2024-05-10T10:54:46.524 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-10T10:54:46.524 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_dparmar@teuthology --archive /home/teuthworker/archive/dparmar-2024-05-10_07:38:09-fs-wip-63896-from-wip-rishabh-testing-20240501.193033-distro-default-smithi/7700928 --verbose --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/suites/dbench}} --name dparmar-2024-05-10_07:38:09-fs-wip-63896-from-wip-rishabh-testing-20240501.193033-distro-default-smithi -- /home/teuthworker/archive/dparmar-2024-05-10_07:38:09-fs-wip-63896-from-wip-rishabh-testing-20240501.193033-distro-default-smithi/7700928/orig.config.yaml 2024-05-10T10:54:46.582 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-10T10:54:46.675 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/dparmar-2024-05-10_07:38:09-fs-wip-63896-from-wip-rishabh-testing-20240501.193033-distro-default-smithi/7700928 branch: wip-63896-from-wip-rishabh-testing-20240501.193033 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/suites/dbench}} email: null first_in_suite: false job_id: '7700928' kernel: kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: dparmar-2024-05-10_07:38:09-fs-wip-63896-from-wip-rishabh-testing-20240501.193033-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-63896-from-wip-rishabh-testing-20240501.193033 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request sha1: d39fa08867836ac070c47cb66c782fa559fd480b subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported install: ceph: flavor: default sha1: d39fa08867836ac070c47cb66c782fa559fd480b kclient: client.0: mount_subvol_num: 1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-63896-from-wip-rishabh-testing-20240501.193033 sha1: d39fa08867836ac070c47cb66c782fa559fd480b owner: scheduled_dparmar@teuthology priority: 100 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 1698 sha1: d39fa08867836ac070c47cb66c782fa559fd480b sleep_before_teardown: 0 subset: 434/512 suite: fs suite_branch: wip-63896-from-wip-rishabh-testing-20240501.193033 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_d39fa08867836ac070c47cb66c782fa559fd480b/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: d39fa08867836ac070c47cb66c782fa559fd480b syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi100.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJt3ipYRQ432lUpm4/AGPX3yBj/0GAz5K85floybCoPuHpQc06vfLSR2OIARet5cxu9aU4b2at+bhV65hOP1tPI= smithi178.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJuZmTHNWAwjSGV2n2XRxRLiENiiqEvpNwqO/nd1+ixZENKQRPBK7HHWqEDjaqfB3U6h5G4+woAQdEh1pWGj2E4= smithi188.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDLA5TUZHoJYRNjyOKi56vQBMgDVnKiHQBobOj4YJoXW8pAlMrQohjTezCcLcajoTSSCXKmYYIyKUsCTdJkyWjs= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - ceph-fuse: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-10_07:38:09 tube: smithi user: dparmar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-10T10:54:46.676 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_d39fa08867836ac070c47cb66c782fa559fd480b/qa; will attempt to use it 2024-05-10T10:54:46.676 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_d39fa08867836ac070c47cb66c782fa559fd480b/qa/tasks 2024-05-10T10:54:46.676 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-10T10:54:46.678 INFO:teuthology.task.internal:Checking packages... 2024-05-10T10:54:46.702 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd39fa08867836ac070c47cb66c782fa559fd480b' 2024-05-10T10:54:46.702 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-10T10:54:46.702 INFO:teuthology.packaging:ref: None 2024-05-10T10:54:46.702 INFO:teuthology.packaging:tag: None 2024-05-10T10:54:46.702 INFO:teuthology.packaging:branch: wip-63896-from-wip-rishabh-testing-20240501.193033 2024-05-10T10:54:46.702 INFO:teuthology.packaging:sha1: d39fa08867836ac070c47cb66c782fa559fd480b 2024-05-10T10:54:46.702 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-63896-from-wip-rishabh-testing-20240501.193033 2024-05-10T10:54:46.925 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3449.gd39fa088 2024-05-10T10:54:46.927 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-10T10:54:46.934 INFO:teuthology.task.internal:no buildpackages task found 2024-05-10T10:54:46.934 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-10T10:54:46.977 INFO:teuthology.task.internal:Saving configuration 2024-05-10T10:54:46.997 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-10T10:54:47.003 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-10T10:54:47.028 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi100.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dparmar-2024-05-10_07:38:09-fs-wip-63896-from-wip-rishabh-testing-20240501.193033-distro-default-smithi/7700928', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-10 10:49:19.642624', 'locked_by': 'scheduled_dparmar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJt3ipYRQ432lUpm4/AGPX3yBj/0GAz5K85floybCoPuHpQc06vfLSR2OIARet5cxu9aU4b2at+bhV65hOP1tPI='} 2024-05-10T10:54:47.050 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi178.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dparmar-2024-05-10_07:38:09-fs-wip-63896-from-wip-rishabh-testing-20240501.193033-distro-default-smithi/7700928', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-10 10:49:19.639563', 'locked_by': 'scheduled_dparmar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJuZmTHNWAwjSGV2n2XRxRLiENiiqEvpNwqO/nd1+ixZENKQRPBK7HHWqEDjaqfB3U6h5G4+woAQdEh1pWGj2E4='} 2024-05-10T10:54:47.072 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi188.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dparmar-2024-05-10_07:38:09-fs-wip-63896-from-wip-rishabh-testing-20240501.193033-distro-default-smithi/7700928', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-10 10:49:19.641318', 'locked_by': 'scheduled_dparmar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDLA5TUZHoJYRNjyOKi56vQBMgDVnKiHQBobOj4YJoXW8pAlMrQohjTezCcLcajoTSSCXKmYYIyKUsCTdJkyWjs='} 2024-05-10T10:54:47.072 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-10T10:54:47.079 INFO:teuthology.task.internal:roles: ubuntu@smithi100.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-05-10T10:54:47.079 INFO:teuthology.task.internal:roles: ubuntu@smithi178.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-05-10T10:54:47.079 INFO:teuthology.task.internal:roles: ubuntu@smithi188.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-05-10T10:54:47.079 INFO:teuthology.run_tasks:Running task console_log... 2024-05-10T10:54:47.251 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7ff4332cc0d0>, signals=[15]) 2024-05-10T10:54:47.251 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-10T10:54:47.257 INFO:teuthology.task.internal:Opening connections... 2024-05-10T10:54:47.257 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi100.front.sepia.ceph.com 2024-05-10T10:54:47.259 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T10:54:47.336 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi178.front.sepia.ceph.com 2024-05-10T10:54:47.337 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T10:54:47.411 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi188.front.sepia.ceph.com 2024-05-10T10:54:47.412 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T10:54:47.484 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-10T10:54:47.489 DEBUG:teuthology.orchestra.run.smithi100:> uname -m 2024-05-10T10:54:47.508 INFO:teuthology.orchestra.run.smithi100.stdout:x86_64 2024-05-10T10:54:47.508 DEBUG:teuthology.orchestra.run.smithi100:> cat /etc/os-release 2024-05-10T10:54:47.566 INFO:teuthology.orchestra.run.smithi100.stdout:NAME="CentOS Stream" 2024-05-10T10:54:47.566 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION="9" 2024-05-10T10:54:47.566 INFO:teuthology.orchestra.run.smithi100.stdout:ID="centos" 2024-05-10T10:54:47.566 INFO:teuthology.orchestra.run.smithi100.stdout:ID_LIKE="rhel fedora" 2024-05-10T10:54:47.566 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION_ID="9" 2024-05-10T10:54:47.566 INFO:teuthology.orchestra.run.smithi100.stdout:PLATFORM_ID="platform:el9" 2024-05-10T10:54:47.566 INFO:teuthology.orchestra.run.smithi100.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-10T10:54:47.566 INFO:teuthology.orchestra.run.smithi100.stdout:ANSI_COLOR="0;31" 2024-05-10T10:54:47.566 INFO:teuthology.orchestra.run.smithi100.stdout:LOGO="fedora-logo-icon" 2024-05-10T10:54:47.566 INFO:teuthology.orchestra.run.smithi100.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-10T10:54:47.566 INFO:teuthology.orchestra.run.smithi100.stdout:HOME_URL="https://centos.org/" 2024-05-10T10:54:47.567 INFO:teuthology.orchestra.run.smithi100.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-10T10:54:47.567 INFO:teuthology.orchestra.run.smithi100.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-10T10:54:47.567 INFO:teuthology.orchestra.run.smithi100.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-10T10:54:47.567 INFO:teuthology.lock.ops:Updating smithi100.front.sepia.ceph.com on lock server 2024-05-10T10:54:47.594 DEBUG:teuthology.orchestra.run.smithi178:> uname -m 2024-05-10T10:54:47.611 INFO:teuthology.orchestra.run.smithi178.stdout:x86_64 2024-05-10T10:54:47.612 DEBUG:teuthology.orchestra.run.smithi178:> cat /etc/os-release 2024-05-10T10:54:47.668 INFO:teuthology.orchestra.run.smithi178.stdout:NAME="CentOS Stream" 2024-05-10T10:54:47.668 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION="9" 2024-05-10T10:54:47.668 INFO:teuthology.orchestra.run.smithi178.stdout:ID="centos" 2024-05-10T10:54:47.668 INFO:teuthology.orchestra.run.smithi178.stdout:ID_LIKE="rhel fedora" 2024-05-10T10:54:47.668 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION_ID="9" 2024-05-10T10:54:47.668 INFO:teuthology.orchestra.run.smithi178.stdout:PLATFORM_ID="platform:el9" 2024-05-10T10:54:47.668 INFO:teuthology.orchestra.run.smithi178.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-10T10:54:47.668 INFO:teuthology.orchestra.run.smithi178.stdout:ANSI_COLOR="0;31" 2024-05-10T10:54:47.669 INFO:teuthology.orchestra.run.smithi178.stdout:LOGO="fedora-logo-icon" 2024-05-10T10:54:47.669 INFO:teuthology.orchestra.run.smithi178.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-10T10:54:47.669 INFO:teuthology.orchestra.run.smithi178.stdout:HOME_URL="https://centos.org/" 2024-05-10T10:54:47.669 INFO:teuthology.orchestra.run.smithi178.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-10T10:54:47.669 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-10T10:54:47.669 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-10T10:54:47.669 INFO:teuthology.lock.ops:Updating smithi178.front.sepia.ceph.com on lock server 2024-05-10T10:54:47.685 DEBUG:teuthology.orchestra.run.smithi188:> uname -m 2024-05-10T10:54:47.703 INFO:teuthology.orchestra.run.smithi188.stdout:x86_64 2024-05-10T10:54:47.703 DEBUG:teuthology.orchestra.run.smithi188:> cat /etc/os-release 2024-05-10T10:54:47.759 INFO:teuthology.orchestra.run.smithi188.stdout:NAME="CentOS Stream" 2024-05-10T10:54:47.760 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION="9" 2024-05-10T10:54:47.760 INFO:teuthology.orchestra.run.smithi188.stdout:ID="centos" 2024-05-10T10:54:47.760 INFO:teuthology.orchestra.run.smithi188.stdout:ID_LIKE="rhel fedora" 2024-05-10T10:54:47.760 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION_ID="9" 2024-05-10T10:54:47.760 INFO:teuthology.orchestra.run.smithi188.stdout:PLATFORM_ID="platform:el9" 2024-05-10T10:54:47.760 INFO:teuthology.orchestra.run.smithi188.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-10T10:54:47.760 INFO:teuthology.orchestra.run.smithi188.stdout:ANSI_COLOR="0;31" 2024-05-10T10:54:47.760 INFO:teuthology.orchestra.run.smithi188.stdout:LOGO="fedora-logo-icon" 2024-05-10T10:54:47.760 INFO:teuthology.orchestra.run.smithi188.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-10T10:54:47.760 INFO:teuthology.orchestra.run.smithi188.stdout:HOME_URL="https://centos.org/" 2024-05-10T10:54:47.760 INFO:teuthology.orchestra.run.smithi188.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-10T10:54:47.760 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-10T10:54:47.761 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-10T10:54:47.761 INFO:teuthology.lock.ops:Updating smithi188.front.sepia.ceph.com on lock server 2024-05-10T10:54:47.785 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-10T10:54:47.794 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-10T10:54:47.819 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-10T10:54:47.820 DEBUG:teuthology.orchestra.run.smithi100:> test '!' -e /home/ubuntu/cephtest 2024-05-10T10:54:47.823 DEBUG:teuthology.orchestra.run.smithi178:> test '!' -e /home/ubuntu/cephtest 2024-05-10T10:54:47.825 DEBUG:teuthology.orchestra.run.smithi188:> test '!' -e /home/ubuntu/cephtest 2024-05-10T10:54:47.841 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-10T10:54:47.847 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-10T10:54:47.847 DEBUG:teuthology.orchestra.run.smithi100:> test -z $(ls -A /var/lib/ceph) 2024-05-10T10:54:47.879 DEBUG:teuthology.orchestra.run.smithi178:> test -z $(ls -A /var/lib/ceph) 2024-05-10T10:54:47.881 DEBUG:teuthology.orchestra.run.smithi188:> test -z $(ls -A /var/lib/ceph) 2024-05-10T10:54:47.982 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-10T10:54:48.050 INFO:teuthology.run_tasks:Running task kernel... 2024-05-10T10:54:48.066 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-05-10T10:54:48.066 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}, 'mon.c': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-05-10T10:54:48.066 DEBUG:teuthology.orchestra.run.smithi100:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-10T10:54:48.067 DEBUG:teuthology.orchestra.run.smithi178:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-10T10:54:48.067 DEBUG:teuthology.orchestra.run.smithi188:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-10T10:54:48.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T10:54:48.083 DEBUG:teuthology.orchestra.run.smithi178:> uname -r 2024-05-10T10:54:48.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T10:54:48.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T10:54:48.085 DEBUG:teuthology.orchestra.run.smithi188:> uname -r 2024-05-10T10:54:48.085 DEBUG:teuthology.orchestra.run.smithi100:> uname -r 2024-05-10T10:54:48.140 INFO:teuthology.orchestra.run.smithi178.stdout:5.14.0-437.el9.x86_64 2024-05-10T10:54:48.140 INFO:teuthology.task.kernel:Running kernel on smithi178: 5.14.0-437.el9.x86_64 2024-05-10T10:54:48.140 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum install -y kernel 2024-05-10T10:54:48.143 INFO:teuthology.orchestra.run.smithi188.stdout:5.14.0-437.el9.x86_64 2024-05-10T10:54:48.143 INFO:teuthology.task.kernel:Running kernel on smithi188: 5.14.0-437.el9.x86_64 2024-05-10T10:54:48.143 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum install -y kernel 2024-05-10T10:54:48.144 INFO:teuthology.orchestra.run.smithi100.stdout:5.14.0-437.el9.x86_64 2024-05-10T10:54:48.144 INFO:teuthology.task.kernel:Running kernel on smithi100: 5.14.0-437.el9.x86_64 2024-05-10T10:54:48.144 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum install -y kernel 2024-05-10T10:54:50.425 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.1 MB 00:00 2024-05-10T10:54:51.278 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - BaseOS 5.2 MB/s | 8.1 MB 00:01 2024-05-10T10:54:51.624 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - BaseOS 4.6 MB/s | 8.1 MB 00:01 2024-05-10T10:54:52.750 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - AppStream 15 MB/s | 19 MB 00:01 2024-05-10T10:54:54.375 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - AppStream 12 MB/s | 19 MB 00:01 2024-05-10T10:54:54.554 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - AppStream 8.7 MB/s | 19 MB 00:02 2024-05-10T10:54:57.134 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - CRB 8.8 MB/s | 6.2 MB 00:00 2024-05-10T10:54:58.617 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - CRB 9.6 MB/s | 6.2 MB 00:00 2024-05-10T10:54:58.655 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.2 MB 00:00 2024-05-10T10:54:58.743 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - Extras packages 38 kB/s | 16 kB 00:00 2024-05-10T10:55:00.069 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - Extras packages 59 kB/s | 16 kB 00:00 2024-05-10T10:55:00.322 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - Extras packages 53 kB/s | 16 kB 00:00 2024-05-10T10:55:00.388 INFO:teuthology.orchestra.run.smithi178.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-05-10T10:55:01.709 INFO:teuthology.orchestra.run.smithi100.stdout:Extra Packages for Enterprise Linux 17 MB/s | 21 MB 00:01 2024-05-10T10:55:01.760 INFO:teuthology.orchestra.run.smithi188.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-05-10T10:55:05.611 INFO:teuthology.orchestra.run.smithi178.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-05-10T10:55:06.873 INFO:teuthology.orchestra.run.smithi100.stdout:lab-extras 16 kB/s | 1.7 kB 00:00 2024-05-10T10:55:06.893 INFO:teuthology.orchestra.run.smithi188.stdout:lab-extras 21 kB/s | 1.7 kB 00:00 2024-05-10T10:55:07.601 INFO:teuthology.orchestra.run.smithi178.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-10T10:55:07.602 INFO:teuthology.orchestra.run.smithi178.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-10T10:55:07.692 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-05-10T10:55:07.701 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-05-10T10:55:07.701 INFO:teuthology.orchestra.run.smithi178.stdout: Package Architecture Version Repository Size 2024-05-10T10:55:07.701 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-05-10T10:55:07.701 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-05-10T10:55:07.701 INFO:teuthology.orchestra.run.smithi178.stdout: kernel x86_64 5.14.0-444.el9 baseos 484 k 2024-05-10T10:55:07.701 INFO:teuthology.orchestra.run.smithi178.stdout:Installing dependencies: 2024-05-10T10:55:07.701 INFO:teuthology.orchestra.run.smithi178.stdout: kernel-core x86_64 5.14.0-444.el9 baseos 15 M 2024-05-10T10:55:07.702 INFO:teuthology.orchestra.run.smithi178.stdout: kernel-modules x86_64 5.14.0-444.el9 baseos 34 M 2024-05-10T10:55:07.702 INFO:teuthology.orchestra.run.smithi178.stdout: kernel-modules-core x86_64 5.14.0-444.el9 baseos 28 M 2024-05-10T10:55:07.702 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-05-10T10:55:07.702 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-05-10T10:55:07.702 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-05-10T10:55:07.702 INFO:teuthology.orchestra.run.smithi178.stdout:Install 4 Packages 2024-05-10T10:55:07.702 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-05-10T10:55:07.702 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 78 M 2024-05-10T10:55:07.703 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 123 M 2024-05-10T10:55:07.703 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-05-10T10:55:07.912 INFO:teuthology.orchestra.run.smithi178.stdout:(1/4): kernel-5.14.0-444.el9.x86_64.rpm 2.7 MB/s | 484 kB 00:00 2024-05-10T10:55:08.429 INFO:teuthology.orchestra.run.smithi178.stdout:(2/4): kernel-core-5.14.0-444.el9.x86_64.rpm 22 MB/s | 15 MB 00:00 2024-05-10T10:55:08.871 INFO:teuthology.orchestra.run.smithi100.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-10T10:55:08.871 INFO:teuthology.orchestra.run.smithi100.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-10T10:55:08.883 INFO:teuthology.orchestra.run.smithi188.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-10T10:55:08.883 INFO:teuthology.orchestra.run.smithi188.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-10T10:55:08.953 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-05-10T10:55:08.960 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-05-10T10:55:08.960 INFO:teuthology.orchestra.run.smithi100.stdout: Package Architecture Version Repository Size 2024-05-10T10:55:08.960 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-05-10T10:55:08.960 INFO:teuthology.orchestra.run.smithi100.stdout:Installing: 2024-05-10T10:55:08.960 INFO:teuthology.orchestra.run.smithi100.stdout: kernel x86_64 5.14.0-444.el9 baseos 484 k 2024-05-10T10:55:08.960 INFO:teuthology.orchestra.run.smithi100.stdout:Installing dependencies: 2024-05-10T10:55:08.960 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-core x86_64 5.14.0-444.el9 baseos 15 M 2024-05-10T10:55:08.960 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-modules x86_64 5.14.0-444.el9 baseos 34 M 2024-05-10T10:55:08.960 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-modules-core x86_64 5.14.0-444.el9 baseos 28 M 2024-05-10T10:55:08.960 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T10:55:08.960 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction Summary 2024-05-10T10:55:08.961 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-05-10T10:55:08.961 INFO:teuthology.orchestra.run.smithi100.stdout:Install 4 Packages 2024-05-10T10:55:08.961 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T10:55:08.961 INFO:teuthology.orchestra.run.smithi100.stdout:Total download size: 78 M 2024-05-10T10:55:08.961 INFO:teuthology.orchestra.run.smithi100.stdout:Installed size: 123 M 2024-05-10T10:55:08.961 INFO:teuthology.orchestra.run.smithi100.stdout:Downloading Packages: 2024-05-10T10:55:08.970 INFO:teuthology.orchestra.run.smithi188.stdout:Dependencies resolved. 2024-05-10T10:55:08.983 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-05-10T10:55:08.983 INFO:teuthology.orchestra.run.smithi188.stdout: Package Architecture Version Repository Size 2024-05-10T10:55:08.983 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-05-10T10:55:08.983 INFO:teuthology.orchestra.run.smithi188.stdout:Installing: 2024-05-10T10:55:08.983 INFO:teuthology.orchestra.run.smithi188.stdout: kernel x86_64 5.14.0-444.el9 baseos 484 k 2024-05-10T10:55:08.983 INFO:teuthology.orchestra.run.smithi188.stdout:Installing dependencies: 2024-05-10T10:55:08.984 INFO:teuthology.orchestra.run.smithi188.stdout: kernel-core x86_64 5.14.0-444.el9 baseos 15 M 2024-05-10T10:55:08.984 INFO:teuthology.orchestra.run.smithi188.stdout: kernel-modules x86_64 5.14.0-444.el9 baseos 34 M 2024-05-10T10:55:08.984 INFO:teuthology.orchestra.run.smithi188.stdout: kernel-modules-core x86_64 5.14.0-444.el9 baseos 28 M 2024-05-10T10:55:08.984 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-10T10:55:08.984 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction Summary 2024-05-10T10:55:08.984 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-05-10T10:55:08.985 INFO:teuthology.orchestra.run.smithi188.stdout:Install 4 Packages 2024-05-10T10:55:08.985 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-10T10:55:08.985 INFO:teuthology.orchestra.run.smithi188.stdout:Total download size: 78 M 2024-05-10T10:55:08.985 INFO:teuthology.orchestra.run.smithi188.stdout:Installed size: 123 M 2024-05-10T10:55:08.986 INFO:teuthology.orchestra.run.smithi188.stdout:Downloading Packages: 2024-05-10T10:55:09.054 INFO:teuthology.orchestra.run.smithi178.stdout:(3/4): kernel-modules-5.14.0-444.el9.x86_64.rpm 26 MB/s | 34 MB 00:01 2024-05-10T10:55:09.275 INFO:teuthology.orchestra.run.smithi100.stdout:(1/4): kernel-5.14.0-444.el9.x86_64.rpm 3.0 MB/s | 484 kB 00:00 2024-05-10T10:55:09.405 INFO:teuthology.orchestra.run.smithi178.stdout:(4/4): kernel-modules-core-5.14.0-444.el9.x86_6 19 MB/s | 28 MB 00:01 2024-05-10T10:55:09.405 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-05-10T10:55:09.405 INFO:teuthology.orchestra.run.smithi178.stdout:Total 46 MB/s | 78 MB 00:01 2024-05-10T10:55:09.511 INFO:teuthology.orchestra.run.smithi188.stdout:(1/4): kernel-5.14.0-444.el9.x86_64.rpm 1.9 MB/s | 484 kB 00:00 2024-05-10T10:55:09.600 INFO:teuthology.orchestra.run.smithi100.stdout:(2/4): kernel-core-5.14.0-444.el9.x86_64.rpm 32 MB/s | 15 MB 00:00 2024-05-10T10:55:10.084 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-05-10T10:55:10.085 INFO:teuthology.orchestra.run.smithi100.stdout:(3/4): kernel-modules-5.14.0-444.el9.x86_64.rpm 35 MB/s | 34 MB 00:00 2024-05-10T10:55:10.320 INFO:teuthology.orchestra.run.smithi188.stdout:(2/4): kernel-core-5.14.0-444.el9.x86_64.rpm 15 MB/s | 15 MB 00:01 2024-05-10T10:55:10.419 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-05-10T10:55:10.419 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-05-10T10:55:10.525 INFO:teuthology.orchestra.run.smithi100.stdout:(4/4): kernel-modules-core-5.14.0-444.el9.x86_6 22 MB/s | 28 MB 00:01 2024-05-10T10:55:10.526 INFO:teuthology.orchestra.run.smithi100.stdout:-------------------------------------------------------------------------------- 2024-05-10T10:55:10.526 INFO:teuthology.orchestra.run.smithi100.stdout:Total 50 MB/s | 78 MB 00:01 2024-05-10T10:55:10.870 INFO:teuthology.orchestra.run.smithi188.stdout:(3/4): kernel-modules-core-5.14.0-444.el9.x86_6 21 MB/s | 28 MB 00:01 2024-05-10T10:55:11.196 INFO:teuthology.orchestra.run.smithi188.stdout:(4/4): kernel-modules-5.14.0-444.el9.x86_64.rpm 18 MB/s | 34 MB 00:01 2024-05-10T10:55:11.196 INFO:teuthology.orchestra.run.smithi188.stdout:-------------------------------------------------------------------------------- 2024-05-10T10:55:11.196 INFO:teuthology.orchestra.run.smithi188.stdout:Total 35 MB/s | 78 MB 00:02 2024-05-10T10:55:11.205 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction check 2024-05-10T10:55:11.373 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-05-10T10:55:11.373 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-05-10T10:55:11.530 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction check succeeded. 2024-05-10T10:55:11.530 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction test 2024-05-10T10:55:11.875 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction check 2024-05-10T10:55:12.209 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction check succeeded. 2024-05-10T10:55:12.209 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction test 2024-05-10T10:55:12.429 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction test succeeded. 2024-05-10T10:55:12.430 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction 2024-05-10T10:55:13.121 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction test succeeded. 2024-05-10T10:55:13.121 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction 2024-05-10T10:55:13.788 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-05-10T10:55:14.512 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : kernel-modules-core-5.14.0-444.el9.x86_64 1/4 2024-05-10T10:55:14.596 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-10T10:55:14.844 INFO:teuthology.orchestra.run.smithi100.stdout: Preparing : 1/1 2024-05-10T10:55:15.508 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : kernel-modules-core-5.14.0-444.el9.x86_64 1/4 2024-05-10T10:55:15.510 INFO:teuthology.orchestra.run.smithi188.stdout: Preparing : 1/1 2024-05-10T10:55:15.593 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-10T10:55:16.214 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : kernel-modules-core-5.14.0-444.el9.x86_64 1/4 2024-05-10T10:55:16.304 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-10T10:55:17.256 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-10T10:55:17.415 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-10T10:55:18.249 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-10T10:55:18.373 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-10T10:55:18.950 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-10T10:55:19.102 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-10T10:55:22.106 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-10T10:55:22.225 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : kernel-5.14.0-444.el9.x86_64 4/4 2024-05-10T10:55:23.062 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-10T10:55:23.109 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : kernel-5.14.0-444.el9.x86_64 4/4 2024-05-10T10:55:23.811 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-10T10:55:23.938 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : kernel-5.14.0-444.el9.x86_64 4/4 2024-05-10T10:55:26.930 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-10T10:55:27.845 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-10T10:55:28.681 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-10T10:55:56.345 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 4/4 2024-05-10T10:55:56.383 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 4/4 2024-05-10T10:55:56.753 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: kernel-5.14.0-444.el9.x86_64 4/4 2024-05-10T10:55:56.753 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : kernel-5.14.0-444.el9.x86_64 1/4 2024-05-10T10:55:56.753 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-10T10:55:56.753 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-10T10:55:57.269 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 4/4 2024-05-10T10:55:57.306 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 4/4 2024-05-10T10:55:57.586 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-10T10:55:57.586 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-05-10T10:55:57.586 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-05-10T10:55:57.587 INFO:teuthology.orchestra.run.smithi178.stdout: kernel-5.14.0-444.el9.x86_64 2024-05-10T10:55:57.587 INFO:teuthology.orchestra.run.smithi178.stdout: kernel-core-5.14.0-444.el9.x86_64 2024-05-10T10:55:57.587 INFO:teuthology.orchestra.run.smithi178.stdout: kernel-modules-5.14.0-444.el9.x86_64 2024-05-10T10:55:57.587 INFO:teuthology.orchestra.run.smithi178.stdout: kernel-modules-core-5.14.0-444.el9.x86_64 2024-05-10T10:55:57.587 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-05-10T10:55:57.587 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-05-10T10:55:57.603 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: kernel-5.14.0-444.el9.x86_64 4/4 2024-05-10T10:55:57.603 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : kernel-5.14.0-444.el9.x86_64 1/4 2024-05-10T10:55:57.603 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-10T10:55:57.603 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-10T10:55:57.959 DEBUG:teuthology.orchestra.run.smithi178:> rpm -q kernel | sort -rV | head -n 1 2024-05-10T10:55:58.031 INFO:teuthology.orchestra.run.smithi178.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-10T10:55:58.032 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-10T10:55:58.032 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-444.el9.x86_64 2024-05-10T10:55:58.032 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-10T10:55:58.032 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-10T10:55:58.033 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-05-10T10:55:58.033 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi178.front.sepia.ceph.com, path=None, version=distro) 2024-05-10T10:55:58.033 DEBUG:teuthology.orchestra.run.smithi178:> rpm -q kernel | sort -rV | head -n 1 2024-05-10T10:55:58.082 INFO:teuthology.orchestra.run.smithi178.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-10T10:55:58.082 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-10T10:55:58.082 DEBUG:teuthology.orchestra.run.smithi178:> sudo rpm -qi grub2-tools 2024-05-10T10:55:58.113 INFO:teuthology.orchestra.run.smithi178.stdout:Name : grub2-tools 2024-05-10T10:55:58.113 INFO:teuthology.orchestra.run.smithi178.stdout:Epoch : 1 2024-05-10T10:55:58.113 INFO:teuthology.orchestra.run.smithi178.stdout:Version : 2.06 2024-05-10T10:55:58.113 INFO:teuthology.orchestra.run.smithi178.stdout:Release : 68.el9 2024-05-10T10:55:58.113 INFO:teuthology.orchestra.run.smithi178.stdout:Architecture: x86_64 2024-05-10T10:55:58.113 INFO:teuthology.orchestra.run.smithi178.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-10T10:55:58.113 INFO:teuthology.orchestra.run.smithi178.stdout:Group : Unspecified 2024-05-10T10:55:58.113 INFO:teuthology.orchestra.run.smithi178.stdout:Size : 8264254 2024-05-10T10:55:58.113 INFO:teuthology.orchestra.run.smithi178.stdout:License : GPLv3+ 2024-05-10T10:55:58.113 INFO:teuthology.orchestra.run.smithi178.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-10T10:55:58.113 INFO:teuthology.orchestra.run.smithi178.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-10T10:55:58.113 INFO:teuthology.orchestra.run.smithi178.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-10T10:55:58.114 INFO:teuthology.orchestra.run.smithi178.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-10T10:55:58.114 INFO:teuthology.orchestra.run.smithi178.stdout:Packager : builder@centos.org 2024-05-10T10:55:58.114 INFO:teuthology.orchestra.run.smithi178.stdout:Vendor : CentOS 2024-05-10T10:55:58.114 INFO:teuthology.orchestra.run.smithi178.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-10T10:55:58.114 INFO:teuthology.orchestra.run.smithi178.stdout:Summary : Support tools for GRUB. 2024-05-10T10:55:58.114 INFO:teuthology.orchestra.run.smithi178.stdout:Description : 2024-05-10T10:55:58.114 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-05-10T10:55:58.114 INFO:teuthology.orchestra.run.smithi178.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-10T10:55:58.114 INFO:teuthology.orchestra.run.smithi178.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-10T10:55:58.114 INFO:teuthology.orchestra.run.smithi178.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-10T10:55:58.115 INFO:teuthology.orchestra.run.smithi178.stdout:hardware devices. 2024-05-10T10:55:58.115 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-05-10T10:55:58.115 INFO:teuthology.orchestra.run.smithi178.stdout:This subpackage provides tools for support of all platforms. 2024-05-10T10:55:58.115 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-10T10:55:58.116 INFO:teuthology.task.kernel:Updating grub on smithi178 to boot 5.14.0-444.el9.x86_64 2024-05-10T10:55:58.116 DEBUG:teuthology.orchestra.run.smithi178:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-10T10:55:58.118 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 4/4 2024-05-10T10:55:58.156 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 4/4 2024-05-10T10:55:58.414 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-10T10:55:58.414 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T10:55:58.414 INFO:teuthology.orchestra.run.smithi100.stdout:Installed: 2024-05-10T10:55:58.414 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-5.14.0-444.el9.x86_64 2024-05-10T10:55:58.414 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-core-5.14.0-444.el9.x86_64 2024-05-10T10:55:58.414 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-modules-5.14.0-444.el9.x86_64 2024-05-10T10:55:58.414 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-modules-core-5.14.0-444.el9.x86_64 2024-05-10T10:55:58.415 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T10:55:58.415 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-05-10T10:55:58.570 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: kernel-5.14.0-444.el9.x86_64 4/4 2024-05-10T10:55:58.570 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : kernel-5.14.0-444.el9.x86_64 1/4 2024-05-10T10:55:58.571 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-10T10:55:58.571 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-10T10:55:58.701 INFO:teuthology.orchestra.run.smithi178.stderr:Generating grub configuration file ... 2024-05-10T10:55:58.746 DEBUG:teuthology.orchestra.run.smithi100:> rpm -q kernel | sort -rV | head -n 1 2024-05-10T10:55:58.858 INFO:teuthology.orchestra.run.smithi100.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-10T10:55:58.858 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-10T10:55:58.858 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-444.el9.x86_64 2024-05-10T10:55:58.858 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-10T10:55:58.858 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-10T10:55:58.858 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-05-10T10:55:58.859 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi100.front.sepia.ceph.com, path=None, version=distro) 2024-05-10T10:55:58.859 DEBUG:teuthology.orchestra.run.smithi100:> rpm -q kernel | sort -rV | head -n 1 2024-05-10T10:55:58.908 INFO:teuthology.orchestra.run.smithi100.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-10T10:55:58.908 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-10T10:55:58.908 DEBUG:teuthology.orchestra.run.smithi100:> sudo rpm -qi grub2-tools 2024-05-10T10:55:58.943 INFO:teuthology.orchestra.run.smithi100.stdout:Name : grub2-tools 2024-05-10T10:55:58.943 INFO:teuthology.orchestra.run.smithi100.stdout:Epoch : 1 2024-05-10T10:55:58.944 INFO:teuthology.orchestra.run.smithi100.stdout:Version : 2.06 2024-05-10T10:55:58.944 INFO:teuthology.orchestra.run.smithi100.stdout:Release : 68.el9 2024-05-10T10:55:58.944 INFO:teuthology.orchestra.run.smithi100.stdout:Architecture: x86_64 2024-05-10T10:55:58.944 INFO:teuthology.orchestra.run.smithi100.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-10T10:55:58.944 INFO:teuthology.orchestra.run.smithi100.stdout:Group : Unspecified 2024-05-10T10:55:58.944 INFO:teuthology.orchestra.run.smithi100.stdout:Size : 8264254 2024-05-10T10:55:58.944 INFO:teuthology.orchestra.run.smithi100.stdout:License : GPLv3+ 2024-05-10T10:55:58.944 INFO:teuthology.orchestra.run.smithi100.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-10T10:55:58.944 INFO:teuthology.orchestra.run.smithi100.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-10T10:55:58.944 INFO:teuthology.orchestra.run.smithi100.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-10T10:55:58.944 INFO:teuthology.orchestra.run.smithi100.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-10T10:55:58.944 INFO:teuthology.orchestra.run.smithi100.stdout:Packager : builder@centos.org 2024-05-10T10:55:58.944 INFO:teuthology.orchestra.run.smithi100.stdout:Vendor : CentOS 2024-05-10T10:55:58.944 INFO:teuthology.orchestra.run.smithi100.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-10T10:55:58.945 INFO:teuthology.orchestra.run.smithi100.stdout:Summary : Support tools for GRUB. 2024-05-10T10:55:58.945 INFO:teuthology.orchestra.run.smithi100.stdout:Description : 2024-05-10T10:55:58.945 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T10:55:58.945 INFO:teuthology.orchestra.run.smithi100.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-10T10:55:58.945 INFO:teuthology.orchestra.run.smithi100.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-10T10:55:58.945 INFO:teuthology.orchestra.run.smithi100.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-10T10:55:58.945 INFO:teuthology.orchestra.run.smithi100.stdout:hardware devices. 2024-05-10T10:55:58.945 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T10:55:58.945 INFO:teuthology.orchestra.run.smithi100.stdout:This subpackage provides tools for support of all platforms. 2024-05-10T10:55:58.946 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-10T10:55:58.946 INFO:teuthology.task.kernel:Updating grub on smithi100 to boot 5.14.0-444.el9.x86_64 2024-05-10T10:55:58.946 DEBUG:teuthology.orchestra.run.smithi100:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-10T10:55:59.394 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-10T10:55:59.394 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-10T10:55:59.394 INFO:teuthology.orchestra.run.smithi188.stdout:Installed: 2024-05-10T10:55:59.394 INFO:teuthology.orchestra.run.smithi188.stdout: kernel-5.14.0-444.el9.x86_64 2024-05-10T10:55:59.394 INFO:teuthology.orchestra.run.smithi188.stdout: kernel-core-5.14.0-444.el9.x86_64 2024-05-10T10:55:59.395 INFO:teuthology.orchestra.run.smithi188.stdout: kernel-modules-5.14.0-444.el9.x86_64 2024-05-10T10:55:59.395 INFO:teuthology.orchestra.run.smithi188.stdout: kernel-modules-core-5.14.0-444.el9.x86_64 2024-05-10T10:55:59.395 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-10T10:55:59.395 INFO:teuthology.orchestra.run.smithi188.stdout:Complete! 2024-05-10T10:55:59.519 INFO:teuthology.orchestra.run.smithi100.stderr:Generating grub configuration file ... 2024-05-10T10:55:59.741 DEBUG:teuthology.orchestra.run.smithi188:> rpm -q kernel | sort -rV | head -n 1 2024-05-10T10:55:59.853 INFO:teuthology.orchestra.run.smithi188.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-10T10:55:59.854 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-10T10:55:59.854 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-444.el9.x86_64 2024-05-10T10:55:59.854 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-10T10:55:59.854 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-10T10:55:59.854 INFO:teuthology.task.kernel:Installing distro kernel on mon.c... 2024-05-10T10:55:59.854 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi188.front.sepia.ceph.com, path=None, version=distro) 2024-05-10T10:55:59.854 DEBUG:teuthology.orchestra.run.smithi188:> rpm -q kernel | sort -rV | head -n 1 2024-05-10T10:55:59.904 INFO:teuthology.orchestra.run.smithi188.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-10T10:55:59.904 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-10T10:55:59.904 DEBUG:teuthology.orchestra.run.smithi188:> sudo rpm -qi grub2-tools 2024-05-10T10:55:59.934 INFO:teuthology.orchestra.run.smithi188.stdout:Name : grub2-tools 2024-05-10T10:55:59.935 INFO:teuthology.orchestra.run.smithi188.stdout:Epoch : 1 2024-05-10T10:55:59.935 INFO:teuthology.orchestra.run.smithi188.stdout:Version : 2.06 2024-05-10T10:55:59.935 INFO:teuthology.orchestra.run.smithi188.stdout:Release : 68.el9 2024-05-10T10:55:59.935 INFO:teuthology.orchestra.run.smithi188.stdout:Architecture: x86_64 2024-05-10T10:55:59.935 INFO:teuthology.orchestra.run.smithi188.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-10T10:55:59.935 INFO:teuthology.orchestra.run.smithi188.stdout:Group : Unspecified 2024-05-10T10:55:59.935 INFO:teuthology.orchestra.run.smithi188.stdout:Size : 8264254 2024-05-10T10:55:59.935 INFO:teuthology.orchestra.run.smithi188.stdout:License : GPLv3+ 2024-05-10T10:55:59.935 INFO:teuthology.orchestra.run.smithi188.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-10T10:55:59.935 INFO:teuthology.orchestra.run.smithi188.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-10T10:55:59.935 INFO:teuthology.orchestra.run.smithi188.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-10T10:55:59.935 INFO:teuthology.orchestra.run.smithi188.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-10T10:55:59.935 INFO:teuthology.orchestra.run.smithi188.stdout:Packager : builder@centos.org 2024-05-10T10:55:59.935 INFO:teuthology.orchestra.run.smithi188.stdout:Vendor : CentOS 2024-05-10T10:55:59.936 INFO:teuthology.orchestra.run.smithi188.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-10T10:55:59.936 INFO:teuthology.orchestra.run.smithi188.stdout:Summary : Support tools for GRUB. 2024-05-10T10:55:59.936 INFO:teuthology.orchestra.run.smithi188.stdout:Description : 2024-05-10T10:55:59.936 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-10T10:55:59.936 INFO:teuthology.orchestra.run.smithi188.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-10T10:55:59.936 INFO:teuthology.orchestra.run.smithi188.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-10T10:55:59.936 INFO:teuthology.orchestra.run.smithi188.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-10T10:55:59.936 INFO:teuthology.orchestra.run.smithi188.stdout:hardware devices. 2024-05-10T10:55:59.936 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-10T10:55:59.936 INFO:teuthology.orchestra.run.smithi188.stdout:This subpackage provides tools for support of all platforms. 2024-05-10T10:55:59.937 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-10T10:55:59.937 INFO:teuthology.task.kernel:Updating grub on smithi188 to boot 5.14.0-444.el9.x86_64 2024-05-10T10:55:59.937 DEBUG:teuthology.orchestra.run.smithi188:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-10T10:56:00.064 INFO:teuthology.orchestra.run.smithi178.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-10T10:56:00.095 INFO:teuthology.orchestra.run.smithi178.stderr:done 2024-05-10T10:56:00.097 DEBUG:teuthology.orchestra.run.smithi178:> mktemp 2024-05-10T10:56:00.116 INFO:teuthology.orchestra.run.smithi178.stdout:/tmp/tmp.3XPj39BvXd 2024-05-10T10:56:00.116 DEBUG:teuthology.orchestra.run.smithi178:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.3XPj39BvXd 2024-05-10T10:56:00.182 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 0666 /tmp/tmp.3XPj39BvXd 2024-05-10T10:56:00.322 DEBUG:teuthology.orchestra.remote:smithi178:/tmp/tmp.3XPj39BvXd is 6KB 2024-05-10T10:56:00.371 DEBUG:teuthology.orchestra.run.smithi178:> rm -fr /tmp/tmp.3XPj39BvXd 2024-05-10T10:56:00.385 DEBUG:teuthology.orchestra.run.smithi178:> sudo /bin/ls /boot/loader/entries || true 2024-05-10T10:56:00.446 INFO:teuthology.orchestra.run.smithi178.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-10T10:56:00.446 INFO:teuthology.orchestra.run.smithi178.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-10T10:56:00.446 INFO:teuthology.orchestra.run.smithi178.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-10T10:56:00.446 INFO:teuthology.orchestra.run.smithi178.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64.conf 2024-05-10T10:56:00.450 DEBUG:teuthology.orchestra.run.smithi178:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64 2024-05-10T10:56:00.536 INFO:teuthology.orchestra.run.smithi188.stderr:Generating grub configuration file ... 2024-05-10T10:56:00.586 DEBUG:teuthology.orchestra.run.smithi178:> sudo shutdown -r now 2024-05-10T10:56:00.855 INFO:teuthology.orchestra.run.smithi100.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-10T10:56:00.889 INFO:teuthology.orchestra.run.smithi100.stderr:done 2024-05-10T10:56:00.891 DEBUG:teuthology.orchestra.run.smithi100:> mktemp 2024-05-10T10:56:00.907 INFO:teuthology.orchestra.run.smithi100.stdout:/tmp/tmp.G8DxNwrwdQ 2024-05-10T10:56:00.908 DEBUG:teuthology.orchestra.run.smithi100:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.G8DxNwrwdQ 2024-05-10T10:56:00.978 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 0666 /tmp/tmp.G8DxNwrwdQ 2024-05-10T10:56:01.122 DEBUG:teuthology.orchestra.remote:smithi100:/tmp/tmp.G8DxNwrwdQ is 6KB 2024-05-10T10:56:01.170 DEBUG:teuthology.orchestra.run.smithi100:> rm -fr /tmp/tmp.G8DxNwrwdQ 2024-05-10T10:56:01.184 DEBUG:teuthology.orchestra.run.smithi100:> sudo /bin/ls /boot/loader/entries || true 2024-05-10T10:56:01.245 INFO:teuthology.orchestra.run.smithi100.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-10T10:56:01.246 INFO:teuthology.orchestra.run.smithi100.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-10T10:56:01.246 INFO:teuthology.orchestra.run.smithi100.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-10T10:56:01.246 INFO:teuthology.orchestra.run.smithi100.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64.conf 2024-05-10T10:56:01.247 DEBUG:teuthology.orchestra.run.smithi100:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64 2024-05-10T10:56:01.406 DEBUG:teuthology.orchestra.run.smithi100:> sudo shutdown -r now 2024-05-10T10:56:01.905 INFO:teuthology.orchestra.run.smithi188.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-10T10:56:01.934 INFO:teuthology.orchestra.run.smithi188.stderr:done 2024-05-10T10:56:01.936 DEBUG:teuthology.orchestra.run.smithi188:> mktemp 2024-05-10T10:56:01.952 INFO:teuthology.orchestra.run.smithi188.stdout:/tmp/tmp.HgDUXylqyv 2024-05-10T10:56:01.952 DEBUG:teuthology.orchestra.run.smithi188:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.HgDUXylqyv 2024-05-10T10:56:02.019 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 0666 /tmp/tmp.HgDUXylqyv 2024-05-10T10:56:02.158 DEBUG:teuthology.orchestra.remote:smithi188:/tmp/tmp.HgDUXylqyv is 6KB 2024-05-10T10:56:02.206 DEBUG:teuthology.orchestra.run.smithi188:> rm -fr /tmp/tmp.HgDUXylqyv 2024-05-10T10:56:02.220 DEBUG:teuthology.orchestra.run.smithi188:> sudo /bin/ls /boot/loader/entries || true 2024-05-10T10:56:02.290 INFO:teuthology.orchestra.run.smithi188.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-10T10:56:02.290 INFO:teuthology.orchestra.run.smithi188.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-10T10:56:02.290 INFO:teuthology.orchestra.run.smithi188.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-10T10:56:02.290 INFO:teuthology.orchestra.run.smithi188.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64.conf 2024-05-10T10:56:02.291 DEBUG:teuthology.orchestra.run.smithi188:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64 2024-05-10T10:56:02.440 DEBUG:teuthology.orchestra.run.smithi188:> sudo shutdown -r now 2024-05-10T10:56:30.613 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-05-10T10:56:30.614 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-05-10T10:56:30.615 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T10:56:31.410 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-05-10T10:56:31.410 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-05-10T10:56:31.411 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T10:56:32.443 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-05-10T10:56:32.443 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi188.front.sepia.ceph.com' 2024-05-10T10:56:32.444 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T10:56:49.183 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.178 2024-05-10T10:56:49.950 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.100 2024-05-10T10:56:50.974 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.188 2024-05-10T10:56:58.191 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-05-10T10:56:58.192 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T10:56:58.951 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-05-10T10:56:58.952 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T10:56:59.976 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi188.front.sepia.ceph.com' 2024-05-10T10:56:59.977 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T10:57:01.246 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.178 2024-05-10T10:57:02.014 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.100 2024-05-10T10:57:03.038 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.188 2024-05-10T10:57:13.257 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-05-10T10:57:13.258 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T10:57:14.016 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-05-10T10:57:14.016 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T10:57:15.041 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi188.front.sepia.ceph.com' 2024-05-10T10:57:15.041 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T10:57:32.446 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.100 2024-05-10T10:57:33.470 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.188 2024-05-10T10:57:47.458 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-05-10T10:57:47.459 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T10:57:47.858 DEBUG:teuthology.orchestra.run.smithi100:> true 2024-05-10T10:57:48.259 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-05-10T10:57:48.259 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-444.el9.x86_64"... 2024-05-10T10:57:48.259 DEBUG:teuthology.orchestra.run.smithi100:> uname -r 2024-05-10T10:57:48.274 INFO:teuthology.orchestra.run.smithi100.stdout:5.14.0-444.el9.x86_64 2024-05-10T10:57:48.274 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-444.el9.x86_64 vs 5.14.0-444.el9.x86_64 2024-05-10T10:57:48.274 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-10T10:57:48.274 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-10T10:57:48.471 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi188.front.sepia.ceph.com' 2024-05-10T10:57:48.472 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T10:57:48.758 DEBUG:teuthology.orchestra.run.smithi188:> true 2024-05-10T10:57:49.275 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-05-10T10:57:49.275 DEBUG:teuthology.orchestra.run.smithi100:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-10T10:57:49.339 INFO:teuthology.orchestra.run.smithi100.stdout:ttyS1 2024-05-10T10:57:49.375 DEBUG:teuthology.parallel:result is None 2024-05-10T10:57:49.398 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi188.front.sepia.ceph.com' 2024-05-10T10:57:49.398 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "5.14.0-444.el9.x86_64"... 2024-05-10T10:57:49.399 DEBUG:teuthology.orchestra.run.smithi188:> uname -r 2024-05-10T10:57:49.422 INFO:teuthology.orchestra.run.smithi188.stdout:5.14.0-444.el9.x86_64 2024-05-10T10:57:49.423 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-444.el9.x86_64 vs 5.14.0-444.el9.x86_64 2024-05-10T10:57:49.423 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-10T10:57:49.423 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-10T10:57:50.424 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-05-10T10:57:50.424 DEBUG:teuthology.orchestra.run.smithi188:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-10T10:57:50.725 INFO:teuthology.orchestra.run.smithi188.stdout:ttyS1 2024-05-10T10:57:50.759 DEBUG:teuthology.parallel:result is None 2024-05-10T10:58:13.259 DEBUG:teuthology.orchestra.remote:timed out 2024-05-10T10:58:28.260 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-05-10T10:58:28.261 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T10:58:28.721 DEBUG:teuthology.orchestra.run.smithi178:> true 2024-05-10T10:58:29.143 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-05-10T10:58:29.144 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-444.el9.x86_64"... 2024-05-10T10:58:29.144 DEBUG:teuthology.orchestra.run.smithi178:> uname -r 2024-05-10T10:58:29.161 INFO:teuthology.orchestra.run.smithi178.stdout:5.14.0-444.el9.x86_64 2024-05-10T10:58:29.161 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-444.el9.x86_64 vs 5.14.0-444.el9.x86_64 2024-05-10T10:58:29.161 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-10T10:58:29.161 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-10T10:58:30.162 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-05-10T10:58:30.162 DEBUG:teuthology.orchestra.run.smithi178:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-10T10:58:30.260 INFO:teuthology.orchestra.run.smithi178.stdout:ttyS1 2024-05-10T10:58:30.295 DEBUG:teuthology.parallel:result is None 2024-05-10T10:58:30.296 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-10T10:58:30.304 INFO:teuthology.task.internal:Creating test directory... 2024-05-10T10:58:30.304 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-10T10:58:30.308 DEBUG:teuthology.orchestra.run.smithi178:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-10T10:58:30.310 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-10T10:58:30.329 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-10T10:58:30.346 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-10T10:58:30.396 INFO:teuthology.task.internal:Creating archive directory... 2024-05-10T10:58:30.396 DEBUG:teuthology.orchestra.run.smithi100:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-10T10:58:30.399 DEBUG:teuthology.orchestra.run.smithi178:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-10T10:58:30.401 DEBUG:teuthology.orchestra.run.smithi188:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-10T10:58:30.495 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-10T10:58:30.505 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-10T10:58:30.506 DEBUG:teuthology.orchestra.run.smithi100:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-10T10:58:30.508 DEBUG:teuthology.orchestra.run.smithi178:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-10T10:58:30.511 DEBUG:teuthology.orchestra.run.smithi188:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-10T10:58:30.541 INFO:teuthology.orchestra.run.smithi178.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-10T10:58:30.546 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-10T10:58:30.550 INFO:teuthology.orchestra.run.smithi188.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-10T10:58:30.552 INFO:teuthology.orchestra.run.smithi178.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-10T10:58:30.558 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-10T10:58:30.562 INFO:teuthology.orchestra.run.smithi188.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-10T10:58:30.564 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-10T10:58:30.571 INFO:teuthology.task.internal:Configuring sudo... 2024-05-10T10:58:30.572 DEBUG:teuthology.orchestra.run.smithi100:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-10T10:58:30.602 DEBUG:teuthology.orchestra.run.smithi178:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-10T10:58:30.605 DEBUG:teuthology.orchestra.run.smithi188:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-10T10:58:30.634 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-10T10:58:30.643 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-10T10:58:30.643 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-10T10:58:30.672 DEBUG:teuthology.orchestra.run.smithi178:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-10T10:58:30.674 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-10T10:58:30.691 DEBUG:teuthology.orchestra.run.smithi100:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-10T10:58:30.771 DEBUG:teuthology.orchestra.run.smithi100:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-10T10:58:30.817 DEBUG:teuthology.orchestra.run.smithi100:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-10T10:58:30.902 DEBUG:teuthology.orchestra.run.smithi100:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-10T10:58:30.941 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-05-10T10:58:30.941 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-10T10:58:31.007 DEBUG:teuthology.orchestra.run.smithi178:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-10T10:58:31.050 DEBUG:teuthology.orchestra.run.smithi178:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-10T10:58:31.100 DEBUG:teuthology.orchestra.run.smithi178:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-10T10:58:31.185 DEBUG:teuthology.orchestra.run.smithi178:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-10T10:58:31.228 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-05-10T10:58:31.229 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-10T10:58:31.296 DEBUG:teuthology.orchestra.run.smithi188:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-10T10:58:31.340 DEBUG:teuthology.orchestra.run.smithi188:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-10T10:58:31.392 DEBUG:teuthology.orchestra.run.smithi188:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-10T10:58:31.478 DEBUG:teuthology.orchestra.run.smithi188:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-10T10:58:31.523 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-05-10T10:58:31.523 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-10T10:58:31.591 DEBUG:teuthology.orchestra.run.smithi100:> sudo service rsyslog restart 2024-05-10T10:58:31.594 DEBUG:teuthology.orchestra.run.smithi178:> sudo service rsyslog restart 2024-05-10T10:58:31.596 DEBUG:teuthology.orchestra.run.smithi188:> sudo service rsyslog restart 2024-05-10T10:58:31.636 INFO:teuthology.orchestra.run.smithi178.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-10T10:58:31.658 INFO:teuthology.orchestra.run.smithi100.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-10T10:58:31.694 INFO:teuthology.orchestra.run.smithi188.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-10T10:58:32.076 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-10T10:58:32.083 INFO:teuthology.task.internal:Starting timer... 2024-05-10T10:58:32.084 INFO:teuthology.run_tasks:Running task pcp... 2024-05-10T10:58:32.133 INFO:teuthology.run_tasks:Running task selinux... 2024-05-10T10:58:32.182 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-05-10T10:58:32.182 DEBUG:teuthology.orchestra.run.smithi100:> sudo service auditd rotate 2024-05-10T10:58:32.259 INFO:teuthology.orchestra.run.smithi100.stdout:Rotating logs: 2024-05-10T10:58:32.261 DEBUG:teuthology.orchestra.run.smithi178:> sudo service auditd rotate 2024-05-10T10:58:32.306 INFO:teuthology.orchestra.run.smithi178.stdout:Rotating logs: 2024-05-10T10:58:32.308 DEBUG:teuthology.orchestra.run.smithi188:> sudo service auditd rotate 2024-05-10T10:58:32.358 INFO:teuthology.orchestra.run.smithi188.stdout:Rotating logs: 2024-05-10T10:58:32.360 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-10T10:58:32.360 DEBUG:teuthology.orchestra.run.smithi100:> /usr/sbin/getenforce 2024-05-10T10:58:32.389 INFO:teuthology.orchestra.run.smithi100.stdout:Permissive 2024-05-10T10:58:32.389 DEBUG:teuthology.orchestra.run.smithi178:> /usr/sbin/getenforce 2024-05-10T10:58:32.436 INFO:teuthology.orchestra.run.smithi178.stdout:Permissive 2024-05-10T10:58:32.437 DEBUG:teuthology.orchestra.run.smithi188:> /usr/sbin/getenforce 2024-05-10T10:58:32.514 INFO:teuthology.orchestra.run.smithi188.stdout:Permissive 2024-05-10T10:58:32.515 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi100.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi178.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi188.front.sepia.ceph.com': 'permissive'} 2024-05-10T10:58:32.515 DEBUG:teuthology.orchestra.run.smithi100:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-10T10:58:32.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T10:58:32.546 DEBUG:teuthology.orchestra.run.smithi178:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-10T10:58:32.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T10:58:32.576 DEBUG:teuthology.orchestra.run.smithi188:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-10T10:58:32.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T10:58:32.605 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-10T10:58:32.605 DEBUG:teuthology.orchestra.run.smithi100:> sudo /usr/sbin/setenforce permissive 2024-05-10T10:58:32.633 DEBUG:teuthology.orchestra.run.smithi178:> sudo /usr/sbin/setenforce permissive 2024-05-10T10:58:32.661 DEBUG:teuthology.orchestra.run.smithi188:> sudo /usr/sbin/setenforce permissive 2024-05-10T10:58:32.688 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-10T10:58:32.698 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-10T10:58:32.715 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-10T10:58:32.717 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi100.front.sepia.ceph.com,smithi178.front.sepia.ceph.com,smithi188.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-10T11:02:38.865 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi100.front.sepia.ceph.com'), Remote(name='ubuntu@smithi178.front.sepia.ceph.com'), Remote(name='ubuntu@smithi188.front.sepia.ceph.com')] 2024-05-10T11:02:38.866 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-05-10T11:02:38.868 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T11:02:38.955 DEBUG:teuthology.orchestra.run.smithi100:> true 2024-05-10T11:02:39.038 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-05-10T11:02:39.039 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-05-10T11:02:39.040 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T11:02:39.114 DEBUG:teuthology.orchestra.run.smithi178:> true 2024-05-10T11:02:39.201 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-05-10T11:02:39.202 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi188.front.sepia.ceph.com' 2024-05-10T11:02:39.202 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T11:02:39.317 DEBUG:teuthology.orchestra.run.smithi188:> true 2024-05-10T11:02:39.393 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi188.front.sepia.ceph.com' 2024-05-10T11:02:39.393 INFO:teuthology.run_tasks:Running task clock... 2024-05-10T11:02:39.405 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-10T11:02:39.405 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-10T11:02:39.406 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-10T11:02:39.408 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-10T11:02:39.408 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-10T11:02:39.410 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-10T11:02:39.410 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-10T11:02:39.439 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-10T11:02:39.440 INFO:teuthology.orchestra.run.smithi178.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-10T11:02:39.454 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-10T11:02:39.455 INFO:teuthology.orchestra.run.smithi178.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-10T11:02:39.463 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-10T11:02:39.477 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-10T11:02:39.479 INFO:teuthology.orchestra.run.smithi100.stderr:sudo: ntpd: command not found 2024-05-10T11:02:39.480 INFO:teuthology.orchestra.run.smithi178.stderr:sudo: ntpd: command not found 2024-05-10T11:02:39.491 INFO:teuthology.orchestra.run.smithi100.stdout:506 Cannot talk to daemon 2024-05-10T11:02:39.491 INFO:teuthology.orchestra.run.smithi178.stdout:506 Cannot talk to daemon 2024-05-10T11:02:39.501 INFO:teuthology.orchestra.run.smithi188.stderr:sudo: ntpd: command not found 2024-05-10T11:02:39.505 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-10T11:02:39.505 INFO:teuthology.orchestra.run.smithi178.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-10T11:02:39.512 INFO:teuthology.orchestra.run.smithi188.stdout:506 Cannot talk to daemon 2024-05-10T11:02:39.519 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-10T11:02:39.520 INFO:teuthology.orchestra.run.smithi178.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-10T11:02:39.526 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-10T11:02:39.540 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-10T11:02:39.576 INFO:teuthology.orchestra.run.smithi178.stderr:bash: line 1: ntpq: command not found 2024-05-10T11:02:39.576 INFO:teuthology.orchestra.run.smithi100.stderr:bash: line 1: ntpq: command not found 2024-05-10T11:02:39.579 INFO:teuthology.orchestra.run.smithi178.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-10T11:02:39.579 INFO:teuthology.orchestra.run.smithi178.stdout:=============================================================================== 2024-05-10T11:02:39.579 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T11:02:39.579 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T11:02:39.579 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T11:02:39.579 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T11:02:39.580 INFO:teuthology.orchestra.run.smithi100.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-10T11:02:39.581 INFO:teuthology.orchestra.run.smithi100.stdout:=============================================================================== 2024-05-10T11:02:39.581 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T11:02:39.581 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T11:02:39.581 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T11:02:39.581 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T11:02:39.601 INFO:teuthology.orchestra.run.smithi188.stderr:bash: line 1: ntpq: command not found 2024-05-10T11:02:39.605 INFO:teuthology.orchestra.run.smithi188.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-10T11:02:39.605 INFO:teuthology.orchestra.run.smithi188.stdout:=============================================================================== 2024-05-10T11:02:39.605 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T11:02:39.605 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T11:02:39.605 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T11:02:39.605 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T11:02:39.606 INFO:teuthology.run_tasks:Running task install... 2024-05-10T11:02:39.614 DEBUG:teuthology.task.install:project ceph 2024-05-10T11:02:39.614 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd39fa08867836ac070c47cb66c782fa559fd480b'}} 2024-05-10T11:02:39.614 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd39fa08867836ac070c47cb66c782fa559fd480b'} 2024-05-10T11:02:39.614 INFO:teuthology.task.install:Using flavor: default 2024-05-10T11:02:39.620 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-10T11:02:39.620 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-10T11:02:39.620 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd39fa08867836ac070c47cb66c782fa559fd480b', 'tag': None, 'wait_for_package': False} 2024-05-10T11:02:39.621 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d39fa08867836ac070c47cb66c782fa559fd480b 2024-05-10T11:02:39.623 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd39fa08867836ac070c47cb66c782fa559fd480b', 'tag': None, 'wait_for_package': False} 2024-05-10T11:02:39.623 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d39fa08867836ac070c47cb66c782fa559fd480b 2024-05-10T11:02:39.624 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd39fa08867836ac070c47cb66c782fa559fd480b', 'tag': None, 'wait_for_package': False} 2024-05-10T11:02:39.624 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d39fa08867836ac070c47cb66c782fa559fd480b 2024-05-10T11:02:39.770 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-63896-from-wip-rishabh-testing-20240501.193033/d39fa08867836ac070c47cb66c782fa559fd480b/centos/9/flavors/default/ 2024-05-10T11:02:39.771 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3449.gd39fa088 2024-05-10T11:02:39.776 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-63896-from-wip-rishabh-testing-20240501.193033/d39fa08867836ac070c47cb66c782fa559fd480b/centos/9/flavors/default/ 2024-05-10T11:02:39.776 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3449.gd39fa088 2024-05-10T11:02:39.839 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-63896-from-wip-rishabh-testing-20240501.193033/d39fa08867836ac070c47cb66c782fa559fd480b/centos/9/flavors/default/ 2024-05-10T11:02:39.839 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3449.gd39fa088 2024-05-10T11:02:39.888 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-63896-from-wip-rishabh-testing-20240501.193033/d39fa08867836ac070c47cb66c782fa559fd480b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-63896-from-wip-rishabh-testing-20240501.193033/d39fa08867836ac070c47cb66c782fa559fd480b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-63896-from-wip-rishabh-testing-20240501.193033/d39fa08867836ac070c47cb66c782fa559fd480b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-10T11:02:39.888 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-05-10T11:02:39.888 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-10T11:02:39.890 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-63896-from-wip-rishabh-testing-20240501.193033/d39fa08867836ac070c47cb66c782fa559fd480b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-63896-from-wip-rishabh-testing-20240501.193033/d39fa08867836ac070c47cb66c782fa559fd480b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-63896-from-wip-rishabh-testing-20240501.193033/d39fa08867836ac070c47cb66c782fa559fd480b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-10T11:02:39.890 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-05-10T11:02:39.890 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-10T11:02:39.917 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-10T11:02:39.917 DEBUG:teuthology.orchestra.run.smithi188:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d39fa08867836ac070c47cb66c782fa559fd480b/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-10T11:02:39.918 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-10T11:02:39.918 DEBUG:teuthology.orchestra.run.smithi178:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d39fa08867836ac070c47cb66c782fa559fd480b/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-10T11:02:39.984 DEBUG:teuthology.orchestra.run.smithi178:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-10T11:02:39.985 DEBUG:teuthology.orchestra.run.smithi188:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-10T11:02:40.008 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-63896-from-wip-rishabh-testing-20240501.193033/d39fa08867836ac070c47cb66c782fa559fd480b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-63896-from-wip-rishabh-testing-20240501.193033/d39fa08867836ac070c47cb66c782fa559fd480b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-63896-from-wip-rishabh-testing-20240501.193033/d39fa08867836ac070c47cb66c782fa559fd480b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-10T11:02:40.009 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-05-10T11:02:40.009 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-10T11:02:40.036 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-10T11:02:40.036 DEBUG:teuthology.orchestra.run.smithi100:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d39fa08867836ac070c47cb66c782fa559fd480b/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-10T11:02:40.059 DEBUG:teuthology.orchestra.run.smithi178:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-10T11:02:40.060 DEBUG:teuthology.orchestra.run.smithi188:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-10T11:02:40.102 DEBUG:teuthology.orchestra.run.smithi100:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-10T11:02:40.123 INFO:teuthology.orchestra.run.smithi178.stdout:check_obsoletes = 1 2024-05-10T11:02:40.126 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum clean all 2024-05-10T11:02:40.126 INFO:teuthology.orchestra.run.smithi188.stdout:check_obsoletes = 1 2024-05-10T11:02:40.128 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum clean all 2024-05-10T11:02:40.178 DEBUG:teuthology.orchestra.run.smithi100:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-10T11:02:40.243 INFO:teuthology.orchestra.run.smithi100.stdout:check_obsoletes = 1 2024-05-10T11:02:40.245 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum clean all 2024-05-10T11:02:40.361 INFO:teuthology.orchestra.run.smithi178.stdout:54 files removed 2024-05-10T11:02:40.362 INFO:teuthology.orchestra.run.smithi188.stdout:54 files removed 2024-05-10T11:02:40.388 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-10T11:02:40.389 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-10T11:02:40.490 INFO:teuthology.orchestra.run.smithi100.stdout:54 files removed 2024-05-10T11:02:40.516 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-10T11:02:41.065 INFO:teuthology.orchestra.run.smithi178.stdout:ceph packages for x86_64 207 kB/s | 84 kB 00:00 2024-05-10T11:02:41.075 INFO:teuthology.orchestra.run.smithi188.stdout:ceph packages for x86_64 203 kB/s | 84 kB 00:00 2024-05-10T11:02:41.115 INFO:teuthology.orchestra.run.smithi100.stdout:ceph packages for x86_64 257 kB/s | 84 kB 00:00 2024-05-10T11:02:41.440 INFO:teuthology.orchestra.run.smithi178.stdout:ceph noarch packages 36 kB/s | 12 kB 00:00 2024-05-10T11:02:41.442 INFO:teuthology.orchestra.run.smithi188.stdout:ceph noarch packages 37 kB/s | 12 kB 00:00 2024-05-10T11:02:41.541 INFO:teuthology.orchestra.run.smithi100.stdout:ceph noarch packages 31 kB/s | 12 kB 00:00 2024-05-10T11:02:41.808 INFO:teuthology.orchestra.run.smithi178.stdout:ceph source packages 5.8 kB/s | 1.9 kB 00:00 2024-05-10T11:02:41.860 INFO:teuthology.orchestra.run.smithi188.stdout:ceph source packages 5.1 kB/s | 1.9 kB 00:00 2024-05-10T11:02:41.950 INFO:teuthology.orchestra.run.smithi100.stdout:ceph source packages 5.2 kB/s | 1.9 kB 00:00 2024-05-10T11:02:42.425 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - BaseOS 19 MB/s | 8.1 MB 00:00 2024-05-10T11:02:42.733 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - BaseOS 9.1 MB/s | 8.1 MB 00:00 2024-05-10T11:02:44.501 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - AppStream 28 MB/s | 19 MB 00:00 2024-05-10T11:02:46.509 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - BaseOS 1.7 MB/s | 8.1 MB 00:04 2024-05-10T11:02:47.226 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - AppStream 5.2 MB/s | 19 MB 00:03 2024-05-10T11:02:48.710 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-05-10T11:02:48.818 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - AppStream 15 MB/s | 19 MB 00:01 2024-05-10T11:02:50.419 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - Extras packages 31 kB/s | 16 kB 00:00 2024-05-10T11:02:51.462 INFO:teuthology.orchestra.run.smithi178.stdout:Extra Packages for Enterprise Linux 23 MB/s | 21 MB 00:00 2024-05-10T11:02:51.685 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - CRB 6.8 MB/s | 6.2 MB 00:00 2024-05-10T11:02:52.878 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.2 MB 00:00 2024-05-10T11:02:53.078 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - Extras packages 81 kB/s | 16 kB 00:00 2024-05-10T11:02:54.678 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - Extras packages 37 kB/s | 16 kB 00:00 2024-05-10T11:02:54.996 INFO:teuthology.orchestra.run.smithi100.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-05-10T11:02:56.579 INFO:teuthology.orchestra.run.smithi178.stdout:lab-extras 36 kB/s | 1.7 kB 00:00 2024-05-10T11:02:56.741 INFO:teuthology.orchestra.run.smithi188.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-05-10T11:02:58.880 INFO:teuthology.orchestra.run.smithi178.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-10T11:02:58.881 INFO:teuthology.orchestra.run.smithi178.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-10T11:02:58.891 INFO:teuthology.orchestra.run.smithi178.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-10T11:02:58.892 INFO:teuthology.orchestra.run.smithi178.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-10T11:02:58.893 INFO:teuthology.orchestra.run.smithi178.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-10T11:02:58.894 INFO:teuthology.orchestra.run.smithi178.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-10T11:02:58.895 INFO:teuthology.orchestra.run.smithi178.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-10T11:02:58.897 INFO:teuthology.orchestra.run.smithi178.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-10T11:02:58.898 INFO:teuthology.orchestra.run.smithi178.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-10T11:02:58.959 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-05-10T11:02:58.974 INFO:teuthology.orchestra.run.smithi178.stdout:======================================================================================= 2024-05-10T11:02:58.974 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repository Size 2024-05-10T11:02:58.974 INFO:teuthology.orchestra.run.smithi178.stdout:======================================================================================= 2024-05-10T11:02:58.974 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-05-10T11:02:58.974 INFO:teuthology.orchestra.run.smithi178.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-10T11:02:58.974 INFO:teuthology.orchestra.run.smithi178.stdout: ceph x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 6.6 k 2024-05-10T11:02:58.975 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-base x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 5.4 M 2024-05-10T11:02:58.975 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-fuse x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 890 k 2024-05-10T11:02:58.975 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 145 k 2024-05-10T11:02:58.975 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 1.5 M 2024-05-10T11:02:58.975 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 142 k 2024-05-10T11:02:58.975 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 3.5 M 2024-05-10T11:02:58.975 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 7.4 M 2024-05-10T11:02:58.975 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-rook noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 49 k 2024-05-10T11:02:58.975 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-radosgw x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 11 M 2024-05-10T11:02:58.975 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-test x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 49 M 2024-05-10T11:02:58.975 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-volume noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 273 k 2024-05-10T11:02:58.975 INFO:teuthology.orchestra.run.smithi178.stdout: cephadm noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 763 k 2024-05-10T11:02:58.975 INFO:teuthology.orchestra.run.smithi178.stdout: cephfs-mirror x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 225 k 2024-05-10T11:02:58.975 INFO:teuthology.orchestra.run.smithi178.stdout: cephfs-shell x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 27 k 2024-05-10T11:02:58.976 INFO:teuthology.orchestra.run.smithi178.stdout: cephfs-top noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 18 k 2024-05-10T11:02:58.976 INFO:teuthology.orchestra.run.smithi178.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-10T11:02:58.976 INFO:teuthology.orchestra.run.smithi178.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-10T11:02:58.976 INFO:teuthology.orchestra.run.smithi178.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-10T11:02:58.976 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs-devel x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 32 k 2024-05-10T11:02:58.976 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs2 x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 746 k 2024-05-10T11:02:58.976 INFO:teuthology.orchestra.run.smithi178.stdout: librados-devel x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 127 k 2024-05-10T11:02:58.976 INFO:teuthology.orchestra.run.smithi178.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-10T11:02:58.976 INFO:teuthology.orchestra.run.smithi178.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-10T11:02:58.976 INFO:teuthology.orchestra.run.smithi178.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-10T11:02:58.977 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-10T11:02:58.977 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-10T11:02:58.977 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-10T11:02:58.977 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cephfs x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 161 k 2024-05-10T11:02:58.977 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rados x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 321 k 2024-05-10T11:02:58.977 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rbd x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 298 k 2024-05-10T11:02:58.977 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rgw x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 99 k 2024-05-10T11:02:58.977 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-fuse x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 85 k 2024-05-10T11:02:58.977 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-mirror x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 3.1 M 2024-05-10T11:02:58.977 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-nbd x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 170 k 2024-05-10T11:02:58.978 INFO:teuthology.orchestra.run.smithi178.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-10T11:02:58.978 INFO:teuthology.orchestra.run.smithi178.stdout:Upgrading: 2024-05-10T11:02:58.978 INFO:teuthology.orchestra.run.smithi178.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-10T11:02:58.978 INFO:teuthology.orchestra.run.smithi178.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-10T11:02:58.978 INFO:teuthology.orchestra.run.smithi178.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-10T11:02:58.978 INFO:teuthology.orchestra.run.smithi178.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-10T11:02:58.978 INFO:teuthology.orchestra.run.smithi178.stdout: librados2 x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 3.4 M 2024-05-10T11:02:58.978 INFO:teuthology.orchestra.run.smithi178.stdout: librbd1 x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 3.1 M 2024-05-10T11:02:58.978 INFO:teuthology.orchestra.run.smithi178.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-10T11:02:58.978 INFO:teuthology.orchestra.run.smithi178.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-10T11:02:58.978 INFO:teuthology.orchestra.run.smithi178.stdout:Installing dependencies: 2024-05-10T11:02:58.978 INFO:teuthology.orchestra.run.smithi178.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-10T11:02:58.978 INFO:teuthology.orchestra.run.smithi178.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-10T11:02:58.979 INFO:teuthology.orchestra.run.smithi178.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-10T11:02:58.979 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-common x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 21 M 2024-05-10T11:02:58.979 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 34 k 2024-05-10T11:02:58.979 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mds x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 2.4 M 2024-05-10T11:02:58.979 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 274 k 2024-05-10T11:02:58.979 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mon x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 4.7 M 2024-05-10T11:02:58.979 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-osd x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 17 M 2024-05-10T11:02:58.979 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 16 k 2024-05-10T11:02:58.979 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-selinux x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 25 k 2024-05-10T11:02:58.979 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-10T11:02:58.979 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-10T11:02:58.979 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-10T11:02:58.979 INFO:teuthology.orchestra.run.smithi178.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-10T11:02:58.980 INFO:teuthology.orchestra.run.smithi178.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-10T11:02:58.980 INFO:teuthology.orchestra.run.smithi178.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-10T11:02:58.980 INFO:teuthology.orchestra.run.smithi178.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-10T11:02:58.980 INFO:teuthology.orchestra.run.smithi178.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-10T11:02:58.980 INFO:teuthology.orchestra.run.smithi178.stdout: libcephsqlite x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 163 k 2024-05-10T11:02:58.980 INFO:teuthology.orchestra.run.smithi178.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-10T11:02:58.980 INFO:teuthology.orchestra.run.smithi178.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-10T11:02:58.981 INFO:teuthology.orchestra.run.smithi178.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-10T11:02:58.981 INFO:teuthology.orchestra.run.smithi178.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-10T11:02:58.981 INFO:teuthology.orchestra.run.smithi178.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-10T11:02:58.981 INFO:teuthology.orchestra.run.smithi178.stdout: libradosstriper1 x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 500 k 2024-05-10T11:02:58.981 INFO:teuthology.orchestra.run.smithi178.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-10T11:02:58.981 INFO:teuthology.orchestra.run.smithi178.stdout: librgw2 x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 5.4 M 2024-05-10T11:02:58.981 INFO:teuthology.orchestra.run.smithi178.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-10T11:02:58.981 INFO:teuthology.orchestra.run.smithi178.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-10T11:02:58.981 INFO:teuthology.orchestra.run.smithi178.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-10T11:02:58.981 INFO:teuthology.orchestra.run.smithi178.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-10T11:02:58.981 INFO:teuthology.orchestra.run.smithi178.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-10T11:02:58.981 INFO:teuthology.orchestra.run.smithi178.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-10T11:02:58.982 INFO:teuthology.orchestra.run.smithi178.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-10T11:02:58.982 INFO:teuthology.orchestra.run.smithi178.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-10T11:02:58.982 INFO:teuthology.orchestra.run.smithi178.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-10T11:02:58.982 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-10T11:02:58.982 INFO:teuthology.orchestra.run.smithi178.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-10T11:02:58.982 INFO:teuthology.orchestra.run.smithi178.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-10T11:02:58.982 INFO:teuthology.orchestra.run.smithi178.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-10T11:02:58.982 INFO:teuthology.orchestra.run.smithi178.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-10T11:02:58.982 INFO:teuthology.orchestra.run.smithi178.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-10T11:02:58.982 INFO:teuthology.orchestra.run.smithi178.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-10T11:02:58.982 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-10T11:02:58.982 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-argparse x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 45 k 2024-05-10T11:02:58.982 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-common x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 135 k 2024-05-10T11:02:58.983 INFO:teuthology.orchestra.run.smithi178.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-10T11:02:58.983 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-10T11:02:58.983 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-10T11:02:58.983 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-10T11:02:58.983 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-10T11:02:58.983 INFO:teuthology.orchestra.run.smithi178.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-10T11:02:58.983 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-10T11:02:58.983 INFO:teuthology.orchestra.run.smithi178.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-10T11:02:58.983 INFO:teuthology.orchestra.run.smithi178.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-10T11:02:58.984 INFO:teuthology.orchestra.run.smithi178.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-10T11:02:58.984 INFO:teuthology.orchestra.run.smithi178.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-10T11:02:58.984 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-10T11:02:58.984 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-10T11:02:58.984 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-10T11:02:58.984 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-10T11:02:58.984 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-10T11:02:58.984 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-10T11:02:58.984 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-10T11:02:58.984 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-10T11:02:58.985 INFO:teuthology.orchestra.run.smithi178.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-10T11:02:58.985 INFO:teuthology.orchestra.run.smithi178.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-10T11:02:58.985 INFO:teuthology.orchestra.run.smithi178.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-10T11:02:58.985 INFO:teuthology.orchestra.run.smithi178.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-10T11:02:58.985 INFO:teuthology.orchestra.run.smithi178.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-10T11:02:58.985 INFO:teuthology.orchestra.run.smithi178.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-10T11:02:58.985 INFO:teuthology.orchestra.run.smithi178.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-10T11:02:58.985 INFO:teuthology.orchestra.run.smithi178.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-10T11:02:58.985 INFO:teuthology.orchestra.run.smithi178.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-10T11:02:58.985 INFO:teuthology.orchestra.run.smithi178.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-10T11:02:58.985 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-10T11:02:58.985 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-10T11:02:58.985 INFO:teuthology.orchestra.run.smithi178.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-10T11:02:58.986 INFO:teuthology.orchestra.run.smithi178.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-10T11:02:58.986 INFO:teuthology.orchestra.run.smithi178.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-10T11:02:58.986 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-10T11:02:58.986 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-10T11:02:58.986 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-10T11:02:58.986 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-10T11:02:58.986 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-10T11:02:58.986 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-10T11:02:58.986 INFO:teuthology.orchestra.run.smithi178.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-10T11:02:58.986 INFO:teuthology.orchestra.run.smithi178.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-10T11:02:58.986 INFO:teuthology.orchestra.run.smithi178.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-10T11:02:58.987 INFO:teuthology.orchestra.run.smithi178.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-10T11:02:58.987 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-10T11:02:58.987 INFO:teuthology.orchestra.run.smithi178.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-10T11:02:58.987 INFO:teuthology.orchestra.run.smithi178.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-10T11:02:58.987 INFO:teuthology.orchestra.run.smithi178.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-10T11:02:58.987 INFO:teuthology.orchestra.run.smithi178.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-10T11:02:58.987 INFO:teuthology.orchestra.run.smithi178.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-10T11:02:58.987 INFO:teuthology.orchestra.run.smithi178.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-10T11:02:58.987 INFO:teuthology.orchestra.run.smithi178.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-10T11:02:58.987 INFO:teuthology.orchestra.run.smithi178.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-10T11:02:58.988 INFO:teuthology.orchestra.run.smithi178.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-10T11:02:58.988 INFO:teuthology.orchestra.run.smithi178.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-10T11:02:58.988 INFO:teuthology.orchestra.run.smithi178.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-10T11:02:58.988 INFO:teuthology.orchestra.run.smithi178.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-10T11:02:58.988 INFO:teuthology.orchestra.run.smithi178.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-10T11:02:58.988 INFO:teuthology.orchestra.run.smithi178.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-10T11:02:58.988 INFO:teuthology.orchestra.run.smithi178.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-10T11:02:58.988 INFO:teuthology.orchestra.run.smithi178.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-10T11:02:58.988 INFO:teuthology.orchestra.run.smithi178.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-10T11:02:58.988 INFO:teuthology.orchestra.run.smithi178.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-10T11:02:58.989 INFO:teuthology.orchestra.run.smithi178.stdout:Installing weak dependencies: 2024-05-10T11:02:58.989 INFO:teuthology.orchestra.run.smithi178.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-10T11:02:58.989 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-05-10T11:02:58.989 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-05-10T11:02:58.989 INFO:teuthology.orchestra.run.smithi178.stdout:======================================================================================= 2024-05-10T11:02:58.989 INFO:teuthology.orchestra.run.smithi178.stdout:Install 148 Packages 2024-05-10T11:02:58.989 INFO:teuthology.orchestra.run.smithi178.stdout:Upgrade 8 Packages 2024-05-10T11:02:58.989 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-05-10T11:02:58.990 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 226 M 2024-05-10T11:02:58.990 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-05-10T11:02:59.639 INFO:teuthology.orchestra.run.smithi178.stdout:(1/156): ceph-19.0.0-3449.gd39fa088.el9.x86_64. 39 kB/s | 6.6 kB 00:00 2024-05-10T11:02:59.865 INFO:teuthology.orchestra.run.smithi178.stdout:(2/156): ceph-fuse-19.0.0-3449.gd39fa088.el9.x8 3.9 MB/s | 890 kB 00:00 2024-05-10T11:02:59.915 INFO:teuthology.orchestra.run.smithi178.stdout:(3/156): ceph-immutable-object-cache-19.0.0-344 2.8 MB/s | 145 kB 00:00 2024-05-10T11:03:00.190 INFO:teuthology.orchestra.run.smithi178.stdout:(4/156): ceph-mds-19.0.0-3449.gd39fa088.el9.x86 8.6 MB/s | 2.4 MB 00:00 2024-05-10T11:03:00.221 INFO:teuthology.orchestra.run.smithi100.stdout:lab-extras 16 kB/s | 1.7 kB 00:00 2024-05-10T11:03:00.424 INFO:teuthology.orchestra.run.smithi178.stdout:(5/156): ceph-mgr-19.0.0-3449.gd39fa088.el9.x86 6.5 MB/s | 1.5 MB 00:00 2024-05-10T11:03:00.849 INFO:teuthology.orchestra.run.smithi178.stdout:(6/156): ceph-base-19.0.0-3449.gd39fa088.el9.x8 3.9 MB/s | 5.4 MB 00:01 2024-05-10T11:03:01.150 INFO:teuthology.orchestra.run.smithi178.stdout:(7/156): ceph-mon-19.0.0-3449.gd39fa088.el9.x86 6.5 MB/s | 4.7 MB 00:00 2024-05-10T11:03:01.641 INFO:teuthology.orchestra.run.smithi178.stdout:(8/156): ceph-common-19.0.0-3449.gd39fa088.el9. 9.7 MB/s | 21 MB 00:02 2024-05-10T11:03:01.699 INFO:teuthology.orchestra.run.smithi178.stdout:(9/156): ceph-selinux-19.0.0-3449.gd39fa088.el9 432 kB/s | 25 kB 00:00 2024-05-10T11:03:01.788 INFO:teuthology.orchestra.run.smithi188.stdout:lab-extras 41 kB/s | 1.7 kB 00:00 2024-05-10T11:03:02.513 INFO:teuthology.orchestra.run.smithi100.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-10T11:03:02.514 INFO:teuthology.orchestra.run.smithi100.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-10T11:03:02.524 INFO:teuthology.orchestra.run.smithi100.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-10T11:03:02.525 INFO:teuthology.orchestra.run.smithi100.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-10T11:03:02.526 INFO:teuthology.orchestra.run.smithi100.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-10T11:03:02.527 INFO:teuthology.orchestra.run.smithi100.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-10T11:03:02.528 INFO:teuthology.orchestra.run.smithi100.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-10T11:03:02.531 INFO:teuthology.orchestra.run.smithi100.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-10T11:03:02.531 INFO:teuthology.orchestra.run.smithi100.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-10T11:03:02.574 INFO:teuthology.orchestra.run.smithi178.stdout:(10/156): ceph-radosgw-19.0.0-3449.gd39fa088.el 7.4 MB/s | 11 MB 00:01 2024-05-10T11:03:02.592 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-05-10T11:03:02.609 INFO:teuthology.orchestra.run.smithi100.stdout:======================================================================================= 2024-05-10T11:03:02.610 INFO:teuthology.orchestra.run.smithi100.stdout: Package Arch Version Repository Size 2024-05-10T11:03:02.610 INFO:teuthology.orchestra.run.smithi100.stdout:======================================================================================= 2024-05-10T11:03:02.610 INFO:teuthology.orchestra.run.smithi100.stdout:Installing: 2024-05-10T11:03:02.610 INFO:teuthology.orchestra.run.smithi100.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-10T11:03:02.610 INFO:teuthology.orchestra.run.smithi100.stdout: ceph x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 6.6 k 2024-05-10T11:03:02.610 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-base x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 5.4 M 2024-05-10T11:03:02.610 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-fuse x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 890 k 2024-05-10T11:03:02.610 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 145 k 2024-05-10T11:03:02.610 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 1.5 M 2024-05-10T11:03:02.610 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 142 k 2024-05-10T11:03:02.610 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 3.5 M 2024-05-10T11:03:02.610 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 7.4 M 2024-05-10T11:03:02.610 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-rook noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 49 k 2024-05-10T11:03:02.611 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-radosgw x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 11 M 2024-05-10T11:03:02.611 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-test x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 49 M 2024-05-10T11:03:02.611 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-volume noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 273 k 2024-05-10T11:03:02.611 INFO:teuthology.orchestra.run.smithi100.stdout: cephadm noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 763 k 2024-05-10T11:03:02.611 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-mirror x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 225 k 2024-05-10T11:03:02.611 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-shell x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 27 k 2024-05-10T11:03:02.611 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-top noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 18 k 2024-05-10T11:03:02.611 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-10T11:03:02.611 INFO:teuthology.orchestra.run.smithi100.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-10T11:03:02.611 INFO:teuthology.orchestra.run.smithi100.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-10T11:03:02.611 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs-devel x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 32 k 2024-05-10T11:03:02.611 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs2 x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 746 k 2024-05-10T11:03:02.611 INFO:teuthology.orchestra.run.smithi100.stdout: librados-devel x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 127 k 2024-05-10T11:03:02.611 INFO:teuthology.orchestra.run.smithi100.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-10T11:03:02.612 INFO:teuthology.orchestra.run.smithi100.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-10T11:03:02.612 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-10T11:03:02.612 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-10T11:03:02.612 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-10T11:03:02.612 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-10T11:03:02.612 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cephfs x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 161 k 2024-05-10T11:03:02.612 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rados x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 321 k 2024-05-10T11:03:02.612 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rbd x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 298 k 2024-05-10T11:03:02.612 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rgw x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 99 k 2024-05-10T11:03:02.612 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-fuse x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 85 k 2024-05-10T11:03:02.612 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-mirror x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 3.1 M 2024-05-10T11:03:02.612 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-nbd x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 170 k 2024-05-10T11:03:02.612 INFO:teuthology.orchestra.run.smithi100.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-10T11:03:02.612 INFO:teuthology.orchestra.run.smithi100.stdout:Upgrading: 2024-05-10T11:03:02.613 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-10T11:03:02.613 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-10T11:03:02.613 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-10T11:03:02.613 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-10T11:03:02.613 INFO:teuthology.orchestra.run.smithi100.stdout: librados2 x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 3.4 M 2024-05-10T11:03:02.613 INFO:teuthology.orchestra.run.smithi100.stdout: librbd1 x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 3.1 M 2024-05-10T11:03:02.613 INFO:teuthology.orchestra.run.smithi100.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-10T11:03:02.613 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-10T11:03:02.613 INFO:teuthology.orchestra.run.smithi100.stdout:Installing dependencies: 2024-05-10T11:03:02.613 INFO:teuthology.orchestra.run.smithi100.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-10T11:03:02.613 INFO:teuthology.orchestra.run.smithi100.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-10T11:03:02.613 INFO:teuthology.orchestra.run.smithi100.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-10T11:03:02.613 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-common x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 21 M 2024-05-10T11:03:02.614 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 34 k 2024-05-10T11:03:02.614 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mds x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 2.4 M 2024-05-10T11:03:02.614 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 274 k 2024-05-10T11:03:02.614 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mon x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 4.7 M 2024-05-10T11:03:02.614 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-osd x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 17 M 2024-05-10T11:03:02.614 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 16 k 2024-05-10T11:03:02.614 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-selinux x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 25 k 2024-05-10T11:03:02.614 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-10T11:03:02.614 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-10T11:03:02.614 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-10T11:03:02.614 INFO:teuthology.orchestra.run.smithi100.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-10T11:03:02.614 INFO:teuthology.orchestra.run.smithi100.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-10T11:03:02.614 INFO:teuthology.orchestra.run.smithi100.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-10T11:03:02.614 INFO:teuthology.orchestra.run.smithi100.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-10T11:03:02.615 INFO:teuthology.orchestra.run.smithi100.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-10T11:03:02.615 INFO:teuthology.orchestra.run.smithi100.stdout: libcephsqlite x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 163 k 2024-05-10T11:03:02.615 INFO:teuthology.orchestra.run.smithi100.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-10T11:03:02.615 INFO:teuthology.orchestra.run.smithi100.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-10T11:03:02.615 INFO:teuthology.orchestra.run.smithi100.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-10T11:03:02.615 INFO:teuthology.orchestra.run.smithi100.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-10T11:03:02.615 INFO:teuthology.orchestra.run.smithi100.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-10T11:03:02.615 INFO:teuthology.orchestra.run.smithi100.stdout: libradosstriper1 x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 500 k 2024-05-10T11:03:02.615 INFO:teuthology.orchestra.run.smithi100.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-10T11:03:02.615 INFO:teuthology.orchestra.run.smithi100.stdout: librgw2 x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 5.4 M 2024-05-10T11:03:02.615 INFO:teuthology.orchestra.run.smithi100.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-10T11:03:02.615 INFO:teuthology.orchestra.run.smithi100.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-10T11:03:02.615 INFO:teuthology.orchestra.run.smithi100.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-10T11:03:02.615 INFO:teuthology.orchestra.run.smithi100.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-10T11:03:02.615 INFO:teuthology.orchestra.run.smithi100.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-10T11:03:02.616 INFO:teuthology.orchestra.run.smithi100.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-10T11:03:02.616 INFO:teuthology.orchestra.run.smithi100.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-10T11:03:02.616 INFO:teuthology.orchestra.run.smithi100.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-10T11:03:02.616 INFO:teuthology.orchestra.run.smithi100.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-10T11:03:02.616 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-10T11:03:02.616 INFO:teuthology.orchestra.run.smithi100.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-10T11:03:02.616 INFO:teuthology.orchestra.run.smithi100.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-10T11:03:02.616 INFO:teuthology.orchestra.run.smithi100.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-10T11:03:02.616 INFO:teuthology.orchestra.run.smithi100.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-10T11:03:02.616 INFO:teuthology.orchestra.run.smithi100.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-10T11:03:02.616 INFO:teuthology.orchestra.run.smithi100.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-10T11:03:02.616 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-10T11:03:02.616 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-argparse x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 45 k 2024-05-10T11:03:02.617 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-common x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 135 k 2024-05-10T11:03:02.617 INFO:teuthology.orchestra.run.smithi100.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-10T11:03:02.617 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-10T11:03:02.617 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-10T11:03:02.617 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-10T11:03:02.617 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-10T11:03:02.617 INFO:teuthology.orchestra.run.smithi100.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-10T11:03:02.617 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-10T11:03:02.617 INFO:teuthology.orchestra.run.smithi100.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-10T11:03:02.617 INFO:teuthology.orchestra.run.smithi100.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-10T11:03:02.617 INFO:teuthology.orchestra.run.smithi100.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-10T11:03:02.617 INFO:teuthology.orchestra.run.smithi100.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-10T11:03:02.617 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-10T11:03:02.617 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-10T11:03:02.617 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-10T11:03:02.618 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-10T11:03:02.618 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-10T11:03:02.618 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-10T11:03:02.618 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-10T11:03:02.618 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-10T11:03:02.618 INFO:teuthology.orchestra.run.smithi100.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-10T11:03:02.618 INFO:teuthology.orchestra.run.smithi100.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-10T11:03:02.618 INFO:teuthology.orchestra.run.smithi100.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-10T11:03:02.618 INFO:teuthology.orchestra.run.smithi100.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-10T11:03:02.618 INFO:teuthology.orchestra.run.smithi100.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-10T11:03:02.618 INFO:teuthology.orchestra.run.smithi100.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-10T11:03:02.618 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-10T11:03:02.618 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-10T11:03:02.618 INFO:teuthology.orchestra.run.smithi100.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-10T11:03:02.618 INFO:teuthology.orchestra.run.smithi100.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-10T11:03:02.619 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-10T11:03:02.619 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-10T11:03:02.619 INFO:teuthology.orchestra.run.smithi100.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-10T11:03:02.619 INFO:teuthology.orchestra.run.smithi100.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-10T11:03:02.619 INFO:teuthology.orchestra.run.smithi100.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-10T11:03:02.619 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-10T11:03:02.619 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-10T11:03:02.619 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-10T11:03:02.619 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-10T11:03:02.619 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-10T11:03:02.619 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-10T11:03:02.619 INFO:teuthology.orchestra.run.smithi100.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-10T11:03:02.619 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-10T11:03:02.619 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-10T11:03:02.620 INFO:teuthology.orchestra.run.smithi100.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-10T11:03:02.620 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-10T11:03:02.620 INFO:teuthology.orchestra.run.smithi100.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-10T11:03:02.620 INFO:teuthology.orchestra.run.smithi100.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-10T11:03:02.620 INFO:teuthology.orchestra.run.smithi100.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-10T11:03:02.620 INFO:teuthology.orchestra.run.smithi100.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-10T11:03:02.620 INFO:teuthology.orchestra.run.smithi100.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-10T11:03:02.620 INFO:teuthology.orchestra.run.smithi100.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-10T11:03:02.620 INFO:teuthology.orchestra.run.smithi100.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-10T11:03:02.620 INFO:teuthology.orchestra.run.smithi100.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-10T11:03:02.620 INFO:teuthology.orchestra.run.smithi100.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-10T11:03:02.620 INFO:teuthology.orchestra.run.smithi100.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-10T11:03:02.620 INFO:teuthology.orchestra.run.smithi100.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-10T11:03:02.620 INFO:teuthology.orchestra.run.smithi100.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-10T11:03:02.621 INFO:teuthology.orchestra.run.smithi100.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-10T11:03:02.621 INFO:teuthology.orchestra.run.smithi100.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-10T11:03:02.621 INFO:teuthology.orchestra.run.smithi100.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-10T11:03:02.621 INFO:teuthology.orchestra.run.smithi100.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-10T11:03:02.621 INFO:teuthology.orchestra.run.smithi100.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-10T11:03:02.621 INFO:teuthology.orchestra.run.smithi100.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-10T11:03:02.621 INFO:teuthology.orchestra.run.smithi100.stdout:Installing weak dependencies: 2024-05-10T11:03:02.621 INFO:teuthology.orchestra.run.smithi100.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-10T11:03:02.621 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:03:02.621 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction Summary 2024-05-10T11:03:02.621 INFO:teuthology.orchestra.run.smithi100.stdout:======================================================================================= 2024-05-10T11:03:02.621 INFO:teuthology.orchestra.run.smithi100.stdout:Install 148 Packages 2024-05-10T11:03:02.621 INFO:teuthology.orchestra.run.smithi100.stdout:Upgrade 8 Packages 2024-05-10T11:03:02.621 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:03:02.622 INFO:teuthology.orchestra.run.smithi100.stdout:Total download size: 226 M 2024-05-10T11:03:02.622 INFO:teuthology.orchestra.run.smithi100.stdout:Downloading Packages: 2024-05-10T11:03:02.650 INFO:teuthology.orchestra.run.smithi178.stdout:(11/156): cephfs-mirror-19.0.0-3449.gd39fa088.e 2.9 MB/s | 225 kB 00:00 2024-05-10T11:03:02.742 INFO:teuthology.orchestra.run.smithi178.stdout:(12/156): cephfs-shell-19.0.0-3449.gd39fa088.el 295 kB/s | 27 kB 00:00 2024-05-10T11:03:02.833 INFO:teuthology.orchestra.run.smithi178.stdout:(13/156): libcephfs-devel-19.0.0-3449.gd39fa088 346 kB/s | 32 kB 00:00 2024-05-10T11:03:03.076 INFO:teuthology.orchestra.run.smithi178.stdout:(14/156): libcephfs2-19.0.0-3449.gd39fa088.el9. 3.0 MB/s | 746 kB 00:00 2024-05-10T11:03:03.159 INFO:teuthology.orchestra.run.smithi178.stdout:(15/156): libcephsqlite-19.0.0-3449.gd39fa088.e 1.9 MB/s | 163 kB 00:00 2024-05-10T11:03:03.243 INFO:teuthology.orchestra.run.smithi178.stdout:(16/156): librados-devel-19.0.0-3449.gd39fa088. 1.5 MB/s | 127 kB 00:00 2024-05-10T11:03:03.393 INFO:teuthology.orchestra.run.smithi178.stdout:(17/156): libradosstriper1-19.0.0-3449.gd39fa08 3.3 MB/s | 500 kB 00:00 2024-05-10T11:03:03.397 INFO:teuthology.orchestra.run.smithi100.stdout:(1/156): ceph-19.0.0-3449.gd39fa088.el9.x86_64. 23 kB/s | 6.6 kB 00:00 2024-05-10T11:03:04.123 INFO:teuthology.orchestra.run.smithi188.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-10T11:03:04.124 INFO:teuthology.orchestra.run.smithi188.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-10T11:03:04.135 INFO:teuthology.orchestra.run.smithi188.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-10T11:03:04.136 INFO:teuthology.orchestra.run.smithi188.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-10T11:03:04.136 INFO:teuthology.orchestra.run.smithi188.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-10T11:03:04.137 INFO:teuthology.orchestra.run.smithi188.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-10T11:03:04.139 INFO:teuthology.orchestra.run.smithi188.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-10T11:03:04.141 INFO:teuthology.orchestra.run.smithi188.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-10T11:03:04.142 INFO:teuthology.orchestra.run.smithi188.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-10T11:03:04.204 INFO:teuthology.orchestra.run.smithi188.stdout:Dependencies resolved. 2024-05-10T11:03:04.218 INFO:teuthology.orchestra.run.smithi188.stdout:======================================================================================= 2024-05-10T11:03:04.218 INFO:teuthology.orchestra.run.smithi188.stdout: Package Arch Version Repository Size 2024-05-10T11:03:04.218 INFO:teuthology.orchestra.run.smithi188.stdout:======================================================================================= 2024-05-10T11:03:04.218 INFO:teuthology.orchestra.run.smithi188.stdout:Installing: 2024-05-10T11:03:04.218 INFO:teuthology.orchestra.run.smithi188.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-10T11:03:04.219 INFO:teuthology.orchestra.run.smithi188.stdout: ceph x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 6.6 k 2024-05-10T11:03:04.219 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-base x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 5.4 M 2024-05-10T11:03:04.219 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-fuse x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 890 k 2024-05-10T11:03:04.219 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 145 k 2024-05-10T11:03:04.219 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 1.5 M 2024-05-10T11:03:04.219 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 142 k 2024-05-10T11:03:04.219 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 3.5 M 2024-05-10T11:03:04.219 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 7.4 M 2024-05-10T11:03:04.219 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-rook noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 49 k 2024-05-10T11:03:04.219 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-radosgw x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 11 M 2024-05-10T11:03:04.219 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-test x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 49 M 2024-05-10T11:03:04.219 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-volume noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 273 k 2024-05-10T11:03:04.219 INFO:teuthology.orchestra.run.smithi188.stdout: cephadm noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 763 k 2024-05-10T11:03:04.219 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-mirror x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 225 k 2024-05-10T11:03:04.220 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-shell x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 27 k 2024-05-10T11:03:04.220 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-top noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 18 k 2024-05-10T11:03:04.220 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-10T11:03:04.220 INFO:teuthology.orchestra.run.smithi188.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-10T11:03:04.220 INFO:teuthology.orchestra.run.smithi188.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-10T11:03:04.220 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs-devel x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 32 k 2024-05-10T11:03:04.220 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs2 x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 746 k 2024-05-10T11:03:04.220 INFO:teuthology.orchestra.run.smithi188.stdout: librados-devel x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 127 k 2024-05-10T11:03:04.220 INFO:teuthology.orchestra.run.smithi188.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-10T11:03:04.220 INFO:teuthology.orchestra.run.smithi188.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-10T11:03:04.220 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-10T11:03:04.220 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-10T11:03:04.220 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-10T11:03:04.220 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-10T11:03:04.221 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cephfs x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 161 k 2024-05-10T11:03:04.221 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rados x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 321 k 2024-05-10T11:03:04.221 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rbd x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 298 k 2024-05-10T11:03:04.221 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rgw x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 99 k 2024-05-10T11:03:04.221 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-fuse x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 85 k 2024-05-10T11:03:04.221 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-mirror x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 3.1 M 2024-05-10T11:03:04.221 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-nbd x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 170 k 2024-05-10T11:03:04.221 INFO:teuthology.orchestra.run.smithi188.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-10T11:03:04.221 INFO:teuthology.orchestra.run.smithi188.stdout:Upgrading: 2024-05-10T11:03:04.221 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-10T11:03:04.221 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-10T11:03:04.221 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-10T11:03:04.221 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-10T11:03:04.221 INFO:teuthology.orchestra.run.smithi188.stdout: librados2 x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 3.4 M 2024-05-10T11:03:04.222 INFO:teuthology.orchestra.run.smithi188.stdout: librbd1 x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 3.1 M 2024-05-10T11:03:04.222 INFO:teuthology.orchestra.run.smithi188.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-10T11:03:04.222 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-10T11:03:04.222 INFO:teuthology.orchestra.run.smithi188.stdout:Installing dependencies: 2024-05-10T11:03:04.222 INFO:teuthology.orchestra.run.smithi188.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-10T11:03:04.222 INFO:teuthology.orchestra.run.smithi188.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-10T11:03:04.222 INFO:teuthology.orchestra.run.smithi188.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-10T11:03:04.222 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-common x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 21 M 2024-05-10T11:03:04.222 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 34 k 2024-05-10T11:03:04.222 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mds x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 2.4 M 2024-05-10T11:03:04.222 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 274 k 2024-05-10T11:03:04.222 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mon x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 4.7 M 2024-05-10T11:03:04.223 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-osd x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 17 M 2024-05-10T11:03:04.223 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 16 k 2024-05-10T11:03:04.223 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-selinux x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 25 k 2024-05-10T11:03:04.223 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-10T11:03:04.223 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-10T11:03:04.223 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-10T11:03:04.223 INFO:teuthology.orchestra.run.smithi188.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-10T11:03:04.223 INFO:teuthology.orchestra.run.smithi188.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-10T11:03:04.223 INFO:teuthology.orchestra.run.smithi188.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-10T11:03:04.223 INFO:teuthology.orchestra.run.smithi188.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-10T11:03:04.223 INFO:teuthology.orchestra.run.smithi188.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-10T11:03:04.223 INFO:teuthology.orchestra.run.smithi188.stdout: libcephsqlite x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 163 k 2024-05-10T11:03:04.223 INFO:teuthology.orchestra.run.smithi188.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-10T11:03:04.223 INFO:teuthology.orchestra.run.smithi188.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-10T11:03:04.224 INFO:teuthology.orchestra.run.smithi188.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-10T11:03:04.224 INFO:teuthology.orchestra.run.smithi188.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-10T11:03:04.224 INFO:teuthology.orchestra.run.smithi188.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-10T11:03:04.224 INFO:teuthology.orchestra.run.smithi188.stdout: libradosstriper1 x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 500 k 2024-05-10T11:03:04.224 INFO:teuthology.orchestra.run.smithi188.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-10T11:03:04.224 INFO:teuthology.orchestra.run.smithi188.stdout: librgw2 x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 5.4 M 2024-05-10T11:03:04.224 INFO:teuthology.orchestra.run.smithi188.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-10T11:03:04.224 INFO:teuthology.orchestra.run.smithi188.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-10T11:03:04.224 INFO:teuthology.orchestra.run.smithi188.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-10T11:03:04.224 INFO:teuthology.orchestra.run.smithi188.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-10T11:03:04.224 INFO:teuthology.orchestra.run.smithi188.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-10T11:03:04.224 INFO:teuthology.orchestra.run.smithi188.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-10T11:03:04.224 INFO:teuthology.orchestra.run.smithi188.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-10T11:03:04.224 INFO:teuthology.orchestra.run.smithi188.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-10T11:03:04.225 INFO:teuthology.orchestra.run.smithi188.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-10T11:03:04.225 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-10T11:03:04.225 INFO:teuthology.orchestra.run.smithi188.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-10T11:03:04.225 INFO:teuthology.orchestra.run.smithi188.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-10T11:03:04.225 INFO:teuthology.orchestra.run.smithi188.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-10T11:03:04.225 INFO:teuthology.orchestra.run.smithi188.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-10T11:03:04.225 INFO:teuthology.orchestra.run.smithi188.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-10T11:03:04.225 INFO:teuthology.orchestra.run.smithi188.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-10T11:03:04.225 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-10T11:03:04.225 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-argparse x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 45 k 2024-05-10T11:03:04.225 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-common x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 135 k 2024-05-10T11:03:04.225 INFO:teuthology.orchestra.run.smithi188.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-10T11:03:04.225 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-10T11:03:04.226 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-10T11:03:04.226 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-10T11:03:04.226 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-10T11:03:04.226 INFO:teuthology.orchestra.run.smithi188.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-10T11:03:04.226 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-10T11:03:04.226 INFO:teuthology.orchestra.run.smithi188.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-10T11:03:04.226 INFO:teuthology.orchestra.run.smithi188.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-10T11:03:04.226 INFO:teuthology.orchestra.run.smithi188.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-10T11:03:04.226 INFO:teuthology.orchestra.run.smithi188.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-10T11:03:04.226 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-10T11:03:04.226 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-10T11:03:04.226 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-10T11:03:04.226 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-10T11:03:04.227 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-10T11:03:04.227 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-10T11:03:04.227 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-10T11:03:04.227 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-10T11:03:04.227 INFO:teuthology.orchestra.run.smithi188.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-10T11:03:04.227 INFO:teuthology.orchestra.run.smithi188.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-10T11:03:04.227 INFO:teuthology.orchestra.run.smithi188.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-10T11:03:04.227 INFO:teuthology.orchestra.run.smithi188.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-10T11:03:04.227 INFO:teuthology.orchestra.run.smithi188.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-10T11:03:04.227 INFO:teuthology.orchestra.run.smithi188.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-10T11:03:04.227 INFO:teuthology.orchestra.run.smithi188.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-10T11:03:04.227 INFO:teuthology.orchestra.run.smithi188.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-10T11:03:04.227 INFO:teuthology.orchestra.run.smithi188.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-10T11:03:04.227 INFO:teuthology.orchestra.run.smithi188.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-10T11:03:04.227 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-10T11:03:04.228 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-10T11:03:04.228 INFO:teuthology.orchestra.run.smithi188.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-10T11:03:04.228 INFO:teuthology.orchestra.run.smithi188.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-10T11:03:04.228 INFO:teuthology.orchestra.run.smithi188.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-10T11:03:04.228 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-10T11:03:04.228 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-10T11:03:04.228 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-10T11:03:04.228 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-10T11:03:04.228 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-10T11:03:04.228 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-10T11:03:04.228 INFO:teuthology.orchestra.run.smithi188.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-10T11:03:04.228 INFO:teuthology.orchestra.run.smithi188.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-10T11:03:04.228 INFO:teuthology.orchestra.run.smithi188.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-10T11:03:04.228 INFO:teuthology.orchestra.run.smithi188.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-10T11:03:04.229 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-10T11:03:04.229 INFO:teuthology.orchestra.run.smithi188.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-10T11:03:04.229 INFO:teuthology.orchestra.run.smithi188.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-10T11:03:04.229 INFO:teuthology.orchestra.run.smithi188.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-10T11:03:04.229 INFO:teuthology.orchestra.run.smithi188.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-10T11:03:04.229 INFO:teuthology.orchestra.run.smithi188.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-10T11:03:04.229 INFO:teuthology.orchestra.run.smithi188.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-10T11:03:04.229 INFO:teuthology.orchestra.run.smithi188.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-10T11:03:04.229 INFO:teuthology.orchestra.run.smithi188.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-10T11:03:04.229 INFO:teuthology.orchestra.run.smithi188.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-10T11:03:04.229 INFO:teuthology.orchestra.run.smithi188.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-10T11:03:04.229 INFO:teuthology.orchestra.run.smithi188.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-10T11:03:04.229 INFO:teuthology.orchestra.run.smithi188.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-10T11:03:04.230 INFO:teuthology.orchestra.run.smithi188.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-10T11:03:04.230 INFO:teuthology.orchestra.run.smithi188.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-10T11:03:04.230 INFO:teuthology.orchestra.run.smithi188.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-10T11:03:04.230 INFO:teuthology.orchestra.run.smithi188.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-10T11:03:04.230 INFO:teuthology.orchestra.run.smithi188.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-10T11:03:04.230 INFO:teuthology.orchestra.run.smithi188.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-10T11:03:04.230 INFO:teuthology.orchestra.run.smithi188.stdout:Installing weak dependencies: 2024-05-10T11:03:04.230 INFO:teuthology.orchestra.run.smithi188.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-10T11:03:04.230 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-10T11:03:04.230 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction Summary 2024-05-10T11:03:04.230 INFO:teuthology.orchestra.run.smithi188.stdout:======================================================================================= 2024-05-10T11:03:04.230 INFO:teuthology.orchestra.run.smithi188.stdout:Install 148 Packages 2024-05-10T11:03:04.230 INFO:teuthology.orchestra.run.smithi188.stdout:Upgrade 8 Packages 2024-05-10T11:03:04.230 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-10T11:03:04.231 INFO:teuthology.orchestra.run.smithi188.stdout:Total download size: 226 M 2024-05-10T11:03:04.231 INFO:teuthology.orchestra.run.smithi188.stdout:Downloading Packages: 2024-05-10T11:03:04.410 INFO:teuthology.orchestra.run.smithi100.stdout:(2/156): ceph-fuse-19.0.0-3449.gd39fa088.el9.x8 878 kB/s | 890 kB 00:01 2024-05-10T11:03:04.473 INFO:teuthology.orchestra.run.smithi100.stdout:(3/156): ceph-immutable-object-cache-19.0.0-344 2.3 MB/s | 145 kB 00:00 2024-05-10T11:03:04.685 INFO:teuthology.orchestra.run.smithi178.stdout:(18/156): ceph-test-19.0.0-3449.gd39fa088.el9.x 16 MB/s | 49 MB 00:02 2024-05-10T11:03:04.769 INFO:teuthology.orchestra.run.smithi178.stdout:(19/156): librgw2-19.0.0-3449.gd39fa088.el9.x86 3.9 MB/s | 5.4 MB 00:01 2024-05-10T11:03:04.794 INFO:teuthology.orchestra.run.smithi178.stdout:(20/156): python3-ceph-argparse-19.0.0-3449.gd3 416 kB/s | 45 kB 00:00 2024-05-10T11:03:04.873 INFO:teuthology.orchestra.run.smithi100.stdout:(4/156): ceph-mds-19.0.0-3449.gd39fa088.el9.x86 5.9 MB/s | 2.4 MB 00:00 2024-05-10T11:03:04.986 INFO:teuthology.orchestra.run.smithi178.stdout:(21/156): ceph-osd-19.0.0-3449.gd39fa088.el9.x8 4.1 MB/s | 17 MB 00:04 2024-05-10T11:03:05.003 INFO:teuthology.orchestra.run.smithi178.stdout:(22/156): python3-ceph-common-19.0.0-3449.gd39f 580 kB/s | 135 kB 00:00 2024-05-10T11:03:05.028 INFO:teuthology.orchestra.run.smithi178.stdout:(23/156): python3-cephfs-19.0.0-3449.gd39fa088. 691 kB/s | 161 kB 00:00 2024-05-10T11:03:05.065 INFO:teuthology.orchestra.run.smithi188.stdout:(1/156): ceph-19.0.0-3449.gd39fa088.el9.x86_64. 30 kB/s | 6.6 kB 00:00 2024-05-10T11:03:05.103 INFO:teuthology.orchestra.run.smithi178.stdout:(24/156): python3-rbd-19.0.0-3449.gd39fa088.el9 2.9 MB/s | 298 kB 00:00 2024-05-10T11:03:05.128 INFO:teuthology.orchestra.run.smithi178.stdout:(25/156): python3-rgw-19.0.0-3449.gd39fa088.el9 990 kB/s | 99 kB 00:00 2024-05-10T11:03:05.196 INFO:teuthology.orchestra.run.smithi178.stdout:(26/156): rbd-fuse-19.0.0-3449.gd39fa088.el9.x8 930 kB/s | 85 kB 00:00 2024-05-10T11:03:05.288 INFO:teuthology.orchestra.run.smithi178.stdout:(27/156): rbd-nbd-19.0.0-3449.gd39fa088.el9.x86 1.8 MB/s | 170 kB 00:00 2024-05-10T11:03:05.290 INFO:teuthology.orchestra.run.smithi100.stdout:(5/156): ceph-mgr-19.0.0-3449.gd39fa088.el9.x86 3.6 MB/s | 1.5 MB 00:00 2024-05-10T11:03:05.355 INFO:teuthology.orchestra.run.smithi178.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3449.g 511 kB/s | 34 kB 00:00 2024-05-10T11:03:05.447 INFO:teuthology.orchestra.run.smithi178.stdout:(29/156): ceph-mgr-cephadm-19.0.0-3449.gd39fa08 1.5 MB/s | 142 kB 00:00 2024-05-10T11:03:05.547 INFO:teuthology.orchestra.run.smithi178.stdout:(30/156): rbd-mirror-19.0.0-3449.gd39fa088.el9. 7.4 MB/s | 3.1 MB 00:00 2024-05-10T11:03:05.572 INFO:teuthology.orchestra.run.smithi178.stdout:(31/156): python3-rados-19.0.0-3449.gd39fa088.e 548 kB/s | 321 kB 00:00 2024-05-10T11:03:05.664 INFO:teuthology.orchestra.run.smithi178.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3449.gd3 2.9 MB/s | 274 kB 00:00 2024-05-10T11:03:05.739 INFO:teuthology.orchestra.run.smithi178.stdout:(33/156): ceph-mgr-rook-19.0.0-3449.gd39fa088.e 658 kB/s | 49 kB 00:00 2024-05-10T11:03:05.815 INFO:teuthology.orchestra.run.smithi178.stdout:(34/156): ceph-prometheus-alerts-19.0.0-3449.gd 220 kB/s | 16 kB 00:00 2024-05-10T11:03:05.898 INFO:teuthology.orchestra.run.smithi178.stdout:(35/156): ceph-volume-19.0.0-3449.gd39fa088.el9 3.2 MB/s | 273 kB 00:00 2024-05-10T11:03:06.007 INFO:teuthology.orchestra.run.smithi178.stdout:(36/156): cephadm-19.0.0-3449.gd39fa088.el9.noa 6.9 MB/s | 763 kB 00:00 2024-05-10T11:03:06.041 INFO:teuthology.orchestra.run.smithi100.stdout:(6/156): ceph-mon-19.0.0-3449.gd39fa088.el9.x86 6.3 MB/s | 4.7 MB 00:00 2024-05-10T11:03:06.049 INFO:teuthology.orchestra.run.smithi178.stdout:(37/156): cephfs-top-19.0.0-3449.gd39fa088.el9. 438 kB/s | 18 kB 00:00 2024-05-10T11:03:06.116 INFO:teuthology.orchestra.run.smithi178.stdout:(38/156): ceph-mgr-dashboard-19.0.0-3449.gd39fa 5.3 MB/s | 3.5 MB 00:00 2024-05-10T11:03:06.199 INFO:teuthology.orchestra.run.smithi178.stdout:(39/156): fuse-2.9.9-15.el9.x86_64.rpm 535 kB/s | 80 kB 00:00 2024-05-10T11:03:06.257 INFO:teuthology.orchestra.run.smithi100.stdout:(7/156): ceph-base-19.0.0-3449.gd39fa088.el9.x8 1.7 MB/s | 5.4 MB 00:03 2024-05-10T11:03:06.266 INFO:teuthology.orchestra.run.smithi178.stdout:(40/156): libquadmath-11.4.1-3.el9.x86_64.rpm 2.9 MB/s | 195 kB 00:00 2024-05-10T11:03:06.325 INFO:teuthology.orchestra.run.smithi178.stdout:(41/156): libgfortran-11.4.1-3.el9.x86_64.rpm 3.8 MB/s | 803 kB 00:00 2024-05-10T11:03:06.350 INFO:teuthology.orchestra.run.smithi178.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.0 MB/s | 253 kB 00:00 2024-05-10T11:03:06.417 INFO:teuthology.orchestra.run.smithi178.stdout:(43/156): python3-cryptography-36.0.1-4.el9.x86 13 MB/s | 1.2 MB 00:00 2024-05-10T11:03:06.442 INFO:teuthology.orchestra.run.smithi178.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 106 kB 00:00 2024-05-10T11:03:06.468 INFO:teuthology.orchestra.run.smithi178.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-05-10T11:03:06.493 INFO:teuthology.orchestra.run.smithi178.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-05-10T11:03:06.518 INFO:teuthology.orchestra.run.smithi178.stdout:(47/156): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-05-10T11:03:06.752 INFO:teuthology.orchestra.run.smithi178.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 2.6 MB/s | 689 kB 00:00 2024-05-10T11:03:06.811 INFO:teuthology.orchestra.run.smithi178.stdout:(49/156): bison-3.7.4-5.el9.x86_64.rpm 3.2 MB/s | 943 kB 00:00 2024-05-10T11:03:06.836 INFO:teuthology.orchestra.run.smithi178.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 1.2 MB/s | 107 kB 00:00 2024-05-10T11:03:06.839 INFO:teuthology.orchestra.run.smithi188.stdout:(2/156): ceph-fuse-19.0.0-3449.gd39fa088.el9.x8 501 kB/s | 890 kB 00:01 2024-05-10T11:03:06.861 INFO:teuthology.orchestra.run.smithi178.stdout:(51/156): elfutils-libelf-devel-0.190-2.el9.x86 430 kB/s | 21 kB 00:00 2024-05-10T11:03:06.912 INFO:teuthology.orchestra.run.smithi178.stdout:(52/156): flexiblas-3.0.4-8.el9.x86_64.rpm 652 kB/s | 33 kB 00:00 2024-05-10T11:03:06.945 INFO:teuthology.orchestra.run.smithi178.stdout:(53/156): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-05-10T11:03:06.996 INFO:teuthology.orchestra.run.smithi178.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 363 kB/s | 18 kB 00:00 2024-05-10T11:03:07.046 INFO:teuthology.orchestra.run.smithi178.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 224 kB/s | 11 kB 00:00 2024-05-10T11:03:07.104 INFO:teuthology.orchestra.run.smithi178.stdout:(56/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-05-10T11:03:07.140 INFO:teuthology.orchestra.run.smithi188.stdout:(3/156): ceph-immutable-object-cache-19.0.0-344 483 kB/s | 145 kB 00:00 2024-05-10T11:03:07.155 INFO:teuthology.orchestra.run.smithi178.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 903 kB/s | 45 kB 00:00 2024-05-10T11:03:07.305 INFO:teuthology.orchestra.run.smithi178.stdout:(58/156): librdkafka-1.6.1-102.el9.x86_64.rpm 4.3 MB/s | 662 kB 00:00 2024-05-10T11:03:07.366 INFO:teuthology.orchestra.run.smithi100.stdout:(8/156): ceph-osd-19.0.0-3449.gd39fa088.el9.x86 13 MB/s | 17 MB 00:01 2024-05-10T11:03:07.425 INFO:teuthology.orchestra.run.smithi100.stdout:(9/156): ceph-selinux-19.0.0-3449.gd39fa088.el9 432 kB/s | 25 kB 00:00 2024-05-10T11:03:07.430 INFO:teuthology.orchestra.run.smithi178.stdout:(59/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.8 MB/s | 3.0 MB 00:00 2024-05-10T11:03:07.456 INFO:teuthology.orchestra.run.smithi178.stdout:(60/156): libtool-2.4.6-45.el9.x86_64.rpm 3.8 MB/s | 582 kB 00:00 2024-05-10T11:03:07.473 INFO:teuthology.orchestra.run.smithi178.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 608 kB/s | 25 kB 00:00 2024-05-10T11:03:07.506 INFO:teuthology.orchestra.run.smithi178.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-05-10T11:03:07.556 INFO:teuthology.orchestra.run.smithi178.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-05-10T11:03:07.582 INFO:teuthology.orchestra.run.smithi178.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 2.5 MB/s | 188 kB 00:00 2024-05-10T11:03:07.607 INFO:teuthology.orchestra.run.smithi178.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 702 kB/s | 35 kB 00:00 2024-05-10T11:03:07.675 INFO:teuthology.orchestra.run.smithi100.stdout:(10/156): ceph-common-19.0.0-3449.gd39fa088.el9 4.6 MB/s | 21 MB 00:04 2024-05-10T11:03:07.809 INFO:teuthology.orchestra.run.smithi100.stdout:(11/156): cephfs-mirror-19.0.0-3449.gd39fa088.e 1.6 MB/s | 225 kB 00:00 2024-05-10T11:03:07.909 INFO:teuthology.orchestra.run.smithi100.stdout:(12/156): cephfs-shell-19.0.0-3449.gd39fa088.el 270 kB/s | 27 kB 00:00 2024-05-10T11:03:07.957 INFO:teuthology.orchestra.run.smithi178.stdout:(66/156): ceph-mgr-diskprediction-local-19.0.0- 3.1 MB/s | 7.4 MB 00:02 2024-05-10T11:03:07.992 INFO:teuthology.orchestra.run.smithi100.stdout:(13/156): libcephfs-devel-19.0.0-3449.gd39fa088 382 kB/s | 32 kB 00:00 2024-05-10T11:03:08.234 INFO:teuthology.orchestra.run.smithi100.stdout:(14/156): libcephfs2-19.0.0-3449.gd39fa088.el9. 3.0 MB/s | 746 kB 00:00 2024-05-10T11:03:08.308 INFO:teuthology.orchestra.run.smithi178.stdout:(67/156): openblas-openmp-0.3.21-2.el9.x86_64.r 6.5 MB/s | 4.7 MB 00:00 2024-05-10T11:03:08.326 INFO:teuthology.orchestra.run.smithi100.stdout:(15/156): libcephsqlite-19.0.0-3449.gd39fa088.e 1.7 MB/s | 163 kB 00:00 2024-05-10T11:03:08.418 INFO:teuthology.orchestra.run.smithi100.stdout:(16/156): librados-devel-19.0.0-3449.gd39fa088. 1.4 MB/s | 127 kB 00:00 2024-05-10T11:03:08.449 INFO:teuthology.orchestra.run.smithi178.stdout:(68/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 5.3 MB/s | 4.4 MB 00:00 2024-05-10T11:03:08.491 INFO:teuthology.orchestra.run.smithi178.stdout:(69/156): postgresql-13.14-1.el9.x86_64.rpm 3.0 MB/s | 1.6 MB 00:00 2024-05-10T11:03:08.542 INFO:teuthology.orchestra.run.smithi178.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 1.4 MB/s | 133 kB 00:00 2024-05-10T11:03:08.584 INFO:teuthology.orchestra.run.smithi178.stdout:(71/156): postgresql-contrib-13.14-1.el9.x86_64 3.2 MB/s | 911 kB 00:00 2024-05-10T11:03:08.642 INFO:teuthology.orchestra.run.smithi178.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-05-10T11:03:08.767 INFO:teuthology.orchestra.run.smithi178.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 4.5 MB/s | 1.0 MB 00:00 2024-05-10T11:03:08.859 INFO:teuthology.orchestra.run.smithi178.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 2.6 MB/s | 244 kB 00:00 2024-05-10T11:03:08.943 INFO:teuthology.orchestra.run.smithi178.stdout:(75/156): python3-jinja2-2.11.3-5.el9.noarch.rp 2.9 MB/s | 248 kB 00:00 2024-05-10T11:03:08.943 INFO:teuthology.orchestra.run.smithi100.stdout:(17/156): libradosstriper1-19.0.0-3449.gd39fa08 953 kB/s | 500 kB 00:00 2024-05-10T11:03:08.993 INFO:teuthology.orchestra.run.smithi178.stdout:(76/156): python3-jsonpatch-1.21-16.el9.noarch. 512 kB/s | 26 kB 00:00 2024-05-10T11:03:09.044 INFO:teuthology.orchestra.run.smithi178.stdout:(77/156): python3-jsonpointer-2.0-4.el9.noarch. 384 kB/s | 19 kB 00:00 2024-05-10T11:03:09.102 INFO:teuthology.orchestra.run.smithi178.stdout:(78/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-05-10T11:03:09.152 INFO:teuthology.orchestra.run.smithi178.stdout:(79/156): python3-markupsafe-1.1.1-12.el9.x86_6 693 kB/s | 35 kB 00:00 2024-05-10T11:03:09.511 INFO:teuthology.orchestra.run.smithi178.stdout:(80/156): python3-babel-2.9.1-2.el9.noarch.rpm 6.9 MB/s | 6.0 MB 00:00 2024-05-10T11:03:09.586 INFO:teuthology.orchestra.run.smithi178.stdout:(81/156): postgresql-server-13.14-1.el9.x86_64. 5.3 MB/s | 5.8 MB 00:01 2024-05-10T11:03:09.589 INFO:teuthology.orchestra.run.smithi188.stdout:(4/156): ceph-mds-19.0.0-3449.gd39fa088.el9.x86 988 kB/s | 2.4 MB 00:02 2024-05-10T11:03:09.637 INFO:teuthology.orchestra.run.smithi178.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.0 MB/s | 250 kB 00:00 2024-05-10T11:03:09.687 INFO:teuthology.orchestra.run.smithi178.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-05-10T11:03:09.712 INFO:teuthology.orchestra.run.smithi178.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 1.0 MB/s | 77 kB 00:00 2024-05-10T11:03:09.738 INFO:teuthology.orchestra.run.smithi178.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 839 kB/s | 42 kB 00:00 2024-05-10T11:03:09.796 INFO:teuthology.orchestra.run.smithi178.stdout:(86/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-05-10T11:03:09.821 INFO:teuthology.orchestra.run.smithi178.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 2.4 MB/s | 269 kB 00:00 2024-05-10T11:03:09.872 INFO:teuthology.orchestra.run.smithi178.stdout:(88/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-05-10T11:03:09.897 INFO:teuthology.orchestra.run.smithi178.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 279 kB 00:00 2024-05-10T11:03:09.922 INFO:teuthology.orchestra.run.smithi178.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-05-10T11:03:09.973 INFO:teuthology.orchestra.run.smithi178.stdout:(91/156): python3-toml-0.10.2-6.el9.noarch.rpm 832 kB/s | 42 kB 00:00 2024-05-10T11:03:10.023 INFO:teuthology.orchestra.run.smithi178.stdout:(92/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 886 kB/s | 44 kB 00:00 2024-05-10T11:03:10.082 INFO:teuthology.orchestra.run.smithi178.stdout:(93/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-05-10T11:03:10.140 INFO:teuthology.orchestra.run.smithi178.stdout:(94/156): qatlib-service-24.02.0-1.el9.x86_64.r 616 kB/s | 36 kB 00:00 2024-05-10T11:03:10.216 INFO:teuthology.orchestra.run.smithi178.stdout:(95/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.8 MB/s | 5.1 MB 00:01 2024-05-10T11:03:10.223 INFO:teuthology.orchestra.run.smithi188.stdout:(5/156): ceph-common-19.0.0-3449.gd39fa088.el9. 3.9 MB/s | 21 MB 00:05 2024-05-10T11:03:10.232 INFO:teuthology.orchestra.run.smithi178.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 515 kB/s | 47 kB 00:00 2024-05-10T11:03:10.283 INFO:teuthology.orchestra.run.smithi178.stdout:(97/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-05-10T11:03:10.308 INFO:teuthology.orchestra.run.smithi178.stdout:(98/156): socat-1.7.4.1-5.el9.x86_64.rpm 3.2 MB/s | 305 kB 00:00 2024-05-10T11:03:10.342 INFO:teuthology.orchestra.run.smithi178.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.6 MB/s | 94 kB 00:00 2024-05-10T11:03:10.358 INFO:teuthology.orchestra.run.smithi178.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-05-10T11:03:10.392 INFO:teuthology.orchestra.run.smithi178.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 894 kB/s | 45 kB 00:00 2024-05-10T11:03:10.409 INFO:teuthology.orchestra.run.smithi178.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 444 kB/s | 22 kB 00:00 2024-05-10T11:03:10.448 INFO:teuthology.orchestra.run.smithi188.stdout:(6/156): ceph-base-19.0.0-3449.gd39fa088.el9.x8 981 kB/s | 5.4 MB 00:05 2024-05-10T11:03:10.526 INFO:teuthology.orchestra.run.smithi178.stdout:(103/156): abseil-cpp-20211102.0-3.el9.x86_64.r 4.6 MB/s | 548 kB 00:00 2024-05-10T11:03:10.551 INFO:teuthology.orchestra.run.smithi178.stdout:(104/156): protobuf-compiler-3.14.0-13.el9.x86_ 5.3 MB/s | 863 kB 00:00 2024-05-10T11:03:10.577 INFO:teuthology.orchestra.run.smithi178.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 6.0 MB/s | 309 kB 00:00 2024-05-10T11:03:10.607 INFO:teuthology.orchestra.run.smithi188.stdout:(7/156): ceph-mgr-19.0.0-3449.gd39fa088.el9.x86 1.5 MB/s | 1.5 MB 00:01 2024-05-10T11:03:10.627 INFO:teuthology.orchestra.run.smithi178.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 258 kB/s | 19 kB 00:00 2024-05-10T11:03:10.660 INFO:teuthology.orchestra.run.smithi178.stdout:(107/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 784 kB/s | 26 kB 00:00 2024-05-10T11:03:10.724 INFO:teuthology.orchestra.run.smithi188.stdout:(8/156): ceph-mon-19.0.0-3449.gd39fa088.el9.x86 9.4 MB/s | 4.7 MB 00:00 2024-05-10T11:03:10.727 INFO:teuthology.orchestra.run.smithi178.stdout:(108/156): libarrow-9.0.0-11.el9.x86_64.rpm 29 MB/s | 4.4 MB 00:00 2024-05-10T11:03:10.761 INFO:teuthology.orchestra.run.smithi178.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 484 kB/s | 49 kB 00:00 2024-05-10T11:03:10.783 INFO:teuthology.orchestra.run.smithi188.stdout:(9/156): ceph-selinux-19.0.0-3449.gd39fa088.el9 432 kB/s | 25 kB 00:00 2024-05-10T11:03:10.786 INFO:teuthology.orchestra.run.smithi178.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-05-10T11:03:10.812 INFO:teuthology.orchestra.run.smithi178.stdout:(111/156): luarocks-3.9.2-1.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-05-10T11:03:10.854 INFO:teuthology.orchestra.run.smithi178.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 12 MB/s | 839 kB 00:00 2024-05-10T11:03:10.879 INFO:teuthology.orchestra.run.smithi178.stdout:(113/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-05-10T11:03:10.913 INFO:teuthology.orchestra.run.smithi178.stdout:(114/156): python3-asyncssh-2.13.2-5.el9.noarch 5.3 MB/s | 548 kB 00:00 2024-05-10T11:03:10.930 INFO:teuthology.orchestra.run.smithi178.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 639 kB/s | 32 kB 00:00 2024-05-10T11:03:10.946 INFO:teuthology.orchestra.run.smithi178.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 448 kB/s | 15 kB 00:00 2024-05-10T11:03:10.973 INFO:teuthology.orchestra.run.smithi178.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 4.0 MB/s | 172 kB 00:00 2024-05-10T11:03:10.997 INFO:teuthology.orchestra.run.smithi178.stdout:(118/156): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-05-10T11:03:11.022 INFO:teuthology.orchestra.run.smithi178.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-05-10T11:03:11.044 INFO:teuthology.orchestra.run.smithi100.stdout:(18/156): ceph-test-19.0.0-3449.gd39fa088.el9.x 14 MB/s | 49 MB 00:03 2024-05-10T11:03:11.048 INFO:teuthology.orchestra.run.smithi178.stdout:(120/156): python3-colorama-0.4.4-12.el9.noarch 703 kB/s | 35 kB 00:00 2024-05-10T11:03:11.073 INFO:teuthology.orchestra.run.smithi178.stdout:(121/156): python3-google-auth-2.29.0-1.el9.noa 4.0 MB/s | 206 kB 00:00 2024-05-10T11:03:11.119 INFO:teuthology.orchestra.run.smithi100.stdout:(19/156): librgw2-19.0.0-3449.gd39fa088.el9.x86 2.5 MB/s | 5.4 MB 00:02 2024-05-10T11:03:11.132 INFO:teuthology.orchestra.run.smithi178.stdout:(122/156): python3-grpcio-1.46.7-10.el9.x86_64. 24 MB/s | 2.0 MB 00:00 2024-05-10T11:03:11.144 INFO:teuthology.orchestra.run.smithi100.stdout:(20/156): python3-ceph-argparse-19.0.0-3449.gd3 451 kB/s | 45 kB 00:00 2024-05-10T11:03:11.157 INFO:teuthology.orchestra.run.smithi178.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 1.7 MB/s | 144 kB 00:00 2024-05-10T11:03:11.186 INFO:teuthology.orchestra.run.smithi100.stdout:(21/156): python3-ceph-common-19.0.0-3449.gd39f 2.0 MB/s | 135 kB 00:00 2024-05-10T11:03:11.191 INFO:teuthology.orchestra.run.smithi178.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 321 kB/s | 11 kB 00:00 2024-05-10T11:03:11.216 INFO:teuthology.orchestra.run.smithi178.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 302 kB/s | 18 kB 00:00 2024-05-10T11:03:11.219 INFO:teuthology.orchestra.run.smithi100.stdout:(22/156): python3-cephfs-19.0.0-3449.gd39fa088. 2.1 MB/s | 161 kB 00:00 2024-05-10T11:03:11.241 INFO:teuthology.orchestra.run.smithi178.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 462 kB/s | 23 kB 00:00 2024-05-10T11:03:11.266 INFO:teuthology.orchestra.run.smithi178.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 388 kB/s | 19 kB 00:00 2024-05-10T11:03:11.270 INFO:teuthology.orchestra.run.smithi100.stdout:(23/156): python3-rados-19.0.0-3449.gd39fa088.e 3.7 MB/s | 321 kB 00:00 2024-05-10T11:03:11.292 INFO:teuthology.orchestra.run.smithi178.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-05-10T11:03:11.320 INFO:teuthology.orchestra.run.smithi100.stdout:(24/156): python3-rbd-19.0.0-3449.gd39fa088.el9 2.9 MB/s | 298 kB 00:00 2024-05-10T11:03:11.334 INFO:teuthology.orchestra.run.smithi178.stdout:(129/156): python3-kubernetes-26.1.0-2.el9.noar 15 MB/s | 1.0 MB 00:00 2024-05-10T11:03:11.345 INFO:teuthology.orchestra.run.smithi100.stdout:(25/156): python3-rgw-19.0.0-3449.gd39fa088.el9 1.3 MB/s | 99 kB 00:00 2024-05-10T11:03:11.359 INFO:teuthology.orchestra.run.smithi178.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 691 kB/s | 46 kB 00:00 2024-05-10T11:03:11.371 INFO:teuthology.orchestra.run.smithi100.stdout:(26/156): rbd-fuse-19.0.0-3449.gd39fa088.el9.x8 1.7 MB/s | 85 kB 00:00 2024-05-10T11:03:11.376 INFO:teuthology.orchestra.run.smithi178.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 1.8 MB/s | 79 kB 00:00 2024-05-10T11:03:11.401 INFO:teuthology.orchestra.run.smithi178.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 1.3 MB/s | 58 kB 00:00 2024-05-10T11:03:11.426 INFO:teuthology.orchestra.run.smithi178.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 5.3 MB/s | 272 kB 00:00 2024-05-10T11:03:11.438 INFO:teuthology.orchestra.run.smithi100.stdout:(27/156): rbd-nbd-19.0.0-3449.gd39fa088.el9.x86 2.5 MB/s | 170 kB 00:00 2024-05-10T11:03:11.452 INFO:teuthology.orchestra.run.smithi178.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 327 kB/s | 16 kB 00:00 2024-05-10T11:03:11.477 INFO:teuthology.orchestra.run.smithi178.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-05-10T11:03:11.496 INFO:teuthology.orchestra.run.smithi100.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3449.g 588 kB/s | 34 kB 00:00 2024-05-10T11:03:11.502 INFO:teuthology.orchestra.run.smithi178.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 562 kB/s | 28 kB 00:00 2024-05-10T11:03:11.528 INFO:teuthology.orchestra.run.smithi178.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-05-10T11:03:11.553 INFO:teuthology.orchestra.run.smithi178.stdout:(138/156): python3-routes-2.5.1-5.el9.noarch.rp 3.7 MB/s | 188 kB 00:00 2024-05-10T11:03:11.555 INFO:teuthology.orchestra.run.smithi100.stdout:(29/156): ceph-mgr-cephadm-19.0.0-3449.gd39fa08 2.4 MB/s | 142 kB 00:00 2024-05-10T11:03:11.578 INFO:teuthology.orchestra.run.smithi178.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-05-10T11:03:11.603 INFO:teuthology.orchestra.run.smithi178.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 712 kB/s | 36 kB 00:00 2024-05-10T11:03:11.629 INFO:teuthology.orchestra.run.smithi178.stdout:(141/156): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-05-10T11:03:11.654 INFO:teuthology.orchestra.run.smithi178.stdout:(142/156): python3-webob-1.8.7-6.el9.noarch.rpm 4.5 MB/s | 230 kB 00:00 2024-05-10T11:03:11.680 INFO:teuthology.orchestra.run.smithi178.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-05-10T11:03:11.705 INFO:teuthology.orchestra.run.smithi100.stdout:(30/156): ceph-radosgw-19.0.0-3449.gd39fa088.el 1.9 MB/s | 11 MB 00:05 2024-05-10T11:03:11.713 INFO:teuthology.orchestra.run.smithi178.stdout:(144/156): python3-werkzeug-2.0.3-3.el9.1.noarc 7.1 MB/s | 427 kB 00:00 2024-05-10T11:03:11.739 INFO:teuthology.orchestra.run.smithi178.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 341 kB/s | 20 kB 00:00 2024-05-10T11:03:11.764 INFO:teuthology.orchestra.run.smithi178.stdout:(146/156): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-05-10T11:03:11.856 INFO:teuthology.orchestra.run.smithi178.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-05-10T11:03:11.922 INFO:teuthology.orchestra.run.smithi100.stdout:(31/156): ceph-mgr-dashboard-19.0.0-3449.gd39fa 9.6 MB/s | 3.5 MB 00:00 2024-05-10T11:03:11.998 INFO:teuthology.orchestra.run.smithi100.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3449.gd3 3.6 MB/s | 274 kB 00:00 2024-05-10T11:03:12.065 INFO:teuthology.orchestra.run.smithi100.stdout:(33/156): rbd-mirror-19.0.0-3449.gd39fa088.el9. 4.3 MB/s | 3.1 MB 00:00 2024-05-10T11:03:12.090 INFO:teuthology.orchestra.run.smithi100.stdout:(34/156): ceph-mgr-rook-19.0.0-3449.gd39fa088.e 536 kB/s | 49 kB 00:00 2024-05-10T11:03:12.132 INFO:teuthology.orchestra.run.smithi100.stdout:(35/156): ceph-prometheus-alerts-19.0.0-3449.gd 245 kB/s | 16 kB 00:00 2024-05-10T11:03:12.174 INFO:teuthology.orchestra.run.smithi100.stdout:(36/156): ceph-volume-19.0.0-3449.gd39fa088.el9 3.2 MB/s | 273 kB 00:00 2024-05-10T11:03:12.249 INFO:teuthology.orchestra.run.smithi100.stdout:(37/156): cephfs-top-19.0.0-3449.gd39fa088.el9. 243 kB/s | 18 kB 00:00 2024-05-10T11:03:12.383 INFO:teuthology.orchestra.run.smithi100.stdout:(38/156): fuse-2.9.9-15.el9.x86_64.rpm 603 kB/s | 80 kB 00:00 2024-05-10T11:03:12.408 INFO:teuthology.orchestra.run.smithi100.stdout:(39/156): cephadm-19.0.0-3449.gd39fa088.el9.noa 2.7 MB/s | 763 kB 00:00 2024-05-10T11:03:12.492 INFO:teuthology.orchestra.run.smithi100.stdout:(40/156): libgfortran-11.4.1-3.el9.x86_64.rpm 7.2 MB/s | 803 kB 00:00 2024-05-10T11:03:12.498 INFO:teuthology.orchestra.run.smithi178.stdout:(148/156): librados2-19.0.0-3449.gd39fa088.el9. 4.6 MB/s | 3.4 MB 00:00 2024-05-10T11:03:12.534 INFO:teuthology.orchestra.run.smithi100.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.9 MB/s | 253 kB 00:00 2024-05-10T11:03:12.559 INFO:teuthology.orchestra.run.smithi100.stdout:(42/156): libquadmath-11.4.1-3.el9.x86_64.rpm 1.3 MB/s | 195 kB 00:00 2024-05-10T11:03:12.609 INFO:teuthology.orchestra.run.smithi100.stdout:(43/156): python3-cryptography-36.0.1-4.el9.x86 16 MB/s | 1.2 MB 00:00 2024-05-10T11:03:12.635 INFO:teuthology.orchestra.run.smithi100.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-05-10T11:03:12.660 INFO:teuthology.orchestra.run.smithi100.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-05-10T11:03:12.685 INFO:teuthology.orchestra.run.smithi100.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-05-10T11:03:12.698 INFO:teuthology.orchestra.run.smithi178.stdout:(149/156): NetworkManager-1.47.5-1.el9.x86_64.r 12 MB/s | 2.3 MB 00:00 2024-05-10T11:03:12.710 INFO:teuthology.orchestra.run.smithi100.stdout:(47/156): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-05-10T11:03:12.873 INFO:teuthology.orchestra.run.smithi178.stdout:(150/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 5.5 MB/s | 16 MB 00:02 2024-05-10T11:03:12.923 INFO:teuthology.orchestra.run.smithi178.stdout:(151/156): NetworkManager-libnm-1.47.5-1.el9.x8 8.1 MB/s | 1.8 MB 00:00 2024-05-10T11:03:12.957 INFO:teuthology.orchestra.run.smithi178.stdout:(152/156): NetworkManager-team-1.47.5-1.el9.x86 501 kB/s | 42 kB 00:00 2024-05-10T11:03:12.969 INFO:teuthology.orchestra.run.smithi100.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 2.4 MB/s | 689 kB 00:00 2024-05-10T11:03:12.982 INFO:teuthology.orchestra.run.smithi178.stdout:(153/156): NetworkManager-tui-1.47.5-1.el9.x86_ 4.1 MB/s | 248 kB 00:00 2024-05-10T11:03:13.003 INFO:teuthology.orchestra.run.smithi100.stdout:(49/156): bison-3.7.4-5.el9.x86_64.rpm 3.2 MB/s | 943 kB 00:00 2024-05-10T11:03:13.049 INFO:teuthology.orchestra.run.smithi178.stdout:(154/156): librbd1-19.0.0-3449.gd39fa088.el9.x8 2.6 MB/s | 3.1 MB 00:01 2024-05-10T11:03:13.053 INFO:teuthology.orchestra.run.smithi100.stdout:(50/156): elfutils-libelf-devel-0.190-2.el9.x86 430 kB/s | 21 kB 00:00 2024-05-10T11:03:13.078 INFO:teuthology.orchestra.run.smithi100.stdout:(51/156): boost-program-options-1.75.0-8.el9.x8 982 kB/s | 107 kB 00:00 2024-05-10T11:03:13.108 INFO:teuthology.orchestra.run.smithi178.stdout:(155/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 19 MB/s | 2.4 MB 00:00 2024-05-10T11:03:13.129 INFO:teuthology.orchestra.run.smithi100.stdout:(52/156): flexiblas-3.0.4-8.el9.x86_64.rpm 654 kB/s | 33 kB 00:00 2024-05-10T11:03:13.171 INFO:teuthology.orchestra.run.smithi100.stdout:(53/156): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-05-10T11:03:13.208 INFO:teuthology.orchestra.run.smithi178.stdout:(156/156): openssl-3.2.1-1.el9.x86_64.rpm 5.4 MB/s | 1.3 MB 00:00 2024-05-10T11:03:13.210 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-05-10T11:03:13.211 INFO:teuthology.orchestra.run.smithi178.stdout:Total 16 MB/s | 226 MB 00:14 2024-05-10T11:03:13.221 INFO:teuthology.orchestra.run.smithi100.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 364 kB/s | 18 kB 00:00 2024-05-10T11:03:13.263 INFO:teuthology.orchestra.run.smithi100.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 269 kB/s | 11 kB 00:00 2024-05-10T11:03:13.355 INFO:teuthology.orchestra.run.smithi100.stdout:(56/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 13 MB/s | 3.0 MB 00:00 2024-05-10T11:03:13.380 INFO:teuthology.orchestra.run.smithi100.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-05-10T11:03:13.422 INFO:teuthology.orchestra.run.smithi100.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 677 kB/s | 45 kB 00:00 2024-05-10T11:03:13.474 INFO:teuthology.orchestra.run.smithi188.stdout:(10/156): ceph-radosgw-19.0.0-3449.gd39fa088.el 3.7 MB/s | 11 MB 00:02 2024-05-10T11:03:13.489 INFO:teuthology.orchestra.run.smithi100.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 6.0 MB/s | 662 kB 00:00 2024-05-10T11:03:13.514 INFO:teuthology.orchestra.run.smithi100.stdout:(60/156): libtool-2.4.6-45.el9.x86_64.rpm 6.2 MB/s | 582 kB 00:00 2024-05-10T11:03:13.541 INFO:teuthology.orchestra.run.smithi188.stdout:(11/156): cephfs-mirror-19.0.0-3449.gd39fa088.e 3.3 MB/s | 225 kB 00:00 2024-05-10T11:03:13.556 INFO:teuthology.orchestra.run.smithi100.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 381 kB/s | 25 kB 00:00 2024-05-10T11:03:13.573 INFO:teuthology.orchestra.run.smithi100.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 811 kB/s | 47 kB 00:00 2024-05-10T11:03:13.591 INFO:teuthology.orchestra.run.smithi188.stdout:(12/156): cephfs-shell-19.0.0-3449.gd39fa088.el 539 kB/s | 27 kB 00:00 2024-05-10T11:03:13.649 INFO:teuthology.orchestra.run.smithi100.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.1 MB/s | 292 kB 00:00 2024-05-10T11:03:13.650 INFO:teuthology.orchestra.run.smithi188.stdout:(13/156): libcephfs-devel-19.0.0-3449.gd39fa088 544 kB/s | 32 kB 00:00 2024-05-10T11:03:13.674 INFO:teuthology.orchestra.run.smithi100.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 1.8 MB/s | 188 kB 00:00 2024-05-10T11:03:13.716 INFO:teuthology.orchestra.run.smithi100.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 526 kB/s | 35 kB 00:00 2024-05-10T11:03:13.817 INFO:teuthology.orchestra.run.smithi188.stdout:(14/156): libcephfs2-19.0.0-3449.gd39fa088.el9. 4.4 MB/s | 746 kB 00:00 2024-05-10T11:03:13.892 INFO:teuthology.orchestra.run.smithi188.stdout:(15/156): libcephsqlite-19.0.0-3449.gd39fa088.e 2.1 MB/s | 163 kB 00:00 2024-05-10T11:03:13.951 INFO:teuthology.orchestra.run.smithi188.stdout:(16/156): librados-devel-19.0.0-3449.gd39fa088. 2.1 MB/s | 127 kB 00:00 2024-05-10T11:03:14.093 INFO:teuthology.orchestra.run.smithi188.stdout:(17/156): libradosstriper1-19.0.0-3449.gd39fa08 3.7 MB/s | 500 kB 00:00 2024-05-10T11:03:14.093 INFO:teuthology.orchestra.run.smithi100.stdout:(66/156): openblas-openmp-0.3.21-2.el9.x86_64.r 16 MB/s | 4.7 MB 00:00 2024-05-10T11:03:14.093 INFO:teuthology.orchestra.run.smithi100.stdout:(67/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-05-10T11:03:14.151 INFO:teuthology.orchestra.run.smithi100.stdout:(68/156): postgresql-contrib-13.14-1.el9.x86_64 7.6 MB/s | 911 kB 00:00 2024-05-10T11:03:14.192 INFO:teuthology.orchestra.run.smithi100.stdout:(69/156): postgresql-13.14-1.el9.x86_64.rpm 7.3 MB/s | 1.6 MB 00:00 2024-05-10T11:03:14.251 INFO:teuthology.orchestra.run.smithi100.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 1.3 MB/s | 133 kB 00:00 2024-05-10T11:03:14.327 INFO:teuthology.orchestra.run.smithi188.stdout:(18/156): ceph-osd-19.0.0-3449.gd39fa088.el9.x8 4.3 MB/s | 17 MB 00:03 2024-05-10T11:03:14.369 INFO:teuthology.orchestra.run.smithi188.stdout:(19/156): python3-ceph-argparse-19.0.0-3449.gd3 1.1 MB/s | 45 kB 00:00 2024-05-10T11:03:14.418 INFO:teuthology.orchestra.run.smithi100.stdout:(71/156): ceph-mgr-diskprediction-local-19.0.0- 2.7 MB/s | 7.4 MB 00:02 2024-05-10T11:03:14.427 INFO:teuthology.orchestra.run.smithi188.stdout:(20/156): python3-ceph-common-19.0.0-3449.gd39f 2.3 MB/s | 135 kB 00:00 2024-05-10T11:03:14.432 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-05-10T11:03:14.452 INFO:teuthology.orchestra.run.smithi100.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 5.0 MB/s | 1.0 MB 00:00 2024-05-10T11:03:14.486 INFO:teuthology.orchestra.run.smithi188.stdout:(21/156): python3-cephfs-19.0.0-3449.gd39fa088. 2.7 MB/s | 161 kB 00:00 2024-05-10T11:03:14.544 INFO:teuthology.orchestra.run.smithi100.stdout:(73/156): postgresql-server-13.14-1.el9.x86_64. 16 MB/s | 5.8 MB 00:00 2024-05-10T11:03:14.569 INFO:teuthology.orchestra.run.smithi100.stdout:(74/156): python3-attrs-20.3.0-7.el9.noarch.rpm 581 kB/s | 87 kB 00:00 2024-05-10T11:03:14.569 INFO:teuthology.orchestra.run.smithi188.stdout:(22/156): python3-rados-19.0.0-3449.gd39fa088.e 3.8 MB/s | 321 kB 00:00 2024-05-10T11:03:14.581 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-05-10T11:03:14.581 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-05-10T11:03:14.636 INFO:teuthology.orchestra.run.smithi100.stdout:(75/156): python3-devel-3.9.18-3.el9.x86_64.rpm 2.6 MB/s | 244 kB 00:00 2024-05-10T11:03:14.661 INFO:teuthology.orchestra.run.smithi100.stdout:(76/156): python3-jinja2-2.11.3-5.el9.noarch.rp 2.6 MB/s | 248 kB 00:00 2024-05-10T11:03:14.670 INFO:teuthology.orchestra.run.smithi188.stdout:(23/156): python3-rbd-19.0.0-3449.gd39fa088.el9 2.9 MB/s | 298 kB 00:00 2024-05-10T11:03:14.703 INFO:teuthology.orchestra.run.smithi100.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 383 kB/s | 26 kB 00:00 2024-05-10T11:03:14.728 INFO:teuthology.orchestra.run.smithi100.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 287 kB/s | 19 kB 00:00 2024-05-10T11:03:14.729 INFO:teuthology.orchestra.run.smithi188.stdout:(24/156): python3-rgw-19.0.0-3449.gd39fa088.el9 1.7 MB/s | 99 kB 00:00 2024-05-10T11:03:14.770 INFO:teuthology.orchestra.run.smithi188.stdout:(25/156): rbd-fuse-19.0.0-3449.gd39fa088.el9.x8 2.0 MB/s | 85 kB 00:00 2024-05-10T11:03:14.778 INFO:teuthology.orchestra.run.smithi100.stdout:(79/156): python3-markupsafe-1.1.1-12.el9.x86_6 695 kB/s | 35 kB 00:00 2024-05-10T11:03:14.804 INFO:teuthology.orchestra.run.smithi100.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.7 MB/s | 172 kB 00:00 2024-05-10T11:03:14.888 INFO:teuthology.orchestra.run.smithi100.stdout:(81/156): python3-babel-2.9.1-2.el9.noarch.rpm 14 MB/s | 6.0 MB 00:00 2024-05-10T11:03:14.946 INFO:teuthology.orchestra.run.smithi100.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.7 MB/s | 250 kB 00:00 2024-05-10T11:03:14.996 INFO:teuthology.orchestra.run.smithi100.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.0 MB/s | 222 kB 00:00 2024-05-10T11:03:15.022 INFO:teuthology.orchestra.run.smithi100.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 1.0 MB/s | 77 kB 00:00 2024-05-10T11:03:15.071 INFO:teuthology.orchestra.run.smithi188.stdout:(26/156): librgw2-19.0.0-3449.gd39fa088.el9.x86 5.5 MB/s | 5.4 MB 00:00 2024-05-10T11:03:15.072 INFO:teuthology.orchestra.run.smithi100.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 561 kB/s | 42 kB 00:00 2024-05-10T11:03:15.122 INFO:teuthology.orchestra.run.smithi100.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 2.6 MB/s | 269 kB 00:00 2024-05-10T11:03:15.146 INFO:teuthology.orchestra.run.smithi188.stdout:(27/156): rbd-nbd-19.0.0-3449.gd39fa088.el9.x86 2.2 MB/s | 170 kB 00:00 2024-05-10T11:03:15.164 INFO:teuthology.orchestra.run.smithi100.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 159 kB 00:00 2024-05-10T11:03:15.205 INFO:teuthology.orchestra.run.smithi188.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3449.g 585 kB/s | 34 kB 00:00 2024-05-10T11:03:15.214 INFO:teuthology.orchestra.run.smithi100.stdout:(88/156): python3-pyasn1-modules-0.4.8-6.el9.no 3.0 MB/s | 279 kB 00:00 2024-05-10T11:03:15.240 INFO:teuthology.orchestra.run.smithi100.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 677 kB/s | 51 kB 00:00 2024-05-10T11:03:15.315 INFO:teuthology.orchestra.run.smithi100.stdout:(90/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 9.5 MB/s | 5.1 MB 00:00 2024-05-10T11:03:15.341 INFO:teuthology.orchestra.run.smithi100.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 426 kB/s | 54 kB 00:00 2024-05-10T11:03:15.391 INFO:teuthology.orchestra.run.smithi100.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 553 kB/s | 42 kB 00:00 2024-05-10T11:03:15.416 INFO:teuthology.orchestra.run.smithi100.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 587 kB/s | 44 kB 00:00 2024-05-10T11:03:15.467 INFO:teuthology.orchestra.run.smithi100.stdout:(94/156): qatlib-service-24.02.0-1.el9.x86_64.r 716 kB/s | 36 kB 00:00 2024-05-10T11:03:15.492 INFO:teuthology.orchestra.run.smithi100.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 2.1 MB/s | 221 kB 00:00 2024-05-10T11:03:15.542 INFO:teuthology.orchestra.run.smithi100.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 630 kB/s | 47 kB 00:00 2024-05-10T11:03:15.601 INFO:teuthology.orchestra.run.smithi100.stdout:(97/156): socat-1.7.4.1-5.el9.x86_64.rpm 2.7 MB/s | 305 kB 00:00 2024-05-10T11:03:15.626 INFO:teuthology.orchestra.run.smithi100.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 690 kB/s | 58 kB 00:00 2024-05-10T11:03:15.638 INFO:teuthology.orchestra.run.smithi188.stdout:(29/156): ceph-test-19.0.0-3449.gd39fa088.el9.x 10 MB/s | 49 MB 00:04 2024-05-10T11:03:15.655 INFO:teuthology.orchestra.run.smithi188.stdout:(30/156): ceph-mgr-cephadm-19.0.0-3449.gd39fa08 316 kB/s | 142 kB 00:00 2024-05-10T11:03:15.693 INFO:teuthology.orchestra.run.smithi100.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.0 MB/s | 94 kB 00:00 2024-05-10T11:03:15.706 INFO:teuthology.orchestra.run.smithi188.stdout:(31/156): rbd-mirror-19.0.0-3449.gd39fa088.el9. 3.3 MB/s | 3.1 MB 00:00 2024-05-10T11:03:15.718 INFO:teuthology.orchestra.run.smithi100.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 693 kB/s | 64 kB 00:00 2024-05-10T11:03:15.777 INFO:teuthology.orchestra.run.smithi100.stdout:(101/156): lua-devel-5.4.4-4.el9.x86_64.rpm 382 kB/s | 22 kB 00:00 2024-05-10T11:03:15.802 INFO:teuthology.orchestra.run.smithi100.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 411 kB/s | 45 kB 00:00 2024-05-10T11:03:15.806 INFO:teuthology.orchestra.run.smithi188.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3449.gd3 2.7 MB/s | 274 kB 00:00 2024-05-10T11:03:15.865 INFO:teuthology.orchestra.run.smithi188.stdout:(33/156): ceph-mgr-rook-19.0.0-3449.gd39fa088.e 848 kB/s | 49 kB 00:00 2024-05-10T11:03:15.903 INFO:teuthology.orchestra.run.smithi100.stdout:(103/156): protobuf-compiler-3.14.0-13.el9.x86_ 6.7 MB/s | 863 kB 00:00 2024-05-10T11:03:15.923 INFO:teuthology.orchestra.run.smithi188.stdout:(34/156): ceph-prometheus-alerts-19.0.0-3449.gd 283 kB/s | 16 kB 00:00 2024-05-10T11:03:16.007 INFO:teuthology.orchestra.run.smithi188.stdout:(35/156): ceph-volume-19.0.0-3449.gd39fa088.el9 3.2 MB/s | 273 kB 00:00 2024-05-10T11:03:16.065 INFO:teuthology.orchestra.run.smithi188.stdout:(36/156): ceph-mgr-dashboard-19.0.0-3449.gd39fa 8.3 MB/s | 3.5 MB 00:00 2024-05-10T11:03:16.132 INFO:teuthology.orchestra.run.smithi188.stdout:(37/156): cephfs-top-19.0.0-3449.gd39fa088.el9. 274 kB/s | 18 kB 00:00 2024-05-10T11:03:16.136 INFO:teuthology.orchestra.run.smithi100.stdout:(104/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 18 MB/s | 16 MB 00:00 2024-05-10T11:03:16.183 INFO:teuthology.orchestra.run.smithi188.stdout:(38/156): cephadm-19.0.0-3449.gd39fa088.el9.noa 4.2 MB/s | 763 kB 00:00 2024-05-10T11:03:16.291 INFO:teuthology.orchestra.run.smithi188.stdout:(39/156): fuse-2.9.9-15.el9.x86_64.rpm 507 kB/s | 80 kB 00:00 2024-05-10T11:03:16.417 INFO:teuthology.orchestra.run.smithi188.stdout:(40/156): libquadmath-11.4.1-3.el9.x86_64.rpm 1.5 MB/s | 195 kB 00:00 2024-05-10T11:03:16.509 INFO:teuthology.orchestra.run.smithi188.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.7 MB/s | 253 kB 00:00 2024-05-10T11:03:16.542 INFO:teuthology.orchestra.run.smithi188.stdout:(42/156): libgfortran-11.4.1-3.el9.x86_64.rpm 2.2 MB/s | 803 kB 00:00 2024-05-10T11:03:16.545 INFO:teuthology.orchestra.run.smithi100.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 51 kB/s | 19 kB 00:00 2024-05-10T11:03:16.601 INFO:teuthology.orchestra.run.smithi188.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-05-10T11:03:16.685 INFO:teuthology.orchestra.run.smithi188.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 135 kB 00:00 2024-05-10T11:03:16.695 INFO:teuthology.orchestra.run.smithi100.stdout:(106/156): abseil-cpp-20211102.0-3.el9.x86_64.r 614 kB/s | 548 kB 00:00 2024-05-10T11:03:16.721 INFO:teuthology.orchestra.run.smithi100.stdout:(107/156): gperftools-libs-2.9.1-2.el9.x86_64.r 389 kB/s | 309 kB 00:00 2024-05-10T11:03:16.736 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-05-10T11:03:16.737 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-05-10T11:03:16.793 INFO:teuthology.orchestra.run.smithi188.stdout:(45/156): ceph-mgr-diskprediction-local-19.0.0- 6.5 MB/s | 7.4 MB 00:01 2024-05-10T11:03:16.804 INFO:teuthology.orchestra.run.smithi100.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 239 kB/s | 26 kB 00:00 2024-05-10T11:03:16.818 INFO:teuthology.orchestra.run.smithi188.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 934 kB/s | 125 kB 00:00 2024-05-10T11:03:16.821 INFO:teuthology.orchestra.run.smithi100.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 484 kB/s | 49 kB 00:00 2024-05-10T11:03:16.852 INFO:teuthology.orchestra.run.smithi188.stdout:(47/156): python3-cryptography-36.0.1-4.el9.x86 3.6 MB/s | 1.2 MB 00:00 2024-05-10T11:03:16.914 INFO:teuthology.orchestra.run.smithi100.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 620 kB/s | 67 kB 00:00 2024-05-10T11:03:16.955 INFO:teuthology.orchestra.run.smithi100.stdout:(111/156): luarocks-3.9.2-1.el9.noarch.rpm 1.1 MB/s | 151 kB 00:00 2024-05-10T11:03:17.019 INFO:teuthology.orchestra.run.smithi188.stdout:(48/156): python3-urllib3-1.26.5-5.el9.noarch.r 952 kB/s | 215 kB 00:00 2024-05-10T11:03:17.122 INFO:teuthology.orchestra.run.smithi100.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.9 MB/s | 839 kB 00:00 2024-05-10T11:03:17.164 INFO:teuthology.orchestra.run.smithi100.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 2.6 MB/s | 548 kB 00:00 2024-05-10T11:03:17.231 INFO:teuthology.orchestra.run.smithi100.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 400 kB/s | 43 kB 00:00 2024-05-10T11:03:17.315 INFO:teuthology.orchestra.run.smithi100.stdout:(115/156): libarrow-9.0.0-11.el9.x86_64.rpm 5.7 MB/s | 4.4 MB 00:00 2024-05-10T11:03:17.341 INFO:teuthology.orchestra.run.smithi100.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 138 kB/s | 15 kB 00:00 2024-05-10T11:03:17.366 INFO:teuthology.orchestra.run.smithi100.stdout:(117/156): python3-cachetools-4.2.4-1.el9.noarc 160 kB/s | 32 kB 00:00 2024-05-10T11:03:17.433 INFO:teuthology.orchestra.run.smithi100.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 1.4 MB/s | 172 kB 00:00 2024-05-10T11:03:17.453 INFO:teuthology.orchestra.run.smithi188.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 247 kB/s | 107 kB 00:00 2024-05-10T11:03:17.458 INFO:teuthology.orchestra.run.smithi100.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 3.0 MB/s | 358 kB 00:00 2024-05-10T11:03:17.503 INFO:teuthology.orchestra.run.smithi188.stdout:(50/156): automake-1.16.2-8.el9.noarch.rpm 1.0 MB/s | 689 kB 00:00 2024-05-10T11:03:17.508 INFO:teuthology.orchestra.run.smithi100.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.1 MB/s | 303 kB 00:00 2024-05-10T11:03:17.567 INFO:teuthology.orchestra.run.smithi100.stdout:(121/156): python3-colorama-0.4.4-12.el9.noarch 264 kB/s | 35 kB 00:00 2024-05-10T11:03:17.570 INFO:teuthology.orchestra.run.smithi188.stdout:(51/156): elfutils-libelf-devel-0.190-2.el9.x86 183 kB/s | 21 kB 00:00 2024-05-10T11:03:17.593 INFO:teuthology.orchestra.run.smithi100.stdout:(122/156): python3-google-auth-2.29.0-1.el9.noa 1.5 MB/s | 206 kB 00:00 2024-05-10T11:03:17.629 INFO:teuthology.orchestra.run.smithi188.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 2.4 MB/s | 313 kB 00:00 2024-05-10T11:03:17.709 INFO:teuthology.orchestra.run.smithi100.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 1.0 MB/s | 144 kB 00:00 2024-05-10T11:03:17.721 INFO:teuthology.orchestra.run.smithi188.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 217 kB/s | 33 kB 00:00 2024-05-10T11:03:17.735 INFO:teuthology.orchestra.run.smithi100.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 75 kB/s | 11 kB 00:00 2024-05-10T11:03:17.838 INFO:teuthology.orchestra.run.smithi188.stdout:(54/156): bison-3.7.4-5.el9.x86_64.rpm 957 kB/s | 943 kB 00:00 2024-05-10T11:03:17.843 INFO:teuthology.orchestra.run.smithi100.stdout:(125/156): python3-grpcio-1.46.7-10.el9.x86_64. 6.1 MB/s | 2.0 MB 00:00 2024-05-10T11:03:17.863 INFO:teuthology.orchestra.run.smithi188.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 128 kB/s | 18 kB 00:00 2024-05-10T11:03:17.868 INFO:teuthology.orchestra.run.smithi100.stdout:(126/156): python3-jaraco-classes-3.2.1-5.el9.n 111 kB/s | 18 kB 00:00 2024-05-10T11:03:17.894 INFO:teuthology.orchestra.run.smithi100.stdout:(127/156): python3-jaraco-collections-3.0.0-8.e 146 kB/s | 23 kB 00:00 2024-05-10T11:03:17.955 INFO:teuthology.orchestra.run.smithi188.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 95 kB/s | 11 kB 00:00 2024-05-10T11:03:17.969 INFO:teuthology.orchestra.run.smithi100.stdout:(128/156): python3-jaraco-functools-3.5.0-2.el9 155 kB/s | 19 kB 00:00 2024-05-10T11:03:18.002 INFO:teuthology.orchestra.run.smithi100.stdout:(129/156): python3-jaraco-text-3.2.0-6.el9.noar 181 kB/s | 20 kB 00:00 2024-05-10T11:03:18.045 INFO:teuthology.orchestra.run.smithi100.stdout:(130/156): python3-kubernetes-26.1.0-2.el9.noar 6.8 MB/s | 1.0 MB 00:00 2024-05-10T11:03:18.072 INFO:teuthology.orchestra.run.smithi188.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 387 kB/s | 45 kB 00:00 2024-05-10T11:03:18.086 INFO:teuthology.orchestra.run.smithi100.stdout:(131/156): python3-logutils-0.3.5-21.el9.noarch 395 kB/s | 46 kB 00:00 2024-05-10T11:03:18.131 INFO:teuthology.orchestra.run.smithi188.stdout:(58/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.0 MB/s | 3.0 MB 00:00 2024-05-10T11:03:18.136 INFO:teuthology.orchestra.run.smithi100.stdout:(132/156): python3-more-itertools-8.12.0-2.el9. 589 kB/s | 79 kB 00:00 2024-05-10T11:03:18.162 INFO:teuthology.orchestra.run.smithi100.stdout:(133/156): python3-natsort-7.1.1-5.el9.noarch.r 492 kB/s | 58 kB 00:00 2024-05-10T11:03:18.205 INFO:teuthology.orchestra.run.smithi188.stdout:(59/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 469 kB/s | 160 kB 00:00 2024-05-10T11:03:18.229 INFO:teuthology.orchestra.run.smithi100.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 1.9 MB/s | 272 kB 00:00 2024-05-10T11:03:18.248 INFO:teuthology.orchestra.run.smithi188.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 3.7 MB/s | 662 kB 00:00 2024-05-10T11:03:18.254 INFO:teuthology.orchestra.run.smithi100.stdout:(135/156): python3-portend-3.1.0-2.el9.noarch.r 140 kB/s | 16 kB 00:00 2024-05-10T11:03:18.279 INFO:teuthology.orchestra.run.smithi100.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 767 kB/s | 90 kB 00:00 2024-05-10T11:03:18.306 INFO:teuthology.orchestra.run.smithi188.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 251 kB/s | 25 kB 00:00 2024-05-10T11:03:18.329 INFO:teuthology.orchestra.run.smithi100.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 280 kB/s | 28 kB 00:00 2024-05-10T11:03:18.348 INFO:teuthology.orchestra.run.smithi188.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 473 kB/s | 47 kB 00:00 2024-05-10T11:03:18.363 INFO:teuthology.orchestra.run.smithi100.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 282 kB/s | 31 kB 00:00 2024-05-10T11:03:18.389 INFO:teuthology.orchestra.run.smithi100.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-05-10T11:03:18.448 INFO:teuthology.orchestra.run.smithi100.stdout:(140/156): python3-rsa-4.9-2.el9.noarch.rpm 503 kB/s | 59 kB 00:00 2024-05-10T11:03:18.457 INFO:teuthology.orchestra.run.smithi188.stdout:(63/156): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-05-10T11:03:18.473 INFO:teuthology.orchestra.run.smithi100.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 328 kB/s | 36 kB 00:00 2024-05-10T11:03:18.498 INFO:teuthology.orchestra.run.smithi100.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 467 kB/s | 51 kB 00:00 2024-05-10T11:03:18.557 INFO:teuthology.orchestra.run.smithi100.stdout:(143/156): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-05-10T11:03:18.566 INFO:teuthology.orchestra.run.smithi188.stdout:(64/156): openblas-0.3.21-2.el9.x86_64.rpm 325 kB/s | 35 kB 00:00 2024-05-10T11:03:18.591 INFO:teuthology.orchestra.run.smithi100.stdout:(144/156): python3-websocket-client-1.2.3-2.el9 763 kB/s | 90 kB 00:00 2024-05-10T11:03:18.591 INFO:teuthology.orchestra.run.smithi188.stdout:(65/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.0 MB/s | 292 kB 00:00 2024-05-10T11:03:18.624 INFO:teuthology.orchestra.run.smithi100.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-05-10T11:03:18.649 INFO:teuthology.orchestra.run.smithi100.stdout:(146/156): python3-zc-lockfile-2.0-10.el9.noarc 215 kB/s | 20 kB 00:00 2024-05-10T11:03:18.692 INFO:teuthology.orchestra.run.smithi100.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-05-10T11:03:18.767 INFO:teuthology.orchestra.run.smithi100.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-05-10T11:03:18.950 INFO:teuthology.orchestra.run.smithi100.stdout:(149/156): NetworkManager-1.47.5-1.el9.x86_64.r 13 MB/s | 2.3 MB 00:00 2024-05-10T11:03:19.009 INFO:teuthology.orchestra.run.smithi100.stdout:(150/156): librados2-19.0.0-3449.gd39fa088.el9. 9.5 MB/s | 3.4 MB 00:00 2024-05-10T11:03:19.033 INFO:teuthology.orchestra.run.smithi188.stdout:(66/156): libtool-2.4.6-45.el9.x86_64.rpm 646 kB/s | 582 kB 00:00 2024-05-10T11:03:19.051 INFO:teuthology.orchestra.run.smithi100.stdout:(151/156): NetworkManager-libnm-1.47.5-1.el9.x8 18 MB/s | 1.8 MB 00:00 2024-05-10T11:03:19.085 INFO:teuthology.orchestra.run.smithi100.stdout:(152/156): NetworkManager-team-1.47.5-1.el9.x86 553 kB/s | 42 kB 00:00 2024-05-10T11:03:19.108 INFO:teuthology.orchestra.run.smithi188.stdout:(67/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 8.6 MB/s | 4.4 MB 00:00 2024-05-10T11:03:19.109 INFO:teuthology.orchestra.run.smithi100.stdout:(153/156): NetworkManager-tui-1.47.5-1.el9.x86_ 4.1 MB/s | 248 kB 00:00 2024-05-10T11:03:19.177 INFO:teuthology.orchestra.run.smithi100.stdout:(154/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 36 MB/s | 2.4 MB 00:00 2024-05-10T11:03:19.252 INFO:teuthology.orchestra.run.smithi100.stdout:(155/156): openssl-3.2.1-1.el9.x86_64.rpm 8.1 MB/s | 1.3 MB 00:00 2024-05-10T11:03:19.650 INFO:teuthology.orchestra.run.smithi188.stdout:(68/156): openblas-openmp-0.3.21-2.el9.x86_64.r 4.4 MB/s | 4.7 MB 00:01 2024-05-10T11:03:19.652 INFO:teuthology.orchestra.run.smithi100.stdout:(156/156): librbd1-19.0.0-3449.gd39fa088.el9.x8 3.3 MB/s | 3.1 MB 00:00 2024-05-10T11:03:19.657 INFO:teuthology.orchestra.run.smithi100.stdout:-------------------------------------------------------------------------------- 2024-05-10T11:03:19.657 INFO:teuthology.orchestra.run.smithi100.stdout:Total 13 MB/s | 226 MB 00:17 2024-05-10T11:03:19.917 INFO:teuthology.orchestra.run.smithi188.stdout:(69/156): postgresql-private-libs-13.14-1.el9.x 500 kB/s | 133 kB 00:00 2024-05-10T11:03:20.342 INFO:teuthology.orchestra.run.smithi188.stdout:(70/156): postgresql-contrib-13.14-1.el9.x86_64 738 kB/s | 911 kB 00:01 2024-05-10T11:03:20.864 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction check 2024-05-10T11:03:21.017 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction check succeeded. 2024-05-10T11:03:21.017 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction test 2024-05-10T11:03:21.176 INFO:teuthology.orchestra.run.smithi188.stdout:(71/156): postgresql-13.14-1.el9.x86_64.rpm 759 kB/s | 1.6 MB 00:02 2024-05-10T11:03:21.359 INFO:teuthology.orchestra.run.smithi188.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 477 kB/s | 87 kB 00:00 2024-05-10T11:03:21.776 INFO:teuthology.orchestra.run.smithi188.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 721 kB/s | 1.0 MB 00:01 2024-05-10T11:03:22.127 INFO:teuthology.orchestra.run.smithi188.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 698 kB/s | 244 kB 00:00 2024-05-10T11:03:22.469 INFO:teuthology.orchestra.run.smithi188.stdout:(75/156): python3-jinja2-2.11.3-5.el9.noarch.rp 725 kB/s | 248 kB 00:00 2024-05-10T11:03:22.560 INFO:teuthology.orchestra.run.smithi188.stdout:(76/156): python3-jsonpatch-1.21-16.el9.noarch. 280 kB/s | 26 kB 00:00 2024-05-10T11:03:22.652 INFO:teuthology.orchestra.run.smithi188.stdout:(77/156): python3-jsonpointer-2.0-4.el9.noarch. 209 kB/s | 19 kB 00:00 2024-05-10T11:03:22.916 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-05-10T11:03:22.928 INFO:teuthology.orchestra.run.smithi188.stdout:(78/156): python3-mako-1.1.4-6.el9.noarch.rpm 626 kB/s | 172 kB 00:00 2024-05-10T11:03:22.996 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-10T11:03:23.028 INFO:teuthology.orchestra.run.smithi188.stdout:(79/156): python3-markupsafe-1.1.1-12.el9.x86_6 348 kB/s | 35 kB 00:00 2024-05-10T11:03:23.033 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-10T11:03:23.088 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-10T11:03:23.121 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-10T11:03:23.147 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-10T11:03:23.165 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction test succeeded. 2024-05-10T11:03:23.166 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction 2024-05-10T11:03:23.199 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-10T11:03:23.527 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-10T11:03:23.648 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : librados2-2:19.0.0-3449.gd39fa088.el9.x86_64 8/164 2024-05-10T11:03:23.778 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librados2-2:19.0.0-3449.gd39fa088.el9.x86_64 8/164 2024-05-10T11:03:23.815 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libcephfs2-2:19.0.0-3449.gd39fa088.el9.x86_64 9/164 2024-05-10T11:03:23.898 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libcephfs2-2:19.0.0-3449.gd39fa088.el9.x86_64 9/164 2024-05-10T11:03:23.928 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-rados-2:19.0.0-3449.gd39fa088.el9.x86_64 10/164 2024-05-10T11:03:24.106 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-10T11:03:24.170 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-10T11:03:24.195 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libcephsqlite-2:19.0.0-3449.gd39fa088.el9.x86_64 13/164 2024-05-10T11:03:24.299 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3449.gd39fa088.el9.x86_64 13/164 2024-05-10T11:03:24.329 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libradosstriper1-2:19.0.0-3449.gd39fa088.el9.x86 14/164 2024-05-10T11:03:24.392 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3449.gd39fa088.el9.x86 14/164 2024-05-10T11:03:24.421 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-10T11:03:24.445 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-10T11:03:24.639 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-10T11:03:24.818 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-10T11:03:24.960 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-10T11:03:25.004 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-10T11:03:25.124 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-10T11:03:25.206 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-10T11:03:25.420 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-10T11:03:25.594 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-10T11:03:25.739 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-10T11:03:25.778 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-05-10T11:03:25.812 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-05-10T11:03:25.877 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-05-10T11:03:25.912 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-05-10T11:03:25.955 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/164 2024-05-10T11:03:25.988 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/164 2024-05-10T11:03:26.016 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/164 2024-05-10T11:03:26.085 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/164 2024-05-10T11:03:26.138 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/164 2024-05-10T11:03:26.177 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/164 2024-05-10T11:03:26.212 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/164 2024-05-10T11:03:26.290 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/164 2024-05-10T11:03:26.322 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-ceph-common-2:19.0.0-3449.gd39fa088.el9. 33/164 2024-05-10T11:03:26.366 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-ceph-argparse-2:19.0.0-3449.gd39fa088.el 34/164 2024-05-10T11:03:26.395 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cephfs-2:19.0.0-3449.gd39fa088.el9.x86_6 35/164 2024-05-10T11:03:26.465 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/164 2024-05-10T11:03:26.509 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : luarocks-3.9.2-1.el9.noarch 37/164 2024-05-10T11:03:26.583 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/164 2024-05-10T11:03:26.714 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/164 2024-05-10T11:03:26.770 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/164 2024-05-10T11:03:27.286 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/164 2024-05-10T11:03:27.366 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/164 2024-05-10T11:03:27.403 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 43/164 2024-05-10T11:03:27.438 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/164 2024-05-10T11:03:27.475 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : librados-devel-2:19.0.0-3449.gd39fa088.el9.x86_6 45/164 2024-05-10T11:03:27.517 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 46/164 2024-05-10T11:03:27.670 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 47/164 2024-05-10T11:03:27.694 INFO:teuthology.orchestra.run.smithi188.stdout:(80/156): postgresql-server-13.14-1.el9.x86_64. 761 kB/s | 5.8 MB 00:07 2024-05-10T11:03:27.720 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 48/164 2024-05-10T11:03:27.775 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 49/164 2024-05-10T11:03:27.847 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-10T11:03:27.879 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-10T11:03:27.915 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-10T11:03:27.962 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-10T11:03:27.998 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-10T11:03:28.041 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-10T11:03:28.053 INFO:teuthology.orchestra.run.smithi188.stdout:(81/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 699 kB/s | 250 kB 00:00 2024-05-10T11:03:28.083 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-10T11:03:28.217 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-10T11:03:28.390 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-10T11:03:28.412 INFO:teuthology.orchestra.run.smithi188.stdout:(82/156): python3-oauthlib-3.1.1-5.el9.noarch.r 620 kB/s | 222 kB 00:00 2024-05-10T11:03:28.419 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-10T11:03:28.461 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-10T11:03:28.557 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-10T11:03:28.598 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-10T11:03:28.612 INFO:teuthology.orchestra.run.smithi188.stdout:(83/156): python3-packaging-20.9-5.el9.noarch.r 386 kB/s | 77 kB 00:00 2024-05-10T11:03:28.635 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-10T11:03:28.668 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-10T11:03:28.693 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-10T11:03:28.720 INFO:teuthology.orchestra.run.smithi188.stdout:(84/156): python3-prettytable-0.7.2-27.el9.noar 390 kB/s | 42 kB 00:00 2024-05-10T11:03:28.778 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-10T11:03:28.902 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-10T11:03:28.950 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-10T11:03:28.979 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-10T11:03:29.013 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-10T11:03:29.051 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-10T11:03:29.083 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-10T11:03:29.113 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-10T11:03:29.146 INFO:teuthology.orchestra.run.smithi188.stdout:(85/156): python3-protobuf-3.14.0-13.el9.noarch 633 kB/s | 269 kB 00:00 2024-05-10T11:03:29.147 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-10T11:03:29.181 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-10T11:03:29.222 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-10T11:03:29.257 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-10T11:03:29.296 INFO:teuthology.orchestra.run.smithi188.stdout:(86/156): python3-babel-2.9.1-2.el9.noarch.rpm 769 kB/s | 6.0 MB 00:07 2024-05-10T11:03:29.342 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-10T11:03:29.387 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-10T11:03:29.409 INFO:teuthology.orchestra.run.smithi100.stdout: Preparing : 1/1 2024-05-10T11:03:29.421 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-10T11:03:29.455 INFO:teuthology.orchestra.run.smithi188.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 514 kB/s | 159 kB 00:00 2024-05-10T11:03:29.465 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-10T11:03:29.490 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-10T11:03:29.528 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-10T11:03:29.554 INFO:teuthology.orchestra.run.smithi188.stdout:(88/156): python3-pytz-2021.1-5.el9.noarch.rpm 511 kB/s | 51 kB 00:00 2024-05-10T11:03:29.562 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-10T11:03:29.595 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-10T11:03:29.619 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-10T11:03:29.641 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-10T11:03:29.655 INFO:teuthology.orchestra.run.smithi188.stdout:(89/156): python3-requests-oauthlib-1.3.0-12.el 536 kB/s | 54 kB 00:00 2024-05-10T11:03:29.666 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-10T11:03:29.712 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-10T11:03:29.714 INFO:teuthology.orchestra.run.smithi188.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 668 kB/s | 279 kB 00:00 2024-05-10T11:03:29.822 INFO:teuthology.orchestra.run.smithi188.stdout:(91/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 768 kB/s | 5.1 MB 00:06 2024-05-10T11:03:29.839 INFO:teuthology.orchestra.run.smithi188.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 332 kB/s | 42 kB 00:00 2024-05-10T11:03:29.923 INFO:teuthology.orchestra.run.smithi188.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 440 kB/s | 44 kB 00:00 2024-05-10T11:03:30.015 INFO:teuthology.orchestra.run.smithi188.stdout:(94/156): qatlib-service-24.02.0-1.el9.x86_64.r 392 kB/s | 36 kB 00:00 2024-05-10T11:03:30.044 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-10T11:03:30.107 INFO:teuthology.orchestra.run.smithi188.stdout:(95/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 517 kB/s | 47 kB 00:00 2024-05-10T11:03:30.159 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : librados2-2:19.0.0-3449.gd39fa088.el9.x86_64 8/164 2024-05-10T11:03:30.182 INFO:teuthology.orchestra.run.smithi188.stdout:(96/156): qatlib-24.02.0-1.el9.x86_64.rpm 644 kB/s | 221 kB 00:00 2024-05-10T11:03:30.274 INFO:teuthology.orchestra.run.smithi188.stdout:(97/156): uuid-1.6.2-55.el9.x86_64.rpm 632 kB/s | 58 kB 00:00 2024-05-10T11:03:30.280 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librados2-2:19.0.0-3449.gd39fa088.el9.x86_64 8/164 2024-05-10T11:03:30.310 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libcephfs2-2:19.0.0-3449.gd39fa088.el9.x86_64 9/164 2024-05-10T11:03:30.333 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-10T11:03:30.383 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-10T11:03:30.385 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libcephfs2-2:19.0.0-3449.gd39fa088.el9.x86_64 9/164 2024-05-10T11:03:30.414 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rados-2:19.0.0-3449.gd39fa088.el9.x86_64 10/164 2024-05-10T11:03:30.417 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-10T11:03:30.458 INFO:teuthology.orchestra.run.smithi188.stdout:(98/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 511 kB/s | 94 kB 00:00 2024-05-10T11:03:30.514 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-10T11:03:30.541 INFO:teuthology.orchestra.run.smithi188.stdout:(99/156): socat-1.7.4.1-5.el9.x86_64.rpm 702 kB/s | 305 kB 00:00 2024-05-10T11:03:30.566 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-10T11:03:30.581 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-10T11:03:30.641 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-10T11:03:30.642 INFO:teuthology.orchestra.run.smithi188.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 347 kB/s | 64 kB 00:00 2024-05-10T11:03:30.653 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-10T11:03:30.664 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libcephsqlite-2:19.0.0-3449.gd39fa088.el9.x86_64 13/164 2024-05-10T11:03:30.667 INFO:teuthology.orchestra.run.smithi188.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 358 kB/s | 45 kB 00:00 2024-05-10T11:03:30.686 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-10T11:03:30.742 INFO:teuthology.orchestra.run.smithi188.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 222 kB/s | 22 kB 00:00 2024-05-10T11:03:30.752 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3449.gd39fa088.el9.x86_64 13/164 2024-05-10T11:03:30.774 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libradosstriper1-2:19.0.0-3449.gd39fa088.el9.x86 14/164 2024-05-10T11:03:30.837 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3449.gd39fa088.el9.x86 14/164 2024-05-10T11:03:30.854 INFO:teuthology.orchestra.run.smithi188.stdout:(103/156): protobuf-compiler-3.14.0-13.el9.x86_ 4.6 MB/s | 863 kB 00:00 2024-05-10T11:03:30.862 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-10T11:03:30.866 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-10T11:03:30.886 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-10T11:03:30.893 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-10T11:03:30.978 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-10T11:03:31.018 INFO:teuthology.orchestra.run.smithi188.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 1.9 MB/s | 548 kB 00:00 2024-05-10T11:03:31.068 INFO:teuthology.orchestra.run.smithi188.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 392 kB/s | 19 kB 00:00 2024-05-10T11:03:31.076 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-10T11:03:31.093 INFO:teuthology.orchestra.run.smithi188.stdout:(106/156): gperftools-libs-2.9.1-2.el9.x86_64.r 1.2 MB/s | 309 kB 00:00 2024-05-10T11:03:31.152 INFO:teuthology.orchestra.run.smithi188.stdout:(107/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 445 kB/s | 26 kB 00:00 2024-05-10T11:03:31.211 INFO:teuthology.orchestra.run.smithi188.stdout:(108/156): liboath-2.6.7-2.el9.x86_64.rpm 835 kB/s | 49 kB 00:00 2024-05-10T11:03:31.246 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-10T11:03:31.303 INFO:teuthology.orchestra.run.smithi188.stdout:(109/156): libarrow-9.0.0-11.el9.x86_64.rpm 19 MB/s | 4.4 MB 00:00 2024-05-10T11:03:31.328 INFO:teuthology.orchestra.run.smithi188.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 575 kB/s | 67 kB 00:00 2024-05-10T11:03:31.362 INFO:teuthology.orchestra.run.smithi188.stdout:(111/156): luarocks-3.9.2-1.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-05-10T11:03:31.420 INFO:teuthology.orchestra.run.smithi188.stdout:(112/156): python3-asyncssh-2.13.2-5.el9.noarch 9.2 MB/s | 548 kB 00:00 2024-05-10T11:03:31.445 INFO:teuthology.orchestra.run.smithi188.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.0 MB/s | 839 kB 00:00 2024-05-10T11:03:31.447 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-10T11:03:31.471 INFO:teuthology.orchestra.run.smithi188.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 863 kB/s | 43 kB 00:00 2024-05-10T11:03:31.505 INFO:teuthology.orchestra.run.smithi188.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 548 kB/s | 32 kB 00:00 2024-05-10T11:03:31.507 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-10T11:03:31.530 INFO:teuthology.orchestra.run.smithi188.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 256 kB/s | 15 kB 00:00 2024-05-10T11:03:31.555 INFO:teuthology.orchestra.run.smithi188.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-05-10T11:03:31.597 INFO:teuthology.orchestra.run.smithi188.stdout:(118/156): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-05-10T11:03:31.617 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-10T11:03:31.622 INFO:teuthology.orchestra.run.smithi188.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-05-10T11:03:31.631 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-10T11:03:31.648 INFO:teuthology.orchestra.run.smithi188.stdout:(120/156): python3-colorama-0.4.4-12.el9.noarch 701 kB/s | 35 kB 00:00 2024-05-10T11:03:31.681 INFO:teuthology.orchestra.run.smithi188.stdout:(121/156): python3-google-auth-2.29.0-1.el9.noa 3.4 MB/s | 206 kB 00:00 2024-05-10T11:03:31.692 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-10T11:03:31.739 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-10T11:03:31.740 INFO:teuthology.orchestra.run.smithi188.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-05-10T11:03:31.790 INFO:teuthology.orchestra.run.smithi188.stdout:(123/156): python3-grpcio-1.46.7-10.el9.x86_64. 14 MB/s | 2.0 MB 00:00 2024-05-10T11:03:31.816 INFO:teuthology.orchestra.run.smithi188.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 142 kB/s | 11 kB 00:00 2024-05-10T11:03:31.849 INFO:teuthology.orchestra.run.smithi188.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 301 kB/s | 18 kB 00:00 2024-05-10T11:03:31.875 INFO:teuthology.orchestra.run.smithi188.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 395 kB/s | 23 kB 00:00 2024-05-10T11:03:31.894 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-10T11:03:31.900 INFO:teuthology.orchestra.run.smithi188.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-05-10T11:03:31.925 INFO:teuthology.orchestra.run.smithi188.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 390 kB/s | 20 kB 00:00 2024-05-10T11:03:31.999 INFO:teuthology.orchestra.run.smithi188.stdout:(129/156): python3-kubernetes-26.1.0-2.el9.noar 10 MB/s | 1.0 MB 00:00 2024-05-10T11:03:32.001 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-10T11:03:32.026 INFO:teuthology.orchestra.run.smithi188.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 462 kB/s | 46 kB 00:00 2024-05-10T11:03:32.051 INFO:teuthology.orchestra.run.smithi188.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-05-10T11:03:32.078 INFO:teuthology.orchestra.run.smithi188.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-05-10T11:03:32.110 INFO:teuthology.orchestra.run.smithi188.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 4.5 MB/s | 272 kB 00:00 2024-05-10T11:03:32.112 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-10T11:03:32.135 INFO:teuthology.orchestra.run.smithi188.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 280 kB/s | 16 kB 00:00 2024-05-10T11:03:32.148 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-05-10T11:03:32.169 INFO:teuthology.orchestra.run.smithi188.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-05-10T11:03:32.183 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-05-10T11:03:32.194 INFO:teuthology.orchestra.run.smithi188.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 482 kB/s | 28 kB 00:00 2024-05-10T11:03:32.211 INFO:teuthology.orchestra.run.smithi188.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 731 kB/s | 31 kB 00:00 2024-05-10T11:03:32.247 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-05-10T11:03:32.253 INFO:teuthology.orchestra.run.smithi188.stdout:(138/156): python3-routes-2.5.1-5.el9.noarch.rp 3.1 MB/s | 188 kB 00:00 2024-05-10T11:03:32.270 INFO:teuthology.orchestra.run.smithi188.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-05-10T11:03:32.282 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-05-10T11:03:32.295 INFO:teuthology.orchestra.run.smithi188.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 855 kB/s | 36 kB 00:00 2024-05-10T11:03:32.312 INFO:teuthology.orchestra.run.smithi188.stdout:(141/156): python3-typing-extensions-4.4.0-2.el 1.2 MB/s | 51 kB 00:00 2024-05-10T11:03:32.326 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/164 2024-05-10T11:03:32.354 INFO:teuthology.orchestra.run.smithi188.stdout:(142/156): python3-webob-1.8.7-6.el9.noarch.rpm 3.8 MB/s | 230 kB 00:00 2024-05-10T11:03:32.358 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/164 2024-05-10T11:03:32.396 INFO:teuthology.orchestra.run.smithi188.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.0 MB/s | 90 kB 00:00 2024-05-10T11:03:32.416 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/164 2024-05-10T11:03:32.422 INFO:teuthology.orchestra.run.smithi188.stdout:(144/156): python3-werkzeug-2.0.3-3.el9.1.noarc 6.2 MB/s | 427 kB 00:00 2024-05-10T11:03:32.455 INFO:teuthology.orchestra.run.smithi188.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 341 kB/s | 20 kB 00:00 2024-05-10T11:03:32.490 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/164 2024-05-10T11:03:32.490 INFO:teuthology.orchestra.run.smithi188.stdout:(146/156): re2-20211101-3.el9.x86_64.rpm 2.8 MB/s | 192 kB 00:00 2024-05-10T11:03:32.541 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/164 2024-05-10T11:03:32.556 INFO:teuthology.orchestra.run.smithi188.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-05-10T11:03:32.573 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/164 2024-05-10T11:03:32.616 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/164 2024-05-10T11:03:32.692 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/164 2024-05-10T11:03:32.727 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-ceph-common-2:19.0.0-3449.gd39fa088.el9. 33/164 2024-05-10T11:03:32.771 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-ceph-argparse-2:19.0.0-3449.gd39fa088.el 34/164 2024-05-10T11:03:32.799 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cephfs-2:19.0.0-3449.gd39fa088.el9.x86_6 35/164 2024-05-10T11:03:32.868 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/164 2024-05-10T11:03:32.922 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : luarocks-3.9.2-1.el9.noarch 37/164 2024-05-10T11:03:32.948 INFO:teuthology.orchestra.run.smithi188.stdout:(148/156): librados2-19.0.0-3449.gd39fa088.el9. 7.4 MB/s | 3.4 MB 00:00 2024-05-10T11:03:32.987 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-10T11:03:32.997 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/164 2024-05-10T11:03:33.062 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-10T11:03:33.090 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-10T11:03:33.123 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-10T11:03:33.130 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/164 2024-05-10T11:03:33.183 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/164 2024-05-10T11:03:33.257 INFO:teuthology.orchestra.run.smithi188.stdout:(149/156): librbd1-19.0.0-3449.gd39fa088.el9.x8 4.5 MB/s | 3.1 MB 00:00 2024-05-10T11:03:33.436 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-10T11:03:33.477 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : librbd1-2:19.0.0-3449.gd39fa088.el9.x86_64 101/164 2024-05-10T11:03:33.549 INFO:teuthology.orchestra.run.smithi188.stdout:(150/156): NetworkManager-libnm-1.47.5-1.el9.x8 6.3 MB/s | 1.8 MB 00:00 2024-05-10T11:03:33.568 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librbd1-2:19.0.0-3449.gd39fa088.el9.x86_64 101/164 2024-05-10T11:03:33.590 INFO:teuthology.orchestra.run.smithi188.stdout:(151/156): NetworkManager-team-1.47.5-1.el9.x86 1.0 MB/s | 42 kB 00:00 2024-05-10T11:03:33.620 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-rbd-2:19.0.0-3449.gd39fa088.el9.x86_64 102/164 2024-05-10T11:03:33.649 INFO:teuthology.orchestra.run.smithi188.stdout:(152/156): NetworkManager-tui-1.47.5-1.el9.x86_ 4.2 MB/s | 248 kB 00:00 2024-05-10T11:03:33.660 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-10T11:03:33.699 INFO:teuthology.orchestra.run.smithi188.stdout:(153/156): NetworkManager-1.47.5-1.el9.x86_64.r 3.1 MB/s | 2.3 MB 00:00 2024-05-10T11:03:33.704 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/164 2024-05-10T11:03:33.787 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/164 2024-05-10T11:03:33.799 INFO:teuthology.orchestra.run.smithi188.stdout:(154/156): openssl-3.2.1-1.el9.x86_64.rpm 9.0 MB/s | 1.3 MB 00:00 2024-05-10T11:03:33.816 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 43/164 2024-05-10T11:03:33.860 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/164 2024-05-10T11:03:33.897 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librados-devel-2:19.0.0-3449.gd39fa088.el9.x86_6 45/164 2024-05-10T11:03:33.940 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 46/164 2024-05-10T11:03:34.083 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 47/164 2024-05-10T11:03:34.139 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-10T11:03:34.142 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 48/164 2024-05-10T11:03:34.164 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : librgw2-2:19.0.0-3449.gd39fa088.el9.x86_64 105/164 2024-05-10T11:03:34.196 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 49/164 2024-05-10T11:03:34.235 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librgw2-2:19.0.0-3449.gd39fa088.el9.x86_64 105/164 2024-05-10T11:03:34.257 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-rgw-2:19.0.0-3449.gd39fa088.el9.x86_64 106/164 2024-05-10T11:03:34.269 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-10T11:03:34.309 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-10T11:03:34.346 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-10T11:03:34.383 INFO:teuthology.orchestra.run.smithi188.stdout:(155/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 3.5 MB/s | 2.4 MB 00:00 2024-05-10T11:03:34.392 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-10T11:03:34.429 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-10T11:03:34.472 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-10T11:03:34.514 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-10T11:03:34.650 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-10T11:03:34.703 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-10T11:03:34.733 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-10T11:03:34.775 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-10T11:03:34.870 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-10T11:03:34.901 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-10T11:03:34.942 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-10T11:03:35.099 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-10T11:03:35.276 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-10T11:03:35.720 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-10T11:03:36.003 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-10T11:03:36.049 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-10T11:03:36.084 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-10T11:03:36.120 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-10T11:03:36.161 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-10T11:03:36.198 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-10T11:03:36.228 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-10T11:03:36.262 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-10T11:03:36.304 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-10T11:03:36.347 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-10T11:03:36.380 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-10T11:03:36.462 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-10T11:03:36.520 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-10T11:03:36.522 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-common-2:19.0.0-3449.gd39fa088.el9.x86_64 107/164 2024-05-10T11:03:36.550 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-10T11:03:36.553 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-common-2:19.0.0-3449.gd39fa088.el9.x86_64 107/164 2024-05-10T11:03:36.598 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-10T11:03:36.704 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-10T11:03:36.751 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-10T11:03:37.107 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-common-2:19.0.0-3449.gd39fa088.el9.x86_64 107/164 2024-05-10T11:03:37.157 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-base-2:19.0.0-3449.gd39fa088.el9.x86_64 108/164 2024-05-10T11:03:37.265 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-base-2:19.0.0-3449.gd39fa088.el9.x86_64 108/164 2024-05-10T11:03:37.265 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-10T11:03:37.265 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-10T11:03:37.265 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-05-10T11:03:37.300 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-selinux-2:19.0.0-3449.gd39fa088.el9.x86_64 109/164 2024-05-10T11:03:37.471 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-10T11:03:37.524 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-10T11:03:37.558 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-10T11:03:37.657 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-10T11:03:37.707 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-10T11:03:37.794 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-10T11:03:37.835 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-10T11:03:38.011 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-10T11:03:38.041 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-10T11:03:38.119 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-10T11:03:38.786 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-10T11:03:38.906 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-10T11:03:40.165 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-10T11:03:40.254 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-10T11:03:40.282 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-10T11:03:40.314 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-10T11:03:40.635 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-10T11:03:40.678 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : librbd1-2:19.0.0-3449.gd39fa088.el9.x86_64 101/164 2024-05-10T11:03:40.768 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librbd1-2:19.0.0-3449.gd39fa088.el9.x86_64 101/164 2024-05-10T11:03:40.799 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rbd-2:19.0.0-3449.gd39fa088.el9.x86_64 102/164 2024-05-10T11:03:40.840 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-10T11:03:41.314 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-10T11:03:41.356 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librgw2-2:19.0.0-3449.gd39fa088.el9.x86_64 105/164 2024-05-10T11:03:41.419 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librgw2-2:19.0.0-3449.gd39fa088.el9.x86_64 105/164 2024-05-10T11:03:41.449 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rgw-2:19.0.0-3449.gd39fa088.el9.x86_64 106/164 2024-05-10T11:03:43.758 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-common-2:19.0.0-3449.gd39fa088.el9.x86_64 107/164 2024-05-10T11:03:43.804 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-common-2:19.0.0-3449.gd39fa088.el9.x86_64 107/164 2024-05-10T11:03:44.350 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-common-2:19.0.0-3449.gd39fa088.el9.x86_64 107/164 2024-05-10T11:03:44.391 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-base-2:19.0.0-3449.gd39fa088.el9.x86_64 108/164 2024-05-10T11:03:44.486 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-base-2:19.0.0-3449.gd39fa088.el9.x86_64 108/164 2024-05-10T11:03:44.486 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-10T11:03:44.487 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-10T11:03:44.487 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:03:44.525 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-selinux-2:19.0.0-3449.gd39fa088.el9.x86_64 109/164 2024-05-10T11:03:48.813 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3449.gd39fa088.el9.x86_64 109/164 2024-05-10T11:03:48.813 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /sys 2024-05-10T11:03:48.813 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /proc 2024-05-10T11:03:48.814 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /mnt 2024-05-10T11:03:48.814 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /var/tmp 2024-05-10T11:03:48.814 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /home 2024-05-10T11:03:48.814 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /root 2024-05-10T11:03:48.814 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /tmp 2024-05-10T11:03:48.814 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-05-10T11:03:48.991 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-volume-2:19.0.0-3449.gd39fa088.el9.noarch 110/164 2024-05-10T11:03:49.008 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-volume-2:19.0.0-3449.gd39fa088.el9.noarch 110/164 2024-05-10T11:03:49.009 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T11:03:49.009 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-10T11:03:49.009 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-05-10T11:03:50.519 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-osd-2:19.0.0-3449.gd39fa088.el9.x86_64 111/164 2024-05-10T11:03:50.547 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-osd-2:19.0.0-3449.gd39fa088.el9.x86_64 111/164 2024-05-10T11:03:50.547 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T11:03:50.547 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-10T11:03:50.547 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-10T11:03:50.547 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-10T11:03:50.547 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-05-10T11:03:50.770 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mds-2:19.0.0-3449.gd39fa088.el9.x86_64 112/164 2024-05-10T11:03:50.794 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mds-2:19.0.0-3449.gd39fa088.el9.x86_64 112/164 2024-05-10T11:03:50.794 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T11:03:50.794 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-10T11:03:50.794 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-10T11:03:50.794 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-10T11:03:50.794 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-05-10T11:03:51.238 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mon-2:19.0.0-3449.gd39fa088.el9.x86_64 113/164 2024-05-10T11:03:51.261 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mon-2:19.0.0-3449.gd39fa088.el9.x86_64 113/164 2024-05-10T11:03:51.261 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T11:03:51.261 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-10T11:03:51.262 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-10T11:03:51.262 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-10T11:03:51.262 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-05-10T11:03:51.360 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-10T11:03:51.416 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-10T11:03:51.471 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-10T11:03:51.513 INFO:teuthology.orchestra.run.smithi188.stdout:(156/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 761 kB/s | 16 MB 00:21 2024-05-10T11:03:51.516 INFO:teuthology.orchestra.run.smithi188.stdout:-------------------------------------------------------------------------------- 2024-05-10T11:03:51.517 INFO:teuthology.orchestra.run.smithi188.stdout:Total 4.8 MB/s | 226 MB 00:47 2024-05-10T11:03:51.656 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-10T11:03:51.697 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-10T11:03:51.775 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-10T11:03:51.871 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-10T11:03:52.019 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-10T11:03:52.065 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-10T11:03:52.097 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-10T11:03:52.124 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-10T11:03:52.545 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: cephadm-2:19.0.0-3449.gd39fa088.el9.noarch 125/164 2024-05-10T11:03:52.573 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : cephadm-2:19.0.0-3449.gd39fa088.el9.noarch 125/164 2024-05-10T11:03:52.605 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3449.gd39fa088.e 126/164 2024-05-10T11:03:52.666 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3449.gd39fa088. 127/164 2024-05-10T11:03:52.696 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3449.gd39fa088.el9.noa 128/164 2024-05-10T11:03:52.711 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction check 2024-05-10T11:03:52.858 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction check succeeded. 2024-05-10T11:03:52.859 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction test 2024-05-10T11:03:53.735 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3449.gd39fa088.el9.noa 128/164 2024-05-10T11:03:53.789 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3449.gd39fa088.el9.n 129/164 2024-05-10T11:03:54.754 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3449.gd39fa088.el9.n 129/164 2024-05-10T11:03:54.782 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3449.gd39 130/164 2024-05-10T11:03:54.893 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3449.gd39 130/164 2024-05-10T11:03:55.053 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3449.gd39fa088.el 131/164 2024-05-10T11:03:55.084 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-2:19.0.0-3449.gd39fa088.el9.x86_64 132/164 2024-05-10T11:03:55.108 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3449.gd39fa088.el9.x86_64 132/164 2024-05-10T11:03:55.109 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T11:03:55.109 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-10T11:03:55.109 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-10T11:03:55.109 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-10T11:03:55.109 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-05-10T11:03:55.134 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction test succeeded. 2024-05-10T11:03:55.135 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction 2024-05-10T11:03:55.159 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-rook-2:19.0.0-3449.gd39fa088.el9.noarch 133/164 2024-05-10T11:03:55.171 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3449.gd39fa088.el9.noarch 133/164 2024-05-10T11:03:55.278 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-2:19.0.0-3449.gd39fa088.el9.x86_64 134/164 2024-05-10T11:03:55.343 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-fuse-2:19.0.0-3449.gd39fa088.el9.x86_64 135/164 2024-05-10T11:03:56.278 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-10T11:03:56.304 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-radosgw-2:19.0.0-3449.gd39fa088.el9.x86_64 137/164 2024-05-10T11:03:56.328 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3449.gd39fa088.el9.x86_64 137/164 2024-05-10T11:03:56.329 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T11:03:56.329 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-10T11:03:56.329 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-10T11:03:56.329 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-10T11:03:56.329 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-05-10T11:03:56.380 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3449.gd39fa 138/164 2024-05-10T11:03:56.403 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3449.gd39fa 138/164 2024-05-10T11:03:56.403 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T11:03:56.403 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-10T11:03:56.403 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-05-10T11:03:56.430 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3449.gd39fa088.el9.x86_64 109/164 2024-05-10T11:03:56.431 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /sys 2024-05-10T11:03:56.431 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /proc 2024-05-10T11:03:56.431 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /mnt 2024-05-10T11:03:56.431 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /var/tmp 2024-05-10T11:03:56.431 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /home 2024-05-10T11:03:56.431 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /root 2024-05-10T11:03:56.431 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /tmp 2024-05-10T11:03:56.431 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:03:56.455 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : cephfs-mirror-2:19.0.0-3449.gd39fa088.el9.x86_64 139/164 2024-05-10T11:03:56.482 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3449.gd39fa088.el9.x86_64 139/164 2024-05-10T11:03:56.482 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T11:03:56.482 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-10T11:03:56.483 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-10T11:03:56.483 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-10T11:03:56.483 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-05-10T11:03:56.618 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-volume-2:19.0.0-3449.gd39fa088.el9.noarch 110/164 2024-05-10T11:03:56.634 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-volume-2:19.0.0-3449.gd39fa088.el9.noarch 110/164 2024-05-10T11:03:56.635 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T11:03:56.635 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-10T11:03:56.635 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:03:56.782 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : rbd-mirror-2:19.0.0-3449.gd39fa088.el9.x86_64 140/164 2024-05-10T11:03:56.805 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3449.gd39fa088.el9.x86_64 140/164 2024-05-10T11:03:56.805 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T11:03:56.805 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-10T11:03:56.805 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-10T11:03:56.806 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-10T11:03:56.806 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-05-10T11:03:58.112 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-osd-2:19.0.0-3449.gd39fa088.el9.x86_64 111/164 2024-05-10T11:03:58.140 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-osd-2:19.0.0-3449.gd39fa088.el9.x86_64 111/164 2024-05-10T11:03:58.140 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T11:03:58.140 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-10T11:03:58.140 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-10T11:03:58.140 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-10T11:03:58.140 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:03:58.362 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mds-2:19.0.0-3449.gd39fa088.el9.x86_64 112/164 2024-05-10T11:03:58.386 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mds-2:19.0.0-3449.gd39fa088.el9.x86_64 112/164 2024-05-10T11:03:58.386 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T11:03:58.386 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-10T11:03:58.386 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-10T11:03:58.386 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-10T11:03:58.386 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:03:58.839 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mon-2:19.0.0-3449.gd39fa088.el9.x86_64 113/164 2024-05-10T11:03:58.862 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mon-2:19.0.0-3449.gd39fa088.el9.x86_64 113/164 2024-05-10T11:03:58.862 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T11:03:58.862 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-10T11:03:58.862 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-10T11:03:58.862 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-10T11:03:58.862 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:03:58.968 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-10T11:03:59.025 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-10T11:03:59.081 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-10T11:03:59.274 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-10T11:03:59.330 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-10T11:03:59.408 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-10T11:03:59.497 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-10T11:03:59.636 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-10T11:03:59.699 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-10T11:03:59.730 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-10T11:03:59.757 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-10T11:04:00.160 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: cephadm-2:19.0.0-3449.gd39fa088.el9.noarch 125/164 2024-05-10T11:04:00.222 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : cephadm-2:19.0.0-3449.gd39fa088.el9.noarch 125/164 2024-05-10T11:04:00.254 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3449.gd39fa088.e 126/164 2024-05-10T11:04:00.316 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3449.gd39fa088. 127/164 2024-05-10T11:04:00.345 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3449.gd39fa088.el9.noa 128/164 2024-05-10T11:04:01.177 INFO:teuthology.orchestra.run.smithi188.stdout: Preparing : 1/1 2024-05-10T11:04:01.250 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-10T11:04:01.288 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-10T11:04:01.351 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-10T11:04:01.378 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3449.gd39fa088.el9.noa 128/164 2024-05-10T11:04:01.384 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-10T11:04:01.413 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3449.gd39fa088.el9.n 129/164 2024-05-10T11:04:01.418 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-10T11:04:01.444 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-test-2:19.0.0-3449.gd39fa088.el9.x86_64 141/164 2024-05-10T11:04:01.469 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-10T11:04:01.484 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : rbd-fuse-2:19.0.0-3449.gd39fa088.el9.x86_64 142/164 2024-05-10T11:04:01.525 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : rbd-nbd-2:19.0.0-3449.gd39fa088.el9.x86_64 143/164 2024-05-10T11:04:01.557 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-10T11:04:01.585 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-10T11:04:01.770 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : cephfs-shell-2:19.0.0-3449.gd39fa088.el9.x86_64 146/164 2024-05-10T11:04:01.789 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-10T11:04:01.815 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-10T11:04:01.845 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libcephfs-devel-2:19.0.0-3449.gd39fa088.el9.x86_ 148/164 2024-05-10T11:04:01.874 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-10T11:04:01.902 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : librados2-2:19.0.0-3449.gd39fa088.el9.x86_64 8/164 2024-05-10T11:04:01.921 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-10T11:04:02.024 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librados2-2:19.0.0-3449.gd39fa088.el9.x86_64 8/164 2024-05-10T11:04:02.045 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libcephfs2-2:19.0.0-3449.gd39fa088.el9.x86_64 9/164 2024-05-10T11:04:02.154 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libcephfs2-2:19.0.0-3449.gd39fa088.el9.x86_64 9/164 2024-05-10T11:04:02.184 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-rados-2:19.0.0-3449.gd39fa088.el9.x86_64 10/164 2024-05-10T11:04:02.350 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-10T11:04:02.373 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3449.gd39fa088.el9.n 129/164 2024-05-10T11:04:02.405 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3449.gd39 130/164 2024-05-10T11:04:02.418 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-10T11:04:02.441 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libcephsqlite-2:19.0.0-3449.gd39fa088.el9.x86_64 13/164 2024-05-10T11:04:02.509 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-10T11:04:02.517 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3449.gd39 130/164 2024-05-10T11:04:02.537 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3449.gd39fa088.el9.x86_64 13/164 2024-05-10T11:04:02.552 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-10T11:04:02.559 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libradosstriper1-2:19.0.0-3449.gd39fa088.el9.x86 14/164 2024-05-10T11:04:02.573 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-10T11:04:02.605 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3449.gd39fa088.el9.x86 14/164 2024-05-10T11:04:02.627 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-10T11:04:02.649 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-10T11:04:02.694 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3449.gd39fa088.el 131/164 2024-05-10T11:04:02.724 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-2:19.0.0-3449.gd39fa088.el9.x86_64 132/164 2024-05-10T11:04:02.749 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3449.gd39fa088.el9.x86_64 132/164 2024-05-10T11:04:02.749 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T11:04:02.749 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-10T11:04:02.749 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-10T11:04:02.749 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-10T11:04:02.749 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:04:02.800 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-rook-2:19.0.0-3449.gd39fa088.el9.noarch 133/164 2024-05-10T11:04:02.812 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3449.gd39fa088.el9.noarch 133/164 2024-05-10T11:04:02.870 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-10T11:04:02.909 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-2:19.0.0-3449.gd39fa088.el9.x86_64 134/164 2024-05-10T11:04:02.966 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-fuse-2:19.0.0-3449.gd39fa088.el9.x86_64 135/164 2024-05-10T11:04:03.041 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-10T11:04:03.191 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-10T11:04:03.260 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-10T11:04:03.380 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-10T11:04:03.408 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : cephfs-top-2:19.0.0-3449.gd39fa088.el9.noarch 152/164 2024-05-10T11:04:03.436 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-10T11:04:03.501 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-10T11:04:03.552 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-10T11:04:03.649 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-10T11:04:03.652 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-10T11:04:03.687 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-10T11:04:03.722 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-10T11:04:03.722 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-10T11:04:03.762 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-10T11:04:03.806 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-10T11:04:03.847 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-10T11:04:03.848 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-10T11:04:03.860 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-10T11:04:03.890 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-10T11:04:03.892 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-05-10T11:04:03.919 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-radosgw-2:19.0.0-3449.gd39fa088.el9.x86_64 137/164 2024-05-10T11:04:03.926 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-05-10T11:04:03.940 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-10T11:04:03.942 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3449.gd39fa088.el9.x86_64 137/164 2024-05-10T11:04:03.943 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T11:04:03.943 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-10T11:04:03.943 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-10T11:04:03.943 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-10T11:04:03.943 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:04:03.965 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-10T11:04:03.971 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-10T11:04:03.971 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-10T11:04:03.991 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-05-10T11:04:03.994 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3449.gd39fa 138/164 2024-05-10T11:04:04.018 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3449.gd39fa 138/164 2024-05-10T11:04:04.018 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T11:04:04.018 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-10T11:04:04.018 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:04:04.026 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-05-10T11:04:04.062 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : cephfs-mirror-2:19.0.0-3449.gd39fa088.el9.x86_64 139/164 2024-05-10T11:04:04.069 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/164 2024-05-10T11:04:04.085 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3449.gd39fa088.el9.x86_64 139/164 2024-05-10T11:04:04.086 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T11:04:04.086 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-10T11:04:04.086 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-10T11:04:04.086 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-10T11:04:04.086 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:04:04.103 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/164 2024-05-10T11:04:04.130 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/164 2024-05-10T11:04:04.159 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-10T11:04:04.159 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-10T11:04:04.198 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/164 2024-05-10T11:04:04.244 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-10T11:04:04.252 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/164 2024-05-10T11:04:04.284 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/164 2024-05-10T11:04:04.316 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-10T11:04:04.327 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/164 2024-05-10T11:04:04.389 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : rbd-mirror-2:19.0.0-3449.gd39fa088.el9.x86_64 140/164 2024-05-10T11:04:04.402 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/164 2024-05-10T11:04:04.411 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3449.gd39fa088.el9.x86_64 140/164 2024-05-10T11:04:04.411 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T11:04:04.411 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-10T11:04:04.411 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-10T11:04:04.411 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-10T11:04:04.411 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:04:04.438 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-ceph-common-2:19.0.0-3449.gd39fa088.el9. 33/164 2024-05-10T11:04:04.473 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-ceph-argparse-2:19.0.0-3449.gd39fa088.el 34/164 2024-05-10T11:04:04.501 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cephfs-2:19.0.0-3449.gd39fa088.el9.x86_6 35/164 2024-05-10T11:04:04.570 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/164 2024-05-10T11:04:04.616 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : luarocks-3.9.2-1.el9.noarch 37/164 2024-05-10T11:04:04.691 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/164 2024-05-10T11:04:04.824 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/164 2024-05-10T11:04:04.877 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/164 2024-05-10T11:04:05.399 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/164 2024-05-10T11:04:05.489 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/164 2024-05-10T11:04:05.518 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 43/164 2024-05-10T11:04:05.553 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/164 2024-05-10T11:04:05.582 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : librados-devel-2:19.0.0-3449.gd39fa088.el9.x86_6 45/164 2024-05-10T11:04:05.625 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 46/164 2024-05-10T11:04:05.774 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 47/164 2024-05-10T11:04:05.827 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 48/164 2024-05-10T11:04:05.881 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 49/164 2024-05-10T11:04:05.946 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-10T11:04:05.978 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-10T11:04:06.014 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-10T11:04:06.061 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-10T11:04:06.097 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-10T11:04:06.140 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-10T11:04:06.182 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-10T11:04:06.320 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-10T11:04:06.363 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-10T11:04:06.393 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-10T11:04:06.434 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-10T11:04:06.528 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-10T11:04:06.586 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-10T11:04:06.627 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-10T11:04:06.659 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-10T11:04:06.684 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-10T11:04:06.770 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-10T11:04:06.893 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-10T11:04:06.925 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-10T11:04:06.953 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-10T11:04:06.987 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-10T11:04:07.026 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-10T11:04:07.058 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-10T11:04:07.088 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-10T11:04:07.122 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-10T11:04:07.155 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-10T11:04:07.197 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-10T11:04:07.232 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-10T11:04:07.319 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-10T11:04:07.362 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-10T11:04:07.393 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-10T11:04:07.440 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-10T11:04:07.538 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-10T11:04:07.587 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-10T11:04:08.300 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-10T11:04:08.359 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-10T11:04:08.393 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-10T11:04:08.490 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-10T11:04:08.542 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-10T11:04:08.629 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-10T11:04:08.662 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-10T11:04:08.839 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-10T11:04:08.868 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-10T11:04:08.946 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-10T11:04:09.082 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-test-2:19.0.0-3449.gd39fa088.el9.x86_64 141/164 2024-05-10T11:04:09.124 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : rbd-fuse-2:19.0.0-3449.gd39fa088.el9.x86_64 142/164 2024-05-10T11:04:09.165 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : rbd-nbd-2:19.0.0-3449.gd39fa088.el9.x86_64 143/164 2024-05-10T11:04:09.197 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-10T11:04:09.224 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-10T11:04:09.401 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : cephfs-shell-2:19.0.0-3449.gd39fa088.el9.x86_64 146/164 2024-05-10T11:04:09.445 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-10T11:04:09.484 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libcephfs-devel-2:19.0.0-3449.gd39fa088.el9.x86_ 148/164 2024-05-10T11:04:09.513 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-10T11:04:09.560 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-10T11:04:09.590 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-10T11:04:09.700 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-10T11:04:10.081 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-10T11:04:10.081 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-2:19.0.0-3449.gd39fa088.el9.x86_64 1/164 2024-05-10T11:04:10.081 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-base-2:19.0.0-3449.gd39fa088.el9.x86_64 2/164 2024-05-10T11:04:10.081 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-common-2:19.0.0-3449.gd39fa088.el9.x86_64 3/164 2024-05-10T11:04:10.081 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-fuse-2:19.0.0-3449.gd39fa088.el9.x86_64 4/164 2024-05-10T11:04:10.081 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3449.gd39fa 5/164 2024-05-10T11:04:10.081 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mds-2:19.0.0-3449.gd39fa088.el9.x86_64 6/164 2024-05-10T11:04:10.081 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-2:19.0.0-3449.gd39fa088.el9.x86_64 7/164 2024-05-10T11:04:10.082 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mon-2:19.0.0-3449.gd39fa088.el9.x86_64 8/164 2024-05-10T11:04:10.082 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-osd-2:19.0.0-3449.gd39fa088.el9.x86_64 9/164 2024-05-10T11:04:10.082 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-radosgw-2:19.0.0-3449.gd39fa088.el9.x86_64 10/164 2024-05-10T11:04:10.082 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-selinux-2:19.0.0-3449.gd39fa088.el9.x86_64 11/164 2024-05-10T11:04:10.082 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-test-2:19.0.0-3449.gd39fa088.el9.x86_64 12/164 2024-05-10T11:04:10.082 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : cephfs-mirror-2:19.0.0-3449.gd39fa088.el9.x86_64 13/164 2024-05-10T11:04:10.082 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : cephfs-shell-2:19.0.0-3449.gd39fa088.el9.x86_64 14/164 2024-05-10T11:04:10.082 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libcephfs-devel-2:19.0.0-3449.gd39fa088.el9.x86_ 15/164 2024-05-10T11:04:10.082 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libcephfs2-2:19.0.0-3449.gd39fa088.el9.x86_64 16/164 2024-05-10T11:04:10.082 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libcephsqlite-2:19.0.0-3449.gd39fa088.el9.x86_64 17/164 2024-05-10T11:04:10.082 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librados-devel-2:19.0.0-3449.gd39fa088.el9.x86_6 18/164 2024-05-10T11:04:10.082 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libradosstriper1-2:19.0.0-3449.gd39fa088.el9.x86 19/164 2024-05-10T11:04:10.082 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librgw2-2:19.0.0-3449.gd39fa088.el9.x86_64 20/164 2024-05-10T11:04:10.083 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3449.gd39fa088.el 21/164 2024-05-10T11:04:10.083 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-ceph-common-2:19.0.0-3449.gd39fa088.el9. 22/164 2024-05-10T11:04:10.083 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cephfs-2:19.0.0-3449.gd39fa088.el9.x86_6 23/164 2024-05-10T11:04:10.083 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-rados-2:19.0.0-3449.gd39fa088.el9.x86_64 24/164 2024-05-10T11:04:10.083 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-rbd-2:19.0.0-3449.gd39fa088.el9.x86_64 25/164 2024-05-10T11:04:10.083 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-rgw-2:19.0.0-3449.gd39fa088.el9.x86_64 26/164 2024-05-10T11:04:10.083 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : rbd-fuse-2:19.0.0-3449.gd39fa088.el9.x86_64 27/164 2024-05-10T11:04:10.095 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : rbd-mirror-2:19.0.0-3449.gd39fa088.el9.x86_64 28/164 2024-05-10T11:04:10.095 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : rbd-nbd-2:19.0.0-3449.gd39fa088.el9.x86_64 29/164 2024-05-10T11:04:10.096 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3449.gd39fa088. 30/164 2024-05-10T11:04:10.096 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3449.gd39fa088.el9.noa 31/164 2024-05-10T11:04:10.096 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3449.gd39fa088.el9.n 32/164 2024-05-10T11:04:10.096 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3449.gd39 33/164 2024-05-10T11:04:10.096 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3449.gd39fa088.el 34/164 2024-05-10T11:04:10.096 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3449.gd39fa088.el9.noarch 35/164 2024-05-10T11:04:10.096 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3449.gd39fa088.e 36/164 2024-05-10T11:04:10.096 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-volume-2:19.0.0-3449.gd39fa088.el9.noarch 37/164 2024-05-10T11:04:10.096 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : cephadm-2:19.0.0-3449.gd39fa088.el9.noarch 38/164 2024-05-10T11:04:10.096 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : cephfs-top-2:19.0.0-3449.gd39fa088.el9.noarch 39/164 2024-05-10T11:04:10.096 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-10T11:04:10.096 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-10T11:04:10.096 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-10T11:04:10.097 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-10T11:04:10.097 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-10T11:04:10.097 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-10T11:04:10.097 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-10T11:04:10.097 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-10T11:04:10.097 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-10T11:04:10.097 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-10T11:04:10.097 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-10T11:04:10.097 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-10T11:04:10.097 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-10T11:04:10.097 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-10T11:04:10.097 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-10T11:04:10.097 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-10T11:04:10.097 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-10T11:04:10.098 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-10T11:04:10.098 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-10T11:04:10.098 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-10T11:04:10.098 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-10T11:04:10.098 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-10T11:04:10.098 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-10T11:04:10.098 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-10T11:04:10.098 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-10T11:04:10.098 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-10T11:04:10.098 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-10T11:04:10.098 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-10T11:04:10.098 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-10T11:04:10.098 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-10T11:04:10.099 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-10T11:04:10.099 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-10T11:04:10.099 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-10T11:04:10.099 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-10T11:04:10.099 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-10T11:04:10.099 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-10T11:04:10.099 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-10T11:04:10.099 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-10T11:04:10.099 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-10T11:04:10.099 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-10T11:04:10.099 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-10T11:04:10.099 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-10T11:04:10.099 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-10T11:04:10.099 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-10T11:04:10.100 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-10T11:04:10.100 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-10T11:04:10.100 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-10T11:04:10.100 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-10T11:04:10.100 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-10T11:04:10.100 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-10T11:04:10.100 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-10T11:04:10.100 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-10T11:04:10.100 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-10T11:04:10.100 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-10T11:04:10.100 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-10T11:04:10.100 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-10T11:04:10.100 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-10T11:04:10.100 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-10T11:04:10.101 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-10T11:04:10.101 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-10T11:04:10.101 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-10T11:04:10.101 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-10T11:04:10.101 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-10T11:04:10.101 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-10T11:04:10.101 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-10T11:04:10.101 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-10T11:04:10.101 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-10T11:04:10.101 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-10T11:04:10.101 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-10T11:04:10.101 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-10T11:04:10.101 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-10T11:04:10.101 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-10T11:04:10.102 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-10T11:04:10.102 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-10T11:04:10.102 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-10T11:04:10.102 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-10T11:04:10.102 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-10T11:04:10.102 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-10T11:04:10.102 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-10T11:04:10.102 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-10T11:04:10.102 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-10T11:04:10.102 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-10T11:04:10.102 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-10T11:04:10.102 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-10T11:04:10.102 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-10T11:04:10.102 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-10T11:04:10.103 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-10T11:04:10.103 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-10T11:04:10.103 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-10T11:04:10.103 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-10T11:04:10.103 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-10T11:04:10.103 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-10T11:04:10.103 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-10T11:04:10.103 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-10T11:04:10.103 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-10T11:04:10.103 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-10T11:04:10.103 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-10T11:04:10.103 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-10T11:04:10.103 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-10T11:04:10.103 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-10T11:04:10.104 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-10T11:04:10.104 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-10T11:04:10.104 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-10T11:04:10.104 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-10T11:04:10.104 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-10T11:04:10.104 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-10T11:04:10.104 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-10T11:04:10.104 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-10T11:04:10.104 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-10T11:04:10.104 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librados2-2:19.0.0-3449.gd39fa088.el9.x86_64 149/164 2024-05-10T11:04:10.104 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-10T11:04:10.104 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librbd1-2:19.0.0-3449.gd39fa088.el9.x86_64 151/164 2024-05-10T11:04:10.104 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-10T11:04:10.105 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-10T11:04:10.105 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-10T11:04:10.105 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-10T11:04:10.105 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-10T11:04:10.105 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-10T11:04:10.105 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-10T11:04:10.105 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-10T11:04:10.105 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-10T11:04:10.105 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-10T11:04:10.105 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-10T11:04:10.105 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-10T11:04:10.139 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-10T11:04:10.173 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-10T11:04:10.194 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-10T11:04:10.961 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-10T11:04:11.038 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : cephfs-top-2:19.0.0-3449.gd39fa088.el9.noarch 152/164 2024-05-10T11:04:11.047 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-10T11:04:11.075 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-10T11:04:11.108 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-10T11:04:11.128 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-10T11:04:11.174 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-10T11:04:11.267 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-10T11:04:11.298 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-10T11:04:11.324 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-10T11:04:11.324 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-10T11:04:11.400 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-10T11:04:11.431 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-10T11:04:11.441 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-10T11:04:11.441 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-10T11:04:11.471 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : librbd1-2:19.0.0-3449.gd39fa088.el9.x86_64 101/164 2024-05-10T11:04:11.526 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-10T11:04:11.545 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librbd1-2:19.0.0-3449.gd39fa088.el9.x86_64 101/164 2024-05-10T11:04:11.551 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-10T11:04:11.556 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-10T11:04:11.556 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-10T11:04:11.576 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-rbd-2:19.0.0-3449.gd39fa088.el9.x86_64 102/164 2024-05-10T11:04:11.612 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-10T11:04:11.736 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-10T11:04:11.736 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-10T11:04:11.821 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-10T11:04:11.893 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-10T11:04:12.092 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-10T11:04:12.124 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : librgw2-2:19.0.0-3449.gd39fa088.el9.x86_64 105/164 2024-05-10T11:04:12.187 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librgw2-2:19.0.0-3449.gd39fa088.el9.x86_64 105/164 2024-05-10T11:04:12.209 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-rgw-2:19.0.0-3449.gd39fa088.el9.x86_64 106/164 2024-05-10T11:04:12.878 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-10T11:04:12.878 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-05-10T11:04:12.878 INFO:teuthology.orchestra.run.smithi178.stdout:Upgraded: 2024-05-10T11:04:12.878 INFO:teuthology.orchestra.run.smithi178.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-10T11:04:12.878 INFO:teuthology.orchestra.run.smithi178.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-10T11:04:12.878 INFO:teuthology.orchestra.run.smithi178.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-10T11:04:12.878 INFO:teuthology.orchestra.run.smithi178.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-10T11:04:12.878 INFO:teuthology.orchestra.run.smithi178.stdout: librados2-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:12.878 INFO:teuthology.orchestra.run.smithi178.stdout: librbd1-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:12.878 INFO:teuthology.orchestra.run.smithi178.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-10T11:04:12.878 INFO:teuthology.orchestra.run.smithi178.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-10T11:04:12.879 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-05-10T11:04:12.879 INFO:teuthology.orchestra.run.smithi178.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-10T11:04:12.879 INFO:teuthology.orchestra.run.smithi178.stdout: automake-1.16.2-8.el9.noarch 2024-05-10T11:04:12.879 INFO:teuthology.orchestra.run.smithi178.stdout: bison-3.7.4-5.el9.x86_64 2024-05-10T11:04:12.879 INFO:teuthology.orchestra.run.smithi178.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-10T11:04:12.879 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:12.879 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-base-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:12.879 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-common-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:12.879 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-fuse-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:12.879 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-grafana-dashboards-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T11:04:12.879 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-immutable-object-cache-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:12.879 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mds-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:12.879 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:12.879 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-cephadm-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T11:04:12.880 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-dashboard-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T11:04:12.880 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T11:04:12.880 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-modules-core-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T11:04:12.880 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-rook-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T11:04:12.880 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mon-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:12.880 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-osd-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:12.880 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-prometheus-alerts-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T11:04:12.880 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-radosgw-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:12.880 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-selinux-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:12.880 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-test-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:12.880 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-volume-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T11:04:12.880 INFO:teuthology.orchestra.run.smithi178.stdout: cephadm-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T11:04:12.880 INFO:teuthology.orchestra.run.smithi178.stdout: cephfs-mirror-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:12.880 INFO:teuthology.orchestra.run.smithi178.stdout: cephfs-shell-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:12.881 INFO:teuthology.orchestra.run.smithi178.stdout: cephfs-top-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T11:04:12.881 INFO:teuthology.orchestra.run.smithi178.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-10T11:04:12.881 INFO:teuthology.orchestra.run.smithi178.stdout: flex-2.6.4-9.el9.x86_64 2024-05-10T11:04:12.881 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-10T11:04:12.881 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-10T11:04:12.881 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-10T11:04:12.881 INFO:teuthology.orchestra.run.smithi178.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-10T11:04:12.881 INFO:teuthology.orchestra.run.smithi178.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-10T11:04:12.881 INFO:teuthology.orchestra.run.smithi178.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-10T11:04:12.881 INFO:teuthology.orchestra.run.smithi178.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-10T11:04:12.881 INFO:teuthology.orchestra.run.smithi178.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-10T11:04:12.881 INFO:teuthology.orchestra.run.smithi178.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-10T11:04:12.881 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs-devel-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:12.881 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs2-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:12.882 INFO:teuthology.orchestra.run.smithi178.stdout: libcephsqlite-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:12.882 INFO:teuthology.orchestra.run.smithi178.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-10T11:04:12.882 INFO:teuthology.orchestra.run.smithi178.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-10T11:04:12.882 INFO:teuthology.orchestra.run.smithi178.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-10T11:04:12.882 INFO:teuthology.orchestra.run.smithi178.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-10T11:04:12.882 INFO:teuthology.orchestra.run.smithi178.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-10T11:04:12.882 INFO:teuthology.orchestra.run.smithi178.stdout: librados-devel-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:12.882 INFO:teuthology.orchestra.run.smithi178.stdout: libradosstriper1-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:12.882 INFO:teuthology.orchestra.run.smithi178.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-10T11:04:12.882 INFO:teuthology.orchestra.run.smithi178.stdout: librgw2-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:12.882 INFO:teuthology.orchestra.run.smithi178.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-10T11:04:12.882 INFO:teuthology.orchestra.run.smithi178.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-10T11:04:12.882 INFO:teuthology.orchestra.run.smithi178.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-10T11:04:12.883 INFO:teuthology.orchestra.run.smithi178.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-10T11:04:12.883 INFO:teuthology.orchestra.run.smithi178.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-10T11:04:12.883 INFO:teuthology.orchestra.run.smithi178.stdout: lua-5.4.4-4.el9.x86_64 2024-05-10T11:04:12.883 INFO:teuthology.orchestra.run.smithi178.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-10T11:04:12.883 INFO:teuthology.orchestra.run.smithi178.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-10T11:04:12.883 INFO:teuthology.orchestra.run.smithi178.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-10T11:04:12.883 INFO:teuthology.orchestra.run.smithi178.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-10T11:04:12.883 INFO:teuthology.orchestra.run.smithi178.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-10T11:04:12.883 INFO:teuthology.orchestra.run.smithi178.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-10T11:04:12.883 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-13.14-1.el9.x86_64 2024-05-10T11:04:12.883 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-10T11:04:12.883 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-10T11:04:12.883 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-10T11:04:12.883 INFO:teuthology.orchestra.run.smithi178.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-10T11:04:12.884 INFO:teuthology.orchestra.run.smithi178.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-10T11:04:12.884 INFO:teuthology.orchestra.run.smithi178.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-10T11:04:12.884 INFO:teuthology.orchestra.run.smithi178.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-10T11:04:12.884 INFO:teuthology.orchestra.run.smithi178.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-10T11:04:12.884 INFO:teuthology.orchestra.run.smithi178.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-10T11:04:12.884 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-10T11:04:12.884 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-argparse-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:12.884 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-common-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:12.884 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cephfs-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:12.884 INFO:teuthology.orchestra.run.smithi178.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-10T11:04:12.884 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-10T11:04:12.884 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-10T11:04:12.884 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-10T11:04:12.884 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-10T11:04:12.885 INFO:teuthology.orchestra.run.smithi178.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-10T11:04:12.885 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-10T11:04:12.885 INFO:teuthology.orchestra.run.smithi178.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-10T11:04:12.885 INFO:teuthology.orchestra.run.smithi178.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-10T11:04:12.885 INFO:teuthology.orchestra.run.smithi178.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-10T11:04:12.885 INFO:teuthology.orchestra.run.smithi178.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-10T11:04:12.885 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-10T11:04:12.885 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-10T11:04:12.885 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-10T11:04:12.885 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-10T11:04:12.885 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-10T11:04:12.885 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-10T11:04:12.885 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-10T11:04:12.885 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-10T11:04:12.886 INFO:teuthology.orchestra.run.smithi178.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-10T11:04:12.886 INFO:teuthology.orchestra.run.smithi178.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-10T11:04:12.886 INFO:teuthology.orchestra.run.smithi178.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-10T11:04:12.886 INFO:teuthology.orchestra.run.smithi178.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-10T11:04:12.886 INFO:teuthology.orchestra.run.smithi178.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-10T11:04:12.886 INFO:teuthology.orchestra.run.smithi178.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-10T11:04:12.886 INFO:teuthology.orchestra.run.smithi178.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-10T11:04:12.886 INFO:teuthology.orchestra.run.smithi178.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-10T11:04:12.886 INFO:teuthology.orchestra.run.smithi178.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-10T11:04:12.886 INFO:teuthology.orchestra.run.smithi178.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-10T11:04:12.886 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-10T11:04:12.886 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ply-3.11-14.el9.noarch 2024-05-10T11:04:12.886 INFO:teuthology.orchestra.run.smithi178.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-10T11:04:12.886 INFO:teuthology.orchestra.run.smithi178.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-10T11:04:12.887 INFO:teuthology.orchestra.run.smithi178.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-10T11:04:12.887 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-10T11:04:12.887 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-10T11:04:12.887 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-10T11:04:12.887 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-10T11:04:12.887 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-10T11:04:12.887 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-10T11:04:12.887 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rados-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:12.887 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rbd-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:12.887 INFO:teuthology.orchestra.run.smithi178.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-10T11:04:12.887 INFO:teuthology.orchestra.run.smithi178.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-10T11:04:12.887 INFO:teuthology.orchestra.run.smithi178.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-10T11:04:12.887 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rgw-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:12.887 INFO:teuthology.orchestra.run.smithi178.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-10T11:04:12.887 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-10T11:04:12.888 INFO:teuthology.orchestra.run.smithi178.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-10T11:04:12.888 INFO:teuthology.orchestra.run.smithi178.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-10T11:04:12.888 INFO:teuthology.orchestra.run.smithi178.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-10T11:04:12.888 INFO:teuthology.orchestra.run.smithi178.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-10T11:04:12.888 INFO:teuthology.orchestra.run.smithi178.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-10T11:04:12.888 INFO:teuthology.orchestra.run.smithi178.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-10T11:04:12.888 INFO:teuthology.orchestra.run.smithi178.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-10T11:04:12.888 INFO:teuthology.orchestra.run.smithi178.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-10T11:04:12.888 INFO:teuthology.orchestra.run.smithi178.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-10T11:04:12.888 INFO:teuthology.orchestra.run.smithi178.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-10T11:04:12.888 INFO:teuthology.orchestra.run.smithi178.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-10T11:04:12.888 INFO:teuthology.orchestra.run.smithi178.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-10T11:04:12.888 INFO:teuthology.orchestra.run.smithi178.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-10T11:04:12.888 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-fuse-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:12.888 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-mirror-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:12.889 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-nbd-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:12.889 INFO:teuthology.orchestra.run.smithi178.stdout: re2-1:20211101-3.el9.x86_64 2024-05-10T11:04:12.889 INFO:teuthology.orchestra.run.smithi178.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-10T11:04:12.889 INFO:teuthology.orchestra.run.smithi178.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-10T11:04:12.889 INFO:teuthology.orchestra.run.smithi178.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-10T11:04:12.889 INFO:teuthology.orchestra.run.smithi178.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-10T11:04:12.889 INFO:teuthology.orchestra.run.smithi178.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-10T11:04:12.889 INFO:teuthology.orchestra.run.smithi178.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-10T11:04:12.889 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-05-10T11:04:12.889 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-05-10T11:04:13.297 DEBUG:teuthology.parallel:result is None 2024-05-10T11:04:14.532 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-common-2:19.0.0-3449.gd39fa088.el9.x86_64 107/164 2024-05-10T11:04:14.572 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-common-2:19.0.0-3449.gd39fa088.el9.x86_64 107/164 2024-05-10T11:04:15.119 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-common-2:19.0.0-3449.gd39fa088.el9.x86_64 107/164 2024-05-10T11:04:15.159 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-base-2:19.0.0-3449.gd39fa088.el9.x86_64 108/164 2024-05-10T11:04:15.258 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-base-2:19.0.0-3449.gd39fa088.el9.x86_64 108/164 2024-05-10T11:04:15.258 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-10T11:04:15.258 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-10T11:04:15.259 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-10T11:04:15.302 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-selinux-2:19.0.0-3449.gd39fa088.el9.x86_64 109/164 2024-05-10T11:04:17.576 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-10T11:04:17.577 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-2:19.0.0-3449.gd39fa088.el9.x86_64 1/164 2024-05-10T11:04:17.577 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-base-2:19.0.0-3449.gd39fa088.el9.x86_64 2/164 2024-05-10T11:04:17.577 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-common-2:19.0.0-3449.gd39fa088.el9.x86_64 3/164 2024-05-10T11:04:17.577 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-fuse-2:19.0.0-3449.gd39fa088.el9.x86_64 4/164 2024-05-10T11:04:17.577 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3449.gd39fa 5/164 2024-05-10T11:04:17.577 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mds-2:19.0.0-3449.gd39fa088.el9.x86_64 6/164 2024-05-10T11:04:17.577 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-2:19.0.0-3449.gd39fa088.el9.x86_64 7/164 2024-05-10T11:04:17.577 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mon-2:19.0.0-3449.gd39fa088.el9.x86_64 8/164 2024-05-10T11:04:17.577 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-osd-2:19.0.0-3449.gd39fa088.el9.x86_64 9/164 2024-05-10T11:04:17.577 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-radosgw-2:19.0.0-3449.gd39fa088.el9.x86_64 10/164 2024-05-10T11:04:17.577 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-selinux-2:19.0.0-3449.gd39fa088.el9.x86_64 11/164 2024-05-10T11:04:17.577 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-test-2:19.0.0-3449.gd39fa088.el9.x86_64 12/164 2024-05-10T11:04:17.577 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : cephfs-mirror-2:19.0.0-3449.gd39fa088.el9.x86_64 13/164 2024-05-10T11:04:17.578 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : cephfs-shell-2:19.0.0-3449.gd39fa088.el9.x86_64 14/164 2024-05-10T11:04:17.578 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libcephfs-devel-2:19.0.0-3449.gd39fa088.el9.x86_ 15/164 2024-05-10T11:04:17.578 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libcephfs2-2:19.0.0-3449.gd39fa088.el9.x86_64 16/164 2024-05-10T11:04:17.578 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libcephsqlite-2:19.0.0-3449.gd39fa088.el9.x86_64 17/164 2024-05-10T11:04:17.578 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librados-devel-2:19.0.0-3449.gd39fa088.el9.x86_6 18/164 2024-05-10T11:04:17.578 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libradosstriper1-2:19.0.0-3449.gd39fa088.el9.x86 19/164 2024-05-10T11:04:17.578 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librgw2-2:19.0.0-3449.gd39fa088.el9.x86_64 20/164 2024-05-10T11:04:17.578 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3449.gd39fa088.el 21/164 2024-05-10T11:04:17.578 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-ceph-common-2:19.0.0-3449.gd39fa088.el9. 22/164 2024-05-10T11:04:17.578 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cephfs-2:19.0.0-3449.gd39fa088.el9.x86_6 23/164 2024-05-10T11:04:17.578 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rados-2:19.0.0-3449.gd39fa088.el9.x86_64 24/164 2024-05-10T11:04:17.578 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rbd-2:19.0.0-3449.gd39fa088.el9.x86_64 25/164 2024-05-10T11:04:17.578 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rgw-2:19.0.0-3449.gd39fa088.el9.x86_64 26/164 2024-05-10T11:04:17.578 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : rbd-fuse-2:19.0.0-3449.gd39fa088.el9.x86_64 27/164 2024-05-10T11:04:17.579 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : rbd-mirror-2:19.0.0-3449.gd39fa088.el9.x86_64 28/164 2024-05-10T11:04:17.579 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : rbd-nbd-2:19.0.0-3449.gd39fa088.el9.x86_64 29/164 2024-05-10T11:04:17.579 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3449.gd39fa088. 30/164 2024-05-10T11:04:17.579 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3449.gd39fa088.el9.noa 31/164 2024-05-10T11:04:17.579 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3449.gd39fa088.el9.n 32/164 2024-05-10T11:04:17.579 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3449.gd39 33/164 2024-05-10T11:04:17.579 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3449.gd39fa088.el 34/164 2024-05-10T11:04:17.579 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3449.gd39fa088.el9.noarch 35/164 2024-05-10T11:04:17.579 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3449.gd39fa088.e 36/164 2024-05-10T11:04:17.579 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-volume-2:19.0.0-3449.gd39fa088.el9.noarch 37/164 2024-05-10T11:04:17.579 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : cephadm-2:19.0.0-3449.gd39fa088.el9.noarch 38/164 2024-05-10T11:04:17.579 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : cephfs-top-2:19.0.0-3449.gd39fa088.el9.noarch 39/164 2024-05-10T11:04:17.579 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-10T11:04:17.579 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-10T11:04:17.580 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-10T11:04:17.580 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-10T11:04:17.580 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-10T11:04:17.580 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-10T11:04:17.580 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-10T11:04:17.580 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-10T11:04:17.580 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-10T11:04:17.580 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-10T11:04:17.580 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-10T11:04:17.580 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-10T11:04:17.580 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-10T11:04:17.580 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-10T11:04:17.580 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-10T11:04:17.580 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-10T11:04:17.581 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-10T11:04:17.581 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-10T11:04:17.581 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-10T11:04:17.581 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-10T11:04:17.581 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-10T11:04:17.581 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-10T11:04:17.581 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-10T11:04:17.581 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-10T11:04:17.581 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-10T11:04:17.581 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-10T11:04:17.581 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-10T11:04:17.581 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-10T11:04:17.581 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-10T11:04:17.581 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-10T11:04:17.582 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-10T11:04:17.582 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-10T11:04:17.582 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-10T11:04:17.582 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-10T11:04:17.582 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-10T11:04:17.582 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-10T11:04:17.582 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-10T11:04:17.582 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-10T11:04:17.582 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-10T11:04:17.582 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-10T11:04:17.582 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-10T11:04:17.582 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-10T11:04:17.582 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-10T11:04:17.582 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-10T11:04:17.583 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-10T11:04:17.583 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-10T11:04:17.583 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-10T11:04:17.583 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-10T11:04:17.583 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-10T11:04:17.583 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-10T11:04:17.583 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-10T11:04:17.583 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-10T11:04:17.583 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-10T11:04:17.583 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-10T11:04:17.583 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-10T11:04:17.583 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-10T11:04:17.583 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-10T11:04:17.584 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-10T11:04:17.584 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-10T11:04:17.584 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-10T11:04:17.584 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-10T11:04:17.584 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-10T11:04:17.584 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-10T11:04:17.584 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-10T11:04:17.584 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-10T11:04:17.585 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-10T11:04:17.585 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-10T11:04:17.585 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-10T11:04:17.585 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-10T11:04:17.585 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-10T11:04:17.585 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-10T11:04:17.585 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-10T11:04:17.585 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-10T11:04:17.585 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-10T11:04:17.585 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-10T11:04:17.585 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-10T11:04:17.585 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-10T11:04:17.586 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-10T11:04:17.586 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-10T11:04:17.586 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-10T11:04:17.586 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-10T11:04:17.586 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-10T11:04:17.586 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-10T11:04:17.586 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-10T11:04:17.586 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-10T11:04:17.586 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-10T11:04:17.586 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-10T11:04:17.586 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-10T11:04:17.587 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-10T11:04:17.587 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-10T11:04:17.587 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-10T11:04:17.587 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-10T11:04:17.587 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-10T11:04:17.587 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-10T11:04:17.587 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-10T11:04:17.587 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-10T11:04:17.587 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-10T11:04:17.587 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-10T11:04:17.587 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-10T11:04:17.587 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-10T11:04:17.588 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-10T11:04:17.588 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-10T11:04:17.588 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-10T11:04:17.588 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-10T11:04:17.588 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-10T11:04:17.588 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-10T11:04:17.588 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-10T11:04:17.588 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-10T11:04:17.588 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-10T11:04:17.588 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librados2-2:19.0.0-3449.gd39fa088.el9.x86_64 149/164 2024-05-10T11:04:17.588 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-10T11:04:17.588 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librbd1-2:19.0.0-3449.gd39fa088.el9.x86_64 151/164 2024-05-10T11:04:17.589 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-10T11:04:17.589 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-10T11:04:17.589 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-10T11:04:17.589 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-10T11:04:17.589 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-10T11:04:17.589 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-10T11:04:17.589 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-10T11:04:17.589 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-10T11:04:17.589 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-10T11:04:17.589 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-10T11:04:17.589 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-10T11:04:17.589 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-10T11:04:20.040 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-10T11:04:20.041 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:04:20.041 INFO:teuthology.orchestra.run.smithi100.stdout:Upgraded: 2024-05-10T11:04:20.041 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-10T11:04:20.041 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-10T11:04:20.041 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-10T11:04:20.041 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-10T11:04:20.041 INFO:teuthology.orchestra.run.smithi100.stdout: librados2-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:20.041 INFO:teuthology.orchestra.run.smithi100.stdout: librbd1-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:20.041 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-10T11:04:20.041 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-10T11:04:20.041 INFO:teuthology.orchestra.run.smithi100.stdout:Installed: 2024-05-10T11:04:20.041 INFO:teuthology.orchestra.run.smithi100.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-10T11:04:20.041 INFO:teuthology.orchestra.run.smithi100.stdout: automake-1.16.2-8.el9.noarch 2024-05-10T11:04:20.042 INFO:teuthology.orchestra.run.smithi100.stdout: bison-3.7.4-5.el9.x86_64 2024-05-10T11:04:20.042 INFO:teuthology.orchestra.run.smithi100.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-10T11:04:20.042 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:20.042 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-base-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:20.042 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-common-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:20.042 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-fuse-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:20.042 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-grafana-dashboards-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T11:04:20.042 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-immutable-object-cache-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:20.042 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mds-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:20.042 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:20.042 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-cephadm-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T11:04:20.042 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-dashboard-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T11:04:20.043 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T11:04:20.043 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-modules-core-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T11:04:20.043 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-rook-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T11:04:20.043 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mon-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:20.043 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-osd-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:20.043 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-prometheus-alerts-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T11:04:20.043 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-radosgw-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:20.043 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-selinux-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:20.043 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-test-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:20.043 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-volume-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T11:04:20.043 INFO:teuthology.orchestra.run.smithi100.stdout: cephadm-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T11:04:20.043 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-mirror-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:20.043 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-shell-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:20.044 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-top-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T11:04:20.044 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-10T11:04:20.044 INFO:teuthology.orchestra.run.smithi100.stdout: flex-2.6.4-9.el9.x86_64 2024-05-10T11:04:20.044 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-10T11:04:20.044 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-10T11:04:20.044 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-10T11:04:20.044 INFO:teuthology.orchestra.run.smithi100.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-10T11:04:20.044 INFO:teuthology.orchestra.run.smithi100.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-10T11:04:20.044 INFO:teuthology.orchestra.run.smithi100.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-10T11:04:20.045 INFO:teuthology.orchestra.run.smithi100.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-10T11:04:20.045 INFO:teuthology.orchestra.run.smithi100.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-10T11:04:20.045 INFO:teuthology.orchestra.run.smithi100.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-10T11:04:20.045 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs-devel-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:20.045 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs2-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:20.045 INFO:teuthology.orchestra.run.smithi100.stdout: libcephsqlite-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:20.045 INFO:teuthology.orchestra.run.smithi100.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-10T11:04:20.045 INFO:teuthology.orchestra.run.smithi100.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-10T11:04:20.045 INFO:teuthology.orchestra.run.smithi100.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-10T11:04:20.045 INFO:teuthology.orchestra.run.smithi100.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-10T11:04:20.045 INFO:teuthology.orchestra.run.smithi100.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-10T11:04:20.045 INFO:teuthology.orchestra.run.smithi100.stdout: librados-devel-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:20.046 INFO:teuthology.orchestra.run.smithi100.stdout: libradosstriper1-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:20.046 INFO:teuthology.orchestra.run.smithi100.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-10T11:04:20.046 INFO:teuthology.orchestra.run.smithi100.stdout: librgw2-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:20.046 INFO:teuthology.orchestra.run.smithi100.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-10T11:04:20.046 INFO:teuthology.orchestra.run.smithi100.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-10T11:04:20.046 INFO:teuthology.orchestra.run.smithi100.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-10T11:04:20.046 INFO:teuthology.orchestra.run.smithi100.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-10T11:04:20.046 INFO:teuthology.orchestra.run.smithi100.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-10T11:04:20.046 INFO:teuthology.orchestra.run.smithi100.stdout: lua-5.4.4-4.el9.x86_64 2024-05-10T11:04:20.046 INFO:teuthology.orchestra.run.smithi100.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-10T11:04:20.046 INFO:teuthology.orchestra.run.smithi100.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-10T11:04:20.046 INFO:teuthology.orchestra.run.smithi100.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-10T11:04:20.047 INFO:teuthology.orchestra.run.smithi100.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-10T11:04:20.047 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-10T11:04:20.047 INFO:teuthology.orchestra.run.smithi100.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-10T11:04:20.047 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-13.14-1.el9.x86_64 2024-05-10T11:04:20.047 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-10T11:04:20.047 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-10T11:04:20.047 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-10T11:04:20.047 INFO:teuthology.orchestra.run.smithi100.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-10T11:04:20.047 INFO:teuthology.orchestra.run.smithi100.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-10T11:04:20.047 INFO:teuthology.orchestra.run.smithi100.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-10T11:04:20.047 INFO:teuthology.orchestra.run.smithi100.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-10T11:04:20.047 INFO:teuthology.orchestra.run.smithi100.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-10T11:04:20.047 INFO:teuthology.orchestra.run.smithi100.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-10T11:04:20.047 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-10T11:04:20.048 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-argparse-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:20.048 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-common-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:20.048 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cephfs-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:20.048 INFO:teuthology.orchestra.run.smithi100.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-10T11:04:20.048 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-10T11:04:20.048 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-10T11:04:20.048 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-10T11:04:20.048 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-10T11:04:20.048 INFO:teuthology.orchestra.run.smithi100.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-10T11:04:20.048 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-10T11:04:20.048 INFO:teuthology.orchestra.run.smithi100.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-10T11:04:20.048 INFO:teuthology.orchestra.run.smithi100.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-10T11:04:20.048 INFO:teuthology.orchestra.run.smithi100.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-10T11:04:20.048 INFO:teuthology.orchestra.run.smithi100.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-10T11:04:20.049 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-10T11:04:20.049 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-10T11:04:20.049 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-10T11:04:20.049 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-10T11:04:20.049 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-10T11:04:20.049 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-10T11:04:20.049 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-10T11:04:20.049 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-10T11:04:20.049 INFO:teuthology.orchestra.run.smithi100.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-10T11:04:20.049 INFO:teuthology.orchestra.run.smithi100.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-10T11:04:20.049 INFO:teuthology.orchestra.run.smithi100.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-10T11:04:20.049 INFO:teuthology.orchestra.run.smithi100.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-10T11:04:20.049 INFO:teuthology.orchestra.run.smithi100.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-10T11:04:20.050 INFO:teuthology.orchestra.run.smithi100.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-10T11:04:20.050 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-10T11:04:20.050 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-10T11:04:20.050 INFO:teuthology.orchestra.run.smithi100.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-10T11:04:20.050 INFO:teuthology.orchestra.run.smithi100.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-10T11:04:20.050 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-10T11:04:20.050 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ply-3.11-14.el9.noarch 2024-05-10T11:04:20.050 INFO:teuthology.orchestra.run.smithi100.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-10T11:04:20.050 INFO:teuthology.orchestra.run.smithi100.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-10T11:04:20.050 INFO:teuthology.orchestra.run.smithi100.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-10T11:04:20.050 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-10T11:04:20.050 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-10T11:04:20.050 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-10T11:04:20.051 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-10T11:04:20.051 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-10T11:04:20.051 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-10T11:04:20.051 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rados-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:20.051 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rbd-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:20.051 INFO:teuthology.orchestra.run.smithi100.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-10T11:04:20.051 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-10T11:04:20.051 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-10T11:04:20.051 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rgw-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:20.051 INFO:teuthology.orchestra.run.smithi100.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-10T11:04:20.051 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-10T11:04:20.051 INFO:teuthology.orchestra.run.smithi100.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-10T11:04:20.051 INFO:teuthology.orchestra.run.smithi100.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-10T11:04:20.051 INFO:teuthology.orchestra.run.smithi100.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-10T11:04:20.052 INFO:teuthology.orchestra.run.smithi100.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-10T11:04:20.052 INFO:teuthology.orchestra.run.smithi100.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-10T11:04:20.052 INFO:teuthology.orchestra.run.smithi100.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-10T11:04:20.052 INFO:teuthology.orchestra.run.smithi100.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-10T11:04:20.052 INFO:teuthology.orchestra.run.smithi100.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-10T11:04:20.052 INFO:teuthology.orchestra.run.smithi100.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-10T11:04:20.052 INFO:teuthology.orchestra.run.smithi100.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-10T11:04:20.052 INFO:teuthology.orchestra.run.smithi100.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-10T11:04:20.052 INFO:teuthology.orchestra.run.smithi100.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-10T11:04:20.052 INFO:teuthology.orchestra.run.smithi100.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-10T11:04:20.052 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-fuse-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:20.052 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-mirror-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:20.052 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-nbd-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:20.053 INFO:teuthology.orchestra.run.smithi100.stdout: re2-1:20211101-3.el9.x86_64 2024-05-10T11:04:20.053 INFO:teuthology.orchestra.run.smithi100.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-10T11:04:20.053 INFO:teuthology.orchestra.run.smithi100.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-10T11:04:20.053 INFO:teuthology.orchestra.run.smithi100.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-10T11:04:20.053 INFO:teuthology.orchestra.run.smithi100.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-10T11:04:20.053 INFO:teuthology.orchestra.run.smithi100.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-10T11:04:20.053 INFO:teuthology.orchestra.run.smithi100.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-10T11:04:20.054 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:04:20.054 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-05-10T11:04:20.520 DEBUG:teuthology.parallel:result is None 2024-05-10T11:04:27.291 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3449.gd39fa088.el9.x86_64 109/164 2024-05-10T11:04:27.291 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /sys 2024-05-10T11:04:27.292 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /proc 2024-05-10T11:04:27.292 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /mnt 2024-05-10T11:04:27.292 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /var/tmp 2024-05-10T11:04:27.292 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /home 2024-05-10T11:04:27.292 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /root 2024-05-10T11:04:27.292 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /tmp 2024-05-10T11:04:27.292 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-10T11:04:27.468 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-volume-2:19.0.0-3449.gd39fa088.el9.noarch 110/164 2024-05-10T11:04:27.485 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-volume-2:19.0.0-3449.gd39fa088.el9.noarch 110/164 2024-05-10T11:04:27.485 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T11:04:27.485 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-10T11:04:27.485 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-10T11:04:28.988 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-osd-2:19.0.0-3449.gd39fa088.el9.x86_64 111/164 2024-05-10T11:04:29.015 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-osd-2:19.0.0-3449.gd39fa088.el9.x86_64 111/164 2024-05-10T11:04:29.015 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T11:04:29.015 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-10T11:04:29.015 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-10T11:04:29.016 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-10T11:04:29.016 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-10T11:04:29.246 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mds-2:19.0.0-3449.gd39fa088.el9.x86_64 112/164 2024-05-10T11:04:29.269 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mds-2:19.0.0-3449.gd39fa088.el9.x86_64 112/164 2024-05-10T11:04:29.270 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T11:04:29.270 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-10T11:04:29.270 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-10T11:04:29.270 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-10T11:04:29.270 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-10T11:04:29.706 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mon-2:19.0.0-3449.gd39fa088.el9.x86_64 113/164 2024-05-10T11:04:29.729 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mon-2:19.0.0-3449.gd39fa088.el9.x86_64 113/164 2024-05-10T11:04:29.729 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T11:04:29.729 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-10T11:04:29.729 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-10T11:04:29.729 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-10T11:04:29.729 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-10T11:04:29.827 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-10T11:04:29.883 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-10T11:04:29.939 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-10T11:04:30.123 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-10T11:04:30.181 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-10T11:04:30.258 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-10T11:04:30.346 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-10T11:04:30.477 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-10T11:04:30.531 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-10T11:04:30.564 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-10T11:04:30.599 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-10T11:04:30.986 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: cephadm-2:19.0.0-3449.gd39fa088.el9.noarch 125/164 2024-05-10T11:04:31.014 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : cephadm-2:19.0.0-3449.gd39fa088.el9.noarch 125/164 2024-05-10T11:04:31.046 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3449.gd39fa088.e 126/164 2024-05-10T11:04:31.108 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3449.gd39fa088. 127/164 2024-05-10T11:04:31.137 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3449.gd39fa088.el9.noa 128/164 2024-05-10T11:04:32.172 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3449.gd39fa088.el9.noa 128/164 2024-05-10T11:04:32.213 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3449.gd39fa088.el9.n 129/164 2024-05-10T11:04:33.179 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3449.gd39fa088.el9.n 129/164 2024-05-10T11:04:33.206 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3449.gd39 130/164 2024-05-10T11:04:33.315 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3449.gd39 130/164 2024-05-10T11:04:33.475 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3449.gd39fa088.el 131/164 2024-05-10T11:04:33.511 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-2:19.0.0-3449.gd39fa088.el9.x86_64 132/164 2024-05-10T11:04:33.538 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3449.gd39fa088.el9.x86_64 132/164 2024-05-10T11:04:33.538 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T11:04:33.538 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-10T11:04:33.539 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-10T11:04:33.539 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-10T11:04:33.539 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-10T11:04:33.584 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-rook-2:19.0.0-3449.gd39fa088.el9.noarch 133/164 2024-05-10T11:04:33.595 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3449.gd39fa088.el9.noarch 133/164 2024-05-10T11:04:33.701 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-2:19.0.0-3449.gd39fa088.el9.x86_64 134/164 2024-05-10T11:04:33.758 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-fuse-2:19.0.0-3449.gd39fa088.el9.x86_64 135/164 2024-05-10T11:04:34.695 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-10T11:04:34.719 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-radosgw-2:19.0.0-3449.gd39fa088.el9.x86_64 137/164 2024-05-10T11:04:34.743 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3449.gd39fa088.el9.x86_64 137/164 2024-05-10T11:04:34.744 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T11:04:34.744 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-10T11:04:34.744 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-10T11:04:34.744 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-10T11:04:34.744 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-10T11:04:34.795 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3449.gd39fa 138/164 2024-05-10T11:04:34.818 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3449.gd39fa 138/164 2024-05-10T11:04:34.818 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T11:04:34.818 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-10T11:04:34.818 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-10T11:04:34.862 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : cephfs-mirror-2:19.0.0-3449.gd39fa088.el9.x86_64 139/164 2024-05-10T11:04:34.885 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3449.gd39fa088.el9.x86_64 139/164 2024-05-10T11:04:34.885 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T11:04:34.886 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-10T11:04:34.886 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-10T11:04:34.886 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-10T11:04:34.886 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-10T11:04:35.188 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : rbd-mirror-2:19.0.0-3449.gd39fa088.el9.x86_64 140/164 2024-05-10T11:04:35.211 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3449.gd39fa088.el9.x86_64 140/164 2024-05-10T11:04:35.211 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T11:04:35.211 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-10T11:04:35.211 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-10T11:04:35.212 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-10T11:04:35.212 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-10T11:04:39.815 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-test-2:19.0.0-3449.gd39fa088.el9.x86_64 141/164 2024-05-10T11:04:39.857 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : rbd-fuse-2:19.0.0-3449.gd39fa088.el9.x86_64 142/164 2024-05-10T11:04:39.898 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : rbd-nbd-2:19.0.0-3449.gd39fa088.el9.x86_64 143/164 2024-05-10T11:04:39.930 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-10T11:04:39.957 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-10T11:04:40.139 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : cephfs-shell-2:19.0.0-3449.gd39fa088.el9.x86_64 146/164 2024-05-10T11:04:40.186 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-10T11:04:40.217 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libcephfs-devel-2:19.0.0-3449.gd39fa088.el9.x86_ 148/164 2024-05-10T11:04:40.246 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-10T11:04:40.284 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-10T11:04:40.892 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-10T11:04:40.931 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-10T11:04:40.952 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-10T11:04:41.788 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : cephfs-top-2:19.0.0-3449.gd39fa088.el9.noarch 152/164 2024-05-10T11:04:41.886 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-10T11:04:41.930 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-10T11:04:42.025 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-10T11:04:42.056 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-10T11:04:42.082 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-10T11:04:42.082 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-10T11:04:42.166 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-10T11:04:42.199 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-10T11:04:42.199 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-10T11:04:42.283 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-10T11:04:42.308 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-10T11:04:42.314 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-10T11:04:42.314 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-10T11:04:42.494 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-10T11:04:42.494 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-10T11:04:42.571 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-10T11:04:42.643 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-10T11:04:48.274 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-10T11:04:48.275 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-2:19.0.0-3449.gd39fa088.el9.x86_64 1/164 2024-05-10T11:04:48.275 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-base-2:19.0.0-3449.gd39fa088.el9.x86_64 2/164 2024-05-10T11:04:48.275 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-common-2:19.0.0-3449.gd39fa088.el9.x86_64 3/164 2024-05-10T11:04:48.275 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-fuse-2:19.0.0-3449.gd39fa088.el9.x86_64 4/164 2024-05-10T11:04:48.275 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3449.gd39fa 5/164 2024-05-10T11:04:48.275 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mds-2:19.0.0-3449.gd39fa088.el9.x86_64 6/164 2024-05-10T11:04:48.275 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-2:19.0.0-3449.gd39fa088.el9.x86_64 7/164 2024-05-10T11:04:48.275 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mon-2:19.0.0-3449.gd39fa088.el9.x86_64 8/164 2024-05-10T11:04:48.275 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-osd-2:19.0.0-3449.gd39fa088.el9.x86_64 9/164 2024-05-10T11:04:48.275 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-radosgw-2:19.0.0-3449.gd39fa088.el9.x86_64 10/164 2024-05-10T11:04:48.275 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-selinux-2:19.0.0-3449.gd39fa088.el9.x86_64 11/164 2024-05-10T11:04:48.275 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-test-2:19.0.0-3449.gd39fa088.el9.x86_64 12/164 2024-05-10T11:04:48.276 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : cephfs-mirror-2:19.0.0-3449.gd39fa088.el9.x86_64 13/164 2024-05-10T11:04:48.276 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : cephfs-shell-2:19.0.0-3449.gd39fa088.el9.x86_64 14/164 2024-05-10T11:04:48.276 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libcephfs-devel-2:19.0.0-3449.gd39fa088.el9.x86_ 15/164 2024-05-10T11:04:48.276 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libcephfs2-2:19.0.0-3449.gd39fa088.el9.x86_64 16/164 2024-05-10T11:04:48.276 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libcephsqlite-2:19.0.0-3449.gd39fa088.el9.x86_64 17/164 2024-05-10T11:04:48.276 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librados-devel-2:19.0.0-3449.gd39fa088.el9.x86_6 18/164 2024-05-10T11:04:48.276 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libradosstriper1-2:19.0.0-3449.gd39fa088.el9.x86 19/164 2024-05-10T11:04:48.276 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librgw2-2:19.0.0-3449.gd39fa088.el9.x86_64 20/164 2024-05-10T11:04:48.276 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3449.gd39fa088.el 21/164 2024-05-10T11:04:48.276 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-ceph-common-2:19.0.0-3449.gd39fa088.el9. 22/164 2024-05-10T11:04:48.276 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cephfs-2:19.0.0-3449.gd39fa088.el9.x86_6 23/164 2024-05-10T11:04:48.276 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-rados-2:19.0.0-3449.gd39fa088.el9.x86_64 24/164 2024-05-10T11:04:48.276 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-rbd-2:19.0.0-3449.gd39fa088.el9.x86_64 25/164 2024-05-10T11:04:48.276 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-rgw-2:19.0.0-3449.gd39fa088.el9.x86_64 26/164 2024-05-10T11:04:48.277 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : rbd-fuse-2:19.0.0-3449.gd39fa088.el9.x86_64 27/164 2024-05-10T11:04:48.277 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : rbd-mirror-2:19.0.0-3449.gd39fa088.el9.x86_64 28/164 2024-05-10T11:04:48.277 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : rbd-nbd-2:19.0.0-3449.gd39fa088.el9.x86_64 29/164 2024-05-10T11:04:48.277 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3449.gd39fa088. 30/164 2024-05-10T11:04:48.277 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3449.gd39fa088.el9.noa 31/164 2024-05-10T11:04:48.289 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3449.gd39fa088.el9.n 32/164 2024-05-10T11:04:48.289 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3449.gd39 33/164 2024-05-10T11:04:48.289 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3449.gd39fa088.el 34/164 2024-05-10T11:04:48.289 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3449.gd39fa088.el9.noarch 35/164 2024-05-10T11:04:48.289 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3449.gd39fa088.e 36/164 2024-05-10T11:04:48.290 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-volume-2:19.0.0-3449.gd39fa088.el9.noarch 37/164 2024-05-10T11:04:48.290 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : cephadm-2:19.0.0-3449.gd39fa088.el9.noarch 38/164 2024-05-10T11:04:48.290 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : cephfs-top-2:19.0.0-3449.gd39fa088.el9.noarch 39/164 2024-05-10T11:04:48.290 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-10T11:04:48.290 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-10T11:04:48.290 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-10T11:04:48.290 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-10T11:04:48.290 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-10T11:04:48.290 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-10T11:04:48.290 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-10T11:04:48.290 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-10T11:04:48.290 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-10T11:04:48.290 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-10T11:04:48.291 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-10T11:04:48.291 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-10T11:04:48.291 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-10T11:04:48.291 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-10T11:04:48.291 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-10T11:04:48.291 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-10T11:04:48.291 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-10T11:04:48.291 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-10T11:04:48.291 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-10T11:04:48.291 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-10T11:04:48.291 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-10T11:04:48.291 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-10T11:04:48.291 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-10T11:04:48.292 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-10T11:04:48.292 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-10T11:04:48.292 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-10T11:04:48.292 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-10T11:04:48.292 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-10T11:04:48.292 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-10T11:04:48.292 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-10T11:04:48.292 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-10T11:04:48.292 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-10T11:04:48.292 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-10T11:04:48.292 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-10T11:04:48.292 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-10T11:04:48.292 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-10T11:04:48.292 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-10T11:04:48.293 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-10T11:04:48.293 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-10T11:04:48.293 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-10T11:04:48.293 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-10T11:04:48.293 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-10T11:04:48.293 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-10T11:04:48.293 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-10T11:04:48.293 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-10T11:04:48.293 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-10T11:04:48.293 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-10T11:04:48.293 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-10T11:04:48.293 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-10T11:04:48.293 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-10T11:04:48.294 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-10T11:04:48.294 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-10T11:04:48.294 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-10T11:04:48.294 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-10T11:04:48.294 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-10T11:04:48.294 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-10T11:04:48.294 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-10T11:04:48.294 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-10T11:04:48.294 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-10T11:04:48.294 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-10T11:04:48.294 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-10T11:04:48.294 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-10T11:04:48.295 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-10T11:04:48.295 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-10T11:04:48.295 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-10T11:04:48.295 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-10T11:04:48.295 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-10T11:04:48.295 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-10T11:04:48.295 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-10T11:04:48.295 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-10T11:04:48.295 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-10T11:04:48.295 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-10T11:04:48.295 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-10T11:04:48.295 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-10T11:04:48.295 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-10T11:04:48.295 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-10T11:04:48.296 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-10T11:04:48.296 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-10T11:04:48.296 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-10T11:04:48.296 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-10T11:04:48.296 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-10T11:04:48.296 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-10T11:04:48.296 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-10T11:04:48.296 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-10T11:04:48.296 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-10T11:04:48.296 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-10T11:04:48.296 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-10T11:04:48.296 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-10T11:04:48.296 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-10T11:04:48.296 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-10T11:04:48.297 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-10T11:04:48.297 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-10T11:04:48.297 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-10T11:04:48.297 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-10T11:04:48.297 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-10T11:04:48.297 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-10T11:04:48.297 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-10T11:04:48.297 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-10T11:04:48.297 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-10T11:04:48.297 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-10T11:04:48.297 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-10T11:04:48.297 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-10T11:04:48.297 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-10T11:04:48.297 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-10T11:04:48.298 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-10T11:04:48.298 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-10T11:04:48.298 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-10T11:04:48.298 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-10T11:04:48.298 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-10T11:04:48.298 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librados2-2:19.0.0-3449.gd39fa088.el9.x86_64 149/164 2024-05-10T11:04:48.298 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-10T11:04:48.298 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librbd1-2:19.0.0-3449.gd39fa088.el9.x86_64 151/164 2024-05-10T11:04:48.298 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-10T11:04:48.298 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-10T11:04:48.298 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-10T11:04:48.298 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-10T11:04:48.298 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-10T11:04:48.298 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-10T11:04:48.299 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-10T11:04:48.299 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-10T11:04:48.299 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-10T11:04:48.299 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-10T11:04:48.299 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-10T11:04:48.299 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-10T11:04:50.930 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-10T11:04:50.931 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-10T11:04:50.931 INFO:teuthology.orchestra.run.smithi188.stdout:Upgraded: 2024-05-10T11:04:50.931 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-10T11:04:50.931 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-10T11:04:50.931 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-10T11:04:50.931 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-10T11:04:50.931 INFO:teuthology.orchestra.run.smithi188.stdout: librados2-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:50.931 INFO:teuthology.orchestra.run.smithi188.stdout: librbd1-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:50.931 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-10T11:04:50.931 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-10T11:04:50.931 INFO:teuthology.orchestra.run.smithi188.stdout:Installed: 2024-05-10T11:04:50.931 INFO:teuthology.orchestra.run.smithi188.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-10T11:04:50.931 INFO:teuthology.orchestra.run.smithi188.stdout: automake-1.16.2-8.el9.noarch 2024-05-10T11:04:50.932 INFO:teuthology.orchestra.run.smithi188.stdout: bison-3.7.4-5.el9.x86_64 2024-05-10T11:04:50.932 INFO:teuthology.orchestra.run.smithi188.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-10T11:04:50.932 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:50.932 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-base-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:50.932 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-common-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:50.932 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-fuse-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:50.932 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-grafana-dashboards-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T11:04:50.932 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-immutable-object-cache-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:50.932 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mds-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:50.932 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:50.932 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-cephadm-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T11:04:50.932 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-dashboard-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T11:04:50.932 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T11:04:50.932 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-modules-core-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T11:04:50.933 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-rook-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T11:04:50.933 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mon-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:50.933 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-osd-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:50.933 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-prometheus-alerts-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T11:04:50.933 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-radosgw-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:50.933 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-selinux-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:50.933 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-test-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:50.933 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-volume-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T11:04:50.933 INFO:teuthology.orchestra.run.smithi188.stdout: cephadm-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T11:04:50.933 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-mirror-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:50.933 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-shell-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:50.933 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-top-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T11:04:50.933 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-10T11:04:50.933 INFO:teuthology.orchestra.run.smithi188.stdout: flex-2.6.4-9.el9.x86_64 2024-05-10T11:04:50.934 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-10T11:04:50.934 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-10T11:04:50.934 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-10T11:04:50.934 INFO:teuthology.orchestra.run.smithi188.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-10T11:04:50.934 INFO:teuthology.orchestra.run.smithi188.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-10T11:04:50.934 INFO:teuthology.orchestra.run.smithi188.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-10T11:04:50.934 INFO:teuthology.orchestra.run.smithi188.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-10T11:04:50.934 INFO:teuthology.orchestra.run.smithi188.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-10T11:04:50.934 INFO:teuthology.orchestra.run.smithi188.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-10T11:04:50.934 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs-devel-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:50.934 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs2-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:50.934 INFO:teuthology.orchestra.run.smithi188.stdout: libcephsqlite-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:50.934 INFO:teuthology.orchestra.run.smithi188.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-10T11:04:50.935 INFO:teuthology.orchestra.run.smithi188.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-10T11:04:50.935 INFO:teuthology.orchestra.run.smithi188.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-10T11:04:50.935 INFO:teuthology.orchestra.run.smithi188.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-10T11:04:50.935 INFO:teuthology.orchestra.run.smithi188.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-10T11:04:50.935 INFO:teuthology.orchestra.run.smithi188.stdout: librados-devel-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:50.935 INFO:teuthology.orchestra.run.smithi188.stdout: libradosstriper1-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:50.935 INFO:teuthology.orchestra.run.smithi188.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-10T11:04:50.935 INFO:teuthology.orchestra.run.smithi188.stdout: librgw2-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:50.935 INFO:teuthology.orchestra.run.smithi188.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-10T11:04:50.935 INFO:teuthology.orchestra.run.smithi188.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-10T11:04:50.935 INFO:teuthology.orchestra.run.smithi188.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-10T11:04:50.935 INFO:teuthology.orchestra.run.smithi188.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-10T11:04:50.935 INFO:teuthology.orchestra.run.smithi188.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-10T11:04:50.935 INFO:teuthology.orchestra.run.smithi188.stdout: lua-5.4.4-4.el9.x86_64 2024-05-10T11:04:50.936 INFO:teuthology.orchestra.run.smithi188.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-10T11:04:50.936 INFO:teuthology.orchestra.run.smithi188.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-10T11:04:50.936 INFO:teuthology.orchestra.run.smithi188.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-10T11:04:50.936 INFO:teuthology.orchestra.run.smithi188.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-10T11:04:50.936 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-10T11:04:50.936 INFO:teuthology.orchestra.run.smithi188.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-10T11:04:50.936 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-13.14-1.el9.x86_64 2024-05-10T11:04:50.936 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-10T11:04:50.936 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-10T11:04:50.936 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-10T11:04:50.936 INFO:teuthology.orchestra.run.smithi188.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-10T11:04:50.936 INFO:teuthology.orchestra.run.smithi188.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-10T11:04:50.936 INFO:teuthology.orchestra.run.smithi188.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-10T11:04:50.936 INFO:teuthology.orchestra.run.smithi188.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-10T11:04:50.937 INFO:teuthology.orchestra.run.smithi188.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-10T11:04:50.937 INFO:teuthology.orchestra.run.smithi188.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-10T11:04:50.937 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-10T11:04:50.937 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-argparse-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:50.937 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-common-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:50.937 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cephfs-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:50.937 INFO:teuthology.orchestra.run.smithi188.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-10T11:04:50.937 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-10T11:04:50.937 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-10T11:04:50.937 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-10T11:04:50.937 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-10T11:04:50.937 INFO:teuthology.orchestra.run.smithi188.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-10T11:04:50.937 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-10T11:04:50.937 INFO:teuthology.orchestra.run.smithi188.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-10T11:04:50.938 INFO:teuthology.orchestra.run.smithi188.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-10T11:04:50.938 INFO:teuthology.orchestra.run.smithi188.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-10T11:04:50.938 INFO:teuthology.orchestra.run.smithi188.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-10T11:04:50.938 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-10T11:04:50.938 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-10T11:04:50.938 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-10T11:04:50.938 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-10T11:04:50.938 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-10T11:04:50.938 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-10T11:04:50.938 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-10T11:04:50.938 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-10T11:04:50.938 INFO:teuthology.orchestra.run.smithi188.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-10T11:04:50.938 INFO:teuthology.orchestra.run.smithi188.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-10T11:04:50.938 INFO:teuthology.orchestra.run.smithi188.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-10T11:04:50.939 INFO:teuthology.orchestra.run.smithi188.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-10T11:04:50.939 INFO:teuthology.orchestra.run.smithi188.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-10T11:04:50.939 INFO:teuthology.orchestra.run.smithi188.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-10T11:04:50.939 INFO:teuthology.orchestra.run.smithi188.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-10T11:04:50.939 INFO:teuthology.orchestra.run.smithi188.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-10T11:04:50.939 INFO:teuthology.orchestra.run.smithi188.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-10T11:04:50.939 INFO:teuthology.orchestra.run.smithi188.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-10T11:04:50.939 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-10T11:04:50.939 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ply-3.11-14.el9.noarch 2024-05-10T11:04:50.939 INFO:teuthology.orchestra.run.smithi188.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-10T11:04:50.939 INFO:teuthology.orchestra.run.smithi188.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-10T11:04:50.939 INFO:teuthology.orchestra.run.smithi188.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-10T11:04:50.939 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-10T11:04:50.939 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-10T11:04:50.939 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-10T11:04:50.940 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-10T11:04:50.940 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-10T11:04:50.940 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-10T11:04:50.940 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rados-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:50.940 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rbd-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:50.940 INFO:teuthology.orchestra.run.smithi188.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-10T11:04:50.940 INFO:teuthology.orchestra.run.smithi188.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-10T11:04:50.940 INFO:teuthology.orchestra.run.smithi188.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-10T11:04:50.940 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rgw-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:50.940 INFO:teuthology.orchestra.run.smithi188.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-10T11:04:50.940 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-10T11:04:50.940 INFO:teuthology.orchestra.run.smithi188.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-10T11:04:50.940 INFO:teuthology.orchestra.run.smithi188.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-10T11:04:50.940 INFO:teuthology.orchestra.run.smithi188.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-10T11:04:50.941 INFO:teuthology.orchestra.run.smithi188.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-10T11:04:50.941 INFO:teuthology.orchestra.run.smithi188.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-10T11:04:50.941 INFO:teuthology.orchestra.run.smithi188.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-10T11:04:50.941 INFO:teuthology.orchestra.run.smithi188.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-10T11:04:50.941 INFO:teuthology.orchestra.run.smithi188.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-10T11:04:50.941 INFO:teuthology.orchestra.run.smithi188.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-10T11:04:50.941 INFO:teuthology.orchestra.run.smithi188.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-10T11:04:50.941 INFO:teuthology.orchestra.run.smithi188.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-10T11:04:50.941 INFO:teuthology.orchestra.run.smithi188.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-10T11:04:50.941 INFO:teuthology.orchestra.run.smithi188.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-10T11:04:50.941 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-fuse-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:50.941 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-mirror-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:50.941 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-nbd-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T11:04:50.941 INFO:teuthology.orchestra.run.smithi188.stdout: re2-1:20211101-3.el9.x86_64 2024-05-10T11:04:50.941 INFO:teuthology.orchestra.run.smithi188.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-10T11:04:50.942 INFO:teuthology.orchestra.run.smithi188.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-10T11:04:50.942 INFO:teuthology.orchestra.run.smithi188.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-10T11:04:50.942 INFO:teuthology.orchestra.run.smithi188.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-10T11:04:50.942 INFO:teuthology.orchestra.run.smithi188.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-10T11:04:50.942 INFO:teuthology.orchestra.run.smithi188.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-10T11:04:50.942 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-10T11:04:50.942 INFO:teuthology.orchestra.run.smithi188.stdout:Complete! 2024-05-10T11:04:51.363 DEBUG:teuthology.parallel:result is None 2024-05-10T11:04:51.364 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d39fa08867836ac070c47cb66c782fa559fd480b 2024-05-10T11:04:51.585 DEBUG:teuthology.orchestra.run.smithi100:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-10T11:04:51.614 INFO:teuthology.orchestra.run.smithi100.stdout:19.0.0-3449.gd39fa088.el9 2024-05-10T11:04:51.615 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3449.gd39fa088.el9 2024-05-10T11:04:51.615 INFO:teuthology.task.install:The correct ceph version 19.0.0-3449.gd39fa088 is installed. 2024-05-10T11:04:51.617 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d39fa08867836ac070c47cb66c782fa559fd480b 2024-05-10T11:04:51.757 DEBUG:teuthology.orchestra.run.smithi178:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-10T11:04:51.786 INFO:teuthology.orchestra.run.smithi178.stdout:19.0.0-3449.gd39fa088.el9 2024-05-10T11:04:51.786 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3449.gd39fa088.el9 2024-05-10T11:04:51.786 INFO:teuthology.task.install:The correct ceph version 19.0.0-3449.gd39fa088 is installed. 2024-05-10T11:04:51.788 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d39fa08867836ac070c47cb66c782fa559fd480b 2024-05-10T11:04:51.999 DEBUG:teuthology.orchestra.run.smithi188:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-10T11:04:52.025 INFO:teuthology.orchestra.run.smithi188.stdout:19.0.0-3449.gd39fa088.el9 2024-05-10T11:04:52.025 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3449.gd39fa088.el9 2024-05-10T11:04:52.025 INFO:teuthology.task.install:The correct ceph version 19.0.0-3449.gd39fa088 is installed. 2024-05-10T11:04:52.027 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-10T11:04:52.027 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-05-10T11:04:52.028 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-10T11:04:52.061 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-05-10T11:04:52.061 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-10T11:04:52.094 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-05-10T11:04:52.094 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-10T11:04:52.125 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-10T11:04:52.125 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-05-10T11:04:52.126 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/daemon-helper 2024-05-10T11:04:52.155 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-10T11:04:52.222 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-05-10T11:04:52.222 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/usr/bin/daemon-helper 2024-05-10T11:04:52.253 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-10T11:04:52.322 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-05-10T11:04:52.322 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/usr/bin/daemon-helper 2024-05-10T11:04:52.354 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-10T11:04:52.424 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-10T11:04:52.425 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-05-10T11:04:52.425 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-10T11:04:52.455 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-10T11:04:52.523 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-05-10T11:04:52.524 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-10T11:04:52.554 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-10T11:04:52.620 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-05-10T11:04:52.620 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-10T11:04:52.650 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-10T11:04:52.719 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-10T11:04:52.719 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-05-10T11:04:52.720 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/stdin-killer 2024-05-10T11:04:52.745 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-10T11:04:52.811 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-05-10T11:04:52.811 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/usr/bin/stdin-killer 2024-05-10T11:04:52.836 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-10T11:04:52.901 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-05-10T11:04:52.901 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/usr/bin/stdin-killer 2024-05-10T11:04:52.931 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-10T11:04:52.999 INFO:teuthology.run_tasks:Running task cephadm... 2024-05-10T11:04:53.107 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'd39fa08867836ac070c47cb66c782fa559fd480b', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-05-10T11:04:53.108 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b 2024-05-10T11:04:53.108 INFO:tasks.cephadm:Cluster fsid is 20a542cc-0ebd-11ef-bc98-c7b262605968 2024-05-10T11:04:53.108 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-05-10T11:04:53.108 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.100', 'mon.b': '172.21.15.178', 'mon.c': '172.21.15.188'} 2024-05-10T11:04:53.108 INFO:tasks.cephadm:First mon is mon.a on smithi100 2024-05-10T11:04:53.109 INFO:tasks.cephadm:First mgr is x 2024-05-10T11:04:53.109 INFO:tasks.cephadm:Normalizing hostnames... 2024-05-10T11:04:53.109 DEBUG:teuthology.orchestra.run.smithi100:> sudo hostname $(hostname -s) 2024-05-10T11:04:53.134 DEBUG:teuthology.orchestra.run.smithi178:> sudo hostname $(hostname -s) 2024-05-10T11:04:53.160 DEBUG:teuthology.orchestra.run.smithi188:> sudo hostname $(hostname -s) 2024-05-10T11:04:53.190 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-05-10T11:04:53.190 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d39fa08867836ac070c47cb66c782fa559fd480b 2024-05-10T11:04:53.412 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'd39fa08867836ac070c47cb66c782fa559fd480b', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/79273/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3449-gd39fa088', 'node_name': '172.21.2.1+braggi01', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3449.gd39fa088'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-63896-from-wip-rishabh-testing-20240501.193033/d39fa08867836ac070c47cb66c782fa559fd480b/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-10 00:52:03.112070', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-63896-from-wip-rishabh-testing-20240501.193033', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-63896-from-wip-rishabh-testing-20240501.193033/d39fa08867836ac070c47cb66c782fa559fd480b/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-05-10T11:04:53.633 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-63896-from-wip-rishabh-testing-20240501.193033, sha1 d39fa08867836ac070c47cb66c782fa559fd480b from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=d39fa08867836ac070c47cb66c782fa559fd480b 2024-05-10T11:04:53.635 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-63896-from-wip-rishabh-testing-20240501.193033/d39fa08867836ac070c47cb66c782fa559fd480b/centos/9/x86_64/flavors/default/cephadm 2024-05-10T11:04:53.635 DEBUG:teuthology.orchestra.run.smithi100:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-63896-from-wip-rishabh-testing-20240501.193033/d39fa08867836ac070c47cb66c782fa559fd480b/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-10T11:04:54.136 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r--. 1 ubuntu ubuntu 782738 May 10 11:04 /home/ubuntu/cephtest/cephadm 2024-05-10T11:04:54.137 DEBUG:teuthology.orchestra.run.smithi178:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-63896-from-wip-rishabh-testing-20240501.193033/d39fa08867836ac070c47cb66c782fa559fd480b/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-10T11:04:54.633 INFO:teuthology.orchestra.run.smithi178.stdout:-rw-r--r--. 1 ubuntu ubuntu 782738 May 10 11:04 /home/ubuntu/cephtest/cephadm 2024-05-10T11:04:54.633 DEBUG:teuthology.orchestra.run.smithi188:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-63896-from-wip-rishabh-testing-20240501.193033/d39fa08867836ac070c47cb66c782fa559fd480b/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-10T11:04:55.004 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r--. 1 ubuntu ubuntu 782738 May 10 11:04 /home/ubuntu/cephtest/cephadm 2024-05-10T11:04:55.005 DEBUG:teuthology.orchestra.run.smithi100:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-10T11:04:55.024 DEBUG:teuthology.orchestra.run.smithi178:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-10T11:04:55.044 DEBUG:teuthology.orchestra.run.smithi188:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-10T11:04:55.079 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b on all hosts... 2024-05-10T11:04:55.079 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b pull 2024-05-10T11:04:55.081 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b pull 2024-05-10T11:04:55.087 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b pull 2024-05-10T11:04:56.678 INFO:teuthology.orchestra.run.smithi100.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b... 2024-05-10T11:04:56.686 INFO:teuthology.orchestra.run.smithi188.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b... 2024-05-10T11:04:56.736 INFO:teuthology.orchestra.run.smithi178.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b... 2024-05-10T11:05:36.794 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:05:36.794 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_version": "ceph version 19.0.0-3449-gd39fa088 (d39fa08867836ac070c47cb66c782fa559fd480b) squid (dev)", 2024-05-10T11:05:36.794 INFO:teuthology.orchestra.run.smithi100.stdout: "image_id": "ee5e4f21ef58047cb638cccd5cfbe8f6f43412b56c94b1d7fbd9cf3cb90a53a5", 2024-05-10T11:05:36.795 INFO:teuthology.orchestra.run.smithi100.stdout: "repo_digests": [ 2024-05-10T11:05:36.795 INFO:teuthology.orchestra.run.smithi100.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f" 2024-05-10T11:05:36.795 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-05-10T11:05:36.795 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:05:42.610 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-10T11:05:42.610 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_version": "ceph version 19.0.0-3449-gd39fa088 (d39fa08867836ac070c47cb66c782fa559fd480b) squid (dev)", 2024-05-10T11:05:42.610 INFO:teuthology.orchestra.run.smithi188.stdout: "image_id": "ee5e4f21ef58047cb638cccd5cfbe8f6f43412b56c94b1d7fbd9cf3cb90a53a5", 2024-05-10T11:05:42.610 INFO:teuthology.orchestra.run.smithi188.stdout: "repo_digests": [ 2024-05-10T11:05:42.611 INFO:teuthology.orchestra.run.smithi188.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f" 2024-05-10T11:05:42.611 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-10T11:05:42.611 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-10T11:05:46.397 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-05-10T11:05:46.398 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_version": "ceph version 19.0.0-3449-gd39fa088 (d39fa08867836ac070c47cb66c782fa559fd480b) squid (dev)", 2024-05-10T11:05:46.398 INFO:teuthology.orchestra.run.smithi178.stdout: "image_id": "ee5e4f21ef58047cb638cccd5cfbe8f6f43412b56c94b1d7fbd9cf3cb90a53a5", 2024-05-10T11:05:46.398 INFO:teuthology.orchestra.run.smithi178.stdout: "repo_digests": [ 2024-05-10T11:05:46.398 INFO:teuthology.orchestra.run.smithi178.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f" 2024-05-10T11:05:46.398 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-05-10T11:05:46.398 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-05-10T11:05:46.423 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /etc/ceph 2024-05-10T11:05:46.457 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /etc/ceph 2024-05-10T11:05:46.484 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /etc/ceph 2024-05-10T11:05:46.517 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 777 /etc/ceph 2024-05-10T11:05:46.548 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 777 /etc/ceph 2024-05-10T11:05:46.573 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 777 /etc/ceph 2024-05-10T11:05:46.601 INFO:tasks.cephadm:Writing seed config... 2024-05-10T11:05:46.602 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-05-10T11:05:46.602 INFO:tasks.cephadm: override: [client] debug client = 20 2024-05-10T11:05:46.603 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-05-10T11:05:46.603 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-05-10T11:05:46.603 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-05-10T11:05:46.603 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-05-10T11:05:46.603 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-05-10T11:05:46.603 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-05-10T11:05:46.603 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-05-10T11:05:46.603 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-05-10T11:05:46.603 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-05-10T11:05:46.603 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-05-10T11:05:46.603 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-05-10T11:05:46.603 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-05-10T11:05:46.603 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-05-10T11:05:46.603 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-05-10T11:05:46.603 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-05-10T11:05:46.604 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-05-10T11:05:46.604 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-05-10T11:05:46.604 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-05-10T11:05:46.604 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-05-10T11:05:46.604 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-05-10T11:05:46.604 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-05-10T11:05:46.604 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-05-10T11:05:46.604 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-05-10T11:05:46.604 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-05-10T11:05:46.604 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-05-10T11:05:46.604 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-05-10T11:05:46.604 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-05-10T11:05:46.604 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-05-10T11:05:46.604 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-05-10T11:05:46.605 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-05-10T11:05:46.605 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-05-10T11:05:46.605 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-05-10T11:05:46.605 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-05-10T11:05:46.605 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-05-10T11:05:46.605 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-05-10T11:05:46.605 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-05-10T11:05:46.606 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-05-10T11:05:46.606 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-05-10T11:05:46.606 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-05-10T11:05:46.606 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-05-10T11:05:46.606 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-05-10T11:05:46.607 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-05-10T11:05:46.607 DEBUG:teuthology.orchestra.run.smithi100:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-05-10T11:05:46.659 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 20a542cc-0ebd-11ef-bc98-c7b262605968 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-05-10T11:05:46.659 DEBUG:teuthology.orchestra.run.smithi100:mon.a> sudo journalctl -f -n 0 -u ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mon.a.service 2024-05-10T11:05:46.702 DEBUG:teuthology.orchestra.run.smithi100:mgr.x> sudo journalctl -f -n 0 -u ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mgr.x.service 2024-05-10T11:05:46.745 INFO:tasks.cephadm:Bootstrapping... 2024-05-10T11:05:46.745 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b -v bootstrap --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.100 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-05-10T11:05:46.943 INFO:teuthology.orchestra.run.smithi100.stdout:-------------------------------------------------------------------------------- 2024-05-10T11:05:46.944 INFO:teuthology.orchestra.run.smithi100.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b', '-v', 'bootstrap', '--fsid', '20a542cc-0ebd-11ef-bc98-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.100', '--skip-admin-label'] 2024-05-10T11:05:46.944 INFO:teuthology.orchestra.run.smithi100.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-05-10T11:05:46.944 INFO:teuthology.orchestra.run.smithi100.stdout:Verifying podman|docker is present... 2024-05-10T11:05:46.970 INFO:teuthology.orchestra.run.smithi100.stdout:/bin/podman: stdout 4.9.4-dev 2024-05-10T11:05:46.971 INFO:teuthology.orchestra.run.smithi100.stdout:Verifying lvm2 is present... 2024-05-10T11:05:46.971 INFO:teuthology.orchestra.run.smithi100.stdout:Verifying time synchronization is in place... 2024-05-10T11:05:46.976 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-10T11:05:46.977 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-10T11:05:46.983 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-10T11:05:46.983 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout inactive 2024-05-10T11:05:46.989 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout enabled 2024-05-10T11:05:46.997 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout active 2024-05-10T11:05:46.997 INFO:teuthology.orchestra.run.smithi100.stdout:Unit chronyd.service is enabled and running 2024-05-10T11:05:46.997 INFO:teuthology.orchestra.run.smithi100.stdout:Repeating the final host check... 2024-05-10T11:05:47.020 INFO:teuthology.orchestra.run.smithi100.stdout:/bin/podman: stdout 4.9.4-dev 2024-05-10T11:05:47.020 INFO:teuthology.orchestra.run.smithi100.stdout:podman (/bin/podman) version 4.9.4 is present 2024-05-10T11:05:47.020 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl is present 2024-05-10T11:05:47.020 INFO:teuthology.orchestra.run.smithi100.stdout:lvcreate is present 2024-05-10T11:05:47.026 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-10T11:05:47.026 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-10T11:05:47.032 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-10T11:05:47.032 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout inactive 2024-05-10T11:05:47.039 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout enabled 2024-05-10T11:05:47.045 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout active 2024-05-10T11:05:47.045 INFO:teuthology.orchestra.run.smithi100.stdout:Unit chronyd.service is enabled and running 2024-05-10T11:05:47.046 INFO:teuthology.orchestra.run.smithi100.stdout:Host looks OK 2024-05-10T11:05:47.046 INFO:teuthology.orchestra.run.smithi100.stdout:Cluster fsid: 20a542cc-0ebd-11ef-bc98-c7b262605968 2024-05-10T11:05:47.046 INFO:teuthology.orchestra.run.smithi100.stdout:Acquiring lock 140214404054464 on /run/cephadm/20a542cc-0ebd-11ef-bc98-c7b262605968.lock 2024-05-10T11:05:47.046 INFO:teuthology.orchestra.run.smithi100.stdout:Lock 140214404054464 acquired on /run/cephadm/20a542cc-0ebd-11ef-bc98-c7b262605968.lock 2024-05-10T11:05:47.046 INFO:teuthology.orchestra.run.smithi100.stdout:Verifying IP 172.21.15.100 port 3300 ... 2024-05-10T11:05:47.046 INFO:teuthology.orchestra.run.smithi100.stdout:Verifying IP 172.21.15.100 port 6789 ... 2024-05-10T11:05:47.046 INFO:teuthology.orchestra.run.smithi100.stdout:Base mon IP(s) is [172.21.15.100:3300, 172.21.15.100:6789], mon addrv is [v2:172.21.15.100:3300,v1:172.21.15.100:6789] 2024-05-10T11:05:47.050 INFO:teuthology.orchestra.run.smithi100.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.100 metric 100 2024-05-10T11:05:47.050 INFO:teuthology.orchestra.run.smithi100.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.100 metric 100 2024-05-10T11:05:47.052 INFO:teuthology.orchestra.run.smithi100.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-05-10T11:05:47.052 INFO:teuthology.orchestra.run.smithi100.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-05-10T11:05:47.052 INFO:teuthology.orchestra.run.smithi100.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1767sec hoplimit 64 pref medium 2024-05-10T11:05:47.055 INFO:teuthology.orchestra.run.smithi100.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-05-10T11:05:47.055 INFO:teuthology.orchestra.run.smithi100.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-05-10T11:05:47.055 INFO:teuthology.orchestra.run.smithi100.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-10T11:05:47.055 INFO:teuthology.orchestra.run.smithi100.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-05-10T11:05:47.055 INFO:teuthology.orchestra.run.smithi100.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d733/64 scope link 2024-05-10T11:05:47.055 INFO:teuthology.orchestra.run.smithi100.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-10T11:05:47.056 INFO:teuthology.orchestra.run.smithi100.stdout:Mon IP `172.21.15.100` is in CIDR network `172.21.0.0/20` 2024-05-10T11:05:47.056 INFO:teuthology.orchestra.run.smithi100.stdout:Mon IP `172.21.15.100` is in CIDR network `172.21.0.0/20` 2024-05-10T11:05:47.056 INFO:teuthology.orchestra.run.smithi100.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-05-10T11:05:47.056 INFO:teuthology.orchestra.run.smithi100.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-05-10T11:05:47.057 INFO:teuthology.orchestra.run.smithi100.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b... 2024-05-10T11:05:50.625 INFO:teuthology.orchestra.run.smithi100.stdout:/bin/podman: stdout ee5e4f21ef58047cb638cccd5cfbe8f6f43412b56c94b1d7fbd9cf3cb90a53a5 2024-05-10T11:05:50.625 INFO:teuthology.orchestra.run.smithi100.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b... 2024-05-10T11:05:50.625 INFO:teuthology.orchestra.run.smithi100.stdout:/bin/podman: stderr Getting image source signatures 2024-05-10T11:05:50.626 INFO:teuthology.orchestra.run.smithi100.stdout:/bin/podman: stderr Copying blob sha256:5401d9dfdf7b5b1eb30ca8a50398217fe5e68f9c8026e432d9728743b160edef 2024-05-10T11:05:50.626 INFO:teuthology.orchestra.run.smithi100.stdout:/bin/podman: stderr Copying blob sha256:bf5b95bd68251da39c6cd2eb96c6ba4138bb37a62bdc39287908b25e92047d7e 2024-05-10T11:05:50.626 INFO:teuthology.orchestra.run.smithi100.stdout:/bin/podman: stderr Copying config sha256:ee5e4f21ef58047cb638cccd5cfbe8f6f43412b56c94b1d7fbd9cf3cb90a53a5 2024-05-10T11:05:50.626 INFO:teuthology.orchestra.run.smithi100.stdout:/bin/podman: stderr Writing manifest to image destination 2024-05-10T11:05:57.149 INFO:teuthology.orchestra.run.smithi100.stdout:ceph: stdout ceph version 19.0.0-3449-gd39fa088 (d39fa08867836ac070c47cb66c782fa559fd480b) squid (dev) 2024-05-10T11:05:57.149 INFO:teuthology.orchestra.run.smithi100.stdout:Ceph version: ceph version 19.0.0-3449-gd39fa088 (d39fa08867836ac070c47cb66c782fa559fd480b) squid (dev) 2024-05-10T11:05:57.149 INFO:teuthology.orchestra.run.smithi100.stdout:Extracting ceph user uid/gid from container image... 2024-05-10T11:05:58.254 INFO:teuthology.orchestra.run.smithi100.stdout:stat: stdout 167 167 2024-05-10T11:05:58.254 INFO:teuthology.orchestra.run.smithi100.stdout:Creating initial keys... 2024-05-10T11:05:59.259 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-authtool: stdout AQCW/z1m/VcNIhAAJ73m4uxSWXzwV5F7y3dnsw== 2024-05-10T11:06:00.164 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-authtool: stdout AQCX/z1mTtPqIhAAoDlzT4rfosiHmOCIxu6f6Q== 2024-05-10T11:06:01.011 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-authtool: stdout AQCY/z1mfblQHRAAdCRT88WdeuQZyzmAF/HOIQ== 2024-05-10T11:06:01.012 INFO:teuthology.orchestra.run.smithi100.stdout:Creating initial monmap... 2024-05-10T11:06:01.884 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-10T11:06:01.885 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-05-10T11:06:01.885 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 20a542cc-0ebd-11ef-bc98-c7b262605968 2024-05-10T11:06:01.885 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-10T11:06:01.885 INFO:teuthology.orchestra.run.smithi100.stdout:monmaptool for a [v2:172.21.15.100:3300,v1:172.21.15.100:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-10T11:06:01.885 INFO:teuthology.orchestra.run.smithi100.stdout:setting min_mon_release = quincy 2024-05-10T11:06:01.885 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/monmaptool: set fsid to 20a542cc-0ebd-11ef-bc98-c7b262605968 2024-05-10T11:06:01.885 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-10T11:06:01.885 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:06:01.885 INFO:teuthology.orchestra.run.smithi100.stdout:Creating mon... 2024-05-10T11:06:03.997 INFO:teuthology.orchestra.run.smithi100.stdout:create mon.a on 2024-05-10T11:06:04.230 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-05-10T11:06:04.380 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-05-10T11:06:04.549 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-20a542cc-0ebd-11ef-bc98-c7b262605968.target → /etc/systemd/system/ceph-20a542cc-0ebd-11ef-bc98-c7b262605968.target. 2024-05-10T11:06:04.549 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-20a542cc-0ebd-11ef-bc98-c7b262605968.target → /etc/systemd/system/ceph-20a542cc-0ebd-11ef-bc98-c7b262605968.target. 2024-05-10T11:06:04.810 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mon.a 2024-05-10T11:06:04.811 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stderr Failed to reset failed state of unit ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mon.a.service: Unit ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mon.a.service not loaded. 2024-05-10T11:06:04.984 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-20a542cc-0ebd-11ef-bc98-c7b262605968.target.wants/ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mon.a.service → /etc/systemd/system/ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@.service. 2024-05-10T11:06:05.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:05 smithi100 podman[29908]: 2024-05-10 11:06:05.125868704 +0000 UTC m=+0.017993301 image pull ee5e4f21ef58047cb638cccd5cfbe8f6f43412b56c94b1d7fbd9cf3cb90a53a5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b 2024-05-10T11:06:05.505 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-10T11:06:05.506 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout disabled 2024-05-10T11:06:05.512 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-10T11:06:05.513 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout inactive 2024-05-10T11:06:05.513 INFO:teuthology.orchestra.run.smithi100.stdout:firewalld.service is not enabled 2024-05-10T11:06:05.513 INFO:teuthology.orchestra.run.smithi100.stdout:Not possible to enable service . firewalld.service is not available 2024-05-10T11:06:05.513 INFO:teuthology.orchestra.run.smithi100.stdout:Waiting for mon to start... 2024-05-10T11:06:05.513 INFO:teuthology.orchestra.run.smithi100.stdout:Waiting for mon... 2024-05-10T11:06:05.638 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:05 smithi100 podman[29908]: 2024-05-10 11:06:05.408453904 +0000 UTC m=+0.300578486 container init aec1e48d3a833a55f49da4873f9c01b02cbd2f4167a9645ad03a398023b0ecbb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mon-a, GIT_BRANCH=HEAD, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-10T11:06:05.638 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:05 smithi100 podman[29908]: 2024-05-10 11:06:05.411740649 +0000 UTC m=+0.303865231 container start aec1e48d3a833a55f49da4873f9c01b02cbd2f4167a9645ad03a398023b0ecbb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mon-a, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.build-date=20240507, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-05-10T11:06:05.639 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:05 smithi100 ceph-mon[29927]: mkfs 20a542cc-0ebd-11ef-bc98-c7b262605968 2024-05-10T11:06:05.639 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:05 smithi100 ceph-mon[29927]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-10T11:06:05.639 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:05 smithi100 bash[29908]: aec1e48d3a833a55f49da4873f9c01b02cbd2f4167a9645ad03a398023b0ecbb 2024-05-10T11:06:05.639 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:05 smithi100 systemd[1]: Started Ceph mon.a for 20a542cc-0ebd-11ef-bc98-c7b262605968. 2024-05-10T11:06:06.531 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout cluster: 2024-05-10T11:06:06.531 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout id: 20a542cc-0ebd-11ef-bc98-c7b262605968 2024-05-10T11:06:06.531 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-05-10T11:06:06.531 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-05-10T11:06:06.532 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout services: 2024-05-10T11:06:06.532 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.567281s) 2024-05-10T11:06:06.532 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-05-10T11:06:06.532 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-05-10T11:06:06.532 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-05-10T11:06:06.532 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout data: 2024-05-10T11:06:06.532 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-05-10T11:06:06.532 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-05-10T11:06:06.532 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-05-10T11:06:06.532 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout pgs: 2024-05-10T11:06:06.532 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-05-10T11:06:06.532 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:05.942+0000 7f1f6508e700 1 Processor -- start 2024-05-10T11:06:06.532 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:05.942+0000 7f1f6508e700 1 -- start start 2024-05-10T11:06:06.533 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:05.942+0000 7f1f6508e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f60107f60 0x7f1f60108380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:06:06.533 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:05.942+0000 7f1f6508e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1f60108950 con 0x7f1f60107f60 2024-05-10T11:06:06.533 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:05.942+0000 7f1f5ed9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f60107f60 0x7f1f60108380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:06:06.533 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:05.942+0000 7f1f5ed9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f60107f60 0x7f1f60108380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42532/0 (socket says 172.21.15.100:42532) 2024-05-10T11:06:06.533 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:05.942+0000 7f1f5ed9d700 1 -- 172.21.15.100:0/2078561980 learned_addr learned my addr 172.21.15.100:0/2078561980 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:06:06.533 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:05.942+0000 7f1f5ed9d700 1 -- 172.21.15.100:0/2078561980 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f60108c80 con 0x7f1f60107f60 2024-05-10T11:06:06.533 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:05.943+0000 7f1f5ed9d700 1 --2- 172.21.15.100:0/2078561980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f60107f60 0x7f1f60108380 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1f500098a0 tx=0x7f1f50009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e5657c173dec9cc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:06:06.533 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:05.943+0000 7f1f5e59c700 1 -- 172.21.15.100:0/2078561980 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f50004160 con 0x7f1f60107f60 2024-05-10T11:06:06.533 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:05.943+0000 7f1f5e59c700 1 -- 172.21.15.100:0/2078561980 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f50004300 con 0x7f1f60107f60 2024-05-10T11:06:06.533 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:05.943+0000 7f1f5e59c700 1 -- 172.21.15.100:0/2078561980 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f500036a0 con 0x7f1f60107f60 2024-05-10T11:06:06.533 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:05.943+0000 7f1f6508e700 1 -- 172.21.15.100:0/2078561980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f60107f60 msgr2=0x7f1f60108380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:06:06.533 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:05.943+0000 7f1f6508e700 1 --2- 172.21.15.100:0/2078561980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f60107f60 0x7f1f60108380 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1f500098a0 tx=0x7f1f50009bb0 comp rx=0 tx=0).stop 2024-05-10T11:06:06.533 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:05.943+0000 7f1f6508e700 1 -- 172.21.15.100:0/2078561980 shutdown_connections 2024-05-10T11:06:06.533 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:05.943+0000 7f1f6508e700 1 --2- 172.21.15.100:0/2078561980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f60107f60 0x7f1f60108380 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:06:06.533 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:05.943+0000 7f1f6508e700 1 -- 172.21.15.100:0/2078561980 >> 172.21.15.100:0/2078561980 conn(0x7f1f601027a0 msgr2=0x7f1f60104bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:06:06.534 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:05.943+0000 7f1f6508e700 1 -- 172.21.15.100:0/2078561980 shutdown_connections 2024-05-10T11:06:06.534 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:05.943+0000 7f1f6508e700 1 -- 172.21.15.100:0/2078561980 wait complete. 2024-05-10T11:06:06.534 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:05.944+0000 7f1f6508e700 1 Processor -- start 2024-05-10T11:06:06.534 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:05.944+0000 7f1f6508e700 1 -- start start 2024-05-10T11:06:06.534 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:05.944+0000 7f1f6508e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f60107f60 0x7f1f6019b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:06:06.534 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:05.944+0000 7f1f6508e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1f601091b0 con 0x7f1f60107f60 2024-05-10T11:06:06.534 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:05.945+0000 7f1f5ed9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f60107f60 0x7f1f6019b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:06:06.534 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:05.945+0000 7f1f5ed9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f60107f60 0x7f1f6019b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42534/0 (socket says 172.21.15.100:42534) 2024-05-10T11:06:06.534 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:05.945+0000 7f1f5ed9d700 1 -- 172.21.15.100:0/571587946 learned_addr learned my addr 172.21.15.100:0/571587946 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:06:06.534 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:05.945+0000 7f1f5ed9d700 1 -- 172.21.15.100:0/571587946 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f6019b640 con 0x7f1f60107f60 2024-05-10T11:06:06.534 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:05.945+0000 7f1f5ed9d700 1 --2- 172.21.15.100:0/571587946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f60107f60 0x7f1f6019b100 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1f50000c00 tx=0x7f1f50004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:06:06.534 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:05.945+0000 7f1f5cd99700 1 -- 172.21.15.100:0/571587946 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f50003db0 con 0x7f1f60107f60 2024-05-10T11:06:06.534 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:05.945+0000 7f1f5cd99700 1 -- 172.21.15.100:0/571587946 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f50011630 con 0x7f1f60107f60 2024-05-10T11:06:06.534 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:05.945+0000 7f1f6508e700 1 -- 172.21.15.100:0/571587946 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f6019b8d0 con 0x7f1f60107f60 2024-05-10T11:06:06.535 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:05.945+0000 7f1f5cd99700 1 -- 172.21.15.100:0/571587946 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f50011950 con 0x7f1f60107f60 2024-05-10T11:06:06.535 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:05.945+0000 7f1f6508e700 1 -- 172.21.15.100:0/571587946 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f6019e5c0 con 0x7f1f60107f60 2024-05-10T11:06:06.535 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:05.945+0000 7f1f5cd99700 1 -- 172.21.15.100:0/571587946 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1f50011af0 con 0x7f1f60107f60 2024-05-10T11:06:06.535 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:05.945+0000 7f1f5cd99700 1 -- 172.21.15.100:0/571587946 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1f5001c450 con 0x7f1f60107f60 2024-05-10T11:06:06.535 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:05.946+0000 7f1f6508e700 1 -- 172.21.15.100:0/571587946 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f60064b00 con 0x7f1f60107f60 2024-05-10T11:06:06.535 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:05.948+0000 7f1f5cd99700 1 -- 172.21.15.100:0/571587946 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f1f50004260 con 0x7f1f60107f60 2024-05-10T11:06:06.535 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:06.018+0000 7f1f6508e700 1 -- 172.21.15.100:0/571587946 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f1f6002e720 con 0x7f1f60107f60 2024-05-10T11:06:06.535 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:06.019+0000 7f1f5cd99700 1 -- 172.21.15.100:0/571587946 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f1f5001b450 con 0x7f1f60107f60 2024-05-10T11:06:06.535 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:06.019+0000 7f1f6508e700 1 -- 172.21.15.100:0/571587946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f60107f60 msgr2=0x7f1f6019b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:06:06.535 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:06.019+0000 7f1f6508e700 1 --2- 172.21.15.100:0/571587946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f60107f60 0x7f1f6019b100 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1f50000c00 tx=0x7f1f50004750 comp rx=0 tx=0).stop 2024-05-10T11:06:06.535 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:06.019+0000 7f1f6508e700 1 -- 172.21.15.100:0/571587946 shutdown_connections 2024-05-10T11:06:06.535 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:06.019+0000 7f1f6508e700 1 --2- 172.21.15.100:0/571587946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f60107f60 0x7f1f6019b100 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:06:06.535 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:06.019+0000 7f1f6508e700 1 -- 172.21.15.100:0/571587946 >> 172.21.15.100:0/571587946 conn(0x7f1f601027a0 msgr2=0x7f1f60191d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:06:06.535 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:06.019+0000 7f1f6508e700 1 -- 172.21.15.100:0/571587946 shutdown_connections 2024-05-10T11:06:06.536 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:06.019+0000 7f1f6508e700 1 -- 172.21.15.100:0/571587946 wait complete. 2024-05-10T11:06:06.536 INFO:teuthology.orchestra.run.smithi100.stdout:mon is available 2024-05-10T11:06:06.536 INFO:teuthology.orchestra.run.smithi100.stdout:Assimilating anything we can from ceph.conf... 2024-05-10T11:06:06.632 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:06 smithi100 ceph-mon[29927]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-10T11:06:06.632 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:06 smithi100 ceph-mon[29927]: monmap epoch 1 2024-05-10T11:06:06.632 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:06 smithi100 ceph-mon[29927]: fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 2024-05-10T11:06:06.632 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:06 smithi100 ceph-mon[29927]: last_changed 2024-05-10T11:06:01.369875+0000 2024-05-10T11:06:06.632 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:06 smithi100 ceph-mon[29927]: created 2024-05-10T11:06:01.369875+0000 2024-05-10T11:06:06.632 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:06 smithi100 ceph-mon[29927]: min_mon_release 19 (squid) 2024-05-10T11:06:06.632 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:06 smithi100 ceph-mon[29927]: election_strategy: 1 2024-05-10T11:06:06.633 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:06 smithi100 ceph-mon[29927]: 0: [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] mon.a 2024-05-10T11:06:06.633 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:06 smithi100 ceph-mon[29927]: fsmap 2024-05-10T11:06:06.633 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:06 smithi100 ceph-mon[29927]: osdmap e1: 0 total, 0 up, 0 in 2024-05-10T11:06:06.633 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:06 smithi100 ceph-mon[29927]: mgrmap e1: no daemons active 2024-05-10T11:06:06.633 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:06 smithi100 ceph-mon[29927]: from='client.? 172.21.15.100:0/571587946' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-05-10T11:06:07.721 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:07 smithi100 ceph-mon[29927]: from='client.? 172.21.15.100:0/2122077720' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-10T11:06:07.721 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:07 smithi100 ceph-mon[29927]: from='client.? 172.21.15.100:0/2122077720' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-05-10T11:06:07.730 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-05-10T11:06:07.730 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout [global] 2024-05-10T11:06:07.730 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout fsid = 20a542cc-0ebd-11ef-bc98-c7b262605968 2024-05-10T11:06:07.730 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-10T11:06:07.730 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.100:3300,v1:172.21.15.100:6789] 2024-05-10T11:06:07.731 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-10T11:06:07.731 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-10T11:06:07.731 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-10T11:06:07.731 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-10T11:06:07.731 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-05-10T11:06:07.731 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-10T11:06:07.731 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-10T11:06:07.731 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-05-10T11:06:07.731 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout [osd] 2024-05-10T11:06:07.731 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-10T11:06:07.731 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-10T11:06:07.731 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-10T11:06:07.731 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-10T11:06:07.732 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-10T11:06:07.732 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:06.939+0000 7f9aca389700 1 Processor -- start 2024-05-10T11:06:07.732 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:06.939+0000 7f9aca389700 1 -- start start 2024-05-10T11:06:07.732 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:06.939+0000 7f9aca389700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ac41095f0 0x7f9ac410b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:06:07.732 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:06.939+0000 7f9aca389700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac407cde0 con 0x7f9ac41095f0 2024-05-10T11:06:07.732 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:06.940+0000 7f9ac3fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ac41095f0 0x7f9ac410b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:06:07.732 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:06.940+0000 7f9ac3fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ac41095f0 0x7f9ac410b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44742/0 (socket says 172.21.15.100:44742) 2024-05-10T11:06:07.732 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:06.940+0000 7f9ac3fff700 1 -- 172.21.15.100:0/3955789380 learned_addr learned my addr 172.21.15.100:0/3955789380 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:06:07.732 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:06.940+0000 7f9ac3fff700 1 -- 172.21.15.100:0/3955789380 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ac410bf20 con 0x7f9ac41095f0 2024-05-10T11:06:07.732 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:06.940+0000 7f9ac3fff700 1 --2- 172.21.15.100:0/3955789380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ac41095f0 0x7f9ac410b9e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f9ab40098a0 tx=0x7f9ab4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e04039beb020e4ba server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:06:07.732 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:06.940+0000 7f9ac2ffd700 1 -- 172.21.15.100:0/3955789380 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ab4004160 con 0x7f9ac41095f0 2024-05-10T11:06:07.732 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:06.940+0000 7f9ac2ffd700 1 -- 172.21.15.100:0/3955789380 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ab4004300 con 0x7f9ac41095f0 2024-05-10T11:06:07.732 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:06.940+0000 7f9ac2ffd700 1 -- 172.21.15.100:0/3955789380 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ab40036a0 con 0x7f9ac41095f0 2024-05-10T11:06:07.732 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:06.941+0000 7f9aca389700 1 -- 172.21.15.100:0/3955789380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ac41095f0 msgr2=0x7f9ac410b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:06:07.732 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:06.941+0000 7f9aca389700 1 --2- 172.21.15.100:0/3955789380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ac41095f0 0x7f9ac410b9e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f9ab40098a0 tx=0x7f9ab4009bb0 comp rx=0 tx=0).stop 2024-05-10T11:06:07.733 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:06.941+0000 7f9aca389700 1 -- 172.21.15.100:0/3955789380 shutdown_connections 2024-05-10T11:06:07.733 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:06.941+0000 7f9aca389700 1 --2- 172.21.15.100:0/3955789380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ac41095f0 0x7f9ac410b9e0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:06:07.733 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:06.941+0000 7f9aca389700 1 -- 172.21.15.100:0/3955789380 >> 172.21.15.100:0/3955789380 conn(0x7f9ac4102fa0 msgr2=0x7f9ac4105400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:06:07.733 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:06.941+0000 7f9aca389700 1 -- 172.21.15.100:0/3955789380 shutdown_connections 2024-05-10T11:06:07.733 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:06.941+0000 7f9aca389700 1 -- 172.21.15.100:0/3955789380 wait complete. 2024-05-10T11:06:07.733 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:06.942+0000 7f9aca389700 1 Processor -- start 2024-05-10T11:06:07.733 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:06.942+0000 7f9aca389700 1 -- start start 2024-05-10T11:06:07.733 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:06.942+0000 7f9aca389700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ac41095f0 0x7f9ac419c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:06:07.733 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:06.942+0000 7f9aca389700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac410c390 con 0x7f9ac41095f0 2024-05-10T11:06:07.733 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:06.942+0000 7f9ac3fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ac41095f0 0x7f9ac419c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:06:07.733 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:06.942+0000 7f9ac3fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ac41095f0 0x7f9ac419c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44756/0 (socket says 172.21.15.100:44756) 2024-05-10T11:06:07.733 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:06.942+0000 7f9ac3fff700 1 -- 172.21.15.100:0/2122077720 learned_addr learned my addr 172.21.15.100:0/2122077720 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:06:07.733 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:06.942+0000 7f9ac3fff700 1 -- 172.21.15.100:0/2122077720 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ac419c740 con 0x7f9ac41095f0 2024-05-10T11:06:07.734 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:06.942+0000 7f9ac3fff700 1 --2- 172.21.15.100:0/2122077720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ac41095f0 0x7f9ac419c200 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f9ab4000c00 tx=0x7f9ab4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:06:07.734 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:06.943+0000 7f9ac17fa700 1 -- 172.21.15.100:0/2122077720 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ab4003db0 con 0x7f9ac41095f0 2024-05-10T11:06:07.734 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:06.943+0000 7f9ac17fa700 1 -- 172.21.15.100:0/2122077720 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ab4011630 con 0x7f9ac41095f0 2024-05-10T11:06:07.734 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:06.943+0000 7f9ac17fa700 1 -- 172.21.15.100:0/2122077720 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ab4011950 con 0x7f9ac41095f0 2024-05-10T11:06:07.734 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:06.943+0000 7f9aca389700 1 -- 172.21.15.100:0/2122077720 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ac419c9d0 con 0x7f9ac41095f0 2024-05-10T11:06:07.734 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:06.943+0000 7f9aca389700 1 -- 172.21.15.100:0/2122077720 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ac419f6c0 con 0x7f9ac41095f0 2024-05-10T11:06:07.734 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:06.943+0000 7f9ac17fa700 1 -- 172.21.15.100:0/2122077720 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9ab4011af0 con 0x7f9ac41095f0 2024-05-10T11:06:07.734 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:06.943+0000 7f9ac17fa700 1 -- 172.21.15.100:0/2122077720 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9ab401c450 con 0x7f9ac41095f0 2024-05-10T11:06:07.734 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:06.944+0000 7f9aca389700 1 -- 172.21.15.100:0/2122077720 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ac4064b00 con 0x7f9ac41095f0 2024-05-10T11:06:07.734 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:06.946+0000 7f9ac17fa700 1 -- 172.21.15.100:0/2122077720 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f9ab4004260 con 0x7f9ac41095f0 2024-05-10T11:06:07.734 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:07.012+0000 7f9aca389700 1 -- 172.21.15.100:0/2122077720 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f9ac402e710 con 0x7f9ac41095f0 2024-05-10T11:06:07.734 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:07.015+0000 7f9ac17fa700 1 -- 172.21.15.100:0/2122077720 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+467 (secure 0 0 0) 0x7f9ab401b450 con 0x7f9ac41095f0 2024-05-10T11:06:07.734 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:07.016+0000 7f9ac17fa700 1 -- 172.21.15.100:0/2122077720 <== mon.0 v2:172.21.15.100:3300/0 8 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f9ab401b630 con 0x7f9ac41095f0 2024-05-10T11:06:07.734 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:07.017+0000 7f9aca389700 1 -- 172.21.15.100:0/2122077720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ac41095f0 msgr2=0x7f9ac419c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:06:07.735 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:07.017+0000 7f9aca389700 1 --2- 172.21.15.100:0/2122077720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ac41095f0 0x7f9ac419c200 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f9ab4000c00 tx=0x7f9ab4004750 comp rx=0 tx=0).stop 2024-05-10T11:06:07.735 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:07.017+0000 7f9aca389700 1 -- 172.21.15.100:0/2122077720 shutdown_connections 2024-05-10T11:06:07.735 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:07.017+0000 7f9aca389700 1 --2- 172.21.15.100:0/2122077720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ac41095f0 0x7f9ac419c200 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:06:07.735 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:07.017+0000 7f9aca389700 1 -- 172.21.15.100:0/2122077720 >> 172.21.15.100:0/2122077720 conn(0x7f9ac4102fa0 msgr2=0x7f9ac41052d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:06:07.735 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:07.017+0000 7f9aca389700 1 -- 172.21.15.100:0/2122077720 shutdown_connections 2024-05-10T11:06:07.735 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:07.017+0000 7f9aca389700 1 -- 172.21.15.100:0/2122077720 wait complete. 2024-05-10T11:06:07.735 INFO:teuthology.orchestra.run.smithi100.stdout:Generating new minimal ceph.conf... 2024-05-10T11:06:08.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:08 smithi100 ceph-mon[29927]: from='client.? 172.21.15.100:0/3820254586' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:06:09.248 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:08.162+0000 7f0cf9492700 1 Processor -- start 2024-05-10T11:06:09.253 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:08.162+0000 7f0cf9492700 1 -- start start 2024-05-10T11:06:09.253 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:08.162+0000 7f0cf9492700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0cf41095c0 0x7f0cf410b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:06:09.253 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:08.162+0000 7f0cf9492700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf4076980 con 0x7f0cf41095c0 2024-05-10T11:06:09.253 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:08.162+0000 7f0cf2ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0cf41095c0 0x7f0cf410b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:06:09.253 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:08.162+0000 7f0cf2ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0cf41095c0 0x7f0cf410b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44770/0 (socket says 172.21.15.100:44770) 2024-05-10T11:06:09.254 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:08.162+0000 7f0cf2ffd700 1 -- 172.21.15.100:0/1127887670 learned_addr learned my addr 172.21.15.100:0/1127887670 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:06:09.254 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:08.162+0000 7f0cf2ffd700 1 -- 172.21.15.100:0/1127887670 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cf410bef0 con 0x7f0cf41095c0 2024-05-10T11:06:09.254 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:08.162+0000 7f0cf2ffd700 1 --2- 172.21.15.100:0/1127887670 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0cf41095c0 0x7f0cf410b9b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f0ce80098a0 tx=0x7f0ce8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33be8b3e7655f3f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:06:09.254 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:08.162+0000 7f0cf1ffb700 1 -- 172.21.15.100:0/1127887670 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ce8004160 con 0x7f0cf41095c0 2024-05-10T11:06:09.254 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:08.162+0000 7f0cf1ffb700 1 -- 172.21.15.100:0/1127887670 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f0ce8004300 con 0x7f0cf41095c0 2024-05-10T11:06:09.254 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:08.163+0000 7f0cf1ffb700 1 -- 172.21.15.100:0/1127887670 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ce80119e0 con 0x7f0cf41095c0 2024-05-10T11:06:09.254 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:08.163+0000 7f0cf9492700 1 -- 172.21.15.100:0/1127887670 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0cf41095c0 msgr2=0x7f0cf410b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:06:09.254 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:08.163+0000 7f0cf9492700 1 --2- 172.21.15.100:0/1127887670 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0cf41095c0 0x7f0cf410b9b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f0ce80098a0 tx=0x7f0ce8009bb0 comp rx=0 tx=0).stop 2024-05-10T11:06:09.254 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:08.163+0000 7f0cf9492700 1 -- 172.21.15.100:0/1127887670 shutdown_connections 2024-05-10T11:06:09.254 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:08.163+0000 7f0cf9492700 1 --2- 172.21.15.100:0/1127887670 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0cf41095c0 0x7f0cf410b9b0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:06:09.254 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:08.163+0000 7f0cf9492700 1 -- 172.21.15.100:0/1127887670 >> 172.21.15.100:0/1127887670 conn(0x7f0cf4102fd0 msgr2=0x7f0cf41053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:06:09.254 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:08.163+0000 7f0cf9492700 1 -- 172.21.15.100:0/1127887670 shutdown_connections 2024-05-10T11:06:09.254 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:08.163+0000 7f0cf9492700 1 -- 172.21.15.100:0/1127887670 wait complete. 2024-05-10T11:06:09.255 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:08.164+0000 7f0cf9492700 1 Processor -- start 2024-05-10T11:06:09.255 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:08.164+0000 7f0cf9492700 1 -- start start 2024-05-10T11:06:09.255 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:08.164+0000 7f0cf9492700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0cf41095c0 0x7f0cf4108e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:06:09.255 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:08.164+0000 7f0cf9492700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf410ce00 con 0x7f0cf41095c0 2024-05-10T11:06:09.255 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:08.164+0000 7f0cf2ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0cf41095c0 0x7f0cf4108e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:06:09.255 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:08.164+0000 7f0cf2ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0cf41095c0 0x7f0cf4108e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44778/0 (socket says 172.21.15.100:44778) 2024-05-10T11:06:09.255 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:08.164+0000 7f0cf2ffd700 1 -- 172.21.15.100:0/3820254586 learned_addr learned my addr 172.21.15.100:0/3820254586 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:06:09.255 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:08.165+0000 7f0cf2ffd700 1 -- 172.21.15.100:0/3820254586 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cf4109380 con 0x7f0cf41095c0 2024-05-10T11:06:09.255 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:08.165+0000 7f0cf2ffd700 1 --2- 172.21.15.100:0/3820254586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0cf41095c0 0x7f0cf4108e40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0ce80119e0 tx=0x7f0ce80046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:06:09.255 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:08.165+0000 7f0cdbfff700 1 -- 172.21.15.100:0/3820254586 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ce8003bf0 con 0x7f0cf41095c0 2024-05-10T11:06:09.255 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:08.165+0000 7f0cdbfff700 1 -- 172.21.15.100:0/3820254586 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f0ce801a8a0 con 0x7f0cf41095c0 2024-05-10T11:06:09.255 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:08.165+0000 7f0cf9492700 1 -- 172.21.15.100:0/3820254586 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cf4107570 con 0x7f0cf41095c0 2024-05-10T11:06:09.255 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:08.165+0000 7f0cdbfff700 1 -- 172.21.15.100:0/3820254586 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ce80248c0 con 0x7f0cf41095c0 2024-05-10T11:06:09.255 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:08.165+0000 7f0cf9492700 1 -- 172.21.15.100:0/3820254586 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cf4107ab0 con 0x7f0cf41095c0 2024-05-10T11:06:09.256 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:08.165+0000 7f0cdbfff700 1 -- 172.21.15.100:0/3820254586 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0ce8022020 con 0x7f0cf41095c0 2024-05-10T11:06:09.256 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:08.165+0000 7f0cdbfff700 1 -- 172.21.15.100:0/3820254586 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0ce80193c0 con 0x7f0cf41095c0 2024-05-10T11:06:09.256 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:08.166+0000 7f0cf9492700 1 -- 172.21.15.100:0/3820254586 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cf4064b00 con 0x7f0cf41095c0 2024-05-10T11:06:09.256 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:08.168+0000 7f0cdbfff700 1 -- 172.21.15.100:0/3820254586 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f0ce8043e50 con 0x7f0cf41095c0 2024-05-10T11:06:09.256 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:08.238+0000 7f0cf9492700 1 -- 172.21.15.100:0/3820254586 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f0cf4107d60 con 0x7f0cf41095c0 2024-05-10T11:06:09.256 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:08.238+0000 7f0cdbfff700 1 -- 172.21.15.100:0/3820254586 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+177 (secure 0 0 0) 0x7f0ce8032030 con 0x7f0cf41095c0 2024-05-10T11:06:09.256 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:08.239+0000 7f0cf9492700 1 -- 172.21.15.100:0/3820254586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0cf41095c0 msgr2=0x7f0cf4108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:06:09.256 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:08.239+0000 7f0cf9492700 1 --2- 172.21.15.100:0/3820254586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0cf41095c0 0x7f0cf4108e40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0ce80119e0 tx=0x7f0ce80046a0 comp rx=0 tx=0).stop 2024-05-10T11:06:09.256 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:08.239+0000 7f0cf9492700 1 -- 172.21.15.100:0/3820254586 shutdown_connections 2024-05-10T11:06:09.256 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:08.239+0000 7f0cf9492700 1 --2- 172.21.15.100:0/3820254586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0cf41095c0 0x7f0cf4108e40 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:06:09.256 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:08.239+0000 7f0cf9492700 1 -- 172.21.15.100:0/3820254586 >> 172.21.15.100:0/3820254586 conn(0x7f0cf4102fd0 msgr2=0x7f0cf4103b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:06:09.256 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:08.239+0000 7f0cf9492700 1 -- 172.21.15.100:0/3820254586 shutdown_connections 2024-05-10T11:06:09.256 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:08.239+0000 7f0cf9492700 1 -- 172.21.15.100:0/3820254586 wait complete. 2024-05-10T11:06:09.256 INFO:teuthology.orchestra.run.smithi100.stdout:Restarting the monitor... 2024-05-10T11:06:09.516 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:09 smithi100 systemd[1]: Stopping Ceph mon.a for 20a542cc-0ebd-11ef-bc98-c7b262605968... 2024-05-10T11:06:09.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:09 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mon-a[29923]: 2024-05-10T11:06:09.515+0000 7f41ad874700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T11:06:09.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:09 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mon-a[29923]: 2024-05-10T11:06:09.515+0000 7f41ad874700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-05-10T11:06:09.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:09 smithi100 podman[30171]: 2024-05-10 11:06:09.761385427 +0000 UTC m=+0.341989844 container died aec1e48d3a833a55f49da4873f9c01b02cbd2f4167a9645ad03a398023b0ecbb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mon-a, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, CEPH_POINT_RELEASE=, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0) 2024-05-10T11:06:10.274 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 podman[30171]: 2024-05-10 11:06:10.089849147 +0000 UTC m=+0.670453571 container cleanup aec1e48d3a833a55f49da4873f9c01b02cbd2f4167a9645ad03a398023b0ecbb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mon-a, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, maintainer=Guillaume Abrioux , RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-05-10T11:06:10.274 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 bash[30171]: ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mon-a 2024-05-10T11:06:10.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 podman[30185]: 2024-05-10 11:06:10.274179908 +0000 UTC m=+0.510011619 container remove aec1e48d3a833a55f49da4873f9c01b02cbd2f4167a9645ad03a398023b0ecbb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mon-a, org.label-schema.build-date=20240507, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-10T11:06:10.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 systemd[1]: ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mon.a.service: Deactivated successfully. 2024-05-10T11:06:10.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 systemd[1]: Stopped Ceph mon.a for 20a542cc-0ebd-11ef-bc98-c7b262605968. 2024-05-10T11:06:10.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 systemd[1]: Starting Ceph mon.a for 20a542cc-0ebd-11ef-bc98-c7b262605968... 2024-05-10T11:06:10.827 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 podman[30278]: 2024-05-10 11:06:10.716674396 +0000 UTC m=+0.098541869 container create 0469b48457b91a3da3da0578c68bc9d9af67a2aff2d0f427168c619c1078da34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_CLEAN=True) 2024-05-10T11:06:10.827 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 podman[30278]: 2024-05-10 11:06:10.635940781 +0000 UTC m=+0.017808275 image pull ee5e4f21ef58047cb638cccd5cfbe8f6f43412b56c94b1d7fbd9cf3cb90a53a5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b 2024-05-10T11:06:10.981 INFO:teuthology.orchestra.run.smithi100.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-05-10T11:06:11.106 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 podman[30278]: 2024-05-10 11:06:10.910728124 +0000 UTC m=+0.292595597 container init 0469b48457b91a3da3da0578c68bc9d9af67a2aff2d0f427168c619c1078da34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mon-a, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, io.buildah.version=1.33.5) 2024-05-10T11:06:11.106 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 podman[30278]: 2024-05-10 11:06:10.913840872 +0000 UTC m=+0.295708348 container start 0469b48457b91a3da3da0578c68bc9d9af67a2aff2d0f427168c619c1078da34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mon-a, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-05-10T11:06:11.106 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: set uid:gid to 167:167 (ceph:ceph) 2024-05-10T11:06:11.106 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: ceph version 19.0.0-3449-gd39fa088 (d39fa08867836ac070c47cb66c782fa559fd480b) squid (dev), process ceph-mon, pid 2 2024-05-10T11:06:11.106 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: pidfile_write: ignore empty --pid-file 2024-05-10T11:06:11.107 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: load: jerasure load: lrc 2024-05-10T11:06:11.107 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: RocksDB version: 7.9.2 2024-05-10T11:06:11.107 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Git sha 0 2024-05-10T11:06:11.107 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Compile date 2024-05-10 01:38:55 2024-05-10T11:06:11.107 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: DB SUMMARY 2024-05-10T11:06:11.107 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: DB Session ID: AFNX5YK5AHTL2KMFZ1GR 2024-05-10T11:06:11.107 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: CURRENT file: CURRENT 2024-05-10T11:06:11.107 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: IDENTITY file: IDENTITY 2024-05-10T11:06:11.107 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-05-10T11:06:11.107 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-05-10T11:06:11.107 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 103006 ; 2024-05-10T11:06:11.107 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.error_if_exists: 0 2024-05-10T11:06:11.107 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.create_if_missing: 0 2024-05-10T11:06:11.108 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.paranoid_checks: 1 2024-05-10T11:06:11.108 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-10T11:06:11.108 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-10T11:06:11.108 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-10T11:06:11.108 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.env: 0x558f7e89acc0 2024-05-10T11:06:11.108 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.fs: PosixFileSystem 2024-05-10T11:06:11.108 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.info_log: 0x558f7fe19da0 2024-05-10T11:06:11.108 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.max_file_opening_threads: 16 2024-05-10T11:06:11.108 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.statistics: (nil) 2024-05-10T11:06:11.108 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.use_fsync: 0 2024-05-10T11:06:11.108 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.max_log_file_size: 0 2024-05-10T11:06:11.108 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-10T11:06:11.109 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-10T11:06:11.109 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.keep_log_file_num: 1000 2024-05-10T11:06:11.109 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.recycle_log_file_num: 0 2024-05-10T11:06:11.109 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.allow_fallocate: 1 2024-05-10T11:06:11.109 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.allow_mmap_reads: 0 2024-05-10T11:06:11.109 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.allow_mmap_writes: 0 2024-05-10T11:06:11.109 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.use_direct_reads: 0 2024-05-10T11:06:11.109 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-10T11:06:11.110 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.create_missing_column_families: 0 2024-05-10T11:06:11.110 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.db_log_dir: 2024-05-10T11:06:11.110 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.wal_dir: 2024-05-10T11:06:11.110 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-10T11:06:11.110 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-10T11:06:11.110 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-10T11:06:11.110 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-10T11:06:11.110 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-10T11:06:11.111 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-10T11:06:11.111 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.advise_random_on_open: 1 2024-05-10T11:06:11.111 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.db_write_buffer_size: 0 2024-05-10T11:06:11.111 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.write_buffer_manager: 0x558f7fdeb0e0 2024-05-10T11:06:11.111 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-10T11:06:11.111 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-10T11:06:11.111 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-10T11:06:11.111 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.rate_limiter: (nil) 2024-05-10T11:06:11.112 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-10T11:06:11.112 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.wal_recovery_mode: 2 2024-05-10T11:06:11.112 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.enable_thread_tracking: 0 2024-05-10T11:06:11.112 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.enable_pipelined_write: 0 2024-05-10T11:06:11.112 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.unordered_write: 0 2024-05-10T11:06:11.112 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-10T11:06:11.112 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-10T11:06:11.112 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-10T11:06:11.112 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-10T11:06:11.112 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.row_cache: None 2024-05-10T11:06:11.113 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.wal_filter: None 2024-05-10T11:06:11.113 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-10T11:06:11.113 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.allow_ingest_behind: 0 2024-05-10T11:06:11.113 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.two_write_queues: 0 2024-05-10T11:06:11.113 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.manual_wal_flush: 0 2024-05-10T11:06:11.113 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.wal_compression: 0 2024-05-10T11:06:11.113 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.atomic_flush: 0 2024-05-10T11:06:11.113 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-10T11:06:11.113 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-10T11:06:11.113 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-10T11:06:11.113 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.log_readahead_size: 0 2024-05-10T11:06:11.114 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-10T11:06:11.114 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.best_efforts_recovery: 0 2024-05-10T11:06:11.114 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-10T11:06:11.114 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-10T11:06:11.114 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.allow_data_in_errors: 0 2024-05-10T11:06:11.114 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.db_host_id: __hostname__ 2024-05-10T11:06:11.114 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-10T11:06:11.114 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.max_background_jobs: 2 2024-05-10T11:06:11.114 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.max_background_compactions: -1 2024-05-10T11:06:11.114 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.max_subcompactions: 1 2024-05-10T11:06:11.115 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-10T11:06:11.115 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-10T11:06:11.115 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-10T11:06:11.115 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.max_total_wal_size: 0 2024-05-10T11:06:11.115 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-10T11:06:11.115 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-10T11:06:11.115 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-10T11:06:11.115 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-10T11:06:11.115 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.max_open_files: -1 2024-05-10T11:06:11.115 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.bytes_per_sync: 0 2024-05-10T11:06:11.115 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-10T11:06:11.115 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-10T11:06:11.115 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.compaction_readahead_size: 0 2024-05-10T11:06:11.116 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.max_background_flushes: -1 2024-05-10T11:06:11.116 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Compression algorithms supported: 2024-05-10T11:06:11.116 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-10T11:06:11.116 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: kZSTD supported: 0 2024-05-10T11:06:11.116 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: kXpressCompression supported: 0 2024-05-10T11:06:11.116 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: kLZ4HCCompression supported: 1 2024-05-10T11:06:11.116 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: kZlibCompression supported: 1 2024-05-10T11:06:11.116 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: kSnappyCompression supported: 1 2024-05-10T11:06:11.116 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: kLZ4Compression supported: 1 2024-05-10T11:06:11.116 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: kBZip2Compression supported: 0 2024-05-10T11:06:11.116 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-10T11:06:11.117 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-10T11:06:11.117 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-05-10T11:06:11.117 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-10T11:06:11.117 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-10T11:06:11.117 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.merge_operator: 2024-05-10T11:06:11.117 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.compaction_filter: None 2024-05-10T11:06:11.117 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.compaction_filter_factory: None 2024-05-10T11:06:11.117 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.sst_partitioner_factory: None 2024-05-10T11:06:11.117 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-10T11:06:11.117 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-10T11:06:11.117 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x558f7fe19f60) 2024-05-10T11:06:11.117 INFO:journalctl@ceph.mon.a.smithi100.stdout: cache_index_and_filter_blocks: 1 2024-05-10T11:06:11.118 INFO:journalctl@ceph.mon.a.smithi100.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-10T11:06:11.118 INFO:journalctl@ceph.mon.a.smithi100.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-10T11:06:11.118 INFO:journalctl@ceph.mon.a.smithi100.stdout: pin_top_level_index_and_filter: 1 2024-05-10T11:06:11.118 INFO:journalctl@ceph.mon.a.smithi100.stdout: index_type: 0 2024-05-10T11:06:11.118 INFO:journalctl@ceph.mon.a.smithi100.stdout: data_block_index_type: 0 2024-05-10T11:06:11.118 INFO:journalctl@ceph.mon.a.smithi100.stdout: index_shortening: 1 2024-05-10T11:06:11.118 INFO:journalctl@ceph.mon.a.smithi100.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-10T11:06:11.118 INFO:journalctl@ceph.mon.a.smithi100.stdout: checksum: 4 2024-05-10T11:06:11.118 INFO:journalctl@ceph.mon.a.smithi100.stdout: no_block_cache: 0 2024-05-10T11:06:11.118 INFO:journalctl@ceph.mon.a.smithi100.stdout: block_cache: 0x558f7fe1d1f0 2024-05-10T11:06:11.118 INFO:journalctl@ceph.mon.a.smithi100.stdout: block_cache_name: BinnedLRUCache 2024-05-10T11:06:11.119 INFO:journalctl@ceph.mon.a.smithi100.stdout: block_cache_options: 2024-05-10T11:06:11.119 INFO:journalctl@ceph.mon.a.smithi100.stdout: capacity : 536870912 2024-05-10T11:06:11.119 INFO:journalctl@ceph.mon.a.smithi100.stdout: num_shard_bits : 4 2024-05-10T11:06:11.119 INFO:journalctl@ceph.mon.a.smithi100.stdout: strict_capacity_limit : 0 2024-05-10T11:06:11.119 INFO:journalctl@ceph.mon.a.smithi100.stdout: high_pri_pool_ratio: 0.000 2024-05-10T11:06:11.119 INFO:journalctl@ceph.mon.a.smithi100.stdout: block_cache_compressed: (nil) 2024-05-10T11:06:11.119 INFO:journalctl@ceph.mon.a.smithi100.stdout: persistent_cache: (nil) 2024-05-10T11:06:11.119 INFO:journalctl@ceph.mon.a.smithi100.stdout: block_size: 4096 2024-05-10T11:06:11.119 INFO:journalctl@ceph.mon.a.smithi100.stdout: block_size_deviation: 10 2024-05-10T11:06:11.119 INFO:journalctl@ceph.mon.a.smithi100.stdout: block_restart_interval: 16 2024-05-10T11:06:11.119 INFO:journalctl@ceph.mon.a.smithi100.stdout: index_block_restart_interval: 1 2024-05-10T11:06:11.120 INFO:journalctl@ceph.mon.a.smithi100.stdout: metadata_block_size: 4096 2024-05-10T11:06:11.120 INFO:journalctl@ceph.mon.a.smithi100.stdout: partition_filters: 0 2024-05-10T11:06:11.120 INFO:journalctl@ceph.mon.a.smithi100.stdout: use_delta_encoding: 1 2024-05-10T11:06:11.120 INFO:journalctl@ceph.mon.a.smithi100.stdout: filter_policy: bloomfilter 2024-05-10T11:06:11.120 INFO:journalctl@ceph.mon.a.smithi100.stdout: whole_key_filtering: 1 2024-05-10T11:06:11.120 INFO:journalctl@ceph.mon.a.smithi100.stdout: verify_compression: 0 2024-05-10T11:06:11.120 INFO:journalctl@ceph.mon.a.smithi100.stdout: read_amp_bytes_per_bit: 0 2024-05-10T11:06:11.120 INFO:journalctl@ceph.mon.a.smithi100.stdout: format_version: 5 2024-05-10T11:06:11.120 INFO:journalctl@ceph.mon.a.smithi100.stdout: enable_index_compression: 1 2024-05-10T11:06:11.120 INFO:journalctl@ceph.mon.a.smithi100.stdout: block_align: 0 2024-05-10T11:06:11.120 INFO:journalctl@ceph.mon.a.smithi100.stdout: max_auto_readahead_size: 262144 2024-05-10T11:06:11.121 INFO:journalctl@ceph.mon.a.smithi100.stdout: prepopulate_block_cache: 0 2024-05-10T11:06:11.121 INFO:journalctl@ceph.mon.a.smithi100.stdout: initial_auto_readahead_size: 8192 2024-05-10T11:06:11.125 INFO:journalctl@ceph.mon.a.smithi100.stdout: num_file_reads_for_auto_readahead: 2 2024-05-10T11:06:11.126 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.write_buffer_size: 33554432 2024-05-10T11:06:11.126 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.max_write_buffer_number: 2 2024-05-10T11:06:11.126 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.compression: NoCompression 2024-05-10T11:06:11.126 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.bottommost_compression: Disabled 2024-05-10T11:06:11.126 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.prefix_extractor: nullptr 2024-05-10T11:06:11.126 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-10T11:06:11.126 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.num_levels: 7 2024-05-10T11:06:11.126 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-10T11:06:11.126 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-10T11:06:11.127 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-10T11:06:11.127 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-10T11:06:11.127 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-10T11:06:11.127 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-10T11:06:11.127 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-10T11:06:11.127 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-10T11:06:11.127 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-10T11:06:11.127 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-10T11:06:11.127 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-10T11:06:11.127 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-10T11:06:11.127 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-10T11:06:11.127 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.compression_opts.level: 32767 2024-05-10T11:06:11.127 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.compression_opts.strategy: 0 2024-05-10T11:06:11.128 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-10T11:06:11.128 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-10T11:06:11.128 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-10T11:06:11.128 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-10T11:06:11.128 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.compression_opts.enabled: false 2024-05-10T11:06:11.128 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-10T11:06:11.128 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-10T11:06:11.128 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-10T11:06:11.128 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-10T11:06:11.128 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.target_file_size_base: 67108864 2024-05-10T11:06:11.128 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-10T11:06:11.128 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-10T11:06:11.128 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-10T11:06:11.128 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-10T11:06:11.129 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-10T11:06:11.129 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-10T11:06:11.129 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-10T11:06:11.129 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-10T11:06:11.129 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-10T11:06:11.129 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-10T11:06:11.129 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-10T11:06:11.129 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-10T11:06:11.129 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-10T11:06:11.129 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-10T11:06:11.129 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.arena_block_size: 1048576 2024-05-10T11:06:11.129 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-10T11:06:11.129 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-10T11:06:11.129 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.disable_auto_compactions: 0 2024-05-10T11:06:11.130 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-10T11:06:11.130 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-10T11:06:11.130 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-10T11:06:11.130 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-10T11:06:11.130 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-10T11:06:11.130 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-10T11:06:11.130 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-10T11:06:11.130 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-10T11:06:11.130 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-10T11:06:11.130 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-10T11:06:11.130 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-10T11:06:11.130 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.inplace_update_support: 0 2024-05-10T11:06:11.130 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-10T11:06:11.131 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-10T11:06:11.131 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-10T11:06:11.131 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-10T11:06:11.131 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.bloom_locality: 0 2024-05-10T11:06:11.131 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.max_successive_merges: 0 2024-05-10T11:06:11.131 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-10T11:06:11.132 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.paranoid_file_checks: 0 2024-05-10T11:06:11.132 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.force_consistency_checks: 1 2024-05-10T11:06:11.132 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.report_bg_io_stats: 0 2024-05-10T11:06:11.132 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.ttl: 2592000 2024-05-10T11:06:11.132 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-10T11:06:11.132 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-10T11:06:11.132 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-10T11:06:11.132 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.enable_blob_files: false 2024-05-10T11:06:11.132 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.min_blob_size: 0 2024-05-10T11:06:11.132 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.blob_file_size: 268435456 2024-05-10T11:06:11.132 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-10T11:06:11.132 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-10T11:06:11.132 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-10T11:06:11.132 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-10T11:06:11.133 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-10T11:06:11.133 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.blob_file_starting_level: 0 2024-05-10T11:06:11.133 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-10T11:06:11.133 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-05-10T11:06:11.133 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-05-10T11:06:11.133 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: eaf13cb6-97dd-4e94-93b4-9565d7e54b19 2024-05-10T11:06:11.133 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715339170943903, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-05-10T11:06:11.133 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-05-10T11:06:11.133 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715339170944962, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 98284, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 318, "table_properties": {"data_size": 96272, "index_size": 268, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13651, "raw_average_key_size": 49, "raw_value_size": 88820, "raw_average_value_size": 320, "num_data_blocks": 12, "num_entries": 277, "num_filter_entries": 277, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715339170, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "eaf13cb6-97dd-4e94-93b4-9565d7e54b19", "db_session_id": "AFNX5YK5AHTL2KMFZ1GR", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-05-10T11:06:11.133 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715339170945049, "job": 1, "event": "recovery_finished"} 2024-05-10T11:06:11.133 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-05-10T11:06:11.133 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-10T11:06:11.133 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x558f7ff3a000 2024-05-10T11:06:11.133 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: DB pointer 0x558f7ff26000 2024-05-10T11:06:11.134 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-10T11:06:11.134 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-10T11:06:11.134 INFO:journalctl@ceph.mon.a.smithi100.stdout: ** DB Stats ** 2024-05-10T11:06:11.134 INFO:journalctl@ceph.mon.a.smithi100.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-10T11:06:11.134 INFO:journalctl@ceph.mon.a.smithi100.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-10T11:06:11.134 INFO:journalctl@ceph.mon.a.smithi100.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-10T11:06:11.134 INFO:journalctl@ceph.mon.a.smithi100.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-10T11:06:11.134 INFO:journalctl@ceph.mon.a.smithi100.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-10T11:06:11.134 INFO:journalctl@ceph.mon.a.smithi100.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-10T11:06:11.134 INFO:journalctl@ceph.mon.a.smithi100.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-10T11:06:11.134 INFO:journalctl@ceph.mon.a.smithi100.stdout: 2024-05-10T11:06:11.134 INFO:journalctl@ceph.mon.a.smithi100.stdout: ** Compaction Stats [default] ** 2024-05-10T11:06:11.135 INFO:journalctl@ceph.mon.a.smithi100.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-10T11:06:11.135 INFO:journalctl@ceph.mon.a.smithi100.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-10T11:06:11.135 INFO:journalctl@ceph.mon.a.smithi100.stdout: L0 2/0 97.84 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 139.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-10T11:06:11.135 INFO:journalctl@ceph.mon.a.smithi100.stdout: Sum 2/0 97.84 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 139.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-10T11:06:11.135 INFO:journalctl@ceph.mon.a.smithi100.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 139.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-10T11:06:11.135 INFO:journalctl@ceph.mon.a.smithi100.stdout: 2024-05-10T11:06:11.135 INFO:journalctl@ceph.mon.a.smithi100.stdout: ** Compaction Stats [default] ** 2024-05-10T11:06:11.135 INFO:journalctl@ceph.mon.a.smithi100.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-10T11:06:11.135 INFO:journalctl@ceph.mon.a.smithi100.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-10T11:06:11.135 INFO:journalctl@ceph.mon.a.smithi100.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 139.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-10T11:06:11.135 INFO:journalctl@ceph.mon.a.smithi100.stdout: 2024-05-10T11:06:11.136 INFO:journalctl@ceph.mon.a.smithi100.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-10T11:06:11.136 INFO:journalctl@ceph.mon.a.smithi100.stdout: 2024-05-10T11:06:11.136 INFO:journalctl@ceph.mon.a.smithi100.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-10T11:06:11.136 INFO:journalctl@ceph.mon.a.smithi100.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-10T11:06:11.136 INFO:journalctl@ceph.mon.a.smithi100.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-10T11:06:11.136 INFO:journalctl@ceph.mon.a.smithi100.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-10T11:06:11.136 INFO:journalctl@ceph.mon.a.smithi100.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-10T11:06:11.136 INFO:journalctl@ceph.mon.a.smithi100.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-10T11:06:11.136 INFO:journalctl@ceph.mon.a.smithi100.stdout: Cumulative compaction: 0.00 GB write, 30.29 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-10T11:06:11.136 INFO:journalctl@ceph.mon.a.smithi100.stdout: Interval compaction: 0.00 GB write, 30.29 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-10T11:06:11.137 INFO:journalctl@ceph.mon.a.smithi100.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-10T11:06:11.137 INFO:journalctl@ceph.mon.a.smithi100.stdout: Block cache BinnedLRUCache@0x558f7fe1d1f0#2 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-05-10T11:06:11.137 INFO:journalctl@ceph.mon.a.smithi100.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-05-10T11:06:11.137 INFO:journalctl@ceph.mon.a.smithi100.stdout: 2024-05-10T11:06:11.137 INFO:journalctl@ceph.mon.a.smithi100.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-10T11:06:11.137 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: starting mon.a rank 0 at public addrs [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] at bind addrs [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 2024-05-10T11:06:11.137 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: mon.a@-1(???) e1 preinit fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 2024-05-10T11:06:11.137 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: mon.a@-1(???).mds e1 new map 2024-05-10T11:06:11.137 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: mon.a@-1(???).mds e1 print_map 2024-05-10T11:06:11.137 INFO:journalctl@ceph.mon.a.smithi100.stdout: e1 2024-05-10T11:06:11.137 INFO:journalctl@ceph.mon.a.smithi100.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-10T11:06:11.137 INFO:journalctl@ceph.mon.a.smithi100.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-10T11:06:11.137 INFO:journalctl@ceph.mon.a.smithi100.stdout: legacy client fscid: -1 2024-05-10T11:06:11.138 INFO:journalctl@ceph.mon.a.smithi100.stdout: 2024-05-10T11:06:11.138 INFO:journalctl@ceph.mon.a.smithi100.stdout: No filesystems configured 2024-05-10T11:06:11.138 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-10T11:06:11.138 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-10T11:06:11.138 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-10T11:06:11.138 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-10T11:06:11.138 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 ceph-mon[30296]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-05-10T11:06:11.138 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 bash[30278]: 0469b48457b91a3da3da0578c68bc9d9af67a2aff2d0f427168c619c1078da34 2024-05-10T11:06:11.138 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:10 smithi100 systemd[1]: Started Ceph mon.a for 20a542cc-0ebd-11ef-bc98-c7b262605968. 2024-05-10T11:06:11.138 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:11 smithi100 ceph-mon[30296]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-10T11:06:11.138 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:11 smithi100 ceph-mon[30296]: monmap epoch 1 2024-05-10T11:06:11.138 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:11 smithi100 ceph-mon[30296]: fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 2024-05-10T11:06:11.138 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:11 smithi100 ceph-mon[30296]: last_changed 2024-05-10T11:06:01.369875+0000 2024-05-10T11:06:11.139 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:11 smithi100 ceph-mon[30296]: created 2024-05-10T11:06:01.369875+0000 2024-05-10T11:06:11.139 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:11 smithi100 ceph-mon[30296]: min_mon_release 19 (squid) 2024-05-10T11:06:11.139 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:11 smithi100 ceph-mon[30296]: election_strategy: 1 2024-05-10T11:06:11.139 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:11 smithi100 ceph-mon[30296]: 0: [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] mon.a 2024-05-10T11:06:11.139 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:11 smithi100 ceph-mon[30296]: fsmap 2024-05-10T11:06:11.139 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:11 smithi100 ceph-mon[30296]: osdmap e1: 0 total, 0 up, 0 in 2024-05-10T11:06:11.139 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:11 smithi100 ceph-mon[30296]: mgrmap e1: no daemons active 2024-05-10T11:06:12.053 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:11.441+0000 7f3260da5700 1 Processor -- start 2024-05-10T11:06:12.057 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:11.442+0000 7f3260da5700 1 -- start start 2024-05-10T11:06:12.057 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:11.442+0000 7f3260da5700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f325c109600 0x7f325c10b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:06:12.057 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:11.442+0000 7f3260da5700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f325c07ce20 con 0x7f325c109600 2024-05-10T11:06:12.058 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:11.442+0000 7f325a59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f325c109600 0x7f325c10b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:06:12.058 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:11.442+0000 7f325a59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f325c109600 0x7f325c10b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44794/0 (socket says 172.21.15.100:44794) 2024-05-10T11:06:12.058 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:11.442+0000 7f325a59c700 1 -- 172.21.15.100:0/3550172937 learned_addr learned my addr 172.21.15.100:0/3550172937 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:06:12.058 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:11.443+0000 7f325a59c700 1 -- 172.21.15.100:0/3550172937 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f325c10bf30 con 0x7f325c109600 2024-05-10T11:06:12.058 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:11.443+0000 7f325a59c700 1 --2- 172.21.15.100:0/3550172937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f325c109600 0x7f325c10b9f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f324c0098a0 tx=0x7f324c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a80cf56135524a9b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:06:12.058 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:11.443+0000 7f325959a700 1 -- 172.21.15.100:0/3550172937 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f324c004160 con 0x7f325c109600 2024-05-10T11:06:12.058 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:11.443+0000 7f325959a700 1 -- 172.21.15.100:0/3550172937 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f324c004300 con 0x7f325c109600 2024-05-10T11:06:12.058 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:11.443+0000 7f325959a700 1 -- 172.21.15.100:0/3550172937 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f324c0119e0 con 0x7f325c109600 2024-05-10T11:06:12.058 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:11.443+0000 7f3260da5700 1 -- 172.21.15.100:0/3550172937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f325c109600 msgr2=0x7f325c10b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:06:12.058 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:11.443+0000 7f3260da5700 1 --2- 172.21.15.100:0/3550172937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f325c109600 0x7f325c10b9f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f324c0098a0 tx=0x7f324c009bb0 comp rx=0 tx=0).stop 2024-05-10T11:06:12.058 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:11.444+0000 7f3260da5700 1 -- 172.21.15.100:0/3550172937 shutdown_connections 2024-05-10T11:06:12.058 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:11.444+0000 7f3260da5700 1 --2- 172.21.15.100:0/3550172937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f325c109600 0x7f325c10b9f0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:06:12.059 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:11.444+0000 7f3260da5700 1 -- 172.21.15.100:0/3550172937 >> 172.21.15.100:0/3550172937 conn(0x7f325c102fd0 msgr2=0x7f325c105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:06:12.059 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:11.444+0000 7f3260da5700 1 -- 172.21.15.100:0/3550172937 shutdown_connections 2024-05-10T11:06:12.059 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:11.444+0000 7f3260da5700 1 -- 172.21.15.100:0/3550172937 wait complete. 2024-05-10T11:06:12.059 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:11.444+0000 7f3260da5700 1 Processor -- start 2024-05-10T11:06:12.059 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:11.444+0000 7f3260da5700 1 -- start start 2024-05-10T11:06:12.059 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:11.445+0000 7f3260da5700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f325c109600 0x7f325c19c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:06:12.059 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:11.445+0000 7f3260da5700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f325c10ce20 con 0x7f325c109600 2024-05-10T11:06:12.059 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:11.445+0000 7f325a59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f325c109600 0x7f325c19c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:06:12.059 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:11.445+0000 7f325a59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f325c109600 0x7f325c19c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44798/0 (socket says 172.21.15.100:44798) 2024-05-10T11:06:12.059 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:11.445+0000 7f325a59c700 1 -- 172.21.15.100:0/3480502224 learned_addr learned my addr 172.21.15.100:0/3480502224 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:06:12.059 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:11.445+0000 7f325a59c700 1 -- 172.21.15.100:0/3480502224 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f325c19ce70 con 0x7f325c109600 2024-05-10T11:06:12.059 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:11.445+0000 7f325a59c700 1 --2- 172.21.15.100:0/3480502224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f325c109600 0x7f325c19c930 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f324c0119b0 tx=0x7f324c011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:06:12.059 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:11.445+0000 7f324b7fe700 1 -- 172.21.15.100:0/3480502224 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f324c0039f0 con 0x7f325c109600 2024-05-10T11:06:12.059 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:11.445+0000 7f3260da5700 1 -- 172.21.15.100:0/3480502224 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f325c19d160 con 0x7f325c109600 2024-05-10T11:06:12.060 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:11.445+0000 7f324b7fe700 1 -- 172.21.15.100:0/3480502224 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f324c003b90 con 0x7f325c109600 2024-05-10T11:06:12.060 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:11.445+0000 7f324b7fe700 1 -- 172.21.15.100:0/3480502224 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f324c01a5c0 con 0x7f325c109600 2024-05-10T11:06:12.060 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:11.445+0000 7f3260da5700 1 -- 172.21.15.100:0/3480502224 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f325c19fdf0 con 0x7f325c109600 2024-05-10T11:06:12.060 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:11.445+0000 7f324b7fe700 1 -- 172.21.15.100:0/3480502224 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f324c01ac70 con 0x7f325c109600 2024-05-10T11:06:12.060 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:11.446+0000 7f324b7fe700 1 -- 172.21.15.100:0/3480502224 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f324c019700 con 0x7f325c109600 2024-05-10T11:06:12.060 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:11.446+0000 7f3260da5700 1 -- 172.21.15.100:0/3480502224 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f325c064b00 con 0x7f325c109600 2024-05-10T11:06:12.060 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:11.449+0000 7f324b7fe700 1 -- 172.21.15.100:0/3480502224 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f324c032020 con 0x7f325c109600 2024-05-10T11:06:12.060 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:11.514+0000 7f3260da5700 1 -- 172.21.15.100:0/3480502224 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f325c19ffc0 con 0x7f325c109600 2024-05-10T11:06:12.060 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:11.516+0000 7f324b7fe700 1 -- 172.21.15.100:0/3480502224 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f324c024a50 con 0x7f325c109600 2024-05-10T11:06:12.060 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:11.517+0000 7f3260da5700 1 -- 172.21.15.100:0/3480502224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f325c109600 msgr2=0x7f325c19c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:06:12.060 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:11.517+0000 7f3260da5700 1 --2- 172.21.15.100:0/3480502224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f325c109600 0x7f325c19c930 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f324c0119b0 tx=0x7f324c011a90 comp rx=0 tx=0).stop 2024-05-10T11:06:12.060 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:11.518+0000 7f3260da5700 1 -- 172.21.15.100:0/3480502224 shutdown_connections 2024-05-10T11:06:12.060 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:11.518+0000 7f3260da5700 1 --2- 172.21.15.100:0/3480502224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f325c109600 0x7f325c19c930 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:06:12.061 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:11.518+0000 7f3260da5700 1 -- 172.21.15.100:0/3480502224 >> 172.21.15.100:0/3480502224 conn(0x7f325c102fd0 msgr2=0x7f325c103ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:06:12.061 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:11.518+0000 7f3260da5700 1 -- 172.21.15.100:0/3480502224 shutdown_connections 2024-05-10T11:06:12.061 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:11.518+0000 7f3260da5700 1 -- 172.21.15.100:0/3480502224 wait complete. 2024-05-10T11:06:12.061 INFO:teuthology.orchestra.run.smithi100.stdout:Wrote config to /etc/ceph/ceph.conf 2024-05-10T11:06:12.083 INFO:teuthology.orchestra.run.smithi100.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-05-10T11:06:12.083 INFO:teuthology.orchestra.run.smithi100.stdout:Creating mgr... 2024-05-10T11:06:12.083 INFO:teuthology.orchestra.run.smithi100.stdout:Verifying port 0.0.0.0:9283 ... 2024-05-10T11:06:12.084 INFO:teuthology.orchestra.run.smithi100.stdout:Verifying port 0.0.0.0:8765 ... 2024-05-10T11:06:12.317 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mgr.x 2024-05-10T11:06:12.317 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stderr Failed to reset failed state of unit ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mgr.x.service: Unit ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mgr.x.service not loaded. 2024-05-10T11:06:12.470 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-20a542cc-0ebd-11ef-bc98-c7b262605968.target.wants/ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@.service. 2024-05-10T11:06:12.490 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:12 smithi100 systemd[1]: Starting Ceph mgr.x for 20a542cc-0ebd-11ef-bc98-c7b262605968... 2024-05-10T11:06:12.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:12 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/3480502224' entity='client.admin' 2024-05-10T11:06:12.789 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:12 smithi100 podman[30511]: 2024-05-10 11:06:12.627952706 +0000 UTC m=+0.018253814 image pull ee5e4f21ef58047cb638cccd5cfbe8f6f43412b56c94b1d7fbd9cf3cb90a53a5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b 2024-05-10T11:06:12.789 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:12 smithi100 podman[30511]: 2024-05-10 11:06:12.736440187 +0000 UTC m=+0.126741279 container create 81381d9bb2d19d72aa45e62cba5e45e37accf85602ab6b5a526393b06b68e01e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240507, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-05-10T11:06:13.027 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-10T11:06:13.027 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout disabled 2024-05-10T11:06:13.035 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-10T11:06:13.035 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout inactive 2024-05-10T11:06:13.035 INFO:teuthology.orchestra.run.smithi100.stdout:firewalld.service is not enabled 2024-05-10T11:06:13.035 INFO:teuthology.orchestra.run.smithi100.stdout:Not possible to enable service . firewalld.service is not available 2024-05-10T11:06:13.044 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-10T11:06:13.044 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout disabled 2024-05-10T11:06:13.053 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-10T11:06:13.053 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout inactive 2024-05-10T11:06:13.053 INFO:teuthology.orchestra.run.smithi100.stdout:firewalld.service is not enabled 2024-05-10T11:06:13.053 INFO:teuthology.orchestra.run.smithi100.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-05-10T11:06:13.053 INFO:teuthology.orchestra.run.smithi100.stdout:Waiting for mgr to start... 2024-05-10T11:06:13.053 INFO:teuthology.orchestra.run.smithi100.stdout:Waiting for mgr... 2024-05-10T11:06:13.112 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:12 smithi100 podman[30511]: 2024-05-10 11:06:12.947181173 +0000 UTC m=+0.337482278 container init 81381d9bb2d19d72aa45e62cba5e45e37accf85602ab6b5a526393b06b68e01e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, ceph=True) 2024-05-10T11:06:13.112 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:12 smithi100 podman[30511]: 2024-05-10 11:06:12.949888795 +0000 UTC m=+0.340189896 container start 81381d9bb2d19d72aa45e62cba5e45e37accf85602ab6b5a526393b06b68e01e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True) 2024-05-10T11:06:13.112 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:13 smithi100 bash[30511]: 81381d9bb2d19d72aa45e62cba5e45e37accf85602ab6b5a526393b06b68e01e 2024-05-10T11:06:13.112 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:13 smithi100 systemd[1]: Started Ceph mgr.x for 20a542cc-0ebd-11ef-bc98-c7b262605968. 2024-05-10T11:06:13.402 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:13 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:13.111+0000 7f3960dad200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-10T11:06:13.403 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:13 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:13.300+0000 7f3960dad200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-10T11:06:13.403 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:13 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:13.402+0000 7f3960dad200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-10T11:06:13.725 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:13 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/3309073644' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-10T11:06:14.036 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:13 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:13.725+0000 7f3960dad200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-10T11:06:14.036 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:13 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:13.812+0000 7f3960dad200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-10T11:06:14.300 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-05-10T11:06:14.300 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout { 2024-05-10T11:06:14.300 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "fsid": "20a542cc-0ebd-11ef-bc98-c7b262605968", 2024-05-10T11:06:14.300 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "health": { 2024-05-10T11:06:14.300 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-10T11:06:14.301 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-10T11:06:14.301 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-10T11:06:14.301 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-05-10T11:06:14.301 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-10T11:06:14.301 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-10T11:06:14.301 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 0 2024-05-10T11:06:14.301 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ], 2024-05-10T11:06:14.301 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-10T11:06:14.301 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "a" 2024-05-10T11:06:14.301 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ], 2024-05-10T11:06:14.301 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-05-10T11:06:14.301 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-10T11:06:14.301 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T11:06:14.301 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-10T11:06:14.301 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-10T11:06:14.302 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-05-10T11:06:14.302 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-10T11:06:14.302 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T11:06:14.302 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-10T11:06:14.302 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-10T11:06:14.302 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-10T11:06:14.310 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-10T11:06:14.310 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-10T11:06:14.310 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-10T11:06:14.310 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-05-10T11:06:14.310 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-10T11:06:14.310 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-10T11:06:14.310 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-10T11:06:14.311 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-10T11:06:14.311 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-10T11:06:14.311 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-10T11:06:14.311 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-10T11:06:14.311 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-10T11:06:14.311 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-10T11:06:14.311 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-05-10T11:06:14.311 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-10T11:06:14.311 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T11:06:14.311 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-10T11:06:14.311 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-10T11:06:14.311 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-05-10T11:06:14.311 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-10T11:06:14.311 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-10T11:06:14.311 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-10T11:06:14.312 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-10T11:06:14.312 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-10T11:06:14.312 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-10T11:06:14.312 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "restful" 2024-05-10T11:06:14.312 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ], 2024-05-10T11:06:14.312 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-10T11:06:14.312 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-05-10T11:06:14.312 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-10T11:06:14.312 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T11:06:14.312 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "modified": "2024-05-10T11:06:05.455015+0000", 2024-05-10T11:06:14.312 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-10T11:06:14.312 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-05-10T11:06:14.312 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-10T11:06:14.313 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout } 2024-05-10T11:06:14.313 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:13.516+0000 7fb120d03700 1 Processor -- start 2024-05-10T11:06:14.313 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:13.517+0000 7fb120d03700 1 -- start start 2024-05-10T11:06:14.313 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:13.517+0000 7fb120d03700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb11c108b30 0x7fb11c108f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:06:14.313 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:13.517+0000 7fb120d03700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb11c109520 con 0x7fb11c108b30 2024-05-10T11:06:14.313 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:13.517+0000 7fb11a59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb11c108b30 0x7fb11c108f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:06:14.313 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:13.517+0000 7fb11a59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb11c108b30 0x7fb11c108f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44824/0 (socket says 172.21.15.100:44824) 2024-05-10T11:06:14.313 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:13.517+0000 7fb11a59c700 1 -- 172.21.15.100:0/1101126909 learned_addr learned my addr 172.21.15.100:0/1101126909 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:06:14.313 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:13.518+0000 7fb11a59c700 1 -- 172.21.15.100:0/1101126909 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb11c109d30 con 0x7fb11c108b30 2024-05-10T11:06:14.313 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:13.518+0000 7fb11a59c700 1 --2- 172.21.15.100:0/1101126909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb11c108b30 0x7fb11c108f50 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb10c0098a0 tx=0x7fb10c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=602f811d9ccf15ef server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:06:14.313 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:13.518+0000 7fb11959a700 1 -- 172.21.15.100:0/1101126909 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb10c004160 con 0x7fb11c108b30 2024-05-10T11:06:14.313 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:13.518+0000 7fb11959a700 1 -- 172.21.15.100:0/1101126909 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fb10c004300 con 0x7fb11c108b30 2024-05-10T11:06:14.314 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:13.518+0000 7fb11959a700 1 -- 172.21.15.100:0/1101126909 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb10c0119e0 con 0x7fb11c108b30 2024-05-10T11:06:14.314 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:13.518+0000 7fb120d03700 1 -- 172.21.15.100:0/1101126909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb11c108b30 msgr2=0x7fb11c108f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:06:14.314 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:13.519+0000 7fb120d03700 1 --2- 172.21.15.100:0/1101126909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb11c108b30 0x7fb11c108f50 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb10c0098a0 tx=0x7fb10c009bb0 comp rx=0 tx=0).stop 2024-05-10T11:06:14.314 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:13.519+0000 7fb120d03700 1 -- 172.21.15.100:0/1101126909 shutdown_connections 2024-05-10T11:06:14.314 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:13.519+0000 7fb120d03700 1 --2- 172.21.15.100:0/1101126909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb11c108b30 0x7fb11c108f50 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:06:14.314 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:13.519+0000 7fb120d03700 1 -- 172.21.15.100:0/1101126909 >> 172.21.15.100:0/1101126909 conn(0x7fb11c104090 msgr2=0x7fb11c106510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:06:14.314 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:13.519+0000 7fb120d03700 1 -- 172.21.15.100:0/1101126909 shutdown_connections 2024-05-10T11:06:14.314 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:13.519+0000 7fb120d03700 1 -- 172.21.15.100:0/1101126909 wait complete. 2024-05-10T11:06:14.314 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:13.519+0000 7fb120d03700 1 Processor -- start 2024-05-10T11:06:14.314 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:13.519+0000 7fb120d03700 1 -- start start 2024-05-10T11:06:14.314 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:13.520+0000 7fb120d03700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb11c108b30 0x7fb11c19ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:06:14.314 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:13.520+0000 7fb120d03700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb11c10cf60 con 0x7fb11c108b30 2024-05-10T11:06:14.314 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:13.520+0000 7fb11a59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb11c108b30 0x7fb11c19ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:06:14.315 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:13.520+0000 7fb11a59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb11c108b30 0x7fb11c19ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44826/0 (socket says 172.21.15.100:44826) 2024-05-10T11:06:14.315 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:13.520+0000 7fb11a59c700 1 -- 172.21.15.100:0/3309073644 learned_addr learned my addr 172.21.15.100:0/3309073644 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:06:14.315 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:13.520+0000 7fb11a59c700 1 -- 172.21.15.100:0/3309073644 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb11c19cf50 con 0x7fb11c108b30 2024-05-10T11:06:14.315 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:13.520+0000 7fb11a59c700 1 --2- 172.21.15.100:0/3309073644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb11c108b30 0x7fb11c19ca10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb10c006b20 tx=0x7fb10c011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:06:14.315 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:13.520+0000 7fb10b7fe700 1 -- 172.21.15.100:0/3309073644 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb10c003a60 con 0x7fb11c108b30 2024-05-10T11:06:14.315 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:13.520+0000 7fb120d03700 1 -- 172.21.15.100:0/3309073644 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb11c19d1e0 con 0x7fb11c108b30 2024-05-10T11:06:14.315 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:13.520+0000 7fb10b7fe700 1 -- 172.21.15.100:0/3309073644 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fb10c003c00 con 0x7fb11c108b30 2024-05-10T11:06:14.315 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:13.520+0000 7fb10b7fe700 1 -- 172.21.15.100:0/3309073644 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb10c024780 con 0x7fb11c108b30 2024-05-10T11:06:14.315 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:13.520+0000 7fb120d03700 1 -- 172.21.15.100:0/3309073644 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb11c19fed0 con 0x7fb11c108b30 2024-05-10T11:06:14.315 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:13.521+0000 7fb10b7fe700 1 -- 172.21.15.100:0/3309073644 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb10c022020 con 0x7fb11c108b30 2024-05-10T11:06:14.315 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:13.521+0000 7fb10b7fe700 1 -- 172.21.15.100:0/3309073644 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb10c0193c0 con 0x7fb11c108b30 2024-05-10T11:06:14.315 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:13.521+0000 7fb120d03700 1 -- 172.21.15.100:0/3309073644 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb11c19d6c0 con 0x7fb11c108b30 2024-05-10T11:06:14.315 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:13.523+0000 7fb10b7fe700 1 -- 172.21.15.100:0/3309073644 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fb10c042e50 con 0x7fb11c108b30 2024-05-10T11:06:14.315 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:13.596+0000 7fb120d03700 1 -- 172.21.15.100:0/3309073644 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fb11c1a00a0 con 0x7fb11c108b30 2024-05-10T11:06:14.315 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:13.596+0000 7fb10b7fe700 1 -- 172.21.15.100:0/3309073644 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fb10c031020 con 0x7fb11c108b30 2024-05-10T11:06:14.316 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:13.597+0000 7fb120d03700 1 -- 172.21.15.100:0/3309073644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb11c108b30 msgr2=0x7fb11c19ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:06:14.316 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:13.597+0000 7fb120d03700 1 --2- 172.21.15.100:0/3309073644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb11c108b30 0x7fb11c19ca10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb10c006b20 tx=0x7fb10c011a60 comp rx=0 tx=0).stop 2024-05-10T11:06:14.316 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:13.597+0000 7fb120d03700 1 -- 172.21.15.100:0/3309073644 shutdown_connections 2024-05-10T11:06:14.316 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:13.597+0000 7fb120d03700 1 --2- 172.21.15.100:0/3309073644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb11c108b30 0x7fb11c19ca10 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:06:14.316 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:13.597+0000 7fb120d03700 1 -- 172.21.15.100:0/3309073644 >> 172.21.15.100:0/3309073644 conn(0x7fb11c104090 msgr2=0x7fb11c104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:06:14.316 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:13.597+0000 7fb120d03700 1 -- 172.21.15.100:0/3309073644 shutdown_connections 2024-05-10T11:06:14.316 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:13.597+0000 7fb120d03700 1 -- 172.21.15.100:0/3309073644 wait complete. 2024-05-10T11:06:14.316 INFO:teuthology.orchestra.run.smithi100.stdout:mgr not available, waiting (1/15)... 2024-05-10T11:06:15.288 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:14 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:14.998+0000 7f3960dad200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-10T11:06:15.288 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:15 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:15.164+0000 7f3960dad200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-10T11:06:15.613 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:15 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:15.331+0000 7f3960dad200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-10T11:06:15.901 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:15 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:15.613+0000 7f3960dad200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-10T11:06:15.901 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:15 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:15.703+0000 7f3960dad200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-10T11:06:15.901 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:15 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:15.901+0000 7f3960dad200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-10T11:06:16.288 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:16 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:16.095+0000 7f3960dad200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-10T11:06:17.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:16 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/774268397' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-10T11:06:17.038 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:16 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:16.716+0000 7f3960dad200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-10T11:06:17.038 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:16 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:16.948+0000 7f3960dad200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-10T11:06:17.432 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-05-10T11:06:17.432 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout { 2024-05-10T11:06:17.432 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "fsid": "20a542cc-0ebd-11ef-bc98-c7b262605968", 2024-05-10T11:06:17.432 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "health": { 2024-05-10T11:06:17.432 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-10T11:06:17.432 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-10T11:06:17.432 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-10T11:06:17.433 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-05-10T11:06:17.433 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-10T11:06:17.433 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-10T11:06:17.433 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 0 2024-05-10T11:06:17.433 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ], 2024-05-10T11:06:17.433 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-10T11:06:17.433 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "a" 2024-05-10T11:06:17.433 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ], 2024-05-10T11:06:17.433 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-05-10T11:06:17.433 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-10T11:06:17.433 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T11:06:17.433 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-10T11:06:17.433 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-10T11:06:17.433 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-05-10T11:06:17.433 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-10T11:06:17.434 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T11:06:17.434 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-10T11:06:17.434 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-10T11:06:17.434 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-10T11:06:17.434 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-10T11:06:17.434 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-10T11:06:17.434 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-10T11:06:17.434 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-05-10T11:06:17.434 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-10T11:06:17.434 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-10T11:06:17.434 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-10T11:06:17.434 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-10T11:06:17.434 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-10T11:06:17.434 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-10T11:06:17.434 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-10T11:06:17.435 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-10T11:06:17.435 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-10T11:06:17.435 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-05-10T11:06:17.446 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-10T11:06:17.446 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T11:06:17.446 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-10T11:06:17.446 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-10T11:06:17.446 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-05-10T11:06:17.446 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-10T11:06:17.446 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-10T11:06:17.446 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-10T11:06:17.446 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-10T11:06:17.446 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-10T11:06:17.446 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-10T11:06:17.446 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "restful" 2024-05-10T11:06:17.447 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ], 2024-05-10T11:06:17.447 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-10T11:06:17.447 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-05-10T11:06:17.447 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-10T11:06:17.447 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T11:06:17.447 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "modified": "2024-05-10T11:06:05.455015+0000", 2024-05-10T11:06:17.447 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-10T11:06:17.447 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-05-10T11:06:17.447 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-10T11:06:17.447 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout } 2024-05-10T11:06:17.447 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:16.757+0000 7f720dc7d700 1 Processor -- start 2024-05-10T11:06:17.447 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:16.757+0000 7f720dc7d700 1 -- start start 2024-05-10T11:06:17.447 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:16.757+0000 7f720dc7d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7208107a70 0x7f7208107e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:06:17.447 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:16.757+0000 7f720dc7d700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7208108460 con 0x7f7208107a70 2024-05-10T11:06:17.447 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:16.757+0000 7f72077fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7208107a70 0x7f7208107e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:06:17.448 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:16.757+0000 7f72077fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7208107a70 0x7f7208107e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55884/0 (socket says 172.21.15.100:55884) 2024-05-10T11:06:17.448 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:16.757+0000 7f72077fe700 1 -- 172.21.15.100:0/1268133419 learned_addr learned my addr 172.21.15.100:0/1268133419 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:06:17.448 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:16.758+0000 7f72077fe700 1 -- 172.21.15.100:0/1268133419 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7208108c70 con 0x7f7208107a70 2024-05-10T11:06:17.448 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:16.758+0000 7f72077fe700 1 --2- 172.21.15.100:0/1268133419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7208107a70 0x7f7208107e90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f71fc0098a0 tx=0x7f71fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78bb7996556aed17 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:06:17.448 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:16.758+0000 7f72067fc700 1 -- 172.21.15.100:0/1268133419 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71fc004160 con 0x7f7208107a70 2024-05-10T11:06:17.448 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:16.758+0000 7f72067fc700 1 -- 172.21.15.100:0/1268133419 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f71fc004300 con 0x7f7208107a70 2024-05-10T11:06:17.448 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:16.758+0000 7f72067fc700 1 -- 172.21.15.100:0/1268133419 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71fc0119e0 con 0x7f7208107a70 2024-05-10T11:06:17.448 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:16.759+0000 7f720dc7d700 1 -- 172.21.15.100:0/1268133419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7208107a70 msgr2=0x7f7208107e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:06:17.448 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:16.759+0000 7f720dc7d700 1 --2- 172.21.15.100:0/1268133419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7208107a70 0x7f7208107e90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f71fc0098a0 tx=0x7f71fc009bb0 comp rx=0 tx=0).stop 2024-05-10T11:06:17.448 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:16.759+0000 7f720dc7d700 1 -- 172.21.15.100:0/1268133419 shutdown_connections 2024-05-10T11:06:17.448 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:16.759+0000 7f720dc7d700 1 --2- 172.21.15.100:0/1268133419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7208107a70 0x7f7208107e90 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:06:17.448 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:16.759+0000 7f720dc7d700 1 -- 172.21.15.100:0/1268133419 >> 172.21.15.100:0/1268133419 conn(0x7f7208102fd0 msgr2=0x7f7208105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:06:17.449 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:16.759+0000 7f720dc7d700 1 -- 172.21.15.100:0/1268133419 shutdown_connections 2024-05-10T11:06:17.449 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:16.759+0000 7f720dc7d700 1 -- 172.21.15.100:0/1268133419 wait complete. 2024-05-10T11:06:17.449 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:16.759+0000 7f720dc7d700 1 Processor -- start 2024-05-10T11:06:17.449 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:16.760+0000 7f720dc7d700 1 -- start start 2024-05-10T11:06:17.449 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:16.760+0000 7f720dc7d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7208107a70 0x7f720819ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:06:17.449 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:16.760+0000 7f720dc7d700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f720810bea0 con 0x7f7208107a70 2024-05-10T11:06:17.449 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:16.760+0000 7f72077fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7208107a70 0x7f720819ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:06:17.449 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:16.760+0000 7f72077fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7208107a70 0x7f720819ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55890/0 (socket says 172.21.15.100:55890) 2024-05-10T11:06:17.449 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:16.760+0000 7f72077fe700 1 -- 172.21.15.100:0/774268397 learned_addr learned my addr 172.21.15.100:0/774268397 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:06:17.449 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:16.760+0000 7f72077fe700 1 -- 172.21.15.100:0/774268397 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f720819f130 con 0x7f7208107a70 2024-05-10T11:06:17.449 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:16.760+0000 7f72077fe700 1 --2- 172.21.15.100:0/774268397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7208107a70 0x7f720819ebf0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f71fc0119b0 tx=0x7f71fc011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:06:17.449 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:16.761+0000 7f7204ff9700 1 -- 172.21.15.100:0/774268397 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71fc003af0 con 0x7f7208107a70 2024-05-10T11:06:17.449 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:16.761+0000 7f7204ff9700 1 -- 172.21.15.100:0/774268397 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f71fc01a8a0 con 0x7f7208107a70 2024-05-10T11:06:17.449 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:16.761+0000 7f720dc7d700 1 -- 172.21.15.100:0/774268397 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f720819f3c0 con 0x7f7208107a70 2024-05-10T11:06:17.450 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:16.761+0000 7f7204ff9700 1 -- 172.21.15.100:0/774268397 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71fc024830 con 0x7f7208107a70 2024-05-10T11:06:17.450 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:16.761+0000 7f720dc7d700 1 -- 172.21.15.100:0/774268397 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72081a20b0 con 0x7f7208107a70 2024-05-10T11:06:17.450 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:16.761+0000 7f7204ff9700 1 -- 172.21.15.100:0/774268397 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f71fc022020 con 0x7f7208107a70 2024-05-10T11:06:17.450 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:16.761+0000 7f7204ff9700 1 -- 172.21.15.100:0/774268397 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f71fc02dda0 con 0x7f7208107a70 2024-05-10T11:06:17.450 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:16.762+0000 7f720dc7d700 1 -- 172.21.15.100:0/774268397 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7208064b00 con 0x7f7208107a70 2024-05-10T11:06:17.450 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:16.764+0000 7f7204ff9700 1 -- 172.21.15.100:0/774268397 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f71fc01aa50 con 0x7f7208107a70 2024-05-10T11:06:17.450 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:16.836+0000 7f720dc7d700 1 -- 172.21.15.100:0/774268397 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f72080008d0 con 0x7f7208107a70 2024-05-10T11:06:17.450 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:16.837+0000 7f7204ff9700 1 -- 172.21.15.100:0/774268397 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f71fc032030 con 0x7f7208107a70 2024-05-10T11:06:17.450 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:16.837+0000 7f720dc7d700 1 -- 172.21.15.100:0/774268397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7208107a70 msgr2=0x7f720819ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:06:17.450 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:16.837+0000 7f720dc7d700 1 --2- 172.21.15.100:0/774268397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7208107a70 0x7f720819ebf0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f71fc0119b0 tx=0x7f71fc011a90 comp rx=0 tx=0).stop 2024-05-10T11:06:17.450 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:16.838+0000 7f720dc7d700 1 -- 172.21.15.100:0/774268397 shutdown_connections 2024-05-10T11:06:17.450 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:16.838+0000 7f720dc7d700 1 --2- 172.21.15.100:0/774268397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7208107a70 0x7f720819ebf0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:06:17.450 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:16.838+0000 7f720dc7d700 1 -- 172.21.15.100:0/774268397 >> 172.21.15.100:0/774268397 conn(0x7f7208102fd0 msgr2=0x7f7208103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:06:17.451 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:16.838+0000 7f720dc7d700 1 -- 172.21.15.100:0/774268397 shutdown_connections 2024-05-10T11:06:17.451 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:16.838+0000 7f720dc7d700 1 -- 172.21.15.100:0/774268397 wait complete. 2024-05-10T11:06:17.451 INFO:teuthology.orchestra.run.smithi100.stdout:mgr not available, waiting (2/15)... 2024-05-10T11:06:17.788 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:17 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:17.374+0000 7f3960dad200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-10T11:06:17.788 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:17 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:17.474+0000 7f3960dad200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-10T11:06:18.199 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:17 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:17.849+0000 7f3960dad200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-10T11:06:18.199 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:18 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:18.198+0000 7f3960dad200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-10T11:06:18.531 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:18 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:18.291+0000 7f3960dad200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-10T11:06:18.788 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:18 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:18.531+0000 7f3960dad200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-10T11:06:18.788 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:18 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:18.664+0000 7f3960dad200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-10T11:06:18.788 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:18 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:18.758+0000 7f3960dad200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-10T11:06:19.538 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:19 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:19.085+0000 7f3960dad200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-10T11:06:19.538 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:19 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:19.180+0000 7f3960dad200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-10T11:06:20.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:20 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/3539758818' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-10T11:06:20.288 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:19 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:19.809+0000 7f3960dad200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-10T11:06:20.288 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:20 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:20.038+0000 7f3960dad200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-10T11:06:20.661 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-05-10T11:06:20.661 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout { 2024-05-10T11:06:20.661 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "fsid": "20a542cc-0ebd-11ef-bc98-c7b262605968", 2024-05-10T11:06:20.661 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "health": { 2024-05-10T11:06:20.662 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-10T11:06:20.662 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-10T11:06:20.662 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-10T11:06:20.662 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-05-10T11:06:20.662 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-10T11:06:20.662 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-10T11:06:20.662 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 0 2024-05-10T11:06:20.662 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ], 2024-05-10T11:06:20.662 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-10T11:06:20.662 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "a" 2024-05-10T11:06:20.662 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ], 2024-05-10T11:06:20.662 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-05-10T11:06:20.662 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-10T11:06:20.662 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T11:06:20.663 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-10T11:06:20.663 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-10T11:06:20.663 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-05-10T11:06:20.663 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-10T11:06:20.663 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T11:06:20.663 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-10T11:06:20.663 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-10T11:06:20.663 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-10T11:06:20.663 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-10T11:06:20.663 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-10T11:06:20.663 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-10T11:06:20.663 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-05-10T11:06:20.663 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-10T11:06:20.663 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-10T11:06:20.663 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-10T11:06:20.663 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-10T11:06:20.664 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-10T11:06:20.664 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-10T11:06:20.664 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-10T11:06:20.664 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-10T11:06:20.664 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-10T11:06:20.664 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-05-10T11:06:20.664 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-10T11:06:20.664 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T11:06:20.664 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-10T11:06:20.664 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-10T11:06:20.664 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-05-10T11:06:20.664 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-10T11:06:20.665 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-10T11:06:20.665 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-10T11:06:20.665 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-10T11:06:20.665 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-10T11:06:20.665 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-10T11:06:20.665 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "restful" 2024-05-10T11:06:20.665 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ], 2024-05-10T11:06:20.665 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-10T11:06:20.665 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-05-10T11:06:20.665 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-10T11:06:20.665 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T11:06:20.665 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "modified": "2024-05-10T11:06:05.455015+0000", 2024-05-10T11:06:20.665 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-10T11:06:20.665 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-05-10T11:06:20.665 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-10T11:06:20.666 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout } 2024-05-10T11:06:20.666 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:19.903+0000 7f84dd923700 1 Processor -- start 2024-05-10T11:06:20.666 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:19.903+0000 7f84dd923700 1 -- start start 2024-05-10T11:06:20.666 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:19.903+0000 7f84dd923700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84d8109580 0x7f84d810b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:06:20.666 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:19.903+0000 7f84dd923700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f84d80769c0 con 0x7f84d8109580 2024-05-10T11:06:20.666 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:19.903+0000 7f84d6ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84d8109580 0x7f84d810b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:06:20.666 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:19.903+0000 7f84d6ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84d8109580 0x7f84d810b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55892/0 (socket says 172.21.15.100:55892) 2024-05-10T11:06:20.666 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:19.903+0000 7f84d6ffd700 1 -- 172.21.15.100:0/2988205780 learned_addr learned my addr 172.21.15.100:0/2988205780 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:06:20.666 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:19.904+0000 7f84d6ffd700 1 -- 172.21.15.100:0/2988205780 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84d810beb0 con 0x7f84d8109580 2024-05-10T11:06:20.666 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:19.904+0000 7f84d6ffd700 1 --2- 172.21.15.100:0/2988205780 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84d8109580 0x7f84d810b970 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f84cc0098a0 tx=0x7f84cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=345272f7d0aadca8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:06:20.666 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:19.904+0000 7f84d5ffb700 1 -- 172.21.15.100:0/2988205780 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f84cc004160 con 0x7f84d8109580 2024-05-10T11:06:20.666 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:19.904+0000 7f84d5ffb700 1 -- 172.21.15.100:0/2988205780 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f84cc004300 con 0x7f84d8109580 2024-05-10T11:06:20.666 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:19.904+0000 7f84d5ffb700 1 -- 172.21.15.100:0/2988205780 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f84cc0119e0 con 0x7f84d8109580 2024-05-10T11:06:20.667 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:19.905+0000 7f84dd923700 1 -- 172.21.15.100:0/2988205780 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84d8109580 msgr2=0x7f84d810b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:06:20.667 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:19.905+0000 7f84dd923700 1 --2- 172.21.15.100:0/2988205780 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84d8109580 0x7f84d810b970 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f84cc0098a0 tx=0x7f84cc009bb0 comp rx=0 tx=0).stop 2024-05-10T11:06:20.667 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:19.905+0000 7f84dd923700 1 -- 172.21.15.100:0/2988205780 shutdown_connections 2024-05-10T11:06:20.667 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:19.905+0000 7f84dd923700 1 --2- 172.21.15.100:0/2988205780 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84d8109580 0x7f84d810b970 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:06:20.667 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:19.905+0000 7f84dd923700 1 -- 172.21.15.100:0/2988205780 >> 172.21.15.100:0/2988205780 conn(0x7f84d8102fd0 msgr2=0x7f84d81053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:06:20.667 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:19.905+0000 7f84dd923700 1 -- 172.21.15.100:0/2988205780 shutdown_connections 2024-05-10T11:06:20.667 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:19.905+0000 7f84dd923700 1 -- 172.21.15.100:0/2988205780 wait complete. 2024-05-10T11:06:20.667 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:19.905+0000 7f84dd923700 1 Processor -- start 2024-05-10T11:06:20.667 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:19.906+0000 7f84dd923700 1 -- start start 2024-05-10T11:06:20.667 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:19.906+0000 7f84dd923700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84d8109580 0x7f84d819eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:06:20.667 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:19.906+0000 7f84dd923700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f84d810cda0 con 0x7f84d8109580 2024-05-10T11:06:20.667 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:19.906+0000 7f84d6ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84d8109580 0x7f84d819eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:06:20.667 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:19.906+0000 7f84d6ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84d8109580 0x7f84d819eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55908/0 (socket says 172.21.15.100:55908) 2024-05-10T11:06:20.667 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:19.906+0000 7f84d6ffd700 1 -- 172.21.15.100:0/3539758818 learned_addr learned my addr 172.21.15.100:0/3539758818 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:06:20.668 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:19.906+0000 7f84d6ffd700 1 -- 172.21.15.100:0/3539758818 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84d819f0c0 con 0x7f84d8109580 2024-05-10T11:06:20.668 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:19.906+0000 7f84d6ffd700 1 --2- 172.21.15.100:0/3539758818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84d8109580 0x7f84d819eb80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f84cc0116a0 tx=0x7f84cc0046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:06:20.668 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:19.906+0000 7f84dc921700 1 -- 172.21.15.100:0/3539758818 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f84cc003b70 con 0x7f84d8109580 2024-05-10T11:06:20.668 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:19.906+0000 7f84dc921700 1 -- 172.21.15.100:0/3539758818 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f84cc01a8a0 con 0x7f84d8109580 2024-05-10T11:06:20.668 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:19.906+0000 7f84dd923700 1 -- 172.21.15.100:0/3539758818 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84d819f3b0 con 0x7f84d8109580 2024-05-10T11:06:20.668 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:19.906+0000 7f84dc921700 1 -- 172.21.15.100:0/3539758818 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f84cc024890 con 0x7f84d8109580 2024-05-10T11:06:20.668 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:19.906+0000 7f84dd923700 1 -- 172.21.15.100:0/3539758818 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84d81a2040 con 0x7f84d8109580 2024-05-10T11:06:20.668 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:19.907+0000 7f84dc921700 1 -- 172.21.15.100:0/3539758818 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f84cc022020 con 0x7f84d8109580 2024-05-10T11:06:20.668 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:19.907+0000 7f84dc921700 1 -- 172.21.15.100:0/3539758818 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f84cc019620 con 0x7f84d8109580 2024-05-10T11:06:20.668 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:19.907+0000 7f84dd923700 1 -- 172.21.15.100:0/3539758818 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84d8064b00 con 0x7f84d8109580 2024-05-10T11:06:20.668 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:19.909+0000 7f84dc921700 1 -- 172.21.15.100:0/3539758818 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f84cc0094f0 con 0x7f84d8109580 2024-05-10T11:06:20.668 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:19.982+0000 7f84dd923700 1 -- 172.21.15.100:0/3539758818 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f84d81a2660 con 0x7f84d8109580 2024-05-10T11:06:20.668 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:19.982+0000 7f84dc921700 1 -- 172.21.15.100:0/3539758818 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f84cc01aa50 con 0x7f84d8109580 2024-05-10T11:06:20.668 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:19.983+0000 7f84dd923700 1 -- 172.21.15.100:0/3539758818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84d8109580 msgr2=0x7f84d819eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:06:20.668 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:19.983+0000 7f84dd923700 1 --2- 172.21.15.100:0/3539758818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84d8109580 0x7f84d819eb80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f84cc0116a0 tx=0x7f84cc0046a0 comp rx=0 tx=0).stop 2024-05-10T11:06:20.669 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:19.983+0000 7f84dd923700 1 -- 172.21.15.100:0/3539758818 shutdown_connections 2024-05-10T11:06:20.669 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:19.983+0000 7f84dd923700 1 --2- 172.21.15.100:0/3539758818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84d8109580 0x7f84d819eb80 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:06:20.669 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:19.983+0000 7f84dd923700 1 -- 172.21.15.100:0/3539758818 >> 172.21.15.100:0/3539758818 conn(0x7f84d8102fd0 msgr2=0x7f84d8103ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:06:20.669 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:19.983+0000 7f84dd923700 1 -- 172.21.15.100:0/3539758818 shutdown_connections 2024-05-10T11:06:20.669 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:19.983+0000 7f84dd923700 1 -- 172.21.15.100:0/3539758818 wait complete. 2024-05-10T11:06:20.669 INFO:teuthology.orchestra.run.smithi100.stdout:mgr not available, waiting (3/15)... 2024-05-10T11:06:21.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:21 smithi100 ceph-mon[30296]: Activating manager daemon x 2024-05-10T11:06:21.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:21 smithi100 ceph-mon[30296]: mgrmap e2: x(active, starting, since 0.00333934s) 2024-05-10T11:06:21.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:21 smithi100 ceph-mon[30296]: from='mgr.14100 172.21.15.100:0/899838633' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-10T11:06:21.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:21 smithi100 ceph-mon[30296]: from='mgr.14100 172.21.15.100:0/899838633' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-10T11:06:21.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:21 smithi100 ceph-mon[30296]: from='mgr.14100 172.21.15.100:0/899838633' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-10T11:06:21.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:21 smithi100 ceph-mon[30296]: from='mgr.14100 172.21.15.100:0/899838633' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T11:06:21.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:21 smithi100 ceph-mon[30296]: from='mgr.14100 172.21.15.100:0/899838633' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-10T11:06:21.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:21 smithi100 ceph-mon[30296]: Manager daemon x is now available 2024-05-10T11:06:21.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:21 smithi100 ceph-mon[30296]: from='mgr.14100 172.21.15.100:0/899838633' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-10T11:06:21.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:21 smithi100 ceph-mon[30296]: from='mgr.14100 172.21.15.100:0/899838633' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-10T11:06:21.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:21 smithi100 ceph-mon[30296]: from='mgr.14100 172.21.15.100:0/899838633' entity='mgr.x' 2024-05-10T11:06:21.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:21 smithi100 ceph-mon[30296]: from='mgr.14100 172.21.15.100:0/899838633' entity='mgr.x' 2024-05-10T11:06:21.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:21 smithi100 ceph-mon[30296]: from='mgr.14100 172.21.15.100:0/899838633' entity='mgr.x' 2024-05-10T11:06:22.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:22 smithi100 ceph-mon[30296]: mgrmap e3: x(active, since 1.00628s) 2024-05-10T11:06:23.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:23 smithi100 ceph-mon[30296]: mgrmap e4: x(active, since 2s) 2024-05-10T11:06:23.977 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-05-10T11:06:23.977 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout { 2024-05-10T11:06:23.977 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "fsid": "20a542cc-0ebd-11ef-bc98-c7b262605968", 2024-05-10T11:06:23.977 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "health": { 2024-05-10T11:06:23.978 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-10T11:06:23.978 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-10T11:06:23.978 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-10T11:06:23.978 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-05-10T11:06:23.978 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-10T11:06:23.978 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-10T11:06:23.978 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 0 2024-05-10T11:06:23.978 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ], 2024-05-10T11:06:23.978 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-10T11:06:23.978 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "a" 2024-05-10T11:06:23.978 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ], 2024-05-10T11:06:23.978 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-05-10T11:06:23.978 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-10T11:06:23.979 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T11:06:23.979 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-10T11:06:23.979 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-10T11:06:23.979 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-05-10T11:06:23.979 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-10T11:06:23.979 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T11:06:23.979 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-10T11:06:23.979 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-10T11:06:23.979 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-10T11:06:23.989 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-10T11:06:23.989 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-10T11:06:23.989 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-10T11:06:23.989 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-05-10T11:06:23.989 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-10T11:06:23.989 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-10T11:06:23.989 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-10T11:06:23.989 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-10T11:06:23.989 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-10T11:06:23.989 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-10T11:06:23.990 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-10T11:06:23.990 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-10T11:06:23.990 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-10T11:06:23.990 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-05-10T11:06:23.990 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-10T11:06:23.990 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T11:06:23.990 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-10T11:06:23.990 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-10T11:06:23.990 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-05-10T11:06:23.990 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-10T11:06:23.990 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-10T11:06:23.990 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-10T11:06:23.990 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-10T11:06:23.990 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-10T11:06:23.991 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-10T11:06:23.991 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "restful" 2024-05-10T11:06:23.991 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ], 2024-05-10T11:06:23.991 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-10T11:06:23.991 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-05-10T11:06:23.991 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-10T11:06:23.991 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T11:06:23.991 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "modified": "2024-05-10T11:06:05.455015+0000", 2024-05-10T11:06:23.991 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-10T11:06:23.991 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-05-10T11:06:23.991 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-10T11:06:23.991 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout } 2024-05-10T11:06:23.991 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:23.136+0000 7ffb6d901700 1 Processor -- start 2024-05-10T11:06:23.991 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:23.136+0000 7ffb6d901700 1 -- start start 2024-05-10T11:06:23.992 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:23.136+0000 7ffb6d901700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb681079f0 0x7ffb68107e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:06:23.992 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:23.136+0000 7ffb6d901700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ffb681083e0 con 0x7ffb681079f0 2024-05-10T11:06:23.992 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:23.136+0000 7ffb66ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb681079f0 0x7ffb68107e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:06:23.992 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:23.136+0000 7ffb66ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb681079f0 0x7ffb68107e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55976/0 (socket says 172.21.15.100:55976) 2024-05-10T11:06:23.992 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:23.136+0000 7ffb66ffd700 1 -- 172.21.15.100:0/3138943243 learned_addr learned my addr 172.21.15.100:0/3138943243 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:06:23.992 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:23.136+0000 7ffb66ffd700 1 -- 172.21.15.100:0/3138943243 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb68108bf0 con 0x7ffb681079f0 2024-05-10T11:06:23.992 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:23.137+0000 7ffb66ffd700 1 --2- 172.21.15.100:0/3138943243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb681079f0 0x7ffb68107e10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ffb580098a0 tx=0x7ffb58009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41b4f6819874cfee server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:06:23.992 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:23.137+0000 7ffb65ffb700 1 -- 172.21.15.100:0/3138943243 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb58004160 con 0x7ffb681079f0 2024-05-10T11:06:23.992 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:23.137+0000 7ffb65ffb700 1 -- 172.21.15.100:0/3138943243 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7ffb58004300 con 0x7ffb681079f0 2024-05-10T11:06:23.992 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:23.137+0000 7ffb65ffb700 1 -- 172.21.15.100:0/3138943243 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb580119e0 con 0x7ffb681079f0 2024-05-10T11:06:23.992 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:23.137+0000 7ffb6d901700 1 -- 172.21.15.100:0/3138943243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb681079f0 msgr2=0x7ffb68107e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:06:23.992 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:23.137+0000 7ffb6d901700 1 --2- 172.21.15.100:0/3138943243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb681079f0 0x7ffb68107e10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ffb580098a0 tx=0x7ffb58009bb0 comp rx=0 tx=0).stop 2024-05-10T11:06:23.992 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:23.137+0000 7ffb6d901700 1 -- 172.21.15.100:0/3138943243 shutdown_connections 2024-05-10T11:06:23.992 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:23.137+0000 7ffb6d901700 1 --2- 172.21.15.100:0/3138943243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb681079f0 0x7ffb68107e10 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:06:23.992 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:23.137+0000 7ffb6d901700 1 -- 172.21.15.100:0/3138943243 >> 172.21.15.100:0/3138943243 conn(0x7ffb68102fd0 msgr2=0x7ffb681053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:06:23.992 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:23.137+0000 7ffb6d901700 1 -- 172.21.15.100:0/3138943243 shutdown_connections 2024-05-10T11:06:23.993 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:23.138+0000 7ffb6d901700 1 -- 172.21.15.100:0/3138943243 wait complete. 2024-05-10T11:06:23.993 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:23.138+0000 7ffb6d901700 1 Processor -- start 2024-05-10T11:06:23.993 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:23.138+0000 7ffb6d901700 1 -- start start 2024-05-10T11:06:23.993 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:23.138+0000 7ffb6d901700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb681079f0 0x7ffb6819ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:06:23.993 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:23.138+0000 7ffb6d901700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ffb6810be20 con 0x7ffb681079f0 2024-05-10T11:06:23.993 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:23.139+0000 7ffb66ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb681079f0 0x7ffb6819ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:06:23.993 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:23.139+0000 7ffb66ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb681079f0 0x7ffb6819ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55980/0 (socket says 172.21.15.100:55980) 2024-05-10T11:06:23.993 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:23.139+0000 7ffb66ffd700 1 -- 172.21.15.100:0/1981878855 learned_addr learned my addr 172.21.15.100:0/1981878855 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:06:23.993 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:23.139+0000 7ffb66ffd700 1 -- 172.21.15.100:0/1981878855 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb6819cfc0 con 0x7ffb681079f0 2024-05-10T11:06:23.993 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:23.139+0000 7ffb66ffd700 1 --2- 172.21.15.100:0/1981878855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb681079f0 0x7ffb6819ca80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ffb58006b20 tx=0x7ffb58011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:06:23.993 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:23.139+0000 7ffb6c8ff700 1 -- 172.21.15.100:0/1981878855 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb58003a60 con 0x7ffb681079f0 2024-05-10T11:06:23.993 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:23.139+0000 7ffb6c8ff700 1 -- 172.21.15.100:0/1981878855 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7ffb58003c00 con 0x7ffb681079f0 2024-05-10T11:06:23.993 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:23.139+0000 7ffb6d901700 1 -- 172.21.15.100:0/1981878855 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb6819d250 con 0x7ffb681079f0 2024-05-10T11:06:23.993 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:23.139+0000 7ffb6c8ff700 1 -- 172.21.15.100:0/1981878855 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb58024780 con 0x7ffb681079f0 2024-05-10T11:06:23.993 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:23.139+0000 7ffb6d901700 1 -- 172.21.15.100:0/1981878855 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb6819fef0 con 0x7ffb681079f0 2024-05-10T11:06:23.993 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:23.140+0000 7ffb6c8ff700 1 -- 172.21.15.100:0/1981878855 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7ffb58022020 con 0x7ffb681079f0 2024-05-10T11:06:23.994 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:23.140+0000 7ffb6c8ff700 1 --2- 172.21.15.100:0/1981878855 >> [v2:172.21.15.100:6800/1086450256,v1:172.21.15.100:6801/1086450256] conn(0x7ffb5003e420 0x7ffb500408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:06:23.994 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:23.140+0000 7ffb6c8ff700 1 -- 172.21.15.100:0/1981878855 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ffb58053740 con 0x7ffb681079f0 2024-05-10T11:06:23.994 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:23.140+0000 7ffb6d901700 1 -- 172.21.15.100:0/1981878855 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb68064b00 con 0x7ffb681079f0 2024-05-10T11:06:23.994 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:23.140+0000 7ffb667fc700 1 --2- 172.21.15.100:0/1981878855 >> [v2:172.21.15.100:6800/1086450256,v1:172.21.15.100:6801/1086450256] conn(0x7ffb5003e420 0x7ffb500408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:06:23.994 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:23.141+0000 7ffb667fc700 1 --2- 172.21.15.100:0/1981878855 >> [v2:172.21.15.100:6800/1086450256,v1:172.21.15.100:6801/1086450256] conn(0x7ffb5003e420 0x7ffb500408e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c006fd0 tx=0x7ffb5c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:06:23.994 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:23.146+0000 7ffb6c8ff700 1 -- 172.21.15.100:0/1981878855 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7ffb5800e260 con 0x7ffb681079f0 2024-05-10T11:06:23.994 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:23.435+0000 7ffb6d901700 1 -- 172.21.15.100:0/1981878855 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7ffb6802e780 con 0x7ffb681079f0 2024-05-10T11:06:23.994 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:23.435+0000 7ffb6c8ff700 1 -- 172.21.15.100:0/1981878855 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7ffb58053420 con 0x7ffb681079f0 2024-05-10T11:06:23.994 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:23.437+0000 7ffb6d901700 1 -- 172.21.15.100:0/1981878855 >> [v2:172.21.15.100:6800/1086450256,v1:172.21.15.100:6801/1086450256] conn(0x7ffb5003e420 msgr2=0x7ffb500408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:06:23.994 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:23.437+0000 7ffb6d901700 1 --2- 172.21.15.100:0/1981878855 >> [v2:172.21.15.100:6800/1086450256,v1:172.21.15.100:6801/1086450256] conn(0x7ffb5003e420 0x7ffb500408e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c006fd0 tx=0x7ffb5c006e40 comp rx=0 tx=0).stop 2024-05-10T11:06:23.994 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:23.437+0000 7ffb6d901700 1 -- 172.21.15.100:0/1981878855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb681079f0 msgr2=0x7ffb6819ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:06:23.994 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:23.437+0000 7ffb6d901700 1 --2- 172.21.15.100:0/1981878855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb681079f0 0x7ffb6819ca80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ffb58006b20 tx=0x7ffb58011a60 comp rx=0 tx=0).stop 2024-05-10T11:06:23.994 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:23.437+0000 7ffb6d901700 1 -- 172.21.15.100:0/1981878855 shutdown_connections 2024-05-10T11:06:23.994 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:23.437+0000 7ffb6d901700 1 --2- 172.21.15.100:0/1981878855 >> [v2:172.21.15.100:6800/1086450256,v1:172.21.15.100:6801/1086450256] conn(0x7ffb5003e420 0x7ffb500408e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:06:23.995 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:23.437+0000 7ffb6d901700 1 --2- 172.21.15.100:0/1981878855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb681079f0 0x7ffb6819ca80 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:06:23.995 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:23.437+0000 7ffb6d901700 1 -- 172.21.15.100:0/1981878855 >> 172.21.15.100:0/1981878855 conn(0x7ffb68102fd0 msgr2=0x7ffb68103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:06:23.995 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:23.438+0000 7ffb6d901700 1 -- 172.21.15.100:0/1981878855 shutdown_connections 2024-05-10T11:06:23.995 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:23.438+0000 7ffb6d901700 1 -- 172.21.15.100:0/1981878855 wait complete. 2024-05-10T11:06:23.995 INFO:teuthology.orchestra.run.smithi100.stdout:mgr is available 2024-05-10T11:06:24.102 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:24 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/1981878855' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-10T11:06:25.162 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-05-10T11:06:25.162 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout [global] 2024-05-10T11:06:25.163 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout fsid = 20a542cc-0ebd-11ef-bc98-c7b262605968 2024-05-10T11:06:25.163 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-10T11:06:25.163 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-10T11:06:25.163 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-10T11:06:25.163 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-10T11:06:25.163 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-10T11:06:25.163 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-05-10T11:06:25.163 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-10T11:06:25.163 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-10T11:06:25.163 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-05-10T11:06:25.163 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout [osd] 2024-05-10T11:06:25.163 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-10T11:06:25.163 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-10T11:06:25.163 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-10T11:06:25.164 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-10T11:06:25.164 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-10T11:06:25.164 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:24.405+0000 7f4dc8e3e700 1 Processor -- start 2024-05-10T11:06:25.164 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:24.405+0000 7f4dc8e3e700 1 -- start start 2024-05-10T11:06:25.164 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:24.406+0000 7f4dc8e3e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4dc4107a70 0x7f4dc4107e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:06:25.164 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:24.406+0000 7f4dc8e3e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc4108460 con 0x7f4dc4107a70 2024-05-10T11:06:25.164 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:24.406+0000 7f4dc259c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4dc4107a70 0x7f4dc4107e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:06:25.164 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:24.406+0000 7f4dc259c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4dc4107a70 0x7f4dc4107e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55996/0 (socket says 172.21.15.100:55996) 2024-05-10T11:06:25.164 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:24.406+0000 7f4dc259c700 1 -- 172.21.15.100:0/3659642184 learned_addr learned my addr 172.21.15.100:0/3659642184 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:06:25.164 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:24.406+0000 7f4dc259c700 1 -- 172.21.15.100:0/3659642184 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dc4108c70 con 0x7f4dc4107a70 2024-05-10T11:06:25.164 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:24.406+0000 7f4dc259c700 1 --2- 172.21.15.100:0/3659642184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4dc4107a70 0x7f4dc4107e90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4db80098a0 tx=0x7f4db8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8851209aeb25152d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:06:25.164 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:24.406+0000 7f4dc159a700 1 -- 172.21.15.100:0/3659642184 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4db8004160 con 0x7f4dc4107a70 2024-05-10T11:06:25.164 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:24.406+0000 7f4dc159a700 1 -- 172.21.15.100:0/3659642184 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f4db8004300 con 0x7f4dc4107a70 2024-05-10T11:06:25.164 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:24.406+0000 7f4dc159a700 1 -- 172.21.15.100:0/3659642184 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4db80119e0 con 0x7f4dc4107a70 2024-05-10T11:06:25.165 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:24.407+0000 7f4dc8e3e700 1 -- 172.21.15.100:0/3659642184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4dc4107a70 msgr2=0x7f4dc4107e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:06:25.165 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:24.407+0000 7f4dc8e3e700 1 --2- 172.21.15.100:0/3659642184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4dc4107a70 0x7f4dc4107e90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4db80098a0 tx=0x7f4db8009bb0 comp rx=0 tx=0).stop 2024-05-10T11:06:25.165 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:24.407+0000 7f4dc8e3e700 1 -- 172.21.15.100:0/3659642184 shutdown_connections 2024-05-10T11:06:25.165 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:24.407+0000 7f4dc8e3e700 1 --2- 172.21.15.100:0/3659642184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4dc4107a70 0x7f4dc4107e90 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:06:25.165 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:24.407+0000 7f4dc8e3e700 1 -- 172.21.15.100:0/3659642184 >> 172.21.15.100:0/3659642184 conn(0x7f4dc4102fd0 msgr2=0x7f4dc4105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:06:25.165 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:24.407+0000 7f4dc8e3e700 1 -- 172.21.15.100:0/3659642184 shutdown_connections 2024-05-10T11:06:25.165 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:24.407+0000 7f4dc8e3e700 1 -- 172.21.15.100:0/3659642184 wait complete. 2024-05-10T11:06:25.165 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:24.408+0000 7f4dc8e3e700 1 Processor -- start 2024-05-10T11:06:25.165 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:24.408+0000 7f4dc8e3e700 1 -- start start 2024-05-10T11:06:25.165 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:24.408+0000 7f4dc8e3e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4dc4107a70 0x7f4dc407ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:06:25.165 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:24.408+0000 7f4dc8e3e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc410bea0 con 0x7f4dc4107a70 2024-05-10T11:06:25.165 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:24.408+0000 7f4dc259c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4dc4107a70 0x7f4dc407ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:06:25.165 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:24.408+0000 7f4dc259c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4dc4107a70 0x7f4dc407ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56000/0 (socket says 172.21.15.100:56000) 2024-05-10T11:06:25.166 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:24.408+0000 7f4dc259c700 1 -- 172.21.15.100:0/1009682217 learned_addr learned my addr 172.21.15.100:0/1009682217 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:06:25.166 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:24.408+0000 7f4dc259c700 1 -- 172.21.15.100:0/1009682217 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dc419f6f0 con 0x7f4dc4107a70 2024-05-10T11:06:25.166 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:24.409+0000 7f4dc259c700 1 --2- 172.21.15.100:0/1009682217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4dc4107a70 0x7f4dc407ca00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4db80119b0 tx=0x7f4db8011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:06:25.166 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:24.409+0000 7f4db37fe700 1 -- 172.21.15.100:0/1009682217 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4db8003af0 con 0x7f4dc4107a70 2024-05-10T11:06:25.166 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:24.409+0000 7f4db37fe700 1 -- 172.21.15.100:0/1009682217 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f4db801a8a0 con 0x7f4dc4107a70 2024-05-10T11:06:25.166 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:24.409+0000 7f4dc8e3e700 1 -- 172.21.15.100:0/1009682217 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dc419f8c0 con 0x7f4dc4107a70 2024-05-10T11:06:25.166 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:24.409+0000 7f4db37fe700 1 -- 172.21.15.100:0/1009682217 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4db8024830 con 0x7f4dc4107a70 2024-05-10T11:06:25.166 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:24.409+0000 7f4dc8e3e700 1 -- 172.21.15.100:0/1009682217 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dc419fce0 con 0x7f4dc4107a70 2024-05-10T11:06:25.166 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:24.410+0000 7f4db37fe700 1 -- 172.21.15.100:0/1009682217 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7f4db8022020 con 0x7f4dc4107a70 2024-05-10T11:06:25.166 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:24.410+0000 7f4dc8e3e700 1 -- 172.21.15.100:0/1009682217 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dc4064b00 con 0x7f4dc4107a70 2024-05-10T11:06:25.166 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:24.410+0000 7f4db37fe700 1 --2- 172.21.15.100:0/1009682217 >> [v2:172.21.15.100:6800/1086450256,v1:172.21.15.100:6801/1086450256] conn(0x7f4dac03e420 0x7f4dac0408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:06:25.166 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:24.410+0000 7f4db37fe700 1 -- 172.21.15.100:0/1009682217 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4db8053570 con 0x7f4dc4107a70 2024-05-10T11:06:25.166 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:24.410+0000 7f4dc1d9b700 1 --2- 172.21.15.100:0/1009682217 >> [v2:172.21.15.100:6800/1086450256,v1:172.21.15.100:6801/1086450256] conn(0x7f4dac03e420 0x7f4dac0408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:06:25.166 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:24.410+0000 7f4dc1d9b700 1 --2- 172.21.15.100:0/1009682217 >> [v2:172.21.15.100:6800/1086450256,v1:172.21.15.100:6801/1086450256] conn(0x7f4dac03e420 0x7f4dac0408e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4db4006fd0 tx=0x7f4db4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:06:25.167 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:24.415+0000 7f4db37fe700 1 -- 172.21.15.100:0/1009682217 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f4db8024a70 con 0x7f4dc4107a70 2024-05-10T11:06:25.167 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:24.616+0000 7f4dc8e3e700 1 -- 172.21.15.100:0/1009682217 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f4dc41a16a0 con 0x7f4dc4107a70 2024-05-10T11:06:25.167 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:24.617+0000 7f4db37fe700 1 -- 172.21.15.100:0/1009682217 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+409 (secure 0 0 0) 0x7f4db8053250 con 0x7f4dc4107a70 2024-05-10T11:06:25.167 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:24.619+0000 7f4dc8e3e700 1 -- 172.21.15.100:0/1009682217 >> [v2:172.21.15.100:6800/1086450256,v1:172.21.15.100:6801/1086450256] conn(0x7f4dac03e420 msgr2=0x7f4dac0408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:06:25.167 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:24.619+0000 7f4dc8e3e700 1 --2- 172.21.15.100:0/1009682217 >> [v2:172.21.15.100:6800/1086450256,v1:172.21.15.100:6801/1086450256] conn(0x7f4dac03e420 0x7f4dac0408e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4db4006fd0 tx=0x7f4db4006e40 comp rx=0 tx=0).stop 2024-05-10T11:06:25.167 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:24.619+0000 7f4dc8e3e700 1 -- 172.21.15.100:0/1009682217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4dc4107a70 msgr2=0x7f4dc407ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:06:25.167 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:24.619+0000 7f4dc8e3e700 1 --2- 172.21.15.100:0/1009682217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4dc4107a70 0x7f4dc407ca00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4db80119b0 tx=0x7f4db8011a90 comp rx=0 tx=0).stop 2024-05-10T11:06:25.167 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:24.619+0000 7f4dc8e3e700 1 -- 172.21.15.100:0/1009682217 shutdown_connections 2024-05-10T11:06:25.167 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:24.619+0000 7f4dc8e3e700 1 --2- 172.21.15.100:0/1009682217 >> [v2:172.21.15.100:6800/1086450256,v1:172.21.15.100:6801/1086450256] conn(0x7f4dac03e420 0x7f4dac0408e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:06:25.167 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:24.619+0000 7f4dc8e3e700 1 --2- 172.21.15.100:0/1009682217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4dc4107a70 0x7f4dc407ca00 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:06:25.167 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:24.619+0000 7f4dc8e3e700 1 -- 172.21.15.100:0/1009682217 >> 172.21.15.100:0/1009682217 conn(0x7f4dc4102fd0 msgr2=0x7f4dc4103a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:06:25.167 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:24.620+0000 7f4dc8e3e700 1 -- 172.21.15.100:0/1009682217 shutdown_connections 2024-05-10T11:06:25.167 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:24.620+0000 7f4dc8e3e700 1 -- 172.21.15.100:0/1009682217 wait complete. 2024-05-10T11:06:25.168 INFO:teuthology.orchestra.run.smithi100.stdout:Enabling cephadm module... 2024-05-10T11:06:25.282 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:25 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/1009682217' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-10T11:06:26.104 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:26 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/1218553215' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-05-10T11:06:26.367 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:26 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: ignoring --setuser ceph since I am not root 2024-05-10T11:06:26.367 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:26 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: ignoring --setgroup ceph since I am not root 2024-05-10T11:06:26.367 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:26 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:26.226+0000 7fdd07d16200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-10T11:06:26.626 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:26 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:26.412+0000 7fdd07d16200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-10T11:06:26.626 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:26 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:26.514+0000 7fdd07d16200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-10T11:06:26.634 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:25.577+0000 7f8337ece700 1 Processor -- start 2024-05-10T11:06:26.634 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:25.577+0000 7f8337ece700 1 -- start start 2024-05-10T11:06:26.634 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:25.577+0000 7f8337ece700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8330107a10 0x7f8330107e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:06:26.634 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:25.577+0000 7f8337ece700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8330108400 con 0x7f8330107a10 2024-05-10T11:06:26.634 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:25.578+0000 7f8335c6a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8330107a10 0x7f8330107e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:06:26.634 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:25.578+0000 7f8335c6a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8330107a10 0x7f8330107e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56010/0 (socket says 172.21.15.100:56010) 2024-05-10T11:06:26.635 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:25.578+0000 7f8335c6a700 1 -- 172.21.15.100:0/4046000666 learned_addr learned my addr 172.21.15.100:0/4046000666 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:06:26.635 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:25.578+0000 7f8335c6a700 1 -- 172.21.15.100:0/4046000666 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8330108c10 con 0x7f8330107a10 2024-05-10T11:06:26.635 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:25.578+0000 7f8335c6a700 1 --2- 172.21.15.100:0/4046000666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8330107a10 0x7f8330107e30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f832c0098a0 tx=0x7f832c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a5d1bcd0425b604 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:06:26.635 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:25.578+0000 7f8334c68700 1 -- 172.21.15.100:0/4046000666 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f832c004160 con 0x7f8330107a10 2024-05-10T11:06:26.635 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:25.578+0000 7f8334c68700 1 -- 172.21.15.100:0/4046000666 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f832c004300 con 0x7f8330107a10 2024-05-10T11:06:26.635 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:25.578+0000 7f8334c68700 1 -- 172.21.15.100:0/4046000666 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f832c0119e0 con 0x7f8330107a10 2024-05-10T11:06:26.635 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:25.579+0000 7f8337ece700 1 -- 172.21.15.100:0/4046000666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8330107a10 msgr2=0x7f8330107e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:06:26.635 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:25.579+0000 7f8337ece700 1 --2- 172.21.15.100:0/4046000666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8330107a10 0x7f8330107e30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f832c0098a0 tx=0x7f832c009bb0 comp rx=0 tx=0).stop 2024-05-10T11:06:26.635 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:25.579+0000 7f8337ece700 1 -- 172.21.15.100:0/4046000666 shutdown_connections 2024-05-10T11:06:26.635 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:25.579+0000 7f8337ece700 1 --2- 172.21.15.100:0/4046000666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8330107a10 0x7f8330107e30 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:06:26.635 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:25.579+0000 7f8337ece700 1 -- 172.21.15.100:0/4046000666 >> 172.21.15.100:0/4046000666 conn(0x7f8330102f90 msgr2=0x7f83301053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:06:26.635 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:25.579+0000 7f8337ece700 1 -- 172.21.15.100:0/4046000666 shutdown_connections 2024-05-10T11:06:26.636 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:25.579+0000 7f8337ece700 1 -- 172.21.15.100:0/4046000666 wait complete. 2024-05-10T11:06:26.636 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:25.579+0000 7f8337ece700 1 Processor -- start 2024-05-10T11:06:26.636 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:25.580+0000 7f8337ece700 1 -- start start 2024-05-10T11:06:26.636 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:25.580+0000 7f8337ece700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8330107a10 0x7f833019a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:06:26.636 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:25.580+0000 7f8337ece700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f833007bf60 con 0x7f8330107a10 2024-05-10T11:06:26.636 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:25.580+0000 7f8335c6a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8330107a10 0x7f833019a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:06:26.636 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:25.580+0000 7f8335c6a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8330107a10 0x7f833019a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56020/0 (socket says 172.21.15.100:56020) 2024-05-10T11:06:26.636 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:25.580+0000 7f8335c6a700 1 -- 172.21.15.100:0/1218553215 learned_addr learned my addr 172.21.15.100:0/1218553215 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:06:26.636 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:25.580+0000 7f8335c6a700 1 -- 172.21.15.100:0/1218553215 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f833019ad50 con 0x7f8330107a10 2024-05-10T11:06:26.636 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:25.580+0000 7f8335c6a700 1 --2- 172.21.15.100:0/1218553215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8330107a10 0x7f833019a810 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f832c0116a0 tx=0x7f832c011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:06:26.636 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:25.580+0000 7f8326ffd700 1 -- 172.21.15.100:0/1218553215 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f832c0039f0 con 0x7f8330107a10 2024-05-10T11:06:26.636 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:25.580+0000 7f8337ece700 1 -- 172.21.15.100:0/1218553215 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f833019afe0 con 0x7f8330107a10 2024-05-10T11:06:26.636 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:25.580+0000 7f8326ffd700 1 -- 172.21.15.100:0/1218553215 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f832c003b90 con 0x7f8330107a10 2024-05-10T11:06:26.637 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:25.580+0000 7f8326ffd700 1 -- 172.21.15.100:0/1218553215 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f832c01a600 con 0x7f8330107a10 2024-05-10T11:06:26.637 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:25.580+0000 7f8337ece700 1 -- 172.21.15.100:0/1218553215 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f833019dc80 con 0x7f8330107a10 2024-05-10T11:06:26.637 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:25.581+0000 7f8326ffd700 1 -- 172.21.15.100:0/1218553215 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7f832c01a900 con 0x7f8330107a10 2024-05-10T11:06:26.637 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:25.581+0000 7f8337ece700 1 -- 172.21.15.100:0/1218553215 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8330064b00 con 0x7f8330107a10 2024-05-10T11:06:26.637 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:25.581+0000 7f8326ffd700 1 --2- 172.21.15.100:0/1218553215 >> [v2:172.21.15.100:6800/1086450256,v1:172.21.15.100:6801/1086450256] conn(0x7f831c0405e0 0x7f831c042aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:06:26.637 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:25.582+0000 7f8326ffd700 1 -- 172.21.15.100:0/1218553215 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f832c053e00 con 0x7f8330107a10 2024-05-10T11:06:26.637 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:25.582+0000 7f8335469700 1 --2- 172.21.15.100:0/1218553215 >> [v2:172.21.15.100:6800/1086450256,v1:172.21.15.100:6801/1086450256] conn(0x7f831c0405e0 0x7f831c042aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:06:26.637 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:25.582+0000 7f8335469700 1 --2- 172.21.15.100:0/1218553215 >> [v2:172.21.15.100:6800/1086450256,v1:172.21.15.100:6801/1086450256] conn(0x7f831c0405e0 0x7f831c042aa0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8320006fd0 tx=0x7f8320006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:06:26.637 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:25.587+0000 7f8326ffd700 1 -- 172.21.15.100:0/1218553215 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f832c02c350 con 0x7f8330107a10 2024-05-10T11:06:26.637 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:25.844+0000 7f8337ece700 1 -- 172.21.15.100:0/1218553215 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f833019de50 con 0x7f8330107a10 2024-05-10T11:06:26.637 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:26.052+0000 7f8326ffd700 1 -- 172.21.15.100:0/1218553215 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f833019de50 con 0x7f8330107a10 2024-05-10T11:06:26.637 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:26.054+0000 7f8326ffd700 1 -- 172.21.15.100:0/1218553215 <== mon.0 v2:172.21.15.100:3300/0 8 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f832c02d3f0 con 0x7f8330107a10 2024-05-10T11:06:26.637 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:26.055+0000 7f8337ece700 1 -- 172.21.15.100:0/1218553215 >> [v2:172.21.15.100:6800/1086450256,v1:172.21.15.100:6801/1086450256] conn(0x7f831c0405e0 msgr2=0x7f831c042aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:06:26.637 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:26.055+0000 7f8337ece700 1 --2- 172.21.15.100:0/1218553215 >> [v2:172.21.15.100:6800/1086450256,v1:172.21.15.100:6801/1086450256] conn(0x7f831c0405e0 0x7f831c042aa0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8320006fd0 tx=0x7f8320006e40 comp rx=0 tx=0).stop 2024-05-10T11:06:26.638 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:26.055+0000 7f8337ece700 1 -- 172.21.15.100:0/1218553215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8330107a10 msgr2=0x7f833019a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:06:26.638 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:26.055+0000 7f8337ece700 1 --2- 172.21.15.100:0/1218553215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8330107a10 0x7f833019a810 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f832c0116a0 tx=0x7f832c011a60 comp rx=0 tx=0).stop 2024-05-10T11:06:26.638 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:26.055+0000 7f8337ece700 1 -- 172.21.15.100:0/1218553215 shutdown_connections 2024-05-10T11:06:26.638 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:26.055+0000 7f8337ece700 1 --2- 172.21.15.100:0/1218553215 >> [v2:172.21.15.100:6800/1086450256,v1:172.21.15.100:6801/1086450256] conn(0x7f831c0405e0 0x7f831c042aa0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:06:26.638 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:26.055+0000 7f8337ece700 1 --2- 172.21.15.100:0/1218553215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8330107a10 0x7f833019a810 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:06:26.638 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:26.055+0000 7f8337ece700 1 -- 172.21.15.100:0/1218553215 >> 172.21.15.100:0/1218553215 conn(0x7f8330102f90 msgr2=0x7f8330190e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:06:26.638 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:26.056+0000 7f8337ece700 1 -- 172.21.15.100:0/1218553215 shutdown_connections 2024-05-10T11:06:26.638 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:26.056+0000 7f8337ece700 1 -- 172.21.15.100:0/1218553215 wait complete. 2024-05-10T11:06:26.878 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:26 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:26.831+0000 7fdd07d16200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-10T11:06:27.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:27 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/1218553215' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-10T11:06:27.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:27 smithi100 ceph-mon[30296]: mgrmap e5: x(active, since 6s) 2024-05-10T11:06:27.288 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:26 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:26.919+0000 7fdd07d16200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-10T11:06:28.009 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout { 2024-05-10T11:06:28.009 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-05-10T11:06:28.009 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-10T11:06:28.009 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-10T11:06:28.009 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-10T11:06:28.009 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout } 2024-05-10T11:06:28.009 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:27.091+0000 7fed7d6b2700 1 Processor -- start 2024-05-10T11:06:28.009 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:27.091+0000 7fed7d6b2700 1 -- start start 2024-05-10T11:06:28.009 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:27.091+0000 7fed7d6b2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed78109580 0x7fed7810b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:06:28.010 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:27.091+0000 7fed7d6b2700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fed7807ce40 con 0x7fed78109580 2024-05-10T11:06:28.010 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:27.091+0000 7fed76ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed78109580 0x7fed7810b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:06:28.010 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:27.091+0000 7fed76ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed78109580 0x7fed7810b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47018/0 (socket says 172.21.15.100:47018) 2024-05-10T11:06:28.010 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:27.091+0000 7fed76ffd700 1 -- 172.21.15.100:0/1905229544 learned_addr learned my addr 172.21.15.100:0/1905229544 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:06:28.010 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:27.091+0000 7fed76ffd700 1 -- 172.21.15.100:0/1905229544 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed7810beb0 con 0x7fed78109580 2024-05-10T11:06:28.010 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:27.092+0000 7fed76ffd700 1 --2- 172.21.15.100:0/1905229544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed78109580 0x7fed7810b970 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fed680098a0 tx=0x7fed68009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fb1f08677851f03 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:06:28.010 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:27.092+0000 7fed75ffb700 1 -- 172.21.15.100:0/1905229544 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed68004160 con 0x7fed78109580 2024-05-10T11:06:28.010 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:27.092+0000 7fed75ffb700 1 -- 172.21.15.100:0/1905229544 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fed68004300 con 0x7fed78109580 2024-05-10T11:06:28.010 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:27.092+0000 7fed75ffb700 1 -- 172.21.15.100:0/1905229544 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed680119e0 con 0x7fed78109580 2024-05-10T11:06:28.010 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:27.092+0000 7fed7d6b2700 1 -- 172.21.15.100:0/1905229544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed78109580 msgr2=0x7fed7810b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:06:28.010 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:27.092+0000 7fed7d6b2700 1 --2- 172.21.15.100:0/1905229544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed78109580 0x7fed7810b970 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fed680098a0 tx=0x7fed68009bb0 comp rx=0 tx=0).stop 2024-05-10T11:06:28.010 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:27.093+0000 7fed7d6b2700 1 -- 172.21.15.100:0/1905229544 shutdown_connections 2024-05-10T11:06:28.010 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:27.093+0000 7fed7d6b2700 1 --2- 172.21.15.100:0/1905229544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed78109580 0x7fed7810b970 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:06:28.011 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:27.093+0000 7fed7d6b2700 1 -- 172.21.15.100:0/1905229544 >> 172.21.15.100:0/1905229544 conn(0x7fed78102f90 msgr2=0x7fed781053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:06:28.011 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:27.093+0000 7fed7d6b2700 1 -- 172.21.15.100:0/1905229544 shutdown_connections 2024-05-10T11:06:28.011 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:27.093+0000 7fed7d6b2700 1 -- 172.21.15.100:0/1905229544 wait complete. 2024-05-10T11:06:28.011 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:27.093+0000 7fed7d6b2700 1 Processor -- start 2024-05-10T11:06:28.011 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:27.093+0000 7fed7d6b2700 1 -- start start 2024-05-10T11:06:28.011 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:27.094+0000 7fed7d6b2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed78109580 0x7fed78108e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:06:28.011 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:27.094+0000 7fed7d6b2700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fed7810ce20 con 0x7fed78109580 2024-05-10T11:06:28.012 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:27.094+0000 7fed76ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed78109580 0x7fed78108e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:06:28.012 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:27.094+0000 7fed76ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed78109580 0x7fed78108e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47024/0 (socket says 172.21.15.100:47024) 2024-05-10T11:06:28.012 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:27.094+0000 7fed76ffd700 1 -- 172.21.15.100:0/2491894897 learned_addr learned my addr 172.21.15.100:0/2491894897 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:06:28.012 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:27.094+0000 7fed76ffd700 1 -- 172.21.15.100:0/2491894897 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed78109340 con 0x7fed78109580 2024-05-10T11:06:28.012 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:27.094+0000 7fed76ffd700 1 --2- 172.21.15.100:0/2491894897 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed78109580 0x7fed78108e00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fed680119b0 tx=0x7fed68011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:06:28.012 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:27.094+0000 7fed5ffff700 1 -- 172.21.15.100:0/2491894897 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed680039f0 con 0x7fed78109580 2024-05-10T11:06:28.012 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:27.094+0000 7fed7d6b2700 1 -- 172.21.15.100:0/2491894897 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed78107530 con 0x7fed78109580 2024-05-10T11:06:28.012 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:27.094+0000 7fed5ffff700 1 -- 172.21.15.100:0/2491894897 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fed68003b90 con 0x7fed78109580 2024-05-10T11:06:28.012 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:27.094+0000 7fed5ffff700 1 -- 172.21.15.100:0/2491894897 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed6801a620 con 0x7fed78109580 2024-05-10T11:06:28.012 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:27.094+0000 7fed7d6b2700 1 -- 172.21.15.100:0/2491894897 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed78107a70 con 0x7fed78109580 2024-05-10T11:06:28.012 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:27.095+0000 7fed5ffff700 1 -- 172.21.15.100:0/2491894897 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7fed6801a920 con 0x7fed78109580 2024-05-10T11:06:28.012 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:27.095+0000 7fed7d6b2700 1 -- 172.21.15.100:0/2491894897 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed78064b00 con 0x7fed78109580 2024-05-10T11:06:28.012 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:27.096+0000 7fed5ffff700 1 --2- 172.21.15.100:0/2491894897 >> [v2:172.21.15.100:6800/1086450256,v1:172.21.15.100:6801/1086450256] conn(0x7fed6003e420 0x7fed600408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:06:28.012 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:27.096+0000 7fed5ffff700 1 -- 172.21.15.100:0/2491894897 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fed68053c20 con 0x7fed78109580 2024-05-10T11:06:28.013 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:27.096+0000 7fed767fc700 1 -- 172.21.15.100:0/2491894897 >> [v2:172.21.15.100:6800/1086450256,v1:172.21.15.100:6801/1086450256] conn(0x7fed6003e420 msgr2=0x7fed600408e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.100:6800/1086450256 2024-05-10T11:06:28.013 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:27.096+0000 7fed767fc700 1 --2- 172.21.15.100:0/2491894897 >> [v2:172.21.15.100:6800/1086450256,v1:172.21.15.100:6801/1086450256] conn(0x7fed6003e420 0x7fed600408e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-10T11:06:28.013 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:27.101+0000 7fed5ffff700 1 -- 172.21.15.100:0/2491894897 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fed6802c350 con 0x7fed78109580 2024-05-10T11:06:28.013 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:27.297+0000 7fed767fc700 1 -- 172.21.15.100:0/2491894897 >> [v2:172.21.15.100:6800/1086450256,v1:172.21.15.100:6801/1086450256] conn(0x7fed6003e420 msgr2=0x7fed600408e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.100:6800/1086450256 2024-05-10T11:06:28.013 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:27.297+0000 7fed767fc700 1 --2- 172.21.15.100:0/2491894897 >> [v2:172.21.15.100:6800/1086450256,v1:172.21.15.100:6801/1086450256] conn(0x7fed6003e420 0x7fed600408e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-10T11:06:28.013 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:27.401+0000 7fed7d6b2700 1 -- 172.21.15.100:0/2491894897 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fed781083a0 con 0x7fed78109580 2024-05-10T11:06:28.013 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:27.401+0000 7fed5ffff700 1 -- 172.21.15.100:0/2491894897 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fed68053640 con 0x7fed78109580 2024-05-10T11:06:28.013 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:27.403+0000 7fed7d6b2700 1 -- 172.21.15.100:0/2491894897 >> [v2:172.21.15.100:6800/1086450256,v1:172.21.15.100:6801/1086450256] conn(0x7fed6003e420 msgr2=0x7fed600408e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T11:06:28.013 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:27.403+0000 7fed7d6b2700 1 --2- 172.21.15.100:0/2491894897 >> [v2:172.21.15.100:6800/1086450256,v1:172.21.15.100:6801/1086450256] conn(0x7fed6003e420 0x7fed600408e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:06:28.013 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:27.403+0000 7fed7d6b2700 1 -- 172.21.15.100:0/2491894897 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed78109580 msgr2=0x7fed78108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:06:28.013 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:27.403+0000 7fed7d6b2700 1 --2- 172.21.15.100:0/2491894897 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed78109580 0x7fed78108e00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fed680119b0 tx=0x7fed68011a90 comp rx=0 tx=0).stop 2024-05-10T11:06:28.013 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:27.403+0000 7fed7d6b2700 1 -- 172.21.15.100:0/2491894897 shutdown_connections 2024-05-10T11:06:28.013 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:27.403+0000 7fed7d6b2700 1 --2- 172.21.15.100:0/2491894897 >> [v2:172.21.15.100:6800/1086450256,v1:172.21.15.100:6801/1086450256] conn(0x7fed6003e420 0x7fed600408e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:06:28.014 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:27.403+0000 7fed7d6b2700 1 --2- 172.21.15.100:0/2491894897 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed78109580 0x7fed78108e00 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:06:28.014 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:27.403+0000 7fed7d6b2700 1 -- 172.21.15.100:0/2491894897 >> 172.21.15.100:0/2491894897 conn(0x7fed78102f90 msgr2=0x7fed78103a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:06:28.014 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:27.404+0000 7fed7d6b2700 1 -- 172.21.15.100:0/2491894897 shutdown_connections 2024-05-10T11:06:28.014 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:27.404+0000 7fed7d6b2700 1 -- 172.21.15.100:0/2491894897 wait complete. 2024-05-10T11:06:28.014 INFO:teuthology.orchestra.run.smithi100.stdout:Waiting for the mgr to restart... 2024-05-10T11:06:28.014 INFO:teuthology.orchestra.run.smithi100.stdout:Waiting for mgr epoch 5... 2024-05-10T11:06:28.284 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:28 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/2491894897' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-10T11:06:28.284 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:28 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:28.146+0000 7fdd07d16200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-10T11:06:28.538 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:28 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:28.321+0000 7fdd07d16200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-10T11:06:28.538 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:28 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:28.495+0000 7fdd07d16200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-10T11:06:29.038 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:28 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:28.785+0000 7fdd07d16200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-10T11:06:29.038 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:28 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:28.876+0000 7fdd07d16200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-10T11:06:29.538 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:29 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:29.077+0000 7fdd07d16200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-10T11:06:29.538 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:29 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:29.269+0000 7fdd07d16200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-10T11:06:30.288 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:29 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:29.861+0000 7fdd07d16200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-10T11:06:30.288 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:30 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:30.080+0000 7fdd07d16200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-10T11:06:30.788 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:30 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:30.499+0000 7fdd07d16200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-10T11:06:30.788 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:30 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:30.595+0000 7fdd07d16200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-10T11:06:31.288 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:30 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:30.958+0000 7fdd07d16200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-10T11:06:31.641 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:31 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:31.310+0000 7fdd07d16200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-10T11:06:31.641 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:31 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:31.401+0000 7fdd07d16200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-10T11:06:32.038 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:31 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:31.640+0000 7fdd07d16200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-10T11:06:32.038 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:31 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:31.773+0000 7fdd07d16200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-10T11:06:32.038 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:31 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:31.869+0000 7fdd07d16200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-10T11:06:32.538 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:32 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:32.200+0000 7fdd07d16200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-10T11:06:32.538 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:32 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:32.293+0000 7fdd07d16200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-10T11:06:33.165 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:32 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:32.892+0000 7fdd07d16200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-10T11:06:33.166 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:33 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:33.112+0000 7fdd07d16200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-10T11:06:33.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:33 smithi100 ceph-mon[30296]: Active manager daemon x restarted 2024-05-10T11:06:33.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:33 smithi100 ceph-mon[30296]: Activating manager daemon x 2024-05-10T11:06:33.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:33 smithi100 ceph-mon[30296]: osdmap e2: 0 total, 0 up, 0 in 2024-05-10T11:06:33.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:33 smithi100 ceph-mon[30296]: mgrmap e6: x(active, starting, since 0.00379861s) 2024-05-10T11:06:33.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:33 smithi100 ceph-mon[30296]: from='mgr.14120 172.21.15.100:0/2558397845' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T11:06:33.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:33 smithi100 ceph-mon[30296]: from='mgr.14120 172.21.15.100:0/2558397845' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-10T11:06:33.539 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:33 smithi100 ceph-mon[30296]: from='mgr.14120 172.21.15.100:0/2558397845' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-10T11:06:33.539 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:33 smithi100 ceph-mon[30296]: from='mgr.14120 172.21.15.100:0/2558397845' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-10T11:06:33.539 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:33 smithi100 ceph-mon[30296]: from='mgr.14120 172.21.15.100:0/2558397845' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-10T11:06:33.539 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:33 smithi100 ceph-mon[30296]: Manager daemon x is now available 2024-05-10T11:06:33.539 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:33 smithi100 ceph-mon[30296]: from='mgr.14120 172.21.15.100:0/2558397845' entity='mgr.x' 2024-05-10T11:06:33.539 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:33 smithi100 ceph-mon[30296]: from='mgr.14120 172.21.15.100:0/2558397845' entity='mgr.x' 2024-05-10T11:06:33.539 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:33 smithi100 ceph-mon[30296]: from='mgr.14120 172.21.15.100:0/2558397845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:06:33.539 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:33 smithi100 ceph-mon[30296]: from='mgr.14120 172.21.15.100:0/2558397845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:06:33.539 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:33 smithi100 ceph-mon[30296]: from='mgr.14120 172.21.15.100:0/2558397845' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-10T11:06:34.523 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:34 smithi100 ceph-mon[30296]: Found migration_current of "None". Setting to last migration. 2024-05-10T11:06:34.523 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:34 smithi100 ceph-mon[30296]: from='mgr.14120 172.21.15.100:0/2558397845' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-10T11:06:34.523 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:34 smithi100 ceph-mon[30296]: from='mgr.14120 172.21.15.100:0/2558397845' entity='mgr.x' 2024-05-10T11:06:34.523 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:34 smithi100 ceph-mon[30296]: from='mgr.14120 172.21.15.100:0/2558397845' entity='mgr.x' 2024-05-10T11:06:34.523 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:34 smithi100 ceph-mon[30296]: mgrmap e7: x(active, since 1.00598s) 2024-05-10T11:06:34.825 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout { 2024-05-10T11:06:34.825 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-05-10T11:06:34.825 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-10T11:06:34.825 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout } 2024-05-10T11:06:34.825 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:28.503+0000 7f55f7a66700 1 Processor -- start 2024-05-10T11:06:34.825 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:28.503+0000 7f55f7a66700 1 -- start start 2024-05-10T11:06:34.825 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:28.504+0000 7f55f7a66700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55f01079f0 0x7f55f0107e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:06:34.825 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:28.504+0000 7f55f7a66700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f55f01083e0 con 0x7f55f01079f0 2024-05-10T11:06:34.825 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:28.504+0000 7f55f5802700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55f01079f0 0x7f55f0107e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:06:34.825 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:28.504+0000 7f55f5802700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55f01079f0 0x7f55f0107e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47036/0 (socket says 172.21.15.100:47036) 2024-05-10T11:06:34.825 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:28.504+0000 7f55f5802700 1 -- 172.21.15.100:0/3879131538 learned_addr learned my addr 172.21.15.100:0/3879131538 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:06:34.825 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:28.504+0000 7f55f5802700 1 -- 172.21.15.100:0/3879131538 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55f0108bf0 con 0x7f55f01079f0 2024-05-10T11:06:34.826 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:28.504+0000 7f55f5802700 1 --2- 172.21.15.100:0/3879131538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55f01079f0 0x7f55f0107e10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f55ec0098a0 tx=0x7f55ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=368c079aec5f3f0e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:06:34.826 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:28.505+0000 7f55f4800700 1 -- 172.21.15.100:0/3879131538 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55ec004160 con 0x7f55f01079f0 2024-05-10T11:06:34.826 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:28.505+0000 7f55f4800700 1 -- 172.21.15.100:0/3879131538 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f55ec004300 con 0x7f55f01079f0 2024-05-10T11:06:34.826 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:28.505+0000 7f55f4800700 1 -- 172.21.15.100:0/3879131538 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55ec0119e0 con 0x7f55f01079f0 2024-05-10T11:06:34.826 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:28.505+0000 7f55f7a66700 1 -- 172.21.15.100:0/3879131538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55f01079f0 msgr2=0x7f55f0107e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:06:34.826 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:28.505+0000 7f55f7a66700 1 --2- 172.21.15.100:0/3879131538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55f01079f0 0x7f55f0107e10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f55ec0098a0 tx=0x7f55ec009bb0 comp rx=0 tx=0).stop 2024-05-10T11:06:34.826 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:28.505+0000 7f55f7a66700 1 -- 172.21.15.100:0/3879131538 shutdown_connections 2024-05-10T11:06:34.826 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:28.505+0000 7f55f7a66700 1 --2- 172.21.15.100:0/3879131538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55f01079f0 0x7f55f0107e10 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:06:34.826 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:28.505+0000 7f55f7a66700 1 -- 172.21.15.100:0/3879131538 >> 172.21.15.100:0/3879131538 conn(0x7f55f0102fd0 msgr2=0x7f55f01053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:06:34.826 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:28.506+0000 7f55f7a66700 1 -- 172.21.15.100:0/3879131538 shutdown_connections 2024-05-10T11:06:34.826 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:28.506+0000 7f55f7a66700 1 -- 172.21.15.100:0/3879131538 wait complete. 2024-05-10T11:06:34.826 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:28.506+0000 7f55f7a66700 1 Processor -- start 2024-05-10T11:06:34.827 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:28.506+0000 7f55f7a66700 1 -- start start 2024-05-10T11:06:34.827 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:28.507+0000 7f55f7a66700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55f01079f0 0x7f55f019b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:06:34.827 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:28.507+0000 7f55f7a66700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f55f010be20 con 0x7f55f01079f0 2024-05-10T11:06:34.827 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:28.507+0000 7f55f5802700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55f01079f0 0x7f55f019b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:06:34.827 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:28.507+0000 7f55f5802700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55f01079f0 0x7f55f019b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47050/0 (socket says 172.21.15.100:47050) 2024-05-10T11:06:34.827 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:28.507+0000 7f55f5802700 1 -- 172.21.15.100:0/3307427427 learned_addr learned my addr 172.21.15.100:0/3307427427 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:06:34.827 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:28.507+0000 7f55f5802700 1 -- 172.21.15.100:0/3307427427 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55f019be70 con 0x7f55f01079f0 2024-05-10T11:06:34.827 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:28.507+0000 7f55f5802700 1 --2- 172.21.15.100:0/3307427427 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55f01079f0 0x7f55f019b930 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f55ec0119b0 tx=0x7f55ec011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:06:34.827 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:28.507+0000 7f55e6ffd700 1 -- 172.21.15.100:0/3307427427 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55ec003b70 con 0x7f55f01079f0 2024-05-10T11:06:34.827 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:28.507+0000 7f55e6ffd700 1 -- 172.21.15.100:0/3307427427 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f55ec01a8a0 con 0x7f55f01079f0 2024-05-10T11:06:34.827 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:28.507+0000 7f55f7a66700 1 -- 172.21.15.100:0/3307427427 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55f019c100 con 0x7f55f01079f0 2024-05-10T11:06:34.827 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:28.507+0000 7f55e6ffd700 1 -- 172.21.15.100:0/3307427427 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55ec024830 con 0x7f55f01079f0 2024-05-10T11:06:34.827 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:28.508+0000 7f55f7a66700 1 -- 172.21.15.100:0/3307427427 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55f019edf0 con 0x7f55f01079f0 2024-05-10T11:06:34.827 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:28.508+0000 7f55e6ffd700 1 -- 172.21.15.100:0/3307427427 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f55ec022020 con 0x7f55f01079f0 2024-05-10T11:06:34.827 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:28.509+0000 7f55e6ffd700 1 --2- 172.21.15.100:0/3307427427 >> [v2:172.21.15.100:6800/1086450256,v1:172.21.15.100:6801/1086450256] conn(0x7f55dc03e470 0x7f55dc040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:06:34.828 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:28.509+0000 7f55e6ffd700 1 -- 172.21.15.100:0/3307427427 --> [v2:172.21.15.100:6800/1086450256,v1:172.21.15.100:6801/1086450256] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f55dc041040 con 0x7f55dc03e470 2024-05-10T11:06:34.828 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:28.509+0000 7f55e6ffd700 1 -- 172.21.15.100:0/3307427427 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f55ec0536a0 con 0x7f55f01079f0 2024-05-10T11:06:34.828 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:28.509+0000 7f55f5001700 1 -- 172.21.15.100:0/3307427427 >> [v2:172.21.15.100:6800/1086450256,v1:172.21.15.100:6801/1086450256] conn(0x7f55dc03e470 msgr2=0x7f55dc040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.100:6800/1086450256 2024-05-10T11:06:34.828 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:28.509+0000 7f55f5001700 1 --2- 172.21.15.100:0/3307427427 >> [v2:172.21.15.100:6800/1086450256,v1:172.21.15.100:6801/1086450256] conn(0x7f55dc03e470 0x7f55dc040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-10T11:06:34.828 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:28.710+0000 7f55f5001700 1 -- 172.21.15.100:0/3307427427 >> [v2:172.21.15.100:6800/1086450256,v1:172.21.15.100:6801/1086450256] conn(0x7f55dc03e470 msgr2=0x7f55dc040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.100:6800/1086450256 2024-05-10T11:06:34.828 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:28.710+0000 7f55f5001700 1 --2- 172.21.15.100:0/3307427427 >> [v2:172.21.15.100:6800/1086450256,v1:172.21.15.100:6801/1086450256] conn(0x7f55dc03e470 0x7f55dc040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-10T11:06:34.828 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:29.111+0000 7f55f5001700 1 -- 172.21.15.100:0/3307427427 >> [v2:172.21.15.100:6800/1086450256,v1:172.21.15.100:6801/1086450256] conn(0x7f55dc03e470 msgr2=0x7f55dc040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.100:6800/1086450256 2024-05-10T11:06:34.828 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:29.111+0000 7f55f5001700 1 --2- 172.21.15.100:0/3307427427 >> [v2:172.21.15.100:6800/1086450256,v1:172.21.15.100:6801/1086450256] conn(0x7f55dc03e470 0x7f55dc040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-10T11:06:34.828 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:29.912+0000 7f55f5001700 1 -- 172.21.15.100:0/3307427427 >> [v2:172.21.15.100:6800/1086450256,v1:172.21.15.100:6801/1086450256] conn(0x7f55dc03e470 msgr2=0x7f55dc040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.100:6800/1086450256 2024-05-10T11:06:34.828 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:29.912+0000 7f55f5001700 1 --2- 172.21.15.100:0/3307427427 >> [v2:172.21.15.100:6800/1086450256,v1:172.21.15.100:6801/1086450256] conn(0x7f55dc03e470 0x7f55dc040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-10T11:06:34.828 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:31.513+0000 7f55f5001700 1 -- 172.21.15.100:0/3307427427 >> [v2:172.21.15.100:6800/1086450256,v1:172.21.15.100:6801/1086450256] conn(0x7f55dc03e470 msgr2=0x7f55dc040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.100:6800/1086450256 2024-05-10T11:06:34.828 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:31.513+0000 7f55f5001700 1 --2- 172.21.15.100:0/3307427427 >> [v2:172.21.15.100:6800/1086450256,v1:172.21.15.100:6801/1086450256] conn(0x7f55dc03e470 0x7f55dc040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-10T11:06:34.828 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:33.115+0000 7f55e6ffd700 1 -- 172.21.15.100:0/3307427427 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mgrmap(e 6) ==== 49989+0+0 (secure 0 0 0) 0x7f55ec02d3f0 con 0x7f55f01079f0 2024-05-10T11:06:34.828 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:33.116+0000 7f55e6ffd700 1 -- 172.21.15.100:0/3307427427 >> [v2:172.21.15.100:6800/1086450256,v1:172.21.15.100:6801/1086450256] conn(0x7f55dc03e470 msgr2=0x7f55dc040930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T11:06:34.829 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:33.116+0000 7f55e6ffd700 1 --2- 172.21.15.100:0/3307427427 >> [v2:172.21.15.100:6800/1086450256,v1:172.21.15.100:6801/1086450256] conn(0x7f55dc03e470 0x7f55dc040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:06:34.829 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:34.117+0000 7f55e6ffd700 1 -- 172.21.15.100:0/3307427427 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mgrmap(e 7) ==== 50116+0+0 (secure 0 0 0) 0x7f55ec054de0 con 0x7f55f01079f0 2024-05-10T11:06:34.829 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:34.117+0000 7f55e6ffd700 1 --2- 172.21.15.100:0/3307427427 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7f55dc041d60 0x7f55dc044150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:06:34.829 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:34.117+0000 7f55e6ffd700 1 -- 172.21.15.100:0/3307427427 --> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f55ec00ae50 con 0x7f55dc041d60 2024-05-10T11:06:34.829 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:34.117+0000 7f55f5001700 1 --2- 172.21.15.100:0/3307427427 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7f55dc041d60 0x7f55dc044150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:06:34.829 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:34.118+0000 7f55f5001700 1 --2- 172.21.15.100:0/3307427427 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7f55dc041d60 0x7f55dc044150 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f55e0003a10 tx=0x7f55e00092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:06:34.829 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:34.119+0000 7f55e6ffd700 1 -- 172.21.15.100:0/3307427427 <== mgr.14120 v2:172.21.15.100:6800/1597433518 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f55ec00ae50 con 0x7f55dc041d60 2024-05-10T11:06:34.829 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:34.127+0000 7f55f7a66700 1 -- 172.21.15.100:0/3307427427 --> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f55f0064b00 con 0x7f55dc041d60 2024-05-10T11:06:34.829 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:34.127+0000 7f55e6ffd700 1 -- 172.21.15.100:0/3307427427 <== mgr.14120 v2:172.21.15.100:6800/1597433518 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f55f0064b00 con 0x7f55dc041d60 2024-05-10T11:06:34.829 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:34.127+0000 7f55f7a66700 1 -- 172.21.15.100:0/3307427427 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7f55dc041d60 msgr2=0x7f55dc044150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:06:34.829 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:34.127+0000 7f55f7a66700 1 --2- 172.21.15.100:0/3307427427 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7f55dc041d60 0x7f55dc044150 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f55e0003a10 tx=0x7f55e00092b0 comp rx=0 tx=0).stop 2024-05-10T11:06:34.829 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:34.127+0000 7f55f7a66700 1 -- 172.21.15.100:0/3307427427 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55f01079f0 msgr2=0x7f55f019b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:06:34.829 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:34.127+0000 7f55f7a66700 1 --2- 172.21.15.100:0/3307427427 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55f01079f0 0x7f55f019b930 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f55ec0119b0 tx=0x7f55ec011a90 comp rx=0 tx=0).stop 2024-05-10T11:06:34.829 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:34.127+0000 7f55f7a66700 1 -- 172.21.15.100:0/3307427427 shutdown_connections 2024-05-10T11:06:34.830 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:34.127+0000 7f55f7a66700 1 --2- 172.21.15.100:0/3307427427 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7f55dc041d60 0x7f55dc044150 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:06:34.830 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:34.127+0000 7f55f7a66700 1 --2- 172.21.15.100:0/3307427427 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55f01079f0 0x7f55f019b930 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:06:34.830 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:34.127+0000 7f55f7a66700 1 --2- 172.21.15.100:0/3307427427 >> [v2:172.21.15.100:6800/1086450256,v1:172.21.15.100:6801/1086450256] conn(0x7f55dc03e470 0x7f55dc040930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:06:34.830 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:34.127+0000 7f55f7a66700 1 -- 172.21.15.100:0/3307427427 >> 172.21.15.100:0/3307427427 conn(0x7f55f0102fd0 msgr2=0x7f55f0103b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:06:34.830 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:34.127+0000 7f55f7a66700 1 -- 172.21.15.100:0/3307427427 shutdown_connections 2024-05-10T11:06:34.830 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:34.127+0000 7f55f7a66700 1 -- 172.21.15.100:0/3307427427 wait complete. 2024-05-10T11:06:34.830 INFO:teuthology.orchestra.run.smithi100.stdout:mgr epoch 5 is available 2024-05-10T11:06:34.830 INFO:teuthology.orchestra.run.smithi100.stdout:Setting orchestrator backend to cephadm... 2024-05-10T11:06:35.502 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:35 smithi100 ceph-mon[30296]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-10T11:06:35.502 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:35 smithi100 ceph-mon[30296]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-10T11:06:35.503 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:35 smithi100 ceph-mon[30296]: [10/May/2024:11:06:34] ENGINE Bus STARTING 2024-05-10T11:06:35.503 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:35 smithi100 ceph-mon[30296]: [10/May/2024:11:06:34] ENGINE Serving on https://172.21.15.100:7150 2024-05-10T11:06:35.503 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:35 smithi100 ceph-mon[30296]: [10/May/2024:11:06:34] ENGINE Serving on http://172.21.15.100:8765 2024-05-10T11:06:35.503 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:35 smithi100 ceph-mon[30296]: [10/May/2024:11:06:34] ENGINE Bus STARTED 2024-05-10T11:06:35.503 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:35 smithi100 ceph-mon[30296]: from='mgr.14120 172.21.15.100:0/2558397845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:06:36.088 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:35.269+0000 7f1cf1760700 1 Processor -- start 2024-05-10T11:06:36.089 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:35.269+0000 7f1cf1760700 1 -- start start 2024-05-10T11:06:36.089 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:35.269+0000 7f1cf1760700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cec107410 0x7f1cec107830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:06:36.089 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:35.269+0000 7f1cf1760700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1cec076940 con 0x7f1cec107410 2024-05-10T11:06:36.089 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:35.270+0000 7f1ceaffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cec107410 0x7f1cec107830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:06:36.089 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:35.270+0000 7f1ceaffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cec107410 0x7f1cec107830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47132/0 (socket says 172.21.15.100:47132) 2024-05-10T11:06:36.089 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:35.270+0000 7f1ceaffd700 1 -- 172.21.15.100:0/3126213033 learned_addr learned my addr 172.21.15.100:0/3126213033 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:06:36.089 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:35.270+0000 7f1ceaffd700 1 -- 172.21.15.100:0/3126213033 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cec107d70 con 0x7f1cec107410 2024-05-10T11:06:36.089 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:35.270+0000 7f1ceaffd700 1 --2- 172.21.15.100:0/3126213033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cec107410 0x7f1cec107830 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc0098a0 tx=0x7f1cdc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3f1408977c5ca03 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:06:36.089 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:35.270+0000 7f1cea7fc700 1 -- 172.21.15.100:0/3126213033 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1cdc004160 con 0x7f1cec107410 2024-05-10T11:06:36.089 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:35.270+0000 7f1cea7fc700 1 -- 172.21.15.100:0/3126213033 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f1cdc004300 con 0x7f1cec107410 2024-05-10T11:06:36.089 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:35.270+0000 7f1cea7fc700 1 -- 172.21.15.100:0/3126213033 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1cdc0119e0 con 0x7f1cec107410 2024-05-10T11:06:36.090 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:35.271+0000 7f1cf1760700 1 -- 172.21.15.100:0/3126213033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cec107410 msgr2=0x7f1cec107830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:06:36.090 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:35.271+0000 7f1cf1760700 1 --2- 172.21.15.100:0/3126213033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cec107410 0x7f1cec107830 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc0098a0 tx=0x7f1cdc009bb0 comp rx=0 tx=0).stop 2024-05-10T11:06:36.090 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:35.271+0000 7f1cf1760700 1 -- 172.21.15.100:0/3126213033 shutdown_connections 2024-05-10T11:06:36.090 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:35.271+0000 7f1cf1760700 1 --2- 172.21.15.100:0/3126213033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cec107410 0x7f1cec107830 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:06:36.090 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:35.271+0000 7f1cf1760700 1 -- 172.21.15.100:0/3126213033 >> 172.21.15.100:0/3126213033 conn(0x7f1cec102fd0 msgr2=0x7f1cec1053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:06:36.090 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:35.271+0000 7f1cf1760700 1 -- 172.21.15.100:0/3126213033 shutdown_connections 2024-05-10T11:06:36.090 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:35.271+0000 7f1cf1760700 1 -- 172.21.15.100:0/3126213033 wait complete. 2024-05-10T11:06:36.090 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:35.271+0000 7f1cf1760700 1 Processor -- start 2024-05-10T11:06:36.090 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:35.272+0000 7f1cf1760700 1 -- start start 2024-05-10T11:06:36.090 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:35.272+0000 7f1cf1760700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cec107410 0x7f1cec19ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:06:36.090 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:35.272+0000 7f1cf1760700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1cec108d40 con 0x7f1cec107410 2024-05-10T11:06:36.090 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:35.272+0000 7f1ceaffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cec107410 0x7f1cec19ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:06:36.090 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:35.272+0000 7f1ceaffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cec107410 0x7f1cec19ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47142/0 (socket says 172.21.15.100:47142) 2024-05-10T11:06:36.091 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:35.272+0000 7f1ceaffd700 1 -- 172.21.15.100:0/3685505459 learned_addr learned my addr 172.21.15.100:0/3685505459 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:06:36.091 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:35.272+0000 7f1ceaffd700 1 -- 172.21.15.100:0/3685505459 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cec19f100 con 0x7f1cec107410 2024-05-10T11:06:36.091 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:35.272+0000 7f1ceaffd700 1 --2- 172.21.15.100:0/3685505459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cec107410 0x7f1cec19ebc0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc0116a0 tx=0x7f1cdc0046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:06:36.091 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:35.273+0000 7f1ce8ff9700 1 -- 172.21.15.100:0/3685505459 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1cdc003b70 con 0x7f1cec107410 2024-05-10T11:06:36.091 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:35.273+0000 7f1ce8ff9700 1 -- 172.21.15.100:0/3685505459 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f1cdc01a8a0 con 0x7f1cec107410 2024-05-10T11:06:36.091 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:35.273+0000 7f1ce8ff9700 1 -- 172.21.15.100:0/3685505459 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1cdc024890 con 0x7f1cec107410 2024-05-10T11:06:36.091 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:35.273+0000 7f1cf1760700 1 -- 172.21.15.100:0/3685505459 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cec19f390 con 0x7f1cec107410 2024-05-10T11:06:36.091 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:35.273+0000 7f1cf1760700 1 -- 172.21.15.100:0/3685505459 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cec1a2080 con 0x7f1cec107410 2024-05-10T11:06:36.091 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:35.273+0000 7f1ce8ff9700 1 -- 172.21.15.100:0/3685505459 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f1cdc022020 con 0x7f1cec107410 2024-05-10T11:06:36.092 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:35.274+0000 7f1cf1760700 1 -- 172.21.15.100:0/3685505459 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cec064b00 con 0x7f1cec107410 2024-05-10T11:06:36.092 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:35.274+0000 7f1ce8ff9700 1 --2- 172.21.15.100:0/3685505459 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7f1cd803e4c0 0x7f1cd8040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:06:36.092 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:35.274+0000 7f1ce8ff9700 1 -- 172.21.15.100:0/3685505459 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1cdc053b40 con 0x7f1cec107410 2024-05-10T11:06:36.092 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:35.274+0000 7f1ce27fc700 1 --2- 172.21.15.100:0/3685505459 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7f1cd803e4c0 0x7f1cd8040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:06:36.092 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:35.274+0000 7f1ce27fc700 1 --2- 172.21.15.100:0/3685505459 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7f1cd803e4c0 0x7f1cd8040980 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1cd4006fd0 tx=0x7f1cd4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:06:36.092 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:35.279+0000 7f1ce8ff9700 1 -- 172.21.15.100:0/3685505459 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f1cdc017e90 con 0x7f1cec107410 2024-05-10T11:06:36.092 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:35.502+0000 7f1cf1760700 1 -- 172.21.15.100:0/3685505459 --> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f1cec0008d0 con 0x7f1cd803e4c0 2024-05-10T11:06:36.092 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:35.507+0000 7f1ce8ff9700 1 -- 172.21.15.100:0/3685505459 <== mgr.14120 v2:172.21.15.100:6800/1597433518 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1cec0008d0 con 0x7f1cd803e4c0 2024-05-10T11:06:36.092 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:35.510+0000 7f1cf1760700 1 -- 172.21.15.100:0/3685505459 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7f1cd803e4c0 msgr2=0x7f1cd8040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:06:36.092 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:35.510+0000 7f1cf1760700 1 --2- 172.21.15.100:0/3685505459 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7f1cd803e4c0 0x7f1cd8040980 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1cd4006fd0 tx=0x7f1cd4006e40 comp rx=0 tx=0).stop 2024-05-10T11:06:36.093 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:35.510+0000 7f1cf1760700 1 -- 172.21.15.100:0/3685505459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cec107410 msgr2=0x7f1cec19ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:06:36.093 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:35.510+0000 7f1cf1760700 1 --2- 172.21.15.100:0/3685505459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cec107410 0x7f1cec19ebc0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc0116a0 tx=0x7f1cdc0046a0 comp rx=0 tx=0).stop 2024-05-10T11:06:36.093 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:35.510+0000 7f1cf1760700 1 -- 172.21.15.100:0/3685505459 shutdown_connections 2024-05-10T11:06:36.093 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:35.510+0000 7f1cf1760700 1 --2- 172.21.15.100:0/3685505459 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7f1cd803e4c0 0x7f1cd8040980 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:06:36.093 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:35.510+0000 7f1cf1760700 1 --2- 172.21.15.100:0/3685505459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cec107410 0x7f1cec19ebc0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:06:36.093 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:35.510+0000 7f1cf1760700 1 -- 172.21.15.100:0/3685505459 >> 172.21.15.100:0/3685505459 conn(0x7f1cec102fd0 msgr2=0x7f1cec195830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:06:36.094 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:35.510+0000 7f1cf1760700 1 -- 172.21.15.100:0/3685505459 shutdown_connections 2024-05-10T11:06:36.094 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:35.510+0000 7f1cf1760700 1 -- 172.21.15.100:0/3685505459 wait complete. 2024-05-10T11:06:36.407 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:36 smithi100 ceph-mon[30296]: mgrmap e8: x(active, since 2s) 2024-05-10T11:06:36.407 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:36 smithi100 ceph-mon[30296]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:06:36.407 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:36 smithi100 ceph-mon[30296]: from='mgr.14120 172.21.15.100:0/2558397845' entity='mgr.x' 2024-05-10T11:06:36.407 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:36 smithi100 ceph-mon[30296]: from='mgr.14120 172.21.15.100:0/2558397845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:06:37.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:37 smithi100 ceph-mon[30296]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:06:37.312 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout value unchanged 2024-05-10T11:06:37.318 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:36.507+0000 7fd60341d700 1 Processor -- start 2024-05-10T11:06:37.318 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:36.507+0000 7fd60341d700 1 -- start start 2024-05-10T11:06:37.318 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:36.507+0000 7fd60341d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5fc07ee50 0x7fd5fc07d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:06:37.318 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:36.507+0000 7fd60341d700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd5fc07d880 con 0x7fd5fc07ee50 2024-05-10T11:06:37.318 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:36.508+0000 7fd6011b9700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5fc07ee50 0x7fd5fc07d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:06:37.318 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:36.508+0000 7fd6011b9700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5fc07ee50 0x7fd5fc07d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51954/0 (socket says 172.21.15.100:51954) 2024-05-10T11:06:37.318 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:36.508+0000 7fd6011b9700 1 -- 172.21.15.100:0/2991813526 learned_addr learned my addr 172.21.15.100:0/2991813526 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:06:37.318 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:36.508+0000 7fd6011b9700 1 -- 172.21.15.100:0/2991813526 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5fc07da00 con 0x7fd5fc07ee50 2024-05-10T11:06:37.318 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:36.508+0000 7fd6011b9700 1 --2- 172.21.15.100:0/2991813526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5fc07ee50 0x7fd5fc07d2b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd5f80098a0 tx=0x7fd5f8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7f436e7e4b40511 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:06:37.319 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:36.508+0000 7fd5f3fff700 1 -- 172.21.15.100:0/2991813526 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5f8004160 con 0x7fd5fc07ee50 2024-05-10T11:06:37.319 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:36.508+0000 7fd5f3fff700 1 -- 172.21.15.100:0/2991813526 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fd5f8004300 con 0x7fd5fc07ee50 2024-05-10T11:06:37.319 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:36.508+0000 7fd5f3fff700 1 -- 172.21.15.100:0/2991813526 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5f80119e0 con 0x7fd5fc07ee50 2024-05-10T11:06:37.319 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:36.509+0000 7fd60341d700 1 -- 172.21.15.100:0/2991813526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5fc07ee50 msgr2=0x7fd5fc07d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:06:37.319 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:36.509+0000 7fd60341d700 1 --2- 172.21.15.100:0/2991813526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5fc07ee50 0x7fd5fc07d2b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd5f80098a0 tx=0x7fd5f8009bb0 comp rx=0 tx=0).stop 2024-05-10T11:06:37.319 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:36.509+0000 7fd60341d700 1 -- 172.21.15.100:0/2991813526 shutdown_connections 2024-05-10T11:06:37.319 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:36.509+0000 7fd60341d700 1 --2- 172.21.15.100:0/2991813526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5fc07ee50 0x7fd5fc07d2b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:06:37.319 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:36.509+0000 7fd60341d700 1 -- 172.21.15.100:0/2991813526 >> 172.21.15.100:0/2991813526 conn(0x7fd5fc104070 msgr2=0x7fd5fc1064d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:06:37.319 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:36.509+0000 7fd60341d700 1 -- 172.21.15.100:0/2991813526 shutdown_connections 2024-05-10T11:06:37.319 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:36.509+0000 7fd60341d700 1 -- 172.21.15.100:0/2991813526 wait complete. 2024-05-10T11:06:37.319 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:36.510+0000 7fd60341d700 1 Processor -- start 2024-05-10T11:06:37.319 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:36.510+0000 7fd60341d700 1 -- start start 2024-05-10T11:06:37.319 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:36.510+0000 7fd60341d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5fc07ee50 0x7fd5fc19eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:06:37.320 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:36.510+0000 7fd60341d700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd5fc10f020 con 0x7fd5fc07ee50 2024-05-10T11:06:37.320 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:36.510+0000 7fd6011b9700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5fc07ee50 0x7fd5fc19eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:06:37.320 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:36.510+0000 7fd6011b9700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5fc07ee50 0x7fd5fc19eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51964/0 (socket says 172.21.15.100:51964) 2024-05-10T11:06:37.320 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:36.510+0000 7fd6011b9700 1 -- 172.21.15.100:0/2086175491 learned_addr learned my addr 172.21.15.100:0/2086175491 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:06:37.320 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:36.510+0000 7fd6011b9700 1 -- 172.21.15.100:0/2086175491 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5fc19f0b0 con 0x7fd5fc07ee50 2024-05-10T11:06:37.320 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:36.510+0000 7fd6011b9700 1 --2- 172.21.15.100:0/2086175491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5fc07ee50 0x7fd5fc19eb70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd5f80116a0 tx=0x7fd5f8011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:06:37.320 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:36.510+0000 7fd5f27fc700 1 -- 172.21.15.100:0/2086175491 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5f80039f0 con 0x7fd5fc07ee50 2024-05-10T11:06:37.320 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:36.510+0000 7fd5f27fc700 1 -- 172.21.15.100:0/2086175491 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fd5f8003b90 con 0x7fd5fc07ee50 2024-05-10T11:06:37.320 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:36.511+0000 7fd60341d700 1 -- 172.21.15.100:0/2086175491 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5fc19f340 con 0x7fd5fc07ee50 2024-05-10T11:06:37.320 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:36.511+0000 7fd5f27fc700 1 -- 172.21.15.100:0/2086175491 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5f801a600 con 0x7fd5fc07ee50 2024-05-10T11:06:37.320 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:36.511+0000 7fd60341d700 1 -- 172.21.15.100:0/2086175491 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5fc1a2030 con 0x7fd5fc07ee50 2024-05-10T11:06:37.320 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:36.511+0000 7fd5f27fc700 1 -- 172.21.15.100:0/2086175491 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fd5f801a900 con 0x7fd5fc07ee50 2024-05-10T11:06:37.320 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:36.511+0000 7fd60341d700 1 -- 172.21.15.100:0/2086175491 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5fc064b00 con 0x7fd5fc07ee50 2024-05-10T11:06:37.320 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:36.512+0000 7fd5f27fc700 1 --2- 172.21.15.100:0/2086175491 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7fd5e803e420 0x7fd5e80408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:06:37.320 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:36.512+0000 7fd5f27fc700 1 -- 172.21.15.100:0/2086175491 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd5f8053bc0 con 0x7fd5fc07ee50 2024-05-10T11:06:37.321 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:36.512+0000 7fd6009b8700 1 --2- 172.21.15.100:0/2086175491 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7fd5e803e420 0x7fd5e80408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:06:37.321 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:36.512+0000 7fd6009b8700 1 --2- 172.21.15.100:0/2086175491 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7fd5e803e420 0x7fd5e80408e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec006fd0 tx=0x7fd5ec006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:06:37.321 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:36.517+0000 7fd5f27fc700 1 -- 172.21.15.100:0/2086175491 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fd5f802c350 con 0x7fd5fc07ee50 2024-05-10T11:06:37.321 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:36.732+0000 7fd60341d700 1 -- 172.21.15.100:0/2086175491 --> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fd5fc105220 con 0x7fd5e803e420 2024-05-10T11:06:37.321 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:36.733+0000 7fd5f27fc700 1 -- 172.21.15.100:0/2086175491 <== mgr.14120 v2:172.21.15.100:6800/1597433518 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fd5fc105220 con 0x7fd5e803e420 2024-05-10T11:06:37.321 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:36.735+0000 7fd60341d700 1 -- 172.21.15.100:0/2086175491 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7fd5e803e420 msgr2=0x7fd5e80408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:06:37.321 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:36.735+0000 7fd60341d700 1 --2- 172.21.15.100:0/2086175491 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7fd5e803e420 0x7fd5e80408e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec006fd0 tx=0x7fd5ec006e40 comp rx=0 tx=0).stop 2024-05-10T11:06:37.321 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:36.735+0000 7fd60341d700 1 -- 172.21.15.100:0/2086175491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5fc07ee50 msgr2=0x7fd5fc19eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:06:37.321 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:36.735+0000 7fd60341d700 1 --2- 172.21.15.100:0/2086175491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5fc07ee50 0x7fd5fc19eb70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd5f80116a0 tx=0x7fd5f8011a60 comp rx=0 tx=0).stop 2024-05-10T11:06:37.321 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:36.735+0000 7fd60341d700 1 -- 172.21.15.100:0/2086175491 shutdown_connections 2024-05-10T11:06:37.321 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:36.735+0000 7fd60341d700 1 --2- 172.21.15.100:0/2086175491 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7fd5e803e420 0x7fd5e80408e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:06:37.321 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:36.735+0000 7fd60341d700 1 --2- 172.21.15.100:0/2086175491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5fc07ee50 0x7fd5fc19eb70 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:06:37.322 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:36.736+0000 7fd60341d700 1 -- 172.21.15.100:0/2086175491 >> 172.21.15.100:0/2086175491 conn(0x7fd5fc104070 msgr2=0x7fd5fc104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:06:37.322 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:36.736+0000 7fd60341d700 1 -- 172.21.15.100:0/2086175491 shutdown_connections 2024-05-10T11:06:37.322 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:36.736+0000 7fd60341d700 1 -- 172.21.15.100:0/2086175491 wait complete. 2024-05-10T11:06:37.322 INFO:teuthology.orchestra.run.smithi100.stdout:Generating ssh key... 2024-05-10T11:06:38.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:38 smithi100 ceph-mon[30296]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:06:38.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:38 smithi100 ceph-mon[30296]: Generating ssh key... 2024-05-10T11:06:38.666 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:38 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: Generating public/private rsa key pair. 2024-05-10T11:06:38.666 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:38 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: Your identification has been saved in /tmp/tmpo4a9y2a1/key. 2024-05-10T11:06:38.666 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:38 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: Your public key has been saved in /tmp/tmpo4a9y2a1/key.pub. 2024-05-10T11:06:38.666 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:38 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: The key fingerprint is: 2024-05-10T11:06:38.666 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:38 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: SHA256:DmlN6IDKyezF4oeeG+hdFmvLWTVeqRSeGMgJQJmlJv8 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968 2024-05-10T11:06:38.667 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:38 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: The key's randomart image is: 2024-05-10T11:06:38.667 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:38 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: +---[RSA 3072]----+ 2024-05-10T11:06:38.667 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:38 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: |.o=o | 2024-05-10T11:06:38.667 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:38 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: | o..o o. | 2024-05-10T11:06:38.667 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:38 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: |.o. .+.... | 2024-05-10T11:06:38.667 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:38 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: |*oo o ++ o . | 2024-05-10T11:06:38.667 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:38 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: |.*.o .=.S* o | 2024-05-10T11:06:38.667 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:38 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: |+ +. .oo+ + | 2024-05-10T11:06:38.667 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:38 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: |.= .E= ..o | 2024-05-10T11:06:38.667 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:38 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: |o = = + | 2024-05-10T11:06:38.667 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:38 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: | =.. + | 2024-05-10T11:06:38.667 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:38 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: +----[SHA256]-----+ 2024-05-10T11:06:38.949 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:37.747+0000 7f5069781700 1 Processor -- start 2024-05-10T11:06:38.949 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:37.747+0000 7f5069781700 1 -- start start 2024-05-10T11:06:38.950 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:37.747+0000 7f5069781700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5064106ec0 0x7f50641072e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:06:38.950 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:37.747+0000 7f5069781700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f50641078b0 con 0x7f5064106ec0 2024-05-10T11:06:38.950 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:37.747+0000 7f5062ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5064106ec0 0x7f50641072e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:06:38.950 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:37.747+0000 7f5062ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5064106ec0 0x7f50641072e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51978/0 (socket says 172.21.15.100:51978) 2024-05-10T11:06:38.950 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:37.747+0000 7f5062ffd700 1 -- 172.21.15.100:0/3811920170 learned_addr learned my addr 172.21.15.100:0/3811920170 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:06:38.950 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:37.747+0000 7f5062ffd700 1 -- 172.21.15.100:0/3811920170 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5064109e60 con 0x7f5064106ec0 2024-05-10T11:06:38.950 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:37.748+0000 7f5062ffd700 1 --2- 172.21.15.100:0/3811920170 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5064106ec0 0x7f50641072e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f50540098a0 tx=0x7f5054009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c29a783b4c4bfc3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:06:38.950 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:37.748+0000 7f5061ffb700 1 -- 172.21.15.100:0/3811920170 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5054004160 con 0x7f5064106ec0 2024-05-10T11:06:38.950 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:37.748+0000 7f5061ffb700 1 -- 172.21.15.100:0/3811920170 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f5054004300 con 0x7f5064106ec0 2024-05-10T11:06:38.950 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:37.748+0000 7f5061ffb700 1 -- 172.21.15.100:0/3811920170 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50540119e0 con 0x7f5064106ec0 2024-05-10T11:06:38.950 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:37.748+0000 7f5069781700 1 -- 172.21.15.100:0/3811920170 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5064106ec0 msgr2=0x7f50641072e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:06:38.950 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:37.748+0000 7f5069781700 1 --2- 172.21.15.100:0/3811920170 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5064106ec0 0x7f50641072e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f50540098a0 tx=0x7f5054009bb0 comp rx=0 tx=0).stop 2024-05-10T11:06:38.950 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:37.748+0000 7f5069781700 1 -- 172.21.15.100:0/3811920170 shutdown_connections 2024-05-10T11:06:38.951 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:37.748+0000 7f5069781700 1 --2- 172.21.15.100:0/3811920170 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5064106ec0 0x7f50641072e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:06:38.951 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:37.748+0000 7f5069781700 1 -- 172.21.15.100:0/3811920170 >> 172.21.15.100:0/3811920170 conn(0x7f50641027a0 msgr2=0x7f5064104bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:06:38.951 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:37.748+0000 7f5069781700 1 -- 172.21.15.100:0/3811920170 shutdown_connections 2024-05-10T11:06:38.951 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:37.748+0000 7f5069781700 1 -- 172.21.15.100:0/3811920170 wait complete. 2024-05-10T11:06:38.951 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:37.749+0000 7f5069781700 1 Processor -- start 2024-05-10T11:06:38.951 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:37.749+0000 7f5069781700 1 -- start start 2024-05-10T11:06:38.951 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:37.749+0000 7f5069781700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5064106ec0 0x7f506419c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:06:38.951 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:37.749+0000 7f5069781700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f506410d090 con 0x7f5064106ec0 2024-05-10T11:06:38.951 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:37.749+0000 7f5062ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5064106ec0 0x7f506419c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:06:38.951 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:37.749+0000 7f5062ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5064106ec0 0x7f506419c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51984/0 (socket says 172.21.15.100:51984) 2024-05-10T11:06:38.951 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:37.749+0000 7f5062ffd700 1 -- 172.21.15.100:0/3612820530 learned_addr learned my addr 172.21.15.100:0/3612820530 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:06:38.951 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:37.750+0000 7f5062ffd700 1 -- 172.21.15.100:0/3612820530 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f506419cea0 con 0x7f5064106ec0 2024-05-10T11:06:38.951 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:37.750+0000 7f5062ffd700 1 --2- 172.21.15.100:0/3612820530 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5064106ec0 0x7f506419c960 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5054009130 tx=0x7f5054011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:06:38.951 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:37.750+0000 7f504bfff700 1 -- 172.21.15.100:0/3612820530 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5054003b20 con 0x7f5064106ec0 2024-05-10T11:06:38.952 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:37.750+0000 7f504bfff700 1 -- 172.21.15.100:0/3612820530 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f505401a8a0 con 0x7f5064106ec0 2024-05-10T11:06:38.952 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:37.750+0000 7f5069781700 1 -- 172.21.15.100:0/3612820530 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f506419d130 con 0x7f5064106ec0 2024-05-10T11:06:38.952 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:37.750+0000 7f504bfff700 1 -- 172.21.15.100:0/3612820530 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5054024810 con 0x7f5064106ec0 2024-05-10T11:06:38.952 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:37.750+0000 7f5069781700 1 -- 172.21.15.100:0/3612820530 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f506419fe20 con 0x7f5064106ec0 2024-05-10T11:06:38.952 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:37.751+0000 7f504bfff700 1 -- 172.21.15.100:0/3612820530 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f5054022020 con 0x7f5064106ec0 2024-05-10T11:06:38.952 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:37.751+0000 7f5069781700 1 -- 172.21.15.100:0/3612820530 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5064064b00 con 0x7f5064106ec0 2024-05-10T11:06:38.952 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:37.751+0000 7f504bfff700 1 --2- 172.21.15.100:0/3612820530 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7f504c03e470 0x7f504c040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:06:38.952 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:37.751+0000 7f504bfff700 1 -- 172.21.15.100:0/3612820530 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5054053ae0 con 0x7f5064106ec0 2024-05-10T11:06:38.952 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:37.751+0000 7f50627fc700 1 --2- 172.21.15.100:0/3612820530 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7f504c03e470 0x7f504c040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:06:38.952 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:37.751+0000 7f50627fc700 1 --2- 172.21.15.100:0/3612820530 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7f504c03e470 0x7f504c040930 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5058006fd0 tx=0x7f5058006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:06:38.952 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:37.756+0000 7f504bfff700 1 -- 172.21.15.100:0/3612820530 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f5054053470 con 0x7f5064106ec0 2024-05-10T11:06:38.952 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:37.962+0000 7f5069781700 1 -- 172.21.15.100:0/3612820530 --> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f5064103890 con 0x7f504c03e470 2024-05-10T11:06:38.952 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:38.364+0000 7f504bfff700 1 -- 172.21.15.100:0/3612820530 <== mgr.14120 v2:172.21.15.100:6800/1597433518 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5064103890 con 0x7f504c03e470 2024-05-10T11:06:38.952 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:38.367+0000 7f5069781700 1 -- 172.21.15.100:0/3612820530 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7f504c03e470 msgr2=0x7f504c040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:06:38.953 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:38.367+0000 7f5069781700 1 --2- 172.21.15.100:0/3612820530 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7f504c03e470 0x7f504c040930 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5058006fd0 tx=0x7f5058006e40 comp rx=0 tx=0).stop 2024-05-10T11:06:38.953 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:38.367+0000 7f5069781700 1 -- 172.21.15.100:0/3612820530 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5064106ec0 msgr2=0x7f506419c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:06:38.953 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:38.367+0000 7f5069781700 1 --2- 172.21.15.100:0/3612820530 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5064106ec0 0x7f506419c960 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5054009130 tx=0x7f5054011a60 comp rx=0 tx=0).stop 2024-05-10T11:06:38.953 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:38.367+0000 7f5069781700 1 -- 172.21.15.100:0/3612820530 shutdown_connections 2024-05-10T11:06:38.953 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:38.367+0000 7f5069781700 1 --2- 172.21.15.100:0/3612820530 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7f504c03e470 0x7f504c040930 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:06:38.953 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:38.367+0000 7f5069781700 1 --2- 172.21.15.100:0/3612820530 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5064106ec0 0x7f506419c960 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:06:38.953 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:38.367+0000 7f5069781700 1 -- 172.21.15.100:0/3612820530 >> 172.21.15.100:0/3612820530 conn(0x7f50641027a0 msgr2=0x7f5064103180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:06:38.953 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:38.367+0000 7f5069781700 1 -- 172.21.15.100:0/3612820530 shutdown_connections 2024-05-10T11:06:38.953 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:38.367+0000 7f5069781700 1 -- 172.21.15.100:0/3612820530 wait complete. 2024-05-10T11:06:39.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:39 smithi100 ceph-mon[30296]: from='mgr.14120 172.21.15.100:0/2558397845' entity='mgr.x' 2024-05-10T11:06:39.591 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:39 smithi100 ceph-mon[30296]: from='mgr.14120 172.21.15.100:0/2558397845' entity='mgr.x' 2024-05-10T11:06:40.173 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHI7fh65mIvcXCL2la/FdpvKciZgCQgkPBa8Tmz8PvjNhZZjI7QGS9cPd1X7hpcgX60P14LEvPD5yCIPKcSo5d5Ne+aBAWkpSlQz04HmwR+GLOFhXzdta8QSA/aogcRrxzIiI3ZI+tyknmb1pJcZ9v1Emdy6Gtz3l429Gq9XDIJJUSHh6TBHlgu607bwETANsPZgmw18cwD8i0IvnRI4oFbssqjoqEqgn+Pt0NcZnKyJ6DSK62t0HsPU7amlpsot6+A+JCg/bNvBB1I5WlEVVpzY/eTJTjT+HhpiN3C1328xJNU+msWK+jCEWswXJy3n19gMKQOs2tMj8TmaMeGXPp4oJRjyOt7Aw8GlFs9pss0xd8djsh4cybbsw6Ld3egej9qYww412XIAOpM1d7euuaC8X64pQPI4oURZTa4L9wm+31CKLKoaamMgwzYxVGF+xwvvvuIa//3lc9VrKHzo6rDyTnoLbTDddEmR5eQsxzkxkfBhLD0cbq00L5+ahiaTE= ceph-20a542cc-0ebd-11ef-bc98-c7b262605968 2024-05-10T11:06:40.173 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:39.355+0000 7f0318a8f700 1 Processor -- start 2024-05-10T11:06:40.173 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:39.355+0000 7f0318a8f700 1 -- start start 2024-05-10T11:06:40.173 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:39.355+0000 7f0318a8f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0314107a70 0x7f0314107e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:06:40.173 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:39.355+0000 7f0318a8f700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0314108460 con 0x7f0314107a70 2024-05-10T11:06:40.173 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:39.356+0000 7f031259c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0314107a70 0x7f0314107e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:06:40.173 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:39.356+0000 7f031259c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0314107a70 0x7f0314107e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51990/0 (socket says 172.21.15.100:51990) 2024-05-10T11:06:40.174 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:39.356+0000 7f031259c700 1 -- 172.21.15.100:0/2697198923 learned_addr learned my addr 172.21.15.100:0/2697198923 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:06:40.174 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:39.356+0000 7f031259c700 1 -- 172.21.15.100:0/2697198923 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0314108c70 con 0x7f0314107a70 2024-05-10T11:06:40.174 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:39.356+0000 7f031259c700 1 --2- 172.21.15.100:0/2697198923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0314107a70 0x7f0314107e90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f03040098a0 tx=0x7f0304009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=689ec469fc68b833 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:06:40.174 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:39.356+0000 7f031159a700 1 -- 172.21.15.100:0/2697198923 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0304004160 con 0x7f0314107a70 2024-05-10T11:06:40.174 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:39.356+0000 7f031159a700 1 -- 172.21.15.100:0/2697198923 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f0304004300 con 0x7f0314107a70 2024-05-10T11:06:40.174 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:39.356+0000 7f031159a700 1 -- 172.21.15.100:0/2697198923 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f03040119e0 con 0x7f0314107a70 2024-05-10T11:06:40.174 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:39.357+0000 7f0318a8f700 1 -- 172.21.15.100:0/2697198923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0314107a70 msgr2=0x7f0314107e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:06:40.174 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:39.357+0000 7f0318a8f700 1 --2- 172.21.15.100:0/2697198923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0314107a70 0x7f0314107e90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f03040098a0 tx=0x7f0304009bb0 comp rx=0 tx=0).stop 2024-05-10T11:06:40.174 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:39.357+0000 7f0318a8f700 1 -- 172.21.15.100:0/2697198923 shutdown_connections 2024-05-10T11:06:40.174 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:39.357+0000 7f0318a8f700 1 --2- 172.21.15.100:0/2697198923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0314107a70 0x7f0314107e90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:06:40.174 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:39.357+0000 7f0318a8f700 1 -- 172.21.15.100:0/2697198923 >> 172.21.15.100:0/2697198923 conn(0x7f0314102fd0 msgr2=0x7f0314105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:06:40.174 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:39.357+0000 7f0318a8f700 1 -- 172.21.15.100:0/2697198923 shutdown_connections 2024-05-10T11:06:40.174 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:39.357+0000 7f0318a8f700 1 -- 172.21.15.100:0/2697198923 wait complete. 2024-05-10T11:06:40.174 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:39.358+0000 7f0318a8f700 1 Processor -- start 2024-05-10T11:06:40.175 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:39.358+0000 7f0318a8f700 1 -- start start 2024-05-10T11:06:40.175 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:39.358+0000 7f0318a8f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0314107a70 0x7f031407ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:06:40.175 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:39.358+0000 7f0318a8f700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f031410bea0 con 0x7f0314107a70 2024-05-10T11:06:40.175 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:39.358+0000 7f031259c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0314107a70 0x7f031407ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:06:40.175 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:39.358+0000 7f031259c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0314107a70 0x7f031407ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52002/0 (socket says 172.21.15.100:52002) 2024-05-10T11:06:40.175 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:39.358+0000 7f031259c700 1 -- 172.21.15.100:0/2084553049 learned_addr learned my addr 172.21.15.100:0/2084553049 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:06:40.175 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:39.358+0000 7f031259c700 1 -- 172.21.15.100:0/2084553049 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f031419f6c0 con 0x7f0314107a70 2024-05-10T11:06:40.175 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:39.358+0000 7f031259c700 1 --2- 172.21.15.100:0/2084553049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0314107a70 0x7f031407ca00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f03040116a0 tx=0x7f0304011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:06:40.175 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:39.358+0000 7f03037fe700 1 -- 172.21.15.100:0/2084553049 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f03040039f0 con 0x7f0314107a70 2024-05-10T11:06:40.175 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:39.359+0000 7f0318a8f700 1 -- 172.21.15.100:0/2084553049 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f031419f890 con 0x7f0314107a70 2024-05-10T11:06:40.175 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:39.359+0000 7f03037fe700 1 -- 172.21.15.100:0/2084553049 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f0304003b90 con 0x7f0314107a70 2024-05-10T11:06:40.175 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:39.359+0000 7f03037fe700 1 -- 172.21.15.100:0/2084553049 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f030401a600 con 0x7f0314107a70 2024-05-10T11:06:40.175 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:39.359+0000 7f0318a8f700 1 -- 172.21.15.100:0/2084553049 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f031419fcb0 con 0x7f0314107a70 2024-05-10T11:06:40.176 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:39.360+0000 7f03037fe700 1 -- 172.21.15.100:0/2084553049 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f030401a8f0 con 0x7f0314107a70 2024-05-10T11:06:40.176 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:39.360+0000 7f0318a8f700 1 -- 172.21.15.100:0/2084553049 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0314064b00 con 0x7f0314107a70 2024-05-10T11:06:40.176 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:39.360+0000 7f03037fe700 1 --2- 172.21.15.100:0/2084553049 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7f02fc03e470 0x7f02fc040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:06:40.176 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:39.360+0000 7f03037fe700 1 -- 172.21.15.100:0/2084553049 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0304053ce0 con 0x7f0314107a70 2024-05-10T11:06:40.176 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:39.360+0000 7f0311d9b700 1 --2- 172.21.15.100:0/2084553049 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7f02fc03e470 0x7f02fc040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:06:40.176 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:39.360+0000 7f0311d9b700 1 --2- 172.21.15.100:0/2084553049 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7f02fc03e470 0x7f02fc040930 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0308006fd0 tx=0x7f0308006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:06:40.176 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:39.365+0000 7f03037fe700 1 -- 172.21.15.100:0/2084553049 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f030402c350 con 0x7f0314107a70 2024-05-10T11:06:40.176 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:39.572+0000 7f0318a8f700 1 -- 172.21.15.100:0/2084553049 --> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f03141041c0 con 0x7f02fc03e470 2024-05-10T11:06:40.176 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:39.573+0000 7f03037fe700 1 -- 172.21.15.100:0/2084553049 <== mgr.14120 v2:172.21.15.100:6800/1597433518 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f03141041c0 con 0x7f02fc03e470 2024-05-10T11:06:40.176 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:39.575+0000 7f0318a8f700 1 -- 172.21.15.100:0/2084553049 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7f02fc03e470 msgr2=0x7f02fc040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:06:40.176 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:39.575+0000 7f0318a8f700 1 --2- 172.21.15.100:0/2084553049 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7f02fc03e470 0x7f02fc040930 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0308006fd0 tx=0x7f0308006e40 comp rx=0 tx=0).stop 2024-05-10T11:06:40.176 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:39.575+0000 7f0318a8f700 1 -- 172.21.15.100:0/2084553049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0314107a70 msgr2=0x7f031407ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:06:40.176 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:39.575+0000 7f0318a8f700 1 --2- 172.21.15.100:0/2084553049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0314107a70 0x7f031407ca00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f03040116a0 tx=0x7f0304011a60 comp rx=0 tx=0).stop 2024-05-10T11:06:40.177 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:39.575+0000 7f0318a8f700 1 -- 172.21.15.100:0/2084553049 shutdown_connections 2024-05-10T11:06:40.177 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:39.575+0000 7f0318a8f700 1 --2- 172.21.15.100:0/2084553049 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7f02fc03e470 0x7f02fc040930 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:06:40.177 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:39.575+0000 7f0318a8f700 1 --2- 172.21.15.100:0/2084553049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0314107a70 0x7f031407ca00 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:06:40.177 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:39.575+0000 7f0318a8f700 1 -- 172.21.15.100:0/2084553049 >> 172.21.15.100:0/2084553049 conn(0x7f0314102fd0 msgr2=0x7f0314103ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:06:40.177 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:39.576+0000 7f0318a8f700 1 -- 172.21.15.100:0/2084553049 shutdown_connections 2024-05-10T11:06:40.177 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:39.576+0000 7f0318a8f700 1 -- 172.21.15.100:0/2084553049 wait complete. 2024-05-10T11:06:40.177 INFO:teuthology.orchestra.run.smithi100.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-05-10T11:06:40.177 INFO:teuthology.orchestra.run.smithi100.stdout:Adding key to root@localhost authorized_keys... 2024-05-10T11:06:40.177 INFO:teuthology.orchestra.run.smithi100.stdout:Adding host smithi100... 2024-05-10T11:06:40.476 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:40 smithi100 ceph-mon[30296]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:06:41.675 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:41 smithi100 ceph-mon[30296]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi100", "addr": "172.21.15.100", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:06:42.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:42 smithi100 ceph-mon[30296]: Deploying cephadm binary to smithi100 2024-05-10T11:06:45.507 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout Added host 'smithi100' with addr '172.21.15.100' 2024-05-10T11:06:45.507 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:40.582+0000 7f59aa610700 1 Processor -- start 2024-05-10T11:06:45.507 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:40.582+0000 7f59aa610700 1 -- start start 2024-05-10T11:06:45.507 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:40.582+0000 7f59aa610700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59a407edf0 0x7f59a407d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:06:45.507 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:40.582+0000 7f59aa610700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f59a407d820 con 0x7f59a407edf0 2024-05-10T11:06:45.507 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:40.582+0000 7f59a3fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59a407edf0 0x7f59a407d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:06:45.507 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:40.582+0000 7f59a3fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59a407edf0 0x7f59a407d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52008/0 (socket says 172.21.15.100:52008) 2024-05-10T11:06:45.507 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:40.582+0000 7f59a3fff700 1 -- 172.21.15.100:0/761007218 learned_addr learned my addr 172.21.15.100:0/761007218 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:06:45.507 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:40.583+0000 7f59a3fff700 1 -- 172.21.15.100:0/761007218 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59a407d9a0 con 0x7f59a407edf0 2024-05-10T11:06:45.507 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:40.583+0000 7f59a3fff700 1 --2- 172.21.15.100:0/761007218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59a407edf0 0x7f59a407d250 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f59940098a0 tx=0x7f5994009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a8878768119dfcf server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:06:45.508 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:40.583+0000 7f59a2ffd700 1 -- 172.21.15.100:0/761007218 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5994004160 con 0x7f59a407edf0 2024-05-10T11:06:45.508 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:40.583+0000 7f59a2ffd700 1 -- 172.21.15.100:0/761007218 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f5994004300 con 0x7f59a407edf0 2024-05-10T11:06:45.508 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:40.583+0000 7f59a2ffd700 1 -- 172.21.15.100:0/761007218 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59940119e0 con 0x7f59a407edf0 2024-05-10T11:06:45.508 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:40.584+0000 7f59aa610700 1 -- 172.21.15.100:0/761007218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59a407edf0 msgr2=0x7f59a407d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:06:45.508 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:40.584+0000 7f59aa610700 1 --2- 172.21.15.100:0/761007218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59a407edf0 0x7f59a407d250 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f59940098a0 tx=0x7f5994009bb0 comp rx=0 tx=0).stop 2024-05-10T11:06:45.508 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:40.584+0000 7f59aa610700 1 -- 172.21.15.100:0/761007218 shutdown_connections 2024-05-10T11:06:45.508 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:40.584+0000 7f59aa610700 1 --2- 172.21.15.100:0/761007218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59a407edf0 0x7f59a407d250 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:06:45.508 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:40.584+0000 7f59aa610700 1 -- 172.21.15.100:0/761007218 >> 172.21.15.100:0/761007218 conn(0x7f59a4104130 msgr2=0x7f59a4106550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:06:45.508 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:40.584+0000 7f59aa610700 1 -- 172.21.15.100:0/761007218 shutdown_connections 2024-05-10T11:06:45.508 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:40.584+0000 7f59aa610700 1 -- 172.21.15.100:0/761007218 wait complete. 2024-05-10T11:06:45.508 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:40.584+0000 7f59aa610700 1 Processor -- start 2024-05-10T11:06:45.508 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:40.584+0000 7f59aa610700 1 -- start start 2024-05-10T11:06:45.508 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:40.585+0000 7f59aa610700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59a407edf0 0x7f59a419eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:06:45.508 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:40.585+0000 7f59aa610700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f59a410f0a0 con 0x7f59a407edf0 2024-05-10T11:06:45.509 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:40.585+0000 7f59a3fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59a407edf0 0x7f59a419eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:06:45.509 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:40.585+0000 7f59a3fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59a407edf0 0x7f59a419eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52018/0 (socket says 172.21.15.100:52018) 2024-05-10T11:06:45.509 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:40.585+0000 7f59a3fff700 1 -- 172.21.15.100:0/3470342086 learned_addr learned my addr 172.21.15.100:0/3470342086 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:06:45.509 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:40.585+0000 7f59a3fff700 1 -- 172.21.15.100:0/3470342086 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59a419f0a0 con 0x7f59a407edf0 2024-05-10T11:06:45.509 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:40.585+0000 7f59a3fff700 1 --2- 172.21.15.100:0/3470342086 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59a407edf0 0x7f59a419eb60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f59940119b0 tx=0x7f5994011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:06:45.509 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:40.585+0000 7f59a17fa700 1 -- 172.21.15.100:0/3470342086 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59940039f0 con 0x7f59a407edf0 2024-05-10T11:06:45.509 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:40.585+0000 7f59a17fa700 1 -- 172.21.15.100:0/3470342086 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f5994003b90 con 0x7f59a407edf0 2024-05-10T11:06:45.509 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:40.585+0000 7f59aa610700 1 -- 172.21.15.100:0/3470342086 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59a419f330 con 0x7f59a407edf0 2024-05-10T11:06:45.509 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:40.585+0000 7f59a17fa700 1 -- 172.21.15.100:0/3470342086 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f599401a620 con 0x7f59a407edf0 2024-05-10T11:06:45.509 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:40.585+0000 7f59aa610700 1 -- 172.21.15.100:0/3470342086 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59a41a2020 con 0x7f59a407edf0 2024-05-10T11:06:45.509 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:40.586+0000 7f59a17fa700 1 -- 172.21.15.100:0/3470342086 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f599401a920 con 0x7f59a407edf0 2024-05-10T11:06:45.509 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:40.586+0000 7f59aa610700 1 -- 172.21.15.100:0/3470342086 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59a4064b00 con 0x7f59a407edf0 2024-05-10T11:06:45.509 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:40.587+0000 7f59a17fa700 1 --2- 172.21.15.100:0/3470342086 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7f598c03e420 0x7f598c0408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:06:45.509 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:40.587+0000 7f59a17fa700 1 -- 172.21.15.100:0/3470342086 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5994053c80 con 0x7f59a407edf0 2024-05-10T11:06:45.510 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:40.587+0000 7f59a37fe700 1 --2- 172.21.15.100:0/3470342086 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7f598c03e420 0x7f598c0408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:06:45.510 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:40.587+0000 7f59a37fe700 1 --2- 172.21.15.100:0/3470342086 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7f598c03e420 0x7f598c0408e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5998006fd0 tx=0x7f5998006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:06:45.510 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:40.591+0000 7f59a17fa700 1 -- 172.21.15.100:0/3470342086 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f5994024b00 con 0x7f59a407edf0 2024-05-10T11:06:45.510 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:40.795+0000 7f59aa610700 1 -- 172.21.15.100:0/3470342086 --> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi100", "addr": "172.21.15.100", "target": ["mon-mgr", ""]}) -- 0x7f59a41052b0 con 0x7f598c03e420 2024-05-10T11:06:45.510 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:44.902+0000 7f59a17fa700 1 -- 172.21.15.100:0/3470342086 <== mgr.14120 v2:172.21.15.100:6800/1597433518 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f59a41052b0 con 0x7f598c03e420 2024-05-10T11:06:45.510 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:44.906+0000 7f59aa610700 1 -- 172.21.15.100:0/3470342086 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7f598c03e420 msgr2=0x7f598c0408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:06:45.510 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:44.906+0000 7f59aa610700 1 --2- 172.21.15.100:0/3470342086 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7f598c03e420 0x7f598c0408e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5998006fd0 tx=0x7f5998006e40 comp rx=0 tx=0).stop 2024-05-10T11:06:45.510 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:44.906+0000 7f59aa610700 1 -- 172.21.15.100:0/3470342086 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59a407edf0 msgr2=0x7f59a419eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:06:45.510 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:44.906+0000 7f59aa610700 1 --2- 172.21.15.100:0/3470342086 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59a407edf0 0x7f59a419eb60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f59940119b0 tx=0x7f5994011a90 comp rx=0 tx=0).stop 2024-05-10T11:06:45.510 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:44.906+0000 7f59aa610700 1 -- 172.21.15.100:0/3470342086 shutdown_connections 2024-05-10T11:06:45.510 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:44.906+0000 7f59aa610700 1 --2- 172.21.15.100:0/3470342086 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7f598c03e420 0x7f598c0408e0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:06:45.512 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:44.906+0000 7f59aa610700 1 --2- 172.21.15.100:0/3470342086 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59a407edf0 0x7f59a419eb60 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:06:45.512 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:44.906+0000 7f59aa610700 1 -- 172.21.15.100:0/3470342086 >> 172.21.15.100:0/3470342086 conn(0x7f59a4104130 msgr2=0x7f59a4104ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:06:45.512 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:44.906+0000 7f59aa610700 1 -- 172.21.15.100:0/3470342086 shutdown_connections 2024-05-10T11:06:45.512 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:44.906+0000 7f59aa610700 1 -- 172.21.15.100:0/3470342086 wait complete. 2024-05-10T11:06:45.512 INFO:teuthology.orchestra.run.smithi100.stdout:Deploying unmanaged mon service... 2024-05-10T11:06:46.159 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:45 smithi100 ceph-mon[30296]: from='mgr.14120 172.21.15.100:0/2558397845' entity='mgr.x' 2024-05-10T11:06:46.159 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:45 smithi100 ceph-mon[30296]: Added host smithi100 2024-05-10T11:06:46.159 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:45 smithi100 ceph-mon[30296]: from='mgr.14120 172.21.15.100:0/2558397845' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:06:46.714 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-05-10T11:06:46.714 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:45.930+0000 7f113966a700 1 Processor -- start 2024-05-10T11:06:46.714 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:45.930+0000 7f113966a700 1 -- start start 2024-05-10T11:06:46.714 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:45.930+0000 7f113966a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1134106e60 0x7f1134107280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:06:46.714 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:45.930+0000 7f113966a700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1134107850 con 0x7f1134106e60 2024-05-10T11:06:46.714 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:45.930+0000 7f1132ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1134106e60 0x7f1134107280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:06:46.714 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:45.930+0000 7f1132ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1134106e60 0x7f1134107280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52034/0 (socket says 172.21.15.100:52034) 2024-05-10T11:06:46.714 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:45.930+0000 7f1132ffd700 1 -- 172.21.15.100:0/1077171022 learned_addr learned my addr 172.21.15.100:0/1077171022 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:06:46.715 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:45.931+0000 7f1132ffd700 1 -- 172.21.15.100:0/1077171022 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1134109df0 con 0x7f1134106e60 2024-05-10T11:06:46.715 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:45.931+0000 7f1132ffd700 1 --2- 172.21.15.100:0/1077171022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1134106e60 0x7f1134107280 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1124009e80 tx=0x7f112400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47634d4773413cfe server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:06:46.715 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:45.931+0000 7f1131ffb700 1 -- 172.21.15.100:0/1077171022 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1124017070 con 0x7f1134106e60 2024-05-10T11:06:46.715 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:45.931+0000 7f1131ffb700 1 -- 172.21.15.100:0/1077171022 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f112400b8c0 con 0x7f1134106e60 2024-05-10T11:06:46.715 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:45.931+0000 7f1131ffb700 1 -- 172.21.15.100:0/1077171022 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1124004420 con 0x7f1134106e60 2024-05-10T11:06:46.715 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:45.931+0000 7f113966a700 1 -- 172.21.15.100:0/1077171022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1134106e60 msgr2=0x7f1134107280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:06:46.715 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:45.931+0000 7f113966a700 1 --2- 172.21.15.100:0/1077171022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1134106e60 0x7f1134107280 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1124009e80 tx=0x7f112400b330 comp rx=0 tx=0).stop 2024-05-10T11:06:46.715 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:45.931+0000 7f113966a700 1 -- 172.21.15.100:0/1077171022 shutdown_connections 2024-05-10T11:06:46.715 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:45.931+0000 7f113966a700 1 --2- 172.21.15.100:0/1077171022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1134106e60 0x7f1134107280 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:06:46.715 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:45.931+0000 7f113966a700 1 -- 172.21.15.100:0/1077171022 >> 172.21.15.100:0/1077171022 conn(0x7f1134102790 msgr2=0x7f1134104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:06:46.715 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:45.932+0000 7f113966a700 1 -- 172.21.15.100:0/1077171022 shutdown_connections 2024-05-10T11:06:46.715 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:45.932+0000 7f113966a700 1 -- 172.21.15.100:0/1077171022 wait complete. 2024-05-10T11:06:46.715 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:45.932+0000 7f113966a700 1 Processor -- start 2024-05-10T11:06:46.715 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:45.932+0000 7f113966a700 1 -- start start 2024-05-10T11:06:46.716 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:45.932+0000 7f113966a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1134106e60 0x7f113419c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:06:46.716 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:45.932+0000 7f113966a700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f113410d020 con 0x7f1134106e60 2024-05-10T11:06:46.716 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:45.933+0000 7f1132ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1134106e60 0x7f113419c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:06:46.716 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:45.933+0000 7f1132ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1134106e60 0x7f113419c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52050/0 (socket says 172.21.15.100:52050) 2024-05-10T11:06:46.716 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:45.933+0000 7f1132ffd700 1 -- 172.21.15.100:0/1821551404 learned_addr learned my addr 172.21.15.100:0/1821551404 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:06:46.716 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:45.933+0000 7f1132ffd700 1 -- 172.21.15.100:0/1821551404 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f113419cf10 con 0x7f1134106e60 2024-05-10T11:06:46.716 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:45.933+0000 7f1132ffd700 1 --2- 172.21.15.100:0/1821551404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1134106e60 0x7f113419c9d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1124009e50 tx=0x7f112400bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:06:46.716 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:45.933+0000 7f112bfff700 1 -- 172.21.15.100:0/1821551404 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1124020020 con 0x7f1134106e60 2024-05-10T11:06:46.716 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:45.933+0000 7f112bfff700 1 -- 172.21.15.100:0/1821551404 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f1124012430 con 0x7f1134106e60 2024-05-10T11:06:46.716 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:45.933+0000 7f112bfff700 1 -- 172.21.15.100:0/1821551404 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1124017040 con 0x7f1134106e60 2024-05-10T11:06:46.716 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:45.933+0000 7f113966a700 1 -- 172.21.15.100:0/1821551404 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f113419d1a0 con 0x7f1134106e60 2024-05-10T11:06:46.716 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:45.933+0000 7f113966a700 1 -- 172.21.15.100:0/1821551404 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f113419fe90 con 0x7f1134106e60 2024-05-10T11:06:46.716 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:45.934+0000 7f113966a700 1 -- 172.21.15.100:0/1821551404 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f113410d1a0 con 0x7f1134106e60 2024-05-10T11:06:46.716 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:45.934+0000 7f112bfff700 1 -- 172.21.15.100:0/1821551404 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f1124023020 con 0x7f1134106e60 2024-05-10T11:06:46.716 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:45.934+0000 7f112bfff700 1 --2- 172.21.15.100:0/1821551404 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7f112003e420 0x7f11200408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:06:46.717 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:45.934+0000 7f112bfff700 1 -- 172.21.15.100:0/1821551404 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f112400f050 con 0x7f1134106e60 2024-05-10T11:06:46.717 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:45.935+0000 7f11327fc700 1 --2- 172.21.15.100:0/1821551404 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7f112003e420 0x7f11200408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:06:46.717 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:45.935+0000 7f11327fc700 1 --2- 172.21.15.100:0/1821551404 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7f112003e420 0x7f11200408e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f111c006fd0 tx=0x7f111c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:06:46.717 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:45.939+0000 7f112bfff700 1 -- 172.21.15.100:0/1821551404 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f112402a020 con 0x7f1134106e60 2024-05-10T11:06:46.717 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:46.158+0000 7f113966a700 1 -- 172.21.15.100:0/1821551404 --> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f11341039b0 con 0x7f112003e420 2024-05-10T11:06:46.717 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:46.165+0000 7f112bfff700 1 -- 172.21.15.100:0/1821551404 <== mgr.14120 v2:172.21.15.100:6800/1597433518 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f11341039b0 con 0x7f112003e420 2024-05-10T11:06:46.717 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:46.168+0000 7f113966a700 1 -- 172.21.15.100:0/1821551404 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7f112003e420 msgr2=0x7f11200408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:06:46.717 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:46.168+0000 7f113966a700 1 --2- 172.21.15.100:0/1821551404 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7f112003e420 0x7f11200408e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f111c006fd0 tx=0x7f111c006e40 comp rx=0 tx=0).stop 2024-05-10T11:06:46.717 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:46.168+0000 7f113966a700 1 -- 172.21.15.100:0/1821551404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1134106e60 msgr2=0x7f113419c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:06:46.717 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:46.168+0000 7f113966a700 1 --2- 172.21.15.100:0/1821551404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1134106e60 0x7f113419c9d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1124009e50 tx=0x7f112400bd50 comp rx=0 tx=0).stop 2024-05-10T11:06:46.717 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:46.168+0000 7f113966a700 1 -- 172.21.15.100:0/1821551404 shutdown_connections 2024-05-10T11:06:46.717 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:46.168+0000 7f113966a700 1 --2- 172.21.15.100:0/1821551404 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7f112003e420 0x7f11200408e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:06:46.717 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:46.168+0000 7f113966a700 1 --2- 172.21.15.100:0/1821551404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1134106e60 0x7f113419c9d0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:06:46.717 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:46.168+0000 7f113966a700 1 -- 172.21.15.100:0/1821551404 >> 172.21.15.100:0/1821551404 conn(0x7f1134102790 msgr2=0x7f11341032a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:06:46.718 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:46.168+0000 7f113966a700 1 -- 172.21.15.100:0/1821551404 shutdown_connections 2024-05-10T11:06:46.718 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:46.168+0000 7f113966a700 1 -- 172.21.15.100:0/1821551404 wait complete. 2024-05-10T11:06:46.718 INFO:teuthology.orchestra.run.smithi100.stdout:Deploying unmanaged mgr service... 2024-05-10T11:06:47.385 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:47 smithi100 ceph-mon[30296]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:06:47.385 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:47 smithi100 ceph-mon[30296]: Saving service mon spec with placement count:5 2024-05-10T11:06:47.385 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:47 smithi100 ceph-mon[30296]: from='mgr.14120 172.21.15.100:0/2558397845' entity='mgr.x' 2024-05-10T11:06:48.050 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-05-10T11:06:48.050 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:47.145+0000 7f1c07e3b700 1 Processor -- start 2024-05-10T11:06:48.050 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:47.145+0000 7f1c07e3b700 1 -- start start 2024-05-10T11:06:48.050 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:47.146+0000 7f1c07e3b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c0007ee30 0x7f1c0007d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:06:48.050 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:47.146+0000 7f1c07e3b700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0007d860 con 0x7f1c0007ee30 2024-05-10T11:06:48.051 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:47.146+0000 7f1c05bd7700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c0007ee30 0x7f1c0007d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:06:48.051 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:47.146+0000 7f1c05bd7700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c0007ee30 0x7f1c0007d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33496/0 (socket says 172.21.15.100:33496) 2024-05-10T11:06:48.051 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:47.146+0000 7f1c05bd7700 1 -- 172.21.15.100:0/1541135880 learned_addr learned my addr 172.21.15.100:0/1541135880 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:06:48.051 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:47.146+0000 7f1c05bd7700 1 -- 172.21.15.100:0/1541135880 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c0007d9e0 con 0x7f1c0007ee30 2024-05-10T11:06:48.051 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:47.146+0000 7f1c05bd7700 1 --2- 172.21.15.100:0/1541135880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c0007ee30 0x7f1c0007d290 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc0098a0 tx=0x7f1bfc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=614c9f35d5a01203 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:06:48.051 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:47.146+0000 7f1c04bd5700 1 -- 172.21.15.100:0/1541135880 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1bfc004160 con 0x7f1c0007ee30 2024-05-10T11:06:48.051 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:47.146+0000 7f1c04bd5700 1 -- 172.21.15.100:0/1541135880 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f1bfc004300 con 0x7f1c0007ee30 2024-05-10T11:06:48.051 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:47.146+0000 7f1c04bd5700 1 -- 172.21.15.100:0/1541135880 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1bfc0119e0 con 0x7f1c0007ee30 2024-05-10T11:06:48.051 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:47.147+0000 7f1c07e3b700 1 -- 172.21.15.100:0/1541135880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c0007ee30 msgr2=0x7f1c0007d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:06:48.051 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:47.147+0000 7f1c07e3b700 1 --2- 172.21.15.100:0/1541135880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c0007ee30 0x7f1c0007d290 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc0098a0 tx=0x7f1bfc009bb0 comp rx=0 tx=0).stop 2024-05-10T11:06:48.051 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:47.147+0000 7f1c07e3b700 1 -- 172.21.15.100:0/1541135880 shutdown_connections 2024-05-10T11:06:48.051 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:47.147+0000 7f1c07e3b700 1 --2- 172.21.15.100:0/1541135880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c0007ee30 0x7f1c0007d290 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:06:48.051 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:47.147+0000 7f1c07e3b700 1 -- 172.21.15.100:0/1541135880 >> 172.21.15.100:0/1541135880 conn(0x7f1c001040b0 msgr2=0x7f1c001064d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:06:48.051 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:47.147+0000 7f1c07e3b700 1 -- 172.21.15.100:0/1541135880 shutdown_connections 2024-05-10T11:06:48.052 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:47.147+0000 7f1c07e3b700 1 -- 172.21.15.100:0/1541135880 wait complete. 2024-05-10T11:06:48.052 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:47.148+0000 7f1c07e3b700 1 Processor -- start 2024-05-10T11:06:48.052 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:47.148+0000 7f1c07e3b700 1 -- start start 2024-05-10T11:06:48.052 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:47.148+0000 7f1c07e3b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c0007ee30 0x7f1c0019eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:06:48.052 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:47.148+0000 7f1c07e3b700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0010f040 con 0x7f1c0007ee30 2024-05-10T11:06:48.052 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:47.148+0000 7f1c05bd7700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c0007ee30 0x7f1c0019eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:06:48.052 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:47.148+0000 7f1c05bd7700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c0007ee30 0x7f1c0019eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33512/0 (socket says 172.21.15.100:33512) 2024-05-10T11:06:48.052 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:47.148+0000 7f1c05bd7700 1 -- 172.21.15.100:0/1808326231 learned_addr learned my addr 172.21.15.100:0/1808326231 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:06:48.052 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:47.148+0000 7f1c05bd7700 1 -- 172.21.15.100:0/1808326231 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c0019f0e0 con 0x7f1c0007ee30 2024-05-10T11:06:48.052 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:47.148+0000 7f1c05bd7700 1 --2- 172.21.15.100:0/1808326231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c0007ee30 0x7f1c0019eba0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc0116a0 tx=0x7f1bfc011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:06:48.052 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:47.148+0000 7f1bf6ffd700 1 -- 172.21.15.100:0/1808326231 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1bfc0039f0 con 0x7f1c0007ee30 2024-05-10T11:06:48.052 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:47.148+0000 7f1bf6ffd700 1 -- 172.21.15.100:0/1808326231 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f1bfc003b90 con 0x7f1c0007ee30 2024-05-10T11:06:48.052 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:47.148+0000 7f1c07e3b700 1 -- 172.21.15.100:0/1808326231 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c0019f370 con 0x7f1c0007ee30 2024-05-10T11:06:48.052 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:47.148+0000 7f1bf6ffd700 1 -- 172.21.15.100:0/1808326231 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1bfc01a600 con 0x7f1c0007ee30 2024-05-10T11:06:48.053 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:47.149+0000 7f1c07e3b700 1 -- 172.21.15.100:0/1808326231 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c001a2060 con 0x7f1c0007ee30 2024-05-10T11:06:48.053 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:47.149+0000 7f1bf6ffd700 1 -- 172.21.15.100:0/1808326231 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f1bfc024430 con 0x7f1c0007ee30 2024-05-10T11:06:48.053 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:47.149+0000 7f1c07e3b700 1 -- 172.21.15.100:0/1808326231 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c00064b00 con 0x7f1c0007ee30 2024-05-10T11:06:48.053 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:47.150+0000 7f1bf6ffd700 1 --2- 172.21.15.100:0/1808326231 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7f1bec03e420 0x7f1bec0408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:06:48.053 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:47.150+0000 7f1bf6ffd700 1 -- 172.21.15.100:0/1808326231 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1bfc053bb0 con 0x7f1c0007ee30 2024-05-10T11:06:48.053 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:47.150+0000 7f1c053d6700 1 --2- 172.21.15.100:0/1808326231 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7f1bec03e420 0x7f1bec0408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:06:48.053 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:47.150+0000 7f1c053d6700 1 --2- 172.21.15.100:0/1808326231 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7f1bec03e420 0x7f1bec0408e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1bf0006fd0 tx=0x7f1bf0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:06:48.053 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:47.155+0000 7f1bf6ffd700 1 -- 172.21.15.100:0/1808326231 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f1bfc018a40 con 0x7f1c0007ee30 2024-05-10T11:06:48.053 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:47.384+0000 7f1c07e3b700 1 -- 172.21.15.100:0/1808326231 --> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f1c00105280 con 0x7f1bec03e420 2024-05-10T11:06:48.053 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:47.392+0000 7f1bf6ffd700 1 -- 172.21.15.100:0/1808326231 <== mgr.14120 v2:172.21.15.100:6800/1597433518 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f1c00105280 con 0x7f1bec03e420 2024-05-10T11:06:48.053 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:47.394+0000 7f1c07e3b700 1 -- 172.21.15.100:0/1808326231 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7f1bec03e420 msgr2=0x7f1bec0408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:06:48.053 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:47.394+0000 7f1c07e3b700 1 --2- 172.21.15.100:0/1808326231 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7f1bec03e420 0x7f1bec0408e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1bf0006fd0 tx=0x7f1bf0006e40 comp rx=0 tx=0).stop 2024-05-10T11:06:48.053 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:47.394+0000 7f1c07e3b700 1 -- 172.21.15.100:0/1808326231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c0007ee30 msgr2=0x7f1c0019eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:06:48.053 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:47.394+0000 7f1c07e3b700 1 --2- 172.21.15.100:0/1808326231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c0007ee30 0x7f1c0019eba0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc0116a0 tx=0x7f1bfc011a60 comp rx=0 tx=0).stop 2024-05-10T11:06:48.054 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:47.394+0000 7f1c07e3b700 1 -- 172.21.15.100:0/1808326231 shutdown_connections 2024-05-10T11:06:48.054 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:47.394+0000 7f1c07e3b700 1 --2- 172.21.15.100:0/1808326231 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7f1bec03e420 0x7f1bec0408e0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:06:48.054 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:47.394+0000 7f1c07e3b700 1 --2- 172.21.15.100:0/1808326231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c0007ee30 0x7f1c0019eba0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:06:48.054 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:47.394+0000 7f1c07e3b700 1 -- 172.21.15.100:0/1808326231 >> 172.21.15.100:0/1808326231 conn(0x7f1c001040b0 msgr2=0x7f1c00104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:06:48.054 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:47.394+0000 7f1c07e3b700 1 -- 172.21.15.100:0/1808326231 shutdown_connections 2024-05-10T11:06:48.054 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:47.394+0000 7f1c07e3b700 1 -- 172.21.15.100:0/1808326231 wait complete. 2024-05-10T11:06:48.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:48 smithi100 ceph-mon[30296]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:06:48.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:48 smithi100 ceph-mon[30296]: Saving service mgr spec with placement count:2 2024-05-10T11:06:48.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:48 smithi100 ceph-mon[30296]: from='mgr.14120 172.21.15.100:0/2558397845' entity='mgr.x' 2024-05-10T11:06:50.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:49 smithi100 ceph-mon[30296]: from='mgr.14120 172.21.15.100:0/2558397845' entity='mgr.x' 2024-05-10T11:06:50.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:49 smithi100 ceph-mon[30296]: from='mgr.14120 172.21.15.100:0/2558397845' entity='mgr.x' 2024-05-10T11:06:50.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:49 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/574982756' entity='client.admin' 2024-05-10T11:06:50.193 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:49.348+0000 7fd5d48e4700 1 Processor -- start 2024-05-10T11:06:50.193 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:49.348+0000 7fd5d48e4700 1 -- start start 2024-05-10T11:06:50.193 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:49.349+0000 7fd5d48e4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5d0106db0 0x7fd5d01071d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:06:50.194 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:49.349+0000 7fd5d48e4700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d01077a0 con 0x7fd5d0106db0 2024-05-10T11:06:50.194 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:49.349+0000 7fd5ce59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5d0106db0 0x7fd5d01071d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:06:50.194 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:49.349+0000 7fd5ce59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5d0106db0 0x7fd5d01071d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33522/0 (socket says 172.21.15.100:33522) 2024-05-10T11:06:50.194 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:49.349+0000 7fd5ce59c700 1 -- 172.21.15.100:0/3858504477 learned_addr learned my addr 172.21.15.100:0/3858504477 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:06:50.194 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:49.349+0000 7fd5ce59c700 1 -- 172.21.15.100:0/3858504477 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5d0109d90 con 0x7fd5d0106db0 2024-05-10T11:06:50.194 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:49.349+0000 7fd5ce59c700 1 --2- 172.21.15.100:0/3858504477 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5d0106db0 0x7fd5d01071d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0009e80 tx=0x7fd5c000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a16d466914a0c820 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:06:50.194 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:49.349+0000 7fd5cd59a700 1 -- 172.21.15.100:0/3858504477 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5c0017070 con 0x7fd5d0106db0 2024-05-10T11:06:50.194 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:49.349+0000 7fd5cd59a700 1 -- 172.21.15.100:0/3858504477 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd5c000b8c0 con 0x7fd5d0106db0 2024-05-10T11:06:50.194 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:49.349+0000 7fd5cd59a700 1 -- 172.21.15.100:0/3858504477 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5c0004420 con 0x7fd5d0106db0 2024-05-10T11:06:50.194 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:49.350+0000 7fd5d48e4700 1 -- 172.21.15.100:0/3858504477 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5d0106db0 msgr2=0x7fd5d01071d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:06:50.194 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:49.350+0000 7fd5d48e4700 1 --2- 172.21.15.100:0/3858504477 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5d0106db0 0x7fd5d01071d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0009e80 tx=0x7fd5c000b330 comp rx=0 tx=0).stop 2024-05-10T11:06:50.194 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:49.350+0000 7fd5d48e4700 1 -- 172.21.15.100:0/3858504477 shutdown_connections 2024-05-10T11:06:50.194 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:49.350+0000 7fd5d48e4700 1 --2- 172.21.15.100:0/3858504477 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5d0106db0 0x7fd5d01071d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:06:50.195 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:49.350+0000 7fd5d48e4700 1 -- 172.21.15.100:0/3858504477 >> 172.21.15.100:0/3858504477 conn(0x7fd5d0102770 msgr2=0x7fd5d0104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:06:50.195 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:49.350+0000 7fd5d48e4700 1 -- 172.21.15.100:0/3858504477 shutdown_connections 2024-05-10T11:06:50.195 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:49.350+0000 7fd5d48e4700 1 -- 172.21.15.100:0/3858504477 wait complete. 2024-05-10T11:06:50.195 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:49.351+0000 7fd5d48e4700 1 Processor -- start 2024-05-10T11:06:50.195 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:49.351+0000 7fd5d48e4700 1 -- start start 2024-05-10T11:06:50.195 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:49.351+0000 7fd5d48e4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5d0106db0 0x7fd5d019cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:06:50.195 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:49.351+0000 7fd5d48e4700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d010cfc0 con 0x7fd5d0106db0 2024-05-10T11:06:50.195 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:49.351+0000 7fd5ce59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5d0106db0 0x7fd5d019cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:06:50.195 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:49.351+0000 7fd5ce59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5d0106db0 0x7fd5d019cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33536/0 (socket says 172.21.15.100:33536) 2024-05-10T11:06:50.195 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:49.352+0000 7fd5ce59c700 1 -- 172.21.15.100:0/574982756 learned_addr learned my addr 172.21.15.100:0/574982756 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:06:50.195 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:49.352+0000 7fd5ce59c700 1 -- 172.21.15.100:0/574982756 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5d019cff0 con 0x7fd5d0106db0 2024-05-10T11:06:50.195 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:49.352+0000 7fd5ce59c700 1 --2- 172.21.15.100:0/574982756 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5d0106db0 0x7fd5d019cab0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0009e50 tx=0x7fd5c000bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:06:50.195 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:49.352+0000 7fd5c77fe700 1 -- 172.21.15.100:0/574982756 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5c0020020 con 0x7fd5d0106db0 2024-05-10T11:06:50.196 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:49.352+0000 7fd5c77fe700 1 -- 172.21.15.100:0/574982756 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd5c0003d90 con 0x7fd5d0106db0 2024-05-10T11:06:50.196 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:49.352+0000 7fd5c77fe700 1 -- 172.21.15.100:0/574982756 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5c0017040 con 0x7fd5d0106db0 2024-05-10T11:06:50.196 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:49.352+0000 7fd5d48e4700 1 -- 172.21.15.100:0/574982756 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5d019d280 con 0x7fd5d0106db0 2024-05-10T11:06:50.196 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:49.352+0000 7fd5d48e4700 1 -- 172.21.15.100:0/574982756 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5d019ff70 con 0x7fd5d0106db0 2024-05-10T11:06:50.196 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:49.353+0000 7fd5c77fe700 1 -- 172.21.15.100:0/574982756 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fd5c0023020 con 0x7fd5d0106db0 2024-05-10T11:06:50.196 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:49.353+0000 7fd5d48e4700 1 -- 172.21.15.100:0/574982756 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5d0064b00 con 0x7fd5d0106db0 2024-05-10T11:06:50.196 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:49.353+0000 7fd5c77fe700 1 --2- 172.21.15.100:0/574982756 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7fd5bc03e470 0x7fd5bc040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:06:50.196 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:49.354+0000 7fd5c77fe700 1 -- 172.21.15.100:0/574982756 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd5c000f050 con 0x7fd5d0106db0 2024-05-10T11:06:50.196 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:49.354+0000 7fd5cdd9b700 1 --2- 172.21.15.100:0/574982756 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7fd5bc03e470 0x7fd5bc040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:06:50.196 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:49.354+0000 7fd5cdd9b700 1 --2- 172.21.15.100:0/574982756 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7fd5bc03e470 0x7fd5bc040930 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8006fd0 tx=0x7fd5b8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:06:50.196 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:49.359+0000 7fd5c77fe700 1 -- 172.21.15.100:0/574982756 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fd5c002a020 con 0x7fd5d0106db0 2024-05-10T11:06:50.196 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:49.581+0000 7fd5d48e4700 1 -- 172.21.15.100:0/574982756 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fd5d01a0540 con 0x7fd5d0106db0 2024-05-10T11:06:50.196 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:49.584+0000 7fd5c77fe700 1 -- 172.21.15.100:0/574982756 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7fd5c001b370 con 0x7fd5d0106db0 2024-05-10T11:06:50.196 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:49.586+0000 7fd5d48e4700 1 -- 172.21.15.100:0/574982756 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7fd5bc03e470 msgr2=0x7fd5bc040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:06:50.197 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:49.586+0000 7fd5d48e4700 1 --2- 172.21.15.100:0/574982756 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7fd5bc03e470 0x7fd5bc040930 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8006fd0 tx=0x7fd5b8006e40 comp rx=0 tx=0).stop 2024-05-10T11:06:50.197 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:49.586+0000 7fd5d48e4700 1 -- 172.21.15.100:0/574982756 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5d0106db0 msgr2=0x7fd5d019cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:06:50.197 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:49.586+0000 7fd5d48e4700 1 --2- 172.21.15.100:0/574982756 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5d0106db0 0x7fd5d019cab0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0009e50 tx=0x7fd5c000bd50 comp rx=0 tx=0).stop 2024-05-10T11:06:50.197 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:49.587+0000 7fd5d48e4700 1 -- 172.21.15.100:0/574982756 shutdown_connections 2024-05-10T11:06:50.197 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:49.587+0000 7fd5d48e4700 1 --2- 172.21.15.100:0/574982756 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7fd5bc03e470 0x7fd5bc040930 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:06:50.197 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:49.587+0000 7fd5d48e4700 1 --2- 172.21.15.100:0/574982756 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5d0106db0 0x7fd5d019cab0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:06:50.197 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:49.587+0000 7fd5d48e4700 1 -- 172.21.15.100:0/574982756 >> 172.21.15.100:0/574982756 conn(0x7fd5d0102770 msgr2=0x7fd5d0103390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:06:50.197 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:49.587+0000 7fd5d48e4700 1 -- 172.21.15.100:0/574982756 shutdown_connections 2024-05-10T11:06:50.198 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:49.587+0000 7fd5d48e4700 1 -- 172.21.15.100:0/574982756 wait complete. 2024-05-10T11:06:51.601 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:50.660+0000 7f2125038700 1 Processor -- start 2024-05-10T11:06:51.601 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:50.660+0000 7f2125038700 1 -- start start 2024-05-10T11:06:51.602 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:50.661+0000 7f2125038700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2120109030 0x7f2120109450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:06:51.603 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:50.661+0000 7f2125038700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2120109a20 con 0x7f2120109030 2024-05-10T11:06:51.603 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:50.661+0000 7f211ed9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2120109030 0x7f2120109450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:06:51.603 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:50.661+0000 7f211ed9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2120109030 0x7f2120109450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33542/0 (socket says 172.21.15.100:33542) 2024-05-10T11:06:51.603 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:50.661+0000 7f211ed9d700 1 -- 172.21.15.100:0/2188948607 learned_addr learned my addr 172.21.15.100:0/2188948607 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:06:51.603 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:50.661+0000 7f211ed9d700 1 -- 172.21.15.100:0/2188948607 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2120107c90 con 0x7f2120109030 2024-05-10T11:06:51.603 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:50.661+0000 7f211ed9d700 1 --2- 172.21.15.100:0/2188948607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2120109030 0x7f2120109450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f21080098a0 tx=0x7f2108009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d024aa3eedd9a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:06:51.603 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:50.661+0000 7f211dd9b700 1 -- 172.21.15.100:0/2188948607 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2108004160 con 0x7f2120109030 2024-05-10T11:06:51.603 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:50.661+0000 7f211dd9b700 1 -- 172.21.15.100:0/2188948607 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2108004300 con 0x7f2120109030 2024-05-10T11:06:51.603 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:50.661+0000 7f211dd9b700 1 -- 172.21.15.100:0/2188948607 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2108011960 con 0x7f2120109030 2024-05-10T11:06:51.603 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:50.662+0000 7f2125038700 1 -- 172.21.15.100:0/2188948607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2120109030 msgr2=0x7f2120109450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:06:51.603 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:50.662+0000 7f2125038700 1 --2- 172.21.15.100:0/2188948607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2120109030 0x7f2120109450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f21080098a0 tx=0x7f2108009bb0 comp rx=0 tx=0).stop 2024-05-10T11:06:51.603 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:50.662+0000 7f2125038700 1 -- 172.21.15.100:0/2188948607 shutdown_connections 2024-05-10T11:06:51.603 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:50.662+0000 7f2125038700 1 --2- 172.21.15.100:0/2188948607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2120109030 0x7f2120109450 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:06:51.603 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:50.662+0000 7f2125038700 1 -- 172.21.15.100:0/2188948607 >> 172.21.15.100:0/2188948607 conn(0x7f21201027a0 msgr2=0x7f2120104bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:06:51.604 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:50.662+0000 7f2125038700 1 -- 172.21.15.100:0/2188948607 shutdown_connections 2024-05-10T11:06:51.604 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:50.662+0000 7f2125038700 1 -- 172.21.15.100:0/2188948607 wait complete. 2024-05-10T11:06:51.604 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:50.663+0000 7f2125038700 1 Processor -- start 2024-05-10T11:06:51.604 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:50.663+0000 7f2125038700 1 -- start start 2024-05-10T11:06:51.604 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:50.663+0000 7f2125038700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2120109030 0x7f212019cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:06:51.604 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:50.663+0000 7f2125038700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f212010bed0 con 0x7f2120109030 2024-05-10T11:06:51.604 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:50.663+0000 7f211ed9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2120109030 0x7f212019cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:06:51.604 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:50.663+0000 7f211ed9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2120109030 0x7f212019cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33554/0 (socket says 172.21.15.100:33554) 2024-05-10T11:06:51.604 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:50.663+0000 7f211ed9d700 1 -- 172.21.15.100:0/3133684249 learned_addr learned my addr 172.21.15.100:0/3133684249 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:06:51.604 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:50.663+0000 7f211ed9d700 1 -- 172.21.15.100:0/3133684249 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f212019d0c0 con 0x7f2120109030 2024-05-10T11:06:51.604 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:50.664+0000 7f211ed9d700 1 --2- 172.21.15.100:0/3133684249 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2120109030 0x7f212019cb80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2108011670 tx=0x7f2108011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:06:51.604 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:50.664+0000 7f2117fff700 1 -- 172.21.15.100:0/3133684249 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2108003a60 con 0x7f2120109030 2024-05-10T11:06:51.604 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:50.664+0000 7f2117fff700 1 -- 172.21.15.100:0/3133684249 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f210801b8d0 con 0x7f2120109030 2024-05-10T11:06:51.605 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:50.664+0000 7f2125038700 1 -- 172.21.15.100:0/3133684249 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f212019d350 con 0x7f2120109030 2024-05-10T11:06:51.605 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:50.664+0000 7f2117fff700 1 -- 172.21.15.100:0/3133684249 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2108023750 con 0x7f2120109030 2024-05-10T11:06:51.605 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:50.664+0000 7f2125038700 1 -- 172.21.15.100:0/3133684249 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21201a0040 con 0x7f2120109030 2024-05-10T11:06:51.605 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:50.665+0000 7f2117fff700 1 -- 172.21.15.100:0/3133684249 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f2108023a70 con 0x7f2120109030 2024-05-10T11:06:51.605 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:50.665+0000 7f2117fff700 1 --2- 172.21.15.100:0/3133684249 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7f210c03e420 0x7f210c0408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:06:51.605 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:50.665+0000 7f2117fff700 1 -- 172.21.15.100:0/3133684249 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2108054db0 con 0x7f2120109030 2024-05-10T11:06:51.605 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:50.665+0000 7f2125038700 1 -- 172.21.15.100:0/3133684249 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2120064b00 con 0x7f2120109030 2024-05-10T11:06:51.605 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:50.665+0000 7f211e59c700 1 --2- 172.21.15.100:0/3133684249 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7f210c03e420 0x7f210c0408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:06:51.605 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:50.665+0000 7f211e59c700 1 --2- 172.21.15.100:0/3133684249 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7f210c03e420 0x7f210c0408e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2110006fd0 tx=0x7f2110006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:06:51.605 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:50.670+0000 7f2117fff700 1 -- 172.21.15.100:0/3133684249 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f210801f030 con 0x7f2120109030 2024-05-10T11:06:51.605 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:50.895+0000 7f2125038700 1 -- 172.21.15.100:0/3133684249 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f212010c050 con 0x7f2120109030 2024-05-10T11:06:51.605 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:50.897+0000 7f2117fff700 1 -- 172.21.15.100:0/3133684249 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f210800b3e0 con 0x7f2120109030 2024-05-10T11:06:51.605 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:50.900+0000 7f2125038700 1 -- 172.21.15.100:0/3133684249 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7f210c03e420 msgr2=0x7f210c0408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:06:51.605 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:50.900+0000 7f2125038700 1 --2- 172.21.15.100:0/3133684249 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7f210c03e420 0x7f210c0408e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2110006fd0 tx=0x7f2110006e40 comp rx=0 tx=0).stop 2024-05-10T11:06:51.606 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:50.900+0000 7f2125038700 1 -- 172.21.15.100:0/3133684249 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2120109030 msgr2=0x7f212019cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:06:51.606 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:50.900+0000 7f2125038700 1 --2- 172.21.15.100:0/3133684249 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2120109030 0x7f212019cb80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2108011670 tx=0x7f2108011a30 comp rx=0 tx=0).stop 2024-05-10T11:06:51.606 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:50.900+0000 7f2125038700 1 -- 172.21.15.100:0/3133684249 shutdown_connections 2024-05-10T11:06:51.606 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:50.900+0000 7f2125038700 1 --2- 172.21.15.100:0/3133684249 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7f210c03e420 0x7f210c0408e0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:06:51.606 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:50.900+0000 7f2125038700 1 --2- 172.21.15.100:0/3133684249 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2120109030 0x7f212019cb80 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:06:51.606 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:50.900+0000 7f2125038700 1 -- 172.21.15.100:0/3133684249 >> 172.21.15.100:0/3133684249 conn(0x7f21201027a0 msgr2=0x7f2120103240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:06:51.606 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:50.900+0000 7f2125038700 1 -- 172.21.15.100:0/3133684249 shutdown_connections 2024-05-10T11:06:51.606 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:50.900+0000 7f2125038700 1 -- 172.21.15.100:0/3133684249 wait complete. 2024-05-10T11:06:51.606 INFO:teuthology.orchestra.run.smithi100.stdout:Enabling the dashboard module... 2024-05-10T11:06:52.180 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:51 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/3133684249' entity='client.admin' 2024-05-10T11:06:53.218 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:52 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/1032214209' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-05-10T11:06:53.218 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:52 smithi100 ceph-mon[30296]: from='mgr.14120 172.21.15.100:0/2558397845' entity='mgr.x' 2024-05-10T11:06:53.219 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:52 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: ignoring --setuser ceph since I am not root 2024-05-10T11:06:53.219 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:52 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: ignoring --setgroup ceph since I am not root 2024-05-10T11:06:53.219 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:53 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:53.082+0000 7f6216ae5200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-10T11:06:53.503 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:53 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:53.268+0000 7f6216ae5200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-10T11:06:53.503 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:53 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:53.370+0000 7f6216ae5200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-10T11:06:53.505 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:52.023+0000 7f2855713700 1 Processor -- start 2024-05-10T11:06:53.511 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:52.023+0000 7f2855713700 1 -- start start 2024-05-10T11:06:53.511 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:52.023+0000 7f2855713700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28501073c0 0x7f28501077e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:06:53.511 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:52.023+0000 7f2855713700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f285007ce00 con 0x7f28501073c0 2024-05-10T11:06:53.511 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:52.023+0000 7f284effd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28501073c0 0x7f28501077e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:06:53.511 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:52.023+0000 7f284effd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28501073c0 0x7f28501077e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33556/0 (socket says 172.21.15.100:33556) 2024-05-10T11:06:53.511 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:52.023+0000 7f284effd700 1 -- 172.21.15.100:0/2239633121 learned_addr learned my addr 172.21.15.100:0/2239633121 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:06:53.512 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:52.024+0000 7f284effd700 1 -- 172.21.15.100:0/2239633121 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2850107d20 con 0x7f28501073c0 2024-05-10T11:06:53.512 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:52.024+0000 7f284effd700 1 --2- 172.21.15.100:0/2239633121 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28501073c0 0x7f28501077e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f28400098a0 tx=0x7f2840009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb0fa1a89240c146 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:06:53.512 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:52.024+0000 7f284dffb700 1 -- 172.21.15.100:0/2239633121 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2840004160 con 0x7f28501073c0 2024-05-10T11:06:53.512 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:52.024+0000 7f284dffb700 1 -- 172.21.15.100:0/2239633121 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2840004300 con 0x7f28501073c0 2024-05-10T11:06:53.512 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:52.024+0000 7f284dffb700 1 -- 172.21.15.100:0/2239633121 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2840011960 con 0x7f28501073c0 2024-05-10T11:06:53.512 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:52.024+0000 7f2855713700 1 -- 172.21.15.100:0/2239633121 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28501073c0 msgr2=0x7f28501077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:06:53.512 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:52.024+0000 7f2855713700 1 --2- 172.21.15.100:0/2239633121 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28501073c0 0x7f28501077e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f28400098a0 tx=0x7f2840009bb0 comp rx=0 tx=0).stop 2024-05-10T11:06:53.512 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:52.024+0000 7f2855713700 1 -- 172.21.15.100:0/2239633121 shutdown_connections 2024-05-10T11:06:53.512 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:52.024+0000 7f2855713700 1 --2- 172.21.15.100:0/2239633121 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28501073c0 0x7f28501077e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:06:53.512 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:52.024+0000 7f2855713700 1 -- 172.21.15.100:0/2239633121 >> 172.21.15.100:0/2239633121 conn(0x7f2850102fa0 msgr2=0x7f2850105400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:06:53.512 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:52.025+0000 7f2855713700 1 -- 172.21.15.100:0/2239633121 shutdown_connections 2024-05-10T11:06:53.512 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:52.025+0000 7f2855713700 1 -- 172.21.15.100:0/2239633121 wait complete. 2024-05-10T11:06:53.513 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:52.025+0000 7f2855713700 1 Processor -- start 2024-05-10T11:06:53.513 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:52.025+0000 7f2855713700 1 -- start start 2024-05-10T11:06:53.513 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:52.025+0000 7f2855713700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28501073c0 0x7f285019ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:06:53.513 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:52.025+0000 7f2855713700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2850108d30 con 0x7f28501073c0 2024-05-10T11:06:53.513 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:52.026+0000 7f284effd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28501073c0 0x7f285019ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:06:53.513 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:52.026+0000 7f284effd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28501073c0 0x7f285019ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33572/0 (socket says 172.21.15.100:33572) 2024-05-10T11:06:53.513 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:52.026+0000 7f284effd700 1 -- 172.21.15.100:0/1032214209 learned_addr learned my addr 172.21.15.100:0/1032214209 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:06:53.513 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:52.026+0000 7f284effd700 1 -- 172.21.15.100:0/1032214209 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f285019f120 con 0x7f28501073c0 2024-05-10T11:06:53.513 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:52.026+0000 7f284effd700 1 --2- 172.21.15.100:0/1032214209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28501073c0 0x7f285019ebe0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2840011980 tx=0x7f2840011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:06:53.513 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:52.026+0000 7f2837fff700 1 -- 172.21.15.100:0/1032214209 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f28400039e0 con 0x7f28501073c0 2024-05-10T11:06:53.513 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:52.026+0000 7f2855713700 1 -- 172.21.15.100:0/1032214209 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f285019f3b0 con 0x7f28501073c0 2024-05-10T11:06:53.513 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:52.026+0000 7f2837fff700 1 -- 172.21.15.100:0/1032214209 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f284001b8d0 con 0x7f28501073c0 2024-05-10T11:06:53.514 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:52.026+0000 7f2837fff700 1 -- 172.21.15.100:0/1032214209 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2840023710 con 0x7f28501073c0 2024-05-10T11:06:53.514 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:52.026+0000 7f2855713700 1 -- 172.21.15.100:0/1032214209 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28501a20a0 con 0x7f28501073c0 2024-05-10T11:06:53.514 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:52.027+0000 7f2837fff700 1 -- 172.21.15.100:0/1032214209 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f284002c3f0 con 0x7f28501073c0 2024-05-10T11:06:53.514 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:52.027+0000 7f2855713700 1 -- 172.21.15.100:0/1032214209 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2850064b00 con 0x7f28501073c0 2024-05-10T11:06:53.514 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:52.027+0000 7f2837fff700 1 --2- 172.21.15.100:0/1032214209 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7f283803e420 0x7f28380408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:06:53.514 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:52.027+0000 7f2837fff700 1 -- 172.21.15.100:0/1032214209 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f28400540d0 con 0x7f28501073c0 2024-05-10T11:06:53.514 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:52.027+0000 7f284e7fc700 1 --2- 172.21.15.100:0/1032214209 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7f283803e420 0x7f28380408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:06:53.514 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:52.028+0000 7f284e7fc700 1 --2- 172.21.15.100:0/1032214209 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7f283803e420 0x7f28380408e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2844006fd0 tx=0x7f2844006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:06:53.514 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:52.032+0000 7f2837fff700 1 -- 172.21.15.100:0/1032214209 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f284001f030 con 0x7f28501073c0 2024-05-10T11:06:53.514 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:52.297+0000 7f2855713700 1 -- 172.21.15.100:0/1032214209 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f285002e7a0 con 0x7f28501073c0 2024-05-10T11:06:53.514 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:52.902+0000 7f2837fff700 1 -- 172.21.15.100:0/1032214209 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f284001ba80 con 0x7f28501073c0 2024-05-10T11:06:53.514 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:52.904+0000 7f2837fff700 1 -- 172.21.15.100:0/1032214209 <== mon.0 v2:172.21.15.100:3300/0 8 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f284000b3e0 con 0x7f28501073c0 2024-05-10T11:06:53.515 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:52.905+0000 7f2855713700 1 -- 172.21.15.100:0/1032214209 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7f283803e420 msgr2=0x7f28380408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:06:53.515 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:52.905+0000 7f2855713700 1 --2- 172.21.15.100:0/1032214209 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7f283803e420 0x7f28380408e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2844006fd0 tx=0x7f2844006e40 comp rx=0 tx=0).stop 2024-05-10T11:06:53.515 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:52.905+0000 7f2855713700 1 -- 172.21.15.100:0/1032214209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28501073c0 msgr2=0x7f285019ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:06:53.515 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:52.905+0000 7f2855713700 1 --2- 172.21.15.100:0/1032214209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28501073c0 0x7f285019ebe0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2840011980 tx=0x7f2840011a60 comp rx=0 tx=0).stop 2024-05-10T11:06:53.515 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:52.905+0000 7f2855713700 1 -- 172.21.15.100:0/1032214209 shutdown_connections 2024-05-10T11:06:53.515 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:52.905+0000 7f2855713700 1 --2- 172.21.15.100:0/1032214209 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7f283803e420 0x7f28380408e0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:06:53.515 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:52.905+0000 7f2855713700 1 --2- 172.21.15.100:0/1032214209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28501073c0 0x7f285019ebe0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:06:53.515 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:52.905+0000 7f2855713700 1 -- 172.21.15.100:0/1032214209 >> 172.21.15.100:0/1032214209 conn(0x7f2850102fa0 msgr2=0x7f28501958d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:06:53.515 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:52.905+0000 7f2855713700 1 -- 172.21.15.100:0/1032214209 shutdown_connections 2024-05-10T11:06:53.515 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:52.905+0000 7f2855713700 1 -- 172.21.15.100:0/1032214209 wait complete. 2024-05-10T11:06:53.769 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:53 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:53.681+0000 7f6216ae5200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-10T11:06:53.769 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:53 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:53.769+0000 7f6216ae5200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-10T11:06:54.022 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:53 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/1032214209' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-10T11:06:54.023 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:53 smithi100 ceph-mon[30296]: mgrmap e9: x(active, since 19s) 2024-05-10T11:06:55.025 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:06:54 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/3329756194' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-10T11:06:55.026 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:54 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:54.990+0000 7f6216ae5200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-10T11:06:55.034 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout { 2024-05-10T11:06:55.034 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-05-10T11:06:55.034 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-10T11:06:55.034 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-10T11:06:55.034 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-10T11:06:55.034 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout } 2024-05-10T11:06:55.035 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:54.127+0000 7fe314eeb700 1 Processor -- start 2024-05-10T11:06:55.035 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:54.128+0000 7fe314eeb700 1 -- start start 2024-05-10T11:06:55.035 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:54.128+0000 7fe314eeb700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe310109590 0x7fe31010b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:06:55.035 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:54.128+0000 7fe314eeb700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe31007ce40 con 0x7fe310109590 2024-05-10T11:06:55.035 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:54.129+0000 7fe30e59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe310109590 0x7fe31010b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:06:55.035 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:54.129+0000 7fe30e59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe310109590 0x7fe31010b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33596/0 (socket says 172.21.15.100:33596) 2024-05-10T11:06:55.035 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:54.129+0000 7fe30e59c700 1 -- 172.21.15.100:0/1301906403 learned_addr learned my addr 172.21.15.100:0/1301906403 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:06:55.035 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:54.129+0000 7fe30e59c700 1 -- 172.21.15.100:0/1301906403 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe31010bec0 con 0x7fe310109590 2024-05-10T11:06:55.035 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:54.129+0000 7fe30e59c700 1 --2- 172.21.15.100:0/1301906403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe310109590 0x7fe31010b980 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe3000098a0 tx=0x7fe300009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e1a4fd28a75210f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:06:55.035 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:54.129+0000 7fe30d59a700 1 -- 172.21.15.100:0/1301906403 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe300004160 con 0x7fe310109590 2024-05-10T11:06:55.035 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:54.129+0000 7fe30d59a700 1 -- 172.21.15.100:0/1301906403 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe300004300 con 0x7fe310109590 2024-05-10T11:06:55.035 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:54.129+0000 7fe30d59a700 1 -- 172.21.15.100:0/1301906403 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe300011960 con 0x7fe310109590 2024-05-10T11:06:55.035 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:54.130+0000 7fe314eeb700 1 -- 172.21.15.100:0/1301906403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe310109590 msgr2=0x7fe31010b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:06:55.036 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:54.130+0000 7fe314eeb700 1 --2- 172.21.15.100:0/1301906403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe310109590 0x7fe31010b980 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe3000098a0 tx=0x7fe300009bb0 comp rx=0 tx=0).stop 2024-05-10T11:06:55.036 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:54.130+0000 7fe314eeb700 1 -- 172.21.15.100:0/1301906403 shutdown_connections 2024-05-10T11:06:55.036 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:54.130+0000 7fe314eeb700 1 --2- 172.21.15.100:0/1301906403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe310109590 0x7fe31010b980 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:06:55.036 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:54.130+0000 7fe314eeb700 1 -- 172.21.15.100:0/1301906403 >> 172.21.15.100:0/1301906403 conn(0x7fe310102fa0 msgr2=0x7fe310105400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:06:55.036 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:54.130+0000 7fe314eeb700 1 -- 172.21.15.100:0/1301906403 shutdown_connections 2024-05-10T11:06:55.036 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:54.130+0000 7fe314eeb700 1 -- 172.21.15.100:0/1301906403 wait complete. 2024-05-10T11:06:55.036 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:54.131+0000 7fe314eeb700 1 Processor -- start 2024-05-10T11:06:55.037 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:54.131+0000 7fe314eeb700 1 -- start start 2024-05-10T11:06:55.037 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:54.131+0000 7fe314eeb700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe310109590 0x7fe31019c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:06:55.037 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:54.131+0000 7fe314eeb700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe31010ce70 con 0x7fe310109590 2024-05-10T11:06:55.037 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:54.132+0000 7fe30e59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe310109590 0x7fe31019c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:06:55.037 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:54.132+0000 7fe30e59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe310109590 0x7fe31019c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33598/0 (socket says 172.21.15.100:33598) 2024-05-10T11:06:55.037 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:54.132+0000 7fe30e59c700 1 -- 172.21.15.100:0/3329756194 learned_addr learned my addr 172.21.15.100:0/3329756194 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:06:55.037 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:54.132+0000 7fe30e59c700 1 -- 172.21.15.100:0/3329756194 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe31019ced0 con 0x7fe310109590 2024-05-10T11:06:55.037 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:54.132+0000 7fe30e59c700 1 --2- 172.21.15.100:0/3329756194 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe310109590 0x7fe31019c990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe300003e50 tx=0x7fe300011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:06:55.037 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:54.132+0000 7fe2ff7fe700 1 -- 172.21.15.100:0/3329756194 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe300003a60 con 0x7fe310109590 2024-05-10T11:06:55.037 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:54.132+0000 7fe2ff7fe700 1 -- 172.21.15.100:0/3329756194 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe30001b8d0 con 0x7fe310109590 2024-05-10T11:06:55.037 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:54.132+0000 7fe314eeb700 1 -- 172.21.15.100:0/3329756194 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe31019d160 con 0x7fe310109590 2024-05-10T11:06:55.037 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:54.132+0000 7fe2ff7fe700 1 -- 172.21.15.100:0/3329756194 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe3000237d0 con 0x7fe310109590 2024-05-10T11:06:55.037 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:54.132+0000 7fe314eeb700 1 -- 172.21.15.100:0/3329756194 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe31019fe50 con 0x7fe310109590 2024-05-10T11:06:55.037 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:54.133+0000 7fe314eeb700 1 -- 172.21.15.100:0/3329756194 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe31019d640 con 0x7fe310109590 2024-05-10T11:06:55.037 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:54.134+0000 7fe2ff7fe700 1 -- 172.21.15.100:0/3329756194 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7fe30001b440 con 0x7fe310109590 2024-05-10T11:06:55.038 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:54.134+0000 7fe2ff7fe700 1 --2- 172.21.15.100:0/3329756194 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7fe2f803e470 0x7fe2f8040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:06:55.038 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:54.134+0000 7fe2ff7fe700 1 -- 172.21.15.100:0/3329756194 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe3000540a0 con 0x7fe310109590 2024-05-10T11:06:55.038 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:54.134+0000 7fe30dd9b700 1 -- 172.21.15.100:0/3329756194 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7fe2f803e470 msgr2=0x7fe2f8040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.100:6800/1597433518 2024-05-10T11:06:55.038 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:54.134+0000 7fe30dd9b700 1 --2- 172.21.15.100:0/3329756194 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7fe2f803e470 0x7fe2f8040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-10T11:06:55.038 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:54.139+0000 7fe2ff7fe700 1 -- 172.21.15.100:0/3329756194 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fe31019d640 con 0x7fe310109590 2024-05-10T11:06:55.038 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:54.335+0000 7fe30dd9b700 1 -- 172.21.15.100:0/3329756194 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7fe2f803e470 msgr2=0x7fe2f8040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.100:6800/1597433518 2024-05-10T11:06:55.038 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:54.335+0000 7fe30dd9b700 1 --2- 172.21.15.100:0/3329756194 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7fe2f803e470 0x7fe2f8040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-10T11:06:55.038 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:54.440+0000 7fe314eeb700 1 -- 172.21.15.100:0/3329756194 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fe3101a0420 con 0x7fe310109590 2024-05-10T11:06:55.038 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:54.440+0000 7fe2ff7fe700 1 -- 172.21.15.100:0/3329756194 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fe30001f030 con 0x7fe310109590 2024-05-10T11:06:55.038 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:54.443+0000 7fe314eeb700 1 -- 172.21.15.100:0/3329756194 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7fe2f803e470 msgr2=0x7fe2f8040930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T11:06:55.038 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:54.443+0000 7fe314eeb700 1 --2- 172.21.15.100:0/3329756194 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7fe2f803e470 0x7fe2f8040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:06:55.038 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:54.443+0000 7fe314eeb700 1 -- 172.21.15.100:0/3329756194 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe310109590 msgr2=0x7fe31019c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:06:55.039 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:54.443+0000 7fe314eeb700 1 --2- 172.21.15.100:0/3329756194 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe310109590 0x7fe31019c990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe300003e50 tx=0x7fe300011a30 comp rx=0 tx=0).stop 2024-05-10T11:06:55.039 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:54.443+0000 7fe314eeb700 1 -- 172.21.15.100:0/3329756194 shutdown_connections 2024-05-10T11:06:55.039 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:54.443+0000 7fe314eeb700 1 --2- 172.21.15.100:0/3329756194 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7fe2f803e470 0x7fe2f8040930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:06:55.039 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:54.443+0000 7fe314eeb700 1 --2- 172.21.15.100:0/3329756194 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe310109590 0x7fe31019c990 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:06:55.039 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:54.443+0000 7fe314eeb700 1 -- 172.21.15.100:0/3329756194 >> 172.21.15.100:0/3329756194 conn(0x7fe310102fa0 msgr2=0x7fe310103a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:06:55.039 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:54.443+0000 7fe314eeb700 1 -- 172.21.15.100:0/3329756194 shutdown_connections 2024-05-10T11:06:55.039 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:54.443+0000 7fe314eeb700 1 -- 172.21.15.100:0/3329756194 wait complete. 2024-05-10T11:06:55.039 INFO:teuthology.orchestra.run.smithi100.stdout:Waiting for the mgr to restart... 2024-05-10T11:06:55.039 INFO:teuthology.orchestra.run.smithi100.stdout:Waiting for mgr epoch 9... 2024-05-10T11:06:55.278 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:55 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:55.168+0000 7f6216ae5200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-10T11:06:55.540 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:55 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:55.344+0000 7f6216ae5200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-10T11:06:55.955 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:55 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:55.641+0000 7f6216ae5200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-10T11:06:55.955 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:55 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:55.736+0000 7f6216ae5200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-10T11:06:55.955 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:55 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:55.955+0000 7f6216ae5200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-10T11:06:56.288 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:56 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:56.151+0000 7f6216ae5200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-10T11:06:57.038 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:56 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:56.774+0000 7f6216ae5200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-10T11:06:57.038 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:57 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:57.000+0000 7f6216ae5200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-10T11:06:57.788 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:57 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:57.430+0000 7f6216ae5200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-10T11:06:57.788 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:57 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:57.528+0000 7f6216ae5200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-10T11:06:58.277 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:57 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:57.922+0000 7f6216ae5200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-10T11:06:58.277 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:58 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:58.277+0000 7f6216ae5200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-10T11:06:58.538 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:58 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:58.371+0000 7f6216ae5200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-10T11:06:59.038 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:58 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:58.617+0000 7f6216ae5200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-10T11:06:59.038 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:58 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:58.752+0000 7f6216ae5200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-10T11:06:59.038 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:58 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:58.848+0000 7f6216ae5200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-10T11:06:59.538 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:59 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:59.173+0000 7f6216ae5200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-10T11:06:59.538 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:59 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:59.263+0000 7f6216ae5200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-10T11:07:00.144 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:06:59 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:06:59.870+0000 7f6216ae5200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-10T11:07:00.144 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:07:00 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:07:00.090+0000 7f6216ae5200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-10T11:07:00.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:00 smithi100 ceph-mon[30296]: Active manager daemon x restarted 2024-05-10T11:07:00.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:00 smithi100 ceph-mon[30296]: Activating manager daemon x 2024-05-10T11:07:00.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:00 smithi100 ceph-mon[30296]: osdmap e3: 0 total, 0 up, 0 in 2024-05-10T11:07:00.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:00 smithi100 ceph-mon[30296]: mgrmap e10: x(active, starting, since 0.00389247s) 2024-05-10T11:07:00.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:00 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T11:07:00.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:00 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-10T11:07:00.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:00 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-10T11:07:00.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:00 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-10T11:07:00.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:00 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-10T11:07:00.546 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:00 smithi100 ceph-mon[30296]: Manager daemon x is now available 2024-05-10T11:07:00.546 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:00 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:07:00.546 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:00 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-10T11:07:01.296 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:01 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-10T11:07:01.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:01 smithi100 ceph-mon[30296]: [10/May/2024:11:07:00] ENGINE Bus STARTING 2024-05-10T11:07:01.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:01 smithi100 ceph-mon[30296]: [10/May/2024:11:07:00] ENGINE Serving on http://172.21.15.100:8765 2024-05-10T11:07:01.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:01 smithi100 ceph-mon[30296]: [10/May/2024:11:07:00] ENGINE Serving on https://172.21.15.100:7150 2024-05-10T11:07:01.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:01 smithi100 ceph-mon[30296]: [10/May/2024:11:07:00] ENGINE Bus STARTED 2024-05-10T11:07:01.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:01 smithi100 ceph-mon[30296]: mgrmap e11: x(active, since 1.00647s) 2024-05-10T11:07:01.987 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout { 2024-05-10T11:07:01.987 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-05-10T11:07:01.987 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-10T11:07:01.988 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout } 2024-05-10T11:07:01.988 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:55.473+0000 7fa4d155e700 1 Processor -- start 2024-05-10T11:07:01.988 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:55.473+0000 7fa4d155e700 1 -- start start 2024-05-10T11:07:01.988 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:55.473+0000 7fa4d155e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4cc107450 0x7fa4cc107870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:07:01.988 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:55.473+0000 7fa4d155e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa4cc07cdc0 con 0x7fa4cc107450 2024-05-10T11:07:01.988 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:55.473+0000 7fa4caffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4cc107450 0x7fa4cc107870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:07:01.988 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:55.473+0000 7fa4caffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4cc107450 0x7fa4cc107870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33610/0 (socket says 172.21.15.100:33610) 2024-05-10T11:07:01.988 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:55.473+0000 7fa4caffd700 1 -- 172.21.15.100:0/2285489818 learned_addr learned my addr 172.21.15.100:0/2285489818 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:07:01.988 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:55.473+0000 7fa4caffd700 1 -- 172.21.15.100:0/2285489818 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4cc107db0 con 0x7fa4cc107450 2024-05-10T11:07:01.988 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:55.473+0000 7fa4caffd700 1 --2- 172.21.15.100:0/2285489818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4cc107450 0x7fa4cc107870 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc0098a0 tx=0x7fa4bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e9237a9e8c0d89f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:07:01.989 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:55.473+0000 7fa4c9ffb700 1 -- 172.21.15.100:0/2285489818 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4bc004160 con 0x7fa4cc107450 2024-05-10T11:07:01.989 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:55.474+0000 7fa4c9ffb700 1 -- 172.21.15.100:0/2285489818 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa4bc004300 con 0x7fa4cc107450 2024-05-10T11:07:01.989 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:55.474+0000 7fa4c9ffb700 1 -- 172.21.15.100:0/2285489818 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4bc011960 con 0x7fa4cc107450 2024-05-10T11:07:01.989 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:55.474+0000 7fa4d155e700 1 -- 172.21.15.100:0/2285489818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4cc107450 msgr2=0x7fa4cc107870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:07:01.989 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:55.474+0000 7fa4d155e700 1 --2- 172.21.15.100:0/2285489818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4cc107450 0x7fa4cc107870 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc0098a0 tx=0x7fa4bc009bb0 comp rx=0 tx=0).stop 2024-05-10T11:07:01.989 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:55.474+0000 7fa4d155e700 1 -- 172.21.15.100:0/2285489818 shutdown_connections 2024-05-10T11:07:01.989 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:55.474+0000 7fa4d155e700 1 --2- 172.21.15.100:0/2285489818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4cc107450 0x7fa4cc107870 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:07:01.989 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:55.474+0000 7fa4d155e700 1 -- 172.21.15.100:0/2285489818 >> 172.21.15.100:0/2285489818 conn(0x7fa4cc102fd0 msgr2=0x7fa4cc105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:07:01.989 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:55.474+0000 7fa4d155e700 1 -- 172.21.15.100:0/2285489818 shutdown_connections 2024-05-10T11:07:01.990 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:55.474+0000 7fa4d155e700 1 -- 172.21.15.100:0/2285489818 wait complete. 2024-05-10T11:07:01.990 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:55.475+0000 7fa4d155e700 1 Processor -- start 2024-05-10T11:07:01.990 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:55.475+0000 7fa4d155e700 1 -- start start 2024-05-10T11:07:01.990 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:55.475+0000 7fa4d155e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4cc107450 0x7fa4cc19eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:07:01.990 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:55.475+0000 7fa4d155e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa4cc108dc0 con 0x7fa4cc107450 2024-05-10T11:07:01.990 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:55.475+0000 7fa4caffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4cc107450 0x7fa4cc19eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:07:01.990 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:55.475+0000 7fa4caffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4cc107450 0x7fa4cc19eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33612/0 (socket says 172.21.15.100:33612) 2024-05-10T11:07:01.990 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:55.475+0000 7fa4caffd700 1 -- 172.21.15.100:0/549404729 learned_addr learned my addr 172.21.15.100:0/549404729 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:07:01.990 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:55.476+0000 7fa4caffd700 1 -- 172.21.15.100:0/549404729 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4cc19f0c0 con 0x7fa4cc107450 2024-05-10T11:07:01.990 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:55.476+0000 7fa4caffd700 1 --2- 172.21.15.100:0/549404729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4cc107450 0x7fa4cc19eb80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc011670 tx=0x7fa4bc011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:07:01.991 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:55.476+0000 7fa4b3fff700 1 -- 172.21.15.100:0/549404729 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4bc003a60 con 0x7fa4cc107450 2024-05-10T11:07:01.991 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:55.476+0000 7fa4b3fff700 1 -- 172.21.15.100:0/549404729 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa4bc01b8d0 con 0x7fa4cc107450 2024-05-10T11:07:01.991 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:55.476+0000 7fa4b3fff700 1 -- 172.21.15.100:0/549404729 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4bc023790 con 0x7fa4cc107450 2024-05-10T11:07:01.991 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:55.476+0000 7fa4d155e700 1 -- 172.21.15.100:0/549404729 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4cc19f350 con 0x7fa4cc107450 2024-05-10T11:07:01.991 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:55.476+0000 7fa4d155e700 1 -- 172.21.15.100:0/549404729 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4cc1a2040 con 0x7fa4cc107450 2024-05-10T11:07:01.991 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:55.477+0000 7fa4b3fff700 1 -- 172.21.15.100:0/549404729 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7fa4bc01b440 con 0x7fa4cc107450 2024-05-10T11:07:01.991 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:55.477+0000 7fa4b3fff700 1 --2- 172.21.15.100:0/549404729 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7fa4b403e4c0 0x7fa4b4040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:07:01.991 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:55.477+0000 7fa4b3fff700 1 -- 172.21.15.100:0/549404729 --> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa4b4041090 con 0x7fa4b403e4c0 2024-05-10T11:07:01.991 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:55.477+0000 7fa4b3fff700 1 -- 172.21.15.100:0/549404729 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa4bc053f50 con 0x7fa4cc107450 2024-05-10T11:07:01.991 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:55.477+0000 7fa4ca7fc700 1 -- 172.21.15.100:0/549404729 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7fa4b403e4c0 msgr2=0x7fa4b4040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.100:6800/1597433518 2024-05-10T11:07:01.992 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:55.477+0000 7fa4ca7fc700 1 --2- 172.21.15.100:0/549404729 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7fa4b403e4c0 0x7fa4b4040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-10T11:07:01.992 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:55.678+0000 7fa4ca7fc700 1 -- 172.21.15.100:0/549404729 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7fa4b403e4c0 msgr2=0x7fa4b4040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.100:6800/1597433518 2024-05-10T11:07:01.992 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:55.678+0000 7fa4ca7fc700 1 --2- 172.21.15.100:0/549404729 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7fa4b403e4c0 0x7fa4b4040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-10T11:07:01.992 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:56.079+0000 7fa4ca7fc700 1 -- 172.21.15.100:0/549404729 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7fa4b403e4c0 msgr2=0x7fa4b4040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.100:6800/1597433518 2024-05-10T11:07:01.992 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:56.079+0000 7fa4ca7fc700 1 --2- 172.21.15.100:0/549404729 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7fa4b403e4c0 0x7fa4b4040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-10T11:07:01.992 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:56.880+0000 7fa4ca7fc700 1 -- 172.21.15.100:0/549404729 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7fa4b403e4c0 msgr2=0x7fa4b4040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.100:6800/1597433518 2024-05-10T11:07:01.992 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:56.880+0000 7fa4ca7fc700 1 --2- 172.21.15.100:0/549404729 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7fa4b403e4c0 0x7fa4b4040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-10T11:07:01.992 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:58.481+0000 7fa4ca7fc700 1 -- 172.21.15.100:0/549404729 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7fa4b403e4c0 msgr2=0x7fa4b4040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.100:6800/1597433518 2024-05-10T11:07:01.992 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:06:58.481+0000 7fa4ca7fc700 1 --2- 172.21.15.100:0/549404729 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7fa4b403e4c0 0x7fa4b4040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-10T11:07:01.992 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:00.094+0000 7fa4b3fff700 1 -- 172.21.15.100:0/549404729 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mgrmap(e 10) ==== 50002+0+0 (secure 0 0 0) 0x7fa4bc02bcd0 con 0x7fa4cc107450 2024-05-10T11:07:01.992 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:00.094+0000 7fa4b3fff700 1 -- 172.21.15.100:0/549404729 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7fa4b403e4c0 msgr2=0x7fa4b4040980 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T11:07:01.992 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:00.094+0000 7fa4b3fff700 1 --2- 172.21.15.100:0/549404729 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7fa4b403e4c0 0x7fa4b4040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:07:01.992 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:01.095+0000 7fa4b3fff700 1 -- 172.21.15.100:0/549404729 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mgrmap(e 11) ==== 50129+0+0 (secure 0 0 0) 0x7fa4bc0554d0 con 0x7fa4cc107450 2024-05-10T11:07:01.992 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:01.095+0000 7fa4b3fff700 1 --2- 172.21.15.100:0/549404729 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa4b4041f40 0x7fa4b4044330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:07:01.993 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:01.095+0000 7fa4b3fff700 1 -- 172.21.15.100:0/549404729 --> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa4bc008e50 con 0x7fa4b4041f40 2024-05-10T11:07:01.993 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:01.096+0000 7fa4ca7fc700 1 --2- 172.21.15.100:0/549404729 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa4b4041f40 0x7fa4b4044330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:07:01.993 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:01.097+0000 7fa4ca7fc700 1 --2- 172.21.15.100:0/549404729 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa4b4041f40 0x7fa4b4044330 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa4c0003a10 tx=0x7fa4c00092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:07:01.993 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:01.097+0000 7fa4b3fff700 1 -- 172.21.15.100:0/549404729 <== mgr.14152 v2:172.21.15.100:6800/2507287356 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fa4bc008e50 con 0x7fa4b4041f40 2024-05-10T11:07:01.993 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:01.108+0000 7fa4d155e700 1 -- 172.21.15.100:0/549404729 --> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fa4cc064b00 con 0x7fa4b4041f40 2024-05-10T11:07:01.993 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:01.109+0000 7fa4b3fff700 1 -- 172.21.15.100:0/549404729 <== mgr.14152 v2:172.21.15.100:6800/2507287356 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7fa4cc064b00 con 0x7fa4b4041f40 2024-05-10T11:07:01.993 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:01.109+0000 7fa4d155e700 1 -- 172.21.15.100:0/549404729 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa4b4041f40 msgr2=0x7fa4b4044330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:07:01.993 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:01.109+0000 7fa4d155e700 1 --2- 172.21.15.100:0/549404729 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa4b4041f40 0x7fa4b4044330 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa4c0003a10 tx=0x7fa4c00092b0 comp rx=0 tx=0).stop 2024-05-10T11:07:01.993 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:01.109+0000 7fa4d155e700 1 -- 172.21.15.100:0/549404729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4cc107450 msgr2=0x7fa4cc19eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:07:01.993 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:01.109+0000 7fa4d155e700 1 --2- 172.21.15.100:0/549404729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4cc107450 0x7fa4cc19eb80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc011670 tx=0x7fa4bc011a30 comp rx=0 tx=0).stop 2024-05-10T11:07:01.993 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:01.109+0000 7fa4d155e700 1 -- 172.21.15.100:0/549404729 shutdown_connections 2024-05-10T11:07:01.993 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:01.109+0000 7fa4d155e700 1 --2- 172.21.15.100:0/549404729 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa4b4041f40 0x7fa4b4044330 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:07:01.993 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:01.109+0000 7fa4d155e700 1 --2- 172.21.15.100:0/549404729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4cc107450 0x7fa4cc19eb80 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:07:01.994 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:01.109+0000 7fa4d155e700 1 --2- 172.21.15.100:0/549404729 >> [v2:172.21.15.100:6800/1597433518,v1:172.21.15.100:6801/1597433518] conn(0x7fa4b403e4c0 0x7fa4b4040980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:07:01.994 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:01.109+0000 7fa4d155e700 1 -- 172.21.15.100:0/549404729 >> 172.21.15.100:0/549404729 conn(0x7fa4cc102fd0 msgr2=0x7fa4cc195810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:07:01.994 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:01.109+0000 7fa4d155e700 1 -- 172.21.15.100:0/549404729 shutdown_connections 2024-05-10T11:07:01.994 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:01.109+0000 7fa4d155e700 1 -- 172.21.15.100:0/549404729 wait complete. 2024-05-10T11:07:01.994 INFO:teuthology.orchestra.run.smithi100.stdout:mgr epoch 9 is available 2024-05-10T11:07:01.994 INFO:teuthology.orchestra.run.smithi100.stdout:Generating a dashboard self-signed certificate... 2024-05-10T11:07:02.855 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:02 smithi100 ceph-mon[30296]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-10T11:07:02.855 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:02 smithi100 ceph-mon[30296]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-10T11:07:02.856 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:02 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:03.457 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-05-10T11:07:03.457 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:02.429+0000 7f17e741b700 1 Processor -- start 2024-05-10T11:07:03.458 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:02.429+0000 7f17e741b700 1 -- start start 2024-05-10T11:07:03.458 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:02.429+0000 7f17e741b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17e0107430 0x7f17e0107850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:07:03.458 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:02.429+0000 7f17e741b700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f17e007cde0 con 0x7f17e0107430 2024-05-10T11:07:03.458 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:02.429+0000 7f17e51b7700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17e0107430 0x7f17e0107850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:07:03.458 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:02.429+0000 7f17e51b7700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17e0107430 0x7f17e0107850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43264/0 (socket says 172.21.15.100:43264) 2024-05-10T11:07:03.458 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:02.429+0000 7f17e51b7700 1 -- 172.21.15.100:0/212944803 learned_addr learned my addr 172.21.15.100:0/212944803 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:07:03.458 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:02.430+0000 7f17e51b7700 1 -- 172.21.15.100:0/212944803 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17e0107d90 con 0x7f17e0107430 2024-05-10T11:07:03.458 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:02.430+0000 7f17e51b7700 1 --2- 172.21.15.100:0/212944803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17e0107430 0x7f17e0107850 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f17dc0098a0 tx=0x7f17dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caa4076a833904a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:07:03.458 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:02.430+0000 7f17d7fff700 1 -- 172.21.15.100:0/212944803 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f17dc004160 con 0x7f17e0107430 2024-05-10T11:07:03.458 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:02.430+0000 7f17d7fff700 1 -- 172.21.15.100:0/212944803 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f17dc004300 con 0x7f17e0107430 2024-05-10T11:07:03.458 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:02.430+0000 7f17d7fff700 1 -- 172.21.15.100:0/212944803 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f17dc011960 con 0x7f17e0107430 2024-05-10T11:07:03.458 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:02.430+0000 7f17e741b700 1 -- 172.21.15.100:0/212944803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17e0107430 msgr2=0x7f17e0107850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:07:03.458 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:02.430+0000 7f17e741b700 1 --2- 172.21.15.100:0/212944803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17e0107430 0x7f17e0107850 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f17dc0098a0 tx=0x7f17dc009bb0 comp rx=0 tx=0).stop 2024-05-10T11:07:03.459 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:02.430+0000 7f17e741b700 1 -- 172.21.15.100:0/212944803 shutdown_connections 2024-05-10T11:07:03.459 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:02.430+0000 7f17e741b700 1 --2- 172.21.15.100:0/212944803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17e0107430 0x7f17e0107850 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:07:03.459 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:02.430+0000 7f17e741b700 1 -- 172.21.15.100:0/212944803 >> 172.21.15.100:0/212944803 conn(0x7f17e0102fd0 msgr2=0x7f17e0105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:07:03.459 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:02.431+0000 7f17e741b700 1 -- 172.21.15.100:0/212944803 shutdown_connections 2024-05-10T11:07:03.459 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:02.431+0000 7f17e741b700 1 -- 172.21.15.100:0/212944803 wait complete. 2024-05-10T11:07:03.459 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:02.431+0000 7f17e741b700 1 Processor -- start 2024-05-10T11:07:03.459 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:02.431+0000 7f17e741b700 1 -- start start 2024-05-10T11:07:03.459 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:02.431+0000 7f17e741b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17e0107430 0x7f17e019ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:07:03.459 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:02.431+0000 7f17e741b700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f17e0108da0 con 0x7f17e0107430 2024-05-10T11:07:03.459 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:02.431+0000 7f17e51b7700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17e0107430 0x7f17e019ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:07:03.459 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:02.431+0000 7f17e51b7700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17e0107430 0x7f17e019ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43270/0 (socket says 172.21.15.100:43270) 2024-05-10T11:07:03.459 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:02.431+0000 7f17e51b7700 1 -- 172.21.15.100:0/3608040926 learned_addr learned my addr 172.21.15.100:0/3608040926 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:07:03.459 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:02.432+0000 7f17e51b7700 1 -- 172.21.15.100:0/3608040926 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17e019f0f0 con 0x7f17e0107430 2024-05-10T11:07:03.459 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:02.432+0000 7f17e51b7700 1 --2- 172.21.15.100:0/3608040926 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17e0107430 0x7f17e019ebb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f17dc003e50 tx=0x7f17dc011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:07:03.460 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:02.432+0000 7f17d67fc700 1 -- 172.21.15.100:0/3608040926 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f17dc003a60 con 0x7f17e0107430 2024-05-10T11:07:03.460 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:02.432+0000 7f17d67fc700 1 -- 172.21.15.100:0/3608040926 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f17dc01b8d0 con 0x7f17e0107430 2024-05-10T11:07:03.460 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:02.432+0000 7f17e741b700 1 -- 172.21.15.100:0/3608040926 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17e019f380 con 0x7f17e0107430 2024-05-10T11:07:03.460 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:02.432+0000 7f17d67fc700 1 -- 172.21.15.100:0/3608040926 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f17dc0237d0 con 0x7f17e0107430 2024-05-10T11:07:03.460 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:02.432+0000 7f17e741b700 1 -- 172.21.15.100:0/3608040926 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17e01a2070 con 0x7f17e0107430 2024-05-10T11:07:03.460 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:02.433+0000 7f17d67fc700 1 -- 172.21.15.100:0/3608040926 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 11) ==== 50129+0+0 (secure 0 0 0) 0x7f17dc02c3f0 con 0x7f17e0107430 2024-05-10T11:07:03.460 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:02.433+0000 7f17d67fc700 1 --2- 172.21.15.100:0/3608040926 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f17cc03e360 0x7f17cc040820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:07:03.460 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:02.433+0000 7f17e741b700 1 -- 172.21.15.100:0/3608040926 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17e0064b00 con 0x7f17e0107430 2024-05-10T11:07:03.460 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:02.433+0000 7f17d67fc700 1 -- 172.21.15.100:0/3608040926 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f17dc053fb0 con 0x7f17e0107430 2024-05-10T11:07:03.460 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:02.433+0000 7f17e49b6700 1 --2- 172.21.15.100:0/3608040926 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f17cc03e360 0x7f17cc040820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:07:03.460 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:02.434+0000 7f17e49b6700 1 --2- 172.21.15.100:0/3608040926 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f17cc03e360 0x7f17cc040820 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f17d0006fd0 tx=0x7f17d0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:07:03.460 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:02.438+0000 7f17d67fc700 1 -- 172.21.15.100:0/3608040926 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f17dc01f030 con 0x7f17e0107430 2024-05-10T11:07:03.460 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:02.604+0000 7f17d67fc700 1 -- 172.21.15.100:0/3608040926 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f17dc083d70 con 0x7f17e0107430 2024-05-10T11:07:03.460 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:02.682+0000 7f17e741b700 1 -- 172.21.15.100:0/3608040926 --> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f17e002e7d0 con 0x7f17cc03e360 2024-05-10T11:07:03.461 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:02.841+0000 7f17d67fc700 1 -- 172.21.15.100:0/3608040926 <== mgr.14152 v2:172.21.15.100:6800/2507287356 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f17e002e7d0 con 0x7f17cc03e360 2024-05-10T11:07:03.461 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:02.844+0000 7f17e741b700 1 -- 172.21.15.100:0/3608040926 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f17cc03e360 msgr2=0x7f17cc040820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:07:03.461 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:02.844+0000 7f17e741b700 1 --2- 172.21.15.100:0/3608040926 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f17cc03e360 0x7f17cc040820 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f17d0006fd0 tx=0x7f17d0006e40 comp rx=0 tx=0).stop 2024-05-10T11:07:03.461 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:02.844+0000 7f17e741b700 1 -- 172.21.15.100:0/3608040926 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17e0107430 msgr2=0x7f17e019ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:07:03.461 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:02.844+0000 7f17e741b700 1 --2- 172.21.15.100:0/3608040926 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17e0107430 0x7f17e019ebb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f17dc003e50 tx=0x7f17dc011a30 comp rx=0 tx=0).stop 2024-05-10T11:07:03.461 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:02.844+0000 7f17e741b700 1 -- 172.21.15.100:0/3608040926 shutdown_connections 2024-05-10T11:07:03.461 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:02.844+0000 7f17e741b700 1 --2- 172.21.15.100:0/3608040926 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f17cc03e360 0x7f17cc040820 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:07:03.461 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:02.844+0000 7f17e741b700 1 --2- 172.21.15.100:0/3608040926 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17e0107430 0x7f17e019ebb0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:07:03.461 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:02.844+0000 7f17e741b700 1 -- 172.21.15.100:0/3608040926 >> 172.21.15.100:0/3608040926 conn(0x7f17e0102fd0 msgr2=0x7f17e0195850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:07:03.461 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:02.844+0000 7f17e741b700 1 -- 172.21.15.100:0/3608040926 shutdown_connections 2024-05-10T11:07:03.461 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:02.844+0000 7f17e741b700 1 -- 172.21.15.100:0/3608040926 wait complete. 2024-05-10T11:07:03.461 INFO:teuthology.orchestra.run.smithi100.stdout:Creating initial admin user... 2024-05-10T11:07:03.707 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:03 smithi100 ceph-mon[30296]: mgrmap e12: x(active, since 2s) 2024-05-10T11:07:03.707 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:03 smithi100 ceph-mon[30296]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:07:03.707 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:03 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:03.707 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:03 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:03.707 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:03 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:05.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:05 smithi100 ceph-mon[30296]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:07:05.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:05 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:05.705 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$qcjeQCLI5TnsCjnAjSlr/.GMzLfGKsi.DSTEeg29fom2Ys4uSZ4au", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1715339224, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-05-10T11:07:05.705 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:03.922+0000 7f2b9c658700 1 Processor -- start 2024-05-10T11:07:05.705 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:03.922+0000 7f2b9c658700 1 -- start start 2024-05-10T11:07:05.705 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:03.922+0000 7f2b9c658700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b941095e0 0x7f2b9410b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:07:05.705 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:03.922+0000 7f2b9c658700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2b9407ce40 con 0x7f2b941095e0 2024-05-10T11:07:05.705 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:03.922+0000 7f2b9a3f4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b941095e0 0x7f2b9410b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:07:05.706 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:03.922+0000 7f2b9a3f4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b941095e0 0x7f2b9410b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43282/0 (socket says 172.21.15.100:43282) 2024-05-10T11:07:05.706 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:03.922+0000 7f2b9a3f4700 1 -- 172.21.15.100:0/505904451 learned_addr learned my addr 172.21.15.100:0/505904451 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:07:05.706 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:03.922+0000 7f2b9a3f4700 1 -- 172.21.15.100:0/505904451 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b9410bf10 con 0x7f2b941095e0 2024-05-10T11:07:05.706 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:03.923+0000 7f2b9a3f4700 1 --2- 172.21.15.100:0/505904451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b941095e0 0x7f2b9410b9d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2b900098a0 tx=0x7f2b90009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e40a7321c3f3283b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:07:05.706 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:03.923+0000 7f2b993f2700 1 -- 172.21.15.100:0/505904451 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2b90004160 con 0x7f2b941095e0 2024-05-10T11:07:05.706 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:03.923+0000 7f2b993f2700 1 -- 172.21.15.100:0/505904451 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2b90004300 con 0x7f2b941095e0 2024-05-10T11:07:05.706 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:03.923+0000 7f2b993f2700 1 -- 172.21.15.100:0/505904451 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2b90011960 con 0x7f2b941095e0 2024-05-10T11:07:05.706 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:03.923+0000 7f2b9c658700 1 -- 172.21.15.100:0/505904451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b941095e0 msgr2=0x7f2b9410b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:07:05.706 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:03.923+0000 7f2b9c658700 1 --2- 172.21.15.100:0/505904451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b941095e0 0x7f2b9410b9d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2b900098a0 tx=0x7f2b90009bb0 comp rx=0 tx=0).stop 2024-05-10T11:07:05.706 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:03.924+0000 7f2b9c658700 1 -- 172.21.15.100:0/505904451 shutdown_connections 2024-05-10T11:07:05.706 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:03.924+0000 7f2b9c658700 1 --2- 172.21.15.100:0/505904451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b941095e0 0x7f2b9410b9d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:07:05.706 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:03.924+0000 7f2b9c658700 1 -- 172.21.15.100:0/505904451 >> 172.21.15.100:0/505904451 conn(0x7f2b94102fd0 msgr2=0x7f2b94105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:07:05.706 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:03.924+0000 7f2b9c658700 1 -- 172.21.15.100:0/505904451 shutdown_connections 2024-05-10T11:07:05.707 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:03.924+0000 7f2b9c658700 1 -- 172.21.15.100:0/505904451 wait complete. 2024-05-10T11:07:05.707 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:03.924+0000 7f2b9c658700 1 Processor -- start 2024-05-10T11:07:05.707 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:03.924+0000 7f2b9c658700 1 -- start start 2024-05-10T11:07:05.707 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:03.924+0000 7f2b9c658700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b941095e0 0x7f2b9419c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:07:05.707 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:03.924+0000 7f2b9c658700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2b9410cec0 con 0x7f2b941095e0 2024-05-10T11:07:05.707 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:03.924+0000 7f2b9a3f4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b941095e0 0x7f2b9419c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:07:05.707 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:03.925+0000 7f2b9a3f4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b941095e0 0x7f2b9419c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43292/0 (socket says 172.21.15.100:43292) 2024-05-10T11:07:05.707 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:03.925+0000 7f2b9a3f4700 1 -- 172.21.15.100:0/2499938269 learned_addr learned my addr 172.21.15.100:0/2499938269 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:07:05.707 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:03.925+0000 7f2b9a3f4700 1 -- 172.21.15.100:0/2499938269 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b9419cef0 con 0x7f2b941095e0 2024-05-10T11:07:05.707 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:03.925+0000 7f2b9a3f4700 1 --2- 172.21.15.100:0/2499938269 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b941095e0 0x7f2b9419c9b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2b90003e50 tx=0x7f2b90011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:07:05.707 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:03.925+0000 7f2b8b7fe700 1 -- 172.21.15.100:0/2499938269 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2b90003a60 con 0x7f2b941095e0 2024-05-10T11:07:05.707 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:03.925+0000 7f2b8b7fe700 1 -- 172.21.15.100:0/2499938269 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2b9001b8d0 con 0x7f2b941095e0 2024-05-10T11:07:05.707 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:03.925+0000 7f2b9c658700 1 -- 172.21.15.100:0/2499938269 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b9419d180 con 0x7f2b941095e0 2024-05-10T11:07:05.708 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:03.925+0000 7f2b8b7fe700 1 -- 172.21.15.100:0/2499938269 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2b900237d0 con 0x7f2b941095e0 2024-05-10T11:07:05.708 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:03.925+0000 7f2b9c658700 1 -- 172.21.15.100:0/2499938269 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b9419fe70 con 0x7f2b941095e0 2024-05-10T11:07:05.708 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:03.926+0000 7f2b8b7fe700 1 -- 172.21.15.100:0/2499938269 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f2b9001b440 con 0x7f2b941095e0 2024-05-10T11:07:05.708 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:03.926+0000 7f2b9c658700 1 -- 172.21.15.100:0/2499938269 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b94064b00 con 0x7f2b941095e0 2024-05-10T11:07:05.708 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:03.926+0000 7f2b8b7fe700 1 --2- 172.21.15.100:0/2499938269 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2b8003e4d0 0x7f2b80040990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:07:05.708 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:03.926+0000 7f2b8b7fe700 1 -- 172.21.15.100:0/2499938269 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f2b90054210 con 0x7f2b941095e0 2024-05-10T11:07:05.708 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:03.926+0000 7f2b99bf3700 1 --2- 172.21.15.100:0/2499938269 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2b8003e4d0 0x7f2b80040990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:07:05.708 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:03.927+0000 7f2b99bf3700 1 --2- 172.21.15.100:0/2499938269 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2b8003e4d0 0x7f2b80040990 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2b84006fd0 tx=0x7f2b84006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:07:05.708 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:03.931+0000 7f2b8b7fe700 1 -- 172.21.15.100:0/2499938269 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f2b9001f030 con 0x7f2b941095e0 2024-05-10T11:07:05.708 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:04.160+0000 7f2b9c658700 1 -- 172.21.15.100:0/2499938269 --> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f2b9419d660 con 0x7f2b8003e4d0 2024-05-10T11:07:05.708 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:04.384+0000 7f2b8b7fe700 1 -- 172.21.15.100:0/2499938269 <== mgr.14152 v2:172.21.15.100:6800/2507287356 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f2b9419d660 con 0x7f2b8003e4d0 2024-05-10T11:07:05.708 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:04.386+0000 7f2b9c658700 1 -- 172.21.15.100:0/2499938269 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2b8003e4d0 msgr2=0x7f2b80040990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:07:05.708 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:04.386+0000 7f2b9c658700 1 --2- 172.21.15.100:0/2499938269 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2b8003e4d0 0x7f2b80040990 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2b84006fd0 tx=0x7f2b84006e40 comp rx=0 tx=0).stop 2024-05-10T11:07:05.708 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:04.386+0000 7f2b9c658700 1 -- 172.21.15.100:0/2499938269 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b941095e0 msgr2=0x7f2b9419c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:07:05.709 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:04.386+0000 7f2b9c658700 1 --2- 172.21.15.100:0/2499938269 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b941095e0 0x7f2b9419c9b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2b90003e50 tx=0x7f2b90011a30 comp rx=0 tx=0).stop 2024-05-10T11:07:05.709 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:04.386+0000 7f2b9c658700 1 -- 172.21.15.100:0/2499938269 shutdown_connections 2024-05-10T11:07:05.709 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:04.386+0000 7f2b9c658700 1 --2- 172.21.15.100:0/2499938269 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2b8003e4d0 0x7f2b80040990 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:07:05.709 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:04.386+0000 7f2b9c658700 1 --2- 172.21.15.100:0/2499938269 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b941095e0 0x7f2b9419c9b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:07:05.709 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:04.386+0000 7f2b9c658700 1 -- 172.21.15.100:0/2499938269 >> 172.21.15.100:0/2499938269 conn(0x7f2b94102fd0 msgr2=0x7f2b94103ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:07:05.709 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:04.386+0000 7f2b9c658700 1 -- 172.21.15.100:0/2499938269 shutdown_connections 2024-05-10T11:07:05.709 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:04.386+0000 7f2b9c658700 1 -- 172.21.15.100:0/2499938269 wait complete. 2024-05-10T11:07:05.709 INFO:teuthology.orchestra.run.smithi100.stdout:Fetching dashboard port number... 2024-05-10T11:07:06.995 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 8443 2024-05-10T11:07:06.995 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:06.183+0000 7f7a73c67700 1 Processor -- start 2024-05-10T11:07:06.995 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:06.183+0000 7f7a73c67700 1 -- start start 2024-05-10T11:07:06.995 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:06.183+0000 7f7a73c67700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a6c107a70 0x7f7a6c107e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:07:06.995 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:06.183+0000 7f7a73c67700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7a6c108460 con 0x7f7a6c107a70 2024-05-10T11:07:06.995 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:06.183+0000 7f7a71a03700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a6c107a70 0x7f7a6c107e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:07:06.995 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:06.183+0000 7f7a71a03700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a6c107a70 0x7f7a6c107e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43298/0 (socket says 172.21.15.100:43298) 2024-05-10T11:07:06.995 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:06.183+0000 7f7a71a03700 1 -- 172.21.15.100:0/1332538193 learned_addr learned my addr 172.21.15.100:0/1332538193 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:07:06.995 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:06.184+0000 7f7a71a03700 1 -- 172.21.15.100:0/1332538193 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a6c108c70 con 0x7f7a6c107a70 2024-05-10T11:07:06.995 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:06.184+0000 7f7a71a03700 1 --2- 172.21.15.100:0/1332538193 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a6c107a70 0x7f7a6c107e90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7a680098a0 tx=0x7f7a68009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4ffd5fcdc776b0c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:07:06.995 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:06.184+0000 7f7a70a01700 1 -- 172.21.15.100:0/1332538193 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a68004160 con 0x7f7a6c107a70 2024-05-10T11:07:06.996 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:06.184+0000 7f7a70a01700 1 -- 172.21.15.100:0/1332538193 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7a68004300 con 0x7f7a6c107a70 2024-05-10T11:07:06.996 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:06.184+0000 7f7a70a01700 1 -- 172.21.15.100:0/1332538193 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a68011960 con 0x7f7a6c107a70 2024-05-10T11:07:06.996 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:06.185+0000 7f7a73c67700 1 -- 172.21.15.100:0/1332538193 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a6c107a70 msgr2=0x7f7a6c107e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:07:06.996 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:06.185+0000 7f7a73c67700 1 --2- 172.21.15.100:0/1332538193 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a6c107a70 0x7f7a6c107e90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7a680098a0 tx=0x7f7a68009bb0 comp rx=0 tx=0).stop 2024-05-10T11:07:06.996 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:06.185+0000 7f7a73c67700 1 -- 172.21.15.100:0/1332538193 shutdown_connections 2024-05-10T11:07:06.996 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:06.185+0000 7f7a73c67700 1 --2- 172.21.15.100:0/1332538193 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a6c107a70 0x7f7a6c107e90 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:07:06.996 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:06.185+0000 7f7a73c67700 1 -- 172.21.15.100:0/1332538193 >> 172.21.15.100:0/1332538193 conn(0x7f7a6c102fd0 msgr2=0x7f7a6c105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:07:06.996 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:06.185+0000 7f7a73c67700 1 -- 172.21.15.100:0/1332538193 shutdown_connections 2024-05-10T11:07:06.996 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:06.185+0000 7f7a73c67700 1 -- 172.21.15.100:0/1332538193 wait complete. 2024-05-10T11:07:06.996 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:06.186+0000 7f7a73c67700 1 Processor -- start 2024-05-10T11:07:06.996 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:06.186+0000 7f7a73c67700 1 -- start start 2024-05-10T11:07:06.996 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:06.186+0000 7f7a73c67700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a6c107a70 0x7f7a6c07ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:07:06.996 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:06.186+0000 7f7a73c67700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7a6c10bea0 con 0x7f7a6c107a70 2024-05-10T11:07:06.996 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:06.186+0000 7f7a71a03700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a6c107a70 0x7f7a6c07ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:07:06.997 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:06.186+0000 7f7a71a03700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a6c107a70 0x7f7a6c07ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43300/0 (socket says 172.21.15.100:43300) 2024-05-10T11:07:06.997 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:06.186+0000 7f7a71a03700 1 -- 172.21.15.100:0/1643425211 learned_addr learned my addr 172.21.15.100:0/1643425211 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:07:06.997 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:06.186+0000 7f7a71a03700 1 -- 172.21.15.100:0/1643425211 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a6c19f6a0 con 0x7f7a6c107a70 2024-05-10T11:07:06.997 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:06.186+0000 7f7a71a03700 1 --2- 172.21.15.100:0/1643425211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a6c107a70 0x7f7a6c07ca40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7a68011670 tx=0x7f7a68011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:07:06.997 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:06.186+0000 7f7a62ffd700 1 -- 172.21.15.100:0/1643425211 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a68003a60 con 0x7f7a6c107a70 2024-05-10T11:07:06.997 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:06.186+0000 7f7a62ffd700 1 -- 172.21.15.100:0/1643425211 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7a6801b8d0 con 0x7f7a6c107a70 2024-05-10T11:07:06.997 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:06.186+0000 7f7a73c67700 1 -- 172.21.15.100:0/1643425211 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a6c19f870 con 0x7f7a6c107a70 2024-05-10T11:07:06.997 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:06.186+0000 7f7a62ffd700 1 -- 172.21.15.100:0/1643425211 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a68023790 con 0x7f7a6c107a70 2024-05-10T11:07:06.997 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:06.187+0000 7f7a73c67700 1 -- 172.21.15.100:0/1643425211 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a6c19fc90 con 0x7f7a6c107a70 2024-05-10T11:07:06.997 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:06.188+0000 7f7a62ffd700 1 -- 172.21.15.100:0/1643425211 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f7a68023a90 con 0x7f7a6c107a70 2024-05-10T11:07:06.997 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:06.188+0000 7f7a73c67700 1 -- 172.21.15.100:0/1643425211 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a6c10c020 con 0x7f7a6c107a70 2024-05-10T11:07:06.997 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:06.188+0000 7f7a62ffd700 1 --2- 172.21.15.100:0/1643425211 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f7a5803e4c0 0x7f7a58040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:07:06.997 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:06.188+0000 7f7a62ffd700 1 -- 172.21.15.100:0/1643425211 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f7a680541e0 con 0x7f7a6c107a70 2024-05-10T11:07:06.997 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:06.188+0000 7f7a71202700 1 --2- 172.21.15.100:0/1643425211 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f7a5803e4c0 0x7f7a58040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:07:06.998 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:06.188+0000 7f7a71202700 1 --2- 172.21.15.100:0/1643425211 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f7a5803e4c0 0x7f7a58040980 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c006fd0 tx=0x7f7a5c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:07:06.998 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:06.193+0000 7f7a62ffd700 1 -- 172.21.15.100:0/1643425211 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f7a6801f030 con 0x7f7a6c107a70 2024-05-10T11:07:06.998 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:06.385+0000 7f7a62ffd700 1 -- 172.21.15.100:0/1643425211 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f7a6802bcd0 con 0x7f7a6c107a70 2024-05-10T11:07:06.998 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:06.397+0000 7f7a73c67700 1 -- 172.21.15.100:0/1643425211 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f7a6c064b00 con 0x7f7a6c107a70 2024-05-10T11:07:06.998 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:06.398+0000 7f7a62ffd700 1 -- 172.21.15.100:0/1643425211 <== mon.0 v2:172.21.15.100:3300/0 8 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f7a68019df0 con 0x7f7a6c107a70 2024-05-10T11:07:06.998 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:06.400+0000 7f7a73c67700 1 -- 172.21.15.100:0/1643425211 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f7a5803e4c0 msgr2=0x7f7a58040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:07:06.998 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:06.400+0000 7f7a73c67700 1 --2- 172.21.15.100:0/1643425211 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f7a5803e4c0 0x7f7a58040980 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c006fd0 tx=0x7f7a5c006e40 comp rx=0 tx=0).stop 2024-05-10T11:07:06.998 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:06.400+0000 7f7a73c67700 1 -- 172.21.15.100:0/1643425211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a6c107a70 msgr2=0x7f7a6c07ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:07:06.998 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:06.400+0000 7f7a73c67700 1 --2- 172.21.15.100:0/1643425211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a6c107a70 0x7f7a6c07ca40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7a68011670 tx=0x7f7a68011a30 comp rx=0 tx=0).stop 2024-05-10T11:07:06.998 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:06.400+0000 7f7a73c67700 1 -- 172.21.15.100:0/1643425211 shutdown_connections 2024-05-10T11:07:06.998 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:06.400+0000 7f7a73c67700 1 --2- 172.21.15.100:0/1643425211 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f7a5803e4c0 0x7f7a58040980 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:07:06.998 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:06.400+0000 7f7a73c67700 1 --2- 172.21.15.100:0/1643425211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a6c107a70 0x7f7a6c07ca40 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:07:06.998 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:06.400+0000 7f7a73c67700 1 -- 172.21.15.100:0/1643425211 >> 172.21.15.100:0/1643425211 conn(0x7f7a6c102fd0 msgr2=0x7f7a6c103ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:07:06.998 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:06.400+0000 7f7a73c67700 1 -- 172.21.15.100:0/1643425211 shutdown_connections 2024-05-10T11:07:06.999 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:06.400+0000 7f7a73c67700 1 -- 172.21.15.100:0/1643425211 wait complete. 2024-05-10T11:07:07.000 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-10T11:07:07.000 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout disabled 2024-05-10T11:07:07.008 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-10T11:07:07.008 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout inactive 2024-05-10T11:07:07.008 INFO:teuthology.orchestra.run.smithi100.stdout:firewalld.service is not enabled 2024-05-10T11:07:07.008 INFO:teuthology.orchestra.run.smithi100.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-05-10T11:07:07.010 INFO:teuthology.orchestra.run.smithi100.stdout:Ceph Dashboard is now available at: 2024-05-10T11:07:07.010 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:07:07.010 INFO:teuthology.orchestra.run.smithi100.stdout: URL: https://smithi100.front.sepia.ceph.com:8443/ 2024-05-10T11:07:07.010 INFO:teuthology.orchestra.run.smithi100.stdout: User: admin 2024-05-10T11:07:07.010 INFO:teuthology.orchestra.run.smithi100.stdout: Password: j91q3mdbiv 2024-05-10T11:07:07.011 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:07:07.011 INFO:teuthology.orchestra.run.smithi100.stdout:Saving cluster configuration to /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/config directory 2024-05-10T11:07:07.011 INFO:teuthology.orchestra.run.smithi100.stdout:Enabling autotune for osd_memory_target 2024-05-10T11:07:07.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:07 smithi100 ceph-mon[30296]: mgrmap e13: x(active, since 6s) 2024-05-10T11:07:07.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:07 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/1643425211' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-05-10T11:07:08.324 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:07.434+0000 7f2268dda700 1 Processor -- start 2024-05-10T11:07:08.324 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:07.434+0000 7f2268dda700 1 -- start start 2024-05-10T11:07:08.324 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:07.434+0000 7f2268dda700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22641095e0 0x7f226410b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:07:08.324 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:07.434+0000 7f2268dda700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f226407ce40 con 0x7f22641095e0 2024-05-10T11:07:08.324 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:07.435+0000 7f226259c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22641095e0 0x7f226410b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:07:08.324 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:07.435+0000 7f226259c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22641095e0 0x7f226410b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51366/0 (socket says 172.21.15.100:51366) 2024-05-10T11:07:08.325 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:07.435+0000 7f226259c700 1 -- 172.21.15.100:0/2829454904 learned_addr learned my addr 172.21.15.100:0/2829454904 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:07:08.325 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:07.435+0000 7f226259c700 1 -- 172.21.15.100:0/2829454904 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f226410bf10 con 0x7f22641095e0 2024-05-10T11:07:08.325 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:07.435+0000 7f226259c700 1 --2- 172.21.15.100:0/2829454904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22641095e0 0x7f226410b9d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f22580098a0 tx=0x7f2258009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=865cad9d43a3dbbf server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:07:08.325 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:07.435+0000 7f226159a700 1 -- 172.21.15.100:0/2829454904 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2258004160 con 0x7f22641095e0 2024-05-10T11:07:08.325 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:07.435+0000 7f226159a700 1 -- 172.21.15.100:0/2829454904 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2258004300 con 0x7f22641095e0 2024-05-10T11:07:08.325 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:07.435+0000 7f226159a700 1 -- 172.21.15.100:0/2829454904 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2258011960 con 0x7f22641095e0 2024-05-10T11:07:08.325 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:07.436+0000 7f2268dda700 1 -- 172.21.15.100:0/2829454904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22641095e0 msgr2=0x7f226410b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:07:08.325 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:07.436+0000 7f2268dda700 1 --2- 172.21.15.100:0/2829454904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22641095e0 0x7f226410b9d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f22580098a0 tx=0x7f2258009bb0 comp rx=0 tx=0).stop 2024-05-10T11:07:08.325 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:07.436+0000 7f2268dda700 1 -- 172.21.15.100:0/2829454904 shutdown_connections 2024-05-10T11:07:08.325 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:07.436+0000 7f2268dda700 1 --2- 172.21.15.100:0/2829454904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22641095e0 0x7f226410b9d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:07:08.325 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:07.436+0000 7f2268dda700 1 -- 172.21.15.100:0/2829454904 >> 172.21.15.100:0/2829454904 conn(0x7f2264102fd0 msgr2=0x7f2264105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:07:08.325 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:07.436+0000 7f2268dda700 1 -- 172.21.15.100:0/2829454904 shutdown_connections 2024-05-10T11:07:08.325 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:07.436+0000 7f2268dda700 1 -- 172.21.15.100:0/2829454904 wait complete. 2024-05-10T11:07:08.326 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:07.437+0000 7f2268dda700 1 Processor -- start 2024-05-10T11:07:08.326 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:07.437+0000 7f2268dda700 1 -- start start 2024-05-10T11:07:08.326 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:07.437+0000 7f2268dda700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22641095e0 0x7f226419c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:07:08.326 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:07.437+0000 7f2268dda700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f226410cec0 con 0x7f22641095e0 2024-05-10T11:07:08.326 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:07.437+0000 7f226259c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22641095e0 0x7f226419c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:07:08.326 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:07.437+0000 7f226259c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22641095e0 0x7f226419c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51370/0 (socket says 172.21.15.100:51370) 2024-05-10T11:07:08.326 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:07.437+0000 7f226259c700 1 -- 172.21.15.100:0/1840444620 learned_addr learned my addr 172.21.15.100:0/1840444620 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:07:08.326 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:07.437+0000 7f226259c700 1 -- 172.21.15.100:0/1840444620 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f226419cef0 con 0x7f22641095e0 2024-05-10T11:07:08.326 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:07.437+0000 7f226259c700 1 --2- 172.21.15.100:0/1840444620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22641095e0 0x7f226419c9b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2258011670 tx=0x7f2258011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:07:08.326 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:07.438+0000 7f22537fe700 1 -- 172.21.15.100:0/1840444620 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2258003a60 con 0x7f22641095e0 2024-05-10T11:07:08.326 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:07.438+0000 7f22537fe700 1 -- 172.21.15.100:0/1840444620 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f225801b8d0 con 0x7f22641095e0 2024-05-10T11:07:08.326 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:07.438+0000 7f22537fe700 1 -- 172.21.15.100:0/1840444620 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2258023790 con 0x7f22641095e0 2024-05-10T11:07:08.326 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:07.438+0000 7f2268dda700 1 -- 172.21.15.100:0/1840444620 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f226419d180 con 0x7f22641095e0 2024-05-10T11:07:08.326 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:07.438+0000 7f2268dda700 1 -- 172.21.15.100:0/1840444620 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f226419fe70 con 0x7f22641095e0 2024-05-10T11:07:08.327 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:07.439+0000 7f22537fe700 1 -- 172.21.15.100:0/1840444620 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f225801b440 con 0x7f22641095e0 2024-05-10T11:07:08.327 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:07.439+0000 7f2268dda700 1 -- 172.21.15.100:0/1840444620 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2264064b00 con 0x7f22641095e0 2024-05-10T11:07:08.327 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:07.439+0000 7f22537fe700 1 --2- 172.21.15.100:0/1840444620 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f224c03e560 0x7f224c040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:07:08.327 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:07.439+0000 7f22537fe700 1 -- 172.21.15.100:0/1840444620 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f2258054180 con 0x7f22641095e0 2024-05-10T11:07:08.327 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:07.439+0000 7f2261d9b700 1 --2- 172.21.15.100:0/1840444620 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f224c03e560 0x7f224c040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:07:08.327 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:07.439+0000 7f2261d9b700 1 --2- 172.21.15.100:0/1840444620 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f224c03e560 0x7f224c040a20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2254006fd0 tx=0x7f2254006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:07:08.327 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:07.448+0000 7f22537fe700 1 -- 172.21.15.100:0/1840444620 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f225801f030 con 0x7f22641095e0 2024-05-10T11:07:08.327 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:07.654+0000 7f2268dda700 1 -- 172.21.15.100:0/1840444620 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f22641a0490 con 0x7f22641095e0 2024-05-10T11:07:08.327 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:07.655+0000 7f22537fe700 1 -- 172.21.15.100:0/1840444620 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f2258016260 con 0x7f22641095e0 2024-05-10T11:07:08.327 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:07.657+0000 7f2268dda700 1 -- 172.21.15.100:0/1840444620 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f224c03e560 msgr2=0x7f224c040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:07:08.327 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:07.657+0000 7f2268dda700 1 --2- 172.21.15.100:0/1840444620 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f224c03e560 0x7f224c040a20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2254006fd0 tx=0x7f2254006e40 comp rx=0 tx=0).stop 2024-05-10T11:07:08.327 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:07.657+0000 7f2268dda700 1 -- 172.21.15.100:0/1840444620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22641095e0 msgr2=0x7f226419c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:07:08.327 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:07.657+0000 7f2268dda700 1 --2- 172.21.15.100:0/1840444620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22641095e0 0x7f226419c9b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2258011670 tx=0x7f2258011a30 comp rx=0 tx=0).stop 2024-05-10T11:07:08.327 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:07.657+0000 7f2268dda700 1 -- 172.21.15.100:0/1840444620 shutdown_connections 2024-05-10T11:07:08.327 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:07.657+0000 7f2268dda700 1 --2- 172.21.15.100:0/1840444620 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f224c03e560 0x7f224c040a20 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:07:08.328 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:07.657+0000 7f2268dda700 1 --2- 172.21.15.100:0/1840444620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22641095e0 0x7f226419c9b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:07:08.328 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:07.657+0000 7f2268dda700 1 -- 172.21.15.100:0/1840444620 >> 172.21.15.100:0/1840444620 conn(0x7f2264102fd0 msgr2=0x7f2264103ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:07:08.328 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:07.657+0000 7f2268dda700 1 -- 172.21.15.100:0/1840444620 shutdown_connections 2024-05-10T11:07:08.328 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:07.657+0000 7f2268dda700 1 -- 172.21.15.100:0/1840444620 wait complete. 2024-05-10T11:07:09.657 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:08.748+0000 7f8c6c61b700 1 Processor -- start 2024-05-10T11:07:09.657 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:08.748+0000 7f8c6c61b700 1 -- start start 2024-05-10T11:07:09.657 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:08.748+0000 7f8c6c61b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c6407ee30 0x7f8c6407d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:07:09.657 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:08.748+0000 7f8c6c61b700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8c6407d860 con 0x7f8c6407ee30 2024-05-10T11:07:09.657 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:08.749+0000 7f8c6a3b7700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c6407ee30 0x7f8c6407d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:07:09.658 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:08.749+0000 7f8c6a3b7700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c6407ee30 0x7f8c6407d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51380/0 (socket says 172.21.15.100:51380) 2024-05-10T11:07:09.658 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:08.749+0000 7f8c6a3b7700 1 -- 172.21.15.100:0/1615270531 learned_addr learned my addr 172.21.15.100:0/1615270531 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:07:09.658 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:08.749+0000 7f8c6a3b7700 1 -- 172.21.15.100:0/1615270531 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c6407d9e0 con 0x7f8c6407ee30 2024-05-10T11:07:09.658 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:08.749+0000 7f8c6a3b7700 1 --2- 172.21.15.100:0/1615270531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c6407ee30 0x7f8c6407d290 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8c600098a0 tx=0x7f8c60009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66ff1a8bd054c0b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:07:09.658 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:08.749+0000 7f8c693b5700 1 -- 172.21.15.100:0/1615270531 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c60004160 con 0x7f8c6407ee30 2024-05-10T11:07:09.658 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:08.749+0000 7f8c693b5700 1 -- 172.21.15.100:0/1615270531 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8c60004300 con 0x7f8c6407ee30 2024-05-10T11:07:09.658 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:08.749+0000 7f8c693b5700 1 -- 172.21.15.100:0/1615270531 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c60011960 con 0x7f8c6407ee30 2024-05-10T11:07:09.658 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:08.750+0000 7f8c6c61b700 1 -- 172.21.15.100:0/1615270531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c6407ee30 msgr2=0x7f8c6407d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:07:09.658 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:08.750+0000 7f8c6c61b700 1 --2- 172.21.15.100:0/1615270531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c6407ee30 0x7f8c6407d290 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8c600098a0 tx=0x7f8c60009bb0 comp rx=0 tx=0).stop 2024-05-10T11:07:09.658 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:08.750+0000 7f8c6c61b700 1 -- 172.21.15.100:0/1615270531 shutdown_connections 2024-05-10T11:07:09.658 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:08.750+0000 7f8c6c61b700 1 --2- 172.21.15.100:0/1615270531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c6407ee30 0x7f8c6407d290 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:07:09.658 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:08.750+0000 7f8c6c61b700 1 -- 172.21.15.100:0/1615270531 >> 172.21.15.100:0/1615270531 conn(0x7f8c64104130 msgr2=0x7f8c64106550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:07:09.658 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:08.750+0000 7f8c6c61b700 1 -- 172.21.15.100:0/1615270531 shutdown_connections 2024-05-10T11:07:09.659 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:08.750+0000 7f8c6c61b700 1 -- 172.21.15.100:0/1615270531 wait complete. 2024-05-10T11:07:09.659 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:08.750+0000 7f8c6c61b700 1 Processor -- start 2024-05-10T11:07:09.659 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:08.750+0000 7f8c6c61b700 1 -- start start 2024-05-10T11:07:09.659 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:08.751+0000 7f8c6c61b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c6407ee30 0x7f8c6419ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:07:09.659 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:08.751+0000 7f8c6c61b700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8c6410f110 con 0x7f8c6407ee30 2024-05-10T11:07:09.659 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:08.751+0000 7f8c6a3b7700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c6407ee30 0x7f8c6419ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:07:09.659 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:08.751+0000 7f8c6a3b7700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c6407ee30 0x7f8c6419ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51390/0 (socket says 172.21.15.100:51390) 2024-05-10T11:07:09.659 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:08.751+0000 7f8c6a3b7700 1 -- 172.21.15.100:0/3070736050 learned_addr learned my addr 172.21.15.100:0/3070736050 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:07:09.659 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:08.751+0000 7f8c6a3b7700 1 -- 172.21.15.100:0/3070736050 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c6419f100 con 0x7f8c6407ee30 2024-05-10T11:07:09.659 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:08.751+0000 7f8c6a3b7700 1 --2- 172.21.15.100:0/3070736050 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c6407ee30 0x7f8c6419ebc0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8c60011670 tx=0x7f8c60011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:07:09.659 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:08.751+0000 7f8c5b7fe700 1 -- 172.21.15.100:0/3070736050 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c60003a60 con 0x7f8c6407ee30 2024-05-10T11:07:09.659 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:08.751+0000 7f8c5b7fe700 1 -- 172.21.15.100:0/3070736050 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8c6001b8d0 con 0x7f8c6407ee30 2024-05-10T11:07:09.659 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:08.751+0000 7f8c5b7fe700 1 -- 172.21.15.100:0/3070736050 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c60023750 con 0x7f8c6407ee30 2024-05-10T11:07:09.659 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:08.751+0000 7f8c6c61b700 1 -- 172.21.15.100:0/3070736050 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c6419f390 con 0x7f8c6407ee30 2024-05-10T11:07:09.659 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:08.751+0000 7f8c6c61b700 1 -- 172.21.15.100:0/3070736050 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c641a2080 con 0x7f8c6407ee30 2024-05-10T11:07:09.660 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:08.752+0000 7f8c5b7fe700 1 -- 172.21.15.100:0/3070736050 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f8c6001b440 con 0x7f8c6407ee30 2024-05-10T11:07:09.660 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:08.752+0000 7f8c6c61b700 1 -- 172.21.15.100:0/3070736050 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c64064b00 con 0x7f8c6407ee30 2024-05-10T11:07:09.662 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:08.752+0000 7f8c5b7fe700 1 --2- 172.21.15.100:0/3070736050 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8c5003e570 0x7f8c50040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:07:09.662 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:08.752+0000 7f8c5b7fe700 1 -- 172.21.15.100:0/3070736050 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8c60054170 con 0x7f8c6407ee30 2024-05-10T11:07:09.662 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:08.753+0000 7f8c69bb6700 1 --2- 172.21.15.100:0/3070736050 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8c5003e570 0x7f8c50040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:07:09.662 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:08.753+0000 7f8c69bb6700 1 --2- 172.21.15.100:0/3070736050 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8c5003e570 0x7f8c50040a30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8c54006fd0 tx=0x7f8c54006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:07:09.662 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:08.757+0000 7f8c5b7fe700 1 -- 172.21.15.100:0/3070736050 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f8c6001f030 con 0x7f8c6407ee30 2024-05-10T11:07:09.662 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:09.077+0000 7f8c6c61b700 1 -- 172.21.15.100:0/3070736050 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f8c641a2630 con 0x7f8c6407ee30 2024-05-10T11:07:09.662 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:09.080+0000 7f8c5b7fe700 1 -- 172.21.15.100:0/3070736050 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7f8c600187d0 con 0x7f8c6407ee30 2024-05-10T11:07:09.662 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:09.082+0000 7f8c6c61b700 1 -- 172.21.15.100:0/3070736050 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8c5003e570 msgr2=0x7f8c50040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:07:09.662 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:09.082+0000 7f8c6c61b700 1 --2- 172.21.15.100:0/3070736050 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8c5003e570 0x7f8c50040a30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8c54006fd0 tx=0x7f8c54006e40 comp rx=0 tx=0).stop 2024-05-10T11:07:09.662 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:09.082+0000 7f8c6c61b700 1 -- 172.21.15.100:0/3070736050 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c6407ee30 msgr2=0x7f8c6419ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:07:09.662 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:09.082+0000 7f8c6c61b700 1 --2- 172.21.15.100:0/3070736050 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c6407ee30 0x7f8c6419ebc0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8c60011670 tx=0x7f8c60011a30 comp rx=0 tx=0).stop 2024-05-10T11:07:09.662 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:09.082+0000 7f8c6c61b700 1 -- 172.21.15.100:0/3070736050 shutdown_connections 2024-05-10T11:07:09.663 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:09.082+0000 7f8c6c61b700 1 --2- 172.21.15.100:0/3070736050 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8c5003e570 0x7f8c50040a30 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:07:09.663 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:09.082+0000 7f8c6c61b700 1 --2- 172.21.15.100:0/3070736050 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c6407ee30 0x7f8c6419ebc0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:07:09.663 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:09.082+0000 7f8c6c61b700 1 -- 172.21.15.100:0/3070736050 >> 172.21.15.100:0/3070736050 conn(0x7f8c64104130 msgr2=0x7f8c64104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:07:09.663 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:09.082+0000 7f8c6c61b700 1 -- 172.21.15.100:0/3070736050 shutdown_connections 2024-05-10T11:07:09.663 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-05-10T11:07:09.082+0000 7f8c6c61b700 1 -- 172.21.15.100:0/3070736050 wait complete. 2024-05-10T11:07:09.663 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-05-10T11:07:09.663 INFO:teuthology.orchestra.run.smithi100.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-05-10T11:07:09.663 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:07:09.663 INFO:teuthology.orchestra.run.smithi100.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-05-10T11:07:09.664 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:07:09.664 INFO:teuthology.orchestra.run.smithi100.stdout:Or, if you are only running a single cluster on this host: 2024-05-10T11:07:09.664 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:07:09.664 INFO:teuthology.orchestra.run.smithi100.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-05-10T11:07:09.664 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:07:09.664 INFO:teuthology.orchestra.run.smithi100.stdout:Please consider enabling telemetry to help improve Ceph: 2024-05-10T11:07:09.664 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:07:09.664 INFO:teuthology.orchestra.run.smithi100.stdout: ceph telemetry on 2024-05-10T11:07:09.664 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:07:09.664 INFO:teuthology.orchestra.run.smithi100.stdout:For more information see: 2024-05-10T11:07:09.664 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:07:09.664 INFO:teuthology.orchestra.run.smithi100.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-05-10T11:07:09.664 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:07:09.664 INFO:teuthology.orchestra.run.smithi100.stdout:Bootstrap complete. 2024-05-10T11:07:09.696 INFO:tasks.cephadm:Fetching config... 2024-05-10T11:07:09.697 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-05-10T11:07:09.697 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-05-10T11:07:09.712 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-05-10T11:07:09.713 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-05-10T11:07:09.713 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-05-10T11:07:09.768 INFO:tasks.cephadm:Fetching mon keyring... 2024-05-10T11:07:09.768 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-05-10T11:07:09.768 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/keyring of=/dev/stdout 2024-05-10T11:07:09.832 INFO:tasks.cephadm:Fetching pub ssh key... 2024-05-10T11:07:09.833 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-05-10T11:07:09.833 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-05-10T11:07:09.888 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-05-10T11:07:09.888 DEBUG:teuthology.orchestra.run.smithi100:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHI7fh65mIvcXCL2la/FdpvKciZgCQgkPBa8Tmz8PvjNhZZjI7QGS9cPd1X7hpcgX60P14LEvPD5yCIPKcSo5d5Ne+aBAWkpSlQz04HmwR+GLOFhXzdta8QSA/aogcRrxzIiI3ZI+tyknmb1pJcZ9v1Emdy6Gtz3l429Gq9XDIJJUSHh6TBHlgu607bwETANsPZgmw18cwD8i0IvnRI4oFbssqjoqEqgn+Pt0NcZnKyJ6DSK62t0HsPU7amlpsot6+A+JCg/bNvBB1I5WlEVVpzY/eTJTjT+HhpiN3C1328xJNU+msWK+jCEWswXJy3n19gMKQOs2tMj8TmaMeGXPp4oJRjyOt7Aw8GlFs9pss0xd8djsh4cybbsw6Ld3egej9qYww412XIAOpM1d7euuaC8X64pQPI4oURZTa4L9wm+31CKLKoaamMgwzYxVGF+xwvvvuIa//3lc9VrKHzo6rDyTnoLbTDddEmR5eQsxzkxkfBhLD0cbq00L5+ahiaTE= ceph-20a542cc-0ebd-11ef-bc98-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-10T11:07:09.966 INFO:teuthology.orchestra.run.smithi100.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHI7fh65mIvcXCL2la/FdpvKciZgCQgkPBa8Tmz8PvjNhZZjI7QGS9cPd1X7hpcgX60P14LEvPD5yCIPKcSo5d5Ne+aBAWkpSlQz04HmwR+GLOFhXzdta8QSA/aogcRrxzIiI3ZI+tyknmb1pJcZ9v1Emdy6Gtz3l429Gq9XDIJJUSHh6TBHlgu607bwETANsPZgmw18cwD8i0IvnRI4oFbssqjoqEqgn+Pt0NcZnKyJ6DSK62t0HsPU7amlpsot6+A+JCg/bNvBB1I5WlEVVpzY/eTJTjT+HhpiN3C1328xJNU+msWK+jCEWswXJy3n19gMKQOs2tMj8TmaMeGXPp4oJRjyOt7Aw8GlFs9pss0xd8djsh4cybbsw6Ld3egej9qYww412XIAOpM1d7euuaC8X64pQPI4oURZTa4L9wm+31CKLKoaamMgwzYxVGF+xwvvvuIa//3lc9VrKHzo6rDyTnoLbTDddEmR5eQsxzkxkfBhLD0cbq00L5+ahiaTE= ceph-20a542cc-0ebd-11ef-bc98-c7b262605968 2024-05-10T11:07:09.977 DEBUG:teuthology.orchestra.run.smithi178:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHI7fh65mIvcXCL2la/FdpvKciZgCQgkPBa8Tmz8PvjNhZZjI7QGS9cPd1X7hpcgX60P14LEvPD5yCIPKcSo5d5Ne+aBAWkpSlQz04HmwR+GLOFhXzdta8QSA/aogcRrxzIiI3ZI+tyknmb1pJcZ9v1Emdy6Gtz3l429Gq9XDIJJUSHh6TBHlgu607bwETANsPZgmw18cwD8i0IvnRI4oFbssqjoqEqgn+Pt0NcZnKyJ6DSK62t0HsPU7amlpsot6+A+JCg/bNvBB1I5WlEVVpzY/eTJTjT+HhpiN3C1328xJNU+msWK+jCEWswXJy3n19gMKQOs2tMj8TmaMeGXPp4oJRjyOt7Aw8GlFs9pss0xd8djsh4cybbsw6Ld3egej9qYww412XIAOpM1d7euuaC8X64pQPI4oURZTa4L9wm+31CKLKoaamMgwzYxVGF+xwvvvuIa//3lc9VrKHzo6rDyTnoLbTDddEmR5eQsxzkxkfBhLD0cbq00L5+ahiaTE= ceph-20a542cc-0ebd-11ef-bc98-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-10T11:07:10.018 INFO:teuthology.orchestra.run.smithi178.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHI7fh65mIvcXCL2la/FdpvKciZgCQgkPBa8Tmz8PvjNhZZjI7QGS9cPd1X7hpcgX60P14LEvPD5yCIPKcSo5d5Ne+aBAWkpSlQz04HmwR+GLOFhXzdta8QSA/aogcRrxzIiI3ZI+tyknmb1pJcZ9v1Emdy6Gtz3l429Gq9XDIJJUSHh6TBHlgu607bwETANsPZgmw18cwD8i0IvnRI4oFbssqjoqEqgn+Pt0NcZnKyJ6DSK62t0HsPU7amlpsot6+A+JCg/bNvBB1I5WlEVVpzY/eTJTjT+HhpiN3C1328xJNU+msWK+jCEWswXJy3n19gMKQOs2tMj8TmaMeGXPp4oJRjyOt7Aw8GlFs9pss0xd8djsh4cybbsw6Ld3egej9qYww412XIAOpM1d7euuaC8X64pQPI4oURZTa4L9wm+31CKLKoaamMgwzYxVGF+xwvvvuIa//3lc9VrKHzo6rDyTnoLbTDddEmR5eQsxzkxkfBhLD0cbq00L5+ahiaTE= ceph-20a542cc-0ebd-11ef-bc98-c7b262605968 2024-05-10T11:07:10.031 DEBUG:teuthology.orchestra.run.smithi188:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHI7fh65mIvcXCL2la/FdpvKciZgCQgkPBa8Tmz8PvjNhZZjI7QGS9cPd1X7hpcgX60P14LEvPD5yCIPKcSo5d5Ne+aBAWkpSlQz04HmwR+GLOFhXzdta8QSA/aogcRrxzIiI3ZI+tyknmb1pJcZ9v1Emdy6Gtz3l429Gq9XDIJJUSHh6TBHlgu607bwETANsPZgmw18cwD8i0IvnRI4oFbssqjoqEqgn+Pt0NcZnKyJ6DSK62t0HsPU7amlpsot6+A+JCg/bNvBB1I5WlEVVpzY/eTJTjT+HhpiN3C1328xJNU+msWK+jCEWswXJy3n19gMKQOs2tMj8TmaMeGXPp4oJRjyOt7Aw8GlFs9pss0xd8djsh4cybbsw6Ld3egej9qYww412XIAOpM1d7euuaC8X64pQPI4oURZTa4L9wm+31CKLKoaamMgwzYxVGF+xwvvvuIa//3lc9VrKHzo6rDyTnoLbTDddEmR5eQsxzkxkfBhLD0cbq00L5+ahiaTE= ceph-20a542cc-0ebd-11ef-bc98-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-10T11:07:10.072 INFO:teuthology.orchestra.run.smithi188.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHI7fh65mIvcXCL2la/FdpvKciZgCQgkPBa8Tmz8PvjNhZZjI7QGS9cPd1X7hpcgX60P14LEvPD5yCIPKcSo5d5Ne+aBAWkpSlQz04HmwR+GLOFhXzdta8QSA/aogcRrxzIiI3ZI+tyknmb1pJcZ9v1Emdy6Gtz3l429Gq9XDIJJUSHh6TBHlgu607bwETANsPZgmw18cwD8i0IvnRI4oFbssqjoqEqgn+Pt0NcZnKyJ6DSK62t0HsPU7amlpsot6+A+JCg/bNvBB1I5WlEVVpzY/eTJTjT+HhpiN3C1328xJNU+msWK+jCEWswXJy3n19gMKQOs2tMj8TmaMeGXPp4oJRjyOt7Aw8GlFs9pss0xd8djsh4cybbsw6Ld3egej9qYww412XIAOpM1d7euuaC8X64pQPI4oURZTa4L9wm+31CKLKoaamMgwzYxVGF+xwvvvuIa//3lc9VrKHzo6rDyTnoLbTDddEmR5eQsxzkxkfBhLD0cbq00L5+ahiaTE= ceph-20a542cc-0ebd-11ef-bc98-c7b262605968 2024-05-10T11:07:10.085 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-05-10T11:07:10.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:10 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/3070736050' entity='client.admin' 2024-05-10T11:07:10.333 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:07:12.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:12.504+0000 7f4bed199700 1 -- 172.21.15.100:0/1878367503 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4be8101e10 msgr2=0x7f4be8102230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:07:12.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:12.504+0000 7f4bed199700 1 --2- 172.21.15.100:0/1878367503 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4be8101e10 0x7f4be8102230 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4bd8009910 tx=0x7f4bd8009c20 comp rx=0 tx=0).stop 2024-05-10T11:07:12.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:12.504+0000 7f4bed199700 1 -- 172.21.15.100:0/1878367503 shutdown_connections 2024-05-10T11:07:12.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:12.504+0000 7f4bed199700 1 --2- 172.21.15.100:0/1878367503 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4be8101e10 0x7f4be8102230 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:07:12.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:12.504+0000 7f4bed199700 1 -- 172.21.15.100:0/1878367503 >> 172.21.15.100:0/1878367503 conn(0x7f4be80fd3b0 msgr2=0x7f4be80ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:07:12.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:12.504+0000 7f4bed199700 1 -- 172.21.15.100:0/1878367503 shutdown_connections 2024-05-10T11:07:12.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:12.504+0000 7f4bed199700 1 -- 172.21.15.100:0/1878367503 wait complete. 2024-05-10T11:07:12.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:12.505+0000 7f4bed199700 1 Processor -- start 2024-05-10T11:07:12.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:12.505+0000 7f4bed199700 1 -- start start 2024-05-10T11:07:12.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:12.505+0000 7f4bed199700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4be8101e10 0x7f4be818ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:07:12.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:12.505+0000 7f4bed199700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4be8107b80 con 0x7f4be8101e10 2024-05-10T11:07:12.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:12.506+0000 7f4be6d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4be8101e10 0x7f4be818ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:07:12.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:12.506+0000 7f4be6d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4be8101e10 0x7f4be818ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51412/0 (socket says 172.21.15.100:51412) 2024-05-10T11:07:12.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:12.506+0000 7f4be6d9d700 1 -- 172.21.15.100:0/194201475 learned_addr learned my addr 172.21.15.100:0/194201475 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:07:12.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:12.506+0000 7f4be6d9d700 1 -- 172.21.15.100:0/194201475 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4be818f1f0 con 0x7f4be8101e10 2024-05-10T11:07:12.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:12.506+0000 7f4be6d9d700 1 --2- 172.21.15.100:0/194201475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4be8101e10 0x7f4be818ecb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4bd8005b40 tx=0x7f4bd8005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:07:12.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:12.506+0000 7f4bcffff700 1 -- 172.21.15.100:0/194201475 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4bd8027070 con 0x7f4be8101e10 2024-05-10T11:07:12.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:12.506+0000 7f4bcffff700 1 -- 172.21.15.100:0/194201475 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4bd800bdf0 con 0x7f4be8101e10 2024-05-10T11:07:12.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:12.506+0000 7f4bed199700 1 -- 172.21.15.100:0/194201475 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4be818d3e0 con 0x7f4be8101e10 2024-05-10T11:07:12.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:12.506+0000 7f4bcffff700 1 -- 172.21.15.100:0/194201475 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4bd8022cb0 con 0x7f4be8101e10 2024-05-10T11:07:12.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:12.506+0000 7f4bed199700 1 -- 172.21.15.100:0/194201475 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4be818d920 con 0x7f4be8101e10 2024-05-10T11:07:12.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:12.507+0000 7f4bcffff700 1 -- 172.21.15.100:0/194201475 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f4bd802a070 con 0x7f4be8101e10 2024-05-10T11:07:12.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:12.507+0000 7f4bed199700 1 -- 172.21.15.100:0/194201475 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4be8108300 con 0x7f4be8101e10 2024-05-10T11:07:12.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:12.507+0000 7f4bcffff700 1 --2- 172.21.15.100:0/194201475 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4bd003e510 0x7f4bd00409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:07:12.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:12.508+0000 7f4bcffff700 1 -- 172.21.15.100:0/194201475 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f4bd801e030 con 0x7f4be8101e10 2024-05-10T11:07:12.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:12.508+0000 7f4be659c700 1 --2- 172.21.15.100:0/194201475 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4bd003e510 0x7f4bd00409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:07:12.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:12.508+0000 7f4be659c700 1 --2- 172.21.15.100:0/194201475 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4bd003e510 0x7f4bd00409d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc006fd0 tx=0x7f4bdc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:07:12.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:12.512+0000 7f4bcffff700 1 -- 172.21.15.100:0/194201475 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f4bd8013070 con 0x7f4be8101e10 2024-05-10T11:07:12.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:12.719+0000 7f4bed199700 1 -- 172.21.15.100:0/194201475 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f4be8064b00 con 0x7f4be8101e10 2024-05-10T11:07:12.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:12.723+0000 7f4bcffff700 1 -- 172.21.15.100:0/194201475 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f4bd8004800 con 0x7f4be8101e10 2024-05-10T11:07:12.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:12.726+0000 7f4bed199700 1 -- 172.21.15.100:0/194201475 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4bd003e510 msgr2=0x7f4bd00409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:07:12.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:12.726+0000 7f4bed199700 1 --2- 172.21.15.100:0/194201475 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4bd003e510 0x7f4bd00409d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc006fd0 tx=0x7f4bdc006e40 comp rx=0 tx=0).stop 2024-05-10T11:07:12.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:12.726+0000 7f4bed199700 1 -- 172.21.15.100:0/194201475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4be8101e10 msgr2=0x7f4be818ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:07:12.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:12.726+0000 7f4bed199700 1 --2- 172.21.15.100:0/194201475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4be8101e10 0x7f4be818ecb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4bd8005b40 tx=0x7f4bd8005810 comp rx=0 tx=0).stop 2024-05-10T11:07:12.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:12.726+0000 7f4bed199700 1 -- 172.21.15.100:0/194201475 shutdown_connections 2024-05-10T11:07:12.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:12.726+0000 7f4bed199700 1 --2- 172.21.15.100:0/194201475 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4bd003e510 0x7f4bd00409d0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:07:12.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:12.726+0000 7f4bed199700 1 --2- 172.21.15.100:0/194201475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4be8101e10 0x7f4be818ecb0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:07:12.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:12.726+0000 7f4bed199700 1 -- 172.21.15.100:0/194201475 >> 172.21.15.100:0/194201475 conn(0x7f4be80fd3b0 msgr2=0x7f4be806b2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:07:12.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:12.726+0000 7f4bed199700 1 -- 172.21.15.100:0/194201475 shutdown_connections 2024-05-10T11:07:12.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:12.726+0000 7f4bed199700 1 -- 172.21.15.100:0/194201475 wait complete. 2024-05-10T11:07:13.036 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:12 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:13.036 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:12 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:13.036 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:12 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi100", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:07:13.036 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:12 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:13.036 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:12 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:07:13.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:12 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:13.363 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-05-10T11:07:13.363 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-05-10T11:07:13.598 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:07:14.029 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:13 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/194201475' entity='client.admin' 2024-05-10T11:07:14.030 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:13 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:07:14.030 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:13 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:15.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:15.127+0000 7f480c961700 1 -- 172.21.15.100:0/23027039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4808101e20 msgr2=0x7f4808102240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:07:15.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:15.127+0000 7f480c961700 1 --2- 172.21.15.100:0/23027039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4808101e20 0x7f4808102240 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f47f8009910 tx=0x7f47f8009c20 comp rx=0 tx=0).stop 2024-05-10T11:07:15.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:15.127+0000 7f480c961700 1 -- 172.21.15.100:0/23027039 shutdown_connections 2024-05-10T11:07:15.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:15.127+0000 7f480c961700 1 --2- 172.21.15.100:0/23027039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4808101e20 0x7f4808102240 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:07:15.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:15.127+0000 7f480c961700 1 -- 172.21.15.100:0/23027039 >> 172.21.15.100:0/23027039 conn(0x7f48080fd3a0 msgr2=0x7f48080ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:07:15.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:15.127+0000 7f480c961700 1 -- 172.21.15.100:0/23027039 shutdown_connections 2024-05-10T11:07:15.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:15.127+0000 7f480c961700 1 -- 172.21.15.100:0/23027039 wait complete. 2024-05-10T11:07:15.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:15.128+0000 7f480c961700 1 Processor -- start 2024-05-10T11:07:15.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:15.128+0000 7f480c961700 1 -- start start 2024-05-10T11:07:15.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:15.128+0000 7f480c961700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4808101e20 0x7f48081973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:07:15.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:15.128+0000 7f480c961700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4808107b90 con 0x7f4808101e20 2024-05-10T11:07:15.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:15.129+0000 7f480659c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4808101e20 0x7f48081973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:07:15.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:15.129+0000 7f480659c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4808101e20 0x7f48081973c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51426/0 (socket says 172.21.15.100:51426) 2024-05-10T11:07:15.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:15.129+0000 7f480659c700 1 -- 172.21.15.100:0/1003277040 learned_addr learned my addr 172.21.15.100:0/1003277040 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:07:15.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:15.129+0000 7f480659c700 1 -- 172.21.15.100:0/1003277040 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4808197900 con 0x7f4808101e20 2024-05-10T11:07:15.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:15.129+0000 7f480659c700 1 --2- 172.21.15.100:0/1003277040 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4808101e20 0x7f48081973c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f47f8005b40 tx=0x7f47f8005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:07:15.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:15.129+0000 7f47f77fe700 1 -- 172.21.15.100:0/1003277040 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f47f8027070 con 0x7f4808101e20 2024-05-10T11:07:15.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:15.129+0000 7f47f77fe700 1 -- 172.21.15.100:0/1003277040 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f47f800bdf0 con 0x7f4808101e20 2024-05-10T11:07:15.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:15.129+0000 7f480c961700 1 -- 172.21.15.100:0/1003277040 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4808197b90 con 0x7f4808101e20 2024-05-10T11:07:15.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:15.129+0000 7f47f77fe700 1 -- 172.21.15.100:0/1003277040 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f47f8022cb0 con 0x7f4808101e20 2024-05-10T11:07:15.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:15.129+0000 7f480c961700 1 -- 172.21.15.100:0/1003277040 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f480819a880 con 0x7f4808101e20 2024-05-10T11:07:15.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:15.130+0000 7f47f77fe700 1 -- 172.21.15.100:0/1003277040 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f47f802a070 con 0x7f4808101e20 2024-05-10T11:07:15.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:15.130+0000 7f480c961700 1 -- 172.21.15.100:0/1003277040 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4808108310 con 0x7f4808101e20 2024-05-10T11:07:15.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:15.130+0000 7f47f77fe700 1 --2- 172.21.15.100:0/1003277040 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f47f003e570 0x7f47f0040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:07:15.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:15.130+0000 7f47f77fe700 1 -- 172.21.15.100:0/1003277040 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f47f801e030 con 0x7f4808101e20 2024-05-10T11:07:15.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:15.130+0000 7f4805d9b700 1 --2- 172.21.15.100:0/1003277040 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f47f003e570 0x7f47f0040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:07:15.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:15.131+0000 7f4805d9b700 1 --2- 172.21.15.100:0/1003277040 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f47f003e570 0x7f47f0040a30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f47fc006fd0 tx=0x7f47fc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:07:15.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:15.135+0000 7f47f77fe700 1 -- 172.21.15.100:0/1003277040 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f47f8013070 con 0x7f4808101e20 2024-05-10T11:07:15.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:15.344+0000 7f480c961700 1 -- 172.21.15.100:0/1003277040 --> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f4808190610 con 0x7f47f003e570 2024-05-10T11:07:15.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:15.348+0000 7f47f77fe700 1 -- 172.21.15.100:0/1003277040 <== mgr.14152 v2:172.21.15.100:6800/2507287356 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4808190610 con 0x7f47f003e570 2024-05-10T11:07:15.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:15.350+0000 7f480c961700 1 -- 172.21.15.100:0/1003277040 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f47f003e570 msgr2=0x7f47f0040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:07:15.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:15.350+0000 7f480c961700 1 --2- 172.21.15.100:0/1003277040 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f47f003e570 0x7f47f0040a30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f47fc006fd0 tx=0x7f47fc006e40 comp rx=0 tx=0).stop 2024-05-10T11:07:15.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:15.350+0000 7f480c961700 1 -- 172.21.15.100:0/1003277040 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4808101e20 msgr2=0x7f48081973c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:07:15.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:15.350+0000 7f480c961700 1 --2- 172.21.15.100:0/1003277040 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4808101e20 0x7f48081973c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f47f8005b40 tx=0x7f47f8005810 comp rx=0 tx=0).stop 2024-05-10T11:07:15.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:15.350+0000 7f480c961700 1 -- 172.21.15.100:0/1003277040 shutdown_connections 2024-05-10T11:07:15.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:15.350+0000 7f480c961700 1 --2- 172.21.15.100:0/1003277040 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f47f003e570 0x7f47f0040a30 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:07:15.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:15.350+0000 7f480c961700 1 --2- 172.21.15.100:0/1003277040 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4808101e20 0x7f48081973c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:07:15.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:15.350+0000 7f480c961700 1 -- 172.21.15.100:0/1003277040 >> 172.21.15.100:0/1003277040 conn(0x7f48080fd3a0 msgr2=0x7f48080fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:07:15.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:15.350+0000 7f480c961700 1 -- 172.21.15.100:0/1003277040 shutdown_connections 2024-05-10T11:07:15.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:15.350+0000 7f480c961700 1 -- 172.21.15.100:0/1003277040 wait complete. 2024-05-10T11:07:16.071 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi178 2024-05-10T11:07:16.071 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-05-10T11:07:16.071 DEBUG:teuthology.orchestra.run.smithi178:> dd of=/etc/ceph/ceph.conf 2024-05-10T11:07:16.090 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-05-10T11:07:16.090 DEBUG:teuthology.orchestra.run.smithi178:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-10T11:07:16.148 INFO:tasks.cephadm:Adding host smithi178 to orchestrator... 2024-05-10T11:07:16.149 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph orch host add smithi178 2024-05-10T11:07:16.391 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:07:16.615 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:16 smithi100 ceph-mon[30296]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:07:16.616 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:16 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:16.616 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:16 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:07:16.616 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:16 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:07:16.616 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:16 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:07:16.616 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:16 smithi100 ceph-mon[30296]: Updating smithi100:/etc/ceph/ceph.conf 2024-05-10T11:07:16.616 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:16 smithi100 ceph-mon[30296]: Updating smithi100:/var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T11:07:16.616 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:16 smithi100 ceph-mon[30296]: Updating smithi100:/etc/ceph/ceph.client.admin.keyring 2024-05-10T11:07:16.616 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:16 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:16.616 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:16 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:16.617 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:16 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:17.578 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:17 smithi100 ceph-mon[30296]: Updating smithi100:/var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-10T11:07:17.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:17.795+0000 7f660259f700 1 -- 172.21.15.100:0/934316574 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65fc1030e0 msgr2=0x7f65fc1034c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:07:17.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:17.795+0000 7f660259f700 1 --2- 172.21.15.100:0/934316574 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65fc1030e0 0x7f65fc1034c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f65ec009910 tx=0x7f65ec009c20 comp rx=0 tx=0).stop 2024-05-10T11:07:17.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:17.795+0000 7f660259f700 1 -- 172.21.15.100:0/934316574 shutdown_connections 2024-05-10T11:07:17.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:17.795+0000 7f660259f700 1 --2- 172.21.15.100:0/934316574 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65fc1030e0 0x7f65fc1034c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:07:17.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:17.795+0000 7f660259f700 1 -- 172.21.15.100:0/934316574 >> 172.21.15.100:0/934316574 conn(0x7f65fc0fe990 msgr2=0x7f65fc100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:07:17.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:17.795+0000 7f660259f700 1 -- 172.21.15.100:0/934316574 shutdown_connections 2024-05-10T11:07:17.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:17.795+0000 7f660259f700 1 -- 172.21.15.100:0/934316574 wait complete. 2024-05-10T11:07:17.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:17.796+0000 7f660259f700 1 Processor -- start 2024-05-10T11:07:17.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:17.796+0000 7f660259f700 1 -- start start 2024-05-10T11:07:17.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:17.796+0000 7f660259f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65fc1030e0 0x7f65fc19e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:07:17.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:17.796+0000 7f660259f700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f65fc10d210 con 0x7f65fc1030e0 2024-05-10T11:07:17.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:17.796+0000 7f65fbfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65fc1030e0 0x7f65fc19e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:07:17.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:17.796+0000 7f65fbfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65fc1030e0 0x7f65fc19e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43444/0 (socket says 172.21.15.100:43444) 2024-05-10T11:07:17.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:17.796+0000 7f65fbfff700 1 -- 172.21.15.100:0/1422951961 learned_addr learned my addr 172.21.15.100:0/1422951961 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:07:17.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:17.796+0000 7f65fbfff700 1 -- 172.21.15.100:0/1422951961 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65fc19eb70 con 0x7f65fc1030e0 2024-05-10T11:07:17.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:17.797+0000 7f65fbfff700 1 --2- 172.21.15.100:0/1422951961 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65fc1030e0 0x7f65fc19e630 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f65ec005660 tx=0x7f65ec005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:07:17.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:17.797+0000 7f65f9ffb700 1 -- 172.21.15.100:0/1422951961 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65ec026070 con 0x7f65fc1030e0 2024-05-10T11:07:17.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:17.797+0000 7f65f9ffb700 1 -- 172.21.15.100:0/1422951961 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f65ec00b6f0 con 0x7f65fc1030e0 2024-05-10T11:07:17.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:17.797+0000 7f660259f700 1 -- 172.21.15.100:0/1422951961 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65fc19ee60 con 0x7f65fc1030e0 2024-05-10T11:07:17.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:17.797+0000 7f65f9ffb700 1 -- 172.21.15.100:0/1422951961 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65ec0037b0 con 0x7f65fc1030e0 2024-05-10T11:07:17.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:17.797+0000 7f660259f700 1 -- 172.21.15.100:0/1422951961 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65fc1988d0 con 0x7f65fc1030e0 2024-05-10T11:07:17.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:17.798+0000 7f65f9ffb700 1 -- 172.21.15.100:0/1422951961 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f65ec029070 con 0x7f65fc1030e0 2024-05-10T11:07:17.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:17.798+0000 7f660259f700 1 -- 172.21.15.100:0/1422951961 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65fc1045c0 con 0x7f65fc1030e0 2024-05-10T11:07:17.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:17.798+0000 7f65f9ffb700 1 --2- 172.21.15.100:0/1422951961 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f65e803e510 0x7f65e80409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:07:17.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:17.798+0000 7f65f9ffb700 1 -- 172.21.15.100:0/1422951961 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f65ec01e070 con 0x7f65fc1030e0 2024-05-10T11:07:17.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:17.798+0000 7f65f3fff700 1 --2- 172.21.15.100:0/1422951961 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f65e803e510 0x7f65e80409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:07:17.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:17.798+0000 7f65f3fff700 1 --2- 172.21.15.100:0/1422951961 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f65e803e510 0x7f65e80409d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f65e4006fd0 tx=0x7f65e4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:07:17.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:17.803+0000 7f65f9ffb700 1 -- 172.21.15.100:0/1422951961 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f65ec013070 con 0x7f65fc1030e0 2024-05-10T11:07:18.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:18.011+0000 7f660259f700 1 -- 172.21.15.100:0/1422951961 --> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi178", "target": ["mon-mgr", ""]}) -- 0x7f65fc199c30 con 0x7f65e803e510 2024-05-10T11:07:19.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:19 smithi100 ceph-mon[30296]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi178", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:07:19.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:19 smithi100 ceph-mon[30296]: Deploying cephadm binary to smithi178 2024-05-10T11:07:21.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:21 smithi100 ceph-mon[30296]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:07:22.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:22.136+0000 7f65f9ffb700 1 -- 172.21.15.100:0/1422951961 <== mgr.14152 v2:172.21.15.100:6800/2507287356 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f65fc199c30 con 0x7f65e803e510 2024-05-10T11:07:22.138 INFO:teuthology.orchestra.run.smithi100.stdout:Added host 'smithi178' with addr '172.21.15.178' 2024-05-10T11:07:22.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:22.139+0000 7f660259f700 1 -- 172.21.15.100:0/1422951961 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f65e803e510 msgr2=0x7f65e80409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:07:22.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:22.139+0000 7f660259f700 1 --2- 172.21.15.100:0/1422951961 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f65e803e510 0x7f65e80409d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f65e4006fd0 tx=0x7f65e4006e40 comp rx=0 tx=0).stop 2024-05-10T11:07:22.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:22.139+0000 7f660259f700 1 -- 172.21.15.100:0/1422951961 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65fc1030e0 msgr2=0x7f65fc19e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:07:22.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:22.139+0000 7f660259f700 1 --2- 172.21.15.100:0/1422951961 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65fc1030e0 0x7f65fc19e630 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f65ec005660 tx=0x7f65ec005740 comp rx=0 tx=0).stop 2024-05-10T11:07:22.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:22.139+0000 7f660259f700 1 -- 172.21.15.100:0/1422951961 shutdown_connections 2024-05-10T11:07:22.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:22.139+0000 7f660259f700 1 --2- 172.21.15.100:0/1422951961 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f65e803e510 0x7f65e80409d0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:07:22.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:22.139+0000 7f660259f700 1 --2- 172.21.15.100:0/1422951961 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65fc1030e0 0x7f65fc19e630 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:07:22.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:22.139+0000 7f660259f700 1 -- 172.21.15.100:0/1422951961 >> 172.21.15.100:0/1422951961 conn(0x7f65fc0fe990 msgr2=0x7f65fc10ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:07:22.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:22.139+0000 7f660259f700 1 -- 172.21.15.100:0/1422951961 shutdown_connections 2024-05-10T11:07:22.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:22.139+0000 7f660259f700 1 -- 172.21.15.100:0/1422951961 wait complete. 2024-05-10T11:07:22.749 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph orch host ls --format=json 2024-05-10T11:07:22.986 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:07:23.359 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:23 smithi100 ceph-mon[30296]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:07:23.359 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:23 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:23.359 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:23 smithi100 ceph-mon[30296]: Added host smithi178 2024-05-10T11:07:23.359 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:23 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:07:23.359 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:23 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:24.331 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:24 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:24.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:24.623+0000 7f2aaf95b700 1 -- 172.21.15.100:0/476064141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2aa81040f0 msgr2=0x7f2aa81044d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:07:24.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:24.623+0000 7f2aaf95b700 1 --2- 172.21.15.100:0/476064141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2aa81040f0 0x7f2aa81044d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2aa4009910 tx=0x7f2aa4009c20 comp rx=0 tx=0).stop 2024-05-10T11:07:24.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:24.623+0000 7f2aaf95b700 1 -- 172.21.15.100:0/476064141 shutdown_connections 2024-05-10T11:07:24.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:24.623+0000 7f2aaf95b700 1 --2- 172.21.15.100:0/476064141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2aa81040f0 0x7f2aa81044d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:07:24.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:24.623+0000 7f2aaf95b700 1 -- 172.21.15.100:0/476064141 >> 172.21.15.100:0/476064141 conn(0x7f2aa80fd8e0 msgr2=0x7f2aa80ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:07:24.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:24.623+0000 7f2aaf95b700 1 -- 172.21.15.100:0/476064141 shutdown_connections 2024-05-10T11:07:24.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:24.623+0000 7f2aaf95b700 1 -- 172.21.15.100:0/476064141 wait complete. 2024-05-10T11:07:24.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:24.623+0000 7f2aaf95b700 1 Processor -- start 2024-05-10T11:07:24.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:24.623+0000 7f2aaf95b700 1 -- start start 2024-05-10T11:07:24.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:24.624+0000 7f2aaf95b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2aa81040f0 0x7f2aa819e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:07:24.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:24.624+0000 7f2aaf95b700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa810d2b0 con 0x7f2aa81040f0 2024-05-10T11:07:24.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:24.624+0000 7f2aad6f7700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2aa81040f0 0x7f2aa819e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:07:24.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:24.624+0000 7f2aad6f7700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2aa81040f0 0x7f2aa819e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43460/0 (socket says 172.21.15.100:43460) 2024-05-10T11:07:24.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:24.624+0000 7f2aad6f7700 1 -- 172.21.15.100:0/2388377199 learned_addr learned my addr 172.21.15.100:0/2388377199 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:07:24.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:24.624+0000 7f2aad6f7700 1 -- 172.21.15.100:0/2388377199 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aa819eba0 con 0x7f2aa81040f0 2024-05-10T11:07:24.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:24.624+0000 7f2aad6f7700 1 --2- 172.21.15.100:0/2388377199 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2aa81040f0 0x7f2aa819e660 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2aa4005060 tx=0x7f2aa4005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:07:24.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:24.624+0000 7f2a9e7fc700 1 -- 172.21.15.100:0/2388377199 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2aa4026070 con 0x7f2aa81040f0 2024-05-10T11:07:24.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:24.624+0000 7f2a9e7fc700 1 -- 172.21.15.100:0/2388377199 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2aa400b750 con 0x7f2aa81040f0 2024-05-10T11:07:24.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:24.624+0000 7f2aaf95b700 1 -- 172.21.15.100:0/2388377199 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2aa819ee30 con 0x7f2aa81040f0 2024-05-10T11:07:24.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:24.624+0000 7f2a9e7fc700 1 -- 172.21.15.100:0/2388377199 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2aa4003850 con 0x7f2aa81040f0 2024-05-10T11:07:24.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:24.624+0000 7f2aaf95b700 1 -- 172.21.15.100:0/2388377199 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2aa8198730 con 0x7f2aa81040f0 2024-05-10T11:07:24.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:24.625+0000 7f2a9e7fc700 1 -- 172.21.15.100:0/2388377199 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f2aa4029070 con 0x7f2aa81040f0 2024-05-10T11:07:24.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:24.625+0000 7f2aaf95b700 1 -- 172.21.15.100:0/2388377199 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2aa806b9c0 con 0x7f2aa81040f0 2024-05-10T11:07:24.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:24.625+0000 7f2a9e7fc700 1 --2- 172.21.15.100:0/2388377199 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2a9403e510 0x7f2a940409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:07:24.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:24.625+0000 7f2a9e7fc700 1 -- 172.21.15.100:0/2388377199 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f2aa401e070 con 0x7f2aa81040f0 2024-05-10T11:07:24.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:24.626+0000 7f2aacef6700 1 --2- 172.21.15.100:0/2388377199 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2a9403e510 0x7f2a940409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:07:24.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:24.626+0000 7f2aacef6700 1 --2- 172.21.15.100:0/2388377199 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2a9403e510 0x7f2a940409d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2a98006fd0 tx=0x7f2a98006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:07:24.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:24.630+0000 7f2a9e7fc700 1 -- 172.21.15.100:0/2388377199 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f2aa4013070 con 0x7f2aa81040f0 2024-05-10T11:07:24.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:24.838+0000 7f2aaf95b700 1 -- 172.21.15.100:0/2388377199 --> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2aa8199c60 con 0x7f2a9403e510 2024-05-10T11:07:24.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:24.839+0000 7f2a9e7fc700 1 -- 172.21.15.100:0/2388377199 <== mgr.14152 v2:172.21.15.100:6800/2507287356 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+161 (secure 0 0 0) 0x7f2aa8199c60 con 0x7f2a9403e510 2024-05-10T11:07:24.839 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:07:24.839 INFO:teuthology.orchestra.run.smithi100.stdout:[{"addr": "172.21.15.100", "hostname": "smithi100", "labels": [], "status": ""}, {"addr": "172.21.15.178", "hostname": "smithi178", "labels": [], "status": ""}] 2024-05-10T11:07:24.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:24.842+0000 7f2aaf95b700 1 -- 172.21.15.100:0/2388377199 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2a9403e510 msgr2=0x7f2a940409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:07:24.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:24.842+0000 7f2aaf95b700 1 --2- 172.21.15.100:0/2388377199 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2a9403e510 0x7f2a940409d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2a98006fd0 tx=0x7f2a98006e40 comp rx=0 tx=0).stop 2024-05-10T11:07:24.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:24.842+0000 7f2aaf95b700 1 -- 172.21.15.100:0/2388377199 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2aa81040f0 msgr2=0x7f2aa819e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:07:24.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:24.842+0000 7f2aaf95b700 1 --2- 172.21.15.100:0/2388377199 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2aa81040f0 0x7f2aa819e660 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2aa4005060 tx=0x7f2aa4005710 comp rx=0 tx=0).stop 2024-05-10T11:07:24.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:24.842+0000 7f2aaf95b700 1 -- 172.21.15.100:0/2388377199 shutdown_connections 2024-05-10T11:07:24.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:24.842+0000 7f2aaf95b700 1 --2- 172.21.15.100:0/2388377199 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2a9403e510 0x7f2a940409d0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:07:24.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:24.842+0000 7f2aaf95b700 1 --2- 172.21.15.100:0/2388377199 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2aa81040f0 0x7f2aa819e660 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:07:24.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:24.842+0000 7f2aaf95b700 1 -- 172.21.15.100:0/2388377199 >> 172.21.15.100:0/2388377199 conn(0x7f2aa80fd8e0 msgr2=0x7f2aa8106eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:07:24.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:24.842+0000 7f2aaf95b700 1 -- 172.21.15.100:0/2388377199 shutdown_connections 2024-05-10T11:07:24.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:24.842+0000 7f2aaf95b700 1 -- 172.21.15.100:0/2388377199 wait complete. 2024-05-10T11:07:25.476 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi188 2024-05-10T11:07:25.476 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-05-10T11:07:25.476 DEBUG:teuthology.orchestra.run.smithi188:> dd of=/etc/ceph/ceph.conf 2024-05-10T11:07:25.495 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-05-10T11:07:25.495 DEBUG:teuthology.orchestra.run.smithi188:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-10T11:07:25.553 INFO:tasks.cephadm:Adding host smithi188 to orchestrator... 2024-05-10T11:07:25.553 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph orch host add smithi188 2024-05-10T11:07:25.786 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:07:26.001 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:25 smithi100 ceph-mon[30296]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:07:26.001 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:25 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:26.001 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:25 smithi100 ceph-mon[30296]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T11:07:27.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:27.194+0000 7f00f0010700 1 -- 172.21.15.100:0/4206257204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00e8103dd0 msgr2=0x7f00e81012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:07:27.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:27.194+0000 7f00f0010700 1 --2- 172.21.15.100:0/4206257204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00e8103dd0 0x7f00e81012e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f00e4009910 tx=0x7f00e4009c20 comp rx=0 tx=0).stop 2024-05-10T11:07:27.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:27.195+0000 7f00f0010700 1 -- 172.21.15.100:0/4206257204 shutdown_connections 2024-05-10T11:07:27.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:27.195+0000 7f00f0010700 1 --2- 172.21.15.100:0/4206257204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00e8103dd0 0x7f00e81012e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:07:27.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:27.195+0000 7f00f0010700 1 -- 172.21.15.100:0/4206257204 >> 172.21.15.100:0/4206257204 conn(0x7f00e80fd020 msgr2=0x7f00e80ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:07:27.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:27.195+0000 7f00f0010700 1 -- 172.21.15.100:0/4206257204 shutdown_connections 2024-05-10T11:07:27.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:27.195+0000 7f00f0010700 1 -- 172.21.15.100:0/4206257204 wait complete. 2024-05-10T11:07:27.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:27.195+0000 7f00f0010700 1 Processor -- start 2024-05-10T11:07:27.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:27.195+0000 7f00f0010700 1 -- start start 2024-05-10T11:07:27.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:27.196+0000 7f00f0010700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00e8103dd0 0x7f00e81a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:07:27.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:27.196+0000 7f00f0010700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f00e810f480 con 0x7f00e8103dd0 2024-05-10T11:07:27.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:27.196+0000 7f00eddac700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00e8103dd0 0x7f00e81a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:07:27.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:27.196+0000 7f00eddac700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00e8103dd0 0x7f00e81a06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33856/0 (socket says 172.21.15.100:33856) 2024-05-10T11:07:27.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:27.196+0000 7f00eddac700 1 -- 172.21.15.100:0/3953900332 learned_addr learned my addr 172.21.15.100:0/3953900332 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:07:27.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:27.196+0000 7f00eddac700 1 -- 172.21.15.100:0/3953900332 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00e81a0bf0 con 0x7f00e8103dd0 2024-05-10T11:07:27.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:27.196+0000 7f00eddac700 1 --2- 172.21.15.100:0/3953900332 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00e8103dd0 0x7f00e81a06b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f00e4005060 tx=0x7f00e4005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:07:27.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:27.196+0000 7f00deffd700 1 -- 172.21.15.100:0/3953900332 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f00e4026070 con 0x7f00e8103dd0 2024-05-10T11:07:27.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:27.196+0000 7f00deffd700 1 -- 172.21.15.100:0/3953900332 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f00e400b750 con 0x7f00e8103dd0 2024-05-10T11:07:27.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:27.196+0000 7f00f0010700 1 -- 172.21.15.100:0/3953900332 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00e81a0e80 con 0x7f00e8103dd0 2024-05-10T11:07:27.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:27.196+0000 7f00deffd700 1 -- 172.21.15.100:0/3953900332 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f00e4003850 con 0x7f00e8103dd0 2024-05-10T11:07:27.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:27.196+0000 7f00f0010700 1 -- 172.21.15.100:0/3953900332 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00e819a870 con 0x7f00e8103dd0 2024-05-10T11:07:27.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:27.197+0000 7f00f0010700 1 -- 172.21.15.100:0/3953900332 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00e8102500 con 0x7f00e8103dd0 2024-05-10T11:07:27.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:27.197+0000 7f00deffd700 1 -- 172.21.15.100:0/3953900332 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f00e4029070 con 0x7f00e8103dd0 2024-05-10T11:07:27.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:27.198+0000 7f00deffd700 1 --2- 172.21.15.100:0/3953900332 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f00d403e510 0x7f00d40409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:07:27.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:27.198+0000 7f00deffd700 1 -- 172.21.15.100:0/3953900332 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f00e401e070 con 0x7f00e8103dd0 2024-05-10T11:07:27.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:27.198+0000 7f00ed5ab700 1 --2- 172.21.15.100:0/3953900332 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f00d403e510 0x7f00d40409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:07:27.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:27.198+0000 7f00ed5ab700 1 --2- 172.21.15.100:0/3953900332 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f00d403e510 0x7f00d40409d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f00d8006fd0 tx=0x7f00d8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:07:27.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:27.203+0000 7f00deffd700 1 -- 172.21.15.100:0/3953900332 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f00e4013070 con 0x7f00e8103dd0 2024-05-10T11:07:27.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:27.409+0000 7f00f0010700 1 -- 172.21.15.100:0/3953900332 --> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi188", "target": ["mon-mgr", ""]}) -- 0x7f00e819bda0 con 0x7f00d403e510 2024-05-10T11:07:28.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:27 smithi100 ceph-mon[30296]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:07:28.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:27 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:28.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:27 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:28.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:27 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:28.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:27 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:28.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:27 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi178", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:07:28.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:27 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:07:28.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:27 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:07:29.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:28 smithi100 ceph-mon[30296]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi188", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:07:29.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:28 smithi100 ceph-mon[30296]: Updating smithi178:/etc/ceph/ceph.conf 2024-05-10T11:07:29.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:28 smithi100 ceph-mon[30296]: Updating smithi178:/var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T11:07:30.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:29 smithi100 ceph-mon[30296]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:07:30.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:29 smithi100 ceph-mon[30296]: Deploying cephadm binary to smithi188 2024-05-10T11:07:30.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:29 smithi100 ceph-mon[30296]: Updating smithi178:/etc/ceph/ceph.client.admin.keyring 2024-05-10T11:07:30.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:29 smithi100 ceph-mon[30296]: Updating smithi178:/var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-10T11:07:31.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:31.606+0000 7f00deffd700 1 -- 172.21.15.100:0/3953900332 <== mgr.14152 v2:172.21.15.100:6800/2507287356 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f00e819bda0 con 0x7f00d403e510 2024-05-10T11:07:31.607 INFO:teuthology.orchestra.run.smithi100.stdout:Added host 'smithi188' with addr '172.21.15.188' 2024-05-10T11:07:31.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:31.609+0000 7f00f0010700 1 -- 172.21.15.100:0/3953900332 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f00d403e510 msgr2=0x7f00d40409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:07:31.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:31.609+0000 7f00f0010700 1 --2- 172.21.15.100:0/3953900332 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f00d403e510 0x7f00d40409d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f00d8006fd0 tx=0x7f00d8006e40 comp rx=0 tx=0).stop 2024-05-10T11:07:31.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:31.609+0000 7f00f0010700 1 -- 172.21.15.100:0/3953900332 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00e8103dd0 msgr2=0x7f00e81a06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:07:31.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:31.609+0000 7f00f0010700 1 --2- 172.21.15.100:0/3953900332 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00e8103dd0 0x7f00e81a06b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f00e4005060 tx=0x7f00e4005710 comp rx=0 tx=0).stop 2024-05-10T11:07:31.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:31.609+0000 7f00f0010700 1 -- 172.21.15.100:0/3953900332 shutdown_connections 2024-05-10T11:07:31.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:31.609+0000 7f00f0010700 1 --2- 172.21.15.100:0/3953900332 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f00d403e510 0x7f00d40409d0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:07:31.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:31.609+0000 7f00f0010700 1 --2- 172.21.15.100:0/3953900332 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00e8103dd0 0x7f00e81a06b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:07:31.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:31.609+0000 7f00f0010700 1 -- 172.21.15.100:0/3953900332 >> 172.21.15.100:0/3953900332 conn(0x7f00e80fd020 msgr2=0x7f00e80fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:07:31.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:31.609+0000 7f00f0010700 1 -- 172.21.15.100:0/3953900332 shutdown_connections 2024-05-10T11:07:31.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:31.609+0000 7f00f0010700 1 -- 172.21.15.100:0/3953900332 wait complete. 2024-05-10T11:07:32.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:31 smithi100 ceph-mon[30296]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:07:32.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:31 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:32.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:31 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:32.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:31 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:32.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:31 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:32.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:31 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:07:32.274 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph orch host ls --format=json 2024-05-10T11:07:32.509 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:07:33.282 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:32 smithi100 ceph-mon[30296]: Added host smithi188 2024-05-10T11:07:33.283 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:32 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:33.283 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:32 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:34.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:34.111+0000 7f6115999700 1 -- 172.21.15.100:0/2131313268 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61101040f0 msgr2=0x7f61101044d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:07:34.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:34.111+0000 7f6115999700 1 --2- 172.21.15.100:0/2131313268 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61101040f0 0x7f61101044d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6100009910 tx=0x7f6100009c20 comp rx=0 tx=0).stop 2024-05-10T11:07:34.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:34.111+0000 7f6115999700 1 -- 172.21.15.100:0/2131313268 shutdown_connections 2024-05-10T11:07:34.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:34.111+0000 7f6115999700 1 --2- 172.21.15.100:0/2131313268 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61101040f0 0x7f61101044d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:07:34.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:34.111+0000 7f6115999700 1 -- 172.21.15.100:0/2131313268 >> 172.21.15.100:0/2131313268 conn(0x7f61100fd8e0 msgr2=0x7f61100ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:07:34.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:34.111+0000 7f6115999700 1 -- 172.21.15.100:0/2131313268 shutdown_connections 2024-05-10T11:07:34.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:34.111+0000 7f6115999700 1 -- 172.21.15.100:0/2131313268 wait complete. 2024-05-10T11:07:34.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:34.112+0000 7f6115999700 1 Processor -- start 2024-05-10T11:07:34.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:34.112+0000 7f6115999700 1 -- start start 2024-05-10T11:07:34.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:34.112+0000 7f6115999700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61101040f0 0x7f611019e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:07:34.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:34.112+0000 7f6115999700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f611010d2b0 con 0x7f61101040f0 2024-05-10T11:07:34.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:34.112+0000 7f610effd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61101040f0 0x7f611019e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:07:34.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:34.112+0000 7f610effd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61101040f0 0x7f611019e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33872/0 (socket says 172.21.15.100:33872) 2024-05-10T11:07:34.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:34.112+0000 7f610effd700 1 -- 172.21.15.100:0/263907666 learned_addr learned my addr 172.21.15.100:0/263907666 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:07:34.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:34.112+0000 7f610effd700 1 -- 172.21.15.100:0/263907666 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f611019eba0 con 0x7f61101040f0 2024-05-10T11:07:34.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:34.112+0000 7f610effd700 1 --2- 172.21.15.100:0/263907666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61101040f0 0x7f611019e660 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6100005060 tx=0x7f6100005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:07:34.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:34.113+0000 7f6114997700 1 -- 172.21.15.100:0/263907666 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6100026070 con 0x7f61101040f0 2024-05-10T11:07:34.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:34.113+0000 7f6115999700 1 -- 172.21.15.100:0/263907666 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f611019ee30 con 0x7f61101040f0 2024-05-10T11:07:34.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:34.113+0000 7f6114997700 1 -- 172.21.15.100:0/263907666 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f610000b750 con 0x7f61101040f0 2024-05-10T11:07:34.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:34.113+0000 7f6114997700 1 -- 172.21.15.100:0/263907666 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6100003850 con 0x7f61101040f0 2024-05-10T11:07:34.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:34.113+0000 7f6115999700 1 -- 172.21.15.100:0/263907666 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6110198730 con 0x7f61101040f0 2024-05-10T11:07:34.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:34.113+0000 7f6115999700 1 -- 172.21.15.100:0/263907666 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f611006b9c0 con 0x7f61101040f0 2024-05-10T11:07:34.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:34.113+0000 7f6114997700 1 -- 172.21.15.100:0/263907666 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f6100029070 con 0x7f61101040f0 2024-05-10T11:07:34.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:34.114+0000 7f6114997700 1 --2- 172.21.15.100:0/263907666 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f60fc03e510 0x7f60fc0409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:07:34.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:34.114+0000 7f6114997700 1 -- 172.21.15.100:0/263907666 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f610001e070 con 0x7f61101040f0 2024-05-10T11:07:34.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:34.114+0000 7f6107fff700 1 --2- 172.21.15.100:0/263907666 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f60fc03e510 0x7f60fc0409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:07:34.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:34.114+0000 7f6107fff700 1 --2- 172.21.15.100:0/263907666 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f60fc03e510 0x7f60fc0409d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f60f8006fd0 tx=0x7f60f8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:07:34.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:34.119+0000 7f6114997700 1 -- 172.21.15.100:0/263907666 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f6100013070 con 0x7f61101040f0 2024-05-10T11:07:34.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:33 smithi100 ceph-mon[30296]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:07:34.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:34.330+0000 7f6115999700 1 -- 172.21.15.100:0/263907666 --> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6110064b00 con 0x7f60fc03e510 2024-05-10T11:07:34.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:34.331+0000 7f6114997700 1 -- 172.21.15.100:0/263907666 <== mgr.14152 v2:172.21.15.100:6800/2507287356 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+241 (secure 0 0 0) 0x7f6110064b00 con 0x7f60fc03e510 2024-05-10T11:07:34.331 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:07:34.331 INFO:teuthology.orchestra.run.smithi100.stdout:[{"addr": "172.21.15.100", "hostname": "smithi100", "labels": [], "status": ""}, {"addr": "172.21.15.178", "hostname": "smithi178", "labels": [], "status": ""}, {"addr": "172.21.15.188", "hostname": "smithi188", "labels": [], "status": ""}] 2024-05-10T11:07:34.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:34.333+0000 7f6115999700 1 -- 172.21.15.100:0/263907666 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f60fc03e510 msgr2=0x7f60fc0409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:07:34.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:34.333+0000 7f6115999700 1 --2- 172.21.15.100:0/263907666 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f60fc03e510 0x7f60fc0409d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f60f8006fd0 tx=0x7f60f8006e40 comp rx=0 tx=0).stop 2024-05-10T11:07:34.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:34.333+0000 7f6115999700 1 -- 172.21.15.100:0/263907666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61101040f0 msgr2=0x7f611019e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:07:34.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:34.333+0000 7f6115999700 1 --2- 172.21.15.100:0/263907666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61101040f0 0x7f611019e660 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6100005060 tx=0x7f6100005710 comp rx=0 tx=0).stop 2024-05-10T11:07:34.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:34.333+0000 7f6115999700 1 -- 172.21.15.100:0/263907666 shutdown_connections 2024-05-10T11:07:34.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:34.333+0000 7f6115999700 1 --2- 172.21.15.100:0/263907666 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f60fc03e510 0x7f60fc0409d0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:07:34.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:34.333+0000 7f6115999700 1 --2- 172.21.15.100:0/263907666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61101040f0 0x7f611019e660 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:07:34.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:34.333+0000 7f6115999700 1 -- 172.21.15.100:0/263907666 >> 172.21.15.100:0/263907666 conn(0x7f61100fd8e0 msgr2=0x7f6110106eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:07:34.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:34.333+0000 7f6115999700 1 -- 172.21.15.100:0/263907666 shutdown_connections 2024-05-10T11:07:34.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:34.333+0000 7f6115999700 1 -- 172.21.15.100:0/263907666 wait complete. 2024-05-10T11:07:34.968 INFO:tasks.cephadm:Setting crush tunables to default 2024-05-10T11:07:34.968 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph osd crush tunables default 2024-05-10T11:07:35.203 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:07:35.533 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:35 smithi100 ceph-mon[30296]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:07:35.533 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:35 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:35.534 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:35 smithi100 ceph-mon[30296]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T11:07:36.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:36.762+0000 7f2c1fae5700 1 -- 172.21.15.100:0/3781234347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c18103100 msgr2=0x7f2c181034e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:07:36.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:36.762+0000 7f2c1fae5700 1 --2- 172.21.15.100:0/3781234347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c18103100 0x7f2c181034e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2c14009910 tx=0x7f2c14009c20 comp rx=0 tx=0).stop 2024-05-10T11:07:36.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:36.762+0000 7f2c1fae5700 1 -- 172.21.15.100:0/3781234347 shutdown_connections 2024-05-10T11:07:36.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:36.762+0000 7f2c1fae5700 1 --2- 172.21.15.100:0/3781234347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c18103100 0x7f2c181034e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:07:36.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:36.762+0000 7f2c1fae5700 1 -- 172.21.15.100:0/3781234347 >> 172.21.15.100:0/3781234347 conn(0x7f2c180fe990 msgr2=0x7f2c18100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:07:36.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:36.762+0000 7f2c1fae5700 1 -- 172.21.15.100:0/3781234347 shutdown_connections 2024-05-10T11:07:36.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:36.762+0000 7f2c1fae5700 1 -- 172.21.15.100:0/3781234347 wait complete. 2024-05-10T11:07:36.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:36.762+0000 7f2c1fae5700 1 Processor -- start 2024-05-10T11:07:36.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:36.762+0000 7f2c1fae5700 1 -- start start 2024-05-10T11:07:36.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:36.763+0000 7f2c1fae5700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c18103100 0x7f2c1819e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:07:36.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:36.763+0000 7f2c1fae5700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2c1810d230 con 0x7f2c18103100 2024-05-10T11:07:36.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:36.763+0000 7f2c1d881700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c18103100 0x7f2c1819e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:07:36.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:36.763+0000 7f2c1d881700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c18103100 0x7f2c1819e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34594/0 (socket says 172.21.15.100:34594) 2024-05-10T11:07:36.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:36.763+0000 7f2c1d881700 1 -- 172.21.15.100:0/3754236342 learned_addr learned my addr 172.21.15.100:0/3754236342 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:07:36.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:36.763+0000 7f2c1d881700 1 -- 172.21.15.100:0/3754236342 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c1819eab0 con 0x7f2c18103100 2024-05-10T11:07:36.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:36.763+0000 7f2c1d881700 1 --2- 172.21.15.100:0/3754236342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c18103100 0x7f2c1819e570 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2c14005660 tx=0x7f2c14005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:07:36.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:36.763+0000 7f2c0effd700 1 -- 172.21.15.100:0/3754236342 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c14026070 con 0x7f2c18103100 2024-05-10T11:07:36.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:36.763+0000 7f2c1fae5700 1 -- 172.21.15.100:0/3754236342 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c1819eda0 con 0x7f2c18103100 2024-05-10T11:07:36.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:36.763+0000 7f2c0effd700 1 -- 172.21.15.100:0/3754236342 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2c1400b6f0 con 0x7f2c18103100 2024-05-10T11:07:36.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:36.763+0000 7f2c0effd700 1 -- 172.21.15.100:0/3754236342 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c140037b0 con 0x7f2c18103100 2024-05-10T11:07:36.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:36.763+0000 7f2c1fae5700 1 -- 172.21.15.100:0/3754236342 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c18198810 con 0x7f2c18103100 2024-05-10T11:07:36.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:36.764+0000 7f2c1fae5700 1 -- 172.21.15.100:0/3754236342 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c181045e0 con 0x7f2c18103100 2024-05-10T11:07:36.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:36.764+0000 7f2c0effd700 1 -- 172.21.15.100:0/3754236342 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f2c14029070 con 0x7f2c18103100 2024-05-10T11:07:36.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:36.765+0000 7f2c0effd700 1 --2- 172.21.15.100:0/3754236342 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2c0403e510 0x7f2c040409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:07:36.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:36.765+0000 7f2c0effd700 1 -- 172.21.15.100:0/3754236342 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f2c1401e070 con 0x7f2c18103100 2024-05-10T11:07:36.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:36.765+0000 7f2c1d080700 1 --2- 172.21.15.100:0/3754236342 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2c0403e510 0x7f2c040409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:07:36.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:36.765+0000 7f2c1d080700 1 --2- 172.21.15.100:0/3754236342 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2c0403e510 0x7f2c040409d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2c08006fd0 tx=0x7f2c08006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:07:36.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:36.770+0000 7f2c0effd700 1 -- 172.21.15.100:0/3754236342 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f2c14013070 con 0x7f2c18103100 2024-05-10T11:07:36.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:36.986+0000 7f2c1fae5700 1 -- 172.21.15.100:0/3754236342 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f2c18199b70 con 0x7f2c18103100 2024-05-10T11:07:37.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:37.224+0000 7f2c0effd700 1 -- 172.21.15.100:0/3754236342 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f2c1400d340 con 0x7f2c18103100 2024-05-10T11:07:37.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:37.227+0000 7f2c1fae5700 1 -- 172.21.15.100:0/3754236342 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2c0403e510 msgr2=0x7f2c040409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:07:37.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:37.227+0000 7f2c1fae5700 1 --2- 172.21.15.100:0/3754236342 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2c0403e510 0x7f2c040409d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2c08006fd0 tx=0x7f2c08006e40 comp rx=0 tx=0).stop 2024-05-10T11:07:37.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:37.227+0000 7f2c1fae5700 1 -- 172.21.15.100:0/3754236342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c18103100 msgr2=0x7f2c1819e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:07:37.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:37.227+0000 7f2c1fae5700 1 --2- 172.21.15.100:0/3754236342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c18103100 0x7f2c1819e570 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2c14005660 tx=0x7f2c14005740 comp rx=0 tx=0).stop 2024-05-10T11:07:37.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:37.227+0000 7f2c1fae5700 1 -- 172.21.15.100:0/3754236342 shutdown_connections 2024-05-10T11:07:37.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:37.227+0000 7f2c1fae5700 1 --2- 172.21.15.100:0/3754236342 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2c0403e510 0x7f2c040409d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:07:37.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:37.227+0000 7f2c1fae5700 1 --2- 172.21.15.100:0/3754236342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c18103100 0x7f2c1819e570 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:07:37.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:37.227+0000 7f2c1fae5700 1 -- 172.21.15.100:0/3754236342 >> 172.21.15.100:0/3754236342 conn(0x7f2c180fe990 msgr2=0x7f2c18108c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:07:37.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:37.228+0000 7f2c1fae5700 1 -- 172.21.15.100:0/3754236342 shutdown_connections 2024-05-10T11:07:37.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:07:37.228+0000 7f2c1fae5700 1 -- 172.21.15.100:0/3754236342 wait complete. 2024-05-10T11:07:37.228 INFO:teuthology.orchestra.run.smithi100.stderr:adjusted tunables profile to default 2024-05-10T11:07:37.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:37 smithi100 ceph-mon[30296]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:07:37.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:37 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/3754236342' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-10T11:07:37.845 INFO:tasks.cephadm:Adding mon.a on smithi100 2024-05-10T11:07:37.845 INFO:tasks.cephadm:Adding mon.b on smithi178 2024-05-10T11:07:37.845 INFO:tasks.cephadm:Adding mon.c on smithi188 2024-05-10T11:07:37.845 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph orch apply mon '3;smithi100:172.21.15.100=a;smithi178:172.21.15.178=b;smithi188:172.21.15.188=c' 2024-05-10T11:07:38.086 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T11:07:38.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:38 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/3754236342' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-10T11:07:38.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:38 smithi100 ceph-mon[30296]: osdmap e4: 0 total, 0 up, 0 in 2024-05-10T11:07:38.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:38 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:38.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:38 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:38.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:38 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:38.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:38 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:38.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:38 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi188", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:07:38.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:38 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:07:38.539 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:38 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:07:38.539 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:38 smithi100 ceph-mon[30296]: Updating smithi188:/etc/ceph/ceph.conf 2024-05-10T11:07:38.539 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:38 smithi100 ceph-mon[30296]: Updating smithi188:/var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T11:07:39.591 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:39.590+0000 7fc9e5940700 1 -- 172.21.15.188:0/1487977766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9e0107ed0 msgr2=0x7fc9e01082b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:07:39.591 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:39.590+0000 7fc9e5940700 1 --2- 172.21.15.188:0/1487977766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9e0107ed0 0x7fc9e01082b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc9d0009910 tx=0x7fc9d0009c20 comp rx=0 tx=0).stop 2024-05-10T11:07:39.591 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:39.590+0000 7fc9e5940700 1 -- 172.21.15.188:0/1487977766 shutdown_connections 2024-05-10T11:07:39.591 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:39.590+0000 7fc9e5940700 1 --2- 172.21.15.188:0/1487977766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9e0107ed0 0x7fc9e01082b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:07:39.591 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:39.590+0000 7fc9e5940700 1 -- 172.21.15.188:0/1487977766 >> 172.21.15.188:0/1487977766 conn(0x7fc9e00fcff0 msgr2=0x7fc9e00ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:07:39.591 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:39.590+0000 7fc9e5940700 1 -- 172.21.15.188:0/1487977766 shutdown_connections 2024-05-10T11:07:39.591 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:39.590+0000 7fc9e5940700 1 -- 172.21.15.188:0/1487977766 wait complete. 2024-05-10T11:07:39.592 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:39.591+0000 7fc9e5940700 1 Processor -- start 2024-05-10T11:07:39.592 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:39.591+0000 7fc9e5940700 1 -- start start 2024-05-10T11:07:39.592 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:39.591+0000 7fc9e5940700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9e0107ed0 0x7fc9e019e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:07:39.592 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:39.591+0000 7fc9e5940700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e010d390 con 0x7fc9e0107ed0 2024-05-10T11:07:39.594 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:39.591+0000 7fc9deffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9e0107ed0 0x7fc9e019e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:07:39.594 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:39.591+0000 7fc9deffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9e0107ed0 0x7fc9e019e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.188:46814/0 (socket says 172.21.15.188:46814) 2024-05-10T11:07:39.594 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:39.591+0000 7fc9deffd700 1 -- 172.21.15.188:0/1640342132 learned_addr learned my addr 172.21.15.188:0/1640342132 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-05-10T11:07:39.594 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:39.592+0000 7fc9deffd700 1 -- 172.21.15.188:0/1640342132 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9e019eb40 con 0x7fc9e0107ed0 2024-05-10T11:07:39.594 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:39.592+0000 7fc9deffd700 1 --2- 172.21.15.188:0/1640342132 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9e0107ed0 0x7fc9e019e600 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc9d0005660 tx=0x7fc9d0005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:07:39.594 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:39.592+0000 7fc9e493e700 1 -- 172.21.15.188:0/1640342132 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc9d0026070 con 0x7fc9e0107ed0 2024-05-10T11:07:39.594 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:39.592+0000 7fc9e5940700 1 -- 172.21.15.188:0/1640342132 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9e019edd0 con 0x7fc9e0107ed0 2024-05-10T11:07:39.594 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:39.592+0000 7fc9e493e700 1 -- 172.21.15.188:0/1640342132 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc9d000b750 con 0x7fc9e0107ed0 2024-05-10T11:07:39.594 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:39.592+0000 7fc9e493e700 1 -- 172.21.15.188:0/1640342132 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc9d00037b0 con 0x7fc9e0107ed0 2024-05-10T11:07:39.594 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:39.592+0000 7fc9e5940700 1 -- 172.21.15.188:0/1640342132 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9e01986d0 con 0x7fc9e0107ed0 2024-05-10T11:07:39.594 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:39.593+0000 7fc9e5940700 1 -- 172.21.15.188:0/1640342132 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9e0104610 con 0x7fc9e0107ed0 2024-05-10T11:07:39.594 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:39.593+0000 7fc9e493e700 1 -- 172.21.15.188:0/1640342132 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7fc9d0029070 con 0x7fc9e0107ed0 2024-05-10T11:07:39.595 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:39.593+0000 7fc9e493e700 1 --2- 172.21.15.188:0/1640342132 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc9c803e510 0x7fc9c80409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:07:39.595 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:39.593+0000 7fc9e493e700 1 -- 172.21.15.188:0/1640342132 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc9d001e070 con 0x7fc9e0107ed0 2024-05-10T11:07:39.595 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:39.594+0000 7fc9de7fc700 1 --2- 172.21.15.188:0/1640342132 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc9c803e510 0x7fc9c80409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:07:39.595 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:39.594+0000 7fc9de7fc700 1 --2- 172.21.15.188:0/1640342132 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc9c803e510 0x7fc9c80409d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc9d4006fd0 tx=0x7fc9d4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:07:39.599 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:39.599+0000 7fc9e493e700 1 -- 172.21.15.188:0/1640342132 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fc9d0013070 con 0x7fc9e0107ed0 2024-05-10T11:07:39.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:39 smithi100 ceph-mon[30296]: Updating smithi188:/etc/ceph/ceph.client.admin.keyring 2024-05-10T11:07:39.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:39 smithi100 ceph-mon[30296]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:07:39.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:39 smithi100 ceph-mon[30296]: Updating smithi188:/var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-10T11:07:39.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:39 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:39.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:39 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:39.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:39 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:39.819 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:39.819+0000 7fc9e5940700 1 -- 172.21.15.188:0/1640342132 --> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi100:172.21.15.100=a;smithi178:172.21.15.178=b;smithi188:172.21.15.188=c", "target": ["mon-mgr", ""]}) -- 0x7fc9e0199c00 con 0x7fc9c803e510 2024-05-10T11:07:39.825 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:39.825+0000 7fc9e493e700 1 -- 172.21.15.188:0/1640342132 <== mgr.14152 v2:172.21.15.100:6800/2507287356 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fc9e0199c00 con 0x7fc9c803e510 2024-05-10T11:07:39.826 INFO:teuthology.orchestra.run.smithi188.stdout:Scheduled mon update... 2024-05-10T11:07:39.828 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:39.827+0000 7fc9e5940700 1 -- 172.21.15.188:0/1640342132 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc9c803e510 msgr2=0x7fc9c80409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:07:39.828 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:39.827+0000 7fc9e5940700 1 --2- 172.21.15.188:0/1640342132 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc9c803e510 0x7fc9c80409d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc9d4006fd0 tx=0x7fc9d4006e40 comp rx=0 tx=0).stop 2024-05-10T11:07:39.828 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:39.827+0000 7fc9e5940700 1 -- 172.21.15.188:0/1640342132 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9e0107ed0 msgr2=0x7fc9e019e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:07:39.828 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:39.827+0000 7fc9e5940700 1 --2- 172.21.15.188:0/1640342132 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9e0107ed0 0x7fc9e019e600 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc9d0005660 tx=0x7fc9d0005740 comp rx=0 tx=0).stop 2024-05-10T11:07:39.828 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:39.827+0000 7fc9e5940700 1 -- 172.21.15.188:0/1640342132 shutdown_connections 2024-05-10T11:07:39.828 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:39.828+0000 7fc9e5940700 1 --2- 172.21.15.188:0/1640342132 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc9c803e510 0x7fc9c80409d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:07:39.828 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:39.828+0000 7fc9e5940700 1 --2- 172.21.15.188:0/1640342132 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9e0107ed0 0x7fc9e019e600 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:07:39.828 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:39.828+0000 7fc9e5940700 1 -- 172.21.15.188:0/1640342132 >> 172.21.15.188:0/1640342132 conn(0x7fc9e00fcff0 msgr2=0x7fc9e00fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:07:39.828 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:39.828+0000 7fc9e5940700 1 -- 172.21.15.188:0/1640342132 shutdown_connections 2024-05-10T11:07:39.828 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:39.828+0000 7fc9e5940700 1 -- 172.21.15.188:0/1640342132 wait complete. 2024-05-10T11:07:40.419 DEBUG:teuthology.orchestra.run.smithi178:mon.b> sudo journalctl -f -n 0 -u ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mon.b.service 2024-05-10T11:07:40.422 DEBUG:teuthology.orchestra.run.smithi188:mon.c> sudo journalctl -f -n 0 -u ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mon.c.service 2024-05-10T11:07:40.424 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-10T11:07:40.424 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-10T11:07:41.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:40 smithi100 ceph-mon[30296]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi100:172.21.15.100=a;smithi178:172.21.15.178=b;smithi188:172.21.15.188=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:07:41.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:40 smithi100 ceph-mon[30296]: Saving service mon spec with placement smithi100:172.21.15.100=a;smithi178:172.21.15.178=b;smithi188:172.21.15.188=c;count:3 2024-05-10T11:07:41.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:40 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:41.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:40 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:07:41.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:40 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:07:41.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:40 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:07:41.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:40 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:41.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:40 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T11:07:41.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:40 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:07:41.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:40 smithi100 ceph-mon[30296]: Deploying daemon mon.c on smithi188 2024-05-10T11:07:41.401 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T11:07:42.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:41 smithi100 ceph-mon[30296]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:07:44.057 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:44.054+0000 7f887c30a700 1 -- 172.21.15.188:0/2186941431 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8874101c00 msgr2=0x7f8874101fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:07:44.057 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:44.054+0000 7f887c30a700 1 --2- 172.21.15.188:0/2186941431 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8874101c00 0x7f8874101fe0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8870009910 tx=0x7f8870009c20 comp rx=0 tx=0).stop 2024-05-10T11:07:44.057 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:44.054+0000 7f887c30a700 1 -- 172.21.15.188:0/2186941431 shutdown_connections 2024-05-10T11:07:44.057 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:44.054+0000 7f887c30a700 1 --2- 172.21.15.188:0/2186941431 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8874101c00 0x7f8874101fe0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:07:44.057 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:44.054+0000 7f887c30a700 1 -- 172.21.15.188:0/2186941431 >> 172.21.15.188:0/2186941431 conn(0x7f88740fd8e0 msgr2=0x7f88740ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:07:44.057 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:44.054+0000 7f887c30a700 1 -- 172.21.15.188:0/2186941431 shutdown_connections 2024-05-10T11:07:44.057 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:44.054+0000 7f887c30a700 1 -- 172.21.15.188:0/2186941431 wait complete. 2024-05-10T11:07:44.057 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:44.055+0000 7f887c30a700 1 Processor -- start 2024-05-10T11:07:44.057 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:44.055+0000 7f887c30a700 1 -- start start 2024-05-10T11:07:44.058 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:44.055+0000 7f887c30a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8874101c00 0x7f88741a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:07:44.058 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:44.055+0000 7f887c30a700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f887410f4a0 con 0x7f8874101c00 2024-05-10T11:07:44.058 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:44.056+0000 7f887a0a6700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8874101c00 0x7f88741a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:07:44.058 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:44.056+0000 7f887a0a6700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8874101c00 0x7f88741a06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.188:46844/0 (socket says 172.21.15.188:46844) 2024-05-10T11:07:44.058 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:44.056+0000 7f887a0a6700 1 -- 172.21.15.188:0/29952076 learned_addr learned my addr 172.21.15.188:0/29952076 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-05-10T11:07:44.058 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:44.056+0000 7f887a0a6700 1 -- 172.21.15.188:0/29952076 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88741a0c10 con 0x7f8874101c00 2024-05-10T11:07:44.058 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:44.056+0000 7f887a0a6700 1 --2- 172.21.15.188:0/29952076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8874101c00 0x7f88741a06d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8870005660 tx=0x7f8870005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:07:44.058 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:44.056+0000 7f886b7fe700 1 -- 172.21.15.188:0/29952076 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8870026070 con 0x7f8874101c00 2024-05-10T11:07:44.058 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:44.056+0000 7f886b7fe700 1 -- 172.21.15.188:0/29952076 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f887000b750 con 0x7f8874101c00 2024-05-10T11:07:44.059 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:44.056+0000 7f887c30a700 1 -- 172.21.15.188:0/29952076 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88741a0ea0 con 0x7f8874101c00 2024-05-10T11:07:44.059 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:44.056+0000 7f886b7fe700 1 -- 172.21.15.188:0/29952076 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88700037b0 con 0x7f8874101c00 2024-05-10T11:07:44.059 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:44.056+0000 7f887c30a700 1 -- 172.21.15.188:0/29952076 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f887419a890 con 0x7f8874101c00 2024-05-10T11:07:44.060 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:44.057+0000 7f886b7fe700 1 -- 172.21.15.188:0/29952076 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f8870029070 con 0x7f8874101c00 2024-05-10T11:07:44.060 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:44.058+0000 7f886b7fe700 1 --2- 172.21.15.188:0/29952076 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f886003e510 0x7f88600409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:07:44.060 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:44.058+0000 7f886b7fe700 1 -- 172.21.15.188:0/29952076 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f887001e070 con 0x7f8874101c00 2024-05-10T11:07:44.060 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:44.058+0000 7f887c30a700 1 -- 172.21.15.188:0/29952076 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f887406ba40 con 0x7f8874101c00 2024-05-10T11:07:44.060 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:44.058+0000 7f88798a5700 1 --2- 172.21.15.188:0/29952076 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f886003e510 0x7f88600409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:07:44.060 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:44.058+0000 7f88798a5700 1 --2- 172.21.15.188:0/29952076 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f886003e510 0x7f88600409d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8864006fd0 tx=0x7f8864006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:07:44.064 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:44.063+0000 7f886b7fe700 1 -- 172.21.15.188:0/29952076 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f8870030020 con 0x7f8874101c00 2024-05-10T11:07:44.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:43 smithi100 ceph-mon[30296]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:07:44.376 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:44.375+0000 7f887c30a700 1 -- 172.21.15.188:0/29952076 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f887419bdc0 con 0x7f8874101c00 2024-05-10T11:07:44.376 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:44.376+0000 7f886b7fe700 1 -- 172.21.15.188:0/29952076 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+749 (secure 0 0 0) 0x7f8870013070 con 0x7f8874101c00 2024-05-10T11:07:44.376 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-10T11:07:44.376 INFO:teuthology.orchestra.run.smithi188.stdout:{"epoch":1,"fsid":"20a542cc-0ebd-11ef-bc98-c7b262605968","modified":"2024-05-10T11:06:01.369875Z","created":"2024-05-10T11:06:01.369875Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:3300","nonce":0},{"type":"v1","addr":"172.21.15.100:6789","nonce":0}]},"addr":"172.21.15.100:6789/0","public_addr":"172.21.15.100:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-10T11:07:44.379 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:44.379+0000 7f887c30a700 1 -- 172.21.15.188:0/29952076 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f886003e510 msgr2=0x7f88600409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:07:44.379 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:44.379+0000 7f887c30a700 1 --2- 172.21.15.188:0/29952076 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f886003e510 0x7f88600409d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8864006fd0 tx=0x7f8864006e40 comp rx=0 tx=0).stop 2024-05-10T11:07:44.379 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:44.379+0000 7f887c30a700 1 -- 172.21.15.188:0/29952076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8874101c00 msgr2=0x7f88741a06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:07:44.379 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:44.379+0000 7f887c30a700 1 --2- 172.21.15.188:0/29952076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8874101c00 0x7f88741a06d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8870005660 tx=0x7f8870005740 comp rx=0 tx=0).stop 2024-05-10T11:07:44.379 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:44.379+0000 7f887c30a700 1 -- 172.21.15.188:0/29952076 shutdown_connections 2024-05-10T11:07:44.379 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:44.379+0000 7f887c30a700 1 --2- 172.21.15.188:0/29952076 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f886003e510 0x7f88600409d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:07:44.379 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:44.379+0000 7f887c30a700 1 --2- 172.21.15.188:0/29952076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8874101c00 0x7f88741a06d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:07:44.379 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:44.379+0000 7f887c30a700 1 -- 172.21.15.188:0/29952076 >> 172.21.15.188:0/29952076 conn(0x7f88740fd8e0 msgr2=0x7f88740feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:07:44.380 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:44.379+0000 7f887c30a700 1 -- 172.21.15.188:0/29952076 shutdown_connections 2024-05-10T11:07:44.380 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:44.379+0000 7f887c30a700 1 -- 172.21.15.188:0/29952076 wait complete. 2024-05-10T11:07:44.380 INFO:teuthology.orchestra.run.smithi188.stderr:dumped monmap epoch 1 2024-05-10T11:07:44.507 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 podman[34042]: 2024-05-10 11:07:44.482721947 +0000 UTC m=+0.356682903 container start 7e0cb31e6fa7547ee407f9878ca96109d76a0d223e7dc02e5dcf6354bbd440bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mon-c, ceph=True, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0) 2024-05-10T11:07:44.507 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: set uid:gid to 167:167 (ceph:ceph) 2024-05-10T11:07:44.507 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: ceph version 19.0.0-3449-gd39fa088 (d39fa08867836ac070c47cb66c782fa559fd480b) squid (dev), process ceph-mon, pid 2 2024-05-10T11:07:44.507 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: pidfile_write: ignore empty --pid-file 2024-05-10T11:07:44.780 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: load: jerasure load: lrc 2024-05-10T11:07:44.780 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: RocksDB version: 7.9.2 2024-05-10T11:07:44.780 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Git sha 0 2024-05-10T11:07:44.780 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Compile date 2024-05-10 01:38:55 2024-05-10T11:07:44.780 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: DB SUMMARY 2024-05-10T11:07:44.780 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: DB Session ID: 1AZ098P4KANBQL9BMLI7 2024-05-10T11:07:44.780 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: CURRENT file: CURRENT 2024-05-10T11:07:44.780 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: IDENTITY file: IDENTITY 2024-05-10T11:07:44.781 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-05-10T11:07:44.781 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-05-10T11:07:44.781 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-05-10T11:07:44.781 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.error_if_exists: 0 2024-05-10T11:07:44.781 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.create_if_missing: 0 2024-05-10T11:07:44.781 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.paranoid_checks: 1 2024-05-10T11:07:44.781 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-10T11:07:44.781 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-10T11:07:44.782 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-10T11:07:44.782 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.env: 0x55e19aa14cc0 2024-05-10T11:07:44.782 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.fs: PosixFileSystem 2024-05-10T11:07:44.782 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.info_log: 0x55e19c075da0 2024-05-10T11:07:44.782 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.max_file_opening_threads: 16 2024-05-10T11:07:44.782 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.statistics: (nil) 2024-05-10T11:07:44.782 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.use_fsync: 0 2024-05-10T11:07:44.782 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.max_log_file_size: 0 2024-05-10T11:07:44.782 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-10T11:07:44.782 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-10T11:07:44.782 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.keep_log_file_num: 1000 2024-05-10T11:07:44.783 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.recycle_log_file_num: 0 2024-05-10T11:07:44.783 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.allow_fallocate: 1 2024-05-10T11:07:44.783 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.allow_mmap_reads: 0 2024-05-10T11:07:44.783 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.allow_mmap_writes: 0 2024-05-10T11:07:44.783 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.use_direct_reads: 0 2024-05-10T11:07:44.783 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-10T11:07:44.783 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.create_missing_column_families: 0 2024-05-10T11:07:44.783 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.db_log_dir: 2024-05-10T11:07:44.784 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.wal_dir: 2024-05-10T11:07:44.784 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-10T11:07:44.784 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-10T11:07:44.784 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-10T11:07:44.784 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-10T11:07:44.784 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-10T11:07:44.784 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-10T11:07:44.784 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.advise_random_on_open: 1 2024-05-10T11:07:44.784 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.db_write_buffer_size: 0 2024-05-10T11:07:44.785 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.write_buffer_manager: 0x55e19c0470e0 2024-05-10T11:07:44.785 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-10T11:07:44.785 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-10T11:07:44.785 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-10T11:07:44.785 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.rate_limiter: (nil) 2024-05-10T11:07:44.785 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-10T11:07:44.785 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.wal_recovery_mode: 2 2024-05-10T11:07:44.785 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.enable_thread_tracking: 0 2024-05-10T11:07:44.786 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.enable_pipelined_write: 0 2024-05-10T11:07:44.786 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.unordered_write: 0 2024-05-10T11:07:44.786 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-10T11:07:44.786 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-10T11:07:44.786 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-10T11:07:44.786 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-10T11:07:44.786 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.row_cache: None 2024-05-10T11:07:44.786 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.wal_filter: None 2024-05-10T11:07:44.786 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-10T11:07:44.786 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.allow_ingest_behind: 0 2024-05-10T11:07:44.787 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.two_write_queues: 0 2024-05-10T11:07:44.787 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.manual_wal_flush: 0 2024-05-10T11:07:44.787 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.wal_compression: 0 2024-05-10T11:07:44.787 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.atomic_flush: 0 2024-05-10T11:07:44.787 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-10T11:07:44.787 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-10T11:07:44.787 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-10T11:07:44.787 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.log_readahead_size: 0 2024-05-10T11:07:44.787 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-10T11:07:44.787 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.best_efforts_recovery: 0 2024-05-10T11:07:44.787 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-10T11:07:44.788 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-10T11:07:44.788 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.allow_data_in_errors: 0 2024-05-10T11:07:44.788 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.db_host_id: __hostname__ 2024-05-10T11:07:44.788 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-10T11:07:44.788 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.max_background_jobs: 2 2024-05-10T11:07:44.788 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.max_background_compactions: -1 2024-05-10T11:07:44.788 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.max_subcompactions: 1 2024-05-10T11:07:44.788 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-10T11:07:44.788 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-10T11:07:44.789 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-10T11:07:44.789 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.max_total_wal_size: 0 2024-05-10T11:07:44.789 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-10T11:07:44.789 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-10T11:07:44.789 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-10T11:07:44.789 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-10T11:07:44.789 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.max_open_files: -1 2024-05-10T11:07:44.790 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.bytes_per_sync: 0 2024-05-10T11:07:44.790 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-10T11:07:44.790 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-10T11:07:44.790 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.compaction_readahead_size: 0 2024-05-10T11:07:44.790 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.max_background_flushes: -1 2024-05-10T11:07:44.790 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Compression algorithms supported: 2024-05-10T11:07:44.790 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-10T11:07:44.790 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: kZSTD supported: 0 2024-05-10T11:07:44.791 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: kXpressCompression supported: 0 2024-05-10T11:07:44.791 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: kLZ4HCCompression supported: 1 2024-05-10T11:07:44.791 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: kZlibCompression supported: 1 2024-05-10T11:07:44.791 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: kSnappyCompression supported: 1 2024-05-10T11:07:44.791 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: kLZ4Compression supported: 1 2024-05-10T11:07:44.791 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: kBZip2Compression supported: 0 2024-05-10T11:07:44.791 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-10T11:07:44.791 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-10T11:07:44.791 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-05-10T11:07:44.792 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-10T11:07:44.792 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-10T11:07:44.792 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.merge_operator: 2024-05-10T11:07:44.792 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.compaction_filter: None 2024-05-10T11:07:44.792 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.compaction_filter_factory: None 2024-05-10T11:07:44.792 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.sst_partitioner_factory: None 2024-05-10T11:07:44.792 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-10T11:07:44.792 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-10T11:07:44.792 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55e19c075f60) 2024-05-10T11:07:44.792 INFO:journalctl@ceph.mon.c.smithi188.stdout: cache_index_and_filter_blocks: 1 2024-05-10T11:07:44.792 INFO:journalctl@ceph.mon.c.smithi188.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-10T11:07:44.792 INFO:journalctl@ceph.mon.c.smithi188.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-10T11:07:44.793 INFO:journalctl@ceph.mon.c.smithi188.stdout: pin_top_level_index_and_filter: 1 2024-05-10T11:07:44.793 INFO:journalctl@ceph.mon.c.smithi188.stdout: index_type: 0 2024-05-10T11:07:44.793 INFO:journalctl@ceph.mon.c.smithi188.stdout: data_block_index_type: 0 2024-05-10T11:07:44.793 INFO:journalctl@ceph.mon.c.smithi188.stdout: index_shortening: 1 2024-05-10T11:07:44.793 INFO:journalctl@ceph.mon.c.smithi188.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-10T11:07:44.793 INFO:journalctl@ceph.mon.c.smithi188.stdout: checksum: 4 2024-05-10T11:07:44.793 INFO:journalctl@ceph.mon.c.smithi188.stdout: no_block_cache: 0 2024-05-10T11:07:44.793 INFO:journalctl@ceph.mon.c.smithi188.stdout: block_cache: 0x55e19c0791f0 2024-05-10T11:07:44.793 INFO:journalctl@ceph.mon.c.smithi188.stdout: block_cache_name: BinnedLRUCache 2024-05-10T11:07:44.793 INFO:journalctl@ceph.mon.c.smithi188.stdout: block_cache_options: 2024-05-10T11:07:44.793 INFO:journalctl@ceph.mon.c.smithi188.stdout: capacity : 536870912 2024-05-10T11:07:44.794 INFO:journalctl@ceph.mon.c.smithi188.stdout: num_shard_bits : 4 2024-05-10T11:07:44.794 INFO:journalctl@ceph.mon.c.smithi188.stdout: strict_capacity_limit : 0 2024-05-10T11:07:44.794 INFO:journalctl@ceph.mon.c.smithi188.stdout: high_pri_pool_ratio: 0.000 2024-05-10T11:07:44.794 INFO:journalctl@ceph.mon.c.smithi188.stdout: block_cache_compressed: (nil) 2024-05-10T11:07:44.794 INFO:journalctl@ceph.mon.c.smithi188.stdout: persistent_cache: (nil) 2024-05-10T11:07:44.794 INFO:journalctl@ceph.mon.c.smithi188.stdout: block_size: 4096 2024-05-10T11:07:44.794 INFO:journalctl@ceph.mon.c.smithi188.stdout: block_size_deviation: 10 2024-05-10T11:07:44.794 INFO:journalctl@ceph.mon.c.smithi188.stdout: block_restart_interval: 16 2024-05-10T11:07:44.794 INFO:journalctl@ceph.mon.c.smithi188.stdout: index_block_restart_interval: 1 2024-05-10T11:07:44.794 INFO:journalctl@ceph.mon.c.smithi188.stdout: metadata_block_size: 4096 2024-05-10T11:07:44.794 INFO:journalctl@ceph.mon.c.smithi188.stdout: partition_filters: 0 2024-05-10T11:07:44.794 INFO:journalctl@ceph.mon.c.smithi188.stdout: use_delta_encoding: 1 2024-05-10T11:07:44.795 INFO:journalctl@ceph.mon.c.smithi188.stdout: filter_policy: bloomfilter 2024-05-10T11:07:44.795 INFO:journalctl@ceph.mon.c.smithi188.stdout: whole_key_filtering: 1 2024-05-10T11:07:44.795 INFO:journalctl@ceph.mon.c.smithi188.stdout: verify_compression: 0 2024-05-10T11:07:44.795 INFO:journalctl@ceph.mon.c.smithi188.stdout: read_amp_bytes_per_bit: 0 2024-05-10T11:07:44.795 INFO:journalctl@ceph.mon.c.smithi188.stdout: format_version: 5 2024-05-10T11:07:44.795 INFO:journalctl@ceph.mon.c.smithi188.stdout: enable_index_compression: 1 2024-05-10T11:07:44.795 INFO:journalctl@ceph.mon.c.smithi188.stdout: block_align: 0 2024-05-10T11:07:44.795 INFO:journalctl@ceph.mon.c.smithi188.stdout: max_auto_readahead_size: 262144 2024-05-10T11:07:44.795 INFO:journalctl@ceph.mon.c.smithi188.stdout: prepopulate_block_cache: 0 2024-05-10T11:07:44.795 INFO:journalctl@ceph.mon.c.smithi188.stdout: initial_auto_readahead_size: 8192 2024-05-10T11:07:44.795 INFO:journalctl@ceph.mon.c.smithi188.stdout: num_file_reads_for_auto_readahead: 2 2024-05-10T11:07:44.795 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.write_buffer_size: 33554432 2024-05-10T11:07:44.796 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.max_write_buffer_number: 2 2024-05-10T11:07:44.796 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.compression: NoCompression 2024-05-10T11:07:44.796 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.bottommost_compression: Disabled 2024-05-10T11:07:44.796 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.prefix_extractor: nullptr 2024-05-10T11:07:44.796 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-10T11:07:44.796 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.num_levels: 7 2024-05-10T11:07:44.796 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-10T11:07:44.796 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-10T11:07:44.796 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-10T11:07:44.796 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-10T11:07:44.796 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-10T11:07:44.796 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-10T11:07:44.796 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-10T11:07:44.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-10T11:07:44.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-10T11:07:44.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-10T11:07:44.797 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-10T11:07:44.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-10T11:07:44.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-10T11:07:44.798 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.compression_opts.level: 32767 2024-05-10T11:07:44.799 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.compression_opts.strategy: 0 2024-05-10T11:07:44.799 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-10T11:07:44.799 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-10T11:07:44.799 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-10T11:07:44.799 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-10T11:07:44.799 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.compression_opts.enabled: false 2024-05-10T11:07:44.799 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-10T11:07:44.799 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-10T11:07:44.799 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-10T11:07:44.799 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-10T11:07:44.799 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.target_file_size_base: 67108864 2024-05-10T11:07:44.799 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-10T11:07:44.800 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-10T11:07:44.800 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-10T11:07:44.800 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-10T11:07:44.800 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-10T11:07:44.800 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-10T11:07:44.800 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-10T11:07:44.800 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-10T11:07:44.800 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-10T11:07:44.800 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-10T11:07:44.800 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-10T11:07:44.800 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-10T11:07:44.800 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-10T11:07:44.801 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-10T11:07:44.801 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.arena_block_size: 1048576 2024-05-10T11:07:44.801 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-10T11:07:44.801 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-10T11:07:44.801 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.disable_auto_compactions: 0 2024-05-10T11:07:44.801 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-10T11:07:44.801 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-10T11:07:44.801 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-10T11:07:44.801 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-10T11:07:44.801 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-10T11:07:44.801 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-10T11:07:44.801 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-10T11:07:44.801 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-10T11:07:44.802 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-10T11:07:44.802 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-10T11:07:44.802 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-10T11:07:44.802 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.inplace_update_support: 0 2024-05-10T11:07:44.802 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-10T11:07:44.802 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-10T11:07:44.802 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-10T11:07:44.802 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-10T11:07:44.802 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.bloom_locality: 0 2024-05-10T11:07:44.802 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.max_successive_merges: 0 2024-05-10T11:07:44.802 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-10T11:07:44.802 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.paranoid_file_checks: 0 2024-05-10T11:07:44.803 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.force_consistency_checks: 1 2024-05-10T11:07:44.803 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.report_bg_io_stats: 0 2024-05-10T11:07:44.803 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.ttl: 2592000 2024-05-10T11:07:44.803 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-10T11:07:44.803 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-10T11:07:44.803 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-10T11:07:44.803 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.enable_blob_files: false 2024-05-10T11:07:44.803 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.min_blob_size: 0 2024-05-10T11:07:44.803 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.blob_file_size: 268435456 2024-05-10T11:07:44.803 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-10T11:07:44.803 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-10T11:07:44.803 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-10T11:07:44.803 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-10T11:07:44.804 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-10T11:07:44.804 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.blob_file_starting_level: 0 2024-05-10T11:07:44.804 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-10T11:07:44.804 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-10T11:07:44.804 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-05-10T11:07:44.804 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 5d516075-f325-4db1-811f-58a8b2e23d09 2024-05-10T11:07:44.804 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715339264513304, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-10T11:07:44.804 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-05-10T11:07:44.804 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715339264513710, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715339264, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "5d516075-f325-4db1-811f-58a8b2e23d09", "db_session_id": "1AZ098P4KANBQL9BMLI7", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-05-10T11:07:44.804 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715339264513787, "job": 1, "event": "recovery_finished"} 2024-05-10T11:07:44.804 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-05-10T11:07:44.804 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-10T11:07:44.804 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55e19c196000 2024-05-10T11:07:44.805 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: DB pointer 0x55e19c182000 2024-05-10T11:07:44.805 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-10T11:07:44.805 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-10T11:07:44.805 INFO:journalctl@ceph.mon.c.smithi188.stdout: ** DB Stats ** 2024-05-10T11:07:44.805 INFO:journalctl@ceph.mon.c.smithi188.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-10T11:07:44.805 INFO:journalctl@ceph.mon.c.smithi188.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-10T11:07:44.805 INFO:journalctl@ceph.mon.c.smithi188.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-10T11:07:44.805 INFO:journalctl@ceph.mon.c.smithi188.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-10T11:07:44.805 INFO:journalctl@ceph.mon.c.smithi188.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-10T11:07:44.805 INFO:journalctl@ceph.mon.c.smithi188.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-10T11:07:44.805 INFO:journalctl@ceph.mon.c.smithi188.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-10T11:07:44.806 INFO:journalctl@ceph.mon.c.smithi188.stdout: 2024-05-10T11:07:44.806 INFO:journalctl@ceph.mon.c.smithi188.stdout: ** Compaction Stats [default] ** 2024-05-10T11:07:44.806 INFO:journalctl@ceph.mon.c.smithi188.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-10T11:07:44.806 INFO:journalctl@ceph.mon.c.smithi188.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-10T11:07:44.806 INFO:journalctl@ceph.mon.c.smithi188.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-10T11:07:44.806 INFO:journalctl@ceph.mon.c.smithi188.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-10T11:07:44.806 INFO:journalctl@ceph.mon.c.smithi188.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-10T11:07:44.806 INFO:journalctl@ceph.mon.c.smithi188.stdout: 2024-05-10T11:07:44.806 INFO:journalctl@ceph.mon.c.smithi188.stdout: ** Compaction Stats [default] ** 2024-05-10T11:07:44.806 INFO:journalctl@ceph.mon.c.smithi188.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-10T11:07:44.806 INFO:journalctl@ceph.mon.c.smithi188.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-10T11:07:44.806 INFO:journalctl@ceph.mon.c.smithi188.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-10T11:07:44.807 INFO:journalctl@ceph.mon.c.smithi188.stdout: 2024-05-10T11:07:44.807 INFO:journalctl@ceph.mon.c.smithi188.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-10T11:07:44.807 INFO:journalctl@ceph.mon.c.smithi188.stdout: 2024-05-10T11:07:44.807 INFO:journalctl@ceph.mon.c.smithi188.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-10T11:07:44.807 INFO:journalctl@ceph.mon.c.smithi188.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-10T11:07:44.807 INFO:journalctl@ceph.mon.c.smithi188.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-10T11:07:44.807 INFO:journalctl@ceph.mon.c.smithi188.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-10T11:07:44.807 INFO:journalctl@ceph.mon.c.smithi188.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-10T11:07:44.807 INFO:journalctl@ceph.mon.c.smithi188.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-10T11:07:44.807 INFO:journalctl@ceph.mon.c.smithi188.stdout: Cumulative compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-10T11:07:44.807 INFO:journalctl@ceph.mon.c.smithi188.stdout: Interval compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-10T11:07:44.807 INFO:journalctl@ceph.mon.c.smithi188.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-10T11:07:44.807 INFO:journalctl@ceph.mon.c.smithi188.stdout: Block cache BinnedLRUCache@0x55e19c0791f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-05-10T11:07:44.808 INFO:journalctl@ceph.mon.c.smithi188.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-10T11:07:44.808 INFO:journalctl@ceph.mon.c.smithi188.stdout: 2024-05-10T11:07:44.808 INFO:journalctl@ceph.mon.c.smithi188.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-10T11:07:44.808 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-05-10T11:07:44.808 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: using public_addr v2:172.21.15.188:0/0 -> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] 2024-05-10T11:07:44.808 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: starting mon.c rank -1 at public addrs [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] at bind addrs [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 2024-05-10T11:07:44.808 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: mon.c@-1(???) e0 preinit fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 2024-05-10T11:07:44.808 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: mon.c@-1(synchronizing).mds e1 new map 2024-05-10T11:07:44.808 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: mon.c@-1(synchronizing).mds e1 print_map 2024-05-10T11:07:44.808 INFO:journalctl@ceph.mon.c.smithi188.stdout: e1 2024-05-10T11:07:44.808 INFO:journalctl@ceph.mon.c.smithi188.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-10T11:07:44.809 INFO:journalctl@ceph.mon.c.smithi188.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-10T11:07:44.809 INFO:journalctl@ceph.mon.c.smithi188.stdout: legacy client fscid: -1 2024-05-10T11:07:44.809 INFO:journalctl@ceph.mon.c.smithi188.stdout: 2024-05-10T11:07:44.809 INFO:journalctl@ceph.mon.c.smithi188.stdout: No filesystems configured 2024-05-10T11:07:44.809 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-10T11:07:44.809 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-10T11:07:44.809 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-10T11:07:44.809 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-10T11:07:44.809 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-10T11:07:44.809 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-10T11:07:44.809 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-10T11:07:44.809 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-10T11:07:44.810 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-10T11:07:44.810 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-10T11:07:44.810 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:07:44.810 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:44.810 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T11:07:44.810 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:07:44.810 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:44.810 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:44.810 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:44.810 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:44.810 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi178", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:07:44.810 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:07:44.811 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:07:44.811 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi188", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:07:44.811 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: Updating smithi178:/etc/ceph/ceph.conf 2024-05-10T11:07:44.811 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: Updating smithi178:/var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T11:07:44.811 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:07:44.811 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: Deploying cephadm binary to smithi188 2024-05-10T11:07:44.811 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: Updating smithi178:/etc/ceph/ceph.client.admin.keyring 2024-05-10T11:07:44.811 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: Updating smithi178:/var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-10T11:07:44.811 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:07:44.811 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:44.811 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:44.811 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:44.812 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:44.812 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:07:44.812 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: Added host smithi188 2024-05-10T11:07:44.812 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:44.812 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:44.812 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:07:44.812 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:07:44.812 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:44.812 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T11:07:44.812 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:07:44.812 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/3754236342' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-10T11:07:44.812 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/3754236342' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-10T11:07:44.813 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: osdmap e4: 0 total, 0 up, 0 in 2024-05-10T11:07:44.813 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:44.813 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:44.813 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:44.813 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:44.813 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi188", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:07:44.813 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:07:44.813 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:07:44.813 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: Updating smithi188:/etc/ceph/ceph.conf 2024-05-10T11:07:44.813 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: Updating smithi188:/var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T11:07:44.813 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: Updating smithi188:/etc/ceph/ceph.client.admin.keyring 2024-05-10T11:07:44.813 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:07:44.814 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: Updating smithi188:/var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-10T11:07:44.814 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:44.814 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:44.814 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:44.814 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi100:172.21.15.100=a;smithi178:172.21.15.178=b;smithi188:172.21.15.188=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:07:44.814 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: Saving service mon spec with placement smithi100:172.21.15.100=a;smithi178:172.21.15.178=b;smithi188:172.21.15.188=c;count:3 2024-05-10T11:07:44.814 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:44.814 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:07:44.814 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:07:44.815 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:07:44.815 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:44.815 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T11:07:44.815 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:07:44.815 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: Deploying daemon mon.c on smithi188 2024-05-10T11:07:44.815 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:07:44.815 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:07:44.815 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 ceph-mon[34075]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-10T11:07:45.066 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 bash[34042]: 7e0cb31e6fa7547ee407f9878ca96109d76a0d223e7dc02e5dcf6354bbd440bd 2024-05-10T11:07:45.066 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:44 smithi188 systemd[1]: Started Ceph mon.c for 20a542cc-0ebd-11ef-bc98-c7b262605968. 2024-05-10T11:07:46.250 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-10T11:07:46.251 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-10T11:07:46.495 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T11:07:50.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:49 smithi188 ceph-mon[34075]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:07:50.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:49 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T11:07:50.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:49 smithi188 ceph-mon[34075]: mon.a calling monitor election 2024-05-10T11:07:50.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:49 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T11:07:50.030 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:49 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T11:07:50.030 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:49 smithi188 ceph-mon[34075]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:07:50.030 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:49 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T11:07:50.030 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:49 smithi188 ceph-mon[34075]: mon.c calling monitor election 2024-05-10T11:07:50.030 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:49 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T11:07:50.030 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:49 smithi188 ceph-mon[34075]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:07:50.030 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:49 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T11:07:50.030 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:49 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T11:07:50.030 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:49 smithi188 ceph-mon[34075]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-10T11:07:50.030 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:49 smithi188 ceph-mon[34075]: monmap epoch 2 2024-05-10T11:07:50.030 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:49 smithi188 ceph-mon[34075]: fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 2024-05-10T11:07:50.030 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:49 smithi188 ceph-mon[34075]: last_changed 2024-05-10T11:07:44.536696+0000 2024-05-10T11:07:50.030 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:49 smithi188 ceph-mon[34075]: created 2024-05-10T11:06:01.369875+0000 2024-05-10T11:07:50.030 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:49 smithi188 ceph-mon[34075]: min_mon_release 19 (squid) 2024-05-10T11:07:50.031 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:49 smithi188 ceph-mon[34075]: election_strategy: 1 2024-05-10T11:07:50.031 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:49 smithi188 ceph-mon[34075]: 0: [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] mon.a 2024-05-10T11:07:50.031 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:49 smithi188 ceph-mon[34075]: 1: [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] mon.c 2024-05-10T11:07:50.031 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:49 smithi188 ceph-mon[34075]: fsmap 2024-05-10T11:07:50.031 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:49 smithi188 ceph-mon[34075]: osdmap e4: 0 total, 0 up, 0 in 2024-05-10T11:07:50.031 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:49 smithi188 ceph-mon[34075]: mgrmap e13: x(active, since 49s) 2024-05-10T11:07:50.031 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:49 smithi188 ceph-mon[34075]: overall HEALTH_OK 2024-05-10T11:07:50.031 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:49 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:50.031 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:49 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:50.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:49 smithi100 ceph-mon[30296]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:07:50.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:49 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T11:07:50.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:49 smithi100 ceph-mon[30296]: mon.a calling monitor election 2024-05-10T11:07:50.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:49 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T11:07:50.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:49 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T11:07:50.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:49 smithi100 ceph-mon[30296]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:07:50.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:49 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T11:07:50.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:49 smithi100 ceph-mon[30296]: mon.c calling monitor election 2024-05-10T11:07:50.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:49 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T11:07:50.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:49 smithi100 ceph-mon[30296]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:07:50.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:49 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T11:07:50.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:49 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T11:07:50.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:49 smithi100 ceph-mon[30296]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-10T11:07:50.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:49 smithi100 ceph-mon[30296]: monmap epoch 2 2024-05-10T11:07:50.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:49 smithi100 ceph-mon[30296]: fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 2024-05-10T11:07:50.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:49 smithi100 ceph-mon[30296]: last_changed 2024-05-10T11:07:44.536696+0000 2024-05-10T11:07:50.040 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:49 smithi100 ceph-mon[30296]: created 2024-05-10T11:06:01.369875+0000 2024-05-10T11:07:50.040 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:49 smithi100 ceph-mon[30296]: min_mon_release 19 (squid) 2024-05-10T11:07:50.040 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:49 smithi100 ceph-mon[30296]: election_strategy: 1 2024-05-10T11:07:50.040 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:49 smithi100 ceph-mon[30296]: 0: [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] mon.a 2024-05-10T11:07:50.040 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:49 smithi100 ceph-mon[30296]: 1: [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] mon.c 2024-05-10T11:07:50.040 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:49 smithi100 ceph-mon[30296]: fsmap 2024-05-10T11:07:50.040 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:49 smithi100 ceph-mon[30296]: osdmap e4: 0 total, 0 up, 0 in 2024-05-10T11:07:50.040 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:49 smithi100 ceph-mon[30296]: mgrmap e13: x(active, since 49s) 2024-05-10T11:07:50.040 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:49 smithi100 ceph-mon[30296]: overall HEALTH_OK 2024-05-10T11:07:50.040 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:49 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:50.040 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:49 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:50.819 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:50.818+0000 7fb663a63700 1 -- 172.21.15.188:0/1240786491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6480035c0 msgr2=0x7fb648005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:07:50.819 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:50.818+0000 7fb663a63700 1 --2- 172.21.15.188:0/1240786491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6480035c0 0x7fb648005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb65c102f20 tx=0x7fb65800feb0 comp rx=0 tx=0).stop 2024-05-10T11:07:50.819 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:50.818+0000 7fb663a63700 1 -- 172.21.15.188:0/1240786491 shutdown_connections 2024-05-10T11:07:50.820 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:50.818+0000 7fb663a63700 1 --2- 172.21.15.188:0/1240786491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6480035c0 0x7fb648005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:07:50.820 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:50.818+0000 7fb663a63700 1 -- 172.21.15.188:0/1240786491 >> 172.21.15.188:0/1240786491 conn(0x7fb65c0feae0 msgr2=0x7fb65c100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:07:50.820 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:50.818+0000 7fb663a63700 1 -- 172.21.15.188:0/1240786491 shutdown_connections 2024-05-10T11:07:50.820 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:50.818+0000 7fb663a63700 1 -- 172.21.15.188:0/1240786491 wait complete. 2024-05-10T11:07:50.820 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:50.819+0000 7fb663a63700 1 Processor -- start 2024-05-10T11:07:50.820 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:50.819+0000 7fb663a63700 1 -- start start 2024-05-10T11:07:50.821 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:50.820+0000 7fb663a63700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6480035c0 0x7fb65c19e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:07:50.821 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:50.820+0000 7fb663a63700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb65c19ebd0 0x7fb65c1988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:07:50.822 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:50.820+0000 7fb663a63700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb65c10adf0 con 0x7fb6480035c0 2024-05-10T11:07:50.822 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:50.820+0000 7fb663a63700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb65c10ac70 con 0x7fb65c19ebd0 2024-05-10T11:07:50.822 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:50.820+0000 7fb6617ff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6480035c0 0x7fb65c19e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:07:50.822 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:50.820+0000 7fb660ffe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb65c19ebd0 0x7fb65c1988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:07:50.822 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:50.820+0000 7fb6617ff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6480035c0 0x7fb65c19e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.188:52042/0 (socket says 172.21.15.188:52042) 2024-05-10T11:07:50.822 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:50.820+0000 7fb6617ff700 1 -- 172.21.15.188:0/1709588222 learned_addr learned my addr 172.21.15.188:0/1709588222 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-05-10T11:07:50.822 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:50.820+0000 7fb660ffe700 1 -- 172.21.15.188:0/1709588222 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb65c19ebd0 msgr2=0x7fb65c1988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-10T11:07:50.822 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:50.820+0000 7fb660ffe700 1 -- 172.21.15.188:0/1709588222 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb65c19ebd0 msgr2=0x7fb65c1988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-10T11:07:50.822 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:50.820+0000 7fb660ffe700 1 --2- 172.21.15.188:0/1709588222 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb65c19ebd0 0x7fb65c1988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-10T11:07:50.822 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:50.820+0000 7fb660ffe700 1 --2- 172.21.15.188:0/1709588222 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb65c19ebd0 0x7fb65c1988b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-10T11:07:50.822 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:50.820+0000 7fb6617ff700 1 -- 172.21.15.188:0/1709588222 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb65c19ebd0 msgr2=0x7fb65c1988b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T11:07:50.822 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:50.820+0000 7fb6617ff700 1 --2- 172.21.15.188:0/1709588222 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb65c19ebd0 0x7fb65c1988b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:07:50.822 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:50.820+0000 7fb6617ff700 1 -- 172.21.15.188:0/1709588222 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb65c19f210 con 0x7fb6480035c0 2024-05-10T11:07:50.823 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:50.821+0000 7fb6617ff700 1 --2- 172.21.15.188:0/1709588222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6480035c0 0x7fb65c19e690 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb658005b40 tx=0x7fb6580052a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:07:50.823 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:50.821+0000 7fb6527fc700 1 -- 172.21.15.188:0/1709588222 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fb658013b00 con 0x7fb6480035c0 2024-05-10T11:07:50.823 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:50.821+0000 7fb6527fc700 1 -- 172.21.15.188:0/1709588222 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb6580044c0 con 0x7fb6480035c0 2024-05-10T11:07:50.823 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:50.821+0000 7fb6527fc700 1 -- 172.21.15.188:0/1709588222 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fb658003c60 con 0x7fb6480035c0 2024-05-10T11:07:50.823 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:50.821+0000 7fb663a63700 1 -- 172.21.15.188:0/1709588222 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb65c198e50 con 0x7fb6480035c0 2024-05-10T11:07:50.823 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:50.821+0000 7fb663a63700 1 -- 172.21.15.188:0/1709588222 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb65c1993b0 con 0x7fb6480035c0 2024-05-10T11:07:50.824 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:50.822+0000 7fb6527fc700 1 -- 172.21.15.188:0/1709588222 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7fb658023070 con 0x7fb6480035c0 2024-05-10T11:07:50.824 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:50.822+0000 7fb663a63700 1 -- 172.21.15.188:0/1709588222 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb65c104730 con 0x7fb6480035c0 2024-05-10T11:07:50.824 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:50.822+0000 7fb6527fc700 1 --2- 172.21.15.188:0/1709588222 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fb64003e300 0x7fb6400407c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:07:50.824 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:50.822+0000 7fb6527fc700 1 -- 172.21.15.188:0/1709588222 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb658020050 con 0x7fb6480035c0 2024-05-10T11:07:50.824 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:50.822+0000 7fb660ffe700 1 --2- 172.21.15.188:0/1709588222 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fb64003e300 0x7fb6400407c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:07:50.824 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:50.823+0000 7fb660ffe700 1 --2- 172.21.15.188:0/1709588222 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fb64003e300 0x7fb6400407c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb64c009850 tx=0x7fb64c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:07:50.827 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:50.828+0000 7fb6527fc700 1 -- 172.21.15.188:0/1709588222 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fb658028020 con 0x7fb6480035c0 2024-05-10T11:07:51.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:50 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:51.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:50 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T11:07:51.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:50 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:07:51.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:50 smithi188 ceph-mon[34075]: Deploying daemon mon.b on smithi178 2024-05-10T11:07:51.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:50 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T11:07:51.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:50 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:51.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:50 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T11:07:51.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:50 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:07:51.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:50 smithi100 ceph-mon[30296]: Deploying daemon mon.b on smithi178 2024-05-10T11:07:51.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:50 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T11:07:51.116 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:51.116+0000 7fb663a63700 1 -- 172.21.15.188:0/1709588222 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fb65c1996c0 con 0x7fb6480035c0 2024-05-10T11:07:51.117 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:51.117+0000 7fb6527fc700 1 -- 172.21.15.188:0/1709588222 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7fb6580103d0 con 0x7fb6480035c0 2024-05-10T11:07:51.117 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-10T11:07:51.117 INFO:teuthology.orchestra.run.smithi188.stdout:{"epoch":2,"fsid":"20a542cc-0ebd-11ef-bc98-c7b262605968","modified":"2024-05-10T11:07:44.536696Z","created":"2024-05-10T11:06:01.369875Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:3300","nonce":0},{"type":"v1","addr":"172.21.15.100:6789","nonce":0}]},"addr":"172.21.15.100:6789/0","public_addr":"172.21.15.100:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:3300","nonce":0},{"type":"v1","addr":"172.21.15.188:6789","nonce":0}]},"addr":"172.21.15.188:6789/0","public_addr":"172.21.15.188:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-10T11:07:51.119 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:51.119+0000 7fb663a63700 1 -- 172.21.15.188:0/1709588222 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fb64003e300 msgr2=0x7fb6400407c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:07:51.119 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:51.119+0000 7fb663a63700 1 --2- 172.21.15.188:0/1709588222 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fb64003e300 0x7fb6400407c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb64c009850 tx=0x7fb64c008040 comp rx=0 tx=0).stop 2024-05-10T11:07:51.119 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:51.119+0000 7fb663a63700 1 -- 172.21.15.188:0/1709588222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6480035c0 msgr2=0x7fb65c19e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:07:51.119 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:51.119+0000 7fb663a63700 1 --2- 172.21.15.188:0/1709588222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6480035c0 0x7fb65c19e690 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb658005b40 tx=0x7fb6580052a0 comp rx=0 tx=0).stop 2024-05-10T11:07:51.120 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:51.119+0000 7fb663a63700 1 -- 172.21.15.188:0/1709588222 shutdown_connections 2024-05-10T11:07:51.120 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:51.119+0000 7fb663a63700 1 --2- 172.21.15.188:0/1709588222 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fb64003e300 0x7fb6400407c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:07:51.120 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:51.119+0000 7fb663a63700 1 --2- 172.21.15.188:0/1709588222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6480035c0 0x7fb65c19e690 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:07:51.120 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:51.119+0000 7fb663a63700 1 --2- 172.21.15.188:0/1709588222 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb65c19ebd0 0x7fb65c1988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:07:51.120 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:51.119+0000 7fb663a63700 1 -- 172.21.15.188:0/1709588222 >> 172.21.15.188:0/1709588222 conn(0x7fb65c0feae0 msgr2=0x7fb65c106c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:07:51.120 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:51.119+0000 7fb663a63700 1 -- 172.21.15.188:0/1709588222 shutdown_connections 2024-05-10T11:07:51.120 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:51.119+0000 7fb663a63700 1 -- 172.21.15.188:0/1709588222 wait complete. 2024-05-10T11:07:51.120 INFO:teuthology.orchestra.run.smithi188.stderr:dumped monmap epoch 2 2024-05-10T11:07:51.744 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:51 smithi188 ceph-mon[34075]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:07:51.744 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:51 smithi188 ceph-mon[34075]: from='client.? 172.21.15.188:0/1709588222' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-10T11:07:52.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:51 smithi100 ceph-mon[30296]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:07:52.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:51 smithi100 ceph-mon[30296]: from='client.? 172.21.15.188:0/1709588222' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-10T11:07:52.746 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-10T11:07:52.746 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-10T11:07:52.989 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T11:07:53.287 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 podman[33340]: 2024-05-10 11:07:53.287204437 +0000 UTC m=+0.119431531 container create 2b35ae6b2dd96afb36329937240d31f0539cfd9ad405d794e48e78e77203acc8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mon-b, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240507, ceph=True, GIT_BRANCH=HEAD) 2024-05-10T11:07:53.537 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 podman[33340]: 2024-05-10 11:07:53.506106304 +0000 UTC m=+0.338333401 container init 2b35ae6b2dd96afb36329937240d31f0539cfd9ad405d794e48e78e77203acc8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mon-b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-10T11:07:53.537 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 podman[33340]: 2024-05-10 11:07:53.509092961 +0000 UTC m=+0.341320064 container start 2b35ae6b2dd96afb36329937240d31f0539cfd9ad405d794e48e78e77203acc8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mon-b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, maintainer=Guillaume Abrioux , RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-05-10T11:07:53.537 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: set uid:gid to 167:167 (ceph:ceph) 2024-05-10T11:07:53.537 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: ceph version 19.0.0-3449-gd39fa088 (d39fa08867836ac070c47cb66c782fa559fd480b) squid (dev), process ceph-mon, pid 2 2024-05-10T11:07:53.537 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: pidfile_write: ignore empty --pid-file 2024-05-10T11:07:53.537 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: load: jerasure load: lrc 2024-05-10T11:07:54.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: RocksDB version: 7.9.2 2024-05-10T11:07:54.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Git sha 0 2024-05-10T11:07:54.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Compile date 2024-05-10 01:38:55 2024-05-10T11:07:54.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: DB SUMMARY 2024-05-10T11:07:54.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: DB Session ID: UF6X0DTPGU1K8HGTMML2 2024-05-10T11:07:54.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: CURRENT file: CURRENT 2024-05-10T11:07:54.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: IDENTITY file: IDENTITY 2024-05-10T11:07:54.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-05-10T11:07:54.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-05-10T11:07:54.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-05-10T11:07:54.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.error_if_exists: 0 2024-05-10T11:07:54.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.create_if_missing: 0 2024-05-10T11:07:54.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.paranoid_checks: 1 2024-05-10T11:07:54.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-10T11:07:54.004 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-10T11:07:54.004 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-10T11:07:54.004 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.env: 0x5623349d6cc0 2024-05-10T11:07:54.004 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.fs: PosixFileSystem 2024-05-10T11:07:54.004 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.info_log: 0x5623364bfda0 2024-05-10T11:07:54.004 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.max_file_opening_threads: 16 2024-05-10T11:07:54.004 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.statistics: (nil) 2024-05-10T11:07:54.004 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.use_fsync: 0 2024-05-10T11:07:54.004 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.max_log_file_size: 0 2024-05-10T11:07:54.004 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-10T11:07:54.004 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-10T11:07:54.004 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.keep_log_file_num: 1000 2024-05-10T11:07:54.004 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.recycle_log_file_num: 0 2024-05-10T11:07:54.004 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.allow_fallocate: 1 2024-05-10T11:07:54.005 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.allow_mmap_reads: 0 2024-05-10T11:07:54.005 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.allow_mmap_writes: 0 2024-05-10T11:07:54.005 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.use_direct_reads: 0 2024-05-10T11:07:54.005 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-10T11:07:54.005 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.create_missing_column_families: 0 2024-05-10T11:07:54.005 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.db_log_dir: 2024-05-10T11:07:54.005 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.wal_dir: 2024-05-10T11:07:54.005 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-10T11:07:54.005 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-10T11:07:54.005 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-10T11:07:54.005 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-10T11:07:54.007 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-10T11:07:54.007 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-10T11:07:54.007 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.advise_random_on_open: 1 2024-05-10T11:07:54.007 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.db_write_buffer_size: 0 2024-05-10T11:07:54.007 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.write_buffer_manager: 0x5623364910e0 2024-05-10T11:07:54.008 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-10T11:07:54.008 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-10T11:07:54.008 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-10T11:07:54.008 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.rate_limiter: (nil) 2024-05-10T11:07:54.008 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-10T11:07:54.008 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.wal_recovery_mode: 2 2024-05-10T11:07:54.008 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.enable_thread_tracking: 0 2024-05-10T11:07:54.008 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.enable_pipelined_write: 0 2024-05-10T11:07:54.008 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.unordered_write: 0 2024-05-10T11:07:54.008 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-10T11:07:54.008 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-10T11:07:54.008 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-10T11:07:54.008 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-10T11:07:54.008 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.row_cache: None 2024-05-10T11:07:54.008 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.wal_filter: None 2024-05-10T11:07:54.009 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-10T11:07:54.009 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.allow_ingest_behind: 0 2024-05-10T11:07:54.009 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.two_write_queues: 0 2024-05-10T11:07:54.009 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.manual_wal_flush: 0 2024-05-10T11:07:54.009 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.wal_compression: 0 2024-05-10T11:07:54.009 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.atomic_flush: 0 2024-05-10T11:07:54.009 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-10T11:07:54.009 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-10T11:07:54.009 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-10T11:07:54.009 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.log_readahead_size: 0 2024-05-10T11:07:54.009 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-10T11:07:54.009 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.best_efforts_recovery: 0 2024-05-10T11:07:54.009 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-10T11:07:54.009 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-10T11:07:54.010 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.allow_data_in_errors: 0 2024-05-10T11:07:54.010 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.db_host_id: __hostname__ 2024-05-10T11:07:54.010 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-10T11:07:54.010 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.max_background_jobs: 2 2024-05-10T11:07:54.010 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.max_background_compactions: -1 2024-05-10T11:07:54.010 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.max_subcompactions: 1 2024-05-10T11:07:54.010 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-10T11:07:54.010 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-10T11:07:54.010 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-10T11:07:54.010 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.max_total_wal_size: 0 2024-05-10T11:07:54.010 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-10T11:07:54.010 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-10T11:07:54.010 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-10T11:07:54.010 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-10T11:07:54.011 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.max_open_files: -1 2024-05-10T11:07:54.011 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.bytes_per_sync: 0 2024-05-10T11:07:54.011 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-10T11:07:54.011 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-10T11:07:54.011 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.compaction_readahead_size: 0 2024-05-10T11:07:54.011 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.max_background_flushes: -1 2024-05-10T11:07:54.011 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Compression algorithms supported: 2024-05-10T11:07:54.011 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-10T11:07:54.011 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: kZSTD supported: 0 2024-05-10T11:07:54.011 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: kXpressCompression supported: 0 2024-05-10T11:07:54.011 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: kLZ4HCCompression supported: 1 2024-05-10T11:07:54.011 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: kZlibCompression supported: 1 2024-05-10T11:07:54.011 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: kSnappyCompression supported: 1 2024-05-10T11:07:54.011 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: kLZ4Compression supported: 1 2024-05-10T11:07:54.011 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: kBZip2Compression supported: 0 2024-05-10T11:07:54.012 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-10T11:07:54.012 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-10T11:07:54.012 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-05-10T11:07:54.012 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-10T11:07:54.012 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-10T11:07:54.012 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.merge_operator: 2024-05-10T11:07:54.012 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.compaction_filter: None 2024-05-10T11:07:54.012 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.compaction_filter_factory: None 2024-05-10T11:07:54.012 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.sst_partitioner_factory: None 2024-05-10T11:07:54.012 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-10T11:07:54.012 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-10T11:07:54.012 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5623364bff60) 2024-05-10T11:07:54.012 INFO:journalctl@ceph.mon.b.smithi178.stdout: cache_index_and_filter_blocks: 1 2024-05-10T11:07:54.012 INFO:journalctl@ceph.mon.b.smithi178.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-10T11:07:54.012 INFO:journalctl@ceph.mon.b.smithi178.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-10T11:07:54.013 INFO:journalctl@ceph.mon.b.smithi178.stdout: pin_top_level_index_and_filter: 1 2024-05-10T11:07:54.013 INFO:journalctl@ceph.mon.b.smithi178.stdout: index_type: 0 2024-05-10T11:07:54.013 INFO:journalctl@ceph.mon.b.smithi178.stdout: data_block_index_type: 0 2024-05-10T11:07:54.013 INFO:journalctl@ceph.mon.b.smithi178.stdout: index_shortening: 1 2024-05-10T11:07:54.013 INFO:journalctl@ceph.mon.b.smithi178.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-10T11:07:54.013 INFO:journalctl@ceph.mon.b.smithi178.stdout: checksum: 4 2024-05-10T11:07:54.013 INFO:journalctl@ceph.mon.b.smithi178.stdout: no_block_cache: 0 2024-05-10T11:07:54.013 INFO:journalctl@ceph.mon.b.smithi178.stdout: block_cache: 0x5623364c31f0 2024-05-10T11:07:54.013 INFO:journalctl@ceph.mon.b.smithi178.stdout: block_cache_name: BinnedLRUCache 2024-05-10T11:07:54.013 INFO:journalctl@ceph.mon.b.smithi178.stdout: block_cache_options: 2024-05-10T11:07:54.013 INFO:journalctl@ceph.mon.b.smithi178.stdout: capacity : 536870912 2024-05-10T11:07:54.013 INFO:journalctl@ceph.mon.b.smithi178.stdout: num_shard_bits : 4 2024-05-10T11:07:54.013 INFO:journalctl@ceph.mon.b.smithi178.stdout: strict_capacity_limit : 0 2024-05-10T11:07:54.013 INFO:journalctl@ceph.mon.b.smithi178.stdout: high_pri_pool_ratio: 0.000 2024-05-10T11:07:54.014 INFO:journalctl@ceph.mon.b.smithi178.stdout: block_cache_compressed: (nil) 2024-05-10T11:07:54.014 INFO:journalctl@ceph.mon.b.smithi178.stdout: persistent_cache: (nil) 2024-05-10T11:07:54.014 INFO:journalctl@ceph.mon.b.smithi178.stdout: block_size: 4096 2024-05-10T11:07:54.014 INFO:journalctl@ceph.mon.b.smithi178.stdout: block_size_deviation: 10 2024-05-10T11:07:54.014 INFO:journalctl@ceph.mon.b.smithi178.stdout: block_restart_interval: 16 2024-05-10T11:07:54.014 INFO:journalctl@ceph.mon.b.smithi178.stdout: index_block_restart_interval: 1 2024-05-10T11:07:54.014 INFO:journalctl@ceph.mon.b.smithi178.stdout: metadata_block_size: 4096 2024-05-10T11:07:54.014 INFO:journalctl@ceph.mon.b.smithi178.stdout: partition_filters: 0 2024-05-10T11:07:54.014 INFO:journalctl@ceph.mon.b.smithi178.stdout: use_delta_encoding: 1 2024-05-10T11:07:54.014 INFO:journalctl@ceph.mon.b.smithi178.stdout: filter_policy: bloomfilter 2024-05-10T11:07:54.014 INFO:journalctl@ceph.mon.b.smithi178.stdout: whole_key_filtering: 1 2024-05-10T11:07:54.014 INFO:journalctl@ceph.mon.b.smithi178.stdout: verify_compression: 0 2024-05-10T11:07:54.014 INFO:journalctl@ceph.mon.b.smithi178.stdout: read_amp_bytes_per_bit: 0 2024-05-10T11:07:54.014 INFO:journalctl@ceph.mon.b.smithi178.stdout: format_version: 5 2024-05-10T11:07:54.014 INFO:journalctl@ceph.mon.b.smithi178.stdout: enable_index_compression: 1 2024-05-10T11:07:54.015 INFO:journalctl@ceph.mon.b.smithi178.stdout: block_align: 0 2024-05-10T11:07:54.015 INFO:journalctl@ceph.mon.b.smithi178.stdout: max_auto_readahead_size: 262144 2024-05-10T11:07:54.015 INFO:journalctl@ceph.mon.b.smithi178.stdout: prepopulate_block_cache: 0 2024-05-10T11:07:54.015 INFO:journalctl@ceph.mon.b.smithi178.stdout: initial_auto_readahead_size: 8192 2024-05-10T11:07:54.015 INFO:journalctl@ceph.mon.b.smithi178.stdout: num_file_reads_for_auto_readahead: 2 2024-05-10T11:07:54.015 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.write_buffer_size: 33554432 2024-05-10T11:07:54.015 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.max_write_buffer_number: 2 2024-05-10T11:07:54.015 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.compression: NoCompression 2024-05-10T11:07:54.015 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.bottommost_compression: Disabled 2024-05-10T11:07:54.015 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.prefix_extractor: nullptr 2024-05-10T11:07:54.015 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-10T11:07:54.015 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.num_levels: 7 2024-05-10T11:07:54.015 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-10T11:07:54.016 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-10T11:07:54.016 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-10T11:07:54.016 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-10T11:07:54.016 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-10T11:07:54.016 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-10T11:07:54.016 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-10T11:07:54.016 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-10T11:07:54.016 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-10T11:07:54.016 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-10T11:07:54.016 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-10T11:07:54.016 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-10T11:07:54.016 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-10T11:07:54.016 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.compression_opts.level: 32767 2024-05-10T11:07:54.017 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.compression_opts.strategy: 0 2024-05-10T11:07:54.017 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-10T11:07:54.017 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-10T11:07:54.017 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-10T11:07:54.017 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-10T11:07:54.017 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.compression_opts.enabled: false 2024-05-10T11:07:54.017 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-10T11:07:54.017 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-10T11:07:54.017 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-10T11:07:54.017 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-10T11:07:54.017 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.target_file_size_base: 67108864 2024-05-10T11:07:54.017 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-10T11:07:54.017 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-10T11:07:54.017 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-10T11:07:54.018 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-10T11:07:54.018 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-10T11:07:54.018 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-10T11:07:54.018 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-10T11:07:54.018 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-10T11:07:54.018 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-10T11:07:54.018 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-10T11:07:54.018 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-10T11:07:54.018 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-10T11:07:54.018 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-10T11:07:54.018 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-10T11:07:54.018 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.arena_block_size: 1048576 2024-05-10T11:07:54.018 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-10T11:07:54.018 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-10T11:07:54.018 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.disable_auto_compactions: 0 2024-05-10T11:07:54.019 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-10T11:07:54.019 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-10T11:07:54.019 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-10T11:07:54.019 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-10T11:07:54.019 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-10T11:07:54.019 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-10T11:07:54.019 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-10T11:07:54.019 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-10T11:07:54.019 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-10T11:07:54.019 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-10T11:07:54.019 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-10T11:07:54.019 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.inplace_update_support: 0 2024-05-10T11:07:54.019 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-10T11:07:54.020 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-10T11:07:54.020 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-10T11:07:54.020 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-10T11:07:54.020 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.bloom_locality: 0 2024-05-10T11:07:54.020 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.max_successive_merges: 0 2024-05-10T11:07:54.020 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-10T11:07:54.020 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.paranoid_file_checks: 0 2024-05-10T11:07:54.020 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.force_consistency_checks: 1 2024-05-10T11:07:54.020 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.report_bg_io_stats: 0 2024-05-10T11:07:54.020 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.ttl: 2592000 2024-05-10T11:07:54.020 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-10T11:07:54.020 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-10T11:07:54.020 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-10T11:07:54.021 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.enable_blob_files: false 2024-05-10T11:07:54.021 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.min_blob_size: 0 2024-05-10T11:07:54.021 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.blob_file_size: 268435456 2024-05-10T11:07:54.021 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-10T11:07:54.021 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-10T11:07:54.021 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-10T11:07:54.021 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-10T11:07:54.021 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-10T11:07:54.021 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.blob_file_starting_level: 0 2024-05-10T11:07:54.021 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-10T11:07:54.022 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-10T11:07:54.022 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-05-10T11:07:54.022 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 41295ccd-3670-42ec-8481-78d153e87265 2024-05-10T11:07:54.022 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715339273540283, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-10T11:07:54.022 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-05-10T11:07:54.022 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715339273540839, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715339273, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "41295ccd-3670-42ec-8481-78d153e87265", "db_session_id": "UF6X0DTPGU1K8HGTMML2", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-05-10T11:07:54.022 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715339273540945, "job": 1, "event": "recovery_finished"} 2024-05-10T11:07:54.022 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-05-10T11:07:54.022 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-10T11:07:54.022 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5623365e0000 2024-05-10T11:07:54.022 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: DB pointer 0x5623365cc000 2024-05-10T11:07:54.022 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-10T11:07:54.023 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-10T11:07:54.023 INFO:journalctl@ceph.mon.b.smithi178.stdout: ** DB Stats ** 2024-05-10T11:07:54.023 INFO:journalctl@ceph.mon.b.smithi178.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-10T11:07:54.023 INFO:journalctl@ceph.mon.b.smithi178.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-10T11:07:54.023 INFO:journalctl@ceph.mon.b.smithi178.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-10T11:07:54.023 INFO:journalctl@ceph.mon.b.smithi178.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-10T11:07:54.023 INFO:journalctl@ceph.mon.b.smithi178.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-10T11:07:54.023 INFO:journalctl@ceph.mon.b.smithi178.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-10T11:07:54.023 INFO:journalctl@ceph.mon.b.smithi178.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-10T11:07:54.023 INFO:journalctl@ceph.mon.b.smithi178.stdout: 2024-05-10T11:07:54.023 INFO:journalctl@ceph.mon.b.smithi178.stdout: ** Compaction Stats [default] ** 2024-05-10T11:07:54.023 INFO:journalctl@ceph.mon.b.smithi178.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-10T11:07:54.023 INFO:journalctl@ceph.mon.b.smithi178.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-10T11:07:54.023 INFO:journalctl@ceph.mon.b.smithi178.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-10T11:07:54.024 INFO:journalctl@ceph.mon.b.smithi178.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-10T11:07:54.024 INFO:journalctl@ceph.mon.b.smithi178.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-10T11:07:54.024 INFO:journalctl@ceph.mon.b.smithi178.stdout: 2024-05-10T11:07:54.024 INFO:journalctl@ceph.mon.b.smithi178.stdout: ** Compaction Stats [default] ** 2024-05-10T11:07:54.024 INFO:journalctl@ceph.mon.b.smithi178.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-10T11:07:54.024 INFO:journalctl@ceph.mon.b.smithi178.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-10T11:07:54.024 INFO:journalctl@ceph.mon.b.smithi178.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-10T11:07:54.024 INFO:journalctl@ceph.mon.b.smithi178.stdout: 2024-05-10T11:07:54.024 INFO:journalctl@ceph.mon.b.smithi178.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-10T11:07:54.024 INFO:journalctl@ceph.mon.b.smithi178.stdout: 2024-05-10T11:07:54.024 INFO:journalctl@ceph.mon.b.smithi178.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-10T11:07:54.024 INFO:journalctl@ceph.mon.b.smithi178.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-10T11:07:54.024 INFO:journalctl@ceph.mon.b.smithi178.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-10T11:07:54.025 INFO:journalctl@ceph.mon.b.smithi178.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-10T11:07:54.025 INFO:journalctl@ceph.mon.b.smithi178.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-10T11:07:54.025 INFO:journalctl@ceph.mon.b.smithi178.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-10T11:07:54.025 INFO:journalctl@ceph.mon.b.smithi178.stdout: Cumulative compaction: 0.00 GB write, 0.49 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-10T11:07:54.025 INFO:journalctl@ceph.mon.b.smithi178.stdout: Interval compaction: 0.00 GB write, 0.49 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-10T11:07:54.025 INFO:journalctl@ceph.mon.b.smithi178.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-10T11:07:54.025 INFO:journalctl@ceph.mon.b.smithi178.stdout: Block cache BinnedLRUCache@0x5623364c31f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-05-10T11:07:54.025 INFO:journalctl@ceph.mon.b.smithi178.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-10T11:07:54.025 INFO:journalctl@ceph.mon.b.smithi178.stdout: 2024-05-10T11:07:54.025 INFO:journalctl@ceph.mon.b.smithi178.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-10T11:07:54.025 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-05-10T11:07:54.025 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: using public_addr v2:172.21.15.178:0/0 -> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-05-10T11:07:54.025 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: starting mon.b rank -1 at public addrs [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] at bind addrs [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 2024-05-10T11:07:54.025 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: mon.b@-1(???) e0 preinit fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 2024-05-10T11:07:54.025 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: mon.b@-1(synchronizing).mds e1 new map 2024-05-10T11:07:54.026 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: mon.b@-1(synchronizing).mds e1 print_map 2024-05-10T11:07:54.026 INFO:journalctl@ceph.mon.b.smithi178.stdout: e1 2024-05-10T11:07:54.026 INFO:journalctl@ceph.mon.b.smithi178.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-10T11:07:54.026 INFO:journalctl@ceph.mon.b.smithi178.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-10T11:07:54.026 INFO:journalctl@ceph.mon.b.smithi178.stdout: legacy client fscid: -1 2024-05-10T11:07:54.026 INFO:journalctl@ceph.mon.b.smithi178.stdout: 2024-05-10T11:07:54.026 INFO:journalctl@ceph.mon.b.smithi178.stdout: No filesystems configured 2024-05-10T11:07:54.026 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-10T11:07:54.026 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-10T11:07:54.026 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-10T11:07:54.026 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-10T11:07:54.026 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-10T11:07:54.026 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-10T11:07:54.027 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-10T11:07:54.027 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-10T11:07:54.027 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-10T11:07:54.027 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-10T11:07:54.027 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:07:54.027 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:54.027 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T11:07:54.027 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:07:54.027 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:54.027 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:54.027 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:54.027 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:54.027 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi178", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:07:54.027 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:07:54.027 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:07:54.028 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi188", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:07:54.028 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: Updating smithi178:/etc/ceph/ceph.conf 2024-05-10T11:07:54.028 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: Updating smithi178:/var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T11:07:54.028 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:07:54.028 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: Deploying cephadm binary to smithi188 2024-05-10T11:07:54.028 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: Updating smithi178:/etc/ceph/ceph.client.admin.keyring 2024-05-10T11:07:54.028 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: Updating smithi178:/var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-10T11:07:54.028 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:07:54.028 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:54.028 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:54.028 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:54.028 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:54.028 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:07:54.028 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: Added host smithi188 2024-05-10T11:07:54.028 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:54.029 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:54.029 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:07:54.029 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:07:54.029 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:54.029 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T11:07:54.029 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:07:54.029 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/3754236342' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-10T11:07:54.029 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/3754236342' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-10T11:07:54.029 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: osdmap e4: 0 total, 0 up, 0 in 2024-05-10T11:07:54.029 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:54.029 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:54.029 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:54.029 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:54.029 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi188", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:07:54.030 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:07:54.030 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:07:54.030 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: Updating smithi188:/etc/ceph/ceph.conf 2024-05-10T11:07:54.030 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: Updating smithi188:/var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T11:07:54.030 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: Updating smithi188:/etc/ceph/ceph.client.admin.keyring 2024-05-10T11:07:54.030 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:07:54.030 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: Updating smithi188:/var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-10T11:07:54.030 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:54.030 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:54.030 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:54.030 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi100:172.21.15.100=a;smithi178:172.21.15.178=b;smithi188:172.21.15.188=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:07:54.030 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: Saving service mon spec with placement smithi100:172.21.15.100=a;smithi178:172.21.15.178=b;smithi188:172.21.15.188=c;count:3 2024-05-10T11:07:54.030 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:54.030 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:07:54.030 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:07:54.031 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:07:54.031 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:54.031 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T11:07:54.031 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:07:54.031 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: Deploying daemon mon.c on smithi188 2024-05-10T11:07:54.031 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:07:54.031 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:07:54.031 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:07:54.031 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T11:07:54.031 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: mon.a calling monitor election 2024-05-10T11:07:54.031 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T11:07:54.031 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T11:07:54.031 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:07:54.031 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T11:07:54.031 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: mon.c calling monitor election 2024-05-10T11:07:54.031 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T11:07:54.032 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:07:54.032 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T11:07:54.032 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T11:07:54.032 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-10T11:07:54.032 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: monmap epoch 2 2024-05-10T11:07:54.032 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 2024-05-10T11:07:54.032 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: last_changed 2024-05-10T11:07:44.536696+0000 2024-05-10T11:07:54.032 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: created 2024-05-10T11:06:01.369875+0000 2024-05-10T11:07:54.032 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: min_mon_release 19 (squid) 2024-05-10T11:07:54.032 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: election_strategy: 1 2024-05-10T11:07:54.032 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: 0: [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] mon.a 2024-05-10T11:07:54.032 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: 1: [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] mon.c 2024-05-10T11:07:54.032 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: fsmap 2024-05-10T11:07:54.032 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: osdmap e4: 0 total, 0 up, 0 in 2024-05-10T11:07:54.032 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: mgrmap e13: x(active, since 49s) 2024-05-10T11:07:54.033 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: overall HEALTH_OK 2024-05-10T11:07:54.033 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:54.033 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:54.033 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:54.033 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T11:07:54.033 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:07:54.033 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: Deploying daemon mon.b on smithi178 2024-05-10T11:07:54.033 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T11:07:54.033 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:07:54.033 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: from='client.? 172.21.15.188:0/1709588222' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-10T11:07:54.033 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 ceph-mon[33359]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-10T11:07:54.033 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 bash[33340]: 2b35ae6b2dd96afb36329937240d31f0539cfd9ad405d794e48e78e77203acc8 2024-05-10T11:07:54.033 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:53 smithi178 systemd[1]: Started Ceph mon.b for 20a542cc-0ebd-11ef-bc98-c7b262605968. 2024-05-10T11:07:59.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:58 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T11:07:59.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:58 smithi178 ceph-mon[33359]: mon.a calling monitor election 2024-05-10T11:07:59.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:58 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T11:07:59.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:58 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T11:07:59.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:58 smithi178 ceph-mon[33359]: mon.c calling monitor election 2024-05-10T11:07:59.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:58 smithi178 ceph-mon[33359]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:07:59.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:58 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T11:07:59.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:58 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T11:07:59.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:58 smithi178 ceph-mon[33359]: mon.b calling monitor election 2024-05-10T11:07:59.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:58 smithi178 ceph-mon[33359]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:07:59.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:58 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T11:07:59.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:58 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T11:07:59.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:58 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T11:07:59.004 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:58 smithi178 ceph-mon[33359]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-10T11:07:59.004 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:58 smithi178 ceph-mon[33359]: monmap epoch 3 2024-05-10T11:07:59.004 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:58 smithi178 ceph-mon[33359]: fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 2024-05-10T11:07:59.004 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:58 smithi178 ceph-mon[33359]: last_changed 2024-05-10T11:07:53.565231+0000 2024-05-10T11:07:59.004 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:58 smithi178 ceph-mon[33359]: created 2024-05-10T11:06:01.369875+0000 2024-05-10T11:07:59.004 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:58 smithi178 ceph-mon[33359]: min_mon_release 19 (squid) 2024-05-10T11:07:59.004 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:58 smithi178 ceph-mon[33359]: election_strategy: 1 2024-05-10T11:07:59.004 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:58 smithi178 ceph-mon[33359]: 0: [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] mon.a 2024-05-10T11:07:59.004 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:58 smithi178 ceph-mon[33359]: 1: [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] mon.c 2024-05-10T11:07:59.004 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:58 smithi178 ceph-mon[33359]: 2: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.b 2024-05-10T11:07:59.004 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:58 smithi178 ceph-mon[33359]: fsmap 2024-05-10T11:07:59.004 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:58 smithi178 ceph-mon[33359]: osdmap e4: 0 total, 0 up, 0 in 2024-05-10T11:07:59.004 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:58 smithi178 ceph-mon[33359]: mgrmap e13: x(active, since 58s) 2024-05-10T11:07:59.004 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:58 smithi178 ceph-mon[33359]: overall HEALTH_OK 2024-05-10T11:07:59.004 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:58 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:59.005 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:58 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:59.005 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:58 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:59.005 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:58 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:59.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:58 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T11:07:59.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:58 smithi188 ceph-mon[34075]: mon.a calling monitor election 2024-05-10T11:07:59.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:58 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T11:07:59.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:58 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T11:07:59.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:58 smithi188 ceph-mon[34075]: mon.c calling monitor election 2024-05-10T11:07:59.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:58 smithi188 ceph-mon[34075]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:07:59.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:58 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T11:07:59.030 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:58 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T11:07:59.030 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:58 smithi188 ceph-mon[34075]: mon.b calling monitor election 2024-05-10T11:07:59.030 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:58 smithi188 ceph-mon[34075]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:07:59.030 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:58 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T11:07:59.030 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:58 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T11:07:59.030 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:58 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T11:07:59.030 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:58 smithi188 ceph-mon[34075]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-10T11:07:59.030 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:58 smithi188 ceph-mon[34075]: monmap epoch 3 2024-05-10T11:07:59.030 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:58 smithi188 ceph-mon[34075]: fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 2024-05-10T11:07:59.030 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:58 smithi188 ceph-mon[34075]: last_changed 2024-05-10T11:07:53.565231+0000 2024-05-10T11:07:59.030 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:58 smithi188 ceph-mon[34075]: created 2024-05-10T11:06:01.369875+0000 2024-05-10T11:07:59.030 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:58 smithi188 ceph-mon[34075]: min_mon_release 19 (squid) 2024-05-10T11:07:59.030 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:58 smithi188 ceph-mon[34075]: election_strategy: 1 2024-05-10T11:07:59.031 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:58 smithi188 ceph-mon[34075]: 0: [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] mon.a 2024-05-10T11:07:59.031 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:58 smithi188 ceph-mon[34075]: 1: [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] mon.c 2024-05-10T11:07:59.031 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:58 smithi188 ceph-mon[34075]: 2: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.b 2024-05-10T11:07:59.031 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:58 smithi188 ceph-mon[34075]: fsmap 2024-05-10T11:07:59.031 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:58 smithi188 ceph-mon[34075]: osdmap e4: 0 total, 0 up, 0 in 2024-05-10T11:07:59.031 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:58 smithi188 ceph-mon[34075]: mgrmap e13: x(active, since 58s) 2024-05-10T11:07:59.031 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:58 smithi188 ceph-mon[34075]: overall HEALTH_OK 2024-05-10T11:07:59.031 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:58 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:59.031 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:58 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:59.031 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:58 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:59.031 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:58 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:59.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:58 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T11:07:59.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:58 smithi100 ceph-mon[30296]: mon.a calling monitor election 2024-05-10T11:07:59.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:58 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T11:07:59.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:58 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T11:07:59.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:58 smithi100 ceph-mon[30296]: mon.c calling monitor election 2024-05-10T11:07:59.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:58 smithi100 ceph-mon[30296]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:07:59.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:58 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T11:07:59.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:58 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T11:07:59.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:58 smithi100 ceph-mon[30296]: mon.b calling monitor election 2024-05-10T11:07:59.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:58 smithi100 ceph-mon[30296]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:07:59.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:58 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T11:07:59.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:58 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T11:07:59.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:58 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T11:07:59.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:58 smithi100 ceph-mon[30296]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-10T11:07:59.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:58 smithi100 ceph-mon[30296]: monmap epoch 3 2024-05-10T11:07:59.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:58 smithi100 ceph-mon[30296]: fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 2024-05-10T11:07:59.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:58 smithi100 ceph-mon[30296]: last_changed 2024-05-10T11:07:53.565231+0000 2024-05-10T11:07:59.040 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:58 smithi100 ceph-mon[30296]: created 2024-05-10T11:06:01.369875+0000 2024-05-10T11:07:59.040 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:58 smithi100 ceph-mon[30296]: min_mon_release 19 (squid) 2024-05-10T11:07:59.040 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:58 smithi100 ceph-mon[30296]: election_strategy: 1 2024-05-10T11:07:59.040 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:58 smithi100 ceph-mon[30296]: 0: [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] mon.a 2024-05-10T11:07:59.040 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:58 smithi100 ceph-mon[30296]: 1: [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] mon.c 2024-05-10T11:07:59.040 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:58 smithi100 ceph-mon[30296]: 2: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.b 2024-05-10T11:07:59.040 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:58 smithi100 ceph-mon[30296]: fsmap 2024-05-10T11:07:59.040 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:58 smithi100 ceph-mon[30296]: osdmap e4: 0 total, 0 up, 0 in 2024-05-10T11:07:59.040 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:58 smithi100 ceph-mon[30296]: mgrmap e13: x(active, since 58s) 2024-05-10T11:07:59.040 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:58 smithi100 ceph-mon[30296]: overall HEALTH_OK 2024-05-10T11:07:59.040 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:58 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:59.040 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:58 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:59.040 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:58 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:59.040 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:58 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:07:59.075 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:59.074+0000 7f35c621c700 1 -- 172.21.15.188:0/983040069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35a80035c0 msgr2=0x7f35a8005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:07:59.075 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:59.074+0000 7f35c621c700 1 --2- 172.21.15.188:0/983040069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35a80035c0 0x7f35a8005a50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f35c006b790 tx=0x7f35b000fe60 comp rx=0 tx=0).stop 2024-05-10T11:07:59.075 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:59.074+0000 7f35c621c700 1 -- 172.21.15.188:0/983040069 shutdown_connections 2024-05-10T11:07:59.075 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:59.074+0000 7f35c621c700 1 --2- 172.21.15.188:0/983040069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35a80035c0 0x7f35a8005a50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:07:59.075 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:59.074+0000 7f35c621c700 1 -- 172.21.15.188:0/983040069 >> 172.21.15.188:0/983040069 conn(0x7f35c00fd9d0 msgr2=0x7f35c00ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:07:59.075 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:59.074+0000 7f35c621c700 1 -- 172.21.15.188:0/983040069 shutdown_connections 2024-05-10T11:07:59.076 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:59.074+0000 7f35c621c700 1 -- 172.21.15.188:0/983040069 wait complete. 2024-05-10T11:07:59.076 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:59.075+0000 7f35c621c700 1 Processor -- start 2024-05-10T11:07:59.076 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:59.075+0000 7f35c621c700 1 -- start start 2024-05-10T11:07:59.078 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:59.075+0000 7f35c621c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35a80035c0 0x7f35c019e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:07:59.078 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:59.075+0000 7f35c621c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35c019ec90 0x7f35c0198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:07:59.078 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:59.075+0000 7f35c621c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35c0198e50 0x7f35c0199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:07:59.078 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:59.076+0000 7f35c621c700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f35c010ad50 con 0x7f35a80035c0 2024-05-10T11:07:59.078 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:59.076+0000 7f35c621c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f35c010abd0 con 0x7f35c019ec90 2024-05-10T11:07:59.078 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:59.076+0000 7f35c621c700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f35c010aed0 con 0x7f35c0198e50 2024-05-10T11:07:59.078 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:59.076+0000 7f35bffff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35c0198e50 0x7f35c0199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:07:59.078 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:59.076+0000 7f35bffff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35c0198e50 0x7f35c0199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:51386/0 (socket says 172.21.15.188:51386) 2024-05-10T11:07:59.078 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:59.076+0000 7f35bffff700 1 -- 172.21.15.188:0/1098050216 learned_addr learned my addr 172.21.15.188:0/1098050216 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-05-10T11:07:59.079 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:59.076+0000 7f35bf7fe700 1 --2- 172.21.15.188:0/1098050216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35a80035c0 0x7f35c019e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:07:59.079 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:59.076+0000 7f35beffd700 1 --2- 172.21.15.188:0/1098050216 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35c019ec90 0x7f35c0198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:07:59.079 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:59.076+0000 7f35bffff700 1 -- 172.21.15.188:0/1098050216 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35c0198e50 msgr2=0x7f35c0199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-05-10T11:07:59.079 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:59.076+0000 7f35bffff700 1 -- 172.21.15.188:0/1098050216 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35c0198e50 msgr2=0x7f35c0199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-10T11:07:59.079 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:59.076+0000 7f35bffff700 1 --2- 172.21.15.188:0/1098050216 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35c0198e50 0x7f35c0199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-10T11:07:59.079 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:59.076+0000 7f35beffd700 1 -- 172.21.15.188:0/1098050216 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35c019ec90 msgr2=0x7f35c0198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-10T11:07:59.079 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:59.076+0000 7f35beffd700 1 -- 172.21.15.188:0/1098050216 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35c019ec90 msgr2=0x7f35c0198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-10T11:07:59.079 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:59.076+0000 7f35bffff700 1 --2- 172.21.15.188:0/1098050216 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35c0198e50 0x7f35c0199300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-10T11:07:59.079 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:59.076+0000 7f35beffd700 1 --2- 172.21.15.188:0/1098050216 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35c019ec90 0x7f35c0198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-10T11:07:59.079 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:59.076+0000 7f35beffd700 1 --2- 172.21.15.188:0/1098050216 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35c019ec90 0x7f35c0198910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-10T11:07:59.079 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:59.076+0000 7f35bf7fe700 1 -- 172.21.15.188:0/1098050216 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35c0198e50 msgr2=0x7f35c0199300 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T11:07:59.079 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:59.076+0000 7f35bf7fe700 1 --2- 172.21.15.188:0/1098050216 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35c0198e50 0x7f35c0199300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:07:59.079 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:59.076+0000 7f35bf7fe700 1 -- 172.21.15.188:0/1098050216 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35c019ec90 msgr2=0x7f35c0198910 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T11:07:59.079 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:59.076+0000 7f35bf7fe700 1 --2- 172.21.15.188:0/1098050216 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35c019ec90 0x7f35c0198910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:07:59.080 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:59.076+0000 7f35bf7fe700 1 -- 172.21.15.188:0/1098050216 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35c01a5620 con 0x7f35a80035c0 2024-05-10T11:07:59.080 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:59.076+0000 7f35bf7fe700 1 --2- 172.21.15.188:0/1098050216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35a80035c0 0x7f35c019e750 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f35b001f000 tx=0x7f35b0005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:07:59.080 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:59.077+0000 7f35bcff9700 1 -- 172.21.15.188:0/1098050216 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b002b020 con 0x7f35a80035c0 2024-05-10T11:07:59.080 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:59.077+0000 7f35bcff9700 1 -- 172.21.15.188:0/1098050216 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f35b001d440 con 0x7f35a80035c0 2024-05-10T11:07:59.080 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:59.077+0000 7f35bcff9700 1 -- 172.21.15.188:0/1098050216 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b0022040 con 0x7f35a80035c0 2024-05-10T11:07:59.080 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:59.077+0000 7f35c621c700 1 -- 172.21.15.188:0/1098050216 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35c01a57f0 con 0x7f35a80035c0 2024-05-10T11:07:59.080 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:59.077+0000 7f35c621c700 1 -- 172.21.15.188:0/1098050216 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35c0101ec0 con 0x7f35a80035c0 2024-05-10T11:07:59.080 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:59.078+0000 7f35c621c700 1 -- 172.21.15.188:0/1098050216 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35c0064b00 con 0x7f35a80035c0 2024-05-10T11:07:59.080 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:59.078+0000 7f35bcff9700 1 -- 172.21.15.188:0/1098050216 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f35b002e020 con 0x7f35a80035c0 2024-05-10T11:07:59.080 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:59.078+0000 7f35bcff9700 1 --2- 172.21.15.188:0/1098050216 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f35ac042db0 0x7f35ac045270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:07:59.080 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:59.078+0000 7f35bcff9700 1 -- 172.21.15.188:0/1098050216 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f35b0055160 con 0x7f35a80035c0 2024-05-10T11:07:59.081 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:59.078+0000 7f35beffd700 1 --2- 172.21.15.188:0/1098050216 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f35ac042db0 0x7f35ac045270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:07:59.081 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:59.079+0000 7f35beffd700 1 --2- 172.21.15.188:0/1098050216 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f35ac042db0 0x7f35ac045270 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f35b4009850 tx=0x7f35b4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:07:59.083 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:59.083+0000 7f35bcff9700 1 -- 172.21.15.188:0/1098050216 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f35b0035020 con 0x7f35a80035c0 2024-05-10T11:07:59.399 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:59.398+0000 7f35c621c700 1 -- 172.21.15.188:0/1098050216 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f35c0102120 con 0x7f35a80035c0 2024-05-10T11:07:59.399 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:59.399+0000 7f35bcff9700 1 -- 172.21.15.188:0/1098050216 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1289 (secure 0 0 0) 0x7f35b00389e0 con 0x7f35a80035c0 2024-05-10T11:07:59.399 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-10T11:07:59.399 INFO:teuthology.orchestra.run.smithi188.stdout:{"epoch":3,"fsid":"20a542cc-0ebd-11ef-bc98-c7b262605968","modified":"2024-05-10T11:07:53.565231Z","created":"2024-05-10T11:06:01.369875Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:3300","nonce":0},{"type":"v1","addr":"172.21.15.100:6789","nonce":0}]},"addr":"172.21.15.100:6789/0","public_addr":"172.21.15.100:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:3300","nonce":0},{"type":"v1","addr":"172.21.15.188:6789","nonce":0}]},"addr":"172.21.15.188:6789/0","public_addr":"172.21.15.188:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3300","nonce":0},{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-05-10T11:07:59.402 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:59.402+0000 7f35c621c700 1 -- 172.21.15.188:0/1098050216 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f35ac042db0 msgr2=0x7f35ac045270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:07:59.402 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:59.402+0000 7f35c621c700 1 --2- 172.21.15.188:0/1098050216 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f35ac042db0 0x7f35ac045270 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f35b4009850 tx=0x7f35b4008040 comp rx=0 tx=0).stop 2024-05-10T11:07:59.402 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:59.402+0000 7f35c621c700 1 -- 172.21.15.188:0/1098050216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35a80035c0 msgr2=0x7f35c019e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:07:59.402 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:59.402+0000 7f35c621c700 1 --2- 172.21.15.188:0/1098050216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35a80035c0 0x7f35c019e750 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f35b001f000 tx=0x7f35b0005810 comp rx=0 tx=0).stop 2024-05-10T11:07:59.402 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:59.402+0000 7f35c621c700 1 -- 172.21.15.188:0/1098050216 shutdown_connections 2024-05-10T11:07:59.402 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:59.402+0000 7f35c621c700 1 --2- 172.21.15.188:0/1098050216 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f35ac042db0 0x7f35ac045270 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:07:59.402 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:59.402+0000 7f35c621c700 1 --2- 172.21.15.188:0/1098050216 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35c0198e50 0x7f35c0199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:07:59.402 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:59.402+0000 7f35c621c700 1 --2- 172.21.15.188:0/1098050216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35a80035c0 0x7f35c019e750 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:07:59.403 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:59.402+0000 7f35c621c700 1 --2- 172.21.15.188:0/1098050216 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35c019ec90 0x7f35c0198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:07:59.403 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:59.402+0000 7f35c621c700 1 -- 172.21.15.188:0/1098050216 >> 172.21.15.188:0/1098050216 conn(0x7f35c00fd9d0 msgr2=0x7f35c0104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:07:59.403 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:59.402+0000 7f35c621c700 1 -- 172.21.15.188:0/1098050216 shutdown_connections 2024-05-10T11:07:59.403 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:07:59.402+0000 7f35c621c700 1 -- 172.21.15.188:0/1098050216 wait complete. 2024-05-10T11:07:59.403 INFO:teuthology.orchestra.run.smithi188.stderr:dumped monmap epoch 3 2024-05-10T11:07:59.742 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:59 smithi188 ceph-mon[34075]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:07:59.742 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:59 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:07:59.742 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:59 smithi188 ceph-mon[34075]: from='client.? 172.21.15.188:0/1098050216' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-10T11:07:59.742 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:07:59 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T11:08:00.006 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:59 smithi178 ceph-mon[33359]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:08:00.006 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:59 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:08:00.006 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:59 smithi178 ceph-mon[33359]: from='client.? 172.21.15.188:0/1098050216' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-10T11:08:00.006 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:07:59 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T11:08:00.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:59 smithi100 ceph-mon[30296]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:08:00.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:59 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:08:00.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:59 smithi100 ceph-mon[30296]: from='client.? 172.21.15.188:0/1098050216' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-10T11:08:00.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:07:59 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T11:08:00.762 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-05-10T11:08:00.763 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph config generate-minimal-conf 2024-05-10T11:08:01.005 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:08:01.379 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:01 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:01.379 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:01 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:01.379 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:01 smithi100 ceph-mon[30296]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:08:01.379 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:01 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:01.379 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:01 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:01.379 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:01 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:01.379 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:01 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:08:01.379 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:01 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:08:01.379 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:01 smithi100 ceph-mon[30296]: Updating smithi100:/etc/ceph/ceph.conf 2024-05-10T11:08:01.380 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:01 smithi100 ceph-mon[30296]: Updating smithi178:/etc/ceph/ceph.conf 2024-05-10T11:08:01.380 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:01 smithi100 ceph-mon[30296]: Updating smithi188:/etc/ceph/ceph.conf 2024-05-10T11:08:01.380 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:01 smithi100 ceph-mon[30296]: Updating smithi100:/var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T11:08:01.380 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:01 smithi100 ceph-mon[30296]: Updating smithi178:/var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T11:08:01.380 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:01 smithi100 ceph-mon[30296]: Updating smithi188:/var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T11:08:01.380 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:01 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:01.380 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:01 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:01.380 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:01 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:01.380 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:01 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:01.380 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:01 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:01.380 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:01 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:01.380 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:01 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:01.380 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:01 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:01.380 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:01 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:01.380 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:01 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:01.381 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:01 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:01.381 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:01 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T11:08:01.381 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:01 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-10T11:08:01.381 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:01 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:08:01.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:01 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:01.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:01 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:01.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:01 smithi178 ceph-mon[33359]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:08:01.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:01 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:01.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:01 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:01.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:01 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:01.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:01 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:08:01.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:01 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:08:01.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:01 smithi178 ceph-mon[33359]: Updating smithi100:/etc/ceph/ceph.conf 2024-05-10T11:08:01.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:01 smithi178 ceph-mon[33359]: Updating smithi178:/etc/ceph/ceph.conf 2024-05-10T11:08:01.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:01 smithi178 ceph-mon[33359]: Updating smithi188:/etc/ceph/ceph.conf 2024-05-10T11:08:01.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:01 smithi178 ceph-mon[33359]: Updating smithi100:/var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T11:08:01.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:01 smithi178 ceph-mon[33359]: Updating smithi178:/var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T11:08:01.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:01 smithi178 ceph-mon[33359]: Updating smithi188:/var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T11:08:01.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:01 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:01.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:01 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:01.504 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:01 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:01.504 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:01 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:01.504 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:01 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:01.504 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:01 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:01.504 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:01 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:01.504 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:01 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:01.504 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:01 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:01.504 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:01 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:01.504 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:01 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:01.504 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:01 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T11:08:01.504 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:01 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-10T11:08:01.504 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:01 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:08:01.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:01 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:01.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:01 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:01.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:01 smithi188 ceph-mon[34075]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:08:01.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:01 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:01.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:01 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:01.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:01 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:01.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:01 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:08:01.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:01 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:08:01.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:01 smithi188 ceph-mon[34075]: Updating smithi100:/etc/ceph/ceph.conf 2024-05-10T11:08:01.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:01 smithi188 ceph-mon[34075]: Updating smithi178:/etc/ceph/ceph.conf 2024-05-10T11:08:01.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:01 smithi188 ceph-mon[34075]: Updating smithi188:/etc/ceph/ceph.conf 2024-05-10T11:08:01.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:01 smithi188 ceph-mon[34075]: Updating smithi100:/var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T11:08:01.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:01 smithi188 ceph-mon[34075]: Updating smithi178:/var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T11:08:01.530 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:01 smithi188 ceph-mon[34075]: Updating smithi188:/var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T11:08:01.530 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:01 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:01.530 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:01 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:01.530 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:01 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:01.530 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:01 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:01.530 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:01 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:01.530 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:01 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:01.530 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:01 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:01.530 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:01 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:01.530 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:01 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:01.530 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:01 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:01.530 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:01 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:01.530 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:01 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T11:08:01.530 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:01 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-10T11:08:01.531 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:01 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:08:02.426 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:02 smithi100 ceph-mon[30296]: Reconfiguring mon.a (unknown last config time)... 2024-05-10T11:08:02.427 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:02 smithi100 ceph-mon[30296]: Reconfiguring daemon mon.a on smithi100 2024-05-10T11:08:02.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:02 smithi178 ceph-mon[33359]: Reconfiguring mon.a (unknown last config time)... 2024-05-10T11:08:02.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:02 smithi178 ceph-mon[33359]: Reconfiguring daemon mon.a on smithi100 2024-05-10T11:08:02.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:02 smithi188 ceph-mon[34075]: Reconfiguring mon.a (unknown last config time)... 2024-05-10T11:08:02.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:02 smithi188 ceph-mon[34075]: Reconfiguring daemon mon.a on smithi100 2024-05-10T11:08:03.335 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:03 smithi100 ceph-mon[30296]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:08:03.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:03 smithi178 ceph-mon[33359]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:08:03.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:03.516+0000 7fdffaf23700 1 -- 172.21.15.100:0/2965843418 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdff4103560 msgr2=0x7fdff41039e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:08:03.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:03.516+0000 7fdffaf23700 1 --2- 172.21.15.100:0/2965843418 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdff4103560 0x7fdff41039e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdfe00099c0 tx=0x7fdfe0009cd0 comp rx=0 tx=0).stop 2024-05-10T11:08:03.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:03.517+0000 7fdffaf23700 1 -- 172.21.15.100:0/2965843418 shutdown_connections 2024-05-10T11:08:03.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:03.517+0000 7fdffaf23700 1 --2- 172.21.15.100:0/2965843418 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdff4103f20 0x7fdff4110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:08:03.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:03.517+0000 7fdffaf23700 1 --2- 172.21.15.100:0/2965843418 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdff4109540 0x7fdff4109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:08:03.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:03.517+0000 7fdffaf23700 1 --2- 172.21.15.100:0/2965843418 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdff4103560 0x7fdff41039e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:08:03.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:03.517+0000 7fdffaf23700 1 -- 172.21.15.100:0/2965843418 >> 172.21.15.100:0/2965843418 conn(0x7fdff407a660 msgr2=0x7fdff407aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:08:03.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:03.517+0000 7fdffaf23700 1 -- 172.21.15.100:0/2965843418 shutdown_connections 2024-05-10T11:08:03.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:03.517+0000 7fdffaf23700 1 -- 172.21.15.100:0/2965843418 wait complete. 2024-05-10T11:08:03.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:03.517+0000 7fdffaf23700 1 Processor -- start 2024-05-10T11:08:03.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:03.517+0000 7fdffaf23700 1 -- start start 2024-05-10T11:08:03.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:03.517+0000 7fdffaf23700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdff4103560 0x7fdff419f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:08:03.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:03.517+0000 7fdffaf23700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdff4103f20 0x7fdff419fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:08:03.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:03.517+0000 7fdffaf23700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdff4109540 0x7fdff4199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:08:03.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:03.517+0000 7fdffaf23700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdff41134d0 con 0x7fdff4103560 2024-05-10T11:08:03.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:03.517+0000 7fdffaf23700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdff4113350 con 0x7fdff4109540 2024-05-10T11:08:03.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:03.517+0000 7fdffaf23700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fdff4113650 con 0x7fdff4103f20 2024-05-10T11:08:03.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:03.518+0000 7fdff8cbf700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdff4103560 0x7fdff419f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:08:03.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:03.518+0000 7fdff8cbf700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdff4103560 0x7fdff419f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43710/0 (socket says 172.21.15.100:43710) 2024-05-10T11:08:03.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:03.518+0000 7fdff8cbf700 1 -- 172.21.15.100:0/2943437174 learned_addr learned my addr 172.21.15.100:0/2943437174 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:08:03.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:03.518+0000 7fdff94c0700 1 --2- 172.21.15.100:0/2943437174 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdff4109540 0x7fdff4199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:08:03.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:03.518+0000 7fdff3fff700 1 --2- 172.21.15.100:0/2943437174 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdff4103f20 0x7fdff419fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:08:03.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:03.518+0000 7fdff8cbf700 1 -- 172.21.15.100:0/2943437174 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdff4103f20 msgr2=0x7fdff419fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:08:03.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:03.518+0000 7fdff8cbf700 1 --2- 172.21.15.100:0/2943437174 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdff4103f20 0x7fdff419fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:08:03.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:03.518+0000 7fdff8cbf700 1 -- 172.21.15.100:0/2943437174 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdff4109540 msgr2=0x7fdff4199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:08:03.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:03.518+0000 7fdff8cbf700 1 --2- 172.21.15.100:0/2943437174 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdff4109540 0x7fdff4199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:08:03.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:03.518+0000 7fdff8cbf700 1 -- 172.21.15.100:0/2943437174 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdff419a010 con 0x7fdff4103560 2024-05-10T11:08:03.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:03.518+0000 7fdff94c0700 1 --2- 172.21.15.100:0/2943437174 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdff4109540 0x7fdff4199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:08:03.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:03.518+0000 7fdff8cbf700 1 --2- 172.21.15.100:0/2943437174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdff4103560 0x7fdff419f6b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdfe800d980 tx=0x7fdfe800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:08:03.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:03.518+0000 7fdff1ffb700 1 -- 172.21.15.100:0/2943437174 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe8016070 con 0x7fdff4103560 2024-05-10T11:08:03.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:03.518+0000 7fdff1ffb700 1 -- 172.21.15.100:0/2943437174 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdfe8004830 con 0x7fdff4103560 2024-05-10T11:08:03.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:03.518+0000 7fdff1ffb700 1 -- 172.21.15.100:0/2943437174 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe80054c0 con 0x7fdff4103560 2024-05-10T11:08:03.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:03.518+0000 7fdffaf23700 1 -- 172.21.15.100:0/2943437174 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdff419a300 con 0x7fdff4103560 2024-05-10T11:08:03.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:03.519+0000 7fdffaf23700 1 -- 172.21.15.100:0/2943437174 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdff41a6510 con 0x7fdff4103560 2024-05-10T11:08:03.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:03.519+0000 7fdff1ffb700 1 -- 172.21.15.100:0/2943437174 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7fdfe8004220 con 0x7fdff4103560 2024-05-10T11:08:03.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:03.519+0000 7fdff1ffb700 1 --2- 172.21.15.100:0/2943437174 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fdfe403e750 0x7fdfe4040c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:08:03.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:03.519+0000 7fdff1ffb700 1 -- 172.21.15.100:0/2943437174 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fdfe8052530 con 0x7fdff4103560 2024-05-10T11:08:03.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:03.519+0000 7fdffaf23700 1 -- 172.21.15.100:0/2943437174 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdff4104bb0 con 0x7fdff4103560 2024-05-10T11:08:03.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:03.520+0000 7fdff3fff700 1 --2- 172.21.15.100:0/2943437174 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fdfe403e750 0x7fdfe4040c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:08:03.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:03.520+0000 7fdff3fff700 1 --2- 172.21.15.100:0/2943437174 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fdfe403e750 0x7fdfe4040c10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fdfe00058e0 tx=0x7fdfe0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:08:03.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:03.525+0000 7fdff1ffb700 1 -- 172.21.15.100:0/2943437174 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fdfe801f020 con 0x7fdff4103560 2024-05-10T11:08:03.530 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:03 smithi188 ceph-mon[34075]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:08:03.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:03.729+0000 7fdffaf23700 1 -- 172.21.15.100:0/2943437174 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fdff419ad60 con 0x7fdff4103560 2024-05-10T11:08:03.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:03.729+0000 7fdff1ffb700 1 -- 172.21.15.100:0/2943437174 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+277 (secure 0 0 0) 0x7fdfe80227d0 con 0x7fdff4103560 2024-05-10T11:08:03.732 INFO:teuthology.orchestra.run.smithi100.stdout:# minimal ceph.conf for 20a542cc-0ebd-11ef-bc98-c7b262605968 2024-05-10T11:08:03.732 INFO:teuthology.orchestra.run.smithi100.stdout:[global] 2024-05-10T11:08:03.732 INFO:teuthology.orchestra.run.smithi100.stdout: fsid = 20a542cc-0ebd-11ef-bc98-c7b262605968 2024-05-10T11:08:03.732 INFO:teuthology.orchestra.run.smithi100.stdout: mon_host = [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] 2024-05-10T11:08:03.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:03.731+0000 7fdffaf23700 1 -- 172.21.15.100:0/2943437174 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fdfe403e750 msgr2=0x7fdfe4040c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:08:03.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:03.732+0000 7fdffaf23700 1 --2- 172.21.15.100:0/2943437174 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fdfe403e750 0x7fdfe4040c10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fdfe00058e0 tx=0x7fdfe0005fb0 comp rx=0 tx=0).stop 2024-05-10T11:08:03.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:03.732+0000 7fdffaf23700 1 -- 172.21.15.100:0/2943437174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdff4103560 msgr2=0x7fdff419f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:08:03.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:03.732+0000 7fdffaf23700 1 --2- 172.21.15.100:0/2943437174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdff4103560 0x7fdff419f6b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdfe800d980 tx=0x7fdfe800dc90 comp rx=0 tx=0).stop 2024-05-10T11:08:03.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:03.732+0000 7fdffaf23700 1 -- 172.21.15.100:0/2943437174 shutdown_connections 2024-05-10T11:08:03.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:03.732+0000 7fdffaf23700 1 --2- 172.21.15.100:0/2943437174 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fdfe403e750 0x7fdfe4040c10 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:08:03.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:03.732+0000 7fdffaf23700 1 --2- 172.21.15.100:0/2943437174 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdff4109540 0x7fdff4199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:08:03.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:03.732+0000 7fdffaf23700 1 --2- 172.21.15.100:0/2943437174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdff4103560 0x7fdff419f6b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:08:03.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:03.732+0000 7fdffaf23700 1 --2- 172.21.15.100:0/2943437174 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdff4103f20 0x7fdff419fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:08:03.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:03.732+0000 7fdffaf23700 1 -- 172.21.15.100:0/2943437174 >> 172.21.15.100:0/2943437174 conn(0x7fdff407a660 msgr2=0x7fdff4102300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:08:03.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:03.732+0000 7fdffaf23700 1 -- 172.21.15.100:0/2943437174 shutdown_connections 2024-05-10T11:08:03.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:03.732+0000 7fdffaf23700 1 -- 172.21.15.100:0/2943437174 wait complete. 2024-05-10T11:08:04.380 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-05-10T11:08:04.380 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-05-10T11:08:04.381 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/ceph/ceph.conf 2024-05-10T11:08:04.395 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:04 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:04.395 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:04 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:04.395 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:04 smithi100 ceph-mon[30296]: Reconfiguring mon.b (monmap changed)... 2024-05-10T11:08:04.395 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:04 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T11:08:04.395 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:04 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-10T11:08:04.396 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:04 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:08:04.396 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:04 smithi100 ceph-mon[30296]: Reconfiguring daemon mon.b on smithi178 2024-05-10T11:08:04.396 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:04 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/2943437174' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:08:04.409 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-05-10T11:08:04.409 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-10T11:08:04.477 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-05-10T11:08:04.477 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/ceph/ceph.conf 2024-05-10T11:08:04.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:04 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:04.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:04 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:04.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:04 smithi178 ceph-mon[33359]: Reconfiguring mon.b (monmap changed)... 2024-05-10T11:08:04.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:04 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T11:08:04.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:04 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-10T11:08:04.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:04 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:08:04.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:04 smithi178 ceph-mon[33359]: Reconfiguring daemon mon.b on smithi178 2024-05-10T11:08:04.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:04 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/2943437174' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:08:04.510 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-05-10T11:08:04.510 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-10T11:08:04.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:04 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:04.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:04 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:04.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:04 smithi188 ceph-mon[34075]: Reconfiguring mon.b (monmap changed)... 2024-05-10T11:08:04.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:04 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T11:08:04.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:04 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-10T11:08:04.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:04 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:08:04.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:04 smithi188 ceph-mon[34075]: Reconfiguring daemon mon.b on smithi178 2024-05-10T11:08:04.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:04 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/2943437174' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:08:04.574 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-05-10T11:08:04.574 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/ceph/ceph.conf 2024-05-10T11:08:04.608 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-05-10T11:08:04.608 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-10T11:08:04.678 INFO:tasks.cephadm:Adding mgr.x on smithi100 2024-05-10T11:08:04.679 INFO:tasks.cephadm:Adding mgr.y on smithi178 2024-05-10T11:08:04.679 INFO:tasks.cephadm:Adding mgr.z on smithi188 2024-05-10T11:08:04.679 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph orch apply mgr '3;smithi100=x;smithi178=y;smithi188=z' 2024-05-10T11:08:04.965 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T11:08:05.732 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:05 smithi188 ceph-mon[34075]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:08:05.733 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:05 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:05.733 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:05 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:05.733 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:05 smithi188 ceph-mon[34075]: Reconfiguring mon.c (monmap changed)... 2024-05-10T11:08:05.733 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:05 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T11:08:05.733 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:05 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-10T11:08:05.733 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:05 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:08:05.733 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:05 smithi188 ceph-mon[34075]: Reconfiguring daemon mon.c on smithi188 2024-05-10T11:08:06.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:05 smithi178 ceph-mon[33359]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:08:06.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:05 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:06.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:05 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:06.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:05 smithi178 ceph-mon[33359]: Reconfiguring mon.c (monmap changed)... 2024-05-10T11:08:06.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:05 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T11:08:06.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:05 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-10T11:08:06.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:05 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:08:06.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:05 smithi178 ceph-mon[33359]: Reconfiguring daemon mon.c on smithi188 2024-05-10T11:08:06.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:05 smithi100 ceph-mon[30296]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:08:06.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:05 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:06.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:05 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:06.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:05 smithi100 ceph-mon[30296]: Reconfiguring mon.c (monmap changed)... 2024-05-10T11:08:06.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:05 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T11:08:06.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:05 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-10T11:08:06.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:05 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:08:06.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:05 smithi100 ceph-mon[30296]: Reconfiguring daemon mon.c on smithi188 2024-05-10T11:08:06.468 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:08:06.465+0000 7f4005772700 1 -- 172.21.15.188:0/361268115 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4000103a90 msgr2=0x7f4000105e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:08:06.468 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:08:06.465+0000 7f4005772700 1 --2- 172.21.15.188:0/361268115 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4000103a90 0x7f4000105e80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0009910 tx=0x7f3ff0009c20 comp rx=0 tx=0).stop 2024-05-10T11:08:06.468 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:08:06.465+0000 7f4005772700 1 -- 172.21.15.188:0/361268115 shutdown_connections 2024-05-10T11:08:06.468 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:08:06.465+0000 7f4005772700 1 --2- 172.21.15.188:0/361268115 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4000103a90 0x7f4000105e80 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:08:06.468 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:08:06.465+0000 7f4005772700 1 -- 172.21.15.188:0/361268115 >> 172.21.15.188:0/361268115 conn(0x7f40000fd4a0 msgr2=0x7f40000ff8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:08:06.468 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:08:06.465+0000 7f4005772700 1 -- 172.21.15.188:0/361268115 shutdown_connections 2024-05-10T11:08:06.468 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:08:06.465+0000 7f4005772700 1 -- 172.21.15.188:0/361268115 wait complete. 2024-05-10T11:08:06.469 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:08:06.465+0000 7f4005772700 1 Processor -- start 2024-05-10T11:08:06.469 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:08:06.465+0000 7f4005772700 1 -- start start 2024-05-10T11:08:06.469 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:08:06.465+0000 7f4005772700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4000103a90 0x7f4000198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:08:06.469 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:08:06.465+0000 7f4005772700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4000198b60 0x7f400019dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:08:06.469 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:08:06.465+0000 7f4005772700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4000198fe0 0x7f40001a0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:08:06.469 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:08:06.465+0000 7f4005772700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4000108d40 con 0x7f4000198b60 2024-05-10T11:08:06.469 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:08:06.465+0000 7f4005772700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4000108bc0 con 0x7f4000103a90 2024-05-10T11:08:06.469 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:08:06.465+0000 7f4005772700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4000108ec0 con 0x7f4000198fe0 2024-05-10T11:08:06.469 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:08:06.466+0000 7f3fff7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4000198fe0 0x7f40001a0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:08:06.469 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:08:06.466+0000 7f3fff7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4000198fe0 0x7f40001a0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:57088/0 (socket says 172.21.15.188:57088) 2024-05-10T11:08:06.469 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:08:06.466+0000 7f3fff7fe700 1 -- 172.21.15.188:0/1884017673 learned_addr learned my addr 172.21.15.188:0/1884017673 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-05-10T11:08:06.469 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:08:06.466+0000 7f3ff7fff700 1 --2- 172.21.15.188:0/1884017673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4000198b60 0x7f400019dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:08:06.470 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:08:06.466+0000 7f3ffeffd700 1 --2- 172.21.15.188:0/1884017673 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4000103a90 0x7f4000198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:08:06.470 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:08:06.466+0000 7f3fff7fe700 1 -- 172.21.15.188:0/1884017673 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4000103a90 msgr2=0x7f4000198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:08:06.470 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:08:06.466+0000 7f3fff7fe700 1 --2- 172.21.15.188:0/1884017673 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4000103a90 0x7f4000198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:08:06.470 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:08:06.466+0000 7f3fff7fe700 1 -- 172.21.15.188:0/1884017673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4000198b60 msgr2=0x7f400019dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:08:06.470 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:08:06.466+0000 7f3fff7fe700 1 --2- 172.21.15.188:0/1884017673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4000198b60 0x7f400019dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:08:06.470 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:08:06.466+0000 7f3fff7fe700 1 -- 172.21.15.188:0/1884017673 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40001a0a30 con 0x7f4000198fe0 2024-05-10T11:08:06.470 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:08:06.466+0000 7f3ff7fff700 1 --2- 172.21.15.188:0/1884017673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4000198b60 0x7f400019dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:08:06.470 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:08:06.466+0000 7f3fff7fe700 1 --2- 172.21.15.188:0/1884017673 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4000198fe0 0x7f40001a0160 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3ff800ba50 tx=0x7f3ff800bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:08:06.470 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:08:06.466+0000 7f3ffcff9700 1 -- 172.21.15.188:0/1884017673 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff8019070 con 0x7f4000198fe0 2024-05-10T11:08:06.470 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:08:06.466+0000 7f3ffcff9700 1 -- 172.21.15.188:0/1884017673 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3ff800cdc0 con 0x7f4000198fe0 2024-05-10T11:08:06.470 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:08:06.466+0000 7f4005772700 1 -- 172.21.15.188:0/1884017673 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40001a0d20 con 0x7f4000198fe0 2024-05-10T11:08:06.471 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:08:06.466+0000 7f3ffcff9700 1 -- 172.21.15.188:0/1884017673 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff8014e60 con 0x7f4000198fe0 2024-05-10T11:08:06.471 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:08:06.466+0000 7f4005772700 1 -- 172.21.15.188:0/1884017673 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40001a12d0 con 0x7f4000198fe0 2024-05-10T11:08:06.471 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:08:06.467+0000 7f3ffcff9700 1 -- 172.21.15.188:0/1884017673 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f3ff801c070 con 0x7f4000198fe0 2024-05-10T11:08:06.471 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:08:06.467+0000 7f4005772700 1 -- 172.21.15.188:0/1884017673 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4000064b30 con 0x7f4000198fe0 2024-05-10T11:08:06.471 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:08:06.468+0000 7f3ffcff9700 1 --2- 172.21.15.188:0/1884017673 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f3fec03e9a0 0x7f3fec040e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:08:06.471 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:08:06.468+0000 7f3ffcff9700 1 -- 172.21.15.188:0/1884017673 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f3ff8051c20 con 0x7f4000198fe0 2024-05-10T11:08:06.471 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:08:06.468+0000 7f3ffeffd700 1 --2- 172.21.15.188:0/1884017673 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f3fec03e9a0 0x7f3fec040e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:08:06.471 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:08:06.469+0000 7f3ffeffd700 1 --2- 172.21.15.188:0/1884017673 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f3fec03e9a0 0x7f3fec040e60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0009750 tx=0x7f3ff0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:08:06.473 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:08:06.473+0000 7f3ffcff9700 1 -- 172.21.15.188:0/1884017673 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f4000064b30 con 0x7f4000198fe0 2024-05-10T11:08:06.698 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:08:06.698+0000 7f4005772700 1 -- 172.21.15.188:0/1884017673 --> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi100=x;smithi178=y;smithi188=z", "target": ["mon-mgr", ""]}) -- 0x7f40000ff4b0 con 0x7f3fec03e9a0 2024-05-10T11:08:06.704 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:08:06.704+0000 7f3ffcff9700 1 -- 172.21.15.188:0/1884017673 <== mgr.14152 v2:172.21.15.100:6800/2507287356 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f40000ff4b0 con 0x7f3fec03e9a0 2024-05-10T11:08:06.704 INFO:teuthology.orchestra.run.smithi188.stdout:Scheduled mgr update... 2024-05-10T11:08:06.707 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:08:06.706+0000 7f4005772700 1 -- 172.21.15.188:0/1884017673 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f3fec03e9a0 msgr2=0x7f3fec040e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:08:06.707 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:08:06.706+0000 7f4005772700 1 --2- 172.21.15.188:0/1884017673 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f3fec03e9a0 0x7f3fec040e60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0009750 tx=0x7f3ff0005fb0 comp rx=0 tx=0).stop 2024-05-10T11:08:06.708 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:08:06.706+0000 7f4005772700 1 -- 172.21.15.188:0/1884017673 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4000198fe0 msgr2=0x7f40001a0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:08:06.708 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:08:06.706+0000 7f4005772700 1 --2- 172.21.15.188:0/1884017673 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4000198fe0 0x7f40001a0160 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3ff800ba50 tx=0x7f3ff800bd60 comp rx=0 tx=0).stop 2024-05-10T11:08:06.708 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:08:06.706+0000 7f4005772700 1 -- 172.21.15.188:0/1884017673 shutdown_connections 2024-05-10T11:08:06.708 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:08:06.706+0000 7f4005772700 1 --2- 172.21.15.188:0/1884017673 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f3fec03e9a0 0x7f3fec040e60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:08:06.708 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:08:06.707+0000 7f4005772700 1 --2- 172.21.15.188:0/1884017673 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4000198fe0 0x7f40001a0160 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:08:06.708 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:08:06.707+0000 7f4005772700 1 --2- 172.21.15.188:0/1884017673 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4000103a90 0x7f4000198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:08:06.709 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:08:06.707+0000 7f4005772700 1 --2- 172.21.15.188:0/1884017673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4000198b60 0x7f400019dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:08:06.709 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:08:06.707+0000 7f4005772700 1 -- 172.21.15.188:0/1884017673 >> 172.21.15.188:0/1884017673 conn(0x7f40000fd4a0 msgr2=0x7f40000feda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:08:06.709 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:08:06.707+0000 7f4005772700 1 -- 172.21.15.188:0/1884017673 shutdown_connections 2024-05-10T11:08:06.709 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:08:06.707+0000 7f4005772700 1 -- 172.21.15.188:0/1884017673 wait complete. 2024-05-10T11:08:07.692 DEBUG:teuthology.orchestra.run.smithi178:mgr.y> sudo journalctl -f -n 0 -u ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mgr.y.service 2024-05-10T11:08:07.694 DEBUG:teuthology.orchestra.run.smithi188:mgr.z> sudo journalctl -f -n 0 -u ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mgr.z.service 2024-05-10T11:08:07.697 INFO:tasks.cephadm:Deploying OSDs... 2024-05-10T11:08:07.697 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-05-10T11:08:07.697 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/scratch_devs of=/dev/stdout 2024-05-10T11:08:07.717 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-10T11:08:07.717 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_1 2024-05-10T11:08:07.777 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-10T11:08:07.777 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T11:08:07.777 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-05-10T11:08:07.777 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T11:08:07.777 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T11:08:07.777 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-05-10 11:07:10.634271148 +0000 2024-05-10T11:08:07.777 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-05-10 11:02:11.997035531 +0000 2024-05-10T11:08:07.777 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-05-10 11:02:11.997035531 +0000 2024-05-10T11:08:07.777 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-05-10 11:02:11.997035531 +0000 2024-05-10T11:08:07.778 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-10T11:08:07.846 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-05-10T11:08:07.846 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-05-10T11:08:07.846 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000229629 s, 2.2 MB/s 2024-05-10T11:08:07.847 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-10T11:08:07.907 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_2 2024-05-10T11:08:07.965 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-10T11:08:07.965 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T11:08:07.965 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-05-10T11:08:07.965 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T11:08:07.965 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T11:08:07.965 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-05-10 11:07:10.660270463 +0000 2024-05-10T11:08:07.965 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-05-10 11:02:12.350026212 +0000 2024-05-10T11:08:07.965 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-05-10 11:02:12.350026212 +0000 2024-05-10T11:08:07.965 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-05-10 11:02:12.350026212 +0000 2024-05-10T11:08:07.966 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-10T11:08:08.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:07 smithi178 ceph-mon[33359]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:08:08.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:07 smithi178 ceph-mon[33359]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi100=x;smithi178=y;smithi188=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:08:08.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:07 smithi178 ceph-mon[33359]: Saving service mgr spec with placement smithi100=x;smithi178=y;smithi188=z;count:3 2024-05-10T11:08:08.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:07 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:08.021 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:07 smithi188 ceph-mon[34075]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:08:08.021 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:07 smithi188 ceph-mon[34075]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi100=x;smithi178=y;smithi188=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:08:08.021 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:07 smithi188 ceph-mon[34075]: Saving service mgr spec with placement smithi100=x;smithi178=y;smithi188=z;count:3 2024-05-10T11:08:08.021 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:07 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:08.028 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:07 smithi100 ceph-mon[30296]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:08:08.029 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:07 smithi100 ceph-mon[30296]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi100=x;smithi178=y;smithi188=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:08:08.029 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:07 smithi100 ceph-mon[30296]: Saving service mgr spec with placement smithi100=x;smithi178=y;smithi188=z;count:3 2024-05-10T11:08:08.029 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:07 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:08.030 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-05-10T11:08:08.031 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-05-10T11:08:08.031 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000213997 s, 2.4 MB/s 2024-05-10T11:08:08.032 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-10T11:08:08.092 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_3 2024-05-10T11:08:08.148 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-10T11:08:08.148 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T11:08:08.148 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-05-10T11:08:08.148 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T11:08:08.148 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T11:08:08.148 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-05-10 11:07:10.684269830 +0000 2024-05-10T11:08:08.148 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-05-10 11:02:12.688017289 +0000 2024-05-10T11:08:08.148 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-05-10 11:02:12.688017289 +0000 2024-05-10T11:08:08.148 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-05-10 11:02:12.688017289 +0000 2024-05-10T11:08:08.148 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-10T11:08:08.212 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-05-10T11:08:08.212 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-05-10T11:08:08.212 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000187226 s, 2.7 MB/s 2024-05-10T11:08:08.213 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-10T11:08:08.270 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_4 2024-05-10T11:08:08.326 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-10T11:08:08.326 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T11:08:08.326 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-05-10T11:08:08.327 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T11:08:08.327 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T11:08:08.327 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-05-10 11:07:10.707269224 +0000 2024-05-10T11:08:08.327 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-05-10 11:02:13.039008023 +0000 2024-05-10T11:08:08.327 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-05-10 11:02:13.039008023 +0000 2024-05-10T11:08:08.327 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-05-10 11:02:13.039008023 +0000 2024-05-10T11:08:08.327 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-10T11:08:08.390 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-05-10T11:08:08.390 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-05-10T11:08:08.390 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000171768 s, 3.0 MB/s 2024-05-10T11:08:08.391 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-10T11:08:08.449 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-05-10T11:08:08.450 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/scratch_devs of=/dev/stdout 2024-05-10T11:08:08.470 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-10T11:08:08.470 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_1 2024-05-10T11:08:08.529 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-10T11:08:08.529 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T11:08:08.529 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-05-10T11:08:08.529 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T11:08:08.529 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T11:08:08.529 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2024-05-10 11:07:52.969317337 +0000 2024-05-10T11:08:08.529 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2024-05-10 11:02:20.004845641 +0000 2024-05-10T11:08:08.529 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2024-05-10 11:02:20.004845641 +0000 2024-05-10T11:08:08.529 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: 2024-05-10 11:02:20.004845641 +0000 2024-05-10T11:08:08.530 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-10T11:08:08.597 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2024-05-10T11:08:08.597 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2024-05-10T11:08:08.597 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 0.000124088 s, 4.1 MB/s 2024-05-10T11:08:08.598 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-10T11:08:08.658 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_2 2024-05-10T11:08:08.715 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-10T11:08:08.715 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T11:08:08.716 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-05-10T11:08:08.716 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T11:08:08.716 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T11:08:08.716 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2024-05-10 11:07:52.969317337 +0000 2024-05-10T11:08:08.716 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2024-05-10 11:02:20.388839019 +0000 2024-05-10T11:08:08.716 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2024-05-10 11:02:20.388839019 +0000 2024-05-10T11:08:08.716 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: 2024-05-10 11:02:20.388839019 +0000 2024-05-10T11:08:08.716 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-10T11:08:08.781 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2024-05-10T11:08:08.781 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2024-05-10T11:08:08.782 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 0.000118322 s, 4.3 MB/s 2024-05-10T11:08:08.783 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-10T11:08:08.840 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_3 2024-05-10T11:08:08.896 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-10T11:08:08.896 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T11:08:08.897 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-05-10T11:08:08.897 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T11:08:08.897 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T11:08:08.897 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2024-05-10 11:07:52.970317320 +0000 2024-05-10T11:08:08.897 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2024-05-10 11:02:20.711833449 +0000 2024-05-10T11:08:08.897 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2024-05-10 11:02:20.711833449 +0000 2024-05-10T11:08:08.897 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: 2024-05-10 11:02:20.711833449 +0000 2024-05-10T11:08:08.897 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-10T11:08:08.961 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2024-05-10T11:08:08.961 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2024-05-10T11:08:08.962 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 0.00011179 s, 4.6 MB/s 2024-05-10T11:08:08.963 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-10T11:08:09.021 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_4 2024-05-10T11:08:09.077 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-10T11:08:09.077 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T11:08:09.077 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-05-10T11:08:09.077 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T11:08:09.077 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T11:08:09.077 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2024-05-10 11:07:52.970317320 +0000 2024-05-10T11:08:09.078 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2024-05-10 11:02:21.042827741 +0000 2024-05-10T11:08:09.078 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2024-05-10 11:02:21.042827741 +0000 2024-05-10T11:08:09.078 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: 2024-05-10 11:02:21.042827741 +0000 2024-05-10T11:08:09.078 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-10T11:08:09.141 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2024-05-10T11:08:09.141 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2024-05-10T11:08:09.141 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 0.00012664 s, 4.0 MB/s 2024-05-10T11:08:09.142 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-10T11:08:09.199 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-05-10T11:08:09.199 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/scratch_devs of=/dev/stdout 2024-05-10T11:08:09.215 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-10T11:08:09.216 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_1 2024-05-10T11:08:09.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:09 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:09.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:09 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:09.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:09 smithi178 ceph-mon[33359]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:08:09.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:09 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:08:09.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:09 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:08:09.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:09 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:08:09.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:09 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:09.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:09 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-10T11:08:09.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:09 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-10T11:08:09.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:09 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-10T11:08:09.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:09 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:08:09.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:09 smithi178 ceph-mon[33359]: Deploying daemon mgr.z on smithi188 2024-05-10T11:08:09.271 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-10T11:08:09.271 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T11:08:09.271 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-05-10T11:08:09.271 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T11:08:09.271 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T11:08:09.271 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2024-05-10 11:07:43.748751143 +0000 2024-05-10T11:08:09.271 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2024-05-10 11:02:13.417970846 +0000 2024-05-10T11:08:09.271 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2024-05-10 11:02:13.417970846 +0000 2024-05-10T11:08:09.272 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: 2024-05-10 11:02:13.417970846 +0000 2024-05-10T11:08:09.272 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-10T11:08:09.332 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:09 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:09.332 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:09 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:09.332 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:09 smithi188 ceph-mon[34075]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:08:09.332 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:09 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:08:09.332 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:09 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:08:09.333 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:09 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:08:09.333 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:09 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:09.333 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:09 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-10T11:08:09.333 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:09 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-10T11:08:09.333 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:09 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-10T11:08:09.333 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:09 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:08:09.333 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:09 smithi188 ceph-mon[34075]: Deploying daemon mgr.z on smithi188 2024-05-10T11:08:09.334 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2024-05-10T11:08:09.335 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2024-05-10T11:08:09.335 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000213843 s, 2.4 MB/s 2024-05-10T11:08:09.336 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-10T11:08:09.393 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_2 2024-05-10T11:08:09.449 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-10T11:08:09.449 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T11:08:09.449 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-05-10T11:08:09.449 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T11:08:09.450 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T11:08:09.450 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2024-05-10 11:07:43.749751125 +0000 2024-05-10T11:08:09.450 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2024-05-10 11:02:13.757964615 +0000 2024-05-10T11:08:09.450 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2024-05-10 11:02:13.757964615 +0000 2024-05-10T11:08:09.450 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: 2024-05-10 11:02:13.757964615 +0000 2024-05-10T11:08:09.450 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-10T11:08:09.514 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2024-05-10T11:08:09.514 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2024-05-10T11:08:09.514 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000167994 s, 3.0 MB/s 2024-05-10T11:08:09.516 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-10T11:08:09.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:09 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:09.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:09 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:09.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:09 smithi100 ceph-mon[30296]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:08:09.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:09 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:08:09.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:09 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:08:09.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:09 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:08:09.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:09 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:09.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:09 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-10T11:08:09.539 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:09 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-10T11:08:09.539 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:09 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-10T11:08:09.539 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:09 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:08:09.539 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:09 smithi100 ceph-mon[30296]: Deploying daemon mgr.z on smithi188 2024-05-10T11:08:09.572 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_3 2024-05-10T11:08:09.628 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-10T11:08:09.628 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T11:08:09.628 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-05-10T11:08:09.628 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T11:08:09.628 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T11:08:09.628 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2024-05-10 11:07:43.749751125 +0000 2024-05-10T11:08:09.628 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2024-05-10 11:02:14.103958274 +0000 2024-05-10T11:08:09.628 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2024-05-10 11:02:14.103958274 +0000 2024-05-10T11:08:09.628 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: 2024-05-10 11:02:14.103958274 +0000 2024-05-10T11:08:09.629 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-10T11:08:09.693 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2024-05-10T11:08:09.694 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2024-05-10T11:08:09.694 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000203453 s, 2.5 MB/s 2024-05-10T11:08:09.695 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-10T11:08:09.751 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_4 2024-05-10T11:08:09.806 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-10T11:08:09.807 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T11:08:09.807 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-05-10T11:08:09.807 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T11:08:09.807 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T11:08:09.807 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2024-05-10 11:07:43.750751107 +0000 2024-05-10T11:08:09.807 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2024-05-10 11:02:14.432952244 +0000 2024-05-10T11:08:09.807 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2024-05-10 11:02:14.432952244 +0000 2024-05-10T11:08:09.807 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: 2024-05-10 11:02:14.432952244 +0000 2024-05-10T11:08:09.807 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-10T11:08:09.870 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2024-05-10T11:08:09.870 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2024-05-10T11:08:09.870 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000184876 s, 2.8 MB/s 2024-05-10T11:08:09.871 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-10T11:08:09.928 INFO:tasks.cephadm:Deploying osd.0 on smithi100 with /dev/vg_nvme/lv_4... 2024-05-10T11:08:09.928 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-10T11:08:10.173 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:08:10.993 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 10 11:08:10 smithi188 systemd[1]: Starting Ceph mgr.z for 20a542cc-0ebd-11ef-bc98-c7b262605968... 2024-05-10T11:08:11.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:11 smithi188 ceph-mon[34075]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:08:11.280 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 10 11:08:10 smithi188 podman[35917]: 2024-05-10 11:08:10.89633005 +0000 UTC m=+0.018131848 image pull ee5e4f21ef58047cb638cccd5cfbe8f6f43412b56c94b1d7fbd9cf3cb90a53a5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f 2024-05-10T11:08:11.280 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 10 11:08:11 smithi188 podman[35917]: 2024-05-10 11:08:11.005524975 +0000 UTC m=+0.127326763 container create 2e5740865245418813b84e800c68aaa6ce3802b46ef7fa7996ea58d8736060c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-05-10T11:08:11.280 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 10 11:08:11 smithi188 podman[35917]: 2024-05-10 11:08:11.16618941 +0000 UTC m=+0.287991197 container init 2e5740865245418813b84e800c68aaa6ce3802b46ef7fa7996ea58d8736060c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-z, GIT_CLEAN=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240507, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-05-10T11:08:11.280 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 10 11:08:11 smithi188 podman[35917]: 2024-05-10 11:08:11.169586391 +0000 UTC m=+0.291388172 container start 2e5740865245418813b84e800c68aaa6ce3802b46ef7fa7996ea58d8736060c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-z, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , ceph=True) 2024-05-10T11:08:11.280 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 10 11:08:11 smithi188 ceph-mgr[35937]: -- 172.21.15.188:0/4031809350 <== mon.1 v2:172.21.15.188:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562b1aa665a0 con 0x562b1aa3ac00 2024-05-10T11:08:11.280 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 10 11:08:11 smithi188 bash[35917]: 2e5740865245418813b84e800c68aaa6ce3802b46ef7fa7996ea58d8736060c5 2024-05-10T11:08:11.280 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 10 11:08:11 smithi188 systemd[1]: Started Ceph mgr.z for 20a542cc-0ebd-11ef-bc98-c7b262605968. 2024-05-10T11:08:11.458 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:11 smithi100 ceph-mon[30296]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:08:11.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:11 smithi178 ceph-mon[33359]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:08:11.532 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 10 11:08:11 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-z[35932]: 2024-05-10T11:08:11.335+0000 7f058a460200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-10T11:08:11.532 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 10 11:08:11 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-z[35932]: 2024-05-10T11:08:11.532+0000 7f058a460200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-10T11:08:11.970 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 10 11:08:11 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-z[35932]: 2024-05-10T11:08:11.639+0000 7f058a460200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-10T11:08:11.970 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 10 11:08:11 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-z[35932]: 2024-05-10T11:08:11.969+0000 7f058a460200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-10T11:08:12.137 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-10T11:08:12.137 INFO:teuthology.orchestra.run.smithi100.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-10T11:08:12.137 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10+0 records in 2024-05-10T11:08:12.137 INFO:teuthology.orchestra.run.smithi100.stderr:10+0 records out 2024-05-10T11:08:12.138 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0129507 s, 810 MB/s 2024-05-10T11:08:12.138 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping successful for: 2024-05-10T11:08:12.278 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 10 11:08:12 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-z[35932]: 2024-05-10T11:08:12.057+0000 7f058a460200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-10T11:08:12.456 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:12 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:12.456 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:12 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:12.456 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:12 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:12.456 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:12 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-10T11:08:12.456 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:12 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-10T11:08:12.456 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:12 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-10T11:08:12.457 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:12 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:08:12.457 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:12 smithi100 ceph-mon[30296]: Deploying daemon mgr.y on smithi178 2024-05-10T11:08:12.561 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:12 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:12.562 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:12 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:12.562 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:12 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:12.562 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:12 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-10T11:08:12.562 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:12 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-10T11:08:12.562 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:12 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-10T11:08:12.562 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:12 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:08:12.562 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:12 smithi188 ceph-mon[34075]: Deploying daemon mgr.y on smithi178 2024-05-10T11:08:12.669 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:12 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:12.670 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:12 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:12.670 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:12 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:12.670 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:12 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-10T11:08:12.670 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:12 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-10T11:08:12.670 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:12 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-10T11:08:12.670 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:12 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:08:12.670 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:12 smithi178 ceph-mon[33359]: Deploying daemon mgr.y on smithi178 2024-05-10T11:08:12.777 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi100:vg_nvme/lv_4 2024-05-10T11:08:13.011 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:08:13.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:13 smithi100 ceph-mon[30296]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:08:13.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:13 smithi188 ceph-mon[34075]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:08:13.529 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 10 11:08:13 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-z[35932]: 2024-05-10T11:08:13.260+0000 7f058a460200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-10T11:08:13.529 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 10 11:08:13 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-z[35932]: 2024-05-10T11:08:13.430+0000 7f058a460200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-10T11:08:13.645 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:13 smithi178 ceph-mon[33359]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:08:13.889 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 10 11:08:13 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-z[35932]: 2024-05-10T11:08:13.603+0000 7f058a460200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-10T11:08:13.889 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 10 11:08:13 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-z[35932]: 2024-05-10T11:08:13.888+0000 7f058a460200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-10T11:08:14.002 INFO:journalctl@ceph.mgr.y.smithi178.stdout:May 10 11:08:13 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-y[34745]: 2024-05-10T11:08:13.908+0000 7f92e435a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-10T11:08:14.190 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 10 11:08:13 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-z[35932]: 2024-05-10T11:08:13.981+0000 7f058a460200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-10T11:08:14.190 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 10 11:08:14 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-z[35932]: 2024-05-10T11:08:14.189+0000 7f058a460200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-10T11:08:14.502 INFO:journalctl@ceph.mgr.y.smithi178.stdout:May 10 11:08:14 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-y[34745]: 2024-05-10T11:08:14.098+0000 7f92e435a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-10T11:08:14.502 INFO:journalctl@ceph.mgr.y.smithi178.stdout:May 10 11:08:14 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-y[34745]: 2024-05-10T11:08:14.205+0000 7f92e435a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-10T11:08:14.528 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 10 11:08:14 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-z[35932]: 2024-05-10T11:08:14.391+0000 7f058a460200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-10T11:08:14.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:14.589+0000 7f38da758700 1 -- 172.21.15.100:0/909288002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38d4103600 msgr2=0x7f38d4103a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:08:14.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:14.589+0000 7f38da758700 1 --2- 172.21.15.100:0/909288002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38d4103600 0x7f38d4103a60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f38bc0099c0 tx=0x7f38bc009cd0 comp rx=0 tx=0).stop 2024-05-10T11:08:14.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:14.589+0000 7f38da758700 1 -- 172.21.15.100:0/909288002 shutdown_connections 2024-05-10T11:08:14.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:14.589+0000 7f38da758700 1 --2- 172.21.15.100:0/909288002 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f38d4103fa0 0x7f38d410a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:08:14.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:14.589+0000 7f38da758700 1 --2- 172.21.15.100:0/909288002 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38d4102400 0x7f38d4102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:08:14.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:14.589+0000 7f38da758700 1 --2- 172.21.15.100:0/909288002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38d4103600 0x7f38d4103a60 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:08:14.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:14.589+0000 7f38da758700 1 -- 172.21.15.100:0/909288002 >> 172.21.15.100:0/909288002 conn(0x7f38d40fd9c0 msgr2=0x7f38d40ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:08:14.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:14.589+0000 7f38da758700 1 -- 172.21.15.100:0/909288002 shutdown_connections 2024-05-10T11:08:14.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:14.589+0000 7f38da758700 1 -- 172.21.15.100:0/909288002 wait complete. 2024-05-10T11:08:14.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:14.590+0000 7f38da758700 1 Processor -- start 2024-05-10T11:08:14.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:14.590+0000 7f38da758700 1 -- start start 2024-05-10T11:08:14.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:14.590+0000 7f38da758700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38d4102400 0x7f38d4199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:08:14.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:14.590+0000 7f38da758700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38d4103600 0x7f38d4199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:08:14.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:14.590+0000 7f38da758700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f38d4103fa0 0x7f38d41a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:08:14.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:14.590+0000 7f38da758700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f38d410dd30 con 0x7f38d4103600 2024-05-10T11:08:14.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:14.590+0000 7f38da758700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f38d410dbb0 con 0x7f38d4103fa0 2024-05-10T11:08:14.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:14.590+0000 7f38da758700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f38d410deb0 con 0x7f38d4102400 2024-05-10T11:08:14.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:14.590+0000 7f38d37fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38d4103600 0x7f38d4199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:08:14.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:14.590+0000 7f38d37fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38d4103600 0x7f38d4199a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40416/0 (socket says 172.21.15.100:40416) 2024-05-10T11:08:14.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:14.590+0000 7f38d37fe700 1 -- 172.21.15.100:0/4216976492 learned_addr learned my addr 172.21.15.100:0/4216976492 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:08:14.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:14.590+0000 7f38d3fff700 1 --2- 172.21.15.100:0/4216976492 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38d4102400 0x7f38d4199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:08:14.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:14.590+0000 7f38d8cf5700 1 --2- 172.21.15.100:0/4216976492 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f38d4103fa0 0x7f38d41a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:08:14.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:14.591+0000 7f38d37fe700 1 -- 172.21.15.100:0/4216976492 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38d4102400 msgr2=0x7f38d4199540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:08:14.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:14.591+0000 7f38d37fe700 1 --2- 172.21.15.100:0/4216976492 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38d4102400 0x7f38d4199540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:08:14.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:14.591+0000 7f38d37fe700 1 -- 172.21.15.100:0/4216976492 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f38d4103fa0 msgr2=0x7f38d41a0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:08:14.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:14.591+0000 7f38d37fe700 1 --2- 172.21.15.100:0/4216976492 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f38d4103fa0 0x7f38d41a0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:08:14.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:14.591+0000 7f38d37fe700 1 -- 172.21.15.100:0/4216976492 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38d41a11a0 con 0x7f38d4103600 2024-05-10T11:08:14.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:14.591+0000 7f38d37fe700 1 --2- 172.21.15.100:0/4216976492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38d4103600 0x7f38d4199a80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f38bc00b580 tx=0x7f38bc00fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:08:14.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:14.591+0000 7f38d17fa700 1 -- 172.21.15.100:0/4216976492 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38bc027070 con 0x7f38d4103600 2024-05-10T11:08:14.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:14.591+0000 7f38da758700 1 -- 172.21.15.100:0/4216976492 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38d41a1430 con 0x7f38d4103600 2024-05-10T11:08:14.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:14.591+0000 7f38d17fa700 1 -- 172.21.15.100:0/4216976492 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f38bc00fe30 con 0x7f38d4103600 2024-05-10T11:08:14.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:14.591+0000 7f38d17fa700 1 -- 172.21.15.100:0/4216976492 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38bc0173f0 con 0x7f38d4103600 2024-05-10T11:08:14.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:14.591+0000 7f38da758700 1 -- 172.21.15.100:0/4216976492 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38d41a1880 con 0x7f38d4103600 2024-05-10T11:08:14.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:14.592+0000 7f38d17fa700 1 -- 172.21.15.100:0/4216976492 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f38bc02a070 con 0x7f38d4103600 2024-05-10T11:08:14.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:14.592+0000 7f38da758700 1 -- 172.21.15.100:0/4216976492 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38d406a190 con 0x7f38d4103600 2024-05-10T11:08:14.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:14.592+0000 7f38d17fa700 1 --2- 172.21.15.100:0/4216976492 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f38c003e750 0x7f38c0040c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:08:14.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:14.592+0000 7f38d17fa700 1 -- 172.21.15.100:0/4216976492 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f38bc053770 con 0x7f38d4103600 2024-05-10T11:08:14.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:14.592+0000 7f38d3fff700 1 --2- 172.21.15.100:0/4216976492 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f38c003e750 0x7f38c0040c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:08:14.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:14.593+0000 7f38d3fff700 1 --2- 172.21.15.100:0/4216976492 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f38c003e750 0x7f38c0040c10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f38d4103460 tx=0x7f38c400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:08:14.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:14.597+0000 7f38d17fa700 1 -- 172.21.15.100:0/4216976492 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f38bc013070 con 0x7f38d4103600 2024-05-10T11:08:14.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:14.809+0000 7f38da758700 1 -- 172.21.15.100:0/4216976492 --> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f38d406c350 con 0x7f38c003e750 2024-05-10T11:08:14.886 INFO:journalctl@ceph.mgr.y.smithi178.stdout:May 10 11:08:14 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-y[34745]: 2024-05-10T11:08:14.533+0000 7f92e435a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-10T11:08:14.887 INFO:journalctl@ceph.mgr.y.smithi178.stdout:May 10 11:08:14 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-y[34745]: 2024-05-10T11:08:14.627+0000 7f92e435a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-10T11:08:15.185 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 10 11:08:15 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-z[35932]: 2024-05-10T11:08:15.030+0000 7f058a460200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-10T11:08:15.185 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:14 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:15.185 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:14 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:15.185 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:14 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:15.185 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:14 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:15.185 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:14 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:08:15.185 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:14 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T11:08:15.186 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:14 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T11:08:15.186 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:14 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:08:15.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:14 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:15.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:14 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:15.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:14 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:15.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:14 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:15.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:14 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:08:15.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:14 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T11:08:15.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:14 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T11:08:15.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:14 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:08:15.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:14 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:15.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:14 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:15.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:14 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:15.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:14 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:15.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:14 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:08:15.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:14 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T11:08:15.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:14 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T11:08:15.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:14 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:08:15.528 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 10 11:08:15 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-z[35932]: 2024-05-10T11:08:15.256+0000 7f058a460200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-10T11:08:15.945 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 10 11:08:15 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-z[35932]: 2024-05-10T11:08:15.691+0000 7f058a460200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-10T11:08:15.945 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 10 11:08:15 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-z[35932]: 2024-05-10T11:08:15.788+0000 7f058a460200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-10T11:08:16.227 INFO:journalctl@ceph.mgr.y.smithi178.stdout:May 10 11:08:15 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-y[34745]: 2024-05-10T11:08:15.882+0000 7f92e435a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-10T11:08:16.227 INFO:journalctl@ceph.mgr.y.smithi178.stdout:May 10 11:08:16 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-y[34745]: 2024-05-10T11:08:16.053+0000 7f92e435a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-10T11:08:16.227 INFO:journalctl@ceph.mgr.y.smithi178.stdout:May 10 11:08:16 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-y[34745]: 2024-05-10T11:08:16.227+0000 7f92e435a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-10T11:08:16.228 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:15 smithi178 ceph-mon[33359]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:08:16.228 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:15 smithi178 ceph-mon[33359]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:08:16.228 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:15 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:16.228 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:15 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:16.228 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:15 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:16.228 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:15 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:16.228 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:15 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:16.228 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:15 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:08:16.228 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:15 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:08:16.228 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:15 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:16.228 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:15 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-10T11:08:16.228 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:15 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-10T11:08:16.228 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:15 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:08:16.278 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 10 11:08:16 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-z[35932]: 2024-05-10T11:08:16.165+0000 7f058a460200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-10T11:08:16.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:15 smithi188 ceph-mon[34075]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:08:16.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:15 smithi188 ceph-mon[34075]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:08:16.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:15 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:16.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:15 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:16.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:15 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:16.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:15 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:16.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:15 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:16.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:15 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:08:16.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:15 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:08:16.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:15 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:16.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:15 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-10T11:08:16.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:15 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-10T11:08:16.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:15 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:08:16.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:15 smithi100 ceph-mon[30296]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:08:16.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:15 smithi100 ceph-mon[30296]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:08:16.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:15 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:16.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:15 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:16.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:15 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:16.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:15 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:16.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:15 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:16.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:15 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:08:16.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:15 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:08:16.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:15 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:16.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:15 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-10T11:08:16.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:15 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-10T11:08:16.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:15 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:08:16.778 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 10 11:08:16 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-z[35932]: 2024-05-10T11:08:16.520+0000 7f058a460200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-10T11:08:16.778 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 10 11:08:16 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-z[35932]: 2024-05-10T11:08:16.612+0000 7f058a460200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-10T11:08:16.812 INFO:journalctl@ceph.mgr.y.smithi178.stdout:May 10 11:08:16 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-y[34745]: 2024-05-10T11:08:16.515+0000 7f92e435a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-10T11:08:16.812 INFO:journalctl@ceph.mgr.y.smithi178.stdout:May 10 11:08:16 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-y[34745]: 2024-05-10T11:08:16.608+0000 7f92e435a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-10T11:08:16.813 INFO:journalctl@ceph.mgr.y.smithi178.stdout:May 10 11:08:16 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-y[34745]: 2024-05-10T11:08:16.812+0000 7f92e435a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-10T11:08:17.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:16 smithi178 ceph-mon[33359]: Reconfiguring mgr.x (unknown last config time)... 2024-05-10T11:08:17.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:16 smithi178 ceph-mon[33359]: Reconfiguring daemon mgr.x on smithi100 2024-05-10T11:08:17.252 INFO:journalctl@ceph.mgr.y.smithi178.stdout:May 10 11:08:17 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-y[34745]: 2024-05-10T11:08:17.005+0000 7f92e435a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-10T11:08:17.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:16 smithi188 ceph-mon[34075]: Reconfiguring mgr.x (unknown last config time)... 2024-05-10T11:08:17.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:16 smithi188 ceph-mon[34075]: Reconfiguring daemon mgr.x on smithi100 2024-05-10T11:08:17.278 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 10 11:08:16 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-z[35932]: 2024-05-10T11:08:16.854+0000 7f058a460200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-10T11:08:17.279 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 10 11:08:16 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-z[35932]: 2024-05-10T11:08:16.989+0000 7f058a460200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-10T11:08:17.279 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 10 11:08:17 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-z[35932]: 2024-05-10T11:08:17.084+0000 7f058a460200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-10T11:08:17.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:16 smithi100 ceph-mon[30296]: Reconfiguring mgr.x (unknown last config time)... 2024-05-10T11:08:17.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:16 smithi100 ceph-mon[30296]: Reconfiguring daemon mgr.x on smithi100 2024-05-10T11:08:17.778 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 10 11:08:17 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-z[35932]: 2024-05-10T11:08:17.414+0000 7f058a460200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-10T11:08:17.778 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 10 11:08:17 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-z[35932]: 2024-05-10T11:08:17.508+0000 7f058a460200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-10T11:08:17.950 INFO:journalctl@ceph.mgr.y.smithi178.stdout:May 10 11:08:17 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-y[34745]: 2024-05-10T11:08:17.608+0000 7f92e435a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-10T11:08:17.950 INFO:journalctl@ceph.mgr.y.smithi178.stdout:May 10 11:08:17 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-y[34745]: 2024-05-10T11:08:17.827+0000 7f92e435a200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-10T11:08:18.249 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:17 smithi178 ceph-mon[33359]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:08:18.249 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:17 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/1183155677' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5f031418-da23-4e85-8669-160960dadd09"}]: dispatch 2024-05-10T11:08:18.249 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:17 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/1183155677' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5f031418-da23-4e85-8669-160960dadd09"}]': finished 2024-05-10T11:08:18.249 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:17 smithi178 ceph-mon[33359]: osdmap e5: 1 total, 0 up, 1 in 2024-05-10T11:08:18.249 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:17 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T11:08:18.278 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 10 11:08:18 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-z[35932]: 2024-05-10T11:08:18.107+0000 7f058a460200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-10T11:08:18.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:17 smithi188 ceph-mon[34075]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:08:18.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:17 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/1183155677' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5f031418-da23-4e85-8669-160960dadd09"}]: dispatch 2024-05-10T11:08:18.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:17 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/1183155677' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5f031418-da23-4e85-8669-160960dadd09"}]': finished 2024-05-10T11:08:18.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:17 smithi188 ceph-mon[34075]: osdmap e5: 1 total, 0 up, 1 in 2024-05-10T11:08:18.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:17 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T11:08:18.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:17 smithi100 ceph-mon[30296]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:08:18.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:17 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/1183155677' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5f031418-da23-4e85-8669-160960dadd09"}]: dispatch 2024-05-10T11:08:18.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:17 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/1183155677' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5f031418-da23-4e85-8669-160960dadd09"}]': finished 2024-05-10T11:08:18.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:17 smithi100 ceph-mon[30296]: osdmap e5: 1 total, 0 up, 1 in 2024-05-10T11:08:18.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:17 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T11:08:18.502 INFO:journalctl@ceph.mgr.y.smithi178.stdout:May 10 11:08:18 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-y[34745]: 2024-05-10T11:08:18.248+0000 7f92e435a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-10T11:08:18.502 INFO:journalctl@ceph.mgr.y.smithi178.stdout:May 10 11:08:18 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-y[34745]: 2024-05-10T11:08:18.345+0000 7f92e435a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-10T11:08:18.778 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 10 11:08:18 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-z[35932]: 2024-05-10T11:08:18.321+0000 7f058a460200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-10T11:08:18.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:18.962+0000 7f38d17fa700 1 -- 172.21.15.100:0/4216976492 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mgrmap(e 14) ==== 99938+0+0 (secure 0 0 0) 0x7f38bc055d10 con 0x7f38d4103600 2024-05-10T11:08:19.002 INFO:journalctl@ceph.mgr.y.smithi178.stdout:May 10 11:08:18 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-y[34745]: 2024-05-10T11:08:18.717+0000 7f92e435a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-10T11:08:19.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:18 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/2533432678' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T11:08:19.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:18 smithi178 ceph-mon[33359]: Standby manager daemon z started 2024-05-10T11:08:19.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:18 smithi178 ceph-mon[33359]: from='mgr.? 172.21.15.188:0/1388262782' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-10T11:08:19.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:18 smithi178 ceph-mon[33359]: from='mgr.? 172.21.15.188:0/1388262782' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-10T11:08:19.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:18 smithi178 ceph-mon[33359]: from='mgr.? 172.21.15.188:0/1388262782' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-10T11:08:19.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:18 smithi178 ceph-mon[33359]: from='mgr.? 172.21.15.188:0/1388262782' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-10T11:08:19.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:18 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/2533432678' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T11:08:19.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:18 smithi188 ceph-mon[34075]: Standby manager daemon z started 2024-05-10T11:08:19.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:18 smithi188 ceph-mon[34075]: from='mgr.? 172.21.15.188:0/1388262782' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-10T11:08:19.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:18 smithi188 ceph-mon[34075]: from='mgr.? 172.21.15.188:0/1388262782' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-10T11:08:19.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:18 smithi188 ceph-mon[34075]: from='mgr.? 172.21.15.188:0/1388262782' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-10T11:08:19.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:18 smithi188 ceph-mon[34075]: from='mgr.? 172.21.15.188:0/1388262782' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-10T11:08:19.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:18 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/2533432678' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T11:08:19.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:18 smithi100 ceph-mon[30296]: Standby manager daemon z started 2024-05-10T11:08:19.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:18 smithi100 ceph-mon[30296]: from='mgr.? 172.21.15.188:0/1388262782' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-10T11:08:19.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:18 smithi100 ceph-mon[30296]: from='mgr.? 172.21.15.188:0/1388262782' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-10T11:08:19.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:18 smithi100 ceph-mon[30296]: from='mgr.? 172.21.15.188:0/1388262782' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-10T11:08:19.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:18 smithi100 ceph-mon[30296]: from='mgr.? 172.21.15.188:0/1388262782' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-10T11:08:19.413 INFO:journalctl@ceph.mgr.y.smithi178.stdout:May 10 11:08:19 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-y[34745]: 2024-05-10T11:08:19.075+0000 7f92e435a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-10T11:08:19.413 INFO:journalctl@ceph.mgr.y.smithi178.stdout:May 10 11:08:19 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-y[34745]: 2024-05-10T11:08:19.168+0000 7f92e435a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-10T11:08:19.752 INFO:journalctl@ceph.mgr.y.smithi178.stdout:May 10 11:08:19 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-y[34745]: 2024-05-10T11:08:19.412+0000 7f92e435a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-10T11:08:19.752 INFO:journalctl@ceph.mgr.y.smithi178.stdout:May 10 11:08:19 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-y[34745]: 2024-05-10T11:08:19.545+0000 7f92e435a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-10T11:08:19.753 INFO:journalctl@ceph.mgr.y.smithi178.stdout:May 10 11:08:19 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-y[34745]: 2024-05-10T11:08:19.640+0000 7f92e435a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-10T11:08:20.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:19 smithi178 ceph-mon[33359]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:08:20.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:19 smithi178 ceph-mon[33359]: mgrmap e14: x(active, since 78s), standbys: z 2024-05-10T11:08:20.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:19 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-10T11:08:20.252 INFO:journalctl@ceph.mgr.y.smithi178.stdout:May 10 11:08:19 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-y[34745]: 2024-05-10T11:08:19.973+0000 7f92e435a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-10T11:08:20.253 INFO:journalctl@ceph.mgr.y.smithi178.stdout:May 10 11:08:20 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-y[34745]: 2024-05-10T11:08:20.064+0000 7f92e435a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-10T11:08:20.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:19 smithi188 ceph-mon[34075]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:08:20.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:19 smithi188 ceph-mon[34075]: mgrmap e14: x(active, since 78s), standbys: z 2024-05-10T11:08:20.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:19 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-10T11:08:20.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:19 smithi100 ceph-mon[30296]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:08:20.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:19 smithi100 ceph-mon[30296]: mgrmap e14: x(active, since 78s), standbys: z 2024-05-10T11:08:20.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:19 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-10T11:08:20.959 INFO:journalctl@ceph.mgr.y.smithi178.stdout:May 10 11:08:20 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-y[34745]: 2024-05-10T11:08:20.664+0000 7f92e435a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-10T11:08:20.959 INFO:journalctl@ceph.mgr.y.smithi178.stdout:May 10 11:08:20 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-y[34745]: 2024-05-10T11:08:20.880+0000 7f92e435a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-10T11:08:20.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:20.970+0000 7f38d17fa700 1 -- 172.21.15.100:0/4216976492 <== mon.0 v2:172.21.15.100:3300/0 8 ==== mgrmap(e 15) ==== 149597+0+0 (secure 0 0 0) 0x7f38bc016cc0 con 0x7f38d4103600 2024-05-10T11:08:21.230 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:20 smithi100 ceph-mon[30296]: Standby manager daemon y started 2024-05-10T11:08:21.230 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:20 smithi100 ceph-mon[30296]: from='mgr.? 172.21.15.178:0/3877240084' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-10T11:08:21.230 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:20 smithi100 ceph-mon[30296]: from='mgr.? 172.21.15.178:0/3877240084' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-10T11:08:21.230 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:20 smithi100 ceph-mon[30296]: from='mgr.? 172.21.15.178:0/3877240084' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-10T11:08:21.230 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:20 smithi100 ceph-mon[30296]: from='mgr.? 172.21.15.178:0/3877240084' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-10T11:08:21.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:20 smithi178 ceph-mon[33359]: Standby manager daemon y started 2024-05-10T11:08:21.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:20 smithi178 ceph-mon[33359]: from='mgr.? 172.21.15.178:0/3877240084' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-10T11:08:21.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:20 smithi178 ceph-mon[33359]: from='mgr.? 172.21.15.178:0/3877240084' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-10T11:08:21.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:20 smithi178 ceph-mon[33359]: from='mgr.? 172.21.15.178:0/3877240084' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-10T11:08:21.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:20 smithi178 ceph-mon[33359]: from='mgr.? 172.21.15.178:0/3877240084' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-10T11:08:21.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:20 smithi188 ceph-mon[34075]: Standby manager daemon y started 2024-05-10T11:08:21.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:20 smithi188 ceph-mon[34075]: from='mgr.? 172.21.15.178:0/3877240084' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-10T11:08:21.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:20 smithi188 ceph-mon[34075]: from='mgr.? 172.21.15.178:0/3877240084' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-10T11:08:21.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:20 smithi188 ceph-mon[34075]: from='mgr.? 172.21.15.178:0/3877240084' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-10T11:08:21.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:20 smithi188 ceph-mon[34075]: from='mgr.? 172.21.15.178:0/3877240084' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-10T11:08:22.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:21 smithi178 ceph-mon[33359]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:08:22.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:21 smithi178 ceph-mon[33359]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-05-10T11:08:22.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:21 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-10T11:08:22.154 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:21 smithi100 ceph-mon[30296]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:08:22.154 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:21 smithi100 ceph-mon[30296]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-05-10T11:08:22.154 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:21 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-10T11:08:22.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:21 smithi188 ceph-mon[34075]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:08:22.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:21 smithi188 ceph-mon[34075]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-05-10T11:08:22.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:21 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-10T11:08:24.211 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:23 smithi100 ceph-mon[30296]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:08:24.211 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:23 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:24.211 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:23 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:24.211 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:23 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:08:24.211 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:23 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:08:24.211 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:23 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:08:24.211 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:23 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:24.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:23 smithi178 ceph-mon[33359]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:08:24.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:23 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:24.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:23 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:24.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:23 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:08:24.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:23 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:08:24.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:23 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:08:24.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:23 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:24.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:23 smithi188 ceph-mon[34075]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:08:24.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:23 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:24.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:23 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:24.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:23 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:08:24.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:23 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:08:24.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:23 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:08:24.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:23 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:26.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:25 smithi178 ceph-mon[33359]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:08:26.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:25 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-10T11:08:26.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:25 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:08:26.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:25 smithi188 ceph-mon[34075]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:08:26.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:25 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-10T11:08:26.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:25 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:08:26.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:25 smithi100 ceph-mon[30296]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:08:26.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:25 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-10T11:08:26.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:25 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:08:27.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:26 smithi178 ceph-mon[33359]: Deploying daemon osd.0 on smithi100 2024-05-10T11:08:27.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:26 smithi188 ceph-mon[34075]: Deploying daemon osd.0 on smithi100 2024-05-10T11:08:27.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:26 smithi100 ceph-mon[30296]: Deploying daemon osd.0 on smithi100 2024-05-10T11:08:28.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:27 smithi178 ceph-mon[33359]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:08:28.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:27 smithi188 ceph-mon[34075]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:08:28.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:27 smithi100 ceph-mon[30296]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:08:29.192 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:29 smithi100 ceph-mon[30296]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:08:29.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:29 smithi178 ceph-mon[33359]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:08:29.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:29 smithi188 ceph-mon[34075]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:08:31.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:31 smithi178 ceph-mon[33359]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:08:31.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:31 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:31.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:31 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:31.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:31 smithi188 ceph-mon[34075]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:08:31.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:31 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:31.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:31 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:31.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:31 smithi100 ceph-mon[30296]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:08:31.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:31 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:31.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:31 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:32.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:32 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:08:32.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:32 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:08:32.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:32 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:08:33.286 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:33 smithi100 ceph-mon[30296]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:08:33.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:33 smithi178 ceph-mon[33359]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:08:33.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:33 smithi188 ceph-mon[34075]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:08:34.183 INFO:teuthology.orchestra.run.smithi100.stdout:Created osd(s) 0 on host 'smithi100' 2024-05-10T11:08:34.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:34.182+0000 7f38d17fa700 1 -- 172.21.15.100:0/4216976492 <== mgr.14152 v2:172.21.15.100:6800/2507287356 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f38d406c350 con 0x7f38c003e750 2024-05-10T11:08:34.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:34.185+0000 7f38da758700 1 -- 172.21.15.100:0/4216976492 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f38c003e750 msgr2=0x7f38c0040c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:08:34.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:34.185+0000 7f38da758700 1 --2- 172.21.15.100:0/4216976492 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f38c003e750 0x7f38c0040c10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f38d4103460 tx=0x7f38c400b410 comp rx=0 tx=0).stop 2024-05-10T11:08:34.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:34.185+0000 7f38da758700 1 -- 172.21.15.100:0/4216976492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38d4103600 msgr2=0x7f38d4199a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:08:34.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:34.185+0000 7f38da758700 1 --2- 172.21.15.100:0/4216976492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38d4103600 0x7f38d4199a80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f38bc00b580 tx=0x7f38bc00fa00 comp rx=0 tx=0).stop 2024-05-10T11:08:34.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:34.185+0000 7f38da758700 1 -- 172.21.15.100:0/4216976492 shutdown_connections 2024-05-10T11:08:34.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:34.185+0000 7f38da758700 1 --2- 172.21.15.100:0/4216976492 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f38c003e750 0x7f38c0040c10 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:08:34.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:34.185+0000 7f38da758700 1 --2- 172.21.15.100:0/4216976492 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f38d4103fa0 0x7f38d41a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:08:34.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:34.185+0000 7f38da758700 1 --2- 172.21.15.100:0/4216976492 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38d4102400 0x7f38d4199540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:08:34.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:34.185+0000 7f38da758700 1 --2- 172.21.15.100:0/4216976492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38d4103600 0x7f38d4199a80 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:08:34.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:34.185+0000 7f38da758700 1 -- 172.21.15.100:0/4216976492 >> 172.21.15.100:0/4216976492 conn(0x7f38d40fd9c0 msgr2=0x7f38d406bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:08:34.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:34.185+0000 7f38da758700 1 -- 172.21.15.100:0/4216976492 shutdown_connections 2024-05-10T11:08:34.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:08:34.185+0000 7f38da758700 1 -- 172.21.15.100:0/4216976492 wait complete. 2024-05-10T11:08:34.912 DEBUG:teuthology.orchestra.run.smithi100:osd.0> sudo journalctl -f -n 0 -u ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@osd.0.service 2024-05-10T11:08:34.914 INFO:tasks.cephadm:Deploying osd.1 on smithi178 with /dev/vg_nvme/lv_4... 2024-05-10T11:08:34.914 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-10T11:08:35.162 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T11:08:35.170 INFO:journalctl@ceph.osd.0.smithi100.stdout:May 10 11:08:35 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-0[42301]: 2024-05-10T11:08:35.038+0000 7f0beeb477c0 -1 osd.0 0 log_to_monitors true 2024-05-10T11:08:35.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:35 smithi178 ceph-mon[33359]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:08:35.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:35 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:35.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:35 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:35.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:35 smithi178 ceph-mon[33359]: from='osd.0 [v2:172.21.15.100:6802/643028278,v1:172.21.15.100:6803/643028278]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-10T11:08:35.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:35 smithi100 ceph-mon[30296]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:08:35.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:35 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:35.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:35 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:35.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:35 smithi100 ceph-mon[30296]: from='osd.0 [v2:172.21.15.100:6802/643028278,v1:172.21.15.100:6803/643028278]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-10T11:08:35.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:35 smithi188 ceph-mon[34075]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:08:35.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:35 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:35.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:35 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:35.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:35 smithi188 ceph-mon[34075]: from='osd.0 [v2:172.21.15.100:6802/643028278,v1:172.21.15.100:6803/643028278]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-10T11:08:36.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:36 smithi100 ceph-mon[30296]: from='osd.0 [v2:172.21.15.100:6802/643028278,v1:172.21.15.100:6803/643028278]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-10T11:08:36.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:36 smithi100 ceph-mon[30296]: osdmap e6: 1 total, 0 up, 1 in 2024-05-10T11:08:36.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:36 smithi100 ceph-mon[30296]: from='osd.0 [v2:172.21.15.100:6802/643028278,v1:172.21.15.100:6803/643028278]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-05-10T11:08:36.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:36 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T11:08:36.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:36 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:36.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:36 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:36.323 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:36 smithi178 ceph-mon[33359]: from='osd.0 [v2:172.21.15.100:6802/643028278,v1:172.21.15.100:6803/643028278]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-10T11:08:36.323 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:36 smithi178 ceph-mon[33359]: osdmap e6: 1 total, 0 up, 1 in 2024-05-10T11:08:36.323 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:36 smithi178 ceph-mon[33359]: from='osd.0 [v2:172.21.15.100:6802/643028278,v1:172.21.15.100:6803/643028278]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-05-10T11:08:36.323 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:36 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T11:08:36.324 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:36 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:36.324 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:36 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:36.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:36 smithi188 ceph-mon[34075]: from='osd.0 [v2:172.21.15.100:6802/643028278,v1:172.21.15.100:6803/643028278]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-10T11:08:36.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:36 smithi188 ceph-mon[34075]: osdmap e6: 1 total, 0 up, 1 in 2024-05-10T11:08:36.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:36 smithi188 ceph-mon[34075]: from='osd.0 [v2:172.21.15.100:6802/643028278,v1:172.21.15.100:6803/643028278]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-05-10T11:08:36.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:36 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T11:08:36.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:36 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:36.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:36 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:36.897 INFO:teuthology.orchestra.run.smithi178.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-10T11:08:36.897 INFO:teuthology.orchestra.run.smithi178.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-10T11:08:36.897 INFO:teuthology.orchestra.run.smithi178.stderr: stderr: 10+0 records in 2024-05-10T11:08:36.897 INFO:teuthology.orchestra.run.smithi178.stderr:10+0 records out 2024-05-10T11:08:36.897 INFO:teuthology.orchestra.run.smithi178.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00834651 s, 1.3 GB/s 2024-05-10T11:08:36.897 INFO:teuthology.orchestra.run.smithi178.stderr:--> Zapping successful for: 2024-05-10T11:08:37.192 INFO:journalctl@ceph.osd.0.smithi100.stdout:May 10 11:08:36 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-0[42301]: 2024-05-10T11:08:36.877+0000 7f0be49ac700 -1 osd.0 0 waiting for initial osdmap 2024-05-10T11:08:37.192 INFO:journalctl@ceph.osd.0.smithi100.stdout:May 10 11:08:36 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-0[42301]: 2024-05-10T11:08:36.882+0000 7f0bdbf96700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T11:08:37.444 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:37 smithi100 ceph-mon[30296]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:08:37.444 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:37 smithi100 ceph-mon[30296]: from='osd.0 [v2:172.21.15.100:6802/643028278,v1:172.21.15.100:6803/643028278]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-05-10T11:08:37.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:37 smithi100 ceph-mon[30296]: osdmap e7: 1 total, 0 up, 1 in 2024-05-10T11:08:37.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:37 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T11:08:37.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:37 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T11:08:37.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:37 smithi100 ceph-mon[30296]: from='osd.0 [v2:172.21.15.100:6802/643028278,v1:172.21.15.100:6803/643028278]' entity='osd.0' 2024-05-10T11:08:37.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:37 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T11:08:37.454 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:37 smithi178 ceph-mon[33359]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:08:37.454 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:37 smithi178 ceph-mon[33359]: from='osd.0 [v2:172.21.15.100:6802/643028278,v1:172.21.15.100:6803/643028278]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-05-10T11:08:37.454 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:37 smithi178 ceph-mon[33359]: osdmap e7: 1 total, 0 up, 1 in 2024-05-10T11:08:37.454 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:37 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T11:08:37.454 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:37 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T11:08:37.454 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:37 smithi178 ceph-mon[33359]: from='osd.0 [v2:172.21.15.100:6802/643028278,v1:172.21.15.100:6803/643028278]' entity='osd.0' 2024-05-10T11:08:37.454 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:37 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T11:08:37.480 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi178:vg_nvme/lv_4 2024-05-10T11:08:37.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:37 smithi188 ceph-mon[34075]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:08:37.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:37 smithi188 ceph-mon[34075]: from='osd.0 [v2:172.21.15.100:6802/643028278,v1:172.21.15.100:6803/643028278]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-05-10T11:08:37.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:37 smithi188 ceph-mon[34075]: osdmap e7: 1 total, 0 up, 1 in 2024-05-10T11:08:37.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:37 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T11:08:37.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:37 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T11:08:37.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:37 smithi188 ceph-mon[34075]: from='osd.0 [v2:172.21.15.100:6802/643028278,v1:172.21.15.100:6803/643028278]' entity='osd.0' 2024-05-10T11:08:37.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:37 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T11:08:37.725 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T11:08:38.960 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:38 smithi100 ceph-mon[30296]: purged_snaps scrub starts 2024-05-10T11:08:38.960 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:38 smithi100 ceph-mon[30296]: purged_snaps scrub ok 2024-05-10T11:08:38.961 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:38 smithi100 ceph-mon[30296]: osd.0 [v2:172.21.15.100:6802/643028278,v1:172.21.15.100:6803/643028278] boot 2024-05-10T11:08:38.961 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:38 smithi100 ceph-mon[30296]: osdmap e8: 1 total, 1 up, 1 in 2024-05-10T11:08:38.961 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:38 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T11:08:39.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:38 smithi178 ceph-mon[33359]: purged_snaps scrub starts 2024-05-10T11:08:39.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:38 smithi178 ceph-mon[33359]: purged_snaps scrub ok 2024-05-10T11:08:39.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:38 smithi178 ceph-mon[33359]: osd.0 [v2:172.21.15.100:6802/643028278,v1:172.21.15.100:6803/643028278] boot 2024-05-10T11:08:39.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:38 smithi178 ceph-mon[33359]: osdmap e8: 1 total, 1 up, 1 in 2024-05-10T11:08:39.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:38 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T11:08:39.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:08:39.064+0000 7fee56054700 1 -- 172.21.15.178:0/86241103 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee50105130 msgr2=0x7fee5010b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:08:39.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:08:39.064+0000 7fee56054700 1 --2- 172.21.15.178:0/86241103 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee50105130 0x7fee5010b9e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fee4800b0a0 tx=0x7fee4800b3b0 comp rx=0 tx=0).stop 2024-05-10T11:08:39.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:08:39.064+0000 7fee56054700 1 -- 172.21.15.178:0/86241103 shutdown_connections 2024-05-10T11:08:39.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:08:39.064+0000 7fee56054700 1 --2- 172.21.15.178:0/86241103 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee50105130 0x7fee5010b9e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:08:39.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:08:39.064+0000 7fee56054700 1 --2- 172.21.15.178:0/86241103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee501017b0 0x7fee50101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:08:39.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:08:39.064+0000 7fee56054700 1 --2- 172.21.15.178:0/86241103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee50104790 0x7fee50104bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:08:39.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:08:39.064+0000 7fee56054700 1 -- 172.21.15.178:0/86241103 >> 172.21.15.178:0/86241103 conn(0x7fee500fd130 msgr2=0x7fee500ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:08:39.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:08:39.064+0000 7fee56054700 1 -- 172.21.15.178:0/86241103 shutdown_connections 2024-05-10T11:08:39.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:08:39.064+0000 7fee56054700 1 -- 172.21.15.178:0/86241103 wait complete. 2024-05-10T11:08:39.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:08:39.065+0000 7fee56054700 1 Processor -- start 2024-05-10T11:08:39.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:08:39.065+0000 7fee56054700 1 -- start start 2024-05-10T11:08:39.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:08:39.065+0000 7fee56054700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee501017b0 0x7fee501996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:08:39.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:08:39.065+0000 7fee56054700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee50104790 0x7fee50199c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:08:39.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:08:39.065+0000 7fee56054700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee50105130 0x7fee501a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:08:39.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:08:39.065+0000 7fee56054700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fee5010ef70 con 0x7fee50104790 2024-05-10T11:08:39.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:08:39.065+0000 7fee56054700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fee5010edf0 con 0x7fee501017b0 2024-05-10T11:08:39.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:08:39.065+0000 7fee56054700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fee5010f0f0 con 0x7fee50105130 2024-05-10T11:08:39.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:08:39.065+0000 7fee4f7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee501017b0 0x7fee501996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:08:39.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:08:39.065+0000 7fee4f7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee501017b0 0x7fee501996c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:56032/0 (socket says 172.21.15.178:56032) 2024-05-10T11:08:39.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:08:39.065+0000 7fee4f7fe700 1 -- 172.21.15.178:0/1948968986 learned_addr learned my addr 172.21.15.178:0/1948968986 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-05-10T11:08:39.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:08:39.065+0000 7fee4ffff700 1 --2- 172.21.15.178:0/1948968986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee50105130 0x7fee501a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:08:39.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:08:39.065+0000 7fee4f7fe700 1 -- 172.21.15.178:0/1948968986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee50105130 msgr2=0x7fee501a0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:08:39.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:08:39.065+0000 7fee4f7fe700 1 --2- 172.21.15.178:0/1948968986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee50105130 0x7fee501a0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:08:39.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:08:39.066+0000 7fee4f7fe700 1 -- 172.21.15.178:0/1948968986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee50104790 msgr2=0x7fee50199c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:08:39.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:08:39.066+0000 7fee46ffd700 1 --2- 172.21.15.178:0/1948968986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee50104790 0x7fee50199c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:08:39.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:08:39.066+0000 7fee4f7fe700 1 --2- 172.21.15.178:0/1948968986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee50104790 0x7fee50199c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:08:39.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:08:39.066+0000 7fee4f7fe700 1 -- 172.21.15.178:0/1948968986 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee501a1320 con 0x7fee501017b0 2024-05-10T11:08:39.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:08:39.066+0000 7fee4ffff700 1 --2- 172.21.15.178:0/1948968986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee50105130 0x7fee501a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:08:39.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:08:39.066+0000 7fee4f7fe700 1 --2- 172.21.15.178:0/1948968986 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee501017b0 0x7fee501996c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fee4000b500 tx=0x7fee4000b8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:08:39.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:08:39.066+0000 7fee4d7fa700 1 -- 172.21.15.178:0/1948968986 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee40015020 con 0x7fee501017b0 2024-05-10T11:08:39.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:08:39.066+0000 7fee4d7fa700 1 -- 172.21.15.178:0/1948968986 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fee40011cc0 con 0x7fee501017b0 2024-05-10T11:08:39.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:08:39.066+0000 7fee56054700 1 -- 172.21.15.178:0/1948968986 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee501a1610 con 0x7fee501017b0 2024-05-10T11:08:39.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:08:39.066+0000 7fee4d7fa700 1 -- 172.21.15.178:0/1948968986 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee40010cd0 con 0x7fee501017b0 2024-05-10T11:08:39.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:08:39.066+0000 7fee56054700 1 -- 172.21.15.178:0/1948968986 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee501a1c20 con 0x7fee501017b0 2024-05-10T11:08:39.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:08:39.067+0000 7fee56054700 1 -- 172.21.15.178:0/1948968986 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee5006a190 con 0x7fee501017b0 2024-05-10T11:08:39.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:08:39.068+0000 7fee4d7fa700 1 -- 172.21.15.178:0/1948968986 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 15) ==== 149597+0+0 (secure 0 0 0) 0x7fee40018020 con 0x7fee501017b0 2024-05-10T11:08:39.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:08:39.069+0000 7fee4d7fa700 1 --2- 172.21.15.178:0/1948968986 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fee3c0b2870 0x7fee3c0b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:08:39.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:08:39.069+0000 7fee4d7fa700 1 -- 172.21.15.178:0/1948968986 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7fee400e0480 con 0x7fee501017b0 2024-05-10T11:08:39.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:08:39.069+0000 7fee46ffd700 1 --2- 172.21.15.178:0/1948968986 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fee3c0b2870 0x7fee3c0b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:08:39.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:08:39.070+0000 7fee46ffd700 1 --2- 172.21.15.178:0/1948968986 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fee3c0b2870 0x7fee3c0b4d30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fee38006fd0 tx=0x7fee38008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:08:39.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:08:39.073+0000 7fee4d7fa700 1 -- 172.21.15.178:0/1948968986 <== mon.2 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fee4001f020 con 0x7fee501017b0 2024-05-10T11:08:39.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:38 smithi188 ceph-mon[34075]: purged_snaps scrub starts 2024-05-10T11:08:39.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:38 smithi188 ceph-mon[34075]: purged_snaps scrub ok 2024-05-10T11:08:39.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:38 smithi188 ceph-mon[34075]: osd.0 [v2:172.21.15.100:6802/643028278,v1:172.21.15.100:6803/643028278] boot 2024-05-10T11:08:39.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:38 smithi188 ceph-mon[34075]: osdmap e8: 1 total, 1 up, 1 in 2024-05-10T11:08:39.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:38 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T11:08:39.289 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:08:39.289+0000 7fee56054700 1 -- 172.21.15.178:0/1948968986 --> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fee500638e0 con 0x7fee3c0b2870 2024-05-10T11:08:40.180 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:39 smithi188 ceph-mon[34075]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:08:40.180 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:39 smithi188 ceph-mon[34075]: osdmap e9: 1 total, 1 up, 1 in 2024-05-10T11:08:40.180 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:39 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:40.180 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:39 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:40.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:39 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:08:40.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:39 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:40.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:39 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:08:40.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:39 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:08:40.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:39 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:40.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:39 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:08:40.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:39 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T11:08:40.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:39 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T11:08:40.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:39 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:08:40.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:39 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:08:40.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:39 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:08:40.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:39 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:40.192 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:39 smithi178 ceph-mon[33359]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:08:40.192 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:39 smithi178 ceph-mon[33359]: osdmap e9: 1 total, 1 up, 1 in 2024-05-10T11:08:40.192 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:39 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:40.192 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:39 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:40.193 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:39 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:08:40.193 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:39 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:40.193 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:39 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:08:40.193 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:39 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:08:40.193 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:39 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:40.193 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:39 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:08:40.193 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:39 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T11:08:40.193 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:39 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T11:08:40.193 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:39 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:08:40.193 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:39 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:08:40.193 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:39 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:08:40.193 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:39 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:40.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:39 smithi100 ceph-mon[30296]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T11:08:40.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:39 smithi100 ceph-mon[30296]: osdmap e9: 1 total, 1 up, 1 in 2024-05-10T11:08:40.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:39 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:40.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:39 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:40.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:39 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:08:40.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:39 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:40.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:39 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:08:40.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:39 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:08:40.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:39 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:40.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:39 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:08:40.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:39 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T11:08:40.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:39 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T11:08:40.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:39 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:08:40.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:39 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:08:40.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:39 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:08:40.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:39 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:41.110 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:40 smithi178 ceph-mon[33359]: Detected new or changed devices on smithi100 2024-05-10T11:08:41.110 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:40 smithi178 ceph-mon[33359]: Adjusting osd_memory_target on smithi100 to 17088M 2024-05-10T11:08:41.110 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:40 smithi178 ceph-mon[33359]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:08:41.198 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:40 smithi188 ceph-mon[34075]: Detected new or changed devices on smithi100 2024-05-10T11:08:41.198 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:40 smithi188 ceph-mon[34075]: Adjusting osd_memory_target on smithi100 to 17088M 2024-05-10T11:08:41.198 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:40 smithi188 ceph-mon[34075]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:08:41.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:40 smithi100 ceph-mon[30296]: Detected new or changed devices on smithi100 2024-05-10T11:08:41.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:40 smithi100 ceph-mon[30296]: Adjusting osd_memory_target on smithi100 to 17088M 2024-05-10T11:08:41.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:40 smithi100 ceph-mon[30296]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:08:42.243 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:41 smithi178 ceph-mon[33359]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T11:08:42.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:41 smithi188 ceph-mon[34075]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T11:08:42.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:41 smithi100 ceph-mon[30296]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T11:08:43.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:42 smithi178 ceph-mon[33359]: from='client.? 172.21.15.178:0/3577137556' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a908cdc9-001e-4b65-9ab0-0dd08dba20f3"}]: dispatch 2024-05-10T11:08:43.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:42 smithi178 ceph-mon[33359]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a908cdc9-001e-4b65-9ab0-0dd08dba20f3"}]: dispatch 2024-05-10T11:08:43.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:42 smithi178 ceph-mon[33359]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a908cdc9-001e-4b65-9ab0-0dd08dba20f3"}]': finished 2024-05-10T11:08:43.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:42 smithi178 ceph-mon[33359]: osdmap e10: 2 total, 1 up, 2 in 2024-05-10T11:08:43.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:42 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T11:08:43.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:42 smithi178 ceph-mon[33359]: from='client.? 172.21.15.178:0/1736580387' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T11:08:43.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:42 smithi188 ceph-mon[34075]: from='client.? 172.21.15.178:0/3577137556' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a908cdc9-001e-4b65-9ab0-0dd08dba20f3"}]: dispatch 2024-05-10T11:08:43.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:42 smithi188 ceph-mon[34075]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a908cdc9-001e-4b65-9ab0-0dd08dba20f3"}]: dispatch 2024-05-10T11:08:43.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:42 smithi188 ceph-mon[34075]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a908cdc9-001e-4b65-9ab0-0dd08dba20f3"}]': finished 2024-05-10T11:08:43.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:42 smithi188 ceph-mon[34075]: osdmap e10: 2 total, 1 up, 2 in 2024-05-10T11:08:43.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:42 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T11:08:43.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:42 smithi188 ceph-mon[34075]: from='client.? 172.21.15.178:0/1736580387' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T11:08:43.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:42 smithi100 ceph-mon[30296]: from='client.? 172.21.15.178:0/3577137556' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a908cdc9-001e-4b65-9ab0-0dd08dba20f3"}]: dispatch 2024-05-10T11:08:43.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:42 smithi100 ceph-mon[30296]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a908cdc9-001e-4b65-9ab0-0dd08dba20f3"}]: dispatch 2024-05-10T11:08:43.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:42 smithi100 ceph-mon[30296]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a908cdc9-001e-4b65-9ab0-0dd08dba20f3"}]': finished 2024-05-10T11:08:43.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:42 smithi100 ceph-mon[30296]: osdmap e10: 2 total, 1 up, 2 in 2024-05-10T11:08:43.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:42 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T11:08:43.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:42 smithi100 ceph-mon[30296]: from='client.? 172.21.15.178:0/1736580387' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T11:08:44.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:43 smithi178 ceph-mon[33359]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T11:08:44.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:43 smithi188 ceph-mon[34075]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T11:08:44.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:43 smithi100 ceph-mon[30296]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T11:08:46.244 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:45 smithi178 ceph-mon[33359]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T11:08:46.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:45 smithi188 ceph-mon[34075]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T11:08:46.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:45 smithi100 ceph-mon[30296]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T11:08:48.215 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:47 smithi178 ceph-mon[33359]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T11:08:48.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:47 smithi188 ceph-mon[34075]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T11:08:48.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:47 smithi100 ceph-mon[30296]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T11:08:49.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:48 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-10T11:08:49.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:48 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:08:49.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:48 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-10T11:08:49.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:48 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:08:49.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:48 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-10T11:08:49.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:48 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:08:50.219 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:49 smithi178 ceph-mon[33359]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T11:08:50.219 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:49 smithi178 ceph-mon[33359]: Deploying daemon osd.1 on smithi178 2024-05-10T11:08:50.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:49 smithi188 ceph-mon[34075]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T11:08:50.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:49 smithi188 ceph-mon[34075]: Deploying daemon osd.1 on smithi178 2024-05-10T11:08:50.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:49 smithi100 ceph-mon[30296]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T11:08:50.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:49 smithi100 ceph-mon[30296]: Deploying daemon osd.1 on smithi178 2024-05-10T11:08:52.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:51 smithi178 ceph-mon[33359]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T11:08:52.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:51 smithi188 ceph-mon[34075]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T11:08:52.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:51 smithi100 ceph-mon[30296]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T11:08:54.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:53 smithi178 ceph-mon[33359]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T11:08:54.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:53 smithi188 ceph-mon[34075]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T11:08:54.742 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:53 smithi100 ceph-mon[30296]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T11:08:55.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:55 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:55.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:55 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:55.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:55 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:08:55.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:55 smithi188 ceph-mon[34075]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T11:08:55.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:55 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:55.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:55 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:55.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:55 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:08:55.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:55 smithi100 ceph-mon[30296]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T11:08:55.375 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:55 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:55.375 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:55 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:55.375 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:55 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:08:55.375 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:55 smithi178 ceph-mon[33359]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T11:08:56.650 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:08:56.650+0000 7fee4d7fa700 1 -- 172.21.15.178:0/1948968986 <== mgr.14152 v2:172.21.15.100:6800/2507287356 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fee500638e0 con 0x7fee3c0b2870 2024-05-10T11:08:56.650 INFO:teuthology.orchestra.run.smithi178.stdout:Created osd(s) 1 on host 'smithi178' 2024-05-10T11:08:56.652 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:08:56.652+0000 7fee56054700 1 -- 172.21.15.178:0/1948968986 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fee3c0b2870 msgr2=0x7fee3c0b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:08:56.652 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:08:56.652+0000 7fee56054700 1 --2- 172.21.15.178:0/1948968986 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fee3c0b2870 0x7fee3c0b4d30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fee38006fd0 tx=0x7fee38008040 comp rx=0 tx=0).stop 2024-05-10T11:08:56.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:08:56.652+0000 7fee56054700 1 -- 172.21.15.178:0/1948968986 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee501017b0 msgr2=0x7fee501996c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:08:56.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:08:56.652+0000 7fee56054700 1 --2- 172.21.15.178:0/1948968986 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee501017b0 0x7fee501996c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fee4000b500 tx=0x7fee4000b8c0 comp rx=0 tx=0).stop 2024-05-10T11:08:56.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:08:56.652+0000 7fee56054700 1 -- 172.21.15.178:0/1948968986 shutdown_connections 2024-05-10T11:08:56.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:08:56.652+0000 7fee56054700 1 --2- 172.21.15.178:0/1948968986 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fee3c0b2870 0x7fee3c0b4d30 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:08:56.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:08:56.652+0000 7fee56054700 1 --2- 172.21.15.178:0/1948968986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee50105130 0x7fee501a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:08:56.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:08:56.652+0000 7fee56054700 1 --2- 172.21.15.178:0/1948968986 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee501017b0 0x7fee501996c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:08:56.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:08:56.652+0000 7fee56054700 1 --2- 172.21.15.178:0/1948968986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee50104790 0x7fee50199c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:08:56.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:08:56.652+0000 7fee56054700 1 -- 172.21.15.178:0/1948968986 >> 172.21.15.178:0/1948968986 conn(0x7fee500fd130 msgr2=0x7fee50107bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:08:56.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:08:56.652+0000 7fee56054700 1 -- 172.21.15.178:0/1948968986 shutdown_connections 2024-05-10T11:08:56.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:08:56.652+0000 7fee56054700 1 -- 172.21.15.178:0/1948968986 wait complete. 2024-05-10T11:08:57.431 DEBUG:teuthology.orchestra.run.smithi178:osd.1> sudo journalctl -f -n 0 -u ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@osd.1.service 2024-05-10T11:08:57.434 INFO:tasks.cephadm:Deploying osd.2 on smithi188 with /dev/vg_nvme/lv_4... 2024-05-10T11:08:57.434 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-10T11:08:57.684 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T11:08:57.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:57 smithi178 ceph-mon[33359]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T11:08:57.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:57 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:57.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:57 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:57.754 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:57 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:57.754 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:57 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:57.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:57 smithi188 ceph-mon[34075]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T11:08:57.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:57 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:57.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:57 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:57.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:57 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:57.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:57 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:58.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:57 smithi100 ceph-mon[30296]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T11:08:58.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:57 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:58.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:57 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:58.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:57 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:58.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:57 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:08:58.234 INFO:journalctl@ceph.osd.1.smithi178.stdout:May 10 11:08:57 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-1[37942]: 2024-05-10T11:08:57.901+0000 7f9018be07c0 -1 osd.1 0 log_to_monitors true 2024-05-10T11:08:58.949 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:58 smithi188 ceph-mon[34075]: from='osd.1 [v2:172.21.15.178:6800/3103987017,v1:172.21.15.178:6801/3103987017]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-10T11:08:58.949 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:58 smithi188 ceph-mon[34075]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-10T11:08:59.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:58 smithi178 ceph-mon[33359]: from='osd.1 [v2:172.21.15.178:6800/3103987017,v1:172.21.15.178:6801/3103987017]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-10T11:08:59.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:58 smithi178 ceph-mon[33359]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-10T11:08:59.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:58 smithi100 ceph-mon[30296]: from='osd.1 [v2:172.21.15.178:6800/3103987017,v1:172.21.15.178:6801/3103987017]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-10T11:08:59.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:58 smithi100 ceph-mon[30296]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-10T11:08:59.547 INFO:teuthology.orchestra.run.smithi188.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-10T11:08:59.547 INFO:teuthology.orchestra.run.smithi188.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-10T11:08:59.547 INFO:teuthology.orchestra.run.smithi188.stderr: stderr: 10+0 records in 2024-05-10T11:08:59.547 INFO:teuthology.orchestra.run.smithi188.stderr:10+0 records out 2024-05-10T11:08:59.547 INFO:teuthology.orchestra.run.smithi188.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112286 s, 934 MB/s 2024-05-10T11:08:59.547 INFO:teuthology.orchestra.run.smithi188.stderr:--> Zapping successful for: 2024-05-10T11:08:59.872 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:59 smithi188 ceph-mon[34075]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T11:08:59.872 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:59 smithi188 ceph-mon[34075]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-10T11:08:59.872 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:59 smithi188 ceph-mon[34075]: from='osd.1 [v2:172.21.15.178:6800/3103987017,v1:172.21.15.178:6801/3103987017]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-05-10T11:08:59.872 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:59 smithi188 ceph-mon[34075]: osdmap e11: 2 total, 1 up, 2 in 2024-05-10T11:08:59.872 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:59 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T11:08:59.872 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:08:59 smithi188 ceph-mon[34075]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-05-10T11:09:00.006 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:59 smithi178 ceph-mon[33359]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T11:09:00.006 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:59 smithi178 ceph-mon[33359]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-10T11:09:00.006 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:59 smithi178 ceph-mon[33359]: from='osd.1 [v2:172.21.15.178:6800/3103987017,v1:172.21.15.178:6801/3103987017]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-05-10T11:09:00.006 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:59 smithi178 ceph-mon[33359]: osdmap e11: 2 total, 1 up, 2 in 2024-05-10T11:09:00.006 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:59 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T11:09:00.006 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:08:59 smithi178 ceph-mon[33359]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-05-10T11:09:00.006 INFO:journalctl@ceph.osd.1.smithi178.stdout:May 10 11:08:59 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-1[37942]: 2024-05-10T11:08:59.620+0000 7f900e244700 -1 osd.1 0 waiting for initial osdmap 2024-05-10T11:09:00.006 INFO:journalctl@ceph.osd.1.smithi178.stdout:May 10 11:08:59 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-1[37942]: 2024-05-10T11:08:59.625+0000 7f900a037700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T11:09:00.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:59 smithi100 ceph-mon[30296]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T11:09:00.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:59 smithi100 ceph-mon[30296]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-10T11:09:00.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:59 smithi100 ceph-mon[30296]: from='osd.1 [v2:172.21.15.178:6800/3103987017,v1:172.21.15.178:6801/3103987017]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-05-10T11:09:00.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:59 smithi100 ceph-mon[30296]: osdmap e11: 2 total, 1 up, 2 in 2024-05-10T11:09:00.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:59 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T11:09:00.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:08:59 smithi100 ceph-mon[30296]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-05-10T11:09:00.148 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi188:vg_nvme/lv_4 2024-05-10T11:09:00.396 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T11:09:00.728 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:00 smithi188 ceph-mon[34075]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-05-10T11:09:00.728 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:00 smithi188 ceph-mon[34075]: osdmap e12: 2 total, 1 up, 2 in 2024-05-10T11:09:00.729 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:00 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T11:09:00.729 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:00 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T11:09:00.743 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:00 smithi178 ceph-mon[33359]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-05-10T11:09:00.743 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:00 smithi178 ceph-mon[33359]: osdmap e12: 2 total, 1 up, 2 in 2024-05-10T11:09:00.743 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:00 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T11:09:00.743 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:00 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T11:09:01.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:00 smithi100 ceph-mon[30296]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-05-10T11:09:01.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:00 smithi100 ceph-mon[30296]: osdmap e12: 2 total, 1 up, 2 in 2024-05-10T11:09:01.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:00 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T11:09:01.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:00 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T11:09:01.743 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:01 smithi188 ceph-mon[34075]: purged_snaps scrub starts 2024-05-10T11:09:01.743 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:01 smithi188 ceph-mon[34075]: purged_snaps scrub ok 2024-05-10T11:09:01.743 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:01 smithi188 ceph-mon[34075]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T11:09:01.743 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:01 smithi188 ceph-mon[34075]: osd.1 [v2:172.21.15.178:6800/3103987017,v1:172.21.15.178:6801/3103987017] boot 2024-05-10T11:09:01.744 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:01 smithi188 ceph-mon[34075]: osdmap e13: 2 total, 2 up, 2 in 2024-05-10T11:09:01.744 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:01 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T11:09:01.744 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:01 smithi188 ceph-mon[34075]: Detected new or changed devices on smithi178 2024-05-10T11:09:01.744 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:01 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:01.744 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:01 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:01.744 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:01 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:09:01.744 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:01 smithi188 ceph-mon[34075]: Adjusting osd_memory_target on smithi178 to 17088M 2024-05-10T11:09:01.744 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:01 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:01.744 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:01 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:09:01.745 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:01 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:09:01.745 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:01 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:01.745 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:01 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:09:01.745 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:01 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:09:01.745 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:01 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:09:01.745 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:01 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:01.985 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:09:01.981+0000 7f333f790700 1 -- 172.21.15.188:0/2658269567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3338103fa0 msgr2=0x7f333810a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:09:01.986 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:09:01.981+0000 7f333f790700 1 --2- 172.21.15.188:0/2658269567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3338103fa0 0x7f333810a850 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f333000b0a0 tx=0x7f333000b3b0 comp rx=0 tx=0).stop 2024-05-10T11:09:01.986 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:09:01.981+0000 7f333f790700 1 -- 172.21.15.188:0/2658269567 shutdown_connections 2024-05-10T11:09:01.986 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:09:01.981+0000 7f333f790700 1 --2- 172.21.15.188:0/2658269567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3338103fa0 0x7f333810a850 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:09:01.986 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:09:01.981+0000 7f333f790700 1 --2- 172.21.15.188:0/2658269567 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3338102400 0x7f3338102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:09:01.986 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:09:01.981+0000 7f333f790700 1 --2- 172.21.15.188:0/2658269567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3338103600 0x7f3338103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:09:01.986 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:09:01.981+0000 7f333f790700 1 -- 172.21.15.188:0/2658269567 >> 172.21.15.188:0/2658269567 conn(0x7f33380fd9a0 msgr2=0x7f33380ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:09:01.986 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:09:01.981+0000 7f333f790700 1 -- 172.21.15.188:0/2658269567 shutdown_connections 2024-05-10T11:09:01.987 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:09:01.981+0000 7f333f790700 1 -- 172.21.15.188:0/2658269567 wait complete. 2024-05-10T11:09:01.987 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:09:01.982+0000 7f333f790700 1 Processor -- start 2024-05-10T11:09:01.987 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:09:01.982+0000 7f333f790700 1 -- start start 2024-05-10T11:09:01.987 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:09:01.982+0000 7f333f790700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3338102400 0x7f333806bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:09:01.987 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:09:01.982+0000 7f333f790700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3338103600 0x7f33381a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:09:01.987 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:09:01.982+0000 7f333f790700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3338103fa0 0x7f33381a2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:09:01.988 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:09:01.982+0000 7f333f790700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f333810de20 con 0x7f3338103fa0 2024-05-10T11:09:01.988 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:09:01.982+0000 7f333f790700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f333810dca0 con 0x7f3338103600 2024-05-10T11:09:01.988 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:09:01.982+0000 7f333f790700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f333810dfa0 con 0x7f3338102400 2024-05-10T11:09:01.988 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:09:01.982+0000 7f333d52c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3338102400 0x7f333806bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:09:01.988 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:09:01.982+0000 7f333d52c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3338102400 0x7f333806bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:52100/0 (socket says 172.21.15.188:52100) 2024-05-10T11:09:01.988 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:09:01.982+0000 7f333d52c700 1 -- 172.21.15.188:0/162804944 learned_addr learned my addr 172.21.15.188:0/162804944 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-05-10T11:09:01.988 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:09:01.982+0000 7f333dd2d700 1 --2- 172.21.15.188:0/162804944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3338103fa0 0x7f33381a2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:09:01.989 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:09:01.982+0000 7f333d52c700 1 -- 172.21.15.188:0/162804944 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3338103600 msgr2=0x7f33381a0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:09:01.989 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:09:01.982+0000 7f333cd2b700 1 --2- 172.21.15.188:0/162804944 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3338103600 0x7f33381a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:09:01.989 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:09:01.982+0000 7f333d52c700 1 --2- 172.21.15.188:0/162804944 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3338103600 0x7f33381a0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:09:01.989 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:09:01.982+0000 7f333d52c700 1 -- 172.21.15.188:0/162804944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3338103fa0 msgr2=0x7f33381a2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:09:01.989 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:09:01.982+0000 7f333d52c700 1 --2- 172.21.15.188:0/162804944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3338103fa0 0x7f33381a2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:09:01.989 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:09:01.982+0000 7f333d52c700 1 -- 172.21.15.188:0/162804944 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33381a3440 con 0x7f3338102400 2024-05-10T11:09:01.989 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:09:01.982+0000 7f333dd2d700 1 --2- 172.21.15.188:0/162804944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3338103fa0 0x7f33381a2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:09:01.989 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:09:01.982+0000 7f333cd2b700 1 --2- 172.21.15.188:0/162804944 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3338103600 0x7f33381a0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:09:01.989 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:09:01.983+0000 7f333d52c700 1 --2- 172.21.15.188:0/162804944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3338102400 0x7f333806bce0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f332c00b500 tx=0x7f332c00b8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:09:01.989 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:09:01.983+0000 7f332a7fc700 1 -- 172.21.15.188:0/162804944 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f332c015020 con 0x7f3338102400 2024-05-10T11:09:01.989 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:09:01.983+0000 7f332a7fc700 1 -- 172.21.15.188:0/162804944 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f332c011cc0 con 0x7f3338102400 2024-05-10T11:09:01.989 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:09:01.983+0000 7f333f790700 1 -- 172.21.15.188:0/162804944 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33381a3730 con 0x7f3338102400 2024-05-10T11:09:01.989 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:09:01.983+0000 7f332a7fc700 1 -- 172.21.15.188:0/162804944 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f332c010ca0 con 0x7f3338102400 2024-05-10T11:09:01.989 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:09:01.983+0000 7f333f790700 1 -- 172.21.15.188:0/162804944 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33381a3cf0 con 0x7f3338102400 2024-05-10T11:09:01.990 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:09:01.984+0000 7f333f790700 1 -- 172.21.15.188:0/162804944 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f333806a190 con 0x7f3338102400 2024-05-10T11:09:01.990 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:09:01.985+0000 7f332a7fc700 1 -- 172.21.15.188:0/162804944 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 15) ==== 149597+0+0 (secure 0 0 0) 0x7f332c018020 con 0x7f3338102400 2024-05-10T11:09:01.990 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:09:01.986+0000 7f332a7fc700 1 --2- 172.21.15.188:0/162804944 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f33240b27f0 0x7f33240b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:09:01.991 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:09:01.986+0000 7f332a7fc700 1 -- 172.21.15.188:0/162804944 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f332c0e0790 con 0x7f3338102400 2024-05-10T11:09:01.991 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:09:01.986+0000 7f333cd2b700 1 --2- 172.21.15.188:0/162804944 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f33240b27f0 0x7f33240b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:09:01.991 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:09:01.987+0000 7f333cd2b700 1 --2- 172.21.15.188:0/162804944 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f33240b27f0 0x7f33240b4cb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3334005fd0 tx=0x7f3334005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:09:01.991 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:09:01.990+0000 7f332a7fc700 1 -- 172.21.15.188:0/162804944 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f332c01f020 con 0x7f3338102400 2024-05-10T11:09:02.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:01 smithi178 ceph-mon[33359]: purged_snaps scrub starts 2024-05-10T11:09:02.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:01 smithi178 ceph-mon[33359]: purged_snaps scrub ok 2024-05-10T11:09:02.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:01 smithi178 ceph-mon[33359]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T11:09:02.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:01 smithi178 ceph-mon[33359]: osd.1 [v2:172.21.15.178:6800/3103987017,v1:172.21.15.178:6801/3103987017] boot 2024-05-10T11:09:02.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:01 smithi178 ceph-mon[33359]: osdmap e13: 2 total, 2 up, 2 in 2024-05-10T11:09:02.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:01 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T11:09:02.004 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:01 smithi178 ceph-mon[33359]: Detected new or changed devices on smithi178 2024-05-10T11:09:02.004 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:01 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:02.004 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:01 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:02.004 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:01 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:09:02.004 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:01 smithi178 ceph-mon[33359]: Adjusting osd_memory_target on smithi178 to 17088M 2024-05-10T11:09:02.004 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:01 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:02.004 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:01 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:09:02.005 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:01 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:09:02.005 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:01 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:02.005 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:01 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:09:02.005 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:01 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:09:02.005 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:01 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:09:02.005 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:01 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:02.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:01 smithi100 ceph-mon[30296]: purged_snaps scrub starts 2024-05-10T11:09:02.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:01 smithi100 ceph-mon[30296]: purged_snaps scrub ok 2024-05-10T11:09:02.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:01 smithi100 ceph-mon[30296]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-10T11:09:02.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:01 smithi100 ceph-mon[30296]: osd.1 [v2:172.21.15.178:6800/3103987017,v1:172.21.15.178:6801/3103987017] boot 2024-05-10T11:09:02.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:01 smithi100 ceph-mon[30296]: osdmap e13: 2 total, 2 up, 2 in 2024-05-10T11:09:02.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:01 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T11:09:02.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:01 smithi100 ceph-mon[30296]: Detected new or changed devices on smithi178 2024-05-10T11:09:02.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:01 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:02.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:01 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:02.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:01 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:09:02.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:01 smithi100 ceph-mon[30296]: Adjusting osd_memory_target on smithi178 to 17088M 2024-05-10T11:09:02.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:01 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:02.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:01 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:09:02.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:01 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:09:02.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:01 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:02.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:01 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:09:02.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:01 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:09:02.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:01 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:09:02.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:01 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:02.200 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:09:02.200+0000 7f333f790700 1 -- 172.21.15.188:0/162804944 --> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f33380638e0 con 0x7f33240b27f0 2024-05-10T11:09:03.139 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:02 smithi188 ceph-mon[34075]: osdmap e14: 2 total, 2 up, 2 in 2024-05-10T11:09:03.139 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:02 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T11:09:03.139 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:02 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T11:09:03.139 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:02 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:09:03.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:02 smithi178 ceph-mon[33359]: osdmap e14: 2 total, 2 up, 2 in 2024-05-10T11:09:03.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:02 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T11:09:03.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:02 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T11:09:03.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:02 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:09:03.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:02 smithi100 ceph-mon[30296]: osdmap e14: 2 total, 2 up, 2 in 2024-05-10T11:09:03.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:02 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T11:09:03.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:02 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T11:09:03.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:02 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:09:03.950 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:03 smithi188 ceph-mon[34075]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T11:09:03.950 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:03 smithi188 ceph-mon[34075]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:09:04.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:03 smithi178 ceph-mon[33359]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T11:09:04.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:03 smithi178 ceph-mon[33359]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:09:04.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:03 smithi100 ceph-mon[30296]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T11:09:04.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:03 smithi100 ceph-mon[30296]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:09:05.132 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:04 smithi188 ceph-mon[34075]: from='client.? 172.21.15.188:0/2010558133' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4d8f89ae-9424-46e9-9c80-17c47cc5e8ef"}]: dispatch 2024-05-10T11:09:05.133 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:04 smithi188 ceph-mon[34075]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4d8f89ae-9424-46e9-9c80-17c47cc5e8ef"}]: dispatch 2024-05-10T11:09:05.133 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:04 smithi188 ceph-mon[34075]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4d8f89ae-9424-46e9-9c80-17c47cc5e8ef"}]': finished 2024-05-10T11:09:05.133 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:04 smithi188 ceph-mon[34075]: osdmap e15: 3 total, 2 up, 3 in 2024-05-10T11:09:05.133 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:04 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T11:09:05.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:04 smithi100 ceph-mon[30296]: from='client.? 172.21.15.188:0/2010558133' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4d8f89ae-9424-46e9-9c80-17c47cc5e8ef"}]: dispatch 2024-05-10T11:09:05.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:04 smithi100 ceph-mon[30296]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4d8f89ae-9424-46e9-9c80-17c47cc5e8ef"}]: dispatch 2024-05-10T11:09:05.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:04 smithi100 ceph-mon[30296]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4d8f89ae-9424-46e9-9c80-17c47cc5e8ef"}]': finished 2024-05-10T11:09:05.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:04 smithi100 ceph-mon[30296]: osdmap e15: 3 total, 2 up, 3 in 2024-05-10T11:09:05.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:04 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T11:09:05.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:04 smithi178 ceph-mon[33359]: from='client.? 172.21.15.188:0/2010558133' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4d8f89ae-9424-46e9-9c80-17c47cc5e8ef"}]: dispatch 2024-05-10T11:09:05.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:04 smithi178 ceph-mon[33359]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4d8f89ae-9424-46e9-9c80-17c47cc5e8ef"}]: dispatch 2024-05-10T11:09:05.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:04 smithi178 ceph-mon[33359]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4d8f89ae-9424-46e9-9c80-17c47cc5e8ef"}]': finished 2024-05-10T11:09:05.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:04 smithi178 ceph-mon[33359]: osdmap e15: 3 total, 2 up, 3 in 2024-05-10T11:09:05.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:04 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T11:09:06.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:05 smithi100 ceph-mon[30296]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T11:09:06.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:05 smithi100 ceph-mon[30296]: from='client.? 172.21.15.188:0/2993601424' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T11:09:06.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:05 smithi178 ceph-mon[33359]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T11:09:06.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:05 smithi178 ceph-mon[33359]: from='client.? 172.21.15.188:0/2993601424' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T11:09:06.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:05 smithi188 ceph-mon[34075]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T11:09:06.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:05 smithi188 ceph-mon[34075]: from='client.? 172.21.15.188:0/2993601424' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T11:09:08.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:07 smithi178 ceph-mon[33359]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T11:09:08.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:07 smithi188 ceph-mon[34075]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T11:09:08.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:07 smithi100 ceph-mon[30296]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T11:09:10.011 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:09 smithi188 ceph-mon[34075]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T11:09:10.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:09 smithi178 ceph-mon[33359]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T11:09:10.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:09 smithi100 ceph-mon[30296]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T11:09:12.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:11 smithi188 ceph-mon[34075]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T11:09:12.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:11 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-10T11:09:12.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:11 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:09:12.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:11 smithi178 ceph-mon[33359]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T11:09:12.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:11 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-10T11:09:12.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:11 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:09:12.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:11 smithi100 ceph-mon[30296]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T11:09:12.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:11 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-10T11:09:12.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:11 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:09:13.119 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:12 smithi188 ceph-mon[34075]: Deploying daemon osd.2 on smithi188 2024-05-10T11:09:13.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:12 smithi178 ceph-mon[33359]: Deploying daemon osd.2 on smithi188 2024-05-10T11:09:13.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:12 smithi100 ceph-mon[30296]: Deploying daemon osd.2 on smithi188 2024-05-10T11:09:14.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:13 smithi188 ceph-mon[34075]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T11:09:14.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:13 smithi178 ceph-mon[33359]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T11:09:14.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:13 smithi100 ceph-mon[30296]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T11:09:16.024 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:15 smithi188 ceph-mon[34075]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T11:09:16.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:15 smithi178 ceph-mon[33359]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T11:09:16.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:15 smithi100 ceph-mon[30296]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T11:09:17.900 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:17 smithi188 ceph-mon[34075]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T11:09:17.900 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:17 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:17.900 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:17 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:17.900 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:17 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:09:18.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:17 smithi178 ceph-mon[33359]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T11:09:18.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:17 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:18.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:17 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:18.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:17 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:09:18.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:17 smithi100 ceph-mon[30296]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T11:09:18.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:17 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:18.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:17 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:18.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:17 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:09:20.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:19 smithi178 ceph-mon[33359]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T11:09:20.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:19 smithi188 ceph-mon[34075]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T11:09:20.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:19 smithi100 ceph-mon[30296]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T11:09:20.813 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:09:20.813+0000 7f332a7fc700 1 -- 172.21.15.188:0/162804944 <== mgr.14152 v2:172.21.15.100:6800/2507287356 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f33380638e0 con 0x7f33240b27f0 2024-05-10T11:09:20.813 INFO:teuthology.orchestra.run.smithi188.stdout:Created osd(s) 2 on host 'smithi188' 2024-05-10T11:09:20.816 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:09:20.815+0000 7f333f790700 1 -- 172.21.15.188:0/162804944 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f33240b27f0 msgr2=0x7f33240b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:09:20.816 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:09:20.815+0000 7f333f790700 1 --2- 172.21.15.188:0/162804944 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f33240b27f0 0x7f33240b4cb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3334005fd0 tx=0x7f3334005e20 comp rx=0 tx=0).stop 2024-05-10T11:09:20.816 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:09:20.815+0000 7f333f790700 1 -- 172.21.15.188:0/162804944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3338102400 msgr2=0x7f333806bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:09:20.816 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:09:20.815+0000 7f333f790700 1 --2- 172.21.15.188:0/162804944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3338102400 0x7f333806bce0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f332c00b500 tx=0x7f332c00b8c0 comp rx=0 tx=0).stop 2024-05-10T11:09:20.816 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:09:20.816+0000 7f333f790700 1 -- 172.21.15.188:0/162804944 shutdown_connections 2024-05-10T11:09:20.816 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:09:20.816+0000 7f333f790700 1 --2- 172.21.15.188:0/162804944 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f33240b27f0 0x7f33240b4cb0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:09:20.816 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:09:20.816+0000 7f333f790700 1 --2- 172.21.15.188:0/162804944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3338103fa0 0x7f33381a2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:09:20.816 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:09:20.816+0000 7f333f790700 1 --2- 172.21.15.188:0/162804944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3338102400 0x7f333806bce0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:09:20.816 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:09:20.816+0000 7f333f790700 1 --2- 172.21.15.188:0/162804944 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3338103600 0x7f33381a0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:09:20.816 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:09:20.816+0000 7f333f790700 1 -- 172.21.15.188:0/162804944 >> 172.21.15.188:0/162804944 conn(0x7f33380fd9a0 msgr2=0x7f33381069d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:09:20.816 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:09:20.816+0000 7f333f790700 1 -- 172.21.15.188:0/162804944 shutdown_connections 2024-05-10T11:09:20.816 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:09:20.816+0000 7f333f790700 1 -- 172.21.15.188:0/162804944 wait complete. 2024-05-10T11:09:21.122 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:21 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:21.122 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:21 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:21.122 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:21 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:09:21.122 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:21 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:09:21.122 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:21 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:21.122 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:21 smithi188 ceph-mon[34075]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T11:09:21.122 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:21 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:21.123 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:21 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:21.123 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:21 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:09:21.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:21 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:21.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:21 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:21.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:21 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:09:21.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:21 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:09:21.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:21 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:21.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:21 smithi100 ceph-mon[30296]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T11:09:21.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:21 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:21.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:21 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:21.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:21 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:09:21.449 DEBUG:teuthology.orchestra.run.smithi188:osd.2> sudo journalctl -f -n 0 -u ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@osd.2.service 2024-05-10T11:09:21.492 INFO:tasks.cephadm:Deploying osd.3 on smithi100 with /dev/vg_nvme/lv_3... 2024-05-10T11:09:21.492 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-10T11:09:21.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:21 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:21.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:21 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:21.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:21 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:09:21.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:21 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:09:21.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:21 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:21.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:21 smithi178 ceph-mon[33359]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T11:09:21.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:21 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:21.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:21 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:21.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:21 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:09:21.740 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:09:22.118 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:22 smithi100 ceph-mon[30296]: from='osd.2 [v2:172.21.15.188:6800/3991466468,v1:172.21.15.188:6801/3991466468]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-10T11:09:22.119 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:22 smithi100 ceph-mon[30296]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-10T11:09:22.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:22 smithi188 ceph-mon[34075]: from='osd.2 [v2:172.21.15.188:6800/3991466468,v1:172.21.15.188:6801/3991466468]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-10T11:09:22.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:22 smithi188 ceph-mon[34075]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-10T11:09:22.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:22 smithi178 ceph-mon[33359]: from='osd.2 [v2:172.21.15.188:6800/3991466468,v1:172.21.15.188:6801/3991466468]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-10T11:09:22.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:22 smithi178 ceph-mon[33359]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-10T11:09:23.192 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:23 smithi100 ceph-mon[30296]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-10T11:09:23.192 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:23 smithi100 ceph-mon[30296]: osdmap e16: 3 total, 2 up, 3 in 2024-05-10T11:09:23.192 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:23 smithi100 ceph-mon[30296]: from='osd.2 [v2:172.21.15.188:6800/3991466468,v1:172.21.15.188:6801/3991466468]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-05-10T11:09:23.192 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:23 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T11:09:23.192 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:23 smithi100 ceph-mon[30296]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-05-10T11:09:23.192 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:23 smithi100 ceph-mon[30296]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T11:09:23.193 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:23 smithi100 ceph-mon[30296]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-05-10T11:09:23.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:23 smithi188 ceph-mon[34075]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-10T11:09:23.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:23 smithi188 ceph-mon[34075]: osdmap e16: 3 total, 2 up, 3 in 2024-05-10T11:09:23.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:23 smithi188 ceph-mon[34075]: from='osd.2 [v2:172.21.15.188:6800/3991466468,v1:172.21.15.188:6801/3991466468]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-05-10T11:09:23.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:23 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T11:09:23.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:23 smithi188 ceph-mon[34075]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-05-10T11:09:23.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:23 smithi188 ceph-mon[34075]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T11:09:23.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:23 smithi188 ceph-mon[34075]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-05-10T11:09:23.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:23 smithi178 ceph-mon[33359]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-10T11:09:23.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:23 smithi178 ceph-mon[33359]: osdmap e16: 3 total, 2 up, 3 in 2024-05-10T11:09:23.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:23 smithi178 ceph-mon[33359]: from='osd.2 [v2:172.21.15.188:6800/3991466468,v1:172.21.15.188:6801/3991466468]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-05-10T11:09:23.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:23 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T11:09:23.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:23 smithi178 ceph-mon[33359]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-05-10T11:09:23.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:23 smithi178 ceph-mon[33359]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T11:09:23.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:23 smithi178 ceph-mon[33359]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-05-10T11:09:23.746 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-10T11:09:23.746 INFO:teuthology.orchestra.run.smithi100.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-10T11:09:23.747 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10+0 records in 2024-05-10T11:09:23.747 INFO:teuthology.orchestra.run.smithi100.stderr:10+0 records out 2024-05-10T11:09:23.747 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0124309 s, 844 MB/s 2024-05-10T11:09:23.747 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping successful for: 2024-05-10T11:09:24.029 INFO:journalctl@ceph.osd.2.smithi188.stdout:May 10 11:09:23 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-2[39236]: 2024-05-10T11:09:23.736+0000 7f1cfa587700 -1 osd.2 0 waiting for initial osdmap 2024-05-10T11:09:24.029 INFO:journalctl@ceph.osd.2.smithi188.stdout:May 10 11:09:23 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-2[39236]: 2024-05-10T11:09:23.741+0000 7f1cf2372700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T11:09:24.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:24 smithi100 ceph-mon[30296]: osdmap e17: 3 total, 2 up, 3 in 2024-05-10T11:09:24.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:24 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T11:09:24.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:24 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T11:09:24.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:24 smithi100 ceph-mon[30296]: from='osd.2 ' entity='osd.2' 2024-05-10T11:09:24.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:24 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T11:09:24.467 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi100:vg_nvme/lv_3 2024-05-10T11:09:24.469 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:24 smithi188 ceph-mon[34075]: osdmap e17: 3 total, 2 up, 3 in 2024-05-10T11:09:24.469 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:24 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T11:09:24.469 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:24 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T11:09:24.469 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:24 smithi188 ceph-mon[34075]: from='osd.2 ' entity='osd.2' 2024-05-10T11:09:24.469 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:24 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T11:09:24.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:24 smithi178 ceph-mon[33359]: osdmap e17: 3 total, 2 up, 3 in 2024-05-10T11:09:24.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:24 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T11:09:24.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:24 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T11:09:24.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:24 smithi178 ceph-mon[33359]: from='osd.2 ' entity='osd.2' 2024-05-10T11:09:24.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:24 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T11:09:24.732 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:09:25.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:25 smithi100 ceph-mon[30296]: purged_snaps scrub starts 2024-05-10T11:09:25.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:25 smithi100 ceph-mon[30296]: purged_snaps scrub ok 2024-05-10T11:09:25.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:25 smithi100 ceph-mon[30296]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T11:09:25.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:25 smithi100 ceph-mon[30296]: Detected new or changed devices on smithi188 2024-05-10T11:09:25.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:25 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:25.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:25 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:25.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:25 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:09:25.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:25 smithi100 ceph-mon[30296]: Adjusting osd_memory_target on smithi188 to 17088M 2024-05-10T11:09:25.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:25 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:25.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:25 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:09:25.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:25 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:09:25.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:25 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:25.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:25 smithi100 ceph-mon[30296]: osd.2 [v2:172.21.15.188:6800/3991466468,v1:172.21.15.188:6801/3991466468] boot 2024-05-10T11:09:25.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:25 smithi100 ceph-mon[30296]: osdmap e18: 3 total, 3 up, 3 in 2024-05-10T11:09:25.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:25 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T11:09:25.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:25 smithi178 ceph-mon[33359]: purged_snaps scrub starts 2024-05-10T11:09:25.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:25 smithi178 ceph-mon[33359]: purged_snaps scrub ok 2024-05-10T11:09:25.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:25 smithi178 ceph-mon[33359]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T11:09:25.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:25 smithi178 ceph-mon[33359]: Detected new or changed devices on smithi188 2024-05-10T11:09:25.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:25 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:25.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:25 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:25.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:25 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:09:25.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:25 smithi178 ceph-mon[33359]: Adjusting osd_memory_target on smithi188 to 17088M 2024-05-10T11:09:25.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:25 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:25.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:25 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:09:25.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:25 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:09:25.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:25 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:25.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:25 smithi178 ceph-mon[33359]: osd.2 [v2:172.21.15.188:6800/3991466468,v1:172.21.15.188:6801/3991466468] boot 2024-05-10T11:09:25.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:25 smithi178 ceph-mon[33359]: osdmap e18: 3 total, 3 up, 3 in 2024-05-10T11:09:25.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:25 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T11:09:25.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:25 smithi188 ceph-mon[34075]: purged_snaps scrub starts 2024-05-10T11:09:25.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:25 smithi188 ceph-mon[34075]: purged_snaps scrub ok 2024-05-10T11:09:25.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:25 smithi188 ceph-mon[34075]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T11:09:25.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:25 smithi188 ceph-mon[34075]: Detected new or changed devices on smithi188 2024-05-10T11:09:25.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:25 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:25.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:25 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:25.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:25 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:09:25.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:25 smithi188 ceph-mon[34075]: Adjusting osd_memory_target on smithi188 to 17088M 2024-05-10T11:09:25.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:25 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:25.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:25 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:09:25.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:25 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:09:25.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:25 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:25.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:25 smithi188 ceph-mon[34075]: osd.2 [v2:172.21.15.188:6800/3991466468,v1:172.21.15.188:6801/3991466468] boot 2024-05-10T11:09:25.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:25 smithi188 ceph-mon[34075]: osdmap e18: 3 total, 3 up, 3 in 2024-05-10T11:09:25.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:25 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T11:09:26.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:09:26.319+0000 7fb96c19e700 1 -- 172.21.15.100:0/2885105701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb964105090 msgr2=0x7fb96410b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:09:26.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:09:26.319+0000 7fb96c19e700 1 --2- 172.21.15.100:0/2885105701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb964105090 0x7fb96410b940 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb95c00b0d0 tx=0x7fb95c00b3e0 comp rx=0 tx=0).stop 2024-05-10T11:09:26.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:09:26.319+0000 7fb96c19e700 1 -- 172.21.15.100:0/2885105701 shutdown_connections 2024-05-10T11:09:26.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:09:26.319+0000 7fb96c19e700 1 --2- 172.21.15.100:0/2885105701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb964105090 0x7fb96410b940 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:09:26.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:09:26.319+0000 7fb96c19e700 1 --2- 172.21.15.100:0/2885105701 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9641034f0 0x7fb964103910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:09:26.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:09:26.319+0000 7fb96c19e700 1 --2- 172.21.15.100:0/2885105701 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9641046f0 0x7fb964104b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:09:26.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:09:26.319+0000 7fb96c19e700 1 -- 172.21.15.100:0/2885105701 >> 172.21.15.100:0/2885105701 conn(0x7fb9640fea90 msgr2=0x7fb964100ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:09:26.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:09:26.319+0000 7fb96c19e700 1 -- 172.21.15.100:0/2885105701 shutdown_connections 2024-05-10T11:09:26.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:09:26.319+0000 7fb96c19e700 1 -- 172.21.15.100:0/2885105701 wait complete. 2024-05-10T11:09:26.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:09:26.320+0000 7fb96c19e700 1 Processor -- start 2024-05-10T11:09:26.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:09:26.320+0000 7fb96c19e700 1 -- start start 2024-05-10T11:09:26.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:09:26.320+0000 7fb96c19e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9641034f0 0x7fb9641994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:09:26.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:09:26.320+0000 7fb96c19e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9641046f0 0x7fb964199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:09:26.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:09:26.320+0000 7fb96c19e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb964105090 0x7fb9641a0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:09:26.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:09:26.320+0000 7fb96c19e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb96410ee40 con 0x7fb9641034f0 2024-05-10T11:09:26.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:09:26.320+0000 7fb96c19e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb96410ecc0 con 0x7fb964105090 2024-05-10T11:09:26.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:09:26.320+0000 7fb96c19e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb96410efc0 con 0x7fb9641046f0 2024-05-10T11:09:26.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:09:26.320+0000 7fb969f3a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9641034f0 0x7fb9641994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:09:26.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:09:26.320+0000 7fb969f3a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9641034f0 0x7fb9641994e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34572/0 (socket says 172.21.15.100:34572) 2024-05-10T11:09:26.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:09:26.320+0000 7fb969f3a700 1 -- 172.21.15.100:0/785051727 learned_addr learned my addr 172.21.15.100:0/785051727 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:09:26.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:09:26.320+0000 7fb969739700 1 --2- 172.21.15.100:0/785051727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9641046f0 0x7fb964199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:09:26.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:09:26.320+0000 7fb96a73b700 1 --2- 172.21.15.100:0/785051727 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb964105090 0x7fb9641a0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:09:26.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:09:26.321+0000 7fb969f3a700 1 -- 172.21.15.100:0/785051727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9641046f0 msgr2=0x7fb964199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:09:26.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:09:26.321+0000 7fb969f3a700 1 --2- 172.21.15.100:0/785051727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9641046f0 0x7fb964199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:09:26.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:09:26.321+0000 7fb969f3a700 1 -- 172.21.15.100:0/785051727 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb964105090 msgr2=0x7fb9641a0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:09:26.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:09:26.321+0000 7fb969f3a700 1 --2- 172.21.15.100:0/785051727 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb964105090 0x7fb9641a0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:09:26.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:09:26.321+0000 7fb969f3a700 1 -- 172.21.15.100:0/785051727 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9641a1140 con 0x7fb9641034f0 2024-05-10T11:09:26.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:09:26.321+0000 7fb969f3a700 1 --2- 172.21.15.100:0/785051727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9641034f0 0x7fb9641994e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb96000d9b0 tx=0x7fb96000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:09:26.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:09:26.321+0000 7fb95affd700 1 -- 172.21.15.100:0/785051727 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb960017030 con 0x7fb9641034f0 2024-05-10T11:09:26.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:09:26.321+0000 7fb96c19e700 1 -- 172.21.15.100:0/785051727 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9641a1430 con 0x7fb9641034f0 2024-05-10T11:09:26.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:09:26.321+0000 7fb95affd700 1 -- 172.21.15.100:0/785051727 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb9600044b0 con 0x7fb9641034f0 2024-05-10T11:09:26.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:09:26.321+0000 7fb95affd700 1 -- 172.21.15.100:0/785051727 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9600038e0 con 0x7fb9641034f0 2024-05-10T11:09:26.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:09:26.321+0000 7fb96c19e700 1 -- 172.21.15.100:0/785051727 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9641a1970 con 0x7fb9641034f0 2024-05-10T11:09:26.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:09:26.322+0000 7fb96c19e700 1 -- 172.21.15.100:0/785051727 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb96406a190 con 0x7fb9641034f0 2024-05-10T11:09:26.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:09:26.324+0000 7fb95affd700 1 -- 172.21.15.100:0/785051727 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 15) ==== 149597+0+0 (secure 0 0 0) 0x7fb96001a070 con 0x7fb9641034f0 2024-05-10T11:09:26.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:09:26.324+0000 7fb95affd700 1 --2- 172.21.15.100:0/785051727 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fb9500b27f0 0x7fb9500b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:09:26.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:09:26.324+0000 7fb969739700 1 --2- 172.21.15.100:0/785051727 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fb9500b27f0 0x7fb9500b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:09:26.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:09:26.324+0000 7fb95affd700 1 -- 172.21.15.100:0/785051727 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 2851+0+0 (secure 0 0 0) 0x7fb9600e0880 con 0x7fb9641034f0 2024-05-10T11:09:26.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:09:26.325+0000 7fb969739700 1 --2- 172.21.15.100:0/785051727 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fb9500b27f0 0x7fb9500b4cb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb954009c60 tx=0x7fb954009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:09:26.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:09:26.328+0000 7fb95affd700 1 -- 172.21.15.100:0/785051727 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fb9600a9bd0 con 0x7fb9641034f0 2024-05-10T11:09:26.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:09:26.543+0000 7fb96c19e700 1 -- 172.21.15.100:0/785051727 --> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fb9640638e0 con 0x7fb9500b27f0 2024-05-10T11:09:27.452 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:27 smithi100 ceph-mon[30296]: osdmap e19: 3 total, 3 up, 3 in 2024-05-10T11:09:27.452 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:27 smithi100 ceph-mon[30296]: pgmap v82: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-10T11:09:27.452 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:27 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-10T11:09:27.452 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:27 smithi100 ceph-mon[30296]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:09:27.452 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:27 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T11:09:27.452 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:27 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T11:09:27.452 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:27 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:09:27.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:27 smithi178 ceph-mon[33359]: osdmap e19: 3 total, 3 up, 3 in 2024-05-10T11:09:27.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:27 smithi178 ceph-mon[33359]: pgmap v82: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-10T11:09:27.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:27 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-10T11:09:27.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:27 smithi178 ceph-mon[33359]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:09:27.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:27 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T11:09:27.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:27 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T11:09:27.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:27 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:09:27.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:27 smithi188 ceph-mon[34075]: osdmap e19: 3 total, 3 up, 3 in 2024-05-10T11:09:27.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:27 smithi188 ceph-mon[34075]: pgmap v82: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-10T11:09:27.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:27 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-10T11:09:27.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:27 smithi188 ceph-mon[34075]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:09:27.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:27 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T11:09:27.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:27 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T11:09:27.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:27 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:09:28.163 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:28 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-10T11:09:28.164 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:28 smithi100 ceph-mon[30296]: osdmap e20: 3 total, 3 up, 3 in 2024-05-10T11:09:28.164 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:28 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-10T11:09:28.400 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:28 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-10T11:09:28.400 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:28 smithi188 ceph-mon[34075]: osdmap e20: 3 total, 3 up, 3 in 2024-05-10T11:09:28.401 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:28 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-10T11:09:28.401 INFO:journalctl@ceph.osd.2.smithi188.stdout:May 10 11:09:28 smithi188 sudo[42820]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-10T11:09:28.401 INFO:journalctl@ceph.osd.2.smithi188.stdout:May 10 11:09:28 smithi188 sudo[42820]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T11:09:28.401 INFO:journalctl@ceph.osd.2.smithi188.stdout:May 10 11:09:28 smithi188 sudo[42820]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T11:09:28.401 INFO:journalctl@ceph.osd.2.smithi188.stdout:May 10 11:09:28 smithi188 sudo[42820]: pam_unix(sudo:session): session closed for user root 2024-05-10T11:09:28.401 INFO:journalctl@ceph.osd.2.smithi188.stdout:May 10 11:09:28 smithi188 sudo[42823]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-10T11:09:28.401 INFO:journalctl@ceph.osd.2.smithi188.stdout:May 10 11:09:28 smithi188 sudo[42823]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T11:09:28.401 INFO:journalctl@ceph.osd.2.smithi188.stdout:May 10 11:09:28 smithi188 sudo[42823]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T11:09:28.401 INFO:journalctl@ceph.osd.2.smithi188.stdout:May 10 11:09:28 smithi188 sudo[42823]: pam_unix(sudo:session): session closed for user root 2024-05-10T11:09:28.438 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:28 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-10T11:09:28.438 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:28 smithi178 ceph-mon[33359]: osdmap e20: 3 total, 3 up, 3 in 2024-05-10T11:09:28.438 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:28 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-10T11:09:28.438 INFO:journalctl@ceph.osd.1.smithi178.stdout:May 10 11:09:28 smithi178 sudo[41904]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-10T11:09:28.439 INFO:journalctl@ceph.osd.1.smithi178.stdout:May 10 11:09:28 smithi178 sudo[41904]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T11:09:28.439 INFO:journalctl@ceph.osd.1.smithi178.stdout:May 10 11:09:28 smithi178 sudo[41904]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T11:09:28.439 INFO:journalctl@ceph.osd.1.smithi178.stdout:May 10 11:09:28 smithi178 sudo[41904]: pam_unix(sudo:session): session closed for user root 2024-05-10T11:09:28.439 INFO:journalctl@ceph.osd.1.smithi178.stdout:May 10 11:09:28 smithi178 sudo[41907]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-10T11:09:28.439 INFO:journalctl@ceph.osd.1.smithi178.stdout:May 10 11:09:28 smithi178 sudo[41907]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T11:09:28.439 INFO:journalctl@ceph.osd.1.smithi178.stdout:May 10 11:09:28 smithi178 sudo[41907]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T11:09:28.439 INFO:journalctl@ceph.osd.1.smithi178.stdout:May 10 11:09:28 smithi178 sudo[41907]: pam_unix(sudo:session): session closed for user root 2024-05-10T11:09:28.492 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:28 smithi100 sudo[47166]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-10T11:09:28.492 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:28 smithi100 sudo[47166]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T11:09:28.492 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:28 smithi100 sudo[47166]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T11:09:28.492 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:28 smithi100 sudo[47166]: pam_unix(sudo:session): session closed for user root 2024-05-10T11:09:28.492 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:28 smithi100 sudo[47169]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-10T11:09:28.492 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:28 smithi100 sudo[47169]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T11:09:28.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:28 smithi100 sudo[47169]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T11:09:28.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:28 smithi100 sudo[47169]: pam_unix(sudo:session): session closed for user root 2024-05-10T11:09:28.493 INFO:journalctl@ceph.osd.0.smithi100.stdout:May 10 11:09:28 smithi100 sudo[47139]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-10T11:09:28.493 INFO:journalctl@ceph.osd.0.smithi100.stdout:May 10 11:09:28 smithi100 sudo[47139]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T11:09:28.493 INFO:journalctl@ceph.osd.0.smithi100.stdout:May 10 11:09:28 smithi100 sudo[47139]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T11:09:28.493 INFO:journalctl@ceph.osd.0.smithi100.stdout:May 10 11:09:28 smithi100 sudo[47139]: pam_unix(sudo:session): session closed for user root 2024-05-10T11:09:28.493 INFO:journalctl@ceph.osd.0.smithi100.stdout:May 10 11:09:28 smithi100 sudo[47142]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-10T11:09:28.493 INFO:journalctl@ceph.osd.0.smithi100.stdout:May 10 11:09:28 smithi100 sudo[47142]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T11:09:28.493 INFO:journalctl@ceph.osd.0.smithi100.stdout:May 10 11:09:28 smithi100 sudo[47142]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T11:09:28.494 INFO:journalctl@ceph.osd.0.smithi100.stdout:May 10 11:09:28 smithi100 sudo[47142]: pam_unix(sudo:session): session closed for user root 2024-05-10T11:09:28.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:28 smithi178 sudo[41910]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-10T11:09:28.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:28 smithi178 sudo[41910]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T11:09:28.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:28 smithi178 sudo[41910]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T11:09:28.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:28 smithi178 sudo[41910]: pam_unix(sudo:session): session closed for user root 2024-05-10T11:09:28.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:28 smithi178 sudo[41913]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-10T11:09:28.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:28 smithi178 sudo[41913]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T11:09:28.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:28 smithi178 sudo[41913]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T11:09:28.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:28 smithi178 sudo[41913]: pam_unix(sudo:session): session closed for user root 2024-05-10T11:09:28.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:28 smithi188 sudo[42826]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-10T11:09:28.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:28 smithi188 sudo[42826]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T11:09:28.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:28 smithi188 sudo[42826]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T11:09:28.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:28 smithi188 sudo[42826]: pam_unix(sudo:session): session closed for user root 2024-05-10T11:09:28.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:28 smithi188 sudo[42829]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-10T11:09:28.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:28 smithi188 sudo[42829]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T11:09:28.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:28 smithi188 sudo[42829]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T11:09:28.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:28 smithi188 sudo[42829]: pam_unix(sudo:session): session closed for user root 2024-05-10T11:09:29.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:29 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-10T11:09:29.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:29 smithi178 ceph-mon[33359]: osdmap e21: 3 total, 3 up, 3 in 2024-05-10T11:09:29.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:29 smithi178 ceph-mon[33359]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-10T11:09:29.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:29 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:29.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:29 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:29.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:29 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:29.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:29 smithi178 ceph-mon[33359]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-10T11:09:29.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:29 smithi178 ceph-mon[33359]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-10T11:09:29.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:29 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T11:09:29.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:29 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T11:09:29.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:29 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T11:09:29.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:29 smithi178 ceph-mon[33359]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-10T11:09:29.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:29 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T11:09:29.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:29 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T11:09:29.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:29 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T11:09:29.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:29 smithi178 ceph-mon[33359]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-10T11:09:29.504 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:29 smithi178 ceph-mon[33359]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-10T11:09:29.504 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:29 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T11:09:29.504 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:29 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T11:09:29.504 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:29 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T11:09:29.504 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:29 smithi178 ceph-mon[33359]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-10T11:09:29.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:29 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-10T11:09:29.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:29 smithi188 ceph-mon[34075]: osdmap e21: 3 total, 3 up, 3 in 2024-05-10T11:09:29.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:29 smithi188 ceph-mon[34075]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-10T11:09:29.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:29 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:29.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:29 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:29.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:29 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:29.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:29 smithi188 ceph-mon[34075]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-10T11:09:29.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:29 smithi188 ceph-mon[34075]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-10T11:09:29.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:29 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T11:09:29.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:29 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T11:09:29.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:29 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T11:09:29.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:29 smithi188 ceph-mon[34075]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-10T11:09:29.530 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:29 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T11:09:29.530 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:29 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T11:09:29.530 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:29 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T11:09:29.530 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:29 smithi188 ceph-mon[34075]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-10T11:09:29.530 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:29 smithi188 ceph-mon[34075]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-10T11:09:29.530 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:29 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T11:09:29.530 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:29 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T11:09:29.530 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:29 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T11:09:29.530 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:29 smithi188 ceph-mon[34075]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-10T11:09:29.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:29 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-10T11:09:29.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:29 smithi100 ceph-mon[30296]: osdmap e21: 3 total, 3 up, 3 in 2024-05-10T11:09:29.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:29 smithi100 ceph-mon[30296]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-10T11:09:29.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:29 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:29.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:29 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:29.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:29 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:29.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:29 smithi100 ceph-mon[30296]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-10T11:09:29.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:29 smithi100 ceph-mon[30296]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-10T11:09:29.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:29 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T11:09:29.539 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:29 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T11:09:29.539 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:29 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T11:09:29.539 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:29 smithi100 ceph-mon[30296]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-10T11:09:29.539 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:29 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T11:09:29.539 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:29 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T11:09:29.539 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:29 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T11:09:29.539 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:29 smithi100 ceph-mon[30296]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-10T11:09:29.539 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:29 smithi100 ceph-mon[30296]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-10T11:09:29.539 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:29 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T11:09:29.539 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:29 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T11:09:29.539 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:29 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T11:09:29.539 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:29 smithi100 ceph-mon[30296]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-10T11:09:30.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:30 smithi178 ceph-mon[33359]: osdmap e22: 3 total, 3 up, 3 in 2024-05-10T11:09:30.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:30 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/228162283' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4c8865aa-fef4-4710-b766-1385adb9462c"}]: dispatch 2024-05-10T11:09:30.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:30 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/228162283' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4c8865aa-fef4-4710-b766-1385adb9462c"}]': finished 2024-05-10T11:09:30.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:30 smithi178 ceph-mon[33359]: osdmap e23: 4 total, 3 up, 4 in 2024-05-10T11:09:30.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:30 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T11:09:30.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:30 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/3199214304' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T11:09:30.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:30 smithi188 ceph-mon[34075]: osdmap e22: 3 total, 3 up, 3 in 2024-05-10T11:09:30.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:30 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/228162283' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4c8865aa-fef4-4710-b766-1385adb9462c"}]: dispatch 2024-05-10T11:09:30.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:30 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/228162283' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4c8865aa-fef4-4710-b766-1385adb9462c"}]': finished 2024-05-10T11:09:30.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:30 smithi188 ceph-mon[34075]: osdmap e23: 4 total, 3 up, 4 in 2024-05-10T11:09:30.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:30 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T11:09:30.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:30 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/3199214304' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T11:09:30.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:30 smithi100 ceph-mon[30296]: osdmap e22: 3 total, 3 up, 3 in 2024-05-10T11:09:30.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:30 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/228162283' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4c8865aa-fef4-4710-b766-1385adb9462c"}]: dispatch 2024-05-10T11:09:30.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:30 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/228162283' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4c8865aa-fef4-4710-b766-1385adb9462c"}]': finished 2024-05-10T11:09:30.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:30 smithi100 ceph-mon[30296]: osdmap e23: 4 total, 3 up, 4 in 2024-05-10T11:09:30.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:30 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T11:09:30.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:30 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/3199214304' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T11:09:31.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:09:31.130+0000 7fb95affd700 1 -- 172.21.15.100:0/785051727 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fb9600a5760 con 0x7fb9641034f0 2024-05-10T11:09:31.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:31 smithi178 ceph-mon[33359]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-05-10T11:09:31.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:31 smithi188 ceph-mon[34075]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-05-10T11:09:31.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:31 smithi100 ceph-mon[30296]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-05-10T11:09:32.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:32 smithi178 ceph-mon[33359]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-10T11:09:32.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:32 smithi188 ceph-mon[34075]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-10T11:09:32.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:32 smithi100 ceph-mon[30296]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-10T11:09:33.423 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:33 smithi100 ceph-mon[30296]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 89 MiB used, 268 GiB / 268 GiB avail 2024-05-10T11:09:33.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:33 smithi178 ceph-mon[33359]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 89 MiB used, 268 GiB / 268 GiB avail 2024-05-10T11:09:33.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:33 smithi188 ceph-mon[34075]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 89 MiB used, 268 GiB / 268 GiB avail 2024-05-10T11:09:35.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:35 smithi100 ceph-mon[30296]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-10T11:09:35.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:35 smithi178 ceph-mon[33359]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-10T11:09:35.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:35 smithi188 ceph-mon[34075]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-10T11:09:37.469 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:37 smithi100 ceph-mon[30296]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-10T11:09:37.469 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:37 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-10T11:09:37.470 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:37 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:09:37.470 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:37 smithi100 ceph-mon[30296]: Deploying daemon osd.3 on smithi100 2024-05-10T11:09:37.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:37 smithi178 ceph-mon[33359]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-10T11:09:37.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:37 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-10T11:09:37.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:37 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:09:37.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:37 smithi178 ceph-mon[33359]: Deploying daemon osd.3 on smithi100 2024-05-10T11:09:37.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:37 smithi188 ceph-mon[34075]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-10T11:09:37.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:37 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-10T11:09:37.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:37 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:09:37.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:37 smithi188 ceph-mon[34075]: Deploying daemon osd.3 on smithi100 2024-05-10T11:09:39.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:39 smithi100 ceph-mon[30296]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-10T11:09:39.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:39 smithi178 ceph-mon[33359]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-10T11:09:39.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:39 smithi188 ceph-mon[34075]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-10T11:09:41.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:41 smithi100 ceph-mon[30296]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-10T11:09:41.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:41 smithi178 ceph-mon[33359]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-10T11:09:41.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:41 smithi188 ceph-mon[34075]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-10T11:09:44.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:43 smithi188 ceph-mon[34075]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-10T11:09:44.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:43 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:44.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:43 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:44.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:43 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:09:44.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:43 smithi178 ceph-mon[33359]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-10T11:09:44.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:43 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:44.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:43 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:44.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:43 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:09:44.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:43 smithi100 ceph-mon[30296]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-10T11:09:44.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:43 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:44.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:43 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:44.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:43 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:09:46.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:45 smithi100 ceph-mon[30296]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-10T11:09:46.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:45 smithi178 ceph-mon[33359]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-10T11:09:46.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:45 smithi188 ceph-mon[34075]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-10T11:09:46.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:09:46.809+0000 7fb95affd700 1 -- 172.21.15.100:0/785051727 <== mgr.14152 v2:172.21.15.100:6800/2507287356 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb9640638e0 con 0x7fb9500b27f0 2024-05-10T11:09:46.810 INFO:teuthology.orchestra.run.smithi100.stdout:Created osd(s) 3 on host 'smithi100' 2024-05-10T11:09:46.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:09:46.812+0000 7fb96c19e700 1 -- 172.21.15.100:0/785051727 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fb9500b27f0 msgr2=0x7fb9500b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:09:46.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:09:46.812+0000 7fb96c19e700 1 --2- 172.21.15.100:0/785051727 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fb9500b27f0 0x7fb9500b4cb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb954009c60 tx=0x7fb954009260 comp rx=0 tx=0).stop 2024-05-10T11:09:46.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:09:46.812+0000 7fb96c19e700 1 -- 172.21.15.100:0/785051727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9641034f0 msgr2=0x7fb9641994e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:09:46.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:09:46.812+0000 7fb96c19e700 1 --2- 172.21.15.100:0/785051727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9641034f0 0x7fb9641994e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb96000d9b0 tx=0x7fb96000dcc0 comp rx=0 tx=0).stop 2024-05-10T11:09:46.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:09:46.812+0000 7fb96c19e700 1 -- 172.21.15.100:0/785051727 shutdown_connections 2024-05-10T11:09:46.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:09:46.812+0000 7fb96c19e700 1 --2- 172.21.15.100:0/785051727 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fb9500b27f0 0x7fb9500b4cb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:09:46.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:09:46.812+0000 7fb96c19e700 1 --2- 172.21.15.100:0/785051727 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb964105090 0x7fb9641a0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:09:46.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:09:46.812+0000 7fb96c19e700 1 --2- 172.21.15.100:0/785051727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9641034f0 0x7fb9641994e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:09:46.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:09:46.812+0000 7fb96c19e700 1 --2- 172.21.15.100:0/785051727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9641046f0 0x7fb964199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:09:46.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:09:46.812+0000 7fb96c19e700 1 -- 172.21.15.100:0/785051727 >> 172.21.15.100:0/785051727 conn(0x7fb9640fea90 msgr2=0x7fb964107b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:09:46.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:09:46.812+0000 7fb96c19e700 1 -- 172.21.15.100:0/785051727 shutdown_connections 2024-05-10T11:09:46.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:09:46.812+0000 7fb96c19e700 1 -- 172.21.15.100:0/785051727 wait complete. 2024-05-10T11:09:47.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:46 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:47.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:46 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:47.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:46 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:09:47.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:46 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:09:47.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:46 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:47.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:46 smithi100 ceph-mon[30296]: from='osd.3 [v2:172.21.15.100:6810/1713357192,v1:172.21.15.100:6811/1713357192]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-10T11:09:47.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:46 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:09:47.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:46 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:47.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:46 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:47.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:46 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:47.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:46 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:47.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:46 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:09:47.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:46 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:09:47.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:46 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:47.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:46 smithi178 ceph-mon[33359]: from='osd.3 [v2:172.21.15.100:6810/1713357192,v1:172.21.15.100:6811/1713357192]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-10T11:09:47.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:46 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:09:47.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:46 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:47.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:46 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:47.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:46 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:47.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:46 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:47.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:46 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:09:47.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:46 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:09:47.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:46 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:47.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:46 smithi188 ceph-mon[34075]: from='osd.3 [v2:172.21.15.100:6810/1713357192,v1:172.21.15.100:6811/1713357192]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-10T11:09:47.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:46 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:09:47.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:46 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:47.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:46 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:47.481 DEBUG:teuthology.orchestra.run.smithi100:osd.3> sudo journalctl -f -n 0 -u ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@osd.3.service 2024-05-10T11:09:47.483 INFO:tasks.cephadm:Deploying osd.4 on smithi178 with /dev/vg_nvme/lv_3... 2024-05-10T11:09:47.483 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-10T11:09:47.730 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T11:09:48.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:47 smithi100 ceph-mon[30296]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-10T11:09:48.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:47 smithi100 ceph-mon[30296]: from='osd.3 [v2:172.21.15.100:6810/1713357192,v1:172.21.15.100:6811/1713357192]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-10T11:09:48.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:47 smithi100 ceph-mon[30296]: osdmap e24: 4 total, 3 up, 4 in 2024-05-10T11:09:48.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:47 smithi100 ceph-mon[30296]: from='osd.3 [v2:172.21.15.100:6810/1713357192,v1:172.21.15.100:6811/1713357192]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-05-10T11:09:48.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:47 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T11:09:48.165 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:47 smithi178 ceph-mon[33359]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-10T11:09:48.165 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:47 smithi178 ceph-mon[33359]: from='osd.3 [v2:172.21.15.100:6810/1713357192,v1:172.21.15.100:6811/1713357192]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-10T11:09:48.165 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:47 smithi178 ceph-mon[33359]: osdmap e24: 4 total, 3 up, 4 in 2024-05-10T11:09:48.165 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:47 smithi178 ceph-mon[33359]: from='osd.3 [v2:172.21.15.100:6810/1713357192,v1:172.21.15.100:6811/1713357192]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-05-10T11:09:48.165 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:47 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T11:09:48.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:47 smithi188 ceph-mon[34075]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-10T11:09:48.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:47 smithi188 ceph-mon[34075]: from='osd.3 [v2:172.21.15.100:6810/1713357192,v1:172.21.15.100:6811/1713357192]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-10T11:09:48.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:47 smithi188 ceph-mon[34075]: osdmap e24: 4 total, 3 up, 4 in 2024-05-10T11:09:48.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:47 smithi188 ceph-mon[34075]: from='osd.3 [v2:172.21.15.100:6810/1713357192,v1:172.21.15.100:6811/1713357192]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-05-10T11:09:48.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:47 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T11:09:48.637 INFO:journalctl@ceph.osd.3.smithi100.stdout:May 10 11:09:48 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-3[49281]: 2024-05-10T11:09:48.550+0000 7f950d573700 -1 osd.3 0 waiting for initial osdmap 2024-05-10T11:09:48.637 INFO:journalctl@ceph.osd.3.smithi100.stdout:May 10 11:09:48 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-3[49281]: 2024-05-10T11:09:48.557+0000 7f950535e700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T11:09:49.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:48 smithi178 ceph-mon[33359]: from='osd.3 [v2:172.21.15.100:6810/1713357192,v1:172.21.15.100:6811/1713357192]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-05-10T11:09:49.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:48 smithi178 ceph-mon[33359]: osdmap e25: 4 total, 3 up, 4 in 2024-05-10T11:09:49.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:48 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T11:09:49.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:48 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T11:09:49.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:48 smithi178 ceph-mon[33359]: from='osd.3 [v2:172.21.15.100:6810/1713357192,v1:172.21.15.100:6811/1713357192]' entity='osd.3' 2024-05-10T11:09:49.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:48 smithi178 ceph-mon[33359]: osd.3 [v2:172.21.15.100:6810/1713357192,v1:172.21.15.100:6811/1713357192] boot 2024-05-10T11:09:49.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:48 smithi178 ceph-mon[33359]: osdmap e26: 4 total, 4 up, 4 in 2024-05-10T11:09:49.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:48 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T11:09:49.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:48 smithi188 ceph-mon[34075]: from='osd.3 [v2:172.21.15.100:6810/1713357192,v1:172.21.15.100:6811/1713357192]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-05-10T11:09:49.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:48 smithi188 ceph-mon[34075]: osdmap e25: 4 total, 3 up, 4 in 2024-05-10T11:09:49.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:48 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T11:09:49.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:48 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T11:09:49.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:48 smithi188 ceph-mon[34075]: from='osd.3 [v2:172.21.15.100:6810/1713357192,v1:172.21.15.100:6811/1713357192]' entity='osd.3' 2024-05-10T11:09:49.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:48 smithi188 ceph-mon[34075]: osd.3 [v2:172.21.15.100:6810/1713357192,v1:172.21.15.100:6811/1713357192] boot 2024-05-10T11:09:49.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:48 smithi188 ceph-mon[34075]: osdmap e26: 4 total, 4 up, 4 in 2024-05-10T11:09:49.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:48 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T11:09:49.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:48 smithi100 ceph-mon[30296]: from='osd.3 [v2:172.21.15.100:6810/1713357192,v1:172.21.15.100:6811/1713357192]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-05-10T11:09:49.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:48 smithi100 ceph-mon[30296]: osdmap e25: 4 total, 3 up, 4 in 2024-05-10T11:09:49.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:48 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T11:09:49.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:48 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T11:09:49.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:48 smithi100 ceph-mon[30296]: from='osd.3 [v2:172.21.15.100:6810/1713357192,v1:172.21.15.100:6811/1713357192]' entity='osd.3' 2024-05-10T11:09:49.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:48 smithi100 ceph-mon[30296]: osd.3 [v2:172.21.15.100:6810/1713357192,v1:172.21.15.100:6811/1713357192] boot 2024-05-10T11:09:49.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:48 smithi100 ceph-mon[30296]: osdmap e26: 4 total, 4 up, 4 in 2024-05-10T11:09:49.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:48 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T11:09:49.861 INFO:teuthology.orchestra.run.smithi178.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-10T11:09:49.861 INFO:teuthology.orchestra.run.smithi178.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-10T11:09:49.861 INFO:teuthology.orchestra.run.smithi178.stderr: stderr: 10+0 records in 2024-05-10T11:09:49.861 INFO:teuthology.orchestra.run.smithi178.stderr:10+0 records out 2024-05-10T11:09:49.861 INFO:teuthology.orchestra.run.smithi178.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00941235 s, 1.1 GB/s 2024-05-10T11:09:49.861 INFO:teuthology.orchestra.run.smithi178.stderr:--> Zapping successful for: 2024-05-10T11:09:50.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:49 smithi100 ceph-mon[30296]: purged_snaps scrub starts 2024-05-10T11:09:50.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:49 smithi100 ceph-mon[30296]: purged_snaps scrub ok 2024-05-10T11:09:50.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:49 smithi100 ceph-mon[30296]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-10T11:09:50.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:49 smithi100 ceph-mon[30296]: osdmap e27: 4 total, 4 up, 4 in 2024-05-10T11:09:50.213 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:49 smithi178 ceph-mon[33359]: purged_snaps scrub starts 2024-05-10T11:09:50.214 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:49 smithi178 ceph-mon[33359]: purged_snaps scrub ok 2024-05-10T11:09:50.214 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:49 smithi178 ceph-mon[33359]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-10T11:09:50.214 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:49 smithi178 ceph-mon[33359]: osdmap e27: 4 total, 4 up, 4 in 2024-05-10T11:09:50.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:49 smithi188 ceph-mon[34075]: purged_snaps scrub starts 2024-05-10T11:09:50.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:49 smithi188 ceph-mon[34075]: purged_snaps scrub ok 2024-05-10T11:09:50.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:49 smithi188 ceph-mon[34075]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-10T11:09:50.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:49 smithi188 ceph-mon[34075]: osdmap e27: 4 total, 4 up, 4 in 2024-05-10T11:09:50.461 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi178:vg_nvme/lv_3 2024-05-10T11:09:50.714 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T11:09:51.700 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:51 smithi178 ceph-mon[33359]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-10T11:09:51.700 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:51 smithi178 ceph-mon[33359]: Detected new or changed devices on smithi100 2024-05-10T11:09:51.700 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:51 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:51.700 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:51 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:51.700 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:51 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:09:51.700 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:51 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:09:51.700 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:51 smithi178 ceph-mon[33359]: Adjusting osd_memory_target on smithi100 to 8544M 2024-05-10T11:09:51.700 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:51 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:51.700 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:51 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:09:51.700 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:51 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:09:51.701 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:51 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:51.701 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:51 smithi178 ceph-mon[33359]: osdmap e28: 4 total, 4 up, 4 in 2024-05-10T11:09:51.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:51 smithi188 ceph-mon[34075]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-10T11:09:51.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:51 smithi188 ceph-mon[34075]: Detected new or changed devices on smithi100 2024-05-10T11:09:51.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:51 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:51.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:51 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:51.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:51 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:09:51.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:51 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:09:51.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:51 smithi188 ceph-mon[34075]: Adjusting osd_memory_target on smithi100 to 8544M 2024-05-10T11:09:51.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:51 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:51.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:51 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:09:51.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:51 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:09:51.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:51 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:51.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:51 smithi188 ceph-mon[34075]: osdmap e28: 4 total, 4 up, 4 in 2024-05-10T11:09:51.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:51 smithi100 ceph-mon[30296]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-10T11:09:51.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:51 smithi100 ceph-mon[30296]: Detected new or changed devices on smithi100 2024-05-10T11:09:51.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:51 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:51.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:51 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:51.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:51 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:09:51.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:51 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:09:51.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:51 smithi100 ceph-mon[30296]: Adjusting osd_memory_target on smithi100 to 8544M 2024-05-10T11:09:51.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:51 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:51.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:51 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:09:51.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:51 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:09:51.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:51 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:09:51.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:51 smithi100 ceph-mon[30296]: osdmap e28: 4 total, 4 up, 4 in 2024-05-10T11:09:52.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:09:52.251+0000 7f674bf5a700 1 -- 172.21.15.178:0/898349539 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6744101b90 msgr2=0x7f6744101fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:09:52.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:09:52.251+0000 7f674bf5a700 1 --2- 172.21.15.178:0/898349539 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6744101b90 0x7f6744101fb0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f67400099c0 tx=0x7f6740009cd0 comp rx=0 tx=0).stop 2024-05-10T11:09:52.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:09:52.251+0000 7f674bf5a700 1 -- 172.21.15.178:0/898349539 shutdown_connections 2024-05-10T11:09:52.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:09:52.251+0000 7f674bf5a700 1 --2- 172.21.15.178:0/898349539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6744103730 0x7f6744109fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:09:52.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:09:52.251+0000 7f674bf5a700 1 --2- 172.21.15.178:0/898349539 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6744101b90 0x7f6744101fb0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:09:52.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:09:52.251+0000 7f674bf5a700 1 --2- 172.21.15.178:0/898349539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6744102d90 0x7f67441031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:09:52.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:09:52.251+0000 7f674bf5a700 1 -- 172.21.15.178:0/898349539 >> 172.21.15.178:0/898349539 conn(0x7f67440fd110 msgr2=0x7f67440ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:09:52.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:09:52.251+0000 7f674bf5a700 1 -- 172.21.15.178:0/898349539 shutdown_connections 2024-05-10T11:09:52.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:09:52.251+0000 7f674bf5a700 1 -- 172.21.15.178:0/898349539 wait complete. 2024-05-10T11:09:52.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:09:52.251+0000 7f674bf5a700 1 Processor -- start 2024-05-10T11:09:52.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:09:52.251+0000 7f674bf5a700 1 -- start start 2024-05-10T11:09:52.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:09:52.252+0000 7f674bf5a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6744101b90 0x7f674410b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:09:52.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:09:52.252+0000 7f674bf5a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6744102d90 0x7f674419e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:09:52.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:09:52.252+0000 7f674bf5a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6744103730 0x7f67441a0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:09:52.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:09:52.252+0000 7f674bf5a700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f674410ddd0 con 0x7f6744101b90 2024-05-10T11:09:52.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:09:52.252+0000 7f674bf5a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f674410dc50 con 0x7f6744103730 2024-05-10T11:09:52.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:09:52.252+0000 7f674bf5a700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f674410df50 con 0x7f6744102d90 2024-05-10T11:09:52.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:09:52.252+0000 7f674a4f7700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6744103730 0x7f67441a0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:09:52.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:09:52.252+0000 7f674a4f7700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6744103730 0x7f67441a0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:40592/0 (socket says 172.21.15.178:40592) 2024-05-10T11:09:52.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:09:52.252+0000 7f674a4f7700 1 -- 172.21.15.178:0/2848041423 learned_addr learned my addr 172.21.15.178:0/2848041423 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-05-10T11:09:52.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:09:52.252+0000 7f6749cf6700 1 --2- 172.21.15.178:0/2848041423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6744101b90 0x7f674410b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:09:52.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:09:52.252+0000 7f67494f5700 1 --2- 172.21.15.178:0/2848041423 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6744102d90 0x7f674419e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:09:52.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:09:52.252+0000 7f674a4f7700 1 -- 172.21.15.178:0/2848041423 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6744102d90 msgr2=0x7f674419e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:09:52.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:09:52.252+0000 7f674a4f7700 1 --2- 172.21.15.178:0/2848041423 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6744102d90 0x7f674419e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:09:52.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:09:52.252+0000 7f674a4f7700 1 -- 172.21.15.178:0/2848041423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6744101b90 msgr2=0x7f674410b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:09:52.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:09:52.252+0000 7f674a4f7700 1 --2- 172.21.15.178:0/2848041423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6744101b90 0x7f674410b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:09:52.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:09:52.252+0000 7f674a4f7700 1 -- 172.21.15.178:0/2848041423 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67441a1220 con 0x7f6744103730 2024-05-10T11:09:52.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:09:52.252+0000 7f67494f5700 1 --2- 172.21.15.178:0/2848041423 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6744102d90 0x7f674419e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:09:52.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:09:52.252+0000 7f674a4f7700 1 --2- 172.21.15.178:0/2848041423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6744103730 0x7f67441a0950 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f673c007ee0 tx=0x7f673c00d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:09:52.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:09:52.252+0000 7f673affd700 1 -- 172.21.15.178:0/2848041423 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f673c00dbd0 con 0x7f6744103730 2024-05-10T11:09:52.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:09:52.252+0000 7f673affd700 1 -- 172.21.15.178:0/2848041423 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f673c00dd70 con 0x7f6744103730 2024-05-10T11:09:52.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:09:52.252+0000 7f673affd700 1 -- 172.21.15.178:0/2848041423 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f673c0058d0 con 0x7f6744103730 2024-05-10T11:09:52.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:09:52.252+0000 7f674bf5a700 1 -- 172.21.15.178:0/2848041423 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67441a1510 con 0x7f6744103730 2024-05-10T11:09:52.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:09:52.252+0000 7f674bf5a700 1 -- 172.21.15.178:0/2848041423 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6744198300 con 0x7f6744103730 2024-05-10T11:09:52.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:09:52.253+0000 7f674bf5a700 1 -- 172.21.15.178:0/2848041423 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f674402e510 con 0x7f6744103730 2024-05-10T11:09:52.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:09:52.255+0000 7f673affd700 1 -- 172.21.15.178:0/2848041423 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f673c01a020 con 0x7f6744103730 2024-05-10T11:09:52.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:09:52.255+0000 7f673affd700 1 --2- 172.21.15.178:0/2848041423 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f67300b2900 0x7f67300b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:09:52.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:09:52.255+0000 7f673affd700 1 -- 172.21.15.178:0/2848041423 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3669+0+0 (secure 0 0 0) 0x7f673c0e0df0 con 0x7f6744103730 2024-05-10T11:09:52.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:09:52.256+0000 7f6749cf6700 1 --2- 172.21.15.178:0/2848041423 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f67300b2900 0x7f67300b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:09:52.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:09:52.256+0000 7f6749cf6700 1 --2- 172.21.15.178:0/2848041423 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f67300b2900 0x7f67300b4dc0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6740009990 tx=0x7f674001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:09:52.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:09:52.259+0000 7f673affd700 1 -- 172.21.15.178:0/2848041423 <== mon.2 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f673c010070 con 0x7f6744103730 2024-05-10T11:09:52.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:09:52.468+0000 7f674bf5a700 1 -- 172.21.15.178:0/2848041423 --> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f67440638e0 con 0x7f67300b2900 2024-05-10T11:09:52.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:52 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T11:09:52.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:52 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T11:09:52.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:52 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:09:52.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:52 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T11:09:52.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:52 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T11:09:52.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:52 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:09:53.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:52 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T11:09:53.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:52 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T11:09:53.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:52 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:09:53.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:53 smithi178 ceph-mon[33359]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 135 KiB/s, 0 objects/s recovering 2024-05-10T11:09:53.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:53 smithi178 ceph-mon[33359]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:09:53.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:53 smithi100 ceph-mon[30296]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 135 KiB/s, 0 objects/s recovering 2024-05-10T11:09:53.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:53 smithi100 ceph-mon[30296]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:09:54.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:53 smithi188 ceph-mon[34075]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 135 KiB/s, 0 objects/s recovering 2024-05-10T11:09:54.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:53 smithi188 ceph-mon[34075]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:09:55.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:55 smithi178 ceph-mon[33359]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-10T11:09:55.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:55 smithi178 ceph-mon[33359]: from='client.? 172.21.15.178:0/1932010850' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a9653800-da57-43eb-8781-3188e8a3949f"}]: dispatch 2024-05-10T11:09:55.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:55 smithi178 ceph-mon[33359]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a9653800-da57-43eb-8781-3188e8a3949f"}]: dispatch 2024-05-10T11:09:55.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:55 smithi178 ceph-mon[33359]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a9653800-da57-43eb-8781-3188e8a3949f"}]': finished 2024-05-10T11:09:55.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:55 smithi178 ceph-mon[33359]: osdmap e29: 5 total, 4 up, 5 in 2024-05-10T11:09:55.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:55 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T11:09:55.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:55 smithi100 ceph-mon[30296]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-10T11:09:55.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:55 smithi100 ceph-mon[30296]: from='client.? 172.21.15.178:0/1932010850' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a9653800-da57-43eb-8781-3188e8a3949f"}]: dispatch 2024-05-10T11:09:55.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:55 smithi100 ceph-mon[30296]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a9653800-da57-43eb-8781-3188e8a3949f"}]: dispatch 2024-05-10T11:09:55.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:55 smithi100 ceph-mon[30296]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a9653800-da57-43eb-8781-3188e8a3949f"}]': finished 2024-05-10T11:09:55.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:55 smithi100 ceph-mon[30296]: osdmap e29: 5 total, 4 up, 5 in 2024-05-10T11:09:55.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:55 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T11:09:56.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:55 smithi188 ceph-mon[34075]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-10T11:09:56.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:55 smithi188 ceph-mon[34075]: from='client.? 172.21.15.178:0/1932010850' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a9653800-da57-43eb-8781-3188e8a3949f"}]: dispatch 2024-05-10T11:09:56.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:55 smithi188 ceph-mon[34075]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a9653800-da57-43eb-8781-3188e8a3949f"}]: dispatch 2024-05-10T11:09:56.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:55 smithi188 ceph-mon[34075]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a9653800-da57-43eb-8781-3188e8a3949f"}]': finished 2024-05-10T11:09:56.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:55 smithi188 ceph-mon[34075]: osdmap e29: 5 total, 4 up, 5 in 2024-05-10T11:09:56.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:55 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T11:09:57.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:56 smithi178 ceph-mon[33359]: from='client.? 172.21.15.178:0/3754240275' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T11:09:57.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:56 smithi188 ceph-mon[34075]: from='client.? 172.21.15.178:0/3754240275' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T11:09:57.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:56 smithi100 ceph-mon[30296]: from='client.? 172.21.15.178:0/3754240275' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T11:09:58.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:57 smithi178 ceph-mon[33359]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-05-10T11:09:58.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:57 smithi188 ceph-mon[34075]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-05-10T11:09:58.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:57 smithi100 ceph-mon[30296]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-05-10T11:09:59.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:09:59 smithi178 ceph-mon[33359]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-10T11:10:00.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:09:59 smithi188 ceph-mon[34075]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-10T11:10:00.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:09:59 smithi100 ceph-mon[30296]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-10T11:10:00.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:00 smithi178 ceph-mon[33359]: overall HEALTH_OK 2024-05-10T11:10:01.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:00 smithi188 ceph-mon[34075]: overall HEALTH_OK 2024-05-10T11:10:01.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:00 smithi100 ceph-mon[30296]: overall HEALTH_OK 2024-05-10T11:10:01.707 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:01 smithi178 ceph-mon[33359]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-05-10T11:10:02.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:01 smithi188 ceph-mon[34075]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-05-10T11:10:02.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:01 smithi100 ceph-mon[30296]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-05-10T11:10:03.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:02 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-10T11:10:03.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:02 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:10:03.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:02 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-10T11:10:03.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:02 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:10:03.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:02 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-10T11:10:03.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:02 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:10:03.692 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:03 smithi178 ceph-mon[33359]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-10T11:10:03.692 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:03 smithi178 ceph-mon[33359]: Deploying daemon osd.4 on smithi178 2024-05-10T11:10:04.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:03 smithi188 ceph-mon[34075]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-10T11:10:04.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:03 smithi188 ceph-mon[34075]: Deploying daemon osd.4 on smithi178 2024-05-10T11:10:04.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:03 smithi100 ceph-mon[30296]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-10T11:10:04.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:03 smithi100 ceph-mon[30296]: Deploying daemon osd.4 on smithi178 2024-05-10T11:10:05.840 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:05 smithi100 ceph-mon[30296]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-10T11:10:06.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:05 smithi178 ceph-mon[33359]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-10T11:10:06.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:05 smithi188 ceph-mon[34075]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-10T11:10:07.688 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:07 smithi178 ceph-mon[33359]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-10T11:10:08.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:07 smithi188 ceph-mon[34075]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-10T11:10:08.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:07 smithi100 ceph-mon[30296]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-10T11:10:09.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:08 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:10:09.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:08 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:10:09.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:08 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:10:09.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:08 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:10:09.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:08 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:10:09.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:08 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:10:09.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:08 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:10:09.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:08 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:10:09.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:08 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:10:10.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:09 smithi178 ceph-mon[33359]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-10T11:10:10.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:09 smithi188 ceph-mon[34075]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-10T11:10:10.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:09 smithi100 ceph-mon[30296]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-10T11:10:11.832 INFO:teuthology.orchestra.run.smithi178.stdout:Created osd(s) 4 on host 'smithi178' 2024-05-10T11:10:11.832 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:10:11.831+0000 7f673affd700 1 -- 172.21.15.178:0/2848041423 <== mgr.14152 v2:172.21.15.100:6800/2507287356 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f67440638e0 con 0x7f67300b2900 2024-05-10T11:10:11.834 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:10:11.834+0000 7f674bf5a700 1 -- 172.21.15.178:0/2848041423 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f67300b2900 msgr2=0x7f67300b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:10:11.834 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:10:11.834+0000 7f674bf5a700 1 --2- 172.21.15.178:0/2848041423 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f67300b2900 0x7f67300b4dc0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6740009990 tx=0x7f674001a040 comp rx=0 tx=0).stop 2024-05-10T11:10:11.834 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:10:11.834+0000 7f674bf5a700 1 -- 172.21.15.178:0/2848041423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6744103730 msgr2=0x7f67441a0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:10:11.835 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:10:11.834+0000 7f674bf5a700 1 --2- 172.21.15.178:0/2848041423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6744103730 0x7f67441a0950 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f673c007ee0 tx=0x7f673c00d3b0 comp rx=0 tx=0).stop 2024-05-10T11:10:11.835 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:10:11.834+0000 7f674bf5a700 1 -- 172.21.15.178:0/2848041423 shutdown_connections 2024-05-10T11:10:11.835 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:10:11.834+0000 7f674bf5a700 1 --2- 172.21.15.178:0/2848041423 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f67300b2900 0x7f67300b4dc0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:10:11.835 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:10:11.834+0000 7f674bf5a700 1 --2- 172.21.15.178:0/2848041423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6744103730 0x7f67441a0950 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:10:11.835 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:10:11.834+0000 7f674bf5a700 1 --2- 172.21.15.178:0/2848041423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6744101b90 0x7f674410b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:10:11.835 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:10:11.834+0000 7f674bf5a700 1 --2- 172.21.15.178:0/2848041423 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6744102d90 0x7f674419e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:10:11.835 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:10:11.834+0000 7f674bf5a700 1 -- 172.21.15.178:0/2848041423 >> 172.21.15.178:0/2848041423 conn(0x7f67440fd110 msgr2=0x7f6744106160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:10:11.835 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:10:11.834+0000 7f674bf5a700 1 -- 172.21.15.178:0/2848041423 shutdown_connections 2024-05-10T11:10:11.835 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:10:11.834+0000 7f674bf5a700 1 -- 172.21.15.178:0/2848041423 wait complete. 2024-05-10T11:10:12.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:11 smithi178 ceph-mon[33359]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-10T11:10:12.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:11 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:10:12.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:11 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:10:12.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:11 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:10:12.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:11 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:10:12.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:11 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:10:12.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:11 smithi188 ceph-mon[34075]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-10T11:10:12.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:11 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:10:12.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:11 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:10:12.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:11 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:10:12.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:11 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:10:12.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:11 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:10:12.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:11 smithi100 ceph-mon[30296]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-10T11:10:12.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:11 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:10:12.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:11 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:10:12.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:11 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:10:12.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:11 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:10:12.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:11 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:10:12.420 DEBUG:teuthology.orchestra.run.smithi178:osd.4> sudo journalctl -f -n 0 -u ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@osd.4.service 2024-05-10T11:10:12.423 INFO:tasks.cephadm:Deploying osd.5 on smithi188 with /dev/vg_nvme/lv_3... 2024-05-10T11:10:12.423 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-10T11:10:12.672 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T11:10:13.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:12 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:10:13.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:12 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:10:13.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:12 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:10:13.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:12 smithi178 ceph-mon[33359]: from='osd.4 [v2:172.21.15.178:6808/2222814192,v1:172.21.15.178:6809/2222814192]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-10T11:10:13.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:12 smithi178 ceph-mon[33359]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-10T11:10:13.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:12 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:10:13.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:12 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:10:13.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:12 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:10:13.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:12 smithi188 ceph-mon[34075]: from='osd.4 [v2:172.21.15.178:6808/2222814192,v1:172.21.15.178:6809/2222814192]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-10T11:10:13.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:12 smithi188 ceph-mon[34075]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-10T11:10:13.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:12 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:10:13.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:12 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:10:13.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:12 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:10:13.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:12 smithi100 ceph-mon[30296]: from='osd.4 [v2:172.21.15.178:6808/2222814192,v1:172.21.15.178:6809/2222814192]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-10T11:10:13.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:12 smithi100 ceph-mon[30296]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-10T11:10:13.833 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:13 smithi178 ceph-mon[33359]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-10T11:10:13.833 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:13 smithi178 ceph-mon[33359]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-10T11:10:13.833 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:13 smithi178 ceph-mon[33359]: from='osd.4 [v2:172.21.15.178:6808/2222814192,v1:172.21.15.178:6809/2222814192]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-05-10T11:10:13.834 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:13 smithi178 ceph-mon[33359]: osdmap e30: 5 total, 4 up, 5 in 2024-05-10T11:10:13.834 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:13 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T11:10:13.834 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:13 smithi178 ceph-mon[33359]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-05-10T11:10:14.016 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:13 smithi188 ceph-mon[34075]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-10T11:10:14.016 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:13 smithi188 ceph-mon[34075]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-10T11:10:14.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:13 smithi188 ceph-mon[34075]: from='osd.4 [v2:172.21.15.178:6808/2222814192,v1:172.21.15.178:6809/2222814192]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-05-10T11:10:14.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:13 smithi188 ceph-mon[34075]: osdmap e30: 5 total, 4 up, 5 in 2024-05-10T11:10:14.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:13 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T11:10:14.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:13 smithi188 ceph-mon[34075]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-05-10T11:10:14.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:13 smithi100 ceph-mon[30296]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-10T11:10:14.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:13 smithi100 ceph-mon[30296]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-10T11:10:14.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:13 smithi100 ceph-mon[30296]: from='osd.4 [v2:172.21.15.178:6808/2222814192,v1:172.21.15.178:6809/2222814192]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-05-10T11:10:14.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:13 smithi100 ceph-mon[30296]: osdmap e30: 5 total, 4 up, 5 in 2024-05-10T11:10:14.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:13 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T11:10:14.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:13 smithi100 ceph-mon[30296]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-05-10T11:10:14.599 INFO:teuthology.orchestra.run.smithi188.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-10T11:10:14.599 INFO:teuthology.orchestra.run.smithi188.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-10T11:10:14.599 INFO:teuthology.orchestra.run.smithi188.stderr: stderr: 10+0 records in 2024-05-10T11:10:14.599 INFO:teuthology.orchestra.run.smithi188.stderr:10+0 records out 2024-05-10T11:10:14.599 INFO:teuthology.orchestra.run.smithi188.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112973 s, 928 MB/s 2024-05-10T11:10:14.599 INFO:teuthology.orchestra.run.smithi188.stderr:--> Zapping successful for: 2024-05-10T11:10:14.753 INFO:journalctl@ceph.osd.4.smithi178.stdout:May 10 11:10:14 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-4[44733]: 2024-05-10T11:10:14.309+0000 7f56366ae700 -1 osd.4 0 waiting for initial osdmap 2024-05-10T11:10:14.753 INFO:journalctl@ceph.osd.4.smithi178.stdout:May 10 11:10:14 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-4[44733]: 2024-05-10T11:10:14.315+0000 7f562e499700 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T11:10:14.941 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:14 smithi188 ceph-mon[34075]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-05-10T11:10:14.942 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:14 smithi188 ceph-mon[34075]: osdmap e31: 5 total, 4 up, 5 in 2024-05-10T11:10:14.942 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:14 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T11:10:14.942 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:14 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T11:10:14.942 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:14 smithi188 ceph-mon[34075]: from='osd.4 ' entity='osd.4' 2024-05-10T11:10:15.061 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:14 smithi178 ceph-mon[33359]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-05-10T11:10:15.061 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:14 smithi178 ceph-mon[33359]: osdmap e31: 5 total, 4 up, 5 in 2024-05-10T11:10:15.062 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:14 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T11:10:15.062 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:14 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T11:10:15.062 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:14 smithi178 ceph-mon[33359]: from='osd.4 ' entity='osd.4' 2024-05-10T11:10:15.251 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi188:vg_nvme/lv_3 2024-05-10T11:10:15.290 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:14 smithi100 ceph-mon[30296]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-05-10T11:10:15.290 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:14 smithi100 ceph-mon[30296]: osdmap e31: 5 total, 4 up, 5 in 2024-05-10T11:10:15.290 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:14 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T11:10:15.290 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:14 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T11:10:15.290 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:14 smithi100 ceph-mon[30296]: from='osd.4 ' entity='osd.4' 2024-05-10T11:10:15.495 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T11:10:15.870 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:15 smithi188 ceph-mon[34075]: purged_snaps scrub starts 2024-05-10T11:10:15.870 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:15 smithi188 ceph-mon[34075]: purged_snaps scrub ok 2024-05-10T11:10:15.870 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:15 smithi188 ceph-mon[34075]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-10T11:10:15.870 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:15 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T11:10:15.870 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:15 smithi188 ceph-mon[34075]: osd.4 [v2:172.21.15.178:6808/2222814192,v1:172.21.15.178:6809/2222814192] boot 2024-05-10T11:10:15.870 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:15 smithi188 ceph-mon[34075]: osdmap e32: 5 total, 5 up, 5 in 2024-05-10T11:10:15.870 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:15 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T11:10:15.870 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:15 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:10:15.871 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:15 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:10:15.871 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:15 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:10:15.871 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:15 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:10:15.871 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:15 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:10:15.871 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:15 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:10:15.871 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:15 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:10:15.871 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:15 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:10:16.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:15 smithi178 ceph-mon[33359]: purged_snaps scrub starts 2024-05-10T11:10:16.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:15 smithi178 ceph-mon[33359]: purged_snaps scrub ok 2024-05-10T11:10:16.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:15 smithi178 ceph-mon[33359]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-10T11:10:16.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:15 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T11:10:16.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:15 smithi178 ceph-mon[33359]: osd.4 [v2:172.21.15.178:6808/2222814192,v1:172.21.15.178:6809/2222814192] boot 2024-05-10T11:10:16.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:15 smithi178 ceph-mon[33359]: osdmap e32: 5 total, 5 up, 5 in 2024-05-10T11:10:16.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:15 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T11:10:16.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:15 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:10:16.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:15 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:10:16.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:15 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:10:16.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:15 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:10:16.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:15 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:10:16.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:15 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:10:16.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:15 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:10:16.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:15 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:10:16.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:15 smithi100 ceph-mon[30296]: purged_snaps scrub starts 2024-05-10T11:10:16.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:15 smithi100 ceph-mon[30296]: purged_snaps scrub ok 2024-05-10T11:10:16.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:15 smithi100 ceph-mon[30296]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-10T11:10:16.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:15 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T11:10:16.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:15 smithi100 ceph-mon[30296]: osd.4 [v2:172.21.15.178:6808/2222814192,v1:172.21.15.178:6809/2222814192] boot 2024-05-10T11:10:16.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:15 smithi100 ceph-mon[30296]: osdmap e32: 5 total, 5 up, 5 in 2024-05-10T11:10:16.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:15 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T11:10:16.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:15 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:10:16.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:15 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:10:16.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:15 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:10:16.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:15 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:10:16.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:15 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:10:16.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:15 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:10:16.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:15 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:10:16.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:15 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:10:16.986 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:16 smithi188 ceph-mon[34075]: Detected new or changed devices on smithi178 2024-05-10T11:10:16.986 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:16 smithi188 ceph-mon[34075]: Adjusting osd_memory_target on smithi178 to 8544M 2024-05-10T11:10:16.987 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:16 smithi188 ceph-mon[34075]: osdmap e33: 5 total, 5 up, 5 in 2024-05-10T11:10:17.114 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:16 smithi100 ceph-mon[30296]: Detected new or changed devices on smithi178 2024-05-10T11:10:17.114 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:16 smithi100 ceph-mon[30296]: Adjusting osd_memory_target on smithi178 to 8544M 2024-05-10T11:10:17.114 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:16 smithi100 ceph-mon[30296]: osdmap e33: 5 total, 5 up, 5 in 2024-05-10T11:10:17.239 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:10:17.237+0000 7fdeb1e58700 1 -- 172.21.15.188:0/2580310457 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdeac1055f0 msgr2=0x7fdeac107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:10:17.239 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:10:17.237+0000 7fdeb1e58700 1 --2- 172.21.15.188:0/2580310457 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdeac1055f0 0x7fdeac107a00 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fde94009990 tx=0x7fde94009ca0 comp rx=0 tx=0).stop 2024-05-10T11:10:17.239 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:10:17.237+0000 7fdeb1e58700 1 -- 172.21.15.188:0/2580310457 shutdown_connections 2024-05-10T11:10:17.239 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:10:17.237+0000 7fdeb1e58700 1 --2- 172.21.15.188:0/2580310457 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdeac108090 0x7fdeac10a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:10:17.239 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:10:17.237+0000 7fdeb1e58700 1 --2- 172.21.15.188:0/2580310457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdeac102c30 0x7fdeac105020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:10:17.240 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:10:17.237+0000 7fdeb1e58700 1 --2- 172.21.15.188:0/2580310457 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdeac1055f0 0x7fdeac107a00 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:10:17.240 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:10:17.237+0000 7fdeb1e58700 1 -- 172.21.15.188:0/2580310457 >> 172.21.15.188:0/2580310457 conn(0x7fdeac0fccf0 msgr2=0x7fdeac0ff110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:10:17.240 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:10:17.237+0000 7fdeb1e58700 1 -- 172.21.15.188:0/2580310457 shutdown_connections 2024-05-10T11:10:17.240 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:10:17.237+0000 7fdeb1e58700 1 -- 172.21.15.188:0/2580310457 wait complete. 2024-05-10T11:10:17.240 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:10:17.237+0000 7fdeb1e58700 1 Processor -- start 2024-05-10T11:10:17.240 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:10:17.238+0000 7fdeb1e58700 1 -- start start 2024-05-10T11:10:17.240 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:10:17.238+0000 7fdeb1e58700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdeac102c30 0x7fdeac199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:10:17.240 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:10:17.238+0000 7fdeb1e58700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdeac1055f0 0x7fdeac199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:10:17.240 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:10:17.238+0000 7fdeb1e58700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdeac108090 0x7fdeac1a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:10:17.240 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:10:17.238+0000 7fdeb1e58700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdeac10dd50 con 0x7fdeac1055f0 2024-05-10T11:10:17.240 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:10:17.238+0000 7fdeb1e58700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdeac10dbd0 con 0x7fdeac108090 2024-05-10T11:10:17.240 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:10:17.238+0000 7fdeb1e58700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fdeac10ded0 con 0x7fdeac102c30 2024-05-10T11:10:17.240 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:10:17.238+0000 7fdeab7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdeac102c30 0x7fdeac199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:10:17.241 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:10:17.238+0000 7fdeab7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdeac102c30 0x7fdeac199600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:57844/0 (socket says 172.21.15.188:57844) 2024-05-10T11:10:17.241 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:10:17.238+0000 7fdeab7fe700 1 -- 172.21.15.188:0/1912588887 learned_addr learned my addr 172.21.15.188:0/1912588887 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-05-10T11:10:17.241 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:10:17.238+0000 7fdeaaffd700 1 --2- 172.21.15.188:0/1912588887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdeac1055f0 0x7fdeac199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:10:17.241 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:10:17.238+0000 7fdeabfff700 1 --2- 172.21.15.188:0/1912588887 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdeac108090 0x7fdeac1a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:10:17.241 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:10:17.238+0000 7fdeab7fe700 1 -- 172.21.15.188:0/1912588887 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdeac108090 msgr2=0x7fdeac1a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:10:17.241 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:10:17.238+0000 7fdeab7fe700 1 --2- 172.21.15.188:0/1912588887 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdeac108090 0x7fdeac1a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:10:17.241 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:10:17.238+0000 7fdeab7fe700 1 -- 172.21.15.188:0/1912588887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdeac1055f0 msgr2=0x7fdeac199b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:10:17.241 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:10:17.238+0000 7fdeab7fe700 1 --2- 172.21.15.188:0/1912588887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdeac1055f0 0x7fdeac199b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:10:17.241 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:10:17.238+0000 7fdeab7fe700 1 -- 172.21.15.188:0/1912588887 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdeac1a11f0 con 0x7fdeac102c30 2024-05-10T11:10:17.241 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:10:17.239+0000 7fdeab7fe700 1 --2- 172.21.15.188:0/1912588887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdeac102c30 0x7fdeac199600 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fde9c00c920 tx=0x7fde9c00cc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:10:17.241 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:10:17.239+0000 7fdea8ff9700 1 -- 172.21.15.188:0/1912588887 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde9c0078d0 con 0x7fdeac102c30 2024-05-10T11:10:17.241 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:10:17.239+0000 7fdea8ff9700 1 -- 172.21.15.188:0/1912588887 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fde9c00f450 con 0x7fdeac102c30 2024-05-10T11:10:17.241 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:10:17.239+0000 7fdeb1e58700 1 -- 172.21.15.188:0/1912588887 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdeac1a14e0 con 0x7fdeac102c30 2024-05-10T11:10:17.241 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:10:17.239+0000 7fdea8ff9700 1 -- 172.21.15.188:0/1912588887 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde9c018660 con 0x7fdeac102c30 2024-05-10T11:10:17.242 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:10:17.239+0000 7fdeb1e58700 1 -- 172.21.15.188:0/1912588887 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdeac1a1a70 con 0x7fdeac102c30 2024-05-10T11:10:17.243 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:10:17.240+0000 7fdeb1e58700 1 -- 172.21.15.188:0/1912588887 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdeac06a190 con 0x7fdeac102c30 2024-05-10T11:10:17.243 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:10:17.241+0000 7fdea8ff9700 1 -- 172.21.15.188:0/1912588887 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fde9c00f900 con 0x7fdeac102c30 2024-05-10T11:10:17.243 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:10:17.242+0000 7fdea8ff9700 1 --2- 172.21.15.188:0/1912588887 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fde980b2900 0x7fde980b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:10:17.243 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:10:17.242+0000 7fdea8ff9700 1 -- 172.21.15.188:0/1912588887 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4101+0+0 (secure 0 0 0) 0x7fde9c0e0f50 con 0x7fdeac102c30 2024-05-10T11:10:17.243 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:10:17.242+0000 7fdeaaffd700 1 --2- 172.21.15.188:0/1912588887 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fde980b2900 0x7fde980b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:10:17.243 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:10:17.243+0000 7fdeaaffd700 1 --2- 172.21.15.188:0/1912588887 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fde980b2900 0x7fde980b4dc0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fde940036f0 tx=0x7fde94005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:10:17.246 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:10:17.246+0000 7fdea8ff9700 1 -- 172.21.15.188:0/1912588887 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fde9c014030 con 0x7fdeac102c30 2024-05-10T11:10:17.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:16 smithi178 ceph-mon[33359]: Detected new or changed devices on smithi178 2024-05-10T11:10:17.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:16 smithi178 ceph-mon[33359]: Adjusting osd_memory_target on smithi178 to 8544M 2024-05-10T11:10:17.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:16 smithi178 ceph-mon[33359]: osdmap e33: 5 total, 5 up, 5 in 2024-05-10T11:10:17.456 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:10:17.456+0000 7fdeb1e58700 1 -- 172.21.15.188:0/1912588887 --> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fdeac0fe0e0 con 0x7fde980b2900 2024-05-10T11:10:18.072 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:17 smithi188 ceph-mon[34075]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-10T11:10:18.072 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:17 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T11:10:18.072 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:17 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T11:10:18.072 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:17 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:10:18.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:17 smithi178 ceph-mon[33359]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-10T11:10:18.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:17 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T11:10:18.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:17 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T11:10:18.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:17 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:10:18.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:17 smithi100 ceph-mon[30296]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-10T11:10:18.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:17 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T11:10:18.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:17 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T11:10:18.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:17 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:10:19.066 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:18 smithi188 ceph-mon[34075]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:10:19.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:18 smithi178 ceph-mon[33359]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:10:19.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:18 smithi100 ceph-mon[30296]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:10:20.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:19 smithi178 ceph-mon[33359]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-10T11:10:20.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:19 smithi188 ceph-mon[34075]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-10T11:10:20.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:19 smithi100 ceph-mon[30296]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-10T11:10:21.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:20 smithi188 ceph-mon[34075]: from='client.? 172.21.15.188:0/3182058588' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fb5f8af2-1eb0-4d8a-a7a3-ed6dd68e692e"}]: dispatch 2024-05-10T11:10:21.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:20 smithi188 ceph-mon[34075]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fb5f8af2-1eb0-4d8a-a7a3-ed6dd68e692e"}]: dispatch 2024-05-10T11:10:21.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:20 smithi188 ceph-mon[34075]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fb5f8af2-1eb0-4d8a-a7a3-ed6dd68e692e"}]': finished 2024-05-10T11:10:21.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:20 smithi188 ceph-mon[34075]: osdmap e34: 6 total, 5 up, 6 in 2024-05-10T11:10:21.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:20 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T11:10:21.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:20 smithi178 ceph-mon[33359]: from='client.? 172.21.15.188:0/3182058588' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fb5f8af2-1eb0-4d8a-a7a3-ed6dd68e692e"}]: dispatch 2024-05-10T11:10:21.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:20 smithi178 ceph-mon[33359]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fb5f8af2-1eb0-4d8a-a7a3-ed6dd68e692e"}]: dispatch 2024-05-10T11:10:21.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:20 smithi178 ceph-mon[33359]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fb5f8af2-1eb0-4d8a-a7a3-ed6dd68e692e"}]': finished 2024-05-10T11:10:21.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:20 smithi178 ceph-mon[33359]: osdmap e34: 6 total, 5 up, 6 in 2024-05-10T11:10:21.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:20 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T11:10:21.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:20 smithi100 ceph-mon[30296]: from='client.? 172.21.15.188:0/3182058588' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fb5f8af2-1eb0-4d8a-a7a3-ed6dd68e692e"}]: dispatch 2024-05-10T11:10:21.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:20 smithi100 ceph-mon[30296]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fb5f8af2-1eb0-4d8a-a7a3-ed6dd68e692e"}]: dispatch 2024-05-10T11:10:21.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:20 smithi100 ceph-mon[30296]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fb5f8af2-1eb0-4d8a-a7a3-ed6dd68e692e"}]': finished 2024-05-10T11:10:21.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:20 smithi100 ceph-mon[30296]: osdmap e34: 6 total, 5 up, 6 in 2024-05-10T11:10:21.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:20 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T11:10:22.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:21 smithi178 ceph-mon[33359]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-10T11:10:22.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:21 smithi178 ceph-mon[33359]: from='client.? 172.21.15.188:0/2093525863' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T11:10:22.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:21 smithi188 ceph-mon[34075]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-10T11:10:22.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:21 smithi188 ceph-mon[34075]: from='client.? 172.21.15.188:0/2093525863' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T11:10:22.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:21 smithi100 ceph-mon[30296]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-10T11:10:22.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:21 smithi100 ceph-mon[30296]: from='client.? 172.21.15.188:0/2093525863' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T11:10:24.240 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:23 smithi188 ceph-mon[34075]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-10T11:10:24.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:23 smithi178 ceph-mon[33359]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-10T11:10:24.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:23 smithi100 ceph-mon[30296]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-10T11:10:26.093 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:25 smithi178 ceph-mon[33359]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-10T11:10:26.146 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:25 smithi188 ceph-mon[34075]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-10T11:10:26.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:25 smithi100 ceph-mon[30296]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-10T11:10:28.151 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:27 smithi188 ceph-mon[34075]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-10T11:10:28.151 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:27 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-10T11:10:28.151 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:27 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:10:28.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:27 smithi178 ceph-mon[33359]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-10T11:10:28.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:27 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-10T11:10:28.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:27 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:10:28.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:27 smithi100 ceph-mon[30296]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-10T11:10:28.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:27 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-10T11:10:28.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:27 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:10:29.092 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:28 smithi188 ceph-mon[34075]: Deploying daemon osd.5 on smithi188 2024-05-10T11:10:29.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:28 smithi178 ceph-mon[33359]: Deploying daemon osd.5 on smithi188 2024-05-10T11:10:29.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:28 smithi100 ceph-mon[30296]: Deploying daemon osd.5 on smithi188 2024-05-10T11:10:29.889 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:29 smithi188 ceph-mon[34075]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-10T11:10:30.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:29 smithi178 ceph-mon[33359]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-10T11:10:30.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:29 smithi100 ceph-mon[30296]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-10T11:10:32.173 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:31 smithi188 ceph-mon[34075]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-10T11:10:32.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:31 smithi178 ceph-mon[33359]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-10T11:10:32.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:31 smithi100 ceph-mon[30296]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-10T11:10:34.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:33 smithi188 ceph-mon[34075]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-10T11:10:34.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:33 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:10:34.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:33 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:10:34.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:33 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:10:34.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:33 smithi178 ceph-mon[33359]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-10T11:10:34.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:33 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:10:34.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:33 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:10:34.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:33 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:10:34.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:33 smithi100 ceph-mon[30296]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-10T11:10:34.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:33 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:10:34.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:33 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:10:34.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:33 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:10:36.015 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:35 smithi188 ceph-mon[34075]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-10T11:10:36.105 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:35 smithi100 ceph-mon[30296]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-10T11:10:36.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:35 smithi178 ceph-mon[33359]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-10T11:10:37.071 INFO:teuthology.orchestra.run.smithi188.stdout:Created osd(s) 5 on host 'smithi188' 2024-05-10T11:10:37.071 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:10:37.070+0000 7fdea8ff9700 1 -- 172.21.15.188:0/1912588887 <== mgr.14152 v2:172.21.15.100:6800/2507287356 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fdeac0fe0e0 con 0x7fde980b2900 2024-05-10T11:10:37.073 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:10:37.073+0000 7fdeb1e58700 1 -- 172.21.15.188:0/1912588887 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fde980b2900 msgr2=0x7fde980b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:10:37.073 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:10:37.073+0000 7fdeb1e58700 1 --2- 172.21.15.188:0/1912588887 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fde980b2900 0x7fde980b4dc0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fde940036f0 tx=0x7fde94005fd0 comp rx=0 tx=0).stop 2024-05-10T11:10:37.073 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:10:37.073+0000 7fdeb1e58700 1 -- 172.21.15.188:0/1912588887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdeac102c30 msgr2=0x7fdeac199600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:10:37.073 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:10:37.073+0000 7fdeb1e58700 1 --2- 172.21.15.188:0/1912588887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdeac102c30 0x7fdeac199600 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fde9c00c920 tx=0x7fde9c00cc30 comp rx=0 tx=0).stop 2024-05-10T11:10:37.073 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:10:37.073+0000 7fdeb1e58700 1 -- 172.21.15.188:0/1912588887 shutdown_connections 2024-05-10T11:10:37.073 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:10:37.073+0000 7fdeb1e58700 1 --2- 172.21.15.188:0/1912588887 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fde980b2900 0x7fde980b4dc0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:10:37.073 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:10:37.073+0000 7fdeb1e58700 1 --2- 172.21.15.188:0/1912588887 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdeac108090 0x7fdeac1a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:10:37.074 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:10:37.073+0000 7fdeb1e58700 1 --2- 172.21.15.188:0/1912588887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdeac102c30 0x7fdeac199600 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:10:37.074 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:10:37.073+0000 7fdeb1e58700 1 --2- 172.21.15.188:0/1912588887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdeac1055f0 0x7fdeac199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:10:37.074 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:10:37.073+0000 7fdeb1e58700 1 -- 172.21.15.188:0/1912588887 >> 172.21.15.188:0/1912588887 conn(0x7fdeac0fccf0 msgr2=0x7fdeac101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:10:37.074 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:10:37.073+0000 7fdeb1e58700 1 -- 172.21.15.188:0/1912588887 shutdown_connections 2024-05-10T11:10:37.074 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:10:37.073+0000 7fdeb1e58700 1 -- 172.21.15.188:0/1912588887 wait complete. 2024-05-10T11:10:37.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:37 smithi188 ceph-mon[34075]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-10T11:10:37.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:37 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:10:37.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:37 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:10:37.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:37 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:10:37.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:37 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:10:37.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:37 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:10:37.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:37 smithi188 ceph-mon[34075]: from='osd.5 [v2:172.21.15.188:6808/3517375935,v1:172.21.15.188:6809/3517375935]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-10T11:10:37.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:37 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:10:37.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:37 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:10:37.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:37 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:10:37.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:37 smithi178 ceph-mon[33359]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-10T11:10:37.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:37 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:10:37.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:37 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:10:37.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:37 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:10:37.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:37 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:10:37.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:37 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:10:37.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:37 smithi178 ceph-mon[33359]: from='osd.5 [v2:172.21.15.188:6808/3517375935,v1:172.21.15.188:6809/3517375935]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-10T11:10:37.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:37 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:10:37.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:37 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:10:37.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:37 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:10:37.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:37 smithi100 ceph-mon[30296]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-10T11:10:37.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:37 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:10:37.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:37 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:10:37.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:37 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:10:37.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:37 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:10:37.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:37 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:10:37.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:37 smithi100 ceph-mon[30296]: from='osd.5 [v2:172.21.15.188:6808/3517375935,v1:172.21.15.188:6809/3517375935]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-10T11:10:37.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:37 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:10:37.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:37 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:10:37.539 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:37 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:10:37.736 DEBUG:teuthology.orchestra.run.smithi188:osd.5> sudo journalctl -f -n 0 -u ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@osd.5.service 2024-05-10T11:10:37.739 INFO:tasks.cephadm:Deploying osd.6 on smithi100 with /dev/vg_nvme/lv_2... 2024-05-10T11:10:37.739 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-10T11:10:37.977 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:10:39.153 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:38 smithi100 ceph-mon[30296]: from='osd.5 [v2:172.21.15.188:6808/3517375935,v1:172.21.15.188:6809/3517375935]' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-10T11:10:39.153 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:38 smithi100 ceph-mon[30296]: osdmap e35: 6 total, 5 up, 6 in 2024-05-10T11:10:39.153 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:38 smithi100 ceph-mon[30296]: from='osd.5 [v2:172.21.15.188:6808/3517375935,v1:172.21.15.188:6809/3517375935]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-05-10T11:10:39.153 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:38 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T11:10:39.198 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:38 smithi188 ceph-mon[34075]: from='osd.5 [v2:172.21.15.188:6808/3517375935,v1:172.21.15.188:6809/3517375935]' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-10T11:10:39.198 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:38 smithi188 ceph-mon[34075]: osdmap e35: 6 total, 5 up, 6 in 2024-05-10T11:10:39.198 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:38 smithi188 ceph-mon[34075]: from='osd.5 [v2:172.21.15.188:6808/3517375935,v1:172.21.15.188:6809/3517375935]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-05-10T11:10:39.199 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:38 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T11:10:39.199 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 10 11:10:38 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-5[45716]: 2024-05-10T11:10:38.865+0000 7fca3c88d700 -1 osd.5 0 waiting for initial osdmap 2024-05-10T11:10:39.199 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 10 11:10:38 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-5[45716]: 2024-05-10T11:10:38.872+0000 7fca38680700 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T11:10:39.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:38 smithi178 ceph-mon[33359]: from='osd.5 [v2:172.21.15.188:6808/3517375935,v1:172.21.15.188:6809/3517375935]' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-10T11:10:39.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:38 smithi178 ceph-mon[33359]: osdmap e35: 6 total, 5 up, 6 in 2024-05-10T11:10:39.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:38 smithi178 ceph-mon[33359]: from='osd.5 [v2:172.21.15.188:6808/3517375935,v1:172.21.15.188:6809/3517375935]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-05-10T11:10:39.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:38 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T11:10:40.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:39 smithi100 ceph-mon[30296]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-10T11:10:40.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:39 smithi100 ceph-mon[30296]: from='osd.5 [v2:172.21.15.188:6808/3517375935,v1:172.21.15.188:6809/3517375935]' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-05-10T11:10:40.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:39 smithi100 ceph-mon[30296]: osdmap e36: 6 total, 5 up, 6 in 2024-05-10T11:10:40.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:39 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T11:10:40.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:39 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T11:10:40.140 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-10T11:10:40.140 INFO:teuthology.orchestra.run.smithi100.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-10T11:10:40.140 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10+0 records in 2024-05-10T11:10:40.140 INFO:teuthology.orchestra.run.smithi100.stderr:10+0 records out 2024-05-10T11:10:40.140 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113426 s, 924 MB/s 2024-05-10T11:10:40.141 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping successful for: 2024-05-10T11:10:40.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:39 smithi178 ceph-mon[33359]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-10T11:10:40.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:39 smithi178 ceph-mon[33359]: from='osd.5 [v2:172.21.15.188:6808/3517375935,v1:172.21.15.188:6809/3517375935]' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-05-10T11:10:40.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:39 smithi178 ceph-mon[33359]: osdmap e36: 6 total, 5 up, 6 in 2024-05-10T11:10:40.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:39 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T11:10:40.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:39 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T11:10:40.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:39 smithi188 ceph-mon[34075]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-10T11:10:40.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:39 smithi188 ceph-mon[34075]: from='osd.5 [v2:172.21.15.188:6808/3517375935,v1:172.21.15.188:6809/3517375935]' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-05-10T11:10:40.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:39 smithi188 ceph-mon[34075]: osdmap e36: 6 total, 5 up, 6 in 2024-05-10T11:10:40.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:39 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T11:10:40.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:39 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T11:10:40.870 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi100:vg_nvme/lv_2 2024-05-10T11:10:41.106 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:10:41.227 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:40 smithi100 ceph-mon[30296]: purged_snaps scrub starts 2024-05-10T11:10:41.227 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:40 smithi100 ceph-mon[30296]: purged_snaps scrub ok 2024-05-10T11:10:41.228 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:40 smithi100 ceph-mon[30296]: osd.5 [v2:172.21.15.188:6808/3517375935,v1:172.21.15.188:6809/3517375935] boot 2024-05-10T11:10:41.228 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:40 smithi100 ceph-mon[30296]: osdmap e37: 6 total, 6 up, 6 in 2024-05-10T11:10:41.228 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:40 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T11:10:41.228 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:40 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:10:41.228 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:40 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:10:41.228 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:40 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:10:41.228 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:40 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:10:41.228 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:40 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:10:41.228 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:40 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:10:41.228 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:40 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:10:41.228 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:40 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:10:41.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:40 smithi178 ceph-mon[33359]: purged_snaps scrub starts 2024-05-10T11:10:41.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:40 smithi178 ceph-mon[33359]: purged_snaps scrub ok 2024-05-10T11:10:41.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:40 smithi178 ceph-mon[33359]: osd.5 [v2:172.21.15.188:6808/3517375935,v1:172.21.15.188:6809/3517375935] boot 2024-05-10T11:10:41.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:40 smithi178 ceph-mon[33359]: osdmap e37: 6 total, 6 up, 6 in 2024-05-10T11:10:41.254 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:40 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T11:10:41.254 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:40 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:10:41.254 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:40 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:10:41.254 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:40 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:10:41.254 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:40 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:10:41.254 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:40 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:10:41.254 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:40 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:10:41.254 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:40 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:10:41.254 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:40 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:10:41.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:40 smithi188 ceph-mon[34075]: purged_snaps scrub starts 2024-05-10T11:10:41.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:40 smithi188 ceph-mon[34075]: purged_snaps scrub ok 2024-05-10T11:10:41.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:40 smithi188 ceph-mon[34075]: osd.5 [v2:172.21.15.188:6808/3517375935,v1:172.21.15.188:6809/3517375935] boot 2024-05-10T11:10:41.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:40 smithi188 ceph-mon[34075]: osdmap e37: 6 total, 6 up, 6 in 2024-05-10T11:10:41.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:40 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T11:10:41.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:40 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:10:41.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:40 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:10:41.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:40 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:10:41.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:40 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:10:41.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:40 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:10:41.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:40 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:10:41.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:40 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:10:41.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:40 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:10:42.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:41 smithi188 ceph-mon[34075]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-05-10T11:10:42.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:41 smithi188 ceph-mon[34075]: Detected new or changed devices on smithi188 2024-05-10T11:10:42.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:41 smithi188 ceph-mon[34075]: Adjusting osd_memory_target on smithi188 to 8544M 2024-05-10T11:10:42.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:41 smithi188 ceph-mon[34075]: osdmap e38: 6 total, 6 up, 6 in 2024-05-10T11:10:42.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:41 smithi178 ceph-mon[33359]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-05-10T11:10:42.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:41 smithi178 ceph-mon[33359]: Detected new or changed devices on smithi188 2024-05-10T11:10:42.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:41 smithi178 ceph-mon[33359]: Adjusting osd_memory_target on smithi188 to 8544M 2024-05-10T11:10:42.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:41 smithi178 ceph-mon[33359]: osdmap e38: 6 total, 6 up, 6 in 2024-05-10T11:10:42.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:41 smithi100 ceph-mon[30296]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-05-10T11:10:42.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:41 smithi100 ceph-mon[30296]: Detected new or changed devices on smithi188 2024-05-10T11:10:42.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:41 smithi100 ceph-mon[30296]: Adjusting osd_memory_target on smithi188 to 8544M 2024-05-10T11:10:42.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:41 smithi100 ceph-mon[30296]: osdmap e38: 6 total, 6 up, 6 in 2024-05-10T11:10:42.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:10:42.838+0000 7f7817669700 1 -- 172.21.15.100:0/4056304137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78101050c0 msgr2=0x7f781010b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:10:42.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:10:42.838+0000 7f7817669700 1 --2- 172.21.15.100:0/4056304137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78101050c0 0x7f781010b970 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f780800b0d0 tx=0x7f780800b3e0 comp rx=0 tx=0).stop 2024-05-10T11:10:42.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:10:42.838+0000 7f7817669700 1 -- 172.21.15.100:0/4056304137 shutdown_connections 2024-05-10T11:10:42.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:10:42.838+0000 7f7817669700 1 --2- 172.21.15.100:0/4056304137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78101050c0 0x7f781010b970 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:10:42.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:10:42.838+0000 7f7817669700 1 --2- 172.21.15.100:0/4056304137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7810103520 0x7f7810103940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:10:42.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:10:42.838+0000 7f7817669700 1 --2- 172.21.15.100:0/4056304137 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7810104720 0x7f7810104b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:10:42.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:10:42.838+0000 7f7817669700 1 -- 172.21.15.100:0/4056304137 >> 172.21.15.100:0/4056304137 conn(0x7f78100feac0 msgr2=0x7f7810100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:10:42.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:10:42.839+0000 7f7817669700 1 -- 172.21.15.100:0/4056304137 shutdown_connections 2024-05-10T11:10:42.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:10:42.839+0000 7f7817669700 1 -- 172.21.15.100:0/4056304137 wait complete. 2024-05-10T11:10:42.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:10:42.839+0000 7f7817669700 1 Processor -- start 2024-05-10T11:10:42.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:10:42.839+0000 7f7817669700 1 -- start start 2024-05-10T11:10:42.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:10:42.840+0000 7f7817669700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7810103520 0x7f78101995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:10:42.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:10:42.840+0000 7f7817669700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7810104720 0x7f7810199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:10:42.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:10:42.840+0000 7f7817669700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78101050c0 0x7f78101a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:10:42.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:10:42.840+0000 7f7817669700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f781010eec0 con 0x7f7810104720 2024-05-10T11:10:42.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:10:42.840+0000 7f7817669700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f781010ed40 con 0x7f78101050c0 2024-05-10T11:10:42.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:10:42.840+0000 7f7817669700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f781010f040 con 0x7f7810103520 2024-05-10T11:10:42.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:10:42.840+0000 7f7814c04700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7810104720 0x7f7810199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:10:42.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:10:42.840+0000 7f7814c04700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7810104720 0x7f7810199b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49342/0 (socket says 172.21.15.100:49342) 2024-05-10T11:10:42.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:10:42.840+0000 7f7814c04700 1 -- 172.21.15.100:0/674500187 learned_addr learned my addr 172.21.15.100:0/674500187 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:10:42.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:10:42.840+0000 7f7815405700 1 --2- 172.21.15.100:0/674500187 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7810103520 0x7f78101995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:10:42.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:10:42.840+0000 7f7815c06700 1 --2- 172.21.15.100:0/674500187 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78101050c0 0x7f78101a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:10:42.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:10:42.840+0000 7f7814c04700 1 -- 172.21.15.100:0/674500187 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7810103520 msgr2=0x7f78101995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:10:42.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:10:42.840+0000 7f7814c04700 1 --2- 172.21.15.100:0/674500187 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7810103520 0x7f78101995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:10:42.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:10:42.840+0000 7f7814c04700 1 -- 172.21.15.100:0/674500187 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78101050c0 msgr2=0x7f78101a0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:10:42.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:10:42.840+0000 7f7814c04700 1 --2- 172.21.15.100:0/674500187 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78101050c0 0x7f78101a0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:10:42.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:10:42.840+0000 7f7814c04700 1 -- 172.21.15.100:0/674500187 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78101a11d0 con 0x7f7810104720 2024-05-10T11:10:42.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:10:42.840+0000 7f7814c04700 1 --2- 172.21.15.100:0/674500187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7810104720 0x7f7810199b00 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f780c00c950 tx=0x7f780c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:10:42.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:10:42.840+0000 7f78027fc700 1 -- 172.21.15.100:0/674500187 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780c007ac0 con 0x7f7810104720 2024-05-10T11:10:42.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:10:42.840+0000 7f78027fc700 1 -- 172.21.15.100:0/674500187 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f780c004cc0 con 0x7f7810104720 2024-05-10T11:10:42.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:10:42.840+0000 7f78027fc700 1 -- 172.21.15.100:0/674500187 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780c005440 con 0x7f7810104720 2024-05-10T11:10:42.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:10:42.840+0000 7f7817669700 1 -- 172.21.15.100:0/674500187 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78101a14c0 con 0x7f7810104720 2024-05-10T11:10:42.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:10:42.840+0000 7f7817669700 1 -- 172.21.15.100:0/674500187 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78101a1a50 con 0x7f7810104720 2024-05-10T11:10:42.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:10:42.841+0000 7f7817669700 1 -- 172.21.15.100:0/674500187 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f781006a190 con 0x7f7810104720 2024-05-10T11:10:42.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:10:42.843+0000 7f78027fc700 1 -- 172.21.15.100:0/674500187 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f780c004750 con 0x7f7810104720 2024-05-10T11:10:42.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:10:42.843+0000 7f78027fc700 1 --2- 172.21.15.100:0/674500187 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f77fc0b2900 0x7f77fc0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:10:42.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:10:42.843+0000 7f7815405700 1 --2- 172.21.15.100:0/674500187 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f77fc0b2900 0x7f77fc0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:10:42.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:10:42.844+0000 7f78027fc700 1 -- 172.21.15.100:0/674500187 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4533+0+0 (secure 0 0 0) 0x7f780c0e1300 con 0x7f7810104720 2024-05-10T11:10:42.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:10:42.844+0000 7f7815405700 1 --2- 172.21.15.100:0/674500187 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f77fc0b2900 0x7f77fc0b4dc0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7810104580 tx=0x7f780400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:10:42.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:10:42.847+0000 7f78027fc700 1 -- 172.21.15.100:0/674500187 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f780c014030 con 0x7f7810104720 2024-05-10T11:10:43.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:10:43.059+0000 7f7817669700 1 -- 172.21.15.100:0/674500187 --> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f78100638e0 con 0x7f77fc0b2900 2024-05-10T11:10:43.197 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:42 smithi188 ceph-mon[34075]: osdmap e39: 6 total, 6 up, 6 in 2024-05-10T11:10:43.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:42 smithi178 ceph-mon[33359]: osdmap e39: 6 total, 6 up, 6 in 2024-05-10T11:10:43.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:42 smithi100 ceph-mon[30296]: osdmap e39: 6 total, 6 up, 6 in 2024-05-10T11:10:43.935 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:43 smithi100 ceph-mon[30296]: pgmap v140: 1 pgs: 1 peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-05-10T11:10:43.936 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:43 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T11:10:43.936 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:43 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T11:10:43.936 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:43 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:10:44.153 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:43 smithi178 ceph-mon[33359]: pgmap v140: 1 pgs: 1 peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-05-10T11:10:44.154 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:43 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T11:10:44.154 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:43 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T11:10:44.154 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:43 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:10:44.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:43 smithi188 ceph-mon[34075]: pgmap v140: 1 pgs: 1 peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-05-10T11:10:44.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:43 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T11:10:44.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:43 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T11:10:44.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:43 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:10:45.017 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:44 smithi100 ceph-mon[30296]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:10:45.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:44 smithi178 ceph-mon[33359]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:10:45.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:44 smithi188 ceph-mon[34075]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:10:46.250 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:45 smithi100 ceph-mon[30296]: pgmap v141: 1 pgs: 1 peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-05-10T11:10:46.250 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:45 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/3741771191' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "84b57387-0a94-4e49-b099-493d329b87dd"}]: dispatch 2024-05-10T11:10:46.250 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:45 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/3741771191' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "84b57387-0a94-4e49-b099-493d329b87dd"}]': finished 2024-05-10T11:10:46.250 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:45 smithi100 ceph-mon[30296]: osdmap e40: 7 total, 6 up, 7 in 2024-05-10T11:10:46.250 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:45 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T11:10:46.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:45 smithi178 ceph-mon[33359]: pgmap v141: 1 pgs: 1 peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-05-10T11:10:46.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:45 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/3741771191' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "84b57387-0a94-4e49-b099-493d329b87dd"}]: dispatch 2024-05-10T11:10:46.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:45 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/3741771191' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "84b57387-0a94-4e49-b099-493d329b87dd"}]': finished 2024-05-10T11:10:46.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:45 smithi178 ceph-mon[33359]: osdmap e40: 7 total, 6 up, 7 in 2024-05-10T11:10:46.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:45 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T11:10:46.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:45 smithi188 ceph-mon[34075]: pgmap v141: 1 pgs: 1 peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-05-10T11:10:46.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:45 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/3741771191' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "84b57387-0a94-4e49-b099-493d329b87dd"}]: dispatch 2024-05-10T11:10:46.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:45 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/3741771191' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "84b57387-0a94-4e49-b099-493d329b87dd"}]': finished 2024-05-10T11:10:46.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:45 smithi188 ceph-mon[34075]: osdmap e40: 7 total, 6 up, 7 in 2024-05-10T11:10:46.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:45 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T11:10:47.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:46 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/2947628242' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T11:10:47.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:46 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/2947628242' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T11:10:47.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:46 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/2947628242' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T11:10:48.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:47 smithi178 ceph-mon[33359]: pgmap v143: 1 pgs: 1 peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-05-10T11:10:48.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:47 smithi188 ceph-mon[34075]: pgmap v143: 1 pgs: 1 peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-05-10T11:10:48.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:47 smithi100 ceph-mon[30296]: pgmap v143: 1 pgs: 1 peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-05-10T11:10:50.224 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:49 smithi100 ceph-mon[30296]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-05-10T11:10:50.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:49 smithi178 ceph-mon[33359]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-05-10T11:10:50.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:49 smithi188 ceph-mon[34075]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-05-10T11:10:52.251 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:51 smithi100 ceph-mon[30296]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-05-10T11:10:52.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:51 smithi178 ceph-mon[33359]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-05-10T11:10:52.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:51 smithi188 ceph-mon[34075]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-05-10T11:10:54.180 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:53 smithi100 ceph-mon[30296]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-10T11:10:54.181 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:53 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-10T11:10:54.181 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:53 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:10:54.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:53 smithi178 ceph-mon[33359]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-10T11:10:54.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:53 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-10T11:10:54.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:53 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:10:54.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:53 smithi188 ceph-mon[34075]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-10T11:10:54.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:53 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-10T11:10:54.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:53 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:10:55.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:54 smithi100 ceph-mon[30296]: Deploying daemon osd.6 on smithi100 2024-05-10T11:10:55.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:54 smithi178 ceph-mon[33359]: Deploying daemon osd.6 on smithi100 2024-05-10T11:10:55.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:54 smithi188 ceph-mon[34075]: Deploying daemon osd.6 on smithi100 2024-05-10T11:10:56.024 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:55 smithi100 ceph-mon[30296]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-10T11:10:56.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:55 smithi178 ceph-mon[33359]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-10T11:10:56.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:55 smithi188 ceph-mon[34075]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-10T11:10:58.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:57 smithi100 ceph-mon[30296]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-05-10T11:10:58.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:57 smithi178 ceph-mon[33359]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-05-10T11:10:58.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:57 smithi188 ceph-mon[34075]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-05-10T11:11:00.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:59 smithi178 ceph-mon[33359]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-10T11:11:00.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:59 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:00.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:59 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:00.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:10:59 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:11:00.266 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:59 smithi100 ceph-mon[30296]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-10T11:11:00.266 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:59 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:00.266 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:59 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:00.266 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:10:59 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:11:00.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:59 smithi188 ceph-mon[34075]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-10T11:11:00.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:59 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:00.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:59 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:00.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:10:59 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:11:02.230 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:01 smithi100 ceph-mon[30296]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-05-10T11:11:02.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:01 smithi178 ceph-mon[33359]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-05-10T11:11:02.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:01 smithi188 ceph-mon[34075]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-05-10T11:11:03.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:03 smithi100 ceph-mon[30296]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-05-10T11:11:03.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:03 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:03.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:03 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:03.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:03 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:11:03.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:03 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:11:03.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:03 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:03.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:03 smithi100 ceph-mon[30296]: from='osd.6 [v2:172.21.15.100:6818/1410593862,v1:172.21.15.100:6819/1410593862]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-10T11:11:04.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:03 smithi178 ceph-mon[33359]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-05-10T11:11:04.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:03 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:04.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:03 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:04.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:03 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:11:04.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:03 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:11:04.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:03 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:04.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:03 smithi178 ceph-mon[33359]: from='osd.6 [v2:172.21.15.100:6818/1410593862,v1:172.21.15.100:6819/1410593862]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-10T11:11:04.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:03 smithi188 ceph-mon[34075]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-05-10T11:11:04.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:03 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:04.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:03 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:04.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:03 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:11:04.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:03 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:11:04.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:03 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:04.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:03 smithi188 ceph-mon[34075]: from='osd.6 [v2:172.21.15.100:6818/1410593862,v1:172.21.15.100:6819/1410593862]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-10T11:11:04.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:11:04.067+0000 7f78027fc700 1 -- 172.21.15.100:0/674500187 <== mgr.14152 v2:172.21.15.100:6800/2507287356 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f78100638e0 con 0x7f77fc0b2900 2024-05-10T11:11:04.068 INFO:teuthology.orchestra.run.smithi100.stdout:Created osd(s) 6 on host 'smithi100' 2024-05-10T11:11:04.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:11:04.070+0000 7f7817669700 1 -- 172.21.15.100:0/674500187 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f77fc0b2900 msgr2=0x7f77fc0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:11:04.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:11:04.070+0000 7f7817669700 1 --2- 172.21.15.100:0/674500187 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f77fc0b2900 0x7f77fc0b4dc0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7810104580 tx=0x7f780400b410 comp rx=0 tx=0).stop 2024-05-10T11:11:04.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:11:04.070+0000 7f7817669700 1 -- 172.21.15.100:0/674500187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7810104720 msgr2=0x7f7810199b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:11:04.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:11:04.070+0000 7f7817669700 1 --2- 172.21.15.100:0/674500187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7810104720 0x7f7810199b00 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f780c00c950 tx=0x7f780c00cc60 comp rx=0 tx=0).stop 2024-05-10T11:11:04.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:11:04.070+0000 7f7817669700 1 -- 172.21.15.100:0/674500187 shutdown_connections 2024-05-10T11:11:04.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:11:04.070+0000 7f7817669700 1 --2- 172.21.15.100:0/674500187 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f77fc0b2900 0x7f77fc0b4dc0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:11:04.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:11:04.070+0000 7f7817669700 1 --2- 172.21.15.100:0/674500187 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78101050c0 0x7f78101a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:11:04.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:11:04.070+0000 7f7817669700 1 --2- 172.21.15.100:0/674500187 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7810103520 0x7f78101995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:11:04.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:11:04.070+0000 7f7817669700 1 --2- 172.21.15.100:0/674500187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7810104720 0x7f7810199b00 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:11:04.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:11:04.070+0000 7f7817669700 1 -- 172.21.15.100:0/674500187 >> 172.21.15.100:0/674500187 conn(0x7f78100feac0 msgr2=0x7f7810107a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:11:04.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:11:04.070+0000 7f7817669700 1 -- 172.21.15.100:0/674500187 shutdown_connections 2024-05-10T11:11:04.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:11:04.070+0000 7f7817669700 1 -- 172.21.15.100:0/674500187 wait complete. 2024-05-10T11:11:04.776 DEBUG:teuthology.orchestra.run.smithi100:osd.6> sudo journalctl -f -n 0 -u ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@osd.6.service 2024-05-10T11:11:04.778 INFO:tasks.cephadm:Deploying osd.7 on smithi178 with /dev/vg_nvme/lv_2... 2024-05-10T11:11:04.778 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-10T11:11:05.027 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:04 smithi100 ceph-mon[30296]: from='osd.6 [v2:172.21.15.100:6818/1410593862,v1:172.21.15.100:6819/1410593862]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-10T11:11:05.027 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:04 smithi100 ceph-mon[30296]: osdmap e41: 7 total, 6 up, 7 in 2024-05-10T11:11:05.027 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:04 smithi100 ceph-mon[30296]: from='osd.6 [v2:172.21.15.100:6818/1410593862,v1:172.21.15.100:6819/1410593862]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-05-10T11:11:05.027 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:04 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T11:11:05.028 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:04 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:05.028 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:04 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:05.028 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:04 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:11:05.028 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:11:04 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6[56225]: 2024-05-10T11:11:04.934+0000 7ff3da5da700 -1 osd.6 0 waiting for initial osdmap 2024-05-10T11:11:05.028 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:11:04 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6[56225]: 2024-05-10T11:11:04.944+0000 7ff3d63cd700 -1 osd.6 42 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T11:11:05.030 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T11:11:05.152 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:04 smithi178 ceph-mon[33359]: from='osd.6 [v2:172.21.15.100:6818/1410593862,v1:172.21.15.100:6819/1410593862]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-10T11:11:05.153 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:04 smithi178 ceph-mon[33359]: osdmap e41: 7 total, 6 up, 7 in 2024-05-10T11:11:05.153 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:04 smithi178 ceph-mon[33359]: from='osd.6 [v2:172.21.15.100:6818/1410593862,v1:172.21.15.100:6819/1410593862]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-05-10T11:11:05.153 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:04 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T11:11:05.153 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:04 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:05.153 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:04 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:05.153 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:04 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:11:05.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:04 smithi188 ceph-mon[34075]: from='osd.6 [v2:172.21.15.100:6818/1410593862,v1:172.21.15.100:6819/1410593862]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-10T11:11:05.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:04 smithi188 ceph-mon[34075]: osdmap e41: 7 total, 6 up, 7 in 2024-05-10T11:11:05.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:04 smithi188 ceph-mon[34075]: from='osd.6 [v2:172.21.15.100:6818/1410593862,v1:172.21.15.100:6819/1410593862]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-05-10T11:11:05.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:04 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T11:11:05.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:04 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:05.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:04 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:05.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:04 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:11:06.034 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:05 smithi100 ceph-mon[30296]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-05-10T11:11:06.034 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:05 smithi100 ceph-mon[30296]: from='osd.6 [v2:172.21.15.100:6818/1410593862,v1:172.21.15.100:6819/1410593862]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-05-10T11:11:06.034 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:05 smithi100 ceph-mon[30296]: osdmap e42: 7 total, 6 up, 7 in 2024-05-10T11:11:06.034 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:05 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T11:11:06.034 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:05 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T11:11:06.185 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:05 smithi178 ceph-mon[33359]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-05-10T11:11:06.185 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:05 smithi178 ceph-mon[33359]: from='osd.6 [v2:172.21.15.100:6818/1410593862,v1:172.21.15.100:6819/1410593862]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-05-10T11:11:06.185 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:05 smithi178 ceph-mon[33359]: osdmap e42: 7 total, 6 up, 7 in 2024-05-10T11:11:06.185 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:05 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T11:11:06.185 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:05 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T11:11:06.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:05 smithi188 ceph-mon[34075]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-05-10T11:11:06.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:05 smithi188 ceph-mon[34075]: from='osd.6 [v2:172.21.15.100:6818/1410593862,v1:172.21.15.100:6819/1410593862]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-05-10T11:11:06.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:05 smithi188 ceph-mon[34075]: osdmap e42: 7 total, 6 up, 7 in 2024-05-10T11:11:06.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:05 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T11:11:06.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:05 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T11:11:07.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:06 smithi178 ceph-mon[33359]: purged_snaps scrub starts 2024-05-10T11:11:07.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:06 smithi178 ceph-mon[33359]: purged_snaps scrub ok 2024-05-10T11:11:07.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:06 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T11:11:07.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:06 smithi178 ceph-mon[33359]: osd.6 [v2:172.21.15.100:6818/1410593862,v1:172.21.15.100:6819/1410593862] boot 2024-05-10T11:11:07.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:06 smithi178 ceph-mon[33359]: osdmap e43: 7 total, 7 up, 7 in 2024-05-10T11:11:07.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:06 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T11:11:07.202 INFO:teuthology.orchestra.run.smithi178.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-10T11:11:07.202 INFO:teuthology.orchestra.run.smithi178.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-10T11:11:07.202 INFO:teuthology.orchestra.run.smithi178.stderr: stderr: 10+0 records in 2024-05-10T11:11:07.202 INFO:teuthology.orchestra.run.smithi178.stderr:10+0 records out 2024-05-10T11:11:07.202 INFO:teuthology.orchestra.run.smithi178.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00818541 s, 1.3 GB/s 2024-05-10T11:11:07.202 INFO:teuthology.orchestra.run.smithi178.stderr: stderr: 2024-05-10T11:11:07.203 INFO:teuthology.orchestra.run.smithi178.stderr:--> Zapping successful for: 2024-05-10T11:11:07.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:06 smithi188 ceph-mon[34075]: purged_snaps scrub starts 2024-05-10T11:11:07.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:06 smithi188 ceph-mon[34075]: purged_snaps scrub ok 2024-05-10T11:11:07.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:06 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T11:11:07.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:06 smithi188 ceph-mon[34075]: osd.6 [v2:172.21.15.100:6818/1410593862,v1:172.21.15.100:6819/1410593862] boot 2024-05-10T11:11:07.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:06 smithi188 ceph-mon[34075]: osdmap e43: 7 total, 7 up, 7 in 2024-05-10T11:11:07.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:06 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T11:11:07.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:06 smithi100 ceph-mon[30296]: purged_snaps scrub starts 2024-05-10T11:11:07.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:06 smithi100 ceph-mon[30296]: purged_snaps scrub ok 2024-05-10T11:11:07.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:06 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T11:11:07.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:06 smithi100 ceph-mon[30296]: osd.6 [v2:172.21.15.100:6818/1410593862,v1:172.21.15.100:6819/1410593862] boot 2024-05-10T11:11:07.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:06 smithi100 ceph-mon[30296]: osdmap e43: 7 total, 7 up, 7 in 2024-05-10T11:11:07.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:06 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T11:11:07.876 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi178:vg_nvme/lv_2 2024-05-10T11:11:08.121 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T11:11:08.243 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:07 smithi178 ceph-mon[33359]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-10T11:11:08.243 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:07 smithi178 ceph-mon[33359]: osdmap e44: 7 total, 7 up, 7 in 2024-05-10T11:11:08.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:07 smithi188 ceph-mon[34075]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-10T11:11:08.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:07 smithi188 ceph-mon[34075]: osdmap e44: 7 total, 7 up, 7 in 2024-05-10T11:11:08.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:07 smithi100 ceph-mon[30296]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-10T11:11:08.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:07 smithi100 ceph-mon[30296]: osdmap e44: 7 total, 7 up, 7 in 2024-05-10T11:11:09.230 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:09 smithi178 ceph-mon[33359]: Detected new or changed devices on smithi100 2024-05-10T11:11:09.231 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:09 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:09.231 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:09 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:09.231 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:09 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:11:09.231 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:09 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:11:09.231 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:09 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:11:09.231 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:09 smithi178 ceph-mon[33359]: Adjusting osd_memory_target on smithi100 to 5696M 2024-05-10T11:11:09.231 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:09 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:09.231 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:09 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:11:09.231 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:09 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:11:09.231 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:09 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:09.231 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:09 smithi178 ceph-mon[33359]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 213 MiB used, 626 GiB / 626 GiB avail 2024-05-10T11:11:09.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:09 smithi188 ceph-mon[34075]: Detected new or changed devices on smithi100 2024-05-10T11:11:09.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:09 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:09.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:09 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:09.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:09 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:11:09.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:09 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:11:09.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:09 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:11:09.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:09 smithi188 ceph-mon[34075]: Adjusting osd_memory_target on smithi100 to 5696M 2024-05-10T11:11:09.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:09 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:09.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:09 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:11:09.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:09 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:11:09.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:09 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:09.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:09 smithi188 ceph-mon[34075]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 213 MiB used, 626 GiB / 626 GiB avail 2024-05-10T11:11:09.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:09 smithi100 ceph-mon[30296]: Detected new or changed devices on smithi100 2024-05-10T11:11:09.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:09 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:09.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:09 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:09.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:09 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:11:09.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:09 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:11:09.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:09 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:11:09.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:09 smithi100 ceph-mon[30296]: Adjusting osd_memory_target on smithi100 to 5696M 2024-05-10T11:11:09.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:09 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:09.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:09 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:11:09.539 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:09 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:11:09.539 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:09 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:09.539 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:09 smithi100 ceph-mon[30296]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 213 MiB used, 626 GiB / 626 GiB avail 2024-05-10T11:11:09.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:11:09.765+0000 7f080fb91700 1 -- 172.21.15.178:0/3881417107 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0808102c40 msgr2=0x7f0808105030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:11:09.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:11:09.765+0000 7f080fb91700 1 --2- 172.21.15.178:0/3881417107 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0808102c40 0x7f0808105030 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f07fc009990 tx=0x7f07fc009ca0 comp rx=0 tx=0).stop 2024-05-10T11:11:09.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:11:09.765+0000 7f080fb91700 1 -- 172.21.15.178:0/3881417107 shutdown_connections 2024-05-10T11:11:09.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:11:09.765+0000 7f080fb91700 1 --2- 172.21.15.178:0/3881417107 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0808108090 0x7f080810a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:11:09.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:11:09.765+0000 7f080fb91700 1 --2- 172.21.15.178:0/3881417107 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0808102c40 0x7f0808105030 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:11:09.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:11:09.765+0000 7f080fb91700 1 --2- 172.21.15.178:0/3881417107 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0808105570 0x7f0808107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:11:09.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:11:09.765+0000 7f080fb91700 1 -- 172.21.15.178:0/3881417107 >> 172.21.15.178:0/3881417107 conn(0x7f08080fccf0 msgr2=0x7f08080ff110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:11:09.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:11:09.765+0000 7f080fb91700 1 -- 172.21.15.178:0/3881417107 shutdown_connections 2024-05-10T11:11:09.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:11:09.765+0000 7f080fb91700 1 -- 172.21.15.178:0/3881417107 wait complete. 2024-05-10T11:11:09.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:11:09.766+0000 7f080fb91700 1 Processor -- start 2024-05-10T11:11:09.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:11:09.766+0000 7f080fb91700 1 -- start start 2024-05-10T11:11:09.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:11:09.766+0000 7f080fb91700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0808102c40 0x7f0808198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:11:09.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:11:09.766+0000 7f080fb91700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0808105570 0x7f08081989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:11:09.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:11:09.766+0000 7f080fb91700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0808108090 0x7f080819fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:11:09.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:11:09.766+0000 7f080fb91700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f080810dd60 con 0x7f0808108090 2024-05-10T11:11:09.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:11:09.766+0000 7f080fb91700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f080810dbe0 con 0x7f0808105570 2024-05-10T11:11:09.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:11:09.766+0000 7f080fb91700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f080810dee0 con 0x7f0808102c40 2024-05-10T11:11:09.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:11:09.766+0000 7f080d12c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0808105570 0x7f08081989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:11:09.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:11:09.766+0000 7f080d12c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0808105570 0x7f08081989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:42032/0 (socket says 172.21.15.178:42032) 2024-05-10T11:11:09.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:11:09.766+0000 7f080d12c700 1 -- 172.21.15.178:0/1424224442 learned_addr learned my addr 172.21.15.178:0/1424224442 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-05-10T11:11:09.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:11:09.767+0000 7f080e12e700 1 --2- 172.21.15.178:0/1424224442 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0808108090 0x7f080819fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:11:09.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:11:09.767+0000 7f080d92d700 1 --2- 172.21.15.178:0/1424224442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0808102c40 0x7f0808198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:11:09.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:11:09.767+0000 7f080d12c700 1 -- 172.21.15.178:0/1424224442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0808102c40 msgr2=0x7f0808198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:11:09.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:11:09.767+0000 7f080d12c700 1 --2- 172.21.15.178:0/1424224442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0808102c40 0x7f0808198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:11:09.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:11:09.767+0000 7f080d12c700 1 -- 172.21.15.178:0/1424224442 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0808108090 msgr2=0x7f080819fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:11:09.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:11:09.767+0000 7f080d12c700 1 --2- 172.21.15.178:0/1424224442 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0808108090 0x7f080819fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:11:09.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:11:09.767+0000 7f080d12c700 1 -- 172.21.15.178:0/1424224442 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08081a00c0 con 0x7f0808105570 2024-05-10T11:11:09.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:11:09.767+0000 7f080d92d700 1 --2- 172.21.15.178:0/1424224442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0808102c40 0x7f0808198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:11:09.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:11:09.767+0000 7f080d12c700 1 --2- 172.21.15.178:0/1424224442 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0808105570 0x7f08081989a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f080400d980 tx=0x7f080400dd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:11:09.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:11:09.767+0000 7f07faffd700 1 -- 172.21.15.178:0/1424224442 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0804017070 con 0x7f0808105570 2024-05-10T11:11:09.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:11:09.767+0000 7f07faffd700 1 -- 172.21.15.178:0/1424224442 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0804004830 con 0x7f0808105570 2024-05-10T11:11:09.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:11:09.767+0000 7f080fb91700 1 -- 172.21.15.178:0/1424224442 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08081a03b0 con 0x7f0808105570 2024-05-10T11:11:09.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:11:09.767+0000 7f07faffd700 1 -- 172.21.15.178:0/1424224442 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08040053a0 con 0x7f0808105570 2024-05-10T11:11:09.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:11:09.767+0000 7f080fb91700 1 -- 172.21.15.178:0/1424224442 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08081a08f0 con 0x7f0808105570 2024-05-10T11:11:09.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:11:09.768+0000 7f080fb91700 1 -- 172.21.15.178:0/1424224442 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f080806a190 con 0x7f0808105570 2024-05-10T11:11:09.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:11:09.770+0000 7f07faffd700 1 -- 172.21.15.178:0/1424224442 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f08040049e0 con 0x7f0808105570 2024-05-10T11:11:09.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:11:09.771+0000 7f07faffd700 1 --2- 172.21.15.178:0/1424224442 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f07f40b2900 0x7f07f40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:11:09.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:11:09.771+0000 7f07faffd700 1 -- 172.21.15.178:0/1424224442 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 4965+0+0 (secure 0 0 0) 0x7f08040e1820 con 0x7f0808105570 2024-05-10T11:11:09.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:11:09.771+0000 7f080d92d700 1 --2- 172.21.15.178:0/1424224442 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f07f40b2900 0x7f07f40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:11:09.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:11:09.772+0000 7f080d92d700 1 --2- 172.21.15.178:0/1424224442 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f07f40b2900 0x7f07f40b4dc0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f07fc003820 tx=0x7f07fc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:11:09.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:11:09.775+0000 7f07faffd700 1 -- 172.21.15.178:0/1424224442 <== mon.2 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f0804020020 con 0x7f0808105570 2024-05-10T11:11:09.988 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:11:09.988+0000 7f080fb91700 1 -- 172.21.15.178:0/1424224442 --> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f08080639a0 con 0x7f07f40b2900 2024-05-10T11:11:10.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:10 smithi178 ceph-mon[33359]: from='client.24257 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:11:10.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:10 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T11:11:10.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:10 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T11:11:10.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:10 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:11:10.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:10 smithi188 ceph-mon[34075]: from='client.24257 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:11:10.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:10 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T11:11:10.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:10 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T11:11:10.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:10 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:11:10.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:10 smithi100 ceph-mon[30296]: from='client.24257 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:11:10.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:10 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T11:11:10.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:10 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T11:11:10.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:10 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:11:11.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:11 smithi178 ceph-mon[33359]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 213 MiB used, 626 GiB / 626 GiB avail 2024-05-10T11:11:11.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:11 smithi188 ceph-mon[34075]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 213 MiB used, 626 GiB / 626 GiB avail 2024-05-10T11:11:11.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:11 smithi100 ceph-mon[30296]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 213 MiB used, 626 GiB / 626 GiB avail 2024-05-10T11:11:13.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:13 smithi178 ceph-mon[33359]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-10T11:11:13.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:13 smithi178 ceph-mon[33359]: from='client.? 172.21.15.178:0/1291291494' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "02ada28f-ac4f-47cf-983f-8a6da1140abe"}]: dispatch 2024-05-10T11:11:13.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:13 smithi178 ceph-mon[33359]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "02ada28f-ac4f-47cf-983f-8a6da1140abe"}]: dispatch 2024-05-10T11:11:13.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:13 smithi178 ceph-mon[33359]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "02ada28f-ac4f-47cf-983f-8a6da1140abe"}]': finished 2024-05-10T11:11:13.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:13 smithi178 ceph-mon[33359]: osdmap e45: 8 total, 7 up, 8 in 2024-05-10T11:11:13.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:13 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T11:11:13.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:13 smithi188 ceph-mon[34075]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-10T11:11:13.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:13 smithi188 ceph-mon[34075]: from='client.? 172.21.15.178:0/1291291494' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "02ada28f-ac4f-47cf-983f-8a6da1140abe"}]: dispatch 2024-05-10T11:11:13.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:13 smithi188 ceph-mon[34075]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "02ada28f-ac4f-47cf-983f-8a6da1140abe"}]: dispatch 2024-05-10T11:11:13.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:13 smithi188 ceph-mon[34075]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "02ada28f-ac4f-47cf-983f-8a6da1140abe"}]': finished 2024-05-10T11:11:13.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:13 smithi188 ceph-mon[34075]: osdmap e45: 8 total, 7 up, 8 in 2024-05-10T11:11:13.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:13 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T11:11:13.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:13 smithi100 ceph-mon[30296]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-10T11:11:13.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:13 smithi100 ceph-mon[30296]: from='client.? 172.21.15.178:0/1291291494' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "02ada28f-ac4f-47cf-983f-8a6da1140abe"}]: dispatch 2024-05-10T11:11:13.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:13 smithi100 ceph-mon[30296]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "02ada28f-ac4f-47cf-983f-8a6da1140abe"}]: dispatch 2024-05-10T11:11:13.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:13 smithi100 ceph-mon[30296]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "02ada28f-ac4f-47cf-983f-8a6da1140abe"}]': finished 2024-05-10T11:11:13.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:13 smithi100 ceph-mon[30296]: osdmap e45: 8 total, 7 up, 8 in 2024-05-10T11:11:13.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:13 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T11:11:14.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:14 smithi178 ceph-mon[33359]: from='client.? 172.21.15.178:0/1478323292' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T11:11:14.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:14 smithi188 ceph-mon[34075]: from='client.? 172.21.15.178:0/1478323292' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T11:11:14.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:14 smithi100 ceph-mon[30296]: from='client.? 172.21.15.178:0/1478323292' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T11:11:15.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:15 smithi178 ceph-mon[33359]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-10T11:11:15.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:15 smithi188 ceph-mon[34075]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-10T11:11:15.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:15 smithi100 ceph-mon[30296]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-10T11:11:17.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:17 smithi178 ceph-mon[33359]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-10T11:11:17.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:17 smithi188 ceph-mon[34075]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-10T11:11:17.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:17 smithi100 ceph-mon[30296]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-10T11:11:19.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:19 smithi178 ceph-mon[33359]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-10T11:11:19.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:19 smithi188 ceph-mon[34075]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-10T11:11:19.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:19 smithi100 ceph-mon[30296]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-10T11:11:20.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:20 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-10T11:11:20.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:20 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:11:20.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:20 smithi178 ceph-mon[33359]: Deploying daemon osd.7 on smithi178 2024-05-10T11:11:20.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:20 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-10T11:11:20.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:20 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:11:20.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:20 smithi188 ceph-mon[34075]: Deploying daemon osd.7 on smithi178 2024-05-10T11:11:20.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:20 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-10T11:11:20.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:20 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:11:20.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:20 smithi100 ceph-mon[30296]: Deploying daemon osd.7 on smithi178 2024-05-10T11:11:21.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:21 smithi178 ceph-mon[33359]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-10T11:11:21.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:21 smithi188 ceph-mon[34075]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-10T11:11:21.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:21 smithi100 ceph-mon[30296]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-10T11:11:23.440 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:23 smithi178 ceph-mon[33359]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-10T11:11:23.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:23 smithi188 ceph-mon[34075]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-10T11:11:23.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:23 smithi100 ceph-mon[30296]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-10T11:11:25.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:25 smithi188 ceph-mon[34075]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-10T11:11:25.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:25 smithi100 ceph-mon[30296]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-10T11:11:25.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:25 smithi178 ceph-mon[33359]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-10T11:11:26.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:26 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:26.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:26 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:26.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:26 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:11:26.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:26 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:26.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:26 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:26.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:26 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:11:26.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:26 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:26.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:26 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:26.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:26 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:11:27.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:27 smithi178 ceph-mon[33359]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-10T11:11:27.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:27 smithi188 ceph-mon[34075]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-10T11:11:27.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:27 smithi100 ceph-mon[30296]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-10T11:11:29.152 INFO:teuthology.orchestra.run.smithi178.stdout:Created osd(s) 7 on host 'smithi178' 2024-05-10T11:11:29.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:11:29.152+0000 7f07faffd700 1 -- 172.21.15.178:0/1424224442 <== mgr.14152 v2:172.21.15.100:6800/2507287356 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f08080639a0 con 0x7f07f40b2900 2024-05-10T11:11:29.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:11:29.154+0000 7f080fb91700 1 -- 172.21.15.178:0/1424224442 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f07f40b2900 msgr2=0x7f07f40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:11:29.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:11:29.154+0000 7f080fb91700 1 --2- 172.21.15.178:0/1424224442 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f07f40b2900 0x7f07f40b4dc0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f07fc003820 tx=0x7f07fc005fb0 comp rx=0 tx=0).stop 2024-05-10T11:11:29.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:11:29.155+0000 7f080fb91700 1 -- 172.21.15.178:0/1424224442 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0808105570 msgr2=0x7f08081989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:11:29.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:11:29.155+0000 7f080fb91700 1 --2- 172.21.15.178:0/1424224442 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0808105570 0x7f08081989a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f080400d980 tx=0x7f080400dd40 comp rx=0 tx=0).stop 2024-05-10T11:11:29.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:11:29.155+0000 7f080fb91700 1 -- 172.21.15.178:0/1424224442 shutdown_connections 2024-05-10T11:11:29.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:11:29.155+0000 7f080fb91700 1 --2- 172.21.15.178:0/1424224442 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f07f40b2900 0x7f07f40b4dc0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:11:29.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:11:29.155+0000 7f080fb91700 1 --2- 172.21.15.178:0/1424224442 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0808108090 0x7f080819fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:11:29.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:11:29.155+0000 7f080fb91700 1 --2- 172.21.15.178:0/1424224442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0808102c40 0x7f0808198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:11:29.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:11:29.155+0000 7f080fb91700 1 --2- 172.21.15.178:0/1424224442 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0808105570 0x7f08081989a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:11:29.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:11:29.155+0000 7f080fb91700 1 -- 172.21.15.178:0/1424224442 >> 172.21.15.178:0/1424224442 conn(0x7f08080fccf0 msgr2=0x7f0808104b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:11:29.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:11:29.155+0000 7f080fb91700 1 -- 172.21.15.178:0/1424224442 shutdown_connections 2024-05-10T11:11:29.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:11:29.155+0000 7f080fb91700 1 -- 172.21.15.178:0/1424224442 wait complete. 2024-05-10T11:11:29.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:29 smithi178 ceph-mon[33359]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-10T11:11:29.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:29 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:29.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:29 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:29.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:29 smithi178 ceph-mon[33359]: from='osd.7 [v2:172.21.15.178:6816/2458690455,v1:172.21.15.178:6817/2458690455]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-10T11:11:29.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:29 smithi178 ceph-mon[33359]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-10T11:11:29.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:29 smithi188 ceph-mon[34075]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-10T11:11:29.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:29 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:29.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:29 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:29.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:29 smithi188 ceph-mon[34075]: from='osd.7 [v2:172.21.15.178:6816/2458690455,v1:172.21.15.178:6817/2458690455]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-10T11:11:29.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:29 smithi188 ceph-mon[34075]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-10T11:11:29.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:29 smithi100 ceph-mon[30296]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-10T11:11:29.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:29 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:29.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:29 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:29.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:29 smithi100 ceph-mon[30296]: from='osd.7 [v2:172.21.15.178:6816/2458690455,v1:172.21.15.178:6817/2458690455]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-10T11:11:29.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:29 smithi100 ceph-mon[30296]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-10T11:11:29.873 DEBUG:teuthology.orchestra.run.smithi178:osd.7> sudo journalctl -f -n 0 -u ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@osd.7.service 2024-05-10T11:11:29.876 INFO:tasks.cephadm:Deploying osd.8 on smithi188 with /dev/vg_nvme/lv_2... 2024-05-10T11:11:29.876 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-10T11:11:30.126 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T11:11:31.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:30 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:31.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:30 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:31.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:30 smithi178 ceph-mon[33359]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-10T11:11:31.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:30 smithi178 ceph-mon[33359]: from='osd.7 [v2:172.21.15.178:6816/2458690455,v1:172.21.15.178:6817/2458690455]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-05-10T11:11:31.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:30 smithi178 ceph-mon[33359]: osdmap e46: 8 total, 7 up, 8 in 2024-05-10T11:11:31.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:30 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T11:11:31.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:30 smithi178 ceph-mon[33359]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-05-10T11:11:31.253 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:11:31 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7[51714]: 2024-05-10T11:11:31.112+0000 7f2b94bfb700 -1 osd.7 0 waiting for initial osdmap 2024-05-10T11:11:31.253 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:11:31 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7[51714]: 2024-05-10T11:11:31.121+0000 7f2b909ee700 -1 osd.7 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T11:11:31.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:30 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:31.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:30 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:31.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:30 smithi188 ceph-mon[34075]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-10T11:11:31.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:30 smithi188 ceph-mon[34075]: from='osd.7 [v2:172.21.15.178:6816/2458690455,v1:172.21.15.178:6817/2458690455]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-05-10T11:11:31.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:30 smithi188 ceph-mon[34075]: osdmap e46: 8 total, 7 up, 8 in 2024-05-10T11:11:31.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:30 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T11:11:31.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:30 smithi188 ceph-mon[34075]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-05-10T11:11:31.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:30 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:31.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:30 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:31.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:30 smithi100 ceph-mon[30296]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-10T11:11:31.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:30 smithi100 ceph-mon[30296]: from='osd.7 [v2:172.21.15.178:6816/2458690455,v1:172.21.15.178:6817/2458690455]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-05-10T11:11:31.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:30 smithi100 ceph-mon[30296]: osdmap e46: 8 total, 7 up, 8 in 2024-05-10T11:11:31.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:30 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T11:11:31.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:30 smithi100 ceph-mon[30296]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-05-10T11:11:32.182 INFO:teuthology.orchestra.run.smithi188.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-10T11:11:32.183 INFO:teuthology.orchestra.run.smithi188.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-10T11:11:32.183 INFO:teuthology.orchestra.run.smithi188.stderr: stderr: 10+0 records in 2024-05-10T11:11:32.183 INFO:teuthology.orchestra.run.smithi188.stderr:10+0 records out 2024-05-10T11:11:32.183 INFO:teuthology.orchestra.run.smithi188.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.011292 s, 929 MB/s 2024-05-10T11:11:32.183 INFO:teuthology.orchestra.run.smithi188.stderr:--> Zapping successful for: 2024-05-10T11:11:32.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:31 smithi178 ceph-mon[33359]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-10T11:11:32.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:31 smithi178 ceph-mon[33359]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-05-10T11:11:32.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:31 smithi178 ceph-mon[33359]: osdmap e47: 8 total, 7 up, 8 in 2024-05-10T11:11:32.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:31 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T11:11:32.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:31 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T11:11:32.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:31 smithi188 ceph-mon[34075]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-10T11:11:32.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:31 smithi188 ceph-mon[34075]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-05-10T11:11:32.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:31 smithi188 ceph-mon[34075]: osdmap e47: 8 total, 7 up, 8 in 2024-05-10T11:11:32.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:31 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T11:11:32.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:31 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T11:11:32.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:31 smithi100 ceph-mon[30296]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-10T11:11:32.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:31 smithi100 ceph-mon[30296]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-05-10T11:11:32.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:31 smithi100 ceph-mon[30296]: osdmap e47: 8 total, 7 up, 8 in 2024-05-10T11:11:32.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:31 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T11:11:32.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:31 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T11:11:32.848 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi188:vg_nvme/lv_2 2024-05-10T11:11:33.091 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T11:11:33.329 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:33 smithi178 ceph-mon[33359]: purged_snaps scrub starts 2024-05-10T11:11:33.330 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:33 smithi178 ceph-mon[33359]: purged_snaps scrub ok 2024-05-10T11:11:33.330 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:33 smithi178 ceph-mon[33359]: osd.7 [v2:172.21.15.178:6816/2458690455,v1:172.21.15.178:6817/2458690455] boot 2024-05-10T11:11:33.330 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:33 smithi178 ceph-mon[33359]: osdmap e48: 8 total, 8 up, 8 in 2024-05-10T11:11:33.330 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:33 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T11:11:33.330 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:33 smithi178 ceph-mon[33359]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-10T11:11:33.383 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:33 smithi188 ceph-mon[34075]: purged_snaps scrub starts 2024-05-10T11:11:33.383 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:33 smithi188 ceph-mon[34075]: purged_snaps scrub ok 2024-05-10T11:11:33.383 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:33 smithi188 ceph-mon[34075]: osd.7 [v2:172.21.15.178:6816/2458690455,v1:172.21.15.178:6817/2458690455] boot 2024-05-10T11:11:33.383 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:33 smithi188 ceph-mon[34075]: osdmap e48: 8 total, 8 up, 8 in 2024-05-10T11:11:33.383 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:33 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T11:11:33.383 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:33 smithi188 ceph-mon[34075]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-10T11:11:33.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:33 smithi100 ceph-mon[30296]: purged_snaps scrub starts 2024-05-10T11:11:33.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:33 smithi100 ceph-mon[30296]: purged_snaps scrub ok 2024-05-10T11:11:33.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:33 smithi100 ceph-mon[30296]: osd.7 [v2:172.21.15.178:6816/2458690455,v1:172.21.15.178:6817/2458690455] boot 2024-05-10T11:11:33.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:33 smithi100 ceph-mon[30296]: osdmap e48: 8 total, 8 up, 8 in 2024-05-10T11:11:33.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:33 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T11:11:33.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:33 smithi100 ceph-mon[30296]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-10T11:11:34.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:34 smithi188 ceph-mon[34075]: osdmap e49: 8 total, 8 up, 8 in 2024-05-10T11:11:34.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:34 smithi188 ceph-mon[34075]: Detected new or changed devices on smithi178 2024-05-10T11:11:34.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:34 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:34.269 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:34 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:34.269 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:34 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:11:34.269 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:34 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:11:34.269 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:34 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:11:34.269 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:34 smithi188 ceph-mon[34075]: Adjusting osd_memory_target on smithi178 to 5696M 2024-05-10T11:11:34.269 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:34 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:34.269 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:34 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:11:34.269 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:34 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:11:34.269 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:34 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:34.269 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:34 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:11:34.269 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:34 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:11:34.270 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:34 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:11:34.270 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:34 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:34.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:34 smithi178 ceph-mon[33359]: osdmap e49: 8 total, 8 up, 8 in 2024-05-10T11:11:34.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:34 smithi178 ceph-mon[33359]: Detected new or changed devices on smithi178 2024-05-10T11:11:34.504 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:34 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:34.504 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:34 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:34.504 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:34 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:11:34.504 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:34 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:11:34.504 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:34 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:11:34.504 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:34 smithi178 ceph-mon[33359]: Adjusting osd_memory_target on smithi178 to 5696M 2024-05-10T11:11:34.504 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:34 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:34.504 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:34 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:11:34.504 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:34 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:11:34.504 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:34 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:34.504 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:34 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:11:34.504 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:34 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:11:34.504 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:34 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:11:34.504 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:34 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:34.539 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:34 smithi100 ceph-mon[30296]: osdmap e49: 8 total, 8 up, 8 in 2024-05-10T11:11:34.539 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:34 smithi100 ceph-mon[30296]: Detected new or changed devices on smithi178 2024-05-10T11:11:34.539 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:34 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:34.539 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:34 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:34.539 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:34 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:11:34.539 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:34 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:11:34.539 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:34 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:11:34.539 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:34 smithi100 ceph-mon[30296]: Adjusting osd_memory_target on smithi178 to 5696M 2024-05-10T11:11:34.539 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:34 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:34.539 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:34 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:11:34.539 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:34 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:11:34.539 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:34 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:34.539 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:34 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:11:34.540 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:34 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:11:34.540 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:34 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:11:34.540 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:34 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:34.777 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:11:34.776+0000 7f61ad818700 1 -- 172.21.15.188:0/1392803645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61a8103060 msgr2=0x7f61a810a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:11:34.778 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:11:34.776+0000 7f61ad818700 1 --2- 172.21.15.188:0/1392803645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61a8103060 0x7f61a810a920 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f61a000b0d0 tx=0x7f61a000b3e0 comp rx=0 tx=0).stop 2024-05-10T11:11:34.778 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:11:34.776+0000 7f61ad818700 1 -- 172.21.15.188:0/1392803645 shutdown_connections 2024-05-10T11:11:34.778 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:11:34.776+0000 7f61ad818700 1 --2- 172.21.15.188:0/1392803645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61a8103060 0x7f61a810a920 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:11:34.778 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:11:34.776+0000 7f61ad818700 1 --2- 172.21.15.188:0/1392803645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61a8103980 0x7f61a8103da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:11:34.778 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:11:34.776+0000 7f61ad818700 1 --2- 172.21.15.188:0/1392803645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61a81026c0 0x7f61a8102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:11:34.778 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:11:34.776+0000 7f61ad818700 1 -- 172.21.15.188:0/1392803645 >> 172.21.15.188:0/1392803645 conn(0x7f61a80fd130 msgr2=0x7f61a80ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:11:34.778 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:11:34.776+0000 7f61ad818700 1 -- 172.21.15.188:0/1392803645 shutdown_connections 2024-05-10T11:11:34.778 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:11:34.776+0000 7f61ad818700 1 -- 172.21.15.188:0/1392803645 wait complete. 2024-05-10T11:11:34.778 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:11:34.777+0000 7f61ad818700 1 Processor -- start 2024-05-10T11:11:34.778 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:11:34.777+0000 7f61ad818700 1 -- start start 2024-05-10T11:11:34.778 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:11:34.777+0000 7f61ad818700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61a81026c0 0x7f61a8102130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:11:34.779 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:11:34.777+0000 7f61ad818700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61a8103060 0x7f61a819e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:11:34.779 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:11:34.777+0000 7f61ad818700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61a8103980 0x7f61a81a0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:11:34.779 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:11:34.777+0000 7f61ad818700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f61a810dd10 con 0x7f61a81026c0 2024-05-10T11:11:34.779 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:11:34.777+0000 7f61ad818700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f61a810db90 con 0x7f61a8103980 2024-05-10T11:11:34.779 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:11:34.777+0000 7f61ad818700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f61a810de90 con 0x7f61a8103060 2024-05-10T11:11:34.779 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:11:34.777+0000 7f61a67fc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61a8103060 0x7f61a819e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:11:34.779 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:11:34.777+0000 7f61a67fc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61a8103060 0x7f61a819e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:60130/0 (socket says 172.21.15.188:60130) 2024-05-10T11:11:34.779 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:11:34.777+0000 7f61a67fc700 1 -- 172.21.15.188:0/2912346830 learned_addr learned my addr 172.21.15.188:0/2912346830 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-05-10T11:11:34.779 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:11:34.777+0000 7f61a6ffd700 1 --2- 172.21.15.188:0/2912346830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61a81026c0 0x7f61a8102130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:11:34.779 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:11:34.777+0000 7f61a67fc700 1 -- 172.21.15.188:0/2912346830 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61a8103980 msgr2=0x7f61a81a0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:11:34.779 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:11:34.777+0000 7f61a77fe700 1 --2- 172.21.15.188:0/2912346830 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61a8103980 0x7f61a81a0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:11:34.779 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:11:34.777+0000 7f61a67fc700 1 --2- 172.21.15.188:0/2912346830 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61a8103980 0x7f61a81a0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:11:34.779 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:11:34.777+0000 7f61a67fc700 1 -- 172.21.15.188:0/2912346830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61a81026c0 msgr2=0x7f61a8102130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:11:34.779 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:11:34.777+0000 7f61a67fc700 1 --2- 172.21.15.188:0/2912346830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61a81026c0 0x7f61a8102130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:11:34.780 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:11:34.777+0000 7f61a67fc700 1 -- 172.21.15.188:0/2912346830 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61a81a1110 con 0x7f61a8103060 2024-05-10T11:11:34.780 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:11:34.777+0000 7f61a77fe700 1 --2- 172.21.15.188:0/2912346830 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61a8103980 0x7f61a81a0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:11:34.781 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:11:34.778+0000 7f61a67fc700 1 --2- 172.21.15.188:0/2912346830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61a8103060 0x7f61a819e1f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f619c00d950 tx=0x7f619c00dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:11:34.781 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:11:34.778+0000 7f61ac816700 1 -- 172.21.15.188:0/2912346830 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f619c016070 con 0x7f61a8103060 2024-05-10T11:11:34.781 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:11:34.778+0000 7f61ac816700 1 -- 172.21.15.188:0/2912346830 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f619c009dd0 con 0x7f61a8103060 2024-05-10T11:11:34.781 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:11:34.778+0000 7f61ad818700 1 -- 172.21.15.188:0/2912346830 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61a81a1400 con 0x7f61a8103060 2024-05-10T11:11:34.781 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:11:34.778+0000 7f61ac816700 1 -- 172.21.15.188:0/2912346830 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f619c00f3c0 con 0x7f61a8103060 2024-05-10T11:11:34.781 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:11:34.778+0000 7f61ad818700 1 -- 172.21.15.188:0/2912346830 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61a81a1940 con 0x7f61a8103060 2024-05-10T11:11:34.781 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:11:34.779+0000 7f61ad818700 1 -- 172.21.15.188:0/2912346830 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61a806a190 con 0x7f61a8103060 2024-05-10T11:11:34.781 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:11:34.780+0000 7f61ac816700 1 -- 172.21.15.188:0/2912346830 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f619c019070 con 0x7f61a8103060 2024-05-10T11:11:34.781 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:11:34.781+0000 7f61ac816700 1 --2- 172.21.15.188:0/2912346830 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f61900b2880 0x7f61900b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:11:34.781 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:11:34.781+0000 7f61ac816700 1 -- 172.21.15.188:0/2912346830 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5397+0+0 (secure 0 0 0) 0x7f619c0e12e0 con 0x7f61a8103060 2024-05-10T11:11:34.782 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:11:34.781+0000 7f61a6ffd700 1 --2- 172.21.15.188:0/2912346830 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f61900b2880 0x7f61900b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:11:34.782 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:11:34.782+0000 7f61a6ffd700 1 --2- 172.21.15.188:0/2912346830 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f61900b2880 0x7f61900b4d40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f61a806af00 tx=0x7f619800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:11:34.785 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:11:34.785+0000 7f61ac816700 1 -- 172.21.15.188:0/2912346830 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f619c0a9b60 con 0x7f61a8103060 2024-05-10T11:11:34.995 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:11:34.995+0000 7f61ad818700 1 -- 172.21.15.188:0/2912346830 --> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f61a80639a0 con 0x7f61900b2880 2024-05-10T11:11:35.647 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:35 smithi188 ceph-mon[34075]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-05-10T11:11:35.647 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:35 smithi188 ceph-mon[34075]: osdmap e50: 8 total, 8 up, 8 in 2024-05-10T11:11:35.647 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:35 smithi188 ceph-mon[34075]: from='client.24286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:11:35.647 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:35 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T11:11:35.647 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:35 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T11:11:35.648 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:35 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:11:35.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:35 smithi178 ceph-mon[33359]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-05-10T11:11:35.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:35 smithi178 ceph-mon[33359]: osdmap e50: 8 total, 8 up, 8 in 2024-05-10T11:11:35.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:35 smithi178 ceph-mon[33359]: from='client.24286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:11:35.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:35 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T11:11:35.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:35 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T11:11:35.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:35 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:11:35.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:35 smithi100 ceph-mon[30296]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-05-10T11:11:35.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:35 smithi100 ceph-mon[30296]: osdmap e50: 8 total, 8 up, 8 in 2024-05-10T11:11:35.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:35 smithi100 ceph-mon[30296]: from='client.24286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:11:35.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:35 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T11:11:35.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:35 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T11:11:35.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:35 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:11:37.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:37 smithi178 ceph-mon[33359]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-05-10T11:11:37.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:37 smithi188 ceph-mon[34075]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-05-10T11:11:37.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:37 smithi100 ceph-mon[30296]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-05-10T11:11:38.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:38 smithi178 ceph-mon[33359]: from='client.? 172.21.15.188:0/3448478551' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7e489bc-15fc-4a4c-b297-d1bb626513ce"}]: dispatch 2024-05-10T11:11:38.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:38 smithi178 ceph-mon[33359]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7e489bc-15fc-4a4c-b297-d1bb626513ce"}]: dispatch 2024-05-10T11:11:38.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:38 smithi178 ceph-mon[33359]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e7e489bc-15fc-4a4c-b297-d1bb626513ce"}]': finished 2024-05-10T11:11:38.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:38 smithi178 ceph-mon[33359]: osdmap e51: 9 total, 8 up, 9 in 2024-05-10T11:11:38.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:38 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T11:11:38.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:38 smithi188 ceph-mon[34075]: from='client.? 172.21.15.188:0/3448478551' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7e489bc-15fc-4a4c-b297-d1bb626513ce"}]: dispatch 2024-05-10T11:11:38.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:38 smithi188 ceph-mon[34075]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7e489bc-15fc-4a4c-b297-d1bb626513ce"}]: dispatch 2024-05-10T11:11:38.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:38 smithi188 ceph-mon[34075]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e7e489bc-15fc-4a4c-b297-d1bb626513ce"}]': finished 2024-05-10T11:11:38.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:38 smithi188 ceph-mon[34075]: osdmap e51: 9 total, 8 up, 9 in 2024-05-10T11:11:38.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:38 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T11:11:38.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:38 smithi100 ceph-mon[30296]: from='client.? 172.21.15.188:0/3448478551' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7e489bc-15fc-4a4c-b297-d1bb626513ce"}]: dispatch 2024-05-10T11:11:38.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:38 smithi100 ceph-mon[30296]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7e489bc-15fc-4a4c-b297-d1bb626513ce"}]: dispatch 2024-05-10T11:11:38.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:38 smithi100 ceph-mon[30296]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e7e489bc-15fc-4a4c-b297-d1bb626513ce"}]': finished 2024-05-10T11:11:38.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:38 smithi100 ceph-mon[30296]: osdmap e51: 9 total, 8 up, 9 in 2024-05-10T11:11:38.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:38 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T11:11:39.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:39 smithi178 ceph-mon[33359]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-10T11:11:39.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:39 smithi178 ceph-mon[33359]: from='client.? 172.21.15.188:0/957251309' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T11:11:39.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:39 smithi188 ceph-mon[34075]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-10T11:11:39.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:39 smithi188 ceph-mon[34075]: from='client.? 172.21.15.188:0/957251309' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T11:11:39.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:39 smithi100 ceph-mon[30296]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-10T11:11:39.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:39 smithi100 ceph-mon[30296]: from='client.? 172.21.15.188:0/957251309' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T11:11:41.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:41 smithi178 ceph-mon[33359]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-05-10T11:11:41.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:41 smithi188 ceph-mon[34075]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-05-10T11:11:41.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:41 smithi100 ceph-mon[30296]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-05-10T11:11:43.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:43 smithi178 ceph-mon[33359]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-10T11:11:43.765 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:43 smithi188 ceph-mon[34075]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-10T11:11:43.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:43 smithi100 ceph-mon[30296]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-10T11:11:45.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:45 smithi188 ceph-mon[34075]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-10T11:11:45.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:45 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-10T11:11:45.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:45 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:11:45.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:45 smithi178 ceph-mon[33359]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-10T11:11:45.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:45 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-10T11:11:45.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:45 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:11:45.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:45 smithi100 ceph-mon[30296]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-10T11:11:45.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:45 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-10T11:11:45.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:45 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:11:46.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:46 smithi178 ceph-mon[33359]: Deploying daemon osd.8 on smithi188 2024-05-10T11:11:46.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:46 smithi188 ceph-mon[34075]: Deploying daemon osd.8 on smithi188 2024-05-10T11:11:46.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:46 smithi100 ceph-mon[30296]: Deploying daemon osd.8 on smithi188 2024-05-10T11:11:47.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:47 smithi188 ceph-mon[34075]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-10T11:11:47.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:47 smithi178 ceph-mon[33359]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-10T11:11:47.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:47 smithi100 ceph-mon[30296]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-10T11:11:49.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:49 smithi188 ceph-mon[34075]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-05-10T11:11:49.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:49 smithi178 ceph-mon[33359]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-05-10T11:11:49.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:49 smithi100 ceph-mon[30296]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-05-10T11:11:52.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:51 smithi178 ceph-mon[33359]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-05-10T11:11:52.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:51 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:52.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:51 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:11:52.254 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:51 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:52.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:51 smithi188 ceph-mon[34075]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-05-10T11:11:52.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:51 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:52.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:51 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:11:52.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:51 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:52.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:51 smithi100 ceph-mon[30296]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-05-10T11:11:52.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:51 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:52.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:51 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:11:52.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:51 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:54.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:53 smithi178 ceph-mon[33359]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-05-10T11:11:54.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:53 smithi188 ceph-mon[34075]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-05-10T11:11:54.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:53 smithi100 ceph-mon[30296]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-05-10T11:11:55.011 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:11:55.011+0000 7f61ac816700 1 -- 172.21.15.188:0/2912346830 <== mgr.14152 v2:172.21.15.100:6800/2507287356 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f61a80639a0 con 0x7f61900b2880 2024-05-10T11:11:55.011 INFO:teuthology.orchestra.run.smithi188.stdout:Created osd(s) 8 on host 'smithi188' 2024-05-10T11:11:55.013 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:11:55.013+0000 7f61ad818700 1 -- 172.21.15.188:0/2912346830 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f61900b2880 msgr2=0x7f61900b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:11:55.013 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:11:55.013+0000 7f61ad818700 1 --2- 172.21.15.188:0/2912346830 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f61900b2880 0x7f61900b4d40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f61a806af00 tx=0x7f619800b410 comp rx=0 tx=0).stop 2024-05-10T11:11:55.013 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:11:55.013+0000 7f61ad818700 1 -- 172.21.15.188:0/2912346830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61a8103060 msgr2=0x7f61a819e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:11:55.013 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:11:55.013+0000 7f61ad818700 1 --2- 172.21.15.188:0/2912346830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61a8103060 0x7f61a819e1f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f619c00d950 tx=0x7f619c00dc60 comp rx=0 tx=0).stop 2024-05-10T11:11:55.013 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:11:55.013+0000 7f61ad818700 1 -- 172.21.15.188:0/2912346830 shutdown_connections 2024-05-10T11:11:55.013 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:11:55.013+0000 7f61ad818700 1 --2- 172.21.15.188:0/2912346830 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f61900b2880 0x7f61900b4d40 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:11:55.013 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:11:55.013+0000 7f61ad818700 1 --2- 172.21.15.188:0/2912346830 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61a8103980 0x7f61a81a0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:11:55.013 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:11:55.013+0000 7f61ad818700 1 --2- 172.21.15.188:0/2912346830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61a81026c0 0x7f61a8102130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:11:55.014 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:11:55.013+0000 7f61ad818700 1 --2- 172.21.15.188:0/2912346830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61a8103060 0x7f61a819e1f0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:11:55.014 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:11:55.013+0000 7f61ad818700 1 -- 172.21.15.188:0/2912346830 >> 172.21.15.188:0/2912346830 conn(0x7f61a80fd130 msgr2=0x7f61a8106a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:11:55.014 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:11:55.013+0000 7f61ad818700 1 -- 172.21.15.188:0/2912346830 shutdown_connections 2024-05-10T11:11:55.014 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:11:55.013+0000 7f61ad818700 1 -- 172.21.15.188:0/2912346830 wait complete. 2024-05-10T11:11:55.024 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:54 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:55.024 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:54 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:55.024 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:54 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:11:55.024 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:54 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:11:55.024 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:54 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:55.024 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:54 smithi188 ceph-mon[34075]: from='osd.8 [v2:172.21.15.188:6816/1523872826,v1:172.21.15.188:6817/1523872826]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-10T11:11:55.024 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:54 smithi188 ceph-mon[34075]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-10T11:11:55.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:54 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:55.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:54 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:55.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:54 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:11:55.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:54 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:11:55.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:54 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:55.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:54 smithi178 ceph-mon[33359]: from='osd.8 [v2:172.21.15.188:6816/1523872826,v1:172.21.15.188:6817/1523872826]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-10T11:11:55.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:54 smithi178 ceph-mon[33359]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-10T11:11:55.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:54 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:55.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:54 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:55.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:54 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:11:55.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:54 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:11:55.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:54 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:55.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:54 smithi100 ceph-mon[30296]: from='osd.8 [v2:172.21.15.188:6816/1523872826,v1:172.21.15.188:6817/1523872826]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-10T11:11:55.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:54 smithi100 ceph-mon[30296]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-10T11:11:55.681 DEBUG:teuthology.orchestra.run.smithi188:osd.8> sudo journalctl -f -n 0 -u ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@osd.8.service 2024-05-10T11:11:55.684 INFO:tasks.cephadm:Deploying osd.9 on smithi100 with /dev/vg_nvme/lv_1... 2024-05-10T11:11:55.684 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-10T11:11:55.936 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:11:55.967 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:55 smithi100 ceph-mon[30296]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-05-10T11:11:55.967 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:55 smithi100 ceph-mon[30296]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-10T11:11:55.967 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:55 smithi100 ceph-mon[30296]: osdmap e52: 9 total, 8 up, 9 in 2024-05-10T11:11:55.967 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:55 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T11:11:55.967 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:55 smithi100 ceph-mon[30296]: from='osd.8 [v2:172.21.15.188:6816/1523872826,v1:172.21.15.188:6817/1523872826]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-05-10T11:11:55.967 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:55 smithi100 ceph-mon[30296]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-05-10T11:11:55.967 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:55 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:55.967 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:55 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:55.967 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:55 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:11:56.233 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:55 smithi188 ceph-mon[34075]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-05-10T11:11:56.233 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:55 smithi188 ceph-mon[34075]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-10T11:11:56.234 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:55 smithi188 ceph-mon[34075]: osdmap e52: 9 total, 8 up, 9 in 2024-05-10T11:11:56.234 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:55 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T11:11:56.234 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:55 smithi188 ceph-mon[34075]: from='osd.8 [v2:172.21.15.188:6816/1523872826,v1:172.21.15.188:6817/1523872826]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-05-10T11:11:56.234 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:55 smithi188 ceph-mon[34075]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-05-10T11:11:56.234 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:55 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:56.234 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:55 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:56.234 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:55 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:11:56.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:55 smithi178 ceph-mon[33359]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-05-10T11:11:56.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:55 smithi178 ceph-mon[33359]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-10T11:11:56.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:55 smithi178 ceph-mon[33359]: osdmap e52: 9 total, 8 up, 9 in 2024-05-10T11:11:56.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:55 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T11:11:56.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:55 smithi178 ceph-mon[33359]: from='osd.8 [v2:172.21.15.188:6816/1523872826,v1:172.21.15.188:6817/1523872826]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-05-10T11:11:56.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:55 smithi178 ceph-mon[33359]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-05-10T11:11:56.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:55 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:56.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:55 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:56.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:55 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:11:56.968 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:11:56 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:11:56.658+0000 7f01aee20700 -1 osd.8 0 waiting for initial osdmap 2024-05-10T11:11:56.968 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:11:56 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:11:56.667+0000 7f01a6c0b700 -1 osd.8 54 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T11:11:57.235 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:56 smithi100 ceph-mon[30296]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-05-10T11:11:57.235 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:56 smithi100 ceph-mon[30296]: osdmap e53: 9 total, 8 up, 9 in 2024-05-10T11:11:57.235 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:56 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T11:11:57.235 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:56 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T11:11:57.236 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:56 smithi100 ceph-mon[30296]: osdmap e54: 9 total, 8 up, 9 in 2024-05-10T11:11:57.236 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:56 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T11:11:57.236 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:56 smithi100 ceph-mon[30296]: from='osd.8 ' entity='osd.8' 2024-05-10T11:11:57.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:56 smithi178 ceph-mon[33359]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-05-10T11:11:57.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:56 smithi178 ceph-mon[33359]: osdmap e53: 9 total, 8 up, 9 in 2024-05-10T11:11:57.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:56 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T11:11:57.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:56 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T11:11:57.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:56 smithi178 ceph-mon[33359]: osdmap e54: 9 total, 8 up, 9 in 2024-05-10T11:11:57.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:56 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T11:11:57.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:56 smithi178 ceph-mon[33359]: from='osd.8 ' entity='osd.8' 2024-05-10T11:11:57.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:56 smithi188 ceph-mon[34075]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-05-10T11:11:57.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:56 smithi188 ceph-mon[34075]: osdmap e53: 9 total, 8 up, 9 in 2024-05-10T11:11:57.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:56 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T11:11:57.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:56 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T11:11:57.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:56 smithi188 ceph-mon[34075]: osdmap e54: 9 total, 8 up, 9 in 2024-05-10T11:11:57.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:56 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T11:11:57.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:56 smithi188 ceph-mon[34075]: from='osd.8 ' entity='osd.8' 2024-05-10T11:11:58.156 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-10T11:11:58.156 INFO:teuthology.orchestra.run.smithi100.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-10T11:11:58.156 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10+0 records in 2024-05-10T11:11:58.156 INFO:teuthology.orchestra.run.smithi100.stderr:10+0 records out 2024-05-10T11:11:58.156 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111592 s, 940 MB/s 2024-05-10T11:11:58.156 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping successful for: 2024-05-10T11:11:58.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:57 smithi178 ceph-mon[33359]: purged_snaps scrub starts 2024-05-10T11:11:58.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:57 smithi178 ceph-mon[33359]: purged_snaps scrub ok 2024-05-10T11:11:58.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:57 smithi178 ceph-mon[33359]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-05-10T11:11:58.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:57 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T11:11:58.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:57 smithi178 ceph-mon[33359]: osd.8 [v2:172.21.15.188:6816/1523872826,v1:172.21.15.188:6817/1523872826] boot 2024-05-10T11:11:58.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:57 smithi178 ceph-mon[33359]: osdmap e55: 9 total, 9 up, 9 in 2024-05-10T11:11:58.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:57 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T11:11:58.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:57 smithi188 ceph-mon[34075]: purged_snaps scrub starts 2024-05-10T11:11:58.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:57 smithi188 ceph-mon[34075]: purged_snaps scrub ok 2024-05-10T11:11:58.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:57 smithi188 ceph-mon[34075]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-05-10T11:11:58.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:57 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T11:11:58.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:57 smithi188 ceph-mon[34075]: osd.8 [v2:172.21.15.188:6816/1523872826,v1:172.21.15.188:6817/1523872826] boot 2024-05-10T11:11:58.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:57 smithi188 ceph-mon[34075]: osdmap e55: 9 total, 9 up, 9 in 2024-05-10T11:11:58.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:57 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T11:11:58.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:57 smithi100 ceph-mon[30296]: purged_snaps scrub starts 2024-05-10T11:11:58.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:57 smithi100 ceph-mon[30296]: purged_snaps scrub ok 2024-05-10T11:11:58.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:57 smithi100 ceph-mon[30296]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-05-10T11:11:58.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:57 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T11:11:58.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:57 smithi100 ceph-mon[30296]: osd.8 [v2:172.21.15.188:6816/1523872826,v1:172.21.15.188:6817/1523872826] boot 2024-05-10T11:11:58.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:57 smithi100 ceph-mon[30296]: osdmap e55: 9 total, 9 up, 9 in 2024-05-10T11:11:58.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:57 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T11:11:58.914 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi100:vg_nvme/lv_1 2024-05-10T11:11:59.175 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:11:59.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:59 smithi100 ceph-mon[30296]: osdmap e56: 9 total, 9 up, 9 in 2024-05-10T11:11:59.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:59 smithi100 ceph-mon[30296]: pgmap v195: 1 pgs: 1 peering; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-10T11:11:59.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:59 smithi100 ceph-mon[30296]: Detected new or changed devices on smithi188 2024-05-10T11:11:59.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:59 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:59.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:59 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:59.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:59 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:11:59.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:59 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:11:59.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:59 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:11:59.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:59 smithi100 ceph-mon[30296]: Adjusting osd_memory_target on smithi188 to 5696M 2024-05-10T11:11:59.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:59 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:59.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:59 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:11:59.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:59 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:11:59.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:11:59 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:59.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:59 smithi178 ceph-mon[33359]: osdmap e56: 9 total, 9 up, 9 in 2024-05-10T11:11:59.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:59 smithi178 ceph-mon[33359]: pgmap v195: 1 pgs: 1 peering; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-10T11:11:59.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:59 smithi178 ceph-mon[33359]: Detected new or changed devices on smithi188 2024-05-10T11:11:59.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:59 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:59.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:59 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:59.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:59 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:11:59.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:59 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:11:59.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:59 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:11:59.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:59 smithi178 ceph-mon[33359]: Adjusting osd_memory_target on smithi188 to 5696M 2024-05-10T11:11:59.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:59 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:59.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:59 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:11:59.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:59 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:11:59.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:11:59 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:59.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:59 smithi188 ceph-mon[34075]: osdmap e56: 9 total, 9 up, 9 in 2024-05-10T11:11:59.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:59 smithi188 ceph-mon[34075]: pgmap v195: 1 pgs: 1 peering; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-10T11:11:59.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:59 smithi188 ceph-mon[34075]: Detected new or changed devices on smithi188 2024-05-10T11:11:59.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:59 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:59.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:59 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:59.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:59 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:11:59.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:59 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:11:59.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:59 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:11:59.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:59 smithi188 ceph-mon[34075]: Adjusting osd_memory_target on smithi188 to 5696M 2024-05-10T11:11:59.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:59 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:11:59.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:59 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:11:59.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:59 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:11:59.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:11:59 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:12:00.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:00 smithi100 ceph-mon[30296]: osdmap e57: 9 total, 9 up, 9 in 2024-05-10T11:12:00.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:00 smithi178 ceph-mon[33359]: osdmap e57: 9 total, 9 up, 9 in 2024-05-10T11:12:00.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:00 smithi188 ceph-mon[34075]: osdmap e57: 9 total, 9 up, 9 in 2024-05-10T11:12:00.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:12:00.881+0000 7f7c14c88700 1 -- 172.21.15.100:0/3433247302 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c10101ef0 msgr2=0x7f7c1010bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:12:00.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:12:00.881+0000 7f7c14c88700 1 --2- 172.21.15.100:0/3433247302 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c10101ef0 0x7f7c1010bac0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7c0800b0a0 tx=0x7f7c0800b3b0 comp rx=0 tx=0).stop 2024-05-10T11:12:00.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:12:00.881+0000 7f7c14c88700 1 -- 172.21.15.100:0/3433247302 shutdown_connections 2024-05-10T11:12:00.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:12:00.881+0000 7f7c14c88700 1 --2- 172.21.15.100:0/3433247302 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c10101ef0 0x7f7c1010bac0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:12:00.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:12:00.881+0000 7f7c14c88700 1 --2- 172.21.15.100:0/3433247302 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c10104ad0 0x7f7c10104ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:12:00.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:12:00.881+0000 7f7c14c88700 1 --2- 172.21.15.100:0/3433247302 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c10101550 0x7f7c101019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:12:00.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:12:00.881+0000 7f7c14c88700 1 -- 172.21.15.100:0/3433247302 >> 172.21.15.100:0/3433247302 conn(0x7f7c100fd130 msgr2=0x7f7c100ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:12:00.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:12:00.881+0000 7f7c14c88700 1 -- 172.21.15.100:0/3433247302 shutdown_connections 2024-05-10T11:12:00.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:12:00.881+0000 7f7c14c88700 1 -- 172.21.15.100:0/3433247302 wait complete. 2024-05-10T11:12:00.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:12:00.882+0000 7f7c14c88700 1 Processor -- start 2024-05-10T11:12:00.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:12:00.882+0000 7f7c14c88700 1 -- start start 2024-05-10T11:12:00.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:12:00.882+0000 7f7c14c88700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c10101550 0x7f7c101995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:12:00.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:12:00.882+0000 7f7c14c88700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c10101ef0 0x7f7c10199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:12:00.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:12:00.882+0000 7f7c14c88700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c10104ad0 0x7f7c101a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:12:00.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:12:00.882+0000 7f7c14c88700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7c1010eec0 con 0x7f7c10101ef0 2024-05-10T11:12:00.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:12:00.882+0000 7f7c14c88700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7c1010ed40 con 0x7f7c10104ad0 2024-05-10T11:12:00.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:12:00.882+0000 7f7c14c88700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7c1010f040 con 0x7f7c10101550 2024-05-10T11:12:00.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:12:00.882+0000 7f7c07fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c10101ef0 0x7f7c10199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:12:00.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:12:00.882+0000 7f7c07fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c10101ef0 0x7f7c10199b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52352/0 (socket says 172.21.15.100:52352) 2024-05-10T11:12:00.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:12:00.882+0000 7f7c07fff700 1 -- 172.21.15.100:0/2246518280 learned_addr learned my addr 172.21.15.100:0/2246518280 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:12:00.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:12:00.882+0000 7f7c0e59c700 1 --2- 172.21.15.100:0/2246518280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c10101550 0x7f7c101995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:12:00.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:12:00.882+0000 7f7c0ed9d700 1 --2- 172.21.15.100:0/2246518280 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c10104ad0 0x7f7c101a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:12:00.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:12:00.882+0000 7f7c07fff700 1 -- 172.21.15.100:0/2246518280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c10101550 msgr2=0x7f7c101995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:12:00.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:12:00.882+0000 7f7c07fff700 1 --2- 172.21.15.100:0/2246518280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c10101550 0x7f7c101995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:12:00.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:12:00.882+0000 7f7c07fff700 1 -- 172.21.15.100:0/2246518280 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c10104ad0 msgr2=0x7f7c101a0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:12:00.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:12:00.882+0000 7f7c07fff700 1 --2- 172.21.15.100:0/2246518280 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c10104ad0 0x7f7c101a0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:12:00.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:12:00.882+0000 7f7c07fff700 1 -- 172.21.15.100:0/2246518280 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c101a11d0 con 0x7f7c10101ef0 2024-05-10T11:12:00.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:12:00.882+0000 7f7c0ed9d700 1 --2- 172.21.15.100:0/2246518280 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c10104ad0 0x7f7c101a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:12:00.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:12:00.883+0000 7f7c07fff700 1 --2- 172.21.15.100:0/2246518280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c10101ef0 0x7f7c10199b00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7bf800c950 tx=0x7f7bf800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:12:00.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:12:00.883+0000 7f7c077fe700 1 -- 172.21.15.100:0/2246518280 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf8007ac0 con 0x7f7c10101ef0 2024-05-10T11:12:00.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:12:00.883+0000 7f7c077fe700 1 -- 172.21.15.100:0/2246518280 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7bf8004cc0 con 0x7f7c10101ef0 2024-05-10T11:12:00.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:12:00.883+0000 7f7c077fe700 1 -- 172.21.15.100:0/2246518280 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf8005440 con 0x7f7c10101ef0 2024-05-10T11:12:00.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:12:00.883+0000 7f7c14c88700 1 -- 172.21.15.100:0/2246518280 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c101a14c0 con 0x7f7c10101ef0 2024-05-10T11:12:00.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:12:00.883+0000 7f7c14c88700 1 -- 172.21.15.100:0/2246518280 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c101a1a80 con 0x7f7c10101ef0 2024-05-10T11:12:00.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:12:00.884+0000 7f7c14c88700 1 -- 172.21.15.100:0/2246518280 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c1006a190 con 0x7f7c10101ef0 2024-05-10T11:12:00.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:12:00.885+0000 7f7c077fe700 1 -- 172.21.15.100:0/2246518280 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f7bf8004750 con 0x7f7c10101ef0 2024-05-10T11:12:00.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:12:00.886+0000 7f7c077fe700 1 --2- 172.21.15.100:0/2246518280 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f7bfc0b2880 0x7f7bfc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:12:00.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:12:00.886+0000 7f7c0e59c700 1 --2- 172.21.15.100:0/2246518280 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f7bfc0b2880 0x7f7bfc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:12:00.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:12:00.886+0000 7f7c077fe700 1 -- 172.21.15.100:0/2246518280 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5829+0+0 (secure 0 0 0) 0x7f7bf80e1440 con 0x7f7c10101ef0 2024-05-10T11:12:00.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:12:00.886+0000 7f7c0e59c700 1 --2- 172.21.15.100:0/2246518280 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f7bfc0b2880 0x7f7bfc0b4d40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7c1006aea0 tx=0x7f7c00009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:12:00.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:12:00.891+0000 7f7c077fe700 1 -- 172.21.15.100:0/2246518280 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f7bf8014030 con 0x7f7c10101ef0 2024-05-10T11:12:01.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:12:01.106+0000 7f7c14c88700 1 -- 172.21.15.100:0/2246518280 --> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f7c100638e0 con 0x7f7bfc0b2880 2024-05-10T11:12:01.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:01 smithi178 ceph-mon[33359]: pgmap v197: 1 pgs: 1 peering; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-10T11:12:01.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:01 smithi188 ceph-mon[34075]: pgmap v197: 1 pgs: 1 peering; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-10T11:12:01.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:01 smithi100 ceph-mon[30296]: pgmap v197: 1 pgs: 1 peering; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-10T11:12:02.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:02 smithi100 ceph-mon[30296]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:12:02.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:02 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T11:12:02.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:02 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T11:12:02.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:02 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:12:02.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:02 smithi178 ceph-mon[33359]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:12:02.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:02 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T11:12:02.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:02 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T11:12:02.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:02 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:12:02.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:02 smithi188 ceph-mon[34075]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:12:02.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:02 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T11:12:02.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:02 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T11:12:02.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:02 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:12:03.353 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:03 smithi100 ceph-mon[30296]: pgmap v198: 1 pgs: 1 peering; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-10T11:12:03.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:03 smithi178 ceph-mon[33359]: pgmap v198: 1 pgs: 1 peering; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-10T11:12:03.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:03 smithi188 ceph-mon[34075]: pgmap v198: 1 pgs: 1 peering; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-10T11:12:04.467 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:04 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/3108206436' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "72328326-cd16-45d9-bfd9-40f6c5c20f5c"}]: dispatch 2024-05-10T11:12:04.467 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:04 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/3108206436' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "72328326-cd16-45d9-bfd9-40f6c5c20f5c"}]': finished 2024-05-10T11:12:04.467 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:04 smithi100 ceph-mon[30296]: osdmap e58: 10 total, 9 up, 10 in 2024-05-10T11:12:04.467 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:04 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T11:12:04.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:04 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/3108206436' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "72328326-cd16-45d9-bfd9-40f6c5c20f5c"}]: dispatch 2024-05-10T11:12:04.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:04 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/3108206436' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "72328326-cd16-45d9-bfd9-40f6c5c20f5c"}]': finished 2024-05-10T11:12:04.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:04 smithi178 ceph-mon[33359]: osdmap e58: 10 total, 9 up, 10 in 2024-05-10T11:12:04.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:04 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T11:12:04.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:04 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/3108206436' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "72328326-cd16-45d9-bfd9-40f6c5c20f5c"}]: dispatch 2024-05-10T11:12:04.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:04 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/3108206436' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "72328326-cd16-45d9-bfd9-40f6c5c20f5c"}]': finished 2024-05-10T11:12:04.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:04 smithi188 ceph-mon[34075]: osdmap e58: 10 total, 9 up, 10 in 2024-05-10T11:12:04.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:04 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T11:12:05.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:05 smithi178 ceph-mon[33359]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-05-10T11:12:05.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:05 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/1204091909' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T11:12:05.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:05 smithi188 ceph-mon[34075]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-05-10T11:12:05.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:05 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/1204091909' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T11:12:05.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:05 smithi100 ceph-mon[30296]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-05-10T11:12:05.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:05 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/1204091909' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T11:12:07.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:07 smithi178 ceph-mon[33359]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-10T11:12:07.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:07 smithi188 ceph-mon[34075]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-10T11:12:07.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:07 smithi100 ceph-mon[30296]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-10T11:12:09.497 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:09 smithi100 ceph-mon[30296]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-05-10T11:12:09.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:09 smithi178 ceph-mon[33359]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-05-10T11:12:09.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:09 smithi188 ceph-mon[34075]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-05-10T11:12:11.268 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:11 smithi100 ceph-mon[30296]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-10T11:12:11.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:11 smithi178 ceph-mon[33359]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-10T11:12:11.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:11 smithi188 ceph-mon[34075]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-10T11:12:12.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:12 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-10T11:12:12.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:12 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:12:12.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:12 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-10T11:12:12.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:12 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:12:12.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:12 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-10T11:12:12.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:12 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:12:13.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:13 smithi100 ceph-mon[30296]: Deploying daemon osd.9 on smithi100 2024-05-10T11:12:13.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:13 smithi100 ceph-mon[30296]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-10T11:12:13.429 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:13 smithi188 ceph-mon[34075]: Deploying daemon osd.9 on smithi100 2024-05-10T11:12:13.430 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:13 smithi188 ceph-mon[34075]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-10T11:12:13.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:13 smithi178 ceph-mon[33359]: Deploying daemon osd.9 on smithi100 2024-05-10T11:12:13.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:13 smithi178 ceph-mon[33359]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-10T11:12:15.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:15 smithi100 ceph-mon[30296]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-10T11:12:15.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:15 smithi178 ceph-mon[33359]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-10T11:12:15.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:15 smithi188 ceph-mon[34075]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-10T11:12:17.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:17 smithi178 ceph-mon[33359]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-10T11:12:17.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:17 smithi188 ceph-mon[34075]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-10T11:12:17.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:17 smithi100 ceph-mon[30296]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-10T11:12:19.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:19 smithi100 ceph-mon[30296]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-10T11:12:19.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:19 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:12:19.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:19 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:12:19.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:19 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:12:19.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:19 smithi178 ceph-mon[33359]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-10T11:12:19.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:19 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:12:19.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:19 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:12:19.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:19 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:12:19.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:19 smithi188 ceph-mon[34075]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-10T11:12:19.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:19 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:12:19.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:19 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:12:19.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:19 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:12:21.625 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:21 smithi100 ceph-mon[30296]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-10T11:12:21.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:21 smithi178 ceph-mon[33359]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-10T11:12:21.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:21 smithi188 ceph-mon[34075]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-10T11:12:22.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:12:22.466+0000 7f7c077fe700 1 -- 172.21.15.100:0/2246518280 <== mgr.14152 v2:172.21.15.100:6800/2507287356 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7c100638e0 con 0x7f7bfc0b2880 2024-05-10T11:12:22.466 INFO:teuthology.orchestra.run.smithi100.stdout:Created osd(s) 9 on host 'smithi100' 2024-05-10T11:12:22.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:12:22.469+0000 7f7c14c88700 1 -- 172.21.15.100:0/2246518280 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f7bfc0b2880 msgr2=0x7f7bfc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:12:22.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:12:22.469+0000 7f7c14c88700 1 --2- 172.21.15.100:0/2246518280 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f7bfc0b2880 0x7f7bfc0b4d40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7c1006aea0 tx=0x7f7c00009190 comp rx=0 tx=0).stop 2024-05-10T11:12:22.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:12:22.469+0000 7f7c14c88700 1 -- 172.21.15.100:0/2246518280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c10101ef0 msgr2=0x7f7c10199b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:12:22.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:12:22.469+0000 7f7c14c88700 1 --2- 172.21.15.100:0/2246518280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c10101ef0 0x7f7c10199b00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7bf800c950 tx=0x7f7bf800cc60 comp rx=0 tx=0).stop 2024-05-10T11:12:22.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:12:22.469+0000 7f7c14c88700 1 -- 172.21.15.100:0/2246518280 shutdown_connections 2024-05-10T11:12:22.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:12:22.469+0000 7f7c14c88700 1 --2- 172.21.15.100:0/2246518280 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f7bfc0b2880 0x7f7bfc0b4d40 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:12:22.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:12:22.469+0000 7f7c14c88700 1 --2- 172.21.15.100:0/2246518280 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c10104ad0 0x7f7c101a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:12:22.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:12:22.469+0000 7f7c14c88700 1 --2- 172.21.15.100:0/2246518280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c10101550 0x7f7c101995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:12:22.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:12:22.469+0000 7f7c14c88700 1 --2- 172.21.15.100:0/2246518280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c10101ef0 0x7f7c10199b00 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:12:22.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:12:22.469+0000 7f7c14c88700 1 -- 172.21.15.100:0/2246518280 >> 172.21.15.100:0/2246518280 conn(0x7f7c100fd130 msgr2=0x7f7c10107bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:12:22.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:12:22.469+0000 7f7c14c88700 1 -- 172.21.15.100:0/2246518280 shutdown_connections 2024-05-10T11:12:22.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:12:22.469+0000 7f7c14c88700 1 -- 172.21.15.100:0/2246518280 wait complete. 2024-05-10T11:12:22.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:22 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:12:22.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:22 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:12:22.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:22 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:12:22.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:22 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:12:22.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:22 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:12:22.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:22 smithi178 ceph-mon[33359]: from='osd.9 [v2:172.21.15.100:6826/3585761978,v1:172.21.15.100:6827/3585761978]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-10T11:12:22.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:22 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:12:22.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:22 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:12:22.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:22 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:12:22.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:22 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:12:22.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:22 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:12:22.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:22 smithi188 ceph-mon[34075]: from='osd.9 [v2:172.21.15.100:6826/3585761978,v1:172.21.15.100:6827/3585761978]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-10T11:12:22.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:22 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:12:22.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:22 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:12:22.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:22 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:12:22.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:22 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:12:22.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:22 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:12:22.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:22 smithi100 ceph-mon[30296]: from='osd.9 [v2:172.21.15.100:6826/3585761978,v1:172.21.15.100:6827/3585761978]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-10T11:12:23.214 DEBUG:teuthology.orchestra.run.smithi100:osd.9> sudo journalctl -f -n 0 -u ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@osd.9.service 2024-05-10T11:12:23.217 INFO:tasks.cephadm:Deploying osd.10 on smithi178 with /dev/vg_nvme/lv_1... 2024-05-10T11:12:23.217 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-10T11:12:23.467 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T11:12:23.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:23 smithi178 ceph-mon[33359]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-10T11:12:23.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:23 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:12:23.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:23 smithi178 ceph-mon[33359]: from='osd.9 [v2:172.21.15.100:6826/3585761978,v1:172.21.15.100:6827/3585761978]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-10T11:12:23.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:23 smithi178 ceph-mon[33359]: osdmap e59: 10 total, 9 up, 10 in 2024-05-10T11:12:23.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:23 smithi178 ceph-mon[33359]: from='osd.9 [v2:172.21.15.100:6826/3585761978,v1:172.21.15.100:6827/3585761978]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-05-10T11:12:23.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:23 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T11:12:23.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:23 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:12:23.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:23 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:12:23.766 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:23 smithi100 ceph-mon[30296]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-10T11:12:23.766 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:23 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:12:23.766 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:23 smithi100 ceph-mon[30296]: from='osd.9 [v2:172.21.15.100:6826/3585761978,v1:172.21.15.100:6827/3585761978]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-10T11:12:23.766 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:23 smithi100 ceph-mon[30296]: osdmap e59: 10 total, 9 up, 10 in 2024-05-10T11:12:23.766 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:23 smithi100 ceph-mon[30296]: from='osd.9 [v2:172.21.15.100:6826/3585761978,v1:172.21.15.100:6827/3585761978]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-05-10T11:12:23.766 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:23 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T11:12:23.766 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:23 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:12:23.766 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:23 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:12:23.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:23 smithi188 ceph-mon[34075]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-10T11:12:23.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:23 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:12:23.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:23 smithi188 ceph-mon[34075]: from='osd.9 [v2:172.21.15.100:6826/3585761978,v1:172.21.15.100:6827/3585761978]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-10T11:12:23.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:23 smithi188 ceph-mon[34075]: osdmap e59: 10 total, 9 up, 10 in 2024-05-10T11:12:23.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:23 smithi188 ceph-mon[34075]: from='osd.9 [v2:172.21.15.100:6826/3585761978,v1:172.21.15.100:6827/3585761978]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-05-10T11:12:23.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:23 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T11:12:23.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:23 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:12:23.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:23 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:12:24.449 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:12:24 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:12:24.144+0000 7fb62ba9b700 -1 osd.9 0 waiting for initial osdmap 2024-05-10T11:12:24.449 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:12:24 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:12:24.155+0000 7fb623886700 -1 osd.9 60 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T11:12:24.702 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:24 smithi188 ceph-mon[34075]: from='osd.9 [v2:172.21.15.100:6826/3585761978,v1:172.21.15.100:6827/3585761978]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-05-10T11:12:24.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:24 smithi188 ceph-mon[34075]: osdmap e60: 10 total, 9 up, 10 in 2024-05-10T11:12:24.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:24 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T11:12:24.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:24 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T11:12:24.703 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:24 smithi188 ceph-mon[34075]: from='osd.9 [v2:172.21.15.100:6826/3585761978,v1:172.21.15.100:6827/3585761978]' entity='osd.9' 2024-05-10T11:12:24.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:24 smithi178 ceph-mon[33359]: from='osd.9 [v2:172.21.15.100:6826/3585761978,v1:172.21.15.100:6827/3585761978]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-05-10T11:12:24.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:24 smithi178 ceph-mon[33359]: osdmap e60: 10 total, 9 up, 10 in 2024-05-10T11:12:24.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:24 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T11:12:24.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:24 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T11:12:24.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:24 smithi178 ceph-mon[33359]: from='osd.9 [v2:172.21.15.100:6826/3585761978,v1:172.21.15.100:6827/3585761978]' entity='osd.9' 2024-05-10T11:12:24.780 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:24 smithi100 ceph-mon[30296]: from='osd.9 [v2:172.21.15.100:6826/3585761978,v1:172.21.15.100:6827/3585761978]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-05-10T11:12:24.780 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:24 smithi100 ceph-mon[30296]: osdmap e60: 10 total, 9 up, 10 in 2024-05-10T11:12:24.780 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:24 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T11:12:24.780 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:24 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T11:12:24.780 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:24 smithi100 ceph-mon[30296]: from='osd.9 [v2:172.21.15.100:6826/3585761978,v1:172.21.15.100:6827/3585761978]' entity='osd.9' 2024-05-10T11:12:25.489 INFO:teuthology.orchestra.run.smithi178.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-10T11:12:25.489 INFO:teuthology.orchestra.run.smithi178.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-10T11:12:25.489 INFO:teuthology.orchestra.run.smithi178.stderr: stderr: 10+0 records in 2024-05-10T11:12:25.489 INFO:teuthology.orchestra.run.smithi178.stderr:10+0 records out 2024-05-10T11:12:25.489 INFO:teuthology.orchestra.run.smithi178.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00950839 s, 1.1 GB/s 2024-05-10T11:12:25.489 INFO:teuthology.orchestra.run.smithi178.stderr:--> Zapping successful for: 2024-05-10T11:12:25.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:25 smithi178 ceph-mon[33359]: purged_snaps scrub starts 2024-05-10T11:12:25.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:25 smithi178 ceph-mon[33359]: purged_snaps scrub ok 2024-05-10T11:12:25.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:25 smithi178 ceph-mon[33359]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-10T11:12:25.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:25 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T11:12:25.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:25 smithi178 ceph-mon[33359]: osd.9 [v2:172.21.15.100:6826/3585761978,v1:172.21.15.100:6827/3585761978] boot 2024-05-10T11:12:25.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:25 smithi178 ceph-mon[33359]: osdmap e61: 10 total, 10 up, 10 in 2024-05-10T11:12:25.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:25 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T11:12:25.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:25 smithi188 ceph-mon[34075]: purged_snaps scrub starts 2024-05-10T11:12:25.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:25 smithi188 ceph-mon[34075]: purged_snaps scrub ok 2024-05-10T11:12:25.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:25 smithi188 ceph-mon[34075]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-10T11:12:25.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:25 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T11:12:25.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:25 smithi188 ceph-mon[34075]: osd.9 [v2:172.21.15.100:6826/3585761978,v1:172.21.15.100:6827/3585761978] boot 2024-05-10T11:12:25.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:25 smithi188 ceph-mon[34075]: osdmap e61: 10 total, 10 up, 10 in 2024-05-10T11:12:25.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:25 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T11:12:25.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:25 smithi100 ceph-mon[30296]: purged_snaps scrub starts 2024-05-10T11:12:25.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:25 smithi100 ceph-mon[30296]: purged_snaps scrub ok 2024-05-10T11:12:25.789 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:25 smithi100 ceph-mon[30296]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-10T11:12:25.789 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:25 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T11:12:25.789 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:25 smithi100 ceph-mon[30296]: osd.9 [v2:172.21.15.100:6826/3585761978,v1:172.21.15.100:6827/3585761978] boot 2024-05-10T11:12:25.789 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:25 smithi100 ceph-mon[30296]: osdmap e61: 10 total, 10 up, 10 in 2024-05-10T11:12:25.789 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:25 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T11:12:26.265 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi178:vg_nvme/lv_1 2024-05-10T11:12:26.504 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T11:12:27.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:27 smithi178 ceph-mon[33359]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-10T11:12:27.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:27 smithi178 ceph-mon[33359]: osdmap e62: 10 total, 10 up, 10 in 2024-05-10T11:12:27.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:27 smithi178 ceph-mon[33359]: Detected new or changed devices on smithi100 2024-05-10T11:12:27.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:27 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:12:27.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:27 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:12:27.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:27 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:12:27.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:27 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:12:27.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:27 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:12:27.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:27 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:12:27.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:27 smithi178 ceph-mon[33359]: Adjusting osd_memory_target on smithi100 to 4272M 2024-05-10T11:12:27.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:27 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:12:27.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:27 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:12:27.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:27 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:12:27.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:27 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:12:27.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:27 smithi188 ceph-mon[34075]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-10T11:12:27.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:27 smithi188 ceph-mon[34075]: osdmap e62: 10 total, 10 up, 10 in 2024-05-10T11:12:27.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:27 smithi188 ceph-mon[34075]: Detected new or changed devices on smithi100 2024-05-10T11:12:27.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:27 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:12:27.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:27 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:12:27.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:27 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:12:27.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:27 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:12:27.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:27 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:12:27.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:27 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:12:27.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:27 smithi188 ceph-mon[34075]: Adjusting osd_memory_target on smithi100 to 4272M 2024-05-10T11:12:27.780 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:27 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:12:27.780 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:27 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:12:27.780 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:27 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:12:27.780 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:27 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:12:27.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:27 smithi100 ceph-mon[30296]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-10T11:12:27.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:27 smithi100 ceph-mon[30296]: osdmap e62: 10 total, 10 up, 10 in 2024-05-10T11:12:27.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:27 smithi100 ceph-mon[30296]: Detected new or changed devices on smithi100 2024-05-10T11:12:27.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:27 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:12:27.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:27 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:12:27.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:27 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:12:27.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:27 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:12:27.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:27 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:12:27.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:27 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:12:27.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:27 smithi100 ceph-mon[30296]: Adjusting osd_memory_target on smithi100 to 4272M 2024-05-10T11:12:27.789 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:27 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:12:27.789 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:27 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:12:27.789 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:27 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:12:27.789 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:27 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:12:28.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:12:28.129+0000 7f7faba45700 1 -- 172.21.15.178:0/154443112 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fa4102410 msgr2=0x7f7fa4102830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:12:28.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:12:28.129+0000 7f7faba45700 1 --2- 172.21.15.178:0/154443112 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fa4102410 0x7f7fa4102830 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7f940099c0 tx=0x7f7f94009cd0 comp rx=0 tx=0).stop 2024-05-10T11:12:28.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:12:28.129+0000 7f7faba45700 1 -- 172.21.15.178:0/154443112 shutdown_connections 2024-05-10T11:12:28.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:12:28.129+0000 7f7faba45700 1 --2- 172.21.15.178:0/154443112 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fa4103fb0 0x7f7fa410a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:12:28.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:12:28.129+0000 7f7faba45700 1 --2- 172.21.15.178:0/154443112 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fa4102410 0x7f7fa4102830 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:12:28.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:12:28.129+0000 7f7faba45700 1 --2- 172.21.15.178:0/154443112 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fa4103610 0x7f7fa4103a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:12:28.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:12:28.129+0000 7f7faba45700 1 -- 172.21.15.178:0/154443112 >> 172.21.15.178:0/154443112 conn(0x7f7fa40fd9d0 msgr2=0x7f7fa40ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:12:28.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:12:28.129+0000 7f7faba45700 1 -- 172.21.15.178:0/154443112 shutdown_connections 2024-05-10T11:12:28.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:12:28.129+0000 7f7faba45700 1 -- 172.21.15.178:0/154443112 wait complete. 2024-05-10T11:12:28.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:12:28.130+0000 7f7faba45700 1 Processor -- start 2024-05-10T11:12:28.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:12:28.130+0000 7f7faba45700 1 -- start start 2024-05-10T11:12:28.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:12:28.130+0000 7f7faba45700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fa4102410 0x7f7fa4199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:12:28.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:12:28.130+0000 7f7faba45700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fa4103610 0x7f7fa4199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:12:28.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:12:28.130+0000 7f7faba45700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fa4103fb0 0x7f7fa41a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:12:28.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:12:28.130+0000 7f7faba45700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa410eee0 con 0x7f7fa4103610 2024-05-10T11:12:28.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:12:28.130+0000 7f7faba45700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa410ed60 con 0x7f7fa4103fb0 2024-05-10T11:12:28.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:12:28.130+0000 7f7faba45700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa410f060 con 0x7f7fa4102410 2024-05-10T11:12:28.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:12:28.130+0000 7f7fa9fe2700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fa4103fb0 0x7f7fa41a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:12:28.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:12:28.130+0000 7f7fa9fe2700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fa4103fb0 0x7f7fa41a0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:48636/0 (socket says 172.21.15.178:48636) 2024-05-10T11:12:28.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:12:28.130+0000 7f7fa9fe2700 1 -- 172.21.15.178:0/4063797035 learned_addr learned my addr 172.21.15.178:0/4063797035 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-05-10T11:12:28.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:12:28.130+0000 7f7fa97e1700 1 --2- 172.21.15.178:0/4063797035 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fa4102410 0x7f7fa4199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:12:28.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:12:28.130+0000 7f7fa8fe0700 1 --2- 172.21.15.178:0/4063797035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fa4103610 0x7f7fa4199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:12:28.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:12:28.130+0000 7f7fa9fe2700 1 -- 172.21.15.178:0/4063797035 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fa4102410 msgr2=0x7f7fa4199620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:12:28.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:12:28.130+0000 7f7fa9fe2700 1 --2- 172.21.15.178:0/4063797035 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fa4102410 0x7f7fa4199620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:12:28.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:12:28.130+0000 7f7fa9fe2700 1 -- 172.21.15.178:0/4063797035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fa4103610 msgr2=0x7f7fa4199b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:12:28.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:12:28.130+0000 7f7fa9fe2700 1 --2- 172.21.15.178:0/4063797035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fa4103610 0x7f7fa4199b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:12:28.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:12:28.130+0000 7f7fa9fe2700 1 -- 172.21.15.178:0/4063797035 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fa41a1230 con 0x7f7fa4103fb0 2024-05-10T11:12:28.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:12:28.130+0000 7f7fa9fe2700 1 --2- 172.21.15.178:0/4063797035 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fa4103fb0 0x7f7fa41a0b90 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c00b970 tx=0x7f7f9c00bd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:12:28.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:12:28.130+0000 7f7f9a7fc700 1 -- 172.21.15.178:0/4063797035 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c019070 con 0x7f7fa4103fb0 2024-05-10T11:12:28.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:12:28.131+0000 7f7faba45700 1 -- 172.21.15.178:0/4063797035 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fa41a1520 con 0x7f7fa4103fb0 2024-05-10T11:12:28.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:12:28.131+0000 7f7f9a7fc700 1 -- 172.21.15.178:0/4063797035 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7f9c00ce40 con 0x7f7fa4103fb0 2024-05-10T11:12:28.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:12:28.131+0000 7f7f9a7fc700 1 -- 172.21.15.178:0/4063797035 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c007370 con 0x7f7fa4103fb0 2024-05-10T11:12:28.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:12:28.131+0000 7f7faba45700 1 -- 172.21.15.178:0/4063797035 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fa41a1a60 con 0x7f7fa4103fb0 2024-05-10T11:12:28.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:12:28.132+0000 7f7faba45700 1 -- 172.21.15.178:0/4063797035 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fa406a190 con 0x7f7fa4103fb0 2024-05-10T11:12:28.135 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:12:28.133+0000 7f7f9a7fc700 1 -- 172.21.15.178:0/4063797035 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f7f9c01c070 con 0x7f7fa4103fb0 2024-05-10T11:12:28.135 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:12:28.133+0000 7f7f9a7fc700 1 --2- 172.21.15.178:0/4063797035 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f7f900b2880 0x7f7f900b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:12:28.135 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:12:28.134+0000 7f7f9a7fc700 1 -- 172.21.15.178:0/4063797035 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 6261+0+0 (secure 0 0 0) 0x7f7f9c0e1070 con 0x7f7fa4103fb0 2024-05-10T11:12:28.135 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:12:28.134+0000 7f7fa97e1700 1 --2- 172.21.15.178:0/4063797035 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f7f900b2880 0x7f7f900b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:12:28.135 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:12:28.134+0000 7f7fa97e1700 1 --2- 172.21.15.178:0/4063797035 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f7f900b2880 0x7f7f900b4d40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7f94006010 tx=0x7f7f94019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:12:28.138 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:12:28.138+0000 7f7f9a7fc700 1 -- 172.21.15.178:0/4063797035 <== mon.2 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f7f9c051d70 con 0x7f7fa4103fb0 2024-05-10T11:12:28.349 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:12:28.349+0000 7f7faba45700 1 -- 172.21.15.178:0/4063797035 --> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f7fa40638e0 con 0x7f7f900b2880 2024-05-10T11:12:28.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:28 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T11:12:28.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:28 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T11:12:28.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:28 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:12:28.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:28 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T11:12:28.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:28 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T11:12:28.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:28 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:12:28.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:28 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T11:12:28.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:28 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T11:12:28.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:28 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:12:29.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:29 smithi178 ceph-mon[33359]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-10T11:12:29.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:29 smithi178 ceph-mon[33359]: from='client.24314 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:12:29.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:29 smithi188 ceph-mon[34075]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-10T11:12:29.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:29 smithi188 ceph-mon[34075]: from='client.24314 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:12:29.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:29 smithi100 ceph-mon[30296]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-10T11:12:29.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:29 smithi100 ceph-mon[30296]: from='client.24314 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:12:31.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:31 smithi178 ceph-mon[33359]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-10T11:12:31.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:31 smithi178 ceph-mon[33359]: from='client.? 172.21.15.178:0/54009845' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a355f9a8-91a3-41d4-814d-5a9c9b953706"}]: dispatch 2024-05-10T11:12:31.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:31 smithi178 ceph-mon[33359]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a355f9a8-91a3-41d4-814d-5a9c9b953706"}]: dispatch 2024-05-10T11:12:31.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:31 smithi178 ceph-mon[33359]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a355f9a8-91a3-41d4-814d-5a9c9b953706"}]': finished 2024-05-10T11:12:31.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:31 smithi178 ceph-mon[33359]: osdmap e63: 11 total, 10 up, 11 in 2024-05-10T11:12:31.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:31 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T11:12:31.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:31 smithi188 ceph-mon[34075]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-10T11:12:31.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:31 smithi188 ceph-mon[34075]: from='client.? 172.21.15.178:0/54009845' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a355f9a8-91a3-41d4-814d-5a9c9b953706"}]: dispatch 2024-05-10T11:12:31.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:31 smithi188 ceph-mon[34075]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a355f9a8-91a3-41d4-814d-5a9c9b953706"}]: dispatch 2024-05-10T11:12:31.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:31 smithi188 ceph-mon[34075]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a355f9a8-91a3-41d4-814d-5a9c9b953706"}]': finished 2024-05-10T11:12:31.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:31 smithi188 ceph-mon[34075]: osdmap e63: 11 total, 10 up, 11 in 2024-05-10T11:12:31.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:31 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T11:12:31.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:31 smithi100 ceph-mon[30296]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-10T11:12:31.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:31 smithi100 ceph-mon[30296]: from='client.? 172.21.15.178:0/54009845' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a355f9a8-91a3-41d4-814d-5a9c9b953706"}]: dispatch 2024-05-10T11:12:31.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:31 smithi100 ceph-mon[30296]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a355f9a8-91a3-41d4-814d-5a9c9b953706"}]: dispatch 2024-05-10T11:12:31.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:31 smithi100 ceph-mon[30296]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a355f9a8-91a3-41d4-814d-5a9c9b953706"}]': finished 2024-05-10T11:12:31.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:31 smithi100 ceph-mon[30296]: osdmap e63: 11 total, 10 up, 11 in 2024-05-10T11:12:31.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:31 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T11:12:32.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:32 smithi178 ceph-mon[33359]: from='client.? 172.21.15.178:0/530151187' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T11:12:32.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:32 smithi188 ceph-mon[34075]: from='client.? 172.21.15.178:0/530151187' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T11:12:32.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:32 smithi100 ceph-mon[30296]: from='client.? 172.21.15.178:0/530151187' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T11:12:33.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:33 smithi178 ceph-mon[33359]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-10T11:12:33.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:33 smithi188 ceph-mon[34075]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-10T11:12:33.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:33 smithi100 ceph-mon[30296]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-10T11:12:35.618 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:35 smithi178 ceph-mon[33359]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-10T11:12:35.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:35 smithi188 ceph-mon[34075]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-10T11:12:35.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:35 smithi100 ceph-mon[30296]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-10T11:12:37.632 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:37 smithi178 ceph-mon[33359]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-10T11:12:37.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:37 smithi188 ceph-mon[34075]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-10T11:12:37.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:37 smithi100 ceph-mon[30296]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-10T11:12:39.695 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:39 smithi178 ceph-mon[33359]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-10T11:12:39.695 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:39 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-10T11:12:39.695 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:39 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:12:39.695 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:39 smithi178 ceph-mon[33359]: Deploying daemon osd.10 on smithi178 2024-05-10T11:12:39.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:39 smithi188 ceph-mon[34075]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-10T11:12:39.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:39 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-10T11:12:39.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:39 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:12:39.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:39 smithi188 ceph-mon[34075]: Deploying daemon osd.10 on smithi178 2024-05-10T11:12:39.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:39 smithi100 ceph-mon[30296]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-10T11:12:39.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:39 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-10T11:12:39.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:39 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:12:39.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:39 smithi100 ceph-mon[30296]: Deploying daemon osd.10 on smithi178 2024-05-10T11:12:41.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:41 smithi178 ceph-mon[33359]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-10T11:12:41.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:41 smithi188 ceph-mon[34075]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-10T11:12:41.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:41 smithi100 ceph-mon[30296]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-10T11:12:43.671 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:43 smithi178 ceph-mon[33359]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-10T11:12:43.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:43 smithi188 ceph-mon[34075]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-10T11:12:43.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:43 smithi100 ceph-mon[30296]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-10T11:12:45.828 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:45 smithi178 ceph-mon[33359]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-10T11:12:45.828 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:45 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:12:45.828 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:45 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:12:45.828 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:45 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:12:46.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:45 smithi188 ceph-mon[34075]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-10T11:12:46.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:45 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:12:46.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:45 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:12:46.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:45 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:12:46.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:45 smithi100 ceph-mon[30296]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-10T11:12:46.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:45 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:12:46.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:45 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:12:46.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:45 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:12:47.894 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:47 smithi178 ceph-mon[33359]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-10T11:12:48.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:47 smithi188 ceph-mon[34075]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-10T11:12:48.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:47 smithi100 ceph-mon[30296]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-10T11:12:48.266 INFO:teuthology.orchestra.run.smithi178.stdout:Created osd(s) 10 on host 'smithi178' 2024-05-10T11:12:48.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:12:48.265+0000 7f7f9a7fc700 1 -- 172.21.15.178:0/4063797035 <== mgr.14152 v2:172.21.15.100:6800/2507287356 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f7fa40638e0 con 0x7f7f900b2880 2024-05-10T11:12:48.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:12:48.268+0000 7f7faba45700 1 -- 172.21.15.178:0/4063797035 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f7f900b2880 msgr2=0x7f7f900b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:12:48.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:12:48.268+0000 7f7faba45700 1 --2- 172.21.15.178:0/4063797035 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f7f900b2880 0x7f7f900b4d40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7f94006010 tx=0x7f7f94019040 comp rx=0 tx=0).stop 2024-05-10T11:12:48.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:12:48.268+0000 7f7faba45700 1 -- 172.21.15.178:0/4063797035 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fa4103fb0 msgr2=0x7f7fa41a0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:12:48.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:12:48.268+0000 7f7faba45700 1 --2- 172.21.15.178:0/4063797035 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fa4103fb0 0x7f7fa41a0b90 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c00b970 tx=0x7f7f9c00bd30 comp rx=0 tx=0).stop 2024-05-10T11:12:48.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:12:48.268+0000 7f7faba45700 1 -- 172.21.15.178:0/4063797035 shutdown_connections 2024-05-10T11:12:48.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:12:48.268+0000 7f7faba45700 1 --2- 172.21.15.178:0/4063797035 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f7f900b2880 0x7f7f900b4d40 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:12:48.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:12:48.268+0000 7f7faba45700 1 --2- 172.21.15.178:0/4063797035 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fa4103fb0 0x7f7fa41a0b90 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:12:48.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:12:48.268+0000 7f7faba45700 1 --2- 172.21.15.178:0/4063797035 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fa4102410 0x7f7fa4199620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:12:48.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:12:48.268+0000 7f7faba45700 1 --2- 172.21.15.178:0/4063797035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fa4103610 0x7f7fa4199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:12:48.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:12:48.268+0000 7f7faba45700 1 -- 172.21.15.178:0/4063797035 >> 172.21.15.178:0/4063797035 conn(0x7f7fa40fd9d0 msgr2=0x7f7fa4106a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:12:48.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:12:48.268+0000 7f7faba45700 1 -- 172.21.15.178:0/4063797035 shutdown_connections 2024-05-10T11:12:48.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-05-10T11:12:48.268+0000 7f7faba45700 1 -- 172.21.15.178:0/4063797035 wait complete. 2024-05-10T11:12:48.977 DEBUG:teuthology.orchestra.run.smithi178:osd.10> sudo journalctl -f -n 0 -u ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@osd.10.service 2024-05-10T11:12:48.979 INFO:tasks.cephadm:Deploying osd.11 on smithi188 with /dev/vg_nvme/lv_1... 2024-05-10T11:12:48.980 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-10T11:12:49.234 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T11:12:49.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:49 smithi188 ceph-mon[34075]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-10T11:12:49.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:49 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:12:49.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:49 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:12:49.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:49 smithi188 ceph-mon[34075]: from='osd.10 [v2:172.21.15.178:6824/2985841725,v1:172.21.15.178:6825/2985841725]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-10T11:12:49.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:49 smithi188 ceph-mon[34075]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-10T11:12:49.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:49 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:12:49.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:49 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:12:49.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:49 smithi178 ceph-mon[33359]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-10T11:12:49.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:49 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:12:49.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:49 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:12:49.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:49 smithi178 ceph-mon[33359]: from='osd.10 [v2:172.21.15.178:6824/2985841725,v1:172.21.15.178:6825/2985841725]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-10T11:12:49.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:49 smithi178 ceph-mon[33359]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-10T11:12:49.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:49 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:12:49.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:49 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:12:49.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:49 smithi100 ceph-mon[30296]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-10T11:12:49.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:49 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:12:49.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:49 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:12:49.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:49 smithi100 ceph-mon[30296]: from='osd.10 [v2:172.21.15.178:6824/2985841725,v1:172.21.15.178:6825/2985841725]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-10T11:12:49.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:49 smithi100 ceph-mon[30296]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-10T11:12:49.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:49 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:12:49.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:49 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:12:50.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:50 smithi178 ceph-mon[33359]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-10T11:12:50.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:50 smithi178 ceph-mon[33359]: osdmap e64: 11 total, 10 up, 11 in 2024-05-10T11:12:50.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:50 smithi178 ceph-mon[33359]: from='osd.10 [v2:172.21.15.178:6824/2985841725,v1:172.21.15.178:6825/2985841725]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-05-10T11:12:50.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:50 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T11:12:50.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:50 smithi178 ceph-mon[33359]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-05-10T11:12:50.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:50 smithi188 ceph-mon[34075]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-10T11:12:50.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:50 smithi188 ceph-mon[34075]: osdmap e64: 11 total, 10 up, 11 in 2024-05-10T11:12:50.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:50 smithi188 ceph-mon[34075]: from='osd.10 [v2:172.21.15.178:6824/2985841725,v1:172.21.15.178:6825/2985841725]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-05-10T11:12:50.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:50 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T11:12:50.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:50 smithi188 ceph-mon[34075]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-05-10T11:12:50.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:50 smithi100 ceph-mon[30296]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-10T11:12:50.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:50 smithi100 ceph-mon[30296]: osdmap e64: 11 total, 10 up, 11 in 2024-05-10T11:12:50.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:50 smithi100 ceph-mon[30296]: from='osd.10 [v2:172.21.15.178:6824/2985841725,v1:172.21.15.178:6825/2985841725]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-05-10T11:12:50.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:50 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T11:12:50.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:50 smithi100 ceph-mon[30296]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-05-10T11:12:51.003 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:12:50 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:12:50.760+0000 7fbce5f8f700 -1 osd.10 0 waiting for initial osdmap 2024-05-10T11:12:51.003 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:12:50 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:12:50.771+0000 7fbcdd579700 -1 osd.10 65 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T11:12:51.281 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:51 smithi178 ceph-mon[33359]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-10T11:12:51.281 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:51 smithi178 ceph-mon[33359]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-05-10T11:12:51.281 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:51 smithi178 ceph-mon[33359]: osdmap e65: 11 total, 10 up, 11 in 2024-05-10T11:12:51.281 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:51 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T11:12:51.282 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:51 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T11:12:51.282 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:51 smithi178 ceph-mon[33359]: from='osd.10 ' entity='osd.10' 2024-05-10T11:12:51.282 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:51 smithi178 ceph-mon[33359]: osd.10 [v2:172.21.15.178:6824/2985841725,v1:172.21.15.178:6825/2985841725] boot 2024-05-10T11:12:51.282 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:51 smithi178 ceph-mon[33359]: osdmap e66: 11 total, 11 up, 11 in 2024-05-10T11:12:51.282 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:51 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T11:12:51.461 INFO:teuthology.orchestra.run.smithi188.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-10T11:12:51.461 INFO:teuthology.orchestra.run.smithi188.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-10T11:12:51.461 INFO:teuthology.orchestra.run.smithi188.stderr: stderr: 10+0 records in 2024-05-10T11:12:51.461 INFO:teuthology.orchestra.run.smithi188.stderr:10+0 records out 2024-05-10T11:12:51.461 INFO:teuthology.orchestra.run.smithi188.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0130308 s, 805 MB/s 2024-05-10T11:12:51.461 INFO:teuthology.orchestra.run.smithi188.stderr:--> Zapping successful for: 2024-05-10T11:12:51.507 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:51 smithi188 ceph-mon[34075]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-10T11:12:51.507 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:51 smithi188 ceph-mon[34075]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-05-10T11:12:51.507 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:51 smithi188 ceph-mon[34075]: osdmap e65: 11 total, 10 up, 11 in 2024-05-10T11:12:51.507 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:51 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T11:12:51.507 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:51 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T11:12:51.507 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:51 smithi188 ceph-mon[34075]: from='osd.10 ' entity='osd.10' 2024-05-10T11:12:51.508 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:51 smithi188 ceph-mon[34075]: osd.10 [v2:172.21.15.178:6824/2985841725,v1:172.21.15.178:6825/2985841725] boot 2024-05-10T11:12:51.508 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:51 smithi188 ceph-mon[34075]: osdmap e66: 11 total, 11 up, 11 in 2024-05-10T11:12:51.508 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:51 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T11:12:51.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:51 smithi100 ceph-mon[30296]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-10T11:12:51.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:51 smithi100 ceph-mon[30296]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-05-10T11:12:51.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:51 smithi100 ceph-mon[30296]: osdmap e65: 11 total, 10 up, 11 in 2024-05-10T11:12:51.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:51 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T11:12:51.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:51 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T11:12:51.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:51 smithi100 ceph-mon[30296]: from='osd.10 ' entity='osd.10' 2024-05-10T11:12:51.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:51 smithi100 ceph-mon[30296]: osd.10 [v2:172.21.15.178:6824/2985841725,v1:172.21.15.178:6825/2985841725] boot 2024-05-10T11:12:51.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:51 smithi100 ceph-mon[30296]: osdmap e66: 11 total, 11 up, 11 in 2024-05-10T11:12:51.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:51 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T11:12:52.243 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi188:vg_nvme/lv_1 2024-05-10T11:12:52.495 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T11:12:52.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:52 smithi178 ceph-mon[33359]: purged_snaps scrub starts 2024-05-10T11:12:52.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:52 smithi178 ceph-mon[33359]: purged_snaps scrub ok 2024-05-10T11:12:52.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:52 smithi178 ceph-mon[33359]: osdmap e67: 11 total, 11 up, 11 in 2024-05-10T11:12:52.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:52 smithi188 ceph-mon[34075]: purged_snaps scrub starts 2024-05-10T11:12:52.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:52 smithi188 ceph-mon[34075]: purged_snaps scrub ok 2024-05-10T11:12:52.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:52 smithi188 ceph-mon[34075]: osdmap e67: 11 total, 11 up, 11 in 2024-05-10T11:12:52.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:52 smithi100 ceph-mon[30296]: purged_snaps scrub starts 2024-05-10T11:12:52.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:52 smithi100 ceph-mon[30296]: purged_snaps scrub ok 2024-05-10T11:12:52.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:52 smithi100 ceph-mon[30296]: osdmap e67: 11 total, 11 up, 11 in 2024-05-10T11:12:54.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:53 smithi188 ceph-mon[34075]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 735 MiB used, 983 GiB / 983 GiB avail 2024-05-10T11:12:54.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:53 smithi188 ceph-mon[34075]: Detected new or changed devices on smithi178 2024-05-10T11:12:54.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:53 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:12:54.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:53 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:12:54.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:53 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:12:54.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:53 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:12:54.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:53 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:12:54.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:53 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:12:54.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:53 smithi188 ceph-mon[34075]: Adjusting osd_memory_target on smithi178 to 4272M 2024-05-10T11:12:54.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:53 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:12:54.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:53 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:12:54.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:53 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:12:54.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:53 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:12:54.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:53 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:12:54.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:53 smithi188 ceph-mon[34075]: osdmap e68: 11 total, 11 up, 11 in 2024-05-10T11:12:54.030 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:53 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:12:54.030 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:53 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:12:54.030 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:53 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:12:54.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:53 smithi100 ceph-mon[30296]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 735 MiB used, 983 GiB / 983 GiB avail 2024-05-10T11:12:54.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:53 smithi100 ceph-mon[30296]: Detected new or changed devices on smithi178 2024-05-10T11:12:54.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:53 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:12:54.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:53 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:12:54.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:53 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:12:54.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:53 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:12:54.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:53 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:12:54.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:53 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:12:54.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:53 smithi100 ceph-mon[30296]: Adjusting osd_memory_target on smithi178 to 4272M 2024-05-10T11:12:54.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:53 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:12:54.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:53 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:12:54.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:53 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:12:54.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:53 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:12:54.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:53 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:12:54.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:53 smithi100 ceph-mon[30296]: osdmap e68: 11 total, 11 up, 11 in 2024-05-10T11:12:54.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:53 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:12:54.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:53 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:12:54.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:53 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:12:54.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:53 smithi178 ceph-mon[33359]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 735 MiB used, 983 GiB / 983 GiB avail 2024-05-10T11:12:54.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:53 smithi178 ceph-mon[33359]: Detected new or changed devices on smithi178 2024-05-10T11:12:54.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:53 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:12:54.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:53 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:12:54.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:53 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:12:54.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:53 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:12:54.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:53 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:12:54.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:53 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:12:54.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:53 smithi178 ceph-mon[33359]: Adjusting osd_memory_target on smithi178 to 4272M 2024-05-10T11:12:54.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:53 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:12:54.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:53 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:12:54.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:53 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:12:54.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:53 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:12:54.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:53 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:12:54.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:53 smithi178 ceph-mon[33359]: osdmap e68: 11 total, 11 up, 11 in 2024-05-10T11:12:54.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:53 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:12:54.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:53 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:12:54.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:53 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:12:54.270 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:12:54.270+0000 7f4c5faa6700 1 -- 172.21.15.188:0/1680991854 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c58103600 msgr2=0x7f4c58103a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:12:54.270 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:12:54.270+0000 7f4c5faa6700 1 --2- 172.21.15.188:0/1680991854 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c58103600 0x7f4c58103a60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4c54009990 tx=0x7f4c54009ca0 comp rx=0 tx=0).stop 2024-05-10T11:12:54.270 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:12:54.270+0000 7f4c5faa6700 1 -- 172.21.15.188:0/1680991854 shutdown_connections 2024-05-10T11:12:54.271 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:12:54.270+0000 7f4c5faa6700 1 --2- 172.21.15.188:0/1680991854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c58103fa0 0x7f4c5810a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:12:54.271 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:12:54.270+0000 7f4c5faa6700 1 --2- 172.21.15.188:0/1680991854 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c58102400 0x7f4c58102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:12:54.271 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:12:54.270+0000 7f4c5faa6700 1 --2- 172.21.15.188:0/1680991854 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c58103600 0x7f4c58103a60 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:12:54.271 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:12:54.270+0000 7f4c5faa6700 1 -- 172.21.15.188:0/1680991854 >> 172.21.15.188:0/1680991854 conn(0x7f4c580fd9a0 msgr2=0x7f4c580ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:12:54.271 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:12:54.270+0000 7f4c5faa6700 1 -- 172.21.15.188:0/1680991854 shutdown_connections 2024-05-10T11:12:54.271 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:12:54.270+0000 7f4c5faa6700 1 -- 172.21.15.188:0/1680991854 wait complete. 2024-05-10T11:12:54.273 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:12:54.271+0000 7f4c5faa6700 1 Processor -- start 2024-05-10T11:12:54.273 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:12:54.271+0000 7f4c5faa6700 1 -- start start 2024-05-10T11:12:54.273 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:12:54.271+0000 7f4c5faa6700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c58102400 0x7f4c58199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:12:54.273 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:12:54.271+0000 7f4c5faa6700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c58103600 0x7f4c58199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:12:54.273 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:12:54.271+0000 7f4c5faa6700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c58103fa0 0x7f4c581a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:12:54.273 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:12:54.271+0000 7f4c5faa6700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5810dda0 con 0x7f4c58103600 2024-05-10T11:12:54.273 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:12:54.271+0000 7f4c5faa6700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5810dc20 con 0x7f4c58102400 2024-05-10T11:12:54.274 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:12:54.271+0000 7f4c5faa6700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5810df20 con 0x7f4c58103fa0 2024-05-10T11:12:54.274 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:12:54.271+0000 7f4c5e043700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c58103fa0 0x7f4c581a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:12:54.274 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:12:54.271+0000 7f4c5e043700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c58103fa0 0x7f4c581a0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:57530/0 (socket says 172.21.15.188:57530) 2024-05-10T11:12:54.274 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:12:54.271+0000 7f4c5e043700 1 -- 172.21.15.188:0/2394050044 learned_addr learned my addr 172.21.15.188:0/2394050044 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-05-10T11:12:54.274 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:12:54.271+0000 7f4c5d842700 1 --2- 172.21.15.188:0/2394050044 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c58102400 0x7f4c58199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:12:54.274 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:12:54.271+0000 7f4c5d041700 1 --2- 172.21.15.188:0/2394050044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c58103600 0x7f4c58199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:12:54.274 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:12:54.271+0000 7f4c5e043700 1 -- 172.21.15.188:0/2394050044 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c58102400 msgr2=0x7f4c58199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:12:54.274 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:12:54.271+0000 7f4c5e043700 1 --2- 172.21.15.188:0/2394050044 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c58102400 0x7f4c58199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:12:54.274 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:12:54.271+0000 7f4c5e043700 1 -- 172.21.15.188:0/2394050044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c58103600 msgr2=0x7f4c58199b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:12:54.274 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:12:54.271+0000 7f4c5e043700 1 --2- 172.21.15.188:0/2394050044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c58103600 0x7f4c58199b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:12:54.274 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:12:54.271+0000 7f4c5e043700 1 -- 172.21.15.188:0/2394050044 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c581a1260 con 0x7f4c58103fa0 2024-05-10T11:12:54.274 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:12:54.272+0000 7f4c5e043700 1 --2- 172.21.15.188:0/2394050044 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c58103fa0 0x7f4c581a0bc0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4c5000add0 tx=0x7f4c5000c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:12:54.275 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:12:54.272+0000 7f4c4effd700 1 -- 172.21.15.188:0/2394050044 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c50017020 con 0x7f4c58103fa0 2024-05-10T11:12:54.275 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:12:54.272+0000 7f4c4effd700 1 -- 172.21.15.188:0/2394050044 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4c5000ae00 con 0x7f4c58103fa0 2024-05-10T11:12:54.275 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:12:54.272+0000 7f4c5faa6700 1 -- 172.21.15.188:0/2394050044 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c581a1550 con 0x7f4c58103fa0 2024-05-10T11:12:54.275 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:12:54.272+0000 7f4c4effd700 1 -- 172.21.15.188:0/2394050044 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c50005350 con 0x7f4c58103fa0 2024-05-10T11:12:54.275 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:12:54.272+0000 7f4c5faa6700 1 -- 172.21.15.188:0/2394050044 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c581a1a40 con 0x7f4c58103fa0 2024-05-10T11:12:54.275 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:12:54.273+0000 7f4c5faa6700 1 -- 172.21.15.188:0/2394050044 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c5806a190 con 0x7f4c58103fa0 2024-05-10T11:12:54.276 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:12:54.274+0000 7f4c4effd700 1 -- 172.21.15.188:0/2394050044 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f4c50004ad0 con 0x7f4c58103fa0 2024-05-10T11:12:54.276 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:12:54.275+0000 7f4c4effd700 1 --2- 172.21.15.188:0/2394050044 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4c440b2880 0x7f4c440b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:12:54.276 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:12:54.275+0000 7f4c5d842700 1 --2- 172.21.15.188:0/2394050044 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4c440b2880 0x7f4c440b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:12:54.276 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:12:54.275+0000 7f4c4effd700 1 -- 172.21.15.188:0/2394050044 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6694+0+0 (secure 0 0 0) 0x7f4c500e18d0 con 0x7f4c58103fa0 2024-05-10T11:12:54.276 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:12:54.275+0000 7f4c5d842700 1 --2- 172.21.15.188:0/2394050044 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4c440b2880 0x7f4c440b4d40 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4c58103460 tx=0x7f4c4800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:12:54.278 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:12:54.278+0000 7f4c4effd700 1 -- 172.21.15.188:0/2394050044 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f4c50021020 con 0x7f4c58103fa0 2024-05-10T11:12:54.488 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:12:54.488+0000 7f4c5faa6700 1 -- 172.21.15.188:0/2394050044 --> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f4c5806c2e0 con 0x7f4c440b2880 2024-05-10T11:12:55.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:54 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T11:12:55.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:54 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T11:12:55.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:54 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:12:55.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:54 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T11:12:55.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:54 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T11:12:55.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:54 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:12:55.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:54 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T11:12:55.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:54 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T11:12:55.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:54 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:12:56.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:55 smithi188 ceph-mon[34075]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 735 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-10T11:12:56.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:55 smithi188 ceph-mon[34075]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:12:56.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:55 smithi100 ceph-mon[30296]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 735 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-10T11:12:56.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:55 smithi100 ceph-mon[30296]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:12:56.097 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:55 smithi178 ceph-mon[33359]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 735 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-10T11:12:56.097 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:55 smithi178 ceph-mon[33359]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:12:58.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:57 smithi188 ceph-mon[34075]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 735 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-05-10T11:12:58.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:57 smithi188 ceph-mon[34075]: from='client.? 172.21.15.188:0/2212352537' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "76a5e13c-7e38-42f9-bfa7-7fedec7c7184"}]: dispatch 2024-05-10T11:12:58.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:57 smithi188 ceph-mon[34075]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "76a5e13c-7e38-42f9-bfa7-7fedec7c7184"}]: dispatch 2024-05-10T11:12:58.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:57 smithi188 ceph-mon[34075]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "76a5e13c-7e38-42f9-bfa7-7fedec7c7184"}]': finished 2024-05-10T11:12:58.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:57 smithi188 ceph-mon[34075]: osdmap e69: 12 total, 11 up, 12 in 2024-05-10T11:12:58.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:57 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T11:12:58.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:57 smithi100 ceph-mon[30296]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 735 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-05-10T11:12:58.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:57 smithi100 ceph-mon[30296]: from='client.? 172.21.15.188:0/2212352537' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "76a5e13c-7e38-42f9-bfa7-7fedec7c7184"}]: dispatch 2024-05-10T11:12:58.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:57 smithi100 ceph-mon[30296]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "76a5e13c-7e38-42f9-bfa7-7fedec7c7184"}]: dispatch 2024-05-10T11:12:58.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:57 smithi100 ceph-mon[30296]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "76a5e13c-7e38-42f9-bfa7-7fedec7c7184"}]': finished 2024-05-10T11:12:58.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:57 smithi100 ceph-mon[30296]: osdmap e69: 12 total, 11 up, 12 in 2024-05-10T11:12:58.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:57 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T11:12:58.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:57 smithi178 ceph-mon[33359]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 735 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-05-10T11:12:58.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:57 smithi178 ceph-mon[33359]: from='client.? 172.21.15.188:0/2212352537' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "76a5e13c-7e38-42f9-bfa7-7fedec7c7184"}]: dispatch 2024-05-10T11:12:58.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:57 smithi178 ceph-mon[33359]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "76a5e13c-7e38-42f9-bfa7-7fedec7c7184"}]: dispatch 2024-05-10T11:12:58.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:57 smithi178 ceph-mon[33359]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "76a5e13c-7e38-42f9-bfa7-7fedec7c7184"}]': finished 2024-05-10T11:12:58.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:57 smithi178 ceph-mon[33359]: osdmap e69: 12 total, 11 up, 12 in 2024-05-10T11:12:58.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:57 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T11:12:59.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:58 smithi178 ceph-mon[33359]: from='client.? 172.21.15.188:0/2139916680' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T11:12:59.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:58 smithi188 ceph-mon[34075]: from='client.? 172.21.15.188:0/2139916680' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T11:12:59.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:58 smithi100 ceph-mon[30296]: from='client.? 172.21.15.188:0/2139916680' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T11:13:00.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:12:59 smithi178 ceph-mon[33359]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-05-10T11:13:00.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:12:59 smithi188 ceph-mon[34075]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-05-10T11:13:00.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:12:59 smithi100 ceph-mon[30296]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-05-10T11:13:02.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:01 smithi188 ceph-mon[34075]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-10T11:13:02.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:01 smithi178 ceph-mon[33359]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-10T11:13:02.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:01 smithi100 ceph-mon[30296]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-10T11:13:04.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:03 smithi188 ceph-mon[34075]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-05-10T11:13:04.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:03 smithi178 ceph-mon[33359]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-05-10T11:13:04.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:03 smithi100 ceph-mon[30296]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-05-10T11:13:06.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:05 smithi188 ceph-mon[34075]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-05-10T11:13:06.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:05 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-10T11:13:06.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:05 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:13:06.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:05 smithi188 ceph-mon[34075]: Deploying daemon osd.11 on smithi188 2024-05-10T11:13:06.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:05 smithi178 ceph-mon[33359]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-05-10T11:13:06.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:05 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-10T11:13:06.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:05 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:13:06.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:05 smithi178 ceph-mon[33359]: Deploying daemon osd.11 on smithi188 2024-05-10T11:13:06.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:05 smithi100 ceph-mon[30296]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-05-10T11:13:06.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:05 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-10T11:13:06.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:05 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:13:06.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:05 smithi100 ceph-mon[30296]: Deploying daemon osd.11 on smithi188 2024-05-10T11:13:07.978 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:07 smithi188 ceph-mon[34075]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-05-10T11:13:08.090 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:07 smithi100 ceph-mon[30296]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-05-10T11:13:08.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:07 smithi178 ceph-mon[33359]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-05-10T11:13:10.222 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:09 smithi188 ceph-mon[34075]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-05-10T11:13:10.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:09 smithi178 ceph-mon[33359]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-05-10T11:13:10.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:09 smithi100 ceph-mon[30296]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-05-10T11:13:12.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:11 smithi178 ceph-mon[33359]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-05-10T11:13:12.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:11 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:12.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:11 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:12.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:11 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:13:12.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:11 smithi188 ceph-mon[34075]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-05-10T11:13:12.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:11 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:12.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:11 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:12.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:11 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:13:12.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:11 smithi100 ceph-mon[30296]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-05-10T11:13:12.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:11 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:12.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:11 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:12.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:11 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:13:14.187 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:13 smithi188 ceph-mon[34075]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-05-10T11:13:14.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:13 smithi178 ceph-mon[33359]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-05-10T11:13:14.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:13 smithi100 ceph-mon[30296]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-05-10T11:13:14.505 INFO:teuthology.orchestra.run.smithi188.stdout:Created osd(s) 11 on host 'smithi188' 2024-05-10T11:13:14.506 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:13:14.505+0000 7f4c4effd700 1 -- 172.21.15.188:0/2394050044 <== mgr.14152 v2:172.21.15.100:6800/2507287356 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f4c5806c2e0 con 0x7f4c440b2880 2024-05-10T11:13:14.509 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:13:14.508+0000 7f4c5faa6700 1 -- 172.21.15.188:0/2394050044 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4c440b2880 msgr2=0x7f4c440b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:13:14.509 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:13:14.508+0000 7f4c5faa6700 1 --2- 172.21.15.188:0/2394050044 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4c440b2880 0x7f4c440b4d40 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4c58103460 tx=0x7f4c4800b410 comp rx=0 tx=0).stop 2024-05-10T11:13:14.509 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:13:14.508+0000 7f4c5faa6700 1 -- 172.21.15.188:0/2394050044 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c58103fa0 msgr2=0x7f4c581a0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:13:14.509 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:13:14.508+0000 7f4c5faa6700 1 --2- 172.21.15.188:0/2394050044 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c58103fa0 0x7f4c581a0bc0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4c5000add0 tx=0x7f4c5000c330 comp rx=0 tx=0).stop 2024-05-10T11:13:14.509 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:13:14.508+0000 7f4c5faa6700 1 -- 172.21.15.188:0/2394050044 shutdown_connections 2024-05-10T11:13:14.509 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:13:14.508+0000 7f4c5faa6700 1 --2- 172.21.15.188:0/2394050044 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4c440b2880 0x7f4c440b4d40 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:14.509 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:13:14.508+0000 7f4c5faa6700 1 --2- 172.21.15.188:0/2394050044 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c58103fa0 0x7f4c581a0bc0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:14.509 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:13:14.508+0000 7f4c5faa6700 1 --2- 172.21.15.188:0/2394050044 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c58102400 0x7f4c58199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:14.509 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:13:14.508+0000 7f4c5faa6700 1 --2- 172.21.15.188:0/2394050044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c58103600 0x7f4c58199b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:14.509 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:13:14.508+0000 7f4c5faa6700 1 -- 172.21.15.188:0/2394050044 >> 172.21.15.188:0/2394050044 conn(0x7f4c580fd9a0 msgr2=0x7f4c5806bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:13:14.509 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:13:14.509+0000 7f4c5faa6700 1 -- 172.21.15.188:0/2394050044 shutdown_connections 2024-05-10T11:13:14.509 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:13:14.509+0000 7f4c5faa6700 1 -- 172.21.15.188:0/2394050044 wait complete. 2024-05-10T11:13:15.156 DEBUG:teuthology.orchestra.run.smithi188:osd.11> sudo journalctl -f -n 0 -u ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@osd.11.service 2024-05-10T11:13:15.159 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-05-10T11:13:15.159 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph osd stat -f json 2024-05-10T11:13:15.417 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:13:15.701 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:15 smithi100 ceph-mon[30296]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-05-10T11:13:15.702 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:15 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:15.702 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:15 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:15.702 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:15 smithi100 ceph-mon[30296]: from='osd.11 [v2:172.21.15.188:6824/2574955118,v1:172.21.15.188:6825/2574955118]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-10T11:13:15.702 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:15 smithi100 ceph-mon[30296]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-10T11:13:15.702 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:15 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:15.702 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:15 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:15.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:15 smithi178 ceph-mon[33359]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-05-10T11:13:15.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:15 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:15.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:15 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:15.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:15 smithi178 ceph-mon[33359]: from='osd.11 [v2:172.21.15.188:6824/2574955118,v1:172.21.15.188:6825/2574955118]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-10T11:13:15.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:15 smithi178 ceph-mon[33359]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-10T11:13:15.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:15 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:15.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:15 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:15.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:15 smithi188 ceph-mon[34075]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-05-10T11:13:15.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:15 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:15.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:15 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:15.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:15 smithi188 ceph-mon[34075]: from='osd.11 [v2:172.21.15.188:6824/2574955118,v1:172.21.15.188:6825/2574955118]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-10T11:13:15.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:15 smithi188 ceph-mon[34075]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-10T11:13:15.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:15 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:15.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:15 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:16.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:16 smithi178 ceph-mon[33359]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-10T11:13:16.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:16 smithi178 ceph-mon[33359]: osdmap e70: 12 total, 11 up, 12 in 2024-05-10T11:13:16.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:16 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T11:13:16.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:16 smithi178 ceph-mon[33359]: from='osd.11 [v2:172.21.15.188:6824/2574955118,v1:172.21.15.188:6825/2574955118]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-05-10T11:13:16.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:16 smithi178 ceph-mon[33359]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-05-10T11:13:16.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:16 smithi188 ceph-mon[34075]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-10T11:13:16.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:16 smithi188 ceph-mon[34075]: osdmap e70: 12 total, 11 up, 12 in 2024-05-10T11:13:16.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:16 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T11:13:16.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:16 smithi188 ceph-mon[34075]: from='osd.11 [v2:172.21.15.188:6824/2574955118,v1:172.21.15.188:6825/2574955118]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-05-10T11:13:16.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:16 smithi188 ceph-mon[34075]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-05-10T11:13:16.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:16 smithi100 ceph-mon[30296]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-10T11:13:16.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:16 smithi100 ceph-mon[30296]: osdmap e70: 12 total, 11 up, 12 in 2024-05-10T11:13:16.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:16 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T11:13:16.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:16 smithi100 ceph-mon[30296]: from='osd.11 [v2:172.21.15.188:6824/2574955118,v1:172.21.15.188:6825/2574955118]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-05-10T11:13:16.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:16 smithi100 ceph-mon[30296]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-05-10T11:13:17.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:17.189+0000 7f4489d85700 1 -- 172.21.15.100:0/2622384938 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4484103770 msgr2=0x7f448410a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:13:17.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:17.189+0000 7f4489d85700 1 --2- 172.21.15.100:0/2622384938 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4484103770 0x7f448410a020 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f447c00b0d0 tx=0x7f447c00b3e0 comp rx=0 tx=0).stop 2024-05-10T11:13:17.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:17.189+0000 7f4489d85700 1 -- 172.21.15.100:0/2622384938 shutdown_connections 2024-05-10T11:13:17.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:17.189+0000 7f4489d85700 1 --2- 172.21.15.100:0/2622384938 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4484103770 0x7f448410a020 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:17.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:17.189+0000 7f4489d85700 1 --2- 172.21.15.100:0/2622384938 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4484101bd0 0x7f4484101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:17.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:17.189+0000 7f4489d85700 1 --2- 172.21.15.100:0/2622384938 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4484102dd0 0x7f4484103230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:17.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:17.189+0000 7f4489d85700 1 -- 172.21.15.100:0/2622384938 >> 172.21.15.100:0/2622384938 conn(0x7f44840fd170 msgr2=0x7f44840ff5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:13:17.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:17.189+0000 7f4489d85700 1 -- 172.21.15.100:0/2622384938 shutdown_connections 2024-05-10T11:13:17.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:17.189+0000 7f4489d85700 1 -- 172.21.15.100:0/2622384938 wait complete. 2024-05-10T11:13:17.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:17.189+0000 7f4489d85700 1 Processor -- start 2024-05-10T11:13:17.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:17.190+0000 7f4489d85700 1 -- start start 2024-05-10T11:13:17.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:17.190+0000 7f4489d85700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4484101bd0 0x7f44841995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:13:17.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:17.190+0000 7f4489d85700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4484102dd0 0x7f4484199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:13:17.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:17.190+0000 7f4489d85700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4484103770 0x7f44841a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:13:17.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:17.190+0000 7f4489d85700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f448410ccd0 con 0x7f4484101bd0 2024-05-10T11:13:17.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:17.190+0000 7f4489d85700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f448410cb50 con 0x7f4484102dd0 2024-05-10T11:13:17.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:17.190+0000 7f4489d85700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f448410ce50 con 0x7f4484103770 2024-05-10T11:13:17.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:17.190+0000 7f44837fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4484101bd0 0x7f44841995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:13:17.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:17.190+0000 7f44837fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4484101bd0 0x7f44841995e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43398/0 (socket says 172.21.15.100:43398) 2024-05-10T11:13:17.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:17.190+0000 7f4482ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4484102dd0 0x7f4484199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:13:17.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:17.190+0000 7f44837fe700 1 -- 172.21.15.100:0/1567826855 learned_addr learned my addr 172.21.15.100:0/1567826855 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:13:17.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:17.190+0000 7f4483fff700 1 --2- 172.21.15.100:0/1567826855 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4484103770 0x7f44841a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:13:17.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:17.190+0000 7f44837fe700 1 -- 172.21.15.100:0/1567826855 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4484103770 msgr2=0x7f44841a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:13:17.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:17.190+0000 7f44837fe700 1 --2- 172.21.15.100:0/1567826855 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4484103770 0x7f44841a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:17.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:17.190+0000 7f44837fe700 1 -- 172.21.15.100:0/1567826855 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4484102dd0 msgr2=0x7f4484199b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:13:17.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:17.190+0000 7f44837fe700 1 --2- 172.21.15.100:0/1567826855 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4484102dd0 0x7f4484199b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:17.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:17.190+0000 7f44837fe700 1 -- 172.21.15.100:0/1567826855 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44841a1240 con 0x7f4484101bd0 2024-05-10T11:13:17.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:17.190+0000 7f4482ffd700 1 --2- 172.21.15.100:0/1567826855 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4484102dd0 0x7f4484199b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:13:17.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:17.190+0000 7f44837fe700 1 --2- 172.21.15.100:0/1567826855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4484101bd0 0x7f44841995e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f446c00d9b0 tx=0x7f446c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:13:17.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:17.190+0000 7f4480ff9700 1 -- 172.21.15.100:0/1567826855 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f446c016070 con 0x7f4484101bd0 2024-05-10T11:13:17.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:17.190+0000 7f4489d85700 1 -- 172.21.15.100:0/1567826855 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44841a1530 con 0x7f4484101bd0 2024-05-10T11:13:17.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:17.190+0000 7f4480ff9700 1 -- 172.21.15.100:0/1567826855 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f446c00bde0 con 0x7f4484101bd0 2024-05-10T11:13:17.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:17.190+0000 7f4480ff9700 1 -- 172.21.15.100:0/1567826855 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f446c010d70 con 0x7f4484101bd0 2024-05-10T11:13:17.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:17.191+0000 7f4489d85700 1 -- 172.21.15.100:0/1567826855 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44841a1a70 con 0x7f4484101bd0 2024-05-10T11:13:17.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:17.191+0000 7f4489d85700 1 -- 172.21.15.100:0/1567826855 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f448406a190 con 0x7f4484101bd0 2024-05-10T11:13:17.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:17.193+0000 7f4480ff9700 1 -- 172.21.15.100:0/1567826855 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f446c019070 con 0x7f4484101bd0 2024-05-10T11:13:17.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:17.194+0000 7f4480ff9700 1 --2- 172.21.15.100:0/1567826855 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f44700b2880 0x7f44700b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:13:17.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:17.194+0000 7f4482ffd700 1 --2- 172.21.15.100:0/1567826855 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f44700b2880 0x7f44700b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:13:17.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:17.194+0000 7f4480ff9700 1 -- 172.21.15.100:0/1567826855 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6847+0+0 (secure 0 0 0) 0x7f446c0e2010 con 0x7f4484101bd0 2024-05-10T11:13:17.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:17.194+0000 7f4482ffd700 1 --2- 172.21.15.100:0/1567826855 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f44700b2880 0x7f44700b4d40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f447400e4a0 tx=0x7f4474010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:13:17.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:17.197+0000 7f4480ff9700 1 -- 172.21.15.100:0/1567826855 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f446c020020 con 0x7f4484101bd0 2024-05-10T11:13:17.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:13:17 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:13:17.182+0000 7f872e35c700 -1 osd.11 0 waiting for initial osdmap 2024-05-10T11:13:17.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:13:17 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:13:17.196+0000 7f8726147700 -1 osd.11 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T11:13:17.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:17.398+0000 7f4489d85700 1 -- 172.21.15.100:0/1567826855 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f44841a1f40 con 0x7f4484101bd0 2024-05-10T11:13:17.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:17.399+0000 7f4480ff9700 1 -- 172.21.15.100:0/1567826855 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v71) ==== 74+0+133 (secure 0 0 0) 0x7f446c055e20 con 0x7f4484101bd0 2024-05-10T11:13:17.399 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:13:17.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:17.401+0000 7f4489d85700 1 -- 172.21.15.100:0/1567826855 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f44700b2880 msgr2=0x7f44700b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:13:17.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:17.401+0000 7f4489d85700 1 --2- 172.21.15.100:0/1567826855 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f44700b2880 0x7f44700b4d40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f447400e4a0 tx=0x7f4474010040 comp rx=0 tx=0).stop 2024-05-10T11:13:17.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:17.401+0000 7f4489d85700 1 -- 172.21.15.100:0/1567826855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4484101bd0 msgr2=0x7f44841995e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:13:17.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:17.401+0000 7f4489d85700 1 --2- 172.21.15.100:0/1567826855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4484101bd0 0x7f44841995e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f446c00d9b0 tx=0x7f446c00dcc0 comp rx=0 tx=0).stop 2024-05-10T11:13:17.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:17.401+0000 7f4489d85700 1 -- 172.21.15.100:0/1567826855 shutdown_connections 2024-05-10T11:13:17.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:17.401+0000 7f4489d85700 1 --2- 172.21.15.100:0/1567826855 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f44700b2880 0x7f44700b4d40 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:17.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:17.401+0000 7f4489d85700 1 --2- 172.21.15.100:0/1567826855 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4484103770 0x7f44841a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:17.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:17.401+0000 7f4489d85700 1 --2- 172.21.15.100:0/1567826855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4484101bd0 0x7f44841995e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:17.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:17.401+0000 7f4489d85700 1 --2- 172.21.15.100:0/1567826855 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4484102dd0 0x7f4484199b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:17.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:17.401+0000 7f4489d85700 1 -- 172.21.15.100:0/1567826855 >> 172.21.15.100:0/1567826855 conn(0x7f44840fd170 msgr2=0x7f4484108280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:13:17.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:17.401+0000 7f4489d85700 1 -- 172.21.15.100:0/1567826855 shutdown_connections 2024-05-10T11:13:17.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:17.401+0000 7f4489d85700 1 -- 172.21.15.100:0/1567826855 wait complete. 2024-05-10T11:13:17.572 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:17 smithi188 ceph-mon[34075]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-05-10T11:13:17.572 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:17 smithi188 ceph-mon[34075]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-05-10T11:13:17.572 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:17 smithi188 ceph-mon[34075]: osdmap e71: 12 total, 11 up, 12 in 2024-05-10T11:13:17.572 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:17 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T11:13:17.572 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:17 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T11:13:17.572 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:17 smithi188 ceph-mon[34075]: from='osd.11 ' entity='osd.11' 2024-05-10T11:13:17.572 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:17 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/1567826855' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-10T11:13:17.754 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:17 smithi100 ceph-mon[30296]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-05-10T11:13:17.755 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:17 smithi100 ceph-mon[30296]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-05-10T11:13:17.755 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:17 smithi100 ceph-mon[30296]: osdmap e71: 12 total, 11 up, 12 in 2024-05-10T11:13:17.755 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:17 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T11:13:17.755 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:17 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T11:13:17.755 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:17 smithi100 ceph-mon[30296]: from='osd.11 ' entity='osd.11' 2024-05-10T11:13:17.755 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:17 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/1567826855' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-10T11:13:17.759 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:17 smithi178 ceph-mon[33359]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-05-10T11:13:17.759 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:17 smithi178 ceph-mon[33359]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-05-10T11:13:17.759 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:17 smithi178 ceph-mon[33359]: osdmap e71: 12 total, 11 up, 12 in 2024-05-10T11:13:17.759 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:17 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T11:13:17.759 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:17 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T11:13:17.759 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:17 smithi178 ceph-mon[33359]: from='osd.11 ' entity='osd.11' 2024-05-10T11:13:17.759 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:17 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/1567826855' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-10T11:13:18.068 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":71,"num_osds":12,"num_up_osds":11,"osd_up_since":1715339571,"num_in_osds":12,"osd_in_since":1715339577,"num_remapped_pgs":0} 2024-05-10T11:13:18.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:18 smithi188 ceph-mon[34075]: purged_snaps scrub starts 2024-05-10T11:13:18.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:18 smithi188 ceph-mon[34075]: purged_snaps scrub ok 2024-05-10T11:13:18.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:18 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T11:13:18.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:18 smithi188 ceph-mon[34075]: osd.11 [v2:172.21.15.188:6824/2574955118,v1:172.21.15.188:6825/2574955118] boot 2024-05-10T11:13:18.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:18 smithi188 ceph-mon[34075]: osdmap e72: 12 total, 12 up, 12 in 2024-05-10T11:13:18.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:18 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T11:13:18.754 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:18 smithi178 ceph-mon[33359]: purged_snaps scrub starts 2024-05-10T11:13:18.754 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:18 smithi178 ceph-mon[33359]: purged_snaps scrub ok 2024-05-10T11:13:18.754 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:18 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T11:13:18.754 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:18 smithi178 ceph-mon[33359]: osd.11 [v2:172.21.15.188:6824/2574955118,v1:172.21.15.188:6825/2574955118] boot 2024-05-10T11:13:18.754 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:18 smithi178 ceph-mon[33359]: osdmap e72: 12 total, 12 up, 12 in 2024-05-10T11:13:18.754 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:18 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T11:13:18.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:18 smithi100 ceph-mon[30296]: purged_snaps scrub starts 2024-05-10T11:13:18.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:18 smithi100 ceph-mon[30296]: purged_snaps scrub ok 2024-05-10T11:13:18.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:18 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T11:13:18.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:18 smithi100 ceph-mon[30296]: osd.11 [v2:172.21.15.188:6824/2574955118,v1:172.21.15.188:6825/2574955118] boot 2024-05-10T11:13:18.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:18 smithi100 ceph-mon[30296]: osdmap e72: 12 total, 12 up, 12 in 2024-05-10T11:13:18.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:18 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T11:13:19.070 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph osd stat -f json 2024-05-10T11:13:19.316 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:13:20.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:19 smithi100 ceph-mon[30296]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-05-10T11:13:20.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:19 smithi100 ceph-mon[30296]: Detected new or changed devices on smithi188 2024-05-10T11:13:20.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:19 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:20.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:19 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:20.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:19 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:13:20.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:19 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:13:20.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:19 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:13:20.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:19 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:13:20.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:19 smithi100 ceph-mon[30296]: Adjusting osd_memory_target on smithi188 to 4272M 2024-05-10T11:13:20.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:19 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:20.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:19 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:13:20.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:19 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:13:20.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:19 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:20.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:19 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:13:20.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:19 smithi100 ceph-mon[30296]: osdmap e73: 12 total, 12 up, 12 in 2024-05-10T11:13:20.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:19 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:13:20.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:19 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:13:20.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:19 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:20.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:19 smithi178 ceph-mon[33359]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-05-10T11:13:20.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:19 smithi178 ceph-mon[33359]: Detected new or changed devices on smithi188 2024-05-10T11:13:20.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:19 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:20.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:19 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:20.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:19 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:13:20.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:19 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:13:20.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:19 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:13:20.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:19 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:13:20.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:19 smithi178 ceph-mon[33359]: Adjusting osd_memory_target on smithi188 to 4272M 2024-05-10T11:13:20.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:19 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:20.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:19 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:13:20.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:19 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:13:20.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:19 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:20.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:19 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:13:20.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:19 smithi178 ceph-mon[33359]: osdmap e73: 12 total, 12 up, 12 in 2024-05-10T11:13:20.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:19 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:13:20.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:19 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:13:20.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:19 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:20.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:19 smithi188 ceph-mon[34075]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-05-10T11:13:20.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:19 smithi188 ceph-mon[34075]: Detected new or changed devices on smithi188 2024-05-10T11:13:20.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:19 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:20.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:19 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:20.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:19 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:13:20.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:19 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:13:20.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:19 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:13:20.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:19 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:13:20.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:19 smithi188 ceph-mon[34075]: Adjusting osd_memory_target on smithi188 to 4272M 2024-05-10T11:13:20.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:19 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:20.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:19 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:13:20.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:19 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:13:20.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:19 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:20.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:19 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:13:20.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:19 smithi188 ceph-mon[34075]: osdmap e73: 12 total, 12 up, 12 in 2024-05-10T11:13:20.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:19 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:13:20.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:19 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:13:20.280 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:19 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:21.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:20.996+0000 7facaaaf2700 1 -- 172.21.15.100:0/320752858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faca4105090 msgr2=0x7faca410b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:13:21.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:20.996+0000 7facaaaf2700 1 --2- 172.21.15.100:0/320752858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faca4105090 0x7faca410b940 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fac9c0099c0 tx=0x7fac9c009cd0 comp rx=0 tx=0).stop 2024-05-10T11:13:21.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:20.996+0000 7facaaaf2700 1 -- 172.21.15.100:0/320752858 shutdown_connections 2024-05-10T11:13:21.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:20.996+0000 7facaaaf2700 1 --2- 172.21.15.100:0/320752858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faca4105090 0x7faca410b940 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:21.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:20.996+0000 7facaaaf2700 1 --2- 172.21.15.100:0/320752858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faca41034f0 0x7faca4103910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:21.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:20.996+0000 7facaaaf2700 1 --2- 172.21.15.100:0/320752858 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faca41046f0 0x7faca4104b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:21.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:20.996+0000 7facaaaf2700 1 -- 172.21.15.100:0/320752858 >> 172.21.15.100:0/320752858 conn(0x7faca40fea90 msgr2=0x7faca4100ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:13:21.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:20.996+0000 7facaaaf2700 1 -- 172.21.15.100:0/320752858 shutdown_connections 2024-05-10T11:13:21.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:20.996+0000 7facaaaf2700 1 -- 172.21.15.100:0/320752858 wait complete. 2024-05-10T11:13:21.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:20.997+0000 7facaaaf2700 1 Processor -- start 2024-05-10T11:13:21.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:20.997+0000 7facaaaf2700 1 -- start start 2024-05-10T11:13:21.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:20.997+0000 7facaaaf2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faca41034f0 0x7faca41995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:13:21.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:20.997+0000 7facaaaf2700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faca41046f0 0x7faca4199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:13:21.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:20.997+0000 7facaaaf2700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faca4105090 0x7faca41a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:13:21.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:20.997+0000 7facaaaf2700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7faca410ef30 con 0x7faca41034f0 2024-05-10T11:13:21.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:20.997+0000 7facaaaf2700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7faca410edb0 con 0x7faca4105090 2024-05-10T11:13:21.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:20.997+0000 7facaaaf2700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7faca410f0b0 con 0x7faca41046f0 2024-05-10T11:13:21.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:20.997+0000 7faca888e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faca41034f0 0x7faca41995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:13:21.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:20.997+0000 7faca888e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faca41034f0 0x7faca41995d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43410/0 (socket says 172.21.15.100:43410) 2024-05-10T11:13:21.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:20.997+0000 7faca888e700 1 -- 172.21.15.100:0/2368367590 learned_addr learned my addr 172.21.15.100:0/2368367590 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:13:21.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:20.997+0000 7faca3fff700 1 --2- 172.21.15.100:0/2368367590 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faca41046f0 0x7faca4199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:13:21.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:20.997+0000 7faca908f700 1 --2- 172.21.15.100:0/2368367590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faca4105090 0x7faca41a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:13:21.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:20.997+0000 7faca888e700 1 -- 172.21.15.100:0/2368367590 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faca41046f0 msgr2=0x7faca4199b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:13:21.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:20.997+0000 7faca888e700 1 --2- 172.21.15.100:0/2368367590 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faca41046f0 0x7faca4199b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:21.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:20.997+0000 7faca888e700 1 -- 172.21.15.100:0/2368367590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faca4105090 msgr2=0x7faca41a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:13:21.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:20.997+0000 7faca888e700 1 --2- 172.21.15.100:0/2368367590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faca4105090 0x7faca41a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:21.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:20.997+0000 7faca888e700 1 -- 172.21.15.100:0/2368367590 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faca41a1230 con 0x7faca41034f0 2024-05-10T11:13:21.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:20.997+0000 7faca908f700 1 --2- 172.21.15.100:0/2368367590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faca4105090 0x7faca41a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:13:21.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:20.998+0000 7faca888e700 1 --2- 172.21.15.100:0/2368367590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faca41034f0 0x7faca41995d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fac8c00b530 tx=0x7fac8c00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:13:21.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:20.998+0000 7faca1ffb700 1 -- 172.21.15.100:0/2368367590 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac8c015020 con 0x7faca41034f0 2024-05-10T11:13:21.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:20.998+0000 7faca1ffb700 1 -- 172.21.15.100:0/2368367590 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fac8c011cc0 con 0x7faca41034f0 2024-05-10T11:13:21.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:20.998+0000 7facaaaf2700 1 -- 172.21.15.100:0/2368367590 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faca41a1520 con 0x7faca41034f0 2024-05-10T11:13:21.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:20.998+0000 7faca1ffb700 1 -- 172.21.15.100:0/2368367590 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac8c010ca0 con 0x7faca41034f0 2024-05-10T11:13:21.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:20.998+0000 7facaaaf2700 1 -- 172.21.15.100:0/2368367590 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faca41a1a60 con 0x7faca41034f0 2024-05-10T11:13:21.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:20.999+0000 7facaaaf2700 1 -- 172.21.15.100:0/2368367590 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faca406a190 con 0x7faca41034f0 2024-05-10T11:13:21.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:21.000+0000 7faca1ffb700 1 -- 172.21.15.100:0/2368367590 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fac8c018020 con 0x7faca41034f0 2024-05-10T11:13:21.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:21.001+0000 7faca1ffb700 1 --2- 172.21.15.100:0/2368367590 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fac900b2880 0x7fac900b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:13:21.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:21.001+0000 7faca3fff700 1 --2- 172.21.15.100:0/2368367590 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fac900b2880 0x7fac900b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:13:21.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:21.001+0000 7faca1ffb700 1 -- 172.21.15.100:0/2368367590 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7fac8c0e1560 con 0x7faca41034f0 2024-05-10T11:13:21.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:21.001+0000 7faca3fff700 1 --2- 172.21.15.100:0/2368367590 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fac900b2880 0x7fac900b4d40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fac94009f20 tx=0x7fac940094f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:13:21.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:21.005+0000 7faca1ffb700 1 -- 172.21.15.100:0/2368367590 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fac8c01f020 con 0x7faca41034f0 2024-05-10T11:13:21.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:21.209+0000 7facaaaf2700 1 -- 172.21.15.100:0/2368367590 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7faca402e590 con 0x7faca41034f0 2024-05-10T11:13:21.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:21.209+0000 7faca1ffb700 1 -- 172.21.15.100:0/2368367590 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v73) ==== 74+0+133 (secure 0 0 0) 0x7fac8c0a9720 con 0x7faca41034f0 2024-05-10T11:13:21.209 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:13:21.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:21.211+0000 7facaaaf2700 1 -- 172.21.15.100:0/2368367590 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fac900b2880 msgr2=0x7fac900b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:13:21.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:21.211+0000 7facaaaf2700 1 --2- 172.21.15.100:0/2368367590 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fac900b2880 0x7fac900b4d40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fac94009f20 tx=0x7fac940094f0 comp rx=0 tx=0).stop 2024-05-10T11:13:21.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:21.211+0000 7facaaaf2700 1 -- 172.21.15.100:0/2368367590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faca41034f0 msgr2=0x7faca41995d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:13:21.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:21.211+0000 7facaaaf2700 1 --2- 172.21.15.100:0/2368367590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faca41034f0 0x7faca41995d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fac8c00b530 tx=0x7fac8c00b8f0 comp rx=0 tx=0).stop 2024-05-10T11:13:21.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:21.211+0000 7facaaaf2700 1 -- 172.21.15.100:0/2368367590 shutdown_connections 2024-05-10T11:13:21.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:21.211+0000 7facaaaf2700 1 --2- 172.21.15.100:0/2368367590 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fac900b2880 0x7fac900b4d40 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:21.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:21.211+0000 7facaaaf2700 1 --2- 172.21.15.100:0/2368367590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faca4105090 0x7faca41a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:21.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:21.211+0000 7facaaaf2700 1 --2- 172.21.15.100:0/2368367590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faca41034f0 0x7faca41995d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:21.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:21.211+0000 7facaaaf2700 1 --2- 172.21.15.100:0/2368367590 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faca41046f0 0x7faca4199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:21.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:21.211+0000 7facaaaf2700 1 -- 172.21.15.100:0/2368367590 >> 172.21.15.100:0/2368367590 conn(0x7faca40fea90 msgr2=0x7faca4107b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:13:21.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:21.212+0000 7facaaaf2700 1 -- 172.21.15.100:0/2368367590 shutdown_connections 2024-05-10T11:13:21.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:21.212+0000 7facaaaf2700 1 -- 172.21.15.100:0/2368367590 wait complete. 2024-05-10T11:13:22.040 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":73,"num_osds":12,"num_up_osds":12,"osd_up_since":1715339598,"num_in_osds":12,"osd_in_since":1715339577,"num_remapped_pgs":0} 2024-05-10T11:13:22.040 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-10T11:13:22.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:21 smithi178 ceph-mon[33359]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:13:22.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:21 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/2368367590' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-10T11:13:22.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:21 smithi188 ceph-mon[34075]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:13:22.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:21 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/2368367590' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-10T11:13:22.283 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:13:22.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:21 smithi100 ceph-mon[30296]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:13:22.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:21 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/2368367590' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-10T11:13:24.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:24.049+0000 7f8228ddf700 1 -- 172.21.15.100:0/1793584144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f822406be70 msgr2=0x7f822406c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:13:24.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:24.049+0000 7f8228ddf700 1 --2- 172.21.15.100:0/1793584144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f822406be70 0x7f822406c250 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f82140099c0 tx=0x7f8214009cd0 comp rx=0 tx=0).stop 2024-05-10T11:13:24.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:24.050+0000 7f8228ddf700 1 -- 172.21.15.100:0/1793584144 shutdown_connections 2024-05-10T11:13:24.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:24.050+0000 7f8228ddf700 1 --2- 172.21.15.100:0/1793584144 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82241087c0 0x7f82241103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:24.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:24.050+0000 7f8228ddf700 1 --2- 172.21.15.100:0/1793584144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f822406be70 0x7f822406c250 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:24.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:24.050+0000 7f8228ddf700 1 --2- 172.21.15.100:0/1793584144 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f822406c820 0x7f82241080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:24.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:24.050+0000 7f8228ddf700 1 -- 172.21.15.100:0/1793584144 >> 172.21.15.100:0/1793584144 conn(0x7f82240783f0 msgr2=0x7f8224078800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:13:24.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:24.050+0000 7f8228ddf700 1 -- 172.21.15.100:0/1793584144 shutdown_connections 2024-05-10T11:13:24.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:24.050+0000 7f8228ddf700 1 -- 172.21.15.100:0/1793584144 wait complete. 2024-05-10T11:13:24.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:24.050+0000 7f8228ddf700 1 Processor -- start 2024-05-10T11:13:24.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:24.050+0000 7f8228ddf700 1 -- start start 2024-05-10T11:13:24.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:24.051+0000 7f8228ddf700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f822406be70 0x7f8224102500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:13:24.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:24.051+0000 7f8228ddf700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f822406c820 0x7f8224102a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:13:24.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:24.051+0000 7f8228ddf700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82241087c0 0x7f8224102f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:13:24.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:24.051+0000 7f8228ddf700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8224113560 con 0x7f822406c820 2024-05-10T11:13:24.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:24.051+0000 7f8228ddf700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f82241133e0 con 0x7f822406be70 2024-05-10T11:13:24.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:24.051+0000 7f8228ddf700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f82241136e0 con 0x7f82241087c0 2024-05-10T11:13:24.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:24.051+0000 7f8221d9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f822406c820 0x7f8224102a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:13:24.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:24.051+0000 7f8221d9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f822406c820 0x7f8224102a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43428/0 (socket says 172.21.15.100:43428) 2024-05-10T11:13:24.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:24.051+0000 7f8221d9b700 1 -- 172.21.15.100:0/3050715103 learned_addr learned my addr 172.21.15.100:0/3050715103 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:13:24.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:24.051+0000 7f822259c700 1 --2- 172.21.15.100:0/3050715103 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f822406be70 0x7f8224102500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:13:24.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:24.051+0000 7f8222d9d700 1 --2- 172.21.15.100:0/3050715103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82241087c0 0x7f8224102f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:13:24.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:24.051+0000 7f8221d9b700 1 -- 172.21.15.100:0/3050715103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82241087c0 msgr2=0x7f8224102f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:13:24.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:24.051+0000 7f8221d9b700 1 --2- 172.21.15.100:0/3050715103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82241087c0 0x7f8224102f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:24.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:24.051+0000 7f8221d9b700 1 -- 172.21.15.100:0/3050715103 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f822406be70 msgr2=0x7f8224102500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:13:24.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:24.051+0000 7f8221d9b700 1 --2- 172.21.15.100:0/3050715103 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f822406be70 0x7f8224102500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:24.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:24.051+0000 7f8221d9b700 1 -- 172.21.15.100:0/3050715103 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82241a8010 con 0x7f822406c820 2024-05-10T11:13:24.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:24.051+0000 7f8221d9b700 1 --2- 172.21.15.100:0/3050715103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f822406c820 0x7f8224102a40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f821800d950 tx=0x7f821800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:13:24.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:24.051+0000 7f82137fe700 1 -- 172.21.15.100:0/3050715103 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8218017070 con 0x7f822406c820 2024-05-10T11:13:24.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:24.051+0000 7f82137fe700 1 -- 172.21.15.100:0/3050715103 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8218005020 con 0x7f822406c820 2024-05-10T11:13:24.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:24.051+0000 7f8228ddf700 1 -- 172.21.15.100:0/3050715103 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82241a8300 con 0x7f822406c820 2024-05-10T11:13:24.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:24.052+0000 7f82137fe700 1 -- 172.21.15.100:0/3050715103 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8218009a20 con 0x7f822406c820 2024-05-10T11:13:24.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:24.052+0000 7f8228ddf700 1 -- 172.21.15.100:0/3050715103 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82241a87f0 con 0x7f822406c820 2024-05-10T11:13:24.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:24.052+0000 7f8228ddf700 1 -- 172.21.15.100:0/3050715103 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f822406a190 con 0x7f822406c820 2024-05-10T11:13:24.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:24.054+0000 7f82137fe700 1 -- 172.21.15.100:0/3050715103 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f8218009bc0 con 0x7f822406c820 2024-05-10T11:13:24.060 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:23 smithi100 ceph-mon[30296]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:13:24.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:24.054+0000 7f82137fe700 1 --2- 172.21.15.100:0/3050715103 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f820c0b2880 0x7f820c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:13:24.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:24.055+0000 7f822259c700 1 --2- 172.21.15.100:0/3050715103 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f820c0b2880 0x7f820c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:13:24.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:24.055+0000 7f82137fe700 1 -- 172.21.15.100:0/3050715103 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f82180e1a40 con 0x7f822406c820 2024-05-10T11:13:24.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:24.055+0000 7f822259c700 1 --2- 172.21.15.100:0/3050715103 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f820c0b2880 0x7f820c0b4d40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8214000c00 tx=0x7f8214005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:13:24.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:24.060+0000 7f82137fe700 1 -- 172.21.15.100:0/3050715103 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f82180a9c00 con 0x7f822406c820 2024-05-10T11:13:24.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:23 smithi178 ceph-mon[33359]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:13:24.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:24.270+0000 7f8228ddf700 1 -- 172.21.15.100:0/3050715103 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f822402e510 con 0x7f822406c820 2024-05-10T11:13:24.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:24.271+0000 7f82137fe700 1 -- 172.21.15.100:0/3050715103 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) ==== 74+0+18539 (secure 0 0 0) 0x7f82180a9500 con 0x7f822406c820 2024-05-10T11:13:24.271 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:13:24.271 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":73,"fsid":"20a542cc-0ebd-11ef-bc98-c7b262605968","created":"2024-05-10T11:06:05.454446+0000","modified":"2024-05-10T11:13:19.471994+0000","last_up_change":"2024-05-10T11:13:18.185471+0000","last_in_change":"2024-05-10T11:12:57.268678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T11:09:26.176441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5f031418-da23-4e85-8669-160960dadd09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":643028278},{"type":"v1","addr":"172.21.15.100:6803","nonce":643028278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":643028278},{"type":"v1","addr":"172.21.15.100:6805","nonce":643028278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":643028278},{"type":"v1","addr":"172.21.15.100:6809","nonce":643028278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":643028278},{"type":"v1","addr":"172.21.15.100:6807","nonce":643028278}]},"public_addr":"172.21.15.100:6803/643028278","cluster_addr":"172.21.15.100:6805/643028278","heartbeat_back_addr":"172.21.15.100:6809/643028278","heartbeat_front_addr":"172.21.15.100:6807/643028278","state":["exists","up"]},{"osd":1,"uuid":"a908cdc9-001e-4b65-9ab0-0dd08dba20f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":3103987017},{"type":"v1","addr":"172.21.15.178:6801","nonce":3103987017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":3103987017},{"type":"v1","addr":"172.21.15.178:6803","nonce":3103987017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":3103987017},{"type":"v1","addr":"172.21.15.178:6807","nonce":3103987017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":3103987017},{"type":"v1","addr":"172.21.15.178:6805","nonce":3103987017}]},"public_addr":"172.21.15.178:6801/3103987017","cluster_addr":"172.21.15.178:6803/3103987017","heartbeat_back_addr":"172.21.15.178:6807/3103987017","heartbeat_front_addr":"172.21.15.178:6805/3103987017","state":["exists","up"]},{"osd":2,"uuid":"4d8f89ae-9424-46e9-9c80-17c47cc5e8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3991466468},{"type":"v1","addr":"172.21.15.188:6801","nonce":3991466468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3991466468},{"type":"v1","addr":"172.21.15.188:6803","nonce":3991466468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3991466468},{"type":"v1","addr":"172.21.15.188:6807","nonce":3991466468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3991466468},{"type":"v1","addr":"172.21.15.188:6805","nonce":3991466468}]},"public_addr":"172.21.15.188:6801/3991466468","cluster_addr":"172.21.15.188:6803/3991466468","heartbeat_back_addr":"172.21.15.188:6807/3991466468","heartbeat_front_addr":"172.21.15.188:6805/3991466468","state":["exists","up"]},{"osd":3,"uuid":"4c8865aa-fef4-4710-b766-1385adb9462c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1713357192},{"type":"v1","addr":"172.21.15.100:6811","nonce":1713357192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1713357192},{"type":"v1","addr":"172.21.15.100:6813","nonce":1713357192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1713357192},{"type":"v1","addr":"172.21.15.100:6817","nonce":1713357192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1713357192},{"type":"v1","addr":"172.21.15.100:6815","nonce":1713357192}]},"public_addr":"172.21.15.100:6811/1713357192","cluster_addr":"172.21.15.100:6813/1713357192","heartbeat_back_addr":"172.21.15.100:6817/1713357192","heartbeat_front_addr":"172.21.15.100:6815/1713357192","state":["exists","up"]},{"osd":4,"uuid":"a9653800-da57-43eb-8781-3188e8a3949f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":2222814192},{"type":"v1","addr":"172.21.15.178:6809","nonce":2222814192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":2222814192},{"type":"v1","addr":"172.21.15.178:6811","nonce":2222814192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":2222814192},{"type":"v1","addr":"172.21.15.178:6815","nonce":2222814192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":2222814192},{"type":"v1","addr":"172.21.15.178:6813","nonce":2222814192}]},"public_addr":"172.21.15.178:6809/2222814192","cluster_addr":"172.21.15.178:6811/2222814192","heartbeat_back_addr":"172.21.15.178:6815/2222814192","heartbeat_front_addr":"172.21.15.178:6813/2222814192","state":["exists","up"]},{"osd":5,"uuid":"fb5f8af2-1eb0-4d8a-a7a3-ed6dd68e692e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3517375935},{"type":"v1","addr":"172.21.15.188:6809","nonce":3517375935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3517375935},{"type":"v1","addr":"172.21.15.188:6811","nonce":3517375935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3517375935},{"type":"v1","addr":"172.21.15.188:6815","nonce":3517375935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3517375935},{"type":"v1","addr":"172.21.15.188:6813","nonce":3517375935}]},"public_addr":"172.21.15.188:6809/3517375935","cluster_addr":"172.21.15.188:6811/3517375935","heartbeat_back_addr":"172.21.15.188:6815/3517375935","heartbeat_front_addr":"172.21.15.188:6813/3517375935","state":["exists","up"]},{"osd":6,"uuid":"84b57387-0a94-4e49-b099-493d329b87dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1410593862},{"type":"v1","addr":"172.21.15.100:6819","nonce":1410593862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1410593862},{"type":"v1","addr":"172.21.15.100:6821","nonce":1410593862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1410593862},{"type":"v1","addr":"172.21.15.100:6825","nonce":1410593862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1410593862},{"type":"v1","addr":"172.21.15.100:6823","nonce":1410593862}]},"public_addr":"172.21.15.100:6819/1410593862","cluster_addr":"172.21.15.100:6821/1410593862","heartbeat_back_addr":"172.21.15.100:6825/1410593862","heartbeat_front_addr":"172.21.15.100:6823/1410593862","state":["exists","up"]},{"osd":7,"uuid":"02ada28f-ac4f-47cf-983f-8a6da1140abe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2458690455},{"type":"v1","addr":"172.21.15.178:6817","nonce":2458690455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2458690455},{"type":"v1","addr":"172.21.15.178:6819","nonce":2458690455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2458690455},{"type":"v1","addr":"172.21.15.178:6823","nonce":2458690455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2458690455},{"type":"v1","addr":"172.21.15.178:6821","nonce":2458690455}]},"public_addr":"172.21.15.178:6817/2458690455","cluster_addr":"172.21.15.178:6819/2458690455","heartbeat_back_addr":"172.21.15.178:6823/2458690455","heartbeat_front_addr":"172.21.15.178:6821/2458690455","state":["exists","up"]},{"osd":8,"uuid":"e7e489bc-15fc-4a4c-b297-d1bb626513ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1523872826},{"type":"v1","addr":"172.21.15.188:6817","nonce":1523872826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1523872826},{"type":"v1","addr":"172.21.15.188:6819","nonce":1523872826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1523872826},{"type":"v1","addr":"172.21.15.188:6823","nonce":1523872826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1523872826},{"type":"v1","addr":"172.21.15.188:6821","nonce":1523872826}]},"public_addr":"172.21.15.188:6817/1523872826","cluster_addr":"172.21.15.188:6819/1523872826","heartbeat_back_addr":"172.21.15.188:6823/1523872826","heartbeat_front_addr":"172.21.15.188:6821/1523872826","state":["exists","up"]},{"osd":9,"uuid":"72328326-cd16-45d9-bfd9-40f6c5c20f5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3585761978},{"type":"v1","addr":"172.21.15.100:6827","nonce":3585761978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3585761978},{"type":"v1","addr":"172.21.15.100:6829","nonce":3585761978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3585761978},{"type":"v1","addr":"172.21.15.100:6833","nonce":3585761978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3585761978},{"type":"v1","addr":"172.21.15.100:6831","nonce":3585761978}]},"public_addr":"172.21.15.100:6827/3585761978","cluster_addr":"172.21.15.100:6829/3585761978","heartbeat_back_addr":"172.21.15.100:6833/3585761978","heartbeat_front_addr":"172.21.15.100:6831/3585761978","state":["exists","up"]},{"osd":10,"uuid":"a355f9a8-91a3-41d4-814d-5a9c9b953706","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2985841725},{"type":"v1","addr":"172.21.15.178:6825","nonce":2985841725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2985841725},{"type":"v1","addr":"172.21.15.178:6827","nonce":2985841725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2985841725},{"type":"v1","addr":"172.21.15.178:6831","nonce":2985841725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2985841725},{"type":"v1","addr":"172.21.15.178:6829","nonce":2985841725}]},"public_addr":"172.21.15.178:6825/2985841725","cluster_addr":"172.21.15.178:6827/2985841725","heartbeat_back_addr":"172.21.15.178:6831/2985841725","heartbeat_front_addr":"172.21.15.178:6829/2985841725","state":["exists","up"]},{"osd":11,"uuid":"76a5e13c-7e38-42f9-bfa7-7fedec7c7184","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2574955118},{"type":"v1","addr":"172.21.15.188:6825","nonce":2574955118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2574955118},{"type":"v1","addr":"172.21.15.188:6827","nonce":2574955118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2574955118},{"type":"v1","addr":"172.21.15.188:6831","nonce":2574955118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2574955118},{"type":"v1","addr":"172.21.15.188:6829","nonce":2574955118}]},"public_addr":"172.21.15.188:6825/2574955118","cluster_addr":"172.21.15.188:6827/2574955118","heartbeat_back_addr":"172.21.15.188:6831/2574955118","heartbeat_front_addr":"172.21.15.188:6829/2574955118","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:08:36.086637+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:08:58.948031+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:09:22.142517+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:09:47.655748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:10:12.853999+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:10:37.911383+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:11:04.448815+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:11:30.331740+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:11:55.819278+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:12:23.204612+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:12:49.484835+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:13:15.783531+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:6800/1597433518":"2024-05-11T11:07:00.092732+0000","172.21.15.100:6800/1086450256":"2024-05-11T11:06:33.114176+0000","172.21.15.100:0/767964410":"2024-05-11T11:07:00.092732+0000","172.21.15.100:6801/1597433518":"2024-05-11T11:07:00.092732+0000","172.21.15.100:6801/1086450256":"2024-05-11T11:06:33.114176+0000","172.21.15.100:0/2770572070":"2024-05-11T11:06:33.114176+0000","172.21.15.100:0/1379340513":"2024-05-11T11:06:33.114176+0000","172.21.15.100:0/3025646953":"2024-05-11T11:07:00.092732+0000","172.21.15.100:0/4099258930":"2024-05-11T11:07:00.092732+0000","172.21.15.100:0/4267693567":"2024-05-11T11:06:33.114176+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T11:13:24.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:24.273+0000 7f8228ddf700 1 -- 172.21.15.100:0/3050715103 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f820c0b2880 msgr2=0x7f820c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:13:24.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:24.273+0000 7f8228ddf700 1 --2- 172.21.15.100:0/3050715103 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f820c0b2880 0x7f820c0b4d40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8214000c00 tx=0x7f8214005fb0 comp rx=0 tx=0).stop 2024-05-10T11:13:24.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:24.273+0000 7f8228ddf700 1 -- 172.21.15.100:0/3050715103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f822406c820 msgr2=0x7f8224102a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:13:24.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:24.273+0000 7f8228ddf700 1 --2- 172.21.15.100:0/3050715103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f822406c820 0x7f8224102a40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f821800d950 tx=0x7f821800dc60 comp rx=0 tx=0).stop 2024-05-10T11:13:24.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:24.273+0000 7f8228ddf700 1 -- 172.21.15.100:0/3050715103 shutdown_connections 2024-05-10T11:13:24.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:24.273+0000 7f8228ddf700 1 --2- 172.21.15.100:0/3050715103 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f820c0b2880 0x7f820c0b4d40 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:24.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:24.273+0000 7f8228ddf700 1 --2- 172.21.15.100:0/3050715103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82241087c0 0x7f8224102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:24.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:24.273+0000 7f8228ddf700 1 --2- 172.21.15.100:0/3050715103 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f822406be70 0x7f8224102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:24.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:24.273+0000 7f8228ddf700 1 --2- 172.21.15.100:0/3050715103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f822406c820 0x7f8224102a40 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:24.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:24.273+0000 7f8228ddf700 1 -- 172.21.15.100:0/3050715103 >> 172.21.15.100:0/3050715103 conn(0x7f82240783f0 msgr2=0x7f8224107440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:13:24.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:24.273+0000 7f8228ddf700 1 -- 172.21.15.100:0/3050715103 shutdown_connections 2024-05-10T11:13:24.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:24.273+0000 7f8228ddf700 1 -- 172.21.15.100:0/3050715103 wait complete. 2024-05-10T11:13:24.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:23 smithi188 ceph-mon[34075]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:13:25.089 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T11:09:26.176441+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-10T11:13:25.089 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-05-10T11:13:25.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:24 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/3050715103' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T11:13:25.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:24 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/3050715103' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T11:13:25.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:24 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/3050715103' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T11:13:25.328 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:13:26.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:25 smithi100 ceph-mon[30296]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:13:26.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:25 smithi178 ceph-mon[33359]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:13:26.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:25 smithi188 ceph-mon[34075]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:13:27.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:27.034+0000 7f8d6e7c3700 1 -- 172.21.15.100:0/917317434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d68102600 msgr2=0x7f8d68108160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:13:27.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:27.034+0000 7f8d6e7c3700 1 --2- 172.21.15.100:0/917317434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d68102600 0x7f8d68108160 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8d500099c0 tx=0x7f8d50009cd0 comp rx=0 tx=0).stop 2024-05-10T11:13:27.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:27.034+0000 7f8d6e7c3700 1 -- 172.21.15.100:0/917317434 shutdown_connections 2024-05-10T11:13:27.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:27.034+0000 7f8d6e7c3700 1 --2- 172.21.15.100:0/917317434 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d68108890 0x7f8d68110410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:27.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:27.034+0000 7f8d6e7c3700 1 --2- 172.21.15.100:0/917317434 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d68101c50 0x7f8d68102030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:27.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:27.034+0000 7f8d6e7c3700 1 --2- 172.21.15.100:0/917317434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d68102600 0x7f8d68108160 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:27.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:27.034+0000 7f8d6e7c3700 1 -- 172.21.15.100:0/917317434 >> 172.21.15.100:0/917317434 conn(0x7f8d680789a0 msgr2=0x7f8d680ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:13:27.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:27.034+0000 7f8d6e7c3700 1 -- 172.21.15.100:0/917317434 shutdown_connections 2024-05-10T11:13:27.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:27.034+0000 7f8d6e7c3700 1 -- 172.21.15.100:0/917317434 wait complete. 2024-05-10T11:13:27.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:27.034+0000 7f8d6e7c3700 1 Processor -- start 2024-05-10T11:13:27.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:27.034+0000 7f8d6e7c3700 1 -- start start 2024-05-10T11:13:27.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:27.035+0000 7f8d6e7c3700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d68101c50 0x7f8d68102bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:13:27.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:27.035+0000 7f8d6e7c3700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d68102600 0x7f8d68103110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:13:27.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:27.035+0000 7f8d6e7c3700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d68108890 0x7f8d68103650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:13:27.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:27.035+0000 7f8d6e7c3700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8d68113580 con 0x7f8d68102600 2024-05-10T11:13:27.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:27.035+0000 7f8d6e7c3700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8d68113400 con 0x7f8d68101c50 2024-05-10T11:13:27.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:27.035+0000 7f8d6e7c3700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8d68113700 con 0x7f8d68108890 2024-05-10T11:13:27.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:27.035+0000 7f8d677fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d68102600 0x7f8d68103110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:13:27.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:27.035+0000 7f8d677fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d68102600 0x7f8d68103110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58782/0 (socket says 172.21.15.100:58782) 2024-05-10T11:13:27.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:27.035+0000 7f8d677fe700 1 -- 172.21.15.100:0/3209767621 learned_addr learned my addr 172.21.15.100:0/3209767621 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:13:27.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:27.035+0000 7f8d67fff700 1 --2- 172.21.15.100:0/3209767621 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d68101c50 0x7f8d68102bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:13:27.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:27.035+0000 7f8d6cd60700 1 --2- 172.21.15.100:0/3209767621 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d68108890 0x7f8d68103650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:13:27.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:27.035+0000 7f8d677fe700 1 -- 172.21.15.100:0/3209767621 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d68108890 msgr2=0x7f8d68103650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:13:27.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:27.035+0000 7f8d677fe700 1 --2- 172.21.15.100:0/3209767621 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d68108890 0x7f8d68103650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:27.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:27.035+0000 7f8d677fe700 1 -- 172.21.15.100:0/3209767621 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d68101c50 msgr2=0x7f8d68102bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:13:27.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:27.035+0000 7f8d677fe700 1 --2- 172.21.15.100:0/3209767621 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d68101c50 0x7f8d68102bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:27.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:27.035+0000 7f8d677fe700 1 -- 172.21.15.100:0/3209767621 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d681a7f80 con 0x7f8d68102600 2024-05-10T11:13:27.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:27.035+0000 7f8d677fe700 1 --2- 172.21.15.100:0/3209767621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d68102600 0x7f8d68103110 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8d50000c00 tx=0x7f8d5000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:13:27.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:27.036+0000 7f8d657fa700 1 -- 172.21.15.100:0/3209767621 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d50026070 con 0x7f8d68102600 2024-05-10T11:13:27.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:27.036+0000 7f8d6e7c3700 1 -- 172.21.15.100:0/3209767621 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d681a8210 con 0x7f8d68102600 2024-05-10T11:13:27.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:27.036+0000 7f8d657fa700 1 -- 172.21.15.100:0/3209767621 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8d50005030 con 0x7f8d68102600 2024-05-10T11:13:27.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:27.036+0000 7f8d657fa700 1 -- 172.21.15.100:0/3209767621 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d50004b70 con 0x7f8d68102600 2024-05-10T11:13:27.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:27.036+0000 7f8d6e7c3700 1 -- 172.21.15.100:0/3209767621 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d681a86c0 con 0x7f8d68102600 2024-05-10T11:13:27.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:27.037+0000 7f8d6e7c3700 1 -- 172.21.15.100:0/3209767621 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d6806a190 con 0x7f8d68102600 2024-05-10T11:13:27.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:27.038+0000 7f8d657fa700 1 -- 172.21.15.100:0/3209767621 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f8d50029070 con 0x7f8d68102600 2024-05-10T11:13:27.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:27.038+0000 7f8d657fa700 1 --2- 172.21.15.100:0/3209767621 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8d540b2880 0x7f8d540b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:13:27.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:27.039+0000 7f8d67fff700 1 --2- 172.21.15.100:0/3209767621 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8d540b2880 0x7f8d540b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:13:27.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:27.039+0000 7f8d657fa700 1 -- 172.21.15.100:0/3209767621 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f8d500e2bb0 con 0x7f8d68102600 2024-05-10T11:13:27.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:27.039+0000 7f8d67fff700 1 --2- 172.21.15.100:0/3209767621 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8d540b2880 0x7f8d540b4d40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8d58009a10 tx=0x7f8d58009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:13:27.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:27.042+0000 7f8d657fa700 1 -- 172.21.15.100:0/3209767621 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f8d50030020 con 0x7f8d68102600 2024-05-10T11:13:27.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:27.250+0000 7f8d6e7c3700 1 -- 172.21.15.100:0/3209767621 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8d6802e510 con 0x7f8d68102600 2024-05-10T11:13:27.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:27.250+0000 7f8d657fa700 1 -- 172.21.15.100:0/3209767621 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v73) ==== 93+0+10 (secure 0 0 0) 0x7f8d500a4e20 con 0x7f8d68102600 2024-05-10T11:13:27.251 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-05-10T11:13:27.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:27.253+0000 7f8d6e7c3700 1 -- 172.21.15.100:0/3209767621 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8d540b2880 msgr2=0x7f8d540b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:13:27.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:27.253+0000 7f8d6e7c3700 1 --2- 172.21.15.100:0/3209767621 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8d540b2880 0x7f8d540b4d40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8d58009a10 tx=0x7f8d58009190 comp rx=0 tx=0).stop 2024-05-10T11:13:27.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:27.253+0000 7f8d6e7c3700 1 -- 172.21.15.100:0/3209767621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d68102600 msgr2=0x7f8d68103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:13:27.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:27.253+0000 7f8d6e7c3700 1 --2- 172.21.15.100:0/3209767621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d68102600 0x7f8d68103110 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8d50000c00 tx=0x7f8d5000f690 comp rx=0 tx=0).stop 2024-05-10T11:13:27.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:27.253+0000 7f8d6e7c3700 1 -- 172.21.15.100:0/3209767621 shutdown_connections 2024-05-10T11:13:27.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:27.253+0000 7f8d6e7c3700 1 --2- 172.21.15.100:0/3209767621 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8d540b2880 0x7f8d540b4d40 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:27.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:27.253+0000 7f8d6e7c3700 1 --2- 172.21.15.100:0/3209767621 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d68108890 0x7f8d68103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:27.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:27.253+0000 7f8d6e7c3700 1 --2- 172.21.15.100:0/3209767621 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d68101c50 0x7f8d68102bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:27.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:27.253+0000 7f8d6e7c3700 1 --2- 172.21.15.100:0/3209767621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d68102600 0x7f8d68103110 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:27.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:27.253+0000 7f8d6e7c3700 1 -- 172.21.15.100:0/3209767621 >> 172.21.15.100:0/3209767621 conn(0x7f8d680789a0 msgr2=0x7f8d680ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:13:27.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:27.253+0000 7f8d6e7c3700 1 -- 172.21.15.100:0/3209767621 shutdown_connections 2024-05-10T11:13:27.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:27.253+0000 7f8d6e7c3700 1 -- 172.21.15.100:0/3209767621 wait complete. 2024-05-10T11:13:28.083 INFO:tasks.cephadm:Adding mds.a on smithi100 2024-05-10T11:13:28.083 INFO:tasks.cephadm:Adding mds.d on smithi100 2024-05-10T11:13:28.083 INFO:tasks.cephadm:Adding mds.g on smithi100 2024-05-10T11:13:28.083 INFO:tasks.cephadm:Adding mds.j on smithi100 2024-05-10T11:13:28.083 INFO:tasks.cephadm:Adding mds.b on smithi178 2024-05-10T11:13:28.083 INFO:tasks.cephadm:Adding mds.e on smithi178 2024-05-10T11:13:28.084 INFO:tasks.cephadm:Adding mds.h on smithi178 2024-05-10T11:13:28.084 INFO:tasks.cephadm:Adding mds.k on smithi178 2024-05-10T11:13:28.084 INFO:tasks.cephadm:Adding mds.c on smithi188 2024-05-10T11:13:28.084 INFO:tasks.cephadm:Adding mds.f on smithi188 2024-05-10T11:13:28.084 INFO:tasks.cephadm:Adding mds.i on smithi188 2024-05-10T11:13:28.084 INFO:tasks.cephadm:Adding mds.l on smithi188 2024-05-10T11:13:28.084 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph orch apply mds all '12;smithi100=a;smithi100=d;smithi100=g;smithi100=j;smithi178=b;smithi178=e;smithi178=h;smithi178=k;smithi188=c;smithi188=f;smithi188=i;smithi188=l' 2024-05-10T11:13:28.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:27 smithi178 ceph-mon[33359]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:13:28.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:27 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/3209767621' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T11:13:28.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:27 smithi188 ceph-mon[34075]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:13:28.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:27 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/3209767621' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T11:13:28.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:27 smithi100 ceph-mon[30296]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:13:28.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:27 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/3209767621' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T11:13:28.336 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T11:13:30.142 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:13:30.140+0000 7f595cc2c700 1 -- 172.21.15.188:0/3249237041 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5958103fa0 msgr2=0x7f595810a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:13:30.142 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:13:30.140+0000 7f595cc2c700 1 --2- 172.21.15.188:0/3249237041 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5958103fa0 0x7f595810a850 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f595000b0a0 tx=0x7f595000b3b0 comp rx=0 tx=0).stop 2024-05-10T11:13:30.143 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:13:30.140+0000 7f595cc2c700 1 -- 172.21.15.188:0/3249237041 shutdown_connections 2024-05-10T11:13:30.143 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:13:30.140+0000 7f595cc2c700 1 --2- 172.21.15.188:0/3249237041 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5958103fa0 0x7f595810a850 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:30.143 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:13:30.140+0000 7f595cc2c700 1 --2- 172.21.15.188:0/3249237041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5958102400 0x7f5958102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:30.143 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:13:30.140+0000 7f595cc2c700 1 --2- 172.21.15.188:0/3249237041 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5958103600 0x7f5958103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:30.143 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:13:30.140+0000 7f595cc2c700 1 -- 172.21.15.188:0/3249237041 >> 172.21.15.188:0/3249237041 conn(0x7f59580fd9a0 msgr2=0x7f59580ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:13:30.143 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:13:30.140+0000 7f595cc2c700 1 -- 172.21.15.188:0/3249237041 shutdown_connections 2024-05-10T11:13:30.143 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:13:30.140+0000 7f595cc2c700 1 -- 172.21.15.188:0/3249237041 wait complete. 2024-05-10T11:13:30.143 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:13:30.141+0000 7f595cc2c700 1 Processor -- start 2024-05-10T11:13:30.143 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:13:30.141+0000 7f595cc2c700 1 -- start start 2024-05-10T11:13:30.143 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:13:30.141+0000 7f595cc2c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5958102400 0x7f595806bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:13:30.143 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:13:30.141+0000 7f595cc2c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5958103600 0x7f595819e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:13:30.143 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:13:30.141+0000 7f595cc2c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5958103fa0 0x7f59581a08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:13:30.144 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:13:30.141+0000 7f595cc2c700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f595810dd40 con 0x7f5958102400 2024-05-10T11:13:30.144 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:13:30.141+0000 7f595cc2c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f595810dbc0 con 0x7f5958103600 2024-05-10T11:13:30.144 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:13:30.141+0000 7f595cc2c700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f595810dec0 con 0x7f5958103fa0 2024-05-10T11:13:30.144 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:13:30.141+0000 7f5956d9d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5958103fa0 0x7f59581a08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:13:30.144 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:13:30.141+0000 7f5956d9d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5958103fa0 0x7f59581a08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:58866/0 (socket says 172.21.15.188:58866) 2024-05-10T11:13:30.144 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:13:30.141+0000 7f5956d9d700 1 -- 172.21.15.188:0/2021174775 learned_addr learned my addr 172.21.15.188:0/2021174775 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-05-10T11:13:30.144 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:13:30.141+0000 7f595659c700 1 --2- 172.21.15.188:0/2021174775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5958102400 0x7f595806bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:13:30.144 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:13:30.141+0000 7f5955d9b700 1 --2- 172.21.15.188:0/2021174775 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5958103600 0x7f595819e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:13:30.144 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:13:30.141+0000 7f5956d9d700 1 -- 172.21.15.188:0/2021174775 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5958103600 msgr2=0x7f595819e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:13:30.144 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:13:30.141+0000 7f5956d9d700 1 --2- 172.21.15.188:0/2021174775 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5958103600 0x7f595819e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:30.144 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:13:30.141+0000 7f5956d9d700 1 -- 172.21.15.188:0/2021174775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5958102400 msgr2=0x7f595806bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:13:30.144 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:13:30.141+0000 7f5956d9d700 1 --2- 172.21.15.188:0/2021174775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5958102400 0x7f595806bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:30.144 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:13:30.141+0000 7f5956d9d700 1 -- 172.21.15.188:0/2021174775 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59581a11c0 con 0x7f5958103fa0 2024-05-10T11:13:30.144 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:13:30.141+0000 7f595659c700 1 --2- 172.21.15.188:0/2021174775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5958102400 0x7f595806bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:13:30.145 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:13:30.141+0000 7f5956d9d700 1 --2- 172.21.15.188:0/2021174775 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5958103fa0 0x7f59581a08f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f595000b830 tx=0x7f5950007d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:13:30.145 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:13:30.142+0000 7f59477fe700 1 -- 172.21.15.188:0/2021174775 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5950021020 con 0x7f5958103fa0 2024-05-10T11:13:30.145 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:13:30.142+0000 7f59477fe700 1 -- 172.21.15.188:0/2021174775 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f595001d900 con 0x7f5958103fa0 2024-05-10T11:13:30.145 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:13:30.142+0000 7f595cc2c700 1 -- 172.21.15.188:0/2021174775 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59581a1450 con 0x7f5958103fa0 2024-05-10T11:13:30.145 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:13:30.142+0000 7f59477fe700 1 -- 172.21.15.188:0/2021174775 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59500053d0 con 0x7f5958103fa0 2024-05-10T11:13:30.145 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:13:30.142+0000 7f595cc2c700 1 -- 172.21.15.188:0/2021174775 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59581a1900 con 0x7f5958103fa0 2024-05-10T11:13:30.145 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:13:30.143+0000 7f595cc2c700 1 -- 172.21.15.188:0/2021174775 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f595806a190 con 0x7f5958103fa0 2024-05-10T11:13:30.146 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:13:30.144+0000 7f59477fe700 1 -- 172.21.15.188:0/2021174775 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f5950024020 con 0x7f5958103fa0 2024-05-10T11:13:30.146 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:13:30.144+0000 7f59477fe700 1 --2- 172.21.15.188:0/2021174775 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f59400b2880 0x7f59400b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:13:30.146 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:13:30.145+0000 7f59477fe700 1 -- 172.21.15.188:0/2021174775 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f595000f020 con 0x7f5958103fa0 2024-05-10T11:13:30.146 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:13:30.145+0000 7f595659c700 1 --2- 172.21.15.188:0/2021174775 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f59400b2880 0x7f59400b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:13:30.146 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:13:30.145+0000 7f595659c700 1 --2- 172.21.15.188:0/2021174775 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f59400b2880 0x7f59400b4d40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5948005950 tx=0x7f59480058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:13:30.148 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:13:30.148+0000 7f59477fe700 1 -- 172.21.15.188:0/2021174775 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f59500ab8d0 con 0x7f5958103fa0 2024-05-10T11:13:30.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:29 smithi178 ceph-mon[33359]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:13:30.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:29 smithi188 ceph-mon[34075]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:13:30.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:29 smithi100 ceph-mon[30296]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:13:30.365 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:13:30.365+0000 7f595cc2c700 1 -- 172.21.15.188:0/2021174775 --> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi100=a;smithi100=d;smithi100=g;smithi100=j;smithi178=b;smithi178=e;smithi178=h;smithi178=k;smithi188=c;smithi188=f;smithi188=i;smithi188=l", "target": ["mon-mgr", ""]}) -- 0x7f59580639e0 con 0x7f59400b2880 2024-05-10T11:13:30.373 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:13:30.374+0000 7f59477fe700 1 -- 172.21.15.188:0/2021174775 <== mgr.14152 v2:172.21.15.100:6800/2507287356 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f59580639e0 con 0x7f59400b2880 2024-05-10T11:13:30.374 INFO:teuthology.orchestra.run.smithi188.stdout:Scheduled mds.all update... 2024-05-10T11:13:30.376 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:13:30.376+0000 7f595cc2c700 1 -- 172.21.15.188:0/2021174775 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f59400b2880 msgr2=0x7f59400b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:13:30.376 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:13:30.376+0000 7f595cc2c700 1 --2- 172.21.15.188:0/2021174775 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f59400b2880 0x7f59400b4d40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5948005950 tx=0x7f59480058e0 comp rx=0 tx=0).stop 2024-05-10T11:13:30.376 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:13:30.376+0000 7f595cc2c700 1 -- 172.21.15.188:0/2021174775 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5958103fa0 msgr2=0x7f59581a08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:13:30.376 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:13:30.376+0000 7f595cc2c700 1 --2- 172.21.15.188:0/2021174775 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5958103fa0 0x7f59581a08f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f595000b830 tx=0x7f5950007d10 comp rx=0 tx=0).stop 2024-05-10T11:13:30.376 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:13:30.376+0000 7f595cc2c700 1 -- 172.21.15.188:0/2021174775 shutdown_connections 2024-05-10T11:13:30.377 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:13:30.376+0000 7f595cc2c700 1 --2- 172.21.15.188:0/2021174775 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f59400b2880 0x7f59400b4d40 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:30.377 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:13:30.376+0000 7f595cc2c700 1 --2- 172.21.15.188:0/2021174775 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5958103fa0 0x7f59581a08f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:30.377 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:13:30.376+0000 7f595cc2c700 1 --2- 172.21.15.188:0/2021174775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5958102400 0x7f595806bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:30.377 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:13:30.376+0000 7f595cc2c700 1 --2- 172.21.15.188:0/2021174775 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5958103600 0x7f595819e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:30.377 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:13:30.376+0000 7f595cc2c700 1 -- 172.21.15.188:0/2021174775 >> 172.21.15.188:0/2021174775 conn(0x7f59580fd9a0 msgr2=0x7f59581069c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:13:30.377 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:13:30.376+0000 7f595cc2c700 1 -- 172.21.15.188:0/2021174775 shutdown_connections 2024-05-10T11:13:30.377 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-10T11:13:30.376+0000 7f595cc2c700 1 -- 172.21.15.188:0/2021174775 wait complete. 2024-05-10T11:13:31.118 DEBUG:teuthology.orchestra.run.smithi100:mds.a> sudo journalctl -f -n 0 -u ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mds.a.service 2024-05-10T11:13:31.121 DEBUG:teuthology.orchestra.run.smithi100:mds.d> sudo journalctl -f -n 0 -u ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mds.d.service 2024-05-10T11:13:31.165 DEBUG:teuthology.orchestra.run.smithi100:mds.g> sudo journalctl -f -n 0 -u ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mds.g.service 2024-05-10T11:13:31.208 DEBUG:teuthology.orchestra.run.smithi100:mds.j> sudo journalctl -f -n 0 -u ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mds.j.service 2024-05-10T11:13:31.250 DEBUG:teuthology.orchestra.run.smithi178:mds.b> sudo journalctl -f -n 0 -u ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mds.b.service 2024-05-10T11:13:31.253 DEBUG:teuthology.orchestra.run.smithi178:mds.e> sudo journalctl -f -n 0 -u ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mds.e.service 2024-05-10T11:13:31.297 DEBUG:teuthology.orchestra.run.smithi178:mds.h> sudo journalctl -f -n 0 -u ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mds.h.service 2024-05-10T11:13:31.341 DEBUG:teuthology.orchestra.run.smithi178:mds.k> sudo journalctl -f -n 0 -u ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mds.k.service 2024-05-10T11:13:31.384 DEBUG:teuthology.orchestra.run.smithi188:mds.c> sudo journalctl -f -n 0 -u ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mds.c.service 2024-05-10T11:13:31.386 DEBUG:teuthology.orchestra.run.smithi188:mds.f> sudo journalctl -f -n 0 -u ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mds.f.service 2024-05-10T11:13:31.428 DEBUG:teuthology.orchestra.run.smithi188:mds.i> sudo journalctl -f -n 0 -u ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mds.i.service 2024-05-10T11:13:31.471 DEBUG:teuthology.orchestra.run.smithi188:mds.l> sudo journalctl -f -n 0 -u ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mds.l.service 2024-05-10T11:13:31.514 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-05-10T11:13:31.515 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T11:13:31.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:31 smithi100 ceph-mon[30296]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:13:31.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:31 smithi100 ceph-mon[30296]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi100=a;smithi100=d;smithi100=g;smithi100=j;smithi178=b;smithi178=e;smithi178=h;smithi178=k;smithi188=c;smithi188=f;smithi188=i;smithi188=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:13:31.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:31 smithi100 ceph-mon[30296]: Saving service mds.all spec with placement smithi100=a;smithi100=d;smithi100=g;smithi100=j;smithi178=b;smithi178=e;smithi178=h;smithi178=k;smithi188=c;smithi188=f;smithi188=i;smithi188=l;count:12 2024-05-10T11:13:31.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:31 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:31.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:31 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:13:31.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:31 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:13:31.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:31 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:13:31.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:31 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:31.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:31 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T11:13:31.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:31 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T11:13:31.539 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:31 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:13:31.539 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:31 smithi100 ceph-mon[30296]: Deploying daemon mds.l on smithi188 2024-05-10T11:13:31.656 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:31 smithi188 ceph-mon[34075]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:13:31.656 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:31 smithi188 ceph-mon[34075]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi100=a;smithi100=d;smithi100=g;smithi100=j;smithi178=b;smithi178=e;smithi178=h;smithi178=k;smithi188=c;smithi188=f;smithi188=i;smithi188=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:13:31.657 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:31 smithi188 ceph-mon[34075]: Saving service mds.all spec with placement smithi100=a;smithi100=d;smithi100=g;smithi100=j;smithi178=b;smithi178=e;smithi178=h;smithi178=k;smithi188=c;smithi188=f;smithi188=i;smithi188=l;count:12 2024-05-10T11:13:31.657 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:31 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:31.657 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:31 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:13:31.657 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:31 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:13:31.657 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:31 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:13:31.657 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:31 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:31.657 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:31 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T11:13:31.657 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:31 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T11:13:31.658 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:31 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:13:31.658 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:31 smithi188 ceph-mon[34075]: Deploying daemon mds.l on smithi188 2024-05-10T11:13:31.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:31 smithi178 ceph-mon[33359]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:13:31.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:31 smithi178 ceph-mon[33359]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi100=a;smithi100=d;smithi100=g;smithi100=j;smithi178=b;smithi178=e;smithi178=h;smithi178=k;smithi188=c;smithi188=f;smithi188=i;smithi188=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:13:31.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:31 smithi178 ceph-mon[33359]: Saving service mds.all spec with placement smithi100=a;smithi100=d;smithi100=g;smithi100=j;smithi178=b;smithi178=e;smithi178=h;smithi178=k;smithi188=c;smithi188=f;smithi188=i;smithi188=l;count:12 2024-05-10T11:13:31.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:31 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:31.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:31 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:13:31.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:31 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:13:31.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:31 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:13:31.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:31 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:31.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:31 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T11:13:31.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:31 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T11:13:31.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:31 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:13:31.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:31 smithi178 ceph-mon[33359]: Deploying daemon mds.l on smithi188 2024-05-10T11:13:31.757 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:13:33.028 INFO:journalctl@ceph.mds.l.smithi188.stdout:May 10 11:13:32 smithi188 systemd[1]: Starting Ceph mds.l for 20a542cc-0ebd-11ef-bc98-c7b262605968... 2024-05-10T11:13:33.325 INFO:journalctl@ceph.mds.l.smithi188.stdout:May 10 11:13:33 smithi188 podman[63601]: 2024-05-10 11:13:32.940750144 +0000 UTC m=+0.018006246 image pull ee5e4f21ef58047cb638cccd5cfbe8f6f43412b56c94b1d7fbd9cf3cb90a53a5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f 2024-05-10T11:13:33.325 INFO:journalctl@ceph.mds.l.smithi188.stdout:May 10 11:13:33 smithi188 podman[63601]: 2024-05-10 11:13:33.048178997 +0000 UTC m=+0.125435098 container create f411399636efe8355c2c672777fdab789ea85b8a377c4a410547d3d83cc28943 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-l, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_BRANCH=HEAD) 2024-05-10T11:13:33.325 INFO:journalctl@ceph.mds.l.smithi188.stdout:May 10 11:13:33 smithi188 podman[63601]: 2024-05-10 11:13:33.325704257 +0000 UTC m=+0.402960351 container init f411399636efe8355c2c672777fdab789ea85b8a377c4a410547d3d83cc28943 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-l, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-10T11:13:33.449 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:33 smithi100 ceph-mon[30296]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:13:33.449 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:33 smithi100 ceph-mon[30296]: mds.? [v2:172.21.15.188:6832/1704891041,v1:172.21.15.188:6833/1704891041] up:boot 2024-05-10T11:13:33.449 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:33 smithi100 ceph-mon[30296]: fsmap 1 up:standby 2024-05-10T11:13:33.450 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:33 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-10T11:13:33.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:33.554+0000 7f7a3dd96700 1 -- 172.21.15.100:0/3029979469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a38102540 msgr2=0x7f7a3810e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:13:33.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:33.554+0000 7f7a3dd96700 1 --2- 172.21.15.100:0/3029979469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a38102540 0x7f7a3810e9b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7a3000b0a0 tx=0x7f7a3000b3b0 comp rx=0 tx=0).stop 2024-05-10T11:13:33.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:33.554+0000 7f7a3dd96700 1 -- 172.21.15.100:0/3029979469 shutdown_connections 2024-05-10T11:13:33.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:33.554+0000 7f7a3dd96700 1 --2- 172.21.15.100:0/3029979469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a38102540 0x7f7a3810e9b0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:33.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:33.554+0000 7f7a3dd96700 1 --2- 172.21.15.100:0/3029979469 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a38107be0 0x7f7a38107fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:33.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:33.554+0000 7f7a3dd96700 1 --2- 172.21.15.100:0/3029979469 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a38101b80 0x7f7a38102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:33.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:33.554+0000 7f7a3dd96700 1 -- 172.21.15.100:0/3029979469 >> 172.21.15.100:0/3029979469 conn(0x7f7a38078a10 msgr2=0x7f7a380ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:13:33.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:33.554+0000 7f7a3dd96700 1 -- 172.21.15.100:0/3029979469 shutdown_connections 2024-05-10T11:13:33.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:33.554+0000 7f7a3dd96700 1 -- 172.21.15.100:0/3029979469 wait complete. 2024-05-10T11:13:33.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:33.555+0000 7f7a3dd96700 1 Processor -- start 2024-05-10T11:13:33.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:33.555+0000 7f7a3dd96700 1 -- start start 2024-05-10T11:13:33.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:33.555+0000 7f7a3dd96700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a38101b80 0x7f7a381a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:13:33.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:33.555+0000 7f7a3dd96700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a38102540 0x7f7a381a1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:13:33.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:33.555+0000 7f7a3dd96700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a38107be0 0x7f7a3819b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:13:33.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:33.555+0000 7f7a3dd96700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7a38112490 con 0x7f7a38101b80 2024-05-10T11:13:33.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:33.555+0000 7f7a3dd96700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7a38112310 con 0x7f7a38102540 2024-05-10T11:13:33.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:33.555+0000 7f7a3dd96700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7a38112610 con 0x7f7a38107be0 2024-05-10T11:13:33.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:33.555+0000 7f7a377fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a38101b80 0x7f7a381a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:13:33.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:33.555+0000 7f7a377fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a38101b80 0x7f7a381a17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58806/0 (socket says 172.21.15.100:58806) 2024-05-10T11:13:33.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:33.555+0000 7f7a377fe700 1 -- 172.21.15.100:0/2975018120 learned_addr learned my addr 172.21.15.100:0/2975018120 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:13:33.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:33.556+0000 7f7a37fff700 1 --2- 172.21.15.100:0/2975018120 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a38107be0 0x7f7a3819b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:13:33.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:33.556+0000 7f7a2effd700 1 --2- 172.21.15.100:0/2975018120 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a38102540 0x7f7a381a1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:13:33.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:33.556+0000 7f7a377fe700 1 -- 172.21.15.100:0/2975018120 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a38107be0 msgr2=0x7f7a3819b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:13:33.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:33.556+0000 7f7a377fe700 1 --2- 172.21.15.100:0/2975018120 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a38107be0 0x7f7a3819b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:33.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:33.556+0000 7f7a377fe700 1 -- 172.21.15.100:0/2975018120 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a38102540 msgr2=0x7f7a381a1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:13:33.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:33.556+0000 7f7a377fe700 1 --2- 172.21.15.100:0/2975018120 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a38102540 0x7f7a381a1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:33.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:33.556+0000 7f7a377fe700 1 -- 172.21.15.100:0/2975018120 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a3819c110 con 0x7f7a38101b80 2024-05-10T11:13:33.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:33.556+0000 7f7a377fe700 1 --2- 172.21.15.100:0/2975018120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a38101b80 0x7f7a381a17f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7a2800c970 tx=0x7f7a2800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:13:33.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:33.556+0000 7f7a357fa700 1 -- 172.21.15.100:0/2975018120 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a280078d0 con 0x7f7a38101b80 2024-05-10T11:13:33.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:33.556+0000 7f7a357fa700 1 -- 172.21.15.100:0/2975018120 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7a28018480 con 0x7f7a38101b80 2024-05-10T11:13:33.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:33.556+0000 7f7a3dd96700 1 -- 172.21.15.100:0/2975018120 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a3819c3a0 con 0x7f7a38101b80 2024-05-10T11:13:33.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:33.556+0000 7f7a357fa700 1 -- 172.21.15.100:0/2975018120 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a2800f6b0 con 0x7f7a38101b80 2024-05-10T11:13:33.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:33.556+0000 7f7a3dd96700 1 -- 172.21.15.100:0/2975018120 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a381a86c0 con 0x7f7a38101b80 2024-05-10T11:13:33.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:33.557+0000 7f7a3dd96700 1 -- 172.21.15.100:0/2975018120 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a3806a190 con 0x7f7a38101b80 2024-05-10T11:13:33.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:33.558+0000 7f7a357fa700 1 -- 172.21.15.100:0/2975018120 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f7a28018930 con 0x7f7a38101b80 2024-05-10T11:13:33.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:33.559+0000 7f7a357fa700 1 --2- 172.21.15.100:0/2975018120 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f7a240b2880 0x7f7a240b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:13:33.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:33.559+0000 7f7a2effd700 1 --2- 172.21.15.100:0/2975018120 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f7a240b2880 0x7f7a240b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:13:33.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:33.559+0000 7f7a357fa700 1 -- 172.21.15.100:0/2975018120 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f7a280e1dd0 con 0x7f7a38101b80 2024-05-10T11:13:33.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:33.559+0000 7f7a2effd700 1 --2- 172.21.15.100:0/2975018120 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f7a240b2880 0x7f7a240b4d40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7a38100660 tx=0x7f7a200073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:13:33.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:33.563+0000 7f7a357fa700 1 -- 172.21.15.100:0/2975018120 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f7a280aa020 con 0x7f7a38101b80 2024-05-10T11:13:33.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:33 smithi178 ceph-mon[33359]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:13:33.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:33 smithi178 ceph-mon[33359]: mds.? [v2:172.21.15.188:6832/1704891041,v1:172.21.15.188:6833/1704891041] up:boot 2024-05-10T11:13:33.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:33 smithi178 ceph-mon[33359]: fsmap 1 up:standby 2024-05-10T11:13:33.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:33 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-10T11:13:33.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:33 smithi188 ceph-mon[34075]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:13:33.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:33 smithi188 ceph-mon[34075]: mds.? [v2:172.21.15.188:6832/1704891041,v1:172.21.15.188:6833/1704891041] up:boot 2024-05-10T11:13:33.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:33 smithi188 ceph-mon[34075]: fsmap 1 up:standby 2024-05-10T11:13:33.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:33 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-10T11:13:33.779 INFO:journalctl@ceph.mds.l.smithi188.stdout:May 10 11:13:33 smithi188 podman[63601]: 2024-05-10 11:13:33.328459039 +0000 UTC m=+0.405715134 container start f411399636efe8355c2c672777fdab789ea85b8a377c4a410547d3d83cc28943 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, ceph=True) 2024-05-10T11:13:33.779 INFO:journalctl@ceph.mds.l.smithi188.stdout:May 10 11:13:33 smithi188 ceph-mds[63621]: -- 172.21.15.188:0/145314549 <== mon.1 v2:172.21.15.188:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561124913a40 con 0x5611248e7000 2024-05-10T11:13:33.779 INFO:journalctl@ceph.mds.l.smithi188.stdout:May 10 11:13:33 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-l[63617]: starting mds.l at 2024-05-10T11:13:33.779 INFO:journalctl@ceph.mds.l.smithi188.stdout:May 10 11:13:33 smithi188 bash[63601]: f411399636efe8355c2c672777fdab789ea85b8a377c4a410547d3d83cc28943 2024-05-10T11:13:33.779 INFO:journalctl@ceph.mds.l.smithi188.stdout:May 10 11:13:33 smithi188 systemd[1]: Started Ceph mds.l for 20a542cc-0ebd-11ef-bc98-c7b262605968. 2024-05-10T11:13:33.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:33.814+0000 7f7a3dd96700 1 -- 172.21.15.100:0/2975018120 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7a381a8890 con 0x7f7a38101b80 2024-05-10T11:13:33.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:33.815+0000 7f7a357fa700 1 -- 172.21.15.100:0/2975018120 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f7a280a9920 con 0x7f7a38101b80 2024-05-10T11:13:33.815 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:13:33.815 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1704891041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1704891041},{"type":"v1","addr":"172.21.15.188:6833","nonce":1704891041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-10T11:13:33.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:33.817+0000 7f7a3dd96700 1 -- 172.21.15.100:0/2975018120 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f7a240b2880 msgr2=0x7f7a240b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:13:33.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:33.817+0000 7f7a3dd96700 1 --2- 172.21.15.100:0/2975018120 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f7a240b2880 0x7f7a240b4d40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7a38100660 tx=0x7f7a200073d0 comp rx=0 tx=0).stop 2024-05-10T11:13:33.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:33.817+0000 7f7a3dd96700 1 -- 172.21.15.100:0/2975018120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a38101b80 msgr2=0x7f7a381a17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:13:33.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:33.817+0000 7f7a3dd96700 1 --2- 172.21.15.100:0/2975018120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a38101b80 0x7f7a381a17f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7a2800c970 tx=0x7f7a2800cc80 comp rx=0 tx=0).stop 2024-05-10T11:13:33.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:33.817+0000 7f7a3dd96700 1 -- 172.21.15.100:0/2975018120 shutdown_connections 2024-05-10T11:13:33.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:33.817+0000 7f7a3dd96700 1 --2- 172.21.15.100:0/2975018120 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f7a240b2880 0x7f7a240b4d40 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:33.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:33.817+0000 7f7a3dd96700 1 --2- 172.21.15.100:0/2975018120 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a38107be0 0x7f7a3819b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:33.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:33.817+0000 7f7a3dd96700 1 --2- 172.21.15.100:0/2975018120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a38101b80 0x7f7a381a17f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:33.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:33.817+0000 7f7a3dd96700 1 --2- 172.21.15.100:0/2975018120 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a38102540 0x7f7a381a1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:33.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:33.817+0000 7f7a3dd96700 1 -- 172.21.15.100:0/2975018120 >> 172.21.15.100:0/2975018120 conn(0x7f7a38078a10 msgr2=0x7f7a38106780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:13:33.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:33.817+0000 7f7a3dd96700 1 -- 172.21.15.100:0/2975018120 shutdown_connections 2024-05-10T11:13:33.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:33.817+0000 7f7a3dd96700 1 -- 172.21.15.100:0/2975018120 wait complete. 2024-05-10T11:13:33.819 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 2 2024-05-10T11:13:34.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:34 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:34.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:34 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:34.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:34 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:34.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:34 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T11:13:34.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:34 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T11:13:34.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:34 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:13:34.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:34 smithi178 ceph-mon[33359]: Deploying daemon mds.k on smithi178 2024-05-10T11:13:34.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:34 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/2975018120' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:13:34.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:34 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:34.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:34 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:34.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:34 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:34.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:34 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T11:13:34.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:34 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T11:13:34.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:34 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:13:34.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:34 smithi188 ceph-mon[34075]: Deploying daemon mds.k on smithi178 2024-05-10T11:13:34.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:34 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/2975018120' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:13:34.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:34 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:34.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:34 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:34.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:34 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:34.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:34 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T11:13:34.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:34 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T11:13:34.789 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:34 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:13:34.789 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:34 smithi100 ceph-mon[30296]: Deploying daemon mds.k on smithi178 2024-05-10T11:13:34.789 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:34 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/2975018120' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:13:35.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:35 smithi178 ceph-mon[33359]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:13:35.753 INFO:journalctl@ceph.mds.k.smithi178.stdout:May 10 11:13:35 smithi178 systemd[1]: Starting Ceph mds.k for 20a542cc-0ebd-11ef-bc98-c7b262605968... 2024-05-10T11:13:35.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:35 smithi188 ceph-mon[34075]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:13:35.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:35 smithi100 ceph-mon[30296]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:13:36.167 INFO:journalctl@ceph.mds.k.smithi178.stdout:May 10 11:13:35 smithi178 podman[62981]: 2024-05-10 11:13:35.774121921 +0000 UTC m=+0.018069704 image pull ee5e4f21ef58047cb638cccd5cfbe8f6f43412b56c94b1d7fbd9cf3cb90a53a5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f 2024-05-10T11:13:36.167 INFO:journalctl@ceph.mds.k.smithi178.stdout:May 10 11:13:35 smithi178 podman[62981]: 2024-05-10 11:13:35.915196084 +0000 UTC m=+0.159143851 container create 348fd0498391e5b3e79ac16d3fcbc08d472a81825d9d5d0f2d7778d22281ed38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-k, org.label-schema.license=GPLv2, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507) 2024-05-10T11:13:36.168 INFO:journalctl@ceph.mds.k.smithi178.stdout:May 10 11:13:36 smithi178 podman[62981]: 2024-05-10 11:13:36.16805662 +0000 UTC m=+0.412004383 container init 348fd0498391e5b3e79ac16d3fcbc08d472a81825d9d5d0f2d7778d22281ed38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-k, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_BRANCH=HEAD, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.schema-version=1.0) 2024-05-10T11:13:36.502 INFO:journalctl@ceph.mds.k.smithi178.stdout:May 10 11:13:36 smithi178 podman[62981]: 2024-05-10 11:13:36.17130615 +0000 UTC m=+0.415253922 container start 348fd0498391e5b3e79ac16d3fcbc08d472a81825d9d5d0f2d7778d22281ed38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-k, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08) 2024-05-10T11:13:36.502 INFO:journalctl@ceph.mds.k.smithi178.stdout:May 10 11:13:36 smithi178 ceph-mds[63000]: -- 172.21.15.178:0/806454319 <== mon.2 v2:172.21.15.178:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562c3e61fa40 con 0x562c3e5f2800 2024-05-10T11:13:36.502 INFO:journalctl@ceph.mds.k.smithi178.stdout:May 10 11:13:36 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-k[62996]: starting mds.k at 2024-05-10T11:13:36.502 INFO:journalctl@ceph.mds.k.smithi178.stdout:May 10 11:13:36 smithi178 bash[62981]: 348fd0498391e5b3e79ac16d3fcbc08d472a81825d9d5d0f2d7778d22281ed38 2024-05-10T11:13:36.502 INFO:journalctl@ceph.mds.k.smithi178.stdout:May 10 11:13:36 smithi178 systemd[1]: Started Ceph mds.k for 20a542cc-0ebd-11ef-bc98-c7b262605968. 2024-05-10T11:13:36.604 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T11:13:36.851 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:13:37.590 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:37 smithi178 ceph-mon[33359]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:13:37.591 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:37 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:37.591 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:37 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:37.591 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:37 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:37.591 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:37 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T11:13:37.591 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:37 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T11:13:37.591 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:37 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:13:37.591 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:37 smithi178 ceph-mon[33359]: Deploying daemon mds.j on smithi100 2024-05-10T11:13:37.591 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:37 smithi178 ceph-mon[33359]: mds.? [v2:172.21.15.178:6832/2741810791,v1:172.21.15.178:6833/2741810791] up:boot 2024-05-10T11:13:37.591 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:37 smithi178 ceph-mon[33359]: fsmap 2 up:standby 2024-05-10T11:13:37.591 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:37 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-10T11:13:37.670 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:37 smithi100 ceph-mon[30296]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:13:37.670 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:37 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:37.670 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:37 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:37.670 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:37 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:37.670 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:37 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T11:13:37.671 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:37 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T11:13:37.671 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:37 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:13:37.671 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:37 smithi100 ceph-mon[30296]: Deploying daemon mds.j on smithi100 2024-05-10T11:13:37.671 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:37 smithi100 ceph-mon[30296]: mds.? [v2:172.21.15.178:6832/2741810791,v1:172.21.15.178:6833/2741810791] up:boot 2024-05-10T11:13:37.671 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:37 smithi100 ceph-mon[30296]: fsmap 2 up:standby 2024-05-10T11:13:37.671 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:37 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-10T11:13:37.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:37 smithi188 ceph-mon[34075]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:13:37.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:37 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:37.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:37 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:37.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:37 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:37.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:37 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T11:13:37.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:37 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T11:13:37.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:37 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:13:37.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:37 smithi188 ceph-mon[34075]: Deploying daemon mds.j on smithi100 2024-05-10T11:13:37.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:37 smithi188 ceph-mon[34075]: mds.? [v2:172.21.15.178:6832/2741810791,v1:172.21.15.178:6833/2741810791] up:boot 2024-05-10T11:13:37.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:37 smithi188 ceph-mon[34075]: fsmap 2 up:standby 2024-05-10T11:13:37.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:37 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-10T11:13:39.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:39.304+0000 7f18a7c6b700 1 -- 172.21.15.100:0/2120813133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18a0102d90 msgr2=0x7f18a010f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:13:39.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:39.304+0000 7f18a7c6b700 1 --2- 172.21.15.100:0/2120813133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18a0102d90 0x7f18a010f250 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1898009990 tx=0x7f1898009ca0 comp rx=0 tx=0).stop 2024-05-10T11:13:39.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:39.304+0000 7f18a7c6b700 1 -- 172.21.15.100:0/2120813133 shutdown_connections 2024-05-10T11:13:39.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:39.304+0000 7f18a7c6b700 1 --2- 172.21.15.100:0/2120813133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18a0102d90 0x7f18a010f250 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:39.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:39.304+0000 7f18a7c6b700 1 --2- 172.21.15.100:0/2120813133 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f18a0108450 0x7f18a0108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:39.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:39.304+0000 7f18a7c6b700 1 --2- 172.21.15.100:0/2120813133 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f18a01023d0 0x7f18a0102850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:39.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:39.304+0000 7f18a7c6b700 1 -- 172.21.15.100:0/2120813133 >> 172.21.15.100:0/2120813133 conn(0x7f18a0078410 msgr2=0x7f18a0078820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:13:39.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:39.304+0000 7f18a7c6b700 1 -- 172.21.15.100:0/2120813133 shutdown_connections 2024-05-10T11:13:39.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:39.304+0000 7f18a7c6b700 1 -- 172.21.15.100:0/2120813133 wait complete. 2024-05-10T11:13:39.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:39.305+0000 7f18a7c6b700 1 Processor -- start 2024-05-10T11:13:39.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:39.305+0000 7f18a7c6b700 1 -- start start 2024-05-10T11:13:39.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:39.305+0000 7f18a7c6b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18a01023d0 0x7f18a019a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:13:39.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:39.305+0000 7f18a7c6b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f18a0102d90 0x7f18a019ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:13:39.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:39.305+0000 7f18a7c6b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f18a0108450 0x7f18a01a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:13:39.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:39.305+0000 7f18a7c6b700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f18a0112470 con 0x7f18a01023d0 2024-05-10T11:13:39.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:39.305+0000 7f18a7c6b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f18a01122f0 con 0x7f18a0108450 2024-05-10T11:13:39.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:39.305+0000 7f18a7c6b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f18a01125f0 con 0x7f18a0102d90 2024-05-10T11:13:39.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:39.305+0000 7f18a5a07700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18a01023d0 0x7f18a019a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:13:39.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:39.305+0000 7f18a5a07700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18a01023d0 0x7f18a019a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51646/0 (socket says 172.21.15.100:51646) 2024-05-10T11:13:39.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:39.305+0000 7f18a5a07700 1 -- 172.21.15.100:0/2404901727 learned_addr learned my addr 172.21.15.100:0/2404901727 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:13:39.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:39.305+0000 7f18a6208700 1 --2- 172.21.15.100:0/2404901727 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f18a0108450 0x7f18a01a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:13:39.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:39.305+0000 7f18a5a07700 1 -- 172.21.15.100:0/2404901727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f18a0102d90 msgr2=0x7f18a019ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:13:39.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:39.305+0000 7f18a5a07700 1 --2- 172.21.15.100:0/2404901727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f18a0102d90 0x7f18a019ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:39.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:39.305+0000 7f18a5a07700 1 -- 172.21.15.100:0/2404901727 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f18a0108450 msgr2=0x7f18a01a5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:13:39.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:39.305+0000 7f18a5a07700 1 --2- 172.21.15.100:0/2404901727 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f18a0108450 0x7f18a01a5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:39.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:39.305+0000 7f18a5a07700 1 -- 172.21.15.100:0/2404901727 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18a01a5eb0 con 0x7f18a01023d0 2024-05-10T11:13:39.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:39.305+0000 7f18a6208700 1 --2- 172.21.15.100:0/2404901727 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f18a0108450 0x7f18a01a5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:13:39.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:39.306+0000 7f18a5a07700 1 --2- 172.21.15.100:0/2404901727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18a01023d0 0x7f18a019a8f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f189c0060b0 tx=0x7f189c007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:13:39.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:39.306+0000 7f1896ffd700 1 -- 172.21.15.100:0/2404901727 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189c01a070 con 0x7f18a01023d0 2024-05-10T11:13:39.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:39.306+0000 7f1896ffd700 1 -- 172.21.15.100:0/2404901727 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f189c00de20 con 0x7f18a01023d0 2024-05-10T11:13:39.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:39.306+0000 7f18a7c6b700 1 -- 172.21.15.100:0/2404901727 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18a01a60e0 con 0x7f18a01023d0 2024-05-10T11:13:39.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:39.306+0000 7f1896ffd700 1 -- 172.21.15.100:0/2404901727 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189c014690 con 0x7f18a01023d0 2024-05-10T11:13:39.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:39.306+0000 7f18a7c6b700 1 -- 172.21.15.100:0/2404901727 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18a01a6670 con 0x7f18a01023d0 2024-05-10T11:13:39.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:39.307+0000 7f18a7c6b700 1 -- 172.21.15.100:0/2404901727 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18a0103ac0 con 0x7f18a01023d0 2024-05-10T11:13:39.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:39.308+0000 7f1896ffd700 1 -- 172.21.15.100:0/2404901727 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f189c01d070 con 0x7f18a01023d0 2024-05-10T11:13:39.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:39.309+0000 7f1896ffd700 1 --2- 172.21.15.100:0/2404901727 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f188c0b2900 0x7f188c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:13:39.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:39.309+0000 7f18a5206700 1 --2- 172.21.15.100:0/2404901727 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f188c0b2900 0x7f188c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:13:39.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:39.309+0000 7f1896ffd700 1 -- 172.21.15.100:0/2404901727 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f189c0e1a90 con 0x7f18a01023d0 2024-05-10T11:13:39.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:39.309+0000 7f18a5206700 1 --2- 172.21.15.100:0/2404901727 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f188c0b2900 0x7f188c0b4dc0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f18a0078c90 tx=0x7f1890006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:13:39.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:39.312+0000 7f1896ffd700 1 -- 172.21.15.100:0/2404901727 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f189c010070 con 0x7f18a01023d0 2024-05-10T11:13:39.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:39 smithi100 ceph-mon[30296]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:13:39.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:39.575+0000 7f18a7c6b700 1 -- 172.21.15.100:0/2404901727 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f18a002e6d0 con 0x7f18a01023d0 2024-05-10T11:13:39.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:39.575+0000 7f1896ffd700 1 -- 172.21.15.100:0/2404901727 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2224 (secure 0 0 0) 0x7f189c0a9c50 con 0x7f18a01023d0 2024-05-10T11:13:39.575 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:13:39.575 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/2741810791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2741810791},{"type":"v1","addr":"172.21.15.178:6833","nonce":2741810791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1704891041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1704891041},{"type":"v1","addr":"172.21.15.188:6833","nonce":1704891041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-10T11:13:39.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:39.577+0000 7f18a7c6b700 1 -- 172.21.15.100:0/2404901727 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f188c0b2900 msgr2=0x7f188c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:13:39.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:39.577+0000 7f18a7c6b700 1 --2- 172.21.15.100:0/2404901727 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f188c0b2900 0x7f188c0b4dc0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f18a0078c90 tx=0x7f1890006d20 comp rx=0 tx=0).stop 2024-05-10T11:13:39.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:39.577+0000 7f18a7c6b700 1 -- 172.21.15.100:0/2404901727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18a01023d0 msgr2=0x7f18a019a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:13:39.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:39.577+0000 7f18a7c6b700 1 --2- 172.21.15.100:0/2404901727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18a01023d0 0x7f18a019a8f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f189c0060b0 tx=0x7f189c007f90 comp rx=0 tx=0).stop 2024-05-10T11:13:39.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:39.577+0000 7f18a7c6b700 1 -- 172.21.15.100:0/2404901727 shutdown_connections 2024-05-10T11:13:39.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:39.577+0000 7f18a7c6b700 1 --2- 172.21.15.100:0/2404901727 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f188c0b2900 0x7f188c0b4dc0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:39.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:39.577+0000 7f18a7c6b700 1 --2- 172.21.15.100:0/2404901727 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f18a0108450 0x7f18a01a5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:39.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:39.577+0000 7f18a7c6b700 1 --2- 172.21.15.100:0/2404901727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18a01023d0 0x7f18a019a8f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:39.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:39.577+0000 7f18a7c6b700 1 --2- 172.21.15.100:0/2404901727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f18a0102d90 0x7f18a019ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:39.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:39.577+0000 7f18a7c6b700 1 -- 172.21.15.100:0/2404901727 >> 172.21.15.100:0/2404901727 conn(0x7f18a0078410 msgr2=0x7f18a0101440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:13:39.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:39.577+0000 7f18a7c6b700 1 -- 172.21.15.100:0/2404901727 shutdown_connections 2024-05-10T11:13:39.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:39.578+0000 7f18a7c6b700 1 -- 172.21.15.100:0/2404901727 wait complete. 2024-05-10T11:13:39.579 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 3 2024-05-10T11:13:39.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:39 smithi178 ceph-mon[33359]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:13:39.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:39 smithi188 ceph-mon[34075]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:13:40.519 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:40 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/2404901727' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:13:40.519 INFO:journalctl@ceph.mds.j.smithi100.stdout:May 10 11:13:40 smithi100 systemd[1]: Starting Ceph mds.j for 20a542cc-0ebd-11ef-bc98-c7b262605968... 2024-05-10T11:13:40.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:40 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/2404901727' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:13:40.769 INFO:journalctl@ceph.mds.j.smithi100.stdout:May 10 11:13:40 smithi100 podman[68093]: 2024-05-10 11:13:40.638300393 +0000 UTC m=+0.018770283 image pull ee5e4f21ef58047cb638cccd5cfbe8f6f43412b56c94b1d7fbd9cf3cb90a53a5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f 2024-05-10T11:13:40.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:40 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/2404901727' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:13:41.038 INFO:journalctl@ceph.mds.j.smithi100.stdout:May 10 11:13:40 smithi100 podman[68093]: 2024-05-10 11:13:40.77041584 +0000 UTC m=+0.150885711 container create cfaf0ab1a1056904a0b34e7fe9dbad07a7e1b65b5cdf8f41d016b2b57c0003ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-j, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-05-10T11:13:41.333 INFO:journalctl@ceph.mds.j.smithi100.stdout:May 10 11:13:41 smithi100 podman[68093]: 2024-05-10 11:13:41.039880776 +0000 UTC m=+0.420350652 container init cfaf0ab1a1056904a0b34e7fe9dbad07a7e1b65b5cdf8f41d016b2b57c0003ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-j, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5) 2024-05-10T11:13:41.333 INFO:journalctl@ceph.mds.j.smithi100.stdout:May 10 11:13:41 smithi100 podman[68093]: 2024-05-10 11:13:41.043063785 +0000 UTC m=+0.423533657 container start cfaf0ab1a1056904a0b34e7fe9dbad07a7e1b65b5cdf8f41d016b2b57c0003ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-j, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-10T11:13:41.334 INFO:journalctl@ceph.mds.j.smithi100.stdout:May 10 11:13:41 smithi100 ceph-mds[68113]: -- 172.21.15.100:0/884058284 <== mon.0 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5605234f3a40 con 0x5605234c7000 2024-05-10T11:13:41.334 INFO:journalctl@ceph.mds.j.smithi100.stdout:May 10 11:13:41 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-j[68109]: starting mds.j at 2024-05-10T11:13:41.334 INFO:journalctl@ceph.mds.j.smithi100.stdout:May 10 11:13:41 smithi100 bash[68093]: cfaf0ab1a1056904a0b34e7fe9dbad07a7e1b65b5cdf8f41d016b2b57c0003ff 2024-05-10T11:13:41.334 INFO:journalctl@ceph.mds.j.smithi100.stdout:May 10 11:13:41 smithi100 systemd[1]: Started Ceph mds.j for 20a542cc-0ebd-11ef-bc98-c7b262605968. 2024-05-10T11:13:41.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:41 smithi188 ceph-mon[34075]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:13:41.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:41 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:41.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:41 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:41.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:41 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:41.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:41 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T11:13:41.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:41 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T11:13:41.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:41 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:13:41.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:41 smithi178 ceph-mon[33359]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:13:41.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:41 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:41.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:41 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:41.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:41 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:41.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:41 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T11:13:41.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:41 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T11:13:41.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:41 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:13:41.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:41 smithi100 ceph-mon[30296]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:13:41.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:41 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:41.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:41 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:41.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:41 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:41.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:41 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T11:13:41.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:41 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T11:13:41.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:41 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:13:42.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:42 smithi100 ceph-mon[30296]: Deploying daemon mds.d on smithi100 2024-05-10T11:13:42.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:42 smithi100 ceph-mon[30296]: mds.? [v2:172.21.15.100:6834/2072225012,v1:172.21.15.100:6835/2072225012] up:boot 2024-05-10T11:13:42.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:42 smithi100 ceph-mon[30296]: fsmap 3 up:standby 2024-05-10T11:13:42.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:42 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-10T11:13:42.546 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T11:13:42.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:42 smithi178 ceph-mon[33359]: Deploying daemon mds.d on smithi100 2024-05-10T11:13:42.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:42 smithi178 ceph-mon[33359]: mds.? [v2:172.21.15.100:6834/2072225012,v1:172.21.15.100:6835/2072225012] up:boot 2024-05-10T11:13:42.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:42 smithi178 ceph-mon[33359]: fsmap 3 up:standby 2024-05-10T11:13:42.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:42 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-10T11:13:42.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:42 smithi188 ceph-mon[34075]: Deploying daemon mds.d on smithi100 2024-05-10T11:13:42.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:42 smithi188 ceph-mon[34075]: mds.? [v2:172.21.15.100:6834/2072225012,v1:172.21.15.100:6835/2072225012] up:boot 2024-05-10T11:13:42.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:42 smithi188 ceph-mon[34075]: fsmap 3 up:standby 2024-05-10T11:13:42.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:42 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-10T11:13:43.025 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:13:43.513 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:43 smithi100 ceph-mon[30296]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:13:43.680 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:43 smithi188 ceph-mon[34075]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:13:43.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:43 smithi178 ceph-mon[33359]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:13:44.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:44.833+0000 7fc0393f0700 1 -- 172.21.15.100:0/3718391568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0341042f0 msgr2=0x7fc0341046d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:13:44.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:44.833+0000 7fc0393f0700 1 --2- 172.21.15.100:0/3718391568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0341042f0 0x7fc0341046d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc024009990 tx=0x7fc024009ca0 comp rx=0 tx=0).stop 2024-05-10T11:13:44.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:44.833+0000 7fc0393f0700 1 -- 172.21.15.100:0/3718391568 shutdown_connections 2024-05-10T11:13:44.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:44.833+0000 7fc0393f0700 1 --2- 172.21.15.100:0/3718391568 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc034102d90 0x7fc03410f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:44.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:44.833+0000 7fc0393f0700 1 --2- 172.21.15.100:0/3718391568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0341042f0 0x7fc0341046d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:44.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:44.833+0000 7fc0393f0700 1 --2- 172.21.15.100:0/3718391568 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0341023d0 0x7fc034102850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:44.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:44.833+0000 7fc0393f0700 1 -- 172.21.15.100:0/3718391568 >> 172.21.15.100:0/3718391568 conn(0x7fc034078410 msgr2=0x7fc034078820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:13:44.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:44.833+0000 7fc0393f0700 1 -- 172.21.15.100:0/3718391568 shutdown_connections 2024-05-10T11:13:44.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:44.833+0000 7fc0393f0700 1 -- 172.21.15.100:0/3718391568 wait complete. 2024-05-10T11:13:44.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:44.834+0000 7fc0393f0700 1 Processor -- start 2024-05-10T11:13:44.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:44.834+0000 7fc0393f0700 1 -- start start 2024-05-10T11:13:44.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:44.834+0000 7fc0393f0700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0341023d0 0x7fc03419f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:13:44.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:44.834+0000 7fc0393f0700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc034102d90 0x7fc03419fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:13:44.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:44.834+0000 7fc0393f0700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0341042f0 0x7fc034199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:13:44.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:44.834+0000 7fc0393f0700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc034112560 con 0x7fc034102d90 2024-05-10T11:13:44.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:44.834+0000 7fc0393f0700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc0341123e0 con 0x7fc0341042f0 2024-05-10T11:13:44.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:44.834+0000 7fc0393f0700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc0341126e0 con 0x7fc0341023d0 2024-05-10T11:13:44.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:44.835+0000 7fc0327fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc034102d90 0x7fc03419fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:13:44.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:44.835+0000 7fc0327fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc034102d90 0x7fc03419fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51678/0 (socket says 172.21.15.100:51678) 2024-05-10T11:13:44.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:44.835+0000 7fc032ffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0341023d0 0x7fc03419f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:13:44.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:44.835+0000 7fc0327fc700 1 -- 172.21.15.100:0/1114799227 learned_addr learned my addr 172.21.15.100:0/1114799227 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:13:44.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:44.835+0000 7fc0337fe700 1 --2- 172.21.15.100:0/1114799227 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0341042f0 0x7fc034199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:13:44.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:44.835+0000 7fc0327fc700 1 -- 172.21.15.100:0/1114799227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0341023d0 msgr2=0x7fc03419f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:13:44.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:44.835+0000 7fc0327fc700 1 --2- 172.21.15.100:0/1114799227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0341023d0 0x7fc03419f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:44.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:44.835+0000 7fc0327fc700 1 -- 172.21.15.100:0/1114799227 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0341042f0 msgr2=0x7fc034199a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:13:44.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:44.835+0000 7fc0327fc700 1 --2- 172.21.15.100:0/1114799227 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0341042f0 0x7fc034199a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:44.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:44.835+0000 7fc0327fc700 1 -- 172.21.15.100:0/1114799227 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc03419a160 con 0x7fc034102d90 2024-05-10T11:13:44.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:44.835+0000 7fc0327fc700 1 --2- 172.21.15.100:0/1114799227 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc034102d90 0x7fc03419fd30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc02800b530 tx=0x7fc02800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:13:44.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:44.835+0000 7fc01bfff700 1 -- 172.21.15.100:0/1114799227 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc028015020 con 0x7fc034102d90 2024-05-10T11:13:44.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:44.835+0000 7fc0393f0700 1 -- 172.21.15.100:0/1114799227 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc03419a450 con 0x7fc034102d90 2024-05-10T11:13:44.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:44.835+0000 7fc01bfff700 1 -- 172.21.15.100:0/1114799227 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc028011cc0 con 0x7fc034102d90 2024-05-10T11:13:44.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:44.835+0000 7fc01bfff700 1 -- 172.21.15.100:0/1114799227 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc028010c80 con 0x7fc034102d90 2024-05-10T11:13:44.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:44.835+0000 7fc0393f0700 1 -- 172.21.15.100:0/1114799227 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0341a6740 con 0x7fc034102d90 2024-05-10T11:13:44.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:44.836+0000 7fc0393f0700 1 -- 172.21.15.100:0/1114799227 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc03406a190 con 0x7fc034102d90 2024-05-10T11:13:44.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:44.838+0000 7fc01bfff700 1 -- 172.21.15.100:0/1114799227 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fc028018020 con 0x7fc034102d90 2024-05-10T11:13:44.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:44.838+0000 7fc01bfff700 1 --2- 172.21.15.100:0/1114799227 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc01c0b2880 0x7fc01c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:13:44.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:44.838+0000 7fc032ffd700 1 --2- 172.21.15.100:0/1114799227 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc01c0b2880 0x7fc01c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:13:44.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:44.838+0000 7fc01bfff700 1 -- 172.21.15.100:0/1114799227 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7fc0280e19f0 con 0x7fc034102d90 2024-05-10T11:13:44.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:44.839+0000 7fc032ffd700 1 --2- 172.21.15.100:0/1114799227 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc01c0b2880 0x7fc01c0b4d40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc034108b60 tx=0x7fc024005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:13:44.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:44.842+0000 7fc01bfff700 1 -- 172.21.15.100:0/1114799227 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fc02801f020 con 0x7fc034102d90 2024-05-10T11:13:45.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:45.103+0000 7fc0393f0700 1 -- 172.21.15.100:0/1114799227 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc03419aee0 con 0x7fc034102d90 2024-05-10T11:13:45.104 INFO:journalctl@ceph.mds.d.smithi100.stdout:May 10 11:13:44 smithi100 podman[68515]: 2024-05-10 11:13:44.983632129 +0000 UTC m=+0.362203888 container create 6bdce29d310a345ccbc028835efff3a2c069e22d88c609a294a9b35a4ed5ccd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-05-10T11:13:45.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:45.104+0000 7fc01bfff700 1 -- 172.21.15.100:0/1114799227 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3052 (secure 0 0 0) 0x7fc0280a9bb0 con 0x7fc034102d90 2024-05-10T11:13:45.104 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:13:45.104 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2072225012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2072225012},{"type":"v1","addr":"172.21.15.100:6835","nonce":2072225012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/2741810791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2741810791},{"type":"v1","addr":"172.21.15.178:6833","nonce":2741810791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1704891041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1704891041},{"type":"v1","addr":"172.21.15.188:6833","nonce":1704891041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-10T11:13:45.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:45.107+0000 7fc0393f0700 1 -- 172.21.15.100:0/1114799227 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc01c0b2880 msgr2=0x7fc01c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:13:45.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:45.107+0000 7fc0393f0700 1 --2- 172.21.15.100:0/1114799227 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc01c0b2880 0x7fc01c0b4d40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc034108b60 tx=0x7fc024005960 comp rx=0 tx=0).stop 2024-05-10T11:13:45.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:45.107+0000 7fc0393f0700 1 -- 172.21.15.100:0/1114799227 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc034102d90 msgr2=0x7fc03419fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:13:45.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:45.107+0000 7fc0393f0700 1 --2- 172.21.15.100:0/1114799227 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc034102d90 0x7fc03419fd30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc02800b530 tx=0x7fc02800b840 comp rx=0 tx=0).stop 2024-05-10T11:13:45.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:45.107+0000 7fc0393f0700 1 -- 172.21.15.100:0/1114799227 shutdown_connections 2024-05-10T11:13:45.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:45.107+0000 7fc0393f0700 1 --2- 172.21.15.100:0/1114799227 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc01c0b2880 0x7fc01c0b4d40 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:45.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:45.107+0000 7fc0393f0700 1 --2- 172.21.15.100:0/1114799227 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0341042f0 0x7fc034199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:45.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:45.107+0000 7fc0393f0700 1 --2- 172.21.15.100:0/1114799227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0341023d0 0x7fc03419f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:45.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:45.107+0000 7fc0393f0700 1 --2- 172.21.15.100:0/1114799227 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc034102d90 0x7fc03419fd30 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:45.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:45.107+0000 7fc0393f0700 1 -- 172.21.15.100:0/1114799227 >> 172.21.15.100:0/1114799227 conn(0x7fc034078410 msgr2=0x7fc03406c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:13:45.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:45.107+0000 7fc0393f0700 1 -- 172.21.15.100:0/1114799227 shutdown_connections 2024-05-10T11:13:45.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:45.107+0000 7fc0393f0700 1 -- 172.21.15.100:0/1114799227 wait complete. 2024-05-10T11:13:45.108 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 4 2024-05-10T11:13:45.375 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:45 smithi100 ceph-mon[30296]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:13:45.375 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:45 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/1114799227' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:13:45.375 INFO:journalctl@ceph.mds.d.smithi100.stdout:May 10 11:13:45 smithi100 podman[68515]: 2024-05-10 11:13:45.245647575 +0000 UTC m=+0.624219342 container init 6bdce29d310a345ccbc028835efff3a2c069e22d88c609a294a9b35a4ed5ccd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-d, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, ceph=True) 2024-05-10T11:13:45.375 INFO:journalctl@ceph.mds.d.smithi100.stdout:May 10 11:13:45 smithi100 podman[68515]: 2024-05-10 11:13:45.249007693 +0000 UTC m=+0.627579456 container start 6bdce29d310a345ccbc028835efff3a2c069e22d88c609a294a9b35a4ed5ccd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-d, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True) 2024-05-10T11:13:45.375 INFO:journalctl@ceph.mds.d.smithi100.stdout:May 10 11:13:45 smithi100 ceph-mds[68571]: -- 172.21.15.100:0/2488174815 <== mon.0 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b2ea461a40 con 0x55b2ea434800 2024-05-10T11:13:45.375 INFO:journalctl@ceph.mds.d.smithi100.stdout:May 10 11:13:45 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-d[68554]: starting mds.d at 2024-05-10T11:13:45.375 INFO:journalctl@ceph.mds.d.smithi100.stdout:May 10 11:13:45 smithi100 bash[68515]: 6bdce29d310a345ccbc028835efff3a2c069e22d88c609a294a9b35a4ed5ccd5 2024-05-10T11:13:45.376 INFO:journalctl@ceph.mds.d.smithi100.stdout:May 10 11:13:45 smithi100 systemd[1]: Started Ceph mds.d for 20a542cc-0ebd-11ef-bc98-c7b262605968. 2024-05-10T11:13:45.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:45 smithi178 ceph-mon[33359]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:13:45.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:45 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/1114799227' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:13:45.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:45 smithi188 ceph-mon[34075]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:13:45.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:45 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/1114799227' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:13:46.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:46 smithi178 ceph-mon[33359]: mds.? [v2:172.21.15.100:6836/401939540,v1:172.21.15.100:6837/401939540] up:boot 2024-05-10T11:13:46.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:46 smithi178 ceph-mon[33359]: fsmap 4 up:standby 2024-05-10T11:13:46.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:46 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-10T11:13:46.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:46 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:46.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:46 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:46.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:46 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:46.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:46 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T11:13:46.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:46 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T11:13:46.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:46 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:13:46.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:46 smithi178 ceph-mon[33359]: Deploying daemon mds.e on smithi178 2024-05-10T11:13:46.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:46 smithi188 ceph-mon[34075]: mds.? [v2:172.21.15.100:6836/401939540,v1:172.21.15.100:6837/401939540] up:boot 2024-05-10T11:13:46.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:46 smithi188 ceph-mon[34075]: fsmap 4 up:standby 2024-05-10T11:13:46.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:46 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-10T11:13:46.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:46 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:46.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:46 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:46.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:46 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:46.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:46 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T11:13:46.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:46 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T11:13:46.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:46 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:13:46.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:46 smithi188 ceph-mon[34075]: Deploying daemon mds.e on smithi178 2024-05-10T11:13:46.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:46 smithi100 ceph-mon[30296]: mds.? [v2:172.21.15.100:6836/401939540,v1:172.21.15.100:6837/401939540] up:boot 2024-05-10T11:13:46.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:46 smithi100 ceph-mon[30296]: fsmap 4 up:standby 2024-05-10T11:13:46.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:46 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-10T11:13:46.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:46 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:46.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:46 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:46.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:46 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:46.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:46 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T11:13:46.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:46 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T11:13:46.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:46 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:13:46.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:46 smithi100 ceph-mon[30296]: Deploying daemon mds.e on smithi178 2024-05-10T11:13:47.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:47 smithi178 ceph-mon[33359]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:13:47.752 INFO:journalctl@ceph.mds.e.smithi178.stdout:May 10 11:13:47 smithi178 systemd[1]: Starting Ceph mds.e for 20a542cc-0ebd-11ef-bc98-c7b262605968... 2024-05-10T11:13:47.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:47 smithi188 ceph-mon[34075]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:13:47.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:47 smithi100 ceph-mon[30296]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:13:48.046 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T11:13:48.129 INFO:journalctl@ceph.mds.e.smithi178.stdout:May 10 11:13:47 smithi178 podman[63298]: 2024-05-10 11:13:47.781819523 +0000 UTC m=+0.018006150 image pull ee5e4f21ef58047cb638cccd5cfbe8f6f43412b56c94b1d7fbd9cf3cb90a53a5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f 2024-05-10T11:13:48.129 INFO:journalctl@ceph.mds.e.smithi178.stdout:May 10 11:13:47 smithi178 podman[63298]: 2024-05-10 11:13:47.886047161 +0000 UTC m=+0.122233817 container create b445c9f9eac521c55765b83e47ce5c952ff374ccaf60c4a76f699cd2cc2cf982 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-e, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-10T11:13:48.129 INFO:journalctl@ceph.mds.e.smithi178.stdout:May 10 11:13:48 smithi178 podman[63298]: 2024-05-10 11:13:48.130308542 +0000 UTC m=+0.366495154 container init b445c9f9eac521c55765b83e47ce5c952ff374ccaf60c4a76f699cd2cc2cf982 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-e, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True) 2024-05-10T11:13:48.302 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:13:48.502 INFO:journalctl@ceph.mds.e.smithi178.stdout:May 10 11:13:48 smithi178 podman[63298]: 2024-05-10 11:13:48.13319915 +0000 UTC m=+0.369385772 container start b445c9f9eac521c55765b83e47ce5c952ff374ccaf60c4a76f699cd2cc2cf982 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507) 2024-05-10T11:13:48.503 INFO:journalctl@ceph.mds.e.smithi178.stdout:May 10 11:13:48 smithi178 ceph-mds[63318]: -- 172.21.15.178:0/3873571456 <== mon.2 v2:172.21.15.178:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e3c654ba40 con 0x55e3c651f000 2024-05-10T11:13:48.503 INFO:journalctl@ceph.mds.e.smithi178.stdout:May 10 11:13:48 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-e[63314]: starting mds.e at 2024-05-10T11:13:48.503 INFO:journalctl@ceph.mds.e.smithi178.stdout:May 10 11:13:48 smithi178 bash[63298]: b445c9f9eac521c55765b83e47ce5c952ff374ccaf60c4a76f699cd2cc2cf982 2024-05-10T11:13:48.503 INFO:journalctl@ceph.mds.e.smithi178.stdout:May 10 11:13:48 smithi178 systemd[1]: Started Ceph mds.e for 20a542cc-0ebd-11ef-bc98-c7b262605968. 2024-05-10T11:13:49.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:49 smithi100 ceph-mon[30296]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:13:49.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:49 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:49.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:49 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:49.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:49 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:49.539 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:49 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T11:13:49.539 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:49 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T11:13:49.539 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:49 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:13:49.539 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:49 smithi100 ceph-mon[30296]: Deploying daemon mds.f on smithi188 2024-05-10T11:13:49.539 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:49 smithi100 ceph-mon[30296]: mds.? [v2:172.21.15.178:6834/2124076312,v1:172.21.15.178:6835/2124076312] up:boot 2024-05-10T11:13:49.539 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:49 smithi100 ceph-mon[30296]: fsmap 5 up:standby 2024-05-10T11:13:49.539 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:49 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-10T11:13:49.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:49 smithi178 ceph-mon[33359]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:13:49.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:49 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:49.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:49 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:49.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:49 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:49.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:49 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T11:13:49.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:49 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T11:13:49.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:49 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:13:49.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:49 smithi178 ceph-mon[33359]: Deploying daemon mds.f on smithi188 2024-05-10T11:13:49.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:49 smithi178 ceph-mon[33359]: mds.? [v2:172.21.15.178:6834/2124076312,v1:172.21.15.178:6835/2124076312] up:boot 2024-05-10T11:13:49.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:49 smithi178 ceph-mon[33359]: fsmap 5 up:standby 2024-05-10T11:13:49.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:49 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-10T11:13:49.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:49 smithi188 ceph-mon[34075]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:13:49.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:49 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:49.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:49 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:49.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:49 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:49.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:49 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T11:13:49.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:49 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T11:13:49.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:49 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:13:49.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:49 smithi188 ceph-mon[34075]: Deploying daemon mds.f on smithi188 2024-05-10T11:13:49.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:49 smithi188 ceph-mon[34075]: mds.? [v2:172.21.15.178:6834/2124076312,v1:172.21.15.178:6835/2124076312] up:boot 2024-05-10T11:13:49.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:49 smithi188 ceph-mon[34075]: fsmap 5 up:standby 2024-05-10T11:13:49.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:49 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-10T11:13:50.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:49.998+0000 7f61945e9700 1 -- 172.21.15.100:0/95373008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f618c1023d0 msgr2=0x7f618c102850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:13:50.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:49.998+0000 7f61945e9700 1 --2- 172.21.15.100:0/95373008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f618c1023d0 0x7f618c102850 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6180009990 tx=0x7f6180009ca0 comp rx=0 tx=0).stop 2024-05-10T11:13:50.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:49.998+0000 7f61945e9700 1 -- 172.21.15.100:0/95373008 shutdown_connections 2024-05-10T11:13:50.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:49.998+0000 7f61945e9700 1 --2- 172.21.15.100:0/95373008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f618c102d90 0x7f618c10f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:50.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:49.998+0000 7f61945e9700 1 --2- 172.21.15.100:0/95373008 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f618c108440 0x7f618c108820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:50.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:49.998+0000 7f61945e9700 1 --2- 172.21.15.100:0/95373008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f618c1023d0 0x7f618c102850 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:50.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:49.998+0000 7f61945e9700 1 -- 172.21.15.100:0/95373008 >> 172.21.15.100:0/95373008 conn(0x7f618c0793e0 msgr2=0x7f618c0797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:13:50.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:49.998+0000 7f61945e9700 1 -- 172.21.15.100:0/95373008 shutdown_connections 2024-05-10T11:13:50.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:49.998+0000 7f61945e9700 1 -- 172.21.15.100:0/95373008 wait complete. 2024-05-10T11:13:50.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:49.999+0000 7f61945e9700 1 Processor -- start 2024-05-10T11:13:50.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:49.999+0000 7f61945e9700 1 -- start start 2024-05-10T11:13:50.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:49.999+0000 7f61945e9700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f618c1023d0 0x7f618c19f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:13:50.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:49.999+0000 7f61945e9700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f618c102d90 0x7f618c19fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:13:50.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:49.999+0000 7f61945e9700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f618c108440 0x7f618c199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:13:50.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:49.999+0000 7f61945e9700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f618c112410 con 0x7f618c102d90 2024-05-10T11:13:50.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:49.999+0000 7f61945e9700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f618c112290 con 0x7f618c108440 2024-05-10T11:13:50.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:49.999+0000 7f61945e9700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f618c112590 con 0x7f618c1023d0 2024-05-10T11:13:50.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:49.999+0000 7f6191b84700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f618c102d90 0x7f618c19fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:13:50.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:49.999+0000 7f6191b84700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f618c102d90 0x7f618c19fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57196/0 (socket says 172.21.15.100:57196) 2024-05-10T11:13:50.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:49.999+0000 7f6191b84700 1 -- 172.21.15.100:0/1418462064 learned_addr learned my addr 172.21.15.100:0/1418462064 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:13:50.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:49.999+0000 7f6192385700 1 --2- 172.21.15.100:0/1418462064 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f618c1023d0 0x7f618c19f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:13:50.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:49.999+0000 7f6192b86700 1 --2- 172.21.15.100:0/1418462064 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f618c108440 0x7f618c199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:13:50.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:49.999+0000 7f6191b84700 1 -- 172.21.15.100:0/1418462064 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f618c1023d0 msgr2=0x7f618c19f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:13:50.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:49.999+0000 7f6191b84700 1 --2- 172.21.15.100:0/1418462064 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f618c1023d0 0x7f618c19f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:50.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:49.999+0000 7f6191b84700 1 -- 172.21.15.100:0/1418462064 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f618c108440 msgr2=0x7f618c199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:13:50.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:49.999+0000 7f6191b84700 1 --2- 172.21.15.100:0/1418462064 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f618c108440 0x7f618c199920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:50.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:49.999+0000 7f6191b84700 1 -- 172.21.15.100:0/1418462064 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f618c19a080 con 0x7f618c102d90 2024-05-10T11:13:50.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:49.999+0000 7f6192b86700 1 --2- 172.21.15.100:0/1418462064 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f618c108440 0x7f618c199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:13:50.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:50.000+0000 7f6191b84700 1 --2- 172.21.15.100:0/1418462064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f618c102d90 0x7f618c19fca0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f61800057c0 tx=0x7f61800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:13:50.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:50.000+0000 7f618b7fe700 1 -- 172.21.15.100:0/1418462064 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6180027070 con 0x7f618c102d90 2024-05-10T11:13:50.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:50.000+0000 7f618b7fe700 1 -- 172.21.15.100:0/1418462064 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f618000fc00 con 0x7f618c102d90 2024-05-10T11:13:50.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:50.000+0000 7f61945e9700 1 -- 172.21.15.100:0/1418462064 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f618c19a310 con 0x7f618c102d90 2024-05-10T11:13:50.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:50.000+0000 7f618b7fe700 1 -- 172.21.15.100:0/1418462064 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6180022b30 con 0x7f618c102d90 2024-05-10T11:13:50.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:50.000+0000 7f61945e9700 1 -- 172.21.15.100:0/1418462064 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f618c1a6630 con 0x7f618c102d90 2024-05-10T11:13:50.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:50.001+0000 7f61945e9700 1 -- 172.21.15.100:0/1418462064 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f618c103af0 con 0x7f618c102d90 2024-05-10T11:13:50.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:50.002+0000 7f618b7fe700 1 -- 172.21.15.100:0/1418462064 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f618002a070 con 0x7f618c102d90 2024-05-10T11:13:50.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:50.003+0000 7f618b7fe700 1 --2- 172.21.15.100:0/1418462064 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f61740b2880 0x7f61740b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:13:50.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:50.003+0000 7f6192385700 1 --2- 172.21.15.100:0/1418462064 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f61740b2880 0x7f61740b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:13:50.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:50.003+0000 7f618b7fe700 1 -- 172.21.15.100:0/1418462064 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f61800e2ea0 con 0x7f618c102d90 2024-05-10T11:13:50.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:50.003+0000 7f6192385700 1 --2- 172.21.15.100:0/1418462064 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f61740b2880 0x7f61740b4d40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f618c19aa60 tx=0x7f61840073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:13:50.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:50.007+0000 7f618b7fe700 1 -- 172.21.15.100:0/1418462064 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f6180031080 con 0x7f618c102d90 2024-05-10T11:13:50.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:50.260+0000 7f61945e9700 1 -- 172.21.15.100:0/1418462064 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f618c1a6800 con 0x7f618c102d90 2024-05-10T11:13:50.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:50.261+0000 7f618b7fe700 1 -- 172.21.15.100:0/1418462064 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4705 (secure 0 0 0) 0x7f6180013070 con 0x7f618c102d90 2024-05-10T11:13:50.261 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:13:50.261 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2072225012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2072225012},{"type":"v1","addr":"172.21.15.100:6835","nonce":2072225012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/401939540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":401939540},{"type":"v1","addr":"172.21.15.100:6837","nonce":401939540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/2741810791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2741810791},{"type":"v1","addr":"172.21.15.178:6833","nonce":2741810791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1704891041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1704891041},{"type":"v1","addr":"172.21.15.188:6833","nonce":1704891041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24395,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2124076312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2124076312},{"type":"v1","addr":"172.21.15.178:6835","nonce":2124076312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-05-10T11:13:50.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:50.263+0000 7f61945e9700 1 -- 172.21.15.100:0/1418462064 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f61740b2880 msgr2=0x7f61740b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:13:50.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:50.263+0000 7f61945e9700 1 --2- 172.21.15.100:0/1418462064 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f61740b2880 0x7f61740b4d40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f618c19aa60 tx=0x7f61840073d0 comp rx=0 tx=0).stop 2024-05-10T11:13:50.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:50.263+0000 7f61945e9700 1 -- 172.21.15.100:0/1418462064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f618c102d90 msgr2=0x7f618c19fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:13:50.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:50.263+0000 7f61945e9700 1 --2- 172.21.15.100:0/1418462064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f618c102d90 0x7f618c19fca0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f61800057c0 tx=0x7f61800057f0 comp rx=0 tx=0).stop 2024-05-10T11:13:50.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:50.263+0000 7f61945e9700 1 -- 172.21.15.100:0/1418462064 shutdown_connections 2024-05-10T11:13:50.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:50.263+0000 7f61945e9700 1 --2- 172.21.15.100:0/1418462064 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f61740b2880 0x7f61740b4d40 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:50.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:50.263+0000 7f61945e9700 1 --2- 172.21.15.100:0/1418462064 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f618c108440 0x7f618c199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:50.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:50.263+0000 7f61945e9700 1 --2- 172.21.15.100:0/1418462064 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f618c1023d0 0x7f618c19f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:50.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:50.263+0000 7f61945e9700 1 --2- 172.21.15.100:0/1418462064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f618c102d90 0x7f618c19fca0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:50.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:50.263+0000 7f61945e9700 1 -- 172.21.15.100:0/1418462064 >> 172.21.15.100:0/1418462064 conn(0x7f618c0793e0 msgr2=0x7f618c106920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:13:50.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:50.264+0000 7f61945e9700 1 -- 172.21.15.100:0/1418462064 shutdown_connections 2024-05-10T11:13:50.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:50.264+0000 7f61945e9700 1 -- 172.21.15.100:0/1418462064 wait complete. 2024-05-10T11:13:50.264 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 6 2024-05-10T11:13:50.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:50 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/1418462064' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:13:50.529 INFO:journalctl@ceph.mds.f.smithi188.stdout:May 10 11:13:50 smithi188 systemd[1]: Starting Ceph mds.f for 20a542cc-0ebd-11ef-bc98-c7b262605968... 2024-05-10T11:13:50.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:50 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/1418462064' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:13:50.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:50 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/1418462064' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:13:50.922 INFO:journalctl@ceph.mds.f.smithi188.stdout:May 10 11:13:50 smithi188 podman[63925]: 2024-05-10 11:13:50.565801763 +0000 UTC m=+0.018106993 image pull ee5e4f21ef58047cb638cccd5cfbe8f6f43412b56c94b1d7fbd9cf3cb90a53a5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f 2024-05-10T11:13:50.922 INFO:journalctl@ceph.mds.f.smithi188.stdout:May 10 11:13:50 smithi188 podman[63925]: 2024-05-10 11:13:50.687091032 +0000 UTC m=+0.139396289 container create 172d7613ab043329f870e210e9e87ffb3cd7f44b242e16848ec6acc269ce6a32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-f, org.label-schema.build-date=20240507, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.vendor=CentOS, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-10T11:13:50.922 INFO:journalctl@ceph.mds.f.smithi188.stdout:May 10 11:13:50 smithi188 podman[63925]: 2024-05-10 11:13:50.92278541 +0000 UTC m=+0.375090625 container init 172d7613ab043329f870e210e9e87ffb3cd7f44b242e16848ec6acc269ce6a32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-f, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-05-10T11:13:51.278 INFO:journalctl@ceph.mds.f.smithi188.stdout:May 10 11:13:50 smithi188 podman[63925]: 2024-05-10 11:13:50.925558747 +0000 UTC m=+0.377863959 container start 172d7613ab043329f870e210e9e87ffb3cd7f44b242e16848ec6acc269ce6a32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-f, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-05-10T11:13:51.278 INFO:journalctl@ceph.mds.f.smithi188.stdout:May 10 11:13:50 smithi188 ceph-mds[63944]: -- 172.21.15.188:0/2878390466 <== mon.1 v2:172.21.15.188:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5610d289fa40 con 0x5610d2872c00 2024-05-10T11:13:51.278 INFO:journalctl@ceph.mds.f.smithi188.stdout:May 10 11:13:50 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-f[63940]: starting mds.f at 2024-05-10T11:13:51.279 INFO:journalctl@ceph.mds.f.smithi188.stdout:May 10 11:13:51 smithi188 bash[63925]: 172d7613ab043329f870e210e9e87ffb3cd7f44b242e16848ec6acc269ce6a32 2024-05-10T11:13:51.279 INFO:journalctl@ceph.mds.f.smithi188.stdout:May 10 11:13:51 smithi188 systemd[1]: Started Ceph mds.f for 20a542cc-0ebd-11ef-bc98-c7b262605968. 2024-05-10T11:13:51.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:51 smithi100 ceph-mon[30296]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:13:51.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:51 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:51.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:51 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:51.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:51 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:51.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:51 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T11:13:51.539 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:51 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T11:13:51.539 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:51 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:13:51.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:51 smithi178 ceph-mon[33359]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:13:51.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:51 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:51.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:51 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:51.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:51 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:51.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:51 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T11:13:51.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:51 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T11:13:51.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:51 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:13:51.777 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:51 smithi188 ceph-mon[34075]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:13:51.777 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:51 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:51.777 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:51 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:51.777 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:51 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:51.777 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:51 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T11:13:51.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:51 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T11:13:51.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:51 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:13:52.474 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:52 smithi188 ceph-mon[34075]: Deploying daemon mds.c on smithi188 2024-05-10T11:13:52.475 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:52 smithi188 ceph-mon[34075]: mds.? [v2:172.21.15.188:6834/515547690,v1:172.21.15.188:6835/515547690] up:boot 2024-05-10T11:13:52.475 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:52 smithi188 ceph-mon[34075]: fsmap 6 up:standby 2024-05-10T11:13:52.475 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:52 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-10T11:13:52.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:52 smithi178 ceph-mon[33359]: Deploying daemon mds.c on smithi188 2024-05-10T11:13:52.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:52 smithi178 ceph-mon[33359]: mds.? [v2:172.21.15.188:6834/515547690,v1:172.21.15.188:6835/515547690] up:boot 2024-05-10T11:13:52.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:52 smithi178 ceph-mon[33359]: fsmap 6 up:standby 2024-05-10T11:13:52.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:52 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-10T11:13:52.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:52 smithi100 ceph-mon[30296]: Deploying daemon mds.c on smithi188 2024-05-10T11:13:52.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:52 smithi100 ceph-mon[30296]: mds.? [v2:172.21.15.188:6834/515547690,v1:172.21.15.188:6835/515547690] up:boot 2024-05-10T11:13:52.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:52 smithi100 ceph-mon[30296]: fsmap 6 up:standby 2024-05-10T11:13:52.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:52 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-10T11:13:52.948 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T11:13:53.202 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:13:53.472 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:53 smithi188 ceph-mon[34075]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:13:53.596 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:53 smithi100 ceph-mon[30296]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:13:53.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:53 smithi178 ceph-mon[33359]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:13:54.028 INFO:journalctl@ceph.mds.c.smithi188.stdout:May 10 11:13:53 smithi188 systemd[1]: Started Ceph mds.c for 20a542cc-0ebd-11ef-bc98-c7b262605968. 2024-05-10T11:13:54.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:54.921+0000 7f515fd73700 1 -- 172.21.15.100:0/818079220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51581084b0 msgr2=0x7f5158108890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:13:54.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:54.921+0000 7f515fd73700 1 --2- 172.21.15.100:0/818079220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51581084b0 0x7f5158108890 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5154009990 tx=0x7f5154009ca0 comp rx=0 tx=0).stop 2024-05-10T11:13:54.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:54.922+0000 7f515fd73700 1 -- 172.21.15.100:0/818079220 shutdown_connections 2024-05-10T11:13:54.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:54.922+0000 7f515fd73700 1 --2- 172.21.15.100:0/818079220 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51581023b0 0x7f5158105f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:54.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:54.922+0000 7f515fd73700 1 --2- 172.21.15.100:0/818079220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51581084b0 0x7f5158108890 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:54.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:54.922+0000 7f515fd73700 1 --2- 172.21.15.100:0/818079220 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51581019f0 0x7f5158101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:54.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:54.922+0000 7f515fd73700 1 -- 172.21.15.100:0/818079220 >> 172.21.15.100:0/818079220 conn(0x7f51580789a0 msgr2=0x7f51580ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:13:54.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:54.922+0000 7f515fd73700 1 -- 172.21.15.100:0/818079220 shutdown_connections 2024-05-10T11:13:54.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:54.922+0000 7f515fd73700 1 -- 172.21.15.100:0/818079220 wait complete. 2024-05-10T11:13:54.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:54.922+0000 7f515fd73700 1 Processor -- start 2024-05-10T11:13:54.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:54.923+0000 7f515fd73700 1 -- start start 2024-05-10T11:13:54.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:54.923+0000 7f515fd73700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51581019f0 0x7f515819ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:13:54.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:54.923+0000 7f515fd73700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51581023b0 0x7f51581a04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:13:54.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:54.923+0000 7f515fd73700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51581084b0 0x7f51581a4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:13:54.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:54.923+0000 7f515fd73700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5158119b80 con 0x7f51581019f0 2024-05-10T11:13:54.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:54.923+0000 7f515fd73700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5158119a00 con 0x7f51581084b0 2024-05-10T11:13:54.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:54.923+0000 7f515fd73700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5158119d00 con 0x7f51581023b0 2024-05-10T11:13:54.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:54.923+0000 7f515db0f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51581019f0 0x7f515819ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:13:54.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:54.923+0000 7f515db0f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51581019f0 0x7f515819ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57212/0 (socket says 172.21.15.100:57212) 2024-05-10T11:13:54.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:54.923+0000 7f515db0f700 1 -- 172.21.15.100:0/2213383398 learned_addr learned my addr 172.21.15.100:0/2213383398 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:13:54.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:54.923+0000 7f515d30e700 1 --2- 172.21.15.100:0/2213383398 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51581023b0 0x7f51581a04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:13:54.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:54.923+0000 7f515e310700 1 --2- 172.21.15.100:0/2213383398 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51581084b0 0x7f51581a4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:13:54.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:54.923+0000 7f515db0f700 1 -- 172.21.15.100:0/2213383398 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51581023b0 msgr2=0x7f51581a04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:13:54.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:54.923+0000 7f515db0f700 1 --2- 172.21.15.100:0/2213383398 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51581023b0 0x7f51581a04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:54.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:54.923+0000 7f515db0f700 1 -- 172.21.15.100:0/2213383398 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51581084b0 msgr2=0x7f51581a4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:13:54.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:54.923+0000 7f515db0f700 1 --2- 172.21.15.100:0/2213383398 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51581084b0 0x7f51581a4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:54.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:54.923+0000 7f515db0f700 1 -- 172.21.15.100:0/2213383398 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51581a4f70 con 0x7f51581019f0 2024-05-10T11:13:54.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:54.923+0000 7f515db0f700 1 --2- 172.21.15.100:0/2213383398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51581019f0 0x7f515819ffb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5154000c00 tx=0x7f51540057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:13:54.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:54.924+0000 7f514effd700 1 -- 172.21.15.100:0/2213383398 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5154027070 con 0x7f51581019f0 2024-05-10T11:13:54.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:54.924+0000 7f515fd73700 1 -- 172.21.15.100:0/2213383398 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51581a5200 con 0x7f51581019f0 2024-05-10T11:13:54.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:54.924+0000 7f514effd700 1 -- 172.21.15.100:0/2213383398 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f515400fc00 con 0x7f51581019f0 2024-05-10T11:13:54.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:54.924+0000 7f514effd700 1 -- 172.21.15.100:0/2213383398 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5154022cd0 con 0x7f51581019f0 2024-05-10T11:13:54.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:54.924+0000 7f515fd73700 1 -- 172.21.15.100:0/2213383398 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51581acae0 con 0x7f51581019f0 2024-05-10T11:13:54.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:54.925+0000 7f515fd73700 1 -- 172.21.15.100:0/2213383398 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5158102a00 con 0x7f51581019f0 2024-05-10T11:13:54.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:54.926+0000 7f514effd700 1 -- 172.21.15.100:0/2213383398 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f515402a070 con 0x7f51581019f0 2024-05-10T11:13:54.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:54.926+0000 7f514effd700 1 --2- 172.21.15.100:0/2213383398 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f51440b2880 0x7f51440b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:13:54.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:54.927+0000 7f515d30e700 1 --2- 172.21.15.100:0/2213383398 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f51440b2880 0x7f51440b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:13:54.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:54.927+0000 7f514effd700 1 -- 172.21.15.100:0/2213383398 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f51540e2e20 con 0x7f51581019f0 2024-05-10T11:13:54.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:54.927+0000 7f515d30e700 1 --2- 172.21.15.100:0/2213383398 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f51440b2880 0x7f51440b4d40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f51581088d0 tx=0x7f514800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:13:54.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:54.930+0000 7f514effd700 1 -- 172.21.15.100:0/2213383398 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f5154031080 con 0x7f51581019f0 2024-05-10T11:13:54.930 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:54 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:54.930 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:54 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:54.930 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:54 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:54.930 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:54 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T11:13:54.930 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:54 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T11:13:54.931 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:54 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:13:54.931 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:54 smithi100 ceph-mon[30296]: Deploying daemon mds.h on smithi178 2024-05-10T11:13:54.931 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:54 smithi100 ceph-mon[30296]: mds.? [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] up:boot 2024-05-10T11:13:54.931 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:54 smithi100 ceph-mon[30296]: fsmap 7 up:standby 2024-05-10T11:13:54.931 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:54 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-10T11:13:55.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:55.180+0000 7f515fd73700 1 -- 172.21.15.100:0/2213383398 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f515806a190 con 0x7f51581019f0 2024-05-10T11:13:55.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:55.180+0000 7f514effd700 1 -- 172.21.15.100:0/2213383398 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6358 (secure 0 0 0) 0x7f515806a190 con 0x7f51581019f0 2024-05-10T11:13:55.180 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:13:55.181 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2072225012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2072225012},{"type":"v1","addr":"172.21.15.100:6835","nonce":2072225012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/401939540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":401939540},{"type":"v1","addr":"172.21.15.100:6837","nonce":401939540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/2741810791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2741810791},{"type":"v1","addr":"172.21.15.178:6833","nonce":2741810791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1704891041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1704891041},{"type":"v1","addr":"172.21.15.188:6833","nonce":1704891041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24395,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2124076312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2124076312},{"type":"v1","addr":"172.21.15.178:6835","nonce":2124076312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/515547690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":515547690},{"type":"v1","addr":"172.21.15.188:6835","nonce":515547690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24427,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1943342620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1943342620},{"type":"v1","addr":"172.21.15.188:6837","nonce":1943342620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-05-10T11:13:55.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:55.182+0000 7f515fd73700 1 -- 172.21.15.100:0/2213383398 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f51440b2880 msgr2=0x7f51440b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:13:55.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:55.182+0000 7f515fd73700 1 --2- 172.21.15.100:0/2213383398 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f51440b2880 0x7f51440b4d40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f51581088d0 tx=0x7f514800a380 comp rx=0 tx=0).stop 2024-05-10T11:13:55.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:55.182+0000 7f515fd73700 1 -- 172.21.15.100:0/2213383398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51581019f0 msgr2=0x7f515819ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:13:55.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:55.182+0000 7f515fd73700 1 --2- 172.21.15.100:0/2213383398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51581019f0 0x7f515819ffb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5154000c00 tx=0x7f51540057c0 comp rx=0 tx=0).stop 2024-05-10T11:13:55.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:55.182+0000 7f515fd73700 1 -- 172.21.15.100:0/2213383398 shutdown_connections 2024-05-10T11:13:55.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:55.182+0000 7f515fd73700 1 --2- 172.21.15.100:0/2213383398 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f51440b2880 0x7f51440b4d40 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:55.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:55.182+0000 7f515fd73700 1 --2- 172.21.15.100:0/2213383398 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51581084b0 0x7f51581a4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:55.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:55.182+0000 7f515fd73700 1 --2- 172.21.15.100:0/2213383398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51581019f0 0x7f515819ffb0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:55.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:55.182+0000 7f515fd73700 1 --2- 172.21.15.100:0/2213383398 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51581023b0 0x7f51581a04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:55.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:55.182+0000 7f515fd73700 1 -- 172.21.15.100:0/2213383398 >> 172.21.15.100:0/2213383398 conn(0x7f51580789a0 msgr2=0x7f5158103d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:13:55.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:55.183+0000 7f515fd73700 1 -- 172.21.15.100:0/2213383398 shutdown_connections 2024-05-10T11:13:55.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:55.183+0000 7f515fd73700 1 -- 172.21.15.100:0/2213383398 wait complete. 2024-05-10T11:13:55.184 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 8 2024-05-10T11:13:55.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:54 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:55.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:54 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:55.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:54 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:55.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:54 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T11:13:55.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:54 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T11:13:55.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:54 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:13:55.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:54 smithi178 ceph-mon[33359]: Deploying daemon mds.h on smithi178 2024-05-10T11:13:55.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:54 smithi178 ceph-mon[33359]: mds.? [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] up:boot 2024-05-10T11:13:55.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:54 smithi178 ceph-mon[33359]: fsmap 7 up:standby 2024-05-10T11:13:55.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:54 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-10T11:13:55.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:54 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:55.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:54 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:55.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:54 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:55.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:54 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T11:13:55.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:54 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T11:13:55.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:54 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:13:55.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:54 smithi188 ceph-mon[34075]: Deploying daemon mds.h on smithi178 2024-05-10T11:13:55.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:54 smithi188 ceph-mon[34075]: mds.? [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] up:boot 2024-05-10T11:13:55.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:54 smithi188 ceph-mon[34075]: fsmap 7 up:standby 2024-05-10T11:13:55.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:54 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-10T11:13:56.250 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:55 smithi178 ceph-mon[33359]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:13:56.250 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:55 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/2213383398' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:13:56.250 INFO:journalctl@ceph.mds.h.smithi178.stdout:May 10 11:13:56 smithi178 systemd[1]: Starting Ceph mds.h for 20a542cc-0ebd-11ef-bc98-c7b262605968... 2024-05-10T11:13:56.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:55 smithi188 ceph-mon[34075]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:13:56.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:55 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/2213383398' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:13:56.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:55 smithi100 ceph-mon[30296]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:13:56.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:55 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/2213383398' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:13:56.502 INFO:journalctl@ceph.mds.h.smithi178.stdout:May 10 11:13:56 smithi178 podman[63604]: 2024-05-10 11:13:56.248504824 +0000 UTC m=+0.020486740 image pull ee5e4f21ef58047cb638cccd5cfbe8f6f43412b56c94b1d7fbd9cf3cb90a53a5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f 2024-05-10T11:13:56.502 INFO:journalctl@ceph.mds.h.smithi178.stdout:May 10 11:13:56 smithi178 podman[63604]: 2024-05-10 11:13:56.365648197 +0000 UTC m=+0.137630103 container create 6939a764e16d473fc77acbb004d30986f0a4e6fbb3bc4f271b653143fe097d8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-h, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-10T11:13:57.002 INFO:journalctl@ceph.mds.h.smithi178.stdout:May 10 11:13:56 smithi178 podman[63604]: 2024-05-10 11:13:56.601623706 +0000 UTC m=+0.373605616 container init 6939a764e16d473fc77acbb004d30986f0a4e6fbb3bc4f271b653143fe097d8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-h, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-05-10T11:13:57.002 INFO:journalctl@ceph.mds.h.smithi178.stdout:May 10 11:13:56 smithi178 podman[63604]: 2024-05-10 11:13:56.604629898 +0000 UTC m=+0.376611806 container start 6939a764e16d473fc77acbb004d30986f0a4e6fbb3bc4f271b653143fe097d8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-h, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-05-10T11:13:57.002 INFO:journalctl@ceph.mds.h.smithi178.stdout:May 10 11:13:56 smithi178 ceph-mds[63623]: -- 172.21.15.178:0/3189243801 <== mon.2 v2:172.21.15.178:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f6a616da40 con 0x55f6a6140800 2024-05-10T11:13:57.002 INFO:journalctl@ceph.mds.h.smithi178.stdout:May 10 11:13:56 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-h[63619]: starting mds.h at 2024-05-10T11:13:57.002 INFO:journalctl@ceph.mds.h.smithi178.stdout:May 10 11:13:56 smithi178 bash[63604]: 6939a764e16d473fc77acbb004d30986f0a4e6fbb3bc4f271b653143fe097d8a 2024-05-10T11:13:57.003 INFO:journalctl@ceph.mds.h.smithi178.stdout:May 10 11:13:56 smithi178 systemd[1]: Started Ceph mds.h for 20a542cc-0ebd-11ef-bc98-c7b262605968. 2024-05-10T11:13:57.891 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T11:13:58.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:57 smithi178 ceph-mon[33359]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:13:58.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:57 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:58.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:57 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:58.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:57 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:58.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:57 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T11:13:58.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:57 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T11:13:58.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:57 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:13:58.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:57 smithi178 ceph-mon[33359]: Deploying daemon mds.i on smithi188 2024-05-10T11:13:58.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:57 smithi178 ceph-mon[33359]: mds.? [v2:172.21.15.178:6836/3224358425,v1:172.21.15.178:6837/3224358425] up:boot 2024-05-10T11:13:58.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:57 smithi178 ceph-mon[33359]: fsmap 8 up:standby 2024-05-10T11:13:58.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:57 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-10T11:13:58.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:57 smithi188 ceph-mon[34075]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:13:58.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:57 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:58.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:57 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:58.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:57 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:58.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:57 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T11:13:58.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:57 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T11:13:58.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:57 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:13:58.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:57 smithi188 ceph-mon[34075]: Deploying daemon mds.i on smithi188 2024-05-10T11:13:58.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:57 smithi188 ceph-mon[34075]: mds.? [v2:172.21.15.178:6836/3224358425,v1:172.21.15.178:6837/3224358425] up:boot 2024-05-10T11:13:58.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:57 smithi188 ceph-mon[34075]: fsmap 8 up:standby 2024-05-10T11:13:58.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:57 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-10T11:13:58.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:57 smithi100 ceph-mon[30296]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:13:58.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:57 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:58.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:57 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:58.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:57 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:58.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:57 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T11:13:58.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:57 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T11:13:58.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:57 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:13:58.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:57 smithi100 ceph-mon[30296]: Deploying daemon mds.i on smithi188 2024-05-10T11:13:58.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:57 smithi100 ceph-mon[30296]: mds.? [v2:172.21.15.178:6836/3224358425,v1:172.21.15.178:6837/3224358425] up:boot 2024-05-10T11:13:58.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:57 smithi100 ceph-mon[30296]: fsmap 8 up:standby 2024-05-10T11:13:58.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:57 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-10T11:13:58.145 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:13:59.226 INFO:journalctl@ceph.mds.i.smithi188.stdout:May 10 11:13:59 smithi188 systemd[1]: Starting Ceph mds.i for 20a542cc-0ebd-11ef-bc98-c7b262605968... 2024-05-10T11:13:59.486 INFO:journalctl@ceph.mds.i.smithi188.stdout:May 10 11:13:59 smithi188 podman[64539]: 2024-05-10 11:13:59.130170424 +0000 UTC m=+0.018532266 image pull ee5e4f21ef58047cb638cccd5cfbe8f6f43412b56c94b1d7fbd9cf3cb90a53a5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f 2024-05-10T11:13:59.486 INFO:journalctl@ceph.mds.i.smithi188.stdout:May 10 11:13:59 smithi188 podman[64539]: 2024-05-10 11:13:59.242056213 +0000 UTC m=+0.130418028 container create 9d9fc5b815d34f96217acd961717fc89203a306fcaf730ddceff700ed14d3cdd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-i, org.label-schema.license=GPLv2, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240507, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-10T11:13:59.486 INFO:journalctl@ceph.mds.i.smithi188.stdout:May 10 11:13:59 smithi188 podman[64539]: 2024-05-10 11:13:59.486641464 +0000 UTC m=+0.375003284 container init 9d9fc5b815d34f96217acd961717fc89203a306fcaf730ddceff700ed14d3cdd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-i, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, CEPH_POINT_RELEASE=, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507) 2024-05-10T11:13:59.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:59 smithi188 ceph-mon[34075]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:13:59.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:59 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:59.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:59 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:59.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:59 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:13:59.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:59 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T11:13:59.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:59 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T11:13:59.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:13:59 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:13:59.779 INFO:journalctl@ceph.mds.i.smithi188.stdout:May 10 11:13:59 smithi188 podman[64539]: 2024-05-10 11:13:59.489734303 +0000 UTC m=+0.378096126 container start 9d9fc5b815d34f96217acd961717fc89203a306fcaf730ddceff700ed14d3cdd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-05-10T11:13:59.779 INFO:journalctl@ceph.mds.i.smithi188.stdout:May 10 11:13:59 smithi188 ceph-mds[64558]: -- 172.21.15.188:0/1792343905 <== mon.1 v2:172.21.15.188:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5607ffb0ba40 con 0x5607ffadf000 2024-05-10T11:13:59.779 INFO:journalctl@ceph.mds.i.smithi188.stdout:May 10 11:13:59 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-i[64554]: starting mds.i at 2024-05-10T11:13:59.779 INFO:journalctl@ceph.mds.i.smithi188.stdout:May 10 11:13:59 smithi188 bash[64539]: 9d9fc5b815d34f96217acd961717fc89203a306fcaf730ddceff700ed14d3cdd 2024-05-10T11:13:59.779 INFO:journalctl@ceph.mds.i.smithi188.stdout:May 10 11:13:59 smithi188 systemd[1]: Started Ceph mds.i for 20a542cc-0ebd-11ef-bc98-c7b262605968. 2024-05-10T11:13:59.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:59.852+0000 7f6929e75700 1 -- 172.21.15.100:0/3146852937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69241034f0 msgr2=0x7f6924103970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:13:59.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:59.852+0000 7f6929e75700 1 --2- 172.21.15.100:0/3146852937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69241034f0 0x7f6924103970 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6918009990 tx=0x7f6918009ca0 comp rx=0 tx=0).stop 2024-05-10T11:13:59.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:59.852+0000 7f6929e75700 1 -- 172.21.15.100:0/3146852937 shutdown_connections 2024-05-10T11:13:59.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:59.852+0000 7f6929e75700 1 --2- 172.21.15.100:0/3146852937 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6924103eb0 0x7f6924110390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:59.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:59.852+0000 7f6929e75700 1 --2- 172.21.15.100:0/3146852937 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6924109600 0x7f69241099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:59.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:59.852+0000 7f6929e75700 1 --2- 172.21.15.100:0/3146852937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69241034f0 0x7f6924103970 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:59.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:59.852+0000 7f6929e75700 1 -- 172.21.15.100:0/3146852937 >> 172.21.15.100:0/3146852937 conn(0x7f692407a650 msgr2=0x7f692407aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:13:59.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:59.852+0000 7f6929e75700 1 -- 172.21.15.100:0/3146852937 shutdown_connections 2024-05-10T11:13:59.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:59.852+0000 7f6929e75700 1 -- 172.21.15.100:0/3146852937 wait complete. 2024-05-10T11:13:59.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:59.853+0000 7f6929e75700 1 Processor -- start 2024-05-10T11:13:59.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:59.853+0000 7f6929e75700 1 -- start start 2024-05-10T11:13:59.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:59.853+0000 7f6929e75700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69241034f0 0x7f69240757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:13:59.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:59.853+0000 7f6929e75700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6924103eb0 0x7f6924075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:13:59.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:59.853+0000 7f6929e75700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6924109600 0x7f6924076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:13:59.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:59.853+0000 7f6929e75700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6924113500 con 0x7f6924109600 2024-05-10T11:13:59.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:59.853+0000 7f6929e75700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6924113380 con 0x7f6924103eb0 2024-05-10T11:13:59.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:59.853+0000 7f6929e75700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6924113680 con 0x7f69241034f0 2024-05-10T11:13:59.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:59.853+0000 7f6923fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6924109600 0x7f6924076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:13:59.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:59.853+0000 7f6923fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6924109600 0x7f6924076220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43648/0 (socket says 172.21.15.100:43648) 2024-05-10T11:13:59.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:59.853+0000 7f6923fff700 1 -- 172.21.15.100:0/2352473065 learned_addr learned my addr 172.21.15.100:0/2352473065 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:13:59.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:59.853+0000 7f69237fe700 1 --2- 172.21.15.100:0/2352473065 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69241034f0 0x7f69240757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:13:59.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:59.853+0000 7f6922ffd700 1 --2- 172.21.15.100:0/2352473065 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6924103eb0 0x7f6924075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:13:59.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:59.854+0000 7f6923fff700 1 -- 172.21.15.100:0/2352473065 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69241034f0 msgr2=0x7f69240757a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:13:59.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:59.854+0000 7f6923fff700 1 --2- 172.21.15.100:0/2352473065 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69241034f0 0x7f69240757a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:59.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:59.854+0000 7f6923fff700 1 -- 172.21.15.100:0/2352473065 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6924103eb0 msgr2=0x7f6924075ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:13:59.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:59.854+0000 7f6923fff700 1 --2- 172.21.15.100:0/2352473065 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6924103eb0 0x7f6924075ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:13:59.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:59.854+0000 7f6923fff700 1 -- 172.21.15.100:0/2352473065 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69241a7ee0 con 0x7f6924109600 2024-05-10T11:13:59.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:59.854+0000 7f6922ffd700 1 --2- 172.21.15.100:0/2352473065 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6924103eb0 0x7f6924075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:13:59.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:59.854+0000 7f6923fff700 1 --2- 172.21.15.100:0/2352473065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6924109600 0x7f6924076220 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f691c00efc0 tx=0x7f691c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:13:59.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:59.854+0000 7f6920ff9700 1 -- 172.21.15.100:0/2352473065 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f691c0092e0 con 0x7f6924109600 2024-05-10T11:13:59.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:59.854+0000 7f6929e75700 1 -- 172.21.15.100:0/2352473065 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69241a8170 con 0x7f6924109600 2024-05-10T11:13:59.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:59.854+0000 7f6920ff9700 1 -- 172.21.15.100:0/2352473065 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f691c004bb0 con 0x7f6924109600 2024-05-10T11:13:59.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:59.854+0000 7f6920ff9700 1 -- 172.21.15.100:0/2352473065 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f691c004020 con 0x7f6924109600 2024-05-10T11:13:59.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:59.854+0000 7f6929e75700 1 -- 172.21.15.100:0/2352473065 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69241a8700 con 0x7f6924109600 2024-05-10T11:13:59.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:59.855+0000 7f6929e75700 1 -- 172.21.15.100:0/2352473065 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f692406a190 con 0x7f6924109600 2024-05-10T11:13:59.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:59.856+0000 7f6920ff9700 1 -- 172.21.15.100:0/2352473065 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f691c01b030 con 0x7f6924109600 2024-05-10T11:13:59.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:59.857+0000 7f6920ff9700 1 --2- 172.21.15.100:0/2352473065 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f690c0b2880 0x7f690c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:13:59.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:59.857+0000 7f6920ff9700 1 -- 172.21.15.100:0/2352473065 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f691c0e20c0 con 0x7f6924109600 2024-05-10T11:13:59.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:59.857+0000 7f69237fe700 1 --2- 172.21.15.100:0/2352473065 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f690c0b2880 0x7f690c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:13:59.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:59.858+0000 7f69237fe700 1 --2- 172.21.15.100:0/2352473065 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f690c0b2880 0x7f690c0b4d40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f691400abd0 tx=0x7f691400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:13:59.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:13:59.861+0000 7f6920ff9700 1 -- 172.21.15.100:0/2352473065 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f691c022020 con 0x7f6924109600 2024-05-10T11:14:00.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:59 smithi178 ceph-mon[33359]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:00.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:59 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:00.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:59 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:00.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:59 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:00.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:59 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T11:14:00.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:59 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T11:14:00.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:13:59 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:14:00.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:59 smithi100 ceph-mon[30296]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:00.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:59 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:00.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:59 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:00.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:59 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:00.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:59 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T11:14:00.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:59 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T11:14:00.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:13:59 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:14:00.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:00.121+0000 7f6929e75700 1 -- 172.21.15.100:0/2352473065 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f692402e510 con 0x7f6924109600 2024-05-10T11:14:00.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:00.122+0000 7f6920ff9700 1 -- 172.21.15.100:0/2352473065 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8016 (secure 0 0 0) 0x7f691c010040 con 0x7f6924109600 2024-05-10T11:14:00.122 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:14:00.122 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2072225012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2072225012},{"type":"v1","addr":"172.21.15.100:6835","nonce":2072225012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/401939540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":401939540},{"type":"v1","addr":"172.21.15.100:6837","nonce":401939540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/2741810791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2741810791},{"type":"v1","addr":"172.21.15.178:6833","nonce":2741810791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1704891041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1704891041},{"type":"v1","addr":"172.21.15.188:6833","nonce":1704891041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24395,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2124076312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2124076312},{"type":"v1","addr":"172.21.15.178:6835","nonce":2124076312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24413,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/3224358425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3224358425},{"type":"v1","addr":"172.21.15.178:6837","nonce":3224358425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24421,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/515547690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":515547690},{"type":"v1","addr":"172.21.15.188:6835","nonce":515547690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24427,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1943342620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1943342620},{"type":"v1","addr":"172.21.15.188:6837","nonce":1943342620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24439,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6839/3194917070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":3194917070},{"type":"v1","addr":"172.21.15.188:6839","nonce":3194917070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-10T11:14:00.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:00.124+0000 7f6929e75700 1 -- 172.21.15.100:0/2352473065 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f690c0b2880 msgr2=0x7f690c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:00.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:00.124+0000 7f6929e75700 1 --2- 172.21.15.100:0/2352473065 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f690c0b2880 0x7f690c0b4d40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f691400abd0 tx=0x7f691400a380 comp rx=0 tx=0).stop 2024-05-10T11:14:00.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:00.124+0000 7f6929e75700 1 -- 172.21.15.100:0/2352473065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6924109600 msgr2=0x7f6924076220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:00.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:00.124+0000 7f6929e75700 1 --2- 172.21.15.100:0/2352473065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6924109600 0x7f6924076220 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f691c00efc0 tx=0x7f691c00c3a0 comp rx=0 tx=0).stop 2024-05-10T11:14:00.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:00.124+0000 7f6929e75700 1 -- 172.21.15.100:0/2352473065 shutdown_connections 2024-05-10T11:14:00.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:00.124+0000 7f6929e75700 1 --2- 172.21.15.100:0/2352473065 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f690c0b2880 0x7f690c0b4d40 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:00.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:00.124+0000 7f6929e75700 1 --2- 172.21.15.100:0/2352473065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6924109600 0x7f6924076220 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:00.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:00.124+0000 7f6929e75700 1 --2- 172.21.15.100:0/2352473065 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69241034f0 0x7f69240757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:00.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:00.124+0000 7f6929e75700 1 --2- 172.21.15.100:0/2352473065 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6924103eb0 0x7f6924075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:00.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:00.124+0000 7f6929e75700 1 -- 172.21.15.100:0/2352473065 >> 172.21.15.100:0/2352473065 conn(0x7f692407a650 msgr2=0x7f6924102140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:14:00.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:00.124+0000 7f6929e75700 1 -- 172.21.15.100:0/2352473065 shutdown_connections 2024-05-10T11:14:00.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:00.124+0000 7f6929e75700 1 -- 172.21.15.100:0/2352473065 wait complete. 2024-05-10T11:14:00.125 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 10 2024-05-10T11:14:00.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:00 smithi100 ceph-mon[30296]: Deploying daemon mds.g on smithi100 2024-05-10T11:14:00.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:00 smithi100 ceph-mon[30296]: mds.? [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] up:boot 2024-05-10T11:14:00.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:00 smithi100 ceph-mon[30296]: fsmap 9 up:standby 2024-05-10T11:14:00.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:00 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-10T11:14:00.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:00 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/2352473065' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:14:00.922 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:00 smithi188 ceph-mon[34075]: Deploying daemon mds.g on smithi100 2024-05-10T11:14:00.922 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:00 smithi188 ceph-mon[34075]: mds.? [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] up:boot 2024-05-10T11:14:00.922 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:00 smithi188 ceph-mon[34075]: fsmap 9 up:standby 2024-05-10T11:14:00.922 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:00 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-10T11:14:00.922 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:00 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/2352473065' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:14:01.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:00 smithi178 ceph-mon[33359]: Deploying daemon mds.g on smithi100 2024-05-10T11:14:01.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:00 smithi178 ceph-mon[33359]: mds.? [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] up:boot 2024-05-10T11:14:01.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:00 smithi178 ceph-mon[33359]: fsmap 9 up:standby 2024-05-10T11:14:01.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:00 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-10T11:14:01.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:00 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/2352473065' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:14:01.956 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:01 smithi100 ceph-mon[30296]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:02.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:01 smithi178 ceph-mon[33359]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:02.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:01 smithi188 ceph-mon[34075]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:02.538 INFO:journalctl@ceph.mds.g.smithi100.stdout:May 10 11:14:02 smithi100 systemd[1]: Starting Ceph mds.g for 20a542cc-0ebd-11ef-bc98-c7b262605968... 2024-05-10T11:14:02.840 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T11:14:02.903 INFO:journalctl@ceph.mds.g.smithi100.stdout:May 10 11:14:02 smithi100 podman[69344]: 2024-05-10 11:14:02.539024265 +0000 UTC m=+0.018182690 image pull ee5e4f21ef58047cb638cccd5cfbe8f6f43412b56c94b1d7fbd9cf3cb90a53a5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f 2024-05-10T11:14:02.903 INFO:journalctl@ceph.mds.g.smithi100.stdout:May 10 11:14:02 smithi100 podman[69344]: 2024-05-10 11:14:02.668152025 +0000 UTC m=+0.147310426 container create b9c096f696f7e84ad69bf1dbb8b1600e6bc4880e08d747d4bb4a10a56e14b59d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-g, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-10T11:14:03.092 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:14:03.216 INFO:journalctl@ceph.mds.g.smithi100.stdout:May 10 11:14:02 smithi100 podman[69344]: 2024-05-10 11:14:02.904228591 +0000 UTC m=+0.383386993 container init b9c096f696f7e84ad69bf1dbb8b1600e6bc4880e08d747d4bb4a10a56e14b59d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-10T11:14:03.217 INFO:journalctl@ceph.mds.g.smithi100.stdout:May 10 11:14:02 smithi100 podman[69344]: 2024-05-10 11:14:02.907200555 +0000 UTC m=+0.386358965 container start b9c096f696f7e84ad69bf1dbb8b1600e6bc4880e08d747d4bb4a10a56e14b59d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-g, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08) 2024-05-10T11:14:03.217 INFO:journalctl@ceph.mds.g.smithi100.stdout:May 10 11:14:02 smithi100 ceph-mds[69381]: -- 172.21.15.100:0/1610942031 <== mon.0 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a01fe85a40 con 0x55a01fe58800 2024-05-10T11:14:03.217 INFO:journalctl@ceph.mds.g.smithi100.stdout:May 10 11:14:02 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-g[69359]: starting mds.g at 2024-05-10T11:14:03.217 INFO:journalctl@ceph.mds.g.smithi100.stdout:May 10 11:14:03 smithi100 bash[69344]: b9c096f696f7e84ad69bf1dbb8b1600e6bc4880e08d747d4bb4a10a56e14b59d 2024-05-10T11:14:03.217 INFO:journalctl@ceph.mds.g.smithi100.stdout:May 10 11:14:03 smithi100 systemd[1]: Started Ceph mds.g for 20a542cc-0ebd-11ef-bc98-c7b262605968. 2024-05-10T11:14:04.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:03 smithi178 ceph-mon[33359]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:04.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:03 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:04.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:03 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:04.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:03 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:04.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:03 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T11:14:04.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:03 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T11:14:04.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:03 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:14:04.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:03 smithi188 ceph-mon[34075]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:04.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:03 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:04.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:03 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:04.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:03 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:04.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:03 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T11:14:04.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:03 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T11:14:04.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:03 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:14:04.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:03 smithi100 ceph-mon[30296]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:04.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:03 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:04.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:03 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:04.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:03 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:04.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:03 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T11:14:04.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:03 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T11:14:04.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:03 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:14:04.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:04.826+0000 7fcfa5605700 1 -- 172.21.15.100:0/2016665458 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfa0109640 msgr2=0x7fcfa0109a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:04.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:04.826+0000 7fcfa5605700 1 --2- 172.21.15.100:0/2016665458 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfa0109640 0x7fcfa0109a20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fcf94009990 tx=0x7fcf94009ca0 comp rx=0 tx=0).stop 2024-05-10T11:14:04.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:04.826+0000 7fcfa5605700 1 -- 172.21.15.100:0/2016665458 shutdown_connections 2024-05-10T11:14:04.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:04.826+0000 7fcfa5605700 1 --2- 172.21.15.100:0/2016665458 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcfa006c5a0 0x7fcfa01103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:04.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:04.826+0000 7fcfa5605700 1 --2- 172.21.15.100:0/2016665458 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfa0109640 0x7fcfa0109a20 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:04.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:04.826+0000 7fcfa5605700 1 --2- 172.21.15.100:0/2016665458 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcfa006bbe0 0x7fcfa006c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:04.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:04.826+0000 7fcfa5605700 1 -- 172.21.15.100:0/2016665458 >> 172.21.15.100:0/2016665458 conn(0x7fcfa00791a0 msgr2=0x7fcfa00795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:14:04.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:04.826+0000 7fcfa5605700 1 -- 172.21.15.100:0/2016665458 shutdown_connections 2024-05-10T11:14:04.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:04.826+0000 7fcfa5605700 1 -- 172.21.15.100:0/2016665458 wait complete. 2024-05-10T11:14:04.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:04.826+0000 7fcfa5605700 1 Processor -- start 2024-05-10T11:14:04.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:04.827+0000 7fcfa5605700 1 -- start start 2024-05-10T11:14:04.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:04.827+0000 7fcfa5605700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcfa006bbe0 0x7fcfa019f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:04.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:04.827+0000 7fcfa5605700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcfa006c5a0 0x7fcfa019fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:04.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:04.827+0000 7fcfa5605700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfa0109640 0x7fcfa0199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:04.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:04.827+0000 7fcfa5605700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa01135e0 con 0x7fcfa0109640 2024-05-10T11:14:04.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:04.827+0000 7fcfa5605700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa0113460 con 0x7fcfa006c5a0 2024-05-10T11:14:04.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:04.827+0000 7fcfa5605700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa0113760 con 0x7fcfa006bbe0 2024-05-10T11:14:04.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:04.827+0000 7fcf9f7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfa0109640 0x7fcfa0199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:04.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:04.827+0000 7fcf9f7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfa0109640 0x7fcfa0199900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43676/0 (socket says 172.21.15.100:43676) 2024-05-10T11:14:04.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:04.827+0000 7fcf9f7fe700 1 -- 172.21.15.100:0/3536054206 learned_addr learned my addr 172.21.15.100:0/3536054206 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:14:04.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:04.827+0000 7fcf9e7fc700 1 --2- 172.21.15.100:0/3536054206 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcfa006c5a0 0x7fcfa019fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:04.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:04.827+0000 7fcf9effd700 1 --2- 172.21.15.100:0/3536054206 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcfa006bbe0 0x7fcfa019f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:04.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:04.827+0000 7fcf9f7fe700 1 -- 172.21.15.100:0/3536054206 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcfa006bbe0 msgr2=0x7fcfa019f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:04.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:04.827+0000 7fcf9f7fe700 1 --2- 172.21.15.100:0/3536054206 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcfa006bbe0 0x7fcfa019f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:04.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:04.827+0000 7fcf9f7fe700 1 -- 172.21.15.100:0/3536054206 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcfa006c5a0 msgr2=0x7fcfa019fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:04.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:04.827+0000 7fcf9f7fe700 1 --2- 172.21.15.100:0/3536054206 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcfa006c5a0 0x7fcfa019fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:04.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:04.827+0000 7fcf9f7fe700 1 -- 172.21.15.100:0/3536054206 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfa019a060 con 0x7fcfa0109640 2024-05-10T11:14:04.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:04.827+0000 7fcf9e7fc700 1 --2- 172.21.15.100:0/3536054206 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcfa006c5a0 0x7fcfa019fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:14:04.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:04.827+0000 7fcf9f7fe700 1 --2- 172.21.15.100:0/3536054206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfa0109640 0x7fcfa0199900 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fcf9800cca0 tx=0x7fcf9800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:14:04.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:04.828+0000 7fcf87fff700 1 -- 172.21.15.100:0/3536054206 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf98016020 con 0x7fcfa0109640 2024-05-10T11:14:04.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:04.828+0000 7fcfa5605700 1 -- 172.21.15.100:0/3536054206 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfa019a350 con 0x7fcfa0109640 2024-05-10T11:14:04.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:04.828+0000 7fcf87fff700 1 -- 172.21.15.100:0/3536054206 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcf9800ad60 con 0x7fcfa0109640 2024-05-10T11:14:04.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:04.828+0000 7fcf87fff700 1 -- 172.21.15.100:0/3536054206 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf980115e0 con 0x7fcfa0109640 2024-05-10T11:14:04.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:04.828+0000 7fcfa5605700 1 -- 172.21.15.100:0/3536054206 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfa01a65c0 con 0x7fcfa0109640 2024-05-10T11:14:04.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:04.829+0000 7fcfa5605700 1 -- 172.21.15.100:0/3536054206 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfa010ba90 con 0x7fcfa0109640 2024-05-10T11:14:04.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:04.830+0000 7fcf87fff700 1 -- 172.21.15.100:0/3536054206 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fcf98019020 con 0x7fcfa0109640 2024-05-10T11:14:04.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:04.831+0000 7fcf87fff700 1 --2- 172.21.15.100:0/3536054206 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fcf880b2880 0x7fcf880b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:04.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:04.831+0000 7fcf9effd700 1 --2- 172.21.15.100:0/3536054206 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fcf880b2880 0x7fcf880b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:04.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:04.831+0000 7fcf87fff700 1 -- 172.21.15.100:0/3536054206 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7fcf980e1d30 con 0x7fcfa0109640 2024-05-10T11:14:04.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:04.831+0000 7fcf9effd700 1 --2- 172.21.15.100:0/3536054206 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fcf880b2880 0x7fcf880b4d40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fcf94006010 tx=0x7fcf9401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:14:04.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:04.835+0000 7fcf87fff700 1 -- 172.21.15.100:0/3536054206 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fcf98020020 con 0x7fcfa0109640 2024-05-10T11:14:04.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:04 smithi100 ceph-mon[30296]: Deploying daemon mds.a on smithi100 2024-05-10T11:14:04.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:04 smithi100 ceph-mon[30296]: mds.? [v2:172.21.15.100:6838/3946311025,v1:172.21.15.100:6839/3946311025] up:boot 2024-05-10T11:14:04.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:04 smithi100 ceph-mon[30296]: fsmap 10 up:standby 2024-05-10T11:14:04.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:04 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-10T11:14:05.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:04 smithi188 ceph-mon[34075]: Deploying daemon mds.a on smithi100 2024-05-10T11:14:05.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:04 smithi188 ceph-mon[34075]: mds.? [v2:172.21.15.100:6838/3946311025,v1:172.21.15.100:6839/3946311025] up:boot 2024-05-10T11:14:05.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:04 smithi188 ceph-mon[34075]: fsmap 10 up:standby 2024-05-10T11:14:05.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:04 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-10T11:14:05.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:05.097+0000 7fcfa5605700 1 -- 172.21.15.100:0/3536054206 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcfa010b470 con 0x7fcfa0109640 2024-05-10T11:14:05.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:05.098+0000 7fcf87fff700 1 -- 172.21.15.100:0/3536054206 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8845 (secure 0 0 0) 0x7fcf980a9ef0 con 0x7fcfa0109640 2024-05-10T11:14:05.098 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:14:05.099 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2072225012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2072225012},{"type":"v1","addr":"172.21.15.100:6835","nonce":2072225012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/401939540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":401939540},{"type":"v1","addr":"172.21.15.100:6837","nonce":401939540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14571,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/3946311025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3946311025},{"type":"v1","addr":"172.21.15.100:6839","nonce":3946311025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/2741810791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2741810791},{"type":"v1","addr":"172.21.15.178:6833","nonce":2741810791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1704891041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1704891041},{"type":"v1","addr":"172.21.15.188:6833","nonce":1704891041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24395,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2124076312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2124076312},{"type":"v1","addr":"172.21.15.178:6835","nonce":2124076312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24413,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/3224358425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3224358425},{"type":"v1","addr":"172.21.15.178:6837","nonce":3224358425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24421,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/515547690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":515547690},{"type":"v1","addr":"172.21.15.188:6835","nonce":515547690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24427,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1943342620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1943342620},{"type":"v1","addr":"172.21.15.188:6837","nonce":1943342620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24439,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6839/3194917070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":3194917070},{"type":"v1","addr":"172.21.15.188:6839","nonce":3194917070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-10T11:14:05.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:05.100+0000 7fcfa5605700 1 -- 172.21.15.100:0/3536054206 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fcf880b2880 msgr2=0x7fcf880b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:05.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:05.100+0000 7fcfa5605700 1 --2- 172.21.15.100:0/3536054206 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fcf880b2880 0x7fcf880b4d40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fcf94006010 tx=0x7fcf9401a040 comp rx=0 tx=0).stop 2024-05-10T11:14:05.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:05.100+0000 7fcfa5605700 1 -- 172.21.15.100:0/3536054206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfa0109640 msgr2=0x7fcfa0199900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:05.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:05.100+0000 7fcfa5605700 1 --2- 172.21.15.100:0/3536054206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfa0109640 0x7fcfa0199900 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fcf9800cca0 tx=0x7fcf9800a340 comp rx=0 tx=0).stop 2024-05-10T11:14:05.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:05.100+0000 7fcfa5605700 1 -- 172.21.15.100:0/3536054206 shutdown_connections 2024-05-10T11:14:05.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:05.100+0000 7fcfa5605700 1 --2- 172.21.15.100:0/3536054206 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fcf880b2880 0x7fcf880b4d40 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:05.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:05.100+0000 7fcfa5605700 1 --2- 172.21.15.100:0/3536054206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfa0109640 0x7fcfa0199900 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:05.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:05.100+0000 7fcfa5605700 1 --2- 172.21.15.100:0/3536054206 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcfa006bbe0 0x7fcfa019f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:05.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:05.100+0000 7fcfa5605700 1 --2- 172.21.15.100:0/3536054206 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcfa006c5a0 0x7fcfa019fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:05.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:05.100+0000 7fcfa5605700 1 -- 172.21.15.100:0/3536054206 >> 172.21.15.100:0/3536054206 conn(0x7fcfa00791a0 msgr2=0x7fcfa0101500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:14:05.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:05.100+0000 7fcfa5605700 1 -- 172.21.15.100:0/3536054206 shutdown_connections 2024-05-10T11:14:05.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:05.100+0000 7fcfa5605700 1 -- 172.21.15.100:0/3536054206 wait complete. 2024-05-10T11:14:05.103 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 11 2024-05-10T11:14:05.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:04 smithi178 ceph-mon[33359]: Deploying daemon mds.a on smithi100 2024-05-10T11:14:05.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:04 smithi178 ceph-mon[33359]: mds.? [v2:172.21.15.100:6838/3946311025,v1:172.21.15.100:6839/3946311025] up:boot 2024-05-10T11:14:05.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:04 smithi178 ceph-mon[33359]: fsmap 10 up:standby 2024-05-10T11:14:05.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:04 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-10T11:14:05.965 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:05 smithi100 ceph-mon[30296]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:05.965 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:05 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/3536054206' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:14:06.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:05 smithi188 ceph-mon[34075]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:06.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:05 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/3536054206' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:14:06.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:05 smithi178 ceph-mon[33359]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:06.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:05 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/3536054206' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:14:07.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:07 smithi100 ceph-mon[30296]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:07.288 INFO:journalctl@ceph.mds.a.smithi100.stdout:May 10 11:14:07 smithi100 systemd[1]: Starting Ceph mds.a for 20a542cc-0ebd-11ef-bc98-c7b262605968... 2024-05-10T11:14:07.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:07 smithi178 ceph-mon[33359]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:07.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:07 smithi188 ceph-mon[34075]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:07.602 INFO:journalctl@ceph.mds.a.smithi100.stdout:May 10 11:14:07 smithi100 podman[69812]: 2024-05-10 11:14:07.245360593 +0000 UTC m=+0.017894438 image pull ee5e4f21ef58047cb638cccd5cfbe8f6f43412b56c94b1d7fbd9cf3cb90a53a5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f 2024-05-10T11:14:07.602 INFO:journalctl@ceph.mds.a.smithi100.stdout:May 10 11:14:07 smithi100 podman[69812]: 2024-05-10 11:14:07.364811009 +0000 UTC m=+0.137344877 container create e7edb955eb9705efb9011e0c3a3c21258d85efef0780dcaa146c6253fc046a7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-a, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, GIT_CLEAN=True, ceph=True, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e) 2024-05-10T11:14:07.602 INFO:journalctl@ceph.mds.a.smithi100.stdout:May 10 11:14:07 smithi100 podman[69812]: 2024-05-10 11:14:07.575850208 +0000 UTC m=+0.348384033 container init e7edb955eb9705efb9011e0c3a3c21258d85efef0780dcaa146c6253fc046a7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-a, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-05-10T11:14:07.602 INFO:journalctl@ceph.mds.a.smithi100.stdout:May 10 11:14:07 smithi100 podman[69812]: 2024-05-10 11:14:07.578748918 +0000 UTC m=+0.351282754 container start e7edb955eb9705efb9011e0c3a3c21258d85efef0780dcaa146c6253fc046a7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-a, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, io.buildah.version=1.33.5, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=) 2024-05-10T11:14:07.602 INFO:journalctl@ceph.mds.a.smithi100.stdout:May 10 11:14:07 smithi100 ceph-mds[69831]: -- 172.21.15.100:0/2273562893 <== mon.0 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b25aaffa40 con 0x55b25aad2800 2024-05-10T11:14:07.742 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T11:14:07.988 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:14:08.037 INFO:journalctl@ceph.mds.a.smithi100.stdout:May 10 11:14:07 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-a[69827]: starting mds.a at 2024-05-10T11:14:08.038 INFO:journalctl@ceph.mds.a.smithi100.stdout:May 10 11:14:07 smithi100 bash[69812]: e7edb955eb9705efb9011e0c3a3c21258d85efef0780dcaa146c6253fc046a7c 2024-05-10T11:14:08.038 INFO:journalctl@ceph.mds.a.smithi100.stdout:May 10 11:14:07 smithi100 systemd[1]: Started Ceph mds.a for 20a542cc-0ebd-11ef-bc98-c7b262605968. 2024-05-10T11:14:09.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:08 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:09.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:08 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:09.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:08 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:09.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:08 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T11:14:09.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:08 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T11:14:09.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:08 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:14:09.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:08 smithi178 ceph-mon[33359]: Deploying daemon mds.b on smithi178 2024-05-10T11:14:09.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:08 smithi178 ceph-mon[33359]: mds.? [v2:172.21.15.100:6840/3736371660,v1:172.21.15.100:6841/3736371660] up:boot 2024-05-10T11:14:09.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:08 smithi178 ceph-mon[33359]: fsmap 11 up:standby 2024-05-10T11:14:09.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:08 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-10T11:14:09.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:08 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:09.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:08 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:09.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:08 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:09.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:08 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T11:14:09.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:08 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T11:14:09.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:08 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:14:09.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:08 smithi188 ceph-mon[34075]: Deploying daemon mds.b on smithi178 2024-05-10T11:14:09.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:08 smithi188 ceph-mon[34075]: mds.? [v2:172.21.15.100:6840/3736371660,v1:172.21.15.100:6841/3736371660] up:boot 2024-05-10T11:14:09.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:08 smithi188 ceph-mon[34075]: fsmap 11 up:standby 2024-05-10T11:14:09.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:08 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-10T11:14:09.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:08 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:09.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:08 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:09.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:08 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:09.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:08 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T11:14:09.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:08 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T11:14:09.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:08 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:14:09.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:08 smithi100 ceph-mon[30296]: Deploying daemon mds.b on smithi178 2024-05-10T11:14:09.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:08 smithi100 ceph-mon[30296]: mds.? [v2:172.21.15.100:6840/3736371660,v1:172.21.15.100:6841/3736371660] up:boot 2024-05-10T11:14:09.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:08 smithi100 ceph-mon[30296]: fsmap 11 up:standby 2024-05-10T11:14:09.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:08 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-10T11:14:09.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:09.610+0000 7f9b20d25700 1 -- 172.21.15.100:0/1408874800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b1c101a90 msgr2=0x7f9b1c101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:09.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:09.610+0000 7f9b20d25700 1 --2- 172.21.15.100:0/1408874800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b1c101a90 0x7f9b1c101f10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9b04009990 tx=0x7f9b04009ca0 comp rx=0 tx=0).stop 2024-05-10T11:14:09.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:09.610+0000 7f9b20d25700 1 -- 172.21.15.100:0/1408874800 shutdown_connections 2024-05-10T11:14:09.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:09.610+0000 7f9b20d25700 1 --2- 172.21.15.100:0/1408874800 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b1c102450 0x7f9b1c110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:09.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:09.610+0000 7f9b20d25700 1 --2- 172.21.15.100:0/1408874800 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b1c109520 0x7f9b1c109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:09.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:09.610+0000 7f9b20d25700 1 --2- 172.21.15.100:0/1408874800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b1c101a90 0x7f9b1c101f10 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:09.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:09.610+0000 7f9b20d25700 1 -- 172.21.15.100:0/1408874800 >> 172.21.15.100:0/1408874800 conn(0x7f9b1c078a10 msgr2=0x7f9b1c0ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:14:09.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:09.610+0000 7f9b20d25700 1 -- 172.21.15.100:0/1408874800 shutdown_connections 2024-05-10T11:14:09.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:09.610+0000 7f9b20d25700 1 -- 172.21.15.100:0/1408874800 wait complete. 2024-05-10T11:14:09.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:09.611+0000 7f9b20d25700 1 Processor -- start 2024-05-10T11:14:09.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:09.611+0000 7f9b20d25700 1 -- start start 2024-05-10T11:14:09.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:09.612+0000 7f9b20d25700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b1c101a90 0x7f9b1c19f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:09.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:09.612+0000 7f9b20d25700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b1c102450 0x7f9b1c19fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:09.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:09.612+0000 7f9b20d25700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b1c109520 0x7f9b1c199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:09.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:09.612+0000 7f9b20d25700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9b1c1134e0 con 0x7f9b1c101a90 2024-05-10T11:14:09.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:09.612+0000 7f9b20d25700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9b1c113360 con 0x7f9b1c109520 2024-05-10T11:14:09.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:09.612+0000 7f9b20d25700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9b1c113660 con 0x7f9b1c102450 2024-05-10T11:14:09.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:09.612+0000 7f9b1a59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b1c101a90 0x7f9b1c19f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:09.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:09.612+0000 7f9b1a59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b1c101a90 0x7f9b1c19f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39690/0 (socket says 172.21.15.100:39690) 2024-05-10T11:14:09.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:09.612+0000 7f9b1a59c700 1 -- 172.21.15.100:0/457878481 learned_addr learned my addr 172.21.15.100:0/457878481 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:14:09.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:09.612+0000 7f9b19d9b700 1 --2- 172.21.15.100:0/457878481 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b1c102450 0x7f9b1c19fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:09.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:09.612+0000 7f9b1ad9d700 1 --2- 172.21.15.100:0/457878481 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b1c109520 0x7f9b1c199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:09.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:09.612+0000 7f9b1a59c700 1 -- 172.21.15.100:0/457878481 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b1c102450 msgr2=0x7f9b1c19fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:09.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:09.612+0000 7f9b1a59c700 1 --2- 172.21.15.100:0/457878481 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b1c102450 0x7f9b1c19fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:09.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:09.612+0000 7f9b1a59c700 1 -- 172.21.15.100:0/457878481 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b1c109520 msgr2=0x7f9b1c199880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:09.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:09.612+0000 7f9b1a59c700 1 --2- 172.21.15.100:0/457878481 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b1c109520 0x7f9b1c199880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:09.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:09.612+0000 7f9b1a59c700 1 -- 172.21.15.100:0/457878481 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b1c199fe0 con 0x7f9b1c101a90 2024-05-10T11:14:09.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:09.613+0000 7f9b1a59c700 1 --2- 172.21.15.100:0/457878481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b1c101a90 0x7f9b1c19f6c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c00eb30 tx=0x7f9b0c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:14:09.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:09.613+0000 7f9b137fe700 1 -- 172.21.15.100:0/457878481 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b0c00cbd0 con 0x7f9b1c101a90 2024-05-10T11:14:09.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:09.613+0000 7f9b20d25700 1 -- 172.21.15.100:0/457878481 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b1c19a2d0 con 0x7f9b1c101a90 2024-05-10T11:14:09.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:09.613+0000 7f9b137fe700 1 -- 172.21.15.100:0/457878481 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9b0c0044b0 con 0x7f9b1c101a90 2024-05-10T11:14:09.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:09.613+0000 7f9b137fe700 1 -- 172.21.15.100:0/457878481 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b0c003dc0 con 0x7f9b1c101a90 2024-05-10T11:14:09.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:09.613+0000 7f9b20d25700 1 -- 172.21.15.100:0/457878481 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b1c102ba0 con 0x7f9b1c101a90 2024-05-10T11:14:09.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:09.614+0000 7f9b20d25700 1 -- 172.21.15.100:0/457878481 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b1c19adb0 con 0x7f9b1c101a90 2024-05-10T11:14:09.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:09.615+0000 7f9b137fe700 1 -- 172.21.15.100:0/457878481 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f9b0c003f60 con 0x7f9b1c101a90 2024-05-10T11:14:09.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:09.616+0000 7f9b137fe700 1 --2- 172.21.15.100:0/457878481 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f9b080b2880 0x7f9b080b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:09.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:09.616+0000 7f9b19d9b700 1 --2- 172.21.15.100:0/457878481 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f9b080b2880 0x7f9b080b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:09.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:09.616+0000 7f9b137fe700 1 -- 172.21.15.100:0/457878481 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f9b0c0e2080 con 0x7f9b1c101a90 2024-05-10T11:14:09.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:09.617+0000 7f9b19d9b700 1 --2- 172.21.15.100:0/457878481 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f9b080b2880 0x7f9b080b4d40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c0ffb20 tx=0x7f9b04005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:14:09.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:09.620+0000 7f9b137fe700 1 -- 172.21.15.100:0/457878481 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f9b0c0aa2d0 con 0x7f9b1c101a90 2024-05-10T11:14:09.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:09.878+0000 7f9b20d25700 1 -- 172.21.15.100:0/457878481 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9b1c02e510 con 0x7f9b1c101a90 2024-05-10T11:14:09.879 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:09 smithi100 ceph-mon[30296]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:09.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:09.879+0000 7f9b137fe700 1 -- 172.21.15.100:0/457878481 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9674 (secure 0 0 0) 0x7f9b0c0b22c0 con 0x7f9b1c101a90 2024-05-10T11:14:09.880 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:14:09.880 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2072225012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2072225012},{"type":"v1","addr":"172.21.15.100:6835","nonce":2072225012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/401939540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":401939540},{"type":"v1","addr":"172.21.15.100:6837","nonce":401939540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14571,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/3946311025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3946311025},{"type":"v1","addr":"172.21.15.100:6839","nonce":3946311025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3736371660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3736371660},{"type":"v1","addr":"172.21.15.100:6841","nonce":3736371660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/2741810791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2741810791},{"type":"v1","addr":"172.21.15.178:6833","nonce":2741810791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1704891041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1704891041},{"type":"v1","addr":"172.21.15.188:6833","nonce":1704891041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24395,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2124076312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2124076312},{"type":"v1","addr":"172.21.15.178:6835","nonce":2124076312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24413,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/3224358425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3224358425},{"type":"v1","addr":"172.21.15.178:6837","nonce":3224358425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24421,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/515547690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":515547690},{"type":"v1","addr":"172.21.15.188:6835","nonce":515547690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24427,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1943342620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1943342620},{"type":"v1","addr":"172.21.15.188:6837","nonce":1943342620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24439,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6839/3194917070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":3194917070},{"type":"v1","addr":"172.21.15.188:6839","nonce":3194917070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-10T11:14:09.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:09.881+0000 7f9b20d25700 1 -- 172.21.15.100:0/457878481 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f9b080b2880 msgr2=0x7f9b080b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:09.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:09.881+0000 7f9b20d25700 1 --2- 172.21.15.100:0/457878481 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f9b080b2880 0x7f9b080b4d40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c0ffb20 tx=0x7f9b04005fb0 comp rx=0 tx=0).stop 2024-05-10T11:14:09.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:09.881+0000 7f9b20d25700 1 -- 172.21.15.100:0/457878481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b1c101a90 msgr2=0x7f9b1c19f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:09.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:09.881+0000 7f9b20d25700 1 --2- 172.21.15.100:0/457878481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b1c101a90 0x7f9b1c19f6c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c00eb30 tx=0x7f9b0c00ee40 comp rx=0 tx=0).stop 2024-05-10T11:14:09.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:09.882+0000 7f9b20d25700 1 -- 172.21.15.100:0/457878481 shutdown_connections 2024-05-10T11:14:09.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:09.882+0000 7f9b20d25700 1 --2- 172.21.15.100:0/457878481 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f9b080b2880 0x7f9b080b4d40 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:09.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:09.882+0000 7f9b20d25700 1 --2- 172.21.15.100:0/457878481 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b1c109520 0x7f9b1c199880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:09.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:09.882+0000 7f9b20d25700 1 --2- 172.21.15.100:0/457878481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b1c101a90 0x7f9b1c19f6c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:09.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:09.882+0000 7f9b20d25700 1 --2- 172.21.15.100:0/457878481 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b1c102450 0x7f9b1c19fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:09.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:09.882+0000 7f9b20d25700 1 -- 172.21.15.100:0/457878481 >> 172.21.15.100:0/457878481 conn(0x7f9b1c078a10 msgr2=0x7f9b1c100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:14:09.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:09.882+0000 7f9b20d25700 1 -- 172.21.15.100:0/457878481 shutdown_connections 2024-05-10T11:14:09.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:09.882+0000 7f9b20d25700 1 -- 172.21.15.100:0/457878481 wait complete. 2024-05-10T11:14:09.883 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 12 2024-05-10T11:14:10.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:09 smithi178 ceph-mon[33359]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:10.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:09 smithi188 ceph-mon[34075]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:10.730 INFO:journalctl@ceph.mds.b.smithi178.stdout:May 10 11:14:10 smithi178 systemd[1]: Started Ceph mds.b for 20a542cc-0ebd-11ef-bc98-c7b262605968. 2024-05-10T11:14:11.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:10 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/457878481' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:14:11.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:10 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:11.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:10 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:11.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:10 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:11.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:10 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:11.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:10 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:11.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:10 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/457878481' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:14:11.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:10 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:11.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:10 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:11.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:10 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:11.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:10 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:11.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:10 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:11.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:10 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/457878481' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:14:11.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:10 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:11.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:10 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:11.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:10 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:11.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:10 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:11.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:10 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:11.975 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:11 smithi100 ceph-mon[30296]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:11.975 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:11 smithi100 ceph-mon[30296]: mds.? [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] up:boot 2024-05-10T11:14:11.975 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:11 smithi100 ceph-mon[30296]: fsmap 12 up:standby 2024-05-10T11:14:11.975 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:11 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-10T11:14:11.975 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:11 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:14:11.995 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:11 smithi188 ceph-mon[34075]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:11.995 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:11 smithi188 ceph-mon[34075]: mds.? [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] up:boot 2024-05-10T11:14:11.995 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:11 smithi188 ceph-mon[34075]: fsmap 12 up:standby 2024-05-10T11:14:11.995 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:11 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-10T11:14:11.995 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:11 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:14:12.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:11 smithi178 ceph-mon[33359]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:12.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:11 smithi178 ceph-mon[33359]: mds.? [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] up:boot 2024-05-10T11:14:12.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:11 smithi178 ceph-mon[33359]: fsmap 12 up:standby 2024-05-10T11:14:12.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:11 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-10T11:14:12.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:11 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:14:12.597 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T11:14:12.851 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:14:13.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:13 smithi188 ceph-mon[34075]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:13.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:13 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:13.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:13 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:13.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:13 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:13.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:13 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:13.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:13 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:13.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:13 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:13.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:13 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:14:13.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:13 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:14:13.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:13 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:13.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:13 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:14:13.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:13 smithi100 ceph-mon[30296]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:13.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:13 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:13.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:13 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:13.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:13 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:13.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:13 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:13.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:13 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:13.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:13 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:13.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:13 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:14:13.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:13 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:14:13.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:13 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:13.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:13 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:14:14.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:13 smithi178 ceph-mon[33359]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:14.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:13 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:14.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:13 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:14.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:13 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:14.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:13 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:14.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:13 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:14.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:13 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:14.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:13 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:14:14.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:13 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:14:14.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:13 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:14.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:13 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:14:14.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:14.521+0000 7f8d8e393700 1 -- 172.21.15.100:0/1501734139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d881037b0 msgr2=0x7f8d88103b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:14.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:14.521+0000 7f8d8e393700 1 --2- 172.21.15.100:0/1501734139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d881037b0 0x7f8d88103b90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8d780099c0 tx=0x7f8d78009cd0 comp rx=0 tx=0).stop 2024-05-10T11:14:14.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:14.521+0000 7f8d8e393700 1 -- 172.21.15.100:0/1501734139 shutdown_connections 2024-05-10T11:14:14.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:14.521+0000 7f8d8e393700 1 --2- 172.21.15.100:0/1501734139 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d88108880 0x7f8d88110400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:14.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:14.521+0000 7f8d8e393700 1 --2- 172.21.15.100:0/1501734139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d881037b0 0x7f8d88103b90 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:14.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:14.521+0000 7f8d8e393700 1 --2- 172.21.15.100:0/1501734139 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d88104160 0x7f8d88108150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:14.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:14.521+0000 7f8d8e393700 1 -- 172.21.15.100:0/1501734139 >> 172.21.15.100:0/1501734139 conn(0x7f8d8807a530 msgr2=0x7f8d8807a940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:14:14.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:14.521+0000 7f8d8e393700 1 -- 172.21.15.100:0/1501734139 shutdown_connections 2024-05-10T11:14:14.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:14.521+0000 7f8d8e393700 1 -- 172.21.15.100:0/1501734139 wait complete. 2024-05-10T11:14:14.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:14.521+0000 7f8d8e393700 1 Processor -- start 2024-05-10T11:14:14.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:14.521+0000 7f8d8e393700 1 -- start start 2024-05-10T11:14:14.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:14.522+0000 7f8d8e393700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d881037b0 0x7f8d8819f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:14.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:14.522+0000 7f8d8e393700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d88104160 0x7f8d8819fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:14.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:14.522+0000 7f8d8e393700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d88108880 0x7f8d88199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:14.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:14.522+0000 7f8d8e393700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8d88113500 con 0x7f8d88108880 2024-05-10T11:14:14.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:14.522+0000 7f8d8e393700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8d88113380 con 0x7f8d88104160 2024-05-10T11:14:14.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:14.522+0000 7f8d8e393700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8d88113680 con 0x7f8d881037b0 2024-05-10T11:14:14.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:14.522+0000 7f8d87fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d881037b0 0x7f8d8819f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:14.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:14.522+0000 7f8d7ffff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d88104160 0x7f8d8819fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:14.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:14.522+0000 7f8d87fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d881037b0 0x7f8d8819f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:50216/0 (socket says 172.21.15.100:50216) 2024-05-10T11:14:14.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:14.522+0000 7f8d87fff700 1 -- 172.21.15.100:0/2994284968 learned_addr learned my addr 172.21.15.100:0/2994284968 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:14:14.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:14.522+0000 7f8d8c930700 1 --2- 172.21.15.100:0/2994284968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d88108880 0x7f8d88199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:14.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:14.522+0000 7f8d87fff700 1 -- 172.21.15.100:0/2994284968 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d88104160 msgr2=0x7f8d8819fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:14.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:14.522+0000 7f8d87fff700 1 --2- 172.21.15.100:0/2994284968 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d88104160 0x7f8d8819fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:14.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:14.522+0000 7f8d87fff700 1 -- 172.21.15.100:0/2994284968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d88108880 msgr2=0x7f8d88199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:14.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:14.522+0000 7f8d87fff700 1 --2- 172.21.15.100:0/2994284968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d88108880 0x7f8d88199840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:14.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:14.522+0000 7f8d87fff700 1 -- 172.21.15.100:0/2994284968 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d8819a020 con 0x7f8d881037b0 2024-05-10T11:14:14.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:14.522+0000 7f8d87fff700 1 --2- 172.21.15.100:0/2994284968 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d881037b0 0x7f8d8819f6c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8d78006010 tx=0x7f8d780045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:14:14.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:14.522+0000 7f8d85ffb700 1 -- 172.21.15.100:0/2994284968 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d78026070 con 0x7f8d881037b0 2024-05-10T11:14:14.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:14.522+0000 7f8d8e393700 1 -- 172.21.15.100:0/2994284968 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d8819a2b0 con 0x7f8d881037b0 2024-05-10T11:14:14.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:14.522+0000 7f8d85ffb700 1 -- 172.21.15.100:0/2994284968 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8d7800f460 con 0x7f8d881037b0 2024-05-10T11:14:14.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:14.522+0000 7f8d85ffb700 1 -- 172.21.15.100:0/2994284968 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d78009e60 con 0x7f8d881037b0 2024-05-10T11:14:14.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:14.523+0000 7f8d8e393700 1 -- 172.21.15.100:0/2994284968 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d880756d0 con 0x7f8d881037b0 2024-05-10T11:14:14.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:14.523+0000 7f8d8e393700 1 -- 172.21.15.100:0/2994284968 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d88104c10 con 0x7f8d881037b0 2024-05-10T11:14:14.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:14.525+0000 7f8d85ffb700 1 -- 172.21.15.100:0/2994284968 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f8d78029070 con 0x7f8d881037b0 2024-05-10T11:14:14.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:14.525+0000 7f8d85ffb700 1 --2- 172.21.15.100:0/2994284968 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8d740b2880 0x7f8d740b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:14.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:14.525+0000 7f8d7ffff700 1 --2- 172.21.15.100:0/2994284968 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8d740b2880 0x7f8d740b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:14.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:14.526+0000 7f8d85ffb700 1 -- 172.21.15.100:0/2994284968 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f8d780e2800 con 0x7f8d881037b0 2024-05-10T11:14:14.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:14.526+0000 7f8d7ffff700 1 --2- 172.21.15.100:0/2994284968 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8d740b2880 0x7f8d740b4d40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8d881030b0 tx=0x7f8d7000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:14:14.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:14.529+0000 7f8d85ffb700 1 -- 172.21.15.100:0/2994284968 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f8d78030080 con 0x7f8d881037b0 2024-05-10T11:14:14.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:14.801+0000 7f8d8e393700 1 -- 172.21.15.100:0/2994284968 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8d8819ad70 con 0x7f8d881037b0 2024-05-10T11:14:14.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:14.802+0000 7f8d85ffb700 1 -- 172.21.15.100:0/2994284968 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10503 (secure 0 0 0) 0x7f8d78013070 con 0x7f8d881037b0 2024-05-10T11:14:14.802 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:14:14.802 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2072225012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2072225012},{"type":"v1","addr":"172.21.15.100:6835","nonce":2072225012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/401939540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":401939540},{"type":"v1","addr":"172.21.15.100:6837","nonce":401939540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14571,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/3946311025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3946311025},{"type":"v1","addr":"172.21.15.100:6839","nonce":3946311025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3736371660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3736371660},{"type":"v1","addr":"172.21.15.100:6841","nonce":3736371660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/2741810791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2741810791},{"type":"v1","addr":"172.21.15.178:6833","nonce":2741810791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1704891041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1704891041},{"type":"v1","addr":"172.21.15.188:6833","nonce":1704891041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24395,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2124076312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2124076312},{"type":"v1","addr":"172.21.15.178:6835","nonce":2124076312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24413,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/3224358425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3224358425},{"type":"v1","addr":"172.21.15.178:6837","nonce":3224358425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24421,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/515547690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":515547690},{"type":"v1","addr":"172.21.15.188:6835","nonce":515547690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24427,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1943342620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1943342620},{"type":"v1","addr":"172.21.15.188:6837","nonce":1943342620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24437,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6839/3760985404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3760985404},{"type":"v1","addr":"172.21.15.178:6839","nonce":3760985404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24439,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6839/3194917070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":3194917070},{"type":"v1","addr":"172.21.15.188:6839","nonce":3194917070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-10T11:14:14.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:14.804+0000 7f8d8e393700 1 -- 172.21.15.100:0/2994284968 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8d740b2880 msgr2=0x7f8d740b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:14.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:14.804+0000 7f8d8e393700 1 --2- 172.21.15.100:0/2994284968 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8d740b2880 0x7f8d740b4d40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8d881030b0 tx=0x7f8d7000b410 comp rx=0 tx=0).stop 2024-05-10T11:14:14.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:14.804+0000 7f8d8e393700 1 -- 172.21.15.100:0/2994284968 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d881037b0 msgr2=0x7f8d8819f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:14.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:14.804+0000 7f8d8e393700 1 --2- 172.21.15.100:0/2994284968 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d881037b0 0x7f8d8819f6c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8d78006010 tx=0x7f8d780045e0 comp rx=0 tx=0).stop 2024-05-10T11:14:14.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:14.804+0000 7f8d8e393700 1 -- 172.21.15.100:0/2994284968 shutdown_connections 2024-05-10T11:14:14.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:14.804+0000 7f8d8e393700 1 --2- 172.21.15.100:0/2994284968 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8d740b2880 0x7f8d740b4d40 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:14.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:14.804+0000 7f8d8e393700 1 --2- 172.21.15.100:0/2994284968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d88108880 0x7f8d88199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:14.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:14.804+0000 7f8d8e393700 1 --2- 172.21.15.100:0/2994284968 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d881037b0 0x7f8d8819f6c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:14.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:14.804+0000 7f8d8e393700 1 --2- 172.21.15.100:0/2994284968 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d88104160 0x7f8d8819fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:14.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:14.804+0000 7f8d8e393700 1 -- 172.21.15.100:0/2994284968 >> 172.21.15.100:0/2994284968 conn(0x7f8d8807a530 msgr2=0x7f8d88101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:14:14.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:14.804+0000 7f8d8e393700 1 -- 172.21.15.100:0/2994284968 shutdown_connections 2024-05-10T11:14:14.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:14.804+0000 7f8d8e393700 1 -- 172.21.15.100:0/2994284968 wait complete. 2024-05-10T11:14:14.805 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 13 2024-05-10T11:14:15.643 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:15 smithi100 ceph-mon[30296]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:15.643 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:15 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/2994284968' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:14:15.643 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:15 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:15.643 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:15 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:15.643 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:15 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:15.643 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:15 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:15.643 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:15 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:15.670 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-05-10T11:14:15.903 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:15 smithi178 ceph-mon[33359]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:15.903 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:15 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/2994284968' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:14:15.903 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:15 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:15.904 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:15 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:15.904 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:15 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:15.904 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:15 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:15.904 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:15 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:16.019 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:15 smithi188 ceph-mon[34075]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:16.019 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:15 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/2994284968' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:14:16.020 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:15 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:16.020 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:15 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:16.020 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:15 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:16.020 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:15 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:16.020 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:15 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:16.148 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:14:17.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:17 smithi178 ceph-mon[33359]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:17.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:17 smithi100 ceph-mon[30296]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:17.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:17 smithi188 ceph-mon[34075]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:17.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:17.860+0000 7fb2e6f9d700 1 -- 172.21.15.100:0/1082894431 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2e0108400 msgr2=0x7fb2e01087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:17.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:17.860+0000 7fb2e6f9d700 1 --2- 172.21.15.100:0/1082894431 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2e0108400 0x7fb2e01087e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4009990 tx=0x7fb2d4009ca0 comp rx=0 tx=0).stop 2024-05-10T11:14:17.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:17.860+0000 7fb2e6f9d700 1 -- 172.21.15.100:0/1082894431 shutdown_connections 2024-05-10T11:14:17.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:17.860+0000 7fb2e6f9d700 1 --2- 172.21.15.100:0/1082894431 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2e0102d50 0x7fb2e010f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:17.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:17.860+0000 7fb2e6f9d700 1 --2- 172.21.15.100:0/1082894431 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2e0108400 0x7fb2e01087e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:17.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:17.860+0000 7fb2e6f9d700 1 --2- 172.21.15.100:0/1082894431 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb2e0102390 0x7fb2e0102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:17.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:17.860+0000 7fb2e6f9d700 1 -- 172.21.15.100:0/1082894431 >> 172.21.15.100:0/1082894431 conn(0x7fb2e0079310 msgr2=0x7fb2e0079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:14:17.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:17.860+0000 7fb2e6f9d700 1 -- 172.21.15.100:0/1082894431 shutdown_connections 2024-05-10T11:14:17.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:17.860+0000 7fb2e6f9d700 1 -- 172.21.15.100:0/1082894431 wait complete. 2024-05-10T11:14:17.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:17.861+0000 7fb2e6f9d700 1 Processor -- start 2024-05-10T11:14:17.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:17.861+0000 7fb2e6f9d700 1 -- start start 2024-05-10T11:14:17.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:17.862+0000 7fb2e6f9d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb2e0102390 0x7fb2e019a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:17.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:17.862+0000 7fb2e6f9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2e0102d50 0x7fb2e019adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:17.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:17.862+0000 7fb2e6f9d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2e0108400 0x7fb2e01a5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:17.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:17.862+0000 7fb2e6f9d700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e01123d0 con 0x7fb2e0102d50 2024-05-10T11:14:17.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:17.862+0000 7fb2e6f9d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e0112250 con 0x7fb2e0108400 2024-05-10T11:14:17.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:17.862+0000 7fb2e6f9d700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e0112550 con 0x7fb2e0102390 2024-05-10T11:14:17.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:17.862+0000 7fb2dffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2e0102d50 0x7fb2e019adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:17.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:17.862+0000 7fb2dffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2e0102d50 0x7fb2e019adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54912/0 (socket says 172.21.15.100:54912) 2024-05-10T11:14:17.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:17.862+0000 7fb2dffff700 1 -- 172.21.15.100:0/2372716782 learned_addr learned my addr 172.21.15.100:0/2372716782 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:14:17.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:17.862+0000 7fb2e4d39700 1 --2- 172.21.15.100:0/2372716782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb2e0102390 0x7fb2e019a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:17.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:17.862+0000 7fb2e553a700 1 --2- 172.21.15.100:0/2372716782 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2e0108400 0x7fb2e01a5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:17.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:17.862+0000 7fb2dffff700 1 -- 172.21.15.100:0/2372716782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb2e0102390 msgr2=0x7fb2e019a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:17.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:17.862+0000 7fb2dffff700 1 --2- 172.21.15.100:0/2372716782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb2e0102390 0x7fb2e019a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:17.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:17.862+0000 7fb2dffff700 1 -- 172.21.15.100:0/2372716782 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2e0108400 msgr2=0x7fb2e01a5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:17.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:17.862+0000 7fb2dffff700 1 --2- 172.21.15.100:0/2372716782 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2e0108400 0x7fb2e01a5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:17.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:17.862+0000 7fb2dffff700 1 -- 172.21.15.100:0/2372716782 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2e01a5e70 con 0x7fb2e0102d50 2024-05-10T11:14:17.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:17.862+0000 7fb2e4d39700 1 --2- 172.21.15.100:0/2372716782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb2e0102390 0x7fb2e019a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:14:17.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:17.862+0000 7fb2e553a700 1 --2- 172.21.15.100:0/2372716782 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2e0108400 0x7fb2e01a5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:14:17.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:17.862+0000 7fb2dffff700 1 --2- 172.21.15.100:0/2372716782 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2e0102d50 0x7fb2e019adf0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc00d950 tx=0x7fb2cc00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:14:17.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:17.862+0000 7fb2ddffb700 1 -- 172.21.15.100:0/2372716782 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc016070 con 0x7fb2e0102d50 2024-05-10T11:14:17.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:17.862+0000 7fb2e6f9d700 1 -- 172.21.15.100:0/2372716782 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2e01a60a0 con 0x7fb2e0102d50 2024-05-10T11:14:17.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:17.863+0000 7fb2ddffb700 1 -- 172.21.15.100:0/2372716782 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb2cc010460 con 0x7fb2e0102d50 2024-05-10T11:14:17.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:17.863+0000 7fb2ddffb700 1 -- 172.21.15.100:0/2372716782 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc00f620 con 0x7fb2e0102d50 2024-05-10T11:14:17.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:17.863+0000 7fb2e6f9d700 1 -- 172.21.15.100:0/2372716782 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2e01a6600 con 0x7fb2e0102d50 2024-05-10T11:14:17.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:17.864+0000 7fb2e6f9d700 1 -- 172.21.15.100:0/2372716782 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2e006a190 con 0x7fb2e0102d50 2024-05-10T11:14:17.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:17.865+0000 7fb2ddffb700 1 -- 172.21.15.100:0/2372716782 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fb2cc019070 con 0x7fb2e0102d50 2024-05-10T11:14:17.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:17.865+0000 7fb2ddffb700 1 --2- 172.21.15.100:0/2372716782 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fb2d00b2900 0x7fb2d00b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:17.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:17.866+0000 7fb2e4d39700 1 --2- 172.21.15.100:0/2372716782 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fb2d00b2900 0x7fb2d00b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:17.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:17.866+0000 7fb2ddffb700 1 -- 172.21.15.100:0/2372716782 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7fb2cc0e1c70 con 0x7fb2e0102d50 2024-05-10T11:14:17.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:17.866+0000 7fb2e4d39700 1 --2- 172.21.15.100:0/2372716782 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fb2d00b2900 0x7fb2d00b4dc0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4006010 tx=0x7fb2d401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:14:17.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:17.870+0000 7fb2ddffb700 1 -- 172.21.15.100:0/2372716782 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fb2cc0a9e30 con 0x7fb2e0102d50 2024-05-10T11:14:18.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:18.095+0000 7fb2e6f9d700 1 -- 172.21.15.100:0/2372716782 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fb2e002e510 con 0x7fb2e0102d50 2024-05-10T11:14:18.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:18.095+0000 7fb2ddffb700 1 -- 172.21.15.100:0/2372716782 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v73) ==== 84+0+65 (secure 0 0 0) 0x7fb2cc0a9730 con 0x7fb2e0102d50 2024-05-10T11:14:18.095 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:14:18.096 INFO:teuthology.orchestra.run.smithi100.stdout:[ 2024-05-10T11:14:18.096 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:14:18.096 INFO:teuthology.orchestra.run.smithi100.stdout: "poolnum": 1, 2024-05-10T11:14:18.096 INFO:teuthology.orchestra.run.smithi100.stdout: "poolname": ".mgr" 2024-05-10T11:14:18.096 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:14:18.096 INFO:teuthology.orchestra.run.smithi100.stdout:] 2024-05-10T11:14:18.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:18.097+0000 7fb2e6f9d700 1 -- 172.21.15.100:0/2372716782 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fb2d00b2900 msgr2=0x7fb2d00b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:18.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:18.097+0000 7fb2e6f9d700 1 --2- 172.21.15.100:0/2372716782 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fb2d00b2900 0x7fb2d00b4dc0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4006010 tx=0x7fb2d401a040 comp rx=0 tx=0).stop 2024-05-10T11:14:18.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:18.097+0000 7fb2e6f9d700 1 -- 172.21.15.100:0/2372716782 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2e0102d50 msgr2=0x7fb2e019adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:18.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:18.097+0000 7fb2e6f9d700 1 --2- 172.21.15.100:0/2372716782 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2e0102d50 0x7fb2e019adf0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc00d950 tx=0x7fb2cc00dd10 comp rx=0 tx=0).stop 2024-05-10T11:14:18.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:18.097+0000 7fb2e6f9d700 1 -- 172.21.15.100:0/2372716782 shutdown_connections 2024-05-10T11:14:18.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:18.097+0000 7fb2e6f9d700 1 --2- 172.21.15.100:0/2372716782 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fb2d00b2900 0x7fb2d00b4dc0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:18.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:18.097+0000 7fb2e6f9d700 1 --2- 172.21.15.100:0/2372716782 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2e0108400 0x7fb2e01a5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:18.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:18.097+0000 7fb2e6f9d700 1 --2- 172.21.15.100:0/2372716782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb2e0102390 0x7fb2e019a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:18.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:18.097+0000 7fb2e6f9d700 1 --2- 172.21.15.100:0/2372716782 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2e0102d50 0x7fb2e019adf0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:18.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:18.098+0000 7fb2e6f9d700 1 -- 172.21.15.100:0/2372716782 >> 172.21.15.100:0/2372716782 conn(0x7fb2e0079310 msgr2=0x7fb2e0101280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:14:18.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:18.098+0000 7fb2e6f9d700 1 -- 172.21.15.100:0/2372716782 shutdown_connections 2024-05-10T11:14:18.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:18.098+0000 7fb2e6f9d700 1 -- 172.21.15.100:0/2372716782 wait complete. 2024-05-10T11:14:18.831 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-10T11:14:18.831 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-10T11:14:19.073 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:14:19.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:19 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:19.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:19 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:19.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:19 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:14:19.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:19 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:14:19.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:19 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:19.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:19 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/2372716782' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-10T11:14:19.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:19 smithi100 ceph-mon[30296]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:19.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:19 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:19.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:19 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:19.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:19 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:14:19.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:19 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:14:19.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:19 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:19.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:19 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/2372716782' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-10T11:14:19.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:19 smithi178 ceph-mon[33359]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:19.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:19 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:19.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:19 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:19.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:19 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:14:19.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:19 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:14:19.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:19 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:14:19.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:19 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/2372716782' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-10T11:14:19.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:19 smithi188 ceph-mon[34075]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:20.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:20.788+0000 7fc6321b2700 1 -- 172.21.15.100:0/749737157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc62c102820 msgr2=0x7fc62c110290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:20.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:20.788+0000 7fc6321b2700 1 --2- 172.21.15.100:0/749737157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc62c102820 0x7fc62c110290 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc62400b0a0 tx=0x7fc62400b3b0 comp rx=0 tx=0).stop 2024-05-10T11:14:20.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:20.788+0000 7fc6321b2700 1 -- 172.21.15.100:0/749737157 shutdown_connections 2024-05-10T11:14:20.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:20.788+0000 7fc6321b2700 1 --2- 172.21.15.100:0/749737157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc62c102820 0x7fc62c110290 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:20.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:20.788+0000 7fc6321b2700 1 --2- 172.21.15.100:0/749737157 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc62c108580 0x7fc62c108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:20.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:20.788+0000 7fc6321b2700 1 --2- 172.21.15.100:0/749737157 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc62c101e60 0x7fc62c1022e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:20.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:20.788+0000 7fc6321b2700 1 -- 172.21.15.100:0/749737157 >> 172.21.15.100:0/749737157 conn(0x7fc62c0776c0 msgr2=0x7fc62c077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:14:20.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:20.788+0000 7fc6321b2700 1 -- 172.21.15.100:0/749737157 shutdown_connections 2024-05-10T11:14:20.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:20.788+0000 7fc6321b2700 1 -- 172.21.15.100:0/749737157 wait complete. 2024-05-10T11:14:20.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:20.789+0000 7fc6321b2700 1 Processor -- start 2024-05-10T11:14:20.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:20.789+0000 7fc6321b2700 1 -- start start 2024-05-10T11:14:20.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:20.789+0000 7fc6321b2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc62c101e60 0x7fc62c19f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:20.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:20.789+0000 7fc6321b2700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc62c102820 0x7fc62c19fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:20.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:20.789+0000 7fc6321b2700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc62c108580 0x7fc62c199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:20.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:20.789+0000 7fc6321b2700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc62c113520 con 0x7fc62c101e60 2024-05-10T11:14:20.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:20.789+0000 7fc6321b2700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc62c1133a0 con 0x7fc62c108580 2024-05-10T11:14:20.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:20.789+0000 7fc6321b2700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc62c1136a0 con 0x7fc62c102820 2024-05-10T11:14:20.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:20.790+0000 7fc62b7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc62c101e60 0x7fc62c19f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:20.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:20.790+0000 7fc62b7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc62c101e60 0x7fc62c19f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54922/0 (socket says 172.21.15.100:54922) 2024-05-10T11:14:20.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:20.790+0000 7fc62b7fe700 1 -- 172.21.15.100:0/689273307 learned_addr learned my addr 172.21.15.100:0/689273307 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:14:20.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:20.790+0000 7fc62affd700 1 --2- 172.21.15.100:0/689273307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc62c102820 0x7fc62c19fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:20.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:20.790+0000 7fc62bfff700 1 --2- 172.21.15.100:0/689273307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc62c108580 0x7fc62c199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:20.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:20.790+0000 7fc62b7fe700 1 -- 172.21.15.100:0/689273307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc62c102820 msgr2=0x7fc62c19fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:20.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:20.790+0000 7fc62b7fe700 1 --2- 172.21.15.100:0/689273307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc62c102820 0x7fc62c19fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:20.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:20.790+0000 7fc62b7fe700 1 -- 172.21.15.100:0/689273307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc62c108580 msgr2=0x7fc62c199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:20.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:20.790+0000 7fc62b7fe700 1 --2- 172.21.15.100:0/689273307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc62c108580 0x7fc62c199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:20.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:20.790+0000 7fc62b7fe700 1 -- 172.21.15.100:0/689273307 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc62c19a000 con 0x7fc62c101e60 2024-05-10T11:14:20.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:20.790+0000 7fc62bfff700 1 --2- 172.21.15.100:0/689273307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc62c108580 0x7fc62c199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:14:20.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:20.790+0000 7fc62b7fe700 1 --2- 172.21.15.100:0/689273307 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc62c101e60 0x7fc62c19f650 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc61c00d980 tx=0x7fc61c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:14:20.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:20.790+0000 7fc628ff9700 1 -- 172.21.15.100:0/689273307 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61c017030 con 0x7fc62c101e60 2024-05-10T11:14:20.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:20.790+0000 7fc628ff9700 1 -- 172.21.15.100:0/689273307 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc61c0044b0 con 0x7fc62c101e60 2024-05-10T11:14:20.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:20.790+0000 7fc6321b2700 1 -- 172.21.15.100:0/689273307 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc62c19a290 con 0x7fc62c101e60 2024-05-10T11:14:20.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:20.790+0000 7fc628ff9700 1 -- 172.21.15.100:0/689273307 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61c003970 con 0x7fc62c101e60 2024-05-10T11:14:20.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:20.790+0000 7fc6321b2700 1 -- 172.21.15.100:0/689273307 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc62c1a64b0 con 0x7fc62c101e60 2024-05-10T11:14:20.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:20.791+0000 7fc6321b2700 1 -- 172.21.15.100:0/689273307 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc62c06b690 con 0x7fc62c101e60 2024-05-10T11:14:20.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:20.793+0000 7fc628ff9700 1 -- 172.21.15.100:0/689273307 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fc61c01a070 con 0x7fc62c101e60 2024-05-10T11:14:20.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:20.793+0000 7fc628ff9700 1 --2- 172.21.15.100:0/689273307 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc6140b2880 0x7fc6140b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:20.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:20.793+0000 7fc62affd700 1 --2- 172.21.15.100:0/689273307 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc6140b2880 0x7fc6140b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:20.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:20.794+0000 7fc628ff9700 1 -- 172.21.15.100:0/689273307 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7fc61c0e1b10 con 0x7fc62c101e60 2024-05-10T11:14:20.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:20.794+0000 7fc62affd700 1 --2- 172.21.15.100:0/689273307 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc6140b2880 0x7fc6140b4d40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc620005fd0 tx=0x7fc62000a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:14:20.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:20.797+0000 7fc628ff9700 1 -- 172.21.15.100:0/689273307 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fc61c0a9cd0 con 0x7fc62c101e60 2024-05-10T11:14:20.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:20.998+0000 7fc6321b2700 1 -- 172.21.15.100:0/689273307 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fc62c19ad00 con 0x7fc62c101e60 2024-05-10T11:14:21.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:21.234+0000 7fc628ff9700 1 -- 172.21.15.100:0/689273307 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v74) ==== 138+0+0 (secure 0 0 0) 0x7fc61c0a95d0 con 0x7fc62c101e60 2024-05-10T11:14:21.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:21.237+0000 7fc6321b2700 1 -- 172.21.15.100:0/689273307 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc6140b2880 msgr2=0x7fc6140b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:21.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:21.237+0000 7fc6321b2700 1 --2- 172.21.15.100:0/689273307 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc6140b2880 0x7fc6140b4d40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc620005fd0 tx=0x7fc62000a560 comp rx=0 tx=0).stop 2024-05-10T11:14:21.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:21.237+0000 7fc6321b2700 1 -- 172.21.15.100:0/689273307 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc62c101e60 msgr2=0x7fc62c19f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:21.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:21.237+0000 7fc6321b2700 1 --2- 172.21.15.100:0/689273307 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc62c101e60 0x7fc62c19f650 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc61c00d980 tx=0x7fc61c00dd40 comp rx=0 tx=0).stop 2024-05-10T11:14:21.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:21.237+0000 7fc6321b2700 1 -- 172.21.15.100:0/689273307 shutdown_connections 2024-05-10T11:14:21.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:21.237+0000 7fc6321b2700 1 --2- 172.21.15.100:0/689273307 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc6140b2880 0x7fc6140b4d40 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:21.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:21.237+0000 7fc6321b2700 1 --2- 172.21.15.100:0/689273307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc62c108580 0x7fc62c199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:21.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:21.237+0000 7fc6321b2700 1 --2- 172.21.15.100:0/689273307 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc62c101e60 0x7fc62c19f650 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:21.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:21.237+0000 7fc6321b2700 1 --2- 172.21.15.100:0/689273307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc62c102820 0x7fc62c19fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:21.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:21.237+0000 7fc6321b2700 1 -- 172.21.15.100:0/689273307 >> 172.21.15.100:0/689273307 conn(0x7fc62c0776c0 msgr2=0x7fc62c100ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:14:21.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:21.237+0000 7fc6321b2700 1 -- 172.21.15.100:0/689273307 shutdown_connections 2024-05-10T11:14:21.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:21.238+0000 7fc6321b2700 1 -- 172.21.15.100:0/689273307 wait complete. 2024-05-10T11:14:21.240 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_metadata' created 2024-05-10T11:14:21.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:21 smithi178 ceph-mon[33359]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:21.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:21 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/689273307' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-10T11:14:21.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:21 smithi188 ceph-mon[34075]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:21.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:21 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/689273307' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-10T11:14:21.532 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:21 smithi100 ceph-mon[30296]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:21.532 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:21 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/689273307' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-10T11:14:21.959 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-05-10T11:14:22.212 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:14:22.241 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:22 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/689273307' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-10T11:14:22.242 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:22 smithi100 ceph-mon[30296]: osdmap e74: 12 total, 12 up, 12 in 2024-05-10T11:14:22.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:22 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/689273307' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-10T11:14:22.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:22 smithi178 ceph-mon[33359]: osdmap e74: 12 total, 12 up, 12 in 2024-05-10T11:14:22.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:22 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/689273307' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-10T11:14:22.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:22 smithi188 ceph-mon[34075]: osdmap e74: 12 total, 12 up, 12 in 2024-05-10T11:14:23.423 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:23 smithi100 ceph-mon[30296]: pgmap v285: 33 pgs: 7 creating+peering, 1 active+clean, 25 unknown; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:23.424 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:23 smithi100 ceph-mon[30296]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-10T11:14:23.424 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:23 smithi100 ceph-mon[30296]: osdmap e75: 12 total, 12 up, 12 in 2024-05-10T11:14:23.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:23 smithi178 ceph-mon[33359]: pgmap v285: 33 pgs: 7 creating+peering, 1 active+clean, 25 unknown; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:23.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:23 smithi178 ceph-mon[33359]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-10T11:14:23.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:23 smithi178 ceph-mon[33359]: osdmap e75: 12 total, 12 up, 12 in 2024-05-10T11:14:23.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:23 smithi188 ceph-mon[34075]: pgmap v285: 33 pgs: 7 creating+peering, 1 active+clean, 25 unknown; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:23.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:23 smithi188 ceph-mon[34075]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-10T11:14:23.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:23 smithi188 ceph-mon[34075]: osdmap e75: 12 total, 12 up, 12 in 2024-05-10T11:14:23.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:23.963+0000 7f7ec7e7b700 1 -- 172.21.15.100:0/2653815509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ec0108b50 msgr2=0x7f7ec010f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:23.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:23.963+0000 7f7ec7e7b700 1 --2- 172.21.15.100:0/2653815509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ec0108b50 0x7f7ec010f310 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7eb800b0d0 tx=0x7f7eb800b3e0 comp rx=0 tx=0).stop 2024-05-10T11:14:23.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:23.963+0000 7f7ec7e7b700 1 -- 172.21.15.100:0/2653815509 shutdown_connections 2024-05-10T11:14:23.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:23.963+0000 7f7ec7e7b700 1 --2- 172.21.15.100:0/2653815509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ec0108b50 0x7f7ec010f310 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:23.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:23.963+0000 7f7ec7e7b700 1 --2- 172.21.15.100:0/2653815509 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ec0106320 0x7f7ec0106700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:23.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:23.963+0000 7f7ec7e7b700 1 --2- 172.21.15.100:0/2653815509 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ec0108190 0x7f7ec0108610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:23.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:23.963+0000 7f7ec7e7b700 1 -- 172.21.15.100:0/2653815509 >> 172.21.15.100:0/2653815509 conn(0x7f7ec00776c0 msgr2=0x7f7ec0077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:14:23.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:23.963+0000 7f7ec7e7b700 1 -- 172.21.15.100:0/2653815509 shutdown_connections 2024-05-10T11:14:23.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:23.963+0000 7f7ec7e7b700 1 -- 172.21.15.100:0/2653815509 wait complete. 2024-05-10T11:14:23.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:23.964+0000 7f7ec7e7b700 1 Processor -- start 2024-05-10T11:14:23.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:23.964+0000 7f7ec7e7b700 1 -- start start 2024-05-10T11:14:23.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:23.964+0000 7f7ec7e7b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ec0106320 0x7f7ec019e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:23.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:23.964+0000 7f7ec7e7b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ec0108190 0x7f7ec019ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:23.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:23.964+0000 7f7ec7e7b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ec0108b50 0x7f7ec01988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:23.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:23.964+0000 7f7ec7e7b700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec0112520 con 0x7f7ec0108b50 2024-05-10T11:14:23.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:23.964+0000 7f7ec7e7b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec01123a0 con 0x7f7ec0106320 2024-05-10T11:14:23.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:23.964+0000 7f7ec7e7b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec01126a0 con 0x7f7ec0108190 2024-05-10T11:14:23.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:23.964+0000 7f7ec6418700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ec0108b50 0x7f7ec01988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:23.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:23.964+0000 7f7ec6418700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ec0108b50 0x7f7ec01988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54950/0 (socket says 172.21.15.100:54950) 2024-05-10T11:14:23.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:23.964+0000 7f7ec6418700 1 -- 172.21.15.100:0/1874234358 learned_addr learned my addr 172.21.15.100:0/1874234358 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:14:23.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:23.964+0000 7f7ec5c17700 1 --2- 172.21.15.100:0/1874234358 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ec0106320 0x7f7ec019e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:23.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:23.964+0000 7f7ec5416700 1 --2- 172.21.15.100:0/1874234358 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ec0108190 0x7f7ec019ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:23.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:23.965+0000 7f7ec6418700 1 -- 172.21.15.100:0/1874234358 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ec0108190 msgr2=0x7f7ec019ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:23.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:23.965+0000 7f7ec6418700 1 --2- 172.21.15.100:0/1874234358 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ec0108190 0x7f7ec019ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:23.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:23.965+0000 7f7ec6418700 1 -- 172.21.15.100:0/1874234358 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ec0106320 msgr2=0x7f7ec019e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:23.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:23.965+0000 7f7ec6418700 1 --2- 172.21.15.100:0/1874234358 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ec0106320 0x7f7ec019e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:23.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:23.965+0000 7f7ec6418700 1 -- 172.21.15.100:0/1874234358 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ec0199040 con 0x7f7ec0108b50 2024-05-10T11:14:23.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:23.965+0000 7f7ec5416700 1 --2- 172.21.15.100:0/1874234358 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ec0108190 0x7f7ec019ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:14:23.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:23.965+0000 7f7ec5c17700 1 --2- 172.21.15.100:0/1874234358 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ec0106320 0x7f7ec019e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:14:23.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:23.965+0000 7f7ec6418700 1 --2- 172.21.15.100:0/1874234358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ec0108b50 0x7f7ec01988e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8007d90 tx=0x7f7eb8007e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:14:23.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:23.965+0000 7f7eb6ffd700 1 -- 172.21.15.100:0/1874234358 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb801f030 con 0x7f7ec0108b50 2024-05-10T11:14:23.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:23.965+0000 7f7eb6ffd700 1 -- 172.21.15.100:0/1874234358 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7eb801bb90 con 0x7f7ec0108b50 2024-05-10T11:14:23.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:23.965+0000 7f7eb6ffd700 1 -- 172.21.15.100:0/1874234358 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb8012c90 con 0x7f7ec0108b50 2024-05-10T11:14:23.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:23.965+0000 7f7ec7e7b700 1 -- 172.21.15.100:0/1874234358 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ec01992d0 con 0x7f7ec0108b50 2024-05-10T11:14:23.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:23.965+0000 7f7ec7e7b700 1 -- 172.21.15.100:0/1874234358 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ec0101e40 con 0x7f7ec0108b50 2024-05-10T11:14:23.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:23.966+0000 7f7ec7e7b700 1 -- 172.21.15.100:0/1874234358 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ec006a190 con 0x7f7ec0108b50 2024-05-10T11:14:23.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:23.968+0000 7f7eb6ffd700 1 -- 172.21.15.100:0/1874234358 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f7eb8022030 con 0x7f7ec0108b50 2024-05-10T11:14:23.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:23.968+0000 7f7eb6ffd700 1 --2- 172.21.15.100:0/1874234358 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f7eac0b2880 0x7f7eac0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:23.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:23.968+0000 7f7ec5c17700 1 --2- 172.21.15.100:0/1874234358 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f7eac0b2880 0x7f7eac0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:23.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:23.968+0000 7f7eb6ffd700 1 -- 172.21.15.100:0/1874234358 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7497+0+0 (secure 0 0 0) 0x7f7eb800f020 con 0x7f7ec0108b50 2024-05-10T11:14:23.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:23.969+0000 7f7ec5c17700 1 --2- 172.21.15.100:0/1874234358 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f7eac0b2880 0x7f7eac0b4d40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0107870 tx=0x7f7ebc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:14:23.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:23.972+0000 7f7eb6ffd700 1 -- 172.21.15.100:0/1874234358 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f7eb8029020 con 0x7f7ec0108b50 2024-05-10T11:14:24.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:24.174+0000 7f7ec7e7b700 1 -- 172.21.15.100:0/1874234358 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f7ec01027e0 con 0x7f7ec0108b50 2024-05-10T11:14:24.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:24.252+0000 7f7eb6ffd700 1 -- 172.21.15.100:0/1874234358 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v77) ==== 171+0+0 (secure 0 0 0) 0x7f7eb80ab880 con 0x7f7ec0108b50 2024-05-10T11:14:24.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:24.255+0000 7f7ec7e7b700 1 -- 172.21.15.100:0/1874234358 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f7eac0b2880 msgr2=0x7f7eac0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:24.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:24.255+0000 7f7ec7e7b700 1 --2- 172.21.15.100:0/1874234358 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f7eac0b2880 0x7f7eac0b4d40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0107870 tx=0x7f7ebc008040 comp rx=0 tx=0).stop 2024-05-10T11:14:24.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:24.255+0000 7f7ec7e7b700 1 -- 172.21.15.100:0/1874234358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ec0108b50 msgr2=0x7f7ec01988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:24.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:24.255+0000 7f7ec7e7b700 1 --2- 172.21.15.100:0/1874234358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ec0108b50 0x7f7ec01988e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8007d90 tx=0x7f7eb8007e70 comp rx=0 tx=0).stop 2024-05-10T11:14:24.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:24.255+0000 7f7ec7e7b700 1 -- 172.21.15.100:0/1874234358 shutdown_connections 2024-05-10T11:14:24.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:24.255+0000 7f7ec7e7b700 1 --2- 172.21.15.100:0/1874234358 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f7eac0b2880 0x7f7eac0b4d40 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:24.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:24.255+0000 7f7ec7e7b700 1 --2- 172.21.15.100:0/1874234358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ec0108b50 0x7f7ec01988e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:24.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:24.255+0000 7f7ec7e7b700 1 --2- 172.21.15.100:0/1874234358 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ec0106320 0x7f7ec019e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:24.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:24.255+0000 7f7ec7e7b700 1 --2- 172.21.15.100:0/1874234358 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ec0108190 0x7f7ec019ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:24.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:24.255+0000 7f7ec7e7b700 1 -- 172.21.15.100:0/1874234358 >> 172.21.15.100:0/1874234358 conn(0x7f7ec00776c0 msgr2=0x7f7ec00fff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:14:24.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:24.255+0000 7f7ec7e7b700 1 -- 172.21.15.100:0/1874234358 shutdown_connections 2024-05-10T11:14:24.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:24.255+0000 7f7ec7e7b700 1 -- 172.21.15.100:0/1874234358 wait complete. 2024-05-10T11:14:24.256 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_data' created 2024-05-10T11:14:24.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:24 smithi188 ceph-mon[34075]: osdmap e76: 12 total, 12 up, 12 in 2024-05-10T11:14:24.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:24 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/1874234358' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-10T11:14:24.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:24 smithi100 ceph-mon[30296]: osdmap e76: 12 total, 12 up, 12 in 2024-05-10T11:14:24.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:24 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/1874234358' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-10T11:14:24.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:24 smithi178 ceph-mon[33359]: osdmap e76: 12 total, 12 up, 12 in 2024-05-10T11:14:24.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:24 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/1874234358' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-10T11:14:24.935 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-10T11:14:25.178 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:14:25.264 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:25 smithi100 ceph-mon[30296]: pgmap v288: 33 pgs: 12 active+clean, 7 creating+peering, 14 unknown; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:25.264 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:25 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/1874234358' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-10T11:14:25.264 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:25 smithi100 ceph-mon[30296]: osdmap e77: 12 total, 12 up, 12 in 2024-05-10T11:14:25.526 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:25 smithi188 ceph-mon[34075]: pgmap v288: 33 pgs: 12 active+clean, 7 creating+peering, 14 unknown; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:25.526 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:25 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/1874234358' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-10T11:14:25.526 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:25 smithi188 ceph-mon[34075]: osdmap e77: 12 total, 12 up, 12 in 2024-05-10T11:14:25.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:25 smithi178 ceph-mon[33359]: pgmap v288: 33 pgs: 12 active+clean, 7 creating+peering, 14 unknown; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:25.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:25 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/1874234358' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-10T11:14:25.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:25 smithi178 ceph-mon[33359]: osdmap e77: 12 total, 12 up, 12 in 2024-05-10T11:14:26.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:26 smithi188 ceph-mon[34075]: osdmap e78: 12 total, 12 up, 12 in 2024-05-10T11:14:26.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:26 smithi100 ceph-mon[30296]: osdmap e78: 12 total, 12 up, 12 in 2024-05-10T11:14:26.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:26 smithi178 ceph-mon[33359]: osdmap e78: 12 total, 12 up, 12 in 2024-05-10T11:14:26.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:26.985+0000 7f712b2a3700 1 -- 172.21.15.100:0/10786774 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7124103560 msgr2=0x7f71241039e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:26.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:26.985+0000 7f712b2a3700 1 --2- 172.21.15.100:0/10786774 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7124103560 0x7f71241039e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f71140099c0 tx=0x7f7114009cd0 comp rx=0 tx=0).stop 2024-05-10T11:14:26.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:26.985+0000 7f712b2a3700 1 -- 172.21.15.100:0/10786774 shutdown_connections 2024-05-10T11:14:26.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:26.985+0000 7f712b2a3700 1 --2- 172.21.15.100:0/10786774 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7124103f20 0x7f7124110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:26.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:26.985+0000 7f712b2a3700 1 --2- 172.21.15.100:0/10786774 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7124109540 0x7f7124109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:26.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:26.985+0000 7f712b2a3700 1 --2- 172.21.15.100:0/10786774 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7124103560 0x7f71241039e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:26.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:26.985+0000 7f712b2a3700 1 -- 172.21.15.100:0/10786774 >> 172.21.15.100:0/10786774 conn(0x7f712407a660 msgr2=0x7f712407aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:14:26.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:26.985+0000 7f712b2a3700 1 -- 172.21.15.100:0/10786774 shutdown_connections 2024-05-10T11:14:26.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:26.985+0000 7f712b2a3700 1 -- 172.21.15.100:0/10786774 wait complete. 2024-05-10T11:14:26.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:26.986+0000 7f712b2a3700 1 Processor -- start 2024-05-10T11:14:26.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:26.986+0000 7f712b2a3700 1 -- start start 2024-05-10T11:14:26.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:26.986+0000 7f712b2a3700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7124103560 0x7f71240757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:26.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:26.986+0000 7f712b2a3700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7124103f20 0x7f7124075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:26.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:26.986+0000 7f712b2a3700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7124109540 0x7f7124076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:26.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:26.986+0000 7f712b2a3700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7124113550 con 0x7f7124103560 2024-05-10T11:14:26.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:26.986+0000 7f712b2a3700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f71241133d0 con 0x7f7124103f20 2024-05-10T11:14:26.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:26.986+0000 7f712b2a3700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f71241136d0 con 0x7f7124109540 2024-05-10T11:14:26.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:26.986+0000 7f712903f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7124103560 0x7f71240757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:26.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:26.986+0000 7f712903f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7124103560 0x7f71240757a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36990/0 (socket says 172.21.15.100:36990) 2024-05-10T11:14:26.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:26.986+0000 7f712903f700 1 -- 172.21.15.100:0/665898341 learned_addr learned my addr 172.21.15.100:0/665898341 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:14:26.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:26.986+0000 7f712883e700 1 --2- 172.21.15.100:0/665898341 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7124103f20 0x7f7124075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:26.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:26.986+0000 7f7129840700 1 --2- 172.21.15.100:0/665898341 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7124109540 0x7f7124076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:26.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:26.986+0000 7f712903f700 1 -- 172.21.15.100:0/665898341 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7124109540 msgr2=0x7f7124076220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:26.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:26.986+0000 7f712903f700 1 --2- 172.21.15.100:0/665898341 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7124109540 0x7f7124076220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:26.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:26.986+0000 7f712903f700 1 -- 172.21.15.100:0/665898341 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7124103f20 msgr2=0x7f7124075ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:26.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:26.986+0000 7f712903f700 1 --2- 172.21.15.100:0/665898341 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7124103f20 0x7f7124075ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:26.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:26.986+0000 7f712903f700 1 -- 172.21.15.100:0/665898341 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71241a7f90 con 0x7f7124103560 2024-05-10T11:14:26.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:26.986+0000 7f712903f700 1 --2- 172.21.15.100:0/665898341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7124103560 0x7f71240757a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f712000d950 tx=0x7f712000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:14:26.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:26.987+0000 7f711a7fc700 1 -- 172.21.15.100:0/665898341 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7120016070 con 0x7f7124103560 2024-05-10T11:14:26.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:26.987+0000 7f712b2a3700 1 -- 172.21.15.100:0/665898341 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71241a8280 con 0x7f7124103560 2024-05-10T11:14:26.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:26.987+0000 7f711a7fc700 1 -- 172.21.15.100:0/665898341 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7120009dd0 con 0x7f7124103560 2024-05-10T11:14:26.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:26.987+0000 7f711a7fc700 1 -- 172.21.15.100:0/665898341 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f712000f3c0 con 0x7f7124103560 2024-05-10T11:14:26.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:26.987+0000 7f712b2a3700 1 -- 172.21.15.100:0/665898341 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71241a8790 con 0x7f7124103560 2024-05-10T11:14:26.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:26.988+0000 7f712b2a3700 1 -- 172.21.15.100:0/665898341 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7124104bb0 con 0x7f7124103560 2024-05-10T11:14:26.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:26.989+0000 7f711a7fc700 1 -- 172.21.15.100:0/665898341 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f7120019070 con 0x7f7124103560 2024-05-10T11:14:26.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:26.990+0000 7f711a7fc700 1 --2- 172.21.15.100:0/665898341 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f71100b2880 0x7f71100b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:26.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:26.990+0000 7f712883e700 1 --2- 172.21.15.100:0/665898341 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f71100b2880 0x7f71100b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:26.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:26.990+0000 7f711a7fc700 1 -- 172.21.15.100:0/665898341 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7879+0+0 (secure 0 0 0) 0x7f71200e1c90 con 0x7f7124103560 2024-05-10T11:14:26.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:26.990+0000 7f712883e700 1 --2- 172.21.15.100:0/665898341 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f71100b2880 0x7f71100b4d40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7114000c00 tx=0x7f7114005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:14:26.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:26.994+0000 7f711a7fc700 1 -- 172.21.15.100:0/665898341 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f71200a9b60 con 0x7f7124103560 2024-05-10T11:14:27.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:27.241+0000 7f712b2a3700 1 -- 172.21.15.100:0/665898341 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f712402e510 con 0x7f7124103560 2024-05-10T11:14:27.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:27.271+0000 7f711a7fc700 1 -- 172.21.15.100:0/665898341 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f71200a9460 con 0x7f7124103560 2024-05-10T11:14:27.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:27.273+0000 7f712b2a3700 1 -- 172.21.15.100:0/665898341 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f71100b2880 msgr2=0x7f71100b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:27.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:27.273+0000 7f712b2a3700 1 --2- 172.21.15.100:0/665898341 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f71100b2880 0x7f71100b4d40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7114000c00 tx=0x7f7114005fb0 comp rx=0 tx=0).stop 2024-05-10T11:14:27.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:27.273+0000 7f712b2a3700 1 -- 172.21.15.100:0/665898341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7124103560 msgr2=0x7f71240757a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:27.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:27.273+0000 7f712b2a3700 1 --2- 172.21.15.100:0/665898341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7124103560 0x7f71240757a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f712000d950 tx=0x7f712000dc60 comp rx=0 tx=0).stop 2024-05-10T11:14:27.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:27.273+0000 7f712b2a3700 1 -- 172.21.15.100:0/665898341 shutdown_connections 2024-05-10T11:14:27.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:27.273+0000 7f712b2a3700 1 --2- 172.21.15.100:0/665898341 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f71100b2880 0x7f71100b4d40 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:27.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:27.273+0000 7f712b2a3700 1 --2- 172.21.15.100:0/665898341 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7124109540 0x7f7124076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:27.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:27.273+0000 7f712b2a3700 1 --2- 172.21.15.100:0/665898341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7124103560 0x7f71240757a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:27.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:27.273+0000 7f712b2a3700 1 --2- 172.21.15.100:0/665898341 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7124103f20 0x7f7124075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:27.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:27.274+0000 7f712b2a3700 1 -- 172.21.15.100:0/665898341 >> 172.21.15.100:0/665898341 conn(0x7f712407a660 msgr2=0x7f7124102300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:14:27.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:27.274+0000 7f712b2a3700 1 -- 172.21.15.100:0/665898341 shutdown_connections 2024-05-10T11:14:27.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:27.274+0000 7f712b2a3700 1 -- 172.21.15.100:0/665898341 wait complete. 2024-05-10T11:14:27.275 INFO:teuthology.orchestra.run.smithi100.stderr:new fs with metadata pool 2 and data pool 3 2024-05-10T11:14:27.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:27 smithi100 ceph-mon[30296]: pgmap v291: 97 pgs: 30 active+clean, 16 creating+peering, 51 unknown; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:27.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:27 smithi100 ceph-mon[30296]: osdmap e79: 12 total, 12 up, 12 in 2024-05-10T11:14:27.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:27 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/665898341' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-10T11:14:27.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:27 smithi100 ceph-mon[30296]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-05-10T11:14:27.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:27 smithi178 ceph-mon[33359]: pgmap v291: 97 pgs: 30 active+clean, 16 creating+peering, 51 unknown; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:27.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:27 smithi178 ceph-mon[33359]: osdmap e79: 12 total, 12 up, 12 in 2024-05-10T11:14:27.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:27 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/665898341' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-10T11:14:27.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:27 smithi178 ceph-mon[33359]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-05-10T11:14:27.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:27 smithi188 ceph-mon[34075]: pgmap v291: 97 pgs: 30 active+clean, 16 creating+peering, 51 unknown; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:27.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:27 smithi188 ceph-mon[34075]: osdmap e79: 12 total, 12 up, 12 in 2024-05-10T11:14:27.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:27 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/665898341' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-10T11:14:27.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:27 smithi188 ceph-mon[34075]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-05-10T11:14:28.052 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-05-10T11:14:28.053 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-05-10T11:14:28.299 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:14:28.603 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:28 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/665898341' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-10T11:14:28.603 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:28 smithi100 ceph-mon[30296]: osdmap e80: 12 total, 12 up, 12 in 2024-05-10T11:14:28.603 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:28 smithi100 ceph-mon[30296]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-05-10T11:14:28.603 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:28 smithi100 ceph-mon[30296]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-05-10T11:14:28.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:28 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/665898341' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-10T11:14:28.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:28 smithi178 ceph-mon[33359]: osdmap e80: 12 total, 12 up, 12 in 2024-05-10T11:14:28.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:28 smithi178 ceph-mon[33359]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-05-10T11:14:28.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:28 smithi178 ceph-mon[33359]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-05-10T11:14:28.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:28 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/665898341' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-10T11:14:28.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:28 smithi188 ceph-mon[34075]: osdmap e80: 12 total, 12 up, 12 in 2024-05-10T11:14:28.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:28 smithi188 ceph-mon[34075]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-05-10T11:14:28.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:28 smithi188 ceph-mon[34075]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-05-10T11:14:29.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:29 smithi100 ceph-mon[30296]: pgmap v294: 97 pgs: 65 active+clean, 9 creating+peering, 23 unknown; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 4 op/s 2024-05-10T11:14:29.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:29 smithi100 ceph-mon[30296]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-10T11:14:29.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:29 smithi100 ceph-mon[30296]: Cluster is now healthy 2024-05-10T11:14:29.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:29 smithi100 ceph-mon[30296]: mds.? [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] up:active 2024-05-10T11:14:29.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:29 smithi100 ceph-mon[30296]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-10T11:14:29.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:29 smithi100 ceph-mon[30296]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-10T11:14:29.741 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:29 smithi188 ceph-mon[34075]: pgmap v294: 97 pgs: 65 active+clean, 9 creating+peering, 23 unknown; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 4 op/s 2024-05-10T11:14:29.741 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:29 smithi188 ceph-mon[34075]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-10T11:14:29.742 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:29 smithi188 ceph-mon[34075]: Cluster is now healthy 2024-05-10T11:14:29.742 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:29 smithi188 ceph-mon[34075]: mds.? [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] up:active 2024-05-10T11:14:29.742 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:29 smithi188 ceph-mon[34075]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-10T11:14:29.742 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:29 smithi188 ceph-mon[34075]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-10T11:14:29.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:29 smithi178 ceph-mon[33359]: pgmap v294: 97 pgs: 65 active+clean, 9 creating+peering, 23 unknown; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 4 op/s 2024-05-10T11:14:29.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:29 smithi178 ceph-mon[33359]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-10T11:14:29.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:29 smithi178 ceph-mon[33359]: Cluster is now healthy 2024-05-10T11:14:29.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:29 smithi178 ceph-mon[33359]: mds.? [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] up:active 2024-05-10T11:14:29.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:29 smithi178 ceph-mon[33359]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-10T11:14:29.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:29 smithi178 ceph-mon[33359]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-10T11:14:30.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:30.013+0000 7fdc8db19700 1 -- 172.21.15.100:0/4068066340 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc881095d0 msgr2=0x7fdc881099b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:30.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:30.013+0000 7fdc8db19700 1 --2- 172.21.15.100:0/4068066340 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc881095d0 0x7fdc881099b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fdc780099c0 tx=0x7fdc78009cd0 comp rx=0 tx=0).stop 2024-05-10T11:14:30.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:30.013+0000 7fdc8db19700 1 -- 172.21.15.100:0/4068066340 shutdown_connections 2024-05-10T11:14:30.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:30.013+0000 7fdc8db19700 1 --2- 172.21.15.100:0/4068066340 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc88103ef0 0x7fdc881103a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:30.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:30.013+0000 7fdc8db19700 1 --2- 172.21.15.100:0/4068066340 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc881095d0 0x7fdc881099b0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:30.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:30.013+0000 7fdc8db19700 1 --2- 172.21.15.100:0/4068066340 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc88103530 0x7fdc881039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:30.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:30.013+0000 7fdc8db19700 1 -- 172.21.15.100:0/4068066340 >> 172.21.15.100:0/4068066340 conn(0x7fdc8807a690 msgr2=0x7fdc8807aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:14:30.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:30.013+0000 7fdc8db19700 1 -- 172.21.15.100:0/4068066340 shutdown_connections 2024-05-10T11:14:30.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:30.013+0000 7fdc8db19700 1 -- 172.21.15.100:0/4068066340 wait complete. 2024-05-10T11:14:30.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:30.014+0000 7fdc8db19700 1 Processor -- start 2024-05-10T11:14:30.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:30.014+0000 7fdc8db19700 1 -- start start 2024-05-10T11:14:30.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:30.014+0000 7fdc8db19700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc88103530 0x7fdc880757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:30.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:30.014+0000 7fdc8db19700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc88103ef0 0x7fdc88075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:30.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:30.014+0000 7fdc8db19700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc881095d0 0x7fdc88076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:30.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:30.014+0000 7fdc8db19700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdc88113580 con 0x7fdc88103530 2024-05-10T11:14:30.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:30.014+0000 7fdc8db19700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdc88113400 con 0x7fdc88103ef0 2024-05-10T11:14:30.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:30.014+0000 7fdc8db19700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fdc88113700 con 0x7fdc881095d0 2024-05-10T11:14:30.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:30.014+0000 7fdc877fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc88103530 0x7fdc880757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:30.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:30.014+0000 7fdc877fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc88103530 0x7fdc880757a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37016/0 (socket says 172.21.15.100:37016) 2024-05-10T11:14:30.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:30.014+0000 7fdc877fe700 1 -- 172.21.15.100:0/504252880 learned_addr learned my addr 172.21.15.100:0/504252880 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:14:30.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:30.014+0000 7fdc7ffff700 1 --2- 172.21.15.100:0/504252880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc88103ef0 0x7fdc88075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:30.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:30.014+0000 7fdc877fe700 1 -- 172.21.15.100:0/504252880 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc881095d0 msgr2=0x7fdc88076220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:30.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:30.014+0000 7fdc877fe700 1 --2- 172.21.15.100:0/504252880 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc881095d0 0x7fdc88076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:30.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:30.014+0000 7fdc877fe700 1 -- 172.21.15.100:0/504252880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc88103ef0 msgr2=0x7fdc88075ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:30.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:30.015+0000 7fdc877fe700 1 --2- 172.21.15.100:0/504252880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc88103ef0 0x7fdc88075ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:30.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:30.015+0000 7fdc877fe700 1 -- 172.21.15.100:0/504252880 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc881a7fc0 con 0x7fdc88103530 2024-05-10T11:14:30.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:30.015+0000 7fdc877fe700 1 --2- 172.21.15.100:0/504252880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc88103530 0x7fdc880757a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdc78000c00 tx=0x7fdc7800f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:14:30.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:30.015+0000 7fdc857fa700 1 -- 172.21.15.100:0/504252880 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc7800f940 con 0x7fdc88103530 2024-05-10T11:14:30.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:30.015+0000 7fdc8db19700 1 -- 172.21.15.100:0/504252880 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc881a8250 con 0x7fdc88103530 2024-05-10T11:14:30.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:30.015+0000 7fdc857fa700 1 -- 172.21.15.100:0/504252880 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdc78004830 con 0x7fdc88103530 2024-05-10T11:14:30.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:30.015+0000 7fdc857fa700 1 -- 172.21.15.100:0/504252880 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc780054b0 con 0x7fdc88103530 2024-05-10T11:14:30.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:30.015+0000 7fdc8db19700 1 -- 172.21.15.100:0/504252880 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc881a86a0 con 0x7fdc88103530 2024-05-10T11:14:30.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:30.016+0000 7fdc8db19700 1 -- 172.21.15.100:0/504252880 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc8810ba60 con 0x7fdc88103530 2024-05-10T11:14:30.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:30.017+0000 7fdc857fa700 1 -- 172.21.15.100:0/504252880 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fdc78029060 con 0x7fdc88103530 2024-05-10T11:14:30.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:30.018+0000 7fdc857fa700 1 --2- 172.21.15.100:0/504252880 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fdc740b2880 0x7fdc740b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:30.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:30.019+0000 7fdc7ffff700 1 --2- 172.21.15.100:0/504252880 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fdc740b2880 0x7fdc740b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:30.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:30.019+0000 7fdc857fa700 1 -- 172.21.15.100:0/504252880 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fdc780e2eb0 con 0x7fdc88103530 2024-05-10T11:14:30.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:30.019+0000 7fdc7ffff700 1 --2- 172.21.15.100:0/504252880 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fdc740b2880 0x7fdc740b4d40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdc70005fd0 tx=0x7fdc7000a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:14:30.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:30.022+0000 7fdc857fa700 1 -- 172.21.15.100:0/504252880 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fdc78013070 con 0x7fdc88103530 2024-05-10T11:14:30.028 INFO:journalctl@ceph.mds.i.smithi188.stdout:May 10 11:14:29 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-i[64554]: 2024-05-10T11:14:29.740+0000 7fd14fc2e700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-10T11:14:30.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:30.237+0000 7fdc8db19700 1 -- 172.21.15.100:0/504252880 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7fdc8802e510 con 0x7fdc88103530 2024-05-10T11:14:30.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:30.297+0000 7fdc857fa700 1 -- 172.21.15.100:0/504252880 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v81) ==== 157+0+0 (secure 0 0 0) 0x7fdc780aad10 con 0x7fdc88103530 2024-05-10T11:14:30.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:30.300+0000 7fdc8db19700 1 -- 172.21.15.100:0/504252880 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fdc740b2880 msgr2=0x7fdc740b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:30.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:30.300+0000 7fdc8db19700 1 --2- 172.21.15.100:0/504252880 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fdc740b2880 0x7fdc740b4d40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdc70005fd0 tx=0x7fdc7000a560 comp rx=0 tx=0).stop 2024-05-10T11:14:30.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:30.300+0000 7fdc8db19700 1 -- 172.21.15.100:0/504252880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc88103530 msgr2=0x7fdc880757a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:30.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:30.300+0000 7fdc8db19700 1 --2- 172.21.15.100:0/504252880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc88103530 0x7fdc880757a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdc78000c00 tx=0x7fdc7800f510 comp rx=0 tx=0).stop 2024-05-10T11:14:30.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:30.300+0000 7fdc8db19700 1 -- 172.21.15.100:0/504252880 shutdown_connections 2024-05-10T11:14:30.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:30.300+0000 7fdc8db19700 1 --2- 172.21.15.100:0/504252880 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fdc740b2880 0x7fdc740b4d40 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:30.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:30.300+0000 7fdc8db19700 1 --2- 172.21.15.100:0/504252880 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc881095d0 0x7fdc88076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:30.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:30.300+0000 7fdc8db19700 1 --2- 172.21.15.100:0/504252880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc88103530 0x7fdc880757a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:30.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:30.300+0000 7fdc8db19700 1 --2- 172.21.15.100:0/504252880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc88103ef0 0x7fdc88075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:30.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:30.300+0000 7fdc8db19700 1 -- 172.21.15.100:0/504252880 >> 172.21.15.100:0/504252880 conn(0x7fdc8807a690 msgr2=0x7fdc88102230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:14:30.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:30.300+0000 7fdc8db19700 1 -- 172.21.15.100:0/504252880 shutdown_connections 2024-05-10T11:14:30.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:30.300+0000 7fdc8db19700 1 -- 172.21.15.100:0/504252880 wait complete. 2024-05-10T11:14:30.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:30 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/504252880' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-10T11:14:30.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:30 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/504252880' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-10T11:14:30.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:30 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/504252880' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-10T11:14:30.958 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-05-10T11:14:31.201 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:14:31.616 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:31 smithi100 ceph-mon[30296]: pgmap v295: 97 pgs: 97 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 4 op/s 2024-05-10T11:14:31.616 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:31 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/504252880' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-10T11:14:31.616 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:31 smithi100 ceph-mon[30296]: osdmap e81: 12 total, 12 up, 12 in 2024-05-10T11:14:31.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:31 smithi178 ceph-mon[33359]: pgmap v295: 97 pgs: 97 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 4 op/s 2024-05-10T11:14:31.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:31 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/504252880' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-10T11:14:31.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:31 smithi178 ceph-mon[33359]: osdmap e81: 12 total, 12 up, 12 in 2024-05-10T11:14:31.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:31 smithi188 ceph-mon[34075]: pgmap v295: 97 pgs: 97 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 4 op/s 2024-05-10T11:14:31.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:31 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/504252880' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-10T11:14:31.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:31 smithi188 ceph-mon[34075]: osdmap e81: 12 total, 12 up, 12 in 2024-05-10T11:14:33.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:33.061+0000 7f95f721f700 1 -- 172.21.15.100:0/2685291460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95f0102da0 msgr2=0x7f95f010f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:33.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:33.061+0000 7f95f721f700 1 --2- 172.21.15.100:0/2685291460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95f0102da0 0x7f95f010f240 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f95e800b0a0 tx=0x7f95e800b3b0 comp rx=0 tx=0).stop 2024-05-10T11:14:33.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:33.061+0000 7f95f721f700 1 -- 172.21.15.100:0/2685291460 shutdown_connections 2024-05-10T11:14:33.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:33.061+0000 7f95f721f700 1 --2- 172.21.15.100:0/2685291460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95f0102da0 0x7f95f010f240 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:33.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:33.061+0000 7f95f721f700 1 --2- 172.21.15.100:0/2685291460 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95f0108490 0x7f95f0108870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:33.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:33.061+0000 7f95f721f700 1 --2- 172.21.15.100:0/2685291460 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95f01023e0 0x7f95f0102860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:33.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:33.061+0000 7f95f721f700 1 -- 172.21.15.100:0/2685291460 >> 172.21.15.100:0/2685291460 conn(0x7f95f0079430 msgr2=0x7f95f0079840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:14:33.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:33.061+0000 7f95f721f700 1 -- 172.21.15.100:0/2685291460 shutdown_connections 2024-05-10T11:14:33.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:33.061+0000 7f95f721f700 1 -- 172.21.15.100:0/2685291460 wait complete. 2024-05-10T11:14:33.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:33.062+0000 7f95f721f700 1 Processor -- start 2024-05-10T11:14:33.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:33.062+0000 7f95f721f700 1 -- start start 2024-05-10T11:14:33.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:33.063+0000 7f95f721f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95f01023e0 0x7f95f0104090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:33.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:33.063+0000 7f95f721f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95f0102da0 0x7f95f01045d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:33.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:33.063+0000 7f95f721f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95f0108490 0x7f95f0104b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:33.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:33.063+0000 7f95f721f700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f95f0112400 con 0x7f95f01023e0 2024-05-10T11:14:33.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:33.063+0000 7f95f721f700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f95f0112280 con 0x7f95f0102da0 2024-05-10T11:14:33.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:33.063+0000 7f95f721f700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f95f0112580 con 0x7f95f0108490 2024-05-10T11:14:33.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:33.063+0000 7f95f4fbb700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95f01023e0 0x7f95f0104090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:33.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:33.063+0000 7f95f4fbb700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95f01023e0 0x7f95f0104090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37048/0 (socket says 172.21.15.100:37048) 2024-05-10T11:14:33.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:33.063+0000 7f95f4fbb700 1 -- 172.21.15.100:0/1141675165 learned_addr learned my addr 172.21.15.100:0/1141675165 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:14:33.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:33.063+0000 7f95f57bc700 1 --2- 172.21.15.100:0/1141675165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95f0108490 0x7f95f0104b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:33.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:33.063+0000 7f95effff700 1 --2- 172.21.15.100:0/1141675165 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95f0102da0 0x7f95f01045d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:33.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:33.063+0000 7f95f4fbb700 1 -- 172.21.15.100:0/1141675165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95f0108490 msgr2=0x7f95f0104b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:33.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:33.063+0000 7f95f4fbb700 1 --2- 172.21.15.100:0/1141675165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95f0108490 0x7f95f0104b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:33.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:33.063+0000 7f95f4fbb700 1 -- 172.21.15.100:0/1141675165 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95f0102da0 msgr2=0x7f95f01045d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:33.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:33.063+0000 7f95f4fbb700 1 --2- 172.21.15.100:0/1141675165 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95f0102da0 0x7f95f01045d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:33.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:33.063+0000 7f95f4fbb700 1 -- 172.21.15.100:0/1141675165 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95f01ac1d0 con 0x7f95f01023e0 2024-05-10T11:14:33.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:33.063+0000 7f95f4fbb700 1 --2- 172.21.15.100:0/1141675165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95f01023e0 0x7f95f0104090 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f95e400c9b0 tx=0x7f95e400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:14:33.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:33.063+0000 7f95edffb700 1 -- 172.21.15.100:0/1141675165 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e4007b00 con 0x7f95f01023e0 2024-05-10T11:14:33.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:33.063+0000 7f95edffb700 1 -- 172.21.15.100:0/1141675165 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f95e4007ca0 con 0x7f95f01023e0 2024-05-10T11:14:33.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:33.063+0000 7f95f721f700 1 -- 172.21.15.100:0/1141675165 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95f01ac400 con 0x7f95f01023e0 2024-05-10T11:14:33.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:33.063+0000 7f95edffb700 1 -- 172.21.15.100:0/1141675165 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e4018660 con 0x7f95f01023e0 2024-05-10T11:14:33.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:33.063+0000 7f95f721f700 1 -- 172.21.15.100:0/1141675165 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95f01ac990 con 0x7f95f01023e0 2024-05-10T11:14:33.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:33.064+0000 7f95f721f700 1 -- 172.21.15.100:0/1141675165 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95f0103ad0 con 0x7f95f01023e0 2024-05-10T11:14:33.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:33.065+0000 7f95edffb700 1 -- 172.21.15.100:0/1141675165 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f95e400f450 con 0x7f95f01023e0 2024-05-10T11:14:33.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:33.066+0000 7f95edffb700 1 --2- 172.21.15.100:0/1141675165 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f95e00b29b0 0x7f95e00b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:33.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:33.067+0000 7f95effff700 1 --2- 172.21.15.100:0/1141675165 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f95e00b29b0 0x7f95e00b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:33.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:33.067+0000 7f95edffb700 1 -- 172.21.15.100:0/1141675165 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8277+0+0 (secure 0 0 0) 0x7f95e40e2180 con 0x7f95f01023e0 2024-05-10T11:14:33.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:33.067+0000 7f95effff700 1 --2- 172.21.15.100:0/1141675165 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f95e00b29b0 0x7f95e00b4e70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f95dc005950 tx=0x7f95dc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:14:33.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:33.071+0000 7f95edffb700 1 -- 172.21.15.100:0/1141675165 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f95e4014030 con 0x7f95f01023e0 2024-05-10T11:14:33.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:33.278+0000 7f95f721f700 1 -- 172.21.15.100:0/1141675165 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f95f01accb0 con 0x7f95f01023e0 2024-05-10T11:14:33.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:33 smithi100 ceph-mon[30296]: pgmap v297: 97 pgs: 97 active+clean; 579 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-05-10T11:14:33.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:33 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/1141675165' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-10T11:14:33.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:33 smithi178 ceph-mon[33359]: pgmap v297: 97 pgs: 97 active+clean; 579 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-05-10T11:14:33.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:33 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/1141675165' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-10T11:14:33.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:33 smithi188 ceph-mon[34075]: pgmap v297: 97 pgs: 97 active+clean; 579 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-05-10T11:14:33.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:33 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/1141675165' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-10T11:14:34.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:34.303+0000 7f95edffb700 1 -- 172.21.15.100:0/1141675165 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v83) ==== 228+0+0 (secure 0 0 0) 0x7f95e40a9e70 con 0x7f95f01023e0 2024-05-10T11:14:34.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:34.308+0000 7f95f721f700 1 -- 172.21.15.100:0/1141675165 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f95e00b29b0 msgr2=0x7f95e00b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:34.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:34.308+0000 7f95f721f700 1 --2- 172.21.15.100:0/1141675165 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f95e00b29b0 0x7f95e00b4e70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f95dc005950 tx=0x7f95dc0058e0 comp rx=0 tx=0).stop 2024-05-10T11:14:34.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:34.308+0000 7f95f721f700 1 -- 172.21.15.100:0/1141675165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95f01023e0 msgr2=0x7f95f0104090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:34.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:34.308+0000 7f95f721f700 1 --2- 172.21.15.100:0/1141675165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95f01023e0 0x7f95f0104090 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f95e400c9b0 tx=0x7f95e400ccc0 comp rx=0 tx=0).stop 2024-05-10T11:14:34.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:34.308+0000 7f95f721f700 1 -- 172.21.15.100:0/1141675165 shutdown_connections 2024-05-10T11:14:34.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:34.308+0000 7f95f721f700 1 --2- 172.21.15.100:0/1141675165 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f95e00b29b0 0x7f95e00b4e70 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:34.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:34.308+0000 7f95f721f700 1 --2- 172.21.15.100:0/1141675165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95f0108490 0x7f95f0104b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:34.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:34.308+0000 7f95f721f700 1 --2- 172.21.15.100:0/1141675165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95f01023e0 0x7f95f0104090 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:34.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:34.308+0000 7f95f721f700 1 --2- 172.21.15.100:0/1141675165 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95f0102da0 0x7f95f01045d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:34.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:34.308+0000 7f95f721f700 1 -- 172.21.15.100:0/1141675165 >> 172.21.15.100:0/1141675165 conn(0x7f95f0079430 msgr2=0x7f95f01016e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:14:34.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:34.308+0000 7f95f721f700 1 -- 172.21.15.100:0/1141675165 shutdown_connections 2024-05-10T11:14:34.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:34.308+0000 7f95f721f700 1 -- 172.21.15.100:0/1141675165 wait complete. 2024-05-10T11:14:34.311 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_data_ec' created 2024-05-10T11:14:34.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:34 smithi100 ceph-mon[30296]: osdmap e82: 12 total, 12 up, 12 in 2024-05-10T11:14:34.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:34 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/1141675165' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-10T11:14:34.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:34 smithi178 ceph-mon[33359]: osdmap e82: 12 total, 12 up, 12 in 2024-05-10T11:14:34.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:34 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/1141675165' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-10T11:14:34.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:34 smithi188 ceph-mon[34075]: osdmap e82: 12 total, 12 up, 12 in 2024-05-10T11:14:34.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:34 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/1141675165' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-10T11:14:35.112 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-05-10T11:14:35.371 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:14:35.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:35 smithi100 ceph-mon[30296]: pgmap v299: 97 pgs: 97 active+clean; 579 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-05-10T11:14:35.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:35 smithi100 ceph-mon[30296]: osdmap e83: 12 total, 12 up, 12 in 2024-05-10T11:14:35.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:35 smithi100 ceph-mon[30296]: osdmap e84: 12 total, 12 up, 12 in 2024-05-10T11:14:35.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:35 smithi178 ceph-mon[33359]: pgmap v299: 97 pgs: 97 active+clean; 579 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-05-10T11:14:35.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:35 smithi178 ceph-mon[33359]: osdmap e83: 12 total, 12 up, 12 in 2024-05-10T11:14:35.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:35 smithi178 ceph-mon[33359]: osdmap e84: 12 total, 12 up, 12 in 2024-05-10T11:14:35.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:35 smithi188 ceph-mon[34075]: pgmap v299: 97 pgs: 97 active+clean; 579 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-05-10T11:14:35.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:35 smithi188 ceph-mon[34075]: osdmap e83: 12 total, 12 up, 12 in 2024-05-10T11:14:35.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:35 smithi188 ceph-mon[34075]: osdmap e84: 12 total, 12 up, 12 in 2024-05-10T11:14:37.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:37.156+0000 7efe5bb14700 1 -- 172.21.15.100:0/2791885723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe54102750 msgr2=0x7efe541067a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:37.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:37.156+0000 7efe5bb14700 1 --2- 172.21.15.100:0/2791885723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe54102750 0x7efe541067a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7efe5000aad0 tx=0x7efe5000ade0 comp rx=0 tx=0).stop 2024-05-10T11:14:37.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:37.156+0000 7efe5bb14700 1 -- 172.21.15.100:0/2791885723 shutdown_connections 2024-05-10T11:14:37.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:37.156+0000 7efe5bb14700 1 --2- 172.21.15.100:0/2791885723 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe54106e80 0x7efe5410ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:37.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:37.156+0000 7efe5bb14700 1 --2- 172.21.15.100:0/2791885723 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe54101da0 0x7efe54102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:37.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:37.156+0000 7efe5bb14700 1 --2- 172.21.15.100:0/2791885723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe54102750 0x7efe541067a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:37.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:37.156+0000 7efe5bb14700 1 -- 172.21.15.100:0/2791885723 >> 172.21.15.100:0/2791885723 conn(0x7efe540789a0 msgr2=0x7efe540ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:14:37.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:37.157+0000 7efe5bb14700 1 -- 172.21.15.100:0/2791885723 shutdown_connections 2024-05-10T11:14:37.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:37.157+0000 7efe5bb14700 1 -- 172.21.15.100:0/2791885723 wait complete. 2024-05-10T11:14:37.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:37.157+0000 7efe5bb14700 1 Processor -- start 2024-05-10T11:14:37.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:37.157+0000 7efe5bb14700 1 -- start start 2024-05-10T11:14:37.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:37.157+0000 7efe5bb14700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe54101da0 0x7efe5419f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:37.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:37.157+0000 7efe5bb14700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe54102750 0x7efe5419fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:37.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:37.157+0000 7efe5bb14700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe54106e80 0x7efe54199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:37.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:37.157+0000 7efe5bb14700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7efe541113a0 con 0x7efe54102750 2024-05-10T11:14:37.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:37.157+0000 7efe5bb14700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efe54111220 con 0x7efe54101da0 2024-05-10T11:14:37.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:37.157+0000 7efe5bb14700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7efe54111520 con 0x7efe54106e80 2024-05-10T11:14:37.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:37.157+0000 7efe590af700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe54102750 0x7efe5419fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:37.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:37.157+0000 7efe590af700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe54102750 0x7efe5419fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50774/0 (socket says 172.21.15.100:50774) 2024-05-10T11:14:37.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:37.157+0000 7efe590af700 1 -- 172.21.15.100:0/2607658718 learned_addr learned my addr 172.21.15.100:0/2607658718 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:14:37.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:37.158+0000 7efe598b0700 1 --2- 172.21.15.100:0/2607658718 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe54101da0 0x7efe5419f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:37.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:37.158+0000 7efe5a0b1700 1 --2- 172.21.15.100:0/2607658718 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe54106e80 0x7efe54199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:37.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:37.158+0000 7efe590af700 1 -- 172.21.15.100:0/2607658718 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe54106e80 msgr2=0x7efe54199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:37.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:37.158+0000 7efe590af700 1 --2- 172.21.15.100:0/2607658718 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe54106e80 0x7efe54199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:37.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:37.158+0000 7efe590af700 1 -- 172.21.15.100:0/2607658718 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe54101da0 msgr2=0x7efe5419f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:37.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:37.158+0000 7efe590af700 1 --2- 172.21.15.100:0/2607658718 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe54101da0 0x7efe5419f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:37.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:37.158+0000 7efe590af700 1 -- 172.21.15.100:0/2607658718 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe5419a0a0 con 0x7efe54102750 2024-05-10T11:14:37.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:37.158+0000 7efe590af700 1 --2- 172.21.15.100:0/2607658718 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe54102750 0x7efe5419fcc0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7efe50006010 tx=0x7efe50003e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:14:37.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:37.158+0000 7efe46ffd700 1 -- 172.21.15.100:0/2607658718 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe50023020 con 0x7efe54102750 2024-05-10T11:14:37.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:37.158+0000 7efe5bb14700 1 -- 172.21.15.100:0/2607658718 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe5419a330 con 0x7efe54102750 2024-05-10T11:14:37.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:37.158+0000 7efe46ffd700 1 -- 172.21.15.100:0/2607658718 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efe5000ce80 con 0x7efe54102750 2024-05-10T11:14:37.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:37.158+0000 7efe46ffd700 1 -- 172.21.15.100:0/2607658718 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5000ee00 con 0x7efe54102750 2024-05-10T11:14:37.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:37.158+0000 7efe5bb14700 1 -- 172.21.15.100:0/2607658718 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe541a6650 con 0x7efe54102750 2024-05-10T11:14:37.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:37.159+0000 7efe5bb14700 1 -- 172.21.15.100:0/2607658718 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe5406a190 con 0x7efe54102750 2024-05-10T11:14:37.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:37.161+0000 7efe46ffd700 1 -- 172.21.15.100:0/2607658718 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7efe50026020 con 0x7efe54102750 2024-05-10T11:14:37.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:37.161+0000 7efe46ffd700 1 --2- 172.21.15.100:0/2607658718 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7efe400b2880 0x7efe400b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:37.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:37.161+0000 7efe598b0700 1 --2- 172.21.15.100:0/2607658718 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7efe400b2880 0x7efe400b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:37.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:37.161+0000 7efe46ffd700 1 -- 172.21.15.100:0/2607658718 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8770+0+0 (secure 0 0 0) 0x7efe500e3a50 con 0x7efe54102750 2024-05-10T11:14:37.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:37.162+0000 7efe598b0700 1 --2- 172.21.15.100:0/2607658718 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7efe400b2880 0x7efe400b4d40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7efe5419aa80 tx=0x7efe4800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:14:37.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:37.167+0000 7efe46ffd700 1 -- 172.21.15.100:0/2607658718 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7efe5002d020 con 0x7efe54102750 2024-05-10T11:14:37.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:37.367+0000 7efe5bb14700 1 -- 172.21.15.100:0/2607658718 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7efe5402e510 con 0x7efe54102750 2024-05-10T11:14:37.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:37 smithi100 ceph-mon[30296]: pgmap v302: 129 pgs: 8 creating+peering, 13 unknown, 108 active+clean; 579 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 696 B/s wr, 2 op/s 2024-05-10T11:14:37.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:37 smithi100 ceph-mon[30296]: osdmap e85: 12 total, 12 up, 12 in 2024-05-10T11:14:37.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:37 smithi100 ceph-mon[30296]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-10T11:14:37.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:37 smithi178 ceph-mon[33359]: pgmap v302: 129 pgs: 8 creating+peering, 13 unknown, 108 active+clean; 579 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 696 B/s wr, 2 op/s 2024-05-10T11:14:37.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:37 smithi178 ceph-mon[33359]: osdmap e85: 12 total, 12 up, 12 in 2024-05-10T11:14:37.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:37 smithi178 ceph-mon[33359]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-10T11:14:37.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:37 smithi188 ceph-mon[34075]: pgmap v302: 129 pgs: 8 creating+peering, 13 unknown, 108 active+clean; 579 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 696 B/s wr, 2 op/s 2024-05-10T11:14:37.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:37 smithi188 ceph-mon[34075]: osdmap e85: 12 total, 12 up, 12 in 2024-05-10T11:14:37.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:37 smithi188 ceph-mon[34075]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-10T11:14:38.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:38.313+0000 7efe46ffd700 1 -- 172.21.15.100:0/2607658718 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v86) ==== 169+0+0 (secure 0 0 0) 0x7efe500ab5e0 con 0x7efe54102750 2024-05-10T11:14:38.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:38.318+0000 7efe5bb14700 1 -- 172.21.15.100:0/2607658718 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7efe400b2880 msgr2=0x7efe400b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:38.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:38.318+0000 7efe5bb14700 1 --2- 172.21.15.100:0/2607658718 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7efe400b2880 0x7efe400b4d40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7efe5419aa80 tx=0x7efe4800b410 comp rx=0 tx=0).stop 2024-05-10T11:14:38.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:38.318+0000 7efe5bb14700 1 -- 172.21.15.100:0/2607658718 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe54102750 msgr2=0x7efe5419fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:38.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:38.318+0000 7efe5bb14700 1 --2- 172.21.15.100:0/2607658718 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe54102750 0x7efe5419fcc0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7efe50006010 tx=0x7efe50003e80 comp rx=0 tx=0).stop 2024-05-10T11:14:38.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:38.318+0000 7efe5bb14700 1 -- 172.21.15.100:0/2607658718 shutdown_connections 2024-05-10T11:14:38.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:38.318+0000 7efe5bb14700 1 --2- 172.21.15.100:0/2607658718 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7efe400b2880 0x7efe400b4d40 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:38.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:38.318+0000 7efe5bb14700 1 --2- 172.21.15.100:0/2607658718 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe54106e80 0x7efe54199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:38.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:38.318+0000 7efe5bb14700 1 --2- 172.21.15.100:0/2607658718 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe54101da0 0x7efe5419f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:38.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:38.318+0000 7efe5bb14700 1 --2- 172.21.15.100:0/2607658718 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe54102750 0x7efe5419fcc0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:38.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:38.318+0000 7efe5bb14700 1 -- 172.21.15.100:0/2607658718 >> 172.21.15.100:0/2607658718 conn(0x7efe540789a0 msgr2=0x7efe54105620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:14:38.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:38.318+0000 7efe5bb14700 1 -- 172.21.15.100:0/2607658718 shutdown_connections 2024-05-10T11:14:38.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:38.318+0000 7efe5bb14700 1 -- 172.21.15.100:0/2607658718 wait complete. 2024-05-10T11:14:38.319 INFO:teuthology.orchestra.run.smithi100.stderr:set pool 4 allow_ec_overwrites to true 2024-05-10T11:14:38.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:38 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/2607658718' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-10T11:14:38.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:38 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/2607658718' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-10T11:14:38.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:38 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/2607658718' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-10T11:14:39.073 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-05-10T11:14:39.318 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:14:39.567 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:39 smithi100 ceph-mon[30296]: pgmap v304: 129 pgs: 8 creating+peering, 7 unknown, 114 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:39.567 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:39 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/2607658718' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-10T11:14:39.567 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:39 smithi100 ceph-mon[30296]: osdmap e86: 12 total, 12 up, 12 in 2024-05-10T11:14:39.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:39 smithi178 ceph-mon[33359]: pgmap v304: 129 pgs: 8 creating+peering, 7 unknown, 114 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:39.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:39 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/2607658718' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-10T11:14:39.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:39 smithi178 ceph-mon[33359]: osdmap e86: 12 total, 12 up, 12 in 2024-05-10T11:14:39.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:39 smithi188 ceph-mon[34075]: pgmap v304: 129 pgs: 8 creating+peering, 7 unknown, 114 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:39.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:39 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/2607658718' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-10T11:14:39.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:39 smithi188 ceph-mon[34075]: osdmap e86: 12 total, 12 up, 12 in 2024-05-10T11:14:41.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:41.118+0000 7f25536c9700 1 -- 172.21.15.100:0/1886929746 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f254c102d50 msgr2=0x7f254c10f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:41.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:41.118+0000 7f25536c9700 1 --2- 172.21.15.100:0/1886929746 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f254c102d50 0x7f254c10f240 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f254400b0a0 tx=0x7f254400b3b0 comp rx=0 tx=0).stop 2024-05-10T11:14:41.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:41.118+0000 7f25536c9700 1 -- 172.21.15.100:0/1886929746 shutdown_connections 2024-05-10T11:14:41.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:41.118+0000 7f25536c9700 1 --2- 172.21.15.100:0/1886929746 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f254c102d50 0x7f254c10f240 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:41.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:41.118+0000 7f25536c9700 1 --2- 172.21.15.100:0/1886929746 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f254c108400 0x7f254c1087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:41.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:41.118+0000 7f25536c9700 1 --2- 172.21.15.100:0/1886929746 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f254c102390 0x7f254c102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:41.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:41.118+0000 7f25536c9700 1 -- 172.21.15.100:0/1886929746 >> 172.21.15.100:0/1886929746 conn(0x7f254c079310 msgr2=0x7f254c079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:14:41.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:41.118+0000 7f25536c9700 1 -- 172.21.15.100:0/1886929746 shutdown_connections 2024-05-10T11:14:41.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:41.118+0000 7f25536c9700 1 -- 172.21.15.100:0/1886929746 wait complete. 2024-05-10T11:14:41.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:41.119+0000 7f25536c9700 1 Processor -- start 2024-05-10T11:14:41.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:41.119+0000 7f25536c9700 1 -- start start 2024-05-10T11:14:41.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:41.119+0000 7f25536c9700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f254c102390 0x7f254c19aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:41.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:41.119+0000 7f25536c9700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f254c102d50 0x7f254c19afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:41.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:41.119+0000 7f25536c9700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f254c108400 0x7f254c1a5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:41.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:41.119+0000 7f25536c9700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f254c1124d0 con 0x7f254c102390 2024-05-10T11:14:41.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:41.119+0000 7f25536c9700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f254c112350 con 0x7f254c102d50 2024-05-10T11:14:41.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:41.119+0000 7f25536c9700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f254c112650 con 0x7f254c108400 2024-05-10T11:14:41.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:41.119+0000 7f2551465700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f254c102390 0x7f254c19aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:41.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:41.120+0000 7f2551465700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f254c102390 0x7f254c19aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50804/0 (socket says 172.21.15.100:50804) 2024-05-10T11:14:41.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:41.120+0000 7f2551465700 1 -- 172.21.15.100:0/2425945684 learned_addr learned my addr 172.21.15.100:0/2425945684 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:14:41.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:41.120+0000 7f2551c66700 1 --2- 172.21.15.100:0/2425945684 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f254c108400 0x7f254c1a5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:41.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:41.120+0000 7f2550c64700 1 --2- 172.21.15.100:0/2425945684 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f254c102d50 0x7f254c19afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:41.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:41.120+0000 7f2551465700 1 -- 172.21.15.100:0/2425945684 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f254c108400 msgr2=0x7f254c1a5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:41.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:41.120+0000 7f2551465700 1 --2- 172.21.15.100:0/2425945684 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f254c108400 0x7f254c1a5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:41.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:41.120+0000 7f2551465700 1 -- 172.21.15.100:0/2425945684 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f254c102d50 msgr2=0x7f254c19afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:41.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:41.120+0000 7f2551465700 1 --2- 172.21.15.100:0/2425945684 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f254c102d50 0x7f254c19afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:41.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:41.120+0000 7f2551465700 1 -- 172.21.15.100:0/2425945684 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f254c1a6030 con 0x7f254c102390 2024-05-10T11:14:41.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:41.120+0000 7f2551c66700 1 --2- 172.21.15.100:0/2425945684 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f254c108400 0x7f254c1a5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:14:41.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:41.120+0000 7f2551465700 1 --2- 172.21.15.100:0/2425945684 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f254c102390 0x7f254c19aa70 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f253c00b530 tx=0x7f253c00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:14:41.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:41.120+0000 7f25427fc700 1 -- 172.21.15.100:0/2425945684 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253c015020 con 0x7f254c102390 2024-05-10T11:14:41.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:41.120+0000 7f25427fc700 1 -- 172.21.15.100:0/2425945684 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f253c011cc0 con 0x7f254c102390 2024-05-10T11:14:41.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:41.120+0000 7f25536c9700 1 -- 172.21.15.100:0/2425945684 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f254c1a6260 con 0x7f254c102390 2024-05-10T11:14:41.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:41.120+0000 7f25427fc700 1 -- 172.21.15.100:0/2425945684 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253c010d30 con 0x7f254c102390 2024-05-10T11:14:41.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:41.120+0000 7f25536c9700 1 -- 172.21.15.100:0/2425945684 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f254c1a6770 con 0x7f254c102390 2024-05-10T11:14:41.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:41.121+0000 7f25536c9700 1 -- 172.21.15.100:0/2425945684 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f254c10a970 con 0x7f254c102390 2024-05-10T11:14:41.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:41.123+0000 7f25427fc700 1 -- 172.21.15.100:0/2425945684 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f253c018020 con 0x7f254c102390 2024-05-10T11:14:41.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:41.123+0000 7f25427fc700 1 --2- 172.21.15.100:0/2425945684 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f25380b2880 0x7f25380b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:41.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:41.124+0000 7f2550c64700 1 --2- 172.21.15.100:0/2425945684 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f25380b2880 0x7f25380b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:41.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:41.124+0000 7f25427fc700 1 -- 172.21.15.100:0/2425945684 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8770+0+0 (secure 0 0 0) 0x7f253c0e21d0 con 0x7f254c102390 2024-05-10T11:14:41.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:41.124+0000 7f2550c64700 1 --2- 172.21.15.100:0/2425945684 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f25380b2880 0x7f25380b4d40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2548006fd0 tx=0x7f2548008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:14:41.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:41.127+0000 7f25427fc700 1 -- 172.21.15.100:0/2425945684 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f253c01f020 con 0x7f254c102390 2024-05-10T11:14:41.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:41.398+0000 7f25536c9700 1 -- 172.21.15.100:0/2425945684 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f254c06c000 con 0x7f254c102390 2024-05-10T11:14:41.398 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:41 smithi100 ceph-mon[30296]: pgmap v306: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:41.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:41 smithi178 ceph-mon[33359]: pgmap v306: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:41.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:41 smithi188 ceph-mon[34075]: pgmap v306: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:42.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:42.324+0000 7f25427fc700 1 -- 172.21.15.100:0/2425945684 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f253c0a9d70 con 0x7f254c102390 2024-05-10T11:14:42.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:42.327+0000 7f25536c9700 1 -- 172.21.15.100:0/2425945684 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f25380b2880 msgr2=0x7f25380b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:42.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:42.327+0000 7f25536c9700 1 --2- 172.21.15.100:0/2425945684 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f25380b2880 0x7f25380b4d40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2548006fd0 tx=0x7f2548008040 comp rx=0 tx=0).stop 2024-05-10T11:14:42.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:42.327+0000 7f25536c9700 1 -- 172.21.15.100:0/2425945684 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f254c102390 msgr2=0x7f254c19aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:42.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:42.327+0000 7f25536c9700 1 --2- 172.21.15.100:0/2425945684 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f254c102390 0x7f254c19aa70 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f253c00b530 tx=0x7f253c00b8f0 comp rx=0 tx=0).stop 2024-05-10T11:14:42.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:42.327+0000 7f25536c9700 1 -- 172.21.15.100:0/2425945684 shutdown_connections 2024-05-10T11:14:42.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:42.327+0000 7f25536c9700 1 --2- 172.21.15.100:0/2425945684 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f25380b2880 0x7f25380b4d40 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:42.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:42.327+0000 7f25536c9700 1 --2- 172.21.15.100:0/2425945684 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f254c108400 0x7f254c1a5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:42.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:42.327+0000 7f25536c9700 1 --2- 172.21.15.100:0/2425945684 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f254c102390 0x7f254c19aa70 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:42.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:42.327+0000 7f25536c9700 1 --2- 172.21.15.100:0/2425945684 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f254c102d50 0x7f254c19afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:42.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:42.327+0000 7f25536c9700 1 -- 172.21.15.100:0/2425945684 >> 172.21.15.100:0/2425945684 conn(0x7f254c079310 msgr2=0x7f254c1010e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:14:42.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:42.327+0000 7f25536c9700 1 -- 172.21.15.100:0/2425945684 shutdown_connections 2024-05-10T11:14:42.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:42.327+0000 7f25536c9700 1 -- 172.21.15.100:0/2425945684 wait complete. 2024-05-10T11:14:42.328 INFO:teuthology.orchestra.run.smithi100.stderr:added data pool 4 to fsmap 2024-05-10T11:14:42.674 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:42 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/2425945684' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-10T11:14:42.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:42 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/2425945684' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-10T11:14:42.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:42 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/2425945684' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-10T11:14:43.131 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T11:14:43.381 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:14:43.503 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:43 smithi100 ceph-mon[30296]: pgmap v307: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:43.503 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:43 smithi100 ceph-mon[30296]: osdmap e87: 12 total, 12 up, 12 in 2024-05-10T11:14:43.504 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:43 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/2425945684' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-10T11:14:43.504 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:43 smithi100 ceph-mon[30296]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-10T11:14:43.680 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:43 smithi188 ceph-mon[34075]: pgmap v307: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:43.680 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:43 smithi188 ceph-mon[34075]: osdmap e87: 12 total, 12 up, 12 in 2024-05-10T11:14:43.680 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:43 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/2425945684' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-10T11:14:43.680 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:43 smithi188 ceph-mon[34075]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-10T11:14:43.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:43 smithi178 ceph-mon[33359]: pgmap v307: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:43.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:43 smithi178 ceph-mon[33359]: osdmap e87: 12 total, 12 up, 12 in 2024-05-10T11:14:43.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:43 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/2425945684' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-10T11:14:43.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:43 smithi178 ceph-mon[33359]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-10T11:14:44.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:44 smithi100 ceph-mon[30296]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-10T11:14:44.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:44 smithi100 ceph-mon[30296]: Cluster is now healthy 2024-05-10T11:14:44.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:44 smithi178 ceph-mon[33359]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-10T11:14:44.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:44 smithi178 ceph-mon[33359]: Cluster is now healthy 2024-05-10T11:14:44.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:44 smithi188 ceph-mon[34075]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-10T11:14:44.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:44 smithi188 ceph-mon[34075]: Cluster is now healthy 2024-05-10T11:14:45.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:45.206+0000 7f9a05b56700 1 -- 172.21.15.100:0/2344425989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a00105ed0 msgr2=0x7f9a00106380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:45.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:45.206+0000 7f9a05b56700 1 --2- 172.21.15.100:0/2344425989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a00105ed0 0x7f9a00106380 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f99f800b0a0 tx=0x7f99f800b3b0 comp rx=0 tx=0).stop 2024-05-10T11:14:45.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:45.206+0000 7f9a05b56700 1 -- 172.21.15.100:0/2344425989 shutdown_connections 2024-05-10T11:14:45.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:45.206+0000 7f9a05b56700 1 --2- 172.21.15.100:0/2344425989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a00105ed0 0x7f9a00106380 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:45.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:45.206+0000 7f9a05b56700 1 --2- 172.21.15.100:0/2344425989 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a001083c0 0x7f9a0006bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:45.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:45.206+0000 7f9a05b56700 1 --2- 172.21.15.100:0/2344425989 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a0006c130 0x7f9a0006c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:45.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:45.206+0000 7f9a05b56700 1 -- 172.21.15.100:0/2344425989 >> 172.21.15.100:0/2344425989 conn(0x7f9a00079370 msgr2=0x7f9a00079780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:14:45.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:45.206+0000 7f9a05b56700 1 -- 172.21.15.100:0/2344425989 shutdown_connections 2024-05-10T11:14:45.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:45.206+0000 7f9a05b56700 1 -- 172.21.15.100:0/2344425989 wait complete. 2024-05-10T11:14:45.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:45.207+0000 7f9a05b56700 1 Processor -- start 2024-05-10T11:14:45.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:45.207+0000 7f9a05b56700 1 -- start start 2024-05-10T11:14:45.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:45.207+0000 7f9a05b56700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a0006c130 0x7f9a001a0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:45.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:45.207+0000 7f9a05b56700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a00105ed0 0x7f9a001a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:45.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:45.207+0000 7f9a05b56700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a001083c0 0x7f9a001a49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:45.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:45.207+0000 7f9a05b56700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9a00119bc0 con 0x7f9a0006c130 2024-05-10T11:14:45.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:45.207+0000 7f9a05b56700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9a00119a40 con 0x7f9a00105ed0 2024-05-10T11:14:45.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:45.207+0000 7f9a05b56700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9a00119d40 con 0x7f9a001083c0 2024-05-10T11:14:45.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:45.207+0000 7f99ff7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a0006c130 0x7f9a001a0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:45.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:45.207+0000 7f99ff7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a0006c130 0x7f9a001a0110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50832/0 (socket says 172.21.15.100:50832) 2024-05-10T11:14:45.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:45.207+0000 7f99ff7fe700 1 -- 172.21.15.100:0/3203008745 learned_addr learned my addr 172.21.15.100:0/3203008745 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:14:45.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:45.208+0000 7f99feffd700 1 --2- 172.21.15.100:0/3203008745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a00105ed0 0x7f9a001a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:45.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:45.208+0000 7f99fffff700 1 --2- 172.21.15.100:0/3203008745 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a001083c0 0x7f9a001a49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:45.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:45.208+0000 7f99ff7fe700 1 -- 172.21.15.100:0/3203008745 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a001083c0 msgr2=0x7f9a001a49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:45.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:45.208+0000 7f99ff7fe700 1 --2- 172.21.15.100:0/3203008745 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a001083c0 0x7f9a001a49e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:45.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:45.208+0000 7f99ff7fe700 1 -- 172.21.15.100:0/3203008745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a00105ed0 msgr2=0x7f9a001a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:45.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:45.208+0000 7f99ff7fe700 1 --2- 172.21.15.100:0/3203008745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a00105ed0 0x7f9a001a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:45.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:45.208+0000 7f99ff7fe700 1 -- 172.21.15.100:0/3203008745 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a001a5120 con 0x7f9a0006c130 2024-05-10T11:14:45.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:45.208+0000 7f99ff7fe700 1 --2- 172.21.15.100:0/3203008745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a0006c130 0x7f9a001a0110 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f99f400d950 tx=0x7f99f400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:14:45.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:45.208+0000 7f99fcff9700 1 -- 172.21.15.100:0/3203008745 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f4016070 con 0x7f9a0006c130 2024-05-10T11:14:45.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:45.208+0000 7f9a05b56700 1 -- 172.21.15.100:0/3203008745 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a001a53b0 con 0x7f9a0006c130 2024-05-10T11:14:45.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:45.208+0000 7f99fcff9700 1 -- 172.21.15.100:0/3203008745 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f99f4009dd0 con 0x7f9a0006c130 2024-05-10T11:14:45.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:45.208+0000 7f99fcff9700 1 -- 172.21.15.100:0/3203008745 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f400f3c0 con 0x7f9a0006c130 2024-05-10T11:14:45.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:45.208+0000 7f9a05b56700 1 -- 172.21.15.100:0/3203008745 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a001acc90 con 0x7f9a0006c130 2024-05-10T11:14:45.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:45.209+0000 7f9a05b56700 1 -- 172.21.15.100:0/3203008745 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a0006a190 con 0x7f9a0006c130 2024-05-10T11:14:45.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:45.211+0000 7f99fcff9700 1 -- 172.21.15.100:0/3203008745 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f99f4019070 con 0x7f9a0006c130 2024-05-10T11:14:45.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:45.212+0000 7f99fcff9700 1 --2- 172.21.15.100:0/3203008745 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f99e80b28c0 0x7f99e80b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:45.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:45.212+0000 7f99feffd700 1 --2- 172.21.15.100:0/3203008745 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f99e80b28c0 0x7f99e80b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:45.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:45.212+0000 7f99fcff9700 1 -- 172.21.15.100:0/3203008745 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f99f40e2460 con 0x7f9a0006c130 2024-05-10T11:14:45.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:45.212+0000 7f99feffd700 1 --2- 172.21.15.100:0/3203008745 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f99e80b28c0 0x7f99e80b4d80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9a00079c70 tx=0x7f99f0009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:14:45.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:45.215+0000 7f99fcff9700 1 -- 172.21.15.100:0/3203008745 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f99f40a9f40 con 0x7f9a0006c130 2024-05-10T11:14:45.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:45.471+0000 7f9a05b56700 1 -- 172.21.15.100:0/3203008745 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9a001ace60 con 0x7f9a0006c130 2024-05-10T11:14:45.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:45.471+0000 7f99fcff9700 1 -- 172.21.15.100:0/3203008745 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11821 (secure 0 0 0) 0x7f99f40a9840 con 0x7f9a0006c130 2024-05-10T11:14:45.475 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:14:45.475 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2072225012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2072225012},{"type":"v1","addr":"172.21.15.100:6835","nonce":2072225012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/401939540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":401939540},{"type":"v1","addr":"172.21.15.100:6837","nonce":401939540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14571,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/3946311025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3946311025},{"type":"v1","addr":"172.21.15.100:6839","nonce":3946311025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3736371660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3736371660},{"type":"v1","addr":"172.21.15.100:6841","nonce":3736371660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/2741810791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2741810791},{"type":"v1","addr":"172.21.15.178:6833","nonce":2741810791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1704891041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1704891041},{"type":"v1","addr":"172.21.15.188:6833","nonce":1704891041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24395,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2124076312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2124076312},{"type":"v1","addr":"172.21.15.178:6835","nonce":2124076312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24413,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/3224358425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3224358425},{"type":"v1","addr":"172.21.15.178:6837","nonce":3224358425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24421,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/515547690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":515547690},{"type":"v1","addr":"172.21.15.188:6835","nonce":515547690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24427,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1943342620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1943342620},{"type":"v1","addr":"172.21.15.188:6837","nonce":1943342620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24437,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6839/3760985404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3760985404},{"type":"v1","addr":"172.21.15.178:6839","nonce":3760985404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T11:14:27.242452+0000","modified":"2024-05-10T11:14:41.399890+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.188:6839/3194917070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":3194917070},{"type":"v1","addr":"172.21.15.188:6839","nonce":3194917070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24439,"qdb_cluster":[24439]},"id":1}]} 2024-05-10T11:14:45.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:45.474+0000 7f9a05b56700 1 -- 172.21.15.100:0/3203008745 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f99e80b28c0 msgr2=0x7f99e80b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:45.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:45.474+0000 7f9a05b56700 1 --2- 172.21.15.100:0/3203008745 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f99e80b28c0 0x7f99e80b4d80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9a00079c70 tx=0x7f99f0009190 comp rx=0 tx=0).stop 2024-05-10T11:14:45.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:45.474+0000 7f9a05b56700 1 -- 172.21.15.100:0/3203008745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a0006c130 msgr2=0x7f9a001a0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:45.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:45.474+0000 7f9a05b56700 1 --2- 172.21.15.100:0/3203008745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a0006c130 0x7f9a001a0110 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f99f400d950 tx=0x7f99f400dc60 comp rx=0 tx=0).stop 2024-05-10T11:14:45.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:45.474+0000 7f9a05b56700 1 -- 172.21.15.100:0/3203008745 shutdown_connections 2024-05-10T11:14:45.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:45.474+0000 7f9a05b56700 1 --2- 172.21.15.100:0/3203008745 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f99e80b28c0 0x7f99e80b4d80 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:45.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:45.474+0000 7f9a05b56700 1 --2- 172.21.15.100:0/3203008745 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a001083c0 0x7f9a001a49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:45.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:45.474+0000 7f9a05b56700 1 --2- 172.21.15.100:0/3203008745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a0006c130 0x7f9a001a0110 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:45.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:45.474+0000 7f9a05b56700 1 --2- 172.21.15.100:0/3203008745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a00105ed0 0x7f9a001a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:45.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:45.474+0000 7f9a05b56700 1 -- 172.21.15.100:0/3203008745 >> 172.21.15.100:0/3203008745 conn(0x7f9a00079370 msgr2=0x7f9a00104eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:14:45.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:45.474+0000 7f9a05b56700 1 -- 172.21.15.100:0/3203008745 shutdown_connections 2024-05-10T11:14:45.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:45.474+0000 7f9a05b56700 1 -- 172.21.15.100:0/3203008745 wait complete. 2024-05-10T11:14:45.477 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 17 2024-05-10T11:14:45.653 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:45 smithi178 ceph-mon[33359]: pgmap v309: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:45.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:45 smithi188 ceph-mon[34075]: pgmap v309: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:45.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:45 smithi100 ceph-mon[30296]: pgmap v309: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:46.105 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-10T11:14:46.348 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:14:46.730 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:46 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/3203008745' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:14:46.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:46 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/3203008745' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:14:46.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:46 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/3203008745' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:14:47.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:47 smithi100 ceph-mon[30296]: pgmap v310: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:47.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:47 smithi178 ceph-mon[33359]: pgmap v310: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:47.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:47 smithi188 ceph-mon[34075]: pgmap v310: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:48.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:48.133+0000 7f84637eb700 1 -- 172.21.15.100:0/3025334690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f845c106f60 msgr2=0x7f845c10eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:48.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:48.133+0000 7f84637eb700 1 --2- 172.21.15.100:0/3025334690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f845c106f60 0x7f845c10eaa0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f845400c210 tx=0x7f845400c520 comp rx=0 tx=0).stop 2024-05-10T11:14:48.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:48.133+0000 7f84637eb700 1 -- 172.21.15.100:0/3025334690 shutdown_connections 2024-05-10T11:14:48.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:48.133+0000 7f84637eb700 1 --2- 172.21.15.100:0/3025334690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f845c106f60 0x7f845c10eaa0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:48.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:48.133+0000 7f84637eb700 1 --2- 172.21.15.100:0/3025334690 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f845c1020c0 0x7f845c1024a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:48.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:48.133+0000 7f84637eb700 1 --2- 172.21.15.100:0/3025334690 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f845c102a70 0x7f845c106830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:48.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:48.133+0000 7f84637eb700 1 -- 172.21.15.100:0/3025334690 >> 172.21.15.100:0/3025334690 conn(0x7f845c0776c0 msgr2=0x7f845c077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:14:48.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:48.133+0000 7f84637eb700 1 -- 172.21.15.100:0/3025334690 shutdown_connections 2024-05-10T11:14:48.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:48.133+0000 7f84637eb700 1 -- 172.21.15.100:0/3025334690 wait complete. 2024-05-10T11:14:48.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:48.133+0000 7f84637eb700 1 Processor -- start 2024-05-10T11:14:48.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:48.133+0000 7f84637eb700 1 -- start start 2024-05-10T11:14:48.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:48.134+0000 7f84637eb700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f845c1020c0 0x7f845c19f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:48.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:48.134+0000 7f84637eb700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f845c102a70 0x7f845c19fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:48.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:48.134+0000 7f84637eb700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f845c106f60 0x7f845c199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:48.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:48.134+0000 7f84637eb700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f845c1113d0 con 0x7f845c106f60 2024-05-10T11:14:48.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:48.134+0000 7f84637eb700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f845c111250 con 0x7f845c102a70 2024-05-10T11:14:48.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:48.134+0000 7f84637eb700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f845c111550 con 0x7f845c1020c0 2024-05-10T11:14:48.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:48.134+0000 7f8461d88700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f845c106f60 0x7f845c199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:48.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:48.134+0000 7f8461d88700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f845c106f60 0x7f845c199860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37376/0 (socket says 172.21.15.100:37376) 2024-05-10T11:14:48.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:48.134+0000 7f8461587700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f845c1020c0 0x7f845c19f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:48.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:48.134+0000 7f8461d88700 1 -- 172.21.15.100:0/1924196276 learned_addr learned my addr 172.21.15.100:0/1924196276 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:14:48.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:48.134+0000 7f8460d86700 1 --2- 172.21.15.100:0/1924196276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f845c102a70 0x7f845c19fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:48.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:48.134+0000 7f8461d88700 1 -- 172.21.15.100:0/1924196276 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f845c1020c0 msgr2=0x7f845c19f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:48.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:48.134+0000 7f8461d88700 1 --2- 172.21.15.100:0/1924196276 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f845c1020c0 0x7f845c19f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:48.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:48.134+0000 7f8461d88700 1 -- 172.21.15.100:0/1924196276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f845c102a70 msgr2=0x7f845c19fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:48.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:48.134+0000 7f8461d88700 1 --2- 172.21.15.100:0/1924196276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f845c102a70 0x7f845c19fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:48.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:48.134+0000 7f8461d88700 1 -- 172.21.15.100:0/1924196276 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f845c199fc0 con 0x7f845c106f60 2024-05-10T11:14:48.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:48.134+0000 7f8461d88700 1 --2- 172.21.15.100:0/1924196276 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f845c106f60 0x7f845c199860 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f84540085c0 tx=0x7f845400a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:14:48.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:48.134+0000 7f84527fc700 1 -- 172.21.15.100:0/1924196276 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8454008c90 con 0x7f845c106f60 2024-05-10T11:14:48.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:48.134+0000 7f84527fc700 1 -- 172.21.15.100:0/1924196276 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8454008e30 con 0x7f845c106f60 2024-05-10T11:14:48.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:48.134+0000 7f84637eb700 1 -- 172.21.15.100:0/1924196276 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f845c19a250 con 0x7f845c106f60 2024-05-10T11:14:48.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:48.134+0000 7f84527fc700 1 -- 172.21.15.100:0/1924196276 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84540074a0 con 0x7f845c106f60 2024-05-10T11:14:48.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:48.135+0000 7f84637eb700 1 -- 172.21.15.100:0/1924196276 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f845c1a6570 con 0x7f845c106f60 2024-05-10T11:14:48.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:48.136+0000 7f84637eb700 1 -- 172.21.15.100:0/1924196276 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f845c06a190 con 0x7f845c106f60 2024-05-10T11:14:48.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:48.137+0000 7f84527fc700 1 -- 172.21.15.100:0/1924196276 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f84540050d0 con 0x7f845c106f60 2024-05-10T11:14:48.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:48.137+0000 7f84527fc700 1 --2- 172.21.15.100:0/1924196276 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f84480b2900 0x7f84480b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:48.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:48.138+0000 7f8461587700 1 --2- 172.21.15.100:0/1924196276 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f84480b2900 0x7f84480b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:48.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:48.138+0000 7f84527fc700 1 -- 172.21.15.100:0/1924196276 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f8454010020 con 0x7f845c106f60 2024-05-10T11:14:48.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:48.138+0000 7f8461587700 1 --2- 172.21.15.100:0/1924196276 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f84480b2900 0x7f84480b4dc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f845c19a9a0 tx=0x7f845800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:14:48.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:48.142+0000 7f84527fc700 1 -- 172.21.15.100:0/1924196276 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f845402c020 con 0x7f845c106f60 2024-05-10T11:14:48.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:48.356+0000 7f84637eb700 1 -- 172.21.15.100:0/1924196276 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f845c02e510 con 0x7f845c106f60 2024-05-10T11:14:48.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:48.358+0000 7f84527fc700 1 -- 172.21.15.100:0/1924196276 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24211 (secure 0 0 0) 0x7f84540ab070 con 0x7f845c106f60 2024-05-10T11:14:48.358 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:14:48.358 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":87,"fsid":"20a542cc-0ebd-11ef-bc98-c7b262605968","created":"2024-05-10T11:06:05.454446+0000","modified":"2024-05-10T11:14:41.399746+0000","last_up_change":"2024-05-10T11:13:18.185471+0000","last_in_change":"2024-05-10T11:12:57.268678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T11:09:26.176441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T11:14:21.004285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T11:14:24.179414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T11:14:33.308517+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5f031418-da23-4e85-8669-160960dadd09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":643028278},{"type":"v1","addr":"172.21.15.100:6803","nonce":643028278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":643028278},{"type":"v1","addr":"172.21.15.100:6805","nonce":643028278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":643028278},{"type":"v1","addr":"172.21.15.100:6809","nonce":643028278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":643028278},{"type":"v1","addr":"172.21.15.100:6807","nonce":643028278}]},"public_addr":"172.21.15.100:6803/643028278","cluster_addr":"172.21.15.100:6805/643028278","heartbeat_back_addr":"172.21.15.100:6809/643028278","heartbeat_front_addr":"172.21.15.100:6807/643028278","state":["exists","up"]},{"osd":1,"uuid":"a908cdc9-001e-4b65-9ab0-0dd08dba20f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":3103987017},{"type":"v1","addr":"172.21.15.178:6801","nonce":3103987017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":3103987017},{"type":"v1","addr":"172.21.15.178:6803","nonce":3103987017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":3103987017},{"type":"v1","addr":"172.21.15.178:6807","nonce":3103987017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":3103987017},{"type":"v1","addr":"172.21.15.178:6805","nonce":3103987017}]},"public_addr":"172.21.15.178:6801/3103987017","cluster_addr":"172.21.15.178:6803/3103987017","heartbeat_back_addr":"172.21.15.178:6807/3103987017","heartbeat_front_addr":"172.21.15.178:6805/3103987017","state":["exists","up"]},{"osd":2,"uuid":"4d8f89ae-9424-46e9-9c80-17c47cc5e8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3991466468},{"type":"v1","addr":"172.21.15.188:6801","nonce":3991466468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3991466468},{"type":"v1","addr":"172.21.15.188:6803","nonce":3991466468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3991466468},{"type":"v1","addr":"172.21.15.188:6807","nonce":3991466468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3991466468},{"type":"v1","addr":"172.21.15.188:6805","nonce":3991466468}]},"public_addr":"172.21.15.188:6801/3991466468","cluster_addr":"172.21.15.188:6803/3991466468","heartbeat_back_addr":"172.21.15.188:6807/3991466468","heartbeat_front_addr":"172.21.15.188:6805/3991466468","state":["exists","up"]},{"osd":3,"uuid":"4c8865aa-fef4-4710-b766-1385adb9462c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1713357192},{"type":"v1","addr":"172.21.15.100:6811","nonce":1713357192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1713357192},{"type":"v1","addr":"172.21.15.100:6813","nonce":1713357192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1713357192},{"type":"v1","addr":"172.21.15.100:6817","nonce":1713357192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1713357192},{"type":"v1","addr":"172.21.15.100:6815","nonce":1713357192}]},"public_addr":"172.21.15.100:6811/1713357192","cluster_addr":"172.21.15.100:6813/1713357192","heartbeat_back_addr":"172.21.15.100:6817/1713357192","heartbeat_front_addr":"172.21.15.100:6815/1713357192","state":["exists","up"]},{"osd":4,"uuid":"a9653800-da57-43eb-8781-3188e8a3949f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":2222814192},{"type":"v1","addr":"172.21.15.178:6809","nonce":2222814192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":2222814192},{"type":"v1","addr":"172.21.15.178:6811","nonce":2222814192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":2222814192},{"type":"v1","addr":"172.21.15.178:6815","nonce":2222814192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":2222814192},{"type":"v1","addr":"172.21.15.178:6813","nonce":2222814192}]},"public_addr":"172.21.15.178:6809/2222814192","cluster_addr":"172.21.15.178:6811/2222814192","heartbeat_back_addr":"172.21.15.178:6815/2222814192","heartbeat_front_addr":"172.21.15.178:6813/2222814192","state":["exists","up"]},{"osd":5,"uuid":"fb5f8af2-1eb0-4d8a-a7a3-ed6dd68e692e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3517375935},{"type":"v1","addr":"172.21.15.188:6809","nonce":3517375935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3517375935},{"type":"v1","addr":"172.21.15.188:6811","nonce":3517375935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3517375935},{"type":"v1","addr":"172.21.15.188:6815","nonce":3517375935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3517375935},{"type":"v1","addr":"172.21.15.188:6813","nonce":3517375935}]},"public_addr":"172.21.15.188:6809/3517375935","cluster_addr":"172.21.15.188:6811/3517375935","heartbeat_back_addr":"172.21.15.188:6815/3517375935","heartbeat_front_addr":"172.21.15.188:6813/3517375935","state":["exists","up"]},{"osd":6,"uuid":"84b57387-0a94-4e49-b099-493d329b87dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1410593862},{"type":"v1","addr":"172.21.15.100:6819","nonce":1410593862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1410593862},{"type":"v1","addr":"172.21.15.100:6821","nonce":1410593862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1410593862},{"type":"v1","addr":"172.21.15.100:6825","nonce":1410593862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1410593862},{"type":"v1","addr":"172.21.15.100:6823","nonce":1410593862}]},"public_addr":"172.21.15.100:6819/1410593862","cluster_addr":"172.21.15.100:6821/1410593862","heartbeat_back_addr":"172.21.15.100:6825/1410593862","heartbeat_front_addr":"172.21.15.100:6823/1410593862","state":["exists","up"]},{"osd":7,"uuid":"02ada28f-ac4f-47cf-983f-8a6da1140abe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2458690455},{"type":"v1","addr":"172.21.15.178:6817","nonce":2458690455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2458690455},{"type":"v1","addr":"172.21.15.178:6819","nonce":2458690455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2458690455},{"type":"v1","addr":"172.21.15.178:6823","nonce":2458690455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2458690455},{"type":"v1","addr":"172.21.15.178:6821","nonce":2458690455}]},"public_addr":"172.21.15.178:6817/2458690455","cluster_addr":"172.21.15.178:6819/2458690455","heartbeat_back_addr":"172.21.15.178:6823/2458690455","heartbeat_front_addr":"172.21.15.178:6821/2458690455","state":["exists","up"]},{"osd":8,"uuid":"e7e489bc-15fc-4a4c-b297-d1bb626513ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1523872826},{"type":"v1","addr":"172.21.15.188:6817","nonce":1523872826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1523872826},{"type":"v1","addr":"172.21.15.188:6819","nonce":1523872826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1523872826},{"type":"v1","addr":"172.21.15.188:6823","nonce":1523872826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1523872826},{"type":"v1","addr":"172.21.15.188:6821","nonce":1523872826}]},"public_addr":"172.21.15.188:6817/1523872826","cluster_addr":"172.21.15.188:6819/1523872826","heartbeat_back_addr":"172.21.15.188:6823/1523872826","heartbeat_front_addr":"172.21.15.188:6821/1523872826","state":["exists","up"]},{"osd":9,"uuid":"72328326-cd16-45d9-bfd9-40f6c5c20f5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3585761978},{"type":"v1","addr":"172.21.15.100:6827","nonce":3585761978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3585761978},{"type":"v1","addr":"172.21.15.100:6829","nonce":3585761978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3585761978},{"type":"v1","addr":"172.21.15.100:6833","nonce":3585761978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3585761978},{"type":"v1","addr":"172.21.15.100:6831","nonce":3585761978}]},"public_addr":"172.21.15.100:6827/3585761978","cluster_addr":"172.21.15.100:6829/3585761978","heartbeat_back_addr":"172.21.15.100:6833/3585761978","heartbeat_front_addr":"172.21.15.100:6831/3585761978","state":["exists","up"]},{"osd":10,"uuid":"a355f9a8-91a3-41d4-814d-5a9c9b953706","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2985841725},{"type":"v1","addr":"172.21.15.178:6825","nonce":2985841725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2985841725},{"type":"v1","addr":"172.21.15.178:6827","nonce":2985841725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2985841725},{"type":"v1","addr":"172.21.15.178:6831","nonce":2985841725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2985841725},{"type":"v1","addr":"172.21.15.178:6829","nonce":2985841725}]},"public_addr":"172.21.15.178:6825/2985841725","cluster_addr":"172.21.15.178:6827/2985841725","heartbeat_back_addr":"172.21.15.178:6831/2985841725","heartbeat_front_addr":"172.21.15.178:6829/2985841725","state":["exists","up"]},{"osd":11,"uuid":"76a5e13c-7e38-42f9-bfa7-7fedec7c7184","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2574955118},{"type":"v1","addr":"172.21.15.188:6825","nonce":2574955118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2574955118},{"type":"v1","addr":"172.21.15.188:6827","nonce":2574955118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2574955118},{"type":"v1","addr":"172.21.15.188:6831","nonce":2574955118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2574955118},{"type":"v1","addr":"172.21.15.188:6829","nonce":2574955118}]},"public_addr":"172.21.15.188:6825/2574955118","cluster_addr":"172.21.15.188:6827/2574955118","heartbeat_back_addr":"172.21.15.188:6831/2574955118","heartbeat_front_addr":"172.21.15.188:6829/2574955118","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:08:36.086637+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:08:58.948031+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:09:22.142517+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:09:47.655748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:10:12.853999+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:10:37.911383+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:11:04.448815+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:11:30.331740+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:11:55.819278+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:12:23.204612+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:12:49.484835+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:13:15.783531+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:6800/1597433518":"2024-05-11T11:07:00.092732+0000","172.21.15.100:6800/1086450256":"2024-05-11T11:06:33.114176+0000","172.21.15.100:0/767964410":"2024-05-11T11:07:00.092732+0000","172.21.15.100:6801/1597433518":"2024-05-11T11:07:00.092732+0000","172.21.15.100:6801/1086450256":"2024-05-11T11:06:33.114176+0000","172.21.15.100:0/2770572070":"2024-05-11T11:06:33.114176+0000","172.21.15.100:0/1379340513":"2024-05-11T11:06:33.114176+0000","172.21.15.100:0/3025646953":"2024-05-11T11:07:00.092732+0000","172.21.15.100:0/4099258930":"2024-05-11T11:07:00.092732+0000","172.21.15.100:0/4267693567":"2024-05-11T11:06:33.114176+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T11:14:48.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:48.360+0000 7f84637eb700 1 -- 172.21.15.100:0/1924196276 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f84480b2900 msgr2=0x7f84480b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:48.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:48.360+0000 7f84637eb700 1 --2- 172.21.15.100:0/1924196276 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f84480b2900 0x7f84480b4dc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f845c19a9a0 tx=0x7f845800b410 comp rx=0 tx=0).stop 2024-05-10T11:14:48.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:48.360+0000 7f84637eb700 1 -- 172.21.15.100:0/1924196276 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f845c106f60 msgr2=0x7f845c199860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:48.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:48.360+0000 7f84637eb700 1 --2- 172.21.15.100:0/1924196276 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f845c106f60 0x7f845c199860 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f84540085c0 tx=0x7f845400a890 comp rx=0 tx=0).stop 2024-05-10T11:14:48.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:48.360+0000 7f84637eb700 1 -- 172.21.15.100:0/1924196276 shutdown_connections 2024-05-10T11:14:48.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:48.360+0000 7f84637eb700 1 --2- 172.21.15.100:0/1924196276 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f84480b2900 0x7f84480b4dc0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:48.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:48.360+0000 7f84637eb700 1 --2- 172.21.15.100:0/1924196276 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f845c106f60 0x7f845c199860 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:48.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:48.360+0000 7f84637eb700 1 --2- 172.21.15.100:0/1924196276 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f845c1020c0 0x7f845c19f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:48.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:48.360+0000 7f84637eb700 1 --2- 172.21.15.100:0/1924196276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f845c102a70 0x7f845c19fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:48.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:48.360+0000 7f84637eb700 1 -- 172.21.15.100:0/1924196276 >> 172.21.15.100:0/1924196276 conn(0x7f845c0776c0 msgr2=0x7f845c105a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:14:48.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:48.360+0000 7f84637eb700 1 -- 172.21.15.100:0/1924196276 shutdown_connections 2024-05-10T11:14:48.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:48.360+0000 7f84637eb700 1 -- 172.21.15.100:0/1924196276 wait complete. 2024-05-10T11:14:48.725 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:48 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/1924196276' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T11:14:48.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:48 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/1924196276' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T11:14:48.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:48 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/1924196276' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T11:14:49.063 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-10T11:14:49.306 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:14:49.680 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:49 smithi100 ceph-mon[30296]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:49.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:49 smithi178 ceph-mon[33359]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:49.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:49 smithi188 ceph-mon[34075]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:51.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:51.082+0000 7f4967450700 1 -- 172.21.15.100:0/1662023513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f496006c590 msgr2=0x7f4960110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:51.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:51.082+0000 7f4967450700 1 --2- 172.21.15.100:0/1662023513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f496006c590 0x7f4960110330 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f495800b0a0 tx=0x7f495800b3b0 comp rx=0 tx=0).stop 2024-05-10T11:14:51.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:51.082+0000 7f4967450700 1 -- 172.21.15.100:0/1662023513 shutdown_connections 2024-05-10T11:14:51.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:51.082+0000 7f4967450700 1 --2- 172.21.15.100:0/1662023513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f496006c590 0x7f4960110330 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:51.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:51.082+0000 7f4967450700 1 --2- 172.21.15.100:0/1662023513 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4960109510 0x7f49601098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:51.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:51.082+0000 7f4967450700 1 --2- 172.21.15.100:0/1662023513 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f496006bbd0 0x7f496006c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:51.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:51.082+0000 7f4967450700 1 -- 172.21.15.100:0/1662023513 >> 172.21.15.100:0/1662023513 conn(0x7f4960079350 msgr2=0x7f4960079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:14:51.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:51.082+0000 7f4967450700 1 -- 172.21.15.100:0/1662023513 shutdown_connections 2024-05-10T11:14:51.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:51.082+0000 7f4967450700 1 -- 172.21.15.100:0/1662023513 wait complete. 2024-05-10T11:14:51.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:51.083+0000 7f4967450700 1 Processor -- start 2024-05-10T11:14:51.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:51.083+0000 7f4967450700 1 -- start start 2024-05-10T11:14:51.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:51.083+0000 7f4967450700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f496006bbd0 0x7f496019f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:51.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:51.083+0000 7f4967450700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f496006c590 0x7f496019fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:51.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:51.083+0000 7f4967450700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4960109510 0x7f4960199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:51.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:51.083+0000 7f4967450700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f49601135d0 con 0x7f496006c590 2024-05-10T11:14:51.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:51.083+0000 7f4967450700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4960113450 con 0x7f496006bbd0 2024-05-10T11:14:51.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:51.083+0000 7f4967450700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4960113750 con 0x7f4960109510 2024-05-10T11:14:51.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:51.083+0000 7f49649eb700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f496006c590 0x7f496019fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:51.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:51.083+0000 7f49649eb700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f496006c590 0x7f496019fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37400/0 (socket says 172.21.15.100:37400) 2024-05-10T11:14:51.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:51.083+0000 7f49649eb700 1 -- 172.21.15.100:0/3226495539 learned_addr learned my addr 172.21.15.100:0/3226495539 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:14:51.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:51.083+0000 7f49651ec700 1 --2- 172.21.15.100:0/3226495539 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f496006bbd0 0x7f496019f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:51.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:51.083+0000 7f49659ed700 1 --2- 172.21.15.100:0/3226495539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4960109510 0x7f4960199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:51.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:51.084+0000 7f49649eb700 1 -- 172.21.15.100:0/3226495539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4960109510 msgr2=0x7f4960199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:51.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:51.084+0000 7f49649eb700 1 --2- 172.21.15.100:0/3226495539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4960109510 0x7f4960199a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:51.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:51.084+0000 7f49649eb700 1 -- 172.21.15.100:0/3226495539 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f496006bbd0 msgr2=0x7f496019f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:51.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:51.084+0000 7f49649eb700 1 --2- 172.21.15.100:0/3226495539 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f496006bbd0 0x7f496019f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:51.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:51.084+0000 7f49649eb700 1 -- 172.21.15.100:0/3226495539 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f496019a190 con 0x7f496006c590 2024-05-10T11:14:51.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:51.084+0000 7f49659ed700 1 --2- 172.21.15.100:0/3226495539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4960109510 0x7f4960199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:14:51.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:51.084+0000 7f49649eb700 1 --2- 172.21.15.100:0/3226495539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f496006c590 0x7f496019fd60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f495000b530 tx=0x7f495000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:14:51.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:51.084+0000 7f49567fc700 1 -- 172.21.15.100:0/3226495539 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4950015020 con 0x7f496006c590 2024-05-10T11:14:51.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:51.084+0000 7f49567fc700 1 -- 172.21.15.100:0/3226495539 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4950011cc0 con 0x7f496006c590 2024-05-10T11:14:51.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:51.084+0000 7f4967450700 1 -- 172.21.15.100:0/3226495539 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f496019a480 con 0x7f496006c590 2024-05-10T11:14:51.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:51.084+0000 7f49567fc700 1 -- 172.21.15.100:0/3226495539 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4950010dc0 con 0x7f496006c590 2024-05-10T11:14:51.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:51.084+0000 7f4967450700 1 -- 172.21.15.100:0/3226495539 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49601a66f0 con 0x7f496006c590 2024-05-10T11:14:51.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:51.085+0000 7f4967450700 1 -- 172.21.15.100:0/3226495539 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f496010b9f0 con 0x7f496006c590 2024-05-10T11:14:51.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:51.086+0000 7f49567fc700 1 -- 172.21.15.100:0/3226495539 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f4950018020 con 0x7f496006c590 2024-05-10T11:14:51.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:51.087+0000 7f49567fc700 1 --2- 172.21.15.100:0/3226495539 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f494c0b2900 0x7f494c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:51.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:51.087+0000 7f49651ec700 1 --2- 172.21.15.100:0/3226495539 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f494c0b2900 0x7f494c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:51.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:51.087+0000 7f49567fc700 1 -- 172.21.15.100:0/3226495539 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f49500e2260 con 0x7f496006c590 2024-05-10T11:14:51.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:51.087+0000 7f49651ec700 1 --2- 172.21.15.100:0/3226495539 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f494c0b2900 0x7f494c0b4dc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4960100c20 tx=0x7f495c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:14:51.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:51.090+0000 7f49567fc700 1 -- 172.21.15.100:0/3226495539 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f495001f020 con 0x7f496006c590 2024-05-10T11:14:51.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:51.294+0000 7f4967450700 1 -- 172.21.15.100:0/3226495539 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f496019af10 con 0x7f496006c590 2024-05-10T11:14:51.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:51.296+0000 7f49567fc700 1 -- 172.21.15.100:0/3226495539 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24211 (secure 0 0 0) 0x7f49500a9d40 con 0x7f496006c590 2024-05-10T11:14:51.296 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:14:51.296 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":87,"fsid":"20a542cc-0ebd-11ef-bc98-c7b262605968","created":"2024-05-10T11:06:05.454446+0000","modified":"2024-05-10T11:14:41.399746+0000","last_up_change":"2024-05-10T11:13:18.185471+0000","last_in_change":"2024-05-10T11:12:57.268678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T11:09:26.176441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T11:14:21.004285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T11:14:24.179414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T11:14:33.308517+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5f031418-da23-4e85-8669-160960dadd09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":643028278},{"type":"v1","addr":"172.21.15.100:6803","nonce":643028278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":643028278},{"type":"v1","addr":"172.21.15.100:6805","nonce":643028278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":643028278},{"type":"v1","addr":"172.21.15.100:6809","nonce":643028278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":643028278},{"type":"v1","addr":"172.21.15.100:6807","nonce":643028278}]},"public_addr":"172.21.15.100:6803/643028278","cluster_addr":"172.21.15.100:6805/643028278","heartbeat_back_addr":"172.21.15.100:6809/643028278","heartbeat_front_addr":"172.21.15.100:6807/643028278","state":["exists","up"]},{"osd":1,"uuid":"a908cdc9-001e-4b65-9ab0-0dd08dba20f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":3103987017},{"type":"v1","addr":"172.21.15.178:6801","nonce":3103987017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":3103987017},{"type":"v1","addr":"172.21.15.178:6803","nonce":3103987017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":3103987017},{"type":"v1","addr":"172.21.15.178:6807","nonce":3103987017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":3103987017},{"type":"v1","addr":"172.21.15.178:6805","nonce":3103987017}]},"public_addr":"172.21.15.178:6801/3103987017","cluster_addr":"172.21.15.178:6803/3103987017","heartbeat_back_addr":"172.21.15.178:6807/3103987017","heartbeat_front_addr":"172.21.15.178:6805/3103987017","state":["exists","up"]},{"osd":2,"uuid":"4d8f89ae-9424-46e9-9c80-17c47cc5e8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3991466468},{"type":"v1","addr":"172.21.15.188:6801","nonce":3991466468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3991466468},{"type":"v1","addr":"172.21.15.188:6803","nonce":3991466468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3991466468},{"type":"v1","addr":"172.21.15.188:6807","nonce":3991466468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3991466468},{"type":"v1","addr":"172.21.15.188:6805","nonce":3991466468}]},"public_addr":"172.21.15.188:6801/3991466468","cluster_addr":"172.21.15.188:6803/3991466468","heartbeat_back_addr":"172.21.15.188:6807/3991466468","heartbeat_front_addr":"172.21.15.188:6805/3991466468","state":["exists","up"]},{"osd":3,"uuid":"4c8865aa-fef4-4710-b766-1385adb9462c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1713357192},{"type":"v1","addr":"172.21.15.100:6811","nonce":1713357192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1713357192},{"type":"v1","addr":"172.21.15.100:6813","nonce":1713357192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1713357192},{"type":"v1","addr":"172.21.15.100:6817","nonce":1713357192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1713357192},{"type":"v1","addr":"172.21.15.100:6815","nonce":1713357192}]},"public_addr":"172.21.15.100:6811/1713357192","cluster_addr":"172.21.15.100:6813/1713357192","heartbeat_back_addr":"172.21.15.100:6817/1713357192","heartbeat_front_addr":"172.21.15.100:6815/1713357192","state":["exists","up"]},{"osd":4,"uuid":"a9653800-da57-43eb-8781-3188e8a3949f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":2222814192},{"type":"v1","addr":"172.21.15.178:6809","nonce":2222814192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":2222814192},{"type":"v1","addr":"172.21.15.178:6811","nonce":2222814192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":2222814192},{"type":"v1","addr":"172.21.15.178:6815","nonce":2222814192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":2222814192},{"type":"v1","addr":"172.21.15.178:6813","nonce":2222814192}]},"public_addr":"172.21.15.178:6809/2222814192","cluster_addr":"172.21.15.178:6811/2222814192","heartbeat_back_addr":"172.21.15.178:6815/2222814192","heartbeat_front_addr":"172.21.15.178:6813/2222814192","state":["exists","up"]},{"osd":5,"uuid":"fb5f8af2-1eb0-4d8a-a7a3-ed6dd68e692e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3517375935},{"type":"v1","addr":"172.21.15.188:6809","nonce":3517375935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3517375935},{"type":"v1","addr":"172.21.15.188:6811","nonce":3517375935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3517375935},{"type":"v1","addr":"172.21.15.188:6815","nonce":3517375935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3517375935},{"type":"v1","addr":"172.21.15.188:6813","nonce":3517375935}]},"public_addr":"172.21.15.188:6809/3517375935","cluster_addr":"172.21.15.188:6811/3517375935","heartbeat_back_addr":"172.21.15.188:6815/3517375935","heartbeat_front_addr":"172.21.15.188:6813/3517375935","state":["exists","up"]},{"osd":6,"uuid":"84b57387-0a94-4e49-b099-493d329b87dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1410593862},{"type":"v1","addr":"172.21.15.100:6819","nonce":1410593862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1410593862},{"type":"v1","addr":"172.21.15.100:6821","nonce":1410593862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1410593862},{"type":"v1","addr":"172.21.15.100:6825","nonce":1410593862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1410593862},{"type":"v1","addr":"172.21.15.100:6823","nonce":1410593862}]},"public_addr":"172.21.15.100:6819/1410593862","cluster_addr":"172.21.15.100:6821/1410593862","heartbeat_back_addr":"172.21.15.100:6825/1410593862","heartbeat_front_addr":"172.21.15.100:6823/1410593862","state":["exists","up"]},{"osd":7,"uuid":"02ada28f-ac4f-47cf-983f-8a6da1140abe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2458690455},{"type":"v1","addr":"172.21.15.178:6817","nonce":2458690455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2458690455},{"type":"v1","addr":"172.21.15.178:6819","nonce":2458690455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2458690455},{"type":"v1","addr":"172.21.15.178:6823","nonce":2458690455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2458690455},{"type":"v1","addr":"172.21.15.178:6821","nonce":2458690455}]},"public_addr":"172.21.15.178:6817/2458690455","cluster_addr":"172.21.15.178:6819/2458690455","heartbeat_back_addr":"172.21.15.178:6823/2458690455","heartbeat_front_addr":"172.21.15.178:6821/2458690455","state":["exists","up"]},{"osd":8,"uuid":"e7e489bc-15fc-4a4c-b297-d1bb626513ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1523872826},{"type":"v1","addr":"172.21.15.188:6817","nonce":1523872826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1523872826},{"type":"v1","addr":"172.21.15.188:6819","nonce":1523872826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1523872826},{"type":"v1","addr":"172.21.15.188:6823","nonce":1523872826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1523872826},{"type":"v1","addr":"172.21.15.188:6821","nonce":1523872826}]},"public_addr":"172.21.15.188:6817/1523872826","cluster_addr":"172.21.15.188:6819/1523872826","heartbeat_back_addr":"172.21.15.188:6823/1523872826","heartbeat_front_addr":"172.21.15.188:6821/1523872826","state":["exists","up"]},{"osd":9,"uuid":"72328326-cd16-45d9-bfd9-40f6c5c20f5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3585761978},{"type":"v1","addr":"172.21.15.100:6827","nonce":3585761978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3585761978},{"type":"v1","addr":"172.21.15.100:6829","nonce":3585761978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3585761978},{"type":"v1","addr":"172.21.15.100:6833","nonce":3585761978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3585761978},{"type":"v1","addr":"172.21.15.100:6831","nonce":3585761978}]},"public_addr":"172.21.15.100:6827/3585761978","cluster_addr":"172.21.15.100:6829/3585761978","heartbeat_back_addr":"172.21.15.100:6833/3585761978","heartbeat_front_addr":"172.21.15.100:6831/3585761978","state":["exists","up"]},{"osd":10,"uuid":"a355f9a8-91a3-41d4-814d-5a9c9b953706","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2985841725},{"type":"v1","addr":"172.21.15.178:6825","nonce":2985841725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2985841725},{"type":"v1","addr":"172.21.15.178:6827","nonce":2985841725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2985841725},{"type":"v1","addr":"172.21.15.178:6831","nonce":2985841725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2985841725},{"type":"v1","addr":"172.21.15.178:6829","nonce":2985841725}]},"public_addr":"172.21.15.178:6825/2985841725","cluster_addr":"172.21.15.178:6827/2985841725","heartbeat_back_addr":"172.21.15.178:6831/2985841725","heartbeat_front_addr":"172.21.15.178:6829/2985841725","state":["exists","up"]},{"osd":11,"uuid":"76a5e13c-7e38-42f9-bfa7-7fedec7c7184","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2574955118},{"type":"v1","addr":"172.21.15.188:6825","nonce":2574955118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2574955118},{"type":"v1","addr":"172.21.15.188:6827","nonce":2574955118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2574955118},{"type":"v1","addr":"172.21.15.188:6831","nonce":2574955118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2574955118},{"type":"v1","addr":"172.21.15.188:6829","nonce":2574955118}]},"public_addr":"172.21.15.188:6825/2574955118","cluster_addr":"172.21.15.188:6827/2574955118","heartbeat_back_addr":"172.21.15.188:6831/2574955118","heartbeat_front_addr":"172.21.15.188:6829/2574955118","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:08:36.086637+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:08:58.948031+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:09:22.142517+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:09:47.655748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:10:12.853999+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:10:37.911383+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:11:04.448815+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:11:30.331740+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:11:55.819278+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:12:23.204612+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:12:49.484835+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:13:15.783531+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:6800/1597433518":"2024-05-11T11:07:00.092732+0000","172.21.15.100:6800/1086450256":"2024-05-11T11:06:33.114176+0000","172.21.15.100:0/767964410":"2024-05-11T11:07:00.092732+0000","172.21.15.100:6801/1597433518":"2024-05-11T11:07:00.092732+0000","172.21.15.100:6801/1086450256":"2024-05-11T11:06:33.114176+0000","172.21.15.100:0/2770572070":"2024-05-11T11:06:33.114176+0000","172.21.15.100:0/1379340513":"2024-05-11T11:06:33.114176+0000","172.21.15.100:0/3025646953":"2024-05-11T11:07:00.092732+0000","172.21.15.100:0/4099258930":"2024-05-11T11:07:00.092732+0000","172.21.15.100:0/4267693567":"2024-05-11T11:06:33.114176+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T11:14:51.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:51.298+0000 7f4967450700 1 -- 172.21.15.100:0/3226495539 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f494c0b2900 msgr2=0x7f494c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:51.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:51.298+0000 7f4967450700 1 --2- 172.21.15.100:0/3226495539 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f494c0b2900 0x7f494c0b4dc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4960100c20 tx=0x7f495c009190 comp rx=0 tx=0).stop 2024-05-10T11:14:51.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:51.298+0000 7f4967450700 1 -- 172.21.15.100:0/3226495539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f496006c590 msgr2=0x7f496019fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:51.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:51.298+0000 7f4967450700 1 --2- 172.21.15.100:0/3226495539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f496006c590 0x7f496019fd60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f495000b530 tx=0x7f495000b840 comp rx=0 tx=0).stop 2024-05-10T11:14:51.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:51.298+0000 7f4967450700 1 -- 172.21.15.100:0/3226495539 shutdown_connections 2024-05-10T11:14:51.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:51.298+0000 7f4967450700 1 --2- 172.21.15.100:0/3226495539 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f494c0b2900 0x7f494c0b4dc0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:51.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:51.298+0000 7f4967450700 1 --2- 172.21.15.100:0/3226495539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4960109510 0x7f4960199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:51.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:51.298+0000 7f4967450700 1 --2- 172.21.15.100:0/3226495539 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f496006bbd0 0x7f496019f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:51.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:51.298+0000 7f4967450700 1 --2- 172.21.15.100:0/3226495539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f496006c590 0x7f496019fd60 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:51.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:51.298+0000 7f4967450700 1 -- 172.21.15.100:0/3226495539 >> 172.21.15.100:0/3226495539 conn(0x7f4960079350 msgr2=0x7f4960101360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:14:51.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:51.298+0000 7f4967450700 1 -- 172.21.15.100:0/3226495539 shutdown_connections 2024-05-10T11:14:51.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:51.298+0000 7f4967450700 1 -- 172.21.15.100:0/3226495539 wait complete. 2024-05-10T11:14:51.730 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:51 smithi100 ceph-mon[30296]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:51.731 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:51 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/3226495539' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T11:14:51.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:51 smithi178 ceph-mon[33359]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:51.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:51 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/3226495539' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T11:14:51.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:51 smithi188 ceph-mon[34075]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:51.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:51 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/3226495539' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T11:14:51.988 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T11:14:52.231 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:14:53.663 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:53 smithi100 ceph-mon[30296]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:53.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:53 smithi178 ceph-mon[33359]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:53.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:53 smithi188 ceph-mon[34075]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:53.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:53.891+0000 7fd784213700 1 -- 172.21.15.100:0/681221049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd77c1023a0 msgr2=0x7fd77c102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:53.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:53.891+0000 7fd784213700 1 --2- 172.21.15.100:0/681221049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd77c1023a0 0x7fd77c102820 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd76c009990 tx=0x7fd76c009ca0 comp rx=0 tx=0).stop 2024-05-10T11:14:53.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:53.891+0000 7fd784213700 1 -- 172.21.15.100:0/681221049 shutdown_connections 2024-05-10T11:14:53.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:53.891+0000 7fd784213700 1 --2- 172.21.15.100:0/681221049 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd77c102d60 0x7fd77c10f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:53.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:53.891+0000 7fd784213700 1 --2- 172.21.15.100:0/681221049 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd77c108410 0x7fd77c1087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:53.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:53.891+0000 7fd784213700 1 --2- 172.21.15.100:0/681221049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd77c1023a0 0x7fd77c102820 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:53.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:53.891+0000 7fd784213700 1 -- 172.21.15.100:0/681221049 >> 172.21.15.100:0/681221049 conn(0x7fd77c079350 msgr2=0x7fd77c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:14:53.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:53.891+0000 7fd784213700 1 -- 172.21.15.100:0/681221049 shutdown_connections 2024-05-10T11:14:53.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:53.891+0000 7fd784213700 1 -- 172.21.15.100:0/681221049 wait complete. 2024-05-10T11:14:53.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:53.892+0000 7fd784213700 1 Processor -- start 2024-05-10T11:14:53.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:53.892+0000 7fd784213700 1 -- start start 2024-05-10T11:14:53.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:53.892+0000 7fd784213700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd77c1023a0 0x7fd77c104460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:53.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:53.892+0000 7fd784213700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd77c102d60 0x7fd77c1049a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:53.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:53.892+0000 7fd784213700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd77c108410 0x7fd77c104ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:53.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:53.892+0000 7fd784213700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd77c1124e0 con 0x7fd77c108410 2024-05-10T11:14:53.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:53.892+0000 7fd784213700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd77c112360 con 0x7fd77c102d60 2024-05-10T11:14:53.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:53.892+0000 7fd784213700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd77c112660 con 0x7fd77c1023a0 2024-05-10T11:14:53.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:53.892+0000 7fd7827b0700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd77c108410 0x7fd77c104ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:53.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:53.892+0000 7fd7827b0700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd77c108410 0x7fd77c104ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37416/0 (socket says 172.21.15.100:37416) 2024-05-10T11:14:53.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:53.892+0000 7fd7827b0700 1 -- 172.21.15.100:0/3135494063 learned_addr learned my addr 172.21.15.100:0/3135494063 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:14:53.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:53.892+0000 7fd781faf700 1 --2- 172.21.15.100:0/3135494063 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd77c1023a0 0x7fd77c104460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:53.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:53.892+0000 7fd7817ae700 1 --2- 172.21.15.100:0/3135494063 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd77c102d60 0x7fd77c1049a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:53.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:53.892+0000 7fd7827b0700 1 -- 172.21.15.100:0/3135494063 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd77c1023a0 msgr2=0x7fd77c104460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:53.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:53.892+0000 7fd7827b0700 1 --2- 172.21.15.100:0/3135494063 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd77c1023a0 0x7fd77c104460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:53.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:53.892+0000 7fd7827b0700 1 -- 172.21.15.100:0/3135494063 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd77c102d60 msgr2=0x7fd77c1049a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:53.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:53.892+0000 7fd7827b0700 1 --2- 172.21.15.100:0/3135494063 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd77c102d60 0x7fd77c1049a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:53.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:53.892+0000 7fd7827b0700 1 -- 172.21.15.100:0/3135494063 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd77c1ac650 con 0x7fd77c108410 2024-05-10T11:14:53.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:53.892+0000 7fd7817ae700 1 --2- 172.21.15.100:0/3135494063 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd77c102d60 0x7fd77c1049a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:14:53.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:53.893+0000 7fd7827b0700 1 --2- 172.21.15.100:0/3135494063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd77c108410 0x7fd77c104ee0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd77400b970 tx=0x7fd77400bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:14:53.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:53.893+0000 7fd772ffd700 1 -- 172.21.15.100:0/3135494063 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd774019070 con 0x7fd77c108410 2024-05-10T11:14:53.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:53.893+0000 7fd772ffd700 1 -- 172.21.15.100:0/3135494063 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd77400ce40 con 0x7fd77c108410 2024-05-10T11:14:53.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:53.893+0000 7fd772ffd700 1 -- 172.21.15.100:0/3135494063 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd774007370 con 0x7fd77c108410 2024-05-10T11:14:53.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:53.893+0000 7fd784213700 1 -- 172.21.15.100:0/3135494063 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd77c1ac8e0 con 0x7fd77c108410 2024-05-10T11:14:53.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:53.893+0000 7fd784213700 1 -- 172.21.15.100:0/3135494063 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd77c1ace20 con 0x7fd77c108410 2024-05-10T11:14:53.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:53.894+0000 7fd784213700 1 -- 172.21.15.100:0/3135494063 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd77c103ac0 con 0x7fd77c108410 2024-05-10T11:14:53.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:53.895+0000 7fd772ffd700 1 -- 172.21.15.100:0/3135494063 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fd77401c070 con 0x7fd77c108410 2024-05-10T11:14:53.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:53.896+0000 7fd772ffd700 1 --2- 172.21.15.100:0/3135494063 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fd7680b2880 0x7fd7680b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:53.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:53.896+0000 7fd781faf700 1 --2- 172.21.15.100:0/3135494063 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fd7680b2880 0x7fd7680b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:53.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:53.896+0000 7fd772ffd700 1 -- 172.21.15.100:0/3135494063 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7fd7740e21f0 con 0x7fd77c108410 2024-05-10T11:14:53.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:53.896+0000 7fd781faf700 1 --2- 172.21.15.100:0/3135494063 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fd7680b2880 0x7fd7680b4d40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd778006fd0 tx=0x7fd778008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:14:53.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:53.899+0000 7fd772ffd700 1 -- 172.21.15.100:0/3135494063 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fd774023070 con 0x7fd77c108410 2024-05-10T11:14:54.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:54.151+0000 7fd784213700 1 -- 172.21.15.100:0/3135494063 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd77c1ad110 con 0x7fd77c108410 2024-05-10T11:14:54.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:54.152+0000 7fd772ffd700 1 -- 172.21.15.100:0/3135494063 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11821 (secure 0 0 0) 0x7fd7740a9cd0 con 0x7fd77c108410 2024-05-10T11:14:54.152 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:14:54.152 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2072225012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2072225012},{"type":"v1","addr":"172.21.15.100:6835","nonce":2072225012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/401939540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":401939540},{"type":"v1","addr":"172.21.15.100:6837","nonce":401939540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14571,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/3946311025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3946311025},{"type":"v1","addr":"172.21.15.100:6839","nonce":3946311025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3736371660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3736371660},{"type":"v1","addr":"172.21.15.100:6841","nonce":3736371660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/2741810791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2741810791},{"type":"v1","addr":"172.21.15.178:6833","nonce":2741810791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1704891041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1704891041},{"type":"v1","addr":"172.21.15.188:6833","nonce":1704891041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24395,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2124076312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2124076312},{"type":"v1","addr":"172.21.15.178:6835","nonce":2124076312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24413,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/3224358425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3224358425},{"type":"v1","addr":"172.21.15.178:6837","nonce":3224358425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24421,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/515547690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":515547690},{"type":"v1","addr":"172.21.15.188:6835","nonce":515547690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24427,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1943342620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1943342620},{"type":"v1","addr":"172.21.15.188:6837","nonce":1943342620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24437,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6839/3760985404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3760985404},{"type":"v1","addr":"172.21.15.178:6839","nonce":3760985404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T11:14:27.242452+0000","modified":"2024-05-10T11:14:41.399890+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.188:6839/3194917070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":3194917070},{"type":"v1","addr":"172.21.15.188:6839","nonce":3194917070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24439,"qdb_cluster":[24439]},"id":1}]} 2024-05-10T11:14:54.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:54.154+0000 7fd784213700 1 -- 172.21.15.100:0/3135494063 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fd7680b2880 msgr2=0x7fd7680b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:54.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:54.154+0000 7fd784213700 1 --2- 172.21.15.100:0/3135494063 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fd7680b2880 0x7fd7680b4d40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd778006fd0 tx=0x7fd778008040 comp rx=0 tx=0).stop 2024-05-10T11:14:54.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:54.154+0000 7fd784213700 1 -- 172.21.15.100:0/3135494063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd77c108410 msgr2=0x7fd77c104ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:54.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:54.154+0000 7fd784213700 1 --2- 172.21.15.100:0/3135494063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd77c108410 0x7fd77c104ee0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd77400b970 tx=0x7fd77400bd30 comp rx=0 tx=0).stop 2024-05-10T11:14:54.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:54.154+0000 7fd784213700 1 -- 172.21.15.100:0/3135494063 shutdown_connections 2024-05-10T11:14:54.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:54.154+0000 7fd784213700 1 --2- 172.21.15.100:0/3135494063 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fd7680b2880 0x7fd7680b4d40 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:54.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:54.154+0000 7fd784213700 1 --2- 172.21.15.100:0/3135494063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd77c108410 0x7fd77c104ee0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:54.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:54.154+0000 7fd784213700 1 --2- 172.21.15.100:0/3135494063 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd77c1023a0 0x7fd77c104460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:54.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:54.154+0000 7fd784213700 1 --2- 172.21.15.100:0/3135494063 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd77c102d60 0x7fd77c1049a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:54.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:54.154+0000 7fd784213700 1 -- 172.21.15.100:0/3135494063 >> 172.21.15.100:0/3135494063 conn(0x7fd77c079350 msgr2=0x7fd77c1068e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:14:54.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:54.154+0000 7fd784213700 1 -- 172.21.15.100:0/3135494063 shutdown_connections 2024-05-10T11:14:54.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:54.154+0000 7fd784213700 1 -- 172.21.15.100:0/3135494063 wait complete. 2024-05-10T11:14:54.155 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 17 2024-05-10T11:14:54.731 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:54 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/3135494063' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:14:54.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:54 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/3135494063' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:14:54.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:54 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/3135494063' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:14:54.810 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-10T11:14:55.055 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:14:55.515 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:55 smithi100 ceph-mon[30296]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:55.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:55 smithi178 ceph-mon[33359]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:55.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:55 smithi188 ceph-mon[34075]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:56.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:56.689+0000 7f1c51dd3700 1 -- 172.21.15.100:0/3840139229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c4c1023a0 msgr2=0x7f1c4c102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:56.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:56.689+0000 7f1c51dd3700 1 --2- 172.21.15.100:0/3840139229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c4c1023a0 0x7f1c4c102820 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1c34009990 tx=0x7f1c34009ca0 comp rx=0 tx=0).stop 2024-05-10T11:14:56.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:56.689+0000 7f1c51dd3700 1 -- 172.21.15.100:0/3840139229 shutdown_connections 2024-05-10T11:14:56.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:56.689+0000 7f1c51dd3700 1 --2- 172.21.15.100:0/3840139229 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c4c102d60 0x7f1c4c10f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:56.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:56.689+0000 7f1c51dd3700 1 --2- 172.21.15.100:0/3840139229 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c4c108410 0x7f1c4c1087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:56.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:56.689+0000 7f1c51dd3700 1 --2- 172.21.15.100:0/3840139229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c4c1023a0 0x7f1c4c102820 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:56.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:56.689+0000 7f1c51dd3700 1 -- 172.21.15.100:0/3840139229 >> 172.21.15.100:0/3840139229 conn(0x7f1c4c079350 msgr2=0x7f1c4c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:14:56.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:56.689+0000 7f1c51dd3700 1 -- 172.21.15.100:0/3840139229 shutdown_connections 2024-05-10T11:14:56.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:56.689+0000 7f1c51dd3700 1 -- 172.21.15.100:0/3840139229 wait complete. 2024-05-10T11:14:56.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:56.690+0000 7f1c51dd3700 1 Processor -- start 2024-05-10T11:14:56.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:56.690+0000 7f1c51dd3700 1 -- start start 2024-05-10T11:14:56.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:56.690+0000 7f1c51dd3700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c4c1023a0 0x7f1c4c19a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:56.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:56.690+0000 7f1c51dd3700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c4c102d60 0x7f1c4c19ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:56.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:56.690+0000 7f1c51dd3700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c4c108410 0x7f1c4c1a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:56.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:56.690+0000 7f1c51dd3700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1c4c112490 con 0x7f1c4c102d60 2024-05-10T11:14:56.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:56.690+0000 7f1c51dd3700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1c4c112310 con 0x7f1c4c108410 2024-05-10T11:14:56.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:56.690+0000 7f1c51dd3700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1c4c112610 con 0x7f1c4c1023a0 2024-05-10T11:14:56.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:56.690+0000 7f1c43fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c4c102d60 0x7f1c4c19ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:56.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:56.690+0000 7f1c43fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c4c102d60 0x7f1c4c19ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50374/0 (socket says 172.21.15.100:50374) 2024-05-10T11:14:56.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:56.690+0000 7f1c43fff700 1 -- 172.21.15.100:0/671402890 learned_addr learned my addr 172.21.15.100:0/671402890 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:14:56.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:56.690+0000 7f1c4b7fe700 1 --2- 172.21.15.100:0/671402890 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c4c1023a0 0x7f1c4c19a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:56.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:56.691+0000 7f1c4bfff700 1 --2- 172.21.15.100:0/671402890 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c4c108410 0x7f1c4c1a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:56.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:56.691+0000 7f1c43fff700 1 -- 172.21.15.100:0/671402890 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c4c1023a0 msgr2=0x7f1c4c19a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:56.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:56.691+0000 7f1c43fff700 1 --2- 172.21.15.100:0/671402890 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c4c1023a0 0x7f1c4c19a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:56.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:56.691+0000 7f1c43fff700 1 -- 172.21.15.100:0/671402890 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c4c108410 msgr2=0x7f1c4c1a5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:56.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:56.691+0000 7f1c43fff700 1 --2- 172.21.15.100:0/671402890 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c4c108410 0x7f1c4c1a5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:56.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:56.691+0000 7f1c43fff700 1 -- 172.21.15.100:0/671402890 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c4c1a5ed0 con 0x7f1c4c102d60 2024-05-10T11:14:56.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:56.691+0000 7f1c4bfff700 1 --2- 172.21.15.100:0/671402890 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c4c108410 0x7f1c4c1a5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:14:56.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:56.691+0000 7f1c43fff700 1 --2- 172.21.15.100:0/671402890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c4c102d60 0x7f1c4c19ae50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1c34000c00 tx=0x7f1c340047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:14:56.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:56.691+0000 7f1c497fa700 1 -- 172.21.15.100:0/671402890 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c34027070 con 0x7f1c4c102d60 2024-05-10T11:14:56.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:56.691+0000 7f1c51dd3700 1 -- 172.21.15.100:0/671402890 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c4c1a6100 con 0x7f1c4c102d60 2024-05-10T11:14:56.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:56.691+0000 7f1c497fa700 1 -- 172.21.15.100:0/671402890 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1c3400fd00 con 0x7f1c4c102d60 2024-05-10T11:14:56.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:56.691+0000 7f1c497fa700 1 -- 172.21.15.100:0/671402890 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c34022e20 con 0x7f1c4c102d60 2024-05-10T11:14:56.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:56.691+0000 7f1c51dd3700 1 -- 172.21.15.100:0/671402890 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c4c1a6560 con 0x7f1c4c102d60 2024-05-10T11:14:56.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:56.692+0000 7f1c51dd3700 1 -- 172.21.15.100:0/671402890 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c4c10a980 con 0x7f1c4c102d60 2024-05-10T11:14:56.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:56.694+0000 7f1c497fa700 1 -- 172.21.15.100:0/671402890 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f1c3402a070 con 0x7f1c4c102d60 2024-05-10T11:14:56.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:56.695+0000 7f1c497fa700 1 --2- 172.21.15.100:0/671402890 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f1c380b2880 0x7f1c380b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:56.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:56.695+0000 7f1c4b7fe700 1 --2- 172.21.15.100:0/671402890 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f1c380b2880 0x7f1c380b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:56.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:56.695+0000 7f1c497fa700 1 -- 172.21.15.100:0/671402890 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f1c340e3430 con 0x7f1c4c102d60 2024-05-10T11:14:56.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:56.695+0000 7f1c4b7fe700 1 --2- 172.21.15.100:0/671402890 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f1c380b2880 0x7f1c380b4d40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c009a10 tx=0x7f1c3c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:14:56.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:56.698+0000 7f1c497fa700 1 -- 172.21.15.100:0/671402890 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f1c34013070 con 0x7f1c4c102d60 2024-05-10T11:14:56.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:56.909+0000 7f1c51dd3700 1 -- 172.21.15.100:0/671402890 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1c4c06a190 con 0x7f1c4c102d60 2024-05-10T11:14:56.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:56.911+0000 7f1c497fa700 1 -- 172.21.15.100:0/671402890 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24211 (secure 0 0 0) 0x7f1c340aafa0 con 0x7f1c4c102d60 2024-05-10T11:14:56.912 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:14:56.912 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":87,"fsid":"20a542cc-0ebd-11ef-bc98-c7b262605968","created":"2024-05-10T11:06:05.454446+0000","modified":"2024-05-10T11:14:41.399746+0000","last_up_change":"2024-05-10T11:13:18.185471+0000","last_in_change":"2024-05-10T11:12:57.268678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T11:09:26.176441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T11:14:21.004285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T11:14:24.179414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T11:14:33.308517+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5f031418-da23-4e85-8669-160960dadd09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":643028278},{"type":"v1","addr":"172.21.15.100:6803","nonce":643028278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":643028278},{"type":"v1","addr":"172.21.15.100:6805","nonce":643028278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":643028278},{"type":"v1","addr":"172.21.15.100:6809","nonce":643028278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":643028278},{"type":"v1","addr":"172.21.15.100:6807","nonce":643028278}]},"public_addr":"172.21.15.100:6803/643028278","cluster_addr":"172.21.15.100:6805/643028278","heartbeat_back_addr":"172.21.15.100:6809/643028278","heartbeat_front_addr":"172.21.15.100:6807/643028278","state":["exists","up"]},{"osd":1,"uuid":"a908cdc9-001e-4b65-9ab0-0dd08dba20f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":3103987017},{"type":"v1","addr":"172.21.15.178:6801","nonce":3103987017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":3103987017},{"type":"v1","addr":"172.21.15.178:6803","nonce":3103987017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":3103987017},{"type":"v1","addr":"172.21.15.178:6807","nonce":3103987017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":3103987017},{"type":"v1","addr":"172.21.15.178:6805","nonce":3103987017}]},"public_addr":"172.21.15.178:6801/3103987017","cluster_addr":"172.21.15.178:6803/3103987017","heartbeat_back_addr":"172.21.15.178:6807/3103987017","heartbeat_front_addr":"172.21.15.178:6805/3103987017","state":["exists","up"]},{"osd":2,"uuid":"4d8f89ae-9424-46e9-9c80-17c47cc5e8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3991466468},{"type":"v1","addr":"172.21.15.188:6801","nonce":3991466468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3991466468},{"type":"v1","addr":"172.21.15.188:6803","nonce":3991466468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3991466468},{"type":"v1","addr":"172.21.15.188:6807","nonce":3991466468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3991466468},{"type":"v1","addr":"172.21.15.188:6805","nonce":3991466468}]},"public_addr":"172.21.15.188:6801/3991466468","cluster_addr":"172.21.15.188:6803/3991466468","heartbeat_back_addr":"172.21.15.188:6807/3991466468","heartbeat_front_addr":"172.21.15.188:6805/3991466468","state":["exists","up"]},{"osd":3,"uuid":"4c8865aa-fef4-4710-b766-1385adb9462c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1713357192},{"type":"v1","addr":"172.21.15.100:6811","nonce":1713357192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1713357192},{"type":"v1","addr":"172.21.15.100:6813","nonce":1713357192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1713357192},{"type":"v1","addr":"172.21.15.100:6817","nonce":1713357192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1713357192},{"type":"v1","addr":"172.21.15.100:6815","nonce":1713357192}]},"public_addr":"172.21.15.100:6811/1713357192","cluster_addr":"172.21.15.100:6813/1713357192","heartbeat_back_addr":"172.21.15.100:6817/1713357192","heartbeat_front_addr":"172.21.15.100:6815/1713357192","state":["exists","up"]},{"osd":4,"uuid":"a9653800-da57-43eb-8781-3188e8a3949f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":2222814192},{"type":"v1","addr":"172.21.15.178:6809","nonce":2222814192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":2222814192},{"type":"v1","addr":"172.21.15.178:6811","nonce":2222814192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":2222814192},{"type":"v1","addr":"172.21.15.178:6815","nonce":2222814192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":2222814192},{"type":"v1","addr":"172.21.15.178:6813","nonce":2222814192}]},"public_addr":"172.21.15.178:6809/2222814192","cluster_addr":"172.21.15.178:6811/2222814192","heartbeat_back_addr":"172.21.15.178:6815/2222814192","heartbeat_front_addr":"172.21.15.178:6813/2222814192","state":["exists","up"]},{"osd":5,"uuid":"fb5f8af2-1eb0-4d8a-a7a3-ed6dd68e692e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3517375935},{"type":"v1","addr":"172.21.15.188:6809","nonce":3517375935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3517375935},{"type":"v1","addr":"172.21.15.188:6811","nonce":3517375935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3517375935},{"type":"v1","addr":"172.21.15.188:6815","nonce":3517375935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3517375935},{"type":"v1","addr":"172.21.15.188:6813","nonce":3517375935}]},"public_addr":"172.21.15.188:6809/3517375935","cluster_addr":"172.21.15.188:6811/3517375935","heartbeat_back_addr":"172.21.15.188:6815/3517375935","heartbeat_front_addr":"172.21.15.188:6813/3517375935","state":["exists","up"]},{"osd":6,"uuid":"84b57387-0a94-4e49-b099-493d329b87dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1410593862},{"type":"v1","addr":"172.21.15.100:6819","nonce":1410593862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1410593862},{"type":"v1","addr":"172.21.15.100:6821","nonce":1410593862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1410593862},{"type":"v1","addr":"172.21.15.100:6825","nonce":1410593862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1410593862},{"type":"v1","addr":"172.21.15.100:6823","nonce":1410593862}]},"public_addr":"172.21.15.100:6819/1410593862","cluster_addr":"172.21.15.100:6821/1410593862","heartbeat_back_addr":"172.21.15.100:6825/1410593862","heartbeat_front_addr":"172.21.15.100:6823/1410593862","state":["exists","up"]},{"osd":7,"uuid":"02ada28f-ac4f-47cf-983f-8a6da1140abe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2458690455},{"type":"v1","addr":"172.21.15.178:6817","nonce":2458690455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2458690455},{"type":"v1","addr":"172.21.15.178:6819","nonce":2458690455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2458690455},{"type":"v1","addr":"172.21.15.178:6823","nonce":2458690455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2458690455},{"type":"v1","addr":"172.21.15.178:6821","nonce":2458690455}]},"public_addr":"172.21.15.178:6817/2458690455","cluster_addr":"172.21.15.178:6819/2458690455","heartbeat_back_addr":"172.21.15.178:6823/2458690455","heartbeat_front_addr":"172.21.15.178:6821/2458690455","state":["exists","up"]},{"osd":8,"uuid":"e7e489bc-15fc-4a4c-b297-d1bb626513ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1523872826},{"type":"v1","addr":"172.21.15.188:6817","nonce":1523872826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1523872826},{"type":"v1","addr":"172.21.15.188:6819","nonce":1523872826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1523872826},{"type":"v1","addr":"172.21.15.188:6823","nonce":1523872826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1523872826},{"type":"v1","addr":"172.21.15.188:6821","nonce":1523872826}]},"public_addr":"172.21.15.188:6817/1523872826","cluster_addr":"172.21.15.188:6819/1523872826","heartbeat_back_addr":"172.21.15.188:6823/1523872826","heartbeat_front_addr":"172.21.15.188:6821/1523872826","state":["exists","up"]},{"osd":9,"uuid":"72328326-cd16-45d9-bfd9-40f6c5c20f5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3585761978},{"type":"v1","addr":"172.21.15.100:6827","nonce":3585761978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3585761978},{"type":"v1","addr":"172.21.15.100:6829","nonce":3585761978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3585761978},{"type":"v1","addr":"172.21.15.100:6833","nonce":3585761978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3585761978},{"type":"v1","addr":"172.21.15.100:6831","nonce":3585761978}]},"public_addr":"172.21.15.100:6827/3585761978","cluster_addr":"172.21.15.100:6829/3585761978","heartbeat_back_addr":"172.21.15.100:6833/3585761978","heartbeat_front_addr":"172.21.15.100:6831/3585761978","state":["exists","up"]},{"osd":10,"uuid":"a355f9a8-91a3-41d4-814d-5a9c9b953706","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2985841725},{"type":"v1","addr":"172.21.15.178:6825","nonce":2985841725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2985841725},{"type":"v1","addr":"172.21.15.178:6827","nonce":2985841725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2985841725},{"type":"v1","addr":"172.21.15.178:6831","nonce":2985841725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2985841725},{"type":"v1","addr":"172.21.15.178:6829","nonce":2985841725}]},"public_addr":"172.21.15.178:6825/2985841725","cluster_addr":"172.21.15.178:6827/2985841725","heartbeat_back_addr":"172.21.15.178:6831/2985841725","heartbeat_front_addr":"172.21.15.178:6829/2985841725","state":["exists","up"]},{"osd":11,"uuid":"76a5e13c-7e38-42f9-bfa7-7fedec7c7184","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2574955118},{"type":"v1","addr":"172.21.15.188:6825","nonce":2574955118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2574955118},{"type":"v1","addr":"172.21.15.188:6827","nonce":2574955118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2574955118},{"type":"v1","addr":"172.21.15.188:6831","nonce":2574955118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2574955118},{"type":"v1","addr":"172.21.15.188:6829","nonce":2574955118}]},"public_addr":"172.21.15.188:6825/2574955118","cluster_addr":"172.21.15.188:6827/2574955118","heartbeat_back_addr":"172.21.15.188:6831/2574955118","heartbeat_front_addr":"172.21.15.188:6829/2574955118","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:08:36.086637+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:08:58.948031+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:09:22.142517+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:09:47.655748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:10:12.853999+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:10:37.911383+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:11:04.448815+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:11:30.331740+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:11:55.819278+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:12:23.204612+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:12:49.484835+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:13:15.783531+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:6800/1597433518":"2024-05-11T11:07:00.092732+0000","172.21.15.100:6800/1086450256":"2024-05-11T11:06:33.114176+0000","172.21.15.100:0/767964410":"2024-05-11T11:07:00.092732+0000","172.21.15.100:6801/1597433518":"2024-05-11T11:07:00.092732+0000","172.21.15.100:6801/1086450256":"2024-05-11T11:06:33.114176+0000","172.21.15.100:0/2770572070":"2024-05-11T11:06:33.114176+0000","172.21.15.100:0/1379340513":"2024-05-11T11:06:33.114176+0000","172.21.15.100:0/3025646953":"2024-05-11T11:07:00.092732+0000","172.21.15.100:0/4099258930":"2024-05-11T11:07:00.092732+0000","172.21.15.100:0/4267693567":"2024-05-11T11:06:33.114176+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T11:14:56.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:56.913+0000 7f1c51dd3700 1 -- 172.21.15.100:0/671402890 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f1c380b2880 msgr2=0x7f1c380b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:56.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:56.913+0000 7f1c51dd3700 1 --2- 172.21.15.100:0/671402890 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f1c380b2880 0x7f1c380b4d40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c009a10 tx=0x7f1c3c009190 comp rx=0 tx=0).stop 2024-05-10T11:14:56.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:56.913+0000 7f1c51dd3700 1 -- 172.21.15.100:0/671402890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c4c102d60 msgr2=0x7f1c4c19ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:56.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:56.913+0000 7f1c51dd3700 1 --2- 172.21.15.100:0/671402890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c4c102d60 0x7f1c4c19ae50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1c34000c00 tx=0x7f1c340047b0 comp rx=0 tx=0).stop 2024-05-10T11:14:56.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:56.913+0000 7f1c51dd3700 1 -- 172.21.15.100:0/671402890 shutdown_connections 2024-05-10T11:14:56.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:56.913+0000 7f1c51dd3700 1 --2- 172.21.15.100:0/671402890 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f1c380b2880 0x7f1c380b4d40 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:56.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:56.913+0000 7f1c51dd3700 1 --2- 172.21.15.100:0/671402890 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c4c108410 0x7f1c4c1a5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:56.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:56.913+0000 7f1c51dd3700 1 --2- 172.21.15.100:0/671402890 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c4c1023a0 0x7f1c4c19a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:56.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:56.913+0000 7f1c51dd3700 1 --2- 172.21.15.100:0/671402890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c4c102d60 0x7f1c4c19ae50 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:56.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:56.913+0000 7f1c51dd3700 1 -- 172.21.15.100:0/671402890 >> 172.21.15.100:0/671402890 conn(0x7f1c4c079350 msgr2=0x7f1c4c1011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:14:56.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:56.913+0000 7f1c51dd3700 1 -- 172.21.15.100:0/671402890 shutdown_connections 2024-05-10T11:14:56.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:56.913+0000 7f1c51dd3700 1 -- 172.21.15.100:0/671402890 wait complete. 2024-05-10T11:14:57.512 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:57 smithi100 ceph-mon[30296]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:57.512 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:57 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/671402890' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T11:14:57.540 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-10T11:14:27.242452+0000', 'modified': '2024-05-10T11:14:41.399890+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24439}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24439': {'gid': 24439, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.188:6839/3194917070', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6838', 'nonce': 3194917070}, {'type': 'v1', 'addr': '172.21.15.188:6839', 'nonce': 3194917070}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24439, 'qdb_cluster': [24439]} 2024-05-10T11:14:57.540 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-10T11:14:57.540 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.i status 2024-05-10T11:14:57.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:57 smithi178 ceph-mon[33359]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:57.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:57 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/671402890' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T11:14:57.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:57 smithi188 ceph-mon[34075]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:57.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:57 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/671402890' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T11:14:57.780 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:14:59.475 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:14:59 smithi100 ceph-mon[30296]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:59.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.581+0000 7f3bcb3bd700 1 -- 172.21.15.100:0/517585611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bc4104100 msgr2=0x7f3bc4108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:59.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.581+0000 7f3bcb3bd700 1 --2- 172.21.15.100:0/517585611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bc4104100 0x7f3bc4108120 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3bb40099c0 tx=0x7f3bb4009cd0 comp rx=0 tx=0).stop 2024-05-10T11:14:59.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.581+0000 7f3bcb3bd700 1 -- 172.21.15.100:0/517585611 shutdown_connections 2024-05-10T11:14:59.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.581+0000 7f3bcb3bd700 1 --2- 172.21.15.100:0/517585611 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3bc4108850 0x7f3bc41103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:59.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.581+0000 7f3bcb3bd700 1 --2- 172.21.15.100:0/517585611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3bc4103750 0x7f3bc4103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:59.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.581+0000 7f3bcb3bd700 1 --2- 172.21.15.100:0/517585611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bc4104100 0x7f3bc4108120 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:59.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.581+0000 7f3bcb3bd700 1 -- 172.21.15.100:0/517585611 >> 172.21.15.100:0/517585611 conn(0x7f3bc407a600 msgr2=0x7f3bc407aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:14:59.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.581+0000 7f3bcb3bd700 1 -- 172.21.15.100:0/517585611 shutdown_connections 2024-05-10T11:14:59.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.581+0000 7f3bcb3bd700 1 -- 172.21.15.100:0/517585611 wait complete. 2024-05-10T11:14:59.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.582+0000 7f3bcb3bd700 1 Processor -- start 2024-05-10T11:14:59.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.582+0000 7f3bcb3bd700 1 -- start start 2024-05-10T11:14:59.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.582+0000 7f3bcb3bd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3bc4103750 0x7f3bc419f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:59.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.582+0000 7f3bcb3bd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bc4104100 0x7f3bc419fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:59.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.582+0000 7f3bcb3bd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3bc4108850 0x7f3bc41997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:59.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.582+0000 7f3bcb3bd700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc41134c0 con 0x7f3bc4104100 2024-05-10T11:14:59.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.582+0000 7f3bcb3bd700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc4113340 con 0x7f3bc4103750 2024-05-10T11:14:59.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.582+0000 7f3bcb3bd700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc4113640 con 0x7f3bc4108850 2024-05-10T11:14:59.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.582+0000 7f3bc8958700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bc4104100 0x7f3bc419fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:59.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.582+0000 7f3bc8958700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bc4104100 0x7f3bc419fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50396/0 (socket says 172.21.15.100:50396) 2024-05-10T11:14:59.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.582+0000 7f3bc8958700 1 -- 172.21.15.100:0/3928899077 learned_addr learned my addr 172.21.15.100:0/3928899077 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:14:59.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.582+0000 7f3bc9159700 1 --2- 172.21.15.100:0/3928899077 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3bc4103750 0x7f3bc419f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:59.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.582+0000 7f3bc995a700 1 --2- 172.21.15.100:0/3928899077 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3bc4108850 0x7f3bc41997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:59.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.582+0000 7f3bc8958700 1 -- 172.21.15.100:0/3928899077 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3bc4108850 msgr2=0x7f3bc41997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:59.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.582+0000 7f3bc8958700 1 --2- 172.21.15.100:0/3928899077 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3bc4108850 0x7f3bc41997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:59.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.582+0000 7f3bc8958700 1 -- 172.21.15.100:0/3928899077 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3bc4103750 msgr2=0x7f3bc419f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:59.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.582+0000 7f3bc8958700 1 --2- 172.21.15.100:0/3928899077 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3bc4103750 0x7f3bc419f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:59.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.582+0000 7f3bc8958700 1 -- 172.21.15.100:0/3928899077 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bc4199fc0 con 0x7f3bc4104100 2024-05-10T11:14:59.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.582+0000 7f3bc995a700 1 --2- 172.21.15.100:0/3928899077 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3bc4108850 0x7f3bc41997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:14:59.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.582+0000 7f3bc8958700 1 --2- 172.21.15.100:0/3928899077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bc4104100 0x7f3bc419fba0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4005f50 tx=0x7f3bb40057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:14:59.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.583+0000 7f3bba7fc700 1 -- 172.21.15.100:0/3928899077 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb4027070 con 0x7f3bc4104100 2024-05-10T11:14:59.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.583+0000 7f3bcb3bd700 1 -- 172.21.15.100:0/3928899077 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bc419a250 con 0x7f3bc4104100 2024-05-10T11:14:59.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.583+0000 7f3bba7fc700 1 -- 172.21.15.100:0/3928899077 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3bb400fd60 con 0x7f3bc4104100 2024-05-10T11:14:59.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.583+0000 7f3bba7fc700 1 -- 172.21.15.100:0/3928899077 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb4022e30 con 0x7f3bc4104100 2024-05-10T11:14:59.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.583+0000 7f3bcb3bd700 1 -- 172.21.15.100:0/3928899077 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bc41a64c0 con 0x7f3bc4104100 2024-05-10T11:14:59.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.585+0000 7f3bba7fc700 1 -- 172.21.15.100:0/3928899077 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f3bb402a070 con 0x7f3bc4104100 2024-05-10T11:14:59.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.586+0000 7f3bba7fc700 1 --2- 172.21.15.100:0/3928899077 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f3bb00b2880 0x7f3bb00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:59.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.586+0000 7f3bc9159700 1 --2- 172.21.15.100:0/3928899077 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f3bb00b2880 0x7f3bb00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:59.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.586+0000 7f3bba7fc700 1 -- 172.21.15.100:0/3928899077 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f3bb40e30e0 con 0x7f3bc4104100 2024-05-10T11:14:59.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.586+0000 7f3bd5c50b80 1 Processor -- start 2024-05-10T11:14:59.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.587+0000 7f3bc9159700 1 --2- 172.21.15.100:0/3928899077 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f3bb00b2880 0x7f3bb00b4d40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3bc0005950 tx=0x7f3bc000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:14:59.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.587+0000 7f3bd5c50b80 1 -- start start 2024-05-10T11:14:59.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.587+0000 7f3bd5c50b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fa4c860160 0x55fa4c860540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:59.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.587+0000 7f3bd5c50b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa4c860a80 0x55fa4c85a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:59.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.587+0000 7f3bd5c50b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa4c85ad00 0x55fa4c85b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:59.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.587+0000 7f3bd5c50b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55fa4c7ac8f0 con 0x55fa4c860a80 2024-05-10T11:14:59.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.587+0000 7f3bd5c50b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55fa4c711d50 con 0x55fa4c85ad00 2024-05-10T11:14:59.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.587+0000 7f3bd5c50b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55fa4c7a88b0 con 0x55fa4c860160 2024-05-10T11:14:59.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.587+0000 7f3bc8958700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa4c860a80 0x55fa4c85a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:59.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.587+0000 7f3bc8958700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa4c860a80 0x55fa4c85a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50398/0 (socket says 172.21.15.100:50398) 2024-05-10T11:14:59.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.587+0000 7f3bc8958700 1 -- 172.21.15.100:0/15584783 learned_addr learned my addr 172.21.15.100:0/15584783 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:14:59.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.587+0000 7f3bc9159700 1 --2- 172.21.15.100:0/15584783 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fa4c860160 0x55fa4c860540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:59.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.587+0000 7f3bc995a700 1 --2- 172.21.15.100:0/15584783 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa4c85ad00 0x55fa4c85b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:59.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.587+0000 7f3bc8958700 1 -- 172.21.15.100:0/15584783 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fa4c860160 msgr2=0x55fa4c860540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:14:59.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.587+0000 7f3bc8958700 1 --2- 172.21.15.100:0/15584783 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fa4c860160 0x55fa4c860540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:59.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.587+0000 7f3bc8958700 1 -- 172.21.15.100:0/15584783 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa4c85ad00 msgr2=0x55fa4c85b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:14:59.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.587+0000 7f3bc8958700 1 --2- 172.21.15.100:0/15584783 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa4c85ad00 0x55fa4c85b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:59.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.587+0000 7f3bc8958700 1 -- 172.21.15.100:0/15584783 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa4c6f9880 con 0x55fa4c860a80 2024-05-10T11:14:59.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.587+0000 7f3bc8958700 1 --2- 172.21.15.100:0/15584783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa4c860a80 0x55fa4c85a7c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3bb40aa250 tx=0x7f3bb400ff10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcc8b4f2069f96e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:14:59.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.587+0000 7f3baf7fe700 1 -- 172.21.15.100:0/15584783 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb40e0e60 con 0x55fa4c860a80 2024-05-10T11:14:59.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.588+0000 7f3baf7fe700 1 -- 172.21.15.100:0/15584783 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3bb40aa060 con 0x55fa4c860a80 2024-05-10T11:14:59.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.588+0000 7f3baf7fe700 1 -- 172.21.15.100:0/15584783 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb40aa7b0 con 0x55fa4c860a80 2024-05-10T11:14:59.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.588+0000 7f3bd5c50b80 1 -- 172.21.15.100:0/15584783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa4c860a80 msgr2=0x55fa4c85a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:59.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.588+0000 7f3bd5c50b80 1 --2- 172.21.15.100:0/15584783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa4c860a80 0x55fa4c85a7c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3bb40aa250 tx=0x7f3bb400ff10 comp rx=0 tx=0).stop 2024-05-10T11:14:59.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.588+0000 7f3bd5c50b80 1 -- 172.21.15.100:0/15584783 shutdown_connections 2024-05-10T11:14:59.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.588+0000 7f3bd5c50b80 1 --2- 172.21.15.100:0/15584783 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa4c85ad00 0x55fa4c85b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:59.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.588+0000 7f3bd5c50b80 1 --2- 172.21.15.100:0/15584783 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fa4c860160 0x55fa4c860540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:59.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.588+0000 7f3bd5c50b80 1 --2- 172.21.15.100:0/15584783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa4c860a80 0x55fa4c85a7c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:59.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.588+0000 7f3bd5c50b80 1 -- 172.21.15.100:0/15584783 >> 172.21.15.100:0/15584783 conn(0x55fa4c786b20 msgr2=0x55fa4c855250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:14:59.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.588+0000 7f3bd5c50b80 1 -- 172.21.15.100:0/15584783 shutdown_connections 2024-05-10T11:14:59.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.588+0000 7f3bd5c50b80 1 -- 172.21.15.100:0/15584783 wait complete. 2024-05-10T11:14:59.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.589+0000 7f3bd5c50b80 1 -- start start 2024-05-10T11:14:59.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.589+0000 7f3bd5c50b80 1 Processor -- start 2024-05-10T11:14:59.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.589+0000 7f3bd5c50b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fa4c85ad00 0x55fa4c866720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:59.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.589+0000 7f3bd5c50b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa4c860160 0x55fa4c863ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:59.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.589+0000 7f3bd5c50b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa4c860a80 0x55fa4c864000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:59.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.589+0000 7f3bd5c50b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55fa4c7ac8f0 con 0x55fa4c860160 2024-05-10T11:14:59.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.589+0000 7f3bd5c50b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55fa4c765e80 con 0x55fa4c860a80 2024-05-10T11:14:59.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.589+0000 7f3bd5c50b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55fa4c8671c0 con 0x55fa4c85ad00 2024-05-10T11:14:59.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.589+0000 7f3bd5c50b80 10 client.0 _ll_register_callbacks cb 0x55fa4c740560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:14:59.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.589+0000 7f3bc8958700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa4c860160 0x55fa4c863ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:59.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.589+0000 7f3bc8958700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa4c860160 0x55fa4c863ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50404/0 (socket says 172.21.15.100:50404) 2024-05-10T11:14:59.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.589+0000 7f3bc8958700 1 -- 172.21.15.100:0/2581335624 learned_addr learned my addr 172.21.15.100:0/2581335624 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:14:59.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.589+0000 7f3bc9159700 1 --2- 172.21.15.100:0/2581335624 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fa4c85ad00 0x55fa4c866720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:59.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.589+0000 7f3bc995a700 1 --2- 172.21.15.100:0/2581335624 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa4c860a80 0x55fa4c864000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:59.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.590+0000 7f3bc8958700 1 -- 172.21.15.100:0/2581335624 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fa4c85ad00 msgr2=0x55fa4c866720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:14:59.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.590+0000 7f3bc8958700 1 --2- 172.21.15.100:0/2581335624 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fa4c85ad00 0x55fa4c866720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:59.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.590+0000 7f3bc8958700 1 -- 172.21.15.100:0/2581335624 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa4c860a80 msgr2=0x55fa4c864000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:14:59.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.590+0000 7f3bc8958700 1 --2- 172.21.15.100:0/2581335624 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa4c860a80 0x55fa4c864000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:59.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.590+0000 7f3bc8958700 1 -- 172.21.15.100:0/2581335624 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa4c763710 con 0x55fa4c860160 2024-05-10T11:14:59.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.590+0000 7f3bc8958700 1 --2- 172.21.15.100:0/2581335624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa4c860160 0x55fa4c863ac0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3bb40e0d10 tx=0x7f3bb40aa550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1909715511b99ce7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:14:59.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.590+0000 7f3bacff9700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:14:59.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.590+0000 7f3bacff9700 1 -- 172.21.15.100:0/2581335624 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb40c4060 con 0x55fa4c860160 2024-05-10T11:14:59.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.590+0000 7f3bacff9700 1 -- 172.21.15.100:0/2581335624 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f3bb40b30a0 con 0x55fa4c860160 2024-05-10T11:14:59.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.590+0000 7f3bd5c50b80 1 -- 172.21.15.100:0/2581335624 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bb40aa060 con 0x55fa4c860160 2024-05-10T11:14:59.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.590+0000 7f3bacff9700 1 -- 172.21.15.100:0/2581335624 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f3bb40e0e60 con 0x55fa4c860160 2024-05-10T11:14:59.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.590+0000 7f3bd5c50b80 10 client.14691 fetch_fsmap learned FSMap version 17 2024-05-10T11:14:59.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.590+0000 7f3bd5c50b80 1 -- 172.21.15.100:0/2581335624 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55fa4c871920 con 0x55fa4c860160 2024-05-10T11:14:59.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.590+0000 7f3bacff9700 1 -- 172.21.15.100:0/2581335624 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3bb40aa060 con 0x55fa4c860160 2024-05-10T11:14:59.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.590+0000 7f3bacff9700 1 -- 172.21.15.100:0/2581335624 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7f3bb40a97b0 con 0x55fa4c860160 2024-05-10T11:14:59.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.590+0000 7f3bd5c50b80 10 client.14691 fetch_fsmap finished waiting for FSMap version 17 2024-05-10T11:14:59.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.590+0000 7f3bd5c50b80 10 client.14691 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-10T11:14:59.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.590+0000 7f3bd5c50b80 20 client.14691 populate_metadata read hostname 'smithi100' 2024-05-10T11:14:59.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.590+0000 7f3bd5c50b80 1 --2- 172.21.15.100:0/2581335624 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55fa4c873d50 0x55fa4c8751c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:59.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.590+0000 7f3bd5c50b80 4 client.14691 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:14:59.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.590+0000 7f3bd5c50b80 1 -- 172.21.15.100:0/2581335624 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa4c7ba2d0 con 0x55fa4c873d50 2024-05-10T11:14:59.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.591+0000 7f3bc9159700 1 --2- 172.21.15.100:0/2581335624 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55fa4c873d50 0x55fa4c8751c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:59.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.591+0000 7f3bacff9700 10 client.14691 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:14:59.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.591+0000 7f3bc9159700 1 --2- 172.21.15.100:0/2581335624 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55fa4c873d50 0x55fa4c8751c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9d2cf74f71c98cd server_cookie=f106138347c1b7e1 in_seq=0 out_seq=0 2024-05-10T11:14:59.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.592+0000 7f3bacff9700 1 -- 172.21.15.100:0/2581335624 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fa4c7ba2d0 con 0x55fa4c873d50 2024-05-10T11:14:59.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.592+0000 7f3bacff9700 10 client.14691 handle_command_reply: tid=0 2024-05-10T11:14:59.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.592+0000 7f3bd5c50b80 1 client.14691 shutdown 2024-05-10T11:14:59.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.592+0000 7f3bd5c50b80 1 -- 172.21.15.100:0/2581335624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa4c860160 msgr2=0x55fa4c863ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:59.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.592+0000 7f3bd5c50b80 1 --2- 172.21.15.100:0/2581335624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa4c860160 0x55fa4c863ac0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3bb40e0d10 tx=0x7f3bb40aa550 comp rx=0 tx=0).stop 2024-05-10T11:14:59.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.592+0000 7f3bd5c50b80 1 -- 172.21.15.100:0/2581335624 shutdown_connections 2024-05-10T11:14:59.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.592+0000 7f3bd5c50b80 1 --2- 172.21.15.100:0/2581335624 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55fa4c873d50 0x55fa4c8751c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:59.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.592+0000 7f3bd5c50b80 1 --2- 172.21.15.100:0/2581335624 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa4c860a80 0x55fa4c864000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:59.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.592+0000 7f3bd5c50b80 1 --2- 172.21.15.100:0/2581335624 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fa4c85ad00 0x55fa4c866720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:59.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.592+0000 7f3bd5c50b80 1 --2- 172.21.15.100:0/2581335624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa4c860160 0x55fa4c863ac0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:59.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.592+0000 7f3bd5c50b80 1 -- 172.21.15.100:0/2581335624 >> 172.21.15.100:0/2581335624 conn(0x55fa4c786b20 msgr2=0x55fa4c84fae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:14:59.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.592+0000 7f3bacff9700 0 client.14691 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:14:59.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.592+0000 7f3bd5c50b80 1 -- 172.21.15.100:0/2581335624 shutdown_connections 2024-05-10T11:14:59.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.592+0000 7f3bd5c50b80 1 -- 172.21.15.100:0/2581335624 wait complete. 2024-05-10T11:14:59.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.592+0000 7f3bd5c50b80 20 client.14691 trim_cache size 0 max 16384 2024-05-10T11:14:59.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.613+0000 7f3bd5c50b80 1 Processor -- start 2024-05-10T11:14:59.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.613+0000 7f3bd5c50b80 1 -- start start 2024-05-10T11:14:59.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.613+0000 7f3bd5c50b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa4c85ad00 0x55fa4c85a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:59.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.613+0000 7f3bd5c50b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fa4c860160 0x55fa4c84fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:59.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.613+0000 7f3bd5c50b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa4c860a80 0x55fa4c860e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:59.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.613+0000 7f3bd5c50b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55fa4c873500 con 0x55fa4c85ad00 2024-05-10T11:14:59.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.613+0000 7f3bd5c50b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55fa4c873bd0 con 0x55fa4c860a80 2024-05-10T11:14:59.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.613+0000 7f3bd5c50b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55fa4c8742a0 con 0x55fa4c860160 2024-05-10T11:14:59.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.613+0000 7f3bc9159700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa4c85ad00 0x55fa4c85a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:59.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.613+0000 7f3bc9159700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa4c85ad00 0x55fa4c85a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50408/0 (socket says 172.21.15.100:50408) 2024-05-10T11:14:59.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.613+0000 7f3bc9159700 1 -- 172.21.15.100:0/665685407 learned_addr learned my addr 172.21.15.100:0/665685407 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:14:59.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.614+0000 7f3bc995a700 1 --2- 172.21.15.100:0/665685407 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa4c860a80 0x55fa4c860e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:59.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.614+0000 7f3bc9159700 1 -- 172.21.15.100:0/665685407 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fa4c860160 msgr2=0x55fa4c84fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:14:59.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.614+0000 7f3bc9159700 1 --2- 172.21.15.100:0/665685407 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fa4c860160 0x55fa4c84fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:59.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.614+0000 7f3bc9159700 1 -- 172.21.15.100:0/665685407 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa4c860a80 msgr2=0x55fa4c860e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:14:59.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.614+0000 7f3bc9159700 1 --2- 172.21.15.100:0/665685407 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa4c860a80 0x55fa4c860e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:59.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.614+0000 7f3bc9159700 1 -- 172.21.15.100:0/665685407 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa4c862030 con 0x55fa4c85ad00 2024-05-10T11:14:59.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.614+0000 7f3bc9159700 1 --2- 172.21.15.100:0/665685407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa4c85ad00 0x55fa4c85a7c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3bc0010a40 tx=0x7f3bc0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7720b9a42eae80e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:14:59.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.614+0000 7f3baeffd700 1 -- 172.21.15.100:0/665685407 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc0021a40 con 0x55fa4c85ad00 2024-05-10T11:14:59.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.614+0000 7f3baeffd700 1 -- 172.21.15.100:0/665685407 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3bc0004cc0 con 0x55fa4c85ad00 2024-05-10T11:14:59.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.614+0000 7f3baeffd700 1 -- 172.21.15.100:0/665685407 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc002a7b0 con 0x55fa4c85ad00 2024-05-10T11:14:59.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.615+0000 7f3bd5c50b80 1 -- 172.21.15.100:0/665685407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa4c85ad00 msgr2=0x55fa4c85a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:59.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.615+0000 7f3bd5c50b80 1 --2- 172.21.15.100:0/665685407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa4c85ad00 0x55fa4c85a7c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3bc0010a40 tx=0x7f3bc0013000 comp rx=0 tx=0).stop 2024-05-10T11:14:59.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.615+0000 7f3bd5c50b80 1 -- 172.21.15.100:0/665685407 shutdown_connections 2024-05-10T11:14:59.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.615+0000 7f3bd5c50b80 1 --2- 172.21.15.100:0/665685407 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa4c860a80 0x55fa4c860e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:59.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.615+0000 7f3bd5c50b80 1 --2- 172.21.15.100:0/665685407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa4c85ad00 0x55fa4c85a7c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:59.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.615+0000 7f3bd5c50b80 1 --2- 172.21.15.100:0/665685407 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fa4c860160 0x55fa4c84fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:59.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.615+0000 7f3bd5c50b80 1 -- 172.21.15.100:0/665685407 >> 172.21.15.100:0/665685407 conn(0x55fa4c786b20 msgr2=0x55fa4c8623b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:14:59.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.615+0000 7f3bd5c50b80 1 -- 172.21.15.100:0/665685407 shutdown_connections 2024-05-10T11:14:59.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.615+0000 7f3bd5c50b80 1 -- 172.21.15.100:0/665685407 wait complete. 2024-05-10T11:14:59.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.615+0000 7f3bd5c50b80 1 -- start start 2024-05-10T11:14:59.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.615+0000 7f3bd5c50b80 1 Processor -- start 2024-05-10T11:14:59.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.615+0000 7f3bd5c50b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fa4c85ad00 0x55fa4c84efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:59.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.615+0000 7f3bd5c50b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa4c860160 0x55fa4c84f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:59.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.615+0000 7f3bd5c50b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa4c860a80 0x55fa4c84fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:59.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.615+0000 7f3bd5c50b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55fa4c875040 con 0x55fa4c860160 2024-05-10T11:14:59.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.615+0000 7f3bd5c50b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55fa4c874970 con 0x55fa4c860a80 2024-05-10T11:14:59.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.615+0000 7f3bd5c50b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55fa4c875710 con 0x55fa4c85ad00 2024-05-10T11:14:59.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.615+0000 7f3bd5c50b80 10 client.0 _ll_register_callbacks cb 0x55fa4c740560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:14:59.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.616+0000 7f3bc8958700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa4c860160 0x55fa4c84f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:59.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.616+0000 7f3bc8958700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa4c860160 0x55fa4c84f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50412/0 (socket says 172.21.15.100:50412) 2024-05-10T11:14:59.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.616+0000 7f3bc8958700 1 -- 172.21.15.100:0/3610266855 learned_addr learned my addr 172.21.15.100:0/3610266855 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:14:59.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.616+0000 7f3bc995a700 1 --2- 172.21.15.100:0/3610266855 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa4c860a80 0x55fa4c84fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:59.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.616+0000 7f3bc9159700 1 --2- 172.21.15.100:0/3610266855 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fa4c85ad00 0x55fa4c84efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:59.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.616+0000 7f3bc8958700 1 -- 172.21.15.100:0/3610266855 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fa4c85ad00 msgr2=0x55fa4c84efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:14:59.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.616+0000 7f3bc8958700 1 --2- 172.21.15.100:0/3610266855 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fa4c85ad00 0x55fa4c84efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:59.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.616+0000 7f3bc8958700 1 -- 172.21.15.100:0/3610266855 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa4c860a80 msgr2=0x55fa4c84fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:14:59.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.616+0000 7f3bc8958700 1 --2- 172.21.15.100:0/3610266855 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa4c860a80 0x55fa4c84fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:59.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.616+0000 7f3bc8958700 1 -- 172.21.15.100:0/3610266855 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa4c87c8b0 con 0x55fa4c860160 2024-05-10T11:14:59.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.616+0000 7f3bc995a700 1 --2- 172.21.15.100:0/3610266855 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa4c860a80 0x55fa4c84fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:14:59.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.616+0000 7f3bc8958700 1 --2- 172.21.15.100:0/3610266855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa4c860160 0x55fa4c84f4e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3bb40aa250 tx=0x7f3bb4009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e15fc1cc25dd2248 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:14:59.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.616+0000 7f3bacff9700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:14:59.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.616+0000 7f3bacff9700 1 -- 172.21.15.100:0/3610266855 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb40c4040 con 0x55fa4c860160 2024-05-10T11:14:59.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.616+0000 7f3bacff9700 1 -- 172.21.15.100:0/3610266855 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f3bb40050f0 con 0x55fa4c860160 2024-05-10T11:14:59.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.616+0000 7f3bd5c50b80 1 -- 172.21.15.100:0/3610266855 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bc0004cc0 con 0x55fa4c860160 2024-05-10T11:14:59.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.616+0000 7f3bacff9700 1 -- 172.21.15.100:0/3610266855 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f3bb40a9ea0 con 0x55fa4c860160 2024-05-10T11:14:59.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.616+0000 7f3bd5c50b80 10 client.14697 fetch_fsmap learned FSMap version 17 2024-05-10T11:14:59.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.616+0000 7f3bd5c50b80 1 -- 172.21.15.100:0/3610266855 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55fa4c87c8b0 con 0x55fa4c860160 2024-05-10T11:14:59.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.616+0000 7f3bacff9700 1 -- 172.21.15.100:0/3610266855 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3bc0004cc0 con 0x55fa4c860160 2024-05-10T11:14:59.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.617+0000 7f3bacff9700 1 -- 172.21.15.100:0/3610266855 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7f3bb40a97b0 con 0x55fa4c860160 2024-05-10T11:14:59.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.617+0000 7f3bd5c50b80 10 client.14697 fetch_fsmap finished waiting for FSMap version 17 2024-05-10T11:14:59.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.617+0000 7f3bd5c50b80 10 client.14697 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-10T11:14:59.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.617+0000 7f3bd5c50b80 20 client.14697 populate_metadata read hostname 'smithi100' 2024-05-10T11:14:59.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.617+0000 7f3bd5c50b80 1 --2- 172.21.15.100:0/3610266855 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55fa4c8e3c10 0x55fa4c8e2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:14:59.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.617+0000 7f3bd5c50b80 4 client.14697 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T11:14:59.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.617+0000 7f3bd5c50b80 1 -- 172.21.15.100:0/3610266855 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "status"}) -- 0x55fa4c7b4370 con 0x55fa4c8e3c10 2024-05-10T11:14:59.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.617+0000 7f3bc9159700 1 --2- 172.21.15.100:0/3610266855 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55fa4c8e3c10 0x55fa4c8e2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:14:59.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.617+0000 7f3bc9159700 1 --2- 172.21.15.100:0/3610266855 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55fa4c8e3c10 0x55fa4c8e2920 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d612a81b6d72f5e server_cookie=305405f077547de4 in_seq=0 out_seq=0 2024-05-10T11:14:59.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.617+0000 7f3bacff9700 10 client.14697 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:14:59.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.618+0000 7f3bacff9700 1 -- 172.21.15.100:0/3610266855 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55fa4c7b4370 con 0x55fa4c8e3c10 2024-05-10T11:14:59.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.618+0000 7f3bacff9700 10 client.14697 handle_command_reply: tid=0 2024-05-10T11:14:59.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.618+0000 7f3bd5c50b80 1 client.14697 shutdown 2024-05-10T11:14:59.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.618+0000 7f3bd5c50b80 1 -- 172.21.15.100:0/3610266855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa4c860160 msgr2=0x55fa4c84f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:59.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.618+0000 7f3bd5c50b80 1 --2- 172.21.15.100:0/3610266855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa4c860160 0x55fa4c84f4e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3bb40aa250 tx=0x7f3bb4009f50 comp rx=0 tx=0).stop 2024-05-10T11:14:59.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.618+0000 7f3bd5c50b80 1 -- 172.21.15.100:0/3610266855 shutdown_connections 2024-05-10T11:14:59.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.618+0000 7f3bd5c50b80 1 --2- 172.21.15.100:0/3610266855 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55fa4c8e3c10 0x55fa4c8e2920 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:59.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.618+0000 7f3bd5c50b80 1 --2- 172.21.15.100:0/3610266855 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa4c860a80 0x55fa4c84fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:59.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.618+0000 7f3bd5c50b80 1 --2- 172.21.15.100:0/3610266855 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fa4c85ad00 0x55fa4c84efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:59.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.618+0000 7f3bacff9700 0 client.14697 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:14:59.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.618+0000 7f3bd5c50b80 1 --2- 172.21.15.100:0/3610266855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa4c860160 0x55fa4c84f4e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:59.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.618+0000 7f3bd5c50b80 1 -- 172.21.15.100:0/3610266855 >> 172.21.15.100:0/3610266855 conn(0x55fa4c786b20 msgr2=0x55fa4c860e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:14:59.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.618+0000 7f3bd5c50b80 1 -- 172.21.15.100:0/3610266855 shutdown_connections 2024-05-10T11:14:59.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.618+0000 7f3bd5c50b80 1 -- 172.21.15.100:0/3610266855 wait complete. 2024-05-10T11:14:59.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.618+0000 7f3bd5c50b80 20 client.14697 trim_cache size 0 max 16384 2024-05-10T11:14:59.625 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:14:59.625 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_fsid": "20a542cc-0ebd-11ef-bc98-c7b262605968", 2024-05-10T11:14:59.625 INFO:teuthology.orchestra.run.smithi100.stdout: "whoami": 0, 2024-05-10T11:14:59.625 INFO:teuthology.orchestra.run.smithi100.stdout: "id": 24439, 2024-05-10T11:14:59.625 INFO:teuthology.orchestra.run.smithi100.stdout: "want_state": "up:active", 2024-05-10T11:14:59.625 INFO:teuthology.orchestra.run.smithi100.stdout: "state": "up:active", 2024-05-10T11:14:59.625 INFO:teuthology.orchestra.run.smithi100.stdout: "fs_name": "cephfs", 2024-05-10T11:14:59.625 INFO:teuthology.orchestra.run.smithi100.stdout: "rank_uptime": 32.345991781000002, 2024-05-10T11:14:59.625 INFO:teuthology.orchestra.run.smithi100.stdout: "mdsmap_epoch": 17, 2024-05-10T11:14:59.625 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch": 87, 2024-05-10T11:14:59.625 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch_barrier": 87, 2024-05-10T11:14:59.625 INFO:teuthology.orchestra.run.smithi100.stdout: "uptime": 60.103921096000001 2024-05-10T11:14:59.626 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:14:59.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.618+0000 7f3bcb3bd700 1 -- 172.21.15.100:0/3928899077 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f3bb00b2880 msgr2=0x7f3bb00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:59.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.618+0000 7f3bcb3bd700 1 --2- 172.21.15.100:0/3928899077 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f3bb00b2880 0x7f3bb00b4d40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3bc0005950 tx=0x7f3bc000b410 comp rx=0 tx=0).stop 2024-05-10T11:14:59.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.618+0000 7f3bcb3bd700 1 -- 172.21.15.100:0/3928899077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bc4104100 msgr2=0x7f3bc419fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:14:59.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.618+0000 7f3bcb3bd700 1 --2- 172.21.15.100:0/3928899077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bc4104100 0x7f3bc419fba0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4005f50 tx=0x7f3bb40057c0 comp rx=0 tx=0).stop 2024-05-10T11:14:59.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.619+0000 7f3bcb3bd700 1 -- 172.21.15.100:0/3928899077 shutdown_connections 2024-05-10T11:14:59.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.619+0000 7f3bcb3bd700 1 --2- 172.21.15.100:0/3928899077 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f3bb00b2880 0x7f3bb00b4d40 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:59.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.619+0000 7f3bcb3bd700 1 --2- 172.21.15.100:0/3928899077 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3bc4108850 0x7f3bc41997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:59.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.619+0000 7f3bcb3bd700 1 --2- 172.21.15.100:0/3928899077 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3bc4103750 0x7f3bc419f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:59.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.619+0000 7f3bcb3bd700 1 --2- 172.21.15.100:0/3928899077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bc4104100 0x7f3bc419fba0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:14:59.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.619+0000 7f3bcb3bd700 1 -- 172.21.15.100:0/3928899077 >> 172.21.15.100:0/3928899077 conn(0x7f3bc407a600 msgr2=0x7f3bc4101930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:14:59.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.619+0000 7f3bcb3bd700 1 -- 172.21.15.100:0/3928899077 shutdown_connections 2024-05-10T11:14:59.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:14:59.619+0000 7f3bcb3bd700 1 -- 172.21.15.100:0/3928899077 wait complete. 2024-05-10T11:14:59.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:14:59 smithi178 ceph-mon[33359]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:14:59.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:14:59 smithi188 ceph-mon[34075]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:15:00.440 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-10T11:15:00.440 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-10T11:15:00.441 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph fs ls 2024-05-10T11:15:00.681 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:15:00.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:00 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-05-10T11:15:00.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:00 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-05-10T11:15:00.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:00 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-05-10T11:15:00.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:00 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-05-10T11:15:00.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:00 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-05-10T11:15:00.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:00 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-05-10T11:15:00.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:00 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 8]}]: dispatch 2024-05-10T11:15:00.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:00 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-05-10T11:15:00.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:00 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 10]}]: dispatch 2024-05-10T11:15:00.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:00 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-05-10T11:15:00.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:00 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-05-10T11:15:00.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:00 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-05-10T11:15:00.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:00 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-05-10T11:15:00.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:00 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-05-10T11:15:00.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:00 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-05-10T11:15:00.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:00 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-05-10T11:15:00.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:00 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 8]}]: dispatch 2024-05-10T11:15:00.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:00 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-05-10T11:15:00.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:00 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 10]}]: dispatch 2024-05-10T11:15:00.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:00 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-05-10T11:15:00.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:00 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-05-10T11:15:00.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:00 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-05-10T11:15:00.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:00 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-05-10T11:15:00.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:00 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-05-10T11:15:00.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:00 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-05-10T11:15:00.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:00 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-05-10T11:15:00.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:00 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 8]}]: dispatch 2024-05-10T11:15:00.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:00 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-05-10T11:15:00.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:00 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 10]}]: dispatch 2024-05-10T11:15:00.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:00 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-05-10T11:15:01.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:01 smithi178 ceph-mon[33359]: pgmap v317: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:15:01.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:01 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-05-10T11:15:01.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:01 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-05-10T11:15:01.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:01 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-05-10T11:15:01.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:01 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-05-10T11:15:01.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:01 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-05-10T11:15:01.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:01 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-05-10T11:15:01.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:01 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 8]}]': finished 2024-05-10T11:15:01.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:01 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-05-10T11:15:01.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:01 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 10]}]': finished 2024-05-10T11:15:01.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:01 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-05-10T11:15:01.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:01 smithi178 ceph-mon[33359]: osdmap e88: 12 total, 12 up, 12 in 2024-05-10T11:15:01.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:01 smithi188 ceph-mon[34075]: pgmap v317: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:15:01.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:01 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-05-10T11:15:01.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:01 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-05-10T11:15:01.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:01 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-05-10T11:15:01.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:01 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-05-10T11:15:01.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:01 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-05-10T11:15:01.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:01 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-05-10T11:15:01.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:01 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 8]}]': finished 2024-05-10T11:15:01.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:01 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-05-10T11:15:01.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:01 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 10]}]': finished 2024-05-10T11:15:01.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:01 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-05-10T11:15:01.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:01 smithi188 ceph-mon[34075]: osdmap e88: 12 total, 12 up, 12 in 2024-05-10T11:15:01.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:01 smithi100 ceph-mon[30296]: pgmap v317: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:15:01.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:01 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-05-10T11:15:01.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:01 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-05-10T11:15:01.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:01 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-05-10T11:15:01.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:01 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-05-10T11:15:01.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:01 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-05-10T11:15:01.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:01 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-05-10T11:15:01.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:01 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 8]}]': finished 2024-05-10T11:15:01.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:01 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-05-10T11:15:01.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:01 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 10]}]': finished 2024-05-10T11:15:01.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:01 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-05-10T11:15:01.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:01 smithi100 ceph-mon[30296]: osdmap e88: 12 total, 12 up, 12 in 2024-05-10T11:15:02.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:02.385+0000 7efe26dc1700 1 -- 172.21.15.100:0/833868139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe2006c820 msgr2=0x7efe201080e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:02.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:02.385+0000 7efe26dc1700 1 --2- 172.21.15.100:0/833868139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe2006c820 0x7efe201080e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7efe1000b0d0 tx=0x7efe1000b3e0 comp rx=0 tx=0).stop 2024-05-10T11:15:02.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:02.385+0000 7efe26dc1700 1 -- 172.21.15.100:0/833868139 shutdown_connections 2024-05-10T11:15:02.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:02.385+0000 7efe26dc1700 1 --2- 172.21.15.100:0/833868139 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe201087c0 0x7efe201103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:02.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:02.385+0000 7efe26dc1700 1 --2- 172.21.15.100:0/833868139 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe2006be70 0x7efe2006c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:02.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:02.385+0000 7efe26dc1700 1 --2- 172.21.15.100:0/833868139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe2006c820 0x7efe201080e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:02.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:02.385+0000 7efe26dc1700 1 -- 172.21.15.100:0/833868139 >> 172.21.15.100:0/833868139 conn(0x7efe200783f0 msgr2=0x7efe20078800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:15:02.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:02.385+0000 7efe26dc1700 1 -- 172.21.15.100:0/833868139 shutdown_connections 2024-05-10T11:15:02.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:02.385+0000 7efe26dc1700 1 -- 172.21.15.100:0/833868139 wait complete. 2024-05-10T11:15:02.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:02.385+0000 7efe26dc1700 1 Processor -- start 2024-05-10T11:15:02.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:02.385+0000 7efe26dc1700 1 -- start start 2024-05-10T11:15:02.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:02.386+0000 7efe26dc1700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe2006be70 0x7efe20102500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:15:02.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:02.386+0000 7efe26dc1700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe2006c820 0x7efe20102a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:15:02.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:02.386+0000 7efe26dc1700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe201087c0 0x7efe20102f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:15:02.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:02.386+0000 7efe26dc1700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7efe20113620 con 0x7efe2006c820 2024-05-10T11:15:02.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:02.386+0000 7efe26dc1700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efe201134a0 con 0x7efe2006be70 2024-05-10T11:15:02.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:02.386+0000 7efe26dc1700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7efe201137a0 con 0x7efe201087c0 2024-05-10T11:15:02.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:02.386+0000 7efe1ffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe2006c820 0x7efe20102a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:15:02.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:02.386+0000 7efe1ffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe2006c820 0x7efe20102a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50430/0 (socket says 172.21.15.100:50430) 2024-05-10T11:15:02.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:02.386+0000 7efe1ffff700 1 -- 172.21.15.100:0/2322852395 learned_addr learned my addr 172.21.15.100:0/2322852395 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:15:02.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:02.386+0000 7efe24b5d700 1 --2- 172.21.15.100:0/2322852395 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe2006be70 0x7efe20102500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:15:02.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:02.386+0000 7efe2535e700 1 --2- 172.21.15.100:0/2322852395 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe201087c0 0x7efe20102f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:15:02.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:02.386+0000 7efe1ffff700 1 -- 172.21.15.100:0/2322852395 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe201087c0 msgr2=0x7efe20102f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:02.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:02.386+0000 7efe1ffff700 1 --2- 172.21.15.100:0/2322852395 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe201087c0 0x7efe20102f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:02.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:02.386+0000 7efe1ffff700 1 -- 172.21.15.100:0/2322852395 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe2006be70 msgr2=0x7efe20102500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:02.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:02.386+0000 7efe1ffff700 1 --2- 172.21.15.100:0/2322852395 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe2006be70 0x7efe20102500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:02.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:02.386+0000 7efe1ffff700 1 -- 172.21.15.100:0/2322852395 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe201ac630 con 0x7efe2006c820 2024-05-10T11:15:02.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:02.386+0000 7efe1ffff700 1 --2- 172.21.15.100:0/2322852395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe2006c820 0x7efe20102a40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7efe10007d90 tx=0x7efe10007dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:15:02.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:02.386+0000 7efe1dffb700 1 -- 172.21.15.100:0/2322852395 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe1001f030 con 0x7efe2006c820 2024-05-10T11:15:02.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:02.386+0000 7efe1dffb700 1 -- 172.21.15.100:0/2322852395 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efe1001bad0 con 0x7efe2006c820 2024-05-10T11:15:02.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:02.386+0000 7efe26dc1700 1 -- 172.21.15.100:0/2322852395 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe201ac860 con 0x7efe2006c820 2024-05-10T11:15:02.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:02.387+0000 7efe1dffb700 1 -- 172.21.15.100:0/2322852395 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe10012c50 con 0x7efe2006c820 2024-05-10T11:15:02.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:02.387+0000 7efe26dc1700 1 -- 172.21.15.100:0/2322852395 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe201acd70 con 0x7efe2006c820 2024-05-10T11:15:02.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:02.388+0000 7efe26dc1700 1 -- 172.21.15.100:0/2322852395 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe20104c30 con 0x7efe2006c820 2024-05-10T11:15:02.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:02.389+0000 7efe1dffb700 1 -- 172.21.15.100:0/2322852395 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7efe10022030 con 0x7efe2006c820 2024-05-10T11:15:02.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:02.390+0000 7efe1dffb700 1 --2- 172.21.15.100:0/2322852395 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7efe0c0b2880 0x7efe0c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:15:02.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:02.390+0000 7efe24b5d700 1 --2- 172.21.15.100:0/2322852395 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7efe0c0b2880 0x7efe0c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:15:02.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:02.390+0000 7efe1dffb700 1 -- 172.21.15.100:0/2322852395 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7efe1000f020 con 0x7efe2006c820 2024-05-10T11:15:02.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:02.390+0000 7efe24b5d700 1 --2- 172.21.15.100:0/2322852395 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7efe0c0b2880 0x7efe0c0b4d40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7efe08009a10 tx=0x7efe08009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:15:02.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:02.393+0000 7efe1dffb700 1 -- 172.21.15.100:0/2322852395 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7efe10029020 con 0x7efe2006c820 2024-05-10T11:15:02.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:02.643+0000 7efe26dc1700 1 -- 172.21.15.100:0/2322852395 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7efe2002e510 con 0x7efe2006c820 2024-05-10T11:15:02.644 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:02 smithi100 ceph-mon[30296]: osdmap e89: 12 total, 12 up, 12 in 2024-05-10T11:15:02.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:02.644+0000 7efe1dffb700 1 -- 172.21.15.100:0/2322852395 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7efe100ab720 con 0x7efe2006c820 2024-05-10T11:15:02.644 INFO:teuthology.orchestra.run.smithi100.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-10T11:15:02.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:02.646+0000 7efe26dc1700 1 -- 172.21.15.100:0/2322852395 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7efe0c0b2880 msgr2=0x7efe0c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:02.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:02.646+0000 7efe26dc1700 1 --2- 172.21.15.100:0/2322852395 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7efe0c0b2880 0x7efe0c0b4d40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7efe08009a10 tx=0x7efe08009190 comp rx=0 tx=0).stop 2024-05-10T11:15:02.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:02.646+0000 7efe26dc1700 1 -- 172.21.15.100:0/2322852395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe2006c820 msgr2=0x7efe20102a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:02.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:02.646+0000 7efe26dc1700 1 --2- 172.21.15.100:0/2322852395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe2006c820 0x7efe20102a40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7efe10007d90 tx=0x7efe10007dc0 comp rx=0 tx=0).stop 2024-05-10T11:15:02.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:02.646+0000 7efe26dc1700 1 -- 172.21.15.100:0/2322852395 shutdown_connections 2024-05-10T11:15:02.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:02.646+0000 7efe26dc1700 1 --2- 172.21.15.100:0/2322852395 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7efe0c0b2880 0x7efe0c0b4d40 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:02.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:02.646+0000 7efe26dc1700 1 --2- 172.21.15.100:0/2322852395 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe201087c0 0x7efe20102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:02.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:02.646+0000 7efe26dc1700 1 --2- 172.21.15.100:0/2322852395 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe2006be70 0x7efe20102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:02.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:02.646+0000 7efe26dc1700 1 --2- 172.21.15.100:0/2322852395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe2006c820 0x7efe20102a40 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:02.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:02.646+0000 7efe26dc1700 1 -- 172.21.15.100:0/2322852395 >> 172.21.15.100:0/2322852395 conn(0x7efe200783f0 msgr2=0x7efe201075a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:15:02.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:02.646+0000 7efe26dc1700 1 -- 172.21.15.100:0/2322852395 shutdown_connections 2024-05-10T11:15:02.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:02.646+0000 7efe26dc1700 1 -- 172.21.15.100:0/2322852395 wait complete. 2024-05-10T11:15:02.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:02 smithi178 ceph-mon[33359]: osdmap e89: 12 total, 12 up, 12 in 2024-05-10T11:15:02.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:02 smithi188 ceph-mon[34075]: osdmap e89: 12 total, 12 up, 12 in 2024-05-10T11:15:03.362 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-10T11:15:03.362 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-10T11:15:03.362 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi100.front.sepia.ceph.com 2024-05-10T11:15:03.363 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-05-10T11:15:03.363 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-10T11:15:03.363 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-10T11:15:03.363 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-10T11:15:03.363 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-10T11:15:03.363 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-05-10T11:15:03.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T11:15:03.363 DEBUG:teuthology.orchestra.run.smithi100:> ip addr 2024-05-10T11:15:03.382 INFO:teuthology.orchestra.run.smithi100.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-10T11:15:03.382 INFO:teuthology.orchestra.run.smithi100.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-10T11:15:03.382 INFO:teuthology.orchestra.run.smithi100.stdout: inet 127.0.0.1/8 scope host lo 2024-05-10T11:15:03.382 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-05-10T11:15:03.382 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 ::1/128 scope host 2024-05-10T11:15:03.382 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-05-10T11:15:03.382 INFO:teuthology.orchestra.run.smithi100.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T11:15:03.382 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:28 brd ff:ff:ff:ff:ff:ff 2024-05-10T11:15:03.382 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f0 2024-05-10T11:15:03.382 INFO:teuthology.orchestra.run.smithi100.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T11:15:03.382 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:29 brd ff:ff:ff:ff:ff:ff 2024-05-10T11:15:03.383 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f1 2024-05-10T11:15:03.383 INFO:teuthology.orchestra.run.smithi100.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T11:15:03.383 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:32 brd ff:ff:ff:ff:ff:ff 2024-05-10T11:15:03.383 INFO:teuthology.orchestra.run.smithi100.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-10T11:15:03.383 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:33 brd ff:ff:ff:ff:ff:ff 2024-05-10T11:15:03.383 INFO:teuthology.orchestra.run.smithi100.stdout: inet 172.21.15.100/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-10T11:15:03.383 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft 42162sec preferred_lft 42162sec 2024-05-10T11:15:03.383 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::ec4:7aff:fe8f:d733/64 scope link 2024-05-10T11:15:03.383 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-05-10T11:15:03.384 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-10T11:15:03.384 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-10T11:15:03.384 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-05-10T11:15:03.384 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link add name ceph-brx type bridge 2024-05-10T11:15:03.384 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip addr flush dev ceph-brx 2024-05-10T11:15:03.384 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set ceph-brx up 2024-05-10T11:15:03.384 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-10T11:15:03.384 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-05-10T11:15:03.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-10T11:15:03.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T11:15:03.580 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:03 smithi100 ceph-mon[30296]: pgmap v320: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:15:03.580 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:03 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/2322852395' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-10T11:15:03.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T11:15:03.587 DEBUG:teuthology.orchestra.run.smithi100:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-10T11:15:03.612 INFO:teuthology.orchestra.run.smithi100.stdout:1 2024-05-10T11:15:03.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T11:15:03.617 DEBUG:teuthology.orchestra.run.smithi100:> route 2024-05-10T11:15:03.751 INFO:teuthology.orchestra.run.smithi100.stdout:Kernel IP routing table 2024-05-10T11:15:03.751 INFO:teuthology.orchestra.run.smithi100.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-10T11:15:03.751 INFO:teuthology.orchestra.run.smithi100.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-10T11:15:03.751 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-10T11:15:03.751 INFO:teuthology.orchestra.run.smithi100.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-10T11:15:03.752 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-10T11:15:03.752 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-05-10T11:15:03.752 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-10T11:15:03.752 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-10T11:15:03.752 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-10T11:15:03.752 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-05-10T11:15:03.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:03 smithi178 ceph-mon[33359]: pgmap v320: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:15:03.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:03 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/2322852395' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-10T11:15:03.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:03 smithi188 ceph-mon[34075]: pgmap v320: 129 pgs: 129 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:15:03.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:03 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/2322852395' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-10T11:15:03.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-10T11:15:03.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T11:15:04.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T11:15:04.006 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-05-10T11:15:04.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T11:15:04.022 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list-id 2024-05-10T11:15:04.077 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-10T11:15:04.077 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-05-10T11:15:04.077 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-10T11:15:04.077 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-05-10T11:15:04.077 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-05-10T11:15:04.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-10T11:15:04.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T11:15:04.196 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-05-10T11:15:04.196 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-10T11:15:04.196 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-05-10T11:15:04.197 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-05-10T11:15:04.197 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-10T11:15:04.197 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-05-10T11:15:04.197 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-05-10T11:15:04.197 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-05-10T11:15:04.197 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-05-10T11:15:04.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-10T11:15:04.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T11:15:04.320 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-10T11:15:04.320 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-05-10T11:15:04.320 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set brx.0 up 2024-05-10T11:15:04.320 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set dev brx.0 master ceph-brx 2024-05-10T11:15:04.320 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-05-10T11:15:04.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-10T11:15:04.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T11:15:04.397 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-05-10T11:15:04.397 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T11:15:04.397 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-05-10T11:15:04.422 INFO:teuthology.orchestra.run.smithi100.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-05-10T11:15:04.422 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T11:15:04.422 DEBUG:teuthology.orchestra.run.smithi100:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-05-10T11:15:04.477 DEBUG:teuthology.orchestra.run.smithi100:> sudo modprobe fuse 2024-05-10T11:15:04.542 DEBUG:teuthology.orchestra.run.smithi100:> cat /proc/self/mounts | awk '{print $2}' 2024-05-10T11:15:04.599 INFO:teuthology.orchestra.run.smithi100.stdout:/proc 2024-05-10T11:15:04.599 INFO:teuthology.orchestra.run.smithi100.stdout:/sys 2024-05-10T11:15:04.599 INFO:teuthology.orchestra.run.smithi100.stdout:/dev 2024-05-10T11:15:04.599 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/security 2024-05-10T11:15:04.599 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/shm 2024-05-10T11:15:04.599 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/pts 2024-05-10T11:15:04.599 INFO:teuthology.orchestra.run.smithi100.stdout:/run 2024-05-10T11:15:04.599 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/cgroup 2024-05-10T11:15:04.599 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/pstore 2024-05-10T11:15:04.599 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/bpf 2024-05-10T11:15:04.600 INFO:teuthology.orchestra.run.smithi100.stdout:/ 2024-05-10T11:15:04.600 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/selinux 2024-05-10T11:15:04.600 INFO:teuthology.orchestra.run.smithi100.stdout:/proc/sys/fs/binfmt_misc 2024-05-10T11:15:04.600 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/mqueue 2024-05-10T11:15:04.600 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/hugepages 2024-05-10T11:15:04.600 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/debug 2024-05-10T11:15:04.600 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/tracing 2024-05-10T11:15:04.600 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/fuse/connections 2024-05-10T11:15:04.600 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/config 2024-05-10T11:15:04.600 INFO:teuthology.orchestra.run.smithi100.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-10T11:15:04.600 INFO:teuthology.orchestra.run.smithi100.stdout:/run/credentials/systemd-sysctl.service 2024-05-10T11:15:04.601 INFO:teuthology.orchestra.run.smithi100.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-10T11:15:04.601 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/nfs/rpc_pipefs 2024-05-10T11:15:04.601 INFO:teuthology.orchestra.run.smithi100.stdout:/run/user/1000 2024-05-10T11:15:04.601 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph 2024-05-10T11:15:04.601 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/containers/storage/overlay 2024-05-10T11:15:04.601 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/containers/storage/overlay/14c99dacab4f21afb763bc6738d3e41459fdd962ca149e9c806d7829d3a5e462/merged 2024-05-10T11:15:04.601 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/containers/storage/overlay/2ebaa2bbbfc96bd2425dfcc96728833728884750949c6a40f662fee33f4b994e/merged 2024-05-10T11:15:04.601 INFO:teuthology.orchestra.run.smithi100.stdout:/run/user/0 2024-05-10T11:15:04.601 INFO:teuthology.orchestra.run.smithi100.stdout:/proc/sys/fs/binfmt_misc 2024-05-10T11:15:04.601 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/containers/storage/overlay/13714d117f6e122b4c4f01bbf753a62c3541f3a730f30f49f2d5da6513f0305b/merged 2024-05-10T11:15:04.601 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/containers/storage/overlay/988d20f7e65ec06d4fb2acc5e7d5d246afd534d6aed6382840fb610666191a30/merged 2024-05-10T11:15:04.601 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/containers/storage/overlay/1a048bfd015e9bc34b400c599594a2b6829bf1318c996c13a6fd3fbf62e3169d/merged 2024-05-10T11:15:04.601 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/containers/storage/overlay/1276224ce171df2818ae1c61891eaad31014b7880cccc995fa2486d0d06d219c/merged 2024-05-10T11:15:04.601 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/containers/storage/overlay/5709ec87e7ac4fbfb97f09ec611630239a2d6f1420a85fdcdd9f37695b0fbb2e/merged 2024-05-10T11:15:04.602 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/containers/storage/overlay/886205e188c7b8d2d33b5282abae9196118fbb834277d2f9299693dcb24588e4/merged 2024-05-10T11:15:04.602 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/containers/storage/overlay/c6edc39d3dd307ad783812ab8e2e0121a898f29037388320c1802e0f88c0b564/merged 2024-05-10T11:15:04.602 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/containers/storage/overlay/8c972768aeb67ffe9b37821baac38161338670696cda25fea89a37cf99a46197/merged 2024-05-10T11:15:04.602 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns 2024-05-10T11:15:04.602 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-10T11:15:04.602 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-10T11:15:04.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T11:15:04.602 DEBUG:teuthology.orchestra.run.smithi100:> ls /sys/fs/fuse/connections 2024-05-10T11:15:04.655 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-10T11:15:04.655 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-05-10T11:15:04.698 DEBUG:teuthology.orchestra.run.smithi100:> sudo modprobe fuse 2024-05-10T11:15:04.722 DEBUG:teuthology.orchestra.run.smithi100:> cat /proc/self/mounts | awk '{print $2}' 2024-05-10T11:15:04.779 INFO:teuthology.orchestra.run.smithi100.stdout:/proc 2024-05-10T11:15:04.779 INFO:teuthology.orchestra.run.smithi100.stdout:/sys 2024-05-10T11:15:04.779 INFO:teuthology.orchestra.run.smithi100.stdout:/dev 2024-05-10T11:15:04.779 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/security 2024-05-10T11:15:04.779 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/shm 2024-05-10T11:15:04.779 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/pts 2024-05-10T11:15:04.779 INFO:teuthology.orchestra.run.smithi100.stdout:/run 2024-05-10T11:15:04.780 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/cgroup 2024-05-10T11:15:04.780 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/pstore 2024-05-10T11:15:04.780 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/bpf 2024-05-10T11:15:04.780 INFO:teuthology.orchestra.run.smithi100.stdout:/ 2024-05-10T11:15:04.780 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/selinux 2024-05-10T11:15:04.780 INFO:teuthology.orchestra.run.smithi100.stdout:/proc/sys/fs/binfmt_misc 2024-05-10T11:15:04.780 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/mqueue 2024-05-10T11:15:04.780 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/hugepages 2024-05-10T11:15:04.780 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/debug 2024-05-10T11:15:04.780 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/tracing 2024-05-10T11:15:04.780 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/fuse/connections 2024-05-10T11:15:04.780 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/config 2024-05-10T11:15:04.780 INFO:teuthology.orchestra.run.smithi100.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-10T11:15:04.780 INFO:teuthology.orchestra.run.smithi100.stdout:/run/credentials/systemd-sysctl.service 2024-05-10T11:15:04.781 INFO:teuthology.orchestra.run.smithi100.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-10T11:15:04.781 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/nfs/rpc_pipefs 2024-05-10T11:15:04.781 INFO:teuthology.orchestra.run.smithi100.stdout:/run/user/1000 2024-05-10T11:15:04.781 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph 2024-05-10T11:15:04.781 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/containers/storage/overlay 2024-05-10T11:15:04.781 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/containers/storage/overlay/14c99dacab4f21afb763bc6738d3e41459fdd962ca149e9c806d7829d3a5e462/merged 2024-05-10T11:15:04.781 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/containers/storage/overlay/2ebaa2bbbfc96bd2425dfcc96728833728884750949c6a40f662fee33f4b994e/merged 2024-05-10T11:15:04.781 INFO:teuthology.orchestra.run.smithi100.stdout:/run/user/0 2024-05-10T11:15:04.781 INFO:teuthology.orchestra.run.smithi100.stdout:/proc/sys/fs/binfmt_misc 2024-05-10T11:15:04.781 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/containers/storage/overlay/13714d117f6e122b4c4f01bbf753a62c3541f3a730f30f49f2d5da6513f0305b/merged 2024-05-10T11:15:04.781 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/containers/storage/overlay/988d20f7e65ec06d4fb2acc5e7d5d246afd534d6aed6382840fb610666191a30/merged 2024-05-10T11:15:04.781 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/containers/storage/overlay/1a048bfd015e9bc34b400c599594a2b6829bf1318c996c13a6fd3fbf62e3169d/merged 2024-05-10T11:15:04.781 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/containers/storage/overlay/1276224ce171df2818ae1c61891eaad31014b7880cccc995fa2486d0d06d219c/merged 2024-05-10T11:15:04.782 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/containers/storage/overlay/5709ec87e7ac4fbfb97f09ec611630239a2d6f1420a85fdcdd9f37695b0fbb2e/merged 2024-05-10T11:15:04.782 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/containers/storage/overlay/886205e188c7b8d2d33b5282abae9196118fbb834277d2f9299693dcb24588e4/merged 2024-05-10T11:15:04.782 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/containers/storage/overlay/c6edc39d3dd307ad783812ab8e2e0121a898f29037388320c1802e0f88c0b564/merged 2024-05-10T11:15:04.782 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/containers/storage/overlay/8c972768aeb67ffe9b37821baac38161338670696cda25fea89a37cf99a46197/merged 2024-05-10T11:15:04.782 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns 2024-05-10T11:15:04.782 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-10T11:15:04.782 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-10T11:15:04.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T11:15:04.782 DEBUG:teuthology.orchestra.run.smithi100:> ls /sys/fs/fuse/connections 2024-05-10T11:15:04.809 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi100.stderr:2024-05-10T11:15:04.809+0000 7f22ca218480 -1 init, newargv = 0x55f5de7cf230 newargc=15 2024-05-10T11:15:04.809 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi100.stderr:ceph-fuse[73896]: starting ceph client 2024-05-10T11:15:04.815 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi100.stderr:ceph-fuse[73896]: starting fuse 2024-05-10T11:15:04.824 INFO:teuthology.orchestra.run.smithi100.stdout:114 2024-05-10T11:15:04.824 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [114] 2024-05-10T11:15:04.824 DEBUG:teuthology.orchestra.run.smithi100:> sudo stdin-killer -- python3 -c ' 2024-05-10T11:15:04.824 DEBUG:teuthology.orchestra.run.smithi100:> import glob 2024-05-10T11:15:04.824 DEBUG:teuthology.orchestra.run.smithi100:> import re 2024-05-10T11:15:04.824 DEBUG:teuthology.orchestra.run.smithi100:> import os 2024-05-10T11:15:04.824 DEBUG:teuthology.orchestra.run.smithi100:> import subprocess 2024-05-10T11:15:04.825 DEBUG:teuthology.orchestra.run.smithi100:> 2024-05-10T11:15:04.825 DEBUG:teuthology.orchestra.run.smithi100:> def _find_admin_socket(client_name): 2024-05-10T11:15:04.825 DEBUG:teuthology.orchestra.run.smithi100:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-05-10T11:15:04.825 DEBUG:teuthology.orchestra.run.smithi100:> files = glob.glob(asok_path) 2024-05-10T11:15:04.825 DEBUG:teuthology.orchestra.run.smithi100:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-05-10T11:15:04.825 DEBUG:teuthology.orchestra.run.smithi100:> 2024-05-10T11:15:04.825 DEBUG:teuthology.orchestra.run.smithi100:> # Given a non-glob path, it better be there 2024-05-10T11:15:04.825 DEBUG:teuthology.orchestra.run.smithi100:> if "*" not in asok_path: 2024-05-10T11:15:04.825 DEBUG:teuthology.orchestra.run.smithi100:> assert(len(files) == 1) 2024-05-10T11:15:04.825 DEBUG:teuthology.orchestra.run.smithi100:> return files[0] 2024-05-10T11:15:04.825 DEBUG:teuthology.orchestra.run.smithi100:> 2024-05-10T11:15:04.825 DEBUG:teuthology.orchestra.run.smithi100:> for f in files: 2024-05-10T11:15:04.825 DEBUG:teuthology.orchestra.run.smithi100:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-10T11:15:04.825 DEBUG:teuthology.orchestra.run.smithi100:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-10T11:15:04.825 DEBUG:teuthology.orchestra.run.smithi100:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-10T11:15:04.825 DEBUG:teuthology.orchestra.run.smithi100:> contents = proc_f.read() 2024-05-10T11:15:04.825 DEBUG:teuthology.orchestra.run.smithi100:> if mountpoint in contents: 2024-05-10T11:15:04.826 DEBUG:teuthology.orchestra.run.smithi100:> return f 2024-05-10T11:15:04.826 DEBUG:teuthology.orchestra.run.smithi100:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-10T11:15:04.826 DEBUG:teuthology.orchestra.run.smithi100:> 2024-05-10T11:15:04.826 DEBUG:teuthology.orchestra.run.smithi100:> print(_find_admin_socket("client.admin")) 2024-05-10T11:15:04.826 DEBUG:teuthology.orchestra.run.smithi100:> ' 2024-05-10T11:15:04.943 INFO:teuthology.orchestra.run.smithi100.stdout:/var/run/ceph/ceph-client.admin.73896.asok 2024-05-10T11:15:04.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T11:15:04.953 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.73896.asok 2024-05-10T11:15:04.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T11:15:04.953 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.73896.asok status 2024-05-10T11:15:05.088 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:15:05.088 INFO:teuthology.orchestra.run.smithi100.stdout: "metadata": { 2024-05-10T11:15:05.089 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_sha1": "d39fa08867836ac070c47cb66c782fa559fd480b", 2024-05-10T11:15:05.089 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_version": "ceph version 19.0.0-3449-gd39fa088 (d39fa08867836ac070c47cb66c782fa559fd480b) squid (dev)", 2024-05-10T11:15:05.089 INFO:teuthology.orchestra.run.smithi100.stdout: "entity_id": "admin", 2024-05-10T11:15:05.089 INFO:teuthology.orchestra.run.smithi100.stdout: "hostname": "smithi100", 2024-05-10T11:15:05.089 INFO:teuthology.orchestra.run.smithi100.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-05-10T11:15:05.089 INFO:teuthology.orchestra.run.smithi100.stdout: "pid": "73896", 2024-05-10T11:15:05.089 INFO:teuthology.orchestra.run.smithi100.stdout: "root": "/" 2024-05-10T11:15:05.089 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:15:05.089 INFO:teuthology.orchestra.run.smithi100.stdout: "dentry_count": 0, 2024-05-10T11:15:05.089 INFO:teuthology.orchestra.run.smithi100.stdout: "dentry_pinned_count": 0, 2024-05-10T11:15:05.089 INFO:teuthology.orchestra.run.smithi100.stdout: "id": 14709, 2024-05-10T11:15:05.089 INFO:teuthology.orchestra.run.smithi100.stdout: "inst": { 2024-05-10T11:15:05.089 INFO:teuthology.orchestra.run.smithi100.stdout: "name": { 2024-05-10T11:15:05.089 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "client", 2024-05-10T11:15:05.090 INFO:teuthology.orchestra.run.smithi100.stdout: "num": 14709 2024-05-10T11:15:05.090 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:15:05.090 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": { 2024-05-10T11:15:05.090 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-05-10T11:15:05.090 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "192.168.0.1:0", 2024-05-10T11:15:05.090 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1002159992 2024-05-10T11:15:05.090 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:15:05.090 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:15:05.090 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": { 2024-05-10T11:15:05.090 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-05-10T11:15:05.090 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "192.168.0.1:0", 2024-05-10T11:15:05.090 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1002159992 2024-05-10T11:15:05.090 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:15:05.090 INFO:teuthology.orchestra.run.smithi100.stdout: "inst_str": "client.14709 192.168.0.1:0/1002159992", 2024-05-10T11:15:05.090 INFO:teuthology.orchestra.run.smithi100.stdout: "addr_str": "192.168.0.1:0/1002159992", 2024-05-10T11:15:05.091 INFO:teuthology.orchestra.run.smithi100.stdout: "inode_count": 1, 2024-05-10T11:15:05.091 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_epoch": 17, 2024-05-10T11:15:05.091 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_epoch": 89, 2024-05-10T11:15:05.091 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_epoch_barrier": 0, 2024-05-10T11:15:05.091 INFO:teuthology.orchestra.run.smithi100.stdout: "blocklisted": false, 2024-05-10T11:15:05.091 INFO:teuthology.orchestra.run.smithi100.stdout: "fs_name": "cephfs" 2024-05-10T11:15:05.091 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:15:05.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T11:15:05.097 DEBUG:teuthology.orchestra.run.smithi100:> stat --file-system '--printf=%T 2024-05-10T11:15:05.098 DEBUG:teuthology.orchestra.run.smithi100:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-05-10T11:15:05.113 INFO:teuthology.orchestra.run.smithi100.stdout:fuseblk 2024-05-10T11:15:05.113 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-05-10T11:15:05.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T11:15:05.114 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-05-10T11:15:05.179 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-05-10T11:15:05.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:05 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-10T11:15:05.301 INFO:teuthology.orchestra.run.smithi100.stdout:# file: . 2024-05-10T11:15:05.301 INFO:teuthology.orchestra.run.smithi100.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-05-10T11:15:05.301 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:15:05.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T11:15:05.308 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-05-10T11:15:05.308 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T11:15:05.324 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph osd blocklist ls 2024-05-10T11:15:05.607 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:15:05.729 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:05 smithi100 ceph-mon[30296]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:15:05.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:05 smithi178 ceph-mon[33359]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:15:05.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:05 smithi188 ceph-mon[34075]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:15:07.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:07.332+0000 7fde15a43700 1 -- 172.21.15.100:0/3364150355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde10108880 msgr2=0x7fde10110400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:07.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:07.332+0000 7fde15a43700 1 --2- 172.21.15.100:0/3364150355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde10108880 0x7fde10110400 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fde0800b0d0 tx=0x7fde0800b3e0 comp rx=0 tx=0).stop 2024-05-10T11:15:07.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:07.332+0000 7fde15a43700 1 -- 172.21.15.100:0/3364150355 shutdown_connections 2024-05-10T11:15:07.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:07.332+0000 7fde15a43700 1 --2- 172.21.15.100:0/3364150355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde10108880 0x7fde10110400 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:07.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:07.332+0000 7fde15a43700 1 --2- 172.21.15.100:0/3364150355 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde101037b0 0x7fde10103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:07.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:07.332+0000 7fde15a43700 1 --2- 172.21.15.100:0/3364150355 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde10104160 0x7fde10108150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:07.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:07.332+0000 7fde15a43700 1 -- 172.21.15.100:0/3364150355 >> 172.21.15.100:0/3364150355 conn(0x7fde1007a530 msgr2=0x7fde1007a940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:15:07.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:07.332+0000 7fde15a43700 1 -- 172.21.15.100:0/3364150355 shutdown_connections 2024-05-10T11:15:07.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:07.332+0000 7fde15a43700 1 -- 172.21.15.100:0/3364150355 wait complete. 2024-05-10T11:15:07.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:07.333+0000 7fde15a43700 1 Processor -- start 2024-05-10T11:15:07.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:07.333+0000 7fde15a43700 1 -- start start 2024-05-10T11:15:07.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:07.333+0000 7fde15a43700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde101037b0 0x7fde1019f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:15:07.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:07.333+0000 7fde15a43700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde10104160 0x7fde1019fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:15:07.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:07.333+0000 7fde15a43700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde10108880 0x7fde10199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:15:07.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:07.333+0000 7fde15a43700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fde10113520 con 0x7fde10104160 2024-05-10T11:15:07.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:07.333+0000 7fde15a43700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fde101133a0 con 0x7fde10108880 2024-05-10T11:15:07.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:07.333+0000 7fde15a43700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fde101136a0 con 0x7fde101037b0 2024-05-10T11:15:07.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:07.333+0000 7fde0e7fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde10104160 0x7fde1019fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:15:07.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:07.333+0000 7fde0e7fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde10104160 0x7fde1019fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40294/0 (socket says 172.21.15.100:40294) 2024-05-10T11:15:07.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:07.333+0000 7fde0e7fc700 1 -- 172.21.15.100:0/2891065312 learned_addr learned my addr 172.21.15.100:0/2891065312 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:15:07.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:07.334+0000 7fde0effd700 1 --2- 172.21.15.100:0/2891065312 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde101037b0 0x7fde1019f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:15:07.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:07.334+0000 7fde0e7fc700 1 -- 172.21.15.100:0/2891065312 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde101037b0 msgr2=0x7fde1019f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:07.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:07.334+0000 7fde0e7fc700 1 --2- 172.21.15.100:0/2891065312 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde101037b0 0x7fde1019f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:07.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:07.334+0000 7fde0e7fc700 1 -- 172.21.15.100:0/2891065312 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde10108880 msgr2=0x7fde10199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:07.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:07.334+0000 7fde0e7fc700 1 --2- 172.21.15.100:0/2891065312 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde10108880 0x7fde10199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:07.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:07.334+0000 7fde0e7fc700 1 -- 172.21.15.100:0/2891065312 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde10199fe0 con 0x7fde10104160 2024-05-10T11:15:07.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:07.334+0000 7fde0effd700 1 --2- 172.21.15.100:0/2891065312 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde101037b0 0x7fde1019f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:15:07.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:07.334+0000 7fde0e7fc700 1 --2- 172.21.15.100:0/2891065312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde10104160 0x7fde1019fb70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fde0400e9c0 tx=0x7fde0400ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:15:07.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:07.334+0000 7fde14a41700 1 -- 172.21.15.100:0/2891065312 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde0400cbd0 con 0x7fde10104160 2024-05-10T11:15:07.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:07.334+0000 7fde14a41700 1 -- 172.21.15.100:0/2891065312 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fde0400cd70 con 0x7fde10104160 2024-05-10T11:15:07.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:07.334+0000 7fde15a43700 1 -- 172.21.15.100:0/2891065312 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde1019a2d0 con 0x7fde10104160 2024-05-10T11:15:07.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:07.334+0000 7fde14a41700 1 -- 172.21.15.100:0/2891065312 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde04005610 con 0x7fde10104160 2024-05-10T11:15:07.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:07.334+0000 7fde15a43700 1 -- 172.21.15.100:0/2891065312 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde101a6490 con 0x7fde10104160 2024-05-10T11:15:07.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:07.335+0000 7fde15a43700 1 -- 172.21.15.100:0/2891065312 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde10104c10 con 0x7fde10104160 2024-05-10T11:15:07.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:07.337+0000 7fde14a41700 1 -- 172.21.15.100:0/2891065312 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fde04004750 con 0x7fde10104160 2024-05-10T11:15:07.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:07.337+0000 7fde14a41700 1 --2- 172.21.15.100:0/2891065312 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fddf80b2900 0x7fddf80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:15:07.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:07.338+0000 7fde0effd700 1 --2- 172.21.15.100:0/2891065312 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fddf80b2900 0x7fddf80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:15:07.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:07.338+0000 7fde14a41700 1 -- 172.21.15.100:0/2891065312 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fde040e2b50 con 0x7fde10104160 2024-05-10T11:15:07.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:07.338+0000 7fde0effd700 1 --2- 172.21.15.100:0/2891065312 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fddf80b2900 0x7fddf80b4dc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fde00009cb0 tx=0x7fde00009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:15:07.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:07.343+0000 7fde14a41700 1 -- 172.21.15.100:0/2891065312 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fde04054d70 con 0x7fde10104160 2024-05-10T11:15:07.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:07.565+0000 7fde15a43700 1 -- 172.21.15.100:0/2891065312 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fde1002e510 con 0x7fde10104160 2024-05-10T11:15:07.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:07.566+0000 7fde14a41700 1 -- 172.21.15.100:0/2891065312 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) ==== 81+0+601 (secure 0 0 0) 0x7fde040aa510 con 0x7fde10104160 2024-05-10T11:15:07.566 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.100:6800/1597433518 2024-05-11T11:07:00.092732+0000 2024-05-10T11:15:07.566 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.100:6800/1086450256 2024-05-11T11:06:33.114176+0000 2024-05-10T11:15:07.566 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.100:0/767964410 2024-05-11T11:07:00.092732+0000 2024-05-10T11:15:07.566 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.100:6801/1597433518 2024-05-11T11:07:00.092732+0000 2024-05-10T11:15:07.566 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.100:6801/1086450256 2024-05-11T11:06:33.114176+0000 2024-05-10T11:15:07.566 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.100:0/2770572070 2024-05-11T11:06:33.114176+0000 2024-05-10T11:15:07.566 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.100:0/1379340513 2024-05-11T11:06:33.114176+0000 2024-05-10T11:15:07.566 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.100:0/3025646953 2024-05-11T11:07:00.092732+0000 2024-05-10T11:15:07.566 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.100:0/4099258930 2024-05-11T11:07:00.092732+0000 2024-05-10T11:15:07.566 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.100:0/4267693567 2024-05-11T11:06:33.114176+0000 2024-05-10T11:15:07.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:07.568+0000 7fde15a43700 1 -- 172.21.15.100:0/2891065312 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fddf80b2900 msgr2=0x7fddf80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:07.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:07.568+0000 7fde15a43700 1 --2- 172.21.15.100:0/2891065312 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fddf80b2900 0x7fddf80b4dc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fde00009cb0 tx=0x7fde00009260 comp rx=0 tx=0).stop 2024-05-10T11:15:07.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:07.568+0000 7fde15a43700 1 -- 172.21.15.100:0/2891065312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde10104160 msgr2=0x7fde1019fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:07.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:07.568+0000 7fde15a43700 1 --2- 172.21.15.100:0/2891065312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde10104160 0x7fde1019fb70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fde0400e9c0 tx=0x7fde0400ed80 comp rx=0 tx=0).stop 2024-05-10T11:15:07.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:07.568+0000 7fde15a43700 1 -- 172.21.15.100:0/2891065312 shutdown_connections 2024-05-10T11:15:07.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:07.568+0000 7fde15a43700 1 --2- 172.21.15.100:0/2891065312 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fddf80b2900 0x7fddf80b4dc0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:07.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:07.568+0000 7fde15a43700 1 --2- 172.21.15.100:0/2891065312 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde10108880 0x7fde10199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:07.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:07.568+0000 7fde15a43700 1 --2- 172.21.15.100:0/2891065312 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde101037b0 0x7fde1019f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:07.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:07.568+0000 7fde15a43700 1 --2- 172.21.15.100:0/2891065312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde10104160 0x7fde1019fb70 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:07.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:07.568+0000 7fde15a43700 1 -- 172.21.15.100:0/2891065312 >> 172.21.15.100:0/2891065312 conn(0x7fde1007a530 msgr2=0x7fde10101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:15:07.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:07.568+0000 7fde15a43700 1 -- 172.21.15.100:0/2891065312 shutdown_connections 2024-05-10T11:15:07.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:07.568+0000 7fde15a43700 1 -- 172.21.15.100:0/2891065312 wait complete. 2024-05-10T11:15:07.569 INFO:teuthology.orchestra.run.smithi100.stderr:listed 10 entries 2024-05-10T11:15:07.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:07 smithi178 ceph-mon[33359]: pgmap v322: 129 pgs: 129 active+clean; 579 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:15:07.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:07 smithi188 ceph-mon[34075]: pgmap v322: 129 pgs: 129 active+clean; 579 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:15:07.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:07 smithi100 ceph-mon[30296]: pgmap v322: 129 pgs: 129 active+clean; 579 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:15:08.248 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-05-10T11:15:08.248 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T11:15:08.266 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph osd blocklist ls 2024-05-10T11:15:08.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:08 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/2891065312' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-10T11:15:08.558 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:15:08.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:08 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/2891065312' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-10T11:15:08.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:08 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/2891065312' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-10T11:15:09.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:09 smithi100 ceph-mon[30296]: pgmap v323: 129 pgs: 129 active+clean; 581 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-05-10T11:15:09.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:09 smithi178 ceph-mon[33359]: pgmap v323: 129 pgs: 129 active+clean; 581 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-05-10T11:15:09.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:09 smithi188 ceph-mon[34075]: pgmap v323: 129 pgs: 129 active+clean; 581 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-05-10T11:15:10.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:10.295+0000 7f6221287700 1 -- 172.21.15.100:0/2306422663 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f621c103f20 msgr2=0x7f621c110310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:10.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:10.295+0000 7f6221287700 1 --2- 172.21.15.100:0/2306422663 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f621c103f20 0x7f621c110310 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f621400b0d0 tx=0x7f621400b3e0 comp rx=0 tx=0).stop 2024-05-10T11:15:10.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:10.295+0000 7f6221287700 1 -- 172.21.15.100:0/2306422663 shutdown_connections 2024-05-10T11:15:10.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:10.295+0000 7f6221287700 1 --2- 172.21.15.100:0/2306422663 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f621c103f20 0x7f621c110310 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:10.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:10.295+0000 7f6221287700 1 --2- 172.21.15.100:0/2306422663 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f621c109540 0x7f621c109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:10.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:10.295+0000 7f6221287700 1 --2- 172.21.15.100:0/2306422663 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f621c103560 0x7f621c1039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:10.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:10.295+0000 7f6221287700 1 -- 172.21.15.100:0/2306422663 >> 172.21.15.100:0/2306422663 conn(0x7f621c07a660 msgr2=0x7f621c07aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:15:10.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:10.295+0000 7f6221287700 1 -- 172.21.15.100:0/2306422663 shutdown_connections 2024-05-10T11:15:10.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:10.295+0000 7f6221287700 1 -- 172.21.15.100:0/2306422663 wait complete. 2024-05-10T11:15:10.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:10.295+0000 7f6221287700 1 Processor -- start 2024-05-10T11:15:10.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:10.295+0000 7f6221287700 1 -- start start 2024-05-10T11:15:10.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:10.296+0000 7f6221287700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f621c103560 0x7f621c19f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:15:10.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:10.296+0000 7f6221287700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f621c103f20 0x7f621c19fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:15:10.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:10.296+0000 7f6221287700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f621c109540 0x7f621c1999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:15:10.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:10.296+0000 7f6221287700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f621c113620 con 0x7f621c103560 2024-05-10T11:15:10.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:10.296+0000 7f6221287700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f621c1134a0 con 0x7f621c103f20 2024-05-10T11:15:10.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:10.296+0000 7f6221287700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f621c1137a0 con 0x7f621c109540 2024-05-10T11:15:10.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:10.296+0000 7f621affd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f621c103560 0x7f621c19f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:15:10.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:10.296+0000 7f621affd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f621c103560 0x7f621c19f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40304/0 (socket says 172.21.15.100:40304) 2024-05-10T11:15:10.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:10.296+0000 7f621affd700 1 -- 172.21.15.100:0/1751416872 learned_addr learned my addr 172.21.15.100:0/1751416872 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:15:10.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:10.296+0000 7f621b7fe700 1 --2- 172.21.15.100:0/1751416872 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f621c109540 0x7f621c1999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:15:10.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:10.296+0000 7f6213fff700 1 --2- 172.21.15.100:0/1751416872 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f621c103f20 0x7f621c19fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:15:10.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:10.296+0000 7f621affd700 1 -- 172.21.15.100:0/1751416872 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f621c109540 msgr2=0x7f621c1999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:10.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:10.296+0000 7f621affd700 1 --2- 172.21.15.100:0/1751416872 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f621c109540 0x7f621c1999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:10.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:10.296+0000 7f621affd700 1 -- 172.21.15.100:0/1751416872 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f621c103f20 msgr2=0x7f621c19fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:10.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:10.296+0000 7f621affd700 1 --2- 172.21.15.100:0/1751416872 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f621c103f20 0x7f621c19fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:10.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:10.296+0000 7f621affd700 1 -- 172.21.15.100:0/1751416872 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f621c19a120 con 0x7f621c103560 2024-05-10T11:15:10.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:10.296+0000 7f6213fff700 1 --2- 172.21.15.100:0/1751416872 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f621c103f20 0x7f621c19fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:15:10.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:10.296+0000 7f621affd700 1 --2- 172.21.15.100:0/1751416872 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f621c103560 0x7f621c19f800 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f620c00b530 tx=0x7f620c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:15:10.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:10.296+0000 7f6218ff9700 1 -- 172.21.15.100:0/1751416872 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620c015020 con 0x7f621c103560 2024-05-10T11:15:10.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:10.296+0000 7f6218ff9700 1 -- 172.21.15.100:0/1751416872 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f620c011cc0 con 0x7f621c103560 2024-05-10T11:15:10.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:10.296+0000 7f6221287700 1 -- 172.21.15.100:0/1751416872 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f621c19a410 con 0x7f621c103560 2024-05-10T11:15:10.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:10.296+0000 7f6218ff9700 1 -- 172.21.15.100:0/1751416872 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620c010dc0 con 0x7f621c103560 2024-05-10T11:15:10.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:10.297+0000 7f6221287700 1 -- 172.21.15.100:0/1751416872 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f621c1a66d0 con 0x7f621c103560 2024-05-10T11:15:10.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:10.298+0000 7f6221287700 1 -- 172.21.15.100:0/1751416872 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f621c10b9d0 con 0x7f621c103560 2024-05-10T11:15:10.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:10.299+0000 7f6218ff9700 1 -- 172.21.15.100:0/1751416872 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f620c018020 con 0x7f621c103560 2024-05-10T11:15:10.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:10.300+0000 7f6218ff9700 1 --2- 172.21.15.100:0/1751416872 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f62080b2900 0x7f62080b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:15:10.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:10.300+0000 7f6213fff700 1 --2- 172.21.15.100:0/1751416872 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f62080b2900 0x7f62080b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:15:10.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:10.300+0000 7f6218ff9700 1 -- 172.21.15.100:0/1751416872 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f620c0e23f0 con 0x7f621c103560 2024-05-10T11:15:10.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:10.301+0000 7f6213fff700 1 --2- 172.21.15.100:0/1751416872 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f62080b2900 0x7f62080b4dc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f621c07ada0 tx=0x7f6204008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:15:10.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:10.304+0000 7f6218ff9700 1 -- 172.21.15.100:0/1751416872 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f620c01f020 con 0x7f621c103560 2024-05-10T11:15:10.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:10.519+0000 7f6221287700 1 -- 172.21.15.100:0/1751416872 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f621c19af20 con 0x7f621c103560 2024-05-10T11:15:10.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:10.519+0000 7f6218ff9700 1 -- 172.21.15.100:0/1751416872 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) ==== 81+0+601 (secure 0 0 0) 0x7f621c19af20 con 0x7f621c103560 2024-05-10T11:15:10.519 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.100:6800/1597433518 2024-05-11T11:07:00.092732+0000 2024-05-10T11:15:10.520 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.100:6800/1086450256 2024-05-11T11:06:33.114176+0000 2024-05-10T11:15:10.520 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.100:0/767964410 2024-05-11T11:07:00.092732+0000 2024-05-10T11:15:10.520 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.100:6801/1597433518 2024-05-11T11:07:00.092732+0000 2024-05-10T11:15:10.520 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.100:6801/1086450256 2024-05-11T11:06:33.114176+0000 2024-05-10T11:15:10.520 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.100:0/2770572070 2024-05-11T11:06:33.114176+0000 2024-05-10T11:15:10.520 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.100:0/1379340513 2024-05-11T11:06:33.114176+0000 2024-05-10T11:15:10.520 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.100:0/3025646953 2024-05-11T11:07:00.092732+0000 2024-05-10T11:15:10.520 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.100:0/4099258930 2024-05-11T11:07:00.092732+0000 2024-05-10T11:15:10.520 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.100:0/4267693567 2024-05-11T11:06:33.114176+0000 2024-05-10T11:15:10.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:10.523+0000 7f6221287700 1 -- 172.21.15.100:0/1751416872 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f62080b2900 msgr2=0x7f62080b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:10.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:10.523+0000 7f6221287700 1 --2- 172.21.15.100:0/1751416872 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f62080b2900 0x7f62080b4dc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f621c07ada0 tx=0x7f6204008040 comp rx=0 tx=0).stop 2024-05-10T11:15:10.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:10.523+0000 7f6221287700 1 -- 172.21.15.100:0/1751416872 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f621c103560 msgr2=0x7f621c19f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:10.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:10.523+0000 7f6221287700 1 --2- 172.21.15.100:0/1751416872 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f621c103560 0x7f621c19f800 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f620c00b530 tx=0x7f620c00b840 comp rx=0 tx=0).stop 2024-05-10T11:15:10.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:10.523+0000 7f6221287700 1 -- 172.21.15.100:0/1751416872 shutdown_connections 2024-05-10T11:15:10.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:10.523+0000 7f6221287700 1 --2- 172.21.15.100:0/1751416872 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f62080b2900 0x7f62080b4dc0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:10.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:10.523+0000 7f6221287700 1 --2- 172.21.15.100:0/1751416872 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f621c109540 0x7f621c1999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:10.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:10.523+0000 7f6221287700 1 --2- 172.21.15.100:0/1751416872 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f621c103560 0x7f621c19f800 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:10.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:10.523+0000 7f6221287700 1 --2- 172.21.15.100:0/1751416872 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f621c103f20 0x7f621c19fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:10.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:10.523+0000 7f6221287700 1 -- 172.21.15.100:0/1751416872 >> 172.21.15.100:0/1751416872 conn(0x7f621c07a660 msgr2=0x7f621c102400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:15:10.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:10.523+0000 7f6221287700 1 -- 172.21.15.100:0/1751416872 shutdown_connections 2024-05-10T11:15:10.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:10.523+0000 7f6221287700 1 -- 172.21.15.100:0/1751416872 wait complete. 2024-05-10T11:15:10.524 INFO:teuthology.orchestra.run.smithi100.stderr:listed 10 entries 2024-05-10T11:15:11.156 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi100.front.sepia.ceph.com... 2024-05-10T11:15:11.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T11:15:11.157 DEBUG:teuthology.orchestra.run.smithi100:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-05-10T11:15:11.185 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi100.stderr:ceph-fuse[73896]: fuse finished with error 0 and tester_r 0 2024-05-10T11:15:11.187 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T11:15:11.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:11 smithi178 ceph-mon[33359]: pgmap v324: 129 pgs: 129 active+clean; 581 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 419 B/s wr, 0 op/s 2024-05-10T11:15:11.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:11 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/1751416872' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-10T11:15:11.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:11 smithi188 ceph-mon[34075]: pgmap v324: 129 pgs: 129 active+clean; 581 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 419 B/s wr, 0 op/s 2024-05-10T11:15:11.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:11 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/1751416872' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-10T11:15:11.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:11 smithi100 ceph-mon[30296]: pgmap v324: 129 pgs: 129 active+clean; 581 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 419 B/s wr, 0 op/s 2024-05-10T11:15:11.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:11 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/1751416872' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-10T11:15:13.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:13 smithi178 ceph-mon[33359]: pgmap v325: 129 pgs: 129 active+clean; 581 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 475 B/s wr, 0 op/s 2024-05-10T11:15:13.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:13 smithi188 ceph-mon[34075]: pgmap v325: 129 pgs: 129 active+clean; 581 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 475 B/s wr, 0 op/s 2024-05-10T11:15:13.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:13 smithi100 ceph-mon[30296]: pgmap v325: 129 pgs: 129 active+clean; 581 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 475 B/s wr, 0 op/s 2024-05-10T11:15:15.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:15 smithi178 ceph-mon[33359]: pgmap v326: 129 pgs: 129 active+clean; 581 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-10T11:15:15.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:15 smithi188 ceph-mon[34075]: pgmap v326: 129 pgs: 129 active+clean; 581 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-10T11:15:15.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:15 smithi100 ceph-mon[30296]: pgmap v326: 129 pgs: 129 active+clean; 581 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-10T11:15:17.288 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-05-10T11:15:17.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T11:15:17.289 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-05-10T11:15:17.319 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-10T11:15:17.614 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:15:17.736 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:17 smithi100 ceph-mon[30296]: pgmap v327: 129 pgs: 129 active+clean; 581 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-10T11:15:17.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:17 smithi178 ceph-mon[33359]: pgmap v327: 129 pgs: 129 active+clean; 581 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-10T11:15:17.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:17 smithi188 ceph-mon[34075]: pgmap v327: 129 pgs: 129 active+clean; 581 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-10T11:15:18.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:18 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:15:18.775 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:18 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:15:18.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:18 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:15:19.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:19.340+0000 7f9fc4185700 1 -- 172.21.15.100:0/2263282757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9fbc108400 msgr2=0x7f9fbc1087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:19.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:19.340+0000 7f9fc4185700 1 --2- 172.21.15.100:0/2263282757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9fbc108400 0x7f9fbc1087e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8009990 tx=0x7f9fb8009ca0 comp rx=0 tx=0).stop 2024-05-10T11:15:19.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:19.340+0000 7f9fc4185700 1 -- 172.21.15.100:0/2263282757 shutdown_connections 2024-05-10T11:15:19.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:19.340+0000 7f9fc4185700 1 --2- 172.21.15.100:0/2263282757 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9fbc102d50 0x7f9fbc10f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:19.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:19.340+0000 7f9fc4185700 1 --2- 172.21.15.100:0/2263282757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9fbc108400 0x7f9fbc1087e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:19.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:19.340+0000 7f9fc4185700 1 --2- 172.21.15.100:0/2263282757 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9fbc102390 0x7f9fbc102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:19.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:19.340+0000 7f9fc4185700 1 -- 172.21.15.100:0/2263282757 >> 172.21.15.100:0/2263282757 conn(0x7f9fbc079310 msgr2=0x7f9fbc079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:15:19.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:19.340+0000 7f9fc4185700 1 -- 172.21.15.100:0/2263282757 shutdown_connections 2024-05-10T11:15:19.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:19.341+0000 7f9fc4185700 1 -- 172.21.15.100:0/2263282757 wait complete. 2024-05-10T11:15:19.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:19.341+0000 7f9fc4185700 1 Processor -- start 2024-05-10T11:15:19.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:19.341+0000 7f9fc4185700 1 -- start start 2024-05-10T11:15:19.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:19.342+0000 7f9fc4185700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9fbc102390 0x7f9fbc19f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:15:19.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:19.342+0000 7f9fc4185700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9fbc102d50 0x7f9fbc19fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:15:19.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:19.342+0000 7f9fc4185700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9fbc108400 0x7f9fbc199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:15:19.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:19.342+0000 7f9fc4185700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9fbc1123d0 con 0x7f9fbc108400 2024-05-10T11:15:19.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:19.342+0000 7f9fc4185700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9fbc112250 con 0x7f9fbc102d50 2024-05-10T11:15:19.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:19.342+0000 7f9fc4185700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9fbc112550 con 0x7f9fbc102390 2024-05-10T11:15:19.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:19.342+0000 7f9fc2722700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9fbc108400 0x7f9fbc199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:15:19.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:19.342+0000 7f9fc1720700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9fbc102d50 0x7f9fbc19fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:15:19.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:19.342+0000 7f9fc2722700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9fbc108400 0x7f9fbc199800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37318/0 (socket says 172.21.15.100:37318) 2024-05-10T11:15:19.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:19.342+0000 7f9fc2722700 1 -- 172.21.15.100:0/2717414798 learned_addr learned my addr 172.21.15.100:0/2717414798 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:15:19.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:19.342+0000 7f9fc1f21700 1 --2- 172.21.15.100:0/2717414798 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9fbc102390 0x7f9fbc19f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:15:19.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:19.342+0000 7f9fc2722700 1 -- 172.21.15.100:0/2717414798 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9fbc102390 msgr2=0x7f9fbc19f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:19.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:19.342+0000 7f9fc2722700 1 --2- 172.21.15.100:0/2717414798 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9fbc102390 0x7f9fbc19f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:19.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:19.342+0000 7f9fc2722700 1 -- 172.21.15.100:0/2717414798 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9fbc102d50 msgr2=0x7f9fbc19fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:19.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:19.342+0000 7f9fc2722700 1 --2- 172.21.15.100:0/2717414798 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9fbc102d50 0x7f9fbc19fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:19.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:19.342+0000 7f9fc2722700 1 -- 172.21.15.100:0/2717414798 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fbc199fe0 con 0x7f9fbc108400 2024-05-10T11:15:19.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:19.342+0000 7f9fc2722700 1 --2- 172.21.15.100:0/2717414798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9fbc108400 0x7f9fbc199800 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9fb400ada0 tx=0x7f9fb400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:15:19.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:19.342+0000 7f9fb2ffd700 1 -- 172.21.15.100:0/2717414798 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb4017020 con 0x7f9fbc108400 2024-05-10T11:15:19.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:19.342+0000 7f9fb2ffd700 1 -- 172.21.15.100:0/2717414798 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9fb400cd60 con 0x7f9fbc108400 2024-05-10T11:15:19.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:19.342+0000 7f9fc4185700 1 -- 172.21.15.100:0/2717414798 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fbc19a2d0 con 0x7f9fbc108400 2024-05-10T11:15:19.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:19.342+0000 7f9fb2ffd700 1 -- 172.21.15.100:0/2717414798 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb4005280 con 0x7f9fbc108400 2024-05-10T11:15:19.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:19.343+0000 7f9fc4185700 1 -- 172.21.15.100:0/2717414798 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fbc1a64e0 con 0x7f9fbc108400 2024-05-10T11:15:19.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:19.344+0000 7f9fc4185700 1 -- 172.21.15.100:0/2717414798 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fbc103ab0 con 0x7f9fbc108400 2024-05-10T11:15:19.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:19.346+0000 7f9fb2ffd700 1 -- 172.21.15.100:0/2717414798 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f9fb400c8d0 con 0x7f9fbc108400 2024-05-10T11:15:19.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:19.346+0000 7f9fb2ffd700 1 --2- 172.21.15.100:0/2717414798 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f9fa80b2880 0x7f9fa80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:15:19.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:19.347+0000 7f9fc1f21700 1 --2- 172.21.15.100:0/2717414798 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f9fa80b2880 0x7f9fa80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:15:19.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:19.347+0000 7f9fb2ffd700 1 -- 172.21.15.100:0/2717414798 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f9fb40e2220 con 0x7f9fbc108400 2024-05-10T11:15:19.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:19.347+0000 7f9fc1f21700 1 --2- 172.21.15.100:0/2717414798 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f9fa80b2880 0x7f9fa80b4d40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8000c00 tx=0x7f9fb8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:15:19.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:19.349+0000 7f9fb2ffd700 1 -- 172.21.15.100:0/2717414798 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f9fb4021020 con 0x7f9fbc108400 2024-05-10T11:15:19.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:19.568+0000 7f9fc4185700 1 -- 172.21.15.100:0/2717414798 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9fbc1a6770 con 0x7f9fbc108400 2024-05-10T11:15:19.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:19.569+0000 7f9fb2ffd700 1 -- 172.21.15.100:0/2717414798 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24737 (secure 0 0 0) 0x7f9fb40a9c70 con 0x7f9fbc108400 2024-05-10T11:15:19.570 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:15:19.570 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":89,"fsid":"20a542cc-0ebd-11ef-bc98-c7b262605968","created":"2024-05-10T11:06:05.454446+0000","modified":"2024-05-10T11:15:01.418144+0000","last_up_change":"2024-05-10T11:13:18.185471+0000","last_in_change":"2024-05-10T11:12:57.268678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T11:09:26.176441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T11:14:21.004285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T11:14:24.179414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T11:14:33.308517+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5f031418-da23-4e85-8669-160960dadd09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":643028278},{"type":"v1","addr":"172.21.15.100:6803","nonce":643028278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":643028278},{"type":"v1","addr":"172.21.15.100:6805","nonce":643028278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":643028278},{"type":"v1","addr":"172.21.15.100:6809","nonce":643028278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":643028278},{"type":"v1","addr":"172.21.15.100:6807","nonce":643028278}]},"public_addr":"172.21.15.100:6803/643028278","cluster_addr":"172.21.15.100:6805/643028278","heartbeat_back_addr":"172.21.15.100:6809/643028278","heartbeat_front_addr":"172.21.15.100:6807/643028278","state":["exists","up"]},{"osd":1,"uuid":"a908cdc9-001e-4b65-9ab0-0dd08dba20f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":3103987017},{"type":"v1","addr":"172.21.15.178:6801","nonce":3103987017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":3103987017},{"type":"v1","addr":"172.21.15.178:6803","nonce":3103987017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":3103987017},{"type":"v1","addr":"172.21.15.178:6807","nonce":3103987017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":3103987017},{"type":"v1","addr":"172.21.15.178:6805","nonce":3103987017}]},"public_addr":"172.21.15.178:6801/3103987017","cluster_addr":"172.21.15.178:6803/3103987017","heartbeat_back_addr":"172.21.15.178:6807/3103987017","heartbeat_front_addr":"172.21.15.178:6805/3103987017","state":["exists","up"]},{"osd":2,"uuid":"4d8f89ae-9424-46e9-9c80-17c47cc5e8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3991466468},{"type":"v1","addr":"172.21.15.188:6801","nonce":3991466468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3991466468},{"type":"v1","addr":"172.21.15.188:6803","nonce":3991466468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3991466468},{"type":"v1","addr":"172.21.15.188:6807","nonce":3991466468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3991466468},{"type":"v1","addr":"172.21.15.188:6805","nonce":3991466468}]},"public_addr":"172.21.15.188:6801/3991466468","cluster_addr":"172.21.15.188:6803/3991466468","heartbeat_back_addr":"172.21.15.188:6807/3991466468","heartbeat_front_addr":"172.21.15.188:6805/3991466468","state":["exists","up"]},{"osd":3,"uuid":"4c8865aa-fef4-4710-b766-1385adb9462c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1713357192},{"type":"v1","addr":"172.21.15.100:6811","nonce":1713357192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1713357192},{"type":"v1","addr":"172.21.15.100:6813","nonce":1713357192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1713357192},{"type":"v1","addr":"172.21.15.100:6817","nonce":1713357192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1713357192},{"type":"v1","addr":"172.21.15.100:6815","nonce":1713357192}]},"public_addr":"172.21.15.100:6811/1713357192","cluster_addr":"172.21.15.100:6813/1713357192","heartbeat_back_addr":"172.21.15.100:6817/1713357192","heartbeat_front_addr":"172.21.15.100:6815/1713357192","state":["exists","up"]},{"osd":4,"uuid":"a9653800-da57-43eb-8781-3188e8a3949f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":2222814192},{"type":"v1","addr":"172.21.15.178:6809","nonce":2222814192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":2222814192},{"type":"v1","addr":"172.21.15.178:6811","nonce":2222814192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":2222814192},{"type":"v1","addr":"172.21.15.178:6815","nonce":2222814192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":2222814192},{"type":"v1","addr":"172.21.15.178:6813","nonce":2222814192}]},"public_addr":"172.21.15.178:6809/2222814192","cluster_addr":"172.21.15.178:6811/2222814192","heartbeat_back_addr":"172.21.15.178:6815/2222814192","heartbeat_front_addr":"172.21.15.178:6813/2222814192","state":["exists","up"]},{"osd":5,"uuid":"fb5f8af2-1eb0-4d8a-a7a3-ed6dd68e692e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3517375935},{"type":"v1","addr":"172.21.15.188:6809","nonce":3517375935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3517375935},{"type":"v1","addr":"172.21.15.188:6811","nonce":3517375935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3517375935},{"type":"v1","addr":"172.21.15.188:6815","nonce":3517375935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3517375935},{"type":"v1","addr":"172.21.15.188:6813","nonce":3517375935}]},"public_addr":"172.21.15.188:6809/3517375935","cluster_addr":"172.21.15.188:6811/3517375935","heartbeat_back_addr":"172.21.15.188:6815/3517375935","heartbeat_front_addr":"172.21.15.188:6813/3517375935","state":["exists","up"]},{"osd":6,"uuid":"84b57387-0a94-4e49-b099-493d329b87dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1410593862},{"type":"v1","addr":"172.21.15.100:6819","nonce":1410593862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1410593862},{"type":"v1","addr":"172.21.15.100:6821","nonce":1410593862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1410593862},{"type":"v1","addr":"172.21.15.100:6825","nonce":1410593862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1410593862},{"type":"v1","addr":"172.21.15.100:6823","nonce":1410593862}]},"public_addr":"172.21.15.100:6819/1410593862","cluster_addr":"172.21.15.100:6821/1410593862","heartbeat_back_addr":"172.21.15.100:6825/1410593862","heartbeat_front_addr":"172.21.15.100:6823/1410593862","state":["exists","up"]},{"osd":7,"uuid":"02ada28f-ac4f-47cf-983f-8a6da1140abe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2458690455},{"type":"v1","addr":"172.21.15.178:6817","nonce":2458690455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2458690455},{"type":"v1","addr":"172.21.15.178:6819","nonce":2458690455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2458690455},{"type":"v1","addr":"172.21.15.178:6823","nonce":2458690455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2458690455},{"type":"v1","addr":"172.21.15.178:6821","nonce":2458690455}]},"public_addr":"172.21.15.178:6817/2458690455","cluster_addr":"172.21.15.178:6819/2458690455","heartbeat_back_addr":"172.21.15.178:6823/2458690455","heartbeat_front_addr":"172.21.15.178:6821/2458690455","state":["exists","up"]},{"osd":8,"uuid":"e7e489bc-15fc-4a4c-b297-d1bb626513ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1523872826},{"type":"v1","addr":"172.21.15.188:6817","nonce":1523872826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1523872826},{"type":"v1","addr":"172.21.15.188:6819","nonce":1523872826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1523872826},{"type":"v1","addr":"172.21.15.188:6823","nonce":1523872826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1523872826},{"type":"v1","addr":"172.21.15.188:6821","nonce":1523872826}]},"public_addr":"172.21.15.188:6817/1523872826","cluster_addr":"172.21.15.188:6819/1523872826","heartbeat_back_addr":"172.21.15.188:6823/1523872826","heartbeat_front_addr":"172.21.15.188:6821/1523872826","state":["exists","up"]},{"osd":9,"uuid":"72328326-cd16-45d9-bfd9-40f6c5c20f5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3585761978},{"type":"v1","addr":"172.21.15.100:6827","nonce":3585761978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3585761978},{"type":"v1","addr":"172.21.15.100:6829","nonce":3585761978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3585761978},{"type":"v1","addr":"172.21.15.100:6833","nonce":3585761978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3585761978},{"type":"v1","addr":"172.21.15.100:6831","nonce":3585761978}]},"public_addr":"172.21.15.100:6827/3585761978","cluster_addr":"172.21.15.100:6829/3585761978","heartbeat_back_addr":"172.21.15.100:6833/3585761978","heartbeat_front_addr":"172.21.15.100:6831/3585761978","state":["exists","up"]},{"osd":10,"uuid":"a355f9a8-91a3-41d4-814d-5a9c9b953706","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2985841725},{"type":"v1","addr":"172.21.15.178:6825","nonce":2985841725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2985841725},{"type":"v1","addr":"172.21.15.178:6827","nonce":2985841725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2985841725},{"type":"v1","addr":"172.21.15.178:6831","nonce":2985841725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2985841725},{"type":"v1","addr":"172.21.15.178:6829","nonce":2985841725}]},"public_addr":"172.21.15.178:6825/2985841725","cluster_addr":"172.21.15.178:6827/2985841725","heartbeat_back_addr":"172.21.15.178:6831/2985841725","heartbeat_front_addr":"172.21.15.178:6829/2985841725","state":["exists","up"]},{"osd":11,"uuid":"76a5e13c-7e38-42f9-bfa7-7fedec7c7184","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2574955118},{"type":"v1","addr":"172.21.15.188:6825","nonce":2574955118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2574955118},{"type":"v1","addr":"172.21.15.188:6827","nonce":2574955118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2574955118},{"type":"v1","addr":"172.21.15.188:6831","nonce":2574955118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2574955118},{"type":"v1","addr":"172.21.15.188:6829","nonce":2574955118}]},"public_addr":"172.21.15.188:6825/2574955118","cluster_addr":"172.21.15.188:6827/2574955118","heartbeat_back_addr":"172.21.15.188:6831/2574955118","heartbeat_front_addr":"172.21.15.188:6829/2574955118","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:08:36.086637+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:08:58.948031+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:09:22.142517+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:09:47.655748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:10:12.853999+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:10:37.911383+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:11:04.448815+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:11:30.331740+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:11:55.819278+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:12:23.204612+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:12:49.484835+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:13:15.783531+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:6800/1597433518":"2024-05-11T11:07:00.092732+0000","172.21.15.100:6800/1086450256":"2024-05-11T11:06:33.114176+0000","172.21.15.100:0/767964410":"2024-05-11T11:07:00.092732+0000","172.21.15.100:6801/1597433518":"2024-05-11T11:07:00.092732+0000","172.21.15.100:6801/1086450256":"2024-05-11T11:06:33.114176+0000","172.21.15.100:0/2770572070":"2024-05-11T11:06:33.114176+0000","172.21.15.100:0/1379340513":"2024-05-11T11:06:33.114176+0000","172.21.15.100:0/3025646953":"2024-05-11T11:07:00.092732+0000","172.21.15.100:0/4099258930":"2024-05-11T11:07:00.092732+0000","172.21.15.100:0/4267693567":"2024-05-11T11:06:33.114176+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T11:15:19.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:19.572+0000 7f9fc4185700 1 -- 172.21.15.100:0/2717414798 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f9fa80b2880 msgr2=0x7f9fa80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:19.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:19.572+0000 7f9fc4185700 1 --2- 172.21.15.100:0/2717414798 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f9fa80b2880 0x7f9fa80b4d40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8000c00 tx=0x7f9fb8005960 comp rx=0 tx=0).stop 2024-05-10T11:15:19.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:19.572+0000 7f9fc4185700 1 -- 172.21.15.100:0/2717414798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9fbc108400 msgr2=0x7f9fbc199800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:19.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:19.572+0000 7f9fc4185700 1 --2- 172.21.15.100:0/2717414798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9fbc108400 0x7f9fbc199800 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9fb400ada0 tx=0x7f9fb400c330 comp rx=0 tx=0).stop 2024-05-10T11:15:19.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:19.572+0000 7f9fc4185700 1 -- 172.21.15.100:0/2717414798 shutdown_connections 2024-05-10T11:15:19.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:19.572+0000 7f9fc4185700 1 --2- 172.21.15.100:0/2717414798 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f9fa80b2880 0x7f9fa80b4d40 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:19.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:19.572+0000 7f9fc4185700 1 --2- 172.21.15.100:0/2717414798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9fbc108400 0x7f9fbc199800 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:19.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:19.572+0000 7f9fc4185700 1 --2- 172.21.15.100:0/2717414798 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9fbc102390 0x7f9fbc19f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:19.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:19.572+0000 7f9fc4185700 1 --2- 172.21.15.100:0/2717414798 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9fbc102d50 0x7f9fbc19fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:19.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:19.572+0000 7f9fc4185700 1 -- 172.21.15.100:0/2717414798 >> 172.21.15.100:0/2717414798 conn(0x7f9fbc079310 msgr2=0x7f9fbc101700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:15:19.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:19.572+0000 7f9fc4185700 1 -- 172.21.15.100:0/2717414798 shutdown_connections 2024-05-10T11:15:19.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:19.572+0000 7f9fc4185700 1 -- 172.21.15.100:0/2717414798 wait complete. 2024-05-10T11:15:19.725 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:19 smithi100 ceph-mon[30296]: pgmap v328: 129 pgs: 129 active+clean; 581 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-10T11:15:19.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:19 smithi178 ceph-mon[33359]: pgmap v328: 129 pgs: 129 active+clean; 581 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-10T11:15:19.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:19 smithi188 ceph-mon[34075]: pgmap v328: 129 pgs: 129 active+clean; 581 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-10T11:15:20.430 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-10T11:15:20.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:20 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/2717414798' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T11:15:20.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:20 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:15:20.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:20 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:15:20.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:20 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:15:20.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:20 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:15:20.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:20 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/2717414798' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T11:15:20.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:20 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:15:20.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:20 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:15:20.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:20 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:15:20.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:20 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:15:20.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:20 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/2717414798' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T11:15:20.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:20 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:15:20.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:20 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:15:20.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:20 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:15:20.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:20 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:15:20.859 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:15:22.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:21 smithi178 ceph-mon[33359]: pgmap v329: 129 pgs: 129 active+clean; 581 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-10T11:15:22.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:21 smithi188 ceph-mon[34075]: pgmap v329: 129 pgs: 129 active+clean; 581 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-10T11:15:22.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:21 smithi100 ceph-mon[30296]: pgmap v329: 129 pgs: 129 active+clean; 581 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-10T11:15:22.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:22.540+0000 7f03ab445700 1 -- 172.21.15.100:0/1232258326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03a4108410 msgr2=0x7f03a41087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:22.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:22.540+0000 7f03ab445700 1 --2- 172.21.15.100:0/1232258326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03a4108410 0x7f03a41087f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f03a0009990 tx=0x7f03a0009ca0 comp rx=0 tx=0).stop 2024-05-10T11:15:22.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:22.540+0000 7f03ab445700 1 -- 172.21.15.100:0/1232258326 shutdown_connections 2024-05-10T11:15:22.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:22.540+0000 7f03ab445700 1 --2- 172.21.15.100:0/1232258326 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03a4102d60 0x7f03a410f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:22.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:22.540+0000 7f03ab445700 1 --2- 172.21.15.100:0/1232258326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03a4108410 0x7f03a41087f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:22.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:22.540+0000 7f03ab445700 1 --2- 172.21.15.100:0/1232258326 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f03a41023a0 0x7f03a4102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:22.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:22.540+0000 7f03ab445700 1 -- 172.21.15.100:0/1232258326 >> 172.21.15.100:0/1232258326 conn(0x7f03a4079350 msgr2=0x7f03a4079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:15:22.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:22.540+0000 7f03ab445700 1 -- 172.21.15.100:0/1232258326 shutdown_connections 2024-05-10T11:15:22.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:22.540+0000 7f03ab445700 1 -- 172.21.15.100:0/1232258326 wait complete. 2024-05-10T11:15:22.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:22.541+0000 7f03ab445700 1 Processor -- start 2024-05-10T11:15:22.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:22.541+0000 7f03ab445700 1 -- start start 2024-05-10T11:15:22.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:22.541+0000 7f03ab445700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f03a41023a0 0x7f03a419f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:15:22.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:22.541+0000 7f03ab445700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03a4102d60 0x7f03a419fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:15:22.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:22.541+0000 7f03ab445700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03a4108410 0x7f03a4199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:15:22.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:22.541+0000 7f03ab445700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f03a4113510 con 0x7f03a4108410 2024-05-10T11:15:22.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:22.541+0000 7f03ab445700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f03a4113390 con 0x7f03a41023a0 2024-05-10T11:15:22.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:22.541+0000 7f03ab445700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f03a4113690 con 0x7f03a4102d60 2024-05-10T11:15:22.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:22.542+0000 7f03a99e2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03a4108410 0x7f03a4199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:15:22.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:22.542+0000 7f03a99e2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03a4108410 0x7f03a4199870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37336/0 (socket says 172.21.15.100:37336) 2024-05-10T11:15:22.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:22.542+0000 7f03a99e2700 1 -- 172.21.15.100:0/4007779544 learned_addr learned my addr 172.21.15.100:0/4007779544 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:15:22.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:22.542+0000 7f03a89e0700 1 --2- 172.21.15.100:0/4007779544 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03a4102d60 0x7f03a419fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:15:22.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:22.542+0000 7f03a91e1700 1 --2- 172.21.15.100:0/4007779544 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f03a41023a0 0x7f03a419f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:15:22.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:22.542+0000 7f03a99e2700 1 -- 172.21.15.100:0/4007779544 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03a4102d60 msgr2=0x7f03a419fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:22.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:22.542+0000 7f03a99e2700 1 --2- 172.21.15.100:0/4007779544 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03a4102d60 0x7f03a419fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:22.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:22.542+0000 7f03a99e2700 1 -- 172.21.15.100:0/4007779544 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f03a41023a0 msgr2=0x7f03a419f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:22.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:22.542+0000 7f03a99e2700 1 --2- 172.21.15.100:0/4007779544 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f03a41023a0 0x7f03a419f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:22.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:22.542+0000 7f03a99e2700 1 -- 172.21.15.100:0/4007779544 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03a419a050 con 0x7f03a4108410 2024-05-10T11:15:22.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:22.542+0000 7f03a99e2700 1 --2- 172.21.15.100:0/4007779544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03a4108410 0x7f03a4199870 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f039c007ee0 tx=0x7f039c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:15:22.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:22.542+0000 7f039a7fc700 1 -- 172.21.15.100:0/4007779544 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f039c004bb0 con 0x7f03a4108410 2024-05-10T11:15:22.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:22.542+0000 7f039a7fc700 1 -- 172.21.15.100:0/4007779544 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f039c004d50 con 0x7f03a4108410 2024-05-10T11:15:22.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:22.542+0000 7f03ab445700 1 -- 172.21.15.100:0/4007779544 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03a419a340 con 0x7f03a4108410 2024-05-10T11:15:22.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:22.543+0000 7f039a7fc700 1 -- 172.21.15.100:0/4007779544 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f039c0058b0 con 0x7f03a4108410 2024-05-10T11:15:22.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:22.543+0000 7f03ab445700 1 -- 172.21.15.100:0/4007779544 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03a41a6500 con 0x7f03a4108410 2024-05-10T11:15:22.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:22.543+0000 7f03ab445700 1 -- 172.21.15.100:0/4007779544 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03a4103ac0 con 0x7f03a4108410 2024-05-10T11:15:22.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:22.545+0000 7f039a7fc700 1 -- 172.21.15.100:0/4007779544 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f039c01a020 con 0x7f03a4108410 2024-05-10T11:15:22.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:22.546+0000 7f039a7fc700 1 --2- 172.21.15.100:0/4007779544 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f03900b2900 0x7f03900b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:15:22.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:22.546+0000 7f03a91e1700 1 --2- 172.21.15.100:0/4007779544 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f03900b2900 0x7f03900b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:15:22.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:22.546+0000 7f039a7fc700 1 -- 172.21.15.100:0/4007779544 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f039c0e2150 con 0x7f03a4108410 2024-05-10T11:15:22.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:22.547+0000 7f03a91e1700 1 --2- 172.21.15.100:0/4007779544 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f03900b2900 0x7f03900b4dc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f03a000b5c0 tx=0x7f03a0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:15:22.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:22.550+0000 7f039a7fc700 1 -- 172.21.15.100:0/4007779544 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f039c010070 con 0x7f03a4108410 2024-05-10T11:15:22.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:22.766+0000 7f03ab445700 1 -- 172.21.15.100:0/4007779544 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f03a402e510 con 0x7f03a4108410 2024-05-10T11:15:22.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:22.768+0000 7f039a7fc700 1 -- 172.21.15.100:0/4007779544 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24737 (secure 0 0 0) 0x7f039c0a9b10 con 0x7f03a4108410 2024-05-10T11:15:22.768 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:15:22.769 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":89,"fsid":"20a542cc-0ebd-11ef-bc98-c7b262605968","created":"2024-05-10T11:06:05.454446+0000","modified":"2024-05-10T11:15:01.418144+0000","last_up_change":"2024-05-10T11:13:18.185471+0000","last_in_change":"2024-05-10T11:12:57.268678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T11:09:26.176441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T11:14:21.004285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T11:14:24.179414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T11:14:33.308517+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5f031418-da23-4e85-8669-160960dadd09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":643028278},{"type":"v1","addr":"172.21.15.100:6803","nonce":643028278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":643028278},{"type":"v1","addr":"172.21.15.100:6805","nonce":643028278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":643028278},{"type":"v1","addr":"172.21.15.100:6809","nonce":643028278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":643028278},{"type":"v1","addr":"172.21.15.100:6807","nonce":643028278}]},"public_addr":"172.21.15.100:6803/643028278","cluster_addr":"172.21.15.100:6805/643028278","heartbeat_back_addr":"172.21.15.100:6809/643028278","heartbeat_front_addr":"172.21.15.100:6807/643028278","state":["exists","up"]},{"osd":1,"uuid":"a908cdc9-001e-4b65-9ab0-0dd08dba20f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":3103987017},{"type":"v1","addr":"172.21.15.178:6801","nonce":3103987017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":3103987017},{"type":"v1","addr":"172.21.15.178:6803","nonce":3103987017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":3103987017},{"type":"v1","addr":"172.21.15.178:6807","nonce":3103987017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":3103987017},{"type":"v1","addr":"172.21.15.178:6805","nonce":3103987017}]},"public_addr":"172.21.15.178:6801/3103987017","cluster_addr":"172.21.15.178:6803/3103987017","heartbeat_back_addr":"172.21.15.178:6807/3103987017","heartbeat_front_addr":"172.21.15.178:6805/3103987017","state":["exists","up"]},{"osd":2,"uuid":"4d8f89ae-9424-46e9-9c80-17c47cc5e8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3991466468},{"type":"v1","addr":"172.21.15.188:6801","nonce":3991466468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3991466468},{"type":"v1","addr":"172.21.15.188:6803","nonce":3991466468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3991466468},{"type":"v1","addr":"172.21.15.188:6807","nonce":3991466468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3991466468},{"type":"v1","addr":"172.21.15.188:6805","nonce":3991466468}]},"public_addr":"172.21.15.188:6801/3991466468","cluster_addr":"172.21.15.188:6803/3991466468","heartbeat_back_addr":"172.21.15.188:6807/3991466468","heartbeat_front_addr":"172.21.15.188:6805/3991466468","state":["exists","up"]},{"osd":3,"uuid":"4c8865aa-fef4-4710-b766-1385adb9462c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1713357192},{"type":"v1","addr":"172.21.15.100:6811","nonce":1713357192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1713357192},{"type":"v1","addr":"172.21.15.100:6813","nonce":1713357192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1713357192},{"type":"v1","addr":"172.21.15.100:6817","nonce":1713357192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1713357192},{"type":"v1","addr":"172.21.15.100:6815","nonce":1713357192}]},"public_addr":"172.21.15.100:6811/1713357192","cluster_addr":"172.21.15.100:6813/1713357192","heartbeat_back_addr":"172.21.15.100:6817/1713357192","heartbeat_front_addr":"172.21.15.100:6815/1713357192","state":["exists","up"]},{"osd":4,"uuid":"a9653800-da57-43eb-8781-3188e8a3949f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":2222814192},{"type":"v1","addr":"172.21.15.178:6809","nonce":2222814192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":2222814192},{"type":"v1","addr":"172.21.15.178:6811","nonce":2222814192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":2222814192},{"type":"v1","addr":"172.21.15.178:6815","nonce":2222814192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":2222814192},{"type":"v1","addr":"172.21.15.178:6813","nonce":2222814192}]},"public_addr":"172.21.15.178:6809/2222814192","cluster_addr":"172.21.15.178:6811/2222814192","heartbeat_back_addr":"172.21.15.178:6815/2222814192","heartbeat_front_addr":"172.21.15.178:6813/2222814192","state":["exists","up"]},{"osd":5,"uuid":"fb5f8af2-1eb0-4d8a-a7a3-ed6dd68e692e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3517375935},{"type":"v1","addr":"172.21.15.188:6809","nonce":3517375935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3517375935},{"type":"v1","addr":"172.21.15.188:6811","nonce":3517375935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3517375935},{"type":"v1","addr":"172.21.15.188:6815","nonce":3517375935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3517375935},{"type":"v1","addr":"172.21.15.188:6813","nonce":3517375935}]},"public_addr":"172.21.15.188:6809/3517375935","cluster_addr":"172.21.15.188:6811/3517375935","heartbeat_back_addr":"172.21.15.188:6815/3517375935","heartbeat_front_addr":"172.21.15.188:6813/3517375935","state":["exists","up"]},{"osd":6,"uuid":"84b57387-0a94-4e49-b099-493d329b87dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1410593862},{"type":"v1","addr":"172.21.15.100:6819","nonce":1410593862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1410593862},{"type":"v1","addr":"172.21.15.100:6821","nonce":1410593862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1410593862},{"type":"v1","addr":"172.21.15.100:6825","nonce":1410593862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1410593862},{"type":"v1","addr":"172.21.15.100:6823","nonce":1410593862}]},"public_addr":"172.21.15.100:6819/1410593862","cluster_addr":"172.21.15.100:6821/1410593862","heartbeat_back_addr":"172.21.15.100:6825/1410593862","heartbeat_front_addr":"172.21.15.100:6823/1410593862","state":["exists","up"]},{"osd":7,"uuid":"02ada28f-ac4f-47cf-983f-8a6da1140abe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2458690455},{"type":"v1","addr":"172.21.15.178:6817","nonce":2458690455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2458690455},{"type":"v1","addr":"172.21.15.178:6819","nonce":2458690455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2458690455},{"type":"v1","addr":"172.21.15.178:6823","nonce":2458690455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2458690455},{"type":"v1","addr":"172.21.15.178:6821","nonce":2458690455}]},"public_addr":"172.21.15.178:6817/2458690455","cluster_addr":"172.21.15.178:6819/2458690455","heartbeat_back_addr":"172.21.15.178:6823/2458690455","heartbeat_front_addr":"172.21.15.178:6821/2458690455","state":["exists","up"]},{"osd":8,"uuid":"e7e489bc-15fc-4a4c-b297-d1bb626513ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1523872826},{"type":"v1","addr":"172.21.15.188:6817","nonce":1523872826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1523872826},{"type":"v1","addr":"172.21.15.188:6819","nonce":1523872826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1523872826},{"type":"v1","addr":"172.21.15.188:6823","nonce":1523872826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1523872826},{"type":"v1","addr":"172.21.15.188:6821","nonce":1523872826}]},"public_addr":"172.21.15.188:6817/1523872826","cluster_addr":"172.21.15.188:6819/1523872826","heartbeat_back_addr":"172.21.15.188:6823/1523872826","heartbeat_front_addr":"172.21.15.188:6821/1523872826","state":["exists","up"]},{"osd":9,"uuid":"72328326-cd16-45d9-bfd9-40f6c5c20f5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3585761978},{"type":"v1","addr":"172.21.15.100:6827","nonce":3585761978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3585761978},{"type":"v1","addr":"172.21.15.100:6829","nonce":3585761978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3585761978},{"type":"v1","addr":"172.21.15.100:6833","nonce":3585761978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3585761978},{"type":"v1","addr":"172.21.15.100:6831","nonce":3585761978}]},"public_addr":"172.21.15.100:6827/3585761978","cluster_addr":"172.21.15.100:6829/3585761978","heartbeat_back_addr":"172.21.15.100:6833/3585761978","heartbeat_front_addr":"172.21.15.100:6831/3585761978","state":["exists","up"]},{"osd":10,"uuid":"a355f9a8-91a3-41d4-814d-5a9c9b953706","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2985841725},{"type":"v1","addr":"172.21.15.178:6825","nonce":2985841725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2985841725},{"type":"v1","addr":"172.21.15.178:6827","nonce":2985841725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2985841725},{"type":"v1","addr":"172.21.15.178:6831","nonce":2985841725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2985841725},{"type":"v1","addr":"172.21.15.178:6829","nonce":2985841725}]},"public_addr":"172.21.15.178:6825/2985841725","cluster_addr":"172.21.15.178:6827/2985841725","heartbeat_back_addr":"172.21.15.178:6831/2985841725","heartbeat_front_addr":"172.21.15.178:6829/2985841725","state":["exists","up"]},{"osd":11,"uuid":"76a5e13c-7e38-42f9-bfa7-7fedec7c7184","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2574955118},{"type":"v1","addr":"172.21.15.188:6825","nonce":2574955118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2574955118},{"type":"v1","addr":"172.21.15.188:6827","nonce":2574955118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2574955118},{"type":"v1","addr":"172.21.15.188:6831","nonce":2574955118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2574955118},{"type":"v1","addr":"172.21.15.188:6829","nonce":2574955118}]},"public_addr":"172.21.15.188:6825/2574955118","cluster_addr":"172.21.15.188:6827/2574955118","heartbeat_back_addr":"172.21.15.188:6831/2574955118","heartbeat_front_addr":"172.21.15.188:6829/2574955118","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:08:36.086637+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:08:58.948031+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:09:22.142517+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:09:47.655748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:10:12.853999+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:10:37.911383+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:11:04.448815+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:11:30.331740+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:11:55.819278+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:12:23.204612+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:12:49.484835+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:13:15.783531+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:6800/1597433518":"2024-05-11T11:07:00.092732+0000","172.21.15.100:6800/1086450256":"2024-05-11T11:06:33.114176+0000","172.21.15.100:0/767964410":"2024-05-11T11:07:00.092732+0000","172.21.15.100:6801/1597433518":"2024-05-11T11:07:00.092732+0000","172.21.15.100:6801/1086450256":"2024-05-11T11:06:33.114176+0000","172.21.15.100:0/2770572070":"2024-05-11T11:06:33.114176+0000","172.21.15.100:0/1379340513":"2024-05-11T11:06:33.114176+0000","172.21.15.100:0/3025646953":"2024-05-11T11:07:00.092732+0000","172.21.15.100:0/4099258930":"2024-05-11T11:07:00.092732+0000","172.21.15.100:0/4267693567":"2024-05-11T11:06:33.114176+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T11:15:22.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:22.770+0000 7f03ab445700 1 -- 172.21.15.100:0/4007779544 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f03900b2900 msgr2=0x7f03900b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:22.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:22.770+0000 7f03ab445700 1 --2- 172.21.15.100:0/4007779544 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f03900b2900 0x7f03900b4dc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f03a000b5c0 tx=0x7f03a0005960 comp rx=0 tx=0).stop 2024-05-10T11:15:22.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:22.770+0000 7f03ab445700 1 -- 172.21.15.100:0/4007779544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03a4108410 msgr2=0x7f03a4199870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:22.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:22.770+0000 7f03ab445700 1 --2- 172.21.15.100:0/4007779544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03a4108410 0x7f03a4199870 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f039c007ee0 tx=0x7f039c00d3b0 comp rx=0 tx=0).stop 2024-05-10T11:15:22.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:22.770+0000 7f03ab445700 1 -- 172.21.15.100:0/4007779544 shutdown_connections 2024-05-10T11:15:22.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:22.770+0000 7f03ab445700 1 --2- 172.21.15.100:0/4007779544 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f03900b2900 0x7f03900b4dc0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:22.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:22.770+0000 7f03ab445700 1 --2- 172.21.15.100:0/4007779544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03a4108410 0x7f03a4199870 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:22.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:22.770+0000 7f03ab445700 1 --2- 172.21.15.100:0/4007779544 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f03a41023a0 0x7f03a419f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:22.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:22.770+0000 7f03ab445700 1 --2- 172.21.15.100:0/4007779544 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03a4102d60 0x7f03a419fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:22.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:22.770+0000 7f03ab445700 1 -- 172.21.15.100:0/4007779544 >> 172.21.15.100:0/4007779544 conn(0x7f03a4079350 msgr2=0x7f03a41010d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:15:22.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:22.771+0000 7f03ab445700 1 -- 172.21.15.100:0/4007779544 shutdown_connections 2024-05-10T11:15:22.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:22.771+0000 7f03ab445700 1 -- 172.21.15.100:0/4007779544 wait complete. 2024-05-10T11:15:23.427 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-05-10T11:15:23.679 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:15:23.957 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:23 smithi100 ceph-mon[30296]: pgmap v330: 129 pgs: 129 active+clean; 581 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-10T11:15:23.957 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:23 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:15:23.957 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:23 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:15:23.957 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:23 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/4007779544' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T11:15:23.957 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:23 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:15:23.957 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:23 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:15:23.957 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:23 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:15:24.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:23 smithi178 ceph-mon[33359]: pgmap v330: 129 pgs: 129 active+clean; 581 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-10T11:15:24.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:23 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:15:24.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:23 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:15:24.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:23 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/4007779544' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T11:15:24.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:23 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:15:24.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:23 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:15:24.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:23 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:15:24.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:23 smithi188 ceph-mon[34075]: pgmap v330: 129 pgs: 129 active+clean; 581 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-10T11:15:24.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:23 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:15:24.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:23 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:15:24.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:23 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/4007779544' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T11:15:24.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:23 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:15:24.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:23 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:15:24.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:23 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:15:25.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:25.435+0000 7f3dfb228700 1 -- 172.21.15.100:0/1980520166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3df41037b0 msgr2=0x7f3df4103b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:25.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:25.435+0000 7f3dfb228700 1 --2- 172.21.15.100:0/1980520166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3df41037b0 0x7f3df4103b90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3de80099c0 tx=0x7f3de8009cd0 comp rx=0 tx=0).stop 2024-05-10T11:15:25.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:25.435+0000 7f3dfb228700 1 -- 172.21.15.100:0/1980520166 shutdown_connections 2024-05-10T11:15:25.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:25.435+0000 7f3dfb228700 1 --2- 172.21.15.100:0/1980520166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3df4108880 0x7f3df4110400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:25.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:25.435+0000 7f3dfb228700 1 --2- 172.21.15.100:0/1980520166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3df41037b0 0x7f3df4103b90 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:25.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:25.435+0000 7f3dfb228700 1 --2- 172.21.15.100:0/1980520166 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3df4104160 0x7f3df4108150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:25.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:25.435+0000 7f3dfb228700 1 -- 172.21.15.100:0/1980520166 >> 172.21.15.100:0/1980520166 conn(0x7f3df407a530 msgr2=0x7f3df407a940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:15:25.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:25.435+0000 7f3dfb228700 1 -- 172.21.15.100:0/1980520166 shutdown_connections 2024-05-10T11:15:25.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:25.436+0000 7f3dfb228700 1 -- 172.21.15.100:0/1980520166 wait complete. 2024-05-10T11:15:25.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:25.437+0000 7f3dfb228700 1 Processor -- start 2024-05-10T11:15:25.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:25.437+0000 7f3dfb228700 1 -- start start 2024-05-10T11:15:25.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:25.437+0000 7f3dfb228700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3df41037b0 0x7f3df419f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:15:25.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:25.437+0000 7f3dfb228700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3df4104160 0x7f3df419fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:15:25.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:25.437+0000 7f3dfb228700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3df4108880 0x7f3df4199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:15:25.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:25.437+0000 7f3dfb228700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3df4113520 con 0x7f3df4108880 2024-05-10T11:15:25.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:25.437+0000 7f3dfb228700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3df41133a0 con 0x7f3df4104160 2024-05-10T11:15:25.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:25.437+0000 7f3dfb228700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3df41136a0 con 0x7f3df41037b0 2024-05-10T11:15:25.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:25.437+0000 7f3df97c5700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3df4108880 0x7f3df4199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:15:25.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:25.437+0000 7f3df97c5700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3df4108880 0x7f3df4199800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37362/0 (socket says 172.21.15.100:37362) 2024-05-10T11:15:25.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:25.437+0000 7f3df97c5700 1 -- 172.21.15.100:0/3935332835 learned_addr learned my addr 172.21.15.100:0/3935332835 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:15:25.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:25.437+0000 7f3df3fff700 1 --2- 172.21.15.100:0/3935332835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3df4104160 0x7f3df419fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:15:25.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:25.437+0000 7f3df8fc4700 1 --2- 172.21.15.100:0/3935332835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3df41037b0 0x7f3df419f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:15:25.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:25.437+0000 7f3df97c5700 1 -- 172.21.15.100:0/3935332835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3df41037b0 msgr2=0x7f3df419f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:25.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:25.437+0000 7f3df97c5700 1 --2- 172.21.15.100:0/3935332835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3df41037b0 0x7f3df419f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:25.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:25.437+0000 7f3df97c5700 1 -- 172.21.15.100:0/3935332835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3df4104160 msgr2=0x7f3df419fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:25.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:25.437+0000 7f3df97c5700 1 --2- 172.21.15.100:0/3935332835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3df4104160 0x7f3df419fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:25.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:25.437+0000 7f3df97c5700 1 -- 172.21.15.100:0/3935332835 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3df4199fe0 con 0x7f3df4108880 2024-05-10T11:15:25.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:25.437+0000 7f3df8fc4700 1 --2- 172.21.15.100:0/3935332835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3df41037b0 0x7f3df419f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:15:25.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:25.438+0000 7f3df97c5700 1 --2- 172.21.15.100:0/3935332835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3df4108880 0x7f3df4199800 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3dec007ee0 tx=0x7f3dec00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:15:25.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:25.438+0000 7f3df1ffb700 1 -- 172.21.15.100:0/3935332835 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dec00dbd0 con 0x7f3df4108880 2024-05-10T11:15:25.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:25.438+0000 7f3df1ffb700 1 -- 172.21.15.100:0/3935332835 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3dec00dd70 con 0x7f3df4108880 2024-05-10T11:15:25.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:25.438+0000 7f3dfb228700 1 -- 172.21.15.100:0/3935332835 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3df419a2d0 con 0x7f3df4108880 2024-05-10T11:15:25.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:25.438+0000 7f3df1ffb700 1 -- 172.21.15.100:0/3935332835 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dec0058d0 con 0x7f3df4108880 2024-05-10T11:15:25.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:25.438+0000 7f3dfb228700 1 -- 172.21.15.100:0/3935332835 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3df41a64e0 con 0x7f3df4108880 2024-05-10T11:15:25.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:25.439+0000 7f3dfb228700 1 -- 172.21.15.100:0/3935332835 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3df410bac0 con 0x7f3df4108880 2024-05-10T11:15:25.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:25.440+0000 7f3df1ffb700 1 -- 172.21.15.100:0/3935332835 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f3dec01a020 con 0x7f3df4108880 2024-05-10T11:15:25.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:25.441+0000 7f3df1ffb700 1 --2- 172.21.15.100:0/3935332835 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f3de40b2900 0x7f3de40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:15:25.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:25.441+0000 7f3df8fc4700 1 --2- 172.21.15.100:0/3935332835 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f3de40b2900 0x7f3de40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:15:25.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:25.441+0000 7f3df1ffb700 1 -- 172.21.15.100:0/3935332835 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f3dec0e1cd0 con 0x7f3df4108880 2024-05-10T11:15:25.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:25.441+0000 7f3df8fc4700 1 --2- 172.21.15.100:0/3935332835 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f3de40b2900 0x7f3de40b4dc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3de8005b40 tx=0x7f3de8005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:15:25.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:25.445+0000 7f3df1ffb700 1 -- 172.21.15.100:0/3935332835 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f3dec010070 con 0x7f3df4108880 2024-05-10T11:15:25.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:25.694+0000 7f3dfb228700 1 -- 172.21.15.100:0/3935332835 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f3df4104c10 con 0x7f3df4108880 2024-05-10T11:15:25.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:25.709+0000 7f3df1ffb700 1 -- 172.21.15.100:0/3935332835 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7f3dec0a9690 con 0x7f3df4108880 2024-05-10T11:15:25.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:25.711+0000 7f3dfb228700 1 -- 172.21.15.100:0/3935332835 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f3de40b2900 msgr2=0x7f3de40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:25.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:25.711+0000 7f3dfb228700 1 --2- 172.21.15.100:0/3935332835 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f3de40b2900 0x7f3de40b4dc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3de8005b40 tx=0x7f3de8005a90 comp rx=0 tx=0).stop 2024-05-10T11:15:25.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:25.711+0000 7f3dfb228700 1 -- 172.21.15.100:0/3935332835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3df4108880 msgr2=0x7f3df4199800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:25.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:25.711+0000 7f3dfb228700 1 --2- 172.21.15.100:0/3935332835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3df4108880 0x7f3df4199800 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3dec007ee0 tx=0x7f3dec00d3b0 comp rx=0 tx=0).stop 2024-05-10T11:15:25.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:25.711+0000 7f3dfb228700 1 -- 172.21.15.100:0/3935332835 shutdown_connections 2024-05-10T11:15:25.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:25.711+0000 7f3dfb228700 1 --2- 172.21.15.100:0/3935332835 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f3de40b2900 0x7f3de40b4dc0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:25.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:25.711+0000 7f3dfb228700 1 --2- 172.21.15.100:0/3935332835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3df4108880 0x7f3df4199800 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:25.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:25.711+0000 7f3dfb228700 1 --2- 172.21.15.100:0/3935332835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3df41037b0 0x7f3df419f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:25.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:25.711+0000 7f3dfb228700 1 --2- 172.21.15.100:0/3935332835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3df4104160 0x7f3df419fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:25.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:25.711+0000 7f3dfb228700 1 -- 172.21.15.100:0/3935332835 >> 172.21.15.100:0/3935332835 conn(0x7f3df407a530 msgr2=0x7f3df4101810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:15:25.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:25.712+0000 7f3dfb228700 1 -- 172.21.15.100:0/3935332835 shutdown_connections 2024-05-10T11:15:25.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:25.712+0000 7f3dfb228700 1 -- 172.21.15.100:0/3935332835 wait complete. 2024-05-10T11:15:26.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:25 smithi178 ceph-mon[33359]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:15:26.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:25 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/3935332835' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-10T11:15:26.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:25 smithi188 ceph-mon[34075]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:15:26.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:25 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/3935332835' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-10T11:15:26.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:25 smithi100 ceph-mon[30296]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:15:26.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:25 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/3935332835' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-10T11:15:26.448 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-05-10T11:15:26.449 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-05-10T11:15:26.694 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:15:26.974 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:26 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/3935332835' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-10T11:15:26.974 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:26 smithi100 ceph-mon[30296]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-10T11:15:27.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:26 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/3935332835' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-10T11:15:27.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:26 smithi178 ceph-mon[33359]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-10T11:15:27.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:26 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/3935332835' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-10T11:15:27.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:26 smithi188 ceph-mon[34075]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-10T11:15:28.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:27 smithi178 ceph-mon[33359]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:15:28.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:27 smithi188 ceph-mon[34075]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:15:28.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:27 smithi100 ceph-mon[30296]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:15:28.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:28.462+0000 7fc94ba76700 1 -- 172.21.15.100:0/960346934 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc944103750 msgr2=0x7fc944103b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:28.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:28.462+0000 7fc94ba76700 1 --2- 172.21.15.100:0/960346934 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc944103750 0x7fc944103b30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc9400099c0 tx=0x7fc940009cd0 comp rx=0 tx=0).stop 2024-05-10T11:15:28.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:28.463+0000 7fc94ba76700 1 -- 172.21.15.100:0/960346934 shutdown_connections 2024-05-10T11:15:28.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:28.463+0000 7fc94ba76700 1 --2- 172.21.15.100:0/960346934 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc944108850 0x7fc9441103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:28.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:28.463+0000 7fc94ba76700 1 --2- 172.21.15.100:0/960346934 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc944103750 0x7fc944103b30 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:28.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:28.463+0000 7fc94ba76700 1 --2- 172.21.15.100:0/960346934 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc944104100 0x7fc944108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:28.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:28.463+0000 7fc94ba76700 1 -- 172.21.15.100:0/960346934 >> 172.21.15.100:0/960346934 conn(0x7fc94407a600 msgr2=0x7fc94407aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:15:28.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:28.463+0000 7fc94ba76700 1 -- 172.21.15.100:0/960346934 shutdown_connections 2024-05-10T11:15:28.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:28.463+0000 7fc94ba76700 1 -- 172.21.15.100:0/960346934 wait complete. 2024-05-10T11:15:28.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:28.463+0000 7fc94ba76700 1 Processor -- start 2024-05-10T11:15:28.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:28.463+0000 7fc94ba76700 1 -- start start 2024-05-10T11:15:28.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:28.463+0000 7fc94ba76700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc944103750 0x7fc94419f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:15:28.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:28.464+0000 7fc94ba76700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc944104100 0x7fc94419fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:15:28.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:28.464+0000 7fc94ba76700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc944108850 0x7fc9441998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:15:28.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:28.464+0000 7fc94ba76700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc9441134f0 con 0x7fc944104100 2024-05-10T11:15:28.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:28.464+0000 7fc94ba76700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc944113370 con 0x7fc944103750 2024-05-10T11:15:28.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:28.464+0000 7fc94ba76700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc944113670 con 0x7fc944108850 2024-05-10T11:15:28.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:28.464+0000 7fc949011700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc944104100 0x7fc94419fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:15:28.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:28.464+0000 7fc949011700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc944104100 0x7fc94419fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54678/0 (socket says 172.21.15.100:54678) 2024-05-10T11:15:28.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:28.464+0000 7fc949011700 1 -- 172.21.15.100:0/241077649 learned_addr learned my addr 172.21.15.100:0/241077649 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:15:28.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:28.464+0000 7fc949812700 1 --2- 172.21.15.100:0/241077649 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc944103750 0x7fc94419f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:15:28.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:28.464+0000 7fc94a013700 1 --2- 172.21.15.100:0/241077649 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc944108850 0x7fc9441998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:15:28.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:28.464+0000 7fc949011700 1 -- 172.21.15.100:0/241077649 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc944108850 msgr2=0x7fc9441998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:28.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:28.464+0000 7fc949011700 1 --2- 172.21.15.100:0/241077649 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc944108850 0x7fc9441998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:28.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:28.464+0000 7fc949011700 1 -- 172.21.15.100:0/241077649 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc944103750 msgr2=0x7fc94419f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:28.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:28.464+0000 7fc949011700 1 --2- 172.21.15.100:0/241077649 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc944103750 0x7fc94419f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:28.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:28.464+0000 7fc949011700 1 -- 172.21.15.100:0/241077649 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc94419a010 con 0x7fc944104100 2024-05-10T11:15:28.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:28.464+0000 7fc949011700 1 --2- 172.21.15.100:0/241077649 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc944104100 0x7fc94419fc30 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc93400c970 tx=0x7fc93400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:15:28.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:28.464+0000 7fc93affd700 1 -- 172.21.15.100:0/241077649 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc934007aa0 con 0x7fc944104100 2024-05-10T11:15:28.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:28.464+0000 7fc93affd700 1 -- 172.21.15.100:0/241077649 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc934004cc0 con 0x7fc944104100 2024-05-10T11:15:28.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:28.464+0000 7fc94ba76700 1 -- 172.21.15.100:0/241077649 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc94419a300 con 0x7fc944104100 2024-05-10T11:15:28.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:28.464+0000 7fc93affd700 1 -- 172.21.15.100:0/241077649 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc934005450 con 0x7fc944104100 2024-05-10T11:15:28.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:28.464+0000 7fc94ba76700 1 -- 172.21.15.100:0/241077649 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9441a65c0 con 0x7fc944104100 2024-05-10T11:15:28.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:28.465+0000 7fc94ba76700 1 -- 172.21.15.100:0/241077649 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc94410ba90 con 0x7fc944104100 2024-05-10T11:15:28.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:28.467+0000 7fc93affd700 1 -- 172.21.15.100:0/241077649 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fc934007c40 con 0x7fc944104100 2024-05-10T11:15:28.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:28.467+0000 7fc93affd700 1 --2- 172.21.15.100:0/241077649 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc9300b2880 0x7fc9300b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:15:28.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:28.467+0000 7fc949812700 1 --2- 172.21.15.100:0/241077649 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc9300b2880 0x7fc9300b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:15:28.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:28.467+0000 7fc93affd700 1 -- 172.21.15.100:0/241077649 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fc9340e2350 con 0x7fc944104100 2024-05-10T11:15:28.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:28.468+0000 7fc949812700 1 --2- 172.21.15.100:0/241077649 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc9300b2880 0x7fc9300b4d40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc940009990 tx=0x7fc94001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:15:28.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:28.472+0000 7fc93affd700 1 -- 172.21.15.100:0/241077649 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fc9340a9da0 con 0x7fc944104100 2024-05-10T11:15:28.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:28.721+0000 7fc94ba76700 1 -- 172.21.15.100:0/241077649 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7fc94410b470 con 0x7fc944104100 2024-05-10T11:15:28.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:28.772+0000 7fc93affd700 1 -- 172.21.15.100:0/241077649 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7fc9340a96a0 con 0x7fc944104100 2024-05-10T11:15:28.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:28.775+0000 7fc94ba76700 1 -- 172.21.15.100:0/241077649 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc9300b2880 msgr2=0x7fc9300b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:28.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:28.775+0000 7fc94ba76700 1 --2- 172.21.15.100:0/241077649 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc9300b2880 0x7fc9300b4d40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc940009990 tx=0x7fc94001a040 comp rx=0 tx=0).stop 2024-05-10T11:15:28.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:28.775+0000 7fc94ba76700 1 -- 172.21.15.100:0/241077649 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc944104100 msgr2=0x7fc94419fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:28.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:28.775+0000 7fc94ba76700 1 --2- 172.21.15.100:0/241077649 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc944104100 0x7fc94419fc30 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc93400c970 tx=0x7fc93400cc80 comp rx=0 tx=0).stop 2024-05-10T11:15:28.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:28.775+0000 7fc94ba76700 1 -- 172.21.15.100:0/241077649 shutdown_connections 2024-05-10T11:15:28.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:28.775+0000 7fc94ba76700 1 --2- 172.21.15.100:0/241077649 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc9300b2880 0x7fc9300b4d40 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:28.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:28.775+0000 7fc94ba76700 1 --2- 172.21.15.100:0/241077649 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc944108850 0x7fc9441998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:28.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:28.775+0000 7fc94ba76700 1 --2- 172.21.15.100:0/241077649 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc944103750 0x7fc94419f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:28.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:28.775+0000 7fc94ba76700 1 --2- 172.21.15.100:0/241077649 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc944104100 0x7fc94419fc30 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:28.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:28.775+0000 7fc94ba76700 1 -- 172.21.15.100:0/241077649 >> 172.21.15.100:0/241077649 conn(0x7fc94407a600 msgr2=0x7fc944101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:15:28.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:28.775+0000 7fc94ba76700 1 -- 172.21.15.100:0/241077649 shutdown_connections 2024-05-10T11:15:28.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:28.775+0000 7fc94ba76700 1 -- 172.21.15.100:0/241077649 wait complete. 2024-05-10T11:15:29.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:28 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/241077649' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-10T11:15:29.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:28 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/241077649' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-10T11:15:29.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:28 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/241077649' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-10T11:15:29.422 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-05-10T11:15:29.666 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:15:30.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:29 smithi100 ceph-mon[30296]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:15:30.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:29 smithi100 ceph-mon[30296]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-10T11:15:30.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:29 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/241077649' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-10T11:15:30.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:29 smithi100 ceph-mon[30296]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-10T11:15:30.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:29 smithi100 ceph-mon[30296]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-10T11:15:30.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:29 smithi100 ceph-mon[30296]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-05-10T11:15:30.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:29 smithi100 ceph-mon[30296]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-05-10T11:15:30.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:29 smithi178 ceph-mon[33359]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:15:30.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:29 smithi178 ceph-mon[33359]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-10T11:15:30.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:29 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/241077649' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-10T11:15:30.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:29 smithi178 ceph-mon[33359]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-10T11:15:30.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:29 smithi178 ceph-mon[33359]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-10T11:15:30.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:29 smithi178 ceph-mon[33359]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-05-10T11:15:30.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:29 smithi178 ceph-mon[33359]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-05-10T11:15:30.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:29 smithi188 ceph-mon[34075]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:15:30.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:29 smithi188 ceph-mon[34075]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-10T11:15:30.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:29 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/241077649' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-10T11:15:30.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:29 smithi188 ceph-mon[34075]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-10T11:15:30.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:29 smithi188 ceph-mon[34075]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-10T11:15:30.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:29 smithi188 ceph-mon[34075]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-05-10T11:15:30.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:29 smithi188 ceph-mon[34075]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-05-10T11:15:30.979 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:30 smithi100 ceph-mon[30296]: mds.? [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] up:active 2024-05-10T11:15:30.979 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:30 smithi100 ceph-mon[30296]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-10T11:15:30.979 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:30 smithi100 ceph-mon[30296]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-10T11:15:30.979 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:30 smithi100 ceph-mon[30296]: Cluster is now healthy 2024-05-10T11:15:30.979 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:30 smithi100 ceph-mon[30296]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-05-10T11:15:30.979 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:30 smithi100 ceph-mon[30296]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-05-10T11:15:30.979 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:30 smithi100 ceph-mon[30296]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-05-10T11:15:31.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:30 smithi178 ceph-mon[33359]: mds.? [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] up:active 2024-05-10T11:15:31.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:30 smithi178 ceph-mon[33359]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-10T11:15:31.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:30 smithi178 ceph-mon[33359]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-10T11:15:31.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:30 smithi178 ceph-mon[33359]: Cluster is now healthy 2024-05-10T11:15:31.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:30 smithi178 ceph-mon[33359]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-05-10T11:15:31.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:30 smithi178 ceph-mon[33359]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-05-10T11:15:31.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:30 smithi178 ceph-mon[33359]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-05-10T11:15:31.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:30 smithi188 ceph-mon[34075]: mds.? [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] up:active 2024-05-10T11:15:31.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:30 smithi188 ceph-mon[34075]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-10T11:15:31.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:30 smithi188 ceph-mon[34075]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-10T11:15:31.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:30 smithi188 ceph-mon[34075]: Cluster is now healthy 2024-05-10T11:15:31.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:30 smithi188 ceph-mon[34075]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-05-10T11:15:31.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:30 smithi188 ceph-mon[34075]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-05-10T11:15:31.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:30 smithi188 ceph-mon[34075]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-05-10T11:15:31.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:31.539+0000 7f62addea700 1 -- 172.21.15.100:0/1080469276 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62a8109500 msgr2=0x7f62a81098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:31.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:31.539+0000 7f62addea700 1 --2- 172.21.15.100:0/1080469276 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62a8109500 0x7f62a81098e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f629800b0a0 tx=0x7f629800b3b0 comp rx=0 tx=0).stop 2024-05-10T11:15:31.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:31.539+0000 7f62addea700 1 -- 172.21.15.100:0/1080469276 shutdown_connections 2024-05-10T11:15:31.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:31.539+0000 7f62addea700 1 --2- 172.21.15.100:0/1080469276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f62a806c3d0 0x7f62a8110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:31.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:31.539+0000 7f62addea700 1 --2- 172.21.15.100:0/1080469276 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62a8109500 0x7f62a81098e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:31.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:31.539+0000 7f62addea700 1 --2- 172.21.15.100:0/1080469276 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62a806ba10 0x7f62a806be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:31.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:31.539+0000 7f62addea700 1 -- 172.21.15.100:0/1080469276 >> 172.21.15.100:0/1080469276 conn(0x7f62a8079310 msgr2=0x7f62a8079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:15:31.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:31.539+0000 7f62addea700 1 -- 172.21.15.100:0/1080469276 shutdown_connections 2024-05-10T11:15:31.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:31.539+0000 7f62addea700 1 -- 172.21.15.100:0/1080469276 wait complete. 2024-05-10T11:15:31.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:31.540+0000 7f62addea700 1 Processor -- start 2024-05-10T11:15:31.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:31.540+0000 7f62addea700 1 -- start start 2024-05-10T11:15:31.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:31.540+0000 7f62addea700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f62a806ba10 0x7f62a819f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:15:31.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:31.540+0000 7f62addea700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62a806c3d0 0x7f62a819fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:15:31.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:31.540+0000 7f62addea700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62a8109500 0x7f62a8199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:15:31.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:31.540+0000 7f62addea700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f62a81134c0 con 0x7f62a8109500 2024-05-10T11:15:31.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:31.540+0000 7f62addea700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f62a8113340 con 0x7f62a806ba10 2024-05-10T11:15:31.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:31.541+0000 7f62addea700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f62a8113640 con 0x7f62a806c3d0 2024-05-10T11:15:31.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:31.541+0000 7f62a7fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62a8109500 0x7f62a8199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:15:31.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:31.541+0000 7f62a6ffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62a806c3d0 0x7f62a819fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:15:31.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:31.541+0000 7f62a7fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62a8109500 0x7f62a8199940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54694/0 (socket says 172.21.15.100:54694) 2024-05-10T11:15:31.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:31.541+0000 7f62a7fff700 1 -- 172.21.15.100:0/2463160128 learned_addr learned my addr 172.21.15.100:0/2463160128 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:15:31.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:31.541+0000 7f62a77fe700 1 --2- 172.21.15.100:0/2463160128 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f62a806ba10 0x7f62a819f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:15:31.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:31.541+0000 7f62a6ffd700 1 -- 172.21.15.100:0/2463160128 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f62a806ba10 msgr2=0x7f62a819f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:31.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:31.541+0000 7f62a6ffd700 1 --2- 172.21.15.100:0/2463160128 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f62a806ba10 0x7f62a819f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:31.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:31.541+0000 7f62a6ffd700 1 -- 172.21.15.100:0/2463160128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62a8109500 msgr2=0x7f62a8199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:31.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:31.541+0000 7f62a6ffd700 1 --2- 172.21.15.100:0/2463160128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62a8109500 0x7f62a8199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:31.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:31.541+0000 7f62a6ffd700 1 -- 172.21.15.100:0/2463160128 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62a819a0a0 con 0x7f62a806c3d0 2024-05-10T11:15:31.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:31.541+0000 7f62a7fff700 1 --2- 172.21.15.100:0/2463160128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62a8109500 0x7f62a8199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:15:31.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:31.541+0000 7f62a6ffd700 1 --2- 172.21.15.100:0/2463160128 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62a806c3d0 0x7f62a819fcc0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f629c00e9f0 tx=0x7f629c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:15:31.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:31.541+0000 7f62a4ff9700 1 -- 172.21.15.100:0/2463160128 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f629c00c9c0 con 0x7f62a806c3d0 2024-05-10T11:15:31.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:31.541+0000 7f62a4ff9700 1 -- 172.21.15.100:0/2463160128 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f629c010430 con 0x7f62a806c3d0 2024-05-10T11:15:31.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:31.541+0000 7f62addea700 1 -- 172.21.15.100:0/2463160128 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62a819a330 con 0x7f62a806c3d0 2024-05-10T11:15:31.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:31.542+0000 7f62a4ff9700 1 -- 172.21.15.100:0/2463160128 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f629c018680 con 0x7f62a806c3d0 2024-05-10T11:15:31.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:31.542+0000 7f62addea700 1 -- 172.21.15.100:0/2463160128 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62a81a6650 con 0x7f62a806c3d0 2024-05-10T11:15:31.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:31.543+0000 7f62addea700 1 -- 172.21.15.100:0/2463160128 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62a810b9e0 con 0x7f62a806c3d0 2024-05-10T11:15:31.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:31.544+0000 7f62a4ff9700 1 -- 172.21.15.100:0/2463160128 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f629c00cb60 con 0x7f62a806c3d0 2024-05-10T11:15:31.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:31.545+0000 7f62a4ff9700 1 --2- 172.21.15.100:0/2463160128 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f62900b2880 0x7f62900b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:15:31.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:31.545+0000 7f62a77fe700 1 --2- 172.21.15.100:0/2463160128 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f62900b2880 0x7f62900b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:15:31.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:31.545+0000 7f62a4ff9700 1 -- 172.21.15.100:0/2463160128 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f629c0e2650 con 0x7f62a806c3d0 2024-05-10T11:15:31.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:31.545+0000 7f62a77fe700 1 --2- 172.21.15.100:0/2463160128 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f62900b2880 0x7f62900b4d40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6298009e60 tx=0x7f629800b700 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:15:31.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:31.548+0000 7f62a4ff9700 1 -- 172.21.15.100:0/2463160128 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f629c0aa010 con 0x7f62a806c3d0 2024-05-10T11:15:31.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:31.798+0000 7f62addea700 1 -- 172.21.15.100:0/2463160128 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f62a806a190 con 0x7f62a806c3d0 2024-05-10T11:15:32.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:31 smithi178 ceph-mon[33359]: pgmap v334: 129 pgs: 129 active+clean; 583 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 1 op/s 2024-05-10T11:15:32.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:31 smithi178 ceph-mon[33359]: mds.? [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] up:active 2024-05-10T11:15:32.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:31 smithi178 ceph-mon[33359]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-05-10T11:15:32.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:31 smithi188 ceph-mon[34075]: pgmap v334: 129 pgs: 129 active+clean; 583 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 1 op/s 2024-05-10T11:15:32.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:31 smithi188 ceph-mon[34075]: mds.? [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] up:active 2024-05-10T11:15:32.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:31 smithi188 ceph-mon[34075]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-05-10T11:15:32.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:31 smithi100 ceph-mon[30296]: pgmap v334: 129 pgs: 129 active+clean; 583 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 1 op/s 2024-05-10T11:15:32.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:31 smithi100 ceph-mon[30296]: mds.? [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] up:active 2024-05-10T11:15:32.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:31 smithi100 ceph-mon[30296]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-05-10T11:15:32.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:32.803+0000 7f62a4ff9700 1 -- 172.21.15.100:0/2463160128 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v24) ==== 121+0+0 (secure 0 0 0) 0x7f629c0b2090 con 0x7f62a806c3d0 2024-05-10T11:15:32.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:32.806+0000 7f62addea700 1 -- 172.21.15.100:0/2463160128 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f62900b2880 msgr2=0x7f62900b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:32.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:32.806+0000 7f62addea700 1 --2- 172.21.15.100:0/2463160128 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f62900b2880 0x7f62900b4d40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6298009e60 tx=0x7f629800b700 comp rx=0 tx=0).stop 2024-05-10T11:15:32.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:32.806+0000 7f62addea700 1 -- 172.21.15.100:0/2463160128 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62a806c3d0 msgr2=0x7f62a819fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:32.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:32.806+0000 7f62addea700 1 --2- 172.21.15.100:0/2463160128 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62a806c3d0 0x7f62a819fcc0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f629c00e9f0 tx=0x7f629c00ed00 comp rx=0 tx=0).stop 2024-05-10T11:15:32.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:32.806+0000 7f62addea700 1 -- 172.21.15.100:0/2463160128 shutdown_connections 2024-05-10T11:15:32.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:32.806+0000 7f62addea700 1 --2- 172.21.15.100:0/2463160128 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f62900b2880 0x7f62900b4d40 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:32.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:32.806+0000 7f62addea700 1 --2- 172.21.15.100:0/2463160128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62a8109500 0x7f62a8199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:32.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:32.806+0000 7f62addea700 1 --2- 172.21.15.100:0/2463160128 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f62a806ba10 0x7f62a819f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:32.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:32.806+0000 7f62addea700 1 --2- 172.21.15.100:0/2463160128 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62a806c3d0 0x7f62a819fcc0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:32.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:32.806+0000 7f62addea700 1 -- 172.21.15.100:0/2463160128 >> 172.21.15.100:0/2463160128 conn(0x7f62a8079310 msgr2=0x7f62a8101230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:15:32.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:32.807+0000 7f62addea700 1 -- 172.21.15.100:0/2463160128 shutdown_connections 2024-05-10T11:15:32.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:32.807+0000 7f62addea700 1 -- 172.21.15.100:0/2463160128 wait complete. 2024-05-10T11:15:33.129 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:32 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/2463160128' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-10T11:15:33.129 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:32 smithi100 ceph-mon[30296]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-10T11:15:33.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:32 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/2463160128' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-10T11:15:33.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:32 smithi178 ceph-mon[33359]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-10T11:15:33.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:32 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/2463160128' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-10T11:15:33.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:32 smithi188 ceph-mon[34075]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-10T11:15:33.471 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-05-10T11:15:33.716 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:15:34.083 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:33 smithi100 ceph-mon[30296]: pgmap v335: 129 pgs: 129 active+clean; 584 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 4 op/s 2024-05-10T11:15:34.083 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:33 smithi100 ceph-mon[30296]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-10T11:15:34.083 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:33 smithi100 ceph-mon[30296]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-05-10T11:15:34.083 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:33 smithi100 ceph-mon[30296]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T11:15:34.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:33 smithi178 ceph-mon[33359]: pgmap v335: 129 pgs: 129 active+clean; 584 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 4 op/s 2024-05-10T11:15:34.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:33 smithi178 ceph-mon[33359]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-10T11:15:34.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:33 smithi178 ceph-mon[33359]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-05-10T11:15:34.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:33 smithi178 ceph-mon[33359]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T11:15:34.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:33 smithi188 ceph-mon[34075]: pgmap v335: 129 pgs: 129 active+clean; 584 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 4 op/s 2024-05-10T11:15:34.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:33 smithi188 ceph-mon[34075]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-10T11:15:34.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:33 smithi188 ceph-mon[34075]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-05-10T11:15:34.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:33 smithi188 ceph-mon[34075]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T11:15:35.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:35.414+0000 7f35bba3f700 1 -- 172.21.15.100:0/438850926 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35b4102630 msgr2=0x7f35b4102a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:35.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:35.414+0000 7f35bba3f700 1 --2- 172.21.15.100:0/438850926 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35b4102630 0x7f35b4102a10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f35b000b0d0 tx=0x7f35b000b3e0 comp rx=0 tx=0).stop 2024-05-10T11:15:35.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:35.414+0000 7f35bba3f700 1 -- 172.21.15.100:0/438850926 shutdown_connections 2024-05-10T11:15:35.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:35.414+0000 7f35bba3f700 1 --2- 172.21.15.100:0/438850926 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35b41076c0 0x7f35b410f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:35.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:35.414+0000 7f35bba3f700 1 --2- 172.21.15.100:0/438850926 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35b4102630 0x7f35b4102a10 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:35.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:35.414+0000 7f35bba3f700 1 --2- 172.21.15.100:0/438850926 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35b4102fe0 0x7f35b4106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:35.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:35.414+0000 7f35bba3f700 1 -- 172.21.15.100:0/438850926 >> 172.21.15.100:0/438850926 conn(0x7f35b4079370 msgr2=0x7f35b4079780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:15:35.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:35.414+0000 7f35bba3f700 1 -- 172.21.15.100:0/438850926 shutdown_connections 2024-05-10T11:15:35.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:35.414+0000 7f35bba3f700 1 -- 172.21.15.100:0/438850926 wait complete. 2024-05-10T11:15:35.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:35.414+0000 7f35bba3f700 1 Processor -- start 2024-05-10T11:15:35.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:35.414+0000 7f35bba3f700 1 -- start start 2024-05-10T11:15:35.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:35.415+0000 7f35bba3f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35b4102630 0x7f35b419a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:15:35.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:35.415+0000 7f35bba3f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35b4102fe0 0x7f35b419af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:15:35.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:35.415+0000 7f35bba3f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35b41076c0 0x7f35b41a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:15:35.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:35.415+0000 7f35bba3f700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f35b4112420 con 0x7f35b41076c0 2024-05-10T11:15:35.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:35.415+0000 7f35bba3f700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f35b41122a0 con 0x7f35b4102fe0 2024-05-10T11:15:35.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:35.415+0000 7f35bba3f700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f35b41125a0 con 0x7f35b4102630 2024-05-10T11:15:35.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:35.415+0000 7f35b9fdc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35b41076c0 0x7f35b41a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:15:35.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:35.415+0000 7f35b8fda700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35b4102fe0 0x7f35b419af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:15:35.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:35.415+0000 7f35b8fda700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35b4102fe0 0x7f35b419af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:38220/0 (socket says 172.21.15.100:38220) 2024-05-10T11:15:35.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:35.415+0000 7f35b9fdc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35b41076c0 0x7f35b41a5a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54712/0 (socket says 172.21.15.100:54712) 2024-05-10T11:15:35.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:35.415+0000 7f35b8fda700 1 -- 172.21.15.100:0/47354126 learned_addr learned my addr 172.21.15.100:0/47354126 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:15:35.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:35.415+0000 7f35b97db700 1 --2- 172.21.15.100:0/47354126 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35b4102630 0x7f35b419a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:15:35.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:35.415+0000 7f35b9fdc700 1 -- 172.21.15.100:0/47354126 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35b4102630 msgr2=0x7f35b419a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:35.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:35.415+0000 7f35b9fdc700 1 --2- 172.21.15.100:0/47354126 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35b4102630 0x7f35b419a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:35.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:35.415+0000 7f35b9fdc700 1 -- 172.21.15.100:0/47354126 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35b4102fe0 msgr2=0x7f35b419af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:35.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:35.415+0000 7f35b9fdc700 1 --2- 172.21.15.100:0/47354126 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35b4102fe0 0x7f35b419af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:35.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:35.415+0000 7f35b9fdc700 1 -- 172.21.15.100:0/47354126 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35b41a5f80 con 0x7f35b41076c0 2024-05-10T11:15:35.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:35.415+0000 7f35b97db700 1 --2- 172.21.15.100:0/47354126 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35b4102630 0x7f35b419a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:15:35.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:35.415+0000 7f35b9fdc700 1 --2- 172.21.15.100:0/47354126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35b41076c0 0x7f35b41a5a40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f35ac00d720 tx=0x7f35ac00da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:15:35.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:35.416+0000 7f35aa7fc700 1 -- 172.21.15.100:0/47354126 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ac016070 con 0x7f35b41076c0 2024-05-10T11:15:35.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:35.416+0000 7f35aa7fc700 1 -- 172.21.15.100:0/47354126 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f35ac004830 con 0x7f35b41076c0 2024-05-10T11:15:35.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:35.416+0000 7f35aa7fc700 1 -- 172.21.15.100:0/47354126 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ac005490 con 0x7f35b41076c0 2024-05-10T11:15:35.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:35.416+0000 7f35bba3f700 1 -- 172.21.15.100:0/47354126 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35b41a61b0 con 0x7f35b41076c0 2024-05-10T11:15:35.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:35.416+0000 7f35bba3f700 1 -- 172.21.15.100:0/47354126 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35b41a6670 con 0x7f35b41076c0 2024-05-10T11:15:35.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:35.417+0000 7f35bba3f700 1 -- 172.21.15.100:0/47354126 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35b406a190 con 0x7f35b41076c0 2024-05-10T11:15:35.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:35.418+0000 7f35aa7fc700 1 -- 172.21.15.100:0/47354126 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f35ac004220 con 0x7f35b41076c0 2024-05-10T11:15:35.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:35.419+0000 7f35aa7fc700 1 --2- 172.21.15.100:0/47354126 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f35a00b2880 0x7f35a00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:15:35.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:35.419+0000 7f35b97db700 1 --2- 172.21.15.100:0/47354126 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f35a00b2880 0x7f35a00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:15:35.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:35.419+0000 7f35aa7fc700 1 -- 172.21.15.100:0/47354126 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f35ac0e21d0 con 0x7f35b41076c0 2024-05-10T11:15:35.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:35.419+0000 7f35b97db700 1 --2- 172.21.15.100:0/47354126 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f35a00b2880 0x7f35a00b4d40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f35b0009e60 tx=0x7f35b000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:15:35.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:35.423+0000 7f35aa7fc700 1 -- 172.21.15.100:0/47354126 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f35ac01f020 con 0x7f35b41076c0 2024-05-10T11:15:35.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:35.670+0000 7f35bba3f700 1 -- 172.21.15.100:0/47354126 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f35b402e510 con 0x7f35b41076c0 2024-05-10T11:15:35.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:35.803+0000 7f35aa7fc700 1 -- 172.21.15.100:0/47354126 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v26) ==== 115+0+0 (secure 0 0 0) 0x7f35ac0a9b90 con 0x7f35b41076c0 2024-05-10T11:15:35.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:35.806+0000 7f35bba3f700 1 -- 172.21.15.100:0/47354126 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f35a00b2880 msgr2=0x7f35a00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:35.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:35.806+0000 7f35bba3f700 1 --2- 172.21.15.100:0/47354126 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f35a00b2880 0x7f35a00b4d40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f35b0009e60 tx=0x7f35b000b040 comp rx=0 tx=0).stop 2024-05-10T11:15:35.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:35.806+0000 7f35bba3f700 1 -- 172.21.15.100:0/47354126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35b41076c0 msgr2=0x7f35b41a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:35.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:35.806+0000 7f35bba3f700 1 --2- 172.21.15.100:0/47354126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35b41076c0 0x7f35b41a5a40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f35ac00d720 tx=0x7f35ac00da30 comp rx=0 tx=0).stop 2024-05-10T11:15:35.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:35.806+0000 7f35bba3f700 1 -- 172.21.15.100:0/47354126 shutdown_connections 2024-05-10T11:15:35.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:35.806+0000 7f35bba3f700 1 --2- 172.21.15.100:0/47354126 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f35a00b2880 0x7f35a00b4d40 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:35.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:35.806+0000 7f35bba3f700 1 --2- 172.21.15.100:0/47354126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35b41076c0 0x7f35b41a5a40 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:35.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:35.806+0000 7f35bba3f700 1 --2- 172.21.15.100:0/47354126 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35b4102630 0x7f35b419a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:35.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:35.806+0000 7f35bba3f700 1 --2- 172.21.15.100:0/47354126 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35b4102fe0 0x7f35b419af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:35.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:35.806+0000 7f35bba3f700 1 -- 172.21.15.100:0/47354126 >> 172.21.15.100:0/47354126 conn(0x7f35b4079370 msgr2=0x7f35b4100830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:15:35.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:35.806+0000 7f35bba3f700 1 -- 172.21.15.100:0/47354126 shutdown_connections 2024-05-10T11:15:35.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:35.806+0000 7f35bba3f700 1 -- 172.21.15.100:0/47354126 wait complete. 2024-05-10T11:15:36.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:35 smithi100 ceph-mon[30296]: pgmap v336: 129 pgs: 129 active+clean; 584 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s rd, 2.0 KiB/s wr, 6 op/s 2024-05-10T11:15:36.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:35 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/47354126' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-10T11:15:36.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:35 smithi178 ceph-mon[33359]: pgmap v336: 129 pgs: 129 active+clean; 584 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s rd, 2.0 KiB/s wr, 6 op/s 2024-05-10T11:15:36.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:35 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/47354126' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-10T11:15:36.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:35 smithi188 ceph-mon[34075]: pgmap v336: 129 pgs: 129 active+clean; 584 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s rd, 2.0 KiB/s wr, 6 op/s 2024-05-10T11:15:36.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:35 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/47354126' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-10T11:15:36.525 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-05-10T11:15:36.525 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-05-10T11:15:36.769 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:15:37.134 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:36 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/47354126' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-10T11:15:37.134 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:36 smithi100 ceph-mon[30296]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T11:15:37.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:36 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/47354126' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-10T11:15:37.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:36 smithi178 ceph-mon[33359]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T11:15:37.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:36 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/47354126' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-10T11:15:37.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:36 smithi188 ceph-mon[34075]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T11:15:37.898 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:37 smithi100 ceph-mon[30296]: pgmap v337: 129 pgs: 129 active+clean; 584 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-05-10T11:15:38.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:37 smithi178 ceph-mon[33359]: pgmap v337: 129 pgs: 129 active+clean; 584 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-05-10T11:15:38.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:37 smithi188 ceph-mon[34075]: pgmap v337: 129 pgs: 129 active+clean; 584 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-05-10T11:15:38.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:38.439+0000 7f131ec3f700 1 -- 172.21.15.100:0/4134726643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f131806c3d0 msgr2=0x7f1318110320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:38.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:38.439+0000 7f131ec3f700 1 --2- 172.21.15.100:0/4134726643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f131806c3d0 0x7f1318110320 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1310009990 tx=0x7f1310009ca0 comp rx=0 tx=0).stop 2024-05-10T11:15:38.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:38.439+0000 7f131ec3f700 1 -- 172.21.15.100:0/4134726643 shutdown_connections 2024-05-10T11:15:38.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:38.439+0000 7f131ec3f700 1 --2- 172.21.15.100:0/4134726643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f131806c3d0 0x7f1318110320 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:38.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:38.439+0000 7f131ec3f700 1 --2- 172.21.15.100:0/4134726643 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1318109500 0x7f13181098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:38.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:38.439+0000 7f131ec3f700 1 --2- 172.21.15.100:0/4134726643 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f131806ba10 0x7f131806be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:38.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:38.439+0000 7f131ec3f700 1 -- 172.21.15.100:0/4134726643 >> 172.21.15.100:0/4134726643 conn(0x7f1318079310 msgr2=0x7f1318079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:15:38.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:38.439+0000 7f131ec3f700 1 -- 172.21.15.100:0/4134726643 shutdown_connections 2024-05-10T11:15:38.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:38.439+0000 7f131ec3f700 1 -- 172.21.15.100:0/4134726643 wait complete. 2024-05-10T11:15:38.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:38.440+0000 7f131ec3f700 1 Processor -- start 2024-05-10T11:15:38.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:38.440+0000 7f131ec3f700 1 -- start start 2024-05-10T11:15:38.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:38.440+0000 7f131ec3f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f131806ba10 0x7f1318102460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:15:38.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:38.440+0000 7f131ec3f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f131806c3d0 0x7f13181029a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:15:38.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:38.440+0000 7f131ec3f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1318109500 0x7f1318102ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:15:38.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:38.440+0000 7f131ec3f700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1318113540 con 0x7f131806ba10 2024-05-10T11:15:38.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:38.440+0000 7f131ec3f700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f13181133c0 con 0x7f131806c3d0 2024-05-10T11:15:38.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:38.440+0000 7f131ec3f700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f13181136c0 con 0x7f1318109500 2024-05-10T11:15:38.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:38.440+0000 7f131c9db700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f131806ba10 0x7f1318102460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:15:38.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:38.440+0000 7f131c9db700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f131806ba10 0x7f1318102460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43168/0 (socket says 172.21.15.100:43168) 2024-05-10T11:15:38.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:38.440+0000 7f131c9db700 1 -- 172.21.15.100:0/1541743098 learned_addr learned my addr 172.21.15.100:0/1541743098 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:15:38.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:38.440+0000 7f131d1dc700 1 --2- 172.21.15.100:0/1541743098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1318109500 0x7f1318102ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:15:38.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:38.441+0000 7f130ffff700 1 --2- 172.21.15.100:0/1541743098 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f131806c3d0 0x7f13181029a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:15:38.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:38.441+0000 7f131c9db700 1 -- 172.21.15.100:0/1541743098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1318109500 msgr2=0x7f1318102ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:38.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:38.441+0000 7f131c9db700 1 --2- 172.21.15.100:0/1541743098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1318109500 0x7f1318102ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:38.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:38.441+0000 7f131c9db700 1 -- 172.21.15.100:0/1541743098 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f131806c3d0 msgr2=0x7f13181029a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:38.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:38.441+0000 7f131c9db700 1 --2- 172.21.15.100:0/1541743098 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f131806c3d0 0x7f13181029a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:38.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:38.441+0000 7f131c9db700 1 -- 172.21.15.100:0/1541743098 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13181a7fc0 con 0x7f131806ba10 2024-05-10T11:15:38.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:38.441+0000 7f131d1dc700 1 --2- 172.21.15.100:0/1541743098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1318109500 0x7f1318102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:15:38.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:38.441+0000 7f131c9db700 1 --2- 172.21.15.100:0/1541743098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f131806ba10 0x7f1318102460 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f131400ada0 tx=0x7f131400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:15:38.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:38.441+0000 7f130dffb700 1 -- 172.21.15.100:0/1541743098 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1314017020 con 0x7f131806ba10 2024-05-10T11:15:38.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:38.441+0000 7f131ec3f700 1 -- 172.21.15.100:0/1541743098 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13181a82b0 con 0x7f131806ba10 2024-05-10T11:15:38.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:38.441+0000 7f130dffb700 1 -- 172.21.15.100:0/1541743098 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f131400ce10 con 0x7f131806ba10 2024-05-10T11:15:38.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:38.441+0000 7f130dffb700 1 -- 172.21.15.100:0/1541743098 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1314005440 con 0x7f131806ba10 2024-05-10T11:15:38.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:38.441+0000 7f131ec3f700 1 -- 172.21.15.100:0/1541743098 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13181a87f0 con 0x7f131806ba10 2024-05-10T11:15:38.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:38.442+0000 7f131ec3f700 1 -- 172.21.15.100:0/1541743098 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f131806a190 con 0x7f131806ba10 2024-05-10T11:15:38.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:38.443+0000 7f130dffb700 1 -- 172.21.15.100:0/1541743098 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f1314004ad0 con 0x7f131806ba10 2024-05-10T11:15:38.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:38.444+0000 7f130dffb700 1 --2- 172.21.15.100:0/1541743098 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f13000b2880 0x7f13000b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:15:38.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:38.444+0000 7f130ffff700 1 --2- 172.21.15.100:0/1541743098 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f13000b2880 0x7f13000b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:15:38.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:38.444+0000 7f130dffb700 1 -- 172.21.15.100:0/1541743098 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f13140e1de0 con 0x7f131806ba10 2024-05-10T11:15:38.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:38.444+0000 7f130ffff700 1 --2- 172.21.15.100:0/1541743098 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f13000b2880 0x7f13000b4d40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1308006fd0 tx=0x7f1308008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:15:38.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:38.448+0000 7f130dffb700 1 -- 172.21.15.100:0/1541743098 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f1314021020 con 0x7f131806ba10 2024-05-10T11:15:38.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:38.682+0000 7f131ec3f700 1 -- 172.21.15.100:0/1541743098 --> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f13180638e0 con 0x7f13000b2880 2024-05-10T11:15:38.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:38.709+0000 7f130dffb700 1 -- 172.21.15.100:0/1541743098 <== mgr.14152 v2:172.21.15.100:6800/2507287356 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f13180638e0 con 0x7f13000b2880 2024-05-10T11:15:38.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:38.712+0000 7f131ec3f700 1 -- 172.21.15.100:0/1541743098 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f13000b2880 msgr2=0x7f13000b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:38.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:38.712+0000 7f131ec3f700 1 --2- 172.21.15.100:0/1541743098 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f13000b2880 0x7f13000b4d40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1308006fd0 tx=0x7f1308008040 comp rx=0 tx=0).stop 2024-05-10T11:15:38.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:38.712+0000 7f131ec3f700 1 -- 172.21.15.100:0/1541743098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f131806ba10 msgr2=0x7f1318102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:38.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:38.712+0000 7f131ec3f700 1 --2- 172.21.15.100:0/1541743098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f131806ba10 0x7f1318102460 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f131400ada0 tx=0x7f131400c330 comp rx=0 tx=0).stop 2024-05-10T11:15:38.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:38.712+0000 7f131ec3f700 1 -- 172.21.15.100:0/1541743098 shutdown_connections 2024-05-10T11:15:38.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:38.712+0000 7f131ec3f700 1 --2- 172.21.15.100:0/1541743098 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f13000b2880 0x7f13000b4d40 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:38.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:38.712+0000 7f131ec3f700 1 --2- 172.21.15.100:0/1541743098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1318109500 0x7f1318102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:38.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:38.712+0000 7f131ec3f700 1 --2- 172.21.15.100:0/1541743098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f131806ba10 0x7f1318102460 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:38.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:38.712+0000 7f131ec3f700 1 --2- 172.21.15.100:0/1541743098 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f131806c3d0 0x7f13181029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:38.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:38.712+0000 7f131ec3f700 1 -- 172.21.15.100:0/1541743098 >> 172.21.15.100:0/1541743098 conn(0x7f1318079310 msgr2=0x7f1318101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:15:38.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:38.712+0000 7f131ec3f700 1 -- 172.21.15.100:0/1541743098 shutdown_connections 2024-05-10T11:15:38.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:38.712+0000 7f131ec3f700 1 -- 172.21.15.100:0/1541743098 wait complete. 2024-05-10T11:15:39.038 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:15:38 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:15:38.687+0000 7f6177e24700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-10T11:15:39.038 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:15:38 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:15:38.687+0000 7f6177e24700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-10T11:15:39.038 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:15:38 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:15:38.687+0000 7f6177e24700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-10T11:15:39.038 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:15:38 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:15:38.687+0000 7f6177e24700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-10T11:15:39.038 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:15:38 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:15:38.687+0000 7f6177e24700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-10T11:15:39.561 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-05-10T11:15:39.812 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:15:40.080 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:39 smithi100 ceph-mon[30296]: pgmap v338: 129 pgs: 129 active+clean; 584 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-05-10T11:15:40.080 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:39 smithi100 ceph-mon[30296]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:15:40.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:39 smithi178 ceph-mon[33359]: pgmap v338: 129 pgs: 129 active+clean; 584 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-05-10T11:15:40.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:39 smithi178 ceph-mon[33359]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:15:40.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:39 smithi188 ceph-mon[34075]: pgmap v338: 129 pgs: 129 active+clean; 584 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-05-10T11:15:40.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:39 smithi188 ceph-mon[34075]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:15:41.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:41.637+0000 7f2bada3d700 1 -- 172.21.15.100:0/4248270606 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ba8075ff0 msgr2=0x7f2ba810f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:41.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:41.637+0000 7f2bada3d700 1 --2- 172.21.15.100:0/4248270606 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ba8075ff0 0x7f2ba810f9c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2b90009990 tx=0x7f2b90009ca0 comp rx=0 tx=0).stop 2024-05-10T11:15:41.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:41.637+0000 7f2bada3d700 1 -- 172.21.15.100:0/4248270606 shutdown_connections 2024-05-10T11:15:41.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:41.637+0000 7f2bada3d700 1 --2- 172.21.15.100:0/4248270606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ba810ff00 0x7f2ba81122f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:41.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:41.637+0000 7f2bada3d700 1 --2- 172.21.15.100:0/4248270606 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ba80756d0 0x7f2ba8075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:41.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:41.637+0000 7f2bada3d700 1 --2- 172.21.15.100:0/4248270606 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ba8075ff0 0x7f2ba810f9c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:41.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:41.637+0000 7f2bada3d700 1 -- 172.21.15.100:0/4248270606 >> 172.21.15.100:0/4248270606 conn(0x7f2ba807a530 msgr2=0x7f2ba807a940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:15:41.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:41.637+0000 7f2bada3d700 1 -- 172.21.15.100:0/4248270606 shutdown_connections 2024-05-10T11:15:41.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:41.637+0000 7f2bada3d700 1 -- 172.21.15.100:0/4248270606 wait complete. 2024-05-10T11:15:41.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:41.638+0000 7f2bada3d700 1 Processor -- start 2024-05-10T11:15:41.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:41.638+0000 7f2bada3d700 1 -- start start 2024-05-10T11:15:41.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:41.638+0000 7f2bada3d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ba80756d0 0x7f2ba819be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:15:41.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:41.638+0000 7f2bada3d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ba8075ff0 0x7f2ba819c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:15:41.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:41.638+0000 7f2bada3d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ba810ff00 0x7f2ba81a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:15:41.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:41.638+0000 7f2bada3d700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba8115800 con 0x7f2ba810ff00 2024-05-10T11:15:41.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:41.638+0000 7f2bada3d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba8115680 con 0x7f2ba8075ff0 2024-05-10T11:15:41.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:41.638+0000 7f2bada3d700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba8115980 con 0x7f2ba80756d0 2024-05-10T11:15:41.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:41.638+0000 7f2ba77fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ba810ff00 0x7f2ba81a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:15:41.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:41.638+0000 7f2ba77fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ba810ff00 0x7f2ba81a0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43200/0 (socket says 172.21.15.100:43200) 2024-05-10T11:15:41.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:41.638+0000 7f2ba77fe700 1 -- 172.21.15.100:0/1613840610 learned_addr learned my addr 172.21.15.100:0/1613840610 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:15:41.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:41.639+0000 7f2b9e7fc700 1 --2- 172.21.15.100:0/1613840610 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ba8075ff0 0x7f2ba819c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:15:41.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:41.639+0000 7f2ba6ffd700 1 --2- 172.21.15.100:0/1613840610 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ba80756d0 0x7f2ba819be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:15:41.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:41.639+0000 7f2ba77fe700 1 -- 172.21.15.100:0/1613840610 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ba80756d0 msgr2=0x7f2ba819be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:41.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:41.639+0000 7f2ba77fe700 1 --2- 172.21.15.100:0/1613840610 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ba80756d0 0x7f2ba819be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:41.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:41.639+0000 7f2ba77fe700 1 -- 172.21.15.100:0/1613840610 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ba8075ff0 msgr2=0x7f2ba819c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:41.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:41.639+0000 7f2ba77fe700 1 --2- 172.21.15.100:0/1613840610 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ba8075ff0 0x7f2ba819c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:41.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:41.639+0000 7f2ba77fe700 1 -- 172.21.15.100:0/1613840610 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ba81a0ea0 con 0x7f2ba810ff00 2024-05-10T11:15:41.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:41.639+0000 7f2ba77fe700 1 --2- 172.21.15.100:0/1613840610 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ba810ff00 0x7f2ba81a0760 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2ba000b970 tx=0x7f2ba000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:15:41.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:41.639+0000 7f2baca3b700 1 -- 172.21.15.100:0/1613840610 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba0019070 con 0x7f2ba810ff00 2024-05-10T11:15:41.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:41.639+0000 7f2bada3d700 1 -- 172.21.15.100:0/1613840610 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ba81a1190 con 0x7f2ba810ff00 2024-05-10T11:15:41.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:41.639+0000 7f2baca3b700 1 -- 172.21.15.100:0/1613840610 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2ba000cdc0 con 0x7f2ba810ff00 2024-05-10T11:15:41.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:41.639+0000 7f2baca3b700 1 -- 172.21.15.100:0/1613840610 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba0014e30 con 0x7f2ba810ff00 2024-05-10T11:15:41.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:41.639+0000 7f2bada3d700 1 -- 172.21.15.100:0/1613840610 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ba81a8a10 con 0x7f2ba810ff00 2024-05-10T11:15:41.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:41.640+0000 7f2bada3d700 1 -- 172.21.15.100:0/1613840610 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ba8076d70 con 0x7f2ba810ff00 2024-05-10T11:15:41.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:41.641+0000 7f2baca3b700 1 -- 172.21.15.100:0/1613840610 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f2ba001c070 con 0x7f2ba810ff00 2024-05-10T11:15:41.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:41.642+0000 7f2baca3b700 1 --2- 172.21.15.100:0/1613840610 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2b940b2910 0x7f2b940b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:15:41.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:41.642+0000 7f2ba6ffd700 1 --2- 172.21.15.100:0/1613840610 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2b940b2910 0x7f2b940b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:15:41.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:41.642+0000 7f2baca3b700 1 -- 172.21.15.100:0/1613840610 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f2ba00e1980 con 0x7f2ba810ff00 2024-05-10T11:15:41.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:41.643+0000 7f2ba6ffd700 1 --2- 172.21.15.100:0/1613840610 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2b940b2910 0x7f2b940b4dd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2b98006fd0 tx=0x7f2b98008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:15:41.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:41.647+0000 7f2baca3b700 1 -- 172.21.15.100:0/1613840610 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f2ba00a92f0 con 0x7f2ba810ff00 2024-05-10T11:15:41.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:41.889+0000 7f2bada3d700 1 -- 172.21.15.100:0/1613840610 --> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f2ba80638e0 con 0x7f2b940b2910 2024-05-10T11:15:41.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:41.916+0000 7f2baca3b700 1 -- 172.21.15.100:0/1613840610 <== mgr.14152 v2:172.21.15.100:6800/2507287356 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2ba80638e0 con 0x7f2b940b2910 2024-05-10T11:15:41.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:41.918+0000 7f2bada3d700 1 -- 172.21.15.100:0/1613840610 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2b940b2910 msgr2=0x7f2b940b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:41.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:41.918+0000 7f2bada3d700 1 --2- 172.21.15.100:0/1613840610 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2b940b2910 0x7f2b940b4dd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2b98006fd0 tx=0x7f2b98008040 comp rx=0 tx=0).stop 2024-05-10T11:15:41.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:41.918+0000 7f2bada3d700 1 -- 172.21.15.100:0/1613840610 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ba810ff00 msgr2=0x7f2ba81a0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:41.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:41.918+0000 7f2bada3d700 1 --2- 172.21.15.100:0/1613840610 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ba810ff00 0x7f2ba81a0760 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2ba000b970 tx=0x7f2ba000bc80 comp rx=0 tx=0).stop 2024-05-10T11:15:41.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:41.918+0000 7f2bada3d700 1 -- 172.21.15.100:0/1613840610 shutdown_connections 2024-05-10T11:15:41.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:41.918+0000 7f2bada3d700 1 --2- 172.21.15.100:0/1613840610 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2b940b2910 0x7f2b940b4dd0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:41.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:41.918+0000 7f2bada3d700 1 --2- 172.21.15.100:0/1613840610 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ba810ff00 0x7f2ba81a0760 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:41.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:41.918+0000 7f2bada3d700 1 --2- 172.21.15.100:0/1613840610 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ba80756d0 0x7f2ba819be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:41.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:41.918+0000 7f2bada3d700 1 --2- 172.21.15.100:0/1613840610 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ba8075ff0 0x7f2ba819c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:41.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:41.918+0000 7f2bada3d700 1 -- 172.21.15.100:0/1613840610 >> 172.21.15.100:0/1613840610 conn(0x7f2ba807a530 msgr2=0x7f2ba8101c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:15:41.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:41.918+0000 7f2bada3d700 1 -- 172.21.15.100:0/1613840610 shutdown_connections 2024-05-10T11:15:41.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:41.918+0000 7f2bada3d700 1 -- 172.21.15.100:0/1613840610 wait complete. 2024-05-10T11:15:42.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:41 smithi178 ceph-mon[33359]: pgmap v339: 129 pgs: 129 active+clean; 584 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-05-10T11:15:42.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:41 smithi178 ceph-mon[33359]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-05-10T11:15:42.276 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:41 smithi100 ceph-mon[30296]: pgmap v339: 129 pgs: 129 active+clean; 584 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-05-10T11:15:42.276 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:41 smithi100 ceph-mon[30296]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-05-10T11:15:42.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:41 smithi188 ceph-mon[34075]: pgmap v339: 129 pgs: 129 active+clean; 584 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-05-10T11:15:42.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:41 smithi188 ceph-mon[34075]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-05-10T11:15:42.717 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-05-10T11:15:42.965 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:15:43.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:42 smithi100 ceph-mon[30296]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:15:43.180 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:42 smithi188 ceph-mon[34075]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:15:43.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:42 smithi178 ceph-mon[33359]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:15:44.094 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:43 smithi100 ceph-mon[30296]: pgmap v340: 129 pgs: 129 active+clean; 591 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.6 KiB/s wr, 12 op/s 2024-05-10T11:15:44.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:43 smithi178 ceph-mon[33359]: pgmap v340: 129 pgs: 129 active+clean; 591 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.6 KiB/s wr, 12 op/s 2024-05-10T11:15:44.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:43 smithi188 ceph-mon[34075]: pgmap v340: 129 pgs: 129 active+clean; 591 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.6 KiB/s wr, 12 op/s 2024-05-10T11:15:44.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:44.671+0000 7f3d5f631700 1 -- 172.21.15.100:0/1142587890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d581023b0 msgr2=0x7f3d58110340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:44.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:44.671+0000 7f3d5f631700 1 --2- 172.21.15.100:0/1142587890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d581023b0 0x7f3d58110340 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3d5000b0d0 tx=0x7f3d5000b3e0 comp rx=0 tx=0).stop 2024-05-10T11:15:44.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:44.671+0000 7f3d5f631700 1 -- 172.21.15.100:0/1142587890 shutdown_connections 2024-05-10T11:15:44.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:44.671+0000 7f3d5f631700 1 --2- 172.21.15.100:0/1142587890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d581023b0 0x7f3d58110340 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:44.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:44.671+0000 7f3d5f631700 1 --2- 172.21.15.100:0/1142587890 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d58109570 0x7f3d58109950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:44.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:44.671+0000 7f3d5f631700 1 --2- 172.21.15.100:0/1142587890 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d581019f0 0x7f3d58101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:44.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:44.671+0000 7f3d5f631700 1 -- 172.21.15.100:0/1142587890 >> 172.21.15.100:0/1142587890 conn(0x7f3d580789a0 msgr2=0x7f3d580ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:15:44.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:44.671+0000 7f3d5f631700 1 -- 172.21.15.100:0/1142587890 shutdown_connections 2024-05-10T11:15:44.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:44.671+0000 7f3d5f631700 1 -- 172.21.15.100:0/1142587890 wait complete. 2024-05-10T11:15:44.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:44.672+0000 7f3d5f631700 1 Processor -- start 2024-05-10T11:15:44.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:44.672+0000 7f3d5f631700 1 -- start start 2024-05-10T11:15:44.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:44.672+0000 7f3d5f631700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d581019f0 0x7f3d58102bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:15:44.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:44.672+0000 7f3d5f631700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d581023b0 0x7f3d58103110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:15:44.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:44.672+0000 7f3d5f631700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d58109570 0x7f3d58103650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:15:44.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:44.672+0000 7f3d5f631700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3d58113530 con 0x7f3d581023b0 2024-05-10T11:15:44.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:44.672+0000 7f3d5f631700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3d581133b0 con 0x7f3d581019f0 2024-05-10T11:15:44.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:44.672+0000 7f3d5f631700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3d581136b0 con 0x7f3d58109570 2024-05-10T11:15:44.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:44.672+0000 7f3d5cbcc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d581023b0 0x7f3d58103110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:15:44.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:44.672+0000 7f3d5cbcc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d581023b0 0x7f3d58103110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43230/0 (socket says 172.21.15.100:43230) 2024-05-10T11:15:44.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:44.672+0000 7f3d5cbcc700 1 -- 172.21.15.100:0/8563868 learned_addr learned my addr 172.21.15.100:0/8563868 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:15:44.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:44.673+0000 7f3d5d3cd700 1 --2- 172.21.15.100:0/8563868 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d581019f0 0x7f3d58102bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:15:44.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:44.673+0000 7f3d5dbce700 1 --2- 172.21.15.100:0/8563868 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d58109570 0x7f3d58103650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:15:44.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:44.673+0000 7f3d5cbcc700 1 -- 172.21.15.100:0/8563868 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d58109570 msgr2=0x7f3d58103650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:44.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:44.673+0000 7f3d5cbcc700 1 --2- 172.21.15.100:0/8563868 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d58109570 0x7f3d58103650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:44.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:44.673+0000 7f3d5cbcc700 1 -- 172.21.15.100:0/8563868 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d581019f0 msgr2=0x7f3d58102bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:44.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:44.673+0000 7f3d5cbcc700 1 --2- 172.21.15.100:0/8563868 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d581019f0 0x7f3d58102bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:44.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:44.673+0000 7f3d5cbcc700 1 -- 172.21.15.100:0/8563868 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d581a7f10 con 0x7f3d581023b0 2024-05-10T11:15:44.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:44.673+0000 7f3d5dbce700 1 --2- 172.21.15.100:0/8563868 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d58109570 0x7f3d58103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:15:44.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:44.673+0000 7f3d5cbcc700 1 --2- 172.21.15.100:0/8563868 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d581023b0 0x7f3d58103110 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3d4800d980 tx=0x7f3d4800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:15:44.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:44.673+0000 7f3d4e7fc700 1 -- 172.21.15.100:0/8563868 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d48017070 con 0x7f3d581023b0 2024-05-10T11:15:44.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:44.673+0000 7f3d5f631700 1 -- 172.21.15.100:0/8563868 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d581a8200 con 0x7f3d581023b0 2024-05-10T11:15:44.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:44.673+0000 7f3d4e7fc700 1 -- 172.21.15.100:0/8563868 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3d48004830 con 0x7f3d581023b0 2024-05-10T11:15:44.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:44.673+0000 7f3d4e7fc700 1 -- 172.21.15.100:0/8563868 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d480053a0 con 0x7f3d581023b0 2024-05-10T11:15:44.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:44.673+0000 7f3d5f631700 1 -- 172.21.15.100:0/8563868 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d581a8740 con 0x7f3d581023b0 2024-05-10T11:15:44.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:44.674+0000 7f3d5f631700 1 -- 172.21.15.100:0/8563868 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d5806a190 con 0x7f3d581023b0 2024-05-10T11:15:44.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:44.676+0000 7f3d4e7fc700 1 -- 172.21.15.100:0/8563868 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f3d480049e0 con 0x7f3d581023b0 2024-05-10T11:15:44.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:44.676+0000 7f3d4e7fc700 1 --2- 172.21.15.100:0/8563868 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f3d440b2990 0x7f3d440b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:15:44.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:44.676+0000 7f3d5d3cd700 1 --2- 172.21.15.100:0/8563868 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f3d440b2990 0x7f3d440b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:15:44.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:44.676+0000 7f3d4e7fc700 1 -- 172.21.15.100:0/8563868 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f3d480e2620 con 0x7f3d581023b0 2024-05-10T11:15:44.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:44.677+0000 7f3d5d3cd700 1 --2- 172.21.15.100:0/8563868 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f3d440b2990 0x7f3d440b4e50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3d54005950 tx=0x7f3d540058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:15:44.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:44.681+0000 7f3d4e7fc700 1 -- 172.21.15.100:0/8563868 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f3d48020020 con 0x7f3d581023b0 2024-05-10T11:15:44.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:44.919+0000 7f3d5f631700 1 -- 172.21.15.100:0/8563868 --> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f3d580638e0 con 0x7f3d440b2990 2024-05-10T11:15:44.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:44.923+0000 7f3d4e7fc700 1 -- 172.21.15.100:0/8563868 <== mgr.14152 v2:172.21.15.100:6800/2507287356 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f3d580638e0 con 0x7f3d440b2990 2024-05-10T11:15:44.923 INFO:teuthology.orchestra.run.smithi100.stdout:/volumes/qa/sv_0/96765652-39ef-4cf3-b20a-a70f7e9d774d 2024-05-10T11:15:44.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:44.926+0000 7f3d5f631700 1 -- 172.21.15.100:0/8563868 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f3d440b2990 msgr2=0x7f3d440b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:44.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:44.926+0000 7f3d5f631700 1 --2- 172.21.15.100:0/8563868 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f3d440b2990 0x7f3d440b4e50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3d54005950 tx=0x7f3d540058e0 comp rx=0 tx=0).stop 2024-05-10T11:15:44.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:44.926+0000 7f3d5f631700 1 -- 172.21.15.100:0/8563868 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d581023b0 msgr2=0x7f3d58103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:44.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:44.926+0000 7f3d5f631700 1 --2- 172.21.15.100:0/8563868 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d581023b0 0x7f3d58103110 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3d4800d980 tx=0x7f3d4800dd40 comp rx=0 tx=0).stop 2024-05-10T11:15:44.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:44.926+0000 7f3d5f631700 1 -- 172.21.15.100:0/8563868 shutdown_connections 2024-05-10T11:15:44.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:44.926+0000 7f3d5f631700 1 --2- 172.21.15.100:0/8563868 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f3d440b2990 0x7f3d440b4e50 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:44.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:44.926+0000 7f3d5f631700 1 --2- 172.21.15.100:0/8563868 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d58109570 0x7f3d58103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:44.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:44.926+0000 7f3d5f631700 1 --2- 172.21.15.100:0/8563868 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d581019f0 0x7f3d58102bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:44.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:44.926+0000 7f3d5f631700 1 --2- 172.21.15.100:0/8563868 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d581023b0 0x7f3d58103110 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:44.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:44.926+0000 7f3d5f631700 1 -- 172.21.15.100:0/8563868 >> 172.21.15.100:0/8563868 conn(0x7f3d580789a0 msgr2=0x7f3d581001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:15:44.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:44.926+0000 7f3d5f631700 1 -- 172.21.15.100:0/8563868 shutdown_connections 2024-05-10T11:15:44.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:44.926+0000 7f3d5f631700 1 -- 172.21.15.100:0/8563868 wait complete. 2024-05-10T11:15:45.724 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-05-10T11:15:45.968 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:15:46.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:45 smithi100 ceph-mon[30296]: pgmap v341: 129 pgs: 129 active+clean; 591 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-05-10T11:15:46.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:45 smithi100 ceph-mon[30296]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:15:46.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:45 smithi178 ceph-mon[33359]: pgmap v341: 129 pgs: 129 active+clean; 591 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-05-10T11:15:46.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:45 smithi178 ceph-mon[33359]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:15:46.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:45 smithi188 ceph-mon[34075]: pgmap v341: 129 pgs: 129 active+clean; 591 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-05-10T11:15:46.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:45 smithi188 ceph-mon[34075]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:15:47.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:47.733+0000 7f448c060700 1 -- 172.21.15.100:0/2825020730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44840756d0 msgr2=0x7f4484075ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:47.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:47.733+0000 7f448c060700 1 --2- 172.21.15.100:0/2825020730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44840756d0 0x7f4484075ab0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4480009990 tx=0x7f4480009ca0 comp rx=0 tx=0).stop 2024-05-10T11:15:47.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:47.733+0000 7f448c060700 1 -- 172.21.15.100:0/2825020730 shutdown_connections 2024-05-10T11:15:47.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:47.733+0000 7f448c060700 1 --2- 172.21.15.100:0/2825020730 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4484114670 0x7f4484116a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:47.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:47.733+0000 7f448c060700 1 --2- 172.21.15.100:0/2825020730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44840756d0 0x7f4484075ab0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:47.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:47.733+0000 7f448c060700 1 --2- 172.21.15.100:0/2825020730 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4484075ff0 0x7f4484114130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:47.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:47.733+0000 7f448c060700 1 -- 172.21.15.100:0/2825020730 >> 172.21.15.100:0/2825020730 conn(0x7f448407a600 msgr2=0x7f448407aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:15:47.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:47.733+0000 7f448c060700 1 -- 172.21.15.100:0/2825020730 shutdown_connections 2024-05-10T11:15:47.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:47.733+0000 7f448c060700 1 -- 172.21.15.100:0/2825020730 wait complete. 2024-05-10T11:15:47.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:47.734+0000 7f448c060700 1 Processor -- start 2024-05-10T11:15:47.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:47.734+0000 7f448c060700 1 -- start start 2024-05-10T11:15:47.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:47.734+0000 7f448c060700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44840756d0 0x7f44841a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:15:47.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:47.734+0000 7f448c060700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4484075ff0 0x7f44841a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:15:47.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:47.734+0000 7f448c060700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4484114670 0x7f44841a4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:15:47.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:47.734+0000 7f448c060700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4484119c00 con 0x7f44840756d0 2024-05-10T11:15:47.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:47.734+0000 7f448c060700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4484119a80 con 0x7f4484075ff0 2024-05-10T11:15:47.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:47.734+0000 7f448c060700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4484119d80 con 0x7f4484114670 2024-05-10T11:15:47.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:47.734+0000 7f4489dfc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44840756d0 0x7f44841a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:15:47.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:47.734+0000 7f4489dfc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44840756d0 0x7f44841a01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53552/0 (socket says 172.21.15.100:53552) 2024-05-10T11:15:47.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:47.734+0000 7f4489dfc700 1 -- 172.21.15.100:0/3457579063 learned_addr learned my addr 172.21.15.100:0/3457579063 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:15:47.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:47.735+0000 7f44895fb700 1 --2- 172.21.15.100:0/3457579063 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4484075ff0 0x7f44841a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:15:47.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:47.735+0000 7f448a5fd700 1 --2- 172.21.15.100:0/3457579063 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4484114670 0x7f44841a4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:15:47.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:47.735+0000 7f4489dfc700 1 -- 172.21.15.100:0/3457579063 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4484114670 msgr2=0x7f44841a4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:47.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:47.735+0000 7f4489dfc700 1 --2- 172.21.15.100:0/3457579063 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4484114670 0x7f44841a4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:47.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:47.735+0000 7f4489dfc700 1 -- 172.21.15.100:0/3457579063 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4484075ff0 msgr2=0x7f44841a0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:47.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:47.735+0000 7f4489dfc700 1 --2- 172.21.15.100:0/3457579063 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4484075ff0 0x7f44841a0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:47.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:47.735+0000 7f4489dfc700 1 -- 172.21.15.100:0/3457579063 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44841a5200 con 0x7f44840756d0 2024-05-10T11:15:47.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:47.735+0000 7f448a5fd700 1 --2- 172.21.15.100:0/3457579063 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4484114670 0x7f44841a4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:15:47.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:47.735+0000 7f44895fb700 1 --2- 172.21.15.100:0/3457579063 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4484075ff0 0x7f44841a0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:15:47.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:47.735+0000 7f4489dfc700 1 --2- 172.21.15.100:0/3457579063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44840756d0 0x7f44841a01f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4480000c00 tx=0x7f44800047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:15:47.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:47.735+0000 7f447affd700 1 -- 172.21.15.100:0/3457579063 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4480027070 con 0x7f44840756d0 2024-05-10T11:15:47.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:47.735+0000 7f448c060700 1 -- 172.21.15.100:0/3457579063 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44841a5490 con 0x7f44840756d0 2024-05-10T11:15:47.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:47.735+0000 7f447affd700 1 -- 172.21.15.100:0/3457579063 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f448000fdb0 con 0x7f44840756d0 2024-05-10T11:15:47.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:47.735+0000 7f447affd700 1 -- 172.21.15.100:0/3457579063 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44800173f0 con 0x7f44840756d0 2024-05-10T11:15:47.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:47.735+0000 7f448c060700 1 -- 172.21.15.100:0/3457579063 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44841acd70 con 0x7f44840756d0 2024-05-10T11:15:47.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:47.736+0000 7f448c060700 1 -- 172.21.15.100:0/3457579063 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f448406a190 con 0x7f44840756d0 2024-05-10T11:15:47.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:47.738+0000 7f447affd700 1 -- 172.21.15.100:0/3457579063 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f448002a070 con 0x7f44840756d0 2024-05-10T11:15:47.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:47.738+0000 7f447affd700 1 --2- 172.21.15.100:0/3457579063 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f44700b2990 0x7f44700b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:15:47.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:47.738+0000 7f44895fb700 1 --2- 172.21.15.100:0/3457579063 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f44700b2990 0x7f44700b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:15:47.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:47.738+0000 7f447affd700 1 -- 172.21.15.100:0/3457579063 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f44800e3c30 con 0x7f44840756d0 2024-05-10T11:15:47.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:47.739+0000 7f44895fb700 1 --2- 172.21.15.100:0/3457579063 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f44700b2990 0x7f44700b4e50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4484113e10 tx=0x7f4474008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:15:47.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:47.742+0000 7f447affd700 1 -- 172.21.15.100:0/3457579063 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f4480031070 con 0x7f44840756d0 2024-05-10T11:15:47.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:47.980+0000 7f448c060700 1 -- 172.21.15.100:0/3457579063 --> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f4484110820 con 0x7f44700b2990 2024-05-10T11:15:48.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:48.006+0000 7f447affd700 1 -- 172.21.15.100:0/3457579063 <== mgr.14152 v2:172.21.15.100:6800/2507287356 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4484110820 con 0x7f44700b2990 2024-05-10T11:15:48.006 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:47 smithi100 ceph-mon[30296]: pgmap v342: 129 pgs: 129 active+clean; 591 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-05-10T11:15:48.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:48.008+0000 7f448c060700 1 -- 172.21.15.100:0/3457579063 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f44700b2990 msgr2=0x7f44700b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:48.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:48.008+0000 7f448c060700 1 --2- 172.21.15.100:0/3457579063 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f44700b2990 0x7f44700b4e50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4484113e10 tx=0x7f4474008040 comp rx=0 tx=0).stop 2024-05-10T11:15:48.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:48.008+0000 7f448c060700 1 -- 172.21.15.100:0/3457579063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44840756d0 msgr2=0x7f44841a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:48.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:48.008+0000 7f448c060700 1 --2- 172.21.15.100:0/3457579063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44840756d0 0x7f44841a01f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4480000c00 tx=0x7f44800047b0 comp rx=0 tx=0).stop 2024-05-10T11:15:48.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:48.008+0000 7f448c060700 1 -- 172.21.15.100:0/3457579063 shutdown_connections 2024-05-10T11:15:48.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:48.008+0000 7f448c060700 1 --2- 172.21.15.100:0/3457579063 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f44700b2990 0x7f44700b4e50 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:48.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:48.008+0000 7f448c060700 1 --2- 172.21.15.100:0/3457579063 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4484114670 0x7f44841a4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:48.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:48.008+0000 7f448c060700 1 --2- 172.21.15.100:0/3457579063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44840756d0 0x7f44841a01f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:48.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:48.008+0000 7f448c060700 1 --2- 172.21.15.100:0/3457579063 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4484075ff0 0x7f44841a0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:48.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:48.008+0000 7f448c060700 1 -- 172.21.15.100:0/3457579063 >> 172.21.15.100:0/3457579063 conn(0x7f448407a600 msgr2=0x7f4484110110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:15:48.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:48.008+0000 7f448c060700 1 -- 172.21.15.100:0/3457579063 shutdown_connections 2024-05-10T11:15:48.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:48.008+0000 7f448c060700 1 -- 172.21.15.100:0/3457579063 wait complete. 2024-05-10T11:15:48.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:47 smithi178 ceph-mon[33359]: pgmap v342: 129 pgs: 129 active+clean; 591 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-05-10T11:15:48.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:47 smithi188 ceph-mon[34075]: pgmap v342: 129 pgs: 129 active+clean; 591 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-05-10T11:15:48.718 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-05-10T11:15:48.965 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:15:49.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:48 smithi100 ceph-mon[30296]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:15:49.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:48 smithi178 ceph-mon[33359]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:15:49.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:48 smithi188 ceph-mon[34075]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:15:50.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:49 smithi100 ceph-mon[30296]: pgmap v343: 129 pgs: 129 active+clean; 611 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.1 KiB/s wr, 8 op/s 2024-05-10T11:15:50.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:49 smithi178 ceph-mon[33359]: pgmap v343: 129 pgs: 129 active+clean; 611 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.1 KiB/s wr, 8 op/s 2024-05-10T11:15:50.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:49 smithi188 ceph-mon[34075]: pgmap v343: 129 pgs: 129 active+clean; 611 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.1 KiB/s wr, 8 op/s 2024-05-10T11:15:50.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:50.717+0000 7f5647c0d700 1 -- 172.21.15.100:0/2260979419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56401023a0 msgr2=0x7f5640102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:50.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:50.717+0000 7f5647c0d700 1 --2- 172.21.15.100:0/2260979419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56401023a0 0x7f5640102820 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5630009990 tx=0x7f5630009ca0 comp rx=0 tx=0).stop 2024-05-10T11:15:50.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:50.717+0000 7f5647c0d700 1 -- 172.21.15.100:0/2260979419 shutdown_connections 2024-05-10T11:15:50.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:50.717+0000 7f5647c0d700 1 --2- 172.21.15.100:0/2260979419 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5640102d60 0x7f564010f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:50.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:50.717+0000 7f5647c0d700 1 --2- 172.21.15.100:0/2260979419 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5640108410 0x7f56401087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:50.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:50.717+0000 7f5647c0d700 1 --2- 172.21.15.100:0/2260979419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56401023a0 0x7f5640102820 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:50.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:50.717+0000 7f5647c0d700 1 -- 172.21.15.100:0/2260979419 >> 172.21.15.100:0/2260979419 conn(0x7f5640079350 msgr2=0x7f5640079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:15:50.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:50.717+0000 7f5647c0d700 1 -- 172.21.15.100:0/2260979419 shutdown_connections 2024-05-10T11:15:50.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:50.717+0000 7f5647c0d700 1 -- 172.21.15.100:0/2260979419 wait complete. 2024-05-10T11:15:50.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:50.718+0000 7f5647c0d700 1 Processor -- start 2024-05-10T11:15:50.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:50.718+0000 7f5647c0d700 1 -- start start 2024-05-10T11:15:50.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:50.718+0000 7f5647c0d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56401023a0 0x7f564019f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:15:50.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:50.718+0000 7f5647c0d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5640102d60 0x7f564019fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:15:50.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:50.718+0000 7f5647c0d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5640108410 0x7f56401998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:15:50.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:50.718+0000 7f5647c0d700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5640112410 con 0x7f56401023a0 2024-05-10T11:15:50.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:50.718+0000 7f5647c0d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5640112290 con 0x7f5640102d60 2024-05-10T11:15:50.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:50.718+0000 7f5647c0d700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5640112590 con 0x7f5640108410 2024-05-10T11:15:50.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:50.718+0000 7f56459a9700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56401023a0 0x7f564019f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:15:50.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:50.718+0000 7f56459a9700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56401023a0 0x7f564019f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53566/0 (socket says 172.21.15.100:53566) 2024-05-10T11:15:50.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:50.718+0000 7f56459a9700 1 -- 172.21.15.100:0/124930908 learned_addr learned my addr 172.21.15.100:0/124930908 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:15:50.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:50.719+0000 7f56451a8700 1 --2- 172.21.15.100:0/124930908 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5640102d60 0x7f564019fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:15:50.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:50.719+0000 7f56461aa700 1 --2- 172.21.15.100:0/124930908 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5640108410 0x7f56401998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:15:50.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:50.719+0000 7f56459a9700 1 -- 172.21.15.100:0/124930908 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5640108410 msgr2=0x7f56401998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:50.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:50.719+0000 7f56459a9700 1 --2- 172.21.15.100:0/124930908 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5640108410 0x7f56401998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:50.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:50.719+0000 7f56459a9700 1 -- 172.21.15.100:0/124930908 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5640102d60 msgr2=0x7f564019fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:50.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:50.719+0000 7f56459a9700 1 --2- 172.21.15.100:0/124930908 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5640102d60 0x7f564019fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:50.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:50.719+0000 7f56459a9700 1 -- 172.21.15.100:0/124930908 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f564019a010 con 0x7f56401023a0 2024-05-10T11:15:50.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:50.719+0000 7f56461aa700 1 --2- 172.21.15.100:0/124930908 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5640108410 0x7f56401998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:15:50.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:50.719+0000 7f56459a9700 1 --2- 172.21.15.100:0/124930908 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56401023a0 0x7f564019f6f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f563c00d980 tx=0x7f563c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:15:50.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:50.719+0000 7f5636ffd700 1 -- 172.21.15.100:0/124930908 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f563c016070 con 0x7f56401023a0 2024-05-10T11:15:50.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:50.719+0000 7f5647c0d700 1 -- 172.21.15.100:0/124930908 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f564019a300 con 0x7f56401023a0 2024-05-10T11:15:50.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:50.719+0000 7f5636ffd700 1 -- 172.21.15.100:0/124930908 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f563c004020 con 0x7f56401023a0 2024-05-10T11:15:50.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:50.719+0000 7f5636ffd700 1 -- 172.21.15.100:0/124930908 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f563c004b90 con 0x7f56401023a0 2024-05-10T11:15:50.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:50.719+0000 7f5647c0d700 1 -- 172.21.15.100:0/124930908 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56401a65c0 con 0x7f56401023a0 2024-05-10T11:15:50.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:50.720+0000 7f5647c0d700 1 -- 172.21.15.100:0/124930908 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f564006a190 con 0x7f56401023a0 2024-05-10T11:15:50.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:50.721+0000 7f5636ffd700 1 -- 172.21.15.100:0/124930908 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f563c00bb60 con 0x7f56401023a0 2024-05-10T11:15:50.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:50.722+0000 7f5636ffd700 1 --2- 172.21.15.100:0/124930908 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f562c0b2910 0x7f562c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:15:50.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:50.722+0000 7f56451a8700 1 --2- 172.21.15.100:0/124930908 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f562c0b2910 0x7f562c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:15:50.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:50.722+0000 7f5636ffd700 1 -- 172.21.15.100:0/124930908 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f563c0e2a10 con 0x7f56401023a0 2024-05-10T11:15:50.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:50.722+0000 7f56451a8700 1 --2- 172.21.15.100:0/124930908 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f562c0b2910 0x7f562c0b4dd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5630005cd0 tx=0x7f5630005c40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:15:50.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:50.725+0000 7f5636ffd700 1 -- 172.21.15.100:0/124930908 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f563c01f020 con 0x7f56401023a0 2024-05-10T11:15:50.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:50.965+0000 7f5647c0d700 1 -- 172.21.15.100:0/124930908 --> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f564006b180 con 0x7f562c0b2910 2024-05-10T11:15:50.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:50.970+0000 7f5636ffd700 1 -- 172.21.15.100:0/124930908 <== mgr.14152 v2:172.21.15.100:6800/2507287356 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f564006b180 con 0x7f562c0b2910 2024-05-10T11:15:50.970 INFO:teuthology.orchestra.run.smithi100.stdout:/volumes/qa/sv_1/a6e142aa-4998-4a49-a18e-698cf65a34e6 2024-05-10T11:15:50.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:50.972+0000 7f5647c0d700 1 -- 172.21.15.100:0/124930908 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f562c0b2910 msgr2=0x7f562c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:50.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:50.972+0000 7f5647c0d700 1 --2- 172.21.15.100:0/124930908 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f562c0b2910 0x7f562c0b4dd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5630005cd0 tx=0x7f5630005c40 comp rx=0 tx=0).stop 2024-05-10T11:15:50.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:50.972+0000 7f5647c0d700 1 -- 172.21.15.100:0/124930908 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56401023a0 msgr2=0x7f564019f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:50.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:50.972+0000 7f5647c0d700 1 --2- 172.21.15.100:0/124930908 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56401023a0 0x7f564019f6f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f563c00d980 tx=0x7f563c00dd40 comp rx=0 tx=0).stop 2024-05-10T11:15:50.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:50.972+0000 7f5647c0d700 1 -- 172.21.15.100:0/124930908 shutdown_connections 2024-05-10T11:15:50.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:50.972+0000 7f5647c0d700 1 --2- 172.21.15.100:0/124930908 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f562c0b2910 0x7f562c0b4dd0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:50.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:50.972+0000 7f5647c0d700 1 --2- 172.21.15.100:0/124930908 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5640108410 0x7f56401998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:50.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:50.972+0000 7f5647c0d700 1 --2- 172.21.15.100:0/124930908 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56401023a0 0x7f564019f6f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:50.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:50.972+0000 7f5647c0d700 1 --2- 172.21.15.100:0/124930908 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5640102d60 0x7f564019fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:50.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:50.973+0000 7f5647c0d700 1 -- 172.21.15.100:0/124930908 >> 172.21.15.100:0/124930908 conn(0x7f5640079350 msgr2=0x7f5640106700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:15:50.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:50.973+0000 7f5647c0d700 1 -- 172.21.15.100:0/124930908 shutdown_connections 2024-05-10T11:15:50.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:50.973+0000 7f5647c0d700 1 -- 172.21.15.100:0/124930908 wait complete. 2024-05-10T11:15:51.761 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T11:15:52.005 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:15:52.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:51 smithi100 ceph-mon[30296]: pgmap v344: 129 pgs: 129 active+clean; 611 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 3.1 KiB/s wr, 7 op/s 2024-05-10T11:15:52.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:51 smithi100 ceph-mon[30296]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:15:52.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:51 smithi178 ceph-mon[33359]: pgmap v344: 129 pgs: 129 active+clean; 611 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 3.1 KiB/s wr, 7 op/s 2024-05-10T11:15:52.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:51 smithi178 ceph-mon[33359]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:15:52.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:51 smithi188 ceph-mon[34075]: pgmap v344: 129 pgs: 129 active+clean; 611 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 3.1 KiB/s wr, 7 op/s 2024-05-10T11:15:52.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:51 smithi188 ceph-mon[34075]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:15:53.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:53.806+0000 7f0711d03700 1 -- 172.21.15.100:0/3273133733 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f070c103560 msgr2=0x7f070c1039e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:53.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:53.806+0000 7f0711d03700 1 --2- 172.21.15.100:0/3273133733 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f070c103560 0x7f070c1039e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f07000099c0 tx=0x7f0700009cd0 comp rx=0 tx=0).stop 2024-05-10T11:15:53.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:53.806+0000 7f0711d03700 1 -- 172.21.15.100:0/3273133733 shutdown_connections 2024-05-10T11:15:53.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:53.806+0000 7f0711d03700 1 --2- 172.21.15.100:0/3273133733 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f070c103f20 0x7f070c110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:53.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:53.806+0000 7f0711d03700 1 --2- 172.21.15.100:0/3273133733 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f070c109540 0x7f070c109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:53.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:53.806+0000 7f0711d03700 1 --2- 172.21.15.100:0/3273133733 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f070c103560 0x7f070c1039e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:53.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:53.806+0000 7f0711d03700 1 -- 172.21.15.100:0/3273133733 >> 172.21.15.100:0/3273133733 conn(0x7f070c07a660 msgr2=0x7f070c07aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:15:53.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:53.806+0000 7f0711d03700 1 -- 172.21.15.100:0/3273133733 shutdown_connections 2024-05-10T11:15:53.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:53.806+0000 7f0711d03700 1 -- 172.21.15.100:0/3273133733 wait complete. 2024-05-10T11:15:53.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:53.807+0000 7f0711d03700 1 Processor -- start 2024-05-10T11:15:53.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:53.807+0000 7f0711d03700 1 -- start start 2024-05-10T11:15:53.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:53.807+0000 7f0711d03700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f070c103560 0x7f070c19f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:15:53.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:53.807+0000 7f0711d03700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f070c103f20 0x7f070c19fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:15:53.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:53.807+0000 7f0711d03700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f070c109540 0x7f070c1998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:15:53.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:53.807+0000 7f0711d03700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f070c113580 con 0x7f070c103f20 2024-05-10T11:15:53.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:53.807+0000 7f0711d03700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f070c113400 con 0x7f070c109540 2024-05-10T11:15:53.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:53.807+0000 7f0711d03700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f070c113700 con 0x7f070c103560 2024-05-10T11:15:53.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:53.807+0000 7f070affd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f070c103f20 0x7f070c19fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:15:53.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:53.807+0000 7f070affd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f070c103f20 0x7f070c19fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53588/0 (socket says 172.21.15.100:53588) 2024-05-10T11:15:53.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:53.807+0000 7f070affd700 1 -- 172.21.15.100:0/2615170914 learned_addr learned my addr 172.21.15.100:0/2615170914 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:15:53.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:53.807+0000 7f070b7fe700 1 --2- 172.21.15.100:0/2615170914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f070c103560 0x7f070c19f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:15:53.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:53.808+0000 7f070affd700 1 -- 172.21.15.100:0/2615170914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f070c103560 msgr2=0x7f070c19f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:53.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:53.808+0000 7f070affd700 1 --2- 172.21.15.100:0/2615170914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f070c103560 0x7f070c19f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:53.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:53.808+0000 7f070bfff700 1 --2- 172.21.15.100:0/2615170914 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f070c109540 0x7f070c1998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:15:53.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:53.808+0000 7f070affd700 1 -- 172.21.15.100:0/2615170914 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f070c109540 msgr2=0x7f070c1998c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:53.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:53.808+0000 7f070affd700 1 --2- 172.21.15.100:0/2615170914 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f070c109540 0x7f070c1998c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:53.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:53.808+0000 7f070affd700 1 -- 172.21.15.100:0/2615170914 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f070c19a020 con 0x7f070c103f20 2024-05-10T11:15:53.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:53.808+0000 7f070b7fe700 1 --2- 172.21.15.100:0/2615170914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f070c103560 0x7f070c19f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:15:53.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:53.808+0000 7f070affd700 1 --2- 172.21.15.100:0/2615170914 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f070c103f20 0x7f070c19fc40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f070000f690 tx=0x7f070000f770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:15:53.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:53.808+0000 7f0708ff9700 1 -- 172.21.15.100:0/2615170914 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0700027070 con 0x7f070c103f20 2024-05-10T11:15:53.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:53.808+0000 7f0708ff9700 1 -- 172.21.15.100:0/2615170914 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f07000056d0 con 0x7f070c103f20 2024-05-10T11:15:53.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:53.808+0000 7f0711d03700 1 -- 172.21.15.100:0/2615170914 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f070c19a2b0 con 0x7f070c103f20 2024-05-10T11:15:53.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:53.808+0000 7f0708ff9700 1 -- 172.21.15.100:0/2615170914 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07000173f0 con 0x7f070c103f20 2024-05-10T11:15:53.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:53.808+0000 7f0711d03700 1 -- 172.21.15.100:0/2615170914 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f070c1a65d0 con 0x7f070c103f20 2024-05-10T11:15:53.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:53.809+0000 7f0711d03700 1 -- 172.21.15.100:0/2615170914 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f070c10b9d0 con 0x7f070c103f20 2024-05-10T11:15:53.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:53.810+0000 7f0708ff9700 1 -- 172.21.15.100:0/2615170914 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f070002a070 con 0x7f070c103f20 2024-05-10T11:15:53.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:53.811+0000 7f0708ff9700 1 --2- 172.21.15.100:0/2615170914 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f06f40b2910 0x7f06f40b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:15:53.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:53.811+0000 7f070b7fe700 1 --2- 172.21.15.100:0/2615170914 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f06f40b2910 0x7f06f40b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:15:53.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:53.811+0000 7f0708ff9700 1 -- 172.21.15.100:0/2615170914 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f07000e37b0 con 0x7f070c103f20 2024-05-10T11:15:53.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:53.811+0000 7f070b7fe700 1 --2- 172.21.15.100:0/2615170914 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f06f40b2910 0x7f06f40b4dd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f070c19aa00 tx=0x7f06fc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:15:53.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:53.814+0000 7f0708ff9700 1 -- 172.21.15.100:0/2615170914 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f0700013070 con 0x7f070c103f20 2024-05-10T11:15:54.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:54.071+0000 7f0711d03700 1 -- 172.21.15.100:0/2615170914 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f070c06a190 con 0x7f070c103f20 2024-05-10T11:15:54.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:54.072+0000 7f0708ff9700 1 -- 172.21.15.100:0/2615170914 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11894 (secure 0 0 0) 0x7f07000ab130 con 0x7f070c103f20 2024-05-10T11:15:54.072 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:15:54.072 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2072225012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2072225012},{"type":"v1","addr":"172.21.15.100:6835","nonce":2072225012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/401939540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":401939540},{"type":"v1","addr":"172.21.15.100:6837","nonce":401939540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14571,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/3946311025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3946311025},{"type":"v1","addr":"172.21.15.100:6839","nonce":3946311025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3736371660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3736371660},{"type":"v1","addr":"172.21.15.100:6841","nonce":3736371660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/2741810791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2741810791},{"type":"v1","addr":"172.21.15.178:6833","nonce":2741810791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1704891041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1704891041},{"type":"v1","addr":"172.21.15.188:6833","nonce":1704891041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T11:14:27.242452+0000","modified":"2024-05-10T11:15:35.672461+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24439,"mds_1":24437,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/2124076312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2124076312},{"type":"v1","addr":"172.21.15.178:6835","nonce":2124076312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/3224358425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3224358425},{"type":"v1","addr":"172.21.15.178:6837","nonce":3224358425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/515547690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":515547690},{"type":"v1","addr":"172.21.15.188:6835","nonce":515547690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/1943342620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1943342620},{"type":"v1","addr":"172.21.15.188:6837","nonce":1943342620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.178:6839/3760985404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3760985404},{"type":"v1","addr":"172.21.15.178:6839","nonce":3760985404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.188:6839/3194917070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":3194917070},{"type":"v1","addr":"172.21.15.188:6839","nonce":3194917070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24439,"qdb_cluster":[24437,24439,24427]},"id":1}]} 2024-05-10T11:15:54.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:54.074+0000 7f0711d03700 1 -- 172.21.15.100:0/2615170914 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f06f40b2910 msgr2=0x7f06f40b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:54.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:54.074+0000 7f0711d03700 1 --2- 172.21.15.100:0/2615170914 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f06f40b2910 0x7f06f40b4dd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f070c19aa00 tx=0x7f06fc008040 comp rx=0 tx=0).stop 2024-05-10T11:15:54.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:54.074+0000 7f0711d03700 1 -- 172.21.15.100:0/2615170914 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f070c103f20 msgr2=0x7f070c19fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:54.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:54.074+0000 7f0711d03700 1 --2- 172.21.15.100:0/2615170914 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f070c103f20 0x7f070c19fc40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f070000f690 tx=0x7f070000f770 comp rx=0 tx=0).stop 2024-05-10T11:15:54.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:54.074+0000 7f0711d03700 1 -- 172.21.15.100:0/2615170914 shutdown_connections 2024-05-10T11:15:54.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:54.074+0000 7f0711d03700 1 --2- 172.21.15.100:0/2615170914 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f06f40b2910 0x7f06f40b4dd0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:54.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:54.074+0000 7f0711d03700 1 --2- 172.21.15.100:0/2615170914 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f070c109540 0x7f070c1998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:54.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:54.074+0000 7f0711d03700 1 --2- 172.21.15.100:0/2615170914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f070c103560 0x7f070c19f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:54.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:54.074+0000 7f0711d03700 1 --2- 172.21.15.100:0/2615170914 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f070c103f20 0x7f070c19fc40 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:54.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:54.074+0000 7f0711d03700 1 -- 172.21.15.100:0/2615170914 >> 172.21.15.100:0/2615170914 conn(0x7f070c07a660 msgr2=0x7f070c1023e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:15:54.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:54.074+0000 7f0711d03700 1 -- 172.21.15.100:0/2615170914 shutdown_connections 2024-05-10T11:15:54.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:54.074+0000 7f0711d03700 1 -- 172.21.15.100:0/2615170914 wait complete. 2024-05-10T11:15:54.075 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 26 2024-05-10T11:15:54.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:53 smithi178 ceph-mon[33359]: pgmap v345: 129 pgs: 129 active+clean; 632 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.2 KiB/s wr, 10 op/s 2024-05-10T11:15:54.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:53 smithi188 ceph-mon[34075]: pgmap v345: 129 pgs: 129 active+clean; 632 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.2 KiB/s wr, 10 op/s 2024-05-10T11:15:54.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:53 smithi100 ceph-mon[30296]: pgmap v345: 129 pgs: 129 active+clean; 632 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.2 KiB/s wr, 10 op/s 2024-05-10T11:15:54.855 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-10T11:15:55.109 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:15:55.231 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:54 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/2615170914' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:15:55.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:54 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/2615170914' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:15:55.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:54 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/2615170914' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:15:56.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:55 smithi178 ceph-mon[33359]: pgmap v346: 129 pgs: 129 active+clean; 632 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 4.3 KiB/s wr, 7 op/s 2024-05-10T11:15:56.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:55 smithi188 ceph-mon[34075]: pgmap v346: 129 pgs: 129 active+clean; 632 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 4.3 KiB/s wr, 7 op/s 2024-05-10T11:15:56.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:55 smithi100 ceph-mon[30296]: pgmap v346: 129 pgs: 129 active+clean; 632 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 4.3 KiB/s wr, 7 op/s 2024-05-10T11:15:56.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:56.937+0000 7f414bac1700 1 -- 172.21.15.100:0/4282715839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4144108410 msgr2=0x7f41441087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:56.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:56.937+0000 7f414bac1700 1 --2- 172.21.15.100:0/4282715839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4144108410 0x7f41441087f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4140009990 tx=0x7f4140009ca0 comp rx=0 tx=0).stop 2024-05-10T11:15:56.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:56.937+0000 7f414bac1700 1 -- 172.21.15.100:0/4282715839 shutdown_connections 2024-05-10T11:15:56.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:56.937+0000 7f414bac1700 1 --2- 172.21.15.100:0/4282715839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4144102d60 0x7f414410f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:56.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:56.937+0000 7f414bac1700 1 --2- 172.21.15.100:0/4282715839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4144108410 0x7f41441087f0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:56.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:56.937+0000 7f414bac1700 1 --2- 172.21.15.100:0/4282715839 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41441023a0 0x7f4144102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:56.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:56.937+0000 7f414bac1700 1 -- 172.21.15.100:0/4282715839 >> 172.21.15.100:0/4282715839 conn(0x7f4144079350 msgr2=0x7f4144079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:15:56.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:56.937+0000 7f414bac1700 1 -- 172.21.15.100:0/4282715839 shutdown_connections 2024-05-10T11:15:56.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:56.937+0000 7f414bac1700 1 -- 172.21.15.100:0/4282715839 wait complete. 2024-05-10T11:15:56.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:56.938+0000 7f414bac1700 1 Processor -- start 2024-05-10T11:15:56.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:56.938+0000 7f414bac1700 1 -- start start 2024-05-10T11:15:56.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:56.938+0000 7f414bac1700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41441023a0 0x7f4144198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:15:56.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:56.938+0000 7f414bac1700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4144102d60 0x7f4144198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:15:56.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:56.938+0000 7f414bac1700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4144108410 0x7f4144199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:15:56.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:56.938+0000 7f414bac1700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f41441124e0 con 0x7f41441023a0 2024-05-10T11:15:56.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:56.938+0000 7f414bac1700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4144112360 con 0x7f4144108410 2024-05-10T11:15:56.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:56.938+0000 7f414bac1700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4144112660 con 0x7f4144102d60 2024-05-10T11:15:56.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:56.938+0000 7f414985d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41441023a0 0x7f4144198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:15:56.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:56.938+0000 7f414985d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41441023a0 0x7f4144198810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37052/0 (socket says 172.21.15.100:37052) 2024-05-10T11:15:56.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:56.938+0000 7f414985d700 1 -- 172.21.15.100:0/2427839977 learned_addr learned my addr 172.21.15.100:0/2427839977 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:15:56.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:56.938+0000 7f414905c700 1 --2- 172.21.15.100:0/2427839977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4144102d60 0x7f4144198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:15:56.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:56.938+0000 7f414a05e700 1 --2- 172.21.15.100:0/2427839977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4144108410 0x7f4144199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:15:56.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:56.938+0000 7f414985d700 1 -- 172.21.15.100:0/2427839977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4144102d60 msgr2=0x7f4144198d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:56.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:56.938+0000 7f414985d700 1 --2- 172.21.15.100:0/2427839977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4144102d60 0x7f4144198d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:56.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:56.938+0000 7f414985d700 1 -- 172.21.15.100:0/2427839977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4144108410 msgr2=0x7f4144199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:56.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:56.938+0000 7f414985d700 1 --2- 172.21.15.100:0/2427839977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4144108410 0x7f4144199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:56.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:56.938+0000 7f414985d700 1 -- 172.21.15.100:0/2427839977 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41441a8080 con 0x7f41441023a0 2024-05-10T11:15:56.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:56.938+0000 7f414905c700 1 --2- 172.21.15.100:0/2427839977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4144102d60 0x7f4144198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:15:56.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:56.938+0000 7f414a05e700 1 --2- 172.21.15.100:0/2427839977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4144108410 0x7f4144199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:15:56.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:56.938+0000 7f414985d700 1 --2- 172.21.15.100:0/2427839977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41441023a0 0x7f4144198810 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4140000c00 tx=0x7f41400047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:15:56.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:56.938+0000 7f413affd700 1 -- 172.21.15.100:0/2427839977 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4140027070 con 0x7f41441023a0 2024-05-10T11:15:56.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:56.939+0000 7f413affd700 1 -- 172.21.15.100:0/2427839977 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f414000fdb0 con 0x7f41441023a0 2024-05-10T11:15:56.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:56.939+0000 7f413affd700 1 -- 172.21.15.100:0/2427839977 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41400173f0 con 0x7f41441023a0 2024-05-10T11:15:56.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:56.939+0000 7f414bac1700 1 -- 172.21.15.100:0/2427839977 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41441a8310 con 0x7f41441023a0 2024-05-10T11:15:56.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:56.939+0000 7f414bac1700 1 -- 172.21.15.100:0/2427839977 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41441a8760 con 0x7f41441023a0 2024-05-10T11:15:56.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:56.939+0000 7f414bac1700 1 -- 172.21.15.100:0/2427839977 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f414406a190 con 0x7f41441023a0 2024-05-10T11:15:56.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:56.942+0000 7f413affd700 1 -- 172.21.15.100:0/2427839977 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f414002a070 con 0x7f41441023a0 2024-05-10T11:15:56.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:56.942+0000 7f413affd700 1 --2- 172.21.15.100:0/2427839977 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f41300b2910 0x7f41300b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:15:56.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:56.942+0000 7f414905c700 1 --2- 172.21.15.100:0/2427839977 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f41300b2910 0x7f41300b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:15:56.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:56.942+0000 7f413affd700 1 -- 172.21.15.100:0/2427839977 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f41400e3cb0 con 0x7f41441023a0 2024-05-10T11:15:56.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:56.943+0000 7f414905c700 1 --2- 172.21.15.100:0/2427839977 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f41300b2910 0x7f41300b4dd0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f414419a970 tx=0x7f4134008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:15:56.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:56.946+0000 7f413affd700 1 -- 172.21.15.100:0/2427839977 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f4140031070 con 0x7f41441023a0 2024-05-10T11:15:57.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:57.149+0000 7f414bac1700 1 -- 172.21.15.100:0/2427839977 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f414402e6d0 con 0x7f41441023a0 2024-05-10T11:15:57.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:57.151+0000 7f413affd700 1 -- 172.21.15.100:0/2427839977 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24737 (secure 0 0 0) 0x7f4140013070 con 0x7f41441023a0 2024-05-10T11:15:57.151 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:15:57.152 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":89,"fsid":"20a542cc-0ebd-11ef-bc98-c7b262605968","created":"2024-05-10T11:06:05.454446+0000","modified":"2024-05-10T11:15:01.418144+0000","last_up_change":"2024-05-10T11:13:18.185471+0000","last_in_change":"2024-05-10T11:12:57.268678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T11:09:26.176441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T11:14:21.004285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T11:14:24.179414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T11:14:33.308517+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5f031418-da23-4e85-8669-160960dadd09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":643028278},{"type":"v1","addr":"172.21.15.100:6803","nonce":643028278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":643028278},{"type":"v1","addr":"172.21.15.100:6805","nonce":643028278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":643028278},{"type":"v1","addr":"172.21.15.100:6809","nonce":643028278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":643028278},{"type":"v1","addr":"172.21.15.100:6807","nonce":643028278}]},"public_addr":"172.21.15.100:6803/643028278","cluster_addr":"172.21.15.100:6805/643028278","heartbeat_back_addr":"172.21.15.100:6809/643028278","heartbeat_front_addr":"172.21.15.100:6807/643028278","state":["exists","up"]},{"osd":1,"uuid":"a908cdc9-001e-4b65-9ab0-0dd08dba20f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":3103987017},{"type":"v1","addr":"172.21.15.178:6801","nonce":3103987017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":3103987017},{"type":"v1","addr":"172.21.15.178:6803","nonce":3103987017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":3103987017},{"type":"v1","addr":"172.21.15.178:6807","nonce":3103987017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":3103987017},{"type":"v1","addr":"172.21.15.178:6805","nonce":3103987017}]},"public_addr":"172.21.15.178:6801/3103987017","cluster_addr":"172.21.15.178:6803/3103987017","heartbeat_back_addr":"172.21.15.178:6807/3103987017","heartbeat_front_addr":"172.21.15.178:6805/3103987017","state":["exists","up"]},{"osd":2,"uuid":"4d8f89ae-9424-46e9-9c80-17c47cc5e8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3991466468},{"type":"v1","addr":"172.21.15.188:6801","nonce":3991466468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3991466468},{"type":"v1","addr":"172.21.15.188:6803","nonce":3991466468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3991466468},{"type":"v1","addr":"172.21.15.188:6807","nonce":3991466468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3991466468},{"type":"v1","addr":"172.21.15.188:6805","nonce":3991466468}]},"public_addr":"172.21.15.188:6801/3991466468","cluster_addr":"172.21.15.188:6803/3991466468","heartbeat_back_addr":"172.21.15.188:6807/3991466468","heartbeat_front_addr":"172.21.15.188:6805/3991466468","state":["exists","up"]},{"osd":3,"uuid":"4c8865aa-fef4-4710-b766-1385adb9462c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1713357192},{"type":"v1","addr":"172.21.15.100:6811","nonce":1713357192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1713357192},{"type":"v1","addr":"172.21.15.100:6813","nonce":1713357192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1713357192},{"type":"v1","addr":"172.21.15.100:6817","nonce":1713357192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1713357192},{"type":"v1","addr":"172.21.15.100:6815","nonce":1713357192}]},"public_addr":"172.21.15.100:6811/1713357192","cluster_addr":"172.21.15.100:6813/1713357192","heartbeat_back_addr":"172.21.15.100:6817/1713357192","heartbeat_front_addr":"172.21.15.100:6815/1713357192","state":["exists","up"]},{"osd":4,"uuid":"a9653800-da57-43eb-8781-3188e8a3949f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":2222814192},{"type":"v1","addr":"172.21.15.178:6809","nonce":2222814192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":2222814192},{"type":"v1","addr":"172.21.15.178:6811","nonce":2222814192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":2222814192},{"type":"v1","addr":"172.21.15.178:6815","nonce":2222814192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":2222814192},{"type":"v1","addr":"172.21.15.178:6813","nonce":2222814192}]},"public_addr":"172.21.15.178:6809/2222814192","cluster_addr":"172.21.15.178:6811/2222814192","heartbeat_back_addr":"172.21.15.178:6815/2222814192","heartbeat_front_addr":"172.21.15.178:6813/2222814192","state":["exists","up"]},{"osd":5,"uuid":"fb5f8af2-1eb0-4d8a-a7a3-ed6dd68e692e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3517375935},{"type":"v1","addr":"172.21.15.188:6809","nonce":3517375935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3517375935},{"type":"v1","addr":"172.21.15.188:6811","nonce":3517375935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3517375935},{"type":"v1","addr":"172.21.15.188:6815","nonce":3517375935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3517375935},{"type":"v1","addr":"172.21.15.188:6813","nonce":3517375935}]},"public_addr":"172.21.15.188:6809/3517375935","cluster_addr":"172.21.15.188:6811/3517375935","heartbeat_back_addr":"172.21.15.188:6815/3517375935","heartbeat_front_addr":"172.21.15.188:6813/3517375935","state":["exists","up"]},{"osd":6,"uuid":"84b57387-0a94-4e49-b099-493d329b87dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1410593862},{"type":"v1","addr":"172.21.15.100:6819","nonce":1410593862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1410593862},{"type":"v1","addr":"172.21.15.100:6821","nonce":1410593862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1410593862},{"type":"v1","addr":"172.21.15.100:6825","nonce":1410593862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1410593862},{"type":"v1","addr":"172.21.15.100:6823","nonce":1410593862}]},"public_addr":"172.21.15.100:6819/1410593862","cluster_addr":"172.21.15.100:6821/1410593862","heartbeat_back_addr":"172.21.15.100:6825/1410593862","heartbeat_front_addr":"172.21.15.100:6823/1410593862","state":["exists","up"]},{"osd":7,"uuid":"02ada28f-ac4f-47cf-983f-8a6da1140abe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2458690455},{"type":"v1","addr":"172.21.15.178:6817","nonce":2458690455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2458690455},{"type":"v1","addr":"172.21.15.178:6819","nonce":2458690455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2458690455},{"type":"v1","addr":"172.21.15.178:6823","nonce":2458690455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2458690455},{"type":"v1","addr":"172.21.15.178:6821","nonce":2458690455}]},"public_addr":"172.21.15.178:6817/2458690455","cluster_addr":"172.21.15.178:6819/2458690455","heartbeat_back_addr":"172.21.15.178:6823/2458690455","heartbeat_front_addr":"172.21.15.178:6821/2458690455","state":["exists","up"]},{"osd":8,"uuid":"e7e489bc-15fc-4a4c-b297-d1bb626513ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1523872826},{"type":"v1","addr":"172.21.15.188:6817","nonce":1523872826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1523872826},{"type":"v1","addr":"172.21.15.188:6819","nonce":1523872826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1523872826},{"type":"v1","addr":"172.21.15.188:6823","nonce":1523872826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1523872826},{"type":"v1","addr":"172.21.15.188:6821","nonce":1523872826}]},"public_addr":"172.21.15.188:6817/1523872826","cluster_addr":"172.21.15.188:6819/1523872826","heartbeat_back_addr":"172.21.15.188:6823/1523872826","heartbeat_front_addr":"172.21.15.188:6821/1523872826","state":["exists","up"]},{"osd":9,"uuid":"72328326-cd16-45d9-bfd9-40f6c5c20f5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3585761978},{"type":"v1","addr":"172.21.15.100:6827","nonce":3585761978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3585761978},{"type":"v1","addr":"172.21.15.100:6829","nonce":3585761978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3585761978},{"type":"v1","addr":"172.21.15.100:6833","nonce":3585761978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3585761978},{"type":"v1","addr":"172.21.15.100:6831","nonce":3585761978}]},"public_addr":"172.21.15.100:6827/3585761978","cluster_addr":"172.21.15.100:6829/3585761978","heartbeat_back_addr":"172.21.15.100:6833/3585761978","heartbeat_front_addr":"172.21.15.100:6831/3585761978","state":["exists","up"]},{"osd":10,"uuid":"a355f9a8-91a3-41d4-814d-5a9c9b953706","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2985841725},{"type":"v1","addr":"172.21.15.178:6825","nonce":2985841725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2985841725},{"type":"v1","addr":"172.21.15.178:6827","nonce":2985841725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2985841725},{"type":"v1","addr":"172.21.15.178:6831","nonce":2985841725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2985841725},{"type":"v1","addr":"172.21.15.178:6829","nonce":2985841725}]},"public_addr":"172.21.15.178:6825/2985841725","cluster_addr":"172.21.15.178:6827/2985841725","heartbeat_back_addr":"172.21.15.178:6831/2985841725","heartbeat_front_addr":"172.21.15.178:6829/2985841725","state":["exists","up"]},{"osd":11,"uuid":"76a5e13c-7e38-42f9-bfa7-7fedec7c7184","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2574955118},{"type":"v1","addr":"172.21.15.188:6825","nonce":2574955118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2574955118},{"type":"v1","addr":"172.21.15.188:6827","nonce":2574955118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2574955118},{"type":"v1","addr":"172.21.15.188:6831","nonce":2574955118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2574955118},{"type":"v1","addr":"172.21.15.188:6829","nonce":2574955118}]},"public_addr":"172.21.15.188:6825/2574955118","cluster_addr":"172.21.15.188:6827/2574955118","heartbeat_back_addr":"172.21.15.188:6831/2574955118","heartbeat_front_addr":"172.21.15.188:6829/2574955118","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:08:36.086637+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:08:58.948031+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:09:22.142517+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:09:47.655748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:10:12.853999+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:10:37.911383+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:11:04.448815+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:11:30.331740+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:11:55.819278+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:12:23.204612+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:12:49.484835+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:13:15.783531+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:6800/1597433518":"2024-05-11T11:07:00.092732+0000","172.21.15.100:6800/1086450256":"2024-05-11T11:06:33.114176+0000","172.21.15.100:0/767964410":"2024-05-11T11:07:00.092732+0000","172.21.15.100:6801/1597433518":"2024-05-11T11:07:00.092732+0000","172.21.15.100:6801/1086450256":"2024-05-11T11:06:33.114176+0000","172.21.15.100:0/2770572070":"2024-05-11T11:06:33.114176+0000","172.21.15.100:0/1379340513":"2024-05-11T11:06:33.114176+0000","172.21.15.100:0/3025646953":"2024-05-11T11:07:00.092732+0000","172.21.15.100:0/4099258930":"2024-05-11T11:07:00.092732+0000","172.21.15.100:0/4267693567":"2024-05-11T11:06:33.114176+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T11:15:57.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:57.153+0000 7f414bac1700 1 -- 172.21.15.100:0/2427839977 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f41300b2910 msgr2=0x7f41300b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:57.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:57.153+0000 7f414bac1700 1 --2- 172.21.15.100:0/2427839977 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f41300b2910 0x7f41300b4dd0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f414419a970 tx=0x7f4134008040 comp rx=0 tx=0).stop 2024-05-10T11:15:57.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:57.153+0000 7f414bac1700 1 -- 172.21.15.100:0/2427839977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41441023a0 msgr2=0x7f4144198810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:57.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:57.153+0000 7f414bac1700 1 --2- 172.21.15.100:0/2427839977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41441023a0 0x7f4144198810 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4140000c00 tx=0x7f41400047b0 comp rx=0 tx=0).stop 2024-05-10T11:15:57.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:57.153+0000 7f414bac1700 1 -- 172.21.15.100:0/2427839977 shutdown_connections 2024-05-10T11:15:57.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:57.153+0000 7f414bac1700 1 --2- 172.21.15.100:0/2427839977 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f41300b2910 0x7f41300b4dd0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:57.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:57.153+0000 7f414bac1700 1 --2- 172.21.15.100:0/2427839977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4144108410 0x7f4144199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:57.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:57.153+0000 7f414bac1700 1 --2- 172.21.15.100:0/2427839977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41441023a0 0x7f4144198810 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:57.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:57.153+0000 7f414bac1700 1 --2- 172.21.15.100:0/2427839977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4144102d60 0x7f4144198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:57.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:57.153+0000 7f414bac1700 1 -- 172.21.15.100:0/2427839977 >> 172.21.15.100:0/2427839977 conn(0x7f4144079350 msgr2=0x7f4144101230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:15:57.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:57.153+0000 7f414bac1700 1 -- 172.21.15.100:0/2427839977 shutdown_connections 2024-05-10T11:15:57.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:57.154+0000 7f414bac1700 1 -- 172.21.15.100:0/2427839977 wait complete. 2024-05-10T11:15:57.783 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-10T11:15:57.784 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-10T11:15:58.028 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:15:58.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:57 smithi100 ceph-mon[30296]: pgmap v347: 129 pgs: 129 active+clean; 632 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-05-10T11:15:58.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:57 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/2427839977' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T11:15:58.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:57 smithi178 ceph-mon[33359]: pgmap v347: 129 pgs: 129 active+clean; 632 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-05-10T11:15:58.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:57 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/2427839977' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T11:15:58.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:57 smithi188 ceph-mon[34075]: pgmap v347: 129 pgs: 129 active+clean; 632 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-05-10T11:15:58.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:57 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/2427839977' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T11:15:59.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:59.786+0000 7f1003f42700 1 -- 172.21.15.100:0/4246633147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ffc06bc10 msgr2=0x7f0ffc06c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:59.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:59.786+0000 7f1003f42700 1 --2- 172.21.15.100:0/4246633147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ffc06bc10 0x7f0ffc06c090 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0fec009990 tx=0x7f0fec009ca0 comp rx=0 tx=0).stop 2024-05-10T11:15:59.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:59.786+0000 7f1003f42700 1 -- 172.21.15.100:0/4246633147 shutdown_connections 2024-05-10T11:15:59.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:59.786+0000 7f1003f42700 1 --2- 172.21.15.100:0/4246633147 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ffc06c5d0 0x7f0ffc110380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:59.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:59.786+0000 7f1003f42700 1 --2- 172.21.15.100:0/4246633147 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ffc109540 0x7f0ffc109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:59.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:59.786+0000 7f1003f42700 1 --2- 172.21.15.100:0/4246633147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ffc06bc10 0x7f0ffc06c090 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:59.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:59.786+0000 7f1003f42700 1 -- 172.21.15.100:0/4246633147 >> 172.21.15.100:0/4246633147 conn(0x7f0ffc078410 msgr2=0x7f0ffc078820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:15:59.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:59.786+0000 7f1003f42700 1 -- 172.21.15.100:0/4246633147 shutdown_connections 2024-05-10T11:15:59.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:59.787+0000 7f1003f42700 1 -- 172.21.15.100:0/4246633147 wait complete. 2024-05-10T11:15:59.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:59.787+0000 7f1003f42700 1 Processor -- start 2024-05-10T11:15:59.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:59.787+0000 7f1003f42700 1 -- start start 2024-05-10T11:15:59.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:59.787+0000 7f1003f42700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ffc06bc10 0x7f0ffc19f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:15:59.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:59.787+0000 7f1003f42700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ffc06c5d0 0x7f0ffc19fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:15:59.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:59.787+0000 7f1003f42700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ffc109540 0x7f0ffc1998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:15:59.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:59.787+0000 7f1003f42700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc113550 con 0x7f0ffc109540 2024-05-10T11:15:59.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:59.787+0000 7f1003f42700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc1133d0 con 0x7f0ffc06c5d0 2024-05-10T11:15:59.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:59.787+0000 7f1003f42700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc1136d0 con 0x7f0ffc06bc10 2024-05-10T11:15:59.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:59.787+0000 7f10024df700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ffc109540 0x7f0ffc1998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:15:59.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:59.787+0000 7f10024df700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ffc109540 0x7f0ffc1998b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37070/0 (socket says 172.21.15.100:37070) 2024-05-10T11:15:59.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:59.787+0000 7f10024df700 1 -- 172.21.15.100:0/463381954 learned_addr learned my addr 172.21.15.100:0/463381954 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:15:59.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:59.788+0000 7f1001cde700 1 --2- 172.21.15.100:0/463381954 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ffc06bc10 0x7f0ffc19f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:15:59.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:59.788+0000 7f10024df700 1 -- 172.21.15.100:0/463381954 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ffc06bc10 msgr2=0x7f0ffc19f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:59.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:59.788+0000 7f10014dd700 1 --2- 172.21.15.100:0/463381954 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ffc06c5d0 0x7f0ffc19fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:15:59.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:59.788+0000 7f10024df700 1 --2- 172.21.15.100:0/463381954 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ffc06bc10 0x7f0ffc19f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:59.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:59.788+0000 7f10024df700 1 -- 172.21.15.100:0/463381954 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ffc06c5d0 msgr2=0x7f0ffc19fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:15:59.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:59.788+0000 7f10024df700 1 --2- 172.21.15.100:0/463381954 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ffc06c5d0 0x7f0ffc19fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:15:59.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:59.788+0000 7f10024df700 1 -- 172.21.15.100:0/463381954 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ffc19a010 con 0x7f0ffc109540 2024-05-10T11:15:59.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:59.788+0000 7f10014dd700 1 --2- 172.21.15.100:0/463381954 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ffc06c5d0 0x7f0ffc19fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:15:59.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:59.788+0000 7f10024df700 1 --2- 172.21.15.100:0/463381954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ffc109540 0x7f0ffc1998b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4007d70 tx=0x7f0ff400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:15:59.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:59.788+0000 7f0ff2ffd700 1 -- 172.21.15.100:0/463381954 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff4016020 con 0x7f0ffc109540 2024-05-10T11:15:59.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:59.788+0000 7f0ff2ffd700 1 -- 172.21.15.100:0/463381954 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0ff400ad40 con 0x7f0ffc109540 2024-05-10T11:15:59.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:59.788+0000 7f1003f42700 1 -- 172.21.15.100:0/463381954 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ffc19a300 con 0x7f0ffc109540 2024-05-10T11:15:59.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:59.788+0000 7f0ff2ffd700 1 -- 172.21.15.100:0/463381954 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff40115e0 con 0x7f0ffc109540 2024-05-10T11:15:59.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:59.788+0000 7f1003f42700 1 -- 172.21.15.100:0/463381954 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ffc1a65c0 con 0x7f0ffc109540 2024-05-10T11:15:59.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:59.789+0000 7f1003f42700 1 -- 172.21.15.100:0/463381954 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ffc06a190 con 0x7f0ffc109540 2024-05-10T11:15:59.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:59.791+0000 7f0ff2ffd700 1 -- 172.21.15.100:0/463381954 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f0ff4019020 con 0x7f0ffc109540 2024-05-10T11:15:59.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:59.791+0000 7f0ff2ffd700 1 --2- 172.21.15.100:0/463381954 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0fe80b2910 0x7f0fe80b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:15:59.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:59.792+0000 7f1001cde700 1 --2- 172.21.15.100:0/463381954 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0fe80b2910 0x7f0fe80b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:15:59.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:59.792+0000 7f0ff2ffd700 1 -- 172.21.15.100:0/463381954 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f0ff40e2a70 con 0x7f0ffc109540 2024-05-10T11:15:59.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:59.792+0000 7f1001cde700 1 --2- 172.21.15.100:0/463381954 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0fe80b2910 0x7f0fe80b4dd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc19a9f0 tx=0x7f0ff8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:15:59.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:15:59.795+0000 7f0ff2ffd700 1 -- 172.21.15.100:0/463381954 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f0ff4020020 con 0x7f0ffc109540 2024-05-10T11:16:00.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:00.002+0000 7f1003f42700 1 -- 172.21.15.100:0/463381954 --> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0ffc0638e0 con 0x7f0fe80b2910 2024-05-10T11:16:00.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:00.012+0000 7f0ff2ffd700 1 -- 172.21.15.100:0/463381954 <== mgr.14152 v2:172.21.15.100:6800/2507287356 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340353 (secure 0 0 0) 0x7f0ffc0638e0 con 0x7f0fe80b2910 2024-05-10T11:16:00.018 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:16:00.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:00.016+0000 7f1003f42700 1 -- 172.21.15.100:0/463381954 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0fe80b2910 msgr2=0x7f0fe80b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:00.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:00.016+0000 7f1003f42700 1 --2- 172.21.15.100:0/463381954 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0fe80b2910 0x7f0fe80b4dd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc19a9f0 tx=0x7f0ff8008040 comp rx=0 tx=0).stop 2024-05-10T11:16:00.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:00.016+0000 7f1003f42700 1 -- 172.21.15.100:0/463381954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ffc109540 msgr2=0x7f0ffc1998b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:00.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:00.016+0000 7f1003f42700 1 --2- 172.21.15.100:0/463381954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ffc109540 0x7f0ffc1998b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4007d70 tx=0x7f0ff400a340 comp rx=0 tx=0).stop 2024-05-10T11:16:00.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:00.016+0000 7f1003f42700 1 -- 172.21.15.100:0/463381954 shutdown_connections 2024-05-10T11:16:00.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:00.016+0000 7f1003f42700 1 --2- 172.21.15.100:0/463381954 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0fe80b2910 0x7f0fe80b4dd0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:00.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:00.016+0000 7f1003f42700 1 --2- 172.21.15.100:0/463381954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ffc109540 0x7f0ffc1998b0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:00.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:00.016+0000 7f1003f42700 1 --2- 172.21.15.100:0/463381954 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ffc06bc10 0x7f0ffc19f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:00.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:00.016+0000 7f1003f42700 1 --2- 172.21.15.100:0/463381954 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ffc06c5d0 0x7f0ffc19fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:00.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:00.016+0000 7f1003f42700 1 -- 172.21.15.100:0/463381954 >> 172.21.15.100:0/463381954 conn(0x7f0ffc078410 msgr2=0x7f0ffc1074e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:16:00.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:00.016+0000 7f1003f42700 1 -- 172.21.15.100:0/463381954 shutdown_connections 2024-05-10T11:16:00.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:00.016+0000 7f1003f42700 1 -- 172.21.15.100:0/463381954 wait complete. 2024-05-10T11:16:00.023 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-05-10T11:16:00.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:15:59 smithi178 ceph-mon[33359]: pgmap v348: 129 pgs: 129 active+clean; 638 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.7 KiB/s wr, 9 op/s 2024-05-10T11:16:00.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:15:59 smithi188 ceph-mon[34075]: pgmap v348: 129 pgs: 129 active+clean; 638 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.7 KiB/s wr, 9 op/s 2024-05-10T11:16:00.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:15:59 smithi100 ceph-mon[30296]: pgmap v348: 129 pgs: 129 active+clean; 638 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.7 KiB/s wr, 9 op/s 2024-05-10T11:16:00.972 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":348,"stamp":"2024-05-10T11:15:58.204878+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":652877,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":247,"num_read_kb":217,"num_write":291,"num_write_kb":2265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":236,"ondisk_log_size":236,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":392196,"kb_used_data":13924,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124549628,"statfs":{"total":1151940427776,"available":1151538819072,"internally_reserved":0,"allocated":14258176,"data_stored":7990034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":387233083},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":47558,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":85,"num_read_kb":93,"num_write":26,"num_write_kb":57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003408"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.425894+0000","last_change":"2024-05-10T11:14:25.286878+0000","last_active":"2024-05-10T11:15:01.425894+0000","last_peered":"2024-05-10T11:15:01.425894+0000","last_clean":"2024-05-10T11:15:01.425894+0000","last_became_active":"2024-05-10T11:14:25.286462+0000","last_became_peered":"2024-05-10T11:14:25.286462+0000","last_unstale":"2024-05-10T11:15:01.425894+0000","last_undegraded":"2024-05-10T11:15:01.425894+0000","last_fullsized":"2024-05-10T11:15:01.425894+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:13:40.627138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.425944+0000","last_change":"2024-05-10T11:14:25.280884+0000","last_active":"2024-05-10T11:15:01.425944+0000","last_peered":"2024-05-10T11:15:01.425944+0000","last_clean":"2024-05-10T11:15:01.425944+0000","last_became_active":"2024-05-10T11:14:25.280564+0000","last_became_peered":"2024-05-10T11:14:25.280564+0000","last_unstale":"2024-05-10T11:15:01.425944+0000","last_undegraded":"2024-05-10T11:15:01.425944+0000","last_fullsized":"2024-05-10T11:15:01.425944+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:14:16.735326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.425779+0000","last_change":"2024-05-10T11:14:25.278676+0000","last_active":"2024-05-10T11:15:01.425779+0000","last_peered":"2024-05-10T11:15:01.425779+0000","last_clean":"2024-05-10T11:15:01.425779+0000","last_became_active":"2024-05-10T11:14:25.278196+0000","last_became_peered":"2024-05-10T11:14:25.278196+0000","last_unstale":"2024-05-10T11:15:01.425779+0000","last_undegraded":"2024-05-10T11:15:01.425779+0000","last_fullsized":"2024-05-10T11:15:01.425779+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:50:29.390014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459617+0000","last_change":"2024-05-10T11:14:25.274741+0000","last_active":"2024-05-10T11:15:01.459617+0000","last_peered":"2024-05-10T11:15:01.459617+0000","last_clean":"2024-05-10T11:15:01.459617+0000","last_became_active":"2024-05-10T11:14:25.274251+0000","last_became_peered":"2024-05-10T11:14:25.274251+0000","last_unstale":"2024-05-10T11:15:01.459617+0000","last_undegraded":"2024-05-10T11:15:01.459617+0000","last_fullsized":"2024-05-10T11:15:01.459617+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:04:41.552042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.429957+0000","last_change":"2024-05-10T11:14:25.265232+0000","last_active":"2024-05-10T11:15:01.429957+0000","last_peered":"2024-05-10T11:15:01.429957+0000","last_clean":"2024-05-10T11:15:01.429957+0000","last_became_active":"2024-05-10T11:14:25.264852+0000","last_became_peered":"2024-05-10T11:14:25.264852+0000","last_unstale":"2024-05-10T11:15:01.429957+0000","last_undegraded":"2024-05-10T11:15:01.429957+0000","last_fullsized":"2024-05-10T11:15:01.429957+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:50:27.207258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.421885+0000","last_change":"2024-05-10T11:14:25.272626+0000","last_active":"2024-05-10T11:15:01.421885+0000","last_peered":"2024-05-10T11:15:01.421885+0000","last_clean":"2024-05-10T11:15:01.421885+0000","last_became_active":"2024-05-10T11:14:25.272305+0000","last_became_peered":"2024-05-10T11:14:25.272305+0000","last_unstale":"2024-05-10T11:15:01.421885+0000","last_undegraded":"2024-05-10T11:15:01.421885+0000","last_fullsized":"2024-05-10T11:15:01.421885+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:44:16.725453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.426391+0000","last_change":"2024-05-10T11:14:25.275460+0000","last_active":"2024-05-10T11:15:01.426391+0000","last_peered":"2024-05-10T11:15:01.426391+0000","last_clean":"2024-05-10T11:15:01.426391+0000","last_became_active":"2024-05-10T11:14:25.274571+0000","last_became_peered":"2024-05-10T11:14:25.274571+0000","last_unstale":"2024-05-10T11:15:01.426391+0000","last_undegraded":"2024-05-10T11:15:01.426391+0000","last_fullsized":"2024-05-10T11:15:01.426391+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:33:20.335738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.458437+0000","last_change":"2024-05-10T11:14:25.273975+0000","last_active":"2024-05-10T11:15:01.458437+0000","last_peered":"2024-05-10T11:15:01.458437+0000","last_clean":"2024-05-10T11:15:01.458437+0000","last_became_active":"2024-05-10T11:14:25.273652+0000","last_became_peered":"2024-05-10T11:14:25.273652+0000","last_unstale":"2024-05-10T11:15:01.458437+0000","last_undegraded":"2024-05-10T11:15:01.458437+0000","last_fullsized":"2024-05-10T11:15:01.458437+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:37:55.354449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.427092+0000","last_change":"2024-05-10T11:14:25.285530+0000","last_active":"2024-05-10T11:15:01.427092+0000","last_peered":"2024-05-10T11:15:01.427092+0000","last_clean":"2024-05-10T11:15:01.427092+0000","last_became_active":"2024-05-10T11:14:25.285235+0000","last_became_peered":"2024-05-10T11:14:25.285235+0000","last_unstale":"2024-05-10T11:15:01.427092+0000","last_undegraded":"2024-05-10T11:15:01.427092+0000","last_fullsized":"2024-05-10T11:15:01.427092+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:11:14.225339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.422343+0000","last_change":"2024-05-10T11:14:25.274738+0000","last_active":"2024-05-10T11:15:01.422343+0000","last_peered":"2024-05-10T11:15:01.422343+0000","last_clean":"2024-05-10T11:15:01.422343+0000","last_became_active":"2024-05-10T11:14:25.274107+0000","last_became_peered":"2024-05-10T11:14:25.274107+0000","last_unstale":"2024-05-10T11:15:01.422343+0000","last_undegraded":"2024-05-10T11:15:01.422343+0000","last_fullsized":"2024-05-10T11:15:01.422343+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:02:01.489930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.427600+0000","last_change":"2024-05-10T11:14:25.277431+0000","last_active":"2024-05-10T11:15:01.427600+0000","last_peered":"2024-05-10T11:15:01.427600+0000","last_clean":"2024-05-10T11:15:01.427600+0000","last_became_active":"2024-05-10T11:14:25.277130+0000","last_became_peered":"2024-05-10T11:14:25.277130+0000","last_unstale":"2024-05-10T11:15:01.427600+0000","last_undegraded":"2024-05-10T11:15:01.427600+0000","last_fullsized":"2024-05-10T11:15:01.427600+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:38:17.192282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459572+0000","last_change":"2024-05-10T11:14:25.276291+0000","last_active":"2024-05-10T11:15:01.459572+0000","last_peered":"2024-05-10T11:15:01.459572+0000","last_clean":"2024-05-10T11:15:01.459572+0000","last_became_active":"2024-05-10T11:14:25.275842+0000","last_became_peered":"2024-05-10T11:14:25.275842+0000","last_unstale":"2024-05-10T11:15:01.459572+0000","last_undegraded":"2024-05-10T11:15:01.459572+0000","last_fullsized":"2024-05-10T11:15:01.459572+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:20:53.443135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459970+0000","last_change":"2024-05-10T11:14:25.273950+0000","last_active":"2024-05-10T11:15:01.459970+0000","last_peered":"2024-05-10T11:15:01.459970+0000","last_clean":"2024-05-10T11:15:01.459970+0000","last_became_active":"2024-05-10T11:14:25.273644+0000","last_became_peered":"2024-05-10T11:14:25.273644+0000","last_unstale":"2024-05-10T11:15:01.459970+0000","last_undegraded":"2024-05-10T11:15:01.459970+0000","last_fullsized":"2024-05-10T11:15:01.459970+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:46:05.112575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.425711+0000","last_change":"2024-05-10T11:14:25.277930+0000","last_active":"2024-05-10T11:15:01.425711+0000","last_peered":"2024-05-10T11:15:01.425711+0000","last_clean":"2024-05-10T11:15:01.425711+0000","last_became_active":"2024-05-10T11:14:25.276793+0000","last_became_peered":"2024-05-10T11:14:25.276793+0000","last_unstale":"2024-05-10T11:15:01.425711+0000","last_undegraded":"2024-05-10T11:15:01.425711+0000","last_fullsized":"2024-05-10T11:15:01.425711+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:47:03.592882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.425713+0000","last_change":"2024-05-10T11:14:25.270434+0000","last_active":"2024-05-10T11:15:01.425713+0000","last_peered":"2024-05-10T11:15:01.425713+0000","last_clean":"2024-05-10T11:15:01.425713+0000","last_became_active":"2024-05-10T11:14:25.270139+0000","last_became_peered":"2024-05-10T11:14:25.270139+0000","last_unstale":"2024-05-10T11:15:01.425713+0000","last_undegraded":"2024-05-10T11:15:01.425713+0000","last_fullsized":"2024-05-10T11:15:01.425713+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:58:40.506560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.425989+0000","last_change":"2024-05-10T11:14:25.272981+0000","last_active":"2024-05-10T11:15:01.425989+0000","last_peered":"2024-05-10T11:15:01.425989+0000","last_clean":"2024-05-10T11:15:01.425989+0000","last_became_active":"2024-05-10T11:14:25.272592+0000","last_became_peered":"2024-05-10T11:14:25.272592+0000","last_unstale":"2024-05-10T11:15:01.425989+0000","last_undegraded":"2024-05-10T11:15:01.425989+0000","last_fullsized":"2024-05-10T11:15:01.425989+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:46:09.673013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.425631+0000","last_change":"2024-05-10T11:14:25.287304+0000","last_active":"2024-05-10T11:15:01.425631+0000","last_peered":"2024-05-10T11:15:01.425631+0000","last_clean":"2024-05-10T11:15:01.425631+0000","last_became_active":"2024-05-10T11:14:25.286964+0000","last_became_peered":"2024-05-10T11:14:25.286964+0000","last_unstale":"2024-05-10T11:15:01.425631+0000","last_undegraded":"2024-05-10T11:15:01.425631+0000","last_fullsized":"2024-05-10T11:15:01.425631+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:32:27.006840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.422582+0000","last_change":"2024-05-10T11:14:25.275366+0000","last_active":"2024-05-10T11:15:01.422582+0000","last_peered":"2024-05-10T11:15:01.422582+0000","last_clean":"2024-05-10T11:15:01.422582+0000","last_became_active":"2024-05-10T11:14:25.274986+0000","last_became_peered":"2024-05-10T11:14:25.274986+0000","last_unstale":"2024-05-10T11:15:01.422582+0000","last_undegraded":"2024-05-10T11:15:01.422582+0000","last_fullsized":"2024-05-10T11:15:01.422582+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:50:50.071975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.426182+0000","last_change":"2024-05-10T11:14:25.274588+0000","last_active":"2024-05-10T11:15:01.426182+0000","last_peered":"2024-05-10T11:15:01.426182+0000","last_clean":"2024-05-10T11:15:01.426182+0000","last_became_active":"2024-05-10T11:14:25.274179+0000","last_became_peered":"2024-05-10T11:14:25.274179+0000","last_unstale":"2024-05-10T11:15:01.426182+0000","last_undegraded":"2024-05-10T11:15:01.426182+0000","last_fullsized":"2024-05-10T11:15:01.426182+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:06:23.517082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.461337+0000","last_change":"2024-05-10T11:15:01.461337+0000","last_active":"2024-05-10T11:15:01.461337+0000","last_peered":"2024-05-10T11:15:01.461337+0000","last_clean":"2024-05-10T11:15:01.461337+0000","last_became_active":"2024-05-10T11:15:01.460776+0000","last_became_peered":"2024-05-10T11:15:01.460776+0000","last_unstale":"2024-05-10T11:15:01.461337+0000","last_undegraded":"2024-05-10T11:15:01.461337+0000","last_fullsized":"2024-05-10T11:15:01.461337+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:06:49.156483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,10,0],"acting":[2,3,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:29.797616+0000","last_change":"2024-05-10T11:14:22.253909+0000","last_active":"2024-05-10T11:15:29.797616+0000","last_peered":"2024-05-10T11:15:29.797616+0000","last_clean":"2024-05-10T11:15:29.797616+0000","last_became_active":"2024-05-10T11:14:22.253400+0000","last_became_peered":"2024-05-10T11:14:22.253400+0000","last_unstale":"2024-05-10T11:15:29.797616+0000","last_undegraded":"2024-05-10T11:15:29.797616+0000","last_fullsized":"2024-05-10T11:15:29.797616+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:06:46.173511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.458680+0000","last_change":"2024-05-10T11:14:25.273675+0000","last_active":"2024-05-10T11:15:01.458680+0000","last_peered":"2024-05-10T11:15:01.458680+0000","last_clean":"2024-05-10T11:15:01.458680+0000","last_became_active":"2024-05-10T11:14:25.273320+0000","last_became_peered":"2024-05-10T11:14:25.273320+0000","last_unstale":"2024-05-10T11:15:01.458680+0000","last_undegraded":"2024-05-10T11:15:01.458680+0000","last_fullsized":"2024-05-10T11:15:01.458680+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:09:18.232236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.458938+0000","last_change":"2024-05-10T11:14:35.326204+0000","last_active":"2024-05-10T11:15:01.458938+0000","last_peered":"2024-05-10T11:15:01.458938+0000","last_clean":"2024-05-10T11:15:01.458938+0000","last_became_active":"2024-05-10T11:14:35.325715+0000","last_became_peered":"2024-05-10T11:14:35.325715+0000","last_unstale":"2024-05-10T11:15:01.458938+0000","last_undegraded":"2024-05-10T11:15:01.458938+0000","last_fullsized":"2024-05-10T11:15:01.458938+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:06:30.665861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.458982+0000","last_change":"2024-05-10T11:14:22.299194+0000","last_active":"2024-05-10T11:15:01.458982+0000","last_peered":"2024-05-10T11:15:01.458982+0000","last_clean":"2024-05-10T11:15:01.458982+0000","last_became_active":"2024-05-10T11:14:22.298807+0000","last_became_peered":"2024-05-10T11:14:22.298807+0000","last_unstale":"2024-05-10T11:15:01.458982+0000","last_undegraded":"2024-05-10T11:15:01.458982+0000","last_fullsized":"2024-05-10T11:15:01.458982+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:24:51.054834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.425865+0000","last_change":"2024-05-10T11:14:25.277860+0000","last_active":"2024-05-10T11:15:01.425865+0000","last_peered":"2024-05-10T11:15:01.425865+0000","last_clean":"2024-05-10T11:15:01.425865+0000","last_became_active":"2024-05-10T11:14:25.276784+0000","last_became_peered":"2024-05-10T11:14:25.276784+0000","last_unstale":"2024-05-10T11:15:01.425865+0000","last_undegraded":"2024-05-10T11:15:01.425865+0000","last_fullsized":"2024-05-10T11:15:01.425865+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:42:57.673595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459231+0000","last_change":"2024-05-10T11:14:35.330679+0000","last_active":"2024-05-10T11:15:01.459231+0000","last_peered":"2024-05-10T11:15:01.459231+0000","last_clean":"2024-05-10T11:15:01.459231+0000","last_became_active":"2024-05-10T11:14:35.329819+0000","last_became_peered":"2024-05-10T11:14:35.329819+0000","last_unstale":"2024-05-10T11:15:01.459231+0000","last_undegraded":"2024-05-10T11:15:01.459231+0000","last_fullsized":"2024-05-10T11:15:01.459231+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:31:26.982784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:29.798596+0000","last_change":"2024-05-10T11:14:22.260402+0000","last_active":"2024-05-10T11:15:29.798596+0000","last_peered":"2024-05-10T11:15:29.798596+0000","last_clean":"2024-05-10T11:15:29.798596+0000","last_became_active":"2024-05-10T11:14:22.259619+0000","last_became_peered":"2024-05-10T11:14:22.259619+0000","last_unstale":"2024-05-10T11:15:29.798596+0000","last_undegraded":"2024-05-10T11:15:29.798596+0000","last_fullsized":"2024-05-10T11:15:29.798596+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:56:34.491626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.426783+0000","last_change":"2024-05-10T11:14:25.276117+0000","last_active":"2024-05-10T11:15:01.426783+0000","last_peered":"2024-05-10T11:15:01.426783+0000","last_clean":"2024-05-10T11:15:01.426783+0000","last_became_active":"2024-05-10T11:14:25.275758+0000","last_became_peered":"2024-05-10T11:14:25.275758+0000","last_unstale":"2024-05-10T11:15:01.426783+0000","last_undegraded":"2024-05-10T11:15:01.426783+0000","last_fullsized":"2024-05-10T11:15:01.426783+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:12:16.582244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459124+0000","last_change":"2024-05-10T11:14:35.324633+0000","last_active":"2024-05-10T11:15:01.459124+0000","last_peered":"2024-05-10T11:15:01.459124+0000","last_clean":"2024-05-10T11:15:01.459124+0000","last_became_active":"2024-05-10T11:14:35.324152+0000","last_became_peered":"2024-05-10T11:14:35.324152+0000","last_unstale":"2024-05-10T11:15:01.459124+0000","last_undegraded":"2024-05-10T11:15:01.459124+0000","last_fullsized":"2024-05-10T11:15:01.459124+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:56:50.008782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.430295+0000","last_change":"2024-05-10T11:14:22.259086+0000","last_active":"2024-05-10T11:15:01.430295+0000","last_peered":"2024-05-10T11:15:01.430295+0000","last_clean":"2024-05-10T11:15:01.430295+0000","last_became_active":"2024-05-10T11:14:22.258628+0000","last_became_peered":"2024-05-10T11:14:22.258628+0000","last_unstale":"2024-05-10T11:15:01.430295+0000","last_undegraded":"2024-05-10T11:15:01.430295+0000","last_fullsized":"2024-05-10T11:15:01.430295+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:39:13.472622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459840+0000","last_change":"2024-05-10T11:14:25.277556+0000","last_active":"2024-05-10T11:15:01.459840+0000","last_peered":"2024-05-10T11:15:01.459840+0000","last_clean":"2024-05-10T11:15:01.459840+0000","last_became_active":"2024-05-10T11:14:25.277201+0000","last_became_peered":"2024-05-10T11:14:25.277201+0000","last_unstale":"2024-05-10T11:15:01.459840+0000","last_undegraded":"2024-05-10T11:15:01.459840+0000","last_fullsized":"2024-05-10T11:15:01.459840+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:04:17.028305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.426304+0000","last_change":"2024-05-10T11:14:35.323338+0000","last_active":"2024-05-10T11:15:01.426304+0000","last_peered":"2024-05-10T11:15:01.426304+0000","last_clean":"2024-05-10T11:15:01.426304+0000","last_became_active":"2024-05-10T11:14:35.321948+0000","last_became_peered":"2024-05-10T11:14:35.321948+0000","last_unstale":"2024-05-10T11:15:01.426304+0000","last_undegraded":"2024-05-10T11:15:01.426304+0000","last_fullsized":"2024-05-10T11:15:01.426304+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:41:30.569125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":71,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:53.842867+0000","last_change":"2024-05-10T11:14:22.259187+0000","last_active":"2024-05-10T11:15:53.842867+0000","last_peered":"2024-05-10T11:15:53.842867+0000","last_clean":"2024-05-10T11:15:53.842867+0000","last_became_active":"2024-05-10T11:14:22.258812+0000","last_became_peered":"2024-05-10T11:14:22.258812+0000","last_unstale":"2024-05-10T11:15:53.842867+0000","last_undegraded":"2024-05-10T11:15:53.842867+0000","last_fullsized":"2024-05-10T11:15:53.842867+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:15:38.769705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":26,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.429212+0000","last_change":"2024-05-10T11:14:25.262469+0000","last_active":"2024-05-10T11:15:01.429212+0000","last_peered":"2024-05-10T11:15:01.429212+0000","last_clean":"2024-05-10T11:15:01.429212+0000","last_became_active":"2024-05-10T11:14:25.262104+0000","last_became_peered":"2024-05-10T11:14:25.262104+0000","last_unstale":"2024-05-10T11:15:01.429212+0000","last_undegraded":"2024-05-10T11:15:01.429212+0000","last_fullsized":"2024-05-10T11:15:01.429212+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:48:24.329136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.426854+0000","last_change":"2024-05-10T11:14:35.327737+0000","last_active":"2024-05-10T11:15:01.426854+0000","last_peered":"2024-05-10T11:15:01.426854+0000","last_clean":"2024-05-10T11:15:01.426854+0000","last_became_active":"2024-05-10T11:14:35.327209+0000","last_became_peered":"2024-05-10T11:14:35.327209+0000","last_unstale":"2024-05-10T11:15:01.426854+0000","last_undegraded":"2024-05-10T11:15:01.426854+0000","last_fullsized":"2024-05-10T11:15:01.426854+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:32:11.784752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:29.796781+0000","last_change":"2024-05-10T11:14:22.254853+0000","last_active":"2024-05-10T11:15:29.796781+0000","last_peered":"2024-05-10T11:15:29.796781+0000","last_clean":"2024-05-10T11:15:29.796781+0000","last_became_active":"2024-05-10T11:14:22.253352+0000","last_became_peered":"2024-05-10T11:14:22.253352+0000","last_unstale":"2024-05-10T11:15:29.796781+0000","last_undegraded":"2024-05-10T11:15:29.796781+0000","last_fullsized":"2024-05-10T11:15:29.796781+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:14:15.871042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459935+0000","last_change":"2024-05-10T11:14:25.266798+0000","last_active":"2024-05-10T11:15:01.459935+0000","last_peered":"2024-05-10T11:15:01.459935+0000","last_clean":"2024-05-10T11:15:01.459935+0000","last_became_active":"2024-05-10T11:14:25.266384+0000","last_became_peered":"2024-05-10T11:14:25.266384+0000","last_unstale":"2024-05-10T11:15:01.459935+0000","last_undegraded":"2024-05-10T11:15:01.459935+0000","last_fullsized":"2024-05-10T11:15:01.459935+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:18:07.338987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.462376+0000","last_change":"2024-05-10T11:15:01.462376+0000","last_active":"2024-05-10T11:15:01.462376+0000","last_peered":"2024-05-10T11:15:01.462376+0000","last_clean":"2024-05-10T11:15:01.462376+0000","last_became_active":"2024-05-10T11:15:01.461876+0000","last_became_peered":"2024-05-10T11:15:01.461876+0000","last_unstale":"2024-05-10T11:15:01.462376+0000","last_undegraded":"2024-05-10T11:15:01.462376+0000","last_fullsized":"2024-05-10T11:15:01.462376+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:35:52.076414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:29.796950+0000","last_change":"2024-05-10T11:14:22.253976+0000","last_active":"2024-05-10T11:15:29.796950+0000","last_peered":"2024-05-10T11:15:29.796950+0000","last_clean":"2024-05-10T11:15:29.796950+0000","last_became_active":"2024-05-10T11:14:22.252230+0000","last_became_peered":"2024-05-10T11:14:22.252230+0000","last_unstale":"2024-05-10T11:15:29.796950+0000","last_undegraded":"2024-05-10T11:15:29.796950+0000","last_fullsized":"2024-05-10T11:15:29.796950+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:14:03.208811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.458344+0000","last_change":"2024-05-10T11:14:25.279627+0000","last_active":"2024-05-10T11:15:01.458344+0000","last_peered":"2024-05-10T11:15:01.458344+0000","last_clean":"2024-05-10T11:15:01.458344+0000","last_became_active":"2024-05-10T11:14:25.279330+0000","last_became_peered":"2024-05-10T11:14:25.279330+0000","last_unstale":"2024-05-10T11:15:01.458344+0000","last_undegraded":"2024-05-10T11:15:01.458344+0000","last_fullsized":"2024-05-10T11:15:01.458344+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:50:19.608053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.426021+0000","last_change":"2024-05-10T11:14:35.321723+0000","last_active":"2024-05-10T11:15:01.426021+0000","last_peered":"2024-05-10T11:15:01.426021+0000","last_clean":"2024-05-10T11:15:01.426021+0000","last_became_active":"2024-05-10T11:14:35.321384+0000","last_became_peered":"2024-05-10T11:14:35.321384+0000","last_unstale":"2024-05-10T11:15:01.426021+0000","last_undegraded":"2024-05-10T11:15:01.426021+0000","last_fullsized":"2024-05-10T11:15:01.426021+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:04:07.770889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.460917+0000","last_change":"2024-05-10T11:14:22.255511+0000","last_active":"2024-05-10T11:15:01.460917+0000","last_peered":"2024-05-10T11:15:01.460917+0000","last_clean":"2024-05-10T11:15:01.460917+0000","last_became_active":"2024-05-10T11:14:22.254722+0000","last_became_peered":"2024-05-10T11:14:22.254722+0000","last_unstale":"2024-05-10T11:15:01.460917+0000","last_undegraded":"2024-05-10T11:15:01.460917+0000","last_fullsized":"2024-05-10T11:15:01.460917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:38:27.096011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.422230+0000","last_change":"2024-05-10T11:14:25.274009+0000","last_active":"2024-05-10T11:15:01.422230+0000","last_peered":"2024-05-10T11:15:01.422230+0000","last_clean":"2024-05-10T11:15:01.422230+0000","last_became_active":"2024-05-10T11:14:25.273672+0000","last_became_peered":"2024-05-10T11:14:25.273672+0000","last_unstale":"2024-05-10T11:15:01.422230+0000","last_undegraded":"2024-05-10T11:15:01.422230+0000","last_fullsized":"2024-05-10T11:15:01.422230+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:42:57.210131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.458974+0000","last_change":"2024-05-10T11:14:35.321520+0000","last_active":"2024-05-10T11:15:01.458974+0000","last_peered":"2024-05-10T11:15:01.458974+0000","last_clean":"2024-05-10T11:15:01.458974+0000","last_became_active":"2024-05-10T11:14:35.320575+0000","last_became_peered":"2024-05-10T11:14:35.320575+0000","last_unstale":"2024-05-10T11:15:01.458974+0000","last_undegraded":"2024-05-10T11:15:01.458974+0000","last_fullsized":"2024-05-10T11:15:01.458974+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:54:03.183426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":66,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:54.843515+0000","last_change":"2024-05-10T11:14:22.251463+0000","last_active":"2024-05-10T11:15:54.843515+0000","last_peered":"2024-05-10T11:15:54.843515+0000","last_clean":"2024-05-10T11:15:54.843515+0000","last_became_active":"2024-05-10T11:14:22.251132+0000","last_became_peered":"2024-05-10T11:14:22.251132+0000","last_unstale":"2024-05-10T11:15:54.843515+0000","last_undegraded":"2024-05-10T11:15:54.843515+0000","last_fullsized":"2024-05-10T11:15:54.843515+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:29:18.125380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.426728+0000","last_change":"2024-05-10T11:14:25.276333+0000","last_active":"2024-05-10T11:15:01.426728+0000","last_peered":"2024-05-10T11:15:01.426728+0000","last_clean":"2024-05-10T11:15:01.426728+0000","last_became_active":"2024-05-10T11:14:25.275818+0000","last_became_peered":"2024-05-10T11:14:25.275818+0000","last_unstale":"2024-05-10T11:15:01.426728+0000","last_undegraded":"2024-05-10T11:15:01.426728+0000","last_fullsized":"2024-05-10T11:15:01.426728+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:27:19.542645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.422080+0000","last_change":"2024-05-10T11:14:35.328589+0000","last_active":"2024-05-10T11:15:01.422080+0000","last_peered":"2024-05-10T11:15:01.422080+0000","last_clean":"2024-05-10T11:15:01.422080+0000","last_became_active":"2024-05-10T11:14:35.328052+0000","last_became_peered":"2024-05-10T11:14:35.328052+0000","last_unstale":"2024-05-10T11:15:01.422080+0000","last_undegraded":"2024-05-10T11:15:01.422080+0000","last_fullsized":"2024-05-10T11:15:01.422080+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:09:42.763932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:28.789882+0000","last_change":"2024-05-10T11:14:22.257872+0000","last_active":"2024-05-10T11:15:28.789882+0000","last_peered":"2024-05-10T11:15:28.789882+0000","last_clean":"2024-05-10T11:15:28.789882+0000","last_became_active":"2024-05-10T11:14:22.257530+0000","last_became_peered":"2024-05-10T11:14:22.257530+0000","last_unstale":"2024-05-10T11:15:28.789882+0000","last_undegraded":"2024-05-10T11:15:28.789882+0000","last_fullsized":"2024-05-10T11:15:28.789882+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:26:30.367209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.425956+0000","last_change":"2024-05-10T11:14:25.269436+0000","last_active":"2024-05-10T11:15:01.425956+0000","last_peered":"2024-05-10T11:15:01.425956+0000","last_clean":"2024-05-10T11:15:01.425956+0000","last_became_active":"2024-05-10T11:14:25.269102+0000","last_became_peered":"2024-05-10T11:14:25.269102+0000","last_unstale":"2024-05-10T11:15:01.425956+0000","last_undegraded":"2024-05-10T11:15:01.425956+0000","last_fullsized":"2024-05-10T11:15:01.425956+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:51:50.172464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459463+0000","last_change":"2024-05-10T11:14:35.330272+0000","last_active":"2024-05-10T11:15:01.459463+0000","last_peered":"2024-05-10T11:15:01.459463+0000","last_clean":"2024-05-10T11:15:01.459463+0000","last_became_active":"2024-05-10T11:14:35.329438+0000","last_became_peered":"2024-05-10T11:14:35.329438+0000","last_unstale":"2024-05-10T11:15:01.459463+0000","last_undegraded":"2024-05-10T11:15:01.459463+0000","last_fullsized":"2024-05-10T11:15:01.459463+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:07:00.559447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:28.788966+0000","last_change":"2024-05-10T11:14:22.299302+0000","last_active":"2024-05-10T11:15:28.788966+0000","last_peered":"2024-05-10T11:15:28.788966+0000","last_clean":"2024-05-10T11:15:28.788966+0000","last_became_active":"2024-05-10T11:14:22.298792+0000","last_became_peered":"2024-05-10T11:14:22.298792+0000","last_unstale":"2024-05-10T11:15:28.788966+0000","last_undegraded":"2024-05-10T11:15:28.788966+0000","last_fullsized":"2024-05-10T11:15:28.788966+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:12:35.280135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.426961+0000","last_change":"2024-05-10T11:14:25.275380+0000","last_active":"2024-05-10T11:15:01.426961+0000","last_peered":"2024-05-10T11:15:01.426961+0000","last_clean":"2024-05-10T11:15:01.426961+0000","last_became_active":"2024-05-10T11:14:25.275042+0000","last_became_peered":"2024-05-10T11:14:25.275042+0000","last_unstale":"2024-05-10T11:15:01.426961+0000","last_undegraded":"2024-05-10T11:15:01.426961+0000","last_fullsized":"2024-05-10T11:15:01.426961+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:22:49.598602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.422328+0000","last_change":"2024-05-10T11:14:35.327505+0000","last_active":"2024-05-10T11:15:01.422328+0000","last_peered":"2024-05-10T11:15:01.422328+0000","last_clean":"2024-05-10T11:15:01.422328+0000","last_became_active":"2024-05-10T11:14:35.326885+0000","last_became_peered":"2024-05-10T11:14:35.326885+0000","last_unstale":"2024-05-10T11:15:01.422328+0000","last_undegraded":"2024-05-10T11:15:01.422328+0000","last_fullsized":"2024-05-10T11:15:01.422328+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:23:09.010545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'32","reported_seq":127,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:55.847752+0000","last_change":"2024-05-10T11:14:22.257571+0000","last_active":"2024-05-10T11:15:55.847752+0000","last_peered":"2024-05-10T11:15:55.847752+0000","last_clean":"2024-05-10T11:15:55.847752+0000","last_became_active":"2024-05-10T11:14:22.257156+0000","last_became_peered":"2024-05-10T11:14:22.257156+0000","last_unstale":"2024-05-10T11:15:55.847752+0000","last_undegraded":"2024-05-10T11:15:55.847752+0000","last_fullsized":"2024-05-10T11:15:55.847752+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:28:06.397262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57883,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":56,"num_read_kb":84,"num_write":33,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.430478+0000","last_change":"2024-05-10T11:14:25.281160+0000","last_active":"2024-05-10T11:15:01.430478+0000","last_peered":"2024-05-10T11:15:01.430478+0000","last_clean":"2024-05-10T11:15:01.430478+0000","last_became_active":"2024-05-10T11:14:25.280763+0000","last_became_peered":"2024-05-10T11:14:25.280763+0000","last_unstale":"2024-05-10T11:15:01.430478+0000","last_undegraded":"2024-05-10T11:15:01.430478+0000","last_fullsized":"2024-05-10T11:15:01.430478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:19:53.350792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.460232+0000","last_change":"2024-05-10T11:14:35.320918+0000","last_active":"2024-05-10T11:15:01.460232+0000","last_peered":"2024-05-10T11:15:01.460232+0000","last_clean":"2024-05-10T11:15:01.460232+0000","last_became_active":"2024-05-10T11:14:35.320412+0000","last_became_peered":"2024-05-10T11:14:35.320412+0000","last_unstale":"2024-05-10T11:15:01.460232+0000","last_undegraded":"2024-05-10T11:15:01.460232+0000","last_fullsized":"2024-05-10T11:15:01.460232+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:55:52.585430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:29.798998+0000","last_change":"2024-05-10T11:14:22.257771+0000","last_active":"2024-05-10T11:15:29.798998+0000","last_peered":"2024-05-10T11:15:29.798998+0000","last_clean":"2024-05-10T11:15:29.798998+0000","last_became_active":"2024-05-10T11:14:22.257430+0000","last_became_peered":"2024-05-10T11:14:22.257430+0000","last_unstale":"2024-05-10T11:15:29.798998+0000","last_undegraded":"2024-05-10T11:15:29.798998+0000","last_fullsized":"2024-05-10T11:15:29.798998+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:23:50.131136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459594+0000","last_change":"2024-05-10T11:14:25.272276+0000","last_active":"2024-05-10T11:15:01.459594+0000","last_peered":"2024-05-10T11:15:01.459594+0000","last_clean":"2024-05-10T11:15:01.459594+0000","last_became_active":"2024-05-10T11:14:25.271932+0000","last_became_peered":"2024-05-10T11:14:25.271932+0000","last_unstale":"2024-05-10T11:15:01.459594+0000","last_undegraded":"2024-05-10T11:15:01.459594+0000","last_fullsized":"2024-05-10T11:15:01.459594+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:49:44.479056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.462435+0000","last_change":"2024-05-10T11:15:01.462435+0000","last_active":"2024-05-10T11:15:01.462435+0000","last_peered":"2024-05-10T11:15:01.462435+0000","last_clean":"2024-05-10T11:15:01.462435+0000","last_became_active":"2024-05-10T11:15:01.461890+0000","last_became_peered":"2024-05-10T11:15:01.461890+0000","last_unstale":"2024-05-10T11:15:01.462435+0000","last_undegraded":"2024-05-10T11:15:01.462435+0000","last_fullsized":"2024-05-10T11:15:01.462435+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:01:13.350655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:28.787852+0000","last_change":"2024-05-10T11:14:22.256713+0000","last_active":"2024-05-10T11:15:28.787852+0000","last_peered":"2024-05-10T11:15:28.787852+0000","last_clean":"2024-05-10T11:15:28.787852+0000","last_became_active":"2024-05-10T11:14:22.255847+0000","last_became_peered":"2024-05-10T11:14:22.255847+0000","last_unstale":"2024-05-10T11:15:28.787852+0000","last_undegraded":"2024-05-10T11:15:28.787852+0000","last_fullsized":"2024-05-10T11:15:28.787852+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:00:54.605506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.426720+0000","last_change":"2024-05-10T11:14:25.282118+0000","last_active":"2024-05-10T11:15:01.426720+0000","last_peered":"2024-05-10T11:15:01.426720+0000","last_clean":"2024-05-10T11:15:01.426720+0000","last_became_active":"2024-05-10T11:14:25.280958+0000","last_became_peered":"2024-05-10T11:14:25.280958+0000","last_unstale":"2024-05-10T11:15:01.426720+0000","last_undegraded":"2024-05-10T11:15:01.426720+0000","last_fullsized":"2024-05-10T11:15:01.426720+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:02:10.815733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.422251+0000","last_change":"2024-05-10T11:14:35.329876+0000","last_active":"2024-05-10T11:15:01.422251+0000","last_peered":"2024-05-10T11:15:01.422251+0000","last_clean":"2024-05-10T11:15:01.422251+0000","last_became_active":"2024-05-10T11:14:35.328923+0000","last_became_peered":"2024-05-10T11:14:35.328923+0000","last_unstale":"2024-05-10T11:15:01.422251+0000","last_undegraded":"2024-05-10T11:15:01.422251+0000","last_fullsized":"2024-05-10T11:15:01.422251+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:04:51.356552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'4","reported_seq":68,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:53.842697+0000","last_change":"2024-05-10T11:14:22.254515+0000","last_active":"2024-05-10T11:15:53.842697+0000","last_peered":"2024-05-10T11:15:53.842697+0000","last_clean":"2024-05-10T11:15:53.842697+0000","last_became_active":"2024-05-10T11:14:22.253648+0000","last_became_peered":"2024-05-10T11:14:22.253648+0000","last_unstale":"2024-05-10T11:15:53.842697+0000","last_undegraded":"2024-05-10T11:15:53.842697+0000","last_fullsized":"2024-05-10T11:15:53.842697+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:10:22.144446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":27,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.426521+0000","last_change":"2024-05-10T11:14:25.278841+0000","last_active":"2024-05-10T11:15:01.426521+0000","last_peered":"2024-05-10T11:15:01.426521+0000","last_clean":"2024-05-10T11:15:01.426521+0000","last_became_active":"2024-05-10T11:14:25.278527+0000","last_became_peered":"2024-05-10T11:14:25.278527+0000","last_unstale":"2024-05-10T11:15:01.426521+0000","last_undegraded":"2024-05-10T11:15:01.426521+0000","last_fullsized":"2024-05-10T11:15:01.426521+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:01:55.902587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459717+0000","last_change":"2024-05-10T11:14:35.331334+0000","last_active":"2024-05-10T11:15:01.459717+0000","last_peered":"2024-05-10T11:15:01.459717+0000","last_clean":"2024-05-10T11:15:01.459717+0000","last_became_active":"2024-05-10T11:14:35.330940+0000","last_became_peered":"2024-05-10T11:14:35.330940+0000","last_unstale":"2024-05-10T11:15:01.459717+0000","last_undegraded":"2024-05-10T11:15:01.459717+0000","last_fullsized":"2024-05-10T11:15:01.459717+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:11:12.241551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:29.799044+0000","last_change":"2024-05-10T11:14:22.299917+0000","last_active":"2024-05-10T11:15:29.799044+0000","last_peered":"2024-05-10T11:15:29.799044+0000","last_clean":"2024-05-10T11:15:29.799044+0000","last_became_active":"2024-05-10T11:14:22.299212+0000","last_became_peered":"2024-05-10T11:14:22.299212+0000","last_unstale":"2024-05-10T11:15:29.799044+0000","last_undegraded":"2024-05-10T11:15:29.799044+0000","last_fullsized":"2024-05-10T11:15:29.799044+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:52:48.740339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459876+0000","last_change":"2024-05-10T11:14:25.274693+0000","last_active":"2024-05-10T11:15:01.459876+0000","last_peered":"2024-05-10T11:15:01.459876+0000","last_clean":"2024-05-10T11:15:01.459876+0000","last_became_active":"2024-05-10T11:14:25.274378+0000","last_became_peered":"2024-05-10T11:14:25.274378+0000","last_unstale":"2024-05-10T11:15:01.459876+0000","last_undegraded":"2024-05-10T11:15:01.459876+0000","last_fullsized":"2024-05-10T11:15:01.459876+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:53:24.082742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.461224+0000","last_change":"2024-05-10T11:15:01.460541+0000","last_active":"2024-05-10T11:15:01.461224+0000","last_peered":"2024-05-10T11:15:01.461224+0000","last_clean":"2024-05-10T11:15:01.461224+0000","last_became_active":"2024-05-10T11:15:01.459954+0000","last_became_peered":"2024-05-10T11:15:01.459954+0000","last_unstale":"2024-05-10T11:15:01.461224+0000","last_undegraded":"2024-05-10T11:15:01.461224+0000","last_fullsized":"2024-05-10T11:15:01.461224+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:25:43.642441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,3,1],"acting":[8,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:31.793479+0000","last_change":"2024-05-10T11:14:22.258797+0000","last_active":"2024-05-10T11:15:31.793479+0000","last_peered":"2024-05-10T11:15:31.793479+0000","last_clean":"2024-05-10T11:15:31.793479+0000","last_became_active":"2024-05-10T11:14:22.258365+0000","last_became_peered":"2024-05-10T11:14:22.258365+0000","last_unstale":"2024-05-10T11:15:31.793479+0000","last_undegraded":"2024-05-10T11:15:31.793479+0000","last_fullsized":"2024-05-10T11:15:31.793479+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:58:53.291644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.460502+0000","last_change":"2024-05-10T11:15:01.460502+0000","last_active":"2024-05-10T11:15:01.460502+0000","last_peered":"2024-05-10T11:15:01.460502+0000","last_clean":"2024-05-10T11:15:01.460502+0000","last_became_active":"2024-05-10T11:15:01.460101+0000","last_became_peered":"2024-05-10T11:15:01.460101+0000","last_unstale":"2024-05-10T11:15:01.460502+0000","last_undegraded":"2024-05-10T11:15:01.460502+0000","last_fullsized":"2024-05-10T11:15:01.460502+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:22:39.735251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.429163+0000","last_change":"2024-05-10T11:14:35.419037+0000","last_active":"2024-05-10T11:15:01.429163+0000","last_peered":"2024-05-10T11:15:01.429163+0000","last_clean":"2024-05-10T11:15:01.429163+0000","last_became_active":"2024-05-10T11:14:35.418541+0000","last_became_peered":"2024-05-10T11:14:35.418541+0000","last_unstale":"2024-05-10T11:15:01.429163+0000","last_undegraded":"2024-05-10T11:15:01.429163+0000","last_fullsized":"2024-05-10T11:15:01.429163+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:05:49.560608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.422500+0000","last_change":"2024-05-10T11:14:22.299538+0000","last_active":"2024-05-10T11:15:01.422500+0000","last_peered":"2024-05-10T11:15:01.422500+0000","last_clean":"2024-05-10T11:15:01.422500+0000","last_became_active":"2024-05-10T11:14:22.299192+0000","last_became_peered":"2024-05-10T11:14:22.299192+0000","last_unstale":"2024-05-10T11:15:01.422500+0000","last_undegraded":"2024-05-10T11:15:01.422500+0000","last_fullsized":"2024-05-10T11:15:01.422500+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:28:43.041513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.421964+0000","last_change":"2024-05-10T11:14:25.272238+0000","last_active":"2024-05-10T11:15:01.421964+0000","last_peered":"2024-05-10T11:15:01.421964+0000","last_clean":"2024-05-10T11:15:01.421964+0000","last_became_active":"2024-05-10T11:14:25.271861+0000","last_became_peered":"2024-05-10T11:14:25.271861+0000","last_unstale":"2024-05-10T11:15:01.421964+0000","last_undegraded":"2024-05-10T11:15:01.421964+0000","last_fullsized":"2024-05-10T11:15:01.421964+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:00:08.821316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459561+0000","last_change":"2024-05-10T11:14:35.322239+0000","last_active":"2024-05-10T11:15:01.459561+0000","last_peered":"2024-05-10T11:15:01.459561+0000","last_clean":"2024-05-10T11:15:01.459561+0000","last_became_active":"2024-05-10T11:14:35.321864+0000","last_became_peered":"2024-05-10T11:14:35.321864+0000","last_unstale":"2024-05-10T11:15:01.459561+0000","last_undegraded":"2024-05-10T11:15:01.459561+0000","last_fullsized":"2024-05-10T11:15:01.459561+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:42:06.905879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.427027+0000","last_change":"2024-05-10T11:14:22.255449+0000","last_active":"2024-05-10T11:15:01.427027+0000","last_peered":"2024-05-10T11:15:01.427027+0000","last_clean":"2024-05-10T11:15:01.427027+0000","last_became_active":"2024-05-10T11:14:22.254966+0000","last_became_peered":"2024-05-10T11:14:22.254966+0000","last_unstale":"2024-05-10T11:15:01.427027+0000","last_undegraded":"2024-05-10T11:15:01.427027+0000","last_fullsized":"2024-05-10T11:15:01.427027+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:55:40.746115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.421870+0000","last_change":"2024-05-10T11:14:25.274050+0000","last_active":"2024-05-10T11:15:01.421870+0000","last_peered":"2024-05-10T11:15:01.421870+0000","last_clean":"2024-05-10T11:15:01.421870+0000","last_became_active":"2024-05-10T11:14:25.273720+0000","last_became_peered":"2024-05-10T11:14:25.273720+0000","last_unstale":"2024-05-10T11:15:01.421870+0000","last_undegraded":"2024-05-10T11:15:01.421870+0000","last_fullsized":"2024-05-10T11:15:01.421870+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:22:05.977673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459587+0000","last_change":"2024-05-10T11:14:35.418989+0000","last_active":"2024-05-10T11:15:01.459587+0000","last_peered":"2024-05-10T11:15:01.459587+0000","last_clean":"2024-05-10T11:15:01.459587+0000","last_became_active":"2024-05-10T11:14:35.418546+0000","last_became_peered":"2024-05-10T11:14:35.418546+0000","last_unstale":"2024-05-10T11:15:01.459587+0000","last_undegraded":"2024-05-10T11:15:01.459587+0000","last_fullsized":"2024-05-10T11:15:01.459587+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:23:37.875409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459653+0000","last_change":"2024-05-10T11:14:25.272694+0000","last_active":"2024-05-10T11:15:01.459653+0000","last_peered":"2024-05-10T11:15:01.459653+0000","last_clean":"2024-05-10T11:15:01.459653+0000","last_became_active":"2024-05-10T11:14:25.272303+0000","last_became_peered":"2024-05-10T11:14:25.272303+0000","last_unstale":"2024-05-10T11:15:01.459653+0000","last_undegraded":"2024-05-10T11:15:01.459653+0000","last_fullsized":"2024-05-10T11:15:01.459653+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:51:39.039122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":67,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:55.844613+0000","last_change":"2024-05-10T11:14:22.299359+0000","last_active":"2024-05-10T11:15:55.844613+0000","last_peered":"2024-05-10T11:15:55.844613+0000","last_clean":"2024-05-10T11:15:55.844613+0000","last_became_active":"2024-05-10T11:14:22.299023+0000","last_became_peered":"2024-05-10T11:14:22.299023+0000","last_unstale":"2024-05-10T11:15:55.844613+0000","last_undegraded":"2024-05-10T11:15:55.844613+0000","last_fullsized":"2024-05-10T11:15:55.844613+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:53:06.301774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.427641+0000","last_change":"2024-05-10T11:14:35.322640+0000","last_active":"2024-05-10T11:15:01.427641+0000","last_peered":"2024-05-10T11:15:01.427641+0000","last_clean":"2024-05-10T11:15:01.427641+0000","last_became_active":"2024-05-10T11:14:35.322226+0000","last_became_peered":"2024-05-10T11:14:35.322226+0000","last_unstale":"2024-05-10T11:15:01.427641+0000","last_undegraded":"2024-05-10T11:15:01.427641+0000","last_fullsized":"2024-05-10T11:15:01.427641+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:34:11.574438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.458527+0000","last_change":"2024-05-10T11:14:25.276645+0000","last_active":"2024-05-10T11:15:01.458527+0000","last_peered":"2024-05-10T11:15:01.458527+0000","last_clean":"2024-05-10T11:15:01.458527+0000","last_became_active":"2024-05-10T11:14:25.276295+0000","last_became_peered":"2024-05-10T11:14:25.276295+0000","last_unstale":"2024-05-10T11:15:01.458527+0000","last_undegraded":"2024-05-10T11:15:01.458527+0000","last_fullsized":"2024-05-10T11:15:01.458527+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:52:06.715073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:32.810362+0000","last_change":"2024-05-10T11:14:22.252908+0000","last_active":"2024-05-10T11:15:32.810362+0000","last_peered":"2024-05-10T11:15:32.810362+0000","last_clean":"2024-05-10T11:15:32.810362+0000","last_became_active":"2024-05-10T11:14:22.252532+0000","last_became_peered":"2024-05-10T11:14:22.252532+0000","last_unstale":"2024-05-10T11:15:32.810362+0000","last_undegraded":"2024-05-10T11:15:32.810362+0000","last_fullsized":"2024-05-10T11:15:32.810362+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:41:59.651442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":113,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.426311+0000","last_change":"2024-05-10T11:12:53.090796+0000","last_active":"2024-05-10T11:15:01.426311+0000","last_peered":"2024-05-10T11:15:01.426311+0000","last_clean":"2024-05-10T11:15:01.426311+0000","last_became_active":"2024-05-10T11:12:53.086566+0000","last_became_peered":"2024-05-10T11:12:53.086566+0000","last_unstale":"2024-05-10T11:15:01.426311+0000","last_undegraded":"2024-05-10T11:15:01.426311+0000","last_fullsized":"2024-05-10T11:15:01.426311+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:09:27.097043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:09:27.097043+0000","last_clean_scrub_stamp":"2024-05-10T11:09:27.097043+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:42:17.759719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459876+0000","last_change":"2024-05-10T11:14:35.419970+0000","last_active":"2024-05-10T11:15:01.459876+0000","last_peered":"2024-05-10T11:15:01.459876+0000","last_clean":"2024-05-10T11:15:01.459876+0000","last_became_active":"2024-05-10T11:14:35.419516+0000","last_became_peered":"2024-05-10T11:14:35.419516+0000","last_unstale":"2024-05-10T11:15:01.459876+0000","last_undegraded":"2024-05-10T11:15:01.459876+0000","last_fullsized":"2024-05-10T11:15:01.459876+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:01:25.148155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459243+0000","last_change":"2024-05-10T11:14:25.271478+0000","last_active":"2024-05-10T11:15:01.459243+0000","last_peered":"2024-05-10T11:15:01.459243+0000","last_clean":"2024-05-10T11:15:01.459243+0000","last_became_active":"2024-05-10T11:14:25.271143+0000","last_became_peered":"2024-05-10T11:14:25.271143+0000","last_unstale":"2024-05-10T11:15:01.459243+0000","last_undegraded":"2024-05-10T11:15:01.459243+0000","last_fullsized":"2024-05-10T11:15:01.459243+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:50:55.771659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:29.796526+0000","last_change":"2024-05-10T11:14:22.258704+0000","last_active":"2024-05-10T11:15:29.796526+0000","last_peered":"2024-05-10T11:15:29.796526+0000","last_clean":"2024-05-10T11:15:29.796526+0000","last_became_active":"2024-05-10T11:14:22.258304+0000","last_became_peered":"2024-05-10T11:14:22.258304+0000","last_unstale":"2024-05-10T11:15:29.796526+0000","last_undegraded":"2024-05-10T11:15:29.796526+0000","last_fullsized":"2024-05-10T11:15:29.796526+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:02:41.451394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.425419+0000","last_change":"2024-05-10T11:14:35.325151+0000","last_active":"2024-05-10T11:15:01.425419+0000","last_peered":"2024-05-10T11:15:01.425419+0000","last_clean":"2024-05-10T11:15:01.425419+0000","last_became_active":"2024-05-10T11:14:35.324680+0000","last_became_peered":"2024-05-10T11:14:35.324680+0000","last_unstale":"2024-05-10T11:15:01.425419+0000","last_undegraded":"2024-05-10T11:15:01.425419+0000","last_fullsized":"2024-05-10T11:15:01.425419+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:12:17.578364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:28.787705+0000","last_change":"2024-05-10T11:15:01.428210+0000","last_active":"2024-05-10T11:15:28.787705+0000","last_peered":"2024-05-10T11:15:28.787705+0000","last_clean":"2024-05-10T11:15:28.787705+0000","last_became_active":"2024-05-10T11:15:01.427671+0000","last_became_peered":"2024-05-10T11:15:01.427671+0000","last_unstale":"2024-05-10T11:15:28.787705+0000","last_undegraded":"2024-05-10T11:15:28.787705+0000","last_fullsized":"2024-05-10T11:15:28.787705+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:44:51.312743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459730+0000","last_change":"2024-05-10T11:14:25.271350+0000","last_active":"2024-05-10T11:15:01.459730+0000","last_peered":"2024-05-10T11:15:01.459730+0000","last_clean":"2024-05-10T11:15:01.459730+0000","last_became_active":"2024-05-10T11:14:25.271047+0000","last_became_peered":"2024-05-10T11:14:25.271047+0000","last_unstale":"2024-05-10T11:15:01.459730+0000","last_undegraded":"2024-05-10T11:15:01.459730+0000","last_fullsized":"2024-05-10T11:15:01.459730+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:44:21.074007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.458323+0000","last_change":"2024-05-10T11:14:35.418910+0000","last_active":"2024-05-10T11:15:01.458323+0000","last_peered":"2024-05-10T11:15:01.458323+0000","last_clean":"2024-05-10T11:15:01.458323+0000","last_became_active":"2024-05-10T11:14:35.418460+0000","last_became_peered":"2024-05-10T11:14:35.418460+0000","last_unstale":"2024-05-10T11:15:01.458323+0000","last_undegraded":"2024-05-10T11:15:01.458323+0000","last_fullsized":"2024-05-10T11:15:01.458323+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:36:04.057776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'10","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:51.290625+0000","last_change":"2024-05-10T11:14:22.253673+0000","last_active":"2024-05-10T11:15:51.290625+0000","last_peered":"2024-05-10T11:15:51.290625+0000","last_clean":"2024-05-10T11:15:51.290625+0000","last_became_active":"2024-05-10T11:14:22.253304+0000","last_became_peered":"2024-05-10T11:14:22.253304+0000","last_unstale":"2024-05-10T11:15:51.290625+0000","last_undegraded":"2024-05-10T11:15:51.290625+0000","last_fullsized":"2024-05-10T11:15:51.290625+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:11:37.108621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.458871+0000","last_change":"2024-05-10T11:14:25.280590+0000","last_active":"2024-05-10T11:15:01.458871+0000","last_peered":"2024-05-10T11:15:01.458871+0000","last_clean":"2024-05-10T11:15:01.458871+0000","last_became_active":"2024-05-10T11:14:25.279990+0000","last_became_peered":"2024-05-10T11:14:25.279990+0000","last_unstale":"2024-05-10T11:15:01.458871+0000","last_undegraded":"2024-05-10T11:15:01.458871+0000","last_fullsized":"2024-05-10T11:15:01.458871+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:46:55.394767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.424240+0000","last_change":"2024-05-10T11:14:35.320931+0000","last_active":"2024-05-10T11:15:01.424240+0000","last_peered":"2024-05-10T11:15:01.424240+0000","last_clean":"2024-05-10T11:15:01.424240+0000","last_became_active":"2024-05-10T11:14:35.320523+0000","last_became_peered":"2024-05-10T11:14:35.320523+0000","last_unstale":"2024-05-10T11:15:01.424240+0000","last_undegraded":"2024-05-10T11:15:01.424240+0000","last_fullsized":"2024-05-10T11:15:01.424240+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:50:37.533397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:28.788398+0000","last_change":"2024-05-10T11:14:22.255115+0000","last_active":"2024-05-10T11:15:28.788398+0000","last_peered":"2024-05-10T11:15:28.788398+0000","last_clean":"2024-05-10T11:15:28.788398+0000","last_became_active":"2024-05-10T11:14:22.254732+0000","last_became_peered":"2024-05-10T11:14:22.254732+0000","last_unstale":"2024-05-10T11:15:28.788398+0000","last_undegraded":"2024-05-10T11:15:28.788398+0000","last_fullsized":"2024-05-10T11:15:28.788398+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:49:54.807312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.426039+0000","last_change":"2024-05-10T11:14:25.270766+0000","last_active":"2024-05-10T11:15:01.426039+0000","last_peered":"2024-05-10T11:15:01.426039+0000","last_clean":"2024-05-10T11:15:01.426039+0000","last_became_active":"2024-05-10T11:14:25.268958+0000","last_became_peered":"2024-05-10T11:14:25.268958+0000","last_unstale":"2024-05-10T11:15:01.426039+0000","last_undegraded":"2024-05-10T11:15:01.426039+0000","last_fullsized":"2024-05-10T11:15:01.426039+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:21:28.421478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459164+0000","last_change":"2024-05-10T11:14:35.420030+0000","last_active":"2024-05-10T11:15:01.459164+0000","last_peered":"2024-05-10T11:15:01.459164+0000","last_clean":"2024-05-10T11:15:01.459164+0000","last_became_active":"2024-05-10T11:14:35.419475+0000","last_became_peered":"2024-05-10T11:14:35.419475+0000","last_unstale":"2024-05-10T11:15:01.459164+0000","last_undegraded":"2024-05-10T11:15:01.459164+0000","last_fullsized":"2024-05-10T11:15:01.459164+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:13:51.207146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:28.789188+0000","last_change":"2024-05-10T11:15:01.429683+0000","last_active":"2024-05-10T11:15:28.789188+0000","last_peered":"2024-05-10T11:15:28.789188+0000","last_clean":"2024-05-10T11:15:28.789188+0000","last_became_active":"2024-05-10T11:15:01.429162+0000","last_became_peered":"2024-05-10T11:15:01.429162+0000","last_unstale":"2024-05-10T11:15:28.789188+0000","last_undegraded":"2024-05-10T11:15:28.789188+0000","last_fullsized":"2024-05-10T11:15:28.789188+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:57:35.166686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.429278+0000","last_change":"2024-05-10T11:14:25.282381+0000","last_active":"2024-05-10T11:15:01.429278+0000","last_peered":"2024-05-10T11:15:01.429278+0000","last_clean":"2024-05-10T11:15:01.429278+0000","last_became_active":"2024-05-10T11:14:25.281990+0000","last_became_peered":"2024-05-10T11:14:25.281990+0000","last_unstale":"2024-05-10T11:15:01.429278+0000","last_undegraded":"2024-05-10T11:15:01.429278+0000","last_fullsized":"2024-05-10T11:15:01.429278+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:39:03.726547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.422348+0000","last_change":"2024-05-10T11:14:35.328242+0000","last_active":"2024-05-10T11:15:01.422348+0000","last_peered":"2024-05-10T11:15:01.422348+0000","last_clean":"2024-05-10T11:15:01.422348+0000","last_became_active":"2024-05-10T11:14:35.327877+0000","last_became_peered":"2024-05-10T11:14:35.327877+0000","last_unstale":"2024-05-10T11:15:01.422348+0000","last_undegraded":"2024-05-10T11:15:01.422348+0000","last_fullsized":"2024-05-10T11:15:01.422348+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:01:57.826334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:28.789038+0000","last_change":"2024-05-10T11:14:22.251057+0000","last_active":"2024-05-10T11:15:28.789038+0000","last_peered":"2024-05-10T11:15:28.789038+0000","last_clean":"2024-05-10T11:15:28.789038+0000","last_became_active":"2024-05-10T11:14:22.250696+0000","last_became_peered":"2024-05-10T11:14:22.250696+0000","last_unstale":"2024-05-10T11:15:28.789038+0000","last_undegraded":"2024-05-10T11:15:28.789038+0000","last_fullsized":"2024-05-10T11:15:28.789038+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:38:53.780643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.424685+0000","last_change":"2024-05-10T11:14:25.283806+0000","last_active":"2024-05-10T11:15:01.424685+0000","last_peered":"2024-05-10T11:15:01.424685+0000","last_clean":"2024-05-10T11:15:01.424685+0000","last_became_active":"2024-05-10T11:14:25.282714+0000","last_became_peered":"2024-05-10T11:14:25.282714+0000","last_unstale":"2024-05-10T11:15:01.424685+0000","last_undegraded":"2024-05-10T11:15:01.424685+0000","last_fullsized":"2024-05-10T11:15:01.424685+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:05:11.940408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459414+0000","last_change":"2024-05-10T11:14:35.329986+0000","last_active":"2024-05-10T11:15:01.459414+0000","last_peered":"2024-05-10T11:15:01.459414+0000","last_clean":"2024-05-10T11:15:01.459414+0000","last_became_active":"2024-05-10T11:14:35.329477+0000","last_became_peered":"2024-05-10T11:14:35.329477+0000","last_unstale":"2024-05-10T11:15:01.459414+0000","last_undegraded":"2024-05-10T11:15:01.459414+0000","last_fullsized":"2024-05-10T11:15:01.459414+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:03:19.203580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.425866+0000","last_change":"2024-05-10T11:14:22.257627+0000","last_active":"2024-05-10T11:15:01.425866+0000","last_peered":"2024-05-10T11:15:01.425866+0000","last_clean":"2024-05-10T11:15:01.425866+0000","last_became_active":"2024-05-10T11:14:22.256990+0000","last_became_peered":"2024-05-10T11:14:22.256990+0000","last_unstale":"2024-05-10T11:15:01.425866+0000","last_undegraded":"2024-05-10T11:15:01.425866+0000","last_fullsized":"2024-05-10T11:15:01.425866+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:28:18.926821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.458675+0000","last_change":"2024-05-10T11:14:25.274799+0000","last_active":"2024-05-10T11:15:01.458675+0000","last_peered":"2024-05-10T11:15:01.458675+0000","last_clean":"2024-05-10T11:15:01.458675+0000","last_became_active":"2024-05-10T11:14:25.274443+0000","last_became_peered":"2024-05-10T11:14:25.274443+0000","last_unstale":"2024-05-10T11:15:01.458675+0000","last_undegraded":"2024-05-10T11:15:01.458675+0000","last_fullsized":"2024-05-10T11:15:01.458675+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:25:23.399798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459175+0000","last_change":"2024-05-10T11:14:35.418581+0000","last_active":"2024-05-10T11:15:01.459175+0000","last_peered":"2024-05-10T11:15:01.459175+0000","last_clean":"2024-05-10T11:15:01.459175+0000","last_became_active":"2024-05-10T11:14:35.418160+0000","last_became_peered":"2024-05-10T11:14:35.418160+0000","last_unstale":"2024-05-10T11:15:01.459175+0000","last_undegraded":"2024-05-10T11:15:01.459175+0000","last_fullsized":"2024-05-10T11:15:01.459175+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:39:01.873626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:28.788117+0000","last_change":"2024-05-10T11:14:22.260325+0000","last_active":"2024-05-10T11:15:28.788117+0000","last_peered":"2024-05-10T11:15:28.788117+0000","last_clean":"2024-05-10T11:15:28.788117+0000","last_became_active":"2024-05-10T11:14:22.259971+0000","last_became_peered":"2024-05-10T11:14:22.259971+0000","last_unstale":"2024-05-10T11:15:28.788117+0000","last_undegraded":"2024-05-10T11:15:28.788117+0000","last_fullsized":"2024-05-10T11:15:28.788117+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:34:55.547811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.427093+0000","last_change":"2024-05-10T11:14:25.279009+0000","last_active":"2024-05-10T11:15:01.427093+0000","last_peered":"2024-05-10T11:15:01.427093+0000","last_clean":"2024-05-10T11:15:01.427093+0000","last_became_active":"2024-05-10T11:14:25.278443+0000","last_became_peered":"2024-05-10T11:14:25.278443+0000","last_unstale":"2024-05-10T11:15:01.427093+0000","last_undegraded":"2024-05-10T11:15:01.427093+0000","last_fullsized":"2024-05-10T11:15:01.427093+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:25:22.306555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.426463+0000","last_change":"2024-05-10T11:14:35.319405+0000","last_active":"2024-05-10T11:15:01.426463+0000","last_peered":"2024-05-10T11:15:01.426463+0000","last_clean":"2024-05-10T11:15:01.426463+0000","last_became_active":"2024-05-10T11:14:35.318865+0000","last_became_peered":"2024-05-10T11:14:35.318865+0000","last_unstale":"2024-05-10T11:15:01.426463+0000","last_undegraded":"2024-05-10T11:15:01.426463+0000","last_fullsized":"2024-05-10T11:15:01.426463+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:22:41.592141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:29.798740+0000","last_change":"2024-05-10T11:14:22.299243+0000","last_active":"2024-05-10T11:15:29.798740+0000","last_peered":"2024-05-10T11:15:29.798740+0000","last_clean":"2024-05-10T11:15:29.798740+0000","last_became_active":"2024-05-10T11:14:22.298868+0000","last_became_peered":"2024-05-10T11:14:22.298868+0000","last_unstale":"2024-05-10T11:15:29.798740+0000","last_undegraded":"2024-05-10T11:15:29.798740+0000","last_fullsized":"2024-05-10T11:15:29.798740+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:54:15.078141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.427748+0000","last_change":"2024-05-10T11:14:25.281198+0000","last_active":"2024-05-10T11:15:01.427748+0000","last_peered":"2024-05-10T11:15:01.427748+0000","last_clean":"2024-05-10T11:15:01.427748+0000","last_became_active":"2024-05-10T11:14:25.280871+0000","last_became_peered":"2024-05-10T11:14:25.280871+0000","last_unstale":"2024-05-10T11:15:01.427748+0000","last_undegraded":"2024-05-10T11:15:01.427748+0000","last_fullsized":"2024-05-10T11:15:01.427748+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:24:55.843054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.425696+0000","last_change":"2024-05-10T11:14:35.329517+0000","last_active":"2024-05-10T11:15:01.425696+0000","last_peered":"2024-05-10T11:15:01.425696+0000","last_clean":"2024-05-10T11:15:01.425696+0000","last_became_active":"2024-05-10T11:14:35.329056+0000","last_became_peered":"2024-05-10T11:14:35.329056+0000","last_unstale":"2024-05-10T11:15:01.425696+0000","last_undegraded":"2024-05-10T11:15:01.425696+0000","last_fullsized":"2024-05-10T11:15:01.425696+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:09:24.844256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.460195+0000","last_change":"2024-05-10T11:14:22.257206+0000","last_active":"2024-05-10T11:15:01.460195+0000","last_peered":"2024-05-10T11:15:01.460195+0000","last_clean":"2024-05-10T11:15:01.460195+0000","last_became_active":"2024-05-10T11:14:22.256850+0000","last_became_peered":"2024-05-10T11:14:22.256850+0000","last_unstale":"2024-05-10T11:15:01.460195+0000","last_undegraded":"2024-05-10T11:15:01.460195+0000","last_fullsized":"2024-05-10T11:15:01.460195+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:34:31.973034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.461372+0000","last_change":"2024-05-10T11:15:01.461372+0000","last_active":"2024-05-10T11:15:01.461372+0000","last_peered":"2024-05-10T11:15:01.461372+0000","last_clean":"2024-05-10T11:15:01.461372+0000","last_became_active":"2024-05-10T11:15:01.460978+0000","last_became_peered":"2024-05-10T11:15:01.460978+0000","last_unstale":"2024-05-10T11:15:01.461372+0000","last_undegraded":"2024-05-10T11:15:01.461372+0000","last_fullsized":"2024-05-10T11:15:01.461372+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:18:45.750486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459010+0000","last_change":"2024-05-10T11:14:35.322387+0000","last_active":"2024-05-10T11:15:01.459010+0000","last_peered":"2024-05-10T11:15:01.459010+0000","last_clean":"2024-05-10T11:15:01.459010+0000","last_became_active":"2024-05-10T11:14:35.321939+0000","last_became_peered":"2024-05-10T11:14:35.321939+0000","last_unstale":"2024-05-10T11:15:01.459010+0000","last_undegraded":"2024-05-10T11:15:01.459010+0000","last_fullsized":"2024-05-10T11:15:01.459010+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:28:29.085363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459587+0000","last_change":"2024-05-10T11:14:22.258603+0000","last_active":"2024-05-10T11:15:01.459587+0000","last_peered":"2024-05-10T11:15:01.459587+0000","last_clean":"2024-05-10T11:15:01.459587+0000","last_became_active":"2024-05-10T11:14:22.258269+0000","last_became_peered":"2024-05-10T11:14:22.258269+0000","last_unstale":"2024-05-10T11:15:01.459587+0000","last_undegraded":"2024-05-10T11:15:01.459587+0000","last_fullsized":"2024-05-10T11:15:01.459587+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:30:04.240962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.427771+0000","last_change":"2024-05-10T11:14:25.281959+0000","last_active":"2024-05-10T11:15:01.427771+0000","last_peered":"2024-05-10T11:15:01.427771+0000","last_clean":"2024-05-10T11:15:01.427771+0000","last_became_active":"2024-05-10T11:14:25.281648+0000","last_became_peered":"2024-05-10T11:14:25.281648+0000","last_unstale":"2024-05-10T11:15:01.427771+0000","last_undegraded":"2024-05-10T11:15:01.427771+0000","last_fullsized":"2024-05-10T11:15:01.427771+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:24:01.569685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.424811+0000","last_change":"2024-05-10T11:14:25.284892+0000","last_active":"2024-05-10T11:15:01.424811+0000","last_peered":"2024-05-10T11:15:01.424811+0000","last_clean":"2024-05-10T11:15:01.424811+0000","last_became_active":"2024-05-10T11:14:25.284078+0000","last_became_peered":"2024-05-10T11:14:25.284078+0000","last_unstale":"2024-05-10T11:15:01.424811+0000","last_undegraded":"2024-05-10T11:15:01.424811+0000","last_fullsized":"2024-05-10T11:15:01.424811+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:49:43.348463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.425709+0000","last_change":"2024-05-10T11:14:25.274739+0000","last_active":"2024-05-10T11:15:01.425709+0000","last_peered":"2024-05-10T11:15:01.425709+0000","last_clean":"2024-05-10T11:15:01.425709+0000","last_became_active":"2024-05-10T11:14:25.274310+0000","last_became_peered":"2024-05-10T11:14:25.274310+0000","last_unstale":"2024-05-10T11:15:01.425709+0000","last_undegraded":"2024-05-10T11:15:01.425709+0000","last_fullsized":"2024-05-10T11:15:01.425709+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:21:16.935189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.425463+0000","last_change":"2024-05-10T11:14:25.272484+0000","last_active":"2024-05-10T11:15:01.425463+0000","last_peered":"2024-05-10T11:15:01.425463+0000","last_clean":"2024-05-10T11:15:01.425463+0000","last_became_active":"2024-05-10T11:14:25.272114+0000","last_became_peered":"2024-05-10T11:14:25.272114+0000","last_unstale":"2024-05-10T11:15:01.425463+0000","last_undegraded":"2024-05-10T11:15:01.425463+0000","last_fullsized":"2024-05-10T11:15:01.425463+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:46:34.537959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.460763+0000","last_change":"2024-05-10T11:15:01.460763+0000","last_active":"2024-05-10T11:15:01.460763+0000","last_peered":"2024-05-10T11:15:01.460763+0000","last_clean":"2024-05-10T11:15:01.460763+0000","last_became_active":"2024-05-10T11:15:01.460392+0000","last_became_peered":"2024-05-10T11:15:01.460392+0000","last_unstale":"2024-05-10T11:15:01.460763+0000","last_undegraded":"2024-05-10T11:15:01.460763+0000","last_fullsized":"2024-05-10T11:15:01.460763+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:15:42.859813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459826+0000","last_change":"2024-05-10T11:14:25.269640+0000","last_active":"2024-05-10T11:15:01.459826+0000","last_peered":"2024-05-10T11:15:01.459826+0000","last_clean":"2024-05-10T11:15:01.459826+0000","last_became_active":"2024-05-10T11:14:25.269348+0000","last_became_peered":"2024-05-10T11:14:25.269348+0000","last_unstale":"2024-05-10T11:15:01.459826+0000","last_undegraded":"2024-05-10T11:15:01.459826+0000","last_fullsized":"2024-05-10T11:15:01.459826+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:40:25.276433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.425328+0000","last_change":"2024-05-10T11:14:25.282563+0000","last_active":"2024-05-10T11:15:01.425328+0000","last_peered":"2024-05-10T11:15:01.425328+0000","last_clean":"2024-05-10T11:15:01.425328+0000","last_became_active":"2024-05-10T11:14:25.282145+0000","last_became_peered":"2024-05-10T11:14:25.282145+0000","last_unstale":"2024-05-10T11:15:01.425328+0000","last_undegraded":"2024-05-10T11:15:01.425328+0000","last_fullsized":"2024-05-10T11:15:01.425328+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:54:38.436669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.426576+0000","last_change":"2024-05-10T11:14:25.276753+0000","last_active":"2024-05-10T11:15:01.426576+0000","last_peered":"2024-05-10T11:15:01.426576+0000","last_clean":"2024-05-10T11:15:01.426576+0000","last_became_active":"2024-05-10T11:14:25.276342+0000","last_became_peered":"2024-05-10T11:14:25.276342+0000","last_unstale":"2024-05-10T11:15:01.426576+0000","last_undegraded":"2024-05-10T11:15:01.426576+0000","last_fullsized":"2024-05-10T11:15:01.426576+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:31:42.766069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459001+0000","last_change":"2024-05-10T11:14:25.275905+0000","last_active":"2024-05-10T11:15:01.459001+0000","last_peered":"2024-05-10T11:15:01.459001+0000","last_clean":"2024-05-10T11:15:01.459001+0000","last_became_active":"2024-05-10T11:14:25.275422+0000","last_became_peered":"2024-05-10T11:14:25.275422+0000","last_unstale":"2024-05-10T11:15:01.459001+0000","last_undegraded":"2024-05-10T11:15:01.459001+0000","last_fullsized":"2024-05-10T11:15:01.459001+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:39:24.033519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459199+0000","last_change":"2024-05-10T11:14:25.271602+0000","last_active":"2024-05-10T11:15:01.459199+0000","last_peered":"2024-05-10T11:15:01.459199+0000","last_clean":"2024-05-10T11:15:01.459199+0000","last_became_active":"2024-05-10T11:14:25.267206+0000","last_became_peered":"2024-05-10T11:14:25.267206+0000","last_unstale":"2024-05-10T11:15:01.459199+0000","last_undegraded":"2024-05-10T11:15:01.459199+0000","last_fullsized":"2024-05-10T11:15:01.459199+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:28:24.708297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459137+0000","last_change":"2024-05-10T11:14:25.280378+0000","last_active":"2024-05-10T11:15:01.459137+0000","last_peered":"2024-05-10T11:15:01.459137+0000","last_clean":"2024-05-10T11:15:01.459137+0000","last_became_active":"2024-05-10T11:14:25.280079+0000","last_became_peered":"2024-05-10T11:14:25.280079+0000","last_unstale":"2024-05-10T11:15:01.459137+0000","last_undegraded":"2024-05-10T11:15:01.459137+0000","last_fullsized":"2024-05-10T11:15:01.459137+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:13:47.781060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.426758+0000","last_change":"2024-05-10T11:14:25.272363+0000","last_active":"2024-05-10T11:15:01.426758+0000","last_peered":"2024-05-10T11:15:01.426758+0000","last_clean":"2024-05-10T11:15:01.426758+0000","last_became_active":"2024-05-10T11:14:25.271847+0000","last_became_peered":"2024-05-10T11:14:25.271847+0000","last_unstale":"2024-05-10T11:15:01.426758+0000","last_undegraded":"2024-05-10T11:15:01.426758+0000","last_fullsized":"2024-05-10T11:15:01.426758+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:47:54.661069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459897+0000","last_change":"2024-05-10T11:14:25.272688+0000","last_active":"2024-05-10T11:15:01.459897+0000","last_peered":"2024-05-10T11:15:01.459897+0000","last_clean":"2024-05-10T11:15:01.459897+0000","last_became_active":"2024-05-10T11:14:25.271654+0000","last_became_peered":"2024-05-10T11:14:25.271654+0000","last_unstale":"2024-05-10T11:15:01.459897+0000","last_undegraded":"2024-05-10T11:15:01.459897+0000","last_fullsized":"2024-05-10T11:15:01.459897+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:50:21.149707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.432279+0000","last_change":"2024-05-10T11:15:01.432279+0000","last_active":"2024-05-10T11:15:01.432279+0000","last_peered":"2024-05-10T11:15:01.432279+0000","last_clean":"2024-05-10T11:15:01.432279+0000","last_became_active":"2024-05-10T11:15:01.431751+0000","last_became_peered":"2024-05-10T11:15:01.431751+0000","last_unstale":"2024-05-10T11:15:01.432279+0000","last_undegraded":"2024-05-10T11:15:01.432279+0000","last_fullsized":"2024-05-10T11:15:01.432279+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:25:44.196618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":62275,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":171,"num_read_kb":153,"num_write":166,"num_write_kb":113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":186825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142,"ondisk_log_size":142,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645346,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1015808,"data_stored":505095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841575,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36992,"kb_used_data":1656,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708160,"statfs":{"total":95995035648,"available":95957155840,"internally_reserved":0,"allocated":1695744,"data_stored":1150615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36174273},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005101,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1032192,"data_stored":509405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201330,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1024000,"data_stored":505207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430264,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28600,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716552,"statfs":{"total":95995035648,"available":95965749248,"internally_reserved":0,"allocated":1622016,"data_stored":1101529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593789,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":500485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1069056,"data_stored":558278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1032192,"data_stored":501873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36400,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708752,"statfs":{"total":95995035648,"available":95957762048,"internally_reserved":0,"allocated":1089536,"data_stored":554616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149771,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1019904,"data_stored":500679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953543,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1019904,"data_stored":505526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790017,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28608,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716544,"statfs":{"total":95995035648,"available":95965741056,"internally_reserved":0,"allocated":1630208,"data_stored":1096726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":57883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":59624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":59852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T11:16:00.979 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-10T11:16:01.231 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:16:01.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:16:00 smithi178 ceph-mon[33359]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T11:16:01.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:16:00 smithi188 ceph-mon[34075]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T11:16:01.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:16:00 smithi100 ceph-mon[30296]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T11:16:02.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:16:01 smithi100 ceph-mon[30296]: pgmap v349: 129 pgs: 129 active+clean; 638 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.9 KiB/s wr, 7 op/s 2024-05-10T11:16:02.254 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:16:01 smithi178 ceph-mon[33359]: pgmap v349: 129 pgs: 129 active+clean; 638 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.9 KiB/s wr, 7 op/s 2024-05-10T11:16:02.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:16:01 smithi188 ceph-mon[34075]: pgmap v349: 129 pgs: 129 active+clean; 638 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.9 KiB/s wr, 7 op/s 2024-05-10T11:16:02.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:02.925+0000 7f412f4ed700 1 -- 172.21.15.100:0/4072071748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4128109510 msgr2=0x7f41281098f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:02.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:02.925+0000 7f412f4ed700 1 --2- 172.21.15.100:0/4072071748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4128109510 0x7f41281098f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4124009990 tx=0x7f4124009ca0 comp rx=0 tx=0).stop 2024-05-10T11:16:02.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:02.925+0000 7f412f4ed700 1 -- 172.21.15.100:0/4072071748 shutdown_connections 2024-05-10T11:16:02.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:02.925+0000 7f412f4ed700 1 --2- 172.21.15.100:0/4072071748 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f412806c5b0 0x7f4128110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:02.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:02.925+0000 7f412f4ed700 1 --2- 172.21.15.100:0/4072071748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4128109510 0x7f41281098f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:02.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:02.925+0000 7f412f4ed700 1 --2- 172.21.15.100:0/4072071748 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f412806bbf0 0x7f412806c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:02.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:02.925+0000 7f412f4ed700 1 -- 172.21.15.100:0/4072071748 >> 172.21.15.100:0/4072071748 conn(0x7f4128079370 msgr2=0x7f4128079780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:16:02.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:02.925+0000 7f412f4ed700 1 -- 172.21.15.100:0/4072071748 shutdown_connections 2024-05-10T11:16:02.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:02.925+0000 7f412f4ed700 1 -- 172.21.15.100:0/4072071748 wait complete. 2024-05-10T11:16:02.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:02.925+0000 7f412f4ed700 1 Processor -- start 2024-05-10T11:16:02.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:02.925+0000 7f412f4ed700 1 -- start start 2024-05-10T11:16:02.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:02.926+0000 7f412f4ed700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f412806bbf0 0x7f41281024a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:02.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:02.926+0000 7f412f4ed700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f412806c5b0 0x7f41281029e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:02.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:02.926+0000 7f412f4ed700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4128109510 0x7f4128102f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:02.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:02.926+0000 7f412f4ed700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f41281135a0 con 0x7f412806c5b0 2024-05-10T11:16:02.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:02.926+0000 7f412f4ed700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4128113420 con 0x7f412806bbf0 2024-05-10T11:16:02.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:02.926+0000 7f412f4ed700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4128113720 con 0x7f4128109510 2024-05-10T11:16:02.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:02.926+0000 7f412ca88700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f412806c5b0 0x7f41281029e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:02.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:02.926+0000 7f412ca88700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f412806c5b0 0x7f41281029e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37100/0 (socket says 172.21.15.100:37100) 2024-05-10T11:16:02.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:02.926+0000 7f412ca88700 1 -- 172.21.15.100:0/1894115337 learned_addr learned my addr 172.21.15.100:0/1894115337 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:16:02.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:02.926+0000 7f412d289700 1 --2- 172.21.15.100:0/1894115337 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f412806bbf0 0x7f41281024a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:02.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:02.926+0000 7f412da8a700 1 --2- 172.21.15.100:0/1894115337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4128109510 0x7f4128102f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:02.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:02.926+0000 7f412ca88700 1 -- 172.21.15.100:0/1894115337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4128109510 msgr2=0x7f4128102f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:02.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:02.926+0000 7f412ca88700 1 --2- 172.21.15.100:0/1894115337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4128109510 0x7f4128102f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:02.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:02.926+0000 7f412ca88700 1 -- 172.21.15.100:0/1894115337 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f412806bbf0 msgr2=0x7f41281024a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:02.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:02.926+0000 7f412ca88700 1 --2- 172.21.15.100:0/1894115337 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f412806bbf0 0x7f41281024a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:02.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:02.926+0000 7f412ca88700 1 -- 172.21.15.100:0/1894115337 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41281a80a0 con 0x7f412806c5b0 2024-05-10T11:16:02.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:02.926+0000 7f412ca88700 1 --2- 172.21.15.100:0/1894115337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f412806c5b0 0x7f41281029e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f411800d950 tx=0x7f411800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:16:02.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:02.927+0000 7f411e7fc700 1 -- 172.21.15.100:0/1894115337 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4118016070 con 0x7f412806c5b0 2024-05-10T11:16:02.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:02.927+0000 7f412f4ed700 1 -- 172.21.15.100:0/1894115337 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41281a8390 con 0x7f412806c5b0 2024-05-10T11:16:02.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:02.927+0000 7f412f4ed700 1 -- 172.21.15.100:0/1894115337 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41281a8920 con 0x7f412806c5b0 2024-05-10T11:16:02.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:02.927+0000 7f411e7fc700 1 -- 172.21.15.100:0/1894115337 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f411800fd20 con 0x7f412806c5b0 2024-05-10T11:16:02.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:02.927+0000 7f411e7fc700 1 -- 172.21.15.100:0/1894115337 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f411800e3b0 con 0x7f412806c5b0 2024-05-10T11:16:02.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:02.928+0000 7f412f4ed700 1 -- 172.21.15.100:0/1894115337 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f410c005330 con 0x7f412806c5b0 2024-05-10T11:16:02.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:02.929+0000 7f411e7fc700 1 -- 172.21.15.100:0/1894115337 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f4118019070 con 0x7f412806c5b0 2024-05-10T11:16:02.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:02.930+0000 7f411e7fc700 1 --2- 172.21.15.100:0/1894115337 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f41140b2990 0x7f41140b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:02.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:02.930+0000 7f412d289700 1 --2- 172.21.15.100:0/1894115337 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f41140b2990 0x7f41140b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:02.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:02.930+0000 7f411e7fc700 1 -- 172.21.15.100:0/1894115337 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f41180e2350 con 0x7f412806c5b0 2024-05-10T11:16:02.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:02.930+0000 7f412d289700 1 --2- 172.21.15.100:0/1894115337 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f41140b2990 0x7f41140b4e50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4124003760 tx=0x7f4124005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:16:02.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:02.934+0000 7f411e7fc700 1 -- 172.21.15.100:0/1894115337 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f41180a9bb0 con 0x7f412806c5b0 2024-05-10T11:16:03.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:03.139+0000 7f412f4ed700 1 -- 172.21.15.100:0/1894115337 --> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f410c000bc0 con 0x7f41140b2990 2024-05-10T11:16:03.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:03.148+0000 7f411e7fc700 1 -- 172.21.15.100:0/1894115337 <== mgr.14152 v2:172.21.15.100:6800/2507287356 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340354 (secure 0 0 0) 0x7f410c000bc0 con 0x7f41140b2990 2024-05-10T11:16:03.153 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:16:03.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:03.152+0000 7f412f4ed700 1 -- 172.21.15.100:0/1894115337 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f41140b2990 msgr2=0x7f41140b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:03.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:03.152+0000 7f412f4ed700 1 --2- 172.21.15.100:0/1894115337 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f41140b2990 0x7f41140b4e50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4124003760 tx=0x7f4124005fd0 comp rx=0 tx=0).stop 2024-05-10T11:16:03.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:03.152+0000 7f412f4ed700 1 -- 172.21.15.100:0/1894115337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f412806c5b0 msgr2=0x7f41281029e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:03.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:03.152+0000 7f412f4ed700 1 --2- 172.21.15.100:0/1894115337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f412806c5b0 0x7f41281029e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f411800d950 tx=0x7f411800dc60 comp rx=0 tx=0).stop 2024-05-10T11:16:03.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:03.152+0000 7f412f4ed700 1 -- 172.21.15.100:0/1894115337 shutdown_connections 2024-05-10T11:16:03.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:03.152+0000 7f412f4ed700 1 --2- 172.21.15.100:0/1894115337 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f41140b2990 0x7f41140b4e50 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:03.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:03.152+0000 7f412f4ed700 1 --2- 172.21.15.100:0/1894115337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4128109510 0x7f4128102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:03.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:03.152+0000 7f412f4ed700 1 --2- 172.21.15.100:0/1894115337 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f412806bbf0 0x7f41281024a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:03.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:03.152+0000 7f412f4ed700 1 --2- 172.21.15.100:0/1894115337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f412806c5b0 0x7f41281029e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:03.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:03.152+0000 7f412f4ed700 1 -- 172.21.15.100:0/1894115337 >> 172.21.15.100:0/1894115337 conn(0x7f4128079370 msgr2=0x7f41281011f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:16:03.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:03.152+0000 7f412f4ed700 1 -- 172.21.15.100:0/1894115337 shutdown_connections 2024-05-10T11:16:03.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:03.152+0000 7f412f4ed700 1 -- 172.21.15.100:0/1894115337 wait complete. 2024-05-10T11:16:03.156 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-05-10T11:16:03.965 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":350,"stamp":"2024-05-10T11:16:02.206016+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659130,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":278,"num_read_kb":239,"num_write":295,"num_write_kb":2272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":239,"ondisk_log_size":239,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":392216,"kb_used_data":13944,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124549608,"statfs":{"total":1151940427776,"available":1151538798592,"internally_reserved":0,"allocated":14278656,"data_stored":8007943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":387233083},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":33908,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":89,"num_read_kb":80,"num_write":24,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003271"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.425894+0000","last_change":"2024-05-10T11:14:25.286878+0000","last_active":"2024-05-10T11:15:01.425894+0000","last_peered":"2024-05-10T11:15:01.425894+0000","last_clean":"2024-05-10T11:15:01.425894+0000","last_became_active":"2024-05-10T11:14:25.286462+0000","last_became_peered":"2024-05-10T11:14:25.286462+0000","last_unstale":"2024-05-10T11:15:01.425894+0000","last_undegraded":"2024-05-10T11:15:01.425894+0000","last_fullsized":"2024-05-10T11:15:01.425894+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:13:40.627138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.425944+0000","last_change":"2024-05-10T11:14:25.280884+0000","last_active":"2024-05-10T11:15:01.425944+0000","last_peered":"2024-05-10T11:15:01.425944+0000","last_clean":"2024-05-10T11:15:01.425944+0000","last_became_active":"2024-05-10T11:14:25.280564+0000","last_became_peered":"2024-05-10T11:14:25.280564+0000","last_unstale":"2024-05-10T11:15:01.425944+0000","last_undegraded":"2024-05-10T11:15:01.425944+0000","last_fullsized":"2024-05-10T11:15:01.425944+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:14:16.735326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.425779+0000","last_change":"2024-05-10T11:14:25.278676+0000","last_active":"2024-05-10T11:15:01.425779+0000","last_peered":"2024-05-10T11:15:01.425779+0000","last_clean":"2024-05-10T11:15:01.425779+0000","last_became_active":"2024-05-10T11:14:25.278196+0000","last_became_peered":"2024-05-10T11:14:25.278196+0000","last_unstale":"2024-05-10T11:15:01.425779+0000","last_undegraded":"2024-05-10T11:15:01.425779+0000","last_fullsized":"2024-05-10T11:15:01.425779+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:50:29.390014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459617+0000","last_change":"2024-05-10T11:14:25.274741+0000","last_active":"2024-05-10T11:15:01.459617+0000","last_peered":"2024-05-10T11:15:01.459617+0000","last_clean":"2024-05-10T11:15:01.459617+0000","last_became_active":"2024-05-10T11:14:25.274251+0000","last_became_peered":"2024-05-10T11:14:25.274251+0000","last_unstale":"2024-05-10T11:15:01.459617+0000","last_undegraded":"2024-05-10T11:15:01.459617+0000","last_fullsized":"2024-05-10T11:15:01.459617+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:04:41.552042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.429957+0000","last_change":"2024-05-10T11:14:25.265232+0000","last_active":"2024-05-10T11:15:01.429957+0000","last_peered":"2024-05-10T11:15:01.429957+0000","last_clean":"2024-05-10T11:15:01.429957+0000","last_became_active":"2024-05-10T11:14:25.264852+0000","last_became_peered":"2024-05-10T11:14:25.264852+0000","last_unstale":"2024-05-10T11:15:01.429957+0000","last_undegraded":"2024-05-10T11:15:01.429957+0000","last_fullsized":"2024-05-10T11:15:01.429957+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:50:27.207258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.421885+0000","last_change":"2024-05-10T11:14:25.272626+0000","last_active":"2024-05-10T11:15:01.421885+0000","last_peered":"2024-05-10T11:15:01.421885+0000","last_clean":"2024-05-10T11:15:01.421885+0000","last_became_active":"2024-05-10T11:14:25.272305+0000","last_became_peered":"2024-05-10T11:14:25.272305+0000","last_unstale":"2024-05-10T11:15:01.421885+0000","last_undegraded":"2024-05-10T11:15:01.421885+0000","last_fullsized":"2024-05-10T11:15:01.421885+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:44:16.725453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.426391+0000","last_change":"2024-05-10T11:14:25.275460+0000","last_active":"2024-05-10T11:15:01.426391+0000","last_peered":"2024-05-10T11:15:01.426391+0000","last_clean":"2024-05-10T11:15:01.426391+0000","last_became_active":"2024-05-10T11:14:25.274571+0000","last_became_peered":"2024-05-10T11:14:25.274571+0000","last_unstale":"2024-05-10T11:15:01.426391+0000","last_undegraded":"2024-05-10T11:15:01.426391+0000","last_fullsized":"2024-05-10T11:15:01.426391+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:33:20.335738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.458437+0000","last_change":"2024-05-10T11:14:25.273975+0000","last_active":"2024-05-10T11:15:01.458437+0000","last_peered":"2024-05-10T11:15:01.458437+0000","last_clean":"2024-05-10T11:15:01.458437+0000","last_became_active":"2024-05-10T11:14:25.273652+0000","last_became_peered":"2024-05-10T11:14:25.273652+0000","last_unstale":"2024-05-10T11:15:01.458437+0000","last_undegraded":"2024-05-10T11:15:01.458437+0000","last_fullsized":"2024-05-10T11:15:01.458437+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:37:55.354449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.427092+0000","last_change":"2024-05-10T11:14:25.285530+0000","last_active":"2024-05-10T11:15:01.427092+0000","last_peered":"2024-05-10T11:15:01.427092+0000","last_clean":"2024-05-10T11:15:01.427092+0000","last_became_active":"2024-05-10T11:14:25.285235+0000","last_became_peered":"2024-05-10T11:14:25.285235+0000","last_unstale":"2024-05-10T11:15:01.427092+0000","last_undegraded":"2024-05-10T11:15:01.427092+0000","last_fullsized":"2024-05-10T11:15:01.427092+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:11:14.225339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.422343+0000","last_change":"2024-05-10T11:14:25.274738+0000","last_active":"2024-05-10T11:15:01.422343+0000","last_peered":"2024-05-10T11:15:01.422343+0000","last_clean":"2024-05-10T11:15:01.422343+0000","last_became_active":"2024-05-10T11:14:25.274107+0000","last_became_peered":"2024-05-10T11:14:25.274107+0000","last_unstale":"2024-05-10T11:15:01.422343+0000","last_undegraded":"2024-05-10T11:15:01.422343+0000","last_fullsized":"2024-05-10T11:15:01.422343+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:02:01.489930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.427600+0000","last_change":"2024-05-10T11:14:25.277431+0000","last_active":"2024-05-10T11:15:01.427600+0000","last_peered":"2024-05-10T11:15:01.427600+0000","last_clean":"2024-05-10T11:15:01.427600+0000","last_became_active":"2024-05-10T11:14:25.277130+0000","last_became_peered":"2024-05-10T11:14:25.277130+0000","last_unstale":"2024-05-10T11:15:01.427600+0000","last_undegraded":"2024-05-10T11:15:01.427600+0000","last_fullsized":"2024-05-10T11:15:01.427600+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:38:17.192282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459572+0000","last_change":"2024-05-10T11:14:25.276291+0000","last_active":"2024-05-10T11:15:01.459572+0000","last_peered":"2024-05-10T11:15:01.459572+0000","last_clean":"2024-05-10T11:15:01.459572+0000","last_became_active":"2024-05-10T11:14:25.275842+0000","last_became_peered":"2024-05-10T11:14:25.275842+0000","last_unstale":"2024-05-10T11:15:01.459572+0000","last_undegraded":"2024-05-10T11:15:01.459572+0000","last_fullsized":"2024-05-10T11:15:01.459572+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:20:53.443135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459970+0000","last_change":"2024-05-10T11:14:25.273950+0000","last_active":"2024-05-10T11:15:01.459970+0000","last_peered":"2024-05-10T11:15:01.459970+0000","last_clean":"2024-05-10T11:15:01.459970+0000","last_became_active":"2024-05-10T11:14:25.273644+0000","last_became_peered":"2024-05-10T11:14:25.273644+0000","last_unstale":"2024-05-10T11:15:01.459970+0000","last_undegraded":"2024-05-10T11:15:01.459970+0000","last_fullsized":"2024-05-10T11:15:01.459970+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:46:05.112575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.425711+0000","last_change":"2024-05-10T11:14:25.277930+0000","last_active":"2024-05-10T11:15:01.425711+0000","last_peered":"2024-05-10T11:15:01.425711+0000","last_clean":"2024-05-10T11:15:01.425711+0000","last_became_active":"2024-05-10T11:14:25.276793+0000","last_became_peered":"2024-05-10T11:14:25.276793+0000","last_unstale":"2024-05-10T11:15:01.425711+0000","last_undegraded":"2024-05-10T11:15:01.425711+0000","last_fullsized":"2024-05-10T11:15:01.425711+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:47:03.592882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.425713+0000","last_change":"2024-05-10T11:14:25.270434+0000","last_active":"2024-05-10T11:15:01.425713+0000","last_peered":"2024-05-10T11:15:01.425713+0000","last_clean":"2024-05-10T11:15:01.425713+0000","last_became_active":"2024-05-10T11:14:25.270139+0000","last_became_peered":"2024-05-10T11:14:25.270139+0000","last_unstale":"2024-05-10T11:15:01.425713+0000","last_undegraded":"2024-05-10T11:15:01.425713+0000","last_fullsized":"2024-05-10T11:15:01.425713+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:58:40.506560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.425989+0000","last_change":"2024-05-10T11:14:25.272981+0000","last_active":"2024-05-10T11:15:01.425989+0000","last_peered":"2024-05-10T11:15:01.425989+0000","last_clean":"2024-05-10T11:15:01.425989+0000","last_became_active":"2024-05-10T11:14:25.272592+0000","last_became_peered":"2024-05-10T11:14:25.272592+0000","last_unstale":"2024-05-10T11:15:01.425989+0000","last_undegraded":"2024-05-10T11:15:01.425989+0000","last_fullsized":"2024-05-10T11:15:01.425989+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:46:09.673013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.425631+0000","last_change":"2024-05-10T11:14:25.287304+0000","last_active":"2024-05-10T11:15:01.425631+0000","last_peered":"2024-05-10T11:15:01.425631+0000","last_clean":"2024-05-10T11:15:01.425631+0000","last_became_active":"2024-05-10T11:14:25.286964+0000","last_became_peered":"2024-05-10T11:14:25.286964+0000","last_unstale":"2024-05-10T11:15:01.425631+0000","last_undegraded":"2024-05-10T11:15:01.425631+0000","last_fullsized":"2024-05-10T11:15:01.425631+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:32:27.006840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.422582+0000","last_change":"2024-05-10T11:14:25.275366+0000","last_active":"2024-05-10T11:15:01.422582+0000","last_peered":"2024-05-10T11:15:01.422582+0000","last_clean":"2024-05-10T11:15:01.422582+0000","last_became_active":"2024-05-10T11:14:25.274986+0000","last_became_peered":"2024-05-10T11:14:25.274986+0000","last_unstale":"2024-05-10T11:15:01.422582+0000","last_undegraded":"2024-05-10T11:15:01.422582+0000","last_fullsized":"2024-05-10T11:15:01.422582+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:50:50.071975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.426182+0000","last_change":"2024-05-10T11:14:25.274588+0000","last_active":"2024-05-10T11:15:01.426182+0000","last_peered":"2024-05-10T11:15:01.426182+0000","last_clean":"2024-05-10T11:15:01.426182+0000","last_became_active":"2024-05-10T11:14:25.274179+0000","last_became_peered":"2024-05-10T11:14:25.274179+0000","last_unstale":"2024-05-10T11:15:01.426182+0000","last_undegraded":"2024-05-10T11:15:01.426182+0000","last_fullsized":"2024-05-10T11:15:01.426182+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:06:23.517082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.461337+0000","last_change":"2024-05-10T11:15:01.461337+0000","last_active":"2024-05-10T11:15:01.461337+0000","last_peered":"2024-05-10T11:15:01.461337+0000","last_clean":"2024-05-10T11:15:01.461337+0000","last_became_active":"2024-05-10T11:15:01.460776+0000","last_became_peered":"2024-05-10T11:15:01.460776+0000","last_unstale":"2024-05-10T11:15:01.461337+0000","last_undegraded":"2024-05-10T11:15:01.461337+0000","last_fullsized":"2024-05-10T11:15:01.461337+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:06:49.156483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,10,0],"acting":[2,3,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:29.797616+0000","last_change":"2024-05-10T11:14:22.253909+0000","last_active":"2024-05-10T11:15:29.797616+0000","last_peered":"2024-05-10T11:15:29.797616+0000","last_clean":"2024-05-10T11:15:29.797616+0000","last_became_active":"2024-05-10T11:14:22.253400+0000","last_became_peered":"2024-05-10T11:14:22.253400+0000","last_unstale":"2024-05-10T11:15:29.797616+0000","last_undegraded":"2024-05-10T11:15:29.797616+0000","last_fullsized":"2024-05-10T11:15:29.797616+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:06:46.173511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.458680+0000","last_change":"2024-05-10T11:14:25.273675+0000","last_active":"2024-05-10T11:15:01.458680+0000","last_peered":"2024-05-10T11:15:01.458680+0000","last_clean":"2024-05-10T11:15:01.458680+0000","last_became_active":"2024-05-10T11:14:25.273320+0000","last_became_peered":"2024-05-10T11:14:25.273320+0000","last_unstale":"2024-05-10T11:15:01.458680+0000","last_undegraded":"2024-05-10T11:15:01.458680+0000","last_fullsized":"2024-05-10T11:15:01.458680+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:09:18.232236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.458938+0000","last_change":"2024-05-10T11:14:35.326204+0000","last_active":"2024-05-10T11:15:01.458938+0000","last_peered":"2024-05-10T11:15:01.458938+0000","last_clean":"2024-05-10T11:15:01.458938+0000","last_became_active":"2024-05-10T11:14:35.325715+0000","last_became_peered":"2024-05-10T11:14:35.325715+0000","last_unstale":"2024-05-10T11:15:01.458938+0000","last_undegraded":"2024-05-10T11:15:01.458938+0000","last_fullsized":"2024-05-10T11:15:01.458938+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:06:30.665861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.458982+0000","last_change":"2024-05-10T11:14:22.299194+0000","last_active":"2024-05-10T11:15:01.458982+0000","last_peered":"2024-05-10T11:15:01.458982+0000","last_clean":"2024-05-10T11:15:01.458982+0000","last_became_active":"2024-05-10T11:14:22.298807+0000","last_became_peered":"2024-05-10T11:14:22.298807+0000","last_unstale":"2024-05-10T11:15:01.458982+0000","last_undegraded":"2024-05-10T11:15:01.458982+0000","last_fullsized":"2024-05-10T11:15:01.458982+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:24:51.054834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.425865+0000","last_change":"2024-05-10T11:14:25.277860+0000","last_active":"2024-05-10T11:15:01.425865+0000","last_peered":"2024-05-10T11:15:01.425865+0000","last_clean":"2024-05-10T11:15:01.425865+0000","last_became_active":"2024-05-10T11:14:25.276784+0000","last_became_peered":"2024-05-10T11:14:25.276784+0000","last_unstale":"2024-05-10T11:15:01.425865+0000","last_undegraded":"2024-05-10T11:15:01.425865+0000","last_fullsized":"2024-05-10T11:15:01.425865+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:42:57.673595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459231+0000","last_change":"2024-05-10T11:14:35.330679+0000","last_active":"2024-05-10T11:15:01.459231+0000","last_peered":"2024-05-10T11:15:01.459231+0000","last_clean":"2024-05-10T11:15:01.459231+0000","last_became_active":"2024-05-10T11:14:35.329819+0000","last_became_peered":"2024-05-10T11:14:35.329819+0000","last_unstale":"2024-05-10T11:15:01.459231+0000","last_undegraded":"2024-05-10T11:15:01.459231+0000","last_fullsized":"2024-05-10T11:15:01.459231+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:31:26.982784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:29.798596+0000","last_change":"2024-05-10T11:14:22.260402+0000","last_active":"2024-05-10T11:15:29.798596+0000","last_peered":"2024-05-10T11:15:29.798596+0000","last_clean":"2024-05-10T11:15:29.798596+0000","last_became_active":"2024-05-10T11:14:22.259619+0000","last_became_peered":"2024-05-10T11:14:22.259619+0000","last_unstale":"2024-05-10T11:15:29.798596+0000","last_undegraded":"2024-05-10T11:15:29.798596+0000","last_fullsized":"2024-05-10T11:15:29.798596+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:56:34.491626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.426783+0000","last_change":"2024-05-10T11:14:25.276117+0000","last_active":"2024-05-10T11:15:01.426783+0000","last_peered":"2024-05-10T11:15:01.426783+0000","last_clean":"2024-05-10T11:15:01.426783+0000","last_became_active":"2024-05-10T11:14:25.275758+0000","last_became_peered":"2024-05-10T11:14:25.275758+0000","last_unstale":"2024-05-10T11:15:01.426783+0000","last_undegraded":"2024-05-10T11:15:01.426783+0000","last_fullsized":"2024-05-10T11:15:01.426783+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:12:16.582244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459124+0000","last_change":"2024-05-10T11:14:35.324633+0000","last_active":"2024-05-10T11:15:01.459124+0000","last_peered":"2024-05-10T11:15:01.459124+0000","last_clean":"2024-05-10T11:15:01.459124+0000","last_became_active":"2024-05-10T11:14:35.324152+0000","last_became_peered":"2024-05-10T11:14:35.324152+0000","last_unstale":"2024-05-10T11:15:01.459124+0000","last_undegraded":"2024-05-10T11:15:01.459124+0000","last_fullsized":"2024-05-10T11:15:01.459124+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:56:50.008782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.430295+0000","last_change":"2024-05-10T11:14:22.259086+0000","last_active":"2024-05-10T11:15:01.430295+0000","last_peered":"2024-05-10T11:15:01.430295+0000","last_clean":"2024-05-10T11:15:01.430295+0000","last_became_active":"2024-05-10T11:14:22.258628+0000","last_became_peered":"2024-05-10T11:14:22.258628+0000","last_unstale":"2024-05-10T11:15:01.430295+0000","last_undegraded":"2024-05-10T11:15:01.430295+0000","last_fullsized":"2024-05-10T11:15:01.430295+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:39:13.472622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459840+0000","last_change":"2024-05-10T11:14:25.277556+0000","last_active":"2024-05-10T11:15:01.459840+0000","last_peered":"2024-05-10T11:15:01.459840+0000","last_clean":"2024-05-10T11:15:01.459840+0000","last_became_active":"2024-05-10T11:14:25.277201+0000","last_became_peered":"2024-05-10T11:14:25.277201+0000","last_unstale":"2024-05-10T11:15:01.459840+0000","last_undegraded":"2024-05-10T11:15:01.459840+0000","last_fullsized":"2024-05-10T11:15:01.459840+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:04:17.028305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.426304+0000","last_change":"2024-05-10T11:14:35.323338+0000","last_active":"2024-05-10T11:15:01.426304+0000","last_peered":"2024-05-10T11:15:01.426304+0000","last_clean":"2024-05-10T11:15:01.426304+0000","last_became_active":"2024-05-10T11:14:35.321948+0000","last_became_peered":"2024-05-10T11:14:35.321948+0000","last_unstale":"2024-05-10T11:15:01.426304+0000","last_undegraded":"2024-05-10T11:15:01.426304+0000","last_fullsized":"2024-05-10T11:15:01.426304+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:41:30.569125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":76,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:58.849842+0000","last_change":"2024-05-10T11:14:22.259187+0000","last_active":"2024-05-10T11:15:58.849842+0000","last_peered":"2024-05-10T11:15:58.849842+0000","last_clean":"2024-05-10T11:15:58.849842+0000","last_became_active":"2024-05-10T11:14:22.258812+0000","last_became_peered":"2024-05-10T11:14:22.258812+0000","last_unstale":"2024-05-10T11:15:58.849842+0000","last_undegraded":"2024-05-10T11:15:58.849842+0000","last_fullsized":"2024-05-10T11:15:58.849842+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:15:38.769705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":31,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.429212+0000","last_change":"2024-05-10T11:14:25.262469+0000","last_active":"2024-05-10T11:15:01.429212+0000","last_peered":"2024-05-10T11:15:01.429212+0000","last_clean":"2024-05-10T11:15:01.429212+0000","last_became_active":"2024-05-10T11:14:25.262104+0000","last_became_peered":"2024-05-10T11:14:25.262104+0000","last_unstale":"2024-05-10T11:15:01.429212+0000","last_undegraded":"2024-05-10T11:15:01.429212+0000","last_fullsized":"2024-05-10T11:15:01.429212+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:48:24.329136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.426854+0000","last_change":"2024-05-10T11:14:35.327737+0000","last_active":"2024-05-10T11:15:01.426854+0000","last_peered":"2024-05-10T11:15:01.426854+0000","last_clean":"2024-05-10T11:15:01.426854+0000","last_became_active":"2024-05-10T11:14:35.327209+0000","last_became_peered":"2024-05-10T11:14:35.327209+0000","last_unstale":"2024-05-10T11:15:01.426854+0000","last_undegraded":"2024-05-10T11:15:01.426854+0000","last_fullsized":"2024-05-10T11:15:01.426854+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:32:11.784752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:29.796781+0000","last_change":"2024-05-10T11:14:22.254853+0000","last_active":"2024-05-10T11:15:29.796781+0000","last_peered":"2024-05-10T11:15:29.796781+0000","last_clean":"2024-05-10T11:15:29.796781+0000","last_became_active":"2024-05-10T11:14:22.253352+0000","last_became_peered":"2024-05-10T11:14:22.253352+0000","last_unstale":"2024-05-10T11:15:29.796781+0000","last_undegraded":"2024-05-10T11:15:29.796781+0000","last_fullsized":"2024-05-10T11:15:29.796781+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:14:15.871042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459935+0000","last_change":"2024-05-10T11:14:25.266798+0000","last_active":"2024-05-10T11:15:01.459935+0000","last_peered":"2024-05-10T11:15:01.459935+0000","last_clean":"2024-05-10T11:15:01.459935+0000","last_became_active":"2024-05-10T11:14:25.266384+0000","last_became_peered":"2024-05-10T11:14:25.266384+0000","last_unstale":"2024-05-10T11:15:01.459935+0000","last_undegraded":"2024-05-10T11:15:01.459935+0000","last_fullsized":"2024-05-10T11:15:01.459935+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:18:07.338987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.462376+0000","last_change":"2024-05-10T11:15:01.462376+0000","last_active":"2024-05-10T11:15:01.462376+0000","last_peered":"2024-05-10T11:15:01.462376+0000","last_clean":"2024-05-10T11:15:01.462376+0000","last_became_active":"2024-05-10T11:15:01.461876+0000","last_became_peered":"2024-05-10T11:15:01.461876+0000","last_unstale":"2024-05-10T11:15:01.462376+0000","last_undegraded":"2024-05-10T11:15:01.462376+0000","last_fullsized":"2024-05-10T11:15:01.462376+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:35:52.076414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:29.796950+0000","last_change":"2024-05-10T11:14:22.253976+0000","last_active":"2024-05-10T11:15:29.796950+0000","last_peered":"2024-05-10T11:15:29.796950+0000","last_clean":"2024-05-10T11:15:29.796950+0000","last_became_active":"2024-05-10T11:14:22.252230+0000","last_became_peered":"2024-05-10T11:14:22.252230+0000","last_unstale":"2024-05-10T11:15:29.796950+0000","last_undegraded":"2024-05-10T11:15:29.796950+0000","last_fullsized":"2024-05-10T11:15:29.796950+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:14:03.208811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.458344+0000","last_change":"2024-05-10T11:14:25.279627+0000","last_active":"2024-05-10T11:15:01.458344+0000","last_peered":"2024-05-10T11:15:01.458344+0000","last_clean":"2024-05-10T11:15:01.458344+0000","last_became_active":"2024-05-10T11:14:25.279330+0000","last_became_peered":"2024-05-10T11:14:25.279330+0000","last_unstale":"2024-05-10T11:15:01.458344+0000","last_undegraded":"2024-05-10T11:15:01.458344+0000","last_fullsized":"2024-05-10T11:15:01.458344+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:50:19.608053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.426021+0000","last_change":"2024-05-10T11:14:35.321723+0000","last_active":"2024-05-10T11:15:01.426021+0000","last_peered":"2024-05-10T11:15:01.426021+0000","last_clean":"2024-05-10T11:15:01.426021+0000","last_became_active":"2024-05-10T11:14:35.321384+0000","last_became_peered":"2024-05-10T11:14:35.321384+0000","last_unstale":"2024-05-10T11:15:01.426021+0000","last_undegraded":"2024-05-10T11:15:01.426021+0000","last_fullsized":"2024-05-10T11:15:01.426021+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:04:07.770889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.460917+0000","last_change":"2024-05-10T11:14:22.255511+0000","last_active":"2024-05-10T11:15:01.460917+0000","last_peered":"2024-05-10T11:15:01.460917+0000","last_clean":"2024-05-10T11:15:01.460917+0000","last_became_active":"2024-05-10T11:14:22.254722+0000","last_became_peered":"2024-05-10T11:14:22.254722+0000","last_unstale":"2024-05-10T11:15:01.460917+0000","last_undegraded":"2024-05-10T11:15:01.460917+0000","last_fullsized":"2024-05-10T11:15:01.460917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:38:27.096011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.422230+0000","last_change":"2024-05-10T11:14:25.274009+0000","last_active":"2024-05-10T11:15:01.422230+0000","last_peered":"2024-05-10T11:15:01.422230+0000","last_clean":"2024-05-10T11:15:01.422230+0000","last_became_active":"2024-05-10T11:14:25.273672+0000","last_became_peered":"2024-05-10T11:14:25.273672+0000","last_unstale":"2024-05-10T11:15:01.422230+0000","last_undegraded":"2024-05-10T11:15:01.422230+0000","last_fullsized":"2024-05-10T11:15:01.422230+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:42:57.210131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.458974+0000","last_change":"2024-05-10T11:14:35.321520+0000","last_active":"2024-05-10T11:15:01.458974+0000","last_peered":"2024-05-10T11:15:01.458974+0000","last_clean":"2024-05-10T11:15:01.458974+0000","last_became_active":"2024-05-10T11:14:35.320575+0000","last_became_peered":"2024-05-10T11:14:35.320575+0000","last_unstale":"2024-05-10T11:15:01.458974+0000","last_undegraded":"2024-05-10T11:15:01.458974+0000","last_fullsized":"2024-05-10T11:15:01.458974+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:54:03.183426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":71,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:59.850623+0000","last_change":"2024-05-10T11:14:22.251463+0000","last_active":"2024-05-10T11:15:59.850623+0000","last_peered":"2024-05-10T11:15:59.850623+0000","last_clean":"2024-05-10T11:15:59.850623+0000","last_became_active":"2024-05-10T11:14:22.251132+0000","last_became_peered":"2024-05-10T11:14:22.251132+0000","last_unstale":"2024-05-10T11:15:59.850623+0000","last_undegraded":"2024-05-10T11:15:59.850623+0000","last_fullsized":"2024-05-10T11:15:59.850623+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:29:18.125380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.426728+0000","last_change":"2024-05-10T11:14:25.276333+0000","last_active":"2024-05-10T11:15:01.426728+0000","last_peered":"2024-05-10T11:15:01.426728+0000","last_clean":"2024-05-10T11:15:01.426728+0000","last_became_active":"2024-05-10T11:14:25.275818+0000","last_became_peered":"2024-05-10T11:14:25.275818+0000","last_unstale":"2024-05-10T11:15:01.426728+0000","last_undegraded":"2024-05-10T11:15:01.426728+0000","last_fullsized":"2024-05-10T11:15:01.426728+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:27:19.542645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.422080+0000","last_change":"2024-05-10T11:14:35.328589+0000","last_active":"2024-05-10T11:15:01.422080+0000","last_peered":"2024-05-10T11:15:01.422080+0000","last_clean":"2024-05-10T11:15:01.422080+0000","last_became_active":"2024-05-10T11:14:35.328052+0000","last_became_peered":"2024-05-10T11:14:35.328052+0000","last_unstale":"2024-05-10T11:15:01.422080+0000","last_undegraded":"2024-05-10T11:15:01.422080+0000","last_fullsized":"2024-05-10T11:15:01.422080+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:09:42.763932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:28.789882+0000","last_change":"2024-05-10T11:14:22.257872+0000","last_active":"2024-05-10T11:15:28.789882+0000","last_peered":"2024-05-10T11:15:28.789882+0000","last_clean":"2024-05-10T11:15:28.789882+0000","last_became_active":"2024-05-10T11:14:22.257530+0000","last_became_peered":"2024-05-10T11:14:22.257530+0000","last_unstale":"2024-05-10T11:15:28.789882+0000","last_undegraded":"2024-05-10T11:15:28.789882+0000","last_fullsized":"2024-05-10T11:15:28.789882+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:26:30.367209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.425956+0000","last_change":"2024-05-10T11:14:25.269436+0000","last_active":"2024-05-10T11:15:01.425956+0000","last_peered":"2024-05-10T11:15:01.425956+0000","last_clean":"2024-05-10T11:15:01.425956+0000","last_became_active":"2024-05-10T11:14:25.269102+0000","last_became_peered":"2024-05-10T11:14:25.269102+0000","last_unstale":"2024-05-10T11:15:01.425956+0000","last_undegraded":"2024-05-10T11:15:01.425956+0000","last_fullsized":"2024-05-10T11:15:01.425956+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:51:50.172464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459463+0000","last_change":"2024-05-10T11:14:35.330272+0000","last_active":"2024-05-10T11:15:01.459463+0000","last_peered":"2024-05-10T11:15:01.459463+0000","last_clean":"2024-05-10T11:15:01.459463+0000","last_became_active":"2024-05-10T11:14:35.329438+0000","last_became_peered":"2024-05-10T11:14:35.329438+0000","last_unstale":"2024-05-10T11:15:01.459463+0000","last_undegraded":"2024-05-10T11:15:01.459463+0000","last_fullsized":"2024-05-10T11:15:01.459463+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:07:00.559447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:28.788966+0000","last_change":"2024-05-10T11:14:22.299302+0000","last_active":"2024-05-10T11:15:28.788966+0000","last_peered":"2024-05-10T11:15:28.788966+0000","last_clean":"2024-05-10T11:15:28.788966+0000","last_became_active":"2024-05-10T11:14:22.298792+0000","last_became_peered":"2024-05-10T11:14:22.298792+0000","last_unstale":"2024-05-10T11:15:28.788966+0000","last_undegraded":"2024-05-10T11:15:28.788966+0000","last_fullsized":"2024-05-10T11:15:28.788966+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:12:35.280135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.426961+0000","last_change":"2024-05-10T11:14:25.275380+0000","last_active":"2024-05-10T11:15:01.426961+0000","last_peered":"2024-05-10T11:15:01.426961+0000","last_clean":"2024-05-10T11:15:01.426961+0000","last_became_active":"2024-05-10T11:14:25.275042+0000","last_became_peered":"2024-05-10T11:14:25.275042+0000","last_unstale":"2024-05-10T11:15:01.426961+0000","last_undegraded":"2024-05-10T11:15:01.426961+0000","last_fullsized":"2024-05-10T11:15:01.426961+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:22:49.598602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.422328+0000","last_change":"2024-05-10T11:14:35.327505+0000","last_active":"2024-05-10T11:15:01.422328+0000","last_peered":"2024-05-10T11:15:01.422328+0000","last_clean":"2024-05-10T11:15:01.422328+0000","last_became_active":"2024-05-10T11:14:35.326885+0000","last_became_peered":"2024-05-10T11:14:35.326885+0000","last_unstale":"2024-05-10T11:15:01.422328+0000","last_undegraded":"2024-05-10T11:15:01.422328+0000","last_fullsized":"2024-05-10T11:15:01.422328+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:23:09.010545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'34","reported_seq":140,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:16:00.855514+0000","last_change":"2024-05-10T11:14:22.257571+0000","last_active":"2024-05-10T11:16:00.855514+0000","last_peered":"2024-05-10T11:16:00.855514+0000","last_clean":"2024-05-10T11:16:00.855514+0000","last_became_active":"2024-05-10T11:14:22.257156+0000","last_became_peered":"2024-05-10T11:14:22.257156+0000","last_unstale":"2024-05-10T11:16:00.855514+0000","last_undegraded":"2024-05-10T11:16:00.855514+0000","last_fullsized":"2024-05-10T11:16:00.855514+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:28:06.397262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64136,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":67,"num_read_kb":96,"num_write":35,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.430478+0000","last_change":"2024-05-10T11:14:25.281160+0000","last_active":"2024-05-10T11:15:01.430478+0000","last_peered":"2024-05-10T11:15:01.430478+0000","last_clean":"2024-05-10T11:15:01.430478+0000","last_became_active":"2024-05-10T11:14:25.280763+0000","last_became_peered":"2024-05-10T11:14:25.280763+0000","last_unstale":"2024-05-10T11:15:01.430478+0000","last_undegraded":"2024-05-10T11:15:01.430478+0000","last_fullsized":"2024-05-10T11:15:01.430478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:19:53.350792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.460232+0000","last_change":"2024-05-10T11:14:35.320918+0000","last_active":"2024-05-10T11:15:01.460232+0000","last_peered":"2024-05-10T11:15:01.460232+0000","last_clean":"2024-05-10T11:15:01.460232+0000","last_became_active":"2024-05-10T11:14:35.320412+0000","last_became_peered":"2024-05-10T11:14:35.320412+0000","last_unstale":"2024-05-10T11:15:01.460232+0000","last_undegraded":"2024-05-10T11:15:01.460232+0000","last_fullsized":"2024-05-10T11:15:01.460232+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:55:52.585430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:29.798998+0000","last_change":"2024-05-10T11:14:22.257771+0000","last_active":"2024-05-10T11:15:29.798998+0000","last_peered":"2024-05-10T11:15:29.798998+0000","last_clean":"2024-05-10T11:15:29.798998+0000","last_became_active":"2024-05-10T11:14:22.257430+0000","last_became_peered":"2024-05-10T11:14:22.257430+0000","last_unstale":"2024-05-10T11:15:29.798998+0000","last_undegraded":"2024-05-10T11:15:29.798998+0000","last_fullsized":"2024-05-10T11:15:29.798998+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:23:50.131136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459594+0000","last_change":"2024-05-10T11:14:25.272276+0000","last_active":"2024-05-10T11:15:01.459594+0000","last_peered":"2024-05-10T11:15:01.459594+0000","last_clean":"2024-05-10T11:15:01.459594+0000","last_became_active":"2024-05-10T11:14:25.271932+0000","last_became_peered":"2024-05-10T11:14:25.271932+0000","last_unstale":"2024-05-10T11:15:01.459594+0000","last_undegraded":"2024-05-10T11:15:01.459594+0000","last_fullsized":"2024-05-10T11:15:01.459594+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:49:44.479056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.462435+0000","last_change":"2024-05-10T11:15:01.462435+0000","last_active":"2024-05-10T11:15:01.462435+0000","last_peered":"2024-05-10T11:15:01.462435+0000","last_clean":"2024-05-10T11:15:01.462435+0000","last_became_active":"2024-05-10T11:15:01.461890+0000","last_became_peered":"2024-05-10T11:15:01.461890+0000","last_unstale":"2024-05-10T11:15:01.462435+0000","last_undegraded":"2024-05-10T11:15:01.462435+0000","last_fullsized":"2024-05-10T11:15:01.462435+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:01:13.350655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:28.787852+0000","last_change":"2024-05-10T11:14:22.256713+0000","last_active":"2024-05-10T11:15:28.787852+0000","last_peered":"2024-05-10T11:15:28.787852+0000","last_clean":"2024-05-10T11:15:28.787852+0000","last_became_active":"2024-05-10T11:14:22.255847+0000","last_became_peered":"2024-05-10T11:14:22.255847+0000","last_unstale":"2024-05-10T11:15:28.787852+0000","last_undegraded":"2024-05-10T11:15:28.787852+0000","last_fullsized":"2024-05-10T11:15:28.787852+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:00:54.605506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.426720+0000","last_change":"2024-05-10T11:14:25.282118+0000","last_active":"2024-05-10T11:15:01.426720+0000","last_peered":"2024-05-10T11:15:01.426720+0000","last_clean":"2024-05-10T11:15:01.426720+0000","last_became_active":"2024-05-10T11:14:25.280958+0000","last_became_peered":"2024-05-10T11:14:25.280958+0000","last_unstale":"2024-05-10T11:15:01.426720+0000","last_undegraded":"2024-05-10T11:15:01.426720+0000","last_fullsized":"2024-05-10T11:15:01.426720+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:02:10.815733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.422251+0000","last_change":"2024-05-10T11:14:35.329876+0000","last_active":"2024-05-10T11:15:01.422251+0000","last_peered":"2024-05-10T11:15:01.422251+0000","last_clean":"2024-05-10T11:15:01.422251+0000","last_became_active":"2024-05-10T11:14:35.328923+0000","last_became_peered":"2024-05-10T11:14:35.328923+0000","last_unstale":"2024-05-10T11:15:01.422251+0000","last_undegraded":"2024-05-10T11:15:01.422251+0000","last_fullsized":"2024-05-10T11:15:01.422251+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:04:51.356552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'4","reported_seq":73,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:58.849072+0000","last_change":"2024-05-10T11:14:22.254515+0000","last_active":"2024-05-10T11:15:58.849072+0000","last_peered":"2024-05-10T11:15:58.849072+0000","last_clean":"2024-05-10T11:15:58.849072+0000","last_became_active":"2024-05-10T11:14:22.253648+0000","last_became_peered":"2024-05-10T11:14:22.253648+0000","last_unstale":"2024-05-10T11:15:58.849072+0000","last_undegraded":"2024-05-10T11:15:58.849072+0000","last_fullsized":"2024-05-10T11:15:58.849072+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:10:22.144446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":32,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.426521+0000","last_change":"2024-05-10T11:14:25.278841+0000","last_active":"2024-05-10T11:15:01.426521+0000","last_peered":"2024-05-10T11:15:01.426521+0000","last_clean":"2024-05-10T11:15:01.426521+0000","last_became_active":"2024-05-10T11:14:25.278527+0000","last_became_peered":"2024-05-10T11:14:25.278527+0000","last_unstale":"2024-05-10T11:15:01.426521+0000","last_undegraded":"2024-05-10T11:15:01.426521+0000","last_fullsized":"2024-05-10T11:15:01.426521+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:01:55.902587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459717+0000","last_change":"2024-05-10T11:14:35.331334+0000","last_active":"2024-05-10T11:15:01.459717+0000","last_peered":"2024-05-10T11:15:01.459717+0000","last_clean":"2024-05-10T11:15:01.459717+0000","last_became_active":"2024-05-10T11:14:35.330940+0000","last_became_peered":"2024-05-10T11:14:35.330940+0000","last_unstale":"2024-05-10T11:15:01.459717+0000","last_undegraded":"2024-05-10T11:15:01.459717+0000","last_fullsized":"2024-05-10T11:15:01.459717+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:11:12.241551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:29.799044+0000","last_change":"2024-05-10T11:14:22.299917+0000","last_active":"2024-05-10T11:15:29.799044+0000","last_peered":"2024-05-10T11:15:29.799044+0000","last_clean":"2024-05-10T11:15:29.799044+0000","last_became_active":"2024-05-10T11:14:22.299212+0000","last_became_peered":"2024-05-10T11:14:22.299212+0000","last_unstale":"2024-05-10T11:15:29.799044+0000","last_undegraded":"2024-05-10T11:15:29.799044+0000","last_fullsized":"2024-05-10T11:15:29.799044+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:52:48.740339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459876+0000","last_change":"2024-05-10T11:14:25.274693+0000","last_active":"2024-05-10T11:15:01.459876+0000","last_peered":"2024-05-10T11:15:01.459876+0000","last_clean":"2024-05-10T11:15:01.459876+0000","last_became_active":"2024-05-10T11:14:25.274378+0000","last_became_peered":"2024-05-10T11:14:25.274378+0000","last_unstale":"2024-05-10T11:15:01.459876+0000","last_undegraded":"2024-05-10T11:15:01.459876+0000","last_fullsized":"2024-05-10T11:15:01.459876+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:53:24.082742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.461224+0000","last_change":"2024-05-10T11:15:01.460541+0000","last_active":"2024-05-10T11:15:01.461224+0000","last_peered":"2024-05-10T11:15:01.461224+0000","last_clean":"2024-05-10T11:15:01.461224+0000","last_became_active":"2024-05-10T11:15:01.459954+0000","last_became_peered":"2024-05-10T11:15:01.459954+0000","last_unstale":"2024-05-10T11:15:01.461224+0000","last_undegraded":"2024-05-10T11:15:01.461224+0000","last_fullsized":"2024-05-10T11:15:01.461224+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:25:43.642441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,3,1],"acting":[8,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:31.793479+0000","last_change":"2024-05-10T11:14:22.258797+0000","last_active":"2024-05-10T11:15:31.793479+0000","last_peered":"2024-05-10T11:15:31.793479+0000","last_clean":"2024-05-10T11:15:31.793479+0000","last_became_active":"2024-05-10T11:14:22.258365+0000","last_became_peered":"2024-05-10T11:14:22.258365+0000","last_unstale":"2024-05-10T11:15:31.793479+0000","last_undegraded":"2024-05-10T11:15:31.793479+0000","last_fullsized":"2024-05-10T11:15:31.793479+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:58:53.291644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.460502+0000","last_change":"2024-05-10T11:15:01.460502+0000","last_active":"2024-05-10T11:15:01.460502+0000","last_peered":"2024-05-10T11:15:01.460502+0000","last_clean":"2024-05-10T11:15:01.460502+0000","last_became_active":"2024-05-10T11:15:01.460101+0000","last_became_peered":"2024-05-10T11:15:01.460101+0000","last_unstale":"2024-05-10T11:15:01.460502+0000","last_undegraded":"2024-05-10T11:15:01.460502+0000","last_fullsized":"2024-05-10T11:15:01.460502+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:22:39.735251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.429163+0000","last_change":"2024-05-10T11:14:35.419037+0000","last_active":"2024-05-10T11:15:01.429163+0000","last_peered":"2024-05-10T11:15:01.429163+0000","last_clean":"2024-05-10T11:15:01.429163+0000","last_became_active":"2024-05-10T11:14:35.418541+0000","last_became_peered":"2024-05-10T11:14:35.418541+0000","last_unstale":"2024-05-10T11:15:01.429163+0000","last_undegraded":"2024-05-10T11:15:01.429163+0000","last_fullsized":"2024-05-10T11:15:01.429163+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:05:49.560608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.422500+0000","last_change":"2024-05-10T11:14:22.299538+0000","last_active":"2024-05-10T11:15:01.422500+0000","last_peered":"2024-05-10T11:15:01.422500+0000","last_clean":"2024-05-10T11:15:01.422500+0000","last_became_active":"2024-05-10T11:14:22.299192+0000","last_became_peered":"2024-05-10T11:14:22.299192+0000","last_unstale":"2024-05-10T11:15:01.422500+0000","last_undegraded":"2024-05-10T11:15:01.422500+0000","last_fullsized":"2024-05-10T11:15:01.422500+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:28:43.041513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.421964+0000","last_change":"2024-05-10T11:14:25.272238+0000","last_active":"2024-05-10T11:15:01.421964+0000","last_peered":"2024-05-10T11:15:01.421964+0000","last_clean":"2024-05-10T11:15:01.421964+0000","last_became_active":"2024-05-10T11:14:25.271861+0000","last_became_peered":"2024-05-10T11:14:25.271861+0000","last_unstale":"2024-05-10T11:15:01.421964+0000","last_undegraded":"2024-05-10T11:15:01.421964+0000","last_fullsized":"2024-05-10T11:15:01.421964+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:00:08.821316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459561+0000","last_change":"2024-05-10T11:14:35.322239+0000","last_active":"2024-05-10T11:15:01.459561+0000","last_peered":"2024-05-10T11:15:01.459561+0000","last_clean":"2024-05-10T11:15:01.459561+0000","last_became_active":"2024-05-10T11:14:35.321864+0000","last_became_peered":"2024-05-10T11:14:35.321864+0000","last_unstale":"2024-05-10T11:15:01.459561+0000","last_undegraded":"2024-05-10T11:15:01.459561+0000","last_fullsized":"2024-05-10T11:15:01.459561+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:42:06.905879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.427027+0000","last_change":"2024-05-10T11:14:22.255449+0000","last_active":"2024-05-10T11:15:01.427027+0000","last_peered":"2024-05-10T11:15:01.427027+0000","last_clean":"2024-05-10T11:15:01.427027+0000","last_became_active":"2024-05-10T11:14:22.254966+0000","last_became_peered":"2024-05-10T11:14:22.254966+0000","last_unstale":"2024-05-10T11:15:01.427027+0000","last_undegraded":"2024-05-10T11:15:01.427027+0000","last_fullsized":"2024-05-10T11:15:01.427027+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:55:40.746115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.421870+0000","last_change":"2024-05-10T11:14:25.274050+0000","last_active":"2024-05-10T11:15:01.421870+0000","last_peered":"2024-05-10T11:15:01.421870+0000","last_clean":"2024-05-10T11:15:01.421870+0000","last_became_active":"2024-05-10T11:14:25.273720+0000","last_became_peered":"2024-05-10T11:14:25.273720+0000","last_unstale":"2024-05-10T11:15:01.421870+0000","last_undegraded":"2024-05-10T11:15:01.421870+0000","last_fullsized":"2024-05-10T11:15:01.421870+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:22:05.977673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459587+0000","last_change":"2024-05-10T11:14:35.418989+0000","last_active":"2024-05-10T11:15:01.459587+0000","last_peered":"2024-05-10T11:15:01.459587+0000","last_clean":"2024-05-10T11:15:01.459587+0000","last_became_active":"2024-05-10T11:14:35.418546+0000","last_became_peered":"2024-05-10T11:14:35.418546+0000","last_unstale":"2024-05-10T11:15:01.459587+0000","last_undegraded":"2024-05-10T11:15:01.459587+0000","last_fullsized":"2024-05-10T11:15:01.459587+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:23:37.875409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459653+0000","last_change":"2024-05-10T11:14:25.272694+0000","last_active":"2024-05-10T11:15:01.459653+0000","last_peered":"2024-05-10T11:15:01.459653+0000","last_clean":"2024-05-10T11:15:01.459653+0000","last_became_active":"2024-05-10T11:14:25.272303+0000","last_became_peered":"2024-05-10T11:14:25.272303+0000","last_unstale":"2024-05-10T11:15:01.459653+0000","last_undegraded":"2024-05-10T11:15:01.459653+0000","last_fullsized":"2024-05-10T11:15:01.459653+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:51:39.039122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":72,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:16:00.851643+0000","last_change":"2024-05-10T11:14:22.299359+0000","last_active":"2024-05-10T11:16:00.851643+0000","last_peered":"2024-05-10T11:16:00.851643+0000","last_clean":"2024-05-10T11:16:00.851643+0000","last_became_active":"2024-05-10T11:14:22.299023+0000","last_became_peered":"2024-05-10T11:14:22.299023+0000","last_unstale":"2024-05-10T11:16:00.851643+0000","last_undegraded":"2024-05-10T11:16:00.851643+0000","last_fullsized":"2024-05-10T11:16:00.851643+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:53:06.301774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.427641+0000","last_change":"2024-05-10T11:14:35.322640+0000","last_active":"2024-05-10T11:15:01.427641+0000","last_peered":"2024-05-10T11:15:01.427641+0000","last_clean":"2024-05-10T11:15:01.427641+0000","last_became_active":"2024-05-10T11:14:35.322226+0000","last_became_peered":"2024-05-10T11:14:35.322226+0000","last_unstale":"2024-05-10T11:15:01.427641+0000","last_undegraded":"2024-05-10T11:15:01.427641+0000","last_fullsized":"2024-05-10T11:15:01.427641+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:34:11.574438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.458527+0000","last_change":"2024-05-10T11:14:25.276645+0000","last_active":"2024-05-10T11:15:01.458527+0000","last_peered":"2024-05-10T11:15:01.458527+0000","last_clean":"2024-05-10T11:15:01.458527+0000","last_became_active":"2024-05-10T11:14:25.276295+0000","last_became_peered":"2024-05-10T11:14:25.276295+0000","last_unstale":"2024-05-10T11:15:01.458527+0000","last_undegraded":"2024-05-10T11:15:01.458527+0000","last_fullsized":"2024-05-10T11:15:01.458527+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:52:06.715073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:32.810362+0000","last_change":"2024-05-10T11:14:22.252908+0000","last_active":"2024-05-10T11:15:32.810362+0000","last_peered":"2024-05-10T11:15:32.810362+0000","last_clean":"2024-05-10T11:15:32.810362+0000","last_became_active":"2024-05-10T11:14:22.252532+0000","last_became_peered":"2024-05-10T11:14:22.252532+0000","last_unstale":"2024-05-10T11:15:32.810362+0000","last_undegraded":"2024-05-10T11:15:32.810362+0000","last_fullsized":"2024-05-10T11:15:32.810362+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:41:59.651442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":113,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.426311+0000","last_change":"2024-05-10T11:12:53.090796+0000","last_active":"2024-05-10T11:15:01.426311+0000","last_peered":"2024-05-10T11:15:01.426311+0000","last_clean":"2024-05-10T11:15:01.426311+0000","last_became_active":"2024-05-10T11:12:53.086566+0000","last_became_peered":"2024-05-10T11:12:53.086566+0000","last_unstale":"2024-05-10T11:15:01.426311+0000","last_undegraded":"2024-05-10T11:15:01.426311+0000","last_fullsized":"2024-05-10T11:15:01.426311+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:09:27.097043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:09:27.097043+0000","last_clean_scrub_stamp":"2024-05-10T11:09:27.097043+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:42:17.759719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459876+0000","last_change":"2024-05-10T11:14:35.419970+0000","last_active":"2024-05-10T11:15:01.459876+0000","last_peered":"2024-05-10T11:15:01.459876+0000","last_clean":"2024-05-10T11:15:01.459876+0000","last_became_active":"2024-05-10T11:14:35.419516+0000","last_became_peered":"2024-05-10T11:14:35.419516+0000","last_unstale":"2024-05-10T11:15:01.459876+0000","last_undegraded":"2024-05-10T11:15:01.459876+0000","last_fullsized":"2024-05-10T11:15:01.459876+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:01:25.148155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459243+0000","last_change":"2024-05-10T11:14:25.271478+0000","last_active":"2024-05-10T11:15:01.459243+0000","last_peered":"2024-05-10T11:15:01.459243+0000","last_clean":"2024-05-10T11:15:01.459243+0000","last_became_active":"2024-05-10T11:14:25.271143+0000","last_became_peered":"2024-05-10T11:14:25.271143+0000","last_unstale":"2024-05-10T11:15:01.459243+0000","last_undegraded":"2024-05-10T11:15:01.459243+0000","last_fullsized":"2024-05-10T11:15:01.459243+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:50:55.771659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:29.796526+0000","last_change":"2024-05-10T11:14:22.258704+0000","last_active":"2024-05-10T11:15:29.796526+0000","last_peered":"2024-05-10T11:15:29.796526+0000","last_clean":"2024-05-10T11:15:29.796526+0000","last_became_active":"2024-05-10T11:14:22.258304+0000","last_became_peered":"2024-05-10T11:14:22.258304+0000","last_unstale":"2024-05-10T11:15:29.796526+0000","last_undegraded":"2024-05-10T11:15:29.796526+0000","last_fullsized":"2024-05-10T11:15:29.796526+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:02:41.451394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.425419+0000","last_change":"2024-05-10T11:14:35.325151+0000","last_active":"2024-05-10T11:15:01.425419+0000","last_peered":"2024-05-10T11:15:01.425419+0000","last_clean":"2024-05-10T11:15:01.425419+0000","last_became_active":"2024-05-10T11:14:35.324680+0000","last_became_peered":"2024-05-10T11:14:35.324680+0000","last_unstale":"2024-05-10T11:15:01.425419+0000","last_undegraded":"2024-05-10T11:15:01.425419+0000","last_fullsized":"2024-05-10T11:15:01.425419+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:12:17.578364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:28.787705+0000","last_change":"2024-05-10T11:15:01.428210+0000","last_active":"2024-05-10T11:15:28.787705+0000","last_peered":"2024-05-10T11:15:28.787705+0000","last_clean":"2024-05-10T11:15:28.787705+0000","last_became_active":"2024-05-10T11:15:01.427671+0000","last_became_peered":"2024-05-10T11:15:01.427671+0000","last_unstale":"2024-05-10T11:15:28.787705+0000","last_undegraded":"2024-05-10T11:15:28.787705+0000","last_fullsized":"2024-05-10T11:15:28.787705+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:44:51.312743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459730+0000","last_change":"2024-05-10T11:14:25.271350+0000","last_active":"2024-05-10T11:15:01.459730+0000","last_peered":"2024-05-10T11:15:01.459730+0000","last_clean":"2024-05-10T11:15:01.459730+0000","last_became_active":"2024-05-10T11:14:25.271047+0000","last_became_peered":"2024-05-10T11:14:25.271047+0000","last_unstale":"2024-05-10T11:15:01.459730+0000","last_undegraded":"2024-05-10T11:15:01.459730+0000","last_fullsized":"2024-05-10T11:15:01.459730+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:44:21.074007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.458323+0000","last_change":"2024-05-10T11:14:35.418910+0000","last_active":"2024-05-10T11:15:01.458323+0000","last_peered":"2024-05-10T11:15:01.458323+0000","last_clean":"2024-05-10T11:15:01.458323+0000","last_became_active":"2024-05-10T11:14:35.418460+0000","last_became_peered":"2024-05-10T11:14:35.418460+0000","last_unstale":"2024-05-10T11:15:01.458323+0000","last_undegraded":"2024-05-10T11:15:01.458323+0000","last_fullsized":"2024-05-10T11:15:01.458323+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:36:04.057776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'11","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:54.291141+0000","last_change":"2024-05-10T11:14:22.253673+0000","last_active":"2024-05-10T11:15:54.291141+0000","last_peered":"2024-05-10T11:15:54.291141+0000","last_clean":"2024-05-10T11:15:54.291141+0000","last_became_active":"2024-05-10T11:14:22.253304+0000","last_became_peered":"2024-05-10T11:14:22.253304+0000","last_unstale":"2024-05-10T11:15:54.291141+0000","last_undegraded":"2024-05-10T11:15:54.291141+0000","last_fullsized":"2024-05-10T11:15:54.291141+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:11:37.108621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.458871+0000","last_change":"2024-05-10T11:14:25.280590+0000","last_active":"2024-05-10T11:15:01.458871+0000","last_peered":"2024-05-10T11:15:01.458871+0000","last_clean":"2024-05-10T11:15:01.458871+0000","last_became_active":"2024-05-10T11:14:25.279990+0000","last_became_peered":"2024-05-10T11:14:25.279990+0000","last_unstale":"2024-05-10T11:15:01.458871+0000","last_undegraded":"2024-05-10T11:15:01.458871+0000","last_fullsized":"2024-05-10T11:15:01.458871+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:46:55.394767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.424240+0000","last_change":"2024-05-10T11:14:35.320931+0000","last_active":"2024-05-10T11:15:01.424240+0000","last_peered":"2024-05-10T11:15:01.424240+0000","last_clean":"2024-05-10T11:15:01.424240+0000","last_became_active":"2024-05-10T11:14:35.320523+0000","last_became_peered":"2024-05-10T11:14:35.320523+0000","last_unstale":"2024-05-10T11:15:01.424240+0000","last_undegraded":"2024-05-10T11:15:01.424240+0000","last_fullsized":"2024-05-10T11:15:01.424240+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:50:37.533397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:28.788398+0000","last_change":"2024-05-10T11:14:22.255115+0000","last_active":"2024-05-10T11:15:28.788398+0000","last_peered":"2024-05-10T11:15:28.788398+0000","last_clean":"2024-05-10T11:15:28.788398+0000","last_became_active":"2024-05-10T11:14:22.254732+0000","last_became_peered":"2024-05-10T11:14:22.254732+0000","last_unstale":"2024-05-10T11:15:28.788398+0000","last_undegraded":"2024-05-10T11:15:28.788398+0000","last_fullsized":"2024-05-10T11:15:28.788398+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:49:54.807312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.426039+0000","last_change":"2024-05-10T11:14:25.270766+0000","last_active":"2024-05-10T11:15:01.426039+0000","last_peered":"2024-05-10T11:15:01.426039+0000","last_clean":"2024-05-10T11:15:01.426039+0000","last_became_active":"2024-05-10T11:14:25.268958+0000","last_became_peered":"2024-05-10T11:14:25.268958+0000","last_unstale":"2024-05-10T11:15:01.426039+0000","last_undegraded":"2024-05-10T11:15:01.426039+0000","last_fullsized":"2024-05-10T11:15:01.426039+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:21:28.421478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459164+0000","last_change":"2024-05-10T11:14:35.420030+0000","last_active":"2024-05-10T11:15:01.459164+0000","last_peered":"2024-05-10T11:15:01.459164+0000","last_clean":"2024-05-10T11:15:01.459164+0000","last_became_active":"2024-05-10T11:14:35.419475+0000","last_became_peered":"2024-05-10T11:14:35.419475+0000","last_unstale":"2024-05-10T11:15:01.459164+0000","last_undegraded":"2024-05-10T11:15:01.459164+0000","last_fullsized":"2024-05-10T11:15:01.459164+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:13:51.207146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:28.789188+0000","last_change":"2024-05-10T11:15:01.429683+0000","last_active":"2024-05-10T11:15:28.789188+0000","last_peered":"2024-05-10T11:15:28.789188+0000","last_clean":"2024-05-10T11:15:28.789188+0000","last_became_active":"2024-05-10T11:15:01.429162+0000","last_became_peered":"2024-05-10T11:15:01.429162+0000","last_unstale":"2024-05-10T11:15:28.789188+0000","last_undegraded":"2024-05-10T11:15:28.789188+0000","last_fullsized":"2024-05-10T11:15:28.789188+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:57:35.166686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.429278+0000","last_change":"2024-05-10T11:14:25.282381+0000","last_active":"2024-05-10T11:15:01.429278+0000","last_peered":"2024-05-10T11:15:01.429278+0000","last_clean":"2024-05-10T11:15:01.429278+0000","last_became_active":"2024-05-10T11:14:25.281990+0000","last_became_peered":"2024-05-10T11:14:25.281990+0000","last_unstale":"2024-05-10T11:15:01.429278+0000","last_undegraded":"2024-05-10T11:15:01.429278+0000","last_fullsized":"2024-05-10T11:15:01.429278+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:39:03.726547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.422348+0000","last_change":"2024-05-10T11:14:35.328242+0000","last_active":"2024-05-10T11:15:01.422348+0000","last_peered":"2024-05-10T11:15:01.422348+0000","last_clean":"2024-05-10T11:15:01.422348+0000","last_became_active":"2024-05-10T11:14:35.327877+0000","last_became_peered":"2024-05-10T11:14:35.327877+0000","last_unstale":"2024-05-10T11:15:01.422348+0000","last_undegraded":"2024-05-10T11:15:01.422348+0000","last_fullsized":"2024-05-10T11:15:01.422348+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:01:57.826334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:28.789038+0000","last_change":"2024-05-10T11:14:22.251057+0000","last_active":"2024-05-10T11:15:28.789038+0000","last_peered":"2024-05-10T11:15:28.789038+0000","last_clean":"2024-05-10T11:15:28.789038+0000","last_became_active":"2024-05-10T11:14:22.250696+0000","last_became_peered":"2024-05-10T11:14:22.250696+0000","last_unstale":"2024-05-10T11:15:28.789038+0000","last_undegraded":"2024-05-10T11:15:28.789038+0000","last_fullsized":"2024-05-10T11:15:28.789038+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:38:53.780643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.424685+0000","last_change":"2024-05-10T11:14:25.283806+0000","last_active":"2024-05-10T11:15:01.424685+0000","last_peered":"2024-05-10T11:15:01.424685+0000","last_clean":"2024-05-10T11:15:01.424685+0000","last_became_active":"2024-05-10T11:14:25.282714+0000","last_became_peered":"2024-05-10T11:14:25.282714+0000","last_unstale":"2024-05-10T11:15:01.424685+0000","last_undegraded":"2024-05-10T11:15:01.424685+0000","last_fullsized":"2024-05-10T11:15:01.424685+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:05:11.940408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459414+0000","last_change":"2024-05-10T11:14:35.329986+0000","last_active":"2024-05-10T11:15:01.459414+0000","last_peered":"2024-05-10T11:15:01.459414+0000","last_clean":"2024-05-10T11:15:01.459414+0000","last_became_active":"2024-05-10T11:14:35.329477+0000","last_became_peered":"2024-05-10T11:14:35.329477+0000","last_unstale":"2024-05-10T11:15:01.459414+0000","last_undegraded":"2024-05-10T11:15:01.459414+0000","last_fullsized":"2024-05-10T11:15:01.459414+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:03:19.203580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.425866+0000","last_change":"2024-05-10T11:14:22.257627+0000","last_active":"2024-05-10T11:15:01.425866+0000","last_peered":"2024-05-10T11:15:01.425866+0000","last_clean":"2024-05-10T11:15:01.425866+0000","last_became_active":"2024-05-10T11:14:22.256990+0000","last_became_peered":"2024-05-10T11:14:22.256990+0000","last_unstale":"2024-05-10T11:15:01.425866+0000","last_undegraded":"2024-05-10T11:15:01.425866+0000","last_fullsized":"2024-05-10T11:15:01.425866+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:28:18.926821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.458675+0000","last_change":"2024-05-10T11:14:25.274799+0000","last_active":"2024-05-10T11:15:01.458675+0000","last_peered":"2024-05-10T11:15:01.458675+0000","last_clean":"2024-05-10T11:15:01.458675+0000","last_became_active":"2024-05-10T11:14:25.274443+0000","last_became_peered":"2024-05-10T11:14:25.274443+0000","last_unstale":"2024-05-10T11:15:01.458675+0000","last_undegraded":"2024-05-10T11:15:01.458675+0000","last_fullsized":"2024-05-10T11:15:01.458675+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:25:23.399798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459175+0000","last_change":"2024-05-10T11:14:35.418581+0000","last_active":"2024-05-10T11:15:01.459175+0000","last_peered":"2024-05-10T11:15:01.459175+0000","last_clean":"2024-05-10T11:15:01.459175+0000","last_became_active":"2024-05-10T11:14:35.418160+0000","last_became_peered":"2024-05-10T11:14:35.418160+0000","last_unstale":"2024-05-10T11:15:01.459175+0000","last_undegraded":"2024-05-10T11:15:01.459175+0000","last_fullsized":"2024-05-10T11:15:01.459175+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:39:01.873626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:28.788117+0000","last_change":"2024-05-10T11:14:22.260325+0000","last_active":"2024-05-10T11:15:28.788117+0000","last_peered":"2024-05-10T11:15:28.788117+0000","last_clean":"2024-05-10T11:15:28.788117+0000","last_became_active":"2024-05-10T11:14:22.259971+0000","last_became_peered":"2024-05-10T11:14:22.259971+0000","last_unstale":"2024-05-10T11:15:28.788117+0000","last_undegraded":"2024-05-10T11:15:28.788117+0000","last_fullsized":"2024-05-10T11:15:28.788117+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:34:55.547811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.427093+0000","last_change":"2024-05-10T11:14:25.279009+0000","last_active":"2024-05-10T11:15:01.427093+0000","last_peered":"2024-05-10T11:15:01.427093+0000","last_clean":"2024-05-10T11:15:01.427093+0000","last_became_active":"2024-05-10T11:14:25.278443+0000","last_became_peered":"2024-05-10T11:14:25.278443+0000","last_unstale":"2024-05-10T11:15:01.427093+0000","last_undegraded":"2024-05-10T11:15:01.427093+0000","last_fullsized":"2024-05-10T11:15:01.427093+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:25:22.306555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.426463+0000","last_change":"2024-05-10T11:14:35.319405+0000","last_active":"2024-05-10T11:15:01.426463+0000","last_peered":"2024-05-10T11:15:01.426463+0000","last_clean":"2024-05-10T11:15:01.426463+0000","last_became_active":"2024-05-10T11:14:35.318865+0000","last_became_peered":"2024-05-10T11:14:35.318865+0000","last_unstale":"2024-05-10T11:15:01.426463+0000","last_undegraded":"2024-05-10T11:15:01.426463+0000","last_fullsized":"2024-05-10T11:15:01.426463+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:22:41.592141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:29.798740+0000","last_change":"2024-05-10T11:14:22.299243+0000","last_active":"2024-05-10T11:15:29.798740+0000","last_peered":"2024-05-10T11:15:29.798740+0000","last_clean":"2024-05-10T11:15:29.798740+0000","last_became_active":"2024-05-10T11:14:22.298868+0000","last_became_peered":"2024-05-10T11:14:22.298868+0000","last_unstale":"2024-05-10T11:15:29.798740+0000","last_undegraded":"2024-05-10T11:15:29.798740+0000","last_fullsized":"2024-05-10T11:15:29.798740+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:54:15.078141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.427748+0000","last_change":"2024-05-10T11:14:25.281198+0000","last_active":"2024-05-10T11:15:01.427748+0000","last_peered":"2024-05-10T11:15:01.427748+0000","last_clean":"2024-05-10T11:15:01.427748+0000","last_became_active":"2024-05-10T11:14:25.280871+0000","last_became_peered":"2024-05-10T11:14:25.280871+0000","last_unstale":"2024-05-10T11:15:01.427748+0000","last_undegraded":"2024-05-10T11:15:01.427748+0000","last_fullsized":"2024-05-10T11:15:01.427748+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:24:55.843054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.425696+0000","last_change":"2024-05-10T11:14:35.329517+0000","last_active":"2024-05-10T11:15:01.425696+0000","last_peered":"2024-05-10T11:15:01.425696+0000","last_clean":"2024-05-10T11:15:01.425696+0000","last_became_active":"2024-05-10T11:14:35.329056+0000","last_became_peered":"2024-05-10T11:14:35.329056+0000","last_unstale":"2024-05-10T11:15:01.425696+0000","last_undegraded":"2024-05-10T11:15:01.425696+0000","last_fullsized":"2024-05-10T11:15:01.425696+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:09:24.844256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.460195+0000","last_change":"2024-05-10T11:14:22.257206+0000","last_active":"2024-05-10T11:15:01.460195+0000","last_peered":"2024-05-10T11:15:01.460195+0000","last_clean":"2024-05-10T11:15:01.460195+0000","last_became_active":"2024-05-10T11:14:22.256850+0000","last_became_peered":"2024-05-10T11:14:22.256850+0000","last_unstale":"2024-05-10T11:15:01.460195+0000","last_undegraded":"2024-05-10T11:15:01.460195+0000","last_fullsized":"2024-05-10T11:15:01.460195+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:34:31.973034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.461372+0000","last_change":"2024-05-10T11:15:01.461372+0000","last_active":"2024-05-10T11:15:01.461372+0000","last_peered":"2024-05-10T11:15:01.461372+0000","last_clean":"2024-05-10T11:15:01.461372+0000","last_became_active":"2024-05-10T11:15:01.460978+0000","last_became_peered":"2024-05-10T11:15:01.460978+0000","last_unstale":"2024-05-10T11:15:01.461372+0000","last_undegraded":"2024-05-10T11:15:01.461372+0000","last_fullsized":"2024-05-10T11:15:01.461372+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:18:45.750486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459010+0000","last_change":"2024-05-10T11:14:35.322387+0000","last_active":"2024-05-10T11:15:01.459010+0000","last_peered":"2024-05-10T11:15:01.459010+0000","last_clean":"2024-05-10T11:15:01.459010+0000","last_became_active":"2024-05-10T11:14:35.321939+0000","last_became_peered":"2024-05-10T11:14:35.321939+0000","last_unstale":"2024-05-10T11:15:01.459010+0000","last_undegraded":"2024-05-10T11:15:01.459010+0000","last_fullsized":"2024-05-10T11:15:01.459010+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:28:29.085363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459587+0000","last_change":"2024-05-10T11:14:22.258603+0000","last_active":"2024-05-10T11:15:01.459587+0000","last_peered":"2024-05-10T11:15:01.459587+0000","last_clean":"2024-05-10T11:15:01.459587+0000","last_became_active":"2024-05-10T11:14:22.258269+0000","last_became_peered":"2024-05-10T11:14:22.258269+0000","last_unstale":"2024-05-10T11:15:01.459587+0000","last_undegraded":"2024-05-10T11:15:01.459587+0000","last_fullsized":"2024-05-10T11:15:01.459587+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:30:04.240962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.427771+0000","last_change":"2024-05-10T11:14:25.281959+0000","last_active":"2024-05-10T11:15:01.427771+0000","last_peered":"2024-05-10T11:15:01.427771+0000","last_clean":"2024-05-10T11:15:01.427771+0000","last_became_active":"2024-05-10T11:14:25.281648+0000","last_became_peered":"2024-05-10T11:14:25.281648+0000","last_unstale":"2024-05-10T11:15:01.427771+0000","last_undegraded":"2024-05-10T11:15:01.427771+0000","last_fullsized":"2024-05-10T11:15:01.427771+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:24:01.569685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.424811+0000","last_change":"2024-05-10T11:14:25.284892+0000","last_active":"2024-05-10T11:15:01.424811+0000","last_peered":"2024-05-10T11:15:01.424811+0000","last_clean":"2024-05-10T11:15:01.424811+0000","last_became_active":"2024-05-10T11:14:25.284078+0000","last_became_peered":"2024-05-10T11:14:25.284078+0000","last_unstale":"2024-05-10T11:15:01.424811+0000","last_undegraded":"2024-05-10T11:15:01.424811+0000","last_fullsized":"2024-05-10T11:15:01.424811+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:49:43.348463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.425709+0000","last_change":"2024-05-10T11:14:25.274739+0000","last_active":"2024-05-10T11:15:01.425709+0000","last_peered":"2024-05-10T11:15:01.425709+0000","last_clean":"2024-05-10T11:15:01.425709+0000","last_became_active":"2024-05-10T11:14:25.274310+0000","last_became_peered":"2024-05-10T11:14:25.274310+0000","last_unstale":"2024-05-10T11:15:01.425709+0000","last_undegraded":"2024-05-10T11:15:01.425709+0000","last_fullsized":"2024-05-10T11:15:01.425709+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:21:16.935189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.425463+0000","last_change":"2024-05-10T11:14:25.272484+0000","last_active":"2024-05-10T11:15:01.425463+0000","last_peered":"2024-05-10T11:15:01.425463+0000","last_clean":"2024-05-10T11:15:01.425463+0000","last_became_active":"2024-05-10T11:14:25.272114+0000","last_became_peered":"2024-05-10T11:14:25.272114+0000","last_unstale":"2024-05-10T11:15:01.425463+0000","last_undegraded":"2024-05-10T11:15:01.425463+0000","last_fullsized":"2024-05-10T11:15:01.425463+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:46:34.537959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.460763+0000","last_change":"2024-05-10T11:15:01.460763+0000","last_active":"2024-05-10T11:15:01.460763+0000","last_peered":"2024-05-10T11:15:01.460763+0000","last_clean":"2024-05-10T11:15:01.460763+0000","last_became_active":"2024-05-10T11:15:01.460392+0000","last_became_peered":"2024-05-10T11:15:01.460392+0000","last_unstale":"2024-05-10T11:15:01.460763+0000","last_undegraded":"2024-05-10T11:15:01.460763+0000","last_fullsized":"2024-05-10T11:15:01.460763+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:15:42.859813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459826+0000","last_change":"2024-05-10T11:14:25.269640+0000","last_active":"2024-05-10T11:15:01.459826+0000","last_peered":"2024-05-10T11:15:01.459826+0000","last_clean":"2024-05-10T11:15:01.459826+0000","last_became_active":"2024-05-10T11:14:25.269348+0000","last_became_peered":"2024-05-10T11:14:25.269348+0000","last_unstale":"2024-05-10T11:15:01.459826+0000","last_undegraded":"2024-05-10T11:15:01.459826+0000","last_fullsized":"2024-05-10T11:15:01.459826+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:40:25.276433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.425328+0000","last_change":"2024-05-10T11:14:25.282563+0000","last_active":"2024-05-10T11:15:01.425328+0000","last_peered":"2024-05-10T11:15:01.425328+0000","last_clean":"2024-05-10T11:15:01.425328+0000","last_became_active":"2024-05-10T11:14:25.282145+0000","last_became_peered":"2024-05-10T11:14:25.282145+0000","last_unstale":"2024-05-10T11:15:01.425328+0000","last_undegraded":"2024-05-10T11:15:01.425328+0000","last_fullsized":"2024-05-10T11:15:01.425328+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:54:38.436669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.426576+0000","last_change":"2024-05-10T11:14:25.276753+0000","last_active":"2024-05-10T11:15:01.426576+0000","last_peered":"2024-05-10T11:15:01.426576+0000","last_clean":"2024-05-10T11:15:01.426576+0000","last_became_active":"2024-05-10T11:14:25.276342+0000","last_became_peered":"2024-05-10T11:14:25.276342+0000","last_unstale":"2024-05-10T11:15:01.426576+0000","last_undegraded":"2024-05-10T11:15:01.426576+0000","last_fullsized":"2024-05-10T11:15:01.426576+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:31:42.766069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459001+0000","last_change":"2024-05-10T11:14:25.275905+0000","last_active":"2024-05-10T11:15:01.459001+0000","last_peered":"2024-05-10T11:15:01.459001+0000","last_clean":"2024-05-10T11:15:01.459001+0000","last_became_active":"2024-05-10T11:14:25.275422+0000","last_became_peered":"2024-05-10T11:14:25.275422+0000","last_unstale":"2024-05-10T11:15:01.459001+0000","last_undegraded":"2024-05-10T11:15:01.459001+0000","last_fullsized":"2024-05-10T11:15:01.459001+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:39:24.033519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459199+0000","last_change":"2024-05-10T11:14:25.271602+0000","last_active":"2024-05-10T11:15:01.459199+0000","last_peered":"2024-05-10T11:15:01.459199+0000","last_clean":"2024-05-10T11:15:01.459199+0000","last_became_active":"2024-05-10T11:14:25.267206+0000","last_became_peered":"2024-05-10T11:14:25.267206+0000","last_unstale":"2024-05-10T11:15:01.459199+0000","last_undegraded":"2024-05-10T11:15:01.459199+0000","last_fullsized":"2024-05-10T11:15:01.459199+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:28:24.708297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459137+0000","last_change":"2024-05-10T11:14:25.280378+0000","last_active":"2024-05-10T11:15:01.459137+0000","last_peered":"2024-05-10T11:15:01.459137+0000","last_clean":"2024-05-10T11:15:01.459137+0000","last_became_active":"2024-05-10T11:14:25.280079+0000","last_became_peered":"2024-05-10T11:14:25.280079+0000","last_unstale":"2024-05-10T11:15:01.459137+0000","last_undegraded":"2024-05-10T11:15:01.459137+0000","last_fullsized":"2024-05-10T11:15:01.459137+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:13:47.781060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.426758+0000","last_change":"2024-05-10T11:14:25.272363+0000","last_active":"2024-05-10T11:15:01.426758+0000","last_peered":"2024-05-10T11:15:01.426758+0000","last_clean":"2024-05-10T11:15:01.426758+0000","last_became_active":"2024-05-10T11:14:25.271847+0000","last_became_peered":"2024-05-10T11:14:25.271847+0000","last_unstale":"2024-05-10T11:15:01.426758+0000","last_undegraded":"2024-05-10T11:15:01.426758+0000","last_fullsized":"2024-05-10T11:15:01.426758+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:47:54.661069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459897+0000","last_change":"2024-05-10T11:14:25.272688+0000","last_active":"2024-05-10T11:15:01.459897+0000","last_peered":"2024-05-10T11:15:01.459897+0000","last_clean":"2024-05-10T11:15:01.459897+0000","last_became_active":"2024-05-10T11:14:25.271654+0000","last_became_peered":"2024-05-10T11:14:25.271654+0000","last_unstale":"2024-05-10T11:15:01.459897+0000","last_undegraded":"2024-05-10T11:15:01.459897+0000","last_fullsized":"2024-05-10T11:15:01.459897+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:50:21.149707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.432279+0000","last_change":"2024-05-10T11:15:01.432279+0000","last_active":"2024-05-10T11:15:01.432279+0000","last_peered":"2024-05-10T11:15:01.432279+0000","last_clean":"2024-05-10T11:15:01.432279+0000","last_became_active":"2024-05-10T11:15:01.431751+0000","last_became_peered":"2024-05-10T11:15:01.431751+0000","last_unstale":"2024-05-10T11:15:01.432279+0000","last_undegraded":"2024-05-10T11:15:01.432279+0000","last_fullsized":"2024-05-10T11:15:01.432279+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:25:44.196618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68528,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":202,"num_read_kb":175,"num_write":170,"num_write_kb":120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":205584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145,"ondisk_log_size":145,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645347,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1015808,"data_stored":505095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841576,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37000,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708152,"statfs":{"total":95995035648,"available":95957147648,"internally_reserved":0,"allocated":1703936,"data_stored":1156868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36174273},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005102,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1032192,"data_stored":509405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201330,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1024000,"data_stored":505207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430264,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28600,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716552,"statfs":{"total":95995035648,"available":95965749248,"internally_reserved":0,"allocated":1622016,"data_stored":1101529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593790,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":500485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1077248,"data_stored":564531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1032192,"data_stored":501873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36404,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708748,"statfs":{"total":95995035648,"available":95957757952,"internally_reserved":0,"allocated":1093632,"data_stored":560019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149772,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1019904,"data_stored":500679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953544,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1019904,"data_stored":505526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790018,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28608,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716544,"statfs":{"total":95995035648,"available":95965741056,"internally_reserved":0,"allocated":1630208,"data_stored":1096726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":65877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T11:16:03.971 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-10T11:16:03.971 INFO:tasks.cephadm:Setting up client nodes... 2024-05-10T11:16:03.972 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-10T11:16:04.214 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:16:04.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:16:03 smithi178 ceph-mon[33359]: pgmap v350: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.5 KiB/s wr, 9 op/s 2024-05-10T11:16:04.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:16:03 smithi188 ceph-mon[34075]: pgmap v350: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.5 KiB/s wr, 9 op/s 2024-05-10T11:16:04.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:16:03 smithi100 ceph-mon[30296]: pgmap v350: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.5 KiB/s wr, 9 op/s 2024-05-10T11:16:05.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:16:04 smithi178 ceph-mon[33359]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T11:16:05.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:16:04 smithi188 ceph-mon[34075]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T11:16:05.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:16:04 smithi100 ceph-mon[30296]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T11:16:05.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:16:05 smithi100 ceph-mon[30296]: pgmap v351: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-05-10T11:16:06.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:06.054+0000 7fc19b8cb700 1 -- 172.21.15.100:0/923093352 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1941050c0 msgr2=0x7fc19410b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:06.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:06.054+0000 7fc19b8cb700 1 --2- 172.21.15.100:0/923093352 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1941050c0 0x7fc19410b970 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc18c00b0d0 tx=0x7fc18c00b3e0 comp rx=0 tx=0).stop 2024-05-10T11:16:06.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:06.054+0000 7fc19b8cb700 1 -- 172.21.15.100:0/923093352 shutdown_connections 2024-05-10T11:16:06.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:06.054+0000 7fc19b8cb700 1 --2- 172.21.15.100:0/923093352 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1941050c0 0x7fc19410b970 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:06.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:06.054+0000 7fc19b8cb700 1 --2- 172.21.15.100:0/923093352 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc194103520 0x7fc194103940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:06.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:06.054+0000 7fc19b8cb700 1 --2- 172.21.15.100:0/923093352 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc194104720 0x7fc194104b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:06.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:06.054+0000 7fc19b8cb700 1 -- 172.21.15.100:0/923093352 >> 172.21.15.100:0/923093352 conn(0x7fc1940feac0 msgr2=0x7fc194100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:16:06.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:06.054+0000 7fc19b8cb700 1 -- 172.21.15.100:0/923093352 shutdown_connections 2024-05-10T11:16:06.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:06.054+0000 7fc19b8cb700 1 -- 172.21.15.100:0/923093352 wait complete. 2024-05-10T11:16:06.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:06.054+0000 7fc19b8cb700 1 Processor -- start 2024-05-10T11:16:06.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:06.054+0000 7fc19b8cb700 1 -- start start 2024-05-10T11:16:06.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:06.054+0000 7fc19b8cb700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc194103520 0x7fc1941994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:06.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:06.055+0000 7fc19b8cb700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc194104720 0x7fc1941999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:06.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:06.055+0000 7fc19b8cb700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1941050c0 0x7fc1941a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:06.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:06.055+0000 7fc19b8cb700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc19410eef0 con 0x7fc194104720 2024-05-10T11:16:06.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:06.055+0000 7fc19b8cb700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc19410ed70 con 0x7fc194103520 2024-05-10T11:16:06.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:06.055+0000 7fc19b8cb700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc19410f070 con 0x7fc1941050c0 2024-05-10T11:16:06.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:06.055+0000 7fc198e66700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc194104720 0x7fc1941999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:06.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:06.055+0000 7fc198e66700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc194104720 0x7fc1941999e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37122/0 (socket says 172.21.15.100:37122) 2024-05-10T11:16:06.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:06.055+0000 7fc198e66700 1 -- 172.21.15.100:0/1765179317 learned_addr learned my addr 172.21.15.100:0/1765179317 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:16:06.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:06.055+0000 7fc199667700 1 --2- 172.21.15.100:0/1765179317 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc194103520 0x7fc1941994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:06.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:06.055+0000 7fc199e68700 1 --2- 172.21.15.100:0/1765179317 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1941050c0 0x7fc1941a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:06.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:06.055+0000 7fc198e66700 1 -- 172.21.15.100:0/1765179317 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1941050c0 msgr2=0x7fc1941a0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:06.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:06.055+0000 7fc198e66700 1 --2- 172.21.15.100:0/1765179317 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1941050c0 0x7fc1941a0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:06.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:06.055+0000 7fc198e66700 1 -- 172.21.15.100:0/1765179317 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc194103520 msgr2=0x7fc1941994a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:06.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:06.055+0000 7fc198e66700 1 --2- 172.21.15.100:0/1765179317 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc194103520 0x7fc1941994a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:06.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:06.055+0000 7fc198e66700 1 -- 172.21.15.100:0/1765179317 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1941a1100 con 0x7fc194104720 2024-05-10T11:16:06.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:06.055+0000 7fc199e68700 1 --2- 172.21.15.100:0/1765179317 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1941050c0 0x7fc1941a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:16:06.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:06.055+0000 7fc198e66700 1 --2- 172.21.15.100:0/1765179317 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc194104720 0x7fc1941999e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc18400d980 tx=0x7fc18400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:16:06.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:06.055+0000 7fc18a7fc700 1 -- 172.21.15.100:0/1765179317 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc184016070 con 0x7fc194104720 2024-05-10T11:16:06.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:06.055+0000 7fc18a7fc700 1 -- 172.21.15.100:0/1765179317 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc18400de70 con 0x7fc194104720 2024-05-10T11:16:06.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:06.055+0000 7fc18a7fc700 1 -- 172.21.15.100:0/1765179317 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc184010e40 con 0x7fc194104720 2024-05-10T11:16:06.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:06.055+0000 7fc19b8cb700 1 -- 172.21.15.100:0/1765179317 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1941a13f0 con 0x7fc194104720 2024-05-10T11:16:06.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:06.055+0000 7fc19b8cb700 1 -- 172.21.15.100:0/1765179317 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1941a1960 con 0x7fc194104720 2024-05-10T11:16:06.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:06.056+0000 7fc19b8cb700 1 -- 172.21.15.100:0/1765179317 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1940519e0 con 0x7fc194104720 2024-05-10T11:16:06.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:06.058+0000 7fc18a7fc700 1 -- 172.21.15.100:0/1765179317 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fc184019070 con 0x7fc194104720 2024-05-10T11:16:06.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:06.059+0000 7fc18a7fc700 1 --2- 172.21.15.100:0/1765179317 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc1800b29b0 0x7fc1800b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:06.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:06.059+0000 7fc199667700 1 --2- 172.21.15.100:0/1765179317 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc1800b29b0 0x7fc1800b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:06.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:06.059+0000 7fc18a7fc700 1 -- 172.21.15.100:0/1765179317 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fc1840e2250 con 0x7fc194104720 2024-05-10T11:16:06.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:06.060+0000 7fc199667700 1 --2- 172.21.15.100:0/1765179317 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc1800b29b0 0x7fc1800b4e70 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc194104580 tx=0x7fc190009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:16:06.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:06.063+0000 7fc18a7fc700 1 -- 172.21.15.100:0/1765179317 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fc1840a9ab0 con 0x7fc194104720 2024-05-10T11:16:06.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:16:05 smithi178 ceph-mon[33359]: pgmap v351: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-05-10T11:16:06.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:16:05 smithi188 ceph-mon[34075]: pgmap v351: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-05-10T11:16:06.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:06.352+0000 7fc19b8cb700 1 -- 172.21.15.100:0/1765179317 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fc1940640b0 con 0x7fc194104720 2024-05-10T11:16:06.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:06.354+0000 7fc18a7fc700 1 -- 172.21.15.100:0/1765179317 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fc1840a93b0 con 0x7fc194104720 2024-05-10T11:16:06.354 INFO:teuthology.orchestra.run.smithi100.stdout:[client.0] 2024-05-10T11:16:06.354 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQD2AT5mmN8TFRAAK1CKb8FAiwh29Q3iTpDmvw== 2024-05-10T11:16:06.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:06.356+0000 7fc19b8cb700 1 -- 172.21.15.100:0/1765179317 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc1800b29b0 msgr2=0x7fc1800b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:06.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:06.356+0000 7fc19b8cb700 1 --2- 172.21.15.100:0/1765179317 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc1800b29b0 0x7fc1800b4e70 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc194104580 tx=0x7fc190009190 comp rx=0 tx=0).stop 2024-05-10T11:16:06.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:06.356+0000 7fc19b8cb700 1 -- 172.21.15.100:0/1765179317 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc194104720 msgr2=0x7fc1941999e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:06.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:06.356+0000 7fc19b8cb700 1 --2- 172.21.15.100:0/1765179317 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc194104720 0x7fc1941999e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc18400d980 tx=0x7fc18400dc90 comp rx=0 tx=0).stop 2024-05-10T11:16:06.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:06.356+0000 7fc19b8cb700 1 -- 172.21.15.100:0/1765179317 shutdown_connections 2024-05-10T11:16:06.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:06.356+0000 7fc19b8cb700 1 --2- 172.21.15.100:0/1765179317 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc1800b29b0 0x7fc1800b4e70 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:06.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:06.356+0000 7fc19b8cb700 1 --2- 172.21.15.100:0/1765179317 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1941050c0 0x7fc1941a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:06.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:06.356+0000 7fc19b8cb700 1 --2- 172.21.15.100:0/1765179317 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc194103520 0x7fc1941994a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:06.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:06.356+0000 7fc19b8cb700 1 --2- 172.21.15.100:0/1765179317 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc194104720 0x7fc1941999e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:06.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:06.356+0000 7fc19b8cb700 1 -- 172.21.15.100:0/1765179317 >> 172.21.15.100:0/1765179317 conn(0x7fc1940feac0 msgr2=0x7fc194107a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:16:06.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:06.357+0000 7fc19b8cb700 1 -- 172.21.15.100:0/1765179317 shutdown_connections 2024-05-10T11:16:06.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:06.357+0000 7fc19b8cb700 1 -- 172.21.15.100:0/1765179317 wait complete. 2024-05-10T11:16:07.032 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:16:06 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/1765179317' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-10T11:16:07.032 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:16:06 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/1765179317' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-10T11:16:07.061 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-05-10T11:16:07.061 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-05-10T11:16:07.061 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-10T11:16:07.100 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-10T11:16:07.100 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-05-10T11:16:07.100 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph mgr dump --format=json 2024-05-10T11:16:07.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:16:06 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/1765179317' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-10T11:16:07.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:16:06 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/1765179317' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-10T11:16:07.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:16:06 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/1765179317' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-10T11:16:07.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:16:06 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/1765179317' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-10T11:16:07.381 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:16:08.215 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:16:07 smithi100 ceph-mon[30296]: pgmap v352: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-05-10T11:16:08.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:16:07 smithi178 ceph-mon[33359]: pgmap v352: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-05-10T11:16:08.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:16:07 smithi188 ceph-mon[34075]: pgmap v352: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-05-10T11:16:09.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:09.155+0000 7efe5dda2700 1 -- 172.21.15.100:0/2033612720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe581086f0 msgr2=0x7efe581102c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:09.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:09.155+0000 7efe5dda2700 1 --2- 172.21.15.100:0/2033612720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe581086f0 0x7efe581102c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7efe5000b0d0 tx=0x7efe5000b3e0 comp rx=0 tx=0).stop 2024-05-10T11:16:09.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:09.155+0000 7efe5dda2700 1 -- 172.21.15.100:0/2033612720 shutdown_connections 2024-05-10T11:16:09.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:09.155+0000 7efe5dda2700 1 --2- 172.21.15.100:0/2033612720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe581086f0 0x7efe581102c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:09.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:09.155+0000 7efe5dda2700 1 --2- 172.21.15.100:0/2033612720 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe58103610 0x7efe581039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:09.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:09.155+0000 7efe5dda2700 1 --2- 172.21.15.100:0/2033612720 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe58103fc0 0x7efe58108010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:09.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:09.155+0000 7efe5dda2700 1 -- 172.21.15.100:0/2033612720 >> 172.21.15.100:0/2033612720 conn(0x7efe580feec0 msgr2=0x7efe581012e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:16:09.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:09.156+0000 7efe5dda2700 1 -- 172.21.15.100:0/2033612720 shutdown_connections 2024-05-10T11:16:09.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:09.156+0000 7efe5dda2700 1 -- 172.21.15.100:0/2033612720 wait complete. 2024-05-10T11:16:09.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:09.156+0000 7efe5dda2700 1 Processor -- start 2024-05-10T11:16:09.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:09.156+0000 7efe5dda2700 1 -- start start 2024-05-10T11:16:09.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:09.156+0000 7efe5dda2700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe58103610 0x7efe5819f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:09.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:09.156+0000 7efe5dda2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe58103fc0 0x7efe5819fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:09.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:09.157+0000 7efe5dda2700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe581086f0 0x7efe58199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:09.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:09.157+0000 7efe5dda2700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7efe58113530 con 0x7efe58103fc0 2024-05-10T11:16:09.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:09.157+0000 7efe5dda2700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efe581133b0 con 0x7efe581086f0 2024-05-10T11:16:09.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:09.157+0000 7efe5dda2700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7efe581136b0 con 0x7efe58103610 2024-05-10T11:16:09.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:09.157+0000 7efe57fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe58103fc0 0x7efe5819fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:09.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:09.157+0000 7efe57fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe58103fc0 0x7efe5819fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59734/0 (socket says 172.21.15.100:59734) 2024-05-10T11:16:09.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:09.157+0000 7efe57fff700 1 -- 172.21.15.100:0/3726126240 learned_addr learned my addr 172.21.15.100:0/3726126240 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:16:09.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:09.157+0000 7efe5d5a1700 1 --2- 172.21.15.100:0/3726126240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe581086f0 0x7efe58199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:09.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:09.157+0000 7efe5cda0700 1 --2- 172.21.15.100:0/3726126240 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe58103610 0x7efe5819f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:09.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:09.157+0000 7efe57fff700 1 -- 172.21.15.100:0/3726126240 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe58103610 msgr2=0x7efe5819f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:09.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:09.157+0000 7efe57fff700 1 --2- 172.21.15.100:0/3726126240 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe58103610 0x7efe5819f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:09.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:09.157+0000 7efe57fff700 1 -- 172.21.15.100:0/3726126240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe581086f0 msgr2=0x7efe58199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:09.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:09.157+0000 7efe57fff700 1 --2- 172.21.15.100:0/3726126240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe581086f0 0x7efe58199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:09.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:09.157+0000 7efe57fff700 1 -- 172.21.15.100:0/3726126240 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe5819a070 con 0x7efe58103fc0 2024-05-10T11:16:09.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:09.157+0000 7efe5cda0700 1 --2- 172.21.15.100:0/3726126240 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe58103610 0x7efe5819f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:16:09.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:09.157+0000 7efe5d5a1700 1 --2- 172.21.15.100:0/3726126240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe581086f0 0x7efe58199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:16:09.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:09.157+0000 7efe57fff700 1 --2- 172.21.15.100:0/3726126240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe58103fc0 0x7efe5819fc90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7efe4400b500 tx=0x7efe4400b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:16:09.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:09.157+0000 7efe55ffb700 1 -- 172.21.15.100:0/3726126240 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe44015020 con 0x7efe58103fc0 2024-05-10T11:16:09.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:09.157+0000 7efe5dda2700 1 -- 172.21.15.100:0/3726126240 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe5819a360 con 0x7efe58103fc0 2024-05-10T11:16:09.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:09.157+0000 7efe55ffb700 1 -- 172.21.15.100:0/3726126240 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efe44011cc0 con 0x7efe58103fc0 2024-05-10T11:16:09.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:09.157+0000 7efe55ffb700 1 -- 172.21.15.100:0/3726126240 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe44010c80 con 0x7efe58103fc0 2024-05-10T11:16:09.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:09.158+0000 7efe5dda2700 1 -- 172.21.15.100:0/3726126240 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe581a6620 con 0x7efe58103fc0 2024-05-10T11:16:09.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:09.158+0000 7efe5dda2700 1 -- 172.21.15.100:0/3726126240 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe58104af0 con 0x7efe58103fc0 2024-05-10T11:16:09.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:09.160+0000 7efe55ffb700 1 -- 172.21.15.100:0/3726126240 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7efe44018020 con 0x7efe58103fc0 2024-05-10T11:16:09.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:09.160+0000 7efe55ffb700 1 --2- 172.21.15.100:0/3726126240 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7efe480b2910 0x7efe480b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:09.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:09.160+0000 7efe55ffb700 1 -- 172.21.15.100:0/3726126240 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7efe440e1e10 con 0x7efe58103fc0 2024-05-10T11:16:09.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:09.160+0000 7efe5cda0700 1 --2- 172.21.15.100:0/3726126240 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7efe480b2910 0x7efe480b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:09.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:09.161+0000 7efe5cda0700 1 --2- 172.21.15.100:0/3726126240 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7efe480b2910 0x7efe480b4dd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7efe5819aa50 tx=0x7efe4c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:16:09.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:09.165+0000 7efe55ffb700 1 -- 172.21.15.100:0/3726126240 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7efe4401f020 con 0x7efe58103fc0 2024-05-10T11:16:09.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:09.428+0000 7efe5dda2700 1 -- 172.21.15.100:0/3726126240 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7efe5819ae40 con 0x7efe58103fc0 2024-05-10T11:16:09.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:09.433+0000 7efe55ffb700 1 -- 172.21.15.100:0/3726126240 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+288656 (secure 0 0 0) 0x7efe440a9780 con 0x7efe58103fc0 2024-05-10T11:16:09.437 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:16:09.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:09.437+0000 7efe5dda2700 1 -- 172.21.15.100:0/3726126240 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7efe480b2910 msgr2=0x7efe480b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:09.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:09.437+0000 7efe5dda2700 1 --2- 172.21.15.100:0/3726126240 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7efe480b2910 0x7efe480b4dd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7efe5819aa50 tx=0x7efe4c008040 comp rx=0 tx=0).stop 2024-05-10T11:16:09.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:09.437+0000 7efe5dda2700 1 -- 172.21.15.100:0/3726126240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe58103fc0 msgr2=0x7efe5819fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:09.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:09.437+0000 7efe5dda2700 1 --2- 172.21.15.100:0/3726126240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe58103fc0 0x7efe5819fc90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7efe4400b500 tx=0x7efe4400b8c0 comp rx=0 tx=0).stop 2024-05-10T11:16:09.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:09.437+0000 7efe5dda2700 1 -- 172.21.15.100:0/3726126240 shutdown_connections 2024-05-10T11:16:09.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:09.437+0000 7efe5dda2700 1 --2- 172.21.15.100:0/3726126240 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7efe480b2910 0x7efe480b4dd0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:09.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:09.437+0000 7efe5dda2700 1 --2- 172.21.15.100:0/3726126240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe581086f0 0x7efe58199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:09.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:09.437+0000 7efe5dda2700 1 --2- 172.21.15.100:0/3726126240 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe58103610 0x7efe5819f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:09.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:09.437+0000 7efe5dda2700 1 --2- 172.21.15.100:0/3726126240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe58103fc0 0x7efe5819fc90 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:09.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:09.437+0000 7efe5dda2700 1 -- 172.21.15.100:0/3726126240 >> 172.21.15.100:0/3726126240 conn(0x7efe580feec0 msgr2=0x7efe581002a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:16:09.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:09.437+0000 7efe5dda2700 1 -- 172.21.15.100:0/3726126240 shutdown_connections 2024-05-10T11:16:09.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:09.437+0000 7efe5dda2700 1 -- 172.21.15.100:0/3726126240 wait complete. 2024-05-10T11:16:10.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:16:09 smithi100 ceph-mon[30296]: pgmap v353: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-05-10T11:16:10.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:16:09 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/3726126240' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-10T11:16:10.155 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2507287356},{"type":"v1","addr":"172.21.15.100:6801","nonce":2507287356}]},"active_addr":"172.21.15.100:6801/2507287356","active_change":"2024-05-10T11:07:00.092843+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.100:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.100:0","nonce":2122358376}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.100:0","nonce":3925073324}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.100:0","nonce":3327438833}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.100:0","nonce":1927272520}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.100:0","nonce":1189213157}]}]} 2024-05-10T11:16:10.160 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-05-10T11:16:10.160 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-05-10T11:16:10.161 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-10T11:16:10.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:16:09 smithi178 ceph-mon[33359]: pgmap v353: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-05-10T11:16:10.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:16:09 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/3726126240' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-10T11:16:10.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:16:09 smithi188 ceph-mon[34075]: pgmap v353: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-05-10T11:16:10.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:16:09 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/3726126240' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-10T11:16:10.409 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:16:12.157 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:16:11 smithi100 ceph-mon[30296]: pgmap v354: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 597 B/s wr, 5 op/s 2024-05-10T11:16:12.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:12.158+0000 7f3643d13700 1 -- 172.21.15.100:0/392514373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f363c102d90 msgr2=0x7f363c10f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:12.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:12.158+0000 7f3643d13700 1 --2- 172.21.15.100:0/392514373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f363c102d90 0x7f363c10f350 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f363400b0a0 tx=0x7f363400b3b0 comp rx=0 tx=0).stop 2024-05-10T11:16:12.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:12.158+0000 7f3643d13700 1 -- 172.21.15.100:0/392514373 shutdown_connections 2024-05-10T11:16:12.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:12.158+0000 7f3643d13700 1 --2- 172.21.15.100:0/392514373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f363c102d90 0x7f363c10f350 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:12.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:12.158+0000 7f3643d13700 1 --2- 172.21.15.100:0/392514373 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f363c1042f0 0x7f363c1046d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:12.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:12.158+0000 7f3643d13700 1 --2- 172.21.15.100:0/392514373 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f363c1023d0 0x7f363c102850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:12.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:12.158+0000 7f3643d13700 1 -- 172.21.15.100:0/392514373 >> 172.21.15.100:0/392514373 conn(0x7f363c078410 msgr2=0x7f363c078820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:16:12.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:12.158+0000 7f3643d13700 1 -- 172.21.15.100:0/392514373 shutdown_connections 2024-05-10T11:16:12.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:12.158+0000 7f3643d13700 1 -- 172.21.15.100:0/392514373 wait complete. 2024-05-10T11:16:12.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:12.158+0000 7f3643d13700 1 Processor -- start 2024-05-10T11:16:12.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:12.158+0000 7f3643d13700 1 -- start start 2024-05-10T11:16:12.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:12.159+0000 7f3643d13700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f363c1023d0 0x7f363c1a1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:12.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:12.159+0000 7f3643d13700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f363c102d90 0x7f363c1a1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:12.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:12.159+0000 7f3643d13700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f363c1042f0 0x7f363c19bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:12.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:12.159+0000 7f3643d13700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f363c112520 con 0x7f363c102d90 2024-05-10T11:16:12.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:12.159+0000 7f3643d13700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f363c1123a0 con 0x7f363c1023d0 2024-05-10T11:16:12.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:12.159+0000 7f3643d13700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f363c1126a0 con 0x7f363c1042f0 2024-05-10T11:16:12.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:12.159+0000 7f36412ae700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f363c102d90 0x7f363c1a1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:12.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:12.159+0000 7f36422b0700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f363c1042f0 0x7f363c19bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:12.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:12.159+0000 7f36412ae700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f363c102d90 0x7f363c1a1e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59754/0 (socket says 172.21.15.100:59754) 2024-05-10T11:16:12.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:12.159+0000 7f36412ae700 1 -- 172.21.15.100:0/2201487087 learned_addr learned my addr 172.21.15.100:0/2201487087 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:16:12.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:12.159+0000 7f36422b0700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f363c1042f0 0x7f363c19bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:42898/0 (socket says 172.21.15.100:42898) 2024-05-10T11:16:12.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:12.159+0000 7f3641aaf700 1 --2- 172.21.15.100:0/2201487087 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f363c1023d0 0x7f363c1a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:12.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:12.159+0000 7f36412ae700 1 -- 172.21.15.100:0/2201487087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f363c1042f0 msgr2=0x7f363c19bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:12.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:12.159+0000 7f36412ae700 1 --2- 172.21.15.100:0/2201487087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f363c1042f0 0x7f363c19bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:12.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:12.159+0000 7f36412ae700 1 -- 172.21.15.100:0/2201487087 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f363c1023d0 msgr2=0x7f363c1a1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:12.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:12.159+0000 7f36412ae700 1 --2- 172.21.15.100:0/2201487087 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f363c1023d0 0x7f363c1a1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:12.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:12.159+0000 7f36412ae700 1 -- 172.21.15.100:0/2201487087 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f363c19c240 con 0x7f363c102d90 2024-05-10T11:16:12.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:12.159+0000 7f36412ae700 1 --2- 172.21.15.100:0/2201487087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f363c102d90 0x7f363c1a1e60 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f362c00e9c0 tx=0x7f362c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:16:12.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:12.160+0000 7f3632ffd700 1 -- 172.21.15.100:0/2201487087 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362c00cbd0 con 0x7f363c102d90 2024-05-10T11:16:12.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:12.160+0000 7f3643d13700 1 -- 172.21.15.100:0/2201487087 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f363c19c530 con 0x7f363c102d90 2024-05-10T11:16:12.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:12.160+0000 7f3632ffd700 1 -- 172.21.15.100:0/2201487087 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f362c00ee60 con 0x7f363c102d90 2024-05-10T11:16:12.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:12.160+0000 7f3632ffd700 1 -- 172.21.15.100:0/2201487087 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362c0054b0 con 0x7f363c102d90 2024-05-10T11:16:12.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:12.160+0000 7f3643d13700 1 -- 172.21.15.100:0/2201487087 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f363c1a87f0 con 0x7f363c102d90 2024-05-10T11:16:12.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:12.161+0000 7f3643d13700 1 -- 172.21.15.100:0/2201487087 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f363c06a190 con 0x7f363c102d90 2024-05-10T11:16:12.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:12.162+0000 7f3632ffd700 1 -- 172.21.15.100:0/2201487087 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f362c004750 con 0x7f363c102d90 2024-05-10T11:16:12.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:12.163+0000 7f3632ffd700 1 --2- 172.21.15.100:0/2201487087 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f36280b2910 0x7f36280b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:12.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:12.163+0000 7f3641aaf700 1 --2- 172.21.15.100:0/2201487087 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f36280b2910 0x7f36280b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:12.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:12.163+0000 7f3632ffd700 1 -- 172.21.15.100:0/2201487087 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f362c0e2c10 con 0x7f363c102d90 2024-05-10T11:16:12.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:12.163+0000 7f3641aaf700 1 --2- 172.21.15.100:0/2201487087 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f36280b2910 0x7f36280b4dd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3638007be0 tx=0x7f36380073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:16:12.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:12.166+0000 7f3632ffd700 1 -- 172.21.15.100:0/2201487087 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f362c054d70 con 0x7f363c102d90 2024-05-10T11:16:12.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:16:11 smithi178 ceph-mon[33359]: pgmap v354: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 597 B/s wr, 5 op/s 2024-05-10T11:16:12.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:16:11 smithi188 ceph-mon[34075]: pgmap v354: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 597 B/s wr, 5 op/s 2024-05-10T11:16:12.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:12.376+0000 7f3643d13700 1 -- 172.21.15.100:0/2201487087 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f363c1a8a20 con 0x7f363c102d90 2024-05-10T11:16:12.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:12.378+0000 7f3632ffd700 1 -- 172.21.15.100:0/2201487087 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24737 (secure 0 0 0) 0x7f362c0aa470 con 0x7f363c102d90 2024-05-10T11:16:12.378 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:16:12.378 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":89,"fsid":"20a542cc-0ebd-11ef-bc98-c7b262605968","created":"2024-05-10T11:06:05.454446+0000","modified":"2024-05-10T11:15:01.418144+0000","last_up_change":"2024-05-10T11:13:18.185471+0000","last_in_change":"2024-05-10T11:12:57.268678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T11:09:26.176441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T11:14:21.004285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T11:14:24.179414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T11:14:33.308517+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5f031418-da23-4e85-8669-160960dadd09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":643028278},{"type":"v1","addr":"172.21.15.100:6803","nonce":643028278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":643028278},{"type":"v1","addr":"172.21.15.100:6805","nonce":643028278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":643028278},{"type":"v1","addr":"172.21.15.100:6809","nonce":643028278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":643028278},{"type":"v1","addr":"172.21.15.100:6807","nonce":643028278}]},"public_addr":"172.21.15.100:6803/643028278","cluster_addr":"172.21.15.100:6805/643028278","heartbeat_back_addr":"172.21.15.100:6809/643028278","heartbeat_front_addr":"172.21.15.100:6807/643028278","state":["exists","up"]},{"osd":1,"uuid":"a908cdc9-001e-4b65-9ab0-0dd08dba20f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":3103987017},{"type":"v1","addr":"172.21.15.178:6801","nonce":3103987017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":3103987017},{"type":"v1","addr":"172.21.15.178:6803","nonce":3103987017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":3103987017},{"type":"v1","addr":"172.21.15.178:6807","nonce":3103987017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":3103987017},{"type":"v1","addr":"172.21.15.178:6805","nonce":3103987017}]},"public_addr":"172.21.15.178:6801/3103987017","cluster_addr":"172.21.15.178:6803/3103987017","heartbeat_back_addr":"172.21.15.178:6807/3103987017","heartbeat_front_addr":"172.21.15.178:6805/3103987017","state":["exists","up"]},{"osd":2,"uuid":"4d8f89ae-9424-46e9-9c80-17c47cc5e8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3991466468},{"type":"v1","addr":"172.21.15.188:6801","nonce":3991466468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3991466468},{"type":"v1","addr":"172.21.15.188:6803","nonce":3991466468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3991466468},{"type":"v1","addr":"172.21.15.188:6807","nonce":3991466468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3991466468},{"type":"v1","addr":"172.21.15.188:6805","nonce":3991466468}]},"public_addr":"172.21.15.188:6801/3991466468","cluster_addr":"172.21.15.188:6803/3991466468","heartbeat_back_addr":"172.21.15.188:6807/3991466468","heartbeat_front_addr":"172.21.15.188:6805/3991466468","state":["exists","up"]},{"osd":3,"uuid":"4c8865aa-fef4-4710-b766-1385adb9462c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1713357192},{"type":"v1","addr":"172.21.15.100:6811","nonce":1713357192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1713357192},{"type":"v1","addr":"172.21.15.100:6813","nonce":1713357192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1713357192},{"type":"v1","addr":"172.21.15.100:6817","nonce":1713357192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1713357192},{"type":"v1","addr":"172.21.15.100:6815","nonce":1713357192}]},"public_addr":"172.21.15.100:6811/1713357192","cluster_addr":"172.21.15.100:6813/1713357192","heartbeat_back_addr":"172.21.15.100:6817/1713357192","heartbeat_front_addr":"172.21.15.100:6815/1713357192","state":["exists","up"]},{"osd":4,"uuid":"a9653800-da57-43eb-8781-3188e8a3949f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":2222814192},{"type":"v1","addr":"172.21.15.178:6809","nonce":2222814192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":2222814192},{"type":"v1","addr":"172.21.15.178:6811","nonce":2222814192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":2222814192},{"type":"v1","addr":"172.21.15.178:6815","nonce":2222814192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":2222814192},{"type":"v1","addr":"172.21.15.178:6813","nonce":2222814192}]},"public_addr":"172.21.15.178:6809/2222814192","cluster_addr":"172.21.15.178:6811/2222814192","heartbeat_back_addr":"172.21.15.178:6815/2222814192","heartbeat_front_addr":"172.21.15.178:6813/2222814192","state":["exists","up"]},{"osd":5,"uuid":"fb5f8af2-1eb0-4d8a-a7a3-ed6dd68e692e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3517375935},{"type":"v1","addr":"172.21.15.188:6809","nonce":3517375935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3517375935},{"type":"v1","addr":"172.21.15.188:6811","nonce":3517375935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3517375935},{"type":"v1","addr":"172.21.15.188:6815","nonce":3517375935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3517375935},{"type":"v1","addr":"172.21.15.188:6813","nonce":3517375935}]},"public_addr":"172.21.15.188:6809/3517375935","cluster_addr":"172.21.15.188:6811/3517375935","heartbeat_back_addr":"172.21.15.188:6815/3517375935","heartbeat_front_addr":"172.21.15.188:6813/3517375935","state":["exists","up"]},{"osd":6,"uuid":"84b57387-0a94-4e49-b099-493d329b87dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1410593862},{"type":"v1","addr":"172.21.15.100:6819","nonce":1410593862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1410593862},{"type":"v1","addr":"172.21.15.100:6821","nonce":1410593862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1410593862},{"type":"v1","addr":"172.21.15.100:6825","nonce":1410593862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1410593862},{"type":"v1","addr":"172.21.15.100:6823","nonce":1410593862}]},"public_addr":"172.21.15.100:6819/1410593862","cluster_addr":"172.21.15.100:6821/1410593862","heartbeat_back_addr":"172.21.15.100:6825/1410593862","heartbeat_front_addr":"172.21.15.100:6823/1410593862","state":["exists","up"]},{"osd":7,"uuid":"02ada28f-ac4f-47cf-983f-8a6da1140abe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2458690455},{"type":"v1","addr":"172.21.15.178:6817","nonce":2458690455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2458690455},{"type":"v1","addr":"172.21.15.178:6819","nonce":2458690455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2458690455},{"type":"v1","addr":"172.21.15.178:6823","nonce":2458690455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2458690455},{"type":"v1","addr":"172.21.15.178:6821","nonce":2458690455}]},"public_addr":"172.21.15.178:6817/2458690455","cluster_addr":"172.21.15.178:6819/2458690455","heartbeat_back_addr":"172.21.15.178:6823/2458690455","heartbeat_front_addr":"172.21.15.178:6821/2458690455","state":["exists","up"]},{"osd":8,"uuid":"e7e489bc-15fc-4a4c-b297-d1bb626513ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1523872826},{"type":"v1","addr":"172.21.15.188:6817","nonce":1523872826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1523872826},{"type":"v1","addr":"172.21.15.188:6819","nonce":1523872826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1523872826},{"type":"v1","addr":"172.21.15.188:6823","nonce":1523872826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1523872826},{"type":"v1","addr":"172.21.15.188:6821","nonce":1523872826}]},"public_addr":"172.21.15.188:6817/1523872826","cluster_addr":"172.21.15.188:6819/1523872826","heartbeat_back_addr":"172.21.15.188:6823/1523872826","heartbeat_front_addr":"172.21.15.188:6821/1523872826","state":["exists","up"]},{"osd":9,"uuid":"72328326-cd16-45d9-bfd9-40f6c5c20f5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3585761978},{"type":"v1","addr":"172.21.15.100:6827","nonce":3585761978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3585761978},{"type":"v1","addr":"172.21.15.100:6829","nonce":3585761978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3585761978},{"type":"v1","addr":"172.21.15.100:6833","nonce":3585761978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3585761978},{"type":"v1","addr":"172.21.15.100:6831","nonce":3585761978}]},"public_addr":"172.21.15.100:6827/3585761978","cluster_addr":"172.21.15.100:6829/3585761978","heartbeat_back_addr":"172.21.15.100:6833/3585761978","heartbeat_front_addr":"172.21.15.100:6831/3585761978","state":["exists","up"]},{"osd":10,"uuid":"a355f9a8-91a3-41d4-814d-5a9c9b953706","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2985841725},{"type":"v1","addr":"172.21.15.178:6825","nonce":2985841725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2985841725},{"type":"v1","addr":"172.21.15.178:6827","nonce":2985841725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2985841725},{"type":"v1","addr":"172.21.15.178:6831","nonce":2985841725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2985841725},{"type":"v1","addr":"172.21.15.178:6829","nonce":2985841725}]},"public_addr":"172.21.15.178:6825/2985841725","cluster_addr":"172.21.15.178:6827/2985841725","heartbeat_back_addr":"172.21.15.178:6831/2985841725","heartbeat_front_addr":"172.21.15.178:6829/2985841725","state":["exists","up"]},{"osd":11,"uuid":"76a5e13c-7e38-42f9-bfa7-7fedec7c7184","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2574955118},{"type":"v1","addr":"172.21.15.188:6825","nonce":2574955118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2574955118},{"type":"v1","addr":"172.21.15.188:6827","nonce":2574955118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2574955118},{"type":"v1","addr":"172.21.15.188:6831","nonce":2574955118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2574955118},{"type":"v1","addr":"172.21.15.188:6829","nonce":2574955118}]},"public_addr":"172.21.15.188:6825/2574955118","cluster_addr":"172.21.15.188:6827/2574955118","heartbeat_back_addr":"172.21.15.188:6831/2574955118","heartbeat_front_addr":"172.21.15.188:6829/2574955118","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:08:36.086637+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:08:58.948031+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:09:22.142517+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:09:47.655748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:10:12.853999+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:10:37.911383+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:11:04.448815+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:11:30.331740+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:11:55.819278+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:12:23.204612+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:12:49.484835+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:13:15.783531+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:6800/1597433518":"2024-05-11T11:07:00.092732+0000","172.21.15.100:6800/1086450256":"2024-05-11T11:06:33.114176+0000","172.21.15.100:0/767964410":"2024-05-11T11:07:00.092732+0000","172.21.15.100:6801/1597433518":"2024-05-11T11:07:00.092732+0000","172.21.15.100:6801/1086450256":"2024-05-11T11:06:33.114176+0000","172.21.15.100:0/2770572070":"2024-05-11T11:06:33.114176+0000","172.21.15.100:0/1379340513":"2024-05-11T11:06:33.114176+0000","172.21.15.100:0/3025646953":"2024-05-11T11:07:00.092732+0000","172.21.15.100:0/4099258930":"2024-05-11T11:07:00.092732+0000","172.21.15.100:0/4267693567":"2024-05-11T11:06:33.114176+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T11:16:12.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:12.380+0000 7f3643d13700 1 -- 172.21.15.100:0/2201487087 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f36280b2910 msgr2=0x7f36280b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:12.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:12.380+0000 7f3643d13700 1 --2- 172.21.15.100:0/2201487087 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f36280b2910 0x7f36280b4dd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3638007be0 tx=0x7f36380073d0 comp rx=0 tx=0).stop 2024-05-10T11:16:12.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:12.380+0000 7f3643d13700 1 -- 172.21.15.100:0/2201487087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f363c102d90 msgr2=0x7f363c1a1e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:12.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:12.380+0000 7f3643d13700 1 --2- 172.21.15.100:0/2201487087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f363c102d90 0x7f363c1a1e60 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f362c00e9c0 tx=0x7f362c00ecd0 comp rx=0 tx=0).stop 2024-05-10T11:16:12.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:12.380+0000 7f3643d13700 1 -- 172.21.15.100:0/2201487087 shutdown_connections 2024-05-10T11:16:12.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:12.380+0000 7f3643d13700 1 --2- 172.21.15.100:0/2201487087 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f36280b2910 0x7f36280b4dd0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:12.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:12.380+0000 7f3643d13700 1 --2- 172.21.15.100:0/2201487087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f363c1042f0 0x7f363c19bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:12.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:12.380+0000 7f3643d13700 1 --2- 172.21.15.100:0/2201487087 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f363c1023d0 0x7f363c1a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:12.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:12.380+0000 7f3643d13700 1 --2- 172.21.15.100:0/2201487087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f363c102d90 0x7f363c1a1e60 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:12.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:12.380+0000 7f3643d13700 1 -- 172.21.15.100:0/2201487087 >> 172.21.15.100:0/2201487087 conn(0x7f363c078410 msgr2=0x7f363c100240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:16:12.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:12.380+0000 7f3643d13700 1 -- 172.21.15.100:0/2201487087 shutdown_connections 2024-05-10T11:16:12.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:12.380+0000 7f3643d13700 1 -- 172.21.15.100:0/2201487087 wait complete. 2024-05-10T11:16:13.073 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-05-10T11:16:13.073 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-10T11:16:13.179 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:16:12 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/2201487087' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T11:16:13.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:16:12 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/2201487087' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T11:16:13.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:16:12 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/2201487087' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T11:16:13.316 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:16:14.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:16:13 smithi100 ceph-mon[30296]: pgmap v355: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-10T11:16:14.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:16:13 smithi178 ceph-mon[33359]: pgmap v355: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-10T11:16:14.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:16:13 smithi188 ceph-mon[34075]: pgmap v355: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-10T11:16:14.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:14.976+0000 7f64a795f700 1 -- 172.21.15.100:0/796090635 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64a006be70 msgr2=0x7f64a006c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:14.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:14.976+0000 7f64a795f700 1 --2- 172.21.15.100:0/796090635 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64a006be70 0x7f64a006c250 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f649c0099c0 tx=0x7f649c009cd0 comp rx=0 tx=0).stop 2024-05-10T11:16:14.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:14.976+0000 7f64a795f700 1 -- 172.21.15.100:0/796090635 shutdown_connections 2024-05-10T11:16:14.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:14.976+0000 7f64a795f700 1 --2- 172.21.15.100:0/796090635 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64a01087c0 0x7f64a01103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:14.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:14.976+0000 7f64a795f700 1 --2- 172.21.15.100:0/796090635 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64a006be70 0x7f64a006c250 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:14.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:14.976+0000 7f64a795f700 1 --2- 172.21.15.100:0/796090635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64a006c820 0x7f64a01080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:14.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:14.976+0000 7f64a795f700 1 -- 172.21.15.100:0/796090635 >> 172.21.15.100:0/796090635 conn(0x7f64a00783f0 msgr2=0x7f64a0078800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:16:14.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:14.977+0000 7f64a795f700 1 -- 172.21.15.100:0/796090635 shutdown_connections 2024-05-10T11:16:14.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:14.977+0000 7f64a795f700 1 -- 172.21.15.100:0/796090635 wait complete. 2024-05-10T11:16:14.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:14.977+0000 7f64a795f700 1 Processor -- start 2024-05-10T11:16:14.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:14.977+0000 7f64a795f700 1 -- start start 2024-05-10T11:16:14.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:14.978+0000 7f64a795f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64a006be70 0x7f64a0102500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:14.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:14.978+0000 7f64a795f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64a006c820 0x7f64a0102a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:14.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:14.978+0000 7f64a795f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64a01087c0 0x7f64a0102f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:14.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:14.978+0000 7f64a795f700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f64a0113640 con 0x7f64a006be70 2024-05-10T11:16:14.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:14.978+0000 7f64a795f700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f64a01134c0 con 0x7f64a006c820 2024-05-10T11:16:14.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:14.978+0000 7f64a795f700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f64a01137c0 con 0x7f64a01087c0 2024-05-10T11:16:14.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:14.978+0000 7f64a56fb700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64a006be70 0x7f64a0102500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:14.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:14.978+0000 7f64a56fb700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64a006be70 0x7f64a0102500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59784/0 (socket says 172.21.15.100:59784) 2024-05-10T11:16:14.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:14.978+0000 7f64a56fb700 1 -- 172.21.15.100:0/2452525397 learned_addr learned my addr 172.21.15.100:0/2452525397 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:16:14.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:14.978+0000 7f64a4efa700 1 --2- 172.21.15.100:0/2452525397 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64a006c820 0x7f64a0102a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:14.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:14.978+0000 7f64a5efc700 1 --2- 172.21.15.100:0/2452525397 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64a01087c0 0x7f64a0102f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:14.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:14.978+0000 7f64a56fb700 1 -- 172.21.15.100:0/2452525397 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64a01087c0 msgr2=0x7f64a0102f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:14.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:14.978+0000 7f64a56fb700 1 --2- 172.21.15.100:0/2452525397 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64a01087c0 0x7f64a0102f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:14.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:14.978+0000 7f64a56fb700 1 -- 172.21.15.100:0/2452525397 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64a006c820 msgr2=0x7f64a0102a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:14.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:14.978+0000 7f64a56fb700 1 --2- 172.21.15.100:0/2452525397 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64a006c820 0x7f64a0102a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:14.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:14.978+0000 7f64a56fb700 1 -- 172.21.15.100:0/2452525397 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64a01a80e0 con 0x7f64a006be70 2024-05-10T11:16:14.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:14.978+0000 7f64a4efa700 1 --2- 172.21.15.100:0/2452525397 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64a006c820 0x7f64a0102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:16:14.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:14.978+0000 7f64a5efc700 1 --2- 172.21.15.100:0/2452525397 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64a01087c0 0x7f64a0102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:16:14.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:14.978+0000 7f64a56fb700 1 --2- 172.21.15.100:0/2452525397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64a006be70 0x7f64a0102500 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f649c000c00 tx=0x7f649c00f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:16:14.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:14.978+0000 7f64967fc700 1 -- 172.21.15.100:0/2452525397 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f649c026070 con 0x7f64a006be70 2024-05-10T11:16:14.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:14.978+0000 7f64a795f700 1 -- 172.21.15.100:0/2452525397 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64a01a8370 con 0x7f64a006be70 2024-05-10T11:16:14.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:14.978+0000 7f64967fc700 1 -- 172.21.15.100:0/2452525397 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f649c0054c0 con 0x7f64a006be70 2024-05-10T11:16:14.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:14.978+0000 7f64967fc700 1 -- 172.21.15.100:0/2452525397 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f649c004c20 con 0x7f64a006be70 2024-05-10T11:16:14.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:14.979+0000 7f64a795f700 1 -- 172.21.15.100:0/2452525397 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64a01a8820 con 0x7f64a006be70 2024-05-10T11:16:14.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:14.979+0000 7f64a795f700 1 -- 172.21.15.100:0/2452525397 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64a006a190 con 0x7f64a006be70 2024-05-10T11:16:14.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:14.982+0000 7f64967fc700 1 -- 172.21.15.100:0/2452525397 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f649c029070 con 0x7f64a006be70 2024-05-10T11:16:14.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:14.982+0000 7f64967fc700 1 --2- 172.21.15.100:0/2452525397 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f648c0b2910 0x7f648c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:14.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:14.982+0000 7f64a4efa700 1 --2- 172.21.15.100:0/2452525397 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f648c0b2910 0x7f648c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:14.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:14.982+0000 7f64967fc700 1 -- 172.21.15.100:0/2452525397 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f649c0e3660 con 0x7f64a006be70 2024-05-10T11:16:14.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:14.983+0000 7f64a4efa700 1 --2- 172.21.15.100:0/2452525397 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f648c0b2910 0x7f648c0b4dd0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f64a0102060 tx=0x7f6490008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:16:14.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:14.986+0000 7f64967fc700 1 -- 172.21.15.100:0/2452525397 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f649c030020 con 0x7f64a006be70 2024-05-10T11:16:15.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:15.190+0000 7f64a795f700 1 -- 172.21.15.100:0/2452525397 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f64a002e510 con 0x7f64a006be70 2024-05-10T11:16:15.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:15.192+0000 7f64967fc700 1 -- 172.21.15.100:0/2452525397 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24737 (secure 0 0 0) 0x7f649c013070 con 0x7f64a006be70 2024-05-10T11:16:15.193 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:16:15.194 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":89,"fsid":"20a542cc-0ebd-11ef-bc98-c7b262605968","created":"2024-05-10T11:06:05.454446+0000","modified":"2024-05-10T11:15:01.418144+0000","last_up_change":"2024-05-10T11:13:18.185471+0000","last_in_change":"2024-05-10T11:12:57.268678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T11:09:26.176441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T11:14:21.004285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T11:14:24.179414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T11:14:33.308517+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5f031418-da23-4e85-8669-160960dadd09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":643028278},{"type":"v1","addr":"172.21.15.100:6803","nonce":643028278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":643028278},{"type":"v1","addr":"172.21.15.100:6805","nonce":643028278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":643028278},{"type":"v1","addr":"172.21.15.100:6809","nonce":643028278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":643028278},{"type":"v1","addr":"172.21.15.100:6807","nonce":643028278}]},"public_addr":"172.21.15.100:6803/643028278","cluster_addr":"172.21.15.100:6805/643028278","heartbeat_back_addr":"172.21.15.100:6809/643028278","heartbeat_front_addr":"172.21.15.100:6807/643028278","state":["exists","up"]},{"osd":1,"uuid":"a908cdc9-001e-4b65-9ab0-0dd08dba20f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":3103987017},{"type":"v1","addr":"172.21.15.178:6801","nonce":3103987017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":3103987017},{"type":"v1","addr":"172.21.15.178:6803","nonce":3103987017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":3103987017},{"type":"v1","addr":"172.21.15.178:6807","nonce":3103987017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":3103987017},{"type":"v1","addr":"172.21.15.178:6805","nonce":3103987017}]},"public_addr":"172.21.15.178:6801/3103987017","cluster_addr":"172.21.15.178:6803/3103987017","heartbeat_back_addr":"172.21.15.178:6807/3103987017","heartbeat_front_addr":"172.21.15.178:6805/3103987017","state":["exists","up"]},{"osd":2,"uuid":"4d8f89ae-9424-46e9-9c80-17c47cc5e8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3991466468},{"type":"v1","addr":"172.21.15.188:6801","nonce":3991466468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3991466468},{"type":"v1","addr":"172.21.15.188:6803","nonce":3991466468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3991466468},{"type":"v1","addr":"172.21.15.188:6807","nonce":3991466468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3991466468},{"type":"v1","addr":"172.21.15.188:6805","nonce":3991466468}]},"public_addr":"172.21.15.188:6801/3991466468","cluster_addr":"172.21.15.188:6803/3991466468","heartbeat_back_addr":"172.21.15.188:6807/3991466468","heartbeat_front_addr":"172.21.15.188:6805/3991466468","state":["exists","up"]},{"osd":3,"uuid":"4c8865aa-fef4-4710-b766-1385adb9462c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1713357192},{"type":"v1","addr":"172.21.15.100:6811","nonce":1713357192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1713357192},{"type":"v1","addr":"172.21.15.100:6813","nonce":1713357192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1713357192},{"type":"v1","addr":"172.21.15.100:6817","nonce":1713357192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1713357192},{"type":"v1","addr":"172.21.15.100:6815","nonce":1713357192}]},"public_addr":"172.21.15.100:6811/1713357192","cluster_addr":"172.21.15.100:6813/1713357192","heartbeat_back_addr":"172.21.15.100:6817/1713357192","heartbeat_front_addr":"172.21.15.100:6815/1713357192","state":["exists","up"]},{"osd":4,"uuid":"a9653800-da57-43eb-8781-3188e8a3949f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":2222814192},{"type":"v1","addr":"172.21.15.178:6809","nonce":2222814192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":2222814192},{"type":"v1","addr":"172.21.15.178:6811","nonce":2222814192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":2222814192},{"type":"v1","addr":"172.21.15.178:6815","nonce":2222814192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":2222814192},{"type":"v1","addr":"172.21.15.178:6813","nonce":2222814192}]},"public_addr":"172.21.15.178:6809/2222814192","cluster_addr":"172.21.15.178:6811/2222814192","heartbeat_back_addr":"172.21.15.178:6815/2222814192","heartbeat_front_addr":"172.21.15.178:6813/2222814192","state":["exists","up"]},{"osd":5,"uuid":"fb5f8af2-1eb0-4d8a-a7a3-ed6dd68e692e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3517375935},{"type":"v1","addr":"172.21.15.188:6809","nonce":3517375935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3517375935},{"type":"v1","addr":"172.21.15.188:6811","nonce":3517375935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3517375935},{"type":"v1","addr":"172.21.15.188:6815","nonce":3517375935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3517375935},{"type":"v1","addr":"172.21.15.188:6813","nonce":3517375935}]},"public_addr":"172.21.15.188:6809/3517375935","cluster_addr":"172.21.15.188:6811/3517375935","heartbeat_back_addr":"172.21.15.188:6815/3517375935","heartbeat_front_addr":"172.21.15.188:6813/3517375935","state":["exists","up"]},{"osd":6,"uuid":"84b57387-0a94-4e49-b099-493d329b87dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1410593862},{"type":"v1","addr":"172.21.15.100:6819","nonce":1410593862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1410593862},{"type":"v1","addr":"172.21.15.100:6821","nonce":1410593862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1410593862},{"type":"v1","addr":"172.21.15.100:6825","nonce":1410593862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1410593862},{"type":"v1","addr":"172.21.15.100:6823","nonce":1410593862}]},"public_addr":"172.21.15.100:6819/1410593862","cluster_addr":"172.21.15.100:6821/1410593862","heartbeat_back_addr":"172.21.15.100:6825/1410593862","heartbeat_front_addr":"172.21.15.100:6823/1410593862","state":["exists","up"]},{"osd":7,"uuid":"02ada28f-ac4f-47cf-983f-8a6da1140abe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2458690455},{"type":"v1","addr":"172.21.15.178:6817","nonce":2458690455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2458690455},{"type":"v1","addr":"172.21.15.178:6819","nonce":2458690455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2458690455},{"type":"v1","addr":"172.21.15.178:6823","nonce":2458690455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2458690455},{"type":"v1","addr":"172.21.15.178:6821","nonce":2458690455}]},"public_addr":"172.21.15.178:6817/2458690455","cluster_addr":"172.21.15.178:6819/2458690455","heartbeat_back_addr":"172.21.15.178:6823/2458690455","heartbeat_front_addr":"172.21.15.178:6821/2458690455","state":["exists","up"]},{"osd":8,"uuid":"e7e489bc-15fc-4a4c-b297-d1bb626513ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1523872826},{"type":"v1","addr":"172.21.15.188:6817","nonce":1523872826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1523872826},{"type":"v1","addr":"172.21.15.188:6819","nonce":1523872826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1523872826},{"type":"v1","addr":"172.21.15.188:6823","nonce":1523872826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1523872826},{"type":"v1","addr":"172.21.15.188:6821","nonce":1523872826}]},"public_addr":"172.21.15.188:6817/1523872826","cluster_addr":"172.21.15.188:6819/1523872826","heartbeat_back_addr":"172.21.15.188:6823/1523872826","heartbeat_front_addr":"172.21.15.188:6821/1523872826","state":["exists","up"]},{"osd":9,"uuid":"72328326-cd16-45d9-bfd9-40f6c5c20f5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3585761978},{"type":"v1","addr":"172.21.15.100:6827","nonce":3585761978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3585761978},{"type":"v1","addr":"172.21.15.100:6829","nonce":3585761978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3585761978},{"type":"v1","addr":"172.21.15.100:6833","nonce":3585761978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3585761978},{"type":"v1","addr":"172.21.15.100:6831","nonce":3585761978}]},"public_addr":"172.21.15.100:6827/3585761978","cluster_addr":"172.21.15.100:6829/3585761978","heartbeat_back_addr":"172.21.15.100:6833/3585761978","heartbeat_front_addr":"172.21.15.100:6831/3585761978","state":["exists","up"]},{"osd":10,"uuid":"a355f9a8-91a3-41d4-814d-5a9c9b953706","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2985841725},{"type":"v1","addr":"172.21.15.178:6825","nonce":2985841725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2985841725},{"type":"v1","addr":"172.21.15.178:6827","nonce":2985841725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2985841725},{"type":"v1","addr":"172.21.15.178:6831","nonce":2985841725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2985841725},{"type":"v1","addr":"172.21.15.178:6829","nonce":2985841725}]},"public_addr":"172.21.15.178:6825/2985841725","cluster_addr":"172.21.15.178:6827/2985841725","heartbeat_back_addr":"172.21.15.178:6831/2985841725","heartbeat_front_addr":"172.21.15.178:6829/2985841725","state":["exists","up"]},{"osd":11,"uuid":"76a5e13c-7e38-42f9-bfa7-7fedec7c7184","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2574955118},{"type":"v1","addr":"172.21.15.188:6825","nonce":2574955118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2574955118},{"type":"v1","addr":"172.21.15.188:6827","nonce":2574955118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2574955118},{"type":"v1","addr":"172.21.15.188:6831","nonce":2574955118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2574955118},{"type":"v1","addr":"172.21.15.188:6829","nonce":2574955118}]},"public_addr":"172.21.15.188:6825/2574955118","cluster_addr":"172.21.15.188:6827/2574955118","heartbeat_back_addr":"172.21.15.188:6831/2574955118","heartbeat_front_addr":"172.21.15.188:6829/2574955118","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:08:36.086637+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:08:58.948031+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:09:22.142517+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:09:47.655748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:10:12.853999+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:10:37.911383+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:11:04.448815+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:11:30.331740+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:11:55.819278+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:12:23.204612+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:12:49.484835+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:13:15.783531+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:6800/1597433518":"2024-05-11T11:07:00.092732+0000","172.21.15.100:6800/1086450256":"2024-05-11T11:06:33.114176+0000","172.21.15.100:0/767964410":"2024-05-11T11:07:00.092732+0000","172.21.15.100:6801/1597433518":"2024-05-11T11:07:00.092732+0000","172.21.15.100:6801/1086450256":"2024-05-11T11:06:33.114176+0000","172.21.15.100:0/2770572070":"2024-05-11T11:06:33.114176+0000","172.21.15.100:0/1379340513":"2024-05-11T11:06:33.114176+0000","172.21.15.100:0/3025646953":"2024-05-11T11:07:00.092732+0000","172.21.15.100:0/4099258930":"2024-05-11T11:07:00.092732+0000","172.21.15.100:0/4267693567":"2024-05-11T11:06:33.114176+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T11:16:15.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:15.194+0000 7f64a795f700 1 -- 172.21.15.100:0/2452525397 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f648c0b2910 msgr2=0x7f648c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:15.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:15.194+0000 7f64a795f700 1 --2- 172.21.15.100:0/2452525397 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f648c0b2910 0x7f648c0b4dd0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f64a0102060 tx=0x7f6490008040 comp rx=0 tx=0).stop 2024-05-10T11:16:15.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:15.194+0000 7f64a795f700 1 -- 172.21.15.100:0/2452525397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64a006be70 msgr2=0x7f64a0102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:15.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:15.194+0000 7f64a795f700 1 --2- 172.21.15.100:0/2452525397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64a006be70 0x7f64a0102500 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f649c000c00 tx=0x7f649c00f740 comp rx=0 tx=0).stop 2024-05-10T11:16:15.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:15.195+0000 7f64a795f700 1 -- 172.21.15.100:0/2452525397 shutdown_connections 2024-05-10T11:16:15.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:15.195+0000 7f64a795f700 1 --2- 172.21.15.100:0/2452525397 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f648c0b2910 0x7f648c0b4dd0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:15.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:15.195+0000 7f64a795f700 1 --2- 172.21.15.100:0/2452525397 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64a01087c0 0x7f64a0102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:15.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:15.195+0000 7f64a795f700 1 --2- 172.21.15.100:0/2452525397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64a006be70 0x7f64a0102500 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:15.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:15.195+0000 7f64a795f700 1 --2- 172.21.15.100:0/2452525397 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64a006c820 0x7f64a0102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:15.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:15.195+0000 7f64a795f700 1 -- 172.21.15.100:0/2452525397 >> 172.21.15.100:0/2452525397 conn(0x7f64a00783f0 msgr2=0x7f64a0100760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:16:15.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:15.195+0000 7f64a795f700 1 -- 172.21.15.100:0/2452525397 shutdown_connections 2024-05-10T11:16:15.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:15.195+0000 7f64a795f700 1 -- 172.21.15.100:0/2452525397 wait complete. 2024-05-10T11:16:15.919 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-05-10T11:16:15.920 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-05-10T11:16:15.920 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-05-10T11:16:15.921 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-05-10T11:16:15.921 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-05-10T11:16:15.922 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-05-10T11:16:15.922 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-05-10T11:16:15.922 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-05-10T11:16:15.923 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-05-10T11:16:15.924 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-05-10T11:16:15.924 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-05-10T11:16:15.924 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-05-10T11:16:16.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:16:15 smithi178 ceph-mon[33359]: pgmap v356: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-10T11:16:16.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:16:15 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/2452525397' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T11:16:16.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:16:15 smithi188 ceph-mon[34075]: pgmap v356: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-10T11:16:16.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:16:15 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/2452525397' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T11:16:16.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:16:15 smithi100 ceph-mon[30296]: pgmap v356: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-10T11:16:16.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:16:15 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/2452525397' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T11:16:16.567 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:16:16.587 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:16:18.118 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:16:17 smithi100 ceph-mon[30296]: pgmap v357: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T11:16:18.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:16:17 smithi178 ceph-mon[33359]: pgmap v357: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T11:16:18.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:16:17 smithi188 ceph-mon[34075]: pgmap v357: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T11:16:18.462 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:16:18.896 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:16:18.896 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:16:18.898 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:16:18.899 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:16:18.899 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:16:18.900 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:16:18.902 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:16:20.168 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:16:19 smithi100 ceph-mon[30296]: pgmap v358: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T11:16:20.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:16:19 smithi178 ceph-mon[33359]: pgmap v358: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T11:16:20.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:16:19 smithi188 ceph-mon[34075]: pgmap v358: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T11:16:22.167 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:16:21 smithi100 ceph-mon[30296]: pgmap v359: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T11:16:22.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:16:21 smithi178 ceph-mon[33359]: pgmap v359: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T11:16:22.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:16:21 smithi188 ceph-mon[34075]: pgmap v359: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T11:16:24.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:16:23 smithi178 ceph-mon[33359]: pgmap v360: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T11:16:24.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:16:23 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:16:24.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:16:23 smithi188 ceph-mon[34075]: pgmap v360: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T11:16:24.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:16:23 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:16:24.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:16:23 smithi100 ceph-mon[30296]: pgmap v360: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T11:16:24.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:16:23 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:16:25.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.616+0000 7fe8b140e700 1 -- 172.21.15.100:0/1710079481 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a8003b80 con 0x7fe8ac102630 2024-05-10T11:16:25.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.617+0000 7fe8b4674700 1 -- 172.21.15.100:0/1710079481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8ac102630 msgr2=0x7fe8ac102a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:25.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.617+0000 7fe8b4674700 1 --2- 172.21.15.100:0/1710079481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8ac102630 0x7fe8ac102a10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe8a80099c0 tx=0x7fe8a8009cd0 comp rx=0 tx=0).stop 2024-05-10T11:16:25.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.617+0000 7fe8b4674700 1 -- 172.21.15.100:0/1710079481 shutdown_connections 2024-05-10T11:16:25.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.617+0000 7fe8b4674700 1 --2- 172.21.15.100:0/1710079481 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8ac1076c0 0x7fe8ac10f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:25.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.617+0000 7fe8b4674700 1 --2- 172.21.15.100:0/1710079481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8ac102630 0x7fe8ac102a10 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:25.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.617+0000 7fe8b4674700 1 --2- 172.21.15.100:0/1710079481 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8ac102fe0 0x7fe8ac106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:25.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.617+0000 7fe8b4674700 1 -- 172.21.15.100:0/1710079481 >> 172.21.15.100:0/1710079481 conn(0x7fe8ac079350 msgr2=0x7fe8ac079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:16:25.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.617+0000 7fe8b4674700 1 -- 172.21.15.100:0/1710079481 shutdown_connections 2024-05-10T11:16:25.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.617+0000 7fe8b4674700 1 -- 172.21.15.100:0/1710079481 wait complete. 2024-05-10T11:16:25.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.618+0000 7fe8b4674700 1 Processor -- start 2024-05-10T11:16:25.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.618+0000 7fe8b4674700 1 -- start start 2024-05-10T11:16:25.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.618+0000 7fe8b4674700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8ac102630 0x7fe8ac104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:25.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.618+0000 7fe8b4674700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8ac102fe0 0x7fe8ac1045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:25.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.618+0000 7fe8b4674700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8ac1076c0 0x7fe8ac104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:25.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.618+0000 7fe8b4674700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe8ac112420 con 0x7fe8ac102630 2024-05-10T11:16:25.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.618+0000 7fe8b4674700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe8ac1122a0 con 0x7fe8ac1076c0 2024-05-10T11:16:25.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.618+0000 7fe8b4674700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe8ac1125a0 con 0x7fe8ac102fe0 2024-05-10T11:16:25.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.618+0000 7fe8b2410700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8ac102630 0x7fe8ac104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:25.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.619+0000 7fe8b2410700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8ac102630 0x7fe8ac104080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56104/0 (socket says 172.21.15.100:56104) 2024-05-10T11:16:25.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.619+0000 7fe8b2410700 1 -- 172.21.15.100:0/1950906565 learned_addr learned my addr 172.21.15.100:0/1950906565 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:16:25.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.619+0000 7fe8b1c0f700 1 --2- 172.21.15.100:0/1950906565 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8ac102fe0 0x7fe8ac1045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:25.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.619+0000 7fe8b2c11700 1 --2- 172.21.15.100:0/1950906565 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8ac1076c0 0x7fe8ac104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:25.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.619+0000 7fe8b2410700 1 -- 172.21.15.100:0/1950906565 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8ac102fe0 msgr2=0x7fe8ac1045c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:25.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.619+0000 7fe8b2410700 1 --2- 172.21.15.100:0/1950906565 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8ac102fe0 0x7fe8ac1045c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:25.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.619+0000 7fe8b2410700 1 -- 172.21.15.100:0/1950906565 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8ac1076c0 msgr2=0x7fe8ac104b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:25.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.619+0000 7fe8b2410700 1 --2- 172.21.15.100:0/1950906565 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8ac1076c0 0x7fe8ac104b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:25.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.619+0000 7fe8b2410700 1 -- 172.21.15.100:0/1950906565 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8ac1a8070 con 0x7fe8ac102630 2024-05-10T11:16:25.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.619+0000 7fe8b1c0f700 1 --2- 172.21.15.100:0/1950906565 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8ac102fe0 0x7fe8ac1045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:16:25.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.619+0000 7fe8b2410700 1 --2- 172.21.15.100:0/1950906565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8ac102630 0x7fe8ac104080 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8006010 tx=0x7fe8a80045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:16:25.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.619+0000 7fe8a37fe700 1 -- 172.21.15.100:0/1950906565 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a8026070 con 0x7fe8ac102630 2024-05-10T11:16:25.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.619+0000 7fe8a37fe700 1 -- 172.21.15.100:0/1950906565 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe8a800f460 con 0x7fe8ac102630 2024-05-10T11:16:25.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.619+0000 7fe8b4674700 1 -- 172.21.15.100:0/1950906565 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8ac1a8300 con 0x7fe8ac102630 2024-05-10T11:16:25.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.619+0000 7fe8a37fe700 1 -- 172.21.15.100:0/1950906565 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a8009e60 con 0x7fe8ac102630 2024-05-10T11:16:25.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.619+0000 7fe8b4674700 1 -- 172.21.15.100:0/1950906565 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8ac1a8750 con 0x7fe8ac102630 2024-05-10T11:16:25.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.620+0000 7fe8b4674700 1 -- 172.21.15.100:0/1950906565 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe8ac1016f0 con 0x7fe8ac102630 2024-05-10T11:16:25.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.622+0000 7fe8a37fe700 1 -- 172.21.15.100:0/1950906565 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fe8a8029070 con 0x7fe8ac102630 2024-05-10T11:16:25.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.623+0000 7fe8a37fe700 1 --2- 172.21.15.100:0/1950906565 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe8980b2910 0x7fe8980b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:25.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.623+0000 7fe8b1c0f700 1 --2- 172.21.15.100:0/1950906565 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe8980b2910 0x7fe8980b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:25.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.623+0000 7fe8a37fe700 1 -- 172.21.15.100:0/1950906565 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fe8a80e38c0 con 0x7fe8ac102630 2024-05-10T11:16:25.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.623+0000 7fe8a37fe700 1 --2- 172.21.15.100:0/1950906565 >> [v2:172.21.15.188:6824/2574955118,v1:172.21.15.188:6825/2574955118] conn(0x7fe8980bda70 0x7fe8980bff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:25.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.623+0000 7fe8a37fe700 1 -- 172.21.15.100:0/1950906565 --> [v2:172.21.15.188:6824/2574955118,v1:172.21.15.188:6825/2574955118] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe8980c05e0 con 0x7fe8980bda70 2024-05-10T11:16:25.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.623+0000 7fe8a37fe700 1 -- 172.21.15.100:0/1950906565 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fe8a80e3d10 con 0x7fe8ac102630 2024-05-10T11:16:25.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.623+0000 7fe8b1c0f700 1 --2- 172.21.15.100:0/1950906565 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe8980b2910 0x7fe8980b4dd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac1071c0 tx=0x7fe89c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:16:25.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.623+0000 7fe8b2c11700 1 --2- 172.21.15.100:0/1950906565 >> [v2:172.21.15.188:6824/2574955118,v1:172.21.15.188:6825/2574955118] conn(0x7fe8980bda70 0x7fe8980bff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:25.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.624+0000 7fe8b2c11700 1 --2- 172.21.15.100:0/1950906565 >> [v2:172.21.15.188:6824/2574955118,v1:172.21.15.188:6825/2574955118] conn(0x7fe8980bda70 0x7fe8980bff10 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:16:25.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.624+0000 7fe8a37fe700 1 -- 172.21.15.100:0/1950906565 <== osd.11 v2:172.21.15.188:6824/2574955118 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe8980c05e0 con 0x7fe8980bda70 2024-05-10T11:16:25.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.654+0000 7fe8b4674700 1 -- 172.21.15.100:0/1950906565 --> [v2:172.21.15.188:6824/2574955118,v1:172.21.15.188:6825/2574955118] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe8ac103ac0 con 0x7fe8980bda70 2024-05-10T11:16:25.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.654+0000 7fe8a37fe700 1 -- 172.21.15.100:0/1950906565 <== osd.11 v2:172.21.15.188:6824/2574955118 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe8ac103ac0 con 0x7fe8980bda70 2024-05-10T11:16:25.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.654+0000 7fe8b4674700 1 -- 172.21.15.100:0/1950906565 >> [v2:172.21.15.188:6824/2574955118,v1:172.21.15.188:6825/2574955118] conn(0x7fe8980bda70 msgr2=0x7fe8980bff10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:25.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.654+0000 7fe8b4674700 1 --2- 172.21.15.100:0/1950906565 >> [v2:172.21.15.188:6824/2574955118,v1:172.21.15.188:6825/2574955118] conn(0x7fe8980bda70 0x7fe8980bff10 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:25.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.655+0000 7fe8b4674700 1 -- 172.21.15.100:0/1950906565 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe8980b2910 msgr2=0x7fe8980b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:25.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.655+0000 7fe8b4674700 1 --2- 172.21.15.100:0/1950906565 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe8980b2910 0x7fe8980b4dd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac1071c0 tx=0x7fe89c009260 comp rx=0 tx=0).stop 2024-05-10T11:16:25.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.655+0000 7fe8b4674700 1 -- 172.21.15.100:0/1950906565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8ac102630 msgr2=0x7fe8ac104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:25.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.655+0000 7fe8b4674700 1 --2- 172.21.15.100:0/1950906565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8ac102630 0x7fe8ac104080 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8006010 tx=0x7fe8a80045e0 comp rx=0 tx=0).stop 2024-05-10T11:16:25.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.655+0000 7fe8b2c11700 1 -- 172.21.15.100:0/1950906565 reap_dead start 2024-05-10T11:16:25.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.655+0000 7fe8b4674700 1 -- 172.21.15.100:0/1950906565 shutdown_connections 2024-05-10T11:16:25.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.655+0000 7fe8b4674700 1 -- 172.21.15.100:0/1950906565 >> 172.21.15.100:0/1950906565 conn(0x7fe8ac079350 msgr2=0x7fe8ac06bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:16:25.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.655+0000 7fe8b4674700 1 -- 172.21.15.100:0/1950906565 shutdown_connections 2024-05-10T11:16:25.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.655+0000 7fe8b4674700 1 -- 172.21.15.100:0/1950906565 wait complete. 2024-05-10T11:16:25.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.384+0000 7fddc0e53700 1 -- 172.21.15.100:0/3467280862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fddbc101a90 msgr2=0x7fddbc101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:25.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.384+0000 7fddc0e53700 1 --2- 172.21.15.100:0/3467280862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fddbc101a90 0x7fddbc101f10 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fddb0009990 tx=0x7fddb0009ca0 comp rx=0 tx=0).stop 2024-05-10T11:16:25.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.384+0000 7fddc0e53700 1 -- 172.21.15.100:0/3467280862 shutdown_connections 2024-05-10T11:16:25.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.384+0000 7fddc0e53700 1 --2- 172.21.15.100:0/3467280862 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fddbc102450 0x7fddbc110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:25.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.384+0000 7fddc0e53700 1 --2- 172.21.15.100:0/3467280862 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fddbc109520 0x7fddbc109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:25.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.384+0000 7fddc0e53700 1 --2- 172.21.15.100:0/3467280862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fddbc101a90 0x7fddbc101f10 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:25.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.384+0000 7fddc0e53700 1 -- 172.21.15.100:0/3467280862 >> 172.21.15.100:0/3467280862 conn(0x7fddbc078a10 msgr2=0x7fddbc0ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:16:25.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.384+0000 7fddc0e53700 1 -- 172.21.15.100:0/3467280862 shutdown_connections 2024-05-10T11:16:25.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.384+0000 7fddc0e53700 1 -- 172.21.15.100:0/3467280862 wait complete. 2024-05-10T11:16:25.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.385+0000 7fddc0e53700 1 Processor -- start 2024-05-10T11:16:25.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.385+0000 7fddc0e53700 1 -- start start 2024-05-10T11:16:25.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.385+0000 7fddc0e53700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fddbc101a90 0x7fddbc19f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:25.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.385+0000 7fddc0e53700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fddbc102450 0x7fddbc19fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:25.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.385+0000 7fddc0e53700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fddbc109520 0x7fddbc199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:25.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.385+0000 7fddc0e53700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fddbc113500 con 0x7fddbc102450 2024-05-10T11:16:25.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.385+0000 7fddc0e53700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fddbc113380 con 0x7fddbc101a90 2024-05-10T11:16:25.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.385+0000 7fddc0e53700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fddbc113680 con 0x7fddbc109520 2024-05-10T11:16:25.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.385+0000 7fddb9d9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fddbc102450 0x7fddbc19fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:25.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.385+0000 7fddb9d9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fddbc102450 0x7fddbc19fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56078/0 (socket says 172.21.15.100:56078) 2024-05-10T11:16:25.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.385+0000 7fddb9d9b700 1 -- 172.21.15.100:0/1300014819 learned_addr learned my addr 172.21.15.100:0/1300014819 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:16:25.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.385+0000 7fddba59c700 1 --2- 172.21.15.100:0/1300014819 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fddbc101a90 0x7fddbc19f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:25.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.385+0000 7fddbad9d700 1 --2- 172.21.15.100:0/1300014819 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fddbc109520 0x7fddbc199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:25.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.386+0000 7fddb9d9b700 1 -- 172.21.15.100:0/1300014819 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fddbc109520 msgr2=0x7fddbc199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:25.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.386+0000 7fddb9d9b700 1 --2- 172.21.15.100:0/1300014819 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fddbc109520 0x7fddbc199840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:25.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.386+0000 7fddb9d9b700 1 -- 172.21.15.100:0/1300014819 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fddbc101a90 msgr2=0x7fddbc19f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:25.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.386+0000 7fddb9d9b700 1 --2- 172.21.15.100:0/1300014819 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fddbc101a90 0x7fddbc19f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:25.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.386+0000 7fddb9d9b700 1 -- 172.21.15.100:0/1300014819 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddbc19a020 con 0x7fddbc102450 2024-05-10T11:16:25.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.386+0000 7fddb9d9b700 1 --2- 172.21.15.100:0/1300014819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fddbc102450 0x7fddbc19fc00 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fddb0006010 tx=0x7fddb00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:16:25.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.386+0000 7fddab7fe700 1 -- 172.21.15.100:0/1300014819 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb0027070 con 0x7fddbc102450 2024-05-10T11:16:25.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.386+0000 7fddab7fe700 1 -- 172.21.15.100:0/1300014819 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fddb0022470 con 0x7fddbc102450 2024-05-10T11:16:25.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.386+0000 7fddc0e53700 1 -- 172.21.15.100:0/1300014819 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddbc19a2b0 con 0x7fddbc102450 2024-05-10T11:16:25.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.386+0000 7fddab7fe700 1 -- 172.21.15.100:0/1300014819 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb0017650 con 0x7fddbc102450 2024-05-10T11:16:25.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.386+0000 7fddc0e53700 1 -- 172.21.15.100:0/1300014819 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddbc1a6520 con 0x7fddbc102450 2024-05-10T11:16:25.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.387+0000 7fddc0e53700 1 -- 172.21.15.100:0/1300014819 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fddbc063c30 con 0x7fddbc102450 2024-05-10T11:16:25.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.388+0000 7fddab7fe700 1 -- 172.21.15.100:0/1300014819 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fddb0009d10 con 0x7fddbc102450 2024-05-10T11:16:25.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.389+0000 7fddab7fe700 1 --2- 172.21.15.100:0/1300014819 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fdda40b2910 0x7fdda40b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:25.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.389+0000 7fddba59c700 1 --2- 172.21.15.100:0/1300014819 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fdda40b2910 0x7fdda40b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:25.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.389+0000 7fddab7fe700 1 -- 172.21.15.100:0/1300014819 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fddb00e3720 con 0x7fddbc102450 2024-05-10T11:16:25.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.389+0000 7fddab7fe700 1 --2- 172.21.15.100:0/1300014819 >> [v2:172.21.15.178:6824/2985841725,v1:172.21.15.178:6825/2985841725] conn(0x7fdda40bda70 0x7fdda40bff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:25.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.389+0000 7fddab7fe700 1 -- 172.21.15.100:0/1300014819 --> [v2:172.21.15.178:6824/2985841725,v1:172.21.15.178:6825/2985841725] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fddb00095d0 con 0x7fdda40bda70 2024-05-10T11:16:25.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.389+0000 7fddab7fe700 1 -- 172.21.15.100:0/1300014819 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fddb00e3b70 con 0x7fddbc102450 2024-05-10T11:16:25.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.390+0000 7fddba59c700 1 --2- 172.21.15.100:0/1300014819 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fdda40b2910 0x7fdda40b4dd0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fddac009be0 tx=0x7fddac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:16:25.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.390+0000 7fddbad9d700 1 --2- 172.21.15.100:0/1300014819 >> [v2:172.21.15.178:6824/2985841725,v1:172.21.15.178:6825/2985841725] conn(0x7fdda40bda70 0x7fdda40bff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:25.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.390+0000 7fddbad9d700 1 --2- 172.21.15.100:0/1300014819 >> [v2:172.21.15.178:6824/2985841725,v1:172.21.15.178:6825/2985841725] conn(0x7fdda40bda70 0x7fdda40bff10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:16:25.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.391+0000 7fddab7fe700 1 -- 172.21.15.100:0/1300014819 <== osd.10 v2:172.21.15.178:6824/2985841725 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fddb00095d0 con 0x7fdda40bda70 2024-05-10T11:16:25.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.408+0000 7fddc0e53700 1 -- 172.21.15.100:0/1300014819 --> [v2:172.21.15.178:6824/2985841725,v1:172.21.15.178:6825/2985841725] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fddbc0640b0 con 0x7fdda40bda70 2024-05-10T11:16:25.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.408+0000 7fddab7fe700 1 -- 172.21.15.100:0/1300014819 <== osd.10 v2:172.21.15.178:6824/2985841725 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fddbc0640b0 con 0x7fdda40bda70 2024-05-10T11:16:25.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.408+0000 7fddc0e53700 1 -- 172.21.15.100:0/1300014819 >> [v2:172.21.15.178:6824/2985841725,v1:172.21.15.178:6825/2985841725] conn(0x7fdda40bda70 msgr2=0x7fdda40bff10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:25.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.408+0000 7fddc0e53700 1 --2- 172.21.15.100:0/1300014819 >> [v2:172.21.15.178:6824/2985841725,v1:172.21.15.178:6825/2985841725] conn(0x7fdda40bda70 0x7fdda40bff10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:25.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.409+0000 7fddc0e53700 1 -- 172.21.15.100:0/1300014819 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fdda40b2910 msgr2=0x7fdda40b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:25.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.409+0000 7fddc0e53700 1 --2- 172.21.15.100:0/1300014819 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fdda40b2910 0x7fdda40b4dd0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fddac009be0 tx=0x7fddac009260 comp rx=0 tx=0).stop 2024-05-10T11:16:25.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.409+0000 7fddc0e53700 1 -- 172.21.15.100:0/1300014819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fddbc102450 msgr2=0x7fddbc19fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:25.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.409+0000 7fddc0e53700 1 --2- 172.21.15.100:0/1300014819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fddbc102450 0x7fddbc19fc00 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fddb0006010 tx=0x7fddb00047b0 comp rx=0 tx=0).stop 2024-05-10T11:16:25.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.409+0000 7fddbad9d700 1 -- 172.21.15.100:0/1300014819 reap_dead start 2024-05-10T11:16:25.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.409+0000 7fddc0e53700 1 -- 172.21.15.100:0/1300014819 shutdown_connections 2024-05-10T11:16:25.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.409+0000 7fddc0e53700 1 -- 172.21.15.100:0/1300014819 >> 172.21.15.100:0/1300014819 conn(0x7fddbc078a10 msgr2=0x7fddbc100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:16:25.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.409+0000 7fddc0e53700 1 -- 172.21.15.100:0/1300014819 shutdown_connections 2024-05-10T11:16:25.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:25.409+0000 7fddc0e53700 1 -- 172.21.15.100:0/1300014819 wait complete. 2024-05-10T11:16:26.211 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:16:25 smithi100 ceph-mon[30296]: pgmap v361: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-10T11:16:26.212 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:16:25 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:16:26.212 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:16:25 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:16:26.212 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:16:25 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:16:26.212 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:16:25 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:16:26.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:16:25 smithi178 ceph-mon[33359]: pgmap v361: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-10T11:16:26.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:16:25 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:16:26.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:16:25 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:16:26.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:16:25 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:16:26.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:16:25 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:16:26.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:16:25 smithi188 ceph-mon[34075]: pgmap v361: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-10T11:16:26.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:16:25 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:16:26.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:16:25 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:16:26.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:16:25 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:16:26.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:16:25 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:16:26.875 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:16:26.876 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:16:28.194 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:16:27 smithi100 ceph-mon[30296]: pgmap v362: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-10T11:16:28.195 INFO:teuthology.orchestra.run.smithi100.stdout:309237645353 2024-05-10T11:16:28.196 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-05-10T11:16:28.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:16:27 smithi178 ceph-mon[33359]: pgmap v362: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-10T11:16:28.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:16:27 smithi188 ceph-mon[34075]: pgmap v362: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-10T11:16:28.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:28.939+0000 7f46fc969700 1 -- 172.21.15.100:0/3108954527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46f8102d50 msgr2=0x7f46f810f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:28.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:28.939+0000 7f46fc969700 1 --2- 172.21.15.100:0/3108954527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46f8102d50 0x7f46f810f240 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f46ec009990 tx=0x7f46ec009ca0 comp rx=0 tx=0).stop 2024-05-10T11:16:28.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:28.939+0000 7f46fc969700 1 -- 172.21.15.100:0/3108954527 shutdown_connections 2024-05-10T11:16:28.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:28.939+0000 7f46fc969700 1 --2- 172.21.15.100:0/3108954527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46f8102d50 0x7f46f810f240 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:28.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:28.939+0000 7f46fc969700 1 --2- 172.21.15.100:0/3108954527 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46f8108400 0x7f46f81087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:28.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:28.939+0000 7f46fc969700 1 --2- 172.21.15.100:0/3108954527 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46f8102390 0x7f46f8102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:28.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:28.939+0000 7f46fc969700 1 -- 172.21.15.100:0/3108954527 >> 172.21.15.100:0/3108954527 conn(0x7f46f8079310 msgr2=0x7f46f8079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:16:28.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:28.939+0000 7f46fc969700 1 -- 172.21.15.100:0/3108954527 shutdown_connections 2024-05-10T11:16:28.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:28.939+0000 7f46fc969700 1 -- 172.21.15.100:0/3108954527 wait complete. 2024-05-10T11:16:28.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:28.940+0000 7f46fc969700 1 Processor -- start 2024-05-10T11:16:28.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:28.940+0000 7f46fc969700 1 -- start start 2024-05-10T11:16:28.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:28.940+0000 7f46fc969700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46f8102390 0x7f46f819a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:28.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:28.940+0000 7f46fc969700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46f8102d50 0x7f46f819ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:28.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:28.940+0000 7f46fc969700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46f8108400 0x7f46f81a59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:28.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:28.940+0000 7f46fc969700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f46f8112450 con 0x7f46f8102390 2024-05-10T11:16:28.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:28.940+0000 7f46fc969700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f46f81122d0 con 0x7f46f8102d50 2024-05-10T11:16:28.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:28.940+0000 7f46fc969700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f46f81125d0 con 0x7f46f8108400 2024-05-10T11:16:28.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:28.940+0000 7f46f659c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46f8102390 0x7f46f819a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:28.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:28.940+0000 7f46f659c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46f8102390 0x7f46f819a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59902/0 (socket says 172.21.15.100:59902) 2024-05-10T11:16:28.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:28.940+0000 7f46f659c700 1 -- 172.21.15.100:0/3762141356 learned_addr learned my addr 172.21.15.100:0/3762141356 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:16:28.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:28.940+0000 7f46f5d9b700 1 --2- 172.21.15.100:0/3762141356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46f8102d50 0x7f46f819ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:28.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:28.940+0000 7f46f6d9d700 1 --2- 172.21.15.100:0/3762141356 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46f8108400 0x7f46f81a59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:28.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:28.941+0000 7f46f5d9b700 1 -- 172.21.15.100:0/3762141356 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46f8108400 msgr2=0x7f46f81a59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:28.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:28.941+0000 7f46f5d9b700 1 --2- 172.21.15.100:0/3762141356 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46f8108400 0x7f46f81a59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:28.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:28.941+0000 7f46f5d9b700 1 -- 172.21.15.100:0/3762141356 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46f8102390 msgr2=0x7f46f819a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:28.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:28.941+0000 7f46f5d9b700 1 --2- 172.21.15.100:0/3762141356 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46f8102390 0x7f46f819a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:28.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:28.941+0000 7f46f5d9b700 1 -- 172.21.15.100:0/3762141356 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46f81a5ef0 con 0x7f46f8102d50 2024-05-10T11:16:28.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:28.941+0000 7f46f6d9d700 1 --2- 172.21.15.100:0/3762141356 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46f8108400 0x7f46f81a59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:16:28.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:28.941+0000 7f46f659c700 1 --2- 172.21.15.100:0/3762141356 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46f8102390 0x7f46f819a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:16:28.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:28.941+0000 7f46f5d9b700 1 --2- 172.21.15.100:0/3762141356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46f8102d50 0x7f46f819ae70 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f46f000ef10 tx=0x7f46f000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:16:28.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:28.941+0000 7f46e77fe700 1 -- 172.21.15.100:0/3762141356 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f000ca70 con 0x7f46f8102d50 2024-05-10T11:16:28.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:28.941+0000 7f46fc969700 1 -- 172.21.15.100:0/3762141356 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46f81a60c0 con 0x7f46f8102d50 2024-05-10T11:16:28.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:28.941+0000 7f46e77fe700 1 -- 172.21.15.100:0/3762141356 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f46f0019070 con 0x7f46f8102d50 2024-05-10T11:16:28.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:28.941+0000 7f46e77fe700 1 -- 172.21.15.100:0/3762141356 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f0014480 con 0x7f46f8102d50 2024-05-10T11:16:28.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:28.941+0000 7f46fc969700 1 -- 172.21.15.100:0/3762141356 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46f81a6650 con 0x7f46f8102d50 2024-05-10T11:16:28.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:28.942+0000 7f46fc969700 1 -- 172.21.15.100:0/3762141356 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f46f8063bb0 con 0x7f46f8102d50 2024-05-10T11:16:28.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:28.943+0000 7f46e77fe700 1 -- 172.21.15.100:0/3762141356 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f46f0012050 con 0x7f46f8102d50 2024-05-10T11:16:28.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:28.944+0000 7f46e77fe700 1 --2- 172.21.15.100:0/3762141356 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f46e00b2910 0x7f46e00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:28.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:28.944+0000 7f46e77fe700 1 -- 172.21.15.100:0/3762141356 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f46f00e32c0 con 0x7f46f8102d50 2024-05-10T11:16:28.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:28.944+0000 7f46f659c700 1 --2- 172.21.15.100:0/3762141356 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f46e00b2910 0x7f46e00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:28.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:28.944+0000 7f46e77fe700 1 --2- 172.21.15.100:0/3762141356 >> [v2:172.21.15.178:6816/2458690455,v1:172.21.15.178:6817/2458690455] conn(0x7f46e00bda70 0x7f46e00bff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:28.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:28.944+0000 7f46e77fe700 1 -- 172.21.15.100:0/3762141356 --> [v2:172.21.15.178:6816/2458690455,v1:172.21.15.178:6817/2458690455] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f46e00c05e0 con 0x7f46e00bda70 2024-05-10T11:16:28.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:28.944+0000 7f46e77fe700 1 -- 172.21.15.100:0/3762141356 <== mon.2 v2:172.21.15.178:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f46f00e3670 con 0x7f46f8102d50 2024-05-10T11:16:28.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:28.945+0000 7f46f6d9d700 1 --2- 172.21.15.100:0/3762141356 >> [v2:172.21.15.178:6816/2458690455,v1:172.21.15.178:6817/2458690455] conn(0x7f46e00bda70 0x7f46e00bff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:28.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:28.945+0000 7f46f659c700 1 --2- 172.21.15.100:0/3762141356 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f46e00b2910 0x7f46e00b4dd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f46e8009ea0 tx=0x7f46e8009290 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:16:28.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:28.945+0000 7f46f6d9d700 1 --2- 172.21.15.100:0/3762141356 >> [v2:172.21.15.178:6816/2458690455,v1:172.21.15.178:6817/2458690455] conn(0x7f46e00bda70 0x7f46e00bff10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:16:28.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:28.946+0000 7f46e77fe700 1 -- 172.21.15.100:0/3762141356 <== osd.7 v2:172.21.15.178:6816/2458690455 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f46e00c05e0 con 0x7f46e00bda70 2024-05-10T11:16:28.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:28.962+0000 7f46fc969700 1 -- 172.21.15.100:0/3762141356 --> [v2:172.21.15.178:6816/2458690455,v1:172.21.15.178:6817/2458690455] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f46f8063fb0 con 0x7f46e00bda70 2024-05-10T11:16:28.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:28.963+0000 7f46e77fe700 1 -- 172.21.15.100:0/3762141356 <== osd.7 v2:172.21.15.178:6816/2458690455 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f46f8063fb0 con 0x7f46e00bda70 2024-05-10T11:16:28.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:28.963+0000 7f46fc969700 1 -- 172.21.15.100:0/3762141356 >> [v2:172.21.15.178:6816/2458690455,v1:172.21.15.178:6817/2458690455] conn(0x7f46e00bda70 msgr2=0x7f46e00bff10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:28.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:28.963+0000 7f46fc969700 1 --2- 172.21.15.100:0/3762141356 >> [v2:172.21.15.178:6816/2458690455,v1:172.21.15.178:6817/2458690455] conn(0x7f46e00bda70 0x7f46e00bff10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:28.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:28.963+0000 7f46fc969700 1 -- 172.21.15.100:0/3762141356 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f46e00b2910 msgr2=0x7f46e00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:28.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:28.963+0000 7f46fc969700 1 --2- 172.21.15.100:0/3762141356 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f46e00b2910 0x7f46e00b4dd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f46e8009ea0 tx=0x7f46e8009290 comp rx=0 tx=0).stop 2024-05-10T11:16:28.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:28.963+0000 7f46fc969700 1 -- 172.21.15.100:0/3762141356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46f8102d50 msgr2=0x7f46f819ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:28.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:28.963+0000 7f46fc969700 1 --2- 172.21.15.100:0/3762141356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46f8102d50 0x7f46f819ae70 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f46f000ef10 tx=0x7f46f000c3a0 comp rx=0 tx=0).stop 2024-05-10T11:16:28.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:28.963+0000 7f46f6d9d700 1 -- 172.21.15.100:0/3762141356 reap_dead start 2024-05-10T11:16:28.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:28.963+0000 7f46fc969700 1 -- 172.21.15.100:0/3762141356 shutdown_connections 2024-05-10T11:16:28.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:28.963+0000 7f46fc969700 1 -- 172.21.15.100:0/3762141356 >> 172.21.15.100:0/3762141356 conn(0x7f46f8079310 msgr2=0x7f46f81010e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:16:28.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:28.963+0000 7f46fc969700 1 -- 172.21.15.100:0/3762141356 shutdown_connections 2024-05-10T11:16:28.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:28.963+0000 7f46fc969700 1 -- 172.21.15.100:0/3762141356 wait complete. 2024-05-10T11:16:29.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:29.145+0000 7f776ecb8700 1 -- 172.21.15.100:0/3198867876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7768103ef0 msgr2=0x7f77681103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:29.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:29.145+0000 7f776ecb8700 1 --2- 172.21.15.100:0/3198867876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7768103ef0 0x7f77681103d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f776000b0a0 tx=0x7f776000b3b0 comp rx=0 tx=0).stop 2024-05-10T11:16:29.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:29.145+0000 7f776ecb8700 1 -- 172.21.15.100:0/3198867876 shutdown_connections 2024-05-10T11:16:29.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:29.145+0000 7f776ecb8700 1 --2- 172.21.15.100:0/3198867876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7768103ef0 0x7f77681103d0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:29.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:29.145+0000 7f776ecb8700 1 --2- 172.21.15.100:0/3198867876 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7768108460 0x7f7768108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:29.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:29.145+0000 7f776ecb8700 1 --2- 172.21.15.100:0/3198867876 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7768103550 0x7f77681039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:29.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:29.145+0000 7f776ecb8700 1 -- 172.21.15.100:0/3198867876 >> 172.21.15.100:0/3198867876 conn(0x7f776807a600 msgr2=0x7f776807aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:16:29.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:29.145+0000 7f776ecb8700 1 -- 172.21.15.100:0/3198867876 shutdown_connections 2024-05-10T11:16:29.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:29.145+0000 7f776ecb8700 1 -- 172.21.15.100:0/3198867876 wait complete. 2024-05-10T11:16:29.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:29.146+0000 7f776ecb8700 1 Processor -- start 2024-05-10T11:16:29.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:29.146+0000 7f776ecb8700 1 -- start start 2024-05-10T11:16:29.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:29.146+0000 7f776ecb8700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7768103550 0x7f776819f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:29.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:29.146+0000 7f776ecb8700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7768103ef0 0x7f776819fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:29.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:29.146+0000 7f776ecb8700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7768108460 0x7f77681998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:29.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:29.146+0000 7f776ecb8700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f77681135f0 con 0x7f7768103550 2024-05-10T11:16:29.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:29.146+0000 7f776ecb8700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7768113470 con 0x7f7768103ef0 2024-05-10T11:16:29.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:29.146+0000 7f776ecb8700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7768113770 con 0x7f7768108460 2024-05-10T11:16:29.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:29.146+0000 7f776ca54700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7768103550 0x7f776819f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:29.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:29.146+0000 7f776ca54700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7768103550 0x7f776819f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59926/0 (socket says 172.21.15.100:59926) 2024-05-10T11:16:29.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:29.146+0000 7f776ca54700 1 -- 172.21.15.100:0/1798529652 learned_addr learned my addr 172.21.15.100:0/1798529652 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:16:29.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:29.146+0000 7f7767fff700 1 --2- 172.21.15.100:0/1798529652 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7768103ef0 0x7f776819fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:29.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:29.146+0000 7f776d255700 1 --2- 172.21.15.100:0/1798529652 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7768108460 0x7f77681998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:29.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:29.146+0000 7f776ca54700 1 -- 172.21.15.100:0/1798529652 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7768108460 msgr2=0x7f77681998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:29.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:29.146+0000 7f776ca54700 1 --2- 172.21.15.100:0/1798529652 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7768108460 0x7f77681998d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:29.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:29.146+0000 7f776ca54700 1 -- 172.21.15.100:0/1798529652 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7768103ef0 msgr2=0x7f776819fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:29.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:29.146+0000 7f776ca54700 1 --2- 172.21.15.100:0/1798529652 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7768103ef0 0x7f776819fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:29.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:29.146+0000 7f776ca54700 1 -- 172.21.15.100:0/1798529652 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f776819a030 con 0x7f7768103550 2024-05-10T11:16:29.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:29.147+0000 7f776ca54700 1 --2- 172.21.15.100:0/1798529652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7768103550 0x7f776819f710 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f775c00d980 tx=0x7f775c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:16:29.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:29.147+0000 7f7765ffb700 1 -- 172.21.15.100:0/1798529652 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775c017070 con 0x7f7768103550 2024-05-10T11:16:29.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:29.147+0000 7f776ecb8700 1 -- 172.21.15.100:0/1798529652 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f776819a2c0 con 0x7f7768103550 2024-05-10T11:16:29.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:29.147+0000 7f7765ffb700 1 -- 172.21.15.100:0/1798529652 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f775c009dc0 con 0x7f7768103550 2024-05-10T11:16:29.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:29.147+0000 7f7765ffb700 1 -- 172.21.15.100:0/1798529652 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775c00f3c0 con 0x7f7768103550 2024-05-10T11:16:29.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:29.147+0000 7f776ecb8700 1 -- 172.21.15.100:0/1798529652 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77681a65e0 con 0x7f7768103550 2024-05-10T11:16:29.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:29.148+0000 7f776ecb8700 1 -- 172.21.15.100:0/1798529652 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7768063c30 con 0x7f7768103550 2024-05-10T11:16:29.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:29.149+0000 7f7765ffb700 1 -- 172.21.15.100:0/1798529652 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f775c009930 con 0x7f7768103550 2024-05-10T11:16:29.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:29.150+0000 7f7765ffb700 1 --2- 172.21.15.100:0/1798529652 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f77580b2910 0x7f77580b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:29.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:29.150+0000 7f7767fff700 1 --2- 172.21.15.100:0/1798529652 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f77580b2910 0x7f77580b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:29.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:29.150+0000 7f7765ffb700 1 -- 172.21.15.100:0/1798529652 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f775c0e26a0 con 0x7f7768103550 2024-05-10T11:16:29.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:29.150+0000 7f7765ffb700 1 --2- 172.21.15.100:0/1798529652 >> [v2:172.21.15.178:6800/3103987017,v1:172.21.15.178:6801/3103987017] conn(0x7f77580bda70 0x7f77580bff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:29.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:29.150+0000 7f7765ffb700 1 -- 172.21.15.100:0/1798529652 --> [v2:172.21.15.178:6800/3103987017,v1:172.21.15.178:6801/3103987017] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f775c022e70 con 0x7f77580bda70 2024-05-10T11:16:29.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:29.150+0000 7f7765ffb700 1 -- 172.21.15.100:0/1798529652 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f775c0e2af0 con 0x7f7768103550 2024-05-10T11:16:29.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:29.150+0000 7f7767fff700 1 --2- 172.21.15.100:0/1798529652 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f77580b2910 0x7f77580b4dd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f776807ace0 tx=0x7f7754009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:16:29.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:29.151+0000 7f776d255700 1 --2- 172.21.15.100:0/1798529652 >> [v2:172.21.15.178:6800/3103987017,v1:172.21.15.178:6801/3103987017] conn(0x7f77580bda70 0x7f77580bff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:29.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:29.151+0000 7f776d255700 1 --2- 172.21.15.100:0/1798529652 >> [v2:172.21.15.178:6800/3103987017,v1:172.21.15.178:6801/3103987017] conn(0x7f77580bda70 0x7f77580bff10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:16:29.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:29.152+0000 7f7765ffb700 1 -- 172.21.15.100:0/1798529652 <== osd.1 v2:172.21.15.178:6800/3103987017 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f775c022e70 con 0x7f77580bda70 2024-05-10T11:16:29.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:29.168+0000 7f776ecb8700 1 -- 172.21.15.100:0/1798529652 --> [v2:172.21.15.178:6800/3103987017,v1:172.21.15.178:6801/3103987017] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f77680640b0 con 0x7f77580bda70 2024-05-10T11:16:29.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:29.169+0000 7f7765ffb700 1 -- 172.21.15.100:0/1798529652 <== osd.1 v2:172.21.15.178:6800/3103987017 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f77680640b0 con 0x7f77580bda70 2024-05-10T11:16:29.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:29.169+0000 7f776ecb8700 1 -- 172.21.15.100:0/1798529652 >> [v2:172.21.15.178:6800/3103987017,v1:172.21.15.178:6801/3103987017] conn(0x7f77580bda70 msgr2=0x7f77580bff10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:29.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:29.169+0000 7f776ecb8700 1 --2- 172.21.15.100:0/1798529652 >> [v2:172.21.15.178:6800/3103987017,v1:172.21.15.178:6801/3103987017] conn(0x7f77580bda70 0x7f77580bff10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:29.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:29.169+0000 7f776ecb8700 1 -- 172.21.15.100:0/1798529652 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f77580b2910 msgr2=0x7f77580b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:29.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:29.169+0000 7f776ecb8700 1 --2- 172.21.15.100:0/1798529652 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f77580b2910 0x7f77580b4dd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f776807ace0 tx=0x7f7754009190 comp rx=0 tx=0).stop 2024-05-10T11:16:29.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:29.169+0000 7f776ecb8700 1 -- 172.21.15.100:0/1798529652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7768103550 msgr2=0x7f776819f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:29.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:29.169+0000 7f776ecb8700 1 --2- 172.21.15.100:0/1798529652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7768103550 0x7f776819f710 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f775c00d980 tx=0x7f775c00dc90 comp rx=0 tx=0).stop 2024-05-10T11:16:29.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:29.169+0000 7f776d255700 1 -- 172.21.15.100:0/1798529652 reap_dead start 2024-05-10T11:16:29.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:29.169+0000 7f776ecb8700 1 -- 172.21.15.100:0/1798529652 shutdown_connections 2024-05-10T11:16:29.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:29.169+0000 7f776ecb8700 1 -- 172.21.15.100:0/1798529652 >> 172.21.15.100:0/1798529652 conn(0x7f776807a600 msgr2=0x7f7768101700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:16:29.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:29.170+0000 7f776ecb8700 1 -- 172.21.15.100:0/1798529652 shutdown_connections 2024-05-10T11:16:29.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:29.170+0000 7f776ecb8700 1 -- 172.21.15.100:0/1798529652 wait complete. 2024-05-10T11:16:29.985 INFO:teuthology.orchestra.run.smithi100.stdout:283467841581 2024-05-10T11:16:29.985 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-05-10T11:16:30.182 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:16:29 smithi100 ceph-mon[30296]: pgmap v363: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T11:16:30.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:16:29 smithi178 ceph-mon[33359]: pgmap v363: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T11:16:30.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:16:29 smithi188 ceph-mon[34075]: pgmap v363: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T11:16:32.217 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:16:31 smithi100 ceph-mon[30296]: pgmap v364: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T11:16:32.244 INFO:teuthology.orchestra.run.smithi100.stdout:55834574940 2024-05-10T11:16:32.244 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-05-10T11:16:32.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:16:31 smithi178 ceph-mon[33359]: pgmap v364: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T11:16:32.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:16:31 smithi188 ceph-mon[34075]: pgmap v364: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T11:16:32.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:31.901+0000 7f322899a700 1 -- 172.21.15.100:0/4249777388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3224108400 msgr2=0x7f32241087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:32.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:31.901+0000 7f322899a700 1 --2- 172.21.15.100:0/4249777388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3224108400 0x7f32241087e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3214009990 tx=0x7f3214009ca0 comp rx=0 tx=0).stop 2024-05-10T11:16:32.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:31.901+0000 7f322899a700 1 -- 172.21.15.100:0/4249777388 shutdown_connections 2024-05-10T11:16:32.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:31.901+0000 7f322899a700 1 --2- 172.21.15.100:0/4249777388 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3224102d50 0x7f322410f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:32.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:31.901+0000 7f322899a700 1 --2- 172.21.15.100:0/4249777388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3224108400 0x7f32241087e0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:32.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:31.901+0000 7f322899a700 1 --2- 172.21.15.100:0/4249777388 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3224102390 0x7f3224102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:32.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:31.901+0000 7f322899a700 1 -- 172.21.15.100:0/4249777388 >> 172.21.15.100:0/4249777388 conn(0x7f3224079310 msgr2=0x7f3224079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:16:32.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:31.901+0000 7f322899a700 1 -- 172.21.15.100:0/4249777388 shutdown_connections 2024-05-10T11:16:32.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:31.901+0000 7f322899a700 1 -- 172.21.15.100:0/4249777388 wait complete. 2024-05-10T11:16:32.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:31.902+0000 7f322899a700 1 Processor -- start 2024-05-10T11:16:32.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:31.902+0000 7f322899a700 1 -- start start 2024-05-10T11:16:32.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:31.902+0000 7f322899a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3224102390 0x7f322419a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:32.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:31.902+0000 7f322899a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3224102d50 0x7f322419add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:32.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:31.902+0000 7f322899a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3224108400 0x7f32241a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:32.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:31.902+0000 7f322899a700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f32241123d0 con 0x7f3224108400 2024-05-10T11:16:32.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:31.902+0000 7f322899a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3224112250 con 0x7f3224102390 2024-05-10T11:16:32.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:31.902+0000 7f322899a700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3224112550 con 0x7f3224102d50 2024-05-10T11:16:32.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:31.902+0000 7f3222d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3224108400 0x7f32241a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:32.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:31.902+0000 7f3222d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3224108400 0x7f32241a5910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59938/0 (socket says 172.21.15.100:59938) 2024-05-10T11:16:32.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:31.902+0000 7f322259c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3224102390 0x7f322419a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:32.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:31.902+0000 7f3222d9d700 1 -- 172.21.15.100:0/2540101433 learned_addr learned my addr 172.21.15.100:0/2540101433 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:16:32.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:31.902+0000 7f3221d9b700 1 --2- 172.21.15.100:0/2540101433 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3224102d50 0x7f322419add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:32.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:31.903+0000 7f3222d9d700 1 -- 172.21.15.100:0/2540101433 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3224102d50 msgr2=0x7f322419add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:32.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:31.903+0000 7f3222d9d700 1 --2- 172.21.15.100:0/2540101433 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3224102d50 0x7f322419add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:32.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:31.903+0000 7f3222d9d700 1 -- 172.21.15.100:0/2540101433 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3224102390 msgr2=0x7f322419a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:32.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:31.903+0000 7f3222d9d700 1 --2- 172.21.15.100:0/2540101433 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3224102390 0x7f322419a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:32.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:31.903+0000 7f3222d9d700 1 -- 172.21.15.100:0/2540101433 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32241a5e50 con 0x7f3224108400 2024-05-10T11:16:32.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:31.903+0000 7f3222d9d700 1 --2- 172.21.15.100:0/2540101433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3224108400 0x7f32241a5910 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f321c007ee0 tx=0x7f321c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:16:32.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:31.903+0000 7f32137fe700 1 -- 172.21.15.100:0/2540101433 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f321c004bb0 con 0x7f3224108400 2024-05-10T11:16:32.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:31.903+0000 7f32137fe700 1 -- 172.21.15.100:0/2540101433 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f321c004d50 con 0x7f3224108400 2024-05-10T11:16:32.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:31.903+0000 7f32137fe700 1 -- 172.21.15.100:0/2540101433 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f321c0058b0 con 0x7f3224108400 2024-05-10T11:16:32.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:31.903+0000 7f322899a700 1 -- 172.21.15.100:0/2540101433 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32241a6080 con 0x7f3224108400 2024-05-10T11:16:32.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:31.903+0000 7f322899a700 1 -- 172.21.15.100:0/2540101433 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32241a6540 con 0x7f3224108400 2024-05-10T11:16:32.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:31.904+0000 7f322899a700 1 -- 172.21.15.100:0/2540101433 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3224063bb0 con 0x7f3224108400 2024-05-10T11:16:32.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:31.906+0000 7f32137fe700 1 -- 172.21.15.100:0/2540101433 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f321c01a020 con 0x7f3224108400 2024-05-10T11:16:32.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:31.906+0000 7f32137fe700 1 --2- 172.21.15.100:0/2540101433 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f320c0b2910 0x7f320c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:32.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:31.906+0000 7f322259c700 1 --2- 172.21.15.100:0/2540101433 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f320c0b2910 0x7f320c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:32.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:31.906+0000 7f32137fe700 1 -- 172.21.15.100:0/2540101433 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f321c0e2350 con 0x7f3224108400 2024-05-10T11:16:32.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:31.907+0000 7f32137fe700 1 --2- 172.21.15.100:0/2540101433 >> [v2:172.21.15.178:6808/2222814192,v1:172.21.15.178:6809/2222814192] conn(0x7f320c0bda70 0x7f320c0bff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:32.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:31.907+0000 7f32137fe700 1 -- 172.21.15.100:0/2540101433 --> [v2:172.21.15.178:6808/2222814192,v1:172.21.15.178:6809/2222814192] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f321c016ec0 con 0x7f320c0bda70 2024-05-10T11:16:32.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:31.907+0000 7f32137fe700 1 -- 172.21.15.100:0/2540101433 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f321c0e2830 con 0x7f3224108400 2024-05-10T11:16:32.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:31.907+0000 7f322259c700 1 --2- 172.21.15.100:0/2540101433 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f320c0b2910 0x7f320c0b4dd0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f32140098e0 tx=0x7f3214005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:16:32.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:31.907+0000 7f3221d9b700 1 --2- 172.21.15.100:0/2540101433 >> [v2:172.21.15.178:6808/2222814192,v1:172.21.15.178:6809/2222814192] conn(0x7f320c0bda70 0x7f320c0bff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:32.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:31.907+0000 7f3221d9b700 1 --2- 172.21.15.100:0/2540101433 >> [v2:172.21.15.178:6808/2222814192,v1:172.21.15.178:6809/2222814192] conn(0x7f320c0bda70 0x7f320c0bff10 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:16:32.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:31.908+0000 7f32137fe700 1 -- 172.21.15.100:0/2540101433 <== osd.4 v2:172.21.15.178:6808/2222814192 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f321c016ec0 con 0x7f320c0bda70 2024-05-10T11:16:32.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:31.924+0000 7f322899a700 1 -- 172.21.15.100:0/2540101433 --> [v2:172.21.15.178:6808/2222814192,v1:172.21.15.178:6809/2222814192] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3224064030 con 0x7f320c0bda70 2024-05-10T11:16:32.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:31.925+0000 7f32137fe700 1 -- 172.21.15.100:0/2540101433 <== osd.4 v2:172.21.15.178:6808/2222814192 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3224064030 con 0x7f320c0bda70 2024-05-10T11:16:32.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:31.925+0000 7f322899a700 1 -- 172.21.15.100:0/2540101433 >> [v2:172.21.15.178:6808/2222814192,v1:172.21.15.178:6809/2222814192] conn(0x7f320c0bda70 msgr2=0x7f320c0bff10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:32.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:31.925+0000 7f322899a700 1 --2- 172.21.15.100:0/2540101433 >> [v2:172.21.15.178:6808/2222814192,v1:172.21.15.178:6809/2222814192] conn(0x7f320c0bda70 0x7f320c0bff10 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:32.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:31.925+0000 7f322899a700 1 -- 172.21.15.100:0/2540101433 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f320c0b2910 msgr2=0x7f320c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:32.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:31.925+0000 7f322899a700 1 --2- 172.21.15.100:0/2540101433 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f320c0b2910 0x7f320c0b4dd0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f32140098e0 tx=0x7f3214005960 comp rx=0 tx=0).stop 2024-05-10T11:16:32.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:31.925+0000 7f322899a700 1 -- 172.21.15.100:0/2540101433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3224108400 msgr2=0x7f32241a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:32.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:31.925+0000 7f322899a700 1 --2- 172.21.15.100:0/2540101433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3224108400 0x7f32241a5910 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f321c007ee0 tx=0x7f321c00d3b0 comp rx=0 tx=0).stop 2024-05-10T11:16:32.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:31.925+0000 7f3222d9d700 1 -- 172.21.15.100:0/2540101433 reap_dead start 2024-05-10T11:16:32.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:31.925+0000 7f322899a700 1 -- 172.21.15.100:0/2540101433 shutdown_connections 2024-05-10T11:16:32.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:31.925+0000 7f322899a700 1 -- 172.21.15.100:0/2540101433 >> 172.21.15.100:0/2540101433 conn(0x7f3224079310 msgr2=0x7f3224101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:16:32.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:31.926+0000 7f322899a700 1 -- 172.21.15.100:0/2540101433 shutdown_connections 2024-05-10T11:16:32.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:31.926+0000 7f322899a700 1 -- 172.21.15.100:0/2540101433 wait complete. 2024-05-10T11:16:33.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:33.169+0000 7f0862e20700 1 -- 172.21.15.100:0/667051862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f085c108850 msgr2=0x7f085c1103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:33.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:33.169+0000 7f0862e20700 1 --2- 172.21.15.100:0/667051862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f085c108850 0x7f085c1103d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f08540099c0 tx=0x7f0854009cd0 comp rx=0 tx=0).stop 2024-05-10T11:16:33.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:33.169+0000 7f0862e20700 1 -- 172.21.15.100:0/667051862 shutdown_connections 2024-05-10T11:16:33.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:33.169+0000 7f0862e20700 1 --2- 172.21.15.100:0/667051862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f085c108850 0x7f085c1103d0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:33.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:33.169+0000 7f0862e20700 1 --2- 172.21.15.100:0/667051862 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f085c103750 0x7f085c103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:33.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:33.169+0000 7f0862e20700 1 --2- 172.21.15.100:0/667051862 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f085c104100 0x7f085c108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:33.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:33.169+0000 7f0862e20700 1 -- 172.21.15.100:0/667051862 >> 172.21.15.100:0/667051862 conn(0x7f085c07a600 msgr2=0x7f085c07aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:16:33.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:33.169+0000 7f0862e20700 1 -- 172.21.15.100:0/667051862 shutdown_connections 2024-05-10T11:16:33.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:33.169+0000 7f0862e20700 1 -- 172.21.15.100:0/667051862 wait complete. 2024-05-10T11:16:33.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:33.170+0000 7f0862e20700 1 Processor -- start 2024-05-10T11:16:33.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:33.170+0000 7f0862e20700 1 -- start start 2024-05-10T11:16:33.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:33.170+0000 7f0862e20700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f085c103750 0x7f085c19f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:33.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:33.170+0000 7f0862e20700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f085c104100 0x7f085c19fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:33.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:33.170+0000 7f0862e20700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f085c108850 0x7f085c1998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:33.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:33.170+0000 7f0862e20700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f085c1135c0 con 0x7f085c108850 2024-05-10T11:16:33.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:33.170+0000 7f0862e20700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f085c113440 con 0x7f085c104100 2024-05-10T11:16:33.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:33.170+0000 7f0862e20700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f085c113740 con 0x7f085c103750 2024-05-10T11:16:33.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:33.170+0000 7f08613bd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f085c108850 0x7f085c1998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:33.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:33.170+0000 7f08613bd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f085c108850 0x7f085c1998c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59948/0 (socket says 172.21.15.100:59948) 2024-05-10T11:16:33.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:33.170+0000 7f08613bd700 1 -- 172.21.15.100:0/1231957494 learned_addr learned my addr 172.21.15.100:0/1231957494 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:16:33.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:33.170+0000 7f085bfff700 1 --2- 172.21.15.100:0/1231957494 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f085c104100 0x7f085c19fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:33.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:33.170+0000 7f0860bbc700 1 --2- 172.21.15.100:0/1231957494 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f085c103750 0x7f085c19f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:33.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:33.170+0000 7f08613bd700 1 -- 172.21.15.100:0/1231957494 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f085c103750 msgr2=0x7f085c19f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:33.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:33.170+0000 7f08613bd700 1 --2- 172.21.15.100:0/1231957494 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f085c103750 0x7f085c19f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:33.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:33.170+0000 7f08613bd700 1 -- 172.21.15.100:0/1231957494 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f085c104100 msgr2=0x7f085c19fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:33.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:33.170+0000 7f08613bd700 1 --2- 172.21.15.100:0/1231957494 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f085c104100 0x7f085c19fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:33.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:33.170+0000 7f08613bd700 1 -- 172.21.15.100:0/1231957494 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f085c19a020 con 0x7f085c108850 2024-05-10T11:16:33.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:33.171+0000 7f08613bd700 1 --2- 172.21.15.100:0/1231957494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f085c108850 0x7f085c1998c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f08540059d0 tx=0x7f085400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:16:33.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:33.171+0000 7f0859ffb700 1 -- 172.21.15.100:0/1231957494 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0854027070 con 0x7f085c108850 2024-05-10T11:16:33.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:33.171+0000 7f0859ffb700 1 -- 172.21.15.100:0/1231957494 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f08540056d0 con 0x7f085c108850 2024-05-10T11:16:33.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:33.171+0000 7f0862e20700 1 -- 172.21.15.100:0/1231957494 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f085c19a2b0 con 0x7f085c108850 2024-05-10T11:16:33.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:33.171+0000 7f0859ffb700 1 -- 172.21.15.100:0/1231957494 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0854022de0 con 0x7f085c108850 2024-05-10T11:16:33.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:33.171+0000 7f0862e20700 1 -- 172.21.15.100:0/1231957494 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f085c1a65d0 con 0x7f085c108850 2024-05-10T11:16:33.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:33.172+0000 7f0862e20700 1 -- 172.21.15.100:0/1231957494 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f085c063c30 con 0x7f085c108850 2024-05-10T11:16:33.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:33.173+0000 7f0859ffb700 1 -- 172.21.15.100:0/1231957494 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f085402a070 con 0x7f085c108850 2024-05-10T11:16:33.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:33.174+0000 7f0859ffb700 1 --2- 172.21.15.100:0/1231957494 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f08480b2990 0x7f08480b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:33.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:33.174+0000 7f0860bbc700 1 --2- 172.21.15.100:0/1231957494 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f08480b2990 0x7f08480b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:33.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:33.174+0000 7f0859ffb700 1 -- 172.21.15.100:0/1231957494 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f08540e39f0 con 0x7f085c108850 2024-05-10T11:16:33.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:33.175+0000 7f0859ffb700 1 --2- 172.21.15.100:0/1231957494 >> [v2:172.21.15.100:6802/643028278,v1:172.21.15.100:6803/643028278] conn(0x7f08480bdaf0 0x7f08480bff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:33.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:33.175+0000 7f0859ffb700 1 -- 172.21.15.100:0/1231957494 --> [v2:172.21.15.100:6802/643028278,v1:172.21.15.100:6803/643028278] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0854007e80 con 0x7f08480bdaf0 2024-05-10T11:16:33.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:33.175+0000 7f0859ffb700 1 -- 172.21.15.100:0/1231957494 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f08540e3e40 con 0x7f085c108850 2024-05-10T11:16:33.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:33.175+0000 7f085bfff700 1 --2- 172.21.15.100:0/1231957494 >> [v2:172.21.15.100:6802/643028278,v1:172.21.15.100:6803/643028278] conn(0x7f08480bdaf0 0x7f08480bff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:33.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:33.175+0000 7f0860bbc700 1 --2- 172.21.15.100:0/1231957494 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f08480b2990 0x7f08480b4e50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f085c19aa00 tx=0x7f0844009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:16:33.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:33.176+0000 7f085bfff700 1 --2- 172.21.15.100:0/1231957494 >> [v2:172.21.15.100:6802/643028278,v1:172.21.15.100:6803/643028278] conn(0x7f08480bdaf0 0x7f08480bff90 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:16:33.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:33.176+0000 7f0859ffb700 1 -- 172.21.15.100:0/1231957494 <== osd.0 v2:172.21.15.100:6802/643028278 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0854007e80 con 0x7f08480bdaf0 2024-05-10T11:16:33.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:33.193+0000 7f0862e20700 1 -- 172.21.15.100:0/1231957494 --> [v2:172.21.15.100:6802/643028278,v1:172.21.15.100:6803/643028278] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f085c064030 con 0x7f08480bdaf0 2024-05-10T11:16:33.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:33.193+0000 7f0859ffb700 1 -- 172.21.15.100:0/1231957494 <== osd.0 v2:172.21.15.100:6802/643028278 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f085c064030 con 0x7f08480bdaf0 2024-05-10T11:16:33.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:33.194+0000 7f0862e20700 1 -- 172.21.15.100:0/1231957494 >> [v2:172.21.15.100:6802/643028278,v1:172.21.15.100:6803/643028278] conn(0x7f08480bdaf0 msgr2=0x7f08480bff90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:33.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:33.194+0000 7f0862e20700 1 --2- 172.21.15.100:0/1231957494 >> [v2:172.21.15.100:6802/643028278,v1:172.21.15.100:6803/643028278] conn(0x7f08480bdaf0 0x7f08480bff90 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:33.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:33.194+0000 7f0862e20700 1 -- 172.21.15.100:0/1231957494 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f08480b2990 msgr2=0x7f08480b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:33.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:33.194+0000 7f0862e20700 1 --2- 172.21.15.100:0/1231957494 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f08480b2990 0x7f08480b4e50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f085c19aa00 tx=0x7f0844009190 comp rx=0 tx=0).stop 2024-05-10T11:16:33.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:33.194+0000 7f0862e20700 1 -- 172.21.15.100:0/1231957494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f085c108850 msgr2=0x7f085c1998c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:33.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:33.194+0000 7f0862e20700 1 --2- 172.21.15.100:0/1231957494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f085c108850 0x7f085c1998c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f08540059d0 tx=0x7f085400f690 comp rx=0 tx=0).stop 2024-05-10T11:16:33.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:33.194+0000 7f08613bd700 1 -- 172.21.15.100:0/1231957494 reap_dead start 2024-05-10T11:16:33.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:33.194+0000 7f0862e20700 1 -- 172.21.15.100:0/1231957494 shutdown_connections 2024-05-10T11:16:33.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:33.194+0000 7f0862e20700 1 -- 172.21.15.100:0/1231957494 >> 172.21.15.100:0/1231957494 conn(0x7f085c07a600 msgr2=0x7f085c1017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:16:33.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:33.194+0000 7f0862e20700 1 -- 172.21.15.100:0/1231957494 shutdown_connections 2024-05-10T11:16:33.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:33.194+0000 7f0862e20700 1 -- 172.21.15.100:0/1231957494 wait complete. 2024-05-10T11:16:34.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:16:33 smithi100 ceph-mon[30296]: pgmap v365: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T11:16:34.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:16:33 smithi178 ceph-mon[33359]: pgmap v365: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T11:16:34.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:16:33 smithi188 ceph-mon[34075]: pgmap v365: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T11:16:34.734 INFO:teuthology.orchestra.run.smithi100.stdout:206158430271 2024-05-10T11:16:34.735 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-05-10T11:16:35.696 INFO:teuthology.orchestra.run.smithi100.stdout:137438953551 2024-05-10T11:16:35.696 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-05-10T11:16:35.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:35.044+0000 7f4f7d856700 1 -- 172.21.15.100:0/3581718106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f781076c0 msgr2=0x7f4f78107aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:35.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:35.044+0000 7f4f7d856700 1 --2- 172.21.15.100:0/3581718106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f781076c0 0x7f4f78107aa0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4f68009990 tx=0x7f4f68009ca0 comp rx=0 tx=0).stop 2024-05-10T11:16:35.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:35.044+0000 7f4f7d856700 1 -- 172.21.15.100:0/3581718106 shutdown_connections 2024-05-10T11:16:35.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:35.044+0000 7f4f7d856700 1 --2- 172.21.15.100:0/3581718106 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f78102080 0x7f4f7810e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:35.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:35.044+0000 7f4f7d856700 1 --2- 172.21.15.100:0/3581718106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f781076c0 0x7f4f78107aa0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:35.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:35.044+0000 7f4f7d856700 1 --2- 172.21.15.100:0/3581718106 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f781016c0 0x7f4f78101b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:35.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:35.044+0000 7f4f7d856700 1 -- 172.21.15.100:0/3581718106 >> 172.21.15.100:0/3581718106 conn(0x7f4f780785a0 msgr2=0x7f4f780ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:16:35.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:35.044+0000 7f4f7d856700 1 -- 172.21.15.100:0/3581718106 shutdown_connections 2024-05-10T11:16:35.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:35.044+0000 7f4f7d856700 1 -- 172.21.15.100:0/3581718106 wait complete. 2024-05-10T11:16:35.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:35.045+0000 7f4f7d856700 1 Processor -- start 2024-05-10T11:16:35.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:35.045+0000 7f4f7d856700 1 -- start start 2024-05-10T11:16:35.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:35.045+0000 7f4f7d856700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f781016c0 0x7f4f7819d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:35.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:35.045+0000 7f4f7d856700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f78102080 0x7f4f7819d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:35.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:35.045+0000 7f4f7d856700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f781076c0 0x7f4f78197660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:35.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:35.045+0000 7f4f7d856700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4f78111320 con 0x7f4f781016c0 2024-05-10T11:16:35.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:35.045+0000 7f4f7d856700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4f781111a0 con 0x7f4f781076c0 2024-05-10T11:16:35.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:35.045+0000 7f4f7d856700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4f781114a0 con 0x7f4f78102080 2024-05-10T11:16:35.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:35.045+0000 7f4f76ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f781016c0 0x7f4f7819d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:35.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:35.045+0000 7f4f76ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f781016c0 0x7f4f7819d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59978/0 (socket says 172.21.15.100:59978) 2024-05-10T11:16:35.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:35.045+0000 7f4f76ffd700 1 -- 172.21.15.100:0/1422337426 learned_addr learned my addr 172.21.15.100:0/1422337426 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:16:35.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:35.046+0000 7f4f767fc700 1 --2- 172.21.15.100:0/1422337426 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f78102080 0x7f4f7819d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:35.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:35.046+0000 7f4f777fe700 1 --2- 172.21.15.100:0/1422337426 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f781076c0 0x7f4f78197660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:35.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:35.046+0000 7f4f76ffd700 1 -- 172.21.15.100:0/1422337426 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f78102080 msgr2=0x7f4f7819d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:35.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:35.046+0000 7f4f76ffd700 1 --2- 172.21.15.100:0/1422337426 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f78102080 0x7f4f7819d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:35.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:35.046+0000 7f4f76ffd700 1 -- 172.21.15.100:0/1422337426 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f781076c0 msgr2=0x7f4f78197660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:35.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:35.046+0000 7f4f76ffd700 1 --2- 172.21.15.100:0/1422337426 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f781076c0 0x7f4f78197660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:35.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:35.046+0000 7f4f76ffd700 1 -- 172.21.15.100:0/1422337426 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f78197dc0 con 0x7f4f781016c0 2024-05-10T11:16:35.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:35.046+0000 7f4f76ffd700 1 --2- 172.21.15.100:0/1422337426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f781016c0 0x7f4f7819d450 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4f68000c00 tx=0x7f4f680047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:16:35.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:35.046+0000 7f4f7c854700 1 -- 172.21.15.100:0/1422337426 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f68027070 con 0x7f4f781016c0 2024-05-10T11:16:35.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:35.046+0000 7f4f7d856700 1 -- 172.21.15.100:0/1422337426 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f78198050 con 0x7f4f781016c0 2024-05-10T11:16:35.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:35.046+0000 7f4f7c854700 1 -- 172.21.15.100:0/1422337426 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4f6800fb80 con 0x7f4f781016c0 2024-05-10T11:16:35.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:35.046+0000 7f4f7c854700 1 -- 172.21.15.100:0/1422337426 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f68022d00 con 0x7f4f781016c0 2024-05-10T11:16:35.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:35.046+0000 7f4f7d856700 1 -- 172.21.15.100:0/1422337426 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f781a4320 con 0x7f4f781016c0 2024-05-10T11:16:35.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:35.047+0000 7f4f7d856700 1 -- 172.21.15.100:0/1422337426 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4f78063bb0 con 0x7f4f781016c0 2024-05-10T11:16:35.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:35.049+0000 7f4f7c854700 1 -- 172.21.15.100:0/1422337426 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f4f6802a070 con 0x7f4f781016c0 2024-05-10T11:16:35.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:35.049+0000 7f4f7c854700 1 --2- 172.21.15.100:0/1422337426 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4f600b2910 0x7f4f600b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:35.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:35.049+0000 7f4f767fc700 1 --2- 172.21.15.100:0/1422337426 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4f600b2910 0x7f4f600b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:35.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:35.049+0000 7f4f7c854700 1 -- 172.21.15.100:0/1422337426 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f4f680e3c20 con 0x7f4f781016c0 2024-05-10T11:16:35.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:35.049+0000 7f4f7c854700 1 --2- 172.21.15.100:0/1422337426 >> [v2:172.21.15.100:6810/1713357192,v1:172.21.15.100:6811/1713357192] conn(0x7f4f600bda70 0x7f4f600bff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:35.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:35.050+0000 7f4f7c854700 1 -- 172.21.15.100:0/1422337426 --> [v2:172.21.15.100:6810/1713357192,v1:172.21.15.100:6811/1713357192] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4f680095d0 con 0x7f4f600bda70 2024-05-10T11:16:35.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:35.050+0000 7f4f7c854700 1 -- 172.21.15.100:0/1422337426 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f4f680e4070 con 0x7f4f781016c0 2024-05-10T11:16:35.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:35.050+0000 7f4f767fc700 1 --2- 172.21.15.100:0/1422337426 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4f600b2910 0x7f4f600b4dd0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4f78100c00 tx=0x7f4f6c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:16:35.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:35.050+0000 7f4f777fe700 1 --2- 172.21.15.100:0/1422337426 >> [v2:172.21.15.100:6810/1713357192,v1:172.21.15.100:6811/1713357192] conn(0x7f4f600bda70 0x7f4f600bff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:35.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:35.050+0000 7f4f777fe700 1 --2- 172.21.15.100:0/1422337426 >> [v2:172.21.15.100:6810/1713357192,v1:172.21.15.100:6811/1713357192] conn(0x7f4f600bda70 0x7f4f600bff10 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:16:35.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:35.051+0000 7f4f7c854700 1 -- 172.21.15.100:0/1422337426 <== osd.3 v2:172.21.15.100:6810/1713357192 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4f680095d0 con 0x7f4f600bda70 2024-05-10T11:16:35.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:35.067+0000 7f4f7d856700 1 -- 172.21.15.100:0/1422337426 --> [v2:172.21.15.100:6810/1713357192,v1:172.21.15.100:6811/1713357192] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4f78064030 con 0x7f4f600bda70 2024-05-10T11:16:35.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:35.067+0000 7f4f7c854700 1 -- 172.21.15.100:0/1422337426 <== osd.3 v2:172.21.15.100:6810/1713357192 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4f78064030 con 0x7f4f600bda70 2024-05-10T11:16:35.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:35.068+0000 7f4f7d856700 1 -- 172.21.15.100:0/1422337426 >> [v2:172.21.15.100:6810/1713357192,v1:172.21.15.100:6811/1713357192] conn(0x7f4f600bda70 msgr2=0x7f4f600bff10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:35.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:35.068+0000 7f4f7d856700 1 --2- 172.21.15.100:0/1422337426 >> [v2:172.21.15.100:6810/1713357192,v1:172.21.15.100:6811/1713357192] conn(0x7f4f600bda70 0x7f4f600bff10 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:35.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:35.068+0000 7f4f7d856700 1 -- 172.21.15.100:0/1422337426 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4f600b2910 msgr2=0x7f4f600b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:35.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:35.068+0000 7f4f7d856700 1 --2- 172.21.15.100:0/1422337426 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4f600b2910 0x7f4f600b4dd0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4f78100c00 tx=0x7f4f6c00a380 comp rx=0 tx=0).stop 2024-05-10T11:16:35.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:35.068+0000 7f4f7d856700 1 -- 172.21.15.100:0/1422337426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f781016c0 msgr2=0x7f4f7819d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:35.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:35.068+0000 7f4f7d856700 1 --2- 172.21.15.100:0/1422337426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f781016c0 0x7f4f7819d450 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4f68000c00 tx=0x7f4f680047b0 comp rx=0 tx=0).stop 2024-05-10T11:16:35.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:35.068+0000 7f4f777fe700 1 -- 172.21.15.100:0/1422337426 reap_dead start 2024-05-10T11:16:35.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:35.068+0000 7f4f7d856700 1 -- 172.21.15.100:0/1422337426 shutdown_connections 2024-05-10T11:16:35.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:35.068+0000 7f4f7d856700 1 -- 172.21.15.100:0/1422337426 >> 172.21.15.100:0/1422337426 conn(0x7f4f780785a0 msgr2=0x7f4f78104660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:16:35.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:35.068+0000 7f4f7d856700 1 -- 172.21.15.100:0/1422337426 shutdown_connections 2024-05-10T11:16:35.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:35.068+0000 7f4f7d856700 1 -- 172.21.15.100:0/1422337426 wait complete. 2024-05-10T11:16:36.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:16:35 smithi100 ceph-mon[30296]: pgmap v366: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T11:16:36.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:16:35 smithi178 ceph-mon[33359]: pgmap v366: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T11:16:36.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:16:35 smithi188 ceph-mon[34075]: pgmap v366: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T11:16:37.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:37.058+0000 7f0049791700 1 -- 172.21.15.100:0/335398006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0044108410 msgr2=0x7f00441087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:37.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:37.058+0000 7f0049791700 1 --2- 172.21.15.100:0/335398006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0044108410 0x7f00441087f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0034009990 tx=0x7f0034009ca0 comp rx=0 tx=0).stop 2024-05-10T11:16:37.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:37.058+0000 7f0049791700 1 -- 172.21.15.100:0/335398006 shutdown_connections 2024-05-10T11:16:37.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:37.058+0000 7f0049791700 1 --2- 172.21.15.100:0/335398006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0044102d60 0x7f004410f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:37.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:37.058+0000 7f0049791700 1 --2- 172.21.15.100:0/335398006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0044108410 0x7f00441087f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:37.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:37.058+0000 7f0049791700 1 --2- 172.21.15.100:0/335398006 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f00441023a0 0x7f0044102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:37.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:37.058+0000 7f0049791700 1 -- 172.21.15.100:0/335398006 >> 172.21.15.100:0/335398006 conn(0x7f0044079350 msgr2=0x7f0044079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:16:37.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:37.058+0000 7f0049791700 1 -- 172.21.15.100:0/335398006 shutdown_connections 2024-05-10T11:16:37.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:37.058+0000 7f0049791700 1 -- 172.21.15.100:0/335398006 wait complete. 2024-05-10T11:16:37.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:37.059+0000 7f0049791700 1 Processor -- start 2024-05-10T11:16:37.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:37.059+0000 7f0049791700 1 -- start start 2024-05-10T11:16:37.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:37.060+0000 7f0049791700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00441023a0 0x7f0044198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:37.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:37.060+0000 7f0042ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00441023a0 0x7f0044198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:37.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:37.060+0000 7f0042ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00441023a0 0x7f0044198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56816/0 (socket says 172.21.15.100:56816) 2024-05-10T11:16:37.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:37.060+0000 7f0042ffd700 1 -- 172.21.15.100:0/1211894827 learned_addr learned my addr 172.21.15.100:0/1211894827 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:16:37.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:37.060+0000 7f0049791700 1 --2- 172.21.15.100:0/1211894827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0044102d60 0x7f0044198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:37.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:37.060+0000 7f0049791700 1 --2- 172.21.15.100:0/1211894827 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0044108410 0x7f0044199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:37.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:37.060+0000 7f0049791700 1 -- 172.21.15.100:0/1211894827 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0044112460 con 0x7f00441023a0 2024-05-10T11:16:37.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:37.060+0000 7f0049791700 1 -- 172.21.15.100:0/1211894827 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f00441122e0 con 0x7f0044102d60 2024-05-10T11:16:37.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:37.060+0000 7f0049791700 1 -- 172.21.15.100:0/1211894827 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f00441125e0 con 0x7f0044108410 2024-05-10T11:16:37.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:37.060+0000 7f00427fc700 1 --2- 172.21.15.100:0/1211894827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0044102d60 0x7f0044198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:37.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:37.061+0000 7f0042ffd700 1 -- 172.21.15.100:0/1211894827 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0044108410 msgr2=0x7f0044199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:37.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:37.061+0000 7f0042ffd700 1 --2- 172.21.15.100:0/1211894827 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0044108410 0x7f0044199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:37.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:37.061+0000 7f0042ffd700 1 -- 172.21.15.100:0/1211894827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0044102d60 msgr2=0x7f0044198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:37.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:37.061+0000 7f0042ffd700 1 --2- 172.21.15.100:0/1211894827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0044102d60 0x7f0044198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:37.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:37.061+0000 7f0042ffd700 1 -- 172.21.15.100:0/1211894827 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00441a7f80 con 0x7f00441023a0 2024-05-10T11:16:37.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:37.061+0000 7f0042ffd700 1 --2- 172.21.15.100:0/1211894827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00441023a0 0x7f0044198710 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f003400b650 tx=0x7f003400bfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:16:37.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:37.061+0000 7f002bfff700 1 -- 172.21.15.100:0/1211894827 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0034027070 con 0x7f00441023a0 2024-05-10T11:16:37.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:37.061+0000 7f0049791700 1 -- 172.21.15.100:0/1211894827 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00441a8210 con 0x7f00441023a0 2024-05-10T11:16:37.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:37.061+0000 7f0049791700 1 -- 172.21.15.100:0/1211894827 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00441a8600 con 0x7f00441023a0 2024-05-10T11:16:37.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:37.061+0000 7f002bfff700 1 -- 172.21.15.100:0/1211894827 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0034022470 con 0x7f00441023a0 2024-05-10T11:16:37.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:37.061+0000 7f002bfff700 1 -- 172.21.15.100:0/1211894827 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0034017650 con 0x7f00441023a0 2024-05-10T11:16:37.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:37.062+0000 7f0049791700 1 -- 172.21.15.100:0/1211894827 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0030001030 con 0x7f00441023a0 2024-05-10T11:16:37.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:37.065+0000 7f002bfff700 1 -- 172.21.15.100:0/1211894827 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f0034009d10 con 0x7f00441023a0 2024-05-10T11:16:37.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:37.066+0000 7f002bfff700 1 --2- 172.21.15.100:0/1211894827 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f002c0b2990 0x7f002c0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:37.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:37.066+0000 7f002bfff700 1 -- 172.21.15.100:0/1211894827 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f00340e3c50 con 0x7f00441023a0 2024-05-10T11:16:37.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:37.066+0000 7f00427fc700 1 --2- 172.21.15.100:0/1211894827 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f002c0b2990 0x7f002c0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:37.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:37.066+0000 7f002bfff700 1 --2- 172.21.15.100:0/1211894827 >> [v2:172.21.15.188:6808/3517375935,v1:172.21.15.188:6809/3517375935] conn(0x7f002c0bd9c0 0x7f002c0bfe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:37.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:37.066+0000 7f002bfff700 1 -- 172.21.15.100:0/1211894827 --> [v2:172.21.15.188:6808/3517375935,v1:172.21.15.188:6809/3517375935] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f002c0c0530 con 0x7f002c0bd9c0 2024-05-10T11:16:37.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:37.066+0000 7f002bfff700 1 -- 172.21.15.100:0/1211894827 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f00340e40a0 con 0x7f00441023a0 2024-05-10T11:16:37.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:37.067+0000 7f00437fe700 1 --2- 172.21.15.100:0/1211894827 >> [v2:172.21.15.188:6808/3517375935,v1:172.21.15.188:6809/3517375935] conn(0x7f002c0bd9c0 0x7f002c0bfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:37.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:37.067+0000 7f00427fc700 1 --2- 172.21.15.100:0/1211894827 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f002c0b2990 0x7f002c0b4e50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0038009680 tx=0x7f0038009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:16:37.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:37.067+0000 7f00437fe700 1 --2- 172.21.15.100:0/1211894827 >> [v2:172.21.15.188:6808/3517375935,v1:172.21.15.188:6809/3517375935] conn(0x7f002c0bd9c0 0x7f002c0bfe60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:16:37.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:37.068+0000 7f002bfff700 1 -- 172.21.15.100:0/1211894827 <== osd.5 v2:172.21.15.188:6808/3517375935 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f002c0c0530 con 0x7f002c0bd9c0 2024-05-10T11:16:37.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:37.085+0000 7f0049791700 1 -- 172.21.15.100:0/1211894827 --> [v2:172.21.15.188:6808/3517375935,v1:172.21.15.188:6809/3517375935] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0030002e60 con 0x7f002c0bd9c0 2024-05-10T11:16:37.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:37.086+0000 7f002bfff700 1 -- 172.21.15.100:0/1211894827 <== osd.5 v2:172.21.15.188:6808/3517375935 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0030002e60 con 0x7f002c0bd9c0 2024-05-10T11:16:37.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:37.086+0000 7f0049791700 1 -- 172.21.15.100:0/1211894827 >> [v2:172.21.15.188:6808/3517375935,v1:172.21.15.188:6809/3517375935] conn(0x7f002c0bd9c0 msgr2=0x7f002c0bfe60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:37.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:37.086+0000 7f0049791700 1 --2- 172.21.15.100:0/1211894827 >> [v2:172.21.15.188:6808/3517375935,v1:172.21.15.188:6809/3517375935] conn(0x7f002c0bd9c0 0x7f002c0bfe60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:37.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:37.086+0000 7f0049791700 1 -- 172.21.15.100:0/1211894827 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f002c0b2990 msgr2=0x7f002c0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:37.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:37.086+0000 7f0049791700 1 --2- 172.21.15.100:0/1211894827 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f002c0b2990 0x7f002c0b4e50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0038009680 tx=0x7f0038009260 comp rx=0 tx=0).stop 2024-05-10T11:16:37.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:37.086+0000 7f0049791700 1 -- 172.21.15.100:0/1211894827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00441023a0 msgr2=0x7f0044198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:37.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:37.086+0000 7f0049791700 1 --2- 172.21.15.100:0/1211894827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00441023a0 0x7f0044198710 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f003400b650 tx=0x7f003400bfc0 comp rx=0 tx=0).stop 2024-05-10T11:16:37.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:37.086+0000 7f00437fe700 1 -- 172.21.15.100:0/1211894827 reap_dead start 2024-05-10T11:16:37.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:37.086+0000 7f0049791700 1 -- 172.21.15.100:0/1211894827 shutdown_connections 2024-05-10T11:16:37.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:37.086+0000 7f0049791700 1 -- 172.21.15.100:0/1211894827 >> 172.21.15.100:0/1211894827 conn(0x7f0044079350 msgr2=0x7f0044101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:16:37.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:37.086+0000 7f0049791700 1 -- 172.21.15.100:0/1211894827 shutdown_connections 2024-05-10T11:16:37.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:37.086+0000 7f0049791700 1 -- 172.21.15.100:0/1211894827 wait complete. 2024-05-10T11:16:38.028 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:16:37 smithi100 ceph-mon[30296]: pgmap v367: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T11:16:38.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:16:37 smithi178 ceph-mon[33359]: pgmap v367: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T11:16:38.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:16:37 smithi188 ceph-mon[34075]: pgmap v367: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T11:16:38.616 INFO:teuthology.orchestra.run.smithi100.stdout:158913790026 2024-05-10T11:16:38.617 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-05-10T11:16:39.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:38.028+0000 7fa7f47cb700 1 -- 172.21.15.100:0/3505731163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7ec102390 msgr2=0x7fa7ec102810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:39.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:38.028+0000 7fa7f47cb700 1 --2- 172.21.15.100:0/3505731163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7ec102390 0x7fa7ec102810 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc009990 tx=0x7fa7dc009ca0 comp rx=0 tx=0).stop 2024-05-10T11:16:39.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:38.028+0000 7fa7f47cb700 1 -- 172.21.15.100:0/3505731163 shutdown_connections 2024-05-10T11:16:39.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:38.028+0000 7fa7f47cb700 1 --2- 172.21.15.100:0/3505731163 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7ec102d50 0x7fa7ec10f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:39.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:38.028+0000 7fa7f47cb700 1 --2- 172.21.15.100:0/3505731163 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7ec108400 0x7fa7ec1087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:39.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:38.028+0000 7fa7f47cb700 1 --2- 172.21.15.100:0/3505731163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7ec102390 0x7fa7ec102810 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:39.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:38.028+0000 7fa7f47cb700 1 -- 172.21.15.100:0/3505731163 >> 172.21.15.100:0/3505731163 conn(0x7fa7ec079310 msgr2=0x7fa7ec079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:16:39.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:38.028+0000 7fa7f47cb700 1 -- 172.21.15.100:0/3505731163 shutdown_connections 2024-05-10T11:16:39.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:38.028+0000 7fa7f47cb700 1 -- 172.21.15.100:0/3505731163 wait complete. 2024-05-10T11:16:39.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:38.029+0000 7fa7f47cb700 1 Processor -- start 2024-05-10T11:16:39.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:38.029+0000 7fa7f47cb700 1 -- start start 2024-05-10T11:16:39.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:38.029+0000 7fa7f47cb700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7ec102390 0x7fa7ec19f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:39.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:38.029+0000 7fa7f47cb700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7ec102d50 0x7fa7ec19fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:39.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:38.029+0000 7fa7f47cb700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7ec108400 0x7fa7ec1999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:39.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:38.029+0000 7fa7f47cb700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa7ec1124d0 con 0x7fa7ec102d50 2024-05-10T11:16:39.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:38.029+0000 7fa7f47cb700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa7ec112350 con 0x7fa7ec108400 2024-05-10T11:16:39.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:38.029+0000 7fa7f47cb700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa7ec112650 con 0x7fa7ec102390 2024-05-10T11:16:39.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:38.029+0000 7fa7f1d66700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7ec102d50 0x7fa7ec19fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:39.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:38.029+0000 7fa7f1d66700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7ec102d50 0x7fa7ec19fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56834/0 (socket says 172.21.15.100:56834) 2024-05-10T11:16:39.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:38.029+0000 7fa7f1d66700 1 -- 172.21.15.100:0/3465402350 learned_addr learned my addr 172.21.15.100:0/3465402350 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:16:39.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:38.029+0000 7fa7f2567700 1 --2- 172.21.15.100:0/3465402350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7ec102390 0x7fa7ec19f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:39.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:38.029+0000 7fa7f2d68700 1 --2- 172.21.15.100:0/3465402350 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7ec108400 0x7fa7ec1999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:39.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:38.029+0000 7fa7f1d66700 1 -- 172.21.15.100:0/3465402350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7ec102390 msgr2=0x7fa7ec19f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:39.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:38.029+0000 7fa7f1d66700 1 --2- 172.21.15.100:0/3465402350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7ec102390 0x7fa7ec19f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:39.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:38.029+0000 7fa7f1d66700 1 -- 172.21.15.100:0/3465402350 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7ec108400 msgr2=0x7fa7ec1999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:39.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:38.029+0000 7fa7f1d66700 1 --2- 172.21.15.100:0/3465402350 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7ec108400 0x7fa7ec1999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:39.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:38.029+0000 7fa7f1d66700 1 -- 172.21.15.100:0/3465402350 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7ec19a140 con 0x7fa7ec102d50 2024-05-10T11:16:39.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:38.029+0000 7fa7f2d68700 1 --2- 172.21.15.100:0/3465402350 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7ec108400 0x7fa7ec1999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:16:39.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:38.029+0000 7fa7f2567700 1 --2- 172.21.15.100:0/3465402350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7ec102390 0x7fa7ec19f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:16:39.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:38.029+0000 7fa7f1d66700 1 --2- 172.21.15.100:0/3465402350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7ec102d50 0x7fa7ec19fd60 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc000c00 tx=0x7fa7dc0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:16:39.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:38.030+0000 7fa7e37fe700 1 -- 172.21.15.100:0/3465402350 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7dc027070 con 0x7fa7ec102d50 2024-05-10T11:16:39.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:38.030+0000 7fa7f47cb700 1 -- 172.21.15.100:0/3465402350 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7ec19a3d0 con 0x7fa7ec102d50 2024-05-10T11:16:39.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:38.030+0000 7fa7e37fe700 1 -- 172.21.15.100:0/3465402350 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa7dc00fdb0 con 0x7fa7ec102d50 2024-05-10T11:16:39.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:38.030+0000 7fa7e37fe700 1 -- 172.21.15.100:0/3465402350 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7dc0173f0 con 0x7fa7ec102d50 2024-05-10T11:16:39.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:38.030+0000 7fa7f47cb700 1 -- 172.21.15.100:0/3465402350 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7ec1a66f0 con 0x7fa7ec102d50 2024-05-10T11:16:39.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:38.031+0000 7fa7f47cb700 1 -- 172.21.15.100:0/3465402350 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa7ec107490 con 0x7fa7ec102d50 2024-05-10T11:16:39.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:38.032+0000 7fa7e37fe700 1 -- 172.21.15.100:0/3465402350 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fa7dc02a070 con 0x7fa7ec102d50 2024-05-10T11:16:39.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:38.033+0000 7fa7e37fe700 1 --2- 172.21.15.100:0/3465402350 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa7d80b2910 0x7fa7d80b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:39.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:38.033+0000 7fa7f2567700 1 --2- 172.21.15.100:0/3465402350 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa7d80b2910 0x7fa7d80b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:39.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:38.033+0000 7fa7e37fe700 1 -- 172.21.15.100:0/3465402350 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fa7dc0e3f40 con 0x7fa7ec102d50 2024-05-10T11:16:39.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:38.033+0000 7fa7e37fe700 1 --2- 172.21.15.100:0/3465402350 >> [v2:172.21.15.100:6818/1410593862,v1:172.21.15.100:6819/1410593862] conn(0x7fa7d80bd940 0x7fa7d80bfde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:39.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:38.033+0000 7fa7e37fe700 1 -- 172.21.15.100:0/3465402350 --> [v2:172.21.15.100:6818/1410593862,v1:172.21.15.100:6819/1410593862] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa7d80c04b0 con 0x7fa7d80bd940 2024-05-10T11:16:39.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:38.033+0000 7fa7e37fe700 1 -- 172.21.15.100:0/3465402350 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fa7dc0e4390 con 0x7fa7ec102d50 2024-05-10T11:16:39.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:38.033+0000 7fa7f2d68700 1 --2- 172.21.15.100:0/3465402350 >> [v2:172.21.15.100:6818/1410593862,v1:172.21.15.100:6819/1410593862] conn(0x7fa7d80bd940 0x7fa7d80bfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:39.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:38.033+0000 7fa7f2567700 1 --2- 172.21.15.100:0/3465402350 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa7d80b2910 0x7fa7d80b4dd0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec19ab20 tx=0x7fa7e8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:16:39.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:38.034+0000 7fa7f2d68700 1 --2- 172.21.15.100:0/3465402350 >> [v2:172.21.15.100:6818/1410593862,v1:172.21.15.100:6819/1410593862] conn(0x7fa7d80bd940 0x7fa7d80bfde0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:16:39.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:38.034+0000 7fa7e37fe700 1 -- 172.21.15.100:0/3465402350 <== osd.6 v2:172.21.15.100:6818/1410593862 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa7d80c04b0 con 0x7fa7d80bd940 2024-05-10T11:16:39.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:38.051+0000 7fa7f47cb700 1 -- 172.21.15.100:0/3465402350 --> [v2:172.21.15.100:6818/1410593862,v1:172.21.15.100:6819/1410593862] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa7ec103ab0 con 0x7fa7d80bd940 2024-05-10T11:16:39.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:38.051+0000 7fa7e37fe700 1 -- 172.21.15.100:0/3465402350 <== osd.6 v2:172.21.15.100:6818/1410593862 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa7ec103ab0 con 0x7fa7d80bd940 2024-05-10T11:16:39.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:38.051+0000 7fa7f47cb700 1 -- 172.21.15.100:0/3465402350 >> [v2:172.21.15.100:6818/1410593862,v1:172.21.15.100:6819/1410593862] conn(0x7fa7d80bd940 msgr2=0x7fa7d80bfde0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:39.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:38.051+0000 7fa7f47cb700 1 --2- 172.21.15.100:0/3465402350 >> [v2:172.21.15.100:6818/1410593862,v1:172.21.15.100:6819/1410593862] conn(0x7fa7d80bd940 0x7fa7d80bfde0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:39.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:38.051+0000 7fa7f47cb700 1 -- 172.21.15.100:0/3465402350 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa7d80b2910 msgr2=0x7fa7d80b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:39.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:38.051+0000 7fa7f47cb700 1 --2- 172.21.15.100:0/3465402350 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa7d80b2910 0x7fa7d80b4dd0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec19ab20 tx=0x7fa7e8008040 comp rx=0 tx=0).stop 2024-05-10T11:16:39.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:38.051+0000 7fa7f47cb700 1 -- 172.21.15.100:0/3465402350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7ec102d50 msgr2=0x7fa7ec19fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:39.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:38.051+0000 7fa7f47cb700 1 --2- 172.21.15.100:0/3465402350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7ec102d50 0x7fa7ec19fd60 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc000c00 tx=0x7fa7dc0047b0 comp rx=0 tx=0).stop 2024-05-10T11:16:39.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:38.051+0000 7fa7f2d68700 1 -- 172.21.15.100:0/3465402350 reap_dead start 2024-05-10T11:16:39.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:38.051+0000 7fa7f47cb700 1 -- 172.21.15.100:0/3465402350 shutdown_connections 2024-05-10T11:16:39.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:38.051+0000 7fa7f47cb700 1 -- 172.21.15.100:0/3465402350 >> 172.21.15.100:0/3465402350 conn(0x7fa7ec079310 msgr2=0x7fa7ec06bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:16:39.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:38.051+0000 7fa7f47cb700 1 -- 172.21.15.100:0/3465402350 shutdown_connections 2024-05-10T11:16:39.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:38.052+0000 7fa7f47cb700 1 -- 172.21.15.100:0/3465402350 wait complete. 2024-05-10T11:16:39.630 INFO:teuthology.orchestra.run.smithi100.stdout:184683593798 2024-05-10T11:16:39.630 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-05-10T11:16:39.994 INFO:teuthology.orchestra.run.smithi100.stdout:34359738466 2024-05-10T11:16:39.994 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-05-10T11:16:40.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:36.949+0000 7f5a75ffb700 1 -- 172.21.15.100:0/998651342 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a70009480 con 0x7f5a78107760 2024-05-10T11:16:40.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:36.950+0000 7f5a7d4a9700 1 -- 172.21.15.100:0/998651342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a78107760 msgr2=0x7f5a7810f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:40.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:36.950+0000 7f5a7d4a9700 1 --2- 172.21.15.100:0/998651342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a78107760 0x7f5a7810f2e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5a7000b0d0 tx=0x7f5a7000b3e0 comp rx=0 tx=0).stop 2024-05-10T11:16:40.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:36.950+0000 7f5a7d4a9700 1 -- 172.21.15.100:0/998651342 shutdown_connections 2024-05-10T11:16:40.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:36.950+0000 7f5a7d4a9700 1 --2- 172.21.15.100:0/998651342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a78107760 0x7f5a7810f2e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:40.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:36.950+0000 7f5a7d4a9700 1 --2- 172.21.15.100:0/998651342 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a78102e00 0x7f5a781031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:40.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:36.950+0000 7f5a7d4a9700 1 --2- 172.21.15.100:0/998651342 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a781037b0 0x7f5a78107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:40.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:36.950+0000 7f5a7d4a9700 1 -- 172.21.15.100:0/998651342 >> 172.21.15.100:0/998651342 conn(0x7f5a78078a10 msgr2=0x7f5a780ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:16:40.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:36.950+0000 7f5a7d4a9700 1 -- 172.21.15.100:0/998651342 shutdown_connections 2024-05-10T11:16:40.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:36.950+0000 7f5a7d4a9700 1 -- 172.21.15.100:0/998651342 wait complete. 2024-05-10T11:16:40.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:36.951+0000 7f5a7d4a9700 1 Processor -- start 2024-05-10T11:16:40.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:36.951+0000 7f5a7d4a9700 1 -- start start 2024-05-10T11:16:40.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:36.951+0000 7f5a7d4a9700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a78102e00 0x7f5a7819e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:40.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:36.951+0000 7f5a7d4a9700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a781037b0 0x7f5a7819eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:40.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:36.952+0000 7f5a7d4a9700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a78107760 0x7f5a781986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:40.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:36.952+0000 7f5a76ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a78102e00 0x7f5a7819e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:40.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:36.952+0000 7f5a7d4a9700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5a781123e0 con 0x7f5a78102e00 2024-05-10T11:16:40.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:36.952+0000 7f5a7d4a9700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5a78112260 con 0x7f5a781037b0 2024-05-10T11:16:40.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:36.952+0000 7f5a7d4a9700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5a78112560 con 0x7f5a78107760 2024-05-10T11:16:40.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:36.952+0000 7f5a76ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a78102e00 0x7f5a7819e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56798/0 (socket says 172.21.15.100:56798) 2024-05-10T11:16:40.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:36.952+0000 7f5a76ffd700 1 -- 172.21.15.100:0/3122307672 learned_addr learned my addr 172.21.15.100:0/3122307672 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:16:40.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:36.952+0000 7f5a767fc700 1 --2- 172.21.15.100:0/3122307672 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a781037b0 0x7f5a7819eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:40.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:36.952+0000 7f5a777fe700 1 --2- 172.21.15.100:0/3122307672 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a78107760 0x7f5a781986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:40.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:36.952+0000 7f5a76ffd700 1 -- 172.21.15.100:0/3122307672 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a78107760 msgr2=0x7f5a781986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:40.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:36.952+0000 7f5a76ffd700 1 --2- 172.21.15.100:0/3122307672 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a78107760 0x7f5a781986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:40.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:36.952+0000 7f5a76ffd700 1 -- 172.21.15.100:0/3122307672 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a781037b0 msgr2=0x7f5a7819eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:40.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:36.952+0000 7f5a76ffd700 1 --2- 172.21.15.100:0/3122307672 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a781037b0 0x7f5a7819eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:40.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:36.952+0000 7f5a76ffd700 1 -- 172.21.15.100:0/3122307672 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a78198fc0 con 0x7f5a78102e00 2024-05-10T11:16:40.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:36.952+0000 7f5a76ffd700 1 --2- 172.21.15.100:0/3122307672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a78102e00 0x7f5a7819e5c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5a68009970 tx=0x7f5a68009c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:16:40.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:36.952+0000 7f5a5ffff700 1 -- 172.21.15.100:0/3122307672 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a6801c070 con 0x7f5a78102e00 2024-05-10T11:16:40.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:36.953+0000 7f5a7d4a9700 1 -- 172.21.15.100:0/3122307672 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a78199250 con 0x7f5a78102e00 2024-05-10T11:16:40.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:36.953+0000 7f5a5ffff700 1 -- 172.21.15.100:0/3122307672 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5a6800d260 con 0x7f5a78102e00 2024-05-10T11:16:40.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:36.953+0000 7f5a5ffff700 1 -- 172.21.15.100:0/3122307672 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a68017890 con 0x7f5a78102e00 2024-05-10T11:16:40.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:36.953+0000 7f5a7d4a9700 1 -- 172.21.15.100:0/3122307672 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a78101c60 con 0x7f5a78102e00 2024-05-10T11:16:40.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:36.954+0000 7f5a7d4a9700 1 -- 172.21.15.100:0/3122307672 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5a78102270 con 0x7f5a78102e00 2024-05-10T11:16:40.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:36.955+0000 7f5a5ffff700 1 -- 172.21.15.100:0/3122307672 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f5a6801f070 con 0x7f5a78102e00 2024-05-10T11:16:40.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:36.956+0000 7f5a5ffff700 1 --2- 172.21.15.100:0/3122307672 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f5a600b2910 0x7f5a600b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:40.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:36.956+0000 7f5a767fc700 1 --2- 172.21.15.100:0/3122307672 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f5a600b2910 0x7f5a600b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:40.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:36.956+0000 7f5a5ffff700 1 -- 172.21.15.100:0/3122307672 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f5a680e3980 con 0x7f5a78102e00 2024-05-10T11:16:40.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:36.956+0000 7f5a5ffff700 1 --2- 172.21.15.100:0/3122307672 >> [v2:172.21.15.188:6800/3991466468,v1:172.21.15.188:6801/3991466468] conn(0x7f5a600bda70 0x7f5a600bff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:40.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:36.956+0000 7f5a5ffff700 1 -- 172.21.15.100:0/3122307672 --> [v2:172.21.15.188:6800/3991466468,v1:172.21.15.188:6801/3991466468] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5a600c05e0 con 0x7f5a600bda70 2024-05-10T11:16:40.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:36.956+0000 7f5a5ffff700 1 -- 172.21.15.100:0/3122307672 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f5a680e3dd0 con 0x7f5a78102e00 2024-05-10T11:16:40.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:36.956+0000 7f5a767fc700 1 --2- 172.21.15.100:0/3122307672 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f5a600b2910 0x7f5a600b4dd0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5a78199e50 tx=0x7f5a6c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:16:40.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:36.957+0000 7f5a777fe700 1 --2- 172.21.15.100:0/3122307672 >> [v2:172.21.15.188:6800/3991466468,v1:172.21.15.188:6801/3991466468] conn(0x7f5a600bda70 0x7f5a600bff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:40.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:36.957+0000 7f5a777fe700 1 --2- 172.21.15.100:0/3122307672 >> [v2:172.21.15.188:6800/3991466468,v1:172.21.15.188:6801/3991466468] conn(0x7f5a600bda70 0x7f5a600bff10 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:16:40.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:36.958+0000 7f5a5ffff700 1 -- 172.21.15.100:0/3122307672 <== osd.2 v2:172.21.15.188:6800/3991466468 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5a600c05e0 con 0x7f5a600bda70 2024-05-10T11:16:40.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:36.974+0000 7f5a7d4a9700 1 -- 172.21.15.100:0/3122307672 --> [v2:172.21.15.188:6800/3991466468,v1:172.21.15.188:6801/3991466468] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5a78102670 con 0x7f5a600bda70 2024-05-10T11:16:40.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:36.975+0000 7f5a5ffff700 1 -- 172.21.15.100:0/3122307672 <== osd.2 v2:172.21.15.188:6800/3991466468 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5a78102670 con 0x7f5a600bda70 2024-05-10T11:16:40.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:36.975+0000 7f5a7d4a9700 1 -- 172.21.15.100:0/3122307672 >> [v2:172.21.15.188:6800/3991466468,v1:172.21.15.188:6801/3991466468] conn(0x7f5a600bda70 msgr2=0x7f5a600bff10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:40.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:36.975+0000 7f5a7d4a9700 1 --2- 172.21.15.100:0/3122307672 >> [v2:172.21.15.188:6800/3991466468,v1:172.21.15.188:6801/3991466468] conn(0x7f5a600bda70 0x7f5a600bff10 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:40.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:36.975+0000 7f5a7d4a9700 1 -- 172.21.15.100:0/3122307672 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f5a600b2910 msgr2=0x7f5a600b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:40.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:36.975+0000 7f5a7d4a9700 1 --2- 172.21.15.100:0/3122307672 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f5a600b2910 0x7f5a600b4dd0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5a78199e50 tx=0x7f5a6c009260 comp rx=0 tx=0).stop 2024-05-10T11:16:40.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:36.975+0000 7f5a7d4a9700 1 -- 172.21.15.100:0/3122307672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a78102e00 msgr2=0x7f5a7819e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:40.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:36.975+0000 7f5a7d4a9700 1 --2- 172.21.15.100:0/3122307672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a78102e00 0x7f5a7819e5c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5a68009970 tx=0x7f5a68009c80 comp rx=0 tx=0).stop 2024-05-10T11:16:40.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:36.975+0000 7f5a777fe700 1 -- 172.21.15.100:0/3122307672 reap_dead start 2024-05-10T11:16:40.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:36.975+0000 7f5a7d4a9700 1 -- 172.21.15.100:0/3122307672 shutdown_connections 2024-05-10T11:16:40.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:36.975+0000 7f5a7d4a9700 1 -- 172.21.15.100:0/3122307672 >> 172.21.15.100:0/3122307672 conn(0x7f5a78078a10 msgr2=0x7f5a780ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:16:40.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:36.976+0000 7f5a7d4a9700 1 -- 172.21.15.100:0/3122307672 shutdown_connections 2024-05-10T11:16:40.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:36.976+0000 7f5a7d4a9700 1 -- 172.21.15.100:0/3122307672 wait complete. 2024-05-10T11:16:40.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:16:39 smithi178 ceph-mon[33359]: pgmap v368: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-05-10T11:16:40.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:16:39 smithi188 ceph-mon[34075]: pgmap v368: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-05-10T11:16:40.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:16:39 smithi100 ceph-mon[30296]: pgmap v368: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-05-10T11:16:40.688 INFO:teuthology.orchestra.run.smithi100.stdout:77309411418 2024-05-10T11:16:40.688 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-05-10T11:16:41.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.052+0000 7fcd54a6e700 1 -- 172.21.15.100:0/3361776570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd50104100 msgr2=0x7fcd50108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:41.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.052+0000 7fcd54a6e700 1 --2- 172.21.15.100:0/3361776570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd50104100 0x7fcd50108120 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fcd440099c0 tx=0x7fcd44009cd0 comp rx=0 tx=0).stop 2024-05-10T11:16:41.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.052+0000 7fcd54a6e700 1 -- 172.21.15.100:0/3361776570 shutdown_connections 2024-05-10T11:16:41.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.052+0000 7fcd54a6e700 1 --2- 172.21.15.100:0/3361776570 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd50108850 0x7fcd501103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:41.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.052+0000 7fcd54a6e700 1 --2- 172.21.15.100:0/3361776570 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd50103750 0x7fcd50103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:41.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.052+0000 7fcd54a6e700 1 --2- 172.21.15.100:0/3361776570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd50104100 0x7fcd50108120 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:41.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.052+0000 7fcd54a6e700 1 -- 172.21.15.100:0/3361776570 >> 172.21.15.100:0/3361776570 conn(0x7fcd5007a600 msgr2=0x7fcd5007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:16:41.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.052+0000 7fcd54a6e700 1 -- 172.21.15.100:0/3361776570 shutdown_connections 2024-05-10T11:16:41.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.052+0000 7fcd54a6e700 1 -- 172.21.15.100:0/3361776570 wait complete. 2024-05-10T11:16:41.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.052+0000 7fcd54a6e700 1 Processor -- start 2024-05-10T11:16:41.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.052+0000 7fcd54a6e700 1 -- start start 2024-05-10T11:16:41.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.053+0000 7fcd54a6e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd50103750 0x7fcd5019f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:41.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.053+0000 7fcd54a6e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd50104100 0x7fcd5019fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:41.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.053+0000 7fcd54a6e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd50108850 0x7fcd50199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:41.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.053+0000 7fcd54a6e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fcd501135c0 con 0x7fcd50104100 2024-05-10T11:16:41.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.053+0000 7fcd54a6e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcd50113440 con 0x7fcd50108850 2024-05-10T11:16:41.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.053+0000 7fcd54a6e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcd50113740 con 0x7fcd50103750 2024-05-10T11:16:41.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.053+0000 7fcd4dd9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd50104100 0x7fcd5019fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:41.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.053+0000 7fcd4dd9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd50104100 0x7fcd5019fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56850/0 (socket says 172.21.15.100:56850) 2024-05-10T11:16:41.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.053+0000 7fcd4dd9b700 1 -- 172.21.15.100:0/1646068671 learned_addr learned my addr 172.21.15.100:0/1646068671 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:16:41.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.053+0000 7fcd4ed9d700 1 --2- 172.21.15.100:0/1646068671 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd50108850 0x7fcd50199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:41.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.053+0000 7fcd4e59c700 1 --2- 172.21.15.100:0/1646068671 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd50103750 0x7fcd5019f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:41.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.053+0000 7fcd4dd9b700 1 -- 172.21.15.100:0/1646068671 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd50103750 msgr2=0x7fcd5019f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:41.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.053+0000 7fcd4dd9b700 1 --2- 172.21.15.100:0/1646068671 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd50103750 0x7fcd5019f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:41.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.053+0000 7fcd4dd9b700 1 -- 172.21.15.100:0/1646068671 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd50108850 msgr2=0x7fcd50199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:41.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.053+0000 7fcd4dd9b700 1 --2- 172.21.15.100:0/1646068671 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd50108850 0x7fcd50199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:41.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.053+0000 7fcd4dd9b700 1 -- 172.21.15.100:0/1646068671 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd5019a0a0 con 0x7fcd50104100 2024-05-10T11:16:41.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.053+0000 7fcd4dd9b700 1 --2- 172.21.15.100:0/1646068671 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd50104100 0x7fcd5019fcc0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fcd44000c00 tx=0x7fcd4400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:16:41.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.053+0000 7fcd3f7fe700 1 -- 172.21.15.100:0/1646068671 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd44026070 con 0x7fcd50104100 2024-05-10T11:16:41.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.054+0000 7fcd54a6e700 1 -- 172.21.15.100:0/1646068671 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd5019a330 con 0x7fcd50104100 2024-05-10T11:16:41.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.054+0000 7fcd3f7fe700 1 -- 172.21.15.100:0/1646068671 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcd44005030 con 0x7fcd50104100 2024-05-10T11:16:41.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.054+0000 7fcd3f7fe700 1 -- 172.21.15.100:0/1646068671 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd44004b70 con 0x7fcd50104100 2024-05-10T11:16:41.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.054+0000 7fcd54a6e700 1 -- 172.21.15.100:0/1646068671 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd501a6650 con 0x7fcd50104100 2024-05-10T11:16:41.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.054+0000 7fcd54a6e700 1 -- 172.21.15.100:0/1646068671 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcd50063bb0 con 0x7fcd50104100 2024-05-10T11:16:41.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.056+0000 7fcd3f7fe700 1 -- 172.21.15.100:0/1646068671 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fcd44029070 con 0x7fcd50104100 2024-05-10T11:16:41.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.057+0000 7fcd3f7fe700 1 --2- 172.21.15.100:0/1646068671 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fcd380b2910 0x7fcd380b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:41.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.057+0000 7fcd4e59c700 1 --2- 172.21.15.100:0/1646068671 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fcd380b2910 0x7fcd380b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:41.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.057+0000 7fcd3f7fe700 1 -- 172.21.15.100:0/1646068671 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fcd440e39b0 con 0x7fcd50104100 2024-05-10T11:16:41.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.057+0000 7fcd3f7fe700 1 --2- 172.21.15.100:0/1646068671 >> [v2:172.21.15.100:6826/3585761978,v1:172.21.15.100:6827/3585761978] conn(0x7fcd380bda70 0x7fcd380bff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:41.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.057+0000 7fcd3f7fe700 1 -- 172.21.15.100:0/1646068671 --> [v2:172.21.15.100:6826/3585761978,v1:172.21.15.100:6827/3585761978] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcd4402ce50 con 0x7fcd380bda70 2024-05-10T11:16:41.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.057+0000 7fcd3f7fe700 1 -- 172.21.15.100:0/1646068671 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fcd440e3e00 con 0x7fcd50104100 2024-05-10T11:16:41.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.057+0000 7fcd4ed9d700 1 --2- 172.21.15.100:0/1646068671 >> [v2:172.21.15.100:6826/3585761978,v1:172.21.15.100:6827/3585761978] conn(0x7fcd380bda70 0x7fcd380bff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:41.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.057+0000 7fcd4e59c700 1 --2- 172.21.15.100:0/1646068671 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fcd380b2910 0x7fcd380b4dd0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcd40006fd0 tx=0x7fcd40008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:16:41.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.058+0000 7fcd4ed9d700 1 --2- 172.21.15.100:0/1646068671 >> [v2:172.21.15.100:6826/3585761978,v1:172.21.15.100:6827/3585761978] conn(0x7fcd380bda70 0x7fcd380bff10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:16:41.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.058+0000 7fcd3f7fe700 1 -- 172.21.15.100:0/1646068671 <== osd.9 v2:172.21.15.100:6826/3585761978 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcd4402ce50 con 0x7fcd380bda70 2024-05-10T11:16:41.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.074+0000 7fcd54a6e700 1 -- 172.21.15.100:0/1646068671 --> [v2:172.21.15.100:6826/3585761978,v1:172.21.15.100:6827/3585761978] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcd50064030 con 0x7fcd380bda70 2024-05-10T11:16:41.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.075+0000 7fcd3f7fe700 1 -- 172.21.15.100:0/1646068671 <== osd.9 v2:172.21.15.100:6826/3585761978 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fcd50064030 con 0x7fcd380bda70 2024-05-10T11:16:41.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.075+0000 7fcd54a6e700 1 -- 172.21.15.100:0/1646068671 >> [v2:172.21.15.100:6826/3585761978,v1:172.21.15.100:6827/3585761978] conn(0x7fcd380bda70 msgr2=0x7fcd380bff10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:41.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.075+0000 7fcd54a6e700 1 --2- 172.21.15.100:0/1646068671 >> [v2:172.21.15.100:6826/3585761978,v1:172.21.15.100:6827/3585761978] conn(0x7fcd380bda70 0x7fcd380bff10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:41.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.075+0000 7fcd54a6e700 1 -- 172.21.15.100:0/1646068671 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fcd380b2910 msgr2=0x7fcd380b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:41.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.075+0000 7fcd54a6e700 1 --2- 172.21.15.100:0/1646068671 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fcd380b2910 0x7fcd380b4dd0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcd40006fd0 tx=0x7fcd40008040 comp rx=0 tx=0).stop 2024-05-10T11:16:41.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.075+0000 7fcd54a6e700 1 -- 172.21.15.100:0/1646068671 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd50104100 msgr2=0x7fcd5019fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:41.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.075+0000 7fcd54a6e700 1 --2- 172.21.15.100:0/1646068671 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd50104100 0x7fcd5019fcc0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fcd44000c00 tx=0x7fcd4400f690 comp rx=0 tx=0).stop 2024-05-10T11:16:41.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.075+0000 7fcd4ed9d700 1 -- 172.21.15.100:0/1646068671 reap_dead start 2024-05-10T11:16:41.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.075+0000 7fcd54a6e700 1 -- 172.21.15.100:0/1646068671 shutdown_connections 2024-05-10T11:16:41.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.075+0000 7fcd54a6e700 1 -- 172.21.15.100:0/1646068671 >> 172.21.15.100:0/1646068671 conn(0x7fcd5007a600 msgr2=0x7fcd50101880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:16:41.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.075+0000 7fcd54a6e700 1 -- 172.21.15.100:0/1646068671 shutdown_connections 2024-05-10T11:16:41.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.075+0000 7fcd54a6e700 1 -- 172.21.15.100:0/1646068671 wait complete. 2024-05-10T11:16:41.631 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:16:41.632 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:16:41.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.842+0000 7f7f90627700 1 -- 172.21.15.100:0/2172821411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f88102d50 msgr2=0x7f7f8810f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:41.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.842+0000 7f7f90627700 1 --2- 172.21.15.100:0/2172821411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f88102d50 0x7f7f8810f240 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7f80009990 tx=0x7f7f80009ca0 comp rx=0 tx=0).stop 2024-05-10T11:16:41.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.842+0000 7f7f90627700 1 -- 172.21.15.100:0/2172821411 shutdown_connections 2024-05-10T11:16:41.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.842+0000 7f7f90627700 1 --2- 172.21.15.100:0/2172821411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f88102d50 0x7f7f8810f240 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:41.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.842+0000 7f7f90627700 1 --2- 172.21.15.100:0/2172821411 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f88108400 0x7f7f881087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:41.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.842+0000 7f7f90627700 1 --2- 172.21.15.100:0/2172821411 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f88102390 0x7f7f88102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:41.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.842+0000 7f7f90627700 1 -- 172.21.15.100:0/2172821411 >> 172.21.15.100:0/2172821411 conn(0x7f7f88079310 msgr2=0x7f7f88079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:16:41.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.842+0000 7f7f90627700 1 -- 172.21.15.100:0/2172821411 shutdown_connections 2024-05-10T11:16:41.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.842+0000 7f7f90627700 1 -- 172.21.15.100:0/2172821411 wait complete. 2024-05-10T11:16:41.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.843+0000 7f7f90627700 1 Processor -- start 2024-05-10T11:16:41.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.843+0000 7f7f90627700 1 -- start start 2024-05-10T11:16:41.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.843+0000 7f7f90627700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f88102390 0x7f7f8819f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:41.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.843+0000 7f7f90627700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f88102d50 0x7f7f8819fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:41.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.843+0000 7f7f90627700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f88108400 0x7f7f88199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:41.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.843+0000 7f7f90627700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7f88112400 con 0x7f7f88108400 2024-05-10T11:16:41.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.843+0000 7f7f90627700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7f88112280 con 0x7f7f88102d50 2024-05-10T11:16:41.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.843+0000 7f7f90627700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7f88112580 con 0x7f7f88102390 2024-05-10T11:16:41.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.843+0000 7f7f8ebc4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f88108400 0x7f7f88199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:41.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.843+0000 7f7f8ebc4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f88108400 0x7f7f88199830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56870/0 (socket says 172.21.15.100:56870) 2024-05-10T11:16:41.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.843+0000 7f7f8ebc4700 1 -- 172.21.15.100:0/137666050 learned_addr learned my addr 172.21.15.100:0/137666050 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:16:41.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.843+0000 7f7f8dbc2700 1 --2- 172.21.15.100:0/137666050 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f88102d50 0x7f7f8819fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:41.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.843+0000 7f7f8e3c3700 1 --2- 172.21.15.100:0/137666050 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f88102390 0x7f7f8819f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:41.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.843+0000 7f7f8ebc4700 1 -- 172.21.15.100:0/137666050 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f88102390 msgr2=0x7f7f8819f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:41.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.843+0000 7f7f8ebc4700 1 --2- 172.21.15.100:0/137666050 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f88102390 0x7f7f8819f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:41.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.843+0000 7f7f8ebc4700 1 -- 172.21.15.100:0/137666050 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f88102d50 msgr2=0x7f7f8819fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:41.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.843+0000 7f7f8ebc4700 1 --2- 172.21.15.100:0/137666050 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f88102d50 0x7f7f8819fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:41.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.843+0000 7f7f8ebc4700 1 -- 172.21.15.100:0/137666050 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f8819a010 con 0x7f7f88108400 2024-05-10T11:16:41.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.843+0000 7f7f8ebc4700 1 --2- 172.21.15.100:0/137666050 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f88108400 0x7f7f88199830 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7f800047b0 tx=0x7f7f80005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:16:41.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.844+0000 7f7f7f7fe700 1 -- 172.21.15.100:0/137666050 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f80027070 con 0x7f7f88108400 2024-05-10T11:16:41.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.844+0000 7f7f7f7fe700 1 -- 172.21.15.100:0/137666050 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7f8000fb00 con 0x7f7f88108400 2024-05-10T11:16:41.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.844+0000 7f7f90627700 1 -- 172.21.15.100:0/137666050 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f8819a2a0 con 0x7f7f88108400 2024-05-10T11:16:41.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.844+0000 7f7f7f7fe700 1 -- 172.21.15.100:0/137666050 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f80022b20 con 0x7f7f88108400 2024-05-10T11:16:41.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.844+0000 7f7f90627700 1 -- 172.21.15.100:0/137666050 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f881a6510 con 0x7f7f88108400 2024-05-10T11:16:41.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.844+0000 7f7f90627700 1 -- 172.21.15.100:0/137666050 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7f8806bc50 con 0x7f7f88108400 2024-05-10T11:16:41.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.847+0000 7f7f7f7fe700 1 -- 172.21.15.100:0/137666050 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f7f8002a070 con 0x7f7f88108400 2024-05-10T11:16:41.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.848+0000 7f7f7f7fe700 1 --2- 172.21.15.100:0/137666050 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f7f740b2910 0x7f7f740b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:41.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.848+0000 7f7f8e3c3700 1 --2- 172.21.15.100:0/137666050 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f7f740b2910 0x7f7f740b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:41.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.848+0000 7f7f7f7fe700 1 -- 172.21.15.100:0/137666050 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f7f800e4100 con 0x7f7f88108400 2024-05-10T11:16:41.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.848+0000 7f7f7f7fe700 1 --2- 172.21.15.100:0/137666050 >> [v2:172.21.15.188:6816/1523872826,v1:172.21.15.188:6817/1523872826] conn(0x7f7f740bda70 0x7f7f740bff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:41.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.848+0000 7f7f7f7fe700 1 -- 172.21.15.100:0/137666050 --> [v2:172.21.15.188:6816/1523872826,v1:172.21.15.188:6817/1523872826] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7f80030e80 con 0x7f7f740bda70 2024-05-10T11:16:41.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.848+0000 7f7f7f7fe700 1 -- 172.21.15.100:0/137666050 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f7f800e4550 con 0x7f7f88108400 2024-05-10T11:16:41.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.848+0000 7f7f8e3c3700 1 --2- 172.21.15.100:0/137666050 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f7f740b2910 0x7f7f740b4dd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7f84007fd0 tx=0x7f7f840079e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:16:41.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.848+0000 7f7f8dbc2700 1 --2- 172.21.15.100:0/137666050 >> [v2:172.21.15.188:6816/1523872826,v1:172.21.15.188:6817/1523872826] conn(0x7f7f740bda70 0x7f7f740bff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:41.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.849+0000 7f7f8dbc2700 1 --2- 172.21.15.100:0/137666050 >> [v2:172.21.15.188:6816/1523872826,v1:172.21.15.188:6817/1523872826] conn(0x7f7f740bda70 0x7f7f740bff10 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:16:41.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.850+0000 7f7f7f7fe700 1 -- 172.21.15.100:0/137666050 <== osd.8 v2:172.21.15.188:6816/1523872826 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7f80030e80 con 0x7f7f740bda70 2024-05-10T11:16:41.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.869+0000 7f7f90627700 1 -- 172.21.15.100:0/137666050 --> [v2:172.21.15.188:6816/1523872826,v1:172.21.15.188:6817/1523872826] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7f88103ab0 con 0x7f7f740bda70 2024-05-10T11:16:41.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.870+0000 7f7f7f7fe700 1 -- 172.21.15.100:0/137666050 <== osd.8 v2:172.21.15.188:6816/1523872826 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7f88103ab0 con 0x7f7f740bda70 2024-05-10T11:16:41.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.870+0000 7f7f90627700 1 -- 172.21.15.100:0/137666050 >> [v2:172.21.15.188:6816/1523872826,v1:172.21.15.188:6817/1523872826] conn(0x7f7f740bda70 msgr2=0x7f7f740bff10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:41.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.870+0000 7f7f90627700 1 --2- 172.21.15.100:0/137666050 >> [v2:172.21.15.188:6816/1523872826,v1:172.21.15.188:6817/1523872826] conn(0x7f7f740bda70 0x7f7f740bff10 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:41.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.871+0000 7f7f90627700 1 -- 172.21.15.100:0/137666050 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f7f740b2910 msgr2=0x7f7f740b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:41.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.871+0000 7f7f90627700 1 --2- 172.21.15.100:0/137666050 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f7f740b2910 0x7f7f740b4dd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7f84007fd0 tx=0x7f7f840079e0 comp rx=0 tx=0).stop 2024-05-10T11:16:41.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.871+0000 7f7f90627700 1 -- 172.21.15.100:0/137666050 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f88108400 msgr2=0x7f7f88199830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:41.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.871+0000 7f7f90627700 1 --2- 172.21.15.100:0/137666050 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f88108400 0x7f7f88199830 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7f800047b0 tx=0x7f7f80005e80 comp rx=0 tx=0).stop 2024-05-10T11:16:41.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.871+0000 7f7f8ebc4700 1 -- 172.21.15.100:0/137666050 reap_dead start 2024-05-10T11:16:41.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.871+0000 7f7f90627700 1 -- 172.21.15.100:0/137666050 shutdown_connections 2024-05-10T11:16:41.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.871+0000 7f7f90627700 1 -- 172.21.15.100:0/137666050 >> 172.21.15.100:0/137666050 conn(0x7f7f88079310 msgr2=0x7f7f881016b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:16:41.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.871+0000 7f7f90627700 1 -- 172.21.15.100:0/137666050 shutdown_connections 2024-05-10T11:16:41.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:41.871+0000 7f7f90627700 1 -- 172.21.15.100:0/137666050 wait complete. 2024-05-10T11:16:42.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:16:41 smithi100 ceph-mon[30296]: pgmap v369: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-10T11:16:42.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:16:41 smithi178 ceph-mon[33359]: pgmap v369: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-10T11:16:42.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:16:41 smithi188 ceph-mon[34075]: pgmap v369: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-10T11:16:42.385 INFO:teuthology.orchestra.run.smithi100.stdout:111669149780 2024-05-10T11:16:42.385 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-05-10T11:16:43.250 INFO:teuthology.orchestra.run.smithi100.stdout:236223201339 2024-05-10T11:16:43.251 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-05-10T11:16:44.064 INFO:teuthology.orchestra.run.smithi100.stdout:261993005111 2024-05-10T11:16:44.065 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-05-10T11:16:44.247 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:16:43 smithi100 ceph-mon[30296]: pgmap v370: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T11:16:44.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:16:43 smithi178 ceph-mon[33359]: pgmap v370: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T11:16:44.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:16:43 smithi188 ceph-mon[34075]: pgmap v370: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T11:16:45.797 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:16:45.798 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:16:46.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:16:45 smithi100 ceph-mon[30296]: pgmap v371: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-10T11:16:46.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:16:45 smithi178 ceph-mon[33359]: pgmap v371: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-10T11:16:46.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:16:45 smithi188 ceph-mon[34075]: pgmap v371: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-10T11:16:47.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:46.937+0000 7fe505380700 1 -- 172.21.15.100:0/3028659160 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe50006b180 msgr2=0x7fe50006b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:47.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:46.937+0000 7fe505380700 1 --2- 172.21.15.100:0/3028659160 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe50006b180 0x7fe50006b600 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8009990 tx=0x7fe4e8009ca0 comp rx=0 tx=0).stop 2024-05-10T11:16:47.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:46.937+0000 7fe505380700 1 -- 172.21.15.100:0/3028659160 shutdown_connections 2024-05-10T11:16:47.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:46.937+0000 7fe505380700 1 --2- 172.21.15.100:0/3028659160 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe50006bb40 0x7fe500110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:47.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:46.937+0000 7fe505380700 1 --2- 172.21.15.100:0/3028659160 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe500109510 0x7fe5001098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:47.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:46.937+0000 7fe505380700 1 --2- 172.21.15.100:0/3028659160 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe50006b180 0x7fe50006b600 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:47.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:46.937+0000 7fe505380700 1 -- 172.21.15.100:0/3028659160 >> 172.21.15.100:0/3028659160 conn(0x7fe500079350 msgr2=0x7fe500079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:16:47.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:46.937+0000 7fe505380700 1 -- 172.21.15.100:0/3028659160 shutdown_connections 2024-05-10T11:16:47.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:46.937+0000 7fe505380700 1 -- 172.21.15.100:0/3028659160 wait complete. 2024-05-10T11:16:47.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:46.938+0000 7fe505380700 1 Processor -- start 2024-05-10T11:16:47.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:46.938+0000 7fe505380700 1 -- start start 2024-05-10T11:16:47.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:46.938+0000 7fe505380700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe50006b180 0x7fe50019f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:47.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:46.938+0000 7fe505380700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe50006bb40 0x7fe50019fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:47.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:46.938+0000 7fe505380700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe500109510 0x7fe5001999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:47.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:46.938+0000 7fe505380700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe500113570 con 0x7fe50006b180 2024-05-10T11:16:47.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:46.938+0000 7fe505380700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe5001133f0 con 0x7fe50006bb40 2024-05-10T11:16:47.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:46.938+0000 7fe505380700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe5001136f0 con 0x7fe500109510 2024-05-10T11:16:47.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:46.938+0000 7fe4feffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe50006b180 0x7fe50019f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:47.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:46.938+0000 7fe4feffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe50006b180 0x7fe50019f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36598/0 (socket says 172.21.15.100:36598) 2024-05-10T11:16:47.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:46.938+0000 7fe4feffd700 1 -- 172.21.15.100:0/309071698 learned_addr learned my addr 172.21.15.100:0/309071698 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:16:47.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:46.938+0000 7fe4ff7fe700 1 --2- 172.21.15.100:0/309071698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe500109510 0x7fe5001999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:47.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:46.939+0000 7fe4f67fc700 1 --2- 172.21.15.100:0/309071698 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe50006bb40 0x7fe50019fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:47.356 INFO:teuthology.orchestra.run.smithi100.stdout:309237645357 2024-05-10T11:16:47.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:46.939+0000 7fe4feffd700 1 -- 172.21.15.100:0/309071698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe500109510 msgr2=0x7fe5001999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:47.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:46.939+0000 7fe4feffd700 1 --2- 172.21.15.100:0/309071698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe500109510 0x7fe5001999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:47.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:46.939+0000 7fe4feffd700 1 -- 172.21.15.100:0/309071698 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe50006bb40 msgr2=0x7fe50019fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:47.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:46.939+0000 7fe4feffd700 1 --2- 172.21.15.100:0/309071698 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe50006bb40 0x7fe50019fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:47.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:46.939+0000 7fe4feffd700 1 -- 172.21.15.100:0/309071698 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe50019a140 con 0x7fe50006b180 2024-05-10T11:16:47.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:46.939+0000 7fe4feffd700 1 --2- 172.21.15.100:0/309071698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe50006b180 0x7fe50019f820 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe4f000b530 tx=0x7fe4f000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:16:47.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:46.939+0000 7fe4fcff9700 1 -- 172.21.15.100:0/309071698 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f0015020 con 0x7fe50006b180 2024-05-10T11:16:47.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:46.939+0000 7fe4fcff9700 1 -- 172.21.15.100:0/309071698 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe4f0011cc0 con 0x7fe50006b180 2024-05-10T11:16:47.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:46.939+0000 7fe505380700 1 -- 172.21.15.100:0/309071698 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe50019a430 con 0x7fe50006b180 2024-05-10T11:16:47.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:46.939+0000 7fe4fcff9700 1 -- 172.21.15.100:0/309071698 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f0010dc0 con 0x7fe50006b180 2024-05-10T11:16:47.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:46.939+0000 7fe505380700 1 -- 172.21.15.100:0/309071698 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5001023a0 con 0x7fe50006b180 2024-05-10T11:16:47.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:46.940+0000 7fe505380700 1 -- 172.21.15.100:0/309071698 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe50010b9f0 con 0x7fe50006b180 2024-05-10T11:16:47.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:46.942+0000 7fe4fcff9700 1 -- 172.21.15.100:0/309071698 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fe4f0018020 con 0x7fe50006b180 2024-05-10T11:16:47.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:46.943+0000 7fe4fcff9700 1 --2- 172.21.15.100:0/309071698 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe4ec0b2910 0x7fe4ec0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:47.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:46.943+0000 7fe4f67fc700 1 --2- 172.21.15.100:0/309071698 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe4ec0b2910 0x7fe4ec0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:47.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:46.943+0000 7fe4fcff9700 1 -- 172.21.15.100:0/309071698 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fe4f00e1e60 con 0x7fe50006b180 2024-05-10T11:16:47.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:46.943+0000 7fe4f67fc700 1 --2- 172.21.15.100:0/309071698 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe4ec0b2910 0x7fe4ec0b4dd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8005850 tx=0x7fe4e801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:16:47.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:46.947+0000 7fe4fcff9700 1 -- 172.21.15.100:0/309071698 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fe4f001f020 con 0x7fe50006b180 2024-05-10T11:16:47.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:47.160+0000 7fe505380700 1 -- 172.21.15.100:0/309071698 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fe50019af10 con 0x7fe50006b180 2024-05-10T11:16:47.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:47.160+0000 7fe4fcff9700 1 -- 172.21.15.100:0/309071698 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fe4f00a97d0 con 0x7fe50006b180 2024-05-10T11:16:47.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:47.162+0000 7fe505380700 1 -- 172.21.15.100:0/309071698 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe4ec0b2910 msgr2=0x7fe4ec0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:47.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:47.162+0000 7fe505380700 1 --2- 172.21.15.100:0/309071698 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe4ec0b2910 0x7fe4ec0b4dd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8005850 tx=0x7fe4e801a040 comp rx=0 tx=0).stop 2024-05-10T11:16:47.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:47.162+0000 7fe505380700 1 -- 172.21.15.100:0/309071698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe50006b180 msgr2=0x7fe50019f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:47.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:47.162+0000 7fe505380700 1 --2- 172.21.15.100:0/309071698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe50006b180 0x7fe50019f820 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe4f000b530 tx=0x7fe4f000b840 comp rx=0 tx=0).stop 2024-05-10T11:16:47.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:47.163+0000 7fe505380700 1 -- 172.21.15.100:0/309071698 shutdown_connections 2024-05-10T11:16:47.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:47.163+0000 7fe505380700 1 --2- 172.21.15.100:0/309071698 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe4ec0b2910 0x7fe4ec0b4dd0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:47.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:47.163+0000 7fe505380700 1 --2- 172.21.15.100:0/309071698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe500109510 0x7fe5001999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:47.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:47.163+0000 7fe505380700 1 --2- 172.21.15.100:0/309071698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe50006b180 0x7fe50019f820 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:47.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:47.163+0000 7fe505380700 1 --2- 172.21.15.100:0/309071698 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe50006bb40 0x7fe50019fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:47.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:47.163+0000 7fe505380700 1 -- 172.21.15.100:0/309071698 >> 172.21.15.100:0/309071698 conn(0x7fe500079350 msgr2=0x7fe500101220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:16:47.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:47.163+0000 7fe505380700 1 -- 172.21.15.100:0/309071698 shutdown_connections 2024-05-10T11:16:47.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:47.163+0000 7fe505380700 1 -- 172.21.15.100:0/309071698 wait complete. 2024-05-10T11:16:47.814 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:16:47.815 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:16:47.815 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:16:47.816 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:16:48.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:16:47 smithi178 ceph-mon[33359]: pgmap v372: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T11:16:48.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:16:47 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/309071698' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-10T11:16:48.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:16:47 smithi188 ceph-mon[34075]: pgmap v372: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T11:16:48.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:16:47 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/309071698' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-10T11:16:48.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:16:47 smithi100 ceph-mon[30296]: pgmap v372: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T11:16:48.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:16:47 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/309071698' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-10T11:16:48.708 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645353 got 309237645357 for osd.11 2024-05-10T11:16:48.708 DEBUG:teuthology.parallel:result is None 2024-05-10T11:16:50.176 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:16:49 smithi100 ceph-mon[30296]: pgmap v373: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T11:16:50.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:16:49 smithi178 ceph-mon[33359]: pgmap v373: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T11:16:50.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:16:49 smithi188 ceph-mon[34075]: pgmap v373: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T11:16:50.469 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:16:50.469 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:16:50.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:50.478+0000 7f862a6db700 1 -- 172.21.15.100:0/3595364917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8624108400 msgr2=0x7f86241087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:50.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:50.478+0000 7f862a6db700 1 --2- 172.21.15.100:0/3595364917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8624108400 0x7f86241087e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8614009990 tx=0x7f8614009ca0 comp rx=0 tx=0).stop 2024-05-10T11:16:50.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:50.478+0000 7f862a6db700 1 -- 172.21.15.100:0/3595364917 shutdown_connections 2024-05-10T11:16:50.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:50.478+0000 7f862a6db700 1 --2- 172.21.15.100:0/3595364917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8624102d50 0x7f862410f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:50.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:50.478+0000 7f862a6db700 1 --2- 172.21.15.100:0/3595364917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8624108400 0x7f86241087e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:50.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:50.478+0000 7f862a6db700 1 --2- 172.21.15.100:0/3595364917 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8624102390 0x7f8624102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:50.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:50.478+0000 7f862a6db700 1 -- 172.21.15.100:0/3595364917 >> 172.21.15.100:0/3595364917 conn(0x7f8624079310 msgr2=0x7f8624079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:16:50.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:50.478+0000 7f862a6db700 1 -- 172.21.15.100:0/3595364917 shutdown_connections 2024-05-10T11:16:50.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:50.478+0000 7f862a6db700 1 -- 172.21.15.100:0/3595364917 wait complete. 2024-05-10T11:16:50.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:50.479+0000 7f862a6db700 1 Processor -- start 2024-05-10T11:16:50.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:50.479+0000 7f862a6db700 1 -- start start 2024-05-10T11:16:50.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:50.479+0000 7f862a6db700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8624102390 0x7f862419e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:50.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:50.479+0000 7f862a6db700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8624102d50 0x7f862419ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:50.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:50.479+0000 7f862a6db700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8624108400 0x7f8624198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:50.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:50.479+0000 7f862a6db700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f86241123d0 con 0x7f8624102d50 2024-05-10T11:16:50.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:50.479+0000 7f862a6db700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8624112250 con 0x7f8624108400 2024-05-10T11:16:50.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:50.479+0000 7f862a6db700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8624112550 con 0x7f8624102390 2024-05-10T11:16:50.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:50.479+0000 7f86237fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8624102d50 0x7f862419ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:50.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:50.479+0000 7f86237fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8624102d50 0x7f862419ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36618/0 (socket says 172.21.15.100:36618) 2024-05-10T11:16:50.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:50.479+0000 7f86237fe700 1 -- 172.21.15.100:0/540495719 learned_addr learned my addr 172.21.15.100:0/540495719 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:16:50.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:50.479+0000 7f8628c78700 1 --2- 172.21.15.100:0/540495719 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8624108400 0x7f8624198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:50.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:50.480+0000 7f86237fe700 1 -- 172.21.15.100:0/540495719 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8624102390 msgr2=0x7f862419e680 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T11:16:50.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:50.480+0000 7f86237fe700 1 --2- 172.21.15.100:0/540495719 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8624102390 0x7f862419e680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:50.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:50.480+0000 7f86237fe700 1 -- 172.21.15.100:0/540495719 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8624108400 msgr2=0x7f8624198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:50.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:50.480+0000 7f86237fe700 1 --2- 172.21.15.100:0/540495719 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8624108400 0x7f8624198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:50.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:50.480+0000 7f86237fe700 1 -- 172.21.15.100:0/540495719 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8624199030 con 0x7f8624102d50 2024-05-10T11:16:50.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:50.480+0000 7f86237fe700 1 --2- 172.21.15.100:0/540495719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8624102d50 0x7f862419ebc0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f861800c9b0 tx=0x7f861800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:16:50.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:50.480+0000 7f86217fa700 1 -- 172.21.15.100:0/540495719 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8618007b00 con 0x7f8624102d50 2024-05-10T11:16:50.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:50.480+0000 7f86217fa700 1 -- 172.21.15.100:0/540495719 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8618007ca0 con 0x7f8624102d50 2024-05-10T11:16:50.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:50.480+0000 7f862a6db700 1 -- 172.21.15.100:0/540495719 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8624199320 con 0x7f8624102d50 2024-05-10T11:16:50.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:50.480+0000 7f86217fa700 1 -- 172.21.15.100:0/540495719 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8618018660 con 0x7f8624102d50 2024-05-10T11:16:50.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:50.480+0000 7f862a6db700 1 -- 172.21.15.100:0/540495719 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86241a5600 con 0x7f8624102d50 2024-05-10T11:16:50.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:50.481+0000 7f862a6db700 1 -- 172.21.15.100:0/540495719 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8624103ab0 con 0x7f8624102d50 2024-05-10T11:16:50.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:50.484+0000 7f86217fa700 1 -- 172.21.15.100:0/540495719 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f861800f450 con 0x7f8624102d50 2024-05-10T11:16:50.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:50.484+0000 7f86217fa700 1 --2- 172.21.15.100:0/540495719 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f860c0b2910 0x7f860c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:50.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:50.485+0000 7f86217fa700 1 -- 172.21.15.100:0/540495719 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f86180e2a30 con 0x7f8624102d50 2024-05-10T11:16:50.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:50.485+0000 7f8623fff700 1 --2- 172.21.15.100:0/540495719 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f860c0b2910 0x7f860c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:50.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:50.485+0000 7f8623fff700 1 --2- 172.21.15.100:0/540495719 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f860c0b2910 0x7f860c0b4dd0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8614005b40 tx=0x7f8614005ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:16:50.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:50.489+0000 7f86217fa700 1 -- 172.21.15.100:0/540495719 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f8618014030 con 0x7f8624102d50 2024-05-10T11:16:50.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:50.705+0000 7f862a6db700 1 -- 172.21.15.100:0/540495719 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f8624199c80 con 0x7f8624102d50 2024-05-10T11:16:50.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:50.705+0000 7f86217fa700 1 -- 172.21.15.100:0/540495719 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f86180aa290 con 0x7f8624102d50 2024-05-10T11:16:50.706 INFO:teuthology.orchestra.run.smithi100.stdout:283467841586 2024-05-10T11:16:50.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:50.708+0000 7f862a6db700 1 -- 172.21.15.100:0/540495719 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f860c0b2910 msgr2=0x7f860c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:50.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:50.708+0000 7f862a6db700 1 --2- 172.21.15.100:0/540495719 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f860c0b2910 0x7f860c0b4dd0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8614005b40 tx=0x7f8614005ab0 comp rx=0 tx=0).stop 2024-05-10T11:16:50.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:50.708+0000 7f862a6db700 1 -- 172.21.15.100:0/540495719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8624102d50 msgr2=0x7f862419ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:50.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:50.708+0000 7f862a6db700 1 --2- 172.21.15.100:0/540495719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8624102d50 0x7f862419ebc0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f861800c9b0 tx=0x7f861800ccc0 comp rx=0 tx=0).stop 2024-05-10T11:16:50.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:50.708+0000 7f862a6db700 1 -- 172.21.15.100:0/540495719 shutdown_connections 2024-05-10T11:16:50.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:50.708+0000 7f862a6db700 1 --2- 172.21.15.100:0/540495719 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f860c0b2910 0x7f860c0b4dd0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:50.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:50.708+0000 7f862a6db700 1 --2- 172.21.15.100:0/540495719 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8624108400 0x7f8624198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:50.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:50.708+0000 7f862a6db700 1 --2- 172.21.15.100:0/540495719 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8624102390 0x7f862419e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:50.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:50.708+0000 7f862a6db700 1 --2- 172.21.15.100:0/540495719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8624102d50 0x7f862419ebc0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:50.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:50.708+0000 7f862a6db700 1 -- 172.21.15.100:0/540495719 >> 172.21.15.100:0/540495719 conn(0x7f8624079310 msgr2=0x7f8624101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:16:50.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:50.708+0000 7f862a6db700 1 -- 172.21.15.100:0/540495719 shutdown_connections 2024-05-10T11:16:50.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:50.708+0000 7f862a6db700 1 -- 172.21.15.100:0/540495719 wait complete. 2024-05-10T11:16:51.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:16:50 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/540495719' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-10T11:16:51.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:16:50 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/540495719' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-10T11:16:51.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:16:50 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/540495719' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-10T11:16:52.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:16:51 smithi178 ceph-mon[33359]: pgmap v374: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-05-10T11:16:52.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:16:51 smithi188 ceph-mon[34075]: pgmap v374: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-05-10T11:16:52.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:16:51 smithi100 ceph-mon[30296]: pgmap v374: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-05-10T11:16:54.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:16:53 smithi178 ceph-mon[33359]: pgmap v375: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T11:16:54.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:16:53 smithi188 ceph-mon[34075]: pgmap v375: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T11:16:54.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:16:53 smithi100 ceph-mon[30296]: pgmap v375: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T11:16:54.781 INFO:tasks.cephadm.ceph_manager.ceph:need seq 283467841581 got 283467841586 for osd.10 2024-05-10T11:16:54.781 DEBUG:teuthology.parallel:result is None 2024-05-10T11:16:56.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:16:55 smithi178 ceph-mon[33359]: pgmap v376: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-10T11:16:56.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:16:55 smithi188 ceph-mon[34075]: pgmap v376: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-10T11:16:56.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:16:55 smithi100 ceph-mon[30296]: pgmap v376: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-10T11:16:56.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.550+0000 7fbb20129700 1 -- 172.21.15.100:0/131149001 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb18104180 msgr2=0x7fbb181103f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:56.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.550+0000 7fbb20129700 1 --2- 172.21.15.100:0/131149001 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb18104180 0x7fbb181103f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fbb1000b0a0 tx=0x7fbb1000b3b0 comp rx=0 tx=0).stop 2024-05-10T11:16:56.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.550+0000 7fbb20129700 1 -- 172.21.15.100:0/131149001 shutdown_connections 2024-05-10T11:16:56.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.550+0000 7fbb20129700 1 --2- 172.21.15.100:0/131149001 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb18104180 0x7fbb181103f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:56.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.550+0000 7fbb20129700 1 --2- 172.21.15.100:0/131149001 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb18109660 0x7fbb18109a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:56.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.550+0000 7fbb20129700 1 --2- 172.21.15.100:0/131149001 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb181037c0 0x7fbb18103c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:56.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.550+0000 7fbb20129700 1 -- 172.21.15.100:0/131149001 >> 172.21.15.100:0/131149001 conn(0x7fbb1807b430 msgr2=0x7fbb1807b840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:16:56.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.550+0000 7fbb20129700 1 -- 172.21.15.100:0/131149001 shutdown_connections 2024-05-10T11:16:56.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.551+0000 7fbb20129700 1 -- 172.21.15.100:0/131149001 wait complete. 2024-05-10T11:16:56.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.551+0000 7fbb20129700 1 Processor -- start 2024-05-10T11:16:56.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.551+0000 7fbb20129700 1 -- start start 2024-05-10T11:16:56.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.551+0000 7fbb20129700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb181037c0 0x7fbb181a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:56.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.551+0000 7fbb20129700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb18104180 0x7fbb181a1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:56.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.551+0000 7fbb20129700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb18109660 0x7fbb1819ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:56.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.552+0000 7fbb20129700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbb18113630 con 0x7fbb181037c0 2024-05-10T11:16:56.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.552+0000 7fbb20129700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbb181134b0 con 0x7fbb18109660 2024-05-10T11:16:56.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.552+0000 7fbb20129700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbb181137b0 con 0x7fbb18104180 2024-05-10T11:16:56.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.552+0000 7fbb1dec5700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb181037c0 0x7fbb181a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:56.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.552+0000 7fbb1dec5700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb181037c0 0x7fbb181a1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37618/0 (socket says 172.21.15.100:37618) 2024-05-10T11:16:56.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.552+0000 7fbb1dec5700 1 -- 172.21.15.100:0/1336872575 learned_addr learned my addr 172.21.15.100:0/1336872575 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:16:56.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.552+0000 7fbb1e6c6700 1 --2- 172.21.15.100:0/1336872575 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb18109660 0x7fbb1819ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:56.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.552+0000 7fbb1d6c4700 1 --2- 172.21.15.100:0/1336872575 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb18104180 0x7fbb181a1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:56.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.552+0000 7fbb1dec5700 1 -- 172.21.15.100:0/1336872575 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb18104180 msgr2=0x7fbb181a1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:56.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.552+0000 7fbb1dec5700 1 --2- 172.21.15.100:0/1336872575 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb18104180 0x7fbb181a1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:56.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.552+0000 7fbb1dec5700 1 -- 172.21.15.100:0/1336872575 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb18109660 msgr2=0x7fbb1819ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:56.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.552+0000 7fbb1dec5700 1 --2- 172.21.15.100:0/1336872575 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb18109660 0x7fbb1819ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:56.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.552+0000 7fbb1dec5700 1 -- 172.21.15.100:0/1336872575 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb1819c190 con 0x7fbb181037c0 2024-05-10T11:16:56.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.552+0000 7fbb1dec5700 1 --2- 172.21.15.100:0/1336872575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb181037c0 0x7fbb181a1870 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fbb1400d980 tx=0x7fbb1400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:16:56.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.552+0000 7fbb0effd700 1 -- 172.21.15.100:0/1336872575 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb14017070 con 0x7fbb181037c0 2024-05-10T11:16:56.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.552+0000 7fbb0effd700 1 -- 172.21.15.100:0/1336872575 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbb14005020 con 0x7fbb181037c0 2024-05-10T11:16:56.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.552+0000 7fbb20129700 1 -- 172.21.15.100:0/1336872575 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb1819c420 con 0x7fbb181037c0 2024-05-10T11:16:56.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.553+0000 7fbb20129700 1 -- 172.21.15.100:0/1336872575 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb181a8740 con 0x7fbb181037c0 2024-05-10T11:16:56.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.553+0000 7fbb0effd700 1 -- 172.21.15.100:0/1336872575 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb14009ab0 con 0x7fbb181037c0 2024-05-10T11:16:56.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.554+0000 7fbb20129700 1 -- 172.21.15.100:0/1336872575 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbafc005330 con 0x7fbb181037c0 2024-05-10T11:16:56.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.555+0000 7fbb0effd700 1 -- 172.21.15.100:0/1336872575 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fbb14009c50 con 0x7fbb181037c0 2024-05-10T11:16:56.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.556+0000 7fbb0effd700 1 --2- 172.21.15.100:0/1336872575 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fbb040b2a60 0x7fbb040b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:56.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.556+0000 7fbb1d6c4700 1 --2- 172.21.15.100:0/1336872575 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fbb040b2a60 0x7fbb040b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:56.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.556+0000 7fbb0effd700 1 -- 172.21.15.100:0/1336872575 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fbb140e2410 con 0x7fbb181037c0 2024-05-10T11:16:56.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.556+0000 7fbb1d6c4700 1 --2- 172.21.15.100:0/1336872575 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fbb040b2a60 0x7fbb040b4f20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbb08005950 tx=0x7fbb080058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:16:56.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.561+0000 7fbb0effd700 1 -- 172.21.15.100:0/1336872575 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fbb14020020 con 0x7fbb181037c0 2024-05-10T11:16:56.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.775+0000 7fbb20129700 1 -- 172.21.15.100:0/1336872575 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fbafc0058f0 con 0x7fbb181037c0 2024-05-10T11:16:56.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.776+0000 7fbb0effd700 1 -- 172.21.15.100:0/1336872575 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fbb140a9c70 con 0x7fbb181037c0 2024-05-10T11:16:56.776 INFO:teuthology.orchestra.run.smithi100.stdout:158913790029 2024-05-10T11:16:56.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.778+0000 7fbb20129700 1 -- 172.21.15.100:0/1336872575 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fbb040b2a60 msgr2=0x7fbb040b4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:56.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.778+0000 7fbb20129700 1 --2- 172.21.15.100:0/1336872575 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fbb040b2a60 0x7fbb040b4f20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbb08005950 tx=0x7fbb080058e0 comp rx=0 tx=0).stop 2024-05-10T11:16:56.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.778+0000 7fbb20129700 1 -- 172.21.15.100:0/1336872575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb181037c0 msgr2=0x7fbb181a1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:56.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.778+0000 7fbb20129700 1 --2- 172.21.15.100:0/1336872575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb181037c0 0x7fbb181a1870 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fbb1400d980 tx=0x7fbb1400dc90 comp rx=0 tx=0).stop 2024-05-10T11:16:56.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.778+0000 7fbb20129700 1 -- 172.21.15.100:0/1336872575 shutdown_connections 2024-05-10T11:16:56.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.778+0000 7fbb20129700 1 --2- 172.21.15.100:0/1336872575 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fbb040b2a60 0x7fbb040b4f20 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:56.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.778+0000 7fbb20129700 1 --2- 172.21.15.100:0/1336872575 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb18109660 0x7fbb1819ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:56.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.778+0000 7fbb20129700 1 --2- 172.21.15.100:0/1336872575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb181037c0 0x7fbb181a1870 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:56.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.778+0000 7fbb20129700 1 --2- 172.21.15.100:0/1336872575 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb18104180 0x7fbb181a1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:56.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.778+0000 7fbb20129700 1 -- 172.21.15.100:0/1336872575 >> 172.21.15.100:0/1336872575 conn(0x7fbb1807b430 msgr2=0x7fbb1810ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:16:56.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.778+0000 7fbb20129700 1 -- 172.21.15.100:0/1336872575 shutdown_connections 2024-05-10T11:16:56.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.778+0000 7fbb20129700 1 -- 172.21.15.100:0/1336872575 wait complete. 2024-05-10T11:16:56.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.805+0000 7fef28df1700 1 -- 172.21.15.100:0/3950703564 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef24102e00 msgr2=0x7fef241031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:56.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.805+0000 7fef28df1700 1 --2- 172.21.15.100:0/3950703564 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef24102e00 0x7fef241031e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fef140099c0 tx=0x7fef14009cd0 comp rx=0 tx=0).stop 2024-05-10T11:16:56.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.805+0000 7fef28df1700 1 -- 172.21.15.100:0/3950703564 shutdown_connections 2024-05-10T11:16:56.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.805+0000 7fef28df1700 1 --2- 172.21.15.100:0/3950703564 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef24107760 0x7fef2410f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:56.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.805+0000 7fef28df1700 1 --2- 172.21.15.100:0/3950703564 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef24102e00 0x7fef241031e0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:56.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.805+0000 7fef28df1700 1 --2- 172.21.15.100:0/3950703564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef241037b0 0x7fef24107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:56.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.805+0000 7fef28df1700 1 -- 172.21.15.100:0/3950703564 >> 172.21.15.100:0/3950703564 conn(0x7fef24078a10 msgr2=0x7fef240ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:16:56.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.805+0000 7fef28df1700 1 -- 172.21.15.100:0/3950703564 shutdown_connections 2024-05-10T11:16:56.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.805+0000 7fef28df1700 1 -- 172.21.15.100:0/3950703564 wait complete. 2024-05-10T11:16:56.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.806+0000 7fef28df1700 1 Processor -- start 2024-05-10T11:16:56.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.806+0000 7fef28df1700 1 -- start start 2024-05-10T11:16:56.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.806+0000 7fef28df1700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef24102e00 0x7fef2419f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:56.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.806+0000 7fef28df1700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef241037b0 0x7fef2419fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:56.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.806+0000 7fef28df1700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef24107760 0x7fef24199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:56.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.806+0000 7fef28df1700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fef24112400 con 0x7fef24102e00 2024-05-10T11:16:56.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.806+0000 7fef28df1700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fef24112280 con 0x7fef24107760 2024-05-10T11:16:56.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.806+0000 7fef28df1700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fef24112580 con 0x7fef241037b0 2024-05-10T11:16:56.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.806+0000 7fef2259c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef24102e00 0x7fef2419f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:56.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.806+0000 7fef2259c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef24102e00 0x7fef2419f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37644/0 (socket says 172.21.15.100:37644) 2024-05-10T11:16:56.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.807+0000 7fef2259c700 1 -- 172.21.15.100:0/501403575 learned_addr learned my addr 172.21.15.100:0/501403575 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:16:56.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.807+0000 7fef21d9b700 1 --2- 172.21.15.100:0/501403575 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef241037b0 0x7fef2419fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:56.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.807+0000 7fef22d9d700 1 --2- 172.21.15.100:0/501403575 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef24107760 0x7fef24199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:56.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.807+0000 7fef2259c700 1 -- 172.21.15.100:0/501403575 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef241037b0 msgr2=0x7fef2419fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:56.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.807+0000 7fef2259c700 1 --2- 172.21.15.100:0/501403575 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef241037b0 0x7fef2419fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:56.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.807+0000 7fef2259c700 1 -- 172.21.15.100:0/501403575 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef24107760 msgr2=0x7fef24199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:56.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.807+0000 7fef2259c700 1 --2- 172.21.15.100:0/501403575 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef24107760 0x7fef24199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:56.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.807+0000 7fef2259c700 1 -- 172.21.15.100:0/501403575 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef2419a010 con 0x7fef24102e00 2024-05-10T11:16:56.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.807+0000 7fef22d9d700 1 --2- 172.21.15.100:0/501403575 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef24107760 0x7fef24199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:16:56.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.807+0000 7fef2259c700 1 --2- 172.21.15.100:0/501403575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef24102e00 0x7fef2419f6b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fef14005f50 tx=0x7fef140057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:16:56.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.807+0000 7fef137fe700 1 -- 172.21.15.100:0/501403575 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef14027070 con 0x7fef24102e00 2024-05-10T11:16:56.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.807+0000 7fef28df1700 1 -- 172.21.15.100:0/501403575 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef2419a2a0 con 0x7fef24102e00 2024-05-10T11:16:56.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.807+0000 7fef137fe700 1 -- 172.21.15.100:0/501403575 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fef1400fd60 con 0x7fef24102e00 2024-05-10T11:16:56.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.807+0000 7fef28df1700 1 -- 172.21.15.100:0/501403575 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef240fddb0 con 0x7fef24102e00 2024-05-10T11:16:56.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.807+0000 7fef137fe700 1 -- 172.21.15.100:0/501403575 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef14022e30 con 0x7fef24102e00 2024-05-10T11:16:56.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.809+0000 7fef28df1700 1 -- 172.21.15.100:0/501403575 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef24103bb0 con 0x7fef24102e00 2024-05-10T11:16:56.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.810+0000 7fef137fe700 1 -- 172.21.15.100:0/501403575 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fef1402a070 con 0x7fef24102e00 2024-05-10T11:16:56.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.811+0000 7fef137fe700 1 --2- 172.21.15.100:0/501403575 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fef0c0b2990 0x7fef0c0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:16:56.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.811+0000 7fef21d9b700 1 --2- 172.21.15.100:0/501403575 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fef0c0b2990 0x7fef0c0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:16:56.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.811+0000 7fef137fe700 1 -- 172.21.15.100:0/501403575 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fef140e33d0 con 0x7fef24102e00 2024-05-10T11:16:56.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.812+0000 7fef21d9b700 1 --2- 172.21.15.100:0/501403575 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fef0c0b2990 0x7fef0c0b4e50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fef241010a0 tx=0x7fef1800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:16:56.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:56.815+0000 7fef137fe700 1 -- 172.21.15.100:0/501403575 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fef14013070 con 0x7fef24102e00 2024-05-10T11:16:57.029 INFO:teuthology.orchestra.run.smithi100.stdout:206158430276 2024-05-10T11:16:57.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:57.029+0000 7fef28df1700 1 -- 172.21.15.100:0/501403575 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fef2419ad60 con 0x7fef24102e00 2024-05-10T11:16:57.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:57.029+0000 7fef137fe700 1 -- 172.21.15.100:0/501403575 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fef140aacc0 con 0x7fef24102e00 2024-05-10T11:16:57.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:57.031+0000 7fef28df1700 1 -- 172.21.15.100:0/501403575 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fef0c0b2990 msgr2=0x7fef0c0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:57.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:57.031+0000 7fef28df1700 1 --2- 172.21.15.100:0/501403575 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fef0c0b2990 0x7fef0c0b4e50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fef241010a0 tx=0x7fef1800a380 comp rx=0 tx=0).stop 2024-05-10T11:16:57.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:57.031+0000 7fef28df1700 1 -- 172.21.15.100:0/501403575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef24102e00 msgr2=0x7fef2419f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:16:57.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:57.031+0000 7fef28df1700 1 --2- 172.21.15.100:0/501403575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef24102e00 0x7fef2419f6b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fef14005f50 tx=0x7fef140057c0 comp rx=0 tx=0).stop 2024-05-10T11:16:57.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:57.031+0000 7fef28df1700 1 -- 172.21.15.100:0/501403575 shutdown_connections 2024-05-10T11:16:57.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:57.031+0000 7fef28df1700 1 --2- 172.21.15.100:0/501403575 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fef0c0b2990 0x7fef0c0b4e50 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:57.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:57.031+0000 7fef28df1700 1 --2- 172.21.15.100:0/501403575 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef24107760 0x7fef24199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:57.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:57.031+0000 7fef28df1700 1 --2- 172.21.15.100:0/501403575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef24102e00 0x7fef2419f6b0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:57.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:57.031+0000 7fef28df1700 1 --2- 172.21.15.100:0/501403575 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef241037b0 0x7fef2419fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:16:57.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:57.031+0000 7fef28df1700 1 -- 172.21.15.100:0/501403575 >> 172.21.15.100:0/501403575 conn(0x7fef24078a10 msgr2=0x7fef24104e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:16:57.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:57.032+0000 7fef28df1700 1 -- 172.21.15.100:0/501403575 shutdown_connections 2024-05-10T11:16:57.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:16:57.032+0000 7fef28df1700 1 -- 172.21.15.100:0/501403575 wait complete. 2024-05-10T11:16:57.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:16:56 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/1336872575' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-10T11:16:57.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:16:56 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/1336872575' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-10T11:16:57.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:16:56 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/1336872575' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-10T11:16:58.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:16:57 smithi100 ceph-mon[30296]: pgmap v377: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T11:16:58.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:16:57 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/501403575' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-10T11:16:58.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:16:57 smithi178 ceph-mon[33359]: pgmap v377: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T11:16:58.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:16:57 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/501403575' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-10T11:16:58.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:16:57 smithi188 ceph-mon[34075]: pgmap v377: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T11:16:58.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:16:57 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/501403575' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-10T11:16:58.895 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913790026 got 158913790029 for osd.5 2024-05-10T11:16:58.895 DEBUG:teuthology.parallel:result is None 2024-05-10T11:17:00.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.092+0000 7fe5d5aed700 1 -- 172.21.15.100:0/1897947437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5d0101e60 msgr2=0x7fe5d01022e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:00.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.092+0000 7fe5d5aed700 1 --2- 172.21.15.100:0/1897947437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5d0101e60 0x7fe5d01022e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4009990 tx=0x7fe5c4009ca0 comp rx=0 tx=0).stop 2024-05-10T11:17:00.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.092+0000 7fe5d5aed700 1 -- 172.21.15.100:0/1897947437 shutdown_connections 2024-05-10T11:17:00.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.092+0000 7fe5d5aed700 1 --2- 172.21.15.100:0/1897947437 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5d0102820 0x7fe5d0110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:00.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.092+0000 7fe5d5aed700 1 --2- 172.21.15.100:0/1897947437 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5d0108580 0x7fe5d0108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:00.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.092+0000 7fe5d5aed700 1 --2- 172.21.15.100:0/1897947437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5d0101e60 0x7fe5d01022e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:00.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.092+0000 7fe5d5aed700 1 -- 172.21.15.100:0/1897947437 >> 172.21.15.100:0/1897947437 conn(0x7fe5d00776c0 msgr2=0x7fe5d0077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:17:00.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.092+0000 7fe5d5aed700 1 -- 172.21.15.100:0/1897947437 shutdown_connections 2024-05-10T11:17:00.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.092+0000 7fe5d5aed700 1 -- 172.21.15.100:0/1897947437 wait complete. 2024-05-10T11:17:00.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.093+0000 7fe5d5aed700 1 Processor -- start 2024-05-10T11:17:00.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.093+0000 7fe5d5aed700 1 -- start start 2024-05-10T11:17:00.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.093+0000 7fe5d5aed700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5d0101e60 0x7fe5d019f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:00.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.093+0000 7fe5d5aed700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5d0102820 0x7fe5d019fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:00.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.093+0000 7fe5d5aed700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5d0108580 0x7fe5d0199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:00.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.093+0000 7fe5d5aed700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d0113520 con 0x7fe5d0101e60 2024-05-10T11:17:00.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.093+0000 7fe5d5aed700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d01133a0 con 0x7fe5d0108580 2024-05-10T11:17:00.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.093+0000 7fe5d5aed700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d01136a0 con 0x7fe5d0102820 2024-05-10T11:17:00.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.093+0000 7fe5cf7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5d0101e60 0x7fe5d019f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:00.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.093+0000 7fe5cf7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5d0101e60 0x7fe5d019f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37668/0 (socket says 172.21.15.100:37668) 2024-05-10T11:17:00.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.093+0000 7fe5cf7fe700 1 -- 172.21.15.100:0/188053662 learned_addr learned my addr 172.21.15.100:0/188053662 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:17:00.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.094+0000 7fe5cffff700 1 --2- 172.21.15.100:0/188053662 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5d0108580 0x7fe5d0199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:00.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.094+0000 7fe5ceffd700 1 --2- 172.21.15.100:0/188053662 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5d0102820 0x7fe5d019fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:00.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.094+0000 7fe5cf7fe700 1 -- 172.21.15.100:0/188053662 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5d0102820 msgr2=0x7fe5d019fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:00.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.094+0000 7fe5cf7fe700 1 --2- 172.21.15.100:0/188053662 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5d0102820 0x7fe5d019fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:00.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.094+0000 7fe5cf7fe700 1 -- 172.21.15.100:0/188053662 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5d0108580 msgr2=0x7fe5d0199860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:00.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.094+0000 7fe5cf7fe700 1 --2- 172.21.15.100:0/188053662 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5d0108580 0x7fe5d0199860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:00.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.094+0000 7fe5cf7fe700 1 -- 172.21.15.100:0/188053662 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5d0199fc0 con 0x7fe5d0101e60 2024-05-10T11:17:00.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.094+0000 7fe5cf7fe700 1 --2- 172.21.15.100:0/188053662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5d0101e60 0x7fe5d019f6a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe5c000c970 tx=0x7fe5c000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:17:00.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.094+0000 7fe5ccff9700 1 -- 172.21.15.100:0/188053662 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c0004220 con 0x7fe5d0101e60 2024-05-10T11:17:00.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.094+0000 7fe5ccff9700 1 -- 172.21.15.100:0/188053662 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe5c00043c0 con 0x7fe5d0101e60 2024-05-10T11:17:00.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.094+0000 7fe5d5aed700 1 -- 172.21.15.100:0/188053662 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5d019a2b0 con 0x7fe5d0101e60 2024-05-10T11:17:00.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.094+0000 7fe5ccff9700 1 -- 172.21.15.100:0/188053662 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c0003b60 con 0x7fe5d0101e60 2024-05-10T11:17:00.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.094+0000 7fe5d5aed700 1 -- 172.21.15.100:0/188053662 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5d0102f70 con 0x7fe5d0101e60 2024-05-10T11:17:00.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.095+0000 7fe5d5aed700 1 -- 172.21.15.100:0/188053662 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5d006a190 con 0x7fe5d0101e60 2024-05-10T11:17:00.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.097+0000 7fe5ccff9700 1 -- 172.21.15.100:0/188053662 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fe5c0003d00 con 0x7fe5d0101e60 2024-05-10T11:17:00.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.097+0000 7fe5ccff9700 1 --2- 172.21.15.100:0/188053662 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe5b80b2910 0x7fe5b80b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:00.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.097+0000 7fe5ceffd700 1 --2- 172.21.15.100:0/188053662 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe5b80b2910 0x7fe5b80b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:00.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.098+0000 7fe5ccff9700 1 -- 172.21.15.100:0/188053662 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fe5c00e2590 con 0x7fe5d0101e60 2024-05-10T11:17:00.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.098+0000 7fe5ceffd700 1 --2- 172.21.15.100:0/188053662 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe5b80b2910 0x7fe5b80b4dd0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe5c400b5c0 tx=0x7fe5c4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:17:00.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.101+0000 7fe5ccff9700 1 -- 172.21.15.100:0/188053662 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fe5c0014030 con 0x7fe5d0101e60 2024-05-10T11:17:00.214 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:16:59 smithi100 ceph-mon[30296]: pgmap v378: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T11:17:00.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:16:59 smithi178 ceph-mon[33359]: pgmap v378: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T11:17:00.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:16:59 smithi188 ceph-mon[34075]: pgmap v378: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T11:17:00.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.320+0000 7fc0b6671700 1 -- 172.21.15.100:0/2442433865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0b0102d60 msgr2=0x7fc0b010f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:00.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.320+0000 7fc0b6671700 1 --2- 172.21.15.100:0/2442433865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0b0102d60 0x7fc0b010f250 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc0a800b0a0 tx=0x7fc0a800b3b0 comp rx=0 tx=0).stop 2024-05-10T11:17:00.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.320+0000 7fc0b6671700 1 -- 172.21.15.100:0/2442433865 shutdown_connections 2024-05-10T11:17:00.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.320+0000 7fc0b6671700 1 --2- 172.21.15.100:0/2442433865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0b0102d60 0x7fc0b010f250 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:00.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.320+0000 7fc0b6671700 1 --2- 172.21.15.100:0/2442433865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0b0108410 0x7fc0b01087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:00.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.320+0000 7fc0b6671700 1 --2- 172.21.15.100:0/2442433865 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0b01023a0 0x7fc0b0102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:00.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.320+0000 7fc0b6671700 1 -- 172.21.15.100:0/2442433865 >> 172.21.15.100:0/2442433865 conn(0x7fc0b0079350 msgr2=0x7fc0b0079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:17:00.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.320+0000 7fc0b6671700 1 -- 172.21.15.100:0/2442433865 shutdown_connections 2024-05-10T11:17:00.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.320+0000 7fc0b6671700 1 -- 172.21.15.100:0/2442433865 wait complete. 2024-05-10T11:17:00.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.321+0000 7fc0b6671700 1 Processor -- start 2024-05-10T11:17:00.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.321+0000 7fc0b6671700 1 -- start start 2024-05-10T11:17:00.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.321+0000 7fc0b6671700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0b01023a0 0x7fc0b019a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:00.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.321+0000 7fc0b6671700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0b0102d60 0x7fc0b019ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:00.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.321+0000 7fc0b6671700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0b0108410 0x7fc0b01a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:00.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.321+0000 7fc0b6671700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b0112400 con 0x7fc0b01023a0 2024-05-10T11:17:00.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.321+0000 7fc0b6671700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b0112280 con 0x7fc0b0108410 2024-05-10T11:17:00.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.321+0000 7fc0b6671700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b0112580 con 0x7fc0b0102d60 2024-05-10T11:17:00.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.321+0000 7fc0affff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0b01023a0 0x7fc0b019a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:00.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.321+0000 7fc0affff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0b01023a0 0x7fc0b019a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37674/0 (socket says 172.21.15.100:37674) 2024-05-10T11:17:00.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.321+0000 7fc0affff700 1 -- 172.21.15.100:0/2034380124 learned_addr learned my addr 172.21.15.100:0/2034380124 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:17:00.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.322+0000 7fc0b4c0e700 1 --2- 172.21.15.100:0/2034380124 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0b0108410 0x7fc0b01a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:00.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.322+0000 7fc0af7fe700 1 --2- 172.21.15.100:0/2034380124 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0b0102d60 0x7fc0b019ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:00.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.322+0000 7fc0affff700 1 -- 172.21.15.100:0/2034380124 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0b0102d60 msgr2=0x7fc0b019ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:00.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.322+0000 7fc0affff700 1 --2- 172.21.15.100:0/2034380124 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0b0102d60 0x7fc0b019ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:00.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.322+0000 7fc0affff700 1 -- 172.21.15.100:0/2034380124 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0b0108410 msgr2=0x7fc0b01a5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:00.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.322+0000 7fc0affff700 1 --2- 172.21.15.100:0/2034380124 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0b0108410 0x7fc0b01a5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:00.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.322+0000 7fc0affff700 1 -- 172.21.15.100:0/2034380124 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0b01a5e60 con 0x7fc0b01023a0 2024-05-10T11:17:00.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.322+0000 7fc0b4c0e700 1 --2- 172.21.15.100:0/2034380124 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0b0108410 0x7fc0b01a5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:17:00.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.322+0000 7fc0af7fe700 1 --2- 172.21.15.100:0/2034380124 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0b0102d60 0x7fc0b019ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:17:00.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.322+0000 7fc0affff700 1 --2- 172.21.15.100:0/2034380124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0b01023a0 0x7fc0b019a8a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc0a000d980 tx=0x7fc0a000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:17:00.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.322+0000 7fc0ad7fa700 1 -- 172.21.15.100:0/2034380124 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a0017070 con 0x7fc0b01023a0 2024-05-10T11:17:00.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.322+0000 7fc0ad7fa700 1 -- 172.21.15.100:0/2034380124 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc0a0004830 con 0x7fc0b01023a0 2024-05-10T11:17:00.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.322+0000 7fc0ad7fa700 1 -- 172.21.15.100:0/2034380124 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a00053a0 con 0x7fc0b01023a0 2024-05-10T11:17:00.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.322+0000 7fc0b6671700 1 -- 172.21.15.100:0/2034380124 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0b01a6090 con 0x7fc0b01023a0 2024-05-10T11:17:00.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.322+0000 7fc0b6671700 1 -- 172.21.15.100:0/2034380124 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0b01a6550 con 0x7fc0b01023a0 2024-05-10T11:17:00.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.323+0000 7fc0b6671700 1 -- 172.21.15.100:0/2034380124 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0b0103ac0 con 0x7fc0b01023a0 2024-05-10T11:17:00.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.326+0000 7fc0ad7fa700 1 -- 172.21.15.100:0/2034380124 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fc0a00049e0 con 0x7fc0b01023a0 2024-05-10T11:17:00.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.327+0000 7fc0ad7fa700 1 --2- 172.21.15.100:0/2034380124 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc0980b2990 0x7fc0980b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:00.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.327+0000 7fc0af7fe700 1 --2- 172.21.15.100:0/2034380124 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc0980b2990 0x7fc0980b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:00.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.327+0000 7fc0ad7fa700 1 -- 172.21.15.100:0/2034380124 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fc0a00e2780 con 0x7fc0b01023a0 2024-05-10T11:17:00.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.327+0000 7fc0af7fe700 1 --2- 172.21.15.100:0/2034380124 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc0980b2990 0x7fc0980b4e50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4005fd0 tx=0x7fc0a4005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:17:00.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.329+0000 7fc0ad7fa700 1 -- 172.21.15.100:0/2034380124 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fc0a0020030 con 0x7fc0b01023a0 2024-05-10T11:17:00.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.352+0000 7fe5d5aed700 1 -- 172.21.15.100:0/188053662 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fe5d019ad90 con 0x7fe5d0101e60 2024-05-10T11:17:00.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.352+0000 7fe5ccff9700 1 -- 172.21.15.100:0/188053662 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe5c00b0e20 con 0x7fe5d0101e60 2024-05-10T11:17:00.353 INFO:teuthology.orchestra.run.smithi100.stdout:55834574946 2024-05-10T11:17:00.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.356+0000 7fe5d5aed700 1 -- 172.21.15.100:0/188053662 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe5b80b2910 msgr2=0x7fe5b80b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:00.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.356+0000 7fe5d5aed700 1 --2- 172.21.15.100:0/188053662 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe5b80b2910 0x7fe5b80b4dd0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe5c400b5c0 tx=0x7fe5c4005960 comp rx=0 tx=0).stop 2024-05-10T11:17:00.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.356+0000 7fe5d5aed700 1 -- 172.21.15.100:0/188053662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5d0101e60 msgr2=0x7fe5d019f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:00.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.356+0000 7fe5d5aed700 1 --2- 172.21.15.100:0/188053662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5d0101e60 0x7fe5d019f6a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe5c000c970 tx=0x7fe5c000cc80 comp rx=0 tx=0).stop 2024-05-10T11:17:00.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.356+0000 7fe5d5aed700 1 -- 172.21.15.100:0/188053662 shutdown_connections 2024-05-10T11:17:00.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.356+0000 7fe5d5aed700 1 --2- 172.21.15.100:0/188053662 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe5b80b2910 0x7fe5b80b4dd0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:00.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.356+0000 7fe5d5aed700 1 --2- 172.21.15.100:0/188053662 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5d0108580 0x7fe5d0199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:00.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.356+0000 7fe5d5aed700 1 --2- 172.21.15.100:0/188053662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5d0101e60 0x7fe5d019f6a0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:00.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.356+0000 7fe5d5aed700 1 --2- 172.21.15.100:0/188053662 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5d0102820 0x7fe5d019fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:00.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.356+0000 7fe5d5aed700 1 -- 172.21.15.100:0/188053662 >> 172.21.15.100:0/188053662 conn(0x7fe5d00776c0 msgr2=0x7fe5d0100ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:17:00.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.356+0000 7fe5d5aed700 1 -- 172.21.15.100:0/188053662 shutdown_connections 2024-05-10T11:17:00.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.356+0000 7fe5d5aed700 1 -- 172.21.15.100:0/188053662 wait complete. 2024-05-10T11:17:00.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.553+0000 7fc0b6671700 1 -- 172.21.15.100:0/2034380124 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fc0b002e6d0 con 0x7fc0b01023a0 2024-05-10T11:17:00.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.553+0000 7fc0ad7fa700 1 -- 172.21.15.100:0/2034380124 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc0a00aa060 con 0x7fc0b01023a0 2024-05-10T11:17:00.553 INFO:teuthology.orchestra.run.smithi100.stdout:77309411422 2024-05-10T11:17:00.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.555+0000 7fc0b6671700 1 -- 172.21.15.100:0/2034380124 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc0980b2990 msgr2=0x7fc0980b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:00.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.555+0000 7fc0b6671700 1 --2- 172.21.15.100:0/2034380124 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc0980b2990 0x7fc0980b4e50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4005fd0 tx=0x7fc0a4005dc0 comp rx=0 tx=0).stop 2024-05-10T11:17:00.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.555+0000 7fc0b6671700 1 -- 172.21.15.100:0/2034380124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0b01023a0 msgr2=0x7fc0b019a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:00.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.555+0000 7fc0b6671700 1 --2- 172.21.15.100:0/2034380124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0b01023a0 0x7fc0b019a8a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc0a000d980 tx=0x7fc0a000dd40 comp rx=0 tx=0).stop 2024-05-10T11:17:00.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.555+0000 7fc0b6671700 1 -- 172.21.15.100:0/2034380124 shutdown_connections 2024-05-10T11:17:00.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.555+0000 7fc0b6671700 1 --2- 172.21.15.100:0/2034380124 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc0980b2990 0x7fc0980b4e50 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:00.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.555+0000 7fc0b6671700 1 --2- 172.21.15.100:0/2034380124 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0b0108410 0x7fc0b01a5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:00.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.555+0000 7fc0b6671700 1 --2- 172.21.15.100:0/2034380124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0b01023a0 0x7fc0b019a8a0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:00.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.555+0000 7fc0b6671700 1 --2- 172.21.15.100:0/2034380124 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0b0102d60 0x7fc0b019ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:00.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.555+0000 7fc0b6671700 1 -- 172.21.15.100:0/2034380124 >> 172.21.15.100:0/2034380124 conn(0x7fc0b0079350 msgr2=0x7fc0b0100fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:17:00.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.556+0000 7fc0b6671700 1 -- 172.21.15.100:0/2034380124 shutdown_connections 2024-05-10T11:17:00.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:00.556+0000 7fc0b6671700 1 -- 172.21.15.100:0/2034380124 wait complete. 2024-05-10T11:17:01.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:17:00 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/188053662' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-10T11:17:01.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:17:00 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/2034380124' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-10T11:17:01.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:17:00 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/188053662' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-10T11:17:01.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:17:00 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/2034380124' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-10T11:17:01.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:17:00 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/188053662' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-10T11:17:01.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:17:00 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/2034380124' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-10T11:17:01.482 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430271 got 206158430276 for osd.7 2024-05-10T11:17:01.482 DEBUG:teuthology.parallel:result is None 2024-05-10T11:17:02.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:01.916+0000 7f9848f71700 1 -- 172.21.15.100:0/1200739750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9844102fe0 msgr2=0x7f9844106fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:02.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:01.916+0000 7f9848f71700 1 --2- 172.21.15.100:0/1200739750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9844102fe0 0x7f9844106fe0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f98340099c0 tx=0x7f9834009cd0 comp rx=0 tx=0).stop 2024-05-10T11:17:02.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:01.917+0000 7f9848f71700 1 -- 172.21.15.100:0/1200739750 shutdown_connections 2024-05-10T11:17:02.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:01.917+0000 7f9848f71700 1 --2- 172.21.15.100:0/1200739750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98441076c0 0x7f984410f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:02.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:01.917+0000 7f9848f71700 1 --2- 172.21.15.100:0/1200739750 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9844102630 0x7f9844102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:02.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:01.917+0000 7f9848f71700 1 --2- 172.21.15.100:0/1200739750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9844102fe0 0x7f9844106fe0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:02.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:01.917+0000 7f9848f71700 1 -- 172.21.15.100:0/1200739750 >> 172.21.15.100:0/1200739750 conn(0x7f9844079350 msgr2=0x7f9844079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:17:02.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:01.917+0000 7f9848f71700 1 -- 172.21.15.100:0/1200739750 shutdown_connections 2024-05-10T11:17:02.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:01.917+0000 7f9848f71700 1 -- 172.21.15.100:0/1200739750 wait complete. 2024-05-10T11:17:02.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:01.917+0000 7f9848f71700 1 Processor -- start 2024-05-10T11:17:02.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:01.917+0000 7f9848f71700 1 -- start start 2024-05-10T11:17:02.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:01.918+0000 7f9848f71700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9844102630 0x7f984419e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:02.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:01.918+0000 7f9848f71700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9844102fe0 0x7f984419ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:02.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:01.918+0000 7f9848f71700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98441076c0 0x7f98441988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:02.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:01.918+0000 7f9848f71700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f98441123e0 con 0x7f9844102630 2024-05-10T11:17:02.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:01.918+0000 7f9848f71700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9844112260 con 0x7f98441076c0 2024-05-10T11:17:02.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:01.918+0000 7f9848f71700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9844112560 con 0x7f9844102fe0 2024-05-10T11:17:02.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:01.918+0000 7f984259c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9844102630 0x7f984419e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:02.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:01.918+0000 7f984259c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9844102630 0x7f984419e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37700/0 (socket says 172.21.15.100:37700) 2024-05-10T11:17:02.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:01.918+0000 7f984259c700 1 -- 172.21.15.100:0/2967599920 learned_addr learned my addr 172.21.15.100:0/2967599920 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:17:02.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:01.918+0000 7f9841d9b700 1 --2- 172.21.15.100:0/2967599920 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9844102fe0 0x7f984419ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:02.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:01.918+0000 7f9842d9d700 1 --2- 172.21.15.100:0/2967599920 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98441076c0 0x7f98441988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:02.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:01.918+0000 7f984259c700 1 -- 172.21.15.100:0/2967599920 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9844102fe0 msgr2=0x7f984419ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:02.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:01.918+0000 7f984259c700 1 --2- 172.21.15.100:0/2967599920 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9844102fe0 0x7f984419ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:02.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:01.918+0000 7f984259c700 1 -- 172.21.15.100:0/2967599920 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98441076c0 msgr2=0x7f98441988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:02.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:01.918+0000 7f984259c700 1 --2- 172.21.15.100:0/2967599920 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98441076c0 0x7f98441988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:02.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:01.918+0000 7f984259c700 1 -- 172.21.15.100:0/2967599920 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9844199030 con 0x7f9844102630 2024-05-10T11:17:02.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:01.918+0000 7f9842d9d700 1 --2- 172.21.15.100:0/2967599920 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98441076c0 0x7f98441988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:17:02.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:01.918+0000 7f984259c700 1 --2- 172.21.15.100:0/2967599920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9844102630 0x7f984419e6c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f983800d980 tx=0x7f983800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:17:02.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:01.918+0000 7f98337fe700 1 -- 172.21.15.100:0/2967599920 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9838016070 con 0x7f9844102630 2024-05-10T11:17:02.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:01.918+0000 7f98337fe700 1 -- 172.21.15.100:0/2967599920 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9838004830 con 0x7f9844102630 2024-05-10T11:17:02.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:01.918+0000 7f9848f71700 1 -- 172.21.15.100:0/2967599920 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9844199320 con 0x7f9844102630 2024-05-10T11:17:02.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:01.918+0000 7f98337fe700 1 -- 172.21.15.100:0/2967599920 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9838005440 con 0x7f9844102630 2024-05-10T11:17:02.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:01.918+0000 7f9848f71700 1 -- 172.21.15.100:0/2967599920 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98441a5590 con 0x7f9844102630 2024-05-10T11:17:02.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:01.919+0000 7f9848f71700 1 -- 172.21.15.100:0/2967599920 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f984406a190 con 0x7f9844102630 2024-05-10T11:17:02.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:01.921+0000 7f98337fe700 1 -- 172.21.15.100:0/2967599920 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f98380055e0 con 0x7f9844102630 2024-05-10T11:17:02.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:01.921+0000 7f98337fe700 1 --2- 172.21.15.100:0/2967599920 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f982c0b2880 0x7f982c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:02.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:01.921+0000 7f9841d9b700 1 --2- 172.21.15.100:0/2967599920 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f982c0b2880 0x7f982c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:02.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:01.921+0000 7f98337fe700 1 -- 172.21.15.100:0/2967599920 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f98380e2870 con 0x7f9844102630 2024-05-10T11:17:02.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:01.922+0000 7f9841d9b700 1 --2- 172.21.15.100:0/2967599920 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f982c0b2880 0x7f982c0b4d40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9834009990 tx=0x7f9834005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:17:02.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:01.925+0000 7f98337fe700 1 -- 172.21.15.100:0/2967599920 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f983801f020 con 0x7f9844102630 2024-05-10T11:17:02.099 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:17:01 smithi100 ceph-mon[30296]: pgmap v379: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T11:17:02.099 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:17:01 smithi100 ceph-mon[30296]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-05-10T11:17:02.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:02.135+0000 7f9848f71700 1 -- 172.21.15.100:0/2967599920 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f984402e510 con 0x7f9844102630 2024-05-10T11:17:02.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:02.135+0000 7f98337fe700 1 -- 172.21.15.100:0/2967599920 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9838055e20 con 0x7f9844102630 2024-05-10T11:17:02.135 INFO:teuthology.orchestra.run.smithi100.stdout:34359738472 2024-05-10T11:17:02.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:02.137+0000 7f9848f71700 1 -- 172.21.15.100:0/2967599920 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f982c0b2880 msgr2=0x7f982c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:02.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:02.137+0000 7f9848f71700 1 --2- 172.21.15.100:0/2967599920 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f982c0b2880 0x7f982c0b4d40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9834009990 tx=0x7f9834005fb0 comp rx=0 tx=0).stop 2024-05-10T11:17:02.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:02.137+0000 7f9848f71700 1 -- 172.21.15.100:0/2967599920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9844102630 msgr2=0x7f984419e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:02.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:02.137+0000 7f9848f71700 1 --2- 172.21.15.100:0/2967599920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9844102630 0x7f984419e6c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f983800d980 tx=0x7f983800dc90 comp rx=0 tx=0).stop 2024-05-10T11:17:02.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:02.137+0000 7f9848f71700 1 -- 172.21.15.100:0/2967599920 shutdown_connections 2024-05-10T11:17:02.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:02.137+0000 7f9848f71700 1 --2- 172.21.15.100:0/2967599920 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f982c0b2880 0x7f982c0b4d40 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:02.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:02.137+0000 7f9848f71700 1 --2- 172.21.15.100:0/2967599920 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98441076c0 0x7f98441988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:02.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:02.137+0000 7f9848f71700 1 --2- 172.21.15.100:0/2967599920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9844102630 0x7f984419e6c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:02.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:02.137+0000 7f9848f71700 1 --2- 172.21.15.100:0/2967599920 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9844102fe0 0x7f984419ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:02.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:02.137+0000 7f9848f71700 1 -- 172.21.15.100:0/2967599920 >> 172.21.15.100:0/2967599920 conn(0x7f9844079350 msgr2=0x7f9844100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:17:02.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:02.138+0000 7f9848f71700 1 -- 172.21.15.100:0/2967599920 shutdown_connections 2024-05-10T11:17:02.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:02.138+0000 7f9848f71700 1 -- 172.21.15.100:0/2967599920 wait complete. 2024-05-10T11:17:02.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:17:01 smithi178 ceph-mon[33359]: pgmap v379: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T11:17:02.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:17:01 smithi178 ceph-mon[33359]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-05-10T11:17:02.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:17:01 smithi188 ceph-mon[34075]: pgmap v379: 129 pgs: 129 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T11:17:02.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:17:01 smithi188 ceph-mon[34075]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-05-10T11:17:02.330 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:17:02.520 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:17:03.079 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574940 got 55834574946 for osd.1 2024-05-10T11:17:03.079 DEBUG:teuthology.parallel:result is None 2024-05-10T11:17:03.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:17:02 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/2967599920' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-10T11:17:03.269 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:17:02 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/2967599920' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-10T11:17:03.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:17:02 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/2967599920' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-10T11:17:03.925 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411418 got 77309411422 for osd.2 2024-05-10T11:17:03.926 DEBUG:teuthology.parallel:result is None 2024-05-10T11:17:04.146 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:17:03 smithi100 ceph-mon[30296]: pgmap v380: 129 pgs: 129 active+clean; 644 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-10T11:17:04.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:17:03 smithi178 ceph-mon[33359]: pgmap v380: 129 pgs: 129 active+clean; 644 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-10T11:17:04.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:17:03 smithi188 ceph-mon[34075]: pgmap v380: 129 pgs: 129 active+clean; 644 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-10T11:17:04.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:04.343+0000 7ff602e65700 1 -- 172.21.15.100:0/2614859883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5fc101c50 msgr2=0x7ff5fc102030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:04.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:04.343+0000 7ff602e65700 1 --2- 172.21.15.100:0/2614859883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5fc101c50 0x7ff5fc102030 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff5f80099c0 tx=0x7ff5f8009cd0 comp rx=0 tx=0).stop 2024-05-10T11:17:04.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:04.343+0000 7ff602e65700 1 -- 172.21.15.100:0/2614859883 shutdown_connections 2024-05-10T11:17:04.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:04.343+0000 7ff602e65700 1 --2- 172.21.15.100:0/2614859883 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5fc108890 0x7ff5fc110410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:04.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:04.343+0000 7ff602e65700 1 --2- 172.21.15.100:0/2614859883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5fc101c50 0x7ff5fc102030 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:04.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:04.343+0000 7ff602e65700 1 --2- 172.21.15.100:0/2614859883 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5fc102600 0x7ff5fc108160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:04.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:04.343+0000 7ff602e65700 1 -- 172.21.15.100:0/2614859883 >> 172.21.15.100:0/2614859883 conn(0x7ff5fc0789a0 msgr2=0x7ff5fc0ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:17:04.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:04.343+0000 7ff602e65700 1 -- 172.21.15.100:0/2614859883 shutdown_connections 2024-05-10T11:17:04.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:04.343+0000 7ff602e65700 1 -- 172.21.15.100:0/2614859883 wait complete. 2024-05-10T11:17:04.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:04.344+0000 7ff602e65700 1 Processor -- start 2024-05-10T11:17:04.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:04.344+0000 7ff602e65700 1 -- start start 2024-05-10T11:17:04.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:04.344+0000 7ff602e65700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5fc101c50 0x7ff5fc19f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:04.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:04.344+0000 7ff602e65700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5fc102600 0x7ff5fc19fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:04.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:04.344+0000 7ff602e65700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5fc108890 0x7ff5fc199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:04.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:04.344+0000 7ff602e65700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff5fc113510 con 0x7ff5fc101c50 2024-05-10T11:17:04.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:04.344+0000 7ff602e65700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff5fc113390 con 0x7ff5fc102600 2024-05-10T11:17:04.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:04.344+0000 7ff602e65700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff5fc113690 con 0x7ff5fc108890 2024-05-10T11:17:04.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:04.344+0000 7ff600c01700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5fc101c50 0x7ff5fc19f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:04.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:04.344+0000 7ff600c01700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5fc101c50 0x7ff5fc19f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37718/0 (socket says 172.21.15.100:37718) 2024-05-10T11:17:04.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:04.344+0000 7ff600c01700 1 -- 172.21.15.100:0/202528045 learned_addr learned my addr 172.21.15.100:0/202528045 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:17:04.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:04.344+0000 7ff601402700 1 --2- 172.21.15.100:0/202528045 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5fc108890 0x7ff5fc199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:04.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:04.344+0000 7ff5f3fff700 1 --2- 172.21.15.100:0/202528045 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5fc102600 0x7ff5fc19fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:04.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:04.344+0000 7ff600c01700 1 -- 172.21.15.100:0/202528045 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5fc108890 msgr2=0x7ff5fc199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:04.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:04.344+0000 7ff600c01700 1 --2- 172.21.15.100:0/202528045 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5fc108890 0x7ff5fc199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:04.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:04.344+0000 7ff600c01700 1 -- 172.21.15.100:0/202528045 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5fc102600 msgr2=0x7ff5fc19fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:04.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:04.344+0000 7ff600c01700 1 --2- 172.21.15.100:0/202528045 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5fc102600 0x7ff5fc19fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:04.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:04.344+0000 7ff600c01700 1 -- 172.21.15.100:0/202528045 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5fc19a010 con 0x7ff5fc101c50 2024-05-10T11:17:04.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:04.344+0000 7ff5f3fff700 1 --2- 172.21.15.100:0/202528045 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5fc102600 0x7ff5fc19fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:17:04.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:04.345+0000 7ff600c01700 1 --2- 172.21.15.100:0/202528045 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5fc101c50 0x7ff5fc19f6b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8005f50 tx=0x7ff5f80057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:17:04.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:04.345+0000 7ff5f1ffb700 1 -- 172.21.15.100:0/202528045 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f8027070 con 0x7ff5fc101c50 2024-05-10T11:17:04.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:04.345+0000 7ff5f1ffb700 1 -- 172.21.15.100:0/202528045 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff5f800fd60 con 0x7ff5fc101c50 2024-05-10T11:17:04.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:04.345+0000 7ff602e65700 1 -- 172.21.15.100:0/202528045 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5fc19a2a0 con 0x7ff5fc101c50 2024-05-10T11:17:04.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:04.345+0000 7ff5f1ffb700 1 -- 172.21.15.100:0/202528045 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f8022e30 con 0x7ff5fc101c50 2024-05-10T11:17:04.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:04.345+0000 7ff602e65700 1 -- 172.21.15.100:0/202528045 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5fc1a6510 con 0x7ff5fc101c50 2024-05-10T11:17:04.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:04.346+0000 7ff602e65700 1 -- 172.21.15.100:0/202528045 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5fc104d10 con 0x7ff5fc101c50 2024-05-10T11:17:04.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:04.348+0000 7ff5f1ffb700 1 -- 172.21.15.100:0/202528045 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7ff5f802a070 con 0x7ff5fc101c50 2024-05-10T11:17:04.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:04.349+0000 7ff5f1ffb700 1 --2- 172.21.15.100:0/202528045 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff5e40b2880 0x7ff5e40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:04.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:04.349+0000 7ff5f3fff700 1 --2- 172.21.15.100:0/202528045 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff5e40b2880 0x7ff5e40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:04.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:04.349+0000 7ff5f1ffb700 1 -- 172.21.15.100:0/202528045 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7ff5f80e32c0 con 0x7ff5fc101c50 2024-05-10T11:17:04.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:04.349+0000 7ff5f3fff700 1 --2- 172.21.15.100:0/202528045 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff5e40b2880 0x7ff5e40b4d40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec005950 tx=0x7ff5ec0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:17:04.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:04.352+0000 7ff5f1ffb700 1 -- 172.21.15.100:0/202528045 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7ff5f8013070 con 0x7ff5fc101c50 2024-05-10T11:17:04.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:04.561+0000 7ff602e65700 1 -- 172.21.15.100:0/202528045 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7ff5fc02e510 con 0x7ff5fc101c50 2024-05-10T11:17:04.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:04.561+0000 7ff5f1ffb700 1 -- 172.21.15.100:0/202528045 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff5f80aad10 con 0x7ff5fc101c50 2024-05-10T11:17:04.561 INFO:teuthology.orchestra.run.smithi100.stdout:184683593803 2024-05-10T11:17:04.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:04.563+0000 7ff602e65700 1 -- 172.21.15.100:0/202528045 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff5e40b2880 msgr2=0x7ff5e40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:04.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:04.563+0000 7ff602e65700 1 --2- 172.21.15.100:0/202528045 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff5e40b2880 0x7ff5e40b4d40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec005950 tx=0x7ff5ec0058e0 comp rx=0 tx=0).stop 2024-05-10T11:17:04.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:04.564+0000 7ff602e65700 1 -- 172.21.15.100:0/202528045 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5fc101c50 msgr2=0x7ff5fc19f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:04.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:04.564+0000 7ff602e65700 1 --2- 172.21.15.100:0/202528045 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5fc101c50 0x7ff5fc19f6b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8005f50 tx=0x7ff5f80057c0 comp rx=0 tx=0).stop 2024-05-10T11:17:04.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:04.564+0000 7ff602e65700 1 -- 172.21.15.100:0/202528045 shutdown_connections 2024-05-10T11:17:04.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:04.564+0000 7ff602e65700 1 --2- 172.21.15.100:0/202528045 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff5e40b2880 0x7ff5e40b4d40 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:04.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:04.564+0000 7ff602e65700 1 --2- 172.21.15.100:0/202528045 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5fc108890 0x7ff5fc199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:04.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:04.564+0000 7ff602e65700 1 --2- 172.21.15.100:0/202528045 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5fc101c50 0x7ff5fc19f6b0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:04.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:04.564+0000 7ff602e65700 1 --2- 172.21.15.100:0/202528045 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5fc102600 0x7ff5fc19fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:04.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:04.564+0000 7ff602e65700 1 -- 172.21.15.100:0/202528045 >> 172.21.15.100:0/202528045 conn(0x7ff5fc0789a0 msgr2=0x7ff5fc0ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:17:04.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:04.564+0000 7ff602e65700 1 -- 172.21.15.100:0/202528045 shutdown_connections 2024-05-10T11:17:04.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:04.564+0000 7ff602e65700 1 -- 172.21.15.100:0/202528045 wait complete. 2024-05-10T11:17:04.803 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738466 got 34359738472 for osd.0 2024-05-10T11:17:04.803 DEBUG:teuthology.parallel:result is None 2024-05-10T11:17:05.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:17:05 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/202528045' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-10T11:17:05.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:17:05 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/202528045' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-10T11:17:05.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.302+0000 7f8568dc9700 1 -- 172.21.15.100:0/3476642308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8564109540 msgr2=0x7f8564109920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:05.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.302+0000 7f8568dc9700 1 --2- 172.21.15.100:0/3476642308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8564109540 0x7f8564109920 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f85540099c0 tx=0x7f8554009cd0 comp rx=0 tx=0).stop 2024-05-10T11:17:05.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.302+0000 7f8568dc9700 1 -- 172.21.15.100:0/3476642308 shutdown_connections 2024-05-10T11:17:05.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.302+0000 7f8568dc9700 1 --2- 172.21.15.100:0/3476642308 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8564103f20 0x7f8564110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:05.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.302+0000 7f8568dc9700 1 --2- 172.21.15.100:0/3476642308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8564109540 0x7f8564109920 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:05.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.302+0000 7f8568dc9700 1 --2- 172.21.15.100:0/3476642308 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8564103560 0x7f85641039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:05.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.302+0000 7f8568dc9700 1 -- 172.21.15.100:0/3476642308 >> 172.21.15.100:0/3476642308 conn(0x7f856407a660 msgr2=0x7f856407aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:17:05.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.302+0000 7f8568dc9700 1 -- 172.21.15.100:0/3476642308 shutdown_connections 2024-05-10T11:17:05.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.302+0000 7f8568dc9700 1 -- 172.21.15.100:0/3476642308 wait complete. 2024-05-10T11:17:05.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.302+0000 7f8568dc9700 1 Processor -- start 2024-05-10T11:17:05.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.303+0000 7f8568dc9700 1 -- start start 2024-05-10T11:17:05.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.303+0000 7f8568dc9700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8564103560 0x7f856419f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:05.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.303+0000 7f8568dc9700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8564103f20 0x7f856419fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:05.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.303+0000 7f8568dc9700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8564109540 0x7f8564199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:05.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.303+0000 7f8568dc9700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f85641134d0 con 0x7f8564103f20 2024-05-10T11:17:05.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.303+0000 7f8568dc9700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8564113350 con 0x7f8564109540 2024-05-10T11:17:05.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.303+0000 7f8568dc9700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8564113650 con 0x7f8564103560 2024-05-10T11:17:05.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.303+0000 7f8561d9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8564103f20 0x7f856419fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:05.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.303+0000 7f8561d9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8564103f20 0x7f856419fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37738/0 (socket says 172.21.15.100:37738) 2024-05-10T11:17:05.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.303+0000 7f8561d9b700 1 -- 172.21.15.100:0/1095691473 learned_addr learned my addr 172.21.15.100:0/1095691473 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:17:05.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.303+0000 7f856259c700 1 --2- 172.21.15.100:0/1095691473 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8564103560 0x7f856419f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:05.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.303+0000 7f8562d9d700 1 --2- 172.21.15.100:0/1095691473 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8564109540 0x7f8564199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:05.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.303+0000 7f8561d9b700 1 -- 172.21.15.100:0/1095691473 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8564103560 msgr2=0x7f856419f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:05.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.303+0000 7f8561d9b700 1 --2- 172.21.15.100:0/1095691473 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8564103560 0x7f856419f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:05.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.303+0000 7f8561d9b700 1 -- 172.21.15.100:0/1095691473 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8564109540 msgr2=0x7f8564199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:05.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.303+0000 7f8561d9b700 1 --2- 172.21.15.100:0/1095691473 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8564109540 0x7f8564199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:05.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.303+0000 7f8561d9b700 1 -- 172.21.15.100:0/1095691473 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8564199ff0 con 0x7f8564103f20 2024-05-10T11:17:05.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.303+0000 7f8561d9b700 1 --2- 172.21.15.100:0/1095691473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8564103f20 0x7f856419fbd0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f855800d950 tx=0x7f855800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:17:05.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.303+0000 7f85537fe700 1 -- 172.21.15.100:0/1095691473 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8558016070 con 0x7f8564103f20 2024-05-10T11:17:05.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.304+0000 7f8568dc9700 1 -- 172.21.15.100:0/1095691473 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f856419a2e0 con 0x7f8564103f20 2024-05-10T11:17:05.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.304+0000 7f85537fe700 1 -- 172.21.15.100:0/1095691473 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8558009dd0 con 0x7f8564103f20 2024-05-10T11:17:05.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.304+0000 7f85537fe700 1 -- 172.21.15.100:0/1095691473 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855800f3c0 con 0x7f8564103f20 2024-05-10T11:17:05.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.304+0000 7f8568dc9700 1 -- 172.21.15.100:0/1095691473 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85641a64f0 con 0x7f8564103f20 2024-05-10T11:17:05.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.305+0000 7f8568dc9700 1 -- 172.21.15.100:0/1095691473 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f856410b9d0 con 0x7f8564103f20 2024-05-10T11:17:05.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.306+0000 7f85537fe700 1 -- 172.21.15.100:0/1095691473 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f8558019070 con 0x7f8564103f20 2024-05-10T11:17:05.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.307+0000 7f85537fe700 1 --2- 172.21.15.100:0/1095691473 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f854c0b2880 0x7f854c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:05.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.307+0000 7f856259c700 1 --2- 172.21.15.100:0/1095691473 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f854c0b2880 0x7f854c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:05.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.307+0000 7f85537fe700 1 -- 172.21.15.100:0/1095691473 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f85580e21e0 con 0x7f8564103f20 2024-05-10T11:17:05.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.307+0000 7f856259c700 1 --2- 172.21.15.100:0/1095691473 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f854c0b2880 0x7f854c0b4d40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8554000c00 tx=0x7f8554005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:17:05.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.311+0000 7f85537fe700 1 -- 172.21.15.100:0/1095691473 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f85580a9ba0 con 0x7f8564103f20 2024-05-10T11:17:05.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:17:05 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/202528045' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-10T11:17:05.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.527+0000 7f8568dc9700 1 -- 172.21.15.100:0/1095691473 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f856419ad40 con 0x7f8564103f20 2024-05-10T11:17:05.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.527+0000 7f85537fe700 1 -- 172.21.15.100:0/1095691473 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f85580a94a0 con 0x7f8564103f20 2024-05-10T11:17:05.527 INFO:teuthology.orchestra.run.smithi100.stdout:111669149786 2024-05-10T11:17:05.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.529+0000 7f8568dc9700 1 -- 172.21.15.100:0/1095691473 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f854c0b2880 msgr2=0x7f854c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:05.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.529+0000 7f8568dc9700 1 --2- 172.21.15.100:0/1095691473 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f854c0b2880 0x7f854c0b4d40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8554000c00 tx=0x7f8554005fb0 comp rx=0 tx=0).stop 2024-05-10T11:17:05.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.529+0000 7f8568dc9700 1 -- 172.21.15.100:0/1095691473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8564103f20 msgr2=0x7f856419fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:05.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.529+0000 7f8568dc9700 1 --2- 172.21.15.100:0/1095691473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8564103f20 0x7f856419fbd0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f855800d950 tx=0x7f855800dc60 comp rx=0 tx=0).stop 2024-05-10T11:17:05.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.529+0000 7f8568dc9700 1 -- 172.21.15.100:0/1095691473 shutdown_connections 2024-05-10T11:17:05.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.529+0000 7f8568dc9700 1 --2- 172.21.15.100:0/1095691473 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f854c0b2880 0x7f854c0b4d40 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:05.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.529+0000 7f8568dc9700 1 --2- 172.21.15.100:0/1095691473 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8564109540 0x7f8564199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:05.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.529+0000 7f8568dc9700 1 --2- 172.21.15.100:0/1095691473 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8564103560 0x7f856419f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:05.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.529+0000 7f8568dc9700 1 --2- 172.21.15.100:0/1095691473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8564103f20 0x7f856419fbd0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:05.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.529+0000 7f8568dc9700 1 -- 172.21.15.100:0/1095691473 >> 172.21.15.100:0/1095691473 conn(0x7f856407a660 msgr2=0x7f85641023a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:17:05.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.529+0000 7f8568dc9700 1 -- 172.21.15.100:0/1095691473 shutdown_connections 2024-05-10T11:17:05.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.529+0000 7f8568dc9700 1 -- 172.21.15.100:0/1095691473 wait complete. 2024-05-10T11:17:05.776 INFO:tasks.cephadm.ceph_manager.ceph:need seq 184683593798 got 184683593803 for osd.6 2024-05-10T11:17:05.776 DEBUG:teuthology.parallel:result is None 2024-05-10T11:17:05.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.988+0000 7f354fa04700 1 -- 172.21.15.100:0/757092082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3548108880 msgr2=0x7f3548110400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:05.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.988+0000 7f354fa04700 1 --2- 172.21.15.100:0/757092082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3548108880 0x7f3548110400 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f354000b0d0 tx=0x7f354000b3e0 comp rx=0 tx=0).stop 2024-05-10T11:17:05.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.988+0000 7f354fa04700 1 -- 172.21.15.100:0/757092082 shutdown_connections 2024-05-10T11:17:05.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.988+0000 7f354fa04700 1 --2- 172.21.15.100:0/757092082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3548108880 0x7f3548110400 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:05.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.988+0000 7f354fa04700 1 --2- 172.21.15.100:0/757092082 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35481037b0 0x7f3548103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:05.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.988+0000 7f354fa04700 1 --2- 172.21.15.100:0/757092082 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3548104160 0x7f3548108150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:05.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.988+0000 7f354fa04700 1 -- 172.21.15.100:0/757092082 >> 172.21.15.100:0/757092082 conn(0x7f354807a580 msgr2=0x7f354807a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:17:05.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.988+0000 7f354fa04700 1 -- 172.21.15.100:0/757092082 shutdown_connections 2024-05-10T11:17:05.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.988+0000 7f354fa04700 1 -- 172.21.15.100:0/757092082 wait complete. 2024-05-10T11:17:05.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.989+0000 7f354fa04700 1 Processor -- start 2024-05-10T11:17:05.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.989+0000 7f354fa04700 1 -- start start 2024-05-10T11:17:05.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.989+0000 7f354fa04700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35481037b0 0x7f354819f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:05.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.989+0000 7f354fa04700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3548104160 0x7f354819fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:05.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.989+0000 7f354fa04700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3548108880 0x7f35481997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:05.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.989+0000 7f354fa04700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3548113570 con 0x7f3548108880 2024-05-10T11:17:05.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.989+0000 7f354fa04700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f35481133f0 con 0x7f35481037b0 2024-05-10T11:17:05.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.989+0000 7f354fa04700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f35481136f0 con 0x7f3548104160 2024-05-10T11:17:05.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.989+0000 7f354dfa1700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3548108880 0x7f35481997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:05.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.989+0000 7f354dfa1700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3548108880 0x7f35481997f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37768/0 (socket says 172.21.15.100:37768) 2024-05-10T11:17:05.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.989+0000 7f354dfa1700 1 -- 172.21.15.100:0/561956238 learned_addr learned my addr 172.21.15.100:0/561956238 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:17:05.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.990+0000 7f354cf9f700 1 --2- 172.21.15.100:0/561956238 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3548104160 0x7f354819fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:05.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.990+0000 7f354d7a0700 1 --2- 172.21.15.100:0/561956238 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35481037b0 0x7f354819f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:05.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.990+0000 7f354dfa1700 1 -- 172.21.15.100:0/561956238 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3548104160 msgr2=0x7f354819fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:05.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.990+0000 7f354dfa1700 1 --2- 172.21.15.100:0/561956238 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3548104160 0x7f354819fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:05.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.990+0000 7f354dfa1700 1 -- 172.21.15.100:0/561956238 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35481037b0 msgr2=0x7f354819f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:05.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.990+0000 7f354dfa1700 1 --2- 172.21.15.100:0/561956238 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35481037b0 0x7f354819f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:05.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.990+0000 7f354dfa1700 1 -- 172.21.15.100:0/561956238 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3548199fd0 con 0x7f3548108880 2024-05-10T11:17:05.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.990+0000 7f354d7a0700 1 --2- 172.21.15.100:0/561956238 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35481037b0 0x7f354819f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:17:05.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.990+0000 7f354dfa1700 1 --2- 172.21.15.100:0/561956238 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3548108880 0x7f35481997f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f3540007d90 tx=0x7f3540007dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:17:05.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.990+0000 7f353e7fc700 1 -- 172.21.15.100:0/561956238 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f354001f030 con 0x7f3548108880 2024-05-10T11:17:05.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.990+0000 7f353e7fc700 1 -- 172.21.15.100:0/561956238 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f354001bb90 con 0x7f3548108880 2024-05-10T11:17:05.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.990+0000 7f354fa04700 1 -- 172.21.15.100:0/561956238 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f354819a260 con 0x7f3548108880 2024-05-10T11:17:05.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.990+0000 7f353e7fc700 1 -- 172.21.15.100:0/561956238 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3540012bd0 con 0x7f3548108880 2024-05-10T11:17:05.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.990+0000 7f354fa04700 1 -- 172.21.15.100:0/561956238 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35481a6480 con 0x7f3548108880 2024-05-10T11:17:05.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.991+0000 7f354fa04700 1 -- 172.21.15.100:0/561956238 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3548104c10 con 0x7f3548108880 2024-05-10T11:17:05.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.993+0000 7f353e7fc700 1 -- 172.21.15.100:0/561956238 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f3540022030 con 0x7f3548108880 2024-05-10T11:17:05.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.994+0000 7f353e7fc700 1 --2- 172.21.15.100:0/561956238 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f35340b2880 0x7f35340b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:05.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.994+0000 7f354d7a0700 1 --2- 172.21.15.100:0/561956238 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f35340b2880 0x7f35340b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:05.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.994+0000 7f353e7fc700 1 -- 172.21.15.100:0/561956238 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f35400e38a0 con 0x7f3548108880 2024-05-10T11:17:05.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.995+0000 7f354d7a0700 1 --2- 172.21.15.100:0/561956238 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f35340b2880 0x7f35340b4d40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3544005fd0 tx=0x7f3544005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:17:05.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:05.997+0000 7f353e7fc700 1 -- 172.21.15.100:0/561956238 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f35400ab2f0 con 0x7f3548108880 2024-05-10T11:17:06.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:06.211+0000 7f354fa04700 1 -- 172.21.15.100:0/561956238 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f354802e510 con 0x7f3548108880 2024-05-10T11:17:06.212 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:17:06 smithi100 ceph-mon[30296]: pgmap v381: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 5 op/s 2024-05-10T11:17:06.212 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:17:06 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/1095691473' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-10T11:17:06.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:06.212+0000 7f353e7fc700 1 -- 172.21.15.100:0/561956238 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f35400aabf0 con 0x7f3548108880 2024-05-10T11:17:06.212 INFO:teuthology.orchestra.run.smithi100.stdout:137438953557 2024-05-10T11:17:06.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:06.215+0000 7f354fa04700 1 -- 172.21.15.100:0/561956238 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f35340b2880 msgr2=0x7f35340b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:06.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:06.215+0000 7f354fa04700 1 --2- 172.21.15.100:0/561956238 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f35340b2880 0x7f35340b4d40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3544005fd0 tx=0x7f3544005e30 comp rx=0 tx=0).stop 2024-05-10T11:17:06.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:06.215+0000 7f354fa04700 1 -- 172.21.15.100:0/561956238 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3548108880 msgr2=0x7f35481997f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:06.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:06.215+0000 7f354fa04700 1 --2- 172.21.15.100:0/561956238 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3548108880 0x7f35481997f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f3540007d90 tx=0x7f3540007dc0 comp rx=0 tx=0).stop 2024-05-10T11:17:06.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:06.216+0000 7f354fa04700 1 -- 172.21.15.100:0/561956238 shutdown_connections 2024-05-10T11:17:06.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:06.216+0000 7f354fa04700 1 --2- 172.21.15.100:0/561956238 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f35340b2880 0x7f35340b4d40 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:06.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:06.216+0000 7f354fa04700 1 --2- 172.21.15.100:0/561956238 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3548108880 0x7f35481997f0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:06.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:06.216+0000 7f354fa04700 1 --2- 172.21.15.100:0/561956238 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35481037b0 0x7f354819f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:06.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:06.216+0000 7f354fa04700 1 --2- 172.21.15.100:0/561956238 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3548104160 0x7f354819fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:06.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:06.216+0000 7f354fa04700 1 -- 172.21.15.100:0/561956238 >> 172.21.15.100:0/561956238 conn(0x7f354807a580 msgr2=0x7f3548102cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:17:06.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:06.216+0000 7f354fa04700 1 -- 172.21.15.100:0/561956238 shutdown_connections 2024-05-10T11:17:06.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:06.216+0000 7f354fa04700 1 -- 172.21.15.100:0/561956238 wait complete. 2024-05-10T11:17:06.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:17:06 smithi188 ceph-mon[34075]: pgmap v381: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 5 op/s 2024-05-10T11:17:06.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:17:06 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/1095691473' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-10T11:17:06.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:17:06 smithi178 ceph-mon[33359]: pgmap v381: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 5 op/s 2024-05-10T11:17:06.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:17:06 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/1095691473' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-10T11:17:07.171 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:17:07 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/561956238' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-10T11:17:07.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:17:07 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/561956238' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-10T11:17:07.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:17:07 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/561956238' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-10T11:17:08.180 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953551 got 137438953557 for osd.4 2024-05-10T11:17:08.181 DEBUG:teuthology.parallel:result is None 2024-05-10T11:17:08.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:17:08 smithi188 ceph-mon[34075]: pgmap v382: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-10T11:17:08.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:17:08 smithi100 ceph-mon[30296]: pgmap v382: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-10T11:17:08.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:17:08 smithi178 ceph-mon[33359]: pgmap v382: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-10T11:17:09.030 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149780 got 111669149786 for osd.3 2024-05-10T11:17:09.030 DEBUG:teuthology.parallel:result is None 2024-05-10T11:17:09.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:17:09 smithi188 ceph-mon[34075]: pgmap v383: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-10T11:17:09.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:17:09 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:17:09.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:17:09 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:17:09.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:17:09 smithi100 ceph-mon[30296]: pgmap v383: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-10T11:17:09.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:17:09 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:17:09.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:17:09 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:17:09.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:17:09 smithi178 ceph-mon[33359]: pgmap v383: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-10T11:17:09.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:17:09 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:17:09.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:17:09 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:17:10.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.347+0000 7fbb1ad89700 1 -- 172.21.15.100:0/1068116883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb14109520 msgr2=0x7fbb14109900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:10.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.347+0000 7fbb1ad89700 1 --2- 172.21.15.100:0/1068116883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb14109520 0x7fbb14109900 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fbb08009990 tx=0x7fbb08009ca0 comp rx=0 tx=0).stop 2024-05-10T11:17:10.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.347+0000 7fbb1ad89700 1 -- 172.21.15.100:0/1068116883 shutdown_connections 2024-05-10T11:17:10.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.347+0000 7fbb1ad89700 1 --2- 172.21.15.100:0/1068116883 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb14102450 0x7fbb14110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:10.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.347+0000 7fbb1ad89700 1 --2- 172.21.15.100:0/1068116883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb14109520 0x7fbb14109900 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:10.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.347+0000 7fbb1ad89700 1 --2- 172.21.15.100:0/1068116883 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb14101a90 0x7fbb14101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:10.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.347+0000 7fbb1ad89700 1 -- 172.21.15.100:0/1068116883 >> 172.21.15.100:0/1068116883 conn(0x7fbb14078a10 msgr2=0x7fbb140ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:17:10.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.347+0000 7fbb1ad89700 1 -- 172.21.15.100:0/1068116883 shutdown_connections 2024-05-10T11:17:10.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.347+0000 7fbb1ad89700 1 -- 172.21.15.100:0/1068116883 wait complete. 2024-05-10T11:17:10.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.348+0000 7fbb1ad89700 1 Processor -- start 2024-05-10T11:17:10.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.348+0000 7fbb1ad89700 1 -- start start 2024-05-10T11:17:10.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.348+0000 7fbb1ad89700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb14101a90 0x7fbb1419f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:10.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.348+0000 7fbb1ad89700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb14102450 0x7fbb1419fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:10.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.348+0000 7fbb1ad89700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb14109520 0x7fbb14199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:10.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.348+0000 7fbb1ad89700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbb141134e0 con 0x7fbb14101a90 2024-05-10T11:17:10.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.348+0000 7fbb1ad89700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbb14113360 con 0x7fbb14102450 2024-05-10T11:17:10.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.348+0000 7fbb1ad89700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbb14113660 con 0x7fbb14109520 2024-05-10T11:17:10.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.349+0000 7fbb18b25700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb14101a90 0x7fbb1419f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:10.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.349+0000 7fbb18b25700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb14101a90 0x7fbb1419f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41602/0 (socket says 172.21.15.100:41602) 2024-05-10T11:17:10.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.349+0000 7fbb18b25700 1 -- 172.21.15.100:0/3743371097 learned_addr learned my addr 172.21.15.100:0/3743371097 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:17:10.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.349+0000 7fbb13fff700 1 --2- 172.21.15.100:0/3743371097 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb14102450 0x7fbb1419fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:10.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.349+0000 7fbb19326700 1 --2- 172.21.15.100:0/3743371097 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb14109520 0x7fbb14199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:10.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.349+0000 7fbb18b25700 1 -- 172.21.15.100:0/3743371097 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb14109520 msgr2=0x7fbb14199860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:10.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.349+0000 7fbb18b25700 1 --2- 172.21.15.100:0/3743371097 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb14109520 0x7fbb14199860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:10.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.349+0000 7fbb18b25700 1 -- 172.21.15.100:0/3743371097 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb14102450 msgr2=0x7fbb1419fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:10.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.349+0000 7fbb18b25700 1 --2- 172.21.15.100:0/3743371097 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb14102450 0x7fbb1419fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:10.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.349+0000 7fbb18b25700 1 -- 172.21.15.100:0/3743371097 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb14199fc0 con 0x7fbb14101a90 2024-05-10T11:17:10.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.349+0000 7fbb18b25700 1 --2- 172.21.15.100:0/3743371097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb14101a90 0x7fbb1419f6a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fbb08006010 tx=0x7fbb080057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:17:10.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.349+0000 7fbb11ffb700 1 -- 172.21.15.100:0/3743371097 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb08027070 con 0x7fbb14101a90 2024-05-10T11:17:10.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.349+0000 7fbb1ad89700 1 -- 172.21.15.100:0/3743371097 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb1419a250 con 0x7fbb14101a90 2024-05-10T11:17:10.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.349+0000 7fbb11ffb700 1 -- 172.21.15.100:0/3743371097 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbb08022470 con 0x7fbb14101a90 2024-05-10T11:17:10.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.349+0000 7fbb11ffb700 1 -- 172.21.15.100:0/3743371097 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb08017650 con 0x7fbb14101a90 2024-05-10T11:17:10.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.350+0000 7fbb1ad89700 1 -- 172.21.15.100:0/3743371097 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb14102ba0 con 0x7fbb14101a90 2024-05-10T11:17:10.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.351+0000 7fbb1ad89700 1 -- 172.21.15.100:0/3743371097 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb1410ba00 con 0x7fbb14101a90 2024-05-10T11:17:10.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.353+0000 7fbb11ffb700 1 -- 172.21.15.100:0/3743371097 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fbb0802a070 con 0x7fbb14101a90 2024-05-10T11:17:10.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.354+0000 7fbb11ffb700 1 --2- 172.21.15.100:0/3743371097 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fbb040b2880 0x7fbb040b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:10.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.354+0000 7fbb13fff700 1 --2- 172.21.15.100:0/3743371097 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fbb040b2880 0x7fbb040b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:10.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.354+0000 7fbb11ffb700 1 -- 172.21.15.100:0/3743371097 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fbb080e34c0 con 0x7fbb14101a90 2024-05-10T11:17:10.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.355+0000 7fbb13fff700 1 --2- 172.21.15.100:0/3743371097 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fbb040b2880 0x7fbb040b4d40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbb140ffb60 tx=0x7fbb000073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:17:10.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.359+0000 7fbb11ffb700 1 -- 172.21.15.100:0/3743371097 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fbb08013070 con 0x7fbb14101a90 2024-05-10T11:17:10.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.242+0000 7f217f9e0700 1 -- 172.21.15.100:0/3352127320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f217806ba10 msgr2=0x7f217806be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:10.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.242+0000 7f217f9e0700 1 --2- 172.21.15.100:0/3352127320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f217806ba10 0x7f217806be90 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2168009990 tx=0x7f2168009ca0 comp rx=0 tx=0).stop 2024-05-10T11:17:10.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.243+0000 7f217f9e0700 1 -- 172.21.15.100:0/3352127320 shutdown_connections 2024-05-10T11:17:10.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.243+0000 7f217f9e0700 1 --2- 172.21.15.100:0/3352127320 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f217806c3d0 0x7f2178110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:10.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.243+0000 7f217f9e0700 1 --2- 172.21.15.100:0/3352127320 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2178109500 0x7f21781098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:10.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.243+0000 7f217f9e0700 1 --2- 172.21.15.100:0/3352127320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f217806ba10 0x7f217806be90 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:10.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.243+0000 7f217f9e0700 1 -- 172.21.15.100:0/3352127320 >> 172.21.15.100:0/3352127320 conn(0x7f2178079310 msgr2=0x7f2178079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:17:10.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.243+0000 7f217f9e0700 1 -- 172.21.15.100:0/3352127320 shutdown_connections 2024-05-10T11:17:10.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.243+0000 7f217f9e0700 1 -- 172.21.15.100:0/3352127320 wait complete. 2024-05-10T11:17:10.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.243+0000 7f217f9e0700 1 Processor -- start 2024-05-10T11:17:10.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.244+0000 7f217f9e0700 1 -- start start 2024-05-10T11:17:10.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.244+0000 7f217f9e0700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f217806ba10 0x7f217819f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:10.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.244+0000 7f217f9e0700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f217806c3d0 0x7f217819fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:10.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.244+0000 7f217f9e0700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2178109500 0x7f21781998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:10.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.244+0000 7f217f9e0700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f21781134e0 con 0x7f217806ba10 2024-05-10T11:17:10.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.244+0000 7f217f9e0700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2178113360 con 0x7f217806c3d0 2024-05-10T11:17:10.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.244+0000 7f217f9e0700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2178113660 con 0x7f2178109500 2024-05-10T11:17:10.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.244+0000 7f217d77c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f217806ba10 0x7f217819f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:10.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.244+0000 7f217d77c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f217806ba10 0x7f217819f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41580/0 (socket says 172.21.15.100:41580) 2024-05-10T11:17:10.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.244+0000 7f217d77c700 1 -- 172.21.15.100:0/2338392244 learned_addr learned my addr 172.21.15.100:0/2338392244 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:17:10.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.244+0000 7f217df7d700 1 --2- 172.21.15.100:0/2338392244 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2178109500 0x7f21781998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:10.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.244+0000 7f217cf7b700 1 --2- 172.21.15.100:0/2338392244 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f217806c3d0 0x7f217819fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:10.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.244+0000 7f217d77c700 1 -- 172.21.15.100:0/2338392244 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2178109500 msgr2=0x7f21781998c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:10.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.244+0000 7f217d77c700 1 --2- 172.21.15.100:0/2338392244 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2178109500 0x7f21781998c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:10.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.244+0000 7f217d77c700 1 -- 172.21.15.100:0/2338392244 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f217806c3d0 msgr2=0x7f217819fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:10.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.244+0000 7f217d77c700 1 --2- 172.21.15.100:0/2338392244 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f217806c3d0 0x7f217819fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:10.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.244+0000 7f217d77c700 1 -- 172.21.15.100:0/2338392244 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f217819a020 con 0x7f217806ba10 2024-05-10T11:17:10.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.245+0000 7f217d77c700 1 --2- 172.21.15.100:0/2338392244 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f217806ba10 0x7f217819f700 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f217400d950 tx=0x7f217400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:17:10.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.245+0000 7f216e7fc700 1 -- 172.21.15.100:0/2338392244 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2174016070 con 0x7f217806ba10 2024-05-10T11:17:10.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.245+0000 7f216e7fc700 1 -- 172.21.15.100:0/2338392244 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f217400fd20 con 0x7f217806ba10 2024-05-10T11:17:10.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.245+0000 7f216e7fc700 1 -- 172.21.15.100:0/2338392244 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f217400e3b0 con 0x7f217806ba10 2024-05-10T11:17:10.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.245+0000 7f217f9e0700 1 -- 172.21.15.100:0/2338392244 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f217819a310 con 0x7f217806ba10 2024-05-10T11:17:10.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.245+0000 7f217f9e0700 1 -- 172.21.15.100:0/2338392244 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2178102390 con 0x7f217806ba10 2024-05-10T11:17:10.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.246+0000 7f217f9e0700 1 -- 172.21.15.100:0/2338392244 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f217806a190 con 0x7f217806ba10 2024-05-10T11:17:10.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.247+0000 7f216e7fc700 1 -- 172.21.15.100:0/2338392244 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f2174019070 con 0x7f217806ba10 2024-05-10T11:17:10.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.248+0000 7f216e7fc700 1 --2- 172.21.15.100:0/2338392244 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f21640b2880 0x7f21640b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:10.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.248+0000 7f217cf7b700 1 --2- 172.21.15.100:0/2338392244 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f21640b2880 0x7f21640b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:10.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.248+0000 7f216e7fc700 1 -- 172.21.15.100:0/2338392244 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f21740e2000 con 0x7f217806ba10 2024-05-10T11:17:10.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.249+0000 7f217cf7b700 1 --2- 172.21.15.100:0/2338392244 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f21640b2880 0x7f21640b4d40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f216800b5c0 tx=0x7f2168005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:17:10.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.253+0000 7f216e7fc700 1 -- 172.21.15.100:0/2338392244 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f21740a99c0 con 0x7f217806ba10 2024-05-10T11:17:10.466 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:17:10 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:17:10.466 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:17:10 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:17:10.466 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:17:10 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:17:10.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.500+0000 7f217f9e0700 1 -- 172.21.15.100:0/2338392244 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f217802e510 con 0x7f217806ba10 2024-05-10T11:17:10.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.500+0000 7f216e7fc700 1 -- 172.21.15.100:0/2338392244 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f21740a92c0 con 0x7f217806ba10 2024-05-10T11:17:10.500 INFO:teuthology.orchestra.run.smithi100.stdout:236223201345 2024-05-10T11:17:10.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.502+0000 7f217f9e0700 1 -- 172.21.15.100:0/2338392244 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f21640b2880 msgr2=0x7f21640b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:10.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.502+0000 7f217f9e0700 1 --2- 172.21.15.100:0/2338392244 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f21640b2880 0x7f21640b4d40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f216800b5c0 tx=0x7f2168005960 comp rx=0 tx=0).stop 2024-05-10T11:17:10.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.502+0000 7f217f9e0700 1 -- 172.21.15.100:0/2338392244 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f217806ba10 msgr2=0x7f217819f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:10.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.502+0000 7f217f9e0700 1 --2- 172.21.15.100:0/2338392244 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f217806ba10 0x7f217819f700 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f217400d950 tx=0x7f217400dc60 comp rx=0 tx=0).stop 2024-05-10T11:17:10.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.502+0000 7f217f9e0700 1 -- 172.21.15.100:0/2338392244 shutdown_connections 2024-05-10T11:17:10.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.502+0000 7f217f9e0700 1 --2- 172.21.15.100:0/2338392244 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f21640b2880 0x7f21640b4d40 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:10.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.502+0000 7f217f9e0700 1 --2- 172.21.15.100:0/2338392244 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2178109500 0x7f21781998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:10.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.502+0000 7f217f9e0700 1 --2- 172.21.15.100:0/2338392244 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f217806ba10 0x7f217819f700 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:10.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.502+0000 7f217f9e0700 1 --2- 172.21.15.100:0/2338392244 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f217806c3d0 0x7f217819fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:10.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.502+0000 7f217f9e0700 1 -- 172.21.15.100:0/2338392244 >> 172.21.15.100:0/2338392244 conn(0x7f2178079310 msgr2=0x7f2178101260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:17:10.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.503+0000 7f217f9e0700 1 -- 172.21.15.100:0/2338392244 shutdown_connections 2024-05-10T11:17:10.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.503+0000 7f217f9e0700 1 -- 172.21.15.100:0/2338392244 wait complete. 2024-05-10T11:17:10.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:17:10 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:17:10.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:17:10 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:17:10.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:17:10 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:17:10.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.571+0000 7fbb1ad89700 1 -- 172.21.15.100:0/3743371097 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fbb1419ad90 con 0x7fbb14101a90 2024-05-10T11:17:10.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.571+0000 7fbb11ffb700 1 -- 172.21.15.100:0/3743371097 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fbb080aaf10 con 0x7fbb14101a90 2024-05-10T11:17:10.571 INFO:teuthology.orchestra.run.smithi100.stdout:261993005116 2024-05-10T11:17:10.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.573+0000 7fbb1ad89700 1 -- 172.21.15.100:0/3743371097 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fbb040b2880 msgr2=0x7fbb040b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:10.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.573+0000 7fbb1ad89700 1 --2- 172.21.15.100:0/3743371097 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fbb040b2880 0x7fbb040b4d40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbb140ffb60 tx=0x7fbb000073d0 comp rx=0 tx=0).stop 2024-05-10T11:17:10.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.573+0000 7fbb1ad89700 1 -- 172.21.15.100:0/3743371097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb14101a90 msgr2=0x7fbb1419f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:10.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.573+0000 7fbb1ad89700 1 --2- 172.21.15.100:0/3743371097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb14101a90 0x7fbb1419f6a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fbb08006010 tx=0x7fbb080057c0 comp rx=0 tx=0).stop 2024-05-10T11:17:10.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.573+0000 7fbb1ad89700 1 -- 172.21.15.100:0/3743371097 shutdown_connections 2024-05-10T11:17:10.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.573+0000 7fbb1ad89700 1 --2- 172.21.15.100:0/3743371097 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fbb040b2880 0x7fbb040b4d40 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:10.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.573+0000 7fbb1ad89700 1 --2- 172.21.15.100:0/3743371097 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb14109520 0x7fbb14199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:10.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.573+0000 7fbb1ad89700 1 --2- 172.21.15.100:0/3743371097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb14101a90 0x7fbb1419f6a0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:10.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.573+0000 7fbb1ad89700 1 --2- 172.21.15.100:0/3743371097 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb14102450 0x7fbb1419fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:10.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.573+0000 7fbb1ad89700 1 -- 172.21.15.100:0/3743371097 >> 172.21.15.100:0/3743371097 conn(0x7fbb14078a10 msgr2=0x7fbb14100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:17:10.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.573+0000 7fbb1ad89700 1 -- 172.21.15.100:0/3743371097 shutdown_connections 2024-05-10T11:17:10.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:10.573+0000 7fbb1ad89700 1 -- 172.21.15.100:0/3743371097 wait complete. 2024-05-10T11:17:10.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:17:10 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:17:10.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:17:10 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:17:10.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:17:10 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:17:11.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:17:11 smithi100 ceph-mon[30296]: pgmap v384: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-10T11:17:11.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:17:11 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/2338392244' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-10T11:17:11.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:17:11 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/3743371097' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-10T11:17:11.431 INFO:tasks.cephadm.ceph_manager.ceph:need seq 261993005111 got 261993005116 for osd.9 2024-05-10T11:17:11.431 DEBUG:teuthology.parallel:result is None 2024-05-10T11:17:11.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:17:11 smithi188 ceph-mon[34075]: pgmap v384: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-10T11:17:11.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:17:11 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/2338392244' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-10T11:17:11.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:17:11 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/3743371097' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-10T11:17:11.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:17:11 smithi178 ceph-mon[33359]: pgmap v384: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-10T11:17:11.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:17:11 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/2338392244' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-10T11:17:11.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:17:11 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/3743371097' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-10T11:17:11.944 INFO:tasks.cephadm.ceph_manager.ceph:need seq 236223201339 got 236223201345 for osd.8 2024-05-10T11:17:11.944 DEBUG:teuthology.parallel:result is None 2024-05-10T11:17:11.944 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-10T11:17:11.944 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-10T11:17:12.195 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:17:13.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:17:13 smithi188 ceph-mon[34075]: pgmap v385: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-05-10T11:17:13.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:17:13 smithi100 ceph-mon[30296]: pgmap v385: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-05-10T11:17:13.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:17:13 smithi178 ceph-mon[33359]: pgmap v385: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-05-10T11:17:13.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:13.965+0000 7ff2c208f700 1 -- 172.21.15.100:0/3496831208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2bc109510 msgr2=0x7ff2bc1098f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:13.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:13.965+0000 7ff2c208f700 1 --2- 172.21.15.100:0/3496831208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2bc109510 0x7ff2bc1098f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac009990 tx=0x7ff2ac009ca0 comp rx=0 tx=0).stop 2024-05-10T11:17:13.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:13.965+0000 7ff2c208f700 1 -- 172.21.15.100:0/3496831208 shutdown_connections 2024-05-10T11:17:13.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:13.965+0000 7ff2c208f700 1 --2- 172.21.15.100:0/3496831208 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2bc06bb40 0x7ff2bc110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:13.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:13.965+0000 7ff2c208f700 1 --2- 172.21.15.100:0/3496831208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2bc109510 0x7ff2bc1098f0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:13.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:13.965+0000 7ff2c208f700 1 --2- 172.21.15.100:0/3496831208 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2bc06b180 0x7ff2bc06b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:13.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:13.965+0000 7ff2c208f700 1 -- 172.21.15.100:0/3496831208 >> 172.21.15.100:0/3496831208 conn(0x7ff2bc079350 msgr2=0x7ff2bc079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:17:13.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:13.965+0000 7ff2c208f700 1 -- 172.21.15.100:0/3496831208 shutdown_connections 2024-05-10T11:17:13.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:13.965+0000 7ff2c208f700 1 -- 172.21.15.100:0/3496831208 wait complete. 2024-05-10T11:17:13.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:13.965+0000 7ff2c208f700 1 Processor -- start 2024-05-10T11:17:13.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:13.966+0000 7ff2c208f700 1 -- start start 2024-05-10T11:17:13.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:13.966+0000 7ff2c208f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2bc06b180 0x7ff2bc19f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:13.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:13.966+0000 7ff2c208f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2bc06bb40 0x7ff2bc19fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:13.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:13.966+0000 7ff2c208f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2bc109510 0x7ff2bc199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:13.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:13.966+0000 7ff2c208f700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff2bc1134f0 con 0x7ff2bc109510 2024-05-10T11:17:13.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:13.966+0000 7ff2c208f700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff2bc113370 con 0x7ff2bc06bb40 2024-05-10T11:17:13.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:13.966+0000 7ff2c208f700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff2bc113670 con 0x7ff2bc06b180 2024-05-10T11:17:13.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:13.966+0000 7ff2bbfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2bc109510 0x7ff2bc199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:13.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:13.966+0000 7ff2bbfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2bc109510 0x7ff2bc199830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41618/0 (socket says 172.21.15.100:41618) 2024-05-10T11:17:13.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:13.966+0000 7ff2bbfff700 1 -- 172.21.15.100:0/2777233229 learned_addr learned my addr 172.21.15.100:0/2777233229 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:17:13.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:13.966+0000 7ff2bb7fe700 1 --2- 172.21.15.100:0/2777233229 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2bc06b180 0x7ff2bc19f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:13.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:13.966+0000 7ff2baffd700 1 --2- 172.21.15.100:0/2777233229 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2bc06bb40 0x7ff2bc19fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:13.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:13.966+0000 7ff2bbfff700 1 -- 172.21.15.100:0/2777233229 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2bc06b180 msgr2=0x7ff2bc19f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:13.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:13.966+0000 7ff2bbfff700 1 --2- 172.21.15.100:0/2777233229 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2bc06b180 0x7ff2bc19f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:13.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:13.966+0000 7ff2bbfff700 1 -- 172.21.15.100:0/2777233229 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2bc06bb40 msgr2=0x7ff2bc19fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:13.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:13.966+0000 7ff2bbfff700 1 --2- 172.21.15.100:0/2777233229 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2bc06bb40 0x7ff2bc19fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:13.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:13.966+0000 7ff2bbfff700 1 -- 172.21.15.100:0/2777233229 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2bc19a010 con 0x7ff2bc109510 2024-05-10T11:17:13.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:13.966+0000 7ff2bb7fe700 1 --2- 172.21.15.100:0/2777233229 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2bc06b180 0x7ff2bc19f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:17:13.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:13.966+0000 7ff2bbfff700 1 --2- 172.21.15.100:0/2777233229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2bc109510 0x7ff2bc199830 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff2b4007d70 tx=0x7ff2b400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:17:13.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:13.967+0000 7ff2b8ff9700 1 -- 172.21.15.100:0/2777233229 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b4016020 con 0x7ff2bc109510 2024-05-10T11:17:13.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:13.967+0000 7ff2b8ff9700 1 -- 172.21.15.100:0/2777233229 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff2b400ac90 con 0x7ff2bc109510 2024-05-10T11:17:13.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:13.967+0000 7ff2c208f700 1 -- 172.21.15.100:0/2777233229 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2bc19a300 con 0x7ff2bc109510 2024-05-10T11:17:13.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:13.967+0000 7ff2b8ff9700 1 -- 172.21.15.100:0/2777233229 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b40115e0 con 0x7ff2bc109510 2024-05-10T11:17:13.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:13.967+0000 7ff2c208f700 1 -- 172.21.15.100:0/2777233229 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2bc1a6510 con 0x7ff2bc109510 2024-05-10T11:17:13.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:13.968+0000 7ff2c208f700 1 -- 172.21.15.100:0/2777233229 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2bc10b9f0 con 0x7ff2bc109510 2024-05-10T11:17:13.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:13.969+0000 7ff2b8ff9700 1 -- 172.21.15.100:0/2777233229 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7ff2b4019020 con 0x7ff2bc109510 2024-05-10T11:17:13.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:13.970+0000 7ff2b8ff9700 1 --2- 172.21.15.100:0/2777233229 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff2a40b2880 0x7ff2a40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:13.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:13.970+0000 7ff2bb7fe700 1 --2- 172.21.15.100:0/2777233229 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff2a40b2880 0x7ff2a40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:13.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:13.970+0000 7ff2b8ff9700 1 -- 172.21.15.100:0/2777233229 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7ff2b40e29e0 con 0x7ff2bc109510 2024-05-10T11:17:13.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:13.970+0000 7ff2bb7fe700 1 --2- 172.21.15.100:0/2777233229 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff2a40b2880 0x7ff2a40b4d40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac00b2e0 tx=0x7ff2ac01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:17:13.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:13.973+0000 7ff2b8ff9700 1 -- 172.21.15.100:0/2777233229 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7ff2b4020020 con 0x7ff2bc109510 2024-05-10T11:17:14.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:14.178+0000 7ff2c208f700 1 -- 172.21.15.100:0/2777233229 --> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff2bc0638e0 con 0x7ff2a40b2880 2024-05-10T11:17:14.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:14.189+0000 7ff2b8ff9700 1 -- 172.21.15.100:0/2777233229 <== mgr.14152 v2:172.21.15.100:6800/2507287356 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340468 (secure 0 0 0) 0x7ff2bc0638e0 con 0x7ff2a40b2880 2024-05-10T11:17:14.195 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:17:14.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:14.193+0000 7ff2c208f700 1 -- 172.21.15.100:0/2777233229 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff2a40b2880 msgr2=0x7ff2a40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:14.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:14.193+0000 7ff2c208f700 1 --2- 172.21.15.100:0/2777233229 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff2a40b2880 0x7ff2a40b4d40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac00b2e0 tx=0x7ff2ac01a040 comp rx=0 tx=0).stop 2024-05-10T11:17:14.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:14.193+0000 7ff2c208f700 1 -- 172.21.15.100:0/2777233229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2bc109510 msgr2=0x7ff2bc199830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:14.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:14.193+0000 7ff2c208f700 1 --2- 172.21.15.100:0/2777233229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2bc109510 0x7ff2bc199830 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff2b4007d70 tx=0x7ff2b400a340 comp rx=0 tx=0).stop 2024-05-10T11:17:14.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:14.193+0000 7ff2c208f700 1 -- 172.21.15.100:0/2777233229 shutdown_connections 2024-05-10T11:17:14.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:14.193+0000 7ff2c208f700 1 --2- 172.21.15.100:0/2777233229 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff2a40b2880 0x7ff2a40b4d40 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:14.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:14.193+0000 7ff2c208f700 1 --2- 172.21.15.100:0/2777233229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2bc109510 0x7ff2bc199830 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:14.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:14.193+0000 7ff2c208f700 1 --2- 172.21.15.100:0/2777233229 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2bc06b180 0x7ff2bc19f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:14.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:14.193+0000 7ff2c208f700 1 --2- 172.21.15.100:0/2777233229 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2bc06bb40 0x7ff2bc19fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:14.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:14.193+0000 7ff2c208f700 1 -- 172.21.15.100:0/2777233229 >> 172.21.15.100:0/2777233229 conn(0x7ff2bc079350 msgr2=0x7ff2bc101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:17:14.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:14.193+0000 7ff2c208f700 1 -- 172.21.15.100:0/2777233229 shutdown_connections 2024-05-10T11:17:14.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:14.193+0000 7ff2c208f700 1 -- 172.21.15.100:0/2777233229 wait complete. 2024-05-10T11:17:14.199 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-05-10T11:17:15.006 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":385,"stamp":"2024-05-10T11:17:12.226251+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659661,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":700,"num_read_kb":452,"num_write":304,"num_write_kb":2279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":747,"ondisk_log_size":747,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":404704,"kb_used_data":13952,"kb_used_omap":18,"kb_used_meta":390637,"kb_avail":1124537120,"statfs":{"total":1151940427776,"available":1151526010880,"internally_reserved":0,"allocated":14286848,"data_stored":8015789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":400012603},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":531,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":88,"num_read_kb":48,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003001"},"pg_stats":[{"pgid":"3.3f","version":"89'8","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.206983+0000","last_change":"2024-05-10T11:14:25.286878+0000","last_active":"2024-05-10T11:17:00.206983+0000","last_peered":"2024-05-10T11:17:00.206983+0000","last_clean":"2024-05-10T11:17:00.206983+0000","last_became_active":"2024-05-10T11:14:25.286462+0000","last_became_peered":"2024-05-10T11:14:25.286462+0000","last_unstale":"2024-05-10T11:17:00.206983+0000","last_undegraded":"2024-05-10T11:17:00.206983+0000","last_fullsized":"2024-05-10T11:17:00.206983+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:13:40.627138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.204906+0000","last_change":"2024-05-10T11:14:25.280884+0000","last_active":"2024-05-10T11:17:00.204906+0000","last_peered":"2024-05-10T11:17:00.204906+0000","last_clean":"2024-05-10T11:17:00.204906+0000","last_became_active":"2024-05-10T11:14:25.280564+0000","last_became_peered":"2024-05-10T11:14:25.280564+0000","last_unstale":"2024-05-10T11:17:00.204906+0000","last_undegraded":"2024-05-10T11:17:00.204906+0000","last_fullsized":"2024-05-10T11:17:00.204906+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:14:16.735326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.208402+0000","last_change":"2024-05-10T11:14:25.278676+0000","last_active":"2024-05-10T11:17:00.208402+0000","last_peered":"2024-05-10T11:17:00.208402+0000","last_clean":"2024-05-10T11:17:00.208402+0000","last_became_active":"2024-05-10T11:14:25.278196+0000","last_became_peered":"2024-05-10T11:14:25.278196+0000","last_unstale":"2024-05-10T11:17:00.208402+0000","last_undegraded":"2024-05-10T11:17:00.208402+0000","last_fullsized":"2024-05-10T11:17:00.208402+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:50:29.390014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.207221+0000","last_change":"2024-05-10T11:14:25.274741+0000","last_active":"2024-05-10T11:17:00.207221+0000","last_peered":"2024-05-10T11:17:00.207221+0000","last_clean":"2024-05-10T11:17:00.207221+0000","last_became_active":"2024-05-10T11:14:25.274251+0000","last_became_peered":"2024-05-10T11:14:25.274251+0000","last_unstale":"2024-05-10T11:17:00.207221+0000","last_undegraded":"2024-05-10T11:17:00.207221+0000","last_fullsized":"2024-05-10T11:17:00.207221+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:04:41.552042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.186404+0000","last_change":"2024-05-10T11:14:25.265232+0000","last_active":"2024-05-10T11:17:00.186404+0000","last_peered":"2024-05-10T11:17:00.186404+0000","last_clean":"2024-05-10T11:17:00.186404+0000","last_became_active":"2024-05-10T11:14:25.264852+0000","last_became_peered":"2024-05-10T11:14:25.264852+0000","last_unstale":"2024-05-10T11:17:00.186404+0000","last_undegraded":"2024-05-10T11:17:00.186404+0000","last_fullsized":"2024-05-10T11:17:00.186404+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:50:27.207258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.210357+0000","last_change":"2024-05-10T11:14:25.272626+0000","last_active":"2024-05-10T11:17:00.210357+0000","last_peered":"2024-05-10T11:17:00.210357+0000","last_clean":"2024-05-10T11:17:00.210357+0000","last_became_active":"2024-05-10T11:14:25.272305+0000","last_became_peered":"2024-05-10T11:14:25.272305+0000","last_unstale":"2024-05-10T11:17:00.210357+0000","last_undegraded":"2024-05-10T11:17:00.210357+0000","last_fullsized":"2024-05-10T11:17:00.210357+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:44:16.725453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.210435+0000","last_change":"2024-05-10T11:14:25.275460+0000","last_active":"2024-05-10T11:17:00.210435+0000","last_peered":"2024-05-10T11:17:00.210435+0000","last_clean":"2024-05-10T11:17:00.210435+0000","last_became_active":"2024-05-10T11:14:25.274571+0000","last_became_peered":"2024-05-10T11:14:25.274571+0000","last_unstale":"2024-05-10T11:17:00.210435+0000","last_undegraded":"2024-05-10T11:17:00.210435+0000","last_fullsized":"2024-05-10T11:17:00.210435+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:33:20.335738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.207412+0000","last_change":"2024-05-10T11:14:25.273975+0000","last_active":"2024-05-10T11:17:00.207412+0000","last_peered":"2024-05-10T11:17:00.207412+0000","last_clean":"2024-05-10T11:17:00.207412+0000","last_became_active":"2024-05-10T11:14:25.273652+0000","last_became_peered":"2024-05-10T11:14:25.273652+0000","last_unstale":"2024-05-10T11:17:00.207412+0000","last_undegraded":"2024-05-10T11:17:00.207412+0000","last_fullsized":"2024-05-10T11:17:00.207412+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:37:55.354449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.207494+0000","last_change":"2024-05-10T11:14:25.285530+0000","last_active":"2024-05-10T11:17:00.207494+0000","last_peered":"2024-05-10T11:17:00.207494+0000","last_clean":"2024-05-10T11:17:00.207494+0000","last_became_active":"2024-05-10T11:14:25.285235+0000","last_became_peered":"2024-05-10T11:14:25.285235+0000","last_unstale":"2024-05-10T11:17:00.207494+0000","last_undegraded":"2024-05-10T11:17:00.207494+0000","last_fullsized":"2024-05-10T11:17:00.207494+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:11:14.225339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.210587+0000","last_change":"2024-05-10T11:14:25.274738+0000","last_active":"2024-05-10T11:17:00.210587+0000","last_peered":"2024-05-10T11:17:00.210587+0000","last_clean":"2024-05-10T11:17:00.210587+0000","last_became_active":"2024-05-10T11:14:25.274107+0000","last_became_peered":"2024-05-10T11:14:25.274107+0000","last_unstale":"2024-05-10T11:17:00.210587+0000","last_undegraded":"2024-05-10T11:17:00.210587+0000","last_fullsized":"2024-05-10T11:17:00.210587+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:02:01.489930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.213489+0000","last_change":"2024-05-10T11:14:25.277431+0000","last_active":"2024-05-10T11:17:00.213489+0000","last_peered":"2024-05-10T11:17:00.213489+0000","last_clean":"2024-05-10T11:17:00.213489+0000","last_became_active":"2024-05-10T11:14:25.277130+0000","last_became_peered":"2024-05-10T11:14:25.277130+0000","last_unstale":"2024-05-10T11:17:00.213489+0000","last_undegraded":"2024-05-10T11:17:00.213489+0000","last_fullsized":"2024-05-10T11:17:00.213489+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:38:17.192282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.206976+0000","last_change":"2024-05-10T11:14:25.276291+0000","last_active":"2024-05-10T11:17:00.206976+0000","last_peered":"2024-05-10T11:17:00.206976+0000","last_clean":"2024-05-10T11:17:00.206976+0000","last_became_active":"2024-05-10T11:14:25.275842+0000","last_became_peered":"2024-05-10T11:14:25.275842+0000","last_unstale":"2024-05-10T11:17:00.206976+0000","last_undegraded":"2024-05-10T11:17:00.206976+0000","last_fullsized":"2024-05-10T11:17:00.206976+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:20:53.443135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.209964+0000","last_change":"2024-05-10T11:14:25.273950+0000","last_active":"2024-05-10T11:17:00.209964+0000","last_peered":"2024-05-10T11:17:00.209964+0000","last_clean":"2024-05-10T11:17:00.209964+0000","last_became_active":"2024-05-10T11:14:25.273644+0000","last_became_peered":"2024-05-10T11:14:25.273644+0000","last_unstale":"2024-05-10T11:17:00.209964+0000","last_undegraded":"2024-05-10T11:17:00.209964+0000","last_fullsized":"2024-05-10T11:17:00.209964+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:46:05.112575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.209077+0000","last_change":"2024-05-10T11:14:25.277930+0000","last_active":"2024-05-10T11:17:00.209077+0000","last_peered":"2024-05-10T11:17:00.209077+0000","last_clean":"2024-05-10T11:17:00.209077+0000","last_became_active":"2024-05-10T11:14:25.276793+0000","last_became_peered":"2024-05-10T11:14:25.276793+0000","last_unstale":"2024-05-10T11:17:00.209077+0000","last_undegraded":"2024-05-10T11:17:00.209077+0000","last_fullsized":"2024-05-10T11:17:00.209077+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:47:03.592882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.214057+0000","last_change":"2024-05-10T11:14:25.270434+0000","last_active":"2024-05-10T11:17:00.214057+0000","last_peered":"2024-05-10T11:17:00.214057+0000","last_clean":"2024-05-10T11:17:00.214057+0000","last_became_active":"2024-05-10T11:14:25.270139+0000","last_became_peered":"2024-05-10T11:14:25.270139+0000","last_unstale":"2024-05-10T11:17:00.214057+0000","last_undegraded":"2024-05-10T11:17:00.214057+0000","last_fullsized":"2024-05-10T11:17:00.214057+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:58:40.506560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.209206+0000","last_change":"2024-05-10T11:14:25.272981+0000","last_active":"2024-05-10T11:17:00.209206+0000","last_peered":"2024-05-10T11:17:00.209206+0000","last_clean":"2024-05-10T11:17:00.209206+0000","last_became_active":"2024-05-10T11:14:25.272592+0000","last_became_peered":"2024-05-10T11:14:25.272592+0000","last_unstale":"2024-05-10T11:17:00.209206+0000","last_undegraded":"2024-05-10T11:17:00.209206+0000","last_fullsized":"2024-05-10T11:17:00.209206+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:46:09.673013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.212493+0000","last_change":"2024-05-10T11:14:25.287304+0000","last_active":"2024-05-10T11:17:00.212493+0000","last_peered":"2024-05-10T11:17:00.212493+0000","last_clean":"2024-05-10T11:17:00.212493+0000","last_became_active":"2024-05-10T11:14:25.286964+0000","last_became_peered":"2024-05-10T11:14:25.286964+0000","last_unstale":"2024-05-10T11:17:00.212493+0000","last_undegraded":"2024-05-10T11:17:00.212493+0000","last_fullsized":"2024-05-10T11:17:00.212493+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:32:27.006840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.209855+0000","last_change":"2024-05-10T11:14:25.275366+0000","last_active":"2024-05-10T11:17:00.209855+0000","last_peered":"2024-05-10T11:17:00.209855+0000","last_clean":"2024-05-10T11:17:00.209855+0000","last_became_active":"2024-05-10T11:14:25.274986+0000","last_became_peered":"2024-05-10T11:14:25.274986+0000","last_unstale":"2024-05-10T11:17:00.209855+0000","last_undegraded":"2024-05-10T11:17:00.209855+0000","last_fullsized":"2024-05-10T11:17:00.209855+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:50:50.071975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.203474+0000","last_change":"2024-05-10T11:14:25.274588+0000","last_active":"2024-05-10T11:17:00.203474+0000","last_peered":"2024-05-10T11:17:00.203474+0000","last_clean":"2024-05-10T11:17:00.203474+0000","last_became_active":"2024-05-10T11:14:25.274179+0000","last_became_peered":"2024-05-10T11:14:25.274179+0000","last_unstale":"2024-05-10T11:17:00.203474+0000","last_undegraded":"2024-05-10T11:17:00.203474+0000","last_fullsized":"2024-05-10T11:17:00.203474+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:06:23.517082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.461771+0000","last_change":"2024-05-10T11:15:01.461337+0000","last_active":"2024-05-10T11:15:01.461771+0000","last_peered":"2024-05-10T11:15:01.461771+0000","last_clean":"2024-05-10T11:15:01.461771+0000","last_became_active":"2024-05-10T11:15:01.460776+0000","last_became_peered":"2024-05-10T11:15:01.460776+0000","last_unstale":"2024-05-10T11:15:01.461771+0000","last_undegraded":"2024-05-10T11:15:01.461771+0000","last_fullsized":"2024-05-10T11:15:01.461771+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:06:49.156483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,10,0],"acting":[2,3,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:32.813572+0000","last_change":"2024-05-10T11:14:22.253909+0000","last_active":"2024-05-10T11:15:32.813572+0000","last_peered":"2024-05-10T11:15:32.813572+0000","last_clean":"2024-05-10T11:15:32.813572+0000","last_became_active":"2024-05-10T11:14:22.253400+0000","last_became_peered":"2024-05-10T11:14:22.253400+0000","last_unstale":"2024-05-10T11:15:32.813572+0000","last_undegraded":"2024-05-10T11:15:32.813572+0000","last_fullsized":"2024-05-10T11:15:32.813572+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:06:46.173511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.206338+0000","last_change":"2024-05-10T11:14:25.273675+0000","last_active":"2024-05-10T11:17:00.206338+0000","last_peered":"2024-05-10T11:17:00.206338+0000","last_clean":"2024-05-10T11:17:00.206338+0000","last_became_active":"2024-05-10T11:14:25.273320+0000","last_became_peered":"2024-05-10T11:14:25.273320+0000","last_unstale":"2024-05-10T11:17:00.206338+0000","last_undegraded":"2024-05-10T11:17:00.206338+0000","last_fullsized":"2024-05-10T11:17:00.206338+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:09:18.232236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459008+0000","last_change":"2024-05-10T11:14:35.326204+0000","last_active":"2024-05-10T11:15:01.459008+0000","last_peered":"2024-05-10T11:15:01.459008+0000","last_clean":"2024-05-10T11:15:01.459008+0000","last_became_active":"2024-05-10T11:14:35.325715+0000","last_became_peered":"2024-05-10T11:14:35.325715+0000","last_unstale":"2024-05-10T11:15:01.459008+0000","last_undegraded":"2024-05-10T11:15:01.459008+0000","last_fullsized":"2024-05-10T11:15:01.459008+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:06:30.665861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:29.796053+0000","last_change":"2024-05-10T11:14:22.299194+0000","last_active":"2024-05-10T11:15:29.796053+0000","last_peered":"2024-05-10T11:15:29.796053+0000","last_clean":"2024-05-10T11:15:29.796053+0000","last_became_active":"2024-05-10T11:14:22.298807+0000","last_became_peered":"2024-05-10T11:14:22.298807+0000","last_unstale":"2024-05-10T11:15:29.796053+0000","last_undegraded":"2024-05-10T11:15:29.796053+0000","last_fullsized":"2024-05-10T11:15:29.796053+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:24:51.054834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.209768+0000","last_change":"2024-05-10T11:14:25.277860+0000","last_active":"2024-05-10T11:17:00.209768+0000","last_peered":"2024-05-10T11:17:00.209768+0000","last_clean":"2024-05-10T11:17:00.209768+0000","last_became_active":"2024-05-10T11:14:25.276784+0000","last_became_peered":"2024-05-10T11:14:25.276784+0000","last_unstale":"2024-05-10T11:17:00.209768+0000","last_undegraded":"2024-05-10T11:17:00.209768+0000","last_fullsized":"2024-05-10T11:17:00.209768+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:42:57.673595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459304+0000","last_change":"2024-05-10T11:14:35.330679+0000","last_active":"2024-05-10T11:15:01.459304+0000","last_peered":"2024-05-10T11:15:01.459304+0000","last_clean":"2024-05-10T11:15:01.459304+0000","last_became_active":"2024-05-10T11:14:35.329819+0000","last_became_peered":"2024-05-10T11:14:35.329819+0000","last_unstale":"2024-05-10T11:15:01.459304+0000","last_undegraded":"2024-05-10T11:15:01.459304+0000","last_fullsized":"2024-05-10T11:15:01.459304+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:31:26.982784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:29.798740+0000","last_change":"2024-05-10T11:14:22.260402+0000","last_active":"2024-05-10T11:15:29.798740+0000","last_peered":"2024-05-10T11:15:29.798740+0000","last_clean":"2024-05-10T11:15:29.798740+0000","last_became_active":"2024-05-10T11:14:22.259619+0000","last_became_peered":"2024-05-10T11:14:22.259619+0000","last_unstale":"2024-05-10T11:15:29.798740+0000","last_undegraded":"2024-05-10T11:15:29.798740+0000","last_fullsized":"2024-05-10T11:15:29.798740+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:56:34.491626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.204002+0000","last_change":"2024-05-10T11:14:25.276117+0000","last_active":"2024-05-10T11:17:00.204002+0000","last_peered":"2024-05-10T11:17:00.204002+0000","last_clean":"2024-05-10T11:17:00.204002+0000","last_became_active":"2024-05-10T11:14:25.275758+0000","last_became_peered":"2024-05-10T11:14:25.275758+0000","last_unstale":"2024-05-10T11:17:00.204002+0000","last_undegraded":"2024-05-10T11:17:00.204002+0000","last_fullsized":"2024-05-10T11:17:00.204002+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:12:16.582244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459213+0000","last_change":"2024-05-10T11:14:35.324633+0000","last_active":"2024-05-10T11:15:01.459213+0000","last_peered":"2024-05-10T11:15:01.459213+0000","last_clean":"2024-05-10T11:15:01.459213+0000","last_became_active":"2024-05-10T11:14:35.324152+0000","last_became_peered":"2024-05-10T11:14:35.324152+0000","last_unstale":"2024-05-10T11:15:01.459213+0000","last_undegraded":"2024-05-10T11:15:01.459213+0000","last_fullsized":"2024-05-10T11:15:01.459213+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:56:50.008782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.430353+0000","last_change":"2024-05-10T11:14:22.259086+0000","last_active":"2024-05-10T11:15:01.430353+0000","last_peered":"2024-05-10T11:15:01.430353+0000","last_clean":"2024-05-10T11:15:01.430353+0000","last_became_active":"2024-05-10T11:14:22.258628+0000","last_became_peered":"2024-05-10T11:14:22.258628+0000","last_unstale":"2024-05-10T11:15:01.430353+0000","last_undegraded":"2024-05-10T11:15:01.430353+0000","last_fullsized":"2024-05-10T11:15:01.430353+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:39:13.472622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.213935+0000","last_change":"2024-05-10T11:14:25.277556+0000","last_active":"2024-05-10T11:17:00.213935+0000","last_peered":"2024-05-10T11:17:00.213935+0000","last_clean":"2024-05-10T11:17:00.213935+0000","last_became_active":"2024-05-10T11:14:25.277201+0000","last_became_peered":"2024-05-10T11:14:25.277201+0000","last_unstale":"2024-05-10T11:17:00.213935+0000","last_undegraded":"2024-05-10T11:17:00.213935+0000","last_fullsized":"2024-05-10T11:17:00.213935+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:04:17.028305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.426356+0000","last_change":"2024-05-10T11:14:35.323338+0000","last_active":"2024-05-10T11:15:01.426356+0000","last_peered":"2024-05-10T11:15:01.426356+0000","last_clean":"2024-05-10T11:15:01.426356+0000","last_became_active":"2024-05-10T11:14:35.321948+0000","last_became_peered":"2024-05-10T11:14:35.321948+0000","last_unstale":"2024-05-10T11:15:01.426356+0000","last_undegraded":"2024-05-10T11:15:01.426356+0000","last_fullsized":"2024-05-10T11:15:01.426356+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:41:30.569125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'7","reported_seq":148,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:07.953382+0000","last_change":"2024-05-10T11:14:22.259187+0000","last_active":"2024-05-10T11:17:07.953382+0000","last_peered":"2024-05-10T11:17:07.953382+0000","last_clean":"2024-05-10T11:17:07.953382+0000","last_became_active":"2024-05-10T11:14:22.258812+0000","last_became_peered":"2024-05-10T11:14:22.258812+0000","last_unstale":"2024-05-10T11:17:07.953382+0000","last_undegraded":"2024-05-10T11:17:07.953382+0000","last_fullsized":"2024-05-10T11:17:07.953382+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:15:38.769705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.188655+0000","last_change":"2024-05-10T11:14:25.262469+0000","last_active":"2024-05-10T11:17:00.188655+0000","last_peered":"2024-05-10T11:17:00.188655+0000","last_clean":"2024-05-10T11:17:00.188655+0000","last_became_active":"2024-05-10T11:14:25.262104+0000","last_became_peered":"2024-05-10T11:14:25.262104+0000","last_unstale":"2024-05-10T11:17:00.188655+0000","last_undegraded":"2024-05-10T11:17:00.188655+0000","last_fullsized":"2024-05-10T11:17:00.188655+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:48:24.329136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.426927+0000","last_change":"2024-05-10T11:14:35.327737+0000","last_active":"2024-05-10T11:15:01.426927+0000","last_peered":"2024-05-10T11:15:01.426927+0000","last_clean":"2024-05-10T11:15:01.426927+0000","last_became_active":"2024-05-10T11:14:35.327209+0000","last_became_peered":"2024-05-10T11:14:35.327209+0000","last_unstale":"2024-05-10T11:15:01.426927+0000","last_undegraded":"2024-05-10T11:15:01.426927+0000","last_fullsized":"2024-05-10T11:15:01.426927+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:32:11.784752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:29.797324+0000","last_change":"2024-05-10T11:14:22.254853+0000","last_active":"2024-05-10T11:15:29.797324+0000","last_peered":"2024-05-10T11:15:29.797324+0000","last_clean":"2024-05-10T11:15:29.797324+0000","last_became_active":"2024-05-10T11:14:22.253352+0000","last_became_peered":"2024-05-10T11:14:22.253352+0000","last_unstale":"2024-05-10T11:15:29.797324+0000","last_undegraded":"2024-05-10T11:15:29.797324+0000","last_fullsized":"2024-05-10T11:15:29.797324+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:14:15.871042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.213028+0000","last_change":"2024-05-10T11:14:25.266798+0000","last_active":"2024-05-10T11:17:00.213028+0000","last_peered":"2024-05-10T11:17:00.213028+0000","last_clean":"2024-05-10T11:17:00.213028+0000","last_became_active":"2024-05-10T11:14:25.266384+0000","last_became_peered":"2024-05-10T11:14:25.266384+0000","last_unstale":"2024-05-10T11:17:00.213028+0000","last_undegraded":"2024-05-10T11:17:00.213028+0000","last_fullsized":"2024-05-10T11:17:00.213028+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:18:07.338987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.463232+0000","last_change":"2024-05-10T11:15:01.462376+0000","last_active":"2024-05-10T11:15:01.463232+0000","last_peered":"2024-05-10T11:15:01.463232+0000","last_clean":"2024-05-10T11:15:01.463232+0000","last_became_active":"2024-05-10T11:15:01.461876+0000","last_became_peered":"2024-05-10T11:15:01.461876+0000","last_unstale":"2024-05-10T11:15:01.463232+0000","last_undegraded":"2024-05-10T11:15:01.463232+0000","last_fullsized":"2024-05-10T11:15:01.463232+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:35:52.076414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:29.797310+0000","last_change":"2024-05-10T11:14:22.253976+0000","last_active":"2024-05-10T11:15:29.797310+0000","last_peered":"2024-05-10T11:15:29.797310+0000","last_clean":"2024-05-10T11:15:29.797310+0000","last_became_active":"2024-05-10T11:14:22.252230+0000","last_became_peered":"2024-05-10T11:14:22.252230+0000","last_unstale":"2024-05-10T11:15:29.797310+0000","last_undegraded":"2024-05-10T11:15:29.797310+0000","last_fullsized":"2024-05-10T11:15:29.797310+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:14:03.208811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.206683+0000","last_change":"2024-05-10T11:14:25.279627+0000","last_active":"2024-05-10T11:17:00.206683+0000","last_peered":"2024-05-10T11:17:00.206683+0000","last_clean":"2024-05-10T11:17:00.206683+0000","last_became_active":"2024-05-10T11:14:25.279330+0000","last_became_peered":"2024-05-10T11:14:25.279330+0000","last_unstale":"2024-05-10T11:17:00.206683+0000","last_undegraded":"2024-05-10T11:17:00.206683+0000","last_fullsized":"2024-05-10T11:17:00.206683+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:50:19.608053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.426065+0000","last_change":"2024-05-10T11:14:35.321723+0000","last_active":"2024-05-10T11:15:01.426065+0000","last_peered":"2024-05-10T11:15:01.426065+0000","last_clean":"2024-05-10T11:15:01.426065+0000","last_became_active":"2024-05-10T11:14:35.321384+0000","last_became_peered":"2024-05-10T11:14:35.321384+0000","last_unstale":"2024-05-10T11:15:01.426065+0000","last_undegraded":"2024-05-10T11:15:01.426065+0000","last_fullsized":"2024-05-10T11:15:01.426065+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:04:07.770889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:28.790869+0000","last_change":"2024-05-10T11:14:22.255511+0000","last_active":"2024-05-10T11:15:28.790869+0000","last_peered":"2024-05-10T11:15:28.790869+0000","last_clean":"2024-05-10T11:15:28.790869+0000","last_became_active":"2024-05-10T11:14:22.254722+0000","last_became_peered":"2024-05-10T11:14:22.254722+0000","last_unstale":"2024-05-10T11:15:28.790869+0000","last_undegraded":"2024-05-10T11:15:28.790869+0000","last_fullsized":"2024-05-10T11:15:28.790869+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:38:27.096011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.209442+0000","last_change":"2024-05-10T11:14:25.274009+0000","last_active":"2024-05-10T11:17:00.209442+0000","last_peered":"2024-05-10T11:17:00.209442+0000","last_clean":"2024-05-10T11:17:00.209442+0000","last_became_active":"2024-05-10T11:14:25.273672+0000","last_became_peered":"2024-05-10T11:14:25.273672+0000","last_unstale":"2024-05-10T11:17:00.209442+0000","last_undegraded":"2024-05-10T11:17:00.209442+0000","last_fullsized":"2024-05-10T11:17:00.209442+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:42:57.210131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459026+0000","last_change":"2024-05-10T11:14:35.321520+0000","last_active":"2024-05-10T11:15:01.459026+0000","last_peered":"2024-05-10T11:15:01.459026+0000","last_clean":"2024-05-10T11:15:01.459026+0000","last_became_active":"2024-05-10T11:14:35.320575+0000","last_became_peered":"2024-05-10T11:14:35.320575+0000","last_unstale":"2024-05-10T11:15:01.459026+0000","last_undegraded":"2024-05-10T11:15:01.459026+0000","last_fullsized":"2024-05-10T11:15:01.459026+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:54:03.183426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":143,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:09.956129+0000","last_change":"2024-05-10T11:14:22.251463+0000","last_active":"2024-05-10T11:17:09.956129+0000","last_peered":"2024-05-10T11:17:09.956129+0000","last_clean":"2024-05-10T11:17:09.956129+0000","last_became_active":"2024-05-10T11:14:22.251132+0000","last_became_peered":"2024-05-10T11:14:22.251132+0000","last_unstale":"2024-05-10T11:17:09.956129+0000","last_undegraded":"2024-05-10T11:17:09.956129+0000","last_fullsized":"2024-05-10T11:17:09.956129+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:29:18.125380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":99,"num_read_kb":99,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.209206+0000","last_change":"2024-05-10T11:14:25.276333+0000","last_active":"2024-05-10T11:17:00.209206+0000","last_peered":"2024-05-10T11:17:00.209206+0000","last_clean":"2024-05-10T11:17:00.209206+0000","last_became_active":"2024-05-10T11:14:25.275818+0000","last_became_peered":"2024-05-10T11:14:25.275818+0000","last_unstale":"2024-05-10T11:17:00.209206+0000","last_undegraded":"2024-05-10T11:17:00.209206+0000","last_fullsized":"2024-05-10T11:17:00.209206+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:27:19.542645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.422145+0000","last_change":"2024-05-10T11:14:35.328589+0000","last_active":"2024-05-10T11:15:01.422145+0000","last_peered":"2024-05-10T11:15:01.422145+0000","last_clean":"2024-05-10T11:15:01.422145+0000","last_became_active":"2024-05-10T11:14:35.328052+0000","last_became_peered":"2024-05-10T11:14:35.328052+0000","last_unstale":"2024-05-10T11:15:01.422145+0000","last_undegraded":"2024-05-10T11:15:01.422145+0000","last_fullsized":"2024-05-10T11:15:01.422145+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:09:42.763932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:28.790098+0000","last_change":"2024-05-10T11:14:22.257872+0000","last_active":"2024-05-10T11:15:28.790098+0000","last_peered":"2024-05-10T11:15:28.790098+0000","last_clean":"2024-05-10T11:15:28.790098+0000","last_became_active":"2024-05-10T11:14:22.257530+0000","last_became_peered":"2024-05-10T11:14:22.257530+0000","last_unstale":"2024-05-10T11:15:28.790098+0000","last_undegraded":"2024-05-10T11:15:28.790098+0000","last_fullsized":"2024-05-10T11:15:28.790098+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:26:30.367209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.214274+0000","last_change":"2024-05-10T11:14:25.269436+0000","last_active":"2024-05-10T11:17:00.214274+0000","last_peered":"2024-05-10T11:17:00.214274+0000","last_clean":"2024-05-10T11:17:00.214274+0000","last_became_active":"2024-05-10T11:14:25.269102+0000","last_became_peered":"2024-05-10T11:14:25.269102+0000","last_unstale":"2024-05-10T11:17:00.214274+0000","last_undegraded":"2024-05-10T11:17:00.214274+0000","last_fullsized":"2024-05-10T11:17:00.214274+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:51:50.172464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459510+0000","last_change":"2024-05-10T11:14:35.330272+0000","last_active":"2024-05-10T11:15:01.459510+0000","last_peered":"2024-05-10T11:15:01.459510+0000","last_clean":"2024-05-10T11:15:01.459510+0000","last_became_active":"2024-05-10T11:14:35.329438+0000","last_became_peered":"2024-05-10T11:14:35.329438+0000","last_unstale":"2024-05-10T11:15:01.459510+0000","last_undegraded":"2024-05-10T11:15:01.459510+0000","last_fullsized":"2024-05-10T11:15:01.459510+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:07:00.559447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:32.813046+0000","last_change":"2024-05-10T11:14:22.299302+0000","last_active":"2024-05-10T11:15:32.813046+0000","last_peered":"2024-05-10T11:15:32.813046+0000","last_clean":"2024-05-10T11:15:32.813046+0000","last_became_active":"2024-05-10T11:14:22.298792+0000","last_became_peered":"2024-05-10T11:14:22.298792+0000","last_unstale":"2024-05-10T11:15:32.813046+0000","last_undegraded":"2024-05-10T11:15:32.813046+0000","last_fullsized":"2024-05-10T11:15:32.813046+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:12:35.280135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.200955+0000","last_change":"2024-05-10T11:14:25.275380+0000","last_active":"2024-05-10T11:17:00.200955+0000","last_peered":"2024-05-10T11:17:00.200955+0000","last_clean":"2024-05-10T11:17:00.200955+0000","last_became_active":"2024-05-10T11:14:25.275042+0000","last_became_peered":"2024-05-10T11:14:25.275042+0000","last_unstale":"2024-05-10T11:17:00.200955+0000","last_undegraded":"2024-05-10T11:17:00.200955+0000","last_fullsized":"2024-05-10T11:17:00.200955+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:22:49.598602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.422374+0000","last_change":"2024-05-10T11:14:35.327505+0000","last_active":"2024-05-10T11:15:01.422374+0000","last_peered":"2024-05-10T11:15:01.422374+0000","last_clean":"2024-05-10T11:15:01.422374+0000","last_became_active":"2024-05-10T11:14:35.326885+0000","last_became_peered":"2024-05-10T11:14:35.326885+0000","last_unstale":"2024-05-10T11:15:01.422374+0000","last_undegraded":"2024-05-10T11:15:01.422374+0000","last_fullsized":"2024-05-10T11:15:01.422374+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:23:09.010545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'37","reported_seq":285,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:10.962979+0000","last_change":"2024-05-10T11:14:22.257571+0000","last_active":"2024-05-10T11:17:10.962979+0000","last_peered":"2024-05-10T11:17:10.962979+0000","last_clean":"2024-05-10T11:17:10.962979+0000","last_became_active":"2024-05-10T11:14:22.257156+0000","last_became_peered":"2024-05-10T11:14:22.257156+0000","last_unstale":"2024-05-10T11:17:10.962979+0000","last_undegraded":"2024-05-10T11:17:10.962979+0000","last_fullsized":"2024-05-10T11:17:10.962979+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:28:06.397262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64385,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":208,"num_read_kb":167,"num_write":38,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.193870+0000","last_change":"2024-05-10T11:14:25.281160+0000","last_active":"2024-05-10T11:17:00.193870+0000","last_peered":"2024-05-10T11:17:00.193870+0000","last_clean":"2024-05-10T11:17:00.193870+0000","last_became_active":"2024-05-10T11:14:25.280763+0000","last_became_peered":"2024-05-10T11:14:25.280763+0000","last_unstale":"2024-05-10T11:17:00.193870+0000","last_undegraded":"2024-05-10T11:17:00.193870+0000","last_fullsized":"2024-05-10T11:17:00.193870+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:19:53.350792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.460284+0000","last_change":"2024-05-10T11:14:35.320918+0000","last_active":"2024-05-10T11:15:01.460284+0000","last_peered":"2024-05-10T11:15:01.460284+0000","last_clean":"2024-05-10T11:15:01.460284+0000","last_became_active":"2024-05-10T11:14:35.320412+0000","last_became_peered":"2024-05-10T11:14:35.320412+0000","last_unstale":"2024-05-10T11:15:01.460284+0000","last_undegraded":"2024-05-10T11:15:01.460284+0000","last_fullsized":"2024-05-10T11:15:01.460284+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:55:52.585430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:32.810406+0000","last_change":"2024-05-10T11:14:22.257771+0000","last_active":"2024-05-10T11:15:32.810406+0000","last_peered":"2024-05-10T11:15:32.810406+0000","last_clean":"2024-05-10T11:15:32.810406+0000","last_became_active":"2024-05-10T11:14:22.257430+0000","last_became_peered":"2024-05-10T11:14:22.257430+0000","last_unstale":"2024-05-10T11:15:32.810406+0000","last_undegraded":"2024-05-10T11:15:32.810406+0000","last_fullsized":"2024-05-10T11:15:32.810406+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:23:50.131136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.203875+0000","last_change":"2024-05-10T11:14:25.272276+0000","last_active":"2024-05-10T11:17:00.203875+0000","last_peered":"2024-05-10T11:17:00.203875+0000","last_clean":"2024-05-10T11:17:00.203875+0000","last_became_active":"2024-05-10T11:14:25.271932+0000","last_became_peered":"2024-05-10T11:14:25.271932+0000","last_unstale":"2024-05-10T11:17:00.203875+0000","last_undegraded":"2024-05-10T11:17:00.203875+0000","last_fullsized":"2024-05-10T11:17:00.203875+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:49:44.479056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.463273+0000","last_change":"2024-05-10T11:15:01.462435+0000","last_active":"2024-05-10T11:15:01.463273+0000","last_peered":"2024-05-10T11:15:01.463273+0000","last_clean":"2024-05-10T11:15:01.463273+0000","last_became_active":"2024-05-10T11:15:01.461890+0000","last_became_peered":"2024-05-10T11:15:01.461890+0000","last_unstale":"2024-05-10T11:15:01.463273+0000","last_undegraded":"2024-05-10T11:15:01.463273+0000","last_fullsized":"2024-05-10T11:15:01.463273+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:01:13.350655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:28.789871+0000","last_change":"2024-05-10T11:14:22.256713+0000","last_active":"2024-05-10T11:15:28.789871+0000","last_peered":"2024-05-10T11:15:28.789871+0000","last_clean":"2024-05-10T11:15:28.789871+0000","last_became_active":"2024-05-10T11:14:22.255847+0000","last_became_peered":"2024-05-10T11:14:22.255847+0000","last_unstale":"2024-05-10T11:15:28.789871+0000","last_undegraded":"2024-05-10T11:15:28.789871+0000","last_fullsized":"2024-05-10T11:15:28.789871+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:00:54.605506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.214050+0000","last_change":"2024-05-10T11:14:25.282118+0000","last_active":"2024-05-10T11:17:00.214050+0000","last_peered":"2024-05-10T11:17:00.214050+0000","last_clean":"2024-05-10T11:17:00.214050+0000","last_became_active":"2024-05-10T11:14:25.280958+0000","last_became_peered":"2024-05-10T11:14:25.280958+0000","last_unstale":"2024-05-10T11:17:00.214050+0000","last_undegraded":"2024-05-10T11:17:00.214050+0000","last_fullsized":"2024-05-10T11:17:00.214050+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:02:10.815733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.422298+0000","last_change":"2024-05-10T11:14:35.329876+0000","last_active":"2024-05-10T11:15:01.422298+0000","last_peered":"2024-05-10T11:15:01.422298+0000","last_clean":"2024-05-10T11:15:01.422298+0000","last_became_active":"2024-05-10T11:14:35.328923+0000","last_became_peered":"2024-05-10T11:14:35.328923+0000","last_unstale":"2024-05-10T11:15:01.422298+0000","last_undegraded":"2024-05-10T11:15:01.422298+0000","last_fullsized":"2024-05-10T11:15:01.422298+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:04:51.356552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'5","reported_seq":146,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:08.963667+0000","last_change":"2024-05-10T11:14:22.254515+0000","last_active":"2024-05-10T11:17:08.963667+0000","last_peered":"2024-05-10T11:17:08.963667+0000","last_clean":"2024-05-10T11:17:08.963667+0000","last_became_active":"2024-05-10T11:14:22.253648+0000","last_became_peered":"2024-05-10T11:14:22.253648+0000","last_unstale":"2024-05-10T11:17:08.963667+0000","last_undegraded":"2024-05-10T11:17:08.963667+0000","last_fullsized":"2024-05-10T11:17:08.963667+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:10:22.144446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":103,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.210635+0000","last_change":"2024-05-10T11:14:25.278841+0000","last_active":"2024-05-10T11:17:00.210635+0000","last_peered":"2024-05-10T11:17:00.210635+0000","last_clean":"2024-05-10T11:17:00.210635+0000","last_became_active":"2024-05-10T11:14:25.278527+0000","last_became_peered":"2024-05-10T11:14:25.278527+0000","last_unstale":"2024-05-10T11:17:00.210635+0000","last_undegraded":"2024-05-10T11:17:00.210635+0000","last_fullsized":"2024-05-10T11:17:00.210635+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:01:55.902587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459763+0000","last_change":"2024-05-10T11:14:35.331334+0000","last_active":"2024-05-10T11:15:01.459763+0000","last_peered":"2024-05-10T11:15:01.459763+0000","last_clean":"2024-05-10T11:15:01.459763+0000","last_became_active":"2024-05-10T11:14:35.330940+0000","last_became_peered":"2024-05-10T11:14:35.330940+0000","last_unstale":"2024-05-10T11:15:01.459763+0000","last_undegraded":"2024-05-10T11:15:01.459763+0000","last_fullsized":"2024-05-10T11:15:01.459763+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:11:12.241551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":56,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:32.810680+0000","last_change":"2024-05-10T11:14:22.299917+0000","last_active":"2024-05-10T11:15:32.810680+0000","last_peered":"2024-05-10T11:15:32.810680+0000","last_clean":"2024-05-10T11:15:32.810680+0000","last_became_active":"2024-05-10T11:14:22.299212+0000","last_became_peered":"2024-05-10T11:14:22.299212+0000","last_unstale":"2024-05-10T11:15:32.810680+0000","last_undegraded":"2024-05-10T11:15:32.810680+0000","last_fullsized":"2024-05-10T11:15:32.810680+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:52:48.740339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.213948+0000","last_change":"2024-05-10T11:14:25.274693+0000","last_active":"2024-05-10T11:17:00.213948+0000","last_peered":"2024-05-10T11:17:00.213948+0000","last_clean":"2024-05-10T11:17:00.213948+0000","last_became_active":"2024-05-10T11:14:25.274378+0000","last_became_peered":"2024-05-10T11:14:25.274378+0000","last_unstale":"2024-05-10T11:17:00.213948+0000","last_undegraded":"2024-05-10T11:17:00.213948+0000","last_fullsized":"2024-05-10T11:17:00.213948+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:53:24.082742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.461224+0000","last_change":"2024-05-10T11:15:01.460541+0000","last_active":"2024-05-10T11:15:01.461224+0000","last_peered":"2024-05-10T11:15:01.461224+0000","last_clean":"2024-05-10T11:15:01.461224+0000","last_became_active":"2024-05-10T11:15:01.459954+0000","last_became_peered":"2024-05-10T11:15:01.459954+0000","last_unstale":"2024-05-10T11:15:01.461224+0000","last_undegraded":"2024-05-10T11:15:01.461224+0000","last_fullsized":"2024-05-10T11:15:01.461224+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:25:43.642441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,3,1],"acting":[8,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:32.810495+0000","last_change":"2024-05-10T11:14:22.258797+0000","last_active":"2024-05-10T11:15:32.810495+0000","last_peered":"2024-05-10T11:15:32.810495+0000","last_clean":"2024-05-10T11:15:32.810495+0000","last_became_active":"2024-05-10T11:14:22.258365+0000","last_became_peered":"2024-05-10T11:14:22.258365+0000","last_unstale":"2024-05-10T11:15:32.810495+0000","last_undegraded":"2024-05-10T11:15:32.810495+0000","last_fullsized":"2024-05-10T11:15:32.810495+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:58:53.291644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.198793+0000","last_change":"2024-05-10T11:15:01.460502+0000","last_active":"2024-05-10T11:17:00.198793+0000","last_peered":"2024-05-10T11:17:00.198793+0000","last_clean":"2024-05-10T11:17:00.198793+0000","last_became_active":"2024-05-10T11:15:01.460101+0000","last_became_peered":"2024-05-10T11:15:01.460101+0000","last_unstale":"2024-05-10T11:17:00.198793+0000","last_undegraded":"2024-05-10T11:17:00.198793+0000","last_fullsized":"2024-05-10T11:17:00.198793+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:22:39.735251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.429220+0000","last_change":"2024-05-10T11:14:35.419037+0000","last_active":"2024-05-10T11:15:01.429220+0000","last_peered":"2024-05-10T11:15:01.429220+0000","last_clean":"2024-05-10T11:15:01.429220+0000","last_became_active":"2024-05-10T11:14:35.418541+0000","last_became_peered":"2024-05-10T11:14:35.418541+0000","last_unstale":"2024-05-10T11:15:01.429220+0000","last_undegraded":"2024-05-10T11:15:01.429220+0000","last_fullsized":"2024-05-10T11:15:01.429220+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:05:49.560608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.422544+0000","last_change":"2024-05-10T11:14:22.299538+0000","last_active":"2024-05-10T11:15:01.422544+0000","last_peered":"2024-05-10T11:15:01.422544+0000","last_clean":"2024-05-10T11:15:01.422544+0000","last_became_active":"2024-05-10T11:14:22.299192+0000","last_became_peered":"2024-05-10T11:14:22.299192+0000","last_unstale":"2024-05-10T11:15:01.422544+0000","last_undegraded":"2024-05-10T11:15:01.422544+0000","last_fullsized":"2024-05-10T11:15:01.422544+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:28:43.041513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.193550+0000","last_change":"2024-05-10T11:14:25.272238+0000","last_active":"2024-05-10T11:17:00.193550+0000","last_peered":"2024-05-10T11:17:00.193550+0000","last_clean":"2024-05-10T11:17:00.193550+0000","last_became_active":"2024-05-10T11:14:25.271861+0000","last_became_peered":"2024-05-10T11:14:25.271861+0000","last_unstale":"2024-05-10T11:17:00.193550+0000","last_undegraded":"2024-05-10T11:17:00.193550+0000","last_fullsized":"2024-05-10T11:17:00.193550+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:00:08.821316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459614+0000","last_change":"2024-05-10T11:14:35.322239+0000","last_active":"2024-05-10T11:15:01.459614+0000","last_peered":"2024-05-10T11:15:01.459614+0000","last_clean":"2024-05-10T11:15:01.459614+0000","last_became_active":"2024-05-10T11:14:35.321864+0000","last_became_peered":"2024-05-10T11:14:35.321864+0000","last_unstale":"2024-05-10T11:15:01.459614+0000","last_undegraded":"2024-05-10T11:15:01.459614+0000","last_fullsized":"2024-05-10T11:15:01.459614+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:42:06.905879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:29.796685+0000","last_change":"2024-05-10T11:14:22.255449+0000","last_active":"2024-05-10T11:15:29.796685+0000","last_peered":"2024-05-10T11:15:29.796685+0000","last_clean":"2024-05-10T11:15:29.796685+0000","last_became_active":"2024-05-10T11:14:22.254966+0000","last_became_peered":"2024-05-10T11:14:22.254966+0000","last_unstale":"2024-05-10T11:15:29.796685+0000","last_undegraded":"2024-05-10T11:15:29.796685+0000","last_fullsized":"2024-05-10T11:15:29.796685+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:55:40.746115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.209423+0000","last_change":"2024-05-10T11:14:25.274050+0000","last_active":"2024-05-10T11:17:00.209423+0000","last_peered":"2024-05-10T11:17:00.209423+0000","last_clean":"2024-05-10T11:17:00.209423+0000","last_became_active":"2024-05-10T11:14:25.273720+0000","last_became_peered":"2024-05-10T11:14:25.273720+0000","last_unstale":"2024-05-10T11:17:00.209423+0000","last_undegraded":"2024-05-10T11:17:00.209423+0000","last_fullsized":"2024-05-10T11:17:00.209423+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:22:05.977673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459638+0000","last_change":"2024-05-10T11:14:35.418989+0000","last_active":"2024-05-10T11:15:01.459638+0000","last_peered":"2024-05-10T11:15:01.459638+0000","last_clean":"2024-05-10T11:15:01.459638+0000","last_became_active":"2024-05-10T11:14:35.418546+0000","last_became_peered":"2024-05-10T11:14:35.418546+0000","last_unstale":"2024-05-10T11:15:01.459638+0000","last_undegraded":"2024-05-10T11:15:01.459638+0000","last_fullsized":"2024-05-10T11:15:01.459638+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:23:37.875409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.210371+0000","last_change":"2024-05-10T11:14:25.272694+0000","last_active":"2024-05-10T11:17:00.210371+0000","last_peered":"2024-05-10T11:17:00.210371+0000","last_clean":"2024-05-10T11:17:00.210371+0000","last_became_active":"2024-05-10T11:14:25.272303+0000","last_became_peered":"2024-05-10T11:14:25.272303+0000","last_unstale":"2024-05-10T11:17:00.210371+0000","last_undegraded":"2024-05-10T11:17:00.210371+0000","last_fullsized":"2024-05-10T11:17:00.210371+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:51:39.039122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":144,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:10.965436+0000","last_change":"2024-05-10T11:14:22.299359+0000","last_active":"2024-05-10T11:17:10.965436+0000","last_peered":"2024-05-10T11:17:10.965436+0000","last_clean":"2024-05-10T11:17:10.965436+0000","last_became_active":"2024-05-10T11:14:22.299023+0000","last_became_peered":"2024-05-10T11:14:22.299023+0000","last_unstale":"2024-05-10T11:17:10.965436+0000","last_undegraded":"2024-05-10T11:17:10.965436+0000","last_fullsized":"2024-05-10T11:17:10.965436+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:53:06.301774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":100,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:41.913182+0000","last_change":"2024-05-10T11:14:35.322640+0000","last_active":"2024-05-10T11:15:41.913182+0000","last_peered":"2024-05-10T11:15:41.913182+0000","last_clean":"2024-05-10T11:15:41.913182+0000","last_became_active":"2024-05-10T11:14:35.322226+0000","last_became_peered":"2024-05-10T11:14:35.322226+0000","last_unstale":"2024-05-10T11:15:41.913182+0000","last_undegraded":"2024-05-10T11:15:41.913182+0000","last_fullsized":"2024-05-10T11:15:41.913182+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:34:11.574438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.206730+0000","last_change":"2024-05-10T11:14:25.276645+0000","last_active":"2024-05-10T11:17:00.206730+0000","last_peered":"2024-05-10T11:17:00.206730+0000","last_clean":"2024-05-10T11:17:00.206730+0000","last_became_active":"2024-05-10T11:14:25.276295+0000","last_became_peered":"2024-05-10T11:14:25.276295+0000","last_unstale":"2024-05-10T11:17:00.206730+0000","last_undegraded":"2024-05-10T11:17:00.206730+0000","last_fullsized":"2024-05-10T11:17:00.206730+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:52:06.715073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:32.810748+0000","last_change":"2024-05-10T11:14:22.252908+0000","last_active":"2024-05-10T11:15:32.810748+0000","last_peered":"2024-05-10T11:15:32.810748+0000","last_clean":"2024-05-10T11:15:32.810748+0000","last_became_active":"2024-05-10T11:14:22.252532+0000","last_became_peered":"2024-05-10T11:14:22.252532+0000","last_unstale":"2024-05-10T11:15:32.810748+0000","last_undegraded":"2024-05-10T11:15:32.810748+0000","last_fullsized":"2024-05-10T11:15:32.810748+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:41:59.651442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":113,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.426311+0000","last_change":"2024-05-10T11:12:53.090796+0000","last_active":"2024-05-10T11:15:01.426311+0000","last_peered":"2024-05-10T11:15:01.426311+0000","last_clean":"2024-05-10T11:15:01.426311+0000","last_became_active":"2024-05-10T11:12:53.086566+0000","last_became_peered":"2024-05-10T11:12:53.086566+0000","last_unstale":"2024-05-10T11:15:01.426311+0000","last_undegraded":"2024-05-10T11:15:01.426311+0000","last_fullsized":"2024-05-10T11:15:01.426311+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:09:27.097043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:09:27.097043+0000","last_clean_scrub_stamp":"2024-05-10T11:09:27.097043+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:42:17.759719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459957+0000","last_change":"2024-05-10T11:14:35.419970+0000","last_active":"2024-05-10T11:15:01.459957+0000","last_peered":"2024-05-10T11:15:01.459957+0000","last_clean":"2024-05-10T11:15:01.459957+0000","last_became_active":"2024-05-10T11:14:35.419516+0000","last_became_peered":"2024-05-10T11:14:35.419516+0000","last_unstale":"2024-05-10T11:15:01.459957+0000","last_undegraded":"2024-05-10T11:15:01.459957+0000","last_fullsized":"2024-05-10T11:15:01.459957+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:01:25.148155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.186817+0000","last_change":"2024-05-10T11:14:25.271478+0000","last_active":"2024-05-10T11:17:00.186817+0000","last_peered":"2024-05-10T11:17:00.186817+0000","last_clean":"2024-05-10T11:17:00.186817+0000","last_became_active":"2024-05-10T11:14:25.271143+0000","last_became_peered":"2024-05-10T11:14:25.271143+0000","last_unstale":"2024-05-10T11:17:00.186817+0000","last_undegraded":"2024-05-10T11:17:00.186817+0000","last_fullsized":"2024-05-10T11:17:00.186817+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:50:55.771659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:29.796678+0000","last_change":"2024-05-10T11:14:22.258704+0000","last_active":"2024-05-10T11:15:29.796678+0000","last_peered":"2024-05-10T11:15:29.796678+0000","last_clean":"2024-05-10T11:15:29.796678+0000","last_became_active":"2024-05-10T11:14:22.258304+0000","last_became_peered":"2024-05-10T11:14:22.258304+0000","last_unstale":"2024-05-10T11:15:29.796678+0000","last_undegraded":"2024-05-10T11:15:29.796678+0000","last_fullsized":"2024-05-10T11:15:29.796678+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:02:41.451394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.425481+0000","last_change":"2024-05-10T11:14:35.325151+0000","last_active":"2024-05-10T11:15:01.425481+0000","last_peered":"2024-05-10T11:15:01.425481+0000","last_clean":"2024-05-10T11:15:01.425481+0000","last_became_active":"2024-05-10T11:14:35.324680+0000","last_became_peered":"2024-05-10T11:14:35.324680+0000","last_unstale":"2024-05-10T11:15:01.425481+0000","last_undegraded":"2024-05-10T11:15:01.425481+0000","last_fullsized":"2024-05-10T11:15:01.425481+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:12:17.578364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:32.810795+0000","last_change":"2024-05-10T11:15:01.428210+0000","last_active":"2024-05-10T11:15:32.810795+0000","last_peered":"2024-05-10T11:15:32.810795+0000","last_clean":"2024-05-10T11:15:32.810795+0000","last_became_active":"2024-05-10T11:15:01.427671+0000","last_became_peered":"2024-05-10T11:15:01.427671+0000","last_unstale":"2024-05-10T11:15:32.810795+0000","last_undegraded":"2024-05-10T11:15:32.810795+0000","last_fullsized":"2024-05-10T11:15:32.810795+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:44:51.312743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.194143+0000","last_change":"2024-05-10T11:14:25.271350+0000","last_active":"2024-05-10T11:17:00.194143+0000","last_peered":"2024-05-10T11:17:00.194143+0000","last_clean":"2024-05-10T11:17:00.194143+0000","last_became_active":"2024-05-10T11:14:25.271047+0000","last_became_peered":"2024-05-10T11:14:25.271047+0000","last_unstale":"2024-05-10T11:17:00.194143+0000","last_undegraded":"2024-05-10T11:17:00.194143+0000","last_fullsized":"2024-05-10T11:17:00.194143+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:44:21.074007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.458379+0000","last_change":"2024-05-10T11:14:35.418910+0000","last_active":"2024-05-10T11:15:01.458379+0000","last_peered":"2024-05-10T11:15:01.458379+0000","last_clean":"2024-05-10T11:15:01.458379+0000","last_became_active":"2024-05-10T11:14:35.418460+0000","last_became_peered":"2024-05-10T11:14:35.418460+0000","last_unstale":"2024-05-10T11:15:01.458379+0000","last_undegraded":"2024-05-10T11:15:01.458379+0000","last_fullsized":"2024-05-10T11:15:01.458379+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:36:04.057776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'12","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:16:35.480421+0000","last_change":"2024-05-10T11:14:22.253673+0000","last_active":"2024-05-10T11:16:35.480421+0000","last_peered":"2024-05-10T11:16:35.480421+0000","last_clean":"2024-05-10T11:16:35.480421+0000","last_became_active":"2024-05-10T11:14:22.253304+0000","last_became_peered":"2024-05-10T11:14:22.253304+0000","last_unstale":"2024-05-10T11:16:35.480421+0000","last_undegraded":"2024-05-10T11:16:35.480421+0000","last_fullsized":"2024-05-10T11:16:35.480421+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:11:37.108621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.211961+0000","last_change":"2024-05-10T11:14:25.280590+0000","last_active":"2024-05-10T11:17:00.211961+0000","last_peered":"2024-05-10T11:17:00.211961+0000","last_clean":"2024-05-10T11:17:00.211961+0000","last_became_active":"2024-05-10T11:14:25.279990+0000","last_became_peered":"2024-05-10T11:14:25.279990+0000","last_unstale":"2024-05-10T11:17:00.211961+0000","last_undegraded":"2024-05-10T11:17:00.211961+0000","last_fullsized":"2024-05-10T11:17:00.211961+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:46:55.394767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.424307+0000","last_change":"2024-05-10T11:14:35.320931+0000","last_active":"2024-05-10T11:15:01.424307+0000","last_peered":"2024-05-10T11:15:01.424307+0000","last_clean":"2024-05-10T11:15:01.424307+0000","last_became_active":"2024-05-10T11:14:35.320523+0000","last_became_peered":"2024-05-10T11:14:35.320523+0000","last_unstale":"2024-05-10T11:15:01.424307+0000","last_undegraded":"2024-05-10T11:15:01.424307+0000","last_fullsized":"2024-05-10T11:15:01.424307+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:50:37.533397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:28.788626+0000","last_change":"2024-05-10T11:14:22.255115+0000","last_active":"2024-05-10T11:15:28.788626+0000","last_peered":"2024-05-10T11:15:28.788626+0000","last_clean":"2024-05-10T11:15:28.788626+0000","last_became_active":"2024-05-10T11:14:22.254732+0000","last_became_peered":"2024-05-10T11:14:22.254732+0000","last_unstale":"2024-05-10T11:15:28.788626+0000","last_undegraded":"2024-05-10T11:15:28.788626+0000","last_fullsized":"2024-05-10T11:15:28.788626+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:49:54.807312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.206879+0000","last_change":"2024-05-10T11:14:25.270766+0000","last_active":"2024-05-10T11:17:00.206879+0000","last_peered":"2024-05-10T11:17:00.206879+0000","last_clean":"2024-05-10T11:17:00.206879+0000","last_became_active":"2024-05-10T11:14:25.268958+0000","last_became_peered":"2024-05-10T11:14:25.268958+0000","last_unstale":"2024-05-10T11:17:00.206879+0000","last_undegraded":"2024-05-10T11:17:00.206879+0000","last_fullsized":"2024-05-10T11:17:00.206879+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:21:28.421478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459244+0000","last_change":"2024-05-10T11:14:35.420030+0000","last_active":"2024-05-10T11:15:01.459244+0000","last_peered":"2024-05-10T11:15:01.459244+0000","last_clean":"2024-05-10T11:15:01.459244+0000","last_became_active":"2024-05-10T11:14:35.419475+0000","last_became_peered":"2024-05-10T11:14:35.419475+0000","last_unstale":"2024-05-10T11:15:01.459244+0000","last_undegraded":"2024-05-10T11:15:01.459244+0000","last_fullsized":"2024-05-10T11:15:01.459244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:13:51.207146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:28.789346+0000","last_change":"2024-05-10T11:15:01.429683+0000","last_active":"2024-05-10T11:15:28.789346+0000","last_peered":"2024-05-10T11:15:28.789346+0000","last_clean":"2024-05-10T11:15:28.789346+0000","last_became_active":"2024-05-10T11:15:01.429162+0000","last_became_peered":"2024-05-10T11:15:01.429162+0000","last_unstale":"2024-05-10T11:15:28.789346+0000","last_undegraded":"2024-05-10T11:15:28.789346+0000","last_fullsized":"2024-05-10T11:15:28.789346+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:57:35.166686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.167290+0000","last_change":"2024-05-10T11:14:25.282381+0000","last_active":"2024-05-10T11:17:00.167290+0000","last_peered":"2024-05-10T11:17:00.167290+0000","last_clean":"2024-05-10T11:17:00.167290+0000","last_became_active":"2024-05-10T11:14:25.281990+0000","last_became_peered":"2024-05-10T11:14:25.281990+0000","last_unstale":"2024-05-10T11:17:00.167290+0000","last_undegraded":"2024-05-10T11:17:00.167290+0000","last_fullsized":"2024-05-10T11:17:00.167290+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:39:03.726547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.422393+0000","last_change":"2024-05-10T11:14:35.328242+0000","last_active":"2024-05-10T11:15:01.422393+0000","last_peered":"2024-05-10T11:15:01.422393+0000","last_clean":"2024-05-10T11:15:01.422393+0000","last_became_active":"2024-05-10T11:14:35.327877+0000","last_became_peered":"2024-05-10T11:14:35.327877+0000","last_unstale":"2024-05-10T11:15:01.422393+0000","last_undegraded":"2024-05-10T11:15:01.422393+0000","last_fullsized":"2024-05-10T11:15:01.422393+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:01:57.826334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:28.789342+0000","last_change":"2024-05-10T11:14:22.251057+0000","last_active":"2024-05-10T11:15:28.789342+0000","last_peered":"2024-05-10T11:15:28.789342+0000","last_clean":"2024-05-10T11:15:28.789342+0000","last_became_active":"2024-05-10T11:14:22.250696+0000","last_became_peered":"2024-05-10T11:14:22.250696+0000","last_unstale":"2024-05-10T11:15:28.789342+0000","last_undegraded":"2024-05-10T11:15:28.789342+0000","last_fullsized":"2024-05-10T11:15:28.789342+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:38:53.780643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.206311+0000","last_change":"2024-05-10T11:14:25.283806+0000","last_active":"2024-05-10T11:17:00.206311+0000","last_peered":"2024-05-10T11:17:00.206311+0000","last_clean":"2024-05-10T11:17:00.206311+0000","last_became_active":"2024-05-10T11:14:25.282714+0000","last_became_peered":"2024-05-10T11:14:25.282714+0000","last_unstale":"2024-05-10T11:17:00.206311+0000","last_undegraded":"2024-05-10T11:17:00.206311+0000","last_fullsized":"2024-05-10T11:17:00.206311+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:05:11.940408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:48.003154+0000","last_change":"2024-05-10T11:14:35.329986+0000","last_active":"2024-05-10T11:15:48.003154+0000","last_peered":"2024-05-10T11:15:48.003154+0000","last_clean":"2024-05-10T11:15:48.003154+0000","last_became_active":"2024-05-10T11:14:35.329477+0000","last_became_peered":"2024-05-10T11:14:35.329477+0000","last_unstale":"2024-05-10T11:15:48.003154+0000","last_undegraded":"2024-05-10T11:15:48.003154+0000","last_fullsized":"2024-05-10T11:15:48.003154+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:03:19.203580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:30.784768+0000","last_change":"2024-05-10T11:14:22.257627+0000","last_active":"2024-05-10T11:15:30.784768+0000","last_peered":"2024-05-10T11:15:30.784768+0000","last_clean":"2024-05-10T11:15:30.784768+0000","last_became_active":"2024-05-10T11:14:22.256990+0000","last_became_peered":"2024-05-10T11:14:22.256990+0000","last_unstale":"2024-05-10T11:15:30.784768+0000","last_undegraded":"2024-05-10T11:15:30.784768+0000","last_fullsized":"2024-05-10T11:15:30.784768+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:28:18.926821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.203131+0000","last_change":"2024-05-10T11:14:25.274799+0000","last_active":"2024-05-10T11:17:00.203131+0000","last_peered":"2024-05-10T11:17:00.203131+0000","last_clean":"2024-05-10T11:17:00.203131+0000","last_became_active":"2024-05-10T11:14:25.274443+0000","last_became_peered":"2024-05-10T11:14:25.274443+0000","last_unstale":"2024-05-10T11:17:00.203131+0000","last_undegraded":"2024-05-10T11:17:00.203131+0000","last_fullsized":"2024-05-10T11:17:00.203131+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:25:23.399798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459376+0000","last_change":"2024-05-10T11:14:35.418581+0000","last_active":"2024-05-10T11:15:01.459376+0000","last_peered":"2024-05-10T11:15:01.459376+0000","last_clean":"2024-05-10T11:15:01.459376+0000","last_became_active":"2024-05-10T11:14:35.418160+0000","last_became_peered":"2024-05-10T11:14:35.418160+0000","last_unstale":"2024-05-10T11:15:01.459376+0000","last_undegraded":"2024-05-10T11:15:01.459376+0000","last_fullsized":"2024-05-10T11:15:01.459376+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:39:01.873626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:32.810420+0000","last_change":"2024-05-10T11:14:22.260325+0000","last_active":"2024-05-10T11:15:32.810420+0000","last_peered":"2024-05-10T11:15:32.810420+0000","last_clean":"2024-05-10T11:15:32.810420+0000","last_became_active":"2024-05-10T11:14:22.259971+0000","last_became_peered":"2024-05-10T11:14:22.259971+0000","last_unstale":"2024-05-10T11:15:32.810420+0000","last_undegraded":"2024-05-10T11:15:32.810420+0000","last_fullsized":"2024-05-10T11:15:32.810420+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:34:55.547811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.214078+0000","last_change":"2024-05-10T11:14:25.279009+0000","last_active":"2024-05-10T11:17:00.214078+0000","last_peered":"2024-05-10T11:17:00.214078+0000","last_clean":"2024-05-10T11:17:00.214078+0000","last_became_active":"2024-05-10T11:14:25.278443+0000","last_became_peered":"2024-05-10T11:14:25.278443+0000","last_unstale":"2024-05-10T11:17:00.214078+0000","last_undegraded":"2024-05-10T11:17:00.214078+0000","last_fullsized":"2024-05-10T11:17:00.214078+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:25:22.306555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.426518+0000","last_change":"2024-05-10T11:14:35.319405+0000","last_active":"2024-05-10T11:15:01.426518+0000","last_peered":"2024-05-10T11:15:01.426518+0000","last_clean":"2024-05-10T11:15:01.426518+0000","last_became_active":"2024-05-10T11:14:35.318865+0000","last_became_peered":"2024-05-10T11:14:35.318865+0000","last_unstale":"2024-05-10T11:15:01.426518+0000","last_undegraded":"2024-05-10T11:15:01.426518+0000","last_fullsized":"2024-05-10T11:15:01.426518+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:22:41.592141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:32.813833+0000","last_change":"2024-05-10T11:14:22.299243+0000","last_active":"2024-05-10T11:15:32.813833+0000","last_peered":"2024-05-10T11:15:32.813833+0000","last_clean":"2024-05-10T11:15:32.813833+0000","last_became_active":"2024-05-10T11:14:22.298868+0000","last_became_peered":"2024-05-10T11:14:22.298868+0000","last_unstale":"2024-05-10T11:15:32.813833+0000","last_undegraded":"2024-05-10T11:15:32.813833+0000","last_fullsized":"2024-05-10T11:15:32.813833+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:54:15.078141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.213861+0000","last_change":"2024-05-10T11:14:25.281198+0000","last_active":"2024-05-10T11:17:00.213861+0000","last_peered":"2024-05-10T11:17:00.213861+0000","last_clean":"2024-05-10T11:17:00.213861+0000","last_became_active":"2024-05-10T11:14:25.280871+0000","last_became_peered":"2024-05-10T11:14:25.280871+0000","last_unstale":"2024-05-10T11:17:00.213861+0000","last_undegraded":"2024-05-10T11:17:00.213861+0000","last_fullsized":"2024-05-10T11:17:00.213861+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:24:55.843054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.425771+0000","last_change":"2024-05-10T11:14:35.329517+0000","last_active":"2024-05-10T11:15:01.425771+0000","last_peered":"2024-05-10T11:15:01.425771+0000","last_clean":"2024-05-10T11:15:01.425771+0000","last_became_active":"2024-05-10T11:14:35.329056+0000","last_became_peered":"2024-05-10T11:14:35.329056+0000","last_unstale":"2024-05-10T11:15:01.425771+0000","last_undegraded":"2024-05-10T11:15:01.425771+0000","last_fullsized":"2024-05-10T11:15:01.425771+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:09:24.844256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:32.810561+0000","last_change":"2024-05-10T11:14:22.257206+0000","last_active":"2024-05-10T11:15:32.810561+0000","last_peered":"2024-05-10T11:15:32.810561+0000","last_clean":"2024-05-10T11:15:32.810561+0000","last_became_active":"2024-05-10T11:14:22.256850+0000","last_became_peered":"2024-05-10T11:14:22.256850+0000","last_unstale":"2024-05-10T11:15:32.810561+0000","last_undegraded":"2024-05-10T11:15:32.810561+0000","last_fullsized":"2024-05-10T11:15:32.810561+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:34:31.973034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.211163+0000","last_change":"2024-05-10T11:15:01.461372+0000","last_active":"2024-05-10T11:17:00.211163+0000","last_peered":"2024-05-10T11:17:00.211163+0000","last_clean":"2024-05-10T11:17:00.211163+0000","last_became_active":"2024-05-10T11:15:01.460978+0000","last_became_peered":"2024-05-10T11:15:01.460978+0000","last_unstale":"2024-05-10T11:17:00.211163+0000","last_undegraded":"2024-05-10T11:17:00.211163+0000","last_fullsized":"2024-05-10T11:17:00.211163+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:18:45.750486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459061+0000","last_change":"2024-05-10T11:14:35.322387+0000","last_active":"2024-05-10T11:15:01.459061+0000","last_peered":"2024-05-10T11:15:01.459061+0000","last_clean":"2024-05-10T11:15:01.459061+0000","last_became_active":"2024-05-10T11:14:35.321939+0000","last_became_peered":"2024-05-10T11:14:35.321939+0000","last_unstale":"2024-05-10T11:15:01.459061+0000","last_undegraded":"2024-05-10T11:15:01.459061+0000","last_fullsized":"2024-05-10T11:15:01.459061+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:28:29.085363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459652+0000","last_change":"2024-05-10T11:14:22.258603+0000","last_active":"2024-05-10T11:15:01.459652+0000","last_peered":"2024-05-10T11:15:01.459652+0000","last_clean":"2024-05-10T11:15:01.459652+0000","last_became_active":"2024-05-10T11:14:22.258269+0000","last_became_peered":"2024-05-10T11:14:22.258269+0000","last_unstale":"2024-05-10T11:15:01.459652+0000","last_undegraded":"2024-05-10T11:15:01.459652+0000","last_fullsized":"2024-05-10T11:15:01.459652+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:30:04.240962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.207686+0000","last_change":"2024-05-10T11:14:25.281959+0000","last_active":"2024-05-10T11:17:00.207686+0000","last_peered":"2024-05-10T11:17:00.207686+0000","last_clean":"2024-05-10T11:17:00.207686+0000","last_became_active":"2024-05-10T11:14:25.281648+0000","last_became_peered":"2024-05-10T11:14:25.281648+0000","last_unstale":"2024-05-10T11:17:00.207686+0000","last_undegraded":"2024-05-10T11:17:00.207686+0000","last_fullsized":"2024-05-10T11:17:00.207686+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:24:01.569685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.195950+0000","last_change":"2024-05-10T11:14:25.284892+0000","last_active":"2024-05-10T11:17:00.195950+0000","last_peered":"2024-05-10T11:17:00.195950+0000","last_clean":"2024-05-10T11:17:00.195950+0000","last_became_active":"2024-05-10T11:14:25.284078+0000","last_became_peered":"2024-05-10T11:14:25.284078+0000","last_unstale":"2024-05-10T11:17:00.195950+0000","last_undegraded":"2024-05-10T11:17:00.195950+0000","last_fullsized":"2024-05-10T11:17:00.195950+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:49:43.348463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.211265+0000","last_change":"2024-05-10T11:14:25.274739+0000","last_active":"2024-05-10T11:17:00.211265+0000","last_peered":"2024-05-10T11:17:00.211265+0000","last_clean":"2024-05-10T11:17:00.211265+0000","last_became_active":"2024-05-10T11:14:25.274310+0000","last_became_peered":"2024-05-10T11:14:25.274310+0000","last_unstale":"2024-05-10T11:17:00.211265+0000","last_undegraded":"2024-05-10T11:17:00.211265+0000","last_fullsized":"2024-05-10T11:17:00.211265+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:21:16.935189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.204984+0000","last_change":"2024-05-10T11:14:25.272484+0000","last_active":"2024-05-10T11:17:00.204984+0000","last_peered":"2024-05-10T11:17:00.204984+0000","last_clean":"2024-05-10T11:17:00.204984+0000","last_became_active":"2024-05-10T11:14:25.272114+0000","last_became_peered":"2024-05-10T11:14:25.272114+0000","last_unstale":"2024-05-10T11:17:00.204984+0000","last_undegraded":"2024-05-10T11:17:00.204984+0000","last_fullsized":"2024-05-10T11:17:00.204984+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:46:34.537959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.208365+0000","last_change":"2024-05-10T11:15:01.460763+0000","last_active":"2024-05-10T11:17:00.208365+0000","last_peered":"2024-05-10T11:17:00.208365+0000","last_clean":"2024-05-10T11:17:00.208365+0000","last_became_active":"2024-05-10T11:15:01.460392+0000","last_became_peered":"2024-05-10T11:15:01.460392+0000","last_unstale":"2024-05-10T11:17:00.208365+0000","last_undegraded":"2024-05-10T11:17:00.208365+0000","last_fullsized":"2024-05-10T11:17:00.208365+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:15:42.859813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.202714+0000","last_change":"2024-05-10T11:14:25.269640+0000","last_active":"2024-05-10T11:17:00.202714+0000","last_peered":"2024-05-10T11:17:00.202714+0000","last_clean":"2024-05-10T11:17:00.202714+0000","last_became_active":"2024-05-10T11:14:25.269348+0000","last_became_peered":"2024-05-10T11:14:25.269348+0000","last_unstale":"2024-05-10T11:17:00.202714+0000","last_undegraded":"2024-05-10T11:17:00.202714+0000","last_fullsized":"2024-05-10T11:17:00.202714+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:40:25.276433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.194352+0000","last_change":"2024-05-10T11:14:25.282563+0000","last_active":"2024-05-10T11:17:00.194352+0000","last_peered":"2024-05-10T11:17:00.194352+0000","last_clean":"2024-05-10T11:17:00.194352+0000","last_became_active":"2024-05-10T11:14:25.282145+0000","last_became_peered":"2024-05-10T11:14:25.282145+0000","last_unstale":"2024-05-10T11:17:00.194352+0000","last_undegraded":"2024-05-10T11:17:00.194352+0000","last_fullsized":"2024-05-10T11:17:00.194352+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:54:38.436669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.202523+0000","last_change":"2024-05-10T11:14:25.276753+0000","last_active":"2024-05-10T11:17:00.202523+0000","last_peered":"2024-05-10T11:17:00.202523+0000","last_clean":"2024-05-10T11:17:00.202523+0000","last_became_active":"2024-05-10T11:14:25.276342+0000","last_became_peered":"2024-05-10T11:14:25.276342+0000","last_unstale":"2024-05-10T11:17:00.202523+0000","last_undegraded":"2024-05-10T11:17:00.202523+0000","last_fullsized":"2024-05-10T11:17:00.202523+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:31:42.766069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.209177+0000","last_change":"2024-05-10T11:14:25.275905+0000","last_active":"2024-05-10T11:17:00.209177+0000","last_peered":"2024-05-10T11:17:00.209177+0000","last_clean":"2024-05-10T11:17:00.209177+0000","last_became_active":"2024-05-10T11:14:25.275422+0000","last_became_peered":"2024-05-10T11:14:25.275422+0000","last_unstale":"2024-05-10T11:17:00.209177+0000","last_undegraded":"2024-05-10T11:17:00.209177+0000","last_fullsized":"2024-05-10T11:17:00.209177+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:39:24.033519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.205191+0000","last_change":"2024-05-10T11:14:25.271602+0000","last_active":"2024-05-10T11:17:00.205191+0000","last_peered":"2024-05-10T11:17:00.205191+0000","last_clean":"2024-05-10T11:17:00.205191+0000","last_became_active":"2024-05-10T11:14:25.267206+0000","last_became_peered":"2024-05-10T11:14:25.267206+0000","last_unstale":"2024-05-10T11:17:00.205191+0000","last_undegraded":"2024-05-10T11:17:00.205191+0000","last_fullsized":"2024-05-10T11:17:00.205191+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:28:24.708297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.206462+0000","last_change":"2024-05-10T11:14:25.280378+0000","last_active":"2024-05-10T11:17:00.206462+0000","last_peered":"2024-05-10T11:17:00.206462+0000","last_clean":"2024-05-10T11:17:00.206462+0000","last_became_active":"2024-05-10T11:14:25.280079+0000","last_became_peered":"2024-05-10T11:14:25.280079+0000","last_unstale":"2024-05-10T11:17:00.206462+0000","last_undegraded":"2024-05-10T11:17:00.206462+0000","last_fullsized":"2024-05-10T11:17:00.206462+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:13:47.781060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.204726+0000","last_change":"2024-05-10T11:14:25.272363+0000","last_active":"2024-05-10T11:17:00.204726+0000","last_peered":"2024-05-10T11:17:00.204726+0000","last_clean":"2024-05-10T11:17:00.204726+0000","last_became_active":"2024-05-10T11:14:25.271847+0000","last_became_peered":"2024-05-10T11:14:25.271847+0000","last_unstale":"2024-05-10T11:17:00.204726+0000","last_undegraded":"2024-05-10T11:17:00.204726+0000","last_fullsized":"2024-05-10T11:17:00.204726+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:47:54.661069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.203714+0000","last_change":"2024-05-10T11:14:25.272688+0000","last_active":"2024-05-10T11:17:00.203714+0000","last_peered":"2024-05-10T11:17:00.203714+0000","last_clean":"2024-05-10T11:17:00.203714+0000","last_became_active":"2024-05-10T11:14:25.271654+0000","last_became_peered":"2024-05-10T11:14:25.271654+0000","last_unstale":"2024-05-10T11:17:00.203714+0000","last_undegraded":"2024-05-10T11:17:00.203714+0000","last_fullsized":"2024-05-10T11:17:00.203714+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:50:21.149707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.199436+0000","last_change":"2024-05-10T11:15:01.432279+0000","last_active":"2024-05-10T11:17:00.199436+0000","last_peered":"2024-05-10T11:17:00.199436+0000","last_clean":"2024-05-10T11:17:00.199436+0000","last_became_active":"2024-05-10T11:15:01.431751+0000","last_became_peered":"2024-05-10T11:15:01.431751+0000","last_unstale":"2024-05-10T11:17:00.199436+0000","last_undegraded":"2024-05-10T11:17:00.199436+0000","last_fullsized":"2024-05-10T11:17:00.199436+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:25:44.196618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69059,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":624,"num_read_kb":388,"num_write":179,"num_write_kb":127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":207177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":153,"ondisk_log_size":153,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645362,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1015808,"data_stored":505095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841591,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":41160,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93703992,"statfs":{"total":95995035648,"available":95952887808,"internally_reserved":0,"allocated":1703936,"data_stored":1157258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":40434113},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005117,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1032192,"data_stored":509405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201345,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1024000,"data_stored":505207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430279,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28600,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716552,"statfs":{"total":95995035648,"available":95965749248,"internally_reserved":0,"allocated":1622016,"data_stored":1101811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593805,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":500485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574949,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1077248,"data_stored":564780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738474,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1032192,"data_stored":501873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411425,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36412,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708740,"statfs":{"total":95995035648,"available":95957749760,"internally_reserved":0,"allocated":1101824,"data_stored":566662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149787,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1019904,"data_stored":500679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953559,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1019904,"data_stored":505667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790033,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28608,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716544,"statfs":{"total":95995035648,"available":95965741056,"internally_reserved":0,"allocated":1630208,"data_stored":1096867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":66267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T11:17:15.013 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-10T11:17:15.258 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:17:15.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:17:15 smithi188 ceph-mon[34075]: from='client.14991 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T11:17:15.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:17:15 smithi188 ceph-mon[34075]: pgmap v386: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-10T11:17:15.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:17:15 smithi100 ceph-mon[30296]: from='client.14991 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T11:17:15.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:17:15 smithi100 ceph-mon[30296]: pgmap v386: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-10T11:17:15.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:17:15 smithi178 ceph-mon[33359]: from='client.14991 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T11:17:15.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:17:15 smithi178 ceph-mon[33359]: pgmap v386: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-10T11:17:17.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:17.028+0000 7f93244e2700 1 -- 172.21.15.100:0/3946788215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f931c06c150 msgr2=0x7f931c06c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:17.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:17.028+0000 7f93244e2700 1 --2- 172.21.15.100:0/3946788215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f931c06c150 0x7f931c06c5d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f930c009990 tx=0x7f930c009ca0 comp rx=0 tx=0).stop 2024-05-10T11:17:17.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:17.028+0000 7f93244e2700 1 -- 172.21.15.100:0/3946788215 shutdown_connections 2024-05-10T11:17:17.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:17.028+0000 7f93244e2700 1 --2- 172.21.15.100:0/3946788215 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f931c105f30 0x7f931c1063b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:17.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:17.028+0000 7f93244e2700 1 --2- 172.21.15.100:0/3946788215 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f931c1084c0 0x7f931c06bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:17.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:17.028+0000 7f93244e2700 1 --2- 172.21.15.100:0/3946788215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f931c06c150 0x7f931c06c5d0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:17.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:17.028+0000 7f93244e2700 1 -- 172.21.15.100:0/3946788215 >> 172.21.15.100:0/3946788215 conn(0x7f931c0783f0 msgr2=0x7f931c078800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:17:17.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:17.028+0000 7f93244e2700 1 -- 172.21.15.100:0/3946788215 shutdown_connections 2024-05-10T11:17:17.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:17.028+0000 7f93244e2700 1 -- 172.21.15.100:0/3946788215 wait complete. 2024-05-10T11:17:17.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:17.029+0000 7f93244e2700 1 Processor -- start 2024-05-10T11:17:17.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:17.029+0000 7f93244e2700 1 -- start start 2024-05-10T11:17:17.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:17.029+0000 7f93244e2700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f931c06c150 0x7f931c1a00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:17.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:17.029+0000 7f93244e2700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f931c105f30 0x7f931c1a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:17.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:17.029+0000 7f93244e2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f931c1084c0 0x7f931c1a4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:17.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:17.029+0000 7f93244e2700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f931c119b80 con 0x7f931c1084c0 2024-05-10T11:17:17.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:17.029+0000 7f93244e2700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f931c119a00 con 0x7f931c06c150 2024-05-10T11:17:17.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:17.029+0000 7f93244e2700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f931c119d00 con 0x7f931c105f30 2024-05-10T11:17:17.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:17.029+0000 7f9322a7f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f931c1084c0 0x7f931c1a4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:17.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:17.029+0000 7f9322a7f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f931c1084c0 0x7f931c1a4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58050/0 (socket says 172.21.15.100:58050) 2024-05-10T11:17:17.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:17.029+0000 7f9322a7f700 1 -- 172.21.15.100:0/3357053240 learned_addr learned my addr 172.21.15.100:0/3357053240 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:17:17.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:17.029+0000 7f932227e700 1 --2- 172.21.15.100:0/3357053240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f931c06c150 0x7f931c1a00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:17.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:17.029+0000 7f9321a7d700 1 --2- 172.21.15.100:0/3357053240 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f931c105f30 0x7f931c1a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:17.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:17.029+0000 7f9322a7f700 1 -- 172.21.15.100:0/3357053240 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f931c105f30 msgr2=0x7f931c1a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:17.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:17.029+0000 7f9322a7f700 1 --2- 172.21.15.100:0/3357053240 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f931c105f30 0x7f931c1a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:17.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:17.029+0000 7f9322a7f700 1 -- 172.21.15.100:0/3357053240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f931c06c150 msgr2=0x7f931c1a00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:17.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:17.029+0000 7f9322a7f700 1 --2- 172.21.15.100:0/3357053240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f931c06c150 0x7f931c1a00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:17.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:17.029+0000 7f9322a7f700 1 -- 172.21.15.100:0/3357053240 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f931c1a5090 con 0x7f931c1084c0 2024-05-10T11:17:17.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:17.029+0000 7f932227e700 1 --2- 172.21.15.100:0/3357053240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f931c06c150 0x7f931c1a00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:17:17.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:17.029+0000 7f9321a7d700 1 --2- 172.21.15.100:0/3357053240 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f931c105f30 0x7f931c1a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:17:17.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:17.030+0000 7f9322a7f700 1 --2- 172.21.15.100:0/3357053240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f931c1084c0 0x7f931c1a4950 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9314007d70 tx=0x7f931400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:17:17.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:17.030+0000 7f93137fe700 1 -- 172.21.15.100:0/3357053240 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9314016020 con 0x7f931c1084c0 2024-05-10T11:17:17.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:17.030+0000 7f93244e2700 1 -- 172.21.15.100:0/3357053240 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f931c1a5380 con 0x7f931c1084c0 2024-05-10T11:17:17.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:17.030+0000 7f93137fe700 1 -- 172.21.15.100:0/3357053240 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f931400ad40 con 0x7f931c1084c0 2024-05-10T11:17:17.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:17.030+0000 7f93137fe700 1 -- 172.21.15.100:0/3357053240 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93140115e0 con 0x7f931c1084c0 2024-05-10T11:17:17.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:17.030+0000 7f93244e2700 1 -- 172.21.15.100:0/3357053240 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f931c1acc80 con 0x7f931c1084c0 2024-05-10T11:17:17.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:17.031+0000 7f93244e2700 1 -- 172.21.15.100:0/3357053240 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f931c06a190 con 0x7f931c1084c0 2024-05-10T11:17:17.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:17.032+0000 7f93137fe700 1 -- 172.21.15.100:0/3357053240 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f9314019020 con 0x7f931c1084c0 2024-05-10T11:17:17.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:17.033+0000 7f93137fe700 1 --2- 172.21.15.100:0/3357053240 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f93080b2880 0x7f93080b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:17.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:17.033+0000 7f932227e700 1 --2- 172.21.15.100:0/3357053240 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f93080b2880 0x7f93080b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:17.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:17.033+0000 7f93137fe700 1 -- 172.21.15.100:0/3357053240 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f93140e2a00 con 0x7f931c1084c0 2024-05-10T11:17:17.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:17.034+0000 7f932227e700 1 --2- 172.21.15.100:0/3357053240 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f93080b2880 0x7f93080b4d40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f93180099d0 tx=0x7f9318008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:17:17.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:17.037+0000 7f93137fe700 1 -- 172.21.15.100:0/3357053240 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f9314020020 con 0x7f931c1084c0 2024-05-10T11:17:17.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:17.241+0000 7f93244e2700 1 -- 172.21.15.100:0/3357053240 --> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f931c1044a0 con 0x7f93080b2880 2024-05-10T11:17:17.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:17.250+0000 7f93137fe700 1 -- 172.21.15.100:0/3357053240 <== mgr.14152 v2:172.21.15.100:6800/2507287356 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340468 (secure 0 0 0) 0x7f931c1044a0 con 0x7f93080b2880 2024-05-10T11:17:17.254 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:17:17.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:17.254+0000 7f93244e2700 1 -- 172.21.15.100:0/3357053240 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f93080b2880 msgr2=0x7f93080b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:17.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:17.254+0000 7f93244e2700 1 --2- 172.21.15.100:0/3357053240 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f93080b2880 0x7f93080b4d40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f93180099d0 tx=0x7f9318008040 comp rx=0 tx=0).stop 2024-05-10T11:17:17.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:17.254+0000 7f93244e2700 1 -- 172.21.15.100:0/3357053240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f931c1084c0 msgr2=0x7f931c1a4950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:17.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:17.254+0000 7f93244e2700 1 --2- 172.21.15.100:0/3357053240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f931c1084c0 0x7f931c1a4950 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9314007d70 tx=0x7f931400a340 comp rx=0 tx=0).stop 2024-05-10T11:17:17.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:17.254+0000 7f93244e2700 1 -- 172.21.15.100:0/3357053240 shutdown_connections 2024-05-10T11:17:17.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:17.254+0000 7f93244e2700 1 --2- 172.21.15.100:0/3357053240 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f93080b2880 0x7f93080b4d40 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:17.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:17.254+0000 7f93244e2700 1 --2- 172.21.15.100:0/3357053240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f931c1084c0 0x7f931c1a4950 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:17.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:17.254+0000 7f93244e2700 1 --2- 172.21.15.100:0/3357053240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f931c06c150 0x7f931c1a00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:17.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:17.254+0000 7f93244e2700 1 --2- 172.21.15.100:0/3357053240 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f931c105f30 0x7f931c1a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:17.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:17.254+0000 7f93244e2700 1 -- 172.21.15.100:0/3357053240 >> 172.21.15.100:0/3357053240 conn(0x7f931c0783f0 msgr2=0x7f931c103d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:17:17.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:17.254+0000 7f93244e2700 1 -- 172.21.15.100:0/3357053240 shutdown_connections 2024-05-10T11:17:17.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:17.254+0000 7f93244e2700 1 -- 172.21.15.100:0/3357053240 wait complete. 2024-05-10T11:17:17.258 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-05-10T11:17:17.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:17:17 smithi188 ceph-mon[34075]: pgmap v387: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-10T11:17:17.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:17:17 smithi100 ceph-mon[30296]: pgmap v387: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-10T11:17:17.554 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:17:17 smithi178 ceph-mon[33359]: pgmap v387: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-10T11:17:17.941 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":387,"stamp":"2024-05-10T11:17:16.227285+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659661,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":710,"num_read_kb":452,"num_write":304,"num_write_kb":2279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":747,"ondisk_log_size":747,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":404704,"kb_used_data":13952,"kb_used_omap":18,"kb_used_meta":390637,"kb_avail":1124537120,"statfs":{"total":1151940427776,"available":1151526010880,"internally_reserved":0,"allocated":14286848,"data_stored":8015789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":400012603},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":72,"num_read_kb":32,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003061"},"pg_stats":[{"pgid":"3.3f","version":"89'8","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.206983+0000","last_change":"2024-05-10T11:14:25.286878+0000","last_active":"2024-05-10T11:17:00.206983+0000","last_peered":"2024-05-10T11:17:00.206983+0000","last_clean":"2024-05-10T11:17:00.206983+0000","last_became_active":"2024-05-10T11:14:25.286462+0000","last_became_peered":"2024-05-10T11:14:25.286462+0000","last_unstale":"2024-05-10T11:17:00.206983+0000","last_undegraded":"2024-05-10T11:17:00.206983+0000","last_fullsized":"2024-05-10T11:17:00.206983+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:13:40.627138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.204906+0000","last_change":"2024-05-10T11:14:25.280884+0000","last_active":"2024-05-10T11:17:00.204906+0000","last_peered":"2024-05-10T11:17:00.204906+0000","last_clean":"2024-05-10T11:17:00.204906+0000","last_became_active":"2024-05-10T11:14:25.280564+0000","last_became_peered":"2024-05-10T11:14:25.280564+0000","last_unstale":"2024-05-10T11:17:00.204906+0000","last_undegraded":"2024-05-10T11:17:00.204906+0000","last_fullsized":"2024-05-10T11:17:00.204906+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:14:16.735326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.208402+0000","last_change":"2024-05-10T11:14:25.278676+0000","last_active":"2024-05-10T11:17:00.208402+0000","last_peered":"2024-05-10T11:17:00.208402+0000","last_clean":"2024-05-10T11:17:00.208402+0000","last_became_active":"2024-05-10T11:14:25.278196+0000","last_became_peered":"2024-05-10T11:14:25.278196+0000","last_unstale":"2024-05-10T11:17:00.208402+0000","last_undegraded":"2024-05-10T11:17:00.208402+0000","last_fullsized":"2024-05-10T11:17:00.208402+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:50:29.390014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.207221+0000","last_change":"2024-05-10T11:14:25.274741+0000","last_active":"2024-05-10T11:17:00.207221+0000","last_peered":"2024-05-10T11:17:00.207221+0000","last_clean":"2024-05-10T11:17:00.207221+0000","last_became_active":"2024-05-10T11:14:25.274251+0000","last_became_peered":"2024-05-10T11:14:25.274251+0000","last_unstale":"2024-05-10T11:17:00.207221+0000","last_undegraded":"2024-05-10T11:17:00.207221+0000","last_fullsized":"2024-05-10T11:17:00.207221+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:04:41.552042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.186404+0000","last_change":"2024-05-10T11:14:25.265232+0000","last_active":"2024-05-10T11:17:00.186404+0000","last_peered":"2024-05-10T11:17:00.186404+0000","last_clean":"2024-05-10T11:17:00.186404+0000","last_became_active":"2024-05-10T11:14:25.264852+0000","last_became_peered":"2024-05-10T11:14:25.264852+0000","last_unstale":"2024-05-10T11:17:00.186404+0000","last_undegraded":"2024-05-10T11:17:00.186404+0000","last_fullsized":"2024-05-10T11:17:00.186404+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:50:27.207258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.210357+0000","last_change":"2024-05-10T11:14:25.272626+0000","last_active":"2024-05-10T11:17:00.210357+0000","last_peered":"2024-05-10T11:17:00.210357+0000","last_clean":"2024-05-10T11:17:00.210357+0000","last_became_active":"2024-05-10T11:14:25.272305+0000","last_became_peered":"2024-05-10T11:14:25.272305+0000","last_unstale":"2024-05-10T11:17:00.210357+0000","last_undegraded":"2024-05-10T11:17:00.210357+0000","last_fullsized":"2024-05-10T11:17:00.210357+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:44:16.725453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.210435+0000","last_change":"2024-05-10T11:14:25.275460+0000","last_active":"2024-05-10T11:17:00.210435+0000","last_peered":"2024-05-10T11:17:00.210435+0000","last_clean":"2024-05-10T11:17:00.210435+0000","last_became_active":"2024-05-10T11:14:25.274571+0000","last_became_peered":"2024-05-10T11:14:25.274571+0000","last_unstale":"2024-05-10T11:17:00.210435+0000","last_undegraded":"2024-05-10T11:17:00.210435+0000","last_fullsized":"2024-05-10T11:17:00.210435+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:33:20.335738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.207412+0000","last_change":"2024-05-10T11:14:25.273975+0000","last_active":"2024-05-10T11:17:00.207412+0000","last_peered":"2024-05-10T11:17:00.207412+0000","last_clean":"2024-05-10T11:17:00.207412+0000","last_became_active":"2024-05-10T11:14:25.273652+0000","last_became_peered":"2024-05-10T11:14:25.273652+0000","last_unstale":"2024-05-10T11:17:00.207412+0000","last_undegraded":"2024-05-10T11:17:00.207412+0000","last_fullsized":"2024-05-10T11:17:00.207412+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:37:55.354449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.207494+0000","last_change":"2024-05-10T11:14:25.285530+0000","last_active":"2024-05-10T11:17:00.207494+0000","last_peered":"2024-05-10T11:17:00.207494+0000","last_clean":"2024-05-10T11:17:00.207494+0000","last_became_active":"2024-05-10T11:14:25.285235+0000","last_became_peered":"2024-05-10T11:14:25.285235+0000","last_unstale":"2024-05-10T11:17:00.207494+0000","last_undegraded":"2024-05-10T11:17:00.207494+0000","last_fullsized":"2024-05-10T11:17:00.207494+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:11:14.225339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.210587+0000","last_change":"2024-05-10T11:14:25.274738+0000","last_active":"2024-05-10T11:17:00.210587+0000","last_peered":"2024-05-10T11:17:00.210587+0000","last_clean":"2024-05-10T11:17:00.210587+0000","last_became_active":"2024-05-10T11:14:25.274107+0000","last_became_peered":"2024-05-10T11:14:25.274107+0000","last_unstale":"2024-05-10T11:17:00.210587+0000","last_undegraded":"2024-05-10T11:17:00.210587+0000","last_fullsized":"2024-05-10T11:17:00.210587+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:02:01.489930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.213489+0000","last_change":"2024-05-10T11:14:25.277431+0000","last_active":"2024-05-10T11:17:00.213489+0000","last_peered":"2024-05-10T11:17:00.213489+0000","last_clean":"2024-05-10T11:17:00.213489+0000","last_became_active":"2024-05-10T11:14:25.277130+0000","last_became_peered":"2024-05-10T11:14:25.277130+0000","last_unstale":"2024-05-10T11:17:00.213489+0000","last_undegraded":"2024-05-10T11:17:00.213489+0000","last_fullsized":"2024-05-10T11:17:00.213489+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:38:17.192282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.206976+0000","last_change":"2024-05-10T11:14:25.276291+0000","last_active":"2024-05-10T11:17:00.206976+0000","last_peered":"2024-05-10T11:17:00.206976+0000","last_clean":"2024-05-10T11:17:00.206976+0000","last_became_active":"2024-05-10T11:14:25.275842+0000","last_became_peered":"2024-05-10T11:14:25.275842+0000","last_unstale":"2024-05-10T11:17:00.206976+0000","last_undegraded":"2024-05-10T11:17:00.206976+0000","last_fullsized":"2024-05-10T11:17:00.206976+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:20:53.443135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.209964+0000","last_change":"2024-05-10T11:14:25.273950+0000","last_active":"2024-05-10T11:17:00.209964+0000","last_peered":"2024-05-10T11:17:00.209964+0000","last_clean":"2024-05-10T11:17:00.209964+0000","last_became_active":"2024-05-10T11:14:25.273644+0000","last_became_peered":"2024-05-10T11:14:25.273644+0000","last_unstale":"2024-05-10T11:17:00.209964+0000","last_undegraded":"2024-05-10T11:17:00.209964+0000","last_fullsized":"2024-05-10T11:17:00.209964+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:46:05.112575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.209077+0000","last_change":"2024-05-10T11:14:25.277930+0000","last_active":"2024-05-10T11:17:00.209077+0000","last_peered":"2024-05-10T11:17:00.209077+0000","last_clean":"2024-05-10T11:17:00.209077+0000","last_became_active":"2024-05-10T11:14:25.276793+0000","last_became_peered":"2024-05-10T11:14:25.276793+0000","last_unstale":"2024-05-10T11:17:00.209077+0000","last_undegraded":"2024-05-10T11:17:00.209077+0000","last_fullsized":"2024-05-10T11:17:00.209077+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:47:03.592882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.214057+0000","last_change":"2024-05-10T11:14:25.270434+0000","last_active":"2024-05-10T11:17:00.214057+0000","last_peered":"2024-05-10T11:17:00.214057+0000","last_clean":"2024-05-10T11:17:00.214057+0000","last_became_active":"2024-05-10T11:14:25.270139+0000","last_became_peered":"2024-05-10T11:14:25.270139+0000","last_unstale":"2024-05-10T11:17:00.214057+0000","last_undegraded":"2024-05-10T11:17:00.214057+0000","last_fullsized":"2024-05-10T11:17:00.214057+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:58:40.506560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.209206+0000","last_change":"2024-05-10T11:14:25.272981+0000","last_active":"2024-05-10T11:17:00.209206+0000","last_peered":"2024-05-10T11:17:00.209206+0000","last_clean":"2024-05-10T11:17:00.209206+0000","last_became_active":"2024-05-10T11:14:25.272592+0000","last_became_peered":"2024-05-10T11:14:25.272592+0000","last_unstale":"2024-05-10T11:17:00.209206+0000","last_undegraded":"2024-05-10T11:17:00.209206+0000","last_fullsized":"2024-05-10T11:17:00.209206+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:46:09.673013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.212493+0000","last_change":"2024-05-10T11:14:25.287304+0000","last_active":"2024-05-10T11:17:00.212493+0000","last_peered":"2024-05-10T11:17:00.212493+0000","last_clean":"2024-05-10T11:17:00.212493+0000","last_became_active":"2024-05-10T11:14:25.286964+0000","last_became_peered":"2024-05-10T11:14:25.286964+0000","last_unstale":"2024-05-10T11:17:00.212493+0000","last_undegraded":"2024-05-10T11:17:00.212493+0000","last_fullsized":"2024-05-10T11:17:00.212493+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:32:27.006840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.209855+0000","last_change":"2024-05-10T11:14:25.275366+0000","last_active":"2024-05-10T11:17:00.209855+0000","last_peered":"2024-05-10T11:17:00.209855+0000","last_clean":"2024-05-10T11:17:00.209855+0000","last_became_active":"2024-05-10T11:14:25.274986+0000","last_became_peered":"2024-05-10T11:14:25.274986+0000","last_unstale":"2024-05-10T11:17:00.209855+0000","last_undegraded":"2024-05-10T11:17:00.209855+0000","last_fullsized":"2024-05-10T11:17:00.209855+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:50:50.071975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.203474+0000","last_change":"2024-05-10T11:14:25.274588+0000","last_active":"2024-05-10T11:17:00.203474+0000","last_peered":"2024-05-10T11:17:00.203474+0000","last_clean":"2024-05-10T11:17:00.203474+0000","last_became_active":"2024-05-10T11:14:25.274179+0000","last_became_peered":"2024-05-10T11:14:25.274179+0000","last_unstale":"2024-05-10T11:17:00.203474+0000","last_undegraded":"2024-05-10T11:17:00.203474+0000","last_fullsized":"2024-05-10T11:17:00.203474+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:06:23.517082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.461771+0000","last_change":"2024-05-10T11:15:01.461337+0000","last_active":"2024-05-10T11:15:01.461771+0000","last_peered":"2024-05-10T11:15:01.461771+0000","last_clean":"2024-05-10T11:15:01.461771+0000","last_became_active":"2024-05-10T11:15:01.460776+0000","last_became_peered":"2024-05-10T11:15:01.460776+0000","last_unstale":"2024-05-10T11:15:01.461771+0000","last_undegraded":"2024-05-10T11:15:01.461771+0000","last_fullsized":"2024-05-10T11:15:01.461771+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:06:49.156483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,10,0],"acting":[2,3,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:32.813572+0000","last_change":"2024-05-10T11:14:22.253909+0000","last_active":"2024-05-10T11:15:32.813572+0000","last_peered":"2024-05-10T11:15:32.813572+0000","last_clean":"2024-05-10T11:15:32.813572+0000","last_became_active":"2024-05-10T11:14:22.253400+0000","last_became_peered":"2024-05-10T11:14:22.253400+0000","last_unstale":"2024-05-10T11:15:32.813572+0000","last_undegraded":"2024-05-10T11:15:32.813572+0000","last_fullsized":"2024-05-10T11:15:32.813572+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:06:46.173511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.206338+0000","last_change":"2024-05-10T11:14:25.273675+0000","last_active":"2024-05-10T11:17:00.206338+0000","last_peered":"2024-05-10T11:17:00.206338+0000","last_clean":"2024-05-10T11:17:00.206338+0000","last_became_active":"2024-05-10T11:14:25.273320+0000","last_became_peered":"2024-05-10T11:14:25.273320+0000","last_unstale":"2024-05-10T11:17:00.206338+0000","last_undegraded":"2024-05-10T11:17:00.206338+0000","last_fullsized":"2024-05-10T11:17:00.206338+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:09:18.232236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459008+0000","last_change":"2024-05-10T11:14:35.326204+0000","last_active":"2024-05-10T11:15:01.459008+0000","last_peered":"2024-05-10T11:15:01.459008+0000","last_clean":"2024-05-10T11:15:01.459008+0000","last_became_active":"2024-05-10T11:14:35.325715+0000","last_became_peered":"2024-05-10T11:14:35.325715+0000","last_unstale":"2024-05-10T11:15:01.459008+0000","last_undegraded":"2024-05-10T11:15:01.459008+0000","last_fullsized":"2024-05-10T11:15:01.459008+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:06:30.665861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:29.796053+0000","last_change":"2024-05-10T11:14:22.299194+0000","last_active":"2024-05-10T11:15:29.796053+0000","last_peered":"2024-05-10T11:15:29.796053+0000","last_clean":"2024-05-10T11:15:29.796053+0000","last_became_active":"2024-05-10T11:14:22.298807+0000","last_became_peered":"2024-05-10T11:14:22.298807+0000","last_unstale":"2024-05-10T11:15:29.796053+0000","last_undegraded":"2024-05-10T11:15:29.796053+0000","last_fullsized":"2024-05-10T11:15:29.796053+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:24:51.054834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.209768+0000","last_change":"2024-05-10T11:14:25.277860+0000","last_active":"2024-05-10T11:17:00.209768+0000","last_peered":"2024-05-10T11:17:00.209768+0000","last_clean":"2024-05-10T11:17:00.209768+0000","last_became_active":"2024-05-10T11:14:25.276784+0000","last_became_peered":"2024-05-10T11:14:25.276784+0000","last_unstale":"2024-05-10T11:17:00.209768+0000","last_undegraded":"2024-05-10T11:17:00.209768+0000","last_fullsized":"2024-05-10T11:17:00.209768+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:42:57.673595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459304+0000","last_change":"2024-05-10T11:14:35.330679+0000","last_active":"2024-05-10T11:15:01.459304+0000","last_peered":"2024-05-10T11:15:01.459304+0000","last_clean":"2024-05-10T11:15:01.459304+0000","last_became_active":"2024-05-10T11:14:35.329819+0000","last_became_peered":"2024-05-10T11:14:35.329819+0000","last_unstale":"2024-05-10T11:15:01.459304+0000","last_undegraded":"2024-05-10T11:15:01.459304+0000","last_fullsized":"2024-05-10T11:15:01.459304+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:31:26.982784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:29.798740+0000","last_change":"2024-05-10T11:14:22.260402+0000","last_active":"2024-05-10T11:15:29.798740+0000","last_peered":"2024-05-10T11:15:29.798740+0000","last_clean":"2024-05-10T11:15:29.798740+0000","last_became_active":"2024-05-10T11:14:22.259619+0000","last_became_peered":"2024-05-10T11:14:22.259619+0000","last_unstale":"2024-05-10T11:15:29.798740+0000","last_undegraded":"2024-05-10T11:15:29.798740+0000","last_fullsized":"2024-05-10T11:15:29.798740+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:56:34.491626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.204002+0000","last_change":"2024-05-10T11:14:25.276117+0000","last_active":"2024-05-10T11:17:00.204002+0000","last_peered":"2024-05-10T11:17:00.204002+0000","last_clean":"2024-05-10T11:17:00.204002+0000","last_became_active":"2024-05-10T11:14:25.275758+0000","last_became_peered":"2024-05-10T11:14:25.275758+0000","last_unstale":"2024-05-10T11:17:00.204002+0000","last_undegraded":"2024-05-10T11:17:00.204002+0000","last_fullsized":"2024-05-10T11:17:00.204002+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:12:16.582244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459213+0000","last_change":"2024-05-10T11:14:35.324633+0000","last_active":"2024-05-10T11:15:01.459213+0000","last_peered":"2024-05-10T11:15:01.459213+0000","last_clean":"2024-05-10T11:15:01.459213+0000","last_became_active":"2024-05-10T11:14:35.324152+0000","last_became_peered":"2024-05-10T11:14:35.324152+0000","last_unstale":"2024-05-10T11:15:01.459213+0000","last_undegraded":"2024-05-10T11:15:01.459213+0000","last_fullsized":"2024-05-10T11:15:01.459213+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:56:50.008782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.430353+0000","last_change":"2024-05-10T11:14:22.259086+0000","last_active":"2024-05-10T11:15:01.430353+0000","last_peered":"2024-05-10T11:15:01.430353+0000","last_clean":"2024-05-10T11:15:01.430353+0000","last_became_active":"2024-05-10T11:14:22.258628+0000","last_became_peered":"2024-05-10T11:14:22.258628+0000","last_unstale":"2024-05-10T11:15:01.430353+0000","last_undegraded":"2024-05-10T11:15:01.430353+0000","last_fullsized":"2024-05-10T11:15:01.430353+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:39:13.472622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.213935+0000","last_change":"2024-05-10T11:14:25.277556+0000","last_active":"2024-05-10T11:17:00.213935+0000","last_peered":"2024-05-10T11:17:00.213935+0000","last_clean":"2024-05-10T11:17:00.213935+0000","last_became_active":"2024-05-10T11:14:25.277201+0000","last_became_peered":"2024-05-10T11:14:25.277201+0000","last_unstale":"2024-05-10T11:17:00.213935+0000","last_undegraded":"2024-05-10T11:17:00.213935+0000","last_fullsized":"2024-05-10T11:17:00.213935+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:04:17.028305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.426356+0000","last_change":"2024-05-10T11:14:35.323338+0000","last_active":"2024-05-10T11:15:01.426356+0000","last_peered":"2024-05-10T11:15:01.426356+0000","last_clean":"2024-05-10T11:15:01.426356+0000","last_became_active":"2024-05-10T11:14:35.321948+0000","last_became_peered":"2024-05-10T11:14:35.321948+0000","last_unstale":"2024-05-10T11:15:01.426356+0000","last_undegraded":"2024-05-10T11:15:01.426356+0000","last_fullsized":"2024-05-10T11:15:01.426356+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:41:30.569125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'7","reported_seq":153,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:12.961435+0000","last_change":"2024-05-10T11:14:22.259187+0000","last_active":"2024-05-10T11:17:12.961435+0000","last_peered":"2024-05-10T11:17:12.961435+0000","last_clean":"2024-05-10T11:17:12.961435+0000","last_became_active":"2024-05-10T11:14:22.258812+0000","last_became_peered":"2024-05-10T11:14:22.258812+0000","last_unstale":"2024-05-10T11:17:12.961435+0000","last_undegraded":"2024-05-10T11:17:12.961435+0000","last_fullsized":"2024-05-10T11:17:12.961435+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:15:38.769705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.188655+0000","last_change":"2024-05-10T11:14:25.262469+0000","last_active":"2024-05-10T11:17:00.188655+0000","last_peered":"2024-05-10T11:17:00.188655+0000","last_clean":"2024-05-10T11:17:00.188655+0000","last_became_active":"2024-05-10T11:14:25.262104+0000","last_became_peered":"2024-05-10T11:14:25.262104+0000","last_unstale":"2024-05-10T11:17:00.188655+0000","last_undegraded":"2024-05-10T11:17:00.188655+0000","last_fullsized":"2024-05-10T11:17:00.188655+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:48:24.329136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.426927+0000","last_change":"2024-05-10T11:14:35.327737+0000","last_active":"2024-05-10T11:15:01.426927+0000","last_peered":"2024-05-10T11:15:01.426927+0000","last_clean":"2024-05-10T11:15:01.426927+0000","last_became_active":"2024-05-10T11:14:35.327209+0000","last_became_peered":"2024-05-10T11:14:35.327209+0000","last_unstale":"2024-05-10T11:15:01.426927+0000","last_undegraded":"2024-05-10T11:15:01.426927+0000","last_fullsized":"2024-05-10T11:15:01.426927+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:32:11.784752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:29.797324+0000","last_change":"2024-05-10T11:14:22.254853+0000","last_active":"2024-05-10T11:15:29.797324+0000","last_peered":"2024-05-10T11:15:29.797324+0000","last_clean":"2024-05-10T11:15:29.797324+0000","last_became_active":"2024-05-10T11:14:22.253352+0000","last_became_peered":"2024-05-10T11:14:22.253352+0000","last_unstale":"2024-05-10T11:15:29.797324+0000","last_undegraded":"2024-05-10T11:15:29.797324+0000","last_fullsized":"2024-05-10T11:15:29.797324+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:14:15.871042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.213028+0000","last_change":"2024-05-10T11:14:25.266798+0000","last_active":"2024-05-10T11:17:00.213028+0000","last_peered":"2024-05-10T11:17:00.213028+0000","last_clean":"2024-05-10T11:17:00.213028+0000","last_became_active":"2024-05-10T11:14:25.266384+0000","last_became_peered":"2024-05-10T11:14:25.266384+0000","last_unstale":"2024-05-10T11:17:00.213028+0000","last_undegraded":"2024-05-10T11:17:00.213028+0000","last_fullsized":"2024-05-10T11:17:00.213028+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:18:07.338987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.463232+0000","last_change":"2024-05-10T11:15:01.462376+0000","last_active":"2024-05-10T11:15:01.463232+0000","last_peered":"2024-05-10T11:15:01.463232+0000","last_clean":"2024-05-10T11:15:01.463232+0000","last_became_active":"2024-05-10T11:15:01.461876+0000","last_became_peered":"2024-05-10T11:15:01.461876+0000","last_unstale":"2024-05-10T11:15:01.463232+0000","last_undegraded":"2024-05-10T11:15:01.463232+0000","last_fullsized":"2024-05-10T11:15:01.463232+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:35:52.076414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:29.797310+0000","last_change":"2024-05-10T11:14:22.253976+0000","last_active":"2024-05-10T11:15:29.797310+0000","last_peered":"2024-05-10T11:15:29.797310+0000","last_clean":"2024-05-10T11:15:29.797310+0000","last_became_active":"2024-05-10T11:14:22.252230+0000","last_became_peered":"2024-05-10T11:14:22.252230+0000","last_unstale":"2024-05-10T11:15:29.797310+0000","last_undegraded":"2024-05-10T11:15:29.797310+0000","last_fullsized":"2024-05-10T11:15:29.797310+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:14:03.208811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.206683+0000","last_change":"2024-05-10T11:14:25.279627+0000","last_active":"2024-05-10T11:17:00.206683+0000","last_peered":"2024-05-10T11:17:00.206683+0000","last_clean":"2024-05-10T11:17:00.206683+0000","last_became_active":"2024-05-10T11:14:25.279330+0000","last_became_peered":"2024-05-10T11:14:25.279330+0000","last_unstale":"2024-05-10T11:17:00.206683+0000","last_undegraded":"2024-05-10T11:17:00.206683+0000","last_fullsized":"2024-05-10T11:17:00.206683+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:50:19.608053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.426065+0000","last_change":"2024-05-10T11:14:35.321723+0000","last_active":"2024-05-10T11:15:01.426065+0000","last_peered":"2024-05-10T11:15:01.426065+0000","last_clean":"2024-05-10T11:15:01.426065+0000","last_became_active":"2024-05-10T11:14:35.321384+0000","last_became_peered":"2024-05-10T11:14:35.321384+0000","last_unstale":"2024-05-10T11:15:01.426065+0000","last_undegraded":"2024-05-10T11:15:01.426065+0000","last_fullsized":"2024-05-10T11:15:01.426065+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:04:07.770889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:28.790869+0000","last_change":"2024-05-10T11:14:22.255511+0000","last_active":"2024-05-10T11:15:28.790869+0000","last_peered":"2024-05-10T11:15:28.790869+0000","last_clean":"2024-05-10T11:15:28.790869+0000","last_became_active":"2024-05-10T11:14:22.254722+0000","last_became_peered":"2024-05-10T11:14:22.254722+0000","last_unstale":"2024-05-10T11:15:28.790869+0000","last_undegraded":"2024-05-10T11:15:28.790869+0000","last_fullsized":"2024-05-10T11:15:28.790869+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:38:27.096011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.209442+0000","last_change":"2024-05-10T11:14:25.274009+0000","last_active":"2024-05-10T11:17:00.209442+0000","last_peered":"2024-05-10T11:17:00.209442+0000","last_clean":"2024-05-10T11:17:00.209442+0000","last_became_active":"2024-05-10T11:14:25.273672+0000","last_became_peered":"2024-05-10T11:14:25.273672+0000","last_unstale":"2024-05-10T11:17:00.209442+0000","last_undegraded":"2024-05-10T11:17:00.209442+0000","last_fullsized":"2024-05-10T11:17:00.209442+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:42:57.210131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459026+0000","last_change":"2024-05-10T11:14:35.321520+0000","last_active":"2024-05-10T11:15:01.459026+0000","last_peered":"2024-05-10T11:15:01.459026+0000","last_clean":"2024-05-10T11:15:01.459026+0000","last_became_active":"2024-05-10T11:14:35.320575+0000","last_became_peered":"2024-05-10T11:14:35.320575+0000","last_unstale":"2024-05-10T11:15:01.459026+0000","last_undegraded":"2024-05-10T11:15:01.459026+0000","last_fullsized":"2024-05-10T11:15:01.459026+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:54:03.183426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":143,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:09.956129+0000","last_change":"2024-05-10T11:14:22.251463+0000","last_active":"2024-05-10T11:17:09.956129+0000","last_peered":"2024-05-10T11:17:09.956129+0000","last_clean":"2024-05-10T11:17:09.956129+0000","last_became_active":"2024-05-10T11:14:22.251132+0000","last_became_peered":"2024-05-10T11:14:22.251132+0000","last_unstale":"2024-05-10T11:17:09.956129+0000","last_undegraded":"2024-05-10T11:17:09.956129+0000","last_fullsized":"2024-05-10T11:17:09.956129+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:29:18.125380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":99,"num_read_kb":99,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.209206+0000","last_change":"2024-05-10T11:14:25.276333+0000","last_active":"2024-05-10T11:17:00.209206+0000","last_peered":"2024-05-10T11:17:00.209206+0000","last_clean":"2024-05-10T11:17:00.209206+0000","last_became_active":"2024-05-10T11:14:25.275818+0000","last_became_peered":"2024-05-10T11:14:25.275818+0000","last_unstale":"2024-05-10T11:17:00.209206+0000","last_undegraded":"2024-05-10T11:17:00.209206+0000","last_fullsized":"2024-05-10T11:17:00.209206+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:27:19.542645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.422145+0000","last_change":"2024-05-10T11:14:35.328589+0000","last_active":"2024-05-10T11:15:01.422145+0000","last_peered":"2024-05-10T11:15:01.422145+0000","last_clean":"2024-05-10T11:15:01.422145+0000","last_became_active":"2024-05-10T11:14:35.328052+0000","last_became_peered":"2024-05-10T11:14:35.328052+0000","last_unstale":"2024-05-10T11:15:01.422145+0000","last_undegraded":"2024-05-10T11:15:01.422145+0000","last_fullsized":"2024-05-10T11:15:01.422145+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:09:42.763932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:28.790098+0000","last_change":"2024-05-10T11:14:22.257872+0000","last_active":"2024-05-10T11:15:28.790098+0000","last_peered":"2024-05-10T11:15:28.790098+0000","last_clean":"2024-05-10T11:15:28.790098+0000","last_became_active":"2024-05-10T11:14:22.257530+0000","last_became_peered":"2024-05-10T11:14:22.257530+0000","last_unstale":"2024-05-10T11:15:28.790098+0000","last_undegraded":"2024-05-10T11:15:28.790098+0000","last_fullsized":"2024-05-10T11:15:28.790098+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:26:30.367209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.214274+0000","last_change":"2024-05-10T11:14:25.269436+0000","last_active":"2024-05-10T11:17:00.214274+0000","last_peered":"2024-05-10T11:17:00.214274+0000","last_clean":"2024-05-10T11:17:00.214274+0000","last_became_active":"2024-05-10T11:14:25.269102+0000","last_became_peered":"2024-05-10T11:14:25.269102+0000","last_unstale":"2024-05-10T11:17:00.214274+0000","last_undegraded":"2024-05-10T11:17:00.214274+0000","last_fullsized":"2024-05-10T11:17:00.214274+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:51:50.172464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459510+0000","last_change":"2024-05-10T11:14:35.330272+0000","last_active":"2024-05-10T11:15:01.459510+0000","last_peered":"2024-05-10T11:15:01.459510+0000","last_clean":"2024-05-10T11:15:01.459510+0000","last_became_active":"2024-05-10T11:14:35.329438+0000","last_became_peered":"2024-05-10T11:14:35.329438+0000","last_unstale":"2024-05-10T11:15:01.459510+0000","last_undegraded":"2024-05-10T11:15:01.459510+0000","last_fullsized":"2024-05-10T11:15:01.459510+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:07:00.559447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:32.813046+0000","last_change":"2024-05-10T11:14:22.299302+0000","last_active":"2024-05-10T11:15:32.813046+0000","last_peered":"2024-05-10T11:15:32.813046+0000","last_clean":"2024-05-10T11:15:32.813046+0000","last_became_active":"2024-05-10T11:14:22.298792+0000","last_became_peered":"2024-05-10T11:14:22.298792+0000","last_unstale":"2024-05-10T11:15:32.813046+0000","last_undegraded":"2024-05-10T11:15:32.813046+0000","last_fullsized":"2024-05-10T11:15:32.813046+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:12:35.280135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.200955+0000","last_change":"2024-05-10T11:14:25.275380+0000","last_active":"2024-05-10T11:17:00.200955+0000","last_peered":"2024-05-10T11:17:00.200955+0000","last_clean":"2024-05-10T11:17:00.200955+0000","last_became_active":"2024-05-10T11:14:25.275042+0000","last_became_peered":"2024-05-10T11:14:25.275042+0000","last_unstale":"2024-05-10T11:17:00.200955+0000","last_undegraded":"2024-05-10T11:17:00.200955+0000","last_fullsized":"2024-05-10T11:17:00.200955+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:22:49.598602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.422374+0000","last_change":"2024-05-10T11:14:35.327505+0000","last_active":"2024-05-10T11:15:01.422374+0000","last_peered":"2024-05-10T11:15:01.422374+0000","last_clean":"2024-05-10T11:15:01.422374+0000","last_became_active":"2024-05-10T11:14:35.326885+0000","last_became_peered":"2024-05-10T11:14:35.326885+0000","last_unstale":"2024-05-10T11:15:01.422374+0000","last_undegraded":"2024-05-10T11:15:01.422374+0000","last_fullsized":"2024-05-10T11:15:01.422374+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:23:09.010545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'37","reported_seq":285,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:10.962979+0000","last_change":"2024-05-10T11:14:22.257571+0000","last_active":"2024-05-10T11:17:10.962979+0000","last_peered":"2024-05-10T11:17:10.962979+0000","last_clean":"2024-05-10T11:17:10.962979+0000","last_became_active":"2024-05-10T11:14:22.257156+0000","last_became_peered":"2024-05-10T11:14:22.257156+0000","last_unstale":"2024-05-10T11:17:10.962979+0000","last_undegraded":"2024-05-10T11:17:10.962979+0000","last_fullsized":"2024-05-10T11:17:10.962979+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:28:06.397262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64385,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":208,"num_read_kb":167,"num_write":38,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.193870+0000","last_change":"2024-05-10T11:14:25.281160+0000","last_active":"2024-05-10T11:17:00.193870+0000","last_peered":"2024-05-10T11:17:00.193870+0000","last_clean":"2024-05-10T11:17:00.193870+0000","last_became_active":"2024-05-10T11:14:25.280763+0000","last_became_peered":"2024-05-10T11:14:25.280763+0000","last_unstale":"2024-05-10T11:17:00.193870+0000","last_undegraded":"2024-05-10T11:17:00.193870+0000","last_fullsized":"2024-05-10T11:17:00.193870+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:19:53.350792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.460284+0000","last_change":"2024-05-10T11:14:35.320918+0000","last_active":"2024-05-10T11:15:01.460284+0000","last_peered":"2024-05-10T11:15:01.460284+0000","last_clean":"2024-05-10T11:15:01.460284+0000","last_became_active":"2024-05-10T11:14:35.320412+0000","last_became_peered":"2024-05-10T11:14:35.320412+0000","last_unstale":"2024-05-10T11:15:01.460284+0000","last_undegraded":"2024-05-10T11:15:01.460284+0000","last_fullsized":"2024-05-10T11:15:01.460284+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:55:52.585430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:32.810406+0000","last_change":"2024-05-10T11:14:22.257771+0000","last_active":"2024-05-10T11:15:32.810406+0000","last_peered":"2024-05-10T11:15:32.810406+0000","last_clean":"2024-05-10T11:15:32.810406+0000","last_became_active":"2024-05-10T11:14:22.257430+0000","last_became_peered":"2024-05-10T11:14:22.257430+0000","last_unstale":"2024-05-10T11:15:32.810406+0000","last_undegraded":"2024-05-10T11:15:32.810406+0000","last_fullsized":"2024-05-10T11:15:32.810406+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:23:50.131136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.203875+0000","last_change":"2024-05-10T11:14:25.272276+0000","last_active":"2024-05-10T11:17:00.203875+0000","last_peered":"2024-05-10T11:17:00.203875+0000","last_clean":"2024-05-10T11:17:00.203875+0000","last_became_active":"2024-05-10T11:14:25.271932+0000","last_became_peered":"2024-05-10T11:14:25.271932+0000","last_unstale":"2024-05-10T11:17:00.203875+0000","last_undegraded":"2024-05-10T11:17:00.203875+0000","last_fullsized":"2024-05-10T11:17:00.203875+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:49:44.479056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.463273+0000","last_change":"2024-05-10T11:15:01.462435+0000","last_active":"2024-05-10T11:15:01.463273+0000","last_peered":"2024-05-10T11:15:01.463273+0000","last_clean":"2024-05-10T11:15:01.463273+0000","last_became_active":"2024-05-10T11:15:01.461890+0000","last_became_peered":"2024-05-10T11:15:01.461890+0000","last_unstale":"2024-05-10T11:15:01.463273+0000","last_undegraded":"2024-05-10T11:15:01.463273+0000","last_fullsized":"2024-05-10T11:15:01.463273+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:01:13.350655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:28.789871+0000","last_change":"2024-05-10T11:14:22.256713+0000","last_active":"2024-05-10T11:15:28.789871+0000","last_peered":"2024-05-10T11:15:28.789871+0000","last_clean":"2024-05-10T11:15:28.789871+0000","last_became_active":"2024-05-10T11:14:22.255847+0000","last_became_peered":"2024-05-10T11:14:22.255847+0000","last_unstale":"2024-05-10T11:15:28.789871+0000","last_undegraded":"2024-05-10T11:15:28.789871+0000","last_fullsized":"2024-05-10T11:15:28.789871+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:00:54.605506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.214050+0000","last_change":"2024-05-10T11:14:25.282118+0000","last_active":"2024-05-10T11:17:00.214050+0000","last_peered":"2024-05-10T11:17:00.214050+0000","last_clean":"2024-05-10T11:17:00.214050+0000","last_became_active":"2024-05-10T11:14:25.280958+0000","last_became_peered":"2024-05-10T11:14:25.280958+0000","last_unstale":"2024-05-10T11:17:00.214050+0000","last_undegraded":"2024-05-10T11:17:00.214050+0000","last_fullsized":"2024-05-10T11:17:00.214050+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:02:10.815733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.422298+0000","last_change":"2024-05-10T11:14:35.329876+0000","last_active":"2024-05-10T11:15:01.422298+0000","last_peered":"2024-05-10T11:15:01.422298+0000","last_clean":"2024-05-10T11:15:01.422298+0000","last_became_active":"2024-05-10T11:14:35.328923+0000","last_became_peered":"2024-05-10T11:14:35.328923+0000","last_unstale":"2024-05-10T11:15:01.422298+0000","last_undegraded":"2024-05-10T11:15:01.422298+0000","last_fullsized":"2024-05-10T11:15:01.422298+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:04:51.356552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'5","reported_seq":151,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:13.970278+0000","last_change":"2024-05-10T11:14:22.254515+0000","last_active":"2024-05-10T11:17:13.970278+0000","last_peered":"2024-05-10T11:17:13.970278+0000","last_clean":"2024-05-10T11:17:13.970278+0000","last_became_active":"2024-05-10T11:14:22.253648+0000","last_became_peered":"2024-05-10T11:14:22.253648+0000","last_unstale":"2024-05-10T11:17:13.970278+0000","last_undegraded":"2024-05-10T11:17:13.970278+0000","last_fullsized":"2024-05-10T11:17:13.970278+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:10:22.144446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":108,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.210635+0000","last_change":"2024-05-10T11:14:25.278841+0000","last_active":"2024-05-10T11:17:00.210635+0000","last_peered":"2024-05-10T11:17:00.210635+0000","last_clean":"2024-05-10T11:17:00.210635+0000","last_became_active":"2024-05-10T11:14:25.278527+0000","last_became_peered":"2024-05-10T11:14:25.278527+0000","last_unstale":"2024-05-10T11:17:00.210635+0000","last_undegraded":"2024-05-10T11:17:00.210635+0000","last_fullsized":"2024-05-10T11:17:00.210635+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:01:55.902587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459763+0000","last_change":"2024-05-10T11:14:35.331334+0000","last_active":"2024-05-10T11:15:01.459763+0000","last_peered":"2024-05-10T11:15:01.459763+0000","last_clean":"2024-05-10T11:15:01.459763+0000","last_became_active":"2024-05-10T11:14:35.330940+0000","last_became_peered":"2024-05-10T11:14:35.330940+0000","last_unstale":"2024-05-10T11:15:01.459763+0000","last_undegraded":"2024-05-10T11:15:01.459763+0000","last_fullsized":"2024-05-10T11:15:01.459763+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:11:12.241551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":56,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:32.810680+0000","last_change":"2024-05-10T11:14:22.299917+0000","last_active":"2024-05-10T11:15:32.810680+0000","last_peered":"2024-05-10T11:15:32.810680+0000","last_clean":"2024-05-10T11:15:32.810680+0000","last_became_active":"2024-05-10T11:14:22.299212+0000","last_became_peered":"2024-05-10T11:14:22.299212+0000","last_unstale":"2024-05-10T11:15:32.810680+0000","last_undegraded":"2024-05-10T11:15:32.810680+0000","last_fullsized":"2024-05-10T11:15:32.810680+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:52:48.740339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.213948+0000","last_change":"2024-05-10T11:14:25.274693+0000","last_active":"2024-05-10T11:17:00.213948+0000","last_peered":"2024-05-10T11:17:00.213948+0000","last_clean":"2024-05-10T11:17:00.213948+0000","last_became_active":"2024-05-10T11:14:25.274378+0000","last_became_peered":"2024-05-10T11:14:25.274378+0000","last_unstale":"2024-05-10T11:17:00.213948+0000","last_undegraded":"2024-05-10T11:17:00.213948+0000","last_fullsized":"2024-05-10T11:17:00.213948+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:53:24.082742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:02.989636+0000","last_change":"2024-05-10T11:15:01.460541+0000","last_active":"2024-05-10T11:15:02.989636+0000","last_peered":"2024-05-10T11:15:02.989636+0000","last_clean":"2024-05-10T11:15:02.989636+0000","last_became_active":"2024-05-10T11:15:01.459954+0000","last_became_peered":"2024-05-10T11:15:01.459954+0000","last_unstale":"2024-05-10T11:15:02.989636+0000","last_undegraded":"2024-05-10T11:15:02.989636+0000","last_fullsized":"2024-05-10T11:15:02.989636+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:25:43.642441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,3,1],"acting":[8,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:32.810495+0000","last_change":"2024-05-10T11:14:22.258797+0000","last_active":"2024-05-10T11:15:32.810495+0000","last_peered":"2024-05-10T11:15:32.810495+0000","last_clean":"2024-05-10T11:15:32.810495+0000","last_became_active":"2024-05-10T11:14:22.258365+0000","last_became_peered":"2024-05-10T11:14:22.258365+0000","last_unstale":"2024-05-10T11:15:32.810495+0000","last_undegraded":"2024-05-10T11:15:32.810495+0000","last_fullsized":"2024-05-10T11:15:32.810495+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:58:53.291644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.198793+0000","last_change":"2024-05-10T11:15:01.460502+0000","last_active":"2024-05-10T11:17:00.198793+0000","last_peered":"2024-05-10T11:17:00.198793+0000","last_clean":"2024-05-10T11:17:00.198793+0000","last_became_active":"2024-05-10T11:15:01.460101+0000","last_became_peered":"2024-05-10T11:15:01.460101+0000","last_unstale":"2024-05-10T11:17:00.198793+0000","last_undegraded":"2024-05-10T11:17:00.198793+0000","last_fullsized":"2024-05-10T11:17:00.198793+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:22:39.735251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.429220+0000","last_change":"2024-05-10T11:14:35.419037+0000","last_active":"2024-05-10T11:15:01.429220+0000","last_peered":"2024-05-10T11:15:01.429220+0000","last_clean":"2024-05-10T11:15:01.429220+0000","last_became_active":"2024-05-10T11:14:35.418541+0000","last_became_peered":"2024-05-10T11:14:35.418541+0000","last_unstale":"2024-05-10T11:15:01.429220+0000","last_undegraded":"2024-05-10T11:15:01.429220+0000","last_fullsized":"2024-05-10T11:15:01.429220+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:05:49.560608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.422544+0000","last_change":"2024-05-10T11:14:22.299538+0000","last_active":"2024-05-10T11:15:01.422544+0000","last_peered":"2024-05-10T11:15:01.422544+0000","last_clean":"2024-05-10T11:15:01.422544+0000","last_became_active":"2024-05-10T11:14:22.299192+0000","last_became_peered":"2024-05-10T11:14:22.299192+0000","last_unstale":"2024-05-10T11:15:01.422544+0000","last_undegraded":"2024-05-10T11:15:01.422544+0000","last_fullsized":"2024-05-10T11:15:01.422544+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:28:43.041513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.193550+0000","last_change":"2024-05-10T11:14:25.272238+0000","last_active":"2024-05-10T11:17:00.193550+0000","last_peered":"2024-05-10T11:17:00.193550+0000","last_clean":"2024-05-10T11:17:00.193550+0000","last_became_active":"2024-05-10T11:14:25.271861+0000","last_became_peered":"2024-05-10T11:14:25.271861+0000","last_unstale":"2024-05-10T11:17:00.193550+0000","last_undegraded":"2024-05-10T11:17:00.193550+0000","last_fullsized":"2024-05-10T11:17:00.193550+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:00:08.821316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459614+0000","last_change":"2024-05-10T11:14:35.322239+0000","last_active":"2024-05-10T11:15:01.459614+0000","last_peered":"2024-05-10T11:15:01.459614+0000","last_clean":"2024-05-10T11:15:01.459614+0000","last_became_active":"2024-05-10T11:14:35.321864+0000","last_became_peered":"2024-05-10T11:14:35.321864+0000","last_unstale":"2024-05-10T11:15:01.459614+0000","last_undegraded":"2024-05-10T11:15:01.459614+0000","last_fullsized":"2024-05-10T11:15:01.459614+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:42:06.905879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:29.796685+0000","last_change":"2024-05-10T11:14:22.255449+0000","last_active":"2024-05-10T11:15:29.796685+0000","last_peered":"2024-05-10T11:15:29.796685+0000","last_clean":"2024-05-10T11:15:29.796685+0000","last_became_active":"2024-05-10T11:14:22.254966+0000","last_became_peered":"2024-05-10T11:14:22.254966+0000","last_unstale":"2024-05-10T11:15:29.796685+0000","last_undegraded":"2024-05-10T11:15:29.796685+0000","last_fullsized":"2024-05-10T11:15:29.796685+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:55:40.746115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.209423+0000","last_change":"2024-05-10T11:14:25.274050+0000","last_active":"2024-05-10T11:17:00.209423+0000","last_peered":"2024-05-10T11:17:00.209423+0000","last_clean":"2024-05-10T11:17:00.209423+0000","last_became_active":"2024-05-10T11:14:25.273720+0000","last_became_peered":"2024-05-10T11:14:25.273720+0000","last_unstale":"2024-05-10T11:17:00.209423+0000","last_undegraded":"2024-05-10T11:17:00.209423+0000","last_fullsized":"2024-05-10T11:17:00.209423+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:22:05.977673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459638+0000","last_change":"2024-05-10T11:14:35.418989+0000","last_active":"2024-05-10T11:15:01.459638+0000","last_peered":"2024-05-10T11:15:01.459638+0000","last_clean":"2024-05-10T11:15:01.459638+0000","last_became_active":"2024-05-10T11:14:35.418546+0000","last_became_peered":"2024-05-10T11:14:35.418546+0000","last_unstale":"2024-05-10T11:15:01.459638+0000","last_undegraded":"2024-05-10T11:15:01.459638+0000","last_fullsized":"2024-05-10T11:15:01.459638+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:23:37.875409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.210371+0000","last_change":"2024-05-10T11:14:25.272694+0000","last_active":"2024-05-10T11:17:00.210371+0000","last_peered":"2024-05-10T11:17:00.210371+0000","last_clean":"2024-05-10T11:17:00.210371+0000","last_became_active":"2024-05-10T11:14:25.272303+0000","last_became_peered":"2024-05-10T11:14:25.272303+0000","last_unstale":"2024-05-10T11:17:00.210371+0000","last_undegraded":"2024-05-10T11:17:00.210371+0000","last_fullsized":"2024-05-10T11:17:00.210371+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:51:39.039122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":144,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:10.965436+0000","last_change":"2024-05-10T11:14:22.299359+0000","last_active":"2024-05-10T11:17:10.965436+0000","last_peered":"2024-05-10T11:17:10.965436+0000","last_clean":"2024-05-10T11:17:10.965436+0000","last_became_active":"2024-05-10T11:14:22.299023+0000","last_became_peered":"2024-05-10T11:14:22.299023+0000","last_unstale":"2024-05-10T11:17:10.965436+0000","last_undegraded":"2024-05-10T11:17:10.965436+0000","last_fullsized":"2024-05-10T11:17:10.965436+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:53:06.301774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":100,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:41.913182+0000","last_change":"2024-05-10T11:14:35.322640+0000","last_active":"2024-05-10T11:15:41.913182+0000","last_peered":"2024-05-10T11:15:41.913182+0000","last_clean":"2024-05-10T11:15:41.913182+0000","last_became_active":"2024-05-10T11:14:35.322226+0000","last_became_peered":"2024-05-10T11:14:35.322226+0000","last_unstale":"2024-05-10T11:15:41.913182+0000","last_undegraded":"2024-05-10T11:15:41.913182+0000","last_fullsized":"2024-05-10T11:15:41.913182+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:34:11.574438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.206730+0000","last_change":"2024-05-10T11:14:25.276645+0000","last_active":"2024-05-10T11:17:00.206730+0000","last_peered":"2024-05-10T11:17:00.206730+0000","last_clean":"2024-05-10T11:17:00.206730+0000","last_became_active":"2024-05-10T11:14:25.276295+0000","last_became_peered":"2024-05-10T11:14:25.276295+0000","last_unstale":"2024-05-10T11:17:00.206730+0000","last_undegraded":"2024-05-10T11:17:00.206730+0000","last_fullsized":"2024-05-10T11:17:00.206730+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:52:06.715073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:32.810748+0000","last_change":"2024-05-10T11:14:22.252908+0000","last_active":"2024-05-10T11:15:32.810748+0000","last_peered":"2024-05-10T11:15:32.810748+0000","last_clean":"2024-05-10T11:15:32.810748+0000","last_became_active":"2024-05-10T11:14:22.252532+0000","last_became_peered":"2024-05-10T11:14:22.252532+0000","last_unstale":"2024-05-10T11:15:32.810748+0000","last_undegraded":"2024-05-10T11:15:32.810748+0000","last_fullsized":"2024-05-10T11:15:32.810748+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:41:59.651442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":113,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.426311+0000","last_change":"2024-05-10T11:12:53.090796+0000","last_active":"2024-05-10T11:15:01.426311+0000","last_peered":"2024-05-10T11:15:01.426311+0000","last_clean":"2024-05-10T11:15:01.426311+0000","last_became_active":"2024-05-10T11:12:53.086566+0000","last_became_peered":"2024-05-10T11:12:53.086566+0000","last_unstale":"2024-05-10T11:15:01.426311+0000","last_undegraded":"2024-05-10T11:15:01.426311+0000","last_fullsized":"2024-05-10T11:15:01.426311+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:09:27.097043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:09:27.097043+0000","last_clean_scrub_stamp":"2024-05-10T11:09:27.097043+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:42:17.759719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459957+0000","last_change":"2024-05-10T11:14:35.419970+0000","last_active":"2024-05-10T11:15:01.459957+0000","last_peered":"2024-05-10T11:15:01.459957+0000","last_clean":"2024-05-10T11:15:01.459957+0000","last_became_active":"2024-05-10T11:14:35.419516+0000","last_became_peered":"2024-05-10T11:14:35.419516+0000","last_unstale":"2024-05-10T11:15:01.459957+0000","last_undegraded":"2024-05-10T11:15:01.459957+0000","last_fullsized":"2024-05-10T11:15:01.459957+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:01:25.148155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.186817+0000","last_change":"2024-05-10T11:14:25.271478+0000","last_active":"2024-05-10T11:17:00.186817+0000","last_peered":"2024-05-10T11:17:00.186817+0000","last_clean":"2024-05-10T11:17:00.186817+0000","last_became_active":"2024-05-10T11:14:25.271143+0000","last_became_peered":"2024-05-10T11:14:25.271143+0000","last_unstale":"2024-05-10T11:17:00.186817+0000","last_undegraded":"2024-05-10T11:17:00.186817+0000","last_fullsized":"2024-05-10T11:17:00.186817+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:50:55.771659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:29.796678+0000","last_change":"2024-05-10T11:14:22.258704+0000","last_active":"2024-05-10T11:15:29.796678+0000","last_peered":"2024-05-10T11:15:29.796678+0000","last_clean":"2024-05-10T11:15:29.796678+0000","last_became_active":"2024-05-10T11:14:22.258304+0000","last_became_peered":"2024-05-10T11:14:22.258304+0000","last_unstale":"2024-05-10T11:15:29.796678+0000","last_undegraded":"2024-05-10T11:15:29.796678+0000","last_fullsized":"2024-05-10T11:15:29.796678+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:02:41.451394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.425481+0000","last_change":"2024-05-10T11:14:35.325151+0000","last_active":"2024-05-10T11:15:01.425481+0000","last_peered":"2024-05-10T11:15:01.425481+0000","last_clean":"2024-05-10T11:15:01.425481+0000","last_became_active":"2024-05-10T11:14:35.324680+0000","last_became_peered":"2024-05-10T11:14:35.324680+0000","last_unstale":"2024-05-10T11:15:01.425481+0000","last_undegraded":"2024-05-10T11:15:01.425481+0000","last_fullsized":"2024-05-10T11:15:01.425481+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:12:17.578364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:32.810795+0000","last_change":"2024-05-10T11:15:01.428210+0000","last_active":"2024-05-10T11:15:32.810795+0000","last_peered":"2024-05-10T11:15:32.810795+0000","last_clean":"2024-05-10T11:15:32.810795+0000","last_became_active":"2024-05-10T11:15:01.427671+0000","last_became_peered":"2024-05-10T11:15:01.427671+0000","last_unstale":"2024-05-10T11:15:32.810795+0000","last_undegraded":"2024-05-10T11:15:32.810795+0000","last_fullsized":"2024-05-10T11:15:32.810795+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:44:51.312743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.194143+0000","last_change":"2024-05-10T11:14:25.271350+0000","last_active":"2024-05-10T11:17:00.194143+0000","last_peered":"2024-05-10T11:17:00.194143+0000","last_clean":"2024-05-10T11:17:00.194143+0000","last_became_active":"2024-05-10T11:14:25.271047+0000","last_became_peered":"2024-05-10T11:14:25.271047+0000","last_unstale":"2024-05-10T11:17:00.194143+0000","last_undegraded":"2024-05-10T11:17:00.194143+0000","last_fullsized":"2024-05-10T11:17:00.194143+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:44:21.074007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.458379+0000","last_change":"2024-05-10T11:14:35.418910+0000","last_active":"2024-05-10T11:15:01.458379+0000","last_peered":"2024-05-10T11:15:01.458379+0000","last_clean":"2024-05-10T11:15:01.458379+0000","last_became_active":"2024-05-10T11:14:35.418460+0000","last_became_peered":"2024-05-10T11:14:35.418460+0000","last_unstale":"2024-05-10T11:15:01.458379+0000","last_undegraded":"2024-05-10T11:15:01.458379+0000","last_fullsized":"2024-05-10T11:15:01.458379+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:36:04.057776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'12","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:16:35.480421+0000","last_change":"2024-05-10T11:14:22.253673+0000","last_active":"2024-05-10T11:16:35.480421+0000","last_peered":"2024-05-10T11:16:35.480421+0000","last_clean":"2024-05-10T11:16:35.480421+0000","last_became_active":"2024-05-10T11:14:22.253304+0000","last_became_peered":"2024-05-10T11:14:22.253304+0000","last_unstale":"2024-05-10T11:16:35.480421+0000","last_undegraded":"2024-05-10T11:16:35.480421+0000","last_fullsized":"2024-05-10T11:16:35.480421+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:11:37.108621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.211961+0000","last_change":"2024-05-10T11:14:25.280590+0000","last_active":"2024-05-10T11:17:00.211961+0000","last_peered":"2024-05-10T11:17:00.211961+0000","last_clean":"2024-05-10T11:17:00.211961+0000","last_became_active":"2024-05-10T11:14:25.279990+0000","last_became_peered":"2024-05-10T11:14:25.279990+0000","last_unstale":"2024-05-10T11:17:00.211961+0000","last_undegraded":"2024-05-10T11:17:00.211961+0000","last_fullsized":"2024-05-10T11:17:00.211961+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:46:55.394767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.424307+0000","last_change":"2024-05-10T11:14:35.320931+0000","last_active":"2024-05-10T11:15:01.424307+0000","last_peered":"2024-05-10T11:15:01.424307+0000","last_clean":"2024-05-10T11:15:01.424307+0000","last_became_active":"2024-05-10T11:14:35.320523+0000","last_became_peered":"2024-05-10T11:14:35.320523+0000","last_unstale":"2024-05-10T11:15:01.424307+0000","last_undegraded":"2024-05-10T11:15:01.424307+0000","last_fullsized":"2024-05-10T11:15:01.424307+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:50:37.533397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:28.788626+0000","last_change":"2024-05-10T11:14:22.255115+0000","last_active":"2024-05-10T11:15:28.788626+0000","last_peered":"2024-05-10T11:15:28.788626+0000","last_clean":"2024-05-10T11:15:28.788626+0000","last_became_active":"2024-05-10T11:14:22.254732+0000","last_became_peered":"2024-05-10T11:14:22.254732+0000","last_unstale":"2024-05-10T11:15:28.788626+0000","last_undegraded":"2024-05-10T11:15:28.788626+0000","last_fullsized":"2024-05-10T11:15:28.788626+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:49:54.807312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.206879+0000","last_change":"2024-05-10T11:14:25.270766+0000","last_active":"2024-05-10T11:17:00.206879+0000","last_peered":"2024-05-10T11:17:00.206879+0000","last_clean":"2024-05-10T11:17:00.206879+0000","last_became_active":"2024-05-10T11:14:25.268958+0000","last_became_peered":"2024-05-10T11:14:25.268958+0000","last_unstale":"2024-05-10T11:17:00.206879+0000","last_undegraded":"2024-05-10T11:17:00.206879+0000","last_fullsized":"2024-05-10T11:17:00.206879+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:21:28.421478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459244+0000","last_change":"2024-05-10T11:14:35.420030+0000","last_active":"2024-05-10T11:15:01.459244+0000","last_peered":"2024-05-10T11:15:01.459244+0000","last_clean":"2024-05-10T11:15:01.459244+0000","last_became_active":"2024-05-10T11:14:35.419475+0000","last_became_peered":"2024-05-10T11:14:35.419475+0000","last_unstale":"2024-05-10T11:15:01.459244+0000","last_undegraded":"2024-05-10T11:15:01.459244+0000","last_fullsized":"2024-05-10T11:15:01.459244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:13:51.207146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:28.789346+0000","last_change":"2024-05-10T11:15:01.429683+0000","last_active":"2024-05-10T11:15:28.789346+0000","last_peered":"2024-05-10T11:15:28.789346+0000","last_clean":"2024-05-10T11:15:28.789346+0000","last_became_active":"2024-05-10T11:15:01.429162+0000","last_became_peered":"2024-05-10T11:15:01.429162+0000","last_unstale":"2024-05-10T11:15:28.789346+0000","last_undegraded":"2024-05-10T11:15:28.789346+0000","last_fullsized":"2024-05-10T11:15:28.789346+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:57:35.166686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.167290+0000","last_change":"2024-05-10T11:14:25.282381+0000","last_active":"2024-05-10T11:17:00.167290+0000","last_peered":"2024-05-10T11:17:00.167290+0000","last_clean":"2024-05-10T11:17:00.167290+0000","last_became_active":"2024-05-10T11:14:25.281990+0000","last_became_peered":"2024-05-10T11:14:25.281990+0000","last_unstale":"2024-05-10T11:17:00.167290+0000","last_undegraded":"2024-05-10T11:17:00.167290+0000","last_fullsized":"2024-05-10T11:17:00.167290+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:39:03.726547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.422393+0000","last_change":"2024-05-10T11:14:35.328242+0000","last_active":"2024-05-10T11:15:01.422393+0000","last_peered":"2024-05-10T11:15:01.422393+0000","last_clean":"2024-05-10T11:15:01.422393+0000","last_became_active":"2024-05-10T11:14:35.327877+0000","last_became_peered":"2024-05-10T11:14:35.327877+0000","last_unstale":"2024-05-10T11:15:01.422393+0000","last_undegraded":"2024-05-10T11:15:01.422393+0000","last_fullsized":"2024-05-10T11:15:01.422393+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:01:57.826334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:28.789342+0000","last_change":"2024-05-10T11:14:22.251057+0000","last_active":"2024-05-10T11:15:28.789342+0000","last_peered":"2024-05-10T11:15:28.789342+0000","last_clean":"2024-05-10T11:15:28.789342+0000","last_became_active":"2024-05-10T11:14:22.250696+0000","last_became_peered":"2024-05-10T11:14:22.250696+0000","last_unstale":"2024-05-10T11:15:28.789342+0000","last_undegraded":"2024-05-10T11:15:28.789342+0000","last_fullsized":"2024-05-10T11:15:28.789342+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:38:53.780643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.206311+0000","last_change":"2024-05-10T11:14:25.283806+0000","last_active":"2024-05-10T11:17:00.206311+0000","last_peered":"2024-05-10T11:17:00.206311+0000","last_clean":"2024-05-10T11:17:00.206311+0000","last_became_active":"2024-05-10T11:14:25.282714+0000","last_became_peered":"2024-05-10T11:14:25.282714+0000","last_unstale":"2024-05-10T11:17:00.206311+0000","last_undegraded":"2024-05-10T11:17:00.206311+0000","last_fullsized":"2024-05-10T11:17:00.206311+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:05:11.940408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:48.003154+0000","last_change":"2024-05-10T11:14:35.329986+0000","last_active":"2024-05-10T11:15:48.003154+0000","last_peered":"2024-05-10T11:15:48.003154+0000","last_clean":"2024-05-10T11:15:48.003154+0000","last_became_active":"2024-05-10T11:14:35.329477+0000","last_became_peered":"2024-05-10T11:14:35.329477+0000","last_unstale":"2024-05-10T11:15:48.003154+0000","last_undegraded":"2024-05-10T11:15:48.003154+0000","last_fullsized":"2024-05-10T11:15:48.003154+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:03:19.203580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:30.784768+0000","last_change":"2024-05-10T11:14:22.257627+0000","last_active":"2024-05-10T11:15:30.784768+0000","last_peered":"2024-05-10T11:15:30.784768+0000","last_clean":"2024-05-10T11:15:30.784768+0000","last_became_active":"2024-05-10T11:14:22.256990+0000","last_became_peered":"2024-05-10T11:14:22.256990+0000","last_unstale":"2024-05-10T11:15:30.784768+0000","last_undegraded":"2024-05-10T11:15:30.784768+0000","last_fullsized":"2024-05-10T11:15:30.784768+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:28:18.926821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.203131+0000","last_change":"2024-05-10T11:14:25.274799+0000","last_active":"2024-05-10T11:17:00.203131+0000","last_peered":"2024-05-10T11:17:00.203131+0000","last_clean":"2024-05-10T11:17:00.203131+0000","last_became_active":"2024-05-10T11:14:25.274443+0000","last_became_peered":"2024-05-10T11:14:25.274443+0000","last_unstale":"2024-05-10T11:17:00.203131+0000","last_undegraded":"2024-05-10T11:17:00.203131+0000","last_fullsized":"2024-05-10T11:17:00.203131+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:25:23.399798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459376+0000","last_change":"2024-05-10T11:14:35.418581+0000","last_active":"2024-05-10T11:15:01.459376+0000","last_peered":"2024-05-10T11:15:01.459376+0000","last_clean":"2024-05-10T11:15:01.459376+0000","last_became_active":"2024-05-10T11:14:35.418160+0000","last_became_peered":"2024-05-10T11:14:35.418160+0000","last_unstale":"2024-05-10T11:15:01.459376+0000","last_undegraded":"2024-05-10T11:15:01.459376+0000","last_fullsized":"2024-05-10T11:15:01.459376+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:39:01.873626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:32.810420+0000","last_change":"2024-05-10T11:14:22.260325+0000","last_active":"2024-05-10T11:15:32.810420+0000","last_peered":"2024-05-10T11:15:32.810420+0000","last_clean":"2024-05-10T11:15:32.810420+0000","last_became_active":"2024-05-10T11:14:22.259971+0000","last_became_peered":"2024-05-10T11:14:22.259971+0000","last_unstale":"2024-05-10T11:15:32.810420+0000","last_undegraded":"2024-05-10T11:15:32.810420+0000","last_fullsized":"2024-05-10T11:15:32.810420+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:34:55.547811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.214078+0000","last_change":"2024-05-10T11:14:25.279009+0000","last_active":"2024-05-10T11:17:00.214078+0000","last_peered":"2024-05-10T11:17:00.214078+0000","last_clean":"2024-05-10T11:17:00.214078+0000","last_became_active":"2024-05-10T11:14:25.278443+0000","last_became_peered":"2024-05-10T11:14:25.278443+0000","last_unstale":"2024-05-10T11:17:00.214078+0000","last_undegraded":"2024-05-10T11:17:00.214078+0000","last_fullsized":"2024-05-10T11:17:00.214078+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:25:22.306555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.426518+0000","last_change":"2024-05-10T11:14:35.319405+0000","last_active":"2024-05-10T11:15:01.426518+0000","last_peered":"2024-05-10T11:15:01.426518+0000","last_clean":"2024-05-10T11:15:01.426518+0000","last_became_active":"2024-05-10T11:14:35.318865+0000","last_became_peered":"2024-05-10T11:14:35.318865+0000","last_unstale":"2024-05-10T11:15:01.426518+0000","last_undegraded":"2024-05-10T11:15:01.426518+0000","last_fullsized":"2024-05-10T11:15:01.426518+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:22:41.592141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:32.813833+0000","last_change":"2024-05-10T11:14:22.299243+0000","last_active":"2024-05-10T11:15:32.813833+0000","last_peered":"2024-05-10T11:15:32.813833+0000","last_clean":"2024-05-10T11:15:32.813833+0000","last_became_active":"2024-05-10T11:14:22.298868+0000","last_became_peered":"2024-05-10T11:14:22.298868+0000","last_unstale":"2024-05-10T11:15:32.813833+0000","last_undegraded":"2024-05-10T11:15:32.813833+0000","last_fullsized":"2024-05-10T11:15:32.813833+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:54:15.078141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.213861+0000","last_change":"2024-05-10T11:14:25.281198+0000","last_active":"2024-05-10T11:17:00.213861+0000","last_peered":"2024-05-10T11:17:00.213861+0000","last_clean":"2024-05-10T11:17:00.213861+0000","last_became_active":"2024-05-10T11:14:25.280871+0000","last_became_peered":"2024-05-10T11:14:25.280871+0000","last_unstale":"2024-05-10T11:17:00.213861+0000","last_undegraded":"2024-05-10T11:17:00.213861+0000","last_fullsized":"2024-05-10T11:17:00.213861+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:24:55.843054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.425771+0000","last_change":"2024-05-10T11:14:35.329517+0000","last_active":"2024-05-10T11:15:01.425771+0000","last_peered":"2024-05-10T11:15:01.425771+0000","last_clean":"2024-05-10T11:15:01.425771+0000","last_became_active":"2024-05-10T11:14:35.329056+0000","last_became_peered":"2024-05-10T11:14:35.329056+0000","last_unstale":"2024-05-10T11:15:01.425771+0000","last_undegraded":"2024-05-10T11:15:01.425771+0000","last_fullsized":"2024-05-10T11:15:01.425771+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:09:24.844256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:32.810561+0000","last_change":"2024-05-10T11:14:22.257206+0000","last_active":"2024-05-10T11:15:32.810561+0000","last_peered":"2024-05-10T11:15:32.810561+0000","last_clean":"2024-05-10T11:15:32.810561+0000","last_became_active":"2024-05-10T11:14:22.256850+0000","last_became_peered":"2024-05-10T11:14:22.256850+0000","last_unstale":"2024-05-10T11:15:32.810561+0000","last_undegraded":"2024-05-10T11:15:32.810561+0000","last_fullsized":"2024-05-10T11:15:32.810561+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:34:31.973034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.211163+0000","last_change":"2024-05-10T11:15:01.461372+0000","last_active":"2024-05-10T11:17:00.211163+0000","last_peered":"2024-05-10T11:17:00.211163+0000","last_clean":"2024-05-10T11:17:00.211163+0000","last_became_active":"2024-05-10T11:15:01.460978+0000","last_became_peered":"2024-05-10T11:15:01.460978+0000","last_unstale":"2024-05-10T11:17:00.211163+0000","last_undegraded":"2024-05-10T11:17:00.211163+0000","last_fullsized":"2024-05-10T11:17:00.211163+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:18:45.750486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459061+0000","last_change":"2024-05-10T11:14:35.322387+0000","last_active":"2024-05-10T11:15:01.459061+0000","last_peered":"2024-05-10T11:15:01.459061+0000","last_clean":"2024-05-10T11:15:01.459061+0000","last_became_active":"2024-05-10T11:14:35.321939+0000","last_became_peered":"2024-05-10T11:14:35.321939+0000","last_unstale":"2024-05-10T11:15:01.459061+0000","last_undegraded":"2024-05-10T11:15:01.459061+0000","last_fullsized":"2024-05-10T11:15:01.459061+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:28:29.085363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459652+0000","last_change":"2024-05-10T11:14:22.258603+0000","last_active":"2024-05-10T11:15:01.459652+0000","last_peered":"2024-05-10T11:15:01.459652+0000","last_clean":"2024-05-10T11:15:01.459652+0000","last_became_active":"2024-05-10T11:14:22.258269+0000","last_became_peered":"2024-05-10T11:14:22.258269+0000","last_unstale":"2024-05-10T11:15:01.459652+0000","last_undegraded":"2024-05-10T11:15:01.459652+0000","last_fullsized":"2024-05-10T11:15:01.459652+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:30:04.240962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.207686+0000","last_change":"2024-05-10T11:14:25.281959+0000","last_active":"2024-05-10T11:17:00.207686+0000","last_peered":"2024-05-10T11:17:00.207686+0000","last_clean":"2024-05-10T11:17:00.207686+0000","last_became_active":"2024-05-10T11:14:25.281648+0000","last_became_peered":"2024-05-10T11:14:25.281648+0000","last_unstale":"2024-05-10T11:17:00.207686+0000","last_undegraded":"2024-05-10T11:17:00.207686+0000","last_fullsized":"2024-05-10T11:17:00.207686+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:24:01.569685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.195950+0000","last_change":"2024-05-10T11:14:25.284892+0000","last_active":"2024-05-10T11:17:00.195950+0000","last_peered":"2024-05-10T11:17:00.195950+0000","last_clean":"2024-05-10T11:17:00.195950+0000","last_became_active":"2024-05-10T11:14:25.284078+0000","last_became_peered":"2024-05-10T11:14:25.284078+0000","last_unstale":"2024-05-10T11:17:00.195950+0000","last_undegraded":"2024-05-10T11:17:00.195950+0000","last_fullsized":"2024-05-10T11:17:00.195950+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:49:43.348463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.211265+0000","last_change":"2024-05-10T11:14:25.274739+0000","last_active":"2024-05-10T11:17:00.211265+0000","last_peered":"2024-05-10T11:17:00.211265+0000","last_clean":"2024-05-10T11:17:00.211265+0000","last_became_active":"2024-05-10T11:14:25.274310+0000","last_became_peered":"2024-05-10T11:14:25.274310+0000","last_unstale":"2024-05-10T11:17:00.211265+0000","last_undegraded":"2024-05-10T11:17:00.211265+0000","last_fullsized":"2024-05-10T11:17:00.211265+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:21:16.935189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.204984+0000","last_change":"2024-05-10T11:14:25.272484+0000","last_active":"2024-05-10T11:17:00.204984+0000","last_peered":"2024-05-10T11:17:00.204984+0000","last_clean":"2024-05-10T11:17:00.204984+0000","last_became_active":"2024-05-10T11:14:25.272114+0000","last_became_peered":"2024-05-10T11:14:25.272114+0000","last_unstale":"2024-05-10T11:17:00.204984+0000","last_undegraded":"2024-05-10T11:17:00.204984+0000","last_fullsized":"2024-05-10T11:17:00.204984+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:46:34.537959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.208365+0000","last_change":"2024-05-10T11:15:01.460763+0000","last_active":"2024-05-10T11:17:00.208365+0000","last_peered":"2024-05-10T11:17:00.208365+0000","last_clean":"2024-05-10T11:17:00.208365+0000","last_became_active":"2024-05-10T11:15:01.460392+0000","last_became_peered":"2024-05-10T11:15:01.460392+0000","last_unstale":"2024-05-10T11:17:00.208365+0000","last_undegraded":"2024-05-10T11:17:00.208365+0000","last_fullsized":"2024-05-10T11:17:00.208365+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:15:42.859813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.202714+0000","last_change":"2024-05-10T11:14:25.269640+0000","last_active":"2024-05-10T11:17:00.202714+0000","last_peered":"2024-05-10T11:17:00.202714+0000","last_clean":"2024-05-10T11:17:00.202714+0000","last_became_active":"2024-05-10T11:14:25.269348+0000","last_became_peered":"2024-05-10T11:14:25.269348+0000","last_unstale":"2024-05-10T11:17:00.202714+0000","last_undegraded":"2024-05-10T11:17:00.202714+0000","last_fullsized":"2024-05-10T11:17:00.202714+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:40:25.276433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.194352+0000","last_change":"2024-05-10T11:14:25.282563+0000","last_active":"2024-05-10T11:17:00.194352+0000","last_peered":"2024-05-10T11:17:00.194352+0000","last_clean":"2024-05-10T11:17:00.194352+0000","last_became_active":"2024-05-10T11:14:25.282145+0000","last_became_peered":"2024-05-10T11:14:25.282145+0000","last_unstale":"2024-05-10T11:17:00.194352+0000","last_undegraded":"2024-05-10T11:17:00.194352+0000","last_fullsized":"2024-05-10T11:17:00.194352+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:54:38.436669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.202523+0000","last_change":"2024-05-10T11:14:25.276753+0000","last_active":"2024-05-10T11:17:00.202523+0000","last_peered":"2024-05-10T11:17:00.202523+0000","last_clean":"2024-05-10T11:17:00.202523+0000","last_became_active":"2024-05-10T11:14:25.276342+0000","last_became_peered":"2024-05-10T11:14:25.276342+0000","last_unstale":"2024-05-10T11:17:00.202523+0000","last_undegraded":"2024-05-10T11:17:00.202523+0000","last_fullsized":"2024-05-10T11:17:00.202523+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:31:42.766069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.209177+0000","last_change":"2024-05-10T11:14:25.275905+0000","last_active":"2024-05-10T11:17:00.209177+0000","last_peered":"2024-05-10T11:17:00.209177+0000","last_clean":"2024-05-10T11:17:00.209177+0000","last_became_active":"2024-05-10T11:14:25.275422+0000","last_became_peered":"2024-05-10T11:14:25.275422+0000","last_unstale":"2024-05-10T11:17:00.209177+0000","last_undegraded":"2024-05-10T11:17:00.209177+0000","last_fullsized":"2024-05-10T11:17:00.209177+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:39:24.033519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.205191+0000","last_change":"2024-05-10T11:14:25.271602+0000","last_active":"2024-05-10T11:17:00.205191+0000","last_peered":"2024-05-10T11:17:00.205191+0000","last_clean":"2024-05-10T11:17:00.205191+0000","last_became_active":"2024-05-10T11:14:25.267206+0000","last_became_peered":"2024-05-10T11:14:25.267206+0000","last_unstale":"2024-05-10T11:17:00.205191+0000","last_undegraded":"2024-05-10T11:17:00.205191+0000","last_fullsized":"2024-05-10T11:17:00.205191+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:28:24.708297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.206462+0000","last_change":"2024-05-10T11:14:25.280378+0000","last_active":"2024-05-10T11:17:00.206462+0000","last_peered":"2024-05-10T11:17:00.206462+0000","last_clean":"2024-05-10T11:17:00.206462+0000","last_became_active":"2024-05-10T11:14:25.280079+0000","last_became_peered":"2024-05-10T11:14:25.280079+0000","last_unstale":"2024-05-10T11:17:00.206462+0000","last_undegraded":"2024-05-10T11:17:00.206462+0000","last_fullsized":"2024-05-10T11:17:00.206462+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:13:47.781060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.204726+0000","last_change":"2024-05-10T11:14:25.272363+0000","last_active":"2024-05-10T11:17:00.204726+0000","last_peered":"2024-05-10T11:17:00.204726+0000","last_clean":"2024-05-10T11:17:00.204726+0000","last_became_active":"2024-05-10T11:14:25.271847+0000","last_became_peered":"2024-05-10T11:14:25.271847+0000","last_unstale":"2024-05-10T11:17:00.204726+0000","last_undegraded":"2024-05-10T11:17:00.204726+0000","last_fullsized":"2024-05-10T11:17:00.204726+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:47:54.661069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.203714+0000","last_change":"2024-05-10T11:14:25.272688+0000","last_active":"2024-05-10T11:17:00.203714+0000","last_peered":"2024-05-10T11:17:00.203714+0000","last_clean":"2024-05-10T11:17:00.203714+0000","last_became_active":"2024-05-10T11:14:25.271654+0000","last_became_peered":"2024-05-10T11:14:25.271654+0000","last_unstale":"2024-05-10T11:17:00.203714+0000","last_undegraded":"2024-05-10T11:17:00.203714+0000","last_fullsized":"2024-05-10T11:17:00.203714+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:50:21.149707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.199436+0000","last_change":"2024-05-10T11:15:01.432279+0000","last_active":"2024-05-10T11:17:00.199436+0000","last_peered":"2024-05-10T11:17:00.199436+0000","last_clean":"2024-05-10T11:17:00.199436+0000","last_became_active":"2024-05-10T11:15:01.431751+0000","last_became_peered":"2024-05-10T11:15:01.431751+0000","last_unstale":"2024-05-10T11:17:00.199436+0000","last_undegraded":"2024-05-10T11:17:00.199436+0000","last_fullsized":"2024-05-10T11:17:00.199436+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:25:44.196618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69059,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":634,"num_read_kb":388,"num_write":179,"num_write_kb":127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":207177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":153,"ondisk_log_size":153,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645363,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1015808,"data_stored":505095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841591,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":41160,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93703992,"statfs":{"total":95995035648,"available":95952887808,"internally_reserved":0,"allocated":1703936,"data_stored":1157258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":40434113},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005118,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1032192,"data_stored":509405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201346,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1024000,"data_stored":505207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430280,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28600,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716552,"statfs":{"total":95995035648,"available":95965749248,"internally_reserved":0,"allocated":1622016,"data_stored":1101811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593805,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":500485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574949,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1077248,"data_stored":564780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738475,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1032192,"data_stored":501873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411426,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36412,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708740,"statfs":{"total":95995035648,"available":95957749760,"internally_reserved":0,"allocated":1101824,"data_stored":566662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149788,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1019904,"data_stored":500679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953560,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1019904,"data_stored":505667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790034,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28608,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716544,"statfs":{"total":95995035648,"available":95965741056,"internally_reserved":0,"allocated":1630208,"data_stored":1096867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":66267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T11:17:17.946 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-10T11:17:17.947 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-10T11:17:17.947 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-05-10T11:17:17.947 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph health --format=json 2024-05-10T11:17:18.187 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:17:18.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:17:18 smithi100 ceph-mon[30296]: from='client.14997 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T11:17:18.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:17:18 smithi178 ceph-mon[33359]: from='client.14997 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T11:17:18.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:17:18 smithi188 ceph-mon[34075]: from='client.14997 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T11:17:19.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:17:19 smithi100 ceph-mon[30296]: pgmap v388: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T11:17:19.754 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:17:19 smithi178 ceph-mon[33359]: pgmap v388: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T11:17:19.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:17:19 smithi188 ceph-mon[34075]: pgmap v388: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T11:17:19.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:19.952+0000 7f507a8ad700 1 -- 172.21.15.100:0/897816264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5074103ef0 msgr2=0x7f50741103a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:19.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:19.952+0000 7f507a8ad700 1 --2- 172.21.15.100:0/897816264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5074103ef0 0x7f50741103a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f506c00b0d0 tx=0x7f506c00b3e0 comp rx=0 tx=0).stop 2024-05-10T11:17:19.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:19.952+0000 7f507a8ad700 1 -- 172.21.15.100:0/897816264 shutdown_connections 2024-05-10T11:17:19.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:19.952+0000 7f507a8ad700 1 --2- 172.21.15.100:0/897816264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5074103ef0 0x7f50741103a0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:19.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:19.952+0000 7f507a8ad700 1 --2- 172.21.15.100:0/897816264 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50741095d0 0x7f50741099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:19.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:19.952+0000 7f507a8ad700 1 --2- 172.21.15.100:0/897816264 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5074103530 0x7f50741039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:19.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:19.952+0000 7f507a8ad700 1 -- 172.21.15.100:0/897816264 >> 172.21.15.100:0/897816264 conn(0x7f507407a690 msgr2=0x7f507407aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:17:19.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:19.952+0000 7f507a8ad700 1 -- 172.21.15.100:0/897816264 shutdown_connections 2024-05-10T11:17:19.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:19.952+0000 7f507a8ad700 1 -- 172.21.15.100:0/897816264 wait complete. 2024-05-10T11:17:19.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:19.953+0000 7f507a8ad700 1 Processor -- start 2024-05-10T11:17:19.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:19.953+0000 7f507a8ad700 1 -- start start 2024-05-10T11:17:19.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:19.953+0000 7f507a8ad700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5074103530 0x7f50741a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:19.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:19.953+0000 7f507a8ad700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5074103ef0 0x7f50741a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:19.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:19.953+0000 7f507a8ad700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50741095d0 0x7f507419baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:19.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:19.953+0000 7f507a8ad700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5074113660 con 0x7f5074103530 2024-05-10T11:17:19.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:19.953+0000 7f507a8ad700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f50741134e0 con 0x7f5074103ef0 2024-05-10T11:17:19.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:19.953+0000 7f507a8ad700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f50741137e0 con 0x7f50741095d0 2024-05-10T11:17:19.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:19.953+0000 7f5073fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5074103530 0x7f50741a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:19.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:19.953+0000 7f5073fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5074103530 0x7f50741a1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58062/0 (socket says 172.21.15.100:58062) 2024-05-10T11:17:19.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:19.953+0000 7f5073fff700 1 -- 172.21.15.100:0/305431373 learned_addr learned my addr 172.21.15.100:0/305431373 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:17:19.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:19.953+0000 7f5078e4a700 1 --2- 172.21.15.100:0/305431373 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50741095d0 0x7f507419baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:19.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:19.954+0000 7f5073fff700 1 -- 172.21.15.100:0/305431373 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50741095d0 msgr2=0x7f507419baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:19.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:19.954+0000 7f5073fff700 1 --2- 172.21.15.100:0/305431373 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50741095d0 0x7f507419baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:19.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:19.954+0000 7f5073fff700 1 -- 172.21.15.100:0/305431373 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5074103ef0 msgr2=0x7f50741a1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:19.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:19.954+0000 7f506bfff700 1 --2- 172.21.15.100:0/305431373 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5074103ef0 0x7f50741a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:19.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:19.954+0000 7f5073fff700 1 --2- 172.21.15.100:0/305431373 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5074103ef0 0x7f50741a1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:19.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:19.954+0000 7f5073fff700 1 -- 172.21.15.100:0/305431373 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f507419c250 con 0x7f5074103530 2024-05-10T11:17:19.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:19.954+0000 7f506bfff700 1 --2- 172.21.15.100:0/305431373 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5074103ef0 0x7f50741a1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:17:19.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:19.954+0000 7f5073fff700 1 --2- 172.21.15.100:0/305431373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5074103530 0x7f50741a1930 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f506400b530 tx=0x7f506400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:17:19.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:19.954+0000 7f5071ffb700 1 -- 172.21.15.100:0/305431373 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5064015020 con 0x7f5074103530 2024-05-10T11:17:19.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:19.954+0000 7f507a8ad700 1 -- 172.21.15.100:0/305431373 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f507419c540 con 0x7f5074103530 2024-05-10T11:17:19.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:19.954+0000 7f5071ffb700 1 -- 172.21.15.100:0/305431373 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5064011cc0 con 0x7f5074103530 2024-05-10T11:17:19.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:19.954+0000 7f5071ffb700 1 -- 172.21.15.100:0/305431373 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5064010d30 con 0x7f5074103530 2024-05-10T11:17:19.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:19.954+0000 7f507a8ad700 1 -- 172.21.15.100:0/305431373 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50741a8800 con 0x7f5074103530 2024-05-10T11:17:19.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:19.955+0000 7f507a8ad700 1 -- 172.21.15.100:0/305431373 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f507406a190 con 0x7f5074103530 2024-05-10T11:17:19.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:19.957+0000 7f5071ffb700 1 -- 172.21.15.100:0/305431373 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f5064018020 con 0x7f5074103530 2024-05-10T11:17:19.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:19.958+0000 7f5071ffb700 1 --2- 172.21.15.100:0/305431373 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f50600b2880 0x7f50600b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:19.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:19.958+0000 7f506bfff700 1 --2- 172.21.15.100:0/305431373 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f50600b2880 0x7f50600b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:19.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:19.958+0000 7f5071ffb700 1 -- 172.21.15.100:0/305431373 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f50640e2340 con 0x7f5074103530 2024-05-10T11:17:19.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:19.958+0000 7f506bfff700 1 --2- 172.21.15.100:0/305431373 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f50600b2880 0x7f50600b4d40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f507407af20 tx=0x7f505c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:17:19.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:19.961+0000 7f5071ffb700 1 -- 172.21.15.100:0/305431373 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f506401f020 con 0x7f5074103530 2024-05-10T11:17:20.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:20.211+0000 7f507a8ad700 1 -- 172.21.15.100:0/305431373 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f507402e510 con 0x7f5074103530 2024-05-10T11:17:20.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:20.212+0000 7f5071ffb700 1 -- 172.21.15.100:0/305431373 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f50640a9da0 con 0x7f5074103530 2024-05-10T11:17:20.212 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:17:20.212 INFO:teuthology.orchestra.run.smithi100.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-10T11:17:20.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:20.214+0000 7f507a8ad700 1 -- 172.21.15.100:0/305431373 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f50600b2880 msgr2=0x7f50600b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:20.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:20.214+0000 7f507a8ad700 1 --2- 172.21.15.100:0/305431373 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f50600b2880 0x7f50600b4d40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f507407af20 tx=0x7f505c008040 comp rx=0 tx=0).stop 2024-05-10T11:17:20.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:20.214+0000 7f507a8ad700 1 -- 172.21.15.100:0/305431373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5074103530 msgr2=0x7f50741a1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:20.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:20.214+0000 7f507a8ad700 1 --2- 172.21.15.100:0/305431373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5074103530 0x7f50741a1930 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f506400b530 tx=0x7f506400b8f0 comp rx=0 tx=0).stop 2024-05-10T11:17:20.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:20.214+0000 7f507a8ad700 1 -- 172.21.15.100:0/305431373 shutdown_connections 2024-05-10T11:17:20.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:20.214+0000 7f507a8ad700 1 --2- 172.21.15.100:0/305431373 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f50600b2880 0x7f50600b4d40 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:20.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:20.214+0000 7f507a8ad700 1 --2- 172.21.15.100:0/305431373 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50741095d0 0x7f507419baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:20.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:20.214+0000 7f507a8ad700 1 --2- 172.21.15.100:0/305431373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5074103530 0x7f50741a1930 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:20.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:20.214+0000 7f507a8ad700 1 --2- 172.21.15.100:0/305431373 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5074103ef0 0x7f50741a1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:20.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:20.214+0000 7f507a8ad700 1 -- 172.21.15.100:0/305431373 >> 172.21.15.100:0/305431373 conn(0x7f507407a690 msgr2=0x7f507410ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:17:20.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:20.214+0000 7f507a8ad700 1 -- 172.21.15.100:0/305431373 shutdown_connections 2024-05-10T11:17:20.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:20.214+0000 7f507a8ad700 1 -- 172.21.15.100:0/305431373 wait complete. 2024-05-10T11:17:20.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:17:20 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/305431373' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-10T11:17:20.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:17:20 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/305431373' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-10T11:17:20.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:17:20 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/305431373' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-10T11:17:20.860 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-05-10T11:17:20.861 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T11:17:21.103 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:17:21.510 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:17:21 smithi100 ceph-mon[30296]: pgmap v389: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-05-10T11:17:21.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:17:21 smithi178 ceph-mon[33359]: pgmap v389: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-05-10T11:17:21.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:17:21 smithi188 ceph-mon[34075]: pgmap v389: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-05-10T11:17:22.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:22.992+0000 7f0f044d1700 1 -- 172.21.15.100:0/4023438609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0efc102690 msgr2=0x7f0efc102a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:22.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:22.992+0000 7f0f044d1700 1 --2- 172.21.15.100:0/4023438609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0efc102690 0x7f0efc102a70 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0ef80099c0 tx=0x7f0ef8009cd0 comp rx=0 tx=0).stop 2024-05-10T11:17:22.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:22.992+0000 7f0f044d1700 1 -- 172.21.15.100:0/4023438609 shutdown_connections 2024-05-10T11:17:22.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:22.992+0000 7f0f044d1700 1 --2- 172.21.15.100:0/4023438609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0efc1076e0 0x7f0efc10f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:22.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:22.992+0000 7f0f044d1700 1 --2- 172.21.15.100:0/4023438609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0efc102690 0x7f0efc102a70 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:22.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:22.992+0000 7f0f044d1700 1 --2- 172.21.15.100:0/4023438609 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0efc103040 0x7f0efc107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:22.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:22.992+0000 7f0f044d1700 1 -- 172.21.15.100:0/4023438609 >> 172.21.15.100:0/4023438609 conn(0x7f0efc0783f0 msgr2=0x7f0efc078800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:17:22.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:22.992+0000 7f0f044d1700 1 -- 172.21.15.100:0/4023438609 shutdown_connections 2024-05-10T11:17:22.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:22.992+0000 7f0f044d1700 1 -- 172.21.15.100:0/4023438609 wait complete. 2024-05-10T11:17:22.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:22.992+0000 7f0f044d1700 1 Processor -- start 2024-05-10T11:17:22.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:22.993+0000 7f0f044d1700 1 -- start start 2024-05-10T11:17:22.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:22.993+0000 7f0f044d1700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0efc102690 0x7f0efc19f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:22.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:22.993+0000 7f0f044d1700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0efc103040 0x7f0efc19fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:22.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:22.993+0000 7f0f044d1700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0efc1076e0 0x7f0efc199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:22.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:22.993+0000 7f0f044d1700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0efc112450 con 0x7f0efc102690 2024-05-10T11:17:22.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:22.993+0000 7f0f044d1700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0efc1122d0 con 0x7f0efc1076e0 2024-05-10T11:17:22.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:22.993+0000 7f0f044d1700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0efc1125d0 con 0x7f0efc103040 2024-05-10T11:17:23.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:22.993+0000 7f0f0226d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0efc102690 0x7f0efc19f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:23.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:22.993+0000 7f0f0226d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0efc102690 0x7f0efc19f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58084/0 (socket says 172.21.15.100:58084) 2024-05-10T11:17:23.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:22.993+0000 7f0f0226d700 1 -- 172.21.15.100:0/3185409611 learned_addr learned my addr 172.21.15.100:0/3185409611 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:17:23.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:22.993+0000 7f0f02a6e700 1 --2- 172.21.15.100:0/3185409611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0efc1076e0 0x7f0efc199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:23.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:22.993+0000 7f0f01a6c700 1 --2- 172.21.15.100:0/3185409611 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0efc103040 0x7f0efc19fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:23.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:22.993+0000 7f0f0226d700 1 -- 172.21.15.100:0/3185409611 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0efc103040 msgr2=0x7f0efc19fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:23.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:22.993+0000 7f0f0226d700 1 --2- 172.21.15.100:0/3185409611 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0efc103040 0x7f0efc19fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:23.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:22.993+0000 7f0f0226d700 1 -- 172.21.15.100:0/3185409611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0efc1076e0 msgr2=0x7f0efc199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:23.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:22.993+0000 7f0f0226d700 1 --2- 172.21.15.100:0/3185409611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0efc1076e0 0x7f0efc199920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:23.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:22.993+0000 7f0f0226d700 1 -- 172.21.15.100:0/3185409611 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0efc19a080 con 0x7f0efc102690 2024-05-10T11:17:23.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:22.993+0000 7f0f01a6c700 1 --2- 172.21.15.100:0/3185409611 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0efc103040 0x7f0efc19fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:17:23.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:22.993+0000 7f0f0226d700 1 --2- 172.21.15.100:0/3185409611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0efc102690 0x7f0efc19f760 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8000c00 tx=0x7f0ef800f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:17:23.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:22.994+0000 7f0ef37fe700 1 -- 172.21.15.100:0/3185409611 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef8026070 con 0x7f0efc102690 2024-05-10T11:17:23.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:22.994+0000 7f0ef37fe700 1 -- 172.21.15.100:0/3185409611 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0ef80054c0 con 0x7f0efc102690 2024-05-10T11:17:23.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:22.994+0000 7f0f044d1700 1 -- 172.21.15.100:0/3185409611 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0efc19a310 con 0x7f0efc102690 2024-05-10T11:17:23.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:22.994+0000 7f0ef37fe700 1 -- 172.21.15.100:0/3185409611 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef8004c20 con 0x7f0efc102690 2024-05-10T11:17:23.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:22.994+0000 7f0f044d1700 1 -- 172.21.15.100:0/3185409611 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0efc1a6630 con 0x7f0efc102690 2024-05-10T11:17:23.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:22.995+0000 7f0f044d1700 1 -- 172.21.15.100:0/3185409611 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0efc06a190 con 0x7f0efc102690 2024-05-10T11:17:23.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:22.996+0000 7f0ef37fe700 1 -- 172.21.15.100:0/3185409611 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f0ef8029070 con 0x7f0efc102690 2024-05-10T11:17:23.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:22.997+0000 7f0ef37fe700 1 --2- 172.21.15.100:0/3185409611 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0ee80b2880 0x7f0ee80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:23.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:22.997+0000 7f0f01a6c700 1 --2- 172.21.15.100:0/3185409611 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0ee80b2880 0x7f0ee80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:23.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:22.997+0000 7f0ef37fe700 1 -- 172.21.15.100:0/3185409611 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f0ef80e34b0 con 0x7f0efc102690 2024-05-10T11:17:23.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:22.997+0000 7f0f01a6c700 1 --2- 172.21.15.100:0/3185409611 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0ee80b2880 0x7f0ee80b4d40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0efc06bd30 tx=0x7f0eec005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:17:23.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:23.000+0000 7f0ef37fe700 1 -- 172.21.15.100:0/3185409611 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f0ef8030020 con 0x7f0efc102690 2024-05-10T11:17:23.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:23.254+0000 7f0f044d1700 1 -- 172.21.15.100:0/3185409611 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0efc02e510 con 0x7f0efc102690 2024-05-10T11:17:23.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:23.255+0000 7f0ef37fe700 1 -- 172.21.15.100:0/3185409611 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11894 (secure 0 0 0) 0x7f0ef8013070 con 0x7f0efc102690 2024-05-10T11:17:23.255 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:17:23.255 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2072225012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2072225012},{"type":"v1","addr":"172.21.15.100:6835","nonce":2072225012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/401939540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":401939540},{"type":"v1","addr":"172.21.15.100:6837","nonce":401939540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14571,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/3946311025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3946311025},{"type":"v1","addr":"172.21.15.100:6839","nonce":3946311025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3736371660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3736371660},{"type":"v1","addr":"172.21.15.100:6841","nonce":3736371660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/2741810791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2741810791},{"type":"v1","addr":"172.21.15.178:6833","nonce":2741810791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1704891041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1704891041},{"type":"v1","addr":"172.21.15.188:6833","nonce":1704891041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T11:14:27.242452+0000","modified":"2024-05-10T11:15:35.672461+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24439,"mds_1":24437,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/2124076312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2124076312},{"type":"v1","addr":"172.21.15.178:6835","nonce":2124076312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/3224358425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3224358425},{"type":"v1","addr":"172.21.15.178:6837","nonce":3224358425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/515547690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":515547690},{"type":"v1","addr":"172.21.15.188:6835","nonce":515547690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/1943342620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1943342620},{"type":"v1","addr":"172.21.15.188:6837","nonce":1943342620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.178:6839/3760985404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3760985404},{"type":"v1","addr":"172.21.15.178:6839","nonce":3760985404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.188:6839/3194917070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":3194917070},{"type":"v1","addr":"172.21.15.188:6839","nonce":3194917070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24439,"qdb_cluster":[24437,24439,24427]},"id":1}]} 2024-05-10T11:17:23.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:23.258+0000 7f0f044d1700 1 -- 172.21.15.100:0/3185409611 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0ee80b2880 msgr2=0x7f0ee80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:23.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:23.258+0000 7f0f044d1700 1 --2- 172.21.15.100:0/3185409611 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0ee80b2880 0x7f0ee80b4d40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0efc06bd30 tx=0x7f0eec005f50 comp rx=0 tx=0).stop 2024-05-10T11:17:23.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:23.258+0000 7f0f044d1700 1 -- 172.21.15.100:0/3185409611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0efc102690 msgr2=0x7f0efc19f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:23.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:23.258+0000 7f0f044d1700 1 --2- 172.21.15.100:0/3185409611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0efc102690 0x7f0efc19f760 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8000c00 tx=0x7f0ef800f740 comp rx=0 tx=0).stop 2024-05-10T11:17:23.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:23.258+0000 7f0f044d1700 1 -- 172.21.15.100:0/3185409611 shutdown_connections 2024-05-10T11:17:23.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:23.258+0000 7f0f044d1700 1 --2- 172.21.15.100:0/3185409611 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0ee80b2880 0x7f0ee80b4d40 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:23.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:23.258+0000 7f0f044d1700 1 --2- 172.21.15.100:0/3185409611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0efc1076e0 0x7f0efc199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:23.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:23.258+0000 7f0f044d1700 1 --2- 172.21.15.100:0/3185409611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0efc102690 0x7f0efc19f760 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:23.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:23.258+0000 7f0f044d1700 1 --2- 172.21.15.100:0/3185409611 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0efc103040 0x7f0efc19fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:23.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:23.258+0000 7f0f044d1700 1 -- 172.21.15.100:0/3185409611 >> 172.21.15.100:0/3185409611 conn(0x7f0efc0783f0 msgr2=0x7f0efc1065c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:17:23.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:23.258+0000 7f0f044d1700 1 -- 172.21.15.100:0/3185409611 shutdown_connections 2024-05-10T11:17:23.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:23.258+0000 7f0f044d1700 1 -- 172.21.15.100:0/3185409611 wait complete. 2024-05-10T11:17:23.259 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 26 2024-05-10T11:17:23.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:17:23 smithi100 ceph-mon[30296]: pgmap v390: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T11:17:23.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:17:23 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/3185409611' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:17:23.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:17:23 smithi178 ceph-mon[33359]: pgmap v390: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T11:17:23.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:17:23 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/3185409611' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:17:23.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:17:23 smithi188 ceph-mon[34075]: pgmap v390: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T11:17:23.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:17:23 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/3185409611' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:17:23.991 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T11:17:24.235 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:17:25.619 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:17:25 smithi100 ceph-mon[30296]: pgmap v391: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-10T11:17:25.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:17:25 smithi178 ceph-mon[33359]: pgmap v391: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-10T11:17:25.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:17:25 smithi188 ceph-mon[34075]: pgmap v391: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-10T11:17:26.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:26.009+0000 7fe87285c700 1 -- 172.21.15.100:0/921319416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe86c108850 msgr2=0x7fe86c1103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:26.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:26.009+0000 7fe87285c700 1 --2- 172.21.15.100:0/921319416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe86c108850 0x7fe86c1103d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe86400b0d0 tx=0x7fe86400b3e0 comp rx=0 tx=0).stop 2024-05-10T11:17:26.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:26.009+0000 7fe87285c700 1 -- 172.21.15.100:0/921319416 shutdown_connections 2024-05-10T11:17:26.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:26.009+0000 7fe87285c700 1 --2- 172.21.15.100:0/921319416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe86c108850 0x7fe86c1103d0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:26.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:26.009+0000 7fe87285c700 1 --2- 172.21.15.100:0/921319416 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe86c103750 0x7fe86c103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:26.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:26.009+0000 7fe87285c700 1 --2- 172.21.15.100:0/921319416 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe86c104100 0x7fe86c108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:26.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:26.009+0000 7fe87285c700 1 -- 172.21.15.100:0/921319416 >> 172.21.15.100:0/921319416 conn(0x7fe86c07a600 msgr2=0x7fe86c07aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:17:26.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:26.009+0000 7fe87285c700 1 -- 172.21.15.100:0/921319416 shutdown_connections 2024-05-10T11:17:26.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:26.009+0000 7fe87285c700 1 -- 172.21.15.100:0/921319416 wait complete. 2024-05-10T11:17:26.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:26.009+0000 7fe87285c700 1 Processor -- start 2024-05-10T11:17:26.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:26.009+0000 7fe87285c700 1 -- start start 2024-05-10T11:17:26.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:26.010+0000 7fe87285c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe86c103750 0x7fe86c19f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:26.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:26.010+0000 7fe87285c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe86c104100 0x7fe86c19fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:26.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:26.010+0000 7fe87285c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe86c108850 0x7fe86c1997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:26.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:26.010+0000 7fe87285c700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe86c1134d0 con 0x7fe86c104100 2024-05-10T11:17:26.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:26.010+0000 7fe87285c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe86c113350 con 0x7fe86c108850 2024-05-10T11:17:26.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:26.010+0000 7fe87285c700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe86c113650 con 0x7fe86c103750 2024-05-10T11:17:26.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:26.010+0000 7fe86b7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe86c104100 0x7fe86c19fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:26.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:26.010+0000 7fe86b7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe86c104100 0x7fe86c19fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58116/0 (socket says 172.21.15.100:58116) 2024-05-10T11:17:26.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:26.010+0000 7fe86b7fe700 1 -- 172.21.15.100:0/3564842627 learned_addr learned my addr 172.21.15.100:0/3564842627 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:17:26.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:26.010+0000 7fe86bfff700 1 --2- 172.21.15.100:0/3564842627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe86c103750 0x7fe86c19f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:26.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:26.010+0000 7fe870df9700 1 --2- 172.21.15.100:0/3564842627 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe86c108850 0x7fe86c1997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:26.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:26.010+0000 7fe86b7fe700 1 -- 172.21.15.100:0/3564842627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe86c103750 msgr2=0x7fe86c19f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:26.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:26.010+0000 7fe86b7fe700 1 --2- 172.21.15.100:0/3564842627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe86c103750 0x7fe86c19f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:26.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:26.010+0000 7fe86b7fe700 1 -- 172.21.15.100:0/3564842627 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe86c108850 msgr2=0x7fe86c1997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:26.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:26.010+0000 7fe86b7fe700 1 --2- 172.21.15.100:0/3564842627 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe86c108850 0x7fe86c1997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:26.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:26.010+0000 7fe86b7fe700 1 -- 172.21.15.100:0/3564842627 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe86c199f90 con 0x7fe86c104100 2024-05-10T11:17:26.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:26.011+0000 7fe86b7fe700 1 --2- 172.21.15.100:0/3564842627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe86c104100 0x7fe86c19fb70 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe86000c920 tx=0x7fe86000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:17:26.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:26.011+0000 7fe8697fa700 1 -- 172.21.15.100:0/3564842627 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe860004a30 con 0x7fe86c104100 2024-05-10T11:17:26.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:26.011+0000 7fe87285c700 1 -- 172.21.15.100:0/3564842627 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe86c19a280 con 0x7fe86c104100 2024-05-10T11:17:26.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:26.011+0000 7fe8697fa700 1 -- 172.21.15.100:0/3564842627 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe8600054b0 con 0x7fe86c104100 2024-05-10T11:17:26.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:26.011+0000 7fe8697fa700 1 -- 172.21.15.100:0/3564842627 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe860007b70 con 0x7fe86c104100 2024-05-10T11:17:26.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:26.011+0000 7fe87285c700 1 -- 172.21.15.100:0/3564842627 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe86c1a6490 con 0x7fe86c104100 2024-05-10T11:17:26.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:26.012+0000 7fe87285c700 1 -- 172.21.15.100:0/3564842627 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe86c104be0 con 0x7fe86c104100 2024-05-10T11:17:26.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:26.013+0000 7fe8697fa700 1 -- 172.21.15.100:0/3564842627 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fe860004bd0 con 0x7fe86c104100 2024-05-10T11:17:26.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:26.014+0000 7fe8697fa700 1 --2- 172.21.15.100:0/3564842627 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe8540b2880 0x7fe8540b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:26.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:26.014+0000 7fe86bfff700 1 --2- 172.21.15.100:0/3564842627 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe8540b2880 0x7fe8540b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:26.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:26.014+0000 7fe8697fa700 1 -- 172.21.15.100:0/3564842627 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fe8600e2480 con 0x7fe86c104100 2024-05-10T11:17:26.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:26.014+0000 7fe86bfff700 1 --2- 172.21.15.100:0/3564842627 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe8540b2880 0x7fe8540b4d40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe85c005950 tx=0x7fe85c00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:17:26.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:26.017+0000 7fe8697fa700 1 -- 172.21.15.100:0/3564842627 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fe860014030 con 0x7fe86c104100 2024-05-10T11:17:26.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:26.270+0000 7fe87285c700 1 -- 172.21.15.100:0/3564842627 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe86c19ace0 con 0x7fe86c104100 2024-05-10T11:17:26.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:26.271+0000 7fe8697fa700 1 -- 172.21.15.100:0/3564842627 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11894 (secure 0 0 0) 0x7fe860005660 con 0x7fe86c104100 2024-05-10T11:17:26.271 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:17:26.271 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2072225012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2072225012},{"type":"v1","addr":"172.21.15.100:6835","nonce":2072225012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/401939540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":401939540},{"type":"v1","addr":"172.21.15.100:6837","nonce":401939540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14571,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/3946311025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3946311025},{"type":"v1","addr":"172.21.15.100:6839","nonce":3946311025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3736371660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3736371660},{"type":"v1","addr":"172.21.15.100:6841","nonce":3736371660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/2741810791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2741810791},{"type":"v1","addr":"172.21.15.178:6833","nonce":2741810791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1704891041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1704891041},{"type":"v1","addr":"172.21.15.188:6833","nonce":1704891041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T11:14:27.242452+0000","modified":"2024-05-10T11:15:35.672461+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24439,"mds_1":24437,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/2124076312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2124076312},{"type":"v1","addr":"172.21.15.178:6835","nonce":2124076312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/3224358425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3224358425},{"type":"v1","addr":"172.21.15.178:6837","nonce":3224358425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/515547690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":515547690},{"type":"v1","addr":"172.21.15.188:6835","nonce":515547690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/1943342620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1943342620},{"type":"v1","addr":"172.21.15.188:6837","nonce":1943342620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.178:6839/3760985404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3760985404},{"type":"v1","addr":"172.21.15.178:6839","nonce":3760985404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.188:6839/3194917070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":3194917070},{"type":"v1","addr":"172.21.15.188:6839","nonce":3194917070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24439,"qdb_cluster":[24437,24439,24427]},"id":1}]} 2024-05-10T11:17:26.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:26.273+0000 7fe87285c700 1 -- 172.21.15.100:0/3564842627 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe8540b2880 msgr2=0x7fe8540b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:26.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:26.273+0000 7fe87285c700 1 --2- 172.21.15.100:0/3564842627 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe8540b2880 0x7fe8540b4d40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe85c005950 tx=0x7fe85c00a300 comp rx=0 tx=0).stop 2024-05-10T11:17:26.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:26.273+0000 7fe87285c700 1 -- 172.21.15.100:0/3564842627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe86c104100 msgr2=0x7fe86c19fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:26.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:26.273+0000 7fe87285c700 1 --2- 172.21.15.100:0/3564842627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe86c104100 0x7fe86c19fb70 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe86000c920 tx=0x7fe86000cc30 comp rx=0 tx=0).stop 2024-05-10T11:17:26.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:26.273+0000 7fe87285c700 1 -- 172.21.15.100:0/3564842627 shutdown_connections 2024-05-10T11:17:26.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:26.273+0000 7fe87285c700 1 --2- 172.21.15.100:0/3564842627 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe8540b2880 0x7fe8540b4d40 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:26.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:26.273+0000 7fe87285c700 1 --2- 172.21.15.100:0/3564842627 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe86c108850 0x7fe86c1997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:26.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:26.273+0000 7fe87285c700 1 --2- 172.21.15.100:0/3564842627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe86c103750 0x7fe86c19f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:26.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:26.273+0000 7fe87285c700 1 --2- 172.21.15.100:0/3564842627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe86c104100 0x7fe86c19fb70 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:26.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:26.273+0000 7fe87285c700 1 -- 172.21.15.100:0/3564842627 >> 172.21.15.100:0/3564842627 conn(0x7fe86c07a600 msgr2=0x7fe86c101880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:17:26.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:26.273+0000 7fe87285c700 1 -- 172.21.15.100:0/3564842627 shutdown_connections 2024-05-10T11:17:26.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:26.273+0000 7fe87285c700 1 -- 172.21.15.100:0/3564842627 wait complete. 2024-05-10T11:17:26.274 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 26 2024-05-10T11:17:26.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:17:26 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/3564842627' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:17:26.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:17:26 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/3564842627' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:17:26.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:17:26 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/3564842627' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:17:26.967 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-10T11:17:27.210 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:17:27.622 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:17:27 smithi100 ceph-mon[30296]: pgmap v392: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-05-10T11:17:27.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:17:27 smithi178 ceph-mon[33359]: pgmap v392: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-05-10T11:17:27.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:17:27 smithi188 ceph-mon[34075]: pgmap v392: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-05-10T11:17:28.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:28.943+0000 7f1910c7e700 1 -- 172.21.15.100:0/2602918909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f190c0756d0 msgr2=0x7f190c075ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:28.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:28.943+0000 7f1910c7e700 1 --2- 172.21.15.100:0/2602918909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f190c0756d0 0x7f190c075ab0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f18fc009990 tx=0x7f18fc009ca0 comp rx=0 tx=0).stop 2024-05-10T11:17:28.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:28.943+0000 7f1910c7e700 1 -- 172.21.15.100:0/2602918909 shutdown_connections 2024-05-10T11:17:28.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:28.943+0000 7f1910c7e700 1 --2- 172.21.15.100:0/2602918909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f190c10fea0 0x7f190c112290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:28.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:28.943+0000 7f1910c7e700 1 --2- 172.21.15.100:0/2602918909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f190c0756d0 0x7f190c075ab0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:28.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:28.943+0000 7f1910c7e700 1 --2- 172.21.15.100:0/2602918909 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f190c075ff0 0x7f190c10f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:28.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:28.943+0000 7f1910c7e700 1 -- 172.21.15.100:0/2602918909 >> 172.21.15.100:0/2602918909 conn(0x7f190c07a600 msgr2=0x7f190c07aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:17:28.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:28.943+0000 7f1910c7e700 1 -- 172.21.15.100:0/2602918909 shutdown_connections 2024-05-10T11:17:28.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:28.943+0000 7f1910c7e700 1 -- 172.21.15.100:0/2602918909 wait complete. 2024-05-10T11:17:28.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:28.944+0000 7f1910c7e700 1 Processor -- start 2024-05-10T11:17:28.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:28.944+0000 7f1910c7e700 1 -- start start 2024-05-10T11:17:28.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:28.944+0000 7f1910c7e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f190c0756d0 0x7f190c1a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:28.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:28.944+0000 7f1910c7e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f190c075ff0 0x7f190c1a1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:28.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:28.944+0000 7f1910c7e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f190c10fea0 0x7f190c19bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:28.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:28.944+0000 7f1910c7e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f190c115730 con 0x7f190c10fea0 2024-05-10T11:17:28.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:28.944+0000 7f1910c7e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f190c1155b0 con 0x7f190c0756d0 2024-05-10T11:17:28.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:28.944+0000 7f1910c7e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f190c1158b0 con 0x7f190c075ff0 2024-05-10T11:17:28.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:28.944+0000 7f190ad9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f190c10fea0 0x7f190c19bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:28.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:28.944+0000 7f190ad9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f190c10fea0 0x7f190c19bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35502/0 (socket says 172.21.15.100:35502) 2024-05-10T11:17:28.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:28.944+0000 7f190ad9d700 1 -- 172.21.15.100:0/1225849875 learned_addr learned my addr 172.21.15.100:0/1225849875 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:17:28.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:28.945+0000 7f190a59c700 1 --2- 172.21.15.100:0/1225849875 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f190c0756d0 0x7f190c1a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:28.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:28.945+0000 7f1903fff700 1 --2- 172.21.15.100:0/1225849875 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f190c075ff0 0x7f190c1a1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:28.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:28.945+0000 7f190ad9d700 1 -- 172.21.15.100:0/1225849875 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f190c075ff0 msgr2=0x7f190c1a1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:28.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:28.945+0000 7f190ad9d700 1 --2- 172.21.15.100:0/1225849875 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f190c075ff0 0x7f190c1a1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:28.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:28.945+0000 7f190ad9d700 1 -- 172.21.15.100:0/1225849875 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f190c0756d0 msgr2=0x7f190c1a18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:28.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:28.945+0000 7f190ad9d700 1 --2- 172.21.15.100:0/1225849875 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f190c0756d0 0x7f190c1a18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:28.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:28.945+0000 7f190ad9d700 1 -- 172.21.15.100:0/1225849875 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f190c19c210 con 0x7f190c10fea0 2024-05-10T11:17:28.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:28.945+0000 7f190ad9d700 1 --2- 172.21.15.100:0/1225849875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f190c10fea0 0x7f190c19bab0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f190400ef90 tx=0x7f190400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:17:28.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:28.945+0000 7f19037fe700 1 -- 172.21.15.100:0/1225849875 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19040092e0 con 0x7f190c10fea0 2024-05-10T11:17:28.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:28.945+0000 7f1910c7e700 1 -- 172.21.15.100:0/1225849875 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f190c19c4a0 con 0x7f190c10fea0 2024-05-10T11:17:28.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:28.945+0000 7f19037fe700 1 -- 172.21.15.100:0/1225849875 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1904004bb0 con 0x7f190c10fea0 2024-05-10T11:17:28.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:28.945+0000 7f19037fe700 1 -- 172.21.15.100:0/1225849875 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1904004040 con 0x7f190c10fea0 2024-05-10T11:17:28.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:28.945+0000 7f1910c7e700 1 -- 172.21.15.100:0/1225849875 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f190c1a8770 con 0x7f190c10fea0 2024-05-10T11:17:28.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:28.946+0000 7f1910c7e700 1 -- 172.21.15.100:0/1225849875 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f190c06a190 con 0x7f190c10fea0 2024-05-10T11:17:28.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:28.947+0000 7f19037fe700 1 -- 172.21.15.100:0/1225849875 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f190401b030 con 0x7f190c10fea0 2024-05-10T11:17:28.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:28.948+0000 7f19037fe700 1 --2- 172.21.15.100:0/1225849875 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f18f80b2880 0x7f18f80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:28.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:28.948+0000 7f190a59c700 1 --2- 172.21.15.100:0/1225849875 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f18f80b2880 0x7f18f80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:28.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:28.948+0000 7f19037fe700 1 -- 172.21.15.100:0/1225849875 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f19040e2b50 con 0x7f190c10fea0 2024-05-10T11:17:28.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:28.949+0000 7f190a59c700 1 --2- 172.21.15.100:0/1225849875 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f18f80b2880 0x7f18f80b4d40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f18fc00b2e0 tx=0x7f18fc01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:17:28.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:28.953+0000 7f19037fe700 1 -- 172.21.15.100:0/1225849875 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f1904022020 con 0x7f190c10fea0 2024-05-10T11:17:29.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:29.157+0000 7f1910c7e700 1 -- 172.21.15.100:0/1225849875 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f190c02e510 con 0x7f190c10fea0 2024-05-10T11:17:29.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:29.159+0000 7f19037fe700 1 -- 172.21.15.100:0/1225849875 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24737 (secure 0 0 0) 0x7f1904010040 con 0x7f190c10fea0 2024-05-10T11:17:29.160 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:17:29.160 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":89,"fsid":"20a542cc-0ebd-11ef-bc98-c7b262605968","created":"2024-05-10T11:06:05.454446+0000","modified":"2024-05-10T11:15:01.418144+0000","last_up_change":"2024-05-10T11:13:18.185471+0000","last_in_change":"2024-05-10T11:12:57.268678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T11:09:26.176441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T11:14:21.004285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T11:14:24.179414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T11:14:33.308517+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5f031418-da23-4e85-8669-160960dadd09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":643028278},{"type":"v1","addr":"172.21.15.100:6803","nonce":643028278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":643028278},{"type":"v1","addr":"172.21.15.100:6805","nonce":643028278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":643028278},{"type":"v1","addr":"172.21.15.100:6809","nonce":643028278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":643028278},{"type":"v1","addr":"172.21.15.100:6807","nonce":643028278}]},"public_addr":"172.21.15.100:6803/643028278","cluster_addr":"172.21.15.100:6805/643028278","heartbeat_back_addr":"172.21.15.100:6809/643028278","heartbeat_front_addr":"172.21.15.100:6807/643028278","state":["exists","up"]},{"osd":1,"uuid":"a908cdc9-001e-4b65-9ab0-0dd08dba20f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":3103987017},{"type":"v1","addr":"172.21.15.178:6801","nonce":3103987017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":3103987017},{"type":"v1","addr":"172.21.15.178:6803","nonce":3103987017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":3103987017},{"type":"v1","addr":"172.21.15.178:6807","nonce":3103987017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":3103987017},{"type":"v1","addr":"172.21.15.178:6805","nonce":3103987017}]},"public_addr":"172.21.15.178:6801/3103987017","cluster_addr":"172.21.15.178:6803/3103987017","heartbeat_back_addr":"172.21.15.178:6807/3103987017","heartbeat_front_addr":"172.21.15.178:6805/3103987017","state":["exists","up"]},{"osd":2,"uuid":"4d8f89ae-9424-46e9-9c80-17c47cc5e8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3991466468},{"type":"v1","addr":"172.21.15.188:6801","nonce":3991466468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3991466468},{"type":"v1","addr":"172.21.15.188:6803","nonce":3991466468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3991466468},{"type":"v1","addr":"172.21.15.188:6807","nonce":3991466468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3991466468},{"type":"v1","addr":"172.21.15.188:6805","nonce":3991466468}]},"public_addr":"172.21.15.188:6801/3991466468","cluster_addr":"172.21.15.188:6803/3991466468","heartbeat_back_addr":"172.21.15.188:6807/3991466468","heartbeat_front_addr":"172.21.15.188:6805/3991466468","state":["exists","up"]},{"osd":3,"uuid":"4c8865aa-fef4-4710-b766-1385adb9462c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1713357192},{"type":"v1","addr":"172.21.15.100:6811","nonce":1713357192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1713357192},{"type":"v1","addr":"172.21.15.100:6813","nonce":1713357192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1713357192},{"type":"v1","addr":"172.21.15.100:6817","nonce":1713357192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1713357192},{"type":"v1","addr":"172.21.15.100:6815","nonce":1713357192}]},"public_addr":"172.21.15.100:6811/1713357192","cluster_addr":"172.21.15.100:6813/1713357192","heartbeat_back_addr":"172.21.15.100:6817/1713357192","heartbeat_front_addr":"172.21.15.100:6815/1713357192","state":["exists","up"]},{"osd":4,"uuid":"a9653800-da57-43eb-8781-3188e8a3949f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":2222814192},{"type":"v1","addr":"172.21.15.178:6809","nonce":2222814192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":2222814192},{"type":"v1","addr":"172.21.15.178:6811","nonce":2222814192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":2222814192},{"type":"v1","addr":"172.21.15.178:6815","nonce":2222814192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":2222814192},{"type":"v1","addr":"172.21.15.178:6813","nonce":2222814192}]},"public_addr":"172.21.15.178:6809/2222814192","cluster_addr":"172.21.15.178:6811/2222814192","heartbeat_back_addr":"172.21.15.178:6815/2222814192","heartbeat_front_addr":"172.21.15.178:6813/2222814192","state":["exists","up"]},{"osd":5,"uuid":"fb5f8af2-1eb0-4d8a-a7a3-ed6dd68e692e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3517375935},{"type":"v1","addr":"172.21.15.188:6809","nonce":3517375935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3517375935},{"type":"v1","addr":"172.21.15.188:6811","nonce":3517375935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3517375935},{"type":"v1","addr":"172.21.15.188:6815","nonce":3517375935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3517375935},{"type":"v1","addr":"172.21.15.188:6813","nonce":3517375935}]},"public_addr":"172.21.15.188:6809/3517375935","cluster_addr":"172.21.15.188:6811/3517375935","heartbeat_back_addr":"172.21.15.188:6815/3517375935","heartbeat_front_addr":"172.21.15.188:6813/3517375935","state":["exists","up"]},{"osd":6,"uuid":"84b57387-0a94-4e49-b099-493d329b87dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1410593862},{"type":"v1","addr":"172.21.15.100:6819","nonce":1410593862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1410593862},{"type":"v1","addr":"172.21.15.100:6821","nonce":1410593862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1410593862},{"type":"v1","addr":"172.21.15.100:6825","nonce":1410593862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1410593862},{"type":"v1","addr":"172.21.15.100:6823","nonce":1410593862}]},"public_addr":"172.21.15.100:6819/1410593862","cluster_addr":"172.21.15.100:6821/1410593862","heartbeat_back_addr":"172.21.15.100:6825/1410593862","heartbeat_front_addr":"172.21.15.100:6823/1410593862","state":["exists","up"]},{"osd":7,"uuid":"02ada28f-ac4f-47cf-983f-8a6da1140abe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2458690455},{"type":"v1","addr":"172.21.15.178:6817","nonce":2458690455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2458690455},{"type":"v1","addr":"172.21.15.178:6819","nonce":2458690455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2458690455},{"type":"v1","addr":"172.21.15.178:6823","nonce":2458690455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2458690455},{"type":"v1","addr":"172.21.15.178:6821","nonce":2458690455}]},"public_addr":"172.21.15.178:6817/2458690455","cluster_addr":"172.21.15.178:6819/2458690455","heartbeat_back_addr":"172.21.15.178:6823/2458690455","heartbeat_front_addr":"172.21.15.178:6821/2458690455","state":["exists","up"]},{"osd":8,"uuid":"e7e489bc-15fc-4a4c-b297-d1bb626513ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1523872826},{"type":"v1","addr":"172.21.15.188:6817","nonce":1523872826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1523872826},{"type":"v1","addr":"172.21.15.188:6819","nonce":1523872826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1523872826},{"type":"v1","addr":"172.21.15.188:6823","nonce":1523872826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1523872826},{"type":"v1","addr":"172.21.15.188:6821","nonce":1523872826}]},"public_addr":"172.21.15.188:6817/1523872826","cluster_addr":"172.21.15.188:6819/1523872826","heartbeat_back_addr":"172.21.15.188:6823/1523872826","heartbeat_front_addr":"172.21.15.188:6821/1523872826","state":["exists","up"]},{"osd":9,"uuid":"72328326-cd16-45d9-bfd9-40f6c5c20f5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3585761978},{"type":"v1","addr":"172.21.15.100:6827","nonce":3585761978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3585761978},{"type":"v1","addr":"172.21.15.100:6829","nonce":3585761978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3585761978},{"type":"v1","addr":"172.21.15.100:6833","nonce":3585761978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3585761978},{"type":"v1","addr":"172.21.15.100:6831","nonce":3585761978}]},"public_addr":"172.21.15.100:6827/3585761978","cluster_addr":"172.21.15.100:6829/3585761978","heartbeat_back_addr":"172.21.15.100:6833/3585761978","heartbeat_front_addr":"172.21.15.100:6831/3585761978","state":["exists","up"]},{"osd":10,"uuid":"a355f9a8-91a3-41d4-814d-5a9c9b953706","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2985841725},{"type":"v1","addr":"172.21.15.178:6825","nonce":2985841725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2985841725},{"type":"v1","addr":"172.21.15.178:6827","nonce":2985841725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2985841725},{"type":"v1","addr":"172.21.15.178:6831","nonce":2985841725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2985841725},{"type":"v1","addr":"172.21.15.178:6829","nonce":2985841725}]},"public_addr":"172.21.15.178:6825/2985841725","cluster_addr":"172.21.15.178:6827/2985841725","heartbeat_back_addr":"172.21.15.178:6831/2985841725","heartbeat_front_addr":"172.21.15.178:6829/2985841725","state":["exists","up"]},{"osd":11,"uuid":"76a5e13c-7e38-42f9-bfa7-7fedec7c7184","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2574955118},{"type":"v1","addr":"172.21.15.188:6825","nonce":2574955118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2574955118},{"type":"v1","addr":"172.21.15.188:6827","nonce":2574955118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2574955118},{"type":"v1","addr":"172.21.15.188:6831","nonce":2574955118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2574955118},{"type":"v1","addr":"172.21.15.188:6829","nonce":2574955118}]},"public_addr":"172.21.15.188:6825/2574955118","cluster_addr":"172.21.15.188:6827/2574955118","heartbeat_back_addr":"172.21.15.188:6831/2574955118","heartbeat_front_addr":"172.21.15.188:6829/2574955118","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:08:36.086637+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:08:58.948031+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:09:22.142517+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:09:47.655748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:10:12.853999+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:10:37.911383+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:11:04.448815+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:11:30.331740+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:11:55.819278+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:12:23.204612+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:12:49.484835+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:13:15.783531+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:6800/1597433518":"2024-05-11T11:07:00.092732+0000","172.21.15.100:6800/1086450256":"2024-05-11T11:06:33.114176+0000","172.21.15.100:0/767964410":"2024-05-11T11:07:00.092732+0000","172.21.15.100:6801/1597433518":"2024-05-11T11:07:00.092732+0000","172.21.15.100:6801/1086450256":"2024-05-11T11:06:33.114176+0000","172.21.15.100:0/2770572070":"2024-05-11T11:06:33.114176+0000","172.21.15.100:0/1379340513":"2024-05-11T11:06:33.114176+0000","172.21.15.100:0/3025646953":"2024-05-11T11:07:00.092732+0000","172.21.15.100:0/4099258930":"2024-05-11T11:07:00.092732+0000","172.21.15.100:0/4267693567":"2024-05-11T11:06:33.114176+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T11:17:29.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:29.161+0000 7f1910c7e700 1 -- 172.21.15.100:0/1225849875 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f18f80b2880 msgr2=0x7f18f80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:29.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:29.161+0000 7f1910c7e700 1 --2- 172.21.15.100:0/1225849875 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f18f80b2880 0x7f18f80b4d40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f18fc00b2e0 tx=0x7f18fc01a040 comp rx=0 tx=0).stop 2024-05-10T11:17:29.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:29.161+0000 7f1910c7e700 1 -- 172.21.15.100:0/1225849875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f190c10fea0 msgr2=0x7f190c19bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:29.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:29.161+0000 7f1910c7e700 1 --2- 172.21.15.100:0/1225849875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f190c10fea0 0x7f190c19bab0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f190400ef90 tx=0x7f190400c3a0 comp rx=0 tx=0).stop 2024-05-10T11:17:29.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:29.161+0000 7f1910c7e700 1 -- 172.21.15.100:0/1225849875 shutdown_connections 2024-05-10T11:17:29.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:29.161+0000 7f1910c7e700 1 --2- 172.21.15.100:0/1225849875 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f18f80b2880 0x7f18f80b4d40 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:29.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:29.161+0000 7f1910c7e700 1 --2- 172.21.15.100:0/1225849875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f190c10fea0 0x7f190c19bab0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:29.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:29.161+0000 7f1910c7e700 1 --2- 172.21.15.100:0/1225849875 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f190c0756d0 0x7f190c1a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:29.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:29.161+0000 7f1910c7e700 1 --2- 172.21.15.100:0/1225849875 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f190c075ff0 0x7f190c1a1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:29.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:29.161+0000 7f1910c7e700 1 -- 172.21.15.100:0/1225849875 >> 172.21.15.100:0/1225849875 conn(0x7f190c07a600 msgr2=0x7f190c100750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:17:29.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:29.161+0000 7f1910c7e700 1 -- 172.21.15.100:0/1225849875 shutdown_connections 2024-05-10T11:17:29.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:29.161+0000 7f1910c7e700 1 -- 172.21.15.100:0/1225849875 wait complete. 2024-05-10T11:17:29.462 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:17:29 smithi100 ceph-mon[30296]: pgmap v393: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T11:17:29.462 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:17:29 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/1225849875' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T11:17:29.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:17:29 smithi178 ceph-mon[33359]: pgmap v393: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T11:17:29.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:17:29 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/1225849875' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T11:17:29.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:17:29 smithi188 ceph-mon[34075]: pgmap v393: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T11:17:29.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:17:29 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/1225849875' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T11:17:29.867 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T11:17:30.116 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:17:31.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:17:31 smithi100 ceph-mon[30296]: pgmap v394: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T11:17:31.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:17:31 smithi178 ceph-mon[33359]: pgmap v394: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T11:17:31.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:17:31 smithi188 ceph-mon[34075]: pgmap v394: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T11:17:31.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:31.826+0000 7fa63f8d4700 1 -- 172.21.15.100:0/4037954264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa638103f20 msgr2=0x7fa638110310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:31.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:31.826+0000 7fa63f8d4700 1 --2- 172.21.15.100:0/4037954264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa638103f20 0x7fa638110310 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa63000b0d0 tx=0x7fa63000b3e0 comp rx=0 tx=0).stop 2024-05-10T11:17:31.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:31.826+0000 7fa63f8d4700 1 -- 172.21.15.100:0/4037954264 shutdown_connections 2024-05-10T11:17:31.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:31.826+0000 7fa63f8d4700 1 --2- 172.21.15.100:0/4037954264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa638103f20 0x7fa638110310 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:31.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:31.826+0000 7fa63f8d4700 1 --2- 172.21.15.100:0/4037954264 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa638109540 0x7fa638109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:31.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:31.826+0000 7fa63f8d4700 1 --2- 172.21.15.100:0/4037954264 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa638103560 0x7fa6381039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:31.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:31.826+0000 7fa63f8d4700 1 -- 172.21.15.100:0/4037954264 >> 172.21.15.100:0/4037954264 conn(0x7fa63807a660 msgr2=0x7fa63807aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:17:31.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:31.826+0000 7fa63f8d4700 1 -- 172.21.15.100:0/4037954264 shutdown_connections 2024-05-10T11:17:31.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:31.826+0000 7fa63f8d4700 1 -- 172.21.15.100:0/4037954264 wait complete. 2024-05-10T11:17:31.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:31.826+0000 7fa63f8d4700 1 Processor -- start 2024-05-10T11:17:31.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:31.827+0000 7fa63f8d4700 1 -- start start 2024-05-10T11:17:31.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:31.827+0000 7fa63f8d4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa638103560 0x7fa63819f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:31.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:31.827+0000 7fa63f8d4700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa638103f20 0x7fa63819fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:31.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:31.827+0000 7fa63f8d4700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa638109540 0x7fa638199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:31.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:31.827+0000 7fa63f8d4700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa638113550 con 0x7fa638103560 2024-05-10T11:17:31.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:31.827+0000 7fa63f8d4700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa6381133d0 con 0x7fa638109540 2024-05-10T11:17:31.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:31.827+0000 7fa63f8d4700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa6381136d0 con 0x7fa638103f20 2024-05-10T11:17:31.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:31.827+0000 7fa63d670700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa638103560 0x7fa63819f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:31.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:31.827+0000 7fa63d670700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa638103560 0x7fa63819f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35522/0 (socket says 172.21.15.100:35522) 2024-05-10T11:17:31.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:31.827+0000 7fa63d670700 1 -- 172.21.15.100:0/2988679397 learned_addr learned my addr 172.21.15.100:0/2988679397 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:17:31.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:31.827+0000 7fa63ce6f700 1 --2- 172.21.15.100:0/2988679397 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa638103f20 0x7fa63819fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:31.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:31.827+0000 7fa63de71700 1 --2- 172.21.15.100:0/2988679397 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa638109540 0x7fa638199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:31.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:31.827+0000 7fa63d670700 1 -- 172.21.15.100:0/2988679397 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa638103f20 msgr2=0x7fa63819fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:31.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:31.827+0000 7fa63d670700 1 --2- 172.21.15.100:0/2988679397 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa638103f20 0x7fa63819fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:31.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:31.827+0000 7fa63d670700 1 -- 172.21.15.100:0/2988679397 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa638109540 msgr2=0x7fa638199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:31.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:31.827+0000 7fa63d670700 1 --2- 172.21.15.100:0/2988679397 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa638109540 0x7fa638199890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:31.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:31.827+0000 7fa63d670700 1 -- 172.21.15.100:0/2988679397 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa63819a070 con 0x7fa638103560 2024-05-10T11:17:31.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:31.827+0000 7fa63de71700 1 --2- 172.21.15.100:0/2988679397 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa638109540 0x7fa638199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:17:31.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:31.827+0000 7fa63d670700 1 --2- 172.21.15.100:0/2988679397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa638103560 0x7fa63819f6c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa63400b530 tx=0x7fa63400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:17:31.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:31.828+0000 7fa62e7fc700 1 -- 172.21.15.100:0/2988679397 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa634015020 con 0x7fa638103560 2024-05-10T11:17:31.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:31.828+0000 7fa63f8d4700 1 -- 172.21.15.100:0/2988679397 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa63819a360 con 0x7fa638103560 2024-05-10T11:17:31.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:31.828+0000 7fa62e7fc700 1 -- 172.21.15.100:0/2988679397 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa634011cc0 con 0x7fa638103560 2024-05-10T11:17:31.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:31.828+0000 7fa62e7fc700 1 -- 172.21.15.100:0/2988679397 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa634010dc0 con 0x7fa638103560 2024-05-10T11:17:31.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:31.828+0000 7fa63f8d4700 1 -- 172.21.15.100:0/2988679397 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6381a6520 con 0x7fa638103560 2024-05-10T11:17:31.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:31.829+0000 7fa63f8d4700 1 -- 172.21.15.100:0/2988679397 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa63810b9d0 con 0x7fa638103560 2024-05-10T11:17:31.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:31.830+0000 7fa62e7fc700 1 -- 172.21.15.100:0/2988679397 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fa634018020 con 0x7fa638103560 2024-05-10T11:17:31.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:31.830+0000 7fa62e7fc700 1 --2- 172.21.15.100:0/2988679397 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa6240b2880 0x7fa6240b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:31.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:31.831+0000 7fa63ce6f700 1 --2- 172.21.15.100:0/2988679397 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa6240b2880 0x7fa6240b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:31.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:31.831+0000 7fa62e7fc700 1 -- 172.21.15.100:0/2988679397 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fa6340e2450 con 0x7fa638103560 2024-05-10T11:17:31.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:31.831+0000 7fa63ce6f700 1 --2- 172.21.15.100:0/2988679397 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa6240b2880 0x7fa6240b4d40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa63807ac80 tx=0x7fa62800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:17:31.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:31.834+0000 7fa62e7fc700 1 -- 172.21.15.100:0/2988679397 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fa63401f020 con 0x7fa638103560 2024-05-10T11:17:32.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:32.082+0000 7fa63f8d4700 1 -- 172.21.15.100:0/2988679397 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa63819ad70 con 0x7fa638103560 2024-05-10T11:17:32.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:32.083+0000 7fa62e7fc700 1 -- 172.21.15.100:0/2988679397 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11894 (secure 0 0 0) 0x7fa6340a9e10 con 0x7fa638103560 2024-05-10T11:17:32.083 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:17:32.083 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2072225012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2072225012},{"type":"v1","addr":"172.21.15.100:6835","nonce":2072225012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/401939540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":401939540},{"type":"v1","addr":"172.21.15.100:6837","nonce":401939540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14571,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/3946311025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3946311025},{"type":"v1","addr":"172.21.15.100:6839","nonce":3946311025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3736371660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3736371660},{"type":"v1","addr":"172.21.15.100:6841","nonce":3736371660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/2741810791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2741810791},{"type":"v1","addr":"172.21.15.178:6833","nonce":2741810791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1704891041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1704891041},{"type":"v1","addr":"172.21.15.188:6833","nonce":1704891041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T11:14:27.242452+0000","modified":"2024-05-10T11:15:35.672461+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24439,"mds_1":24437,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/2124076312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2124076312},{"type":"v1","addr":"172.21.15.178:6835","nonce":2124076312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/3224358425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3224358425},{"type":"v1","addr":"172.21.15.178:6837","nonce":3224358425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/515547690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":515547690},{"type":"v1","addr":"172.21.15.188:6835","nonce":515547690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/1943342620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1943342620},{"type":"v1","addr":"172.21.15.188:6837","nonce":1943342620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.178:6839/3760985404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3760985404},{"type":"v1","addr":"172.21.15.178:6839","nonce":3760985404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.188:6839/3194917070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":3194917070},{"type":"v1","addr":"172.21.15.188:6839","nonce":3194917070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24439,"qdb_cluster":[24437,24439,24427]},"id":1}]} 2024-05-10T11:17:32.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:32.085+0000 7fa63f8d4700 1 -- 172.21.15.100:0/2988679397 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa6240b2880 msgr2=0x7fa6240b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:32.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:32.085+0000 7fa63f8d4700 1 --2- 172.21.15.100:0/2988679397 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa6240b2880 0x7fa6240b4d40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa63807ac80 tx=0x7fa62800a380 comp rx=0 tx=0).stop 2024-05-10T11:17:32.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:32.085+0000 7fa63f8d4700 1 -- 172.21.15.100:0/2988679397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa638103560 msgr2=0x7fa63819f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:32.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:32.085+0000 7fa63f8d4700 1 --2- 172.21.15.100:0/2988679397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa638103560 0x7fa63819f6c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa63400b530 tx=0x7fa63400b840 comp rx=0 tx=0).stop 2024-05-10T11:17:32.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:32.085+0000 7fa63f8d4700 1 -- 172.21.15.100:0/2988679397 shutdown_connections 2024-05-10T11:17:32.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:32.085+0000 7fa63f8d4700 1 --2- 172.21.15.100:0/2988679397 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa6240b2880 0x7fa6240b4d40 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:32.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:32.085+0000 7fa63f8d4700 1 --2- 172.21.15.100:0/2988679397 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa638109540 0x7fa638199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:32.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:32.085+0000 7fa63f8d4700 1 --2- 172.21.15.100:0/2988679397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa638103560 0x7fa63819f6c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:32.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:32.085+0000 7fa63f8d4700 1 --2- 172.21.15.100:0/2988679397 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa638103f20 0x7fa63819fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:32.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:32.085+0000 7fa63f8d4700 1 -- 172.21.15.100:0/2988679397 >> 172.21.15.100:0/2988679397 conn(0x7fa63807a660 msgr2=0x7fa638102180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:17:32.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:32.085+0000 7fa63f8d4700 1 -- 172.21.15.100:0/2988679397 shutdown_connections 2024-05-10T11:17:32.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:32.085+0000 7fa63f8d4700 1 -- 172.21.15.100:0/2988679397 wait complete. 2024-05-10T11:17:32.086 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 26 2024-05-10T11:17:32.413 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:17:32 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/2988679397' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:17:32.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:17:32 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/2988679397' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:17:32.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:17:32 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/2988679397' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:17:32.859 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-10T11:14:27.242452+0000', 'modified': '2024-05-10T11:15:35.672461+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24439, 'mds_1': 24437, 'mds_2': 24427}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24395': {'gid': 24395, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.178:6835/2124076312', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6834', 'nonce': 2124076312}, {'type': 'v1', 'addr': '172.21.15.178:6835', 'nonce': 2124076312}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24413': {'gid': 24413, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.178:6837/3224358425', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6836', 'nonce': 3224358425}, {'type': 'v1', 'addr': '172.21.15.178:6837', 'nonce': 3224358425}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24421': {'gid': 24421, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.188:6835/515547690', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6834', 'nonce': 515547690}, {'type': 'v1', 'addr': '172.21.15.188:6835', 'nonce': 515547690}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24427': {'gid': 24427, 'name': 'c', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 25, 'addr': '172.21.15.188:6837/1943342620', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6836', 'nonce': 1943342620}, {'type': 'v1', 'addr': '172.21.15.188:6837', 'nonce': 1943342620}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24437': {'gid': 24437, 'name': 'b', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 21, 'addr': '172.21.15.178:6839/3760985404', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6838', 'nonce': 3760985404}, {'type': 'v1', 'addr': '172.21.15.178:6839', 'nonce': 3760985404}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24439': {'gid': 24439, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.188:6839/3194917070', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6838', 'nonce': 3194917070}, {'type': 'v1', 'addr': '172.21.15.188:6839', 'nonce': 3194917070}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24439, 'qdb_cluster': [24437, 24439, 24427]} 2024-05-10T11:17:32.860 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-10T11:17:32.860 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.c status 2024-05-10T11:17:33.104 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:17:33.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:17:33 smithi100 ceph-mon[30296]: pgmap v395: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T11:17:33.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:17:33 smithi178 ceph-mon[33359]: pgmap v395: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T11:17:33.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:17:33 smithi188 ceph-mon[34075]: pgmap v395: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T11:17:34.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.752+0000 7f359ea7b700 1 -- 172.21.15.100:0/2927483702 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3598109540 msgr2=0x7f3598109920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:34.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.752+0000 7f359ea7b700 1 --2- 172.21.15.100:0/2927483702 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3598109540 0x7f3598109920 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f358c0099c0 tx=0x7f358c009cd0 comp rx=0 tx=0).stop 2024-05-10T11:17:34.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.752+0000 7f359ea7b700 1 -- 172.21.15.100:0/2927483702 shutdown_connections 2024-05-10T11:17:34.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.752+0000 7f359ea7b700 1 --2- 172.21.15.100:0/2927483702 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3598103f20 0x7f3598110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:34.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.752+0000 7f359ea7b700 1 --2- 172.21.15.100:0/2927483702 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3598109540 0x7f3598109920 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:34.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.752+0000 7f359ea7b700 1 --2- 172.21.15.100:0/2927483702 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3598103560 0x7f35981039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:34.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.752+0000 7f359ea7b700 1 -- 172.21.15.100:0/2927483702 >> 172.21.15.100:0/2927483702 conn(0x7f359807a660 msgr2=0x7f359807aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:17:34.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.752+0000 7f359ea7b700 1 -- 172.21.15.100:0/2927483702 shutdown_connections 2024-05-10T11:17:34.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.752+0000 7f359ea7b700 1 -- 172.21.15.100:0/2927483702 wait complete. 2024-05-10T11:17:34.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.752+0000 7f359ea7b700 1 Processor -- start 2024-05-10T11:17:34.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.752+0000 7f359ea7b700 1 -- start start 2024-05-10T11:17:34.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.753+0000 7f359ea7b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3598103560 0x7f359819f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:34.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.753+0000 7f359ea7b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3598103f20 0x7f359819fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:34.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.753+0000 7f359ea7b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3598109540 0x7f35981997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:34.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.753+0000 7f359ea7b700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f35981134d0 con 0x7f3598109540 2024-05-10T11:17:34.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.753+0000 7f359ea7b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3598113350 con 0x7f3598103560 2024-05-10T11:17:34.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.753+0000 7f359ea7b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3598113650 con 0x7f3598103f20 2024-05-10T11:17:34.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.753+0000 7f359d018700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3598109540 0x7f35981997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:34.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.753+0000 7f359d018700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3598109540 0x7f35981997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35552/0 (socket says 172.21.15.100:35552) 2024-05-10T11:17:34.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.753+0000 7f359d018700 1 -- 172.21.15.100:0/1151778819 learned_addr learned my addr 172.21.15.100:0/1151778819 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:17:34.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.753+0000 7f3597fff700 1 --2- 172.21.15.100:0/1151778819 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3598103f20 0x7f359819fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:34.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.753+0000 7f359c817700 1 --2- 172.21.15.100:0/1151778819 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3598103560 0x7f359819f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:34.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.753+0000 7f359d018700 1 -- 172.21.15.100:0/1151778819 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3598103f20 msgr2=0x7f359819fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:34.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.753+0000 7f359d018700 1 --2- 172.21.15.100:0/1151778819 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3598103f20 0x7f359819fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:34.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.753+0000 7f359d018700 1 -- 172.21.15.100:0/1151778819 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3598103560 msgr2=0x7f359819f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:34.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.753+0000 7f359d018700 1 --2- 172.21.15.100:0/1151778819 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3598103560 0x7f359819f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:34.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.753+0000 7f359d018700 1 -- 172.21.15.100:0/1151778819 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3598199fb0 con 0x7f3598109540 2024-05-10T11:17:34.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.753+0000 7f359d018700 1 --2- 172.21.15.100:0/1151778819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3598109540 0x7f35981997d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3590007ee0 tx=0x7f359000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:17:34.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.753+0000 7f3595ffb700 1 -- 172.21.15.100:0/1151778819 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3590004bb0 con 0x7f3598109540 2024-05-10T11:17:34.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.753+0000 7f3595ffb700 1 -- 172.21.15.100:0/1151778819 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3590004d50 con 0x7f3598109540 2024-05-10T11:17:34.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.753+0000 7f359ea7b700 1 -- 172.21.15.100:0/1151778819 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f359819a2a0 con 0x7f3598109540 2024-05-10T11:17:34.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.753+0000 7f3595ffb700 1 -- 172.21.15.100:0/1151778819 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35900058b0 con 0x7f3598109540 2024-05-10T11:17:34.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.754+0000 7f359ea7b700 1 -- 172.21.15.100:0/1151778819 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35981a64b0 con 0x7f3598109540 2024-05-10T11:17:34.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.756+0000 7f3595ffb700 1 -- 172.21.15.100:0/1151778819 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f359001a020 con 0x7f3598109540 2024-05-10T11:17:34.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.756+0000 7f3595ffb700 1 --2- 172.21.15.100:0/1151778819 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f35880b2880 0x7f35880b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:34.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.757+0000 7f359c817700 1 --2- 172.21.15.100:0/1151778819 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f35880b2880 0x7f35880b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:34.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.757+0000 7f3595ffb700 1 -- 172.21.15.100:0/1151778819 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f35900e1c10 con 0x7f3598109540 2024-05-10T11:17:34.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.757+0000 7f359c817700 1 --2- 172.21.15.100:0/1151778819 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f35880b2880 0x7f35880b4d40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f358c000c00 tx=0x7f358c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:17:34.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.757+0000 7f35a930eb80 1 Processor -- start 2024-05-10T11:17:34.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.757+0000 7f35a930eb80 1 -- start start 2024-05-10T11:17:34.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.757+0000 7f35a930eb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cdf6782160 0x55cdf6782540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:34.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.757+0000 7f35a930eb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdf6782a80 0x55cdf677c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:34.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.757+0000 7f35a930eb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdf677cd00 0x55cdf677d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:34.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.757+0000 7f35a930eb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55cdf66ce8f0 con 0x55cdf6782a80 2024-05-10T11:17:34.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.757+0000 7f35a930eb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55cdf6633d50 con 0x55cdf6782160 2024-05-10T11:17:34.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.757+0000 7f35a930eb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55cdf66ca8b0 con 0x55cdf677cd00 2024-05-10T11:17:34.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.758+0000 7f3597fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdf6782a80 0x55cdf677c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:34.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.758+0000 7f3597fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdf6782a80 0x55cdf677c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35556/0 (socket says 172.21.15.100:35556) 2024-05-10T11:17:34.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.758+0000 7f3597fff700 1 -- 172.21.15.100:0/3321591937 learned_addr learned my addr 172.21.15.100:0/3321591937 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:17:34.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.758+0000 7f359c817700 1 --2- 172.21.15.100:0/3321591937 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cdf6782160 0x55cdf6782540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:34.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.758+0000 7f359d018700 1 --2- 172.21.15.100:0/3321591937 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdf677cd00 0x55cdf677d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:34.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.758+0000 7f3597fff700 1 -- 172.21.15.100:0/3321591937 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdf677cd00 msgr2=0x55cdf677d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:17:34.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.758+0000 7f3597fff700 1 --2- 172.21.15.100:0/3321591937 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdf677cd00 0x55cdf677d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:34.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.758+0000 7f3597fff700 1 -- 172.21.15.100:0/3321591937 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cdf6782160 msgr2=0x55cdf6782540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:17:34.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.758+0000 7f3597fff700 1 --2- 172.21.15.100:0/3321591937 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cdf6782160 0x55cdf6782540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:34.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.758+0000 7f3597fff700 1 -- 172.21.15.100:0/3321591937 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdf661b880 con 0x55cdf6782a80 2024-05-10T11:17:34.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.758+0000 7f3597fff700 1 --2- 172.21.15.100:0/3321591937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdf6782a80 0x55cdf677c7c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f358400bd30 tx=0x7f35840106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aad9b56b4ac6ee5e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:17:34.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.758+0000 7f3582ffd700 1 -- 172.21.15.100:0/3321591937 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3584010e10 con 0x55cdf6782a80 2024-05-10T11:17:34.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.758+0000 7f3582ffd700 1 -- 172.21.15.100:0/3321591937 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f35840044c0 con 0x55cdf6782a80 2024-05-10T11:17:34.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.758+0000 7f3582ffd700 1 -- 172.21.15.100:0/3321591937 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3584011440 con 0x55cdf6782a80 2024-05-10T11:17:34.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.759+0000 7f35a930eb80 1 -- 172.21.15.100:0/3321591937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdf6782a80 msgr2=0x55cdf677c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:34.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.759+0000 7f35a930eb80 1 --2- 172.21.15.100:0/3321591937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdf6782a80 0x55cdf677c7c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f358400bd30 tx=0x7f35840106b0 comp rx=0 tx=0).stop 2024-05-10T11:17:34.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.759+0000 7f35a930eb80 1 -- 172.21.15.100:0/3321591937 shutdown_connections 2024-05-10T11:17:34.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.759+0000 7f35a930eb80 1 --2- 172.21.15.100:0/3321591937 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdf677cd00 0x55cdf677d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:34.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.759+0000 7f35a930eb80 1 --2- 172.21.15.100:0/3321591937 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cdf6782160 0x55cdf6782540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:34.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.759+0000 7f35a930eb80 1 --2- 172.21.15.100:0/3321591937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdf6782a80 0x55cdf677c7c0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:34.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.759+0000 7f35a930eb80 1 -- 172.21.15.100:0/3321591937 >> 172.21.15.100:0/3321591937 conn(0x55cdf66a8b20 msgr2=0x55cdf6777250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:17:34.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.759+0000 7f35a930eb80 1 -- 172.21.15.100:0/3321591937 shutdown_connections 2024-05-10T11:17:34.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.759+0000 7f35a930eb80 1 -- 172.21.15.100:0/3321591937 wait complete. 2024-05-10T11:17:34.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.759+0000 7f35a930eb80 1 -- start start 2024-05-10T11:17:34.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.760+0000 7f35a930eb80 1 Processor -- start 2024-05-10T11:17:34.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.760+0000 7f35a930eb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdf677cd00 0x55cdf6788760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:34.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.760+0000 7f35a930eb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdf6782160 0x55cdf6785b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:34.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.760+0000 7f35a930eb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cdf6782a80 0x55cdf6786040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:34.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.760+0000 7f35a930eb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55cdf6789400 con 0x55cdf677cd00 2024-05-10T11:17:34.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.760+0000 7f35a930eb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55cdf6687e80 con 0x55cdf6782a80 2024-05-10T11:17:34.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.760+0000 7f35a930eb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55cdf6789580 con 0x55cdf6782160 2024-05-10T11:17:34.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.760+0000 7f35a930eb80 10 client.0 _ll_register_callbacks cb 0x55cdf6662560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:17:34.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.760+0000 7f359c817700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdf677cd00 0x55cdf6788760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:34.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.760+0000 7f359c817700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdf677cd00 0x55cdf6788760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35568/0 (socket says 172.21.15.100:35568) 2024-05-10T11:17:34.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.760+0000 7f359c817700 1 -- 172.21.15.100:0/480343941 learned_addr learned my addr 172.21.15.100:0/480343941 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:17:34.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.760+0000 7f359d018700 1 --2- 172.21.15.100:0/480343941 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cdf6782a80 0x55cdf6786040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:34.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.760+0000 7f3597fff700 1 --2- 172.21.15.100:0/480343941 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdf6782160 0x55cdf6785b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:34.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.760+0000 7f359c817700 1 -- 172.21.15.100:0/480343941 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdf6782160 msgr2=0x55cdf6785b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:17:34.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.760+0000 7f359c817700 1 --2- 172.21.15.100:0/480343941 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdf6782160 0x55cdf6785b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:34.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.760+0000 7f359c817700 1 -- 172.21.15.100:0/480343941 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cdf6782a80 msgr2=0x55cdf6786040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:17:34.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.760+0000 7f359c817700 1 --2- 172.21.15.100:0/480343941 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cdf6782a80 0x55cdf6786040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:34.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.760+0000 7f359c817700 1 -- 172.21.15.100:0/480343941 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdf6685710 con 0x55cdf677cd00 2024-05-10T11:17:34.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.761+0000 7f3597fff700 1 --2- 172.21.15.100:0/480343941 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdf6782160 0x55cdf6785b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:17:34.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.761+0000 7f359c817700 1 --2- 172.21.15.100:0/480343941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdf677cd00 0x55cdf6788760 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f358c00b580 tx=0x7f358c009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c769b3583b573c79 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:17:34.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.761+0000 7f3577fff700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:17:34.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.761+0000 7f3577fff700 1 -- 172.21.15.100:0/480343941 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358c035060 con 0x55cdf677cd00 2024-05-10T11:17:34.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.761+0000 7f3577fff700 1 -- 172.21.15.100:0/480343941 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f358c003a90 con 0x55cdf677cd00 2024-05-10T11:17:34.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.761+0000 7f35a930eb80 1 -- 172.21.15.100:0/480343941 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35840044c0 con 0x55cdf677cd00 2024-05-10T11:17:34.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.761+0000 7f3577fff700 1 -- 172.21.15.100:0/480343941 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f358c0375a0 con 0x55cdf677cd00 2024-05-10T11:17:34.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.761+0000 7f35a930eb80 10 client.15039 fetch_fsmap learned FSMap version 26 2024-05-10T11:17:34.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.761+0000 7f35a930eb80 1 -- 172.21.15.100:0/480343941 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cdf6793d30 con 0x55cdf677cd00 2024-05-10T11:17:34.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.761+0000 7f3577fff700 1 -- 172.21.15.100:0/480343941 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f35840044c0 con 0x55cdf677cd00 2024-05-10T11:17:34.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.761+0000 7f3577fff700 1 -- 172.21.15.100:0/480343941 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f358c026430 con 0x55cdf677cd00 2024-05-10T11:17:34.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.761+0000 7f35a930eb80 10 client.15039 fetch_fsmap finished waiting for FSMap version 26 2024-05-10T11:17:34.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.762+0000 7f35a930eb80 10 client.15039 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-10T11:17:34.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.762+0000 7f35a930eb80 20 client.15039 populate_metadata read hostname 'smithi100' 2024-05-10T11:17:34.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.762+0000 7f35a930eb80 1 --2- 172.21.15.100:0/480343941 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55cdf67962f0 0x55cdf6797760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:34.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.762+0000 7f35a930eb80 4 client.15039 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:17:34.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.762+0000 7f35a930eb80 1 -- 172.21.15.100:0/480343941 --> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cdf66dc2d0 con 0x55cdf67962f0 2024-05-10T11:17:34.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.762+0000 7f3597fff700 1 --2- 172.21.15.100:0/480343941 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55cdf67962f0 0x55cdf6797760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:34.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.762+0000 7f3597fff700 1 --2- 172.21.15.100:0/480343941 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55cdf67962f0 0x55cdf6797760 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=abe2d11dc0954887 server_cookie=42bc3db21dfd3681 in_seq=0 out_seq=0 2024-05-10T11:17:34.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.762+0000 7f3577fff700 10 client.15039 ms_handle_connect on v2:172.21.15.188:6836/1943342620 2024-05-10T11:17:34.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.763+0000 7f3577fff700 1 -- 172.21.15.100:0/480343941 <== mds.2 v2:172.21.15.188:6836/1943342620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55cdf66dc2d0 con 0x55cdf67962f0 2024-05-10T11:17:34.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.763+0000 7f3577fff700 10 client.15039 handle_command_reply: tid=0 2024-05-10T11:17:34.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.763+0000 7f35a930eb80 1 client.15039 shutdown 2024-05-10T11:17:34.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.763+0000 7f35a930eb80 1 -- 172.21.15.100:0/480343941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdf677cd00 msgr2=0x55cdf6788760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:34.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.763+0000 7f35a930eb80 1 --2- 172.21.15.100:0/480343941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdf677cd00 0x55cdf6788760 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f358c00b580 tx=0x7f358c009cd0 comp rx=0 tx=0).stop 2024-05-10T11:17:34.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.763+0000 7f35a930eb80 1 -- 172.21.15.100:0/480343941 shutdown_connections 2024-05-10T11:17:34.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.763+0000 7f35a930eb80 1 --2- 172.21.15.100:0/480343941 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55cdf67962f0 0x55cdf6797760 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:34.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.763+0000 7f35a930eb80 1 --2- 172.21.15.100:0/480343941 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cdf6782a80 0x55cdf6786040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:34.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.763+0000 7f35a930eb80 1 --2- 172.21.15.100:0/480343941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdf677cd00 0x55cdf6788760 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:34.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.763+0000 7f3577fff700 0 client.15039 ms_handle_reset on v2:172.21.15.188:6836/1943342620 2024-05-10T11:17:34.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.763+0000 7f35a930eb80 1 --2- 172.21.15.100:0/480343941 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdf6782160 0x55cdf6785b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:34.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.763+0000 7f35a930eb80 1 -- 172.21.15.100:0/480343941 >> 172.21.15.100:0/480343941 conn(0x55cdf66a8b20 msgr2=0x55cdf6771ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:17:34.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.763+0000 7f35a930eb80 1 -- 172.21.15.100:0/480343941 shutdown_connections 2024-05-10T11:17:34.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.763+0000 7f35a930eb80 1 -- 172.21.15.100:0/480343941 wait complete. 2024-05-10T11:17:34.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.763+0000 7f35a930eb80 20 client.15039 trim_cache size 0 max 16384 2024-05-10T11:17:34.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.784+0000 7f35a930eb80 1 Processor -- start 2024-05-10T11:17:34.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.784+0000 7f35a930eb80 1 -- start start 2024-05-10T11:17:34.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.784+0000 7f35a930eb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cdf677cd00 0x55cdf677c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:34.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.784+0000 7f35a930eb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdf6782160 0x55cdf6771bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:34.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.784+0000 7f35a930eb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdf6782a80 0x55cdf6782e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:34.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.784+0000 7f35a930eb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55cdf6795aa0 con 0x55cdf6782160 2024-05-10T11:17:34.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.784+0000 7f35a930eb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55cdf6796170 con 0x55cdf677cd00 2024-05-10T11:17:34.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.784+0000 7f35a930eb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55cdf6796840 con 0x55cdf6782a80 2024-05-10T11:17:34.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.784+0000 7f3597fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdf6782160 0x55cdf6771bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:34.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.784+0000 7f3597fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdf6782160 0x55cdf6771bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35584/0 (socket says 172.21.15.100:35584) 2024-05-10T11:17:34.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.784+0000 7f3597fff700 1 -- 172.21.15.100:0/3276438184 learned_addr learned my addr 172.21.15.100:0/3276438184 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:17:34.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.785+0000 7f359c817700 1 --2- 172.21.15.100:0/3276438184 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cdf677cd00 0x55cdf677c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:34.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.785+0000 7f359d018700 1 --2- 172.21.15.100:0/3276438184 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdf6782a80 0x55cdf6782e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:34.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.785+0000 7f3597fff700 1 -- 172.21.15.100:0/3276438184 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdf6782a80 msgr2=0x55cdf6782e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:17:34.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.785+0000 7f3597fff700 1 --2- 172.21.15.100:0/3276438184 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdf6782a80 0x55cdf6782e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:34.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.785+0000 7f3597fff700 1 -- 172.21.15.100:0/3276438184 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cdf677cd00 msgr2=0x55cdf677c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:17:34.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.785+0000 7f3597fff700 1 --2- 172.21.15.100:0/3276438184 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cdf677cd00 0x55cdf677c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:34.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.785+0000 7f3597fff700 1 -- 172.21.15.100:0/3276438184 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdf6784090 con 0x55cdf6782160 2024-05-10T11:17:34.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.785+0000 7f359d018700 1 --2- 172.21.15.100:0/3276438184 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdf6782a80 0x55cdf6782e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:17:34.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.785+0000 7f3597fff700 1 --2- 172.21.15.100:0/3276438184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdf6782160 0x55cdf6771bf0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f358400bd30 tx=0x7f358400b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b1c8be111d64827 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:17:34.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.785+0000 7f35827fc700 1 -- 172.21.15.100:0/3276438184 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3584010e10 con 0x55cdf6782160 2024-05-10T11:17:34.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.785+0000 7f35827fc700 1 -- 172.21.15.100:0/3276438184 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3584003cb0 con 0x55cdf6782160 2024-05-10T11:17:34.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.785+0000 7f35827fc700 1 -- 172.21.15.100:0/3276438184 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35840056b0 con 0x55cdf6782160 2024-05-10T11:17:34.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.786+0000 7f35a930eb80 1 -- 172.21.15.100:0/3276438184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdf6782160 msgr2=0x55cdf6771bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:34.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.786+0000 7f35a930eb80 1 --2- 172.21.15.100:0/3276438184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdf6782160 0x55cdf6771bf0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f358400bd30 tx=0x7f358400b650 comp rx=0 tx=0).stop 2024-05-10T11:17:34.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.786+0000 7f35a930eb80 1 -- 172.21.15.100:0/3276438184 shutdown_connections 2024-05-10T11:17:34.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.786+0000 7f35a930eb80 1 --2- 172.21.15.100:0/3276438184 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdf6782a80 0x55cdf6782e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:34.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.786+0000 7f35a930eb80 1 --2- 172.21.15.100:0/3276438184 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cdf677cd00 0x55cdf677c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:34.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.786+0000 7f35a930eb80 1 --2- 172.21.15.100:0/3276438184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdf6782160 0x55cdf6771bf0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:34.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.786+0000 7f35a930eb80 1 -- 172.21.15.100:0/3276438184 >> 172.21.15.100:0/3276438184 conn(0x55cdf66a8b20 msgr2=0x55cdf6784410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:17:34.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.786+0000 7f35a930eb80 1 -- 172.21.15.100:0/3276438184 shutdown_connections 2024-05-10T11:17:34.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.786+0000 7f35a930eb80 1 -- 172.21.15.100:0/3276438184 wait complete. 2024-05-10T11:17:34.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.786+0000 7f35a930eb80 1 -- start start 2024-05-10T11:17:34.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.786+0000 7f35a930eb80 1 Processor -- start 2024-05-10T11:17:34.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.787+0000 7f35a930eb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdf677cd00 0x55cdf67711c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:34.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.787+0000 7f35a930eb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdf6782160 0x55cdf6771700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:34.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.787+0000 7f35a930eb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cdf6782a80 0x55cdf6771c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:34.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.787+0000 7f35a930eb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55cdf67975e0 con 0x55cdf677cd00 2024-05-10T11:17:34.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.787+0000 7f35a930eb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55cdf6796f10 con 0x55cdf6782a80 2024-05-10T11:17:34.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.787+0000 7f35a930eb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55cdf6797cb0 con 0x55cdf6782160 2024-05-10T11:17:34.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.787+0000 7f35a930eb80 10 client.0 _ll_register_callbacks cb 0x55cdf6662560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:17:34.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.787+0000 7f359c817700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdf677cd00 0x55cdf67711c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:34.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.787+0000 7f359c817700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdf677cd00 0x55cdf67711c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35586/0 (socket says 172.21.15.100:35586) 2024-05-10T11:17:34.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.787+0000 7f359c817700 1 -- 172.21.15.100:0/2972530342 learned_addr learned my addr 172.21.15.100:0/2972530342 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:17:34.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.787+0000 7f3597fff700 1 --2- 172.21.15.100:0/2972530342 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdf6782160 0x55cdf6771700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:34.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.787+0000 7f359d018700 1 --2- 172.21.15.100:0/2972530342 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cdf6782a80 0x55cdf6771c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:34.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.787+0000 7f359c817700 1 -- 172.21.15.100:0/2972530342 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdf6782160 msgr2=0x55cdf6771700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:17:34.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.787+0000 7f359c817700 1 --2- 172.21.15.100:0/2972530342 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdf6782160 0x55cdf6771700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:34.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.787+0000 7f359c817700 1 -- 172.21.15.100:0/2972530342 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cdf6782a80 msgr2=0x55cdf6771c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:17:34.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.787+0000 7f359c817700 1 --2- 172.21.15.100:0/2972530342 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cdf6782a80 0x55cdf6771c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:34.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.787+0000 7f359c817700 1 -- 172.21.15.100:0/2972530342 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdf679f350 con 0x55cdf677cd00 2024-05-10T11:17:34.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.787+0000 7f359c817700 1 --2- 172.21.15.100:0/2972530342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdf677cd00 0x55cdf67711c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f358c0098e0 tx=0x7f358c00be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19291fafd11a9456 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:17:34.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.787+0000 7f3577fff700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:17:34.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.788+0000 7f3577fff700 1 -- 172.21.15.100:0/2972530342 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358c044060 con 0x55cdf677cd00 2024-05-10T11:17:34.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.788+0000 7f35a930eb80 1 -- 172.21.15.100:0/2972530342 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3584003cb0 con 0x55cdf677cd00 2024-05-10T11:17:34.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.788+0000 7f3577fff700 1 -- 172.21.15.100:0/2972530342 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f358c003a90 con 0x55cdf677cd00 2024-05-10T11:17:34.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.788+0000 7f3577fff700 1 -- 172.21.15.100:0/2972530342 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f358c04b050 con 0x55cdf677cd00 2024-05-10T11:17:34.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.788+0000 7f35a930eb80 10 client.15045 fetch_fsmap learned FSMap version 26 2024-05-10T11:17:34.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.788+0000 7f35a930eb80 1 -- 172.21.15.100:0/2972530342 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cdf6801bb0 con 0x55cdf677cd00 2024-05-10T11:17:34.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.788+0000 7f3577fff700 1 -- 172.21.15.100:0/2972530342 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3584003cb0 con 0x55cdf677cd00 2024-05-10T11:17:34.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.788+0000 7f3577fff700 1 -- 172.21.15.100:0/2972530342 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f358c03e070 con 0x55cdf677cd00 2024-05-10T11:17:34.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.788+0000 7f35a930eb80 10 client.15045 fetch_fsmap finished waiting for FSMap version 26 2024-05-10T11:17:34.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.788+0000 7f35a930eb80 10 client.15045 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-10T11:17:34.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.788+0000 7f35a930eb80 20 client.15045 populate_metadata read hostname 'smithi100' 2024-05-10T11:17:34.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.788+0000 7f35a930eb80 1 --2- 172.21.15.100:0/2972530342 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55cdf67962f0 0x55cdf68054b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:34.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.788+0000 7f35a930eb80 4 client.15045 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T11:17:34.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.788+0000 7f35a930eb80 1 -- 172.21.15.100:0/2972530342 --> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] -- command(tid 0: {"prefix": "status"}) -- 0x55cdf66d6370 con 0x55cdf67962f0 2024-05-10T11:17:34.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.789+0000 7f3597fff700 1 --2- 172.21.15.100:0/2972530342 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55cdf67962f0 0x55cdf68054b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:34.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.789+0000 7f3597fff700 1 --2- 172.21.15.100:0/2972530342 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55cdf67962f0 0x55cdf68054b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1f7622a40be092fa server_cookie=591e2f16b4d3c3a9 in_seq=0 out_seq=0 2024-05-10T11:17:34.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.789+0000 7f3577fff700 10 client.15045 ms_handle_connect on v2:172.21.15.188:6836/1943342620 2024-05-10T11:17:34.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.789+0000 7f3577fff700 1 -- 172.21.15.100:0/2972530342 <== mds.2 v2:172.21.15.188:6836/1943342620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55cdf66d6370 con 0x55cdf67962f0 2024-05-10T11:17:34.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.789+0000 7f3577fff700 10 client.15045 handle_command_reply: tid=0 2024-05-10T11:17:34.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.789+0000 7f35a930eb80 1 client.15045 shutdown 2024-05-10T11:17:34.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.789+0000 7f35a930eb80 1 -- 172.21.15.100:0/2972530342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdf677cd00 msgr2=0x55cdf67711c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:34.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.789+0000 7f35a930eb80 1 --2- 172.21.15.100:0/2972530342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdf677cd00 0x55cdf67711c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f358c0098e0 tx=0x7f358c00be70 comp rx=0 tx=0).stop 2024-05-10T11:17:34.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.789+0000 7f35a930eb80 1 -- 172.21.15.100:0/2972530342 shutdown_connections 2024-05-10T11:17:34.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.789+0000 7f35a930eb80 1 --2- 172.21.15.100:0/2972530342 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55cdf67962f0 0x55cdf68054b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:34.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.789+0000 7f35a930eb80 1 --2- 172.21.15.100:0/2972530342 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cdf6782a80 0x55cdf6771c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:34.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.789+0000 7f35a930eb80 1 --2- 172.21.15.100:0/2972530342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cdf677cd00 0x55cdf67711c0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:34.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.789+0000 7f35a930eb80 1 --2- 172.21.15.100:0/2972530342 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdf6782160 0x55cdf6771700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:34.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.789+0000 7f35a930eb80 1 -- 172.21.15.100:0/2972530342 >> 172.21.15.100:0/2972530342 conn(0x55cdf66a8b20 msgr2=0x55cdf6782e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:17:34.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.789+0000 7f3577fff700 0 client.15045 ms_handle_reset on v2:172.21.15.188:6836/1943342620 2024-05-10T11:17:34.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.790+0000 7f35a930eb80 1 -- 172.21.15.100:0/2972530342 shutdown_connections 2024-05-10T11:17:34.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.790+0000 7f35a930eb80 1 -- 172.21.15.100:0/2972530342 wait complete. 2024-05-10T11:17:34.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.790+0000 7f35a930eb80 20 client.15045 trim_cache size 0 max 16384 2024-05-10T11:17:34.795 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:17:34.795 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_fsid": "20a542cc-0ebd-11ef-bc98-c7b262605968", 2024-05-10T11:17:34.795 INFO:teuthology.orchestra.run.smithi100.stdout: "whoami": 2, 2024-05-10T11:17:34.795 INFO:teuthology.orchestra.run.smithi100.stdout: "id": 24427, 2024-05-10T11:17:34.795 INFO:teuthology.orchestra.run.smithi100.stdout: "want_state": "up:active", 2024-05-10T11:17:34.795 INFO:teuthology.orchestra.run.smithi100.stdout: "state": "up:active", 2024-05-10T11:17:34.795 INFO:teuthology.orchestra.run.smithi100.stdout: "fs_name": "cephfs", 2024-05-10T11:17:34.795 INFO:teuthology.orchestra.run.smithi100.stdout: "rank_uptime": 125.001455526, 2024-05-10T11:17:34.795 INFO:teuthology.orchestra.run.smithi100.stdout: "mdsmap_epoch": 26, 2024-05-10T11:17:34.795 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch": 89, 2024-05-10T11:17:34.795 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch_barrier": 89, 2024-05-10T11:17:34.795 INFO:teuthology.orchestra.run.smithi100.stdout: "uptime": 220.99710349099999 2024-05-10T11:17:34.796 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:17:34.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.790+0000 7f359ea7b700 1 -- 172.21.15.100:0/1151778819 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f35880b2880 msgr2=0x7f35880b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:34.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.790+0000 7f359ea7b700 1 --2- 172.21.15.100:0/1151778819 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f35880b2880 0x7f35880b4d40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f358c000c00 tx=0x7f358c005fb0 comp rx=0 tx=0).stop 2024-05-10T11:17:34.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.790+0000 7f359ea7b700 1 -- 172.21.15.100:0/1151778819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3598109540 msgr2=0x7f35981997d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:34.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.790+0000 7f359ea7b700 1 --2- 172.21.15.100:0/1151778819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3598109540 0x7f35981997d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3590007ee0 tx=0x7f359000d3b0 comp rx=0 tx=0).stop 2024-05-10T11:17:34.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.790+0000 7f359ea7b700 1 -- 172.21.15.100:0/1151778819 shutdown_connections 2024-05-10T11:17:34.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.790+0000 7f359ea7b700 1 --2- 172.21.15.100:0/1151778819 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f35880b2880 0x7f35880b4d40 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:34.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.790+0000 7f359ea7b700 1 --2- 172.21.15.100:0/1151778819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3598109540 0x7f35981997d0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:34.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.790+0000 7f359ea7b700 1 --2- 172.21.15.100:0/1151778819 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3598103560 0x7f359819f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:34.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.791+0000 7f359ea7b700 1 --2- 172.21.15.100:0/1151778819 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3598103f20 0x7f359819fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:34.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.791+0000 7f359ea7b700 1 -- 172.21.15.100:0/1151778819 >> 172.21.15.100:0/1151778819 conn(0x7f359807a660 msgr2=0x7f3598102330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:17:34.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.791+0000 7f359ea7b700 1 -- 172.21.15.100:0/1151778819 shutdown_connections 2024-05-10T11:17:34.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:34.791+0000 7f359ea7b700 1 -- 172.21.15.100:0/1151778819 wait complete. 2024-05-10T11:17:35.476 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.b status 2024-05-10T11:17:35.720 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:17:35.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:17:35 smithi178 ceph-mon[33359]: pgmap v396: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-10T11:17:35.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:17:35 smithi188 ceph-mon[34075]: pgmap v396: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-10T11:17:35.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:17:35 smithi100 ceph-mon[30296]: pgmap v396: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-10T11:17:37.427 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:17:37 smithi100 ceph-mon[30296]: pgmap v397: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T11:17:37.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.432+0000 7fe5aeaf7700 1 -- 172.21.15.100:0/2798342577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5a8108b70 msgr2=0x7fe5a810f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:37.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.432+0000 7fe5aeaf7700 1 --2- 172.21.15.100:0/2798342577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5a8108b70 0x7fe5a810f330 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe5a000b0d0 tx=0x7fe5a000b3e0 comp rx=0 tx=0).stop 2024-05-10T11:17:37.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.432+0000 7fe5aeaf7700 1 -- 172.21.15.100:0/2798342577 shutdown_connections 2024-05-10T11:17:37.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.432+0000 7fe5aeaf7700 1 --2- 172.21.15.100:0/2798342577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5a8108b70 0x7fe5a810f330 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:37.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.432+0000 7fe5aeaf7700 1 --2- 172.21.15.100:0/2798342577 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5a8106340 0x7fe5a8106720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:37.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.432+0000 7fe5aeaf7700 1 --2- 172.21.15.100:0/2798342577 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5a81081b0 0x7fe5a8108630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:37.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.432+0000 7fe5aeaf7700 1 -- 172.21.15.100:0/2798342577 >> 172.21.15.100:0/2798342577 conn(0x7fe5a80776c0 msgr2=0x7fe5a8077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:17:37.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.432+0000 7fe5aeaf7700 1 -- 172.21.15.100:0/2798342577 shutdown_connections 2024-05-10T11:17:37.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.432+0000 7fe5aeaf7700 1 -- 172.21.15.100:0/2798342577 wait complete. 2024-05-10T11:17:37.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.432+0000 7fe5aeaf7700 1 Processor -- start 2024-05-10T11:17:37.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.432+0000 7fe5aeaf7700 1 -- start start 2024-05-10T11:17:37.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.432+0000 7fe5aeaf7700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5a8106340 0x7fe5a819a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:37.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.432+0000 7fe5aeaf7700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5a81081b0 0x7fe5a819adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:37.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.432+0000 7fe5aeaf7700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5a8108b70 0x7fe5a81a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:37.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.432+0000 7fe5aeaf7700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a81124c0 con 0x7fe5a81081b0 2024-05-10T11:17:37.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.433+0000 7fe5aeaf7700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a8112340 con 0x7fe5a8106340 2024-05-10T11:17:37.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.433+0000 7fe5aeaf7700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a8112640 con 0x7fe5a8108b70 2024-05-10T11:17:37.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.433+0000 7fe5a7fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5a81081b0 0x7fe5a819adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:37.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.433+0000 7fe5a7fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5a81081b0 0x7fe5a819adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59852/0 (socket says 172.21.15.100:59852) 2024-05-10T11:17:37.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.433+0000 7fe5a7fff700 1 -- 172.21.15.100:0/192482969 learned_addr learned my addr 172.21.15.100:0/192482969 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:17:37.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.433+0000 7fe5ac893700 1 --2- 172.21.15.100:0/192482969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5a8106340 0x7fe5a819a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:37.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.433+0000 7fe5ad094700 1 --2- 172.21.15.100:0/192482969 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5a8108b70 0x7fe5a81a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:37.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.433+0000 7fe5a7fff700 1 -- 172.21.15.100:0/192482969 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5a8108b70 msgr2=0x7fe5a81a5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:37.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.433+0000 7fe5a7fff700 1 --2- 172.21.15.100:0/192482969 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5a8108b70 0x7fe5a81a5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:37.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.433+0000 7fe5a7fff700 1 -- 172.21.15.100:0/192482969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5a8106340 msgr2=0x7fe5a819a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:37.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.433+0000 7fe5a7fff700 1 --2- 172.21.15.100:0/192482969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5a8106340 0x7fe5a819a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:37.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.433+0000 7fe5a7fff700 1 -- 172.21.15.100:0/192482969 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5a81a5e40 con 0x7fe5a81081b0 2024-05-10T11:17:37.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.433+0000 7fe5a7fff700 1 --2- 172.21.15.100:0/192482969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5a81081b0 0x7fe5a819adc0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe59400c920 tx=0x7fe59400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:17:37.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.433+0000 7fe5a5ffb700 1 -- 172.21.15.100:0/192482969 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe594004a30 con 0x7fe5a81081b0 2024-05-10T11:17:37.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.433+0000 7fe5aeaf7700 1 -- 172.21.15.100:0/192482969 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5a81a6070 con 0x7fe5a81081b0 2024-05-10T11:17:37.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.433+0000 7fe5a5ffb700 1 -- 172.21.15.100:0/192482969 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe5940054b0 con 0x7fe5a81081b0 2024-05-10T11:17:37.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.433+0000 7fe5a5ffb700 1 -- 172.21.15.100:0/192482969 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe594007b70 con 0x7fe5a81081b0 2024-05-10T11:17:37.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.434+0000 7fe5aeaf7700 1 -- 172.21.15.100:0/192482969 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5a81a65d0 con 0x7fe5a81081b0 2024-05-10T11:17:37.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.436+0000 7fe5a5ffb700 1 -- 172.21.15.100:0/192482969 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fe594007d10 con 0x7fe5a81081b0 2024-05-10T11:17:37.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.436+0000 7fe5a5ffb700 1 --2- 172.21.15.100:0/192482969 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe5980b2880 0x7fe5980b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:37.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.436+0000 7fe5ac893700 1 --2- 172.21.15.100:0/192482969 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe5980b2880 0x7fe5980b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:37.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.437+0000 7fe5a5ffb700 1 -- 172.21.15.100:0/192482969 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fe5940e1fb0 con 0x7fe5a81081b0 2024-05-10T11:17:37.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.437+0000 7fe5ac893700 1 --2- 172.21.15.100:0/192482969 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe5980b2880 0x7fe5980b4d40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe59c004630 tx=0x7fe59c00b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:17:37.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.437+0000 7fe5b938ab80 1 Processor -- start 2024-05-10T11:17:37.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.437+0000 7fe5b938ab80 1 -- start start 2024-05-10T11:17:37.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.437+0000 7fe5b938ab80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0c9320160 0x55a0c9320540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:37.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.437+0000 7fe5b938ab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a0c9320a80 0x55a0c931a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:37.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.437+0000 7fe5b938ab80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0c931ad00 0x55a0c931b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:37.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.437+0000 7fe5b938ab80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55a0c926c8f0 con 0x55a0c931ad00 2024-05-10T11:17:37.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.437+0000 7fe5b938ab80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a0c91d1d50 con 0x55a0c9320160 2024-05-10T11:17:37.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.437+0000 7fe5b938ab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55a0c92688b0 con 0x55a0c9320a80 2024-05-10T11:17:37.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.438+0000 7fe5ad094700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0c931ad00 0x55a0c931b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:37.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.438+0000 7fe5ad094700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0c931ad00 0x55a0c931b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59854/0 (socket says 172.21.15.100:59854) 2024-05-10T11:17:37.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.438+0000 7fe5ad094700 1 -- 172.21.15.100:0/3472667850 learned_addr learned my addr 172.21.15.100:0/3472667850 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:17:37.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.438+0000 7fe5ac893700 1 --2- 172.21.15.100:0/3472667850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0c9320160 0x55a0c9320540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:37.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.438+0000 7fe5a7fff700 1 --2- 172.21.15.100:0/3472667850 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a0c9320a80 0x55a0c931a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:37.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.438+0000 7fe5ad094700 1 -- 172.21.15.100:0/3472667850 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a0c9320a80 msgr2=0x55a0c931a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:17:37.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.438+0000 7fe5ad094700 1 --2- 172.21.15.100:0/3472667850 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a0c9320a80 0x55a0c931a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:37.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.438+0000 7fe5ad094700 1 -- 172.21.15.100:0/3472667850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0c9320160 msgr2=0x55a0c9320540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:17:37.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.438+0000 7fe5ad094700 1 --2- 172.21.15.100:0/3472667850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0c9320160 0x55a0c9320540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:37.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.438+0000 7fe5ad094700 1 -- 172.21.15.100:0/3472667850 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0c91b9880 con 0x55a0c931ad00 2024-05-10T11:17:37.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.438+0000 7fe5ad094700 1 --2- 172.21.15.100:0/3472667850 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0c931ad00 0x55a0c931b140 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0005ac0 tx=0x7fe5a0005af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=765d669f82850a43 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:17:37.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.438+0000 7fe592ffd700 1 -- 172.21.15.100:0/3472667850 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a0009450 con 0x55a0c931ad00 2024-05-10T11:17:37.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.438+0000 7fe592ffd700 1 -- 172.21.15.100:0/3472667850 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe5a0020070 con 0x55a0c931ad00 2024-05-10T11:17:37.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.438+0000 7fe592ffd700 1 -- 172.21.15.100:0/3472667850 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a00058b0 con 0x55a0c931ad00 2024-05-10T11:17:37.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.439+0000 7fe5b938ab80 1 -- 172.21.15.100:0/3472667850 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0c931ad00 msgr2=0x55a0c931b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:37.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.439+0000 7fe5b938ab80 1 --2- 172.21.15.100:0/3472667850 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0c931ad00 0x55a0c931b140 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0005ac0 tx=0x7fe5a0005af0 comp rx=0 tx=0).stop 2024-05-10T11:17:37.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.439+0000 7fe5b938ab80 1 -- 172.21.15.100:0/3472667850 shutdown_connections 2024-05-10T11:17:37.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.439+0000 7fe5b938ab80 1 --2- 172.21.15.100:0/3472667850 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0c931ad00 0x55a0c931b140 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:37.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.439+0000 7fe5b938ab80 1 --2- 172.21.15.100:0/3472667850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0c9320160 0x55a0c9320540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:37.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.439+0000 7fe5b938ab80 1 --2- 172.21.15.100:0/3472667850 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a0c9320a80 0x55a0c931a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:37.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.439+0000 7fe5b938ab80 1 -- 172.21.15.100:0/3472667850 >> 172.21.15.100:0/3472667850 conn(0x55a0c9246b20 msgr2=0x55a0c9315250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:17:37.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.439+0000 7fe5b938ab80 1 -- 172.21.15.100:0/3472667850 shutdown_connections 2024-05-10T11:17:37.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.439+0000 7fe5b938ab80 1 -- 172.21.15.100:0/3472667850 wait complete. 2024-05-10T11:17:37.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.439+0000 7fe5b938ab80 1 -- start start 2024-05-10T11:17:37.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.440+0000 7fe5b938ab80 1 Processor -- start 2024-05-10T11:17:37.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.440+0000 7fe5b938ab80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0c931ad00 0x55a0c9326660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:37.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.440+0000 7fe5b938ab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a0c9320160 0x55a0c9323a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:37.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.440+0000 7fe5b938ab80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0c9320a80 0x55a0c9323f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:37.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.440+0000 7fe5b938ab80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55a0c93273a0 con 0x55a0c931ad00 2024-05-10T11:17:37.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.440+0000 7fe5b938ab80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a0c9225e80 con 0x55a0c9320a80 2024-05-10T11:17:37.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.440+0000 7fe5b938ab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55a0c9327520 con 0x55a0c9320160 2024-05-10T11:17:37.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.440+0000 7fe5b938ab80 10 client.0 _ll_register_callbacks cb 0x55a0c9200560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:17:37.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.440+0000 7fe5ac893700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0c931ad00 0x55a0c9326660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:37.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.440+0000 7fe5ac893700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0c931ad00 0x55a0c9326660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59860/0 (socket says 172.21.15.100:59860) 2024-05-10T11:17:37.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.440+0000 7fe5ac893700 1 -- 172.21.15.100:0/2880856810 learned_addr learned my addr 172.21.15.100:0/2880856810 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:17:37.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.440+0000 7fe5a7fff700 1 --2- 172.21.15.100:0/2880856810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a0c9320160 0x55a0c9323a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:37.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.440+0000 7fe5ad094700 1 --2- 172.21.15.100:0/2880856810 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0c9320a80 0x55a0c9323f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:37.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.440+0000 7fe5ac893700 1 -- 172.21.15.100:0/2880856810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a0c9320160 msgr2=0x55a0c9323a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:17:37.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.440+0000 7fe5ac893700 1 --2- 172.21.15.100:0/2880856810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a0c9320160 0x55a0c9323a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:37.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.440+0000 7fe5ac893700 1 -- 172.21.15.100:0/2880856810 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0c9320a80 msgr2=0x55a0c9323f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:17:37.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.440+0000 7fe5ac893700 1 --2- 172.21.15.100:0/2880856810 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0c9320a80 0x55a0c9323f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:37.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.440+0000 7fe5ac893700 1 -- 172.21.15.100:0/2880856810 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0c9223710 con 0x55a0c931ad00 2024-05-10T11:17:37.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.441+0000 7fe5ac893700 1 --2- 172.21.15.100:0/2880856810 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0c931ad00 0x55a0c9326660 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe59c003eb0 tx=0x7fe59c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e70cdc985e9d4b12 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:17:37.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.441+0000 7fe587fff700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:17:37.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.441+0000 7fe587fff700 1 -- 172.21.15.100:0/2880856810 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59c011ce0 con 0x55a0c931ad00 2024-05-10T11:17:37.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.441+0000 7fe587fff700 1 -- 172.21.15.100:0/2880856810 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fe59c028470 con 0x55a0c931ad00 2024-05-10T11:17:37.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.441+0000 7fe5b938ab80 1 -- 172.21.15.100:0/2880856810 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5a0020070 con 0x55a0c931ad00 2024-05-10T11:17:37.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.441+0000 7fe587fff700 1 -- 172.21.15.100:0/2880856810 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe59c031050 con 0x55a0c931ad00 2024-05-10T11:17:37.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.441+0000 7fe5b938ab80 10 client.15057 fetch_fsmap learned FSMap version 26 2024-05-10T11:17:37.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.441+0000 7fe5b938ab80 1 -- 172.21.15.100:0/2880856810 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a0c9331c70 con 0x55a0c931ad00 2024-05-10T11:17:37.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.441+0000 7fe587fff700 1 -- 172.21.15.100:0/2880856810 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe5a0020070 con 0x55a0c931ad00 2024-05-10T11:17:37.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.441+0000 7fe587fff700 1 -- 172.21.15.100:0/2880856810 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fe59c01e430 con 0x55a0c931ad00 2024-05-10T11:17:37.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.441+0000 7fe5b938ab80 10 client.15057 fetch_fsmap finished waiting for FSMap version 26 2024-05-10T11:17:37.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.441+0000 7fe5b938ab80 10 client.15057 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-10T11:17:37.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.441+0000 7fe5b938ab80 20 client.15057 populate_metadata read hostname 'smithi100' 2024-05-10T11:17:37.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.441+0000 7fe5b938ab80 1 --2- 172.21.15.100:0/2880856810 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55a0c93342c0 0x55a0c9335730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:37.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.441+0000 7fe5b938ab80 4 client.15057 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:17:37.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.441+0000 7fe5b938ab80 1 -- 172.21.15.100:0/2880856810 --> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0c927a2d0 con 0x55a0c93342c0 2024-05-10T11:17:37.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.442+0000 7fe5a7fff700 1 --2- 172.21.15.100:0/2880856810 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55a0c93342c0 0x55a0c9335730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:37.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.442+0000 7fe5a7fff700 1 --2- 172.21.15.100:0/2880856810 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55a0c93342c0 0x55a0c9335730 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1318ee00c6ffa488 server_cookie=7aa27dbc19519407 in_seq=0 out_seq=0 2024-05-10T11:17:37.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.442+0000 7fe587fff700 10 client.15057 ms_handle_connect on v2:172.21.15.178:6838/3760985404 2024-05-10T11:17:37.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.443+0000 7fe587fff700 1 -- 172.21.15.100:0/2880856810 <== mds.1 v2:172.21.15.178:6838/3760985404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a0c927a2d0 con 0x55a0c93342c0 2024-05-10T11:17:37.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.443+0000 7fe587fff700 10 client.15057 handle_command_reply: tid=0 2024-05-10T11:17:37.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.443+0000 7fe5b938ab80 1 client.15057 shutdown 2024-05-10T11:17:37.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.443+0000 7fe5b938ab80 1 -- 172.21.15.100:0/2880856810 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0c931ad00 msgr2=0x55a0c9326660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:37.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.443+0000 7fe5b938ab80 1 --2- 172.21.15.100:0/2880856810 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0c931ad00 0x55a0c9326660 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe59c003eb0 tx=0x7fe59c0047c0 comp rx=0 tx=0).stop 2024-05-10T11:17:37.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.443+0000 7fe5b938ab80 1 -- 172.21.15.100:0/2880856810 shutdown_connections 2024-05-10T11:17:37.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.443+0000 7fe5b938ab80 1 --2- 172.21.15.100:0/2880856810 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55a0c93342c0 0x55a0c9335730 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:37.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.443+0000 7fe5b938ab80 1 --2- 172.21.15.100:0/2880856810 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0c9320a80 0x55a0c9323f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:37.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.443+0000 7fe5b938ab80 1 --2- 172.21.15.100:0/2880856810 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0c931ad00 0x55a0c9326660 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:37.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.443+0000 7fe587fff700 0 client.15057 ms_handle_reset on v2:172.21.15.178:6838/3760985404 2024-05-10T11:17:37.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.443+0000 7fe5b938ab80 1 --2- 172.21.15.100:0/2880856810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a0c9320160 0x55a0c9323a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:37.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.443+0000 7fe5b938ab80 1 -- 172.21.15.100:0/2880856810 >> 172.21.15.100:0/2880856810 conn(0x55a0c9246b20 msgr2=0x55a0c930fae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:17:37.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.443+0000 7fe5b938ab80 1 -- 172.21.15.100:0/2880856810 shutdown_connections 2024-05-10T11:17:37.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.443+0000 7fe5b938ab80 1 -- 172.21.15.100:0/2880856810 wait complete. 2024-05-10T11:17:37.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.443+0000 7fe5b938ab80 20 client.15057 trim_cache size 0 max 16384 2024-05-10T11:17:37.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.464+0000 7fe5b938ab80 1 Processor -- start 2024-05-10T11:17:37.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.464+0000 7fe5b938ab80 1 -- start start 2024-05-10T11:17:37.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.464+0000 7fe5b938ab80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0c931ad00 0x55a0c931a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:37.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.464+0000 7fe5b938ab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a0c9320160 0x55a0c930fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:37.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.464+0000 7fe5b938ab80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0c9320a80 0x55a0c9320e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:37.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.464+0000 7fe5b938ab80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55a0c9333a70 con 0x55a0c931ad00 2024-05-10T11:17:37.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.464+0000 7fe5b938ab80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a0c9334140 con 0x55a0c9320a80 2024-05-10T11:17:37.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.464+0000 7fe5b938ab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55a0c9334810 con 0x55a0c9320160 2024-05-10T11:17:37.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.464+0000 7fe5ac893700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0c931ad00 0x55a0c931a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:37.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.464+0000 7fe5ac893700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0c931ad00 0x55a0c931a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59868/0 (socket says 172.21.15.100:59868) 2024-05-10T11:17:37.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.464+0000 7fe5ac893700 1 -- 172.21.15.100:0/1158204750 learned_addr learned my addr 172.21.15.100:0/1158204750 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:17:37.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.465+0000 7fe5ac893700 1 -- 172.21.15.100:0/1158204750 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a0c9320160 msgr2=0x55a0c930fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:17:37.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.465+0000 7fe5ac893700 1 --2- 172.21.15.100:0/1158204750 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a0c9320160 0x55a0c930fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:37.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.465+0000 7fe5ac893700 1 -- 172.21.15.100:0/1158204750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0c9320a80 msgr2=0x55a0c9320e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:17:37.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.465+0000 7fe5ac893700 1 --2- 172.21.15.100:0/1158204750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0c9320a80 0x55a0c9320e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:37.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.465+0000 7fe5ac893700 1 -- 172.21.15.100:0/1158204750 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0c931fba0 con 0x55a0c931ad00 2024-05-10T11:17:37.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.465+0000 7fe5ac893700 1 --2- 172.21.15.100:0/1158204750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0c931ad00 0x55a0c931a7c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe59c005a00 tx=0x7fe59c005a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e32a9d1878ca1de server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:17:37.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.465+0000 7fe5927fc700 1 -- 172.21.15.100:0/1158204750 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59c003680 con 0x55a0c931ad00 2024-05-10T11:17:37.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.465+0000 7fe5927fc700 1 -- 172.21.15.100:0/1158204750 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe59c023070 con 0x55a0c931ad00 2024-05-10T11:17:37.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.465+0000 7fe5927fc700 1 -- 172.21.15.100:0/1158204750 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59c004a90 con 0x55a0c931ad00 2024-05-10T11:17:37.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.465+0000 7fe5b938ab80 1 -- 172.21.15.100:0/1158204750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0c931ad00 msgr2=0x55a0c931a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:37.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.465+0000 7fe5b938ab80 1 --2- 172.21.15.100:0/1158204750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0c931ad00 0x55a0c931a7c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe59c005a00 tx=0x7fe59c005a30 comp rx=0 tx=0).stop 2024-05-10T11:17:37.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.466+0000 7fe5b938ab80 1 -- 172.21.15.100:0/1158204750 shutdown_connections 2024-05-10T11:17:37.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.466+0000 7fe5b938ab80 1 --2- 172.21.15.100:0/1158204750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0c9320a80 0x55a0c9320e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:37.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.466+0000 7fe5b938ab80 1 --2- 172.21.15.100:0/1158204750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0c931ad00 0x55a0c931a7c0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:37.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.466+0000 7fe5b938ab80 1 --2- 172.21.15.100:0/1158204750 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a0c9320160 0x55a0c930fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:37.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.466+0000 7fe5b938ab80 1 -- 172.21.15.100:0/1158204750 >> 172.21.15.100:0/1158204750 conn(0x55a0c9246b20 msgr2=0x55a0c93222f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:17:37.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.466+0000 7fe5b938ab80 1 -- 172.21.15.100:0/1158204750 shutdown_connections 2024-05-10T11:17:37.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.466+0000 7fe5b938ab80 1 -- 172.21.15.100:0/1158204750 wait complete. 2024-05-10T11:17:37.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.466+0000 7fe5b938ab80 1 -- start start 2024-05-10T11:17:37.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.466+0000 7fe5b938ab80 1 Processor -- start 2024-05-10T11:17:37.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.466+0000 7fe5b938ab80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0c931ad00 0x55a0c933ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:37.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.466+0000 7fe5b938ab80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0c9320160 0x55a0c933f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:37.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.466+0000 7fe5b938ab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a0c9320a80 0x55a0c9339b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:37.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.466+0000 7fe5b938ab80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55a0c93355b0 con 0x55a0c9320160 2024-05-10T11:17:37.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.466+0000 7fe5b938ab80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a0c9334ee0 con 0x55a0c931ad00 2024-05-10T11:17:37.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.466+0000 7fe5b938ab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55a0c9335c80 con 0x55a0c9320a80 2024-05-10T11:17:37.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.466+0000 7fe5b938ab80 10 client.0 _ll_register_callbacks cb 0x55a0c9200560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:17:37.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.467+0000 7fe5a7fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0c9320160 0x55a0c933f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:37.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.467+0000 7fe5a7fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0c9320160 0x55a0c933f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59880/0 (socket says 172.21.15.100:59880) 2024-05-10T11:17:37.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.467+0000 7fe5a7fff700 1 -- 172.21.15.100:0/3101082514 learned_addr learned my addr 172.21.15.100:0/3101082514 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:17:37.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.467+0000 7fe5ad094700 1 --2- 172.21.15.100:0/3101082514 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a0c9320a80 0x55a0c9339b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:37.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.467+0000 7fe5ac893700 1 --2- 172.21.15.100:0/3101082514 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0c931ad00 0x55a0c933ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:37.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.467+0000 7fe5a7fff700 1 -- 172.21.15.100:0/3101082514 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a0c9320a80 msgr2=0x55a0c9339b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:17:37.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.467+0000 7fe5a7fff700 1 --2- 172.21.15.100:0/3101082514 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a0c9320a80 0x55a0c9339b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:37.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.467+0000 7fe5a7fff700 1 -- 172.21.15.100:0/3101082514 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0c931ad00 msgr2=0x55a0c933ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:17:37.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.467+0000 7fe5a7fff700 1 --2- 172.21.15.100:0/3101082514 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0c931ad00 0x55a0c933ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:37.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.467+0000 7fe5a7fff700 1 -- 172.21.15.100:0/3101082514 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0c933a1d0 con 0x55a0c9320160 2024-05-10T11:17:37.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.467+0000 7fe5ac893700 1 --2- 172.21.15.100:0/3101082514 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0c931ad00 0x55a0c933ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:17:37.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.467+0000 7fe5a7fff700 1 --2- 172.21.15.100:0/3101082514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0c9320160 0x55a0c933f0f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe5940df6f0 tx=0x7fe5940a9a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=285eb20aa07b833e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:17:37.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.467+0000 7fe587fff700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:17:37.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.467+0000 7fe587fff700 1 -- 172.21.15.100:0/3101082514 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59401d040 con 0x55a0c9320160 2024-05-10T11:17:37.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.467+0000 7fe587fff700 1 -- 172.21.15.100:0/3101082514 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fe5940a94d0 con 0x55a0c9320160 2024-05-10T11:17:37.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.467+0000 7fe5b938ab80 1 -- 172.21.15.100:0/3101082514 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe59c023070 con 0x55a0c9320160 2024-05-10T11:17:37.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.467+0000 7fe587fff700 1 -- 172.21.15.100:0/3101082514 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe5940ad620 con 0x55a0c9320160 2024-05-10T11:17:37.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.467+0000 7fe5b938ab80 10 client.15063 fetch_fsmap learned FSMap version 26 2024-05-10T11:17:37.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.467+0000 7fe5b938ab80 1 -- 172.21.15.100:0/3101082514 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a0c93a0100 con 0x55a0c9320160 2024-05-10T11:17:37.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.468+0000 7fe587fff700 1 -- 172.21.15.100:0/3101082514 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe59c023070 con 0x55a0c9320160 2024-05-10T11:17:37.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.468+0000 7fe587fff700 1 -- 172.21.15.100:0/3101082514 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fe594068ce0 con 0x55a0c9320160 2024-05-10T11:17:37.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.468+0000 7fe5b938ab80 10 client.15063 fetch_fsmap finished waiting for FSMap version 26 2024-05-10T11:17:37.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.468+0000 7fe5b938ab80 10 client.15063 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-10T11:17:37.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.468+0000 7fe5b938ab80 20 client.15063 populate_metadata read hostname 'smithi100' 2024-05-10T11:17:37.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.468+0000 7fe5b938ab80 1 --2- 172.21.15.100:0/3101082514 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55a0c93a4c50 0x55a0c93a3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:37.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.468+0000 7fe5b938ab80 4 client.15063 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T11:17:37.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.468+0000 7fe5b938ab80 1 -- 172.21.15.100:0/3101082514 --> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] -- command(tid 0: {"prefix": "status"}) -- 0x55a0c9274370 con 0x55a0c93a4c50 2024-05-10T11:17:37.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.468+0000 7fe5ac893700 1 --2- 172.21.15.100:0/3101082514 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55a0c93a4c50 0x55a0c93a3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:37.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.469+0000 7fe587fff700 10 client.15063 ms_handle_connect on v2:172.21.15.178:6838/3760985404 2024-05-10T11:17:37.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.469+0000 7fe5ac893700 1 --2- 172.21.15.100:0/3101082514 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55a0c93a4c50 0x55a0c93a3960 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4452c3ab9a0e9ec1 server_cookie=806bc31cc4004b02 in_seq=0 out_seq=0 2024-05-10T11:17:37.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.469+0000 7fe587fff700 1 -- 172.21.15.100:0/3101082514 <== mds.1 v2:172.21.15.178:6838/3760985404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55a0c9274370 con 0x55a0c93a4c50 2024-05-10T11:17:37.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.469+0000 7fe587fff700 10 client.15063 handle_command_reply: tid=0 2024-05-10T11:17:37.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.469+0000 7fe5b938ab80 1 client.15063 shutdown 2024-05-10T11:17:37.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.469+0000 7fe5b938ab80 1 -- 172.21.15.100:0/3101082514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0c9320160 msgr2=0x55a0c933f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:37.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.469+0000 7fe5b938ab80 1 --2- 172.21.15.100:0/3101082514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0c9320160 0x55a0c933f0f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe5940df6f0 tx=0x7fe5940a9a20 comp rx=0 tx=0).stop 2024-05-10T11:17:37.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.469+0000 7fe5b938ab80 1 -- 172.21.15.100:0/3101082514 shutdown_connections 2024-05-10T11:17:37.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.469+0000 7fe5b938ab80 1 --2- 172.21.15.100:0/3101082514 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55a0c93a4c50 0x55a0c93a3960 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:37.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.469+0000 7fe5b938ab80 1 --2- 172.21.15.100:0/3101082514 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a0c9320a80 0x55a0c9339b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:37.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.469+0000 7fe5b938ab80 1 --2- 172.21.15.100:0/3101082514 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0c931ad00 0x55a0c933ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:37.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.469+0000 7fe5b938ab80 1 --2- 172.21.15.100:0/3101082514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0c9320160 0x55a0c933f0f0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:37.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.469+0000 7fe5b938ab80 1 -- 172.21.15.100:0/3101082514 >> 172.21.15.100:0/3101082514 conn(0x55a0c9246b20 msgr2=0x55a0c9320e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:17:37.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.469+0000 7fe587fff700 0 client.15063 ms_handle_reset on v2:172.21.15.178:6838/3760985404 2024-05-10T11:17:37.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.469+0000 7fe5b938ab80 1 -- 172.21.15.100:0/3101082514 shutdown_connections 2024-05-10T11:17:37.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.469+0000 7fe5b938ab80 1 -- 172.21.15.100:0/3101082514 wait complete. 2024-05-10T11:17:37.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.469+0000 7fe5b938ab80 20 client.15063 trim_cache size 0 max 16384 2024-05-10T11:17:37.475 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:17:37.475 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_fsid": "20a542cc-0ebd-11ef-bc98-c7b262605968", 2024-05-10T11:17:37.475 INFO:teuthology.orchestra.run.smithi100.stdout: "whoami": 1, 2024-05-10T11:17:37.475 INFO:teuthology.orchestra.run.smithi100.stdout: "id": 24437, 2024-05-10T11:17:37.475 INFO:teuthology.orchestra.run.smithi100.stdout: "want_state": "up:active", 2024-05-10T11:17:37.476 INFO:teuthology.orchestra.run.smithi100.stdout: "state": "up:active", 2024-05-10T11:17:37.476 INFO:teuthology.orchestra.run.smithi100.stdout: "fs_name": "cephfs", 2024-05-10T11:17:37.476 INFO:teuthology.orchestra.run.smithi100.stdout: "rank_uptime": 128.690617288, 2024-05-10T11:17:37.476 INFO:teuthology.orchestra.run.smithi100.stdout: "mdsmap_epoch": 26, 2024-05-10T11:17:37.476 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch": 89, 2024-05-10T11:17:37.476 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch_barrier": 89, 2024-05-10T11:17:37.476 INFO:teuthology.orchestra.run.smithi100.stdout: "uptime": 206.91899011999999 2024-05-10T11:17:37.476 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:17:37.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.470+0000 7fe5aeaf7700 1 -- 172.21.15.100:0/192482969 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe5980b2880 msgr2=0x7fe5980b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:37.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.470+0000 7fe5aeaf7700 1 --2- 172.21.15.100:0/192482969 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe5980b2880 0x7fe5980b4d40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe59c004630 tx=0x7fe59c00b3c0 comp rx=0 tx=0).stop 2024-05-10T11:17:37.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.470+0000 7fe5aeaf7700 1 -- 172.21.15.100:0/192482969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5a81081b0 msgr2=0x7fe5a819adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:37.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.470+0000 7fe5aeaf7700 1 --2- 172.21.15.100:0/192482969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5a81081b0 0x7fe5a819adc0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe59400c920 tx=0x7fe59400cc30 comp rx=0 tx=0).stop 2024-05-10T11:17:37.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.470+0000 7fe5aeaf7700 1 -- 172.21.15.100:0/192482969 shutdown_connections 2024-05-10T11:17:37.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.470+0000 7fe5aeaf7700 1 --2- 172.21.15.100:0/192482969 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe5980b2880 0x7fe5980b4d40 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:37.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.470+0000 7fe5aeaf7700 1 --2- 172.21.15.100:0/192482969 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5a8108b70 0x7fe5a81a5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:37.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.470+0000 7fe5aeaf7700 1 --2- 172.21.15.100:0/192482969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5a8106340 0x7fe5a819a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:37.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.470+0000 7fe5aeaf7700 1 --2- 172.21.15.100:0/192482969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5a81081b0 0x7fe5a819adc0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:37.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.470+0000 7fe5aeaf7700 1 -- 172.21.15.100:0/192482969 >> 172.21.15.100:0/192482969 conn(0x7fe5a80776c0 msgr2=0x7fe5a80ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:17:37.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.470+0000 7fe5aeaf7700 1 -- 172.21.15.100:0/192482969 shutdown_connections 2024-05-10T11:17:37.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:37.470+0000 7fe5aeaf7700 1 -- 172.21.15.100:0/192482969 wait complete. 2024-05-10T11:17:37.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:17:37 smithi178 ceph-mon[33359]: pgmap v397: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T11:17:37.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:17:37 smithi188 ceph-mon[34075]: pgmap v397: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T11:17:38.228 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.i status 2024-05-10T11:17:38.473 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:17:39.620 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:17:39 smithi100 ceph-mon[30296]: pgmap v398: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T11:17:39.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:17:39 smithi178 ceph-mon[33359]: pgmap v398: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T11:17:39.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:17:39 smithi188 ceph-mon[34075]: pgmap v398: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T11:17:40.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.212+0000 7f3c46799700 1 -- 172.21.15.100:0/3445128214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c40108400 msgr2=0x7f3c401087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:40.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.212+0000 7f3c46799700 1 --2- 172.21.15.100:0/3445128214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c40108400 0x7f3c401087e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3c30009990 tx=0x7f3c30009ca0 comp rx=0 tx=0).stop 2024-05-10T11:17:40.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.212+0000 7f3c46799700 1 -- 172.21.15.100:0/3445128214 shutdown_connections 2024-05-10T11:17:40.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.212+0000 7f3c46799700 1 --2- 172.21.15.100:0/3445128214 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c40102d50 0x7f3c4010f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:40.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.212+0000 7f3c46799700 1 --2- 172.21.15.100:0/3445128214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c40108400 0x7f3c401087e0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:40.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.212+0000 7f3c46799700 1 --2- 172.21.15.100:0/3445128214 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c40102390 0x7f3c40102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:40.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.212+0000 7f3c46799700 1 -- 172.21.15.100:0/3445128214 >> 172.21.15.100:0/3445128214 conn(0x7f3c40079310 msgr2=0x7f3c40079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:17:40.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.212+0000 7f3c46799700 1 -- 172.21.15.100:0/3445128214 shutdown_connections 2024-05-10T11:17:40.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.212+0000 7f3c46799700 1 -- 172.21.15.100:0/3445128214 wait complete. 2024-05-10T11:17:40.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.212+0000 7f3c46799700 1 Processor -- start 2024-05-10T11:17:40.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.213+0000 7f3c46799700 1 -- start start 2024-05-10T11:17:40.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.213+0000 7f3c46799700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c40102390 0x7f3c4019f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:40.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.213+0000 7f3c46799700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c40102d50 0x7f3c4019fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:40.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.213+0000 7f3c46799700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c40108400 0x7f3c40199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:40.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.213+0000 7f3c46799700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3c40112400 con 0x7f3c40108400 2024-05-10T11:17:40.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.213+0000 7f3c46799700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3c40112280 con 0x7f3c40102390 2024-05-10T11:17:40.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.213+0000 7f3c46799700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3c40112580 con 0x7f3c40102d50 2024-05-10T11:17:40.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.213+0000 7f3c44d36700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c40108400 0x7f3c40199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:40.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.213+0000 7f3c44d36700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c40108400 0x7f3c40199970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59896/0 (socket says 172.21.15.100:59896) 2024-05-10T11:17:40.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.213+0000 7f3c44d36700 1 -- 172.21.15.100:0/3980215198 learned_addr learned my addr 172.21.15.100:0/3980215198 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:17:40.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.213+0000 7f3c3ffff700 1 --2- 172.21.15.100:0/3980215198 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c40102390 0x7f3c4019f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:40.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.213+0000 7f3c3f7fe700 1 --2- 172.21.15.100:0/3980215198 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c40102d50 0x7f3c4019fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:40.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.213+0000 7f3c44d36700 1 -- 172.21.15.100:0/3980215198 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c40102d50 msgr2=0x7f3c4019fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:40.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.213+0000 7f3c44d36700 1 --2- 172.21.15.100:0/3980215198 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c40102d50 0x7f3c4019fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:40.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.213+0000 7f3c44d36700 1 -- 172.21.15.100:0/3980215198 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c40102390 msgr2=0x7f3c4019f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:40.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.213+0000 7f3c44d36700 1 --2- 172.21.15.100:0/3980215198 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c40102390 0x7f3c4019f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:40.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.213+0000 7f3c44d36700 1 -- 172.21.15.100:0/3980215198 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c4019a0d0 con 0x7f3c40108400 2024-05-10T11:17:40.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.214+0000 7f3c3f7fe700 1 --2- 172.21.15.100:0/3980215198 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c40102d50 0x7f3c4019fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:17:40.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.214+0000 7f3c44d36700 1 --2- 172.21.15.100:0/3980215198 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c40108400 0x7f3c40199970 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3c3800ada0 tx=0x7f3c3800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:17:40.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.214+0000 7f3c3d7fa700 1 -- 172.21.15.100:0/3980215198 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c38017020 con 0x7f3c40108400 2024-05-10T11:17:40.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.214+0000 7f3c46799700 1 -- 172.21.15.100:0/3980215198 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c4019a3c0 con 0x7f3c40108400 2024-05-10T11:17:40.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.214+0000 7f3c46799700 1 -- 172.21.15.100:0/3980215198 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c401a6630 con 0x7f3c40108400 2024-05-10T11:17:40.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.214+0000 7f3c3d7fa700 1 -- 172.21.15.100:0/3980215198 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3c3800ce10 con 0x7f3c40108400 2024-05-10T11:17:40.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.214+0000 7f3c3d7fa700 1 -- 172.21.15.100:0/3980215198 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c38005280 con 0x7f3c40108400 2024-05-10T11:17:40.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.217+0000 7f3c3d7fa700 1 -- 172.21.15.100:0/3980215198 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f3c38005520 con 0x7f3c40108400 2024-05-10T11:17:40.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.218+0000 7f3c3d7fa700 1 --2- 172.21.15.100:0/3980215198 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f3c2c0b2900 0x7f3c2c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:40.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.218+0000 7f3c3ffff700 1 --2- 172.21.15.100:0/3980215198 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f3c2c0b2900 0x7f3c2c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:40.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.218+0000 7f3c3d7fa700 1 -- 172.21.15.100:0/3980215198 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f3c380e1bf0 con 0x7f3c40108400 2024-05-10T11:17:40.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.218+0000 7f3c5102cb80 1 Processor -- start 2024-05-10T11:17:40.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.218+0000 7f3c5102cb80 1 -- start start 2024-05-10T11:17:40.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.218+0000 7f3c3ffff700 1 --2- 172.21.15.100:0/3980215198 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f3c2c0b2900 0x7f3c2c0b4dc0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3c3000b5c0 tx=0x7f3c3001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:17:40.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.218+0000 7f3c5102cb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5627975b4480 0x5627975b4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:40.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.218+0000 7f3c5102cb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5627975b4da0 0x5627975b86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:40.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.218+0000 7f3c5102cb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5627975b8c30 0x5627975b90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:40.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.218+0000 7f3c5102cb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x562797506810 con 0x5627975b4480 2024-05-10T11:17:40.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.218+0000 7f3c5102cb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5627974a5770 con 0x5627975b8c30 2024-05-10T11:17:40.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.218+0000 7f3c5102cb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5627975027d0 con 0x5627975b4da0 2024-05-10T11:17:40.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.219+0000 7f3c3ffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5627975b4480 0x5627975b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:40.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.219+0000 7f3c3ffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5627975b4480 0x5627975b4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59902/0 (socket says 172.21.15.100:59902) 2024-05-10T11:17:40.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.219+0000 7f3c3ffff700 1 -- 172.21.15.100:0/1528194831 learned_addr learned my addr 172.21.15.100:0/1528194831 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:17:40.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.219+0000 7f3c44d36700 1 --2- 172.21.15.100:0/1528194831 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5627975b8c30 0x5627975b90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:40.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.219+0000 7f3c3f7fe700 1 --2- 172.21.15.100:0/1528194831 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5627975b4da0 0x5627975b86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:40.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.219+0000 7f3c3ffff700 1 -- 172.21.15.100:0/1528194831 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5627975b4da0 msgr2=0x5627975b86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:17:40.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.219+0000 7f3c3ffff700 1 --2- 172.21.15.100:0/1528194831 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5627975b4da0 0x5627975b86a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:40.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.219+0000 7f3c3ffff700 1 -- 172.21.15.100:0/1528194831 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5627975b8c30 msgr2=0x5627975b90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:17:40.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.219+0000 7f3c3ffff700 1 --2- 172.21.15.100:0/1528194831 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5627975b8c30 0x5627975b90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:40.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.219+0000 7f3c3ffff700 1 -- 172.21.15.100:0/1528194831 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562797453880 con 0x5627975b4480 2024-05-10T11:17:40.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.219+0000 7f3c3ffff700 1 --2- 172.21.15.100:0/1528194831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5627975b4480 0x5627975b4860 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3c30015000 tx=0x7f3c3001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1551304c352f7322 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:17:40.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.219+0000 7f3c367fc700 1 -- 172.21.15.100:0/1528194831 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c30004590 con 0x5627975b4480 2024-05-10T11:17:40.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.219+0000 7f3c367fc700 1 -- 172.21.15.100:0/1528194831 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3c30003c50 con 0x5627975b4480 2024-05-10T11:17:40.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.219+0000 7f3c367fc700 1 -- 172.21.15.100:0/1528194831 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c3000f7e0 con 0x5627975b4480 2024-05-10T11:17:40.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.220+0000 7f3c5102cb80 1 -- 172.21.15.100:0/1528194831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5627975b4480 msgr2=0x5627975b4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:40.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.220+0000 7f3c5102cb80 1 --2- 172.21.15.100:0/1528194831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5627975b4480 0x5627975b4860 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3c30015000 tx=0x7f3c3001f000 comp rx=0 tx=0).stop 2024-05-10T11:17:40.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.220+0000 7f3c5102cb80 1 -- 172.21.15.100:0/1528194831 shutdown_connections 2024-05-10T11:17:40.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.220+0000 7f3c5102cb80 1 --2- 172.21.15.100:0/1528194831 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5627975b8c30 0x5627975b90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:40.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.220+0000 7f3c5102cb80 1 --2- 172.21.15.100:0/1528194831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5627975b4480 0x5627975b4860 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:40.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.220+0000 7f3c5102cb80 1 --2- 172.21.15.100:0/1528194831 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5627975b4da0 0x5627975b86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:40.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.220+0000 7f3c5102cb80 1 -- 172.21.15.100:0/1528194831 >> 172.21.15.100:0/1528194831 conn(0x5627974e00b0 msgr2=0x5627975aec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:17:40.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.220+0000 7f3c5102cb80 1 -- 172.21.15.100:0/1528194831 shutdown_connections 2024-05-10T11:17:40.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.220+0000 7f3c5102cb80 1 -- 172.21.15.100:0/1528194831 wait complete. 2024-05-10T11:17:40.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.221+0000 7f3c5102cb80 1 -- start start 2024-05-10T11:17:40.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.221+0000 7f3c5102cb80 1 Processor -- start 2024-05-10T11:17:40.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.221+0000 7f3c5102cb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5627975b4480 0x5627975bffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:40.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.221+0000 7f3c5102cb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5627975b4da0 0x5627975bd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:40.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.221+0000 7f3c5102cb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5627975b8c30 0x5627975bd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:40.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.221+0000 7f3c5102cb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5627975c0d80 con 0x5627975b4da0 2024-05-10T11:17:40.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.221+0000 7f3c5102cb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56279746cbe0 con 0x5627975b8c30 2024-05-10T11:17:40.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.221+0000 7f3c5102cb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5627975c0f00 con 0x5627975b4480 2024-05-10T11:17:40.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.221+0000 7f3c5102cb80 10 client.0 _ll_register_callbacks cb 0x56279749e8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:17:40.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.221+0000 7f3c3f7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5627975b4da0 0x5627975bd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:40.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.221+0000 7f3c44d36700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5627975b8c30 0x5627975bd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:40.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.221+0000 7f3c3f7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5627975b4da0 0x5627975bd3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59914/0 (socket says 172.21.15.100:59914) 2024-05-10T11:17:40.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.221+0000 7f3c3f7fe700 1 -- 172.21.15.100:0/2455279713 learned_addr learned my addr 172.21.15.100:0/2455279713 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:17:40.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.222+0000 7f3c3ffff700 1 --2- 172.21.15.100:0/2455279713 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5627975b4480 0x5627975bffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:40.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.222+0000 7f3c3f7fe700 1 -- 172.21.15.100:0/2455279713 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5627975b4480 msgr2=0x5627975bffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:17:40.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.222+0000 7f3c3f7fe700 1 --2- 172.21.15.100:0/2455279713 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5627975b4480 0x5627975bffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:40.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.222+0000 7f3c3f7fe700 1 -- 172.21.15.100:0/2455279713 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5627975b8c30 msgr2=0x5627975bd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:17:40.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.222+0000 7f3c3f7fe700 1 --2- 172.21.15.100:0/2455279713 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5627975b8c30 0x5627975bd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:40.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.222+0000 7f3c3f7fe700 1 -- 172.21.15.100:0/2455279713 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627974a2640 con 0x5627975b4da0 2024-05-10T11:17:40.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.222+0000 7f3c44d36700 1 --2- 172.21.15.100:0/2455279713 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5627975b8c30 0x5627975bd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:17:40.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.222+0000 7f3c3f7fe700 1 --2- 172.21.15.100:0/2455279713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5627975b4da0 0x5627975bd3d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f3c2800f660 tx=0x7f3c2800f970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecdd88e7d51c102a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:17:40.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.222+0000 7f3c1f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:17:40.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.222+0000 7f3c1f7fe700 1 -- 172.21.15.100:0/2455279713 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c28004a30 con 0x5627975b4da0 2024-05-10T11:17:40.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.222+0000 7f3c1f7fe700 1 -- 172.21.15.100:0/2455279713 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f3c28004bd0 con 0x5627975b4da0 2024-05-10T11:17:40.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.222+0000 7f3c5102cb80 1 -- 172.21.15.100:0/2455279713 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c30003c50 con 0x5627975b4da0 2024-05-10T11:17:40.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.222+0000 7f3c1f7fe700 1 -- 172.21.15.100:0/2455279713 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3c28004dd0 con 0x5627975b4da0 2024-05-10T11:17:40.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.222+0000 7f3c5102cb80 10 client.15075 fetch_fsmap learned FSMap version 26 2024-05-10T11:17:40.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.222+0000 7f3c5102cb80 1 -- 172.21.15.100:0/2455279713 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5627975cb650 con 0x5627975b4da0 2024-05-10T11:17:40.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.223+0000 7f3c1f7fe700 1 -- 172.21.15.100:0/2455279713 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3c30003c50 con 0x5627975b4da0 2024-05-10T11:17:40.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.223+0000 7f3c1f7fe700 1 -- 172.21.15.100:0/2455279713 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f3c280119f0 con 0x5627975b4da0 2024-05-10T11:17:40.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.223+0000 7f3c5102cb80 10 client.15075 fetch_fsmap finished waiting for FSMap version 26 2024-05-10T11:17:40.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.223+0000 7f3c5102cb80 10 client.15075 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-10T11:17:40.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.223+0000 7f3c5102cb80 20 client.15075 populate_metadata read hostname 'smithi100' 2024-05-10T11:17:40.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.223+0000 7f3c5102cb80 1 --2- 172.21.15.100:0/2455279713 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5627975cdcc0 0x5627975cf130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:40.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.223+0000 7f3c5102cb80 4 client.15075 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:17:40.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.223+0000 7f3c5102cb80 1 -- 172.21.15.100:0/2455279713 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562797580600 con 0x5627975cdcc0 2024-05-10T11:17:40.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.223+0000 7f3c3ffff700 1 --2- 172.21.15.100:0/2455279713 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5627975cdcc0 0x5627975cf130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:40.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.224+0000 7f3c1f7fe700 10 client.15075 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:17:40.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.224+0000 7f3c3ffff700 1 --2- 172.21.15.100:0/2455279713 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5627975cdcc0 0x5627975cf130 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d0834f197799093 server_cookie=3faac7fc4bb4c918 in_seq=0 out_seq=0 2024-05-10T11:17:40.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.225+0000 7f3c1f7fe700 1 -- 172.21.15.100:0/2455279713 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562797580600 con 0x5627975cdcc0 2024-05-10T11:17:40.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.225+0000 7f3c1f7fe700 10 client.15075 handle_command_reply: tid=0 2024-05-10T11:17:40.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.225+0000 7f3c5102cb80 1 client.15075 shutdown 2024-05-10T11:17:40.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.225+0000 7f3c5102cb80 1 -- 172.21.15.100:0/2455279713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5627975b4da0 msgr2=0x5627975bd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:40.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.225+0000 7f3c5102cb80 1 --2- 172.21.15.100:0/2455279713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5627975b4da0 0x5627975bd3d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f3c2800f660 tx=0x7f3c2800f970 comp rx=0 tx=0).stop 2024-05-10T11:17:40.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.225+0000 7f3c5102cb80 1 -- 172.21.15.100:0/2455279713 shutdown_connections 2024-05-10T11:17:40.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.225+0000 7f3c5102cb80 1 --2- 172.21.15.100:0/2455279713 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5627975cdcc0 0x5627975cf130 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:40.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.225+0000 7f3c5102cb80 1 --2- 172.21.15.100:0/2455279713 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5627975b8c30 0x5627975bd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:40.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.225+0000 7f3c5102cb80 1 --2- 172.21.15.100:0/2455279713 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5627975b4480 0x5627975bffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:40.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.225+0000 7f3c1f7fe700 0 client.15075 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:17:40.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.225+0000 7f3c5102cb80 1 --2- 172.21.15.100:0/2455279713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5627975b4da0 0x5627975bd3d0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:40.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.225+0000 7f3c5102cb80 1 -- 172.21.15.100:0/2455279713 >> 172.21.15.100:0/2455279713 conn(0x5627974e00b0 msgr2=0x5627975a96d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:17:40.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.225+0000 7f3c5102cb80 1 -- 172.21.15.100:0/2455279713 shutdown_connections 2024-05-10T11:17:40.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.225+0000 7f3c5102cb80 1 -- 172.21.15.100:0/2455279713 wait complete. 2024-05-10T11:17:40.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.225+0000 7f3c5102cb80 20 client.15075 trim_cache size 0 max 16384 2024-05-10T11:17:40.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.245+0000 7f3c5102cb80 1 Processor -- start 2024-05-10T11:17:40.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.246+0000 7f3c5102cb80 1 -- start start 2024-05-10T11:17:40.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.246+0000 7f3c5102cb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5627975b4480 0x5627975b65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:40.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.246+0000 7f3c5102cb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5627975b4da0 0x5627975b6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:40.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.246+0000 7f3c5102cb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5627975b8c30 0x5627975a8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:40.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.246+0000 7f3c5102cb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5627975cd470 con 0x5627975b4480 2024-05-10T11:17:40.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.246+0000 7f3c5102cb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5627975cdb40 con 0x5627975b8c30 2024-05-10T11:17:40.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.246+0000 7f3c5102cb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5627975ce210 con 0x5627975b4da0 2024-05-10T11:17:40.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.246+0000 7f3c3ffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5627975b4480 0x5627975b65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:40.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.246+0000 7f3c3ffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5627975b4480 0x5627975b65a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59920/0 (socket says 172.21.15.100:59920) 2024-05-10T11:17:40.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.246+0000 7f3c3ffff700 1 -- 172.21.15.100:0/2810272633 learned_addr learned my addr 172.21.15.100:0/2810272633 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:17:40.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.246+0000 7f3c3f7fe700 1 --2- 172.21.15.100:0/2810272633 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5627975b4da0 0x5627975b6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:40.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.246+0000 7f3c44d36700 1 --2- 172.21.15.100:0/2810272633 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5627975b8c30 0x5627975a8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:40.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.246+0000 7f3c3ffff700 1 -- 172.21.15.100:0/2810272633 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5627975b4da0 msgr2=0x5627975b6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:17:40.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.246+0000 7f3c3ffff700 1 --2- 172.21.15.100:0/2810272633 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5627975b4da0 0x5627975b6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:40.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.246+0000 7f3c3ffff700 1 -- 172.21.15.100:0/2810272633 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5627975b8c30 msgr2=0x5627975a8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:17:40.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.246+0000 7f3c3ffff700 1 --2- 172.21.15.100:0/2810272633 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5627975b8c30 0x5627975a8e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:40.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.246+0000 7f3c3ffff700 1 -- 172.21.15.100:0/2810272633 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627975b04d0 con 0x5627975b4480 2024-05-10T11:17:40.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.246+0000 7f3c3ffff700 1 --2- 172.21.15.100:0/2810272633 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5627975b4480 0x5627975b65a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3c3000bfc0 tx=0x7f3c3000bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25a5a32aa46093b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:17:40.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.247+0000 7f3c35ffb700 1 -- 172.21.15.100:0/2810272633 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c30004590 con 0x5627975b4480 2024-05-10T11:17:40.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.247+0000 7f3c35ffb700 1 -- 172.21.15.100:0/2810272633 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3c30003c50 con 0x5627975b4480 2024-05-10T11:17:40.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.247+0000 7f3c35ffb700 1 -- 172.21.15.100:0/2810272633 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c3003c630 con 0x5627975b4480 2024-05-10T11:17:40.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.247+0000 7f3c5102cb80 1 -- 172.21.15.100:0/2810272633 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5627975b4480 msgr2=0x5627975b65a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:40.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.247+0000 7f3c5102cb80 1 --2- 172.21.15.100:0/2810272633 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5627975b4480 0x5627975b65a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3c3000bfc0 tx=0x7f3c3000bf50 comp rx=0 tx=0).stop 2024-05-10T11:17:40.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.247+0000 7f3c5102cb80 1 -- 172.21.15.100:0/2810272633 shutdown_connections 2024-05-10T11:17:40.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.247+0000 7f3c5102cb80 1 --2- 172.21.15.100:0/2810272633 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5627975b8c30 0x5627975a8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:40.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.247+0000 7f3c5102cb80 1 --2- 172.21.15.100:0/2810272633 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5627975b4480 0x5627975b65a0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:40.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.247+0000 7f3c5102cb80 1 --2- 172.21.15.100:0/2810272633 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5627975b4da0 0x5627975b6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:40.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.247+0000 7f3c5102cb80 1 -- 172.21.15.100:0/2810272633 >> 172.21.15.100:0/2810272633 conn(0x5627974e00b0 msgr2=0x5627975bbcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:17:40.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.247+0000 7f3c5102cb80 1 -- 172.21.15.100:0/2810272633 shutdown_connections 2024-05-10T11:17:40.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.247+0000 7f3c5102cb80 1 -- 172.21.15.100:0/2810272633 wait complete. 2024-05-10T11:17:40.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.248+0000 7f3c5102cb80 1 -- start start 2024-05-10T11:17:40.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.248+0000 7f3c5102cb80 1 Processor -- start 2024-05-10T11:17:40.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.248+0000 7f3c5102cb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5627975b4480 0x5627975d8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:40.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.248+0000 7f3c5102cb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5627975b4da0 0x5627975d89a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:40.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.248+0000 7f3c5102cb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5627975b8c30 0x5627975d1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:40.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.248+0000 7f3c5102cb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5627975cefb0 con 0x5627975b4da0 2024-05-10T11:17:40.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.248+0000 7f3c5102cb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5627975ce8e0 con 0x5627975b4480 2024-05-10T11:17:40.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.249+0000 7f3c5102cb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5627975cf680 con 0x5627975b8c30 2024-05-10T11:17:40.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.249+0000 7f3c5102cb80 10 client.0 _ll_register_callbacks cb 0x56279749e8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:17:40.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.249+0000 7f3c3f7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5627975b4da0 0x5627975d89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:40.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.249+0000 7f3c3f7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5627975b4da0 0x5627975d89a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59934/0 (socket says 172.21.15.100:59934) 2024-05-10T11:17:40.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.249+0000 7f3c3f7fe700 1 -- 172.21.15.100:0/1627354835 learned_addr learned my addr 172.21.15.100:0/1627354835 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:17:40.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.249+0000 7f3c3ffff700 1 --2- 172.21.15.100:0/1627354835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5627975b4480 0x5627975d8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:40.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.249+0000 7f3c3f7fe700 1 -- 172.21.15.100:0/1627354835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5627975b8c30 msgr2=0x5627975d1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:17:40.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.249+0000 7f3c3f7fe700 1 --2- 172.21.15.100:0/1627354835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5627975b8c30 0x5627975d1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:40.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.249+0000 7f3c3f7fe700 1 -- 172.21.15.100:0/1627354835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5627975b4480 msgr2=0x5627975d8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:17:40.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.249+0000 7f3c3f7fe700 1 --2- 172.21.15.100:0/1627354835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5627975b4480 0x5627975d8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:40.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.249+0000 7f3c3f7fe700 1 -- 172.21.15.100:0/1627354835 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627975c04b0 con 0x5627975b4da0 2024-05-10T11:17:40.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.249+0000 7f3c3f7fe700 1 --2- 172.21.15.100:0/1627354835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5627975b4da0 0x5627975d89a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3c28005dd0 tx=0x7f3c28011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c195efdd2f18c3f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:17:40.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.249+0000 7f3c1f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:17:40.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.249+0000 7f3c1f7fe700 1 -- 172.21.15.100:0/1627354835 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c28011ba0 con 0x5627975b4da0 2024-05-10T11:17:40.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.249+0000 7f3c1f7fe700 1 -- 172.21.15.100:0/1627354835 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f3c28004220 con 0x5627975b4da0 2024-05-10T11:17:40.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.249+0000 7f3c5102cb80 1 -- 172.21.15.100:0/1627354835 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c30003c50 con 0x5627975b4da0 2024-05-10T11:17:40.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.249+0000 7f3c1f7fe700 1 -- 172.21.15.100:0/1627354835 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3c2800fcc0 con 0x5627975b4da0 2024-05-10T11:17:40.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.250+0000 7f3c5102cb80 10 client.15081 fetch_fsmap learned FSMap version 26 2024-05-10T11:17:40.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.250+0000 7f3c5102cb80 1 -- 172.21.15.100:0/1627354835 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5627975d67a0 con 0x5627975b4da0 2024-05-10T11:17:40.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.250+0000 7f3c1f7fe700 1 -- 172.21.15.100:0/1627354835 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3c30003c50 con 0x5627975b4da0 2024-05-10T11:17:40.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.250+0000 7f3c1f7fe700 1 -- 172.21.15.100:0/1627354835 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f3c28030070 con 0x5627975b4da0 2024-05-10T11:17:40.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.250+0000 7f3c5102cb80 10 client.15081 fetch_fsmap finished waiting for FSMap version 26 2024-05-10T11:17:40.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.250+0000 7f3c5102cb80 10 client.15081 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-10T11:17:40.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.250+0000 7f3c5102cb80 20 client.15081 populate_metadata read hostname 'smithi100' 2024-05-10T11:17:40.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.250+0000 7f3c5102cb80 1 --2- 172.21.15.100:0/1627354835 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56279763d600 0x56279763c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:40.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.250+0000 7f3c5102cb80 4 client.15081 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T11:17:40.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.250+0000 7f3c5102cb80 1 -- 172.21.15.100:0/1627354835 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "status"}) -- 0x5627975141f0 con 0x56279763d600 2024-05-10T11:17:40.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.250+0000 7f3c3ffff700 1 --2- 172.21.15.100:0/1627354835 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56279763d600 0x56279763c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:40.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.251+0000 7f3c1f7fe700 10 client.15081 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:17:40.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.251+0000 7f3c3ffff700 1 --2- 172.21.15.100:0/1627354835 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56279763d600 0x56279763c310 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=939dc4c5674e548d server_cookie=ec73c9fc19d81b85 in_seq=0 out_seq=0 2024-05-10T11:17:40.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.251+0000 7f3c1f7fe700 1 -- 172.21.15.100:0/1627354835 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5627975141f0 con 0x56279763d600 2024-05-10T11:17:40.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.251+0000 7f3c1f7fe700 10 client.15081 handle_command_reply: tid=0 2024-05-10T11:17:40.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.251+0000 7f3c5102cb80 1 client.15081 shutdown 2024-05-10T11:17:40.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.251+0000 7f3c5102cb80 1 -- 172.21.15.100:0/1627354835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5627975b4da0 msgr2=0x5627975d89a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:40.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.251+0000 7f3c5102cb80 1 --2- 172.21.15.100:0/1627354835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5627975b4da0 0x5627975d89a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3c28005dd0 tx=0x7f3c28011900 comp rx=0 tx=0).stop 2024-05-10T11:17:40.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.251+0000 7f3c5102cb80 1 -- 172.21.15.100:0/1627354835 shutdown_connections 2024-05-10T11:17:40.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.251+0000 7f3c5102cb80 1 --2- 172.21.15.100:0/1627354835 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56279763d600 0x56279763c310 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:40.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.251+0000 7f3c5102cb80 1 --2- 172.21.15.100:0/1627354835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5627975b8c30 0x5627975d1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:40.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.251+0000 7f3c1f7fe700 0 client.15081 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:17:40.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.251+0000 7f3c5102cb80 1 --2- 172.21.15.100:0/1627354835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5627975b4480 0x5627975d8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:40.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.251+0000 7f3c5102cb80 1 --2- 172.21.15.100:0/1627354835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5627975b4da0 0x5627975d89a0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:40.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.251+0000 7f3c5102cb80 1 -- 172.21.15.100:0/1627354835 >> 172.21.15.100:0/1627354835 conn(0x5627974e00b0 msgr2=0x5627975bb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:17:40.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.251+0000 7f3c5102cb80 1 -- 172.21.15.100:0/1627354835 shutdown_connections 2024-05-10T11:17:40.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.251+0000 7f3c5102cb80 1 -- 172.21.15.100:0/1627354835 wait complete. 2024-05-10T11:17:40.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.251+0000 7f3c5102cb80 20 client.15081 trim_cache size 0 max 16384 2024-05-10T11:17:40.257 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:17:40.257 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_fsid": "20a542cc-0ebd-11ef-bc98-c7b262605968", 2024-05-10T11:17:40.257 INFO:teuthology.orchestra.run.smithi100.stdout: "whoami": 0, 2024-05-10T11:17:40.257 INFO:teuthology.orchestra.run.smithi100.stdout: "id": 24439, 2024-05-10T11:17:40.258 INFO:teuthology.orchestra.run.smithi100.stdout: "want_state": "up:active", 2024-05-10T11:17:40.258 INFO:teuthology.orchestra.run.smithi100.stdout: "state": "up:active", 2024-05-10T11:17:40.258 INFO:teuthology.orchestra.run.smithi100.stdout: "fs_name": "cephfs", 2024-05-10T11:17:40.258 INFO:teuthology.orchestra.run.smithi100.stdout: "rank_uptime": 192.97838350800001, 2024-05-10T11:17:40.258 INFO:teuthology.orchestra.run.smithi100.stdout: "mdsmap_epoch": 26, 2024-05-10T11:17:40.258 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch": 89, 2024-05-10T11:17:40.258 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch_barrier": 89, 2024-05-10T11:17:40.258 INFO:teuthology.orchestra.run.smithi100.stdout: "uptime": 220.73631261899999 2024-05-10T11:17:40.258 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:17:40.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.252+0000 7f3c46799700 1 -- 172.21.15.100:0/3980215198 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f3c2c0b2900 msgr2=0x7f3c2c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:40.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.252+0000 7f3c46799700 1 --2- 172.21.15.100:0/3980215198 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f3c2c0b2900 0x7f3c2c0b4dc0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3c3000b5c0 tx=0x7f3c3001a040 comp rx=0 tx=0).stop 2024-05-10T11:17:40.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.252+0000 7f3c46799700 1 -- 172.21.15.100:0/3980215198 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c40108400 msgr2=0x7f3c40199970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:40.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.252+0000 7f3c46799700 1 --2- 172.21.15.100:0/3980215198 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c40108400 0x7f3c40199970 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3c3800ada0 tx=0x7f3c3800c330 comp rx=0 tx=0).stop 2024-05-10T11:17:40.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.252+0000 7f3c46799700 1 -- 172.21.15.100:0/3980215198 shutdown_connections 2024-05-10T11:17:40.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.252+0000 7f3c46799700 1 --2- 172.21.15.100:0/3980215198 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f3c2c0b2900 0x7f3c2c0b4dc0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:40.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.252+0000 7f3c46799700 1 --2- 172.21.15.100:0/3980215198 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c40108400 0x7f3c40199970 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:40.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.252+0000 7f3c46799700 1 --2- 172.21.15.100:0/3980215198 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c40102390 0x7f3c4019f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:40.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.252+0000 7f3c46799700 1 --2- 172.21.15.100:0/3980215198 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c40102d50 0x7f3c4019fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:40.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.252+0000 7f3c46799700 1 -- 172.21.15.100:0/3980215198 >> 172.21.15.100:0/3980215198 conn(0x7f3c40079310 msgr2=0x7f3c4006bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:17:40.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.252+0000 7f3c46799700 1 -- 172.21.15.100:0/3980215198 shutdown_connections 2024-05-10T11:17:40.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:40.252+0000 7f3c46799700 1 -- 172.21.15.100:0/3980215198 wait complete. 2024-05-10T11:17:40.977 INFO:tasks.cephadm:Setup complete, yielding 2024-05-10T11:17:40.978 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-10T11:17:40.989 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi100.front.sepia.ceph.com 2024-05-10T11:17:40.989 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- bash -c 'ceph orch status' 2024-05-10T11:17:41.235 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:17:41.596 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:17:41 smithi100 ceph-mon[30296]: pgmap v399: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T11:17:41.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:17:41 smithi178 ceph-mon[33359]: pgmap v399: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T11:17:41.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:17:41 smithi188 ceph-mon[34075]: pgmap v399: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T11:17:43.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:43.017+0000 7f1bd1ce1700 1 -- 172.21.15.100:0/1436738311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bcc102d90 msgr2=0x7f1bcc1031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:43.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:43.017+0000 7f1bd1ce1700 1 --2- 172.21.15.100:0/1436738311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bcc102d90 0x7f1bcc1031f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1bb40099c0 tx=0x7f1bb4009cd0 comp rx=0 tx=0).stop 2024-05-10T11:17:43.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:43.017+0000 7f1bd1ce1700 1 -- 172.21.15.100:0/1436738311 shutdown_connections 2024-05-10T11:17:43.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:43.017+0000 7f1bd1ce1700 1 --2- 172.21.15.100:0/1436738311 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1bcc103730 0x7f1bcc109fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:43.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:43.017+0000 7f1bd1ce1700 1 --2- 172.21.15.100:0/1436738311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1bcc101b90 0x7f1bcc101fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:43.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:43.017+0000 7f1bd1ce1700 1 --2- 172.21.15.100:0/1436738311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bcc102d90 0x7f1bcc1031f0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:43.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:43.017+0000 7f1bd1ce1700 1 -- 172.21.15.100:0/1436738311 >> 172.21.15.100:0/1436738311 conn(0x7f1bcc0fd110 msgr2=0x7f1bcc0ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:17:43.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:43.017+0000 7f1bd1ce1700 1 -- 172.21.15.100:0/1436738311 shutdown_connections 2024-05-10T11:17:43.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:43.017+0000 7f1bd1ce1700 1 -- 172.21.15.100:0/1436738311 wait complete. 2024-05-10T11:17:43.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:43.018+0000 7f1bd1ce1700 1 Processor -- start 2024-05-10T11:17:43.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:43.018+0000 7f1bd1ce1700 1 -- start start 2024-05-10T11:17:43.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:43.018+0000 7f1bd1ce1700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1bcc101b90 0x7f1bcc198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:43.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:43.018+0000 7f1bd1ce1700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bcc102d90 0x7f1bcc198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:43.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:43.018+0000 7f1bd1ce1700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1bcc103730 0x7f1bcc19f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:43.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:43.018+0000 7f1bd1ce1700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1bcc10ddd0 con 0x7f1bcc102d90 2024-05-10T11:17:43.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:43.018+0000 7f1bd1ce1700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1bcc10dc50 con 0x7f1bcc101b90 2024-05-10T11:17:43.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:43.018+0000 7f1bd1ce1700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1bcc10df50 con 0x7f1bcc103730 2024-05-10T11:17:43.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:43.018+0000 7f1bc3fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bcc102d90 0x7f1bcc198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:43.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:43.018+0000 7f1bc3fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bcc102d90 0x7f1bcc198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59960/0 (socket says 172.21.15.100:59960) 2024-05-10T11:17:43.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:43.018+0000 7f1bc3fff700 1 -- 172.21.15.100:0/3945034041 learned_addr learned my addr 172.21.15.100:0/3945034041 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:17:43.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:43.018+0000 7f1bcb7fe700 1 --2- 172.21.15.100:0/3945034041 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1bcc101b90 0x7f1bcc198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:43.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:43.018+0000 7f1bcbfff700 1 --2- 172.21.15.100:0/3945034041 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1bcc103730 0x7f1bcc19f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:43.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:43.018+0000 7f1bc3fff700 1 -- 172.21.15.100:0/3945034041 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1bcc103730 msgr2=0x7f1bcc19f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:43.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:43.018+0000 7f1bc3fff700 1 --2- 172.21.15.100:0/3945034041 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1bcc103730 0x7f1bcc19f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:43.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:43.018+0000 7f1bc3fff700 1 -- 172.21.15.100:0/3945034041 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1bcc101b90 msgr2=0x7f1bcc198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:43.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:43.018+0000 7f1bc3fff700 1 --2- 172.21.15.100:0/3945034041 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1bcc101b90 0x7f1bcc198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:43.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:43.018+0000 7f1bc3fff700 1 -- 172.21.15.100:0/3945034041 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bcc1a0070 con 0x7f1bcc102d90 2024-05-10T11:17:43.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:43.018+0000 7f1bcbfff700 1 --2- 172.21.15.100:0/3945034041 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1bcc103730 0x7f1bcc19f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:17:43.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:43.018+0000 7f1bc3fff700 1 --2- 172.21.15.100:0/3945034041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bcc102d90 0x7f1bcc198950 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f1bb401a840 tx=0x7f1bb401a870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:17:43.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:43.019+0000 7f1bc97fa700 1 -- 172.21.15.100:0/3945034041 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb402a070 con 0x7f1bcc102d90 2024-05-10T11:17:43.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:43.019+0000 7f1bc97fa700 1 -- 172.21.15.100:0/3945034041 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1bb400f460 con 0x7f1bcc102d90 2024-05-10T11:17:43.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:43.019+0000 7f1bc97fa700 1 -- 172.21.15.100:0/3945034041 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb4025680 con 0x7f1bcc102d90 2024-05-10T11:17:43.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:43.019+0000 7f1bd1ce1700 1 -- 172.21.15.100:0/3945034041 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bcc10a9f0 con 0x7f1bcc102d90 2024-05-10T11:17:43.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:43.019+0000 7f1bd1ce1700 1 -- 172.21.15.100:0/3945034041 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bcc10aed0 con 0x7f1bcc102d90 2024-05-10T11:17:43.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:43.020+0000 7f1bd1ce1700 1 -- 172.21.15.100:0/3945034041 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bcc06a190 con 0x7f1bcc102d90 2024-05-10T11:17:43.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:43.021+0000 7f1bc97fa700 1 -- 172.21.15.100:0/3945034041 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f1bb402d070 con 0x7f1bcc102d90 2024-05-10T11:17:43.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:43.022+0000 7f1bc97fa700 1 --2- 172.21.15.100:0/3945034041 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f1bb80b2880 0x7f1bb80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:43.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:43.022+0000 7f1bcb7fe700 1 --2- 172.21.15.100:0/3945034041 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f1bb80b2880 0x7f1bb80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:43.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:43.022+0000 7f1bc97fa700 1 -- 172.21.15.100:0/3945034041 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f1bb40e5d60 con 0x7f1bcc102d90 2024-05-10T11:17:43.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:43.022+0000 7f1bcb7fe700 1 --2- 172.21.15.100:0/3945034041 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f1bb80b2880 0x7f1bb80b4d40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc102bf0 tx=0x7f1bbc009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:17:43.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:43.025+0000 7f1bc97fa700 1 -- 172.21.15.100:0/3945034041 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f1bb4034070 con 0x7f1bcc102d90 2024-05-10T11:17:43.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:43.245+0000 7f1bd1ce1700 1 -- 172.21.15.100:0/3945034041 --> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f1bcc10b1f0 con 0x7f1bb80b2880 2024-05-10T11:17:43.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:43.247+0000 7f1bc97fa700 1 -- 172.21.15.100:0/3945034041 <== mgr.14152 v2:172.21.15.100:6800/2507287356 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f1bcc10b1f0 con 0x7f1bb80b2880 2024-05-10T11:17:43.247 INFO:teuthology.orchestra.run.smithi100.stdout:Backend: cephadm 2024-05-10T11:17:43.247 INFO:teuthology.orchestra.run.smithi100.stdout:Available: Yes 2024-05-10T11:17:43.247 INFO:teuthology.orchestra.run.smithi100.stdout:Paused: No 2024-05-10T11:17:43.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:43.249+0000 7f1bd1ce1700 1 -- 172.21.15.100:0/3945034041 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f1bb80b2880 msgr2=0x7f1bb80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:43.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:43.249+0000 7f1bd1ce1700 1 --2- 172.21.15.100:0/3945034041 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f1bb80b2880 0x7f1bb80b4d40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc102bf0 tx=0x7f1bbc009190 comp rx=0 tx=0).stop 2024-05-10T11:17:43.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:43.249+0000 7f1bd1ce1700 1 -- 172.21.15.100:0/3945034041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bcc102d90 msgr2=0x7f1bcc198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:43.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:43.249+0000 7f1bd1ce1700 1 --2- 172.21.15.100:0/3945034041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bcc102d90 0x7f1bcc198950 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f1bb401a840 tx=0x7f1bb401a870 comp rx=0 tx=0).stop 2024-05-10T11:17:43.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:43.249+0000 7f1bd1ce1700 1 -- 172.21.15.100:0/3945034041 shutdown_connections 2024-05-10T11:17:43.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:43.249+0000 7f1bd1ce1700 1 --2- 172.21.15.100:0/3945034041 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f1bb80b2880 0x7f1bb80b4d40 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:43.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:43.249+0000 7f1bd1ce1700 1 --2- 172.21.15.100:0/3945034041 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1bcc103730 0x7f1bcc19f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:43.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:43.249+0000 7f1bd1ce1700 1 --2- 172.21.15.100:0/3945034041 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1bcc101b90 0x7f1bcc198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:43.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:43.249+0000 7f1bd1ce1700 1 --2- 172.21.15.100:0/3945034041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bcc102d90 0x7f1bcc198950 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:43.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:43.249+0000 7f1bd1ce1700 1 -- 172.21.15.100:0/3945034041 >> 172.21.15.100:0/3945034041 conn(0x7f1bcc0fd110 msgr2=0x7f1bcc106090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:17:43.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:43.249+0000 7f1bd1ce1700 1 -- 172.21.15.100:0/3945034041 shutdown_connections 2024-05-10T11:17:43.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:43.249+0000 7f1bd1ce1700 1 -- 172.21.15.100:0/3945034041 wait complete. 2024-05-10T11:17:43.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:17:43 smithi100 ceph-mon[30296]: pgmap v400: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T11:17:43.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:17:43 smithi178 ceph-mon[33359]: pgmap v400: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T11:17:43.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:17:43 smithi188 ceph-mon[34075]: pgmap v400: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T11:17:43.951 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- bash -c 'ceph orch ps' 2024-05-10T11:17:44.195 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:17:44.573 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:17:44 smithi100 ceph-mon[30296]: from='client.15087 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:17:44.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:17:44 smithi178 ceph-mon[33359]: from='client.15087 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:17:44.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:17:44 smithi188 ceph-mon[34075]: from='client.15087 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:17:45.647 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:17:45 smithi100 ceph-mon[30296]: pgmap v401: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-10T11:17:45.679 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:17:45 smithi188 ceph-mon[34075]: pgmap v401: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-10T11:17:45.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:17:45 smithi178 ceph-mon[33359]: pgmap v401: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-10T11:17:45.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:45.920+0000 7f6521767700 1 -- 172.21.15.100:0/2542231485 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f651c1068c0 msgr2=0x7f651c108cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:45.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:45.920+0000 7f6521767700 1 --2- 172.21.15.100:0/2542231485 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f651c1068c0 0x7f651c108cb0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f651000b0a0 tx=0x7f651000b3b0 comp rx=0 tx=0).stop 2024-05-10T11:17:45.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:45.920+0000 7f6521767700 1 -- 172.21.15.100:0/2542231485 shutdown_connections 2024-05-10T11:17:45.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:45.921+0000 7f6521767700 1 --2- 172.21.15.100:0/2542231485 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f651c1091f0 0x7f651c10b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:45.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:45.921+0000 7f6521767700 1 --2- 172.21.15.100:0/2542231485 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f651c103f90 0x7f651c106380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:45.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:45.921+0000 7f6521767700 1 --2- 172.21.15.100:0/2542231485 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f651c1068c0 0x7f651c108cb0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:45.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:45.921+0000 7f6521767700 1 -- 172.21.15.100:0/2542231485 >> 172.21.15.100:0/2542231485 conn(0x7f651c0fd9a0 msgr2=0x7f651c0ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:17:45.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:45.921+0000 7f6521767700 1 -- 172.21.15.100:0/2542231485 shutdown_connections 2024-05-10T11:17:45.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:45.921+0000 7f6521767700 1 -- 172.21.15.100:0/2542231485 wait complete. 2024-05-10T11:17:45.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:45.921+0000 7f6521767700 1 Processor -- start 2024-05-10T11:17:45.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:45.922+0000 7f6521767700 1 -- start start 2024-05-10T11:17:45.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:45.922+0000 7f6521767700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f651c103f90 0x7f651c199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:45.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:45.922+0000 7f6521767700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f651c1068c0 0x7f651c199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:45.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:45.922+0000 7f6521767700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f651c1091f0 0x7f651c1a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:45.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:45.922+0000 7f6521767700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f651c10ee70 con 0x7f651c1068c0 2024-05-10T11:17:45.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:45.922+0000 7f6521767700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f651c10ecf0 con 0x7f651c103f90 2024-05-10T11:17:45.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:45.922+0000 7f6521767700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f651c10eff0 con 0x7f651c1091f0 2024-05-10T11:17:45.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:45.922+0000 7f651a7fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f651c1068c0 0x7f651c199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:45.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:45.922+0000 7f651a7fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f651c1068c0 0x7f651c199aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59980/0 (socket says 172.21.15.100:59980) 2024-05-10T11:17:45.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:45.922+0000 7f651a7fc700 1 -- 172.21.15.100:0/13151699 learned_addr learned my addr 172.21.15.100:0/13151699 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:17:45.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:45.922+0000 7f651affd700 1 --2- 172.21.15.100:0/13151699 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f651c103f90 0x7f651c199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:45.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:45.922+0000 7f651b7fe700 1 --2- 172.21.15.100:0/13151699 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f651c1091f0 0x7f651c1a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:45.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:45.922+0000 7f651a7fc700 1 -- 172.21.15.100:0/13151699 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f651c1091f0 msgr2=0x7f651c1a0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:45.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:45.922+0000 7f651a7fc700 1 --2- 172.21.15.100:0/13151699 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f651c1091f0 0x7f651c1a0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:45.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:45.922+0000 7f651a7fc700 1 -- 172.21.15.100:0/13151699 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f651c103f90 msgr2=0x7f651c199560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:45.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:45.922+0000 7f651a7fc700 1 --2- 172.21.15.100:0/13151699 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f651c103f90 0x7f651c199560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:45.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:45.922+0000 7f651a7fc700 1 -- 172.21.15.100:0/13151699 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f651c1a11c0 con 0x7f651c1068c0 2024-05-10T11:17:45.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:45.922+0000 7f651b7fe700 1 --2- 172.21.15.100:0/13151699 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f651c1091f0 0x7f651c1a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:17:45.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:45.922+0000 7f651a7fc700 1 --2- 172.21.15.100:0/13151699 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f651c1068c0 0x7f651c199aa0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6510000f80 tx=0x7f6510009c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:17:45.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:45.923+0000 7f6503fff700 1 -- 172.21.15.100:0/13151699 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6510021070 con 0x7f651c1068c0 2024-05-10T11:17:45.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:45.923+0000 7f6503fff700 1 -- 172.21.15.100:0/13151699 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6510007c80 con 0x7f651c1068c0 2024-05-10T11:17:45.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:45.923+0000 7f6521767700 1 -- 172.21.15.100:0/13151699 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f651c1a1450 con 0x7f651c1068c0 2024-05-10T11:17:45.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:45.923+0000 7f6503fff700 1 -- 172.21.15.100:0/13151699 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f651001b7a0 con 0x7f651c1068c0 2024-05-10T11:17:45.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:45.923+0000 7f6521767700 1 -- 172.21.15.100:0/13151699 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f651c1a18a0 con 0x7f651c1068c0 2024-05-10T11:17:45.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:45.924+0000 7f6521767700 1 -- 172.21.15.100:0/13151699 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f651c06a190 con 0x7f651c1068c0 2024-05-10T11:17:45.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:45.925+0000 7f6503fff700 1 -- 172.21.15.100:0/13151699 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f6510024070 con 0x7f651c1068c0 2024-05-10T11:17:45.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:45.926+0000 7f6503fff700 1 --2- 172.21.15.100:0/13151699 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f65040b2880 0x7f65040b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:45.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:45.926+0000 7f651affd700 1 --2- 172.21.15.100:0/13151699 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f65040b2880 0x7f65040b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:45.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:45.926+0000 7f6503fff700 1 -- 172.21.15.100:0/13151699 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f651000f020 con 0x7f651c1068c0 2024-05-10T11:17:45.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:45.926+0000 7f651affd700 1 --2- 172.21.15.100:0/13151699 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f65040b2880 0x7f65040b4d40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6514005950 tx=0x7f651400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:17:45.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:45.929+0000 7f6503fff700 1 -- 172.21.15.100:0/13151699 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f651002b070 con 0x7f651c1068c0 2024-05-10T11:17:46.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:46.141+0000 7f6521767700 1 -- 172.21.15.100:0/13151699 --> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f651c0638e0 con 0x7f65040b2880 2024-05-10T11:17:46.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:46.155+0000 7f6503fff700 1 -- 172.21.15.100:0/13151699 <== mgr.14152 v2:172.21.15.100:6800/2507287356 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f651c0638e0 con 0x7f65040b2880 2024-05-10T11:17:46.158 INFO:teuthology.orchestra.run.smithi100.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-10T11:17:46.158 INFO:teuthology.orchestra.run.smithi100.stdout:mds.a smithi100 running (3m) 37s ago 3m 13.5M - 19.0.0-3449-gd39fa088 ee5e4f21ef58 e7edb955eb97 2024-05-10T11:17:46.158 INFO:teuthology.orchestra.run.smithi100.stdout:mds.b smithi178 running (3m) 80s ago 3m 20.8M - 19.0.0-3449-gd39fa088 ee5e4f21ef58 e8a32cd93f44 2024-05-10T11:17:46.159 INFO:teuthology.orchestra.run.smithi100.stdout:mds.c smithi188 running (3m) 80s ago 3m 18.0M - 19.0.0-3449-gd39fa088 ee5e4f21ef58 ae08c30f24b6 2024-05-10T11:17:46.159 INFO:teuthology.orchestra.run.smithi100.stdout:mds.d smithi100 running (4m) 37s ago 4m 14.4M - 19.0.0-3449-gd39fa088 ee5e4f21ef58 6bdce29d310a 2024-05-10T11:17:46.159 INFO:teuthology.orchestra.run.smithi100.stdout:mds.e smithi178 running (3m) 80s ago 3m 17.3M - 19.0.0-3449-gd39fa088 ee5e4f21ef58 b445c9f9eac5 2024-05-10T11:17:46.159 INFO:teuthology.orchestra.run.smithi100.stdout:mds.f smithi188 running (3m) 80s ago 3m 19.8M - 19.0.0-3449-gd39fa088 ee5e4f21ef58 172d7613ab04 2024-05-10T11:17:46.159 INFO:teuthology.orchestra.run.smithi100.stdout:mds.g smithi100 running (3m) 37s ago 3m 16.0M - 19.0.0-3449-gd39fa088 ee5e4f21ef58 b9c096f696f7 2024-05-10T11:17:46.160 INFO:teuthology.orchestra.run.smithi100.stdout:mds.h smithi178 running (3m) 80s ago 3m 17.3M - 19.0.0-3449-gd39fa088 ee5e4f21ef58 6939a764e16d 2024-05-10T11:17:46.160 INFO:teuthology.orchestra.run.smithi100.stdout:mds.i smithi188 running (3m) 80s ago 3m 27.5M - 19.0.0-3449-gd39fa088 ee5e4f21ef58 9d9fc5b815d3 2024-05-10T11:17:46.160 INFO:teuthology.orchestra.run.smithi100.stdout:mds.j smithi100 running (4m) 37s ago 4m 13.9M - 19.0.0-3449-gd39fa088 ee5e4f21ef58 cfaf0ab1a105 2024-05-10T11:17:46.160 INFO:teuthology.orchestra.run.smithi100.stdout:mds.k smithi178 running (4m) 80s ago 4m 17.1M - 19.0.0-3449-gd39fa088 ee5e4f21ef58 348fd0498391 2024-05-10T11:17:46.160 INFO:teuthology.orchestra.run.smithi100.stdout:mds.l smithi188 running (4m) 80s ago 4m 13.7M - 19.0.0-3449-gd39fa088 ee5e4f21ef58 f411399636ef 2024-05-10T11:17:46.161 INFO:teuthology.orchestra.run.smithi100.stdout:mgr.x smithi100 *:9283,8765 running (11m) 37s ago 11m 531M - 19.0.0-3449-gd39fa088 ee5e4f21ef58 81381d9bb2d1 2024-05-10T11:17:46.161 INFO:teuthology.orchestra.run.smithi100.stdout:mgr.y smithi178 *:8443,8765 running (9m) 80s ago 9m 467M - 19.0.0-3449-gd39fa088 ee5e4f21ef58 b2f3553d842c 2024-05-10T11:17:46.161 INFO:teuthology.orchestra.run.smithi100.stdout:mgr.z smithi188 *:8443,8765 running (9m) 80s ago 9m 466M - 19.0.0-3449-gd39fa088 ee5e4f21ef58 2e5740865245 2024-05-10T11:17:46.161 INFO:teuthology.orchestra.run.smithi100.stdout:mon.a smithi100 running (11m) 37s ago 11m 55.0M 2048M 19.0.0-3449-gd39fa088 ee5e4f21ef58 0469b48457b9 2024-05-10T11:17:46.161 INFO:teuthology.orchestra.run.smithi100.stdout:mon.b smithi178 running (9m) 80s ago 9m 41.4M 2048M 19.0.0-3449-gd39fa088 ee5e4f21ef58 2b35ae6b2dd9 2024-05-10T11:17:46.162 INFO:teuthology.orchestra.run.smithi100.stdout:mon.c smithi188 running (10m) 80s ago 10m 41.4M 2048M 19.0.0-3449-gd39fa088 ee5e4f21ef58 7e0cb31e6fa7 2024-05-10T11:17:46.162 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0 smithi100 running (9m) 37s ago 9m 53.7M 4272M 19.0.0-3449-gd39fa088 ee5e4f21ef58 368e76af2a30 2024-05-10T11:17:46.162 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1 smithi178 running (8m) 80s ago 8m 47.9M 4272M 19.0.0-3449-gd39fa088 ee5e4f21ef58 00a623888b32 2024-05-10T11:17:46.162 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2 smithi188 running (8m) 80s ago 8m 56.7M 4272M 19.0.0-3449-gd39fa088 ee5e4f21ef58 66f6e1fdd848 2024-05-10T11:17:46.163 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3 smithi100 running (8m) 37s ago 8m 55.1M 4272M 19.0.0-3449-gd39fa088 ee5e4f21ef58 2e5cf00d9067 2024-05-10T11:17:46.163 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4 smithi178 running (7m) 80s ago 7m 54.0M 4272M 19.0.0-3449-gd39fa088 ee5e4f21ef58 2273740b90eb 2024-05-10T11:17:46.163 INFO:teuthology.orchestra.run.smithi100.stdout:osd.5 smithi188 running (7m) 80s ago 7m 44.2M 4272M 19.0.0-3449-gd39fa088 ee5e4f21ef58 083c0cb86b40 2024-05-10T11:17:46.163 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6 smithi100 running (6m) 37s ago 6m 45.1M 4272M 19.0.0-3449-gd39fa088 ee5e4f21ef58 3d6c7a1eadac 2024-05-10T11:17:46.163 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7 smithi178 running (6m) 80s ago 6m 45.0M 4272M 19.0.0-3449-gd39fa088 ee5e4f21ef58 8450ce57819a 2024-05-10T11:17:46.163 INFO:teuthology.orchestra.run.smithi100.stdout:osd.8 smithi188 running (5m) 80s ago 5m 52.5M 4272M 19.0.0-3449-gd39fa088 ee5e4f21ef58 dca79fe29f17 2024-05-10T11:17:46.164 INFO:teuthology.orchestra.run.smithi100.stdout:osd.9 smithi100 running (5m) 37s ago 5m 50.0M 4272M 19.0.0-3449-gd39fa088 ee5e4f21ef58 f887ad41d083 2024-05-10T11:17:46.164 INFO:teuthology.orchestra.run.smithi100.stdout:osd.10 smithi178 running (5m) 80s ago 5m 51.3M 4272M 19.0.0-3449-gd39fa088 ee5e4f21ef58 e1ed9b9a15fb 2024-05-10T11:17:46.164 INFO:teuthology.orchestra.run.smithi100.stdout:osd.11 smithi188 running (4m) 80s ago 4m 50.8M 4272M 19.0.0-3449-gd39fa088 ee5e4f21ef58 3bba8eb71c40 2024-05-10T11:17:46.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:46.157+0000 7f6521767700 1 -- 172.21.15.100:0/13151699 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f65040b2880 msgr2=0x7f65040b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:46.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:46.157+0000 7f6521767700 1 --2- 172.21.15.100:0/13151699 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f65040b2880 0x7f65040b4d40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6514005950 tx=0x7f651400b410 comp rx=0 tx=0).stop 2024-05-10T11:17:46.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:46.157+0000 7f6521767700 1 -- 172.21.15.100:0/13151699 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f651c1068c0 msgr2=0x7f651c199aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:46.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:46.157+0000 7f6521767700 1 --2- 172.21.15.100:0/13151699 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f651c1068c0 0x7f651c199aa0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6510000f80 tx=0x7f6510009c70 comp rx=0 tx=0).stop 2024-05-10T11:17:46.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:46.157+0000 7f6521767700 1 -- 172.21.15.100:0/13151699 shutdown_connections 2024-05-10T11:17:46.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:46.157+0000 7f6521767700 1 --2- 172.21.15.100:0/13151699 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f65040b2880 0x7f65040b4d40 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:46.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:46.157+0000 7f6521767700 1 --2- 172.21.15.100:0/13151699 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f651c1091f0 0x7f651c1a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:46.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:46.157+0000 7f6521767700 1 --2- 172.21.15.100:0/13151699 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f651c103f90 0x7f651c199560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:46.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:46.157+0000 7f6521767700 1 --2- 172.21.15.100:0/13151699 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f651c1068c0 0x7f651c199aa0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:46.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:46.157+0000 7f6521767700 1 -- 172.21.15.100:0/13151699 >> 172.21.15.100:0/13151699 conn(0x7f651c0fd9a0 msgr2=0x7f651c1049f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:17:46.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:46.157+0000 7f6521767700 1 -- 172.21.15.100:0/13151699 shutdown_connections 2024-05-10T11:17:46.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:46.157+0000 7f6521767700 1 -- 172.21.15.100:0/13151699 wait complete. 2024-05-10T11:17:46.934 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- bash -c 'ceph orch ls' 2024-05-10T11:17:47.178 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:17:47.634 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:17:47 smithi100 ceph-mon[30296]: from='client.15093 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:17:47.634 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:17:47 smithi100 ceph-mon[30296]: pgmap v402: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T11:17:47.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:17:47 smithi178 ceph-mon[33359]: from='client.15093 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:17:47.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:17:47 smithi178 ceph-mon[33359]: pgmap v402: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T11:17:47.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:17:47 smithi188 ceph-mon[34075]: from='client.15093 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:17:47.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:17:47 smithi188 ceph-mon[34075]: pgmap v402: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T11:17:48.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:48.840+0000 7f33aced7700 1 -- 172.21.15.100:0/3784426907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33a81017f0 msgr2=0x7f33a8101c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:48.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:48.840+0000 7f33aced7700 1 --2- 172.21.15.100:0/3784426907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33a81017f0 0x7f33a8101c10 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f339c0099c0 tx=0x7f339c009cd0 comp rx=0 tx=0).stop 2024-05-10T11:17:48.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:48.841+0000 7f33aced7700 1 -- 172.21.15.100:0/3784426907 shutdown_connections 2024-05-10T11:17:48.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:48.841+0000 7f33aced7700 1 --2- 172.21.15.100:0/3784426907 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f33a8105210 0x7f33a810bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:48.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:48.841+0000 7f33aced7700 1 --2- 172.21.15.100:0/3784426907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33a81017f0 0x7f33a8101c10 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:48.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:48.841+0000 7f33aced7700 1 --2- 172.21.15.100:0/3784426907 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33a8104870 0x7f33a8104cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:48.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:48.841+0000 7f33aced7700 1 -- 172.21.15.100:0/3784426907 >> 172.21.15.100:0/3784426907 conn(0x7f33a80fd110 msgr2=0x7f33a80ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:17:48.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:48.841+0000 7f33aced7700 1 -- 172.21.15.100:0/3784426907 shutdown_connections 2024-05-10T11:17:48.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:48.841+0000 7f33aced7700 1 -- 172.21.15.100:0/3784426907 wait complete. 2024-05-10T11:17:48.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:48.841+0000 7f33aced7700 1 Processor -- start 2024-05-10T11:17:48.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:48.841+0000 7f33aced7700 1 -- start start 2024-05-10T11:17:48.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:48.842+0000 7f33aced7700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f33a81017f0 0x7f33a8199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:48.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:48.842+0000 7f33aced7700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33a8104870 0x7f33a8199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:48.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:48.842+0000 7f33aced7700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33a8105210 0x7f33a81a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:48.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:48.842+0000 7f33aced7700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f33a810ee70 con 0x7f33a8105210 2024-05-10T11:17:48.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:48.842+0000 7f33aced7700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f33a810ecf0 con 0x7f33a81017f0 2024-05-10T11:17:48.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:48.842+0000 7f33aced7700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f33a810eff0 con 0x7f33a8104870 2024-05-10T11:17:48.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:48.842+0000 7f33a6d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33a8105210 0x7f33a81a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:48.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:48.842+0000 7f33a6d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33a8105210 0x7f33a81a0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42116/0 (socket says 172.21.15.100:42116) 2024-05-10T11:17:48.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:48.842+0000 7f33a6d9d700 1 -- 172.21.15.100:0/783545410 learned_addr learned my addr 172.21.15.100:0/783545410 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:17:48.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:48.842+0000 7f33a5d9b700 1 --2- 172.21.15.100:0/783545410 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33a8104870 0x7f33a8199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:48.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:48.842+0000 7f33a659c700 1 --2- 172.21.15.100:0/783545410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f33a81017f0 0x7f33a8199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:48.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:48.842+0000 7f33a6d9d700 1 -- 172.21.15.100:0/783545410 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33a8104870 msgr2=0x7f33a8199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:48.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:48.842+0000 7f33a6d9d700 1 --2- 172.21.15.100:0/783545410 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33a8104870 0x7f33a8199a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:48.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:48.842+0000 7f33a6d9d700 1 -- 172.21.15.100:0/783545410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f33a81017f0 msgr2=0x7f33a8199550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:48.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:48.842+0000 7f33a6d9d700 1 --2- 172.21.15.100:0/783545410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f33a81017f0 0x7f33a8199550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:48.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:48.842+0000 7f33a6d9d700 1 -- 172.21.15.100:0/783545410 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33a81a11b0 con 0x7f33a8105210 2024-05-10T11:17:48.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:48.842+0000 7f33a6d9d700 1 --2- 172.21.15.100:0/783545410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33a8105210 0x7f33a81a0b10 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f33a0007f10 tx=0x7f33a000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:17:48.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:48.842+0000 7f33977fe700 1 -- 172.21.15.100:0/783545410 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a0004bb0 con 0x7f33a8105210 2024-05-10T11:17:48.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:48.843+0000 7f33977fe700 1 -- 172.21.15.100:0/783545410 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f33a0004d50 con 0x7f33a8105210 2024-05-10T11:17:48.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:48.843+0000 7f33aced7700 1 -- 172.21.15.100:0/783545410 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33a81a14a0 con 0x7f33a8105210 2024-05-10T11:17:48.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:48.843+0000 7f33977fe700 1 -- 172.21.15.100:0/783545410 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a00058b0 con 0x7f33a8105210 2024-05-10T11:17:48.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:48.843+0000 7f33aced7700 1 -- 172.21.15.100:0/783545410 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33a81a1990 con 0x7f33a8105210 2024-05-10T11:17:48.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:48.843+0000 7f33aced7700 1 -- 172.21.15.100:0/783545410 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33a806a190 con 0x7f33a8105210 2024-05-10T11:17:48.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:48.846+0000 7f33977fe700 1 -- 172.21.15.100:0/783545410 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f33a001a020 con 0x7f33a8105210 2024-05-10T11:17:48.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:48.846+0000 7f33977fe700 1 --2- 172.21.15.100:0/783545410 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f33900b2880 0x7f33900b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:48.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:48.846+0000 7f33a659c700 1 --2- 172.21.15.100:0/783545410 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f33900b2880 0x7f33900b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:48.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:48.847+0000 7f33977fe700 1 -- 172.21.15.100:0/783545410 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f33a00e2260 con 0x7f33a8105210 2024-05-10T11:17:48.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:48.847+0000 7f33a659c700 1 --2- 172.21.15.100:0/783545410 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f33900b2880 0x7f33900b4d40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f339c009990 tx=0x7f339c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:17:48.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:48.850+0000 7f33977fe700 1 -- 172.21.15.100:0/783545410 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f33a0010070 con 0x7f33a8105210 2024-05-10T11:17:49.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:49.062+0000 7f33aced7700 1 -- 172.21.15.100:0/783545410 --> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f33a80638e0 con 0x7f33900b2880 2024-05-10T11:17:49.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:49.068+0000 7f33977fe700 1 -- 172.21.15.100:0/783545410 <== mgr.14152 v2:172.21.15.100:6800/2507287356 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f33a80638e0 con 0x7f33900b2880 2024-05-10T11:17:49.068 INFO:teuthology.orchestra.run.smithi100.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-10T11:17:49.068 INFO:teuthology.orchestra.run.smithi100.stdout:mds.all 12/12 83s ago 4m smithi100=a;smithi100=d;smithi100=g;smithi100=j;smithi178=b;smithi178=e;smithi178=h;smithi178=k;smithi188=c;smithi188=f;smithi188=i;smithi188=l;count:12 2024-05-10T11:17:49.068 INFO:teuthology.orchestra.run.smithi100.stdout:mgr 3/3 83s ago 9m smithi100=x;smithi178=y;smithi188=z;count:3 2024-05-10T11:17:49.068 INFO:teuthology.orchestra.run.smithi100.stdout:mon 3/3 83s ago 10m smithi100:172.21.15.100=a;smithi178:172.21.15.178=b;smithi188:172.21.15.188=c;count:3 2024-05-10T11:17:49.068 INFO:teuthology.orchestra.run.smithi100.stdout:osd 12 83s ago - 2024-05-10T11:17:49.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:49.071+0000 7f33aced7700 1 -- 172.21.15.100:0/783545410 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f33900b2880 msgr2=0x7f33900b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:49.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:49.071+0000 7f33aced7700 1 --2- 172.21.15.100:0/783545410 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f33900b2880 0x7f33900b4d40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f339c009990 tx=0x7f339c005fb0 comp rx=0 tx=0).stop 2024-05-10T11:17:49.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:49.071+0000 7f33aced7700 1 -- 172.21.15.100:0/783545410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33a8105210 msgr2=0x7f33a81a0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:49.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:49.071+0000 7f33aced7700 1 --2- 172.21.15.100:0/783545410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33a8105210 0x7f33a81a0b10 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f33a0007f10 tx=0x7f33a000d3b0 comp rx=0 tx=0).stop 2024-05-10T11:17:49.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:49.071+0000 7f33aced7700 1 -- 172.21.15.100:0/783545410 shutdown_connections 2024-05-10T11:17:49.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:49.071+0000 7f33aced7700 1 --2- 172.21.15.100:0/783545410 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f33900b2880 0x7f33900b4d40 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:49.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:49.071+0000 7f33aced7700 1 --2- 172.21.15.100:0/783545410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33a8105210 0x7f33a81a0b10 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:49.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:49.071+0000 7f33aced7700 1 --2- 172.21.15.100:0/783545410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f33a81017f0 0x7f33a8199550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:49.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:49.071+0000 7f33aced7700 1 --2- 172.21.15.100:0/783545410 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33a8104870 0x7f33a8199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:49.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:49.071+0000 7f33aced7700 1 -- 172.21.15.100:0/783545410 >> 172.21.15.100:0/783545410 conn(0x7f33a80fd110 msgr2=0x7f33a8107c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:17:49.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:49.071+0000 7f33aced7700 1 -- 172.21.15.100:0/783545410 shutdown_connections 2024-05-10T11:17:49.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:49.071+0000 7f33aced7700 1 -- 172.21.15.100:0/783545410 wait complete. 2024-05-10T11:17:49.664 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:17:49 smithi100 ceph-mon[30296]: pgmap v403: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T11:17:49.693 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- bash -c 'ceph orch host ls' 2024-05-10T11:17:49.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:17:49 smithi178 ceph-mon[33359]: pgmap v403: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T11:17:49.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:17:49 smithi188 ceph-mon[34075]: pgmap v403: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T11:17:49.935 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:17:50.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:17:50 smithi178 ceph-mon[33359]: from='client.15099 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:17:50.771 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:17:50 smithi100 ceph-mon[30296]: from='client.15099 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:17:50.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:17:50 smithi188 ceph-mon[34075]: from='client.15099 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:17:51.601 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:17:51 smithi100 ceph-mon[30296]: pgmap v404: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T11:17:51.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:51.645+0000 7fe74193c700 1 -- 172.21.15.100:0/3143372743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe73c102400 msgr2=0x7fe73c102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:51.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:51.645+0000 7fe74193c700 1 --2- 172.21.15.100:0/3143372743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe73c102400 0x7fe73c102820 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe7340099c0 tx=0x7fe734009cd0 comp rx=0 tx=0).stop 2024-05-10T11:17:51.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:51.645+0000 7fe74193c700 1 -- 172.21.15.100:0/3143372743 shutdown_connections 2024-05-10T11:17:51.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:51.645+0000 7fe74193c700 1 --2- 172.21.15.100:0/3143372743 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe73c103fa0 0x7fe73c10a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:51.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:51.645+0000 7fe74193c700 1 --2- 172.21.15.100:0/3143372743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe73c102400 0x7fe73c102820 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:51.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:51.645+0000 7fe74193c700 1 --2- 172.21.15.100:0/3143372743 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe73c103600 0x7fe73c103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:51.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:51.645+0000 7fe74193c700 1 -- 172.21.15.100:0/3143372743 >> 172.21.15.100:0/3143372743 conn(0x7fe73c0fd9c0 msgr2=0x7fe73c0ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:17:51.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:51.645+0000 7fe74193c700 1 -- 172.21.15.100:0/3143372743 shutdown_connections 2024-05-10T11:17:51.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:51.645+0000 7fe74193c700 1 -- 172.21.15.100:0/3143372743 wait complete. 2024-05-10T11:17:51.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:51.646+0000 7fe74193c700 1 Processor -- start 2024-05-10T11:17:51.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:51.646+0000 7fe74193c700 1 -- start start 2024-05-10T11:17:51.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:51.646+0000 7fe74193c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe73c102400 0x7fe73c106390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:51.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:51.646+0000 7fe74193c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe73c103600 0x7fe73c1049e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:51.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:51.646+0000 7fe74193c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe73c103fa0 0x7fe73c104f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:51.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:51.646+0000 7fe74193c700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe73c10dd50 con 0x7fe73c103600 2024-05-10T11:17:51.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:51.646+0000 7fe74193c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe73c10dbd0 con 0x7fe73c102400 2024-05-10T11:17:51.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:51.646+0000 7fe74193c700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe73c10ded0 con 0x7fe73c103fa0 2024-05-10T11:17:51.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:51.646+0000 7fe73a7fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe73c103600 0x7fe73c1049e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:51.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:51.646+0000 7fe73a7fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe73c103600 0x7fe73c1049e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42132/0 (socket says 172.21.15.100:42132) 2024-05-10T11:17:51.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:51.646+0000 7fe73a7fc700 1 -- 172.21.15.100:0/4170288332 learned_addr learned my addr 172.21.15.100:0/4170288332 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:17:51.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:51.646+0000 7fe73affd700 1 --2- 172.21.15.100:0/4170288332 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe73c102400 0x7fe73c106390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:51.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:51.647+0000 7fe73b7fe700 1 --2- 172.21.15.100:0/4170288332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe73c103fa0 0x7fe73c104f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:51.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:51.647+0000 7fe73a7fc700 1 -- 172.21.15.100:0/4170288332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe73c103fa0 msgr2=0x7fe73c104f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:51.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:51.647+0000 7fe73a7fc700 1 --2- 172.21.15.100:0/4170288332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe73c103fa0 0x7fe73c104f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:51.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:51.647+0000 7fe73a7fc700 1 -- 172.21.15.100:0/4170288332 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe73c102400 msgr2=0x7fe73c106390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:51.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:51.647+0000 7fe73a7fc700 1 --2- 172.21.15.100:0/4170288332 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe73c102400 0x7fe73c106390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:51.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:51.647+0000 7fe73a7fc700 1 -- 172.21.15.100:0/4170288332 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe73c1057d0 con 0x7fe73c103600 2024-05-10T11:17:51.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:51.647+0000 7fe73a7fc700 1 --2- 172.21.15.100:0/4170288332 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe73c103600 0x7fe73c1049e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe72400d980 tx=0x7fe72400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:17:51.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:51.647+0000 7fe74093a700 1 -- 172.21.15.100:0/4170288332 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe724017070 con 0x7fe73c103600 2024-05-10T11:17:51.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:51.647+0000 7fe74193c700 1 -- 172.21.15.100:0/4170288332 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe73c1a3720 con 0x7fe73c103600 2024-05-10T11:17:51.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:51.647+0000 7fe74093a700 1 -- 172.21.15.100:0/4170288332 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe724005020 con 0x7fe73c103600 2024-05-10T11:17:51.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:51.647+0000 7fe74093a700 1 -- 172.21.15.100:0/4170288332 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72400f980 con 0x7fe73c103600 2024-05-10T11:17:51.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:51.647+0000 7fe74193c700 1 -- 172.21.15.100:0/4170288332 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe73c1a3c30 con 0x7fe73c103600 2024-05-10T11:17:51.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:51.648+0000 7fe74193c700 1 -- 172.21.15.100:0/4170288332 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe73c06a190 con 0x7fe73c103600 2024-05-10T11:17:51.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:51.649+0000 7fe74093a700 1 -- 172.21.15.100:0/4170288332 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fe724004a30 con 0x7fe73c103600 2024-05-10T11:17:51.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:51.650+0000 7fe74093a700 1 --2- 172.21.15.100:0/4170288332 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe7280b2880 0x7fe7280b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:51.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:51.650+0000 7fe73affd700 1 --2- 172.21.15.100:0/4170288332 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe7280b2880 0x7fe7280b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:51.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:51.650+0000 7fe74093a700 1 -- 172.21.15.100:0/4170288332 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fe7240e27a0 con 0x7fe73c103600 2024-05-10T11:17:51.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:51.651+0000 7fe73affd700 1 --2- 172.21.15.100:0/4170288332 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe7280b2880 0x7fe7280b4d40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe734009990 tx=0x7fe734005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:17:51.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:51.654+0000 7fe74093a700 1 -- 172.21.15.100:0/4170288332 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fe724020020 con 0x7fe73c103600 2024-05-10T11:17:51.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:17:51 smithi178 ceph-mon[33359]: pgmap v404: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T11:17:51.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:17:51 smithi188 ceph-mon[34075]: pgmap v404: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T11:17:51.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:51.861+0000 7fe74193c700 1 -- 172.21.15.100:0/4170288332 --> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7fe73c06c280 con 0x7fe7280b2880 2024-05-10T11:17:51.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:51.863+0000 7fe74093a700 1 -- 172.21.15.100:0/4170288332 <== mgr.14152 v2:172.21.15.100:6800/2507287356 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7fe73c06c280 con 0x7fe7280b2880 2024-05-10T11:17:51.863 INFO:teuthology.orchestra.run.smithi100.stdout:HOST ADDR LABELS STATUS 2024-05-10T11:17:51.863 INFO:teuthology.orchestra.run.smithi100.stdout:smithi100 172.21.15.100 2024-05-10T11:17:51.863 INFO:teuthology.orchestra.run.smithi100.stdout:smithi178 172.21.15.178 2024-05-10T11:17:51.863 INFO:teuthology.orchestra.run.smithi100.stdout:smithi188 172.21.15.188 2024-05-10T11:17:51.863 INFO:teuthology.orchestra.run.smithi100.stdout:3 hosts in cluster 2024-05-10T11:17:51.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:51.865+0000 7fe74193c700 1 -- 172.21.15.100:0/4170288332 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe7280b2880 msgr2=0x7fe7280b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:51.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:51.865+0000 7fe74193c700 1 --2- 172.21.15.100:0/4170288332 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe7280b2880 0x7fe7280b4d40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe734009990 tx=0x7fe734005fb0 comp rx=0 tx=0).stop 2024-05-10T11:17:51.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:51.865+0000 7fe74193c700 1 -- 172.21.15.100:0/4170288332 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe73c103600 msgr2=0x7fe73c1049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:51.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:51.865+0000 7fe74193c700 1 --2- 172.21.15.100:0/4170288332 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe73c103600 0x7fe73c1049e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe72400d980 tx=0x7fe72400dc90 comp rx=0 tx=0).stop 2024-05-10T11:17:51.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:51.865+0000 7fe74193c700 1 -- 172.21.15.100:0/4170288332 shutdown_connections 2024-05-10T11:17:51.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:51.865+0000 7fe74193c700 1 --2- 172.21.15.100:0/4170288332 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe7280b2880 0x7fe7280b4d40 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:51.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:51.865+0000 7fe74193c700 1 --2- 172.21.15.100:0/4170288332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe73c103fa0 0x7fe73c104f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:51.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:51.865+0000 7fe74193c700 1 --2- 172.21.15.100:0/4170288332 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe73c102400 0x7fe73c106390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:51.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:51.865+0000 7fe74193c700 1 --2- 172.21.15.100:0/4170288332 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe73c103600 0x7fe73c1049e0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:51.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:51.865+0000 7fe74193c700 1 -- 172.21.15.100:0/4170288332 >> 172.21.15.100:0/4170288332 conn(0x7fe73c0fd9c0 msgr2=0x7fe73c06bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:17:51.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:51.865+0000 7fe74193c700 1 -- 172.21.15.100:0/4170288332 shutdown_connections 2024-05-10T11:17:51.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:51.865+0000 7fe74193c700 1 -- 172.21.15.100:0/4170288332 wait complete. 2024-05-10T11:17:52.522 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:17:52 smithi100 ceph-mon[30296]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:17:52.552 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- bash -c 'ceph orch device ls' 2024-05-10T11:17:52.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:17:52 smithi178 ceph-mon[33359]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:17:52.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:17:52 smithi188 ceph-mon[34075]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:17:52.794 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:17:53.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:17:53 smithi100 ceph-mon[30296]: pgmap v405: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T11:17:53.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:17:53 smithi178 ceph-mon[33359]: pgmap v405: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T11:17:53.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:17:53 smithi188 ceph-mon[34075]: pgmap v405: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T11:17:54.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:54.473+0000 7fe351e42700 1 -- 172.21.15.100:0/2348842875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe34c104700 msgr2=0x7fe34c104b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:54.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:54.473+0000 7fe351e42700 1 --2- 172.21.15.100:0/2348842875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe34c104700 0x7fe34c104b60 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe33c0099c0 tx=0x7fe33c009cd0 comp rx=0 tx=0).stop 2024-05-10T11:17:54.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:54.473+0000 7fe351e42700 1 -- 172.21.15.100:0/2348842875 shutdown_connections 2024-05-10T11:17:54.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:54.473+0000 7fe351e42700 1 --2- 172.21.15.100:0/2348842875 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe34c1050a0 0x7fe34c10b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:54.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:54.473+0000 7fe351e42700 1 --2- 172.21.15.100:0/2348842875 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe34c103500 0x7fe34c103920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:54.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:54.473+0000 7fe351e42700 1 --2- 172.21.15.100:0/2348842875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe34c104700 0x7fe34c104b60 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:54.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:54.473+0000 7fe351e42700 1 -- 172.21.15.100:0/2348842875 >> 172.21.15.100:0/2348842875 conn(0x7fe34c0feaa0 msgr2=0x7fe34c100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:17:54.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:54.473+0000 7fe351e42700 1 -- 172.21.15.100:0/2348842875 shutdown_connections 2024-05-10T11:17:54.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:54.473+0000 7fe351e42700 1 -- 172.21.15.100:0/2348842875 wait complete. 2024-05-10T11:17:54.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:54.474+0000 7fe351e42700 1 Processor -- start 2024-05-10T11:17:54.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:54.474+0000 7fe351e42700 1 -- start start 2024-05-10T11:17:54.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:54.474+0000 7fe351e42700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe34c103500 0x7fe34c1995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:54.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:54.474+0000 7fe351e42700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe34c104700 0x7fe34c199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:54.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:54.474+0000 7fe351e42700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe34c1050a0 0x7fe34c1a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:54.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:54.474+0000 7fe351e42700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe34c10eec0 con 0x7fe34c103500 2024-05-10T11:17:54.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:54.474+0000 7fe351e42700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe34c10ed40 con 0x7fe34c1050a0 2024-05-10T11:17:54.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:54.474+0000 7fe351e42700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe34c10f040 con 0x7fe34c104700 2024-05-10T11:17:54.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:54.474+0000 7fe34b7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe34c103500 0x7fe34c1995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:54.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:54.474+0000 7fe34b7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe34c103500 0x7fe34c1995f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42154/0 (socket says 172.21.15.100:42154) 2024-05-10T11:17:54.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:54.474+0000 7fe34b7fe700 1 -- 172.21.15.100:0/2277719733 learned_addr learned my addr 172.21.15.100:0/2277719733 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:17:54.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:54.475+0000 7fe34affd700 1 --2- 172.21.15.100:0/2277719733 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe34c104700 0x7fe34c199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:54.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:54.475+0000 7fe34b7fe700 1 -- 172.21.15.100:0/2277719733 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe34c104700 msgr2=0x7fe34c199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:54.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:54.475+0000 7fe34b7fe700 1 --2- 172.21.15.100:0/2277719733 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe34c104700 0x7fe34c199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:54.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:54.475+0000 7fe34bfff700 1 --2- 172.21.15.100:0/2277719733 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe34c1050a0 0x7fe34c1a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:54.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:54.475+0000 7fe34b7fe700 1 -- 172.21.15.100:0/2277719733 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe34c1050a0 msgr2=0x7fe34c1a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:54.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:54.475+0000 7fe34b7fe700 1 --2- 172.21.15.100:0/2277719733 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe34c1050a0 0x7fe34c1a0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:54.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:54.475+0000 7fe34b7fe700 1 -- 172.21.15.100:0/2277719733 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe34c1a1250 con 0x7fe34c103500 2024-05-10T11:17:54.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:54.475+0000 7fe34affd700 1 --2- 172.21.15.100:0/2277719733 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe34c104700 0x7fe34c199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:17:54.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:54.475+0000 7fe34b7fe700 1 --2- 172.21.15.100:0/2277719733 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe34c103500 0x7fe34c1995f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe33400d9b0 tx=0x7fe33400dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:17:54.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:54.475+0000 7fe348ff9700 1 -- 172.21.15.100:0/2277719733 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe334017070 con 0x7fe34c103500 2024-05-10T11:17:54.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:54.475+0000 7fe348ff9700 1 -- 172.21.15.100:0/2277719733 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe334010460 con 0x7fe34c103500 2024-05-10T11:17:54.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:54.475+0000 7fe351e42700 1 -- 172.21.15.100:0/2277719733 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe34c1a1540 con 0x7fe34c103500 2024-05-10T11:17:54.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:54.475+0000 7fe348ff9700 1 -- 172.21.15.100:0/2277719733 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe33400f620 con 0x7fe34c103500 2024-05-10T11:17:54.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:54.475+0000 7fe351e42700 1 -- 172.21.15.100:0/2277719733 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe34c1a1a30 con 0x7fe34c103500 2024-05-10T11:17:54.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:54.476+0000 7fe351e42700 1 -- 172.21.15.100:0/2277719733 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe34c06a190 con 0x7fe34c103500 2024-05-10T11:17:54.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:54.478+0000 7fe348ff9700 1 -- 172.21.15.100:0/2277719733 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fe334010610 con 0x7fe34c103500 2024-05-10T11:17:54.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:54.478+0000 7fe348ff9700 1 --2- 172.21.15.100:0/2277719733 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe3380b2880 0x7fe3380b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:54.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:54.478+0000 7fe34affd700 1 --2- 172.21.15.100:0/2277719733 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe3380b2880 0x7fe3380b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:54.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:54.479+0000 7fe348ff9700 1 -- 172.21.15.100:0/2277719733 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fe3340e20f0 con 0x7fe34c103500 2024-05-10T11:17:54.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:54.479+0000 7fe34affd700 1 --2- 172.21.15.100:0/2277719733 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe3380b2880 0x7fe3380b4d40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe33c00b580 tx=0x7fe33c019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:17:54.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:54.482+0000 7fe348ff9700 1 -- 172.21.15.100:0/2277719733 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fe3340a9ab0 con 0x7fe34c103500 2024-05-10T11:17:54.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:54.695+0000 7fe351e42700 1 -- 172.21.15.100:0/2277719733 --> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fe34c0639a0 con 0x7fe3380b2880 2024-05-10T11:17:54.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:54.697+0000 7fe348ff9700 1 -- 172.21.15.100:0/2277719733 <== mgr.14152 v2:172.21.15.100:6800/2507287356 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fe34c0639a0 con 0x7fe3380b2880 2024-05-10T11:17:54.697 INFO:teuthology.orchestra.run.smithi100.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-05-10T11:17:54.697 INFO:teuthology.orchestra.run.smithi100.stdout:smithi100 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400WR400BGN 372G No 5m ago Has a FileSystem, LVM detected 2024-05-10T11:17:54.697 INFO:teuthology.orchestra.run.smithi100.stdout:smithi178 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB75130029480DGN 447G No 5m ago Has a FileSystem, LVM detected 2024-05-10T11:17:54.697 INFO:teuthology.orchestra.run.smithi100.stdout:smithi188 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500MP400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-05-10T11:17:54.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:54.699+0000 7fe351e42700 1 -- 172.21.15.100:0/2277719733 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe3380b2880 msgr2=0x7fe3380b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:54.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:54.699+0000 7fe351e42700 1 --2- 172.21.15.100:0/2277719733 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe3380b2880 0x7fe3380b4d40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe33c00b580 tx=0x7fe33c019040 comp rx=0 tx=0).stop 2024-05-10T11:17:54.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:54.699+0000 7fe351e42700 1 -- 172.21.15.100:0/2277719733 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe34c103500 msgr2=0x7fe34c1995f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:54.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:54.699+0000 7fe351e42700 1 --2- 172.21.15.100:0/2277719733 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe34c103500 0x7fe34c1995f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe33400d9b0 tx=0x7fe33400dd70 comp rx=0 tx=0).stop 2024-05-10T11:17:54.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:54.700+0000 7fe351e42700 1 -- 172.21.15.100:0/2277719733 shutdown_connections 2024-05-10T11:17:54.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:54.700+0000 7fe351e42700 1 --2- 172.21.15.100:0/2277719733 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe3380b2880 0x7fe3380b4d40 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:54.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:54.700+0000 7fe351e42700 1 --2- 172.21.15.100:0/2277719733 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe34c1050a0 0x7fe34c1a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:54.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:54.700+0000 7fe351e42700 1 --2- 172.21.15.100:0/2277719733 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe34c103500 0x7fe34c1995f0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:54.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:54.700+0000 7fe351e42700 1 --2- 172.21.15.100:0/2277719733 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe34c104700 0x7fe34c199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:54.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:54.700+0000 7fe351e42700 1 -- 172.21.15.100:0/2277719733 >> 172.21.15.100:0/2277719733 conn(0x7fe34c0feaa0 msgr2=0x7fe34c107b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:17:54.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:54.700+0000 7fe351e42700 1 -- 172.21.15.100:0/2277719733 shutdown_connections 2024-05-10T11:17:54.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:54.700+0000 7fe351e42700 1 -- 172.21.15.100:0/2277719733 wait complete. 2024-05-10T11:17:55.344 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-10T11:17:55.354 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi100.front.sepia.ceph.com 2024-05-10T11:17:55.354 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-10T11:17:55.596 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:17:55.718 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:17:55 smithi100 ceph-mon[30296]: pgmap v406: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-10T11:17:55.718 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:17:55 smithi100 ceph-mon[30296]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:17:55.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:17:55 smithi178 ceph-mon[33359]: pgmap v406: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-10T11:17:55.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:17:55 smithi178 ceph-mon[33359]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:17:55.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:17:55 smithi188 ceph-mon[34075]: pgmap v406: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-10T11:17:55.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:17:55 smithi188 ceph-mon[34075]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T11:17:57.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:57.279+0000 7fc2753cd700 1 -- 172.21.15.100:0/3565116934 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2701068e0 msgr2=0x7fc270108cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:57.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:57.279+0000 7fc2753cd700 1 --2- 172.21.15.100:0/3565116934 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2701068e0 0x7fc270108cd0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc258009990 tx=0x7fc258009ca0 comp rx=0 tx=0).stop 2024-05-10T11:17:57.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:57.279+0000 7fc2753cd700 1 -- 172.21.15.100:0/3565116934 shutdown_connections 2024-05-10T11:17:57.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:57.279+0000 7fc2753cd700 1 --2- 172.21.15.100:0/3565116934 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc270109210 0x7fc27010b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:57.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:57.279+0000 7fc2753cd700 1 --2- 172.21.15.100:0/3565116934 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc270103fb0 0x7fc2701063a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:57.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:57.279+0000 7fc2753cd700 1 --2- 172.21.15.100:0/3565116934 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2701068e0 0x7fc270108cd0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:57.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:57.279+0000 7fc2753cd700 1 -- 172.21.15.100:0/3565116934 >> 172.21.15.100:0/3565116934 conn(0x7fc2700fda00 msgr2=0x7fc2700ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:17:57.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:57.279+0000 7fc2753cd700 1 -- 172.21.15.100:0/3565116934 shutdown_connections 2024-05-10T11:17:57.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:57.279+0000 7fc2753cd700 1 -- 172.21.15.100:0/3565116934 wait complete. 2024-05-10T11:17:57.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:57.279+0000 7fc2753cd700 1 Processor -- start 2024-05-10T11:17:57.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:57.280+0000 7fc2753cd700 1 -- start start 2024-05-10T11:17:57.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:57.280+0000 7fc2753cd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc270103fb0 0x7fc2701995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:57.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:57.280+0000 7fc2753cd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2701068e0 0x7fc270199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:57.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:57.280+0000 7fc2753cd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc270109210 0x7fc2701a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:57.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:57.280+0000 7fc2753cd700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc27010ee90 con 0x7fc270103fb0 2024-05-10T11:17:57.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:57.280+0000 7fc2753cd700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc27010ed10 con 0x7fc270109210 2024-05-10T11:17:57.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:57.280+0000 7fc2753cd700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc27010f010 con 0x7fc2701068e0 2024-05-10T11:17:57.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:57.280+0000 7fc26effd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc270103fb0 0x7fc2701995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:57.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:57.280+0000 7fc26effd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc270103fb0 0x7fc2701995a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37142/0 (socket says 172.21.15.100:37142) 2024-05-10T11:17:57.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:57.280+0000 7fc26effd700 1 -- 172.21.15.100:0/572047706 learned_addr learned my addr 172.21.15.100:0/572047706 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:17:57.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:57.280+0000 7fc2667fc700 1 --2- 172.21.15.100:0/572047706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2701068e0 0x7fc270199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:57.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:57.280+0000 7fc26f7fe700 1 --2- 172.21.15.100:0/572047706 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc270109210 0x7fc2701a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:57.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:57.280+0000 7fc26effd700 1 -- 172.21.15.100:0/572047706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2701068e0 msgr2=0x7fc270199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:57.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:57.280+0000 7fc26effd700 1 --2- 172.21.15.100:0/572047706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2701068e0 0x7fc270199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:57.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:57.280+0000 7fc26effd700 1 -- 172.21.15.100:0/572047706 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc270109210 msgr2=0x7fc2701a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:57.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:57.280+0000 7fc26effd700 1 --2- 172.21.15.100:0/572047706 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc270109210 0x7fc2701a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:57.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:57.280+0000 7fc26effd700 1 -- 172.21.15.100:0/572047706 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2701a1200 con 0x7fc270103fb0 2024-05-10T11:17:57.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:57.280+0000 7fc26f7fe700 1 --2- 172.21.15.100:0/572047706 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc270109210 0x7fc2701a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:17:57.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:57.281+0000 7fc26effd700 1 --2- 172.21.15.100:0/572047706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc270103fb0 0x7fc2701995a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc26000d9b0 tx=0x7fc26000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:17:57.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:57.281+0000 7fc26cff9700 1 -- 172.21.15.100:0/572047706 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc260017070 con 0x7fc270103fb0 2024-05-10T11:17:57.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:57.281+0000 7fc2753cd700 1 -- 172.21.15.100:0/572047706 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2701a14f0 con 0x7fc270103fb0 2024-05-10T11:17:57.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:57.281+0000 7fc26cff9700 1 -- 172.21.15.100:0/572047706 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc260005020 con 0x7fc270103fb0 2024-05-10T11:17:57.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:57.281+0000 7fc26cff9700 1 -- 172.21.15.100:0/572047706 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc260009ab0 con 0x7fc270103fb0 2024-05-10T11:17:57.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:57.281+0000 7fc2753cd700 1 -- 172.21.15.100:0/572047706 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2701a1a30 con 0x7fc270103fb0 2024-05-10T11:17:57.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:57.282+0000 7fc2753cd700 1 -- 172.21.15.100:0/572047706 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc27006a190 con 0x7fc270103fb0 2024-05-10T11:17:57.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:57.284+0000 7fc26cff9700 1 -- 172.21.15.100:0/572047706 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fc2600051d0 con 0x7fc270103fb0 2024-05-10T11:17:57.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:57.284+0000 7fc26cff9700 1 --2- 172.21.15.100:0/572047706 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc25c0b2900 0x7fc25c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:17:57.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:57.285+0000 7fc2667fc700 1 --2- 172.21.15.100:0/572047706 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc25c0b2900 0x7fc25c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:17:57.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:57.285+0000 7fc26cff9700 1 -- 172.21.15.100:0/572047706 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fc2600e2840 con 0x7fc270103fb0 2024-05-10T11:17:57.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:57.285+0000 7fc2667fc700 1 --2- 172.21.15.100:0/572047706 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc25c0b2900 0x7fc25c0b4dc0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc25800b5c0 tx=0x7fc258005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:17:57.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:57.288+0000 7fc26cff9700 1 -- 172.21.15.100:0/572047706 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fc260020020 con 0x7fc270103fb0 2024-05-10T11:17:57.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:57.537+0000 7fc2753cd700 1 -- 172.21.15.100:0/572047706 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fc2700640b0 con 0x7fc270103fb0 2024-05-10T11:17:57.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:57.538+0000 7fc26cff9700 1 -- 172.21.15.100:0/572047706 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) ==== 76+0+3066 (secure 0 0 0) 0x7fc2600aa280 con 0x7fc270103fb0 2024-05-10T11:17:57.538 INFO:teuthology.orchestra.run.smithi100.stdout:e26 2024-05-10T11:17:57.538 INFO:teuthology.orchestra.run.smithi100.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-10T11:17:57.538 INFO:teuthology.orchestra.run.smithi100.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-10T11:17:57.538 INFO:teuthology.orchestra.run.smithi100.stdout:legacy client fscid: 1 2024-05-10T11:17:57.538 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:17:57.538 INFO:teuthology.orchestra.run.smithi100.stdout:Filesystem 'cephfs' (1) 2024-05-10T11:17:57.538 INFO:teuthology.orchestra.run.smithi100.stdout:fs_name cephfs 2024-05-10T11:17:57.538 INFO:teuthology.orchestra.run.smithi100.stdout:epoch 26 2024-05-10T11:17:57.539 INFO:teuthology.orchestra.run.smithi100.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-10T11:17:57.539 INFO:teuthology.orchestra.run.smithi100.stdout:created 2024-05-10T11:14:27.242452+0000 2024-05-10T11:17:57.539 INFO:teuthology.orchestra.run.smithi100.stdout:modified 2024-05-10T11:15:35.672461+0000 2024-05-10T11:17:57.539 INFO:teuthology.orchestra.run.smithi100.stdout:tableserver 0 2024-05-10T11:17:57.539 INFO:teuthology.orchestra.run.smithi100.stdout:root 0 2024-05-10T11:17:57.539 INFO:teuthology.orchestra.run.smithi100.stdout:session_timeout 300 2024-05-10T11:17:57.539 INFO:teuthology.orchestra.run.smithi100.stdout:session_autoclose 300 2024-05-10T11:17:57.539 INFO:teuthology.orchestra.run.smithi100.stdout:max_file_size 1099511627776 2024-05-10T11:17:57.539 INFO:teuthology.orchestra.run.smithi100.stdout:max_xattr_size 65536 2024-05-10T11:17:57.539 INFO:teuthology.orchestra.run.smithi100.stdout:required_client_features {} 2024-05-10T11:17:57.539 INFO:teuthology.orchestra.run.smithi100.stdout:last_failure 0 2024-05-10T11:17:57.539 INFO:teuthology.orchestra.run.smithi100.stdout:last_failure_osd_epoch 0 2024-05-10T11:17:57.539 INFO:teuthology.orchestra.run.smithi100.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-10T11:17:57.539 INFO:teuthology.orchestra.run.smithi100.stdout:max_mds 3 2024-05-10T11:17:57.540 INFO:teuthology.orchestra.run.smithi100.stdout:in 0,1,2 2024-05-10T11:17:57.540 INFO:teuthology.orchestra.run.smithi100.stdout:up {0=24439,1=24437,2=24427} 2024-05-10T11:17:57.540 INFO:teuthology.orchestra.run.smithi100.stdout:failed 2024-05-10T11:17:57.540 INFO:teuthology.orchestra.run.smithi100.stdout:damaged 2024-05-10T11:17:57.540 INFO:teuthology.orchestra.run.smithi100.stdout:stopped 2024-05-10T11:17:57.540 INFO:teuthology.orchestra.run.smithi100.stdout:data_pools [3,4] 2024-05-10T11:17:57.540 INFO:teuthology.orchestra.run.smithi100.stdout:metadata_pool 2 2024-05-10T11:17:57.540 INFO:teuthology.orchestra.run.smithi100.stdout:inline_data disabled 2024-05-10T11:17:57.540 INFO:teuthology.orchestra.run.smithi100.stdout:balancer 2024-05-10T11:17:57.540 INFO:teuthology.orchestra.run.smithi100.stdout:bal_rank_mask -1 2024-05-10T11:17:57.540 INFO:teuthology.orchestra.run.smithi100.stdout:standby_count_wanted 0 2024-05-10T11:17:57.540 INFO:teuthology.orchestra.run.smithi100.stdout:qdb_cluster leader: 24439 members: 24437,24439,24427 2024-05-10T11:17:57.540 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.i{0:24439} state up:active seq 8 addr [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] compat {c=[1],r=[1],i=[1fff]}] 2024-05-10T11:17:57.540 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.f{0:24421} state up:standby-replay seq 1 addr [v2:172.21.15.188:6834/515547690,v1:172.21.15.188:6835/515547690] compat {c=[1],r=[1],i=[1fff]}] 2024-05-10T11:17:57.540 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.b{1:24437} state up:active seq 21 addr [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] compat {c=[1],r=[1],i=[1fff]}] 2024-05-10T11:17:57.541 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.h{1:24413} state up:standby-replay seq 1 addr [v2:172.21.15.178:6836/3224358425,v1:172.21.15.178:6837/3224358425] compat {c=[1],r=[1],i=[1fff]}] 2024-05-10T11:17:57.541 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.c{2:24427} state up:active seq 25 addr [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] compat {c=[1],r=[1],i=[1fff]}] 2024-05-10T11:17:57.541 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.e{2:24395} state up:standby-replay seq 1 addr [v2:172.21.15.178:6834/2124076312,v1:172.21.15.178:6835/2124076312] compat {c=[1],r=[1],i=[1fff]}] 2024-05-10T11:17:57.541 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:17:57.541 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:17:57.541 INFO:teuthology.orchestra.run.smithi100.stdout:Standby daemons: 2024-05-10T11:17:57.541 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:17:57.541 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.j{-1:14520} state up:standby seq 1 addr [v2:172.21.15.100:6834/2072225012,v1:172.21.15.100:6835/2072225012] compat {c=[1],r=[1],i=[1fff]}] 2024-05-10T11:17:57.541 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.d{-1:14532} state up:standby seq 1 addr [v2:172.21.15.100:6836/401939540,v1:172.21.15.100:6837/401939540] compat {c=[1],r=[1],i=[1fff]}] 2024-05-10T11:17:57.541 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.g{-1:14571} state up:standby seq 1 addr [v2:172.21.15.100:6838/3946311025,v1:172.21.15.100:6839/3946311025] compat {c=[1],r=[1],i=[1fff]}] 2024-05-10T11:17:57.541 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.a{-1:14583} state up:standby seq 1 addr [v2:172.21.15.100:6840/3736371660,v1:172.21.15.100:6841/3736371660] compat {c=[1],r=[1],i=[1fff]}] 2024-05-10T11:17:57.541 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.k{-1:24383} state up:standby seq 1 addr [v2:172.21.15.178:6832/2741810791,v1:172.21.15.178:6833/2741810791] compat {c=[1],r=[1],i=[1fff]}] 2024-05-10T11:17:57.541 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.l{-1:24388} state up:standby seq 1 addr [v2:172.21.15.188:6832/1704891041,v1:172.21.15.188:6833/1704891041] compat {c=[1],r=[1],i=[1fff]}] 2024-05-10T11:17:57.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:57.541+0000 7fc2753cd700 1 -- 172.21.15.100:0/572047706 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc25c0b2900 msgr2=0x7fc25c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:57.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:57.541+0000 7fc2753cd700 1 --2- 172.21.15.100:0/572047706 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc25c0b2900 0x7fc25c0b4dc0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc25800b5c0 tx=0x7fc258005fd0 comp rx=0 tx=0).stop 2024-05-10T11:17:57.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:57.541+0000 7fc2753cd700 1 -- 172.21.15.100:0/572047706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc270103fb0 msgr2=0x7fc2701995a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:17:57.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:57.541+0000 7fc2753cd700 1 --2- 172.21.15.100:0/572047706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc270103fb0 0x7fc2701995a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc26000d9b0 tx=0x7fc26000dcc0 comp rx=0 tx=0).stop 2024-05-10T11:17:57.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:57.541+0000 7fc2753cd700 1 -- 172.21.15.100:0/572047706 shutdown_connections 2024-05-10T11:17:57.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:57.541+0000 7fc2753cd700 1 --2- 172.21.15.100:0/572047706 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc25c0b2900 0x7fc25c0b4dc0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:57.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:57.541+0000 7fc2753cd700 1 --2- 172.21.15.100:0/572047706 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc270109210 0x7fc2701a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:57.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:57.541+0000 7fc2753cd700 1 --2- 172.21.15.100:0/572047706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc270103fb0 0x7fc2701995a0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:57.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:57.541+0000 7fc2753cd700 1 --2- 172.21.15.100:0/572047706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2701068e0 0x7fc270199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:17:57.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:57.541+0000 7fc2753cd700 1 -- 172.21.15.100:0/572047706 >> 172.21.15.100:0/572047706 conn(0x7fc2700fda00 msgr2=0x7fc270105e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:17:57.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:57.541+0000 7fc2753cd700 1 -- 172.21.15.100:0/572047706 shutdown_connections 2024-05-10T11:17:57.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:17:57.541+0000 7fc2753cd700 1 -- 172.21.15.100:0/572047706 wait complete. 2024-05-10T11:17:57.543 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 26 2024-05-10T11:17:57.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:17:57 smithi178 ceph-mon[33359]: pgmap v407: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T11:17:57.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:17:57 smithi188 ceph-mon[34075]: pgmap v407: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T11:17:57.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:17:57 smithi100 ceph-mon[30296]: pgmap v407: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T11:17:58.262 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- bash -c 'ceph osd dump' 2024-05-10T11:17:58.514 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:17:58.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:17:58 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/572047706' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-10T11:17:58.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:17:58 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/572047706' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-10T11:17:58.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:17:58 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/572047706' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-10T11:17:59.673 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:17:59 smithi100 ceph-mon[30296]: pgmap v408: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T11:17:59.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:17:59 smithi178 ceph-mon[33359]: pgmap v408: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T11:17:59.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:17:59 smithi188 ceph-mon[34075]: pgmap v408: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T11:18:00.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:00.198+0000 7f6d50f87700 1 -- 172.21.15.100:0/1386611867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d4c103600 msgr2=0x7f6d4c103a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:00.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:00.198+0000 7f6d50f87700 1 --2- 172.21.15.100:0/1386611867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d4c103600 0x7f6d4c103a60 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f6d400099c0 tx=0x7f6d40009cd0 comp rx=0 tx=0).stop 2024-05-10T11:18:00.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:00.198+0000 7f6d50f87700 1 -- 172.21.15.100:0/1386611867 shutdown_connections 2024-05-10T11:18:00.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:00.198+0000 7f6d50f87700 1 --2- 172.21.15.100:0/1386611867 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d4c103fa0 0x7f6d4c10a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:00.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:00.198+0000 7f6d50f87700 1 --2- 172.21.15.100:0/1386611867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d4c102400 0x7f6d4c102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:00.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:00.198+0000 7f6d50f87700 1 --2- 172.21.15.100:0/1386611867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d4c103600 0x7f6d4c103a60 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:00.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:00.198+0000 7f6d50f87700 1 -- 172.21.15.100:0/1386611867 >> 172.21.15.100:0/1386611867 conn(0x7f6d4c0fd9a0 msgr2=0x7f6d4c0ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:00.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:00.198+0000 7f6d50f87700 1 -- 172.21.15.100:0/1386611867 shutdown_connections 2024-05-10T11:18:00.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:00.199+0000 7f6d50f87700 1 -- 172.21.15.100:0/1386611867 wait complete. 2024-05-10T11:18:00.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:00.199+0000 7f6d50f87700 1 Processor -- start 2024-05-10T11:18:00.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:00.199+0000 7f6d50f87700 1 -- start start 2024-05-10T11:18:00.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:00.199+0000 7f6d50f87700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d4c102400 0x7f6d4c1995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:00.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:00.199+0000 7f6d50f87700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d4c103600 0x7f6d4c199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:00.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:00.199+0000 7f6d50f87700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d4c103fa0 0x7f6d4c1a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:00.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:00.199+0000 7f6d50f87700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6d4c10dd50 con 0x7f6d4c102400 2024-05-10T11:18:00.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:00.199+0000 7f6d50f87700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6d4c10dbd0 con 0x7f6d4c103fa0 2024-05-10T11:18:00.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:00.199+0000 7f6d50f87700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6d4c10ded0 con 0x7f6d4c103600 2024-05-10T11:18:00.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:00.200+0000 7f6d4a59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d4c102400 0x7f6d4c1995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:00.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:00.200+0000 7f6d4a59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d4c102400 0x7f6d4c1995c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37170/0 (socket says 172.21.15.100:37170) 2024-05-10T11:18:00.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:00.200+0000 7f6d4a59c700 1 -- 172.21.15.100:0/2651614690 learned_addr learned my addr 172.21.15.100:0/2651614690 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:18:00.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:00.200+0000 7f6d49d9b700 1 --2- 172.21.15.100:0/2651614690 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d4c103600 0x7f6d4c199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:00.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:00.200+0000 7f6d4ad9d700 1 --2- 172.21.15.100:0/2651614690 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d4c103fa0 0x7f6d4c1a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:00.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:00.200+0000 7f6d4a59c700 1 -- 172.21.15.100:0/2651614690 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d4c103600 msgr2=0x7f6d4c199b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:00.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:00.200+0000 7f6d4a59c700 1 --2- 172.21.15.100:0/2651614690 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d4c103600 0x7f6d4c199b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:00.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:00.200+0000 7f6d4a59c700 1 -- 172.21.15.100:0/2651614690 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d4c103fa0 msgr2=0x7f6d4c1a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:00.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:00.200+0000 7f6d4a59c700 1 --2- 172.21.15.100:0/2651614690 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d4c103fa0 0x7f6d4c1a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:00.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:00.200+0000 7f6d4a59c700 1 -- 172.21.15.100:0/2651614690 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d4c104820 con 0x7f6d4c102400 2024-05-10T11:18:00.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:00.200+0000 7f6d4a59c700 1 --2- 172.21.15.100:0/2651614690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d4c102400 0x7f6d4c1995c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c00d9b0 tx=0x7f6d3c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:00.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:00.200+0000 7f6d3b7fe700 1 -- 172.21.15.100:0/2651614690 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c017030 con 0x7f6d4c102400 2024-05-10T11:18:00.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:00.200+0000 7f6d50f87700 1 -- 172.21.15.100:0/2651614690 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d4c104b10 con 0x7f6d4c102400 2024-05-10T11:18:00.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:00.200+0000 7f6d3b7fe700 1 -- 172.21.15.100:0/2651614690 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d3c0044b0 con 0x7f6d4c102400 2024-05-10T11:18:00.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:00.200+0000 7f6d3b7fe700 1 -- 172.21.15.100:0/2651614690 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c0038e0 con 0x7f6d4c102400 2024-05-10T11:18:00.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:00.200+0000 7f6d50f87700 1 -- 172.21.15.100:0/2651614690 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d4c1050a0 con 0x7f6d4c102400 2024-05-10T11:18:00.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:00.202+0000 7f6d50f87700 1 -- 172.21.15.100:0/2651614690 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d4c06a190 con 0x7f6d4c102400 2024-05-10T11:18:00.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:00.203+0000 7f6d3b7fe700 1 -- 172.21.15.100:0/2651614690 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f6d3c01a070 con 0x7f6d4c102400 2024-05-10T11:18:00.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:00.203+0000 7f6d3b7fe700 1 --2- 172.21.15.100:0/2651614690 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f6d340b2880 0x7f6d340b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:00.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:00.204+0000 7f6d49d9b700 1 --2- 172.21.15.100:0/2651614690 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f6d340b2880 0x7f6d340b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:00.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:00.204+0000 7f6d3b7fe700 1 -- 172.21.15.100:0/2651614690 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f6d3c0e1e10 con 0x7f6d4c102400 2024-05-10T11:18:00.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:00.204+0000 7f6d49d9b700 1 --2- 172.21.15.100:0/2651614690 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f6d340b2880 0x7f6d340b4d40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6d40009990 tx=0x7f6d40003850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:00.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:00.208+0000 7f6d3b7fe700 1 -- 172.21.15.100:0/2651614690 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f6d3c0a97d0 con 0x7f6d4c102400 2024-05-10T11:18:00.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:00.420+0000 7f6d50f87700 1 -- 172.21.15.100:0/2651614690 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f6d4c105a80 con 0x7f6d4c102400 2024-05-10T11:18:00.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:00.421+0000 7f6d3b7fe700 1 -- 172.21.15.100:0/2651614690 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v89) ==== 56+0+5636 (secure 0 0 0) 0x7f6d3c0a90d0 con 0x7f6d4c102400 2024-05-10T11:18:00.421 INFO:teuthology.orchestra.run.smithi100.stdout:epoch 89 2024-05-10T11:18:00.421 INFO:teuthology.orchestra.run.smithi100.stdout:fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 2024-05-10T11:18:00.422 INFO:teuthology.orchestra.run.smithi100.stdout:created 2024-05-10T11:06:05.454446+0000 2024-05-10T11:18:00.422 INFO:teuthology.orchestra.run.smithi100.stdout:modified 2024-05-10T11:15:01.418144+0000 2024-05-10T11:18:00.422 INFO:teuthology.orchestra.run.smithi100.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-05-10T11:18:00.422 INFO:teuthology.orchestra.run.smithi100.stdout:crush_version 27 2024-05-10T11:18:00.422 INFO:teuthology.orchestra.run.smithi100.stdout:full_ratio 0.95 2024-05-10T11:18:00.422 INFO:teuthology.orchestra.run.smithi100.stdout:backfillfull_ratio 0.9 2024-05-10T11:18:00.422 INFO:teuthology.orchestra.run.smithi100.stdout:nearfull_ratio 0.85 2024-05-10T11:18:00.423 INFO:teuthology.orchestra.run.smithi100.stdout:require_min_compat_client luminous 2024-05-10T11:18:00.423 INFO:teuthology.orchestra.run.smithi100.stdout:min_compat_client luminous 2024-05-10T11:18:00.423 INFO:teuthology.orchestra.run.smithi100.stdout:require_osd_release squid 2024-05-10T11:18:00.423 INFO:teuthology.orchestra.run.smithi100.stdout:stretch_mode_enabled false 2024-05-10T11:18:00.423 INFO:teuthology.orchestra.run.smithi100.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-05-10T11:18:00.423 INFO:teuthology.orchestra.run.smithi100.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 80 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-05-10T11:18:00.423 INFO:teuthology.orchestra.run.smithi100.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 80 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-05-10T11:18:00.423 INFO:teuthology.orchestra.run.smithi100.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 87 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-05-10T11:18:00.423 INFO:teuthology.orchestra.run.smithi100.stdout:max_osd 12 2024-05-10T11:18:00.423 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0 up in weight 1 up_from 8 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.100:6802/643028278,v1:172.21.15.100:6803/643028278] [v2:172.21.15.100:6804/643028278,v1:172.21.15.100:6805/643028278] exists,up 5f031418-da23-4e85-8669-160960dadd09 2024-05-10T11:18:00.424 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1 up in weight 1 up_from 13 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.178:6800/3103987017,v1:172.21.15.178:6801/3103987017] [v2:172.21.15.178:6802/3103987017,v1:172.21.15.178:6803/3103987017] exists,up a908cdc9-001e-4b65-9ab0-0dd08dba20f3 2024-05-10T11:18:00.424 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2 up in weight 1 up_from 18 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.188:6800/3991466468,v1:172.21.15.188:6801/3991466468] [v2:172.21.15.188:6802/3991466468,v1:172.21.15.188:6803/3991466468] exists,up 4d8f89ae-9424-46e9-9c80-17c47cc5e8ef 2024-05-10T11:18:00.424 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3 up in weight 1 up_from 26 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.100:6810/1713357192,v1:172.21.15.100:6811/1713357192] [v2:172.21.15.100:6812/1713357192,v1:172.21.15.100:6813/1713357192] exists,up 4c8865aa-fef4-4710-b766-1385adb9462c 2024-05-10T11:18:00.424 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4 up in weight 1 up_from 32 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.178:6808/2222814192,v1:172.21.15.178:6809/2222814192] [v2:172.21.15.178:6810/2222814192,v1:172.21.15.178:6811/2222814192] exists,up a9653800-da57-43eb-8781-3188e8a3949f 2024-05-10T11:18:00.424 INFO:teuthology.orchestra.run.smithi100.stdout:osd.5 up in weight 1 up_from 37 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.188:6808/3517375935,v1:172.21.15.188:6809/3517375935] [v2:172.21.15.188:6810/3517375935,v1:172.21.15.188:6811/3517375935] exists,up fb5f8af2-1eb0-4d8a-a7a3-ed6dd68e692e 2024-05-10T11:18:00.424 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6 up in weight 1 up_from 43 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.100:6818/1410593862,v1:172.21.15.100:6819/1410593862] [v2:172.21.15.100:6820/1410593862,v1:172.21.15.100:6821/1410593862] exists,up 84b57387-0a94-4e49-b099-493d329b87dd 2024-05-10T11:18:00.424 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7 up in weight 1 up_from 48 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.178:6816/2458690455,v1:172.21.15.178:6817/2458690455] [v2:172.21.15.178:6818/2458690455,v1:172.21.15.178:6819/2458690455] exists,up 02ada28f-ac4f-47cf-983f-8a6da1140abe 2024-05-10T11:18:00.424 INFO:teuthology.orchestra.run.smithi100.stdout:osd.8 up in weight 1 up_from 55 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.188:6816/1523872826,v1:172.21.15.188:6817/1523872826] [v2:172.21.15.188:6818/1523872826,v1:172.21.15.188:6819/1523872826] exists,up e7e489bc-15fc-4a4c-b297-d1bb626513ce 2024-05-10T11:18:00.424 INFO:teuthology.orchestra.run.smithi100.stdout:osd.9 up in weight 1 up_from 61 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.100:6826/3585761978,v1:172.21.15.100:6827/3585761978] [v2:172.21.15.100:6828/3585761978,v1:172.21.15.100:6829/3585761978] exists,up 72328326-cd16-45d9-bfd9-40f6c5c20f5c 2024-05-10T11:18:00.424 INFO:teuthology.orchestra.run.smithi100.stdout:osd.10 up in weight 1 up_from 66 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.178:6824/2985841725,v1:172.21.15.178:6825/2985841725] [v2:172.21.15.178:6826/2985841725,v1:172.21.15.178:6827/2985841725] exists,up a355f9a8-91a3-41d4-814d-5a9c9b953706 2024-05-10T11:18:00.424 INFO:teuthology.orchestra.run.smithi100.stdout:osd.11 up in weight 1 up_from 72 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.188:6824/2574955118,v1:172.21.15.188:6825/2574955118] [v2:172.21.15.188:6826/2574955118,v1:172.21.15.188:6827/2574955118] exists,up 76a5e13c-7e38-42f9-bfa7-7fedec7c7184 2024-05-10T11:18:00.424 INFO:teuthology.orchestra.run.smithi100.stdout:pg_upmap_items 2.5 [8,11] 2024-05-10T11:18:00.425 INFO:teuthology.orchestra.run.smithi100.stdout:pg_upmap_items 2.8 [2,11] 2024-05-10T11:18:00.425 INFO:teuthology.orchestra.run.smithi100.stdout:pg_upmap_items 3.c [3,9] 2024-05-10T11:18:00.425 INFO:teuthology.orchestra.run.smithi100.stdout:pg_upmap_items 3.e [5,9] 2024-05-10T11:18:00.425 INFO:teuthology.orchestra.run.smithi100.stdout:pg_upmap_items 3.23 [4,8] 2024-05-10T11:18:00.425 INFO:teuthology.orchestra.run.smithi100.stdout:pg_upmap_items 3.2c [4,9] 2024-05-10T11:18:00.425 INFO:teuthology.orchestra.run.smithi100.stdout:pg_upmap_items 4.9 [5,8] 2024-05-10T11:18:00.425 INFO:teuthology.orchestra.run.smithi100.stdout:pg_upmap_items 4.14 [9,8] 2024-05-10T11:18:00.425 INFO:teuthology.orchestra.run.smithi100.stdout:pg_upmap_items 4.19 [9,10] 2024-05-10T11:18:00.425 INFO:teuthology.orchestra.run.smithi100.stdout:pg_upmap_items 4.1f [9,8] 2024-05-10T11:18:00.425 INFO:teuthology.orchestra.run.smithi100.stdout:blocklist 172.21.15.100:6800/1597433518 expires 2024-05-11T11:07:00.092732+0000 2024-05-10T11:18:00.425 INFO:teuthology.orchestra.run.smithi100.stdout:blocklist 172.21.15.100:6800/1086450256 expires 2024-05-11T11:06:33.114176+0000 2024-05-10T11:18:00.425 INFO:teuthology.orchestra.run.smithi100.stdout:blocklist 172.21.15.100:0/767964410 expires 2024-05-11T11:07:00.092732+0000 2024-05-10T11:18:00.426 INFO:teuthology.orchestra.run.smithi100.stdout:blocklist 172.21.15.100:6801/1597433518 expires 2024-05-11T11:07:00.092732+0000 2024-05-10T11:18:00.426 INFO:teuthology.orchestra.run.smithi100.stdout:blocklist 172.21.15.100:6801/1086450256 expires 2024-05-11T11:06:33.114176+0000 2024-05-10T11:18:00.426 INFO:teuthology.orchestra.run.smithi100.stdout:blocklist 172.21.15.100:0/2770572070 expires 2024-05-11T11:06:33.114176+0000 2024-05-10T11:18:00.426 INFO:teuthology.orchestra.run.smithi100.stdout:blocklist 172.21.15.100:0/1379340513 expires 2024-05-11T11:06:33.114176+0000 2024-05-10T11:18:00.426 INFO:teuthology.orchestra.run.smithi100.stdout:blocklist 172.21.15.100:0/3025646953 expires 2024-05-11T11:07:00.092732+0000 2024-05-10T11:18:00.426 INFO:teuthology.orchestra.run.smithi100.stdout:blocklist 172.21.15.100:0/4099258930 expires 2024-05-11T11:07:00.092732+0000 2024-05-10T11:18:00.426 INFO:teuthology.orchestra.run.smithi100.stdout:blocklist 172.21.15.100:0/4267693567 expires 2024-05-11T11:06:33.114176+0000 2024-05-10T11:18:00.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:00.423+0000 7f6d50f87700 1 -- 172.21.15.100:0/2651614690 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f6d340b2880 msgr2=0x7f6d340b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:00.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:00.423+0000 7f6d50f87700 1 --2- 172.21.15.100:0/2651614690 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f6d340b2880 0x7f6d340b4d40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6d40009990 tx=0x7f6d40003850 comp rx=0 tx=0).stop 2024-05-10T11:18:00.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:00.423+0000 7f6d50f87700 1 -- 172.21.15.100:0/2651614690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d4c102400 msgr2=0x7f6d4c1995c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:00.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:00.423+0000 7f6d50f87700 1 --2- 172.21.15.100:0/2651614690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d4c102400 0x7f6d4c1995c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c00d9b0 tx=0x7f6d3c00dcc0 comp rx=0 tx=0).stop 2024-05-10T11:18:00.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:00.423+0000 7f6d50f87700 1 -- 172.21.15.100:0/2651614690 shutdown_connections 2024-05-10T11:18:00.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:00.423+0000 7f6d50f87700 1 --2- 172.21.15.100:0/2651614690 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f6d340b2880 0x7f6d340b4d40 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:00.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:00.423+0000 7f6d50f87700 1 --2- 172.21.15.100:0/2651614690 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d4c103fa0 0x7f6d4c1a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:00.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:00.423+0000 7f6d50f87700 1 --2- 172.21.15.100:0/2651614690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d4c102400 0x7f6d4c1995c0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:00.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:00.423+0000 7f6d50f87700 1 --2- 172.21.15.100:0/2651614690 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d4c103600 0x7f6d4c199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:00.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:00.423+0000 7f6d50f87700 1 -- 172.21.15.100:0/2651614690 >> 172.21.15.100:0/2651614690 conn(0x7f6d4c0fd9a0 msgr2=0x7f6d4c06b230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:00.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:00.424+0000 7f6d50f87700 1 -- 172.21.15.100:0/2651614690 shutdown_connections 2024-05-10T11:18:00.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:00.424+0000 7f6d50f87700 1 -- 172.21.15.100:0/2651614690 wait complete. 2024-05-10T11:18:01.153 INFO:teuthology.run_tasks:Running task fs.ready... 2024-05-10T11:18:01.165 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T11:18:01.419 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:18:01.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:18:01 smithi100 ceph-mon[30296]: pgmap v409: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T11:18:01.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:18:01 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/2651614690' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-10T11:18:01.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:18:01 smithi178 ceph-mon[33359]: pgmap v409: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T11:18:01.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:18:01 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/2651614690' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-10T11:18:01.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:18:01 smithi188 ceph-mon[34075]: pgmap v409: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T11:18:01.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:18:01 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/2651614690' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-10T11:18:03.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:03.198+0000 7f25402ae700 1 -- 172.21.15.100:0/3045346056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2538109540 msgr2=0x7f2538109920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:03.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:03.198+0000 7f25402ae700 1 --2- 172.21.15.100:0/3045346056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2538109540 0x7f2538109920 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2534009990 tx=0x7f2534009ca0 comp rx=0 tx=0).stop 2024-05-10T11:18:03.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:03.198+0000 7f25402ae700 1 -- 172.21.15.100:0/3045346056 shutdown_connections 2024-05-10T11:18:03.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:03.198+0000 7f25402ae700 1 --2- 172.21.15.100:0/3045346056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f253806c5d0 0x7f2538110380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:03.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:03.198+0000 7f25402ae700 1 --2- 172.21.15.100:0/3045346056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2538109540 0x7f2538109920 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:03.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:03.198+0000 7f25402ae700 1 --2- 172.21.15.100:0/3045346056 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f253806bc10 0x7f253806c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:03.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:03.198+0000 7f25402ae700 1 -- 172.21.15.100:0/3045346056 >> 172.21.15.100:0/3045346056 conn(0x7f2538078410 msgr2=0x7f2538078820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:03.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:03.198+0000 7f25402ae700 1 -- 172.21.15.100:0/3045346056 shutdown_connections 2024-05-10T11:18:03.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:03.198+0000 7f25402ae700 1 -- 172.21.15.100:0/3045346056 wait complete. 2024-05-10T11:18:03.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:03.199+0000 7f25402ae700 1 Processor -- start 2024-05-10T11:18:03.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:03.199+0000 7f25402ae700 1 -- start start 2024-05-10T11:18:03.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:03.199+0000 7f25402ae700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f253806bc10 0x7f25381024a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:03.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:03.199+0000 7f25402ae700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f253806c5d0 0x7f25381029e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:03.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:03.199+0000 7f25402ae700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2538109540 0x7f2538102f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:03.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:03.199+0000 7f25402ae700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f25381135a0 con 0x7f2538109540 2024-05-10T11:18:03.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:03.199+0000 7f25402ae700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2538113420 con 0x7f253806c5d0 2024-05-10T11:18:03.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:03.199+0000 7f25402ae700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2538113720 con 0x7f253806bc10 2024-05-10T11:18:03.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:03.199+0000 7f253e04a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f253806bc10 0x7f25381024a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:03.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:03.199+0000 7f253e84b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2538109540 0x7f2538102f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:03.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:03.199+0000 7f253e04a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f253806bc10 0x7f25381024a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:50444/0 (socket says 172.21.15.100:50444) 2024-05-10T11:18:03.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:03.199+0000 7f253d849700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f253806c5d0 0x7f25381029e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:03.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:03.199+0000 7f253e04a700 1 -- 172.21.15.100:0/3328634981 learned_addr learned my addr 172.21.15.100:0/3328634981 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:18:03.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:03.199+0000 7f253e84b700 1 -- 172.21.15.100:0/3328634981 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f253806bc10 msgr2=0x7f25381024a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:03.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:03.199+0000 7f253e84b700 1 --2- 172.21.15.100:0/3328634981 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f253806bc10 0x7f25381024a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:03.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:03.199+0000 7f253e84b700 1 -- 172.21.15.100:0/3328634981 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f253806c5d0 msgr2=0x7f25381029e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:03.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:03.199+0000 7f253e84b700 1 --2- 172.21.15.100:0/3328634981 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f253806c5d0 0x7f25381029e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:03.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:03.199+0000 7f253e84b700 1 -- 172.21.15.100:0/3328634981 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25381a8000 con 0x7f2538109540 2024-05-10T11:18:03.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:03.199+0000 7f253e04a700 1 --2- 172.21.15.100:0/3328634981 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f253806bc10 0x7f25381024a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:18:03.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:03.199+0000 7f253d849700 1 --2- 172.21.15.100:0/3328634981 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f253806c5d0 0x7f25381029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:18:03.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:03.200+0000 7f253e84b700 1 --2- 172.21.15.100:0/3328634981 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2538109540 0x7f2538102f20 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f253000be20 tx=0x7f2530009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:03.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:03.200+0000 7f252f7fe700 1 -- 172.21.15.100:0/3328634981 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2530019070 con 0x7f2538109540 2024-05-10T11:18:03.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:03.200+0000 7f25402ae700 1 -- 172.21.15.100:0/3328634981 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25381a82f0 con 0x7f2538109540 2024-05-10T11:18:03.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:03.200+0000 7f252f7fe700 1 -- 172.21.15.100:0/3328634981 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f253000cb90 con 0x7f2538109540 2024-05-10T11:18:03.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:03.200+0000 7f252f7fe700 1 -- 172.21.15.100:0/3328634981 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25300125a0 con 0x7f2538109540 2024-05-10T11:18:03.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:03.200+0000 7f25402ae700 1 -- 172.21.15.100:0/3328634981 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25381a87e0 con 0x7f2538109540 2024-05-10T11:18:03.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:03.201+0000 7f25402ae700 1 -- 172.21.15.100:0/3328634981 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f253810ba40 con 0x7f2538109540 2024-05-10T11:18:03.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:03.202+0000 7f252f7fe700 1 -- 172.21.15.100:0/3328634981 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f253001c070 con 0x7f2538109540 2024-05-10T11:18:03.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:03.203+0000 7f252f7fe700 1 --2- 172.21.15.100:0/3328634981 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f25240b2880 0x7f25240b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:03.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:03.203+0000 7f253e04a700 1 --2- 172.21.15.100:0/3328634981 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f25240b2880 0x7f25240b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:03.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:03.203+0000 7f252f7fe700 1 -- 172.21.15.100:0/3328634981 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f25300e24b0 con 0x7f2538109540 2024-05-10T11:18:03.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:03.203+0000 7f253e04a700 1 --2- 172.21.15.100:0/3328634981 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f25240b2880 0x7f25240b4d40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2534006010 tx=0x7f253400b560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:03.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:03.206+0000 7f252f7fe700 1 -- 172.21.15.100:0/3328634981 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f2530023070 con 0x7f2538109540 2024-05-10T11:18:03.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:03.461+0000 7f25402ae700 1 -- 172.21.15.100:0/3328634981 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2538104c00 con 0x7f2538109540 2024-05-10T11:18:03.461 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:18:03 smithi100 ceph-mon[30296]: pgmap v410: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T11:18:03.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:03.462+0000 7f252f7fe700 1 -- 172.21.15.100:0/3328634981 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11894 (secure 0 0 0) 0x7f25300a9e70 con 0x7f2538109540 2024-05-10T11:18:03.462 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:18:03.462 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2072225012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2072225012},{"type":"v1","addr":"172.21.15.100:6835","nonce":2072225012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/401939540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":401939540},{"type":"v1","addr":"172.21.15.100:6837","nonce":401939540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14571,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/3946311025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3946311025},{"type":"v1","addr":"172.21.15.100:6839","nonce":3946311025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3736371660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3736371660},{"type":"v1","addr":"172.21.15.100:6841","nonce":3736371660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/2741810791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2741810791},{"type":"v1","addr":"172.21.15.178:6833","nonce":2741810791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1704891041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1704891041},{"type":"v1","addr":"172.21.15.188:6833","nonce":1704891041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T11:14:27.242452+0000","modified":"2024-05-10T11:15:35.672461+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24439,"mds_1":24437,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/2124076312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2124076312},{"type":"v1","addr":"172.21.15.178:6835","nonce":2124076312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/3224358425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3224358425},{"type":"v1","addr":"172.21.15.178:6837","nonce":3224358425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/515547690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":515547690},{"type":"v1","addr":"172.21.15.188:6835","nonce":515547690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/1943342620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1943342620},{"type":"v1","addr":"172.21.15.188:6837","nonce":1943342620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.178:6839/3760985404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3760985404},{"type":"v1","addr":"172.21.15.178:6839","nonce":3760985404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.188:6839/3194917070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":3194917070},{"type":"v1","addr":"172.21.15.188:6839","nonce":3194917070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24439,"qdb_cluster":[24437,24439,24427]},"id":1}]} 2024-05-10T11:18:03.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:03.464+0000 7f25402ae700 1 -- 172.21.15.100:0/3328634981 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f25240b2880 msgr2=0x7f25240b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:03.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:03.464+0000 7f25402ae700 1 --2- 172.21.15.100:0/3328634981 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f25240b2880 0x7f25240b4d40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2534006010 tx=0x7f253400b560 comp rx=0 tx=0).stop 2024-05-10T11:18:03.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:03.464+0000 7f25402ae700 1 -- 172.21.15.100:0/3328634981 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2538109540 msgr2=0x7f2538102f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:03.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:03.464+0000 7f25402ae700 1 --2- 172.21.15.100:0/3328634981 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2538109540 0x7f2538102f20 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f253000be20 tx=0x7f2530009ed0 comp rx=0 tx=0).stop 2024-05-10T11:18:03.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:03.464+0000 7f25402ae700 1 -- 172.21.15.100:0/3328634981 shutdown_connections 2024-05-10T11:18:03.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:03.464+0000 7f25402ae700 1 --2- 172.21.15.100:0/3328634981 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f25240b2880 0x7f25240b4d40 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:03.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:03.464+0000 7f25402ae700 1 --2- 172.21.15.100:0/3328634981 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2538109540 0x7f2538102f20 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:03.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:03.464+0000 7f25402ae700 1 --2- 172.21.15.100:0/3328634981 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f253806bc10 0x7f25381024a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:03.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:03.464+0000 7f25402ae700 1 --2- 172.21.15.100:0/3328634981 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f253806c5d0 0x7f25381029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:03.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:03.464+0000 7f25402ae700 1 -- 172.21.15.100:0/3328634981 >> 172.21.15.100:0/3328634981 conn(0x7f2538078410 msgr2=0x7f2538101260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:03.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:03.465+0000 7f25402ae700 1 -- 172.21.15.100:0/3328634981 shutdown_connections 2024-05-10T11:18:03.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:03.465+0000 7f25402ae700 1 -- 172.21.15.100:0/3328634981 wait complete. 2024-05-10T11:18:03.466 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 26 2024-05-10T11:18:03.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:18:03 smithi178 ceph-mon[33359]: pgmap v410: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T11:18:03.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:18:03 smithi188 ceph-mon[34075]: pgmap v410: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T11:18:04.171 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T11:18:04.413 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:18:04.535 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:18:04 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/3328634981' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:18:04.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:18:04 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/3328634981' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:18:04.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:18:04 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/3328634981' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:18:05.736 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:18:05 smithi100 ceph-mon[30296]: pgmap v411: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-10T11:18:05.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:18:05 smithi178 ceph-mon[33359]: pgmap v411: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-10T11:18:05.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:18:05 smithi188 ceph-mon[34075]: pgmap v411: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-10T11:18:06.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:06.115+0000 7fb3130ec700 1 -- 172.21.15.100:0/3298327488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb30c101a90 msgr2=0x7fb30c101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:06.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:06.115+0000 7fb3130ec700 1 --2- 172.21.15.100:0/3298327488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb30c101a90 0x7fb30c101f10 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb2f8009990 tx=0x7fb2f8009ca0 comp rx=0 tx=0).stop 2024-05-10T11:18:06.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:06.115+0000 7fb3130ec700 1 -- 172.21.15.100:0/3298327488 shutdown_connections 2024-05-10T11:18:06.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:06.115+0000 7fb3130ec700 1 --2- 172.21.15.100:0/3298327488 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb30c102450 0x7fb30c110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:06.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:06.115+0000 7fb3130ec700 1 --2- 172.21.15.100:0/3298327488 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb30c109520 0x7fb30c109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:06.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:06.115+0000 7fb3130ec700 1 --2- 172.21.15.100:0/3298327488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb30c101a90 0x7fb30c101f10 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:06.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:06.115+0000 7fb3130ec700 1 -- 172.21.15.100:0/3298327488 >> 172.21.15.100:0/3298327488 conn(0x7fb30c078a10 msgr2=0x7fb30c0ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:06.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:06.115+0000 7fb3130ec700 1 -- 172.21.15.100:0/3298327488 shutdown_connections 2024-05-10T11:18:06.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:06.115+0000 7fb3130ec700 1 -- 172.21.15.100:0/3298327488 wait complete. 2024-05-10T11:18:06.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:06.116+0000 7fb3130ec700 1 Processor -- start 2024-05-10T11:18:06.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:06.116+0000 7fb3130ec700 1 -- start start 2024-05-10T11:18:06.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:06.116+0000 7fb3130ec700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb30c101a90 0x7fb30c19f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:06.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:06.116+0000 7fb3130ec700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb30c102450 0x7fb30c19fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:06.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:06.116+0000 7fb3130ec700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb30c109520 0x7fb30c199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:06.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:06.116+0000 7fb3130ec700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb30c1134e0 con 0x7fb30c109520 2024-05-10T11:18:06.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:06.116+0000 7fb3130ec700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb30c113360 con 0x7fb30c102450 2024-05-10T11:18:06.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:06.116+0000 7fb3130ec700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb30c113660 con 0x7fb30c101a90 2024-05-10T11:18:06.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:06.117+0000 7fb311689700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb30c109520 0x7fb30c199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:06.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:06.117+0000 7fb311689700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb30c109520 0x7fb30c199960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37202/0 (socket says 172.21.15.100:37202) 2024-05-10T11:18:06.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:06.117+0000 7fb311689700 1 -- 172.21.15.100:0/2689837738 learned_addr learned my addr 172.21.15.100:0/2689837738 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:18:06.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:06.117+0000 7fb310e88700 1 --2- 172.21.15.100:0/2689837738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb30c101a90 0x7fb30c19f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:06.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:06.117+0000 7fb30bfff700 1 --2- 172.21.15.100:0/2689837738 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb30c102450 0x7fb30c19fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:06.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:06.117+0000 7fb311689700 1 -- 172.21.15.100:0/2689837738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb30c101a90 msgr2=0x7fb30c19f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:06.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:06.117+0000 7fb311689700 1 --2- 172.21.15.100:0/2689837738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb30c101a90 0x7fb30c19f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:06.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:06.117+0000 7fb311689700 1 -- 172.21.15.100:0/2689837738 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb30c102450 msgr2=0x7fb30c19fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:06.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:06.117+0000 7fb311689700 1 --2- 172.21.15.100:0/2689837738 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb30c102450 0x7fb30c19fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:06.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:06.117+0000 7fb311689700 1 -- 172.21.15.100:0/2689837738 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb30c19a0c0 con 0x7fb30c109520 2024-05-10T11:18:06.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:06.117+0000 7fb311689700 1 --2- 172.21.15.100:0/2689837738 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb30c109520 0x7fb30c199960 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb304007ee0 tx=0x7fb30400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:06.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:06.117+0000 7fb309ffb700 1 -- 172.21.15.100:0/2689837738 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb304004bb0 con 0x7fb30c109520 2024-05-10T11:18:06.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:06.117+0000 7fb3130ec700 1 -- 172.21.15.100:0/2689837738 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb30c19a3b0 con 0x7fb30c109520 2024-05-10T11:18:06.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:06.117+0000 7fb309ffb700 1 -- 172.21.15.100:0/2689837738 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb304004d50 con 0x7fb30c109520 2024-05-10T11:18:06.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:06.117+0000 7fb309ffb700 1 -- 172.21.15.100:0/2689837738 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3040058b0 con 0x7fb30c109520 2024-05-10T11:18:06.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:06.117+0000 7fb3130ec700 1 -- 172.21.15.100:0/2689837738 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb30c1a6620 con 0x7fb30c109520 2024-05-10T11:18:06.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:06.118+0000 7fb3130ec700 1 -- 172.21.15.100:0/2689837738 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb30c06a190 con 0x7fb30c109520 2024-05-10T11:18:06.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:06.118+0000 7fb309ffb700 1 -- 172.21.15.100:0/2689837738 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fb30401a020 con 0x7fb30c109520 2024-05-10T11:18:06.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:06.119+0000 7fb309ffb700 1 --2- 172.21.15.100:0/2689837738 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fb2fc0b2880 0x7fb2fc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:06.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:06.119+0000 7fb310e88700 1 --2- 172.21.15.100:0/2689837738 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fb2fc0b2880 0x7fb2fc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:06.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:06.119+0000 7fb309ffb700 1 -- 172.21.15.100:0/2689837738 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fb3040e1c00 con 0x7fb30c109520 2024-05-10T11:18:06.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:06.120+0000 7fb310e88700 1 --2- 172.21.15.100:0/2689837738 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fb2fc0b2880 0x7fb2fc0b4d40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb30c0ff5f0 tx=0x7fb30000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:06.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:06.124+0000 7fb309ffb700 1 -- 172.21.15.100:0/2689837738 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fb3040e8080 con 0x7fb30c109520 2024-05-10T11:18:06.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:06.378+0000 7fb3130ec700 1 -- 172.21.15.100:0/2689837738 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb30c02e510 con 0x7fb30c109520 2024-05-10T11:18:06.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:06.378+0000 7fb309ffb700 1 -- 172.21.15.100:0/2689837738 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11894 (secure 0 0 0) 0x7fb304010070 con 0x7fb30c109520 2024-05-10T11:18:06.379 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:18:06.380 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2072225012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2072225012},{"type":"v1","addr":"172.21.15.100:6835","nonce":2072225012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/401939540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":401939540},{"type":"v1","addr":"172.21.15.100:6837","nonce":401939540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14571,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/3946311025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3946311025},{"type":"v1","addr":"172.21.15.100:6839","nonce":3946311025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3736371660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3736371660},{"type":"v1","addr":"172.21.15.100:6841","nonce":3736371660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/2741810791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2741810791},{"type":"v1","addr":"172.21.15.178:6833","nonce":2741810791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1704891041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1704891041},{"type":"v1","addr":"172.21.15.188:6833","nonce":1704891041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T11:14:27.242452+0000","modified":"2024-05-10T11:15:35.672461+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24439,"mds_1":24437,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/2124076312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2124076312},{"type":"v1","addr":"172.21.15.178:6835","nonce":2124076312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/3224358425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3224358425},{"type":"v1","addr":"172.21.15.178:6837","nonce":3224358425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/515547690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":515547690},{"type":"v1","addr":"172.21.15.188:6835","nonce":515547690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/1943342620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1943342620},{"type":"v1","addr":"172.21.15.188:6837","nonce":1943342620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.178:6839/3760985404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3760985404},{"type":"v1","addr":"172.21.15.178:6839","nonce":3760985404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.188:6839/3194917070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":3194917070},{"type":"v1","addr":"172.21.15.188:6839","nonce":3194917070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24439,"qdb_cluster":[24437,24439,24427]},"id":1}]} 2024-05-10T11:18:06.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:06.381+0000 7fb3130ec700 1 -- 172.21.15.100:0/2689837738 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fb2fc0b2880 msgr2=0x7fb2fc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:06.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:06.381+0000 7fb3130ec700 1 --2- 172.21.15.100:0/2689837738 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fb2fc0b2880 0x7fb2fc0b4d40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb30c0ff5f0 tx=0x7fb30000b410 comp rx=0 tx=0).stop 2024-05-10T11:18:06.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:06.381+0000 7fb3130ec700 1 -- 172.21.15.100:0/2689837738 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb30c109520 msgr2=0x7fb30c199960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:06.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:06.381+0000 7fb3130ec700 1 --2- 172.21.15.100:0/2689837738 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb30c109520 0x7fb30c199960 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb304007ee0 tx=0x7fb30400d3b0 comp rx=0 tx=0).stop 2024-05-10T11:18:06.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:06.381+0000 7fb3130ec700 1 -- 172.21.15.100:0/2689837738 shutdown_connections 2024-05-10T11:18:06.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:06.381+0000 7fb3130ec700 1 --2- 172.21.15.100:0/2689837738 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fb2fc0b2880 0x7fb2fc0b4d40 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:06.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:06.381+0000 7fb3130ec700 1 --2- 172.21.15.100:0/2689837738 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb30c109520 0x7fb30c199960 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:06.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:06.381+0000 7fb3130ec700 1 --2- 172.21.15.100:0/2689837738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb30c101a90 0x7fb30c19f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:06.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:06.381+0000 7fb3130ec700 1 --2- 172.21.15.100:0/2689837738 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb30c102450 0x7fb30c19fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:06.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:06.381+0000 7fb3130ec700 1 -- 172.21.15.100:0/2689837738 >> 172.21.15.100:0/2689837738 conn(0x7fb30c078a10 msgr2=0x7fb30c1003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:06.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:06.381+0000 7fb3130ec700 1 -- 172.21.15.100:0/2689837738 shutdown_connections 2024-05-10T11:18:06.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:06.381+0000 7fb3130ec700 1 -- 172.21.15.100:0/2689837738 wait complete. 2024-05-10T11:18:06.383 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 26 2024-05-10T11:18:07.096 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-10T11:18:07.341 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:18:07.463 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:18:07 smithi100 ceph-mon[30296]: pgmap v412: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T11:18:07.463 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:18:07 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/2689837738' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:18:07.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:18:07 smithi178 ceph-mon[33359]: pgmap v412: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T11:18:07.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:18:07 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/2689837738' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:18:07.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:18:07 smithi188 ceph-mon[34075]: pgmap v412: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T11:18:07.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:18:07 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/2689837738' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:18:09.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:09.065+0000 7f6155632700 1 -- 172.21.15.100:0/1420151567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6150102d50 msgr2=0x7f615010f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:09.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:09.065+0000 7f6155632700 1 --2- 172.21.15.100:0/1420151567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6150102d50 0x7f615010f240 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f614800b0a0 tx=0x7f614800b3b0 comp rx=0 tx=0).stop 2024-05-10T11:18:09.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:09.065+0000 7f6155632700 1 -- 172.21.15.100:0/1420151567 shutdown_connections 2024-05-10T11:18:09.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:09.065+0000 7f6155632700 1 --2- 172.21.15.100:0/1420151567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6150102d50 0x7f615010f240 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:09.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:09.065+0000 7f6155632700 1 --2- 172.21.15.100:0/1420151567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6150108400 0x7f61501087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:09.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:09.065+0000 7f6155632700 1 --2- 172.21.15.100:0/1420151567 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6150102390 0x7f6150102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:09.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:09.065+0000 7f6155632700 1 -- 172.21.15.100:0/1420151567 >> 172.21.15.100:0/1420151567 conn(0x7f6150079310 msgr2=0x7f6150079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:09.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:09.065+0000 7f6155632700 1 -- 172.21.15.100:0/1420151567 shutdown_connections 2024-05-10T11:18:09.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:09.065+0000 7f6155632700 1 -- 172.21.15.100:0/1420151567 wait complete. 2024-05-10T11:18:09.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:09.066+0000 7f6155632700 1 Processor -- start 2024-05-10T11:18:09.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:09.066+0000 7f6155632700 1 -- start start 2024-05-10T11:18:09.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:09.066+0000 7f6155632700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6150102390 0x7f6150104070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:09.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:09.066+0000 7f6155632700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6150102d50 0x7f61501045b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:09.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:09.066+0000 7f6155632700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6150108400 0x7f6150104af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:09.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:09.066+0000 7f6155632700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6150112450 con 0x7f6150108400 2024-05-10T11:18:09.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:09.066+0000 7f6155632700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f61501122d0 con 0x7f6150102390 2024-05-10T11:18:09.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:09.066+0000 7f6155632700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f61501125d0 con 0x7f6150102d50 2024-05-10T11:18:09.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:09.066+0000 7f614f7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6150108400 0x7f6150104af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:09.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:09.066+0000 7f614f7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6150108400 0x7f6150104af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59418/0 (socket says 172.21.15.100:59418) 2024-05-10T11:18:09.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:09.066+0000 7f614f7fe700 1 -- 172.21.15.100:0/1225761570 learned_addr learned my addr 172.21.15.100:0/1225761570 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:18:09.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:09.066+0000 7f614effd700 1 --2- 172.21.15.100:0/1225761570 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6150102390 0x7f6150104070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:09.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:09.067+0000 7f6147fff700 1 --2- 172.21.15.100:0/1225761570 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6150102d50 0x7f61501045b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:09.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:09.067+0000 7f614f7fe700 1 -- 172.21.15.100:0/1225761570 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6150102d50 msgr2=0x7f61501045b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:09.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:09.067+0000 7f614f7fe700 1 --2- 172.21.15.100:0/1225761570 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6150102d50 0x7f61501045b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:09.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:09.067+0000 7f614f7fe700 1 -- 172.21.15.100:0/1225761570 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6150102390 msgr2=0x7f6150104070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:09.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:09.067+0000 7f614f7fe700 1 --2- 172.21.15.100:0/1225761570 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6150102390 0x7f6150104070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:09.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:09.067+0000 7f614f7fe700 1 -- 172.21.15.100:0/1225761570 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61501a7fa0 con 0x7f6150108400 2024-05-10T11:18:09.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:09.067+0000 7f614f7fe700 1 --2- 172.21.15.100:0/1225761570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6150108400 0x7f6150104af0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f61480062a0 tx=0x7f6148004ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:09.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:09.067+0000 7f614cff9700 1 -- 172.21.15.100:0/1225761570 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6148021070 con 0x7f6150108400 2024-05-10T11:18:09.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:09.067+0000 7f614cff9700 1 -- 172.21.15.100:0/1225761570 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6148007d00 con 0x7f6150108400 2024-05-10T11:18:09.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:09.067+0000 7f6155632700 1 -- 172.21.15.100:0/1225761570 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61501a8230 con 0x7f6150108400 2024-05-10T11:18:09.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:09.067+0000 7f614cff9700 1 -- 172.21.15.100:0/1225761570 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6148005280 con 0x7f6150108400 2024-05-10T11:18:09.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:09.067+0000 7f6155632700 1 -- 172.21.15.100:0/1225761570 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61501a8630 con 0x7f6150108400 2024-05-10T11:18:09.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:09.068+0000 7f6155632700 1 -- 172.21.15.100:0/1225761570 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6150103ab0 con 0x7f6150108400 2024-05-10T11:18:09.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:09.069+0000 7f614cff9700 1 -- 172.21.15.100:0/1225761570 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f6148024070 con 0x7f6150108400 2024-05-10T11:18:09.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:09.070+0000 7f614cff9700 1 --2- 172.21.15.100:0/1225761570 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f613c0b2880 0x7f613c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:09.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:09.070+0000 7f614effd700 1 --2- 172.21.15.100:0/1225761570 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f613c0b2880 0x7f613c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:09.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:09.070+0000 7f614cff9700 1 -- 172.21.15.100:0/1225761570 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f61480e3bd0 con 0x7f6150108400 2024-05-10T11:18:09.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:09.070+0000 7f614effd700 1 --2- 172.21.15.100:0/1225761570 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f613c0b2880 0x7f613c0b4d40 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6140006fd0 tx=0x7f6140008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:09.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:09.073+0000 7f614cff9700 1 -- 172.21.15.100:0/1225761570 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f61480ab620 con 0x7f6150108400 2024-05-10T11:18:09.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:09.284+0000 7f6155632700 1 -- 172.21.15.100:0/1225761570 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f61501a8890 con 0x7f6150108400 2024-05-10T11:18:09.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:09.285+0000 7f614cff9700 1 -- 172.21.15.100:0/1225761570 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24737 (secure 0 0 0) 0x7f61480aaf20 con 0x7f6150108400 2024-05-10T11:18:09.286 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:18:09.287 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":89,"fsid":"20a542cc-0ebd-11ef-bc98-c7b262605968","created":"2024-05-10T11:06:05.454446+0000","modified":"2024-05-10T11:15:01.418144+0000","last_up_change":"2024-05-10T11:13:18.185471+0000","last_in_change":"2024-05-10T11:12:57.268678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T11:09:26.176441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T11:14:21.004285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T11:14:24.179414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T11:14:33.308517+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5f031418-da23-4e85-8669-160960dadd09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":643028278},{"type":"v1","addr":"172.21.15.100:6803","nonce":643028278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":643028278},{"type":"v1","addr":"172.21.15.100:6805","nonce":643028278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":643028278},{"type":"v1","addr":"172.21.15.100:6809","nonce":643028278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":643028278},{"type":"v1","addr":"172.21.15.100:6807","nonce":643028278}]},"public_addr":"172.21.15.100:6803/643028278","cluster_addr":"172.21.15.100:6805/643028278","heartbeat_back_addr":"172.21.15.100:6809/643028278","heartbeat_front_addr":"172.21.15.100:6807/643028278","state":["exists","up"]},{"osd":1,"uuid":"a908cdc9-001e-4b65-9ab0-0dd08dba20f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":3103987017},{"type":"v1","addr":"172.21.15.178:6801","nonce":3103987017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":3103987017},{"type":"v1","addr":"172.21.15.178:6803","nonce":3103987017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":3103987017},{"type":"v1","addr":"172.21.15.178:6807","nonce":3103987017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":3103987017},{"type":"v1","addr":"172.21.15.178:6805","nonce":3103987017}]},"public_addr":"172.21.15.178:6801/3103987017","cluster_addr":"172.21.15.178:6803/3103987017","heartbeat_back_addr":"172.21.15.178:6807/3103987017","heartbeat_front_addr":"172.21.15.178:6805/3103987017","state":["exists","up"]},{"osd":2,"uuid":"4d8f89ae-9424-46e9-9c80-17c47cc5e8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3991466468},{"type":"v1","addr":"172.21.15.188:6801","nonce":3991466468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3991466468},{"type":"v1","addr":"172.21.15.188:6803","nonce":3991466468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3991466468},{"type":"v1","addr":"172.21.15.188:6807","nonce":3991466468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3991466468},{"type":"v1","addr":"172.21.15.188:6805","nonce":3991466468}]},"public_addr":"172.21.15.188:6801/3991466468","cluster_addr":"172.21.15.188:6803/3991466468","heartbeat_back_addr":"172.21.15.188:6807/3991466468","heartbeat_front_addr":"172.21.15.188:6805/3991466468","state":["exists","up"]},{"osd":3,"uuid":"4c8865aa-fef4-4710-b766-1385adb9462c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1713357192},{"type":"v1","addr":"172.21.15.100:6811","nonce":1713357192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1713357192},{"type":"v1","addr":"172.21.15.100:6813","nonce":1713357192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1713357192},{"type":"v1","addr":"172.21.15.100:6817","nonce":1713357192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1713357192},{"type":"v1","addr":"172.21.15.100:6815","nonce":1713357192}]},"public_addr":"172.21.15.100:6811/1713357192","cluster_addr":"172.21.15.100:6813/1713357192","heartbeat_back_addr":"172.21.15.100:6817/1713357192","heartbeat_front_addr":"172.21.15.100:6815/1713357192","state":["exists","up"]},{"osd":4,"uuid":"a9653800-da57-43eb-8781-3188e8a3949f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":2222814192},{"type":"v1","addr":"172.21.15.178:6809","nonce":2222814192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":2222814192},{"type":"v1","addr":"172.21.15.178:6811","nonce":2222814192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":2222814192},{"type":"v1","addr":"172.21.15.178:6815","nonce":2222814192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":2222814192},{"type":"v1","addr":"172.21.15.178:6813","nonce":2222814192}]},"public_addr":"172.21.15.178:6809/2222814192","cluster_addr":"172.21.15.178:6811/2222814192","heartbeat_back_addr":"172.21.15.178:6815/2222814192","heartbeat_front_addr":"172.21.15.178:6813/2222814192","state":["exists","up"]},{"osd":5,"uuid":"fb5f8af2-1eb0-4d8a-a7a3-ed6dd68e692e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3517375935},{"type":"v1","addr":"172.21.15.188:6809","nonce":3517375935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3517375935},{"type":"v1","addr":"172.21.15.188:6811","nonce":3517375935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3517375935},{"type":"v1","addr":"172.21.15.188:6815","nonce":3517375935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3517375935},{"type":"v1","addr":"172.21.15.188:6813","nonce":3517375935}]},"public_addr":"172.21.15.188:6809/3517375935","cluster_addr":"172.21.15.188:6811/3517375935","heartbeat_back_addr":"172.21.15.188:6815/3517375935","heartbeat_front_addr":"172.21.15.188:6813/3517375935","state":["exists","up"]},{"osd":6,"uuid":"84b57387-0a94-4e49-b099-493d329b87dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1410593862},{"type":"v1","addr":"172.21.15.100:6819","nonce":1410593862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1410593862},{"type":"v1","addr":"172.21.15.100:6821","nonce":1410593862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1410593862},{"type":"v1","addr":"172.21.15.100:6825","nonce":1410593862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1410593862},{"type":"v1","addr":"172.21.15.100:6823","nonce":1410593862}]},"public_addr":"172.21.15.100:6819/1410593862","cluster_addr":"172.21.15.100:6821/1410593862","heartbeat_back_addr":"172.21.15.100:6825/1410593862","heartbeat_front_addr":"172.21.15.100:6823/1410593862","state":["exists","up"]},{"osd":7,"uuid":"02ada28f-ac4f-47cf-983f-8a6da1140abe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2458690455},{"type":"v1","addr":"172.21.15.178:6817","nonce":2458690455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2458690455},{"type":"v1","addr":"172.21.15.178:6819","nonce":2458690455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2458690455},{"type":"v1","addr":"172.21.15.178:6823","nonce":2458690455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2458690455},{"type":"v1","addr":"172.21.15.178:6821","nonce":2458690455}]},"public_addr":"172.21.15.178:6817/2458690455","cluster_addr":"172.21.15.178:6819/2458690455","heartbeat_back_addr":"172.21.15.178:6823/2458690455","heartbeat_front_addr":"172.21.15.178:6821/2458690455","state":["exists","up"]},{"osd":8,"uuid":"e7e489bc-15fc-4a4c-b297-d1bb626513ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1523872826},{"type":"v1","addr":"172.21.15.188:6817","nonce":1523872826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1523872826},{"type":"v1","addr":"172.21.15.188:6819","nonce":1523872826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1523872826},{"type":"v1","addr":"172.21.15.188:6823","nonce":1523872826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1523872826},{"type":"v1","addr":"172.21.15.188:6821","nonce":1523872826}]},"public_addr":"172.21.15.188:6817/1523872826","cluster_addr":"172.21.15.188:6819/1523872826","heartbeat_back_addr":"172.21.15.188:6823/1523872826","heartbeat_front_addr":"172.21.15.188:6821/1523872826","state":["exists","up"]},{"osd":9,"uuid":"72328326-cd16-45d9-bfd9-40f6c5c20f5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3585761978},{"type":"v1","addr":"172.21.15.100:6827","nonce":3585761978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3585761978},{"type":"v1","addr":"172.21.15.100:6829","nonce":3585761978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3585761978},{"type":"v1","addr":"172.21.15.100:6833","nonce":3585761978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3585761978},{"type":"v1","addr":"172.21.15.100:6831","nonce":3585761978}]},"public_addr":"172.21.15.100:6827/3585761978","cluster_addr":"172.21.15.100:6829/3585761978","heartbeat_back_addr":"172.21.15.100:6833/3585761978","heartbeat_front_addr":"172.21.15.100:6831/3585761978","state":["exists","up"]},{"osd":10,"uuid":"a355f9a8-91a3-41d4-814d-5a9c9b953706","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2985841725},{"type":"v1","addr":"172.21.15.178:6825","nonce":2985841725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2985841725},{"type":"v1","addr":"172.21.15.178:6827","nonce":2985841725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2985841725},{"type":"v1","addr":"172.21.15.178:6831","nonce":2985841725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2985841725},{"type":"v1","addr":"172.21.15.178:6829","nonce":2985841725}]},"public_addr":"172.21.15.178:6825/2985841725","cluster_addr":"172.21.15.178:6827/2985841725","heartbeat_back_addr":"172.21.15.178:6831/2985841725","heartbeat_front_addr":"172.21.15.178:6829/2985841725","state":["exists","up"]},{"osd":11,"uuid":"76a5e13c-7e38-42f9-bfa7-7fedec7c7184","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2574955118},{"type":"v1","addr":"172.21.15.188:6825","nonce":2574955118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2574955118},{"type":"v1","addr":"172.21.15.188:6827","nonce":2574955118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2574955118},{"type":"v1","addr":"172.21.15.188:6831","nonce":2574955118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2574955118},{"type":"v1","addr":"172.21.15.188:6829","nonce":2574955118}]},"public_addr":"172.21.15.188:6825/2574955118","cluster_addr":"172.21.15.188:6827/2574955118","heartbeat_back_addr":"172.21.15.188:6831/2574955118","heartbeat_front_addr":"172.21.15.188:6829/2574955118","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:08:36.086637+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:08:58.948031+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:09:22.142517+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:09:47.655748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:10:12.853999+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:10:37.911383+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:11:04.448815+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:11:30.331740+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:11:55.819278+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:12:23.204612+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:12:49.484835+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:13:15.783531+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:6800/1597433518":"2024-05-11T11:07:00.092732+0000","172.21.15.100:6800/1086450256":"2024-05-11T11:06:33.114176+0000","172.21.15.100:0/767964410":"2024-05-11T11:07:00.092732+0000","172.21.15.100:6801/1597433518":"2024-05-11T11:07:00.092732+0000","172.21.15.100:6801/1086450256":"2024-05-11T11:06:33.114176+0000","172.21.15.100:0/2770572070":"2024-05-11T11:06:33.114176+0000","172.21.15.100:0/1379340513":"2024-05-11T11:06:33.114176+0000","172.21.15.100:0/3025646953":"2024-05-11T11:07:00.092732+0000","172.21.15.100:0/4099258930":"2024-05-11T11:07:00.092732+0000","172.21.15.100:0/4267693567":"2024-05-11T11:06:33.114176+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T11:18:09.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:09.287+0000 7f6155632700 1 -- 172.21.15.100:0/1225761570 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f613c0b2880 msgr2=0x7f613c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:09.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:09.287+0000 7f6155632700 1 --2- 172.21.15.100:0/1225761570 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f613c0b2880 0x7f613c0b4d40 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6140006fd0 tx=0x7f6140008040 comp rx=0 tx=0).stop 2024-05-10T11:18:09.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:09.287+0000 7f6155632700 1 -- 172.21.15.100:0/1225761570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6150108400 msgr2=0x7f6150104af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:09.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:09.287+0000 7f6155632700 1 --2- 172.21.15.100:0/1225761570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6150108400 0x7f6150104af0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f61480062a0 tx=0x7f6148004ed0 comp rx=0 tx=0).stop 2024-05-10T11:18:09.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:09.287+0000 7f6155632700 1 -- 172.21.15.100:0/1225761570 shutdown_connections 2024-05-10T11:18:09.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:09.287+0000 7f6155632700 1 --2- 172.21.15.100:0/1225761570 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f613c0b2880 0x7f613c0b4d40 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:09.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:09.287+0000 7f6155632700 1 --2- 172.21.15.100:0/1225761570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6150108400 0x7f6150104af0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:09.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:09.287+0000 7f6155632700 1 --2- 172.21.15.100:0/1225761570 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6150102390 0x7f6150104070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:09.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:09.287+0000 7f6155632700 1 --2- 172.21.15.100:0/1225761570 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6150102d50 0x7f61501045b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:09.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:09.287+0000 7f6155632700 1 -- 172.21.15.100:0/1225761570 >> 172.21.15.100:0/1225761570 conn(0x7f6150079310 msgr2=0x7f615006bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:09.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:09.287+0000 7f6155632700 1 -- 172.21.15.100:0/1225761570 shutdown_connections 2024-05-10T11:18:09.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:09.288+0000 7f6155632700 1 -- 172.21.15.100:0/1225761570 wait complete. 2024-05-10T11:18:09.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:18:09 smithi100 ceph-mon[30296]: pgmap v413: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T11:18:09.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:18:09 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/1225761570' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T11:18:09.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:18:09 smithi178 ceph-mon[33359]: pgmap v413: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T11:18:09.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:18:09 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/1225761570' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T11:18:09.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:18:09 smithi188 ceph-mon[34075]: pgmap v413: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T11:18:09.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:18:09 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/1225761570' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T11:18:09.984 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-10T11:14:27.242452+0000', 'modified': '2024-05-10T11:15:35.672461+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24439, 'mds_1': 24437, 'mds_2': 24427}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24395': {'gid': 24395, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.178:6835/2124076312', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6834', 'nonce': 2124076312}, {'type': 'v1', 'addr': '172.21.15.178:6835', 'nonce': 2124076312}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24413': {'gid': 24413, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.178:6837/3224358425', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6836', 'nonce': 3224358425}, {'type': 'v1', 'addr': '172.21.15.178:6837', 'nonce': 3224358425}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24421': {'gid': 24421, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.188:6835/515547690', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6834', 'nonce': 515547690}, {'type': 'v1', 'addr': '172.21.15.188:6835', 'nonce': 515547690}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24427': {'gid': 24427, 'name': 'c', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 25, 'addr': '172.21.15.188:6837/1943342620', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6836', 'nonce': 1943342620}, {'type': 'v1', 'addr': '172.21.15.188:6837', 'nonce': 1943342620}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24437': {'gid': 24437, 'name': 'b', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 21, 'addr': '172.21.15.178:6839/3760985404', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6838', 'nonce': 3760985404}, {'type': 'v1', 'addr': '172.21.15.178:6839', 'nonce': 3760985404}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24439': {'gid': 24439, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.188:6839/3194917070', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6838', 'nonce': 3194917070}, {'type': 'v1', 'addr': '172.21.15.188:6839', 'nonce': 3194917070}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24439, 'qdb_cluster': [24437, 24439, 24427]} 2024-05-10T11:18:09.984 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-10T11:18:09.984 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.c status 2024-05-10T11:18:10.258 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:18:10.553 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:18:10 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:18:10.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:18:10 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:18:10.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:18:10 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:18:11.749 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:18:11 smithi100 ceph-mon[30296]: pgmap v414: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T11:18:11.749 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:18:11 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:18:11.749 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:18:11 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:18:11.749 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:18:11 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:18:11.749 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:18:11 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:18:11.749 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:18:11 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:18:11.750 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:18:11 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:18:11.750 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:18:11 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:18:11.750 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:18:11 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:18:11.750 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:18:11 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:18:11.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.967+0000 7ff51a59a700 1 -- 172.21.15.100:0/1310298151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff514106e80 msgr2=0x7ff51410ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:11.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.967+0000 7ff51a59a700 1 --2- 172.21.15.100:0/1310298151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff514106e80 0x7ff51410ea50 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff50c00b0d0 tx=0x7ff50c00b3e0 comp rx=0 tx=0).stop 2024-05-10T11:18:11.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.967+0000 7ff51a59a700 1 -- 172.21.15.100:0/1310298151 shutdown_connections 2024-05-10T11:18:11.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.967+0000 7ff51a59a700 1 --2- 172.21.15.100:0/1310298151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff514106e80 0x7ff51410ea50 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:11.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.967+0000 7ff51a59a700 1 --2- 172.21.15.100:0/1310298151 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff514101da0 0x7ff514102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:11.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.967+0000 7ff51a59a700 1 --2- 172.21.15.100:0/1310298151 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff514102750 0x7ff5141067a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:11.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.967+0000 7ff51a59a700 1 -- 172.21.15.100:0/1310298151 >> 172.21.15.100:0/1310298151 conn(0x7ff5140789a0 msgr2=0x7ff5140ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:11.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.967+0000 7ff51a59a700 1 -- 172.21.15.100:0/1310298151 shutdown_connections 2024-05-10T11:18:11.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.967+0000 7ff51a59a700 1 -- 172.21.15.100:0/1310298151 wait complete. 2024-05-10T11:18:11.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.968+0000 7ff51a59a700 1 Processor -- start 2024-05-10T11:18:11.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.968+0000 7ff51a59a700 1 -- start start 2024-05-10T11:18:11.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.968+0000 7ff51a59a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff514101da0 0x7ff51419e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:11.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.968+0000 7ff51a59a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff514102750 0x7ff51419eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:11.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.968+0000 7ff51a59a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff514106e80 0x7ff514198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:11.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.968+0000 7ff51a59a700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff514111c40 con 0x7ff514102750 2024-05-10T11:18:11.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.968+0000 7ff51a59a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff514111ac0 con 0x7ff514106e80 2024-05-10T11:18:11.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.968+0000 7ff51a59a700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff514111dc0 con 0x7ff514101da0 2024-05-10T11:18:11.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.968+0000 7ff50bfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff514102750 0x7ff51419eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:11.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.968+0000 7ff50bfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff514102750 0x7ff51419eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59448/0 (socket says 172.21.15.100:59448) 2024-05-10T11:18:11.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.968+0000 7ff50bfff700 1 -- 172.21.15.100:0/2103324492 learned_addr learned my addr 172.21.15.100:0/2103324492 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:18:11.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.968+0000 7ff513fff700 1 --2- 172.21.15.100:0/2103324492 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff514101da0 0x7ff51419e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:11.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.968+0000 7ff518b37700 1 --2- 172.21.15.100:0/2103324492 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff514106e80 0x7ff514198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:11.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.968+0000 7ff50bfff700 1 -- 172.21.15.100:0/2103324492 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff514101da0 msgr2=0x7ff51419e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:11.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.968+0000 7ff50bfff700 1 --2- 172.21.15.100:0/2103324492 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff514101da0 0x7ff51419e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:11.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.968+0000 7ff50bfff700 1 -- 172.21.15.100:0/2103324492 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff514106e80 msgr2=0x7ff514198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:11.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.968+0000 7ff50bfff700 1 --2- 172.21.15.100:0/2103324492 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff514106e80 0x7ff514198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:11.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.968+0000 7ff50bfff700 1 -- 172.21.15.100:0/2103324492 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff514199020 con 0x7ff514102750 2024-05-10T11:18:11.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.968+0000 7ff50bfff700 1 --2- 172.21.15.100:0/2103324492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff514102750 0x7ff51419eb10 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc00c9e0 tx=0x7ff4fc00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:11.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.969+0000 7ff511ffb700 1 -- 172.21.15.100:0/2103324492 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc007aa0 con 0x7ff514102750 2024-05-10T11:18:11.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.969+0000 7ff511ffb700 1 -- 172.21.15.100:0/2103324492 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff4fc007c40 con 0x7ff514102750 2024-05-10T11:18:11.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.969+0000 7ff51a59a700 1 -- 172.21.15.100:0/2103324492 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff514199310 con 0x7ff514102750 2024-05-10T11:18:11.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.969+0000 7ff511ffb700 1 -- 172.21.15.100:0/2103324492 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc018400 con 0x7ff514102750 2024-05-10T11:18:11.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.969+0000 7ff51a59a700 1 -- 172.21.15.100:0/2103324492 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff514104c30 con 0x7ff514102750 2024-05-10T11:18:11.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.971+0000 7ff511ffb700 1 -- 172.21.15.100:0/2103324492 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7ff4fc00f450 con 0x7ff514102750 2024-05-10T11:18:11.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.972+0000 7ff511ffb700 1 --2- 172.21.15.100:0/2103324492 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff5000b2880 0x7ff5000b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:11.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.972+0000 7ff513fff700 1 --2- 172.21.15.100:0/2103324492 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff5000b2880 0x7ff5000b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:11.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.972+0000 7ff511ffb700 1 -- 172.21.15.100:0/2103324492 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7ff4fc0e2490 con 0x7ff514102750 2024-05-10T11:18:11.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.973+0000 7ff513fff700 1 --2- 172.21.15.100:0/2103324492 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff5000b2880 0x7ff5000b4d40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff504009a10 tx=0x7ff504009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:11.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.973+0000 7ff524e2db80 1 Processor -- start 2024-05-10T11:18:11.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.973+0000 7ff524e2db80 1 -- start start 2024-05-10T11:18:11.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.973+0000 7ff524e2db80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ecf339540 0x555ecf339920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:11.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.973+0000 7ff524e2db80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecf339e60 0x555ecf333bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:11.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.973+0000 7ff524e2db80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ecf334160 0x555ecf3345c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:11.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.973+0000 7ff524e2db80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x555ecf2818f0 con 0x555ecf339e60 2024-05-10T11:18:11.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.973+0000 7ff524e2db80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x555ecf1e6d50 con 0x555ecf339540 2024-05-10T11:18:11.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.973+0000 7ff524e2db80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x555ecf27d8b0 con 0x555ecf334160 2024-05-10T11:18:11.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.973+0000 7ff50bfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecf339e60 0x555ecf333bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:11.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.973+0000 7ff513fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ecf339540 0x555ecf339920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:11.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.973+0000 7ff50bfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecf339e60 0x555ecf333bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59454/0 (socket says 172.21.15.100:59454) 2024-05-10T11:18:11.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.973+0000 7ff50bfff700 1 -- 172.21.15.100:0/1307143567 learned_addr learned my addr 172.21.15.100:0/1307143567 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:18:11.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.973+0000 7ff518b37700 1 --2- 172.21.15.100:0/1307143567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ecf334160 0x555ecf3345c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:11.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.973+0000 7ff50bfff700 1 -- 172.21.15.100:0/1307143567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ecf334160 msgr2=0x555ecf3345c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:18:11.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.973+0000 7ff50bfff700 1 --2- 172.21.15.100:0/1307143567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ecf334160 0x555ecf3345c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:11.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.973+0000 7ff50bfff700 1 -- 172.21.15.100:0/1307143567 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ecf339540 msgr2=0x555ecf339920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:18:11.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.973+0000 7ff50bfff700 1 --2- 172.21.15.100:0/1307143567 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ecf339540 0x555ecf339920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:11.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.973+0000 7ff50bfff700 1 -- 172.21.15.100:0/1307143567 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ecf1ce880 con 0x555ecf339e60 2024-05-10T11:18:11.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.973+0000 7ff513fff700 1 --2- 172.21.15.100:0/1307143567 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ecf339540 0x555ecf339920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:18:11.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.973+0000 7ff50bfff700 1 --2- 172.21.15.100:0/1307143567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecf339e60 0x555ecf333bd0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc0a9450 tx=0x7ff4fc0a8f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8ef959beed3d42a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:11.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.974+0000 7ff50a7fc700 1 -- 172.21.15.100:0/1307143567 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc0c3070 con 0x555ecf339e60 2024-05-10T11:18:11.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.974+0000 7ff50a7fc700 1 -- 172.21.15.100:0/1307143567 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff4fc0a9e50 con 0x555ecf339e60 2024-05-10T11:18:11.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.974+0000 7ff50a7fc700 1 -- 172.21.15.100:0/1307143567 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc00f740 con 0x555ecf339e60 2024-05-10T11:18:11.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.974+0000 7ff524e2db80 1 -- 172.21.15.100:0/1307143567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecf339e60 msgr2=0x555ecf333bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:11.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.974+0000 7ff524e2db80 1 --2- 172.21.15.100:0/1307143567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecf339e60 0x555ecf333bd0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc0a9450 tx=0x7ff4fc0a8f40 comp rx=0 tx=0).stop 2024-05-10T11:18:11.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.974+0000 7ff524e2db80 1 -- 172.21.15.100:0/1307143567 shutdown_connections 2024-05-10T11:18:11.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.974+0000 7ff524e2db80 1 --2- 172.21.15.100:0/1307143567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ecf334160 0x555ecf3345c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:11.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.974+0000 7ff524e2db80 1 --2- 172.21.15.100:0/1307143567 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ecf339540 0x555ecf339920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:11.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.974+0000 7ff524e2db80 1 --2- 172.21.15.100:0/1307143567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecf339e60 0x555ecf333bd0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:11.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.974+0000 7ff524e2db80 1 -- 172.21.15.100:0/1307143567 >> 172.21.15.100:0/1307143567 conn(0x555ecf25bb20 msgr2=0x555ecf32a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:11.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.974+0000 7ff524e2db80 1 -- 172.21.15.100:0/1307143567 shutdown_connections 2024-05-10T11:18:11.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.974+0000 7ff524e2db80 1 -- 172.21.15.100:0/1307143567 wait complete. 2024-05-10T11:18:11.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.975+0000 7ff524e2db80 1 -- start start 2024-05-10T11:18:11.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.975+0000 7ff524e2db80 1 Processor -- start 2024-05-10T11:18:11.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.975+0000 7ff524e2db80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ecf334160 0x555ecf33fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:11.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.975+0000 7ff524e2db80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ecf339540 0x555ecf33ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:11.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.975+0000 7ff524e2db80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecf339e60 0x555ecf33d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:11.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.975+0000 7ff524e2db80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x555ecf340710 con 0x555ecf339e60 2024-05-10T11:18:11.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.975+0000 7ff524e2db80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x555ecf23ae80 con 0x555ecf339540 2024-05-10T11:18:11.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.975+0000 7ff524e2db80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x555ecf340890 con 0x555ecf334160 2024-05-10T11:18:11.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.975+0000 7ff524e2db80 10 client.0 _ll_register_callbacks cb 0x555ecf215560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:18:11.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.975+0000 7ff518b37700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecf339e60 0x555ecf33d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:11.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.975+0000 7ff518b37700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecf339e60 0x555ecf33d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59456/0 (socket says 172.21.15.100:59456) 2024-05-10T11:18:11.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.975+0000 7ff518b37700 1 -- 172.21.15.100:0/3788419217 learned_addr learned my addr 172.21.15.100:0/3788419217 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:18:11.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.975+0000 7ff513fff700 1 --2- 172.21.15.100:0/3788419217 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ecf334160 0x555ecf33fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:11.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.976+0000 7ff50bfff700 1 --2- 172.21.15.100:0/3788419217 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ecf339540 0x555ecf33ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:11.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.976+0000 7ff518b37700 1 -- 172.21.15.100:0/3788419217 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ecf334160 msgr2=0x555ecf33fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:18:11.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.976+0000 7ff518b37700 1 --2- 172.21.15.100:0/3788419217 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ecf334160 0x555ecf33fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:11.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.976+0000 7ff518b37700 1 -- 172.21.15.100:0/3788419217 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ecf339540 msgr2=0x555ecf33ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:18:11.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.976+0000 7ff518b37700 1 --2- 172.21.15.100:0/3788419217 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ecf339540 0x555ecf33ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:11.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.976+0000 7ff518b37700 1 -- 172.21.15.100:0/3788419217 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ecf238710 con 0x555ecf339e60 2024-05-10T11:18:11.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.976+0000 7ff50bfff700 1 --2- 172.21.15.100:0/3788419217 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ecf339540 0x555ecf33ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:18:11.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.976+0000 7ff513fff700 1 --2- 172.21.15.100:0/3788419217 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ecf334160 0x555ecf33fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:18:11.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.976+0000 7ff518b37700 1 --2- 172.21.15.100:0/3788419217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecf339e60 0x555ecf33d3b0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff50c009e50 tx=0x7ff50c007590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c4d2d224c682291 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:11.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.976+0000 7ff4f37fe700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:18:11.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.976+0000 7ff4f37fe700 1 -- 172.21.15.100:0/3788419217 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff50c003c10 con 0x555ecf339e60 2024-05-10T11:18:11.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.976+0000 7ff524e2db80 1 -- 172.21.15.100:0/3788419217 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4fc0a9e50 con 0x555ecf339e60 2024-05-10T11:18:11.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.976+0000 7ff4f37fe700 1 -- 172.21.15.100:0/3788419217 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7ff50c003db0 con 0x555ecf339e60 2024-05-10T11:18:11.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.976+0000 7ff4f37fe700 1 -- 172.21.15.100:0/3788419217 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff50c02f050 con 0x555ecf339e60 2024-05-10T11:18:11.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.976+0000 7ff524e2db80 10 client.15153 fetch_fsmap learned FSMap version 26 2024-05-10T11:18:11.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.976+0000 7ff524e2db80 1 -- 172.21.15.100:0/3788419217 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555ecf34af70 con 0x555ecf339e60 2024-05-10T11:18:11.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.976+0000 7ff4f37fe700 1 -- 172.21.15.100:0/3788419217 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff4fc0a9e50 con 0x555ecf339e60 2024-05-10T11:18:11.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.977+0000 7ff4f37fe700 1 -- 172.21.15.100:0/3788419217 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7ff50c011070 con 0x555ecf339e60 2024-05-10T11:18:11.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.977+0000 7ff524e2db80 10 client.15153 fetch_fsmap finished waiting for FSMap version 26 2024-05-10T11:18:11.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.977+0000 7ff524e2db80 10 client.15153 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-10T11:18:11.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.977+0000 7ff524e2db80 20 client.15153 populate_metadata read hostname 'smithi100' 2024-05-10T11:18:11.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.977+0000 7ff524e2db80 1 --2- 172.21.15.100:0/3788419217 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x555ecf34d5b0 0x555ecf34ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:11.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.977+0000 7ff524e2db80 4 client.15153 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:18:11.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.977+0000 7ff524e2db80 1 -- 172.21.15.100:0/3788419217 --> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ecf28f2d0 con 0x555ecf34d5b0 2024-05-10T11:18:11.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.977+0000 7ff513fff700 1 --2- 172.21.15.100:0/3788419217 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x555ecf34d5b0 0x555ecf34ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:11.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.977+0000 7ff4f37fe700 10 client.15153 ms_handle_connect on v2:172.21.15.188:6836/1943342620 2024-05-10T11:18:11.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.977+0000 7ff513fff700 1 --2- 172.21.15.100:0/3788419217 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x555ecf34d5b0 0x555ecf34ea20 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cef2cd53114437c2 server_cookie=ec4270024a892b0e in_seq=0 out_seq=0 2024-05-10T11:18:11.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.978+0000 7ff4f37fe700 1 -- 172.21.15.100:0/3788419217 <== mds.2 v2:172.21.15.188:6836/1943342620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555ecf28f2d0 con 0x555ecf34d5b0 2024-05-10T11:18:11.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.978+0000 7ff4f37fe700 10 client.15153 handle_command_reply: tid=0 2024-05-10T11:18:11.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.978+0000 7ff524e2db80 1 client.15153 shutdown 2024-05-10T11:18:11.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.978+0000 7ff524e2db80 1 -- 172.21.15.100:0/3788419217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecf339e60 msgr2=0x555ecf33d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:11.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.978+0000 7ff524e2db80 1 --2- 172.21.15.100:0/3788419217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecf339e60 0x555ecf33d3b0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff50c009e50 tx=0x7ff50c007590 comp rx=0 tx=0).stop 2024-05-10T11:18:11.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.979+0000 7ff524e2db80 1 -- 172.21.15.100:0/3788419217 shutdown_connections 2024-05-10T11:18:11.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.979+0000 7ff524e2db80 1 --2- 172.21.15.100:0/3788419217 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x555ecf34d5b0 0x555ecf34ea20 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:11.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.979+0000 7ff4f37fe700 0 client.15153 ms_handle_reset on v2:172.21.15.188:6836/1943342620 2024-05-10T11:18:11.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.979+0000 7ff524e2db80 1 --2- 172.21.15.100:0/3788419217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecf339e60 0x555ecf33d3b0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:11.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.979+0000 7ff524e2db80 1 --2- 172.21.15.100:0/3788419217 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ecf334160 0x555ecf33fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:11.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.979+0000 7ff524e2db80 1 --2- 172.21.15.100:0/3788419217 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ecf339540 0x555ecf33ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:11.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.979+0000 7ff524e2db80 1 -- 172.21.15.100:0/3788419217 >> 172.21.15.100:0/3788419217 conn(0x555ecf25bb20 msgr2=0x555ecf324ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:11.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.979+0000 7ff524e2db80 1 -- 172.21.15.100:0/3788419217 shutdown_connections 2024-05-10T11:18:11.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.979+0000 7ff524e2db80 1 -- 172.21.15.100:0/3788419217 wait complete. 2024-05-10T11:18:11.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.979+0000 7ff524e2db80 20 client.15153 trim_cache size 0 max 16384 2024-05-10T11:18:12.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.999+0000 7ff524e2db80 1 Processor -- start 2024-05-10T11:18:12.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.999+0000 7ff524e2db80 1 -- start start 2024-05-10T11:18:12.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.999+0000 7ff524e2db80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ecf334160 0x555ecf33d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:12.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.999+0000 7ff524e2db80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecf339540 0x555ecf324bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:12.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.999+0000 7ff524e2db80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ecf339e60 0x555ecf33a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:12.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.999+0000 7ff524e2db80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x555ecf34cd60 con 0x555ecf339540 2024-05-10T11:18:12.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.999+0000 7ff524e2db80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x555ecf34d430 con 0x555ecf339e60 2024-05-10T11:18:12.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:11.999+0000 7ff524e2db80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x555ecf34db00 con 0x555ecf334160 2024-05-10T11:18:12.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.000+0000 7ff50bfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecf339540 0x555ecf324bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:12.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.000+0000 7ff50bfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecf339540 0x555ecf324bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59470/0 (socket says 172.21.15.100:59470) 2024-05-10T11:18:12.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.000+0000 7ff50bfff700 1 -- 172.21.15.100:0/3349923858 learned_addr learned my addr 172.21.15.100:0/3349923858 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:18:12.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.000+0000 7ff513fff700 1 --2- 172.21.15.100:0/3349923858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ecf334160 0x555ecf33d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:12.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.000+0000 7ff518b37700 1 --2- 172.21.15.100:0/3349923858 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ecf339e60 0x555ecf33a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:12.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.000+0000 7ff50bfff700 1 -- 172.21.15.100:0/3349923858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ecf334160 msgr2=0x555ecf33d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:18:12.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.000+0000 7ff50bfff700 1 --2- 172.21.15.100:0/3349923858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ecf334160 0x555ecf33d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:12.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.000+0000 7ff50bfff700 1 -- 172.21.15.100:0/3349923858 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ecf339e60 msgr2=0x555ecf33a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:18:12.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.000+0000 7ff50bfff700 1 --2- 172.21.15.100:0/3349923858 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ecf339e60 0x555ecf33a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:12.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.000+0000 7ff50bfff700 1 -- 172.21.15.100:0/3349923858 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ecf326c30 con 0x555ecf339540 2024-05-10T11:18:12.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.000+0000 7ff518b37700 1 --2- 172.21.15.100:0/3349923858 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ecf339e60 0x555ecf33a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:18:12.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.000+0000 7ff50bfff700 1 --2- 172.21.15.100:0/3349923858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecf339540 0x555ecf324bf0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc0a9030 tx=0x7ff4fc0a5700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f624f43e0fc1e18 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:12.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.000+0000 7ff509ffb700 1 -- 172.21.15.100:0/3349923858 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc0c3040 con 0x555ecf339540 2024-05-10T11:18:12.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.000+0000 7ff509ffb700 1 -- 172.21.15.100:0/3349923858 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff4fc0a9e50 con 0x555ecf339540 2024-05-10T11:18:12.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.000+0000 7ff509ffb700 1 -- 172.21.15.100:0/3349923858 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc00f740 con 0x555ecf339540 2024-05-10T11:18:12.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:18:11 smithi178 ceph-mon[33359]: pgmap v414: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T11:18:12.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:18:11 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:18:12.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:18:11 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:18:12.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:18:11 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:18:12.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:18:11 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:18:12.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:18:11 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:18:12.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:18:11 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:18:12.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:18:11 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:18:12.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:18:11 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:18:12.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:18:11 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:18:12.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.001+0000 7ff524e2db80 1 -- 172.21.15.100:0/3349923858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecf339540 msgr2=0x555ecf324bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:12.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.001+0000 7ff524e2db80 1 --2- 172.21.15.100:0/3349923858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecf339540 0x555ecf324bf0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc0a9030 tx=0x7ff4fc0a5700 comp rx=0 tx=0).stop 2024-05-10T11:18:12.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.001+0000 7ff524e2db80 1 -- 172.21.15.100:0/3349923858 shutdown_connections 2024-05-10T11:18:12.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.001+0000 7ff524e2db80 1 --2- 172.21.15.100:0/3349923858 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ecf339e60 0x555ecf33a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:12.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.001+0000 7ff524e2db80 1 --2- 172.21.15.100:0/3349923858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ecf334160 0x555ecf33d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:12.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.001+0000 7ff524e2db80 1 --2- 172.21.15.100:0/3349923858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecf339540 0x555ecf324bf0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:12.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.001+0000 7ff524e2db80 1 -- 172.21.15.100:0/3349923858 >> 172.21.15.100:0/3349923858 conn(0x555ecf25bb20 msgr2=0x555ecf33b760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:12.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.001+0000 7ff524e2db80 1 -- 172.21.15.100:0/3349923858 shutdown_connections 2024-05-10T11:18:12.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.001+0000 7ff524e2db80 1 -- 172.21.15.100:0/3349923858 wait complete. 2024-05-10T11:18:12.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.002+0000 7ff524e2db80 1 -- start start 2024-05-10T11:18:12.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.002+0000 7ff524e2db80 1 Processor -- start 2024-05-10T11:18:12.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.002+0000 7ff524e2db80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ecf334160 0x555ecf323fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:12.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.002+0000 7ff524e2db80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecf339540 0x555ecf324520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:12.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.002+0000 7ff524e2db80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ecf339e60 0x555ecf324a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:12.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.002+0000 7ff524e2db80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x555ecf34e8a0 con 0x555ecf339540 2024-05-10T11:18:12.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.002+0000 7ff524e2db80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x555ecf34e1d0 con 0x555ecf339e60 2024-05-10T11:18:12.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.002+0000 7ff524e2db80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x555ecf34ef70 con 0x555ecf334160 2024-05-10T11:18:12.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.002+0000 7ff524e2db80 10 client.0 _ll_register_callbacks cb 0x555ecf215560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:18:12.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.002+0000 7ff50bfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecf339540 0x555ecf324520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:12.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.002+0000 7ff513fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ecf334160 0x555ecf323fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:12.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.002+0000 7ff513fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ecf334160 0x555ecf323fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:40592/0 (socket says 172.21.15.100:40592) 2024-05-10T11:18:12.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.002+0000 7ff50bfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecf339540 0x555ecf324520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59474/0 (socket says 172.21.15.100:59474) 2024-05-10T11:18:12.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.003+0000 7ff513fff700 1 -- 172.21.15.100:0/1688142854 learned_addr learned my addr 172.21.15.100:0/1688142854 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:18:12.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.003+0000 7ff518b37700 1 --2- 172.21.15.100:0/1688142854 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ecf339e60 0x555ecf324a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:12.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.003+0000 7ff513fff700 1 -- 172.21.15.100:0/1688142854 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ecf339e60 msgr2=0x555ecf324a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:18:12.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.003+0000 7ff513fff700 1 --2- 172.21.15.100:0/1688142854 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ecf339e60 0x555ecf324a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:12.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.003+0000 7ff513fff700 1 -- 172.21.15.100:0/1688142854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecf339540 msgr2=0x555ecf324520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:18:12.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.003+0000 7ff513fff700 1 --2- 172.21.15.100:0/1688142854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecf339540 0x555ecf324520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:12.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.003+0000 7ff513fff700 1 -- 172.21.15.100:0/1688142854 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ecf356150 con 0x555ecf334160 2024-05-10T11:18:12.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.003+0000 7ff513fff700 1 --2- 172.21.15.100:0/1688142854 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ecf334160 0x555ecf323fe0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff5040155c0 tx=0x7ff504004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ded8ec596baa587 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:12.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.003+0000 7ff4f37fe700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:18:12.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.003+0000 7ff4f37fe700 1 -- 172.21.15.100:0/1688142854 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff50401b040 con 0x555ecf334160 2024-05-10T11:18:12.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.003+0000 7ff524e2db80 1 -- 172.21.15.100:0/1688142854 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4fc0a9e50 con 0x555ecf334160 2024-05-10T11:18:12.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.003+0000 7ff4f37fe700 1 -- 172.21.15.100:0/1688142854 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7ff504015cd0 con 0x555ecf334160 2024-05-10T11:18:12.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.003+0000 7ff4f37fe700 1 -- 172.21.15.100:0/1688142854 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff50403b050 con 0x555ecf334160 2024-05-10T11:18:12.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.003+0000 7ff524e2db80 10 client.24820 fetch_fsmap learned FSMap version 26 2024-05-10T11:18:12.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.003+0000 7ff524e2db80 1 -- 172.21.15.100:0/1688142854 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555ecf3b8d30 con 0x555ecf334160 2024-05-10T11:18:12.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.004+0000 7ff4f37fe700 1 -- 172.21.15.100:0/1688142854 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff4fc0a9e50 con 0x555ecf334160 2024-05-10T11:18:12.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.004+0000 7ff4f37fe700 1 -- 172.21.15.100:0/1688142854 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7ff504022430 con 0x555ecf334160 2024-05-10T11:18:12.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.004+0000 7ff524e2db80 10 client.24820 fetch_fsmap finished waiting for FSMap version 26 2024-05-10T11:18:12.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.004+0000 7ff524e2db80 10 client.24820 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-10T11:18:12.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.004+0000 7ff524e2db80 20 client.24820 populate_metadata read hostname 'smithi100' 2024-05-10T11:18:12.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.004+0000 7ff524e2db80 1 --2- 172.21.15.100:0/1688142854 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x555ecf34d5b0 0x555ecf3bc610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:12.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.004+0000 7ff524e2db80 4 client.24820 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T11:18:12.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.004+0000 7ff524e2db80 1 -- 172.21.15.100:0/1688142854 --> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] -- command(tid 0: {"prefix": "status"}) -- 0x555ecf289370 con 0x555ecf34d5b0 2024-05-10T11:18:12.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.004+0000 7ff50bfff700 1 --2- 172.21.15.100:0/1688142854 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x555ecf34d5b0 0x555ecf3bc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:12.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.004+0000 7ff4f37fe700 10 client.24820 ms_handle_connect on v2:172.21.15.188:6836/1943342620 2024-05-10T11:18:12.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.004+0000 7ff50bfff700 1 --2- 172.21.15.100:0/1688142854 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x555ecf34d5b0 0x555ecf3bc610 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d4d627a51c344f80 server_cookie=5a758ed24e1263bc in_seq=0 out_seq=0 2024-05-10T11:18:12.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.005+0000 7ff4f37fe700 1 -- 172.21.15.100:0/1688142854 <== mds.2 v2:172.21.15.188:6836/1943342620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x555ecf289370 con 0x555ecf34d5b0 2024-05-10T11:18:12.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.005+0000 7ff4f37fe700 10 client.24820 handle_command_reply: tid=0 2024-05-10T11:18:12.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.005+0000 7ff524e2db80 1 client.24820 shutdown 2024-05-10T11:18:12.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.005+0000 7ff524e2db80 1 -- 172.21.15.100:0/1688142854 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ecf334160 msgr2=0x555ecf323fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:12.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.005+0000 7ff524e2db80 1 --2- 172.21.15.100:0/1688142854 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ecf334160 0x555ecf323fe0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff5040155c0 tx=0x7ff504004750 comp rx=0 tx=0).stop 2024-05-10T11:18:12.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.005+0000 7ff524e2db80 1 -- 172.21.15.100:0/1688142854 shutdown_connections 2024-05-10T11:18:12.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.005+0000 7ff524e2db80 1 --2- 172.21.15.100:0/1688142854 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x555ecf34d5b0 0x555ecf3bc610 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:12.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.005+0000 7ff524e2db80 1 --2- 172.21.15.100:0/1688142854 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ecf339e60 0x555ecf324a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:12.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.005+0000 7ff4f37fe700 0 client.24820 ms_handle_reset on v2:172.21.15.188:6836/1943342620 2024-05-10T11:18:12.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.005+0000 7ff524e2db80 1 --2- 172.21.15.100:0/1688142854 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ecf334160 0x555ecf323fe0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:12.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.005+0000 7ff524e2db80 1 --2- 172.21.15.100:0/1688142854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecf339540 0x555ecf324520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:12.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.005+0000 7ff524e2db80 1 -- 172.21.15.100:0/1688142854 >> 172.21.15.100:0/1688142854 conn(0x555ecf25bb20 msgr2=0x555ecf33b360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:12.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.005+0000 7ff524e2db80 1 -- 172.21.15.100:0/1688142854 shutdown_connections 2024-05-10T11:18:12.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.005+0000 7ff524e2db80 1 -- 172.21.15.100:0/1688142854 wait complete. 2024-05-10T11:18:12.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.005+0000 7ff524e2db80 20 client.24820 trim_cache size 0 max 16384 2024-05-10T11:18:12.016 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:18:12.016 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_fsid": "20a542cc-0ebd-11ef-bc98-c7b262605968", 2024-05-10T11:18:12.016 INFO:teuthology.orchestra.run.smithi100.stdout: "whoami": 2, 2024-05-10T11:18:12.016 INFO:teuthology.orchestra.run.smithi100.stdout: "id": 24427, 2024-05-10T11:18:12.017 INFO:teuthology.orchestra.run.smithi100.stdout: "want_state": "up:active", 2024-05-10T11:18:12.017 INFO:teuthology.orchestra.run.smithi100.stdout: "state": "up:active", 2024-05-10T11:18:12.017 INFO:teuthology.orchestra.run.smithi100.stdout: "fs_name": "cephfs", 2024-05-10T11:18:12.017 INFO:teuthology.orchestra.run.smithi100.stdout: "rank_uptime": 162.21680191499999, 2024-05-10T11:18:12.017 INFO:teuthology.orchestra.run.smithi100.stdout: "mdsmap_epoch": 26, 2024-05-10T11:18:12.017 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch": 89, 2024-05-10T11:18:12.018 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch_barrier": 89, 2024-05-10T11:18:12.018 INFO:teuthology.orchestra.run.smithi100.stdout: "uptime": 258.21244870300001 2024-05-10T11:18:12.018 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:18:12.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.006+0000 7ff51a59a700 1 -- 172.21.15.100:0/2103324492 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff5000b2880 msgr2=0x7ff5000b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:12.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.006+0000 7ff51a59a700 1 --2- 172.21.15.100:0/2103324492 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff5000b2880 0x7ff5000b4d40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff504009a10 tx=0x7ff504009190 comp rx=0 tx=0).stop 2024-05-10T11:18:12.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.006+0000 7ff51a59a700 1 -- 172.21.15.100:0/2103324492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff514102750 msgr2=0x7ff51419eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:12.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.006+0000 7ff51a59a700 1 --2- 172.21.15.100:0/2103324492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff514102750 0x7ff51419eb10 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc00c9e0 tx=0x7ff4fc00ccf0 comp rx=0 tx=0).stop 2024-05-10T11:18:12.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.006+0000 7ff51a59a700 1 -- 172.21.15.100:0/2103324492 shutdown_connections 2024-05-10T11:18:12.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.006+0000 7ff51a59a700 1 --2- 172.21.15.100:0/2103324492 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff5000b2880 0x7ff5000b4d40 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:12.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.006+0000 7ff51a59a700 1 --2- 172.21.15.100:0/2103324492 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff514106e80 0x7ff514198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:12.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.006+0000 7ff51a59a700 1 --2- 172.21.15.100:0/2103324492 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff514101da0 0x7ff51419e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:12.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.006+0000 7ff51a59a700 1 --2- 172.21.15.100:0/2103324492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff514102750 0x7ff51419eb10 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:12.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.006+0000 7ff51a59a700 1 -- 172.21.15.100:0/2103324492 >> 172.21.15.100:0/2103324492 conn(0x7ff5140789a0 msgr2=0x7ff5140ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:12.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.006+0000 7ff51a59a700 1 -- 172.21.15.100:0/2103324492 shutdown_connections 2024-05-10T11:18:12.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:12.006+0000 7ff51a59a700 1 -- 172.21.15.100:0/2103324492 wait complete. 2024-05-10T11:18:12.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:18:11 smithi188 ceph-mon[34075]: pgmap v414: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T11:18:12.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:18:11 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:18:12.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:18:11 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:18:12.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:18:11 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:18:12.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:18:11 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:18:12.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:18:11 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:18:12.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:18:11 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:18:12.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:18:11 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:18:12.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:18:11 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:18:12.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:18:11 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:18:12.691 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.b status 2024-05-10T11:18:12.934 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:18:13.754 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:18:13 smithi100 ceph-mon[30296]: pgmap v415: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T11:18:14.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:18:13 smithi178 ceph-mon[33359]: pgmap v415: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T11:18:14.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:18:13 smithi188 ceph-mon[34075]: pgmap v415: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T11:18:14.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.569+0000 7f998f4e5700 1 -- 172.21.15.100:0/1892335647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9988103eb0 msgr2=0x7f9988110390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:14.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.569+0000 7f998f4e5700 1 --2- 172.21.15.100:0/1892335647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9988103eb0 0x7f9988110390 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f998000b0a0 tx=0x7f998000b3b0 comp rx=0 tx=0).stop 2024-05-10T11:18:14.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.569+0000 7f998f4e5700 1 -- 172.21.15.100:0/1892335647 shutdown_connections 2024-05-10T11:18:14.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.569+0000 7f998f4e5700 1 --2- 172.21.15.100:0/1892335647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9988103eb0 0x7f9988110390 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:14.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.569+0000 7f998f4e5700 1 --2- 172.21.15.100:0/1892335647 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9988109600 0x7f99881099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:14.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.569+0000 7f998f4e5700 1 --2- 172.21.15.100:0/1892335647 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99881034f0 0x7f9988103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:14.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.569+0000 7f998f4e5700 1 -- 172.21.15.100:0/1892335647 >> 172.21.15.100:0/1892335647 conn(0x7f998807a650 msgr2=0x7f998807aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:14.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.569+0000 7f998f4e5700 1 -- 172.21.15.100:0/1892335647 shutdown_connections 2024-05-10T11:18:14.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.569+0000 7f998f4e5700 1 -- 172.21.15.100:0/1892335647 wait complete. 2024-05-10T11:18:14.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.570+0000 7f998f4e5700 1 Processor -- start 2024-05-10T11:18:14.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.570+0000 7f998f4e5700 1 -- start start 2024-05-10T11:18:14.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.570+0000 7f998f4e5700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99881034f0 0x7f99880757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:14.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.570+0000 7f998f4e5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9988103eb0 0x7f9988075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:14.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.570+0000 7f998f4e5700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9988109600 0x7f9988076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:14.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.570+0000 7f998f4e5700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9988113600 con 0x7f9988109600 2024-05-10T11:18:14.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.570+0000 7f998f4e5700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9988113480 con 0x7f9988103eb0 2024-05-10T11:18:14.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.570+0000 7f998f4e5700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9988113780 con 0x7f99881034f0 2024-05-10T11:18:14.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.570+0000 7f998da82700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9988109600 0x7f9988076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:14.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.570+0000 7f998d281700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99881034f0 0x7f99880757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:14.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.571+0000 7f998da82700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9988109600 0x7f9988076220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59490/0 (socket says 172.21.15.100:59490) 2024-05-10T11:18:14.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.571+0000 7f998da82700 1 -- 172.21.15.100:0/3676827936 learned_addr learned my addr 172.21.15.100:0/3676827936 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:18:14.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.571+0000 7f998ca80700 1 --2- 172.21.15.100:0/3676827936 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9988103eb0 0x7f9988075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:14.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.571+0000 7f998da82700 1 -- 172.21.15.100:0/3676827936 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99881034f0 msgr2=0x7f99880757a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:14.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.571+0000 7f998da82700 1 --2- 172.21.15.100:0/3676827936 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99881034f0 0x7f99880757a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:14.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.571+0000 7f998da82700 1 -- 172.21.15.100:0/3676827936 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9988103eb0 msgr2=0x7f9988075ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:14.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.571+0000 7f998da82700 1 --2- 172.21.15.100:0/3676827936 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9988103eb0 0x7f9988075ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:14.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.571+0000 7f998da82700 1 -- 172.21.15.100:0/3676827936 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99881a8110 con 0x7f9988109600 2024-05-10T11:18:14.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.571+0000 7f998da82700 1 --2- 172.21.15.100:0/3676827936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9988109600 0x7f9988076220 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f99800062a0 tx=0x7f9980004dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:14.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.571+0000 7f997e7fc700 1 -- 172.21.15.100:0/3676827936 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9980021070 con 0x7f9988109600 2024-05-10T11:18:14.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.571+0000 7f997e7fc700 1 -- 172.21.15.100:0/3676827936 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9980007c00 con 0x7f9988109600 2024-05-10T11:18:14.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.571+0000 7f997e7fc700 1 -- 172.21.15.100:0/3676827936 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99800052a0 con 0x7f9988109600 2024-05-10T11:18:14.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.571+0000 7f998f4e5700 1 -- 172.21.15.100:0/3676827936 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99881a83a0 con 0x7f9988109600 2024-05-10T11:18:14.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.571+0000 7f998f4e5700 1 -- 172.21.15.100:0/3676827936 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99881a87a0 con 0x7f9988109600 2024-05-10T11:18:14.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.575+0000 7f9999d78b80 1 Processor -- start 2024-05-10T11:18:14.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.576+0000 7f997e7fc700 1 -- 172.21.15.100:0/3676827936 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f9980024070 con 0x7f9988109600 2024-05-10T11:18:14.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.576+0000 7f9999d78b80 1 -- start start 2024-05-10T11:18:14.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.576+0000 7f9999d78b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fe2f054480 0x55fe2f054860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:14.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.576+0000 7f9999d78b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe2f054da0 0x55fe2f0586a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:14.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.576+0000 7f9999d78b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe2f058c30 0x55fe2f0590b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:14.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.576+0000 7f9999d78b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55fe2efa6810 con 0x55fe2f058c30 2024-05-10T11:18:14.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.576+0000 7f9999d78b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55fe2ef45770 con 0x55fe2f054480 2024-05-10T11:18:14.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.576+0000 7f9999d78b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55fe2efa27d0 con 0x55fe2f054da0 2024-05-10T11:18:14.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.576+0000 7f998ca80700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe2f058c30 0x55fe2f0590b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:14.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.576+0000 7f998ca80700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe2f058c30 0x55fe2f0590b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59494/0 (socket says 172.21.15.100:59494) 2024-05-10T11:18:14.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.577+0000 7f998ca80700 1 -- 172.21.15.100:0/4031907278 learned_addr learned my addr 172.21.15.100:0/4031907278 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:18:14.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.577+0000 7f998d281700 1 --2- 172.21.15.100:0/4031907278 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe2f054da0 0x55fe2f0586a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:14.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.577+0000 7f998ca80700 1 --2- 172.21.15.100:0/4031907278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fe2f054480 0x55fe2f054860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:14.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.577+0000 7f997e7fc700 1 --2- 172.21.15.100:0/3676827936 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f99740b2900 0x7f99740b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:14.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.577+0000 7f998ca80700 1 -- 172.21.15.100:0/4031907278 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe2f054da0 msgr2=0x55fe2f0586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:18:14.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.577+0000 7f998ca80700 1 --2- 172.21.15.100:0/4031907278 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe2f054da0 0x55fe2f0586a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:14.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.577+0000 7f998ca80700 1 -- 172.21.15.100:0/4031907278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fe2f054480 msgr2=0x55fe2f054860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:18:14.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.577+0000 7f998ca80700 1 --2- 172.21.15.100:0/4031907278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fe2f054480 0x55fe2f054860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:14.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.577+0000 7f998ca80700 1 -- 172.21.15.100:0/4031907278 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe2eef3880 con 0x55fe2f058c30 2024-05-10T11:18:14.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.577+0000 7f997e7fc700 1 -- 172.21.15.100:0/3676827936 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f99800e3730 con 0x7f9988109600 2024-05-10T11:18:14.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.577+0000 7f998ca80700 1 --2- 172.21.15.100:0/4031907278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe2f058c30 0x55fe2f0590b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f997800cf90 tx=0x7f997800c1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30da7cfced1e4d8b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:14.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.577+0000 7f998da82700 1 --2- 172.21.15.100:0/3676827936 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f99740b2900 0x7f99740b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:14.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.577+0000 7f99737fe700 1 -- 172.21.15.100:0/4031907278 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f997801d070 con 0x55fe2f058c30 2024-05-10T11:18:14.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.577+0000 7f99737fe700 1 -- 172.21.15.100:0/4031907278 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9978015e10 con 0x55fe2f058c30 2024-05-10T11:18:14.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.577+0000 7f99737fe700 1 -- 172.21.15.100:0/4031907278 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f997800f040 con 0x55fe2f058c30 2024-05-10T11:18:14.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.578+0000 7f9999d78b80 1 -- 172.21.15.100:0/4031907278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe2f058c30 msgr2=0x55fe2f0590b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:14.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.578+0000 7f998da82700 1 --2- 172.21.15.100:0/3676827936 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f99740b2900 0x7f99740b4dc0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f99800e1240 tx=0x7f9980009b80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:14.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.578+0000 7f9999d78b80 1 --2- 172.21.15.100:0/4031907278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe2f058c30 0x55fe2f0590b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f997800cf90 tx=0x7f997800c1b0 comp rx=0 tx=0).stop 2024-05-10T11:18:14.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.578+0000 7f9999d78b80 1 -- 172.21.15.100:0/4031907278 shutdown_connections 2024-05-10T11:18:14.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.578+0000 7f9999d78b80 1 --2- 172.21.15.100:0/4031907278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe2f058c30 0x55fe2f0590b0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:14.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.578+0000 7f9999d78b80 1 --2- 172.21.15.100:0/4031907278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fe2f054480 0x55fe2f054860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:14.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.578+0000 7f9999d78b80 1 --2- 172.21.15.100:0/4031907278 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe2f054da0 0x55fe2f0586a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:14.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.578+0000 7f9999d78b80 1 -- 172.21.15.100:0/4031907278 >> 172.21.15.100:0/4031907278 conn(0x55fe2ef800b0 msgr2=0x55fe2f04ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:14.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.578+0000 7f9999d78b80 1 -- 172.21.15.100:0/4031907278 shutdown_connections 2024-05-10T11:18:14.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.578+0000 7f9999d78b80 1 -- 172.21.15.100:0/4031907278 wait complete. 2024-05-10T11:18:14.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.578+0000 7f9999d78b80 1 -- start start 2024-05-10T11:18:14.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.579+0000 7f9999d78b80 1 Processor -- start 2024-05-10T11:18:14.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.579+0000 7f9999d78b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe2f054480 0x55fe2f0600f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:14.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.579+0000 7f9999d78b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe2f054da0 0x55fe2f05d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:14.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.579+0000 7f9999d78b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fe2f058c30 0x55fe2f05d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:14.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.579+0000 7f9999d78b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55fe2f060e40 con 0x55fe2f054da0 2024-05-10T11:18:14.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.579+0000 7f9999d78b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55fe2ef0cbe0 con 0x55fe2f058c30 2024-05-10T11:18:14.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.579+0000 7f9999d78b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55fe2f060fc0 con 0x55fe2f054480 2024-05-10T11:18:14.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.579+0000 7f9999d78b80 10 client.0 _ll_register_callbacks cb 0x55fe2ef3e8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:18:14.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.579+0000 7f998d281700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe2f054da0 0x55fe2f05d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:14.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.579+0000 7f998d281700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe2f054da0 0x55fe2f05d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59500/0 (socket says 172.21.15.100:59500) 2024-05-10T11:18:14.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.579+0000 7f998d281700 1 -- 172.21.15.100:0/1688835985 learned_addr learned my addr 172.21.15.100:0/1688835985 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:18:14.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.579+0000 7f998ca80700 1 --2- 172.21.15.100:0/1688835985 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fe2f058c30 0x55fe2f05d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:14.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.579+0000 7f998ca80700 1 --2- 172.21.15.100:0/1688835985 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe2f054480 0x55fe2f0600f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:14.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.579+0000 7f998d281700 1 -- 172.21.15.100:0/1688835985 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe2f054480 msgr2=0x55fe2f0600f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:18:14.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.579+0000 7f998d281700 1 --2- 172.21.15.100:0/1688835985 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe2f054480 0x55fe2f0600f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:14.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.579+0000 7f998d281700 1 -- 172.21.15.100:0/1688835985 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fe2f058c30 msgr2=0x55fe2f05d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:18:14.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.579+0000 7f998d281700 1 --2- 172.21.15.100:0/1688835985 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fe2f058c30 0x55fe2f05d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:14.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.579+0000 7f998d281700 1 -- 172.21.15.100:0/1688835985 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe2ef42640 con 0x55fe2f054da0 2024-05-10T11:18:14.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.579+0000 7f998ca80700 1 --2- 172.21.15.100:0/1688835985 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe2f054480 0x55fe2f0600f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:18:14.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.580+0000 7f998d281700 1 --2- 172.21.15.100:0/1688835985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe2f054da0 0x55fe2f05d490 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9984011610 tx=0x7f99840119d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d28713a27a1b3ce9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:14.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.580+0000 7f9970ff9700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:18:14.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.580+0000 7f9970ff9700 1 -- 172.21.15.100:0/1688835985 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9984013670 con 0x55fe2f054da0 2024-05-10T11:18:14.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.580+0000 7f9970ff9700 1 -- 172.21.15.100:0/1688835985 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f9984013810 con 0x55fe2f054da0 2024-05-10T11:18:14.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.580+0000 7f9999d78b80 1 -- 172.21.15.100:0/1688835985 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9978015e10 con 0x55fe2f054da0 2024-05-10T11:18:14.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.580+0000 7f9970ff9700 1 -- 172.21.15.100:0/1688835985 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9984026050 con 0x55fe2f054da0 2024-05-10T11:18:14.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.580+0000 7f9999d78b80 10 client.15171 fetch_fsmap learned FSMap version 26 2024-05-10T11:18:14.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.580+0000 7f9999d78b80 1 -- 172.21.15.100:0/1688835985 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fe2f06b770 con 0x55fe2f054da0 2024-05-10T11:18:14.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.580+0000 7f9970ff9700 1 -- 172.21.15.100:0/1688835985 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9978015e10 con 0x55fe2f054da0 2024-05-10T11:18:14.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.580+0000 7f9970ff9700 1 -- 172.21.15.100:0/1688835985 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f9984014b20 con 0x55fe2f054da0 2024-05-10T11:18:14.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.580+0000 7f9999d78b80 10 client.15171 fetch_fsmap finished waiting for FSMap version 26 2024-05-10T11:18:14.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.581+0000 7f9999d78b80 10 client.15171 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-10T11:18:14.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.581+0000 7f9999d78b80 20 client.15171 populate_metadata read hostname 'smithi100' 2024-05-10T11:18:14.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.581+0000 7f9999d78b80 1 --2- 172.21.15.100:0/1688835985 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55fe2f06ddc0 0x55fe2f06f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:14.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.581+0000 7f9999d78b80 4 client.15171 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:18:14.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.581+0000 7f9999d78b80 1 -- 172.21.15.100:0/1688835985 --> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe2f020600 con 0x55fe2f06ddc0 2024-05-10T11:18:14.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.581+0000 7f998ca80700 1 --2- 172.21.15.100:0/1688835985 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55fe2f06ddc0 0x55fe2f06f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:14.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.581+0000 7f9970ff9700 10 client.15171 ms_handle_connect on v2:172.21.15.178:6838/3760985404 2024-05-10T11:18:14.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.581+0000 7f998ca80700 1 --2- 172.21.15.100:0/1688835985 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55fe2f06ddc0 0x55fe2f06f230 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b3aa935e38baba4d server_cookie=5cc017439b6feb65 in_seq=0 out_seq=0 2024-05-10T11:18:14.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.582+0000 7f9970ff9700 1 -- 172.21.15.100:0/1688835985 <== mds.1 v2:172.21.15.178:6838/3760985404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fe2f020600 con 0x55fe2f06ddc0 2024-05-10T11:18:14.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.582+0000 7f9970ff9700 10 client.15171 handle_command_reply: tid=0 2024-05-10T11:18:14.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.582+0000 7f9999d78b80 1 client.15171 shutdown 2024-05-10T11:18:14.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.582+0000 7f9999d78b80 1 -- 172.21.15.100:0/1688835985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe2f054da0 msgr2=0x55fe2f05d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:14.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.582+0000 7f9999d78b80 1 --2- 172.21.15.100:0/1688835985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe2f054da0 0x55fe2f05d490 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9984011610 tx=0x7f99840119d0 comp rx=0 tx=0).stop 2024-05-10T11:18:14.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.582+0000 7f9999d78b80 1 -- 172.21.15.100:0/1688835985 shutdown_connections 2024-05-10T11:18:14.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.582+0000 7f9999d78b80 1 --2- 172.21.15.100:0/1688835985 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55fe2f06ddc0 0x55fe2f06f230 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:14.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.582+0000 7f9999d78b80 1 --2- 172.21.15.100:0/1688835985 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fe2f058c30 0x55fe2f05d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:14.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.582+0000 7f9999d78b80 1 --2- 172.21.15.100:0/1688835985 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe2f054480 0x55fe2f0600f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:14.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.582+0000 7f9999d78b80 1 --2- 172.21.15.100:0/1688835985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe2f054da0 0x55fe2f05d490 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:14.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.582+0000 7f9999d78b80 1 -- 172.21.15.100:0/1688835985 >> 172.21.15.100:0/1688835985 conn(0x55fe2ef800b0 msgr2=0x55fe2f04ebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:14.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.582+0000 7f9970ff9700 0 client.15171 ms_handle_reset on v2:172.21.15.178:6838/3760985404 2024-05-10T11:18:14.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.582+0000 7f9999d78b80 1 -- 172.21.15.100:0/1688835985 shutdown_connections 2024-05-10T11:18:14.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.582+0000 7f9999d78b80 1 -- 172.21.15.100:0/1688835985 wait complete. 2024-05-10T11:18:14.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.583+0000 7f9999d78b80 20 client.15171 trim_cache size 0 max 16384 2024-05-10T11:18:14.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.603+0000 7f9999d78b80 1 Processor -- start 2024-05-10T11:18:14.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.603+0000 7f9999d78b80 1 -- start start 2024-05-10T11:18:14.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.603+0000 7f9999d78b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe2f054480 0x55fe2f048e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:14.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.603+0000 7f9999d78b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fe2f054da0 0x55fe2f049380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:14.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.603+0000 7f9999d78b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe2f058c30 0x55fe2f0498c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:14.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.603+0000 7f9999d78b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55fe2f06d570 con 0x55fe2f058c30 2024-05-10T11:18:14.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.603+0000 7f9999d78b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55fe2f06dc40 con 0x55fe2f054da0 2024-05-10T11:18:14.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.603+0000 7f9999d78b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55fe2f06e310 con 0x55fe2f054480 2024-05-10T11:18:14.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.604+0000 7f998ca80700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe2f058c30 0x55fe2f0498c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:14.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.604+0000 7f998ca80700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe2f058c30 0x55fe2f0498c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59502/0 (socket says 172.21.15.100:59502) 2024-05-10T11:18:14.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.604+0000 7f998ca80700 1 -- 172.21.15.100:0/2821096307 learned_addr learned my addr 172.21.15.100:0/2821096307 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:18:14.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.604+0000 7f998ca80700 1 --2- 172.21.15.100:0/2821096307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe2f054480 0x55fe2f048e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:14.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.604+0000 7f998d281700 1 --2- 172.21.15.100:0/2821096307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fe2f054da0 0x55fe2f049380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:14.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.604+0000 7f998ca80700 1 -- 172.21.15.100:0/2821096307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe2f054480 msgr2=0x55fe2f048e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:18:14.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.604+0000 7f998ca80700 1 --2- 172.21.15.100:0/2821096307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe2f054480 0x55fe2f048e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:14.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.604+0000 7f998ca80700 1 -- 172.21.15.100:0/2821096307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fe2f054da0 msgr2=0x55fe2f049380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:18:14.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.604+0000 7f998ca80700 1 --2- 172.21.15.100:0/2821096307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fe2f054da0 0x55fe2f049380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:14.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.604+0000 7f998ca80700 1 -- 172.21.15.100:0/2821096307 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe2f047280 con 0x55fe2f058c30 2024-05-10T11:18:14.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.604+0000 7f998ca80700 1 --2- 172.21.15.100:0/2821096307 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe2f058c30 0x55fe2f0498c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f99780048a0 tx=0x7f997800c1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ea8026595ccada6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:14.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.604+0000 7f9972ffd700 1 -- 172.21.15.100:0/2821096307 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f997801d040 con 0x55fe2f058c30 2024-05-10T11:18:14.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.604+0000 7f9972ffd700 1 -- 172.21.15.100:0/2821096307 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9978006c30 con 0x55fe2f058c30 2024-05-10T11:18:14.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.604+0000 7f9972ffd700 1 -- 172.21.15.100:0/2821096307 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9978018a50 con 0x55fe2f058c30 2024-05-10T11:18:14.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.605+0000 7f9999d78b80 1 -- 172.21.15.100:0/2821096307 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe2f058c30 msgr2=0x55fe2f0498c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:14.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.605+0000 7f9999d78b80 1 --2- 172.21.15.100:0/2821096307 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe2f058c30 0x55fe2f0498c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f99780048a0 tx=0x7f997800c1b0 comp rx=0 tx=0).stop 2024-05-10T11:18:14.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.605+0000 7f9999d78b80 1 -- 172.21.15.100:0/2821096307 shutdown_connections 2024-05-10T11:18:14.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.605+0000 7f9999d78b80 1 --2- 172.21.15.100:0/2821096307 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe2f058c30 0x55fe2f0498c0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:14.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.605+0000 7f9999d78b80 1 --2- 172.21.15.100:0/2821096307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe2f054480 0x55fe2f048e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:14.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.605+0000 7f9999d78b80 1 --2- 172.21.15.100:0/2821096307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fe2f054da0 0x55fe2f049380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:14.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.605+0000 7f9999d78b80 1 -- 172.21.15.100:0/2821096307 >> 172.21.15.100:0/2821096307 conn(0x55fe2ef800b0 msgr2=0x55fe2f05bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:14.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.605+0000 7f9999d78b80 1 -- 172.21.15.100:0/2821096307 shutdown_connections 2024-05-10T11:18:14.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.605+0000 7f9999d78b80 1 -- 172.21.15.100:0/2821096307 wait complete. 2024-05-10T11:18:14.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.605+0000 7f9999d78b80 1 -- start start 2024-05-10T11:18:14.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.606+0000 7f9999d78b80 1 Processor -- start 2024-05-10T11:18:14.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.606+0000 7f9999d78b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fe2f054480 0x55fe2f078790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:14.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.606+0000 7f9999d78b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe2f054da0 0x55fe2f078cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:14.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.606+0000 7f9999d78b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe2f058c30 0x55fe2f073d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:14.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.606+0000 7f9999d78b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55fe2f06f0b0 con 0x55fe2f054da0 2024-05-10T11:18:14.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.606+0000 7f9999d78b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55fe2f06e9e0 con 0x55fe2f054480 2024-05-10T11:18:14.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.606+0000 7f9999d78b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55fe2f06f780 con 0x55fe2f058c30 2024-05-10T11:18:14.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.606+0000 7f9999d78b80 10 client.0 _ll_register_callbacks cb 0x55fe2ef3e8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:18:14.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.606+0000 7f998d281700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe2f054da0 0x55fe2f078cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:14.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.606+0000 7f998d281700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe2f054da0 0x55fe2f078cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59506/0 (socket says 172.21.15.100:59506) 2024-05-10T11:18:14.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.606+0000 7f998d281700 1 -- 172.21.15.100:0/3136016421 learned_addr learned my addr 172.21.15.100:0/3136016421 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:18:14.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.606+0000 7f998ca80700 1 --2- 172.21.15.100:0/3136016421 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fe2f054480 0x55fe2f078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:14.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.606+0000 7f998ca80700 1 --2- 172.21.15.100:0/3136016421 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe2f058c30 0x55fe2f073d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:14.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.607+0000 7f998d281700 1 -- 172.21.15.100:0/3136016421 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe2f058c30 msgr2=0x55fe2f073d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:18:14.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.607+0000 7f998d281700 1 --2- 172.21.15.100:0/3136016421 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe2f058c30 0x55fe2f073d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:14.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.607+0000 7f998d281700 1 -- 172.21.15.100:0/3136016421 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fe2f054480 msgr2=0x55fe2f078790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:18:14.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.607+0000 7f998d281700 1 --2- 172.21.15.100:0/3136016421 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fe2f054480 0x55fe2f078790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:14.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.607+0000 7f998d281700 1 -- 172.21.15.100:0/3136016421 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe2f0605c0 con 0x55fe2f054da0 2024-05-10T11:18:14.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.607+0000 7f998ca80700 1 --2- 172.21.15.100:0/3136016421 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fe2f054480 0x55fe2f078790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:18:14.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.607+0000 7f998d281700 1 --2- 172.21.15.100:0/3136016421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe2f054da0 0x55fe2f078cd0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f9984000c00 tx=0x7f998401d850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21f720b6472d3cbf server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:14.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.607+0000 7f9970ff9700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:18:14.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.607+0000 7f9970ff9700 1 -- 172.21.15.100:0/3136016421 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f998402f020 con 0x55fe2f054da0 2024-05-10T11:18:14.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.607+0000 7f9970ff9700 1 -- 172.21.15.100:0/3136016421 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f998401dc20 con 0x55fe2f054da0 2024-05-10T11:18:14.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.607+0000 7f9999d78b80 1 -- 172.21.15.100:0/3136016421 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9978006c30 con 0x55fe2f054da0 2024-05-10T11:18:14.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.607+0000 7f9970ff9700 1 -- 172.21.15.100:0/3136016421 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f998403a050 con 0x55fe2f054da0 2024-05-10T11:18:14.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.607+0000 7f9999d78b80 10 client.15177 fetch_fsmap learned FSMap version 26 2024-05-10T11:18:14.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.607+0000 7f9999d78b80 1 -- 172.21.15.100:0/3136016421 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fe2f0d8ce0 con 0x55fe2f054da0 2024-05-10T11:18:14.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.607+0000 7f9970ff9700 1 -- 172.21.15.100:0/3136016421 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9978006c30 con 0x55fe2f054da0 2024-05-10T11:18:14.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.608+0000 7f9970ff9700 1 -- 172.21.15.100:0/3136016421 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f9984013c70 con 0x55fe2f054da0 2024-05-10T11:18:14.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.608+0000 7f9999d78b80 10 client.15177 fetch_fsmap finished waiting for FSMap version 26 2024-05-10T11:18:14.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.608+0000 7f9999d78b80 10 client.15177 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-10T11:18:14.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.608+0000 7f9999d78b80 20 client.15177 populate_metadata read hostname 'smithi100' 2024-05-10T11:18:14.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.608+0000 7f9999d78b80 1 --2- 172.21.15.100:0/3136016421 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55fe2f0dd930 0x55fe2f0dc640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:14.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.608+0000 7f9999d78b80 4 client.15177 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T11:18:14.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.608+0000 7f9999d78b80 1 -- 172.21.15.100:0/3136016421 --> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] -- command(tid 0: {"prefix": "status"}) -- 0x55fe2efb41f0 con 0x55fe2f0dd930 2024-05-10T11:18:14.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.608+0000 7f998ca80700 1 --2- 172.21.15.100:0/3136016421 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55fe2f0dd930 0x55fe2f0dc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:14.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.608+0000 7f998ca80700 1 --2- 172.21.15.100:0/3136016421 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55fe2f0dd930 0x55fe2f0dc640 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c66f8198eb901d43 server_cookie=6bb77f26891d2b5d in_seq=0 out_seq=0 2024-05-10T11:18:14.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.608+0000 7f9970ff9700 10 client.15177 ms_handle_connect on v2:172.21.15.178:6838/3760985404 2024-05-10T11:18:14.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.609+0000 7f9970ff9700 1 -- 172.21.15.100:0/3136016421 <== mds.1 v2:172.21.15.178:6838/3760985404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55fe2efb41f0 con 0x55fe2f0dd930 2024-05-10T11:18:14.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.609+0000 7f9970ff9700 10 client.15177 handle_command_reply: tid=0 2024-05-10T11:18:14.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.609+0000 7f9999d78b80 1 client.15177 shutdown 2024-05-10T11:18:14.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.609+0000 7f9999d78b80 1 -- 172.21.15.100:0/3136016421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe2f054da0 msgr2=0x55fe2f078cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:14.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.609+0000 7f9999d78b80 1 --2- 172.21.15.100:0/3136016421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe2f054da0 0x55fe2f078cd0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f9984000c00 tx=0x7f998401d850 comp rx=0 tx=0).stop 2024-05-10T11:18:14.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.609+0000 7f9999d78b80 1 -- 172.21.15.100:0/3136016421 shutdown_connections 2024-05-10T11:18:14.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.609+0000 7f9999d78b80 1 --2- 172.21.15.100:0/3136016421 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55fe2f0dd930 0x55fe2f0dc640 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:14.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.609+0000 7f9999d78b80 1 --2- 172.21.15.100:0/3136016421 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe2f058c30 0x55fe2f073d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:14.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.609+0000 7f9999d78b80 1 --2- 172.21.15.100:0/3136016421 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fe2f054480 0x55fe2f078790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:14.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.609+0000 7f9999d78b80 1 --2- 172.21.15.100:0/3136016421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe2f054da0 0x55fe2f078cd0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:14.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.609+0000 7f9999d78b80 1 -- 172.21.15.100:0/3136016421 >> 172.21.15.100:0/3136016421 conn(0x55fe2ef800b0 msgr2=0x55fe2f0496d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:14.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.609+0000 7f9970ff9700 0 client.15177 ms_handle_reset on v2:172.21.15.178:6838/3760985404 2024-05-10T11:18:14.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.609+0000 7f9999d78b80 1 -- 172.21.15.100:0/3136016421 shutdown_connections 2024-05-10T11:18:14.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.609+0000 7f9999d78b80 1 -- 172.21.15.100:0/3136016421 wait complete. 2024-05-10T11:18:14.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.609+0000 7f9999d78b80 20 client.15177 trim_cache size 0 max 16384 2024-05-10T11:18:14.650 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:18:14.650 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_fsid": "20a542cc-0ebd-11ef-bc98-c7b262605968", 2024-05-10T11:18:14.650 INFO:teuthology.orchestra.run.smithi100.stdout: "whoami": 1, 2024-05-10T11:18:14.650 INFO:teuthology.orchestra.run.smithi100.stdout: "id": 24437, 2024-05-10T11:18:14.651 INFO:teuthology.orchestra.run.smithi100.stdout: "want_state": "up:active", 2024-05-10T11:18:14.651 INFO:teuthology.orchestra.run.smithi100.stdout: "state": "up:active", 2024-05-10T11:18:14.651 INFO:teuthology.orchestra.run.smithi100.stdout: "fs_name": "cephfs", 2024-05-10T11:18:14.651 INFO:teuthology.orchestra.run.smithi100.stdout: "rank_uptime": 165.83042642500001, 2024-05-10T11:18:14.651 INFO:teuthology.orchestra.run.smithi100.stdout: "mdsmap_epoch": 26, 2024-05-10T11:18:14.651 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch": 89, 2024-05-10T11:18:14.651 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch_barrier": 89, 2024-05-10T11:18:14.651 INFO:teuthology.orchestra.run.smithi100.stdout: "uptime": 244.058798669 2024-05-10T11:18:14.651 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:18:14.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.610+0000 7f998f4e5700 1 -- 172.21.15.100:0/3676827936 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f99740b2900 msgr2=0x7f99740b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:14.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.610+0000 7f998f4e5700 1 --2- 172.21.15.100:0/3676827936 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f99740b2900 0x7f99740b4dc0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f99800e1240 tx=0x7f9980009b80 comp rx=0 tx=0).stop 2024-05-10T11:18:14.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.610+0000 7f998f4e5700 1 -- 172.21.15.100:0/3676827936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9988109600 msgr2=0x7f9988076220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:14.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.610+0000 7f998f4e5700 1 --2- 172.21.15.100:0/3676827936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9988109600 0x7f9988076220 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f99800062a0 tx=0x7f9980004dc0 comp rx=0 tx=0).stop 2024-05-10T11:18:14.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.610+0000 7f998f4e5700 1 -- 172.21.15.100:0/3676827936 shutdown_connections 2024-05-10T11:18:14.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.610+0000 7f998f4e5700 1 --2- 172.21.15.100:0/3676827936 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f99740b2900 0x7f99740b4dc0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:14.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.610+0000 7f998f4e5700 1 --2- 172.21.15.100:0/3676827936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9988109600 0x7f9988076220 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:14.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.610+0000 7f998f4e5700 1 --2- 172.21.15.100:0/3676827936 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99881034f0 0x7f99880757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:14.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.610+0000 7f998f4e5700 1 --2- 172.21.15.100:0/3676827936 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9988103eb0 0x7f9988075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:14.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.610+0000 7f998f4e5700 1 -- 172.21.15.100:0/3676827936 >> 172.21.15.100:0/3676827936 conn(0x7f998807a650 msgr2=0x7f9988102170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:14.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.610+0000 7f998f4e5700 1 -- 172.21.15.100:0/3676827936 shutdown_connections 2024-05-10T11:18:14.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:14.610+0000 7f998f4e5700 1 -- 172.21.15.100:0/3676827936 wait complete. 2024-05-10T11:18:15.419 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.i status 2024-05-10T11:18:15.662 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:18:15.783 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:18:15 smithi100 ceph-mon[30296]: pgmap v416: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-10T11:18:15.830 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:18:15 smithi188 ceph-mon[34075]: pgmap v416: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-10T11:18:16.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:18:15 smithi178 ceph-mon[33359]: pgmap v416: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-10T11:18:17.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.339+0000 7f2c87337700 1 -- 172.21.15.100:0/3724589948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c80108580 msgr2=0x7f2c80108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:17.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.339+0000 7f2c87337700 1 --2- 172.21.15.100:0/3724589948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c80108580 0x7f2c80108960 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2c74009990 tx=0x7f2c74009ca0 comp rx=0 tx=0).stop 2024-05-10T11:18:17.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.340+0000 7f2c87337700 1 -- 172.21.15.100:0/3724589948 shutdown_connections 2024-05-10T11:18:17.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.340+0000 7f2c87337700 1 --2- 172.21.15.100:0/3724589948 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c80102820 0x7f2c80110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:17.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.340+0000 7f2c87337700 1 --2- 172.21.15.100:0/3724589948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c80108580 0x7f2c80108960 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:17.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.340+0000 7f2c87337700 1 --2- 172.21.15.100:0/3724589948 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c80101e60 0x7f2c801022e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:17.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.340+0000 7f2c87337700 1 -- 172.21.15.100:0/3724589948 >> 172.21.15.100:0/3724589948 conn(0x7f2c800776c0 msgr2=0x7f2c80077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:17.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.340+0000 7f2c87337700 1 -- 172.21.15.100:0/3724589948 shutdown_connections 2024-05-10T11:18:17.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.340+0000 7f2c87337700 1 -- 172.21.15.100:0/3724589948 wait complete. 2024-05-10T11:18:17.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.340+0000 7f2c87337700 1 Processor -- start 2024-05-10T11:18:17.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.340+0000 7f2c87337700 1 -- start start 2024-05-10T11:18:17.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.340+0000 7f2c87337700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c80101e60 0x7f2c80103040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:17.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.341+0000 7f2c87337700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c80102820 0x7f2c80103580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:17.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.341+0000 7f2c87337700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c80108580 0x7f2c80103ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:17.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.341+0000 7f2c87337700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2c80113520 con 0x7f2c80108580 2024-05-10T11:18:17.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.341+0000 7f2c87337700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2c801133a0 con 0x7f2c80101e60 2024-05-10T11:18:17.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.341+0000 7f2c87337700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2c801136a0 con 0x7f2c80102820 2024-05-10T11:18:17.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.341+0000 7f2c858d4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c80108580 0x7f2c80103ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:17.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.341+0000 7f2c858d4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c80108580 0x7f2c80103ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60126/0 (socket says 172.21.15.100:60126) 2024-05-10T11:18:17.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.341+0000 7f2c858d4700 1 -- 172.21.15.100:0/3560698564 learned_addr learned my addr 172.21.15.100:0/3560698564 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:18:17.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.341+0000 7f2c848d2700 1 --2- 172.21.15.100:0/3560698564 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c80102820 0x7f2c80103580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:17.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.341+0000 7f2c850d3700 1 --2- 172.21.15.100:0/3560698564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c80101e60 0x7f2c80103040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:17.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.341+0000 7f2c858d4700 1 -- 172.21.15.100:0/3560698564 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c80102820 msgr2=0x7f2c80103580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:17.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.341+0000 7f2c858d4700 1 --2- 172.21.15.100:0/3560698564 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c80102820 0x7f2c80103580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:17.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.341+0000 7f2c858d4700 1 -- 172.21.15.100:0/3560698564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c80101e60 msgr2=0x7f2c80103040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:17.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.341+0000 7f2c858d4700 1 --2- 172.21.15.100:0/3560698564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c80101e60 0x7f2c80103040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:17.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.341+0000 7f2c858d4700 1 -- 172.21.15.100:0/3560698564 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c801a7f40 con 0x7f2c80108580 2024-05-10T11:18:17.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.341+0000 7f2c858d4700 1 --2- 172.21.15.100:0/3560698564 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c80108580 0x7f2c80103ac0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f2c78007d70 tx=0x7f2c7800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:17.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.341+0000 7f2c727fc700 1 -- 172.21.15.100:0/3560698564 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c78016020 con 0x7f2c80108580 2024-05-10T11:18:17.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.341+0000 7f2c87337700 1 -- 172.21.15.100:0/3560698564 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c801a8230 con 0x7f2c80108580 2024-05-10T11:18:17.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.341+0000 7f2c727fc700 1 -- 172.21.15.100:0/3560698564 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2c7800ac90 con 0x7f2c80108580 2024-05-10T11:18:17.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.341+0000 7f2c727fc700 1 -- 172.21.15.100:0/3560698564 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c780115e0 con 0x7f2c80108580 2024-05-10T11:18:17.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.341+0000 7f2c87337700 1 -- 172.21.15.100:0/3560698564 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c801a8720 con 0x7f2c80108580 2024-05-10T11:18:17.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.344+0000 7f2c727fc700 1 -- 172.21.15.100:0/3560698564 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f2c78019020 con 0x7f2c80108580 2024-05-10T11:18:17.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.345+0000 7f2c727fc700 1 --2- 172.21.15.100:0/3560698564 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2c6c0b2880 0x7f2c6c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:17.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.345+0000 7f2c850d3700 1 --2- 172.21.15.100:0/3560698564 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2c6c0b2880 0x7f2c6c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:17.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.345+0000 7f2c727fc700 1 -- 172.21.15.100:0/3560698564 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f2c780e2460 con 0x7f2c80108580 2024-05-10T11:18:17.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.345+0000 7f2c91bcab80 1 Processor -- start 2024-05-10T11:18:17.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.345+0000 7f2c850d3700 1 --2- 172.21.15.100:0/3560698564 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2c6c0b2880 0x7f2c6c0b4d40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2c7400b5c0 tx=0x7f2c74005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:17.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.345+0000 7f2c91bcab80 1 -- start start 2024-05-10T11:18:17.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.345+0000 7f2c91bcab80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558631ae6160 0x558631ae6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:17.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.345+0000 7f2c91bcab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558631ae6a80 0x558631ae07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:17.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.345+0000 7f2c91bcab80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558631ae0d00 0x558631ae1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:17.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.345+0000 7f2c91bcab80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x558631a328f0 con 0x558631ae0d00 2024-05-10T11:18:17.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.345+0000 7f2c91bcab80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558631997d50 con 0x558631ae6160 2024-05-10T11:18:17.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.345+0000 7f2c91bcab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558631a2e8b0 con 0x558631ae6a80 2024-05-10T11:18:17.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.346+0000 7f2c858d4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558631ae0d00 0x558631ae1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:17.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.346+0000 7f2c858d4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558631ae0d00 0x558631ae1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60136/0 (socket says 172.21.15.100:60136) 2024-05-10T11:18:17.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.346+0000 7f2c858d4700 1 -- 172.21.15.100:0/3370031142 learned_addr learned my addr 172.21.15.100:0/3370031142 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:18:17.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.346+0000 7f2c850d3700 1 --2- 172.21.15.100:0/3370031142 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558631ae6160 0x558631ae6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:17.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.346+0000 7f2c848d2700 1 --2- 172.21.15.100:0/3370031142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558631ae6a80 0x558631ae07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:17.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.346+0000 7f2c858d4700 1 -- 172.21.15.100:0/3370031142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558631ae6a80 msgr2=0x558631ae07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:18:17.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.346+0000 7f2c858d4700 1 --2- 172.21.15.100:0/3370031142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558631ae6a80 0x558631ae07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:17.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.346+0000 7f2c858d4700 1 -- 172.21.15.100:0/3370031142 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558631ae6160 msgr2=0x558631ae6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:18:17.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.346+0000 7f2c858d4700 1 --2- 172.21.15.100:0/3370031142 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558631ae6160 0x558631ae6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:17.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.346+0000 7f2c858d4700 1 -- 172.21.15.100:0/3370031142 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55863197f880 con 0x558631ae0d00 2024-05-10T11:18:17.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.346+0000 7f2c858d4700 1 --2- 172.21.15.100:0/3370031142 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558631ae0d00 0x558631ae1140 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f2c78007d40 tx=0x7f2c78013f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b157ec3ec1514a50 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:17.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.346+0000 7f2c6b7fe700 1 -- 172.21.15.100:0/3370031142 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c780a9e20 con 0x558631ae0d00 2024-05-10T11:18:17.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.346+0000 7f2c6b7fe700 1 -- 172.21.15.100:0/3370031142 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2c780b0e50 con 0x558631ae0d00 2024-05-10T11:18:17.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.346+0000 7f2c6b7fe700 1 -- 172.21.15.100:0/3370031142 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c780910f0 con 0x558631ae0d00 2024-05-10T11:18:17.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.347+0000 7f2c91bcab80 1 -- 172.21.15.100:0/3370031142 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558631ae0d00 msgr2=0x558631ae1140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:17.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.347+0000 7f2c91bcab80 1 --2- 172.21.15.100:0/3370031142 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558631ae0d00 0x558631ae1140 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f2c78007d40 tx=0x7f2c78013f50 comp rx=0 tx=0).stop 2024-05-10T11:18:17.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.347+0000 7f2c91bcab80 1 -- 172.21.15.100:0/3370031142 shutdown_connections 2024-05-10T11:18:17.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.347+0000 7f2c91bcab80 1 --2- 172.21.15.100:0/3370031142 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558631ae0d00 0x558631ae1140 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:17.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.347+0000 7f2c91bcab80 1 --2- 172.21.15.100:0/3370031142 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558631ae6160 0x558631ae6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:17.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.347+0000 7f2c91bcab80 1 --2- 172.21.15.100:0/3370031142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558631ae6a80 0x558631ae07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:17.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.347+0000 7f2c91bcab80 1 -- 172.21.15.100:0/3370031142 >> 172.21.15.100:0/3370031142 conn(0x558631a0cb20 msgr2=0x558631adb250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:17.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.347+0000 7f2c91bcab80 1 -- 172.21.15.100:0/3370031142 shutdown_connections 2024-05-10T11:18:17.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.347+0000 7f2c91bcab80 1 -- 172.21.15.100:0/3370031142 wait complete. 2024-05-10T11:18:17.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.347+0000 7f2c91bcab80 1 -- start start 2024-05-10T11:18:17.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.347+0000 7f2c91bcab80 1 Processor -- start 2024-05-10T11:18:17.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.348+0000 7f2c91bcab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558631ae0d00 0x558631aec780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:17.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.348+0000 7f2c91bcab80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558631ae6160 0x558631ae9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:17.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.348+0000 7f2c91bcab80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558631ae6a80 0x558631aea060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:17.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.348+0000 7f2c91bcab80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x558631aed400 con 0x558631ae6a80 2024-05-10T11:18:17.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.348+0000 7f2c91bcab80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5586319ebe80 con 0x558631ae6160 2024-05-10T11:18:17.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.348+0000 7f2c91bcab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558631aed580 con 0x558631ae0d00 2024-05-10T11:18:17.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.348+0000 7f2c91bcab80 10 client.0 _ll_register_callbacks cb 0x5586319c6560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:18:17.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.348+0000 7f2c858d4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558631ae6a80 0x558631aea060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:17.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.348+0000 7f2c858d4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558631ae6a80 0x558631aea060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60138/0 (socket says 172.21.15.100:60138) 2024-05-10T11:18:17.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.348+0000 7f2c858d4700 1 -- 172.21.15.100:0/1221850603 learned_addr learned my addr 172.21.15.100:0/1221850603 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:18:17.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.348+0000 7f2c850d3700 1 --2- 172.21.15.100:0/1221850603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558631ae0d00 0x558631aec780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:17.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.348+0000 7f2c848d2700 1 --2- 172.21.15.100:0/1221850603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558631ae6160 0x558631ae9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:17.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.348+0000 7f2c858d4700 1 -- 172.21.15.100:0/1221850603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558631ae0d00 msgr2=0x558631aec780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:18:17.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.348+0000 7f2c858d4700 1 --2- 172.21.15.100:0/1221850603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558631ae0d00 0x558631aec780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:17.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.348+0000 7f2c858d4700 1 -- 172.21.15.100:0/1221850603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558631ae6160 msgr2=0x558631ae9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:18:17.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.348+0000 7f2c858d4700 1 --2- 172.21.15.100:0/1221850603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558631ae6160 0x558631ae9b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:17.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.348+0000 7f2c858d4700 1 -- 172.21.15.100:0/1221850603 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586319e9710 con 0x558631ae6a80 2024-05-10T11:18:17.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.348+0000 7f2c858d4700 1 --2- 172.21.15.100:0/1221850603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558631ae6a80 0x558631aea060 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f2c780b2050 tx=0x7f2c780e23f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30485d437ca534d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:17.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.348+0000 7f2c68ff9700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:18:17.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.349+0000 7f2c68ff9700 1 -- 172.21.15.100:0/1221850603 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c780b2580 con 0x558631ae6a80 2024-05-10T11:18:17.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.349+0000 7f2c68ff9700 1 -- 172.21.15.100:0/1221850603 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f2c780912d0 con 0x558631ae6a80 2024-05-10T11:18:17.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.349+0000 7f2c91bcab80 1 -- 172.21.15.100:0/1221850603 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c780b0e50 con 0x558631ae6a80 2024-05-10T11:18:17.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.349+0000 7f2c68ff9700 1 -- 172.21.15.100:0/1221850603 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2c780a9e20 con 0x558631ae6a80 2024-05-10T11:18:17.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.349+0000 7f2c91bcab80 10 client.15189 fetch_fsmap learned FSMap version 26 2024-05-10T11:18:17.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.349+0000 7f2c91bcab80 1 -- 172.21.15.100:0/1221850603 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558631aed400 con 0x558631ae6a80 2024-05-10T11:18:17.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.349+0000 7f2c68ff9700 1 -- 172.21.15.100:0/1221850603 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2c780b0e50 con 0x558631ae6a80 2024-05-10T11:18:17.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.349+0000 7f2c68ff9700 1 -- 172.21.15.100:0/1221850603 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f2c780a5090 con 0x558631ae6a80 2024-05-10T11:18:17.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.349+0000 7f2c91bcab80 10 client.15189 fetch_fsmap finished waiting for FSMap version 26 2024-05-10T11:18:17.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.349+0000 7f2c91bcab80 10 client.15189 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-10T11:18:17.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.349+0000 7f2c91bcab80 20 client.15189 populate_metadata read hostname 'smithi100' 2024-05-10T11:18:17.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.349+0000 7f2c91bcab80 1 --2- 172.21.15.100:0/1221850603 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558631af9eb0 0x558631afb320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:17.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.349+0000 7f2c91bcab80 4 client.15189 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:18:17.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.349+0000 7f2c91bcab80 1 -- 172.21.15.100:0/1221850603 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558631a402d0 con 0x558631af9eb0 2024-05-10T11:18:17.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.349+0000 7f2c850d3700 1 --2- 172.21.15.100:0/1221850603 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558631af9eb0 0x558631afb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:17.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.350+0000 7f2c68ff9700 10 client.15189 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:18:17.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.350+0000 7f2c850d3700 1 --2- 172.21.15.100:0/1221850603 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558631af9eb0 0x558631afb320 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb90a25a9ae7016d server_cookie=e39be1980f83b98b in_seq=0 out_seq=0 2024-05-10T11:18:17.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.350+0000 7f2c68ff9700 1 -- 172.21.15.100:0/1221850603 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558631a402d0 con 0x558631af9eb0 2024-05-10T11:18:17.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.350+0000 7f2c68ff9700 10 client.15189 handle_command_reply: tid=0 2024-05-10T11:18:17.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.351+0000 7f2c91bcab80 1 client.15189 shutdown 2024-05-10T11:18:17.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.351+0000 7f2c91bcab80 1 -- 172.21.15.100:0/1221850603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558631ae6a80 msgr2=0x558631aea060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:17.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.351+0000 7f2c91bcab80 1 --2- 172.21.15.100:0/1221850603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558631ae6a80 0x558631aea060 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f2c780b2050 tx=0x7f2c780e23f0 comp rx=0 tx=0).stop 2024-05-10T11:18:17.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.351+0000 7f2c91bcab80 1 -- 172.21.15.100:0/1221850603 shutdown_connections 2024-05-10T11:18:17.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.351+0000 7f2c91bcab80 1 --2- 172.21.15.100:0/1221850603 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558631af9eb0 0x558631afb320 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:17.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.351+0000 7f2c91bcab80 1 --2- 172.21.15.100:0/1221850603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558631ae6a80 0x558631aea060 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:17.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.351+0000 7f2c91bcab80 1 --2- 172.21.15.100:0/1221850603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558631ae0d00 0x558631aec780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:17.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.351+0000 7f2c68ff9700 0 client.15189 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:18:17.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.351+0000 7f2c91bcab80 1 --2- 172.21.15.100:0/1221850603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558631ae6160 0x558631ae9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:17.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.351+0000 7f2c91bcab80 1 -- 172.21.15.100:0/1221850603 >> 172.21.15.100:0/1221850603 conn(0x558631a0cb20 msgr2=0x558631ad5ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:17.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.351+0000 7f2c91bcab80 1 -- 172.21.15.100:0/1221850603 shutdown_connections 2024-05-10T11:18:17.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.351+0000 7f2c91bcab80 1 -- 172.21.15.100:0/1221850603 wait complete. 2024-05-10T11:18:17.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.351+0000 7f2c91bcab80 20 client.15189 trim_cache size 0 max 16384 2024-05-10T11:18:17.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.372+0000 7f2c91bcab80 1 Processor -- start 2024-05-10T11:18:17.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.372+0000 7f2c91bcab80 1 -- start start 2024-05-10T11:18:17.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.372+0000 7f2c91bcab80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558631ae0d00 0x558631ae07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:17.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.372+0000 7f2c91bcab80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558631ae6160 0x558631af97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:17.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.372+0000 7f2c91bcab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558631ae6a80 0x558631aea090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:17.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.372+0000 7f2c91bcab80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x558631af9660 con 0x558631ae0d00 2024-05-10T11:18:17.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.372+0000 7f2c91bcab80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558631af9d30 con 0x558631ae6160 2024-05-10T11:18:17.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.372+0000 7f2c91bcab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558631afa400 con 0x558631ae6a80 2024-05-10T11:18:17.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.372+0000 7f2c850d3700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558631ae0d00 0x558631ae07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:17.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.372+0000 7f2c850d3700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558631ae0d00 0x558631ae07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60146/0 (socket says 172.21.15.100:60146) 2024-05-10T11:18:17.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.372+0000 7f2c850d3700 1 -- 172.21.15.100:0/642515440 learned_addr learned my addr 172.21.15.100:0/642515440 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:18:17.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.372+0000 7f2c848d2700 1 --2- 172.21.15.100:0/642515440 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558631ae6160 0x558631af97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:17.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.372+0000 7f2c858d4700 1 --2- 172.21.15.100:0/642515440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558631ae6a80 0x558631aea090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:17.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.372+0000 7f2c850d3700 1 -- 172.21.15.100:0/642515440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558631ae6a80 msgr2=0x558631aea090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:18:17.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.372+0000 7f2c850d3700 1 --2- 172.21.15.100:0/642515440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558631ae6a80 0x558631aea090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:17.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.372+0000 7f2c850d3700 1 -- 172.21.15.100:0/642515440 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558631ae6160 msgr2=0x558631af97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:18:17.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.372+0000 7f2c850d3700 1 --2- 172.21.15.100:0/642515440 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558631ae6160 0x558631af97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:17.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.372+0000 7f2c850d3700 1 -- 172.21.15.100:0/642515440 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558631ae8090 con 0x558631ae0d00 2024-05-10T11:18:17.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.372+0000 7f2c858d4700 1 --2- 172.21.15.100:0/642515440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558631ae6a80 0x558631aea090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:18:17.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.372+0000 7f2c848d2700 1 --2- 172.21.15.100:0/642515440 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558631ae6160 0x558631af97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:18:17.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.373+0000 7f2c850d3700 1 --2- 172.21.15.100:0/642515440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558631ae0d00 0x558631ae07c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f2c74009990 tx=0x7f2c7401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7518dbec11bb1693 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:17.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.373+0000 7f2c6affd700 1 -- 172.21.15.100:0/642515440 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c74038070 con 0x558631ae0d00 2024-05-10T11:18:17.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.373+0000 7f2c6affd700 1 -- 172.21.15.100:0/642515440 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2c7400fcd0 con 0x558631ae0d00 2024-05-10T11:18:17.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.373+0000 7f2c6affd700 1 -- 172.21.15.100:0/642515440 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c740348a0 con 0x558631ae0d00 2024-05-10T11:18:17.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.373+0000 7f2c91bcab80 1 -- 172.21.15.100:0/642515440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558631ae0d00 msgr2=0x558631ae07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:17.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.373+0000 7f2c91bcab80 1 --2- 172.21.15.100:0/642515440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558631ae0d00 0x558631ae07c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f2c74009990 tx=0x7f2c7401f000 comp rx=0 tx=0).stop 2024-05-10T11:18:17.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.373+0000 7f2c91bcab80 1 -- 172.21.15.100:0/642515440 shutdown_connections 2024-05-10T11:18:17.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.373+0000 7f2c91bcab80 1 --2- 172.21.15.100:0/642515440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558631ae6a80 0x558631aea090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:17.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.373+0000 7f2c91bcab80 1 --2- 172.21.15.100:0/642515440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558631ae0d00 0x558631ae07c0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:17.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.373+0000 7f2c91bcab80 1 --2- 172.21.15.100:0/642515440 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558631ae6160 0x558631af97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:17.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.373+0000 7f2c91bcab80 1 -- 172.21.15.100:0/642515440 >> 172.21.15.100:0/642515440 conn(0x558631a0cb20 msgr2=0x558631ae6e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:17.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.373+0000 7f2c91bcab80 1 -- 172.21.15.100:0/642515440 shutdown_connections 2024-05-10T11:18:17.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.374+0000 7f2c91bcab80 1 -- 172.21.15.100:0/642515440 wait complete. 2024-05-10T11:18:17.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.374+0000 7f2c91bcab80 1 -- start start 2024-05-10T11:18:17.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.374+0000 7f2c91bcab80 1 Processor -- start 2024-05-10T11:18:17.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.374+0000 7f2c91bcab80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558631ae0d00 0x558631b04b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:17.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.374+0000 7f2c91bcab80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558631ae6160 0x558631ad4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:17.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.374+0000 7f2c91bcab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558631ae6a80 0x558631ad5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:17.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.374+0000 7f2c91bcab80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x558631afb1a0 con 0x558631ae0d00 2024-05-10T11:18:17.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.374+0000 7f2c91bcab80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558631afaad0 con 0x558631ae6160 2024-05-10T11:18:17.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.374+0000 7f2c91bcab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558631afb870 con 0x558631ae6a80 2024-05-10T11:18:17.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.374+0000 7f2c91bcab80 10 client.0 _ll_register_callbacks cb 0x5586319c6560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:18:17.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.374+0000 7f2c850d3700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558631ae0d00 0x558631b04b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:17.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.374+0000 7f2c850d3700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558631ae0d00 0x558631b04b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60150/0 (socket says 172.21.15.100:60150) 2024-05-10T11:18:17.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.374+0000 7f2c850d3700 1 -- 172.21.15.100:0/1083884498 learned_addr learned my addr 172.21.15.100:0/1083884498 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:18:17.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.375+0000 7f2c858d4700 1 --2- 172.21.15.100:0/1083884498 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558631ae6a80 0x558631ad5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:17.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.375+0000 7f2c848d2700 1 --2- 172.21.15.100:0/1083884498 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558631ae6160 0x558631ad4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:17.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.375+0000 7f2c850d3700 1 -- 172.21.15.100:0/1083884498 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558631ae6a80 msgr2=0x558631ad5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:18:17.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.375+0000 7f2c850d3700 1 --2- 172.21.15.100:0/1083884498 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558631ae6a80 0x558631ad5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:17.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.375+0000 7f2c850d3700 1 -- 172.21.15.100:0/1083884498 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558631ae6160 msgr2=0x558631ad4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:18:17.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.375+0000 7f2c850d3700 1 --2- 172.21.15.100:0/1083884498 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558631ae6160 0x558631ad4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:17.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.375+0000 7f2c850d3700 1 -- 172.21.15.100:0/1083884498 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558631ad5810 con 0x558631ae0d00 2024-05-10T11:18:17.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.375+0000 7f2c850d3700 1 --2- 172.21.15.100:0/1083884498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558631ae0d00 0x558631b04b50 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2c7400ff50 tx=0x7f2c7400bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdefa0b88686b571 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:17.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.375+0000 7f2c68ff9700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:18:17.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.375+0000 7f2c68ff9700 1 -- 172.21.15.100:0/1083884498 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c74038070 con 0x558631ae0d00 2024-05-10T11:18:17.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.375+0000 7f2c68ff9700 1 -- 172.21.15.100:0/1083884498 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f2c740340f0 con 0x558631ae0d00 2024-05-10T11:18:17.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.375+0000 7f2c91bcab80 1 -- 172.21.15.100:0/1083884498 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c7400fcd0 con 0x558631ae0d00 2024-05-10T11:18:17.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.375+0000 7f2c68ff9700 1 -- 172.21.15.100:0/1083884498 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2c7404c050 con 0x558631ae0d00 2024-05-10T11:18:17.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.375+0000 7f2c91bcab80 10 client.15195 fetch_fsmap learned FSMap version 26 2024-05-10T11:18:17.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.375+0000 7f2c91bcab80 1 -- 172.21.15.100:0/1083884498 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558631b65320 con 0x558631ae0d00 2024-05-10T11:18:17.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.376+0000 7f2c68ff9700 1 -- 172.21.15.100:0/1083884498 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2c7400fcd0 con 0x558631ae0d00 2024-05-10T11:18:17.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.376+0000 7f2c68ff9700 1 -- 172.21.15.100:0/1083884498 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f2c7403d6e0 con 0x558631ae0d00 2024-05-10T11:18:17.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.376+0000 7f2c91bcab80 10 client.15195 fetch_fsmap finished waiting for FSMap version 26 2024-05-10T11:18:17.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.376+0000 7f2c91bcab80 10 client.15195 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-10T11:18:17.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.376+0000 7f2c91bcab80 20 client.15195 populate_metadata read hostname 'smithi100' 2024-05-10T11:18:17.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.376+0000 7f2c91bcab80 1 --2- 172.21.15.100:0/1083884498 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558631af9eb0 0x558631b684e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:17.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.376+0000 7f2c91bcab80 4 client.15195 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T11:18:17.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.376+0000 7f2c91bcab80 1 -- 172.21.15.100:0/1083884498 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "status"}) -- 0x558631a3a370 con 0x558631af9eb0 2024-05-10T11:18:17.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.376+0000 7f2c848d2700 1 --2- 172.21.15.100:0/1083884498 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558631af9eb0 0x558631b684e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:17.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.377+0000 7f2c848d2700 1 --2- 172.21.15.100:0/1083884498 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558631af9eb0 0x558631b684e0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab215d6d0a76853c server_cookie=ee4a15e94aa4988f in_seq=0 out_seq=0 2024-05-10T11:18:17.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.377+0000 7f2c68ff9700 10 client.15195 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:18:17.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.377+0000 7f2c68ff9700 1 -- 172.21.15.100:0/1083884498 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x558631a3a370 con 0x558631af9eb0 2024-05-10T11:18:17.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.377+0000 7f2c68ff9700 10 client.15195 handle_command_reply: tid=0 2024-05-10T11:18:17.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.377+0000 7f2c91bcab80 1 client.15195 shutdown 2024-05-10T11:18:17.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.377+0000 7f2c91bcab80 1 -- 172.21.15.100:0/1083884498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558631ae0d00 msgr2=0x558631b04b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:17.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.377+0000 7f2c91bcab80 1 --2- 172.21.15.100:0/1083884498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558631ae0d00 0x558631b04b50 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2c7400ff50 tx=0x7f2c7400bee0 comp rx=0 tx=0).stop 2024-05-10T11:18:17.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.377+0000 7f2c91bcab80 1 -- 172.21.15.100:0/1083884498 shutdown_connections 2024-05-10T11:18:17.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.377+0000 7f2c91bcab80 1 --2- 172.21.15.100:0/1083884498 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558631af9eb0 0x558631b684e0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:17.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.377+0000 7f2c91bcab80 1 --2- 172.21.15.100:0/1083884498 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558631ae6a80 0x558631ad5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:17.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.377+0000 7f2c91bcab80 1 --2- 172.21.15.100:0/1083884498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558631ae0d00 0x558631b04b50 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:17.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.377+0000 7f2c91bcab80 1 --2- 172.21.15.100:0/1083884498 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558631ae6160 0x558631ad4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:17.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.377+0000 7f2c91bcab80 1 -- 172.21.15.100:0/1083884498 >> 172.21.15.100:0/1083884498 conn(0x558631a0cb20 msgr2=0x558631ae4c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:17.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.377+0000 7f2c68ff9700 0 client.15195 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:18:17.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.377+0000 7f2c91bcab80 1 -- 172.21.15.100:0/1083884498 shutdown_connections 2024-05-10T11:18:17.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.377+0000 7f2c91bcab80 1 -- 172.21.15.100:0/1083884498 wait complete. 2024-05-10T11:18:17.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.377+0000 7f2c91bcab80 20 client.15195 trim_cache size 0 max 16384 2024-05-10T11:18:17.384 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:18:17.384 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_fsid": "20a542cc-0ebd-11ef-bc98-c7b262605968", 2024-05-10T11:18:17.384 INFO:teuthology.orchestra.run.smithi100.stdout: "whoami": 0, 2024-05-10T11:18:17.384 INFO:teuthology.orchestra.run.smithi100.stdout: "id": 24439, 2024-05-10T11:18:17.384 INFO:teuthology.orchestra.run.smithi100.stdout: "want_state": "up:active", 2024-05-10T11:18:17.384 INFO:teuthology.orchestra.run.smithi100.stdout: "state": "up:active", 2024-05-10T11:18:17.384 INFO:teuthology.orchestra.run.smithi100.stdout: "fs_name": "cephfs", 2024-05-10T11:18:17.384 INFO:teuthology.orchestra.run.smithi100.stdout: "rank_uptime": 230.10421983099999, 2024-05-10T11:18:17.384 INFO:teuthology.orchestra.run.smithi100.stdout: "mdsmap_epoch": 26, 2024-05-10T11:18:17.384 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch": 89, 2024-05-10T11:18:17.384 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch_barrier": 89, 2024-05-10T11:18:17.384 INFO:teuthology.orchestra.run.smithi100.stdout: "uptime": 257.86214900599998 2024-05-10T11:18:17.385 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:18:17.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.378+0000 7f2c87337700 1 -- 172.21.15.100:0/3560698564 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2c6c0b2880 msgr2=0x7f2c6c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:17.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.378+0000 7f2c87337700 1 --2- 172.21.15.100:0/3560698564 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2c6c0b2880 0x7f2c6c0b4d40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2c7400b5c0 tx=0x7f2c74005960 comp rx=0 tx=0).stop 2024-05-10T11:18:17.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.378+0000 7f2c87337700 1 -- 172.21.15.100:0/3560698564 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c80108580 msgr2=0x7f2c80103ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:17.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.378+0000 7f2c87337700 1 --2- 172.21.15.100:0/3560698564 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c80108580 0x7f2c80103ac0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f2c78007d70 tx=0x7f2c7800a340 comp rx=0 tx=0).stop 2024-05-10T11:18:17.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.378+0000 7f2c87337700 1 -- 172.21.15.100:0/3560698564 shutdown_connections 2024-05-10T11:18:17.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.378+0000 7f2c87337700 1 --2- 172.21.15.100:0/3560698564 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2c6c0b2880 0x7f2c6c0b4d40 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:17.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.378+0000 7f2c87337700 1 --2- 172.21.15.100:0/3560698564 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c80108580 0x7f2c80103ac0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:17.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.378+0000 7f2c87337700 1 --2- 172.21.15.100:0/3560698564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c80101e60 0x7f2c80103040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:17.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.378+0000 7f2c87337700 1 --2- 172.21.15.100:0/3560698564 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c80102820 0x7f2c80103580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:17.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.378+0000 7f2c87337700 1 -- 172.21.15.100:0/3560698564 >> 172.21.15.100:0/3560698564 conn(0x7f2c800776c0 msgr2=0x7f2c801009d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:17.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.378+0000 7f2c87337700 1 -- 172.21.15.100:0/3560698564 shutdown_connections 2024-05-10T11:18:17.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:17.378+0000 7f2c87337700 1 -- 172.21.15.100:0/3560698564 wait complete. 2024-05-10T11:18:17.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:18:17 smithi100 ceph-mon[30296]: pgmap v417: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T11:18:18.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:18:17 smithi178 ceph-mon[33359]: pgmap v417: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T11:18:18.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:18:17 smithi188 ceph-mon[34075]: pgmap v417: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T11:18:18.152 INFO:teuthology.run_tasks:Running task sequential... 2024-05-10T11:18:18.162 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-10T11:18:18.162 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-10T11:18:18.163 INFO:teuthology.task.print:Enabling mgr modules 2024-05-10T11:18:18.163 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-05-10T11:18:18.203 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-05-10T11:18:18.203 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-05-10T11:18:18.203 INFO:tasks.ceph_fuse:client.0 config is {'mount_subvol_num': 0} 2024-05-10T11:18:18.203 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-10T11:18:18.203 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-10T11:18:18.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T11:18:18.203 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-05-10T11:18:18.220 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-05-10T11:18:18.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T11:18:18.221 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-10T11:18:18.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T11:18:18.289 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link delete ceph-brx 2024-05-10T11:18:18.393 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-05-10T11:18:18.393 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph fs ls 2024-05-10T11:18:18.637 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:18:19.790 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:18:19 smithi100 ceph-mon[30296]: pgmap v418: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T11:18:20.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:18:19 smithi178 ceph-mon[33359]: pgmap v418: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T11:18:20.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:18:19 smithi188 ceph-mon[34075]: pgmap v418: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-10T11:18:20.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:20.489+0000 7f4dd61ba700 1 -- 172.21.15.100:0/2819007659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4dd0106f40 msgr2=0x7f4dd010ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:20.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:20.489+0000 7f4dd61ba700 1 --2- 172.21.15.100:0/2819007659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4dd0106f40 0x7f4dd010ea80 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0009990 tx=0x7f4dc0009ca0 comp rx=0 tx=0).stop 2024-05-10T11:18:20.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:20.489+0000 7f4dd61ba700 1 -- 172.21.15.100:0/2819007659 shutdown_connections 2024-05-10T11:18:20.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:20.489+0000 7f4dd61ba700 1 --2- 172.21.15.100:0/2819007659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4dd0106f40 0x7f4dd010ea80 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:20.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:20.489+0000 7f4dd61ba700 1 --2- 172.21.15.100:0/2819007659 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4dd01020a0 0x7f4dd0102480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:20.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:20.489+0000 7f4dd61ba700 1 --2- 172.21.15.100:0/2819007659 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4dd0102a50 0x7f4dd0106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:20.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:20.489+0000 7f4dd61ba700 1 -- 172.21.15.100:0/2819007659 >> 172.21.15.100:0/2819007659 conn(0x7f4dd00776c0 msgr2=0x7f4dd0077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:20.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:20.489+0000 7f4dd61ba700 1 -- 172.21.15.100:0/2819007659 shutdown_connections 2024-05-10T11:18:20.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:20.489+0000 7f4dd61ba700 1 -- 172.21.15.100:0/2819007659 wait complete. 2024-05-10T11:18:20.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:20.490+0000 7f4dd61ba700 1 Processor -- start 2024-05-10T11:18:20.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:20.490+0000 7f4dd61ba700 1 -- start start 2024-05-10T11:18:20.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:20.491+0000 7f4dd61ba700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4dd01020a0 0x7f4dd019f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:20.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:20.491+0000 7f4dd61ba700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4dd0102a50 0x7f4dd019fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:20.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:20.491+0000 7f4dd61ba700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4dd0106f40 0x7f4dd0199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:20.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:20.491+0000 7f4dd61ba700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd0111420 con 0x7f4dd0106f40 2024-05-10T11:18:20.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:20.491+0000 7f4dd61ba700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd01112a0 con 0x7f4dd01020a0 2024-05-10T11:18:20.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:20.491+0000 7f4dd61ba700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd01115a0 con 0x7f4dd0102a50 2024-05-10T11:18:20.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:20.491+0000 7f4dcffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4dd0106f40 0x7f4dd0199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:20.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:20.491+0000 7f4dcf7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4dd01020a0 0x7f4dd019f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:20.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:20.491+0000 7f4dcffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4dd0106f40 0x7f4dd0199980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60180/0 (socket says 172.21.15.100:60180) 2024-05-10T11:18:20.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:20.491+0000 7f4dcffff700 1 -- 172.21.15.100:0/391154265 learned_addr learned my addr 172.21.15.100:0/391154265 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:18:20.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:20.491+0000 7f4dceffd700 1 --2- 172.21.15.100:0/391154265 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4dd0102a50 0x7f4dd019fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:20.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:20.491+0000 7f4dcffff700 1 -- 172.21.15.100:0/391154265 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4dd0102a50 msgr2=0x7f4dd019fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:20.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:20.491+0000 7f4dcffff700 1 --2- 172.21.15.100:0/391154265 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4dd0102a50 0x7f4dd019fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:20.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:20.491+0000 7f4dcffff700 1 -- 172.21.15.100:0/391154265 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4dd01020a0 msgr2=0x7f4dd019f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:20.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:20.491+0000 7f4dcffff700 1 --2- 172.21.15.100:0/391154265 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4dd01020a0 0x7f4dd019f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:20.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:20.491+0000 7f4dcffff700 1 -- 172.21.15.100:0/391154265 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dd019a0e0 con 0x7f4dd0106f40 2024-05-10T11:18:20.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:20.491+0000 7f4dcffff700 1 --2- 172.21.15.100:0/391154265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4dd0106f40 0x7f4dd0199980 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4dc00047b0 tx=0x7f4dc0005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:20.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:20.491+0000 7f4dccff9700 1 -- 172.21.15.100:0/391154265 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc0004830 con 0x7f4dd0106f40 2024-05-10T11:18:20.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:20.492+0000 7f4dd61ba700 1 -- 172.21.15.100:0/391154265 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dd019a370 con 0x7f4dd0106f40 2024-05-10T11:18:20.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:20.492+0000 7f4dccff9700 1 -- 172.21.15.100:0/391154265 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4dc0005020 con 0x7f4dd0106f40 2024-05-10T11:18:20.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:20.492+0000 7f4dccff9700 1 -- 172.21.15.100:0/391154265 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc000f8d0 con 0x7f4dd0106f40 2024-05-10T11:18:20.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:20.492+0000 7f4dd61ba700 1 -- 172.21.15.100:0/391154265 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dd01a6690 con 0x7f4dd0106f40 2024-05-10T11:18:20.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:20.492+0000 7f4dd61ba700 1 -- 172.21.15.100:0/391154265 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dd0103280 con 0x7f4dd0106f40 2024-05-10T11:18:20.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:20.495+0000 7f4dccff9700 1 -- 172.21.15.100:0/391154265 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f4dc00054d0 con 0x7f4dd0106f40 2024-05-10T11:18:20.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:20.495+0000 7f4dccff9700 1 --2- 172.21.15.100:0/391154265 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4dbc0b2880 0x7f4dbc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:20.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:20.496+0000 7f4dcf7fe700 1 --2- 172.21.15.100:0/391154265 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4dbc0b2880 0x7f4dbc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:20.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:20.496+0000 7f4dccff9700 1 -- 172.21.15.100:0/391154265 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f4dc00e3400 con 0x7f4dd0106f40 2024-05-10T11:18:20.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:20.496+0000 7f4dcf7fe700 1 --2- 172.21.15.100:0/391154265 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4dbc0b2880 0x7f4dbc0b4d40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4dd019aac0 tx=0x7f4dc800d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:20.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:20.499+0000 7f4dccff9700 1 -- 172.21.15.100:0/391154265 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f4dc0013070 con 0x7f4dd0106f40 2024-05-10T11:18:20.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:20.750+0000 7f4dd61ba700 1 -- 172.21.15.100:0/391154265 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f4dd019aeb0 con 0x7f4dd0106f40 2024-05-10T11:18:20.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:20.751+0000 7f4dccff9700 1 -- 172.21.15.100:0/391154265 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+88 (secure 0 0 0) 0x7f4dc00aae50 con 0x7f4dd0106f40 2024-05-10T11:18:20.751 INFO:teuthology.orchestra.run.smithi100.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-10T11:18:20.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:20.753+0000 7f4dd61ba700 1 -- 172.21.15.100:0/391154265 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4dbc0b2880 msgr2=0x7f4dbc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:20.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:20.753+0000 7f4dd61ba700 1 --2- 172.21.15.100:0/391154265 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4dbc0b2880 0x7f4dbc0b4d40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4dd019aac0 tx=0x7f4dc800d040 comp rx=0 tx=0).stop 2024-05-10T11:18:20.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:20.753+0000 7f4dd61ba700 1 -- 172.21.15.100:0/391154265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4dd0106f40 msgr2=0x7f4dd0199980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:20.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:20.753+0000 7f4dd61ba700 1 --2- 172.21.15.100:0/391154265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4dd0106f40 0x7f4dd0199980 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4dc00047b0 tx=0x7f4dc0005ef0 comp rx=0 tx=0).stop 2024-05-10T11:18:20.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:20.753+0000 7f4dd61ba700 1 -- 172.21.15.100:0/391154265 shutdown_connections 2024-05-10T11:18:20.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:20.753+0000 7f4dd61ba700 1 --2- 172.21.15.100:0/391154265 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4dbc0b2880 0x7f4dbc0b4d40 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:20.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:20.753+0000 7f4dd61ba700 1 --2- 172.21.15.100:0/391154265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4dd0106f40 0x7f4dd0199980 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:20.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:20.753+0000 7f4dd61ba700 1 --2- 172.21.15.100:0/391154265 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4dd01020a0 0x7f4dd019f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:20.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:20.753+0000 7f4dd61ba700 1 --2- 172.21.15.100:0/391154265 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4dd0102a50 0x7f4dd019fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:20.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:20.753+0000 7f4dd61ba700 1 -- 172.21.15.100:0/391154265 >> 172.21.15.100:0/391154265 conn(0x7f4dd00776c0 msgr2=0x7f4dd0105bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:20.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:20.754+0000 7f4dd61ba700 1 -- 172.21.15.100:0/391154265 shutdown_connections 2024-05-10T11:18:20.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:20.754+0000 7f4dd61ba700 1 -- 172.21.15.100:0/391154265 wait complete. 2024-05-10T11:18:21.572 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-10T11:18:21.572 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-10T11:18:21.572 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi100.front.sepia.ceph.com 2024-05-10T11:18:21.572 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-10T11:18:21.572 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-10T11:18:21.572 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-10T11:18:21.572 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-10T11:18:21.572 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-10T11:18:21.572 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-10T11:18:21.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T11:18:21.573 DEBUG:teuthology.orchestra.run.smithi100:> ip addr 2024-05-10T11:18:21.590 INFO:teuthology.orchestra.run.smithi100.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-10T11:18:21.591 INFO:teuthology.orchestra.run.smithi100.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-10T11:18:21.591 INFO:teuthology.orchestra.run.smithi100.stdout: inet 127.0.0.1/8 scope host lo 2024-05-10T11:18:21.591 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-05-10T11:18:21.591 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 ::1/128 scope host 2024-05-10T11:18:21.591 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-05-10T11:18:21.591 INFO:teuthology.orchestra.run.smithi100.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T11:18:21.591 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:28 brd ff:ff:ff:ff:ff:ff 2024-05-10T11:18:21.591 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f0 2024-05-10T11:18:21.591 INFO:teuthology.orchestra.run.smithi100.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T11:18:21.591 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:29 brd ff:ff:ff:ff:ff:ff 2024-05-10T11:18:21.591 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f1 2024-05-10T11:18:21.591 INFO:teuthology.orchestra.run.smithi100.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T11:18:21.591 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:32 brd ff:ff:ff:ff:ff:ff 2024-05-10T11:18:21.592 INFO:teuthology.orchestra.run.smithi100.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-10T11:18:21.592 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:33 brd ff:ff:ff:ff:ff:ff 2024-05-10T11:18:21.592 INFO:teuthology.orchestra.run.smithi100.stdout: inet 172.21.15.100/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-10T11:18:21.592 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft 41964sec preferred_lft 41964sec 2024-05-10T11:18:21.592 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::ec4:7aff:fe8f:d733/64 scope link 2024-05-10T11:18:21.592 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-05-10T11:18:21.592 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-10T11:18:21.592 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-10T11:18:21.592 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-05-10T11:18:21.593 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link add name ceph-brx type bridge 2024-05-10T11:18:21.593 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip addr flush dev ceph-brx 2024-05-10T11:18:21.593 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set ceph-brx up 2024-05-10T11:18:21.593 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-10T11:18:21.593 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-05-10T11:18:21.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-10T11:18:21.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T11:18:21.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T11:18:21.733 DEBUG:teuthology.orchestra.run.smithi100:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-10T11:18:21.756 INFO:teuthology.orchestra.run.smithi100.stdout:1 2024-05-10T11:18:21.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T11:18:21.757 DEBUG:teuthology.orchestra.run.smithi100:> route 2024-05-10T11:18:21.815 INFO:teuthology.orchestra.run.smithi100.stdout:Kernel IP routing table 2024-05-10T11:18:21.815 INFO:teuthology.orchestra.run.smithi100.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-10T11:18:21.815 INFO:teuthology.orchestra.run.smithi100.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-10T11:18:21.815 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-10T11:18:21.816 INFO:teuthology.orchestra.run.smithi100.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-10T11:18:21.816 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-10T11:18:21.816 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-05-10T11:18:21.816 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-10T11:18:21.816 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-10T11:18:21.816 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-10T11:18:21.816 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-05-10T11:18:21.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-10T11:18:21.912 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:18:21 smithi100 ceph-mon[30296]: pgmap v419: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T11:18:21.913 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:18:21 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/391154265' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-10T11:18:21.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T11:18:21.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T11:18:21.943 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-05-10T11:18:21.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T11:18:21.960 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list-id 2024-05-10T11:18:22.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:18:21 smithi178 ceph-mon[33359]: pgmap v419: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T11:18:22.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:18:21 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/391154265' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-10T11:18:22.015 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-10T11:18:22.015 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-05-10T11:18:22.015 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-10T11:18:22.015 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-10T11:18:22.016 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-05-10T11:18:22.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:18:21 smithi188 ceph-mon[34075]: pgmap v419: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T11:18:22.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:18:21 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/391154265' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-10T11:18:22.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-10T11:18:22.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T11:18:22.134 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-10T11:18:22.134 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-10T11:18:22.134 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-05-10T11:18:22.134 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-10T11:18:22.134 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-10T11:18:22.134 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-10T11:18:22.134 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-10T11:18:22.134 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-10T11:18:22.135 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-05-10T11:18:22.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-10T11:18:22.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T11:18:22.248 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-10T11:18:22.248 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-05-10T11:18:22.248 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set brx.0 up 2024-05-10T11:18:22.248 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set dev brx.0 master ceph-brx 2024-05-10T11:18:22.248 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-05-10T11:18:22.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-10T11:18:22.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T11:18:22.326 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mount_subvol_num': 0} 2024-05-10T11:18:22.326 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T11:18:22.327 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-10T11:18:22.341 INFO:teuthology.orchestra.run.smithi100.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-10T11:18:22.341 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T11:18:22.342 DEBUG:teuthology.orchestra.run.smithi100:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-10T11:18:22.397 DEBUG:teuthology.orchestra.run.smithi100:> sudo modprobe fuse 2024-05-10T11:18:22.462 DEBUG:teuthology.orchestra.run.smithi100:> cat /proc/self/mounts | awk '{print $2}' 2024-05-10T11:18:22.520 INFO:teuthology.orchestra.run.smithi100.stdout:/proc 2024-05-10T11:18:22.520 INFO:teuthology.orchestra.run.smithi100.stdout:/sys 2024-05-10T11:18:22.520 INFO:teuthology.orchestra.run.smithi100.stdout:/dev 2024-05-10T11:18:22.520 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/security 2024-05-10T11:18:22.520 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/shm 2024-05-10T11:18:22.521 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/pts 2024-05-10T11:18:22.521 INFO:teuthology.orchestra.run.smithi100.stdout:/run 2024-05-10T11:18:22.521 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/cgroup 2024-05-10T11:18:22.521 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/pstore 2024-05-10T11:18:22.521 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/bpf 2024-05-10T11:18:22.521 INFO:teuthology.orchestra.run.smithi100.stdout:/ 2024-05-10T11:18:22.521 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/selinux 2024-05-10T11:18:22.521 INFO:teuthology.orchestra.run.smithi100.stdout:/proc/sys/fs/binfmt_misc 2024-05-10T11:18:22.521 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/mqueue 2024-05-10T11:18:22.521 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/hugepages 2024-05-10T11:18:22.521 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/debug 2024-05-10T11:18:22.521 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/tracing 2024-05-10T11:18:22.521 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/fuse/connections 2024-05-10T11:18:22.521 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/config 2024-05-10T11:18:22.522 INFO:teuthology.orchestra.run.smithi100.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-10T11:18:22.522 INFO:teuthology.orchestra.run.smithi100.stdout:/run/credentials/systemd-sysctl.service 2024-05-10T11:18:22.522 INFO:teuthology.orchestra.run.smithi100.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-10T11:18:22.522 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/nfs/rpc_pipefs 2024-05-10T11:18:22.522 INFO:teuthology.orchestra.run.smithi100.stdout:/run/user/1000 2024-05-10T11:18:22.522 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph 2024-05-10T11:18:22.522 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/containers/storage/overlay 2024-05-10T11:18:22.522 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/containers/storage/overlay/14c99dacab4f21afb763bc6738d3e41459fdd962ca149e9c806d7829d3a5e462/merged 2024-05-10T11:18:22.522 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/containers/storage/overlay/2ebaa2bbbfc96bd2425dfcc96728833728884750949c6a40f662fee33f4b994e/merged 2024-05-10T11:18:22.522 INFO:teuthology.orchestra.run.smithi100.stdout:/run/user/0 2024-05-10T11:18:22.522 INFO:teuthology.orchestra.run.smithi100.stdout:/proc/sys/fs/binfmt_misc 2024-05-10T11:18:22.522 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/containers/storage/overlay/13714d117f6e122b4c4f01bbf753a62c3541f3a730f30f49f2d5da6513f0305b/merged 2024-05-10T11:18:22.522 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/containers/storage/overlay/988d20f7e65ec06d4fb2acc5e7d5d246afd534d6aed6382840fb610666191a30/merged 2024-05-10T11:18:22.523 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/containers/storage/overlay/1a048bfd015e9bc34b400c599594a2b6829bf1318c996c13a6fd3fbf62e3169d/merged 2024-05-10T11:18:22.523 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/containers/storage/overlay/1276224ce171df2818ae1c61891eaad31014b7880cccc995fa2486d0d06d219c/merged 2024-05-10T11:18:22.523 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/containers/storage/overlay/5709ec87e7ac4fbfb97f09ec611630239a2d6f1420a85fdcdd9f37695b0fbb2e/merged 2024-05-10T11:18:22.523 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/containers/storage/overlay/886205e188c7b8d2d33b5282abae9196118fbb834277d2f9299693dcb24588e4/merged 2024-05-10T11:18:22.523 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/containers/storage/overlay/c6edc39d3dd307ad783812ab8e2e0121a898f29037388320c1802e0f88c0b564/merged 2024-05-10T11:18:22.523 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/containers/storage/overlay/8c972768aeb67ffe9b37821baac38161338670696cda25fea89a37cf99a46197/merged 2024-05-10T11:18:22.523 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns 2024-05-10T11:18:22.523 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-10T11:18:22.523 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-10T11:18:22.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T11:18:22.523 DEBUG:teuthology.orchestra.run.smithi100:> ls /sys/fs/fuse/connections 2024-05-10T11:18:22.576 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-10T11:18:22.576 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/volumes/qa/sv_0/96765652-39ef-4cf3-b20a-a70f7e9d774d --client_fs=cephfs) 2024-05-10T11:18:22.619 DEBUG:teuthology.orchestra.run.smithi100:> sudo modprobe fuse 2024-05-10T11:18:22.644 DEBUG:teuthology.orchestra.run.smithi100:> cat /proc/self/mounts | awk '{print $2}' 2024-05-10T11:18:22.700 INFO:teuthology.orchestra.run.smithi100.stdout:/proc 2024-05-10T11:18:22.701 INFO:teuthology.orchestra.run.smithi100.stdout:/sys 2024-05-10T11:18:22.701 INFO:teuthology.orchestra.run.smithi100.stdout:/dev 2024-05-10T11:18:22.701 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/security 2024-05-10T11:18:22.701 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/shm 2024-05-10T11:18:22.701 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/pts 2024-05-10T11:18:22.701 INFO:teuthology.orchestra.run.smithi100.stdout:/run 2024-05-10T11:18:22.701 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/cgroup 2024-05-10T11:18:22.701 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/pstore 2024-05-10T11:18:22.701 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/bpf 2024-05-10T11:18:22.701 INFO:teuthology.orchestra.run.smithi100.stdout:/ 2024-05-10T11:18:22.701 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/selinux 2024-05-10T11:18:22.701 INFO:teuthology.orchestra.run.smithi100.stdout:/proc/sys/fs/binfmt_misc 2024-05-10T11:18:22.701 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/mqueue 2024-05-10T11:18:22.701 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/hugepages 2024-05-10T11:18:22.702 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/debug 2024-05-10T11:18:22.702 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/tracing 2024-05-10T11:18:22.702 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/fuse/connections 2024-05-10T11:18:22.702 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/config 2024-05-10T11:18:22.702 INFO:teuthology.orchestra.run.smithi100.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-10T11:18:22.702 INFO:teuthology.orchestra.run.smithi100.stdout:/run/credentials/systemd-sysctl.service 2024-05-10T11:18:22.702 INFO:teuthology.orchestra.run.smithi100.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-10T11:18:22.702 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/nfs/rpc_pipefs 2024-05-10T11:18:22.702 INFO:teuthology.orchestra.run.smithi100.stdout:/run/user/1000 2024-05-10T11:18:22.702 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph 2024-05-10T11:18:22.702 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/containers/storage/overlay 2024-05-10T11:18:22.702 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/containers/storage/overlay/14c99dacab4f21afb763bc6738d3e41459fdd962ca149e9c806d7829d3a5e462/merged 2024-05-10T11:18:22.702 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/containers/storage/overlay/2ebaa2bbbfc96bd2425dfcc96728833728884750949c6a40f662fee33f4b994e/merged 2024-05-10T11:18:22.703 INFO:teuthology.orchestra.run.smithi100.stdout:/run/user/0 2024-05-10T11:18:22.703 INFO:teuthology.orchestra.run.smithi100.stdout:/proc/sys/fs/binfmt_misc 2024-05-10T11:18:22.703 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/containers/storage/overlay/13714d117f6e122b4c4f01bbf753a62c3541f3a730f30f49f2d5da6513f0305b/merged 2024-05-10T11:18:22.703 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/containers/storage/overlay/988d20f7e65ec06d4fb2acc5e7d5d246afd534d6aed6382840fb610666191a30/merged 2024-05-10T11:18:22.703 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/containers/storage/overlay/1a048bfd015e9bc34b400c599594a2b6829bf1318c996c13a6fd3fbf62e3169d/merged 2024-05-10T11:18:22.703 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/containers/storage/overlay/1276224ce171df2818ae1c61891eaad31014b7880cccc995fa2486d0d06d219c/merged 2024-05-10T11:18:22.703 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/containers/storage/overlay/5709ec87e7ac4fbfb97f09ec611630239a2d6f1420a85fdcdd9f37695b0fbb2e/merged 2024-05-10T11:18:22.703 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/containers/storage/overlay/886205e188c7b8d2d33b5282abae9196118fbb834277d2f9299693dcb24588e4/merged 2024-05-10T11:18:22.703 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/containers/storage/overlay/c6edc39d3dd307ad783812ab8e2e0121a898f29037388320c1802e0f88c0b564/merged 2024-05-10T11:18:22.703 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/containers/storage/overlay/8c972768aeb67ffe9b37821baac38161338670696cda25fea89a37cf99a46197/merged 2024-05-10T11:18:22.703 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns 2024-05-10T11:18:22.703 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-10T11:18:22.703 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-10T11:18:22.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T11:18:22.704 DEBUG:teuthology.orchestra.run.smithi100:> ls /sys/fs/fuse/connections 2024-05-10T11:18:22.705 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi100.stderr:2024-05-10T11:18:22.701+0000 7f20a3c0c480 -1 init, newargv = 0x56508b4f5160 newargc=15 2024-05-10T11:18:22.705 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi100.stderr:ceph-fuse[86801]: starting ceph client 2024-05-10T11:18:22.716 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi100.stderr:ceph-fuse[86801]: starting fuse 2024-05-10T11:18:22.720 INFO:teuthology.orchestra.run.smithi100.stdout:114 2024-05-10T11:18:22.721 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [114] 2024-05-10T11:18:22.721 DEBUG:teuthology.orchestra.run.smithi100:> sudo stdin-killer -- python3 -c ' 2024-05-10T11:18:22.721 DEBUG:teuthology.orchestra.run.smithi100:> import glob 2024-05-10T11:18:22.721 DEBUG:teuthology.orchestra.run.smithi100:> import re 2024-05-10T11:18:22.721 DEBUG:teuthology.orchestra.run.smithi100:> import os 2024-05-10T11:18:22.721 DEBUG:teuthology.orchestra.run.smithi100:> import subprocess 2024-05-10T11:18:22.721 DEBUG:teuthology.orchestra.run.smithi100:> 2024-05-10T11:18:22.721 DEBUG:teuthology.orchestra.run.smithi100:> def _find_admin_socket(client_name): 2024-05-10T11:18:22.721 DEBUG:teuthology.orchestra.run.smithi100:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-10T11:18:22.721 DEBUG:teuthology.orchestra.run.smithi100:> files = glob.glob(asok_path) 2024-05-10T11:18:22.721 DEBUG:teuthology.orchestra.run.smithi100:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-10T11:18:22.721 DEBUG:teuthology.orchestra.run.smithi100:> 2024-05-10T11:18:22.721 DEBUG:teuthology.orchestra.run.smithi100:> # Given a non-glob path, it better be there 2024-05-10T11:18:22.722 DEBUG:teuthology.orchestra.run.smithi100:> if "*" not in asok_path: 2024-05-10T11:18:22.722 DEBUG:teuthology.orchestra.run.smithi100:> assert(len(files) == 1) 2024-05-10T11:18:22.722 DEBUG:teuthology.orchestra.run.smithi100:> return files[0] 2024-05-10T11:18:22.722 DEBUG:teuthology.orchestra.run.smithi100:> 2024-05-10T11:18:22.722 DEBUG:teuthology.orchestra.run.smithi100:> for f in files: 2024-05-10T11:18:22.722 DEBUG:teuthology.orchestra.run.smithi100:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-10T11:18:22.722 DEBUG:teuthology.orchestra.run.smithi100:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-10T11:18:22.722 DEBUG:teuthology.orchestra.run.smithi100:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-10T11:18:22.722 DEBUG:teuthology.orchestra.run.smithi100:> contents = proc_f.read() 2024-05-10T11:18:22.722 DEBUG:teuthology.orchestra.run.smithi100:> if mountpoint in contents: 2024-05-10T11:18:22.722 DEBUG:teuthology.orchestra.run.smithi100:> return f 2024-05-10T11:18:22.722 DEBUG:teuthology.orchestra.run.smithi100:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-10T11:18:22.722 DEBUG:teuthology.orchestra.run.smithi100:> 2024-05-10T11:18:22.722 DEBUG:teuthology.orchestra.run.smithi100:> print(_find_admin_socket("client.0")) 2024-05-10T11:18:22.722 DEBUG:teuthology.orchestra.run.smithi100:> ' 2024-05-10T11:18:22.839 INFO:teuthology.orchestra.run.smithi100.stdout:/var/run/ceph/ceph-client.0.86801.asok 2024-05-10T11:18:22.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T11:18:22.850 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.86801.asok 2024-05-10T11:18:22.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T11:18:22.851 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.86801.asok status 2024-05-10T11:18:22.986 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:18:22.986 INFO:teuthology.orchestra.run.smithi100.stdout: "metadata": { 2024-05-10T11:18:22.986 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_sha1": "d39fa08867836ac070c47cb66c782fa559fd480b", 2024-05-10T11:18:22.986 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_version": "ceph version 19.0.0-3449-gd39fa088 (d39fa08867836ac070c47cb66c782fa559fd480b) squid (dev)", 2024-05-10T11:18:22.986 INFO:teuthology.orchestra.run.smithi100.stdout: "entity_id": "0", 2024-05-10T11:18:22.986 INFO:teuthology.orchestra.run.smithi100.stdout: "hostname": "smithi100", 2024-05-10T11:18:22.986 INFO:teuthology.orchestra.run.smithi100.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-10T11:18:22.986 INFO:teuthology.orchestra.run.smithi100.stdout: "pid": "86801", 2024-05-10T11:18:22.986 INFO:teuthology.orchestra.run.smithi100.stdout: "root": "/volumes/qa/sv_0/96765652-39ef-4cf3-b20a-a70f7e9d774d" 2024-05-10T11:18:22.987 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:18:22.987 INFO:teuthology.orchestra.run.smithi100.stdout: "dentry_count": 0, 2024-05-10T11:18:22.987 INFO:teuthology.orchestra.run.smithi100.stdout: "dentry_pinned_count": 0, 2024-05-10T11:18:22.987 INFO:teuthology.orchestra.run.smithi100.stdout: "id": 15207, 2024-05-10T11:18:22.987 INFO:teuthology.orchestra.run.smithi100.stdout: "inst": { 2024-05-10T11:18:22.987 INFO:teuthology.orchestra.run.smithi100.stdout: "name": { 2024-05-10T11:18:22.987 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "client", 2024-05-10T11:18:22.987 INFO:teuthology.orchestra.run.smithi100.stdout: "num": 15207 2024-05-10T11:18:22.987 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:18:22.987 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": { 2024-05-10T11:18:22.987 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-05-10T11:18:22.987 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "192.168.0.1:0", 2024-05-10T11:18:22.987 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3737523865 2024-05-10T11:18:22.987 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:18:22.987 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:18:22.988 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": { 2024-05-10T11:18:22.988 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-05-10T11:18:22.988 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "192.168.0.1:0", 2024-05-10T11:18:22.988 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3737523865 2024-05-10T11:18:22.988 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:18:22.988 INFO:teuthology.orchestra.run.smithi100.stdout: "inst_str": "client.15207 192.168.0.1:0/3737523865", 2024-05-10T11:18:22.988 INFO:teuthology.orchestra.run.smithi100.stdout: "addr_str": "192.168.0.1:0/3737523865", 2024-05-10T11:18:22.988 INFO:teuthology.orchestra.run.smithi100.stdout: "inode_count": 5, 2024-05-10T11:18:22.988 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_epoch": 26, 2024-05-10T11:18:22.988 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_epoch": 89, 2024-05-10T11:18:22.988 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_epoch_barrier": 0, 2024-05-10T11:18:22.988 INFO:teuthology.orchestra.run.smithi100.stdout: "blocklisted": false, 2024-05-10T11:18:22.988 INFO:teuthology.orchestra.run.smithi100.stdout: "fs_name": "cephfs" 2024-05-10T11:18:22.988 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:18:22.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T11:18:22.995 DEBUG:teuthology.orchestra.run.smithi100:> stat --file-system '--printf=%T 2024-05-10T11:18:22.995 DEBUG:teuthology.orchestra.run.smithi100:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-10T11:18:23.011 INFO:teuthology.orchestra.run.smithi100.stdout:fuseblk 2024-05-10T11:18:23.011 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-10T11:18:23.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T11:18:23.011 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-10T11:18:23.079 INFO:teuthology.run_tasks:Running task exec... 2024-05-10T11:18:23.090 INFO:teuthology.task.exec:Executing custom commands... 2024-05-10T11:18:23.090 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi100.front.sepia.ceph.com 2024-05-10T11:18:23.091 DEBUG:teuthology.orchestra.run.smithi100:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-05-10T11:18:23.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.222+0000 7f802a95f640 1 -- 172.21.15.100:0/2171323600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80241045c0 msgr2=0x7f802410ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:23.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.222+0000 7f802a95f640 1 --2- 172.21.15.100:0/2171323600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80241045c0 0x7f802410ae50 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f801c009ca0 tx=0x7f801c02a8c0 comp rx=0 tx=0).stop 2024-05-10T11:18:23.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.222+0000 7f802a95f640 1 -- 172.21.15.100:0/2171323600 shutdown_connections 2024-05-10T11:18:23.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.222+0000 7f802a95f640 1 --2- 172.21.15.100:0/2171323600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80241045c0 0x7f802410ae50 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:23.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.222+0000 7f802a95f640 1 --2- 172.21.15.100:0/2171323600 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8024103c00 0x7f8024104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:23.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.222+0000 7f802a95f640 1 --2- 172.21.15.100:0/2171323600 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8024102a10 0x7f8024102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:23.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.222+0000 7f802a95f640 1 -- 172.21.15.100:0/2171323600 >> 172.21.15.100:0/2171323600 conn(0x7f80240fdfa0 msgr2=0x7f80241003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:23.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.222+0000 7f802a95f640 1 -- 172.21.15.100:0/2171323600 shutdown_connections 2024-05-10T11:18:23.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.222+0000 7f802a95f640 1 -- 172.21.15.100:0/2171323600 wait complete. 2024-05-10T11:18:23.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.223+0000 7f802a95f640 1 Processor -- start 2024-05-10T11:18:23.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.223+0000 7f802a95f640 1 -- start start 2024-05-10T11:18:23.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.223+0000 7f802a95f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8024102a10 0x7f8024198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:23.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.223+0000 7f802a95f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8024103c00 0x7f8024198fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:23.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.223+0000 7f802a95f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f80241045c0 0x7f80241a0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:23.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.223+0000 7f802a95f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f802410e420 con 0x7f8024103c00 2024-05-10T11:18:23.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.223+0000 7f802a95f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f802410e2a0 con 0x7f80241045c0 2024-05-10T11:18:23.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.223+0000 7f802a95f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f802410e5a0 con 0x7f8024102a10 2024-05-10T11:18:23.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.223+0000 7f80237fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8024103c00 0x7f8024198fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:23.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.223+0000 7f80237fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8024103c00 0x7f8024198fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60208/0 (socket says 172.21.15.100:60208) 2024-05-10T11:18:23.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.223+0000 7f80237fe640 1 -- 172.21.15.100:0/1791103415 learned_addr learned my addr 172.21.15.100:0/1791103415 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:18:23.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.223+0000 7f8023fff640 1 --2- 172.21.15.100:0/1791103415 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8024102a10 0x7f8024198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:23.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.223+0000 7f8028efc640 1 --2- 172.21.15.100:0/1791103415 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f80241045c0 0x7f80241a0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:23.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.223+0000 7f80237fe640 1 -- 172.21.15.100:0/1791103415 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8024102a10 msgr2=0x7f8024198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:23.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.223+0000 7f80237fe640 1 --2- 172.21.15.100:0/1791103415 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8024102a10 0x7f8024198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:23.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.223+0000 7f80237fe640 1 -- 172.21.15.100:0/1791103415 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f80241045c0 msgr2=0x7f80241a0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:23.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.223+0000 7f80237fe640 1 --2- 172.21.15.100:0/1791103415 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f80241045c0 0x7f80241a0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:23.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.223+0000 7f80237fe640 1 -- 172.21.15.100:0/1791103415 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80241a0680 con 0x7f8024103c00 2024-05-10T11:18:23.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.224+0000 7f80237fe640 1 --2- 172.21.15.100:0/1791103415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8024103c00 0x7f8024198fa0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f801800d950 tx=0x7f801800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:23.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.224+0000 7f80217fa640 1 -- 172.21.15.100:0/1791103415 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8018017030 con 0x7f8024103c00 2024-05-10T11:18:23.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.224+0000 7f80217fa640 1 -- 172.21.15.100:0/1791103415 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8018004020 con 0x7f8024103c00 2024-05-10T11:18:23.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.224+0000 7f802a95f640 1 -- 172.21.15.100:0/1791103415 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80241a0970 con 0x7f8024103c00 2024-05-10T11:18:23.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.224+0000 7f80217fa640 1 -- 172.21.15.100:0/1791103415 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80180055d0 con 0x7f8024103c00 2024-05-10T11:18:23.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.224+0000 7f802a95f640 1 -- 172.21.15.100:0/1791103415 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80241a0ec0 con 0x7f8024103c00 2024-05-10T11:18:23.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.225+0000 7f802a95f640 1 -- 172.21.15.100:0/1791103415 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8024109d30 con 0x7f8024103c00 2024-05-10T11:18:23.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.226+0000 7f80217fa640 1 -- 172.21.15.100:0/1791103415 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f8018009a30 con 0x7f8024103c00 2024-05-10T11:18:23.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.227+0000 7f80217fa640 1 --2- 172.21.15.100:0/1791103415 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f800c0b28a0 0x7f800c0b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:23.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.227+0000 7f8023fff640 1 --2- 172.21.15.100:0/1791103415 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f800c0b28a0 0x7f800c0b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:23.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.227+0000 7f80217fa640 1 -- 172.21.15.100:0/1791103415 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f80180e2610 con 0x7f8024103c00 2024-05-10T11:18:23.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.227+0000 7f8023fff640 1 --2- 172.21.15.100:0/1791103415 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f800c0b28a0 0x7f800c0b4d60 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8014005fd0 tx=0x7f8014005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:23.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.230+0000 7f80217fa640 1 -- 172.21.15.100:0/1791103415 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f80180a9fd0 con 0x7f8024103c00 2024-05-10T11:18:23.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.453+0000 7f802a95f640 1 -- 172.21.15.100:0/1791103415 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f802402db50 con 0x7f8024103c00 2024-05-10T11:18:23.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.584+0000 7f80217fa640 1 -- 172.21.15.100:0/1791103415 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v27) ==== 117+0+0 (secure 0 0 0) 0x7f80180a98d0 con 0x7f8024103c00 2024-05-10T11:18:23.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.586+0000 7f802a95f640 1 -- 172.21.15.100:0/1791103415 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f800c0b28a0 msgr2=0x7f800c0b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:23.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.586+0000 7f802a95f640 1 --2- 172.21.15.100:0/1791103415 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f800c0b28a0 0x7f800c0b4d60 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8014005fd0 tx=0x7f8014005c70 comp rx=0 tx=0).stop 2024-05-10T11:18:23.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.586+0000 7f802a95f640 1 -- 172.21.15.100:0/1791103415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8024103c00 msgr2=0x7f8024198fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:23.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.586+0000 7f802a95f640 1 --2- 172.21.15.100:0/1791103415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8024103c00 0x7f8024198fa0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f801800d950 tx=0x7f801800de10 comp rx=0 tx=0).stop 2024-05-10T11:18:23.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.586+0000 7f802a95f640 1 -- 172.21.15.100:0/1791103415 shutdown_connections 2024-05-10T11:18:23.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.586+0000 7f802a95f640 1 --2- 172.21.15.100:0/1791103415 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f800c0b28a0 0x7f800c0b4d60 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:23.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.586+0000 7f802a95f640 1 --2- 172.21.15.100:0/1791103415 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f80241045c0 0x7f80241a0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:23.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.586+0000 7f802a95f640 1 --2- 172.21.15.100:0/1791103415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8024103c00 0x7f8024198fa0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:23.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.586+0000 7f802a95f640 1 --2- 172.21.15.100:0/1791103415 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8024102a10 0x7f8024198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:23.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.586+0000 7f802a95f640 1 -- 172.21.15.100:0/1791103415 >> 172.21.15.100:0/1791103415 conn(0x7f80240fdfa0 msgr2=0x7f8024106f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:23.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.586+0000 7f802a95f640 1 -- 172.21.15.100:0/1791103415 shutdown_connections 2024-05-10T11:18:23.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.586+0000 7f802a95f640 1 -- 172.21.15.100:0/1791103415 wait complete. 2024-05-10T11:18:23.597 INFO:teuthology.run_tasks:Running task check-counter... 2024-05-10T11:18:23.609 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-05-10T11:18:23.663 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T11:18:23.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.764+0000 7f4fc1815640 1 -- 172.21.15.100:0/2859844228 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fbc103480 msgr2=0x7f4fbc109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:23.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.764+0000 7f4fc1815640 1 --2- 172.21.15.100:0/2859844228 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fbc103480 0x7f4fbc109d10 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4009ca0 tx=0x7f4fb402a8c0 comp rx=0 tx=0).stop 2024-05-10T11:18:23.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.764+0000 7f4fc1815640 1 -- 172.21.15.100:0/2859844228 shutdown_connections 2024-05-10T11:18:23.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.764+0000 7f4fc1815640 1 --2- 172.21.15.100:0/2859844228 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fbc103480 0x7f4fbc109d10 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:23.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.764+0000 7f4fc1815640 1 --2- 172.21.15.100:0/2859844228 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fbc102ac0 0x7f4fbc102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:23.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.764+0000 7f4fc1815640 1 --2- 172.21.15.100:0/2859844228 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4fbc1018d0 0x7f4fbc101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:23.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.764+0000 7f4fc1815640 1 -- 172.21.15.100:0/2859844228 >> 172.21.15.100:0/2859844228 conn(0x7f4fbc0fce40 msgr2=0x7f4fbc0ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:23.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.765+0000 7f4fc1815640 1 -- 172.21.15.100:0/2859844228 shutdown_connections 2024-05-10T11:18:23.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.765+0000 7f4fc1815640 1 -- 172.21.15.100:0/2859844228 wait complete. 2024-05-10T11:18:23.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.765+0000 7f4fc1815640 1 Processor -- start 2024-05-10T11:18:23.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.765+0000 7f4fc1815640 1 -- start start 2024-05-10T11:18:23.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.765+0000 7f4fc1815640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4fbc1018d0 0x7f4fbc197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:23.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.765+0000 7f4fc1815640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fbc102ac0 0x7f4fbc197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:23.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.765+0000 7f4fc1815640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fbc103480 0x7f4fbc19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:23.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.765+0000 7f4fc1815640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4fbc10d2e0 con 0x7f4fbc103480 2024-05-10T11:18:23.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.765+0000 7f4fc1815640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4fbc10d160 con 0x7f4fbc1018d0 2024-05-10T11:18:23.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.765+0000 7f4fc1815640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4fbc10d460 con 0x7f4fbc102ac0 2024-05-10T11:18:23.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.766+0000 7f4fbb7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fbc103480 0x7f4fbc19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:23.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.766+0000 7f4fbb7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fbc103480 0x7f4fbc19ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60234/0 (socket says 172.21.15.100:60234) 2024-05-10T11:18:23.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.766+0000 7f4fbb7fe640 1 -- 172.21.15.100:0/4276641183 learned_addr learned my addr 172.21.15.100:0/4276641183 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:18:23.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.766+0000 7f4fba7fc640 1 --2- 172.21.15.100:0/4276641183 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fbc102ac0 0x7f4fbc197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:23.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.766+0000 7f4fbaffd640 1 --2- 172.21.15.100:0/4276641183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4fbc1018d0 0x7f4fbc197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:23.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.766+0000 7f4fbb7fe640 1 -- 172.21.15.100:0/4276641183 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fbc102ac0 msgr2=0x7f4fbc197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:23.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.766+0000 7f4fbb7fe640 1 --2- 172.21.15.100:0/4276641183 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fbc102ac0 0x7f4fbc197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:23.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.766+0000 7f4fbb7fe640 1 -- 172.21.15.100:0/4276641183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4fbc1018d0 msgr2=0x7f4fbc197990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:23.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.766+0000 7f4fbb7fe640 1 --2- 172.21.15.100:0/4276641183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4fbc1018d0 0x7f4fbc197990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:23.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.766+0000 7f4fbb7fe640 1 -- 172.21.15.100:0/4276641183 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fbc19f5b0 con 0x7f4fbc103480 2024-05-10T11:18:23.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.766+0000 7f4fbb7fe640 1 --2- 172.21.15.100:0/4276641183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fbc103480 0x7f4fbc19ef40 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f4fb402aeb0 tx=0x7f4fb4003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:23.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.766+0000 7f4fc0813640 1 -- 172.21.15.100:0/4276641183 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb4032ed0 con 0x7f4fbc103480 2024-05-10T11:18:23.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.766+0000 7f4fc0813640 1 -- 172.21.15.100:0/4276641183 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4fb4003e10 con 0x7f4fbc103480 2024-05-10T11:18:23.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.766+0000 7f4fc1815640 1 -- 172.21.15.100:0/4276641183 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fbc19f840 con 0x7f4fbc103480 2024-05-10T11:18:23.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.766+0000 7f4fc0813640 1 -- 172.21.15.100:0/4276641183 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb4030cd0 con 0x7f4fbc103480 2024-05-10T11:18:23.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.766+0000 7f4fc1815640 1 -- 172.21.15.100:0/4276641183 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fbc19fd90 con 0x7f4fbc103480 2024-05-10T11:18:23.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.767+0000 7f4fc1815640 1 -- 172.21.15.100:0/4276641183 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fbc108c70 con 0x7f4fbc103480 2024-05-10T11:18:23.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.769+0000 7f4fc0813640 1 -- 172.21.15.100:0/4276641183 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f4fb4004240 con 0x7f4fbc103480 2024-05-10T11:18:23.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.769+0000 7f4fc0813640 1 --2- 172.21.15.100:0/4276641183 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4fa40b27f0 0x7f4fa40b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:23.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.770+0000 7f4fbaffd640 1 --2- 172.21.15.100:0/4276641183 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4fa40b27f0 0x7f4fa40b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:23.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.770+0000 7f4fc0813640 1 -- 172.21.15.100:0/4276641183 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f4fb4103cd0 con 0x7f4fbc103480 2024-05-10T11:18:23.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.770+0000 7f4fbaffd640 1 --2- 172.21.15.100:0/4276641183 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4fa40b27f0 0x7f4fa40b4cb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4fac005d00 tx=0x7f4fac005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:23.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.774+0000 7f4fc0813640 1 -- 172.21.15.100:0/4276641183 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f4fb40cb630 con 0x7f4fbc103480 2024-05-10T11:18:23.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.958+0000 7f4fc1815640 1 -- 172.21.15.100:0/4276641183 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4fbc0635b0 con 0x7f4fbc103480 2024-05-10T11:18:23.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.960+0000 7f4fc0813640 1 -- 172.21.15.100:0/4276641183 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24737 (secure 0 0 0) 0x7f4fb40caf30 con 0x7f4fbc103480 2024-05-10T11:18:23.960 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:18:23.961 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":89,"fsid":"20a542cc-0ebd-11ef-bc98-c7b262605968","created":"2024-05-10T11:06:05.454446+0000","modified":"2024-05-10T11:15:01.418144+0000","last_up_change":"2024-05-10T11:13:18.185471+0000","last_in_change":"2024-05-10T11:12:57.268678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T11:09:26.176441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T11:14:21.004285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T11:14:24.179414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T11:14:33.308517+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5f031418-da23-4e85-8669-160960dadd09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":643028278},{"type":"v1","addr":"172.21.15.100:6803","nonce":643028278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":643028278},{"type":"v1","addr":"172.21.15.100:6805","nonce":643028278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":643028278},{"type":"v1","addr":"172.21.15.100:6809","nonce":643028278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":643028278},{"type":"v1","addr":"172.21.15.100:6807","nonce":643028278}]},"public_addr":"172.21.15.100:6803/643028278","cluster_addr":"172.21.15.100:6805/643028278","heartbeat_back_addr":"172.21.15.100:6809/643028278","heartbeat_front_addr":"172.21.15.100:6807/643028278","state":["exists","up"]},{"osd":1,"uuid":"a908cdc9-001e-4b65-9ab0-0dd08dba20f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":3103987017},{"type":"v1","addr":"172.21.15.178:6801","nonce":3103987017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":3103987017},{"type":"v1","addr":"172.21.15.178:6803","nonce":3103987017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":3103987017},{"type":"v1","addr":"172.21.15.178:6807","nonce":3103987017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":3103987017},{"type":"v1","addr":"172.21.15.178:6805","nonce":3103987017}]},"public_addr":"172.21.15.178:6801/3103987017","cluster_addr":"172.21.15.178:6803/3103987017","heartbeat_back_addr":"172.21.15.178:6807/3103987017","heartbeat_front_addr":"172.21.15.178:6805/3103987017","state":["exists","up"]},{"osd":2,"uuid":"4d8f89ae-9424-46e9-9c80-17c47cc5e8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3991466468},{"type":"v1","addr":"172.21.15.188:6801","nonce":3991466468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3991466468},{"type":"v1","addr":"172.21.15.188:6803","nonce":3991466468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3991466468},{"type":"v1","addr":"172.21.15.188:6807","nonce":3991466468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3991466468},{"type":"v1","addr":"172.21.15.188:6805","nonce":3991466468}]},"public_addr":"172.21.15.188:6801/3991466468","cluster_addr":"172.21.15.188:6803/3991466468","heartbeat_back_addr":"172.21.15.188:6807/3991466468","heartbeat_front_addr":"172.21.15.188:6805/3991466468","state":["exists","up"]},{"osd":3,"uuid":"4c8865aa-fef4-4710-b766-1385adb9462c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1713357192},{"type":"v1","addr":"172.21.15.100:6811","nonce":1713357192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1713357192},{"type":"v1","addr":"172.21.15.100:6813","nonce":1713357192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1713357192},{"type":"v1","addr":"172.21.15.100:6817","nonce":1713357192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1713357192},{"type":"v1","addr":"172.21.15.100:6815","nonce":1713357192}]},"public_addr":"172.21.15.100:6811/1713357192","cluster_addr":"172.21.15.100:6813/1713357192","heartbeat_back_addr":"172.21.15.100:6817/1713357192","heartbeat_front_addr":"172.21.15.100:6815/1713357192","state":["exists","up"]},{"osd":4,"uuid":"a9653800-da57-43eb-8781-3188e8a3949f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":2222814192},{"type":"v1","addr":"172.21.15.178:6809","nonce":2222814192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":2222814192},{"type":"v1","addr":"172.21.15.178:6811","nonce":2222814192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":2222814192},{"type":"v1","addr":"172.21.15.178:6815","nonce":2222814192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":2222814192},{"type":"v1","addr":"172.21.15.178:6813","nonce":2222814192}]},"public_addr":"172.21.15.178:6809/2222814192","cluster_addr":"172.21.15.178:6811/2222814192","heartbeat_back_addr":"172.21.15.178:6815/2222814192","heartbeat_front_addr":"172.21.15.178:6813/2222814192","state":["exists","up"]},{"osd":5,"uuid":"fb5f8af2-1eb0-4d8a-a7a3-ed6dd68e692e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3517375935},{"type":"v1","addr":"172.21.15.188:6809","nonce":3517375935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3517375935},{"type":"v1","addr":"172.21.15.188:6811","nonce":3517375935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3517375935},{"type":"v1","addr":"172.21.15.188:6815","nonce":3517375935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3517375935},{"type":"v1","addr":"172.21.15.188:6813","nonce":3517375935}]},"public_addr":"172.21.15.188:6809/3517375935","cluster_addr":"172.21.15.188:6811/3517375935","heartbeat_back_addr":"172.21.15.188:6815/3517375935","heartbeat_front_addr":"172.21.15.188:6813/3517375935","state":["exists","up"]},{"osd":6,"uuid":"84b57387-0a94-4e49-b099-493d329b87dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1410593862},{"type":"v1","addr":"172.21.15.100:6819","nonce":1410593862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1410593862},{"type":"v1","addr":"172.21.15.100:6821","nonce":1410593862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1410593862},{"type":"v1","addr":"172.21.15.100:6825","nonce":1410593862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1410593862},{"type":"v1","addr":"172.21.15.100:6823","nonce":1410593862}]},"public_addr":"172.21.15.100:6819/1410593862","cluster_addr":"172.21.15.100:6821/1410593862","heartbeat_back_addr":"172.21.15.100:6825/1410593862","heartbeat_front_addr":"172.21.15.100:6823/1410593862","state":["exists","up"]},{"osd":7,"uuid":"02ada28f-ac4f-47cf-983f-8a6da1140abe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2458690455},{"type":"v1","addr":"172.21.15.178:6817","nonce":2458690455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2458690455},{"type":"v1","addr":"172.21.15.178:6819","nonce":2458690455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2458690455},{"type":"v1","addr":"172.21.15.178:6823","nonce":2458690455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2458690455},{"type":"v1","addr":"172.21.15.178:6821","nonce":2458690455}]},"public_addr":"172.21.15.178:6817/2458690455","cluster_addr":"172.21.15.178:6819/2458690455","heartbeat_back_addr":"172.21.15.178:6823/2458690455","heartbeat_front_addr":"172.21.15.178:6821/2458690455","state":["exists","up"]},{"osd":8,"uuid":"e7e489bc-15fc-4a4c-b297-d1bb626513ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1523872826},{"type":"v1","addr":"172.21.15.188:6817","nonce":1523872826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1523872826},{"type":"v1","addr":"172.21.15.188:6819","nonce":1523872826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1523872826},{"type":"v1","addr":"172.21.15.188:6823","nonce":1523872826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1523872826},{"type":"v1","addr":"172.21.15.188:6821","nonce":1523872826}]},"public_addr":"172.21.15.188:6817/1523872826","cluster_addr":"172.21.15.188:6819/1523872826","heartbeat_back_addr":"172.21.15.188:6823/1523872826","heartbeat_front_addr":"172.21.15.188:6821/1523872826","state":["exists","up"]},{"osd":9,"uuid":"72328326-cd16-45d9-bfd9-40f6c5c20f5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3585761978},{"type":"v1","addr":"172.21.15.100:6827","nonce":3585761978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3585761978},{"type":"v1","addr":"172.21.15.100:6829","nonce":3585761978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3585761978},{"type":"v1","addr":"172.21.15.100:6833","nonce":3585761978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3585761978},{"type":"v1","addr":"172.21.15.100:6831","nonce":3585761978}]},"public_addr":"172.21.15.100:6827/3585761978","cluster_addr":"172.21.15.100:6829/3585761978","heartbeat_back_addr":"172.21.15.100:6833/3585761978","heartbeat_front_addr":"172.21.15.100:6831/3585761978","state":["exists","up"]},{"osd":10,"uuid":"a355f9a8-91a3-41d4-814d-5a9c9b953706","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2985841725},{"type":"v1","addr":"172.21.15.178:6825","nonce":2985841725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2985841725},{"type":"v1","addr":"172.21.15.178:6827","nonce":2985841725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2985841725},{"type":"v1","addr":"172.21.15.178:6831","nonce":2985841725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2985841725},{"type":"v1","addr":"172.21.15.178:6829","nonce":2985841725}]},"public_addr":"172.21.15.178:6825/2985841725","cluster_addr":"172.21.15.178:6827/2985841725","heartbeat_back_addr":"172.21.15.178:6831/2985841725","heartbeat_front_addr":"172.21.15.178:6829/2985841725","state":["exists","up"]},{"osd":11,"uuid":"76a5e13c-7e38-42f9-bfa7-7fedec7c7184","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2574955118},{"type":"v1","addr":"172.21.15.188:6825","nonce":2574955118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2574955118},{"type":"v1","addr":"172.21.15.188:6827","nonce":2574955118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2574955118},{"type":"v1","addr":"172.21.15.188:6831","nonce":2574955118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2574955118},{"type":"v1","addr":"172.21.15.188:6829","nonce":2574955118}]},"public_addr":"172.21.15.188:6825/2574955118","cluster_addr":"172.21.15.188:6827/2574955118","heartbeat_back_addr":"172.21.15.188:6831/2574955118","heartbeat_front_addr":"172.21.15.188:6829/2574955118","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:08:36.086637+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:08:58.948031+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:09:22.142517+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:09:47.655748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:10:12.853999+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:10:37.911383+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:11:04.448815+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:11:30.331740+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:11:55.819278+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:12:23.204612+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:12:49.484835+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:13:15.783531+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:6800/1597433518":"2024-05-11T11:07:00.092732+0000","172.21.15.100:6800/1086450256":"2024-05-11T11:06:33.114176+0000","172.21.15.100:0/767964410":"2024-05-11T11:07:00.092732+0000","172.21.15.100:6801/1597433518":"2024-05-11T11:07:00.092732+0000","172.21.15.100:6801/1086450256":"2024-05-11T11:06:33.114176+0000","172.21.15.100:0/2770572070":"2024-05-11T11:06:33.114176+0000","172.21.15.100:0/1379340513":"2024-05-11T11:06:33.114176+0000","172.21.15.100:0/3025646953":"2024-05-11T11:07:00.092732+0000","172.21.15.100:0/4099258930":"2024-05-11T11:07:00.092732+0000","172.21.15.100:0/4267693567":"2024-05-11T11:06:33.114176+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T11:18:23.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.962+0000 7f4fc1815640 1 -- 172.21.15.100:0/4276641183 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4fa40b27f0 msgr2=0x7f4fa40b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:23.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.962+0000 7f4fc1815640 1 --2- 172.21.15.100:0/4276641183 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4fa40b27f0 0x7f4fa40b4cb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4fac005d00 tx=0x7f4fac005c50 comp rx=0 tx=0).stop 2024-05-10T11:18:23.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.962+0000 7f4fc1815640 1 -- 172.21.15.100:0/4276641183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fbc103480 msgr2=0x7f4fbc19ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:23.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.962+0000 7f4fc1815640 1 --2- 172.21.15.100:0/4276641183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fbc103480 0x7f4fbc19ef40 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f4fb402aeb0 tx=0x7f4fb4003c20 comp rx=0 tx=0).stop 2024-05-10T11:18:23.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.962+0000 7f4fc1815640 1 -- 172.21.15.100:0/4276641183 shutdown_connections 2024-05-10T11:18:23.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.962+0000 7f4fc1815640 1 --2- 172.21.15.100:0/4276641183 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4fa40b27f0 0x7f4fa40b4cb0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:23.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.962+0000 7f4fc1815640 1 --2- 172.21.15.100:0/4276641183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fbc103480 0x7f4fbc19ef40 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:23.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.962+0000 7f4fc1815640 1 --2- 172.21.15.100:0/4276641183 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fbc102ac0 0x7f4fbc197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:23.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.962+0000 7f4fc1815640 1 --2- 172.21.15.100:0/4276641183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4fbc1018d0 0x7f4fbc197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:23.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.962+0000 7f4fc1815640 1 -- 172.21.15.100:0/4276641183 >> 172.21.15.100:0/4276641183 conn(0x7f4fbc0fce40 msgr2=0x7f4fbc105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:23.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.962+0000 7f4fc1815640 1 -- 172.21.15.100:0/4276641183 shutdown_connections 2024-05-10T11:18:23.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:23.962+0000 7f4fc1815640 1 -- 172.21.15.100:0/4276641183 wait complete. 2024-05-10T11:18:23.973 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:18:23 smithi100 ceph-mon[30296]: pgmap v420: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T11:18:23.973 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:18:23 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/1791103415' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-10T11:18:23.974 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T11:09:26.176441+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T11:14:21.004285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T11:14:24.179414+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-10T11:14:33.308517+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-10T11:18:23.975 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T11:18:24.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:18:23 smithi178 ceph-mon[33359]: pgmap v420: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T11:18:24.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:18:23 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/1791103415' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-10T11:18:24.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:18:23 smithi188 ceph-mon[34075]: pgmap v420: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-10T11:18:24.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:18:23 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/1791103415' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-10T11:18:24.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.078+0000 7f2454322640 1 -- 172.21.15.100:0/2796201251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f244c1086d0 msgr2=0x7f244c10ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:24.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.078+0000 7f2454322640 1 --2- 172.21.15.100:0/2796201251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f244c1086d0 0x7f244c10ab20 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f2444009900 tx=0x7f244402a410 comp rx=0 tx=0).stop 2024-05-10T11:18:24.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.079+0000 7f2454322640 1 -- 172.21.15.100:0/2796201251 shutdown_connections 2024-05-10T11:18:24.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.079+0000 7f2454322640 1 --2- 172.21.15.100:0/2796201251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f244c1086d0 0x7f244c10ab20 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:24.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.079+0000 7f2454322640 1 --2- 172.21.15.100:0/2796201251 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f244c105da0 0x7f244c108190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:24.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.079+0000 7f2454322640 1 --2- 172.21.15.100:0/2796201251 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f244c103470 0x7f244c105860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:24.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.079+0000 7f2454322640 1 -- 172.21.15.100:0/2796201251 >> 172.21.15.100:0/2796201251 conn(0x7f244c0fce60 msgr2=0x7f244c0ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:24.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.079+0000 7f2454322640 1 -- 172.21.15.100:0/2796201251 shutdown_connections 2024-05-10T11:18:24.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.079+0000 7f2454322640 1 -- 172.21.15.100:0/2796201251 wait complete. 2024-05-10T11:18:24.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.079+0000 7f2454322640 1 Processor -- start 2024-05-10T11:18:24.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.079+0000 7f2454322640 1 -- start start 2024-05-10T11:18:24.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.079+0000 7f2454322640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f244c103470 0x7f244c198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:24.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.079+0000 7f2454322640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f244c105da0 0x7f244c199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:24.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.079+0000 7f2454322640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f244c1086d0 0x7f244c1a00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:24.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.079+0000 7f2454322640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f244c10e3d0 con 0x7f244c1086d0 2024-05-10T11:18:24.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.079+0000 7f2454322640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f244c10e250 con 0x7f244c103470 2024-05-10T11:18:24.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.079+0000 7f2454322640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f244c10e550 con 0x7f244c105da0 2024-05-10T11:18:24.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.079+0000 7f24528bf640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f244c1086d0 0x7f244c1a00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:24.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.079+0000 7f24528bf640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f244c1086d0 0x7f244c1a00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60250/0 (socket says 172.21.15.100:60250) 2024-05-10T11:18:24.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.079+0000 7f24528bf640 1 -- 172.21.15.100:0/2494492725 learned_addr learned my addr 172.21.15.100:0/2494492725 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:18:24.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.079+0000 7f24518bd640 1 --2- 172.21.15.100:0/2494492725 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f244c105da0 0x7f244c199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:24.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.080+0000 7f24520be640 1 --2- 172.21.15.100:0/2494492725 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f244c103470 0x7f244c198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:24.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.080+0000 7f24528bf640 1 -- 172.21.15.100:0/2494492725 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f244c105da0 msgr2=0x7f244c199080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:24.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.080+0000 7f24528bf640 1 --2- 172.21.15.100:0/2494492725 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f244c105da0 0x7f244c199080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:24.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.080+0000 7f24528bf640 1 -- 172.21.15.100:0/2494492725 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f244c103470 msgr2=0x7f244c198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:24.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.080+0000 7f24528bf640 1 --2- 172.21.15.100:0/2494492725 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f244c103470 0x7f244c198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:24.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.080+0000 7f24528bf640 1 -- 172.21.15.100:0/2494492725 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f244c1a0760 con 0x7f244c1086d0 2024-05-10T11:18:24.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.080+0000 7f24520be640 1 --2- 172.21.15.100:0/2494492725 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f244c103470 0x7f244c198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:18:24.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.080+0000 7f24528bf640 1 --2- 172.21.15.100:0/2494492725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f244c1086d0 0x7f244c1a00f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2444004770 tx=0x7f24440042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:24.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.080+0000 7f244b7fe640 1 -- 172.21.15.100:0/2494492725 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244403f030 con 0x7f244c1086d0 2024-05-10T11:18:24.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.080+0000 7f244b7fe640 1 -- 172.21.15.100:0/2494492725 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2444003c90 con 0x7f244c1086d0 2024-05-10T11:18:24.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.080+0000 7f2454322640 1 -- 172.21.15.100:0/2494492725 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f244c1a09f0 con 0x7f244c1086d0 2024-05-10T11:18:24.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.080+0000 7f244b7fe640 1 -- 172.21.15.100:0/2494492725 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2444035070 con 0x7f244c1086d0 2024-05-10T11:18:24.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.080+0000 7f2454322640 1 -- 172.21.15.100:0/2494492725 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f244c1a0ee0 con 0x7f244c1086d0 2024-05-10T11:18:24.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.081+0000 7f2454322640 1 -- 172.21.15.100:0/2494492725 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f244c0fe950 con 0x7f244c1086d0 2024-05-10T11:18:24.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.082+0000 7f244b7fe640 1 -- 172.21.15.100:0/2494492725 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f2444042030 con 0x7f244c1086d0 2024-05-10T11:18:24.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.083+0000 7f244b7fe640 1 --2- 172.21.15.100:0/2494492725 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f24340b27f0 0x7f24340b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:24.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.083+0000 7f24520be640 1 --2- 172.21.15.100:0/2494492725 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f24340b27f0 0x7f24340b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:24.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.083+0000 7f244b7fe640 1 -- 172.21.15.100:0/2494492725 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f24441030a0 con 0x7f244c1086d0 2024-05-10T11:18:24.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.083+0000 7f24520be640 1 --2- 172.21.15.100:0/2494492725 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f24340b27f0 0x7f24340b4cb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f243c005fd0 tx=0x7f243c009600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:24.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.087+0000 7f244b7fe640 1 -- 172.21.15.100:0/2494492725 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f24440caa60 con 0x7f244c1086d0 2024-05-10T11:18:24.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.267+0000 7f2454322640 1 -- 172.21.15.100:0/2494492725 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f244c02dce0 con 0x7f244c1086d0 2024-05-10T11:18:24.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.268+0000 7f244b7fe640 1 -- 172.21.15.100:0/2494492725 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7f24440ca360 con 0x7f244c1086d0 2024-05-10T11:18:24.267 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-05-10T11:18:24.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.269+0000 7f2454322640 1 -- 172.21.15.100:0/2494492725 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f24340b27f0 msgr2=0x7f24340b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:24.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.270+0000 7f2454322640 1 --2- 172.21.15.100:0/2494492725 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f24340b27f0 0x7f24340b4cb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f243c005fd0 tx=0x7f243c009600 comp rx=0 tx=0).stop 2024-05-10T11:18:24.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.270+0000 7f2454322640 1 -- 172.21.15.100:0/2494492725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f244c1086d0 msgr2=0x7f244c1a00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:24.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.270+0000 7f2454322640 1 --2- 172.21.15.100:0/2494492725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f244c1086d0 0x7f244c1a00f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2444004770 tx=0x7f24440042f0 comp rx=0 tx=0).stop 2024-05-10T11:18:24.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.270+0000 7f2454322640 1 -- 172.21.15.100:0/2494492725 shutdown_connections 2024-05-10T11:18:24.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.270+0000 7f2454322640 1 --2- 172.21.15.100:0/2494492725 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f24340b27f0 0x7f24340b4cb0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:24.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.270+0000 7f2454322640 1 --2- 172.21.15.100:0/2494492725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f244c1086d0 0x7f244c1a00f0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:24.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.270+0000 7f2454322640 1 --2- 172.21.15.100:0/2494492725 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f244c105da0 0x7f244c199080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:24.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.270+0000 7f2454322640 1 --2- 172.21.15.100:0/2494492725 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f244c103470 0x7f244c198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:24.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.270+0000 7f2454322640 1 -- 172.21.15.100:0/2494492725 >> 172.21.15.100:0/2494492725 conn(0x7f244c0fce60 msgr2=0x7f244c103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:24.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.270+0000 7f2454322640 1 -- 172.21.15.100:0/2494492725 shutdown_connections 2024-05-10T11:18:24.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.270+0000 7f2454322640 1 -- 172.21.15.100:0/2494492725 wait complete. 2024-05-10T11:18:24.281 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T11:18:24.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.385+0000 7f8b425b6640 1 -- 172.21.15.100:0/3431982839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b3c102b00 msgr2=0x7f8b3c102f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:24.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.385+0000 7f8b425b6640 1 --2- 172.21.15.100:0/3431982839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b3c102b00 0x7f8b3c102f80 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8b24009900 tx=0x7f8b2402a410 comp rx=0 tx=0).stop 2024-05-10T11:18:24.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.385+0000 7f8b425b6640 1 -- 172.21.15.100:0/3431982839 shutdown_connections 2024-05-10T11:18:24.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.385+0000 7f8b425b6640 1 --2- 172.21.15.100:0/3431982839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b3c1034c0 0x7f8b3c109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:24.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.385+0000 7f8b425b6640 1 --2- 172.21.15.100:0/3431982839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b3c102b00 0x7f8b3c102f80 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:24.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.385+0000 7f8b425b6640 1 --2- 172.21.15.100:0/3431982839 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b3c101910 0x7f8b3c101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:24.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.385+0000 7f8b425b6640 1 -- 172.21.15.100:0/3431982839 >> 172.21.15.100:0/3431982839 conn(0x7f8b3c0fcea0 msgr2=0x7f8b3c0ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:24.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.385+0000 7f8b425b6640 1 -- 172.21.15.100:0/3431982839 shutdown_connections 2024-05-10T11:18:24.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.385+0000 7f8b425b6640 1 -- 172.21.15.100:0/3431982839 wait complete. 2024-05-10T11:18:24.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.386+0000 7f8b425b6640 1 Processor -- start 2024-05-10T11:18:24.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.386+0000 7f8b425b6640 1 -- start start 2024-05-10T11:18:24.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.386+0000 7f8b425b6640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b3c101910 0x7f8b3c06bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:24.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.386+0000 7f8b425b6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b3c102b00 0x7f8b3c19d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:24.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.386+0000 7f8b425b6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b3c1034c0 0x7f8b3c19fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:24.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.386+0000 7f8b425b6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8b3c10d2d0 con 0x7f8b3c102b00 2024-05-10T11:18:24.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.386+0000 7f8b425b6640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8b3c10d150 con 0x7f8b3c101910 2024-05-10T11:18:24.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.386+0000 7f8b425b6640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8b3c10d450 con 0x7f8b3c1034c0 2024-05-10T11:18:24.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.386+0000 7f8b33fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b3c102b00 0x7f8b3c19d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:24.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.386+0000 7f8b33fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b3c102b00 0x7f8b3c19d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60262/0 (socket says 172.21.15.100:60262) 2024-05-10T11:18:24.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.386+0000 7f8b33fff640 1 -- 172.21.15.100:0/3309664419 learned_addr learned my addr 172.21.15.100:0/3309664419 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:18:24.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.386+0000 7f8b3bfff640 1 --2- 172.21.15.100:0/3309664419 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b3c101910 0x7f8b3c06bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:24.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.386+0000 7f8b40b53640 1 --2- 172.21.15.100:0/3309664419 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b3c1034c0 0x7f8b3c19fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:24.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.386+0000 7f8b33fff640 1 -- 172.21.15.100:0/3309664419 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b3c1034c0 msgr2=0x7f8b3c19fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:24.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.386+0000 7f8b33fff640 1 --2- 172.21.15.100:0/3309664419 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b3c1034c0 0x7f8b3c19fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:24.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.386+0000 7f8b33fff640 1 -- 172.21.15.100:0/3309664419 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b3c101910 msgr2=0x7f8b3c06bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:24.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.386+0000 7f8b33fff640 1 --2- 172.21.15.100:0/3309664419 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b3c101910 0x7f8b3c06bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:24.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.386+0000 7f8b33fff640 1 -- 172.21.15.100:0/3309664419 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b3c1a0780 con 0x7f8b3c102b00 2024-05-10T11:18:24.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.387+0000 7f8b33fff640 1 --2- 172.21.15.100:0/3309664419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b3c102b00 0x7f8b3c19d880 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f8b24009230 tx=0x7f8b240042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:24.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.387+0000 7f8b39ffb640 1 -- 172.21.15.100:0/3309664419 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b2403f030 con 0x7f8b3c102b00 2024-05-10T11:18:24.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.387+0000 7f8b39ffb640 1 -- 172.21.15.100:0/3309664419 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8b24003c90 con 0x7f8b3c102b00 2024-05-10T11:18:24.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.387+0000 7f8b425b6640 1 -- 172.21.15.100:0/3309664419 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b3c1a0a10 con 0x7f8b3c102b00 2024-05-10T11:18:24.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.387+0000 7f8b39ffb640 1 -- 172.21.15.100:0/3309664419 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b24035070 con 0x7f8b3c102b00 2024-05-10T11:18:24.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.387+0000 7f8b425b6640 1 -- 172.21.15.100:0/3309664419 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b3c1a0f00 con 0x7f8b3c102b00 2024-05-10T11:18:24.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.388+0000 7f8b425b6640 1 -- 172.21.15.100:0/3309664419 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b3c108d50 con 0x7f8b3c102b00 2024-05-10T11:18:24.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.389+0000 7f8b39ffb640 1 -- 172.21.15.100:0/3309664419 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f8b24042030 con 0x7f8b3c102b00 2024-05-10T11:18:24.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.390+0000 7f8b39ffb640 1 --2- 172.21.15.100:0/3309664419 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8b280b27f0 0x7f8b280b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:24.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.390+0000 7f8b3bfff640 1 --2- 172.21.15.100:0/3309664419 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8b280b27f0 0x7f8b280b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:24.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.390+0000 7f8b39ffb640 1 -- 172.21.15.100:0/3309664419 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f8b24102c30 con 0x7f8b3c102b00 2024-05-10T11:18:24.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.390+0000 7f8b3bfff640 1 --2- 172.21.15.100:0/3309664419 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8b280b27f0 0x7f8b280b4cb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c102960 tx=0x7f8b2c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:24.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.394+0000 7f8b39ffb640 1 -- 172.21.15.100:0/3309664419 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f8b240ca060 con 0x7f8b3c102b00 2024-05-10T11:18:24.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.576+0000 7f8b425b6640 1 -- 172.21.15.100:0/3309664419 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8b3c0635b0 con 0x7f8b3c102b00 2024-05-10T11:18:24.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.577+0000 7f8b39ffb640 1 -- 172.21.15.100:0/3309664419 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7f8b240ce360 con 0x7f8b3c102b00 2024-05-10T11:18:24.579 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-05-10T11:18:24.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.579+0000 7f8b425b6640 1 -- 172.21.15.100:0/3309664419 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8b280b27f0 msgr2=0x7f8b280b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:24.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.579+0000 7f8b425b6640 1 --2- 172.21.15.100:0/3309664419 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8b280b27f0 0x7f8b280b4cb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c102960 tx=0x7f8b2c00b390 comp rx=0 tx=0).stop 2024-05-10T11:18:24.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.579+0000 7f8b425b6640 1 -- 172.21.15.100:0/3309664419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b3c102b00 msgr2=0x7f8b3c19d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:24.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.579+0000 7f8b425b6640 1 --2- 172.21.15.100:0/3309664419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b3c102b00 0x7f8b3c19d880 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f8b24009230 tx=0x7f8b240042f0 comp rx=0 tx=0).stop 2024-05-10T11:18:24.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.579+0000 7f8b425b6640 1 -- 172.21.15.100:0/3309664419 shutdown_connections 2024-05-10T11:18:24.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.579+0000 7f8b425b6640 1 --2- 172.21.15.100:0/3309664419 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8b280b27f0 0x7f8b280b4cb0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:24.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.579+0000 7f8b425b6640 1 --2- 172.21.15.100:0/3309664419 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b3c1034c0 0x7f8b3c19fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:24.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.579+0000 7f8b425b6640 1 --2- 172.21.15.100:0/3309664419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b3c102b00 0x7f8b3c19d880 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:24.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.579+0000 7f8b425b6640 1 --2- 172.21.15.100:0/3309664419 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b3c101910 0x7f8b3c06bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:24.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.579+0000 7f8b425b6640 1 -- 172.21.15.100:0/3309664419 >> 172.21.15.100:0/3309664419 conn(0x7f8b3c0fcea0 msgr2=0x7f8b3c105f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:24.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.579+0000 7f8b425b6640 1 -- 172.21.15.100:0/3309664419 shutdown_connections 2024-05-10T11:18:24.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.579+0000 7f8b425b6640 1 -- 172.21.15.100:0/3309664419 wait complete. 2024-05-10T11:18:24.591 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T11:18:24.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.695+0000 7f171b01d640 1 -- 172.21.15.100:0/2403089775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1714101c40 msgr2=0x7f17141020a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:24.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.695+0000 7f171b01d640 1 --2- 172.21.15.100:0/2403089775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1714101c40 0x7f17141020a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f16fc009900 tx=0x7f16fc02a480 comp rx=0 tx=0).stop 2024-05-10T11:18:24.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.695+0000 7f171b01d640 1 -- 172.21.15.100:0/2403089775 shutdown_connections 2024-05-10T11:18:24.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.695+0000 7f171b01d640 1 --2- 172.21.15.100:0/2403089775 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17141025e0 0x7f1714102a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:24.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.695+0000 7f171b01d640 1 --2- 172.21.15.100:0/2403089775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1714101c40 0x7f17141020a0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:24.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.695+0000 7f171b01d640 1 --2- 172.21.15.100:0/2403089775 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17141012e0 0x7f1714101700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:24.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.695+0000 7f171b01d640 1 -- 172.21.15.100:0/2403089775 >> 172.21.15.100:0/2403089775 conn(0x7f17140fce60 msgr2=0x7f17140ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:24.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.695+0000 7f171b01d640 1 -- 172.21.15.100:0/2403089775 shutdown_connections 2024-05-10T11:18:24.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.696+0000 7f171b01d640 1 -- 172.21.15.100:0/2403089775 wait complete. 2024-05-10T11:18:24.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.696+0000 7f171b01d640 1 Processor -- start 2024-05-10T11:18:24.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.696+0000 7f171b01d640 1 -- start start 2024-05-10T11:18:24.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.696+0000 7f171b01d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17141012e0 0x7f171419ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:24.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.696+0000 7f171b01d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1714101c40 0x7f171419b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:24.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.696+0000 7f171b01d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17141025e0 0x7f17141a2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:24.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.696+0000 7f171b01d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f17141105d0 con 0x7f17141012e0 2024-05-10T11:18:24.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.696+0000 7f171b01d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1714110450 con 0x7f17141025e0 2024-05-10T11:18:24.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.696+0000 7f171b01d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1714110750 con 0x7f1714101c40 2024-05-10T11:18:24.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.696+0000 7f1718db9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17141012e0 0x7f171419ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:24.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.696+0000 7f1718db9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17141012e0 0x7f171419ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60274/0 (socket says 172.21.15.100:60274) 2024-05-10T11:18:24.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.696+0000 7f1718db9640 1 -- 172.21.15.100:0/3034883279 learned_addr learned my addr 172.21.15.100:0/3034883279 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:18:24.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.697+0000 7f1713fff640 1 --2- 172.21.15.100:0/3034883279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1714101c40 0x7f171419b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:24.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.697+0000 7f17195ba640 1 --2- 172.21.15.100:0/3034883279 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17141025e0 0x7f17141a2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:24.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.697+0000 7f1718db9640 1 -- 172.21.15.100:0/3034883279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1714101c40 msgr2=0x7f171419b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:24.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.697+0000 7f1718db9640 1 --2- 172.21.15.100:0/3034883279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1714101c40 0x7f171419b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:24.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.697+0000 7f1718db9640 1 -- 172.21.15.100:0/3034883279 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17141025e0 msgr2=0x7f17141a2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:24.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.697+0000 7f1718db9640 1 --2- 172.21.15.100:0/3034883279 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17141025e0 0x7f17141a2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:24.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.697+0000 7f1718db9640 1 -- 172.21.15.100:0/3034883279 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17141a2a00 con 0x7f17141012e0 2024-05-10T11:18:24.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.697+0000 7f1713fff640 1 --2- 172.21.15.100:0/3034883279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1714101c40 0x7f171419b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:18:24.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.697+0000 7f17195ba640 1 --2- 172.21.15.100:0/3034883279 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17141025e0 0x7f17141a2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:18:24.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.697+0000 7f1718db9640 1 --2- 172.21.15.100:0/3034883279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17141012e0 0x7f171419ade0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f170400db80 tx=0x7f170400b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:24.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.697+0000 7f1711ffb640 1 -- 172.21.15.100:0/3034883279 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1704017070 con 0x7f17141012e0 2024-05-10T11:18:24.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.697+0000 7f1711ffb640 1 -- 172.21.15.100:0/3034883279 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1704004020 con 0x7f17141012e0 2024-05-10T11:18:24.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.697+0000 7f171b01d640 1 -- 172.21.15.100:0/3034883279 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17141a2cf0 con 0x7f17141012e0 2024-05-10T11:18:24.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.697+0000 7f1711ffb640 1 -- 172.21.15.100:0/3034883279 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f170400be60 con 0x7f17141012e0 2024-05-10T11:18:24.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.697+0000 7f171b01d640 1 -- 172.21.15.100:0/3034883279 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17141a3190 con 0x7f17141012e0 2024-05-10T11:18:24.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.698+0000 7f171b01d640 1 -- 172.21.15.100:0/3034883279 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f171410c360 con 0x7f17141012e0 2024-05-10T11:18:24.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.699+0000 7f1711ffb640 1 -- 172.21.15.100:0/3034883279 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f1704003b70 con 0x7f17141012e0 2024-05-10T11:18:24.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.700+0000 7f1711ffb640 1 --2- 172.21.15.100:0/3034883279 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f17000b27f0 0x7f17000b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:24.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.701+0000 7f1713fff640 1 --2- 172.21.15.100:0/3034883279 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f17000b27f0 0x7f17000b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:24.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.701+0000 7f1711ffb640 1 -- 172.21.15.100:0/3034883279 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f17040e2770 con 0x7f17141012e0 2024-05-10T11:18:24.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.701+0000 7f1713fff640 1 --2- 172.21.15.100:0/3034883279 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f17000b27f0 0x7f17000b4cb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f16fc0096c0 tx=0x7f16fc009650 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:24.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.705+0000 7f1711ffb640 1 -- 172.21.15.100:0/3034883279 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f17040aa130 con 0x7f17141012e0 2024-05-10T11:18:24.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.890+0000 7f171b01d640 1 -- 172.21.15.100:0/3034883279 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f171402db50 con 0x7f17141012e0 2024-05-10T11:18:24.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.890+0000 7f1711ffb640 1 -- 172.21.15.100:0/3034883279 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7f171402db50 con 0x7f17141012e0 2024-05-10T11:18:24.890 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 64 2024-05-10T11:18:24.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.892+0000 7f171b01d640 1 -- 172.21.15.100:0/3034883279 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f17000b27f0 msgr2=0x7f17000b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:24.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.892+0000 7f171b01d640 1 --2- 172.21.15.100:0/3034883279 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f17000b27f0 0x7f17000b4cb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f16fc0096c0 tx=0x7f16fc009650 comp rx=0 tx=0).stop 2024-05-10T11:18:24.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.892+0000 7f171b01d640 1 -- 172.21.15.100:0/3034883279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17141012e0 msgr2=0x7f171419ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:24.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.892+0000 7f171b01d640 1 --2- 172.21.15.100:0/3034883279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17141012e0 0x7f171419ade0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f170400db80 tx=0x7f170400b630 comp rx=0 tx=0).stop 2024-05-10T11:18:24.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.892+0000 7f171b01d640 1 -- 172.21.15.100:0/3034883279 shutdown_connections 2024-05-10T11:18:24.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.892+0000 7f171b01d640 1 --2- 172.21.15.100:0/3034883279 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f17000b27f0 0x7f17000b4cb0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:24.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.892+0000 7f171b01d640 1 --2- 172.21.15.100:0/3034883279 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17141025e0 0x7f17141a2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:24.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.892+0000 7f171b01d640 1 --2- 172.21.15.100:0/3034883279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1714101c40 0x7f171419b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:24.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.892+0000 7f171b01d640 1 --2- 172.21.15.100:0/3034883279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17141012e0 0x7f171419ade0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:24.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.892+0000 7f171b01d640 1 -- 172.21.15.100:0/3034883279 >> 172.21.15.100:0/3034883279 conn(0x7f17140fce60 msgr2=0x7f17140fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:24.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.892+0000 7f171b01d640 1 -- 172.21.15.100:0/3034883279 shutdown_connections 2024-05-10T11:18:24.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:24.892+0000 7f171b01d640 1 -- 172.21.15.100:0/3034883279 wait complete. 2024-05-10T11:18:24.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:18:24 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/1791103415' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-10T11:18:24.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:18:24 smithi100 ceph-mon[30296]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T11:18:24.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:18:24 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/4276641183' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T11:18:24.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:18:24 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/2494492725' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T11:18:24.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:18:24 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/3309664419' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T11:18:24.904 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-10T11:18:25.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:18:24 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/1791103415' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-10T11:18:25.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:18:24 smithi178 ceph-mon[33359]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T11:18:25.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:18:24 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/4276641183' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T11:18:25.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:18:24 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/2494492725' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T11:18:25.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:18:24 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/3309664419' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T11:18:25.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:25.007+0000 7fce0d8bd640 1 -- 172.21.15.100:0/2271083136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce08102a00 msgr2=0x7fce08102e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:25.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:25.007+0000 7fce0d8bd640 1 --2- 172.21.15.100:0/2271083136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce08102a00 0x7fce08102e20 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fcdf8009900 tx=0x7fcdf802a410 comp rx=0 tx=0).stop 2024-05-10T11:18:25.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:25.007+0000 7fce0d8bd640 1 -- 172.21.15.100:0/2271083136 shutdown_connections 2024-05-10T11:18:25.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:25.007+0000 7fce0d8bd640 1 --2- 172.21.15.100:0/2271083136 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce081045b0 0x7fce0810ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:25.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:25.007+0000 7fce0d8bd640 1 --2- 172.21.15.100:0/2271083136 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce08103bf0 0x7fce08104070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:25.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:25.007+0000 7fce0d8bd640 1 --2- 172.21.15.100:0/2271083136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce08102a00 0x7fce08102e20 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:25.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:25.007+0000 7fce0d8bd640 1 -- 172.21.15.100:0/2271083136 >> 172.21.15.100:0/2271083136 conn(0x7fce080fdf90 msgr2=0x7fce081003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:25.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:25.007+0000 7fce0d8bd640 1 -- 172.21.15.100:0/2271083136 shutdown_connections 2024-05-10T11:18:25.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:25.007+0000 7fce0d8bd640 1 -- 172.21.15.100:0/2271083136 wait complete. 2024-05-10T11:18:25.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:25.008+0000 7fce0d8bd640 1 Processor -- start 2024-05-10T11:18:25.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:25.008+0000 7fce0d8bd640 1 -- start start 2024-05-10T11:18:25.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:25.008+0000 7fce0d8bd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce08102a00 0x7fce08198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:25.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:25.008+0000 7fce0d8bd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce08103bf0 0x7fce081990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:25.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:25.008+0000 7fce0d8bd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce081045b0 0x7fce081a0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:25.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:25.008+0000 7fce0d8bd640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fce0810e3b0 con 0x7fce08103bf0 2024-05-10T11:18:25.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:25.008+0000 7fce0d8bd640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fce0810e230 con 0x7fce08102a00 2024-05-10T11:18:25.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:25.008+0000 7fce0d8bd640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fce0810e530 con 0x7fce081045b0 2024-05-10T11:18:25.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:25.008+0000 7fcdfffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce08103bf0 0x7fce081990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:25.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:25.008+0000 7fcdfffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce08103bf0 0x7fce081990e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60302/0 (socket says 172.21.15.100:60302) 2024-05-10T11:18:25.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:25.008+0000 7fcdfffff640 1 -- 172.21.15.100:0/901404566 learned_addr learned my addr 172.21.15.100:0/901404566 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:18:25.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:25.008+0000 7fce06ffd640 1 --2- 172.21.15.100:0/901404566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce08102a00 0x7fce08198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:25.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:25.008+0000 7fce077fe640 1 --2- 172.21.15.100:0/901404566 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce081045b0 0x7fce081a0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:25.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:25.008+0000 7fcdfffff640 1 -- 172.21.15.100:0/901404566 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce081045b0 msgr2=0x7fce081a0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:25.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:25.008+0000 7fcdfffff640 1 --2- 172.21.15.100:0/901404566 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce081045b0 0x7fce081a0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:25.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:25.008+0000 7fcdfffff640 1 -- 172.21.15.100:0/901404566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce08102a00 msgr2=0x7fce08198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:25.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:25.008+0000 7fcdfffff640 1 --2- 172.21.15.100:0/901404566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce08102a00 0x7fce08198ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:25.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:25.008+0000 7fcdfffff640 1 -- 172.21.15.100:0/901404566 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce081a07c0 con 0x7fce08103bf0 2024-05-10T11:18:25.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:25.008+0000 7fce06ffd640 1 --2- 172.21.15.100:0/901404566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce08102a00 0x7fce08198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:18:25.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:25.008+0000 7fce077fe640 1 --2- 172.21.15.100:0/901404566 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce081045b0 0x7fce081a0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:18:25.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:25.009+0000 7fcdfffff640 1 --2- 172.21.15.100:0/901404566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce08103bf0 0x7fce081990e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fcdf000d950 tx=0x7fcdf000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:25.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:25.009+0000 7fce0c8bb640 1 -- 172.21.15.100:0/901404566 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf0017070 con 0x7fce08103bf0 2024-05-10T11:18:25.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:25.009+0000 7fce0c8bb640 1 -- 172.21.15.100:0/901404566 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcdf0009c30 con 0x7fce08103bf0 2024-05-10T11:18:25.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:25.009+0000 7fce0d8bd640 1 -- 172.21.15.100:0/901404566 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce081a0ab0 con 0x7fce08103bf0 2024-05-10T11:18:25.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:25.009+0000 7fce0c8bb640 1 -- 172.21.15.100:0/901404566 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf000f620 con 0x7fce08103bf0 2024-05-10T11:18:25.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:25.009+0000 7fce0d8bd640 1 -- 172.21.15.100:0/901404566 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce081a1000 con 0x7fce08103bf0 2024-05-10T11:18:25.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:25.010+0000 7fce0d8bd640 1 -- 172.21.15.100:0/901404566 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce08109ee0 con 0x7fce08103bf0 2024-05-10T11:18:25.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:25.011+0000 7fce0c8bb640 1 -- 172.21.15.100:0/901404566 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fcdf0010460 con 0x7fce08103bf0 2024-05-10T11:18:25.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:25.012+0000 7fce0c8bb640 1 --2- 172.21.15.100:0/901404566 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fcdf40b27f0 0x7fcdf40b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:25.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:25.012+0000 7fce06ffd640 1 --2- 172.21.15.100:0/901404566 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fcdf40b27f0 0x7fcdf40b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:25.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:25.012+0000 7fce0c8bb640 1 -- 172.21.15.100:0/901404566 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fcdf00e2230 con 0x7fce08103bf0 2024-05-10T11:18:25.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:25.012+0000 7fce06ffd640 1 --2- 172.21.15.100:0/901404566 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fcdf40b27f0 0x7fcdf40b4cb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fcdf8006010 tx=0x7fcdf8031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:25.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:25.016+0000 7fce0c8bb640 1 -- 172.21.15.100:0/901404566 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fcdf00a9bf0 con 0x7fce08103bf0 2024-05-10T11:18:25.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:18:24 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/1791103415' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-10T11:18:25.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:18:24 smithi188 ceph-mon[34075]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T11:18:25.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:18:24 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/4276641183' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T11:18:25.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:18:24 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/2494492725' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T11:18:25.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:18:24 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/3309664419' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T11:18:25.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:25.198+0000 7fce0d8bd640 1 -- 172.21.15.100:0/901404566 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fce08063650 con 0x7fce08103bf0 2024-05-10T11:18:25.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:25.198+0000 7fce0c8bb640 1 -- 172.21.15.100:0/901404566 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) ==== 103+0+11 (secure 0 0 0) 0x7fcdf0009de0 con 0x7fce08103bf0 2024-05-10T11:18:25.198 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-05-10T11:18:25.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:25.200+0000 7fce0d8bd640 1 -- 172.21.15.100:0/901404566 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fcdf40b27f0 msgr2=0x7fcdf40b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:25.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:25.200+0000 7fce0d8bd640 1 --2- 172.21.15.100:0/901404566 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fcdf40b27f0 0x7fcdf40b4cb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fcdf8006010 tx=0x7fcdf8031400 comp rx=0 tx=0).stop 2024-05-10T11:18:25.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:25.200+0000 7fce0d8bd640 1 -- 172.21.15.100:0/901404566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce08103bf0 msgr2=0x7fce081990e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:25.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:25.200+0000 7fce0d8bd640 1 --2- 172.21.15.100:0/901404566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce08103bf0 0x7fce081990e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fcdf000d950 tx=0x7fcdf000de10 comp rx=0 tx=0).stop 2024-05-10T11:18:25.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:25.200+0000 7fce0d8bd640 1 -- 172.21.15.100:0/901404566 shutdown_connections 2024-05-10T11:18:25.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:25.200+0000 7fce0d8bd640 1 --2- 172.21.15.100:0/901404566 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fcdf40b27f0 0x7fcdf40b4cb0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:25.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:25.200+0000 7fce0d8bd640 1 --2- 172.21.15.100:0/901404566 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce081045b0 0x7fce081a0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:25.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:25.200+0000 7fce0d8bd640 1 --2- 172.21.15.100:0/901404566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce08103bf0 0x7fce081990e0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:25.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:25.200+0000 7fce0d8bd640 1 --2- 172.21.15.100:0/901404566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce08102a00 0x7fce08198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:25.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:25.200+0000 7fce0d8bd640 1 -- 172.21.15.100:0/901404566 >> 172.21.15.100:0/901404566 conn(0x7fce080fdf90 msgr2=0x7fce081070b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:25.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:25.200+0000 7fce0d8bd640 1 -- 172.21.15.100:0/901404566 shutdown_connections 2024-05-10T11:18:25.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:25.200+0000 7fce0d8bd640 1 -- 172.21.15.100:0/901404566 wait complete. 2024-05-10T11:18:25.211 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-05-10T11:18:25.211 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T11:18:25.459 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:18:25.832 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:18:25 smithi100 ceph-mon[30296]: pgmap v421: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-10T11:18:25.833 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:18:25 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/3034883279' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T11:18:25.833 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:18:25 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/901404566' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T11:18:26.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:18:25 smithi178 ceph-mon[33359]: pgmap v421: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-10T11:18:26.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:18:25 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/3034883279' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T11:18:26.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:18:25 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/901404566' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T11:18:26.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:18:25 smithi188 ceph-mon[34075]: pgmap v421: 129 pgs: 129 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-10T11:18:26.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:18:25 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/3034883279' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T11:18:26.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:18:25 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/901404566' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T11:18:27.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:27.244+0000 7f4e298d5700 1 -- 172.21.15.100:0/1026646108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e24103540 msgr2=0x7f4e241039c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:27.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:27.244+0000 7f4e298d5700 1 --2- 172.21.15.100:0/1026646108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e24103540 0x7f4e241039c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c0099c0 tx=0x7f4e0c009cd0 comp rx=0 tx=0).stop 2024-05-10T11:18:27.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:27.244+0000 7f4e298d5700 1 -- 172.21.15.100:0/1026646108 shutdown_connections 2024-05-10T11:18:27.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:27.244+0000 7f4e298d5700 1 --2- 172.21.15.100:0/1026646108 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e24103f00 0x7f4e241103b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:27.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:27.244+0000 7f4e298d5700 1 --2- 172.21.15.100:0/1026646108 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e241095e0 0x7f4e241099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:27.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:27.244+0000 7f4e298d5700 1 --2- 172.21.15.100:0/1026646108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e24103540 0x7f4e241039c0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:27.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:27.244+0000 7f4e298d5700 1 -- 172.21.15.100:0/1026646108 >> 172.21.15.100:0/1026646108 conn(0x7f4e2407a650 msgr2=0x7f4e2407aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:27.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:27.244+0000 7f4e298d5700 1 -- 172.21.15.100:0/1026646108 shutdown_connections 2024-05-10T11:18:27.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:27.244+0000 7f4e298d5700 1 -- 172.21.15.100:0/1026646108 wait complete. 2024-05-10T11:18:27.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:27.245+0000 7f4e298d5700 1 Processor -- start 2024-05-10T11:18:27.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:27.245+0000 7f4e298d5700 1 -- start start 2024-05-10T11:18:27.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:27.245+0000 7f4e298d5700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e24103540 0x7f4e2419f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:27.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:27.245+0000 7f4e298d5700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e24103f00 0x7f4e2419fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:27.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:27.245+0000 7f4e298d5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e241095e0 0x7f4e241998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:27.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:27.245+0000 7f4e298d5700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4e241135f0 con 0x7f4e24103f00 2024-05-10T11:18:27.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:27.245+0000 7f4e298d5700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4e24113470 con 0x7f4e241095e0 2024-05-10T11:18:27.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:27.245+0000 7f4e298d5700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4e24113770 con 0x7f4e24103540 2024-05-10T11:18:27.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:27.245+0000 7f4e227fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e24103f00 0x7f4e2419fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:27.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:27.245+0000 7f4e227fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e24103f00 0x7f4e2419fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42240/0 (socket says 172.21.15.100:42240) 2024-05-10T11:18:27.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:27.245+0000 7f4e227fc700 1 -- 172.21.15.100:0/1839723861 learned_addr learned my addr 172.21.15.100:0/1839723861 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:18:27.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:27.246+0000 7f4e237fe700 1 --2- 172.21.15.100:0/1839723861 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e241095e0 0x7f4e241998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:27.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:27.246+0000 7f4e22ffd700 1 --2- 172.21.15.100:0/1839723861 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e24103540 0x7f4e2419f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:27.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:27.246+0000 7f4e227fc700 1 -- 172.21.15.100:0/1839723861 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e24103540 msgr2=0x7f4e2419f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:27.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:27.246+0000 7f4e227fc700 1 --2- 172.21.15.100:0/1839723861 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e24103540 0x7f4e2419f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:27.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:27.246+0000 7f4e227fc700 1 -- 172.21.15.100:0/1839723861 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e241095e0 msgr2=0x7f4e241998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:27.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:27.246+0000 7f4e227fc700 1 --2- 172.21.15.100:0/1839723861 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e241095e0 0x7f4e241998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:27.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:27.246+0000 7f4e227fc700 1 -- 172.21.15.100:0/1839723861 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e2419a050 con 0x7f4e24103f00 2024-05-10T11:18:27.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:27.246+0000 7f4e227fc700 1 --2- 172.21.15.100:0/1839723861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e24103f00 0x7f4e2419fc70 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c000c00 tx=0x7f4e0c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:27.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:27.246+0000 7f4e288d3700 1 -- 172.21.15.100:0/1839723861 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e0c026070 con 0x7f4e24103f00 2024-05-10T11:18:27.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:27.246+0000 7f4e298d5700 1 -- 172.21.15.100:0/1839723861 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e2419a2e0 con 0x7f4e24103f00 2024-05-10T11:18:27.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:27.246+0000 7f4e288d3700 1 -- 172.21.15.100:0/1839723861 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4e0c005030 con 0x7f4e24103f00 2024-05-10T11:18:27.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:27.246+0000 7f4e288d3700 1 -- 172.21.15.100:0/1839723861 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e0c004b70 con 0x7f4e24103f00 2024-05-10T11:18:27.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:27.246+0000 7f4e298d5700 1 -- 172.21.15.100:0/1839723861 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e241a6600 con 0x7f4e24103f00 2024-05-10T11:18:27.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:27.247+0000 7f4e298d5700 1 -- 172.21.15.100:0/1839723861 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e2410ba70 con 0x7f4e24103f00 2024-05-10T11:18:27.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:27.248+0000 7f4e288d3700 1 -- 172.21.15.100:0/1839723861 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f4e0c029070 con 0x7f4e24103f00 2024-05-10T11:18:27.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:27.249+0000 7f4e288d3700 1 --2- 172.21.15.100:0/1839723861 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4e100b2880 0x7f4e100b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:27.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:27.250+0000 7f4e22ffd700 1 --2- 172.21.15.100:0/1839723861 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4e100b2880 0x7f4e100b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:27.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:27.250+0000 7f4e288d3700 1 -- 172.21.15.100:0/1839723861 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f4e0c0e34f0 con 0x7f4e24103f00 2024-05-10T11:18:27.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:27.250+0000 7f4e22ffd700 1 --2- 172.21.15.100:0/1839723861 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4e100b2880 0x7f4e100b4d40 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4e2419aa30 tx=0x7f4e1c0094f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:27.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:27.253+0000 7f4e288d3700 1 -- 172.21.15.100:0/1839723861 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f4e0c030020 con 0x7f4e24103f00 2024-05-10T11:18:27.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:27.502+0000 7f4e298d5700 1 -- 172.21.15.100:0/1839723861 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4e24104c50 con 0x7f4e24103f00 2024-05-10T11:18:27.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:27.502+0000 7f4e288d3700 1 -- 172.21.15.100:0/1839723861 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11894 (secure 0 0 0) 0x7f4e0c013070 con 0x7f4e24103f00 2024-05-10T11:18:27.503 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:18:27.503 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2072225012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2072225012},{"type":"v1","addr":"172.21.15.100:6835","nonce":2072225012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/401939540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":401939540},{"type":"v1","addr":"172.21.15.100:6837","nonce":401939540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14571,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/3946311025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3946311025},{"type":"v1","addr":"172.21.15.100:6839","nonce":3946311025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3736371660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3736371660},{"type":"v1","addr":"172.21.15.100:6841","nonce":3736371660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/2741810791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2741810791},{"type":"v1","addr":"172.21.15.178:6833","nonce":2741810791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1704891041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1704891041},{"type":"v1","addr":"172.21.15.188:6833","nonce":1704891041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T11:14:27.242452+0000","modified":"2024-05-10T11:18:23.454860+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24439,"mds_1":24437,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/2124076312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2124076312},{"type":"v1","addr":"172.21.15.178:6835","nonce":2124076312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/3224358425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3224358425},{"type":"v1","addr":"172.21.15.178:6837","nonce":3224358425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/515547690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":515547690},{"type":"v1","addr":"172.21.15.188:6835","nonce":515547690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/1943342620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1943342620},{"type":"v1","addr":"172.21.15.188:6837","nonce":1943342620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.178:6839/3760985404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3760985404},{"type":"v1","addr":"172.21.15.178:6839","nonce":3760985404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.188:6839/3194917070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":3194917070},{"type":"v1","addr":"172.21.15.188:6839","nonce":3194917070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24439,"qdb_cluster":[24427,24437,24439]},"id":1}]} 2024-05-10T11:18:27.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:27.505+0000 7f4e298d5700 1 -- 172.21.15.100:0/1839723861 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4e100b2880 msgr2=0x7f4e100b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:27.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:27.505+0000 7f4e298d5700 1 --2- 172.21.15.100:0/1839723861 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4e100b2880 0x7f4e100b4d40 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4e2419aa30 tx=0x7f4e1c0094f0 comp rx=0 tx=0).stop 2024-05-10T11:18:27.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:27.505+0000 7f4e298d5700 1 -- 172.21.15.100:0/1839723861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e24103f00 msgr2=0x7f4e2419fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:27.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:27.505+0000 7f4e298d5700 1 --2- 172.21.15.100:0/1839723861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e24103f00 0x7f4e2419fc70 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c000c00 tx=0x7f4e0c00f690 comp rx=0 tx=0).stop 2024-05-10T11:18:27.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:27.505+0000 7f4e298d5700 1 -- 172.21.15.100:0/1839723861 shutdown_connections 2024-05-10T11:18:27.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:27.505+0000 7f4e298d5700 1 --2- 172.21.15.100:0/1839723861 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4e100b2880 0x7f4e100b4d40 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:27.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:27.505+0000 7f4e298d5700 1 --2- 172.21.15.100:0/1839723861 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e241095e0 0x7f4e241998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:27.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:27.505+0000 7f4e298d5700 1 --2- 172.21.15.100:0/1839723861 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e24103540 0x7f4e2419f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:27.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:27.505+0000 7f4e298d5700 1 --2- 172.21.15.100:0/1839723861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e24103f00 0x7f4e2419fc70 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:27.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:27.505+0000 7f4e298d5700 1 -- 172.21.15.100:0/1839723861 >> 172.21.15.100:0/1839723861 conn(0x7f4e2407a650 msgr2=0x7f4e24102360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:27.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:27.505+0000 7f4e298d5700 1 -- 172.21.15.100:0/1839723861 shutdown_connections 2024-05-10T11:18:27.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:27.505+0000 7f4e298d5700 1 -- 172.21.15.100:0/1839723861 wait complete. 2024-05-10T11:18:27.506 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 27 2024-05-10T11:18:27.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:18:27 smithi100 ceph-mon[30296]: pgmap v422: 129 pgs: 129 active+clean; 645 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 6 op/s 2024-05-10T11:18:27.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:18:27 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/1839723861' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:18:28.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:18:27 smithi178 ceph-mon[33359]: pgmap v422: 129 pgs: 129 active+clean; 645 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 6 op/s 2024-05-10T11:18:28.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:18:27 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/1839723861' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:18:28.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:18:27 smithi188 ceph-mon[34075]: pgmap v422: 129 pgs: 129 active+clean; 645 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 6 op/s 2024-05-10T11:18:28.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:18:27 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/1839723861' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:18:28.274 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-05-10T11:18:28.274 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-05-10T11:18:28.274 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T11:18:28.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.378+0000 7f607af80640 1 -- 172.21.15.100:0/784076207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6074102ad0 msgr2=0x7f6074102f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:28.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.378+0000 7f607af80640 1 --2- 172.21.15.100:0/784076207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6074102ad0 0x7f6074102f50 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6060009900 tx=0x7f606002a410 comp rx=0 tx=0).stop 2024-05-10T11:18:28.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.378+0000 7f607af80640 1 -- 172.21.15.100:0/784076207 shutdown_connections 2024-05-10T11:18:28.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.378+0000 7f607af80640 1 --2- 172.21.15.100:0/784076207 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6074103490 0x7f6074109d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:28.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.378+0000 7f607af80640 1 --2- 172.21.15.100:0/784076207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6074102ad0 0x7f6074102f50 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:28.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.378+0000 7f607af80640 1 --2- 172.21.15.100:0/784076207 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60741018e0 0x7f6074101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:28.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.378+0000 7f607af80640 1 -- 172.21.15.100:0/784076207 >> 172.21.15.100:0/784076207 conn(0x7f60740fce70 msgr2=0x7f60740ff290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:28.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.378+0000 7f607af80640 1 -- 172.21.15.100:0/784076207 shutdown_connections 2024-05-10T11:18:28.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.378+0000 7f607af80640 1 -- 172.21.15.100:0/784076207 wait complete. 2024-05-10T11:18:28.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.379+0000 7f607af80640 1 Processor -- start 2024-05-10T11:18:28.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.379+0000 7f607af80640 1 -- start start 2024-05-10T11:18:28.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.379+0000 7f607af80640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60741018e0 0x7f6074197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:28.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.379+0000 7f607af80640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6074102ad0 0x7f6074197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:28.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.379+0000 7f607af80640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6074103490 0x7f607419efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:28.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.379+0000 7f607af80640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f607410d2c0 con 0x7f6074102ad0 2024-05-10T11:18:28.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.379+0000 7f607af80640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f607410d140 con 0x7f6074103490 2024-05-10T11:18:28.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.379+0000 7f607af80640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f607410d440 con 0x7f60741018e0 2024-05-10T11:18:28.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.379+0000 7f6073fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6074102ad0 0x7f6074197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:28.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.379+0000 7f6073fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6074102ad0 0x7f6074197f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42258/0 (socket says 172.21.15.100:42258) 2024-05-10T11:18:28.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.379+0000 7f6073fff640 1 -- 172.21.15.100:0/938922931 learned_addr learned my addr 172.21.15.100:0/938922931 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:18:28.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.379+0000 7f6073fff640 1 -- 172.21.15.100:0/938922931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60741018e0 msgr2=0x7f6074197a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:28.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.379+0000 7f6078d1c640 1 --2- 172.21.15.100:0/938922931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60741018e0 0x7f6074197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:28.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.379+0000 7f6073fff640 1 --2- 172.21.15.100:0/938922931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60741018e0 0x7f6074197a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:28.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.379+0000 7f607951d640 1 --2- 172.21.15.100:0/938922931 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6074103490 0x7f607419efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:28.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.379+0000 7f6073fff640 1 -- 172.21.15.100:0/938922931 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6074103490 msgr2=0x7f607419efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:28.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.379+0000 7f6073fff640 1 --2- 172.21.15.100:0/938922931 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6074103490 0x7f607419efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:28.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.380+0000 7f6073fff640 1 -- 172.21.15.100:0/938922931 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f607419f630 con 0x7f6074102ad0 2024-05-10T11:18:28.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.380+0000 7f6078d1c640 1 --2- 172.21.15.100:0/938922931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60741018e0 0x7f6074197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:18:28.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.380+0000 7f6073fff640 1 --2- 172.21.15.100:0/938922931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6074102ad0 0x7f6074197f50 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f6060009230 tx=0x7f60600042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:28.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.380+0000 7f6071ffb640 1 -- 172.21.15.100:0/938922931 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f606003f030 con 0x7f6074102ad0 2024-05-10T11:18:28.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.380+0000 7f6071ffb640 1 -- 172.21.15.100:0/938922931 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6060032e10 con 0x7f6074102ad0 2024-05-10T11:18:28.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.380+0000 7f607af80640 1 -- 172.21.15.100:0/938922931 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f607419f8c0 con 0x7f6074102ad0 2024-05-10T11:18:28.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.380+0000 7f6071ffb640 1 -- 172.21.15.100:0/938922931 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6060035070 con 0x7f6074102ad0 2024-05-10T11:18:28.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.380+0000 7f607af80640 1 -- 172.21.15.100:0/938922931 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f607419fdb0 con 0x7f6074102ad0 2024-05-10T11:18:28.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.381+0000 7f607af80640 1 -- 172.21.15.100:0/938922931 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6074108cd0 con 0x7f6074102ad0 2024-05-10T11:18:28.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.382+0000 7f6071ffb640 1 -- 172.21.15.100:0/938922931 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f6060042030 con 0x7f6074102ad0 2024-05-10T11:18:28.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.383+0000 7f6071ffb640 1 --2- 172.21.15.100:0/938922931 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f60640b27f0 0x7f60640b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:28.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.383+0000 7f6078d1c640 1 --2- 172.21.15.100:0/938922931 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f60640b27f0 0x7f60640b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:28.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.383+0000 7f6071ffb640 1 -- 172.21.15.100:0/938922931 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f6060102c40 con 0x7f6074102ad0 2024-05-10T11:18:28.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.383+0000 7f6078d1c640 1 --2- 172.21.15.100:0/938922931 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f60640b27f0 0x7f60640b4cb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6074102930 tx=0x7f6068005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:28.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.387+0000 7f6071ffb640 1 -- 172.21.15.100:0/938922931 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f60600ca600 con 0x7f6074102ad0 2024-05-10T11:18:28.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.570+0000 7f607af80640 1 -- 172.21.15.100:0/938922931 --> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6074062df0 con 0x7f60640b27f0 2024-05-10T11:18:28.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.579+0000 7f6071ffb640 1 -- 172.21.15.100:0/938922931 <== mgr.14152 v2:172.21.15.100:6800/2507287356 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340476 (secure 0 0 0) 0x7f6074062df0 con 0x7f60640b27f0 2024-05-10T11:18:28.583 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:18:28.585 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-05-10T11:18:28.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.583+0000 7f607af80640 1 -- 172.21.15.100:0/938922931 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f60640b27f0 msgr2=0x7f60640b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:28.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.583+0000 7f607af80640 1 --2- 172.21.15.100:0/938922931 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f60640b27f0 0x7f60640b4cb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6074102930 tx=0x7f6068005f90 comp rx=0 tx=0).stop 2024-05-10T11:18:28.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.583+0000 7f607af80640 1 -- 172.21.15.100:0/938922931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6074102ad0 msgr2=0x7f6074197f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:28.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.583+0000 7f607af80640 1 --2- 172.21.15.100:0/938922931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6074102ad0 0x7f6074197f50 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f6060009230 tx=0x7f60600042f0 comp rx=0 tx=0).stop 2024-05-10T11:18:28.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.583+0000 7f607af80640 1 -- 172.21.15.100:0/938922931 shutdown_connections 2024-05-10T11:18:28.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.583+0000 7f607af80640 1 --2- 172.21.15.100:0/938922931 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f60640b27f0 0x7f60640b4cb0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:28.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.583+0000 7f607af80640 1 --2- 172.21.15.100:0/938922931 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6074103490 0x7f607419efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:28.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.583+0000 7f607af80640 1 --2- 172.21.15.100:0/938922931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6074102ad0 0x7f6074197f50 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:28.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.583+0000 7f607af80640 1 --2- 172.21.15.100:0/938922931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60741018e0 0x7f6074197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:28.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.583+0000 7f607af80640 1 -- 172.21.15.100:0/938922931 >> 172.21.15.100:0/938922931 conn(0x7f60740fce70 msgr2=0x7f6074105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:28.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.583+0000 7f607af80640 1 -- 172.21.15.100:0/938922931 shutdown_connections 2024-05-10T11:18:28.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.583+0000 7f607af80640 1 -- 172.21.15.100:0/938922931 wait complete. 2024-05-10T11:18:28.594 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":423,"stamp":"2024-05-10T11:18:28.248281+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665454,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1153,"num_read_kb":684,"num_write":315,"num_write_kb":2292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":756,"ondisk_log_size":756,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":404716,"kb_used_data":13964,"kb_used_omap":18,"kb_used_meta":390637,"kb_avail":1124537108,"statfs":{"total":1151940427776,"available":1151525998592,"internally_reserved":0,"allocated":14299136,"data_stored":8033168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":400012603},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5793,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":84,"num_read_kb":53,"num_write":11,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003400"},"pg_stats":[{"pgid":"3.3f","version":"89'8","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.206983+0000","last_change":"2024-05-10T11:14:25.286878+0000","last_active":"2024-05-10T11:17:00.206983+0000","last_peered":"2024-05-10T11:17:00.206983+0000","last_clean":"2024-05-10T11:17:00.206983+0000","last_became_active":"2024-05-10T11:14:25.286462+0000","last_became_peered":"2024-05-10T11:14:25.286462+0000","last_unstale":"2024-05-10T11:17:00.206983+0000","last_undegraded":"2024-05-10T11:17:00.206983+0000","last_fullsized":"2024-05-10T11:17:00.206983+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:13:40.627138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.204906+0000","last_change":"2024-05-10T11:14:25.280884+0000","last_active":"2024-05-10T11:17:00.204906+0000","last_peered":"2024-05-10T11:17:00.204906+0000","last_clean":"2024-05-10T11:17:00.204906+0000","last_became_active":"2024-05-10T11:14:25.280564+0000","last_became_peered":"2024-05-10T11:14:25.280564+0000","last_unstale":"2024-05-10T11:17:00.204906+0000","last_undegraded":"2024-05-10T11:17:00.204906+0000","last_fullsized":"2024-05-10T11:17:00.204906+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:14:16.735326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.208402+0000","last_change":"2024-05-10T11:14:25.278676+0000","last_active":"2024-05-10T11:17:00.208402+0000","last_peered":"2024-05-10T11:17:00.208402+0000","last_clean":"2024-05-10T11:17:00.208402+0000","last_became_active":"2024-05-10T11:14:25.278196+0000","last_became_peered":"2024-05-10T11:14:25.278196+0000","last_unstale":"2024-05-10T11:17:00.208402+0000","last_undegraded":"2024-05-10T11:17:00.208402+0000","last_fullsized":"2024-05-10T11:17:00.208402+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:50:29.390014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.207328+0000","last_change":"2024-05-10T11:14:25.274741+0000","last_active":"2024-05-10T11:17:00.207328+0000","last_peered":"2024-05-10T11:17:00.207328+0000","last_clean":"2024-05-10T11:17:00.207328+0000","last_became_active":"2024-05-10T11:14:25.274251+0000","last_became_peered":"2024-05-10T11:14:25.274251+0000","last_unstale":"2024-05-10T11:17:00.207328+0000","last_undegraded":"2024-05-10T11:17:00.207328+0000","last_fullsized":"2024-05-10T11:17:00.207328+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:04:41.552042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.186404+0000","last_change":"2024-05-10T11:14:25.265232+0000","last_active":"2024-05-10T11:17:00.186404+0000","last_peered":"2024-05-10T11:17:00.186404+0000","last_clean":"2024-05-10T11:17:00.186404+0000","last_became_active":"2024-05-10T11:14:25.264852+0000","last_became_peered":"2024-05-10T11:14:25.264852+0000","last_unstale":"2024-05-10T11:17:00.186404+0000","last_undegraded":"2024-05-10T11:17:00.186404+0000","last_fullsized":"2024-05-10T11:17:00.186404+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:50:27.207258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.210357+0000","last_change":"2024-05-10T11:14:25.272626+0000","last_active":"2024-05-10T11:17:00.210357+0000","last_peered":"2024-05-10T11:17:00.210357+0000","last_clean":"2024-05-10T11:17:00.210357+0000","last_became_active":"2024-05-10T11:14:25.272305+0000","last_became_peered":"2024-05-10T11:14:25.272305+0000","last_unstale":"2024-05-10T11:17:00.210357+0000","last_undegraded":"2024-05-10T11:17:00.210357+0000","last_fullsized":"2024-05-10T11:17:00.210357+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:44:16.725453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.210435+0000","last_change":"2024-05-10T11:14:25.275460+0000","last_active":"2024-05-10T11:17:00.210435+0000","last_peered":"2024-05-10T11:17:00.210435+0000","last_clean":"2024-05-10T11:17:00.210435+0000","last_became_active":"2024-05-10T11:14:25.274571+0000","last_became_peered":"2024-05-10T11:14:25.274571+0000","last_unstale":"2024-05-10T11:17:00.210435+0000","last_undegraded":"2024-05-10T11:17:00.210435+0000","last_fullsized":"2024-05-10T11:17:00.210435+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:33:20.335738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.207412+0000","last_change":"2024-05-10T11:14:25.273975+0000","last_active":"2024-05-10T11:17:00.207412+0000","last_peered":"2024-05-10T11:17:00.207412+0000","last_clean":"2024-05-10T11:17:00.207412+0000","last_became_active":"2024-05-10T11:14:25.273652+0000","last_became_peered":"2024-05-10T11:14:25.273652+0000","last_unstale":"2024-05-10T11:17:00.207412+0000","last_undegraded":"2024-05-10T11:17:00.207412+0000","last_fullsized":"2024-05-10T11:17:00.207412+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:37:55.354449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.207494+0000","last_change":"2024-05-10T11:14:25.285530+0000","last_active":"2024-05-10T11:17:00.207494+0000","last_peered":"2024-05-10T11:17:00.207494+0000","last_clean":"2024-05-10T11:17:00.207494+0000","last_became_active":"2024-05-10T11:14:25.285235+0000","last_became_peered":"2024-05-10T11:14:25.285235+0000","last_unstale":"2024-05-10T11:17:00.207494+0000","last_undegraded":"2024-05-10T11:17:00.207494+0000","last_fullsized":"2024-05-10T11:17:00.207494+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:11:14.225339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.210587+0000","last_change":"2024-05-10T11:14:25.274738+0000","last_active":"2024-05-10T11:17:00.210587+0000","last_peered":"2024-05-10T11:17:00.210587+0000","last_clean":"2024-05-10T11:17:00.210587+0000","last_became_active":"2024-05-10T11:14:25.274107+0000","last_became_peered":"2024-05-10T11:14:25.274107+0000","last_unstale":"2024-05-10T11:17:00.210587+0000","last_undegraded":"2024-05-10T11:17:00.210587+0000","last_fullsized":"2024-05-10T11:17:00.210587+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:02:01.489930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.213489+0000","last_change":"2024-05-10T11:14:25.277431+0000","last_active":"2024-05-10T11:17:00.213489+0000","last_peered":"2024-05-10T11:17:00.213489+0000","last_clean":"2024-05-10T11:17:00.213489+0000","last_became_active":"2024-05-10T11:14:25.277130+0000","last_became_peered":"2024-05-10T11:14:25.277130+0000","last_unstale":"2024-05-10T11:17:00.213489+0000","last_undegraded":"2024-05-10T11:17:00.213489+0000","last_fullsized":"2024-05-10T11:17:00.213489+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:38:17.192282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.206976+0000","last_change":"2024-05-10T11:14:25.276291+0000","last_active":"2024-05-10T11:17:00.206976+0000","last_peered":"2024-05-10T11:17:00.206976+0000","last_clean":"2024-05-10T11:17:00.206976+0000","last_became_active":"2024-05-10T11:14:25.275842+0000","last_became_peered":"2024-05-10T11:14:25.275842+0000","last_unstale":"2024-05-10T11:17:00.206976+0000","last_undegraded":"2024-05-10T11:17:00.206976+0000","last_fullsized":"2024-05-10T11:17:00.206976+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:20:53.443135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.209964+0000","last_change":"2024-05-10T11:14:25.273950+0000","last_active":"2024-05-10T11:17:00.209964+0000","last_peered":"2024-05-10T11:17:00.209964+0000","last_clean":"2024-05-10T11:17:00.209964+0000","last_became_active":"2024-05-10T11:14:25.273644+0000","last_became_peered":"2024-05-10T11:14:25.273644+0000","last_unstale":"2024-05-10T11:17:00.209964+0000","last_undegraded":"2024-05-10T11:17:00.209964+0000","last_fullsized":"2024-05-10T11:17:00.209964+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:46:05.112575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.209077+0000","last_change":"2024-05-10T11:14:25.277930+0000","last_active":"2024-05-10T11:17:00.209077+0000","last_peered":"2024-05-10T11:17:00.209077+0000","last_clean":"2024-05-10T11:17:00.209077+0000","last_became_active":"2024-05-10T11:14:25.276793+0000","last_became_peered":"2024-05-10T11:14:25.276793+0000","last_unstale":"2024-05-10T11:17:00.209077+0000","last_undegraded":"2024-05-10T11:17:00.209077+0000","last_fullsized":"2024-05-10T11:17:00.209077+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:47:03.592882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.214057+0000","last_change":"2024-05-10T11:14:25.270434+0000","last_active":"2024-05-10T11:17:00.214057+0000","last_peered":"2024-05-10T11:17:00.214057+0000","last_clean":"2024-05-10T11:17:00.214057+0000","last_became_active":"2024-05-10T11:14:25.270139+0000","last_became_peered":"2024-05-10T11:14:25.270139+0000","last_unstale":"2024-05-10T11:17:00.214057+0000","last_undegraded":"2024-05-10T11:17:00.214057+0000","last_fullsized":"2024-05-10T11:17:00.214057+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:58:40.506560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.209206+0000","last_change":"2024-05-10T11:14:25.272981+0000","last_active":"2024-05-10T11:17:00.209206+0000","last_peered":"2024-05-10T11:17:00.209206+0000","last_clean":"2024-05-10T11:17:00.209206+0000","last_became_active":"2024-05-10T11:14:25.272592+0000","last_became_peered":"2024-05-10T11:14:25.272592+0000","last_unstale":"2024-05-10T11:17:00.209206+0000","last_undegraded":"2024-05-10T11:17:00.209206+0000","last_fullsized":"2024-05-10T11:17:00.209206+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:46:09.673013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.212493+0000","last_change":"2024-05-10T11:14:25.287304+0000","last_active":"2024-05-10T11:17:00.212493+0000","last_peered":"2024-05-10T11:17:00.212493+0000","last_clean":"2024-05-10T11:17:00.212493+0000","last_became_active":"2024-05-10T11:14:25.286964+0000","last_became_peered":"2024-05-10T11:14:25.286964+0000","last_unstale":"2024-05-10T11:17:00.212493+0000","last_undegraded":"2024-05-10T11:17:00.212493+0000","last_fullsized":"2024-05-10T11:17:00.212493+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:32:27.006840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.209855+0000","last_change":"2024-05-10T11:14:25.275366+0000","last_active":"2024-05-10T11:17:00.209855+0000","last_peered":"2024-05-10T11:17:00.209855+0000","last_clean":"2024-05-10T11:17:00.209855+0000","last_became_active":"2024-05-10T11:14:25.274986+0000","last_became_peered":"2024-05-10T11:14:25.274986+0000","last_unstale":"2024-05-10T11:17:00.209855+0000","last_undegraded":"2024-05-10T11:17:00.209855+0000","last_fullsized":"2024-05-10T11:17:00.209855+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:50:50.071975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.203474+0000","last_change":"2024-05-10T11:14:25.274588+0000","last_active":"2024-05-10T11:17:00.203474+0000","last_peered":"2024-05-10T11:17:00.203474+0000","last_clean":"2024-05-10T11:17:00.203474+0000","last_became_active":"2024-05-10T11:14:25.274179+0000","last_became_peered":"2024-05-10T11:14:25.274179+0000","last_unstale":"2024-05-10T11:17:00.203474+0000","last_undegraded":"2024-05-10T11:17:00.203474+0000","last_fullsized":"2024-05-10T11:17:00.203474+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:06:23.517082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.461771+0000","last_change":"2024-05-10T11:15:01.461337+0000","last_active":"2024-05-10T11:15:01.461771+0000","last_peered":"2024-05-10T11:15:01.461771+0000","last_clean":"2024-05-10T11:15:01.461771+0000","last_became_active":"2024-05-10T11:15:01.460776+0000","last_became_peered":"2024-05-10T11:15:01.460776+0000","last_unstale":"2024-05-10T11:15:01.461771+0000","last_undegraded":"2024-05-10T11:15:01.461771+0000","last_fullsized":"2024-05-10T11:15:01.461771+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:06:49.156483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,10,0],"acting":[2,3,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:32.813572+0000","last_change":"2024-05-10T11:14:22.253909+0000","last_active":"2024-05-10T11:15:32.813572+0000","last_peered":"2024-05-10T11:15:32.813572+0000","last_clean":"2024-05-10T11:15:32.813572+0000","last_became_active":"2024-05-10T11:14:22.253400+0000","last_became_peered":"2024-05-10T11:14:22.253400+0000","last_unstale":"2024-05-10T11:15:32.813572+0000","last_undegraded":"2024-05-10T11:15:32.813572+0000","last_fullsized":"2024-05-10T11:15:32.813572+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:06:46.173511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.206338+0000","last_change":"2024-05-10T11:14:25.273675+0000","last_active":"2024-05-10T11:17:00.206338+0000","last_peered":"2024-05-10T11:17:00.206338+0000","last_clean":"2024-05-10T11:17:00.206338+0000","last_became_active":"2024-05-10T11:14:25.273320+0000","last_became_peered":"2024-05-10T11:14:25.273320+0000","last_unstale":"2024-05-10T11:17:00.206338+0000","last_undegraded":"2024-05-10T11:17:00.206338+0000","last_fullsized":"2024-05-10T11:17:00.206338+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:09:18.232236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459008+0000","last_change":"2024-05-10T11:14:35.326204+0000","last_active":"2024-05-10T11:15:01.459008+0000","last_peered":"2024-05-10T11:15:01.459008+0000","last_clean":"2024-05-10T11:15:01.459008+0000","last_became_active":"2024-05-10T11:14:35.325715+0000","last_became_peered":"2024-05-10T11:14:35.325715+0000","last_unstale":"2024-05-10T11:15:01.459008+0000","last_undegraded":"2024-05-10T11:15:01.459008+0000","last_fullsized":"2024-05-10T11:15:01.459008+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:06:30.665861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:16:23.904221+0000","last_change":"2024-05-10T11:14:22.299194+0000","last_active":"2024-05-10T11:16:23.904221+0000","last_peered":"2024-05-10T11:16:23.904221+0000","last_clean":"2024-05-10T11:16:23.904221+0000","last_became_active":"2024-05-10T11:14:22.298807+0000","last_became_peered":"2024-05-10T11:14:22.298807+0000","last_unstale":"2024-05-10T11:16:23.904221+0000","last_undegraded":"2024-05-10T11:16:23.904221+0000","last_fullsized":"2024-05-10T11:16:23.904221+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:24:51.054834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.209768+0000","last_change":"2024-05-10T11:14:25.277860+0000","last_active":"2024-05-10T11:17:00.209768+0000","last_peered":"2024-05-10T11:17:00.209768+0000","last_clean":"2024-05-10T11:17:00.209768+0000","last_became_active":"2024-05-10T11:14:25.276784+0000","last_became_peered":"2024-05-10T11:14:25.276784+0000","last_unstale":"2024-05-10T11:17:00.209768+0000","last_undegraded":"2024-05-10T11:17:00.209768+0000","last_fullsized":"2024-05-10T11:17:00.209768+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:42:57.673595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459304+0000","last_change":"2024-05-10T11:14:35.330679+0000","last_active":"2024-05-10T11:15:01.459304+0000","last_peered":"2024-05-10T11:15:01.459304+0000","last_clean":"2024-05-10T11:15:01.459304+0000","last_became_active":"2024-05-10T11:14:35.329819+0000","last_became_peered":"2024-05-10T11:14:35.329819+0000","last_unstale":"2024-05-10T11:15:01.459304+0000","last_undegraded":"2024-05-10T11:15:01.459304+0000","last_fullsized":"2024-05-10T11:15:01.459304+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:31:26.982784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:29.798740+0000","last_change":"2024-05-10T11:14:22.260402+0000","last_active":"2024-05-10T11:15:29.798740+0000","last_peered":"2024-05-10T11:15:29.798740+0000","last_clean":"2024-05-10T11:15:29.798740+0000","last_became_active":"2024-05-10T11:14:22.259619+0000","last_became_peered":"2024-05-10T11:14:22.259619+0000","last_unstale":"2024-05-10T11:15:29.798740+0000","last_undegraded":"2024-05-10T11:15:29.798740+0000","last_fullsized":"2024-05-10T11:15:29.798740+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:56:34.491626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.204002+0000","last_change":"2024-05-10T11:14:25.276117+0000","last_active":"2024-05-10T11:17:00.204002+0000","last_peered":"2024-05-10T11:17:00.204002+0000","last_clean":"2024-05-10T11:17:00.204002+0000","last_became_active":"2024-05-10T11:14:25.275758+0000","last_became_peered":"2024-05-10T11:14:25.275758+0000","last_unstale":"2024-05-10T11:17:00.204002+0000","last_undegraded":"2024-05-10T11:17:00.204002+0000","last_fullsized":"2024-05-10T11:17:00.204002+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:12:16.582244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459213+0000","last_change":"2024-05-10T11:14:35.324633+0000","last_active":"2024-05-10T11:15:01.459213+0000","last_peered":"2024-05-10T11:15:01.459213+0000","last_clean":"2024-05-10T11:15:01.459213+0000","last_became_active":"2024-05-10T11:14:35.324152+0000","last_became_peered":"2024-05-10T11:14:35.324152+0000","last_unstale":"2024-05-10T11:15:01.459213+0000","last_undegraded":"2024-05-10T11:15:01.459213+0000","last_fullsized":"2024-05-10T11:15:01.459213+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:56:50.008782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.430353+0000","last_change":"2024-05-10T11:14:22.259086+0000","last_active":"2024-05-10T11:15:01.430353+0000","last_peered":"2024-05-10T11:15:01.430353+0000","last_clean":"2024-05-10T11:15:01.430353+0000","last_became_active":"2024-05-10T11:14:22.258628+0000","last_became_peered":"2024-05-10T11:14:22.258628+0000","last_unstale":"2024-05-10T11:15:01.430353+0000","last_undegraded":"2024-05-10T11:15:01.430353+0000","last_fullsized":"2024-05-10T11:15:01.430353+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:39:13.472622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.213935+0000","last_change":"2024-05-10T11:14:25.277556+0000","last_active":"2024-05-10T11:17:00.213935+0000","last_peered":"2024-05-10T11:17:00.213935+0000","last_clean":"2024-05-10T11:17:00.213935+0000","last_became_active":"2024-05-10T11:14:25.277201+0000","last_became_peered":"2024-05-10T11:14:25.277201+0000","last_unstale":"2024-05-10T11:17:00.213935+0000","last_undegraded":"2024-05-10T11:17:00.213935+0000","last_fullsized":"2024-05-10T11:17:00.213935+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:04:17.028305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.426356+0000","last_change":"2024-05-10T11:14:35.323338+0000","last_active":"2024-05-10T11:15:01.426356+0000","last_peered":"2024-05-10T11:15:01.426356+0000","last_clean":"2024-05-10T11:15:01.426356+0000","last_became_active":"2024-05-10T11:14:35.321948+0000","last_became_peered":"2024-05-10T11:14:35.321948+0000","last_unstale":"2024-05-10T11:15:01.426356+0000","last_undegraded":"2024-05-10T11:15:01.426356+0000","last_fullsized":"2024-05-10T11:15:01.426356+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:41:30.569125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'8","reported_seq":225,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:18:23.069244+0000","last_change":"2024-05-10T11:14:22.259187+0000","last_active":"2024-05-10T11:18:23.069244+0000","last_peered":"2024-05-10T11:18:23.069244+0000","last_clean":"2024-05-10T11:18:23.069244+0000","last_became_active":"2024-05-10T11:14:22.258812+0000","last_became_peered":"2024-05-10T11:14:22.258812+0000","last_unstale":"2024-05-10T11:18:23.069244+0000","last_undegraded":"2024-05-10T11:18:23.069244+0000","last_fullsized":"2024-05-10T11:18:23.069244+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:15:38.769705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1647,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":177,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.188655+0000","last_change":"2024-05-10T11:14:25.262469+0000","last_active":"2024-05-10T11:17:00.188655+0000","last_peered":"2024-05-10T11:17:00.188655+0000","last_clean":"2024-05-10T11:17:00.188655+0000","last_became_active":"2024-05-10T11:14:25.262104+0000","last_became_peered":"2024-05-10T11:14:25.262104+0000","last_unstale":"2024-05-10T11:17:00.188655+0000","last_undegraded":"2024-05-10T11:17:00.188655+0000","last_fullsized":"2024-05-10T11:17:00.188655+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:48:24.329136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.426927+0000","last_change":"2024-05-10T11:14:35.327737+0000","last_active":"2024-05-10T11:15:01.426927+0000","last_peered":"2024-05-10T11:15:01.426927+0000","last_clean":"2024-05-10T11:15:01.426927+0000","last_became_active":"2024-05-10T11:14:35.327209+0000","last_became_peered":"2024-05-10T11:14:35.327209+0000","last_unstale":"2024-05-10T11:15:01.426927+0000","last_undegraded":"2024-05-10T11:15:01.426927+0000","last_fullsized":"2024-05-10T11:15:01.426927+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:32:11.784752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:29.797324+0000","last_change":"2024-05-10T11:14:22.254853+0000","last_active":"2024-05-10T11:15:29.797324+0000","last_peered":"2024-05-10T11:15:29.797324+0000","last_clean":"2024-05-10T11:15:29.797324+0000","last_became_active":"2024-05-10T11:14:22.253352+0000","last_became_peered":"2024-05-10T11:14:22.253352+0000","last_unstale":"2024-05-10T11:15:29.797324+0000","last_undegraded":"2024-05-10T11:15:29.797324+0000","last_fullsized":"2024-05-10T11:15:29.797324+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:14:15.871042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.213028+0000","last_change":"2024-05-10T11:14:25.266798+0000","last_active":"2024-05-10T11:17:00.213028+0000","last_peered":"2024-05-10T11:17:00.213028+0000","last_clean":"2024-05-10T11:17:00.213028+0000","last_became_active":"2024-05-10T11:14:25.266384+0000","last_became_peered":"2024-05-10T11:14:25.266384+0000","last_unstale":"2024-05-10T11:17:00.213028+0000","last_undegraded":"2024-05-10T11:17:00.213028+0000","last_fullsized":"2024-05-10T11:17:00.213028+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:18:07.338987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.463232+0000","last_change":"2024-05-10T11:15:01.462376+0000","last_active":"2024-05-10T11:15:01.463232+0000","last_peered":"2024-05-10T11:15:01.463232+0000","last_clean":"2024-05-10T11:15:01.463232+0000","last_became_active":"2024-05-10T11:15:01.461876+0000","last_became_peered":"2024-05-10T11:15:01.461876+0000","last_unstale":"2024-05-10T11:15:01.463232+0000","last_undegraded":"2024-05-10T11:15:01.463232+0000","last_fullsized":"2024-05-10T11:15:01.463232+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:35:52.076414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:29.797310+0000","last_change":"2024-05-10T11:14:22.253976+0000","last_active":"2024-05-10T11:15:29.797310+0000","last_peered":"2024-05-10T11:15:29.797310+0000","last_clean":"2024-05-10T11:15:29.797310+0000","last_became_active":"2024-05-10T11:14:22.252230+0000","last_became_peered":"2024-05-10T11:14:22.252230+0000","last_unstale":"2024-05-10T11:15:29.797310+0000","last_undegraded":"2024-05-10T11:15:29.797310+0000","last_fullsized":"2024-05-10T11:15:29.797310+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:14:03.208811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.206683+0000","last_change":"2024-05-10T11:14:25.279627+0000","last_active":"2024-05-10T11:17:00.206683+0000","last_peered":"2024-05-10T11:17:00.206683+0000","last_clean":"2024-05-10T11:17:00.206683+0000","last_became_active":"2024-05-10T11:14:25.279330+0000","last_became_peered":"2024-05-10T11:14:25.279330+0000","last_unstale":"2024-05-10T11:17:00.206683+0000","last_undegraded":"2024-05-10T11:17:00.206683+0000","last_fullsized":"2024-05-10T11:17:00.206683+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:50:19.608053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.426065+0000","last_change":"2024-05-10T11:14:35.321723+0000","last_active":"2024-05-10T11:15:01.426065+0000","last_peered":"2024-05-10T11:15:01.426065+0000","last_clean":"2024-05-10T11:15:01.426065+0000","last_became_active":"2024-05-10T11:14:35.321384+0000","last_became_peered":"2024-05-10T11:14:35.321384+0000","last_unstale":"2024-05-10T11:15:01.426065+0000","last_undegraded":"2024-05-10T11:15:01.426065+0000","last_fullsized":"2024-05-10T11:15:01.426065+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:04:07.770889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:28.790869+0000","last_change":"2024-05-10T11:14:22.255511+0000","last_active":"2024-05-10T11:15:28.790869+0000","last_peered":"2024-05-10T11:15:28.790869+0000","last_clean":"2024-05-10T11:15:28.790869+0000","last_became_active":"2024-05-10T11:14:22.254722+0000","last_became_peered":"2024-05-10T11:14:22.254722+0000","last_unstale":"2024-05-10T11:15:28.790869+0000","last_undegraded":"2024-05-10T11:15:28.790869+0000","last_fullsized":"2024-05-10T11:15:28.790869+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:38:27.096011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.209442+0000","last_change":"2024-05-10T11:14:25.274009+0000","last_active":"2024-05-10T11:17:00.209442+0000","last_peered":"2024-05-10T11:17:00.209442+0000","last_clean":"2024-05-10T11:17:00.209442+0000","last_became_active":"2024-05-10T11:14:25.273672+0000","last_became_peered":"2024-05-10T11:14:25.273672+0000","last_unstale":"2024-05-10T11:17:00.209442+0000","last_undegraded":"2024-05-10T11:17:00.209442+0000","last_fullsized":"2024-05-10T11:17:00.209442+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:42:57.210131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459026+0000","last_change":"2024-05-10T11:14:35.321520+0000","last_active":"2024-05-10T11:15:01.459026+0000","last_peered":"2024-05-10T11:15:01.459026+0000","last_clean":"2024-05-10T11:15:01.459026+0000","last_became_active":"2024-05-10T11:14:35.320575+0000","last_became_peered":"2024-05-10T11:14:35.320575+0000","last_unstale":"2024-05-10T11:15:01.459026+0000","last_undegraded":"2024-05-10T11:15:01.459026+0000","last_fullsized":"2024-05-10T11:15:01.459026+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:54:03.183426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'5","reported_seq":219,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:18:25.071520+0000","last_change":"2024-05-10T11:14:22.251463+0000","last_active":"2024-05-10T11:18:25.071520+0000","last_peered":"2024-05-10T11:18:25.071520+0000","last_clean":"2024-05-10T11:18:25.071520+0000","last_became_active":"2024-05-10T11:14:22.251132+0000","last_became_peered":"2024-05-10T11:14:22.251132+0000","last_unstale":"2024-05-10T11:18:25.071520+0000","last_undegraded":"2024-05-10T11:18:25.071520+0000","last_fullsized":"2024-05-10T11:18:25.071520+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:29:18.125380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":174,"num_read_kb":174,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.209206+0000","last_change":"2024-05-10T11:14:25.276333+0000","last_active":"2024-05-10T11:17:00.209206+0000","last_peered":"2024-05-10T11:17:00.209206+0000","last_clean":"2024-05-10T11:17:00.209206+0000","last_became_active":"2024-05-10T11:14:25.275818+0000","last_became_peered":"2024-05-10T11:14:25.275818+0000","last_unstale":"2024-05-10T11:17:00.209206+0000","last_undegraded":"2024-05-10T11:17:00.209206+0000","last_fullsized":"2024-05-10T11:17:00.209206+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:27:19.542645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.422145+0000","last_change":"2024-05-10T11:14:35.328589+0000","last_active":"2024-05-10T11:15:01.422145+0000","last_peered":"2024-05-10T11:15:01.422145+0000","last_clean":"2024-05-10T11:15:01.422145+0000","last_became_active":"2024-05-10T11:14:35.328052+0000","last_became_peered":"2024-05-10T11:14:35.328052+0000","last_unstale":"2024-05-10T11:15:01.422145+0000","last_undegraded":"2024-05-10T11:15:01.422145+0000","last_fullsized":"2024-05-10T11:15:01.422145+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:09:42.763932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:28.790098+0000","last_change":"2024-05-10T11:14:22.257872+0000","last_active":"2024-05-10T11:15:28.790098+0000","last_peered":"2024-05-10T11:15:28.790098+0000","last_clean":"2024-05-10T11:15:28.790098+0000","last_became_active":"2024-05-10T11:14:22.257530+0000","last_became_peered":"2024-05-10T11:14:22.257530+0000","last_unstale":"2024-05-10T11:15:28.790098+0000","last_undegraded":"2024-05-10T11:15:28.790098+0000","last_fullsized":"2024-05-10T11:15:28.790098+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:26:30.367209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.214274+0000","last_change":"2024-05-10T11:14:25.269436+0000","last_active":"2024-05-10T11:17:00.214274+0000","last_peered":"2024-05-10T11:17:00.214274+0000","last_clean":"2024-05-10T11:17:00.214274+0000","last_became_active":"2024-05-10T11:14:25.269102+0000","last_became_peered":"2024-05-10T11:14:25.269102+0000","last_unstale":"2024-05-10T11:17:00.214274+0000","last_undegraded":"2024-05-10T11:17:00.214274+0000","last_fullsized":"2024-05-10T11:17:00.214274+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:51:50.172464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459510+0000","last_change":"2024-05-10T11:14:35.330272+0000","last_active":"2024-05-10T11:15:01.459510+0000","last_peered":"2024-05-10T11:15:01.459510+0000","last_clean":"2024-05-10T11:15:01.459510+0000","last_became_active":"2024-05-10T11:14:35.329438+0000","last_became_peered":"2024-05-10T11:14:35.329438+0000","last_unstale":"2024-05-10T11:15:01.459510+0000","last_undegraded":"2024-05-10T11:15:01.459510+0000","last_fullsized":"2024-05-10T11:15:01.459510+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:07:00.559447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:32.813046+0000","last_change":"2024-05-10T11:14:22.299302+0000","last_active":"2024-05-10T11:15:32.813046+0000","last_peered":"2024-05-10T11:15:32.813046+0000","last_clean":"2024-05-10T11:15:32.813046+0000","last_became_active":"2024-05-10T11:14:22.298792+0000","last_became_peered":"2024-05-10T11:14:22.298792+0000","last_unstale":"2024-05-10T11:15:32.813046+0000","last_undegraded":"2024-05-10T11:15:32.813046+0000","last_fullsized":"2024-05-10T11:15:32.813046+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:12:35.280135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.200955+0000","last_change":"2024-05-10T11:14:25.275380+0000","last_active":"2024-05-10T11:17:00.200955+0000","last_peered":"2024-05-10T11:17:00.200955+0000","last_clean":"2024-05-10T11:17:00.200955+0000","last_became_active":"2024-05-10T11:14:25.275042+0000","last_became_peered":"2024-05-10T11:14:25.275042+0000","last_unstale":"2024-05-10T11:17:00.200955+0000","last_undegraded":"2024-05-10T11:17:00.200955+0000","last_fullsized":"2024-05-10T11:17:00.200955+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:22:49.598602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.422374+0000","last_change":"2024-05-10T11:14:35.327505+0000","last_active":"2024-05-10T11:15:01.422374+0000","last_peered":"2024-05-10T11:15:01.422374+0000","last_clean":"2024-05-10T11:15:01.422374+0000","last_became_active":"2024-05-10T11:14:35.326885+0000","last_became_peered":"2024-05-10T11:14:35.326885+0000","last_unstale":"2024-05-10T11:15:01.422374+0000","last_undegraded":"2024-05-10T11:15:01.422374+0000","last_fullsized":"2024-05-10T11:15:01.422374+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:23:09.010545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'40","reported_seq":440,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:18:26.073755+0000","last_change":"2024-05-10T11:14:22.257571+0000","last_active":"2024-05-10T11:18:26.073755+0000","last_peered":"2024-05-10T11:18:26.073755+0000","last_clean":"2024-05-10T11:18:26.073755+0000","last_became_active":"2024-05-10T11:14:22.257156+0000","last_became_peered":"2024-05-10T11:14:22.257156+0000","last_unstale":"2024-05-10T11:18:26.073755+0000","last_undegraded":"2024-05-10T11:18:26.073755+0000","last_fullsized":"2024-05-10T11:18:26.073755+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:28:06.397262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69192,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":360,"num_read_kb":248,"num_write":41,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.193870+0000","last_change":"2024-05-10T11:14:25.281160+0000","last_active":"2024-05-10T11:17:00.193870+0000","last_peered":"2024-05-10T11:17:00.193870+0000","last_clean":"2024-05-10T11:17:00.193870+0000","last_became_active":"2024-05-10T11:14:25.280763+0000","last_became_peered":"2024-05-10T11:14:25.280763+0000","last_unstale":"2024-05-10T11:17:00.193870+0000","last_undegraded":"2024-05-10T11:17:00.193870+0000","last_fullsized":"2024-05-10T11:17:00.193870+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:19:53.350792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.460284+0000","last_change":"2024-05-10T11:14:35.320918+0000","last_active":"2024-05-10T11:15:01.460284+0000","last_peered":"2024-05-10T11:15:01.460284+0000","last_clean":"2024-05-10T11:15:01.460284+0000","last_became_active":"2024-05-10T11:14:35.320412+0000","last_became_peered":"2024-05-10T11:14:35.320412+0000","last_unstale":"2024-05-10T11:15:01.460284+0000","last_undegraded":"2024-05-10T11:15:01.460284+0000","last_fullsized":"2024-05-10T11:15:01.460284+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:55:52.585430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:32.810406+0000","last_change":"2024-05-10T11:14:22.257771+0000","last_active":"2024-05-10T11:15:32.810406+0000","last_peered":"2024-05-10T11:15:32.810406+0000","last_clean":"2024-05-10T11:15:32.810406+0000","last_became_active":"2024-05-10T11:14:22.257430+0000","last_became_peered":"2024-05-10T11:14:22.257430+0000","last_unstale":"2024-05-10T11:15:32.810406+0000","last_undegraded":"2024-05-10T11:15:32.810406+0000","last_fullsized":"2024-05-10T11:15:32.810406+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:23:50.131136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.203875+0000","last_change":"2024-05-10T11:14:25.272276+0000","last_active":"2024-05-10T11:17:00.203875+0000","last_peered":"2024-05-10T11:17:00.203875+0000","last_clean":"2024-05-10T11:17:00.203875+0000","last_became_active":"2024-05-10T11:14:25.271932+0000","last_became_peered":"2024-05-10T11:14:25.271932+0000","last_unstale":"2024-05-10T11:17:00.203875+0000","last_undegraded":"2024-05-10T11:17:00.203875+0000","last_fullsized":"2024-05-10T11:17:00.203875+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:49:44.479056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.463273+0000","last_change":"2024-05-10T11:15:01.462435+0000","last_active":"2024-05-10T11:15:01.463273+0000","last_peered":"2024-05-10T11:15:01.463273+0000","last_clean":"2024-05-10T11:15:01.463273+0000","last_became_active":"2024-05-10T11:15:01.461890+0000","last_became_peered":"2024-05-10T11:15:01.461890+0000","last_unstale":"2024-05-10T11:15:01.463273+0000","last_undegraded":"2024-05-10T11:15:01.463273+0000","last_fullsized":"2024-05-10T11:15:01.463273+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:01:13.350655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:28.789871+0000","last_change":"2024-05-10T11:14:22.256713+0000","last_active":"2024-05-10T11:15:28.789871+0000","last_peered":"2024-05-10T11:15:28.789871+0000","last_clean":"2024-05-10T11:15:28.789871+0000","last_became_active":"2024-05-10T11:14:22.255847+0000","last_became_peered":"2024-05-10T11:14:22.255847+0000","last_unstale":"2024-05-10T11:15:28.789871+0000","last_undegraded":"2024-05-10T11:15:28.789871+0000","last_fullsized":"2024-05-10T11:15:28.789871+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:00:54.605506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.214050+0000","last_change":"2024-05-10T11:14:25.282118+0000","last_active":"2024-05-10T11:17:00.214050+0000","last_peered":"2024-05-10T11:17:00.214050+0000","last_clean":"2024-05-10T11:17:00.214050+0000","last_became_active":"2024-05-10T11:14:25.280958+0000","last_became_peered":"2024-05-10T11:14:25.280958+0000","last_unstale":"2024-05-10T11:17:00.214050+0000","last_undegraded":"2024-05-10T11:17:00.214050+0000","last_fullsized":"2024-05-10T11:17:00.214050+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:02:10.815733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.422298+0000","last_change":"2024-05-10T11:14:35.329876+0000","last_active":"2024-05-10T11:15:01.422298+0000","last_peered":"2024-05-10T11:15:01.422298+0000","last_clean":"2024-05-10T11:15:01.422298+0000","last_became_active":"2024-05-10T11:14:35.328923+0000","last_became_peered":"2024-05-10T11:14:35.328923+0000","last_unstale":"2024-05-10T11:15:01.422298+0000","last_undegraded":"2024-05-10T11:15:01.422298+0000","last_fullsized":"2024-05-10T11:15:01.422298+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:04:51.356552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'6","reported_seq":223,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:18:24.070053+0000","last_change":"2024-05-10T11:14:22.254515+0000","last_active":"2024-05-10T11:18:24.070053+0000","last_peered":"2024-05-10T11:18:24.070053+0000","last_clean":"2024-05-10T11:18:24.070053+0000","last_became_active":"2024-05-10T11:14:22.253648+0000","last_became_peered":"2024-05-10T11:14:22.253648+0000","last_unstale":"2024-05-10T11:18:24.070053+0000","last_undegraded":"2024-05-10T11:18:24.070053+0000","last_fullsized":"2024-05-10T11:18:24.070053+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:10:22.144446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1613,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":179,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.210635+0000","last_change":"2024-05-10T11:14:25.278841+0000","last_active":"2024-05-10T11:17:00.210635+0000","last_peered":"2024-05-10T11:17:00.210635+0000","last_clean":"2024-05-10T11:17:00.210635+0000","last_became_active":"2024-05-10T11:14:25.278527+0000","last_became_peered":"2024-05-10T11:14:25.278527+0000","last_unstale":"2024-05-10T11:17:00.210635+0000","last_undegraded":"2024-05-10T11:17:00.210635+0000","last_fullsized":"2024-05-10T11:17:00.210635+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:01:55.902587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459763+0000","last_change":"2024-05-10T11:14:35.331334+0000","last_active":"2024-05-10T11:15:01.459763+0000","last_peered":"2024-05-10T11:15:01.459763+0000","last_clean":"2024-05-10T11:15:01.459763+0000","last_became_active":"2024-05-10T11:14:35.330940+0000","last_became_peered":"2024-05-10T11:14:35.330940+0000","last_unstale":"2024-05-10T11:15:01.459763+0000","last_undegraded":"2024-05-10T11:15:01.459763+0000","last_fullsized":"2024-05-10T11:15:01.459763+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:11:12.241551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":56,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:32.810680+0000","last_change":"2024-05-10T11:14:22.299917+0000","last_active":"2024-05-10T11:15:32.810680+0000","last_peered":"2024-05-10T11:15:32.810680+0000","last_clean":"2024-05-10T11:15:32.810680+0000","last_became_active":"2024-05-10T11:14:22.299212+0000","last_became_peered":"2024-05-10T11:14:22.299212+0000","last_unstale":"2024-05-10T11:15:32.810680+0000","last_undegraded":"2024-05-10T11:15:32.810680+0000","last_fullsized":"2024-05-10T11:15:32.810680+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:52:48.740339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.213948+0000","last_change":"2024-05-10T11:14:25.274693+0000","last_active":"2024-05-10T11:17:00.213948+0000","last_peered":"2024-05-10T11:17:00.213948+0000","last_clean":"2024-05-10T11:17:00.213948+0000","last_became_active":"2024-05-10T11:14:25.274378+0000","last_became_peered":"2024-05-10T11:14:25.274378+0000","last_unstale":"2024-05-10T11:17:00.213948+0000","last_undegraded":"2024-05-10T11:17:00.213948+0000","last_fullsized":"2024-05-10T11:17:00.213948+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:53:24.082742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:02.989636+0000","last_change":"2024-05-10T11:15:01.460541+0000","last_active":"2024-05-10T11:15:02.989636+0000","last_peered":"2024-05-10T11:15:02.989636+0000","last_clean":"2024-05-10T11:15:02.989636+0000","last_became_active":"2024-05-10T11:15:01.459954+0000","last_became_peered":"2024-05-10T11:15:01.459954+0000","last_unstale":"2024-05-10T11:15:02.989636+0000","last_undegraded":"2024-05-10T11:15:02.989636+0000","last_fullsized":"2024-05-10T11:15:02.989636+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:25:43.642441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,3,1],"acting":[8,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:32.810495+0000","last_change":"2024-05-10T11:14:22.258797+0000","last_active":"2024-05-10T11:15:32.810495+0000","last_peered":"2024-05-10T11:15:32.810495+0000","last_clean":"2024-05-10T11:15:32.810495+0000","last_became_active":"2024-05-10T11:14:22.258365+0000","last_became_peered":"2024-05-10T11:14:22.258365+0000","last_unstale":"2024-05-10T11:15:32.810495+0000","last_undegraded":"2024-05-10T11:15:32.810495+0000","last_fullsized":"2024-05-10T11:15:32.810495+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:58:53.291644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.198793+0000","last_change":"2024-05-10T11:15:01.460502+0000","last_active":"2024-05-10T11:17:00.198793+0000","last_peered":"2024-05-10T11:17:00.198793+0000","last_clean":"2024-05-10T11:17:00.198793+0000","last_became_active":"2024-05-10T11:15:01.460101+0000","last_became_peered":"2024-05-10T11:15:01.460101+0000","last_unstale":"2024-05-10T11:17:00.198793+0000","last_undegraded":"2024-05-10T11:17:00.198793+0000","last_fullsized":"2024-05-10T11:17:00.198793+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:22:39.735251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.429220+0000","last_change":"2024-05-10T11:14:35.419037+0000","last_active":"2024-05-10T11:15:01.429220+0000","last_peered":"2024-05-10T11:15:01.429220+0000","last_clean":"2024-05-10T11:15:01.429220+0000","last_became_active":"2024-05-10T11:14:35.418541+0000","last_became_peered":"2024-05-10T11:14:35.418541+0000","last_unstale":"2024-05-10T11:15:01.429220+0000","last_undegraded":"2024-05-10T11:15:01.429220+0000","last_fullsized":"2024-05-10T11:15:01.429220+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:05:49.560608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.422544+0000","last_change":"2024-05-10T11:14:22.299538+0000","last_active":"2024-05-10T11:15:01.422544+0000","last_peered":"2024-05-10T11:15:01.422544+0000","last_clean":"2024-05-10T11:15:01.422544+0000","last_became_active":"2024-05-10T11:14:22.299192+0000","last_became_peered":"2024-05-10T11:14:22.299192+0000","last_unstale":"2024-05-10T11:15:01.422544+0000","last_undegraded":"2024-05-10T11:15:01.422544+0000","last_fullsized":"2024-05-10T11:15:01.422544+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:28:43.041513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.193550+0000","last_change":"2024-05-10T11:14:25.272238+0000","last_active":"2024-05-10T11:17:00.193550+0000","last_peered":"2024-05-10T11:17:00.193550+0000","last_clean":"2024-05-10T11:17:00.193550+0000","last_became_active":"2024-05-10T11:14:25.271861+0000","last_became_peered":"2024-05-10T11:14:25.271861+0000","last_unstale":"2024-05-10T11:17:00.193550+0000","last_undegraded":"2024-05-10T11:17:00.193550+0000","last_fullsized":"2024-05-10T11:17:00.193550+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:00:08.821316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459614+0000","last_change":"2024-05-10T11:14:35.322239+0000","last_active":"2024-05-10T11:15:01.459614+0000","last_peered":"2024-05-10T11:15:01.459614+0000","last_clean":"2024-05-10T11:15:01.459614+0000","last_became_active":"2024-05-10T11:14:35.321864+0000","last_became_peered":"2024-05-10T11:14:35.321864+0000","last_unstale":"2024-05-10T11:15:01.459614+0000","last_undegraded":"2024-05-10T11:15:01.459614+0000","last_fullsized":"2024-05-10T11:15:01.459614+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:42:06.905879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:29.796685+0000","last_change":"2024-05-10T11:14:22.255449+0000","last_active":"2024-05-10T11:15:29.796685+0000","last_peered":"2024-05-10T11:15:29.796685+0000","last_clean":"2024-05-10T11:15:29.796685+0000","last_became_active":"2024-05-10T11:14:22.254966+0000","last_became_peered":"2024-05-10T11:14:22.254966+0000","last_unstale":"2024-05-10T11:15:29.796685+0000","last_undegraded":"2024-05-10T11:15:29.796685+0000","last_fullsized":"2024-05-10T11:15:29.796685+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:55:40.746115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.209423+0000","last_change":"2024-05-10T11:14:25.274050+0000","last_active":"2024-05-10T11:17:00.209423+0000","last_peered":"2024-05-10T11:17:00.209423+0000","last_clean":"2024-05-10T11:17:00.209423+0000","last_became_active":"2024-05-10T11:14:25.273720+0000","last_became_peered":"2024-05-10T11:14:25.273720+0000","last_unstale":"2024-05-10T11:17:00.209423+0000","last_undegraded":"2024-05-10T11:17:00.209423+0000","last_fullsized":"2024-05-10T11:17:00.209423+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:22:05.977673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459638+0000","last_change":"2024-05-10T11:14:35.418989+0000","last_active":"2024-05-10T11:15:01.459638+0000","last_peered":"2024-05-10T11:15:01.459638+0000","last_clean":"2024-05-10T11:15:01.459638+0000","last_became_active":"2024-05-10T11:14:35.418546+0000","last_became_peered":"2024-05-10T11:14:35.418546+0000","last_unstale":"2024-05-10T11:15:01.459638+0000","last_undegraded":"2024-05-10T11:15:01.459638+0000","last_fullsized":"2024-05-10T11:15:01.459638+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:23:37.875409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.213026+0000","last_change":"2024-05-10T11:14:25.272694+0000","last_active":"2024-05-10T11:17:00.213026+0000","last_peered":"2024-05-10T11:17:00.213026+0000","last_clean":"2024-05-10T11:17:00.213026+0000","last_became_active":"2024-05-10T11:14:25.272303+0000","last_became_peered":"2024-05-10T11:14:25.272303+0000","last_unstale":"2024-05-10T11:17:00.213026+0000","last_undegraded":"2024-05-10T11:17:00.213026+0000","last_fullsized":"2024-05-10T11:17:00.213026+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:51:39.039122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'6","reported_seq":219,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:18:25.070826+0000","last_change":"2024-05-10T11:14:22.299359+0000","last_active":"2024-05-10T11:18:25.070826+0000","last_peered":"2024-05-10T11:18:25.070826+0000","last_clean":"2024-05-10T11:18:25.070826+0000","last_became_active":"2024-05-10T11:14:22.299023+0000","last_became_peered":"2024-05-10T11:14:22.299023+0000","last_unstale":"2024-05-10T11:18:25.070826+0000","last_undegraded":"2024-05-10T11:18:25.070826+0000","last_fullsized":"2024-05-10T11:18:25.070826+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:53:06.301774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":174,"num_read_kb":174,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:41.913182+0000","last_change":"2024-05-10T11:14:35.322640+0000","last_active":"2024-05-10T11:15:41.913182+0000","last_peered":"2024-05-10T11:15:41.913182+0000","last_clean":"2024-05-10T11:15:41.913182+0000","last_became_active":"2024-05-10T11:14:35.322226+0000","last_became_peered":"2024-05-10T11:14:35.322226+0000","last_unstale":"2024-05-10T11:15:41.913182+0000","last_undegraded":"2024-05-10T11:15:41.913182+0000","last_fullsized":"2024-05-10T11:15:41.913182+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:34:11.574438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.206730+0000","last_change":"2024-05-10T11:14:25.276645+0000","last_active":"2024-05-10T11:17:00.206730+0000","last_peered":"2024-05-10T11:17:00.206730+0000","last_clean":"2024-05-10T11:17:00.206730+0000","last_became_active":"2024-05-10T11:14:25.276295+0000","last_became_peered":"2024-05-10T11:14:25.276295+0000","last_unstale":"2024-05-10T11:17:00.206730+0000","last_undegraded":"2024-05-10T11:17:00.206730+0000","last_fullsized":"2024-05-10T11:17:00.206730+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:52:06.715073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:32.810748+0000","last_change":"2024-05-10T11:14:22.252908+0000","last_active":"2024-05-10T11:15:32.810748+0000","last_peered":"2024-05-10T11:15:32.810748+0000","last_clean":"2024-05-10T11:15:32.810748+0000","last_became_active":"2024-05-10T11:14:22.252532+0000","last_became_peered":"2024-05-10T11:14:22.252532+0000","last_unstale":"2024-05-10T11:15:32.810748+0000","last_undegraded":"2024-05-10T11:15:32.810748+0000","last_fullsized":"2024-05-10T11:15:32.810748+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:41:59.651442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":114,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:38.138493+0000","last_change":"2024-05-10T11:12:53.090796+0000","last_active":"2024-05-10T11:15:38.138493+0000","last_peered":"2024-05-10T11:15:38.138493+0000","last_clean":"2024-05-10T11:15:38.138493+0000","last_became_active":"2024-05-10T11:12:53.086566+0000","last_became_peered":"2024-05-10T11:12:53.086566+0000","last_unstale":"2024-05-10T11:15:38.138493+0000","last_undegraded":"2024-05-10T11:15:38.138493+0000","last_fullsized":"2024-05-10T11:15:38.138493+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:09:27.097043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:09:27.097043+0000","last_clean_scrub_stamp":"2024-05-10T11:09:27.097043+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:42:17.759719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459957+0000","last_change":"2024-05-10T11:14:35.419970+0000","last_active":"2024-05-10T11:15:01.459957+0000","last_peered":"2024-05-10T11:15:01.459957+0000","last_clean":"2024-05-10T11:15:01.459957+0000","last_became_active":"2024-05-10T11:14:35.419516+0000","last_became_peered":"2024-05-10T11:14:35.419516+0000","last_unstale":"2024-05-10T11:15:01.459957+0000","last_undegraded":"2024-05-10T11:15:01.459957+0000","last_fullsized":"2024-05-10T11:15:01.459957+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:01:25.148155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.186817+0000","last_change":"2024-05-10T11:14:25.271478+0000","last_active":"2024-05-10T11:17:00.186817+0000","last_peered":"2024-05-10T11:17:00.186817+0000","last_clean":"2024-05-10T11:17:00.186817+0000","last_became_active":"2024-05-10T11:14:25.271143+0000","last_became_peered":"2024-05-10T11:14:25.271143+0000","last_unstale":"2024-05-10T11:17:00.186817+0000","last_undegraded":"2024-05-10T11:17:00.186817+0000","last_fullsized":"2024-05-10T11:17:00.186817+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:50:55.771659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:16:23.143145+0000","last_change":"2024-05-10T11:14:22.258704+0000","last_active":"2024-05-10T11:16:23.143145+0000","last_peered":"2024-05-10T11:16:23.143145+0000","last_clean":"2024-05-10T11:16:23.143145+0000","last_became_active":"2024-05-10T11:14:22.258304+0000","last_became_peered":"2024-05-10T11:14:22.258304+0000","last_unstale":"2024-05-10T11:16:23.143145+0000","last_undegraded":"2024-05-10T11:16:23.143145+0000","last_fullsized":"2024-05-10T11:16:23.143145+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:02:41.451394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.425481+0000","last_change":"2024-05-10T11:14:35.325151+0000","last_active":"2024-05-10T11:15:01.425481+0000","last_peered":"2024-05-10T11:15:01.425481+0000","last_clean":"2024-05-10T11:15:01.425481+0000","last_became_active":"2024-05-10T11:14:35.324680+0000","last_became_peered":"2024-05-10T11:14:35.324680+0000","last_unstale":"2024-05-10T11:15:01.425481+0000","last_undegraded":"2024-05-10T11:15:01.425481+0000","last_fullsized":"2024-05-10T11:15:01.425481+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:12:17.578364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:32.810795+0000","last_change":"2024-05-10T11:15:01.428210+0000","last_active":"2024-05-10T11:15:32.810795+0000","last_peered":"2024-05-10T11:15:32.810795+0000","last_clean":"2024-05-10T11:15:32.810795+0000","last_became_active":"2024-05-10T11:15:01.427671+0000","last_became_peered":"2024-05-10T11:15:01.427671+0000","last_unstale":"2024-05-10T11:15:32.810795+0000","last_undegraded":"2024-05-10T11:15:32.810795+0000","last_fullsized":"2024-05-10T11:15:32.810795+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:44:51.312743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.194143+0000","last_change":"2024-05-10T11:14:25.271350+0000","last_active":"2024-05-10T11:17:00.194143+0000","last_peered":"2024-05-10T11:17:00.194143+0000","last_clean":"2024-05-10T11:17:00.194143+0000","last_became_active":"2024-05-10T11:14:25.271047+0000","last_became_peered":"2024-05-10T11:14:25.271047+0000","last_unstale":"2024-05-10T11:17:00.194143+0000","last_undegraded":"2024-05-10T11:17:00.194143+0000","last_fullsized":"2024-05-10T11:17:00.194143+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:44:21.074007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.458379+0000","last_change":"2024-05-10T11:14:35.418910+0000","last_active":"2024-05-10T11:15:01.458379+0000","last_peered":"2024-05-10T11:15:01.458379+0000","last_clean":"2024-05-10T11:15:01.458379+0000","last_became_active":"2024-05-10T11:14:35.418460+0000","last_became_peered":"2024-05-10T11:14:35.418460+0000","last_unstale":"2024-05-10T11:15:01.458379+0000","last_undegraded":"2024-05-10T11:15:01.458379+0000","last_fullsized":"2024-05-10T11:15:01.458379+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:36:04.057776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'13","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.299940+0000","last_change":"2024-05-10T11:14:22.253673+0000","last_active":"2024-05-10T11:17:00.299940+0000","last_peered":"2024-05-10T11:17:00.299940+0000","last_clean":"2024-05-10T11:17:00.299940+0000","last_became_active":"2024-05-10T11:14:22.253304+0000","last_became_peered":"2024-05-10T11:14:22.253304+0000","last_unstale":"2024-05-10T11:17:00.299940+0000","last_undegraded":"2024-05-10T11:17:00.299940+0000","last_fullsized":"2024-05-10T11:17:00.299940+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:11:37.108621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.211961+0000","last_change":"2024-05-10T11:14:25.280590+0000","last_active":"2024-05-10T11:17:00.211961+0000","last_peered":"2024-05-10T11:17:00.211961+0000","last_clean":"2024-05-10T11:17:00.211961+0000","last_became_active":"2024-05-10T11:14:25.279990+0000","last_became_peered":"2024-05-10T11:14:25.279990+0000","last_unstale":"2024-05-10T11:17:00.211961+0000","last_undegraded":"2024-05-10T11:17:00.211961+0000","last_fullsized":"2024-05-10T11:17:00.211961+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:46:55.394767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.424307+0000","last_change":"2024-05-10T11:14:35.320931+0000","last_active":"2024-05-10T11:15:01.424307+0000","last_peered":"2024-05-10T11:15:01.424307+0000","last_clean":"2024-05-10T11:15:01.424307+0000","last_became_active":"2024-05-10T11:14:35.320523+0000","last_became_peered":"2024-05-10T11:14:35.320523+0000","last_unstale":"2024-05-10T11:15:01.424307+0000","last_undegraded":"2024-05-10T11:15:01.424307+0000","last_fullsized":"2024-05-10T11:15:01.424307+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:50:37.533397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:28.788626+0000","last_change":"2024-05-10T11:14:22.255115+0000","last_active":"2024-05-10T11:15:28.788626+0000","last_peered":"2024-05-10T11:15:28.788626+0000","last_clean":"2024-05-10T11:15:28.788626+0000","last_became_active":"2024-05-10T11:14:22.254732+0000","last_became_peered":"2024-05-10T11:14:22.254732+0000","last_unstale":"2024-05-10T11:15:28.788626+0000","last_undegraded":"2024-05-10T11:15:28.788626+0000","last_fullsized":"2024-05-10T11:15:28.788626+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:49:54.807312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.206879+0000","last_change":"2024-05-10T11:14:25.270766+0000","last_active":"2024-05-10T11:17:00.206879+0000","last_peered":"2024-05-10T11:17:00.206879+0000","last_clean":"2024-05-10T11:17:00.206879+0000","last_became_active":"2024-05-10T11:14:25.268958+0000","last_became_peered":"2024-05-10T11:14:25.268958+0000","last_unstale":"2024-05-10T11:17:00.206879+0000","last_undegraded":"2024-05-10T11:17:00.206879+0000","last_fullsized":"2024-05-10T11:17:00.206879+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:21:28.421478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459244+0000","last_change":"2024-05-10T11:14:35.420030+0000","last_active":"2024-05-10T11:15:01.459244+0000","last_peered":"2024-05-10T11:15:01.459244+0000","last_clean":"2024-05-10T11:15:01.459244+0000","last_became_active":"2024-05-10T11:14:35.419475+0000","last_became_peered":"2024-05-10T11:14:35.419475+0000","last_unstale":"2024-05-10T11:15:01.459244+0000","last_undegraded":"2024-05-10T11:15:01.459244+0000","last_fullsized":"2024-05-10T11:15:01.459244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:13:51.207146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:28.789346+0000","last_change":"2024-05-10T11:15:01.429683+0000","last_active":"2024-05-10T11:15:28.789346+0000","last_peered":"2024-05-10T11:15:28.789346+0000","last_clean":"2024-05-10T11:15:28.789346+0000","last_became_active":"2024-05-10T11:15:01.429162+0000","last_became_peered":"2024-05-10T11:15:01.429162+0000","last_unstale":"2024-05-10T11:15:28.789346+0000","last_undegraded":"2024-05-10T11:15:28.789346+0000","last_fullsized":"2024-05-10T11:15:28.789346+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:57:35.166686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.167290+0000","last_change":"2024-05-10T11:14:25.282381+0000","last_active":"2024-05-10T11:17:00.167290+0000","last_peered":"2024-05-10T11:17:00.167290+0000","last_clean":"2024-05-10T11:17:00.167290+0000","last_became_active":"2024-05-10T11:14:25.281990+0000","last_became_peered":"2024-05-10T11:14:25.281990+0000","last_unstale":"2024-05-10T11:17:00.167290+0000","last_undegraded":"2024-05-10T11:17:00.167290+0000","last_fullsized":"2024-05-10T11:17:00.167290+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:39:03.726547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.422393+0000","last_change":"2024-05-10T11:14:35.328242+0000","last_active":"2024-05-10T11:15:01.422393+0000","last_peered":"2024-05-10T11:15:01.422393+0000","last_clean":"2024-05-10T11:15:01.422393+0000","last_became_active":"2024-05-10T11:14:35.327877+0000","last_became_peered":"2024-05-10T11:14:35.327877+0000","last_unstale":"2024-05-10T11:15:01.422393+0000","last_undegraded":"2024-05-10T11:15:01.422393+0000","last_fullsized":"2024-05-10T11:15:01.422393+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:01:57.826334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:28.789342+0000","last_change":"2024-05-10T11:14:22.251057+0000","last_active":"2024-05-10T11:15:28.789342+0000","last_peered":"2024-05-10T11:15:28.789342+0000","last_clean":"2024-05-10T11:15:28.789342+0000","last_became_active":"2024-05-10T11:14:22.250696+0000","last_became_peered":"2024-05-10T11:14:22.250696+0000","last_unstale":"2024-05-10T11:15:28.789342+0000","last_undegraded":"2024-05-10T11:15:28.789342+0000","last_fullsized":"2024-05-10T11:15:28.789342+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:38:53.780643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.206311+0000","last_change":"2024-05-10T11:14:25.283806+0000","last_active":"2024-05-10T11:17:00.206311+0000","last_peered":"2024-05-10T11:17:00.206311+0000","last_clean":"2024-05-10T11:17:00.206311+0000","last_became_active":"2024-05-10T11:14:25.282714+0000","last_became_peered":"2024-05-10T11:14:25.282714+0000","last_unstale":"2024-05-10T11:17:00.206311+0000","last_undegraded":"2024-05-10T11:17:00.206311+0000","last_fullsized":"2024-05-10T11:17:00.206311+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:05:11.940408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:48.003154+0000","last_change":"2024-05-10T11:14:35.329986+0000","last_active":"2024-05-10T11:15:48.003154+0000","last_peered":"2024-05-10T11:15:48.003154+0000","last_clean":"2024-05-10T11:15:48.003154+0000","last_became_active":"2024-05-10T11:14:35.329477+0000","last_became_peered":"2024-05-10T11:14:35.329477+0000","last_unstale":"2024-05-10T11:15:48.003154+0000","last_undegraded":"2024-05-10T11:15:48.003154+0000","last_fullsized":"2024-05-10T11:15:48.003154+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:03:19.203580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'2","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:16:35.093919+0000","last_change":"2024-05-10T11:14:22.257627+0000","last_active":"2024-05-10T11:16:35.093919+0000","last_peered":"2024-05-10T11:16:35.093919+0000","last_clean":"2024-05-10T11:16:35.093919+0000","last_became_active":"2024-05-10T11:14:22.256990+0000","last_became_peered":"2024-05-10T11:14:22.256990+0000","last_unstale":"2024-05-10T11:16:35.093919+0000","last_undegraded":"2024-05-10T11:16:35.093919+0000","last_fullsized":"2024-05-10T11:16:35.093919+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:28:18.926821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.203131+0000","last_change":"2024-05-10T11:14:25.274799+0000","last_active":"2024-05-10T11:17:00.203131+0000","last_peered":"2024-05-10T11:17:00.203131+0000","last_clean":"2024-05-10T11:17:00.203131+0000","last_became_active":"2024-05-10T11:14:25.274443+0000","last_became_peered":"2024-05-10T11:14:25.274443+0000","last_unstale":"2024-05-10T11:17:00.203131+0000","last_undegraded":"2024-05-10T11:17:00.203131+0000","last_fullsized":"2024-05-10T11:17:00.203131+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:25:23.399798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459376+0000","last_change":"2024-05-10T11:14:35.418581+0000","last_active":"2024-05-10T11:15:01.459376+0000","last_peered":"2024-05-10T11:15:01.459376+0000","last_clean":"2024-05-10T11:15:01.459376+0000","last_became_active":"2024-05-10T11:14:35.418160+0000","last_became_peered":"2024-05-10T11:14:35.418160+0000","last_unstale":"2024-05-10T11:15:01.459376+0000","last_undegraded":"2024-05-10T11:15:01.459376+0000","last_fullsized":"2024-05-10T11:15:01.459376+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:39:01.873626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:32.810420+0000","last_change":"2024-05-10T11:14:22.260325+0000","last_active":"2024-05-10T11:15:32.810420+0000","last_peered":"2024-05-10T11:15:32.810420+0000","last_clean":"2024-05-10T11:15:32.810420+0000","last_became_active":"2024-05-10T11:14:22.259971+0000","last_became_peered":"2024-05-10T11:14:22.259971+0000","last_unstale":"2024-05-10T11:15:32.810420+0000","last_undegraded":"2024-05-10T11:15:32.810420+0000","last_fullsized":"2024-05-10T11:15:32.810420+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:34:55.547811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.214078+0000","last_change":"2024-05-10T11:14:25.279009+0000","last_active":"2024-05-10T11:17:00.214078+0000","last_peered":"2024-05-10T11:17:00.214078+0000","last_clean":"2024-05-10T11:17:00.214078+0000","last_became_active":"2024-05-10T11:14:25.278443+0000","last_became_peered":"2024-05-10T11:14:25.278443+0000","last_unstale":"2024-05-10T11:17:00.214078+0000","last_undegraded":"2024-05-10T11:17:00.214078+0000","last_fullsized":"2024-05-10T11:17:00.214078+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:25:22.306555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.426518+0000","last_change":"2024-05-10T11:14:35.319405+0000","last_active":"2024-05-10T11:15:01.426518+0000","last_peered":"2024-05-10T11:15:01.426518+0000","last_clean":"2024-05-10T11:15:01.426518+0000","last_became_active":"2024-05-10T11:14:35.318865+0000","last_became_peered":"2024-05-10T11:14:35.318865+0000","last_unstale":"2024-05-10T11:15:01.426518+0000","last_undegraded":"2024-05-10T11:15:01.426518+0000","last_fullsized":"2024-05-10T11:15:01.426518+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:22:41.592141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:32.813833+0000","last_change":"2024-05-10T11:14:22.299243+0000","last_active":"2024-05-10T11:15:32.813833+0000","last_peered":"2024-05-10T11:15:32.813833+0000","last_clean":"2024-05-10T11:15:32.813833+0000","last_became_active":"2024-05-10T11:14:22.298868+0000","last_became_peered":"2024-05-10T11:14:22.298868+0000","last_unstale":"2024-05-10T11:15:32.813833+0000","last_undegraded":"2024-05-10T11:15:32.813833+0000","last_fullsized":"2024-05-10T11:15:32.813833+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:54:15.078141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.213861+0000","last_change":"2024-05-10T11:14:25.281198+0000","last_active":"2024-05-10T11:17:00.213861+0000","last_peered":"2024-05-10T11:17:00.213861+0000","last_clean":"2024-05-10T11:17:00.213861+0000","last_became_active":"2024-05-10T11:14:25.280871+0000","last_became_peered":"2024-05-10T11:14:25.280871+0000","last_unstale":"2024-05-10T11:17:00.213861+0000","last_undegraded":"2024-05-10T11:17:00.213861+0000","last_fullsized":"2024-05-10T11:17:00.213861+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:24:55.843054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.425771+0000","last_change":"2024-05-10T11:14:35.329517+0000","last_active":"2024-05-10T11:15:01.425771+0000","last_peered":"2024-05-10T11:15:01.425771+0000","last_clean":"2024-05-10T11:15:01.425771+0000","last_became_active":"2024-05-10T11:14:35.329056+0000","last_became_peered":"2024-05-10T11:14:35.329056+0000","last_unstale":"2024-05-10T11:15:01.425771+0000","last_undegraded":"2024-05-10T11:15:01.425771+0000","last_fullsized":"2024-05-10T11:15:01.425771+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:09:24.844256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:32.810561+0000","last_change":"2024-05-10T11:14:22.257206+0000","last_active":"2024-05-10T11:15:32.810561+0000","last_peered":"2024-05-10T11:15:32.810561+0000","last_clean":"2024-05-10T11:15:32.810561+0000","last_became_active":"2024-05-10T11:14:22.256850+0000","last_became_peered":"2024-05-10T11:14:22.256850+0000","last_unstale":"2024-05-10T11:15:32.810561+0000","last_undegraded":"2024-05-10T11:15:32.810561+0000","last_fullsized":"2024-05-10T11:15:32.810561+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:34:31.973034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.211163+0000","last_change":"2024-05-10T11:15:01.461372+0000","last_active":"2024-05-10T11:17:00.211163+0000","last_peered":"2024-05-10T11:17:00.211163+0000","last_clean":"2024-05-10T11:17:00.211163+0000","last_became_active":"2024-05-10T11:15:01.460978+0000","last_became_peered":"2024-05-10T11:15:01.460978+0000","last_unstale":"2024-05-10T11:17:00.211163+0000","last_undegraded":"2024-05-10T11:17:00.211163+0000","last_fullsized":"2024-05-10T11:17:00.211163+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:18:45.750486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459061+0000","last_change":"2024-05-10T11:14:35.322387+0000","last_active":"2024-05-10T11:15:01.459061+0000","last_peered":"2024-05-10T11:15:01.459061+0000","last_clean":"2024-05-10T11:15:01.459061+0000","last_became_active":"2024-05-10T11:14:35.321939+0000","last_became_peered":"2024-05-10T11:14:35.321939+0000","last_unstale":"2024-05-10T11:15:01.459061+0000","last_undegraded":"2024-05-10T11:15:01.459061+0000","last_fullsized":"2024-05-10T11:15:01.459061+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:28:29.085363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459652+0000","last_change":"2024-05-10T11:14:22.258603+0000","last_active":"2024-05-10T11:15:01.459652+0000","last_peered":"2024-05-10T11:15:01.459652+0000","last_clean":"2024-05-10T11:15:01.459652+0000","last_became_active":"2024-05-10T11:14:22.258269+0000","last_became_peered":"2024-05-10T11:14:22.258269+0000","last_unstale":"2024-05-10T11:15:01.459652+0000","last_undegraded":"2024-05-10T11:15:01.459652+0000","last_fullsized":"2024-05-10T11:15:01.459652+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:30:04.240962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.207686+0000","last_change":"2024-05-10T11:14:25.281959+0000","last_active":"2024-05-10T11:17:00.207686+0000","last_peered":"2024-05-10T11:17:00.207686+0000","last_clean":"2024-05-10T11:17:00.207686+0000","last_became_active":"2024-05-10T11:14:25.281648+0000","last_became_peered":"2024-05-10T11:14:25.281648+0000","last_unstale":"2024-05-10T11:17:00.207686+0000","last_undegraded":"2024-05-10T11:17:00.207686+0000","last_fullsized":"2024-05-10T11:17:00.207686+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:24:01.569685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.195950+0000","last_change":"2024-05-10T11:14:25.284892+0000","last_active":"2024-05-10T11:17:00.195950+0000","last_peered":"2024-05-10T11:17:00.195950+0000","last_clean":"2024-05-10T11:17:00.195950+0000","last_became_active":"2024-05-10T11:14:25.284078+0000","last_became_peered":"2024-05-10T11:14:25.284078+0000","last_unstale":"2024-05-10T11:17:00.195950+0000","last_undegraded":"2024-05-10T11:17:00.195950+0000","last_fullsized":"2024-05-10T11:17:00.195950+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:49:43.348463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.211265+0000","last_change":"2024-05-10T11:14:25.274739+0000","last_active":"2024-05-10T11:17:00.211265+0000","last_peered":"2024-05-10T11:17:00.211265+0000","last_clean":"2024-05-10T11:17:00.211265+0000","last_became_active":"2024-05-10T11:14:25.274310+0000","last_became_peered":"2024-05-10T11:14:25.274310+0000","last_unstale":"2024-05-10T11:17:00.211265+0000","last_undegraded":"2024-05-10T11:17:00.211265+0000","last_fullsized":"2024-05-10T11:17:00.211265+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:21:16.935189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.204984+0000","last_change":"2024-05-10T11:14:25.272484+0000","last_active":"2024-05-10T11:17:00.204984+0000","last_peered":"2024-05-10T11:17:00.204984+0000","last_clean":"2024-05-10T11:17:00.204984+0000","last_became_active":"2024-05-10T11:14:25.272114+0000","last_became_peered":"2024-05-10T11:14:25.272114+0000","last_unstale":"2024-05-10T11:17:00.204984+0000","last_undegraded":"2024-05-10T11:17:00.204984+0000","last_fullsized":"2024-05-10T11:17:00.204984+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:46:34.537959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.208365+0000","last_change":"2024-05-10T11:15:01.460763+0000","last_active":"2024-05-10T11:17:00.208365+0000","last_peered":"2024-05-10T11:17:00.208365+0000","last_clean":"2024-05-10T11:17:00.208365+0000","last_became_active":"2024-05-10T11:15:01.460392+0000","last_became_peered":"2024-05-10T11:15:01.460392+0000","last_unstale":"2024-05-10T11:17:00.208365+0000","last_undegraded":"2024-05-10T11:17:00.208365+0000","last_fullsized":"2024-05-10T11:17:00.208365+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:15:42.859813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.202714+0000","last_change":"2024-05-10T11:14:25.269640+0000","last_active":"2024-05-10T11:17:00.202714+0000","last_peered":"2024-05-10T11:17:00.202714+0000","last_clean":"2024-05-10T11:17:00.202714+0000","last_became_active":"2024-05-10T11:14:25.269348+0000","last_became_peered":"2024-05-10T11:14:25.269348+0000","last_unstale":"2024-05-10T11:17:00.202714+0000","last_undegraded":"2024-05-10T11:17:00.202714+0000","last_fullsized":"2024-05-10T11:17:00.202714+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:40:25.276433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.194352+0000","last_change":"2024-05-10T11:14:25.282563+0000","last_active":"2024-05-10T11:17:00.194352+0000","last_peered":"2024-05-10T11:17:00.194352+0000","last_clean":"2024-05-10T11:17:00.194352+0000","last_became_active":"2024-05-10T11:14:25.282145+0000","last_became_peered":"2024-05-10T11:14:25.282145+0000","last_unstale":"2024-05-10T11:17:00.194352+0000","last_undegraded":"2024-05-10T11:17:00.194352+0000","last_fullsized":"2024-05-10T11:17:00.194352+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:54:38.436669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.202523+0000","last_change":"2024-05-10T11:14:25.276753+0000","last_active":"2024-05-10T11:17:00.202523+0000","last_peered":"2024-05-10T11:17:00.202523+0000","last_clean":"2024-05-10T11:17:00.202523+0000","last_became_active":"2024-05-10T11:14:25.276342+0000","last_became_peered":"2024-05-10T11:14:25.276342+0000","last_unstale":"2024-05-10T11:17:00.202523+0000","last_undegraded":"2024-05-10T11:17:00.202523+0000","last_fullsized":"2024-05-10T11:17:00.202523+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:31:42.766069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.209177+0000","last_change":"2024-05-10T11:14:25.275905+0000","last_active":"2024-05-10T11:17:00.209177+0000","last_peered":"2024-05-10T11:17:00.209177+0000","last_clean":"2024-05-10T11:17:00.209177+0000","last_became_active":"2024-05-10T11:14:25.275422+0000","last_became_peered":"2024-05-10T11:14:25.275422+0000","last_unstale":"2024-05-10T11:17:00.209177+0000","last_undegraded":"2024-05-10T11:17:00.209177+0000","last_fullsized":"2024-05-10T11:17:00.209177+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:39:24.033519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.205191+0000","last_change":"2024-05-10T11:14:25.271602+0000","last_active":"2024-05-10T11:17:00.205191+0000","last_peered":"2024-05-10T11:17:00.205191+0000","last_clean":"2024-05-10T11:17:00.205191+0000","last_became_active":"2024-05-10T11:14:25.267206+0000","last_became_peered":"2024-05-10T11:14:25.267206+0000","last_unstale":"2024-05-10T11:17:00.205191+0000","last_undegraded":"2024-05-10T11:17:00.205191+0000","last_fullsized":"2024-05-10T11:17:00.205191+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:28:24.708297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.206462+0000","last_change":"2024-05-10T11:14:25.280378+0000","last_active":"2024-05-10T11:17:00.206462+0000","last_peered":"2024-05-10T11:17:00.206462+0000","last_clean":"2024-05-10T11:17:00.206462+0000","last_became_active":"2024-05-10T11:14:25.280079+0000","last_became_peered":"2024-05-10T11:14:25.280079+0000","last_unstale":"2024-05-10T11:17:00.206462+0000","last_undegraded":"2024-05-10T11:17:00.206462+0000","last_fullsized":"2024-05-10T11:17:00.206462+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:13:47.781060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.204726+0000","last_change":"2024-05-10T11:14:25.272363+0000","last_active":"2024-05-10T11:17:00.204726+0000","last_peered":"2024-05-10T11:17:00.204726+0000","last_clean":"2024-05-10T11:17:00.204726+0000","last_became_active":"2024-05-10T11:14:25.271847+0000","last_became_peered":"2024-05-10T11:14:25.271847+0000","last_unstale":"2024-05-10T11:17:00.204726+0000","last_undegraded":"2024-05-10T11:17:00.204726+0000","last_fullsized":"2024-05-10T11:17:00.204726+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:47:54.661069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.203714+0000","last_change":"2024-05-10T11:14:25.272688+0000","last_active":"2024-05-10T11:17:00.203714+0000","last_peered":"2024-05-10T11:17:00.203714+0000","last_clean":"2024-05-10T11:17:00.203714+0000","last_became_active":"2024-05-10T11:14:25.271654+0000","last_became_peered":"2024-05-10T11:14:25.271654+0000","last_unstale":"2024-05-10T11:17:00.203714+0000","last_undegraded":"2024-05-10T11:17:00.203714+0000","last_fullsized":"2024-05-10T11:17:00.203714+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:50:21.149707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.199436+0000","last_change":"2024-05-10T11:15:01.432279+0000","last_active":"2024-05-10T11:17:00.199436+0000","last_peered":"2024-05-10T11:17:00.199436+0000","last_clean":"2024-05-10T11:17:00.199436+0000","last_became_active":"2024-05-10T11:15:01.431751+0000","last_became_peered":"2024-05-10T11:15:01.431751+0000","last_unstale":"2024-05-10T11:17:00.199436+0000","last_undegraded":"2024-05-10T11:17:00.199436+0000","last_fullsized":"2024-05-10T11:17:00.199436+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:25:44.196618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74852,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1077,"num_read_kb":620,"num_write":190,"num_write_kb":140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":224556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":162,"ondisk_log_size":162,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645377,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1015808,"data_stored":505095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841606,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":41164,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93703988,"statfs":{"total":95995035648,"available":95952883712,"internally_reserved":0,"allocated":1708032,"data_stored":1162558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":40434113},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005132,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1032192,"data_stored":509405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201361,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1024000,"data_stored":505207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430295,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28600,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716552,"statfs":{"total":95995035648,"available":95965749248,"internally_reserved":0,"allocated":1622016,"data_stored":1102797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593820,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":500485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574964,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1081344,"data_stored":569587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738490,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1032192,"data_stored":501873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411440,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36416,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708736,"statfs":{"total":95995035648,"available":95957745664,"internally_reserved":0,"allocated":1105920,"data_stored":571962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149802,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1019904,"data_stored":500679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953574,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1019904,"data_stored":506160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790048,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28608,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716544,"statfs":{"total":95995035648,"available":95965741056,"internally_reserved":0,"allocated":1630208,"data_stored":1097360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":69192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":71567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":71795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T11:18:28.602 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T11:18:28.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.736+0000 7f8955d7f640 1 -- 172.21.15.100:0/1010637905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89501086e0 msgr2=0x7f895010ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:28.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.736+0000 7f8955d7f640 1 --2- 172.21.15.100:0/1010637905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89501086e0 0x7f895010ab30 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f8948009900 tx=0x7f894802a410 comp rx=0 tx=0).stop 2024-05-10T11:18:28.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.737+0000 7f8955d7f640 1 -- 172.21.15.100:0/1010637905 shutdown_connections 2024-05-10T11:18:28.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.737+0000 7f8955d7f640 1 --2- 172.21.15.100:0/1010637905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89501086e0 0x7f895010ab30 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:28.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.737+0000 7f8955d7f640 1 --2- 172.21.15.100:0/1010637905 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8950105db0 0x7f89501081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:28.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.737+0000 7f8955d7f640 1 --2- 172.21.15.100:0/1010637905 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8950103480 0x7f8950105870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:28.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.737+0000 7f8955d7f640 1 -- 172.21.15.100:0/1010637905 >> 172.21.15.100:0/1010637905 conn(0x7f89500fce70 msgr2=0x7f89500ff2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:28.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.737+0000 7f8955d7f640 1 -- 172.21.15.100:0/1010637905 shutdown_connections 2024-05-10T11:18:28.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.737+0000 7f8955d7f640 1 -- 172.21.15.100:0/1010637905 wait complete. 2024-05-10T11:18:28.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.737+0000 7f8955d7f640 1 Processor -- start 2024-05-10T11:18:28.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.737+0000 7f8955d7f640 1 -- start start 2024-05-10T11:18:28.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.737+0000 7f8955d7f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8950103480 0x7f8950198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:28.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.737+0000 7f8955d7f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8950105db0 0x7f8950199010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:28.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.738+0000 7f8955d7f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89501086e0 0x7f89501a0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:28.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.738+0000 7f8955d7f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f895010e400 con 0x7f8950103480 2024-05-10T11:18:28.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.738+0000 7f8955d7f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f895010e280 con 0x7f8950105db0 2024-05-10T11:18:28.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.738+0000 7f8955d7f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f895010e580 con 0x7f89501086e0 2024-05-10T11:18:28.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.738+0000 7f894f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8950103480 0x7f8950198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:28.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.738+0000 7f894f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8950103480 0x7f8950198ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42270/0 (socket says 172.21.15.100:42270) 2024-05-10T11:18:28.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.738+0000 7f894f7fe640 1 -- 172.21.15.100:0/2162879383 learned_addr learned my addr 172.21.15.100:0/2162879383 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:18:28.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.738+0000 7f894effd640 1 --2- 172.21.15.100:0/2162879383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8950105db0 0x7f8950199010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:28.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.738+0000 7f894ffff640 1 --2- 172.21.15.100:0/2162879383 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89501086e0 0x7f89501a0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:28.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.738+0000 7f894f7fe640 1 -- 172.21.15.100:0/2162879383 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89501086e0 msgr2=0x7f89501a0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:28.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.738+0000 7f894f7fe640 1 --2- 172.21.15.100:0/2162879383 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89501086e0 0x7f89501a0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:28.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.738+0000 7f894f7fe640 1 -- 172.21.15.100:0/2162879383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8950105db0 msgr2=0x7f8950199010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:28.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.738+0000 7f894f7fe640 1 --2- 172.21.15.100:0/2162879383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8950105db0 0x7f8950199010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:28.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.738+0000 7f894f7fe640 1 -- 172.21.15.100:0/2162879383 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89501a06f0 con 0x7f8950103480 2024-05-10T11:18:28.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.738+0000 7f894f7fe640 1 --2- 172.21.15.100:0/2162879383 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8950103480 0x7f8950198ad0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f894000ea80 tx=0x7f894000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:28.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.738+0000 7f894cff9640 1 -- 172.21.15.100:0/2162879383 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f894000ce40 con 0x7f8950103480 2024-05-10T11:18:28.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.738+0000 7f894cff9640 1 -- 172.21.15.100:0/2162879383 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8940005020 con 0x7f8950103480 2024-05-10T11:18:28.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.738+0000 7f8955d7f640 1 -- 172.21.15.100:0/2162879383 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89501a09e0 con 0x7f8950103480 2024-05-10T11:18:28.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.738+0000 7f894cff9640 1 -- 172.21.15.100:0/2162879383 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89400116d0 con 0x7f8950103480 2024-05-10T11:18:28.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.738+0000 7f8955d7f640 1 -- 172.21.15.100:0/2162879383 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89501a0f30 con 0x7f8950103480 2024-05-10T11:18:28.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.739+0000 7f8955d7f640 1 -- 172.21.15.100:0/2162879383 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89500fe960 con 0x7f8950103480 2024-05-10T11:18:28.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.741+0000 7f894cff9640 1 -- 172.21.15.100:0/2162879383 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f8940004a40 con 0x7f8950103480 2024-05-10T11:18:28.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.741+0000 7f894cff9640 1 --2- 172.21.15.100:0/2162879383 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f89380b2870 0x7f89380b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:28.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.741+0000 7f894effd640 1 --2- 172.21.15.100:0/2162879383 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f89380b2870 0x7f89380b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:28.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.741+0000 7f894cff9640 1 -- 172.21.15.100:0/2162879383 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f89400e2a00 con 0x7f8950103480 2024-05-10T11:18:28.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.742+0000 7f894effd640 1 --2- 172.21.15.100:0/2162879383 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f89380b2870 0x7f89380b4d30 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8950199fe0 tx=0x7f894400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:28.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.745+0000 7f894cff9640 1 -- 172.21.15.100:0/2162879383 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f8940016030 con 0x7f8950103480 2024-05-10T11:18:28.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.922+0000 7f8955d7f640 1 -- 172.21.15.100:0/2162879383 --> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8950062df0 con 0x7f89380b2870 2024-05-10T11:18:28.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.931+0000 7f894cff9640 1 -- 172.21.15.100:0/2162879383 <== mgr.14152 v2:172.21.15.100:6800/2507287356 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340476 (secure 0 0 0) 0x7f8950062df0 con 0x7f89380b2870 2024-05-10T11:18:28.935 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:18:28.936 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-05-10T11:18:28.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.935+0000 7f8955d7f640 1 -- 172.21.15.100:0/2162879383 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f89380b2870 msgr2=0x7f89380b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:28.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.935+0000 7f8955d7f640 1 --2- 172.21.15.100:0/2162879383 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f89380b2870 0x7f89380b4d30 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8950199fe0 tx=0x7f894400b390 comp rx=0 tx=0).stop 2024-05-10T11:18:28.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.935+0000 7f8955d7f640 1 -- 172.21.15.100:0/2162879383 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8950103480 msgr2=0x7f8950198ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:28.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.935+0000 7f8955d7f640 1 --2- 172.21.15.100:0/2162879383 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8950103480 0x7f8950198ad0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f894000ea80 tx=0x7f894000ef40 comp rx=0 tx=0).stop 2024-05-10T11:18:28.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.935+0000 7f8955d7f640 1 -- 172.21.15.100:0/2162879383 shutdown_connections 2024-05-10T11:18:28.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.935+0000 7f8955d7f640 1 --2- 172.21.15.100:0/2162879383 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f89380b2870 0x7f89380b4d30 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:28.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.935+0000 7f8955d7f640 1 --2- 172.21.15.100:0/2162879383 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89501086e0 0x7f89501a0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:28.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.935+0000 7f8955d7f640 1 --2- 172.21.15.100:0/2162879383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8950105db0 0x7f8950199010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:28.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.935+0000 7f8955d7f640 1 --2- 172.21.15.100:0/2162879383 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8950103480 0x7f8950198ad0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:28.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.935+0000 7f8955d7f640 1 -- 172.21.15.100:0/2162879383 >> 172.21.15.100:0/2162879383 conn(0x7f89500fce70 msgr2=0x7f8950103d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:28.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.935+0000 7f8955d7f640 1 -- 172.21.15.100:0/2162879383 shutdown_connections 2024-05-10T11:18:28.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:28.935+0000 7f8955d7f640 1 -- 172.21.15.100:0/2162879383 wait complete. 2024-05-10T11:18:28.946 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":423,"stamp":"2024-05-10T11:18:28.248281+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665454,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1153,"num_read_kb":684,"num_write":315,"num_write_kb":2292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":756,"ondisk_log_size":756,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":404716,"kb_used_data":13964,"kb_used_omap":18,"kb_used_meta":390637,"kb_avail":1124537108,"statfs":{"total":1151940427776,"available":1151525998592,"internally_reserved":0,"allocated":14299136,"data_stored":8033168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":400012603},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5793,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":84,"num_read_kb":53,"num_write":11,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003400"},"pg_stats":[{"pgid":"3.3f","version":"89'8","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.206983+0000","last_change":"2024-05-10T11:14:25.286878+0000","last_active":"2024-05-10T11:17:00.206983+0000","last_peered":"2024-05-10T11:17:00.206983+0000","last_clean":"2024-05-10T11:17:00.206983+0000","last_became_active":"2024-05-10T11:14:25.286462+0000","last_became_peered":"2024-05-10T11:14:25.286462+0000","last_unstale":"2024-05-10T11:17:00.206983+0000","last_undegraded":"2024-05-10T11:17:00.206983+0000","last_fullsized":"2024-05-10T11:17:00.206983+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:13:40.627138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.204906+0000","last_change":"2024-05-10T11:14:25.280884+0000","last_active":"2024-05-10T11:17:00.204906+0000","last_peered":"2024-05-10T11:17:00.204906+0000","last_clean":"2024-05-10T11:17:00.204906+0000","last_became_active":"2024-05-10T11:14:25.280564+0000","last_became_peered":"2024-05-10T11:14:25.280564+0000","last_unstale":"2024-05-10T11:17:00.204906+0000","last_undegraded":"2024-05-10T11:17:00.204906+0000","last_fullsized":"2024-05-10T11:17:00.204906+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:14:16.735326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.208402+0000","last_change":"2024-05-10T11:14:25.278676+0000","last_active":"2024-05-10T11:17:00.208402+0000","last_peered":"2024-05-10T11:17:00.208402+0000","last_clean":"2024-05-10T11:17:00.208402+0000","last_became_active":"2024-05-10T11:14:25.278196+0000","last_became_peered":"2024-05-10T11:14:25.278196+0000","last_unstale":"2024-05-10T11:17:00.208402+0000","last_undegraded":"2024-05-10T11:17:00.208402+0000","last_fullsized":"2024-05-10T11:17:00.208402+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:50:29.390014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.207328+0000","last_change":"2024-05-10T11:14:25.274741+0000","last_active":"2024-05-10T11:17:00.207328+0000","last_peered":"2024-05-10T11:17:00.207328+0000","last_clean":"2024-05-10T11:17:00.207328+0000","last_became_active":"2024-05-10T11:14:25.274251+0000","last_became_peered":"2024-05-10T11:14:25.274251+0000","last_unstale":"2024-05-10T11:17:00.207328+0000","last_undegraded":"2024-05-10T11:17:00.207328+0000","last_fullsized":"2024-05-10T11:17:00.207328+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:04:41.552042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.186404+0000","last_change":"2024-05-10T11:14:25.265232+0000","last_active":"2024-05-10T11:17:00.186404+0000","last_peered":"2024-05-10T11:17:00.186404+0000","last_clean":"2024-05-10T11:17:00.186404+0000","last_became_active":"2024-05-10T11:14:25.264852+0000","last_became_peered":"2024-05-10T11:14:25.264852+0000","last_unstale":"2024-05-10T11:17:00.186404+0000","last_undegraded":"2024-05-10T11:17:00.186404+0000","last_fullsized":"2024-05-10T11:17:00.186404+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:50:27.207258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.210357+0000","last_change":"2024-05-10T11:14:25.272626+0000","last_active":"2024-05-10T11:17:00.210357+0000","last_peered":"2024-05-10T11:17:00.210357+0000","last_clean":"2024-05-10T11:17:00.210357+0000","last_became_active":"2024-05-10T11:14:25.272305+0000","last_became_peered":"2024-05-10T11:14:25.272305+0000","last_unstale":"2024-05-10T11:17:00.210357+0000","last_undegraded":"2024-05-10T11:17:00.210357+0000","last_fullsized":"2024-05-10T11:17:00.210357+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:44:16.725453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.210435+0000","last_change":"2024-05-10T11:14:25.275460+0000","last_active":"2024-05-10T11:17:00.210435+0000","last_peered":"2024-05-10T11:17:00.210435+0000","last_clean":"2024-05-10T11:17:00.210435+0000","last_became_active":"2024-05-10T11:14:25.274571+0000","last_became_peered":"2024-05-10T11:14:25.274571+0000","last_unstale":"2024-05-10T11:17:00.210435+0000","last_undegraded":"2024-05-10T11:17:00.210435+0000","last_fullsized":"2024-05-10T11:17:00.210435+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:33:20.335738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.207412+0000","last_change":"2024-05-10T11:14:25.273975+0000","last_active":"2024-05-10T11:17:00.207412+0000","last_peered":"2024-05-10T11:17:00.207412+0000","last_clean":"2024-05-10T11:17:00.207412+0000","last_became_active":"2024-05-10T11:14:25.273652+0000","last_became_peered":"2024-05-10T11:14:25.273652+0000","last_unstale":"2024-05-10T11:17:00.207412+0000","last_undegraded":"2024-05-10T11:17:00.207412+0000","last_fullsized":"2024-05-10T11:17:00.207412+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:37:55.354449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.207494+0000","last_change":"2024-05-10T11:14:25.285530+0000","last_active":"2024-05-10T11:17:00.207494+0000","last_peered":"2024-05-10T11:17:00.207494+0000","last_clean":"2024-05-10T11:17:00.207494+0000","last_became_active":"2024-05-10T11:14:25.285235+0000","last_became_peered":"2024-05-10T11:14:25.285235+0000","last_unstale":"2024-05-10T11:17:00.207494+0000","last_undegraded":"2024-05-10T11:17:00.207494+0000","last_fullsized":"2024-05-10T11:17:00.207494+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:11:14.225339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.210587+0000","last_change":"2024-05-10T11:14:25.274738+0000","last_active":"2024-05-10T11:17:00.210587+0000","last_peered":"2024-05-10T11:17:00.210587+0000","last_clean":"2024-05-10T11:17:00.210587+0000","last_became_active":"2024-05-10T11:14:25.274107+0000","last_became_peered":"2024-05-10T11:14:25.274107+0000","last_unstale":"2024-05-10T11:17:00.210587+0000","last_undegraded":"2024-05-10T11:17:00.210587+0000","last_fullsized":"2024-05-10T11:17:00.210587+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:02:01.489930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.213489+0000","last_change":"2024-05-10T11:14:25.277431+0000","last_active":"2024-05-10T11:17:00.213489+0000","last_peered":"2024-05-10T11:17:00.213489+0000","last_clean":"2024-05-10T11:17:00.213489+0000","last_became_active":"2024-05-10T11:14:25.277130+0000","last_became_peered":"2024-05-10T11:14:25.277130+0000","last_unstale":"2024-05-10T11:17:00.213489+0000","last_undegraded":"2024-05-10T11:17:00.213489+0000","last_fullsized":"2024-05-10T11:17:00.213489+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:38:17.192282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.206976+0000","last_change":"2024-05-10T11:14:25.276291+0000","last_active":"2024-05-10T11:17:00.206976+0000","last_peered":"2024-05-10T11:17:00.206976+0000","last_clean":"2024-05-10T11:17:00.206976+0000","last_became_active":"2024-05-10T11:14:25.275842+0000","last_became_peered":"2024-05-10T11:14:25.275842+0000","last_unstale":"2024-05-10T11:17:00.206976+0000","last_undegraded":"2024-05-10T11:17:00.206976+0000","last_fullsized":"2024-05-10T11:17:00.206976+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:20:53.443135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.209964+0000","last_change":"2024-05-10T11:14:25.273950+0000","last_active":"2024-05-10T11:17:00.209964+0000","last_peered":"2024-05-10T11:17:00.209964+0000","last_clean":"2024-05-10T11:17:00.209964+0000","last_became_active":"2024-05-10T11:14:25.273644+0000","last_became_peered":"2024-05-10T11:14:25.273644+0000","last_unstale":"2024-05-10T11:17:00.209964+0000","last_undegraded":"2024-05-10T11:17:00.209964+0000","last_fullsized":"2024-05-10T11:17:00.209964+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:46:05.112575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.209077+0000","last_change":"2024-05-10T11:14:25.277930+0000","last_active":"2024-05-10T11:17:00.209077+0000","last_peered":"2024-05-10T11:17:00.209077+0000","last_clean":"2024-05-10T11:17:00.209077+0000","last_became_active":"2024-05-10T11:14:25.276793+0000","last_became_peered":"2024-05-10T11:14:25.276793+0000","last_unstale":"2024-05-10T11:17:00.209077+0000","last_undegraded":"2024-05-10T11:17:00.209077+0000","last_fullsized":"2024-05-10T11:17:00.209077+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:47:03.592882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.214057+0000","last_change":"2024-05-10T11:14:25.270434+0000","last_active":"2024-05-10T11:17:00.214057+0000","last_peered":"2024-05-10T11:17:00.214057+0000","last_clean":"2024-05-10T11:17:00.214057+0000","last_became_active":"2024-05-10T11:14:25.270139+0000","last_became_peered":"2024-05-10T11:14:25.270139+0000","last_unstale":"2024-05-10T11:17:00.214057+0000","last_undegraded":"2024-05-10T11:17:00.214057+0000","last_fullsized":"2024-05-10T11:17:00.214057+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:58:40.506560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.209206+0000","last_change":"2024-05-10T11:14:25.272981+0000","last_active":"2024-05-10T11:17:00.209206+0000","last_peered":"2024-05-10T11:17:00.209206+0000","last_clean":"2024-05-10T11:17:00.209206+0000","last_became_active":"2024-05-10T11:14:25.272592+0000","last_became_peered":"2024-05-10T11:14:25.272592+0000","last_unstale":"2024-05-10T11:17:00.209206+0000","last_undegraded":"2024-05-10T11:17:00.209206+0000","last_fullsized":"2024-05-10T11:17:00.209206+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:46:09.673013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.212493+0000","last_change":"2024-05-10T11:14:25.287304+0000","last_active":"2024-05-10T11:17:00.212493+0000","last_peered":"2024-05-10T11:17:00.212493+0000","last_clean":"2024-05-10T11:17:00.212493+0000","last_became_active":"2024-05-10T11:14:25.286964+0000","last_became_peered":"2024-05-10T11:14:25.286964+0000","last_unstale":"2024-05-10T11:17:00.212493+0000","last_undegraded":"2024-05-10T11:17:00.212493+0000","last_fullsized":"2024-05-10T11:17:00.212493+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:32:27.006840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.209855+0000","last_change":"2024-05-10T11:14:25.275366+0000","last_active":"2024-05-10T11:17:00.209855+0000","last_peered":"2024-05-10T11:17:00.209855+0000","last_clean":"2024-05-10T11:17:00.209855+0000","last_became_active":"2024-05-10T11:14:25.274986+0000","last_became_peered":"2024-05-10T11:14:25.274986+0000","last_unstale":"2024-05-10T11:17:00.209855+0000","last_undegraded":"2024-05-10T11:17:00.209855+0000","last_fullsized":"2024-05-10T11:17:00.209855+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:50:50.071975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.203474+0000","last_change":"2024-05-10T11:14:25.274588+0000","last_active":"2024-05-10T11:17:00.203474+0000","last_peered":"2024-05-10T11:17:00.203474+0000","last_clean":"2024-05-10T11:17:00.203474+0000","last_became_active":"2024-05-10T11:14:25.274179+0000","last_became_peered":"2024-05-10T11:14:25.274179+0000","last_unstale":"2024-05-10T11:17:00.203474+0000","last_undegraded":"2024-05-10T11:17:00.203474+0000","last_fullsized":"2024-05-10T11:17:00.203474+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:06:23.517082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.461771+0000","last_change":"2024-05-10T11:15:01.461337+0000","last_active":"2024-05-10T11:15:01.461771+0000","last_peered":"2024-05-10T11:15:01.461771+0000","last_clean":"2024-05-10T11:15:01.461771+0000","last_became_active":"2024-05-10T11:15:01.460776+0000","last_became_peered":"2024-05-10T11:15:01.460776+0000","last_unstale":"2024-05-10T11:15:01.461771+0000","last_undegraded":"2024-05-10T11:15:01.461771+0000","last_fullsized":"2024-05-10T11:15:01.461771+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:06:49.156483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,10,0],"acting":[2,3,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:32.813572+0000","last_change":"2024-05-10T11:14:22.253909+0000","last_active":"2024-05-10T11:15:32.813572+0000","last_peered":"2024-05-10T11:15:32.813572+0000","last_clean":"2024-05-10T11:15:32.813572+0000","last_became_active":"2024-05-10T11:14:22.253400+0000","last_became_peered":"2024-05-10T11:14:22.253400+0000","last_unstale":"2024-05-10T11:15:32.813572+0000","last_undegraded":"2024-05-10T11:15:32.813572+0000","last_fullsized":"2024-05-10T11:15:32.813572+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:06:46.173511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.206338+0000","last_change":"2024-05-10T11:14:25.273675+0000","last_active":"2024-05-10T11:17:00.206338+0000","last_peered":"2024-05-10T11:17:00.206338+0000","last_clean":"2024-05-10T11:17:00.206338+0000","last_became_active":"2024-05-10T11:14:25.273320+0000","last_became_peered":"2024-05-10T11:14:25.273320+0000","last_unstale":"2024-05-10T11:17:00.206338+0000","last_undegraded":"2024-05-10T11:17:00.206338+0000","last_fullsized":"2024-05-10T11:17:00.206338+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:09:18.232236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459008+0000","last_change":"2024-05-10T11:14:35.326204+0000","last_active":"2024-05-10T11:15:01.459008+0000","last_peered":"2024-05-10T11:15:01.459008+0000","last_clean":"2024-05-10T11:15:01.459008+0000","last_became_active":"2024-05-10T11:14:35.325715+0000","last_became_peered":"2024-05-10T11:14:35.325715+0000","last_unstale":"2024-05-10T11:15:01.459008+0000","last_undegraded":"2024-05-10T11:15:01.459008+0000","last_fullsized":"2024-05-10T11:15:01.459008+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:06:30.665861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:16:23.904221+0000","last_change":"2024-05-10T11:14:22.299194+0000","last_active":"2024-05-10T11:16:23.904221+0000","last_peered":"2024-05-10T11:16:23.904221+0000","last_clean":"2024-05-10T11:16:23.904221+0000","last_became_active":"2024-05-10T11:14:22.298807+0000","last_became_peered":"2024-05-10T11:14:22.298807+0000","last_unstale":"2024-05-10T11:16:23.904221+0000","last_undegraded":"2024-05-10T11:16:23.904221+0000","last_fullsized":"2024-05-10T11:16:23.904221+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:24:51.054834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.209768+0000","last_change":"2024-05-10T11:14:25.277860+0000","last_active":"2024-05-10T11:17:00.209768+0000","last_peered":"2024-05-10T11:17:00.209768+0000","last_clean":"2024-05-10T11:17:00.209768+0000","last_became_active":"2024-05-10T11:14:25.276784+0000","last_became_peered":"2024-05-10T11:14:25.276784+0000","last_unstale":"2024-05-10T11:17:00.209768+0000","last_undegraded":"2024-05-10T11:17:00.209768+0000","last_fullsized":"2024-05-10T11:17:00.209768+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:42:57.673595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459304+0000","last_change":"2024-05-10T11:14:35.330679+0000","last_active":"2024-05-10T11:15:01.459304+0000","last_peered":"2024-05-10T11:15:01.459304+0000","last_clean":"2024-05-10T11:15:01.459304+0000","last_became_active":"2024-05-10T11:14:35.329819+0000","last_became_peered":"2024-05-10T11:14:35.329819+0000","last_unstale":"2024-05-10T11:15:01.459304+0000","last_undegraded":"2024-05-10T11:15:01.459304+0000","last_fullsized":"2024-05-10T11:15:01.459304+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:31:26.982784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:29.798740+0000","last_change":"2024-05-10T11:14:22.260402+0000","last_active":"2024-05-10T11:15:29.798740+0000","last_peered":"2024-05-10T11:15:29.798740+0000","last_clean":"2024-05-10T11:15:29.798740+0000","last_became_active":"2024-05-10T11:14:22.259619+0000","last_became_peered":"2024-05-10T11:14:22.259619+0000","last_unstale":"2024-05-10T11:15:29.798740+0000","last_undegraded":"2024-05-10T11:15:29.798740+0000","last_fullsized":"2024-05-10T11:15:29.798740+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:56:34.491626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.204002+0000","last_change":"2024-05-10T11:14:25.276117+0000","last_active":"2024-05-10T11:17:00.204002+0000","last_peered":"2024-05-10T11:17:00.204002+0000","last_clean":"2024-05-10T11:17:00.204002+0000","last_became_active":"2024-05-10T11:14:25.275758+0000","last_became_peered":"2024-05-10T11:14:25.275758+0000","last_unstale":"2024-05-10T11:17:00.204002+0000","last_undegraded":"2024-05-10T11:17:00.204002+0000","last_fullsized":"2024-05-10T11:17:00.204002+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:12:16.582244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459213+0000","last_change":"2024-05-10T11:14:35.324633+0000","last_active":"2024-05-10T11:15:01.459213+0000","last_peered":"2024-05-10T11:15:01.459213+0000","last_clean":"2024-05-10T11:15:01.459213+0000","last_became_active":"2024-05-10T11:14:35.324152+0000","last_became_peered":"2024-05-10T11:14:35.324152+0000","last_unstale":"2024-05-10T11:15:01.459213+0000","last_undegraded":"2024-05-10T11:15:01.459213+0000","last_fullsized":"2024-05-10T11:15:01.459213+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:56:50.008782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.430353+0000","last_change":"2024-05-10T11:14:22.259086+0000","last_active":"2024-05-10T11:15:01.430353+0000","last_peered":"2024-05-10T11:15:01.430353+0000","last_clean":"2024-05-10T11:15:01.430353+0000","last_became_active":"2024-05-10T11:14:22.258628+0000","last_became_peered":"2024-05-10T11:14:22.258628+0000","last_unstale":"2024-05-10T11:15:01.430353+0000","last_undegraded":"2024-05-10T11:15:01.430353+0000","last_fullsized":"2024-05-10T11:15:01.430353+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:39:13.472622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.213935+0000","last_change":"2024-05-10T11:14:25.277556+0000","last_active":"2024-05-10T11:17:00.213935+0000","last_peered":"2024-05-10T11:17:00.213935+0000","last_clean":"2024-05-10T11:17:00.213935+0000","last_became_active":"2024-05-10T11:14:25.277201+0000","last_became_peered":"2024-05-10T11:14:25.277201+0000","last_unstale":"2024-05-10T11:17:00.213935+0000","last_undegraded":"2024-05-10T11:17:00.213935+0000","last_fullsized":"2024-05-10T11:17:00.213935+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:04:17.028305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.426356+0000","last_change":"2024-05-10T11:14:35.323338+0000","last_active":"2024-05-10T11:15:01.426356+0000","last_peered":"2024-05-10T11:15:01.426356+0000","last_clean":"2024-05-10T11:15:01.426356+0000","last_became_active":"2024-05-10T11:14:35.321948+0000","last_became_peered":"2024-05-10T11:14:35.321948+0000","last_unstale":"2024-05-10T11:15:01.426356+0000","last_undegraded":"2024-05-10T11:15:01.426356+0000","last_fullsized":"2024-05-10T11:15:01.426356+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:41:30.569125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'8","reported_seq":225,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:18:23.069244+0000","last_change":"2024-05-10T11:14:22.259187+0000","last_active":"2024-05-10T11:18:23.069244+0000","last_peered":"2024-05-10T11:18:23.069244+0000","last_clean":"2024-05-10T11:18:23.069244+0000","last_became_active":"2024-05-10T11:14:22.258812+0000","last_became_peered":"2024-05-10T11:14:22.258812+0000","last_unstale":"2024-05-10T11:18:23.069244+0000","last_undegraded":"2024-05-10T11:18:23.069244+0000","last_fullsized":"2024-05-10T11:18:23.069244+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:15:38.769705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1647,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":177,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.188655+0000","last_change":"2024-05-10T11:14:25.262469+0000","last_active":"2024-05-10T11:17:00.188655+0000","last_peered":"2024-05-10T11:17:00.188655+0000","last_clean":"2024-05-10T11:17:00.188655+0000","last_became_active":"2024-05-10T11:14:25.262104+0000","last_became_peered":"2024-05-10T11:14:25.262104+0000","last_unstale":"2024-05-10T11:17:00.188655+0000","last_undegraded":"2024-05-10T11:17:00.188655+0000","last_fullsized":"2024-05-10T11:17:00.188655+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:48:24.329136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.426927+0000","last_change":"2024-05-10T11:14:35.327737+0000","last_active":"2024-05-10T11:15:01.426927+0000","last_peered":"2024-05-10T11:15:01.426927+0000","last_clean":"2024-05-10T11:15:01.426927+0000","last_became_active":"2024-05-10T11:14:35.327209+0000","last_became_peered":"2024-05-10T11:14:35.327209+0000","last_unstale":"2024-05-10T11:15:01.426927+0000","last_undegraded":"2024-05-10T11:15:01.426927+0000","last_fullsized":"2024-05-10T11:15:01.426927+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:32:11.784752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:29.797324+0000","last_change":"2024-05-10T11:14:22.254853+0000","last_active":"2024-05-10T11:15:29.797324+0000","last_peered":"2024-05-10T11:15:29.797324+0000","last_clean":"2024-05-10T11:15:29.797324+0000","last_became_active":"2024-05-10T11:14:22.253352+0000","last_became_peered":"2024-05-10T11:14:22.253352+0000","last_unstale":"2024-05-10T11:15:29.797324+0000","last_undegraded":"2024-05-10T11:15:29.797324+0000","last_fullsized":"2024-05-10T11:15:29.797324+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:14:15.871042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.213028+0000","last_change":"2024-05-10T11:14:25.266798+0000","last_active":"2024-05-10T11:17:00.213028+0000","last_peered":"2024-05-10T11:17:00.213028+0000","last_clean":"2024-05-10T11:17:00.213028+0000","last_became_active":"2024-05-10T11:14:25.266384+0000","last_became_peered":"2024-05-10T11:14:25.266384+0000","last_unstale":"2024-05-10T11:17:00.213028+0000","last_undegraded":"2024-05-10T11:17:00.213028+0000","last_fullsized":"2024-05-10T11:17:00.213028+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:18:07.338987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.463232+0000","last_change":"2024-05-10T11:15:01.462376+0000","last_active":"2024-05-10T11:15:01.463232+0000","last_peered":"2024-05-10T11:15:01.463232+0000","last_clean":"2024-05-10T11:15:01.463232+0000","last_became_active":"2024-05-10T11:15:01.461876+0000","last_became_peered":"2024-05-10T11:15:01.461876+0000","last_unstale":"2024-05-10T11:15:01.463232+0000","last_undegraded":"2024-05-10T11:15:01.463232+0000","last_fullsized":"2024-05-10T11:15:01.463232+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:35:52.076414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:29.797310+0000","last_change":"2024-05-10T11:14:22.253976+0000","last_active":"2024-05-10T11:15:29.797310+0000","last_peered":"2024-05-10T11:15:29.797310+0000","last_clean":"2024-05-10T11:15:29.797310+0000","last_became_active":"2024-05-10T11:14:22.252230+0000","last_became_peered":"2024-05-10T11:14:22.252230+0000","last_unstale":"2024-05-10T11:15:29.797310+0000","last_undegraded":"2024-05-10T11:15:29.797310+0000","last_fullsized":"2024-05-10T11:15:29.797310+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:14:03.208811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.206683+0000","last_change":"2024-05-10T11:14:25.279627+0000","last_active":"2024-05-10T11:17:00.206683+0000","last_peered":"2024-05-10T11:17:00.206683+0000","last_clean":"2024-05-10T11:17:00.206683+0000","last_became_active":"2024-05-10T11:14:25.279330+0000","last_became_peered":"2024-05-10T11:14:25.279330+0000","last_unstale":"2024-05-10T11:17:00.206683+0000","last_undegraded":"2024-05-10T11:17:00.206683+0000","last_fullsized":"2024-05-10T11:17:00.206683+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:50:19.608053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.426065+0000","last_change":"2024-05-10T11:14:35.321723+0000","last_active":"2024-05-10T11:15:01.426065+0000","last_peered":"2024-05-10T11:15:01.426065+0000","last_clean":"2024-05-10T11:15:01.426065+0000","last_became_active":"2024-05-10T11:14:35.321384+0000","last_became_peered":"2024-05-10T11:14:35.321384+0000","last_unstale":"2024-05-10T11:15:01.426065+0000","last_undegraded":"2024-05-10T11:15:01.426065+0000","last_fullsized":"2024-05-10T11:15:01.426065+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:04:07.770889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:28.790869+0000","last_change":"2024-05-10T11:14:22.255511+0000","last_active":"2024-05-10T11:15:28.790869+0000","last_peered":"2024-05-10T11:15:28.790869+0000","last_clean":"2024-05-10T11:15:28.790869+0000","last_became_active":"2024-05-10T11:14:22.254722+0000","last_became_peered":"2024-05-10T11:14:22.254722+0000","last_unstale":"2024-05-10T11:15:28.790869+0000","last_undegraded":"2024-05-10T11:15:28.790869+0000","last_fullsized":"2024-05-10T11:15:28.790869+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:38:27.096011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.209442+0000","last_change":"2024-05-10T11:14:25.274009+0000","last_active":"2024-05-10T11:17:00.209442+0000","last_peered":"2024-05-10T11:17:00.209442+0000","last_clean":"2024-05-10T11:17:00.209442+0000","last_became_active":"2024-05-10T11:14:25.273672+0000","last_became_peered":"2024-05-10T11:14:25.273672+0000","last_unstale":"2024-05-10T11:17:00.209442+0000","last_undegraded":"2024-05-10T11:17:00.209442+0000","last_fullsized":"2024-05-10T11:17:00.209442+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:42:57.210131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459026+0000","last_change":"2024-05-10T11:14:35.321520+0000","last_active":"2024-05-10T11:15:01.459026+0000","last_peered":"2024-05-10T11:15:01.459026+0000","last_clean":"2024-05-10T11:15:01.459026+0000","last_became_active":"2024-05-10T11:14:35.320575+0000","last_became_peered":"2024-05-10T11:14:35.320575+0000","last_unstale":"2024-05-10T11:15:01.459026+0000","last_undegraded":"2024-05-10T11:15:01.459026+0000","last_fullsized":"2024-05-10T11:15:01.459026+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:54:03.183426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'5","reported_seq":219,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:18:25.071520+0000","last_change":"2024-05-10T11:14:22.251463+0000","last_active":"2024-05-10T11:18:25.071520+0000","last_peered":"2024-05-10T11:18:25.071520+0000","last_clean":"2024-05-10T11:18:25.071520+0000","last_became_active":"2024-05-10T11:14:22.251132+0000","last_became_peered":"2024-05-10T11:14:22.251132+0000","last_unstale":"2024-05-10T11:18:25.071520+0000","last_undegraded":"2024-05-10T11:18:25.071520+0000","last_fullsized":"2024-05-10T11:18:25.071520+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:29:18.125380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":174,"num_read_kb":174,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.209206+0000","last_change":"2024-05-10T11:14:25.276333+0000","last_active":"2024-05-10T11:17:00.209206+0000","last_peered":"2024-05-10T11:17:00.209206+0000","last_clean":"2024-05-10T11:17:00.209206+0000","last_became_active":"2024-05-10T11:14:25.275818+0000","last_became_peered":"2024-05-10T11:14:25.275818+0000","last_unstale":"2024-05-10T11:17:00.209206+0000","last_undegraded":"2024-05-10T11:17:00.209206+0000","last_fullsized":"2024-05-10T11:17:00.209206+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:27:19.542645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.422145+0000","last_change":"2024-05-10T11:14:35.328589+0000","last_active":"2024-05-10T11:15:01.422145+0000","last_peered":"2024-05-10T11:15:01.422145+0000","last_clean":"2024-05-10T11:15:01.422145+0000","last_became_active":"2024-05-10T11:14:35.328052+0000","last_became_peered":"2024-05-10T11:14:35.328052+0000","last_unstale":"2024-05-10T11:15:01.422145+0000","last_undegraded":"2024-05-10T11:15:01.422145+0000","last_fullsized":"2024-05-10T11:15:01.422145+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:09:42.763932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:28.790098+0000","last_change":"2024-05-10T11:14:22.257872+0000","last_active":"2024-05-10T11:15:28.790098+0000","last_peered":"2024-05-10T11:15:28.790098+0000","last_clean":"2024-05-10T11:15:28.790098+0000","last_became_active":"2024-05-10T11:14:22.257530+0000","last_became_peered":"2024-05-10T11:14:22.257530+0000","last_unstale":"2024-05-10T11:15:28.790098+0000","last_undegraded":"2024-05-10T11:15:28.790098+0000","last_fullsized":"2024-05-10T11:15:28.790098+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:26:30.367209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.214274+0000","last_change":"2024-05-10T11:14:25.269436+0000","last_active":"2024-05-10T11:17:00.214274+0000","last_peered":"2024-05-10T11:17:00.214274+0000","last_clean":"2024-05-10T11:17:00.214274+0000","last_became_active":"2024-05-10T11:14:25.269102+0000","last_became_peered":"2024-05-10T11:14:25.269102+0000","last_unstale":"2024-05-10T11:17:00.214274+0000","last_undegraded":"2024-05-10T11:17:00.214274+0000","last_fullsized":"2024-05-10T11:17:00.214274+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:51:50.172464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459510+0000","last_change":"2024-05-10T11:14:35.330272+0000","last_active":"2024-05-10T11:15:01.459510+0000","last_peered":"2024-05-10T11:15:01.459510+0000","last_clean":"2024-05-10T11:15:01.459510+0000","last_became_active":"2024-05-10T11:14:35.329438+0000","last_became_peered":"2024-05-10T11:14:35.329438+0000","last_unstale":"2024-05-10T11:15:01.459510+0000","last_undegraded":"2024-05-10T11:15:01.459510+0000","last_fullsized":"2024-05-10T11:15:01.459510+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:07:00.559447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:32.813046+0000","last_change":"2024-05-10T11:14:22.299302+0000","last_active":"2024-05-10T11:15:32.813046+0000","last_peered":"2024-05-10T11:15:32.813046+0000","last_clean":"2024-05-10T11:15:32.813046+0000","last_became_active":"2024-05-10T11:14:22.298792+0000","last_became_peered":"2024-05-10T11:14:22.298792+0000","last_unstale":"2024-05-10T11:15:32.813046+0000","last_undegraded":"2024-05-10T11:15:32.813046+0000","last_fullsized":"2024-05-10T11:15:32.813046+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:12:35.280135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.200955+0000","last_change":"2024-05-10T11:14:25.275380+0000","last_active":"2024-05-10T11:17:00.200955+0000","last_peered":"2024-05-10T11:17:00.200955+0000","last_clean":"2024-05-10T11:17:00.200955+0000","last_became_active":"2024-05-10T11:14:25.275042+0000","last_became_peered":"2024-05-10T11:14:25.275042+0000","last_unstale":"2024-05-10T11:17:00.200955+0000","last_undegraded":"2024-05-10T11:17:00.200955+0000","last_fullsized":"2024-05-10T11:17:00.200955+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:22:49.598602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.422374+0000","last_change":"2024-05-10T11:14:35.327505+0000","last_active":"2024-05-10T11:15:01.422374+0000","last_peered":"2024-05-10T11:15:01.422374+0000","last_clean":"2024-05-10T11:15:01.422374+0000","last_became_active":"2024-05-10T11:14:35.326885+0000","last_became_peered":"2024-05-10T11:14:35.326885+0000","last_unstale":"2024-05-10T11:15:01.422374+0000","last_undegraded":"2024-05-10T11:15:01.422374+0000","last_fullsized":"2024-05-10T11:15:01.422374+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:23:09.010545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'40","reported_seq":440,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:18:26.073755+0000","last_change":"2024-05-10T11:14:22.257571+0000","last_active":"2024-05-10T11:18:26.073755+0000","last_peered":"2024-05-10T11:18:26.073755+0000","last_clean":"2024-05-10T11:18:26.073755+0000","last_became_active":"2024-05-10T11:14:22.257156+0000","last_became_peered":"2024-05-10T11:14:22.257156+0000","last_unstale":"2024-05-10T11:18:26.073755+0000","last_undegraded":"2024-05-10T11:18:26.073755+0000","last_fullsized":"2024-05-10T11:18:26.073755+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:28:06.397262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69192,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":360,"num_read_kb":248,"num_write":41,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.193870+0000","last_change":"2024-05-10T11:14:25.281160+0000","last_active":"2024-05-10T11:17:00.193870+0000","last_peered":"2024-05-10T11:17:00.193870+0000","last_clean":"2024-05-10T11:17:00.193870+0000","last_became_active":"2024-05-10T11:14:25.280763+0000","last_became_peered":"2024-05-10T11:14:25.280763+0000","last_unstale":"2024-05-10T11:17:00.193870+0000","last_undegraded":"2024-05-10T11:17:00.193870+0000","last_fullsized":"2024-05-10T11:17:00.193870+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:19:53.350792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.460284+0000","last_change":"2024-05-10T11:14:35.320918+0000","last_active":"2024-05-10T11:15:01.460284+0000","last_peered":"2024-05-10T11:15:01.460284+0000","last_clean":"2024-05-10T11:15:01.460284+0000","last_became_active":"2024-05-10T11:14:35.320412+0000","last_became_peered":"2024-05-10T11:14:35.320412+0000","last_unstale":"2024-05-10T11:15:01.460284+0000","last_undegraded":"2024-05-10T11:15:01.460284+0000","last_fullsized":"2024-05-10T11:15:01.460284+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:55:52.585430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:32.810406+0000","last_change":"2024-05-10T11:14:22.257771+0000","last_active":"2024-05-10T11:15:32.810406+0000","last_peered":"2024-05-10T11:15:32.810406+0000","last_clean":"2024-05-10T11:15:32.810406+0000","last_became_active":"2024-05-10T11:14:22.257430+0000","last_became_peered":"2024-05-10T11:14:22.257430+0000","last_unstale":"2024-05-10T11:15:32.810406+0000","last_undegraded":"2024-05-10T11:15:32.810406+0000","last_fullsized":"2024-05-10T11:15:32.810406+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:23:50.131136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.203875+0000","last_change":"2024-05-10T11:14:25.272276+0000","last_active":"2024-05-10T11:17:00.203875+0000","last_peered":"2024-05-10T11:17:00.203875+0000","last_clean":"2024-05-10T11:17:00.203875+0000","last_became_active":"2024-05-10T11:14:25.271932+0000","last_became_peered":"2024-05-10T11:14:25.271932+0000","last_unstale":"2024-05-10T11:17:00.203875+0000","last_undegraded":"2024-05-10T11:17:00.203875+0000","last_fullsized":"2024-05-10T11:17:00.203875+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:49:44.479056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.463273+0000","last_change":"2024-05-10T11:15:01.462435+0000","last_active":"2024-05-10T11:15:01.463273+0000","last_peered":"2024-05-10T11:15:01.463273+0000","last_clean":"2024-05-10T11:15:01.463273+0000","last_became_active":"2024-05-10T11:15:01.461890+0000","last_became_peered":"2024-05-10T11:15:01.461890+0000","last_unstale":"2024-05-10T11:15:01.463273+0000","last_undegraded":"2024-05-10T11:15:01.463273+0000","last_fullsized":"2024-05-10T11:15:01.463273+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:01:13.350655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:28.789871+0000","last_change":"2024-05-10T11:14:22.256713+0000","last_active":"2024-05-10T11:15:28.789871+0000","last_peered":"2024-05-10T11:15:28.789871+0000","last_clean":"2024-05-10T11:15:28.789871+0000","last_became_active":"2024-05-10T11:14:22.255847+0000","last_became_peered":"2024-05-10T11:14:22.255847+0000","last_unstale":"2024-05-10T11:15:28.789871+0000","last_undegraded":"2024-05-10T11:15:28.789871+0000","last_fullsized":"2024-05-10T11:15:28.789871+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:00:54.605506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.214050+0000","last_change":"2024-05-10T11:14:25.282118+0000","last_active":"2024-05-10T11:17:00.214050+0000","last_peered":"2024-05-10T11:17:00.214050+0000","last_clean":"2024-05-10T11:17:00.214050+0000","last_became_active":"2024-05-10T11:14:25.280958+0000","last_became_peered":"2024-05-10T11:14:25.280958+0000","last_unstale":"2024-05-10T11:17:00.214050+0000","last_undegraded":"2024-05-10T11:17:00.214050+0000","last_fullsized":"2024-05-10T11:17:00.214050+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:02:10.815733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.422298+0000","last_change":"2024-05-10T11:14:35.329876+0000","last_active":"2024-05-10T11:15:01.422298+0000","last_peered":"2024-05-10T11:15:01.422298+0000","last_clean":"2024-05-10T11:15:01.422298+0000","last_became_active":"2024-05-10T11:14:35.328923+0000","last_became_peered":"2024-05-10T11:14:35.328923+0000","last_unstale":"2024-05-10T11:15:01.422298+0000","last_undegraded":"2024-05-10T11:15:01.422298+0000","last_fullsized":"2024-05-10T11:15:01.422298+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:04:51.356552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'6","reported_seq":223,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:18:24.070053+0000","last_change":"2024-05-10T11:14:22.254515+0000","last_active":"2024-05-10T11:18:24.070053+0000","last_peered":"2024-05-10T11:18:24.070053+0000","last_clean":"2024-05-10T11:18:24.070053+0000","last_became_active":"2024-05-10T11:14:22.253648+0000","last_became_peered":"2024-05-10T11:14:22.253648+0000","last_unstale":"2024-05-10T11:18:24.070053+0000","last_undegraded":"2024-05-10T11:18:24.070053+0000","last_fullsized":"2024-05-10T11:18:24.070053+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:10:22.144446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1613,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":179,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.210635+0000","last_change":"2024-05-10T11:14:25.278841+0000","last_active":"2024-05-10T11:17:00.210635+0000","last_peered":"2024-05-10T11:17:00.210635+0000","last_clean":"2024-05-10T11:17:00.210635+0000","last_became_active":"2024-05-10T11:14:25.278527+0000","last_became_peered":"2024-05-10T11:14:25.278527+0000","last_unstale":"2024-05-10T11:17:00.210635+0000","last_undegraded":"2024-05-10T11:17:00.210635+0000","last_fullsized":"2024-05-10T11:17:00.210635+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:01:55.902587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459763+0000","last_change":"2024-05-10T11:14:35.331334+0000","last_active":"2024-05-10T11:15:01.459763+0000","last_peered":"2024-05-10T11:15:01.459763+0000","last_clean":"2024-05-10T11:15:01.459763+0000","last_became_active":"2024-05-10T11:14:35.330940+0000","last_became_peered":"2024-05-10T11:14:35.330940+0000","last_unstale":"2024-05-10T11:15:01.459763+0000","last_undegraded":"2024-05-10T11:15:01.459763+0000","last_fullsized":"2024-05-10T11:15:01.459763+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:11:12.241551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":56,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:32.810680+0000","last_change":"2024-05-10T11:14:22.299917+0000","last_active":"2024-05-10T11:15:32.810680+0000","last_peered":"2024-05-10T11:15:32.810680+0000","last_clean":"2024-05-10T11:15:32.810680+0000","last_became_active":"2024-05-10T11:14:22.299212+0000","last_became_peered":"2024-05-10T11:14:22.299212+0000","last_unstale":"2024-05-10T11:15:32.810680+0000","last_undegraded":"2024-05-10T11:15:32.810680+0000","last_fullsized":"2024-05-10T11:15:32.810680+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:52:48.740339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.213948+0000","last_change":"2024-05-10T11:14:25.274693+0000","last_active":"2024-05-10T11:17:00.213948+0000","last_peered":"2024-05-10T11:17:00.213948+0000","last_clean":"2024-05-10T11:17:00.213948+0000","last_became_active":"2024-05-10T11:14:25.274378+0000","last_became_peered":"2024-05-10T11:14:25.274378+0000","last_unstale":"2024-05-10T11:17:00.213948+0000","last_undegraded":"2024-05-10T11:17:00.213948+0000","last_fullsized":"2024-05-10T11:17:00.213948+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:53:24.082742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:02.989636+0000","last_change":"2024-05-10T11:15:01.460541+0000","last_active":"2024-05-10T11:15:02.989636+0000","last_peered":"2024-05-10T11:15:02.989636+0000","last_clean":"2024-05-10T11:15:02.989636+0000","last_became_active":"2024-05-10T11:15:01.459954+0000","last_became_peered":"2024-05-10T11:15:01.459954+0000","last_unstale":"2024-05-10T11:15:02.989636+0000","last_undegraded":"2024-05-10T11:15:02.989636+0000","last_fullsized":"2024-05-10T11:15:02.989636+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:25:43.642441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,3,1],"acting":[8,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:32.810495+0000","last_change":"2024-05-10T11:14:22.258797+0000","last_active":"2024-05-10T11:15:32.810495+0000","last_peered":"2024-05-10T11:15:32.810495+0000","last_clean":"2024-05-10T11:15:32.810495+0000","last_became_active":"2024-05-10T11:14:22.258365+0000","last_became_peered":"2024-05-10T11:14:22.258365+0000","last_unstale":"2024-05-10T11:15:32.810495+0000","last_undegraded":"2024-05-10T11:15:32.810495+0000","last_fullsized":"2024-05-10T11:15:32.810495+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:58:53.291644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.198793+0000","last_change":"2024-05-10T11:15:01.460502+0000","last_active":"2024-05-10T11:17:00.198793+0000","last_peered":"2024-05-10T11:17:00.198793+0000","last_clean":"2024-05-10T11:17:00.198793+0000","last_became_active":"2024-05-10T11:15:01.460101+0000","last_became_peered":"2024-05-10T11:15:01.460101+0000","last_unstale":"2024-05-10T11:17:00.198793+0000","last_undegraded":"2024-05-10T11:17:00.198793+0000","last_fullsized":"2024-05-10T11:17:00.198793+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:22:39.735251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.429220+0000","last_change":"2024-05-10T11:14:35.419037+0000","last_active":"2024-05-10T11:15:01.429220+0000","last_peered":"2024-05-10T11:15:01.429220+0000","last_clean":"2024-05-10T11:15:01.429220+0000","last_became_active":"2024-05-10T11:14:35.418541+0000","last_became_peered":"2024-05-10T11:14:35.418541+0000","last_unstale":"2024-05-10T11:15:01.429220+0000","last_undegraded":"2024-05-10T11:15:01.429220+0000","last_fullsized":"2024-05-10T11:15:01.429220+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:05:49.560608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.422544+0000","last_change":"2024-05-10T11:14:22.299538+0000","last_active":"2024-05-10T11:15:01.422544+0000","last_peered":"2024-05-10T11:15:01.422544+0000","last_clean":"2024-05-10T11:15:01.422544+0000","last_became_active":"2024-05-10T11:14:22.299192+0000","last_became_peered":"2024-05-10T11:14:22.299192+0000","last_unstale":"2024-05-10T11:15:01.422544+0000","last_undegraded":"2024-05-10T11:15:01.422544+0000","last_fullsized":"2024-05-10T11:15:01.422544+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:28:43.041513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.193550+0000","last_change":"2024-05-10T11:14:25.272238+0000","last_active":"2024-05-10T11:17:00.193550+0000","last_peered":"2024-05-10T11:17:00.193550+0000","last_clean":"2024-05-10T11:17:00.193550+0000","last_became_active":"2024-05-10T11:14:25.271861+0000","last_became_peered":"2024-05-10T11:14:25.271861+0000","last_unstale":"2024-05-10T11:17:00.193550+0000","last_undegraded":"2024-05-10T11:17:00.193550+0000","last_fullsized":"2024-05-10T11:17:00.193550+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:00:08.821316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459614+0000","last_change":"2024-05-10T11:14:35.322239+0000","last_active":"2024-05-10T11:15:01.459614+0000","last_peered":"2024-05-10T11:15:01.459614+0000","last_clean":"2024-05-10T11:15:01.459614+0000","last_became_active":"2024-05-10T11:14:35.321864+0000","last_became_peered":"2024-05-10T11:14:35.321864+0000","last_unstale":"2024-05-10T11:15:01.459614+0000","last_undegraded":"2024-05-10T11:15:01.459614+0000","last_fullsized":"2024-05-10T11:15:01.459614+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:42:06.905879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:29.796685+0000","last_change":"2024-05-10T11:14:22.255449+0000","last_active":"2024-05-10T11:15:29.796685+0000","last_peered":"2024-05-10T11:15:29.796685+0000","last_clean":"2024-05-10T11:15:29.796685+0000","last_became_active":"2024-05-10T11:14:22.254966+0000","last_became_peered":"2024-05-10T11:14:22.254966+0000","last_unstale":"2024-05-10T11:15:29.796685+0000","last_undegraded":"2024-05-10T11:15:29.796685+0000","last_fullsized":"2024-05-10T11:15:29.796685+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:55:40.746115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.209423+0000","last_change":"2024-05-10T11:14:25.274050+0000","last_active":"2024-05-10T11:17:00.209423+0000","last_peered":"2024-05-10T11:17:00.209423+0000","last_clean":"2024-05-10T11:17:00.209423+0000","last_became_active":"2024-05-10T11:14:25.273720+0000","last_became_peered":"2024-05-10T11:14:25.273720+0000","last_unstale":"2024-05-10T11:17:00.209423+0000","last_undegraded":"2024-05-10T11:17:00.209423+0000","last_fullsized":"2024-05-10T11:17:00.209423+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:22:05.977673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459638+0000","last_change":"2024-05-10T11:14:35.418989+0000","last_active":"2024-05-10T11:15:01.459638+0000","last_peered":"2024-05-10T11:15:01.459638+0000","last_clean":"2024-05-10T11:15:01.459638+0000","last_became_active":"2024-05-10T11:14:35.418546+0000","last_became_peered":"2024-05-10T11:14:35.418546+0000","last_unstale":"2024-05-10T11:15:01.459638+0000","last_undegraded":"2024-05-10T11:15:01.459638+0000","last_fullsized":"2024-05-10T11:15:01.459638+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:23:37.875409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.213026+0000","last_change":"2024-05-10T11:14:25.272694+0000","last_active":"2024-05-10T11:17:00.213026+0000","last_peered":"2024-05-10T11:17:00.213026+0000","last_clean":"2024-05-10T11:17:00.213026+0000","last_became_active":"2024-05-10T11:14:25.272303+0000","last_became_peered":"2024-05-10T11:14:25.272303+0000","last_unstale":"2024-05-10T11:17:00.213026+0000","last_undegraded":"2024-05-10T11:17:00.213026+0000","last_fullsized":"2024-05-10T11:17:00.213026+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:51:39.039122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'6","reported_seq":219,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:18:25.070826+0000","last_change":"2024-05-10T11:14:22.299359+0000","last_active":"2024-05-10T11:18:25.070826+0000","last_peered":"2024-05-10T11:18:25.070826+0000","last_clean":"2024-05-10T11:18:25.070826+0000","last_became_active":"2024-05-10T11:14:22.299023+0000","last_became_peered":"2024-05-10T11:14:22.299023+0000","last_unstale":"2024-05-10T11:18:25.070826+0000","last_undegraded":"2024-05-10T11:18:25.070826+0000","last_fullsized":"2024-05-10T11:18:25.070826+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:53:06.301774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":174,"num_read_kb":174,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:41.913182+0000","last_change":"2024-05-10T11:14:35.322640+0000","last_active":"2024-05-10T11:15:41.913182+0000","last_peered":"2024-05-10T11:15:41.913182+0000","last_clean":"2024-05-10T11:15:41.913182+0000","last_became_active":"2024-05-10T11:14:35.322226+0000","last_became_peered":"2024-05-10T11:14:35.322226+0000","last_unstale":"2024-05-10T11:15:41.913182+0000","last_undegraded":"2024-05-10T11:15:41.913182+0000","last_fullsized":"2024-05-10T11:15:41.913182+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:34:11.574438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.206730+0000","last_change":"2024-05-10T11:14:25.276645+0000","last_active":"2024-05-10T11:17:00.206730+0000","last_peered":"2024-05-10T11:17:00.206730+0000","last_clean":"2024-05-10T11:17:00.206730+0000","last_became_active":"2024-05-10T11:14:25.276295+0000","last_became_peered":"2024-05-10T11:14:25.276295+0000","last_unstale":"2024-05-10T11:17:00.206730+0000","last_undegraded":"2024-05-10T11:17:00.206730+0000","last_fullsized":"2024-05-10T11:17:00.206730+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:52:06.715073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:32.810748+0000","last_change":"2024-05-10T11:14:22.252908+0000","last_active":"2024-05-10T11:15:32.810748+0000","last_peered":"2024-05-10T11:15:32.810748+0000","last_clean":"2024-05-10T11:15:32.810748+0000","last_became_active":"2024-05-10T11:14:22.252532+0000","last_became_peered":"2024-05-10T11:14:22.252532+0000","last_unstale":"2024-05-10T11:15:32.810748+0000","last_undegraded":"2024-05-10T11:15:32.810748+0000","last_fullsized":"2024-05-10T11:15:32.810748+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:41:59.651442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":114,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:38.138493+0000","last_change":"2024-05-10T11:12:53.090796+0000","last_active":"2024-05-10T11:15:38.138493+0000","last_peered":"2024-05-10T11:15:38.138493+0000","last_clean":"2024-05-10T11:15:38.138493+0000","last_became_active":"2024-05-10T11:12:53.086566+0000","last_became_peered":"2024-05-10T11:12:53.086566+0000","last_unstale":"2024-05-10T11:15:38.138493+0000","last_undegraded":"2024-05-10T11:15:38.138493+0000","last_fullsized":"2024-05-10T11:15:38.138493+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:09:27.097043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:09:27.097043+0000","last_clean_scrub_stamp":"2024-05-10T11:09:27.097043+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:42:17.759719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459957+0000","last_change":"2024-05-10T11:14:35.419970+0000","last_active":"2024-05-10T11:15:01.459957+0000","last_peered":"2024-05-10T11:15:01.459957+0000","last_clean":"2024-05-10T11:15:01.459957+0000","last_became_active":"2024-05-10T11:14:35.419516+0000","last_became_peered":"2024-05-10T11:14:35.419516+0000","last_unstale":"2024-05-10T11:15:01.459957+0000","last_undegraded":"2024-05-10T11:15:01.459957+0000","last_fullsized":"2024-05-10T11:15:01.459957+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:01:25.148155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.186817+0000","last_change":"2024-05-10T11:14:25.271478+0000","last_active":"2024-05-10T11:17:00.186817+0000","last_peered":"2024-05-10T11:17:00.186817+0000","last_clean":"2024-05-10T11:17:00.186817+0000","last_became_active":"2024-05-10T11:14:25.271143+0000","last_became_peered":"2024-05-10T11:14:25.271143+0000","last_unstale":"2024-05-10T11:17:00.186817+0000","last_undegraded":"2024-05-10T11:17:00.186817+0000","last_fullsized":"2024-05-10T11:17:00.186817+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:50:55.771659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:16:23.143145+0000","last_change":"2024-05-10T11:14:22.258704+0000","last_active":"2024-05-10T11:16:23.143145+0000","last_peered":"2024-05-10T11:16:23.143145+0000","last_clean":"2024-05-10T11:16:23.143145+0000","last_became_active":"2024-05-10T11:14:22.258304+0000","last_became_peered":"2024-05-10T11:14:22.258304+0000","last_unstale":"2024-05-10T11:16:23.143145+0000","last_undegraded":"2024-05-10T11:16:23.143145+0000","last_fullsized":"2024-05-10T11:16:23.143145+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:02:41.451394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.425481+0000","last_change":"2024-05-10T11:14:35.325151+0000","last_active":"2024-05-10T11:15:01.425481+0000","last_peered":"2024-05-10T11:15:01.425481+0000","last_clean":"2024-05-10T11:15:01.425481+0000","last_became_active":"2024-05-10T11:14:35.324680+0000","last_became_peered":"2024-05-10T11:14:35.324680+0000","last_unstale":"2024-05-10T11:15:01.425481+0000","last_undegraded":"2024-05-10T11:15:01.425481+0000","last_fullsized":"2024-05-10T11:15:01.425481+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:12:17.578364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:32.810795+0000","last_change":"2024-05-10T11:15:01.428210+0000","last_active":"2024-05-10T11:15:32.810795+0000","last_peered":"2024-05-10T11:15:32.810795+0000","last_clean":"2024-05-10T11:15:32.810795+0000","last_became_active":"2024-05-10T11:15:01.427671+0000","last_became_peered":"2024-05-10T11:15:01.427671+0000","last_unstale":"2024-05-10T11:15:32.810795+0000","last_undegraded":"2024-05-10T11:15:32.810795+0000","last_fullsized":"2024-05-10T11:15:32.810795+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:44:51.312743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.194143+0000","last_change":"2024-05-10T11:14:25.271350+0000","last_active":"2024-05-10T11:17:00.194143+0000","last_peered":"2024-05-10T11:17:00.194143+0000","last_clean":"2024-05-10T11:17:00.194143+0000","last_became_active":"2024-05-10T11:14:25.271047+0000","last_became_peered":"2024-05-10T11:14:25.271047+0000","last_unstale":"2024-05-10T11:17:00.194143+0000","last_undegraded":"2024-05-10T11:17:00.194143+0000","last_fullsized":"2024-05-10T11:17:00.194143+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:44:21.074007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.458379+0000","last_change":"2024-05-10T11:14:35.418910+0000","last_active":"2024-05-10T11:15:01.458379+0000","last_peered":"2024-05-10T11:15:01.458379+0000","last_clean":"2024-05-10T11:15:01.458379+0000","last_became_active":"2024-05-10T11:14:35.418460+0000","last_became_peered":"2024-05-10T11:14:35.418460+0000","last_unstale":"2024-05-10T11:15:01.458379+0000","last_undegraded":"2024-05-10T11:15:01.458379+0000","last_fullsized":"2024-05-10T11:15:01.458379+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:36:04.057776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'13","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.299940+0000","last_change":"2024-05-10T11:14:22.253673+0000","last_active":"2024-05-10T11:17:00.299940+0000","last_peered":"2024-05-10T11:17:00.299940+0000","last_clean":"2024-05-10T11:17:00.299940+0000","last_became_active":"2024-05-10T11:14:22.253304+0000","last_became_peered":"2024-05-10T11:14:22.253304+0000","last_unstale":"2024-05-10T11:17:00.299940+0000","last_undegraded":"2024-05-10T11:17:00.299940+0000","last_fullsized":"2024-05-10T11:17:00.299940+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:11:37.108621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.211961+0000","last_change":"2024-05-10T11:14:25.280590+0000","last_active":"2024-05-10T11:17:00.211961+0000","last_peered":"2024-05-10T11:17:00.211961+0000","last_clean":"2024-05-10T11:17:00.211961+0000","last_became_active":"2024-05-10T11:14:25.279990+0000","last_became_peered":"2024-05-10T11:14:25.279990+0000","last_unstale":"2024-05-10T11:17:00.211961+0000","last_undegraded":"2024-05-10T11:17:00.211961+0000","last_fullsized":"2024-05-10T11:17:00.211961+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:46:55.394767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.424307+0000","last_change":"2024-05-10T11:14:35.320931+0000","last_active":"2024-05-10T11:15:01.424307+0000","last_peered":"2024-05-10T11:15:01.424307+0000","last_clean":"2024-05-10T11:15:01.424307+0000","last_became_active":"2024-05-10T11:14:35.320523+0000","last_became_peered":"2024-05-10T11:14:35.320523+0000","last_unstale":"2024-05-10T11:15:01.424307+0000","last_undegraded":"2024-05-10T11:15:01.424307+0000","last_fullsized":"2024-05-10T11:15:01.424307+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:50:37.533397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:28.788626+0000","last_change":"2024-05-10T11:14:22.255115+0000","last_active":"2024-05-10T11:15:28.788626+0000","last_peered":"2024-05-10T11:15:28.788626+0000","last_clean":"2024-05-10T11:15:28.788626+0000","last_became_active":"2024-05-10T11:14:22.254732+0000","last_became_peered":"2024-05-10T11:14:22.254732+0000","last_unstale":"2024-05-10T11:15:28.788626+0000","last_undegraded":"2024-05-10T11:15:28.788626+0000","last_fullsized":"2024-05-10T11:15:28.788626+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:49:54.807312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.206879+0000","last_change":"2024-05-10T11:14:25.270766+0000","last_active":"2024-05-10T11:17:00.206879+0000","last_peered":"2024-05-10T11:17:00.206879+0000","last_clean":"2024-05-10T11:17:00.206879+0000","last_became_active":"2024-05-10T11:14:25.268958+0000","last_became_peered":"2024-05-10T11:14:25.268958+0000","last_unstale":"2024-05-10T11:17:00.206879+0000","last_undegraded":"2024-05-10T11:17:00.206879+0000","last_fullsized":"2024-05-10T11:17:00.206879+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:21:28.421478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459244+0000","last_change":"2024-05-10T11:14:35.420030+0000","last_active":"2024-05-10T11:15:01.459244+0000","last_peered":"2024-05-10T11:15:01.459244+0000","last_clean":"2024-05-10T11:15:01.459244+0000","last_became_active":"2024-05-10T11:14:35.419475+0000","last_became_peered":"2024-05-10T11:14:35.419475+0000","last_unstale":"2024-05-10T11:15:01.459244+0000","last_undegraded":"2024-05-10T11:15:01.459244+0000","last_fullsized":"2024-05-10T11:15:01.459244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:13:51.207146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:28.789346+0000","last_change":"2024-05-10T11:15:01.429683+0000","last_active":"2024-05-10T11:15:28.789346+0000","last_peered":"2024-05-10T11:15:28.789346+0000","last_clean":"2024-05-10T11:15:28.789346+0000","last_became_active":"2024-05-10T11:15:01.429162+0000","last_became_peered":"2024-05-10T11:15:01.429162+0000","last_unstale":"2024-05-10T11:15:28.789346+0000","last_undegraded":"2024-05-10T11:15:28.789346+0000","last_fullsized":"2024-05-10T11:15:28.789346+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:57:35.166686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.167290+0000","last_change":"2024-05-10T11:14:25.282381+0000","last_active":"2024-05-10T11:17:00.167290+0000","last_peered":"2024-05-10T11:17:00.167290+0000","last_clean":"2024-05-10T11:17:00.167290+0000","last_became_active":"2024-05-10T11:14:25.281990+0000","last_became_peered":"2024-05-10T11:14:25.281990+0000","last_unstale":"2024-05-10T11:17:00.167290+0000","last_undegraded":"2024-05-10T11:17:00.167290+0000","last_fullsized":"2024-05-10T11:17:00.167290+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:39:03.726547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.422393+0000","last_change":"2024-05-10T11:14:35.328242+0000","last_active":"2024-05-10T11:15:01.422393+0000","last_peered":"2024-05-10T11:15:01.422393+0000","last_clean":"2024-05-10T11:15:01.422393+0000","last_became_active":"2024-05-10T11:14:35.327877+0000","last_became_peered":"2024-05-10T11:14:35.327877+0000","last_unstale":"2024-05-10T11:15:01.422393+0000","last_undegraded":"2024-05-10T11:15:01.422393+0000","last_fullsized":"2024-05-10T11:15:01.422393+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:01:57.826334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:28.789342+0000","last_change":"2024-05-10T11:14:22.251057+0000","last_active":"2024-05-10T11:15:28.789342+0000","last_peered":"2024-05-10T11:15:28.789342+0000","last_clean":"2024-05-10T11:15:28.789342+0000","last_became_active":"2024-05-10T11:14:22.250696+0000","last_became_peered":"2024-05-10T11:14:22.250696+0000","last_unstale":"2024-05-10T11:15:28.789342+0000","last_undegraded":"2024-05-10T11:15:28.789342+0000","last_fullsized":"2024-05-10T11:15:28.789342+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:38:53.780643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.206311+0000","last_change":"2024-05-10T11:14:25.283806+0000","last_active":"2024-05-10T11:17:00.206311+0000","last_peered":"2024-05-10T11:17:00.206311+0000","last_clean":"2024-05-10T11:17:00.206311+0000","last_became_active":"2024-05-10T11:14:25.282714+0000","last_became_peered":"2024-05-10T11:14:25.282714+0000","last_unstale":"2024-05-10T11:17:00.206311+0000","last_undegraded":"2024-05-10T11:17:00.206311+0000","last_fullsized":"2024-05-10T11:17:00.206311+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:05:11.940408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:48.003154+0000","last_change":"2024-05-10T11:14:35.329986+0000","last_active":"2024-05-10T11:15:48.003154+0000","last_peered":"2024-05-10T11:15:48.003154+0000","last_clean":"2024-05-10T11:15:48.003154+0000","last_became_active":"2024-05-10T11:14:35.329477+0000","last_became_peered":"2024-05-10T11:14:35.329477+0000","last_unstale":"2024-05-10T11:15:48.003154+0000","last_undegraded":"2024-05-10T11:15:48.003154+0000","last_fullsized":"2024-05-10T11:15:48.003154+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:03:19.203580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'2","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:16:35.093919+0000","last_change":"2024-05-10T11:14:22.257627+0000","last_active":"2024-05-10T11:16:35.093919+0000","last_peered":"2024-05-10T11:16:35.093919+0000","last_clean":"2024-05-10T11:16:35.093919+0000","last_became_active":"2024-05-10T11:14:22.256990+0000","last_became_peered":"2024-05-10T11:14:22.256990+0000","last_unstale":"2024-05-10T11:16:35.093919+0000","last_undegraded":"2024-05-10T11:16:35.093919+0000","last_fullsized":"2024-05-10T11:16:35.093919+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:28:18.926821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.203131+0000","last_change":"2024-05-10T11:14:25.274799+0000","last_active":"2024-05-10T11:17:00.203131+0000","last_peered":"2024-05-10T11:17:00.203131+0000","last_clean":"2024-05-10T11:17:00.203131+0000","last_became_active":"2024-05-10T11:14:25.274443+0000","last_became_peered":"2024-05-10T11:14:25.274443+0000","last_unstale":"2024-05-10T11:17:00.203131+0000","last_undegraded":"2024-05-10T11:17:00.203131+0000","last_fullsized":"2024-05-10T11:17:00.203131+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:25:23.399798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459376+0000","last_change":"2024-05-10T11:14:35.418581+0000","last_active":"2024-05-10T11:15:01.459376+0000","last_peered":"2024-05-10T11:15:01.459376+0000","last_clean":"2024-05-10T11:15:01.459376+0000","last_became_active":"2024-05-10T11:14:35.418160+0000","last_became_peered":"2024-05-10T11:14:35.418160+0000","last_unstale":"2024-05-10T11:15:01.459376+0000","last_undegraded":"2024-05-10T11:15:01.459376+0000","last_fullsized":"2024-05-10T11:15:01.459376+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:39:01.873626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:32.810420+0000","last_change":"2024-05-10T11:14:22.260325+0000","last_active":"2024-05-10T11:15:32.810420+0000","last_peered":"2024-05-10T11:15:32.810420+0000","last_clean":"2024-05-10T11:15:32.810420+0000","last_became_active":"2024-05-10T11:14:22.259971+0000","last_became_peered":"2024-05-10T11:14:22.259971+0000","last_unstale":"2024-05-10T11:15:32.810420+0000","last_undegraded":"2024-05-10T11:15:32.810420+0000","last_fullsized":"2024-05-10T11:15:32.810420+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:34:55.547811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.214078+0000","last_change":"2024-05-10T11:14:25.279009+0000","last_active":"2024-05-10T11:17:00.214078+0000","last_peered":"2024-05-10T11:17:00.214078+0000","last_clean":"2024-05-10T11:17:00.214078+0000","last_became_active":"2024-05-10T11:14:25.278443+0000","last_became_peered":"2024-05-10T11:14:25.278443+0000","last_unstale":"2024-05-10T11:17:00.214078+0000","last_undegraded":"2024-05-10T11:17:00.214078+0000","last_fullsized":"2024-05-10T11:17:00.214078+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:25:22.306555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.426518+0000","last_change":"2024-05-10T11:14:35.319405+0000","last_active":"2024-05-10T11:15:01.426518+0000","last_peered":"2024-05-10T11:15:01.426518+0000","last_clean":"2024-05-10T11:15:01.426518+0000","last_became_active":"2024-05-10T11:14:35.318865+0000","last_became_peered":"2024-05-10T11:14:35.318865+0000","last_unstale":"2024-05-10T11:15:01.426518+0000","last_undegraded":"2024-05-10T11:15:01.426518+0000","last_fullsized":"2024-05-10T11:15:01.426518+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:22:41.592141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:32.813833+0000","last_change":"2024-05-10T11:14:22.299243+0000","last_active":"2024-05-10T11:15:32.813833+0000","last_peered":"2024-05-10T11:15:32.813833+0000","last_clean":"2024-05-10T11:15:32.813833+0000","last_became_active":"2024-05-10T11:14:22.298868+0000","last_became_peered":"2024-05-10T11:14:22.298868+0000","last_unstale":"2024-05-10T11:15:32.813833+0000","last_undegraded":"2024-05-10T11:15:32.813833+0000","last_fullsized":"2024-05-10T11:15:32.813833+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:54:15.078141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.213861+0000","last_change":"2024-05-10T11:14:25.281198+0000","last_active":"2024-05-10T11:17:00.213861+0000","last_peered":"2024-05-10T11:17:00.213861+0000","last_clean":"2024-05-10T11:17:00.213861+0000","last_became_active":"2024-05-10T11:14:25.280871+0000","last_became_peered":"2024-05-10T11:14:25.280871+0000","last_unstale":"2024-05-10T11:17:00.213861+0000","last_undegraded":"2024-05-10T11:17:00.213861+0000","last_fullsized":"2024-05-10T11:17:00.213861+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:24:55.843054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.425771+0000","last_change":"2024-05-10T11:14:35.329517+0000","last_active":"2024-05-10T11:15:01.425771+0000","last_peered":"2024-05-10T11:15:01.425771+0000","last_clean":"2024-05-10T11:15:01.425771+0000","last_became_active":"2024-05-10T11:14:35.329056+0000","last_became_peered":"2024-05-10T11:14:35.329056+0000","last_unstale":"2024-05-10T11:15:01.425771+0000","last_undegraded":"2024-05-10T11:15:01.425771+0000","last_fullsized":"2024-05-10T11:15:01.425771+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:09:24.844256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:32.810561+0000","last_change":"2024-05-10T11:14:22.257206+0000","last_active":"2024-05-10T11:15:32.810561+0000","last_peered":"2024-05-10T11:15:32.810561+0000","last_clean":"2024-05-10T11:15:32.810561+0000","last_became_active":"2024-05-10T11:14:22.256850+0000","last_became_peered":"2024-05-10T11:14:22.256850+0000","last_unstale":"2024-05-10T11:15:32.810561+0000","last_undegraded":"2024-05-10T11:15:32.810561+0000","last_fullsized":"2024-05-10T11:15:32.810561+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:34:31.973034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.211163+0000","last_change":"2024-05-10T11:15:01.461372+0000","last_active":"2024-05-10T11:17:00.211163+0000","last_peered":"2024-05-10T11:17:00.211163+0000","last_clean":"2024-05-10T11:17:00.211163+0000","last_became_active":"2024-05-10T11:15:01.460978+0000","last_became_peered":"2024-05-10T11:15:01.460978+0000","last_unstale":"2024-05-10T11:17:00.211163+0000","last_undegraded":"2024-05-10T11:17:00.211163+0000","last_fullsized":"2024-05-10T11:17:00.211163+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:18:45.750486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459061+0000","last_change":"2024-05-10T11:14:35.322387+0000","last_active":"2024-05-10T11:15:01.459061+0000","last_peered":"2024-05-10T11:15:01.459061+0000","last_clean":"2024-05-10T11:15:01.459061+0000","last_became_active":"2024-05-10T11:14:35.321939+0000","last_became_peered":"2024-05-10T11:14:35.321939+0000","last_unstale":"2024-05-10T11:15:01.459061+0000","last_undegraded":"2024-05-10T11:15:01.459061+0000","last_fullsized":"2024-05-10T11:15:01.459061+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:28:29.085363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459652+0000","last_change":"2024-05-10T11:14:22.258603+0000","last_active":"2024-05-10T11:15:01.459652+0000","last_peered":"2024-05-10T11:15:01.459652+0000","last_clean":"2024-05-10T11:15:01.459652+0000","last_became_active":"2024-05-10T11:14:22.258269+0000","last_became_peered":"2024-05-10T11:14:22.258269+0000","last_unstale":"2024-05-10T11:15:01.459652+0000","last_undegraded":"2024-05-10T11:15:01.459652+0000","last_fullsized":"2024-05-10T11:15:01.459652+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:30:04.240962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.207686+0000","last_change":"2024-05-10T11:14:25.281959+0000","last_active":"2024-05-10T11:17:00.207686+0000","last_peered":"2024-05-10T11:17:00.207686+0000","last_clean":"2024-05-10T11:17:00.207686+0000","last_became_active":"2024-05-10T11:14:25.281648+0000","last_became_peered":"2024-05-10T11:14:25.281648+0000","last_unstale":"2024-05-10T11:17:00.207686+0000","last_undegraded":"2024-05-10T11:17:00.207686+0000","last_fullsized":"2024-05-10T11:17:00.207686+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:24:01.569685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.195950+0000","last_change":"2024-05-10T11:14:25.284892+0000","last_active":"2024-05-10T11:17:00.195950+0000","last_peered":"2024-05-10T11:17:00.195950+0000","last_clean":"2024-05-10T11:17:00.195950+0000","last_became_active":"2024-05-10T11:14:25.284078+0000","last_became_peered":"2024-05-10T11:14:25.284078+0000","last_unstale":"2024-05-10T11:17:00.195950+0000","last_undegraded":"2024-05-10T11:17:00.195950+0000","last_fullsized":"2024-05-10T11:17:00.195950+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:49:43.348463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.211265+0000","last_change":"2024-05-10T11:14:25.274739+0000","last_active":"2024-05-10T11:17:00.211265+0000","last_peered":"2024-05-10T11:17:00.211265+0000","last_clean":"2024-05-10T11:17:00.211265+0000","last_became_active":"2024-05-10T11:14:25.274310+0000","last_became_peered":"2024-05-10T11:14:25.274310+0000","last_unstale":"2024-05-10T11:17:00.211265+0000","last_undegraded":"2024-05-10T11:17:00.211265+0000","last_fullsized":"2024-05-10T11:17:00.211265+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:21:16.935189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.204984+0000","last_change":"2024-05-10T11:14:25.272484+0000","last_active":"2024-05-10T11:17:00.204984+0000","last_peered":"2024-05-10T11:17:00.204984+0000","last_clean":"2024-05-10T11:17:00.204984+0000","last_became_active":"2024-05-10T11:14:25.272114+0000","last_became_peered":"2024-05-10T11:14:25.272114+0000","last_unstale":"2024-05-10T11:17:00.204984+0000","last_undegraded":"2024-05-10T11:17:00.204984+0000","last_fullsized":"2024-05-10T11:17:00.204984+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:46:34.537959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.208365+0000","last_change":"2024-05-10T11:15:01.460763+0000","last_active":"2024-05-10T11:17:00.208365+0000","last_peered":"2024-05-10T11:17:00.208365+0000","last_clean":"2024-05-10T11:17:00.208365+0000","last_became_active":"2024-05-10T11:15:01.460392+0000","last_became_peered":"2024-05-10T11:15:01.460392+0000","last_unstale":"2024-05-10T11:17:00.208365+0000","last_undegraded":"2024-05-10T11:17:00.208365+0000","last_fullsized":"2024-05-10T11:17:00.208365+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:15:42.859813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.202714+0000","last_change":"2024-05-10T11:14:25.269640+0000","last_active":"2024-05-10T11:17:00.202714+0000","last_peered":"2024-05-10T11:17:00.202714+0000","last_clean":"2024-05-10T11:17:00.202714+0000","last_became_active":"2024-05-10T11:14:25.269348+0000","last_became_peered":"2024-05-10T11:14:25.269348+0000","last_unstale":"2024-05-10T11:17:00.202714+0000","last_undegraded":"2024-05-10T11:17:00.202714+0000","last_fullsized":"2024-05-10T11:17:00.202714+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:40:25.276433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.194352+0000","last_change":"2024-05-10T11:14:25.282563+0000","last_active":"2024-05-10T11:17:00.194352+0000","last_peered":"2024-05-10T11:17:00.194352+0000","last_clean":"2024-05-10T11:17:00.194352+0000","last_became_active":"2024-05-10T11:14:25.282145+0000","last_became_peered":"2024-05-10T11:14:25.282145+0000","last_unstale":"2024-05-10T11:17:00.194352+0000","last_undegraded":"2024-05-10T11:17:00.194352+0000","last_fullsized":"2024-05-10T11:17:00.194352+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:54:38.436669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.202523+0000","last_change":"2024-05-10T11:14:25.276753+0000","last_active":"2024-05-10T11:17:00.202523+0000","last_peered":"2024-05-10T11:17:00.202523+0000","last_clean":"2024-05-10T11:17:00.202523+0000","last_became_active":"2024-05-10T11:14:25.276342+0000","last_became_peered":"2024-05-10T11:14:25.276342+0000","last_unstale":"2024-05-10T11:17:00.202523+0000","last_undegraded":"2024-05-10T11:17:00.202523+0000","last_fullsized":"2024-05-10T11:17:00.202523+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:31:42.766069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.209177+0000","last_change":"2024-05-10T11:14:25.275905+0000","last_active":"2024-05-10T11:17:00.209177+0000","last_peered":"2024-05-10T11:17:00.209177+0000","last_clean":"2024-05-10T11:17:00.209177+0000","last_became_active":"2024-05-10T11:14:25.275422+0000","last_became_peered":"2024-05-10T11:14:25.275422+0000","last_unstale":"2024-05-10T11:17:00.209177+0000","last_undegraded":"2024-05-10T11:17:00.209177+0000","last_fullsized":"2024-05-10T11:17:00.209177+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:39:24.033519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.205191+0000","last_change":"2024-05-10T11:14:25.271602+0000","last_active":"2024-05-10T11:17:00.205191+0000","last_peered":"2024-05-10T11:17:00.205191+0000","last_clean":"2024-05-10T11:17:00.205191+0000","last_became_active":"2024-05-10T11:14:25.267206+0000","last_became_peered":"2024-05-10T11:14:25.267206+0000","last_unstale":"2024-05-10T11:17:00.205191+0000","last_undegraded":"2024-05-10T11:17:00.205191+0000","last_fullsized":"2024-05-10T11:17:00.205191+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:28:24.708297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.206462+0000","last_change":"2024-05-10T11:14:25.280378+0000","last_active":"2024-05-10T11:17:00.206462+0000","last_peered":"2024-05-10T11:17:00.206462+0000","last_clean":"2024-05-10T11:17:00.206462+0000","last_became_active":"2024-05-10T11:14:25.280079+0000","last_became_peered":"2024-05-10T11:14:25.280079+0000","last_unstale":"2024-05-10T11:17:00.206462+0000","last_undegraded":"2024-05-10T11:17:00.206462+0000","last_fullsized":"2024-05-10T11:17:00.206462+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:13:47.781060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.204726+0000","last_change":"2024-05-10T11:14:25.272363+0000","last_active":"2024-05-10T11:17:00.204726+0000","last_peered":"2024-05-10T11:17:00.204726+0000","last_clean":"2024-05-10T11:17:00.204726+0000","last_became_active":"2024-05-10T11:14:25.271847+0000","last_became_peered":"2024-05-10T11:14:25.271847+0000","last_unstale":"2024-05-10T11:17:00.204726+0000","last_undegraded":"2024-05-10T11:17:00.204726+0000","last_fullsized":"2024-05-10T11:17:00.204726+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:47:54.661069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.203714+0000","last_change":"2024-05-10T11:14:25.272688+0000","last_active":"2024-05-10T11:17:00.203714+0000","last_peered":"2024-05-10T11:17:00.203714+0000","last_clean":"2024-05-10T11:17:00.203714+0000","last_became_active":"2024-05-10T11:14:25.271654+0000","last_became_peered":"2024-05-10T11:14:25.271654+0000","last_unstale":"2024-05-10T11:17:00.203714+0000","last_undegraded":"2024-05-10T11:17:00.203714+0000","last_fullsized":"2024-05-10T11:17:00.203714+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:50:21.149707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.199436+0000","last_change":"2024-05-10T11:15:01.432279+0000","last_active":"2024-05-10T11:17:00.199436+0000","last_peered":"2024-05-10T11:17:00.199436+0000","last_clean":"2024-05-10T11:17:00.199436+0000","last_became_active":"2024-05-10T11:15:01.431751+0000","last_became_peered":"2024-05-10T11:15:01.431751+0000","last_unstale":"2024-05-10T11:17:00.199436+0000","last_undegraded":"2024-05-10T11:17:00.199436+0000","last_fullsized":"2024-05-10T11:17:00.199436+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:25:44.196618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74852,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1077,"num_read_kb":620,"num_write":190,"num_write_kb":140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":224556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":162,"ondisk_log_size":162,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645377,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1015808,"data_stored":505095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841606,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":41164,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93703988,"statfs":{"total":95995035648,"available":95952883712,"internally_reserved":0,"allocated":1708032,"data_stored":1162558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":40434113},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005132,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1032192,"data_stored":509405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201361,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1024000,"data_stored":505207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430295,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28600,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716552,"statfs":{"total":95995035648,"available":95965749248,"internally_reserved":0,"allocated":1622016,"data_stored":1102797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593820,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":500485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574964,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1081344,"data_stored":569587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738490,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1032192,"data_stored":501873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411440,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36416,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708736,"statfs":{"total":95995035648,"available":95957745664,"internally_reserved":0,"allocated":1105920,"data_stored":571962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149802,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1019904,"data_stored":500679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953574,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1019904,"data_stored":506160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790048,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28608,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716544,"statfs":{"total":95995035648,"available":95965741056,"internally_reserved":0,"allocated":1630208,"data_stored":1097360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":69192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":71567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":71795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T11:18:28.951 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-05-10T11:18:28.951 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T11:18:29.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:29.087+0000 7f09c5512640 1 -- 172.21.15.100:0/1211767998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09c0102ac0 msgr2=0x7f09c0102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:29.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:29.087+0000 7f09c5512640 1 --2- 172.21.15.100:0/1211767998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09c0102ac0 0x7f09c0102f40 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f09a8009900 tx=0x7f09a802a410 comp rx=0 tx=0).stop 2024-05-10T11:18:29.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:29.087+0000 7f09c5512640 1 -- 172.21.15.100:0/1211767998 shutdown_connections 2024-05-10T11:18:29.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:29.087+0000 7f09c5512640 1 --2- 172.21.15.100:0/1211767998 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09c0103480 0x7f09c0109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:29.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:29.087+0000 7f09c5512640 1 --2- 172.21.15.100:0/1211767998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09c0102ac0 0x7f09c0102f40 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:29.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:29.087+0000 7f09c5512640 1 --2- 172.21.15.100:0/1211767998 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09c01018d0 0x7f09c0101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:29.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:29.087+0000 7f09c5512640 1 -- 172.21.15.100:0/1211767998 >> 172.21.15.100:0/1211767998 conn(0x7f09c00fce40 msgr2=0x7f09c00ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:29.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:29.087+0000 7f09c5512640 1 -- 172.21.15.100:0/1211767998 shutdown_connections 2024-05-10T11:18:29.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:29.087+0000 7f09c5512640 1 -- 172.21.15.100:0/1211767998 wait complete. 2024-05-10T11:18:29.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:29.088+0000 7f09c5512640 1 Processor -- start 2024-05-10T11:18:29.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:29.088+0000 7f09c5512640 1 -- start start 2024-05-10T11:18:29.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:29.088+0000 7f09c5512640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09c01018d0 0x7f09c006a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:29.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:29.088+0000 7f09c5512640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09c0102ac0 0x7f09c019d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:29.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:29.088+0000 7f09c5512640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09c0103480 0x7f09c019fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:29.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:29.088+0000 7f09c5512640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f09c010d310 con 0x7f09c0103480 2024-05-10T11:18:29.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:29.088+0000 7f09c5512640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f09c010d190 con 0x7f09c0102ac0 2024-05-10T11:18:29.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:29.088+0000 7f09c5512640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f09c010d490 con 0x7f09c01018d0 2024-05-10T11:18:29.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:29.088+0000 7f09bf7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09c0103480 0x7f09c019fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:29.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:29.088+0000 7f09bf7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09c0103480 0x7f09c019fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42300/0 (socket says 172.21.15.100:42300) 2024-05-10T11:18:29.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:29.088+0000 7f09bf7fe640 1 -- 172.21.15.100:0/2716960077 learned_addr learned my addr 172.21.15.100:0/2716960077 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:18:29.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:29.089+0000 7f09b67fc640 1 --2- 172.21.15.100:0/2716960077 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09c0102ac0 0x7f09c019d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:29.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:29.089+0000 7f09bf7fe640 1 -- 172.21.15.100:0/2716960077 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09c01018d0 msgr2=0x7f09c006a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:29.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:29.089+0000 7f09bf7fe640 1 --2- 172.21.15.100:0/2716960077 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09c01018d0 0x7f09c006a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:29.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:29.089+0000 7f09bf7fe640 1 -- 172.21.15.100:0/2716960077 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09c0102ac0 msgr2=0x7f09c019d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:29.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:29.089+0000 7f09bf7fe640 1 --2- 172.21.15.100:0/2716960077 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09c0102ac0 0x7f09c019d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:29.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:29.089+0000 7f09bf7fe640 1 -- 172.21.15.100:0/2716960077 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09c01a0800 con 0x7f09c0103480 2024-05-10T11:18:29.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:29.089+0000 7f09b67fc640 1 --2- 172.21.15.100:0/2716960077 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09c0102ac0 0x7f09c019d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:18:29.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:29.089+0000 7f09bf7fe640 1 --2- 172.21.15.100:0/2716960077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09c0103480 0x7f09c019fef0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f09b800b970 tx=0x7f09b800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:29.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:29.089+0000 7f09bcff9640 1 -- 172.21.15.100:0/2716960077 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b8019070 con 0x7f09c0103480 2024-05-10T11:18:29.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:29.089+0000 7f09c5512640 1 -- 172.21.15.100:0/2716960077 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09c01a0a90 con 0x7f09c0103480 2024-05-10T11:18:29.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:29.089+0000 7f09bcff9640 1 -- 172.21.15.100:0/2716960077 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f09b8014440 con 0x7f09c0103480 2024-05-10T11:18:29.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:29.089+0000 7f09c5512640 1 -- 172.21.15.100:0/2716960077 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09c01a1000 con 0x7f09c0103480 2024-05-10T11:18:29.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:29.089+0000 7f09bcff9640 1 -- 172.21.15.100:0/2716960077 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b801d5b0 con 0x7f09c0103480 2024-05-10T11:18:29.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:29.090+0000 7f09c5512640 1 -- 172.21.15.100:0/2716960077 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09c0108d30 con 0x7f09c0103480 2024-05-10T11:18:29.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:29.092+0000 7f09bcff9640 1 -- 172.21.15.100:0/2716960077 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f09b801c070 con 0x7f09c0103480 2024-05-10T11:18:29.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:29.093+0000 7f09bcff9640 1 --2- 172.21.15.100:0/2716960077 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f09ac0b27f0 0x7f09ac0b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:29.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:29.093+0000 7f09bcff9640 1 -- 172.21.15.100:0/2716960077 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f09b80e1fe0 con 0x7f09c0103480 2024-05-10T11:18:29.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:29.093+0000 7f09beffd640 1 --2- 172.21.15.100:0/2716960077 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f09ac0b27f0 0x7f09ac0b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:29.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:29.093+0000 7f09beffd640 1 --2- 172.21.15.100:0/2716960077 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f09ac0b27f0 0x7f09ac0b4cb0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f09c0102920 tx=0x7f09b0005ed0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:29.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:29.096+0000 7f09bcff9640 1 -- 172.21.15.100:0/2716960077 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f09b80a9940 con 0x7f09c0103480 2024-05-10T11:18:29.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:29.289+0000 7f09c5512640 1 -- 172.21.15.100:0/2716960077 --> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f09c0062df0 con 0x7f09ac0b27f0 2024-05-10T11:18:29.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:29.298+0000 7f09bcff9640 1 -- 172.21.15.100:0/2716960077 <== mgr.14152 v2:172.21.15.100:6800/2507287356 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340476 (secure 0 0 0) 0x7f09c0062df0 con 0x7f09ac0b27f0 2024-05-10T11:18:29.301 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:18:29.303 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-05-10T11:18:29.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:29.301+0000 7f09c5512640 1 -- 172.21.15.100:0/2716960077 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f09ac0b27f0 msgr2=0x7f09ac0b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:29.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:29.301+0000 7f09c5512640 1 --2- 172.21.15.100:0/2716960077 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f09ac0b27f0 0x7f09ac0b4cb0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f09c0102920 tx=0x7f09b0005ed0 comp rx=0 tx=0).stop 2024-05-10T11:18:29.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:29.301+0000 7f09c5512640 1 -- 172.21.15.100:0/2716960077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09c0103480 msgr2=0x7f09c019fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:29.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:29.301+0000 7f09c5512640 1 --2- 172.21.15.100:0/2716960077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09c0103480 0x7f09c019fef0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f09b800b970 tx=0x7f09b800be30 comp rx=0 tx=0).stop 2024-05-10T11:18:29.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:29.301+0000 7f09c5512640 1 -- 172.21.15.100:0/2716960077 shutdown_connections 2024-05-10T11:18:29.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:29.301+0000 7f09c5512640 1 --2- 172.21.15.100:0/2716960077 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f09ac0b27f0 0x7f09ac0b4cb0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:29.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:29.301+0000 7f09c5512640 1 --2- 172.21.15.100:0/2716960077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09c0103480 0x7f09c019fef0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:29.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:29.301+0000 7f09c5512640 1 --2- 172.21.15.100:0/2716960077 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09c0102ac0 0x7f09c019d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:29.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:29.301+0000 7f09c5512640 1 --2- 172.21.15.100:0/2716960077 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09c01018d0 0x7f09c006a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:29.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:29.301+0000 7f09c5512640 1 -- 172.21.15.100:0/2716960077 >> 172.21.15.100:0/2716960077 conn(0x7f09c00fce40 msgr2=0x7f09c0105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:29.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:29.301+0000 7f09c5512640 1 -- 172.21.15.100:0/2716960077 shutdown_connections 2024-05-10T11:18:29.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:29.301+0000 7f09c5512640 1 -- 172.21.15.100:0/2716960077 wait complete. 2024-05-10T11:18:29.313 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":423,"stamp":"2024-05-10T11:18:28.248281+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665454,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1153,"num_read_kb":684,"num_write":315,"num_write_kb":2292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":756,"ondisk_log_size":756,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":404716,"kb_used_data":13964,"kb_used_omap":18,"kb_used_meta":390637,"kb_avail":1124537108,"statfs":{"total":1151940427776,"available":1151525998592,"internally_reserved":0,"allocated":14299136,"data_stored":8033168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":400012603},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5793,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":84,"num_read_kb":53,"num_write":11,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003400"},"pg_stats":[{"pgid":"3.3f","version":"89'8","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.206983+0000","last_change":"2024-05-10T11:14:25.286878+0000","last_active":"2024-05-10T11:17:00.206983+0000","last_peered":"2024-05-10T11:17:00.206983+0000","last_clean":"2024-05-10T11:17:00.206983+0000","last_became_active":"2024-05-10T11:14:25.286462+0000","last_became_peered":"2024-05-10T11:14:25.286462+0000","last_unstale":"2024-05-10T11:17:00.206983+0000","last_undegraded":"2024-05-10T11:17:00.206983+0000","last_fullsized":"2024-05-10T11:17:00.206983+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:13:40.627138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.204906+0000","last_change":"2024-05-10T11:14:25.280884+0000","last_active":"2024-05-10T11:17:00.204906+0000","last_peered":"2024-05-10T11:17:00.204906+0000","last_clean":"2024-05-10T11:17:00.204906+0000","last_became_active":"2024-05-10T11:14:25.280564+0000","last_became_peered":"2024-05-10T11:14:25.280564+0000","last_unstale":"2024-05-10T11:17:00.204906+0000","last_undegraded":"2024-05-10T11:17:00.204906+0000","last_fullsized":"2024-05-10T11:17:00.204906+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:14:16.735326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.208402+0000","last_change":"2024-05-10T11:14:25.278676+0000","last_active":"2024-05-10T11:17:00.208402+0000","last_peered":"2024-05-10T11:17:00.208402+0000","last_clean":"2024-05-10T11:17:00.208402+0000","last_became_active":"2024-05-10T11:14:25.278196+0000","last_became_peered":"2024-05-10T11:14:25.278196+0000","last_unstale":"2024-05-10T11:17:00.208402+0000","last_undegraded":"2024-05-10T11:17:00.208402+0000","last_fullsized":"2024-05-10T11:17:00.208402+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:50:29.390014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.207328+0000","last_change":"2024-05-10T11:14:25.274741+0000","last_active":"2024-05-10T11:17:00.207328+0000","last_peered":"2024-05-10T11:17:00.207328+0000","last_clean":"2024-05-10T11:17:00.207328+0000","last_became_active":"2024-05-10T11:14:25.274251+0000","last_became_peered":"2024-05-10T11:14:25.274251+0000","last_unstale":"2024-05-10T11:17:00.207328+0000","last_undegraded":"2024-05-10T11:17:00.207328+0000","last_fullsized":"2024-05-10T11:17:00.207328+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:04:41.552042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.186404+0000","last_change":"2024-05-10T11:14:25.265232+0000","last_active":"2024-05-10T11:17:00.186404+0000","last_peered":"2024-05-10T11:17:00.186404+0000","last_clean":"2024-05-10T11:17:00.186404+0000","last_became_active":"2024-05-10T11:14:25.264852+0000","last_became_peered":"2024-05-10T11:14:25.264852+0000","last_unstale":"2024-05-10T11:17:00.186404+0000","last_undegraded":"2024-05-10T11:17:00.186404+0000","last_fullsized":"2024-05-10T11:17:00.186404+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:50:27.207258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.210357+0000","last_change":"2024-05-10T11:14:25.272626+0000","last_active":"2024-05-10T11:17:00.210357+0000","last_peered":"2024-05-10T11:17:00.210357+0000","last_clean":"2024-05-10T11:17:00.210357+0000","last_became_active":"2024-05-10T11:14:25.272305+0000","last_became_peered":"2024-05-10T11:14:25.272305+0000","last_unstale":"2024-05-10T11:17:00.210357+0000","last_undegraded":"2024-05-10T11:17:00.210357+0000","last_fullsized":"2024-05-10T11:17:00.210357+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:44:16.725453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.210435+0000","last_change":"2024-05-10T11:14:25.275460+0000","last_active":"2024-05-10T11:17:00.210435+0000","last_peered":"2024-05-10T11:17:00.210435+0000","last_clean":"2024-05-10T11:17:00.210435+0000","last_became_active":"2024-05-10T11:14:25.274571+0000","last_became_peered":"2024-05-10T11:14:25.274571+0000","last_unstale":"2024-05-10T11:17:00.210435+0000","last_undegraded":"2024-05-10T11:17:00.210435+0000","last_fullsized":"2024-05-10T11:17:00.210435+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:33:20.335738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.207412+0000","last_change":"2024-05-10T11:14:25.273975+0000","last_active":"2024-05-10T11:17:00.207412+0000","last_peered":"2024-05-10T11:17:00.207412+0000","last_clean":"2024-05-10T11:17:00.207412+0000","last_became_active":"2024-05-10T11:14:25.273652+0000","last_became_peered":"2024-05-10T11:14:25.273652+0000","last_unstale":"2024-05-10T11:17:00.207412+0000","last_undegraded":"2024-05-10T11:17:00.207412+0000","last_fullsized":"2024-05-10T11:17:00.207412+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:37:55.354449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.207494+0000","last_change":"2024-05-10T11:14:25.285530+0000","last_active":"2024-05-10T11:17:00.207494+0000","last_peered":"2024-05-10T11:17:00.207494+0000","last_clean":"2024-05-10T11:17:00.207494+0000","last_became_active":"2024-05-10T11:14:25.285235+0000","last_became_peered":"2024-05-10T11:14:25.285235+0000","last_unstale":"2024-05-10T11:17:00.207494+0000","last_undegraded":"2024-05-10T11:17:00.207494+0000","last_fullsized":"2024-05-10T11:17:00.207494+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:11:14.225339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.210587+0000","last_change":"2024-05-10T11:14:25.274738+0000","last_active":"2024-05-10T11:17:00.210587+0000","last_peered":"2024-05-10T11:17:00.210587+0000","last_clean":"2024-05-10T11:17:00.210587+0000","last_became_active":"2024-05-10T11:14:25.274107+0000","last_became_peered":"2024-05-10T11:14:25.274107+0000","last_unstale":"2024-05-10T11:17:00.210587+0000","last_undegraded":"2024-05-10T11:17:00.210587+0000","last_fullsized":"2024-05-10T11:17:00.210587+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:02:01.489930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.213489+0000","last_change":"2024-05-10T11:14:25.277431+0000","last_active":"2024-05-10T11:17:00.213489+0000","last_peered":"2024-05-10T11:17:00.213489+0000","last_clean":"2024-05-10T11:17:00.213489+0000","last_became_active":"2024-05-10T11:14:25.277130+0000","last_became_peered":"2024-05-10T11:14:25.277130+0000","last_unstale":"2024-05-10T11:17:00.213489+0000","last_undegraded":"2024-05-10T11:17:00.213489+0000","last_fullsized":"2024-05-10T11:17:00.213489+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:38:17.192282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.206976+0000","last_change":"2024-05-10T11:14:25.276291+0000","last_active":"2024-05-10T11:17:00.206976+0000","last_peered":"2024-05-10T11:17:00.206976+0000","last_clean":"2024-05-10T11:17:00.206976+0000","last_became_active":"2024-05-10T11:14:25.275842+0000","last_became_peered":"2024-05-10T11:14:25.275842+0000","last_unstale":"2024-05-10T11:17:00.206976+0000","last_undegraded":"2024-05-10T11:17:00.206976+0000","last_fullsized":"2024-05-10T11:17:00.206976+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:20:53.443135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.209964+0000","last_change":"2024-05-10T11:14:25.273950+0000","last_active":"2024-05-10T11:17:00.209964+0000","last_peered":"2024-05-10T11:17:00.209964+0000","last_clean":"2024-05-10T11:17:00.209964+0000","last_became_active":"2024-05-10T11:14:25.273644+0000","last_became_peered":"2024-05-10T11:14:25.273644+0000","last_unstale":"2024-05-10T11:17:00.209964+0000","last_undegraded":"2024-05-10T11:17:00.209964+0000","last_fullsized":"2024-05-10T11:17:00.209964+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:46:05.112575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.209077+0000","last_change":"2024-05-10T11:14:25.277930+0000","last_active":"2024-05-10T11:17:00.209077+0000","last_peered":"2024-05-10T11:17:00.209077+0000","last_clean":"2024-05-10T11:17:00.209077+0000","last_became_active":"2024-05-10T11:14:25.276793+0000","last_became_peered":"2024-05-10T11:14:25.276793+0000","last_unstale":"2024-05-10T11:17:00.209077+0000","last_undegraded":"2024-05-10T11:17:00.209077+0000","last_fullsized":"2024-05-10T11:17:00.209077+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:47:03.592882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.214057+0000","last_change":"2024-05-10T11:14:25.270434+0000","last_active":"2024-05-10T11:17:00.214057+0000","last_peered":"2024-05-10T11:17:00.214057+0000","last_clean":"2024-05-10T11:17:00.214057+0000","last_became_active":"2024-05-10T11:14:25.270139+0000","last_became_peered":"2024-05-10T11:14:25.270139+0000","last_unstale":"2024-05-10T11:17:00.214057+0000","last_undegraded":"2024-05-10T11:17:00.214057+0000","last_fullsized":"2024-05-10T11:17:00.214057+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:58:40.506560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.209206+0000","last_change":"2024-05-10T11:14:25.272981+0000","last_active":"2024-05-10T11:17:00.209206+0000","last_peered":"2024-05-10T11:17:00.209206+0000","last_clean":"2024-05-10T11:17:00.209206+0000","last_became_active":"2024-05-10T11:14:25.272592+0000","last_became_peered":"2024-05-10T11:14:25.272592+0000","last_unstale":"2024-05-10T11:17:00.209206+0000","last_undegraded":"2024-05-10T11:17:00.209206+0000","last_fullsized":"2024-05-10T11:17:00.209206+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:46:09.673013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.212493+0000","last_change":"2024-05-10T11:14:25.287304+0000","last_active":"2024-05-10T11:17:00.212493+0000","last_peered":"2024-05-10T11:17:00.212493+0000","last_clean":"2024-05-10T11:17:00.212493+0000","last_became_active":"2024-05-10T11:14:25.286964+0000","last_became_peered":"2024-05-10T11:14:25.286964+0000","last_unstale":"2024-05-10T11:17:00.212493+0000","last_undegraded":"2024-05-10T11:17:00.212493+0000","last_fullsized":"2024-05-10T11:17:00.212493+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:32:27.006840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.209855+0000","last_change":"2024-05-10T11:14:25.275366+0000","last_active":"2024-05-10T11:17:00.209855+0000","last_peered":"2024-05-10T11:17:00.209855+0000","last_clean":"2024-05-10T11:17:00.209855+0000","last_became_active":"2024-05-10T11:14:25.274986+0000","last_became_peered":"2024-05-10T11:14:25.274986+0000","last_unstale":"2024-05-10T11:17:00.209855+0000","last_undegraded":"2024-05-10T11:17:00.209855+0000","last_fullsized":"2024-05-10T11:17:00.209855+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:50:50.071975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.203474+0000","last_change":"2024-05-10T11:14:25.274588+0000","last_active":"2024-05-10T11:17:00.203474+0000","last_peered":"2024-05-10T11:17:00.203474+0000","last_clean":"2024-05-10T11:17:00.203474+0000","last_became_active":"2024-05-10T11:14:25.274179+0000","last_became_peered":"2024-05-10T11:14:25.274179+0000","last_unstale":"2024-05-10T11:17:00.203474+0000","last_undegraded":"2024-05-10T11:17:00.203474+0000","last_fullsized":"2024-05-10T11:17:00.203474+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:06:23.517082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.461771+0000","last_change":"2024-05-10T11:15:01.461337+0000","last_active":"2024-05-10T11:15:01.461771+0000","last_peered":"2024-05-10T11:15:01.461771+0000","last_clean":"2024-05-10T11:15:01.461771+0000","last_became_active":"2024-05-10T11:15:01.460776+0000","last_became_peered":"2024-05-10T11:15:01.460776+0000","last_unstale":"2024-05-10T11:15:01.461771+0000","last_undegraded":"2024-05-10T11:15:01.461771+0000","last_fullsized":"2024-05-10T11:15:01.461771+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:06:49.156483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,10,0],"acting":[2,3,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:32.813572+0000","last_change":"2024-05-10T11:14:22.253909+0000","last_active":"2024-05-10T11:15:32.813572+0000","last_peered":"2024-05-10T11:15:32.813572+0000","last_clean":"2024-05-10T11:15:32.813572+0000","last_became_active":"2024-05-10T11:14:22.253400+0000","last_became_peered":"2024-05-10T11:14:22.253400+0000","last_unstale":"2024-05-10T11:15:32.813572+0000","last_undegraded":"2024-05-10T11:15:32.813572+0000","last_fullsized":"2024-05-10T11:15:32.813572+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:06:46.173511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.206338+0000","last_change":"2024-05-10T11:14:25.273675+0000","last_active":"2024-05-10T11:17:00.206338+0000","last_peered":"2024-05-10T11:17:00.206338+0000","last_clean":"2024-05-10T11:17:00.206338+0000","last_became_active":"2024-05-10T11:14:25.273320+0000","last_became_peered":"2024-05-10T11:14:25.273320+0000","last_unstale":"2024-05-10T11:17:00.206338+0000","last_undegraded":"2024-05-10T11:17:00.206338+0000","last_fullsized":"2024-05-10T11:17:00.206338+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:09:18.232236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459008+0000","last_change":"2024-05-10T11:14:35.326204+0000","last_active":"2024-05-10T11:15:01.459008+0000","last_peered":"2024-05-10T11:15:01.459008+0000","last_clean":"2024-05-10T11:15:01.459008+0000","last_became_active":"2024-05-10T11:14:35.325715+0000","last_became_peered":"2024-05-10T11:14:35.325715+0000","last_unstale":"2024-05-10T11:15:01.459008+0000","last_undegraded":"2024-05-10T11:15:01.459008+0000","last_fullsized":"2024-05-10T11:15:01.459008+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:06:30.665861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:16:23.904221+0000","last_change":"2024-05-10T11:14:22.299194+0000","last_active":"2024-05-10T11:16:23.904221+0000","last_peered":"2024-05-10T11:16:23.904221+0000","last_clean":"2024-05-10T11:16:23.904221+0000","last_became_active":"2024-05-10T11:14:22.298807+0000","last_became_peered":"2024-05-10T11:14:22.298807+0000","last_unstale":"2024-05-10T11:16:23.904221+0000","last_undegraded":"2024-05-10T11:16:23.904221+0000","last_fullsized":"2024-05-10T11:16:23.904221+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:24:51.054834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.209768+0000","last_change":"2024-05-10T11:14:25.277860+0000","last_active":"2024-05-10T11:17:00.209768+0000","last_peered":"2024-05-10T11:17:00.209768+0000","last_clean":"2024-05-10T11:17:00.209768+0000","last_became_active":"2024-05-10T11:14:25.276784+0000","last_became_peered":"2024-05-10T11:14:25.276784+0000","last_unstale":"2024-05-10T11:17:00.209768+0000","last_undegraded":"2024-05-10T11:17:00.209768+0000","last_fullsized":"2024-05-10T11:17:00.209768+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:42:57.673595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459304+0000","last_change":"2024-05-10T11:14:35.330679+0000","last_active":"2024-05-10T11:15:01.459304+0000","last_peered":"2024-05-10T11:15:01.459304+0000","last_clean":"2024-05-10T11:15:01.459304+0000","last_became_active":"2024-05-10T11:14:35.329819+0000","last_became_peered":"2024-05-10T11:14:35.329819+0000","last_unstale":"2024-05-10T11:15:01.459304+0000","last_undegraded":"2024-05-10T11:15:01.459304+0000","last_fullsized":"2024-05-10T11:15:01.459304+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:31:26.982784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:29.798740+0000","last_change":"2024-05-10T11:14:22.260402+0000","last_active":"2024-05-10T11:15:29.798740+0000","last_peered":"2024-05-10T11:15:29.798740+0000","last_clean":"2024-05-10T11:15:29.798740+0000","last_became_active":"2024-05-10T11:14:22.259619+0000","last_became_peered":"2024-05-10T11:14:22.259619+0000","last_unstale":"2024-05-10T11:15:29.798740+0000","last_undegraded":"2024-05-10T11:15:29.798740+0000","last_fullsized":"2024-05-10T11:15:29.798740+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:56:34.491626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.204002+0000","last_change":"2024-05-10T11:14:25.276117+0000","last_active":"2024-05-10T11:17:00.204002+0000","last_peered":"2024-05-10T11:17:00.204002+0000","last_clean":"2024-05-10T11:17:00.204002+0000","last_became_active":"2024-05-10T11:14:25.275758+0000","last_became_peered":"2024-05-10T11:14:25.275758+0000","last_unstale":"2024-05-10T11:17:00.204002+0000","last_undegraded":"2024-05-10T11:17:00.204002+0000","last_fullsized":"2024-05-10T11:17:00.204002+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:12:16.582244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459213+0000","last_change":"2024-05-10T11:14:35.324633+0000","last_active":"2024-05-10T11:15:01.459213+0000","last_peered":"2024-05-10T11:15:01.459213+0000","last_clean":"2024-05-10T11:15:01.459213+0000","last_became_active":"2024-05-10T11:14:35.324152+0000","last_became_peered":"2024-05-10T11:14:35.324152+0000","last_unstale":"2024-05-10T11:15:01.459213+0000","last_undegraded":"2024-05-10T11:15:01.459213+0000","last_fullsized":"2024-05-10T11:15:01.459213+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:56:50.008782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.430353+0000","last_change":"2024-05-10T11:14:22.259086+0000","last_active":"2024-05-10T11:15:01.430353+0000","last_peered":"2024-05-10T11:15:01.430353+0000","last_clean":"2024-05-10T11:15:01.430353+0000","last_became_active":"2024-05-10T11:14:22.258628+0000","last_became_peered":"2024-05-10T11:14:22.258628+0000","last_unstale":"2024-05-10T11:15:01.430353+0000","last_undegraded":"2024-05-10T11:15:01.430353+0000","last_fullsized":"2024-05-10T11:15:01.430353+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:39:13.472622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.213935+0000","last_change":"2024-05-10T11:14:25.277556+0000","last_active":"2024-05-10T11:17:00.213935+0000","last_peered":"2024-05-10T11:17:00.213935+0000","last_clean":"2024-05-10T11:17:00.213935+0000","last_became_active":"2024-05-10T11:14:25.277201+0000","last_became_peered":"2024-05-10T11:14:25.277201+0000","last_unstale":"2024-05-10T11:17:00.213935+0000","last_undegraded":"2024-05-10T11:17:00.213935+0000","last_fullsized":"2024-05-10T11:17:00.213935+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:04:17.028305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.426356+0000","last_change":"2024-05-10T11:14:35.323338+0000","last_active":"2024-05-10T11:15:01.426356+0000","last_peered":"2024-05-10T11:15:01.426356+0000","last_clean":"2024-05-10T11:15:01.426356+0000","last_became_active":"2024-05-10T11:14:35.321948+0000","last_became_peered":"2024-05-10T11:14:35.321948+0000","last_unstale":"2024-05-10T11:15:01.426356+0000","last_undegraded":"2024-05-10T11:15:01.426356+0000","last_fullsized":"2024-05-10T11:15:01.426356+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:41:30.569125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'8","reported_seq":225,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:18:23.069244+0000","last_change":"2024-05-10T11:14:22.259187+0000","last_active":"2024-05-10T11:18:23.069244+0000","last_peered":"2024-05-10T11:18:23.069244+0000","last_clean":"2024-05-10T11:18:23.069244+0000","last_became_active":"2024-05-10T11:14:22.258812+0000","last_became_peered":"2024-05-10T11:14:22.258812+0000","last_unstale":"2024-05-10T11:18:23.069244+0000","last_undegraded":"2024-05-10T11:18:23.069244+0000","last_fullsized":"2024-05-10T11:18:23.069244+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:15:38.769705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1647,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":177,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.188655+0000","last_change":"2024-05-10T11:14:25.262469+0000","last_active":"2024-05-10T11:17:00.188655+0000","last_peered":"2024-05-10T11:17:00.188655+0000","last_clean":"2024-05-10T11:17:00.188655+0000","last_became_active":"2024-05-10T11:14:25.262104+0000","last_became_peered":"2024-05-10T11:14:25.262104+0000","last_unstale":"2024-05-10T11:17:00.188655+0000","last_undegraded":"2024-05-10T11:17:00.188655+0000","last_fullsized":"2024-05-10T11:17:00.188655+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:48:24.329136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.426927+0000","last_change":"2024-05-10T11:14:35.327737+0000","last_active":"2024-05-10T11:15:01.426927+0000","last_peered":"2024-05-10T11:15:01.426927+0000","last_clean":"2024-05-10T11:15:01.426927+0000","last_became_active":"2024-05-10T11:14:35.327209+0000","last_became_peered":"2024-05-10T11:14:35.327209+0000","last_unstale":"2024-05-10T11:15:01.426927+0000","last_undegraded":"2024-05-10T11:15:01.426927+0000","last_fullsized":"2024-05-10T11:15:01.426927+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:32:11.784752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:29.797324+0000","last_change":"2024-05-10T11:14:22.254853+0000","last_active":"2024-05-10T11:15:29.797324+0000","last_peered":"2024-05-10T11:15:29.797324+0000","last_clean":"2024-05-10T11:15:29.797324+0000","last_became_active":"2024-05-10T11:14:22.253352+0000","last_became_peered":"2024-05-10T11:14:22.253352+0000","last_unstale":"2024-05-10T11:15:29.797324+0000","last_undegraded":"2024-05-10T11:15:29.797324+0000","last_fullsized":"2024-05-10T11:15:29.797324+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:14:15.871042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.213028+0000","last_change":"2024-05-10T11:14:25.266798+0000","last_active":"2024-05-10T11:17:00.213028+0000","last_peered":"2024-05-10T11:17:00.213028+0000","last_clean":"2024-05-10T11:17:00.213028+0000","last_became_active":"2024-05-10T11:14:25.266384+0000","last_became_peered":"2024-05-10T11:14:25.266384+0000","last_unstale":"2024-05-10T11:17:00.213028+0000","last_undegraded":"2024-05-10T11:17:00.213028+0000","last_fullsized":"2024-05-10T11:17:00.213028+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:18:07.338987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.463232+0000","last_change":"2024-05-10T11:15:01.462376+0000","last_active":"2024-05-10T11:15:01.463232+0000","last_peered":"2024-05-10T11:15:01.463232+0000","last_clean":"2024-05-10T11:15:01.463232+0000","last_became_active":"2024-05-10T11:15:01.461876+0000","last_became_peered":"2024-05-10T11:15:01.461876+0000","last_unstale":"2024-05-10T11:15:01.463232+0000","last_undegraded":"2024-05-10T11:15:01.463232+0000","last_fullsized":"2024-05-10T11:15:01.463232+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:35:52.076414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:29.797310+0000","last_change":"2024-05-10T11:14:22.253976+0000","last_active":"2024-05-10T11:15:29.797310+0000","last_peered":"2024-05-10T11:15:29.797310+0000","last_clean":"2024-05-10T11:15:29.797310+0000","last_became_active":"2024-05-10T11:14:22.252230+0000","last_became_peered":"2024-05-10T11:14:22.252230+0000","last_unstale":"2024-05-10T11:15:29.797310+0000","last_undegraded":"2024-05-10T11:15:29.797310+0000","last_fullsized":"2024-05-10T11:15:29.797310+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:14:03.208811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.206683+0000","last_change":"2024-05-10T11:14:25.279627+0000","last_active":"2024-05-10T11:17:00.206683+0000","last_peered":"2024-05-10T11:17:00.206683+0000","last_clean":"2024-05-10T11:17:00.206683+0000","last_became_active":"2024-05-10T11:14:25.279330+0000","last_became_peered":"2024-05-10T11:14:25.279330+0000","last_unstale":"2024-05-10T11:17:00.206683+0000","last_undegraded":"2024-05-10T11:17:00.206683+0000","last_fullsized":"2024-05-10T11:17:00.206683+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:50:19.608053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.426065+0000","last_change":"2024-05-10T11:14:35.321723+0000","last_active":"2024-05-10T11:15:01.426065+0000","last_peered":"2024-05-10T11:15:01.426065+0000","last_clean":"2024-05-10T11:15:01.426065+0000","last_became_active":"2024-05-10T11:14:35.321384+0000","last_became_peered":"2024-05-10T11:14:35.321384+0000","last_unstale":"2024-05-10T11:15:01.426065+0000","last_undegraded":"2024-05-10T11:15:01.426065+0000","last_fullsized":"2024-05-10T11:15:01.426065+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:04:07.770889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:28.790869+0000","last_change":"2024-05-10T11:14:22.255511+0000","last_active":"2024-05-10T11:15:28.790869+0000","last_peered":"2024-05-10T11:15:28.790869+0000","last_clean":"2024-05-10T11:15:28.790869+0000","last_became_active":"2024-05-10T11:14:22.254722+0000","last_became_peered":"2024-05-10T11:14:22.254722+0000","last_unstale":"2024-05-10T11:15:28.790869+0000","last_undegraded":"2024-05-10T11:15:28.790869+0000","last_fullsized":"2024-05-10T11:15:28.790869+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:38:27.096011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.209442+0000","last_change":"2024-05-10T11:14:25.274009+0000","last_active":"2024-05-10T11:17:00.209442+0000","last_peered":"2024-05-10T11:17:00.209442+0000","last_clean":"2024-05-10T11:17:00.209442+0000","last_became_active":"2024-05-10T11:14:25.273672+0000","last_became_peered":"2024-05-10T11:14:25.273672+0000","last_unstale":"2024-05-10T11:17:00.209442+0000","last_undegraded":"2024-05-10T11:17:00.209442+0000","last_fullsized":"2024-05-10T11:17:00.209442+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:42:57.210131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459026+0000","last_change":"2024-05-10T11:14:35.321520+0000","last_active":"2024-05-10T11:15:01.459026+0000","last_peered":"2024-05-10T11:15:01.459026+0000","last_clean":"2024-05-10T11:15:01.459026+0000","last_became_active":"2024-05-10T11:14:35.320575+0000","last_became_peered":"2024-05-10T11:14:35.320575+0000","last_unstale":"2024-05-10T11:15:01.459026+0000","last_undegraded":"2024-05-10T11:15:01.459026+0000","last_fullsized":"2024-05-10T11:15:01.459026+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:54:03.183426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'5","reported_seq":219,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:18:25.071520+0000","last_change":"2024-05-10T11:14:22.251463+0000","last_active":"2024-05-10T11:18:25.071520+0000","last_peered":"2024-05-10T11:18:25.071520+0000","last_clean":"2024-05-10T11:18:25.071520+0000","last_became_active":"2024-05-10T11:14:22.251132+0000","last_became_peered":"2024-05-10T11:14:22.251132+0000","last_unstale":"2024-05-10T11:18:25.071520+0000","last_undegraded":"2024-05-10T11:18:25.071520+0000","last_fullsized":"2024-05-10T11:18:25.071520+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:29:18.125380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":174,"num_read_kb":174,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.209206+0000","last_change":"2024-05-10T11:14:25.276333+0000","last_active":"2024-05-10T11:17:00.209206+0000","last_peered":"2024-05-10T11:17:00.209206+0000","last_clean":"2024-05-10T11:17:00.209206+0000","last_became_active":"2024-05-10T11:14:25.275818+0000","last_became_peered":"2024-05-10T11:14:25.275818+0000","last_unstale":"2024-05-10T11:17:00.209206+0000","last_undegraded":"2024-05-10T11:17:00.209206+0000","last_fullsized":"2024-05-10T11:17:00.209206+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:27:19.542645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.422145+0000","last_change":"2024-05-10T11:14:35.328589+0000","last_active":"2024-05-10T11:15:01.422145+0000","last_peered":"2024-05-10T11:15:01.422145+0000","last_clean":"2024-05-10T11:15:01.422145+0000","last_became_active":"2024-05-10T11:14:35.328052+0000","last_became_peered":"2024-05-10T11:14:35.328052+0000","last_unstale":"2024-05-10T11:15:01.422145+0000","last_undegraded":"2024-05-10T11:15:01.422145+0000","last_fullsized":"2024-05-10T11:15:01.422145+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:09:42.763932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:28.790098+0000","last_change":"2024-05-10T11:14:22.257872+0000","last_active":"2024-05-10T11:15:28.790098+0000","last_peered":"2024-05-10T11:15:28.790098+0000","last_clean":"2024-05-10T11:15:28.790098+0000","last_became_active":"2024-05-10T11:14:22.257530+0000","last_became_peered":"2024-05-10T11:14:22.257530+0000","last_unstale":"2024-05-10T11:15:28.790098+0000","last_undegraded":"2024-05-10T11:15:28.790098+0000","last_fullsized":"2024-05-10T11:15:28.790098+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:26:30.367209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.214274+0000","last_change":"2024-05-10T11:14:25.269436+0000","last_active":"2024-05-10T11:17:00.214274+0000","last_peered":"2024-05-10T11:17:00.214274+0000","last_clean":"2024-05-10T11:17:00.214274+0000","last_became_active":"2024-05-10T11:14:25.269102+0000","last_became_peered":"2024-05-10T11:14:25.269102+0000","last_unstale":"2024-05-10T11:17:00.214274+0000","last_undegraded":"2024-05-10T11:17:00.214274+0000","last_fullsized":"2024-05-10T11:17:00.214274+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:51:50.172464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459510+0000","last_change":"2024-05-10T11:14:35.330272+0000","last_active":"2024-05-10T11:15:01.459510+0000","last_peered":"2024-05-10T11:15:01.459510+0000","last_clean":"2024-05-10T11:15:01.459510+0000","last_became_active":"2024-05-10T11:14:35.329438+0000","last_became_peered":"2024-05-10T11:14:35.329438+0000","last_unstale":"2024-05-10T11:15:01.459510+0000","last_undegraded":"2024-05-10T11:15:01.459510+0000","last_fullsized":"2024-05-10T11:15:01.459510+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:07:00.559447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:32.813046+0000","last_change":"2024-05-10T11:14:22.299302+0000","last_active":"2024-05-10T11:15:32.813046+0000","last_peered":"2024-05-10T11:15:32.813046+0000","last_clean":"2024-05-10T11:15:32.813046+0000","last_became_active":"2024-05-10T11:14:22.298792+0000","last_became_peered":"2024-05-10T11:14:22.298792+0000","last_unstale":"2024-05-10T11:15:32.813046+0000","last_undegraded":"2024-05-10T11:15:32.813046+0000","last_fullsized":"2024-05-10T11:15:32.813046+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:12:35.280135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.200955+0000","last_change":"2024-05-10T11:14:25.275380+0000","last_active":"2024-05-10T11:17:00.200955+0000","last_peered":"2024-05-10T11:17:00.200955+0000","last_clean":"2024-05-10T11:17:00.200955+0000","last_became_active":"2024-05-10T11:14:25.275042+0000","last_became_peered":"2024-05-10T11:14:25.275042+0000","last_unstale":"2024-05-10T11:17:00.200955+0000","last_undegraded":"2024-05-10T11:17:00.200955+0000","last_fullsized":"2024-05-10T11:17:00.200955+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:22:49.598602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.422374+0000","last_change":"2024-05-10T11:14:35.327505+0000","last_active":"2024-05-10T11:15:01.422374+0000","last_peered":"2024-05-10T11:15:01.422374+0000","last_clean":"2024-05-10T11:15:01.422374+0000","last_became_active":"2024-05-10T11:14:35.326885+0000","last_became_peered":"2024-05-10T11:14:35.326885+0000","last_unstale":"2024-05-10T11:15:01.422374+0000","last_undegraded":"2024-05-10T11:15:01.422374+0000","last_fullsized":"2024-05-10T11:15:01.422374+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:23:09.010545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'40","reported_seq":440,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:18:26.073755+0000","last_change":"2024-05-10T11:14:22.257571+0000","last_active":"2024-05-10T11:18:26.073755+0000","last_peered":"2024-05-10T11:18:26.073755+0000","last_clean":"2024-05-10T11:18:26.073755+0000","last_became_active":"2024-05-10T11:14:22.257156+0000","last_became_peered":"2024-05-10T11:14:22.257156+0000","last_unstale":"2024-05-10T11:18:26.073755+0000","last_undegraded":"2024-05-10T11:18:26.073755+0000","last_fullsized":"2024-05-10T11:18:26.073755+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:28:06.397262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69192,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":360,"num_read_kb":248,"num_write":41,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.193870+0000","last_change":"2024-05-10T11:14:25.281160+0000","last_active":"2024-05-10T11:17:00.193870+0000","last_peered":"2024-05-10T11:17:00.193870+0000","last_clean":"2024-05-10T11:17:00.193870+0000","last_became_active":"2024-05-10T11:14:25.280763+0000","last_became_peered":"2024-05-10T11:14:25.280763+0000","last_unstale":"2024-05-10T11:17:00.193870+0000","last_undegraded":"2024-05-10T11:17:00.193870+0000","last_fullsized":"2024-05-10T11:17:00.193870+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:19:53.350792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.460284+0000","last_change":"2024-05-10T11:14:35.320918+0000","last_active":"2024-05-10T11:15:01.460284+0000","last_peered":"2024-05-10T11:15:01.460284+0000","last_clean":"2024-05-10T11:15:01.460284+0000","last_became_active":"2024-05-10T11:14:35.320412+0000","last_became_peered":"2024-05-10T11:14:35.320412+0000","last_unstale":"2024-05-10T11:15:01.460284+0000","last_undegraded":"2024-05-10T11:15:01.460284+0000","last_fullsized":"2024-05-10T11:15:01.460284+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:55:52.585430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:32.810406+0000","last_change":"2024-05-10T11:14:22.257771+0000","last_active":"2024-05-10T11:15:32.810406+0000","last_peered":"2024-05-10T11:15:32.810406+0000","last_clean":"2024-05-10T11:15:32.810406+0000","last_became_active":"2024-05-10T11:14:22.257430+0000","last_became_peered":"2024-05-10T11:14:22.257430+0000","last_unstale":"2024-05-10T11:15:32.810406+0000","last_undegraded":"2024-05-10T11:15:32.810406+0000","last_fullsized":"2024-05-10T11:15:32.810406+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:23:50.131136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.203875+0000","last_change":"2024-05-10T11:14:25.272276+0000","last_active":"2024-05-10T11:17:00.203875+0000","last_peered":"2024-05-10T11:17:00.203875+0000","last_clean":"2024-05-10T11:17:00.203875+0000","last_became_active":"2024-05-10T11:14:25.271932+0000","last_became_peered":"2024-05-10T11:14:25.271932+0000","last_unstale":"2024-05-10T11:17:00.203875+0000","last_undegraded":"2024-05-10T11:17:00.203875+0000","last_fullsized":"2024-05-10T11:17:00.203875+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:49:44.479056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.463273+0000","last_change":"2024-05-10T11:15:01.462435+0000","last_active":"2024-05-10T11:15:01.463273+0000","last_peered":"2024-05-10T11:15:01.463273+0000","last_clean":"2024-05-10T11:15:01.463273+0000","last_became_active":"2024-05-10T11:15:01.461890+0000","last_became_peered":"2024-05-10T11:15:01.461890+0000","last_unstale":"2024-05-10T11:15:01.463273+0000","last_undegraded":"2024-05-10T11:15:01.463273+0000","last_fullsized":"2024-05-10T11:15:01.463273+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:01:13.350655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:28.789871+0000","last_change":"2024-05-10T11:14:22.256713+0000","last_active":"2024-05-10T11:15:28.789871+0000","last_peered":"2024-05-10T11:15:28.789871+0000","last_clean":"2024-05-10T11:15:28.789871+0000","last_became_active":"2024-05-10T11:14:22.255847+0000","last_became_peered":"2024-05-10T11:14:22.255847+0000","last_unstale":"2024-05-10T11:15:28.789871+0000","last_undegraded":"2024-05-10T11:15:28.789871+0000","last_fullsized":"2024-05-10T11:15:28.789871+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:00:54.605506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.214050+0000","last_change":"2024-05-10T11:14:25.282118+0000","last_active":"2024-05-10T11:17:00.214050+0000","last_peered":"2024-05-10T11:17:00.214050+0000","last_clean":"2024-05-10T11:17:00.214050+0000","last_became_active":"2024-05-10T11:14:25.280958+0000","last_became_peered":"2024-05-10T11:14:25.280958+0000","last_unstale":"2024-05-10T11:17:00.214050+0000","last_undegraded":"2024-05-10T11:17:00.214050+0000","last_fullsized":"2024-05-10T11:17:00.214050+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:02:10.815733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.422298+0000","last_change":"2024-05-10T11:14:35.329876+0000","last_active":"2024-05-10T11:15:01.422298+0000","last_peered":"2024-05-10T11:15:01.422298+0000","last_clean":"2024-05-10T11:15:01.422298+0000","last_became_active":"2024-05-10T11:14:35.328923+0000","last_became_peered":"2024-05-10T11:14:35.328923+0000","last_unstale":"2024-05-10T11:15:01.422298+0000","last_undegraded":"2024-05-10T11:15:01.422298+0000","last_fullsized":"2024-05-10T11:15:01.422298+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:04:51.356552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'6","reported_seq":223,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:18:24.070053+0000","last_change":"2024-05-10T11:14:22.254515+0000","last_active":"2024-05-10T11:18:24.070053+0000","last_peered":"2024-05-10T11:18:24.070053+0000","last_clean":"2024-05-10T11:18:24.070053+0000","last_became_active":"2024-05-10T11:14:22.253648+0000","last_became_peered":"2024-05-10T11:14:22.253648+0000","last_unstale":"2024-05-10T11:18:24.070053+0000","last_undegraded":"2024-05-10T11:18:24.070053+0000","last_fullsized":"2024-05-10T11:18:24.070053+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:10:22.144446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1613,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":179,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.210635+0000","last_change":"2024-05-10T11:14:25.278841+0000","last_active":"2024-05-10T11:17:00.210635+0000","last_peered":"2024-05-10T11:17:00.210635+0000","last_clean":"2024-05-10T11:17:00.210635+0000","last_became_active":"2024-05-10T11:14:25.278527+0000","last_became_peered":"2024-05-10T11:14:25.278527+0000","last_unstale":"2024-05-10T11:17:00.210635+0000","last_undegraded":"2024-05-10T11:17:00.210635+0000","last_fullsized":"2024-05-10T11:17:00.210635+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T23:01:55.902587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459763+0000","last_change":"2024-05-10T11:14:35.331334+0000","last_active":"2024-05-10T11:15:01.459763+0000","last_peered":"2024-05-10T11:15:01.459763+0000","last_clean":"2024-05-10T11:15:01.459763+0000","last_became_active":"2024-05-10T11:14:35.330940+0000","last_became_peered":"2024-05-10T11:14:35.330940+0000","last_unstale":"2024-05-10T11:15:01.459763+0000","last_undegraded":"2024-05-10T11:15:01.459763+0000","last_fullsized":"2024-05-10T11:15:01.459763+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:11:12.241551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":56,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:32.810680+0000","last_change":"2024-05-10T11:14:22.299917+0000","last_active":"2024-05-10T11:15:32.810680+0000","last_peered":"2024-05-10T11:15:32.810680+0000","last_clean":"2024-05-10T11:15:32.810680+0000","last_became_active":"2024-05-10T11:14:22.299212+0000","last_became_peered":"2024-05-10T11:14:22.299212+0000","last_unstale":"2024-05-10T11:15:32.810680+0000","last_undegraded":"2024-05-10T11:15:32.810680+0000","last_fullsized":"2024-05-10T11:15:32.810680+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:52:48.740339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.213948+0000","last_change":"2024-05-10T11:14:25.274693+0000","last_active":"2024-05-10T11:17:00.213948+0000","last_peered":"2024-05-10T11:17:00.213948+0000","last_clean":"2024-05-10T11:17:00.213948+0000","last_became_active":"2024-05-10T11:14:25.274378+0000","last_became_peered":"2024-05-10T11:14:25.274378+0000","last_unstale":"2024-05-10T11:17:00.213948+0000","last_undegraded":"2024-05-10T11:17:00.213948+0000","last_fullsized":"2024-05-10T11:17:00.213948+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:53:24.082742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:02.989636+0000","last_change":"2024-05-10T11:15:01.460541+0000","last_active":"2024-05-10T11:15:02.989636+0000","last_peered":"2024-05-10T11:15:02.989636+0000","last_clean":"2024-05-10T11:15:02.989636+0000","last_became_active":"2024-05-10T11:15:01.459954+0000","last_became_peered":"2024-05-10T11:15:01.459954+0000","last_unstale":"2024-05-10T11:15:02.989636+0000","last_undegraded":"2024-05-10T11:15:02.989636+0000","last_fullsized":"2024-05-10T11:15:02.989636+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:25:43.642441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,3,1],"acting":[8,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:32.810495+0000","last_change":"2024-05-10T11:14:22.258797+0000","last_active":"2024-05-10T11:15:32.810495+0000","last_peered":"2024-05-10T11:15:32.810495+0000","last_clean":"2024-05-10T11:15:32.810495+0000","last_became_active":"2024-05-10T11:14:22.258365+0000","last_became_peered":"2024-05-10T11:14:22.258365+0000","last_unstale":"2024-05-10T11:15:32.810495+0000","last_undegraded":"2024-05-10T11:15:32.810495+0000","last_fullsized":"2024-05-10T11:15:32.810495+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:58:53.291644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.198793+0000","last_change":"2024-05-10T11:15:01.460502+0000","last_active":"2024-05-10T11:17:00.198793+0000","last_peered":"2024-05-10T11:17:00.198793+0000","last_clean":"2024-05-10T11:17:00.198793+0000","last_became_active":"2024-05-10T11:15:01.460101+0000","last_became_peered":"2024-05-10T11:15:01.460101+0000","last_unstale":"2024-05-10T11:17:00.198793+0000","last_undegraded":"2024-05-10T11:17:00.198793+0000","last_fullsized":"2024-05-10T11:17:00.198793+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:22:39.735251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.429220+0000","last_change":"2024-05-10T11:14:35.419037+0000","last_active":"2024-05-10T11:15:01.429220+0000","last_peered":"2024-05-10T11:15:01.429220+0000","last_clean":"2024-05-10T11:15:01.429220+0000","last_became_active":"2024-05-10T11:14:35.418541+0000","last_became_peered":"2024-05-10T11:14:35.418541+0000","last_unstale":"2024-05-10T11:15:01.429220+0000","last_undegraded":"2024-05-10T11:15:01.429220+0000","last_fullsized":"2024-05-10T11:15:01.429220+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:05:49.560608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.422544+0000","last_change":"2024-05-10T11:14:22.299538+0000","last_active":"2024-05-10T11:15:01.422544+0000","last_peered":"2024-05-10T11:15:01.422544+0000","last_clean":"2024-05-10T11:15:01.422544+0000","last_became_active":"2024-05-10T11:14:22.299192+0000","last_became_peered":"2024-05-10T11:14:22.299192+0000","last_unstale":"2024-05-10T11:15:01.422544+0000","last_undegraded":"2024-05-10T11:15:01.422544+0000","last_fullsized":"2024-05-10T11:15:01.422544+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:28:43.041513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.193550+0000","last_change":"2024-05-10T11:14:25.272238+0000","last_active":"2024-05-10T11:17:00.193550+0000","last_peered":"2024-05-10T11:17:00.193550+0000","last_clean":"2024-05-10T11:17:00.193550+0000","last_became_active":"2024-05-10T11:14:25.271861+0000","last_became_peered":"2024-05-10T11:14:25.271861+0000","last_unstale":"2024-05-10T11:17:00.193550+0000","last_undegraded":"2024-05-10T11:17:00.193550+0000","last_fullsized":"2024-05-10T11:17:00.193550+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:00:08.821316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459614+0000","last_change":"2024-05-10T11:14:35.322239+0000","last_active":"2024-05-10T11:15:01.459614+0000","last_peered":"2024-05-10T11:15:01.459614+0000","last_clean":"2024-05-10T11:15:01.459614+0000","last_became_active":"2024-05-10T11:14:35.321864+0000","last_became_peered":"2024-05-10T11:14:35.321864+0000","last_unstale":"2024-05-10T11:15:01.459614+0000","last_undegraded":"2024-05-10T11:15:01.459614+0000","last_fullsized":"2024-05-10T11:15:01.459614+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:42:06.905879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:29.796685+0000","last_change":"2024-05-10T11:14:22.255449+0000","last_active":"2024-05-10T11:15:29.796685+0000","last_peered":"2024-05-10T11:15:29.796685+0000","last_clean":"2024-05-10T11:15:29.796685+0000","last_became_active":"2024-05-10T11:14:22.254966+0000","last_became_peered":"2024-05-10T11:14:22.254966+0000","last_unstale":"2024-05-10T11:15:29.796685+0000","last_undegraded":"2024-05-10T11:15:29.796685+0000","last_fullsized":"2024-05-10T11:15:29.796685+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:55:40.746115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.209423+0000","last_change":"2024-05-10T11:14:25.274050+0000","last_active":"2024-05-10T11:17:00.209423+0000","last_peered":"2024-05-10T11:17:00.209423+0000","last_clean":"2024-05-10T11:17:00.209423+0000","last_became_active":"2024-05-10T11:14:25.273720+0000","last_became_peered":"2024-05-10T11:14:25.273720+0000","last_unstale":"2024-05-10T11:17:00.209423+0000","last_undegraded":"2024-05-10T11:17:00.209423+0000","last_fullsized":"2024-05-10T11:17:00.209423+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:22:05.977673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459638+0000","last_change":"2024-05-10T11:14:35.418989+0000","last_active":"2024-05-10T11:15:01.459638+0000","last_peered":"2024-05-10T11:15:01.459638+0000","last_clean":"2024-05-10T11:15:01.459638+0000","last_became_active":"2024-05-10T11:14:35.418546+0000","last_became_peered":"2024-05-10T11:14:35.418546+0000","last_unstale":"2024-05-10T11:15:01.459638+0000","last_undegraded":"2024-05-10T11:15:01.459638+0000","last_fullsized":"2024-05-10T11:15:01.459638+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:23:37.875409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.213026+0000","last_change":"2024-05-10T11:14:25.272694+0000","last_active":"2024-05-10T11:17:00.213026+0000","last_peered":"2024-05-10T11:17:00.213026+0000","last_clean":"2024-05-10T11:17:00.213026+0000","last_became_active":"2024-05-10T11:14:25.272303+0000","last_became_peered":"2024-05-10T11:14:25.272303+0000","last_unstale":"2024-05-10T11:17:00.213026+0000","last_undegraded":"2024-05-10T11:17:00.213026+0000","last_fullsized":"2024-05-10T11:17:00.213026+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:51:39.039122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'6","reported_seq":219,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:18:25.070826+0000","last_change":"2024-05-10T11:14:22.299359+0000","last_active":"2024-05-10T11:18:25.070826+0000","last_peered":"2024-05-10T11:18:25.070826+0000","last_clean":"2024-05-10T11:18:25.070826+0000","last_became_active":"2024-05-10T11:14:22.299023+0000","last_became_peered":"2024-05-10T11:14:22.299023+0000","last_unstale":"2024-05-10T11:18:25.070826+0000","last_undegraded":"2024-05-10T11:18:25.070826+0000","last_fullsized":"2024-05-10T11:18:25.070826+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:53:06.301774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":174,"num_read_kb":174,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:41.913182+0000","last_change":"2024-05-10T11:14:35.322640+0000","last_active":"2024-05-10T11:15:41.913182+0000","last_peered":"2024-05-10T11:15:41.913182+0000","last_clean":"2024-05-10T11:15:41.913182+0000","last_became_active":"2024-05-10T11:14:35.322226+0000","last_became_peered":"2024-05-10T11:14:35.322226+0000","last_unstale":"2024-05-10T11:15:41.913182+0000","last_undegraded":"2024-05-10T11:15:41.913182+0000","last_fullsized":"2024-05-10T11:15:41.913182+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:34:11.574438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.206730+0000","last_change":"2024-05-10T11:14:25.276645+0000","last_active":"2024-05-10T11:17:00.206730+0000","last_peered":"2024-05-10T11:17:00.206730+0000","last_clean":"2024-05-10T11:17:00.206730+0000","last_became_active":"2024-05-10T11:14:25.276295+0000","last_became_peered":"2024-05-10T11:14:25.276295+0000","last_unstale":"2024-05-10T11:17:00.206730+0000","last_undegraded":"2024-05-10T11:17:00.206730+0000","last_fullsized":"2024-05-10T11:17:00.206730+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:52:06.715073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:32.810748+0000","last_change":"2024-05-10T11:14:22.252908+0000","last_active":"2024-05-10T11:15:32.810748+0000","last_peered":"2024-05-10T11:15:32.810748+0000","last_clean":"2024-05-10T11:15:32.810748+0000","last_became_active":"2024-05-10T11:14:22.252532+0000","last_became_peered":"2024-05-10T11:14:22.252532+0000","last_unstale":"2024-05-10T11:15:32.810748+0000","last_undegraded":"2024-05-10T11:15:32.810748+0000","last_fullsized":"2024-05-10T11:15:32.810748+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:41:59.651442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":114,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:38.138493+0000","last_change":"2024-05-10T11:12:53.090796+0000","last_active":"2024-05-10T11:15:38.138493+0000","last_peered":"2024-05-10T11:15:38.138493+0000","last_clean":"2024-05-10T11:15:38.138493+0000","last_became_active":"2024-05-10T11:12:53.086566+0000","last_became_peered":"2024-05-10T11:12:53.086566+0000","last_unstale":"2024-05-10T11:15:38.138493+0000","last_undegraded":"2024-05-10T11:15:38.138493+0000","last_fullsized":"2024-05-10T11:15:38.138493+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:09:27.097043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:09:27.097043+0000","last_clean_scrub_stamp":"2024-05-10T11:09:27.097043+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:42:17.759719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459957+0000","last_change":"2024-05-10T11:14:35.419970+0000","last_active":"2024-05-10T11:15:01.459957+0000","last_peered":"2024-05-10T11:15:01.459957+0000","last_clean":"2024-05-10T11:15:01.459957+0000","last_became_active":"2024-05-10T11:14:35.419516+0000","last_became_peered":"2024-05-10T11:14:35.419516+0000","last_unstale":"2024-05-10T11:15:01.459957+0000","last_undegraded":"2024-05-10T11:15:01.459957+0000","last_fullsized":"2024-05-10T11:15:01.459957+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:01:25.148155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.186817+0000","last_change":"2024-05-10T11:14:25.271478+0000","last_active":"2024-05-10T11:17:00.186817+0000","last_peered":"2024-05-10T11:17:00.186817+0000","last_clean":"2024-05-10T11:17:00.186817+0000","last_became_active":"2024-05-10T11:14:25.271143+0000","last_became_peered":"2024-05-10T11:14:25.271143+0000","last_unstale":"2024-05-10T11:17:00.186817+0000","last_undegraded":"2024-05-10T11:17:00.186817+0000","last_fullsized":"2024-05-10T11:17:00.186817+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:50:55.771659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:16:23.143145+0000","last_change":"2024-05-10T11:14:22.258704+0000","last_active":"2024-05-10T11:16:23.143145+0000","last_peered":"2024-05-10T11:16:23.143145+0000","last_clean":"2024-05-10T11:16:23.143145+0000","last_became_active":"2024-05-10T11:14:22.258304+0000","last_became_peered":"2024-05-10T11:14:22.258304+0000","last_unstale":"2024-05-10T11:16:23.143145+0000","last_undegraded":"2024-05-10T11:16:23.143145+0000","last_fullsized":"2024-05-10T11:16:23.143145+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:02:41.451394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.425481+0000","last_change":"2024-05-10T11:14:35.325151+0000","last_active":"2024-05-10T11:15:01.425481+0000","last_peered":"2024-05-10T11:15:01.425481+0000","last_clean":"2024-05-10T11:15:01.425481+0000","last_became_active":"2024-05-10T11:14:35.324680+0000","last_became_peered":"2024-05-10T11:14:35.324680+0000","last_unstale":"2024-05-10T11:15:01.425481+0000","last_undegraded":"2024-05-10T11:15:01.425481+0000","last_fullsized":"2024-05-10T11:15:01.425481+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:12:17.578364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:32.810795+0000","last_change":"2024-05-10T11:15:01.428210+0000","last_active":"2024-05-10T11:15:32.810795+0000","last_peered":"2024-05-10T11:15:32.810795+0000","last_clean":"2024-05-10T11:15:32.810795+0000","last_became_active":"2024-05-10T11:15:01.427671+0000","last_became_peered":"2024-05-10T11:15:01.427671+0000","last_unstale":"2024-05-10T11:15:32.810795+0000","last_undegraded":"2024-05-10T11:15:32.810795+0000","last_fullsized":"2024-05-10T11:15:32.810795+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:44:51.312743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.194143+0000","last_change":"2024-05-10T11:14:25.271350+0000","last_active":"2024-05-10T11:17:00.194143+0000","last_peered":"2024-05-10T11:17:00.194143+0000","last_clean":"2024-05-10T11:17:00.194143+0000","last_became_active":"2024-05-10T11:14:25.271047+0000","last_became_peered":"2024-05-10T11:14:25.271047+0000","last_unstale":"2024-05-10T11:17:00.194143+0000","last_undegraded":"2024-05-10T11:17:00.194143+0000","last_fullsized":"2024-05-10T11:17:00.194143+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:44:21.074007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.458379+0000","last_change":"2024-05-10T11:14:35.418910+0000","last_active":"2024-05-10T11:15:01.458379+0000","last_peered":"2024-05-10T11:15:01.458379+0000","last_clean":"2024-05-10T11:15:01.458379+0000","last_became_active":"2024-05-10T11:14:35.418460+0000","last_became_peered":"2024-05-10T11:14:35.418460+0000","last_unstale":"2024-05-10T11:15:01.458379+0000","last_undegraded":"2024-05-10T11:15:01.458379+0000","last_fullsized":"2024-05-10T11:15:01.458379+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:36:04.057776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'13","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.299940+0000","last_change":"2024-05-10T11:14:22.253673+0000","last_active":"2024-05-10T11:17:00.299940+0000","last_peered":"2024-05-10T11:17:00.299940+0000","last_clean":"2024-05-10T11:17:00.299940+0000","last_became_active":"2024-05-10T11:14:22.253304+0000","last_became_peered":"2024-05-10T11:14:22.253304+0000","last_unstale":"2024-05-10T11:17:00.299940+0000","last_undegraded":"2024-05-10T11:17:00.299940+0000","last_fullsized":"2024-05-10T11:17:00.299940+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:11:37.108621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.211961+0000","last_change":"2024-05-10T11:14:25.280590+0000","last_active":"2024-05-10T11:17:00.211961+0000","last_peered":"2024-05-10T11:17:00.211961+0000","last_clean":"2024-05-10T11:17:00.211961+0000","last_became_active":"2024-05-10T11:14:25.279990+0000","last_became_peered":"2024-05-10T11:14:25.279990+0000","last_unstale":"2024-05-10T11:17:00.211961+0000","last_undegraded":"2024-05-10T11:17:00.211961+0000","last_fullsized":"2024-05-10T11:17:00.211961+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:46:55.394767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.424307+0000","last_change":"2024-05-10T11:14:35.320931+0000","last_active":"2024-05-10T11:15:01.424307+0000","last_peered":"2024-05-10T11:15:01.424307+0000","last_clean":"2024-05-10T11:15:01.424307+0000","last_became_active":"2024-05-10T11:14:35.320523+0000","last_became_peered":"2024-05-10T11:14:35.320523+0000","last_unstale":"2024-05-10T11:15:01.424307+0000","last_undegraded":"2024-05-10T11:15:01.424307+0000","last_fullsized":"2024-05-10T11:15:01.424307+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:50:37.533397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:28.788626+0000","last_change":"2024-05-10T11:14:22.255115+0000","last_active":"2024-05-10T11:15:28.788626+0000","last_peered":"2024-05-10T11:15:28.788626+0000","last_clean":"2024-05-10T11:15:28.788626+0000","last_became_active":"2024-05-10T11:14:22.254732+0000","last_became_peered":"2024-05-10T11:14:22.254732+0000","last_unstale":"2024-05-10T11:15:28.788626+0000","last_undegraded":"2024-05-10T11:15:28.788626+0000","last_fullsized":"2024-05-10T11:15:28.788626+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:49:54.807312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.206879+0000","last_change":"2024-05-10T11:14:25.270766+0000","last_active":"2024-05-10T11:17:00.206879+0000","last_peered":"2024-05-10T11:17:00.206879+0000","last_clean":"2024-05-10T11:17:00.206879+0000","last_became_active":"2024-05-10T11:14:25.268958+0000","last_became_peered":"2024-05-10T11:14:25.268958+0000","last_unstale":"2024-05-10T11:17:00.206879+0000","last_undegraded":"2024-05-10T11:17:00.206879+0000","last_fullsized":"2024-05-10T11:17:00.206879+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:21:28.421478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459244+0000","last_change":"2024-05-10T11:14:35.420030+0000","last_active":"2024-05-10T11:15:01.459244+0000","last_peered":"2024-05-10T11:15:01.459244+0000","last_clean":"2024-05-10T11:15:01.459244+0000","last_became_active":"2024-05-10T11:14:35.419475+0000","last_became_peered":"2024-05-10T11:14:35.419475+0000","last_unstale":"2024-05-10T11:15:01.459244+0000","last_undegraded":"2024-05-10T11:15:01.459244+0000","last_fullsized":"2024-05-10T11:15:01.459244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:13:51.207146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:28.789346+0000","last_change":"2024-05-10T11:15:01.429683+0000","last_active":"2024-05-10T11:15:28.789346+0000","last_peered":"2024-05-10T11:15:28.789346+0000","last_clean":"2024-05-10T11:15:28.789346+0000","last_became_active":"2024-05-10T11:15:01.429162+0000","last_became_peered":"2024-05-10T11:15:01.429162+0000","last_unstale":"2024-05-10T11:15:28.789346+0000","last_undegraded":"2024-05-10T11:15:28.789346+0000","last_fullsized":"2024-05-10T11:15:28.789346+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:57:35.166686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.167290+0000","last_change":"2024-05-10T11:14:25.282381+0000","last_active":"2024-05-10T11:17:00.167290+0000","last_peered":"2024-05-10T11:17:00.167290+0000","last_clean":"2024-05-10T11:17:00.167290+0000","last_became_active":"2024-05-10T11:14:25.281990+0000","last_became_peered":"2024-05-10T11:14:25.281990+0000","last_unstale":"2024-05-10T11:17:00.167290+0000","last_undegraded":"2024-05-10T11:17:00.167290+0000","last_fullsized":"2024-05-10T11:17:00.167290+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:39:03.726547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.422393+0000","last_change":"2024-05-10T11:14:35.328242+0000","last_active":"2024-05-10T11:15:01.422393+0000","last_peered":"2024-05-10T11:15:01.422393+0000","last_clean":"2024-05-10T11:15:01.422393+0000","last_became_active":"2024-05-10T11:14:35.327877+0000","last_became_peered":"2024-05-10T11:14:35.327877+0000","last_unstale":"2024-05-10T11:15:01.422393+0000","last_undegraded":"2024-05-10T11:15:01.422393+0000","last_fullsized":"2024-05-10T11:15:01.422393+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:01:57.826334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:28.789342+0000","last_change":"2024-05-10T11:14:22.251057+0000","last_active":"2024-05-10T11:15:28.789342+0000","last_peered":"2024-05-10T11:15:28.789342+0000","last_clean":"2024-05-10T11:15:28.789342+0000","last_became_active":"2024-05-10T11:14:22.250696+0000","last_became_peered":"2024-05-10T11:14:22.250696+0000","last_unstale":"2024-05-10T11:15:28.789342+0000","last_undegraded":"2024-05-10T11:15:28.789342+0000","last_fullsized":"2024-05-10T11:15:28.789342+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:38:53.780643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.206311+0000","last_change":"2024-05-10T11:14:25.283806+0000","last_active":"2024-05-10T11:17:00.206311+0000","last_peered":"2024-05-10T11:17:00.206311+0000","last_clean":"2024-05-10T11:17:00.206311+0000","last_became_active":"2024-05-10T11:14:25.282714+0000","last_became_peered":"2024-05-10T11:14:25.282714+0000","last_unstale":"2024-05-10T11:17:00.206311+0000","last_undegraded":"2024-05-10T11:17:00.206311+0000","last_fullsized":"2024-05-10T11:17:00.206311+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:05:11.940408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:48.003154+0000","last_change":"2024-05-10T11:14:35.329986+0000","last_active":"2024-05-10T11:15:48.003154+0000","last_peered":"2024-05-10T11:15:48.003154+0000","last_clean":"2024-05-10T11:15:48.003154+0000","last_became_active":"2024-05-10T11:14:35.329477+0000","last_became_peered":"2024-05-10T11:14:35.329477+0000","last_unstale":"2024-05-10T11:15:48.003154+0000","last_undegraded":"2024-05-10T11:15:48.003154+0000","last_fullsized":"2024-05-10T11:15:48.003154+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:03:19.203580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'2","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:16:35.093919+0000","last_change":"2024-05-10T11:14:22.257627+0000","last_active":"2024-05-10T11:16:35.093919+0000","last_peered":"2024-05-10T11:16:35.093919+0000","last_clean":"2024-05-10T11:16:35.093919+0000","last_became_active":"2024-05-10T11:14:22.256990+0000","last_became_peered":"2024-05-10T11:14:22.256990+0000","last_unstale":"2024-05-10T11:16:35.093919+0000","last_undegraded":"2024-05-10T11:16:35.093919+0000","last_fullsized":"2024-05-10T11:16:35.093919+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:28:18.926821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.203131+0000","last_change":"2024-05-10T11:14:25.274799+0000","last_active":"2024-05-10T11:17:00.203131+0000","last_peered":"2024-05-10T11:17:00.203131+0000","last_clean":"2024-05-10T11:17:00.203131+0000","last_became_active":"2024-05-10T11:14:25.274443+0000","last_became_peered":"2024-05-10T11:14:25.274443+0000","last_unstale":"2024-05-10T11:17:00.203131+0000","last_undegraded":"2024-05-10T11:17:00.203131+0000","last_fullsized":"2024-05-10T11:17:00.203131+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:25:23.399798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459376+0000","last_change":"2024-05-10T11:14:35.418581+0000","last_active":"2024-05-10T11:15:01.459376+0000","last_peered":"2024-05-10T11:15:01.459376+0000","last_clean":"2024-05-10T11:15:01.459376+0000","last_became_active":"2024-05-10T11:14:35.418160+0000","last_became_peered":"2024-05-10T11:14:35.418160+0000","last_unstale":"2024-05-10T11:15:01.459376+0000","last_undegraded":"2024-05-10T11:15:01.459376+0000","last_fullsized":"2024-05-10T11:15:01.459376+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:39:01.873626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:32.810420+0000","last_change":"2024-05-10T11:14:22.260325+0000","last_active":"2024-05-10T11:15:32.810420+0000","last_peered":"2024-05-10T11:15:32.810420+0000","last_clean":"2024-05-10T11:15:32.810420+0000","last_became_active":"2024-05-10T11:14:22.259971+0000","last_became_peered":"2024-05-10T11:14:22.259971+0000","last_unstale":"2024-05-10T11:15:32.810420+0000","last_undegraded":"2024-05-10T11:15:32.810420+0000","last_fullsized":"2024-05-10T11:15:32.810420+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:34:55.547811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.214078+0000","last_change":"2024-05-10T11:14:25.279009+0000","last_active":"2024-05-10T11:17:00.214078+0000","last_peered":"2024-05-10T11:17:00.214078+0000","last_clean":"2024-05-10T11:17:00.214078+0000","last_became_active":"2024-05-10T11:14:25.278443+0000","last_became_peered":"2024-05-10T11:14:25.278443+0000","last_unstale":"2024-05-10T11:17:00.214078+0000","last_undegraded":"2024-05-10T11:17:00.214078+0000","last_fullsized":"2024-05-10T11:17:00.214078+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:25:22.306555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.426518+0000","last_change":"2024-05-10T11:14:35.319405+0000","last_active":"2024-05-10T11:15:01.426518+0000","last_peered":"2024-05-10T11:15:01.426518+0000","last_clean":"2024-05-10T11:15:01.426518+0000","last_became_active":"2024-05-10T11:14:35.318865+0000","last_became_peered":"2024-05-10T11:14:35.318865+0000","last_unstale":"2024-05-10T11:15:01.426518+0000","last_undegraded":"2024-05-10T11:15:01.426518+0000","last_fullsized":"2024-05-10T11:15:01.426518+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:22:41.592141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:32.813833+0000","last_change":"2024-05-10T11:14:22.299243+0000","last_active":"2024-05-10T11:15:32.813833+0000","last_peered":"2024-05-10T11:15:32.813833+0000","last_clean":"2024-05-10T11:15:32.813833+0000","last_became_active":"2024-05-10T11:14:22.298868+0000","last_became_peered":"2024-05-10T11:14:22.298868+0000","last_unstale":"2024-05-10T11:15:32.813833+0000","last_undegraded":"2024-05-10T11:15:32.813833+0000","last_fullsized":"2024-05-10T11:15:32.813833+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:54:15.078141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.213861+0000","last_change":"2024-05-10T11:14:25.281198+0000","last_active":"2024-05-10T11:17:00.213861+0000","last_peered":"2024-05-10T11:17:00.213861+0000","last_clean":"2024-05-10T11:17:00.213861+0000","last_became_active":"2024-05-10T11:14:25.280871+0000","last_became_peered":"2024-05-10T11:14:25.280871+0000","last_unstale":"2024-05-10T11:17:00.213861+0000","last_undegraded":"2024-05-10T11:17:00.213861+0000","last_fullsized":"2024-05-10T11:17:00.213861+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:24:55.843054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.425771+0000","last_change":"2024-05-10T11:14:35.329517+0000","last_active":"2024-05-10T11:15:01.425771+0000","last_peered":"2024-05-10T11:15:01.425771+0000","last_clean":"2024-05-10T11:15:01.425771+0000","last_became_active":"2024-05-10T11:14:35.329056+0000","last_became_peered":"2024-05-10T11:14:35.329056+0000","last_unstale":"2024-05-10T11:15:01.425771+0000","last_undegraded":"2024-05-10T11:15:01.425771+0000","last_fullsized":"2024-05-10T11:15:01.425771+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:09:24.844256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:32.810561+0000","last_change":"2024-05-10T11:14:22.257206+0000","last_active":"2024-05-10T11:15:32.810561+0000","last_peered":"2024-05-10T11:15:32.810561+0000","last_clean":"2024-05-10T11:15:32.810561+0000","last_became_active":"2024-05-10T11:14:22.256850+0000","last_became_peered":"2024-05-10T11:14:22.256850+0000","last_unstale":"2024-05-10T11:15:32.810561+0000","last_undegraded":"2024-05-10T11:15:32.810561+0000","last_fullsized":"2024-05-10T11:15:32.810561+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:34:31.973034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.211163+0000","last_change":"2024-05-10T11:15:01.461372+0000","last_active":"2024-05-10T11:17:00.211163+0000","last_peered":"2024-05-10T11:17:00.211163+0000","last_clean":"2024-05-10T11:17:00.211163+0000","last_became_active":"2024-05-10T11:15:01.460978+0000","last_became_peered":"2024-05-10T11:15:01.460978+0000","last_unstale":"2024-05-10T11:17:00.211163+0000","last_undegraded":"2024-05-10T11:17:00.211163+0000","last_fullsized":"2024-05-10T11:17:00.211163+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:18:45.750486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459061+0000","last_change":"2024-05-10T11:14:35.322387+0000","last_active":"2024-05-10T11:15:01.459061+0000","last_peered":"2024-05-10T11:15:01.459061+0000","last_clean":"2024-05-10T11:15:01.459061+0000","last_became_active":"2024-05-10T11:14:35.321939+0000","last_became_peered":"2024-05-10T11:14:35.321939+0000","last_unstale":"2024-05-10T11:15:01.459061+0000","last_undegraded":"2024-05-10T11:15:01.459061+0000","last_fullsized":"2024-05-10T11:15:01.459061+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:34.302269+0000","last_clean_scrub_stamp":"2024-05-10T11:14:34.302269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:28:29.085363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:15:01.459652+0000","last_change":"2024-05-10T11:14:22.258603+0000","last_active":"2024-05-10T11:15:01.459652+0000","last_peered":"2024-05-10T11:15:01.459652+0000","last_clean":"2024-05-10T11:15:01.459652+0000","last_became_active":"2024-05-10T11:14:22.258269+0000","last_became_peered":"2024-05-10T11:14:22.258269+0000","last_unstale":"2024-05-10T11:15:01.459652+0000","last_undegraded":"2024-05-10T11:15:01.459652+0000","last_fullsized":"2024-05-10T11:15:01.459652+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:21.230649+0000","last_clean_scrub_stamp":"2024-05-10T11:14:21.230649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:30:04.240962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.207686+0000","last_change":"2024-05-10T11:14:25.281959+0000","last_active":"2024-05-10T11:17:00.207686+0000","last_peered":"2024-05-10T11:17:00.207686+0000","last_clean":"2024-05-10T11:17:00.207686+0000","last_became_active":"2024-05-10T11:14:25.281648+0000","last_became_peered":"2024-05-10T11:14:25.281648+0000","last_unstale":"2024-05-10T11:17:00.207686+0000","last_undegraded":"2024-05-10T11:17:00.207686+0000","last_fullsized":"2024-05-10T11:17:00.207686+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:24:01.569685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.195950+0000","last_change":"2024-05-10T11:14:25.284892+0000","last_active":"2024-05-10T11:17:00.195950+0000","last_peered":"2024-05-10T11:17:00.195950+0000","last_clean":"2024-05-10T11:17:00.195950+0000","last_became_active":"2024-05-10T11:14:25.284078+0000","last_became_peered":"2024-05-10T11:14:25.284078+0000","last_unstale":"2024-05-10T11:17:00.195950+0000","last_undegraded":"2024-05-10T11:17:00.195950+0000","last_fullsized":"2024-05-10T11:17:00.195950+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:49:43.348463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.211265+0000","last_change":"2024-05-10T11:14:25.274739+0000","last_active":"2024-05-10T11:17:00.211265+0000","last_peered":"2024-05-10T11:17:00.211265+0000","last_clean":"2024-05-10T11:17:00.211265+0000","last_became_active":"2024-05-10T11:14:25.274310+0000","last_became_peered":"2024-05-10T11:14:25.274310+0000","last_unstale":"2024-05-10T11:17:00.211265+0000","last_undegraded":"2024-05-10T11:17:00.211265+0000","last_fullsized":"2024-05-10T11:17:00.211265+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:21:16.935189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.204984+0000","last_change":"2024-05-10T11:14:25.272484+0000","last_active":"2024-05-10T11:17:00.204984+0000","last_peered":"2024-05-10T11:17:00.204984+0000","last_clean":"2024-05-10T11:17:00.204984+0000","last_became_active":"2024-05-10T11:14:25.272114+0000","last_became_peered":"2024-05-10T11:14:25.272114+0000","last_unstale":"2024-05-10T11:17:00.204984+0000","last_undegraded":"2024-05-10T11:17:00.204984+0000","last_fullsized":"2024-05-10T11:17:00.204984+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:46:34.537959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.208365+0000","last_change":"2024-05-10T11:15:01.460763+0000","last_active":"2024-05-10T11:17:00.208365+0000","last_peered":"2024-05-10T11:17:00.208365+0000","last_clean":"2024-05-10T11:17:00.208365+0000","last_became_active":"2024-05-10T11:15:01.460392+0000","last_became_peered":"2024-05-10T11:15:01.460392+0000","last_unstale":"2024-05-10T11:17:00.208365+0000","last_undegraded":"2024-05-10T11:17:00.208365+0000","last_fullsized":"2024-05-10T11:17:00.208365+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:15:42.859813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.202714+0000","last_change":"2024-05-10T11:14:25.269640+0000","last_active":"2024-05-10T11:17:00.202714+0000","last_peered":"2024-05-10T11:17:00.202714+0000","last_clean":"2024-05-10T11:17:00.202714+0000","last_became_active":"2024-05-10T11:14:25.269348+0000","last_became_peered":"2024-05-10T11:14:25.269348+0000","last_unstale":"2024-05-10T11:17:00.202714+0000","last_undegraded":"2024-05-10T11:17:00.202714+0000","last_fullsized":"2024-05-10T11:17:00.202714+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:40:25.276433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.194352+0000","last_change":"2024-05-10T11:14:25.282563+0000","last_active":"2024-05-10T11:17:00.194352+0000","last_peered":"2024-05-10T11:17:00.194352+0000","last_clean":"2024-05-10T11:17:00.194352+0000","last_became_active":"2024-05-10T11:14:25.282145+0000","last_became_peered":"2024-05-10T11:14:25.282145+0000","last_unstale":"2024-05-10T11:17:00.194352+0000","last_undegraded":"2024-05-10T11:17:00.194352+0000","last_fullsized":"2024-05-10T11:17:00.194352+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:54:38.436669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.202523+0000","last_change":"2024-05-10T11:14:25.276753+0000","last_active":"2024-05-10T11:17:00.202523+0000","last_peered":"2024-05-10T11:17:00.202523+0000","last_clean":"2024-05-10T11:17:00.202523+0000","last_became_active":"2024-05-10T11:14:25.276342+0000","last_became_peered":"2024-05-10T11:14:25.276342+0000","last_unstale":"2024-05-10T11:17:00.202523+0000","last_undegraded":"2024-05-10T11:17:00.202523+0000","last_fullsized":"2024-05-10T11:17:00.202523+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:31:42.766069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.209177+0000","last_change":"2024-05-10T11:14:25.275905+0000","last_active":"2024-05-10T11:17:00.209177+0000","last_peered":"2024-05-10T11:17:00.209177+0000","last_clean":"2024-05-10T11:17:00.209177+0000","last_became_active":"2024-05-10T11:14:25.275422+0000","last_became_peered":"2024-05-10T11:14:25.275422+0000","last_unstale":"2024-05-10T11:17:00.209177+0000","last_undegraded":"2024-05-10T11:17:00.209177+0000","last_fullsized":"2024-05-10T11:17:00.209177+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:39:24.033519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.205191+0000","last_change":"2024-05-10T11:14:25.271602+0000","last_active":"2024-05-10T11:17:00.205191+0000","last_peered":"2024-05-10T11:17:00.205191+0000","last_clean":"2024-05-10T11:17:00.205191+0000","last_became_active":"2024-05-10T11:14:25.267206+0000","last_became_peered":"2024-05-10T11:14:25.267206+0000","last_unstale":"2024-05-10T11:17:00.205191+0000","last_undegraded":"2024-05-10T11:17:00.205191+0000","last_fullsized":"2024-05-10T11:17:00.205191+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:28:24.708297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.206462+0000","last_change":"2024-05-10T11:14:25.280378+0000","last_active":"2024-05-10T11:17:00.206462+0000","last_peered":"2024-05-10T11:17:00.206462+0000","last_clean":"2024-05-10T11:17:00.206462+0000","last_became_active":"2024-05-10T11:14:25.280079+0000","last_became_peered":"2024-05-10T11:14:25.280079+0000","last_unstale":"2024-05-10T11:17:00.206462+0000","last_undegraded":"2024-05-10T11:17:00.206462+0000","last_fullsized":"2024-05-10T11:17:00.206462+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:13:47.781060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.204726+0000","last_change":"2024-05-10T11:14:25.272363+0000","last_active":"2024-05-10T11:17:00.204726+0000","last_peered":"2024-05-10T11:17:00.204726+0000","last_clean":"2024-05-10T11:17:00.204726+0000","last_became_active":"2024-05-10T11:14:25.271847+0000","last_became_peered":"2024-05-10T11:14:25.271847+0000","last_unstale":"2024-05-10T11:17:00.204726+0000","last_undegraded":"2024-05-10T11:17:00.204726+0000","last_fullsized":"2024-05-10T11:17:00.204726+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:47:54.661069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.203714+0000","last_change":"2024-05-10T11:14:25.272688+0000","last_active":"2024-05-10T11:17:00.203714+0000","last_peered":"2024-05-10T11:17:00.203714+0000","last_clean":"2024-05-10T11:17:00.203714+0000","last_became_active":"2024-05-10T11:14:25.271654+0000","last_became_peered":"2024-05-10T11:14:25.271654+0000","last_unstale":"2024-05-10T11:17:00.203714+0000","last_undegraded":"2024-05-10T11:17:00.203714+0000","last_fullsized":"2024-05-10T11:17:00.203714+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:50:21.149707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T11:17:00.199436+0000","last_change":"2024-05-10T11:15:01.432279+0000","last_active":"2024-05-10T11:17:00.199436+0000","last_peered":"2024-05-10T11:17:00.199436+0000","last_clean":"2024-05-10T11:17:00.199436+0000","last_became_active":"2024-05-10T11:15:01.431751+0000","last_became_peered":"2024-05-10T11:15:01.431751+0000","last_unstale":"2024-05-10T11:17:00.199436+0000","last_undegraded":"2024-05-10T11:17:00.199436+0000","last_fullsized":"2024-05-10T11:17:00.199436+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T11:14:24.248337+0000","last_clean_scrub_stamp":"2024-05-10T11:14:24.248337+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:25:44.196618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74852,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1077,"num_read_kb":620,"num_write":190,"num_write_kb":140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":224556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":162,"ondisk_log_size":162,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645377,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1015808,"data_stored":505095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841606,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":41164,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93703988,"statfs":{"total":95995035648,"available":95952883712,"internally_reserved":0,"allocated":1708032,"data_stored":1162558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":40434113},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005132,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1032192,"data_stored":509405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201361,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1024000,"data_stored":505207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430295,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28600,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716552,"statfs":{"total":95995035648,"available":95965749248,"internally_reserved":0,"allocated":1622016,"data_stored":1102797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593820,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":500485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574964,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1081344,"data_stored":569587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738490,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1032192,"data_stored":501873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411440,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36416,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708736,"statfs":{"total":95995035648,"available":95957745664,"internally_reserved":0,"allocated":1105920,"data_stored":571962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149802,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1019904,"data_stored":500679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953574,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1019904,"data_stored":506160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790048,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28608,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716544,"statfs":{"total":95995035648,"available":95965741056,"internally_reserved":0,"allocated":1630208,"data_stored":1097360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":69192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":71567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":71795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T11:18:29.319 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T11:18:29.594 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:18:29.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:18:29 smithi100 ceph-mon[30296]: pgmap v423: 129 pgs: 129 active+clean; 650 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-05-10T11:18:29.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:18:29 smithi100 ceph-mon[30296]: from='client.15255 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T11:18:29.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:18:29 smithi100 ceph-mon[30296]: from='client.15261 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T11:18:30.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:18:29 smithi178 ceph-mon[33359]: pgmap v423: 129 pgs: 129 active+clean; 650 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-05-10T11:18:30.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:18:29 smithi178 ceph-mon[33359]: from='client.15255 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T11:18:30.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:18:29 smithi178 ceph-mon[33359]: from='client.15261 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T11:18:30.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:18:29 smithi188 ceph-mon[34075]: pgmap v423: 129 pgs: 129 active+clean; 650 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-05-10T11:18:30.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:18:29 smithi188 ceph-mon[34075]: from='client.15255 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T11:18:30.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:18:29 smithi188 ceph-mon[34075]: from='client.15261 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T11:18:30.877 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:18:30 smithi100 ceph-mon[30296]: from='client.15267 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T11:18:31.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:18:30 smithi178 ceph-mon[33359]: from='client.15267 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T11:18:31.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:18:30 smithi188 ceph-mon[34075]: from='client.15267 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T11:18:31.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:31.285+0000 7f918559f700 1 -- 172.21.15.100:0/3960997579 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f918006ba10 msgr2=0x7f918006be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:31.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:31.285+0000 7f918559f700 1 --2- 172.21.15.100:0/3960997579 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f918006ba10 0x7f918006be90 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f9174009990 tx=0x7f9174009ca0 comp rx=0 tx=0).stop 2024-05-10T11:18:31.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:31.286+0000 7f918559f700 1 -- 172.21.15.100:0/3960997579 shutdown_connections 2024-05-10T11:18:31.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:31.286+0000 7f918559f700 1 --2- 172.21.15.100:0/3960997579 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f918006c3d0 0x7f9180110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:31.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:31.286+0000 7f918559f700 1 --2- 172.21.15.100:0/3960997579 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9180109500 0x7f91801098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:31.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:31.286+0000 7f918559f700 1 --2- 172.21.15.100:0/3960997579 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f918006ba10 0x7f918006be90 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:31.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:31.286+0000 7f918559f700 1 -- 172.21.15.100:0/3960997579 >> 172.21.15.100:0/3960997579 conn(0x7f9180079310 msgr2=0x7f9180079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:31.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:31.286+0000 7f918559f700 1 -- 172.21.15.100:0/3960997579 shutdown_connections 2024-05-10T11:18:31.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:31.286+0000 7f918559f700 1 -- 172.21.15.100:0/3960997579 wait complete. 2024-05-10T11:18:31.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:31.287+0000 7f918559f700 1 Processor -- start 2024-05-10T11:18:31.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:31.287+0000 7f918559f700 1 -- start start 2024-05-10T11:18:31.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:31.287+0000 7f918559f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f918006ba10 0x7f918019f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:31.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:31.287+0000 7f918559f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f918006c3d0 0x7f918019fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:31.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:31.287+0000 7f918559f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9180109500 0x7f9180199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:31.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:31.287+0000 7f918559f700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f91801135c0 con 0x7f918006ba10 2024-05-10T11:18:31.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:31.287+0000 7f918559f700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9180113440 con 0x7f918006c3d0 2024-05-10T11:18:31.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:31.287+0000 7f918559f700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9180113740 con 0x7f9180109500 2024-05-10T11:18:31.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:31.287+0000 7f917effd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f918006ba10 0x7f918019f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:31.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:31.287+0000 7f917effd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f918006ba10 0x7f918019f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42320/0 (socket says 172.21.15.100:42320) 2024-05-10T11:18:31.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:31.287+0000 7f917effd700 1 -- 172.21.15.100:0/2969606821 learned_addr learned my addr 172.21.15.100:0/2969606821 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:18:31.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:31.287+0000 7f917f7fe700 1 --2- 172.21.15.100:0/2969606821 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9180109500 0x7f9180199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:31.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:31.287+0000 7f917e7fc700 1 --2- 172.21.15.100:0/2969606821 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f918006c3d0 0x7f918019fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:31.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:31.288+0000 7f917effd700 1 -- 172.21.15.100:0/2969606821 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9180109500 msgr2=0x7f9180199a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:31.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:31.288+0000 7f917effd700 1 --2- 172.21.15.100:0/2969606821 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9180109500 0x7f9180199a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:31.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:31.288+0000 7f917effd700 1 -- 172.21.15.100:0/2969606821 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f918006c3d0 msgr2=0x7f918019fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:31.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:31.288+0000 7f917effd700 1 --2- 172.21.15.100:0/2969606821 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f918006c3d0 0x7f918019fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:31.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:31.288+0000 7f917effd700 1 -- 172.21.15.100:0/2969606821 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f918019a160 con 0x7f918006ba10 2024-05-10T11:18:31.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:31.288+0000 7f917effd700 1 --2- 172.21.15.100:0/2969606821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f918006ba10 0x7f918019f840 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f917000b530 tx=0x7f917000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:31.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:31.288+0000 7f9167fff700 1 -- 172.21.15.100:0/2969606821 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9170015020 con 0x7f918006ba10 2024-05-10T11:18:31.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:31.288+0000 7f9167fff700 1 -- 172.21.15.100:0/2969606821 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9170011cc0 con 0x7f918006ba10 2024-05-10T11:18:31.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:31.288+0000 7f918559f700 1 -- 172.21.15.100:0/2969606821 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f918019a450 con 0x7f918006ba10 2024-05-10T11:18:31.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:31.288+0000 7f9167fff700 1 -- 172.21.15.100:0/2969606821 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9170010c80 con 0x7f918006ba10 2024-05-10T11:18:31.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:31.288+0000 7f918559f700 1 -- 172.21.15.100:0/2969606821 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91801a6710 con 0x7f918006ba10 2024-05-10T11:18:31.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:31.289+0000 7f918559f700 1 -- 172.21.15.100:0/2969606821 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f918010b9e0 con 0x7f918006ba10 2024-05-10T11:18:31.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:31.290+0000 7f9167fff700 1 -- 172.21.15.100:0/2969606821 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f9170018020 con 0x7f918006ba10 2024-05-10T11:18:31.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:31.291+0000 7f9167fff700 1 --2- 172.21.15.100:0/2969606821 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f91680b2880 0x7f91680b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:31.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:31.291+0000 7f917e7fc700 1 --2- 172.21.15.100:0/2969606821 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f91680b2880 0x7f91680b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:31.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:31.291+0000 7f9167fff700 1 -- 172.21.15.100:0/2969606821 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f91700e21f0 con 0x7f918006ba10 2024-05-10T11:18:31.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:31.292+0000 7f917e7fc700 1 --2- 172.21.15.100:0/2969606821 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f91680b2880 0x7f91680b4d40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f917400b5c0 tx=0x7f9174005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:31.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:31.299+0000 7f9167fff700 1 -- 172.21.15.100:0/2969606821 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f917001f020 con 0x7f918006ba10 2024-05-10T11:18:31.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:31.558+0000 7f918559f700 1 -- 172.21.15.100:0/2969606821 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f918019af30 con 0x7f918006ba10 2024-05-10T11:18:31.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:31.559+0000 7f9167fff700 1 -- 172.21.15.100:0/2969606821 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11894 (secure 0 0 0) 0x7f91700a9bb0 con 0x7f918006ba10 2024-05-10T11:18:31.559 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:18:31.559 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2072225012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2072225012},{"type":"v1","addr":"172.21.15.100:6835","nonce":2072225012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/401939540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":401939540},{"type":"v1","addr":"172.21.15.100:6837","nonce":401939540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14571,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/3946311025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3946311025},{"type":"v1","addr":"172.21.15.100:6839","nonce":3946311025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3736371660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3736371660},{"type":"v1","addr":"172.21.15.100:6841","nonce":3736371660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/2741810791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2741810791},{"type":"v1","addr":"172.21.15.178:6833","nonce":2741810791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1704891041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1704891041},{"type":"v1","addr":"172.21.15.188:6833","nonce":1704891041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T11:14:27.242452+0000","modified":"2024-05-10T11:18:23.454860+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24439,"mds_1":24437,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/2124076312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2124076312},{"type":"v1","addr":"172.21.15.178:6835","nonce":2124076312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/3224358425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3224358425},{"type":"v1","addr":"172.21.15.178:6837","nonce":3224358425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/515547690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":515547690},{"type":"v1","addr":"172.21.15.188:6835","nonce":515547690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/1943342620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1943342620},{"type":"v1","addr":"172.21.15.188:6837","nonce":1943342620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.178:6839/3760985404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3760985404},{"type":"v1","addr":"172.21.15.178:6839","nonce":3760985404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.188:6839/3194917070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":3194917070},{"type":"v1","addr":"172.21.15.188:6839","nonce":3194917070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24439,"qdb_cluster":[24427,24437,24439]},"id":1}]} 2024-05-10T11:18:31.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:31.562+0000 7f918559f700 1 -- 172.21.15.100:0/2969606821 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f91680b2880 msgr2=0x7f91680b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:31.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:31.562+0000 7f918559f700 1 --2- 172.21.15.100:0/2969606821 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f91680b2880 0x7f91680b4d40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f917400b5c0 tx=0x7f9174005960 comp rx=0 tx=0).stop 2024-05-10T11:18:31.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:31.562+0000 7f918559f700 1 -- 172.21.15.100:0/2969606821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f918006ba10 msgr2=0x7f918019f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:31.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:31.562+0000 7f918559f700 1 --2- 172.21.15.100:0/2969606821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f918006ba10 0x7f918019f840 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f917000b530 tx=0x7f917000b840 comp rx=0 tx=0).stop 2024-05-10T11:18:31.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:31.562+0000 7f918559f700 1 -- 172.21.15.100:0/2969606821 shutdown_connections 2024-05-10T11:18:31.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:31.562+0000 7f918559f700 1 --2- 172.21.15.100:0/2969606821 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f91680b2880 0x7f91680b4d40 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:31.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:31.562+0000 7f918559f700 1 --2- 172.21.15.100:0/2969606821 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9180109500 0x7f9180199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:31.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:31.562+0000 7f918559f700 1 --2- 172.21.15.100:0/2969606821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f918006ba10 0x7f918019f840 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:31.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:31.562+0000 7f918559f700 1 --2- 172.21.15.100:0/2969606821 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f918006c3d0 0x7f918019fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:31.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:31.562+0000 7f918559f700 1 -- 172.21.15.100:0/2969606821 >> 172.21.15.100:0/2969606821 conn(0x7f9180079310 msgr2=0x7f91801012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:31.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:31.562+0000 7f918559f700 1 -- 172.21.15.100:0/2969606821 shutdown_connections 2024-05-10T11:18:31.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:31.562+0000 7f918559f700 1 -- 172.21.15.100:0/2969606821 wait complete. 2024-05-10T11:18:31.563 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 27 2024-05-10T11:18:31.750 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:18:31 smithi100 ceph-mon[30296]: pgmap v424: 129 pgs: 129 active+clean; 650 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-05-10T11:18:31.750 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:18:31 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/2969606821' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:18:32.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:18:31 smithi178 ceph-mon[33359]: pgmap v424: 129 pgs: 129 active+clean; 650 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-05-10T11:18:32.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:18:31 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/2969606821' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:18:32.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:18:31 smithi188 ceph-mon[34075]: pgmap v424: 129 pgs: 129 active+clean; 650 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-05-10T11:18:32.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:18:31 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/2969606821' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:18:32.330 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-10T11:18:32.579 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:18:33.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:18:33 smithi100 ceph-mon[30296]: pgmap v425: 129 pgs: 129 active+clean; 650 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-05-10T11:18:34.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:18:33 smithi178 ceph-mon[33359]: pgmap v425: 129 pgs: 129 active+clean; 650 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-05-10T11:18:34.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:18:33 smithi188 ceph-mon[34075]: pgmap v425: 129 pgs: 129 active+clean; 650 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-05-10T11:18:34.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:34.563+0000 7ff9a961e700 1 -- 172.21.15.100:0/3736344136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9a4102690 msgr2=0x7ff9a4102a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:34.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:34.563+0000 7ff9a961e700 1 --2- 172.21.15.100:0/3736344136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9a4102690 0x7ff9a4102a70 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff9940099c0 tx=0x7ff994009cd0 comp rx=0 tx=0).stop 2024-05-10T11:18:34.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:34.563+0000 7ff9a961e700 1 -- 172.21.15.100:0/3736344136 shutdown_connections 2024-05-10T11:18:34.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:34.563+0000 7ff9a961e700 1 --2- 172.21.15.100:0/3736344136 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9a41076e0 0x7ff9a410f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:34.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:34.563+0000 7ff9a961e700 1 --2- 172.21.15.100:0/3736344136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9a4102690 0x7ff9a4102a70 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:34.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:34.563+0000 7ff9a961e700 1 --2- 172.21.15.100:0/3736344136 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9a4103040 0x7ff9a4107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:34.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:34.563+0000 7ff9a961e700 1 -- 172.21.15.100:0/3736344136 >> 172.21.15.100:0/3736344136 conn(0x7ff9a40783f0 msgr2=0x7ff9a4078800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:34.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:34.563+0000 7ff9a961e700 1 -- 172.21.15.100:0/3736344136 shutdown_connections 2024-05-10T11:18:34.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:34.564+0000 7ff9a961e700 1 -- 172.21.15.100:0/3736344136 wait complete. 2024-05-10T11:18:34.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:34.564+0000 7ff9a961e700 1 Processor -- start 2024-05-10T11:18:34.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:34.564+0000 7ff9a961e700 1 -- start start 2024-05-10T11:18:34.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:34.564+0000 7ff9a961e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9a4102690 0x7ff9a419e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:34.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:34.564+0000 7ff9a961e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9a4103040 0x7ff9a419eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:34.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:34.564+0000 7ff9a961e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9a41076e0 0x7ff9a41987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:34.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:34.564+0000 7ff9a961e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a4112450 con 0x7ff9a41076e0 2024-05-10T11:18:34.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:34.565+0000 7ff9a961e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a41122d0 con 0x7ff9a4103040 2024-05-10T11:18:34.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:34.565+0000 7ff9a961e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a41125d0 con 0x7ff9a4102690 2024-05-10T11:18:34.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:34.565+0000 7ff9a37fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9a41076e0 0x7ff9a41987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:34.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:34.565+0000 7ff9a37fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9a41076e0 0x7ff9a41987d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42340/0 (socket says 172.21.15.100:42340) 2024-05-10T11:18:34.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:34.565+0000 7ff9a37fe700 1 -- 172.21.15.100:0/1388432327 learned_addr learned my addr 172.21.15.100:0/1388432327 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:18:34.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:34.565+0000 7ff9a27fc700 1 --2- 172.21.15.100:0/1388432327 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9a4103040 0x7ff9a419eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:34.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:34.565+0000 7ff9a2ffd700 1 --2- 172.21.15.100:0/1388432327 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9a4102690 0x7ff9a419e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:34.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:34.565+0000 7ff9a37fe700 1 -- 172.21.15.100:0/1388432327 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9a4102690 msgr2=0x7ff9a419e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:34.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:34.565+0000 7ff9a37fe700 1 --2- 172.21.15.100:0/1388432327 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9a4102690 0x7ff9a419e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:34.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:34.565+0000 7ff9a37fe700 1 -- 172.21.15.100:0/1388432327 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9a4103040 msgr2=0x7ff9a419eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:34.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:34.565+0000 7ff9a37fe700 1 --2- 172.21.15.100:0/1388432327 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9a4103040 0x7ff9a419eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:34.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:34.565+0000 7ff9a37fe700 1 -- 172.21.15.100:0/1388432327 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9a4198fb0 con 0x7ff9a41076e0 2024-05-10T11:18:34.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:34.565+0000 7ff9a27fc700 1 --2- 172.21.15.100:0/1388432327 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9a4103040 0x7ff9a419eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:18:34.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:34.565+0000 7ff9a37fe700 1 --2- 172.21.15.100:0/1388432327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9a41076e0 0x7ff9a41987d0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff99c007ee0 tx=0x7ff99c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:34.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:34.565+0000 7ff98bfff700 1 -- 172.21.15.100:0/1388432327 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99c00dbd0 con 0x7ff9a41076e0 2024-05-10T11:18:34.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:34.565+0000 7ff9a961e700 1 -- 172.21.15.100:0/1388432327 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9a41992a0 con 0x7ff9a41076e0 2024-05-10T11:18:34.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:34.565+0000 7ff98bfff700 1 -- 172.21.15.100:0/1388432327 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff99c00dd70 con 0x7ff9a41076e0 2024-05-10T11:18:34.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:34.565+0000 7ff98bfff700 1 -- 172.21.15.100:0/1388432327 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99c0058d0 con 0x7ff9a41076e0 2024-05-10T11:18:34.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:34.565+0000 7ff9a961e700 1 -- 172.21.15.100:0/1388432327 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9a406bc10 con 0x7ff9a41076e0 2024-05-10T11:18:34.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:34.566+0000 7ff9a961e700 1 -- 172.21.15.100:0/1388432327 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9a4103ae0 con 0x7ff9a41076e0 2024-05-10T11:18:34.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:34.568+0000 7ff98bfff700 1 -- 172.21.15.100:0/1388432327 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7ff99c01a020 con 0x7ff9a41076e0 2024-05-10T11:18:34.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:34.568+0000 7ff98bfff700 1 --2- 172.21.15.100:0/1388432327 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff98c0b2880 0x7ff98c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:34.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:34.569+0000 7ff9a2ffd700 1 --2- 172.21.15.100:0/1388432327 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff98c0b2880 0x7ff98c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:34.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:34.569+0000 7ff98bfff700 1 -- 172.21.15.100:0/1388432327 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7ff99c0e2300 con 0x7ff9a41076e0 2024-05-10T11:18:34.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:34.569+0000 7ff9a2ffd700 1 --2- 172.21.15.100:0/1388432327 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff98c0b2880 0x7ff98c0b4d40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff994000c00 tx=0x7ff994005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:34.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:34.572+0000 7ff98bfff700 1 -- 172.21.15.100:0/1388432327 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7ff99c010070 con 0x7ff9a41076e0 2024-05-10T11:18:34.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:34.782+0000 7ff9a961e700 1 -- 172.21.15.100:0/1388432327 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff9a406c640 con 0x7ff9a41076e0 2024-05-10T11:18:34.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:34.783+0000 7ff98bfff700 1 -- 172.21.15.100:0/1388432327 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24737 (secure 0 0 0) 0x7ff99c0a9cc0 con 0x7ff9a41076e0 2024-05-10T11:18:34.784 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:18:34.784 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":89,"fsid":"20a542cc-0ebd-11ef-bc98-c7b262605968","created":"2024-05-10T11:06:05.454446+0000","modified":"2024-05-10T11:15:01.418144+0000","last_up_change":"2024-05-10T11:13:18.185471+0000","last_in_change":"2024-05-10T11:12:57.268678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T11:09:26.176441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T11:14:21.004285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T11:14:24.179414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T11:14:33.308517+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5f031418-da23-4e85-8669-160960dadd09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":643028278},{"type":"v1","addr":"172.21.15.100:6803","nonce":643028278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":643028278},{"type":"v1","addr":"172.21.15.100:6805","nonce":643028278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":643028278},{"type":"v1","addr":"172.21.15.100:6809","nonce":643028278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":643028278},{"type":"v1","addr":"172.21.15.100:6807","nonce":643028278}]},"public_addr":"172.21.15.100:6803/643028278","cluster_addr":"172.21.15.100:6805/643028278","heartbeat_back_addr":"172.21.15.100:6809/643028278","heartbeat_front_addr":"172.21.15.100:6807/643028278","state":["exists","up"]},{"osd":1,"uuid":"a908cdc9-001e-4b65-9ab0-0dd08dba20f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":3103987017},{"type":"v1","addr":"172.21.15.178:6801","nonce":3103987017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":3103987017},{"type":"v1","addr":"172.21.15.178:6803","nonce":3103987017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":3103987017},{"type":"v1","addr":"172.21.15.178:6807","nonce":3103987017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":3103987017},{"type":"v1","addr":"172.21.15.178:6805","nonce":3103987017}]},"public_addr":"172.21.15.178:6801/3103987017","cluster_addr":"172.21.15.178:6803/3103987017","heartbeat_back_addr":"172.21.15.178:6807/3103987017","heartbeat_front_addr":"172.21.15.178:6805/3103987017","state":["exists","up"]},{"osd":2,"uuid":"4d8f89ae-9424-46e9-9c80-17c47cc5e8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3991466468},{"type":"v1","addr":"172.21.15.188:6801","nonce":3991466468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3991466468},{"type":"v1","addr":"172.21.15.188:6803","nonce":3991466468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3991466468},{"type":"v1","addr":"172.21.15.188:6807","nonce":3991466468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3991466468},{"type":"v1","addr":"172.21.15.188:6805","nonce":3991466468}]},"public_addr":"172.21.15.188:6801/3991466468","cluster_addr":"172.21.15.188:6803/3991466468","heartbeat_back_addr":"172.21.15.188:6807/3991466468","heartbeat_front_addr":"172.21.15.188:6805/3991466468","state":["exists","up"]},{"osd":3,"uuid":"4c8865aa-fef4-4710-b766-1385adb9462c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1713357192},{"type":"v1","addr":"172.21.15.100:6811","nonce":1713357192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1713357192},{"type":"v1","addr":"172.21.15.100:6813","nonce":1713357192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1713357192},{"type":"v1","addr":"172.21.15.100:6817","nonce":1713357192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1713357192},{"type":"v1","addr":"172.21.15.100:6815","nonce":1713357192}]},"public_addr":"172.21.15.100:6811/1713357192","cluster_addr":"172.21.15.100:6813/1713357192","heartbeat_back_addr":"172.21.15.100:6817/1713357192","heartbeat_front_addr":"172.21.15.100:6815/1713357192","state":["exists","up"]},{"osd":4,"uuid":"a9653800-da57-43eb-8781-3188e8a3949f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":2222814192},{"type":"v1","addr":"172.21.15.178:6809","nonce":2222814192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":2222814192},{"type":"v1","addr":"172.21.15.178:6811","nonce":2222814192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":2222814192},{"type":"v1","addr":"172.21.15.178:6815","nonce":2222814192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":2222814192},{"type":"v1","addr":"172.21.15.178:6813","nonce":2222814192}]},"public_addr":"172.21.15.178:6809/2222814192","cluster_addr":"172.21.15.178:6811/2222814192","heartbeat_back_addr":"172.21.15.178:6815/2222814192","heartbeat_front_addr":"172.21.15.178:6813/2222814192","state":["exists","up"]},{"osd":5,"uuid":"fb5f8af2-1eb0-4d8a-a7a3-ed6dd68e692e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3517375935},{"type":"v1","addr":"172.21.15.188:6809","nonce":3517375935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3517375935},{"type":"v1","addr":"172.21.15.188:6811","nonce":3517375935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3517375935},{"type":"v1","addr":"172.21.15.188:6815","nonce":3517375935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3517375935},{"type":"v1","addr":"172.21.15.188:6813","nonce":3517375935}]},"public_addr":"172.21.15.188:6809/3517375935","cluster_addr":"172.21.15.188:6811/3517375935","heartbeat_back_addr":"172.21.15.188:6815/3517375935","heartbeat_front_addr":"172.21.15.188:6813/3517375935","state":["exists","up"]},{"osd":6,"uuid":"84b57387-0a94-4e49-b099-493d329b87dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1410593862},{"type":"v1","addr":"172.21.15.100:6819","nonce":1410593862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1410593862},{"type":"v1","addr":"172.21.15.100:6821","nonce":1410593862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1410593862},{"type":"v1","addr":"172.21.15.100:6825","nonce":1410593862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1410593862},{"type":"v1","addr":"172.21.15.100:6823","nonce":1410593862}]},"public_addr":"172.21.15.100:6819/1410593862","cluster_addr":"172.21.15.100:6821/1410593862","heartbeat_back_addr":"172.21.15.100:6825/1410593862","heartbeat_front_addr":"172.21.15.100:6823/1410593862","state":["exists","up"]},{"osd":7,"uuid":"02ada28f-ac4f-47cf-983f-8a6da1140abe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2458690455},{"type":"v1","addr":"172.21.15.178:6817","nonce":2458690455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2458690455},{"type":"v1","addr":"172.21.15.178:6819","nonce":2458690455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2458690455},{"type":"v1","addr":"172.21.15.178:6823","nonce":2458690455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2458690455},{"type":"v1","addr":"172.21.15.178:6821","nonce":2458690455}]},"public_addr":"172.21.15.178:6817/2458690455","cluster_addr":"172.21.15.178:6819/2458690455","heartbeat_back_addr":"172.21.15.178:6823/2458690455","heartbeat_front_addr":"172.21.15.178:6821/2458690455","state":["exists","up"]},{"osd":8,"uuid":"e7e489bc-15fc-4a4c-b297-d1bb626513ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1523872826},{"type":"v1","addr":"172.21.15.188:6817","nonce":1523872826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1523872826},{"type":"v1","addr":"172.21.15.188:6819","nonce":1523872826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1523872826},{"type":"v1","addr":"172.21.15.188:6823","nonce":1523872826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1523872826},{"type":"v1","addr":"172.21.15.188:6821","nonce":1523872826}]},"public_addr":"172.21.15.188:6817/1523872826","cluster_addr":"172.21.15.188:6819/1523872826","heartbeat_back_addr":"172.21.15.188:6823/1523872826","heartbeat_front_addr":"172.21.15.188:6821/1523872826","state":["exists","up"]},{"osd":9,"uuid":"72328326-cd16-45d9-bfd9-40f6c5c20f5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3585761978},{"type":"v1","addr":"172.21.15.100:6827","nonce":3585761978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3585761978},{"type":"v1","addr":"172.21.15.100:6829","nonce":3585761978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3585761978},{"type":"v1","addr":"172.21.15.100:6833","nonce":3585761978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3585761978},{"type":"v1","addr":"172.21.15.100:6831","nonce":3585761978}]},"public_addr":"172.21.15.100:6827/3585761978","cluster_addr":"172.21.15.100:6829/3585761978","heartbeat_back_addr":"172.21.15.100:6833/3585761978","heartbeat_front_addr":"172.21.15.100:6831/3585761978","state":["exists","up"]},{"osd":10,"uuid":"a355f9a8-91a3-41d4-814d-5a9c9b953706","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2985841725},{"type":"v1","addr":"172.21.15.178:6825","nonce":2985841725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2985841725},{"type":"v1","addr":"172.21.15.178:6827","nonce":2985841725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2985841725},{"type":"v1","addr":"172.21.15.178:6831","nonce":2985841725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2985841725},{"type":"v1","addr":"172.21.15.178:6829","nonce":2985841725}]},"public_addr":"172.21.15.178:6825/2985841725","cluster_addr":"172.21.15.178:6827/2985841725","heartbeat_back_addr":"172.21.15.178:6831/2985841725","heartbeat_front_addr":"172.21.15.178:6829/2985841725","state":["exists","up"]},{"osd":11,"uuid":"76a5e13c-7e38-42f9-bfa7-7fedec7c7184","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2574955118},{"type":"v1","addr":"172.21.15.188:6825","nonce":2574955118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2574955118},{"type":"v1","addr":"172.21.15.188:6827","nonce":2574955118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2574955118},{"type":"v1","addr":"172.21.15.188:6831","nonce":2574955118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2574955118},{"type":"v1","addr":"172.21.15.188:6829","nonce":2574955118}]},"public_addr":"172.21.15.188:6825/2574955118","cluster_addr":"172.21.15.188:6827/2574955118","heartbeat_back_addr":"172.21.15.188:6831/2574955118","heartbeat_front_addr":"172.21.15.188:6829/2574955118","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:08:36.086637+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:08:58.948031+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:09:22.142517+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:09:47.655748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:10:12.853999+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:10:37.911383+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:11:04.448815+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:11:30.331740+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:11:55.819278+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:12:23.204612+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:12:49.484835+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:13:15.783531+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:6800/1597433518":"2024-05-11T11:07:00.092732+0000","172.21.15.100:6800/1086450256":"2024-05-11T11:06:33.114176+0000","172.21.15.100:0/767964410":"2024-05-11T11:07:00.092732+0000","172.21.15.100:6801/1597433518":"2024-05-11T11:07:00.092732+0000","172.21.15.100:6801/1086450256":"2024-05-11T11:06:33.114176+0000","172.21.15.100:0/2770572070":"2024-05-11T11:06:33.114176+0000","172.21.15.100:0/1379340513":"2024-05-11T11:06:33.114176+0000","172.21.15.100:0/3025646953":"2024-05-11T11:07:00.092732+0000","172.21.15.100:0/4099258930":"2024-05-11T11:07:00.092732+0000","172.21.15.100:0/4267693567":"2024-05-11T11:06:33.114176+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T11:18:34.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:34.786+0000 7ff9a961e700 1 -- 172.21.15.100:0/1388432327 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff98c0b2880 msgr2=0x7ff98c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:34.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:34.786+0000 7ff9a961e700 1 --2- 172.21.15.100:0/1388432327 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff98c0b2880 0x7ff98c0b4d40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff994000c00 tx=0x7ff994005fb0 comp rx=0 tx=0).stop 2024-05-10T11:18:34.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:34.786+0000 7ff9a961e700 1 -- 172.21.15.100:0/1388432327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9a41076e0 msgr2=0x7ff9a41987d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:34.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:34.786+0000 7ff9a961e700 1 --2- 172.21.15.100:0/1388432327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9a41076e0 0x7ff9a41987d0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff99c007ee0 tx=0x7ff99c00d3b0 comp rx=0 tx=0).stop 2024-05-10T11:18:34.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:34.786+0000 7ff9a961e700 1 -- 172.21.15.100:0/1388432327 shutdown_connections 2024-05-10T11:18:34.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:34.786+0000 7ff9a961e700 1 --2- 172.21.15.100:0/1388432327 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff98c0b2880 0x7ff98c0b4d40 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:34.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:34.786+0000 7ff9a961e700 1 --2- 172.21.15.100:0/1388432327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9a41076e0 0x7ff9a41987d0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:34.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:34.786+0000 7ff9a961e700 1 --2- 172.21.15.100:0/1388432327 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9a4102690 0x7ff9a419e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:34.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:34.786+0000 7ff9a961e700 1 --2- 172.21.15.100:0/1388432327 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9a4103040 0x7ff9a419eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:34.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:34.786+0000 7ff9a961e700 1 -- 172.21.15.100:0/1388432327 >> 172.21.15.100:0/1388432327 conn(0x7ff9a40783f0 msgr2=0x7ff9a4101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:34.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:34.786+0000 7ff9a961e700 1 -- 172.21.15.100:0/1388432327 shutdown_connections 2024-05-10T11:18:34.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:34.786+0000 7ff9a961e700 1 -- 172.21.15.100:0/1388432327 wait complete. 2024-05-10T11:18:35.500 DEBUG:tasks.fwd_scrub:Yielding 2024-05-10T11:18:35.500 INFO:teuthology.run_tasks:Running task background_exec... 2024-05-10T11:18:35.514 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi100.front.sepia.ceph.com 2024-05-10T11:18:35.514 DEBUG:teuthology.orchestra.run.smithi100:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-05-10T11:18:35.515 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-05-10T11:18:35.515 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T11:18:35.515 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T11:18:35.517 INFO:teuthology.run_tasks:Running task workunit... 2024-05-10T11:18:35.546 INFO:tasks.workunit:Pulling workunits from ref d39fa08867836ac070c47cb66c782fa559fd480b 2024-05-10T11:18:35.546 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-10T11:18:35.546 INFO:tasks.workunit:timeout=3h 2024-05-10T11:18:35.546 INFO:tasks.workunit:cleanup=True 2024-05-10T11:18:35.547 DEBUG:teuthology.orchestra.run.smithi100:> stat -- /home/ubuntu/cephtest/mnt.0 2024-05-10T11:18:35.563 INFO:teuthology.orchestra.run.smithi100.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-05-10T11:18:35.563 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-05-10T11:18:35.563 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 72h/114d Inode: 1 Links: 2 2024-05-10T11:18:35.563 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T11:18:35.563 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:fusefs_t:s0 2024-05-10T11:18:35.563 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-05-10 11:15:41.895414827 +0000 2024-05-10T11:18:35.563 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-05-10 11:15:41.895414827 +0000 2024-05-10T11:18:35.563 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-05-10 11:18:23.076195039 +0000 2024-05-10T11:18:35.563 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: - 2024-05-10T11:18:35.563 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-05-10T11:18:35.564 DEBUG:teuthology.orchestra.run.smithi100:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-05-10T11:18:35.636 DEBUG:teuthology.orchestra.run.smithi100:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout d39fa08867836ac070c47cb66c782fa559fd480b 2024-05-10T11:18:35.694 INFO:tasks.workunit.client.0.smithi100.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-05-10T11:18:35.763 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:18:35.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:18:35 smithi100 ceph-mon[30296]: pgmap v426: 129 pgs: 129 active+clean; 650 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-05-10T11:18:35.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:18:35 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/1388432327' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T11:18:36.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:18:35 smithi178 ceph-mon[33359]: pgmap v426: 129 pgs: 129 active+clean; 650 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-05-10T11:18:36.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:18:35 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/1388432327' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T11:18:36.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:18:35 smithi188 ceph-mon[34075]: pgmap v426: 129 pgs: 129 active+clean; 650 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-05-10T11:18:36.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:18:35 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/1388432327' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T11:18:37.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.433+0000 7f1e29b83700 1 -- 172.21.15.100:0/740700267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e24101da0 msgr2=0x7f1e24102180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:37.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.433+0000 7f1e29b83700 1 --2- 172.21.15.100:0/740700267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e24101da0 0x7f1e24102180 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f1e1400b0d0 tx=0x7f1e1400b3e0 comp rx=0 tx=0).stop 2024-05-10T11:18:37.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.433+0000 7f1e29b83700 1 -- 172.21.15.100:0/740700267 shutdown_connections 2024-05-10T11:18:37.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.433+0000 7f1e29b83700 1 --2- 172.21.15.100:0/740700267 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e24106e80 0x7f1e2410ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:37.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.433+0000 7f1e29b83700 1 --2- 172.21.15.100:0/740700267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e24101da0 0x7f1e24102180 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:37.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.433+0000 7f1e29b83700 1 --2- 172.21.15.100:0/740700267 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e24102750 0x7f1e241067a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:37.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.433+0000 7f1e29b83700 1 -- 172.21.15.100:0/740700267 >> 172.21.15.100:0/740700267 conn(0x7f1e240789a0 msgr2=0x7f1e240ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:37.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.433+0000 7f1e29b83700 1 -- 172.21.15.100:0/740700267 shutdown_connections 2024-05-10T11:18:37.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.433+0000 7f1e29b83700 1 -- 172.21.15.100:0/740700267 wait complete. 2024-05-10T11:18:37.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.434+0000 7f1e29b83700 1 Processor -- start 2024-05-10T11:18:37.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.434+0000 7f1e29b83700 1 -- start start 2024-05-10T11:18:37.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.434+0000 7f1e29b83700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e24101da0 0x7f1e2419f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:37.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.435+0000 7f1e29b83700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e24102750 0x7f1e2419fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:37.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.435+0000 7f1e29b83700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e24106e80 0x7f1e24199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:37.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.435+0000 7f1e29b83700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1e24111c60 con 0x7f1e24102750 2024-05-10T11:18:37.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.435+0000 7f1e29b83700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1e24111ae0 con 0x7f1e24101da0 2024-05-10T11:18:37.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.435+0000 7f1e29b83700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1e24111de0 con 0x7f1e24106e80 2024-05-10T11:18:37.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.435+0000 7f1e22ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e24102750 0x7f1e2419fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:37.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.435+0000 7f1e22ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e24102750 0x7f1e2419fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49160/0 (socket says 172.21.15.100:49160) 2024-05-10T11:18:37.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.435+0000 7f1e22ffd700 1 -- 172.21.15.100:0/779214881 learned_addr learned my addr 172.21.15.100:0/779214881 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:18:37.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.435+0000 7f1e23fff700 1 --2- 172.21.15.100:0/779214881 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e24106e80 0x7f1e24199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:37.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.435+0000 7f1e237fe700 1 --2- 172.21.15.100:0/779214881 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e24101da0 0x7f1e2419f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:37.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.435+0000 7f1e22ffd700 1 -- 172.21.15.100:0/779214881 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e24106e80 msgr2=0x7f1e24199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:37.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.435+0000 7f1e22ffd700 1 --2- 172.21.15.100:0/779214881 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e24106e80 0x7f1e24199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:37.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.435+0000 7f1e22ffd700 1 -- 172.21.15.100:0/779214881 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e24101da0 msgr2=0x7f1e2419f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:37.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.435+0000 7f1e22ffd700 1 --2- 172.21.15.100:0/779214881 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e24101da0 0x7f1e2419f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:37.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.435+0000 7f1e22ffd700 1 -- 172.21.15.100:0/779214881 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e2419a0b0 con 0x7f1e24102750 2024-05-10T11:18:37.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.435+0000 7f1e237fe700 1 --2- 172.21.15.100:0/779214881 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e24101da0 0x7f1e2419f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:18:37.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.435+0000 7f1e22ffd700 1 --2- 172.21.15.100:0/779214881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e24102750 0x7f1e2419fcd0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f1e1800c970 tx=0x7f1e1800cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:37.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.435+0000 7f1e20ff9700 1 -- 172.21.15.100:0/779214881 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e18007b00 con 0x7f1e24102750 2024-05-10T11:18:37.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.435+0000 7f1e20ff9700 1 -- 172.21.15.100:0/779214881 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1e18004cc0 con 0x7f1e24102750 2024-05-10T11:18:37.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.435+0000 7f1e29b83700 1 -- 172.21.15.100:0/779214881 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e2419a3a0 con 0x7f1e24102750 2024-05-10T11:18:37.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.436+0000 7f1e20ff9700 1 -- 172.21.15.100:0/779214881 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e18005580 con 0x7f1e24102750 2024-05-10T11:18:37.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.436+0000 7f1e29b83700 1 -- 172.21.15.100:0/779214881 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e241a6660 con 0x7f1e24102750 2024-05-10T11:18:37.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.438+0000 7f1e20ff9700 1 -- 172.21.15.100:0/779214881 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f1e18007ca0 con 0x7f1e24102750 2024-05-10T11:18:37.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.439+0000 7f1e20ff9700 1 --2- 172.21.15.100:0/779214881 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f1e0c0b2880 0x7f1e0c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:37.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.439+0000 7f1e237fe700 1 --2- 172.21.15.100:0/779214881 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f1e0c0b2880 0x7f1e0c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:37.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.439+0000 7f1e20ff9700 1 -- 172.21.15.100:0/779214881 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f1e180e21b0 con 0x7f1e24102750 2024-05-10T11:18:37.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.439+0000 7f1e237fe700 1 --2- 172.21.15.100:0/779214881 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f1e0c0b2880 0x7f1e0c0b4d40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1e14009e60 tx=0x7f1e1400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:37.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.440+0000 7f1e34416b80 1 Processor -- start 2024-05-10T11:18:37.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.440+0000 7f1e34416b80 1 -- start start 2024-05-10T11:18:37.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.440+0000 7f1e34416b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee279bf230 0x55ee279bf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:37.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.440+0000 7f1e34416b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee279bfb50 0x55ee279b9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:37.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.440+0000 7f1e34416b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ee279b9dd0 0x55ee279ba210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:37.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.440+0000 7f1e34416b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55ee2790b9c0 con 0x55ee279bfb50 2024-05-10T11:18:37.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.440+0000 7f1e34416b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ee27870e20 con 0x55ee279bf230 2024-05-10T11:18:37.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.440+0000 7f1e34416b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55ee27907980 con 0x55ee279b9dd0 2024-05-10T11:18:37.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.440+0000 7f1e22ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee279bfb50 0x55ee279b9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:37.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.440+0000 7f1e22ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee279bfb50 0x55ee279b9890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49170/0 (socket says 172.21.15.100:49170) 2024-05-10T11:18:37.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.440+0000 7f1e22ffd700 1 -- 172.21.15.100:0/570308421 learned_addr learned my addr 172.21.15.100:0/570308421 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:18:37.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.440+0000 7f1e23fff700 1 --2- 172.21.15.100:0/570308421 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ee279b9dd0 0x55ee279ba210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:37.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.440+0000 7f1e237fe700 1 --2- 172.21.15.100:0/570308421 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee279bf230 0x55ee279bf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:37.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.440+0000 7f1e22ffd700 1 -- 172.21.15.100:0/570308421 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ee279b9dd0 msgr2=0x55ee279ba210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:18:37.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.440+0000 7f1e22ffd700 1 --2- 172.21.15.100:0/570308421 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ee279b9dd0 0x55ee279ba210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:37.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.440+0000 7f1e22ffd700 1 -- 172.21.15.100:0/570308421 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee279bf230 msgr2=0x55ee279bf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:18:37.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.440+0000 7f1e22ffd700 1 --2- 172.21.15.100:0/570308421 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee279bf230 0x55ee279bf610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:37.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.440+0000 7f1e22ffd700 1 -- 172.21.15.100:0/570308421 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee27858950 con 0x55ee279bfb50 2024-05-10T11:18:37.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.440+0000 7f1e22ffd700 1 --2- 172.21.15.100:0/570308421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee279bfb50 0x55ee279b9890 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f1e180a66e0 tx=0x7f1e180057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=277390398a35c8e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:37.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.441+0000 7f1e09ffb700 1 -- 172.21.15.100:0/570308421 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1801d040 con 0x55ee279bfb50 2024-05-10T11:18:37.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.441+0000 7f1e09ffb700 1 -- 172.21.15.100:0/570308421 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1e180a9470 con 0x55ee279bfb50 2024-05-10T11:18:37.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.441+0000 7f1e09ffb700 1 -- 172.21.15.100:0/570308421 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e180b20a0 con 0x55ee279bfb50 2024-05-10T11:18:37.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.441+0000 7f1e34416b80 1 -- 172.21.15.100:0/570308421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee279bfb50 msgr2=0x55ee279b9890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:37.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.441+0000 7f1e34416b80 1 --2- 172.21.15.100:0/570308421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee279bfb50 0x55ee279b9890 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f1e180a66e0 tx=0x7f1e180057a0 comp rx=0 tx=0).stop 2024-05-10T11:18:37.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.441+0000 7f1e34416b80 1 -- 172.21.15.100:0/570308421 shutdown_connections 2024-05-10T11:18:37.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.441+0000 7f1e34416b80 1 --2- 172.21.15.100:0/570308421 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ee279b9dd0 0x55ee279ba210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:37.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.441+0000 7f1e34416b80 1 --2- 172.21.15.100:0/570308421 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee279bf230 0x55ee279bf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:37.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.441+0000 7f1e34416b80 1 --2- 172.21.15.100:0/570308421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee279bfb50 0x55ee279b9890 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:37.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.441+0000 7f1e34416b80 1 -- 172.21.15.100:0/570308421 >> 172.21.15.100:0/570308421 conn(0x55ee278e5bf0 msgr2=0x55ee279b4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:37.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.442+0000 7f1e34416b80 1 -- 172.21.15.100:0/570308421 shutdown_connections 2024-05-10T11:18:37.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.442+0000 7f1e34416b80 1 -- 172.21.15.100:0/570308421 wait complete. 2024-05-10T11:18:37.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.442+0000 7f1e34416b80 1 -- start start 2024-05-10T11:18:37.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.442+0000 7f1e34416b80 1 Processor -- start 2024-05-10T11:18:37.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.442+0000 7f1e34416b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee279b9dd0 0x55ee279c57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:37.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.442+0000 7f1e34416b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee279bf230 0x55ee279c2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:37.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.442+0000 7f1e34416b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ee279bfb50 0x55ee279c30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:37.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.442+0000 7f1e34416b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55ee279c64d0 con 0x55ee279bf230 2024-05-10T11:18:37.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.442+0000 7f1e34416b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ee278c4f50 con 0x55ee279b9dd0 2024-05-10T11:18:37.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.442+0000 7f1e34416b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55ee279c6650 con 0x55ee279bfb50 2024-05-10T11:18:37.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.442+0000 7f1e34416b80 10 client.0 _ll_register_callbacks cb 0x55ee2789f630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:18:37.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.442+0000 7f1e22ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee279bf230 0x55ee279c2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:37.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.443+0000 7f1e22ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee279bf230 0x55ee279c2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49174/0 (socket says 172.21.15.100:49174) 2024-05-10T11:18:37.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.443+0000 7f1e22ffd700 1 -- 172.21.15.100:0/858559146 learned_addr learned my addr 172.21.15.100:0/858559146 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:18:37.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.443+0000 7f1e23fff700 1 --2- 172.21.15.100:0/858559146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ee279bfb50 0x55ee279c30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:37.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.443+0000 7f1e237fe700 1 --2- 172.21.15.100:0/858559146 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee279b9dd0 0x55ee279c57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:37.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.443+0000 7f1e22ffd700 1 -- 172.21.15.100:0/858559146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ee279bfb50 msgr2=0x55ee279c30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:18:37.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.443+0000 7f1e22ffd700 1 --2- 172.21.15.100:0/858559146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ee279bfb50 0x55ee279c30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:37.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.443+0000 7f1e22ffd700 1 -- 172.21.15.100:0/858559146 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee279b9dd0 msgr2=0x55ee279c57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:18:37.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.443+0000 7f1e22ffd700 1 --2- 172.21.15.100:0/858559146 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee279b9dd0 0x55ee279c57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:37.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.443+0000 7f1e22ffd700 1 -- 172.21.15.100:0/858559146 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee278c27e0 con 0x55ee279bf230 2024-05-10T11:18:37.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.443+0000 7f1e23fff700 1 --2- 172.21.15.100:0/858559146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ee279bfb50 0x55ee279c30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:18:37.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.443+0000 7f1e237fe700 1 --2- 172.21.15.100:0/858559146 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee279b9dd0 0x55ee279c57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:18:37.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.443+0000 7f1e22ffd700 1 --2- 172.21.15.100:0/858559146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee279bf230 0x55ee279c2b90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f1e180a54f0 tx=0x7f1e180b2120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b1ce3157e8ffef4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:37.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.443+0000 7f1e02ffd700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:18:37.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.443+0000 7f1e02ffd700 1 -- 172.21.15.100:0/858559146 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1806d060 con 0x55ee279bf230 2024-05-10T11:18:37.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.443+0000 7f1e02ffd700 1 -- 172.21.15.100:0/858559146 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f1e180a5a60 con 0x55ee279bf230 2024-05-10T11:18:37.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.443+0000 7f1e34416b80 1 -- 172.21.15.100:0/858559146 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e180a9470 con 0x55ee279bf230 2024-05-10T11:18:37.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.443+0000 7f1e02ffd700 1 -- 172.21.15.100:0/858559146 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1e180c2030 con 0x55ee279bf230 2024-05-10T11:18:37.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.443+0000 7f1e34416b80 10 client.15291 fetch_fsmap learned FSMap version 27 2024-05-10T11:18:37.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.443+0000 7f1e34416b80 1 -- 172.21.15.100:0/858559146 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ee279d0db0 con 0x55ee279bf230 2024-05-10T11:18:37.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.443+0000 7f1e02ffd700 1 -- 172.21.15.100:0/858559146 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1e180a9470 con 0x55ee279bf230 2024-05-10T11:18:37.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.444+0000 7f1e02ffd700 1 -- 172.21.15.100:0/858559146 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f1e180a3b90 con 0x55ee279bf230 2024-05-10T11:18:37.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.444+0000 7f1e34416b80 10 client.15291 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T11:18:37.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.444+0000 7f1e34416b80 10 client.15291 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:18:37.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.444+0000 7f1e34416b80 20 client.15291 populate_metadata read hostname 'smithi100' 2024-05-10T11:18:37.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.444+0000 7f1e34416b80 1 --2- 172.21.15.100:0/858559146 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ee279d3280 0x55ee279d5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:37.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.444+0000 7f1e34416b80 4 client.15291 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:18:37.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.444+0000 7f1e34416b80 1 -- 172.21.15.100:0/858559146 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee279193a0 con 0x55ee279d3280 2024-05-10T11:18:37.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.444+0000 7f1e237fe700 1 --2- 172.21.15.100:0/858559146 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ee279d3280 0x55ee279d5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:37.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.444+0000 7f1e02ffd700 10 client.15291 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:18:37.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.444+0000 7f1e237fe700 1 --2- 172.21.15.100:0/858559146 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ee279d3280 0x55ee279d5670 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d2a958ff8d8adaa server_cookie=cb887f1f2e84fd39 in_seq=0 out_seq=0 2024-05-10T11:18:37.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.445+0000 7f1e02ffd700 1 -- 172.21.15.100:0/858559146 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ee279193a0 con 0x55ee279d3280 2024-05-10T11:18:37.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.445+0000 7f1e02ffd700 10 client.15291 handle_command_reply: tid=0 2024-05-10T11:18:37.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.445+0000 7f1e34416b80 1 client.15291 shutdown 2024-05-10T11:18:37.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.445+0000 7f1e34416b80 1 -- 172.21.15.100:0/858559146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee279bf230 msgr2=0x55ee279c2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:37.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.445+0000 7f1e34416b80 1 --2- 172.21.15.100:0/858559146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee279bf230 0x55ee279c2b90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f1e180a54f0 tx=0x7f1e180b2120 comp rx=0 tx=0).stop 2024-05-10T11:18:37.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.445+0000 7f1e34416b80 1 -- 172.21.15.100:0/858559146 shutdown_connections 2024-05-10T11:18:37.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.445+0000 7f1e34416b80 1 --2- 172.21.15.100:0/858559146 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ee279d3280 0x55ee279d5670 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:37.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.445+0000 7f1e34416b80 1 --2- 172.21.15.100:0/858559146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ee279bfb50 0x55ee279c30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:37.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.445+0000 7f1e34416b80 1 --2- 172.21.15.100:0/858559146 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee279b9dd0 0x55ee279c57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:37.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.445+0000 7f1e02ffd700 0 client.15291 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:18:37.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.445+0000 7f1e34416b80 1 --2- 172.21.15.100:0/858559146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee279bf230 0x55ee279c2b90 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:37.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.445+0000 7f1e34416b80 1 -- 172.21.15.100:0/858559146 >> 172.21.15.100:0/858559146 conn(0x55ee278e5bf0 msgr2=0x55ee279aebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:37.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.446+0000 7f1e34416b80 1 -- 172.21.15.100:0/858559146 shutdown_connections 2024-05-10T11:18:37.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.446+0000 7f1e34416b80 1 -- 172.21.15.100:0/858559146 wait complete. 2024-05-10T11:18:37.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.446+0000 7f1e34416b80 20 client.15291 trim_cache size 0 max 16384 2024-05-10T11:18:37.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.466+0000 7f1e34416b80 1 Processor -- start 2024-05-10T11:18:37.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.466+0000 7f1e34416b80 1 -- start start 2024-05-10T11:18:37.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.466+0000 7f1e34416b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ee279b9dd0 0x55ee279b9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:37.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.466+0000 7f1e34416b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee279bf230 0x55ee279c07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:37.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.466+0000 7f1e34416b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee279bfb50 0x55ee279c31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:37.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.466+0000 7f1e34416b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1e241b1a40 con 0x55ee279bf230 2024-05-10T11:18:37.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.466+0000 7f1e34416b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1e241af2d0 con 0x55ee279bfb50 2024-05-10T11:18:37.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.466+0000 7f1e34416b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1e241ad120 con 0x55ee279b9dd0 2024-05-10T11:18:37.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.466+0000 7f1e22ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee279bf230 0x55ee279c07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:37.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.466+0000 7f1e22ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee279bf230 0x55ee279c07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49180/0 (socket says 172.21.15.100:49180) 2024-05-10T11:18:37.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.466+0000 7f1e22ffd700 1 -- 172.21.15.100:0/1820175619 learned_addr learned my addr 172.21.15.100:0/1820175619 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:18:37.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.467+0000 7f1e237fe700 1 --2- 172.21.15.100:0/1820175619 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ee279b9dd0 0x55ee279b9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:37.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.467+0000 7f1e22ffd700 1 -- 172.21.15.100:0/1820175619 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ee279b9dd0 msgr2=0x55ee279b9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:18:37.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.467+0000 7f1e22ffd700 1 --2- 172.21.15.100:0/1820175619 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ee279b9dd0 0x55ee279b9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:37.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.467+0000 7f1e23fff700 1 --2- 172.21.15.100:0/1820175619 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee279bfb50 0x55ee279c31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:37.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.467+0000 7f1e22ffd700 1 -- 172.21.15.100:0/1820175619 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee279bfb50 msgr2=0x55ee279c31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:18:37.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.467+0000 7f1e22ffd700 1 --2- 172.21.15.100:0/1820175619 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee279bfb50 0x55ee279c31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:37.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.467+0000 7f1e22ffd700 1 -- 172.21.15.100:0/1820175619 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee279bf860 con 0x55ee279bf230 2024-05-10T11:18:37.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.467+0000 7f1e23fff700 1 --2- 172.21.15.100:0/1820175619 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee279bfb50 0x55ee279c31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:18:37.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.467+0000 7f1e22ffd700 1 --2- 172.21.15.100:0/1820175619 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee279bf230 0x55ee279c07c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f1e180a54a0 tx=0x7f1e1800cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f47288e6452d5302 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:37.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.467+0000 7f1e097fa700 1 -- 172.21.15.100:0/1820175619 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1806d040 con 0x55ee279bf230 2024-05-10T11:18:37.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.467+0000 7f1e097fa700 1 -- 172.21.15.100:0/1820175619 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1e180a9470 con 0x55ee279bf230 2024-05-10T11:18:37.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.467+0000 7f1e097fa700 1 -- 172.21.15.100:0/1820175619 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1801d070 con 0x55ee279bf230 2024-05-10T11:18:37.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.468+0000 7f1e34416b80 1 -- 172.21.15.100:0/1820175619 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee279bf230 msgr2=0x55ee279c07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:37.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.468+0000 7f1e34416b80 1 --2- 172.21.15.100:0/1820175619 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee279bf230 0x55ee279c07c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f1e180a54a0 tx=0x7f1e1800cf10 comp rx=0 tx=0).stop 2024-05-10T11:18:37.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.468+0000 7f1e34416b80 1 -- 172.21.15.100:0/1820175619 shutdown_connections 2024-05-10T11:18:37.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.468+0000 7f1e34416b80 1 --2- 172.21.15.100:0/1820175619 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee279bfb50 0x55ee279c31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:37.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.468+0000 7f1e34416b80 1 --2- 172.21.15.100:0/1820175619 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ee279b9dd0 0x55ee279b9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:37.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.468+0000 7f1e34416b80 1 --2- 172.21.15.100:0/1820175619 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee279bf230 0x55ee279c07c0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:37.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.468+0000 7f1e34416b80 1 -- 172.21.15.100:0/1820175619 >> 172.21.15.100:0/1820175619 conn(0x55ee278e5bf0 msgr2=0x55ee279bdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:37.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.468+0000 7f1e34416b80 1 -- 172.21.15.100:0/1820175619 shutdown_connections 2024-05-10T11:18:37.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.468+0000 7f1e34416b80 1 -- 172.21.15.100:0/1820175619 wait complete. 2024-05-10T11:18:37.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.469+0000 7f1e34416b80 1 -- start start 2024-05-10T11:18:37.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.469+0000 7f1e34416b80 1 Processor -- start 2024-05-10T11:18:37.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.469+0000 7f1e34416b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ee279b9dd0 0x55ee279d8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:37.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.469+0000 7f1e34416b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee279bf230 0x55ee279d8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:37.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.469+0000 7f1e34416b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee279bfb50 0x55ee279d6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:37.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.469+0000 7f1e34416b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1e241ad790 con 0x55ee279bf230 2024-05-10T11:18:37.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.469+0000 7f1e34416b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1e241aec00 con 0x55ee279bfb50 2024-05-10T11:18:37.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.469+0000 7f1e34416b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1e241ade60 con 0x55ee279b9dd0 2024-05-10T11:18:37.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.469+0000 7f1e34416b80 10 client.0 _ll_register_callbacks cb 0x55ee2789f630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:18:37.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.469+0000 7f1e22ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee279bf230 0x55ee279d8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:37.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.469+0000 7f1e22ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee279bf230 0x55ee279d8ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49192/0 (socket says 172.21.15.100:49192) 2024-05-10T11:18:37.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.469+0000 7f1e22ffd700 1 -- 172.21.15.100:0/3199720199 learned_addr learned my addr 172.21.15.100:0/3199720199 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:18:37.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.469+0000 7f1e237fe700 1 --2- 172.21.15.100:0/3199720199 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ee279b9dd0 0x55ee279d8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:37.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.469+0000 7f1e23fff700 1 --2- 172.21.15.100:0/3199720199 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee279bfb50 0x55ee279d6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:37.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.470+0000 7f1e22ffd700 1 -- 172.21.15.100:0/3199720199 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ee279b9dd0 msgr2=0x55ee279d8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:18:37.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.470+0000 7f1e22ffd700 1 --2- 172.21.15.100:0/3199720199 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ee279b9dd0 0x55ee279d8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:37.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.470+0000 7f1e22ffd700 1 -- 172.21.15.100:0/3199720199 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee279bfb50 msgr2=0x55ee279d6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:18:37.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.470+0000 7f1e22ffd700 1 --2- 172.21.15.100:0/3199720199 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee279bfb50 0x55ee279d6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:37.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.470+0000 7f1e22ffd700 1 -- 172.21.15.100:0/3199720199 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee279d72a0 con 0x55ee279bf230 2024-05-10T11:18:37.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.470+0000 7f1e22ffd700 1 --2- 172.21.15.100:0/3199720199 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee279bf230 0x55ee279d8ba0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1e180a54f0 tx=0x7f1e1800a520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd22157b9cdc7fa2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:37.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.470+0000 7f1e02ffd700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:18:37.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.470+0000 7f1e02ffd700 1 -- 172.21.15.100:0/3199720199 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1806d070 con 0x55ee279bf230 2024-05-10T11:18:37.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.470+0000 7f1e02ffd700 1 -- 172.21.15.100:0/3199720199 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f1e180a5a60 con 0x55ee279bf230 2024-05-10T11:18:37.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.470+0000 7f1e34416b80 1 -- 172.21.15.100:0/3199720199 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e180a9470 con 0x55ee279bf230 2024-05-10T11:18:37.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.470+0000 7f1e02ffd700 1 -- 172.21.15.100:0/3199720199 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1e1808d050 con 0x55ee279bf230 2024-05-10T11:18:37.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.470+0000 7f1e34416b80 10 client.15297 fetch_fsmap learned FSMap version 27 2024-05-10T11:18:37.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.470+0000 7f1e34416b80 1 -- 172.21.15.100:0/3199720199 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ee279d72a0 con 0x55ee279bf230 2024-05-10T11:18:37.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.470+0000 7f1e02ffd700 1 -- 172.21.15.100:0/3199720199 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1e180a9470 con 0x55ee279bf230 2024-05-10T11:18:37.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.470+0000 7f1e02ffd700 1 -- 172.21.15.100:0/3199720199 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f1e180a4e70 con 0x55ee279bf230 2024-05-10T11:18:37.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.470+0000 7f1e34416b80 10 client.15297 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T11:18:37.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.470+0000 7f1e34416b80 10 client.15297 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:18:37.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.470+0000 7f1e34416b80 20 client.15297 populate_metadata read hostname 'smithi100' 2024-05-10T11:18:37.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.470+0000 7f1e34416b80 1 --2- 172.21.15.100:0/3199720199 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ee27a3e500 0x55ee27a40920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:37.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.470+0000 7f1e34416b80 4 client.15297 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T11:18:37.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.470+0000 7f1e34416b80 1 -- 172.21.15.100:0/3199720199 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ee27913440 con 0x55ee27a3e500 2024-05-10T11:18:37.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.471+0000 7f1e237fe700 1 --2- 172.21.15.100:0/3199720199 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ee27a3e500 0x55ee27a40920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:37.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.471+0000 7f1e02ffd700 10 client.15297 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:18:37.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.471+0000 7f1e237fe700 1 --2- 172.21.15.100:0/3199720199 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ee27a3e500 0x55ee27a40920 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=376cea894180e466 server_cookie=f8a664d705b6fc56 in_seq=0 out_seq=0 2024-05-10T11:18:37.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.472+0000 7f1e02ffd700 1 -- 172.21.15.100:0/3199720199 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ee27913440 con 0x55ee27a3e500 2024-05-10T11:18:37.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.472+0000 7f1e02ffd700 10 client.15297 handle_command_reply: tid=0 2024-05-10T11:18:37.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.472+0000 7f1e34416b80 1 client.15297 shutdown 2024-05-10T11:18:37.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.473+0000 7f1e34416b80 1 -- 172.21.15.100:0/3199720199 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee279bf230 msgr2=0x55ee279d8ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:37.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.473+0000 7f1e34416b80 1 --2- 172.21.15.100:0/3199720199 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee279bf230 0x55ee279d8ba0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1e180a54f0 tx=0x7f1e1800a520 comp rx=0 tx=0).stop 2024-05-10T11:18:37.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.473+0000 7f1e34416b80 1 -- 172.21.15.100:0/3199720199 shutdown_connections 2024-05-10T11:18:37.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.473+0000 7f1e34416b80 1 --2- 172.21.15.100:0/3199720199 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ee27a3e500 0x55ee27a40920 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:37.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.473+0000 7f1e34416b80 1 --2- 172.21.15.100:0/3199720199 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee279bfb50 0x55ee279d6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:37.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.473+0000 7f1e34416b80 1 --2- 172.21.15.100:0/3199720199 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ee279b9dd0 0x55ee279d8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:37.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.473+0000 7f1e02ffd700 0 client.15297 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:18:37.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.473+0000 7f1e34416b80 1 --2- 172.21.15.100:0/3199720199 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee279bf230 0x55ee279d8ba0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:37.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.473+0000 7f1e34416b80 1 -- 172.21.15.100:0/3199720199 >> 172.21.15.100:0/3199720199 conn(0x55ee278e5bf0 msgr2=0x55ee279be5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:37.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.473+0000 7f1e34416b80 1 -- 172.21.15.100:0/3199720199 shutdown_connections 2024-05-10T11:18:37.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.473+0000 7f1e34416b80 1 -- 172.21.15.100:0/3199720199 wait complete. 2024-05-10T11:18:37.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.473+0000 7f1e34416b80 20 client.15297 trim_cache size 0 max 16384 2024-05-10T11:18:37.478 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:18:37.478 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-05-10T11:18:37.478 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "d17234fa-0680-4bc4-ab72-35c080724035", 2024-05-10T11:18:37.478 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-05-10T11:18:37.478 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:18:37.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.473+0000 7f1e29b83700 1 -- 172.21.15.100:0/779214881 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f1e0c0b2880 msgr2=0x7f1e0c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:37.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.473+0000 7f1e29b83700 1 --2- 172.21.15.100:0/779214881 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f1e0c0b2880 0x7f1e0c0b4d40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1e14009e60 tx=0x7f1e1400b040 comp rx=0 tx=0).stop 2024-05-10T11:18:37.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.473+0000 7f1e29b83700 1 -- 172.21.15.100:0/779214881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e24102750 msgr2=0x7f1e2419fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:37.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.473+0000 7f1e29b83700 1 --2- 172.21.15.100:0/779214881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e24102750 0x7f1e2419fcd0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f1e1800c970 tx=0x7f1e1800cd30 comp rx=0 tx=0).stop 2024-05-10T11:18:37.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.474+0000 7f1e29b83700 1 -- 172.21.15.100:0/779214881 shutdown_connections 2024-05-10T11:18:37.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.474+0000 7f1e29b83700 1 --2- 172.21.15.100:0/779214881 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f1e0c0b2880 0x7f1e0c0b4d40 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:37.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.474+0000 7f1e29b83700 1 --2- 172.21.15.100:0/779214881 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e24106e80 0x7f1e24199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:37.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.474+0000 7f1e29b83700 1 --2- 172.21.15.100:0/779214881 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e24101da0 0x7f1e2419f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:37.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.474+0000 7f1e29b83700 1 --2- 172.21.15.100:0/779214881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e24102750 0x7f1e2419fcd0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:37.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.474+0000 7f1e29b83700 1 -- 172.21.15.100:0/779214881 >> 172.21.15.100:0/779214881 conn(0x7f1e240789a0 msgr2=0x7f1e240ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:37.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.474+0000 7f1e29b83700 1 -- 172.21.15.100:0/779214881 shutdown_connections 2024-05-10T11:18:37.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:37.474+0000 7f1e29b83700 1 -- 172.21.15.100:0/779214881 wait complete. 2024-05-10T11:18:37.933 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:18:37 smithi100 ceph-mon[30296]: pgmap v427: 129 pgs: 129 active+clean; 650 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-05-10T11:18:37.933 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:18:37 smithi100 ceph-mon[30296]: scrub queued for path: / 2024-05-10T11:18:37.933 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:18:37 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:18:37.933 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:18:37 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:18:37.933 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:18:37 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:18:38.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:18:37 smithi178 ceph-mon[33359]: pgmap v427: 129 pgs: 129 active+clean; 650 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-05-10T11:18:38.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:18:37 smithi178 ceph-mon[33359]: scrub queued for path: / 2024-05-10T11:18:38.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:18:37 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:18:38.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:18:37 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:18:38.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:18:37 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:18:38.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:18:37 smithi188 ceph-mon[34075]: pgmap v427: 129 pgs: 129 active+clean; 650 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-05-10T11:18:38.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:18:37 smithi188 ceph-mon[34075]: scrub queued for path: / 2024-05-10T11:18:38.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:18:37 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:18:38.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:18:37 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:18:38.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:18:37 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:18:38.363 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T11:18:38.648 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:18:39.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:18:39 smithi100 ceph-mon[30296]: pgmap v428: 129 pgs: 129 active+clean; 655 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-05-10T11:18:40.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:18:39 smithi178 ceph-mon[33359]: pgmap v428: 129 pgs: 129 active+clean; 655 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-05-10T11:18:40.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:18:39 smithi188 ceph-mon[34075]: pgmap v428: 129 pgs: 129 active+clean; 655 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-05-10T11:18:40.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.334+0000 7faf1b807700 1 -- 172.21.15.100:0/3214928223 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf14108400 msgr2=0x7faf141087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:40.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.334+0000 7faf1b807700 1 --2- 172.21.15.100:0/3214928223 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf14108400 0x7faf141087e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7faf1000b0a0 tx=0x7faf1000b3b0 comp rx=0 tx=0).stop 2024-05-10T11:18:40.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.334+0000 7faf1b807700 1 -- 172.21.15.100:0/3214928223 shutdown_connections 2024-05-10T11:18:40.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.334+0000 7faf1b807700 1 --2- 172.21.15.100:0/3214928223 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf14102d50 0x7faf1410f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:40.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.334+0000 7faf1b807700 1 --2- 172.21.15.100:0/3214928223 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf14108400 0x7faf141087e0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:40.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.334+0000 7faf1b807700 1 --2- 172.21.15.100:0/3214928223 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf14102390 0x7faf14102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:40.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.334+0000 7faf1b807700 1 -- 172.21.15.100:0/3214928223 >> 172.21.15.100:0/3214928223 conn(0x7faf14079310 msgr2=0x7faf14079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:40.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.334+0000 7faf1b807700 1 -- 172.21.15.100:0/3214928223 shutdown_connections 2024-05-10T11:18:40.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.334+0000 7faf1b807700 1 -- 172.21.15.100:0/3214928223 wait complete. 2024-05-10T11:18:40.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.335+0000 7faf1b807700 1 Processor -- start 2024-05-10T11:18:40.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.335+0000 7faf1b807700 1 -- start start 2024-05-10T11:18:40.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.335+0000 7faf1b807700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf14102390 0x7faf1419a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:40.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.335+0000 7faf1b807700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf14102d50 0x7faf1419ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:40.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.335+0000 7faf1b807700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf14108400 0x7faf141a59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:40.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.335+0000 7faf1b807700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7faf14112450 con 0x7faf14102d50 2024-05-10T11:18:40.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.335+0000 7faf1b807700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7faf141122d0 con 0x7faf14108400 2024-05-10T11:18:40.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.335+0000 7faf1b807700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7faf141125d0 con 0x7faf14102390 2024-05-10T11:18:40.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.335+0000 7faf18da2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf14102d50 0x7faf1419ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:40.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.335+0000 7faf18da2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf14102d50 0x7faf1419ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49212/0 (socket says 172.21.15.100:49212) 2024-05-10T11:18:40.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.335+0000 7faf18da2700 1 -- 172.21.15.100:0/1835273358 learned_addr learned my addr 172.21.15.100:0/1835273358 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:18:40.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.335+0000 7faf195a3700 1 --2- 172.21.15.100:0/1835273358 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf14102390 0x7faf1419a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:40.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.335+0000 7faf19da4700 1 --2- 172.21.15.100:0/1835273358 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf14108400 0x7faf141a59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:40.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.335+0000 7faf18da2700 1 -- 172.21.15.100:0/1835273358 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf14102390 msgr2=0x7faf1419a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:40.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.335+0000 7faf18da2700 1 --2- 172.21.15.100:0/1835273358 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf14102390 0x7faf1419a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:40.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.335+0000 7faf18da2700 1 -- 172.21.15.100:0/1835273358 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf14108400 msgr2=0x7faf141a59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:40.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.335+0000 7faf18da2700 1 --2- 172.21.15.100:0/1835273358 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf14108400 0x7faf141a59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:40.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.335+0000 7faf18da2700 1 -- 172.21.15.100:0/1835273358 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf141a5f10 con 0x7faf14102d50 2024-05-10T11:18:40.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.336+0000 7faf18da2700 1 --2- 172.21.15.100:0/1835273358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf14102d50 0x7faf1419ae90 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7faf0400b530 tx=0x7faf0400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:40.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.336+0000 7faf0a7fc700 1 -- 172.21.15.100:0/1835273358 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf04015020 con 0x7faf14102d50 2024-05-10T11:18:40.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.336+0000 7faf1b807700 1 -- 172.21.15.100:0/1835273358 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf141a60e0 con 0x7faf14102d50 2024-05-10T11:18:40.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.336+0000 7faf0a7fc700 1 -- 172.21.15.100:0/1835273358 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faf04011cc0 con 0x7faf14102d50 2024-05-10T11:18:40.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.336+0000 7faf0a7fc700 1 -- 172.21.15.100:0/1835273358 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf04010c80 con 0x7faf14102d50 2024-05-10T11:18:40.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.336+0000 7faf1b807700 1 -- 172.21.15.100:0/1835273358 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf141a6640 con 0x7faf14102d50 2024-05-10T11:18:40.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.338+0000 7faf0a7fc700 1 -- 172.21.15.100:0/1835273358 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7faf04018020 con 0x7faf14102d50 2024-05-10T11:18:40.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.339+0000 7faf0a7fc700 1 --2- 172.21.15.100:0/1835273358 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7faf000b2880 0x7faf000b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:40.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.339+0000 7faf195a3700 1 --2- 172.21.15.100:0/1835273358 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7faf000b2880 0x7faf000b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:40.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.339+0000 7faf0a7fc700 1 -- 172.21.15.100:0/1835273358 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7faf040e1c50 con 0x7faf14102d50 2024-05-10T11:18:40.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.339+0000 7faf195a3700 1 --2- 172.21.15.100:0/1835273358 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7faf000b2880 0x7faf000b4d40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7faf10009e60 tx=0x7faf1000b7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:40.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.340+0000 7faf2609ab80 1 Processor -- start 2024-05-10T11:18:40.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.340+0000 7faf2609ab80 1 -- start start 2024-05-10T11:18:40.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.340+0000 7faf2609ab80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562be3787590 0x562be3787970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:40.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.340+0000 7faf2609ab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562be3787eb0 0x562be3781c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:40.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.340+0000 7faf2609ab80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562be37821b0 0x562be3782610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:40.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.340+0000 7faf2609ab80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x562be36cf940 con 0x562be37821b0 2024-05-10T11:18:40.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.340+0000 7faf2609ab80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562be3634da0 con 0x562be3787590 2024-05-10T11:18:40.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.340+0000 7faf2609ab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x562be36cb900 con 0x562be3787eb0 2024-05-10T11:18:40.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.340+0000 7faf19da4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562be37821b0 0x562be3782610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:40.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.340+0000 7faf19da4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562be37821b0 0x562be3782610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49226/0 (socket says 172.21.15.100:49226) 2024-05-10T11:18:40.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.340+0000 7faf19da4700 1 -- 172.21.15.100:0/682047374 learned_addr learned my addr 172.21.15.100:0/682047374 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:18:40.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.340+0000 7faf195a3700 1 --2- 172.21.15.100:0/682047374 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562be3787590 0x562be3787970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:40.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.340+0000 7faf18da2700 1 --2- 172.21.15.100:0/682047374 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562be3787eb0 0x562be3781c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:40.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.340+0000 7faf19da4700 1 -- 172.21.15.100:0/682047374 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562be3787eb0 msgr2=0x562be3781c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:18:40.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.340+0000 7faf19da4700 1 --2- 172.21.15.100:0/682047374 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562be3787eb0 0x562be3781c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:40.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.340+0000 7faf19da4700 1 -- 172.21.15.100:0/682047374 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562be3787590 msgr2=0x562be3787970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:18:40.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.341+0000 7faf19da4700 1 --2- 172.21.15.100:0/682047374 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562be3787590 0x562be3787970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:40.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.341+0000 7faf19da4700 1 -- 172.21.15.100:0/682047374 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562be361c8d0 con 0x562be37821b0 2024-05-10T11:18:40.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.341+0000 7faf19da4700 1 --2- 172.21.15.100:0/682047374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562be37821b0 0x562be3782610 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7faf0c00b6c0 tx=0x7faf0c00b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=538838adda62c894 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:40.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.341+0000 7faeff7fe700 1 -- 172.21.15.100:0/682047374 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c004a30 con 0x562be37821b0 2024-05-10T11:18:40.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.341+0000 7faeff7fe700 1 -- 172.21.15.100:0/682047374 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faf0c015070 con 0x562be37821b0 2024-05-10T11:18:40.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.341+0000 7faeff7fe700 1 -- 172.21.15.100:0/682047374 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c0039a0 con 0x562be37821b0 2024-05-10T11:18:40.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.342+0000 7faf2609ab80 1 -- 172.21.15.100:0/682047374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562be37821b0 msgr2=0x562be3782610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:40.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.342+0000 7faf2609ab80 1 --2- 172.21.15.100:0/682047374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562be37821b0 0x562be3782610 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7faf0c00b6c0 tx=0x7faf0c00b9d0 comp rx=0 tx=0).stop 2024-05-10T11:18:40.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.342+0000 7faf2609ab80 1 -- 172.21.15.100:0/682047374 shutdown_connections 2024-05-10T11:18:40.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.342+0000 7faf2609ab80 1 --2- 172.21.15.100:0/682047374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562be37821b0 0x562be3782610 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:40.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.342+0000 7faf2609ab80 1 --2- 172.21.15.100:0/682047374 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562be3787590 0x562be3787970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:40.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.342+0000 7faf2609ab80 1 --2- 172.21.15.100:0/682047374 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562be3787eb0 0x562be3781c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:40.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.342+0000 7faf2609ab80 1 -- 172.21.15.100:0/682047374 >> 172.21.15.100:0/682047374 conn(0x562be36a9b70 msgr2=0x562be37782a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:40.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.342+0000 7faf2609ab80 1 -- 172.21.15.100:0/682047374 shutdown_connections 2024-05-10T11:18:40.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.342+0000 7faf2609ab80 1 -- 172.21.15.100:0/682047374 wait complete. 2024-05-10T11:18:40.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.342+0000 7faf2609ab80 1 -- start start 2024-05-10T11:18:40.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.342+0000 7faf2609ab80 1 Processor -- start 2024-05-10T11:18:40.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.343+0000 7faf2609ab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562be37821b0 0x562be378dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:40.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.343+0000 7faf2609ab80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562be3787590 0x562be378b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:40.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.343+0000 7faf2609ab80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562be3787eb0 0x562be378b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:40.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.343+0000 7faf2609ab80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x562be378e8e0 con 0x562be3787590 2024-05-10T11:18:40.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.343+0000 7faf2609ab80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562be3688ed0 con 0x562be3787eb0 2024-05-10T11:18:40.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.343+0000 7faf2609ab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x562be378ea60 con 0x562be37821b0 2024-05-10T11:18:40.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.343+0000 7faf18da2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562be3787590 0x562be378b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:40.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.343+0000 7faf2609ab80 10 client.0 _ll_register_callbacks cb 0x562be36635b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:18:40.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.343+0000 7faf18da2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562be3787590 0x562be378b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49238/0 (socket says 172.21.15.100:49238) 2024-05-10T11:18:40.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.343+0000 7faf18da2700 1 -- 172.21.15.100:0/2636021379 learned_addr learned my addr 172.21.15.100:0/2636021379 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:18:40.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.343+0000 7faf195a3700 1 --2- 172.21.15.100:0/2636021379 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562be37821b0 0x562be378dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:40.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.343+0000 7faf19da4700 1 --2- 172.21.15.100:0/2636021379 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562be3787eb0 0x562be378b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:40.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.343+0000 7faf18da2700 1 -- 172.21.15.100:0/2636021379 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562be37821b0 msgr2=0x562be378dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:18:40.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.343+0000 7faf18da2700 1 --2- 172.21.15.100:0/2636021379 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562be37821b0 0x562be378dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:40.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.343+0000 7faf18da2700 1 -- 172.21.15.100:0/2636021379 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562be3787eb0 msgr2=0x562be378b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:18:40.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.343+0000 7faf18da2700 1 --2- 172.21.15.100:0/2636021379 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562be3787eb0 0x562be378b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:40.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.343+0000 7faf18da2700 1 -- 172.21.15.100:0/2636021379 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562be3686760 con 0x562be3787590 2024-05-10T11:18:40.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.343+0000 7faf18da2700 1 --2- 172.21.15.100:0/2636021379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562be3787590 0x562be378b010 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7faf040df440 tx=0x7faf040a8aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9df9e1e888d7f5f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:40.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.343+0000 7faefcff9700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:18:40.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.343+0000 7faefcff9700 1 -- 172.21.15.100:0/2636021379 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf040a8f10 con 0x562be3787590 2024-05-10T11:18:40.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.343+0000 7faf2609ab80 1 -- 172.21.15.100:0/2636021379 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf0c015070 con 0x562be3787590 2024-05-10T11:18:40.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.343+0000 7faefcff9700 1 -- 172.21.15.100:0/2636021379 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7faf040a9380 con 0x562be3787590 2024-05-10T11:18:40.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.343+0000 7faefcff9700 1 -- 172.21.15.100:0/2636021379 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7faf040ad2c0 con 0x562be3787590 2024-05-10T11:18:40.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.344+0000 7faf2609ab80 10 client.15309 fetch_fsmap learned FSMap version 27 2024-05-10T11:18:40.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.344+0000 7faf2609ab80 1 -- 172.21.15.100:0/2636021379 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562be37992c0 con 0x562be3787590 2024-05-10T11:18:40.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.344+0000 7faefcff9700 1 -- 172.21.15.100:0/2636021379 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faf0c015070 con 0x562be3787590 2024-05-10T11:18:40.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.344+0000 7faefcff9700 1 -- 172.21.15.100:0/2636021379 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7faf040a4880 con 0x562be3787590 2024-05-10T11:18:40.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.344+0000 7faf2609ab80 10 client.15309 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T11:18:40.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.344+0000 7faf2609ab80 10 client.15309 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:18:40.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.344+0000 7faf2609ab80 20 client.15309 populate_metadata read hostname 'smithi100' 2024-05-10T11:18:40.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.344+0000 7faf2609ab80 1 --2- 172.21.15.100:0/2636021379 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562be379b900 0x562be379dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:40.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.344+0000 7faf2609ab80 4 client.15309 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:18:40.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.344+0000 7faf2609ab80 1 -- 172.21.15.100:0/2636021379 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562be36dd320 con 0x562be379b900 2024-05-10T11:18:40.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.344+0000 7faf195a3700 1 --2- 172.21.15.100:0/2636021379 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562be379b900 0x562be379dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:40.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.345+0000 7faefcff9700 10 client.15309 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:18:40.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.345+0000 7faf195a3700 1 --2- 172.21.15.100:0/2636021379 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562be379b900 0x562be379dd20 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f964da88b315680 server_cookie=1650be37c4adfcbb in_seq=0 out_seq=0 2024-05-10T11:18:40.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.345+0000 7faefcff9700 1 -- 172.21.15.100:0/2636021379 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562be36dd320 con 0x562be379b900 2024-05-10T11:18:40.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.345+0000 7faefcff9700 10 client.15309 handle_command_reply: tid=0 2024-05-10T11:18:40.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.345+0000 7faf2609ab80 1 client.15309 shutdown 2024-05-10T11:18:40.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.345+0000 7faf2609ab80 1 -- 172.21.15.100:0/2636021379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562be3787590 msgr2=0x562be378b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:40.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.345+0000 7faf2609ab80 1 --2- 172.21.15.100:0/2636021379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562be3787590 0x562be378b010 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7faf040df440 tx=0x7faf040a8aa0 comp rx=0 tx=0).stop 2024-05-10T11:18:40.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.346+0000 7faf2609ab80 1 -- 172.21.15.100:0/2636021379 shutdown_connections 2024-05-10T11:18:40.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.346+0000 7faf2609ab80 1 --2- 172.21.15.100:0/2636021379 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562be379b900 0x562be379dd20 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:40.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.346+0000 7faf2609ab80 1 --2- 172.21.15.100:0/2636021379 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562be3787eb0 0x562be378b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:40.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.346+0000 7faf2609ab80 1 --2- 172.21.15.100:0/2636021379 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562be37821b0 0x562be378dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:40.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.346+0000 7faefcff9700 0 client.15309 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:18:40.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.346+0000 7faf2609ab80 1 --2- 172.21.15.100:0/2636021379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562be3787590 0x562be378b010 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:40.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.346+0000 7faf2609ab80 1 -- 172.21.15.100:0/2636021379 >> 172.21.15.100:0/2636021379 conn(0x562be36a9b70 msgr2=0x562be3772b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:40.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.346+0000 7faf2609ab80 1 -- 172.21.15.100:0/2636021379 shutdown_connections 2024-05-10T11:18:40.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.346+0000 7faf2609ab80 1 -- 172.21.15.100:0/2636021379 wait complete. 2024-05-10T11:18:40.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.346+0000 7faf2609ab80 20 client.15309 trim_cache size 0 max 16384 2024-05-10T11:18:40.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.365+0000 7faf2609ab80 1 Processor -- start 2024-05-10T11:18:40.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.365+0000 7faf2609ab80 1 -- start start 2024-05-10T11:18:40.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.365+0000 7faf2609ab80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562be37821b0 0x562be3788bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:40.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.365+0000 7faf2609ab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562be3787590 0x562be3797970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:40.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.365+0000 7faf2609ab80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562be3787eb0 0x562be3772c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:40.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.365+0000 7faf2609ab80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7faf141af7f0 con 0x562be3787eb0 2024-05-10T11:18:40.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.365+0000 7faf2609ab80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7faf141ad080 con 0x562be37821b0 2024-05-10T11:18:40.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.365+0000 7faf2609ab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7faf141aaef0 con 0x562be3787590 2024-05-10T11:18:40.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.365+0000 7faf19da4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562be3787eb0 0x562be3772c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:40.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.365+0000 7faf19da4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562be3787eb0 0x562be3772c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49242/0 (socket says 172.21.15.100:49242) 2024-05-10T11:18:40.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.365+0000 7faf19da4700 1 -- 172.21.15.100:0/1843955869 learned_addr learned my addr 172.21.15.100:0/1843955869 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:18:40.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.366+0000 7faf195a3700 1 --2- 172.21.15.100:0/1843955869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562be37821b0 0x562be3788bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:40.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.366+0000 7faf18da2700 1 --2- 172.21.15.100:0/1843955869 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562be3787590 0x562be3797970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:40.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.366+0000 7faf19da4700 1 -- 172.21.15.100:0/1843955869 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562be3787590 msgr2=0x562be3797970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:18:40.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.366+0000 7faf19da4700 1 --2- 172.21.15.100:0/1843955869 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562be3787590 0x562be3797970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:40.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.366+0000 7faf19da4700 1 -- 172.21.15.100:0/1843955869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562be37821b0 msgr2=0x562be3788bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:18:40.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.366+0000 7faf19da4700 1 --2- 172.21.15.100:0/1843955869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562be37821b0 0x562be3788bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:40.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.366+0000 7faf19da4700 1 -- 172.21.15.100:0/1843955869 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562be378d860 con 0x562be3787eb0 2024-05-10T11:18:40.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.366+0000 7faf18da2700 1 --2- 172.21.15.100:0/1843955869 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562be3787590 0x562be3797970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:18:40.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.366+0000 7faf195a3700 1 --2- 172.21.15.100:0/1843955869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562be37821b0 0x562be3788bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:18:40.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.366+0000 7faf19da4700 1 --2- 172.21.15.100:0/1843955869 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562be3787eb0 0x562be3772c40 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7faf0c00b6c0 tx=0x7faf0c00bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9b30d2b2d829148 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:40.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.366+0000 7faefeffd700 1 -- 172.21.15.100:0/1843955869 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c028070 con 0x562be3787eb0 2024-05-10T11:18:40.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.366+0000 7faefeffd700 1 -- 172.21.15.100:0/1843955869 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faf0c015040 con 0x562be3787eb0 2024-05-10T11:18:40.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.366+0000 7faefeffd700 1 -- 172.21.15.100:0/1843955869 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c0238e0 con 0x562be3787eb0 2024-05-10T11:18:40.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.367+0000 7faf2609ab80 1 -- 172.21.15.100:0/1843955869 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562be3787eb0 msgr2=0x562be3772c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:40.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.367+0000 7faf2609ab80 1 --2- 172.21.15.100:0/1843955869 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562be3787eb0 0x562be3772c40 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7faf0c00b6c0 tx=0x7faf0c00bfd0 comp rx=0 tx=0).stop 2024-05-10T11:18:40.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.367+0000 7faf2609ab80 1 -- 172.21.15.100:0/1843955869 shutdown_connections 2024-05-10T11:18:40.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.367+0000 7faf2609ab80 1 --2- 172.21.15.100:0/1843955869 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562be3787eb0 0x562be3772c40 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:40.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.367+0000 7faf2609ab80 1 --2- 172.21.15.100:0/1843955869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562be37821b0 0x562be3788bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:40.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.367+0000 7faf2609ab80 1 --2- 172.21.15.100:0/1843955869 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562be3787590 0x562be3797970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:40.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.367+0000 7faf2609ab80 1 -- 172.21.15.100:0/1843955869 >> 172.21.15.100:0/1843955869 conn(0x562be36a9b70 msgr2=0x562be377c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:40.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.367+0000 7faf2609ab80 1 -- 172.21.15.100:0/1843955869 shutdown_connections 2024-05-10T11:18:40.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.367+0000 7faf2609ab80 1 -- 172.21.15.100:0/1843955869 wait complete. 2024-05-10T11:18:40.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.367+0000 7faf2609ab80 1 -- start start 2024-05-10T11:18:40.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.367+0000 7faf2609ab80 1 Processor -- start 2024-05-10T11:18:40.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.368+0000 7faf2609ab80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562be37821b0 0x562be3793cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:40.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.368+0000 7faf2609ab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562be3787590 0x562be3794230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:40.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.368+0000 7faf2609ab80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562be3787eb0 0x562be3790cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:40.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.368+0000 7faf2609ab80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7faf141ab540 con 0x562be3787eb0 2024-05-10T11:18:40.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.368+0000 7faf2609ab80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7faf141ac9b0 con 0x562be37821b0 2024-05-10T11:18:40.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.368+0000 7faf2609ab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7faf141abc10 con 0x562be3787590 2024-05-10T11:18:40.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.368+0000 7faf2609ab80 10 client.0 _ll_register_callbacks cb 0x562be36635b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:18:40.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.368+0000 7faf19da4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562be3787eb0 0x562be3790cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:40.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.368+0000 7faf19da4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562be3787eb0 0x562be3790cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49246/0 (socket says 172.21.15.100:49246) 2024-05-10T11:18:40.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.368+0000 7faf19da4700 1 -- 172.21.15.100:0/4242430292 learned_addr learned my addr 172.21.15.100:0/4242430292 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:18:40.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.368+0000 7faf195a3700 1 --2- 172.21.15.100:0/4242430292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562be37821b0 0x562be3793cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:40.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.368+0000 7faf19da4700 1 -- 172.21.15.100:0/4242430292 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562be3787590 msgr2=0x562be3794230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:18:40.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.368+0000 7faf19da4700 1 --2- 172.21.15.100:0/4242430292 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562be3787590 0x562be3794230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:40.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.368+0000 7faf19da4700 1 -- 172.21.15.100:0/4242430292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562be37821b0 msgr2=0x562be3793cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:18:40.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.368+0000 7faf19da4700 1 --2- 172.21.15.100:0/4242430292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562be37821b0 0x562be3793cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:40.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.368+0000 7faf19da4700 1 -- 172.21.15.100:0/4242430292 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562be3791200 con 0x562be3787eb0 2024-05-10T11:18:40.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.368+0000 7faf19da4700 1 --2- 172.21.15.100:0/4242430292 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562be3787eb0 0x562be3790cc0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7faf0c00b4e0 tx=0x7faf0c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d6be580d7a312c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:40.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.368+0000 7faefcff9700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:18:40.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.369+0000 7faefcff9700 1 -- 172.21.15.100:0/4242430292 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c028040 con 0x562be3787eb0 2024-05-10T11:18:40.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.369+0000 7faefcff9700 1 -- 172.21.15.100:0/4242430292 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7faf0c003c30 con 0x562be3787eb0 2024-05-10T11:18:40.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.369+0000 7faf2609ab80 1 -- 172.21.15.100:0/4242430292 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf0c015040 con 0x562be3787eb0 2024-05-10T11:18:40.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.369+0000 7faefcff9700 1 -- 172.21.15.100:0/4242430292 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7faf0c003e30 con 0x562be3787eb0 2024-05-10T11:18:40.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.369+0000 7faf2609ab80 10 client.15315 fetch_fsmap learned FSMap version 27 2024-05-10T11:18:40.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.369+0000 7faf2609ab80 1 -- 172.21.15.100:0/4242430292 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562be3806830 con 0x562be3787eb0 2024-05-10T11:18:40.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.369+0000 7faefcff9700 1 -- 172.21.15.100:0/4242430292 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faf0c015040 con 0x562be3787eb0 2024-05-10T11:18:40.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.369+0000 7faefcff9700 1 -- 172.21.15.100:0/4242430292 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7faf0c02b070 con 0x562be3787eb0 2024-05-10T11:18:40.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.369+0000 7faf2609ab80 10 client.15315 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T11:18:40.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.369+0000 7faf2609ab80 10 client.15315 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:18:40.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.369+0000 7faf2609ab80 20 client.15315 populate_metadata read hostname 'smithi100' 2024-05-10T11:18:40.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.369+0000 7faf2609ab80 1 --2- 172.21.15.100:0/4242430292 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562be3806be0 0x562be3808ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:40.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.369+0000 7faf2609ab80 4 client.15315 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T11:18:40.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.369+0000 7faf2609ab80 1 -- 172.21.15.100:0/4242430292 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562be36d73c0 con 0x562be3806be0 2024-05-10T11:18:40.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.370+0000 7faf195a3700 1 --2- 172.21.15.100:0/4242430292 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562be3806be0 0x562be3808ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:40.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.370+0000 7faefcff9700 10 client.15315 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:18:40.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.370+0000 7faf195a3700 1 --2- 172.21.15.100:0/4242430292 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562be3806be0 0x562be3808ff0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cb71933ded43a52 server_cookie=a57f481bf8b3e30b in_seq=0 out_seq=0 2024-05-10T11:18:40.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.370+0000 7faefcff9700 1 -- 172.21.15.100:0/4242430292 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x562be36d73c0 con 0x562be3806be0 2024-05-10T11:18:40.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.370+0000 7faefcff9700 10 client.15315 handle_command_reply: tid=0 2024-05-10T11:18:40.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.370+0000 7faf2609ab80 1 client.15315 shutdown 2024-05-10T11:18:40.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.370+0000 7faf2609ab80 1 -- 172.21.15.100:0/4242430292 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562be3787eb0 msgr2=0x562be3790cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:40.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.370+0000 7faf2609ab80 1 --2- 172.21.15.100:0/4242430292 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562be3787eb0 0x562be3790cc0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7faf0c00b4e0 tx=0x7faf0c005fd0 comp rx=0 tx=0).stop 2024-05-10T11:18:40.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.370+0000 7faf2609ab80 1 -- 172.21.15.100:0/4242430292 shutdown_connections 2024-05-10T11:18:40.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.370+0000 7faf2609ab80 1 --2- 172.21.15.100:0/4242430292 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562be3806be0 0x562be3808ff0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:40.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.370+0000 7faf2609ab80 1 --2- 172.21.15.100:0/4242430292 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562be3787eb0 0x562be3790cc0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:40.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.371+0000 7faf2609ab80 1 --2- 172.21.15.100:0/4242430292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562be37821b0 0x562be3793cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:40.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.371+0000 7faf2609ab80 1 --2- 172.21.15.100:0/4242430292 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562be3787590 0x562be3794230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:40.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.371+0000 7faf2609ab80 1 -- 172.21.15.100:0/4242430292 >> 172.21.15.100:0/4242430292 conn(0x562be36a9b70 msgr2=0x562be3788a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:40.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.371+0000 7faefcff9700 0 client.15315 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:18:40.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.371+0000 7faf2609ab80 1 -- 172.21.15.100:0/4242430292 shutdown_connections 2024-05-10T11:18:40.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.371+0000 7faf2609ab80 1 -- 172.21.15.100:0/4242430292 wait complete. 2024-05-10T11:18:40.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.371+0000 7faf2609ab80 20 client.15315 trim_cache size 0 max 16384 2024-05-10T11:18:40.378 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:18:40.378 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T11:18:40.378 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-05-10T11:18:40.378 INFO:teuthology.orchestra.run.smithi100.stdout: "d17234fa-0680-4bc4-ab72-35c080724035": { 2024-05-10T11:18:40.378 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-05-10T11:18:40.378 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "d17234fa-0680-4bc4-ab72-35c080724035", 2024-05-10T11:18:40.378 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-05-10T11:18:40.378 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:18:40.378 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:18:40.378 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:18:40.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.371+0000 7faf1b807700 1 -- 172.21.15.100:0/1835273358 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7faf000b2880 msgr2=0x7faf000b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:40.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.371+0000 7faf1b807700 1 --2- 172.21.15.100:0/1835273358 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7faf000b2880 0x7faf000b4d40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7faf10009e60 tx=0x7faf1000b7b0 comp rx=0 tx=0).stop 2024-05-10T11:18:40.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.372+0000 7faf1b807700 1 -- 172.21.15.100:0/1835273358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf14102d50 msgr2=0x7faf1419ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:40.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.372+0000 7faf1b807700 1 --2- 172.21.15.100:0/1835273358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf14102d50 0x7faf1419ae90 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7faf0400b530 tx=0x7faf0400b840 comp rx=0 tx=0).stop 2024-05-10T11:18:40.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.372+0000 7faf1b807700 1 -- 172.21.15.100:0/1835273358 shutdown_connections 2024-05-10T11:18:40.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.372+0000 7faf1b807700 1 --2- 172.21.15.100:0/1835273358 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7faf000b2880 0x7faf000b4d40 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:40.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.372+0000 7faf1b807700 1 --2- 172.21.15.100:0/1835273358 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf14108400 0x7faf141a59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:40.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.372+0000 7faf1b807700 1 --2- 172.21.15.100:0/1835273358 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf14102390 0x7faf1419a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:40.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.372+0000 7faf1b807700 1 --2- 172.21.15.100:0/1835273358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf14102d50 0x7faf1419ae90 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:40.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.372+0000 7faf1b807700 1 -- 172.21.15.100:0/1835273358 >> 172.21.15.100:0/1835273358 conn(0x7faf14079310 msgr2=0x7faf14101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:40.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.372+0000 7faf1b807700 1 -- 172.21.15.100:0/1835273358 shutdown_connections 2024-05-10T11:18:40.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:18:40.372+0000 7faf1b807700 1 -- 172.21.15.100:0/1835273358 wait complete. 2024-05-10T11:18:41.009 INFO:tasks.cephfs.filesystem:scrub status for tag:d17234fa-0680-4bc4-ab72-35c080724035 - {'path': '/', 'tag': 'd17234fa-0680-4bc4-ab72-35c080724035', 'options': 'recursive,force'} 2024-05-10T11:18:42.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:18:41 smithi178 ceph-mon[33359]: pgmap v429: 129 pgs: 129 active+clean; 655 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 511 B/s wr, 5 op/s 2024-05-10T11:18:42.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:18:41 smithi188 ceph-mon[34075]: pgmap v429: 129 pgs: 129 active+clean; 655 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 511 B/s wr, 5 op/s 2024-05-10T11:18:42.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:18:41 smithi100 ceph-mon[30296]: pgmap v429: 129 pgs: 129 active+clean; 655 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 511 B/s wr, 5 op/s 2024-05-10T11:18:44.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:18:43 smithi178 ceph-mon[33359]: pgmap v430: 129 pgs: 129 active+clean; 655 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-10T11:18:44.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:18:43 smithi188 ceph-mon[34075]: pgmap v430: 129 pgs: 129 active+clean; 655 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-10T11:18:44.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:18:43 smithi100 ceph-mon[30296]: pgmap v430: 129 pgs: 129 active+clean; 655 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-10T11:18:46.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:18:45 smithi178 ceph-mon[33359]: pgmap v431: 129 pgs: 129 active+clean; 655 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 5 op/s 2024-05-10T11:18:46.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:18:45 smithi188 ceph-mon[34075]: pgmap v431: 129 pgs: 129 active+clean; 655 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 5 op/s 2024-05-10T11:18:46.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:18:45 smithi100 ceph-mon[30296]: pgmap v431: 129 pgs: 129 active+clean; 655 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 5 op/s 2024-05-10T11:18:48.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:18:47 smithi178 ceph-mon[33359]: pgmap v432: 129 pgs: 129 active+clean; 655 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-10T11:18:48.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:18:47 smithi188 ceph-mon[34075]: pgmap v432: 129 pgs: 129 active+clean; 655 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-10T11:18:48.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:18:47 smithi100 ceph-mon[30296]: pgmap v432: 129 pgs: 129 active+clean; 655 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-10T11:18:48.675 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.658+0000 7f83c4608640 1 -- 172.21.15.100:0/4182326322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83bc103c00 msgr2=0x7f83bc104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:48.676 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.658+0000 7f83c4608640 1 --2- 172.21.15.100:0/4182326322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83bc103c00 0x7f83bc104080 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f83b0009900 tx=0x7f83b002a410 comp rx=0 tx=0).stop 2024-05-10T11:18:48.676 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.658+0000 7f83c4608640 1 -- 172.21.15.100:0/4182326322 shutdown_connections 2024-05-10T11:18:48.676 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.658+0000 7f83c4608640 1 --2- 172.21.15.100:0/4182326322 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83bc1045c0 0x7f83bc10ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:48.676 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.658+0000 7f83c4608640 1 --2- 172.21.15.100:0/4182326322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83bc103c00 0x7f83bc104080 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:48.676 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.658+0000 7f83c4608640 1 --2- 172.21.15.100:0/4182326322 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83bc102a10 0x7f83bc102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:48.676 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.658+0000 7f83c4608640 1 -- 172.21.15.100:0/4182326322 >> 172.21.15.100:0/4182326322 conn(0x7f83bc0fdfa0 msgr2=0x7f83bc1003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:48.676 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.659+0000 7f83c4608640 1 -- 172.21.15.100:0/4182326322 shutdown_connections 2024-05-10T11:18:48.676 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.659+0000 7f83c4608640 1 -- 172.21.15.100:0/4182326322 wait complete. 2024-05-10T11:18:48.676 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.659+0000 7f83c4608640 1 Processor -- start 2024-05-10T11:18:48.676 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.659+0000 7f83c4608640 1 -- start start 2024-05-10T11:18:48.676 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.659+0000 7f83c4608640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83bc102a10 0x7f83bc198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:48.676 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.659+0000 7f83c4608640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83bc103c00 0x7f83bc1990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:48.676 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.659+0000 7f83c4608640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83bc1045c0 0x7f83bc1a0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.659+0000 7f83c4608640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f83bc10e460 con 0x7f83bc103c00 2024-05-10T11:18:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.659+0000 7f83c4608640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f83bc10e2e0 con 0x7f83bc1045c0 2024-05-10T11:18:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.659+0000 7f83c4608640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f83bc10e5e0 con 0x7f83bc102a10 2024-05-10T11:18:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.660+0000 7f83c1ba3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83bc103c00 0x7f83bc1990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.660+0000 7f83c1ba3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83bc103c00 0x7f83bc1990a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46990/0 (socket says 172.21.15.100:46990) 2024-05-10T11:18:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.660+0000 7f83c1ba3640 1 -- 172.21.15.100:0/3705479713 learned_addr learned my addr 172.21.15.100:0/3705479713 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:18:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.660+0000 7f83c2ba5640 1 --2- 172.21.15.100:0/3705479713 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83bc1045c0 0x7f83bc1a0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.660+0000 7f83c23a4640 1 --2- 172.21.15.100:0/3705479713 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83bc102a10 0x7f83bc198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.660+0000 7f83c1ba3640 1 -- 172.21.15.100:0/3705479713 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83bc102a10 msgr2=0x7f83bc198b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.660+0000 7f83c1ba3640 1 --2- 172.21.15.100:0/3705479713 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83bc102a10 0x7f83bc198b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.660+0000 7f83c1ba3640 1 -- 172.21.15.100:0/3705479713 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83bc1045c0 msgr2=0x7f83bc1a0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.660+0000 7f83c1ba3640 1 --2- 172.21.15.100:0/3705479713 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83bc1045c0 0x7f83bc1a0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.660+0000 7f83c1ba3640 1 -- 172.21.15.100:0/3705479713 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83bc1a0780 con 0x7f83bc103c00 2024-05-10T11:18:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.660+0000 7f83c2ba5640 1 --2- 172.21.15.100:0/3705479713 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83bc1045c0 0x7f83bc1a0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:18:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.660+0000 7f83c1ba3640 1 --2- 172.21.15.100:0/3705479713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83bc103c00 0x7f83bc1990a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f83b0009230 tx=0x7f83b00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.660+0000 7f83bb7fe640 1 -- 172.21.15.100:0/3705479713 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b003f030 con 0x7f83bc103c00 2024-05-10T11:18:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.660+0000 7f83bb7fe640 1 -- 172.21.15.100:0/3705479713 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f83b0003c90 con 0x7f83bc103c00 2024-05-10T11:18:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.660+0000 7f83c4608640 1 -- 172.21.15.100:0/3705479713 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83bc1a0a10 con 0x7f83bc103c00 2024-05-10T11:18:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.660+0000 7f83bb7fe640 1 -- 172.21.15.100:0/3705479713 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b0035070 con 0x7f83bc103c00 2024-05-10T11:18:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.660+0000 7f83c4608640 1 -- 172.21.15.100:0/3705479713 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83bc1a0fd0 con 0x7f83bc103c00 2024-05-10T11:18:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.663+0000 7f83bb7fe640 1 -- 172.21.15.100:0/3705479713 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f83b0042030 con 0x7f83bc103c00 2024-05-10T11:18:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.663+0000 7f83c73f5740 1 Processor -- start 2024-05-10T11:18:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.663+0000 7f83c73f5740 1 -- start start 2024-05-10T11:18:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.664+0000 7f83c73f5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562a61590db0 0x562a615911d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.664+0000 7f83c73f5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a615917a0 0x562a615962a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.664+0000 7f83c73f5740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562a61596910 0x562a61598da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.664+0000 7f83c73f5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x562a614f55d0 con 0x562a615917a0 2024-05-10T11:18:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.664+0000 7f83c73f5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562a614ae890 con 0x562a61590db0 2024-05-10T11:18:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.664+0000 7f83c73f5740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x562a614f1590 con 0x562a61596910 2024-05-10T11:18:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.664+0000 7f83c23a4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a615917a0 0x562a615962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.664+0000 7f83c2ba5640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562a61590db0 0x562a615911d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.664+0000 7f83c23a4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a615917a0 0x562a615962a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47004/0 (socket says 172.21.15.100:47004) 2024-05-10T11:18:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.664+0000 7f83c23a4640 1 -- 172.21.15.100:0/3068314813 learned_addr learned my addr 172.21.15.100:0/3068314813 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:18:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.664+0000 7f83c1ba3640 1 --2- 172.21.15.100:0/3068314813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562a61596910 0x562a61598da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.664+0000 7f83bb7fe640 1 --2- 172.21.15.100:0/3705479713 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f83a40b2920 0x7f83a40b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.664+0000 7f83c23a4640 1 -- 172.21.15.100:0/3068314813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562a61596910 msgr2=0x562a61598da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:18:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.664+0000 7f83c23a4640 1 --2- 172.21.15.100:0/3068314813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562a61596910 0x562a61598da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.664+0000 7f83c23a4640 1 -- 172.21.15.100:0/3068314813 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562a61590db0 msgr2=0x562a615911d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:18:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.664+0000 7f83c23a4640 1 --2- 172.21.15.100:0/3068314813 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562a61590db0 0x562a615911d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.664+0000 7f83c23a4640 1 -- 172.21.15.100:0/3068314813 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a61207210 con 0x562a615917a0 2024-05-10T11:18:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.664+0000 7f83c2ba5640 1 --2- 172.21.15.100:0/3068314813 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562a61590db0 0x562a615911d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:18:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.664+0000 7f83c2ba5640 1 --2- 172.21.15.100:0/3705479713 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f83a40b2920 0x7f83a40b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.664+0000 7f83bb7fe640 1 -- 172.21.15.100:0/3705479713 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f83b0102d30 con 0x7f83bc103c00 2024-05-10T11:18:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.664+0000 7f83c23a4640 1 --2- 172.21.15.100:0/3068314813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a615917a0 0x562a615962a0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f83ac007de0 tx=0x7f83ac00c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da3f30876c1cce06 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.665+0000 7f83b8bfe640 1 -- 172.21.15.100:0/3068314813 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ac015020 con 0x562a615917a0 2024-05-10T11:18:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.665+0000 7f83b8bfe640 1 -- 172.21.15.100:0/3068314813 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f83ac011ac0 con 0x562a615917a0 2024-05-10T11:18:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.665+0000 7f83b8bfe640 1 -- 172.21.15.100:0/3068314813 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ac010a50 con 0x562a615917a0 2024-05-10T11:18:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.665+0000 7f83c73f5740 1 -- 172.21.15.100:0/3068314813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a615917a0 msgr2=0x562a615962a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.665+0000 7f83c73f5740 1 --2- 172.21.15.100:0/3068314813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a615917a0 0x562a615962a0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f83ac007de0 tx=0x7f83ac00c560 comp rx=0 tx=0).stop 2024-05-10T11:18:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.665+0000 7f83c2ba5640 1 --2- 172.21.15.100:0/3705479713 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f83a40b2920 0x7f83a40b4de0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x562a61592000 tx=0x7f83b40046c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.665+0000 7f83c73f5740 1 -- 172.21.15.100:0/3068314813 shutdown_connections 2024-05-10T11:18:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.665+0000 7f83c73f5740 1 --2- 172.21.15.100:0/3068314813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562a61596910 0x562a61598da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.665+0000 7f83c73f5740 1 --2- 172.21.15.100:0/3068314813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a615917a0 0x562a615962a0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.665+0000 7f83c73f5740 1 --2- 172.21.15.100:0/3068314813 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562a61590db0 0x562a615911d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.665+0000 7f83c73f5740 1 -- 172.21.15.100:0/3068314813 >> 172.21.15.100:0/3068314813 conn(0x562a6158c420 msgr2=0x562a6158c820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.666+0000 7f83c73f5740 1 -- 172.21.15.100:0/3068314813 shutdown_connections 2024-05-10T11:18:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.666+0000 7f83c73f5740 1 -- 172.21.15.100:0/3068314813 wait complete. 2024-05-10T11:18:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.666+0000 7f83c73f5740 1 -- start start 2024-05-10T11:18:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.666+0000 7f83c73f5740 1 Processor -- start 2024-05-10T11:18:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.666+0000 7f83c73f5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a61590db0 0x562a6159b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.666+0000 7f83c73f5740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562a615917a0 0x562a6159b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.666+0000 7f83c73f5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562a61596910 0x562a615a25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.666+0000 7f83c73f5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x562a615998c0 con 0x562a61590db0 2024-05-10T11:18:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.666+0000 7f83c73f5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562a614b2370 con 0x562a61596910 2024-05-10T11:18:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.666+0000 7f83c73f5740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x562a61599a40 con 0x562a615917a0 2024-05-10T11:18:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.666+0000 7f83c23a4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a61590db0 0x562a6159b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.666+0000 7f83c73f5740 10 client.0 _ll_register_callbacks cb 0x562a61206a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:18:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.666+0000 7f83c23a4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a61590db0 0x562a6159b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47014/0 (socket says 172.21.15.100:47014) 2024-05-10T11:18:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.666+0000 7f83c23a4640 1 -- 172.21.15.100:0/1825869706 learned_addr learned my addr 172.21.15.100:0/1825869706 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:18:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.666+0000 7f83c1ba3640 1 --2- 172.21.15.100:0/1825869706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562a615917a0 0x562a6159b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.667+0000 7f83c23a4640 1 -- 172.21.15.100:0/1825869706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562a615917a0 msgr2=0x562a6159b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:18:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.667+0000 7f83c23a4640 1 --2- 172.21.15.100:0/1825869706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562a615917a0 0x562a6159b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.667+0000 7f83c23a4640 1 -- 172.21.15.100:0/1825869706 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562a61596910 msgr2=0x562a615a25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:18:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.667+0000 7f83c23a4640 1 --2- 172.21.15.100:0/1825869706 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562a61596910 0x562a615a25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.667+0000 7f83c23a4640 1 -- 172.21.15.100:0/1825869706 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a612d3620 con 0x562a61590db0 2024-05-10T11:18:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.667+0000 7f83c1ba3640 1 --2- 172.21.15.100:0/1825869706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562a615917a0 0x562a6159b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:18:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.667+0000 7f83c23a4640 1 --2- 172.21.15.100:0/1825869706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a61590db0 0x562a6159b030 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f83ac000f80 tx=0x7f83ac00cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41ca155fed5b562f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.667+0000 7f83a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:18:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.667+0000 7f83a1ffb640 1 -- 172.21.15.100:0/1825869706 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ac029060 con 0x562a61590db0 2024-05-10T11:18:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.667+0000 7f83a1ffb640 1 -- 172.21.15.100:0/1825869706 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f83ac010ba0 con 0x562a61590db0 2024-05-10T11:18:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.667+0000 7f83c73f5740 1 -- 172.21.15.100:0/1825869706 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83ac011ac0 con 0x562a61590db0 2024-05-10T11:18:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.667+0000 7f83a1ffb640 1 -- 172.21.15.100:0/1825869706 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f83ac030050 con 0x562a61590db0 2024-05-10T11:18:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.667+0000 7f83c73f5740 10 client.15327 fetch_fsmap learned FSMap version 27 2024-05-10T11:18:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.667+0000 7f83c73f5740 1 -- 172.21.15.100:0/1825869706 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562a615a5590 con 0x562a61590db0 2024-05-10T11:18:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.667+0000 7f83a1ffb640 1 -- 172.21.15.100:0/1825869706 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f83ac011ac0 con 0x562a61590db0 2024-05-10T11:18:48.683 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.667+0000 7f83a1ffb640 1 -- 172.21.15.100:0/1825869706 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f83ac023030 con 0x562a61590db0 2024-05-10T11:18:48.683 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.668+0000 7f83c73f5740 10 client.15327 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T11:18:48.683 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.668+0000 7f83c73f5740 10 client.15327 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:18:48.683 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.668+0000 7f83c73f5740 20 client.15327 populate_metadata read hostname 'smithi100' 2024-05-10T11:18:48.683 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.668+0000 7f83c73f5740 1 --2- 172.21.15.100:0/1825869706 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562a615a6aa0 0x562a615a8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:48.683 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.668+0000 7f83c73f5740 4 client.15327 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:18:48.683 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.668+0000 7f83c73f5740 1 -- 172.21.15.100:0/1825869706 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a615031e0 con 0x562a615a6aa0 2024-05-10T11:18:48.683 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.668+0000 7f83c1ba3640 1 --2- 172.21.15.100:0/1825869706 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562a615a6aa0 0x562a615a8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:48.683 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.668+0000 7f83a1ffb640 10 client.15327 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:18:48.683 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.668+0000 7f83c1ba3640 1 --2- 172.21.15.100:0/1825869706 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562a615a6aa0 0x562a615a8e90 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=671ad9970b1cdca7 server_cookie=194694acec7bdc96 in_seq=0 out_seq=0 2024-05-10T11:18:48.683 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.669+0000 7f83a1ffb640 1 -- 172.21.15.100:0/1825869706 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562a615031e0 con 0x562a615a6aa0 2024-05-10T11:18:48.683 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.669+0000 7f83a1ffb640 10 client.15327 handle_command_reply: tid=0 2024-05-10T11:18:48.683 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.669+0000 7f83c73f5740 1 client.15327 shutdown 2024-05-10T11:18:48.683 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.669+0000 7f83c73f5740 1 -- 172.21.15.100:0/1825869706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a61590db0 msgr2=0x562a6159b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:48.684 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.669+0000 7f83c73f5740 1 --2- 172.21.15.100:0/1825869706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a61590db0 0x562a6159b030 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f83ac000f80 tx=0x7f83ac00cf00 comp rx=0 tx=0).stop 2024-05-10T11:18:48.684 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.669+0000 7f83c73f5740 1 -- 172.21.15.100:0/1825869706 shutdown_connections 2024-05-10T11:18:48.684 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.669+0000 7f83c73f5740 1 --2- 172.21.15.100:0/1825869706 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562a615a6aa0 0x562a615a8e90 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:48.684 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.669+0000 7f83c73f5740 1 --2- 172.21.15.100:0/1825869706 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562a61596910 0x562a615a25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:48.684 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.669+0000 7f83c73f5740 1 --2- 172.21.15.100:0/1825869706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562a615917a0 0x562a6159b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:48.684 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.669+0000 7f83a1ffb640 0 client.15327 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:18:48.684 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.669+0000 7f83c73f5740 1 --2- 172.21.15.100:0/1825869706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a61590db0 0x562a6159b030 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:48.684 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.669+0000 7f83c73f5740 1 -- 172.21.15.100:0/1825869706 >> 172.21.15.100:0/1825869706 conn(0x562a6158c420 msgr2=0x562a6158b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:48.684 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.670+0000 7f83c73f5740 1 -- 172.21.15.100:0/1825869706 shutdown_connections 2024-05-10T11:18:48.684 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.670+0000 7f83c73f5740 1 -- 172.21.15.100:0/1825869706 wait complete. 2024-05-10T11:18:48.684 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.670+0000 7f83c73f5740 20 client.15327 trim_cache size 0 max 16384 2024-05-10T11:18:48.689 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.685+0000 7f83c73f5740 1 Processor -- start 2024-05-10T11:18:48.689 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.685+0000 7f83c73f5740 1 -- start start 2024-05-10T11:18:48.689 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.685+0000 7f83c73f5740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562a61590db0 0x562a6158f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:48.689 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.685+0000 7f83c73f5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562a615917a0 0x562a6158f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:48.689 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.685+0000 7f83c73f5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a61596910 0x562a6159a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.685+0000 7f83c73f5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f83bc1adc80 con 0x562a61596910 2024-05-10T11:18:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.685+0000 7f83c73f5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f83bc1ab550 con 0x562a615917a0 2024-05-10T11:18:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.685+0000 7f83c73f5740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f83bc1a93a0 con 0x562a61590db0 2024-05-10T11:18:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.685+0000 7f83c2ba5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a61596910 0x562a6159a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.685+0000 7f83c2ba5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a61596910 0x562a6159a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47016/0 (socket says 172.21.15.100:47016) 2024-05-10T11:18:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.685+0000 7f83c2ba5640 1 -- 172.21.15.100:0/4245725395 learned_addr learned my addr 172.21.15.100:0/4245725395 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:18:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.685+0000 7f83c1ba3640 1 --2- 172.21.15.100:0/4245725395 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562a615917a0 0x562a6158f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.685+0000 7f83c2ba5640 1 -- 172.21.15.100:0/4245725395 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562a61590db0 msgr2=0x562a6158f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:18:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.685+0000 7f83c2ba5640 1 --2- 172.21.15.100:0/4245725395 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562a61590db0 0x562a6158f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.685+0000 7f83c2ba5640 1 -- 172.21.15.100:0/4245725395 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562a615917a0 msgr2=0x562a6158f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:18:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.685+0000 7f83c2ba5640 1 --2- 172.21.15.100:0/4245725395 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562a615917a0 0x562a6158f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.685+0000 7f83c2ba5640 1 -- 172.21.15.100:0/4245725395 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a6158c250 con 0x562a61596910 2024-05-10T11:18:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.685+0000 7f83c1ba3640 1 --2- 172.21.15.100:0/4245725395 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562a615917a0 0x562a6158f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:18:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.686+0000 7f83c2ba5640 1 --2- 172.21.15.100:0/4245725395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a61596910 0x562a6159a650 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f83b400b760 tx=0x7f83b4014070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4331698db57eb52 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.686+0000 7f83a3fff640 1 -- 172.21.15.100:0/4245725395 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b4026670 con 0x562a61596910 2024-05-10T11:18:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.686+0000 7f83a3fff640 1 -- 172.21.15.100:0/4245725395 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f83b4021070 con 0x562a61596910 2024-05-10T11:18:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.686+0000 7f83a3fff640 1 -- 172.21.15.100:0/4245725395 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b401da80 con 0x562a61596910 2024-05-10T11:18:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.686+0000 7f83c73f5740 1 -- 172.21.15.100:0/4245725395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a61596910 msgr2=0x562a6159a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.686+0000 7f83c73f5740 1 --2- 172.21.15.100:0/4245725395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a61596910 0x562a6159a650 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f83b400b760 tx=0x7f83b4014070 comp rx=0 tx=0).stop 2024-05-10T11:18:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.686+0000 7f83c73f5740 1 -- 172.21.15.100:0/4245725395 shutdown_connections 2024-05-10T11:18:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.686+0000 7f83c73f5740 1 --2- 172.21.15.100:0/4245725395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a61596910 0x562a6159a650 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.686+0000 7f83c73f5740 1 --2- 172.21.15.100:0/4245725395 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562a615917a0 0x562a6158f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.686+0000 7f83c73f5740 1 --2- 172.21.15.100:0/4245725395 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562a61590db0 0x562a6158f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.686+0000 7f83c73f5740 1 -- 172.21.15.100:0/4245725395 >> 172.21.15.100:0/4245725395 conn(0x562a6158c420 msgr2=0x562a61596030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.686+0000 7f83c73f5740 1 -- 172.21.15.100:0/4245725395 shutdown_connections 2024-05-10T11:18:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.686+0000 7f83c73f5740 1 -- 172.21.15.100:0/4245725395 wait complete. 2024-05-10T11:18:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.687+0000 7f83c73f5740 1 -- start start 2024-05-10T11:18:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.687+0000 7f83c73f5740 1 Processor -- start 2024-05-10T11:18:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.687+0000 7f83c73f5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a61590db0 0x562a6159f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.687+0000 7f83c73f5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562a615917a0 0x562a6159f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.687+0000 7f83c73f5740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562a61596910 0x562a615a0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.687+0000 7f83c73f5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f83bc1a9a10 con 0x562a61590db0 2024-05-10T11:18:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.687+0000 7f83c73f5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f83bc1aae80 con 0x562a615917a0 2024-05-10T11:18:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.687+0000 7f83c73f5740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f83bc1aa0e0 con 0x562a61596910 2024-05-10T11:18:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.687+0000 7f83c73f5740 10 client.0 _ll_register_callbacks cb 0x7f83b4005f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:18:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.687+0000 7f83c23a4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a61590db0 0x562a6159f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.687+0000 7f83c23a4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a61590db0 0x562a6159f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47024/0 (socket says 172.21.15.100:47024) 2024-05-10T11:18:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.687+0000 7f83c23a4640 1 -- 172.21.15.100:0/3712894378 learned_addr learned my addr 172.21.15.100:0/3712894378 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:18:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.687+0000 7f83c1ba3640 1 --2- 172.21.15.100:0/3712894378 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562a615917a0 0x562a6159f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.687+0000 7f83c2ba5640 1 --2- 172.21.15.100:0/3712894378 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562a61596910 0x562a615a0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.687+0000 7f83c23a4640 1 -- 172.21.15.100:0/3712894378 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562a61596910 msgr2=0x562a615a0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:18:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.687+0000 7f83c23a4640 1 --2- 172.21.15.100:0/3712894378 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562a61596910 0x562a615a0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.687+0000 7f83c23a4640 1 -- 172.21.15.100:0/3712894378 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562a615917a0 msgr2=0x562a6159f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:18:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.687+0000 7f83c23a4640 1 --2- 172.21.15.100:0/3712894378 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562a615917a0 0x562a6159f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.687+0000 7f83c23a4640 1 -- 172.21.15.100:0/3712894378 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a615a0af0 con 0x562a61590db0 2024-05-10T11:18:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.687+0000 7f83c2ba5640 1 --2- 172.21.15.100:0/3712894378 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562a61596910 0x562a615a0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:18:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.687+0000 7f83c1ba3640 1 --2- 172.21.15.100:0/3712894378 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562a615917a0 0x562a6159f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:18:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.687+0000 7f83c23a4640 1 --2- 172.21.15.100:0/3712894378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a61590db0 0x562a6159f020 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f83ac002230 tx=0x7f83ac010640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f22a3445990f98b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:18:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.687+0000 7f83a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:18:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.688+0000 7f83a1ffb640 1 -- 172.21.15.100:0/3712894378 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ac015050 con 0x562a61590db0 2024-05-10T11:18:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.688+0000 7f83c73f5740 1 -- 172.21.15.100:0/3712894378 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83b4021070 con 0x562a61590db0 2024-05-10T11:18:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.688+0000 7f83a1ffb640 1 -- 172.21.15.100:0/3712894378 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f83ac0109a0 con 0x562a61590db0 2024-05-10T11:18:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.688+0000 7f83a1ffb640 1 -- 172.21.15.100:0/3712894378 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f83ac03d050 con 0x562a61590db0 2024-05-10T11:18:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.688+0000 7f83c73f5740 10 client.15333 fetch_fsmap learned FSMap version 27 2024-05-10T11:18:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.688+0000 7f83c73f5740 1 -- 172.21.15.100:0/3712894378 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562a615a1d70 con 0x562a61590db0 2024-05-10T11:18:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.688+0000 7f83a1ffb640 1 -- 172.21.15.100:0/3712894378 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f83b4021070 con 0x562a61590db0 2024-05-10T11:18:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.688+0000 7f83a1ffb640 1 -- 172.21.15.100:0/3712894378 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f83ac023040 con 0x562a61590db0 2024-05-10T11:18:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.688+0000 7f83c73f5740 10 client.15333 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T11:18:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.688+0000 7f83c73f5740 10 client.15333 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:18:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.688+0000 7f83c73f5740 20 client.15333 populate_metadata read hostname 'smithi100' 2024-05-10T11:18:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.688+0000 7f83c73f5740 1 --2- 172.21.15.100:0/3712894378 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562a616118b0 0x562a61613cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:18:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.688+0000 7f83c73f5740 4 client.15333 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:18:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.688+0000 7f83c73f5740 1 -- 172.21.15.100:0/3712894378 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562a614fd050 con 0x562a616118b0 2024-05-10T11:18:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.688+0000 7f83c1ba3640 1 --2- 172.21.15.100:0/3712894378 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562a616118b0 0x562a61613cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:18:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.689+0000 7f83a1ffb640 10 client.15333 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:18:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.689+0000 7f83c1ba3640 1 --2- 172.21.15.100:0/3712894378 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562a616118b0 0x562a61613cd0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da982be88e781b6b server_cookie=c3d84d97cc3e4dab in_seq=0 out_seq=0 2024-05-10T11:18:48.701 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.701+0000 7f83a1ffb640 1 -- 172.21.15.100:0/3712894378 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562a614fd050 con 0x562a616118b0 2024-05-10T11:18:48.702 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.701+0000 7f83a1ffb640 10 client.15333 handle_command_reply: tid=0 2024-05-10T11:18:48.702 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.701+0000 7f83c73f5740 1 client.15333 shutdown 2024-05-10T11:18:48.702 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.701+0000 7f83c73f5740 1 -- 172.21.15.100:0/3712894378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a61590db0 msgr2=0x562a6159f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:48.702 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.701+0000 7f83c73f5740 1 --2- 172.21.15.100:0/3712894378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a61590db0 0x562a6159f020 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f83ac002230 tx=0x7f83ac010640 comp rx=0 tx=0).stop 2024-05-10T11:18:48.702 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.701+0000 7f83c73f5740 1 -- 172.21.15.100:0/3712894378 shutdown_connections 2024-05-10T11:18:48.702 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.701+0000 7f83c73f5740 1 --2- 172.21.15.100:0/3712894378 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562a616118b0 0x562a61613cd0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:48.702 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.701+0000 7f83c73f5740 1 --2- 172.21.15.100:0/3712894378 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562a61596910 0x562a615a0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:48.702 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.701+0000 7f83c73f5740 1 --2- 172.21.15.100:0/3712894378 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562a615917a0 0x562a6159f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:48.702 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.701+0000 7f83a1ffb640 0 client.15333 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:18:48.702 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.701+0000 7f83c73f5740 1 --2- 172.21.15.100:0/3712894378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a61590db0 0x562a6159f020 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:48.702 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.701+0000 7f83c73f5740 1 -- 172.21.15.100:0/3712894378 >> 172.21.15.100:0/3712894378 conn(0x562a6158c420 msgr2=0x562a6158f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:48.702 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.701+0000 7f83c73f5740 1 -- 172.21.15.100:0/3712894378 shutdown_connections 2024-05-10T11:18:48.703 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.702+0000 7f83c73f5740 1 -- 172.21.15.100:0/3712894378 wait complete. 2024-05-10T11:18:48.703 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.702+0000 7f83c73f5740 20 client.15333 trim_cache size 0 max 16384 2024-05-10T11:18:48.703 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:18:48.703 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:18:48.703 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:18:48.703 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:18:48.704 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.702+0000 7f83c4608640 1 -- 172.21.15.100:0/3705479713 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f83a40b2920 msgr2=0x7f83a40b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:48.704 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.702+0000 7f83c4608640 1 --2- 172.21.15.100:0/3705479713 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f83a40b2920 0x7f83a40b4de0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x562a61592000 tx=0x7f83b40046c0 comp rx=0 tx=0).stop 2024-05-10T11:18:48.704 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.702+0000 7f83c4608640 1 -- 172.21.15.100:0/3705479713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83bc103c00 msgr2=0x7f83bc1990a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:18:48.704 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.702+0000 7f83c4608640 1 --2- 172.21.15.100:0/3705479713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83bc103c00 0x7f83bc1990a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f83b0009230 tx=0x7f83b00042f0 comp rx=0 tx=0).stop 2024-05-10T11:18:48.704 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.702+0000 7f83c4608640 1 -- 172.21.15.100:0/3705479713 shutdown_connections 2024-05-10T11:18:48.704 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.702+0000 7f83c4608640 1 --2- 172.21.15.100:0/3705479713 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f83a40b2920 0x7f83a40b4de0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:48.704 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.702+0000 7f83c4608640 1 --2- 172.21.15.100:0/3705479713 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83bc1045c0 0x7f83bc1a0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:48.704 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.702+0000 7f83c4608640 1 --2- 172.21.15.100:0/3705479713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83bc103c00 0x7f83bc1990a0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:48.704 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.702+0000 7f83c4608640 1 --2- 172.21.15.100:0/3705479713 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83bc102a10 0x7f83bc198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:18:48.704 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.702+0000 7f83c4608640 1 -- 172.21.15.100:0/3705479713 >> 172.21.15.100:0/3705479713 conn(0x7f83bc0fdfa0 msgr2=0x7f83bc106fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:18:48.705 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.702+0000 7f83c4608640 1 -- 172.21.15.100:0/3705479713 shutdown_connections 2024-05-10T11:18:48.705 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:18:48.702+0000 7f83c4608640 1 -- 172.21.15.100:0/3705479713 wait complete. 2024-05-10T11:18:50.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:18:49 smithi178 ceph-mon[33359]: pgmap v433: 129 pgs: 129 active+clean; 660 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-05-10T11:18:50.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:18:49 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:18:50.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:18:49 smithi188 ceph-mon[34075]: pgmap v433: 129 pgs: 129 active+clean; 660 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-05-10T11:18:50.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:18:49 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:18:50.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:18:49 smithi100 ceph-mon[30296]: pgmap v433: 129 pgs: 129 active+clean; 660 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-05-10T11:18:50.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:18:49 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:18:52.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:18:51 smithi178 ceph-mon[33359]: pgmap v434: 129 pgs: 129 active+clean; 660 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-05-10T11:18:52.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:18:51 smithi188 ceph-mon[34075]: pgmap v434: 129 pgs: 129 active+clean; 660 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-05-10T11:18:52.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:18:51 smithi100 ceph-mon[30296]: pgmap v434: 129 pgs: 129 active+clean; 660 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-05-10T11:18:54.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:18:53 smithi178 ceph-mon[33359]: pgmap v435: 129 pgs: 129 active+clean; 661 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.9 KiB/s wr, 11 op/s 2024-05-10T11:18:54.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:18:53 smithi188 ceph-mon[34075]: pgmap v435: 129 pgs: 129 active+clean; 661 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.9 KiB/s wr, 11 op/s 2024-05-10T11:18:54.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:18:53 smithi100 ceph-mon[30296]: pgmap v435: 129 pgs: 129 active+clean; 661 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.9 KiB/s wr, 11 op/s 2024-05-10T11:18:56.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:18:55 smithi178 ceph-mon[33359]: pgmap v436: 129 pgs: 129 active+clean; 661 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-05-10T11:18:56.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:18:55 smithi188 ceph-mon[34075]: pgmap v436: 129 pgs: 129 active+clean; 661 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-05-10T11:18:56.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:18:55 smithi100 ceph-mon[30296]: pgmap v436: 129 pgs: 129 active+clean; 661 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-05-10T11:18:58.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:18:57 smithi178 ceph-mon[33359]: pgmap v437: 129 pgs: 129 active+clean; 661 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-05-10T11:18:58.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:18:57 smithi188 ceph-mon[34075]: pgmap v437: 129 pgs: 129 active+clean; 661 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-05-10T11:18:58.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:18:57 smithi100 ceph-mon[30296]: pgmap v437: 129 pgs: 129 active+clean; 661 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-05-10T11:19:00.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:18:59 smithi178 ceph-mon[33359]: pgmap v438: 129 pgs: 129 active+clean; 661 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-05-10T11:19:00.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:18:59 smithi188 ceph-mon[34075]: pgmap v438: 129 pgs: 129 active+clean; 661 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-05-10T11:19:00.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:18:59 smithi100 ceph-mon[30296]: pgmap v438: 129 pgs: 129 active+clean; 661 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-05-10T11:19:01.802 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.800+0000 7f2052f32640 1 -- 172.21.15.100:0/3933742002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f204c1086f0 msgr2=0x7f204c10ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:01.802 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.800+0000 7f2052f32640 1 --2- 172.21.15.100:0/3933742002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f204c1086f0 0x7f204c10ab40 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f2044009ca0 tx=0x7f204402a8c0 comp rx=0 tx=0).stop 2024-05-10T11:19:01.802 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.800+0000 7f2052f32640 1 -- 172.21.15.100:0/3933742002 shutdown_connections 2024-05-10T11:19:01.802 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.800+0000 7f2052f32640 1 --2- 172.21.15.100:0/3933742002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f204c1086f0 0x7f204c10ab40 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:01.802 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.800+0000 7f2052f32640 1 --2- 172.21.15.100:0/3933742002 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f204c105dc0 0x7f204c1081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:01.802 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.800+0000 7f2052f32640 1 --2- 172.21.15.100:0/3933742002 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f204c103490 0x7f204c105880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:01.802 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.800+0000 7f2052f32640 1 -- 172.21.15.100:0/3933742002 >> 172.21.15.100:0/3933742002 conn(0x7f204c0fce80 msgr2=0x7f204c0ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.800+0000 7f2052f32640 1 -- 172.21.15.100:0/3933742002 shutdown_connections 2024-05-10T11:19:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.800+0000 7f2052f32640 1 -- 172.21.15.100:0/3933742002 wait complete. 2024-05-10T11:19:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.801+0000 7f2052f32640 1 Processor -- start 2024-05-10T11:19:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.801+0000 7f2052f32640 1 -- start start 2024-05-10T11:19:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.801+0000 7f2052f32640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f204c103490 0x7f204c198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.801+0000 7f2052f32640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f204c105dc0 0x7f204c198fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.801+0000 7f2052f32640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f204c1086f0 0x7f204c1a0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.801+0000 7f2052f32640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f204c10e460 con 0x7f204c105dc0 2024-05-10T11:19:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.801+0000 7f2052f32640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f204c10e2e0 con 0x7f204c103490 2024-05-10T11:19:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.801+0000 7f2052f32640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f204c10e5e0 con 0x7f204c1086f0 2024-05-10T11:19:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.801+0000 7f204bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f204c105dc0 0x7f204c198fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.801+0000 7f204bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f204c105dc0 0x7f204c198fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48678/0 (socket says 172.21.15.100:48678) 2024-05-10T11:19:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.801+0000 7f204bfff640 1 -- 172.21.15.100:0/3787430776 learned_addr learned my addr 172.21.15.100:0/3787430776 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.801+0000 7f20514cf640 1 --2- 172.21.15.100:0/3787430776 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f204c1086f0 0x7f204c1a0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.801+0000 7f2050cce640 1 --2- 172.21.15.100:0/3787430776 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f204c103490 0x7f204c198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.801+0000 7f204bfff640 1 -- 172.21.15.100:0/3787430776 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f204c1086f0 msgr2=0x7f204c1a0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.801+0000 7f204bfff640 1 --2- 172.21.15.100:0/3787430776 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f204c1086f0 0x7f204c1a0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.801+0000 7f204bfff640 1 -- 172.21.15.100:0/3787430776 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f204c103490 msgr2=0x7f204c198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.801+0000 7f204bfff640 1 --2- 172.21.15.100:0/3787430776 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f204c103490 0x7f204c198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.801+0000 7f204bfff640 1 -- 172.21.15.100:0/3787430776 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f204c1a06a0 con 0x7f204c105dc0 2024-05-10T11:19:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.802+0000 7f204bfff640 1 --2- 172.21.15.100:0/3787430776 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f204c105dc0 0x7f204c198fc0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f2038005fd0 tx=0x7f203800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.802+0000 7f2049ffb640 1 -- 172.21.15.100:0/3787430776 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2038017070 con 0x7f204c105dc0 2024-05-10T11:19:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.802+0000 7f2049ffb640 1 -- 172.21.15.100:0/3787430776 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2038009c30 con 0x7f204c105dc0 2024-05-10T11:19:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.802+0000 7f2052f32640 1 -- 172.21.15.100:0/3787430776 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f204c1a0990 con 0x7f204c105dc0 2024-05-10T11:19:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.802+0000 7f2049ffb640 1 -- 172.21.15.100:0/3787430776 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2038003c70 con 0x7f204c105dc0 2024-05-10T11:19:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.802+0000 7f2052f32640 1 -- 172.21.15.100:0/3787430776 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f204c1a0ee0 con 0x7f204c105dc0 2024-05-10T11:19:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.804+0000 7f2049ffb640 1 -- 172.21.15.100:0/3787430776 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f2038003e10 con 0x7f204c105dc0 2024-05-10T11:19:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.805+0000 7f2055c50740 1 Processor -- start 2024-05-10T11:19:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.805+0000 7f2055c50740 1 -- start start 2024-05-10T11:19:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.805+0000 7f2055c50740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56194980adb0 0x56194980b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.805+0000 7f2055c50740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56194980b7a0 0x5619498102a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.805+0000 7f2055c50740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561949810910 0x561949812da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.805+0000 7f2055c50740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56194976f5d0 con 0x56194980b7a0 2024-05-10T11:19:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.805+0000 7f2055c50740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561949728890 con 0x56194980adb0 2024-05-10T11:19:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.805+0000 7f2055c50740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56194976b590 con 0x561949810910 2024-05-10T11:19:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.805+0000 7f2050cce640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56194980b7a0 0x5619498102a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.805+0000 7f2049ffb640 1 --2- 172.21.15.100:0/3787430776 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f203c0b28a0 0x7f203c0b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.805+0000 7f2050cce640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56194980b7a0 0x5619498102a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48692/0 (socket says 172.21.15.100:48692) 2024-05-10T11:19:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.805+0000 7f2050cce640 1 -- 172.21.15.100:0/4085650965 learned_addr learned my addr 172.21.15.100:0/4085650965 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.805+0000 7f20514cf640 1 --2- 172.21.15.100:0/3787430776 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f203c0b28a0 0x7f203c0b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.805+0000 7f2049ffb640 1 -- 172.21.15.100:0/3787430776 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f20380e2160 con 0x7f204c105dc0 2024-05-10T11:19:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.805+0000 7f204bfff640 1 --2- 172.21.15.100:0/4085650965 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561949810910 0x561949812da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.805+0000 7f20514cf640 1 --2- 172.21.15.100:0/4085650965 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56194980adb0 0x56194980b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.805+0000 7f2050cce640 1 -- 172.21.15.100:0/4085650965 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561949810910 msgr2=0x561949812da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.805+0000 7f2050cce640 1 --2- 172.21.15.100:0/4085650965 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561949810910 0x561949812da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.805+0000 7f2050cce640 1 -- 172.21.15.100:0/4085650965 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56194980adb0 msgr2=0x56194980b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.805+0000 7f2050cce640 1 --2- 172.21.15.100:0/4085650965 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56194980adb0 0x56194980b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.805+0000 7f2050cce640 1 -- 172.21.15.100:0/4085650965 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561949481210 con 0x56194980b7a0 2024-05-10T11:19:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.805+0000 7f20514cf640 1 --2- 172.21.15.100:0/4085650965 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56194980adb0 0x56194980b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:19:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.806+0000 7f2050cce640 1 --2- 172.21.15.100:0/4085650965 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56194980b7a0 0x5619498102a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f204000ac10 tx=0x7f204000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f23dcb73a50c9ac server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.806+0000 7f20514cf640 1 --2- 172.21.15.100:0/3787430776 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f203c0b28a0 0x7f203c0b4d60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2044006010 tx=0x7f204403f040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.806+0000 7f2036ffd640 1 -- 172.21.15.100:0/4085650965 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204001d070 con 0x56194980b7a0 2024-05-10T11:19:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.806+0000 7f2036ffd640 1 -- 172.21.15.100:0/4085650965 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2040018ad0 con 0x56194980b7a0 2024-05-10T11:19:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.806+0000 7f2036ffd640 1 -- 172.21.15.100:0/4085650965 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2040010980 con 0x56194980b7a0 2024-05-10T11:19:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.807+0000 7f2055c50740 1 -- 172.21.15.100:0/4085650965 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56194980b7a0 msgr2=0x5619498102a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.807+0000 7f2055c50740 1 --2- 172.21.15.100:0/4085650965 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56194980b7a0 0x5619498102a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f204000ac10 tx=0x7f204000f790 comp rx=0 tx=0).stop 2024-05-10T11:19:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.807+0000 7f2055c50740 1 -- 172.21.15.100:0/4085650965 shutdown_connections 2024-05-10T11:19:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.807+0000 7f2055c50740 1 --2- 172.21.15.100:0/4085650965 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561949810910 0x561949812da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.807+0000 7f2055c50740 1 --2- 172.21.15.100:0/4085650965 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56194980b7a0 0x5619498102a0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.807+0000 7f2055c50740 1 --2- 172.21.15.100:0/4085650965 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56194980adb0 0x56194980b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.807+0000 7f2055c50740 1 -- 172.21.15.100:0/4085650965 >> 172.21.15.100:0/4085650965 conn(0x561949806420 msgr2=0x561949806820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.807+0000 7f2055c50740 1 -- 172.21.15.100:0/4085650965 shutdown_connections 2024-05-10T11:19:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.807+0000 7f2055c50740 1 -- 172.21.15.100:0/4085650965 wait complete. 2024-05-10T11:19:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.807+0000 7f2055c50740 1 -- start start 2024-05-10T11:19:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.807+0000 7f2055c50740 1 Processor -- start 2024-05-10T11:19:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.807+0000 7f2055c50740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56194980adb0 0x561949814f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.807+0000 7f2055c50740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56194980b7a0 0x5619498154a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.808+0000 7f2055c50740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561949810910 0x56194981c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.808+0000 7f2055c50740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5619498138c0 con 0x56194980adb0 2024-05-10T11:19:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.808+0000 7f2055c50740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56194972c370 con 0x561949810910 2024-05-10T11:19:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.808+0000 7f2055c50740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x561949813a40 con 0x56194980b7a0 2024-05-10T11:19:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.808+0000 7f2055c50740 10 client.0 _ll_register_callbacks cb 0x561949480a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:19:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.808+0000 7f2050cce640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56194980adb0 0x561949814f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.808+0000 7f2050cce640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56194980adb0 0x561949814f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48704/0 (socket says 172.21.15.100:48704) 2024-05-10T11:19:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.808+0000 7f2050cce640 1 -- 172.21.15.100:0/2987741137 learned_addr learned my addr 172.21.15.100:0/2987741137 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.808+0000 7f204bfff640 1 --2- 172.21.15.100:0/2987741137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56194980b7a0 0x5619498154a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.808+0000 7f20514cf640 1 --2- 172.21.15.100:0/2987741137 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561949810910 0x56194981c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.808+0000 7f2050cce640 1 -- 172.21.15.100:0/2987741137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56194980b7a0 msgr2=0x5619498154a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.808+0000 7f2050cce640 1 --2- 172.21.15.100:0/2987741137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56194980b7a0 0x5619498154a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.808+0000 7f2050cce640 1 -- 172.21.15.100:0/2987741137 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561949810910 msgr2=0x56194981c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.808+0000 7f2050cce640 1 --2- 172.21.15.100:0/2987741137 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561949810910 0x56194981c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.808+0000 7f2050cce640 1 -- 172.21.15.100:0/2987741137 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56194954d620 con 0x56194980adb0 2024-05-10T11:19:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.808+0000 7f2050cce640 1 --2- 172.21.15.100:0/2987741137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56194980adb0 0x561949814f60 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f20400105d0 tx=0x7f2040018670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=558e352e8053894c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.808+0000 7f202bfff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:19:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.808+0000 7f202bfff640 1 -- 172.21.15.100:0/2987741137 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2040027020 con 0x56194980adb0 2024-05-10T11:19:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.808+0000 7f202bfff640 1 -- 172.21.15.100:0/2987741137 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f2040005040 con 0x56194980adb0 2024-05-10T11:19:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.808+0000 7f2055c50740 1 -- 172.21.15.100:0/2987741137 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2040018ad0 con 0x56194980adb0 2024-05-10T11:19:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.808+0000 7f202bfff640 1 -- 172.21.15.100:0/2987741137 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2040032050 con 0x56194980adb0 2024-05-10T11:19:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.809+0000 7f2055c50740 10 client.15345 fetch_fsmap learned FSMap version 27 2024-05-10T11:19:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.809+0000 7f2055c50740 1 -- 172.21.15.100:0/2987741137 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56194981f430 con 0x56194980adb0 2024-05-10T11:19:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.809+0000 7f202bfff640 1 -- 172.21.15.100:0/2987741137 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2040018ad0 con 0x56194980adb0 2024-05-10T11:19:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.809+0000 7f202bfff640 1 -- 172.21.15.100:0/2987741137 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f20400046c0 con 0x56194980adb0 2024-05-10T11:19:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.809+0000 7f2055c50740 10 client.15345 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T11:19:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.809+0000 7f2055c50740 10 client.15345 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:19:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.809+0000 7f2055c50740 20 client.15345 populate_metadata read hostname 'smithi100' 2024-05-10T11:19:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.809+0000 7f2055c50740 1 --2- 172.21.15.100:0/2987741137 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x561949820830 0x561949822c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.809+0000 7f2055c50740 4 client.15345 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:19:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.809+0000 7f2055c50740 1 -- 172.21.15.100:0/2987741137 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56194977d1e0 con 0x561949820830 2024-05-10T11:19:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.809+0000 7f204bfff640 1 --2- 172.21.15.100:0/2987741137 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x561949820830 0x561949822c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:01.817 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.810+0000 7f202bfff640 10 client.15345 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:19:01.817 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.810+0000 7f204bfff640 1 --2- 172.21.15.100:0/2987741137 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x561949820830 0x561949822c20 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94f30265e87b47d0 server_cookie=9559b19011cc685d in_seq=0 out_seq=0 2024-05-10T11:19:01.817 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.810+0000 7f202bfff640 1 -- 172.21.15.100:0/2987741137 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56194977d1e0 con 0x561949820830 2024-05-10T11:19:01.817 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.810+0000 7f202bfff640 10 client.15345 handle_command_reply: tid=0 2024-05-10T11:19:01.817 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.810+0000 7f2055c50740 1 client.15345 shutdown 2024-05-10T11:19:01.817 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.810+0000 7f2055c50740 1 -- 172.21.15.100:0/2987741137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56194980adb0 msgr2=0x561949814f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:01.817 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.810+0000 7f2055c50740 1 --2- 172.21.15.100:0/2987741137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56194980adb0 0x561949814f60 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f20400105d0 tx=0x7f2040018670 comp rx=0 tx=0).stop 2024-05-10T11:19:01.817 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.811+0000 7f2055c50740 1 -- 172.21.15.100:0/2987741137 shutdown_connections 2024-05-10T11:19:01.817 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.811+0000 7f2055c50740 1 --2- 172.21.15.100:0/2987741137 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x561949820830 0x561949822c20 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:01.817 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.811+0000 7f2055c50740 1 --2- 172.21.15.100:0/2987741137 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561949810910 0x56194981c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:01.817 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.811+0000 7f202bfff640 0 client.15345 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:19:01.817 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.811+0000 7f2055c50740 1 --2- 172.21.15.100:0/2987741137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56194980b7a0 0x5619498154a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:01.817 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.811+0000 7f2055c50740 1 --2- 172.21.15.100:0/2987741137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56194980adb0 0x561949814f60 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:01.817 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.811+0000 7f2055c50740 1 -- 172.21.15.100:0/2987741137 >> 172.21.15.100:0/2987741137 conn(0x561949806420 msgr2=0x561949805580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:01.817 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.811+0000 7f2055c50740 1 -- 172.21.15.100:0/2987741137 shutdown_connections 2024-05-10T11:19:01.818 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.811+0000 7f2055c50740 1 -- 172.21.15.100:0/2987741137 wait complete. 2024-05-10T11:19:01.818 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.811+0000 7f2055c50740 20 client.15345 trim_cache size 0 max 16384 2024-05-10T11:19:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.826+0000 7f2055c50740 1 Processor -- start 2024-05-10T11:19:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.826+0000 7f2055c50740 1 -- start start 2024-05-10T11:19:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.826+0000 7f2055c50740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56194980adb0 0x561949809300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.826+0000 7f2055c50740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56194980b7a0 0x56194981c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.826+0000 7f2055c50740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561949810910 0x561949809840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.826+0000 7f2055c50740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f204c1adb70 con 0x56194980adb0 2024-05-10T11:19:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.826+0000 7f2055c50740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f204c1ab440 con 0x561949810910 2024-05-10T11:19:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.826+0000 7f2055c50740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f204c1a9290 con 0x56194980b7a0 2024-05-10T11:19:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.826+0000 7f2050cce640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56194980adb0 0x561949809300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.826+0000 7f2050cce640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56194980adb0 0x561949809300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48710/0 (socket says 172.21.15.100:48710) 2024-05-10T11:19:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.826+0000 7f2050cce640 1 -- 172.21.15.100:0/314781100 learned_addr learned my addr 172.21.15.100:0/314781100 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.826+0000 7f204bfff640 1 --2- 172.21.15.100:0/314781100 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56194980b7a0 0x56194981c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.827+0000 7f2050cce640 1 -- 172.21.15.100:0/314781100 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56194980b7a0 msgr2=0x56194981c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.827+0000 7f2050cce640 1 --2- 172.21.15.100:0/314781100 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56194980b7a0 0x56194981c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.827+0000 7f2050cce640 1 -- 172.21.15.100:0/314781100 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561949810910 msgr2=0x561949809840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.827+0000 7f2050cce640 1 --2- 172.21.15.100:0/314781100 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561949810910 0x561949809840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.827+0000 7f2050cce640 1 -- 172.21.15.100:0/314781100 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561949806250 con 0x56194980adb0 2024-05-10T11:19:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.827+0000 7f2050cce640 1 --2- 172.21.15.100:0/314781100 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56194980adb0 0x561949809300 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f20400058c0 tx=0x7f204000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50438b6a43df2537 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.827+0000 7f20367fc640 1 -- 172.21.15.100:0/314781100 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2040027040 con 0x56194980adb0 2024-05-10T11:19:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.827+0000 7f20367fc640 1 -- 172.21.15.100:0/314781100 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2040005cc0 con 0x56194980adb0 2024-05-10T11:19:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.827+0000 7f20367fc640 1 -- 172.21.15.100:0/314781100 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204001d040 con 0x56194980adb0 2024-05-10T11:19:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.827+0000 7f2055c50740 1 -- 172.21.15.100:0/314781100 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56194980adb0 msgr2=0x561949809300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.827+0000 7f2055c50740 1 --2- 172.21.15.100:0/314781100 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56194980adb0 0x561949809300 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f20400058c0 tx=0x7f204000a340 comp rx=0 tx=0).stop 2024-05-10T11:19:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.828+0000 7f2055c50740 1 -- 172.21.15.100:0/314781100 shutdown_connections 2024-05-10T11:19:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.828+0000 7f2055c50740 1 --2- 172.21.15.100:0/314781100 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561949810910 0x561949809840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.828+0000 7f2055c50740 1 --2- 172.21.15.100:0/314781100 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56194980b7a0 0x56194981c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.828+0000 7f2055c50740 1 --2- 172.21.15.100:0/314781100 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56194980adb0 0x561949809300 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.828+0000 7f2055c50740 1 -- 172.21.15.100:0/314781100 >> 172.21.15.100:0/314781100 conn(0x561949806420 msgr2=0x561949806800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.828+0000 7f2055c50740 1 -- 172.21.15.100:0/314781100 shutdown_connections 2024-05-10T11:19:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.828+0000 7f2055c50740 1 -- 172.21.15.100:0/314781100 wait complete. 2024-05-10T11:19:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.828+0000 7f2055c50740 1 -- start start 2024-05-10T11:19:01.830 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.828+0000 7f2055c50740 1 Processor -- start 2024-05-10T11:19:01.835 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.828+0000 7f2055c50740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56194980adb0 0x561949819500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:01.835 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.828+0000 7f2055c50740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56194980b7a0 0x561949819a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:01.835 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.828+0000 7f2055c50740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561949810910 0x56194981aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:01.835 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.828+0000 7f2055c50740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f204c1a9900 con 0x56194980b7a0 2024-05-10T11:19:01.835 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.828+0000 7f2055c50740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f204c1aad70 con 0x561949810910 2024-05-10T11:19:01.835 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.828+0000 7f2055c50740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f204c1a9fd0 con 0x56194980adb0 2024-05-10T11:19:01.835 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.828+0000 7f2055c50740 10 client.0 _ll_register_callbacks cb 0x7f204001b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:19:01.835 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.828+0000 7f204bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56194980b7a0 0x561949819a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:01.836 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.828+0000 7f204bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56194980b7a0 0x561949819a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48722/0 (socket says 172.21.15.100:48722) 2024-05-10T11:19:01.836 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.828+0000 7f204bfff640 1 -- 172.21.15.100:0/3388347928 learned_addr learned my addr 172.21.15.100:0/3388347928 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:01.836 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.829+0000 7f2050cce640 1 --2- 172.21.15.100:0/3388347928 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56194980adb0 0x561949819500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:01.836 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.829+0000 7f204bfff640 1 -- 172.21.15.100:0/3388347928 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56194980adb0 msgr2=0x561949819500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:01.836 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.829+0000 7f204bfff640 1 --2- 172.21.15.100:0/3388347928 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56194980adb0 0x561949819500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:01.836 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.829+0000 7f204bfff640 1 -- 172.21.15.100:0/3388347928 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561949810910 msgr2=0x56194981aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:01.836 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.829+0000 7f204bfff640 1 --2- 172.21.15.100:0/3388347928 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561949810910 0x56194981aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:01.836 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.829+0000 7f204bfff640 1 -- 172.21.15.100:0/3388347928 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56194981b040 con 0x56194980b7a0 2024-05-10T11:19:01.836 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.829+0000 7f2050cce640 1 --2- 172.21.15.100:0/3388347928 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56194980adb0 0x561949819500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:19:01.836 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.829+0000 7f204bfff640 1 --2- 172.21.15.100:0/3388347928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56194980b7a0 0x561949819a40 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f20380df900 tx=0x7f2038005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54b865e2d971fa1e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:01.836 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.829+0000 7f202bfff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:19:01.837 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.829+0000 7f202bfff640 1 -- 172.21.15.100:0/3388347928 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20380b2280 con 0x56194980b7a0 2024-05-10T11:19:01.837 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.829+0000 7f202bfff640 1 -- 172.21.15.100:0/3388347928 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f20380b0b50 con 0x56194980b7a0 2024-05-10T11:19:01.837 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.829+0000 7f2055c50740 1 -- 172.21.15.100:0/3388347928 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2040005cc0 con 0x56194980b7a0 2024-05-10T11:19:01.837 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.829+0000 7f202bfff640 1 -- 172.21.15.100:0/3388347928 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f20380a9420 con 0x56194980b7a0 2024-05-10T11:19:01.837 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.829+0000 7f2055c50740 10 client.15351 fetch_fsmap learned FSMap version 27 2024-05-10T11:19:01.837 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.829+0000 7f2055c50740 1 -- 172.21.15.100:0/3388347928 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56194981c3c0 con 0x56194980b7a0 2024-05-10T11:19:01.837 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.829+0000 7f202bfff640 1 -- 172.21.15.100:0/3388347928 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2040005cc0 con 0x56194980b7a0 2024-05-10T11:19:01.837 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.829+0000 7f202bfff640 1 -- 172.21.15.100:0/3388347928 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f20380a8680 con 0x56194980b7a0 2024-05-10T11:19:01.837 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.829+0000 7f2055c50740 10 client.15351 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T11:19:01.837 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.829+0000 7f2055c50740 10 client.15351 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:19:01.837 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.829+0000 7f2055c50740 20 client.15351 populate_metadata read hostname 'smithi100' 2024-05-10T11:19:01.837 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.829+0000 7f2055c50740 1 --2- 172.21.15.100:0/3388347928 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56194988b450 0x56194988d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:01.837 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.829+0000 7f2055c50740 4 client.15351 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:19:01.837 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.829+0000 7f2055c50740 1 -- 172.21.15.100:0/3388347928 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561949777050 con 0x56194988b450 2024-05-10T11:19:01.837 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.830+0000 7f2050cce640 1 --2- 172.21.15.100:0/3388347928 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56194988b450 0x56194988d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.830+0000 7f202bfff640 10 client.15351 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:19:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.830+0000 7f2050cce640 1 --2- 172.21.15.100:0/3388347928 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56194988b450 0x56194988d870 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b77eeb92fd3f1ae7 server_cookie=cda0b51a2d240aed in_seq=0 out_seq=0 2024-05-10T11:19:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.834+0000 7f202bfff640 1 -- 172.21.15.100:0/3388347928 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561949777050 con 0x56194988b450 2024-05-10T11:19:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.834+0000 7f202bfff640 10 client.15351 handle_command_reply: tid=0 2024-05-10T11:19:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.834+0000 7f2055c50740 1 client.15351 shutdown 2024-05-10T11:19:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.834+0000 7f2055c50740 1 -- 172.21.15.100:0/3388347928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56194980b7a0 msgr2=0x561949819a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.834+0000 7f2055c50740 1 --2- 172.21.15.100:0/3388347928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56194980b7a0 0x561949819a40 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f20380df900 tx=0x7f2038005db0 comp rx=0 tx=0).stop 2024-05-10T11:19:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.834+0000 7f2055c50740 1 -- 172.21.15.100:0/3388347928 shutdown_connections 2024-05-10T11:19:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.834+0000 7f2055c50740 1 --2- 172.21.15.100:0/3388347928 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56194988b450 0x56194988d870 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.834+0000 7f2055c50740 1 --2- 172.21.15.100:0/3388347928 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561949810910 0x56194981aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.834+0000 7f2055c50740 1 --2- 172.21.15.100:0/3388347928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56194980b7a0 0x561949819a40 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.834+0000 7f202bfff640 0 client.15351 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:19:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.834+0000 7f2055c50740 1 --2- 172.21.15.100:0/3388347928 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56194980adb0 0x561949819500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.834+0000 7f2055c50740 1 -- 172.21.15.100:0/3388347928 >> 172.21.15.100:0/3388347928 conn(0x561949806420 msgr2=0x561949809300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.834+0000 7f2055c50740 1 -- 172.21.15.100:0/3388347928 shutdown_connections 2024-05-10T11:19:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.834+0000 7f2055c50740 1 -- 172.21.15.100:0/3388347928 wait complete. 2024-05-10T11:19:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.834+0000 7f2055c50740 20 client.15351 trim_cache size 0 max 16384 2024-05-10T11:19:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:19:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:19:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:19:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:19:01.840 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.835+0000 7f2052f32640 1 -- 172.21.15.100:0/3787430776 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f203c0b28a0 msgr2=0x7f203c0b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:01.840 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.835+0000 7f2052f32640 1 --2- 172.21.15.100:0/3787430776 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f203c0b28a0 0x7f203c0b4d60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2044006010 tx=0x7f204403f040 comp rx=0 tx=0).stop 2024-05-10T11:19:01.840 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.835+0000 7f2052f32640 1 -- 172.21.15.100:0/3787430776 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f204c105dc0 msgr2=0x7f204c198fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:01.840 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.835+0000 7f2052f32640 1 --2- 172.21.15.100:0/3787430776 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f204c105dc0 0x7f204c198fc0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f2038005fd0 tx=0x7f203800dde0 comp rx=0 tx=0).stop 2024-05-10T11:19:01.840 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.835+0000 7f2052f32640 1 -- 172.21.15.100:0/3787430776 shutdown_connections 2024-05-10T11:19:01.840 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.835+0000 7f2052f32640 1 --2- 172.21.15.100:0/3787430776 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f203c0b28a0 0x7f203c0b4d60 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:01.841 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.835+0000 7f2052f32640 1 --2- 172.21.15.100:0/3787430776 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f204c1086f0 0x7f204c1a0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:01.841 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.835+0000 7f2052f32640 1 --2- 172.21.15.100:0/3787430776 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f204c105dc0 0x7f204c198fc0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:01.841 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.835+0000 7f2052f32640 1 --2- 172.21.15.100:0/3787430776 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f204c103490 0x7f204c198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:01.841 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.835+0000 7f2052f32640 1 -- 172.21.15.100:0/3787430776 >> 172.21.15.100:0/3787430776 conn(0x7f204c0fce80 msgr2=0x7f204c103dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:01.841 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.835+0000 7f2052f32640 1 -- 172.21.15.100:0/3787430776 shutdown_connections 2024-05-10T11:19:01.841 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:01.835+0000 7f2052f32640 1 -- 172.21.15.100:0/3787430776 wait complete. 2024-05-10T11:19:02.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:19:01 smithi178 ceph-mon[33359]: pgmap v439: 129 pgs: 129 active+clean; 661 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-05-10T11:19:02.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:19:01 smithi188 ceph-mon[34075]: pgmap v439: 129 pgs: 129 active+clean; 661 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-05-10T11:19:02.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:19:01 smithi100 ceph-mon[30296]: pgmap v439: 129 pgs: 129 active+clean; 661 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-05-10T11:19:04.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:19:03 smithi178 ceph-mon[33359]: pgmap v440: 129 pgs: 129 active+clean; 661 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-05-10T11:19:04.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:19:03 smithi188 ceph-mon[34075]: pgmap v440: 129 pgs: 129 active+clean; 661 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-05-10T11:19:04.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:19:03 smithi100 ceph-mon[30296]: pgmap v440: 129 pgs: 129 active+clean; 661 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-05-10T11:19:06.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:19:05 smithi178 ceph-mon[33359]: pgmap v441: 129 pgs: 129 active+clean; 661 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-10T11:19:06.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:19:05 smithi188 ceph-mon[34075]: pgmap v441: 129 pgs: 129 active+clean; 661 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-10T11:19:06.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:19:05 smithi100 ceph-mon[30296]: pgmap v441: 129 pgs: 129 active+clean; 661 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-10T11:19:08.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:19:07 smithi178 ceph-mon[33359]: pgmap v442: 129 pgs: 129 active+clean; 661 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-10T11:19:08.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:19:07 smithi188 ceph-mon[34075]: pgmap v442: 129 pgs: 129 active+clean; 661 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-10T11:19:08.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:19:07 smithi100 ceph-mon[30296]: pgmap v442: 129 pgs: 129 active+clean; 661 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-10T11:19:10.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:19:09 smithi178 ceph-mon[33359]: pgmap v443: 129 pgs: 129 active+clean; 662 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-10T11:19:10.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:19:09 smithi188 ceph-mon[34075]: pgmap v443: 129 pgs: 129 active+clean; 662 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-10T11:19:10.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:19:09 smithi100 ceph-mon[30296]: pgmap v443: 129 pgs: 129 active+clean; 662 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-10T11:19:11.010 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T11:19:11.259 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:19:12.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:19:11 smithi178 ceph-mon[33359]: pgmap v444: 129 pgs: 129 active+clean; 662 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-10T11:19:12.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:19:11 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:19:12.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:19:11 smithi188 ceph-mon[34075]: pgmap v444: 129 pgs: 129 active+clean; 662 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-10T11:19:12.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:19:11 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:19:12.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:19:11 smithi100 ceph-mon[30296]: pgmap v444: 129 pgs: 129 active+clean; 662 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-10T11:19:12.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:19:11 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:19:13.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:19:12 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:19:13.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:19:12 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:19:13.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:19:12 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:19:13.010 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:19:12 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:19:13.010 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:19:12 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:19:13.010 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:19:12 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:19:13.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:19:12 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:19:13.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:19:12 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:19:13.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:19:12 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:19:13.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.116+0000 7f7c1137a700 1 -- 172.21.15.100:0/1169781778 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c0c1023d0 msgr2=0x7f7c0c102850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:13.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.116+0000 7f7c1137a700 1 --2- 172.21.15.100:0/1169781778 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c0c1023d0 0x7f7c0c102850 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f7c00009990 tx=0x7f7c00009ca0 comp rx=0 tx=0).stop 2024-05-10T11:19:13.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.116+0000 7f7c1137a700 1 -- 172.21.15.100:0/1169781778 shutdown_connections 2024-05-10T11:19:13.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.116+0000 7f7c1137a700 1 --2- 172.21.15.100:0/1169781778 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c0c102d90 0x7f7c0c10f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:13.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.116+0000 7f7c1137a700 1 --2- 172.21.15.100:0/1169781778 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c0c108440 0x7f7c0c108820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:13.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.116+0000 7f7c1137a700 1 --2- 172.21.15.100:0/1169781778 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c0c1023d0 0x7f7c0c102850 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:13.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.116+0000 7f7c1137a700 1 -- 172.21.15.100:0/1169781778 >> 172.21.15.100:0/1169781778 conn(0x7f7c0c0793e0 msgr2=0x7f7c0c0797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:13.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.116+0000 7f7c1137a700 1 -- 172.21.15.100:0/1169781778 shutdown_connections 2024-05-10T11:19:13.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.116+0000 7f7c1137a700 1 -- 172.21.15.100:0/1169781778 wait complete. 2024-05-10T11:19:13.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.117+0000 7f7c1137a700 1 Processor -- start 2024-05-10T11:19:13.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.117+0000 7f7c1137a700 1 -- start start 2024-05-10T11:19:13.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.117+0000 7f7c1137a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c0c1023d0 0x7f7c0c19a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:13.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.117+0000 7f7c1137a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c0c102d90 0x7f7c0c19ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:13.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.117+0000 7f7c1137a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c0c108440 0x7f7c0c1a59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:13.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.117+0000 7f7c1137a700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0c112490 con 0x7f7c0c108440 2024-05-10T11:19:13.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.117+0000 7f7c1137a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0c112310 con 0x7f7c0c102d90 2024-05-10T11:19:13.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.117+0000 7f7c1137a700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0c112610 con 0x7f7c0c1023d0 2024-05-10T11:19:13.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.117+0000 7f7c0b7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c0c108440 0x7f7c0c1a59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:13.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.117+0000 7f7c0b7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c0c108440 0x7f7c0c1a59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53396/0 (socket says 172.21.15.100:53396) 2024-05-10T11:19:13.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.117+0000 7f7c0b7fe700 1 -- 172.21.15.100:0/1567527563 learned_addr learned my addr 172.21.15.100:0/1567527563 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:13.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.117+0000 7f7c0a7fc700 1 --2- 172.21.15.100:0/1567527563 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c0c102d90 0x7f7c0c19ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:13.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.118+0000 7f7c0affd700 1 --2- 172.21.15.100:0/1567527563 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c0c1023d0 0x7f7c0c19a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:13.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.118+0000 7f7c0b7fe700 1 -- 172.21.15.100:0/1567527563 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c0c1023d0 msgr2=0x7f7c0c19a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:13.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.118+0000 7f7c0b7fe700 1 --2- 172.21.15.100:0/1567527563 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c0c1023d0 0x7f7c0c19a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:13.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.118+0000 7f7c0b7fe700 1 -- 172.21.15.100:0/1567527563 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c0c102d90 msgr2=0x7f7c0c19ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:13.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.118+0000 7f7c0b7fe700 1 --2- 172.21.15.100:0/1567527563 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c0c102d90 0x7f7c0c19ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:13.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.118+0000 7f7c0b7fe700 1 -- 172.21.15.100:0/1567527563 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c0c1a5f10 con 0x7f7c0c108440 2024-05-10T11:19:13.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.118+0000 7f7c0b7fe700 1 --2- 172.21.15.100:0/1567527563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c0c108440 0x7f7c0c1a59d0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f7c04007ee0 tx=0x7f7c0400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:13.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.118+0000 7f7bf3fff700 1 -- 172.21.15.100:0/1567527563 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c04004bb0 con 0x7f7c0c108440 2024-05-10T11:19:13.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.118+0000 7f7c1137a700 1 -- 172.21.15.100:0/1567527563 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c0c1a6140 con 0x7f7c0c108440 2024-05-10T11:19:13.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.118+0000 7f7bf3fff700 1 -- 172.21.15.100:0/1567527563 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7c04004d50 con 0x7f7c0c108440 2024-05-10T11:19:13.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.118+0000 7f7bf3fff700 1 -- 172.21.15.100:0/1567527563 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c040058b0 con 0x7f7c0c108440 2024-05-10T11:19:13.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.118+0000 7f7c1137a700 1 -- 172.21.15.100:0/1567527563 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c0c1a6650 con 0x7f7c0c108440 2024-05-10T11:19:13.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.121+0000 7f7bf3fff700 1 -- 172.21.15.100:0/1567527563 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f7c0401a020 con 0x7f7c0c108440 2024-05-10T11:19:13.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.121+0000 7f7bf3fff700 1 --2- 172.21.15.100:0/1567527563 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f7bf40b2880 0x7f7bf40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:13.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.121+0000 7f7c0affd700 1 --2- 172.21.15.100:0/1567527563 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f7bf40b2880 0x7f7bf40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:13.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.121+0000 7f7bf3fff700 1 -- 172.21.15.100:0/1567527563 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f7c040e1c00 con 0x7f7c0c108440 2024-05-10T11:19:13.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.122+0000 7f7c0affd700 1 --2- 172.21.15.100:0/1567527563 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f7bf40b2880 0x7f7bf40b4d40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc009a10 tx=0x7f7bfc009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:13.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.122+0000 7f7c1bc0db80 1 Processor -- start 2024-05-10T11:19:13.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.122+0000 7f7c1bc0db80 1 -- start start 2024-05-10T11:19:13.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.122+0000 7f7c1bc0db80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b12eb561b0 0x55b12eb56590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:13.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.122+0000 7f7c1bc0db80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b12eb56ad0 0x55b12eb50810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:13.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.122+0000 7f7c1bc0db80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b12eb50d50 0x55b12eb51190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:13.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.122+0000 7f7c1bc0db80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55b12eaa2940 con 0x55b12eb50d50 2024-05-10T11:19:13.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.122+0000 7f7c1bc0db80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b12ea07da0 con 0x55b12eb561b0 2024-05-10T11:19:13.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.122+0000 7f7c1bc0db80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b12ea9e900 con 0x55b12eb56ad0 2024-05-10T11:19:13.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.122+0000 7f7c0b7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b12eb50d50 0x55b12eb51190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:13.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.122+0000 7f7c0b7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b12eb50d50 0x55b12eb51190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53410/0 (socket says 172.21.15.100:53410) 2024-05-10T11:19:13.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.122+0000 7f7c0b7fe700 1 -- 172.21.15.100:0/1835613946 learned_addr learned my addr 172.21.15.100:0/1835613946 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:13.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.122+0000 7f7c0affd700 1 --2- 172.21.15.100:0/1835613946 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b12eb561b0 0x55b12eb56590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:13.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.122+0000 7f7c0a7fc700 1 --2- 172.21.15.100:0/1835613946 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b12eb56ad0 0x55b12eb50810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:13.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.122+0000 7f7c0b7fe700 1 -- 172.21.15.100:0/1835613946 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b12eb56ad0 msgr2=0x55b12eb50810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:13.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.122+0000 7f7c0b7fe700 1 --2- 172.21.15.100:0/1835613946 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b12eb56ad0 0x55b12eb50810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:13.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.122+0000 7f7c0b7fe700 1 -- 172.21.15.100:0/1835613946 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b12eb561b0 msgr2=0x55b12eb56590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:13.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.122+0000 7f7c0b7fe700 1 --2- 172.21.15.100:0/1835613946 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b12eb561b0 0x55b12eb56590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:13.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.122+0000 7f7c0b7fe700 1 -- 172.21.15.100:0/1835613946 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b12e9ef8d0 con 0x55b12eb50d50 2024-05-10T11:19:13.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.122+0000 7f7c0affd700 1 --2- 172.21.15.100:0/1835613946 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b12eb561b0 0x55b12eb56590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:19:13.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.123+0000 7f7c0b7fe700 1 --2- 172.21.15.100:0/1835613946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b12eb50d50 0x55b12eb51190 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f7c040a8e50 tx=0x7f7c040a88e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c186b26fde64985 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:13.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.123+0000 7f7bf17fa700 1 -- 172.21.15.100:0/1835613946 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c040cb070 con 0x55b12eb50d50 2024-05-10T11:19:13.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.123+0000 7f7bf17fa700 1 -- 172.21.15.100:0/1835613946 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7c040a8ec0 con 0x55b12eb50d50 2024-05-10T11:19:13.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.123+0000 7f7bf17fa700 1 -- 172.21.15.100:0/1835613946 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c040b1890 con 0x55b12eb50d50 2024-05-10T11:19:13.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.123+0000 7f7c1bc0db80 1 -- 172.21.15.100:0/1835613946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b12eb50d50 msgr2=0x55b12eb51190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:13.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.123+0000 7f7c1bc0db80 1 --2- 172.21.15.100:0/1835613946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b12eb50d50 0x55b12eb51190 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f7c040a8e50 tx=0x7f7c040a88e0 comp rx=0 tx=0).stop 2024-05-10T11:19:13.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.123+0000 7f7c1bc0db80 1 -- 172.21.15.100:0/1835613946 shutdown_connections 2024-05-10T11:19:13.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.123+0000 7f7c1bc0db80 1 --2- 172.21.15.100:0/1835613946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b12eb50d50 0x55b12eb51190 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:13.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.124+0000 7f7c1bc0db80 1 --2- 172.21.15.100:0/1835613946 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b12eb561b0 0x55b12eb56590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:13.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.124+0000 7f7c1bc0db80 1 --2- 172.21.15.100:0/1835613946 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b12eb56ad0 0x55b12eb50810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:13.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.124+0000 7f7c1bc0db80 1 -- 172.21.15.100:0/1835613946 >> 172.21.15.100:0/1835613946 conn(0x55b12ea7cb70 msgr2=0x55b12eb4b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:13.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.124+0000 7f7c1bc0db80 1 -- 172.21.15.100:0/1835613946 shutdown_connections 2024-05-10T11:19:13.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.124+0000 7f7c1bc0db80 1 -- 172.21.15.100:0/1835613946 wait complete. 2024-05-10T11:19:13.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.124+0000 7f7c1bc0db80 1 -- start start 2024-05-10T11:19:13.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.124+0000 7f7c1bc0db80 1 Processor -- start 2024-05-10T11:19:13.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.124+0000 7f7c1bc0db80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b12eb50d50 0x55b12eb5c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:13.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.124+0000 7f7c1bc0db80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b12eb561b0 0x55b12eb59b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:13.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.124+0000 7f7c1bc0db80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b12eb56ad0 0x55b12eb5a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:13.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.124+0000 7f7c1bc0db80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55b12eb5d450 con 0x55b12eb56ad0 2024-05-10T11:19:13.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.124+0000 7f7c1bc0db80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b12ea5bed0 con 0x55b12eb50d50 2024-05-10T11:19:13.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.124+0000 7f7c1bc0db80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b12eb5d5d0 con 0x55b12eb561b0 2024-05-10T11:19:13.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.124+0000 7f7c1bc0db80 10 client.0 _ll_register_callbacks cb 0x55b12ea365b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:19:13.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.124+0000 7f7c0b7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b12eb56ad0 0x55b12eb5a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:13.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.124+0000 7f7c0b7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b12eb56ad0 0x55b12eb5a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53420/0 (socket says 172.21.15.100:53420) 2024-05-10T11:19:13.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.124+0000 7f7c0b7fe700 1 -- 172.21.15.100:0/2936402697 learned_addr learned my addr 172.21.15.100:0/2936402697 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:13.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.124+0000 7f7c0affd700 1 --2- 172.21.15.100:0/2936402697 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b12eb50d50 0x55b12eb5c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:13.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.125+0000 7f7c0b7fe700 1 -- 172.21.15.100:0/2936402697 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b12eb561b0 msgr2=0x55b12eb59b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:13.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.125+0000 7f7c0b7fe700 1 --2- 172.21.15.100:0/2936402697 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b12eb561b0 0x55b12eb59b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:13.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.125+0000 7f7c0b7fe700 1 -- 172.21.15.100:0/2936402697 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b12eb50d50 msgr2=0x55b12eb5c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:13.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.125+0000 7f7c0b7fe700 1 --2- 172.21.15.100:0/2936402697 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b12eb50d50 0x55b12eb5c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:13.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.125+0000 7f7c0b7fe700 1 -- 172.21.15.100:0/2936402697 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b12ea59760 con 0x55b12eb56ad0 2024-05-10T11:19:13.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.125+0000 7f7c0b7fe700 1 --2- 172.21.15.100:0/2936402697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b12eb56ad0 0x55b12eb5a080 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f7c040a8bc0 tx=0x7f7c040a9400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f1e69468eac3249 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:13.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.125+0000 7f7beeff5700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:19:13.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.125+0000 7f7beeff5700 1 -- 172.21.15.100:0/2936402697 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c040cb050 con 0x55b12eb56ad0 2024-05-10T11:19:13.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.125+0000 7f7beeff5700 1 -- 172.21.15.100:0/2936402697 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f7c040b1aa0 con 0x55b12eb56ad0 2024-05-10T11:19:13.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.125+0000 7f7c1bc0db80 1 -- 172.21.15.100:0/2936402697 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c040a8ec0 con 0x55b12eb56ad0 2024-05-10T11:19:13.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.125+0000 7f7beeff5700 1 -- 172.21.15.100:0/2936402697 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7c040a8120 con 0x55b12eb56ad0 2024-05-10T11:19:13.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.125+0000 7f7c1bc0db80 10 client.15363 fetch_fsmap learned FSMap version 27 2024-05-10T11:19:13.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.125+0000 7f7c1bc0db80 1 -- 172.21.15.100:0/2936402697 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b12eb67d60 con 0x55b12eb56ad0 2024-05-10T11:19:13.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.125+0000 7f7beeff5700 1 -- 172.21.15.100:0/2936402697 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7c040a8ec0 con 0x55b12eb56ad0 2024-05-10T11:19:13.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.125+0000 7f7beeff5700 1 -- 172.21.15.100:0/2936402697 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f7c040a7b90 con 0x55b12eb56ad0 2024-05-10T11:19:13.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.125+0000 7f7c1bc0db80 10 client.15363 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T11:19:13.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.125+0000 7f7c1bc0db80 10 client.15363 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:19:13.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.125+0000 7f7c1bc0db80 20 client.15363 populate_metadata read hostname 'smithi100' 2024-05-10T11:19:13.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.125+0000 7f7c1bc0db80 1 --2- 172.21.15.100:0/2936402697 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55b12eb6a230 0x55b12eb6c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:13.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.125+0000 7f7c1bc0db80 4 client.15363 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:19:13.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.125+0000 7f7c1bc0db80 1 -- 172.21.15.100:0/2936402697 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b12eab0320 con 0x55b12eb6a230 2024-05-10T11:19:13.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.126+0000 7f7c0affd700 1 --2- 172.21.15.100:0/2936402697 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55b12eb6a230 0x55b12eb6c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:13.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.126+0000 7f7beeff5700 10 client.15363 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:19:13.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.126+0000 7f7c0affd700 1 --2- 172.21.15.100:0/2936402697 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55b12eb6a230 0x55b12eb6c620 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=641271c4dd8e1f31 server_cookie=fb72f487825fd9ba in_seq=0 out_seq=0 2024-05-10T11:19:13.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.127+0000 7f7beeff5700 1 -- 172.21.15.100:0/2936402697 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b12eab0320 con 0x55b12eb6a230 2024-05-10T11:19:13.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.127+0000 7f7beeff5700 10 client.15363 handle_command_reply: tid=0 2024-05-10T11:19:13.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.127+0000 7f7c1bc0db80 1 client.15363 shutdown 2024-05-10T11:19:13.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.127+0000 7f7c1bc0db80 1 -- 172.21.15.100:0/2936402697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b12eb56ad0 msgr2=0x55b12eb5a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:13.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.127+0000 7f7c1bc0db80 1 --2- 172.21.15.100:0/2936402697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b12eb56ad0 0x55b12eb5a080 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f7c040a8bc0 tx=0x7f7c040a9400 comp rx=0 tx=0).stop 2024-05-10T11:19:13.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.127+0000 7f7c1bc0db80 1 -- 172.21.15.100:0/2936402697 shutdown_connections 2024-05-10T11:19:13.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.127+0000 7f7c1bc0db80 1 --2- 172.21.15.100:0/2936402697 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55b12eb6a230 0x55b12eb6c620 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:13.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.127+0000 7f7c1bc0db80 1 --2- 172.21.15.100:0/2936402697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b12eb56ad0 0x55b12eb5a080 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:13.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.127+0000 7f7beeff5700 0 client.15363 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:19:13.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.127+0000 7f7c1bc0db80 1 --2- 172.21.15.100:0/2936402697 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b12eb50d50 0x55b12eb5c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:13.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.127+0000 7f7c1bc0db80 1 --2- 172.21.15.100:0/2936402697 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b12eb561b0 0x55b12eb59b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:13.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.127+0000 7f7c1bc0db80 1 -- 172.21.15.100:0/2936402697 >> 172.21.15.100:0/2936402697 conn(0x55b12ea7cb70 msgr2=0x55b12eb45b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:13.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.127+0000 7f7c1bc0db80 1 -- 172.21.15.100:0/2936402697 shutdown_connections 2024-05-10T11:19:13.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.127+0000 7f7c1bc0db80 1 -- 172.21.15.100:0/2936402697 wait complete. 2024-05-10T11:19:13.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.127+0000 7f7c1bc0db80 20 client.15363 trim_cache size 0 max 16384 2024-05-10T11:19:13.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.147+0000 7f7c1bc0db80 1 Processor -- start 2024-05-10T11:19:13.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.147+0000 7f7c1bc0db80 1 -- start start 2024-05-10T11:19:13.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.147+0000 7f7c1bc0db80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b12eb50d50 0x55b12eb50810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:13.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.147+0000 7f7c1bc0db80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b12eb561b0 0x55b12eb45c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:13.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.147+0000 7f7c1bc0db80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b12eb56ad0 0x55b12eb6fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:13.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.147+0000 7f7c1bc0db80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0c1b1910 con 0x55b12eb50d50 2024-05-10T11:19:13.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.147+0000 7f7c1bc0db80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0c1af1a0 con 0x55b12eb56ad0 2024-05-10T11:19:13.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.147+0000 7f7c1bc0db80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0c1acff0 con 0x55b12eb561b0 2024-05-10T11:19:13.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.147+0000 7f7c0affd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b12eb50d50 0x55b12eb50810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:13.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.147+0000 7f7c0affd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b12eb50d50 0x55b12eb50810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53422/0 (socket says 172.21.15.100:53422) 2024-05-10T11:19:13.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.147+0000 7f7c0affd700 1 -- 172.21.15.100:0/1490855231 learned_addr learned my addr 172.21.15.100:0/1490855231 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:13.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.147+0000 7f7c0affd700 1 -- 172.21.15.100:0/1490855231 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b12eb561b0 msgr2=0x55b12eb45c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:13.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.147+0000 7f7c0affd700 1 --2- 172.21.15.100:0/1490855231 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b12eb561b0 0x55b12eb45c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:13.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.147+0000 7f7c0affd700 1 -- 172.21.15.100:0/1490855231 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b12eb56ad0 msgr2=0x55b12eb6fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:13.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.147+0000 7f7c0affd700 1 --2- 172.21.15.100:0/1490855231 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b12eb56ad0 0x55b12eb6fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:13.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.147+0000 7f7c0affd700 1 -- 172.21.15.100:0/1490855231 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b12eb59910 con 0x55b12eb50d50 2024-05-10T11:19:13.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.147+0000 7f7c0affd700 1 --2- 172.21.15.100:0/1490855231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b12eb50d50 0x55b12eb50810 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x55b12eb5d640 tx=0x7f7bfc011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e558b59a309609e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:13.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.148+0000 7f7bf0ff9700 1 -- 172.21.15.100:0/1490855231 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bfc01b040 con 0x55b12eb50d50 2024-05-10T11:19:13.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.148+0000 7f7bf0ff9700 1 -- 172.21.15.100:0/1490855231 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7bfc0286e0 con 0x55b12eb50d50 2024-05-10T11:19:13.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.148+0000 7f7bf0ff9700 1 -- 172.21.15.100:0/1490855231 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bfc015810 con 0x55b12eb50d50 2024-05-10T11:19:13.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.148+0000 7f7c1bc0db80 1 -- 172.21.15.100:0/1490855231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b12eb50d50 msgr2=0x55b12eb50810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:13.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.148+0000 7f7c1bc0db80 1 --2- 172.21.15.100:0/1490855231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b12eb50d50 0x55b12eb50810 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x55b12eb5d640 tx=0x7f7bfc011000 comp rx=0 tx=0).stop 2024-05-10T11:19:13.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.148+0000 7f7c1bc0db80 1 -- 172.21.15.100:0/1490855231 shutdown_connections 2024-05-10T11:19:13.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.148+0000 7f7c1bc0db80 1 --2- 172.21.15.100:0/1490855231 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b12eb56ad0 0x55b12eb6fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:13.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.148+0000 7f7c1bc0db80 1 --2- 172.21.15.100:0/1490855231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b12eb50d50 0x55b12eb50810 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:13.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.148+0000 7f7c1bc0db80 1 --2- 172.21.15.100:0/1490855231 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b12eb561b0 0x55b12eb45c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:13.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.148+0000 7f7c1bc0db80 1 -- 172.21.15.100:0/1490855231 >> 172.21.15.100:0/1490855231 conn(0x55b12ea7cb70 msgr2=0x55b12eb58300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:13.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.148+0000 7f7c1bc0db80 1 -- 172.21.15.100:0/1490855231 shutdown_connections 2024-05-10T11:19:13.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.148+0000 7f7c1bc0db80 1 -- 172.21.15.100:0/1490855231 wait complete. 2024-05-10T11:19:13.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.149+0000 7f7c1bc0db80 1 -- start start 2024-05-10T11:19:13.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.149+0000 7f7c1bc0db80 1 Processor -- start 2024-05-10T11:19:13.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.149+0000 7f7c1bc0db80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b12eb50d50 0x55b12ebcc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:13.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.149+0000 7f7c1bc0db80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b12eb561b0 0x55b12ebcce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:13.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.149+0000 7f7c1bc0db80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b12eb56ad0 0x55b12ebd7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:13.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.149+0000 7f7c1bc0db80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0c1ad660 con 0x55b12eb561b0 2024-05-10T11:19:13.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.149+0000 7f7c1bc0db80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0c1aead0 con 0x55b12eb56ad0 2024-05-10T11:19:13.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.149+0000 7f7c1bc0db80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0c1add30 con 0x55b12eb50d50 2024-05-10T11:19:13.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.149+0000 7f7c1bc0db80 10 client.0 _ll_register_callbacks cb 0x55b12ea365b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:19:13.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.149+0000 7f7c0a7fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b12eb561b0 0x55b12ebcce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:13.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.149+0000 7f7c0a7fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b12eb561b0 0x55b12ebcce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53424/0 (socket says 172.21.15.100:53424) 2024-05-10T11:19:13.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.149+0000 7f7c0a7fc700 1 -- 172.21.15.100:0/3457131850 learned_addr learned my addr 172.21.15.100:0/3457131850 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:13.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.149+0000 7f7c0b7fe700 1 --2- 172.21.15.100:0/3457131850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b12eb56ad0 0x55b12ebd7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:13.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.149+0000 7f7c0affd700 1 --2- 172.21.15.100:0/3457131850 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b12eb50d50 0x55b12ebcc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:13.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.149+0000 7f7c0a7fc700 1 -- 172.21.15.100:0/3457131850 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b12eb50d50 msgr2=0x55b12ebcc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:13.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.149+0000 7f7c0a7fc700 1 --2- 172.21.15.100:0/3457131850 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b12eb50d50 0x55b12ebcc8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:13.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.149+0000 7f7c0a7fc700 1 -- 172.21.15.100:0/3457131850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b12eb56ad0 msgr2=0x55b12ebd7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:13.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.149+0000 7f7c0a7fc700 1 --2- 172.21.15.100:0/3457131850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b12eb56ad0 0x55b12ebd7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:13.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.149+0000 7f7c0a7fc700 1 -- 172.21.15.100:0/3457131850 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b12ebd7d10 con 0x55b12eb561b0 2024-05-10T11:19:13.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.149+0000 7f7c0b7fe700 1 --2- 172.21.15.100:0/3457131850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b12eb56ad0 0x55b12ebd7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:19:13.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.149+0000 7f7c0a7fc700 1 --2- 172.21.15.100:0/3457131850 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b12eb561b0 0x55b12ebcce20 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7c00015000 tx=0x7f7c00003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5e445e9e3c6c200 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:13.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.149+0000 7f7beeff5700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:19:13.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.150+0000 7f7beeff5700 1 -- 172.21.15.100:0/3457131850 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c000054e0 con 0x55b12eb561b0 2024-05-10T11:19:13.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.150+0000 7f7beeff5700 1 -- 172.21.15.100:0/3457131850 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f7c00005680 con 0x55b12eb561b0 2024-05-10T11:19:13.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.150+0000 7f7c1bc0db80 1 -- 172.21.15.100:0/3457131850 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7bfc0286e0 con 0x55b12eb561b0 2024-05-10T11:19:13.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.150+0000 7f7beeff5700 1 -- 172.21.15.100:0/3457131850 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7c0002f050 con 0x55b12eb561b0 2024-05-10T11:19:13.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.150+0000 7f7c1bc0db80 10 client.15369 fetch_fsmap learned FSMap version 27 2024-05-10T11:19:13.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.150+0000 7f7c1bc0db80 1 -- 172.21.15.100:0/3457131850 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b12ebd9050 con 0x55b12eb561b0 2024-05-10T11:19:13.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.150+0000 7f7beeff5700 1 -- 172.21.15.100:0/3457131850 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7bfc0286e0 con 0x55b12eb561b0 2024-05-10T11:19:13.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.150+0000 7f7beeff5700 1 -- 172.21.15.100:0/3457131850 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f7c0002a030 con 0x55b12eb561b0 2024-05-10T11:19:13.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.150+0000 7f7c1bc0db80 10 client.15369 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T11:19:13.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.150+0000 7f7c1bc0db80 10 client.15369 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:19:13.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.150+0000 7f7c1bc0db80 20 client.15369 populate_metadata read hostname 'smithi100' 2024-05-10T11:19:13.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.150+0000 7f7c1bc0db80 1 --2- 172.21.15.100:0/3457131850 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55b12ebd9810 0x55b12ebdbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:13.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.150+0000 7f7c1bc0db80 4 client.15369 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T11:19:13.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.150+0000 7f7c1bc0db80 1 -- 172.21.15.100:0/3457131850 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b12eaaa3c0 con 0x55b12ebd9810 2024-05-10T11:19:13.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.150+0000 7f7c0affd700 1 --2- 172.21.15.100:0/3457131850 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55b12ebd9810 0x55b12ebdbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:13.194 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:19:13.194 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-05-10T11:19:13.194 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-05-10T11:19:13.194 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:19:13.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.151+0000 7f7c0affd700 1 --2- 172.21.15.100:0/3457131850 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55b12ebd9810 0x55b12ebdbc30 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d48940edabe9b7a6 server_cookie=8f8a10d22486b17b in_seq=0 out_seq=0 2024-05-10T11:19:13.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.151+0000 7f7beeff5700 10 client.15369 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:19:13.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.151+0000 7f7beeff5700 1 -- 172.21.15.100:0/3457131850 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b12eaaa3c0 con 0x55b12ebd9810 2024-05-10T11:19:13.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.151+0000 7f7beeff5700 10 client.15369 handle_command_reply: tid=0 2024-05-10T11:19:13.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.151+0000 7f7c1bc0db80 1 client.15369 shutdown 2024-05-10T11:19:13.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.151+0000 7f7c1bc0db80 1 -- 172.21.15.100:0/3457131850 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b12eb561b0 msgr2=0x55b12ebcce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:13.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.151+0000 7f7c1bc0db80 1 --2- 172.21.15.100:0/3457131850 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b12eb561b0 0x55b12ebcce20 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7c00015000 tx=0x7f7c00003680 comp rx=0 tx=0).stop 2024-05-10T11:19:13.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.151+0000 7f7c1bc0db80 1 -- 172.21.15.100:0/3457131850 shutdown_connections 2024-05-10T11:19:13.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.151+0000 7f7c1bc0db80 1 --2- 172.21.15.100:0/3457131850 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55b12ebd9810 0x55b12ebdbc30 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:13.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.151+0000 7f7c1bc0db80 1 --2- 172.21.15.100:0/3457131850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b12eb56ad0 0x55b12ebd7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:13.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.151+0000 7f7c1bc0db80 1 --2- 172.21.15.100:0/3457131850 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b12eb50d50 0x55b12ebcc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:13.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.151+0000 7f7beeff5700 0 client.15369 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:19:13.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.151+0000 7f7c1bc0db80 1 --2- 172.21.15.100:0/3457131850 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b12eb561b0 0x55b12ebcce20 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:13.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.151+0000 7f7c1bc0db80 1 -- 172.21.15.100:0/3457131850 >> 172.21.15.100:0/3457131850 conn(0x55b12ea7cb70 msgr2=0x55b12eb58b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:13.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.151+0000 7f7c1bc0db80 1 -- 172.21.15.100:0/3457131850 shutdown_connections 2024-05-10T11:19:13.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.151+0000 7f7c1bc0db80 1 -- 172.21.15.100:0/3457131850 wait complete. 2024-05-10T11:19:13.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.151+0000 7f7c1bc0db80 20 client.15369 trim_cache size 0 max 16384 2024-05-10T11:19:13.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.152+0000 7f7c1137a700 1 -- 172.21.15.100:0/1567527563 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f7bf40b2880 msgr2=0x7f7bf40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:13.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.152+0000 7f7c1137a700 1 --2- 172.21.15.100:0/1567527563 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f7bf40b2880 0x7f7bf40b4d40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc009a10 tx=0x7f7bfc009190 comp rx=0 tx=0).stop 2024-05-10T11:19:13.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.152+0000 7f7c1137a700 1 -- 172.21.15.100:0/1567527563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c0c108440 msgr2=0x7f7c0c1a59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:13.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.152+0000 7f7c1137a700 1 --2- 172.21.15.100:0/1567527563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c0c108440 0x7f7c0c1a59d0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f7c04007ee0 tx=0x7f7c0400d3b0 comp rx=0 tx=0).stop 2024-05-10T11:19:13.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.152+0000 7f7c1137a700 1 -- 172.21.15.100:0/1567527563 shutdown_connections 2024-05-10T11:19:13.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.152+0000 7f7c1137a700 1 --2- 172.21.15.100:0/1567527563 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f7bf40b2880 0x7f7bf40b4d40 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:13.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.152+0000 7f7c1137a700 1 --2- 172.21.15.100:0/1567527563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c0c108440 0x7f7c0c1a59d0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:13.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.152+0000 7f7c1137a700 1 --2- 172.21.15.100:0/1567527563 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c0c1023d0 0x7f7c0c19a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:13.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.152+0000 7f7c1137a700 1 --2- 172.21.15.100:0/1567527563 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c0c102d90 0x7f7c0c19ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:13.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.152+0000 7f7c1137a700 1 -- 172.21.15.100:0/1567527563 >> 172.21.15.100:0/1567527563 conn(0x7f7c0c0793e0 msgr2=0x7f7c0c101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:13.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.152+0000 7f7c1137a700 1 -- 172.21.15.100:0/1567527563 shutdown_connections 2024-05-10T11:19:13.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:13.152+0000 7f7c1137a700 1 -- 172.21.15.100:0/1567527563 wait complete. 2024-05-10T11:19:13.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:19:13 smithi100 ceph-mon[30296]: pgmap v445: 129 pgs: 129 active+clean; 662 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-10T11:19:13.911 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T11:19:13.912 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T11:19:14.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:19:13 smithi178 ceph-mon[33359]: pgmap v445: 129 pgs: 129 active+clean; 662 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-10T11:19:14.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:19:13 smithi188 ceph-mon[34075]: pgmap v445: 129 pgs: 129 active+clean; 662 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-10T11:19:14.154 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:19:14.937 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.925+0000 7f1cc1d32640 1 -- 172.21.15.100:0/693552091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cbc1034c0 msgr2=0x7f1cbc109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:14.937 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.925+0000 7f1cc1d32640 1 --2- 172.21.15.100:0/693552091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cbc1034c0 0x7f1cbc109d50 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f1cb4009900 tx=0x7f1cb402a410 comp rx=0 tx=0).stop 2024-05-10T11:19:14.937 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.925+0000 7f1cc1d32640 1 -- 172.21.15.100:0/693552091 shutdown_connections 2024-05-10T11:19:14.937 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.925+0000 7f1cc1d32640 1 --2- 172.21.15.100:0/693552091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cbc1034c0 0x7f1cbc109d50 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:14.938 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.925+0000 7f1cc1d32640 1 --2- 172.21.15.100:0/693552091 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1cbc102b00 0x7f1cbc102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:14.938 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.925+0000 7f1cc1d32640 1 --2- 172.21.15.100:0/693552091 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1cbc101910 0x7f1cbc101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:14.938 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.925+0000 7f1cc1d32640 1 -- 172.21.15.100:0/693552091 >> 172.21.15.100:0/693552091 conn(0x7f1cbc0fcea0 msgr2=0x7f1cbc0ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:14.938 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.925+0000 7f1cc1d32640 1 -- 172.21.15.100:0/693552091 shutdown_connections 2024-05-10T11:19:14.938 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.925+0000 7f1cc1d32640 1 -- 172.21.15.100:0/693552091 wait complete. 2024-05-10T11:19:14.938 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.926+0000 7f1cc1d32640 1 Processor -- start 2024-05-10T11:19:14.938 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.926+0000 7f1cc1d32640 1 -- start start 2024-05-10T11:19:14.938 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.926+0000 7f1cc1d32640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1cbc101910 0x7f1cbc105970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:14.938 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.926+0000 7f1cc1d32640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cbc102b00 0x7f1cbc103fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:14.938 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.926+0000 7f1cc1d32640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1cbc1034c0 0x7f1cbc1044f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:14.938 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.926+0000 7f1cc1d32640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1cbc10d2f0 con 0x7f1cbc102b00 2024-05-10T11:19:14.938 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.926+0000 7f1cc1d32640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1cbc10d170 con 0x7f1cbc101910 2024-05-10T11:19:14.938 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.926+0000 7f1cc1d32640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1cbc10d470 con 0x7f1cbc1034c0 2024-05-10T11:19:14.938 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.926+0000 7f1cbaffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cbc102b00 0x7f1cbc103fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:14.939 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.926+0000 7f1cbaffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cbc102b00 0x7f1cbc103fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53446/0 (socket says 172.21.15.100:53446) 2024-05-10T11:19:14.939 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.926+0000 7f1cbaffd640 1 -- 172.21.15.100:0/3712136497 learned_addr learned my addr 172.21.15.100:0/3712136497 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:14.939 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.926+0000 7f1cbbfff640 1 --2- 172.21.15.100:0/3712136497 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1cbc1034c0 0x7f1cbc1044f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:14.939 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.926+0000 7f1cbb7fe640 1 --2- 172.21.15.100:0/3712136497 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1cbc101910 0x7f1cbc105970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:14.939 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.926+0000 7f1cbaffd640 1 -- 172.21.15.100:0/3712136497 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1cbc1034c0 msgr2=0x7f1cbc1044f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:14.939 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.926+0000 7f1cbaffd640 1 --2- 172.21.15.100:0/3712136497 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1cbc1034c0 0x7f1cbc1044f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:14.939 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.926+0000 7f1cbaffd640 1 -- 172.21.15.100:0/3712136497 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1cbc101910 msgr2=0x7f1cbc105970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:14.939 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.926+0000 7f1cbaffd640 1 --2- 172.21.15.100:0/3712136497 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1cbc101910 0x7f1cbc105970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:14.939 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.926+0000 7f1cbaffd640 1 -- 172.21.15.100:0/3712136497 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cbc104d80 con 0x7f1cbc102b00 2024-05-10T11:19:14.939 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.926+0000 7f1cbb7fe640 1 --2- 172.21.15.100:0/3712136497 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1cbc101910 0x7f1cbc105970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:19:14.939 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.927+0000 7f1cbaffd640 1 --2- 172.21.15.100:0/3712136497 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cbc102b00 0x7f1cbc103fb0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1ca400ebc0 tx=0x7f1ca400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:14.939 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.927+0000 7f1cb8ff9640 1 -- 172.21.15.100:0/3712136497 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca4010630 con 0x7f1cbc102b00 2024-05-10T11:19:14.939 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.927+0000 7f1cb8ff9640 1 -- 172.21.15.100:0/3712136497 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1ca4010c60 con 0x7f1cbc102b00 2024-05-10T11:19:14.939 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.927+0000 7f1cb8ff9640 1 -- 172.21.15.100:0/3712136497 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca4018b00 con 0x7f1cbc102b00 2024-05-10T11:19:14.940 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.927+0000 7f1cc1d32640 1 -- 172.21.15.100:0/3712136497 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cbc1a2bd0 con 0x7f1cbc102b00 2024-05-10T11:19:14.940 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.927+0000 7f1cc1d32640 1 -- 172.21.15.100:0/3712136497 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cbc1a30d0 con 0x7f1cbc102b00 2024-05-10T11:19:14.940 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.929+0000 7f1cb8ff9640 1 -- 172.21.15.100:0/3712136497 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f1ca4018ca0 con 0x7f1cbc102b00 2024-05-10T11:19:14.940 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.929+0000 7f1cc4a4f740 1 Processor -- start 2024-05-10T11:19:14.940 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.929+0000 7f1cc4a4f740 1 -- start start 2024-05-10T11:19:14.940 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.929+0000 7f1cc4a4f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc6c008db0 0x55fc6c0091d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:14.940 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.929+0000 7f1cc4a4f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fc6c0097a0 0x55fc6c00e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:14.940 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.929+0000 7f1cc4a4f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc6c00e910 0x55fc6c010da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:14.940 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.929+0000 7f1cc4a4f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55fc6bf6d5d0 con 0x55fc6c00e910 2024-05-10T11:19:14.940 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.929+0000 7f1cc4a4f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55fc6bf26890 con 0x55fc6c0097a0 2024-05-10T11:19:14.940 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.929+0000 7f1cc4a4f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55fc6bf69590 con 0x55fc6c008db0 2024-05-10T11:19:14.940 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.930+0000 7f1cbaffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc6c00e910 0x55fc6c010da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:14.940 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.930+0000 7f1cbbfff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc6c008db0 0x55fc6c0091d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:14.940 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.930+0000 7f1cbaffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc6c00e910 0x55fc6c010da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53450/0 (socket says 172.21.15.100:53450) 2024-05-10T11:19:14.940 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.930+0000 7f1cbaffd640 1 -- 172.21.15.100:0/776362760 learned_addr learned my addr 172.21.15.100:0/776362760 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.930+0000 7f1cbb7fe640 1 --2- 172.21.15.100:0/776362760 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fc6c0097a0 0x55fc6c00e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.930+0000 7f1cbaffd640 1 -- 172.21.15.100:0/776362760 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc6c008db0 msgr2=0x55fc6c0091d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.930+0000 7f1cbaffd640 1 --2- 172.21.15.100:0/776362760 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc6c008db0 0x55fc6c0091d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.930+0000 7f1cbaffd640 1 -- 172.21.15.100:0/776362760 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fc6c0097a0 msgr2=0x55fc6c00e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.930+0000 7f1cbaffd640 1 --2- 172.21.15.100:0/776362760 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fc6c0097a0 0x55fc6c00e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.930+0000 7f1cbaffd640 1 -- 172.21.15.100:0/776362760 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc6bc7f210 con 0x55fc6c00e910 2024-05-10T11:19:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.930+0000 7f1cbaffd640 1 --2- 172.21.15.100:0/776362760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc6c00e910 0x55fc6c010da0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f1ca40e6d70 tx=0x7f1ca40eb6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b1046e7a6fd947a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.930+0000 7f1cb1ffb640 1 -- 172.21.15.100:0/776362760 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca40f2070 con 0x55fc6c00e910 2024-05-10T11:19:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.930+0000 7f1cb8ff9640 1 --2- 172.21.15.100:0/3712136497 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f1ca80b28a0 0x7f1ca80b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.930+0000 7f1cb1ffb640 1 -- 172.21.15.100:0/776362760 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1ca40a99a0 con 0x55fc6c00e910 2024-05-10T11:19:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.930+0000 7f1cb1ffb640 1 -- 172.21.15.100:0/776362760 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca40bd8e0 con 0x55fc6c00e910 2024-05-10T11:19:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.930+0000 7f1cbbfff640 1 --2- 172.21.15.100:0/3712136497 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f1ca80b28a0 0x7f1ca80b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.931+0000 7f1cb8ff9640 1 -- 172.21.15.100:0/3712136497 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f1ca40e2bd0 con 0x7f1cbc102b00 2024-05-10T11:19:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.931+0000 7f1cc4a4f740 1 -- 172.21.15.100:0/776362760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc6c00e910 msgr2=0x55fc6c010da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.931+0000 7f1cbbfff640 1 --2- 172.21.15.100:0/3712136497 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f1ca80b28a0 0x7f1ca80b4d60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1cb4006010 tx=0x7f1cb4031420 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.931+0000 7f1cc4a4f740 1 --2- 172.21.15.100:0/776362760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc6c00e910 0x55fc6c010da0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f1ca40e6d70 tx=0x7f1ca40eb6a0 comp rx=0 tx=0).stop 2024-05-10T11:19:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.931+0000 7f1cc4a4f740 1 -- 172.21.15.100:0/776362760 shutdown_connections 2024-05-10T11:19:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.931+0000 7f1cc4a4f740 1 --2- 172.21.15.100:0/776362760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc6c00e910 0x55fc6c010da0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.931+0000 7f1cc4a4f740 1 --2- 172.21.15.100:0/776362760 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fc6c0097a0 0x55fc6c00e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.931+0000 7f1cc4a4f740 1 --2- 172.21.15.100:0/776362760 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc6c008db0 0x55fc6c0091d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.931+0000 7f1cc4a4f740 1 -- 172.21.15.100:0/776362760 >> 172.21.15.100:0/776362760 conn(0x55fc6c004420 msgr2=0x55fc6c004820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.931+0000 7f1cc4a4f740 1 -- 172.21.15.100:0/776362760 shutdown_connections 2024-05-10T11:19:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.931+0000 7f1cc4a4f740 1 -- 172.21.15.100:0/776362760 wait complete. 2024-05-10T11:19:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.931+0000 7f1cc4a4f740 1 -- start start 2024-05-10T11:19:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.932+0000 7f1cc4a4f740 1 Processor -- start 2024-05-10T11:19:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.932+0000 7f1cc4a4f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc6c008db0 0x55fc6c012f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.932+0000 7f1cc4a4f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc6c0097a0 0x55fc6c013470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.932+0000 7f1cc4a4f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fc6c00e910 0x55fc6c01a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.932+0000 7f1cc4a4f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55fc6c0118c0 con 0x55fc6c008db0 2024-05-10T11:19:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.932+0000 7f1cc4a4f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55fc6bf2a370 con 0x55fc6c00e910 2024-05-10T11:19:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.932+0000 7f1cc4a4f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55fc6c011a40 con 0x55fc6c0097a0 2024-05-10T11:19:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.932+0000 7f1cc4a4f740 10 client.0 _ll_register_callbacks cb 0x55fc6bc7ea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:19:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.932+0000 7f1cbb7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc6c008db0 0x55fc6c012f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.932+0000 7f1cbb7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc6c008db0 0x55fc6c012f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53464/0 (socket says 172.21.15.100:53464) 2024-05-10T11:19:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.932+0000 7f1cbb7fe640 1 -- 172.21.15.100:0/1260908053 learned_addr learned my addr 172.21.15.100:0/1260908053 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.932+0000 7f1cbaffd640 1 --2- 172.21.15.100:0/1260908053 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc6c0097a0 0x55fc6c013470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.932+0000 7f1cbbfff640 1 --2- 172.21.15.100:0/1260908053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fc6c00e910 0x55fc6c01a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.932+0000 7f1cbb7fe640 1 -- 172.21.15.100:0/1260908053 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc6c0097a0 msgr2=0x55fc6c013470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.932+0000 7f1cbb7fe640 1 --2- 172.21.15.100:0/1260908053 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc6c0097a0 0x55fc6c013470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.932+0000 7f1cbb7fe640 1 -- 172.21.15.100:0/1260908053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fc6c00e910 msgr2=0x55fc6c01a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.932+0000 7f1cbb7fe640 1 --2- 172.21.15.100:0/1260908053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fc6c00e910 0x55fc6c01a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.932+0000 7f1cbb7fe640 1 -- 172.21.15.100:0/1260908053 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc6bd4b620 con 0x55fc6c008db0 2024-05-10T11:19:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.932+0000 7f1cbaffd640 1 --2- 172.21.15.100:0/1260908053 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc6c0097a0 0x55fc6c013470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:19:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.932+0000 7f1cbb7fe640 1 --2- 172.21.15.100:0/1260908053 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc6c008db0 0x55fc6c012f30 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f1cac00be20 tx=0x7f1cac012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fde1056f8924ea server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.932+0000 7f1c9affd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:19:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.933+0000 7f1c9affd640 1 -- 172.21.15.100:0/1260908053 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cac004230 con 0x55fc6c008db0 2024-05-10T11:19:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.933+0000 7f1c9affd640 1 -- 172.21.15.100:0/1260908053 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f1cac0043d0 con 0x55fc6c008db0 2024-05-10T11:19:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.933+0000 7f1cc4a4f740 1 -- 172.21.15.100:0/1260908053 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ca40a99a0 con 0x55fc6c008db0 2024-05-10T11:19:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.933+0000 7f1c9affd640 1 -- 172.21.15.100:0/1260908053 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1cac024050 con 0x55fc6c008db0 2024-05-10T11:19:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.933+0000 7f1cc4a4f740 10 client.15381 fetch_fsmap learned FSMap version 27 2024-05-10T11:19:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.933+0000 7f1cc4a4f740 1 -- 172.21.15.100:0/1260908053 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fc6c01d560 con 0x55fc6c008db0 2024-05-10T11:19:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.933+0000 7f1c9affd640 1 -- 172.21.15.100:0/1260908053 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1ca40a99a0 con 0x55fc6c008db0 2024-05-10T11:19:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.933+0000 7f1c9affd640 1 -- 172.21.15.100:0/1260908053 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f1cac003810 con 0x55fc6c008db0 2024-05-10T11:19:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.933+0000 7f1cc4a4f740 10 client.15381 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T11:19:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.933+0000 7f1cc4a4f740 10 client.15381 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:19:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.933+0000 7f1cc4a4f740 20 client.15381 populate_metadata read hostname 'smithi100' 2024-05-10T11:19:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.933+0000 7f1cc4a4f740 1 --2- 172.21.15.100:0/1260908053 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55fc6c01ec50 0x55fc6c0210e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.933+0000 7f1cc4a4f740 4 client.15381 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:19:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.933+0000 7f1cc4a4f740 1 -- 172.21.15.100:0/1260908053 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc6bf7b1e0 con 0x55fc6c01ec50 2024-05-10T11:19:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.933+0000 7f1cbaffd640 1 --2- 172.21.15.100:0/1260908053 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55fc6c01ec50 0x55fc6c0210e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.934+0000 7f1cbaffd640 1 --2- 172.21.15.100:0/1260908053 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55fc6c01ec50 0x55fc6c0210e0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43dfde422d73514c server_cookie=b46bbccdc8c4c3ce in_seq=0 out_seq=0 2024-05-10T11:19:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.934+0000 7f1c9affd640 10 client.15381 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:19:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.934+0000 7f1c9affd640 1 -- 172.21.15.100:0/1260908053 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fc6bf7b1e0 con 0x55fc6c01ec50 2024-05-10T11:19:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.934+0000 7f1c9affd640 10 client.15381 handle_command_reply: tid=0 2024-05-10T11:19:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.935+0000 7f1cc4a4f740 1 client.15381 shutdown 2024-05-10T11:19:14.946 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.935+0000 7f1cc4a4f740 1 -- 172.21.15.100:0/1260908053 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc6c008db0 msgr2=0x55fc6c012f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:14.946 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.935+0000 7f1cc4a4f740 1 --2- 172.21.15.100:0/1260908053 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc6c008db0 0x55fc6c012f30 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f1cac00be20 tx=0x7f1cac012820 comp rx=0 tx=0).stop 2024-05-10T11:19:14.946 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.935+0000 7f1cc4a4f740 1 -- 172.21.15.100:0/1260908053 shutdown_connections 2024-05-10T11:19:14.946 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.935+0000 7f1cc4a4f740 1 --2- 172.21.15.100:0/1260908053 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55fc6c01ec50 0x55fc6c0210e0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:14.946 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.935+0000 7f1cc4a4f740 1 --2- 172.21.15.100:0/1260908053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fc6c00e910 0x55fc6c01a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:14.946 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.935+0000 7f1cc4a4f740 1 --2- 172.21.15.100:0/1260908053 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc6c0097a0 0x55fc6c013470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:14.946 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.935+0000 7f1c9affd640 0 client.15381 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:19:14.946 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.935+0000 7f1cc4a4f740 1 --2- 172.21.15.100:0/1260908053 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc6c008db0 0x55fc6c012f30 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:14.946 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.935+0000 7f1cc4a4f740 1 -- 172.21.15.100:0/1260908053 >> 172.21.15.100:0/1260908053 conn(0x55fc6c004420 msgr2=0x55fc6c003580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:14.946 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.935+0000 7f1cc4a4f740 1 -- 172.21.15.100:0/1260908053 shutdown_connections 2024-05-10T11:19:14.946 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.935+0000 7f1cc4a4f740 1 -- 172.21.15.100:0/1260908053 wait complete. 2024-05-10T11:19:14.946 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.935+0000 7f1cc4a4f740 20 client.15381 trim_cache size 0 max 16384 2024-05-10T11:19:14.954 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.950+0000 7f1cc4a4f740 1 Processor -- start 2024-05-10T11:19:14.954 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.950+0000 7f1cc4a4f740 1 -- start start 2024-05-10T11:19:14.954 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.950+0000 7f1cc4a4f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc6c008db0 0x55fc6c013210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:14.954 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.950+0000 7f1cc4a4f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fc6c0097a0 0x55fc6c01a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:14.954 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.950+0000 7f1cc4a4f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc6c00e910 0x55fc6c003400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.950+0000 7f1cc4a4f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1cbc1afe50 con 0x55fc6c00e910 2024-05-10T11:19:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.950+0000 7f1cc4a4f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1cbc1ad720 con 0x55fc6c0097a0 2024-05-10T11:19:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.950+0000 7f1cc4a4f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1cbc1ab570 con 0x55fc6c008db0 2024-05-10T11:19:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.950+0000 7f1cbbfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc6c00e910 0x55fc6c003400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.950+0000 7f1cbbfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc6c00e910 0x55fc6c003400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53470/0 (socket says 172.21.15.100:53470) 2024-05-10T11:19:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.950+0000 7f1cbbfff640 1 -- 172.21.15.100:0/3069323944 learned_addr learned my addr 172.21.15.100:0/3069323944 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.950+0000 7f1cbb7fe640 1 --2- 172.21.15.100:0/3069323944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc6c008db0 0x55fc6c013210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.950+0000 7f1cbaffd640 1 --2- 172.21.15.100:0/3069323944 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fc6c0097a0 0x55fc6c01a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.950+0000 7f1cbbfff640 1 -- 172.21.15.100:0/3069323944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc6c008db0 msgr2=0x55fc6c013210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.950+0000 7f1cbbfff640 1 --2- 172.21.15.100:0/3069323944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc6c008db0 0x55fc6c013210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.950+0000 7f1cbbfff640 1 -- 172.21.15.100:0/3069323944 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fc6c0097a0 msgr2=0x55fc6c01a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.950+0000 7f1cbbfff640 1 --2- 172.21.15.100:0/3069323944 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fc6c0097a0 0x55fc6c01a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.951+0000 7f1cbbfff640 1 -- 172.21.15.100:0/3069323944 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc6c009b80 con 0x55fc6c00e910 2024-05-10T11:19:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.951+0000 7f1cbaffd640 1 --2- 172.21.15.100:0/3069323944 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fc6c0097a0 0x55fc6c01a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:19:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.951+0000 7f1cbbfff640 1 --2- 172.21.15.100:0/3069323944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc6c00e910 0x55fc6c003400 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f1cb4009230 tx=0x7f1cb4040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3a7cead8d47f03f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.951+0000 7f1cb17fa640 1 -- 172.21.15.100:0/3069323944 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb4035040 con 0x55fc6c00e910 2024-05-10T11:19:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.951+0000 7f1cb17fa640 1 -- 172.21.15.100:0/3069323944 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1cb404c070 con 0x55fc6c00e910 2024-05-10T11:19:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.951+0000 7f1cb17fa640 1 -- 172.21.15.100:0/3069323944 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb4051470 con 0x55fc6c00e910 2024-05-10T11:19:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.951+0000 7f1cc4a4f740 1 -- 172.21.15.100:0/3069323944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc6c00e910 msgr2=0x55fc6c003400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.951+0000 7f1cc4a4f740 1 --2- 172.21.15.100:0/3069323944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc6c00e910 0x55fc6c003400 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f1cb4009230 tx=0x7f1cb4040000 comp rx=0 tx=0).stop 2024-05-10T11:19:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.951+0000 7f1cc4a4f740 1 -- 172.21.15.100:0/3069323944 shutdown_connections 2024-05-10T11:19:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.951+0000 7f1cc4a4f740 1 --2- 172.21.15.100:0/3069323944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc6c00e910 0x55fc6c003400 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.951+0000 7f1cc4a4f740 1 --2- 172.21.15.100:0/3069323944 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fc6c0097a0 0x55fc6c01a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.952+0000 7f1cc4a4f740 1 --2- 172.21.15.100:0/3069323944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc6c008db0 0x55fc6c013210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.952+0000 7f1cc4a4f740 1 -- 172.21.15.100:0/3069323944 >> 172.21.15.100:0/3069323944 conn(0x55fc6c004420 msgr2=0x55fc6c002cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.952+0000 7f1cc4a4f740 1 -- 172.21.15.100:0/3069323944 shutdown_connections 2024-05-10T11:19:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.952+0000 7f1cc4a4f740 1 -- 172.21.15.100:0/3069323944 wait complete. 2024-05-10T11:19:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.952+0000 7f1cc4a4f740 1 -- start start 2024-05-10T11:19:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.952+0000 7f1cc4a4f740 1 Processor -- start 2024-05-10T11:19:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.952+0000 7f1cc4a4f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc6c008db0 0x55fc6c018070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.952+0000 7f1cc4a4f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc6c0097a0 0x55fc6c0185b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.952+0000 7f1cc4a4f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fc6c00e910 0x55fc6c019570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.952+0000 7f1cc4a4f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1cbc1abbe0 con 0x55fc6c008db0 2024-05-10T11:19:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.952+0000 7f1cc4a4f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1cbc1ad050 con 0x55fc6c00e910 2024-05-10T11:19:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.952+0000 7f1cc4a4f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1cbc1ac2b0 con 0x55fc6c0097a0 2024-05-10T11:19:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.952+0000 7f1cc4a4f740 10 client.0 _ll_register_callbacks cb 0x7f1cac00b800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:19:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.952+0000 7f1cbb7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc6c008db0 0x55fc6c018070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.952+0000 7f1cbb7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc6c008db0 0x55fc6c018070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53474/0 (socket says 172.21.15.100:53474) 2024-05-10T11:19:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.952+0000 7f1cbb7fe640 1 -- 172.21.15.100:0/3800191096 learned_addr learned my addr 172.21.15.100:0/3800191096 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.952+0000 7f1cbaffd640 1 --2- 172.21.15.100:0/3800191096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc6c0097a0 0x55fc6c0185b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.953+0000 7f1cbb7fe640 1 -- 172.21.15.100:0/3800191096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc6c0097a0 msgr2=0x55fc6c0185b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.953+0000 7f1cbb7fe640 1 --2- 172.21.15.100:0/3800191096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc6c0097a0 0x55fc6c0185b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.953+0000 7f1cbb7fe640 1 -- 172.21.15.100:0/3800191096 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fc6c00e910 msgr2=0x55fc6c019570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.953+0000 7f1cbb7fe640 1 --2- 172.21.15.100:0/3800191096 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fc6c00e910 0x55fc6c019570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.953+0000 7f1cbb7fe640 1 -- 172.21.15.100:0/3800191096 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc6c019b60 con 0x55fc6c008db0 2024-05-10T11:19:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.953+0000 7f1cbaffd640 1 --2- 172.21.15.100:0/3800191096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc6c0097a0 0x55fc6c0185b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:19:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.953+0000 7f1cbb7fe640 1 --2- 172.21.15.100:0/3800191096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc6c008db0 0x55fc6c018070 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f1cac003cb0 tx=0x7f1cac003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=238b97abb4f741d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.953+0000 7f1c9affd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:19:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.953+0000 7f1c9affd640 1 -- 172.21.15.100:0/3800191096 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cac021040 con 0x55fc6c008db0 2024-05-10T11:19:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.953+0000 7f1c9affd640 1 -- 172.21.15.100:0/3800191096 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f1cac005440 con 0x55fc6c008db0 2024-05-10T11:19:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.953+0000 7f1cc4a4f740 1 -- 172.21.15.100:0/3800191096 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cb404c070 con 0x55fc6c008db0 2024-05-10T11:19:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.953+0000 7f1c9affd640 1 -- 172.21.15.100:0/3800191096 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1cac038050 con 0x55fc6c008db0 2024-05-10T11:19:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.953+0000 7f1cc4a4f740 10 client.15387 fetch_fsmap learned FSMap version 27 2024-05-10T11:19:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.953+0000 7f1cc4a4f740 1 -- 172.21.15.100:0/3800191096 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fc6c0899c0 con 0x55fc6c008db0 2024-05-10T11:19:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.953+0000 7f1c9affd640 1 -- 172.21.15.100:0/3800191096 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1cb404c070 con 0x55fc6c008db0 2024-05-10T11:19:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.953+0000 7f1c9affd640 1 -- 172.21.15.100:0/3800191096 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f1cac004680 con 0x55fc6c008db0 2024-05-10T11:19:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.953+0000 7f1cc4a4f740 10 client.15387 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T11:19:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.953+0000 7f1cc4a4f740 10 client.15387 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:19:14.959 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.953+0000 7f1cc4a4f740 20 client.15387 populate_metadata read hostname 'smithi100' 2024-05-10T11:19:14.959 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.953+0000 7f1cc4a4f740 1 --2- 172.21.15.100:0/3800191096 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55fc6c08a390 0x55fc6c08c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:14.959 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.953+0000 7f1cc4a4f740 4 client.15387 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:19:14.959 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.953+0000 7f1cc4a4f740 1 -- 172.21.15.100:0/3800191096 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fc6bf75050 con 0x55fc6c08a390 2024-05-10T11:19:14.959 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.954+0000 7f1cbaffd640 1 --2- 172.21.15.100:0/3800191096 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55fc6c08a390 0x55fc6c08c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:14.959 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.954+0000 7f1c9affd640 10 client.15387 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:19:14.959 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.954+0000 7f1cbaffd640 1 --2- 172.21.15.100:0/3800191096 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55fc6c08a390 0x55fc6c08c7b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ebec59ac2453a0f server_cookie=54a1d94848fa45de in_seq=0 out_seq=0 2024-05-10T11:19:14.960 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.958+0000 7f1c9affd640 1 -- 172.21.15.100:0/3800191096 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fc6bf75050 con 0x55fc6c08a390 2024-05-10T11:19:14.960 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.958+0000 7f1c9affd640 10 client.15387 handle_command_reply: tid=0 2024-05-10T11:19:14.960 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.958+0000 7f1cc4a4f740 1 client.15387 shutdown 2024-05-10T11:19:14.960 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.958+0000 7f1cc4a4f740 1 -- 172.21.15.100:0/3800191096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc6c008db0 msgr2=0x55fc6c018070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:14.960 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.958+0000 7f1cc4a4f740 1 --2- 172.21.15.100:0/3800191096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc6c008db0 0x55fc6c018070 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f1cac003cb0 tx=0x7f1cac003820 comp rx=0 tx=0).stop 2024-05-10T11:19:14.960 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.958+0000 7f1cc4a4f740 1 -- 172.21.15.100:0/3800191096 shutdown_connections 2024-05-10T11:19:14.961 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.958+0000 7f1cc4a4f740 1 --2- 172.21.15.100:0/3800191096 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55fc6c08a390 0x55fc6c08c7b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:14.961 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.958+0000 7f1cc4a4f740 1 --2- 172.21.15.100:0/3800191096 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fc6c00e910 0x55fc6c019570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:14.961 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.958+0000 7f1cc4a4f740 1 --2- 172.21.15.100:0/3800191096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc6c0097a0 0x55fc6c0185b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:14.961 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.958+0000 7f1cc4a4f740 1 --2- 172.21.15.100:0/3800191096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc6c008db0 0x55fc6c018070 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:14.961 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.958+0000 7f1c9affd640 0 client.15387 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:19:14.961 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.958+0000 7f1cc4a4f740 1 -- 172.21.15.100:0/3800191096 >> 172.21.15.100:0/3800191096 conn(0x55fc6c004420 msgr2=0x55fc6c003580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:14.961 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.958+0000 7f1cc4a4f740 1 -- 172.21.15.100:0/3800191096 shutdown_connections 2024-05-10T11:19:14.961 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.958+0000 7f1cc4a4f740 1 -- 172.21.15.100:0/3800191096 wait complete. 2024-05-10T11:19:14.961 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.958+0000 7f1cc4a4f740 20 client.15387 trim_cache size 0 max 16384 2024-05-10T11:19:14.961 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:19:14.961 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:19:14.962 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:19:14.962 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:19:14.962 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.958+0000 7f1cc1d32640 1 -- 172.21.15.100:0/3712136497 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f1ca80b28a0 msgr2=0x7f1ca80b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:14.963 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.958+0000 7f1cc1d32640 1 --2- 172.21.15.100:0/3712136497 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f1ca80b28a0 0x7f1ca80b4d60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1cb4006010 tx=0x7f1cb4031420 comp rx=0 tx=0).stop 2024-05-10T11:19:14.963 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.958+0000 7f1cc1d32640 1 -- 172.21.15.100:0/3712136497 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cbc102b00 msgr2=0x7f1cbc103fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:14.963 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.958+0000 7f1cc1d32640 1 --2- 172.21.15.100:0/3712136497 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cbc102b00 0x7f1cbc103fb0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1ca400ebc0 tx=0x7f1ca400c6a0 comp rx=0 tx=0).stop 2024-05-10T11:19:14.963 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.958+0000 7f1cc1d32640 1 -- 172.21.15.100:0/3712136497 shutdown_connections 2024-05-10T11:19:14.963 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.958+0000 7f1cc1d32640 1 --2- 172.21.15.100:0/3712136497 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f1ca80b28a0 0x7f1ca80b4d60 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:14.963 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.958+0000 7f1cc1d32640 1 --2- 172.21.15.100:0/3712136497 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1cbc1034c0 0x7f1cbc1044f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:14.963 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.958+0000 7f1cc1d32640 1 --2- 172.21.15.100:0/3712136497 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cbc102b00 0x7f1cbc103fb0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:14.963 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.958+0000 7f1cc1d32640 1 --2- 172.21.15.100:0/3712136497 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1cbc101910 0x7f1cbc105970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:14.963 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.958+0000 7f1cc1d32640 1 -- 172.21.15.100:0/3712136497 >> 172.21.15.100:0/3712136497 conn(0x7f1cbc0fcea0 msgr2=0x7f1cbc06b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:14.963 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.959+0000 7f1cc1d32640 1 -- 172.21.15.100:0/3712136497 shutdown_connections 2024-05-10T11:19:14.963 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:14.959+0000 7f1cc1d32640 1 -- 172.21.15.100:0/3712136497 wait complete. 2024-05-10T11:19:15.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:15.880+0000 7fb263292700 1 -- 172.21.15.100:0/2131344471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb25c104130 msgr2=0x7fb25c108140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:15.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:15.880+0000 7fb263292700 1 --2- 172.21.15.100:0/2131344471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb25c104130 0x7fb25c108140 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fb24c0099c0 tx=0x7fb24c009cd0 comp rx=0 tx=0).stop 2024-05-10T11:19:15.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:15.880+0000 7fb263292700 1 -- 172.21.15.100:0/2131344471 shutdown_connections 2024-05-10T11:19:15.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:15.880+0000 7fb263292700 1 --2- 172.21.15.100:0/2131344471 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb25c108870 0x7fb25c1103f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:15.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:15.880+0000 7fb263292700 1 --2- 172.21.15.100:0/2131344471 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb25c103780 0x7fb25c103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:15.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:15.880+0000 7fb263292700 1 --2- 172.21.15.100:0/2131344471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb25c104130 0x7fb25c108140 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:15.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:15.880+0000 7fb263292700 1 -- 172.21.15.100:0/2131344471 >> 172.21.15.100:0/2131344471 conn(0x7fb25c07a610 msgr2=0x7fb25c07aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:15.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:15.880+0000 7fb263292700 1 -- 172.21.15.100:0/2131344471 shutdown_connections 2024-05-10T11:19:15.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:15.880+0000 7fb263292700 1 -- 172.21.15.100:0/2131344471 wait complete. 2024-05-10T11:19:15.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:15.880+0000 7fb263292700 1 Processor -- start 2024-05-10T11:19:15.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:15.880+0000 7fb263292700 1 -- start start 2024-05-10T11:19:15.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:15.881+0000 7fb263292700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb25c103780 0x7fb25c0757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:15.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:15.881+0000 7fb263292700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb25c104130 0x7fb25c075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:15.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:15.881+0000 7fb263292700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb25c108870 0x7fb25c076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:15.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:15.881+0000 7fb263292700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb25c113500 con 0x7fb25c103780 2024-05-10T11:19:15.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:15.881+0000 7fb263292700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb25c113380 con 0x7fb25c104130 2024-05-10T11:19:15.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:15.881+0000 7fb263292700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb25c113680 con 0x7fb25c108870 2024-05-10T11:19:15.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:15.881+0000 7fb26102e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb25c103780 0x7fb25c0757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:15.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:15.881+0000 7fb26102e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb25c103780 0x7fb25c0757a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53492/0 (socket says 172.21.15.100:53492) 2024-05-10T11:19:15.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:15.881+0000 7fb26102e700 1 -- 172.21.15.100:0/2592757964 learned_addr learned my addr 172.21.15.100:0/2592757964 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:15.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:15.881+0000 7fb26082d700 1 --2- 172.21.15.100:0/2592757964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb25c104130 0x7fb25c075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:15.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:15.881+0000 7fb26102e700 1 -- 172.21.15.100:0/2592757964 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb25c108870 msgr2=0x7fb25c076220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:15.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:15.881+0000 7fb26182f700 1 --2- 172.21.15.100:0/2592757964 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb25c108870 0x7fb25c076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:15.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:15.881+0000 7fb26102e700 1 --2- 172.21.15.100:0/2592757964 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb25c108870 0x7fb25c076220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:15.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:15.881+0000 7fb26102e700 1 -- 172.21.15.100:0/2592757964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb25c104130 msgr2=0x7fb25c075ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:15.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:15.881+0000 7fb26102e700 1 --2- 172.21.15.100:0/2592757964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb25c104130 0x7fb25c075ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:15.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:15.881+0000 7fb26102e700 1 -- 172.21.15.100:0/2592757964 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb25c1a7f60 con 0x7fb25c103780 2024-05-10T11:19:15.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:15.881+0000 7fb26102e700 1 --2- 172.21.15.100:0/2592757964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb25c103780 0x7fb25c0757a0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fb25800c970 tx=0x7fb25800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:15.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:15.881+0000 7fb2527fc700 1 -- 172.21.15.100:0/2592757964 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb258007aa0 con 0x7fb25c103780 2024-05-10T11:19:15.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:15.881+0000 7fb2527fc700 1 -- 172.21.15.100:0/2592757964 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb258004cc0 con 0x7fb25c103780 2024-05-10T11:19:15.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:15.881+0000 7fb263292700 1 -- 172.21.15.100:0/2592757964 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb25c1a8250 con 0x7fb25c103780 2024-05-10T11:19:15.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:15.881+0000 7fb2527fc700 1 -- 172.21.15.100:0/2592757964 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb258005450 con 0x7fb25c103780 2024-05-10T11:19:15.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:15.882+0000 7fb263292700 1 -- 172.21.15.100:0/2592757964 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb25c1a8710 con 0x7fb25c103780 2024-05-10T11:19:15.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:15.882+0000 7fb263292700 1 -- 172.21.15.100:0/2592757964 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb25c10ba40 con 0x7fb25c103780 2024-05-10T11:19:15.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:15.885+0000 7fb2527fc700 1 -- 172.21.15.100:0/2592757964 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fb258007c40 con 0x7fb25c103780 2024-05-10T11:19:15.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:15.885+0000 7fb2527fc700 1 --2- 172.21.15.100:0/2592757964 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fb2480b2880 0x7fb2480b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:15.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:15.885+0000 7fb26082d700 1 --2- 172.21.15.100:0/2592757964 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fb2480b2880 0x7fb2480b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:15.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:15.886+0000 7fb2527fc700 1 -- 172.21.15.100:0/2592757964 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fb2580e25f0 con 0x7fb25c103780 2024-05-10T11:19:15.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:15.886+0000 7fb26082d700 1 --2- 172.21.15.100:0/2592757964 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fb2480b2880 0x7fb2480b4d40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb24c0058e0 tx=0x7fb24c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:15.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:15.889+0000 7fb2527fc700 1 -- 172.21.15.100:0/2592757964 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fb2580aa0d0 con 0x7fb25c103780 2024-05-10T11:19:16.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:19:15 smithi178 ceph-mon[33359]: pgmap v446: 129 pgs: 129 active+clean; 662 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-10T11:19:16.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:19:15 smithi188 ceph-mon[34075]: pgmap v446: 129 pgs: 129 active+clean; 662 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-10T11:19:16.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:19:15 smithi100 ceph-mon[30296]: pgmap v446: 129 pgs: 129 active+clean; 662 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-10T11:19:16.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:16.141+0000 7fb263292700 1 -- 172.21.15.100:0/2592757964 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb25c06a190 con 0x7fb25c103780 2024-05-10T11:19:16.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:16.142+0000 7fb2527fc700 1 -- 172.21.15.100:0/2592757964 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11894 (secure 0 0 0) 0x7fb2580a99d0 con 0x7fb25c103780 2024-05-10T11:19:16.142 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:19:16.143 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2072225012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2072225012},{"type":"v1","addr":"172.21.15.100:6835","nonce":2072225012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/401939540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":401939540},{"type":"v1","addr":"172.21.15.100:6837","nonce":401939540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14571,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/3946311025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3946311025},{"type":"v1","addr":"172.21.15.100:6839","nonce":3946311025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3736371660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3736371660},{"type":"v1","addr":"172.21.15.100:6841","nonce":3736371660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/2741810791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2741810791},{"type":"v1","addr":"172.21.15.178:6833","nonce":2741810791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1704891041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1704891041},{"type":"v1","addr":"172.21.15.188:6833","nonce":1704891041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T11:14:27.242452+0000","modified":"2024-05-10T11:18:23.454860+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24439,"mds_1":24437,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/2124076312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2124076312},{"type":"v1","addr":"172.21.15.178:6835","nonce":2124076312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/3224358425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3224358425},{"type":"v1","addr":"172.21.15.178:6837","nonce":3224358425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/515547690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":515547690},{"type":"v1","addr":"172.21.15.188:6835","nonce":515547690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/1943342620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1943342620},{"type":"v1","addr":"172.21.15.188:6837","nonce":1943342620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.178:6839/3760985404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3760985404},{"type":"v1","addr":"172.21.15.178:6839","nonce":3760985404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.188:6839/3194917070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":3194917070},{"type":"v1","addr":"172.21.15.188:6839","nonce":3194917070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24439,"qdb_cluster":[24427,24437,24439]},"id":1}]} 2024-05-10T11:19:16.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:16.144+0000 7fb263292700 1 -- 172.21.15.100:0/2592757964 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fb2480b2880 msgr2=0x7fb2480b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:16.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:16.144+0000 7fb263292700 1 --2- 172.21.15.100:0/2592757964 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fb2480b2880 0x7fb2480b4d40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb24c0058e0 tx=0x7fb24c005fb0 comp rx=0 tx=0).stop 2024-05-10T11:19:16.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:16.144+0000 7fb263292700 1 -- 172.21.15.100:0/2592757964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb25c103780 msgr2=0x7fb25c0757a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:16.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:16.144+0000 7fb263292700 1 --2- 172.21.15.100:0/2592757964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb25c103780 0x7fb25c0757a0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fb25800c970 tx=0x7fb25800cc80 comp rx=0 tx=0).stop 2024-05-10T11:19:16.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:16.144+0000 7fb263292700 1 -- 172.21.15.100:0/2592757964 shutdown_connections 2024-05-10T11:19:16.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:16.144+0000 7fb263292700 1 --2- 172.21.15.100:0/2592757964 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fb2480b2880 0x7fb2480b4d40 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:16.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:16.144+0000 7fb263292700 1 --2- 172.21.15.100:0/2592757964 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb25c108870 0x7fb25c076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:16.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:16.144+0000 7fb263292700 1 --2- 172.21.15.100:0/2592757964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb25c103780 0x7fb25c0757a0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:16.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:16.144+0000 7fb263292700 1 --2- 172.21.15.100:0/2592757964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb25c104130 0x7fb25c075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:16.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:16.144+0000 7fb263292700 1 -- 172.21.15.100:0/2592757964 >> 172.21.15.100:0/2592757964 conn(0x7fb25c07a610 msgr2=0x7fb25c101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:16.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:16.144+0000 7fb263292700 1 -- 172.21.15.100:0/2592757964 shutdown_connections 2024-05-10T11:19:16.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:16.144+0000 7fb263292700 1 -- 172.21.15.100:0/2592757964 wait complete. 2024-05-10T11:19:16.146 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 27 2024-05-10T11:19:16.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:19:16 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/2592757964' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:19:16.903 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-10T11:19:17.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:19:16 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/2592757964' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:19:17.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:19:16 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/2592757964' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:19:17.150 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:19:18.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:19:17 smithi178 ceph-mon[33359]: pgmap v447: 129 pgs: 129 active+clean; 662 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-10T11:19:18.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:19:17 smithi188 ceph-mon[34075]: pgmap v447: 129 pgs: 129 active+clean; 662 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-10T11:19:18.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:19:17 smithi100 ceph-mon[30296]: pgmap v447: 129 pgs: 129 active+clean; 662 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-10T11:19:18.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.947+0000 7fa0fb59e700 1 -- 172.21.15.100:0/3336026207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0fc102530 msgr2=0x7fa0fc102910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:18.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.948+0000 7fa0fb59e700 1 --2- 172.21.15.100:0/3336026207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0fc102530 0x7fa0fc102910 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec0099c0 tx=0x7fa0ec009cd0 comp rx=0 tx=0).stop 2024-05-10T11:19:18.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.948+0000 7fa0fb59e700 1 -- 172.21.15.100:0/3336026207 shutdown_connections 2024-05-10T11:19:18.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.948+0000 7fa0fb59e700 1 --2- 172.21.15.100:0/3336026207 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0fc107610 0x7fa0fc10f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:18.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.948+0000 7fa0fb59e700 1 --2- 172.21.15.100:0/3336026207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0fc102530 0x7fa0fc102910 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:18.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.948+0000 7fa0fb59e700 1 --2- 172.21.15.100:0/3336026207 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0fc102ee0 0x7fa0fc106f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:18.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.948+0000 7fa0fb59e700 1 -- 172.21.15.100:0/3336026207 >> 172.21.15.100:0/3336026207 conn(0x7fa0fc0fddc0 msgr2=0x7fa0fc1001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:18.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.948+0000 7fa0fb59e700 1 -- 172.21.15.100:0/3336026207 shutdown_connections 2024-05-10T11:19:18.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.948+0000 7fa0fb59e700 1 -- 172.21.15.100:0/3336026207 wait complete. 2024-05-10T11:19:18.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.948+0000 7fa0fb59e700 1 Processor -- start 2024-05-10T11:19:18.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.949+0000 7fa0fb59e700 1 -- start start 2024-05-10T11:19:18.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.949+0000 7fa0fb59e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0fc102530 0x7fa0fc19f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:18.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.949+0000 7fa0fb59e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0fc102ee0 0x7fa0fc19fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:18.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.949+0000 7fa0fb59e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0fc107610 0x7fa0fc1997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:18.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.949+0000 7fa0fb59e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa0fc1123d0 con 0x7fa0fc107610 2024-05-10T11:19:18.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.949+0000 7fa0fb59e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa0fc112250 con 0x7fa0fc102530 2024-05-10T11:19:18.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.949+0000 7fa0fb59e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa0fc112550 con 0x7fa0fc102ee0 2024-05-10T11:19:18.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.949+0000 7fa0fad9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0fc107610 0x7fa0fc1997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:18.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.949+0000 7fa0fad9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0fc107610 0x7fa0fc1997f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55662/0 (socket says 172.21.15.100:55662) 2024-05-10T11:19:18.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.949+0000 7fa0fad9d700 1 -- 172.21.15.100:0/1507688910 learned_addr learned my addr 172.21.15.100:0/1507688910 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:18.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.949+0000 7fa0f9d9b700 1 --2- 172.21.15.100:0/1507688910 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0fc102ee0 0x7fa0fc19fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:18.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.949+0000 7fa0fa59c700 1 --2- 172.21.15.100:0/1507688910 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0fc102530 0x7fa0fc19f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:18.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.949+0000 7fa0fad9d700 1 -- 172.21.15.100:0/1507688910 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0fc102ee0 msgr2=0x7fa0fc19fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:18.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.949+0000 7fa0fad9d700 1 --2- 172.21.15.100:0/1507688910 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0fc102ee0 0x7fa0fc19fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:18.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.949+0000 7fa0fad9d700 1 -- 172.21.15.100:0/1507688910 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0fc102530 msgr2=0x7fa0fc19f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:18.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.949+0000 7fa0fad9d700 1 --2- 172.21.15.100:0/1507688910 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0fc102530 0x7fa0fc19f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:18.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.949+0000 7fa0fad9d700 1 -- 172.21.15.100:0/1507688910 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0fc199fd0 con 0x7fa0fc107610 2024-05-10T11:19:18.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.950+0000 7fa0fad9d700 1 --2- 172.21.15.100:0/1507688910 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0fc107610 0x7fa0fc1997f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fa0f400b970 tx=0x7fa0f400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:18.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.950+0000 7fa0eb7fe700 1 -- 172.21.15.100:0/1507688910 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f400d860 con 0x7fa0fc107610 2024-05-10T11:19:18.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.950+0000 7fa0eb7fe700 1 -- 172.21.15.100:0/1507688910 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa0f400e430 con 0x7fa0fc107610 2024-05-10T11:19:18.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.950+0000 7fa0fb59e700 1 -- 172.21.15.100:0/1507688910 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0fc19a2c0 con 0x7fa0fc107610 2024-05-10T11:19:18.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.950+0000 7fa0eb7fe700 1 -- 172.21.15.100:0/1507688910 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f4017660 con 0x7fa0fc107610 2024-05-10T11:19:18.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.950+0000 7fa0fb59e700 1 -- 172.21.15.100:0/1507688910 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0fc1a6480 con 0x7fa0fc107610 2024-05-10T11:19:18.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.952+0000 7fa0eb7fe700 1 -- 172.21.15.100:0/1507688910 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fa0f400da00 con 0x7fa0fc107610 2024-05-10T11:19:18.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.953+0000 7fa0eb7fe700 1 --2- 172.21.15.100:0/1507688910 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa0e40b2880 0x7fa0e40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:18.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.953+0000 7fa0fa59c700 1 --2- 172.21.15.100:0/1507688910 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa0e40b2880 0x7fa0e40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:18.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.953+0000 7fa0eb7fe700 1 -- 172.21.15.100:0/1507688910 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fa0f40e21a0 con 0x7fa0fc107610 2024-05-10T11:19:18.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.954+0000 7fa10b492b80 1 Processor -- start 2024-05-10T11:19:18.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.954+0000 7fa0fa59c700 1 --2- 172.21.15.100:0/1507688910 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa0e40b2880 0x7fa0e40b4d40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec009990 tx=0x7fa0ec000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:18.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.954+0000 7fa10b492b80 1 -- start start 2024-05-10T11:19:18.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.954+0000 7fa10b492b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578a0180290 0x5578a0180670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:18.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.954+0000 7fa10b492b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5578a0180bb0 0x5578a017a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:18.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.954+0000 7fa10b492b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578a017ad50 0x5578a017b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:18.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.954+0000 7fa10b492b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5578a00cc940 con 0x5578a017ad50 2024-05-10T11:19:18.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.954+0000 7fa10b492b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5578a0031da0 con 0x5578a0180bb0 2024-05-10T11:19:18.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.954+0000 7fa10b492b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5578a00c8900 con 0x5578a0180290 2024-05-10T11:19:18.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.954+0000 7fa0fad9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578a017ad50 0x5578a017b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:18.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.954+0000 7fa0fad9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578a017ad50 0x5578a017b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55670/0 (socket says 172.21.15.100:55670) 2024-05-10T11:19:18.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.954+0000 7fa0fad9d700 1 -- 172.21.15.100:0/3314017819 learned_addr learned my addr 172.21.15.100:0/3314017819 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:18.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.955+0000 7fa0f9d9b700 1 --2- 172.21.15.100:0/3314017819 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5578a0180bb0 0x5578a017a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:18.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.955+0000 7fa0fa59c700 1 --2- 172.21.15.100:0/3314017819 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578a0180290 0x5578a0180670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:18.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.955+0000 7fa0fad9d700 1 -- 172.21.15.100:0/3314017819 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578a0180290 msgr2=0x5578a0180670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:18.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.955+0000 7fa0fad9d700 1 --2- 172.21.15.100:0/3314017819 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578a0180290 0x5578a0180670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:18.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.955+0000 7fa0fad9d700 1 -- 172.21.15.100:0/3314017819 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5578a0180bb0 msgr2=0x5578a017a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:18.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.955+0000 7fa0fad9d700 1 --2- 172.21.15.100:0/3314017819 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5578a0180bb0 0x5578a017a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:18.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.955+0000 7fa0fad9d700 1 -- 172.21.15.100:0/3314017819 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578a00198d0 con 0x5578a017ad50 2024-05-10T11:19:18.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.955+0000 7fa0f9d9b700 1 --2- 172.21.15.100:0/3314017819 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5578a0180bb0 0x5578a017a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:19:18.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.955+0000 7fa0fa59c700 1 --2- 172.21.15.100:0/3314017819 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578a0180290 0x5578a0180670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:19:18.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.955+0000 7fa0fad9d700 1 --2- 172.21.15.100:0/3314017819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578a017ad50 0x5578a017b170 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa0f40a54e0 tx=0x7fa0f40a9b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96ff335fbd11602e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:18.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.955+0000 7fa0e8ff9700 1 -- 172.21.15.100:0/3314017819 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f40b22c0 con 0x5578a017ad50 2024-05-10T11:19:18.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.955+0000 7fa0e8ff9700 1 -- 172.21.15.100:0/3314017819 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa0f40a5a50 con 0x5578a017ad50 2024-05-10T11:19:18.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.955+0000 7fa0e8ff9700 1 -- 172.21.15.100:0/3314017819 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f40a96c0 con 0x5578a017ad50 2024-05-10T11:19:18.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.956+0000 7fa10b492b80 1 -- 172.21.15.100:0/3314017819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578a017ad50 msgr2=0x5578a017b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:18.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.956+0000 7fa10b492b80 1 --2- 172.21.15.100:0/3314017819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578a017ad50 0x5578a017b170 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa0f40a54e0 tx=0x7fa0f40a9b60 comp rx=0 tx=0).stop 2024-05-10T11:19:18.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.956+0000 7fa10b492b80 1 -- 172.21.15.100:0/3314017819 shutdown_connections 2024-05-10T11:19:18.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.956+0000 7fa10b492b80 1 --2- 172.21.15.100:0/3314017819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578a017ad50 0x5578a017b170 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:18.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.956+0000 7fa10b492b80 1 --2- 172.21.15.100:0/3314017819 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578a0180290 0x5578a0180670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:18.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.956+0000 7fa10b492b80 1 --2- 172.21.15.100:0/3314017819 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5578a0180bb0 0x5578a017a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:18.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.956+0000 7fa10b492b80 1 -- 172.21.15.100:0/3314017819 >> 172.21.15.100:0/3314017819 conn(0x5578a00a6b70 msgr2=0x5578a01752d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:18.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.956+0000 7fa10b492b80 1 -- 172.21.15.100:0/3314017819 shutdown_connections 2024-05-10T11:19:18.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.956+0000 7fa10b492b80 1 -- 172.21.15.100:0/3314017819 wait complete. 2024-05-10T11:19:18.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.956+0000 7fa10b492b80 1 -- start start 2024-05-10T11:19:18.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.957+0000 7fa10b492b80 1 Processor -- start 2024-05-10T11:19:18.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.957+0000 7fa10b492b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578a017ad50 0x5578a0186850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:18.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.957+0000 7fa10b492b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578a0180290 0x5578a0183bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:18.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.957+0000 7fa10b492b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5578a0180bb0 0x5578a0184130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:18.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.957+0000 7fa10b492b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5578a01875e0 con 0x5578a017ad50 2024-05-10T11:19:18.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.957+0000 7fa10b492b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5578a0085ed0 con 0x5578a0180bb0 2024-05-10T11:19:18.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.957+0000 7fa10b492b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5578a0187760 con 0x5578a0180290 2024-05-10T11:19:18.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.957+0000 7fa10b492b80 10 client.0 _ll_register_callbacks cb 0x5578a00605b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:19:18.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.957+0000 7fa0fa59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578a017ad50 0x5578a0186850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:18.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.957+0000 7fa0fa59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578a017ad50 0x5578a0186850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55676/0 (socket says 172.21.15.100:55676) 2024-05-10T11:19:18.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.957+0000 7fa0fa59c700 1 -- 172.21.15.100:0/3660581143 learned_addr learned my addr 172.21.15.100:0/3660581143 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:18.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.957+0000 7fa0fad9d700 1 --2- 172.21.15.100:0/3660581143 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5578a0180bb0 0x5578a0184130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:18.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.957+0000 7fa0f9d9b700 1 --2- 172.21.15.100:0/3660581143 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578a0180290 0x5578a0183bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:18.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.957+0000 7fa0fa59c700 1 -- 172.21.15.100:0/3660581143 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578a0180290 msgr2=0x5578a0183bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:18.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.957+0000 7fa0fa59c700 1 --2- 172.21.15.100:0/3660581143 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578a0180290 0x5578a0183bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:18.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.957+0000 7fa0fa59c700 1 -- 172.21.15.100:0/3660581143 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5578a0180bb0 msgr2=0x5578a0184130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:18.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.957+0000 7fa0fa59c700 1 --2- 172.21.15.100:0/3660581143 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5578a0180bb0 0x5578a0184130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:18.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.957+0000 7fa0fa59c700 1 -- 172.21.15.100:0/3660581143 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578a0083760 con 0x5578a017ad50 2024-05-10T11:19:18.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.957+0000 7fa0f9d9b700 1 --2- 172.21.15.100:0/3660581143 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578a0180290 0x5578a0183bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:19:18.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.958+0000 7fa0fa59c700 1 --2- 172.21.15.100:0/3660581143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578a017ad50 0x5578a0186850 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec0099c0 tx=0x7fa0ec009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8d2d2e16eddb82b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:18.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.958+0000 7fa0e1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:19:18.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.958+0000 7fa0e1ffb700 1 -- 172.21.15.100:0/3660581143 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec033060 con 0x5578a017ad50 2024-05-10T11:19:18.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.958+0000 7fa10b492b80 1 -- 172.21.15.100:0/3660581143 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0f40a5a50 con 0x5578a017ad50 2024-05-10T11:19:18.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.958+0000 7fa0e1ffb700 1 -- 172.21.15.100:0/3660581143 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fa0ec003b10 con 0x5578a017ad50 2024-05-10T11:19:18.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.958+0000 7fa0e1ffb700 1 -- 172.21.15.100:0/3660581143 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa0ec03a050 con 0x5578a017ad50 2024-05-10T11:19:18.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.958+0000 7fa10b492b80 10 client.15405 fetch_fsmap learned FSMap version 27 2024-05-10T11:19:18.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.958+0000 7fa10b492b80 1 -- 172.21.15.100:0/3660581143 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5578a0191de0 con 0x5578a017ad50 2024-05-10T11:19:18.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.958+0000 7fa0e1ffb700 1 -- 172.21.15.100:0/3660581143 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa0f40a5a50 con 0x5578a017ad50 2024-05-10T11:19:18.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.958+0000 7fa0e1ffb700 1 -- 172.21.15.100:0/3660581143 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa0ec02d070 con 0x5578a017ad50 2024-05-10T11:19:18.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.958+0000 7fa10b492b80 10 client.15405 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T11:19:18.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.958+0000 7fa10b492b80 10 client.15405 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-10T11:19:18.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.958+0000 7fa10b492b80 20 client.15405 populate_metadata read hostname 'smithi100' 2024-05-10T11:19:18.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.958+0000 7fa10b492b80 1 --2- 172.21.15.100:0/3660581143 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x5578a0194370 0x5578a01967a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:18.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.958+0000 7fa10b492b80 4 client.15405 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:19:18.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.958+0000 7fa10b492b80 1 -- 172.21.15.100:0/3660581143 --> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578a00da320 con 0x5578a0194370 2024-05-10T11:19:18.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.959+0000 7fa0f9d9b700 1 --2- 172.21.15.100:0/3660581143 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x5578a0194370 0x5578a01967a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:18.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.959+0000 7fa0e1ffb700 10 client.15405 ms_handle_connect on v2:172.21.15.188:6836/1943342620 2024-05-10T11:19:18.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.959+0000 7fa0f9d9b700 1 --2- 172.21.15.100:0/3660581143 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x5578a0194370 0x5578a01967a0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cd343ebf0d394330 server_cookie=bb43396b07d32734 in_seq=0 out_seq=0 2024-05-10T11:19:18.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.960+0000 7fa0e1ffb700 1 -- 172.21.15.100:0/3660581143 <== mds.2 v2:172.21.15.188:6836/1943342620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5578a00da320 con 0x5578a0194370 2024-05-10T11:19:18.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.960+0000 7fa0e1ffb700 10 client.15405 handle_command_reply: tid=0 2024-05-10T11:19:18.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.960+0000 7fa10b492b80 1 client.15405 shutdown 2024-05-10T11:19:18.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.960+0000 7fa10b492b80 1 -- 172.21.15.100:0/3660581143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578a017ad50 msgr2=0x5578a0186850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:18.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.960+0000 7fa10b492b80 1 --2- 172.21.15.100:0/3660581143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578a017ad50 0x5578a0186850 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec0099c0 tx=0x7fa0ec009310 comp rx=0 tx=0).stop 2024-05-10T11:19:18.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.960+0000 7fa10b492b80 1 -- 172.21.15.100:0/3660581143 shutdown_connections 2024-05-10T11:19:18.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.960+0000 7fa10b492b80 1 --2- 172.21.15.100:0/3660581143 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x5578a0194370 0x5578a01967a0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:18.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.960+0000 7fa10b492b80 1 --2- 172.21.15.100:0/3660581143 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5578a0180bb0 0x5578a0184130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:18.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.960+0000 7fa10b492b80 1 --2- 172.21.15.100:0/3660581143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578a017ad50 0x5578a0186850 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:18.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.960+0000 7fa0e1ffb700 0 client.15405 ms_handle_reset on v2:172.21.15.188:6836/1943342620 2024-05-10T11:19:18.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.960+0000 7fa10b492b80 1 --2- 172.21.15.100:0/3660581143 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578a0180290 0x5578a0183bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:18.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.960+0000 7fa10b492b80 1 -- 172.21.15.100:0/3660581143 >> 172.21.15.100:0/3660581143 conn(0x5578a00a6b70 msgr2=0x5578a0174960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:18.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.960+0000 7fa10b492b80 1 -- 172.21.15.100:0/3660581143 shutdown_connections 2024-05-10T11:19:18.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.960+0000 7fa10b492b80 1 -- 172.21.15.100:0/3660581143 wait complete. 2024-05-10T11:19:18.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.960+0000 7fa10b492b80 20 client.15405 trim_cache size 0 max 16384 2024-05-10T11:19:18.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.975+0000 7fa10b492b80 1 Processor -- start 2024-05-10T11:19:18.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.975+0000 7fa10b492b80 1 -- start start 2024-05-10T11:19:18.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.975+0000 7fa10b492b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5578a017ad50 0x5578a017a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:18.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.975+0000 7fa10b492b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578a0180290 0x5578a0190670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:18.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.976+0000 7fa10b492b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578a0180bb0 0x5578a0185b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:18.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.976+0000 7fa10b492b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa0fc1b1790 con 0x5578a0180290 2024-05-10T11:19:18.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.976+0000 7fa10b492b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa0fc1af020 con 0x5578a017ad50 2024-05-10T11:19:18.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.976+0000 7fa10b492b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa0fc1ace70 con 0x5578a0180bb0 2024-05-10T11:19:18.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.976+0000 7fa0f9d9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578a0180290 0x5578a0190670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:18.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.976+0000 7fa0f9d9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578a0180290 0x5578a0190670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55684/0 (socket says 172.21.15.100:55684) 2024-05-10T11:19:18.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.976+0000 7fa0f9d9b700 1 -- 172.21.15.100:0/1906735370 learned_addr learned my addr 172.21.15.100:0/1906735370 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:18.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.976+0000 7fa0fad9d700 1 --2- 172.21.15.100:0/1906735370 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578a0180bb0 0x5578a0185b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:18.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.976+0000 7fa0fa59c700 1 --2- 172.21.15.100:0/1906735370 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5578a017ad50 0x5578a017a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:18.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.976+0000 7fa0f9d9b700 1 -- 172.21.15.100:0/1906735370 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578a0180bb0 msgr2=0x5578a0185b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:18.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.976+0000 7fa0f9d9b700 1 --2- 172.21.15.100:0/1906735370 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578a0180bb0 0x5578a0185b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:18.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.976+0000 7fa0f9d9b700 1 -- 172.21.15.100:0/1906735370 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5578a017ad50 msgr2=0x5578a017a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:18.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.976+0000 7fa0f9d9b700 1 --2- 172.21.15.100:0/1906735370 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5578a017ad50 0x5578a017a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:18.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.976+0000 7fa0f9d9b700 1 -- 172.21.15.100:0/1906735370 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578a0196e20 con 0x5578a0180290 2024-05-10T11:19:18.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.976+0000 7fa0fa59c700 1 --2- 172.21.15.100:0/1906735370 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5578a017ad50 0x5578a017a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:19:18.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.976+0000 7fa0f9d9b700 1 --2- 172.21.15.100:0/1906735370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578a0180290 0x5578a0190670 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc0ff3a0 tx=0x7fa0f000a380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63ca5c08e598a28f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:18.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.976+0000 7fa0e3fff700 1 -- 172.21.15.100:0/1906735370 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f001f070 con 0x5578a0180290 2024-05-10T11:19:18.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.976+0000 7fa0e3fff700 1 -- 172.21.15.100:0/1906735370 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa0f00044b0 con 0x5578a0180290 2024-05-10T11:19:18.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.976+0000 7fa0e3fff700 1 -- 172.21.15.100:0/1906735370 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f0003dd0 con 0x5578a0180290 2024-05-10T11:19:18.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.977+0000 7fa10b492b80 1 -- 172.21.15.100:0/1906735370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578a0180290 msgr2=0x5578a0190670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:18.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.977+0000 7fa10b492b80 1 --2- 172.21.15.100:0/1906735370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578a0180290 0x5578a0190670 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc0ff3a0 tx=0x7fa0f000a380 comp rx=0 tx=0).stop 2024-05-10T11:19:18.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.977+0000 7fa10b492b80 1 -- 172.21.15.100:0/1906735370 shutdown_connections 2024-05-10T11:19:18.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.977+0000 7fa10b492b80 1 --2- 172.21.15.100:0/1906735370 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578a0180bb0 0x5578a0185b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:18.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.977+0000 7fa10b492b80 1 --2- 172.21.15.100:0/1906735370 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5578a017ad50 0x5578a017a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:18.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.977+0000 7fa10b492b80 1 --2- 172.21.15.100:0/1906735370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578a0180290 0x5578a0190670 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:18.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.977+0000 7fa10b492b80 1 -- 172.21.15.100:0/1906735370 >> 172.21.15.100:0/1906735370 conn(0x5578a00a6b70 msgr2=0x5578a0181600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:18.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.978+0000 7fa10b492b80 1 -- 172.21.15.100:0/1906735370 shutdown_connections 2024-05-10T11:19:18.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.978+0000 7fa10b492b80 1 -- 172.21.15.100:0/1906735370 wait complete. 2024-05-10T11:19:18.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.978+0000 7fa10b492b80 1 -- start start 2024-05-10T11:19:18.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.978+0000 7fa10b492b80 1 Processor -- start 2024-05-10T11:19:18.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.978+0000 7fa10b492b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5578a017ad50 0x5578a019bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:18.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.978+0000 7fa10b492b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578a0180290 0x5578a019c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:18.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.978+0000 7fa10b492b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578a0180bb0 0x5578a01fdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:18.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.978+0000 7fa10b492b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa0fc1ad4e0 con 0x5578a0180bb0 2024-05-10T11:19:18.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.978+0000 7fa10b492b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa0fc1ae950 con 0x5578a017ad50 2024-05-10T11:19:18.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.978+0000 7fa10b492b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa0fc1adbb0 con 0x5578a0180290 2024-05-10T11:19:18.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.978+0000 7fa10b492b80 10 client.0 _ll_register_callbacks cb 0x5578a00605b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:19:18.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.978+0000 7fa0fad9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578a0180bb0 0x5578a01fdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:18.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.978+0000 7fa0fad9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578a0180bb0 0x5578a01fdcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55686/0 (socket says 172.21.15.100:55686) 2024-05-10T11:19:18.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.978+0000 7fa0fad9d700 1 -- 172.21.15.100:0/120009898 learned_addr learned my addr 172.21.15.100:0/120009898 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:18.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.978+0000 7fa0f9d9b700 1 --2- 172.21.15.100:0/120009898 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578a0180290 0x5578a019c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:18.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.978+0000 7fa0fa59c700 1 --2- 172.21.15.100:0/120009898 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5578a017ad50 0x5578a019bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:18.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.979+0000 7fa0fad9d700 1 -- 172.21.15.100:0/120009898 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578a0180290 msgr2=0x5578a019c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:18.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.979+0000 7fa0fad9d700 1 --2- 172.21.15.100:0/120009898 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578a0180290 0x5578a019c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:18.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.979+0000 7fa0fad9d700 1 -- 172.21.15.100:0/120009898 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5578a017ad50 msgr2=0x5578a019bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:18.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.979+0000 7fa0fad9d700 1 --2- 172.21.15.100:0/120009898 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5578a017ad50 0x5578a019bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:18.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.979+0000 7fa0fad9d700 1 -- 172.21.15.100:0/120009898 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578a01fe1f0 con 0x5578a0180bb0 2024-05-10T11:19:18.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.979+0000 7fa0fad9d700 1 --2- 172.21.15.100:0/120009898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578a0180bb0 0x5578a01fdcb0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fa0f40a54e0 tx=0x7fa0f400df70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f73e0a3e3754a82 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:18.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.979+0000 7fa0e1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:19:18.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.979+0000 7fa0e1ffb700 1 -- 172.21.15.100:0/120009898 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f40b22c0 con 0x5578a0180bb0 2024-05-10T11:19:18.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.979+0000 7fa0e1ffb700 1 -- 172.21.15.100:0/120009898 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fa0f40a96c0 con 0x5578a0180bb0 2024-05-10T11:19:18.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.979+0000 7fa10b492b80 1 -- 172.21.15.100:0/120009898 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0f00044b0 con 0x5578a0180bb0 2024-05-10T11:19:18.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.979+0000 7fa0e1ffb700 1 -- 172.21.15.100:0/120009898 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa0f40a98c0 con 0x5578a0180bb0 2024-05-10T11:19:18.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.979+0000 7fa10b492b80 10 client.15411 fetch_fsmap learned FSMap version 27 2024-05-10T11:19:18.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.979+0000 7fa10b492b80 1 -- 172.21.15.100:0/120009898 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5578a01ff3b0 con 0x5578a0180bb0 2024-05-10T11:19:18.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.979+0000 7fa0e1ffb700 1 -- 172.21.15.100:0/120009898 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa0f00044b0 con 0x5578a0180bb0 2024-05-10T11:19:18.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.979+0000 7fa0e1ffb700 1 -- 172.21.15.100:0/120009898 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa0f40b0c20 con 0x5578a0180bb0 2024-05-10T11:19:18.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.980+0000 7fa10b492b80 10 client.15411 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T11:19:18.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.980+0000 7fa10b492b80 10 client.15411 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-10T11:19:18.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.980+0000 7fa10b492b80 20 client.15411 populate_metadata read hostname 'smithi100' 2024-05-10T11:19:18.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.980+0000 7fa10b492b80 1 --2- 172.21.15.100:0/120009898 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x5578a01ff690 0x5578a0201a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:18.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.980+0000 7fa10b492b80 4 client.15411 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:19:18.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.980+0000 7fa10b492b80 1 -- 172.21.15.100:0/120009898 --> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5578a00d43c0 con 0x5578a01ff690 2024-05-10T11:19:18.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.980+0000 7fa0fa59c700 1 --2- 172.21.15.100:0/120009898 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x5578a01ff690 0x5578a0201a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:18.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.980+0000 7fa0fa59c700 1 --2- 172.21.15.100:0/120009898 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x5578a01ff690 0x5578a0201a80 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ffacec51e4ee1e19 server_cookie=626a652464845599 in_seq=0 out_seq=0 2024-05-10T11:19:18.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.980+0000 7fa0e1ffb700 10 client.15411 ms_handle_connect on v2:172.21.15.188:6836/1943342620 2024-05-10T11:19:18.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.980+0000 7fa0e1ffb700 1 -- 172.21.15.100:0/120009898 <== mds.2 v2:172.21.15.188:6836/1943342620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5578a00d43c0 con 0x5578a01ff690 2024-05-10T11:19:18.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.980+0000 7fa0e1ffb700 10 client.15411 handle_command_reply: tid=0 2024-05-10T11:19:18.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.980+0000 7fa10b492b80 1 client.15411 shutdown 2024-05-10T11:19:18.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.980+0000 7fa10b492b80 1 -- 172.21.15.100:0/120009898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578a0180bb0 msgr2=0x5578a01fdcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:18.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.980+0000 7fa10b492b80 1 --2- 172.21.15.100:0/120009898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578a0180bb0 0x5578a01fdcb0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fa0f40a54e0 tx=0x7fa0f400df70 comp rx=0 tx=0).stop 2024-05-10T11:19:18.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.981+0000 7fa10b492b80 1 -- 172.21.15.100:0/120009898 shutdown_connections 2024-05-10T11:19:18.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.981+0000 7fa10b492b80 1 --2- 172.21.15.100:0/120009898 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x5578a01ff690 0x5578a0201a80 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:18.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.981+0000 7fa10b492b80 1 --2- 172.21.15.100:0/120009898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578a0180bb0 0x5578a01fdcb0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:18.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.981+0000 7fa10b492b80 1 --2- 172.21.15.100:0/120009898 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5578a017ad50 0x5578a019bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:18.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.981+0000 7fa10b492b80 1 --2- 172.21.15.100:0/120009898 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578a0180290 0x5578a019c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:18.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.981+0000 7fa10b492b80 1 -- 172.21.15.100:0/120009898 >> 172.21.15.100:0/120009898 conn(0x5578a00a6b70 msgr2=0x5578a0182100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:18.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.981+0000 7fa0e1ffb700 0 client.15411 ms_handle_reset on v2:172.21.15.188:6836/1943342620 2024-05-10T11:19:18.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.981+0000 7fa10b492b80 1 -- 172.21.15.100:0/120009898 shutdown_connections 2024-05-10T11:19:18.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.981+0000 7fa10b492b80 1 -- 172.21.15.100:0/120009898 wait complete. 2024-05-10T11:19:18.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.981+0000 7fa10b492b80 20 client.15411 trim_cache size 0 max 16384 2024-05-10T11:19:18.986 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:19:18.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.981+0000 7fa0fb59e700 1 -- 172.21.15.100:0/1507688910 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa0e40b2880 msgr2=0x7fa0e40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:18.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.981+0000 7fa0fb59e700 1 --2- 172.21.15.100:0/1507688910 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa0e40b2880 0x7fa0e40b4d40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec009990 tx=0x7fa0ec000bc0 comp rx=0 tx=0).stop 2024-05-10T11:19:18.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.981+0000 7fa0fb59e700 1 -- 172.21.15.100:0/1507688910 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0fc107610 msgr2=0x7fa0fc1997f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:18.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.981+0000 7fa0fb59e700 1 --2- 172.21.15.100:0/1507688910 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0fc107610 0x7fa0fc1997f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fa0f400b970 tx=0x7fa0f400bc80 comp rx=0 tx=0).stop 2024-05-10T11:19:18.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.981+0000 7fa0fb59e700 1 -- 172.21.15.100:0/1507688910 shutdown_connections 2024-05-10T11:19:18.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.981+0000 7fa0fb59e700 1 --2- 172.21.15.100:0/1507688910 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa0e40b2880 0x7fa0e40b4d40 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:18.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.981+0000 7fa0fb59e700 1 --2- 172.21.15.100:0/1507688910 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0fc107610 0x7fa0fc1997f0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:18.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.981+0000 7fa0fb59e700 1 --2- 172.21.15.100:0/1507688910 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0fc102530 0x7fa0fc19f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:18.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.981+0000 7fa0fb59e700 1 --2- 172.21.15.100:0/1507688910 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0fc102ee0 0x7fa0fc19fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:18.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.981+0000 7fa0fb59e700 1 -- 172.21.15.100:0/1507688910 >> 172.21.15.100:0/1507688910 conn(0x7fa0fc0fddc0 msgr2=0x7fa0fc0ffc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:18.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.982+0000 7fa0fb59e700 1 -- 172.21.15.100:0/1507688910 shutdown_connections 2024-05-10T11:19:18.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:18.982+0000 7fa0fb59e700 1 -- 172.21.15.100:0/1507688910 wait complete. 2024-05-10T11:19:19.867 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-10T11:19:20.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:19:19 smithi178 ceph-mon[33359]: pgmap v448: 129 pgs: 129 active+clean; 663 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-10T11:19:20.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:19:19 smithi188 ceph-mon[34075]: pgmap v448: 129 pgs: 129 active+clean; 663 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-10T11:19:20.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:19:19 smithi100 ceph-mon[30296]: pgmap v448: 129 pgs: 129 active+clean; 663 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-10T11:19:20.117 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:19:21.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.871+0000 7f2e71d1d700 1 -- 172.21.15.100:0/632214739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e6c104160 msgr2=0x7f2e6c108150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:21.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.871+0000 7f2e71d1d700 1 --2- 172.21.15.100:0/632214739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e6c104160 0x7f2e6c108150 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f2e600099c0 tx=0x7f2e60009cd0 comp rx=0 tx=0).stop 2024-05-10T11:19:21.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.871+0000 7f2e71d1d700 1 -- 172.21.15.100:0/632214739 shutdown_connections 2024-05-10T11:19:21.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.871+0000 7f2e71d1d700 1 --2- 172.21.15.100:0/632214739 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e6c108880 0x7f2e6c110400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:21.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.871+0000 7f2e71d1d700 1 --2- 172.21.15.100:0/632214739 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2e6c1037b0 0x7f2e6c103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:21.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.871+0000 7f2e71d1d700 1 --2- 172.21.15.100:0/632214739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e6c104160 0x7f2e6c108150 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:21.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.871+0000 7f2e71d1d700 1 -- 172.21.15.100:0/632214739 >> 172.21.15.100:0/632214739 conn(0x7f2e6c07a530 msgr2=0x7f2e6c07a940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:21.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.871+0000 7f2e71d1d700 1 -- 172.21.15.100:0/632214739 shutdown_connections 2024-05-10T11:19:21.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.871+0000 7f2e71d1d700 1 -- 172.21.15.100:0/632214739 wait complete. 2024-05-10T11:19:21.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.872+0000 7f2e71d1d700 1 Processor -- start 2024-05-10T11:19:21.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.872+0000 7f2e71d1d700 1 -- start start 2024-05-10T11:19:21.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.872+0000 7f2e71d1d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e6c1037b0 0x7f2e6c19f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:21.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.872+0000 7f2e71d1d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e6c104160 0x7f2e6c19fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:21.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.872+0000 7f2e71d1d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2e6c108880 0x7f2e6c1997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:21.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.872+0000 7f2e71d1d700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2e6c1134f0 con 0x7f2e6c104160 2024-05-10T11:19:21.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.872+0000 7f2e71d1d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2e6c113370 con 0x7f2e6c1037b0 2024-05-10T11:19:21.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.872+0000 7f2e71d1d700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2e6c113670 con 0x7f2e6c108880 2024-05-10T11:19:21.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.872+0000 7f2e6affd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e6c104160 0x7f2e6c19fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:21.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.872+0000 7f2e6affd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e6c104160 0x7f2e6c19fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55694/0 (socket says 172.21.15.100:55694) 2024-05-10T11:19:21.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.872+0000 7f2e6affd700 1 -- 172.21.15.100:0/3823631645 learned_addr learned my addr 172.21.15.100:0/3823631645 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:21.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.872+0000 7f2e6b7fe700 1 --2- 172.21.15.100:0/3823631645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e6c1037b0 0x7f2e6c19f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:21.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.872+0000 7f2e6bfff700 1 --2- 172.21.15.100:0/3823631645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2e6c108880 0x7f2e6c1997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:21.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.873+0000 7f2e6affd700 1 -- 172.21.15.100:0/3823631645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2e6c108880 msgr2=0x7f2e6c1997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:21.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.873+0000 7f2e6affd700 1 --2- 172.21.15.100:0/3823631645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2e6c108880 0x7f2e6c1997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:21.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.873+0000 7f2e6affd700 1 -- 172.21.15.100:0/3823631645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e6c1037b0 msgr2=0x7f2e6c19f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:21.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.873+0000 7f2e6affd700 1 --2- 172.21.15.100:0/3823631645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e6c1037b0 0x7f2e6c19f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:21.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.873+0000 7f2e6affd700 1 -- 172.21.15.100:0/3823631645 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e6c199fd0 con 0x7f2e6c104160 2024-05-10T11:19:21.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.873+0000 7f2e6affd700 1 --2- 172.21.15.100:0/3823631645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e6c104160 0x7f2e6c19fb60 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f2e60005b40 tx=0x7f2e600047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:21.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.873+0000 7f2e68ff9700 1 -- 172.21.15.100:0/3823631645 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e60027070 con 0x7f2e6c104160 2024-05-10T11:19:21.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.873+0000 7f2e68ff9700 1 -- 172.21.15.100:0/3823631645 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2e60003ef0 con 0x7f2e6c104160 2024-05-10T11:19:21.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.873+0000 7f2e71d1d700 1 -- 172.21.15.100:0/3823631645 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e6c19a260 con 0x7f2e6c104160 2024-05-10T11:19:21.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.873+0000 7f2e68ff9700 1 -- 172.21.15.100:0/3823631645 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e60022c00 con 0x7f2e6c104160 2024-05-10T11:19:21.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.873+0000 7f2e71d1d700 1 -- 172.21.15.100:0/3823631645 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e6c1a6480 con 0x7f2e6c104160 2024-05-10T11:19:21.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.876+0000 7f2e68ff9700 1 -- 172.21.15.100:0/3823631645 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f2e6002a070 con 0x7f2e6c104160 2024-05-10T11:19:21.877 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:19:21 smithi100 ceph-mon[30296]: pgmap v449: 129 pgs: 129 active+clean; 663 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-10T11:19:21.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.877+0000 7f2e68ff9700 1 --2- 172.21.15.100:0/3823631645 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2e540b2900 0x7f2e540b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:21.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.877+0000 7f2e68ff9700 1 -- 172.21.15.100:0/3823631645 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f2e600e3400 con 0x7f2e6c104160 2024-05-10T11:19:21.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.877+0000 7f2e6b7fe700 1 --2- 172.21.15.100:0/3823631645 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2e540b2900 0x7f2e540b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:21.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.877+0000 7f2e7c5b0b80 1 Processor -- start 2024-05-10T11:19:21.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.877+0000 7f2e7c5b0b80 1 -- start start 2024-05-10T11:19:21.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.877+0000 7f2e6b7fe700 1 --2- 172.21.15.100:0/3823631645 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2e540b2900 0x7f2e540b4dc0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c005950 tx=0x7f2e5c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:21.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.877+0000 7f2e7c5b0b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b05e2f61b0 0x55b05e2f6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:21.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.877+0000 7f2e7c5b0b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b05e2f6ad0 0x55b05e2f0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:21.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.877+0000 7f2e7c5b0b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b05e2f0d50 0x55b05e2f1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:21.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.877+0000 7f2e7c5b0b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55b05e242940 con 0x55b05e2f61b0 2024-05-10T11:19:21.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.877+0000 7f2e7c5b0b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b05e1a7da0 con 0x55b05e2f0d50 2024-05-10T11:19:21.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.877+0000 7f2e7c5b0b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b05e23e900 con 0x55b05e2f6ad0 2024-05-10T11:19:21.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.878+0000 7f2e6b7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b05e2f61b0 0x55b05e2f6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:21.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.878+0000 7f2e6b7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b05e2f61b0 0x55b05e2f6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55698/0 (socket says 172.21.15.100:55698) 2024-05-10T11:19:21.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.878+0000 7f2e6b7fe700 1 -- 172.21.15.100:0/935448408 learned_addr learned my addr 172.21.15.100:0/935448408 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:21.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.878+0000 7f2e6affd700 1 --2- 172.21.15.100:0/935448408 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b05e2f6ad0 0x55b05e2f0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:21.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.878+0000 7f2e6b7fe700 1 -- 172.21.15.100:0/935448408 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b05e2f6ad0 msgr2=0x55b05e2f0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:21.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.878+0000 7f2e6b7fe700 1 --2- 172.21.15.100:0/935448408 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b05e2f6ad0 0x55b05e2f0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:21.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.878+0000 7f2e6b7fe700 1 -- 172.21.15.100:0/935448408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b05e2f0d50 msgr2=0x55b05e2f1190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T11:19:21.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.878+0000 7f2e6bfff700 1 --2- 172.21.15.100:0/935448408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b05e2f0d50 0x55b05e2f1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:21.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.878+0000 7f2e6b7fe700 1 --2- 172.21.15.100:0/935448408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b05e2f0d50 0x55b05e2f1190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:21.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.878+0000 7f2e6b7fe700 1 -- 172.21.15.100:0/935448408 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b05e18f8d0 con 0x55b05e2f61b0 2024-05-10T11:19:21.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.878+0000 7f2e6b7fe700 1 --2- 172.21.15.100:0/935448408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b05e2f61b0 0x55b05e2f6590 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c010fa0 tx=0x7f2e5c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d108c54f69841e63 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:21.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.878+0000 7f2e51ffb700 1 -- 172.21.15.100:0/935448408 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e5c00be60 con 0x55b05e2f61b0 2024-05-10T11:19:21.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.878+0000 7f2e51ffb700 1 -- 172.21.15.100:0/935448408 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2e5c011db0 con 0x55b05e2f61b0 2024-05-10T11:19:21.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.878+0000 7f2e51ffb700 1 -- 172.21.15.100:0/935448408 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e5c01c650 con 0x55b05e2f61b0 2024-05-10T11:19:21.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.879+0000 7f2e7c5b0b80 1 -- 172.21.15.100:0/935448408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b05e2f61b0 msgr2=0x55b05e2f6590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:21.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.879+0000 7f2e7c5b0b80 1 --2- 172.21.15.100:0/935448408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b05e2f61b0 0x55b05e2f6590 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c010fa0 tx=0x7f2e5c013000 comp rx=0 tx=0).stop 2024-05-10T11:19:21.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.879+0000 7f2e7c5b0b80 1 -- 172.21.15.100:0/935448408 shutdown_connections 2024-05-10T11:19:21.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.879+0000 7f2e7c5b0b80 1 --2- 172.21.15.100:0/935448408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b05e2f0d50 0x55b05e2f1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:21.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.879+0000 7f2e7c5b0b80 1 --2- 172.21.15.100:0/935448408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b05e2f61b0 0x55b05e2f6590 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:21.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.879+0000 7f2e7c5b0b80 1 --2- 172.21.15.100:0/935448408 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b05e2f6ad0 0x55b05e2f0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:21.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.879+0000 7f2e7c5b0b80 1 -- 172.21.15.100:0/935448408 >> 172.21.15.100:0/935448408 conn(0x55b05e21cb70 msgr2=0x55b05e2eb2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:21.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.879+0000 7f2e7c5b0b80 1 -- 172.21.15.100:0/935448408 shutdown_connections 2024-05-10T11:19:21.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.879+0000 7f2e7c5b0b80 1 -- 172.21.15.100:0/935448408 wait complete. 2024-05-10T11:19:21.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.880+0000 7f2e7c5b0b80 1 -- start start 2024-05-10T11:19:21.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.880+0000 7f2e7c5b0b80 1 Processor -- start 2024-05-10T11:19:21.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.880+0000 7f2e7c5b0b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b05e2f0d50 0x55b05e2fc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:21.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.880+0000 7f2e7c5b0b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b05e2f6ad0 0x55b05e2f9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:21.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.880+0000 7f2e7c5b0b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b05e2fa220 0x55b05e2fa680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:21.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.880+0000 7f2e7c5b0b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55b05e2fd450 con 0x55b05e2f0d50 2024-05-10T11:19:21.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.880+0000 7f2e7c5b0b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b05e1fbed0 con 0x55b05e2fa220 2024-05-10T11:19:21.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.880+0000 7f2e7c5b0b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b05e2fd5d0 con 0x55b05e2f6ad0 2024-05-10T11:19:21.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.880+0000 7f2e7c5b0b80 10 client.0 _ll_register_callbacks cb 0x55b05e1d65b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:19:21.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.880+0000 7f2e6b7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b05e2f0d50 0x55b05e2fc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:21.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.880+0000 7f2e6b7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b05e2f0d50 0x55b05e2fc990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55702/0 (socket says 172.21.15.100:55702) 2024-05-10T11:19:21.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.880+0000 7f2e6b7fe700 1 -- 172.21.15.100:0/2078904999 learned_addr learned my addr 172.21.15.100:0/2078904999 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:21.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.881+0000 7f2e6affd700 1 --2- 172.21.15.100:0/2078904999 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b05e2f6ad0 0x55b05e2f9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:21.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.881+0000 7f2e6bfff700 1 --2- 172.21.15.100:0/2078904999 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b05e2fa220 0x55b05e2fa680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:21.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.881+0000 7f2e6b7fe700 1 -- 172.21.15.100:0/2078904999 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b05e2f6ad0 msgr2=0x55b05e2f9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:21.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.881+0000 7f2e6b7fe700 1 --2- 172.21.15.100:0/2078904999 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b05e2f6ad0 0x55b05e2f9c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:21.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.881+0000 7f2e6b7fe700 1 -- 172.21.15.100:0/2078904999 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b05e2fa220 msgr2=0x55b05e2fa680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:21.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.881+0000 7f2e6b7fe700 1 --2- 172.21.15.100:0/2078904999 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b05e2fa220 0x55b05e2fa680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:21.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.881+0000 7f2e6b7fe700 1 -- 172.21.15.100:0/2078904999 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b05e1f9760 con 0x55b05e2f0d50 2024-05-10T11:19:21.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.881+0000 7f2e6bfff700 1 --2- 172.21.15.100:0/2078904999 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b05e2fa220 0x55b05e2fa680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:19:21.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.881+0000 7f2e6affd700 1 --2- 172.21.15.100:0/2078904999 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b05e2f6ad0 0x55b05e2f9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:19:21.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.881+0000 7f2e6b7fe700 1 --2- 172.21.15.100:0/2078904999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b05e2f0d50 0x55b05e2fc990 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c005d90 tx=0x7f2e5c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52a43bdeee1dc6d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:21.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.881+0000 7f2e4affd700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:19:21.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.881+0000 7f2e4affd700 1 -- 172.21.15.100:0/2078904999 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e5c036060 con 0x55b05e2f0d50 2024-05-10T11:19:21.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.881+0000 7f2e4affd700 1 -- 172.21.15.100:0/2078904999 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f2e5c011bb0 con 0x55b05e2f0d50 2024-05-10T11:19:21.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.881+0000 7f2e7c5b0b80 1 -- 172.21.15.100:0/2078904999 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e5c011db0 con 0x55b05e2f0d50 2024-05-10T11:19:21.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.881+0000 7f2e4affd700 1 -- 172.21.15.100:0/2078904999 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2e5c03d050 con 0x55b05e2f0d50 2024-05-10T11:19:21.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.881+0000 7f2e7c5b0b80 10 client.15423 fetch_fsmap learned FSMap version 27 2024-05-10T11:19:21.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.881+0000 7f2e7c5b0b80 1 -- 172.21.15.100:0/2078904999 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b05e308350 con 0x55b05e2f0d50 2024-05-10T11:19:21.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.882+0000 7f2e4affd700 1 -- 172.21.15.100:0/2078904999 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2e5c011db0 con 0x55b05e2f0d50 2024-05-10T11:19:21.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.882+0000 7f2e4affd700 1 -- 172.21.15.100:0/2078904999 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f2e5c0318e0 con 0x55b05e2f0d50 2024-05-10T11:19:21.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.882+0000 7f2e7c5b0b80 10 client.15423 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T11:19:21.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.882+0000 7f2e7c5b0b80 10 client.15423 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-10T11:19:21.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.882+0000 7f2e7c5b0b80 20 client.15423 populate_metadata read hostname 'smithi100' 2024-05-10T11:19:21.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.882+0000 7f2e7c5b0b80 1 --2- 172.21.15.100:0/2078904999 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55b05e30a8d0 0x55b05e30ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:21.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.882+0000 7f2e7c5b0b80 4 client.15423 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:19:21.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.882+0000 7f2e7c5b0b80 1 -- 172.21.15.100:0/2078904999 --> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b05e250320 con 0x55b05e30a8d0 2024-05-10T11:19:21.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.882+0000 7f2e6affd700 1 --2- 172.21.15.100:0/2078904999 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55b05e30a8d0 0x55b05e30ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:21.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.883+0000 7f2e4affd700 10 client.15423 ms_handle_connect on v2:172.21.15.178:6838/3760985404 2024-05-10T11:19:21.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.883+0000 7f2e6affd700 1 --2- 172.21.15.100:0/2078904999 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55b05e30a8d0 0x55b05e30ccc0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2e197d5fde871056 server_cookie=9c0a0facc0f07a9c in_seq=0 out_seq=0 2024-05-10T11:19:21.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.883+0000 7f2e4affd700 1 -- 172.21.15.100:0/2078904999 <== mds.1 v2:172.21.15.178:6838/3760985404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b05e250320 con 0x55b05e30a8d0 2024-05-10T11:19:21.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.883+0000 7f2e4affd700 10 client.15423 handle_command_reply: tid=0 2024-05-10T11:19:21.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.883+0000 7f2e7c5b0b80 1 client.15423 shutdown 2024-05-10T11:19:21.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.884+0000 7f2e7c5b0b80 1 -- 172.21.15.100:0/2078904999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b05e2f0d50 msgr2=0x55b05e2fc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:21.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.884+0000 7f2e7c5b0b80 1 --2- 172.21.15.100:0/2078904999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b05e2f0d50 0x55b05e2fc990 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c005d90 tx=0x7f2e5c005800 comp rx=0 tx=0).stop 2024-05-10T11:19:21.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.884+0000 7f2e7c5b0b80 1 -- 172.21.15.100:0/2078904999 shutdown_connections 2024-05-10T11:19:21.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.884+0000 7f2e7c5b0b80 1 --2- 172.21.15.100:0/2078904999 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55b05e30a8d0 0x55b05e30ccc0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:21.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.884+0000 7f2e7c5b0b80 1 --2- 172.21.15.100:0/2078904999 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b05e2fa220 0x55b05e2fa680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:21.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.884+0000 7f2e7c5b0b80 1 --2- 172.21.15.100:0/2078904999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b05e2f0d50 0x55b05e2fc990 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:21.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.884+0000 7f2e7c5b0b80 1 --2- 172.21.15.100:0/2078904999 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b05e2f6ad0 0x55b05e2f9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:21.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.884+0000 7f2e4affd700 0 client.15423 ms_handle_reset on v2:172.21.15.178:6838/3760985404 2024-05-10T11:19:21.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.884+0000 7f2e7c5b0b80 1 -- 172.21.15.100:0/2078904999 >> 172.21.15.100:0/2078904999 conn(0x55b05e21cb70 msgr2=0x55b05e2e5b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:21.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.884+0000 7f2e7c5b0b80 1 -- 172.21.15.100:0/2078904999 shutdown_connections 2024-05-10T11:19:21.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.884+0000 7f2e7c5b0b80 1 -- 172.21.15.100:0/2078904999 wait complete. 2024-05-10T11:19:21.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.884+0000 7f2e7c5b0b80 20 client.15423 trim_cache size 0 max 16384 2024-05-10T11:19:21.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.899+0000 7f2e7c5b0b80 1 Processor -- start 2024-05-10T11:19:21.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.899+0000 7f2e7c5b0b80 1 -- start start 2024-05-10T11:19:21.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.899+0000 7f2e7c5b0b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b05e2f0d50 0x55b05e2f0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:21.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.899+0000 7f2e7c5b0b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b05e2f6ad0 0x55b05e3064e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:21.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.899+0000 7f2e7c5b0b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b05e2fa220 0x55b05e2e5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:21.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.899+0000 7f2e7c5b0b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2e6c1b1800 con 0x55b05e2fa220 2024-05-10T11:19:21.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.899+0000 7f2e7c5b0b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2e6c1af090 con 0x55b05e2f0d50 2024-05-10T11:19:21.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.899+0000 7f2e7c5b0b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2e6c1acee0 con 0x55b05e2f6ad0 2024-05-10T11:19:21.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.900+0000 7f2e6bfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b05e2fa220 0x55b05e2e5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:21.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.900+0000 7f2e6bfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b05e2fa220 0x55b05e2e5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55716/0 (socket says 172.21.15.100:55716) 2024-05-10T11:19:21.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.900+0000 7f2e6bfff700 1 -- 172.21.15.100:0/2251912123 learned_addr learned my addr 172.21.15.100:0/2251912123 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:21.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.900+0000 7f2e6affd700 1 --2- 172.21.15.100:0/2251912123 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b05e2f6ad0 0x55b05e3064e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:21.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.900+0000 7f2e6b7fe700 1 --2- 172.21.15.100:0/2251912123 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b05e2f0d50 0x55b05e2f0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:21.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.900+0000 7f2e6bfff700 1 -- 172.21.15.100:0/2251912123 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b05e2f6ad0 msgr2=0x55b05e3064e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:21.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.900+0000 7f2e6bfff700 1 --2- 172.21.15.100:0/2251912123 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b05e2f6ad0 0x55b05e3064e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:21.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.900+0000 7f2e6bfff700 1 -- 172.21.15.100:0/2251912123 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b05e2f0d50 msgr2=0x55b05e2f0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:21.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.900+0000 7f2e6bfff700 1 --2- 172.21.15.100:0/2251912123 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b05e2f0d50 0x55b05e2f0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:21.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.900+0000 7f2e6bfff700 1 -- 172.21.15.100:0/2251912123 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b05e2fc5d0 con 0x55b05e2fa220 2024-05-10T11:19:21.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.900+0000 7f2e6affd700 1 --2- 172.21.15.100:0/2251912123 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b05e2f6ad0 0x55b05e3064e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:19:21.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.900+0000 7f2e6b7fe700 1 --2- 172.21.15.100:0/2251912123 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b05e2f0d50 0x55b05e2f0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:19:21.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.900+0000 7f2e6bfff700 1 --2- 172.21.15.100:0/2251912123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b05e2fa220 0x55b05e2e5c40 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f2e64012390 tx=0x7f2e64007950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ae10d7e8ef757f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:21.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.900+0000 7f2e517fa700 1 -- 172.21.15.100:0/2251912123 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e6401a070 con 0x55b05e2fa220 2024-05-10T11:19:21.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.900+0000 7f2e517fa700 1 -- 172.21.15.100:0/2251912123 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2e64012d00 con 0x55b05e2fa220 2024-05-10T11:19:21.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.900+0000 7f2e517fa700 1 -- 172.21.15.100:0/2251912123 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e64015640 con 0x55b05e2fa220 2024-05-10T11:19:21.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.901+0000 7f2e7c5b0b80 1 -- 172.21.15.100:0/2251912123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b05e2fa220 msgr2=0x55b05e2e5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:21.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.901+0000 7f2e7c5b0b80 1 --2- 172.21.15.100:0/2251912123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b05e2fa220 0x55b05e2e5c40 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f2e64012390 tx=0x7f2e64007950 comp rx=0 tx=0).stop 2024-05-10T11:19:21.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.901+0000 7f2e7c5b0b80 1 -- 172.21.15.100:0/2251912123 shutdown_connections 2024-05-10T11:19:21.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.901+0000 7f2e7c5b0b80 1 --2- 172.21.15.100:0/2251912123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b05e2fa220 0x55b05e2e5c40 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:21.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.901+0000 7f2e7c5b0b80 1 --2- 172.21.15.100:0/2251912123 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b05e2f0d50 0x55b05e2f0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:21.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.901+0000 7f2e7c5b0b80 1 --2- 172.21.15.100:0/2251912123 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b05e2f6ad0 0x55b05e3064e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:21.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.901+0000 7f2e7c5b0b80 1 -- 172.21.15.100:0/2251912123 >> 172.21.15.100:0/2251912123 conn(0x55b05e21cb70 msgr2=0x55b05e2f4e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:21.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.901+0000 7f2e7c5b0b80 1 -- 172.21.15.100:0/2251912123 shutdown_connections 2024-05-10T11:19:21.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.901+0000 7f2e7c5b0b80 1 -- 172.21.15.100:0/2251912123 wait complete. 2024-05-10T11:19:21.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.901+0000 7f2e7c5b0b80 1 -- start start 2024-05-10T11:19:21.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.902+0000 7f2e7c5b0b80 1 Processor -- start 2024-05-10T11:19:21.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.902+0000 7f2e7c5b0b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b05e2f0d50 0x55b05e3124f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:21.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.902+0000 7f2e7c5b0b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b05e2f6ad0 0x55b05e301c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:21.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.902+0000 7f2e7c5b0b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b05e2fa220 0x55b05e302170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:21.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.902+0000 7f2e7c5b0b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2e6c1ad550 con 0x55b05e2fa220 2024-05-10T11:19:21.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.902+0000 7f2e7c5b0b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2e6c1ae9c0 con 0x55b05e2f6ad0 2024-05-10T11:19:21.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.902+0000 7f2e7c5b0b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2e6c1adc20 con 0x55b05e2f0d50 2024-05-10T11:19:21.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.902+0000 7f2e7c5b0b80 10 client.0 _ll_register_callbacks cb 0x55b05e1d65b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:19:21.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.902+0000 7f2e6bfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b05e2fa220 0x55b05e302170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:21.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.902+0000 7f2e6bfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b05e2fa220 0x55b05e302170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55718/0 (socket says 172.21.15.100:55718) 2024-05-10T11:19:21.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.902+0000 7f2e6bfff700 1 -- 172.21.15.100:0/1007340754 learned_addr learned my addr 172.21.15.100:0/1007340754 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:21.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.902+0000 7f2e6affd700 1 --2- 172.21.15.100:0/1007340754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b05e2f6ad0 0x55b05e301c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:21.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.902+0000 7f2e6b7fe700 1 --2- 172.21.15.100:0/1007340754 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b05e2f0d50 0x55b05e3124f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:21.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.902+0000 7f2e6bfff700 1 -- 172.21.15.100:0/1007340754 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b05e2f0d50 msgr2=0x55b05e3124f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:21.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.902+0000 7f2e6bfff700 1 --2- 172.21.15.100:0/1007340754 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b05e2f0d50 0x55b05e3124f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:21.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.902+0000 7f2e6bfff700 1 -- 172.21.15.100:0/1007340754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b05e2f6ad0 msgr2=0x55b05e301c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:21.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.902+0000 7f2e6bfff700 1 --2- 172.21.15.100:0/1007340754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b05e2f6ad0 0x55b05e301c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:21.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.902+0000 7f2e6bfff700 1 -- 172.21.15.100:0/1007340754 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b05e312d50 con 0x55b05e2fa220 2024-05-10T11:19:21.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.902+0000 7f2e6b7fe700 1 --2- 172.21.15.100:0/1007340754 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b05e2f0d50 0x55b05e3124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:19:21.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.903+0000 7f2e6bfff700 1 --2- 172.21.15.100:0/1007340754 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b05e2fa220 0x55b05e302170 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f2e64009010 tx=0x7f2e640166f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67b8630855fb6af2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:21.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.903+0000 7f2e4affd700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:19:21.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.903+0000 7f2e4affd700 1 -- 172.21.15.100:0/1007340754 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e6401a040 con 0x55b05e2fa220 2024-05-10T11:19:21.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.903+0000 7f2e4affd700 1 -- 172.21.15.100:0/1007340754 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f2e64004040 con 0x55b05e2fa220 2024-05-10T11:19:21.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.903+0000 7f2e7c5b0b80 1 -- 172.21.15.100:0/1007340754 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e64012d00 con 0x55b05e2fa220 2024-05-10T11:19:21.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.903+0000 7f2e4affd700 1 -- 172.21.15.100:0/1007340754 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2e64004240 con 0x55b05e2fa220 2024-05-10T11:19:21.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.903+0000 7f2e7c5b0b80 10 client.15429 fetch_fsmap learned FSMap version 27 2024-05-10T11:19:21.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.903+0000 7f2e7c5b0b80 1 -- 172.21.15.100:0/1007340754 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b05e3789e0 con 0x55b05e2fa220 2024-05-10T11:19:21.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.903+0000 7f2e4affd700 1 -- 172.21.15.100:0/1007340754 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2e64012d00 con 0x55b05e2fa220 2024-05-10T11:19:21.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.903+0000 7f2e4affd700 1 -- 172.21.15.100:0/1007340754 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f2e640294a0 con 0x55b05e2fa220 2024-05-10T11:19:21.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.903+0000 7f2e7c5b0b80 10 client.15429 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T11:19:21.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.903+0000 7f2e7c5b0b80 10 client.15429 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-10T11:19:21.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.903+0000 7f2e7c5b0b80 20 client.15429 populate_metadata read hostname 'smithi100' 2024-05-10T11:19:21.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.903+0000 7f2e7c5b0b80 1 --2- 172.21.15.100:0/1007340754 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55b05e379070 0x55b05e37b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:21.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.903+0000 7f2e7c5b0b80 4 client.15429 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:19:21.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.903+0000 7f2e7c5b0b80 1 -- 172.21.15.100:0/1007340754 --> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b05e24a3c0 con 0x55b05e379070 2024-05-10T11:19:21.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.904+0000 7f2e6b7fe700 1 --2- 172.21.15.100:0/1007340754 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55b05e379070 0x55b05e37b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:21.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.904+0000 7f2e4affd700 10 client.15429 ms_handle_connect on v2:172.21.15.178:6838/3760985404 2024-05-10T11:19:21.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.904+0000 7f2e6b7fe700 1 --2- 172.21.15.100:0/1007340754 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55b05e379070 0x55b05e37b490 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d1bab102288e9832 server_cookie=e424d7734ce9dc65 in_seq=0 out_seq=0 2024-05-10T11:19:21.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.904+0000 7f2e4affd700 1 -- 172.21.15.100:0/1007340754 <== mds.1 v2:172.21.15.178:6838/3760985404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b05e24a3c0 con 0x55b05e379070 2024-05-10T11:19:21.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.904+0000 7f2e4affd700 10 client.15429 handle_command_reply: tid=0 2024-05-10T11:19:21.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.904+0000 7f2e7c5b0b80 1 client.15429 shutdown 2024-05-10T11:19:21.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.904+0000 7f2e7c5b0b80 1 -- 172.21.15.100:0/1007340754 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b05e2fa220 msgr2=0x55b05e302170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:21.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.904+0000 7f2e7c5b0b80 1 --2- 172.21.15.100:0/1007340754 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b05e2fa220 0x55b05e302170 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f2e64009010 tx=0x7f2e640166f0 comp rx=0 tx=0).stop 2024-05-10T11:19:21.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.904+0000 7f2e7c5b0b80 1 -- 172.21.15.100:0/1007340754 shutdown_connections 2024-05-10T11:19:21.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.904+0000 7f2e7c5b0b80 1 --2- 172.21.15.100:0/1007340754 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55b05e379070 0x55b05e37b490 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:21.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.904+0000 7f2e4affd700 0 client.15429 ms_handle_reset on v2:172.21.15.178:6838/3760985404 2024-05-10T11:19:21.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.904+0000 7f2e7c5b0b80 1 --2- 172.21.15.100:0/1007340754 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b05e2fa220 0x55b05e302170 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:21.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.904+0000 7f2e7c5b0b80 1 --2- 172.21.15.100:0/1007340754 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b05e2f0d50 0x55b05e3124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:21.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.904+0000 7f2e7c5b0b80 1 --2- 172.21.15.100:0/1007340754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b05e2f6ad0 0x55b05e301c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:21.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.904+0000 7f2e7c5b0b80 1 -- 172.21.15.100:0/1007340754 >> 172.21.15.100:0/1007340754 conn(0x55b05e21cb70 msgr2=0x55b05e2f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:21.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.904+0000 7f2e7c5b0b80 1 -- 172.21.15.100:0/1007340754 shutdown_connections 2024-05-10T11:19:21.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.904+0000 7f2e7c5b0b80 1 -- 172.21.15.100:0/1007340754 wait complete. 2024-05-10T11:19:21.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.905+0000 7f2e7c5b0b80 20 client.15429 trim_cache size 0 max 16384 2024-05-10T11:19:21.915 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:19:21.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.905+0000 7f2e71d1d700 1 -- 172.21.15.100:0/3823631645 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2e540b2900 msgr2=0x7f2e540b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:21.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.905+0000 7f2e71d1d700 1 --2- 172.21.15.100:0/3823631645 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2e540b2900 0x7f2e540b4dc0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c005950 tx=0x7f2e5c00b410 comp rx=0 tx=0).stop 2024-05-10T11:19:21.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.905+0000 7f2e71d1d700 1 -- 172.21.15.100:0/3823631645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e6c104160 msgr2=0x7f2e6c19fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:21.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.905+0000 7f2e71d1d700 1 --2- 172.21.15.100:0/3823631645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e6c104160 0x7f2e6c19fb60 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f2e60005b40 tx=0x7f2e600047b0 comp rx=0 tx=0).stop 2024-05-10T11:19:21.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.905+0000 7f2e71d1d700 1 -- 172.21.15.100:0/3823631645 shutdown_connections 2024-05-10T11:19:21.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.905+0000 7f2e71d1d700 1 --2- 172.21.15.100:0/3823631645 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2e540b2900 0x7f2e540b4dc0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:21.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.905+0000 7f2e71d1d700 1 --2- 172.21.15.100:0/3823631645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2e6c108880 0x7f2e6c1997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:21.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.905+0000 7f2e71d1d700 1 --2- 172.21.15.100:0/3823631645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e6c1037b0 0x7f2e6c19f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:21.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.905+0000 7f2e71d1d700 1 --2- 172.21.15.100:0/3823631645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e6c104160 0x7f2e6c19fb60 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:21.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.905+0000 7f2e71d1d700 1 -- 172.21.15.100:0/3823631645 >> 172.21.15.100:0/3823631645 conn(0x7f2e6c07a530 msgr2=0x7f2e6c101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:21.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.905+0000 7f2e71d1d700 1 -- 172.21.15.100:0/3823631645 shutdown_connections 2024-05-10T11:19:21.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:21.905+0000 7f2e71d1d700 1 -- 172.21.15.100:0/3823631645 wait complete. 2024-05-10T11:19:22.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:19:21 smithi178 ceph-mon[33359]: pgmap v449: 129 pgs: 129 active+clean; 663 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-10T11:19:22.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:19:21 smithi188 ceph-mon[34075]: pgmap v449: 129 pgs: 129 active+clean; 663 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-10T11:19:22.663 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-10T11:19:22.912 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:19:24.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:19:23 smithi178 ceph-mon[33359]: pgmap v450: 129 pgs: 129 active+clean; 663 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-10T11:19:24.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:19:23 smithi188 ceph-mon[34075]: pgmap v450: 129 pgs: 129 active+clean; 663 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-10T11:19:24.029 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:19:23 smithi100 ceph-mon[30296]: pgmap v450: 129 pgs: 129 active+clean; 663 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-10T11:19:24.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.593+0000 7f4e41d6c700 1 -- 172.21.15.100:0/3475366336 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e3c06c3d0 msgr2=0x7f4e3c110320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:24.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.593+0000 7f4e41d6c700 1 --2- 172.21.15.100:0/3475366336 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e3c06c3d0 0x7f4e3c110320 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f4e3400b0a0 tx=0x7f4e3400b3b0 comp rx=0 tx=0).stop 2024-05-10T11:19:24.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.593+0000 7f4e41d6c700 1 -- 172.21.15.100:0/3475366336 shutdown_connections 2024-05-10T11:19:24.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.593+0000 7f4e41d6c700 1 --2- 172.21.15.100:0/3475366336 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e3c06c3d0 0x7f4e3c110320 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:24.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.593+0000 7f4e41d6c700 1 --2- 172.21.15.100:0/3475366336 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e3c109500 0x7f4e3c1098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:24.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.593+0000 7f4e41d6c700 1 --2- 172.21.15.100:0/3475366336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e3c06ba10 0x7f4e3c06be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:24.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.593+0000 7f4e41d6c700 1 -- 172.21.15.100:0/3475366336 >> 172.21.15.100:0/3475366336 conn(0x7f4e3c079310 msgr2=0x7f4e3c079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:24.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.593+0000 7f4e41d6c700 1 -- 172.21.15.100:0/3475366336 shutdown_connections 2024-05-10T11:19:24.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.593+0000 7f4e41d6c700 1 -- 172.21.15.100:0/3475366336 wait complete. 2024-05-10T11:19:24.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.593+0000 7f4e41d6c700 1 Processor -- start 2024-05-10T11:19:24.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.594+0000 7f4e41d6c700 1 -- start start 2024-05-10T11:19:24.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.594+0000 7f4e41d6c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e3c06ba10 0x7f4e3c19f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:24.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.594+0000 7f4e41d6c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e3c06c3d0 0x7f4e3c19fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:24.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.594+0000 7f4e41d6c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e3c109500 0x7f4e3c1999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:24.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.594+0000 7f4e41d6c700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3c1135c0 con 0x7f4e3c06ba10 2024-05-10T11:19:24.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.594+0000 7f4e41d6c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3c113440 con 0x7f4e3c06c3d0 2024-05-10T11:19:24.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.594+0000 7f4e41d6c700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3c113740 con 0x7f4e3c109500 2024-05-10T11:19:24.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.594+0000 7f4e3b7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e3c06ba10 0x7f4e3c19f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:24.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.594+0000 7f4e3b7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e3c06ba10 0x7f4e3c19f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55736/0 (socket says 172.21.15.100:55736) 2024-05-10T11:19:24.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.594+0000 7f4e3b7fe700 1 -- 172.21.15.100:0/1653164813 learned_addr learned my addr 172.21.15.100:0/1653164813 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:24.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.594+0000 7f4e3bfff700 1 --2- 172.21.15.100:0/1653164813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e3c109500 0x7f4e3c1999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:24.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.594+0000 7f4e3affd700 1 --2- 172.21.15.100:0/1653164813 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e3c06c3d0 0x7f4e3c19fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:24.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.594+0000 7f4e3b7fe700 1 -- 172.21.15.100:0/1653164813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e3c109500 msgr2=0x7f4e3c1999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:24.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.594+0000 7f4e3b7fe700 1 --2- 172.21.15.100:0/1653164813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e3c109500 0x7f4e3c1999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:24.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.594+0000 7f4e3b7fe700 1 -- 172.21.15.100:0/1653164813 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e3c06c3d0 msgr2=0x7f4e3c19fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:24.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.594+0000 7f4e3b7fe700 1 --2- 172.21.15.100:0/1653164813 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e3c06c3d0 0x7f4e3c19fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:24.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.594+0000 7f4e3b7fe700 1 -- 172.21.15.100:0/1653164813 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e3c19a120 con 0x7f4e3c06ba10 2024-05-10T11:19:24.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.594+0000 7f4e3b7fe700 1 --2- 172.21.15.100:0/1653164813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e3c06ba10 0x7f4e3c19f800 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c00b530 tx=0x7f4e2c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:24.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.595+0000 7f4e38ff9700 1 -- 172.21.15.100:0/1653164813 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e2c015020 con 0x7f4e3c06ba10 2024-05-10T11:19:24.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.595+0000 7f4e38ff9700 1 -- 172.21.15.100:0/1653164813 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4e2c011cc0 con 0x7f4e3c06ba10 2024-05-10T11:19:24.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.595+0000 7f4e41d6c700 1 -- 172.21.15.100:0/1653164813 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e3c19a3b0 con 0x7f4e3c06ba10 2024-05-10T11:19:24.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.595+0000 7f4e38ff9700 1 -- 172.21.15.100:0/1653164813 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e2c010dc0 con 0x7f4e3c06ba10 2024-05-10T11:19:24.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.595+0000 7f4e41d6c700 1 -- 172.21.15.100:0/1653164813 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e3c1a66d0 con 0x7f4e3c06ba10 2024-05-10T11:19:24.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.598+0000 7f4e38ff9700 1 -- 172.21.15.100:0/1653164813 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f4e2c018020 con 0x7f4e3c06ba10 2024-05-10T11:19:24.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.599+0000 7f4e4c5ffb80 1 Processor -- start 2024-05-10T11:19:24.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.599+0000 7f4e4c5ffb80 1 -- start start 2024-05-10T11:19:24.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.599+0000 7f4e4c5ffb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d41ee54d0 0x558d41ee58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:24.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.599+0000 7f4e4c5ffb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558d41ee5df0 0x558d41ee96f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:24.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.599+0000 7f4e4c5ffb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558d41ee9c80 0x558d41eea100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:24.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.599+0000 7f4e4c5ffb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x558d41e37860 con 0x558d41ee54d0 2024-05-10T11:19:24.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.599+0000 7f4e4c5ffb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558d41dd67c0 con 0x558d41ee5df0 2024-05-10T11:19:24.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.599+0000 7f4e4c5ffb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558d41e33820 con 0x558d41ee9c80 2024-05-10T11:19:24.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.599+0000 7f4e3bfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d41ee54d0 0x558d41ee58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:24.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.599+0000 7f4e3bfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d41ee54d0 0x558d41ee58b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55746/0 (socket says 172.21.15.100:55746) 2024-05-10T11:19:24.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.599+0000 7f4e3bfff700 1 -- 172.21.15.100:0/3482631577 learned_addr learned my addr 172.21.15.100:0/3482631577 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:24.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.599+0000 7f4e38ff9700 1 --2- 172.21.15.100:0/1653164813 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4e240b2900 0x7f4e240b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:24.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.599+0000 7f4e3affd700 1 --2- 172.21.15.100:0/3482631577 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558d41ee9c80 0x558d41eea100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:24.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.599+0000 7f4e3b7fe700 1 --2- 172.21.15.100:0/3482631577 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558d41ee5df0 0x558d41ee96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:24.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.599+0000 7f4e3bfff700 1 --2- 172.21.15.100:0/1653164813 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4e240b2900 0x7f4e240b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:24.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.599+0000 7f4e3affd700 1 -- 172.21.15.100:0/3482631577 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558d41ee5df0 msgr2=0x558d41ee96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:24.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.599+0000 7f4e38ff9700 1 -- 172.21.15.100:0/1653164813 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f4e2c0e1d10 con 0x7f4e3c06ba10 2024-05-10T11:19:24.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.599+0000 7f4e3affd700 1 --2- 172.21.15.100:0/3482631577 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558d41ee5df0 0x558d41ee96f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:24.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.599+0000 7f4e3affd700 1 -- 172.21.15.100:0/3482631577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d41ee54d0 msgr2=0x558d41ee58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:24.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.599+0000 7f4e3affd700 1 --2- 172.21.15.100:0/3482631577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d41ee54d0 0x558d41ee58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:24.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.600+0000 7f4e3affd700 1 -- 172.21.15.100:0/3482631577 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d41d848d0 con 0x558d41ee9c80 2024-05-10T11:19:24.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.600+0000 7f4e3affd700 1 --2- 172.21.15.100:0/3482631577 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558d41ee9c80 0x558d41eea100 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4e30007720 tx=0x7f4e30007a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64757802265f65de server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:24.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.600+0000 7f4e21ffb700 1 -- 172.21.15.100:0/3482631577 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e30011590 con 0x558d41ee9c80 2024-05-10T11:19:24.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.600+0000 7f4e21ffb700 1 -- 172.21.15.100:0/3482631577 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4e30017070 con 0x558d41ee9c80 2024-05-10T11:19:24.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.600+0000 7f4e21ffb700 1 -- 172.21.15.100:0/3482631577 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e30012a90 con 0x558d41ee9c80 2024-05-10T11:19:24.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.601+0000 7f4e3bfff700 1 --2- 172.21.15.100:0/1653164813 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4e240b2900 0x7f4e240b4dc0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x558d41ee0c80 tx=0x7f4e3400b3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:24.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.601+0000 7f4e4c5ffb80 1 -- 172.21.15.100:0/3482631577 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558d41ee9c80 msgr2=0x558d41eea100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:24.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.601+0000 7f4e4c5ffb80 1 --2- 172.21.15.100:0/3482631577 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558d41ee9c80 0x558d41eea100 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4e30007720 tx=0x7f4e30007a30 comp rx=0 tx=0).stop 2024-05-10T11:19:24.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.601+0000 7f4e4c5ffb80 1 -- 172.21.15.100:0/3482631577 shutdown_connections 2024-05-10T11:19:24.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.601+0000 7f4e4c5ffb80 1 --2- 172.21.15.100:0/3482631577 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558d41ee9c80 0x558d41eea100 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:24.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.601+0000 7f4e4c5ffb80 1 --2- 172.21.15.100:0/3482631577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d41ee54d0 0x558d41ee58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:24.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.601+0000 7f4e4c5ffb80 1 --2- 172.21.15.100:0/3482631577 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558d41ee5df0 0x558d41ee96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:24.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.601+0000 7f4e4c5ffb80 1 -- 172.21.15.100:0/3482631577 >> 172.21.15.100:0/3482631577 conn(0x558d41e11100 msgr2=0x558d41edfc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:24.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.601+0000 7f4e4c5ffb80 1 -- 172.21.15.100:0/3482631577 shutdown_connections 2024-05-10T11:19:24.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.601+0000 7f4e4c5ffb80 1 -- 172.21.15.100:0/3482631577 wait complete. 2024-05-10T11:19:24.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.601+0000 7f4e4c5ffb80 1 -- start start 2024-05-10T11:19:24.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.602+0000 7f4e4c5ffb80 1 Processor -- start 2024-05-10T11:19:24.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.602+0000 7f4e4c5ffb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558d41ee54d0 0x558d41ef0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:24.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.602+0000 7f4e4c5ffb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558d41ee5df0 0x558d41eee3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:24.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.602+0000 7f4e4c5ffb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d41ee9c80 0x558d41eee900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:24.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.602+0000 7f4e4c5ffb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x558d41ef1e90 con 0x558d41ee9c80 2024-05-10T11:19:24.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.602+0000 7f4e4c5ffb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558d41d9dc30 con 0x558d41ee5df0 2024-05-10T11:19:24.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.602+0000 7f4e4c5ffb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558d41ef2010 con 0x558d41ee54d0 2024-05-10T11:19:24.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.602+0000 7f4e4c5ffb80 10 client.0 _ll_register_callbacks cb 0x558d41dcf920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:19:24.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.602+0000 7f4e3bfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d41ee9c80 0x558d41eee900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:24.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.602+0000 7f4e3bfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d41ee9c80 0x558d41eee900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55762/0 (socket says 172.21.15.100:55762) 2024-05-10T11:19:24.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.602+0000 7f4e3bfff700 1 -- 172.21.15.100:0/3251339906 learned_addr learned my addr 172.21.15.100:0/3251339906 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:24.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.602+0000 7f4e3b7fe700 1 --2- 172.21.15.100:0/3251339906 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558d41ee54d0 0x558d41ef0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:24.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.602+0000 7f4e3affd700 1 --2- 172.21.15.100:0/3251339906 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558d41ee5df0 0x558d41eee3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:24.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.603+0000 7f4e3bfff700 1 -- 172.21.15.100:0/3251339906 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558d41ee54d0 msgr2=0x558d41ef0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:24.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.603+0000 7f4e3bfff700 1 --2- 172.21.15.100:0/3251339906 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558d41ee54d0 0x558d41ef0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:24.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.603+0000 7f4e3bfff700 1 -- 172.21.15.100:0/3251339906 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558d41ee5df0 msgr2=0x558d41eee3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:24.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.603+0000 7f4e3bfff700 1 --2- 172.21.15.100:0/3251339906 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558d41ee5df0 0x558d41eee3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:24.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.603+0000 7f4e3bfff700 1 -- 172.21.15.100:0/3251339906 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d41dd3690 con 0x558d41ee9c80 2024-05-10T11:19:24.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.603+0000 7f4e3bfff700 1 --2- 172.21.15.100:0/3251339906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d41ee9c80 0x558d41eee900 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f4e34007b90 tx=0x7f4e340099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88224c3ce6ef97d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:24.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.603+0000 7f4e1affd700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:19:24.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.603+0000 7f4e1affd700 1 -- 172.21.15.100:0/3251339906 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e340052d0 con 0x558d41ee9c80 2024-05-10T11:19:24.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.603+0000 7f4e4c5ffb80 1 -- 172.21.15.100:0/3251339906 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e30017070 con 0x558d41ee9c80 2024-05-10T11:19:24.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.603+0000 7f4e1affd700 1 -- 172.21.15.100:0/3251339906 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f4e34005470 con 0x558d41ee9c80 2024-05-10T11:19:24.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.603+0000 7f4e1affd700 1 -- 172.21.15.100:0/3251339906 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4e34005670 con 0x558d41ee9c80 2024-05-10T11:19:24.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.603+0000 7f4e4c5ffb80 10 client.15441 fetch_fsmap learned FSMap version 27 2024-05-10T11:19:24.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.603+0000 7f4e4c5ffb80 1 -- 172.21.15.100:0/3251339906 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558d41efc660 con 0x558d41ee9c80 2024-05-10T11:19:24.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.603+0000 7f4e1affd700 1 -- 172.21.15.100:0/3251339906 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4e30017070 con 0x558d41ee9c80 2024-05-10T11:19:24.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.604+0000 7f4e1affd700 1 -- 172.21.15.100:0/3251339906 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f4e34011070 con 0x558d41ee9c80 2024-05-10T11:19:24.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.604+0000 7f4e4c5ffb80 10 client.15441 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T11:19:24.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.604+0000 7f4e4c5ffb80 10 client.15441 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:19:24.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.604+0000 7f4e4c5ffb80 20 client.15441 populate_metadata read hostname 'smithi100' 2024-05-10T11:19:24.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.604+0000 7f4e4c5ffb80 1 --2- 172.21.15.100:0/3251339906 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558d41efebf0 0x558d41f01000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:24.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.604+0000 7f4e4c5ffb80 4 client.15441 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:19:24.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.604+0000 7f4e4c5ffb80 1 -- 172.21.15.100:0/3251339906 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d41eb1650 con 0x558d41efebf0 2024-05-10T11:19:24.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.604+0000 7f4e3b7fe700 1 --2- 172.21.15.100:0/3251339906 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558d41efebf0 0x558d41f01000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:24.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.605+0000 7f4e3b7fe700 1 --2- 172.21.15.100:0/3251339906 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558d41efebf0 0x558d41f01000 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52c573f27769f719 server_cookie=ff6392c4c6e7140c in_seq=0 out_seq=0 2024-05-10T11:19:24.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.605+0000 7f4e1affd700 10 client.15441 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:19:24.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.605+0000 7f4e1affd700 1 -- 172.21.15.100:0/3251339906 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558d41eb1650 con 0x558d41efebf0 2024-05-10T11:19:24.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.605+0000 7f4e1affd700 10 client.15441 handle_command_reply: tid=0 2024-05-10T11:19:24.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.605+0000 7f4e4c5ffb80 1 client.15441 shutdown 2024-05-10T11:19:24.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.605+0000 7f4e4c5ffb80 1 -- 172.21.15.100:0/3251339906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d41ee9c80 msgr2=0x558d41eee900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:24.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.605+0000 7f4e4c5ffb80 1 --2- 172.21.15.100:0/3251339906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d41ee9c80 0x558d41eee900 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f4e34007b90 tx=0x7f4e340099e0 comp rx=0 tx=0).stop 2024-05-10T11:19:24.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.605+0000 7f4e4c5ffb80 1 -- 172.21.15.100:0/3251339906 shutdown_connections 2024-05-10T11:19:24.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.605+0000 7f4e4c5ffb80 1 --2- 172.21.15.100:0/3251339906 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558d41efebf0 0x558d41f01000 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:24.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.606+0000 7f4e4c5ffb80 1 --2- 172.21.15.100:0/3251339906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d41ee9c80 0x558d41eee900 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:24.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.606+0000 7f4e4c5ffb80 1 --2- 172.21.15.100:0/3251339906 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558d41ee54d0 0x558d41ef0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:24.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.606+0000 7f4e1affd700 0 client.15441 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:19:24.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.606+0000 7f4e4c5ffb80 1 --2- 172.21.15.100:0/3251339906 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558d41ee5df0 0x558d41eee3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:24.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.606+0000 7f4e4c5ffb80 1 -- 172.21.15.100:0/3251339906 >> 172.21.15.100:0/3251339906 conn(0x558d41e11100 msgr2=0x558d41eea080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:24.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.606+0000 7f4e4c5ffb80 1 -- 172.21.15.100:0/3251339906 shutdown_connections 2024-05-10T11:19:24.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.606+0000 7f4e4c5ffb80 1 -- 172.21.15.100:0/3251339906 wait complete. 2024-05-10T11:19:24.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.606+0000 7f4e4c5ffb80 20 client.15441 trim_cache size 0 max 16384 2024-05-10T11:19:24.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.621+0000 7f4e4c5ffb80 1 Processor -- start 2024-05-10T11:19:24.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.621+0000 7f4e4c5ffb80 1 -- start start 2024-05-10T11:19:24.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.621+0000 7f4e4c5ffb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d41ee54d0 0x558d41f00fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:24.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.621+0000 7f4e4c5ffb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558d41ee5df0 0x558d41eda5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:24.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.621+0000 7f4e4c5ffb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558d41ee9c80 0x558d41ee75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:24.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.621+0000 7f4e4c5ffb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3c1b1990 con 0x558d41ee54d0 2024-05-10T11:19:24.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.621+0000 7f4e4c5ffb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3c1af220 con 0x558d41ee5df0 2024-05-10T11:19:24.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.621+0000 7f4e4c5ffb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3c1ad070 con 0x558d41ee9c80 2024-05-10T11:19:24.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.621+0000 7f4e3b7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d41ee54d0 0x558d41f00fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:24.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.621+0000 7f4e3b7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d41ee54d0 0x558d41f00fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55772/0 (socket says 172.21.15.100:55772) 2024-05-10T11:19:24.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.621+0000 7f4e3b7fe700 1 -- 172.21.15.100:0/4234586672 learned_addr learned my addr 172.21.15.100:0/4234586672 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:24.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.622+0000 7f4e3affd700 1 --2- 172.21.15.100:0/4234586672 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558d41ee5df0 0x558d41eda5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:24.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.622+0000 7f4e3b7fe700 1 -- 172.21.15.100:0/4234586672 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558d41ee9c80 msgr2=0x558d41ee75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:24.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.622+0000 7f4e3b7fe700 1 --2- 172.21.15.100:0/4234586672 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558d41ee9c80 0x558d41ee75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:24.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.622+0000 7f4e3b7fe700 1 -- 172.21.15.100:0/4234586672 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558d41ee5df0 msgr2=0x558d41eda5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:24.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.622+0000 7f4e3b7fe700 1 --2- 172.21.15.100:0/4234586672 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558d41ee5df0 0x558d41eda5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:24.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.622+0000 7f4e3b7fe700 1 -- 172.21.15.100:0/4234586672 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d41ef17d0 con 0x558d41ee54d0 2024-05-10T11:19:24.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.622+0000 7f4e3b7fe700 1 --2- 172.21.15.100:0/4234586672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d41ee54d0 0x558d41f00fe0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c0a8f60 tx=0x7f4e2c076bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5e3eab214fda4ce server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:24.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.622+0000 7f4e217fa700 1 -- 172.21.15.100:0/4234586672 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e2c06ea20 con 0x558d41ee54d0 2024-05-10T11:19:24.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.622+0000 7f4e217fa700 1 -- 172.21.15.100:0/4234586672 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4e2c0a96d0 con 0x558d41ee54d0 2024-05-10T11:19:24.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.622+0000 7f4e217fa700 1 -- 172.21.15.100:0/4234586672 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e2c018df0 con 0x558d41ee54d0 2024-05-10T11:19:24.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.623+0000 7f4e4c5ffb80 1 -- 172.21.15.100:0/4234586672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d41ee54d0 msgr2=0x558d41f00fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:24.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.623+0000 7f4e4c5ffb80 1 --2- 172.21.15.100:0/4234586672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d41ee54d0 0x558d41f00fe0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c0a8f60 tx=0x7f4e2c076bf0 comp rx=0 tx=0).stop 2024-05-10T11:19:24.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.623+0000 7f4e4c5ffb80 1 -- 172.21.15.100:0/4234586672 shutdown_connections 2024-05-10T11:19:24.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.623+0000 7f4e4c5ffb80 1 --2- 172.21.15.100:0/4234586672 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558d41ee9c80 0x558d41ee75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:24.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.623+0000 7f4e4c5ffb80 1 --2- 172.21.15.100:0/4234586672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d41ee54d0 0x558d41f00fe0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:24.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.623+0000 7f4e4c5ffb80 1 --2- 172.21.15.100:0/4234586672 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558d41ee5df0 0x558d41eda5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:24.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.623+0000 7f4e4c5ffb80 1 -- 172.21.15.100:0/4234586672 >> 172.21.15.100:0/4234586672 conn(0x558d41e11100 msgr2=0x558d41ed9e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:24.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.623+0000 7f4e4c5ffb80 1 -- 172.21.15.100:0/4234586672 shutdown_connections 2024-05-10T11:19:24.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.623+0000 7f4e4c5ffb80 1 -- 172.21.15.100:0/4234586672 wait complete. 2024-05-10T11:19:24.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.624+0000 7f4e4c5ffb80 1 -- start start 2024-05-10T11:19:24.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.624+0000 7f4e4c5ffb80 1 Processor -- start 2024-05-10T11:19:24.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.624+0000 7f4e4c5ffb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558d41ee5df0 0x558d41ef7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:24.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.624+0000 7f4e4c5ffb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558d41ee9c80 0x558d41ef7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:24.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.624+0000 7f4e4c5ffb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d41efebf0 0x558d41ef3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:24.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.624+0000 7f4e4c5ffb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3c1ad6e0 con 0x558d41efebf0 2024-05-10T11:19:24.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.624+0000 7f4e4c5ffb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3c1aeb50 con 0x558d41ee9c80 2024-05-10T11:19:24.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.624+0000 7f4e4c5ffb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3c1addb0 con 0x558d41ee5df0 2024-05-10T11:19:24.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.624+0000 7f4e4c5ffb80 10 client.0 _ll_register_callbacks cb 0x558d41dcf920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:19:24.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.624+0000 7f4e3bfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d41efebf0 0x558d41ef3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:24.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.624+0000 7f4e3bfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d41efebf0 0x558d41ef3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55788/0 (socket says 172.21.15.100:55788) 2024-05-10T11:19:24.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.624+0000 7f4e3bfff700 1 -- 172.21.15.100:0/3515285145 learned_addr learned my addr 172.21.15.100:0/3515285145 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:24.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.625+0000 7f4e3b7fe700 1 --2- 172.21.15.100:0/3515285145 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558d41ee5df0 0x558d41ef7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:24.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.625+0000 7f4e3affd700 1 --2- 172.21.15.100:0/3515285145 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558d41ee9c80 0x558d41ef7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:24.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.625+0000 7f4e3bfff700 1 -- 172.21.15.100:0/3515285145 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558d41ee5df0 msgr2=0x558d41ef7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:24.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.625+0000 7f4e3bfff700 1 --2- 172.21.15.100:0/3515285145 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558d41ee5df0 0x558d41ef7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:24.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.625+0000 7f4e3bfff700 1 -- 172.21.15.100:0/3515285145 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558d41ee9c80 msgr2=0x558d41ef7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:24.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.625+0000 7f4e3bfff700 1 --2- 172.21.15.100:0/3515285145 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558d41ee9c80 0x558d41ef7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:24.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.625+0000 7f4e3bfff700 1 -- 172.21.15.100:0/3515285145 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d41edc810 con 0x558d41efebf0 2024-05-10T11:19:24.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.625+0000 7f4e3bfff700 1 --2- 172.21.15.100:0/3515285145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d41efebf0 0x558d41ef3600 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f4e3402cdb0 tx=0x7f4e34005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c50c4a56bf9b1b1d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:24.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.625+0000 7f4e1affd700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:19:24.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.625+0000 7f4e1affd700 1 -- 172.21.15.100:0/3515285145 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e34005a20 con 0x558d41efebf0 2024-05-10T11:19:24.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.625+0000 7f4e1affd700 1 -- 172.21.15.100:0/3515285145 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f4e34028040 con 0x558d41efebf0 2024-05-10T11:19:24.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.625+0000 7f4e4c5ffb80 1 -- 172.21.15.100:0/3515285145 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e2c0a96d0 con 0x558d41efebf0 2024-05-10T11:19:24.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.625+0000 7f4e1affd700 1 -- 172.21.15.100:0/3515285145 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4e340094d0 con 0x558d41efebf0 2024-05-10T11:19:24.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.625+0000 7f4e4c5ffb80 10 client.15447 fetch_fsmap learned FSMap version 27 2024-05-10T11:19:24.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.625+0000 7f4e4c5ffb80 1 -- 172.21.15.100:0/3515285145 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558d41f62870 con 0x558d41efebf0 2024-05-10T11:19:24.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.626+0000 7f4e1affd700 1 -- 172.21.15.100:0/3515285145 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4e3402c8f0 con 0x558d41efebf0 2024-05-10T11:19:24.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.626+0000 7f4e1affd700 1 -- 172.21.15.100:0/3515285145 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f4e34011050 con 0x558d41efebf0 2024-05-10T11:19:24.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.626+0000 7f4e4c5ffb80 10 client.15447 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T11:19:24.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.626+0000 7f4e4c5ffb80 10 client.15447 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:19:24.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.626+0000 7f4e4c5ffb80 20 client.15447 populate_metadata read hostname 'smithi100' 2024-05-10T11:19:24.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.626+0000 7f4e4c5ffb80 1 --2- 172.21.15.100:0/3515285145 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558d41f6db90 0x558d41f6ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:24.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.626+0000 7f4e4c5ffb80 4 client.15447 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:19:24.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.626+0000 7f4e4c5ffb80 1 -- 172.21.15.100:0/3515285145 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f4e2c0a96d0 con 0x558d41f6db90 2024-05-10T11:19:24.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.626+0000 7f4e3b7fe700 1 --2- 172.21.15.100:0/3515285145 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558d41f6db90 0x558d41f6ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:24.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.627+0000 7f4e1affd700 10 client.15447 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:19:24.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.627+0000 7f4e3b7fe700 1 --2- 172.21.15.100:0/3515285145 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558d41f6db90 0x558d41f6ffb0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24cf3efb31234e9c server_cookie=c951e48d19a396c0 in_seq=0 out_seq=0 2024-05-10T11:19:24.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.627+0000 7f4e1affd700 1 -- 172.21.15.100:0/3515285145 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f4e2c0a96d0 con 0x558d41f6db90 2024-05-10T11:19:24.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.627+0000 7f4e1affd700 10 client.15447 handle_command_reply: tid=0 2024-05-10T11:19:24.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.627+0000 7f4e4c5ffb80 1 client.15447 shutdown 2024-05-10T11:19:24.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.627+0000 7f4e4c5ffb80 1 -- 172.21.15.100:0/3515285145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d41efebf0 msgr2=0x558d41ef3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:24.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.627+0000 7f4e4c5ffb80 1 --2- 172.21.15.100:0/3515285145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d41efebf0 0x558d41ef3600 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f4e3402cdb0 tx=0x7f4e34005f10 comp rx=0 tx=0).stop 2024-05-10T11:19:24.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.627+0000 7f4e4c5ffb80 1 -- 172.21.15.100:0/3515285145 shutdown_connections 2024-05-10T11:19:24.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.627+0000 7f4e4c5ffb80 1 --2- 172.21.15.100:0/3515285145 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558d41f6db90 0x558d41f6ffb0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:24.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.627+0000 7f4e4c5ffb80 1 --2- 172.21.15.100:0/3515285145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d41efebf0 0x558d41ef3600 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:24.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.627+0000 7f4e4c5ffb80 1 --2- 172.21.15.100:0/3515285145 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558d41ee5df0 0x558d41ef7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:24.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.627+0000 7f4e4c5ffb80 1 --2- 172.21.15.100:0/3515285145 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558d41ee9c80 0x558d41ef7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:24.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.627+0000 7f4e4c5ffb80 1 -- 172.21.15.100:0/3515285145 >> 172.21.15.100:0/3515285145 conn(0x558d41e11100 msgr2=0x558d41eda720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:24.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.627+0000 7f4e1affd700 0 client.15447 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:19:24.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.627+0000 7f4e4c5ffb80 1 -- 172.21.15.100:0/3515285145 shutdown_connections 2024-05-10T11:19:24.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.627+0000 7f4e4c5ffb80 1 -- 172.21.15.100:0/3515285145 wait complete. 2024-05-10T11:19:24.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.628+0000 7f4e4c5ffb80 20 client.15447 trim_cache size 0 max 16384 2024-05-10T11:19:24.633 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:19:24.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.628+0000 7f4e41d6c700 1 -- 172.21.15.100:0/1653164813 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4e240b2900 msgr2=0x7f4e240b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:24.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.628+0000 7f4e41d6c700 1 --2- 172.21.15.100:0/1653164813 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4e240b2900 0x7f4e240b4dc0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x558d41ee0c80 tx=0x7f4e3400b3b0 comp rx=0 tx=0).stop 2024-05-10T11:19:24.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.628+0000 7f4e41d6c700 1 -- 172.21.15.100:0/1653164813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e3c06ba10 msgr2=0x7f4e3c19f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:24.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.628+0000 7f4e41d6c700 1 --2- 172.21.15.100:0/1653164813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e3c06ba10 0x7f4e3c19f800 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c00b530 tx=0x7f4e2c00b840 comp rx=0 tx=0).stop 2024-05-10T11:19:24.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.628+0000 7f4e41d6c700 1 -- 172.21.15.100:0/1653164813 shutdown_connections 2024-05-10T11:19:24.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.628+0000 7f4e41d6c700 1 --2- 172.21.15.100:0/1653164813 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4e240b2900 0x7f4e240b4dc0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:24.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.628+0000 7f4e41d6c700 1 --2- 172.21.15.100:0/1653164813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e3c109500 0x7f4e3c1999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:24.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.628+0000 7f4e41d6c700 1 --2- 172.21.15.100:0/1653164813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e3c06ba10 0x7f4e3c19f800 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:24.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.628+0000 7f4e41d6c700 1 --2- 172.21.15.100:0/1653164813 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e3c06c3d0 0x7f4e3c19fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:24.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.628+0000 7f4e41d6c700 1 -- 172.21.15.100:0/1653164813 >> 172.21.15.100:0/1653164813 conn(0x7f4e3c079310 msgr2=0x7f4e3c101220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:24.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.629+0000 7f4e41d6c700 1 -- 172.21.15.100:0/1653164813 shutdown_connections 2024-05-10T11:19:24.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:24.629+0000 7f4e41d6c700 1 -- 172.21.15.100:0/1653164813 wait complete. 2024-05-10T11:19:26.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:19:25 smithi178 ceph-mon[33359]: pgmap v451: 129 pgs: 129 active+clean; 663 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-10T11:19:26.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:19:25 smithi188 ceph-mon[34075]: pgmap v451: 129 pgs: 129 active+clean; 663 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-10T11:19:26.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:19:25 smithi100 ceph-mon[30296]: pgmap v451: 129 pgs: 129 active+clean; 663 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-10T11:19:26.427 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T11:19:26.428 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T11:19:26.696 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:19:28.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:19:27 smithi178 ceph-mon[33359]: pgmap v452: 129 pgs: 129 active+clean; 663 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-10T11:19:28.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:19:27 smithi188 ceph-mon[34075]: pgmap v452: 129 pgs: 129 active+clean; 663 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-10T11:19:28.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:19:27 smithi100 ceph-mon[30296]: pgmap v452: 129 pgs: 129 active+clean; 663 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-10T11:19:28.053 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.053+0000 7f6d149c8640 1 -- 172.21.15.100:0/1221793946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d101034c0 msgr2=0x7f6d10109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:28.053 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.053+0000 7f6d149c8640 1 --2- 172.21.15.100:0/1221793946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d101034c0 0x7f6d10109d50 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f6d08009ca0 tx=0x7f6d0802a8c0 comp rx=0 tx=0).stop 2024-05-10T11:19:28.053 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.053+0000 7f6d149c8640 1 -- 172.21.15.100:0/1221793946 shutdown_connections 2024-05-10T11:19:28.053 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.053+0000 7f6d149c8640 1 --2- 172.21.15.100:0/1221793946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d101034c0 0x7f6d10109d50 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:28.053 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.053+0000 7f6d149c8640 1 --2- 172.21.15.100:0/1221793946 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d10102b00 0x7f6d10102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:28.053 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.053+0000 7f6d149c8640 1 --2- 172.21.15.100:0/1221793946 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d10101910 0x7f6d10101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:28.053 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.053+0000 7f6d149c8640 1 -- 172.21.15.100:0/1221793946 >> 172.21.15.100:0/1221793946 conn(0x7f6d100fce80 msgr2=0x7f6d100ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:28.053 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.053+0000 7f6d149c8640 1 -- 172.21.15.100:0/1221793946 shutdown_connections 2024-05-10T11:19:28.053 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.053+0000 7f6d149c8640 1 -- 172.21.15.100:0/1221793946 wait complete. 2024-05-10T11:19:28.055 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.054+0000 7f6d149c8640 1 Processor -- start 2024-05-10T11:19:28.056 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.054+0000 7f6d149c8640 1 -- start start 2024-05-10T11:19:28.056 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.054+0000 7f6d149c8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d10101910 0x7f6d1006abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:28.056 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.054+0000 7f6d149c8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d10102b00 0x7f6d1019d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:28.056 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.054+0000 7f6d149c8640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d101034c0 0x7f6d1019fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:28.056 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.054+0000 7f6d149c8640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6d1010d320 con 0x7f6d10102b00 2024-05-10T11:19:28.056 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.054+0000 7f6d149c8640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6d1010d1a0 con 0x7f6d101034c0 2024-05-10T11:19:28.056 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.054+0000 7f6d149c8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6d1010d4a0 con 0x7f6d10101910 2024-05-10T11:19:28.056 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.054+0000 7f6d0dd9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d10102b00 0x7f6d1019d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:28.056 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.054+0000 7f6d0dd9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d10102b00 0x7f6d1019d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41090/0 (socket says 172.21.15.100:41090) 2024-05-10T11:19:28.056 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.054+0000 7f6d0dd9b640 1 -- 172.21.15.100:0/3676621687 learned_addr learned my addr 172.21.15.100:0/3676621687 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:28.056 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.054+0000 7f6d0e59c640 1 --2- 172.21.15.100:0/3676621687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d10101910 0x7f6d1006abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:28.056 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.054+0000 7f6d0ed9d640 1 --2- 172.21.15.100:0/3676621687 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d101034c0 0x7f6d1019fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:28.056 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.054+0000 7f6d0dd9b640 1 -- 172.21.15.100:0/3676621687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d10101910 msgr2=0x7f6d1006abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:28.057 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.054+0000 7f6d0dd9b640 1 --2- 172.21.15.100:0/3676621687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d10101910 0x7f6d1006abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:28.057 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.054+0000 7f6d0dd9b640 1 -- 172.21.15.100:0/3676621687 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d101034c0 msgr2=0x7f6d1019fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:28.057 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.054+0000 7f6d0dd9b640 1 --2- 172.21.15.100:0/3676621687 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d101034c0 0x7f6d1019fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:28.057 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.054+0000 7f6d0dd9b640 1 -- 172.21.15.100:0/3676621687 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d101a06c0 con 0x7f6d10102b00 2024-05-10T11:19:28.057 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.055+0000 7f6d0dd9b640 1 --2- 172.21.15.100:0/3676621687 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d10102b00 0x7f6d1019d7c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f6d0400d950 tx=0x7f6d0400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:28.057 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.055+0000 7f6cff7fe640 1 -- 172.21.15.100:0/3676621687 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d04017070 con 0x7f6d10102b00 2024-05-10T11:19:28.057 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.055+0000 7f6cff7fe640 1 -- 172.21.15.100:0/3676621687 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d04009c30 con 0x7f6d10102b00 2024-05-10T11:19:28.057 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.055+0000 7f6d149c8640 1 -- 172.21.15.100:0/3676621687 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d101a09b0 con 0x7f6d10102b00 2024-05-10T11:19:28.057 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.055+0000 7f6cff7fe640 1 -- 172.21.15.100:0/3676621687 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d0400f620 con 0x7f6d10102b00 2024-05-10T11:19:28.057 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.055+0000 7f6d149c8640 1 -- 172.21.15.100:0/3676621687 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d101a0f00 con 0x7f6d10102b00 2024-05-10T11:19:28.062 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.057+0000 7f6d177be740 1 Processor -- start 2024-05-10T11:19:28.062 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.058+0000 7f6d177be740 1 -- start start 2024-05-10T11:19:28.062 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.058+0000 7f6d177be740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee6a9c8db0 0x55ee6a9c91d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:28.062 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.058+0000 7f6d177be740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee6a9c97a0 0x55ee6a9ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:28.062 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.058+0000 7f6d177be740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ee6a9ce910 0x55ee6a9d0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:28.063 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.058+0000 7f6d177be740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55ee6a92d5d0 con 0x55ee6a9c97a0 2024-05-10T11:19:28.063 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.058+0000 7f6d177be740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ee6a8e6890 con 0x55ee6a9c8db0 2024-05-10T11:19:28.063 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.058+0000 7f6d177be740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55ee6a929590 con 0x55ee6a9ce910 2024-05-10T11:19:28.063 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.058+0000 7f6d0e59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee6a9c97a0 0x55ee6a9ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:28.063 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.058+0000 7f6d0e59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee6a9c97a0 0x55ee6a9ce2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41100/0 (socket says 172.21.15.100:41100) 2024-05-10T11:19:28.063 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.058+0000 7f6d0e59c640 1 -- 172.21.15.100:0/3984320209 learned_addr learned my addr 172.21.15.100:0/3984320209 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:28.063 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.058+0000 7f6d0ed9d640 1 --2- 172.21.15.100:0/3984320209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee6a9c8db0 0x55ee6a9c91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:28.063 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.058+0000 7f6d0e59c640 1 -- 172.21.15.100:0/3984320209 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ee6a9ce910 msgr2=0x55ee6a9d0da0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T11:19:28.063 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.058+0000 7f6d0e59c640 1 --2- 172.21.15.100:0/3984320209 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ee6a9ce910 0x55ee6a9d0da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:28.063 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.058+0000 7f6d0e59c640 1 -- 172.21.15.100:0/3984320209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee6a9c8db0 msgr2=0x55ee6a9c91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:28.063 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.058+0000 7f6d0e59c640 1 --2- 172.21.15.100:0/3984320209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee6a9c8db0 0x55ee6a9c91d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:28.063 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.058+0000 7f6d0e59c640 1 -- 172.21.15.100:0/3984320209 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee6a63f210 con 0x55ee6a9c97a0 2024-05-10T11:19:28.063 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.058+0000 7f6cff7fe640 1 -- 172.21.15.100:0/3676621687 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f6d04010460 con 0x7f6d10102b00 2024-05-10T11:19:28.063 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.058+0000 7f6d0e59c640 1 --2- 172.21.15.100:0/3984320209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee6a9c97a0 0x55ee6a9ce2a0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f6d0000abf0 tx=0x7f6d0000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1fc7cb4c6cfc125 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:28.064 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.059+0000 7f6cfcff9640 1 -- 172.21.15.100:0/3984320209 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d00004220 con 0x55ee6a9c97a0 2024-05-10T11:19:28.064 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.059+0000 7f6cfcff9640 1 -- 172.21.15.100:0/3984320209 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d000043c0 con 0x55ee6a9c97a0 2024-05-10T11:19:28.064 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.059+0000 7f6cfcff9640 1 -- 172.21.15.100:0/3984320209 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d00018800 con 0x55ee6a9c97a0 2024-05-10T11:19:28.064 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.059+0000 7f6cff7fe640 1 --2- 172.21.15.100:0/3676621687 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f6cf80b28a0 0x7f6cf80b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:28.064 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.059+0000 7f6d177be740 1 -- 172.21.15.100:0/3984320209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee6a9c97a0 msgr2=0x55ee6a9ce2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:28.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.059+0000 7f6d177be740 1 --2- 172.21.15.100:0/3984320209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee6a9c97a0 0x55ee6a9ce2a0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f6d0000abf0 tx=0x7f6d0000f790 comp rx=0 tx=0).stop 2024-05-10T11:19:28.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.059+0000 7f6d0ed9d640 1 --2- 172.21.15.100:0/3676621687 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f6cf80b28a0 0x7f6cf80b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:28.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.060+0000 7f6cff7fe640 1 -- 172.21.15.100:0/3676621687 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f6d040e21a0 con 0x7f6d10102b00 2024-05-10T11:19:28.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.060+0000 7f6d177be740 1 -- 172.21.15.100:0/3984320209 shutdown_connections 2024-05-10T11:19:28.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.060+0000 7f6d177be740 1 --2- 172.21.15.100:0/3984320209 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ee6a9ce910 0x55ee6a9d0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:28.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.060+0000 7f6d177be740 1 --2- 172.21.15.100:0/3984320209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee6a9c97a0 0x55ee6a9ce2a0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:28.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.060+0000 7f6d177be740 1 --2- 172.21.15.100:0/3984320209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee6a9c8db0 0x55ee6a9c91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:28.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.060+0000 7f6d177be740 1 -- 172.21.15.100:0/3984320209 >> 172.21.15.100:0/3984320209 conn(0x55ee6a9c4420 msgr2=0x55ee6a9c4820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:28.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.060+0000 7f6d177be740 1 -- 172.21.15.100:0/3984320209 shutdown_connections 2024-05-10T11:19:28.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.060+0000 7f6d177be740 1 -- 172.21.15.100:0/3984320209 wait complete. 2024-05-10T11:19:28.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.060+0000 7f6d0ed9d640 1 --2- 172.21.15.100:0/3676621687 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f6cf80b28a0 0x7f6cf80b4d60 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x55ee6a9ca000 tx=0x7f6d08009360 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:28.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.060+0000 7f6d177be740 1 -- start start 2024-05-10T11:19:28.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.060+0000 7f6d177be740 1 Processor -- start 2024-05-10T11:19:28.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.060+0000 7f6d177be740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ee6a9c8db0 0x55ee6a9d7460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:28.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.060+0000 7f6d177be740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee6a9c97a0 0x55ee6a9d79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:28.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.060+0000 7f6d177be740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee6a9ce910 0x55ee6a9de9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:28.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.061+0000 7f6d177be740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55ee6a9d18c0 con 0x55ee6a9ce910 2024-05-10T11:19:28.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.061+0000 7f6d177be740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ee6a8ea370 con 0x55ee6a9c97a0 2024-05-10T11:19:28.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.061+0000 7f6d177be740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55ee6a9d1a40 con 0x55ee6a9c8db0 2024-05-10T11:19:28.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.061+0000 7f6d177be740 10 client.0 _ll_register_callbacks cb 0x55ee6a63ea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:19:28.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.061+0000 7f6d0ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee6a9ce910 0x55ee6a9de9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:28.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.061+0000 7f6d0ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee6a9ce910 0x55ee6a9de9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41114/0 (socket says 172.21.15.100:41114) 2024-05-10T11:19:28.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.061+0000 7f6d0ed9d640 1 -- 172.21.15.100:0/2110570908 learned_addr learned my addr 172.21.15.100:0/2110570908 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:28.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.061+0000 7f6d0e59c640 1 --2- 172.21.15.100:0/2110570908 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ee6a9c8db0 0x55ee6a9d7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:28.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.061+0000 7f6d0dd9b640 1 --2- 172.21.15.100:0/2110570908 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee6a9c97a0 0x55ee6a9d79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:28.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.061+0000 7f6d0ed9d640 1 -- 172.21.15.100:0/2110570908 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ee6a9c8db0 msgr2=0x55ee6a9d7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:28.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.061+0000 7f6d0ed9d640 1 --2- 172.21.15.100:0/2110570908 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ee6a9c8db0 0x55ee6a9d7460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:28.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.061+0000 7f6d0ed9d640 1 -- 172.21.15.100:0/2110570908 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee6a9c97a0 msgr2=0x55ee6a9d79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:28.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.061+0000 7f6d0ed9d640 1 --2- 172.21.15.100:0/2110570908 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee6a9c97a0 0x55ee6a9d79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:28.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.061+0000 7f6d0ed9d640 1 -- 172.21.15.100:0/2110570908 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee6a70b620 con 0x55ee6a9ce910 2024-05-10T11:19:28.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.061+0000 7f6d0ed9d640 1 --2- 172.21.15.100:0/2110570908 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee6a9ce910 0x55ee6a9de9e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f6d08009ca0 tx=0x7f6d0802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96e6b8902f25b19a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:28.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.061+0000 7f6cf5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:19:28.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.061+0000 7f6cf5ffb640 1 -- 172.21.15.100:0/2110570908 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d08004250 con 0x55ee6a9ce910 2024-05-10T11:19:28.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.061+0000 7f6cf5ffb640 1 -- 172.21.15.100:0/2110570908 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f6d080043f0 con 0x55ee6a9ce910 2024-05-10T11:19:28.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.061+0000 7f6d177be740 1 -- 172.21.15.100:0/2110570908 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d000043c0 con 0x55ee6a9ce910 2024-05-10T11:19:28.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.061+0000 7f6cf5ffb640 1 -- 172.21.15.100:0/2110570908 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6d080045f0 con 0x55ee6a9ce910 2024-05-10T11:19:28.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.062+0000 7f6d177be740 10 client.15459 fetch_fsmap learned FSMap version 27 2024-05-10T11:19:28.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.062+0000 7f6d177be740 1 -- 172.21.15.100:0/2110570908 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ee6a9e19c0 con 0x55ee6a9ce910 2024-05-10T11:19:28.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.062+0000 7f6cf5ffb640 1 -- 172.21.15.100:0/2110570908 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d000043c0 con 0x55ee6a9ce910 2024-05-10T11:19:28.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.062+0000 7f6cf5ffb640 1 -- 172.21.15.100:0/2110570908 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f6d08051070 con 0x55ee6a9ce910 2024-05-10T11:19:28.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.062+0000 7f6d177be740 10 client.15459 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T11:19:28.068 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.062+0000 7f6d177be740 10 client.15459 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:19:28.068 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.062+0000 7f6d177be740 20 client.15459 populate_metadata read hostname 'smithi100' 2024-05-10T11:19:28.068 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.062+0000 7f6d177be740 1 --2- 172.21.15.100:0/2110570908 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ee6a9e3150 0x55ee6a9e5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:28.068 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.062+0000 7f6d177be740 4 client.15459 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:19:28.068 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.062+0000 7f6d177be740 1 -- 172.21.15.100:0/2110570908 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee6a93b1e0 con 0x55ee6a9e3150 2024-05-10T11:19:28.068 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.062+0000 7f6d0e59c640 1 --2- 172.21.15.100:0/2110570908 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ee6a9e3150 0x55ee6a9e5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:28.068 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.063+0000 7f6cf5ffb640 10 client.15459 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:19:28.068 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.063+0000 7f6d0e59c640 1 --2- 172.21.15.100:0/2110570908 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ee6a9e3150 0x55ee6a9e5540 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=304a069a56c7acfe server_cookie=a879261bb187d073 in_seq=0 out_seq=0 2024-05-10T11:19:28.069 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.063+0000 7f6cf5ffb640 1 -- 172.21.15.100:0/2110570908 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ee6a93b1e0 con 0x55ee6a9e3150 2024-05-10T11:19:28.070 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.063+0000 7f6cf5ffb640 10 client.15459 handle_command_reply: tid=0 2024-05-10T11:19:28.070 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.063+0000 7f6d177be740 1 client.15459 shutdown 2024-05-10T11:19:28.070 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.063+0000 7f6d177be740 1 -- 172.21.15.100:0/2110570908 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee6a9ce910 msgr2=0x55ee6a9de9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:28.070 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.063+0000 7f6d177be740 1 --2- 172.21.15.100:0/2110570908 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee6a9ce910 0x55ee6a9de9e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f6d08009ca0 tx=0x7f6d0802a8c0 comp rx=0 tx=0).stop 2024-05-10T11:19:28.070 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.064+0000 7f6d177be740 1 -- 172.21.15.100:0/2110570908 shutdown_connections 2024-05-10T11:19:28.070 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.064+0000 7f6d177be740 1 --2- 172.21.15.100:0/2110570908 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ee6a9e3150 0x55ee6a9e5540 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:28.070 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.064+0000 7f6d177be740 1 --2- 172.21.15.100:0/2110570908 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee6a9ce910 0x55ee6a9de9e0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:28.070 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.064+0000 7f6cf5ffb640 0 client.15459 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:19:28.070 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.064+0000 7f6d177be740 1 --2- 172.21.15.100:0/2110570908 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee6a9c97a0 0x55ee6a9d79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:28.070 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.064+0000 7f6d177be740 1 --2- 172.21.15.100:0/2110570908 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ee6a9c8db0 0x55ee6a9d7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:28.070 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.064+0000 7f6d177be740 1 -- 172.21.15.100:0/2110570908 >> 172.21.15.100:0/2110570908 conn(0x55ee6a9c4420 msgr2=0x55ee6a9c3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:28.070 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.064+0000 7f6d177be740 1 -- 172.21.15.100:0/2110570908 shutdown_connections 2024-05-10T11:19:28.070 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.064+0000 7f6d177be740 1 -- 172.21.15.100:0/2110570908 wait complete. 2024-05-10T11:19:28.071 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.064+0000 7f6d177be740 20 client.15459 trim_cache size 0 max 16384 2024-05-10T11:19:28.079 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.079+0000 7f6d177be740 1 Processor -- start 2024-05-10T11:19:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.079+0000 7f6d177be740 1 -- start start 2024-05-10T11:19:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.079+0000 7f6d177be740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee6a9c8db0 0x55ee6a9ca9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.079+0000 7f6d177be740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee6a9c97a0 0x55ee6a9caf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.079+0000 7f6d177be740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ee6a9ce910 0x55ee6a9c7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.079+0000 7f6d177be740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6d101ab9d0 con 0x55ee6a9c97a0 2024-05-10T11:19:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.079+0000 7f6d177be740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6d101a92a0 con 0x55ee6a9c8db0 2024-05-10T11:19:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.079+0000 7f6d177be740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6d101a70f0 con 0x55ee6a9ce910 2024-05-10T11:19:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.079+0000 7f6d0dd9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee6a9c97a0 0x55ee6a9caf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.079+0000 7f6d0dd9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee6a9c97a0 0x55ee6a9caf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41126/0 (socket says 172.21.15.100:41126) 2024-05-10T11:19:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.079+0000 7f6d0dd9b640 1 -- 172.21.15.100:0/1200254903 learned_addr learned my addr 172.21.15.100:0/1200254903 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.079+0000 7f6d0e59c640 1 --2- 172.21.15.100:0/1200254903 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee6a9c8db0 0x55ee6a9ca9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.079+0000 7f6d0ed9d640 1 --2- 172.21.15.100:0/1200254903 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ee6a9ce910 0x55ee6a9c7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.079+0000 7f6d0dd9b640 1 -- 172.21.15.100:0/1200254903 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ee6a9ce910 msgr2=0x55ee6a9c7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.079+0000 7f6d0dd9b640 1 --2- 172.21.15.100:0/1200254903 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ee6a9ce910 0x55ee6a9c7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.079+0000 7f6d0dd9b640 1 -- 172.21.15.100:0/1200254903 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee6a9c8db0 msgr2=0x55ee6a9ca9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.079+0000 7f6d0dd9b640 1 --2- 172.21.15.100:0/1200254903 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee6a9c8db0 0x55ee6a9ca9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.079+0000 7f6d0dd9b640 1 -- 172.21.15.100:0/1200254903 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee6a9c9540 con 0x55ee6a9c97a0 2024-05-10T11:19:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.079+0000 7f6d0dd9b640 1 --2- 172.21.15.100:0/1200254903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee6a9c97a0 0x55ee6a9caf00 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f6d040df940 tx=0x7f6d04010f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a786ec4077f0fee server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.080+0000 7f6cf7fff640 1 -- 172.21.15.100:0/1200254903 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d04020070 con 0x55ee6a9c97a0 2024-05-10T11:19:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.080+0000 7f6cf7fff640 1 -- 172.21.15.100:0/1200254903 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d040a9460 con 0x55ee6a9c97a0 2024-05-10T11:19:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.080+0000 7f6cf7fff640 1 -- 172.21.15.100:0/1200254903 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d040b0b90 con 0x55ee6a9c97a0 2024-05-10T11:19:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.080+0000 7f6d177be740 1 -- 172.21.15.100:0/1200254903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee6a9c97a0 msgr2=0x55ee6a9caf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.080+0000 7f6d177be740 1 --2- 172.21.15.100:0/1200254903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee6a9c97a0 0x55ee6a9caf00 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f6d040df940 tx=0x7f6d04010f70 comp rx=0 tx=0).stop 2024-05-10T11:19:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.080+0000 7f6d177be740 1 -- 172.21.15.100:0/1200254903 shutdown_connections 2024-05-10T11:19:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.080+0000 7f6d177be740 1 --2- 172.21.15.100:0/1200254903 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ee6a9ce910 0x55ee6a9c7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.080+0000 7f6d177be740 1 --2- 172.21.15.100:0/1200254903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee6a9c97a0 0x55ee6a9caf00 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.080+0000 7f6d177be740 1 --2- 172.21.15.100:0/1200254903 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee6a9c8db0 0x55ee6a9ca9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.080+0000 7f6d177be740 1 -- 172.21.15.100:0/1200254903 >> 172.21.15.100:0/1200254903 conn(0x55ee6a9c4420 msgr2=0x55ee6a9bf3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.081+0000 7f6d177be740 1 -- 172.21.15.100:0/1200254903 shutdown_connections 2024-05-10T11:19:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.081+0000 7f6d177be740 1 -- 172.21.15.100:0/1200254903 wait complete. 2024-05-10T11:19:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.081+0000 7f6d177be740 1 -- start start 2024-05-10T11:19:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.081+0000 7f6d177be740 1 Processor -- start 2024-05-10T11:19:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.081+0000 7f6d177be740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ee6a9c8db0 0x55ee6a9dba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.081+0000 7f6d177be740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee6a9c97a0 0x55ee6a9dbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.081+0000 7f6d177be740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee6a9ce910 0x55ee6a9dcfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.081+0000 7f6d177be740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6d101a7760 con 0x55ee6a9ce910 2024-05-10T11:19:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.081+0000 7f6d177be740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6d101a8bd0 con 0x55ee6a9c97a0 2024-05-10T11:19:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.081+0000 7f6d177be740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6d101a7e30 con 0x55ee6a9c8db0 2024-05-10T11:19:28.086 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.081+0000 7f6d177be740 10 client.0 _ll_register_callbacks cb 0x7f6d08031450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:19:28.086 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.081+0000 7f6d0ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee6a9ce910 0x55ee6a9dcfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:28.086 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.081+0000 7f6d0ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee6a9ce910 0x55ee6a9dcfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41132/0 (socket says 172.21.15.100:41132) 2024-05-10T11:19:28.086 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.081+0000 7f6d0ed9d640 1 -- 172.21.15.100:0/2150235720 learned_addr learned my addr 172.21.15.100:0/2150235720 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:28.086 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.081+0000 7f6d0dd9b640 1 --2- 172.21.15.100:0/2150235720 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee6a9c97a0 0x55ee6a9dbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:28.086 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.081+0000 7f6d0e59c640 1 --2- 172.21.15.100:0/2150235720 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ee6a9c8db0 0x55ee6a9dba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:28.086 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.081+0000 7f6d0ed9d640 1 -- 172.21.15.100:0/2150235720 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ee6a9c8db0 msgr2=0x55ee6a9dba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:28.086 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.081+0000 7f6d0ed9d640 1 --2- 172.21.15.100:0/2150235720 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ee6a9c8db0 0x55ee6a9dba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:28.086 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.081+0000 7f6d0ed9d640 1 -- 172.21.15.100:0/2150235720 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee6a9c97a0 msgr2=0x55ee6a9dbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:28.086 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.081+0000 7f6d0ed9d640 1 --2- 172.21.15.100:0/2150235720 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee6a9c97a0 0x55ee6a9dbfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:28.086 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.081+0000 7f6d0ed9d640 1 -- 172.21.15.100:0/2150235720 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee6a9dd5a0 con 0x55ee6a9ce910 2024-05-10T11:19:28.086 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.081+0000 7f6d0dd9b640 1 --2- 172.21.15.100:0/2150235720 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee6a9c97a0 0x55ee6a9dbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:19:28.086 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.082+0000 7f6d0ed9d640 1 --2- 172.21.15.100:0/2150235720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee6a9ce910 0x55ee6a9dcfb0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6d0802ade0 tx=0x7f6d08031580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d03d5f639990e35a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:28.086 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.082+0000 7f6cf5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:19:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.082+0000 7f6cf5ffb640 1 -- 172.21.15.100:0/2150235720 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d08032e30 con 0x55ee6a9ce910 2024-05-10T11:19:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.082+0000 7f6cf5ffb640 1 -- 172.21.15.100:0/2150235720 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f6d08053a30 con 0x55ee6a9ce910 2024-05-10T11:19:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.082+0000 7f6d177be740 1 -- 172.21.15.100:0/2150235720 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d040a9460 con 0x55ee6a9ce910 2024-05-10T11:19:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.082+0000 7f6cf5ffb640 1 -- 172.21.15.100:0/2150235720 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6d0806c050 con 0x55ee6a9ce910 2024-05-10T11:19:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.082+0000 7f6d177be740 10 client.15465 fetch_fsmap learned FSMap version 27 2024-05-10T11:19:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.082+0000 7f6d177be740 1 -- 172.21.15.100:0/2150235720 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ee6aa4dca0 con 0x55ee6a9ce910 2024-05-10T11:19:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.082+0000 7f6cf5ffb640 1 -- 172.21.15.100:0/2150235720 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d040a9460 con 0x55ee6a9ce910 2024-05-10T11:19:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.082+0000 7f6cf5ffb640 1 -- 172.21.15.100:0/2150235720 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f6d08051050 con 0x55ee6a9ce910 2024-05-10T11:19:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.082+0000 7f6d177be740 10 client.15465 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T11:19:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.082+0000 7f6d177be740 10 client.15465 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:19:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.082+0000 7f6d177be740 20 client.15465 populate_metadata read hostname 'smithi100' 2024-05-10T11:19:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.082+0000 7f6d177be740 1 --2- 172.21.15.100:0/2150235720 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ee6a9e3150 0x55ee6aa50700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.082+0000 7f6d177be740 4 client.15465 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:19:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.082+0000 7f6d177be740 1 -- 172.21.15.100:0/2150235720 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ee6a935050 con 0x55ee6a9e3150 2024-05-10T11:19:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.083+0000 7f6d0e59c640 1 --2- 172.21.15.100:0/2150235720 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ee6a9e3150 0x55ee6aa50700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.083+0000 7f6cf5ffb640 10 client.15465 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:19:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.083+0000 7f6d0e59c640 1 --2- 172.21.15.100:0/2150235720 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ee6a9e3150 0x55ee6aa50700 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a8faa6c11bdda4c server_cookie=1ce8221aa48e0c1c in_seq=0 out_seq=0 2024-05-10T11:19:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.087+0000 7f6cf5ffb640 1 -- 172.21.15.100:0/2150235720 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ee6a935050 con 0x55ee6a9e3150 2024-05-10T11:19:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.087+0000 7f6cf5ffb640 10 client.15465 handle_command_reply: tid=0 2024-05-10T11:19:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.087+0000 7f6d177be740 1 client.15465 shutdown 2024-05-10T11:19:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.087+0000 7f6d177be740 1 -- 172.21.15.100:0/2150235720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee6a9ce910 msgr2=0x55ee6a9dcfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.087+0000 7f6d177be740 1 --2- 172.21.15.100:0/2150235720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee6a9ce910 0x55ee6a9dcfb0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6d0802ade0 tx=0x7f6d08031580 comp rx=0 tx=0).stop 2024-05-10T11:19:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.087+0000 7f6d177be740 1 -- 172.21.15.100:0/2150235720 shutdown_connections 2024-05-10T11:19:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.087+0000 7f6d177be740 1 --2- 172.21.15.100:0/2150235720 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ee6a9e3150 0x55ee6aa50700 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.087+0000 7f6d177be740 1 --2- 172.21.15.100:0/2150235720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee6a9ce910 0x55ee6a9dcfb0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.087+0000 7f6d177be740 1 --2- 172.21.15.100:0/2150235720 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee6a9c97a0 0x55ee6a9dbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.087+0000 7f6cf5ffb640 0 client.15465 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:19:28.090 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.087+0000 7f6d177be740 1 --2- 172.21.15.100:0/2150235720 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ee6a9c8db0 0x55ee6a9dba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:28.090 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.087+0000 7f6d177be740 1 -- 172.21.15.100:0/2150235720 >> 172.21.15.100:0/2150235720 conn(0x55ee6a9c4420 msgr2=0x55ee6a9d7460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:28.090 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.087+0000 7f6d177be740 1 -- 172.21.15.100:0/2150235720 shutdown_connections 2024-05-10T11:19:28.090 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.087+0000 7f6d177be740 1 -- 172.21.15.100:0/2150235720 wait complete. 2024-05-10T11:19:28.090 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.087+0000 7f6d177be740 20 client.15465 trim_cache size 0 max 16384 2024-05-10T11:19:28.090 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:19:28.090 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:19:28.090 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:19:28.090 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:19:28.091 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.088+0000 7f6d149c8640 1 -- 172.21.15.100:0/3676621687 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f6cf80b28a0 msgr2=0x7f6cf80b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:28.091 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.088+0000 7f6d149c8640 1 --2- 172.21.15.100:0/3676621687 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f6cf80b28a0 0x7f6cf80b4d60 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x55ee6a9ca000 tx=0x7f6d08009360 comp rx=0 tx=0).stop 2024-05-10T11:19:28.091 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.088+0000 7f6d149c8640 1 -- 172.21.15.100:0/3676621687 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d10102b00 msgr2=0x7f6d1019d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:28.091 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.088+0000 7f6d149c8640 1 --2- 172.21.15.100:0/3676621687 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d10102b00 0x7f6d1019d7c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f6d0400d950 tx=0x7f6d0400de10 comp rx=0 tx=0).stop 2024-05-10T11:19:28.091 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.088+0000 7f6d149c8640 1 -- 172.21.15.100:0/3676621687 shutdown_connections 2024-05-10T11:19:28.091 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.088+0000 7f6d149c8640 1 --2- 172.21.15.100:0/3676621687 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f6cf80b28a0 0x7f6cf80b4d60 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:28.092 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.088+0000 7f6d149c8640 1 --2- 172.21.15.100:0/3676621687 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d101034c0 0x7f6d1019fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:28.092 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.088+0000 7f6d149c8640 1 --2- 172.21.15.100:0/3676621687 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d10102b00 0x7f6d1019d7c0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:28.092 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.088+0000 7f6d149c8640 1 --2- 172.21.15.100:0/3676621687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d10101910 0x7f6d1006abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:28.092 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.088+0000 7f6d149c8640 1 -- 172.21.15.100:0/3676621687 >> 172.21.15.100:0/3676621687 conn(0x7f6d100fce80 msgr2=0x7f6d10105e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:28.092 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.088+0000 7f6d149c8640 1 -- 172.21.15.100:0/3676621687 shutdown_connections 2024-05-10T11:19:28.092 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:28.088+0000 7f6d149c8640 1 -- 172.21.15.100:0/3676621687 wait complete. 2024-05-10T11:19:28.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.518+0000 7f97ac793700 1 -- 172.21.15.100:0/1440660875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97a4108400 msgr2=0x7f97a41087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:28.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.518+0000 7f97ac793700 1 --2- 172.21.15.100:0/1440660875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97a4108400 0x7f97a41087e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f97a0009990 tx=0x7f97a0009ca0 comp rx=0 tx=0).stop 2024-05-10T11:19:28.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.519+0000 7f97ac793700 1 -- 172.21.15.100:0/1440660875 shutdown_connections 2024-05-10T11:19:28.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.519+0000 7f97ac793700 1 --2- 172.21.15.100:0/1440660875 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97a4102d50 0x7f97a410f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:28.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.519+0000 7f97ac793700 1 --2- 172.21.15.100:0/1440660875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97a4108400 0x7f97a41087e0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:28.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.519+0000 7f97ac793700 1 --2- 172.21.15.100:0/1440660875 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97a4102390 0x7f97a4102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:28.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.519+0000 7f97ac793700 1 -- 172.21.15.100:0/1440660875 >> 172.21.15.100:0/1440660875 conn(0x7f97a4079310 msgr2=0x7f97a4079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:28.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.519+0000 7f97ac793700 1 -- 172.21.15.100:0/1440660875 shutdown_connections 2024-05-10T11:19:28.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.519+0000 7f97ac793700 1 -- 172.21.15.100:0/1440660875 wait complete. 2024-05-10T11:19:28.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.519+0000 7f97ac793700 1 Processor -- start 2024-05-10T11:19:28.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.519+0000 7f97ac793700 1 -- start start 2024-05-10T11:19:28.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.520+0000 7f97ac793700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97a4102390 0x7f97a419f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:28.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.520+0000 7f97ac793700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97a4102d50 0x7f97a419fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:28.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.520+0000 7f97ac793700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97a4108400 0x7f97a41997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:28.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.520+0000 7f97ac793700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f97a41123d0 con 0x7f97a4108400 2024-05-10T11:19:28.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.520+0000 7f97ac793700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f97a4112250 con 0x7f97a4102d50 2024-05-10T11:19:28.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.520+0000 7f97ac793700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f97a4112550 con 0x7f97a4102390 2024-05-10T11:19:28.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.520+0000 7f97aa52f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97a4102390 0x7f97a419f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:28.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.520+0000 7f97aa52f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97a4102390 0x7f97a419f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:57110/0 (socket says 172.21.15.100:57110) 2024-05-10T11:19:28.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.520+0000 7f97aa52f700 1 -- 172.21.15.100:0/251132858 learned_addr learned my addr 172.21.15.100:0/251132858 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:28.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.520+0000 7f97aad30700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97a4108400 0x7f97a41997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:28.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.520+0000 7f97a9d2e700 1 --2- 172.21.15.100:0/251132858 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97a4102d50 0x7f97a419fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:28.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.520+0000 7f97aad30700 1 -- 172.21.15.100:0/251132858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97a4102390 msgr2=0x7f97a419f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:28.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.520+0000 7f97aad30700 1 --2- 172.21.15.100:0/251132858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97a4102390 0x7f97a419f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:28.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.520+0000 7f97aad30700 1 -- 172.21.15.100:0/251132858 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97a4102d50 msgr2=0x7f97a419fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:28.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.520+0000 7f97aad30700 1 --2- 172.21.15.100:0/251132858 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97a4102d50 0x7f97a419fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:28.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.520+0000 7f97aad30700 1 -- 172.21.15.100:0/251132858 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97a4199f90 con 0x7f97a4108400 2024-05-10T11:19:28.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.520+0000 7f97aa52f700 1 --2- 172.21.15.100:0/251132858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97a4102390 0x7f97a419f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:19:28.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.520+0000 7f97aad30700 1 --2- 172.21.15.100:0/251132858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97a4108400 0x7f97a41997b0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f979c00cc50 tx=0x7f979c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:28.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.520+0000 7f979b7fe700 1 -- 172.21.15.100:0/251132858 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979c016020 con 0x7f97a4108400 2024-05-10T11:19:28.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.521+0000 7f979b7fe700 1 -- 172.21.15.100:0/251132858 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f979c012420 con 0x7f97a4108400 2024-05-10T11:19:28.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.521+0000 7f97ac793700 1 -- 172.21.15.100:0/251132858 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97a419a280 con 0x7f97a4108400 2024-05-10T11:19:28.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.521+0000 7f979b7fe700 1 -- 172.21.15.100:0/251132858 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979c0115e0 con 0x7f97a4108400 2024-05-10T11:19:28.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.521+0000 7f97ac793700 1 -- 172.21.15.100:0/251132858 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97a4103fa0 con 0x7f97a4108400 2024-05-10T11:19:28.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.523+0000 7f979b7fe700 1 -- 172.21.15.100:0/251132858 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f979c019020 con 0x7f97a4108400 2024-05-10T11:19:28.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.524+0000 7f979b7fe700 1 --2- 172.21.15.100:0/251132858 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f97900b2880 0x7f97900b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:28.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.524+0000 7f97aa52f700 1 --2- 172.21.15.100:0/251132858 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f97900b2880 0x7f97900b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:28.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.524+0000 7f979b7fe700 1 -- 172.21.15.100:0/251132858 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f979c0e22f0 con 0x7f97a4108400 2024-05-10T11:19:28.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.524+0000 7f97b7026b80 1 Processor -- start 2024-05-10T11:19:28.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.525+0000 7f97b7026b80 1 -- start start 2024-05-10T11:19:28.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.525+0000 7f97aa52f700 1 --2- 172.21.15.100:0/251132858 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f97900b2880 0x7f97900b4d40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f97a0006010 tx=0x7f97a000bca0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:28.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.525+0000 7f97b7026b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fdbf7d6550 0x55fdbf7d6930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:28.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.525+0000 7f97b7026b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fdbf7d6e70 0x55fdbf7da770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:28.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.525+0000 7f97b7026b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fdbf7dad00 0x55fdbf7db180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:28.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.525+0000 7f97b7026b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55fdbf7288e0 con 0x55fdbf7dad00 2024-05-10T11:19:28.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.525+0000 7f97b7026b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55fdbf6c7840 con 0x55fdbf7d6e70 2024-05-10T11:19:28.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.525+0000 7f97b7026b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55fdbf7248a0 con 0x55fdbf7d6550 2024-05-10T11:19:28.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.525+0000 7f97aad30700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fdbf7dad00 0x55fdbf7db180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:28.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.525+0000 7f97aad30700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fdbf7dad00 0x55fdbf7db180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41146/0 (socket says 172.21.15.100:41146) 2024-05-10T11:19:28.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.525+0000 7f97aad30700 1 -- 172.21.15.100:0/3323001357 learned_addr learned my addr 172.21.15.100:0/3323001357 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:28.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.525+0000 7f97a9d2e700 1 --2- 172.21.15.100:0/3323001357 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fdbf7d6e70 0x55fdbf7da770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:28.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.525+0000 7f97aa52f700 1 --2- 172.21.15.100:0/3323001357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fdbf7d6550 0x55fdbf7d6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:28.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.525+0000 7f97aad30700 1 -- 172.21.15.100:0/3323001357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fdbf7d6550 msgr2=0x55fdbf7d6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:28.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.525+0000 7f97aad30700 1 --2- 172.21.15.100:0/3323001357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fdbf7d6550 0x55fdbf7d6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:28.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.525+0000 7f97aad30700 1 -- 172.21.15.100:0/3323001357 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fdbf7d6e70 msgr2=0x55fdbf7da770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:28.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.525+0000 7f97aad30700 1 --2- 172.21.15.100:0/3323001357 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fdbf7d6e70 0x55fdbf7da770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:28.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.525+0000 7f97aad30700 1 -- 172.21.15.100:0/3323001357 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdbf675950 con 0x55fdbf7dad00 2024-05-10T11:19:28.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.526+0000 7f97aad30700 1 --2- 172.21.15.100:0/3323001357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fdbf7dad00 0x55fdbf7db180 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f979c0a92b0 tx=0x7f979c0a90c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e343e72af0ef4d09 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:28.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.526+0000 7f9798ff9700 1 -- 172.21.15.100:0/3323001357 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979c0ad960 con 0x55fdbf7dad00 2024-05-10T11:19:28.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.526+0000 7f9798ff9700 1 -- 172.21.15.100:0/3323001357 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f979c0b2410 con 0x55fdbf7dad00 2024-05-10T11:19:28.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.526+0000 7f9798ff9700 1 -- 172.21.15.100:0/3323001357 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979c0b1f80 con 0x55fdbf7dad00 2024-05-10T11:19:28.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.527+0000 7f97b7026b80 1 -- 172.21.15.100:0/3323001357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fdbf7dad00 msgr2=0x55fdbf7db180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:28.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.527+0000 7f97b7026b80 1 --2- 172.21.15.100:0/3323001357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fdbf7dad00 0x55fdbf7db180 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f979c0a92b0 tx=0x7f979c0a90c0 comp rx=0 tx=0).stop 2024-05-10T11:19:28.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.527+0000 7f97b7026b80 1 -- 172.21.15.100:0/3323001357 shutdown_connections 2024-05-10T11:19:28.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.527+0000 7f97b7026b80 1 --2- 172.21.15.100:0/3323001357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fdbf7dad00 0x55fdbf7db180 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:28.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.527+0000 7f97b7026b80 1 --2- 172.21.15.100:0/3323001357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fdbf7d6550 0x55fdbf7d6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:28.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.527+0000 7f97b7026b80 1 --2- 172.21.15.100:0/3323001357 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fdbf7d6e70 0x55fdbf7da770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:28.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.527+0000 7f97b7026b80 1 -- 172.21.15.100:0/3323001357 >> 172.21.15.100:0/3323001357 conn(0x55fdbf702180 msgr2=0x55fdbf7d0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:28.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.527+0000 7f97b7026b80 1 -- 172.21.15.100:0/3323001357 shutdown_connections 2024-05-10T11:19:28.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.527+0000 7f97b7026b80 1 -- 172.21.15.100:0/3323001357 wait complete. 2024-05-10T11:19:28.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.527+0000 7f97b7026b80 1 -- start start 2024-05-10T11:19:28.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.527+0000 7f97b7026b80 1 Processor -- start 2024-05-10T11:19:28.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.528+0000 7f97b7026b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fdbf7d6550 0x55fdbf7e20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:28.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.528+0000 7f97b7026b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fdbf7d6e70 0x55fdbf7df470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:28.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.528+0000 7f97b7026b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fdbf7dad00 0x55fdbf7df9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:28.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.528+0000 7f97b7026b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55fdbf7e2f10 con 0x55fdbf7d6550 2024-05-10T11:19:28.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.528+0000 7f97b7026b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55fdbf68ecb0 con 0x55fdbf7dad00 2024-05-10T11:19:28.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.528+0000 7f97b7026b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55fdbf7e3090 con 0x55fdbf7d6e70 2024-05-10T11:19:28.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.528+0000 7f97b7026b80 10 client.0 _ll_register_callbacks cb 0x55fdbf6c09a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:19:28.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.528+0000 7f97aa52f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fdbf7d6550 0x55fdbf7e20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:28.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.528+0000 7f97aa52f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fdbf7d6550 0x55fdbf7e20d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41158/0 (socket says 172.21.15.100:41158) 2024-05-10T11:19:28.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.528+0000 7f97aa52f700 1 -- 172.21.15.100:0/4221255044 learned_addr learned my addr 172.21.15.100:0/4221255044 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:28.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.528+0000 7f97aad30700 1 --2- 172.21.15.100:0/4221255044 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fdbf7dad00 0x55fdbf7df9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:28.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.528+0000 7f97a9d2e700 1 --2- 172.21.15.100:0/4221255044 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fdbf7d6e70 0x55fdbf7df470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:28.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.528+0000 7f97aa52f700 1 -- 172.21.15.100:0/4221255044 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fdbf7d6e70 msgr2=0x55fdbf7df470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:28.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.528+0000 7f97aa52f700 1 --2- 172.21.15.100:0/4221255044 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fdbf7d6e70 0x55fdbf7df470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:28.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.528+0000 7f97aa52f700 1 -- 172.21.15.100:0/4221255044 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fdbf7dad00 msgr2=0x55fdbf7df9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:28.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.528+0000 7f97aa52f700 1 --2- 172.21.15.100:0/4221255044 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fdbf7dad00 0x55fdbf7df9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:28.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.528+0000 7f97aa52f700 1 -- 172.21.15.100:0/4221255044 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdbf6c4710 con 0x55fdbf7d6550 2024-05-10T11:19:28.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.528+0000 7f97aa52f700 1 --2- 172.21.15.100:0/4221255044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fdbf7d6550 0x55fdbf7e20d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f97a00057a0 tx=0x7f97a0009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=759455a8e4b7feb4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:28.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.528+0000 7f978dffb700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:19:28.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.528+0000 7f978dffb700 1 -- 172.21.15.100:0/4221255044 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a0033060 con 0x55fdbf7d6550 2024-05-10T11:19:28.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.528+0000 7f978dffb700 1 -- 172.21.15.100:0/4221255044 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f97a000f870 con 0x55fdbf7d6550 2024-05-10T11:19:28.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.528+0000 7f97b7026b80 1 -- 172.21.15.100:0/4221255044 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f979c0b2410 con 0x55fdbf7d6550 2024-05-10T11:19:28.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.529+0000 7f978dffb700 1 -- 172.21.15.100:0/4221255044 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f97a003a050 con 0x55fdbf7d6550 2024-05-10T11:19:28.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.529+0000 7f97b7026b80 10 client.15477 fetch_fsmap learned FSMap version 27 2024-05-10T11:19:28.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.529+0000 7f97b7026b80 1 -- 172.21.15.100:0/4221255044 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fdbf7ed710 con 0x55fdbf7d6550 2024-05-10T11:19:28.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.529+0000 7f978dffb700 1 -- 172.21.15.100:0/4221255044 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f979c0b2410 con 0x55fdbf7d6550 2024-05-10T11:19:28.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.529+0000 7f978dffb700 1 -- 172.21.15.100:0/4221255044 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f97a002d070 con 0x55fdbf7d6550 2024-05-10T11:19:28.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.529+0000 7f97b7026b80 10 client.15477 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T11:19:28.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.529+0000 7f97b7026b80 10 client.15477 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:19:28.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.529+0000 7f97b7026b80 20 client.15477 populate_metadata read hostname 'smithi100' 2024-05-10T11:19:28.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.529+0000 7f97b7026b80 1 --2- 172.21.15.100:0/4221255044 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55fdbf7efd50 0x55fdbf7f2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:28.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.529+0000 7f97b7026b80 4 client.15477 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:19:28.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.529+0000 7f97b7026b80 1 -- 172.21.15.100:0/4221255044 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fdbf7a26d0 con 0x55fdbf7efd50 2024-05-10T11:19:28.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.529+0000 7f97a9d2e700 1 --2- 172.21.15.100:0/4221255044 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55fdbf7efd50 0x55fdbf7f2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:28.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.530+0000 7f978dffb700 10 client.15477 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:19:28.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.530+0000 7f97a9d2e700 1 --2- 172.21.15.100:0/4221255044 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55fdbf7efd50 0x55fdbf7f2140 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df443738f1fd96fb server_cookie=7f2f77ca0e070abe in_seq=0 out_seq=0 2024-05-10T11:19:28.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.530+0000 7f978dffb700 1 -- 172.21.15.100:0/4221255044 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fdbf7a26d0 con 0x55fdbf7efd50 2024-05-10T11:19:28.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.530+0000 7f978dffb700 10 client.15477 handle_command_reply: tid=0 2024-05-10T11:19:28.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.531+0000 7f97b7026b80 1 client.15477 shutdown 2024-05-10T11:19:28.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.531+0000 7f97b7026b80 1 -- 172.21.15.100:0/4221255044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fdbf7d6550 msgr2=0x55fdbf7e20d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:28.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.531+0000 7f97b7026b80 1 --2- 172.21.15.100:0/4221255044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fdbf7d6550 0x55fdbf7e20d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f97a00057a0 tx=0x7f97a0009310 comp rx=0 tx=0).stop 2024-05-10T11:19:28.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.531+0000 7f97b7026b80 1 -- 172.21.15.100:0/4221255044 shutdown_connections 2024-05-10T11:19:28.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.531+0000 7f97b7026b80 1 --2- 172.21.15.100:0/4221255044 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55fdbf7efd50 0x55fdbf7f2140 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:28.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.531+0000 7f97b7026b80 1 --2- 172.21.15.100:0/4221255044 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fdbf7dad00 0x55fdbf7df9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:28.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.531+0000 7f97b7026b80 1 --2- 172.21.15.100:0/4221255044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fdbf7d6550 0x55fdbf7e20d0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:28.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.531+0000 7f978dffb700 0 client.15477 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:19:28.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.531+0000 7f97b7026b80 1 --2- 172.21.15.100:0/4221255044 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fdbf7d6e70 0x55fdbf7df470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:28.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.531+0000 7f97b7026b80 1 -- 172.21.15.100:0/4221255044 >> 172.21.15.100:0/4221255044 conn(0x55fdbf702180 msgr2=0x55fdbf7d0c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:28.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.531+0000 7f97b7026b80 1 -- 172.21.15.100:0/4221255044 shutdown_connections 2024-05-10T11:19:28.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.531+0000 7f97b7026b80 1 -- 172.21.15.100:0/4221255044 wait complete. 2024-05-10T11:19:28.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.531+0000 7f97b7026b80 20 client.15477 trim_cache size 0 max 16384 2024-05-10T11:19:28.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.567+0000 7f97b7026b80 1 Processor -- start 2024-05-10T11:19:28.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.567+0000 7f97b7026b80 1 -- start start 2024-05-10T11:19:28.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.567+0000 7f97b7026b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fdbf7d6550 0x55fdbf7ebfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:28.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.567+0000 7f97b7026b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fdbf7d6e70 0x55fdbf7d5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:28.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.567+0000 7f97b7026b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fdbf7dad00 0x55fdbf7f7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:28.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.567+0000 7f97b7026b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f97a41b3a60 con 0x55fdbf7dad00 2024-05-10T11:19:28.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.567+0000 7f97b7026b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f97a41b12f0 con 0x55fdbf7d6e70 2024-05-10T11:19:28.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.567+0000 7f97b7026b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f97a41af140 con 0x55fdbf7d6550 2024-05-10T11:19:28.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.567+0000 7f97aad30700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fdbf7dad00 0x55fdbf7f7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:28.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.567+0000 7f97aad30700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fdbf7dad00 0x55fdbf7f7aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41160/0 (socket says 172.21.15.100:41160) 2024-05-10T11:19:28.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.567+0000 7f97aad30700 1 -- 172.21.15.100:0/4170256342 learned_addr learned my addr 172.21.15.100:0/4170256342 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:28.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.567+0000 7f97a9d2e700 1 --2- 172.21.15.100:0/4170256342 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fdbf7d6e70 0x55fdbf7d5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:28.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.568+0000 7f97aa52f700 1 --2- 172.21.15.100:0/4170256342 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fdbf7d6550 0x55fdbf7ebfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:28.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.568+0000 7f97aad30700 1 -- 172.21.15.100:0/4170256342 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fdbf7d6550 msgr2=0x55fdbf7ebfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:28.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.568+0000 7f97aad30700 1 --2- 172.21.15.100:0/4170256342 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fdbf7d6550 0x55fdbf7ebfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:28.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.568+0000 7f97aad30700 1 -- 172.21.15.100:0/4170256342 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fdbf7d6e70 msgr2=0x55fdbf7d5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:28.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.568+0000 7f97aad30700 1 --2- 172.21.15.100:0/4170256342 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fdbf7d6e70 0x55fdbf7d5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:28.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.568+0000 7f97aad30700 1 -- 172.21.15.100:0/4170256342 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdbf7cd890 con 0x55fdbf7dad00 2024-05-10T11:19:28.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.568+0000 7f97aa52f700 1 --2- 172.21.15.100:0/4170256342 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fdbf7d6550 0x55fdbf7ebfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:19:28.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.568+0000 7f97aad30700 1 --2- 172.21.15.100:0/4170256342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fdbf7dad00 0x55fdbf7f7aa0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f979c0a9540 tx=0x7f979c006040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3964cb192b11800b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:28.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.568+0000 7f978ffff700 1 -- 172.21.15.100:0/4170256342 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979c06d070 con 0x55fdbf7dad00 2024-05-10T11:19:28.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.568+0000 7f978ffff700 1 -- 172.21.15.100:0/4170256342 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f979c0b2410 con 0x55fdbf7dad00 2024-05-10T11:19:28.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.568+0000 7f978ffff700 1 -- 172.21.15.100:0/4170256342 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979c0c2070 con 0x55fdbf7dad00 2024-05-10T11:19:28.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.569+0000 7f97b7026b80 1 -- 172.21.15.100:0/4170256342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fdbf7dad00 msgr2=0x55fdbf7f7aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:28.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.569+0000 7f97b7026b80 1 --2- 172.21.15.100:0/4170256342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fdbf7dad00 0x55fdbf7f7aa0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f979c0a9540 tx=0x7f979c006040 comp rx=0 tx=0).stop 2024-05-10T11:19:28.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.569+0000 7f97b7026b80 1 -- 172.21.15.100:0/4170256342 shutdown_connections 2024-05-10T11:19:28.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.569+0000 7f97b7026b80 1 --2- 172.21.15.100:0/4170256342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fdbf7dad00 0x55fdbf7f7aa0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:28.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.569+0000 7f97b7026b80 1 --2- 172.21.15.100:0/4170256342 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fdbf7d6550 0x55fdbf7ebfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:28.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.569+0000 7f97b7026b80 1 --2- 172.21.15.100:0/4170256342 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fdbf7d6e70 0x55fdbf7d5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:28.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.569+0000 7f97b7026b80 1 -- 172.21.15.100:0/4170256342 >> 172.21.15.100:0/4170256342 conn(0x55fdbf702180 msgr2=0x55fdbf7cb7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:28.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.569+0000 7f97b7026b80 1 -- 172.21.15.100:0/4170256342 shutdown_connections 2024-05-10T11:19:28.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.569+0000 7f97b7026b80 1 -- 172.21.15.100:0/4170256342 wait complete. 2024-05-10T11:19:28.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.569+0000 7f97b7026b80 1 -- start start 2024-05-10T11:19:28.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.569+0000 7f97b7026b80 1 Processor -- start 2024-05-10T11:19:28.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.570+0000 7f97b7026b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fdbf7d6550 0x55fdbf7e8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:28.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.570+0000 7f97b7026b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fdbf7d6e70 0x55fdbf7e4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:28.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.570+0000 7f97b7026b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fdbf7dad00 0x55fdbf7e54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:28.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.570+0000 7f97b7026b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f97a41af7b0 con 0x55fdbf7d6e70 2024-05-10T11:19:28.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.570+0000 7f97b7026b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f97a41b0c20 con 0x55fdbf7d6550 2024-05-10T11:19:28.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.570+0000 7f97b7026b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f97a41afe80 con 0x55fdbf7dad00 2024-05-10T11:19:28.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.570+0000 7f97b7026b80 10 client.0 _ll_register_callbacks cb 0x55fdbf6c09a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:19:28.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.570+0000 7f97a9d2e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fdbf7d6e70 0x55fdbf7e4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:28.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.570+0000 7f97a9d2e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fdbf7d6e70 0x55fdbf7e4de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41162/0 (socket says 172.21.15.100:41162) 2024-05-10T11:19:28.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.570+0000 7f97a9d2e700 1 -- 172.21.15.100:0/1810555334 learned_addr learned my addr 172.21.15.100:0/1810555334 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:28.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.570+0000 7f97aad30700 1 --2- 172.21.15.100:0/1810555334 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fdbf7dad00 0x55fdbf7e54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:28.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.570+0000 7f97aa52f700 1 --2- 172.21.15.100:0/1810555334 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fdbf7d6550 0x55fdbf7e8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:28.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.570+0000 7f97a9d2e700 1 -- 172.21.15.100:0/1810555334 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fdbf7dad00 msgr2=0x55fdbf7e54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:28.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.570+0000 7f97a9d2e700 1 --2- 172.21.15.100:0/1810555334 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fdbf7dad00 0x55fdbf7e54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:28.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.570+0000 7f97a9d2e700 1 -- 172.21.15.100:0/1810555334 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fdbf7d6550 msgr2=0x55fdbf7e8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:28.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.570+0000 7f97a9d2e700 1 --2- 172.21.15.100:0/1810555334 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fdbf7d6550 0x55fdbf7e8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:28.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.570+0000 7f97a9d2e700 1 -- 172.21.15.100:0/1810555334 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdbf7d25a0 con 0x55fdbf7d6e70 2024-05-10T11:19:28.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.570+0000 7f97a9d2e700 1 --2- 172.21.15.100:0/1810555334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fdbf7d6e70 0x55fdbf7e4de0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x55fdbf7f2a40 tx=0x7f9794010990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1082da493b86c6b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:28.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.570+0000 7f978dffb700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:19:28.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.570+0000 7f978dffb700 1 -- 172.21.15.100:0/1810555334 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9794004460 con 0x55fdbf7d6e70 2024-05-10T11:19:28.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.570+0000 7f978dffb700 1 -- 172.21.15.100:0/1810555334 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f9794003680 con 0x55fdbf7d6e70 2024-05-10T11:19:28.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.570+0000 7f97b7026b80 1 -- 172.21.15.100:0/1810555334 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f979c0b2410 con 0x55fdbf7d6e70 2024-05-10T11:19:28.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.571+0000 7f978dffb700 1 -- 172.21.15.100:0/1810555334 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9794031050 con 0x55fdbf7d6e70 2024-05-10T11:19:28.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.571+0000 7f97b7026b80 10 client.15483 fetch_fsmap learned FSMap version 27 2024-05-10T11:19:28.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.571+0000 7f97b7026b80 1 -- 172.21.15.100:0/1810555334 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fdbf85b690 con 0x55fdbf7d6e70 2024-05-10T11:19:28.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.571+0000 7f978dffb700 1 -- 172.21.15.100:0/1810555334 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f979c0b2410 con 0x55fdbf7d6e70 2024-05-10T11:19:28.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.571+0000 7f978dffb700 1 -- 172.21.15.100:0/1810555334 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f9794020490 con 0x55fdbf7d6e70 2024-05-10T11:19:28.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.571+0000 7f97b7026b80 10 client.15483 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T11:19:28.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.571+0000 7f97b7026b80 10 client.15483 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:19:28.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.571+0000 7f97b7026b80 20 client.15483 populate_metadata read hostname 'smithi100' 2024-05-10T11:19:28.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.571+0000 7f97b7026b80 1 --2- 172.21.15.100:0/1810555334 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55fdbf7efd50 0x55fdbf85de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:28.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.571+0000 7f97b7026b80 4 client.15483 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T11:19:28.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.571+0000 7f97b7026b80 1 -- 172.21.15.100:0/1810555334 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55fdbf7362c0 con 0x55fdbf7efd50 2024-05-10T11:19:28.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.572+0000 7f97aa52f700 1 --2- 172.21.15.100:0/1810555334 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55fdbf7efd50 0x55fdbf85de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:28.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.572+0000 7f978dffb700 10 client.15483 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:19:28.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.572+0000 7f97aa52f700 1 --2- 172.21.15.100:0/1810555334 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55fdbf7efd50 0x55fdbf85de10 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca9ab79fbcb312ea server_cookie=175f643b91ddc553 in_seq=0 out_seq=0 2024-05-10T11:19:28.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.573+0000 7f978dffb700 1 -- 172.21.15.100:0/1810555334 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55fdbf7362c0 con 0x55fdbf7efd50 2024-05-10T11:19:28.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.573+0000 7f978dffb700 10 client.15483 handle_command_reply: tid=0 2024-05-10T11:19:28.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.573+0000 7f97b7026b80 1 client.15483 shutdown 2024-05-10T11:19:28.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.573+0000 7f97b7026b80 1 -- 172.21.15.100:0/1810555334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fdbf7d6e70 msgr2=0x55fdbf7e4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:28.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.573+0000 7f97b7026b80 1 --2- 172.21.15.100:0/1810555334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fdbf7d6e70 0x55fdbf7e4de0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x55fdbf7f2a40 tx=0x7f9794010990 comp rx=0 tx=0).stop 2024-05-10T11:19:28.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.573+0000 7f97b7026b80 1 -- 172.21.15.100:0/1810555334 shutdown_connections 2024-05-10T11:19:28.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.573+0000 7f97b7026b80 1 --2- 172.21.15.100:0/1810555334 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55fdbf7efd50 0x55fdbf85de10 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:28.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.573+0000 7f97b7026b80 1 --2- 172.21.15.100:0/1810555334 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fdbf7dad00 0x55fdbf7e54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:28.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.573+0000 7f97b7026b80 1 --2- 172.21.15.100:0/1810555334 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fdbf7d6550 0x55fdbf7e8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:28.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.573+0000 7f97b7026b80 1 --2- 172.21.15.100:0/1810555334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fdbf7d6e70 0x55fdbf7e4de0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:28.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.573+0000 7f97b7026b80 1 -- 172.21.15.100:0/1810555334 >> 172.21.15.100:0/1810555334 conn(0x55fdbf702180 msgr2=0x55fdbf7d5bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:28.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.573+0000 7f978dffb700 0 client.15483 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:19:28.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.573+0000 7f97b7026b80 1 -- 172.21.15.100:0/1810555334 shutdown_connections 2024-05-10T11:19:28.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.573+0000 7f97b7026b80 1 -- 172.21.15.100:0/1810555334 wait complete. 2024-05-10T11:19:28.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.574+0000 7f97b7026b80 20 client.15483 trim_cache size 0 max 16384 2024-05-10T11:19:28.580 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:19:28.580 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-05-10T11:19:28.580 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "87bcd327-6303-424a-b038-7be2e37fee9a", 2024-05-10T11:19:28.580 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-05-10T11:19:28.580 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:19:28.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.574+0000 7f97ac793700 1 -- 172.21.15.100:0/251132858 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f97900b2880 msgr2=0x7f97900b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:28.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.574+0000 7f97ac793700 1 --2- 172.21.15.100:0/251132858 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f97900b2880 0x7f97900b4d40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f97a0006010 tx=0x7f97a000bca0 comp rx=0 tx=0).stop 2024-05-10T11:19:28.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.574+0000 7f97ac793700 1 -- 172.21.15.100:0/251132858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97a4108400 msgr2=0x7f97a41997b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:28.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.574+0000 7f97ac793700 1 --2- 172.21.15.100:0/251132858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97a4108400 0x7f97a41997b0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f979c00cc50 tx=0x7f979c00cf60 comp rx=0 tx=0).stop 2024-05-10T11:19:28.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.574+0000 7f97ac793700 1 -- 172.21.15.100:0/251132858 shutdown_connections 2024-05-10T11:19:28.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.574+0000 7f97ac793700 1 --2- 172.21.15.100:0/251132858 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f97900b2880 0x7f97900b4d40 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:28.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.574+0000 7f97ac793700 1 --2- 172.21.15.100:0/251132858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97a4108400 0x7f97a41997b0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:28.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.574+0000 7f97ac793700 1 --2- 172.21.15.100:0/251132858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97a4102390 0x7f97a419f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:28.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.574+0000 7f97ac793700 1 --2- 172.21.15.100:0/251132858 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97a4102d50 0x7f97a419fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:28.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.574+0000 7f97ac793700 1 -- 172.21.15.100:0/251132858 >> 172.21.15.100:0/251132858 conn(0x7f97a4079310 msgr2=0x7f97a4101760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:28.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.574+0000 7f97ac793700 1 -- 172.21.15.100:0/251132858 shutdown_connections 2024-05-10T11:19:28.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:28.574+0000 7f97ac793700 1 -- 172.21.15.100:0/251132858 wait complete. 2024-05-10T11:19:29.387 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T11:19:29.640 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:19:29.965 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:19:29 smithi100 ceph-mon[30296]: pgmap v453: 129 pgs: 129 active+clean; 663 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-10T11:19:29.965 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:19:29 smithi100 ceph-mon[30296]: scrub queued for path: / 2024-05-10T11:19:29.965 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:19:29 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:19:29.965 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:19:29 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:19:29.965 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:19:29 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:19:29.965 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:19:29 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:19:30.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:19:29 smithi178 ceph-mon[33359]: pgmap v453: 129 pgs: 129 active+clean; 663 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-10T11:19:30.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:19:29 smithi178 ceph-mon[33359]: scrub queued for path: / 2024-05-10T11:19:30.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:19:29 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:19:30.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:19:29 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:19:30.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:19:29 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:19:30.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:19:29 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:19:30.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:19:29 smithi188 ceph-mon[34075]: pgmap v453: 129 pgs: 129 active+clean; 663 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-10T11:19:30.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:19:29 smithi188 ceph-mon[34075]: scrub queued for path: / 2024-05-10T11:19:30.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:19:29 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:19:30.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:19:29 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:19:30.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:19:29 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:19:30.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:19:29 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:19:31.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.540+0000 7f69c8d9b700 1 -- 172.21.15.100:0/3957394492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69c4109510 msgr2=0x7f69c41098f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:31.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.540+0000 7f69c8d9b700 1 --2- 172.21.15.100:0/3957394492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69c4109510 0x7f69c41098f0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f69b4009990 tx=0x7f69b4009ca0 comp rx=0 tx=0).stop 2024-05-10T11:19:31.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.541+0000 7f69c8d9b700 1 -- 172.21.15.100:0/3957394492 shutdown_connections 2024-05-10T11:19:31.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.541+0000 7f69c8d9b700 1 --2- 172.21.15.100:0/3957394492 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f69c406bb40 0x7f69c4110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:31.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.541+0000 7f69c8d9b700 1 --2- 172.21.15.100:0/3957394492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69c4109510 0x7f69c41098f0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:31.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.541+0000 7f69c8d9b700 1 --2- 172.21.15.100:0/3957394492 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69c406b180 0x7f69c406b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:31.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.541+0000 7f69c8d9b700 1 -- 172.21.15.100:0/3957394492 >> 172.21.15.100:0/3957394492 conn(0x7f69c4079350 msgr2=0x7f69c4079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:31.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.541+0000 7f69c8d9b700 1 -- 172.21.15.100:0/3957394492 shutdown_connections 2024-05-10T11:19:31.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.541+0000 7f69c8d9b700 1 -- 172.21.15.100:0/3957394492 wait complete. 2024-05-10T11:19:31.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.542+0000 7f69c8d9b700 1 Processor -- start 2024-05-10T11:19:31.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.542+0000 7f69c8d9b700 1 -- start start 2024-05-10T11:19:31.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.542+0000 7f69c8d9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69c406b180 0x7f69c4102470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:31.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.542+0000 7f69c8d9b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69c406bb40 0x7f69c41029b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:31.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.542+0000 7f69c8d9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f69c4109510 0x7f69c4102ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:31.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.542+0000 7f69c8d9b700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f69c41134d0 con 0x7f69c406b180 2024-05-10T11:19:31.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.542+0000 7f69c8d9b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f69c4113350 con 0x7f69c4109510 2024-05-10T11:19:31.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.542+0000 7f69c8d9b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f69c4113650 con 0x7f69c406bb40 2024-05-10T11:19:31.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.542+0000 7f69c259c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69c406b180 0x7f69c4102470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:31.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.543+0000 7f69c259c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69c406b180 0x7f69c4102470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41186/0 (socket says 172.21.15.100:41186) 2024-05-10T11:19:31.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.543+0000 7f69c259c700 1 -- 172.21.15.100:0/1962142665 learned_addr learned my addr 172.21.15.100:0/1962142665 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:31.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.543+0000 7f69c1d9b700 1 --2- 172.21.15.100:0/1962142665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69c406bb40 0x7f69c41029b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:31.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.543+0000 7f69c2d9d700 1 --2- 172.21.15.100:0/1962142665 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f69c4109510 0x7f69c4102ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:31.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.543+0000 7f69c259c700 1 -- 172.21.15.100:0/1962142665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69c406bb40 msgr2=0x7f69c41029b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:31.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.543+0000 7f69c259c700 1 --2- 172.21.15.100:0/1962142665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69c406bb40 0x7f69c41029b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:31.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.543+0000 7f69c259c700 1 -- 172.21.15.100:0/1962142665 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f69c4109510 msgr2=0x7f69c4102ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:31.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.543+0000 7f69c259c700 1 --2- 172.21.15.100:0/1962142665 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f69c4109510 0x7f69c4102ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:31.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.543+0000 7f69c259c700 1 -- 172.21.15.100:0/1962142665 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69c41ac3b0 con 0x7f69c406b180 2024-05-10T11:19:31.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.543+0000 7f69c2d9d700 1 --2- 172.21.15.100:0/1962142665 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f69c4109510 0x7f69c4102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:19:31.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.543+0000 7f69c259c700 1 --2- 172.21.15.100:0/1962142665 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69c406b180 0x7f69c4102470 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f69b4006010 tx=0x7f69b400bfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:31.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.544+0000 7f69b37fe700 1 -- 172.21.15.100:0/1962142665 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b4027070 con 0x7f69c406b180 2024-05-10T11:19:31.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.544+0000 7f69b37fe700 1 -- 172.21.15.100:0/1962142665 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f69b4022470 con 0x7f69c406b180 2024-05-10T11:19:31.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.544+0000 7f69c8d9b700 1 -- 172.21.15.100:0/1962142665 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69c41ac5e0 con 0x7f69c406b180 2024-05-10T11:19:31.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.544+0000 7f69b37fe700 1 -- 172.21.15.100:0/1962142665 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b4017650 con 0x7f69c406b180 2024-05-10T11:19:31.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.544+0000 7f69c8d9b700 1 -- 172.21.15.100:0/1962142665 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69c41acae0 con 0x7f69c406b180 2024-05-10T11:19:31.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.547+0000 7f69b37fe700 1 -- 172.21.15.100:0/1962142665 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f69b4009d10 con 0x7f69c406b180 2024-05-10T11:19:31.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.548+0000 7f69b37fe700 1 --2- 172.21.15.100:0/1962142665 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f69ac0b2900 0x7f69ac0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:31.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.549+0000 7f69c1d9b700 1 --2- 172.21.15.100:0/1962142665 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f69ac0b2900 0x7f69ac0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:31.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.549+0000 7f69b37fe700 1 -- 172.21.15.100:0/1962142665 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f69b40e30f0 con 0x7f69c406b180 2024-05-10T11:19:31.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.549+0000 7f69d362eb80 1 Processor -- start 2024-05-10T11:19:31.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.549+0000 7f69d362eb80 1 -- start start 2024-05-10T11:19:31.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.549+0000 7f69c1d9b700 1 --2- 172.21.15.100:0/1962142665 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f69ac0b2900 0x7f69ac0b4dc0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f69c4103c30 tx=0x7f69b800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:31.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.549+0000 7f69d362eb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aeb2ce34d0 0x55aeb2ce38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:31.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.549+0000 7f69d362eb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aeb2ce3df0 0x55aeb2ce76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:31.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.549+0000 7f69d362eb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55aeb2ce7c80 0x55aeb2ce8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:31.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.549+0000 7f69d362eb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55aeb2c35860 con 0x55aeb2ce34d0 2024-05-10T11:19:31.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.549+0000 7f69d362eb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55aeb2bd47c0 con 0x55aeb2ce7c80 2024-05-10T11:19:31.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.549+0000 7f69d362eb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55aeb2c31820 con 0x55aeb2ce3df0 2024-05-10T11:19:31.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.550+0000 7f69c259c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aeb2ce34d0 0x55aeb2ce38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:31.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.550+0000 7f69c259c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aeb2ce34d0 0x55aeb2ce38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41190/0 (socket says 172.21.15.100:41190) 2024-05-10T11:19:31.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.550+0000 7f69c259c700 1 -- 172.21.15.100:0/3468306750 learned_addr learned my addr 172.21.15.100:0/3468306750 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:31.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.550+0000 7f69c1d9b700 1 --2- 172.21.15.100:0/3468306750 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aeb2ce3df0 0x55aeb2ce76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:31.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.550+0000 7f69c2d9d700 1 --2- 172.21.15.100:0/3468306750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55aeb2ce7c80 0x55aeb2ce8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:31.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.550+0000 7f69c259c700 1 -- 172.21.15.100:0/3468306750 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aeb2ce3df0 msgr2=0x55aeb2ce76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:31.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.550+0000 7f69c259c700 1 --2- 172.21.15.100:0/3468306750 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aeb2ce3df0 0x55aeb2ce76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:31.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.550+0000 7f69c259c700 1 -- 172.21.15.100:0/3468306750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55aeb2ce7c80 msgr2=0x55aeb2ce8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:31.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.550+0000 7f69c259c700 1 --2- 172.21.15.100:0/3468306750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55aeb2ce7c80 0x55aeb2ce8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:31.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.550+0000 7f69c259c700 1 -- 172.21.15.100:0/3468306750 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aeb2b828d0 con 0x55aeb2ce34d0 2024-05-10T11:19:31.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.550+0000 7f69c259c700 1 --2- 172.21.15.100:0/3468306750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aeb2ce34d0 0x55aeb2ce38b0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f69b40aa3d0 tx=0x7f69b40aa020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2bb943a8716831e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:31.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.550+0000 7f69b0ff9700 1 -- 172.21.15.100:0/3468306750 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b40b3210 con 0x55aeb2ce34d0 2024-05-10T11:19:31.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.551+0000 7f69b0ff9700 1 -- 172.21.15.100:0/3468306750 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f69b4022920 con 0x55aeb2ce34d0 2024-05-10T11:19:31.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.551+0000 7f69b0ff9700 1 -- 172.21.15.100:0/3468306750 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b40b2fe0 con 0x55aeb2ce34d0 2024-05-10T11:19:31.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.551+0000 7f69d362eb80 1 -- 172.21.15.100:0/3468306750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aeb2ce34d0 msgr2=0x55aeb2ce38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:31.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.551+0000 7f69d362eb80 1 --2- 172.21.15.100:0/3468306750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aeb2ce34d0 0x55aeb2ce38b0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f69b40aa3d0 tx=0x7f69b40aa020 comp rx=0 tx=0).stop 2024-05-10T11:19:31.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.551+0000 7f69d362eb80 1 -- 172.21.15.100:0/3468306750 shutdown_connections 2024-05-10T11:19:31.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.552+0000 7f69d362eb80 1 --2- 172.21.15.100:0/3468306750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55aeb2ce7c80 0x55aeb2ce8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:31.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.552+0000 7f69d362eb80 1 --2- 172.21.15.100:0/3468306750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aeb2ce34d0 0x55aeb2ce38b0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:31.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.552+0000 7f69d362eb80 1 --2- 172.21.15.100:0/3468306750 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aeb2ce3df0 0x55aeb2ce76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:31.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.552+0000 7f69d362eb80 1 -- 172.21.15.100:0/3468306750 >> 172.21.15.100:0/3468306750 conn(0x55aeb2c0f100 msgr2=0x55aeb2cddc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:31.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.552+0000 7f69d362eb80 1 -- 172.21.15.100:0/3468306750 shutdown_connections 2024-05-10T11:19:31.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.552+0000 7f69d362eb80 1 -- 172.21.15.100:0/3468306750 wait complete. 2024-05-10T11:19:31.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.552+0000 7f69d362eb80 1 -- start start 2024-05-10T11:19:31.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.552+0000 7f69d362eb80 1 Processor -- start 2024-05-10T11:19:31.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.553+0000 7f69d362eb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aeb2ce34d0 0x55aeb2cef1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:31.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.553+0000 7f69d362eb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55aeb2ce3df0 0x55aeb2cec5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:31.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.553+0000 7f69d362eb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aeb2ce7c80 0x55aeb2cecb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:31.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.553+0000 7f69c259c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aeb2ce34d0 0x55aeb2cef1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:31.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.553+0000 7f69c259c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aeb2ce34d0 0x55aeb2cef1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41194/0 (socket says 172.21.15.100:41194) 2024-05-10T11:19:31.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.553+0000 7f69c259c700 1 -- 172.21.15.100:0/2133095539 learned_addr learned my addr 172.21.15.100:0/2133095539 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:31.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.553+0000 7f69d362eb80 1 -- 172.21.15.100:0/2133095539 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55aeb2cefc50 con 0x55aeb2ce34d0 2024-05-10T11:19:31.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.553+0000 7f69d362eb80 1 -- 172.21.15.100:0/2133095539 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55aeb2b9bc30 con 0x55aeb2ce3df0 2024-05-10T11:19:31.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.553+0000 7f69d362eb80 1 -- 172.21.15.100:0/2133095539 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55aeb2cefdd0 con 0x55aeb2ce7c80 2024-05-10T11:19:31.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.553+0000 7f69c1d9b700 1 --2- 172.21.15.100:0/2133095539 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55aeb2ce3df0 0x55aeb2cec5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:31.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.553+0000 7f69c2d9d700 1 --2- 172.21.15.100:0/2133095539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aeb2ce7c80 0x55aeb2cecb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:31.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.553+0000 7f69d362eb80 10 client.0 _ll_register_callbacks cb 0x55aeb2bcd920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:19:31.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.553+0000 7f69c259c700 1 -- 172.21.15.100:0/2133095539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aeb2ce7c80 msgr2=0x55aeb2cecb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:31.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.553+0000 7f69c259c700 1 --2- 172.21.15.100:0/2133095539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aeb2ce7c80 0x55aeb2cecb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:31.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.553+0000 7f69c259c700 1 -- 172.21.15.100:0/2133095539 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55aeb2ce3df0 msgr2=0x55aeb2cec5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:31.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.553+0000 7f69c259c700 1 --2- 172.21.15.100:0/2133095539 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55aeb2ce3df0 0x55aeb2cec5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:31.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.553+0000 7f69c259c700 1 -- 172.21.15.100:0/2133095539 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aeb2bd1690 con 0x55aeb2ce34d0 2024-05-10T11:19:31.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.553+0000 7f69c259c700 1 --2- 172.21.15.100:0/2133095539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aeb2ce34d0 0x55aeb2cef1e0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f69b40059d0 tx=0x7f69b40b2f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ada98303f10daa7b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:31.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.553+0000 7f69a9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:19:31.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.554+0000 7f69a9ffb700 1 -- 172.21.15.100:0/2133095539 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b40b3210 con 0x55aeb2ce34d0 2024-05-10T11:19:31.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.554+0000 7f69a9ffb700 1 -- 172.21.15.100:0/2133095539 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f69b40a6a30 con 0x55aeb2ce34d0 2024-05-10T11:19:31.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.554+0000 7f69d362eb80 1 -- 172.21.15.100:0/2133095539 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69b4022920 con 0x55aeb2ce34d0 2024-05-10T11:19:31.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.554+0000 7f69a9ffb700 1 -- 172.21.15.100:0/2133095539 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f69b40a4d70 con 0x55aeb2ce34d0 2024-05-10T11:19:31.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.554+0000 7f69d362eb80 10 client.15495 fetch_fsmap learned FSMap version 27 2024-05-10T11:19:31.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.554+0000 7f69d362eb80 1 -- 172.21.15.100:0/2133095539 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55aeb2cefc50 con 0x55aeb2ce34d0 2024-05-10T11:19:31.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.554+0000 7f69a9ffb700 1 -- 172.21.15.100:0/2133095539 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f69b4022920 con 0x55aeb2ce34d0 2024-05-10T11:19:31.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.554+0000 7f69a9ffb700 1 -- 172.21.15.100:0/2133095539 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f69b40a9830 con 0x55aeb2ce34d0 2024-05-10T11:19:31.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.555+0000 7f69d362eb80 10 client.15495 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T11:19:31.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.555+0000 7f69d362eb80 10 client.15495 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:19:31.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.555+0000 7f69d362eb80 20 client.15495 populate_metadata read hostname 'smithi100' 2024-05-10T11:19:31.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.555+0000 7f69d362eb80 1 --2- 172.21.15.100:0/2133095539 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55aeb2cfc880 0x55aeb2cfecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:31.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.555+0000 7f69d362eb80 4 client.15495 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:19:31.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.555+0000 7f69d362eb80 1 -- 172.21.15.100:0/2133095539 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aeb2caf650 con 0x55aeb2cfc880 2024-05-10T11:19:31.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.555+0000 7f69c1d9b700 1 --2- 172.21.15.100:0/2133095539 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55aeb2cfc880 0x55aeb2cfecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:31.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.555+0000 7f69c1d9b700 1 --2- 172.21.15.100:0/2133095539 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55aeb2cfc880 0x55aeb2cfecc0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=424627ea0eb59238 server_cookie=dfc15bf31ff5a4e5 in_seq=0 out_seq=0 2024-05-10T11:19:31.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.555+0000 7f69a9ffb700 10 client.15495 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:19:31.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.556+0000 7f69a9ffb700 1 -- 172.21.15.100:0/2133095539 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55aeb2caf650 con 0x55aeb2cfc880 2024-05-10T11:19:31.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.556+0000 7f69a9ffb700 10 client.15495 handle_command_reply: tid=0 2024-05-10T11:19:31.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.556+0000 7f69d362eb80 1 client.15495 shutdown 2024-05-10T11:19:31.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.556+0000 7f69d362eb80 1 -- 172.21.15.100:0/2133095539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aeb2ce34d0 msgr2=0x55aeb2cef1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:31.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.556+0000 7f69d362eb80 1 --2- 172.21.15.100:0/2133095539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aeb2ce34d0 0x55aeb2cef1e0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f69b40059d0 tx=0x7f69b40b2f90 comp rx=0 tx=0).stop 2024-05-10T11:19:31.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.556+0000 7f69d362eb80 1 -- 172.21.15.100:0/2133095539 shutdown_connections 2024-05-10T11:19:31.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.556+0000 7f69d362eb80 1 --2- 172.21.15.100:0/2133095539 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55aeb2cfc880 0x55aeb2cfecc0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:31.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.556+0000 7f69d362eb80 1 --2- 172.21.15.100:0/2133095539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aeb2ce7c80 0x55aeb2cecb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:31.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.556+0000 7f69d362eb80 1 --2- 172.21.15.100:0/2133095539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aeb2ce34d0 0x55aeb2cef1e0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:31.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.556+0000 7f69a9ffb700 0 client.15495 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:19:31.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.556+0000 7f69d362eb80 1 --2- 172.21.15.100:0/2133095539 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55aeb2ce3df0 0x55aeb2cec5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:31.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.556+0000 7f69d362eb80 1 -- 172.21.15.100:0/2133095539 >> 172.21.15.100:0/2133095539 conn(0x55aeb2c0f100 msgr2=0x55aeb2cd8720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:31.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.557+0000 7f69d362eb80 1 -- 172.21.15.100:0/2133095539 shutdown_connections 2024-05-10T11:19:31.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.557+0000 7f69d362eb80 1 -- 172.21.15.100:0/2133095539 wait complete. 2024-05-10T11:19:31.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.557+0000 7f69d362eb80 20 client.15495 trim_cache size 0 max 16384 2024-05-10T11:19:31.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.576+0000 7f69d362eb80 1 Processor -- start 2024-05-10T11:19:31.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.576+0000 7f69d362eb80 1 -- start start 2024-05-10T11:19:31.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.577+0000 7f69d362eb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aeb2c0f100 0x55aeb2cf8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:31.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.577+0000 7f69d362eb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55aeb2ce3df0 0x55aeb2ce29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:31.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.577+0000 7f69d362eb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aeb2ce7c80 0x55aeb2d04590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:31.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.577+0000 7f69d362eb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f69c41b7e20 con 0x55aeb2ce7c80 2024-05-10T11:19:31.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.577+0000 7f69d362eb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f69c41b56b0 con 0x55aeb2ce3df0 2024-05-10T11:19:31.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.577+0000 7f69c259c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aeb2c0f100 0x55aeb2cf8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:31.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.577+0000 7f69c259c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aeb2c0f100 0x55aeb2cf8ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:57188/0 (socket says 172.21.15.100:57188) 2024-05-10T11:19:31.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.577+0000 7f69c259c700 1 -- 172.21.15.100:0/3237701880 learned_addr learned my addr 172.21.15.100:0/3237701880 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:31.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.577+0000 7f69d362eb80 1 -- 172.21.15.100:0/3237701880 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f69c41b3500 con 0x55aeb2c0f100 2024-05-10T11:19:31.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.577+0000 7f69c2d9d700 1 --2- 172.21.15.100:0/3237701880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aeb2ce7c80 0x55aeb2d04590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:31.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.577+0000 7f69c1d9b700 1 --2- 172.21.15.100:0/3237701880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55aeb2ce3df0 0x55aeb2ce29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:31.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.578+0000 7f69c1d9b700 1 -- 172.21.15.100:0/3237701880 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aeb2c0f100 msgr2=0x55aeb2cf8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:31.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.578+0000 7f69c1d9b700 1 --2- 172.21.15.100:0/3237701880 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aeb2c0f100 0x55aeb2cf8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:31.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.578+0000 7f69c1d9b700 1 -- 172.21.15.100:0/3237701880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aeb2ce7c80 msgr2=0x55aeb2d04590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:31.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.578+0000 7f69c1d9b700 1 --2- 172.21.15.100:0/3237701880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aeb2ce7c80 0x55aeb2d04590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:31.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.578+0000 7f69c1d9b700 1 -- 172.21.15.100:0/3237701880 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aeb2ce7530 con 0x55aeb2ce3df0 2024-05-10T11:19:31.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.578+0000 7f69c259c700 1 --2- 172.21.15.100:0/3237701880 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aeb2c0f100 0x55aeb2cf8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:19:31.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.578+0000 7f69c1d9b700 1 --2- 172.21.15.100:0/3237701880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55aeb2ce3df0 0x55aeb2ce29f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x55aeb2cff530 tx=0x7f69b8013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12d90dac04bd4cb4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:31.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.578+0000 7f69abfff700 1 -- 172.21.15.100:0/3237701880 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b8020dd0 con 0x55aeb2ce3df0 2024-05-10T11:19:31.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.578+0000 7f69abfff700 1 -- 172.21.15.100:0/3237701880 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f69b802d070 con 0x55aeb2ce3df0 2024-05-10T11:19:31.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.578+0000 7f69abfff700 1 -- 172.21.15.100:0/3237701880 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b80116a0 con 0x55aeb2ce3df0 2024-05-10T11:19:31.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.579+0000 7f69d362eb80 1 -- 172.21.15.100:0/3237701880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55aeb2ce3df0 msgr2=0x55aeb2ce29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:31.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.579+0000 7f69d362eb80 1 --2- 172.21.15.100:0/3237701880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55aeb2ce3df0 0x55aeb2ce29f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x55aeb2cff530 tx=0x7f69b8013000 comp rx=0 tx=0).stop 2024-05-10T11:19:31.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.579+0000 7f69d362eb80 1 -- 172.21.15.100:0/3237701880 shutdown_connections 2024-05-10T11:19:31.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.579+0000 7f69d362eb80 1 --2- 172.21.15.100:0/3237701880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aeb2ce7c80 0x55aeb2d04590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:31.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.579+0000 7f69d362eb80 1 --2- 172.21.15.100:0/3237701880 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aeb2c0f100 0x55aeb2cf8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:31.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.579+0000 7f69d362eb80 1 --2- 172.21.15.100:0/3237701880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55aeb2ce3df0 0x55aeb2ce29f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:31.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.579+0000 7f69d362eb80 1 -- 172.21.15.100:0/3237701880 >> 172.21.15.100:0/3237701880 conn(0x55aeb2cd8ab0 msgr2=0x55aeb2ce22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:31.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.580+0000 7f69d362eb80 1 -- 172.21.15.100:0/3237701880 shutdown_connections 2024-05-10T11:19:31.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.580+0000 7f69d362eb80 1 -- 172.21.15.100:0/3237701880 wait complete. 2024-05-10T11:19:31.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.580+0000 7f69d362eb80 1 -- start start 2024-05-10T11:19:31.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.580+0000 7f69d362eb80 1 Processor -- start 2024-05-10T11:19:31.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.580+0000 7f69d362eb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aeb2c0f100 0x55aeb2cf3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:31.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.581+0000 7f69d362eb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55aeb2ce3df0 0x55aeb2cf3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:31.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.581+0000 7f69d362eb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aeb2ce7c80 0x55aeb2d66150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:31.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.581+0000 7f69d362eb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f69c41b3b70 con 0x55aeb2c0f100 2024-05-10T11:19:31.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.581+0000 7f69d362eb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f69c41b4fe0 con 0x55aeb2ce3df0 2024-05-10T11:19:31.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.581+0000 7f69d362eb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f69c41b4240 con 0x55aeb2ce7c80 2024-05-10T11:19:31.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.581+0000 7f69d362eb80 10 client.0 _ll_register_callbacks cb 0x55aeb2bcd920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:19:31.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.581+0000 7f69c259c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aeb2c0f100 0x55aeb2cf3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:31.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.581+0000 7f69c259c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aeb2c0f100 0x55aeb2cf3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41212/0 (socket says 172.21.15.100:41212) 2024-05-10T11:19:31.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.581+0000 7f69c259c700 1 -- 172.21.15.100:0/304085718 learned_addr learned my addr 172.21.15.100:0/304085718 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:31.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.581+0000 7f69c1d9b700 1 --2- 172.21.15.100:0/304085718 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55aeb2ce3df0 0x55aeb2cf3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:31.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.581+0000 7f69c2d9d700 1 --2- 172.21.15.100:0/304085718 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aeb2ce7c80 0x55aeb2d66150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:31.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.581+0000 7f69c1d9b700 1 -- 172.21.15.100:0/304085718 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aeb2ce7c80 msgr2=0x55aeb2d66150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:31.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.581+0000 7f69c1d9b700 1 --2- 172.21.15.100:0/304085718 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aeb2ce7c80 0x55aeb2d66150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:31.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.581+0000 7f69c1d9b700 1 -- 172.21.15.100:0/304085718 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aeb2c0f100 msgr2=0x55aeb2cf3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:31.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.581+0000 7f69c1d9b700 1 --2- 172.21.15.100:0/304085718 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aeb2c0f100 0x55aeb2cf3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:31.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.581+0000 7f69c1d9b700 1 -- 172.21.15.100:0/304085718 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aeb2cf91e0 con 0x55aeb2ce3df0 2024-05-10T11:19:31.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.581+0000 7f69c259c700 1 --2- 172.21.15.100:0/304085718 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aeb2c0f100 0x55aeb2cf3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:19:31.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.582+0000 7f69c1d9b700 1 --2- 172.21.15.100:0/304085718 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55aeb2ce3df0 0x55aeb2cf3840 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x55aeb2cdecc0 tx=0x7f69b80282f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ff0b09d45f60538 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:31.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.582+0000 7f69a9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:19:31.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.582+0000 7f69a9ffb700 1 -- 172.21.15.100:0/304085718 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b8028460 con 0x55aeb2ce3df0 2024-05-10T11:19:31.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.582+0000 7f69d362eb80 1 -- 172.21.15.100:0/304085718 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69b802d070 con 0x55aeb2ce3df0 2024-05-10T11:19:31.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.582+0000 7f69a9ffb700 1 -- 172.21.15.100:0/304085718 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f69b8026050 con 0x55aeb2ce3df0 2024-05-10T11:19:31.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.582+0000 7f69a9ffb700 1 -- 172.21.15.100:0/304085718 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f69b803f050 con 0x55aeb2ce3df0 2024-05-10T11:19:31.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.582+0000 7f69d362eb80 10 client.25043 fetch_fsmap learned FSMap version 27 2024-05-10T11:19:31.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.582+0000 7f69d362eb80 1 -- 172.21.15.100:0/304085718 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55aeb2d677d0 con 0x55aeb2ce3df0 2024-05-10T11:19:31.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.582+0000 7f69a9ffb700 1 -- 172.21.15.100:0/304085718 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f69b802d070 con 0x55aeb2ce3df0 2024-05-10T11:19:31.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.582+0000 7f69a9ffb700 1 -- 172.21.15.100:0/304085718 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f69b8028ce0 con 0x55aeb2ce3df0 2024-05-10T11:19:31.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.582+0000 7f69d362eb80 10 client.25043 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T11:19:31.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.582+0000 7f69d362eb80 10 client.25043 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:19:31.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.582+0000 7f69d362eb80 20 client.25043 populate_metadata read hostname 'smithi100' 2024-05-10T11:19:31.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.583+0000 7f69d362eb80 1 --2- 172.21.15.100:0/304085718 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55aeb2d67dd0 0x55aeb2d6a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:31.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.583+0000 7f69d362eb80 4 client.25043 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T11:19:31.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.583+0000 7f69d362eb80 1 -- 172.21.15.100:0/304085718 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55aeb2c43240 con 0x55aeb2d67dd0 2024-05-10T11:19:31.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.583+0000 7f69c259c700 1 --2- 172.21.15.100:0/304085718 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55aeb2d67dd0 0x55aeb2d6a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:31.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.583+0000 7f69a9ffb700 10 client.25043 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:19:31.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.583+0000 7f69c259c700 1 --2- 172.21.15.100:0/304085718 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55aeb2d67dd0 0x55aeb2d6a1c0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98fb970b7706f44a server_cookie=2cb5795bc98f981b in_seq=0 out_seq=0 2024-05-10T11:19:31.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.583+0000 7f69a9ffb700 1 -- 172.21.15.100:0/304085718 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55aeb2c43240 con 0x55aeb2d67dd0 2024-05-10T11:19:31.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.583+0000 7f69a9ffb700 10 client.25043 handle_command_reply: tid=0 2024-05-10T11:19:31.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.584+0000 7f69d362eb80 1 client.25043 shutdown 2024-05-10T11:19:31.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.584+0000 7f69d362eb80 1 -- 172.21.15.100:0/304085718 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55aeb2ce3df0 msgr2=0x55aeb2cf3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:31.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.584+0000 7f69d362eb80 1 --2- 172.21.15.100:0/304085718 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55aeb2ce3df0 0x55aeb2cf3840 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x55aeb2cdecc0 tx=0x7f69b80282f0 comp rx=0 tx=0).stop 2024-05-10T11:19:31.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.584+0000 7f69d362eb80 1 -- 172.21.15.100:0/304085718 shutdown_connections 2024-05-10T11:19:31.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.584+0000 7f69d362eb80 1 --2- 172.21.15.100:0/304085718 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55aeb2d67dd0 0x55aeb2d6a1c0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:31.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.584+0000 7f69d362eb80 1 --2- 172.21.15.100:0/304085718 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aeb2ce7c80 0x55aeb2d66150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:31.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.584+0000 7f69d362eb80 1 --2- 172.21.15.100:0/304085718 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aeb2c0f100 0x55aeb2cf3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:31.591 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:19:31.591 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-05-10T11:19:31.591 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-05-10T11:19:31.591 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:19:31.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.584+0000 7f69a9ffb700 0 client.25043 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:19:31.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.584+0000 7f69d362eb80 1 --2- 172.21.15.100:0/304085718 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55aeb2ce3df0 0x55aeb2cf3840 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:31.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.584+0000 7f69d362eb80 1 -- 172.21.15.100:0/304085718 >> 172.21.15.100:0/304085718 conn(0x55aeb2cd8ab0 msgr2=0x55aeb2ce6db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:31.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.584+0000 7f69d362eb80 1 -- 172.21.15.100:0/304085718 shutdown_connections 2024-05-10T11:19:31.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.584+0000 7f69d362eb80 1 -- 172.21.15.100:0/304085718 wait complete. 2024-05-10T11:19:31.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.584+0000 7f69d362eb80 20 client.25043 trim_cache size 0 max 16384 2024-05-10T11:19:31.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.585+0000 7f69c8d9b700 1 -- 172.21.15.100:0/1962142665 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f69ac0b2900 msgr2=0x7f69ac0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:31.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.585+0000 7f69c8d9b700 1 --2- 172.21.15.100:0/1962142665 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f69ac0b2900 0x7f69ac0b4dc0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f69c4103c30 tx=0x7f69b800b410 comp rx=0 tx=0).stop 2024-05-10T11:19:31.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.585+0000 7f69c8d9b700 1 -- 172.21.15.100:0/1962142665 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69c406b180 msgr2=0x7f69c4102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:31.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.585+0000 7f69c8d9b700 1 --2- 172.21.15.100:0/1962142665 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69c406b180 0x7f69c4102470 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f69b4006010 tx=0x7f69b400bfc0 comp rx=0 tx=0).stop 2024-05-10T11:19:31.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.585+0000 7f69c8d9b700 1 -- 172.21.15.100:0/1962142665 shutdown_connections 2024-05-10T11:19:31.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.585+0000 7f69c8d9b700 1 --2- 172.21.15.100:0/1962142665 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f69ac0b2900 0x7f69ac0b4dc0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:31.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.585+0000 7f69c8d9b700 1 --2- 172.21.15.100:0/1962142665 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f69c4109510 0x7f69c4102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:31.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.585+0000 7f69c8d9b700 1 --2- 172.21.15.100:0/1962142665 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69c406b180 0x7f69c4102470 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:31.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.585+0000 7f69c8d9b700 1 --2- 172.21.15.100:0/1962142665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69c406bb40 0x7f69c41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:31.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.585+0000 7f69c8d9b700 1 -- 172.21.15.100:0/1962142665 >> 172.21.15.100:0/1962142665 conn(0x7f69c4079350 msgr2=0x7f69c4101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:31.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.585+0000 7f69c8d9b700 1 -- 172.21.15.100:0/1962142665 shutdown_connections 2024-05-10T11:19:31.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:31.585+0000 7f69c8d9b700 1 -- 172.21.15.100:0/1962142665 wait complete. 2024-05-10T11:19:31.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:19:31 smithi100 ceph-mon[30296]: pgmap v454: 129 pgs: 129 active+clean; 663 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-10T11:19:32.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:19:31 smithi178 ceph-mon[33359]: pgmap v454: 129 pgs: 129 active+clean; 663 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-10T11:19:32.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:19:31 smithi188 ceph-mon[34075]: pgmap v454: 129 pgs: 129 active+clean; 663 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-10T11:19:32.386 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T11:19:32.386 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T11:19:32.663 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:19:34.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:19:33 smithi178 ceph-mon[33359]: pgmap v455: 129 pgs: 129 active+clean; 664 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-10T11:19:34.004 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:19:33 smithi100 ceph-mon[30296]: pgmap v455: 129 pgs: 129 active+clean; 664 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-10T11:19:34.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:19:33 smithi188 ceph-mon[34075]: pgmap v455: 129 pgs: 129 active+clean; 664 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-10T11:19:34.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:34.413+0000 7f552968c700 1 -- 172.21.15.100:0/531007257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5524107760 msgr2=0x7f552410f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:34.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:34.413+0000 7f552968c700 1 --2- 172.21.15.100:0/531007257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5524107760 0x7f552410f2e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f551c00b0d0 tx=0x7f551c00b3e0 comp rx=0 tx=0).stop 2024-05-10T11:19:34.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:34.413+0000 7f552968c700 1 -- 172.21.15.100:0/531007257 shutdown_connections 2024-05-10T11:19:34.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:34.413+0000 7f552968c700 1 --2- 172.21.15.100:0/531007257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5524107760 0x7f552410f2e0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:34.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:34.413+0000 7f552968c700 1 --2- 172.21.15.100:0/531007257 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5524102e00 0x7f55241031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:34.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:34.413+0000 7f552968c700 1 --2- 172.21.15.100:0/531007257 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55241037b0 0x7f5524107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:34.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:34.413+0000 7f552968c700 1 -- 172.21.15.100:0/531007257 >> 172.21.15.100:0/531007257 conn(0x7f5524078a10 msgr2=0x7f55240ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:34.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:34.413+0000 7f552968c700 1 -- 172.21.15.100:0/531007257 shutdown_connections 2024-05-10T11:19:34.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:34.413+0000 7f552968c700 1 -- 172.21.15.100:0/531007257 wait complete. 2024-05-10T11:19:34.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:34.414+0000 7f552968c700 1 Processor -- start 2024-05-10T11:19:34.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:34.414+0000 7f552968c700 1 -- start start 2024-05-10T11:19:34.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:34.415+0000 7f552968c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5524102e00 0x7f552419e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:34.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:34.415+0000 7f552968c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55241037b0 0x7f552419eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:34.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:34.415+0000 7f552968c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5524107760 0x7f55241986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:34.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:34.415+0000 7f552968c700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f55241123e0 con 0x7f5524107760 2024-05-10T11:19:34.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:34.415+0000 7f552968c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5524112260 con 0x7f5524102e00 2024-05-10T11:19:34.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:34.415+0000 7f552968c700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5524112560 con 0x7f55241037b0 2024-05-10T11:19:34.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:34.415+0000 7f55237fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5524107760 0x7f55241986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:34.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:34.415+0000 7f55237fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5524107760 0x7f55241986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41222/0 (socket says 172.21.15.100:41222) 2024-05-10T11:19:34.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:34.415+0000 7f55237fe700 1 -- 172.21.15.100:0/3678506164 learned_addr learned my addr 172.21.15.100:0/3678506164 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:34.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:34.415+0000 7f55227fc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55241037b0 0x7f552419eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:34.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:34.415+0000 7f5522ffd700 1 --2- 172.21.15.100:0/3678506164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5524102e00 0x7f552419e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:34.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:34.415+0000 7f55237fe700 1 -- 172.21.15.100:0/3678506164 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55241037b0 msgr2=0x7f552419eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:34.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:34.415+0000 7f55237fe700 1 --2- 172.21.15.100:0/3678506164 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55241037b0 0x7f552419eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:34.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:34.415+0000 7f55237fe700 1 -- 172.21.15.100:0/3678506164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5524102e00 msgr2=0x7f552419e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:34.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:34.415+0000 7f55237fe700 1 --2- 172.21.15.100:0/3678506164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5524102e00 0x7f552419e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:34.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:34.415+0000 7f55237fe700 1 -- 172.21.15.100:0/3678506164 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5524198fa0 con 0x7f5524107760 2024-05-10T11:19:34.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:34.415+0000 7f55237fe700 1 --2- 172.21.15.100:0/3678506164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5524107760 0x7f55241986c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f551c009850 tx=0x7f551c007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:34.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:34.416+0000 7f550bfff700 1 -- 172.21.15.100:0/3678506164 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f551c01f070 con 0x7f5524107760 2024-05-10T11:19:34.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:34.416+0000 7f550bfff700 1 -- 172.21.15.100:0/3678506164 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f551c005500 con 0x7f5524107760 2024-05-10T11:19:34.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:34.416+0000 7f550bfff700 1 -- 172.21.15.100:0/3678506164 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f551c01b8f0 con 0x7f5524107760 2024-05-10T11:19:34.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:34.416+0000 7f552968c700 1 -- 172.21.15.100:0/3678506164 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5524199230 con 0x7f5524107760 2024-05-10T11:19:34.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:34.416+0000 7f552968c700 1 -- 172.21.15.100:0/3678506164 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5524101a90 con 0x7f5524107760 2024-05-10T11:19:34.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:34.417+0000 7f552968c700 1 -- 172.21.15.100:0/3678506164 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5524103bb0 con 0x7f5524107760 2024-05-10T11:19:34.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:34.419+0000 7f550bfff700 1 -- 172.21.15.100:0/3678506164 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f551c022070 con 0x7f5524107760 2024-05-10T11:19:34.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:34.420+0000 7f550bfff700 1 --2- 172.21.15.100:0/3678506164 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f550c0b2900 0x7f550c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:34.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:34.420+0000 7f5522ffd700 1 --2- 172.21.15.100:0/3678506164 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f550c0b2900 0x7f550c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:34.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:34.420+0000 7f550bfff700 1 -- 172.21.15.100:0/3678506164 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f551c00f030 con 0x7f5524107760 2024-05-10T11:19:34.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:34.420+0000 7f5522ffd700 1 --2- 172.21.15.100:0/3678506164 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f550c0b2900 0x7f550c0b4dc0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5514004530 tx=0x7f551400f040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:34.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:34.427+0000 7f550bfff700 1 -- 172.21.15.100:0/3678506164 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f551c0ab1c0 con 0x7f5524107760 2024-05-10T11:19:34.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:34.816+0000 7f552968c700 1 -- 172.21.15.100:0/3678506164 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5524102520 con 0x7f5524107760 2024-05-10T11:19:34.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:34.817+0000 7f550bfff700 1 -- 172.21.15.100:0/3678506164 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11894 (secure 0 0 0) 0x7f551c0aaac0 con 0x7f5524107760 2024-05-10T11:19:34.817 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:19:34.817 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2072225012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2072225012},{"type":"v1","addr":"172.21.15.100:6835","nonce":2072225012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/401939540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":401939540},{"type":"v1","addr":"172.21.15.100:6837","nonce":401939540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14571,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/3946311025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3946311025},{"type":"v1","addr":"172.21.15.100:6839","nonce":3946311025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3736371660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3736371660},{"type":"v1","addr":"172.21.15.100:6841","nonce":3736371660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/2741810791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2741810791},{"type":"v1","addr":"172.21.15.178:6833","nonce":2741810791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1704891041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1704891041},{"type":"v1","addr":"172.21.15.188:6833","nonce":1704891041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T11:14:27.242452+0000","modified":"2024-05-10T11:18:23.454860+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24439,"mds_1":24437,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/2124076312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2124076312},{"type":"v1","addr":"172.21.15.178:6835","nonce":2124076312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/3224358425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3224358425},{"type":"v1","addr":"172.21.15.178:6837","nonce":3224358425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/515547690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":515547690},{"type":"v1","addr":"172.21.15.188:6835","nonce":515547690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/1943342620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1943342620},{"type":"v1","addr":"172.21.15.188:6837","nonce":1943342620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.178:6839/3760985404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3760985404},{"type":"v1","addr":"172.21.15.178:6839","nonce":3760985404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.188:6839/3194917070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":3194917070},{"type":"v1","addr":"172.21.15.188:6839","nonce":3194917070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24439,"qdb_cluster":[24427,24437,24439]},"id":1}]} 2024-05-10T11:19:34.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:34.820+0000 7f552968c700 1 -- 172.21.15.100:0/3678506164 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f550c0b2900 msgr2=0x7f550c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:34.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:34.820+0000 7f552968c700 1 --2- 172.21.15.100:0/3678506164 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f550c0b2900 0x7f550c0b4dc0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5514004530 tx=0x7f551400f040 comp rx=0 tx=0).stop 2024-05-10T11:19:34.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:34.820+0000 7f552968c700 1 -- 172.21.15.100:0/3678506164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5524107760 msgr2=0x7f55241986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:34.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:34.820+0000 7f552968c700 1 --2- 172.21.15.100:0/3678506164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5524107760 0x7f55241986c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f551c009850 tx=0x7f551c007d90 comp rx=0 tx=0).stop 2024-05-10T11:19:34.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:34.820+0000 7f552968c700 1 -- 172.21.15.100:0/3678506164 shutdown_connections 2024-05-10T11:19:34.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:34.820+0000 7f552968c700 1 --2- 172.21.15.100:0/3678506164 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f550c0b2900 0x7f550c0b4dc0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:34.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:34.820+0000 7f552968c700 1 --2- 172.21.15.100:0/3678506164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5524107760 0x7f55241986c0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:34.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:34.820+0000 7f552968c700 1 --2- 172.21.15.100:0/3678506164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5524102e00 0x7f552419e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:34.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:34.820+0000 7f552968c700 1 --2- 172.21.15.100:0/3678506164 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55241037b0 0x7f552419eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:34.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:34.820+0000 7f552968c700 1 -- 172.21.15.100:0/3678506164 >> 172.21.15.100:0/3678506164 conn(0x7f5524078a10 msgr2=0x7f55240ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:34.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:34.820+0000 7f552968c700 1 -- 172.21.15.100:0/3678506164 shutdown_connections 2024-05-10T11:19:34.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:34.820+0000 7f552968c700 1 -- 172.21.15.100:0/3678506164 wait complete. 2024-05-10T11:19:34.821 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 27 2024-05-10T11:19:35.614 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-10T11:19:35.885 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:19:36.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:19:35 smithi178 ceph-mon[33359]: pgmap v456: 129 pgs: 129 active+clean; 664 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-10T11:19:36.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:19:35 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/3678506164' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:19:36.013 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:19:35 smithi100 ceph-mon[30296]: pgmap v456: 129 pgs: 129 active+clean; 664 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-10T11:19:36.013 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:19:35 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/3678506164' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:19:36.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:19:35 smithi188 ceph-mon[34075]: pgmap v456: 129 pgs: 129 active+clean; 664 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-10T11:19:36.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:19:35 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/3678506164' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:19:37.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.685+0000 7f58a969b700 1 -- 172.21.15.100:0/2599887830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58a4109540 msgr2=0x7f58a4109920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:37.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.685+0000 7f58a969b700 1 --2- 172.21.15.100:0/2599887830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58a4109540 0x7f58a4109920 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f5894009990 tx=0x7f5894009ca0 comp rx=0 tx=0).stop 2024-05-10T11:19:37.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.685+0000 7f58a969b700 1 -- 172.21.15.100:0/2599887830 shutdown_connections 2024-05-10T11:19:37.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.685+0000 7f58a969b700 1 --2- 172.21.15.100:0/2599887830 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58a406c530 0x7f58a4110360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:37.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.685+0000 7f58a969b700 1 --2- 172.21.15.100:0/2599887830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58a4109540 0x7f58a4109920 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:37.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.685+0000 7f58a969b700 1 --2- 172.21.15.100:0/2599887830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58a406bb70 0x7f58a406bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:37.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.685+0000 7f58a969b700 1 -- 172.21.15.100:0/2599887830 >> 172.21.15.100:0/2599887830 conn(0x7f58a40793e0 msgr2=0x7f58a40797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:37.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.685+0000 7f58a969b700 1 -- 172.21.15.100:0/2599887830 shutdown_connections 2024-05-10T11:19:37.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.685+0000 7f58a969b700 1 -- 172.21.15.100:0/2599887830 wait complete. 2024-05-10T11:19:37.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.686+0000 7f58a969b700 1 Processor -- start 2024-05-10T11:19:37.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.686+0000 7f58a969b700 1 -- start start 2024-05-10T11:19:37.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.687+0000 7f58a969b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58a406bb70 0x7f58a419f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:37.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.687+0000 7f58a969b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58a406c530 0x7f58a419fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:37.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.687+0000 7f58a969b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58a4109540 0x7f58a41998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:37.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.687+0000 7f58a969b700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f58a4113550 con 0x7f58a406bb70 2024-05-10T11:19:37.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.687+0000 7f58a969b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f58a41133d0 con 0x7f58a406c530 2024-05-10T11:19:37.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.687+0000 7f58a969b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f58a41136d0 con 0x7f58a4109540 2024-05-10T11:19:37.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.687+0000 7f58a2ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58a406bb70 0x7f58a419f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:37.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.687+0000 7f58a2ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58a406bb70 0x7f58a419f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60450/0 (socket says 172.21.15.100:60450) 2024-05-10T11:19:37.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.687+0000 7f58a2ffd700 1 -- 172.21.15.100:0/2461955218 learned_addr learned my addr 172.21.15.100:0/2461955218 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:37.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.687+0000 7f58a37fe700 1 --2- 172.21.15.100:0/2461955218 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58a4109540 0x7f58a41998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:37.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.688+0000 7f58a2ffd700 1 -- 172.21.15.100:0/2461955218 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58a4109540 msgr2=0x7f58a41998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:37.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.688+0000 7f58a2ffd700 1 --2- 172.21.15.100:0/2461955218 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58a4109540 0x7f58a41998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:37.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.688+0000 7f58a2ffd700 1 -- 172.21.15.100:0/2461955218 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58a406c530 msgr2=0x7f58a419fc70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T11:19:37.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.688+0000 7f58a2ffd700 1 --2- 172.21.15.100:0/2461955218 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58a406c530 0x7f58a419fc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:37.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.688+0000 7f58a2ffd700 1 -- 172.21.15.100:0/2461955218 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58a419a050 con 0x7f58a406bb70 2024-05-10T11:19:37.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.688+0000 7f58a2ffd700 1 --2- 172.21.15.100:0/2461955218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58a406bb70 0x7f58a419f730 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f5894005910 tx=0x7f58940047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:37.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.689+0000 7f588bfff700 1 -- 172.21.15.100:0/2461955218 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5894004830 con 0x7f58a406bb70 2024-05-10T11:19:37.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.689+0000 7f588bfff700 1 -- 172.21.15.100:0/2461955218 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5894005020 con 0x7f58a406bb70 2024-05-10T11:19:37.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.689+0000 7f588bfff700 1 -- 172.21.15.100:0/2461955218 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589400f8d0 con 0x7f58a406bb70 2024-05-10T11:19:37.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.689+0000 7f58a969b700 1 -- 172.21.15.100:0/2461955218 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58a419a2e0 con 0x7f58a406bb70 2024-05-10T11:19:37.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.689+0000 7f58a969b700 1 -- 172.21.15.100:0/2461955218 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58a41a6600 con 0x7f58a406bb70 2024-05-10T11:19:37.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.693+0000 7f588bfff700 1 -- 172.21.15.100:0/2461955218 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f58940054f0 con 0x7f58a406bb70 2024-05-10T11:19:37.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.694+0000 7f588bfff700 1 --2- 172.21.15.100:0/2461955218 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f588c0b2a50 0x7f588c0b4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:37.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.695+0000 7f588bfff700 1 -- 172.21.15.100:0/2461955218 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f58940e3330 con 0x7f58a406bb70 2024-05-10T11:19:37.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.695+0000 7f58b3f2eb80 1 Processor -- start 2024-05-10T11:19:37.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.695+0000 7f58a27fc700 1 --2- 172.21.15.100:0/2461955218 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f588c0b2a50 0x7f588c0b4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:37.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.695+0000 7f58b3f2eb80 1 -- start start 2024-05-10T11:19:37.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.695+0000 7f58b3f2eb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a641371b0 0x559a64137590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:37.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.696+0000 7f58a27fc700 1 --2- 172.21.15.100:0/2461955218 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f588c0b2a50 0x7f588c0b4f10 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f58a4079b20 tx=0x7f589800b3f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:37.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.696+0000 7f58a2ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a641371b0 0x559a64137590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:37.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.696+0000 7f58b3f2eb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a64137ad0 0x559a64131810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:37.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.696+0000 7f58b3f2eb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a64131d50 0x559a64132190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:37.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.696+0000 7f58b3f2eb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x559a64083940 con 0x559a641371b0 2024-05-10T11:19:37.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.696+0000 7f58b3f2eb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559a63fe8da0 con 0x559a64137ad0 2024-05-10T11:19:37.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.696+0000 7f58a2ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a641371b0 0x559a64137590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60462/0 (socket says 172.21.15.100:60462) 2024-05-10T11:19:37.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.696+0000 7f58a2ffd700 1 -- 172.21.15.100:0/1748978497 learned_addr learned my addr 172.21.15.100:0/1748978497 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:37.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.696+0000 7f58b3f2eb80 1 -- 172.21.15.100:0/1748978497 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x559a6407f900 con 0x559a64131d50 2024-05-10T11:19:37.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.696+0000 7f58a37fe700 1 --2- 172.21.15.100:0/1748978497 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a64131d50 0x559a64132190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:37.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.696+0000 7f58a2ffd700 1 -- 172.21.15.100:0/1748978497 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a64131d50 msgr2=0x559a64132190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:37.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.696+0000 7f58a2ffd700 1 --2- 172.21.15.100:0/1748978497 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a64131d50 0x559a64132190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:37.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.696+0000 7f58a2ffd700 1 -- 172.21.15.100:0/1748978497 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a64137ad0 msgr2=0x559a64131810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:37.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.696+0000 7f58a2ffd700 1 --2- 172.21.15.100:0/1748978497 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a64137ad0 0x559a64131810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:37.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.697+0000 7f58a2ffd700 1 -- 172.21.15.100:0/1748978497 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a63fd08d0 con 0x559a641371b0 2024-05-10T11:19:37.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.697+0000 7f58a2ffd700 1 --2- 172.21.15.100:0/1748978497 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a641371b0 0x559a64137590 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f58940a66a0 tx=0x7f5894009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8c695dde81aa67c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:37.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.697+0000 7f58897fa700 1 -- 172.21.15.100:0/1748978497 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58940aa620 con 0x559a641371b0 2024-05-10T11:19:37.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.697+0000 7f58897fa700 1 -- 172.21.15.100:0/1748978497 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f58940ae9d0 con 0x559a641371b0 2024-05-10T11:19:37.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.697+0000 7f58897fa700 1 -- 172.21.15.100:0/1748978497 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58940b31c0 con 0x559a641371b0 2024-05-10T11:19:37.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.698+0000 7f58b3f2eb80 1 -- 172.21.15.100:0/1748978497 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a641371b0 msgr2=0x559a64137590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:37.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.698+0000 7f58b3f2eb80 1 --2- 172.21.15.100:0/1748978497 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a641371b0 0x559a64137590 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f58940a66a0 tx=0x7f5894009ca0 comp rx=0 tx=0).stop 2024-05-10T11:19:37.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.698+0000 7f58b3f2eb80 1 -- 172.21.15.100:0/1748978497 shutdown_connections 2024-05-10T11:19:37.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.698+0000 7f58b3f2eb80 1 --2- 172.21.15.100:0/1748978497 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a64131d50 0x559a64132190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:37.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.698+0000 7f58b3f2eb80 1 --2- 172.21.15.100:0/1748978497 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a641371b0 0x559a64137590 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:37.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.698+0000 7f58b3f2eb80 1 --2- 172.21.15.100:0/1748978497 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a64137ad0 0x559a64131810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:37.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.698+0000 7f58b3f2eb80 1 -- 172.21.15.100:0/1748978497 >> 172.21.15.100:0/1748978497 conn(0x559a6405db70 msgr2=0x559a6412c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:37.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.698+0000 7f58b3f2eb80 1 -- 172.21.15.100:0/1748978497 shutdown_connections 2024-05-10T11:19:37.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.699+0000 7f58b3f2eb80 1 -- 172.21.15.100:0/1748978497 wait complete. 2024-05-10T11:19:37.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.699+0000 7f58b3f2eb80 1 -- start start 2024-05-10T11:19:37.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.699+0000 7f58b3f2eb80 1 Processor -- start 2024-05-10T11:19:37.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.699+0000 7f58b3f2eb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a64131d50 0x559a6413d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:37.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.699+0000 7f58b3f2eb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a641371b0 0x559a6413ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:37.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.699+0000 7f58b3f2eb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a64137ad0 0x559a6413b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:37.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.699+0000 7f58b3f2eb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x559a6413e210 con 0x559a64137ad0 2024-05-10T11:19:37.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.699+0000 7f58b3f2eb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559a6403ced0 con 0x559a641371b0 2024-05-10T11:19:37.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.699+0000 7f58b3f2eb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x559a6413e390 con 0x559a64131d50 2024-05-10T11:19:37.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.700+0000 7f58b3f2eb80 10 client.0 _ll_register_callbacks cb 0x559a640175b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:19:37.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.700+0000 7f58a37fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a64137ad0 0x559a6413b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:37.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.700+0000 7f58a2ffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a64131d50 0x559a6413d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:37.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.700+0000 7f58a2ffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a64131d50 0x559a6413d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:38930/0 (socket says 172.21.15.100:38930) 2024-05-10T11:19:37.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.700+0000 7f58a37fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a64137ad0 0x559a6413b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60476/0 (socket says 172.21.15.100:60476) 2024-05-10T11:19:37.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.700+0000 7f58a2ffd700 1 -- 172.21.15.100:0/3159064387 learned_addr learned my addr 172.21.15.100:0/3159064387 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:37.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.700+0000 7f58a27fc700 1 --2- 172.21.15.100:0/3159064387 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a641371b0 0x559a6413ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:37.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.700+0000 7f58a2ffd700 1 -- 172.21.15.100:0/3159064387 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a641371b0 msgr2=0x559a6413ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:37.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.700+0000 7f58a2ffd700 1 --2- 172.21.15.100:0/3159064387 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a641371b0 0x559a6413ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:37.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.700+0000 7f58a2ffd700 1 -- 172.21.15.100:0/3159064387 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a64137ad0 msgr2=0x559a6413b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:37.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.700+0000 7f58a2ffd700 1 --2- 172.21.15.100:0/3159064387 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a64137ad0 0x559a6413b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:37.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.700+0000 7f58a2ffd700 1 -- 172.21.15.100:0/3159064387 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a6403a760 con 0x559a64131d50 2024-05-10T11:19:37.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.700+0000 7f58a37fe700 1 --2- 172.21.15.100:0/3159064387 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a64137ad0 0x559a6413b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:19:37.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.701+0000 7f58a2ffd700 1 --2- 172.21.15.100:0/3159064387 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a64131d50 0x559a6413d790 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f58940aabb0 tx=0x7f5894005c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=620b1562832742c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:37.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.701+0000 7f5886ff5700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:19:37.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.701+0000 7f5886ff5700 1 -- 172.21.15.100:0/3159064387 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5894090b90 con 0x559a64131d50 2024-05-10T11:19:37.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.701+0000 7f58b3f2eb80 1 -- 172.21.15.100:0/3159064387 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58940ae9d0 con 0x559a64131d50 2024-05-10T11:19:37.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.701+0000 7f5886ff5700 1 -- 172.21.15.100:0/3159064387 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f5894090d30 con 0x559a64131d50 2024-05-10T11:19:37.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.701+0000 7f5886ff5700 1 -- 172.21.15.100:0/3159064387 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f58940cb3f0 con 0x559a64131d50 2024-05-10T11:19:37.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.701+0000 7f58b3f2eb80 10 client.25057 fetch_fsmap learned FSMap version 27 2024-05-10T11:19:37.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.701+0000 7f58b3f2eb80 1 -- 172.21.15.100:0/3159064387 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559a6413e210 con 0x559a64131d50 2024-05-10T11:19:37.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.701+0000 7f5886ff5700 1 -- 172.21.15.100:0/3159064387 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f58940ae9d0 con 0x559a64131d50 2024-05-10T11:19:37.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.702+0000 7f5886ff5700 1 -- 172.21.15.100:0/3159064387 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f58940a9880 con 0x559a64131d50 2024-05-10T11:19:37.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.702+0000 7f58b3f2eb80 10 client.25057 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T11:19:37.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.702+0000 7f58b3f2eb80 10 client.25057 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-10T11:19:37.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.702+0000 7f58b3f2eb80 20 client.25057 populate_metadata read hostname 'smithi100' 2024-05-10T11:19:37.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.702+0000 7f58b3f2eb80 1 --2- 172.21.15.100:0/3159064387 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x559a6414ad00 0x559a6414d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:37.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.702+0000 7f58b3f2eb80 4 client.25057 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:19:37.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.702+0000 7f58b3f2eb80 1 -- 172.21.15.100:0/3159064387 --> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a64091320 con 0x559a6414ad00 2024-05-10T11:19:37.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.703+0000 7f58a27fc700 1 --2- 172.21.15.100:0/3159064387 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x559a6414ad00 0x559a6414d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:37.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.703+0000 7f58a27fc700 1 --2- 172.21.15.100:0/3159064387 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x559a6414ad00 0x559a6414d110 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8840ebbf1b693984 server_cookie=31082080d4fc1de2 in_seq=0 out_seq=0 2024-05-10T11:19:37.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.703+0000 7f5886ff5700 10 client.25057 ms_handle_connect on v2:172.21.15.188:6836/1943342620 2024-05-10T11:19:37.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.704+0000 7f5886ff5700 1 -- 172.21.15.100:0/3159064387 <== mds.2 v2:172.21.15.188:6836/1943342620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559a64091320 con 0x559a6414ad00 2024-05-10T11:19:37.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.704+0000 7f5886ff5700 10 client.25057 handle_command_reply: tid=0 2024-05-10T11:19:37.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.704+0000 7f58b3f2eb80 1 client.25057 shutdown 2024-05-10T11:19:37.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.704+0000 7f58b3f2eb80 1 -- 172.21.15.100:0/3159064387 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a64131d50 msgr2=0x559a6413d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:37.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.704+0000 7f58b3f2eb80 1 --2- 172.21.15.100:0/3159064387 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a64131d50 0x559a6413d790 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f58940aabb0 tx=0x7f5894005c40 comp rx=0 tx=0).stop 2024-05-10T11:19:37.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.704+0000 7f58b3f2eb80 1 -- 172.21.15.100:0/3159064387 shutdown_connections 2024-05-10T11:19:37.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.704+0000 7f58b3f2eb80 1 --2- 172.21.15.100:0/3159064387 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x559a6414ad00 0x559a6414d110 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:37.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.704+0000 7f58b3f2eb80 1 --2- 172.21.15.100:0/3159064387 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a64137ad0 0x559a6413b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:37.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.704+0000 7f58b3f2eb80 1 --2- 172.21.15.100:0/3159064387 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a64131d50 0x559a6413d790 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:37.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.704+0000 7f58b3f2eb80 1 --2- 172.21.15.100:0/3159064387 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a641371b0 0x559a6413ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:37.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.704+0000 7f5886ff5700 0 client.25057 ms_handle_reset on v2:172.21.15.188:6836/1943342620 2024-05-10T11:19:37.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.704+0000 7f58b3f2eb80 1 -- 172.21.15.100:0/3159064387 >> 172.21.15.100:0/3159064387 conn(0x559a6405db70 msgr2=0x559a64126b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:37.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.704+0000 7f58b3f2eb80 1 -- 172.21.15.100:0/3159064387 shutdown_connections 2024-05-10T11:19:37.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.704+0000 7f58b3f2eb80 1 -- 172.21.15.100:0/3159064387 wait complete. 2024-05-10T11:19:37.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.705+0000 7f58b3f2eb80 20 client.25057 trim_cache size 0 max 16384 2024-05-10T11:19:37.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.734+0000 7f58b3f2eb80 1 Processor -- start 2024-05-10T11:19:37.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.734+0000 7f58b3f2eb80 1 -- start start 2024-05-10T11:19:37.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.734+0000 7f58b3f2eb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a64131d50 0x559a64131810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:37.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.734+0000 7f58b3f2eb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a641371b0 0x559a641471d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:37.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.734+0000 7f58b3f2eb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a64137ad0 0x559a64126c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:37.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.734+0000 7f58b3f2eb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f58a41af600 con 0x559a64131d50 2024-05-10T11:19:37.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.734+0000 7f58b3f2eb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f58a41ace90 con 0x559a64137ad0 2024-05-10T11:19:37.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.734+0000 7f58b3f2eb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f58a41aace0 con 0x559a641371b0 2024-05-10T11:19:37.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.734+0000 7f58a2ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a64131d50 0x559a64131810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:37.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.734+0000 7f58a2ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a64131d50 0x559a64131810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60484/0 (socket says 172.21.15.100:60484) 2024-05-10T11:19:37.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.734+0000 7f58a2ffd700 1 -- 172.21.15.100:0/4190962190 learned_addr learned my addr 172.21.15.100:0/4190962190 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:37.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.735+0000 7f58a37fe700 1 --2- 172.21.15.100:0/4190962190 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a64137ad0 0x559a64126c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:37.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.735+0000 7f58a27fc700 1 --2- 172.21.15.100:0/4190962190 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a641371b0 0x559a641471d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:37.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.735+0000 7f58a2ffd700 1 -- 172.21.15.100:0/4190962190 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a641371b0 msgr2=0x559a641471d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:37.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.735+0000 7f58a2ffd700 1 --2- 172.21.15.100:0/4190962190 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a641371b0 0x559a641471d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:37.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.735+0000 7f58a2ffd700 1 -- 172.21.15.100:0/4190962190 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a64137ad0 msgr2=0x559a64126c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:37.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.735+0000 7f58a2ffd700 1 --2- 172.21.15.100:0/4190962190 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a64137ad0 0x559a64126c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:37.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.735+0000 7f58a2ffd700 1 -- 172.21.15.100:0/4190962190 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a64128c80 con 0x559a64131d50 2024-05-10T11:19:37.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.735+0000 7f58a2ffd700 1 --2- 172.21.15.100:0/4190962190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a64131d50 0x559a64131810 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f58940aa320 tx=0x7f5894009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ab665ac2ec21d6c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:37.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.735+0000 7f5888ff9700 1 -- 172.21.15.100:0/4190962190 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58940ae9d0 con 0x559a64131d50 2024-05-10T11:19:37.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.736+0000 7f5888ff9700 1 -- 172.21.15.100:0/4190962190 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f58940a6280 con 0x559a64131d50 2024-05-10T11:19:37.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.736+0000 7f5888ff9700 1 -- 172.21.15.100:0/4190962190 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58940a9aa0 con 0x559a64131d50 2024-05-10T11:19:37.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.736+0000 7f58b3f2eb80 1 -- 172.21.15.100:0/4190962190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a64131d50 msgr2=0x559a64131810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:37.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.736+0000 7f58b3f2eb80 1 --2- 172.21.15.100:0/4190962190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a64131d50 0x559a64131810 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f58940aa320 tx=0x7f5894009ca0 comp rx=0 tx=0).stop 2024-05-10T11:19:37.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.737+0000 7f58b3f2eb80 1 -- 172.21.15.100:0/4190962190 shutdown_connections 2024-05-10T11:19:37.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.737+0000 7f58b3f2eb80 1 --2- 172.21.15.100:0/4190962190 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a64137ad0 0x559a64126c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:37.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.737+0000 7f58b3f2eb80 1 --2- 172.21.15.100:0/4190962190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a64131d50 0x559a64131810 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:37.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.737+0000 7f58b3f2eb80 1 --2- 172.21.15.100:0/4190962190 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a641371b0 0x559a641471d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:37.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.737+0000 7f58b3f2eb80 1 -- 172.21.15.100:0/4190962190 >> 172.21.15.100:0/4190962190 conn(0x559a6405db70 msgr2=0x559a641364f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:37.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.737+0000 7f58b3f2eb80 1 -- 172.21.15.100:0/4190962190 shutdown_connections 2024-05-10T11:19:37.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.737+0000 7f58b3f2eb80 1 -- 172.21.15.100:0/4190962190 wait complete. 2024-05-10T11:19:37.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.737+0000 7f58b3f2eb80 1 -- start start 2024-05-10T11:19:37.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.737+0000 7f58b3f2eb80 1 Processor -- start 2024-05-10T11:19:37.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.738+0000 7f58b3f2eb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a64131d50 0x559a6413e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:37.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.738+0000 7f58b3f2eb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a641371b0 0x559a6413ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:37.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.738+0000 7f58b3f2eb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a64137ad0 0x559a641bbde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:37.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.738+0000 7f58b3f2eb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f58a41ab350 con 0x559a64131d50 2024-05-10T11:19:37.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.738+0000 7f58b3f2eb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f58a41ac7c0 con 0x559a64137ad0 2024-05-10T11:19:37.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.738+0000 7f58b3f2eb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f58a41aba20 con 0x559a641371b0 2024-05-10T11:19:37.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.738+0000 7f58b3f2eb80 10 client.0 _ll_register_callbacks cb 0x559a640175b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:19:37.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.738+0000 7f58a2ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a64131d50 0x559a6413e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:37.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.738+0000 7f58a2ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a64131d50 0x559a6413e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60492/0 (socket says 172.21.15.100:60492) 2024-05-10T11:19:37.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.738+0000 7f58a27fc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a641371b0 0x559a6413ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:37.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.738+0000 7f58a2ffd700 1 -- 172.21.15.100:0/1846257789 learned_addr learned my addr 172.21.15.100:0/1846257789 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:37.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.738+0000 7f58a37fe700 1 --2- 172.21.15.100:0/1846257789 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a64137ad0 0x559a641bbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:37.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.738+0000 7f58a27fc700 1 -- 172.21.15.100:0/1846257789 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a64137ad0 msgr2=0x559a641bbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:37.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.738+0000 7f58a27fc700 1 --2- 172.21.15.100:0/1846257789 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a64137ad0 0x559a641bbde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:37.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.738+0000 7f58a27fc700 1 -- 172.21.15.100:0/1846257789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a64131d50 msgr2=0x559a6413e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:37.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.738+0000 7f58a27fc700 1 --2- 172.21.15.100:0/1846257789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a64131d50 0x559a6413e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:37.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.738+0000 7f58a27fc700 1 -- 172.21.15.100:0/1846257789 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a641bc320 con 0x559a641371b0 2024-05-10T11:19:37.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.738+0000 7f58a2ffd700 1 --2- 172.21.15.100:0/1846257789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a64131d50 0x559a6413e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:19:37.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.739+0000 7f58a27fc700 1 --2- 172.21.15.100:0/1846257789 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a641371b0 0x559a6413ec60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x559a6412e450 tx=0x7f58980047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5e0506e3c0347ba server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:37.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.739+0000 7f5886ff5700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:19:37.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.739+0000 7f5886ff5700 1 -- 172.21.15.100:0/1846257789 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589800de80 con 0x559a641371b0 2024-05-10T11:19:37.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.739+0000 7f5886ff5700 1 -- 172.21.15.100:0/1846257789 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f5898010cb0 con 0x559a641371b0 2024-05-10T11:19:37.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.739+0000 7f58b3f2eb80 1 -- 172.21.15.100:0/1846257789 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58940a6280 con 0x559a641371b0 2024-05-10T11:19:37.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.739+0000 7f5886ff5700 1 -- 172.21.15.100:0/1846257789 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5898034050 con 0x559a641371b0 2024-05-10T11:19:37.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.739+0000 7f58b3f2eb80 10 client.25063 fetch_fsmap learned FSMap version 27 2024-05-10T11:19:37.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.739+0000 7f58b3f2eb80 1 -- 172.21.15.100:0/1846257789 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559a641bd590 con 0x559a641371b0 2024-05-10T11:19:37.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.739+0000 7f5886ff5700 1 -- 172.21.15.100:0/1846257789 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5898016040 con 0x559a641371b0 2024-05-10T11:19:37.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.740+0000 7f5886ff5700 1 -- 172.21.15.100:0/1846257789 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f5898025ce0 con 0x559a641371b0 2024-05-10T11:19:37.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.740+0000 7f58b3f2eb80 10 client.25063 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T11:19:37.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.740+0000 7f58b3f2eb80 10 client.25063 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-10T11:19:37.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.740+0000 7f58b3f2eb80 20 client.25063 populate_metadata read hostname 'smithi100' 2024-05-10T11:19:37.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.740+0000 7f58b3f2eb80 1 --2- 172.21.15.100:0/1846257789 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x559a6414ad00 0x559a641bfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:37.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.740+0000 7f58b3f2eb80 4 client.25063 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:19:37.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.740+0000 7f58b3f2eb80 1 -- 172.21.15.100:0/1846257789 --> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f58940a6280 con 0x559a6414ad00 2024-05-10T11:19:37.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.741+0000 7f58a2ffd700 1 --2- 172.21.15.100:0/1846257789 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x559a6414ad00 0x559a641bfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:37.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.741+0000 7f5886ff5700 10 client.25063 ms_handle_connect on v2:172.21.15.188:6836/1943342620 2024-05-10T11:19:37.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.741+0000 7f58a2ffd700 1 --2- 172.21.15.100:0/1846257789 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x559a6414ad00 0x559a641bfca0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6716049810b8cdec server_cookie=67b55a6547af82d1 in_seq=0 out_seq=0 2024-05-10T11:19:37.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.741+0000 7f5886ff5700 1 -- 172.21.15.100:0/1846257789 <== mds.2 v2:172.21.15.188:6836/1943342620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f58940a6280 con 0x559a6414ad00 2024-05-10T11:19:37.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.741+0000 7f5886ff5700 10 client.25063 handle_command_reply: tid=0 2024-05-10T11:19:37.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.741+0000 7f58b3f2eb80 1 client.25063 shutdown 2024-05-10T11:19:37.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.741+0000 7f58b3f2eb80 1 -- 172.21.15.100:0/1846257789 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a641371b0 msgr2=0x559a6413ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:37.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.741+0000 7f58b3f2eb80 1 --2- 172.21.15.100:0/1846257789 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a641371b0 0x559a6413ec60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x559a6412e450 tx=0x7f58980047c0 comp rx=0 tx=0).stop 2024-05-10T11:19:37.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.742+0000 7f58b3f2eb80 1 -- 172.21.15.100:0/1846257789 shutdown_connections 2024-05-10T11:19:37.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.742+0000 7f58b3f2eb80 1 --2- 172.21.15.100:0/1846257789 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x559a6414ad00 0x559a641bfca0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:37.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.742+0000 7f5886ff5700 0 client.25063 ms_handle_reset on v2:172.21.15.188:6836/1943342620 2024-05-10T11:19:37.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.742+0000 7f58b3f2eb80 1 --2- 172.21.15.100:0/1846257789 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a64137ad0 0x559a641bbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:37.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.742+0000 7f58b3f2eb80 1 --2- 172.21.15.100:0/1846257789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a64131d50 0x559a6413e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:37.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.742+0000 7f58b3f2eb80 1 --2- 172.21.15.100:0/1846257789 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a641371b0 0x559a6413ec60 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:37.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.742+0000 7f58b3f2eb80 1 -- 172.21.15.100:0/1846257789 >> 172.21.15.100:0/1846257789 conn(0x559a6405db70 msgr2=0x559a64130800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:37.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.742+0000 7f58b3f2eb80 1 -- 172.21.15.100:0/1846257789 shutdown_connections 2024-05-10T11:19:37.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.742+0000 7f58b3f2eb80 1 -- 172.21.15.100:0/1846257789 wait complete. 2024-05-10T11:19:37.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.742+0000 7f58b3f2eb80 20 client.25063 trim_cache size 0 max 16384 2024-05-10T11:19:37.755 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:19:37.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.743+0000 7f58a969b700 1 -- 172.21.15.100:0/2461955218 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f588c0b2a50 msgr2=0x7f588c0b4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:37.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.743+0000 7f58a969b700 1 --2- 172.21.15.100:0/2461955218 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f588c0b2a50 0x7f588c0b4f10 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f58a4079b20 tx=0x7f589800b3f0 comp rx=0 tx=0).stop 2024-05-10T11:19:37.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.743+0000 7f58a969b700 1 -- 172.21.15.100:0/2461955218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58a406bb70 msgr2=0x7f58a419f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:37.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.743+0000 7f58a969b700 1 --2- 172.21.15.100:0/2461955218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58a406bb70 0x7f58a419f730 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f5894005910 tx=0x7f58940047b0 comp rx=0 tx=0).stop 2024-05-10T11:19:37.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.743+0000 7f58a969b700 1 -- 172.21.15.100:0/2461955218 shutdown_connections 2024-05-10T11:19:37.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.743+0000 7f58a969b700 1 --2- 172.21.15.100:0/2461955218 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f588c0b2a50 0x7f588c0b4f10 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:37.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.743+0000 7f58a969b700 1 --2- 172.21.15.100:0/2461955218 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58a4109540 0x7f58a41998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:37.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.743+0000 7f58a969b700 1 --2- 172.21.15.100:0/2461955218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58a406bb70 0x7f58a419f730 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:37.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.743+0000 7f58a969b700 1 --2- 172.21.15.100:0/2461955218 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58a406c530 0x7f58a419fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:37.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.743+0000 7f58a969b700 1 -- 172.21.15.100:0/2461955218 >> 172.21.15.100:0/2461955218 conn(0x7f58a40793e0 msgr2=0x7f58a4101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:37.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.744+0000 7f58a969b700 1 -- 172.21.15.100:0/2461955218 shutdown_connections 2024-05-10T11:19:37.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:37.744+0000 7f58a969b700 1 -- 172.21.15.100:0/2461955218 wait complete. 2024-05-10T11:19:38.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:19:37 smithi178 ceph-mon[33359]: pgmap v457: 129 pgs: 129 active+clean; 664 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-10T11:19:38.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:19:37 smithi188 ceph-mon[34075]: pgmap v457: 129 pgs: 129 active+clean; 664 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-10T11:19:38.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:19:37 smithi100 ceph-mon[30296]: pgmap v457: 129 pgs: 129 active+clean; 664 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-10T11:19:38.568 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-10T11:19:38.843 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:19:39.728 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:19:39 smithi100 ceph-mon[30296]: pgmap v458: 129 pgs: 129 active+clean; 664 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-10T11:19:40.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:19:39 smithi178 ceph-mon[33359]: pgmap v458: 129 pgs: 129 active+clean; 664 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-10T11:19:40.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:19:39 smithi188 ceph-mon[34075]: pgmap v458: 129 pgs: 129 active+clean; 664 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-10T11:19:40.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.684+0000 7ffabb273700 1 -- 172.21.15.100:0/372872264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffab4102d50 msgr2=0x7ffab410f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:40.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.684+0000 7ffabb273700 1 --2- 172.21.15.100:0/372872264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffab4102d50 0x7ffab410f240 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ffaac00b0a0 tx=0x7ffaac00b3b0 comp rx=0 tx=0).stop 2024-05-10T11:19:40.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.685+0000 7ffabb273700 1 -- 172.21.15.100:0/372872264 shutdown_connections 2024-05-10T11:19:40.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.685+0000 7ffabb273700 1 --2- 172.21.15.100:0/372872264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffab4102d50 0x7ffab410f240 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:40.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.685+0000 7ffabb273700 1 --2- 172.21.15.100:0/372872264 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffab4108400 0x7ffab41087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:40.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.685+0000 7ffabb273700 1 --2- 172.21.15.100:0/372872264 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffab4102390 0x7ffab4102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:40.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.685+0000 7ffabb273700 1 -- 172.21.15.100:0/372872264 >> 172.21.15.100:0/372872264 conn(0x7ffab4079310 msgr2=0x7ffab4079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:40.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.686+0000 7ffabb273700 1 -- 172.21.15.100:0/372872264 shutdown_connections 2024-05-10T11:19:40.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.686+0000 7ffabb273700 1 -- 172.21.15.100:0/372872264 wait complete. 2024-05-10T11:19:40.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.687+0000 7ffabb273700 1 Processor -- start 2024-05-10T11:19:40.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.687+0000 7ffabb273700 1 -- start start 2024-05-10T11:19:40.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.687+0000 7ffabb273700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffab4102390 0x7ffab410fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:40.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.687+0000 7ffabb273700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffab4102d50 0x7ffab4110210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:40.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.687+0000 7ffabb273700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffab4108400 0x7ffab4110750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:40.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.687+0000 7ffabb273700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ffab4113520 con 0x7ffab4102390 2024-05-10T11:19:40.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.687+0000 7ffabb273700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ffab41133a0 con 0x7ffab4102d50 2024-05-10T11:19:40.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.687+0000 7ffabb273700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ffab41136a0 con 0x7ffab4108400 2024-05-10T11:19:40.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.687+0000 7ffab900f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffab4102390 0x7ffab410fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:40.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.687+0000 7ffab900f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffab4102390 0x7ffab410fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60512/0 (socket says 172.21.15.100:60512) 2024-05-10T11:19:40.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.687+0000 7ffab900f700 1 -- 172.21.15.100:0/1334141972 learned_addr learned my addr 172.21.15.100:0/1334141972 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:40.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.687+0000 7ffab880e700 1 --2- 172.21.15.100:0/1334141972 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffab4102d50 0x7ffab4110210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:40.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.688+0000 7ffab900f700 1 -- 172.21.15.100:0/1334141972 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffab4108400 msgr2=0x7ffab4110750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:40.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.688+0000 7ffab900f700 1 --2- 172.21.15.100:0/1334141972 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffab4108400 0x7ffab4110750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:40.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.688+0000 7ffab900f700 1 -- 172.21.15.100:0/1334141972 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffab4102d50 msgr2=0x7ffab4110210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:40.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.688+0000 7ffab900f700 1 --2- 172.21.15.100:0/1334141972 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffab4102d50 0x7ffab4110210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:40.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.688+0000 7ffab900f700 1 -- 172.21.15.100:0/1334141972 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffab41ac420 con 0x7ffab4102390 2024-05-10T11:19:40.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.688+0000 7ffab900f700 1 --2- 172.21.15.100:0/1334141972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffab4102390 0x7ffab410fcd0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7ffab000d950 tx=0x7ffab000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:40.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.689+0000 7ffaaa7fc700 1 -- 172.21.15.100:0/1334141972 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab0016070 con 0x7ffab4102390 2024-05-10T11:19:40.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.689+0000 7ffabb273700 1 -- 172.21.15.100:0/1334141972 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffab41ac6b0 con 0x7ffab4102390 2024-05-10T11:19:40.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.689+0000 7ffabb273700 1 -- 172.21.15.100:0/1334141972 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffab41acc40 con 0x7ffab4102390 2024-05-10T11:19:40.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.689+0000 7ffaaa7fc700 1 -- 172.21.15.100:0/1334141972 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffab0009dd0 con 0x7ffab4102390 2024-05-10T11:19:40.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.689+0000 7ffaaa7fc700 1 -- 172.21.15.100:0/1334141972 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab0020060 con 0x7ffab4102390 2024-05-10T11:19:40.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.693+0000 7ffaaa7fc700 1 -- 172.21.15.100:0/1334141972 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7ffab0019070 con 0x7ffab4102390 2024-05-10T11:19:40.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.693+0000 7ffac5b06b80 1 Processor -- start 2024-05-10T11:19:40.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.694+0000 7ffac5b06b80 1 -- start start 2024-05-10T11:19:40.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.694+0000 7ffac5b06b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56307382b4d0 0x56307382b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:40.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.694+0000 7ffac5b06b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56307382bdf0 0x56307382f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:40.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.694+0000 7ffac5b06b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56307382fc80 0x563073830100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:40.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.694+0000 7ffac5b06b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56307377d860 con 0x56307382bdf0 2024-05-10T11:19:40.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.694+0000 7ffac5b06b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56307371c7c0 con 0x56307382fc80 2024-05-10T11:19:40.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.694+0000 7ffac5b06b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563073779820 con 0x56307382b4d0 2024-05-10T11:19:40.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.694+0000 7ffab900f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56307382bdf0 0x56307382f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:40.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.694+0000 7ffab900f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56307382bdf0 0x56307382f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60516/0 (socket says 172.21.15.100:60516) 2024-05-10T11:19:40.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.694+0000 7ffab900f700 1 -- 172.21.15.100:0/621666603 learned_addr learned my addr 172.21.15.100:0/621666603 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:40.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.694+0000 7ffab880e700 1 --2- 172.21.15.100:0/621666603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56307382fc80 0x563073830100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:40.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.695+0000 7ffab9810700 1 --2- 172.21.15.100:0/621666603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56307382b4d0 0x56307382b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:40.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.695+0000 7ffaaa7fc700 1 --2- 172.21.15.100:0/1334141972 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ffaa00b2880 0x7ffaa00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:40.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.695+0000 7ffab900f700 1 -- 172.21.15.100:0/621666603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56307382b4d0 msgr2=0x56307382b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:40.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.695+0000 7ffab900f700 1 --2- 172.21.15.100:0/621666603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56307382b4d0 0x56307382b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:40.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.695+0000 7ffab900f700 1 -- 172.21.15.100:0/621666603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56307382fc80 msgr2=0x563073830100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:40.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.695+0000 7ffab900f700 1 --2- 172.21.15.100:0/621666603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56307382fc80 0x563073830100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:40.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.695+0000 7ffab900f700 1 -- 172.21.15.100:0/621666603 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630736ca8d0 con 0x56307382bdf0 2024-05-10T11:19:40.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.695+0000 7ffaaa7fc700 1 -- 172.21.15.100:0/1334141972 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7ffab00e2190 con 0x7ffab4102390 2024-05-10T11:19:40.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.695+0000 7ffab900f700 1 --2- 172.21.15.100:0/621666603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56307382bdf0 0x56307382f6f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7ffab00df870 tx=0x7ffab001af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3564b888f98c2cab server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:40.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.695+0000 7ffa9f7fe700 1 -- 172.21.15.100:0/621666603 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab00a8da0 con 0x56307382bdf0 2024-05-10T11:19:40.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.695+0000 7ffa9f7fe700 1 -- 172.21.15.100:0/621666603 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffab004cac0 con 0x56307382bdf0 2024-05-10T11:19:40.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.695+0000 7ffab9810700 1 --2- 172.21.15.100:0/1334141972 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ffaa00b2880 0x7ffaa00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:40.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.696+0000 7ffa9f7fe700 1 -- 172.21.15.100:0/621666603 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab00e5d80 con 0x56307382bdf0 2024-05-10T11:19:40.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.696+0000 7ffac5b06b80 1 -- 172.21.15.100:0/621666603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56307382bdf0 msgr2=0x56307382f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:40.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.696+0000 7ffab9810700 1 --2- 172.21.15.100:0/1334141972 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ffaa00b2880 0x7ffaa00b4d40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ffaac006070 tx=0x7ffaac006000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:40.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.696+0000 7ffac5b06b80 1 --2- 172.21.15.100:0/621666603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56307382bdf0 0x56307382f6f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7ffab00df870 tx=0x7ffab001af90 comp rx=0 tx=0).stop 2024-05-10T11:19:40.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.697+0000 7ffac5b06b80 1 -- 172.21.15.100:0/621666603 shutdown_connections 2024-05-10T11:19:40.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.697+0000 7ffac5b06b80 1 --2- 172.21.15.100:0/621666603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56307382fc80 0x563073830100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:40.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.697+0000 7ffac5b06b80 1 --2- 172.21.15.100:0/621666603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56307382b4d0 0x56307382b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:40.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.697+0000 7ffac5b06b80 1 --2- 172.21.15.100:0/621666603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56307382bdf0 0x56307382f6f0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:40.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.697+0000 7ffac5b06b80 1 -- 172.21.15.100:0/621666603 >> 172.21.15.100:0/621666603 conn(0x563073757100 msgr2=0x563073825c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:40.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.697+0000 7ffac5b06b80 1 -- 172.21.15.100:0/621666603 shutdown_connections 2024-05-10T11:19:40.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.697+0000 7ffac5b06b80 1 -- 172.21.15.100:0/621666603 wait complete. 2024-05-10T11:19:40.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.697+0000 7ffac5b06b80 1 -- start start 2024-05-10T11:19:40.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.697+0000 7ffac5b06b80 1 Processor -- start 2024-05-10T11:19:40.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.698+0000 7ffac5b06b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56307382b4d0 0x563073836ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:40.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.698+0000 7ffac5b06b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56307382bdf0 0x563073837420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:40.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.698+0000 7ffac5b06b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56307382fc80 0x563073834480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:40.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.698+0000 7ffab900f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56307382b4d0 0x563073836ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:40.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.698+0000 7ffac5b06b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x563073837e30 con 0x56307382b4d0 2024-05-10T11:19:40.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.698+0000 7ffac5b06b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5630736e3c30 con 0x56307382fc80 2024-05-10T11:19:40.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.698+0000 7ffac5b06b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563073837fb0 con 0x56307382bdf0 2024-05-10T11:19:40.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.698+0000 7ffab900f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56307382b4d0 0x563073836ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60518/0 (socket says 172.21.15.100:60518) 2024-05-10T11:19:40.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.698+0000 7ffab900f700 1 -- 172.21.15.100:0/744094864 learned_addr learned my addr 172.21.15.100:0/744094864 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:40.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.698+0000 7ffab880e700 1 --2- 172.21.15.100:0/744094864 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56307382bdf0 0x563073837420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:40.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.698+0000 7ffac5b06b80 10 client.0 _ll_register_callbacks cb 0x563073715920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:19:40.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.698+0000 7ffab9810700 1 --2- 172.21.15.100:0/744094864 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56307382fc80 0x563073834480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:40.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.698+0000 7ffab880e700 1 -- 172.21.15.100:0/744094864 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56307382fc80 msgr2=0x563073834480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:40.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.698+0000 7ffab880e700 1 --2- 172.21.15.100:0/744094864 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56307382fc80 0x563073834480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:40.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.698+0000 7ffab880e700 1 -- 172.21.15.100:0/744094864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56307382b4d0 msgr2=0x563073836ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:40.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.699+0000 7ffab880e700 1 --2- 172.21.15.100:0/744094864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56307382b4d0 0x563073836ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:40.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.699+0000 7ffab880e700 1 -- 172.21.15.100:0/744094864 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563073719690 con 0x56307382bdf0 2024-05-10T11:19:40.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.699+0000 7ffab900f700 1 --2- 172.21.15.100:0/744094864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56307382b4d0 0x563073836ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:19:40.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.699+0000 7ffab880e700 1 --2- 172.21.15.100:0/744094864 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56307382bdf0 0x563073837420 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ffab4111490 tx=0x7ffaa4009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5345f5d65f072f85 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:40.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.699+0000 7ffa9cff9700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:19:40.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.699+0000 7ffa9cff9700 1 -- 172.21.15.100:0/744094864 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa4004030 con 0x56307382bdf0 2024-05-10T11:19:40.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.699+0000 7ffa9cff9700 1 -- 172.21.15.100:0/744094864 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7ffaa40041d0 con 0x56307382bdf0 2024-05-10T11:19:40.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.699+0000 7ffac5b06b80 1 -- 172.21.15.100:0/744094864 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5630737f7650 con 0x56307382bdf0 2024-05-10T11:19:40.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.699+0000 7ffa9cff9700 1 -- 172.21.15.100:0/744094864 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ffaa4025050 con 0x56307382bdf0 2024-05-10T11:19:40.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.699+0000 7ffac5b06b80 10 client.25072 fetch_fsmap learned FSMap version 27 2024-05-10T11:19:40.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.700+0000 7ffac5b06b80 1 -- 172.21.15.100:0/744094864 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5630738425b0 con 0x56307382bdf0 2024-05-10T11:19:40.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.700+0000 7ffa9cff9700 1 -- 172.21.15.100:0/744094864 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffaa40043d0 con 0x56307382bdf0 2024-05-10T11:19:40.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.700+0000 7ffa9cff9700 1 -- 172.21.15.100:0/744094864 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ffaa40105c0 con 0x56307382bdf0 2024-05-10T11:19:40.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.700+0000 7ffac5b06b80 10 client.25072 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T11:19:40.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.700+0000 7ffac5b06b80 10 client.25072 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-10T11:19:40.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.700+0000 7ffac5b06b80 20 client.25072 populate_metadata read hostname 'smithi100' 2024-05-10T11:19:40.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.700+0000 7ffac5b06b80 1 --2- 172.21.15.100:0/744094864 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x563073844bf0 0x563073846fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:40.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.700+0000 7ffac5b06b80 4 client.25072 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:19:40.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.700+0000 7ffac5b06b80 1 -- 172.21.15.100:0/744094864 --> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630737f7650 con 0x563073844bf0 2024-05-10T11:19:40.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.700+0000 7ffab900f700 1 --2- 172.21.15.100:0/744094864 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x563073844bf0 0x563073846fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:40.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.701+0000 7ffa9cff9700 10 client.25072 ms_handle_connect on v2:172.21.15.178:6838/3760985404 2024-05-10T11:19:40.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.701+0000 7ffab900f700 1 --2- 172.21.15.100:0/744094864 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x563073844bf0 0x563073846fe0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=181d2945648cd8fa server_cookie=87861d7c4ddd5066 in_seq=0 out_seq=0 2024-05-10T11:19:40.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.702+0000 7ffa9cff9700 1 -- 172.21.15.100:0/744094864 <== mds.1 v2:172.21.15.178:6838/3760985404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5630737f7650 con 0x563073844bf0 2024-05-10T11:19:40.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.702+0000 7ffa9cff9700 10 client.25072 handle_command_reply: tid=0 2024-05-10T11:19:40.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.702+0000 7ffac5b06b80 1 client.25072 shutdown 2024-05-10T11:19:40.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.702+0000 7ffac5b06b80 1 -- 172.21.15.100:0/744094864 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56307382bdf0 msgr2=0x563073837420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:40.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.702+0000 7ffac5b06b80 1 --2- 172.21.15.100:0/744094864 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56307382bdf0 0x563073837420 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ffab4111490 tx=0x7ffaa4009f90 comp rx=0 tx=0).stop 2024-05-10T11:19:40.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.702+0000 7ffac5b06b80 1 -- 172.21.15.100:0/744094864 shutdown_connections 2024-05-10T11:19:40.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.702+0000 7ffac5b06b80 1 --2- 172.21.15.100:0/744094864 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x563073844bf0 0x563073846fe0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:40.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.702+0000 7ffac5b06b80 1 --2- 172.21.15.100:0/744094864 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56307382fc80 0x563073834480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:40.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.702+0000 7ffac5b06b80 1 --2- 172.21.15.100:0/744094864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56307382b4d0 0x563073836ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:40.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.702+0000 7ffac5b06b80 1 --2- 172.21.15.100:0/744094864 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56307382bdf0 0x563073837420 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:40.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.702+0000 7ffac5b06b80 1 -- 172.21.15.100:0/744094864 >> 172.21.15.100:0/744094864 conn(0x563073757100 msgr2=0x563073820720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:40.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.702+0000 7ffa9cff9700 0 client.25072 ms_handle_reset on v2:172.21.15.178:6838/3760985404 2024-05-10T11:19:40.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.702+0000 7ffac5b06b80 1 -- 172.21.15.100:0/744094864 shutdown_connections 2024-05-10T11:19:40.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.702+0000 7ffac5b06b80 1 -- 172.21.15.100:0/744094864 wait complete. 2024-05-10T11:19:40.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.702+0000 7ffac5b06b80 20 client.25072 trim_cache size 0 max 16384 2024-05-10T11:19:40.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.717+0000 7ffac5b06b80 1 Processor -- start 2024-05-10T11:19:40.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.717+0000 7ffac5b06b80 1 -- start start 2024-05-10T11:19:40.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.718+0000 7ffac5b06b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56307382b4d0 0x56307382d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:40.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.718+0000 7ffac5b06b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56307382bdf0 0x56307382db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:40.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.718+0000 7ffac5b06b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56307382fc80 0x56307384c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:40.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.718+0000 7ffac5b06b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ffa98011280 con 0x56307382fc80 2024-05-10T11:19:40.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.718+0000 7ffac5b06b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ffa9800eb10 con 0x56307382bdf0 2024-05-10T11:19:40.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.718+0000 7ffac5b06b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ffa9800c960 con 0x56307382b4d0 2024-05-10T11:19:40.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.718+0000 7ffab9810700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56307382fc80 0x56307384c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:40.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.718+0000 7ffab9810700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56307382fc80 0x56307384c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60534/0 (socket says 172.21.15.100:60534) 2024-05-10T11:19:40.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.718+0000 7ffab9810700 1 -- 172.21.15.100:0/2410141514 learned_addr learned my addr 172.21.15.100:0/2410141514 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:40.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.718+0000 7ffab900f700 1 --2- 172.21.15.100:0/2410141514 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56307382b4d0 0x56307382d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:40.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.718+0000 7ffab880e700 1 --2- 172.21.15.100:0/2410141514 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56307382bdf0 0x56307382db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:40.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.718+0000 7ffab9810700 1 -- 172.21.15.100:0/2410141514 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56307382b4d0 msgr2=0x56307382d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:40.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.718+0000 7ffab9810700 1 --2- 172.21.15.100:0/2410141514 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56307382b4d0 0x56307382d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:40.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.718+0000 7ffab9810700 1 -- 172.21.15.100:0/2410141514 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56307382bdf0 msgr2=0x56307382db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:40.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.718+0000 7ffab9810700 1 --2- 172.21.15.100:0/2410141514 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56307382bdf0 0x56307382db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:40.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.718+0000 7ffab9810700 1 -- 172.21.15.100:0/2410141514 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563073826830 con 0x56307382fc80 2024-05-10T11:19:40.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.719+0000 7ffab9810700 1 --2- 172.21.15.100:0/2410141514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56307382fc80 0x56307384c8d0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7ffaac00b0a0 tx=0x7ffaac020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbb6db35f50b610f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:40.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.719+0000 7ffa9effd700 1 -- 172.21.15.100:0/2410141514 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaac02e070 con 0x56307382fc80 2024-05-10T11:19:40.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.719+0000 7ffa9effd700 1 -- 172.21.15.100:0/2410141514 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffaac023040 con 0x56307382fc80 2024-05-10T11:19:40.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.719+0000 7ffa9effd700 1 -- 172.21.15.100:0/2410141514 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaac01f7c0 con 0x56307382fc80 2024-05-10T11:19:40.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.720+0000 7ffac5b06b80 1 -- 172.21.15.100:0/2410141514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56307382fc80 msgr2=0x56307384c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:40.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.720+0000 7ffac5b06b80 1 --2- 172.21.15.100:0/2410141514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56307382fc80 0x56307384c8d0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7ffaac00b0a0 tx=0x7ffaac020000 comp rx=0 tx=0).stop 2024-05-10T11:19:40.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.720+0000 7ffac5b06b80 1 -- 172.21.15.100:0/2410141514 shutdown_connections 2024-05-10T11:19:40.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.720+0000 7ffac5b06b80 1 --2- 172.21.15.100:0/2410141514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56307382fc80 0x56307384c8d0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:40.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.720+0000 7ffac5b06b80 1 --2- 172.21.15.100:0/2410141514 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56307382b4d0 0x56307382d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:40.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.720+0000 7ffac5b06b80 1 --2- 172.21.15.100:0/2410141514 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56307382bdf0 0x56307382db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:40.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.720+0000 7ffac5b06b80 1 -- 172.21.15.100:0/2410141514 >> 172.21.15.100:0/2410141514 conn(0x563073757100 msgr2=0x563073832b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:40.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.720+0000 7ffac5b06b80 1 -- 172.21.15.100:0/2410141514 shutdown_connections 2024-05-10T11:19:40.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.720+0000 7ffac5b06b80 1 -- 172.21.15.100:0/2410141514 wait complete. 2024-05-10T11:19:40.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.720+0000 7ffac5b06b80 1 -- start start 2024-05-10T11:19:40.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.720+0000 7ffac5b06b80 1 Processor -- start 2024-05-10T11:19:40.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.721+0000 7ffac5b06b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56307382b4d0 0x563073838290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:40.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.721+0000 7ffac5b06b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56307382bdf0 0x5630738387d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:40.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.721+0000 7ffac5b06b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56307382fc80 0x5630738be480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:40.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.721+0000 7ffac5b06b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ffa9800cfd0 con 0x56307382bdf0 2024-05-10T11:19:40.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.721+0000 7ffac5b06b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ffa9800e440 con 0x56307382b4d0 2024-05-10T11:19:40.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.721+0000 7ffac5b06b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ffa9800d6a0 con 0x56307382fc80 2024-05-10T11:19:40.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.721+0000 7ffac5b06b80 10 client.0 _ll_register_callbacks cb 0x563073715920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:19:40.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.721+0000 7ffab880e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56307382bdf0 0x5630738387d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:40.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.721+0000 7ffab900f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56307382b4d0 0x563073838290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:40.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.721+0000 7ffab900f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56307382b4d0 0x563073838290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:39140/0 (socket says 172.21.15.100:39140) 2024-05-10T11:19:40.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.721+0000 7ffab900f700 1 -- 172.21.15.100:0/1477389835 learned_addr learned my addr 172.21.15.100:0/1477389835 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:40.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.721+0000 7ffab880e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56307382bdf0 0x5630738387d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60550/0 (socket says 172.21.15.100:60550) 2024-05-10T11:19:40.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.721+0000 7ffab9810700 1 --2- 172.21.15.100:0/1477389835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56307382fc80 0x5630738be480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:40.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.721+0000 7ffab880e700 1 -- 172.21.15.100:0/1477389835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56307382fc80 msgr2=0x5630738be480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:40.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.721+0000 7ffab880e700 1 --2- 172.21.15.100:0/1477389835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56307382fc80 0x5630738be480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:40.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.721+0000 7ffab880e700 1 -- 172.21.15.100:0/1477389835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56307382b4d0 msgr2=0x563073838290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:40.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.721+0000 7ffab880e700 1 --2- 172.21.15.100:0/1477389835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56307382b4d0 0x563073838290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:40.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.721+0000 7ffab880e700 1 -- 172.21.15.100:0/1477389835 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630738be9c0 con 0x56307382bdf0 2024-05-10T11:19:40.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.721+0000 7ffab880e700 1 --2- 172.21.15.100:0/1477389835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56307382bdf0 0x5630738387d0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ffaa4010b80 tx=0x7ffaa4009dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c8cfff207bc898b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:40.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.721+0000 7ffa9cff9700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:19:40.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.722+0000 7ffa9cff9700 1 -- 172.21.15.100:0/1477389835 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa4010640 con 0x56307382bdf0 2024-05-10T11:19:40.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.722+0000 7ffac5b06b80 1 -- 172.21.15.100:0/1477389835 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffaac023040 con 0x56307382bdf0 2024-05-10T11:19:40.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.722+0000 7ffa9cff9700 1 -- 172.21.15.100:0/1477389835 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7ffaa4018040 con 0x56307382bdf0 2024-05-10T11:19:40.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.722+0000 7ffa9cff9700 1 -- 172.21.15.100:0/1477389835 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ffaa4037050 con 0x56307382bdf0 2024-05-10T11:19:40.728 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:19:40.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.722+0000 7ffac5b06b80 10 client.15543 fetch_fsmap learned FSMap version 27 2024-05-10T11:19:40.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.722+0000 7ffac5b06b80 1 -- 172.21.15.100:0/1477389835 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5630738a7570 con 0x56307382bdf0 2024-05-10T11:19:40.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.722+0000 7ffa9cff9700 1 -- 172.21.15.100:0/1477389835 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffaac023040 con 0x56307382bdf0 2024-05-10T11:19:40.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.722+0000 7ffa9cff9700 1 -- 172.21.15.100:0/1477389835 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ffaa4003680 con 0x56307382bdf0 2024-05-10T11:19:40.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.722+0000 7ffac5b06b80 10 client.15543 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T11:19:40.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.722+0000 7ffac5b06b80 10 client.15543 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-10T11:19:40.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.722+0000 7ffac5b06b80 20 client.15543 populate_metadata read hostname 'smithi100' 2024-05-10T11:19:40.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.722+0000 7ffac5b06b80 1 --2- 172.21.15.100:0/1477389835 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x5630738a7cd0 0x5630738aa0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:40.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.722+0000 7ffac5b06b80 4 client.15543 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:19:40.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.722+0000 7ffac5b06b80 1 -- 172.21.15.100:0/1477389835 --> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5630738261b0 con 0x5630738a7cd0 2024-05-10T11:19:40.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.723+0000 7ffab900f700 1 --2- 172.21.15.100:0/1477389835 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x5630738a7cd0 0x5630738aa0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:40.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.723+0000 7ffa9cff9700 10 client.15543 ms_handle_connect on v2:172.21.15.178:6838/3760985404 2024-05-10T11:19:40.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.723+0000 7ffab900f700 1 --2- 172.21.15.100:0/1477389835 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x5630738a7cd0 0x5630738aa0f0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1d157d1e673d25cd server_cookie=c90c39ed5f30a5de in_seq=0 out_seq=0 2024-05-10T11:19:40.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.723+0000 7ffa9cff9700 1 -- 172.21.15.100:0/1477389835 <== mds.1 v2:172.21.15.178:6838/3760985404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5630738261b0 con 0x5630738a7cd0 2024-05-10T11:19:40.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.723+0000 7ffa9cff9700 10 client.15543 handle_command_reply: tid=0 2024-05-10T11:19:40.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.723+0000 7ffac5b06b80 1 client.15543 shutdown 2024-05-10T11:19:40.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.724+0000 7ffac5b06b80 1 -- 172.21.15.100:0/1477389835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56307382bdf0 msgr2=0x5630738387d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:40.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.724+0000 7ffac5b06b80 1 --2- 172.21.15.100:0/1477389835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56307382bdf0 0x5630738387d0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ffaa4010b80 tx=0x7ffaa4009dc0 comp rx=0 tx=0).stop 2024-05-10T11:19:40.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.724+0000 7ffac5b06b80 1 -- 172.21.15.100:0/1477389835 shutdown_connections 2024-05-10T11:19:40.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.724+0000 7ffac5b06b80 1 --2- 172.21.15.100:0/1477389835 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x5630738a7cd0 0x5630738aa0f0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:40.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.724+0000 7ffac5b06b80 1 --2- 172.21.15.100:0/1477389835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56307382fc80 0x5630738be480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:40.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.724+0000 7ffac5b06b80 1 --2- 172.21.15.100:0/1477389835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56307382b4d0 0x563073838290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:40.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.724+0000 7ffac5b06b80 1 --2- 172.21.15.100:0/1477389835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56307382bdf0 0x5630738387d0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:40.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.724+0000 7ffac5b06b80 1 -- 172.21.15.100:0/1477389835 >> 172.21.15.100:0/1477389835 conn(0x563073757100 msgr2=0x56307381fe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:40.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.724+0000 7ffa9cff9700 0 client.15543 ms_handle_reset on v2:172.21.15.178:6838/3760985404 2024-05-10T11:19:40.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.724+0000 7ffac5b06b80 1 -- 172.21.15.100:0/1477389835 shutdown_connections 2024-05-10T11:19:40.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.724+0000 7ffac5b06b80 1 -- 172.21.15.100:0/1477389835 wait complete. 2024-05-10T11:19:40.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.724+0000 7ffac5b06b80 20 client.15543 trim_cache size 0 max 16384 2024-05-10T11:19:40.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.724+0000 7ffabb273700 1 -- 172.21.15.100:0/1334141972 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ffaa00b2880 msgr2=0x7ffaa00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:40.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.724+0000 7ffabb273700 1 --2- 172.21.15.100:0/1334141972 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ffaa00b2880 0x7ffaa00b4d40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ffaac006070 tx=0x7ffaac006000 comp rx=0 tx=0).stop 2024-05-10T11:19:40.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.724+0000 7ffabb273700 1 -- 172.21.15.100:0/1334141972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffab4102390 msgr2=0x7ffab410fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:40.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.724+0000 7ffabb273700 1 --2- 172.21.15.100:0/1334141972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffab4102390 0x7ffab410fcd0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7ffab000d950 tx=0x7ffab000dc60 comp rx=0 tx=0).stop 2024-05-10T11:19:40.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.724+0000 7ffabb273700 1 -- 172.21.15.100:0/1334141972 shutdown_connections 2024-05-10T11:19:40.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.724+0000 7ffabb273700 1 --2- 172.21.15.100:0/1334141972 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ffaa00b2880 0x7ffaa00b4d40 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:40.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.724+0000 7ffabb273700 1 --2- 172.21.15.100:0/1334141972 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffab4108400 0x7ffab4110750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:40.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.724+0000 7ffabb273700 1 --2- 172.21.15.100:0/1334141972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffab4102390 0x7ffab410fcd0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:40.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.724+0000 7ffabb273700 1 --2- 172.21.15.100:0/1334141972 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffab4102d50 0x7ffab4110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:40.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.724+0000 7ffabb273700 1 -- 172.21.15.100:0/1334141972 >> 172.21.15.100:0/1334141972 conn(0x7ffab4079310 msgr2=0x7ffab41010c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:40.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.725+0000 7ffabb273700 1 -- 172.21.15.100:0/1334141972 shutdown_connections 2024-05-10T11:19:40.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:40.725+0000 7ffabb273700 1 -- 172.21.15.100:0/1334141972 wait complete. 2024-05-10T11:19:41.188 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.188+0000 7f8f68dbe640 1 -- 172.21.15.100:0/1463269813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f641034c0 msgr2=0x7f8f64109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:41.188 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.188+0000 7f8f68dbe640 1 --2- 172.21.15.100:0/1463269813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f641034c0 0x7f8f64109d50 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c009ca0 tx=0x7f8f5c02a8c0 comp rx=0 tx=0).stop 2024-05-10T11:19:41.188 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.188+0000 7f8f68dbe640 1 -- 172.21.15.100:0/1463269813 shutdown_connections 2024-05-10T11:19:41.188 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.188+0000 7f8f68dbe640 1 --2- 172.21.15.100:0/1463269813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f641034c0 0x7f8f64109d50 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:41.189 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.188+0000 7f8f68dbe640 1 --2- 172.21.15.100:0/1463269813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f64102b00 0x7f8f64102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:41.189 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.188+0000 7f8f68dbe640 1 --2- 172.21.15.100:0/1463269813 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f64101910 0x7f8f64101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:41.189 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.188+0000 7f8f68dbe640 1 -- 172.21.15.100:0/1463269813 >> 172.21.15.100:0/1463269813 conn(0x7f8f640fce80 msgr2=0x7f8f640ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:41.189 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.188+0000 7f8f68dbe640 1 -- 172.21.15.100:0/1463269813 shutdown_connections 2024-05-10T11:19:41.189 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.188+0000 7f8f68dbe640 1 -- 172.21.15.100:0/1463269813 wait complete. 2024-05-10T11:19:41.191 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.189+0000 7f8f68dbe640 1 Processor -- start 2024-05-10T11:19:41.191 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.189+0000 7f8f68dbe640 1 -- start start 2024-05-10T11:19:41.191 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.190+0000 7f8f68dbe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f64101910 0x7f8f641979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:41.191 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.190+0000 7f8f68dbe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f64102b00 0x7f8f64197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:41.192 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.190+0000 7f8f68dbe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f641034c0 0x7f8f6419ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:41.192 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.190+0000 7f8f68dbe640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6410d340 con 0x7f8f64101910 2024-05-10T11:19:41.192 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.190+0000 7f8f68dbe640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6410d1c0 con 0x7f8f64102b00 2024-05-10T11:19:41.192 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.190+0000 7f8f68dbe640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6410d4c0 con 0x7f8f641034c0 2024-05-10T11:19:41.192 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.190+0000 7f8f6259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f64101910 0x7f8f641979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:41.192 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.190+0000 7f8f6259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f64101910 0x7f8f641979b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60562/0 (socket says 172.21.15.100:60562) 2024-05-10T11:19:41.192 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.190+0000 7f8f6259c640 1 -- 172.21.15.100:0/522515104 learned_addr learned my addr 172.21.15.100:0/522515104 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:41.192 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.190+0000 7f8f61d9b640 1 --2- 172.21.15.100:0/522515104 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f64102b00 0x7f8f64197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:41.192 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.190+0000 7f8f62d9d640 1 --2- 172.21.15.100:0/522515104 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f641034c0 0x7f8f6419ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:41.192 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.190+0000 7f8f6259c640 1 -- 172.21.15.100:0/522515104 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f641034c0 msgr2=0x7f8f6419ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:41.192 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.190+0000 7f8f6259c640 1 --2- 172.21.15.100:0/522515104 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f641034c0 0x7f8f6419ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:41.192 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.190+0000 7f8f6259c640 1 -- 172.21.15.100:0/522515104 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f64102b00 msgr2=0x7f8f64197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:41.192 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.190+0000 7f8f6259c640 1 --2- 172.21.15.100:0/522515104 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f64102b00 0x7f8f64197ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:41.193 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.190+0000 7f8f6259c640 1 -- 172.21.15.100:0/522515104 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f6419f5d0 con 0x7f8f64101910 2024-05-10T11:19:41.193 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.190+0000 7f8f61d9b640 1 --2- 172.21.15.100:0/522515104 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f64102b00 0x7f8f64197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:19:41.193 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.191+0000 7f8f6259c640 1 --2- 172.21.15.100:0/522515104 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f64101910 0x7f8f641979b0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f8f54010f40 tx=0x7f8f5400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:41.193 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.191+0000 7f8f537fe640 1 -- 172.21.15.100:0/522515104 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f54010020 con 0x7f8f64101910 2024-05-10T11:19:41.193 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.191+0000 7f8f537fe640 1 -- 172.21.15.100:0/522515104 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8f540101c0 con 0x7f8f64101910 2024-05-10T11:19:41.193 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.191+0000 7f8f537fe640 1 -- 172.21.15.100:0/522515104 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f54018610 con 0x7f8f64101910 2024-05-10T11:19:41.193 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.191+0000 7f8f68dbe640 1 -- 172.21.15.100:0/522515104 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f6406a0c0 con 0x7f8f64101910 2024-05-10T11:19:41.193 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.191+0000 7f8f68dbe640 1 -- 172.21.15.100:0/522515104 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f6406a5c0 con 0x7f8f64101910 2024-05-10T11:19:41.194 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.194+0000 7f8f537fe640 1 -- 172.21.15.100:0/522515104 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f8f5400f510 con 0x7f8f64101910 2024-05-10T11:19:41.194 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.195+0000 7f8f6bbb0740 1 Processor -- start 2024-05-10T11:19:41.196 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.195+0000 7f8f6bbb0740 1 -- start start 2024-05-10T11:19:41.196 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.195+0000 7f8f537fe640 1 --2- 172.21.15.100:0/522515104 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8f4c0b2920 0x7f8f4c0b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:41.196 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.195+0000 7f8f6bbb0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d58864adb0 0x55d58864b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:41.196 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.195+0000 7f8f6bbb0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d58864b7a0 0x55d5886502a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:41.196 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.195+0000 7f8f6bbb0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d588650910 0x55d588652da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:41.197 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.195+0000 7f8f6bbb0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55d5885af5d0 con 0x55d58864b7a0 2024-05-10T11:19:41.197 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.195+0000 7f8f6bbb0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d588568890 con 0x55d58864adb0 2024-05-10T11:19:41.197 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.195+0000 7f8f6bbb0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d5885ab590 con 0x55d588650910 2024-05-10T11:19:41.197 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.195+0000 7f8f61d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d58864b7a0 0x55d5886502a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:41.197 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.195+0000 7f8f537fe640 1 -- 172.21.15.100:0/522515104 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f8f540e2340 con 0x7f8f64101910 2024-05-10T11:19:41.197 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.195+0000 7f8f62d9d640 1 --2- 172.21.15.100:0/522515104 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8f4c0b2920 0x7f8f4c0b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:41.197 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.196+0000 7f8f6259c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d58864adb0 0x55d58864b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:41.197 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.196+0000 7f8f6259c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d58864adb0 0x55d58864b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:39164/0 (socket says 172.21.15.100:39164) 2024-05-10T11:19:41.197 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.196+0000 7f8f6259c640 1 -- 172.21.15.100:0/1182880334 learned_addr learned my addr 172.21.15.100:0/1182880334 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:41.197 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.196+0000 7f8f6259c640 1 -- 172.21.15.100:0/1182880334 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d588650910 msgr2=0x55d588652da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:41.197 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.196+0000 7f8f6259c640 1 --2- 172.21.15.100:0/1182880334 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d588650910 0x55d588652da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:41.197 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.196+0000 7f8f6259c640 1 -- 172.21.15.100:0/1182880334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d58864b7a0 msgr2=0x55d5886502a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:41.198 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.196+0000 7f8f6259c640 1 --2- 172.21.15.100:0/1182880334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d58864b7a0 0x55d5886502a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:41.198 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.196+0000 7f8f6259c640 1 -- 172.21.15.100:0/1182880334 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5882c1210 con 0x55d58864adb0 2024-05-10T11:19:41.198 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.196+0000 7f8f62d9d640 1 --2- 172.21.15.100:0/522515104 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8f4c0b2920 0x7f8f4c0b4de0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c006010 tx=0x7f8f5c031450 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:41.198 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.196+0000 7f8f6259c640 1 --2- 172.21.15.100:0/1182880334 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d58864adb0 0x55d58864b1d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8f540c3200 tx=0x7f8f540a8ff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ecb44a8aae61e28 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:41.198 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.196+0000 7f8f50ff9640 1 -- 172.21.15.100:0/1182880334 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5401e070 con 0x55d58864adb0 2024-05-10T11:19:41.198 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.196+0000 7f8f50ff9640 1 -- 172.21.15.100:0/1182880334 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8f540a9750 con 0x55d58864adb0 2024-05-10T11:19:41.198 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.196+0000 7f8f50ff9640 1 -- 172.21.15.100:0/1182880334 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f540c48f0 con 0x55d58864adb0 2024-05-10T11:19:41.203 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.197+0000 7f8f6bbb0740 1 -- 172.21.15.100:0/1182880334 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d58864adb0 msgr2=0x55d58864b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:41.203 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.197+0000 7f8f6bbb0740 1 --2- 172.21.15.100:0/1182880334 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d58864adb0 0x55d58864b1d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8f540c3200 tx=0x7f8f540a8ff0 comp rx=0 tx=0).stop 2024-05-10T11:19:41.203 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.197+0000 7f8f6bbb0740 1 -- 172.21.15.100:0/1182880334 shutdown_connections 2024-05-10T11:19:41.203 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.197+0000 7f8f6bbb0740 1 --2- 172.21.15.100:0/1182880334 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d588650910 0x55d588652da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:41.203 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.197+0000 7f8f6bbb0740 1 --2- 172.21.15.100:0/1182880334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d58864b7a0 0x55d5886502a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:41.203 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.197+0000 7f8f6bbb0740 1 --2- 172.21.15.100:0/1182880334 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d58864adb0 0x55d58864b1d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:41.203 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.197+0000 7f8f6bbb0740 1 -- 172.21.15.100:0/1182880334 >> 172.21.15.100:0/1182880334 conn(0x55d588646420 msgr2=0x55d588646820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:41.203 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.198+0000 7f8f6bbb0740 1 -- 172.21.15.100:0/1182880334 shutdown_connections 2024-05-10T11:19:41.203 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.198+0000 7f8f6bbb0740 1 -- 172.21.15.100:0/1182880334 wait complete. 2024-05-10T11:19:41.203 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.198+0000 7f8f6bbb0740 1 -- start start 2024-05-10T11:19:41.203 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.198+0000 7f8f6bbb0740 1 Processor -- start 2024-05-10T11:19:41.203 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.198+0000 7f8f6bbb0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d58864adb0 0x55d588654f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:41.204 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.198+0000 7f8f6bbb0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d58864b7a0 0x55d5886554d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:41.204 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.198+0000 7f8f6bbb0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d588650910 0x55d58865c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:41.204 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.198+0000 7f8f6bbb0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55d588653800 con 0x55d58864b7a0 2024-05-10T11:19:41.204 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.198+0000 7f8f6bbb0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d58856c370 con 0x55d58864adb0 2024-05-10T11:19:41.204 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.198+0000 7f8f6bbb0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d588653980 con 0x55d588650910 2024-05-10T11:19:41.204 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.198+0000 7f8f6bbb0740 10 client.0 _ll_register_callbacks cb 0x55d5882c0a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:19:41.204 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.198+0000 7f8f61d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d58864b7a0 0x55d5886554d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:41.204 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.198+0000 7f8f61d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d58864b7a0 0x55d5886554d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60578/0 (socket says 172.21.15.100:60578) 2024-05-10T11:19:41.204 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.199+0000 7f8f61d9b640 1 -- 172.21.15.100:0/3341825328 learned_addr learned my addr 172.21.15.100:0/3341825328 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:41.204 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.199+0000 7f8f6259c640 1 --2- 172.21.15.100:0/3341825328 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d58864adb0 0x55d588654f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:41.204 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.199+0000 7f8f61d9b640 1 -- 172.21.15.100:0/3341825328 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d588650910 msgr2=0x55d58865c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:41.204 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.199+0000 7f8f62d9d640 1 --2- 172.21.15.100:0/3341825328 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d588650910 0x55d58865c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:41.204 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.199+0000 7f8f61d9b640 1 --2- 172.21.15.100:0/3341825328 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d588650910 0x55d58865c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:41.204 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.199+0000 7f8f61d9b640 1 -- 172.21.15.100:0/3341825328 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d58864adb0 msgr2=0x55d588654f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:41.205 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.199+0000 7f8f61d9b640 1 --2- 172.21.15.100:0/3341825328 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d58864adb0 0x55d588654f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:41.205 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.199+0000 7f8f61d9b640 1 -- 172.21.15.100:0/3341825328 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d58838d620 con 0x55d58864b7a0 2024-05-10T11:19:41.205 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.199+0000 7f8f6259c640 1 --2- 172.21.15.100:0/3341825328 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d58864adb0 0x55d588654f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:19:41.205 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.199+0000 7f8f62d9d640 1 --2- 172.21.15.100:0/3341825328 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d588650910 0x55d58865c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:19:41.205 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.199+0000 7f8f61d9b640 1 --2- 172.21.15.100:0/3341825328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d58864b7a0 0x55d5886554d0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f8f58012a10 tx=0x7f8f58012ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cef3214a3c53e228 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:41.205 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.199+0000 7f8f49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:19:41.205 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.199+0000 7f8f49ffb640 1 -- 172.21.15.100:0/3341825328 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f58004030 con 0x55d58864b7a0 2024-05-10T11:19:41.205 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.199+0000 7f8f49ffb640 1 -- 172.21.15.100:0/3341825328 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f8f580041d0 con 0x55d58864b7a0 2024-05-10T11:19:41.205 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.199+0000 7f8f6bbb0740 1 -- 172.21.15.100:0/3341825328 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f540a9750 con 0x55d58864b7a0 2024-05-10T11:19:41.205 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.199+0000 7f8f49ffb640 1 -- 172.21.15.100:0/3341825328 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8f580043d0 con 0x55d58864b7a0 2024-05-10T11:19:41.205 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.199+0000 7f8f6bbb0740 10 client.15555 fetch_fsmap learned FSMap version 27 2024-05-10T11:19:41.205 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.199+0000 7f8f6bbb0740 1 -- 172.21.15.100:0/3341825328 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d58865f5c0 con 0x55d58864b7a0 2024-05-10T11:19:41.205 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.200+0000 7f8f49ffb640 1 -- 172.21.15.100:0/3341825328 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8f540a9750 con 0x55d58864b7a0 2024-05-10T11:19:41.205 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.200+0000 7f8f49ffb640 1 -- 172.21.15.100:0/3341825328 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f8f58003ba0 con 0x55d58864b7a0 2024-05-10T11:19:41.206 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.200+0000 7f8f6bbb0740 10 client.15555 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T11:19:41.206 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.200+0000 7f8f6bbb0740 10 client.15555 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:19:41.206 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.200+0000 7f8f6bbb0740 20 client.15555 populate_metadata read hostname 'smithi100' 2024-05-10T11:19:41.206 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.200+0000 7f8f6bbb0740 1 --2- 172.21.15.100:0/3341825328 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d588660d00 0x55d588663180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:41.206 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.200+0000 7f8f6bbb0740 4 client.15555 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:19:41.206 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.200+0000 7f8f6bbb0740 1 -- 172.21.15.100:0/3341825328 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5885bd1e0 con 0x55d588660d00 2024-05-10T11:19:41.206 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.200+0000 7f8f6259c640 1 --2- 172.21.15.100:0/3341825328 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d588660d00 0x55d588663180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:41.206 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.201+0000 7f8f6259c640 1 --2- 172.21.15.100:0/3341825328 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d588660d00 0x55d588663180 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92d754b0a2972eaf server_cookie=d26f067d246d3f79 in_seq=0 out_seq=0 2024-05-10T11:19:41.206 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.201+0000 7f8f49ffb640 10 client.15555 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:19:41.206 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.201+0000 7f8f49ffb640 1 -- 172.21.15.100:0/3341825328 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d5885bd1e0 con 0x55d588660d00 2024-05-10T11:19:41.206 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.201+0000 7f8f49ffb640 10 client.15555 handle_command_reply: tid=0 2024-05-10T11:19:41.206 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.201+0000 7f8f6bbb0740 1 client.15555 shutdown 2024-05-10T11:19:41.206 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.202+0000 7f8f6bbb0740 1 -- 172.21.15.100:0/3341825328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d58864b7a0 msgr2=0x55d5886554d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:41.206 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.202+0000 7f8f6bbb0740 1 --2- 172.21.15.100:0/3341825328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d58864b7a0 0x55d5886554d0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f8f58012a10 tx=0x7f8f58012ed0 comp rx=0 tx=0).stop 2024-05-10T11:19:41.207 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.202+0000 7f8f6bbb0740 1 -- 172.21.15.100:0/3341825328 shutdown_connections 2024-05-10T11:19:41.207 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.202+0000 7f8f6bbb0740 1 --2- 172.21.15.100:0/3341825328 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d588660d00 0x55d588663180 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:41.207 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.202+0000 7f8f6bbb0740 1 --2- 172.21.15.100:0/3341825328 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d588650910 0x55d58865c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:41.207 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.202+0000 7f8f6bbb0740 1 --2- 172.21.15.100:0/3341825328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d58864b7a0 0x55d5886554d0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:41.207 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.202+0000 7f8f6bbb0740 1 --2- 172.21.15.100:0/3341825328 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d58864adb0 0x55d588654f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:41.207 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.202+0000 7f8f6bbb0740 1 -- 172.21.15.100:0/3341825328 >> 172.21.15.100:0/3341825328 conn(0x55d588646420 msgr2=0x55d588645580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:41.207 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.202+0000 7f8f49ffb640 0 client.15555 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:19:41.207 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.202+0000 7f8f6bbb0740 1 -- 172.21.15.100:0/3341825328 shutdown_connections 2024-05-10T11:19:41.207 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.202+0000 7f8f6bbb0740 1 -- 172.21.15.100:0/3341825328 wait complete. 2024-05-10T11:19:41.207 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.202+0000 7f8f6bbb0740 20 client.15555 trim_cache size 0 max 16384 2024-05-10T11:19:41.219 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.217+0000 7f8f6bbb0740 1 Processor -- start 2024-05-10T11:19:41.219 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.217+0000 7f8f6bbb0740 1 -- start start 2024-05-10T11:19:41.219 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.217+0000 7f8f6bbb0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d58864adb0 0x55d58864ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:41.220 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.217+0000 7f8f6bbb0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d58864b7a0 0x55d58864cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:41.220 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.217+0000 7f8f6bbb0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d588650910 0x55d588644cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:41.220 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.217+0000 7f8f6bbb0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8f641aba60 con 0x55d58864b7a0 2024-05-10T11:19:41.220 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.217+0000 7f8f6bbb0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8f641a9330 con 0x55d588650910 2024-05-10T11:19:41.220 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.217+0000 7f8f6bbb0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8f641a7180 con 0x55d58864adb0 2024-05-10T11:19:41.220 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.218+0000 7f8f61d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d58864b7a0 0x55d58864cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:41.220 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.218+0000 7f8f61d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d58864b7a0 0x55d58864cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60584/0 (socket says 172.21.15.100:60584) 2024-05-10T11:19:41.220 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.218+0000 7f8f61d9b640 1 -- 172.21.15.100:0/2516318157 learned_addr learned my addr 172.21.15.100:0/2516318157 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:41.220 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.218+0000 7f8f62d9d640 1 --2- 172.21.15.100:0/2516318157 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d588650910 0x55d588644cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:41.220 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.218+0000 7f8f6259c640 1 --2- 172.21.15.100:0/2516318157 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d58864adb0 0x55d58864ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:41.220 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.218+0000 7f8f62d9d640 1 -- 172.21.15.100:0/2516318157 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d58864adb0 msgr2=0x55d58864ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:41.220 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.218+0000 7f8f62d9d640 1 --2- 172.21.15.100:0/2516318157 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d58864adb0 0x55d58864ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:41.220 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.218+0000 7f8f62d9d640 1 -- 172.21.15.100:0/2516318157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d58864b7a0 msgr2=0x55d58864cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:41.220 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.218+0000 7f8f62d9d640 1 --2- 172.21.15.100:0/2516318157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d58864b7a0 0x55d58864cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:41.221 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.218+0000 7f8f62d9d640 1 -- 172.21.15.100:0/2516318157 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d588641f70 con 0x55d588650910 2024-05-10T11:19:41.221 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.218+0000 7f8f61d9b640 1 --2- 172.21.15.100:0/2516318157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d58864b7a0 0x55d58864cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:19:41.221 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.218+0000 7f8f62d9d640 1 --2- 172.21.15.100:0/2516318157 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d588650910 0x55d588644cf0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c031040 tx=0x7f8f5c045000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c62f87711c0f5571 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:41.221 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.219+0000 7f8f4bfff640 1 -- 172.21.15.100:0/2516318157 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5c054070 con 0x55d588650910 2024-05-10T11:19:41.221 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.219+0000 7f8f4bfff640 1 -- 172.21.15.100:0/2516318157 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8f5c036040 con 0x55d588650910 2024-05-10T11:19:41.221 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.219+0000 7f8f4bfff640 1 -- 172.21.15.100:0/2516318157 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5c0045f0 con 0x55d588650910 2024-05-10T11:19:41.225 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.220+0000 7f8f6bbb0740 1 -- 172.21.15.100:0/2516318157 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d588650910 msgr2=0x55d588644cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:41.225 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.220+0000 7f8f6bbb0740 1 --2- 172.21.15.100:0/2516318157 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d588650910 0x55d588644cf0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c031040 tx=0x7f8f5c045000 comp rx=0 tx=0).stop 2024-05-10T11:19:41.225 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.220+0000 7f8f6bbb0740 1 -- 172.21.15.100:0/2516318157 shutdown_connections 2024-05-10T11:19:41.225 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.220+0000 7f8f6bbb0740 1 --2- 172.21.15.100:0/2516318157 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d588650910 0x55d588644cf0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:41.225 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.220+0000 7f8f6bbb0740 1 --2- 172.21.15.100:0/2516318157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d58864b7a0 0x55d58864cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:41.225 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.220+0000 7f8f6bbb0740 1 --2- 172.21.15.100:0/2516318157 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d58864adb0 0x55d58864ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:41.225 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.220+0000 7f8f6bbb0740 1 -- 172.21.15.100:0/2516318157 >> 172.21.15.100:0/2516318157 conn(0x55d588646420 msgr2=0x55d588654f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:41.225 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.221+0000 7f8f6bbb0740 1 -- 172.21.15.100:0/2516318157 shutdown_connections 2024-05-10T11:19:41.225 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.221+0000 7f8f6bbb0740 1 -- 172.21.15.100:0/2516318157 wait complete. 2024-05-10T11:19:41.225 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.221+0000 7f8f6bbb0740 1 -- start start 2024-05-10T11:19:41.225 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.221+0000 7f8f6bbb0740 1 Processor -- start 2024-05-10T11:19:41.225 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.221+0000 7f8f6bbb0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d58864adb0 0x55d588659750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:41.225 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.221+0000 7f8f6bbb0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d58864b7a0 0x55d588659c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.221+0000 7f8f6bbb0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d588650910 0x55d58865ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.221+0000 7f8f6bbb0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8f641a77f0 con 0x55d58864adb0 2024-05-10T11:19:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.221+0000 7f8f6bbb0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8f641a8c60 con 0x55d588650910 2024-05-10T11:19:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.221+0000 7f8f6bbb0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8f641a7ec0 con 0x55d58864b7a0 2024-05-10T11:19:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.221+0000 7f8f6bbb0740 10 client.0 _ll_register_callbacks cb 0x7f8f58009be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:19:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.221+0000 7f8f6259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d58864adb0 0x55d588659750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.222+0000 7f8f6259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d58864adb0 0x55d588659750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60588/0 (socket says 172.21.15.100:60588) 2024-05-10T11:19:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.222+0000 7f8f6259c640 1 -- 172.21.15.100:0/2644766285 learned_addr learned my addr 172.21.15.100:0/2644766285 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.222+0000 7f8f61d9b640 1 --2- 172.21.15.100:0/2644766285 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d58864b7a0 0x55d588659c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.222+0000 7f8f62d9d640 1 --2- 172.21.15.100:0/2644766285 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d588650910 0x55d58865ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.222+0000 7f8f6259c640 1 -- 172.21.15.100:0/2644766285 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d58864b7a0 msgr2=0x55d588659c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.222+0000 7f8f6259c640 1 --2- 172.21.15.100:0/2644766285 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d58864b7a0 0x55d588659c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.222+0000 7f8f6259c640 1 -- 172.21.15.100:0/2644766285 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d588650910 msgr2=0x55d58865ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.222+0000 7f8f6259c640 1 --2- 172.21.15.100:0/2644766285 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d588650910 0x55d58865ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.222+0000 7f8f6259c640 1 -- 172.21.15.100:0/2644766285 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d58865b220 con 0x55d58864adb0 2024-05-10T11:19:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.222+0000 7f8f62d9d640 1 --2- 172.21.15.100:0/2644766285 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d588650910 0x55d58865ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:19:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.222+0000 7f8f6259c640 1 --2- 172.21.15.100:0/2644766285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d58864adb0 0x55d588659750 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f8f540a69c0 tx=0x7f8f540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62ba6c38ef45df58 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.222+0000 7f8f49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:19:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.222+0000 7f8f49ffb640 1 -- 172.21.15.100:0/2644766285 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f540c3060 con 0x55d58864adb0 2024-05-10T11:19:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.222+0000 7f8f49ffb640 1 -- 172.21.15.100:0/2644766285 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f8f540a59e0 con 0x55d58864adb0 2024-05-10T11:19:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.222+0000 7f8f6bbb0740 1 -- 172.21.15.100:0/2644766285 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f5c036040 con 0x55d58864adb0 2024-05-10T11:19:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.223+0000 7f8f49ffb640 1 -- 172.21.15.100:0/2644766285 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8f54083e60 con 0x55d58864adb0 2024-05-10T11:19:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.223+0000 7f8f6bbb0740 10 client.15561 fetch_fsmap learned FSMap version 27 2024-05-10T11:19:41.228 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.223+0000 7f8f6bbb0740 1 -- 172.21.15.100:0/2644766285 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d5886cba30 con 0x55d58864adb0 2024-05-10T11:19:41.228 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.223+0000 7f8f49ffb640 1 -- 172.21.15.100:0/2644766285 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8f5c036040 con 0x55d58864adb0 2024-05-10T11:19:41.228 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.223+0000 7f8f49ffb640 1 -- 172.21.15.100:0/2644766285 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f8f540910b0 con 0x55d58864adb0 2024-05-10T11:19:41.228 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.223+0000 7f8f6bbb0740 10 client.15561 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T11:19:41.228 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.223+0000 7f8f6bbb0740 10 client.15561 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:19:41.228 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.223+0000 7f8f6bbb0740 20 client.15561 populate_metadata read hostname 'smithi100' 2024-05-10T11:19:41.228 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.223+0000 7f8f6bbb0740 1 --2- 172.21.15.100:0/2644766285 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d5886cc010 0x55d5886ce400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:41.228 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.223+0000 7f8f6bbb0740 4 client.15561 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:19:41.228 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.223+0000 7f8f6bbb0740 1 -- 172.21.15.100:0/2644766285 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d5885b7050 con 0x55d5886cc010 2024-05-10T11:19:41.228 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.223+0000 7f8f61d9b640 1 --2- 172.21.15.100:0/2644766285 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d5886cc010 0x55d5886ce400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:41.228 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.224+0000 7f8f61d9b640 1 --2- 172.21.15.100:0/2644766285 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d5886cc010 0x55d5886ce400 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1799019c1e45ce0 server_cookie=fe6b17f25839dab4 in_seq=0 out_seq=0 2024-05-10T11:19:41.228 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.224+0000 7f8f49ffb640 10 client.15561 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:19:41.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.228+0000 7f8f49ffb640 1 -- 172.21.15.100:0/2644766285 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d5885b7050 con 0x55d5886cc010 2024-05-10T11:19:41.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.228+0000 7f8f49ffb640 10 client.15561 handle_command_reply: tid=0 2024-05-10T11:19:41.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.228+0000 7f8f6bbb0740 1 client.15561 shutdown 2024-05-10T11:19:41.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.228+0000 7f8f6bbb0740 1 -- 172.21.15.100:0/2644766285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d58864adb0 msgr2=0x55d588659750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:41.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.228+0000 7f8f6bbb0740 1 --2- 172.21.15.100:0/2644766285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d58864adb0 0x55d588659750 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f8f540a69c0 tx=0x7f8f540057f0 comp rx=0 tx=0).stop 2024-05-10T11:19:41.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.228+0000 7f8f6bbb0740 1 -- 172.21.15.100:0/2644766285 shutdown_connections 2024-05-10T11:19:41.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.228+0000 7f8f6bbb0740 1 --2- 172.21.15.100:0/2644766285 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d5886cc010 0x55d5886ce400 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:41.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.228+0000 7f8f6bbb0740 1 --2- 172.21.15.100:0/2644766285 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d588650910 0x55d58865ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:41.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.228+0000 7f8f6bbb0740 1 --2- 172.21.15.100:0/2644766285 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d58864b7a0 0x55d588659c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:41.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.228+0000 7f8f6bbb0740 1 --2- 172.21.15.100:0/2644766285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d58864adb0 0x55d588659750 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:41.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.228+0000 7f8f6bbb0740 1 -- 172.21.15.100:0/2644766285 >> 172.21.15.100:0/2644766285 conn(0x55d588646420 msgr2=0x55d588654c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:41.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.228+0000 7f8f49ffb640 0 client.15561 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:19:41.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.228+0000 7f8f6bbb0740 1 -- 172.21.15.100:0/2644766285 shutdown_connections 2024-05-10T11:19:41.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.228+0000 7f8f6bbb0740 1 -- 172.21.15.100:0/2644766285 wait complete. 2024-05-10T11:19:41.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.228+0000 7f8f6bbb0740 20 client.15561 trim_cache size 0 max 16384 2024-05-10T11:19:41.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:19:41.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:19:41.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:19:41.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:19:41.232 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.229+0000 7f8f68dbe640 1 -- 172.21.15.100:0/522515104 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8f4c0b2920 msgr2=0x7f8f4c0b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:41.232 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.229+0000 7f8f68dbe640 1 --2- 172.21.15.100:0/522515104 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8f4c0b2920 0x7f8f4c0b4de0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c006010 tx=0x7f8f5c031450 comp rx=0 tx=0).stop 2024-05-10T11:19:41.232 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.229+0000 7f8f68dbe640 1 -- 172.21.15.100:0/522515104 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f64101910 msgr2=0x7f8f641979b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:41.232 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.229+0000 7f8f68dbe640 1 --2- 172.21.15.100:0/522515104 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f64101910 0x7f8f641979b0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f8f54010f40 tx=0x7f8f5400c6a0 comp rx=0 tx=0).stop 2024-05-10T11:19:41.232 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.229+0000 7f8f68dbe640 1 -- 172.21.15.100:0/522515104 shutdown_connections 2024-05-10T11:19:41.232 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.229+0000 7f8f68dbe640 1 --2- 172.21.15.100:0/522515104 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8f4c0b2920 0x7f8f4c0b4de0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:41.232 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.229+0000 7f8f68dbe640 1 --2- 172.21.15.100:0/522515104 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f641034c0 0x7f8f6419ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:41.233 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.229+0000 7f8f68dbe640 1 --2- 172.21.15.100:0/522515104 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f64102b00 0x7f8f64197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:41.233 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.229+0000 7f8f68dbe640 1 --2- 172.21.15.100:0/522515104 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f64101910 0x7f8f641979b0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:41.233 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.229+0000 7f8f68dbe640 1 -- 172.21.15.100:0/522515104 >> 172.21.15.100:0/522515104 conn(0x7f8f640fce80 msgr2=0x7f8f64105e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:41.233 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.229+0000 7f8f68dbe640 1 -- 172.21.15.100:0/522515104 shutdown_connections 2024-05-10T11:19:41.233 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:41.229+0000 7f8f68dbe640 1 -- 172.21.15.100:0/522515104 wait complete. 2024-05-10T11:19:41.468 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-10T11:19:41.731 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:19:41 smithi100 ceph-mon[30296]: pgmap v459: 129 pgs: 129 active+clean; 664 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-10T11:19:41.886 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:19:42.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:19:41 smithi178 ceph-mon[33359]: pgmap v459: 129 pgs: 129 active+clean; 664 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-10T11:19:42.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:19:41 smithi188 ceph-mon[34075]: pgmap v459: 129 pgs: 129 active+clean; 664 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-10T11:19:43.789 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:19:43 smithi100 ceph-mon[30296]: pgmap v460: 129 pgs: 129 active+clean; 665 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 511 B/s wr, 8 op/s 2024-05-10T11:19:43.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.859+0000 7f64ac391700 1 -- 172.21.15.100:0/2218084028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64a41023a0 msgr2=0x7f64a4102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:43.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.859+0000 7f64ac391700 1 --2- 172.21.15.100:0/2218084028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64a41023a0 0x7f64a4102820 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f6494009990 tx=0x7f6494009ca0 comp rx=0 tx=0).stop 2024-05-10T11:19:43.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.859+0000 7f64ac391700 1 -- 172.21.15.100:0/2218084028 shutdown_connections 2024-05-10T11:19:43.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.859+0000 7f64ac391700 1 --2- 172.21.15.100:0/2218084028 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64a4102d60 0x7f64a410f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:43.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.859+0000 7f64ac391700 1 --2- 172.21.15.100:0/2218084028 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64a4108410 0x7f64a41087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:43.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.859+0000 7f64ac391700 1 --2- 172.21.15.100:0/2218084028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64a41023a0 0x7f64a4102820 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:43.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.859+0000 7f64ac391700 1 -- 172.21.15.100:0/2218084028 >> 172.21.15.100:0/2218084028 conn(0x7f64a4079350 msgr2=0x7f64a4079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:43.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.859+0000 7f64ac391700 1 -- 172.21.15.100:0/2218084028 shutdown_connections 2024-05-10T11:19:43.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.859+0000 7f64ac391700 1 -- 172.21.15.100:0/2218084028 wait complete. 2024-05-10T11:19:43.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.860+0000 7f64ac391700 1 Processor -- start 2024-05-10T11:19:43.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.860+0000 7f64ac391700 1 -- start start 2024-05-10T11:19:43.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.861+0000 7f64ac391700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64a41023a0 0x7f64a419a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:43.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.861+0000 7f64ac391700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64a4102d60 0x7f64a419ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:43.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.861+0000 7f64ac391700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64a4108410 0x7f64a41a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:43.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.861+0000 7f64ac391700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f64a4112400 con 0x7f64a41023a0 2024-05-10T11:19:43.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.861+0000 7f64ac391700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f64a4112280 con 0x7f64a4102d60 2024-05-10T11:19:43.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.861+0000 7f64ac391700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f64a4112580 con 0x7f64a4108410 2024-05-10T11:19:43.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.861+0000 7f64aa12d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64a41023a0 0x7f64a419a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:43.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.861+0000 7f64aa12d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64a41023a0 0x7f64a419a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60606/0 (socket says 172.21.15.100:60606) 2024-05-10T11:19:43.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.861+0000 7f64aa12d700 1 -- 172.21.15.100:0/2311860384 learned_addr learned my addr 172.21.15.100:0/2311860384 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:43.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.861+0000 7f64a992c700 1 --2- 172.21.15.100:0/2311860384 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64a4102d60 0x7f64a419ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:43.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.861+0000 7f64aa92e700 1 --2- 172.21.15.100:0/2311860384 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64a4108410 0x7f64a41a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:43.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.861+0000 7f64aa12d700 1 -- 172.21.15.100:0/2311860384 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64a4108410 msgr2=0x7f64a41a59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:43.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.861+0000 7f64aa12d700 1 --2- 172.21.15.100:0/2311860384 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64a4108410 0x7f64a41a59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:43.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.861+0000 7f64aa12d700 1 -- 172.21.15.100:0/2311860384 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64a4102d60 msgr2=0x7f64a419ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:43.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.861+0000 7f64aa12d700 1 --2- 172.21.15.100:0/2311860384 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64a4102d60 0x7f64a419ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:43.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.861+0000 7f64aa12d700 1 -- 172.21.15.100:0/2311860384 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64a4198710 con 0x7f64a41023a0 2024-05-10T11:19:43.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.862+0000 7f64aa12d700 1 --2- 172.21.15.100:0/2311860384 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64a41023a0 0x7f64a419a920 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f64a000d980 tx=0x7f64a000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:43.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.862+0000 7f649b7fe700 1 -- 172.21.15.100:0/2311860384 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a0017070 con 0x7f64a41023a0 2024-05-10T11:19:43.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.862+0000 7f649b7fe700 1 -- 172.21.15.100:0/2311860384 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f64a0005020 con 0x7f64a41023a0 2024-05-10T11:19:43.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.862+0000 7f649b7fe700 1 -- 172.21.15.100:0/2311860384 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a000f980 con 0x7f64a41023a0 2024-05-10T11:19:43.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.862+0000 7f64ac391700 1 -- 172.21.15.100:0/2311860384 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64a4198a00 con 0x7f64a41023a0 2024-05-10T11:19:43.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.862+0000 7f64ac391700 1 -- 172.21.15.100:0/2311860384 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64a4198ec0 con 0x7f64a41023a0 2024-05-10T11:19:43.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.867+0000 7f649b7fe700 1 -- 172.21.15.100:0/2311860384 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f64a0004a30 con 0x7f64a41023a0 2024-05-10T11:19:43.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.867+0000 7f64b6c24b80 1 Processor -- start 2024-05-10T11:19:43.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.868+0000 7f64b6c24b80 1 -- start start 2024-05-10T11:19:43.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.868+0000 7f649b7fe700 1 --2- 172.21.15.100:0/2311860384 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f64900b2900 0x7f64900b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:43.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.868+0000 7f64b6c24b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56443b2364d0 0x56443b2368b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:43.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.868+0000 7f64aa92e700 1 --2- 172.21.15.100:0/2311860384 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f64900b2900 0x7f64900b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:43.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.868+0000 7f64b6c24b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56443b236df0 0x56443b23a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:43.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.868+0000 7f64aa12d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56443b2364d0 0x56443b2368b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:43.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.868+0000 7f64aa12d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56443b2364d0 0x56443b2368b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60610/0 (socket says 172.21.15.100:60610) 2024-05-10T11:19:43.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.868+0000 7f649b7fe700 1 -- 172.21.15.100:0/2311860384 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f64a00e2050 con 0x7f64a41023a0 2024-05-10T11:19:43.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.868+0000 7f64aa12d700 1 -- 172.21.15.100:0/886074327 learned_addr learned my addr 172.21.15.100:0/886074327 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:43.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.868+0000 7f64b6c24b80 1 --2- 172.21.15.100:0/886074327 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56443b23ac80 0x56443b23b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:43.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.869+0000 7f64b6c24b80 1 -- 172.21.15.100:0/886074327 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56443b188860 con 0x56443b2364d0 2024-05-10T11:19:43.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.869+0000 7f64b6c24b80 1 -- 172.21.15.100:0/886074327 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56443b1277c0 con 0x56443b23ac80 2024-05-10T11:19:43.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.869+0000 7f64b6c24b80 1 -- 172.21.15.100:0/886074327 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56443b184820 con 0x56443b236df0 2024-05-10T11:19:43.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.869+0000 7f64aa92e700 1 --2- 172.21.15.100:0/2311860384 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f64900b2900 0x7f64900b4dc0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f649c003ee0 tx=0x7f649c009130 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:43.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.869+0000 7f64aa92e700 1 --2- 172.21.15.100:0/886074327 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56443b23ac80 0x56443b23b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:43.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.869+0000 7f64a992c700 1 --2- 172.21.15.100:0/886074327 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56443b236df0 0x56443b23a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:43.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.869+0000 7f64aa12d700 1 -- 172.21.15.100:0/886074327 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56443b236df0 msgr2=0x56443b23a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:43.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.869+0000 7f64aa12d700 1 --2- 172.21.15.100:0/886074327 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56443b236df0 0x56443b23a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:43.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.869+0000 7f64aa12d700 1 -- 172.21.15.100:0/886074327 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56443b23ac80 msgr2=0x56443b23b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:43.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.869+0000 7f64aa12d700 1 --2- 172.21.15.100:0/886074327 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56443b23ac80 0x56443b23b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:43.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.869+0000 7f64aa12d700 1 -- 172.21.15.100:0/886074327 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56443b0d58d0 con 0x56443b2364d0 2024-05-10T11:19:43.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.869+0000 7f64aa12d700 1 --2- 172.21.15.100:0/886074327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56443b2364d0 0x56443b2368b0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f64a00a92a0 tx=0x7f64a0004e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c553711f3d04eadb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:43.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.870+0000 7f6498d70700 1 -- 172.21.15.100:0/886074327 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a00a9310 con 0x56443b2364d0 2024-05-10T11:19:43.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.870+0000 7f6498d70700 1 -- 172.21.15.100:0/886074327 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f64a00ad6c0 con 0x56443b2364d0 2024-05-10T11:19:43.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.870+0000 7f6498d70700 1 -- 172.21.15.100:0/886074327 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a00a9570 con 0x56443b2364d0 2024-05-10T11:19:43.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.871+0000 7f64b6c24b80 1 -- 172.21.15.100:0/886074327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56443b2364d0 msgr2=0x56443b2368b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:43.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.871+0000 7f64b6c24b80 1 --2- 172.21.15.100:0/886074327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56443b2364d0 0x56443b2368b0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f64a00a92a0 tx=0x7f64a0004e50 comp rx=0 tx=0).stop 2024-05-10T11:19:43.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.871+0000 7f64b6c24b80 1 -- 172.21.15.100:0/886074327 shutdown_connections 2024-05-10T11:19:43.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.871+0000 7f64b6c24b80 1 --2- 172.21.15.100:0/886074327 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56443b23ac80 0x56443b23b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:43.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.871+0000 7f64b6c24b80 1 --2- 172.21.15.100:0/886074327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56443b2364d0 0x56443b2368b0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:43.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.871+0000 7f64b6c24b80 1 --2- 172.21.15.100:0/886074327 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56443b236df0 0x56443b23a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:43.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.871+0000 7f64b6c24b80 1 -- 172.21.15.100:0/886074327 >> 172.21.15.100:0/886074327 conn(0x56443b162100 msgr2=0x56443b230c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:43.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.871+0000 7f64b6c24b80 1 -- 172.21.15.100:0/886074327 shutdown_connections 2024-05-10T11:19:43.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.871+0000 7f64b6c24b80 1 -- 172.21.15.100:0/886074327 wait complete. 2024-05-10T11:19:43.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.872+0000 7f64b6c24b80 1 -- start start 2024-05-10T11:19:43.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.872+0000 7f64b6c24b80 1 Processor -- start 2024-05-10T11:19:43.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.872+0000 7f64b6c24b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56443b2364d0 0x56443b241fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:43.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.872+0000 7f64b6c24b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56443b236df0 0x56443b23f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:43.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.872+0000 7f64b6c24b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56443b23ac80 0x56443b23f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:43.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.872+0000 7f64b6c24b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56443b242e30 con 0x56443b23ac80 2024-05-10T11:19:43.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.872+0000 7f64b6c24b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56443b0eec30 con 0x56443b236df0 2024-05-10T11:19:43.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.872+0000 7f64b6c24b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56443b242fb0 con 0x56443b2364d0 2024-05-10T11:19:43.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.872+0000 7f64b6c24b80 10 client.0 _ll_register_callbacks cb 0x56443b120920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:19:43.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.872+0000 7f64aa12d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56443b2364d0 0x56443b241fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:43.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.873+0000 7f64aa12d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56443b2364d0 0x56443b241fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:39086/0 (socket says 172.21.15.100:39086) 2024-05-10T11:19:43.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.873+0000 7f64aa12d700 1 -- 172.21.15.100:0/584335866 learned_addr learned my addr 172.21.15.100:0/584335866 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:43.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.873+0000 7f64aa92e700 1 --2- 172.21.15.100:0/584335866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56443b23ac80 0x56443b23f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:43.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.873+0000 7f64a992c700 1 --2- 172.21.15.100:0/584335866 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56443b236df0 0x56443b23f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:43.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.873+0000 7f64aa12d700 1 -- 172.21.15.100:0/584335866 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56443b236df0 msgr2=0x56443b23f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:43.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.873+0000 7f64aa12d700 1 --2- 172.21.15.100:0/584335866 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56443b236df0 0x56443b23f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:43.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.873+0000 7f64aa12d700 1 -- 172.21.15.100:0/584335866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56443b23ac80 msgr2=0x56443b23f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:43.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.873+0000 7f64aa12d700 1 --2- 172.21.15.100:0/584335866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56443b23ac80 0x56443b23f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:43.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.873+0000 7f64aa12d700 1 -- 172.21.15.100:0/584335866 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56443b124690 con 0x56443b2364d0 2024-05-10T11:19:43.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.873+0000 7f64aa92e700 1 --2- 172.21.15.100:0/584335866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56443b23ac80 0x56443b23f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:19:43.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.873+0000 7f64aa12d700 1 --2- 172.21.15.100:0/584335866 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56443b2364d0 0x56443b241fe0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f64a00df840 tx=0x7f64a0005700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8aedac68c7efa9c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:43.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.873+0000 7f648dffb700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:19:43.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.873+0000 7f648dffb700 1 -- 172.21.15.100:0/584335866 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a00a9780 con 0x56443b2364d0 2024-05-10T11:19:43.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.873+0000 7f648dffb700 1 -- 172.21.15.100:0/584335866 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f64a00b1ce0 con 0x56443b2364d0 2024-05-10T11:19:43.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.873+0000 7f64b6c24b80 1 -- 172.21.15.100:0/584335866 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64a00ad6c0 con 0x56443b2364d0 2024-05-10T11:19:43.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.874+0000 7f648dffb700 1 -- 172.21.15.100:0/584335866 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f64a00b1ee0 con 0x56443b2364d0 2024-05-10T11:19:43.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.874+0000 7f64b6c24b80 10 client.25099 fetch_fsmap learned FSMap version 27 2024-05-10T11:19:43.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.874+0000 7f64b6c24b80 1 -- 172.21.15.100:0/584335866 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56443b242e30 con 0x56443b2364d0 2024-05-10T11:19:43.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.874+0000 7f648dffb700 1 -- 172.21.15.100:0/584335866 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f64a00ca120 con 0x56443b2364d0 2024-05-10T11:19:43.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.874+0000 7f648dffb700 1 -- 172.21.15.100:0/584335866 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f64a0023060 con 0x56443b2364d0 2024-05-10T11:19:43.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.874+0000 7f64b6c24b80 10 client.25099 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T11:19:43.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.874+0000 7f64b6c24b80 10 client.25099 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:19:43.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.874+0000 7f64b6c24b80 20 client.25099 populate_metadata read hostname 'smithi100' 2024-05-10T11:19:43.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.874+0000 7f64b6c24b80 1 --2- 172.21.15.100:0/584335866 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56443b24f6d0 0x56443b251ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:43.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.874+0000 7f64b6c24b80 4 client.25099 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:19:43.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.874+0000 7f64b6c24b80 1 -- 172.21.15.100:0/584335866 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f64a00ad6c0 con 0x56443b24f6d0 2024-05-10T11:19:43.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.874+0000 7f64a992c700 1 --2- 172.21.15.100:0/584335866 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56443b24f6d0 0x56443b251ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:43.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.875+0000 7f648dffb700 10 client.25099 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:19:43.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.875+0000 7f64a992c700 1 --2- 172.21.15.100:0/584335866 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56443b24f6d0 0x56443b251ac0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1eacd076cca18884 server_cookie=ff6757d735d682e9 in_seq=0 out_seq=0 2024-05-10T11:19:43.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.876+0000 7f648dffb700 1 -- 172.21.15.100:0/584335866 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f64a00ad6c0 con 0x56443b24f6d0 2024-05-10T11:19:43.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.876+0000 7f648dffb700 10 client.25099 handle_command_reply: tid=0 2024-05-10T11:19:43.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.876+0000 7f64b6c24b80 1 client.25099 shutdown 2024-05-10T11:19:43.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.876+0000 7f64b6c24b80 1 -- 172.21.15.100:0/584335866 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56443b2364d0 msgr2=0x56443b241fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:43.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.876+0000 7f64b6c24b80 1 --2- 172.21.15.100:0/584335866 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56443b2364d0 0x56443b241fe0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f64a00df840 tx=0x7f64a0005700 comp rx=0 tx=0).stop 2024-05-10T11:19:43.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.876+0000 7f64b6c24b80 1 -- 172.21.15.100:0/584335866 shutdown_connections 2024-05-10T11:19:43.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.876+0000 7f64b6c24b80 1 --2- 172.21.15.100:0/584335866 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56443b24f6d0 0x56443b251ac0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:43.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.876+0000 7f64b6c24b80 1 --2- 172.21.15.100:0/584335866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56443b23ac80 0x56443b23f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:43.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.876+0000 7f64b6c24b80 1 --2- 172.21.15.100:0/584335866 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56443b2364d0 0x56443b241fe0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:43.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.876+0000 7f648dffb700 0 client.25099 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:19:43.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.876+0000 7f64b6c24b80 1 --2- 172.21.15.100:0/584335866 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56443b236df0 0x56443b23f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:43.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.876+0000 7f64b6c24b80 1 -- 172.21.15.100:0/584335866 >> 172.21.15.100:0/584335866 conn(0x56443b162100 msgr2=0x56443b22b720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:43.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.876+0000 7f64b6c24b80 1 -- 172.21.15.100:0/584335866 shutdown_connections 2024-05-10T11:19:43.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.876+0000 7f64b6c24b80 1 -- 172.21.15.100:0/584335866 wait complete. 2024-05-10T11:19:43.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.877+0000 7f64b6c24b80 20 client.25099 trim_cache size 0 max 16384 2024-05-10T11:19:43.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.906+0000 7f64b6c24b80 1 Processor -- start 2024-05-10T11:19:43.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.906+0000 7f64b6c24b80 1 -- start start 2024-05-10T11:19:43.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.906+0000 7f64b6c24b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56443b2364d0 0x56443b2359f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:43.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.906+0000 7f64b6c24b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56443b236df0 0x56443b2385f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:43.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.906+0000 7f64b6c24b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56443b23ac80 0x56443b238b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:43.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.906+0000 7f64b6c24b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f64a41b3bc0 con 0x56443b236df0 2024-05-10T11:19:43.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.906+0000 7f64b6c24b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f64a41b1450 con 0x56443b23ac80 2024-05-10T11:19:43.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.906+0000 7f64b6c24b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f64a41af2a0 con 0x56443b2364d0 2024-05-10T11:19:43.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.906+0000 7f64a992c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56443b236df0 0x56443b2385f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:43.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.906+0000 7f64a992c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56443b236df0 0x56443b2385f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60628/0 (socket says 172.21.15.100:60628) 2024-05-10T11:19:43.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.906+0000 7f64a992c700 1 -- 172.21.15.100:0/3791193043 learned_addr learned my addr 172.21.15.100:0/3791193043 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:43.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.906+0000 7f64aa12d700 1 --2- 172.21.15.100:0/3791193043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56443b2364d0 0x56443b2359f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:43.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.907+0000 7f64aa12d700 1 -- 172.21.15.100:0/3791193043 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56443b23ac80 msgr2=0x56443b238b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:43.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.907+0000 7f64aa12d700 1 --2- 172.21.15.100:0/3791193043 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56443b23ac80 0x56443b238b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:43.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.907+0000 7f64aa12d700 1 -- 172.21.15.100:0/3791193043 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56443b236df0 msgr2=0x56443b2385f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:43.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.907+0000 7f64aa12d700 1 --2- 172.21.15.100:0/3791193043 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56443b236df0 0x56443b2385f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:43.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.907+0000 7f64aa12d700 1 -- 172.21.15.100:0/3791193043 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56443b231850 con 0x56443b2364d0 2024-05-10T11:19:43.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.907+0000 7f64aa12d700 1 --2- 172.21.15.100:0/3791193043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56443b2364d0 0x56443b2359f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f64a00a9010 tx=0x7f64a0004e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7761aaad7ccb457 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:43.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.907+0000 7f648ffff700 1 -- 172.21.15.100:0/3791193043 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a00ad6c0 con 0x56443b2364d0 2024-05-10T11:19:43.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.907+0000 7f648ffff700 1 -- 172.21.15.100:0/3791193043 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f64a00b2170 con 0x56443b2364d0 2024-05-10T11:19:43.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.907+0000 7f648ffff700 1 -- 172.21.15.100:0/3791193043 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a00ca210 con 0x56443b2364d0 2024-05-10T11:19:43.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.908+0000 7f64b6c24b80 1 -- 172.21.15.100:0/3791193043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56443b2364d0 msgr2=0x56443b2359f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:43.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.908+0000 7f64b6c24b80 1 --2- 172.21.15.100:0/3791193043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56443b2364d0 0x56443b2359f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f64a00a9010 tx=0x7f64a0004e50 comp rx=0 tx=0).stop 2024-05-10T11:19:43.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.908+0000 7f64b6c24b80 1 -- 172.21.15.100:0/3791193043 shutdown_connections 2024-05-10T11:19:43.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.908+0000 7f64b6c24b80 1 --2- 172.21.15.100:0/3791193043 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56443b23ac80 0x56443b238b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:43.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.908+0000 7f64b6c24b80 1 --2- 172.21.15.100:0/3791193043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56443b2364d0 0x56443b2359f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:43.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.908+0000 7f64b6c24b80 1 --2- 172.21.15.100:0/3791193043 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56443b236df0 0x56443b2385f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:43.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.908+0000 7f64b6c24b80 1 -- 172.21.15.100:0/3791193043 >> 172.21.15.100:0/3791193043 conn(0x56443b162100 msgr2=0x56443b2352e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:43.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.908+0000 7f64b6c24b80 1 -- 172.21.15.100:0/3791193043 shutdown_connections 2024-05-10T11:19:43.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.908+0000 7f64b6c24b80 1 -- 172.21.15.100:0/3791193043 wait complete. 2024-05-10T11:19:43.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.909+0000 7f64b6c24b80 1 -- start start 2024-05-10T11:19:43.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.909+0000 7f64b6c24b80 1 Processor -- start 2024-05-10T11:19:43.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.909+0000 7f64b6c24b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56443b2364d0 0x56443b2380f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:43.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.909+0000 7f64b6c24b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56443b236df0 0x56443b238630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:43.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.909+0000 7f64b6c24b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56443b23ac80 0x56443b2c1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:43.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.909+0000 7f64a992c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56443b236df0 0x56443b238630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:43.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.909+0000 7f64a992c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56443b236df0 0x56443b238630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60642/0 (socket says 172.21.15.100:60642) 2024-05-10T11:19:43.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.909+0000 7f64a992c700 1 -- 172.21.15.100:0/3109395094 learned_addr learned my addr 172.21.15.100:0/3109395094 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:43.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.909+0000 7f64b6c24b80 1 -- 172.21.15.100:0/3109395094 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f64a41af910 con 0x56443b236df0 2024-05-10T11:19:43.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.909+0000 7f64b6c24b80 1 -- 172.21.15.100:0/3109395094 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f64a41b0d80 con 0x56443b2364d0 2024-05-10T11:19:43.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.910+0000 7f64b6c24b80 1 -- 172.21.15.100:0/3109395094 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f64a41affe0 con 0x56443b23ac80 2024-05-10T11:19:43.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.909+0000 7f64aa12d700 1 --2- 172.21.15.100:0/3109395094 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56443b2364d0 0x56443b2380f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:43.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.910+0000 7f64b6c24b80 10 client.0 _ll_register_callbacks cb 0x56443b120920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:19:43.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.910+0000 7f64aa92e700 1 --2- 172.21.15.100:0/3109395094 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56443b23ac80 0x56443b2c1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:43.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.910+0000 7f64a992c700 1 -- 172.21.15.100:0/3109395094 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56443b23ac80 msgr2=0x56443b2c1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:43.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.910+0000 7f64a992c700 1 --2- 172.21.15.100:0/3109395094 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56443b23ac80 0x56443b2c1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:43.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.910+0000 7f64a992c700 1 -- 172.21.15.100:0/3109395094 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56443b2364d0 msgr2=0x56443b2380f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:43.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.910+0000 7f64a992c700 1 --2- 172.21.15.100:0/3109395094 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56443b2364d0 0x56443b2380f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:43.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.910+0000 7f64a992c700 1 -- 172.21.15.100:0/3109395094 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56443b2c1a80 con 0x56443b236df0 2024-05-10T11:19:43.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.910+0000 7f64a992c700 1 --2- 172.21.15.100:0/3109395094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56443b236df0 0x56443b238630 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f6494006010 tx=0x7f64940051c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c0d127b4c3def38 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:43.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.910+0000 7f648dffb700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:19:43.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.910+0000 7f648dffb700 1 -- 172.21.15.100:0/3109395094 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64940037c0 con 0x56443b236df0 2024-05-10T11:19:43.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.911+0000 7f648dffb700 1 -- 172.21.15.100:0/3109395094 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f6494003960 con 0x56443b236df0 2024-05-10T11:19:43.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.911+0000 7f64b6c24b80 1 -- 172.21.15.100:0/3109395094 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64a00b2170 con 0x56443b236df0 2024-05-10T11:19:43.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.911+0000 7f648dffb700 1 -- 172.21.15.100:0/3109395094 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6494039050 con 0x56443b236df0 2024-05-10T11:19:43.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.911+0000 7f64b6c24b80 10 client.15579 fetch_fsmap learned FSMap version 27 2024-05-10T11:19:43.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.911+0000 7f64b6c24b80 1 -- 172.21.15.100:0/3109395094 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56443b2c2be0 con 0x56443b236df0 2024-05-10T11:19:43.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.911+0000 7f648dffb700 1 -- 172.21.15.100:0/3109395094 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f64a00b2170 con 0x56443b236df0 2024-05-10T11:19:43.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.912+0000 7f648dffb700 1 -- 172.21.15.100:0/3109395094 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f64940281f0 con 0x56443b236df0 2024-05-10T11:19:43.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.912+0000 7f64b6c24b80 10 client.15579 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T11:19:43.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.912+0000 7f64b6c24b80 10 client.15579 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:19:43.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.912+0000 7f64b6c24b80 20 client.15579 populate_metadata read hostname 'smithi100' 2024-05-10T11:19:43.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.912+0000 7f64b6c24b80 1 --2- 172.21.15.100:0/3109395094 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56443b24f6d0 0x56443b2c5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:43.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.912+0000 7f64b6c24b80 4 client.15579 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:19:43.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.912+0000 7f64b6c24b80 1 -- 172.21.15.100:0/3109395094 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56443b202650 con 0x56443b24f6d0 2024-05-10T11:19:43.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.912+0000 7f64aa12d700 1 --2- 172.21.15.100:0/3109395094 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56443b24f6d0 0x56443b2c5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:43.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.914+0000 7f648dffb700 10 client.15579 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:19:43.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.914+0000 7f64aa12d700 1 --2- 172.21.15.100:0/3109395094 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56443b24f6d0 0x56443b2c5360 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79ccf47b965ba361 server_cookie=9fa8b39c7aaf700d in_seq=0 out_seq=0 2024-05-10T11:19:43.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.915+0000 7f648dffb700 1 -- 172.21.15.100:0/3109395094 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56443b202650 con 0x56443b24f6d0 2024-05-10T11:19:43.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.915+0000 7f648dffb700 10 client.15579 handle_command_reply: tid=0 2024-05-10T11:19:43.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.915+0000 7f64b6c24b80 1 client.15579 shutdown 2024-05-10T11:19:43.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.915+0000 7f64b6c24b80 1 -- 172.21.15.100:0/3109395094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56443b236df0 msgr2=0x56443b238630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:43.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.915+0000 7f64b6c24b80 1 --2- 172.21.15.100:0/3109395094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56443b236df0 0x56443b238630 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f6494006010 tx=0x7f64940051c0 comp rx=0 tx=0).stop 2024-05-10T11:19:43.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.915+0000 7f64b6c24b80 1 -- 172.21.15.100:0/3109395094 shutdown_connections 2024-05-10T11:19:43.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.915+0000 7f64b6c24b80 1 --2- 172.21.15.100:0/3109395094 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56443b24f6d0 0x56443b2c5360 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:43.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.915+0000 7f64b6c24b80 1 --2- 172.21.15.100:0/3109395094 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56443b23ac80 0x56443b2c1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:43.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.915+0000 7f64b6c24b80 1 --2- 172.21.15.100:0/3109395094 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56443b2364d0 0x56443b2380f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:43.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.915+0000 7f648dffb700 0 client.15579 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:19:43.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.915+0000 7f64b6c24b80 1 --2- 172.21.15.100:0/3109395094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56443b236df0 0x56443b238630 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:43.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.915+0000 7f64b6c24b80 1 -- 172.21.15.100:0/3109395094 >> 172.21.15.100:0/3109395094 conn(0x56443b162100 msgr2=0x56443b235b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:43.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.915+0000 7f64b6c24b80 1 -- 172.21.15.100:0/3109395094 shutdown_connections 2024-05-10T11:19:43.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.915+0000 7f64b6c24b80 1 -- 172.21.15.100:0/3109395094 wait complete. 2024-05-10T11:19:43.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.915+0000 7f64b6c24b80 20 client.15579 trim_cache size 0 max 16384 2024-05-10T11:19:43.917 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:19:43.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.917+0000 7f64ac391700 1 -- 172.21.15.100:0/2311860384 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f64900b2900 msgr2=0x7f64900b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:43.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.917+0000 7f64ac391700 1 --2- 172.21.15.100:0/2311860384 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f64900b2900 0x7f64900b4dc0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f649c003ee0 tx=0x7f649c009130 comp rx=0 tx=0).stop 2024-05-10T11:19:43.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.917+0000 7f64ac391700 1 -- 172.21.15.100:0/2311860384 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64a41023a0 msgr2=0x7f64a419a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:43.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.917+0000 7f64ac391700 1 --2- 172.21.15.100:0/2311860384 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64a41023a0 0x7f64a419a920 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f64a000d980 tx=0x7f64a000dc90 comp rx=0 tx=0).stop 2024-05-10T11:19:43.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.917+0000 7f64ac391700 1 -- 172.21.15.100:0/2311860384 shutdown_connections 2024-05-10T11:19:43.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.917+0000 7f64ac391700 1 --2- 172.21.15.100:0/2311860384 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f64900b2900 0x7f64900b4dc0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:43.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.917+0000 7f64ac391700 1 --2- 172.21.15.100:0/2311860384 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64a4108410 0x7f64a41a59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:43.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.917+0000 7f64ac391700 1 --2- 172.21.15.100:0/2311860384 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64a41023a0 0x7f64a419a920 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:43.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.917+0000 7f64ac391700 1 --2- 172.21.15.100:0/2311860384 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64a4102d60 0x7f64a419ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:43.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.917+0000 7f64ac391700 1 -- 172.21.15.100:0/2311860384 >> 172.21.15.100:0/2311860384 conn(0x7f64a4079350 msgr2=0x7f64a41010e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:43.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.917+0000 7f64ac391700 1 -- 172.21.15.100:0/2311860384 shutdown_connections 2024-05-10T11:19:43.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:43.917+0000 7f64ac391700 1 -- 172.21.15.100:0/2311860384 wait complete. 2024-05-10T11:19:44.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:19:43 smithi178 ceph-mon[33359]: pgmap v460: 129 pgs: 129 active+clean; 665 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 511 B/s wr, 8 op/s 2024-05-10T11:19:44.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:19:43 smithi188 ceph-mon[34075]: pgmap v460: 129 pgs: 129 active+clean; 665 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 511 B/s wr, 8 op/s 2024-05-10T11:19:45.774 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T11:19:45.775 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T11:19:46.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:19:45 smithi178 ceph-mon[33359]: pgmap v461: 129 pgs: 129 active+clean; 665 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-10T11:19:46.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:19:45 smithi188 ceph-mon[34075]: pgmap v461: 129 pgs: 129 active+clean; 665 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-10T11:19:46.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:19:45 smithi100 ceph-mon[30296]: pgmap v461: 129 pgs: 129 active+clean; 665 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-10T11:19:46.055 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:19:47.821 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:19:47 smithi100 ceph-mon[30296]: pgmap v462: 129 pgs: 129 active+clean; 665 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-10T11:19:47.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.933+0000 7fcebebe3700 1 -- 172.21.15.100:0/2683985936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fceb8102e00 msgr2=0x7fceb81031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:47.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.933+0000 7fcebebe3700 1 --2- 172.21.15.100:0/2683985936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fceb8102e00 0x7fceb81031e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fceac0099c0 tx=0x7fceac009cd0 comp rx=0 tx=0).stop 2024-05-10T11:19:47.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.933+0000 7fcebebe3700 1 -- 172.21.15.100:0/2683985936 shutdown_connections 2024-05-10T11:19:47.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.933+0000 7fcebebe3700 1 --2- 172.21.15.100:0/2683985936 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fceb8107760 0x7fceb810f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:47.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.933+0000 7fcebebe3700 1 --2- 172.21.15.100:0/2683985936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fceb8102e00 0x7fceb81031e0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:47.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.933+0000 7fcebebe3700 1 --2- 172.21.15.100:0/2683985936 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fceb81037b0 0x7fceb8107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:47.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.933+0000 7fcebebe3700 1 -- 172.21.15.100:0/2683985936 >> 172.21.15.100:0/2683985936 conn(0x7fceb8078a10 msgr2=0x7fceb80ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:47.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.933+0000 7fcebebe3700 1 -- 172.21.15.100:0/2683985936 shutdown_connections 2024-05-10T11:19:47.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.933+0000 7fcebebe3700 1 -- 172.21.15.100:0/2683985936 wait complete. 2024-05-10T11:19:47.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.934+0000 7fcebebe3700 1 Processor -- start 2024-05-10T11:19:47.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.934+0000 7fcebebe3700 1 -- start start 2024-05-10T11:19:47.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.934+0000 7fcebebe3700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fceb8102e00 0x7fceb819e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:47.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.934+0000 7fcebebe3700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fceb81037b0 0x7fceb819ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:47.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.934+0000 7fcebebe3700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fceb8107760 0x7fceb8198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:47.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.934+0000 7fcebebe3700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fceb81124b0 con 0x7fceb81037b0 2024-05-10T11:19:47.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.934+0000 7fcebebe3700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fceb8112330 con 0x7fceb8107760 2024-05-10T11:19:47.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.934+0000 7fcebebe3700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fceb8112630 con 0x7fceb8102e00 2024-05-10T11:19:47.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.935+0000 7fcebd180700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fceb8107760 0x7fceb8198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:47.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.935+0000 7fcebc97f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fceb8102e00 0x7fceb819e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:47.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.935+0000 7fcebd180700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fceb8107760 0x7fceb8198770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:57512/0 (socket says 172.21.15.100:57512) 2024-05-10T11:19:47.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.935+0000 7fcebc97f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fceb8102e00 0x7fceb819e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:59440/0 (socket says 172.21.15.100:59440) 2024-05-10T11:19:47.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.935+0000 7fcebd180700 1 -- 172.21.15.100:0/3016325257 learned_addr learned my addr 172.21.15.100:0/3016325257 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:47.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.935+0000 7fceb7fff700 1 --2- 172.21.15.100:0/3016325257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fceb81037b0 0x7fceb819ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:47.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.935+0000 7fcebc97f700 1 -- 172.21.15.100:0/3016325257 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fceb8107760 msgr2=0x7fceb8198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:47.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.935+0000 7fcebc97f700 1 --2- 172.21.15.100:0/3016325257 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fceb8107760 0x7fceb8198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:47.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.935+0000 7fcebc97f700 1 -- 172.21.15.100:0/3016325257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fceb81037b0 msgr2=0x7fceb819ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:47.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.935+0000 7fcebc97f700 1 --2- 172.21.15.100:0/3016325257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fceb81037b0 0x7fceb819ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:47.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.935+0000 7fcebc97f700 1 -- 172.21.15.100:0/3016325257 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceb8199050 con 0x7fceb8102e00 2024-05-10T11:19:47.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.935+0000 7fcebc97f700 1 --2- 172.21.15.100:0/3016325257 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fceb8102e00 0x7fceb819e6a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fceac0098e0 tx=0x7fceac00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:47.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.936+0000 7fceb5ffb700 1 -- 172.21.15.100:0/3016325257 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceac027070 con 0x7fceb8102e00 2024-05-10T11:19:47.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.936+0000 7fcebebe3700 1 -- 172.21.15.100:0/3016325257 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fceb81992e0 con 0x7fceb8102e00 2024-05-10T11:19:47.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.936+0000 7fcebebe3700 1 -- 172.21.15.100:0/3016325257 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fceb8101c60 con 0x7fceb8102e00 2024-05-10T11:19:47.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.936+0000 7fceb5ffb700 1 -- 172.21.15.100:0/3016325257 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fceac022470 con 0x7fceb8102e00 2024-05-10T11:19:47.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.936+0000 7fceb5ffb700 1 -- 172.21.15.100:0/3016325257 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceac017650 con 0x7fceb8102e00 2024-05-10T11:19:47.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.940+0000 7fcec9476b80 1 Processor -- start 2024-05-10T11:19:47.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.940+0000 7fcec9476b80 1 -- start start 2024-05-10T11:19:47.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.940+0000 7fceb5ffb700 1 -- 172.21.15.100:0/3016325257 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fceac02a070 con 0x7fceb8102e00 2024-05-10T11:19:47.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.940+0000 7fcec9476b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d02042270 0x555d02042650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:47.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.940+0000 7fcec9476b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555d0203e890 0x555d0203ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:47.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.940+0000 7fcec9476b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555d0203f250 0x555d02053500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:47.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.940+0000 7fcec9476b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x555d01f909c0 con 0x555d02042270 2024-05-10T11:19:47.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.940+0000 7fcec9476b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x555d01ef5e20 con 0x555d0203e890 2024-05-10T11:19:47.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.940+0000 7fcec9476b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x555d01f8c980 con 0x555d0203f250 2024-05-10T11:19:47.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.940+0000 7fcebd180700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d02042270 0x555d02042650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:47.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.940+0000 7fcebd180700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d02042270 0x555d02042650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56826/0 (socket says 172.21.15.100:56826) 2024-05-10T11:19:47.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.940+0000 7fcebd180700 1 -- 172.21.15.100:0/4013612962 learned_addr learned my addr 172.21.15.100:0/4013612962 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:47.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.941+0000 7fceb7fff700 1 --2- 172.21.15.100:0/4013612962 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555d0203f250 0x555d02053500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:47.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.941+0000 7fcebd180700 1 -- 172.21.15.100:0/4013612962 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555d0203f250 msgr2=0x555d02053500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:47.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.941+0000 7fcebd180700 1 --2- 172.21.15.100:0/4013612962 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555d0203f250 0x555d02053500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:47.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.941+0000 7fcebd180700 1 -- 172.21.15.100:0/4013612962 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555d0203e890 msgr2=0x555d0203ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:47.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.941+0000 7fcebd180700 1 --2- 172.21.15.100:0/4013612962 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555d0203e890 0x555d0203ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:47.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.941+0000 7fcebd180700 1 -- 172.21.15.100:0/4013612962 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d01edd950 con 0x555d02042270 2024-05-10T11:19:47.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.941+0000 7fcebd180700 1 --2- 172.21.15.100:0/4013612962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d02042270 0x555d02042650 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fceb00074a0 tx=0x7fceb00077b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8618d6c27915f0d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:47.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.941+0000 7fcea2ffd700 1 -- 172.21.15.100:0/4013612962 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb0015030 con 0x555d02042270 2024-05-10T11:19:47.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.941+0000 7fcea2ffd700 1 -- 172.21.15.100:0/4013612962 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fceb0011af0 con 0x555d02042270 2024-05-10T11:19:47.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.941+0000 7fcea2ffd700 1 -- 172.21.15.100:0/4013612962 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb0019980 con 0x555d02042270 2024-05-10T11:19:47.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.942+0000 7fceb5ffb700 1 --2- 172.21.15.100:0/3016325257 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fcea80b2880 0x7fcea80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:47.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.942+0000 7fceb5ffb700 1 -- 172.21.15.100:0/3016325257 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fceac0e3290 con 0x7fceb8102e00 2024-05-10T11:19:47.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.942+0000 7fcec9476b80 1 -- 172.21.15.100:0/4013612962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d02042270 msgr2=0x555d02042650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:47.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.942+0000 7fcec9476b80 1 --2- 172.21.15.100:0/4013612962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d02042270 0x555d02042650 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fceb00074a0 tx=0x7fceb00077b0 comp rx=0 tx=0).stop 2024-05-10T11:19:47.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.942+0000 7fcebc97f700 1 --2- 172.21.15.100:0/3016325257 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fcea80b2880 0x7fcea80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:47.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.943+0000 7fcec9476b80 1 -- 172.21.15.100:0/4013612962 shutdown_connections 2024-05-10T11:19:47.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.943+0000 7fcec9476b80 1 --2- 172.21.15.100:0/4013612962 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555d0203f250 0x555d02053500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:47.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.943+0000 7fcec9476b80 1 --2- 172.21.15.100:0/4013612962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d02042270 0x555d02042650 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:47.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.943+0000 7fcec9476b80 1 --2- 172.21.15.100:0/4013612962 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555d0203e890 0x555d0203ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:47.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.943+0000 7fcec9476b80 1 -- 172.21.15.100:0/4013612962 >> 172.21.15.100:0/4013612962 conn(0x555d01f6abf0 msgr2=0x555d02039320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:47.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.943+0000 7fcec9476b80 1 -- 172.21.15.100:0/4013612962 shutdown_connections 2024-05-10T11:19:47.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.943+0000 7fcec9476b80 1 -- 172.21.15.100:0/4013612962 wait complete. 2024-05-10T11:19:47.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.943+0000 7fcebc97f700 1 --2- 172.21.15.100:0/3016325257 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fcea80b2880 0x7fcea80b4d40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fceac0a7850 tx=0x7fceac0096a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:47.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.943+0000 7fcec9476b80 1 -- start start 2024-05-10T11:19:47.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.943+0000 7fcec9476b80 1 Processor -- start 2024-05-10T11:19:47.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.944+0000 7fcec9476b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d0203e890 0x555d02053230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:47.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.944+0000 7fcec9476b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555d0203f250 0x555d020505d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:47.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.944+0000 7fcec9476b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555d02042270 0x555d02050b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:47.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.944+0000 7fcec9476b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x555d02053ce0 con 0x555d0203e890 2024-05-10T11:19:47.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.944+0000 7fcec9476b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x555d01f49f50 con 0x555d0203f250 2024-05-10T11:19:47.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.944+0000 7fcec9476b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x555d02053e60 con 0x555d02042270 2024-05-10T11:19:47.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.944+0000 7fcec9476b80 10 client.0 _ll_register_callbacks cb 0x555d01f24630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:19:47.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.944+0000 7fceb7fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555d0203f250 0x555d020505d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:47.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.944+0000 7fceb7fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555d0203f250 0x555d020505d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:57518/0 (socket says 172.21.15.100:57518) 2024-05-10T11:19:47.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.944+0000 7fceb7fff700 1 -- 172.21.15.100:0/110285502 learned_addr learned my addr 172.21.15.100:0/110285502 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:47.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.944+0000 7fcebd180700 1 --2- 172.21.15.100:0/110285502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d0203e890 0x555d02053230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:47.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.945+0000 7fcebd180700 1 -- 172.21.15.100:0/110285502 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555d02042270 msgr2=0x555d02050b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:47.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.945+0000 7fcebd180700 1 --2- 172.21.15.100:0/110285502 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555d02042270 0x555d02050b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:47.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.945+0000 7fcebd180700 1 -- 172.21.15.100:0/110285502 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555d0203f250 msgr2=0x555d020505d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:47.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.945+0000 7fcebd180700 1 --2- 172.21.15.100:0/110285502 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555d0203f250 0x555d020505d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:47.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.945+0000 7fcebd180700 1 -- 172.21.15.100:0/110285502 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d01f477e0 con 0x555d0203e890 2024-05-10T11:19:47.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.945+0000 7fceb7fff700 1 --2- 172.21.15.100:0/110285502 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555d0203f250 0x555d020505d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:19:47.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.945+0000 7fcebd180700 1 --2- 172.21.15.100:0/110285502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d0203e890 0x555d02053230 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fceb0004c20 tx=0x7fceb00193c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=784d0405af53ff61 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:47.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.945+0000 7fce97fff700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:19:47.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.946+0000 7fce97fff700 1 -- 172.21.15.100:0/110285502 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb0015040 con 0x555d0203e890 2024-05-10T11:19:47.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.946+0000 7fce97fff700 1 -- 172.21.15.100:0/110285502 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fceb0019800 con 0x555d0203e890 2024-05-10T11:19:47.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.946+0000 7fcec9476b80 1 -- 172.21.15.100:0/110285502 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fceb0011af0 con 0x555d0203e890 2024-05-10T11:19:47.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.946+0000 7fce97fff700 1 -- 172.21.15.100:0/110285502 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fceb0011950 con 0x555d0203e890 2024-05-10T11:19:47.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.946+0000 7fcec9476b80 10 client.15591 fetch_fsmap learned FSMap version 27 2024-05-10T11:19:47.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.946+0000 7fcec9476b80 1 -- 172.21.15.100:0/110285502 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555d0205e670 con 0x555d0203e890 2024-05-10T11:19:47.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.946+0000 7fce97fff700 1 -- 172.21.15.100:0/110285502 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fceb0011af0 con 0x555d0203e890 2024-05-10T11:19:47.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.947+0000 7fce97fff700 1 -- 172.21.15.100:0/110285502 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fceb0024aa0 con 0x555d0203e890 2024-05-10T11:19:47.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.947+0000 7fcec9476b80 10 client.15591 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T11:19:47.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.947+0000 7fcec9476b80 10 client.15591 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:19:47.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.947+0000 7fcec9476b80 20 client.15591 populate_metadata read hostname 'smithi100' 2024-05-10T11:19:47.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.947+0000 7fcec9476b80 1 --2- 172.21.15.100:0/110285502 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x555d02060b40 0x555d02062f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:47.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.947+0000 7fcec9476b80 4 client.15591 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:19:47.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.947+0000 7fcec9476b80 1 -- 172.21.15.100:0/110285502 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d01f9e3a0 con 0x555d02060b40 2024-05-10T11:19:47.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.947+0000 7fceb7fff700 1 --2- 172.21.15.100:0/110285502 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x555d02060b40 0x555d02062f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:47.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.948+0000 7fce97fff700 10 client.15591 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:19:47.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.948+0000 7fceb7fff700 1 --2- 172.21.15.100:0/110285502 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x555d02060b40 0x555d02062f30 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=588f4427e94b8d00 server_cookie=eb364228c18ea34 in_seq=0 out_seq=0 2024-05-10T11:19:47.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.948+0000 7fce97fff700 1 -- 172.21.15.100:0/110285502 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555d01f9e3a0 con 0x555d02060b40 2024-05-10T11:19:47.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.948+0000 7fce97fff700 10 client.15591 handle_command_reply: tid=0 2024-05-10T11:19:47.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.949+0000 7fcec9476b80 1 client.15591 shutdown 2024-05-10T11:19:47.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.949+0000 7fcec9476b80 1 -- 172.21.15.100:0/110285502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d0203e890 msgr2=0x555d02053230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:47.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.949+0000 7fcec9476b80 1 --2- 172.21.15.100:0/110285502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d0203e890 0x555d02053230 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fceb0004c20 tx=0x7fceb00193c0 comp rx=0 tx=0).stop 2024-05-10T11:19:47.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.949+0000 7fcec9476b80 1 -- 172.21.15.100:0/110285502 shutdown_connections 2024-05-10T11:19:47.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.949+0000 7fcec9476b80 1 --2- 172.21.15.100:0/110285502 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x555d02060b40 0x555d02062f30 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:47.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.949+0000 7fce97fff700 0 client.15591 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:19:47.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.949+0000 7fcec9476b80 1 --2- 172.21.15.100:0/110285502 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555d02042270 0x555d02050b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:47.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.949+0000 7fcec9476b80 1 --2- 172.21.15.100:0/110285502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d0203e890 0x555d02053230 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:47.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.949+0000 7fcec9476b80 1 --2- 172.21.15.100:0/110285502 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555d0203f250 0x555d020505d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:47.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.949+0000 7fcec9476b80 1 -- 172.21.15.100:0/110285502 >> 172.21.15.100:0/110285502 conn(0x555d01f6abf0 msgr2=0x555d02033bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:47.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.949+0000 7fcec9476b80 1 -- 172.21.15.100:0/110285502 shutdown_connections 2024-05-10T11:19:47.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.949+0000 7fcec9476b80 1 -- 172.21.15.100:0/110285502 wait complete. 2024-05-10T11:19:47.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.949+0000 7fcec9476b80 20 client.15591 trim_cache size 0 max 16384 2024-05-10T11:19:47.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.989+0000 7fcec9476b80 1 Processor -- start 2024-05-10T11:19:47.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.989+0000 7fcec9476b80 1 -- start start 2024-05-10T11:19:47.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.990+0000 7fcec9476b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d0203e890 0x555d02050b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:47.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.990+0000 7fcec9476b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555d0203f250 0x555d02033cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:47.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.990+0000 7fcec9476b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555d02042270 0x555d020684d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:47.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.990+0000 7fcec9476b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fce9c011280 con 0x555d0203e890 2024-05-10T11:19:47.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.990+0000 7fcec9476b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fce9c00eb10 con 0x555d02042270 2024-05-10T11:19:47.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.990+0000 7fcec9476b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fce9c00c960 con 0x555d0203f250 2024-05-10T11:19:47.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.990+0000 7fcebd180700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d0203e890 0x555d02050b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:47.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.990+0000 7fcebd180700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d0203e890 0x555d02050b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56838/0 (socket says 172.21.15.100:56838) 2024-05-10T11:19:47.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.990+0000 7fcebd180700 1 -- 172.21.15.100:0/3733441241 learned_addr learned my addr 172.21.15.100:0/3733441241 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:47.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.990+0000 7fceb7fff700 1 --2- 172.21.15.100:0/3733441241 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555d0203f250 0x555d02033cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:47.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.991+0000 7fcebd180700 1 -- 172.21.15.100:0/3733441241 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555d0203f250 msgr2=0x555d02033cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:47.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.991+0000 7fcebd180700 1 --2- 172.21.15.100:0/3733441241 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555d0203f250 0x555d02033cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:47.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.991+0000 7fcebd180700 1 -- 172.21.15.100:0/3733441241 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555d02042270 msgr2=0x555d020684d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:47.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.991+0000 7fcebd180700 1 --2- 172.21.15.100:0/3733441241 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555d02042270 0x555d020684d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:47.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.991+0000 7fcebd180700 1 -- 172.21.15.100:0/3733441241 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d0203e6c0 con 0x555d0203e890 2024-05-10T11:19:47.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.991+0000 7fcebd180700 1 --2- 172.21.15.100:0/3733441241 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d0203e890 0x555d02050b80 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fceb0024e40 tx=0x7fceb0009910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f224adb9c3843048 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:47.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.991+0000 7fcea27fc700 1 -- 172.21.15.100:0/3733441241 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb0007bb0 con 0x555d0203e890 2024-05-10T11:19:47.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.991+0000 7fcea27fc700 1 -- 172.21.15.100:0/3733441241 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fceb001f070 con 0x555d0203e890 2024-05-10T11:19:47.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.991+0000 7fcea27fc700 1 -- 172.21.15.100:0/3733441241 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb0024470 con 0x555d0203e890 2024-05-10T11:19:47.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.992+0000 7fcec9476b80 1 -- 172.21.15.100:0/3733441241 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d0203e890 msgr2=0x555d02050b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:47.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.992+0000 7fcec9476b80 1 --2- 172.21.15.100:0/3733441241 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d0203e890 0x555d02050b80 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fceb0024e40 tx=0x7fceb0009910 comp rx=0 tx=0).stop 2024-05-10T11:19:47.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.992+0000 7fcec9476b80 1 -- 172.21.15.100:0/3733441241 shutdown_connections 2024-05-10T11:19:47.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.992+0000 7fcec9476b80 1 --2- 172.21.15.100:0/3733441241 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555d02042270 0x555d020684d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:47.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.992+0000 7fcec9476b80 1 --2- 172.21.15.100:0/3733441241 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d0203e890 0x555d02050b80 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:47.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.992+0000 7fcec9476b80 1 --2- 172.21.15.100:0/3733441241 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555d0203f250 0x555d02033cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:47.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.992+0000 7fcec9476b80 1 -- 172.21.15.100:0/3733441241 >> 172.21.15.100:0/3733441241 conn(0x555d01f6abf0 msgr2=0x555d02052510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:47.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.992+0000 7fcec9476b80 1 -- 172.21.15.100:0/3733441241 shutdown_connections 2024-05-10T11:19:47.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.993+0000 7fcec9476b80 1 -- 172.21.15.100:0/3733441241 wait complete. 2024-05-10T11:19:47.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.993+0000 7fcec9476b80 1 -- start start 2024-05-10T11:19:47.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.993+0000 7fcec9476b80 1 Processor -- start 2024-05-10T11:19:47.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.994+0000 7fcec9476b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d0203e890 0x555d020593a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:47.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.994+0000 7fcec9476b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555d0203f250 0x555d02056120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:47.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.994+0000 7fcec9476b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555d02042270 0x555d02056660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:47.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.994+0000 7fcebd180700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d0203e890 0x555d020593a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:47.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.994+0000 7fceb7fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555d0203f250 0x555d02056120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:47.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.994+0000 7fceb7fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555d0203f250 0x555d02056120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:59458/0 (socket says 172.21.15.100:59458) 2024-05-10T11:19:47.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.994+0000 7fcebd180700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d0203e890 0x555d020593a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56844/0 (socket says 172.21.15.100:56844) 2024-05-10T11:19:47.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.994+0000 7fceb7fff700 1 -- 172.21.15.100:0/3152398339 learned_addr learned my addr 172.21.15.100:0/3152398339 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:47.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.994+0000 7fcec9476b80 1 -- 172.21.15.100:0/3152398339 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fce9c00cfd0 con 0x555d0203e890 2024-05-10T11:19:47.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.994+0000 7fcec9476b80 1 -- 172.21.15.100:0/3152398339 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fce9c00e440 con 0x555d02042270 2024-05-10T11:19:47.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.994+0000 7fcec9476b80 1 -- 172.21.15.100:0/3152398339 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fce9c00d6a0 con 0x555d0203f250 2024-05-10T11:19:47.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.994+0000 7fcebd180700 1 --2- 172.21.15.100:0/3152398339 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555d02042270 0x555d02056660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:47.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.994+0000 7fcec9476b80 10 client.0 _ll_register_callbacks cb 0x555d01f24630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:19:47.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.995+0000 7fcebd180700 1 -- 172.21.15.100:0/3152398339 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555d0203f250 msgr2=0x555d02056120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:47.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.995+0000 7fcebd180700 1 --2- 172.21.15.100:0/3152398339 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555d0203f250 0x555d02056120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:47.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.995+0000 7fcebd180700 1 -- 172.21.15.100:0/3152398339 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555d02042270 msgr2=0x555d02056660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:47.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.995+0000 7fcebd180700 1 --2- 172.21.15.100:0/3152398339 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555d02042270 0x555d02056660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:47.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.995+0000 7fcebd180700 1 -- 172.21.15.100:0/3152398339 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d02056ba0 con 0x555d0203e890 2024-05-10T11:19:47.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.995+0000 7fcebd180700 1 --2- 172.21.15.100:0/3152398339 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d0203e890 0x555d020593a0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fceb0014040 tx=0x7fceb0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=621ece34206de3ec server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:47.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.995+0000 7fce97fff700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:19:47.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.995+0000 7fce97fff700 1 -- 172.21.15.100:0/3152398339 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb0040060 con 0x555d0203e890 2024-05-10T11:19:47.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.995+0000 7fce97fff700 1 -- 172.21.15.100:0/3152398339 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fceb0011cf0 con 0x555d0203e890 2024-05-10T11:19:47.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.995+0000 7fcec9476b80 1 -- 172.21.15.100:0/3152398339 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fceb001f070 con 0x555d0203e890 2024-05-10T11:19:47.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.996+0000 7fce97fff700 1 -- 172.21.15.100:0/3152398339 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fceb0015030 con 0x555d0203e890 2024-05-10T11:19:47.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.996+0000 7fcec9476b80 10 client.15597 fetch_fsmap learned FSMap version 27 2024-05-10T11:19:47.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.996+0000 7fcec9476b80 1 -- 172.21.15.100:0/3152398339 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555d020cbda0 con 0x555d0203e890 2024-05-10T11:19:47.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.996+0000 7fce97fff700 1 -- 172.21.15.100:0/3152398339 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fceb001f070 con 0x555d0203e890 2024-05-10T11:19:47.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.996+0000 7fce97fff700 1 -- 172.21.15.100:0/3152398339 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fceb00194e0 con 0x555d0203e890 2024-05-10T11:19:47.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.996+0000 7fcec9476b80 10 client.15597 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T11:19:47.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.996+0000 7fcec9476b80 10 client.15597 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:19:47.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.996+0000 7fcec9476b80 20 client.15597 populate_metadata read hostname 'smithi100' 2024-05-10T11:19:47.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.996+0000 7fcec9476b80 1 --2- 172.21.15.100:0/3152398339 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x555d02060b40 0x555d020ce210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:48.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.996+0000 7fcec9476b80 4 client.15597 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T11:19:48.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.996+0000 7fcec9476b80 1 -- 172.21.15.100:0/3152398339 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555d01f98440 con 0x555d02060b40 2024-05-10T11:19:48.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.997+0000 7fceb7fff700 1 --2- 172.21.15.100:0/3152398339 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x555d02060b40 0x555d020ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:48.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.997+0000 7fce97fff700 10 client.15597 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:19:48.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.998+0000 7fceb7fff700 1 --2- 172.21.15.100:0/3152398339 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x555d02060b40 0x555d020ce210 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad1be335cf570e5e server_cookie=bd780912b15a2808 in_seq=0 out_seq=0 2024-05-10T11:19:48.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.999+0000 7fce97fff700 1 -- 172.21.15.100:0/3152398339 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555d01f98440 con 0x555d02060b40 2024-05-10T11:19:48.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.999+0000 7fce97fff700 10 client.15597 handle_command_reply: tid=0 2024-05-10T11:19:48.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.999+0000 7fcec9476b80 1 client.15597 shutdown 2024-05-10T11:19:48.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.999+0000 7fcec9476b80 1 -- 172.21.15.100:0/3152398339 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d0203e890 msgr2=0x555d020593a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:48.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.999+0000 7fcec9476b80 1 --2- 172.21.15.100:0/3152398339 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d0203e890 0x555d020593a0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fceb0014040 tx=0x7fceb0004ad0 comp rx=0 tx=0).stop 2024-05-10T11:19:48.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.999+0000 7fcec9476b80 1 -- 172.21.15.100:0/3152398339 shutdown_connections 2024-05-10T11:19:48.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.999+0000 7fcec9476b80 1 --2- 172.21.15.100:0/3152398339 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x555d02060b40 0x555d020ce210 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:48.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.999+0000 7fcec9476b80 1 --2- 172.21.15.100:0/3152398339 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555d02042270 0x555d02056660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:48.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.999+0000 7fcec9476b80 1 --2- 172.21.15.100:0/3152398339 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d0203e890 0x555d020593a0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:48.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.999+0000 7fce97fff700 0 client.15597 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:19:48.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.999+0000 7fcec9476b80 1 --2- 172.21.15.100:0/3152398339 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555d0203f250 0x555d02056120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:48.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:47.999+0000 7fcec9476b80 1 -- 172.21.15.100:0/3152398339 >> 172.21.15.100:0/3152398339 conn(0x555d01f6abf0 msgr2=0x555d0204fb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:48.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:48.000+0000 7fcec9476b80 1 -- 172.21.15.100:0/3152398339 shutdown_connections 2024-05-10T11:19:48.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:48.000+0000 7fcec9476b80 1 -- 172.21.15.100:0/3152398339 wait complete. 2024-05-10T11:19:48.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:48.000+0000 7fcec9476b80 20 client.15597 trim_cache size 0 max 16384 2024-05-10T11:19:48.002 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:19:48.002 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-05-10T11:19:48.002 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "a64b4fa4-0bb8-4cef-8bc7-0e9511013978", 2024-05-10T11:19:48.002 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-05-10T11:19:48.002 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:19:48.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:48.001+0000 7fcebebe3700 1 -- 172.21.15.100:0/3016325257 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fcea80b2880 msgr2=0x7fcea80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:48.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:48.001+0000 7fcebebe3700 1 --2- 172.21.15.100:0/3016325257 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fcea80b2880 0x7fcea80b4d40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fceac0a7850 tx=0x7fceac0096a0 comp rx=0 tx=0).stop 2024-05-10T11:19:48.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:48.001+0000 7fcebebe3700 1 -- 172.21.15.100:0/3016325257 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fceb8102e00 msgr2=0x7fceb819e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:48.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:48.001+0000 7fcebebe3700 1 --2- 172.21.15.100:0/3016325257 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fceb8102e00 0x7fceb819e6a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fceac0098e0 tx=0x7fceac00f690 comp rx=0 tx=0).stop 2024-05-10T11:19:48.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:48.001+0000 7fcebebe3700 1 -- 172.21.15.100:0/3016325257 shutdown_connections 2024-05-10T11:19:48.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:48.001+0000 7fcebebe3700 1 --2- 172.21.15.100:0/3016325257 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fcea80b2880 0x7fcea80b4d40 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:48.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:48.001+0000 7fcebebe3700 1 --2- 172.21.15.100:0/3016325257 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fceb8107760 0x7fceb8198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:48.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:48.001+0000 7fcebebe3700 1 --2- 172.21.15.100:0/3016325257 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fceb8102e00 0x7fceb819e6a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:48.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:48.001+0000 7fcebebe3700 1 --2- 172.21.15.100:0/3016325257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fceb81037b0 0x7fceb819ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:48.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:48.001+0000 7fcebebe3700 1 -- 172.21.15.100:0/3016325257 >> 172.21.15.100:0/3016325257 conn(0x7fceb8078a10 msgr2=0x7fceb80ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:48.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:48.001+0000 7fcebebe3700 1 -- 172.21.15.100:0/3016325257 shutdown_connections 2024-05-10T11:19:48.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:48.001+0000 7fcebebe3700 1 -- 172.21.15.100:0/3016325257 wait complete. 2024-05-10T11:19:48.030 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:19:47 smithi188 ceph-mon[34075]: pgmap v462: 129 pgs: 129 active+clean; 665 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-10T11:19:48.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:19:47 smithi178 ceph-mon[33359]: pgmap v462: 129 pgs: 129 active+clean; 665 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-10T11:19:48.761 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:19:48 smithi100 ceph-mon[30296]: scrub queued for path: / 2024-05-10T11:19:49.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:19:48 smithi188 ceph-mon[34075]: scrub queued for path: / 2024-05-10T11:19:49.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:19:48 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:19:49.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:19:48 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:19:49.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:19:48 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:19:49.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:19:48 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:19:49.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:19:48 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:19:49.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:19:48 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:19:49.168 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T11:19:49.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:19:48 smithi178 ceph-mon[33359]: scrub queued for path: / 2024-05-10T11:19:49.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:19:48 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:19:49.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:19:48 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:19:49.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:19:48 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:19:49.534 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:19:50.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:19:49 smithi188 ceph-mon[34075]: pgmap v463: 129 pgs: 129 active+clean; 665 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-10T11:19:50.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:19:49 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:19:50.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:19:49 smithi100 ceph-mon[30296]: pgmap v463: 129 pgs: 129 active+clean; 665 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-10T11:19:50.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:19:49 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:19:50.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:19:49 smithi178 ceph-mon[33359]: pgmap v463: 129 pgs: 129 active+clean; 665 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-10T11:19:50.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:19:49 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:19:52.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:19:51 smithi188 ceph-mon[34075]: pgmap v464: 129 pgs: 129 active+clean; 665 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-10T11:19:52.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:19:51 smithi100 ceph-mon[30296]: pgmap v464: 129 pgs: 129 active+clean; 665 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-10T11:19:52.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:19:51 smithi178 ceph-mon[33359]: pgmap v464: 129 pgs: 129 active+clean; 665 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-10T11:19:53.836 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:19:53 smithi100 ceph-mon[30296]: pgmap v465: 129 pgs: 129 active+clean; 665 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-10T11:19:53.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.847+0000 7fca73530700 1 -- 172.21.15.100:0/715938429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca6c102d90 msgr2=0x7fca6c10f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:53.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.847+0000 7fca73530700 1 --2- 172.21.15.100:0/715938429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca6c102d90 0x7fca6c10f280 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fca6400b0a0 tx=0x7fca6400b3b0 comp rx=0 tx=0).stop 2024-05-10T11:19:53.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.847+0000 7fca73530700 1 -- 172.21.15.100:0/715938429 shutdown_connections 2024-05-10T11:19:53.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.847+0000 7fca73530700 1 --2- 172.21.15.100:0/715938429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca6c102d90 0x7fca6c10f280 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:53.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.847+0000 7fca73530700 1 --2- 172.21.15.100:0/715938429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca6c108440 0x7fca6c108820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:53.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.847+0000 7fca73530700 1 --2- 172.21.15.100:0/715938429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca6c1023d0 0x7fca6c102850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:53.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.847+0000 7fca73530700 1 -- 172.21.15.100:0/715938429 >> 172.21.15.100:0/715938429 conn(0x7fca6c0793e0 msgr2=0x7fca6c0797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:53.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.847+0000 7fca73530700 1 -- 172.21.15.100:0/715938429 shutdown_connections 2024-05-10T11:19:53.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.847+0000 7fca73530700 1 -- 172.21.15.100:0/715938429 wait complete. 2024-05-10T11:19:53.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.848+0000 7fca73530700 1 Processor -- start 2024-05-10T11:19:53.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.848+0000 7fca73530700 1 -- start start 2024-05-10T11:19:53.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.849+0000 7fca73530700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca6c1023d0 0x7fca6c1a5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:53.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.849+0000 7fca73530700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca6c102d90 0x7fca6c1a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:53.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.849+0000 7fca73530700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca6c108440 0x7fca6c10e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:53.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.849+0000 7fca73530700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c113590 con 0x7fca6c102d90 2024-05-10T11:19:53.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.849+0000 7fca73530700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c113410 con 0x7fca6c1023d0 2024-05-10T11:19:53.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.849+0000 7fca73530700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c113710 con 0x7fca6c108440 2024-05-10T11:19:53.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.849+0000 7fca712cc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca6c1023d0 0x7fca6c1a5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:53.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.849+0000 7fca712cc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca6c1023d0 0x7fca6c1a5d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:57552/0 (socket says 172.21.15.100:57552) 2024-05-10T11:19:53.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.849+0000 7fca712cc700 1 -- 172.21.15.100:0/3828359884 learned_addr learned my addr 172.21.15.100:0/3828359884 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:53.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.849+0000 7fca70acb700 1 --2- 172.21.15.100:0/3828359884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca6c102d90 0x7fca6c1a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:53.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.849+0000 7fca71acd700 1 --2- 172.21.15.100:0/3828359884 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca6c108440 0x7fca6c10e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:53.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.849+0000 7fca712cc700 1 -- 172.21.15.100:0/3828359884 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca6c108440 msgr2=0x7fca6c10e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:53.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.849+0000 7fca712cc700 1 --2- 172.21.15.100:0/3828359884 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca6c108440 0x7fca6c10e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:53.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.849+0000 7fca712cc700 1 -- 172.21.15.100:0/3828359884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca6c102d90 msgr2=0x7fca6c1a6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:53.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.849+0000 7fca712cc700 1 --2- 172.21.15.100:0/3828359884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca6c102d90 0x7fca6c1a6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:53.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.849+0000 7fca712cc700 1 -- 172.21.15.100:0/3828359884 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca6c10ecc0 con 0x7fca6c1023d0 2024-05-10T11:19:53.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.849+0000 7fca712cc700 1 --2- 172.21.15.100:0/3828359884 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca6c1023d0 0x7fca6c1a5d10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fca6800d950 tx=0x7fca6800dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:53.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.850+0000 7fca627fc700 1 -- 172.21.15.100:0/3828359884 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca68016070 con 0x7fca6c1023d0 2024-05-10T11:19:53.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.850+0000 7fca627fc700 1 -- 172.21.15.100:0/3828359884 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fca6800fd20 con 0x7fca6c1023d0 2024-05-10T11:19:53.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.850+0000 7fca73530700 1 -- 172.21.15.100:0/3828359884 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca6c10ef50 con 0x7fca6c1023d0 2024-05-10T11:19:53.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.850+0000 7fca627fc700 1 -- 172.21.15.100:0/3828359884 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca6800e3b0 con 0x7fca6c1023d0 2024-05-10T11:19:53.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.850+0000 7fca73530700 1 -- 172.21.15.100:0/3828359884 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca6c1acb70 con 0x7fca6c1023d0 2024-05-10T11:19:53.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.854+0000 7fca7ddc3b80 1 Processor -- start 2024-05-10T11:19:53.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.854+0000 7fca627fc700 1 -- 172.21.15.100:0/3828359884 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fca68019070 con 0x7fca6c1023d0 2024-05-10T11:19:53.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.854+0000 7fca7ddc3b80 1 -- start start 2024-05-10T11:19:53.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.854+0000 7fca7ddc3b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56308b5241b0 0x56308b524590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:53.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.855+0000 7fca7ddc3b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56308b524ad0 0x56308b51e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:53.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.855+0000 7fca7ddc3b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56308b51ed50 0x56308b51f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:53.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.855+0000 7fca7ddc3b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56308b470940 con 0x56308b524ad0 2024-05-10T11:19:53.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.855+0000 7fca7ddc3b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56308b3d5da0 con 0x56308b51ed50 2024-05-10T11:19:53.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.855+0000 7fca7ddc3b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56308b46c900 con 0x56308b5241b0 2024-05-10T11:19:53.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.855+0000 7fca712cc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56308b524ad0 0x56308b51e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:53.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.855+0000 7fca70acb700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56308b51ed50 0x56308b51f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:53.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.855+0000 7fca70acb700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56308b51ed50 0x56308b51f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:57564/0 (socket says 172.21.15.100:57564) 2024-05-10T11:19:53.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.855+0000 7fca712cc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56308b524ad0 0x56308b51e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56864/0 (socket says 172.21.15.100:56864) 2024-05-10T11:19:53.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.855+0000 7fca70acb700 1 -- 172.21.15.100:0/1930906414 learned_addr learned my addr 172.21.15.100:0/1930906414 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:53.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.855+0000 7fca71acd700 1 --2- 172.21.15.100:0/1930906414 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56308b5241b0 0x56308b524590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:53.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.855+0000 7fca70acb700 1 -- 172.21.15.100:0/1930906414 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56308b5241b0 msgr2=0x56308b524590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:53.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.855+0000 7fca70acb700 1 --2- 172.21.15.100:0/1930906414 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56308b5241b0 0x56308b524590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:53.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.855+0000 7fca70acb700 1 -- 172.21.15.100:0/1930906414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56308b524ad0 msgr2=0x56308b51e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:53.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.855+0000 7fca70acb700 1 --2- 172.21.15.100:0/1930906414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56308b524ad0 0x56308b51e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:53.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.855+0000 7fca70acb700 1 -- 172.21.15.100:0/1930906414 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56308b3bd8d0 con 0x56308b51ed50 2024-05-10T11:19:53.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.856+0000 7fca627fc700 1 --2- 172.21.15.100:0/3828359884 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fca580b2980 0x7fca580b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:53.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.856+0000 7fca71acd700 1 --2- 172.21.15.100:0/3828359884 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fca580b2980 0x7fca580b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:53.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.856+0000 7fca627fc700 1 -- 172.21.15.100:0/3828359884 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fca680e1fe0 con 0x7fca6c1023d0 2024-05-10T11:19:53.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.856+0000 7fca70acb700 1 --2- 172.21.15.100:0/1930906414 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56308b51ed50 0x56308b51f190 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fca5c00bcf0 tx=0x7fca5c0103d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37ba799a8821f29f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:53.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.856+0000 7fca577fe700 1 -- 172.21.15.100:0/1930906414 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5c010d10 con 0x56308b51ed50 2024-05-10T11:19:53.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.856+0000 7fca71acd700 1 --2- 172.21.15.100:0/3828359884 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fca580b2980 0x7fca580b4e40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fca6400b830 tx=0x7fca64006210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:53.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.857+0000 7fca577fe700 1 -- 172.21.15.100:0/1930906414 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fca5c0044c0 con 0x56308b51ed50 2024-05-10T11:19:53.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.857+0000 7fca577fe700 1 -- 172.21.15.100:0/1930906414 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5c005280 con 0x56308b51ed50 2024-05-10T11:19:53.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.857+0000 7fca7ddc3b80 1 -- 172.21.15.100:0/1930906414 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56308b51ed50 msgr2=0x56308b51f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:53.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.857+0000 7fca7ddc3b80 1 --2- 172.21.15.100:0/1930906414 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56308b51ed50 0x56308b51f190 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fca5c00bcf0 tx=0x7fca5c0103d0 comp rx=0 tx=0).stop 2024-05-10T11:19:53.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.857+0000 7fca7ddc3b80 1 -- 172.21.15.100:0/1930906414 shutdown_connections 2024-05-10T11:19:53.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.857+0000 7fca7ddc3b80 1 --2- 172.21.15.100:0/1930906414 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56308b51ed50 0x56308b51f190 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:53.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.858+0000 7fca7ddc3b80 1 --2- 172.21.15.100:0/1930906414 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56308b5241b0 0x56308b524590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:53.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.858+0000 7fca7ddc3b80 1 --2- 172.21.15.100:0/1930906414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56308b524ad0 0x56308b51e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:53.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.858+0000 7fca7ddc3b80 1 -- 172.21.15.100:0/1930906414 >> 172.21.15.100:0/1930906414 conn(0x56308b44ab70 msgr2=0x56308b5192a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:53.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.858+0000 7fca7ddc3b80 1 -- 172.21.15.100:0/1930906414 shutdown_connections 2024-05-10T11:19:53.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.858+0000 7fca7ddc3b80 1 -- 172.21.15.100:0/1930906414 wait complete. 2024-05-10T11:19:53.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.858+0000 7fca7ddc3b80 1 -- start start 2024-05-10T11:19:53.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.858+0000 7fca7ddc3b80 1 Processor -- start 2024-05-10T11:19:53.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.858+0000 7fca7ddc3b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56308b51ed50 0x56308b52a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:53.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.858+0000 7fca7ddc3b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56308b5241b0 0x56308b527aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:53.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.859+0000 7fca7ddc3b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56308b524ad0 0x56308b527fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:53.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.859+0000 7fca7ddc3b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56308b52b450 con 0x56308b5241b0 2024-05-10T11:19:53.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.859+0000 7fca7ddc3b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56308b429ed0 con 0x56308b524ad0 2024-05-10T11:19:53.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.859+0000 7fca7ddc3b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56308b52b5d0 con 0x56308b51ed50 2024-05-10T11:19:53.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.859+0000 7fca712cc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56308b51ed50 0x56308b52a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:53.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.859+0000 7fca712cc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56308b51ed50 0x56308b52a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:59486/0 (socket says 172.21.15.100:59486) 2024-05-10T11:19:53.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.859+0000 7fca712cc700 1 -- 172.21.15.100:0/1743993489 learned_addr learned my addr 172.21.15.100:0/1743993489 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:53.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.859+0000 7fca7ddc3b80 10 client.0 _ll_register_callbacks cb 0x56308b4045b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:19:53.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.859+0000 7fca70acb700 1 --2- 172.21.15.100:0/1743993489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56308b5241b0 0x56308b527aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:53.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.859+0000 7fca71acd700 1 --2- 172.21.15.100:0/1743993489 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56308b524ad0 0x56308b527fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:53.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.859+0000 7fca712cc700 1 -- 172.21.15.100:0/1743993489 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56308b524ad0 msgr2=0x56308b527fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:53.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.859+0000 7fca712cc700 1 --2- 172.21.15.100:0/1743993489 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56308b524ad0 0x56308b527fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:53.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.859+0000 7fca712cc700 1 -- 172.21.15.100:0/1743993489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56308b5241b0 msgr2=0x56308b527aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:53.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.859+0000 7fca712cc700 1 --2- 172.21.15.100:0/1743993489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56308b5241b0 0x56308b527aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:53.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.859+0000 7fca712cc700 1 -- 172.21.15.100:0/1743993489 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56308b427760 con 0x56308b51ed50 2024-05-10T11:19:53.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.859+0000 7fca70acb700 1 --2- 172.21.15.100:0/1743993489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56308b5241b0 0x56308b527aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:19:53.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.859+0000 7fca712cc700 1 --2- 172.21.15.100:0/1743993489 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56308b51ed50 0x56308b52a6b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fca680a91d0 tx=0x7fca680a51b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e50f57b11063d7c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:53.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.859+0000 7fca54ff9700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:19:53.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.860+0000 7fca54ff9700 1 -- 172.21.15.100:0/1743993489 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca68020070 con 0x56308b51ed50 2024-05-10T11:19:53.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.860+0000 7fca54ff9700 1 -- 172.21.15.100:0/1743993489 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fca680dfdf0 con 0x56308b51ed50 2024-05-10T11:19:53.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.860+0000 7fca7ddc3b80 1 -- 172.21.15.100:0/1743993489 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca5c0044c0 con 0x56308b51ed50 2024-05-10T11:19:53.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.860+0000 7fca54ff9700 1 -- 172.21.15.100:0/1743993489 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fca680a89b0 con 0x56308b51ed50 2024-05-10T11:19:53.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.860+0000 7fca7ddc3b80 10 client.25129 fetch_fsmap learned FSMap version 27 2024-05-10T11:19:53.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.860+0000 7fca7ddc3b80 1 -- 172.21.15.100:0/1743993489 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56308b535d00 con 0x56308b51ed50 2024-05-10T11:19:53.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.860+0000 7fca54ff9700 1 -- 172.21.15.100:0/1743993489 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fca680a8d50 con 0x56308b51ed50 2024-05-10T11:19:53.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.860+0000 7fca54ff9700 1 -- 172.21.15.100:0/1743993489 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fca680b1ca0 con 0x56308b51ed50 2024-05-10T11:19:53.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.860+0000 7fca7ddc3b80 10 client.25129 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T11:19:53.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.860+0000 7fca7ddc3b80 10 client.25129 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:19:53.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.861+0000 7fca7ddc3b80 20 client.25129 populate_metadata read hostname 'smithi100' 2024-05-10T11:19:53.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.861+0000 7fca7ddc3b80 1 --2- 172.21.15.100:0/1743993489 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56308b5381d0 0x56308b53a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:53.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.861+0000 7fca7ddc3b80 4 client.25129 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:19:53.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.861+0000 7fca7ddc3b80 1 -- 172.21.15.100:0/1743993489 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fca5c0044c0 con 0x56308b5381d0 2024-05-10T11:19:53.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.861+0000 7fca70acb700 1 --2- 172.21.15.100:0/1743993489 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56308b5381d0 0x56308b53a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:53.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.862+0000 7fca54ff9700 10 client.25129 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:19:53.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.862+0000 7fca70acb700 1 --2- 172.21.15.100:0/1743993489 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56308b5381d0 0x56308b53a600 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af09e1da277097c0 server_cookie=ef7610c1a2b7a432 in_seq=0 out_seq=0 2024-05-10T11:19:53.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.862+0000 7fca54ff9700 1 -- 172.21.15.100:0/1743993489 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fca5c0044c0 con 0x56308b5381d0 2024-05-10T11:19:53.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.862+0000 7fca54ff9700 10 client.25129 handle_command_reply: tid=0 2024-05-10T11:19:53.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.862+0000 7fca7ddc3b80 1 client.25129 shutdown 2024-05-10T11:19:53.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.863+0000 7fca7ddc3b80 1 -- 172.21.15.100:0/1743993489 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56308b51ed50 msgr2=0x56308b52a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:53.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.863+0000 7fca7ddc3b80 1 --2- 172.21.15.100:0/1743993489 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56308b51ed50 0x56308b52a6b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fca680a91d0 tx=0x7fca680a51b0 comp rx=0 tx=0).stop 2024-05-10T11:19:53.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.863+0000 7fca7ddc3b80 1 -- 172.21.15.100:0/1743993489 shutdown_connections 2024-05-10T11:19:53.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.863+0000 7fca7ddc3b80 1 --2- 172.21.15.100:0/1743993489 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56308b5381d0 0x56308b53a600 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:53.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.863+0000 7fca7ddc3b80 1 --2- 172.21.15.100:0/1743993489 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56308b524ad0 0x56308b527fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:53.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.863+0000 7fca7ddc3b80 1 --2- 172.21.15.100:0/1743993489 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56308b51ed50 0x56308b52a6b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:53.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.863+0000 7fca54ff9700 0 client.25129 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:19:53.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.863+0000 7fca7ddc3b80 1 --2- 172.21.15.100:0/1743993489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56308b5241b0 0x56308b527aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:53.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.863+0000 7fca7ddc3b80 1 -- 172.21.15.100:0/1743993489 >> 172.21.15.100:0/1743993489 conn(0x56308b44ab70 msgr2=0x56308b513b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:53.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.863+0000 7fca7ddc3b80 1 -- 172.21.15.100:0/1743993489 shutdown_connections 2024-05-10T11:19:53.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.863+0000 7fca7ddc3b80 1 -- 172.21.15.100:0/1743993489 wait complete. 2024-05-10T11:19:53.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.863+0000 7fca7ddc3b80 20 client.25129 trim_cache size 0 max 16384 2024-05-10T11:19:53.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.903+0000 7fca7ddc3b80 1 Processor -- start 2024-05-10T11:19:53.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.903+0000 7fca7ddc3b80 1 -- start start 2024-05-10T11:19:53.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.904+0000 7fca7ddc3b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56308b51ed50 0x56308b51e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:53.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.904+0000 7fca7ddc3b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56308b5241b0 0x56308b534510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:53.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.904+0000 7fca7ddc3b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56308b524ad0 0x56308b513c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:53.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.904+0000 7fca7ddc3b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c1b7ed0 con 0x56308b524ad0 2024-05-10T11:19:53.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.904+0000 7fca7ddc3b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c1b5760 con 0x56308b5241b0 2024-05-10T11:19:53.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.904+0000 7fca7ddc3b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c1b35b0 con 0x56308b51ed50 2024-05-10T11:19:53.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.904+0000 7fca71acd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56308b524ad0 0x56308b513c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:53.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.904+0000 7fca71acd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56308b524ad0 0x56308b513c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56890/0 (socket says 172.21.15.100:56890) 2024-05-10T11:19:53.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.904+0000 7fca71acd700 1 -- 172.21.15.100:0/3496647135 learned_addr learned my addr 172.21.15.100:0/3496647135 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:53.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.904+0000 7fca70acb700 1 --2- 172.21.15.100:0/3496647135 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56308b5241b0 0x56308b534510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:53.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.904+0000 7fca712cc700 1 --2- 172.21.15.100:0/3496647135 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56308b51ed50 0x56308b51e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:53.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.904+0000 7fca71acd700 1 -- 172.21.15.100:0/3496647135 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56308b51ed50 msgr2=0x56308b51e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:53.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.904+0000 7fca71acd700 1 --2- 172.21.15.100:0/3496647135 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56308b51ed50 0x56308b51e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:53.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.904+0000 7fca71acd700 1 -- 172.21.15.100:0/3496647135 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56308b5241b0 msgr2=0x56308b534510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:53.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.904+0000 7fca71acd700 1 --2- 172.21.15.100:0/3496647135 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56308b5241b0 0x56308b534510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:53.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.904+0000 7fca71acd700 1 -- 172.21.15.100:0/3496647135 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56308b52a2f0 con 0x56308b524ad0 2024-05-10T11:19:53.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.905+0000 7fca70acb700 1 --2- 172.21.15.100:0/3496647135 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56308b5241b0 0x56308b534510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:19:53.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.905+0000 7fca71acd700 1 --2- 172.21.15.100:0/3496647135 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56308b524ad0 0x56308b513c40 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fca6400b070 tx=0x7fca6401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eee3f510344fcdcd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:53.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.905+0000 7fca56ffd700 1 -- 172.21.15.100:0/3496647135 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca64007820 con 0x56308b524ad0 2024-05-10T11:19:53.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.905+0000 7fca56ffd700 1 -- 172.21.15.100:0/3496647135 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fca64005040 con 0x56308b524ad0 2024-05-10T11:19:53.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.905+0000 7fca56ffd700 1 -- 172.21.15.100:0/3496647135 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca6402a680 con 0x56308b524ad0 2024-05-10T11:19:53.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.906+0000 7fca7ddc3b80 1 -- 172.21.15.100:0/3496647135 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56308b524ad0 msgr2=0x56308b513c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:53.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.906+0000 7fca7ddc3b80 1 --2- 172.21.15.100:0/3496647135 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56308b524ad0 0x56308b513c40 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fca6400b070 tx=0x7fca6401f000 comp rx=0 tx=0).stop 2024-05-10T11:19:53.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.906+0000 7fca7ddc3b80 1 -- 172.21.15.100:0/3496647135 shutdown_connections 2024-05-10T11:19:53.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.906+0000 7fca7ddc3b80 1 --2- 172.21.15.100:0/3496647135 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56308b524ad0 0x56308b513c40 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:53.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.906+0000 7fca7ddc3b80 1 --2- 172.21.15.100:0/3496647135 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56308b51ed50 0x56308b51e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:53.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.906+0000 7fca7ddc3b80 1 --2- 172.21.15.100:0/3496647135 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56308b5241b0 0x56308b534510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:53.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.906+0000 7fca7ddc3b80 1 -- 172.21.15.100:0/3496647135 >> 172.21.15.100:0/3496647135 conn(0x56308b44ab70 msgr2=0x56308b527350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:53.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.906+0000 7fca7ddc3b80 1 -- 172.21.15.100:0/3496647135 shutdown_connections 2024-05-10T11:19:53.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.906+0000 7fca7ddc3b80 1 -- 172.21.15.100:0/3496647135 wait complete. 2024-05-10T11:19:53.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.906+0000 7fca7ddc3b80 1 -- start start 2024-05-10T11:19:53.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.907+0000 7fca7ddc3b80 1 Processor -- start 2024-05-10T11:19:53.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.907+0000 7fca7ddc3b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56308b51ed50 0x56308b53fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:53.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.907+0000 7fca712cc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56308b51ed50 0x56308b53fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:53.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.907+0000 7fca712cc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56308b51ed50 0x56308b53fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56896/0 (socket says 172.21.15.100:56896) 2024-05-10T11:19:53.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.907+0000 7fca712cc700 1 -- 172.21.15.100:0/4293189764 learned_addr learned my addr 172.21.15.100:0/4293189764 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:53.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.907+0000 7fca7ddc3b80 1 --2- 172.21.15.100:0/4293189764 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56308b5241b0 0x56308b540010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:53.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.907+0000 7fca7ddc3b80 1 --2- 172.21.15.100:0/4293189764 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56308b524ad0 0x56308b5a1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:53.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.908+0000 7fca7ddc3b80 1 -- 172.21.15.100:0/4293189764 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c1b3c20 con 0x56308b51ed50 2024-05-10T11:19:53.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.908+0000 7fca7ddc3b80 1 -- 172.21.15.100:0/4293189764 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c1b5090 con 0x56308b5241b0 2024-05-10T11:19:53.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.908+0000 7fca7ddc3b80 1 -- 172.21.15.100:0/4293189764 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c1b42f0 con 0x56308b524ad0 2024-05-10T11:19:53.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.908+0000 7fca70acb700 1 --2- 172.21.15.100:0/4293189764 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56308b5241b0 0x56308b540010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:53.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.908+0000 7fca712cc700 1 -- 172.21.15.100:0/4293189764 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56308b524ad0 msgr2=0x56308b5a1a80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T11:19:53.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.908+0000 7fca712cc700 1 --2- 172.21.15.100:0/4293189764 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56308b524ad0 0x56308b5a1a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:53.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.908+0000 7fca712cc700 1 -- 172.21.15.100:0/4293189764 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56308b5241b0 msgr2=0x56308b540010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:53.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.908+0000 7fca712cc700 1 --2- 172.21.15.100:0/4293189764 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56308b5241b0 0x56308b540010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:53.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.908+0000 7fca712cc700 1 -- 172.21.15.100:0/4293189764 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56308b5a1fc0 con 0x56308b51ed50 2024-05-10T11:19:53.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.908+0000 7fca7ddc3b80 10 client.0 _ll_register_callbacks cb 0x56308b4045b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:19:53.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.908+0000 7fca712cc700 1 --2- 172.21.15.100:0/4293189764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56308b51ed50 0x56308b53fad0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fca68010910 tx=0x7fca680a8cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a77692c8157f23a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:53.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.908+0000 7fca54ff9700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:19:53.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.908+0000 7fca7ddc3b80 1 -- 172.21.15.100:0/4293189764 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca64005040 con 0x56308b51ed50 2024-05-10T11:19:53.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.909+0000 7fca54ff9700 1 -- 172.21.15.100:0/4293189764 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca68020040 con 0x56308b51ed50 2024-05-10T11:19:53.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.909+0000 7fca54ff9700 1 -- 172.21.15.100:0/4293189764 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fca680e50f0 con 0x56308b51ed50 2024-05-10T11:19:53.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.909+0000 7fca54ff9700 1 -- 172.21.15.100:0/4293189764 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fca6800fed0 con 0x56308b51ed50 2024-05-10T11:19:53.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.909+0000 7fca7ddc3b80 10 client.15615 fetch_fsmap learned FSMap version 27 2024-05-10T11:19:53.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.909+0000 7fca7ddc3b80 1 -- 172.21.15.100:0/4293189764 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({fsmap=27}) -- 0x56308b5a2fe0 con 0x56308b51ed50 2024-05-10T11:19:53.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.909+0000 7fca54ff9700 1 -- 172.21.15.100:0/4293189764 <== mon.0 v2:172.21.15.100:3300/0 4 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fca6800f890 con 0x56308b51ed50 2024-05-10T11:19:53.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.909+0000 7fca7ddc3b80 10 client.15615 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T11:19:53.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.909+0000 7fca7ddc3b80 10 client.15615 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:19:53.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.909+0000 7fca7ddc3b80 20 client.15615 populate_metadata read hostname 'smithi100' 2024-05-10T11:19:53.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.909+0000 7fca7ddc3b80 1 --2- 172.21.15.100:0/4293189764 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56308b5a3590 0x56308b5a59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:53.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.909+0000 7fca7ddc3b80 4 client.15615 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T11:19:53.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.909+0000 7fca7ddc3b80 1 -- 172.21.15.100:0/4293189764 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56308b47e320 con 0x56308b5a3590 2024-05-10T11:19:53.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.910+0000 7fca70acb700 1 --2- 172.21.15.100:0/4293189764 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56308b5a3590 0x56308b5a59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:53.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.910+0000 7fca70acb700 1 --2- 172.21.15.100:0/4293189764 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56308b5a3590 0x56308b5a59f0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23970a7c9407654b server_cookie=218b176d5f323236 in_seq=0 out_seq=0 2024-05-10T11:19:53.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.910+0000 7fca54ff9700 10 client.15615 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:19:53.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.910+0000 7fca54ff9700 1 -- 172.21.15.100:0/4293189764 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56308b47e320 con 0x56308b5a3590 2024-05-10T11:19:53.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.910+0000 7fca54ff9700 10 client.15615 handle_command_reply: tid=0 2024-05-10T11:19:53.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.910+0000 7fca7ddc3b80 1 client.15615 shutdown 2024-05-10T11:19:53.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.910+0000 7fca7ddc3b80 1 -- 172.21.15.100:0/4293189764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56308b51ed50 msgr2=0x56308b53fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:53.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.910+0000 7fca7ddc3b80 1 --2- 172.21.15.100:0/4293189764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56308b51ed50 0x56308b53fad0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fca68010910 tx=0x7fca680a8cc0 comp rx=0 tx=0).stop 2024-05-10T11:19:53.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.911+0000 7fca7ddc3b80 1 -- 172.21.15.100:0/4293189764 shutdown_connections 2024-05-10T11:19:53.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.911+0000 7fca7ddc3b80 1 --2- 172.21.15.100:0/4293189764 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56308b5a3590 0x56308b5a59f0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:53.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.911+0000 7fca7ddc3b80 1 --2- 172.21.15.100:0/4293189764 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56308b524ad0 0x56308b5a1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:53.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.911+0000 7fca7ddc3b80 1 --2- 172.21.15.100:0/4293189764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56308b51ed50 0x56308b53fad0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:53.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.911+0000 7fca7ddc3b80 1 --2- 172.21.15.100:0/4293189764 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56308b5241b0 0x56308b540010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:53.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.911+0000 7fca7ddc3b80 1 -- 172.21.15.100:0/4293189764 >> 172.21.15.100:0/4293189764 conn(0x56308b44ab70 msgr2=0x56308b5298e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:53.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.911+0000 7fca54ff9700 0 client.15615 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:19:53.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.911+0000 7fca7ddc3b80 1 -- 172.21.15.100:0/4293189764 shutdown_connections 2024-05-10T11:19:53.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.911+0000 7fca7ddc3b80 1 -- 172.21.15.100:0/4293189764 wait complete. 2024-05-10T11:19:53.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.911+0000 7fca7ddc3b80 20 client.15615 trim_cache size 0 max 16384 2024-05-10T11:19:53.915 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:19:53.915 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-05-10T11:19:53.916 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-05-10T11:19:53.916 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:19:53.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.912+0000 7fca73530700 1 -- 172.21.15.100:0/3828359884 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fca580b2980 msgr2=0x7fca580b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:53.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.912+0000 7fca73530700 1 --2- 172.21.15.100:0/3828359884 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fca580b2980 0x7fca580b4e40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fca6400b830 tx=0x7fca64006210 comp rx=0 tx=0).stop 2024-05-10T11:19:53.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.912+0000 7fca73530700 1 -- 172.21.15.100:0/3828359884 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca6c1023d0 msgr2=0x7fca6c1a5d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:53.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.912+0000 7fca73530700 1 --2- 172.21.15.100:0/3828359884 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca6c1023d0 0x7fca6c1a5d10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fca6800d950 tx=0x7fca6800dc60 comp rx=0 tx=0).stop 2024-05-10T11:19:53.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.912+0000 7fca73530700 1 -- 172.21.15.100:0/3828359884 shutdown_connections 2024-05-10T11:19:53.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.912+0000 7fca73530700 1 --2- 172.21.15.100:0/3828359884 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fca580b2980 0x7fca580b4e40 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:53.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.912+0000 7fca73530700 1 --2- 172.21.15.100:0/3828359884 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca6c108440 0x7fca6c10e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:53.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.912+0000 7fca73530700 1 --2- 172.21.15.100:0/3828359884 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca6c1023d0 0x7fca6c1a5d10 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:53.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.912+0000 7fca73530700 1 --2- 172.21.15.100:0/3828359884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca6c102d90 0x7fca6c1a6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:53.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.912+0000 7fca73530700 1 -- 172.21.15.100:0/3828359884 >> 172.21.15.100:0/3828359884 conn(0x7fca6c0793e0 msgr2=0x7fca6c101080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:53.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.913+0000 7fca73530700 1 -- 172.21.15.100:0/3828359884 shutdown_connections 2024-05-10T11:19:53.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:53.913+0000 7fca73530700 1 -- 172.21.15.100:0/3828359884 wait complete. 2024-05-10T11:19:54.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:19:53 smithi188 ceph-mon[34075]: pgmap v465: 129 pgs: 129 active+clean; 665 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-10T11:19:54.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:19:53 smithi178 ceph-mon[33359]: pgmap v465: 129 pgs: 129 active+clean; 665 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-10T11:19:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.363+0000 7ff4ab637640 1 -- 172.21.15.100:0/1880165009 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4a4101910 msgr2=0x7ff4a4101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.363+0000 7ff4ab637640 1 --2- 172.21.15.100:0/1880165009 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4a4101910 0x7ff4a4101d30 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0009900 tx=0x7ff4a002a410 comp rx=0 tx=0).stop 2024-05-10T11:19:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.363+0000 7ff4ab637640 1 -- 172.21.15.100:0/1880165009 shutdown_connections 2024-05-10T11:19:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.363+0000 7ff4ab637640 1 --2- 172.21.15.100:0/1880165009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4a41034c0 0x7ff4a4109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.363+0000 7ff4ab637640 1 --2- 172.21.15.100:0/1880165009 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff4a4102b00 0x7ff4a4102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.363+0000 7ff4ab637640 1 --2- 172.21.15.100:0/1880165009 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4a4101910 0x7ff4a4101d30 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.363+0000 7ff4ab637640 1 -- 172.21.15.100:0/1880165009 >> 172.21.15.100:0/1880165009 conn(0x7ff4a40fce80 msgr2=0x7ff4a40ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.363+0000 7ff4ab637640 1 -- 172.21.15.100:0/1880165009 shutdown_connections 2024-05-10T11:19:54.365 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.364+0000 7ff4ab637640 1 -- 172.21.15.100:0/1880165009 wait complete. 2024-05-10T11:19:54.367 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.365+0000 7ff4ab637640 1 Processor -- start 2024-05-10T11:19:54.367 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.365+0000 7ff4ab637640 1 -- start start 2024-05-10T11:19:54.367 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.365+0000 7ff4ab637640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4a4101910 0x7ff4a41977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:54.367 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.365+0000 7ff4ab637640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff4a4102b00 0x7ff4a4197ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:54.367 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.365+0000 7ff4ab637640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4a41034c0 0x7ff4a419ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:54.367 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.365+0000 7ff4ab637640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a410d310 con 0x7ff4a41034c0 2024-05-10T11:19:54.367 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.365+0000 7ff4ab637640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a410d190 con 0x7ff4a4101910 2024-05-10T11:19:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.365+0000 7ff4a9bd4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4a41034c0 0x7ff4a419ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.366+0000 7ff4a9bd4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4a41034c0 0x7ff4a419ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56916/0 (socket says 172.21.15.100:56916) 2024-05-10T11:19:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.366+0000 7ff4a93d3640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4a4101910 0x7ff4a41977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.366+0000 7ff4a9bd4640 1 -- 172.21.15.100:0/1871833979 learned_addr learned my addr 172.21.15.100:0/1871833979 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.366+0000 7ff4ab637640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a410d490 con 0x7ff4a4102b00 2024-05-10T11:19:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.366+0000 7ff4a8bd2640 1 --2- 172.21.15.100:0/1871833979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff4a4102b00 0x7ff4a4197ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.366+0000 7ff4a9bd4640 1 -- 172.21.15.100:0/1871833979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff4a4102b00 msgr2=0x7ff4a4197ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.366+0000 7ff4a9bd4640 1 --2- 172.21.15.100:0/1871833979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff4a4102b00 0x7ff4a4197ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.366+0000 7ff4a9bd4640 1 -- 172.21.15.100:0/1871833979 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4a4101910 msgr2=0x7ff4a41977a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.366+0000 7ff4a9bd4640 1 --2- 172.21.15.100:0/1871833979 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4a4101910 0x7ff4a41977a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.366+0000 7ff4a9bd4640 1 -- 172.21.15.100:0/1871833979 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4a419f3c0 con 0x7ff4a41034c0 2024-05-10T11:19:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.366+0000 7ff4a9bd4640 1 --2- 172.21.15.100:0/1871833979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4a41034c0 0x7ff4a419ed50 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ff49c00eaa0 tx=0x7ff49c00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.367+0000 7ff49a7fc640 1 -- 172.21.15.100:0/1871833979 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff49c004d20 con 0x7ff4a41034c0 2024-05-10T11:19:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.367+0000 7ff4ab637640 1 -- 172.21.15.100:0/1871833979 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4a419f6b0 con 0x7ff4a41034c0 2024-05-10T11:19:54.372 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.367+0000 7ff49a7fc640 1 -- 172.21.15.100:0/1871833979 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff49c005020 con 0x7ff4a41034c0 2024-05-10T11:19:54.372 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.367+0000 7ff49a7fc640 1 -- 172.21.15.100:0/1871833979 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff49c0116d0 con 0x7ff4a41034c0 2024-05-10T11:19:54.372 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.367+0000 7ff4ab637640 1 -- 172.21.15.100:0/1871833979 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4a419fc00 con 0x7ff4a41034c0 2024-05-10T11:19:54.376 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.370+0000 7ff4ae42f740 1 Processor -- start 2024-05-10T11:19:54.377 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.371+0000 7ff49a7fc640 1 -- 172.21.15.100:0/1871833979 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7ff49c0051d0 con 0x7ff4a41034c0 2024-05-10T11:19:54.377 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.371+0000 7ff4ae42f740 1 -- start start 2024-05-10T11:19:54.377 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.371+0000 7ff4ae42f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a7cf9cdb0 0x556a7cf9d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:54.377 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.371+0000 7ff4ae42f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a7cf9d7a0 0x556a7cfa22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:54.377 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.371+0000 7ff4ae42f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a7cfa2910 0x556a7cfa4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:54.377 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.371+0000 7ff4ae42f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x556a7cf015d0 con 0x556a7cfa2910 2024-05-10T11:19:54.377 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.371+0000 7ff4ae42f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556a7ceba890 con 0x556a7cf9d7a0 2024-05-10T11:19:54.377 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.371+0000 7ff4ae42f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556a7cefd590 con 0x556a7cf9cdb0 2024-05-10T11:19:54.377 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.371+0000 7ff4a8bd2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a7cfa2910 0x556a7cfa4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:54.377 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.371+0000 7ff4a8bd2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a7cfa2910 0x556a7cfa4da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56922/0 (socket says 172.21.15.100:56922) 2024-05-10T11:19:54.378 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.371+0000 7ff4a8bd2640 1 -- 172.21.15.100:0/1256217218 learned_addr learned my addr 172.21.15.100:0/1256217218 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:54.378 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.372+0000 7ff4a93d3640 1 --2- 172.21.15.100:0/1256217218 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a7cf9d7a0 0x556a7cfa22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:54.378 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.372+0000 7ff4a8bd2640 1 --2- 172.21.15.100:0/1256217218 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a7cf9cdb0 0x556a7cf9d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:54.378 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.372+0000 7ff4a8bd2640 1 -- 172.21.15.100:0/1256217218 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a7cf9cdb0 msgr2=0x556a7cf9d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:54.378 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.372+0000 7ff4a8bd2640 1 --2- 172.21.15.100:0/1256217218 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a7cf9cdb0 0x556a7cf9d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:54.378 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.372+0000 7ff4a8bd2640 1 -- 172.21.15.100:0/1256217218 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a7cf9d7a0 msgr2=0x556a7cfa22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:54.378 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.372+0000 7ff4a8bd2640 1 --2- 172.21.15.100:0/1256217218 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a7cf9d7a0 0x556a7cfa22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:54.378 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.372+0000 7ff4a8bd2640 1 -- 172.21.15.100:0/1256217218 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a7cc13210 con 0x556a7cfa2910 2024-05-10T11:19:54.378 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.372+0000 7ff4a8bd2640 1 --2- 172.21.15.100:0/1256217218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a7cfa2910 0x556a7cfa4da0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7ff49400a5c0 tx=0x7ff49400a810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1cdefa7872d649d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:54.378 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.372+0000 7ff48f7fe640 1 -- 172.21.15.100:0/1256217218 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff49401f070 con 0x556a7cfa2910 2024-05-10T11:19:54.378 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.372+0000 7ff48f7fe640 1 -- 172.21.15.100:0/1256217218 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff4940047a0 con 0x556a7cfa2910 2024-05-10T11:19:54.379 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.372+0000 7ff48f7fe640 1 -- 172.21.15.100:0/1256217218 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff49400f980 con 0x556a7cfa2910 2024-05-10T11:19:54.379 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.373+0000 7ff49a7fc640 1 --2- 172.21.15.100:0/1871833979 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff4900b2920 0x7ff4900b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:54.379 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.373+0000 7ff4a93d3640 1 --2- 172.21.15.100:0/1871833979 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff4900b2920 0x7ff4900b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:54.379 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.373+0000 7ff4ae42f740 1 -- 172.21.15.100:0/1256217218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a7cfa2910 msgr2=0x556a7cfa4da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:54.379 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.373+0000 7ff4ae42f740 1 --2- 172.21.15.100:0/1256217218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a7cfa2910 0x556a7cfa4da0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7ff49400a5c0 tx=0x7ff49400a810 comp rx=0 tx=0).stop 2024-05-10T11:19:54.379 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.373+0000 7ff49a7fc640 1 -- 172.21.15.100:0/1871833979 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7ff49c0e2300 con 0x7ff4a41034c0 2024-05-10T11:19:54.379 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.373+0000 7ff4a93d3640 1 --2- 172.21.15.100:0/1871833979 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff4900b2920 0x7ff4900b4de0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0009850 tx=0x7ff4a0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:54.379 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.374+0000 7ff4ae42f740 1 -- 172.21.15.100:0/1256217218 shutdown_connections 2024-05-10T11:19:54.379 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.374+0000 7ff4ae42f740 1 --2- 172.21.15.100:0/1256217218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a7cfa2910 0x556a7cfa4da0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:54.379 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.374+0000 7ff4ae42f740 1 --2- 172.21.15.100:0/1256217218 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a7cf9d7a0 0x556a7cfa22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:54.380 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.374+0000 7ff4ae42f740 1 --2- 172.21.15.100:0/1256217218 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a7cf9cdb0 0x556a7cf9d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:54.380 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.374+0000 7ff4ae42f740 1 -- 172.21.15.100:0/1256217218 >> 172.21.15.100:0/1256217218 conn(0x556a7cf98420 msgr2=0x556a7cf98820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:54.380 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.374+0000 7ff4ae42f740 1 -- 172.21.15.100:0/1256217218 shutdown_connections 2024-05-10T11:19:54.380 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.374+0000 7ff4ae42f740 1 -- 172.21.15.100:0/1256217218 wait complete. 2024-05-10T11:19:54.380 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.374+0000 7ff4ae42f740 1 -- start start 2024-05-10T11:19:54.380 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.374+0000 7ff4ae42f740 1 Processor -- start 2024-05-10T11:19:54.380 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.375+0000 7ff4ae42f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a7cf9cdb0 0x556a7cfa7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:54.380 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.375+0000 7ff4ae42f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a7cf9d7a0 0x556a7cfa75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:54.380 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.375+0000 7ff4ae42f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a7cfa2910 0x556a7cfae610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:54.380 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.375+0000 7ff4ae42f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x556a7cfa58c0 con 0x556a7cfa2910 2024-05-10T11:19:54.380 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.375+0000 7ff4a9bd4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a7cfa2910 0x556a7cfae610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:54.381 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.375+0000 7ff4a9bd4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a7cfa2910 0x556a7cfae610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56924/0 (socket says 172.21.15.100:56924) 2024-05-10T11:19:54.381 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.375+0000 7ff4a9bd4640 1 -- 172.21.15.100:0/2572223862 learned_addr learned my addr 172.21.15.100:0/2572223862 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:54.381 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.375+0000 7ff4ae42f740 1 -- 172.21.15.100:0/2572223862 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556a7cebe370 con 0x556a7cf9d7a0 2024-05-10T11:19:54.381 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.375+0000 7ff4ae42f740 1 -- 172.21.15.100:0/2572223862 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556a7cfa5a40 con 0x556a7cf9cdb0 2024-05-10T11:19:54.381 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.375+0000 7ff4a93d3640 1 --2- 172.21.15.100:0/2572223862 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a7cf9cdb0 0x556a7cfa7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:54.381 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.376+0000 7ff4a8bd2640 1 --2- 172.21.15.100:0/2572223862 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a7cf9d7a0 0x556a7cfa75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:54.381 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.376+0000 7ff4a9bd4640 1 -- 172.21.15.100:0/2572223862 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a7cf9cdb0 msgr2=0x556a7cfa7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:54.381 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.376+0000 7ff4a9bd4640 1 --2- 172.21.15.100:0/2572223862 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a7cf9cdb0 0x556a7cfa7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:54.381 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.376+0000 7ff4a9bd4640 1 -- 172.21.15.100:0/2572223862 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a7cf9d7a0 msgr2=0x556a7cfa75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:54.381 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.376+0000 7ff4a9bd4640 1 --2- 172.21.15.100:0/2572223862 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a7cf9d7a0 0x556a7cfa75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:54.381 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.376+0000 7ff4a9bd4640 1 -- 172.21.15.100:0/2572223862 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a7ccdf620 con 0x556a7cfa2910 2024-05-10T11:19:54.382 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.376+0000 7ff4a9bd4640 1 --2- 172.21.15.100:0/2572223862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a7cfa2910 0x556a7cfae610 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff49c009270 tx=0x7ff49c0a9210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d501d88984614bc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:54.382 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.376+0000 7ff48cff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:19:54.382 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.376+0000 7ff48cff9640 1 -- 172.21.15.100:0/2572223862 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff49c01f040 con 0x556a7cfa2910 2024-05-10T11:19:54.382 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.376+0000 7ff48cff9640 1 -- 172.21.15.100:0/2572223862 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7ff49c0a8820 con 0x556a7cfa2910 2024-05-10T11:19:54.382 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.376+0000 7ff4ae42f740 10 client.0 _ll_register_callbacks cb 0x556a7cc12a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:19:54.382 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.377+0000 7ff4ae42f740 1 -- 172.21.15.100:0/2572223862 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4940047a0 con 0x556a7cfa2910 2024-05-10T11:19:54.382 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.377+0000 7ff48cff9640 1 -- 172.21.15.100:0/2572223862 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff49c0e2160 con 0x556a7cfa2910 2024-05-10T11:19:54.382 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.377+0000 7ff4ae42f740 10 client.15627 fetch_fsmap learned FSMap version 27 2024-05-10T11:19:54.382 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.377+0000 7ff4ae42f740 1 -- 172.21.15.100:0/2572223862 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({fsmap=27}) -- 0x556a7cfb1420 con 0x556a7cfa2910 2024-05-10T11:19:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.377+0000 7ff48cff9640 1 -- 172.21.15.100:0/2572223862 <== mon.0 v2:172.21.15.100:3300/0 4 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ff49c0a3ce0 con 0x556a7cfa2910 2024-05-10T11:19:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.377+0000 7ff4ae42f740 10 client.15627 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T11:19:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.377+0000 7ff4ae42f740 10 client.15627 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:19:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.377+0000 7ff4ae42f740 20 client.15627 populate_metadata read hostname 'smithi100' 2024-05-10T11:19:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.377+0000 7ff4ae42f740 1 --2- 172.21.15.100:0/2572223862 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556a7cfb2b40 0x556a7cfb4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.377+0000 7ff4ae42f740 4 client.15627 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:19:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.377+0000 7ff4ae42f740 1 -- 172.21.15.100:0/2572223862 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff4940047a0 con 0x556a7cfb2b40 2024-05-10T11:19:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.378+0000 7ff4a93d3640 1 --2- 172.21.15.100:0/2572223862 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556a7cfb2b40 0x556a7cfb4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.378+0000 7ff48cff9640 10 client.15627 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:19:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.378+0000 7ff4a93d3640 1 --2- 172.21.15.100:0/2572223862 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556a7cfb2b40 0x556a7cfb4fe0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fac7b288ae1989fa server_cookie=7b9f37f435aa113a in_seq=0 out_seq=0 2024-05-10T11:19:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.379+0000 7ff48cff9640 1 -- 172.21.15.100:0/2572223862 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7ff4940047a0 con 0x556a7cfb2b40 2024-05-10T11:19:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.379+0000 7ff48cff9640 10 client.15627 handle_command_reply: tid=0 2024-05-10T11:19:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.379+0000 7ff4ae42f740 1 client.15627 shutdown 2024-05-10T11:19:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.379+0000 7ff4ae42f740 1 -- 172.21.15.100:0/2572223862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a7cfa2910 msgr2=0x556a7cfae610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.379+0000 7ff4ae42f740 1 --2- 172.21.15.100:0/2572223862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a7cfa2910 0x556a7cfae610 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff49c009270 tx=0x7ff49c0a9210 comp rx=0 tx=0).stop 2024-05-10T11:19:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.379+0000 7ff4ae42f740 1 -- 172.21.15.100:0/2572223862 shutdown_connections 2024-05-10T11:19:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.379+0000 7ff4ae42f740 1 --2- 172.21.15.100:0/2572223862 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556a7cfb2b40 0x556a7cfb4fe0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.379+0000 7ff4ae42f740 1 --2- 172.21.15.100:0/2572223862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a7cfa2910 0x556a7cfae610 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.379+0000 7ff4ae42f740 1 --2- 172.21.15.100:0/2572223862 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a7cf9d7a0 0x556a7cfa75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.379+0000 7ff4ae42f740 1 --2- 172.21.15.100:0/2572223862 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a7cf9cdb0 0x556a7cfa7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:54.386 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.379+0000 7ff48cff9640 0 client.15627 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:19:54.386 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.379+0000 7ff4ae42f740 1 -- 172.21.15.100:0/2572223862 >> 172.21.15.100:0/2572223862 conn(0x556a7cf98420 msgr2=0x556a7cf97580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:54.386 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.379+0000 7ff4ae42f740 1 -- 172.21.15.100:0/2572223862 shutdown_connections 2024-05-10T11:19:54.386 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.379+0000 7ff4ae42f740 1 -- 172.21.15.100:0/2572223862 wait complete. 2024-05-10T11:19:54.386 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.379+0000 7ff4ae42f740 20 client.15627 trim_cache size 0 max 16384 2024-05-10T11:19:54.412 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.409+0000 7ff4ae42f740 1 Processor -- start 2024-05-10T11:19:54.412 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.409+0000 7ff4ae42f740 1 -- start start 2024-05-10T11:19:54.412 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.409+0000 7ff4ae42f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a7cf9cdb0 0x556a7cf9b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:54.412 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.409+0000 7ff4ae42f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a7cf9d7a0 0x556a7cfae5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.409+0000 7ff4ae42f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a7cfa2910 0x556a7cfb98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.409+0000 7ff4ae42f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a41ab780 con 0x556a7cfa2910 2024-05-10T11:19:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.409+0000 7ff4ae42f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a41a8fe0 con 0x556a7cf9cdb0 2024-05-10T11:19:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.409+0000 7ff4ae42f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a41a6e30 con 0x556a7cf9d7a0 2024-05-10T11:19:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.410+0000 7ff4a9bd4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a7cfa2910 0x556a7cfb98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.410+0000 7ff4a9bd4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a7cfa2910 0x556a7cfb98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56940/0 (socket says 172.21.15.100:56940) 2024-05-10T11:19:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.410+0000 7ff4a9bd4640 1 -- 172.21.15.100:0/3682989080 learned_addr learned my addr 172.21.15.100:0/3682989080 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.410+0000 7ff4a8bd2640 1 --2- 172.21.15.100:0/3682989080 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a7cf9d7a0 0x556a7cfae5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.410+0000 7ff4a93d3640 1 --2- 172.21.15.100:0/3682989080 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a7cf9cdb0 0x556a7cf9b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.410+0000 7ff4a9bd4640 1 -- 172.21.15.100:0/3682989080 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a7cf9d7a0 msgr2=0x556a7cfae5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.410+0000 7ff4a9bd4640 1 --2- 172.21.15.100:0/3682989080 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a7cf9d7a0 0x556a7cfae5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.410+0000 7ff4a9bd4640 1 -- 172.21.15.100:0/3682989080 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a7cf9cdb0 msgr2=0x556a7cf9b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.410+0000 7ff4a9bd4640 1 --2- 172.21.15.100:0/3682989080 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a7cf9cdb0 0x556a7cf9b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:54.414 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.410+0000 7ff4a9bd4640 1 -- 172.21.15.100:0/3682989080 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a7cf98250 con 0x556a7cfa2910 2024-05-10T11:19:54.414 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.410+0000 7ff4a93d3640 1 --2- 172.21.15.100:0/3682989080 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a7cf9cdb0 0x556a7cf9b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:19:54.414 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.410+0000 7ff4a9bd4640 1 --2- 172.21.15.100:0/3682989080 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a7cfa2910 0x556a7cfb98e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff49c0e2620 tx=0x7ff49c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64e31e13e20c5bdd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:54.414 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.410+0000 7ff48effd640 1 -- 172.21.15.100:0/3682989080 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff49c0b7070 con 0x556a7cfa2910 2024-05-10T11:19:54.414 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.410+0000 7ff48effd640 1 -- 172.21.15.100:0/3682989080 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff49c00caf0 con 0x556a7cfa2910 2024-05-10T11:19:54.414 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.410+0000 7ff48effd640 1 -- 172.21.15.100:0/3682989080 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff49c01f070 con 0x556a7cfa2910 2024-05-10T11:19:54.414 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.411+0000 7ff4ae42f740 1 -- 172.21.15.100:0/3682989080 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a7cfa2910 msgr2=0x556a7cfb98e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:54.414 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.411+0000 7ff4ae42f740 1 --2- 172.21.15.100:0/3682989080 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a7cfa2910 0x556a7cfb98e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff49c0e2620 tx=0x7ff49c012000 comp rx=0 tx=0).stop 2024-05-10T11:19:54.414 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.411+0000 7ff4ae42f740 1 -- 172.21.15.100:0/3682989080 shutdown_connections 2024-05-10T11:19:54.414 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.411+0000 7ff4ae42f740 1 --2- 172.21.15.100:0/3682989080 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a7cfa2910 0x556a7cfb98e0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:54.414 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.411+0000 7ff4ae42f740 1 --2- 172.21.15.100:0/3682989080 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a7cf9d7a0 0x556a7cfae5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:54.414 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.411+0000 7ff4ae42f740 1 --2- 172.21.15.100:0/3682989080 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a7cf9cdb0 0x556a7cf9b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:54.414 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.411+0000 7ff4ae42f740 1 -- 172.21.15.100:0/3682989080 >> 172.21.15.100:0/3682989080 conn(0x556a7cf98420 msgr2=0x556a7cf9b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:54.414 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.411+0000 7ff4ae42f740 1 -- 172.21.15.100:0/3682989080 shutdown_connections 2024-05-10T11:19:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.411+0000 7ff4ae42f740 1 -- 172.21.15.100:0/3682989080 wait complete. 2024-05-10T11:19:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.412+0000 7ff4ae42f740 1 -- start start 2024-05-10T11:19:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.412+0000 7ff4ae42f740 1 Processor -- start 2024-05-10T11:19:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.412+0000 7ff4ae42f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a7cf9cdb0 0x556a7cfadf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.412+0000 7ff4ae42f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a7cf9d7a0 0x556a7cfae4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.412+0000 7ff4ae42f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a7cfa2910 0x556a7cfac5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.412+0000 7ff4ae42f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a41a74a0 con 0x556a7cf9cdb0 2024-05-10T11:19:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.412+0000 7ff4ae42f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a41a8910 con 0x556a7cf9d7a0 2024-05-10T11:19:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.412+0000 7ff4ae42f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a41a7b70 con 0x556a7cfa2910 2024-05-10T11:19:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.412+0000 7ff4ae42f740 10 client.0 _ll_register_callbacks cb 0x7ff49c004ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:19:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.412+0000 7ff4a93d3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a7cf9cdb0 0x556a7cfadf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.412+0000 7ff4a93d3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a7cf9cdb0 0x556a7cfadf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56952/0 (socket says 172.21.15.100:56952) 2024-05-10T11:19:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.412+0000 7ff4a93d3640 1 -- 172.21.15.100:0/2887300043 learned_addr learned my addr 172.21.15.100:0/2887300043 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.413+0000 7ff4a8bd2640 1 --2- 172.21.15.100:0/2887300043 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a7cf9d7a0 0x556a7cfae4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.413+0000 7ff4a93d3640 1 -- 172.21.15.100:0/2887300043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a7cfa2910 msgr2=0x556a7cfac5e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T11:19:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.413+0000 7ff4a93d3640 1 --2- 172.21.15.100:0/2887300043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a7cfa2910 0x556a7cfac5e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.413+0000 7ff4a93d3640 1 -- 172.21.15.100:0/2887300043 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a7cf9d7a0 msgr2=0x556a7cfae4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:19:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.413+0000 7ff4a93d3640 1 --2- 172.21.15.100:0/2887300043 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a7cf9d7a0 0x556a7cfae4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.413+0000 7ff4a93d3640 1 -- 172.21.15.100:0/2887300043 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a7cfacdf0 con 0x556a7cf9cdb0 2024-05-10T11:19:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.413+0000 7ff4a93d3640 1 --2- 172.21.15.100:0/2887300043 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a7cf9cdb0 0x556a7cfadf80 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0009900 tx=0x7ff4a0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f650f55ca0f91108 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.413+0000 7ff48cff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:19:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.413+0000 7ff48cff9640 1 -- 172.21.15.100:0/2887300043 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a0059060 con 0x556a7cf9cdb0 2024-05-10T11:19:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.413+0000 7ff48cff9640 1 -- 172.21.15.100:0/2887300043 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7ff4a0031880 con 0x556a7cf9cdb0 2024-05-10T11:19:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.413+0000 7ff4ae42f740 1 -- 172.21.15.100:0/2887300043 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff49c00caf0 con 0x556a7cf9cdb0 2024-05-10T11:19:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.413+0000 7ff48cff9640 1 -- 172.21.15.100:0/2887300043 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff4a0060050 con 0x556a7cf9cdb0 2024-05-10T11:19:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.413+0000 7ff4ae42f740 10 client.15633 fetch_fsmap learned FSMap version 27 2024-05-10T11:19:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.413+0000 7ff4ae42f740 1 -- 172.21.15.100:0/2887300043 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556a7d026800 con 0x556a7cf9cdb0 2024-05-10T11:19:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.413+0000 7ff48cff9640 1 -- 172.21.15.100:0/2887300043 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff49c00caf0 con 0x556a7cf9cdb0 2024-05-10T11:19:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.414+0000 7ff48cff9640 1 -- 172.21.15.100:0/2887300043 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ff4a0044040 con 0x556a7cf9cdb0 2024-05-10T11:19:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.414+0000 7ff4ae42f740 10 client.15633 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T11:19:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.414+0000 7ff4ae42f740 10 client.15633 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:19:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.414+0000 7ff4ae42f740 20 client.15633 populate_metadata read hostname 'smithi100' 2024-05-10T11:19:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.414+0000 7ff4ae42f740 1 --2- 172.21.15.100:0/2887300043 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556a7d027150 0x556a7d029570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.414+0000 7ff4ae42f740 4 client.15633 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:19:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.414+0000 7ff4ae42f740 1 -- 172.21.15.100:0/2887300043 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556a7cf0f1e0 con 0x556a7d027150 2024-05-10T11:19:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.414+0000 7ff4a8bd2640 1 --2- 172.21.15.100:0/2887300043 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556a7d027150 0x556a7d029570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.414+0000 7ff4a8bd2640 1 --2- 172.21.15.100:0/2887300043 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556a7d027150 0x556a7d029570 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0f76c6e4404a628 server_cookie=47060f32a1888b8c in_seq=0 out_seq=0 2024-05-10T11:19:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.414+0000 7ff48cff9640 10 client.15633 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:19:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.418+0000 7ff48cff9640 1 -- 172.21.15.100:0/2887300043 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556a7cf0f1e0 con 0x556a7d027150 2024-05-10T11:19:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.418+0000 7ff48cff9640 10 client.15633 handle_command_reply: tid=0 2024-05-10T11:19:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.418+0000 7ff4ae42f740 1 client.15633 shutdown 2024-05-10T11:19:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.418+0000 7ff4ae42f740 1 -- 172.21.15.100:0/2887300043 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a7cf9cdb0 msgr2=0x556a7cfadf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.418+0000 7ff4ae42f740 1 --2- 172.21.15.100:0/2887300043 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a7cf9cdb0 0x556a7cfadf80 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0009900 tx=0x7ff4a0005fd0 comp rx=0 tx=0).stop 2024-05-10T11:19:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.419+0000 7ff4ae42f740 1 -- 172.21.15.100:0/2887300043 shutdown_connections 2024-05-10T11:19:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.419+0000 7ff4ae42f740 1 --2- 172.21.15.100:0/2887300043 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556a7d027150 0x556a7d029570 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.419+0000 7ff4ae42f740 1 --2- 172.21.15.100:0/2887300043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a7cfa2910 0x556a7cfac5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.419+0000 7ff4ae42f740 1 --2- 172.21.15.100:0/2887300043 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a7cf9d7a0 0x556a7cfae4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.419+0000 7ff4ae42f740 1 --2- 172.21.15.100:0/2887300043 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a7cf9cdb0 0x556a7cfadf80 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.419+0000 7ff4ae42f740 1 -- 172.21.15.100:0/2887300043 >> 172.21.15.100:0/2887300043 conn(0x556a7cf98420 msgr2=0x556a7cf9b990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.419+0000 7ff48cff9640 0 client.15633 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:19:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.419+0000 7ff4ae42f740 1 -- 172.21.15.100:0/2887300043 shutdown_connections 2024-05-10T11:19:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.419+0000 7ff4ae42f740 1 -- 172.21.15.100:0/2887300043 wait complete. 2024-05-10T11:19:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.419+0000 7ff4ae42f740 20 client.15633 trim_cache size 0 max 16384 2024-05-10T11:19:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:19:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:19:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:19:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:19:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.419+0000 7ff4ab637640 1 -- 172.21.15.100:0/1871833979 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff4900b2920 msgr2=0x7ff4900b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.419+0000 7ff4ab637640 1 --2- 172.21.15.100:0/1871833979 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff4900b2920 0x7ff4900b4de0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0009850 tx=0x7ff4a0000bc0 comp rx=0 tx=0).stop 2024-05-10T11:19:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.419+0000 7ff4ab637640 1 -- 172.21.15.100:0/1871833979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4a41034c0 msgr2=0x7ff4a419ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.420+0000 7ff4ab637640 1 --2- 172.21.15.100:0/1871833979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4a41034c0 0x7ff4a419ed50 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ff49c00eaa0 tx=0x7ff49c00eea0 comp rx=0 tx=0).stop 2024-05-10T11:19:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.420+0000 7ff4ab637640 1 -- 172.21.15.100:0/1871833979 shutdown_connections 2024-05-10T11:19:54.423 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.420+0000 7ff4ab637640 1 --2- 172.21.15.100:0/1871833979 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff4900b2920 0x7ff4900b4de0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:54.423 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.420+0000 7ff4ab637640 1 --2- 172.21.15.100:0/1871833979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4a41034c0 0x7ff4a419ed50 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:54.423 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.420+0000 7ff4ab637640 1 --2- 172.21.15.100:0/1871833979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff4a4102b00 0x7ff4a4197ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:54.423 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.420+0000 7ff4ab637640 1 --2- 172.21.15.100:0/1871833979 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4a4101910 0x7ff4a41977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:54.423 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.420+0000 7ff4ab637640 1 -- 172.21.15.100:0/1871833979 >> 172.21.15.100:0/1871833979 conn(0x7ff4a40fce80 msgr2=0x7ff4a4105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:54.423 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.420+0000 7ff4ab637640 1 -- 172.21.15.100:0/1871833979 shutdown_connections 2024-05-10T11:19:54.423 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:19:54.420+0000 7ff4ab637640 1 -- 172.21.15.100:0/1871833979 wait complete. 2024-05-10T11:19:54.781 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T11:19:54.781 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T11:19:55.069 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:19:56.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:19:55 smithi188 ceph-mon[34075]: pgmap v466: 129 pgs: 129 active+clean; 665 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 6 op/s 2024-05-10T11:19:56.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:19:55 smithi100 ceph-mon[30296]: pgmap v466: 129 pgs: 129 active+clean; 665 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 6 op/s 2024-05-10T11:19:56.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:19:55 smithi178 ceph-mon[33359]: pgmap v466: 129 pgs: 129 active+clean; 665 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 6 op/s 2024-05-10T11:19:56.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:56.961+0000 7f6a46c32700 1 -- 172.21.15.100:0/730798691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a40102450 msgr2=0x7f6a40110340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:56.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:56.961+0000 7f6a46c32700 1 --2- 172.21.15.100:0/730798691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a40102450 0x7f6a40110340 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f6a3800b0a0 tx=0x7f6a3800b3b0 comp rx=0 tx=0).stop 2024-05-10T11:19:56.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:56.961+0000 7f6a46c32700 1 -- 172.21.15.100:0/730798691 shutdown_connections 2024-05-10T11:19:56.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:56.961+0000 7f6a46c32700 1 --2- 172.21.15.100:0/730798691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a40102450 0x7f6a40110340 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:56.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:56.961+0000 7f6a46c32700 1 --2- 172.21.15.100:0/730798691 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a40109520 0x7f6a40109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:56.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:56.961+0000 7f6a46c32700 1 --2- 172.21.15.100:0/730798691 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a40101a90 0x7f6a40101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:56.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:56.961+0000 7f6a46c32700 1 -- 172.21.15.100:0/730798691 >> 172.21.15.100:0/730798691 conn(0x7f6a40078a10 msgr2=0x7f6a400ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:56.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:56.961+0000 7f6a46c32700 1 -- 172.21.15.100:0/730798691 shutdown_connections 2024-05-10T11:19:56.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:56.961+0000 7f6a46c32700 1 -- 172.21.15.100:0/730798691 wait complete. 2024-05-10T11:19:56.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:56.962+0000 7f6a46c32700 1 Processor -- start 2024-05-10T11:19:56.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:56.962+0000 7f6a46c32700 1 -- start start 2024-05-10T11:19:56.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:56.962+0000 7f6a46c32700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a40101a90 0x7f6a4019f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:56.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:56.962+0000 7f6a46c32700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a40102450 0x7f6a4019fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:56.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:56.963+0000 7f6a46c32700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a40109520 0x7f6a40199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:56.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:56.963+0000 7f6a449ce700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a40101a90 0x7f6a4019f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:56.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:56.963+0000 7f6a449ce700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a40101a90 0x7f6a4019f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:50278/0 (socket says 172.21.15.100:50278) 2024-05-10T11:19:56.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:56.963+0000 7f6a451cf700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a40109520 0x7f6a40199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:56.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:56.963+0000 7f6a449ce700 1 -- 172.21.15.100:0/3424671457 learned_addr learned my addr 172.21.15.100:0/3424671457 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:19:56.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:56.963+0000 7f6a3ffff700 1 --2- 172.21.15.100:0/3424671457 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a40102450 0x7f6a4019fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:56.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:56.963+0000 7f6a46c32700 1 -- 172.21.15.100:0/3424671457 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6a40113500 con 0x7f6a40109520 2024-05-10T11:19:56.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:56.963+0000 7f6a46c32700 1 -- 172.21.15.100:0/3424671457 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6a40113380 con 0x7f6a40101a90 2024-05-10T11:19:56.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:56.963+0000 7f6a46c32700 1 -- 172.21.15.100:0/3424671457 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6a40113680 con 0x7f6a40102450 2024-05-10T11:19:56.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:56.964+0000 7f6a451cf700 1 -- 172.21.15.100:0/3424671457 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a40102450 msgr2=0x7f6a4019fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:56.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:56.964+0000 7f6a451cf700 1 --2- 172.21.15.100:0/3424671457 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a40102450 0x7f6a4019fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:56.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:56.964+0000 7f6a451cf700 1 -- 172.21.15.100:0/3424671457 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a40101a90 msgr2=0x7f6a4019f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:56.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:56.964+0000 7f6a451cf700 1 --2- 172.21.15.100:0/3424671457 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a40101a90 0x7f6a4019f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:56.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:56.964+0000 7f6a451cf700 1 -- 172.21.15.100:0/3424671457 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a4019a020 con 0x7f6a40109520 2024-05-10T11:19:56.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:56.964+0000 7f6a451cf700 1 --2- 172.21.15.100:0/3424671457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a40109520 0x7f6a40199840 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f6a3800b0a0 tx=0x7f6a38004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:56.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:56.964+0000 7f6a3dffb700 1 -- 172.21.15.100:0/3424671457 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a38021070 con 0x7f6a40109520 2024-05-10T11:19:56.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:56.964+0000 7f6a3dffb700 1 -- 172.21.15.100:0/3424671457 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6a38007b80 con 0x7f6a40109520 2024-05-10T11:19:56.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:56.964+0000 7f6a3dffb700 1 -- 172.21.15.100:0/3424671457 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a38005040 con 0x7f6a40109520 2024-05-10T11:19:56.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:56.964+0000 7f6a46c32700 1 -- 172.21.15.100:0/3424671457 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a4019a2b0 con 0x7f6a40109520 2024-05-10T11:19:56.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:56.966+0000 7f6a46c32700 1 -- 172.21.15.100:0/3424671457 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a401a6520 con 0x7f6a40109520 2024-05-10T11:19:56.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:56.967+0000 7f6a46c32700 1 -- 172.21.15.100:0/3424671457 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a4010ba00 con 0x7f6a40109520 2024-05-10T11:19:56.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:56.967+0000 7f6a3dffb700 1 -- 172.21.15.100:0/3424671457 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f6a38024070 con 0x7f6a40109520 2024-05-10T11:19:56.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:56.968+0000 7f6a3dffb700 1 --2- 172.21.15.100:0/3424671457 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f6a300b29d0 0x7f6a300b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:19:56.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:56.969+0000 7f6a3dffb700 1 -- 172.21.15.100:0/3424671457 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f6a3800f020 con 0x7f6a40109520 2024-05-10T11:19:56.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:56.969+0000 7f6a449ce700 1 --2- 172.21.15.100:0/3424671457 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f6a300b29d0 0x7f6a300b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:19:56.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:56.970+0000 7f6a449ce700 1 --2- 172.21.15.100:0/3424671457 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f6a300b29d0 0x7f6a300b4e90 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6a34009ce0 tx=0x7f6a34009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:19:56.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:56.974+0000 7f6a3dffb700 1 -- 172.21.15.100:0/3424671457 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f6a380aa640 con 0x7f6a40109520 2024-05-10T11:19:57.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:57.228+0000 7f6a46c32700 1 -- 172.21.15.100:0/3424671457 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6a4019ad70 con 0x7f6a40109520 2024-05-10T11:19:57.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:57.229+0000 7f6a3dffb700 1 -- 172.21.15.100:0/3424671457 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11894 (secure 0 0 0) 0x7f6a380a9f40 con 0x7f6a40109520 2024-05-10T11:19:57.231 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:19:57.231 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2072225012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2072225012},{"type":"v1","addr":"172.21.15.100:6835","nonce":2072225012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/401939540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":401939540},{"type":"v1","addr":"172.21.15.100:6837","nonce":401939540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14571,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/3946311025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3946311025},{"type":"v1","addr":"172.21.15.100:6839","nonce":3946311025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3736371660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3736371660},{"type":"v1","addr":"172.21.15.100:6841","nonce":3736371660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/2741810791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2741810791},{"type":"v1","addr":"172.21.15.178:6833","nonce":2741810791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1704891041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1704891041},{"type":"v1","addr":"172.21.15.188:6833","nonce":1704891041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T11:14:27.242452+0000","modified":"2024-05-10T11:18:23.454860+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24439,"mds_1":24437,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/2124076312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2124076312},{"type":"v1","addr":"172.21.15.178:6835","nonce":2124076312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/3224358425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3224358425},{"type":"v1","addr":"172.21.15.178:6837","nonce":3224358425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/515547690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":515547690},{"type":"v1","addr":"172.21.15.188:6835","nonce":515547690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/1943342620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1943342620},{"type":"v1","addr":"172.21.15.188:6837","nonce":1943342620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.178:6839/3760985404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3760985404},{"type":"v1","addr":"172.21.15.178:6839","nonce":3760985404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.188:6839/3194917070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":3194917070},{"type":"v1","addr":"172.21.15.188:6839","nonce":3194917070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24439,"qdb_cluster":[24427,24437,24439]},"id":1}]} 2024-05-10T11:19:57.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:57.232+0000 7f6a46c32700 1 -- 172.21.15.100:0/3424671457 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f6a300b29d0 msgr2=0x7f6a300b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:57.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:57.232+0000 7f6a46c32700 1 --2- 172.21.15.100:0/3424671457 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f6a300b29d0 0x7f6a300b4e90 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6a34009ce0 tx=0x7f6a34009260 comp rx=0 tx=0).stop 2024-05-10T11:19:57.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:57.232+0000 7f6a46c32700 1 -- 172.21.15.100:0/3424671457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a40109520 msgr2=0x7f6a40199840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:19:57.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:57.232+0000 7f6a46c32700 1 --2- 172.21.15.100:0/3424671457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a40109520 0x7f6a40199840 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f6a3800b0a0 tx=0x7f6a38004d00 comp rx=0 tx=0).stop 2024-05-10T11:19:57.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:57.232+0000 7f6a46c32700 1 -- 172.21.15.100:0/3424671457 shutdown_connections 2024-05-10T11:19:57.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:57.232+0000 7f6a46c32700 1 --2- 172.21.15.100:0/3424671457 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f6a300b29d0 0x7f6a300b4e90 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:57.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:57.232+0000 7f6a46c32700 1 --2- 172.21.15.100:0/3424671457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a40109520 0x7f6a40199840 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:57.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:57.232+0000 7f6a46c32700 1 --2- 172.21.15.100:0/3424671457 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a40101a90 0x7f6a4019f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:57.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:57.232+0000 7f6a46c32700 1 --2- 172.21.15.100:0/3424671457 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a40102450 0x7f6a4019fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:19:57.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:57.232+0000 7f6a46c32700 1 -- 172.21.15.100:0/3424671457 >> 172.21.15.100:0/3424671457 conn(0x7f6a40078a10 msgr2=0x7f6a401002a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:19:57.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:57.232+0000 7f6a46c32700 1 -- 172.21.15.100:0/3424671457 shutdown_connections 2024-05-10T11:19:57.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:19:57.232+0000 7f6a46c32700 1 -- 172.21.15.100:0/3424671457 wait complete. 2024-05-10T11:19:57.233 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 27 2024-05-10T11:19:57.846 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:19:57 smithi100 ceph-mon[30296]: pgmap v467: 129 pgs: 129 active+clean; 665 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 7 op/s 2024-05-10T11:19:57.846 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:19:57 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/3424671457' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:19:57.885 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-10T11:19:58.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:19:57 smithi188 ceph-mon[34075]: pgmap v467: 129 pgs: 129 active+clean; 665 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 7 op/s 2024-05-10T11:19:58.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:19:57 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/3424671457' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:19:58.163 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:19:58.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:19:57 smithi178 ceph-mon[33359]: pgmap v467: 129 pgs: 129 active+clean; 665 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 7 op/s 2024-05-10T11:19:58.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:19:57 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/3424671457' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:20:00.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.010+0000 7f84ef38f700 1 -- 172.21.15.100:0/1449886120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84e8102d50 msgr2=0x7f84e810f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:00.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.010+0000 7f84ef38f700 1 --2- 172.21.15.100:0/1449886120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84e8102d50 0x7f84e810f240 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f84e000b0a0 tx=0x7f84e000b3b0 comp rx=0 tx=0).stop 2024-05-10T11:20:00.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.010+0000 7f84ef38f700 1 -- 172.21.15.100:0/1449886120 shutdown_connections 2024-05-10T11:20:00.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.010+0000 7f84ef38f700 1 --2- 172.21.15.100:0/1449886120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84e8102d50 0x7f84e810f240 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:00.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.010+0000 7f84ef38f700 1 --2- 172.21.15.100:0/1449886120 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f84e8108400 0x7f84e81087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:00.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.010+0000 7f84ef38f700 1 --2- 172.21.15.100:0/1449886120 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84e8102390 0x7f84e8102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:00.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.010+0000 7f84ef38f700 1 -- 172.21.15.100:0/1449886120 >> 172.21.15.100:0/1449886120 conn(0x7f84e8079310 msgr2=0x7f84e8079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:00.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.010+0000 7f84ef38f700 1 -- 172.21.15.100:0/1449886120 shutdown_connections 2024-05-10T11:20:00.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.010+0000 7f84ef38f700 1 -- 172.21.15.100:0/1449886120 wait complete. 2024-05-10T11:20:00.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.011+0000 7f84ef38f700 1 Processor -- start 2024-05-10T11:20:00.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.011+0000 7f84ef38f700 1 -- start start 2024-05-10T11:20:00.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.011+0000 7f84ef38f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84e8102390 0x7f84e819f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:00.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.011+0000 7f84ef38f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84e8102d50 0x7f84e819fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:00.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.011+0000 7f84ef38f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f84e8108400 0x7f84e81998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:00.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.011+0000 7f84ef38f700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f84e81123f0 con 0x7f84e8102d50 2024-05-10T11:20:00.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.011+0000 7f84ef38f700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f84e8112270 con 0x7f84e8102390 2024-05-10T11:20:00.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.011+0000 7f84ef38f700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f84e8112570 con 0x7f84e8108400 2024-05-10T11:20:00.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.011+0000 7f84ec92a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84e8102d50 0x7f84e819fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:00.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.011+0000 7f84ec92a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84e8102d50 0x7f84e819fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35206/0 (socket says 172.21.15.100:35206) 2024-05-10T11:20:00.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.011+0000 7f84ec92a700 1 -- 172.21.15.100:0/199933497 learned_addr learned my addr 172.21.15.100:0/199933497 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:20:00.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.011+0000 7f84ed92c700 1 --2- 172.21.15.100:0/199933497 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f84e8108400 0x7f84e81998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:00.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.011+0000 7f84ed12b700 1 --2- 172.21.15.100:0/199933497 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84e8102390 0x7f84e819f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:00.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.011+0000 7f84ec92a700 1 -- 172.21.15.100:0/199933497 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f84e8108400 msgr2=0x7f84e81998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:00.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.011+0000 7f84ec92a700 1 --2- 172.21.15.100:0/199933497 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f84e8108400 0x7f84e81998a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:00.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.011+0000 7f84ec92a700 1 -- 172.21.15.100:0/199933497 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84e8102390 msgr2=0x7f84e819f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:00.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.011+0000 7f84ec92a700 1 --2- 172.21.15.100:0/199933497 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84e8102390 0x7f84e819f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:00.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.011+0000 7f84ec92a700 1 -- 172.21.15.100:0/199933497 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84e819a000 con 0x7f84e8102d50 2024-05-10T11:20:00.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.012+0000 7f84ec92a700 1 --2- 172.21.15.100:0/199933497 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84e8102d50 0x7f84e819fc20 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f84d800d950 tx=0x7f84d800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:00.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.012+0000 7f84de7fc700 1 -- 172.21.15.100:0/199933497 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d8016070 con 0x7f84e8102d50 2024-05-10T11:20:00.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.012+0000 7f84de7fc700 1 -- 172.21.15.100:0/199933497 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f84d800fd20 con 0x7f84e8102d50 2024-05-10T11:20:00.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.012+0000 7f84ef38f700 1 -- 172.21.15.100:0/199933497 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84e819a290 con 0x7f84e8102d50 2024-05-10T11:20:00.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.012+0000 7f84de7fc700 1 -- 172.21.15.100:0/199933497 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d800e3b0 con 0x7f84e8102d50 2024-05-10T11:20:00.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.012+0000 7f84ef38f700 1 -- 172.21.15.100:0/199933497 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84e806cec0 con 0x7f84e8102d50 2024-05-10T11:20:00.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.014+0000 7f84de7fc700 1 -- 172.21.15.100:0/199933497 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f84d8019070 con 0x7f84e8102d50 2024-05-10T11:20:00.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.015+0000 7f84de7fc700 1 --2- 172.21.15.100:0/199933497 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f84d40b2880 0x7f84d40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:00.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.015+0000 7f84ed12b700 1 --2- 172.21.15.100:0/199933497 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f84d40b2880 0x7f84d40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:00.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.016+0000 7f84de7fc700 1 -- 172.21.15.100:0/199933497 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f84d80e23b0 con 0x7f84e8102d50 2024-05-10T11:20:00.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.016+0000 7f84ed12b700 1 --2- 172.21.15.100:0/199933497 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f84d40b2880 0x7f84d40b4d40 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f84e819a9e0 tx=0x7f84e400b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:00.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.016+0000 7f84f9c22b80 1 Processor -- start 2024-05-10T11:20:00.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.016+0000 7f84f9c22b80 1 -- start start 2024-05-10T11:20:00.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.017+0000 7f84f9c22b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce859f2590 0x55ce859f2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:00.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.017+0000 7f84f9c22b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce859f2eb0 0x55ce859ecc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:00.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.017+0000 7f84f9c22b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce859ed1b0 0x55ce859ed610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:00.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.017+0000 7f84f9c22b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55ce8593a940 con 0x55ce859f2590 2024-05-10T11:20:00.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.017+0000 7f84f9c22b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ce8589fda0 con 0x55ce859ed1b0 2024-05-10T11:20:00.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.017+0000 7f84f9c22b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55ce85936900 con 0x55ce859f2eb0 2024-05-10T11:20:00.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.017+0000 7f84ed12b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce859f2590 0x55ce859f2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:00.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.017+0000 7f84ed12b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce859f2590 0x55ce859f2970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35222/0 (socket says 172.21.15.100:35222) 2024-05-10T11:20:00.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.017+0000 7f84ed12b700 1 -- 172.21.15.100:0/3462571220 learned_addr learned my addr 172.21.15.100:0/3462571220 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:20:00.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.017+0000 7f84ed92c700 1 --2- 172.21.15.100:0/3462571220 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce859ed1b0 0x55ce859ed610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:00.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.017+0000 7f84ec92a700 1 --2- 172.21.15.100:0/3462571220 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce859f2eb0 0x55ce859ecc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:00.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.017+0000 7f84ed12b700 1 -- 172.21.15.100:0/3462571220 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce859f2eb0 msgr2=0x55ce859ecc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:00.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.017+0000 7f84ed12b700 1 --2- 172.21.15.100:0/3462571220 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce859f2eb0 0x55ce859ecc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:00.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.017+0000 7f84ed12b700 1 -- 172.21.15.100:0/3462571220 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce859ed1b0 msgr2=0x55ce859ed610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:00.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.017+0000 7f84ed12b700 1 --2- 172.21.15.100:0/3462571220 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce859ed1b0 0x55ce859ed610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:00.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.017+0000 7f84ed12b700 1 -- 172.21.15.100:0/3462571220 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce858878d0 con 0x55ce859f2590 2024-05-10T11:20:00.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.017+0000 7f84ed12b700 1 --2- 172.21.15.100:0/3462571220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce859f2590 0x55ce859f2970 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f84e4005e10 tx=0x7f84e4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab8ed583ae8f016 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:00.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.017+0000 7f84d37fe700 1 -- 172.21.15.100:0/3462571220 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e40118d0 con 0x55ce859f2590 2024-05-10T11:20:00.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.017+0000 7f84d37fe700 1 -- 172.21.15.100:0/3462571220 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f84e401e420 con 0x55ce859f2590 2024-05-10T11:20:00.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.018+0000 7f84d37fe700 1 -- 172.21.15.100:0/3462571220 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e401c610 con 0x55ce859f2590 2024-05-10T11:20:00.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.018+0000 7f84f9c22b80 1 -- 172.21.15.100:0/3462571220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce859f2590 msgr2=0x55ce859f2970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:00.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.018+0000 7f84f9c22b80 1 --2- 172.21.15.100:0/3462571220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce859f2590 0x55ce859f2970 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f84e4005e10 tx=0x7f84e4013000 comp rx=0 tx=0).stop 2024-05-10T11:20:00.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.018+0000 7f84f9c22b80 1 -- 172.21.15.100:0/3462571220 shutdown_connections 2024-05-10T11:20:00.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.018+0000 7f84f9c22b80 1 --2- 172.21.15.100:0/3462571220 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce859ed1b0 0x55ce859ed610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:00.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.018+0000 7f84f9c22b80 1 --2- 172.21.15.100:0/3462571220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce859f2590 0x55ce859f2970 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:00.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.018+0000 7f84f9c22b80 1 --2- 172.21.15.100:0/3462571220 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce859f2eb0 0x55ce859ecc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:00.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.018+0000 7f84f9c22b80 1 -- 172.21.15.100:0/3462571220 >> 172.21.15.100:0/3462571220 conn(0x55ce85914b70 msgr2=0x55ce859e32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:00.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.018+0000 7f84f9c22b80 1 -- 172.21.15.100:0/3462571220 shutdown_connections 2024-05-10T11:20:00.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.018+0000 7f84f9c22b80 1 -- 172.21.15.100:0/3462571220 wait complete. 2024-05-10T11:20:00.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.019+0000 7f84f9c22b80 1 -- start start 2024-05-10T11:20:00.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.019+0000 7f84f9c22b80 1 Processor -- start 2024-05-10T11:20:00.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.019+0000 7f84f9c22b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce859ed1b0 0x55ce859f8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:00.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.019+0000 7f84f9c22b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce859f2590 0x55ce859f5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:00.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.019+0000 7f84f9c22b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce859f2eb0 0x55ce859f63d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:00.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.019+0000 7f84f9c22b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55ce859f98e0 con 0x55ce859f2eb0 2024-05-10T11:20:00.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.019+0000 7f84f9c22b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ce858f3ed0 con 0x55ce859f2590 2024-05-10T11:20:00.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.019+0000 7f84f9c22b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55ce859f9a60 con 0x55ce859ed1b0 2024-05-10T11:20:00.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.020+0000 7f84f9c22b80 10 client.0 _ll_register_callbacks cb 0x55ce858ce5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:20:00.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.020+0000 7f84ed92c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce859f2eb0 0x55ce859f63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:00.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.020+0000 7f84ed92c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce859f2eb0 0x55ce859f63d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35228/0 (socket says 172.21.15.100:35228) 2024-05-10T11:20:00.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.020+0000 7f84ed92c700 1 -- 172.21.15.100:0/476020096 learned_addr learned my addr 172.21.15.100:0/476020096 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:20:00.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.020+0000 7f84ed12b700 1 --2- 172.21.15.100:0/476020096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce859ed1b0 0x55ce859f8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:00.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.020+0000 7f84ec92a700 1 --2- 172.21.15.100:0/476020096 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce859f2590 0x55ce859f5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:00.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.020+0000 7f84ed92c700 1 -- 172.21.15.100:0/476020096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce859ed1b0 msgr2=0x55ce859f8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:00.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.020+0000 7f84ed92c700 1 --2- 172.21.15.100:0/476020096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce859ed1b0 0x55ce859f8af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:00.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.020+0000 7f84ed92c700 1 -- 172.21.15.100:0/476020096 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce859f2590 msgr2=0x55ce859f5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:00.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.020+0000 7f84ed92c700 1 --2- 172.21.15.100:0/476020096 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce859f2590 0x55ce859f5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:00.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.020+0000 7f84ed92c700 1 -- 172.21.15.100:0/476020096 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce858f1760 con 0x55ce859f2eb0 2024-05-10T11:20:00.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.020+0000 7f84ec92a700 1 --2- 172.21.15.100:0/476020096 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce859f2590 0x55ce859f5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:20:00.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.020+0000 7f84ed92c700 1 --2- 172.21.15.100:0/476020096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce859f2eb0 0x55ce859f63d0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f84e0004c70 tx=0x7f84e0004ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33df53ba46675beb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:00.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.020+0000 7f84d0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:20:00.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.020+0000 7f84d0ff9700 1 -- 172.21.15.100:0/476020096 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e00078f0 con 0x55ce859f2eb0 2024-05-10T11:20:00.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.020+0000 7f84d0ff9700 1 -- 172.21.15.100:0/476020096 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f84e0007a90 con 0x55ce859f2eb0 2024-05-10T11:20:00.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.020+0000 7f84f9c22b80 1 -- 172.21.15.100:0/476020096 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84e401e420 con 0x55ce859f2eb0 2024-05-10T11:20:00.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.021+0000 7f84d0ff9700 1 -- 172.21.15.100:0/476020096 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f84e0031050 con 0x55ce859f2eb0 2024-05-10T11:20:00.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.021+0000 7f84f9c22b80 10 client.15651 fetch_fsmap learned FSMap version 27 2024-05-10T11:20:00.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.021+0000 7f84f9c22b80 1 -- 172.21.15.100:0/476020096 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ce85a04170 con 0x55ce859f2eb0 2024-05-10T11:20:00.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.021+0000 7f84d0ff9700 1 -- 172.21.15.100:0/476020096 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f84e401e420 con 0x55ce859f2eb0 2024-05-10T11:20:00.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.021+0000 7f84d0ff9700 1 -- 172.21.15.100:0/476020096 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f84e0011070 con 0x55ce859f2eb0 2024-05-10T11:20:00.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.021+0000 7f84f9c22b80 10 client.15651 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T11:20:00.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.021+0000 7f84f9c22b80 10 client.15651 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-10T11:20:00.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.021+0000 7f84f9c22b80 20 client.15651 populate_metadata read hostname 'smithi100' 2024-05-10T11:20:00.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.021+0000 7f84f9c22b80 1 --2- 172.21.15.100:0/476020096 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55ce85a067b0 0x55ce85a08ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:00.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.021+0000 7f84f9c22b80 4 client.15651 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:20:00.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.021+0000 7f84f9c22b80 1 -- 172.21.15.100:0/476020096 --> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ce85948320 con 0x55ce85a067b0 2024-05-10T11:20:00.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.021+0000 7f84ed12b700 1 --2- 172.21.15.100:0/476020096 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55ce85a067b0 0x55ce85a08ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:00.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.022+0000 7f84ed12b700 1 --2- 172.21.15.100:0/476020096 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55ce85a067b0 0x55ce85a08ba0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5b02d10b3fcab3c7 server_cookie=7fd080d8a4361514 in_seq=0 out_seq=0 2024-05-10T11:20:00.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.022+0000 7f84d0ff9700 10 client.15651 ms_handle_connect on v2:172.21.15.188:6836/1943342620 2024-05-10T11:20:00.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.022+0000 7f84d0ff9700 1 -- 172.21.15.100:0/476020096 <== mds.2 v2:172.21.15.188:6836/1943342620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ce85948320 con 0x55ce85a067b0 2024-05-10T11:20:00.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.022+0000 7f84d0ff9700 10 client.15651 handle_command_reply: tid=0 2024-05-10T11:20:00.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.022+0000 7f84f9c22b80 1 client.15651 shutdown 2024-05-10T11:20:00.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.023+0000 7f84f9c22b80 1 -- 172.21.15.100:0/476020096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce859f2eb0 msgr2=0x55ce859f63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:00.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.023+0000 7f84f9c22b80 1 --2- 172.21.15.100:0/476020096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce859f2eb0 0x55ce859f63d0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f84e0004c70 tx=0x7f84e0004ca0 comp rx=0 tx=0).stop 2024-05-10T11:20:00.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.023+0000 7f84f9c22b80 1 -- 172.21.15.100:0/476020096 shutdown_connections 2024-05-10T11:20:00.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.023+0000 7f84f9c22b80 1 --2- 172.21.15.100:0/476020096 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55ce85a067b0 0x55ce85a08ba0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:00.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.023+0000 7f84f9c22b80 1 --2- 172.21.15.100:0/476020096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce859f2eb0 0x55ce859f63d0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:00.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.023+0000 7f84f9c22b80 1 --2- 172.21.15.100:0/476020096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce859ed1b0 0x55ce859f8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:00.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.023+0000 7f84f9c22b80 1 --2- 172.21.15.100:0/476020096 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce859f2590 0x55ce859f5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:00.029 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:19:59 smithi100 ceph-mon[30296]: pgmap v468: 129 pgs: 129 active+clean; 666 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-10T11:20:00.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.023+0000 7f84f9c22b80 1 -- 172.21.15.100:0/476020096 >> 172.21.15.100:0/476020096 conn(0x55ce85914b70 msgr2=0x55ce859ddb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:00.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.023+0000 7f84d0ff9700 0 client.15651 ms_handle_reset on v2:172.21.15.188:6836/1943342620 2024-05-10T11:20:00.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.023+0000 7f84f9c22b80 1 -- 172.21.15.100:0/476020096 shutdown_connections 2024-05-10T11:20:00.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.023+0000 7f84f9c22b80 1 -- 172.21.15.100:0/476020096 wait complete. 2024-05-10T11:20:00.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.023+0000 7f84f9c22b80 20 client.15651 trim_cache size 0 max 16384 2024-05-10T11:20:00.030 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:19:59 smithi188 ceph-mon[34075]: pgmap v468: 129 pgs: 129 active+clean; 666 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-10T11:20:00.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.038+0000 7f84f9c22b80 1 Processor -- start 2024-05-10T11:20:00.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.038+0000 7f84f9c22b80 1 -- start start 2024-05-10T11:20:00.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.039+0000 7f84f9c22b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce859ed1b0 0x55ce859f6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:00.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.039+0000 7f84f9c22b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce859f2590 0x55ce85a02970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:00.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.039+0000 7f84f9c22b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce859f2eb0 0x55ce859f3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:00.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.039+0000 7f84f9c22b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f84e81b7e20 con 0x55ce859f2590 2024-05-10T11:20:00.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.039+0000 7f84f9c22b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f84e80737b0 con 0x55ce859ed1b0 2024-05-10T11:20:00.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.039+0000 7f84f9c22b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f84e8071600 con 0x55ce859f2eb0 2024-05-10T11:20:00.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.039+0000 7f84ec92a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce859f2590 0x55ce85a02970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:00.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.039+0000 7f84ec92a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce859f2590 0x55ce85a02970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35236/0 (socket says 172.21.15.100:35236) 2024-05-10T11:20:00.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.039+0000 7f84ec92a700 1 -- 172.21.15.100:0/2233070022 learned_addr learned my addr 172.21.15.100:0/2233070022 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:20:00.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.039+0000 7f84ed92c700 1 --2- 172.21.15.100:0/2233070022 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce859f2eb0 0x55ce859f3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:00.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.039+0000 7f84ed12b700 1 --2- 172.21.15.100:0/2233070022 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce859ed1b0 0x55ce859f6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:00.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.039+0000 7f84ec92a700 1 -- 172.21.15.100:0/2233070022 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce859f2eb0 msgr2=0x55ce859f3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:00.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.039+0000 7f84ec92a700 1 --2- 172.21.15.100:0/2233070022 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce859f2eb0 0x55ce859f3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:00.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.039+0000 7f84ec92a700 1 -- 172.21.15.100:0/2233070022 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce859ed1b0 msgr2=0x55ce859f6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:00.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.039+0000 7f84ec92a700 1 --2- 172.21.15.100:0/2233070022 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce859ed1b0 0x55ce859f6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:00.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.039+0000 7f84ec92a700 1 -- 172.21.15.100:0/2233070022 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce859dfc80 con 0x55ce859f2590 2024-05-10T11:20:00.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.039+0000 7f84ed92c700 1 --2- 172.21.15.100:0/2233070022 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce859f2eb0 0x55ce859f3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:20:00.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.039+0000 7f84ec92a700 1 --2- 172.21.15.100:0/2233070022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce859f2590 0x55ce85a02970 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f84d80a68e0 tx=0x7f84d80a5660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=463e672bbcd663b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:00.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.040+0000 7f84d2ffd700 1 -- 172.21.15.100:0/2233070022 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d8014020 con 0x55ce859f2590 2024-05-10T11:20:00.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.040+0000 7f84d2ffd700 1 -- 172.21.15.100:0/2233070022 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f84d80ada20 con 0x55ce859f2590 2024-05-10T11:20:00.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.040+0000 7f84d2ffd700 1 -- 172.21.15.100:0/2233070022 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d8020040 con 0x55ce859f2590 2024-05-10T11:20:00.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.040+0000 7f84f9c22b80 1 -- 172.21.15.100:0/2233070022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce859f2590 msgr2=0x55ce85a02970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:00.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.040+0000 7f84f9c22b80 1 --2- 172.21.15.100:0/2233070022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce859f2590 0x55ce85a02970 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f84d80a68e0 tx=0x7f84d80a5660 comp rx=0 tx=0).stop 2024-05-10T11:20:00.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.040+0000 7f84f9c22b80 1 -- 172.21.15.100:0/2233070022 shutdown_connections 2024-05-10T11:20:00.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.040+0000 7f84f9c22b80 1 --2- 172.21.15.100:0/2233070022 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce859f2eb0 0x55ce859f3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:00.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.040+0000 7f84f9c22b80 1 --2- 172.21.15.100:0/2233070022 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce859ed1b0 0x55ce859f6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:00.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.040+0000 7f84f9c22b80 1 --2- 172.21.15.100:0/2233070022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce859f2590 0x55ce85a02970 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:00.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.040+0000 7f84f9c22b80 1 -- 172.21.15.100:0/2233070022 >> 172.21.15.100:0/2233070022 conn(0x55ce85914b70 msgr2=0x55ce859f3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:00.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.041+0000 7f84f9c22b80 1 -- 172.21.15.100:0/2233070022 shutdown_connections 2024-05-10T11:20:00.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.041+0000 7f84f9c22b80 1 -- 172.21.15.100:0/2233070022 wait complete. 2024-05-10T11:20:00.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.041+0000 7f84f9c22b80 1 -- start start 2024-05-10T11:20:00.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.041+0000 7f84f9c22b80 1 Processor -- start 2024-05-10T11:20:00.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.041+0000 7f84f9c22b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce859ed1b0 0x55ce859ff350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:00.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.041+0000 7f84f9c22b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce859f2590 0x55ce85a0da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:00.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.041+0000 7f84f9c22b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce859f2eb0 0x55ce85a0df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:00.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.042+0000 7f84f9c22b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f84e8071c70 con 0x55ce859ed1b0 2024-05-10T11:20:00.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.042+0000 7f84f9c22b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f84e80730e0 con 0x55ce859f2eb0 2024-05-10T11:20:00.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.042+0000 7f84f9c22b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f84e8072340 con 0x55ce859f2590 2024-05-10T11:20:00.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.042+0000 7f84f9c22b80 10 client.0 _ll_register_callbacks cb 0x55ce858ce5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:20:00.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.042+0000 7f84ed12b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce859ed1b0 0x55ce859ff350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:00.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.042+0000 7f84ed12b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce859ed1b0 0x55ce859ff350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35248/0 (socket says 172.21.15.100:35248) 2024-05-10T11:20:00.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.042+0000 7f84ed12b700 1 -- 172.21.15.100:0/2084283716 learned_addr learned my addr 172.21.15.100:0/2084283716 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:20:00.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.042+0000 7f84ed92c700 1 --2- 172.21.15.100:0/2084283716 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce859f2eb0 0x55ce85a0df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:00.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.042+0000 7f84ec92a700 1 --2- 172.21.15.100:0/2084283716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce859f2590 0x55ce85a0da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:00.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.042+0000 7f84ed12b700 1 -- 172.21.15.100:0/2084283716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce859f2590 msgr2=0x55ce85a0da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:00.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.042+0000 7f84ed12b700 1 --2- 172.21.15.100:0/2084283716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce859f2590 0x55ce85a0da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:00.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.042+0000 7f84ed12b700 1 -- 172.21.15.100:0/2084283716 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce859f2eb0 msgr2=0x55ce85a0df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:00.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.042+0000 7f84ed12b700 1 --2- 172.21.15.100:0/2084283716 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce859f2eb0 0x55ce85a0df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:00.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.042+0000 7f84ed12b700 1 -- 172.21.15.100:0/2084283716 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce859f21e0 con 0x55ce859ed1b0 2024-05-10T11:20:00.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.042+0000 7f84ed92c700 1 --2- 172.21.15.100:0/2084283716 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce859f2eb0 0x55ce85a0df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:20:00.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.042+0000 7f84ed12b700 1 --2- 172.21.15.100:0/2084283716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce859ed1b0 0x55ce859ff350 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f84e4005e10 tx=0x7f84e4004310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e01237de17e6ee9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:00.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.042+0000 7f84d0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:20:00.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.042+0000 7f84d0ff9700 1 -- 172.21.15.100:0/2084283716 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e4011ce0 con 0x55ce859ed1b0 2024-05-10T11:20:00.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.042+0000 7f84d0ff9700 1 -- 172.21.15.100:0/2084283716 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f84e4005030 con 0x55ce859ed1b0 2024-05-10T11:20:00.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.042+0000 7f84f9c22b80 1 -- 172.21.15.100:0/2084283716 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84d80ada20 con 0x55ce859ed1b0 2024-05-10T11:20:00.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.043+0000 7f84d0ff9700 1 -- 172.21.15.100:0/2084283716 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f84e4044050 con 0x55ce859ed1b0 2024-05-10T11:20:00.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.043+0000 7f84f9c22b80 10 client.15657 fetch_fsmap learned FSMap version 27 2024-05-10T11:20:00.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.043+0000 7f84f9c22b80 1 -- 172.21.15.100:0/2084283716 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ce85a70eb0 con 0x55ce859ed1b0 2024-05-10T11:20:00.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.043+0000 7f84d0ff9700 1 -- 172.21.15.100:0/2084283716 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f84d80ada20 con 0x55ce859ed1b0 2024-05-10T11:20:00.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.043+0000 7f84d0ff9700 1 -- 172.21.15.100:0/2084283716 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f84e4004a10 con 0x55ce859ed1b0 2024-05-10T11:20:00.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.043+0000 7f84f9c22b80 10 client.15657 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T11:20:00.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.043+0000 7f84f9c22b80 10 client.15657 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-10T11:20:00.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.043+0000 7f84f9c22b80 20 client.15657 populate_metadata read hostname 'smithi100' 2024-05-10T11:20:00.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.043+0000 7f84f9c22b80 1 --2- 172.21.15.100:0/2084283716 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55ce85a71310 0x55ce85a73730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:00.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.043+0000 7f84f9c22b80 4 client.15657 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:20:00.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.043+0000 7f84f9c22b80 1 -- 172.21.15.100:0/2084283716 --> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ce859423c0 con 0x55ce85a71310 2024-05-10T11:20:00.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.043+0000 7f84ec92a700 1 --2- 172.21.15.100:0/2084283716 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55ce85a71310 0x55ce85a73730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:00.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.044+0000 7f84d0ff9700 10 client.15657 ms_handle_connect on v2:172.21.15.188:6836/1943342620 2024-05-10T11:20:00.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.044+0000 7f84ec92a700 1 --2- 172.21.15.100:0/2084283716 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55ce85a71310 0x55ce85a73730 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bed22100afa3de37 server_cookie=7bccfd622eec6d35 in_seq=0 out_seq=0 2024-05-10T11:20:00.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.044+0000 7f84d0ff9700 1 -- 172.21.15.100:0/2084283716 <== mds.2 v2:172.21.15.188:6836/1943342620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ce859423c0 con 0x55ce85a71310 2024-05-10T11:20:00.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.044+0000 7f84d0ff9700 10 client.15657 handle_command_reply: tid=0 2024-05-10T11:20:00.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.044+0000 7f84f9c22b80 1 client.15657 shutdown 2024-05-10T11:20:00.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.044+0000 7f84f9c22b80 1 -- 172.21.15.100:0/2084283716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce859ed1b0 msgr2=0x55ce859ff350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:00.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.044+0000 7f84f9c22b80 1 --2- 172.21.15.100:0/2084283716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce859ed1b0 0x55ce859ff350 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f84e4005e10 tx=0x7f84e4004310 comp rx=0 tx=0).stop 2024-05-10T11:20:00.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.044+0000 7f84f9c22b80 1 -- 172.21.15.100:0/2084283716 shutdown_connections 2024-05-10T11:20:00.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.044+0000 7f84f9c22b80 1 --2- 172.21.15.100:0/2084283716 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55ce85a71310 0x55ce85a73730 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:00.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.044+0000 7f84f9c22b80 1 --2- 172.21.15.100:0/2084283716 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce859f2eb0 0x55ce85a0df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:00.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.044+0000 7f84f9c22b80 1 --2- 172.21.15.100:0/2084283716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce859ed1b0 0x55ce859ff350 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:00.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.044+0000 7f84f9c22b80 1 --2- 172.21.15.100:0/2084283716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce859f2590 0x55ce85a0da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:00.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.044+0000 7f84f9c22b80 1 -- 172.21.15.100:0/2084283716 >> 172.21.15.100:0/2084283716 conn(0x55ce85914b70 msgr2=0x55ce859e7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:00.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.044+0000 7f84d0ff9700 0 client.15657 ms_handle_reset on v2:172.21.15.188:6836/1943342620 2024-05-10T11:20:00.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.044+0000 7f84f9c22b80 1 -- 172.21.15.100:0/2084283716 shutdown_connections 2024-05-10T11:20:00.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.044+0000 7f84f9c22b80 1 -- 172.21.15.100:0/2084283716 wait complete. 2024-05-10T11:20:00.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.044+0000 7f84f9c22b80 20 client.15657 trim_cache size 0 max 16384 2024-05-10T11:20:00.052 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:20:00.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.045+0000 7f84ef38f700 1 -- 172.21.15.100:0/199933497 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f84d40b2880 msgr2=0x7f84d40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:00.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.045+0000 7f84ef38f700 1 --2- 172.21.15.100:0/199933497 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f84d40b2880 0x7f84d40b4d40 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f84e819a9e0 tx=0x7f84e400b3c0 comp rx=0 tx=0).stop 2024-05-10T11:20:00.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.045+0000 7f84ef38f700 1 -- 172.21.15.100:0/199933497 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84e8102d50 msgr2=0x7f84e819fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:00.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.045+0000 7f84ef38f700 1 --2- 172.21.15.100:0/199933497 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84e8102d50 0x7f84e819fc20 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f84d800d950 tx=0x7f84d800dc60 comp rx=0 tx=0).stop 2024-05-10T11:20:00.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.045+0000 7f84ef38f700 1 -- 172.21.15.100:0/199933497 shutdown_connections 2024-05-10T11:20:00.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.045+0000 7f84ef38f700 1 --2- 172.21.15.100:0/199933497 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f84d40b2880 0x7f84d40b4d40 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:00.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.045+0000 7f84ef38f700 1 --2- 172.21.15.100:0/199933497 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f84e8108400 0x7f84e81998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:00.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.045+0000 7f84ef38f700 1 --2- 172.21.15.100:0/199933497 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84e8102390 0x7f84e819f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:00.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.045+0000 7f84ef38f700 1 --2- 172.21.15.100:0/199933497 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84e8102d50 0x7f84e819fc20 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:00.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.045+0000 7f84ef38f700 1 -- 172.21.15.100:0/199933497 >> 172.21.15.100:0/199933497 conn(0x7f84e8079310 msgr2=0x7f84e8106760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:00.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.045+0000 7f84ef38f700 1 -- 172.21.15.100:0/199933497 shutdown_connections 2024-05-10T11:20:00.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:00.045+0000 7f84ef38f700 1 -- 172.21.15.100:0/199933497 wait complete. 2024-05-10T11:20:00.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:19:59 smithi178 ceph-mon[33359]: pgmap v468: 129 pgs: 129 active+clean; 666 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-10T11:20:00.965 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:20:00 smithi100 ceph-mon[30296]: overall HEALTH_OK 2024-05-10T11:20:00.997 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-10T11:20:01.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:20:00 smithi178 ceph-mon[33359]: overall HEALTH_OK 2024-05-10T11:20:01.254 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:20:01.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:20:00 smithi188 ceph-mon[34075]: overall HEALTH_OK 2024-05-10T11:20:02.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:20:01 smithi100 ceph-mon[30296]: pgmap v469: 129 pgs: 129 active+clean; 666 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-10T11:20:02.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:20:01 smithi178 ceph-mon[33359]: pgmap v469: 129 pgs: 129 active+clean; 666 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-10T11:20:02.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:20:01 smithi188 ceph-mon[34075]: pgmap v469: 129 pgs: 129 active+clean; 666 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-10T11:20:02.742 INFO:tasks.workunit.client.0.smithi100.stderr:Updating files: 95% (11684/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-05-10T11:20:03.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.143+0000 7fbcfd080700 1 -- 172.21.15.100:0/1924185488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcf81023a0 msgr2=0x7fbcf8102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:03.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.143+0000 7fbcfd080700 1 --2- 172.21.15.100:0/1924185488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcf81023a0 0x7fbcf8102820 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fbce8009990 tx=0x7fbce8009ca0 comp rx=0 tx=0).stop 2024-05-10T11:20:03.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.143+0000 7fbcfd080700 1 -- 172.21.15.100:0/1924185488 shutdown_connections 2024-05-10T11:20:03.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.143+0000 7fbcfd080700 1 --2- 172.21.15.100:0/1924185488 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbcf8102d60 0x7fbcf810f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:03.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.143+0000 7fbcfd080700 1 --2- 172.21.15.100:0/1924185488 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbcf8108410 0x7fbcf81087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:03.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.143+0000 7fbcfd080700 1 --2- 172.21.15.100:0/1924185488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcf81023a0 0x7fbcf8102820 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:03.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.143+0000 7fbcfd080700 1 -- 172.21.15.100:0/1924185488 >> 172.21.15.100:0/1924185488 conn(0x7fbcf8079350 msgr2=0x7fbcf8079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:03.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.144+0000 7fbcfd080700 1 -- 172.21.15.100:0/1924185488 shutdown_connections 2024-05-10T11:20:03.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.144+0000 7fbcfd080700 1 -- 172.21.15.100:0/1924185488 wait complete. 2024-05-10T11:20:03.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.144+0000 7fbcfd080700 1 Processor -- start 2024-05-10T11:20:03.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.145+0000 7fbcfd080700 1 -- start start 2024-05-10T11:20:03.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.145+0000 7fbcfd080700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbcf81023a0 0x7fbcf8104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:03.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.145+0000 7fbcfd080700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbcf8102d60 0x7fbcf81045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:03.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.145+0000 7fbcfd080700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcf8108410 0x7fbcf8104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:03.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.145+0000 7fbcfd080700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf81123e0 con 0x7fbcf8108410 2024-05-10T11:20:03.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.145+0000 7fbcfd080700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf8112260 con 0x7fbcf8102d60 2024-05-10T11:20:03.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.145+0000 7fbcfd080700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf8112560 con 0x7fbcf81023a0 2024-05-10T11:20:03.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.145+0000 7fbcf759e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcf8108410 0x7fbcf8104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:03.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.145+0000 7fbcf759e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcf8108410 0x7fbcf8104b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35270/0 (socket says 172.21.15.100:35270) 2024-05-10T11:20:03.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.145+0000 7fbcf759e700 1 -- 172.21.15.100:0/2816615293 learned_addr learned my addr 172.21.15.100:0/2816615293 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:20:03.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.145+0000 7fbcf659c700 1 --2- 172.21.15.100:0/2816615293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbcf8102d60 0x7fbcf81045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:03.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.145+0000 7fbcf6d9d700 1 --2- 172.21.15.100:0/2816615293 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbcf81023a0 0x7fbcf8104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:03.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.145+0000 7fbcf759e700 1 -- 172.21.15.100:0/2816615293 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbcf81023a0 msgr2=0x7fbcf8104080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:03.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.145+0000 7fbcf759e700 1 --2- 172.21.15.100:0/2816615293 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbcf81023a0 0x7fbcf8104080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:03.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.145+0000 7fbcf759e700 1 -- 172.21.15.100:0/2816615293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbcf8102d60 msgr2=0x7fbcf81045c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:03.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.145+0000 7fbcf759e700 1 --2- 172.21.15.100:0/2816615293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbcf8102d60 0x7fbcf81045c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:03.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.145+0000 7fbcf759e700 1 -- 172.21.15.100:0/2816615293 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcf81a7f40 con 0x7fbcf8108410 2024-05-10T11:20:03.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.146+0000 7fbcf759e700 1 --2- 172.21.15.100:0/2816615293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcf8108410 0x7fbcf8104b00 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fbcf0007d70 tx=0x7fbcf000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:03.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.146+0000 7fbcdffff700 1 -- 172.21.15.100:0/2816615293 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf0016020 con 0x7fbcf8108410 2024-05-10T11:20:03.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.146+0000 7fbcdffff700 1 -- 172.21.15.100:0/2816615293 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbcf000ad40 con 0x7fbcf8108410 2024-05-10T11:20:03.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.146+0000 7fbcfd080700 1 -- 172.21.15.100:0/2816615293 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcf81a8230 con 0x7fbcf8108410 2024-05-10T11:20:03.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.146+0000 7fbcdffff700 1 -- 172.21.15.100:0/2816615293 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf00115e0 con 0x7fbcf8108410 2024-05-10T11:20:03.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.146+0000 7fbcfd080700 1 -- 172.21.15.100:0/2816615293 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcf81a86f0 con 0x7fbcf8108410 2024-05-10T11:20:03.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.148+0000 7fbcdffff700 1 -- 172.21.15.100:0/2816615293 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fbcf0019020 con 0x7fbcf8108410 2024-05-10T11:20:03.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.149+0000 7fbcdffff700 1 --2- 172.21.15.100:0/2816615293 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fbce00b2880 0x7fbce00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:03.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.149+0000 7fbcf6d9d700 1 --2- 172.21.15.100:0/2816615293 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fbce00b2880 0x7fbce00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:03.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.149+0000 7fbcdffff700 1 -- 172.21.15.100:0/2816615293 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fbcf00e2580 con 0x7fbcf8108410 2024-05-10T11:20:03.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.150+0000 7fbd07913b80 1 Processor -- start 2024-05-10T11:20:03.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.150+0000 7fbd07913b80 1 -- start start 2024-05-10T11:20:03.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.150+0000 7fbcf6d9d700 1 --2- 172.21.15.100:0/2816615293 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fbce00b2880 0x7fbce00b4d40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fbcec005950 tx=0x7fbcec0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:03.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.150+0000 7fbd07913b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56210cda64d0 0x56210cda68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:03.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.150+0000 7fbd07913b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56210cda6df0 0x56210cdaa6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:03.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.150+0000 7fbd07913b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56210cdaac80 0x56210cdab100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:03.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.150+0000 7fbd07913b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56210ccf8860 con 0x56210cda64d0 2024-05-10T11:20:03.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.150+0000 7fbd07913b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56210cc977c0 con 0x56210cdaac80 2024-05-10T11:20:03.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.150+0000 7fbd07913b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56210ccf4820 con 0x56210cda6df0 2024-05-10T11:20:03.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.150+0000 7fbcf6d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56210cda64d0 0x56210cda68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:03.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.150+0000 7fbcf6d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56210cda64d0 0x56210cda68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35282/0 (socket says 172.21.15.100:35282) 2024-05-10T11:20:03.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.150+0000 7fbcf6d9d700 1 -- 172.21.15.100:0/1816950183 learned_addr learned my addr 172.21.15.100:0/1816950183 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:20:03.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.150+0000 7fbcf659c700 1 --2- 172.21.15.100:0/1816950183 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56210cda6df0 0x56210cdaa6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:03.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.150+0000 7fbcf759e700 1 --2- 172.21.15.100:0/1816950183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56210cdaac80 0x56210cdab100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:03.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.150+0000 7fbcf6d9d700 1 -- 172.21.15.100:0/1816950183 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56210cda6df0 msgr2=0x56210cdaa6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:03.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.150+0000 7fbcf6d9d700 1 --2- 172.21.15.100:0/1816950183 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56210cda6df0 0x56210cdaa6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:03.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.150+0000 7fbcf6d9d700 1 -- 172.21.15.100:0/1816950183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56210cdaac80 msgr2=0x56210cdab100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:03.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.150+0000 7fbcf6d9d700 1 --2- 172.21.15.100:0/1816950183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56210cdaac80 0x56210cdab100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:03.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.150+0000 7fbcf6d9d700 1 -- 172.21.15.100:0/1816950183 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56210cc458d0 con 0x56210cda64d0 2024-05-10T11:20:03.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.151+0000 7fbcf6d9d700 1 --2- 172.21.15.100:0/1816950183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56210cda64d0 0x56210cda68b0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fbcec00fcd0 tx=0x7fbcec012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d68505c014685608 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:03.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.151+0000 7fbcdd7fa700 1 -- 172.21.15.100:0/1816950183 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcec022070 con 0x56210cda64d0 2024-05-10T11:20:03.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.151+0000 7fbcdd7fa700 1 -- 172.21.15.100:0/1816950183 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbcec011b90 con 0x56210cda64d0 2024-05-10T11:20:03.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.151+0000 7fbcdd7fa700 1 -- 172.21.15.100:0/1816950183 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcec01d3b0 con 0x56210cda64d0 2024-05-10T11:20:03.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.151+0000 7fbd07913b80 1 -- 172.21.15.100:0/1816950183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56210cda64d0 msgr2=0x56210cda68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:03.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.151+0000 7fbd07913b80 1 --2- 172.21.15.100:0/1816950183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56210cda64d0 0x56210cda68b0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fbcec00fcd0 tx=0x7fbcec012000 comp rx=0 tx=0).stop 2024-05-10T11:20:03.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.152+0000 7fbd07913b80 1 -- 172.21.15.100:0/1816950183 shutdown_connections 2024-05-10T11:20:03.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.152+0000 7fbd07913b80 1 --2- 172.21.15.100:0/1816950183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56210cdaac80 0x56210cdab100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:03.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.152+0000 7fbd07913b80 1 --2- 172.21.15.100:0/1816950183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56210cda64d0 0x56210cda68b0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:03.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.152+0000 7fbd07913b80 1 --2- 172.21.15.100:0/1816950183 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56210cda6df0 0x56210cdaa6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:03.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.152+0000 7fbd07913b80 1 -- 172.21.15.100:0/1816950183 >> 172.21.15.100:0/1816950183 conn(0x56210ccd2100 msgr2=0x56210cda0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:03.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.152+0000 7fbd07913b80 1 -- 172.21.15.100:0/1816950183 shutdown_connections 2024-05-10T11:20:03.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.152+0000 7fbd07913b80 1 -- 172.21.15.100:0/1816950183 wait complete. 2024-05-10T11:20:03.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.152+0000 7fbd07913b80 1 -- start start 2024-05-10T11:20:03.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.152+0000 7fbd07913b80 1 Processor -- start 2024-05-10T11:20:03.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.152+0000 7fbd07913b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56210cda64d0 0x56210cdb2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:03.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.152+0000 7fbd07913b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56210cda6df0 0x56210cdaf4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:03.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.152+0000 7fbd07913b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56210cdaac80 0x56210cdaf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:03.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.152+0000 7fbd07913b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56210cdb2e90 con 0x56210cdaac80 2024-05-10T11:20:03.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.152+0000 7fbd07913b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56210cc5ec30 con 0x56210cda6df0 2024-05-10T11:20:03.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.152+0000 7fbd07913b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56210cdb3010 con 0x56210cda64d0 2024-05-10T11:20:03.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.152+0000 7fbd07913b80 10 client.0 _ll_register_callbacks cb 0x56210cc90920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:20:03.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.153+0000 7fbcf759e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56210cdaac80 0x56210cdaf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:03.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.153+0000 7fbcf6d9d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56210cda64d0 0x56210cdb2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:03.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.153+0000 7fbcf759e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56210cdaac80 0x56210cdaf9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35292/0 (socket says 172.21.15.100:35292) 2024-05-10T11:20:03.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.153+0000 7fbcf6d9d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56210cda64d0 0x56210cdb2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:46902/0 (socket says 172.21.15.100:46902) 2024-05-10T11:20:03.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.153+0000 7fbcf759e700 1 -- 172.21.15.100:0/1845901948 learned_addr learned my addr 172.21.15.100:0/1845901948 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:20:03.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.153+0000 7fbcf659c700 1 --2- 172.21.15.100:0/1845901948 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56210cda6df0 0x56210cdaf4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:03.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.153+0000 7fbcf759e700 1 -- 172.21.15.100:0/1845901948 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56210cda64d0 msgr2=0x56210cdb2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:03.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.153+0000 7fbcf759e700 1 --2- 172.21.15.100:0/1845901948 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56210cda64d0 0x56210cdb2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:03.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.153+0000 7fbcf759e700 1 -- 172.21.15.100:0/1845901948 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56210cda6df0 msgr2=0x56210cdaf4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:03.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.153+0000 7fbcf759e700 1 --2- 172.21.15.100:0/1845901948 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56210cda6df0 0x56210cdaf4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:03.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.153+0000 7fbcf759e700 1 -- 172.21.15.100:0/1845901948 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56210cc94690 con 0x56210cdaac80 2024-05-10T11:20:03.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.153+0000 7fbcf659c700 1 --2- 172.21.15.100:0/1845901948 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56210cda6df0 0x56210cdaf4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:20:03.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.153+0000 7fbcf759e700 1 --2- 172.21.15.100:0/1845901948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56210cdaac80 0x56210cdaf9e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fbcf00a94e0 tx=0x7fbcf00a9d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3417e014c6923dce server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:03.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.153+0000 7fbcdaff5700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:20:03.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.153+0000 7fbcdaff5700 1 -- 172.21.15.100:0/1845901948 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf00a9ee0 con 0x56210cdaac80 2024-05-10T11:20:03.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.153+0000 7fbcdaff5700 1 -- 172.21.15.100:0/1845901948 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fbcf00910d0 con 0x56210cdaac80 2024-05-10T11:20:03.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.153+0000 7fbd07913b80 1 -- 172.21.15.100:0/1845901948 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbcec011b90 con 0x56210cdaac80 2024-05-10T11:20:03.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.154+0000 7fbcdaff5700 1 -- 172.21.15.100:0/1845901948 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbcf00a97e0 con 0x56210cdaac80 2024-05-10T11:20:03.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.154+0000 7fbd07913b80 10 client.15669 fetch_fsmap learned FSMap version 27 2024-05-10T11:20:03.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.154+0000 7fbd07913b80 1 -- 172.21.15.100:0/1845901948 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56210cdbd7d0 con 0x56210cdaac80 2024-05-10T11:20:03.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.154+0000 7fbcdaff5700 1 -- 172.21.15.100:0/1845901948 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbcec011b90 con 0x56210cdaac80 2024-05-10T11:20:03.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.154+0000 7fbcdaff5700 1 -- 172.21.15.100:0/1845901948 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fbcf006f0d0 con 0x56210cdaac80 2024-05-10T11:20:03.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.154+0000 7fbd07913b80 10 client.15669 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T11:20:03.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.154+0000 7fbd07913b80 10 client.15669 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-10T11:20:03.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.154+0000 7fbd07913b80 20 client.15669 populate_metadata read hostname 'smithi100' 2024-05-10T11:20:03.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.154+0000 7fbd07913b80 1 --2- 172.21.15.100:0/1845901948 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x56210cdbfd60 0x56210cdc2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:03.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.154+0000 7fbd07913b80 4 client.15669 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:20:03.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.154+0000 7fbd07913b80 1 -- 172.21.15.100:0/1845901948 --> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56210cd72650 con 0x56210cdbfd60 2024-05-10T11:20:03.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.154+0000 7fbcf6d9d700 1 --2- 172.21.15.100:0/1845901948 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x56210cdbfd60 0x56210cdc2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:03.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.155+0000 7fbcf6d9d700 1 --2- 172.21.15.100:0/1845901948 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x56210cdbfd60 0x56210cdc2150 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fc6cda09ab0df9f4 server_cookie=4228dc560890fd52 in_seq=0 out_seq=0 2024-05-10T11:20:03.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.155+0000 7fbcdaff5700 10 client.15669 ms_handle_connect on v2:172.21.15.178:6838/3760985404 2024-05-10T11:20:03.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.155+0000 7fbcdaff5700 1 -- 172.21.15.100:0/1845901948 <== mds.1 v2:172.21.15.178:6838/3760985404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56210cd72650 con 0x56210cdbfd60 2024-05-10T11:20:03.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.155+0000 7fbcdaff5700 10 client.15669 handle_command_reply: tid=0 2024-05-10T11:20:03.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.156+0000 7fbd07913b80 1 client.15669 shutdown 2024-05-10T11:20:03.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.156+0000 7fbd07913b80 1 -- 172.21.15.100:0/1845901948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56210cdaac80 msgr2=0x56210cdaf9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:03.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.156+0000 7fbd07913b80 1 --2- 172.21.15.100:0/1845901948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56210cdaac80 0x56210cdaf9e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fbcf00a94e0 tx=0x7fbcf00a9d20 comp rx=0 tx=0).stop 2024-05-10T11:20:03.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.156+0000 7fbd07913b80 1 -- 172.21.15.100:0/1845901948 shutdown_connections 2024-05-10T11:20:03.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.156+0000 7fbd07913b80 1 --2- 172.21.15.100:0/1845901948 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x56210cdbfd60 0x56210cdc2150 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:03.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.156+0000 7fbd07913b80 1 --2- 172.21.15.100:0/1845901948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56210cdaac80 0x56210cdaf9e0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:03.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.156+0000 7fbd07913b80 1 --2- 172.21.15.100:0/1845901948 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56210cda64d0 0x56210cdb2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:03.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.156+0000 7fbcdaff5700 0 client.15669 ms_handle_reset on v2:172.21.15.178:6838/3760985404 2024-05-10T11:20:03.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.156+0000 7fbd07913b80 1 --2- 172.21.15.100:0/1845901948 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56210cda6df0 0x56210cdaf4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:03.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.156+0000 7fbd07913b80 1 -- 172.21.15.100:0/1845901948 >> 172.21.15.100:0/1845901948 conn(0x56210ccd2100 msgr2=0x56210cd9b720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:03.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.156+0000 7fbd07913b80 1 -- 172.21.15.100:0/1845901948 shutdown_connections 2024-05-10T11:20:03.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.156+0000 7fbd07913b80 1 -- 172.21.15.100:0/1845901948 wait complete. 2024-05-10T11:20:03.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.156+0000 7fbd07913b80 20 client.15669 trim_cache size 0 max 16384 2024-05-10T11:20:03.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.171+0000 7fbd07913b80 1 Processor -- start 2024-05-10T11:20:03.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.171+0000 7fbd07913b80 1 -- start start 2024-05-10T11:20:03.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.171+0000 7fbd07913b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56210cda64d0 0x56210cdbbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:03.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.171+0000 7fbd07913b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56210cda6df0 0x56210cdb1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:03.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.171+0000 7fbd07913b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56210cdaac80 0x56210cdc7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:03.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.171+0000 7fbd07913b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf81b3b20 con 0x56210cda64d0 2024-05-10T11:20:03.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.171+0000 7fbd07913b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf81b13b0 con 0x56210cda6df0 2024-05-10T11:20:03.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.171+0000 7fbd07913b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf81af200 con 0x56210cdaac80 2024-05-10T11:20:03.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.171+0000 7fbcf6d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56210cda64d0 0x56210cdbbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:03.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.171+0000 7fbcf6d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56210cda64d0 0x56210cdbbf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35306/0 (socket says 172.21.15.100:35306) 2024-05-10T11:20:03.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.171+0000 7fbcf6d9d700 1 -- 172.21.15.100:0/3442257920 learned_addr learned my addr 172.21.15.100:0/3442257920 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:20:03.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.171+0000 7fbcf659c700 1 --2- 172.21.15.100:0/3442257920 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56210cda6df0 0x56210cdb1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:03.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.172+0000 7fbcf6d9d700 1 -- 172.21.15.100:0/3442257920 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56210cdaac80 msgr2=0x56210cdc7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:03.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.172+0000 7fbcf759e700 1 --2- 172.21.15.100:0/3442257920 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56210cdaac80 0x56210cdc7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:03.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.172+0000 7fbcf6d9d700 1 --2- 172.21.15.100:0/3442257920 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56210cdaac80 0x56210cdc7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:03.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.172+0000 7fbcf6d9d700 1 -- 172.21.15.100:0/3442257920 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56210cda6df0 msgr2=0x56210cdb1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:03.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.172+0000 7fbcf6d9d700 1 --2- 172.21.15.100:0/3442257920 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56210cda6df0 0x56210cdb1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:03.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.172+0000 7fbcf6d9d700 1 -- 172.21.15.100:0/3442257920 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56210cda6b00 con 0x56210cda64d0 2024-05-10T11:20:03.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.172+0000 7fbcf659c700 1 --2- 172.21.15.100:0/3442257920 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56210cda6df0 0x56210cdb1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:20:03.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.172+0000 7fbcf759e700 1 --2- 172.21.15.100:0/3442257920 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56210cdaac80 0x56210cdc7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:20:03.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.172+0000 7fbcf6d9d700 1 --2- 172.21.15.100:0/3442257920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56210cda64d0 0x56210cdbbf80 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fbcec00fcd0 tx=0x7fbcec015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac5de3585e7cbf14 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:03.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.172+0000 7fbcdcff9700 1 -- 172.21.15.100:0/3442257920 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcec039070 con 0x56210cda64d0 2024-05-10T11:20:03.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.172+0000 7fbcdcff9700 1 -- 172.21.15.100:0/3442257920 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbcec0054e0 con 0x56210cda64d0 2024-05-10T11:20:03.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.172+0000 7fbcdcff9700 1 -- 172.21.15.100:0/3442257920 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcec022040 con 0x56210cda64d0 2024-05-10T11:20:03.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.173+0000 7fbd07913b80 1 -- 172.21.15.100:0/3442257920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56210cda64d0 msgr2=0x56210cdbbf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:03.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.173+0000 7fbd07913b80 1 --2- 172.21.15.100:0/3442257920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56210cda64d0 0x56210cdbbf80 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fbcec00fcd0 tx=0x7fbcec015000 comp rx=0 tx=0).stop 2024-05-10T11:20:03.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.173+0000 7fbd07913b80 1 -- 172.21.15.100:0/3442257920 shutdown_connections 2024-05-10T11:20:03.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.173+0000 7fbd07913b80 1 --2- 172.21.15.100:0/3442257920 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56210cdaac80 0x56210cdc7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:03.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.173+0000 7fbd07913b80 1 --2- 172.21.15.100:0/3442257920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56210cda64d0 0x56210cdbbf80 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:03.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.173+0000 7fbd07913b80 1 --2- 172.21.15.100:0/3442257920 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56210cda6df0 0x56210cdb1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:03.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.173+0000 7fbd07913b80 1 -- 172.21.15.100:0/3442257920 >> 172.21.15.100:0/3442257920 conn(0x56210ccd2100 msgr2=0x56210cdb0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:03.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.173+0000 7fbd07913b80 1 -- 172.21.15.100:0/3442257920 shutdown_connections 2024-05-10T11:20:03.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.173+0000 7fbd07913b80 1 -- 172.21.15.100:0/3442257920 wait complete. 2024-05-10T11:20:03.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.173+0000 7fbd07913b80 1 -- start start 2024-05-10T11:20:03.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.173+0000 7fbd07913b80 1 Processor -- start 2024-05-10T11:20:03.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.174+0000 7fbd07913b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56210cda64d0 0x56210cdb65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:03.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.174+0000 7fbd07913b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56210cda6df0 0x56210cdb6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:03.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.174+0000 7fbd07913b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56210cdaac80 0x56210ce29540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:03.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.174+0000 7fbd07913b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf81af870 con 0x56210cda6df0 2024-05-10T11:20:03.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.174+0000 7fbd07913b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf81b0ce0 con 0x56210cda64d0 2024-05-10T11:20:03.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.174+0000 7fbd07913b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf81aff40 con 0x56210cdaac80 2024-05-10T11:20:03.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.174+0000 7fbd07913b80 10 client.0 _ll_register_callbacks cb 0x56210cc90920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:20:03.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.174+0000 7fbcf659c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56210cda6df0 0x56210cdb6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:03.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.174+0000 7fbcf659c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56210cda6df0 0x56210cdb6ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35320/0 (socket says 172.21.15.100:35320) 2024-05-10T11:20:03.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.174+0000 7fbcf659c700 1 -- 172.21.15.100:0/3236367901 learned_addr learned my addr 172.21.15.100:0/3236367901 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:20:03.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.174+0000 7fbcf759e700 1 --2- 172.21.15.100:0/3236367901 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56210cdaac80 0x56210ce29540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:03.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.174+0000 7fbcf6d9d700 1 --2- 172.21.15.100:0/3236367901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56210cda64d0 0x56210cdb65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:03.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.174+0000 7fbcf659c700 1 -- 172.21.15.100:0/3236367901 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56210cdaac80 msgr2=0x56210ce29540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:03.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.174+0000 7fbcf659c700 1 --2- 172.21.15.100:0/3236367901 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56210cdaac80 0x56210ce29540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:03.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.174+0000 7fbcf659c700 1 -- 172.21.15.100:0/3236367901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56210cda64d0 msgr2=0x56210cdb65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:03.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.174+0000 7fbcf659c700 1 --2- 172.21.15.100:0/3236367901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56210cda64d0 0x56210cdb65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:03.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.174+0000 7fbcf659c700 1 -- 172.21.15.100:0/3236367901 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56210ce29a80 con 0x56210cda6df0 2024-05-10T11:20:03.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.174+0000 7fbcf759e700 1 --2- 172.21.15.100:0/3236367901 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56210cdaac80 0x56210ce29540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:20:03.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.174+0000 7fbcf659c700 1 --2- 172.21.15.100:0/3236367901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56210cda6df0 0x56210cdb6ae0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fbce8000c00 tx=0x7fbce8009610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dd6210752ce1b94 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:03.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.174+0000 7fbcdaff5700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:20:03.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.174+0000 7fbcdaff5700 1 -- 172.21.15.100:0/3236367901 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce8028020 con 0x56210cda6df0 2024-05-10T11:20:03.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.174+0000 7fbd07913b80 1 -- 172.21.15.100:0/3236367901 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbcec0054e0 con 0x56210cda6df0 2024-05-10T11:20:03.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.174+0000 7fbcdaff5700 1 -- 172.21.15.100:0/3236367901 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fbce8003d90 con 0x56210cda6df0 2024-05-10T11:20:03.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.175+0000 7fbcdaff5700 1 -- 172.21.15.100:0/3236367901 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbce8033050 con 0x56210cda6df0 2024-05-10T11:20:03.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.175+0000 7fbd07913b80 10 client.15675 fetch_fsmap learned FSMap version 27 2024-05-10T11:20:03.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.175+0000 7fbd07913b80 1 -- 172.21.15.100:0/3236367901 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56210ce2acf0 con 0x56210cda6df0 2024-05-10T11:20:03.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.175+0000 7fbcdaff5700 1 -- 172.21.15.100:0/3236367901 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbcec0054e0 con 0x56210cda6df0 2024-05-10T11:20:03.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.175+0000 7fbcdaff5700 1 -- 172.21.15.100:0/3236367901 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fbce800f800 con 0x56210cda6df0 2024-05-10T11:20:03.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.175+0000 7fbd07913b80 10 client.15675 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T11:20:03.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.175+0000 7fbd07913b80 10 client.15675 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-10T11:20:03.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.175+0000 7fbd07913b80 20 client.15675 populate_metadata read hostname 'smithi100' 2024-05-10T11:20:03.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.175+0000 7fbd07913b80 1 --2- 172.21.15.100:0/3236367901 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x56210ce2b480 0x56210ce2d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:03.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.175+0000 7fbd07913b80 4 client.15675 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:20:03.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.175+0000 7fbd07913b80 1 -- 172.21.15.100:0/3236367901 --> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56210cd06240 con 0x56210ce2b480 2024-05-10T11:20:03.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.175+0000 7fbcf6d9d700 1 --2- 172.21.15.100:0/3236367901 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x56210ce2b480 0x56210ce2d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:03.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.176+0000 7fbcdaff5700 10 client.15675 ms_handle_connect on v2:172.21.15.178:6838/3760985404 2024-05-10T11:20:03.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.176+0000 7fbcf6d9d700 1 --2- 172.21.15.100:0/3236367901 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x56210ce2b480 0x56210ce2d8a0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d7bde84a8ac6b2d2 server_cookie=3d937b9ac4e2ce44 in_seq=0 out_seq=0 2024-05-10T11:20:03.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.176+0000 7fbcdaff5700 1 -- 172.21.15.100:0/3236367901 <== mds.1 v2:172.21.15.178:6838/3760985404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56210cd06240 con 0x56210ce2b480 2024-05-10T11:20:03.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.176+0000 7fbcdaff5700 10 client.15675 handle_command_reply: tid=0 2024-05-10T11:20:03.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.176+0000 7fbd07913b80 1 client.15675 shutdown 2024-05-10T11:20:03.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.176+0000 7fbd07913b80 1 -- 172.21.15.100:0/3236367901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56210cda6df0 msgr2=0x56210cdb6ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:03.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.176+0000 7fbd07913b80 1 --2- 172.21.15.100:0/3236367901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56210cda6df0 0x56210cdb6ae0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fbce8000c00 tx=0x7fbce8009610 comp rx=0 tx=0).stop 2024-05-10T11:20:03.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.176+0000 7fbd07913b80 1 -- 172.21.15.100:0/3236367901 shutdown_connections 2024-05-10T11:20:03.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.176+0000 7fbd07913b80 1 --2- 172.21.15.100:0/3236367901 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x56210ce2b480 0x56210ce2d8a0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:03.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.176+0000 7fbd07913b80 1 --2- 172.21.15.100:0/3236367901 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56210cdaac80 0x56210ce29540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:03.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.176+0000 7fbd07913b80 1 --2- 172.21.15.100:0/3236367901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56210cda64d0 0x56210cdb65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:03.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.176+0000 7fbd07913b80 1 --2- 172.21.15.100:0/3236367901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56210cda6df0 0x56210cdb6ae0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:03.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.176+0000 7fbd07913b80 1 -- 172.21.15.100:0/3236367901 >> 172.21.15.100:0/3236367901 conn(0x56210ccd2100 msgr2=0x56210cda5b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:03.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.176+0000 7fbcdaff5700 0 client.15675 ms_handle_reset on v2:172.21.15.178:6838/3760985404 2024-05-10T11:20:03.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.176+0000 7fbd07913b80 1 -- 172.21.15.100:0/3236367901 shutdown_connections 2024-05-10T11:20:03.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.176+0000 7fbd07913b80 1 -- 172.21.15.100:0/3236367901 wait complete. 2024-05-10T11:20:03.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.176+0000 7fbd07913b80 20 client.15675 trim_cache size 0 max 16384 2024-05-10T11:20:03.182 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:20:03.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.177+0000 7fbcfd080700 1 -- 172.21.15.100:0/2816615293 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fbce00b2880 msgr2=0x7fbce00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:03.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.177+0000 7fbcfd080700 1 --2- 172.21.15.100:0/2816615293 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fbce00b2880 0x7fbce00b4d40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fbcec005950 tx=0x7fbcec0058e0 comp rx=0 tx=0).stop 2024-05-10T11:20:03.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.177+0000 7fbcfd080700 1 -- 172.21.15.100:0/2816615293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcf8108410 msgr2=0x7fbcf8104b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:03.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.177+0000 7fbcfd080700 1 --2- 172.21.15.100:0/2816615293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcf8108410 0x7fbcf8104b00 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fbcf0007d70 tx=0x7fbcf000a340 comp rx=0 tx=0).stop 2024-05-10T11:20:03.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.177+0000 7fbcfd080700 1 -- 172.21.15.100:0/2816615293 shutdown_connections 2024-05-10T11:20:03.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.177+0000 7fbcfd080700 1 --2- 172.21.15.100:0/2816615293 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fbce00b2880 0x7fbce00b4d40 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:03.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.177+0000 7fbcfd080700 1 --2- 172.21.15.100:0/2816615293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcf8108410 0x7fbcf8104b00 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:03.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.177+0000 7fbcfd080700 1 --2- 172.21.15.100:0/2816615293 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbcf81023a0 0x7fbcf8104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:03.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.177+0000 7fbcfd080700 1 --2- 172.21.15.100:0/2816615293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbcf8102d60 0x7fbcf81045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:03.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.177+0000 7fbcfd080700 1 -- 172.21.15.100:0/2816615293 >> 172.21.15.100:0/2816615293 conn(0x7fbcf8079350 msgr2=0x7fbcf81016d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:03.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.177+0000 7fbcfd080700 1 -- 172.21.15.100:0/2816615293 shutdown_connections 2024-05-10T11:20:03.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:03.177+0000 7fbcfd080700 1 -- 172.21.15.100:0/2816615293 wait complete. 2024-05-10T11:20:03.578 INFO:tasks.workunit.client.0.smithi100.stderr:Note: switching to 'd39fa08867836ac070c47cb66c782fa559fd480b'. 2024-05-10T11:20:03.578 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-05-10T11:20:03.578 INFO:tasks.workunit.client.0.smithi100.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-10T11:20:03.578 INFO:tasks.workunit.client.0.smithi100.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-10T11:20:03.578 INFO:tasks.workunit.client.0.smithi100.stderr:state without impacting any branches by switching back to a branch. 2024-05-10T11:20:03.578 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-05-10T11:20:03.578 INFO:tasks.workunit.client.0.smithi100.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-10T11:20:03.578 INFO:tasks.workunit.client.0.smithi100.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-10T11:20:03.578 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-05-10T11:20:03.578 INFO:tasks.workunit.client.0.smithi100.stderr: git switch -c 2024-05-10T11:20:03.578 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-05-10T11:20:03.579 INFO:tasks.workunit.client.0.smithi100.stderr:Or undo this operation with: 2024-05-10T11:20:03.579 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-05-10T11:20:03.579 INFO:tasks.workunit.client.0.smithi100.stderr: git switch - 2024-05-10T11:20:03.579 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-05-10T11:20:03.579 INFO:tasks.workunit.client.0.smithi100.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-10T11:20:03.579 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-05-10T11:20:03.579 INFO:tasks.workunit.client.0.smithi100.stderr:HEAD is now at d39fa088678 client: fix file cache cap leak which can stall async read call 2024-05-10T11:20:03.587 DEBUG:teuthology.orchestra.run.smithi100:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-05-10T11:20:03.679 INFO:tasks.workunit.client.0.smithi100.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-10T11:20:03.681 INFO:tasks.workunit.client.0.smithi100.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-10T11:20:03.681 INFO:tasks.workunit.client.0.smithi100.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-10T11:20:04.020 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-10T11:20:04.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:20:03 smithi100 ceph-mon[30296]: pgmap v470: 129 pgs: 129 active+clean; 666 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-10T11:20:04.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:20:03 smithi178 ceph-mon[33359]: pgmap v470: 129 pgs: 129 active+clean; 666 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-10T11:20:04.266 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:20:04.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:20:03 smithi188 ceph-mon[34075]: pgmap v470: 129 pgs: 129 active+clean; 666 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-10T11:20:05.492 INFO:tasks.workunit.client.0.smithi100.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-10T11:20:05.590 INFO:tasks.workunit.client.0.smithi100.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-10T11:20:05.634 INFO:tasks.workunit.client.0.smithi100.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-10T11:20:05.636 INFO:tasks.workunit.client.0.smithi100.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-10T11:20:05.636 INFO:tasks.workunit.client.0.smithi100.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-10T11:20:05.679 INFO:tasks.workunit.client.0.smithi100.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-10T11:20:05.682 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-05-10T11:20:05.682 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-05-10T11:20:05.739 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-05-10T11:20:05.741 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-05-10T11:20:05.741 DEBUG:teuthology.orchestra.run.smithi100:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d39fa08867836ac070c47cb66c782fa559fd480b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-05-10T11:20:05.805 INFO:tasks.workunit.client.0.smithi100.stderr:+ dbench 1 2024-05-10T11:20:05.822 INFO:tasks.workunit.client.0.smithi100.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-05-10T11:20:05.822 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-05-10T11:20:05.822 INFO:tasks.workunit.client.0.smithi100.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-05-10T11:20:05.830 INFO:tasks.workunit.client.0.smithi100.stdout:0 of 1 processes prepared for launch 0 sec 2024-05-10T11:20:05.873 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:20:05 smithi100 ceph-mon[30296]: pgmap v471: 129 pgs: 129 active+clean; 666 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-10T11:20:05.930 INFO:tasks.workunit.client.0.smithi100.stdout:1 of 1 processes prepared for launch 0 sec 2024-05-10T11:20:05.930 INFO:tasks.workunit.client.0.smithi100.stdout:releasing clients 2024-05-10T11:20:06.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.121+0000 7f4eeb6ed700 1 -- 172.21.15.100:0/1781453338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ee4108880 msgr2=0x7f4ee4110400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:06.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.121+0000 7f4eeb6ed700 1 --2- 172.21.15.100:0/1781453338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ee4108880 0x7f4ee4110400 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f4edc00b0d0 tx=0x7f4edc00b3e0 comp rx=0 tx=0).stop 2024-05-10T11:20:06.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.121+0000 7f4eeb6ed700 1 -- 172.21.15.100:0/1781453338 shutdown_connections 2024-05-10T11:20:06.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.121+0000 7f4eeb6ed700 1 --2- 172.21.15.100:0/1781453338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ee4108880 0x7f4ee4110400 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:06.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.121+0000 7f4eeb6ed700 1 --2- 172.21.15.100:0/1781453338 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ee41037b0 0x7f4ee4103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:06.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.121+0000 7f4eeb6ed700 1 --2- 172.21.15.100:0/1781453338 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ee4104160 0x7f4ee4108150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:06.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.121+0000 7f4eeb6ed700 1 -- 172.21.15.100:0/1781453338 >> 172.21.15.100:0/1781453338 conn(0x7f4ee407a530 msgr2=0x7f4ee407a940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:06.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.122+0000 7f4eeb6ed700 1 -- 172.21.15.100:0/1781453338 shutdown_connections 2024-05-10T11:20:06.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.122+0000 7f4eeb6ed700 1 -- 172.21.15.100:0/1781453338 wait complete. 2024-05-10T11:20:06.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.122+0000 7f4eeb6ed700 1 Processor -- start 2024-05-10T11:20:06.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.122+0000 7f4eeb6ed700 1 -- start start 2024-05-10T11:20:06.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.122+0000 7f4eeb6ed700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ee41037b0 0x7f4ee419f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:06.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.123+0000 7f4eeb6ed700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ee4104160 0x7f4ee419fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:06.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.123+0000 7f4eeb6ed700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ee4108880 0x7f4ee41997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:06.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.123+0000 7f4eeb6ed700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee4113520 con 0x7f4ee4108880 2024-05-10T11:20:06.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.123+0000 7f4eeb6ed700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee41133a0 con 0x7f4ee41037b0 2024-05-10T11:20:06.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.123+0000 7f4eeb6ed700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee41136a0 con 0x7f4ee4104160 2024-05-10T11:20:06.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.123+0000 7f4ee9489700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ee41037b0 0x7f4ee419f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:06.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.123+0000 7f4ee9489700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ee41037b0 0x7f4ee419f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:50412/0 (socket says 172.21.15.100:50412) 2024-05-10T11:20:06.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.123+0000 7f4ee9489700 1 -- 172.21.15.100:0/1839175694 learned_addr learned my addr 172.21.15.100:0/1839175694 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:20:06.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.123+0000 7f4ee9c8a700 1 --2- 172.21.15.100:0/1839175694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ee4108880 0x7f4ee41997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:06.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.123+0000 7f4ee8c88700 1 --2- 172.21.15.100:0/1839175694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ee4104160 0x7f4ee419fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:06.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.123+0000 7f4ee9489700 1 -- 172.21.15.100:0/1839175694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ee4104160 msgr2=0x7f4ee419fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:06.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.123+0000 7f4ee9489700 1 --2- 172.21.15.100:0/1839175694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ee4104160 0x7f4ee419fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:06.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.123+0000 7f4ee9489700 1 -- 172.21.15.100:0/1839175694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ee4108880 msgr2=0x7f4ee41997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:06.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.123+0000 7f4ee9489700 1 --2- 172.21.15.100:0/1839175694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ee4108880 0x7f4ee41997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:06.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.123+0000 7f4ee9489700 1 -- 172.21.15.100:0/1839175694 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ee4199fc0 con 0x7f4ee41037b0 2024-05-10T11:20:06.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.123+0000 7f4ee9c8a700 1 --2- 172.21.15.100:0/1839175694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ee4108880 0x7f4ee41997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:20:06.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.123+0000 7f4ee9489700 1 --2- 172.21.15.100:0/1839175694 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ee41037b0 0x7f4ee419f610 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4ee000d980 tx=0x7f4ee000dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:06.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.124+0000 7f4eda7fc700 1 -- 172.21.15.100:0/1839175694 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee0016070 con 0x7f4ee41037b0 2024-05-10T11:20:06.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.124+0000 7f4eeb6ed700 1 -- 172.21.15.100:0/1839175694 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ee419a2b0 con 0x7f4ee41037b0 2024-05-10T11:20:06.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.124+0000 7f4eda7fc700 1 -- 172.21.15.100:0/1839175694 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4ee0004830 con 0x7f4ee41037b0 2024-05-10T11:20:06.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.124+0000 7f4eeb6ed700 1 -- 172.21.15.100:0/1839175694 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ee41a6470 con 0x7f4ee41037b0 2024-05-10T11:20:06.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.124+0000 7f4eda7fc700 1 -- 172.21.15.100:0/1839175694 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee0005410 con 0x7f4ee41037b0 2024-05-10T11:20:06.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.127+0000 7f4eda7fc700 1 -- 172.21.15.100:0/1839175694 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f4ee00055b0 con 0x7f4ee41037b0 2024-05-10T11:20:06.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.128+0000 7f4eda7fc700 1 --2- 172.21.15.100:0/1839175694 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4ed00b29d0 0x7f4ed00b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:06.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.128+0000 7f4ef5f80b80 1 Processor -- start 2024-05-10T11:20:06.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.128+0000 7f4ee8c88700 1 --2- 172.21.15.100:0/1839175694 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4ed00b29d0 0x7f4ed00b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:06.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.128+0000 7f4eda7fc700 1 -- 172.21.15.100:0/1839175694 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f4ee00e22f0 con 0x7f4ee41037b0 2024-05-10T11:20:06.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.128+0000 7f4ef5f80b80 1 -- start start 2024-05-10T11:20:06.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.128+0000 7f4ef5f80b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557bc0c344d0 0x557bc0c348b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:06.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.128+0000 7f4ef5f80b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557bc0c34df0 0x557bc0c386f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:06.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.128+0000 7f4ef5f80b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557bc0c38c80 0x557bc0c39100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:06.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.128+0000 7f4ef5f80b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x557bc0b86860 con 0x557bc0c34df0 2024-05-10T11:20:06.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.128+0000 7f4ef5f80b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x557bc0b257c0 con 0x557bc0c38c80 2024-05-10T11:20:06.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.128+0000 7f4ef5f80b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x557bc0b82820 con 0x557bc0c344d0 2024-05-10T11:20:06.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.128+0000 7f4ee8c88700 1 --2- 172.21.15.100:0/1839175694 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4ed00b29d0 0x7f4ed00b4e90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4102fe0 tx=0x7f4ed4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:06.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.129+0000 7f4ee8c88700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557bc0c34df0 0x557bc0c386f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:06.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.129+0000 7f4ee9489700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557bc0c344d0 0x557bc0c348b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:06.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.129+0000 7f4ee8c88700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557bc0c34df0 0x557bc0c386f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35358/0 (socket says 172.21.15.100:35358) 2024-05-10T11:20:06.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.129+0000 7f4ee8c88700 1 -- 172.21.15.100:0/274866884 learned_addr learned my addr 172.21.15.100:0/274866884 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:20:06.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.129+0000 7f4ee9c8a700 1 --2- 172.21.15.100:0/274866884 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557bc0c38c80 0x557bc0c39100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:06.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.129+0000 7f4ee9489700 1 -- 172.21.15.100:0/274866884 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557bc0c38c80 msgr2=0x557bc0c39100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:06.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.129+0000 7f4ee9489700 1 --2- 172.21.15.100:0/274866884 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557bc0c38c80 0x557bc0c39100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:06.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.129+0000 7f4ee9489700 1 -- 172.21.15.100:0/274866884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557bc0c34df0 msgr2=0x557bc0c386f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:06.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.129+0000 7f4ee9489700 1 --2- 172.21.15.100:0/274866884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557bc0c34df0 0x557bc0c386f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:06.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.129+0000 7f4ee9489700 1 -- 172.21.15.100:0/274866884 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bc0ad38d0 con 0x557bc0c344d0 2024-05-10T11:20:06.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.129+0000 7f4ee8c88700 1 --2- 172.21.15.100:0/274866884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557bc0c34df0 0x557bc0c386f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:20:06.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.129+0000 7f4ee9489700 1 --2- 172.21.15.100:0/274866884 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557bc0c344d0 0x557bc0c348b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4ee00dfae0 tx=0x7f4ee00076c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7fe6875b6fe8be63 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:06.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.129+0000 7f4ecf7fe700 1 -- 172.21.15.100:0/274866884 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee00ad960 con 0x557bc0c344d0 2024-05-10T11:20:06.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.130+0000 7f4ecf7fe700 1 -- 172.21.15.100:0/274866884 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4ee00b2410 con 0x557bc0c344d0 2024-05-10T11:20:06.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.130+0000 7f4ecf7fe700 1 -- 172.21.15.100:0/274866884 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee00a9a70 con 0x557bc0c344d0 2024-05-10T11:20:06.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.130+0000 7f4ef5f80b80 1 -- 172.21.15.100:0/274866884 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557bc0c344d0 msgr2=0x557bc0c348b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:06.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.130+0000 7f4ef5f80b80 1 --2- 172.21.15.100:0/274866884 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557bc0c344d0 0x557bc0c348b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4ee00dfae0 tx=0x7f4ee00076c0 comp rx=0 tx=0).stop 2024-05-10T11:20:06.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.130+0000 7f4ef5f80b80 1 -- 172.21.15.100:0/274866884 shutdown_connections 2024-05-10T11:20:06.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.130+0000 7f4ef5f80b80 1 --2- 172.21.15.100:0/274866884 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557bc0c38c80 0x557bc0c39100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:06.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.130+0000 7f4ef5f80b80 1 --2- 172.21.15.100:0/274866884 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557bc0c344d0 0x557bc0c348b0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:06.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.130+0000 7f4ef5f80b80 1 --2- 172.21.15.100:0/274866884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557bc0c34df0 0x557bc0c386f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:06.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.130+0000 7f4ef5f80b80 1 -- 172.21.15.100:0/274866884 >> 172.21.15.100:0/274866884 conn(0x557bc0b60100 msgr2=0x557bc0c2ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:06.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.130+0000 7f4ef5f80b80 1 -- 172.21.15.100:0/274866884 shutdown_connections 2024-05-10T11:20:06.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.130+0000 7f4ef5f80b80 1 -- 172.21.15.100:0/274866884 wait complete. 2024-05-10T11:20:06.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.131+0000 7f4ef5f80b80 1 -- start start 2024-05-10T11:20:06.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.131+0000 7f4ef5f80b80 1 Processor -- start 2024-05-10T11:20:06.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.131+0000 7f4ef5f80b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557bc0c344d0 0x557bc0c400b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:06.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.131+0000 7f4ef5f80b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557bc0c34df0 0x557bc0c3d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:06.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.131+0000 7f4ef5f80b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557bc0c38c80 0x557bc0c3d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:06.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.131+0000 7f4ef5f80b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x557bc0c40dd0 con 0x557bc0c34df0 2024-05-10T11:20:06.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.131+0000 7f4ef5f80b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x557bc0aecc30 con 0x557bc0c38c80 2024-05-10T11:20:06.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.131+0000 7f4ef5f80b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x557bc0c40f50 con 0x557bc0c344d0 2024-05-10T11:20:06.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.131+0000 7f4ee8c88700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557bc0c34df0 0x557bc0c3d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:06.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.132+0000 7f4ee9489700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557bc0c344d0 0x557bc0c400b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:06.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.132+0000 7f4ee8c88700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557bc0c34df0 0x557bc0c3d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35370/0 (socket says 172.21.15.100:35370) 2024-05-10T11:20:06.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.132+0000 7f4ee9c8a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557bc0c38c80 0x557bc0c3d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:06.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.132+0000 7f4ee8c88700 1 -- 172.21.15.100:0/2386895848 learned_addr learned my addr 172.21.15.100:0/2386895848 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:20:06.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.132+0000 7f4ef5f80b80 10 client.0 _ll_register_callbacks cb 0x557bc0b1e920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:20:06.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.132+0000 7f4ee8c88700 1 -- 172.21.15.100:0/2386895848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557bc0c344d0 msgr2=0x557bc0c400b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:06.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.132+0000 7f4ee8c88700 1 --2- 172.21.15.100:0/2386895848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557bc0c344d0 0x557bc0c400b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:06.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.132+0000 7f4ee8c88700 1 -- 172.21.15.100:0/2386895848 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557bc0c38c80 msgr2=0x557bc0c3d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:06.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.132+0000 7f4ee8c88700 1 --2- 172.21.15.100:0/2386895848 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557bc0c38c80 0x557bc0c3d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:06.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.132+0000 7f4ee8c88700 1 -- 172.21.15.100:0/2386895848 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bc0b22690 con 0x557bc0c34df0 2024-05-10T11:20:06.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.132+0000 7f4ee9489700 1 --2- 172.21.15.100:0/2386895848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557bc0c344d0 0x557bc0c400b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:20:06.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.132+0000 7f4ee8c88700 1 --2- 172.21.15.100:0/2386895848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557bc0c34df0 0x557bc0c3d450 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f4ed40116c0 tx=0x7f4ed4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60e4dea12016692c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:06.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.132+0000 7f4eccff9700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:20:06.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.132+0000 7f4eccff9700 1 -- 172.21.15.100:0/2386895848 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed4011e60 con 0x557bc0c34df0 2024-05-10T11:20:06.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.132+0000 7f4eccff9700 1 -- 172.21.15.100:0/2386895848 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f4ed401f840 con 0x557bc0c34df0 2024-05-10T11:20:06.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.132+0000 7f4ef5f80b80 1 -- 172.21.15.100:0/2386895848 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ee00b2410 con 0x557bc0c34df0 2024-05-10T11:20:06.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.132+0000 7f4eccff9700 1 -- 172.21.15.100:0/2386895848 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4ed400fe70 con 0x557bc0c34df0 2024-05-10T11:20:06.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.132+0000 7f4ef5f80b80 10 client.15687 fetch_fsmap learned FSMap version 27 2024-05-10T11:20:06.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.132+0000 7f4ef5f80b80 1 -- 172.21.15.100:0/2386895848 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557bc0c4fad0 con 0x557bc0c34df0 2024-05-10T11:20:06.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.133+0000 7f4eccff9700 1 -- 172.21.15.100:0/2386895848 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4ee00b2410 con 0x557bc0c34df0 2024-05-10T11:20:06.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.133+0000 7f4eccff9700 1 -- 172.21.15.100:0/2386895848 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f4ed4028bf0 con 0x557bc0c34df0 2024-05-10T11:20:06.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.133+0000 7f4ef5f80b80 10 client.15687 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T11:20:06.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.133+0000 7f4ef5f80b80 10 client.15687 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:20:06.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.133+0000 7f4ef5f80b80 20 client.15687 populate_metadata read hostname 'smithi100' 2024-05-10T11:20:06.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.133+0000 7f4ef5f80b80 1 --2- 172.21.15.100:0/2386895848 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x557bc0c52110 0x557bc0c54500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:06.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.133+0000 7f4ef5f80b80 4 client.15687 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:20:06.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.133+0000 7f4ef5f80b80 1 -- 172.21.15.100:0/2386895848 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557bc0c00650 con 0x557bc0c52110 2024-05-10T11:20:06.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.133+0000 7f4ee9489700 1 --2- 172.21.15.100:0/2386895848 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x557bc0c52110 0x557bc0c54500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:06.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.133+0000 7f4eccff9700 10 client.15687 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:20:06.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.133+0000 7f4ee9489700 1 --2- 172.21.15.100:0/2386895848 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x557bc0c52110 0x557bc0c54500 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3665c9bdcfdb05e server_cookie=a8c1f7a409cd3bc in_seq=0 out_seq=0 2024-05-10T11:20:06.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.134+0000 7f4eccff9700 1 -- 172.21.15.100:0/2386895848 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557bc0c00650 con 0x557bc0c52110 2024-05-10T11:20:06.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.134+0000 7f4eccff9700 10 client.15687 handle_command_reply: tid=0 2024-05-10T11:20:06.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.134+0000 7f4ef5f80b80 1 client.15687 shutdown 2024-05-10T11:20:06.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.134+0000 7f4ef5f80b80 1 -- 172.21.15.100:0/2386895848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557bc0c34df0 msgr2=0x557bc0c3d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:06.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.134+0000 7f4ef5f80b80 1 --2- 172.21.15.100:0/2386895848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557bc0c34df0 0x557bc0c3d450 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f4ed40116c0 tx=0x7f4ed4004750 comp rx=0 tx=0).stop 2024-05-10T11:20:06.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.135+0000 7f4ef5f80b80 1 -- 172.21.15.100:0/2386895848 shutdown_connections 2024-05-10T11:20:06.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.135+0000 7f4ef5f80b80 1 --2- 172.21.15.100:0/2386895848 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x557bc0c52110 0x557bc0c54500 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:06.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.135+0000 7f4ef5f80b80 1 --2- 172.21.15.100:0/2386895848 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557bc0c38c80 0x557bc0c3d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:06.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.135+0000 7f4ef5f80b80 1 --2- 172.21.15.100:0/2386895848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557bc0c344d0 0x557bc0c400b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:06.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.135+0000 7f4ef5f80b80 1 --2- 172.21.15.100:0/2386895848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557bc0c34df0 0x557bc0c3d450 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:06.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.135+0000 7f4ef5f80b80 1 -- 172.21.15.100:0/2386895848 >> 172.21.15.100:0/2386895848 conn(0x557bc0b60100 msgr2=0x557bc0c29720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:06.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.135+0000 7f4eccff9700 0 client.15687 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:20:06.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.135+0000 7f4ef5f80b80 1 -- 172.21.15.100:0/2386895848 shutdown_connections 2024-05-10T11:20:06.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.135+0000 7f4ef5f80b80 1 -- 172.21.15.100:0/2386895848 wait complete. 2024-05-10T11:20:06.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.135+0000 7f4ef5f80b80 20 client.15687 trim_cache size 0 max 16384 2024-05-10T11:20:06.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.153+0000 7f4ef5f80b80 1 Processor -- start 2024-05-10T11:20:06.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.153+0000 7f4ef5f80b80 1 -- start start 2024-05-10T11:20:06.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.154+0000 7f4ef5f80b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557bc0c344d0 0x557bc0c49e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:06.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.154+0000 7f4ef5f80b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557bc0c34df0 0x557bc0c59dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:06.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.154+0000 7f4ef5f80b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557bc0c38c80 0x557bc0c5a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:06.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.154+0000 7f4ef5f80b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee41af660 con 0x557bc0c344d0 2024-05-10T11:20:06.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.154+0000 7f4ef5f80b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee41acef0 con 0x557bc0c38c80 2024-05-10T11:20:06.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.154+0000 7f4ef5f80b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee41aad40 con 0x557bc0c34df0 2024-05-10T11:20:06.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.154+0000 7f4ee9489700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557bc0c344d0 0x557bc0c49e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:06.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.154+0000 7f4ee8c88700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557bc0c34df0 0x557bc0c59dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:06.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.154+0000 7f4ee9489700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557bc0c344d0 0x557bc0c49e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35374/0 (socket says 172.21.15.100:35374) 2024-05-10T11:20:06.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.154+0000 7f4ee9489700 1 -- 172.21.15.100:0/2114820007 learned_addr learned my addr 172.21.15.100:0/2114820007 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:20:06.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.154+0000 7f4ee9c8a700 1 --2- 172.21.15.100:0/2114820007 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557bc0c38c80 0x557bc0c5a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:06.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.154+0000 7f4ee9489700 1 -- 172.21.15.100:0/2114820007 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557bc0c34df0 msgr2=0x557bc0c59dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:06.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.154+0000 7f4ee9489700 1 --2- 172.21.15.100:0/2114820007 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557bc0c34df0 0x557bc0c59dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:06.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.154+0000 7f4ee9489700 1 -- 172.21.15.100:0/2114820007 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557bc0c38c80 msgr2=0x557bc0c5a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:06.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.154+0000 7f4ee9489700 1 --2- 172.21.15.100:0/2114820007 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557bc0c38c80 0x557bc0c5a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:06.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.154+0000 7f4ee9489700 1 -- 172.21.15.100:0/2114820007 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bc0c38a40 con 0x557bc0c344d0 2024-05-10T11:20:06.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.154+0000 7f4ee9489700 1 --2- 172.21.15.100:0/2114820007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557bc0c344d0 0x557bc0c49e60 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f4ee00a92b0 tx=0x7f4ee00e1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40a765e059896f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:06.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.155+0000 7f4eceffd700 1 -- 172.21.15.100:0/2114820007 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee00b7070 con 0x557bc0c344d0 2024-05-10T11:20:06.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.155+0000 7f4eceffd700 1 -- 172.21.15.100:0/2114820007 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4ee00e2120 con 0x557bc0c344d0 2024-05-10T11:20:06.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.155+0000 7f4eceffd700 1 -- 172.21.15.100:0/2114820007 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee00ca030 con 0x557bc0c344d0 2024-05-10T11:20:06.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.155+0000 7f4ef5f80b80 1 -- 172.21.15.100:0/2114820007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557bc0c344d0 msgr2=0x557bc0c49e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:06.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.155+0000 7f4ef5f80b80 1 --2- 172.21.15.100:0/2114820007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557bc0c344d0 0x557bc0c49e60 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f4ee00a92b0 tx=0x7f4ee00e1040 comp rx=0 tx=0).stop 2024-05-10T11:20:06.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.156+0000 7f4ef5f80b80 1 -- 172.21.15.100:0/2114820007 shutdown_connections 2024-05-10T11:20:06.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.156+0000 7f4ef5f80b80 1 --2- 172.21.15.100:0/2114820007 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557bc0c38c80 0x557bc0c5a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:06.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.156+0000 7f4ef5f80b80 1 --2- 172.21.15.100:0/2114820007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557bc0c344d0 0x557bc0c49e60 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:06.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.156+0000 7f4ef5f80b80 1 --2- 172.21.15.100:0/2114820007 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557bc0c34df0 0x557bc0c59dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:06.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.156+0000 7f4ef5f80b80 1 -- 172.21.15.100:0/2114820007 >> 172.21.15.100:0/2114820007 conn(0x557bc0b60100 msgr2=0x557bc0c3bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:06.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.156+0000 7f4ef5f80b80 1 -- 172.21.15.100:0/2114820007 shutdown_connections 2024-05-10T11:20:06.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.156+0000 7f4ef5f80b80 1 -- 172.21.15.100:0/2114820007 wait complete. 2024-05-10T11:20:06.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.156+0000 7f4ef5f80b80 1 -- start start 2024-05-10T11:20:06.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.156+0000 7f4ef5f80b80 1 Processor -- start 2024-05-10T11:20:06.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.156+0000 7f4ef5f80b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557bc0c344d0 0x557bc0c411a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:06.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.156+0000 7f4ef5f80b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557bc0c34df0 0x557bc0c416e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:06.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.156+0000 7f4ef5f80b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557bc0c38c80 0x557bc0cba960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:06.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.156+0000 7f4ef5f80b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee41ab3b0 con 0x557bc0c38c80 2024-05-10T11:20:06.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.156+0000 7f4ef5f80b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee41ac820 con 0x557bc0c344d0 2024-05-10T11:20:06.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.156+0000 7f4ef5f80b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee41aba80 con 0x557bc0c34df0 2024-05-10T11:20:06.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.156+0000 7f4ef5f80b80 10 client.0 _ll_register_callbacks cb 0x557bc0b1e920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:20:06.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.157+0000 7f4ee9489700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557bc0c344d0 0x557bc0c411a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:06.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.157+0000 7f4ee8c88700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557bc0c34df0 0x557bc0c416e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:06.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.157+0000 7f4ee9489700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557bc0c344d0 0x557bc0c411a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:50456/0 (socket says 172.21.15.100:50456) 2024-05-10T11:20:06.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.157+0000 7f4ee8c88700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557bc0c34df0 0x557bc0c416e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:46998/0 (socket says 172.21.15.100:46998) 2024-05-10T11:20:06.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.157+0000 7f4ee9489700 1 -- 172.21.15.100:0/2566807572 learned_addr learned my addr 172.21.15.100:0/2566807572 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:20:06.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.157+0000 7f4ee9c8a700 1 --2- 172.21.15.100:0/2566807572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557bc0c38c80 0x557bc0cba960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:06.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.157+0000 7f4ee9489700 1 -- 172.21.15.100:0/2566807572 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557bc0c34df0 msgr2=0x557bc0c416e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:06.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.157+0000 7f4ee9489700 1 --2- 172.21.15.100:0/2566807572 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557bc0c34df0 0x557bc0c416e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:06.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.157+0000 7f4ee9489700 1 -- 172.21.15.100:0/2566807572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557bc0c38c80 msgr2=0x557bc0cba960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:06.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.157+0000 7f4ee9489700 1 --2- 172.21.15.100:0/2566807572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557bc0c38c80 0x557bc0cba960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:06.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.157+0000 7f4ee9489700 1 -- 172.21.15.100:0/2566807572 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bc0cbaea0 con 0x557bc0c344d0 2024-05-10T11:20:06.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.157+0000 7f4ee9c8a700 1 --2- 172.21.15.100:0/2566807572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557bc0c38c80 0x557bc0cba960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:20:06.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.157+0000 7f4ee9489700 1 --2- 172.21.15.100:0/2566807572 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557bc0c344d0 0x557bc0c411a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4ee00dfe00 tx=0x7f4ee00a9720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e12741d42c119f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:06.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.157+0000 7f4eccff9700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:20:06.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.157+0000 7f4eccff9700 1 -- 172.21.15.100:0/2566807572 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee00ca050 con 0x557bc0c344d0 2024-05-10T11:20:06.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.157+0000 7f4eccff9700 1 -- 172.21.15.100:0/2566807572 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f4ee00a9a10 con 0x557bc0c344d0 2024-05-10T11:20:06.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.158+0000 7f4ef5f80b80 1 -- 172.21.15.100:0/2566807572 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ee00e2120 con 0x557bc0c344d0 2024-05-10T11:20:06.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.158+0000 7f4eccff9700 1 -- 172.21.15.100:0/2566807572 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4ee0004d80 con 0x557bc0c344d0 2024-05-10T11:20:06.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.158+0000 7f4ef5f80b80 10 client.25199 fetch_fsmap learned FSMap version 27 2024-05-10T11:20:06.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.158+0000 7f4ef5f80b80 1 -- 172.21.15.100:0/2566807572 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557bc0cbc090 con 0x557bc0c344d0 2024-05-10T11:20:06.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.158+0000 7f4eccff9700 1 -- 172.21.15.100:0/2566807572 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4ee00e2120 con 0x557bc0c344d0 2024-05-10T11:20:06.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.158+0000 7f4eccff9700 1 -- 172.21.15.100:0/2566807572 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f4ee00b0ce0 con 0x557bc0c344d0 2024-05-10T11:20:06.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.158+0000 7f4ef5f80b80 10 client.25199 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T11:20:06.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.158+0000 7f4ef5f80b80 10 client.25199 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:20:06.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.158+0000 7f4ef5f80b80 20 client.25199 populate_metadata read hostname 'smithi100' 2024-05-10T11:20:06.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.158+0000 7f4ef5f80b80 1 --2- 172.21.15.100:0/2566807572 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x557bc0cbc370 0x557bc0cbe760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:06.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.158+0000 7f4ef5f80b80 4 client.25199 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:20:06.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.158+0000 7f4ef5f80b80 1 -- 172.21.15.100:0/2566807572 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557bc0b94240 con 0x557bc0cbc370 2024-05-10T11:20:06.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.159+0000 7f4ee8c88700 1 --2- 172.21.15.100:0/2566807572 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x557bc0cbc370 0x557bc0cbe760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:06.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.159+0000 7f4eccff9700 10 client.25199 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:20:06.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.159+0000 7f4ee8c88700 1 --2- 172.21.15.100:0/2566807572 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x557bc0cbc370 0x557bc0cbe760 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39babd1c3ec0fe41 server_cookie=ce76b6feedf8c6fb in_seq=0 out_seq=0 2024-05-10T11:20:06.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.159+0000 7f4eccff9700 1 -- 172.21.15.100:0/2566807572 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557bc0b94240 con 0x557bc0cbc370 2024-05-10T11:20:06.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.159+0000 7f4eccff9700 10 client.25199 handle_command_reply: tid=0 2024-05-10T11:20:06.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.159+0000 7f4ef5f80b80 1 client.25199 shutdown 2024-05-10T11:20:06.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.159+0000 7f4ef5f80b80 1 -- 172.21.15.100:0/2566807572 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557bc0c344d0 msgr2=0x557bc0c411a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:06.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.159+0000 7f4ef5f80b80 1 --2- 172.21.15.100:0/2566807572 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557bc0c344d0 0x557bc0c411a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4ee00dfe00 tx=0x7f4ee00a9720 comp rx=0 tx=0).stop 2024-05-10T11:20:06.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.159+0000 7f4ef5f80b80 1 -- 172.21.15.100:0/2566807572 shutdown_connections 2024-05-10T11:20:06.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.159+0000 7f4ef5f80b80 1 --2- 172.21.15.100:0/2566807572 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x557bc0cbc370 0x557bc0cbe760 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:06.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.159+0000 7f4ef5f80b80 1 --2- 172.21.15.100:0/2566807572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557bc0c38c80 0x557bc0cba960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:06.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.159+0000 7f4ef5f80b80 1 --2- 172.21.15.100:0/2566807572 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557bc0c344d0 0x557bc0c411a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:06.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.159+0000 7f4eccff9700 0 client.25199 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:20:06.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.159+0000 7f4ef5f80b80 1 --2- 172.21.15.100:0/2566807572 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557bc0c34df0 0x557bc0c416e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:06.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.159+0000 7f4ef5f80b80 1 -- 172.21.15.100:0/2566807572 >> 172.21.15.100:0/2566807572 conn(0x557bc0b60100 msgr2=0x557bc0c28e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:06.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.160+0000 7f4ef5f80b80 1 -- 172.21.15.100:0/2566807572 shutdown_connections 2024-05-10T11:20:06.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.160+0000 7f4ef5f80b80 1 -- 172.21.15.100:0/2566807572 wait complete. 2024-05-10T11:20:06.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.160+0000 7f4ef5f80b80 20 client.25199 trim_cache size 0 max 16384 2024-05-10T11:20:06.165 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:20:06.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.160+0000 7f4eeb6ed700 1 -- 172.21.15.100:0/1839175694 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4ed00b29d0 msgr2=0x7f4ed00b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:06.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.160+0000 7f4eeb6ed700 1 --2- 172.21.15.100:0/1839175694 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4ed00b29d0 0x7f4ed00b4e90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4102fe0 tx=0x7f4ed4009260 comp rx=0 tx=0).stop 2024-05-10T11:20:06.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.160+0000 7f4eeb6ed700 1 -- 172.21.15.100:0/1839175694 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ee41037b0 msgr2=0x7f4ee419f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:06.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.160+0000 7f4eeb6ed700 1 --2- 172.21.15.100:0/1839175694 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ee41037b0 0x7f4ee419f610 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4ee000d980 tx=0x7f4ee000dc90 comp rx=0 tx=0).stop 2024-05-10T11:20:06.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.160+0000 7f4eeb6ed700 1 -- 172.21.15.100:0/1839175694 shutdown_connections 2024-05-10T11:20:06.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.160+0000 7f4eeb6ed700 1 --2- 172.21.15.100:0/1839175694 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4ed00b29d0 0x7f4ed00b4e90 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:06.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.160+0000 7f4eeb6ed700 1 --2- 172.21.15.100:0/1839175694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ee4108880 0x7f4ee41997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:06.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.160+0000 7f4eeb6ed700 1 --2- 172.21.15.100:0/1839175694 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ee41037b0 0x7f4ee419f610 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:06.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.160+0000 7f4eeb6ed700 1 --2- 172.21.15.100:0/1839175694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ee4104160 0x7f4ee419fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:06.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.160+0000 7f4eeb6ed700 1 -- 172.21.15.100:0/1839175694 >> 172.21.15.100:0/1839175694 conn(0x7f4ee407a530 msgr2=0x7f4ee4101690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:06.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.161+0000 7f4eeb6ed700 1 -- 172.21.15.100:0/1839175694 shutdown_connections 2024-05-10T11:20:06.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:06.161+0000 7f4eeb6ed700 1 -- 172.21.15.100:0/1839175694 wait complete. 2024-05-10T11:20:06.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:20:05 smithi178 ceph-mon[33359]: pgmap v471: 129 pgs: 129 active+clean; 666 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-10T11:20:06.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:20:05 smithi188 ceph-mon[34075]: pgmap v471: 129 pgs: 129 active+clean; 666 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-10T11:20:06.930 INFO:tasks.workunit.client.0.smithi100.stdout: 1 931 21.78 MB/sec warmup 1 sec latency 13.380 ms 2024-05-10T11:20:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.525+0000 7fda7b7fe640 1 -- 172.21.15.100:0/484256474 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda64031800 con 0x7fda7c102b00 2024-05-10T11:20:07.528 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.526+0000 7fda828fc640 1 -- 172.21.15.100:0/484256474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda7c102b00 msgr2=0x7fda7c102f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:07.528 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.526+0000 7fda828fc640 1 --2- 172.21.15.100:0/484256474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda7c102b00 0x7fda7c102f80 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fda64009900 tx=0x7fda6402a410 comp rx=0 tx=0).stop 2024-05-10T11:20:07.528 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.526+0000 7fda828fc640 1 -- 172.21.15.100:0/484256474 shutdown_connections 2024-05-10T11:20:07.528 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.526+0000 7fda828fc640 1 --2- 172.21.15.100:0/484256474 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda7c1034c0 0x7fda7c109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:07.528 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.526+0000 7fda828fc640 1 --2- 172.21.15.100:0/484256474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda7c102b00 0x7fda7c102f80 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:07.528 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.526+0000 7fda828fc640 1 --2- 172.21.15.100:0/484256474 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda7c101910 0x7fda7c101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:07.528 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.526+0000 7fda828fc640 1 -- 172.21.15.100:0/484256474 >> 172.21.15.100:0/484256474 conn(0x7fda7c0fce80 msgr2=0x7fda7c0ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:07.528 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.526+0000 7fda828fc640 1 -- 172.21.15.100:0/484256474 shutdown_connections 2024-05-10T11:20:07.528 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.526+0000 7fda828fc640 1 -- 172.21.15.100:0/484256474 wait complete. 2024-05-10T11:20:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.527+0000 7fda828fc640 1 Processor -- start 2024-05-10T11:20:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.527+0000 7fda828fc640 1 -- start start 2024-05-10T11:20:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.527+0000 7fda828fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda7c101910 0x7fda7c06ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.527+0000 7fda828fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda7c102b00 0x7fda7c19d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.527+0000 7fda828fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda7c1034c0 0x7fda7c19fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.527+0000 7fda828fc640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fda7c10d320 con 0x7fda7c102b00 2024-05-10T11:20:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.527+0000 7fda828fc640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fda7c10d1a0 con 0x7fda7c1034c0 2024-05-10T11:20:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.527+0000 7fda828fc640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fda7c10d4a0 con 0x7fda7c101910 2024-05-10T11:20:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.527+0000 7fda73fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda7c102b00 0x7fda7c19d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.527+0000 7fda73fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda7c102b00 0x7fda7c19d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45428/0 (socket says 172.21.15.100:45428) 2024-05-10T11:20:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.527+0000 7fda73fff640 1 -- 172.21.15.100:0/3758401729 learned_addr learned my addr 172.21.15.100:0/3758401729 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:20:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.527+0000 7fda7bfff640 1 --2- 172.21.15.100:0/3758401729 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda7c101910 0x7fda7c06ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.527+0000 7fda80e99640 1 --2- 172.21.15.100:0/3758401729 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda7c1034c0 0x7fda7c19fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.527+0000 7fda7bfff640 1 -- 172.21.15.100:0/3758401729 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda7c1034c0 msgr2=0x7fda7c19fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:07.530 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.527+0000 7fda7bfff640 1 --2- 172.21.15.100:0/3758401729 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda7c1034c0 0x7fda7c19fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:07.530 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.527+0000 7fda7bfff640 1 -- 172.21.15.100:0/3758401729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda7c102b00 msgr2=0x7fda7c19d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:07.530 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.527+0000 7fda7bfff640 1 --2- 172.21.15.100:0/3758401729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda7c102b00 0x7fda7c19d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:07.530 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.527+0000 7fda7bfff640 1 -- 172.21.15.100:0/3758401729 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda7c1a0700 con 0x7fda7c101910 2024-05-10T11:20:07.530 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.527+0000 7fda73fff640 1 --2- 172.21.15.100:0/3758401729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda7c102b00 0x7fda7c19d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:20:07.530 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.528+0000 7fda7bfff640 1 --2- 172.21.15.100:0/3758401729 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda7c101910 0x7fda7c06ab80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fda6c00e9c0 tx=0x7fda6c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:07.530 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.528+0000 7fda79ffb640 1 -- 172.21.15.100:0/3758401729 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda6c00ccc0 con 0x7fda7c101910 2024-05-10T11:20:07.530 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.528+0000 7fda828fc640 1 -- 172.21.15.100:0/3758401729 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda7c1a09f0 con 0x7fda7c101910 2024-05-10T11:20:07.530 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.528+0000 7fda79ffb640 1 -- 172.21.15.100:0/3758401729 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fda6c0044b0 con 0x7fda7c101910 2024-05-10T11:20:07.530 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.528+0000 7fda79ffb640 1 -- 172.21.15.100:0/3758401729 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda6c003e10 con 0x7fda7c101910 2024-05-10T11:20:07.530 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.528+0000 7fda828fc640 1 -- 172.21.15.100:0/3758401729 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda7c1a0f60 con 0x7fda7c101910 2024-05-10T11:20:07.536 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.530+0000 7fda85615740 1 Processor -- start 2024-05-10T11:20:07.536 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.530+0000 7fda85615740 1 -- start start 2024-05-10T11:20:07.536 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.531+0000 7fda79ffb640 1 -- 172.21.15.100:0/3758401729 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fda6c003fd0 con 0x7fda7c101910 2024-05-10T11:20:07.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.531+0000 7fda85615740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556c4da38db0 0x556c4da391d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:07.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.531+0000 7fda85615740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c4da397a0 0x556c4da3e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:07.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.531+0000 7fda85615740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c4da3e910 0x556c4da40da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:07.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.531+0000 7fda85615740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x556c4d99d5d0 con 0x556c4da397a0 2024-05-10T11:20:07.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.531+0000 7fda85615740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556c4d956890 con 0x556c4da3e910 2024-05-10T11:20:07.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.531+0000 7fda85615740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556c4d999590 con 0x556c4da38db0 2024-05-10T11:20:07.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.531+0000 7fda7bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c4da397a0 0x556c4da3e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:07.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.531+0000 7fda80e99640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556c4da38db0 0x556c4da391d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:07.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.531+0000 7fda80e99640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556c4da38db0 0x556c4da391d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:40302/0 (socket says 172.21.15.100:40302) 2024-05-10T11:20:07.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.531+0000 7fda80e99640 1 -- 172.21.15.100:0/2467382168 learned_addr learned my addr 172.21.15.100:0/2467382168 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:20:07.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.531+0000 7fda73fff640 1 --2- 172.21.15.100:0/2467382168 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c4da3e910 0x556c4da40da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:07.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.531+0000 7fda80e99640 1 -- 172.21.15.100:0/2467382168 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c4da3e910 msgr2=0x556c4da40da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:07.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.531+0000 7fda80e99640 1 --2- 172.21.15.100:0/2467382168 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c4da3e910 0x556c4da40da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:07.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.531+0000 7fda80e99640 1 -- 172.21.15.100:0/2467382168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c4da397a0 msgr2=0x556c4da3e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:07.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.531+0000 7fda80e99640 1 --2- 172.21.15.100:0/2467382168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c4da397a0 0x556c4da3e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:07.538 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.531+0000 7fda80e99640 1 -- 172.21.15.100:0/2467382168 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c4d6af210 con 0x556c4da38db0 2024-05-10T11:20:07.538 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.531+0000 7fda80e99640 1 --2- 172.21.15.100:0/2467382168 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556c4da38db0 0x556c4da391d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fda74009fc0 tx=0x7fda74010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e01d5e8fcfe605f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:07.538 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.532+0000 7fda727fc640 1 -- 172.21.15.100:0/2467382168 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda74003b70 con 0x556c4da38db0 2024-05-10T11:20:07.538 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.532+0000 7fda727fc640 1 -- 172.21.15.100:0/2467382168 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fda74003d10 con 0x556c4da38db0 2024-05-10T11:20:07.538 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.532+0000 7fda727fc640 1 -- 172.21.15.100:0/2467382168 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda74011980 con 0x556c4da38db0 2024-05-10T11:20:07.538 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.532+0000 7fda79ffb640 1 --2- 172.21.15.100:0/3758401729 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fda680b28a0 0x7fda680b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:07.538 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.532+0000 7fda7bfff640 1 --2- 172.21.15.100:0/3758401729 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fda680b28a0 0x7fda680b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:07.538 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.532+0000 7fda79ffb640 1 -- 172.21.15.100:0/3758401729 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fda6c0e2ae0 con 0x7fda7c101910 2024-05-10T11:20:07.538 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.532+0000 7fda85615740 1 -- 172.21.15.100:0/2467382168 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556c4da38db0 msgr2=0x556c4da391d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:07.538 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.532+0000 7fda85615740 1 --2- 172.21.15.100:0/2467382168 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556c4da38db0 0x556c4da391d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fda74009fc0 tx=0x7fda74010860 comp rx=0 tx=0).stop 2024-05-10T11:20:07.538 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.532+0000 7fda7bfff640 1 --2- 172.21.15.100:0/3758401729 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fda680b28a0 0x7fda680b4d60 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fda6c0e05f0 tx=0x7fda6c0e65c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:07.538 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.532+0000 7fda85615740 1 -- 172.21.15.100:0/2467382168 shutdown_connections 2024-05-10T11:20:07.538 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.532+0000 7fda85615740 1 --2- 172.21.15.100:0/2467382168 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c4da3e910 0x556c4da40da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:07.538 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.532+0000 7fda85615740 1 --2- 172.21.15.100:0/2467382168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c4da397a0 0x556c4da3e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:07.539 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.532+0000 7fda85615740 1 --2- 172.21.15.100:0/2467382168 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556c4da38db0 0x556c4da391d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:07.539 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.532+0000 7fda85615740 1 -- 172.21.15.100:0/2467382168 >> 172.21.15.100:0/2467382168 conn(0x556c4da34420 msgr2=0x556c4da34820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:07.539 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.532+0000 7fda85615740 1 -- 172.21.15.100:0/2467382168 shutdown_connections 2024-05-10T11:20:07.539 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.532+0000 7fda85615740 1 -- 172.21.15.100:0/2467382168 wait complete. 2024-05-10T11:20:07.539 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.533+0000 7fda85615740 1 -- start start 2024-05-10T11:20:07.539 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.533+0000 7fda85615740 1 Processor -- start 2024-05-10T11:20:07.539 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.533+0000 7fda85615740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c4da38db0 0x556c4da43020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:07.539 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.533+0000 7fda85615740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c4da397a0 0x556c4da43560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:07.539 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.533+0000 7fda85615740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556c4da3e910 0x556c4da4a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:07.539 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.533+0000 7fda85615740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x556c4da418c0 con 0x556c4da38db0 2024-05-10T11:20:07.539 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.533+0000 7fda85615740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556c4d95a370 con 0x556c4da397a0 2024-05-10T11:20:07.539 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.533+0000 7fda85615740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556c4da41a40 con 0x556c4da3e910 2024-05-10T11:20:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.533+0000 7fda85615740 10 client.0 _ll_register_callbacks cb 0x556c4d6aea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:20:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.533+0000 7fda80e99640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c4da38db0 0x556c4da43020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.533+0000 7fda80e99640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c4da38db0 0x556c4da43020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45452/0 (socket says 172.21.15.100:45452) 2024-05-10T11:20:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.533+0000 7fda80e99640 1 -- 172.21.15.100:0/3213253317 learned_addr learned my addr 172.21.15.100:0/3213253317 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:20:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.533+0000 7fda73fff640 1 --2- 172.21.15.100:0/3213253317 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c4da397a0 0x556c4da43560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.533+0000 7fda80e99640 1 --2- 172.21.15.100:0/3213253317 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556c4da3e910 0x556c4da4a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.534+0000 7fda80e99640 1 -- 172.21.15.100:0/3213253317 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556c4da3e910 msgr2=0x556c4da4a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.534+0000 7fda80e99640 1 --2- 172.21.15.100:0/3213253317 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556c4da3e910 0x556c4da4a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.534+0000 7fda80e99640 1 -- 172.21.15.100:0/3213253317 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c4da397a0 msgr2=0x556c4da43560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.534+0000 7fda80e99640 1 --2- 172.21.15.100:0/3213253317 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c4da397a0 0x556c4da43560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.534+0000 7fda80e99640 1 -- 172.21.15.100:0/3213253317 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c4d77b620 con 0x556c4da38db0 2024-05-10T11:20:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.534+0000 7fda80e99640 1 --2- 172.21.15.100:0/3213253317 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c4da38db0 0x556c4da43020 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fda740048f0 tx=0x7fda740092c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e932fd1e807de6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.534+0000 7fda5b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:20:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.534+0000 7fda5b7fe640 1 -- 172.21.15.100:0/3213253317 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda74003ec0 con 0x556c4da38db0 2024-05-10T11:20:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.534+0000 7fda5b7fe640 1 -- 172.21.15.100:0/3213253317 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fda74023070 con 0x556c4da38db0 2024-05-10T11:20:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.534+0000 7fda85615740 1 -- 172.21.15.100:0/3213253317 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda74003d10 con 0x556c4da38db0 2024-05-10T11:20:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.534+0000 7fda5b7fe640 1 -- 172.21.15.100:0/3213253317 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fda74004060 con 0x556c4da38db0 2024-05-10T11:20:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.534+0000 7fda85615740 10 client.15705 fetch_fsmap learned FSMap version 27 2024-05-10T11:20:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.534+0000 7fda85615740 1 -- 172.21.15.100:0/3213253317 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556c4da4d560 con 0x556c4da38db0 2024-05-10T11:20:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.534+0000 7fda5b7fe640 1 -- 172.21.15.100:0/3213253317 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fda74003d10 con 0x556c4da38db0 2024-05-10T11:20:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.535+0000 7fda5b7fe640 1 -- 172.21.15.100:0/3213253317 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fda74029040 con 0x556c4da38db0 2024-05-10T11:20:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.535+0000 7fda85615740 10 client.15705 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T11:20:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.535+0000 7fda85615740 10 client.15705 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:20:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.535+0000 7fda85615740 20 client.15705 populate_metadata read hostname 'smithi100' 2024-05-10T11:20:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.535+0000 7fda85615740 1 --2- 172.21.15.100:0/3213253317 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556c4da4e960 0x556c4da50d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.535+0000 7fda85615740 4 client.15705 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:20:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.535+0000 7fda85615740 1 -- 172.21.15.100:0/3213253317 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c4d9ab1e0 con 0x556c4da4e960 2024-05-10T11:20:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.535+0000 7fda73fff640 1 --2- 172.21.15.100:0/3213253317 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556c4da4e960 0x556c4da50d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.535+0000 7fda5b7fe640 10 client.15705 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:20:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.535+0000 7fda73fff640 1 --2- 172.21.15.100:0/3213253317 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556c4da4e960 0x556c4da50d50 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2775ea02beffe80 server_cookie=b5752029e3539176 in_seq=0 out_seq=0 2024-05-10T11:20:07.542 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.536+0000 7fda5b7fe640 1 -- 172.21.15.100:0/3213253317 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556c4d9ab1e0 con 0x556c4da4e960 2024-05-10T11:20:07.542 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.536+0000 7fda5b7fe640 10 client.15705 handle_command_reply: tid=0 2024-05-10T11:20:07.542 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.536+0000 7fda85615740 1 client.15705 shutdown 2024-05-10T11:20:07.542 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.536+0000 7fda85615740 1 -- 172.21.15.100:0/3213253317 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c4da38db0 msgr2=0x556c4da43020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:07.542 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.536+0000 7fda85615740 1 --2- 172.21.15.100:0/3213253317 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c4da38db0 0x556c4da43020 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fda740048f0 tx=0x7fda740092c0 comp rx=0 tx=0).stop 2024-05-10T11:20:07.542 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.536+0000 7fda85615740 1 -- 172.21.15.100:0/3213253317 shutdown_connections 2024-05-10T11:20:07.542 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.536+0000 7fda85615740 1 --2- 172.21.15.100:0/3213253317 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556c4da4e960 0x556c4da50d50 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:07.542 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.536+0000 7fda85615740 1 --2- 172.21.15.100:0/3213253317 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556c4da3e910 0x556c4da4a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:07.542 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.536+0000 7fda85615740 1 --2- 172.21.15.100:0/3213253317 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c4da397a0 0x556c4da43560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:07.542 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.536+0000 7fda5b7fe640 0 client.15705 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:20:07.542 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.536+0000 7fda85615740 1 --2- 172.21.15.100:0/3213253317 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c4da38db0 0x556c4da43020 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:07.542 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.536+0000 7fda85615740 1 -- 172.21.15.100:0/3213253317 >> 172.21.15.100:0/3213253317 conn(0x556c4da34420 msgr2=0x556c4da33580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:07.542 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.536+0000 7fda85615740 1 -- 172.21.15.100:0/3213253317 shutdown_connections 2024-05-10T11:20:07.542 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.536+0000 7fda85615740 1 -- 172.21.15.100:0/3213253317 wait complete. 2024-05-10T11:20:07.543 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.536+0000 7fda85615740 20 client.15705 trim_cache size 0 max 16384 2024-05-10T11:20:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.553+0000 7fda85615740 1 Processor -- start 2024-05-10T11:20:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.553+0000 7fda85615740 1 -- start start 2024-05-10T11:20:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.553+0000 7fda85615740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556c4da38db0 0x556c4da422f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.553+0000 7fda85615740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c4da397a0 0x556c4da42830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.553+0000 7fda85615740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c4da3e910 0x556c4da42d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.553+0000 7fda85615740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fda7c1adbe0 con 0x556c4da3e910 2024-05-10T11:20:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.553+0000 7fda85615740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fda7c1ab4b0 con 0x556c4da397a0 2024-05-10T11:20:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.553+0000 7fda85615740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fda7c1a9300 con 0x556c4da38db0 2024-05-10T11:20:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.554+0000 7fda80e99640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556c4da38db0 0x556c4da422f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.554+0000 7fda80e99640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556c4da38db0 0x556c4da422f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:40318/0 (socket says 172.21.15.100:40318) 2024-05-10T11:20:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.554+0000 7fda73fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c4da397a0 0x556c4da42830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.554+0000 7fda80e99640 1 -- 172.21.15.100:0/3332818183 learned_addr learned my addr 172.21.15.100:0/3332818183 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:20:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.554+0000 7fda80e99640 1 --2- 172.21.15.100:0/3332818183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c4da3e910 0x556c4da42d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.554+0000 7fda80e99640 1 -- 172.21.15.100:0/3332818183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c4da397a0 msgr2=0x556c4da42830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.554+0000 7fda80e99640 1 --2- 172.21.15.100:0/3332818183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c4da397a0 0x556c4da42830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.554+0000 7fda80e99640 1 -- 172.21.15.100:0/3332818183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c4da3e910 msgr2=0x556c4da42d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.554+0000 7fda80e99640 1 --2- 172.21.15.100:0/3332818183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c4da3e910 0x556c4da42d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.554+0000 7fda80e99640 1 -- 172.21.15.100:0/3332818183 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c4da389f0 con 0x556c4da38db0 2024-05-10T11:20:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.554+0000 7fda73fff640 1 --2- 172.21.15.100:0/3332818183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c4da397a0 0x556c4da42830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:20:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.554+0000 7fda80e99640 1 --2- 172.21.15.100:0/3332818183 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556c4da38db0 0x556c4da422f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fda74009ae0 tx=0x7fda74010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=776518b9bee2167b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.554+0000 7fda71ffb640 1 -- 172.21.15.100:0/3332818183 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda74009bd0 con 0x556c4da38db0 2024-05-10T11:20:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.554+0000 7fda71ffb640 1 -- 172.21.15.100:0/3332818183 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fda74004690 con 0x556c4da38db0 2024-05-10T11:20:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.554+0000 7fda71ffb640 1 -- 172.21.15.100:0/3332818183 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda74011570 con 0x556c4da38db0 2024-05-10T11:20:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.555+0000 7fda85615740 1 -- 172.21.15.100:0/3332818183 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556c4da38db0 msgr2=0x556c4da422f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.555+0000 7fda85615740 1 --2- 172.21.15.100:0/3332818183 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556c4da38db0 0x556c4da422f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fda74009ae0 tx=0x7fda74010860 comp rx=0 tx=0).stop 2024-05-10T11:20:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.555+0000 7fda85615740 1 -- 172.21.15.100:0/3332818183 shutdown_connections 2024-05-10T11:20:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.555+0000 7fda85615740 1 --2- 172.21.15.100:0/3332818183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c4da3e910 0x556c4da42d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:07.562 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.555+0000 7fda85615740 1 --2- 172.21.15.100:0/3332818183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c4da397a0 0x556c4da42830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:07.562 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.555+0000 7fda85615740 1 --2- 172.21.15.100:0/3332818183 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556c4da38db0 0x556c4da422f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:07.562 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.555+0000 7fda85615740 1 -- 172.21.15.100:0/3332818183 >> 172.21.15.100:0/3332818183 conn(0x556c4da34420 msgr2=0x556c4da37990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:07.562 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.555+0000 7fda85615740 1 -- 172.21.15.100:0/3332818183 shutdown_connections 2024-05-10T11:20:07.562 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.555+0000 7fda85615740 1 -- 172.21.15.100:0/3332818183 wait complete. 2024-05-10T11:20:07.562 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.555+0000 7fda85615740 1 -- start start 2024-05-10T11:20:07.562 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.556+0000 7fda85615740 1 Processor -- start 2024-05-10T11:20:07.562 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.556+0000 7fda85615740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c4da38db0 0x556c4da47600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:07.562 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.556+0000 7fda85615740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556c4da397a0 0x556c4da47b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:07.562 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.556+0000 7fda85615740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c4da3e910 0x556c4da48b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:07.562 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.556+0000 7fda85615740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fda7c1a9970 con 0x556c4da3e910 2024-05-10T11:20:07.562 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.556+0000 7fda85615740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fda7c1aade0 con 0x556c4da38db0 2024-05-10T11:20:07.562 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.556+0000 7fda85615740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fda7c1aa040 con 0x556c4da397a0 2024-05-10T11:20:07.562 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.556+0000 7fda85615740 10 client.0 _ll_register_callbacks cb 0x7fda6c0e6490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:20:07.563 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.556+0000 7fda80e99640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c4da3e910 0x556c4da48b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:07.563 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.556+0000 7fda80e99640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c4da3e910 0x556c4da48b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45470/0 (socket says 172.21.15.100:45470) 2024-05-10T11:20:07.563 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.556+0000 7fda80e99640 1 -- 172.21.15.100:0/3161030586 learned_addr learned my addr 172.21.15.100:0/3161030586 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:20:07.563 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.556+0000 7fda73fff640 1 --2- 172.21.15.100:0/3161030586 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556c4da397a0 0x556c4da47b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:07.563 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.556+0000 7fda80e99640 1 --2- 172.21.15.100:0/3161030586 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c4da38db0 0x556c4da47600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:07.563 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.556+0000 7fda80e99640 1 -- 172.21.15.100:0/3161030586 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556c4da397a0 msgr2=0x556c4da47b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:07.563 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.556+0000 7fda80e99640 1 --2- 172.21.15.100:0/3161030586 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556c4da397a0 0x556c4da47b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:07.563 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.556+0000 7fda80e99640 1 -- 172.21.15.100:0/3161030586 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c4da38db0 msgr2=0x556c4da47600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:07.563 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.556+0000 7fda80e99640 1 --2- 172.21.15.100:0/3161030586 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c4da38db0 0x556c4da47600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:07.563 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.556+0000 7fda80e99640 1 -- 172.21.15.100:0/3161030586 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c4da34250 con 0x556c4da3e910 2024-05-10T11:20:07.563 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.556+0000 7fda80e99640 1 --2- 172.21.15.100:0/3161030586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c4da3e910 0x556c4da48b50 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fda74004970 tx=0x7fda74004840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3b7d14cc34050eb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:07.563 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.556+0000 7fda5b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:20:07.563 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.557+0000 7fda5b7fe640 1 -- 172.21.15.100:0/3161030586 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda74004200 con 0x556c4da3e910 2024-05-10T11:20:07.563 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.557+0000 7fda5b7fe640 1 -- 172.21.15.100:0/3161030586 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fda74023070 con 0x556c4da3e910 2024-05-10T11:20:07.564 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.557+0000 7fda85615740 1 -- 172.21.15.100:0/3161030586 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda74004690 con 0x556c4da3e910 2024-05-10T11:20:07.564 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.557+0000 7fda5b7fe640 1 -- 172.21.15.100:0/3161030586 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fda740043a0 con 0x556c4da3e910 2024-05-10T11:20:07.564 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.557+0000 7fda85615740 10 client.15711 fetch_fsmap learned FSMap version 27 2024-05-10T11:20:07.564 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.557+0000 7fda85615740 1 -- 172.21.15.100:0/3161030586 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556c4dab9190 con 0x556c4da3e910 2024-05-10T11:20:07.564 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.557+0000 7fda5b7fe640 1 -- 172.21.15.100:0/3161030586 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fda74004690 con 0x556c4da3e910 2024-05-10T11:20:07.564 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.557+0000 7fda5b7fe640 1 -- 172.21.15.100:0/3161030586 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fda74011800 con 0x556c4da3e910 2024-05-10T11:20:07.564 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.557+0000 7fda85615740 10 client.15711 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T11:20:07.564 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.557+0000 7fda85615740 10 client.15711 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:20:07.564 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.557+0000 7fda85615740 20 client.15711 populate_metadata read hostname 'smithi100' 2024-05-10T11:20:07.564 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.557+0000 7fda85615740 1 --2- 172.21.15.100:0/3161030586 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556c4dab9ab0 0x556c4dabbed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:07.564 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.557+0000 7fda85615740 4 client.15711 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:20:07.564 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.557+0000 7fda85615740 1 -- 172.21.15.100:0/3161030586 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556c4d9a5050 con 0x556c4dab9ab0 2024-05-10T11:20:07.564 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.557+0000 7fda73fff640 1 --2- 172.21.15.100:0/3161030586 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556c4dab9ab0 0x556c4dabbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:07.564 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.558+0000 7fda5b7fe640 10 client.15711 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:20:07.565 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.558+0000 7fda73fff640 1 --2- 172.21.15.100:0/3161030586 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556c4dab9ab0 0x556c4dabbed0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=619ac4de697c2e88 server_cookie=20f5ed705f97977b in_seq=0 out_seq=0 2024-05-10T11:20:07.605 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.605+0000 7fda5b7fe640 1 -- 172.21.15.100:0/3161030586 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556c4d9a5050 con 0x556c4dab9ab0 2024-05-10T11:20:07.605 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.605+0000 7fda5b7fe640 10 client.15711 handle_command_reply: tid=0 2024-05-10T11:20:07.605 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.605+0000 7fda85615740 1 client.15711 shutdown 2024-05-10T11:20:07.605 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.605+0000 7fda85615740 1 -- 172.21.15.100:0/3161030586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c4da3e910 msgr2=0x556c4da48b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:07.606 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.605+0000 7fda85615740 1 --2- 172.21.15.100:0/3161030586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c4da3e910 0x556c4da48b50 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fda74004970 tx=0x7fda74004840 comp rx=0 tx=0).stop 2024-05-10T11:20:07.606 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.605+0000 7fda85615740 1 -- 172.21.15.100:0/3161030586 shutdown_connections 2024-05-10T11:20:07.606 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.605+0000 7fda85615740 1 --2- 172.21.15.100:0/3161030586 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556c4dab9ab0 0x556c4dabbed0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:07.606 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.605+0000 7fda85615740 1 --2- 172.21.15.100:0/3161030586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c4da3e910 0x556c4da48b50 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:07.606 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.605+0000 7fda85615740 1 --2- 172.21.15.100:0/3161030586 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556c4da397a0 0x556c4da47b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:07.606 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.605+0000 7fda5b7fe640 0 client.15711 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:20:07.606 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.605+0000 7fda85615740 1 --2- 172.21.15.100:0/3161030586 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c4da38db0 0x556c4da47600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:07.606 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.605+0000 7fda85615740 1 -- 172.21.15.100:0/3161030586 >> 172.21.15.100:0/3161030586 conn(0x556c4da34420 msgr2=0x556c4da32cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:07.606 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.605+0000 7fda85615740 1 -- 172.21.15.100:0/3161030586 shutdown_connections 2024-05-10T11:20:07.606 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.605+0000 7fda85615740 1 -- 172.21.15.100:0/3161030586 wait complete. 2024-05-10T11:20:07.606 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.605+0000 7fda85615740 20 client.15711 trim_cache size 0 max 16384 2024-05-10T11:20:07.607 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:20:07.607 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:20:07.607 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:20:07.607 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:20:07.608 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.606+0000 7fda828fc640 1 -- 172.21.15.100:0/3758401729 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fda680b28a0 msgr2=0x7fda680b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:07.608 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.606+0000 7fda828fc640 1 --2- 172.21.15.100:0/3758401729 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fda680b28a0 0x7fda680b4d60 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fda6c0e05f0 tx=0x7fda6c0e65c0 comp rx=0 tx=0).stop 2024-05-10T11:20:07.608 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.606+0000 7fda828fc640 1 -- 172.21.15.100:0/3758401729 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda7c101910 msgr2=0x7fda7c06ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:07.608 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.606+0000 7fda828fc640 1 --2- 172.21.15.100:0/3758401729 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda7c101910 0x7fda7c06ab80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fda6c00e9c0 tx=0x7fda6c00ee80 comp rx=0 tx=0).stop 2024-05-10T11:20:07.608 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.606+0000 7fda828fc640 1 -- 172.21.15.100:0/3758401729 shutdown_connections 2024-05-10T11:20:07.608 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.606+0000 7fda828fc640 1 --2- 172.21.15.100:0/3758401729 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fda680b28a0 0x7fda680b4d60 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:07.608 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.606+0000 7fda828fc640 1 --2- 172.21.15.100:0/3758401729 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda7c1034c0 0x7fda7c19fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:07.608 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.606+0000 7fda828fc640 1 --2- 172.21.15.100:0/3758401729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda7c102b00 0x7fda7c19d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:07.608 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.606+0000 7fda828fc640 1 --2- 172.21.15.100:0/3758401729 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda7c101910 0x7fda7c06ab80 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:07.608 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.606+0000 7fda828fc640 1 -- 172.21.15.100:0/3758401729 >> 172.21.15.100:0/3758401729 conn(0x7fda7c0fce80 msgr2=0x7fda7c105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:07.608 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.606+0000 7fda828fc640 1 -- 172.21.15.100:0/3758401729 shutdown_connections 2024-05-10T11:20:07.608 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:07.606+0000 7fda828fc640 1 -- 172.21.15.100:0/3758401729 wait complete. 2024-05-10T11:20:07.930 INFO:tasks.workunit.client.0.smithi100.stdout: 1 1660 13.11 MB/sec warmup 2 sec latency 15.500 ms 2024-05-10T11:20:08.021 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T11:20:08.022 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T11:20:08.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:20:07 smithi100 ceph-mon[30296]: pgmap v472: 129 pgs: 129 active+clean; 666 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-10T11:20:08.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:20:07 smithi178 ceph-mon[33359]: pgmap v472: 129 pgs: 129 active+clean; 666 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-10T11:20:08.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:20:07 smithi188 ceph-mon[34075]: pgmap v472: 129 pgs: 129 active+clean; 666 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-10T11:20:08.284 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:20:08.930 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2416 9.37 MB/sec warmup 3 sec latency 15.597 ms 2024-05-10T11:20:09.930 INFO:tasks.workunit.client.0.smithi100.stdout: 1 3311 8.84 MB/sec warmup 4 sec latency 11.589 ms 2024-05-10T11:20:10.048 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:20:09 smithi100 ceph-mon[30296]: pgmap v473: 129 pgs: 129 active+clean; 10 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 891 KiB/s wr, 20 op/s 2024-05-10T11:20:10.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.070+0000 7fa2eb0ca700 1 -- 172.21.15.100:0/4149326393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2e4103390 msgr2=0x7fa2e4107000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:10.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.070+0000 7fa2eb0ca700 1 --2- 172.21.15.100:0/4149326393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2e4103390 0x7fa2e4107000 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fa2d00099c0 tx=0x7fa2d0009cd0 comp rx=0 tx=0).stop 2024-05-10T11:20:10.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.070+0000 7fa2eb0ca700 1 -- 172.21.15.100:0/4149326393 shutdown_connections 2024-05-10T11:20:10.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.070+0000 7fa2eb0ca700 1 --2- 172.21.15.100:0/4149326393 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa2e4107730 0x7fa2e410f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:10.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.070+0000 7fa2eb0ca700 1 --2- 172.21.15.100:0/4149326393 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2e41029e0 0x7fa2e4102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:10.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.070+0000 7fa2eb0ca700 1 --2- 172.21.15.100:0/4149326393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2e4103390 0x7fa2e4107000 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:10.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.070+0000 7fa2eb0ca700 1 -- 172.21.15.100:0/4149326393 >> 172.21.15.100:0/4149326393 conn(0x7fa2e4079350 msgr2=0x7fa2e4079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:10.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.070+0000 7fa2eb0ca700 1 -- 172.21.15.100:0/4149326393 shutdown_connections 2024-05-10T11:20:10.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.070+0000 7fa2eb0ca700 1 -- 172.21.15.100:0/4149326393 wait complete. 2024-05-10T11:20:10.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.071+0000 7fa2eb0ca700 1 Processor -- start 2024-05-10T11:20:10.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.071+0000 7fa2eb0ca700 1 -- start start 2024-05-10T11:20:10.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.071+0000 7fa2eb0ca700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2e41029e0 0x7fa2e41a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:10.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.071+0000 7fa2eb0ca700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2e4103390 0x7fa2e41a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:10.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.071+0000 7fa2eb0ca700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa2e4107730 0x7fa2e419baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:10.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.071+0000 7fa2eb0ca700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e4112430 con 0x7fa2e4103390 2024-05-10T11:20:10.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.071+0000 7fa2eb0ca700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e41122b0 con 0x7fa2e41029e0 2024-05-10T11:20:10.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.071+0000 7fa2eb0ca700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e41125b0 con 0x7fa2e4107730 2024-05-10T11:20:10.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.071+0000 7fa2e3fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2e4103390 0x7fa2e41a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:10.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.071+0000 7fa2e9667700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa2e4107730 0x7fa2e419baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:10.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.071+0000 7fa2e3fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2e4103390 0x7fa2e41a1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45474/0 (socket says 172.21.15.100:45474) 2024-05-10T11:20:10.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.071+0000 7fa2e3fff700 1 -- 172.21.15.100:0/1591371785 learned_addr learned my addr 172.21.15.100:0/1591371785 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:20:10.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.071+0000 7fa2e8e66700 1 --2- 172.21.15.100:0/1591371785 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2e41029e0 0x7fa2e41a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:10.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.071+0000 7fa2e3fff700 1 -- 172.21.15.100:0/1591371785 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa2e4107730 msgr2=0x7fa2e419baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:10.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.071+0000 7fa2e3fff700 1 --2- 172.21.15.100:0/1591371785 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa2e4107730 0x7fa2e419baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:10.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.071+0000 7fa2e3fff700 1 -- 172.21.15.100:0/1591371785 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2e41029e0 msgr2=0x7fa2e41a1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:10.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.071+0000 7fa2e3fff700 1 --2- 172.21.15.100:0/1591371785 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2e41029e0 0x7fa2e41a1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:10.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.071+0000 7fa2e3fff700 1 -- 172.21.15.100:0/1591371785 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2e419c250 con 0x7fa2e4103390 2024-05-10T11:20:10.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.071+0000 7fa2e9667700 1 --2- 172.21.15.100:0/1591371785 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa2e4107730 0x7fa2e419baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:20:10.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.072+0000 7fa2e3fff700 1 --2- 172.21.15.100:0/1591371785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2e4103390 0x7fa2e41a1e70 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fa2d00098e0 tx=0x7fa2d00045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:10.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.072+0000 7fa2e1ffb700 1 -- 172.21.15.100:0/1591371785 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d0026070 con 0x7fa2e4103390 2024-05-10T11:20:10.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.072+0000 7fa2e1ffb700 1 -- 172.21.15.100:0/1591371785 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa2d000f460 con 0x7fa2e4103390 2024-05-10T11:20:10.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.072+0000 7fa2eb0ca700 1 -- 172.21.15.100:0/1591371785 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2e419c4e0 con 0x7fa2e4103390 2024-05-10T11:20:10.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.072+0000 7fa2e1ffb700 1 -- 172.21.15.100:0/1591371785 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d0009e60 con 0x7fa2e4103390 2024-05-10T11:20:10.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.072+0000 7fa2eb0ca700 1 -- 172.21.15.100:0/1591371785 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2e41a8800 con 0x7fa2e4103390 2024-05-10T11:20:10.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.075+0000 7fa2e1ffb700 1 -- 172.21.15.100:0/1591371785 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fa2d0029070 con 0x7fa2e4103390 2024-05-10T11:20:10.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.076+0000 7fa2e1ffb700 1 --2- 172.21.15.100:0/1591371785 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa2d40b2900 0x7fa2d40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:10.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.076+0000 7fa2e8e66700 1 --2- 172.21.15.100:0/1591371785 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa2d40b2900 0x7fa2d40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:10.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.076+0000 7fa2e1ffb700 1 -- 172.21.15.100:0/1591371785 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fa2d00e2d80 con 0x7fa2e4103390 2024-05-10T11:20:10.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.076+0000 7fa2e8e66700 1 --2- 172.21.15.100:0/1591371785 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa2d40b2900 0x7fa2d40b4dc0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8007be0 tx=0x7fa2d80073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:10.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.076+0000 7fa2f595db80 1 Processor -- start 2024-05-10T11:20:10.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.077+0000 7fa2f595db80 1 -- start start 2024-05-10T11:20:10.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.077+0000 7fa2f595db80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56490095e230 0x56490095e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:10.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.077+0000 7fa2f595db80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56490095eb50 0x564900958890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:10.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.077+0000 7fa2f595db80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564900958dd0 0x564900959210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:10.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.077+0000 7fa2f595db80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5649008aa9c0 con 0x56490095eb50 2024-05-10T11:20:10.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.077+0000 7fa2f595db80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56490080fe20 con 0x564900958dd0 2024-05-10T11:20:10.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.077+0000 7fa2f595db80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5649008a6980 con 0x56490095e230 2024-05-10T11:20:10.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.077+0000 7fa2e8e66700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56490095e230 0x56490095e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:10.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.077+0000 7fa2e3fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56490095eb50 0x564900958890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:10.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.077+0000 7fa2e8e66700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56490095e230 0x56490095e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:40360/0 (socket says 172.21.15.100:40360) 2024-05-10T11:20:10.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.077+0000 7fa2e8e66700 1 -- 172.21.15.100:0/448958811 learned_addr learned my addr 172.21.15.100:0/448958811 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:20:10.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.077+0000 7fa2e9667700 1 --2- 172.21.15.100:0/448958811 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564900958dd0 0x564900959210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:10.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.077+0000 7fa2e8e66700 1 -- 172.21.15.100:0/448958811 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564900958dd0 msgr2=0x564900959210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:10.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.077+0000 7fa2e8e66700 1 --2- 172.21.15.100:0/448958811 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564900958dd0 0x564900959210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:10.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.077+0000 7fa2e8e66700 1 -- 172.21.15.100:0/448958811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56490095eb50 msgr2=0x564900958890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:10.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.077+0000 7fa2e8e66700 1 --2- 172.21.15.100:0/448958811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56490095eb50 0x564900958890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:10.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.077+0000 7fa2e8e66700 1 -- 172.21.15.100:0/448958811 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649007f7950 con 0x56490095e230 2024-05-10T11:20:10.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.078+0000 7fa2e8e66700 1 --2- 172.21.15.100:0/448958811 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56490095e230 0x56490095e610 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8010e80 tx=0x7fa2d8013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1cad18f55dcf3688 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:10.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.078+0000 7fa2ceffd700 1 -- 172.21.15.100:0/448958811 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d8024070 con 0x56490095e230 2024-05-10T11:20:10.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.078+0000 7fa2ceffd700 1 -- 172.21.15.100:0/448958811 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa2d8015070 con 0x56490095e230 2024-05-10T11:20:10.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.078+0000 7fa2ceffd700 1 -- 172.21.15.100:0/448958811 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d80118a0 con 0x56490095e230 2024-05-10T11:20:10.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.078+0000 7fa2f595db80 1 -- 172.21.15.100:0/448958811 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56490095e230 msgr2=0x56490095e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:10.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.078+0000 7fa2f595db80 1 --2- 172.21.15.100:0/448958811 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56490095e230 0x56490095e610 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8010e80 tx=0x7fa2d8013000 comp rx=0 tx=0).stop 2024-05-10T11:20:10.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.079+0000 7fa2f595db80 1 -- 172.21.15.100:0/448958811 shutdown_connections 2024-05-10T11:20:10.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.079+0000 7fa2f595db80 1 --2- 172.21.15.100:0/448958811 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564900958dd0 0x564900959210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:10.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.079+0000 7fa2f595db80 1 --2- 172.21.15.100:0/448958811 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56490095e230 0x56490095e610 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:10.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.079+0000 7fa2f595db80 1 --2- 172.21.15.100:0/448958811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56490095eb50 0x564900958890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:10.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.079+0000 7fa2f595db80 1 -- 172.21.15.100:0/448958811 >> 172.21.15.100:0/448958811 conn(0x564900884bf0 msgr2=0x564900953320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:10.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.079+0000 7fa2f595db80 1 -- 172.21.15.100:0/448958811 shutdown_connections 2024-05-10T11:20:10.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.079+0000 7fa2f595db80 1 -- 172.21.15.100:0/448958811 wait complete. 2024-05-10T11:20:10.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.079+0000 7fa2f595db80 1 -- start start 2024-05-10T11:20:10.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.079+0000 7fa2f595db80 1 Processor -- start 2024-05-10T11:20:10.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.079+0000 7fa2f595db80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564900958dd0 0x564900964780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:10.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.080+0000 7fa2f595db80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56490095e230 0x564900961b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:10.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.080+0000 7fa2f595db80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56490095eb50 0x564900962060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:10.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.080+0000 7fa2f595db80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5649009654d0 con 0x56490095eb50 2024-05-10T11:20:10.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.080+0000 7fa2f595db80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x564900863f50 con 0x564900958dd0 2024-05-10T11:20:10.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.080+0000 7fa2f595db80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x564900965650 con 0x56490095e230 2024-05-10T11:20:10.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.080+0000 7fa2f595db80 10 client.0 _ll_register_callbacks cb 0x56490083e630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:20:10.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.080+0000 7fa2e8e66700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564900958dd0 0x564900964780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:10.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.080+0000 7fa2e3fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56490095e230 0x564900961b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:10.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.080+0000 7fa2e3fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56490095e230 0x564900961b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:40364/0 (socket says 172.21.15.100:40364) 2024-05-10T11:20:10.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.080+0000 7fa2e8e66700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564900958dd0 0x564900964780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:58402/0 (socket says 172.21.15.100:58402) 2024-05-10T11:20:10.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.080+0000 7fa2e3fff700 1 -- 172.21.15.100:0/2705673535 learned_addr learned my addr 172.21.15.100:0/2705673535 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:20:10.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.080+0000 7fa2e9667700 1 --2- 172.21.15.100:0/2705673535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56490095eb50 0x564900962060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:10.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.080+0000 7fa2e3fff700 1 -- 172.21.15.100:0/2705673535 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564900958dd0 msgr2=0x564900964780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:10.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.080+0000 7fa2e3fff700 1 --2- 172.21.15.100:0/2705673535 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564900958dd0 0x564900964780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:10.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.080+0000 7fa2e3fff700 1 -- 172.21.15.100:0/2705673535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56490095eb50 msgr2=0x564900962060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:10.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.080+0000 7fa2e3fff700 1 --2- 172.21.15.100:0/2705673535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56490095eb50 0x564900962060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:10.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.080+0000 7fa2e3fff700 1 -- 172.21.15.100:0/2705673535 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649008617e0 con 0x56490095e230 2024-05-10T11:20:10.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.080+0000 7fa2e8e66700 1 --2- 172.21.15.100:0/2705673535 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564900958dd0 0x564900964780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:20:10.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.080+0000 7fa2e3fff700 1 --2- 172.21.15.100:0/2705673535 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56490095e230 0x564900961b20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa2d00a72e0 tx=0x7fa2d000fb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2df415942f99ab37 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:10.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.080+0000 7fa2c3fff700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:20:10.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.080+0000 7fa2c3fff700 1 -- 172.21.15.100:0/2705673535 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d00aa070 con 0x56490095e230 2024-05-10T11:20:10.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.080+0000 7fa2c3fff700 1 -- 172.21.15.100:0/2705673535 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fa2d00ae080 con 0x56490095e230 2024-05-10T11:20:10.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.080+0000 7fa2f595db80 1 -- 172.21.15.100:0/2705673535 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2d8015070 con 0x56490095e230 2024-05-10T11:20:10.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.081+0000 7fa2c3fff700 1 -- 172.21.15.100:0/2705673535 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa2d00ae420 con 0x56490095e230 2024-05-10T11:20:10.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.081+0000 7fa2f595db80 10 client.25228 fetch_fsmap learned FSMap version 27 2024-05-10T11:20:10.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.081+0000 7fa2f595db80 1 -- 172.21.15.100:0/2705673535 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564900965530 con 0x56490095e230 2024-05-10T11:20:10.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.081+0000 7fa2c3fff700 1 -- 172.21.15.100:0/2705673535 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa2d8015070 con 0x56490095e230 2024-05-10T11:20:10.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.081+0000 7fa2c3fff700 1 -- 172.21.15.100:0/2705673535 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa2d00a59e0 con 0x56490095e230 2024-05-10T11:20:10.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.081+0000 7fa2f595db80 10 client.25228 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T11:20:10.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.081+0000 7fa2f595db80 10 client.25228 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:20:10.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.081+0000 7fa2f595db80 20 client.25228 populate_metadata read hostname 'smithi100' 2024-05-10T11:20:10.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.081+0000 7fa2f595db80 1 --2- 172.21.15.100:0/2705673535 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x564900972130 0x5649009745b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:10.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.081+0000 7fa2f595db80 4 client.25228 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:20:10.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.081+0000 7fa2f595db80 1 -- 172.21.15.100:0/2705673535 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649008b83a0 con 0x564900972130 2024-05-10T11:20:10.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.081+0000 7fa2e8e66700 1 --2- 172.21.15.100:0/2705673535 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x564900972130 0x5649009745b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:10.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.082+0000 7fa2c3fff700 10 client.25228 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:20:10.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.082+0000 7fa2e8e66700 1 --2- 172.21.15.100:0/2705673535 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x564900972130 0x5649009745b0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6da05eb0c105364 server_cookie=60aceef6f046e229 in_seq=0 out_seq=0 2024-05-10T11:20:10.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.082+0000 7fa2c3fff700 1 -- 172.21.15.100:0/2705673535 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5649008b83a0 con 0x564900972130 2024-05-10T11:20:10.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.082+0000 7fa2c3fff700 10 client.25228 handle_command_reply: tid=0 2024-05-10T11:20:10.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.082+0000 7fa2f595db80 1 client.25228 shutdown 2024-05-10T11:20:10.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.083+0000 7fa2f595db80 1 -- 172.21.15.100:0/2705673535 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56490095e230 msgr2=0x564900961b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:10.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.083+0000 7fa2f595db80 1 --2- 172.21.15.100:0/2705673535 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56490095e230 0x564900961b20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa2d00a72e0 tx=0x7fa2d000fb80 comp rx=0 tx=0).stop 2024-05-10T11:20:10.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.083+0000 7fa2f595db80 1 -- 172.21.15.100:0/2705673535 shutdown_connections 2024-05-10T11:20:10.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.083+0000 7fa2f595db80 1 --2- 172.21.15.100:0/2705673535 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x564900972130 0x5649009745b0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:10.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.083+0000 7fa2f595db80 1 --2- 172.21.15.100:0/2705673535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56490095eb50 0x564900962060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:10.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.083+0000 7fa2f595db80 1 --2- 172.21.15.100:0/2705673535 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564900958dd0 0x564900964780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:10.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.083+0000 7fa2c3fff700 0 client.25228 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:20:10.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.083+0000 7fa2f595db80 1 --2- 172.21.15.100:0/2705673535 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56490095e230 0x564900961b20 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:10.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.083+0000 7fa2f595db80 1 -- 172.21.15.100:0/2705673535 >> 172.21.15.100:0/2705673535 conn(0x564900884bf0 msgr2=0x56490094dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:10.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.083+0000 7fa2f595db80 1 -- 172.21.15.100:0/2705673535 shutdown_connections 2024-05-10T11:20:10.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.083+0000 7fa2f595db80 1 -- 172.21.15.100:0/2705673535 wait complete. 2024-05-10T11:20:10.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.083+0000 7fa2f595db80 20 client.25228 trim_cache size 0 max 16384 2024-05-10T11:20:10.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.105+0000 7fa2f595db80 1 Processor -- start 2024-05-10T11:20:10.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.105+0000 7fa2f595db80 1 -- start start 2024-05-10T11:20:10.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.105+0000 7fa2f595db80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564900958dd0 0x56490095f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:10.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.105+0000 7fa2f595db80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56490095e230 0x564900960a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:10.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.105+0000 7fa2f595db80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56490095eb50 0x56490096e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:10.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.105+0000 7fa2f595db80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e41b3ae0 con 0x56490095e230 2024-05-10T11:20:10.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.105+0000 7fa2f595db80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e41b1370 con 0x56490095eb50 2024-05-10T11:20:10.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.105+0000 7fa2f595db80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e41af1c0 con 0x564900958dd0 2024-05-10T11:20:10.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.105+0000 7fa2e3fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56490095e230 0x564900960a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:10.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.105+0000 7fa2e3fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56490095e230 0x564900960a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45494/0 (socket says 172.21.15.100:45494) 2024-05-10T11:20:10.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.105+0000 7fa2e3fff700 1 -- 172.21.15.100:0/1839515095 learned_addr learned my addr 172.21.15.100:0/1839515095 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:20:10.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.105+0000 7fa2e8e66700 1 --2- 172.21.15.100:0/1839515095 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564900958dd0 0x56490095f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:10.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.106+0000 7fa2e9667700 1 --2- 172.21.15.100:0/1839515095 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56490095eb50 0x56490096e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:10.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.106+0000 7fa2e3fff700 1 -- 172.21.15.100:0/1839515095 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564900958dd0 msgr2=0x56490095f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:10.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.106+0000 7fa2e3fff700 1 --2- 172.21.15.100:0/1839515095 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564900958dd0 0x56490095f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:10.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.106+0000 7fa2e3fff700 1 -- 172.21.15.100:0/1839515095 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56490095eb50 msgr2=0x56490096e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:10.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.106+0000 7fa2e3fff700 1 --2- 172.21.15.100:0/1839515095 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56490095eb50 0x56490096e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:10.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.106+0000 7fa2e3fff700 1 -- 172.21.15.100:0/1839515095 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564900953df0 con 0x56490095e230 2024-05-10T11:20:10.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.106+0000 7fa2e3fff700 1 --2- 172.21.15.100:0/1839515095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56490095e230 0x564900960a80 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa2d000fbf0 tx=0x7fa2d0009530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=989f9918497c32ae server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:10.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.106+0000 7fa2ce7fc700 1 -- 172.21.15.100:0/1839515095 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d00aa070 con 0x56490095e230 2024-05-10T11:20:10.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.106+0000 7fa2ce7fc700 1 -- 172.21.15.100:0/1839515095 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa2d00b2e40 con 0x56490095e230 2024-05-10T11:20:10.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.106+0000 7fa2ce7fc700 1 -- 172.21.15.100:0/1839515095 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d00b2c10 con 0x56490095e230 2024-05-10T11:20:10.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.107+0000 7fa2f595db80 1 -- 172.21.15.100:0/1839515095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56490095e230 msgr2=0x564900960a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:10.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.107+0000 7fa2f595db80 1 --2- 172.21.15.100:0/1839515095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56490095e230 0x564900960a80 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa2d000fbf0 tx=0x7fa2d0009530 comp rx=0 tx=0).stop 2024-05-10T11:20:10.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.107+0000 7fa2f595db80 1 -- 172.21.15.100:0/1839515095 shutdown_connections 2024-05-10T11:20:10.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.107+0000 7fa2f595db80 1 --2- 172.21.15.100:0/1839515095 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56490095eb50 0x56490096e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:10.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.107+0000 7fa2f595db80 1 --2- 172.21.15.100:0/1839515095 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564900958dd0 0x56490095f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:10.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.107+0000 7fa2f595db80 1 --2- 172.21.15.100:0/1839515095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56490095e230 0x564900960a80 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:10.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.107+0000 7fa2f595db80 1 -- 172.21.15.100:0/1839515095 >> 172.21.15.100:0/1839515095 conn(0x564900884bf0 msgr2=0x5649009631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:10.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.107+0000 7fa2f595db80 1 -- 172.21.15.100:0/1839515095 shutdown_connections 2024-05-10T11:20:10.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.107+0000 7fa2f595db80 1 -- 172.21.15.100:0/1839515095 wait complete. 2024-05-10T11:20:10.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.107+0000 7fa2f595db80 1 -- start start 2024-05-10T11:20:10.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.107+0000 7fa2f595db80 1 Processor -- start 2024-05-10T11:20:10.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.108+0000 7fa2f595db80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564900958dd0 0x564900979a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:10.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.108+0000 7fa2f595db80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56490095e230 0x564900979f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:10.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.108+0000 7fa2f595db80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56490095eb50 0x5649009dba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:10.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.108+0000 7fa2f595db80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e41af830 con 0x56490095eb50 2024-05-10T11:20:10.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.108+0000 7fa2f595db80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e41b0ca0 con 0x564900958dd0 2024-05-10T11:20:10.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.108+0000 7fa2f595db80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e41aff00 con 0x56490095e230 2024-05-10T11:20:10.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.108+0000 7fa2f595db80 10 client.0 _ll_register_callbacks cb 0x564900974a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:20:10.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.108+0000 7fa2e9667700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56490095eb50 0x5649009dba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:10.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.108+0000 7fa2e8e66700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564900958dd0 0x564900979a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:10.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.108+0000 7fa2e9667700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56490095eb50 0x5649009dba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45506/0 (socket says 172.21.15.100:45506) 2024-05-10T11:20:10.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.108+0000 7fa2e9667700 1 -- 172.21.15.100:0/2758435955 learned_addr learned my addr 172.21.15.100:0/2758435955 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:20:10.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.108+0000 7fa2e3fff700 1 --2- 172.21.15.100:0/2758435955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56490095e230 0x564900979f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:10.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.108+0000 7fa2e8e66700 1 -- 172.21.15.100:0/2758435955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56490095e230 msgr2=0x564900979f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:10.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.108+0000 7fa2e8e66700 1 --2- 172.21.15.100:0/2758435955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56490095e230 0x564900979f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:10.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.108+0000 7fa2e8e66700 1 -- 172.21.15.100:0/2758435955 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56490095eb50 msgr2=0x5649009dba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:10.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.108+0000 7fa2e8e66700 1 --2- 172.21.15.100:0/2758435955 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56490095eb50 0x5649009dba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:10.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.108+0000 7fa2e8e66700 1 -- 172.21.15.100:0/2758435955 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649009dbfb0 con 0x564900958dd0 2024-05-10T11:20:10.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.108+0000 7fa2e9667700 1 --2- 172.21.15.100:0/2758435955 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56490095eb50 0x5649009dba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:20:10.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.108+0000 7fa2e8e66700 1 --2- 172.21.15.100:0/2758435955 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564900958dd0 0x564900979a00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8022040 tx=0x7fa2d8010b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc0ae51aae667d01 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:10.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.108+0000 7fa2c3fff700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:20:10.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.109+0000 7fa2c3fff700 1 -- 172.21.15.100:0/2758435955 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d8035070 con 0x564900958dd0 2024-05-10T11:20:10.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.109+0000 7fa2f595db80 1 -- 172.21.15.100:0/2758435955 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2d00b2e40 con 0x564900958dd0 2024-05-10T11:20:10.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.109+0000 7fa2c3fff700 1 -- 172.21.15.100:0/2758435955 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fa2d8003880 con 0x564900958dd0 2024-05-10T11:20:10.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.109+0000 7fa2c3fff700 1 -- 172.21.15.100:0/2758435955 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa2d8041050 con 0x564900958dd0 2024-05-10T11:20:10.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.109+0000 7fa2f595db80 10 client.25232 fetch_fsmap learned FSMap version 27 2024-05-10T11:20:10.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.109+0000 7fa2f595db80 1 -- 172.21.15.100:0/2758435955 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5649009dd0e0 con 0x564900958dd0 2024-05-10T11:20:10.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.109+0000 7fa2c3fff700 1 -- 172.21.15.100:0/2758435955 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa2d00b2e40 con 0x564900958dd0 2024-05-10T11:20:10.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.109+0000 7fa2c3fff700 1 -- 172.21.15.100:0/2758435955 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa2d802a040 con 0x564900958dd0 2024-05-10T11:20:10.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.109+0000 7fa2f595db80 10 client.25232 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T11:20:10.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.109+0000 7fa2f595db80 10 client.25232 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:20:10.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.109+0000 7fa2f595db80 20 client.25232 populate_metadata read hostname 'smithi100' 2024-05-10T11:20:10.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.109+0000 7fa2f595db80 1 --2- 172.21.15.100:0/2758435955 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5649009dd540 0x5649009df960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:10.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.109+0000 7fa2f595db80 4 client.25232 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T11:20:10.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.109+0000 7fa2f595db80 1 -- 172.21.15.100:0/2758435955 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5649008b2440 con 0x5649009dd540 2024-05-10T11:20:10.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.109+0000 7fa2e3fff700 1 --2- 172.21.15.100:0/2758435955 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5649009dd540 0x5649009df960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:10.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.110+0000 7fa2c3fff700 10 client.25232 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:20:10.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.110+0000 7fa2e3fff700 1 --2- 172.21.15.100:0/2758435955 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5649009dd540 0x5649009df960 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed248bb0c00f77e7 server_cookie=60314a130728cd68 in_seq=0 out_seq=0 2024-05-10T11:20:10.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.112+0000 7fa2c3fff700 1 -- 172.21.15.100:0/2758435955 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5649008b2440 con 0x5649009dd540 2024-05-10T11:20:10.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.112+0000 7fa2c3fff700 10 client.25232 handle_command_reply: tid=0 2024-05-10T11:20:10.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.112+0000 7fa2f595db80 1 client.25232 shutdown 2024-05-10T11:20:10.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.112+0000 7fa2f595db80 1 -- 172.21.15.100:0/2758435955 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564900958dd0 msgr2=0x564900979a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:10.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.112+0000 7fa2f595db80 1 --2- 172.21.15.100:0/2758435955 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564900958dd0 0x564900979a00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8022040 tx=0x7fa2d8010b70 comp rx=0 tx=0).stop 2024-05-10T11:20:10.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.112+0000 7fa2f595db80 1 -- 172.21.15.100:0/2758435955 shutdown_connections 2024-05-10T11:20:10.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.112+0000 7fa2f595db80 1 --2- 172.21.15.100:0/2758435955 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5649009dd540 0x5649009df960 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:10.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.112+0000 7fa2f595db80 1 --2- 172.21.15.100:0/2758435955 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56490095eb50 0x5649009dba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:10.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.112+0000 7fa2f595db80 1 --2- 172.21.15.100:0/2758435955 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564900958dd0 0x564900979a00 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:10.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.112+0000 7fa2f595db80 1 --2- 172.21.15.100:0/2758435955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56490095e230 0x564900979f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:10.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.112+0000 7fa2f595db80 1 -- 172.21.15.100:0/2758435955 >> 172.21.15.100:0/2758435955 conn(0x564900884bf0 msgr2=0x56490095f640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:10.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.112+0000 7fa2c3fff700 0 client.25232 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:20:10.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.112+0000 7fa2f595db80 1 -- 172.21.15.100:0/2758435955 shutdown_connections 2024-05-10T11:20:10.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.112+0000 7fa2f595db80 1 -- 172.21.15.100:0/2758435955 wait complete. 2024-05-10T11:20:10.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.112+0000 7fa2f595db80 20 client.25232 trim_cache size 0 max 16384 2024-05-10T11:20:10.117 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:20:10.117 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-05-10T11:20:10.117 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "25c52851-183e-453a-9fe4-75e0308845b0", 2024-05-10T11:20:10.117 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-05-10T11:20:10.117 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:20:10.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.113+0000 7fa2eb0ca700 1 -- 172.21.15.100:0/1591371785 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa2d40b2900 msgr2=0x7fa2d40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:10.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.113+0000 7fa2eb0ca700 1 --2- 172.21.15.100:0/1591371785 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa2d40b2900 0x7fa2d40b4dc0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8007be0 tx=0x7fa2d80073d0 comp rx=0 tx=0).stop 2024-05-10T11:20:10.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.113+0000 7fa2eb0ca700 1 -- 172.21.15.100:0/1591371785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2e4103390 msgr2=0x7fa2e41a1e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:10.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.113+0000 7fa2eb0ca700 1 --2- 172.21.15.100:0/1591371785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2e4103390 0x7fa2e41a1e70 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fa2d00098e0 tx=0x7fa2d00045e0 comp rx=0 tx=0).stop 2024-05-10T11:20:10.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.113+0000 7fa2eb0ca700 1 -- 172.21.15.100:0/1591371785 shutdown_connections 2024-05-10T11:20:10.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.113+0000 7fa2eb0ca700 1 --2- 172.21.15.100:0/1591371785 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa2d40b2900 0x7fa2d40b4dc0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:10.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.113+0000 7fa2eb0ca700 1 --2- 172.21.15.100:0/1591371785 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa2e4107730 0x7fa2e419baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:10.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.113+0000 7fa2eb0ca700 1 --2- 172.21.15.100:0/1591371785 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2e41029e0 0x7fa2e41a1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:10.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.113+0000 7fa2eb0ca700 1 --2- 172.21.15.100:0/1591371785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2e4103390 0x7fa2e41a1e70 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:10.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.113+0000 7fa2eb0ca700 1 -- 172.21.15.100:0/1591371785 >> 172.21.15.100:0/1591371785 conn(0x7fa2e4079350 msgr2=0x7fa2e406bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:10.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.113+0000 7fa2eb0ca700 1 -- 172.21.15.100:0/1591371785 shutdown_connections 2024-05-10T11:20:10.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:10.113+0000 7fa2eb0ca700 1 -- 172.21.15.100:0/1591371785 wait complete. 2024-05-10T11:20:10.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:20:09 smithi178 ceph-mon[33359]: pgmap v473: 129 pgs: 129 active+clean; 10 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 891 KiB/s wr, 20 op/s 2024-05-10T11:20:10.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:20:09 smithi188 ceph-mon[34075]: pgmap v473: 129 pgs: 129 active+clean; 10 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 891 KiB/s wr, 20 op/s 2024-05-10T11:20:10.931 INFO:tasks.workunit.client.0.smithi100.stdout: 1 3876 7.95 MB/sec warmup 5 sec latency 20.439 ms 2024-05-10T11:20:11.058 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:20:10 smithi100 ceph-mon[30296]: scrub queued for path: / 2024-05-10T11:20:11.058 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:20:10 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:20:11.059 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:20:10 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:20:11.059 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:20:10 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:20:11.060 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T11:20:11.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:20:10 smithi178 ceph-mon[33359]: scrub queued for path: / 2024-05-10T11:20:11.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:20:10 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:20:11.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:20:10 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:20:11.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:20:10 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:20:11.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:20:10 smithi188 ceph-mon[34075]: scrub queued for path: / 2024-05-10T11:20:11.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:20:10 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:20:11.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:20:10 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:20:11.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:20:10 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:20:11.380 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:20:11.931 INFO:tasks.workunit.client.0.smithi100.stdout: 1 4383 6.77 MB/sec warmup 6 sec latency 17.863 ms 2024-05-10T11:20:12.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:20:11 smithi100 ceph-mon[30296]: pgmap v474: 129 pgs: 129 active+clean; 17 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 1.4 MiB/s wr, 31 op/s 2024-05-10T11:20:12.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:20:11 smithi178 ceph-mon[33359]: pgmap v474: 129 pgs: 129 active+clean; 17 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 1.4 MiB/s wr, 31 op/s 2024-05-10T11:20:12.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:20:11 smithi188 ceph-mon[34075]: pgmap v474: 129 pgs: 129 active+clean; 17 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 1.4 MiB/s wr, 31 op/s 2024-05-10T11:20:12.931 INFO:tasks.workunit.client.0.smithi100.stdout: 1 5067 6.45 MB/sec warmup 7 sec latency 15.460 ms 2024-05-10T11:20:13.129 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:20:12 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:20:13.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.154+0000 7f8c3d042700 1 -- 172.21.15.100:0/3063440130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c3806b180 msgr2=0x7f8c3806b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:13.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.154+0000 7f8c3d042700 1 --2- 172.21.15.100:0/3063440130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c3806b180 0x7f8c3806b600 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c009990 tx=0x7f8c2c009ca0 comp rx=0 tx=0).stop 2024-05-10T11:20:13.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.154+0000 7f8c3d042700 1 -- 172.21.15.100:0/3063440130 shutdown_connections 2024-05-10T11:20:13.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.154+0000 7f8c3d042700 1 --2- 172.21.15.100:0/3063440130 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c3806bb40 0x7f8c38110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:13.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.154+0000 7f8c3d042700 1 --2- 172.21.15.100:0/3063440130 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c38109510 0x7f8c381098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:13.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.154+0000 7f8c3d042700 1 --2- 172.21.15.100:0/3063440130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c3806b180 0x7f8c3806b600 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:13.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.154+0000 7f8c3d042700 1 -- 172.21.15.100:0/3063440130 >> 172.21.15.100:0/3063440130 conn(0x7f8c38079350 msgr2=0x7f8c38079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:13.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.154+0000 7f8c3d042700 1 -- 172.21.15.100:0/3063440130 shutdown_connections 2024-05-10T11:20:13.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.155+0000 7f8c3d042700 1 -- 172.21.15.100:0/3063440130 wait complete. 2024-05-10T11:20:13.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.155+0000 7f8c3d042700 1 Processor -- start 2024-05-10T11:20:13.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.156+0000 7f8c3d042700 1 -- start start 2024-05-10T11:20:13.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.156+0000 7f8c3d042700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c3806b180 0x7f8c3819f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:13.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.156+0000 7f8c3d042700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c3806bb40 0x7f8c3819fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:13.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.156+0000 7f8c3d042700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c38109510 0x7f8c38199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:13.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.156+0000 7f8c3d042700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8c38113520 con 0x7f8c3806b180 2024-05-10T11:20:13.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.156+0000 7f8c3d042700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8c381133a0 con 0x7f8c38109510 2024-05-10T11:20:13.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.156+0000 7f8c3d042700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8c381136a0 con 0x7f8c3806bb40 2024-05-10T11:20:13.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.156+0000 7f8c36d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c3806b180 0x7f8c3819f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:13.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.156+0000 7f8c36d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c3806b180 0x7f8c3819f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45522/0 (socket says 172.21.15.100:45522) 2024-05-10T11:20:13.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.156+0000 7f8c36d9d700 1 -- 172.21.15.100:0/353491323 learned_addr learned my addr 172.21.15.100:0/353491323 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:20:13.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.156+0000 7f8c3659c700 1 --2- 172.21.15.100:0/353491323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c3806bb40 0x7f8c3819fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:13.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.156+0000 7f8c3759e700 1 --2- 172.21.15.100:0/353491323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c38109510 0x7f8c38199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:13.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.157+0000 7f8c3659c700 1 -- 172.21.15.100:0/353491323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c38109510 msgr2=0x7f8c38199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:13.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.157+0000 7f8c3659c700 1 --2- 172.21.15.100:0/353491323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c38109510 0x7f8c38199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:13.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.157+0000 7f8c3659c700 1 -- 172.21.15.100:0/353491323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c3806b180 msgr2=0x7f8c3819f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:13.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.157+0000 7f8c3659c700 1 --2- 172.21.15.100:0/353491323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c3806b180 0x7f8c3819f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:13.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.157+0000 7f8c3659c700 1 -- 172.21.15.100:0/353491323 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c3819a060 con 0x7f8c3806bb40 2024-05-10T11:20:13.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.157+0000 7f8c36d9d700 1 --2- 172.21.15.100:0/353491323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c3806b180 0x7f8c3819f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:20:13.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.157+0000 7f8c3759e700 1 --2- 172.21.15.100:0/353491323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c38109510 0x7f8c38199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:20:13.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.157+0000 7f8c3659c700 1 --2- 172.21.15.100:0/353491323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c3806bb40 0x7f8c3819fc80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c0098e0 tx=0x7f8c2c0047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:13.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.157+0000 7f8c1ffff700 1 -- 172.21.15.100:0/353491323 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c2c026070 con 0x7f8c3806bb40 2024-05-10T11:20:13.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.157+0000 7f8c3d042700 1 -- 172.21.15.100:0/353491323 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c3819a2f0 con 0x7f8c3806bb40 2024-05-10T11:20:13.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.157+0000 7f8c1ffff700 1 -- 172.21.15.100:0/353491323 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8c2c021410 con 0x7f8c3806bb40 2024-05-10T11:20:13.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.157+0000 7f8c1ffff700 1 -- 172.21.15.100:0/353491323 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c2c017650 con 0x7f8c3806bb40 2024-05-10T11:20:13.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.157+0000 7f8c3d042700 1 -- 172.21.15.100:0/353491323 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c381a6610 con 0x7f8c3806bb40 2024-05-10T11:20:13.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.160+0000 7f8c1ffff700 1 -- 172.21.15.100:0/353491323 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f8c2c029070 con 0x7f8c3806bb40 2024-05-10T11:20:13.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.161+0000 7f8c1ffff700 1 --2- 172.21.15.100:0/353491323 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8c200b2880 0x7f8c200b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:13.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.161+0000 7f8c36d9d700 1 --2- 172.21.15.100:0/353491323 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8c200b2880 0x7f8c200b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:13.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.161+0000 7f8c1ffff700 1 -- 172.21.15.100:0/353491323 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f8c2c0e3330 con 0x7f8c3806bb40 2024-05-10T11:20:13.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.161+0000 7f8c478d5b80 1 Processor -- start 2024-05-10T11:20:13.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.162+0000 7f8c478d5b80 1 -- start start 2024-05-10T11:20:13.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.162+0000 7f8c36d9d700 1 --2- 172.21.15.100:0/353491323 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8c200b2880 0x7f8c200b4d40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8c3819aa40 tx=0x7f8c28009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:13.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.162+0000 7f8c478d5b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba33581b0 0x55eba3358590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:13.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.162+0000 7f8c478d5b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eba3358ad0 0x55eba3352810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:13.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.162+0000 7f8c478d5b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eba3352d50 0x55eba3353190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:13.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.162+0000 7f8c478d5b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55eba32a4940 con 0x55eba3358ad0 2024-05-10T11:20:13.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.162+0000 7f8c478d5b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55eba3209da0 con 0x55eba3352d50 2024-05-10T11:20:13.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.162+0000 7f8c478d5b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55eba32a0900 con 0x55eba33581b0 2024-05-10T11:20:13.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.162+0000 7f8c3659c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eba3358ad0 0x55eba3352810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:13.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.162+0000 7f8c36d9d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba33581b0 0x55eba3358590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:13.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.162+0000 7f8c36d9d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba33581b0 0x55eba3358590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:40400/0 (socket says 172.21.15.100:40400) 2024-05-10T11:20:13.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.162+0000 7f8c36d9d700 1 -- 172.21.15.100:0/1902067046 learned_addr learned my addr 172.21.15.100:0/1902067046 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:20:13.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.162+0000 7f8c3759e700 1 --2- 172.21.15.100:0/1902067046 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eba3352d50 0x55eba3353190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:13.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.162+0000 7f8c36d9d700 1 -- 172.21.15.100:0/1902067046 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eba3352d50 msgr2=0x55eba3353190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:13.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.162+0000 7f8c36d9d700 1 --2- 172.21.15.100:0/1902067046 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eba3352d50 0x55eba3353190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:13.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.162+0000 7f8c36d9d700 1 -- 172.21.15.100:0/1902067046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eba3358ad0 msgr2=0x55eba3352810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:13.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.162+0000 7f8c36d9d700 1 --2- 172.21.15.100:0/1902067046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eba3358ad0 0x55eba3352810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:13.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.162+0000 7f8c36d9d700 1 -- 172.21.15.100:0/1902067046 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eba31f18d0 con 0x55eba33581b0 2024-05-10T11:20:13.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.162+0000 7f8c3659c700 1 --2- 172.21.15.100:0/1902067046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eba3358ad0 0x55eba3352810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:20:13.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.162+0000 7f8c36d9d700 1 --2- 172.21.15.100:0/1902067046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba33581b0 0x55eba3358590 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8c280043b0 tx=0x7f8c28014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e4b403863a1c3c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:13.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.163+0000 7f8c1d7fa700 1 -- 172.21.15.100:0/1902067046 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c28011e60 con 0x55eba33581b0 2024-05-10T11:20:13.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.163+0000 7f8c1d7fa700 1 -- 172.21.15.100:0/1902067046 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8c28003ce0 con 0x55eba33581b0 2024-05-10T11:20:13.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.163+0000 7f8c1d7fa700 1 -- 172.21.15.100:0/1902067046 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c28026660 con 0x55eba33581b0 2024-05-10T11:20:13.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.163+0000 7f8c478d5b80 1 -- 172.21.15.100:0/1902067046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba33581b0 msgr2=0x55eba3358590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:13.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.163+0000 7f8c478d5b80 1 --2- 172.21.15.100:0/1902067046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba33581b0 0x55eba3358590 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8c280043b0 tx=0x7f8c28014000 comp rx=0 tx=0).stop 2024-05-10T11:20:13.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.164+0000 7f8c478d5b80 1 -- 172.21.15.100:0/1902067046 shutdown_connections 2024-05-10T11:20:13.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.164+0000 7f8c478d5b80 1 --2- 172.21.15.100:0/1902067046 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eba3352d50 0x55eba3353190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:13.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.164+0000 7f8c478d5b80 1 --2- 172.21.15.100:0/1902067046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba33581b0 0x55eba3358590 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:13.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.164+0000 7f8c478d5b80 1 --2- 172.21.15.100:0/1902067046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eba3358ad0 0x55eba3352810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:13.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.164+0000 7f8c478d5b80 1 -- 172.21.15.100:0/1902067046 >> 172.21.15.100:0/1902067046 conn(0x55eba327eb70 msgr2=0x55eba334d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:13.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.164+0000 7f8c478d5b80 1 -- 172.21.15.100:0/1902067046 shutdown_connections 2024-05-10T11:20:13.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.164+0000 7f8c478d5b80 1 -- 172.21.15.100:0/1902067046 wait complete. 2024-05-10T11:20:13.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.164+0000 7f8c478d5b80 1 -- start start 2024-05-10T11:20:13.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.164+0000 7f8c478d5b80 1 Processor -- start 2024-05-10T11:20:13.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.164+0000 7f8c478d5b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba3352d50 0x55eba335e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:13.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.164+0000 7f8c478d5b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eba33581b0 0x55eba335bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:13.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.164+0000 7f8c478d5b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eba3358ad0 0x55eba335c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:13.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.164+0000 7f8c478d5b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55eba335f450 con 0x55eba3358ad0 2024-05-10T11:20:13.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.164+0000 7f8c478d5b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55eba325ded0 con 0x55eba33581b0 2024-05-10T11:20:13.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.164+0000 7f8c478d5b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55eba335f5d0 con 0x55eba3352d50 2024-05-10T11:20:13.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.165+0000 7f8c478d5b80 10 client.0 _ll_register_callbacks cb 0x55eba32385b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:20:13.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.165+0000 7f8c3759e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eba3358ad0 0x55eba335c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:13.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.165+0000 7f8c3759e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eba3358ad0 0x55eba335c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45538/0 (socket says 172.21.15.100:45538) 2024-05-10T11:20:13.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.165+0000 7f8c3759e700 1 -- 172.21.15.100:0/3175348432 learned_addr learned my addr 172.21.15.100:0/3175348432 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:20:13.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.165+0000 7f8c36d9d700 1 --2- 172.21.15.100:0/3175348432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba3352d50 0x55eba335e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:13.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.165+0000 7f8c3659c700 1 --2- 172.21.15.100:0/3175348432 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eba33581b0 0x55eba335bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:13.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.165+0000 7f8c36d9d700 1 -- 172.21.15.100:0/3175348432 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eba33581b0 msgr2=0x55eba335bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:13.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.165+0000 7f8c36d9d700 1 --2- 172.21.15.100:0/3175348432 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eba33581b0 0x55eba335bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:13.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.165+0000 7f8c36d9d700 1 -- 172.21.15.100:0/3175348432 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eba3358ad0 msgr2=0x55eba335c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:13.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.165+0000 7f8c36d9d700 1 --2- 172.21.15.100:0/3175348432 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eba3358ad0 0x55eba335c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:13.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.165+0000 7f8c36d9d700 1 -- 172.21.15.100:0/3175348432 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eba325b760 con 0x55eba3352d50 2024-05-10T11:20:13.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.165+0000 7f8c3759e700 1 --2- 172.21.15.100:0/3175348432 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eba3358ad0 0x55eba335c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:20:13.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.165+0000 7f8c36d9d700 1 --2- 172.21.15.100:0/3175348432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba3352d50 0x55eba335e770 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8c28011c70 tx=0x7f8c2801eb00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d927100dc287cdd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:13.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.165+0000 7f8c1aff5700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:20:13.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.165+0000 7f8c1aff5700 1 -- 172.21.15.100:0/3175348432 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c28003e90 con 0x55eba3352d50 2024-05-10T11:20:13.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.165+0000 7f8c478d5b80 1 -- 172.21.15.100:0/3175348432 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c28003ce0 con 0x55eba3352d50 2024-05-10T11:20:13.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.165+0000 7f8c1aff5700 1 -- 172.21.15.100:0/3175348432 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f8c28005260 con 0x55eba3352d50 2024-05-10T11:20:13.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.166+0000 7f8c1aff5700 1 -- 172.21.15.100:0/3175348432 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8c28005460 con 0x55eba3352d50 2024-05-10T11:20:13.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.166+0000 7f8c478d5b80 10 client.25246 fetch_fsmap learned FSMap version 27 2024-05-10T11:20:13.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.166+0000 7f8c478d5b80 1 -- 172.21.15.100:0/3175348432 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55eba3369d30 con 0x55eba3352d50 2024-05-10T11:20:13.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.166+0000 7f8c1aff5700 1 -- 172.21.15.100:0/3175348432 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8c28003ce0 con 0x55eba3352d50 2024-05-10T11:20:13.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.166+0000 7f8c1aff5700 1 -- 172.21.15.100:0/3175348432 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f8c2802e070 con 0x55eba3352d50 2024-05-10T11:20:13.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.166+0000 7f8c478d5b80 10 client.25246 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T11:20:13.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.166+0000 7f8c478d5b80 10 client.25246 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:20:13.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.166+0000 7f8c478d5b80 20 client.25246 populate_metadata read hostname 'smithi100' 2024-05-10T11:20:13.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.166+0000 7f8c478d5b80 1 --2- 172.21.15.100:0/3175348432 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55eba336c200 0x55eba336e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:13.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.166+0000 7f8c478d5b80 4 client.25246 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:20:13.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.166+0000 7f8c478d5b80 1 -- 172.21.15.100:0/3175348432 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eba32b2320 con 0x55eba336c200 2024-05-10T11:20:13.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.166+0000 7f8c3659c700 1 --2- 172.21.15.100:0/3175348432 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55eba336c200 0x55eba336e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:13.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.167+0000 7f8c1aff5700 10 client.25246 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:20:13.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.167+0000 7f8c3659c700 1 --2- 172.21.15.100:0/3175348432 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55eba336c200 0x55eba336e5f0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcc15bccc1e1e330 server_cookie=359bbe25b1532346 in_seq=0 out_seq=0 2024-05-10T11:20:13.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.167+0000 7f8c1aff5700 1 -- 172.21.15.100:0/3175348432 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55eba32b2320 con 0x55eba336c200 2024-05-10T11:20:13.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.167+0000 7f8c1aff5700 10 client.25246 handle_command_reply: tid=0 2024-05-10T11:20:13.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.167+0000 7f8c478d5b80 1 client.25246 shutdown 2024-05-10T11:20:13.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.168+0000 7f8c478d5b80 1 -- 172.21.15.100:0/3175348432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba3352d50 msgr2=0x55eba335e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:13.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.168+0000 7f8c478d5b80 1 --2- 172.21.15.100:0/3175348432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba3352d50 0x55eba335e770 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8c28011c70 tx=0x7f8c2801eb00 comp rx=0 tx=0).stop 2024-05-10T11:20:13.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.168+0000 7f8c478d5b80 1 -- 172.21.15.100:0/3175348432 shutdown_connections 2024-05-10T11:20:13.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.168+0000 7f8c478d5b80 1 --2- 172.21.15.100:0/3175348432 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55eba336c200 0x55eba336e5f0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:13.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.168+0000 7f8c478d5b80 1 --2- 172.21.15.100:0/3175348432 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eba3358ad0 0x55eba335c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:13.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.168+0000 7f8c478d5b80 1 --2- 172.21.15.100:0/3175348432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba3352d50 0x55eba335e770 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:13.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.168+0000 7f8c478d5b80 1 --2- 172.21.15.100:0/3175348432 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eba33581b0 0x55eba335bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:13.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.168+0000 7f8c478d5b80 1 -- 172.21.15.100:0/3175348432 >> 172.21.15.100:0/3175348432 conn(0x55eba327eb70 msgr2=0x55eba3347b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:13.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.168+0000 7f8c1aff5700 0 client.25246 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:20:13.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.168+0000 7f8c478d5b80 1 -- 172.21.15.100:0/3175348432 shutdown_connections 2024-05-10T11:20:13.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.168+0000 7f8c478d5b80 1 -- 172.21.15.100:0/3175348432 wait complete. 2024-05-10T11:20:13.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.168+0000 7f8c478d5b80 20 client.25246 trim_cache size 0 max 16384 2024-05-10T11:20:13.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.191+0000 7f8c478d5b80 1 Processor -- start 2024-05-10T11:20:13.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.191+0000 7f8c478d5b80 1 -- start start 2024-05-10T11:20:13.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.191+0000 7f8c478d5b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba3352d50 0x55eba3352810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:13.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.191+0000 7f8c478d5b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eba33581b0 0x55eba335ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:13.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.191+0000 7f8c478d5b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eba3358ad0 0x55eba3347c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:13.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.191+0000 7f8c478d5b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8c381b3a90 con 0x55eba3358ad0 2024-05-10T11:20:13.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.191+0000 7f8c478d5b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8c381b1320 con 0x55eba33581b0 2024-05-10T11:20:13.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.191+0000 7f8c478d5b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8c381af190 con 0x55eba3352d50 2024-05-10T11:20:13.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.191+0000 7f8c3759e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eba3358ad0 0x55eba3347c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:13.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.191+0000 7f8c36d9d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba3352d50 0x55eba3352810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:13.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.191+0000 7f8c3759e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eba3358ad0 0x55eba3347c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45548/0 (socket says 172.21.15.100:45548) 2024-05-10T11:20:13.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.191+0000 7f8c3759e700 1 -- 172.21.15.100:0/3144730056 learned_addr learned my addr 172.21.15.100:0/3144730056 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:20:13.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.191+0000 7f8c36d9d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba3352d50 0x55eba3352810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:40410/0 (socket says 172.21.15.100:40410) 2024-05-10T11:20:13.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.191+0000 7f8c3659c700 1 --2- 172.21.15.100:0/3144730056 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eba33581b0 0x55eba335ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:13.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.191+0000 7f8c3759e700 1 -- 172.21.15.100:0/3144730056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba3352d50 msgr2=0x55eba3352810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:13.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.191+0000 7f8c3759e700 1 --2- 172.21.15.100:0/3144730056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba3352d50 0x55eba3352810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:13.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.191+0000 7f8c3759e700 1 -- 172.21.15.100:0/3144730056 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eba33581b0 msgr2=0x55eba335ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:13.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.191+0000 7f8c3759e700 1 --2- 172.21.15.100:0/3144730056 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eba33581b0 0x55eba335ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:13.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.191+0000 7f8c3759e700 1 -- 172.21.15.100:0/3144730056 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eba33587e0 con 0x55eba3358ad0 2024-05-10T11:20:13.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.192+0000 7f8c3759e700 1 --2- 172.21.15.100:0/3144730056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eba3358ad0 0x55eba3347c40 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f8c30007b90 tx=0x7f8c3000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1958e9b4d34b88f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:13.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.192+0000 7f8c1cff9700 1 -- 172.21.15.100:0/3144730056 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c30003c00 con 0x55eba3358ad0 2024-05-10T11:20:13.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.192+0000 7f8c1cff9700 1 -- 172.21.15.100:0/3144730056 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8c30003da0 con 0x55eba3358ad0 2024-05-10T11:20:13.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.192+0000 7f8c1cff9700 1 -- 172.21.15.100:0/3144730056 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c300044e0 con 0x55eba3358ad0 2024-05-10T11:20:13.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.193+0000 7f8c478d5b80 1 -- 172.21.15.100:0/3144730056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eba3358ad0 msgr2=0x55eba3347c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:13.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.193+0000 7f8c478d5b80 1 --2- 172.21.15.100:0/3144730056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eba3358ad0 0x55eba3347c40 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f8c30007b90 tx=0x7f8c3000b040 comp rx=0 tx=0).stop 2024-05-10T11:20:13.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.193+0000 7f8c478d5b80 1 -- 172.21.15.100:0/3144730056 shutdown_connections 2024-05-10T11:20:13.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.193+0000 7f8c478d5b80 1 --2- 172.21.15.100:0/3144730056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eba3358ad0 0x55eba3347c40 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:13.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.193+0000 7f8c478d5b80 1 --2- 172.21.15.100:0/3144730056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba3352d50 0x55eba3352810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:13.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.193+0000 7f8c478d5b80 1 --2- 172.21.15.100:0/3144730056 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eba33581b0 0x55eba335ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:13.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.193+0000 7f8c478d5b80 1 -- 172.21.15.100:0/3144730056 >> 172.21.15.100:0/3144730056 conn(0x55eba327eb70 msgr2=0x55eba3356c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:13.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.193+0000 7f8c478d5b80 1 -- 172.21.15.100:0/3144730056 shutdown_connections 2024-05-10T11:20:13.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.193+0000 7f8c478d5b80 1 -- 172.21.15.100:0/3144730056 wait complete. 2024-05-10T11:20:13.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.193+0000 7f8c478d5b80 1 -- start start 2024-05-10T11:20:13.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.193+0000 7f8c478d5b80 1 Processor -- start 2024-05-10T11:20:13.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.194+0000 7f8c478d5b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eba3352d50 0x55eba336e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:13.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.194+0000 7f8c478d5b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba33581b0 0x55eba336ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:13.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.194+0000 7f8c478d5b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eba3358ad0 0x55eba33d4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:13.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.194+0000 7f8c36d9d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eba3352d50 0x55eba336e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:13.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.194+0000 7f8c3659c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba33581b0 0x55eba336ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:13.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.194+0000 7f8c36d9d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eba3352d50 0x55eba336e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:58466/0 (socket says 172.21.15.100:58466) 2024-05-10T11:20:13.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.194+0000 7f8c36d9d700 1 -- 172.21.15.100:0/1961659572 learned_addr learned my addr 172.21.15.100:0/1961659572 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:20:13.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.194+0000 7f8c478d5b80 1 -- 172.21.15.100:0/1961659572 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8c381af7e0 con 0x55eba3358ad0 2024-05-10T11:20:13.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.194+0000 7f8c478d5b80 1 -- 172.21.15.100:0/1961659572 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8c381b0c50 con 0x55eba3352d50 2024-05-10T11:20:13.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.194+0000 7f8c478d5b80 1 -- 172.21.15.100:0/1961659572 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8c381afeb0 con 0x55eba33581b0 2024-05-10T11:20:13.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.194+0000 7f8c478d5b80 10 client.0 _ll_register_callbacks cb 0x55eba32385b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:20:13.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.194+0000 7f8c3759e700 1 --2- 172.21.15.100:0/1961659572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eba3358ad0 0x55eba33d4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:13.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.194+0000 7f8c36d9d700 1 -- 172.21.15.100:0/1961659572 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba33581b0 msgr2=0x55eba336ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:13.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.194+0000 7f8c36d9d700 1 --2- 172.21.15.100:0/1961659572 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba33581b0 0x55eba336ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:13.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.194+0000 7f8c36d9d700 1 -- 172.21.15.100:0/1961659572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eba3358ad0 msgr2=0x55eba33d4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:13.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.194+0000 7f8c36d9d700 1 --2- 172.21.15.100:0/1961659572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eba3358ad0 0x55eba33d4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:13.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.194+0000 7f8c36d9d700 1 -- 172.21.15.100:0/1961659572 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eba33d4c50 con 0x55eba3352d50 2024-05-10T11:20:13.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.195+0000 7f8c36d9d700 1 --2- 172.21.15.100:0/1961659572 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eba3352d50 0x55eba336e510 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8c28030520 tx=0x7f8c28005ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e1717fda3f5610 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:13.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.195+0000 7f8c1aff5700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:20:13.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.195+0000 7f8c1aff5700 1 -- 172.21.15.100:0/1961659572 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c28009340 con 0x55eba3352d50 2024-05-10T11:20:13.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.195+0000 7f8c478d5b80 1 -- 172.21.15.100:0/1961659572 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c30003da0 con 0x55eba3352d50 2024-05-10T11:20:13.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.195+0000 7f8c1aff5700 1 -- 172.21.15.100:0/1961659572 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f8c28009040 con 0x55eba3352d50 2024-05-10T11:20:13.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.195+0000 7f8c1aff5700 1 -- 172.21.15.100:0/1961659572 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8c28005260 con 0x55eba3352d50 2024-05-10T11:20:13.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.195+0000 7f8c478d5b80 10 client.25250 fetch_fsmap learned FSMap version 27 2024-05-10T11:20:13.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.195+0000 7f8c478d5b80 1 -- 172.21.15.100:0/1961659572 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55eba33d5f40 con 0x55eba3352d50 2024-05-10T11:20:13.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.195+0000 7f8c1aff5700 1 -- 172.21.15.100:0/1961659572 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8c30003da0 con 0x55eba3352d50 2024-05-10T11:20:13.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.195+0000 7f8c1aff5700 1 -- 172.21.15.100:0/1961659572 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f8c2802e050 con 0x55eba3352d50 2024-05-10T11:20:13.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.195+0000 7f8c478d5b80 10 client.25250 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T11:20:13.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.195+0000 7f8c478d5b80 10 client.25250 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:20:13.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.195+0000 7f8c478d5b80 20 client.25250 populate_metadata read hostname 'smithi100' 2024-05-10T11:20:13.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.196+0000 7f8c478d5b80 1 --2- 172.21.15.100:0/1961659572 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55eba33d6590 0x55eba33d89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:13.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.196+0000 7f8c478d5b80 4 client.25250 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T11:20:13.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.196+0000 7f8c478d5b80 1 -- 172.21.15.100:0/1961659572 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55eba32ac3c0 con 0x55eba33d6590 2024-05-10T11:20:13.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.196+0000 7f8c3659c700 1 --2- 172.21.15.100:0/1961659572 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55eba33d6590 0x55eba33d89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:13.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.196+0000 7f8c1aff5700 10 client.25250 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:20:13.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.196+0000 7f8c3659c700 1 --2- 172.21.15.100:0/1961659572 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55eba33d6590 0x55eba33d89d0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50c1e83c91e5706c server_cookie=cb291eb4449c5ce9 in_seq=0 out_seq=0 2024-05-10T11:20:13.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.197+0000 7f8c1aff5700 1 -- 172.21.15.100:0/1961659572 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55eba32ac3c0 con 0x55eba33d6590 2024-05-10T11:20:13.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.197+0000 7f8c1aff5700 10 client.25250 handle_command_reply: tid=0 2024-05-10T11:20:13.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.197+0000 7f8c478d5b80 1 client.25250 shutdown 2024-05-10T11:20:13.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.197+0000 7f8c478d5b80 1 -- 172.21.15.100:0/1961659572 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eba3352d50 msgr2=0x55eba336e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:13.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.197+0000 7f8c478d5b80 1 --2- 172.21.15.100:0/1961659572 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eba3352d50 0x55eba336e510 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8c28030520 tx=0x7f8c28005ce0 comp rx=0 tx=0).stop 2024-05-10T11:20:13.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.197+0000 7f8c478d5b80 1 -- 172.21.15.100:0/1961659572 shutdown_connections 2024-05-10T11:20:13.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.197+0000 7f8c478d5b80 1 --2- 172.21.15.100:0/1961659572 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55eba33d6590 0x55eba33d89d0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:13.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.197+0000 7f8c478d5b80 1 --2- 172.21.15.100:0/1961659572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eba3358ad0 0x55eba33d4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:13.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.197+0000 7f8c478d5b80 1 --2- 172.21.15.100:0/1961659572 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eba3352d50 0x55eba336e510 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:13.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.197+0000 7f8c478d5b80 1 --2- 172.21.15.100:0/1961659572 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba33581b0 0x55eba336ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:13.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.197+0000 7f8c478d5b80 1 -- 172.21.15.100:0/1961659572 >> 172.21.15.100:0/1961659572 conn(0x55eba327eb70 msgr2=0x55eba335cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:13.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.197+0000 7f8c1aff5700 0 client.25250 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:20:13.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.197+0000 7f8c478d5b80 1 -- 172.21.15.100:0/1961659572 shutdown_connections 2024-05-10T11:20:13.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.197+0000 7f8c478d5b80 1 -- 172.21.15.100:0/1961659572 wait complete. 2024-05-10T11:20:13.205 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:20:13.205 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T11:20:13.206 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-05-10T11:20:13.206 INFO:teuthology.orchestra.run.smithi100.stdout: "25c52851-183e-453a-9fe4-75e0308845b0": { 2024-05-10T11:20:13.206 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-05-10T11:20:13.206 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "25c52851-183e-453a-9fe4-75e0308845b0", 2024-05-10T11:20:13.206 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-05-10T11:20:13.206 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:20:13.206 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:20:13.206 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:20:13.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.197+0000 7f8c478d5b80 20 client.25250 trim_cache size 0 max 16384 2024-05-10T11:20:13.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.198+0000 7f8c3d042700 1 -- 172.21.15.100:0/353491323 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8c200b2880 msgr2=0x7f8c200b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:13.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.198+0000 7f8c3d042700 1 --2- 172.21.15.100:0/353491323 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8c200b2880 0x7f8c200b4d40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8c3819aa40 tx=0x7f8c28009260 comp rx=0 tx=0).stop 2024-05-10T11:20:13.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.198+0000 7f8c3d042700 1 -- 172.21.15.100:0/353491323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c3806bb40 msgr2=0x7f8c3819fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:13.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.198+0000 7f8c3d042700 1 --2- 172.21.15.100:0/353491323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c3806bb40 0x7f8c3819fc80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c0098e0 tx=0x7f8c2c0047b0 comp rx=0 tx=0).stop 2024-05-10T11:20:13.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.198+0000 7f8c3d042700 1 -- 172.21.15.100:0/353491323 shutdown_connections 2024-05-10T11:20:13.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.198+0000 7f8c3d042700 1 --2- 172.21.15.100:0/353491323 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8c200b2880 0x7f8c200b4d40 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:13.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.198+0000 7f8c3d042700 1 --2- 172.21.15.100:0/353491323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c38109510 0x7f8c38199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:13.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.198+0000 7f8c3d042700 1 --2- 172.21.15.100:0/353491323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c3806b180 0x7f8c3819f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:13.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.198+0000 7f8c3d042700 1 --2- 172.21.15.100:0/353491323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c3806bb40 0x7f8c3819fc80 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:13.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.198+0000 7f8c3d042700 1 -- 172.21.15.100:0/353491323 >> 172.21.15.100:0/353491323 conn(0x7f8c38079350 msgr2=0x7f8c38101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:13.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.198+0000 7f8c3d042700 1 -- 172.21.15.100:0/353491323 shutdown_connections 2024-05-10T11:20:13.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:13.198+0000 7f8c3d042700 1 -- 172.21.15.100:0/353491323 wait complete. 2024-05-10T11:20:13.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:20:12 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:20:13.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:20:12 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:20:13.930 INFO:tasks.workunit.client.0.smithi100.stdout: 1 5866 5.86 MB/sec warmup 8 sec latency 16.639 ms 2024-05-10T11:20:13.944 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:20:13 smithi100 ceph-mon[30296]: pgmap v475: 129 pgs: 129 active+clean; 25 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 2.2 MiB/s wr, 78 op/s 2024-05-10T11:20:13.944 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:20:13 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:20:13.944 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:20:13 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:20:13.944 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:20:13 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:20:13.971 INFO:tasks.cephfs.filesystem:scrub status for tag:25c52851-183e-453a-9fe4-75e0308845b0 - {'path': '/', 'tag': '25c52851-183e-453a-9fe4-75e0308845b0', 'options': 'recursive,force'} 2024-05-10T11:20:14.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:20:13 smithi178 ceph-mon[33359]: pgmap v475: 129 pgs: 129 active+clean; 25 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 2.2 MiB/s wr, 78 op/s 2024-05-10T11:20:14.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:20:13 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:20:14.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:20:13 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:20:14.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:20:13 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:20:14.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:20:13 smithi188 ceph-mon[34075]: pgmap v475: 129 pgs: 129 active+clean; 25 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 2.2 MiB/s wr, 78 op/s 2024-05-10T11:20:14.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:20:13 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:20:14.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:20:13 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:20:14.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:20:13 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:20:14.930 INFO:tasks.workunit.client.0.smithi100.stdout: 1 6779 5.94 MB/sec warmup 9 sec latency 15.963 ms 2024-05-10T11:20:15.179 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:20:14 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:20:15.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:20:14 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:20:15.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:20:14 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:20:15.931 INFO:tasks.workunit.client.0.smithi100.stdout: 1 7333 5.89 MB/sec warmup 10 sec latency 15.473 ms 2024-05-10T11:20:16.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:20:15 smithi178 ceph-mon[33359]: pgmap v476: 129 pgs: 129 active+clean; 25 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 2.5 MiB/s wr, 85 op/s 2024-05-10T11:20:16.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:20:15 smithi188 ceph-mon[34075]: pgmap v476: 129 pgs: 129 active+clean; 25 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 2.5 MiB/s wr, 85 op/s 2024-05-10T11:20:16.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:20:15 smithi100 ceph-mon[30296]: pgmap v476: 129 pgs: 129 active+clean; 25 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 2.5 MiB/s wr, 85 op/s 2024-05-10T11:20:16.931 INFO:tasks.workunit.client.0.smithi100.stdout: 1 7855 5.41 MB/sec warmup 11 sec latency 23.284 ms 2024-05-10T11:20:17.931 INFO:tasks.workunit.client.0.smithi100.stdout: 1 8583 5.36 MB/sec warmup 12 sec latency 15.568 ms 2024-05-10T11:20:18.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:20:17 smithi178 ceph-mon[33359]: pgmap v477: 129 pgs: 129 active+clean; 24 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 2.7 MiB/s wr, 97 op/s 2024-05-10T11:20:18.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:20:17 smithi188 ceph-mon[34075]: pgmap v477: 129 pgs: 129 active+clean; 24 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 2.7 MiB/s wr, 97 op/s 2024-05-10T11:20:18.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:20:17 smithi100 ceph-mon[30296]: pgmap v477: 129 pgs: 129 active+clean; 24 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 2.7 MiB/s wr, 97 op/s 2024-05-10T11:20:18.931 INFO:tasks.workunit.client.0.smithi100.stdout: 1 9401 5.08 MB/sec warmup 13 sec latency 20.504 ms 2024-05-10T11:20:19.931 INFO:tasks.workunit.client.0.smithi100.stdout: 1 10289 5.18 MB/sec warmup 14 sec latency 37.869 ms 2024-05-10T11:20:20.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:20:19 smithi178 ceph-mon[33359]: pgmap v478: 129 pgs: 129 active+clean; 26 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 3.1 MiB/s wr, 122 op/s 2024-05-10T11:20:20.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:20:19 smithi188 ceph-mon[34075]: pgmap v478: 129 pgs: 129 active+clean; 26 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 3.1 MiB/s wr, 122 op/s 2024-05-10T11:20:20.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:20:19 smithi100 ceph-mon[30296]: pgmap v478: 129 pgs: 129 active+clean; 26 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 3.1 MiB/s wr, 122 op/s 2024-05-10T11:20:20.702 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.701+0000 7f54b5189640 1 -- 172.21.15.100:0/1296768522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54b0103490 msgr2=0x7f54b0105880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:20.702 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.701+0000 7f54b5189640 1 --2- 172.21.15.100:0/1296768522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54b0103490 0x7f54b0105880 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f54a0009900 tx=0x7f54a002a410 comp rx=0 tx=0).stop 2024-05-10T11:20:20.702 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.701+0000 7f54b5189640 1 -- 172.21.15.100:0/1296768522 shutdown_connections 2024-05-10T11:20:20.702 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.701+0000 7f54b5189640 1 --2- 172.21.15.100:0/1296768522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f54b01086f0 0x7f54b010ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:20.702 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.702+0000 7f54b5189640 1 --2- 172.21.15.100:0/1296768522 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54b0105dc0 0x7f54b01081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:20.702 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.702+0000 7f54b5189640 1 --2- 172.21.15.100:0/1296768522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54b0103490 0x7f54b0105880 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:20.702 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.702+0000 7f54b5189640 1 -- 172.21.15.100:0/1296768522 >> 172.21.15.100:0/1296768522 conn(0x7f54b00fce80 msgr2=0x7f54b00ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:20.703 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.702+0000 7f54b5189640 1 -- 172.21.15.100:0/1296768522 shutdown_connections 2024-05-10T11:20:20.703 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.702+0000 7f54b5189640 1 -- 172.21.15.100:0/1296768522 wait complete. 2024-05-10T11:20:20.703 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.702+0000 7f54b5189640 1 Processor -- start 2024-05-10T11:20:20.705 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.703+0000 7f54b5189640 1 -- start start 2024-05-10T11:20:20.705 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.703+0000 7f54b5189640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f54b0103490 0x7f54b0198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:20.705 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.703+0000 7f54b5189640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54b0105dc0 0x7f54b0199060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:20.705 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.703+0000 7f54b5189640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54b01086f0 0x7f54b01a00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:20.705 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.703+0000 7f54b5189640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f54b010e420 con 0x7f54b01086f0 2024-05-10T11:20:20.705 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.703+0000 7f54b5189640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f54b010e2a0 con 0x7f54b0103490 2024-05-10T11:20:20.705 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.703+0000 7f54b5189640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f54b010e5a0 con 0x7f54b0105dc0 2024-05-10T11:20:20.705 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.703+0000 7f54af59e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54b01086f0 0x7f54b01a00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:20.705 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.703+0000 7f54af59e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54b01086f0 0x7f54b01a00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35456/0 (socket says 172.21.15.100:35456) 2024-05-10T11:20:20.706 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.703+0000 7f54af59e640 1 -- 172.21.15.100:0/2410725880 learned_addr learned my addr 172.21.15.100:0/2410725880 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:20:20.706 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.703+0000 7f54aed9d640 1 --2- 172.21.15.100:0/2410725880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f54b0103490 0x7f54b0198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:20.706 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.703+0000 7f54ae59c640 1 --2- 172.21.15.100:0/2410725880 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54b0105dc0 0x7f54b0199060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:20.706 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.703+0000 7f54af59e640 1 -- 172.21.15.100:0/2410725880 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54b0105dc0 msgr2=0x7f54b0199060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:20.706 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.703+0000 7f54af59e640 1 --2- 172.21.15.100:0/2410725880 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54b0105dc0 0x7f54b0199060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:20.706 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.703+0000 7f54af59e640 1 -- 172.21.15.100:0/2410725880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f54b0103490 msgr2=0x7f54b0198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:20.706 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.703+0000 7f54af59e640 1 --2- 172.21.15.100:0/2410725880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f54b0103490 0x7f54b0198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:20.706 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.703+0000 7f54af59e640 1 -- 172.21.15.100:0/2410725880 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54b01a0740 con 0x7f54b01086f0 2024-05-10T11:20:20.706 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.703+0000 7f54aed9d640 1 --2- 172.21.15.100:0/2410725880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f54b0103490 0x7f54b0198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:20:20.706 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.703+0000 7f54af59e640 1 --2- 172.21.15.100:0/2410725880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54b01086f0 0x7f54b01a00d0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f54a800eb10 tx=0x7f54a800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:20.706 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.704+0000 7f5497fff640 1 -- 172.21.15.100:0/2410725880 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a8004b20 con 0x7f54b01086f0 2024-05-10T11:20:20.706 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.704+0000 7f54b5189640 1 -- 172.21.15.100:0/2410725880 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54b01a0a30 con 0x7f54b01086f0 2024-05-10T11:20:20.706 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.704+0000 7f5497fff640 1 -- 172.21.15.100:0/2410725880 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f54a8004cc0 con 0x7f54b01086f0 2024-05-10T11:20:20.706 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.704+0000 7f5497fff640 1 -- 172.21.15.100:0/2410725880 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a8010690 con 0x7f54b01086f0 2024-05-10T11:20:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.704+0000 7f54b5189640 1 -- 172.21.15.100:0/2410725880 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54b01a0f80 con 0x7f54b01086f0 2024-05-10T11:20:20.713 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.706+0000 7f5497fff640 1 -- 172.21.15.100:0/2410725880 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f54a8005020 con 0x7f54b01086f0 2024-05-10T11:20:20.713 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.707+0000 7f54b7f80740 1 Processor -- start 2024-05-10T11:20:20.713 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.707+0000 7f54b7f80740 1 -- start start 2024-05-10T11:20:20.713 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.707+0000 7f54b7f80740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55733bfef1c0 0x55733bfef5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:20.713 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.707+0000 7f54b7f80740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55733bfefbb0 0x55733bff46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:20.713 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.707+0000 7f54b7f80740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55733bff4d20 0x55733bff71b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:20.713 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.707+0000 7f54b7f80740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55733bf4f5d0 con 0x55733bff4d20 2024-05-10T11:20:20.713 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.707+0000 7f54b7f80740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55733bf08890 con 0x55733bfefbb0 2024-05-10T11:20:20.713 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.707+0000 7f54b7f80740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55733bf4b590 con 0x55733bfef1c0 2024-05-10T11:20:20.714 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.707+0000 7f54ae59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55733bff4d20 0x55733bff71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:20.714 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.707+0000 7f54ae59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55733bff4d20 0x55733bff71b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35466/0 (socket says 172.21.15.100:35466) 2024-05-10T11:20:20.714 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.707+0000 7f54ae59c640 1 -- 172.21.15.100:0/356050814 learned_addr learned my addr 172.21.15.100:0/356050814 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:20:20.714 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.707+0000 7f54ae59c640 1 --2- 172.21.15.100:0/356050814 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55733bfef1c0 0x55733bfef5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:20.714 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.707+0000 7f54aed9d640 1 --2- 172.21.15.100:0/356050814 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55733bfefbb0 0x55733bff46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:20.714 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.707+0000 7f5497fff640 1 --2- 172.21.15.100:0/2410725880 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f54980b28a0 0x7f54980b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:20.714 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.707+0000 7f54ae59c640 1 -- 172.21.15.100:0/356050814 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55733bfef1c0 msgr2=0x55733bfef5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:20.714 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.707+0000 7f54ae59c640 1 --2- 172.21.15.100:0/356050814 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55733bfef1c0 0x55733bfef5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:20.714 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.707+0000 7f54ae59c640 1 -- 172.21.15.100:0/356050814 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55733bfefbb0 msgr2=0x55733bff46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:20.714 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.707+0000 7f54ae59c640 1 --2- 172.21.15.100:0/356050814 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55733bfefbb0 0x55733bff46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:20.714 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.707+0000 7f54ae59c640 1 -- 172.21.15.100:0/356050814 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55733bc61210 con 0x55733bff4d20 2024-05-10T11:20:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.707+0000 7f54af59e640 1 --2- 172.21.15.100:0/2410725880 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f54980b28a0 0x7f54980b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.708+0000 7f54ae59c640 1 --2- 172.21.15.100:0/356050814 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55733bff4d20 0x55733bff71b0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f54a4009fd0 tx=0x7f54a4005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc18b0e0a1a9643c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.708+0000 7f5497fff640 1 -- 172.21.15.100:0/2410725880 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f54a80e2b20 con 0x7f54b01086f0 2024-05-10T11:20:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.708+0000 7f54957fa640 1 -- 172.21.15.100:0/356050814 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a4004030 con 0x55733bff4d20 2024-05-10T11:20:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.708+0000 7f54957fa640 1 -- 172.21.15.100:0/356050814 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f54a400f790 con 0x55733bff4d20 2024-05-10T11:20:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.708+0000 7f54957fa640 1 -- 172.21.15.100:0/356050814 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a4003850 con 0x55733bff4d20 2024-05-10T11:20:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.708+0000 7f54b7f80740 1 -- 172.21.15.100:0/356050814 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55733bff4d20 msgr2=0x55733bff71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.708+0000 7f54af59e640 1 --2- 172.21.15.100:0/2410725880 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f54980b28a0 0x7f54980b4d60 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f54a80a9d70 tx=0x7f54a80d93f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.708+0000 7f54b7f80740 1 --2- 172.21.15.100:0/356050814 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55733bff4d20 0x55733bff71b0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f54a4009fd0 tx=0x7f54a4005850 comp rx=0 tx=0).stop 2024-05-10T11:20:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.709+0000 7f54b7f80740 1 -- 172.21.15.100:0/356050814 shutdown_connections 2024-05-10T11:20:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.709+0000 7f54b7f80740 1 --2- 172.21.15.100:0/356050814 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55733bff4d20 0x55733bff71b0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.709+0000 7f54b7f80740 1 --2- 172.21.15.100:0/356050814 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55733bfefbb0 0x55733bff46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.709+0000 7f54b7f80740 1 --2- 172.21.15.100:0/356050814 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55733bfef1c0 0x55733bfef5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.709+0000 7f54b7f80740 1 -- 172.21.15.100:0/356050814 >> 172.21.15.100:0/356050814 conn(0x55733bfe6420 msgr2=0x55733bfe6820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.709+0000 7f54b7f80740 1 -- 172.21.15.100:0/356050814 shutdown_connections 2024-05-10T11:20:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.709+0000 7f54b7f80740 1 -- 172.21.15.100:0/356050814 wait complete. 2024-05-10T11:20:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.709+0000 7f54b7f80740 1 -- start start 2024-05-10T11:20:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.709+0000 7f54b7f80740 1 Processor -- start 2024-05-10T11:20:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.709+0000 7f54b7f80740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55733bfef1c0 0x55733bff93b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.709+0000 7f54b7f80740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55733bfefbb0 0x55733bff98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.709+0000 7f54b7f80740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55733bff4d20 0x55733c000930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.709+0000 7f54b7f80740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55733bff7cd0 con 0x55733bfef1c0 2024-05-10T11:20:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.709+0000 7f54b7f80740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55733bf0c370 con 0x55733bfefbb0 2024-05-10T11:20:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.709+0000 7f54b7f80740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55733bff7e50 con 0x55733bff4d20 2024-05-10T11:20:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.709+0000 7f54b7f80740 10 client.0 _ll_register_callbacks cb 0x55733bc60a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:20:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.709+0000 7f54ae59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55733bfef1c0 0x55733bff93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.709+0000 7f54ae59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55733bfef1c0 0x55733bff93b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35468/0 (socket says 172.21.15.100:35468) 2024-05-10T11:20:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.709+0000 7f54ae59c640 1 -- 172.21.15.100:0/493547469 learned_addr learned my addr 172.21.15.100:0/493547469 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:20:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.709+0000 7f54aed9d640 1 --2- 172.21.15.100:0/493547469 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55733bfefbb0 0x55733bff98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.709+0000 7f54ae59c640 1 --2- 172.21.15.100:0/493547469 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55733bff4d20 0x55733c000930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.710+0000 7f54ae59c640 1 -- 172.21.15.100:0/493547469 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55733bff4d20 msgr2=0x55733c000930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.710+0000 7f54ae59c640 1 --2- 172.21.15.100:0/493547469 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55733bff4d20 0x55733c000930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.710+0000 7f54ae59c640 1 -- 172.21.15.100:0/493547469 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55733bfefbb0 msgr2=0x55733bff98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.710+0000 7f54ae59c640 1 --2- 172.21.15.100:0/493547469 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55733bfefbb0 0x55733bff98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.710+0000 7f54ae59c640 1 -- 172.21.15.100:0/493547469 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55733bd2d620 con 0x55733bfef1c0 2024-05-10T11:20:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.710+0000 7f54aed9d640 1 --2- 172.21.15.100:0/493547469 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55733bfefbb0 0x55733bff98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:20:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.710+0000 7f54ae59c640 1 --2- 172.21.15.100:0/493547469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55733bfef1c0 0x55733bff93b0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f54a4006010 tx=0x7f54a40056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25fef6fe74184b57 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.710+0000 7f5492ff5640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:20:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.710+0000 7f5492ff5640 1 -- 172.21.15.100:0/493547469 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a402c060 con 0x55733bfef1c0 2024-05-10T11:20:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.710+0000 7f54b7f80740 1 -- 172.21.15.100:0/493547469 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54a400f790 con 0x55733bfef1c0 2024-05-10T11:20:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.710+0000 7f5492ff5640 1 -- 172.21.15.100:0/493547469 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f54a4004410 con 0x55733bfef1c0 2024-05-10T11:20:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.710+0000 7f5492ff5640 1 -- 172.21.15.100:0/493547469 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f54a4033050 con 0x55733bfef1c0 2024-05-10T11:20:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.710+0000 7f54b7f80740 10 client.15759 fetch_fsmap learned FSMap version 27 2024-05-10T11:20:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.710+0000 7f54b7f80740 1 -- 172.21.15.100:0/493547469 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55733c003910 con 0x55733bfef1c0 2024-05-10T11:20:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.710+0000 7f5492ff5640 1 -- 172.21.15.100:0/493547469 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f54a400f790 con 0x55733bfef1c0 2024-05-10T11:20:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.710+0000 7f5492ff5640 1 -- 172.21.15.100:0/493547469 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f54a401f480 con 0x55733bfef1c0 2024-05-10T11:20:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.710+0000 7f54b7f80740 10 client.15759 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T11:20:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.710+0000 7f54b7f80740 10 client.15759 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:20:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.711+0000 7f54b7f80740 20 client.15759 populate_metadata read hostname 'smithi100' 2024-05-10T11:20:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.711+0000 7f54b7f80740 1 --2- 172.21.15.100:0/493547469 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55733c004d80 0x55733c007170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.711+0000 7f54b7f80740 4 client.15759 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:20:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.711+0000 7f54b7f80740 1 -- 172.21.15.100:0/493547469 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55733bf5d1e0 con 0x55733c004d80 2024-05-10T11:20:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.711+0000 7f54aed9d640 1 --2- 172.21.15.100:0/493547469 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55733c004d80 0x55733c007170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.711+0000 7f5492ff5640 10 client.15759 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:20:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.711+0000 7f54aed9d640 1 --2- 172.21.15.100:0/493547469 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55733c004d80 0x55733c007170 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a573257ebf7b48f7 server_cookie=c640b4f7eb529389 in_seq=0 out_seq=0 2024-05-10T11:20:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.712+0000 7f5492ff5640 1 -- 172.21.15.100:0/493547469 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55733bf5d1e0 con 0x55733c004d80 2024-05-10T11:20:20.719 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.712+0000 7f5492ff5640 10 client.15759 handle_command_reply: tid=0 2024-05-10T11:20:20.719 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.712+0000 7f54b7f80740 1 client.15759 shutdown 2024-05-10T11:20:20.719 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.712+0000 7f54b7f80740 1 -- 172.21.15.100:0/493547469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55733bfef1c0 msgr2=0x55733bff93b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:20.719 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.712+0000 7f54b7f80740 1 --2- 172.21.15.100:0/493547469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55733bfef1c0 0x55733bff93b0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f54a4006010 tx=0x7f54a40056d0 comp rx=0 tx=0).stop 2024-05-10T11:20:20.719 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.712+0000 7f54b7f80740 1 -- 172.21.15.100:0/493547469 shutdown_connections 2024-05-10T11:20:20.719 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.712+0000 7f54b7f80740 1 --2- 172.21.15.100:0/493547469 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55733c004d80 0x55733c007170 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:20.719 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.712+0000 7f54b7f80740 1 --2- 172.21.15.100:0/493547469 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55733bff4d20 0x55733c000930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:20.719 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.712+0000 7f54b7f80740 1 --2- 172.21.15.100:0/493547469 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55733bfefbb0 0x55733bff98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:20.719 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.712+0000 7f54b7f80740 1 --2- 172.21.15.100:0/493547469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55733bfef1c0 0x55733bff93b0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:20.719 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.712+0000 7f5492ff5640 0 client.15759 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:20:20.719 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.712+0000 7f54b7f80740 1 -- 172.21.15.100:0/493547469 >> 172.21.15.100:0/493547469 conn(0x55733bfe6420 msgr2=0x55733bfe5580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:20.719 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.712+0000 7f54b7f80740 1 -- 172.21.15.100:0/493547469 shutdown_connections 2024-05-10T11:20:20.719 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.712+0000 7f54b7f80740 1 -- 172.21.15.100:0/493547469 wait complete. 2024-05-10T11:20:20.719 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.712+0000 7f54b7f80740 20 client.15759 trim_cache size 0 max 16384 2024-05-10T11:20:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.728+0000 7f54b7f80740 1 Processor -- start 2024-05-10T11:20:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.728+0000 7f54b7f80740 1 -- start start 2024-05-10T11:20:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.728+0000 7f54b7f80740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55733bfef1c0 0x55733bff9850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.728+0000 7f54b7f80740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55733bfefbb0 0x55733c0008c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.728+0000 7f54b7f80740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55733bff4d20 0x55733bfe9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.728+0000 7f54b7f80740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f54b01aba10 con 0x55733bfefbb0 2024-05-10T11:20:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.728+0000 7f54b7f80740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f54b01a92e0 con 0x55733bfef1c0 2024-05-10T11:20:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.728+0000 7f54b7f80740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f54b01a7130 con 0x55733bff4d20 2024-05-10T11:20:20.734 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.729+0000 7f54aed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55733bfefbb0 0x55733c0008c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:20.734 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.729+0000 7f54aed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55733bfefbb0 0x55733c0008c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35470/0 (socket says 172.21.15.100:35470) 2024-05-10T11:20:20.734 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.729+0000 7f54aed9d640 1 -- 172.21.15.100:0/3337751291 learned_addr learned my addr 172.21.15.100:0/3337751291 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:20:20.734 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.729+0000 7f54ae59c640 1 --2- 172.21.15.100:0/3337751291 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55733bfef1c0 0x55733bff9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:20.734 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.729+0000 7f54ae59c640 1 --2- 172.21.15.100:0/3337751291 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55733bff4d20 0x55733bfe9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:20.734 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.729+0000 7f54aed9d640 1 -- 172.21.15.100:0/3337751291 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55733bff4d20 msgr2=0x55733bfe9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:20.734 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.729+0000 7f54aed9d640 1 --2- 172.21.15.100:0/3337751291 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55733bff4d20 0x55733bfe9810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:20.734 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.729+0000 7f54aed9d640 1 -- 172.21.15.100:0/3337751291 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55733bfef1c0 msgr2=0x55733bff9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:20.734 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.729+0000 7f54aed9d640 1 --2- 172.21.15.100:0/3337751291 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55733bfef1c0 0x55733bff9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:20.734 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.729+0000 7f54aed9d640 1 -- 172.21.15.100:0/3337751291 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55733bfe6250 con 0x55733bfefbb0 2024-05-10T11:20:20.734 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.729+0000 7f54ae59c640 1 --2- 172.21.15.100:0/3337751291 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55733bfef1c0 0x55733bff9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:20:20.734 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.729+0000 7f54aed9d640 1 --2- 172.21.15.100:0/3337751291 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55733bfefbb0 0x55733c0008c0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f54a00096c0 tx=0x7f54a0031bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b2f7b5827d84e9f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:20.734 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.729+0000 7f5494ff9640 1 -- 172.21.15.100:0/3337751291 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a0035040 con 0x55733bfefbb0 2024-05-10T11:20:20.734 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.729+0000 7f5494ff9640 1 -- 172.21.15.100:0/3337751291 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f54a004b070 con 0x55733bfefbb0 2024-05-10T11:20:20.735 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.729+0000 7f5494ff9640 1 -- 172.21.15.100:0/3337751291 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a00035f0 con 0x55733bfefbb0 2024-05-10T11:20:20.735 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.730+0000 7f54b7f80740 1 -- 172.21.15.100:0/3337751291 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55733bfefbb0 msgr2=0x55733c0008c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:20.735 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.730+0000 7f54b7f80740 1 --2- 172.21.15.100:0/3337751291 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55733bfefbb0 0x55733c0008c0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f54a00096c0 tx=0x7f54a0031bc0 comp rx=0 tx=0).stop 2024-05-10T11:20:20.735 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.730+0000 7f54b7f80740 1 -- 172.21.15.100:0/3337751291 shutdown_connections 2024-05-10T11:20:20.735 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.730+0000 7f54b7f80740 1 --2- 172.21.15.100:0/3337751291 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55733bff4d20 0x55733bfe9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:20.735 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.730+0000 7f54b7f80740 1 --2- 172.21.15.100:0/3337751291 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55733bfefbb0 0x55733c0008c0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:20.735 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.730+0000 7f54b7f80740 1 --2- 172.21.15.100:0/3337751291 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55733bfef1c0 0x55733bff9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:20.735 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.730+0000 7f54b7f80740 1 -- 172.21.15.100:0/3337751291 >> 172.21.15.100:0/3337751291 conn(0x55733bfe6420 msgr2=0x55733bfe9100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:20.735 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.730+0000 7f54b7f80740 1 -- 172.21.15.100:0/3337751291 shutdown_connections 2024-05-10T11:20:20.735 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.730+0000 7f54b7f80740 1 -- 172.21.15.100:0/3337751291 wait complete. 2024-05-10T11:20:20.735 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.730+0000 7f54b7f80740 1 -- start start 2024-05-10T11:20:20.735 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.730+0000 7f54b7f80740 1 Processor -- start 2024-05-10T11:20:20.735 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.731+0000 7f54b7f80740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55733bfef1c0 0x55733bffd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:20.735 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.731+0000 7f54b7f80740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55733bfefbb0 0x55733bffdc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:20.735 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.731+0000 7f54b7f80740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55733bff4d20 0x55733c06d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:20.736 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.731+0000 7f54b7f80740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f54b01a77a0 con 0x55733bff4d20 2024-05-10T11:20:20.736 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.731+0000 7f54b7f80740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f54b01a8c10 con 0x55733bfefbb0 2024-05-10T11:20:20.736 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.731+0000 7f54b7f80740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f54b01a7e70 con 0x55733bfef1c0 2024-05-10T11:20:20.736 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.731+0000 7f54b7f80740 10 client.0 _ll_register_callbacks cb 0x7f54a4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:20:20.736 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.731+0000 7f54ae59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55733bff4d20 0x55733c06d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:20.736 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.731+0000 7f54ae59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55733bff4d20 0x55733c06d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35482/0 (socket says 172.21.15.100:35482) 2024-05-10T11:20:20.736 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.731+0000 7f54ae59c640 1 -- 172.21.15.100:0/2430376962 learned_addr learned my addr 172.21.15.100:0/2430376962 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:20:20.736 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.731+0000 7f54ae59c640 1 --2- 172.21.15.100:0/2430376962 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55733bfef1c0 0x55733bffd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:20.736 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.731+0000 7f54aed9d640 1 --2- 172.21.15.100:0/2430376962 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55733bfefbb0 0x55733bffdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:20.736 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.731+0000 7f54ae59c640 1 -- 172.21.15.100:0/2430376962 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55733bfef1c0 msgr2=0x55733bffd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:20.736 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.731+0000 7f54ae59c640 1 --2- 172.21.15.100:0/2430376962 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55733bfef1c0 0x55733bffd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:20.736 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.731+0000 7f54ae59c640 1 -- 172.21.15.100:0/2430376962 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55733bfefbb0 msgr2=0x55733bffdc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:20.736 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.731+0000 7f54ae59c640 1 --2- 172.21.15.100:0/2430376962 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55733bfefbb0 0x55733bffdc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:20.736 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.731+0000 7f54ae59c640 1 -- 172.21.15.100:0/2430376962 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55733c06dad0 con 0x55733bff4d20 2024-05-10T11:20:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.731+0000 7f54aed9d640 1 --2- 172.21.15.100:0/2430376962 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55733bfefbb0 0x55733bffdc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:20:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.731+0000 7f54ae59c640 1 --2- 172.21.15.100:0/2430376962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55733bff4d20 0x55733c06d590 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f54a40059f0 tx=0x7f54a40056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6211ab5d63cdccb8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.731+0000 7f5492ff5640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:20:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.731+0000 7f5492ff5640 1 -- 172.21.15.100:0/2430376962 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a402c070 con 0x55733bff4d20 2024-05-10T11:20:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.731+0000 7f54b7f80740 1 -- 172.21.15.100:0/2430376962 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54a004b070 con 0x55733bff4d20 2024-05-10T11:20:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.731+0000 7f5492ff5640 1 -- 172.21.15.100:0/2430376962 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f54a401fac0 con 0x55733bff4d20 2024-05-10T11:20:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.731+0000 7f5492ff5640 1 -- 172.21.15.100:0/2430376962 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f54a401fcc0 con 0x55733bff4d20 2024-05-10T11:20:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.731+0000 7f54b7f80740 10 client.15765 fetch_fsmap learned FSMap version 27 2024-05-10T11:20:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.731+0000 7f54b7f80740 1 -- 172.21.15.100:0/2430376962 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55733c06ec60 con 0x55733bff4d20 2024-05-10T11:20:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.732+0000 7f5492ff5640 1 -- 172.21.15.100:0/2430376962 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f54a004b070 con 0x55733bff4d20 2024-05-10T11:20:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.732+0000 7f5492ff5640 1 -- 172.21.15.100:0/2430376962 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f54a4003a80 con 0x55733bff4d20 2024-05-10T11:20:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.732+0000 7f54b7f80740 10 client.15765 fetch_fsmap finished waiting for FSMap version 27 2024-05-10T11:20:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.732+0000 7f54b7f80740 10 client.15765 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:20:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.732+0000 7f54b7f80740 20 client.15765 populate_metadata read hostname 'smithi100' 2024-05-10T11:20:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.732+0000 7f54b7f80740 1 --2- 172.21.15.100:0/2430376962 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55733c06f550 0x55733c071970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:20.738 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.732+0000 7f54b7f80740 4 client.15765 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:20:20.738 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.732+0000 7f54b7f80740 1 -- 172.21.15.100:0/2430376962 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55733bf57050 con 0x55733c06f550 2024-05-10T11:20:20.738 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.732+0000 7f54aed9d640 1 --2- 172.21.15.100:0/2430376962 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55733c06f550 0x55733c071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:20.738 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.732+0000 7f5492ff5640 10 client.15765 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:20:20.738 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.732+0000 7f54aed9d640 1 --2- 172.21.15.100:0/2430376962 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55733c06f550 0x55733c071970 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1348f98b6acada42 server_cookie=1d1b6a6cf162ea04 in_seq=0 out_seq=0 2024-05-10T11:20:20.779 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.778+0000 7f5492ff5640 1 -- 172.21.15.100:0/2430376962 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55733bf57050 con 0x55733c06f550 2024-05-10T11:20:20.779 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.778+0000 7f5492ff5640 10 client.15765 handle_command_reply: tid=0 2024-05-10T11:20:20.779 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.778+0000 7f54b7f80740 1 client.15765 shutdown 2024-05-10T11:20:20.779 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.778+0000 7f54b7f80740 1 -- 172.21.15.100:0/2430376962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55733bff4d20 msgr2=0x55733c06d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:20.779 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.778+0000 7f54b7f80740 1 --2- 172.21.15.100:0/2430376962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55733bff4d20 0x55733c06d590 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f54a40059f0 tx=0x7f54a40056d0 comp rx=0 tx=0).stop 2024-05-10T11:20:20.779 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.778+0000 7f54b7f80740 1 -- 172.21.15.100:0/2430376962 shutdown_connections 2024-05-10T11:20:20.779 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.778+0000 7f54b7f80740 1 --2- 172.21.15.100:0/2430376962 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55733c06f550 0x55733c071970 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:20.779 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.778+0000 7f54b7f80740 1 --2- 172.21.15.100:0/2430376962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55733bff4d20 0x55733c06d590 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:20.779 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.778+0000 7f5492ff5640 0 client.15765 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:20:20.779 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.778+0000 7f54b7f80740 1 --2- 172.21.15.100:0/2430376962 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55733bfefbb0 0x55733bffdc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:20.780 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.778+0000 7f54b7f80740 1 --2- 172.21.15.100:0/2430376962 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55733bfef1c0 0x55733bffd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:20.780 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.778+0000 7f54b7f80740 1 -- 172.21.15.100:0/2430376962 >> 172.21.15.100:0/2430376962 conn(0x55733bfe6420 msgr2=0x55733bff96d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:20.780 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.779+0000 7f54b7f80740 1 -- 172.21.15.100:0/2430376962 shutdown_connections 2024-05-10T11:20:20.780 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.779+0000 7f54b7f80740 1 -- 172.21.15.100:0/2430376962 wait complete. 2024-05-10T11:20:20.780 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.779+0000 7f54b7f80740 20 client.15765 trim_cache size 0 max 16384 2024-05-10T11:20:20.780 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:20:20.780 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:20:20.780 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:20:20.780 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:20:20.781 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.779+0000 7f54b5189640 1 -- 172.21.15.100:0/2410725880 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f54980b28a0 msgr2=0x7f54980b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:20.781 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.779+0000 7f54b5189640 1 --2- 172.21.15.100:0/2410725880 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f54980b28a0 0x7f54980b4d60 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f54a80a9d70 tx=0x7f54a80d93f0 comp rx=0 tx=0).stop 2024-05-10T11:20:20.781 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.779+0000 7f54b5189640 1 -- 172.21.15.100:0/2410725880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54b01086f0 msgr2=0x7f54b01a00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:20.781 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.779+0000 7f54b5189640 1 --2- 172.21.15.100:0/2410725880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54b01086f0 0x7f54b01a00d0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f54a800eb10 tx=0x7f54a800c2d0 comp rx=0 tx=0).stop 2024-05-10T11:20:20.781 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.779+0000 7f54b5189640 1 -- 172.21.15.100:0/2410725880 shutdown_connections 2024-05-10T11:20:20.782 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.779+0000 7f54b5189640 1 --2- 172.21.15.100:0/2410725880 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f54980b28a0 0x7f54980b4d60 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:20.782 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.779+0000 7f54b5189640 1 --2- 172.21.15.100:0/2410725880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54b01086f0 0x7f54b01a00d0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:20.782 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.779+0000 7f54b5189640 1 --2- 172.21.15.100:0/2410725880 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54b0105dc0 0x7f54b0199060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:20.782 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.779+0000 7f54b5189640 1 --2- 172.21.15.100:0/2410725880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f54b0103490 0x7f54b0198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:20.782 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.779+0000 7f54b5189640 1 -- 172.21.15.100:0/2410725880 >> 172.21.15.100:0/2410725880 conn(0x7f54b00fce80 msgr2=0x7f54b0103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:20.782 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.780+0000 7f54b5189640 1 -- 172.21.15.100:0/2410725880 shutdown_connections 2024-05-10T11:20:20.782 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:20.780+0000 7f54b5189640 1 -- 172.21.15.100:0/2410725880 wait complete. 2024-05-10T11:20:20.931 INFO:tasks.workunit.client.0.smithi100.stdout: 1 10866 5.20 MB/sec warmup 15 sec latency 42.080 ms 2024-05-10T11:20:21.931 INFO:tasks.workunit.client.0.smithi100.stdout: 1 11332 4.91 MB/sec warmup 16 sec latency 14.972 ms 2024-05-10T11:20:22.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:20:21 smithi178 ceph-mon[33359]: pgmap v479: 129 pgs: 129 active+clean; 26 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 2.3 MiB/s wr, 111 op/s 2024-05-10T11:20:22.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:20:21 smithi188 ceph-mon[34075]: pgmap v479: 129 pgs: 129 active+clean; 26 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 2.3 MiB/s wr, 111 op/s 2024-05-10T11:20:22.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:20:21 smithi100 ceph-mon[30296]: pgmap v479: 129 pgs: 129 active+clean; 26 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 2.3 MiB/s wr, 111 op/s 2024-05-10T11:20:22.931 INFO:tasks.workunit.client.0.smithi100.stdout: 1 12080 4.90 MB/sec warmup 17 sec latency 17.916 ms 2024-05-10T11:20:23.931 INFO:tasks.workunit.client.0.smithi100.stdout: 1 12795 4.67 MB/sec warmup 18 sec latency 15.997 ms 2024-05-10T11:20:24.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:20:23 smithi178 ceph-mon[33359]: pgmap v480: 129 pgs: 129 active+clean; 27 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 2.2 MiB/s wr, 136 op/s 2024-05-10T11:20:24.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:20:23 smithi188 ceph-mon[34075]: pgmap v480: 129 pgs: 129 active+clean; 27 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 2.2 MiB/s wr, 136 op/s 2024-05-10T11:20:24.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:20:23 smithi100 ceph-mon[30296]: pgmap v480: 129 pgs: 129 active+clean; 27 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 2.2 MiB/s wr, 136 op/s 2024-05-10T11:20:24.931 INFO:tasks.workunit.client.0.smithi100.stdout: 1 13572 4.78 MB/sec warmup 19 sec latency 28.504 ms 2024-05-10T11:20:25.931 INFO:tasks.workunit.client.0.smithi100.stdout: 1 14180 4.79 MB/sec warmup 20 sec latency 15.053 ms 2024-05-10T11:20:26.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:20:25 smithi178 ceph-mon[33359]: pgmap v481: 129 pgs: 129 active+clean; 27 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 1.6 MiB/s wr, 103 op/s 2024-05-10T11:20:26.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:20:25 smithi178 ceph-mon[33359]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T11:20:26.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:20:25 smithi188 ceph-mon[34075]: pgmap v481: 129 pgs: 129 active+clean; 27 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 1.6 MiB/s wr, 103 op/s 2024-05-10T11:20:26.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:20:25 smithi188 ceph-mon[34075]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T11:20:26.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:20:25 smithi100 ceph-mon[30296]: pgmap v481: 129 pgs: 129 active+clean; 27 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 1.6 MiB/s wr, 103 op/s 2024-05-10T11:20:26.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:20:25 smithi100 ceph-mon[30296]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T11:20:26.931 INFO:tasks.workunit.client.0.smithi100.stdout: 1 14583 4.64 MB/sec warmup 21 sec latency 15.041 ms 2024-05-10T11:20:27.931 INFO:tasks.workunit.client.0.smithi100.stdout: 1 14930 4.44 MB/sec warmup 22 sec latency 27.727 ms 2024-05-10T11:20:28.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:20:27 smithi178 ceph-mon[33359]: pgmap v482: 129 pgs: 129 active+clean; 28 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 1.8 MiB/s wr, 130 op/s 2024-05-10T11:20:28.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:20:27 smithi188 ceph-mon[34075]: pgmap v482: 129 pgs: 129 active+clean; 28 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 1.8 MiB/s wr, 130 op/s 2024-05-10T11:20:28.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:20:27 smithi100 ceph-mon[30296]: pgmap v482: 129 pgs: 129 active+clean; 28 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 1.8 MiB/s wr, 130 op/s 2024-05-10T11:20:28.931 INFO:tasks.workunit.client.0.smithi100.stdout: 1 15448 4.33 MB/sec warmup 23 sec latency 52.753 ms 2024-05-10T11:20:29.931 INFO:tasks.workunit.client.0.smithi100.stdout: 1 16116 4.29 MB/sec warmup 24 sec latency 18.802 ms 2024-05-10T11:20:30.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:20:29 smithi178 ceph-mon[33359]: pgmap v483: 129 pgs: 129 active+clean; 28 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 1.7 MiB/s wr, 128 op/s 2024-05-10T11:20:30.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:20:29 smithi188 ceph-mon[34075]: pgmap v483: 129 pgs: 129 active+clean; 28 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 1.7 MiB/s wr, 128 op/s 2024-05-10T11:20:30.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:20:29 smithi100 ceph-mon[30296]: pgmap v483: 129 pgs: 129 active+clean; 28 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 1.7 MiB/s wr, 128 op/s 2024-05-10T11:20:30.931 INFO:tasks.workunit.client.0.smithi100.stdout: 1 16701 4.21 MB/sec warmup 25 sec latency 15.578 ms 2024-05-10T11:20:31.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:20:30 smithi178 ceph-mon[33359]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T11:20:31.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:20:30 smithi188 ceph-mon[34075]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T11:20:31.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:20:30 smithi100 ceph-mon[30296]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T11:20:31.931 INFO:tasks.workunit.client.0.smithi100.stdout: 1 17484 4.30 MB/sec warmup 26 sec latency 18.684 ms 2024-05-10T11:20:32.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:20:31 smithi178 ceph-mon[33359]: pgmap v484: 129 pgs: 129 active+clean; 30 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 1.6 MiB/s wr, 151 op/s 2024-05-10T11:20:32.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:20:31 smithi188 ceph-mon[34075]: pgmap v484: 129 pgs: 129 active+clean; 30 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 1.6 MiB/s wr, 151 op/s 2024-05-10T11:20:32.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:20:31 smithi100 ceph-mon[30296]: pgmap v484: 129 pgs: 129 active+clean; 30 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 1.6 MiB/s wr, 151 op/s 2024-05-10T11:20:32.931 INFO:tasks.workunit.client.0.smithi100.stdout: 1 17956 4.31 MB/sec warmup 27 sec latency 13.397 ms 2024-05-10T11:20:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.873+0000 7fd3d3faf640 1 -- 172.21.15.100:0/2723208843 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3cc102b00 msgr2=0x7fd3cc102f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.873+0000 7fd3d3faf640 1 --2- 172.21.15.100:0/2723208843 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3cc102b00 0x7fd3cc102f80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc009900 tx=0x7fd3bc02a410 comp rx=0 tx=0).stop 2024-05-10T11:20:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.873+0000 7fd3d3faf640 1 -- 172.21.15.100:0/2723208843 shutdown_connections 2024-05-10T11:20:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.873+0000 7fd3d3faf640 1 --2- 172.21.15.100:0/2723208843 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3cc1034c0 0x7fd3cc109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.873+0000 7fd3d3faf640 1 --2- 172.21.15.100:0/2723208843 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3cc102b00 0x7fd3cc102f80 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.873+0000 7fd3d3faf640 1 --2- 172.21.15.100:0/2723208843 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3cc101910 0x7fd3cc101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.873+0000 7fd3d3faf640 1 -- 172.21.15.100:0/2723208843 >> 172.21.15.100:0/2723208843 conn(0x7fd3cc0fce80 msgr2=0x7fd3cc0ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.873+0000 7fd3d3faf640 1 -- 172.21.15.100:0/2723208843 shutdown_connections 2024-05-10T11:20:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.873+0000 7fd3d3faf640 1 -- 172.21.15.100:0/2723208843 wait complete. 2024-05-10T11:20:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.873+0000 7fd3d3faf640 1 Processor -- start 2024-05-10T11:20:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.874+0000 7fd3d3faf640 1 -- start start 2024-05-10T11:20:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.874+0000 7fd3d3faf640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3cc101910 0x7fd3cc198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.874+0000 7fd3d3faf640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3cc102b00 0x7fd3cc199090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.874+0000 7fd3d3faf640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3cc1034c0 0x7fd3cc1a00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.874+0000 7fd3d3faf640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd3cc10e410 con 0x7fd3cc101910 2024-05-10T11:20:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.874+0000 7fd3d3faf640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd3cc10e290 con 0x7fd3cc1034c0 2024-05-10T11:20:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.874+0000 7fd3d3faf640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd3cc10e590 con 0x7fd3cc102b00 2024-05-10T11:20:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.874+0000 7fd3d1d4b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3cc101910 0x7fd3cc198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.874+0000 7fd3d1d4b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3cc101910 0x7fd3cc198b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55342/0 (socket says 172.21.15.100:55342) 2024-05-10T11:20:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.874+0000 7fd3d1d4b640 1 -- 172.21.15.100:0/1376331183 learned_addr learned my addr 172.21.15.100:0/1376331183 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:20:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.874+0000 7fd3d154a640 1 --2- 172.21.15.100:0/1376331183 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3cc102b00 0x7fd3cc199090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:33.877 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.874+0000 7fd3d254c640 1 --2- 172.21.15.100:0/1376331183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3cc1034c0 0x7fd3cc1a00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:33.877 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.874+0000 7fd3d1d4b640 1 -- 172.21.15.100:0/1376331183 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3cc102b00 msgr2=0x7fd3cc199090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:33.877 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.874+0000 7fd3d1d4b640 1 --2- 172.21.15.100:0/1376331183 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3cc102b00 0x7fd3cc199090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:33.877 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.874+0000 7fd3d1d4b640 1 -- 172.21.15.100:0/1376331183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3cc1034c0 msgr2=0x7fd3cc1a00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:33.877 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.874+0000 7fd3d1d4b640 1 --2- 172.21.15.100:0/1376331183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3cc1034c0 0x7fd3cc1a00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:33.877 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.874+0000 7fd3d1d4b640 1 -- 172.21.15.100:0/1376331183 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3cc1a0720 con 0x7fd3cc101910 2024-05-10T11:20:33.877 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.874+0000 7fd3d1d4b640 1 --2- 172.21.15.100:0/1376331183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3cc101910 0x7fd3cc198b50 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fd3c800e9c0 tx=0x7fd3c800edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:33.877 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.874+0000 7fd3c2ffd640 1 -- 172.21.15.100:0/1376331183 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c800ccc0 con 0x7fd3cc101910 2024-05-10T11:20:33.877 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.875+0000 7fd3c2ffd640 1 -- 172.21.15.100:0/1376331183 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd3c80035f0 con 0x7fd3cc101910 2024-05-10T11:20:33.877 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.875+0000 7fd3d3faf640 1 -- 172.21.15.100:0/1376331183 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3cc1a0a10 con 0x7fd3cc101910 2024-05-10T11:20:33.877 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.875+0000 7fd3c2ffd640 1 -- 172.21.15.100:0/1376331183 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c800ce60 con 0x7fd3cc101910 2024-05-10T11:20:33.877 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.875+0000 7fd3d3faf640 1 -- 172.21.15.100:0/1376331183 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3cc1a0ee0 con 0x7fd3cc101910 2024-05-10T11:20:33.885 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.877+0000 7fd3c2ffd640 1 -- 172.21.15.100:0/1376331183 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fd3c8004000 con 0x7fd3cc101910 2024-05-10T11:20:33.885 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.877+0000 7fd3d6da1740 1 Processor -- start 2024-05-10T11:20:33.885 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.878+0000 7fd3d6da1740 1 -- start start 2024-05-10T11:20:33.885 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.878+0000 7fd3d6da1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556621f901c0 0x556621f905e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.878+0000 7fd3d6da1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556621f90bb0 0x556621f956b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.878+0000 7fd3d6da1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556621f95d20 0x556621f981b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.878+0000 7fd3d6da1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x556621ef05d0 con 0x556621f95d20 2024-05-10T11:20:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.878+0000 7fd3d6da1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556621ea9890 con 0x556621f901c0 2024-05-10T11:20:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.878+0000 7fd3d6da1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556621eec590 con 0x556621f90bb0 2024-05-10T11:20:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.878+0000 7fd3d1d4b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556621f90bb0 0x556621f956b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.878+0000 7fd3d154a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556621f95d20 0x556621f981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.878+0000 7fd3d154a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556621f95d20 0x556621f981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55354/0 (socket says 172.21.15.100:55354) 2024-05-10T11:20:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.878+0000 7fd3d154a640 1 -- 172.21.15.100:0/3816639247 learned_addr learned my addr 172.21.15.100:0/3816639247 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:20:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.878+0000 7fd3d254c640 1 --2- 172.21.15.100:0/3816639247 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556621f901c0 0x556621f905e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.878+0000 7fd3d1d4b640 1 -- 172.21.15.100:0/3816639247 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556621f901c0 msgr2=0x556621f905e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.878+0000 7fd3d1d4b640 1 --2- 172.21.15.100:0/3816639247 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556621f901c0 0x556621f905e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.878+0000 7fd3d1d4b640 1 -- 172.21.15.100:0/3816639247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556621f95d20 msgr2=0x556621f981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.878+0000 7fd3d1d4b640 1 --2- 172.21.15.100:0/3816639247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556621f95d20 0x556621f981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.878+0000 7fd3d1d4b640 1 -- 172.21.15.100:0/3816639247 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556621c02210 con 0x556621f90bb0 2024-05-10T11:20:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.878+0000 7fd3c2ffd640 1 --2- 172.21.15.100:0/1376331183 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fd3b80b28a0 0x7fd3b80b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.879+0000 7fd3d254c640 1 --2- 172.21.15.100:0/1376331183 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fd3b80b28a0 0x7fd3b80b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.879+0000 7fd3d1d4b640 1 --2- 172.21.15.100:0/3816639247 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556621f90bb0 0x556621f956b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd3c80e0060 tx=0x7fd3c80e8b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbc6e5850ef3b20 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.879+0000 7fd3c2ffd640 1 -- 172.21.15.100:0/1376331183 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fd3c80e28f0 con 0x7fd3cc101910 2024-05-10T11:20:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.879+0000 7fd3b7fff640 1 -- 172.21.15.100:0/3816639247 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c80cae60 con 0x556621f90bb0 2024-05-10T11:20:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.879+0000 7fd3b7fff640 1 -- 172.21.15.100:0/3816639247 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd3c80b2360 con 0x556621f90bb0 2024-05-10T11:20:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.879+0000 7fd3b7fff640 1 -- 172.21.15.100:0/3816639247 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c80a9500 con 0x556621f90bb0 2024-05-10T11:20:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.880+0000 7fd3d6da1740 1 -- 172.21.15.100:0/3816639247 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556621f90bb0 msgr2=0x556621f956b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.880+0000 7fd3d254c640 1 --2- 172.21.15.100:0/1376331183 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fd3b80b28a0 0x7fd3b80b4d60 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x556621f91410 tx=0x7fd3c4009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.880+0000 7fd3d6da1740 1 --2- 172.21.15.100:0/3816639247 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556621f90bb0 0x556621f956b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd3c80e0060 tx=0x7fd3c80e8b40 comp rx=0 tx=0).stop 2024-05-10T11:20:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.880+0000 7fd3d6da1740 1 -- 172.21.15.100:0/3816639247 shutdown_connections 2024-05-10T11:20:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.880+0000 7fd3d6da1740 1 --2- 172.21.15.100:0/3816639247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556621f95d20 0x556621f981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.880+0000 7fd3d6da1740 1 --2- 172.21.15.100:0/3816639247 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556621f90bb0 0x556621f956b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.880+0000 7fd3d6da1740 1 --2- 172.21.15.100:0/3816639247 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556621f901c0 0x556621f905e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.880+0000 7fd3d6da1740 1 -- 172.21.15.100:0/3816639247 >> 172.21.15.100:0/3816639247 conn(0x556621f87420 msgr2=0x556621f87820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.880+0000 7fd3d6da1740 1 -- 172.21.15.100:0/3816639247 shutdown_connections 2024-05-10T11:20:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.880+0000 7fd3d6da1740 1 -- 172.21.15.100:0/3816639247 wait complete. 2024-05-10T11:20:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.880+0000 7fd3d6da1740 1 -- start start 2024-05-10T11:20:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.880+0000 7fd3d6da1740 1 Processor -- start 2024-05-10T11:20:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.881+0000 7fd3d6da1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556621f901c0 0x556621f9a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.881+0000 7fd3d6da1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556621f90bb0 0x556621f9a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.881+0000 7fd3d6da1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556621f95d20 0x556621fa18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.881+0000 7fd3d6da1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x556621f98cd0 con 0x556621f95d20 2024-05-10T11:20:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.881+0000 7fd3d6da1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556621ead370 con 0x556621f90bb0 2024-05-10T11:20:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.881+0000 7fd3d6da1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556621f98e50 con 0x556621f901c0 2024-05-10T11:20:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.881+0000 7fd3d6da1740 10 client.0 _ll_register_callbacks cb 0x556621c01a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:20:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.881+0000 7fd3d254c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556621f95d20 0x556621fa18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:33.889 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.881+0000 7fd3d1d4b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556621f901c0 0x556621f9a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:33.889 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.881+0000 7fd3d154a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556621f90bb0 0x556621f9a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:33.889 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.881+0000 7fd3d254c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556621f95d20 0x556621fa18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55358/0 (socket says 172.21.15.100:55358) 2024-05-10T11:20:33.889 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.881+0000 7fd3d1d4b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556621f901c0 0x556621f9a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:44356/0 (socket says 172.21.15.100:44356) 2024-05-10T11:20:33.889 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.881+0000 7fd3d254c640 1 -- 172.21.15.100:0/3430742938 learned_addr learned my addr 172.21.15.100:0/3430742938 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:20:33.889 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.881+0000 7fd3d254c640 1 -- 172.21.15.100:0/3430742938 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556621f901c0 msgr2=0x556621f9a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:33.889 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.881+0000 7fd3d254c640 1 --2- 172.21.15.100:0/3430742938 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556621f901c0 0x556621f9a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:33.889 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.881+0000 7fd3d254c640 1 -- 172.21.15.100:0/3430742938 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556621f90bb0 msgr2=0x556621f9a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:33.889 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.881+0000 7fd3d254c640 1 --2- 172.21.15.100:0/3430742938 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556621f90bb0 0x556621f9a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:33.889 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.881+0000 7fd3d254c640 1 -- 172.21.15.100:0/3430742938 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556621cce620 con 0x556621f95d20 2024-05-10T11:20:33.889 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.881+0000 7fd3d154a640 1 --2- 172.21.15.100:0/3430742938 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556621f90bb0 0x556621f9a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:20:33.889 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.881+0000 7fd3d254c640 1 --2- 172.21.15.100:0/3430742938 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556621f95d20 0x556621fa18f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fd3c400fa30 tx=0x7fd3c4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d694632175a238c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:33.889 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.881+0000 7fd3b57fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:20:33.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.882+0000 7fd3b57fa640 1 -- 172.21.15.100:0/3430742938 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c402d060 con 0x556621f95d20 2024-05-10T11:20:33.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.882+0000 7fd3b57fa640 1 -- 172.21.15.100:0/3430742938 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fd3c4018030 con 0x556621f95d20 2024-05-10T11:20:33.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.882+0000 7fd3d6da1740 1 -- 172.21.15.100:0/3430742938 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3c80b2360 con 0x556621f95d20 2024-05-10T11:20:33.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.882+0000 7fd3b57fa640 1 -- 172.21.15.100:0/3430742938 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd3c4034050 con 0x556621f95d20 2024-05-10T11:20:33.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.882+0000 7fd3d6da1740 10 client.15777 fetch_fsmap learned FSMap version 29 2024-05-10T11:20:33.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.882+0000 7fd3d6da1740 1 -- 172.21.15.100:0/3430742938 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556621fa49a0 con 0x556621f95d20 2024-05-10T11:20:33.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.882+0000 7fd3b57fa640 1 -- 172.21.15.100:0/3430742938 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd3c80b2360 con 0x556621f95d20 2024-05-10T11:20:33.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.882+0000 7fd3b57fa640 1 -- 172.21.15.100:0/3430742938 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd3c4028700 con 0x556621f95d20 2024-05-10T11:20:33.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.882+0000 7fd3d6da1740 10 client.15777 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T11:20:33.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.882+0000 7fd3d6da1740 10 client.15777 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:20:33.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.882+0000 7fd3d6da1740 20 client.15777 populate_metadata read hostname 'smithi100' 2024-05-10T11:20:33.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.882+0000 7fd3d6da1740 1 --2- 172.21.15.100:0/3430742938 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556621fa6150 0x556621fa8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:33.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.882+0000 7fd3d6da1740 4 client.15777 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:20:33.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.882+0000 7fd3d6da1740 1 -- 172.21.15.100:0/3430742938 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556621efe1e0 con 0x556621fa6150 2024-05-10T11:20:33.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.882+0000 7fd3d1d4b640 1 --2- 172.21.15.100:0/3430742938 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556621fa6150 0x556621fa8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:33.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.883+0000 7fd3b57fa640 10 client.15777 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:20:33.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.883+0000 7fd3d1d4b640 1 --2- 172.21.15.100:0/3430742938 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556621fa6150 0x556621fa8540 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67d26b28afaa65f2 server_cookie=d2039d1d8e74eaf0 in_seq=0 out_seq=0 2024-05-10T11:20:33.892 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.883+0000 7fd3b57fa640 1 -- 172.21.15.100:0/3430742938 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556621efe1e0 con 0x556621fa6150 2024-05-10T11:20:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.883+0000 7fd3b57fa640 10 client.15777 handle_command_reply: tid=0 2024-05-10T11:20:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.883+0000 7fd3d6da1740 1 client.15777 shutdown 2024-05-10T11:20:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.884+0000 7fd3d6da1740 1 -- 172.21.15.100:0/3430742938 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556621f95d20 msgr2=0x556621fa18f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.884+0000 7fd3d6da1740 1 --2- 172.21.15.100:0/3430742938 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556621f95d20 0x556621fa18f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fd3c400fa30 tx=0x7fd3c4004ab0 comp rx=0 tx=0).stop 2024-05-10T11:20:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.884+0000 7fd3d6da1740 1 -- 172.21.15.100:0/3430742938 shutdown_connections 2024-05-10T11:20:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.884+0000 7fd3d6da1740 1 --2- 172.21.15.100:0/3430742938 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556621fa6150 0x556621fa8540 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.884+0000 7fd3d6da1740 1 --2- 172.21.15.100:0/3430742938 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556621f95d20 0x556621fa18f0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.884+0000 7fd3d6da1740 1 --2- 172.21.15.100:0/3430742938 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556621f90bb0 0x556621f9a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.884+0000 7fd3d6da1740 1 --2- 172.21.15.100:0/3430742938 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556621f901c0 0x556621f9a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.884+0000 7fd3b57fa640 0 client.15777 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:20:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.884+0000 7fd3d6da1740 1 -- 172.21.15.100:0/3430742938 >> 172.21.15.100:0/3430742938 conn(0x556621f87420 msgr2=0x556621f86580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.884+0000 7fd3d6da1740 1 -- 172.21.15.100:0/3430742938 shutdown_connections 2024-05-10T11:20:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.884+0000 7fd3d6da1740 1 -- 172.21.15.100:0/3430742938 wait complete. 2024-05-10T11:20:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.884+0000 7fd3d6da1740 20 client.15777 trim_cache size 0 max 16384 2024-05-10T11:20:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.900+0000 7fd3d6da1740 1 Processor -- start 2024-05-10T11:20:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.900+0000 7fd3d6da1740 1 -- start start 2024-05-10T11:20:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.900+0000 7fd3d6da1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556621f87420 0x556621f86400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.900+0000 7fd3d6da1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556621f901c0 0x556621f86940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.900+0000 7fd3d6da1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556621f90bb0 0x556621fa18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.900+0000 7fd3d6da1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd3cc1ab9a0 con 0x556621f901c0 2024-05-10T11:20:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.900+0000 7fd3d6da1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd3cc1a9270 con 0x556621f87420 2024-05-10T11:20:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.900+0000 7fd3d6da1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd3cc1a70c0 con 0x556621f90bb0 2024-05-10T11:20:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.900+0000 7fd3d154a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556621f901c0 0x556621f86940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.900+0000 7fd3d154a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556621f901c0 0x556621f86940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55370/0 (socket says 172.21.15.100:55370) 2024-05-10T11:20:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.900+0000 7fd3d154a640 1 -- 172.21.15.100:0/1603284770 learned_addr learned my addr 172.21.15.100:0/1603284770 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:20:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.900+0000 7fd3d1d4b640 1 --2- 172.21.15.100:0/1603284770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556621f87420 0x556621f86400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.900+0000 7fd3d254c640 1 --2- 172.21.15.100:0/1603284770 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556621f90bb0 0x556621fa18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.900+0000 7fd3d154a640 1 -- 172.21.15.100:0/1603284770 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556621f90bb0 msgr2=0x556621fa18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.900+0000 7fd3d154a640 1 --2- 172.21.15.100:0/1603284770 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556621f90bb0 0x556621fa18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.900+0000 7fd3d154a640 1 -- 172.21.15.100:0/1603284770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556621f87420 msgr2=0x556621f86400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.900+0000 7fd3d154a640 1 --2- 172.21.15.100:0/1603284770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556621f87420 0x556621f86400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.900+0000 7fd3d154a640 1 -- 172.21.15.100:0/1603284770 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556621f81a00 con 0x556621f901c0 2024-05-10T11:20:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.901+0000 7fd3d154a640 1 --2- 172.21.15.100:0/1603284770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556621f901c0 0x556621f86940 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc02a930 tx=0x7fd3bc031640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79e2be1b42d4cbb8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.901+0000 7fd3b77fe640 1 -- 172.21.15.100:0/1603284770 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc035040 con 0x556621f901c0 2024-05-10T11:20:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.901+0000 7fd3b77fe640 1 -- 172.21.15.100:0/1603284770 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd3bc0317f0 con 0x556621f901c0 2024-05-10T11:20:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.901+0000 7fd3b77fe640 1 -- 172.21.15.100:0/1603284770 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc003850 con 0x556621f901c0 2024-05-10T11:20:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.901+0000 7fd3d6da1740 1 -- 172.21.15.100:0/1603284770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556621f901c0 msgr2=0x556621f86940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.901+0000 7fd3d6da1740 1 --2- 172.21.15.100:0/1603284770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556621f901c0 0x556621f86940 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc02a930 tx=0x7fd3bc031640 comp rx=0 tx=0).stop 2024-05-10T11:20:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.901+0000 7fd3d6da1740 1 -- 172.21.15.100:0/1603284770 shutdown_connections 2024-05-10T11:20:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.901+0000 7fd3d6da1740 1 --2- 172.21.15.100:0/1603284770 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556621f90bb0 0x556621fa18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.901+0000 7fd3d6da1740 1 --2- 172.21.15.100:0/1603284770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556621f901c0 0x556621f86940 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.901+0000 7fd3d6da1740 1 --2- 172.21.15.100:0/1603284770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556621f87420 0x556621f86400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.901+0000 7fd3d6da1740 1 -- 172.21.15.100:0/1603284770 >> 172.21.15.100:0/1603284770 conn(0x556621f99f30 msgr2=0x556621f85cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.902+0000 7fd3d6da1740 1 -- 172.21.15.100:0/1603284770 shutdown_connections 2024-05-10T11:20:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.902+0000 7fd3d6da1740 1 -- 172.21.15.100:0/1603284770 wait complete. 2024-05-10T11:20:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.902+0000 7fd3d6da1740 1 -- start start 2024-05-10T11:20:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.902+0000 7fd3d6da1740 1 Processor -- start 2024-05-10T11:20:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.902+0000 7fd3d6da1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556621f87420 0x556621fa1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.902+0000 7fd3d6da1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556621f901c0 0x556621fa1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.902+0000 7fd3d6da1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556621f90bb0 0x556621f9f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.902+0000 7fd3d6da1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd3cc1a7730 con 0x556621f90bb0 2024-05-10T11:20:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.902+0000 7fd3d6da1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd3cc1a8ba0 con 0x556621f901c0 2024-05-10T11:20:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.902+0000 7fd3d6da1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd3cc1a7e00 con 0x556621f87420 2024-05-10T11:20:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.902+0000 7fd3d6da1740 10 client.0 _ll_register_callbacks cb 0x7fd3c400f8f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:20:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.903+0000 7fd3d254c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556621f90bb0 0x556621f9f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.903+0000 7fd3d1d4b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556621f87420 0x556621fa1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.903+0000 7fd3d254c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556621f90bb0 0x556621f9f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55386/0 (socket says 172.21.15.100:55386) 2024-05-10T11:20:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.903+0000 7fd3d254c640 1 -- 172.21.15.100:0/1444414275 learned_addr learned my addr 172.21.15.100:0/1444414275 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:20:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.903+0000 7fd3d154a640 1 --2- 172.21.15.100:0/1444414275 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556621f901c0 0x556621fa1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.903+0000 7fd3d254c640 1 -- 172.21.15.100:0/1444414275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556621f87420 msgr2=0x556621fa1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.903+0000 7fd3d254c640 1 --2- 172.21.15.100:0/1444414275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556621f87420 0x556621fa1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.903+0000 7fd3d254c640 1 -- 172.21.15.100:0/1444414275 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556621f901c0 msgr2=0x556621fa1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.903+0000 7fd3d254c640 1 --2- 172.21.15.100:0/1444414275 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556621f901c0 0x556621fa1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.903+0000 7fd3d254c640 1 -- 172.21.15.100:0/1444414275 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556621f87250 con 0x556621f90bb0 2024-05-10T11:20:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.903+0000 7fd3d254c640 1 --2- 172.21.15.100:0/1444414275 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556621f90bb0 0x556621f9f950 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fd3c4028b70 tx=0x7fd3c4009e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f83f3276191ac8e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.903+0000 7fd3b57fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:20:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.903+0000 7fd3b57fa640 1 -- 172.21.15.100:0/1444414275 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c4028680 con 0x556621f90bb0 2024-05-10T11:20:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.903+0000 7fd3b57fa640 1 -- 172.21.15.100:0/1444414275 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fd3c4028820 con 0x556621f90bb0 2024-05-10T11:20:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.903+0000 7fd3d6da1740 1 -- 172.21.15.100:0/1444414275 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3bc0317f0 con 0x556621f90bb0 2024-05-10T11:20:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.903+0000 7fd3b57fa640 1 -- 172.21.15.100:0/1444414275 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd3c4046050 con 0x556621f90bb0 2024-05-10T11:20:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.903+0000 7fd3d6da1740 10 client.15783 fetch_fsmap learned FSMap version 29 2024-05-10T11:20:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.903+0000 7fd3d6da1740 1 -- 172.21.15.100:0/1444414275 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556622011360 con 0x556621f90bb0 2024-05-10T11:20:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.904+0000 7fd3b57fa640 1 -- 172.21.15.100:0/1444414275 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd3bc0317f0 con 0x556621f90bb0 2024-05-10T11:20:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.904+0000 7fd3b57fa640 1 -- 172.21.15.100:0/1444414275 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd3c401c6e0 con 0x556621f90bb0 2024-05-10T11:20:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.904+0000 7fd3d6da1740 10 client.15783 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T11:20:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.904+0000 7fd3d6da1740 10 client.15783 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:20:33.909 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.904+0000 7fd3d6da1740 20 client.15783 populate_metadata read hostname 'smithi100' 2024-05-10T11:20:33.909 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.904+0000 7fd3d6da1740 1 --2- 172.21.15.100:0/1444414275 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556621f95d20 0x556622013e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:33.909 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.904+0000 7fd3d6da1740 4 client.15783 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:20:33.909 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.904+0000 7fd3d6da1740 1 -- 172.21.15.100:0/1444414275 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556621ef8050 con 0x556621f95d20 2024-05-10T11:20:33.909 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.904+0000 7fd3d1d4b640 1 --2- 172.21.15.100:0/1444414275 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556621f95d20 0x556622013e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:33.909 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.904+0000 7fd3b57fa640 10 client.15783 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:20:33.909 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.904+0000 7fd3d1d4b640 1 --2- 172.21.15.100:0/1444414275 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556621f95d20 0x556622013e30 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d140a7d4929b0ab server_cookie=94fcf2d881656073 in_seq=0 out_seq=0 2024-05-10T11:20:33.926 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.926+0000 7fd3b57fa640 1 -- 172.21.15.100:0/1444414275 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556621ef8050 con 0x556621f95d20 2024-05-10T11:20:33.927 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.926+0000 7fd3b57fa640 10 client.15783 handle_command_reply: tid=0 2024-05-10T11:20:33.927 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.926+0000 7fd3d6da1740 1 client.15783 shutdown 2024-05-10T11:20:33.927 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.926+0000 7fd3d6da1740 1 -- 172.21.15.100:0/1444414275 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556621f90bb0 msgr2=0x556621f9f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:33.927 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.926+0000 7fd3d6da1740 1 --2- 172.21.15.100:0/1444414275 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556621f90bb0 0x556621f9f950 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fd3c4028b70 tx=0x7fd3c4009e70 comp rx=0 tx=0).stop 2024-05-10T11:20:33.927 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.926+0000 7fd3d6da1740 1 -- 172.21.15.100:0/1444414275 shutdown_connections 2024-05-10T11:20:33.927 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.926+0000 7fd3d6da1740 1 --2- 172.21.15.100:0/1444414275 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556621f95d20 0x556622013e30 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:33.927 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.926+0000 7fd3d6da1740 1 --2- 172.21.15.100:0/1444414275 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556621f90bb0 0x556621f9f950 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:33.927 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.926+0000 7fd3d6da1740 1 --2- 172.21.15.100:0/1444414275 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556621f901c0 0x556621fa1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:33.927 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.926+0000 7fd3d6da1740 1 --2- 172.21.15.100:0/1444414275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556621f87420 0x556621fa1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:33.927 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.926+0000 7fd3d6da1740 1 -- 172.21.15.100:0/1444414275 >> 172.21.15.100:0/1444414275 conn(0x556621f99f30 msgr2=0x556621f86580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:33.928 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.926+0000 7fd3b57fa640 0 client.15783 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:20:33.928 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.926+0000 7fd3d6da1740 1 -- 172.21.15.100:0/1444414275 shutdown_connections 2024-05-10T11:20:33.928 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.926+0000 7fd3d6da1740 1 -- 172.21.15.100:0/1444414275 wait complete. 2024-05-10T11:20:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.926+0000 7fd3d6da1740 20 client.15783 trim_cache size 0 max 16384 2024-05-10T11:20:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:20:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:20:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:20:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:20:33.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.927+0000 7fd3d3faf640 1 -- 172.21.15.100:0/1376331183 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fd3b80b28a0 msgr2=0x7fd3b80b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:33.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.927+0000 7fd3d3faf640 1 --2- 172.21.15.100:0/1376331183 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fd3b80b28a0 0x7fd3b80b4d60 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x556621f91410 tx=0x7fd3c4009500 comp rx=0 tx=0).stop 2024-05-10T11:20:33.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.927+0000 7fd3d3faf640 1 -- 172.21.15.100:0/1376331183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3cc101910 msgr2=0x7fd3cc198b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:33.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.927+0000 7fd3d3faf640 1 --2- 172.21.15.100:0/1376331183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3cc101910 0x7fd3cc198b50 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fd3c800e9c0 tx=0x7fd3c800edc0 comp rx=0 tx=0).stop 2024-05-10T11:20:33.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.927+0000 7fd3d3faf640 1 -- 172.21.15.100:0/1376331183 shutdown_connections 2024-05-10T11:20:33.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.927+0000 7fd3d3faf640 1 --2- 172.21.15.100:0/1376331183 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fd3b80b28a0 0x7fd3b80b4d60 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:33.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.927+0000 7fd3d3faf640 1 --2- 172.21.15.100:0/1376331183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3cc1034c0 0x7fd3cc1a00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:33.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.927+0000 7fd3d3faf640 1 --2- 172.21.15.100:0/1376331183 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3cc102b00 0x7fd3cc199090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:33.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.927+0000 7fd3d3faf640 1 --2- 172.21.15.100:0/1376331183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3cc101910 0x7fd3cc198b50 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:33.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.927+0000 7fd3d3faf640 1 -- 172.21.15.100:0/1376331183 >> 172.21.15.100:0/1376331183 conn(0x7fd3cc0fce80 msgr2=0x7fd3cc105f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:33.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.927+0000 7fd3d3faf640 1 -- 172.21.15.100:0/1376331183 shutdown_connections 2024-05-10T11:20:33.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:33.927+0000 7fd3d3faf640 1 -- 172.21.15.100:0/1376331183 wait complete. 2024-05-10T11:20:33.933 INFO:tasks.workunit.client.0.smithi100.stdout: 1 18217 4.16 MB/sec warmup 28 sec latency 22.165 ms 2024-05-10T11:20:34.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:20:33 smithi178 ceph-mon[33359]: pgmap v485: 129 pgs: 129 active+clean; 28 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 1.6 MiB/s wr, 158 op/s 2024-05-10T11:20:34.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:20:33 smithi188 ceph-mon[34075]: pgmap v485: 129 pgs: 129 active+clean; 28 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 1.6 MiB/s wr, 158 op/s 2024-05-10T11:20:34.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:20:33 smithi100 ceph-mon[30296]: pgmap v485: 129 pgs: 129 active+clean; 28 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 1.6 MiB/s wr, 158 op/s 2024-05-10T11:20:34.932 INFO:tasks.workunit.client.0.smithi100.stdout: 1 18319 4.02 MB/sec warmup 29 sec latency 718.409 ms 2024-05-10T11:20:35.932 INFO:tasks.workunit.client.0.smithi100.stdout: 1 18319 3.89 MB/sec warmup 30 sec latency 1718.455 ms 2024-05-10T11:20:36.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:20:35 smithi178 ceph-mon[33359]: pgmap v486: 129 pgs: 129 active+clean; 28 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 1.5 MiB/s wr, 127 op/s 2024-05-10T11:20:36.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:20:35 smithi188 ceph-mon[34075]: pgmap v486: 129 pgs: 129 active+clean; 28 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 1.5 MiB/s wr, 127 op/s 2024-05-10T11:20:36.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:20:35 smithi100 ceph-mon[30296]: pgmap v486: 129 pgs: 129 active+clean; 28 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 1.5 MiB/s wr, 127 op/s 2024-05-10T11:20:36.932 INFO:tasks.workunit.client.0.smithi100.stdout: 1 18319 3.76 MB/sec warmup 31 sec latency 2718.494 ms 2024-05-10T11:20:37.933 INFO:tasks.workunit.client.0.smithi100.stdout: 1 18319 3.64 MB/sec warmup 32 sec latency 3718.633 ms 2024-05-10T11:20:38.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:20:37 smithi178 ceph-mon[33359]: pgmap v487: 129 pgs: 129 active+clean; 31 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 1.6 MiB/s wr, 159 op/s 2024-05-10T11:20:38.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:20:37 smithi188 ceph-mon[34075]: pgmap v487: 129 pgs: 129 active+clean; 31 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 1.6 MiB/s wr, 159 op/s 2024-05-10T11:20:38.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:20:37 smithi100 ceph-mon[30296]: pgmap v487: 129 pgs: 129 active+clean; 31 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 1.6 MiB/s wr, 159 op/s 2024-05-10T11:20:38.933 INFO:tasks.workunit.client.0.smithi100.stdout: 1 18346 3.53 MB/sec warmup 33 sec latency 4592.797 ms 2024-05-10T11:20:39.932 INFO:tasks.workunit.client.0.smithi100.stdout: 1 18877 3.49 MB/sec warmup 34 sec latency 15.576 ms 2024-05-10T11:20:40.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:20:39 smithi178 ceph-mon[33359]: pgmap v488: 129 pgs: 129 active+clean; 27 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 1.1 MiB/s wr, 130 op/s 2024-05-10T11:20:40.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:20:39 smithi188 ceph-mon[34075]: pgmap v488: 129 pgs: 129 active+clean; 27 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 1.1 MiB/s wr, 130 op/s 2024-05-10T11:20:40.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:20:39 smithi100 ceph-mon[30296]: pgmap v488: 129 pgs: 129 active+clean; 27 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 1.1 MiB/s wr, 130 op/s 2024-05-10T11:20:40.932 INFO:tasks.workunit.client.0.smithi100.stdout: 1 19521 3.48 MB/sec warmup 35 sec latency 16.206 ms 2024-05-10T11:20:41.932 INFO:tasks.workunit.client.0.smithi100.stdout: 1 20195 3.44 MB/sec warmup 36 sec latency 17.153 ms 2024-05-10T11:20:42.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:20:41 smithi178 ceph-mon[33359]: pgmap v489: 129 pgs: 129 active+clean; 27 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 1.0 MiB/s wr, 134 op/s 2024-05-10T11:20:42.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:20:41 smithi188 ceph-mon[34075]: pgmap v489: 129 pgs: 129 active+clean; 27 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 1.0 MiB/s wr, 134 op/s 2024-05-10T11:20:42.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:20:41 smithi100 ceph-mon[30296]: pgmap v489: 129 pgs: 129 active+clean; 27 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 1.0 MiB/s wr, 134 op/s 2024-05-10T11:20:42.934 INFO:tasks.workunit.client.0.smithi100.stdout: 1 20969 3.51 MB/sec warmup 37 sec latency 14.956 ms 2024-05-10T11:20:43.932 INFO:tasks.workunit.client.0.smithi100.stdout: 1 21444 3.54 MB/sec warmup 38 sec latency 12.199 ms 2024-05-10T11:20:43.973 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T11:20:44.234 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:20:44.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:20:43 smithi178 ceph-mon[33359]: pgmap v490: 129 pgs: 129 active+clean; 26 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 881 KiB/s wr, 96 op/s 2024-05-10T11:20:44.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:20:43 smithi188 ceph-mon[34075]: pgmap v490: 129 pgs: 129 active+clean; 26 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 881 KiB/s wr, 96 op/s 2024-05-10T11:20:44.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:20:43 smithi100 ceph-mon[30296]: pgmap v490: 129 pgs: 129 active+clean; 26 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 881 KiB/s wr, 96 op/s 2024-05-10T11:20:44.932 INFO:tasks.workunit.client.0.smithi100.stdout: 1 21758 3.48 MB/sec warmup 39 sec latency 14.116 ms 2024-05-10T11:20:45.932 INFO:tasks.workunit.client.0.smithi100.stdout: 1 22152 3.41 MB/sec warmup 40 sec latency 20.236 ms 2024-05-10T11:20:45.984 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:20:45 smithi100 ceph-mon[30296]: pgmap v491: 129 pgs: 129 active+clean; 26 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 829 KiB/s wr, 88 op/s 2024-05-10T11:20:46.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.101+0000 7f0f49540700 1 -- 172.21.15.100:0/1553198501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f44108400 msgr2=0x7f0f441087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:46.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.101+0000 7f0f49540700 1 --2- 172.21.15.100:0/1553198501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f44108400 0x7f0f441087e0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f0f34009990 tx=0x7f0f34009ca0 comp rx=0 tx=0).stop 2024-05-10T11:20:46.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.101+0000 7f0f49540700 1 -- 172.21.15.100:0/1553198501 shutdown_connections 2024-05-10T11:20:46.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.101+0000 7f0f49540700 1 --2- 172.21.15.100:0/1553198501 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f44102d50 0x7f0f4410f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:46.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.101+0000 7f0f49540700 1 --2- 172.21.15.100:0/1553198501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f44108400 0x7f0f441087e0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:46.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.101+0000 7f0f49540700 1 --2- 172.21.15.100:0/1553198501 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f44102390 0x7f0f44102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:46.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.101+0000 7f0f49540700 1 -- 172.21.15.100:0/1553198501 >> 172.21.15.100:0/1553198501 conn(0x7f0f44079310 msgr2=0x7f0f44079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:46.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.101+0000 7f0f49540700 1 -- 172.21.15.100:0/1553198501 shutdown_connections 2024-05-10T11:20:46.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.101+0000 7f0f49540700 1 -- 172.21.15.100:0/1553198501 wait complete. 2024-05-10T11:20:46.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.102+0000 7f0f49540700 1 Processor -- start 2024-05-10T11:20:46.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.102+0000 7f0f49540700 1 -- start start 2024-05-10T11:20:46.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.102+0000 7f0f49540700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f44102390 0x7f0f4419f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:46.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.102+0000 7f0f49540700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f44102d50 0x7f0f4419fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:46.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.102+0000 7f0f49540700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f44108400 0x7f0f441998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:46.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.102+0000 7f0f49540700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0f44112400 con 0x7f0f44108400 2024-05-10T11:20:46.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.102+0000 7f0f49540700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0f44112280 con 0x7f0f44102390 2024-05-10T11:20:46.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.102+0000 7f0f49540700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0f44112580 con 0x7f0f44102d50 2024-05-10T11:20:46.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.103+0000 7f0f437fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f44108400 0x7f0f441998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:46.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.103+0000 7f0f427fc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f44102d50 0x7f0f4419fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:46.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.103+0000 7f0f42ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f44102390 0x7f0f4419f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:46.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.103+0000 7f0f427fc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f44102d50 0x7f0f4419fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:40344/0 (socket says 172.21.15.100:40344) 2024-05-10T11:20:46.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.103+0000 7f0f437fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f44108400 0x7f0f441998d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47442/0 (socket says 172.21.15.100:47442) 2024-05-10T11:20:46.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.103+0000 7f0f42ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f44102390 0x7f0f4419f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:54640/0 (socket says 172.21.15.100:54640) 2024-05-10T11:20:46.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.103+0000 7f0f427fc700 1 -- 172.21.15.100:0/2630440731 learned_addr learned my addr 172.21.15.100:0/2630440731 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:20:46.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.103+0000 7f0f427fc700 1 -- 172.21.15.100:0/2630440731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f44102390 msgr2=0x7f0f4419f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:46.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.103+0000 7f0f427fc700 1 --2- 172.21.15.100:0/2630440731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f44102390 0x7f0f4419f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:46.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.103+0000 7f0f427fc700 1 -- 172.21.15.100:0/2630440731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f44108400 msgr2=0x7f0f441998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:46.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.103+0000 7f0f427fc700 1 --2- 172.21.15.100:0/2630440731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f44108400 0x7f0f441998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:46.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.103+0000 7f0f427fc700 1 -- 172.21.15.100:0/2630440731 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f4419a030 con 0x7f0f44102d50 2024-05-10T11:20:46.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.103+0000 7f0f42ffd700 1 --2- 172.21.15.100:0/2630440731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f44102390 0x7f0f4419f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:20:46.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.103+0000 7f0f427fc700 1 --2- 172.21.15.100:0/2630440731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f44102d50 0x7f0f4419fc50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0f3800e9a0 tx=0x7f0f3800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:46.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.104+0000 7f0f2bfff700 1 -- 172.21.15.100:0/2630440731 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3800ca10 con 0x7f0f44102d50 2024-05-10T11:20:46.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.104+0000 7f0f49540700 1 -- 172.21.15.100:0/2630440731 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f4419a320 con 0x7f0f44102d50 2024-05-10T11:20:46.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.104+0000 7f0f49540700 1 -- 172.21.15.100:0/2630440731 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f441a65e0 con 0x7f0f44102d50 2024-05-10T11:20:46.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.104+0000 7f0f2bfff700 1 -- 172.21.15.100:0/2630440731 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0f3800cbb0 con 0x7f0f44102d50 2024-05-10T11:20:46.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.104+0000 7f0f2bfff700 1 -- 172.21.15.100:0/2630440731 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f38026060 con 0x7f0f44102d50 2024-05-10T11:20:46.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.106+0000 7f0f2bfff700 1 -- 172.21.15.100:0/2630440731 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f0f38018420 con 0x7f0f44102d50 2024-05-10T11:20:46.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.107+0000 7f0f2bfff700 1 --2- 172.21.15.100:0/2630440731 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0f2c0b2880 0x7f0f2c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:46.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.107+0000 7f0f2bfff700 1 -- 172.21.15.100:0/2630440731 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f0f380e26e0 con 0x7f0f44102d50 2024-05-10T11:20:46.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.107+0000 7f0f42ffd700 1 --2- 172.21.15.100:0/2630440731 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0f2c0b2880 0x7f0f2c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:46.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.107+0000 7f0f53dd3b80 1 Processor -- start 2024-05-10T11:20:46.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.107+0000 7f0f53dd3b80 1 -- start start 2024-05-10T11:20:46.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.108+0000 7f0f42ffd700 1 --2- 172.21.15.100:0/2630440731 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0f2c0b2880 0x7f0f2c0b4d40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0f3400b2e0 tx=0x7f0f3401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:46.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.108+0000 7f0f53dd3b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55accd0005c0 0x55accd0009a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:46.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.108+0000 7f0f53dd3b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55accd000ee0 0x55acccffac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:46.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.108+0000 7f0f53dd3b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55acccffb160 0x55acccffb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:46.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.108+0000 7f0f53dd3b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55acccf48940 con 0x55acccffb160 2024-05-10T11:20:46.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.108+0000 7f0f53dd3b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55accceadda0 con 0x55accd000ee0 2024-05-10T11:20:46.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.108+0000 7f0f53dd3b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55acccf44900 con 0x55accd0005c0 2024-05-10T11:20:46.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.108+0000 7f0f42ffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55accd0005c0 0x55accd0009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:46.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.108+0000 7f0f437fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55acccffb160 0x55acccffb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:46.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.108+0000 7f0f42ffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55accd0005c0 0x55accd0009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:40358/0 (socket says 172.21.15.100:40358) 2024-05-10T11:20:46.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.108+0000 7f0f42ffd700 1 -- 172.21.15.100:0/177938097 learned_addr learned my addr 172.21.15.100:0/177938097 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:20:46.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.108+0000 7f0f427fc700 1 --2- 172.21.15.100:0/177938097 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55accd000ee0 0x55acccffac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:46.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.108+0000 7f0f427fc700 1 -- 172.21.15.100:0/177938097 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55accd0005c0 msgr2=0x55accd0009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:46.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.108+0000 7f0f427fc700 1 --2- 172.21.15.100:0/177938097 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55accd0005c0 0x55accd0009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:46.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.108+0000 7f0f427fc700 1 -- 172.21.15.100:0/177938097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55acccffb160 msgr2=0x55acccffb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:46.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.108+0000 7f0f427fc700 1 --2- 172.21.15.100:0/177938097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55acccffb160 0x55acccffb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:46.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.108+0000 7f0f427fc700 1 -- 172.21.15.100:0/177938097 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55accce958d0 con 0x55accd000ee0 2024-05-10T11:20:46.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.108+0000 7f0f427fc700 1 --2- 172.21.15.100:0/177938097 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55accd000ee0 0x55acccffac20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0f380dfe80 tx=0x7f0f380a9530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4645c36c0363740 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:46.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.109+0000 7f0f297fa700 1 -- 172.21.15.100:0/177938097 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f380a99a0 con 0x55accd000ee0 2024-05-10T11:20:46.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.109+0000 7f0f297fa700 1 -- 172.21.15.100:0/177938097 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0f380add50 con 0x55accd000ee0 2024-05-10T11:20:46.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.109+0000 7f0f297fa700 1 -- 172.21.15.100:0/177938097 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f380b2370 con 0x55accd000ee0 2024-05-10T11:20:46.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.109+0000 7f0f53dd3b80 1 -- 172.21.15.100:0/177938097 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55accd000ee0 msgr2=0x55acccffac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:46.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.109+0000 7f0f53dd3b80 1 --2- 172.21.15.100:0/177938097 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55accd000ee0 0x55acccffac20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0f380dfe80 tx=0x7f0f380a9530 comp rx=0 tx=0).stop 2024-05-10T11:20:46.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.109+0000 7f0f53dd3b80 1 -- 172.21.15.100:0/177938097 shutdown_connections 2024-05-10T11:20:46.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.109+0000 7f0f53dd3b80 1 --2- 172.21.15.100:0/177938097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55acccffb160 0x55acccffb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:46.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.109+0000 7f0f53dd3b80 1 --2- 172.21.15.100:0/177938097 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55accd0005c0 0x55accd0009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:46.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.109+0000 7f0f53dd3b80 1 --2- 172.21.15.100:0/177938097 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55accd000ee0 0x55acccffac20 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:46.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.109+0000 7f0f53dd3b80 1 -- 172.21.15.100:0/177938097 >> 172.21.15.100:0/177938097 conn(0x55acccf22b70 msgr2=0x55acccff12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:46.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.110+0000 7f0f53dd3b80 1 -- 172.21.15.100:0/177938097 shutdown_connections 2024-05-10T11:20:46.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.110+0000 7f0f53dd3b80 1 -- 172.21.15.100:0/177938097 wait complete. 2024-05-10T11:20:46.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.110+0000 7f0f53dd3b80 1 -- start start 2024-05-10T11:20:46.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.110+0000 7f0f53dd3b80 1 Processor -- start 2024-05-10T11:20:46.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.110+0000 7f0f53dd3b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55acccffb160 0x55accd006a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:46.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.110+0000 7f0f53dd3b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55accd0005c0 0x55accd003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:46.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.110+0000 7f0f53dd3b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55accd000ee0 0x55accd004340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:46.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.110+0000 7f0f53dd3b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55accd007860 con 0x55accd000ee0 2024-05-10T11:20:46.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.110+0000 7f0f53dd3b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55acccf01ed0 con 0x55acccffb160 2024-05-10T11:20:46.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.110+0000 7f0f53dd3b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55accd0079e0 con 0x55accd0005c0 2024-05-10T11:20:46.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.110+0000 7f0f53dd3b80 10 client.0 _ll_register_callbacks cb 0x55acccedc5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:20:46.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.110+0000 7f0f427fc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55accd0005c0 0x55accd003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:46.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.110+0000 7f0f437fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55accd000ee0 0x55accd004340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:46.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.110+0000 7f0f427fc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55accd0005c0 0x55accd003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:40366/0 (socket says 172.21.15.100:40366) 2024-05-10T11:20:46.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.110+0000 7f0f427fc700 1 -- 172.21.15.100:0/626178077 learned_addr learned my addr 172.21.15.100:0/626178077 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:20:46.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.110+0000 7f0f42ffd700 1 --2- 172.21.15.100:0/626178077 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55acccffb160 0x55accd006a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:46.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.111+0000 7f0f427fc700 1 -- 172.21.15.100:0/626178077 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55acccffb160 msgr2=0x55accd006a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:46.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.111+0000 7f0f427fc700 1 --2- 172.21.15.100:0/626178077 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55acccffb160 0x55accd006a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:46.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.111+0000 7f0f427fc700 1 -- 172.21.15.100:0/626178077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55accd000ee0 msgr2=0x55accd004340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:46.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.111+0000 7f0f427fc700 1 --2- 172.21.15.100:0/626178077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55accd000ee0 0x55accd004340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:46.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.111+0000 7f0f427fc700 1 -- 172.21.15.100:0/626178077 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55accceff760 con 0x55accd0005c0 2024-05-10T11:20:46.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.111+0000 7f0f427fc700 1 --2- 172.21.15.100:0/626178077 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55accd0005c0 0x55accd003e00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0f380aa000 tx=0x7f0f380a5200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cdc7ac40a9024b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:46.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.111+0000 7f0f26ff5700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:20:46.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.111+0000 7f0f26ff5700 1 -- 172.21.15.100:0/626178077 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f380c2030 con 0x55accd0005c0 2024-05-10T11:20:46.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.111+0000 7f0f26ff5700 1 -- 172.21.15.100:0/626178077 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f0f380a5f90 con 0x55accd0005c0 2024-05-10T11:20:46.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.111+0000 7f0f53dd3b80 1 -- 172.21.15.100:0/626178077 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f380add50 con 0x55accd0005c0 2024-05-10T11:20:46.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.111+0000 7f0f26ff5700 1 -- 172.21.15.100:0/626178077 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0f380a99a0 con 0x55accd0005c0 2024-05-10T11:20:46.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.111+0000 7f0f53dd3b80 10 client.25297 fetch_fsmap learned FSMap version 29 2024-05-10T11:20:46.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.111+0000 7f0f53dd3b80 1 -- 172.21.15.100:0/626178077 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55accd007860 con 0x55accd0005c0 2024-05-10T11:20:46.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.112+0000 7f0f26ff5700 1 -- 172.21.15.100:0/626178077 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0f380add50 con 0x55accd0005c0 2024-05-10T11:20:46.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.112+0000 7f0f26ff5700 1 -- 172.21.15.100:0/626178077 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f0f380a53a0 con 0x55accd0005c0 2024-05-10T11:20:46.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.112+0000 7f0f53dd3b80 10 client.25297 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T11:20:46.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.112+0000 7f0f53dd3b80 10 client.25297 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:20:46.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.112+0000 7f0f53dd3b80 20 client.25297 populate_metadata read hostname 'smithi100' 2024-05-10T11:20:46.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.112+0000 7f0f53dd3b80 1 --2- 172.21.15.100:0/626178077 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55accd014350 0x55accd016740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:46.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.112+0000 7f0f53dd3b80 4 client.25297 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:20:46.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.112+0000 7f0f53dd3b80 1 -- 172.21.15.100:0/626178077 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55acccf56320 con 0x55accd014350 2024-05-10T11:20:46.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.112+0000 7f0f42ffd700 1 --2- 172.21.15.100:0/626178077 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55accd014350 0x55accd016740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:46.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.112+0000 7f0f42ffd700 1 --2- 172.21.15.100:0/626178077 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55accd014350 0x55accd016740 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72d8230f4c6ccf07 server_cookie=c96d2d6f17d24e30 in_seq=0 out_seq=0 2024-05-10T11:20:46.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.112+0000 7f0f26ff5700 10 client.25297 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:20:46.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.113+0000 7f0f26ff5700 1 -- 172.21.15.100:0/626178077 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55acccf56320 con 0x55accd014350 2024-05-10T11:20:46.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.113+0000 7f0f26ff5700 10 client.25297 handle_command_reply: tid=0 2024-05-10T11:20:46.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.113+0000 7f0f53dd3b80 1 client.25297 shutdown 2024-05-10T11:20:46.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.113+0000 7f0f53dd3b80 1 -- 172.21.15.100:0/626178077 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55accd0005c0 msgr2=0x55accd003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:46.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.113+0000 7f0f53dd3b80 1 --2- 172.21.15.100:0/626178077 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55accd0005c0 0x55accd003e00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0f380aa000 tx=0x7f0f380a5200 comp rx=0 tx=0).stop 2024-05-10T11:20:46.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.113+0000 7f0f53dd3b80 1 -- 172.21.15.100:0/626178077 shutdown_connections 2024-05-10T11:20:46.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.113+0000 7f0f53dd3b80 1 --2- 172.21.15.100:0/626178077 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55accd014350 0x55accd016740 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:46.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.113+0000 7f0f26ff5700 0 client.25297 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:20:46.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.113+0000 7f0f53dd3b80 1 --2- 172.21.15.100:0/626178077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55accd000ee0 0x55accd004340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:46.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.113+0000 7f0f53dd3b80 1 --2- 172.21.15.100:0/626178077 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55acccffb160 0x55accd006a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:46.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.113+0000 7f0f53dd3b80 1 --2- 172.21.15.100:0/626178077 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55accd0005c0 0x55accd003e00 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:46.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.113+0000 7f0f53dd3b80 1 -- 172.21.15.100:0/626178077 >> 172.21.15.100:0/626178077 conn(0x55acccf22b70 msgr2=0x55acccfebb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:46.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.113+0000 7f0f53dd3b80 1 -- 172.21.15.100:0/626178077 shutdown_connections 2024-05-10T11:20:46.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.113+0000 7f0f53dd3b80 1 -- 172.21.15.100:0/626178077 wait complete. 2024-05-10T11:20:46.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.114+0000 7f0f53dd3b80 20 client.25297 trim_cache size 0 max 16384 2024-05-10T11:20:46.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.134+0000 7f0f53dd3b80 1 Processor -- start 2024-05-10T11:20:46.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.134+0000 7f0f53dd3b80 1 -- start start 2024-05-10T11:20:46.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.134+0000 7f0f53dd3b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55acccffb160 0x55acccffac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:46.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.134+0000 7f0f53dd3b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55accd0005c0 0x55accd010920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:46.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.134+0000 7f0f53dd3b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55accd000ee0 0x55accd0019d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:46.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.134+0000 7f0f53dd3b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0f30011260 con 0x55acccffb160 2024-05-10T11:20:46.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.134+0000 7f0f53dd3b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3000eaf0 con 0x55accd0005c0 2024-05-10T11:20:46.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.134+0000 7f0f53dd3b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3000c940 con 0x55accd000ee0 2024-05-10T11:20:46.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.134+0000 7f0f42ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55acccffb160 0x55acccffac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:46.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.134+0000 7f0f42ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55acccffb160 0x55acccffac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47466/0 (socket says 172.21.15.100:47466) 2024-05-10T11:20:46.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.134+0000 7f0f42ffd700 1 -- 172.21.15.100:0/2216465653 learned_addr learned my addr 172.21.15.100:0/2216465653 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:20:46.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.134+0000 7f0f437fe700 1 --2- 172.21.15.100:0/2216465653 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55accd000ee0 0x55accd0019d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:46.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.134+0000 7f0f427fc700 1 --2- 172.21.15.100:0/2216465653 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55accd0005c0 0x55accd010920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:46.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.134+0000 7f0f42ffd700 1 -- 172.21.15.100:0/2216465653 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55accd000ee0 msgr2=0x55accd0019d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:46.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.134+0000 7f0f42ffd700 1 --2- 172.21.15.100:0/2216465653 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55accd000ee0 0x55accd0019d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:46.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.134+0000 7f0f42ffd700 1 -- 172.21.15.100:0/2216465653 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55accd0005c0 msgr2=0x55accd010920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:46.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.134+0000 7f0f42ffd700 1 --2- 172.21.15.100:0/2216465653 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55accd0005c0 0x55accd010920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:46.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.134+0000 7f0f42ffd700 1 -- 172.21.15.100:0/2216465653 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acccffe910 con 0x55acccffb160 2024-05-10T11:20:46.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.135+0000 7f0f42ffd700 1 --2- 172.21.15.100:0/2216465653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55acccffb160 0x55acccffac20 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f0f34005cd0 tx=0x7f0f3401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a4a28262bfcc241 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:46.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.135+0000 7f0f28ff9700 1 -- 172.21.15.100:0/2216465653 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f34039070 con 0x55acccffb160 2024-05-10T11:20:46.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.135+0000 7f0f28ff9700 1 -- 172.21.15.100:0/2216465653 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0f34005210 con 0x55acccffb160 2024-05-10T11:20:46.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.135+0000 7f0f28ff9700 1 -- 172.21.15.100:0/2216465653 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f34026d80 con 0x55acccffb160 2024-05-10T11:20:46.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.136+0000 7f0f53dd3b80 1 -- 172.21.15.100:0/2216465653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55acccffb160 msgr2=0x55acccffac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:46.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.136+0000 7f0f53dd3b80 1 --2- 172.21.15.100:0/2216465653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55acccffb160 0x55acccffac20 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f0f34005cd0 tx=0x7f0f3401f000 comp rx=0 tx=0).stop 2024-05-10T11:20:46.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.136+0000 7f0f53dd3b80 1 -- 172.21.15.100:0/2216465653 shutdown_connections 2024-05-10T11:20:46.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.136+0000 7f0f53dd3b80 1 --2- 172.21.15.100:0/2216465653 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55accd000ee0 0x55accd0019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:46.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.136+0000 7f0f53dd3b80 1 --2- 172.21.15.100:0/2216465653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55acccffb160 0x55acccffac20 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:46.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.136+0000 7f0f53dd3b80 1 --2- 172.21.15.100:0/2216465653 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55accd0005c0 0x55accd010920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:46.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.136+0000 7f0f53dd3b80 1 -- 172.21.15.100:0/2216465653 >> 172.21.15.100:0/2216465653 conn(0x55acccf22b70 msgr2=0x55accd0012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:46.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.136+0000 7f0f53dd3b80 1 -- 172.21.15.100:0/2216465653 shutdown_connections 2024-05-10T11:20:46.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.136+0000 7f0f53dd3b80 1 -- 172.21.15.100:0/2216465653 wait complete. 2024-05-10T11:20:46.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.136+0000 7f0f53dd3b80 1 -- start start 2024-05-10T11:20:46.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.136+0000 7f0f53dd3b80 1 Processor -- start 2024-05-10T11:20:46.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.136+0000 7f0f53dd3b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55acccffb160 0x55accd00cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:46.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.136+0000 7f0f53dd3b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55accd0005c0 0x55accd00d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:46.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.136+0000 7f0f53dd3b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55accd000ee0 0x55accd009c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:46.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.136+0000 7f0f53dd3b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3000cfb0 con 0x55accd0005c0 2024-05-10T11:20:46.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.136+0000 7f0f53dd3b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3000e420 con 0x55accd000ee0 2024-05-10T11:20:46.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.136+0000 7f0f53dd3b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3000d680 con 0x55acccffb160 2024-05-10T11:20:46.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.136+0000 7f0f53dd3b80 10 client.0 _ll_register_callbacks cb 0x55acccedc5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:20:46.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.137+0000 7f0f42ffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55acccffb160 0x55accd00cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:46.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.137+0000 7f0f427fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55accd0005c0 0x55accd00d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:46.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.137+0000 7f0f42ffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55acccffb160 0x55accd00cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:40370/0 (socket says 172.21.15.100:40370) 2024-05-10T11:20:46.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.137+0000 7f0f42ffd700 1 -- 172.21.15.100:0/1948476498 learned_addr learned my addr 172.21.15.100:0/1948476498 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:20:46.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.137+0000 7f0f437fe700 1 --2- 172.21.15.100:0/1948476498 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55accd000ee0 0x55accd009c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:46.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.137+0000 7f0f42ffd700 1 -- 172.21.15.100:0/1948476498 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55accd000ee0 msgr2=0x55accd009c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:46.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.137+0000 7f0f42ffd700 1 --2- 172.21.15.100:0/1948476498 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55accd000ee0 0x55accd009c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:46.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.137+0000 7f0f42ffd700 1 -- 172.21.15.100:0/1948476498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55accd0005c0 msgr2=0x55accd00d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:46.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.137+0000 7f0f42ffd700 1 --2- 172.21.15.100:0/1948476498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55accd0005c0 0x55accd00d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:46.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.137+0000 7f0f42ffd700 1 -- 172.21.15.100:0/1948476498 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55accd00a1b0 con 0x55acccffb160 2024-05-10T11:20:46.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.137+0000 7f0f427fc700 1 --2- 172.21.15.100:0/1948476498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55accd0005c0 0x55accd00d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:20:46.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.137+0000 7f0f42ffd700 1 --2- 172.21.15.100:0/1948476498 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55acccffb160 0x55accd00cca0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0f340269a0 tx=0x7f0f34005650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e65ac68c09dac6e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:46.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.137+0000 7f0f26ff5700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:20:46.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.137+0000 7f0f26ff5700 1 -- 172.21.15.100:0/1948476498 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f34041020 con 0x55acccffb160 2024-05-10T11:20:46.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.137+0000 7f0f26ff5700 1 -- 172.21.15.100:0/1948476498 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f0f34026630 con 0x55acccffb160 2024-05-10T11:20:46.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.137+0000 7f0f53dd3b80 1 -- 172.21.15.100:0/1948476498 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f34005210 con 0x55acccffb160 2024-05-10T11:20:46.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.138+0000 7f0f26ff5700 1 -- 172.21.15.100:0/1948476498 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0f34039040 con 0x55acccffb160 2024-05-10T11:20:46.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.138+0000 7f0f53dd3b80 10 client.25303 fetch_fsmap learned FSMap version 29 2024-05-10T11:20:46.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.138+0000 7f0f53dd3b80 1 -- 172.21.15.100:0/1948476498 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55accd07f0b0 con 0x55acccffb160 2024-05-10T11:20:46.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.138+0000 7f0f26ff5700 1 -- 172.21.15.100:0/1948476498 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0f34005210 con 0x55acccffb160 2024-05-10T11:20:46.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.138+0000 7f0f26ff5700 1 -- 172.21.15.100:0/1948476498 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f0f3403c030 con 0x55acccffb160 2024-05-10T11:20:46.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.138+0000 7f0f53dd3b80 10 client.25303 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T11:20:46.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.138+0000 7f0f53dd3b80 10 client.25303 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:20:46.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.138+0000 7f0f53dd3b80 20 client.25303 populate_metadata read hostname 'smithi100' 2024-05-10T11:20:46.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.138+0000 7f0f53dd3b80 1 --2- 172.21.15.100:0/1948476498 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55accd07f700 0x55accd081b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:46.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.138+0000 7f0f53dd3b80 4 client.25303 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T11:20:46.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.138+0000 7f0f53dd3b80 1 -- 172.21.15.100:0/1948476498 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55acccf503c0 con 0x55accd07f700 2024-05-10T11:20:46.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.138+0000 7f0f427fc700 1 --2- 172.21.15.100:0/1948476498 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55accd07f700 0x55accd081b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:46.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.139+0000 7f0f26ff5700 10 client.25303 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:20:46.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.139+0000 7f0f427fc700 1 --2- 172.21.15.100:0/1948476498 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55accd07f700 0x55accd081b20 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3833855f9c25290d server_cookie=c3733d99f7d8f57d in_seq=0 out_seq=0 2024-05-10T11:20:46.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.139+0000 7f0f26ff5700 1 -- 172.21.15.100:0/1948476498 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55acccf503c0 con 0x55accd07f700 2024-05-10T11:20:46.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.139+0000 7f0f26ff5700 10 client.25303 handle_command_reply: tid=0 2024-05-10T11:20:46.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.139+0000 7f0f53dd3b80 1 client.25303 shutdown 2024-05-10T11:20:46.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.139+0000 7f0f53dd3b80 1 -- 172.21.15.100:0/1948476498 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55acccffb160 msgr2=0x55accd00cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:46.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.139+0000 7f0f53dd3b80 1 --2- 172.21.15.100:0/1948476498 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55acccffb160 0x55accd00cca0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0f340269a0 tx=0x7f0f34005650 comp rx=0 tx=0).stop 2024-05-10T11:20:46.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.139+0000 7f0f53dd3b80 1 -- 172.21.15.100:0/1948476498 shutdown_connections 2024-05-10T11:20:46.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.139+0000 7f0f53dd3b80 1 --2- 172.21.15.100:0/1948476498 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55accd07f700 0x55accd081b20 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:46.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.139+0000 7f0f53dd3b80 1 --2- 172.21.15.100:0/1948476498 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55accd000ee0 0x55accd009c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:46.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.139+0000 7f0f53dd3b80 1 --2- 172.21.15.100:0/1948476498 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55acccffb160 0x55accd00cca0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:46.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.139+0000 7f0f53dd3b80 1 --2- 172.21.15.100:0/1948476498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55accd0005c0 0x55accd00d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:46.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.139+0000 7f0f53dd3b80 1 -- 172.21.15.100:0/1948476498 >> 172.21.15.100:0/1948476498 conn(0x55acccf22b70 msgr2=0x55acccffef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:46.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.139+0000 7f0f26ff5700 0 client.25303 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:20:46.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.139+0000 7f0f53dd3b80 1 -- 172.21.15.100:0/1948476498 shutdown_connections 2024-05-10T11:20:46.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.139+0000 7f0f53dd3b80 1 -- 172.21.15.100:0/1948476498 wait complete. 2024-05-10T11:20:46.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.139+0000 7f0f53dd3b80 20 client.25303 trim_cache size 0 max 16384 2024-05-10T11:20:46.146 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:20:46.146 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-05-10T11:20:46.146 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-05-10T11:20:46.146 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:20:46.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.140+0000 7f0f49540700 1 -- 172.21.15.100:0/2630440731 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0f2c0b2880 msgr2=0x7f0f2c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:46.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.140+0000 7f0f49540700 1 --2- 172.21.15.100:0/2630440731 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0f2c0b2880 0x7f0f2c0b4d40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0f3400b2e0 tx=0x7f0f3401a040 comp rx=0 tx=0).stop 2024-05-10T11:20:46.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.140+0000 7f0f49540700 1 -- 172.21.15.100:0/2630440731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f44102d50 msgr2=0x7f0f4419fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:46.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.140+0000 7f0f49540700 1 --2- 172.21.15.100:0/2630440731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f44102d50 0x7f0f4419fc50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0f3800e9a0 tx=0x7f0f3800ecb0 comp rx=0 tx=0).stop 2024-05-10T11:20:46.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.140+0000 7f0f49540700 1 -- 172.21.15.100:0/2630440731 shutdown_connections 2024-05-10T11:20:46.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.140+0000 7f0f49540700 1 --2- 172.21.15.100:0/2630440731 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0f2c0b2880 0x7f0f2c0b4d40 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:46.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.140+0000 7f0f49540700 1 --2- 172.21.15.100:0/2630440731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f44108400 0x7f0f441998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:46.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.140+0000 7f0f49540700 1 --2- 172.21.15.100:0/2630440731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f44102390 0x7f0f4419f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:46.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.140+0000 7f0f49540700 1 --2- 172.21.15.100:0/2630440731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f44102d50 0x7f0f4419fc50 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:46.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.140+0000 7f0f49540700 1 -- 172.21.15.100:0/2630440731 >> 172.21.15.100:0/2630440731 conn(0x7f0f44079310 msgr2=0x7f0f4406bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:46.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.140+0000 7f0f49540700 1 -- 172.21.15.100:0/2630440731 shutdown_connections 2024-05-10T11:20:46.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:46.140+0000 7f0f49540700 1 -- 172.21.15.100:0/2630440731 wait complete. 2024-05-10T11:20:46.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:20:45 smithi178 ceph-mon[33359]: pgmap v491: 129 pgs: 129 active+clean; 26 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 829 KiB/s wr, 88 op/s 2024-05-10T11:20:46.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:20:45 smithi188 ceph-mon[34075]: pgmap v491: 129 pgs: 129 active+clean; 26 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 829 KiB/s wr, 88 op/s 2024-05-10T11:20:46.922 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T11:20:46.923 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T11:20:46.932 INFO:tasks.workunit.client.0.smithi100.stdout: 1 22749 3.43 MB/sec warmup 41 sec latency 17.154 ms 2024-05-10T11:20:47.062 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.062+0000 7f959d4da640 1 -- 172.21.15.100:0/3602778609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95981034c0 msgr2=0x7f9598109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:47.063 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.062+0000 7f959d4da640 1 --2- 172.21.15.100:0/3602778609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95981034c0 0x7f9598109d50 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f9590009ca0 tx=0x7f959002a8c0 comp rx=0 tx=0).stop 2024-05-10T11:20:47.063 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.062+0000 7f959d4da640 1 -- 172.21.15.100:0/3602778609 shutdown_connections 2024-05-10T11:20:47.063 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.062+0000 7f959d4da640 1 --2- 172.21.15.100:0/3602778609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95981034c0 0x7f9598109d50 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:47.063 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.062+0000 7f959d4da640 1 --2- 172.21.15.100:0/3602778609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9598102b00 0x7f9598102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:47.063 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.062+0000 7f959d4da640 1 --2- 172.21.15.100:0/3602778609 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9598101910 0x7f9598101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:47.063 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.062+0000 7f959d4da640 1 -- 172.21.15.100:0/3602778609 >> 172.21.15.100:0/3602778609 conn(0x7f95980fce80 msgr2=0x7f95980ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:47.063 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.062+0000 7f959d4da640 1 -- 172.21.15.100:0/3602778609 shutdown_connections 2024-05-10T11:20:47.063 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.062+0000 7f959d4da640 1 -- 172.21.15.100:0/3602778609 wait complete. 2024-05-10T11:20:47.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.063+0000 7f959d4da640 1 Processor -- start 2024-05-10T11:20:47.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.063+0000 7f959d4da640 1 -- start start 2024-05-10T11:20:47.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.063+0000 7f959d4da640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9598101910 0x7f9598197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:47.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.063+0000 7f959d4da640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9598102b00 0x7f9598197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:47.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.063+0000 7f959d4da640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95981034c0 0x7f959819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:47.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.063+0000 7f959d4da640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f959810d320 con 0x7f9598101910 2024-05-10T11:20:47.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.064+0000 7f959d4da640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f959810d1a0 con 0x7f95981034c0 2024-05-10T11:20:47.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.064+0000 7f959d4da640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f959810d4a0 con 0x7f9598102b00 2024-05-10T11:20:47.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.064+0000 7f9596ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9598101910 0x7f9598197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:47.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.064+0000 7f9596ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9598101910 0x7f9598197990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46366/0 (socket says 172.21.15.100:46366) 2024-05-10T11:20:47.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.064+0000 7f9596ffd640 1 -- 172.21.15.100:0/2923973766 learned_addr learned my addr 172.21.15.100:0/2923973766 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:20:47.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.064+0000 7f958ffff640 1 --2- 172.21.15.100:0/2923973766 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9598102b00 0x7f9598197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:47.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.064+0000 7f95977fe640 1 --2- 172.21.15.100:0/2923973766 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95981034c0 0x7f959819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:47.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.064+0000 7f9596ffd640 1 -- 172.21.15.100:0/2923973766 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9598102b00 msgr2=0x7f9598197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:47.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.064+0000 7f9596ffd640 1 --2- 172.21.15.100:0/2923973766 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9598102b00 0x7f9598197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:47.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.064+0000 7f9596ffd640 1 -- 172.21.15.100:0/2923973766 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95981034c0 msgr2=0x7f959819ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:47.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.064+0000 7f9596ffd640 1 --2- 172.21.15.100:0/2923973766 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95981034c0 0x7f959819ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:47.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.064+0000 7f9596ffd640 1 -- 172.21.15.100:0/2923973766 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f959819f5b0 con 0x7f9598101910 2024-05-10T11:20:47.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.064+0000 7f9596ffd640 1 --2- 172.21.15.100:0/2923973766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9598101910 0x7f9598197990 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f958800ead0 tx=0x7f958800ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:47.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.064+0000 7f9594ff9640 1 -- 172.21.15.100:0/2923973766 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f958801c070 con 0x7f9598101910 2024-05-10T11:20:47.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.064+0000 7f959d4da640 1 -- 172.21.15.100:0/2923973766 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f959806a0c0 con 0x7f9598101910 2024-05-10T11:20:47.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.064+0000 7f9594ff9640 1 -- 172.21.15.100:0/2923973766 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f958800bc90 con 0x7f9598101910 2024-05-10T11:20:47.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.064+0000 7f9594ff9640 1 -- 172.21.15.100:0/2923973766 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9588016610 con 0x7f9598101910 2024-05-10T11:20:47.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.064+0000 7f959d4da640 1 -- 172.21.15.100:0/2923973766 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f959806a660 con 0x7f9598101910 2024-05-10T11:20:47.070 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.067+0000 7f95a01fc740 1 Processor -- start 2024-05-10T11:20:47.070 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.067+0000 7f95a01fc740 1 -- start start 2024-05-10T11:20:47.070 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.067+0000 7f95a01fc740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56026b07bdb0 0x56026b07c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:47.070 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.067+0000 7f95a01fc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56026b07c7a0 0x56026b0812a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:47.071 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.067+0000 7f95a01fc740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56026b081910 0x56026b083da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:47.071 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.067+0000 7f95a01fc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56026afe05d0 con 0x56026b07c7a0 2024-05-10T11:20:47.071 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.067+0000 7f95a01fc740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56026af99890 con 0x56026b07bdb0 2024-05-10T11:20:47.071 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.067+0000 7f95a01fc740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56026afdc590 con 0x56026b081910 2024-05-10T11:20:47.071 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.068+0000 7f958ffff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56026b081910 0x56026b083da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:47.071 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.068+0000 7f95977fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56026b07bdb0 0x56026b07c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:47.071 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.068+0000 7f958ffff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56026b081910 0x56026b083da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:49892/0 (socket says 172.21.15.100:49892) 2024-05-10T11:20:47.071 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.068+0000 7f958ffff640 1 -- 172.21.15.100:0/243122572 learned_addr learned my addr 172.21.15.100:0/243122572 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:20:47.071 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.068+0000 7f9594ff9640 1 -- 172.21.15.100:0/2923973766 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f958801f070 con 0x7f9598101910 2024-05-10T11:20:47.071 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.068+0000 7f958ffff640 1 -- 172.21.15.100:0/243122572 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56026b07bdb0 msgr2=0x56026b07c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:47.071 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.068+0000 7f958ffff640 1 --2- 172.21.15.100:0/243122572 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56026b07bdb0 0x56026b07c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:47.071 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.068+0000 7f958ffff640 1 -- 172.21.15.100:0/243122572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56026b07c7a0 msgr2=0x56026b0812a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T11:20:47.071 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.068+0000 7f9596ffd640 1 --2- 172.21.15.100:0/243122572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56026b07c7a0 0x56026b0812a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:47.071 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.068+0000 7f958ffff640 1 --2- 172.21.15.100:0/243122572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56026b07c7a0 0x56026b0812a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.068+0000 7f958ffff640 1 -- 172.21.15.100:0/243122572 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56026acf2210 con 0x56026b081910 2024-05-10T11:20:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.068+0000 7f95977fe640 1 --2- 172.21.15.100:0/243122572 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56026b07bdb0 0x56026b07c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:20:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.069+0000 7f9596ffd640 1 --2- 172.21.15.100:0/243122572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56026b07c7a0 0x56026b0812a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:20:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.069+0000 7f958ffff640 1 --2- 172.21.15.100:0/243122572 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56026b081910 0x56026b083da0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f958000bc90 tx=0x7f95800107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b1a4189252266d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.069+0000 7f9594ff9640 1 --2- 172.21.15.100:0/2923973766 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f95840b28a0 0x7f95840b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.069+0000 7f95977fe640 1 --2- 172.21.15.100:0/2923973766 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f95840b28a0 0x7f95840b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.069+0000 7f9594ff9640 1 -- 172.21.15.100:0/2923973766 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f95880e3c40 con 0x7f9598101910 2024-05-10T11:20:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.069+0000 7f958d7fa640 1 -- 172.21.15.100:0/243122572 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9580005220 con 0x56026b081910 2024-05-10T11:20:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.069+0000 7f958d7fa640 1 -- 172.21.15.100:0/243122572 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f958000be40 con 0x56026b081910 2024-05-10T11:20:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.069+0000 7f958d7fa640 1 -- 172.21.15.100:0/243122572 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95800116a0 con 0x56026b081910 2024-05-10T11:20:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.070+0000 7f95a01fc740 1 -- 172.21.15.100:0/243122572 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56026b081910 msgr2=0x56026b083da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.070+0000 7f95a01fc740 1 --2- 172.21.15.100:0/243122572 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56026b081910 0x56026b083da0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f958000bc90 tx=0x7f95800107a0 comp rx=0 tx=0).stop 2024-05-10T11:20:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.070+0000 7f95977fe640 1 --2- 172.21.15.100:0/2923973766 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f95840b28a0 0x7f95840b4d60 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9590006010 tx=0x7f95900319a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.070+0000 7f95a01fc740 1 -- 172.21.15.100:0/243122572 shutdown_connections 2024-05-10T11:20:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.070+0000 7f95a01fc740 1 --2- 172.21.15.100:0/243122572 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56026b081910 0x56026b083da0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.070+0000 7f95a01fc740 1 --2- 172.21.15.100:0/243122572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56026b07c7a0 0x56026b0812a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.070+0000 7f95a01fc740 1 --2- 172.21.15.100:0/243122572 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56026b07bdb0 0x56026b07c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.070+0000 7f95a01fc740 1 -- 172.21.15.100:0/243122572 >> 172.21.15.100:0/243122572 conn(0x56026b077420 msgr2=0x56026b077820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.070+0000 7f95a01fc740 1 -- 172.21.15.100:0/243122572 shutdown_connections 2024-05-10T11:20:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.070+0000 7f95a01fc740 1 -- 172.21.15.100:0/243122572 wait complete. 2024-05-10T11:20:47.076 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.071+0000 7f95a01fc740 1 -- start start 2024-05-10T11:20:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.071+0000 7f95a01fc740 1 Processor -- start 2024-05-10T11:20:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.071+0000 7f95a01fc740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56026b07bdb0 0x56026b085f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.071+0000 7f95a01fc740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56026b07c7a0 0x56026b0864d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.071+0000 7f95a01fc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56026b081910 0x56026b08d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.071+0000 7f95a01fc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56026b0848c0 con 0x56026b081910 2024-05-10T11:20:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.071+0000 7f95a01fc740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56026af9d370 con 0x56026b07bdb0 2024-05-10T11:20:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.071+0000 7f95a01fc740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56026b084a40 con 0x56026b07c7a0 2024-05-10T11:20:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.071+0000 7f95a01fc740 10 client.0 _ll_register_callbacks cb 0x56026acf1a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:20:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.071+0000 7f95977fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56026b081910 0x56026b08d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.071+0000 7f95977fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56026b081910 0x56026b08d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46384/0 (socket says 172.21.15.100:46384) 2024-05-10T11:20:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.071+0000 7f95977fe640 1 -- 172.21.15.100:0/3850206134 learned_addr learned my addr 172.21.15.100:0/3850206134 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:20:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.071+0000 7f958ffff640 1 --2- 172.21.15.100:0/3850206134 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56026b07c7a0 0x56026b0864d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.072+0000 7f9596ffd640 1 --2- 172.21.15.100:0/3850206134 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56026b07bdb0 0x56026b085f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.072+0000 7f95977fe640 1 -- 172.21.15.100:0/3850206134 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56026b07c7a0 msgr2=0x56026b0864d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:47.078 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.072+0000 7f95977fe640 1 --2- 172.21.15.100:0/3850206134 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56026b07c7a0 0x56026b0864d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:47.078 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.072+0000 7f95977fe640 1 -- 172.21.15.100:0/3850206134 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56026b07bdb0 msgr2=0x56026b085f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:47.078 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.072+0000 7f95977fe640 1 --2- 172.21.15.100:0/3850206134 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56026b07bdb0 0x56026b085f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:47.078 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.072+0000 7f95977fe640 1 -- 172.21.15.100:0/3850206134 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56026adbe620 con 0x56026b081910 2024-05-10T11:20:47.078 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.072+0000 7f9596ffd640 1 --2- 172.21.15.100:0/3850206134 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56026b07bdb0 0x56026b085f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:20:47.078 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.072+0000 7f95977fe640 1 --2- 172.21.15.100:0/3850206134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56026b081910 0x56026b08d510 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f9590009ca0 tx=0x7f959002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a78d96bee3140945 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:47.078 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.072+0000 7f957e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:20:47.078 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.072+0000 7f958ffff640 1 --2- 172.21.15.100:0/3850206134 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56026b07c7a0 0x56026b0864d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:20:47.078 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.073+0000 7f957e7fc640 1 -- 172.21.15.100:0/3850206134 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9590003e40 con 0x56026b081910 2024-05-10T11:20:47.078 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.073+0000 7f957e7fc640 1 -- 172.21.15.100:0/3850206134 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f9590004250 con 0x56026b081910 2024-05-10T11:20:47.078 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.073+0000 7f95a01fc740 1 -- 172.21.15.100:0/3850206134 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56026afee1e0 con 0x56026b081910 2024-05-10T11:20:47.078 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.073+0000 7f957e7fc640 1 -- 172.21.15.100:0/3850206134 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9590004450 con 0x56026b081910 2024-05-10T11:20:47.078 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.073+0000 7f95a01fc740 10 client.15810 fetch_fsmap learned FSMap version 29 2024-05-10T11:20:47.078 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.073+0000 7f95a01fc740 1 -- 172.21.15.100:0/3850206134 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56026b0904d0 con 0x56026b081910 2024-05-10T11:20:47.079 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.073+0000 7f957e7fc640 1 -- 172.21.15.100:0/3850206134 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x56026afee1e0 con 0x56026b081910 2024-05-10T11:20:47.079 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.074+0000 7f957e7fc640 1 -- 172.21.15.100:0/3850206134 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f9590058470 con 0x56026b081910 2024-05-10T11:20:47.079 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.074+0000 7f95a01fc740 10 client.15810 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T11:20:47.079 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.074+0000 7f95a01fc740 10 client.15810 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:20:47.079 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.074+0000 7f95a01fc740 20 client.15810 populate_metadata read hostname 'smithi100' 2024-05-10T11:20:47.079 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.074+0000 7f95a01fc740 1 --2- 172.21.15.100:0/3850206134 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56026b091c30 0x56026b094080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:47.079 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.074+0000 7f95a01fc740 4 client.15810 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:20:47.079 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.074+0000 7f95a01fc740 1 -- 172.21.15.100:0/3850206134 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56026afe8050 con 0x56026b091c30 2024-05-10T11:20:47.079 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.074+0000 7f9596ffd640 1 --2- 172.21.15.100:0/3850206134 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56026b091c30 0x56026b094080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:47.079 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.074+0000 7f957e7fc640 10 client.15810 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:20:47.079 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.074+0000 7f9596ffd640 1 --2- 172.21.15.100:0/3850206134 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56026b091c30 0x56026b094080 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51eba18e41c9f952 server_cookie=da6df4a8e7fea465 in_seq=0 out_seq=0 2024-05-10T11:20:47.079 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.075+0000 7f957e7fc640 1 -- 172.21.15.100:0/3850206134 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56026afe8050 con 0x56026b091c30 2024-05-10T11:20:47.080 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.075+0000 7f957e7fc640 10 client.15810 handle_command_reply: tid=0 2024-05-10T11:20:47.080 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.075+0000 7f95a01fc740 1 client.15810 shutdown 2024-05-10T11:20:47.080 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.075+0000 7f95a01fc740 1 -- 172.21.15.100:0/3850206134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56026b081910 msgr2=0x56026b08d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:47.080 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.075+0000 7f95a01fc740 1 --2- 172.21.15.100:0/3850206134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56026b081910 0x56026b08d510 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f9590009ca0 tx=0x7f959002a8c0 comp rx=0 tx=0).stop 2024-05-10T11:20:47.080 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.075+0000 7f95a01fc740 1 -- 172.21.15.100:0/3850206134 shutdown_connections 2024-05-10T11:20:47.080 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.075+0000 7f95a01fc740 1 --2- 172.21.15.100:0/3850206134 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56026b091c30 0x56026b094080 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:47.080 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.075+0000 7f95a01fc740 1 --2- 172.21.15.100:0/3850206134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56026b081910 0x56026b08d510 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:47.080 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.075+0000 7f95a01fc740 1 --2- 172.21.15.100:0/3850206134 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56026b07c7a0 0x56026b0864d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:47.080 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.075+0000 7f957e7fc640 0 client.15810 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:20:47.080 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.075+0000 7f95a01fc740 1 --2- 172.21.15.100:0/3850206134 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56026b07bdb0 0x56026b085f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:47.080 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.075+0000 7f95a01fc740 1 -- 172.21.15.100:0/3850206134 >> 172.21.15.100:0/3850206134 conn(0x56026b077420 msgr2=0x56026b076580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:47.080 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.075+0000 7f95a01fc740 1 -- 172.21.15.100:0/3850206134 shutdown_connections 2024-05-10T11:20:47.080 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.076+0000 7f95a01fc740 1 -- 172.21.15.100:0/3850206134 wait complete. 2024-05-10T11:20:47.080 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.076+0000 7f95a01fc740 20 client.15810 trim_cache size 0 max 16384 2024-05-10T11:20:47.113 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.105+0000 7f95a01fc740 1 Processor -- start 2024-05-10T11:20:47.113 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.105+0000 7f95a01fc740 1 -- start start 2024-05-10T11:20:47.113 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.105+0000 7f95a01fc740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56026b07bdb0 0x56026b0723e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:47.113 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.105+0000 7f95a01fc740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56026b07c7a0 0x56026b08d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:47.113 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.105+0000 7f95a01fc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56026b081910 0x56026b07d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:47.113 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.105+0000 7f95a01fc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f95981ab990 con 0x56026b081910 2024-05-10T11:20:47.113 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.105+0000 7f95a01fc740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f95981a9260 con 0x56026b07bdb0 2024-05-10T11:20:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.105+0000 7f95a01fc740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f95981a70b0 con 0x56026b07c7a0 2024-05-10T11:20:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.105+0000 7f95977fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56026b081910 0x56026b07d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.105+0000 7f95977fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56026b081910 0x56026b07d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46396/0 (socket says 172.21.15.100:46396) 2024-05-10T11:20:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.105+0000 7f95977fe640 1 -- 172.21.15.100:0/759179928 learned_addr learned my addr 172.21.15.100:0/759179928 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:20:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.105+0000 7f958ffff640 1 --2- 172.21.15.100:0/759179928 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56026b07c7a0 0x56026b08d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.105+0000 7f9596ffd640 1 --2- 172.21.15.100:0/759179928 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56026b07bdb0 0x56026b0723e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.106+0000 7f958ffff640 1 -- 172.21.15.100:0/759179928 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56026b07bdb0 msgr2=0x56026b0723e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.106+0000 7f958ffff640 1 --2- 172.21.15.100:0/759179928 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56026b07bdb0 0x56026b0723e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.106+0000 7f958ffff640 1 -- 172.21.15.100:0/759179928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56026b081910 msgr2=0x56026b07d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.106+0000 7f958ffff640 1 --2- 172.21.15.100:0/759179928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56026b081910 0x56026b07d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.106+0000 7f958ffff640 1 -- 172.21.15.100:0/759179928 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56026b07d470 con 0x56026b07c7a0 2024-05-10T11:20:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.106+0000 7f95977fe640 1 --2- 172.21.15.100:0/759179928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56026b081910 0x56026b07d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:20:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.106+0000 7f958ffff640 1 --2- 172.21.15.100:0/759179928 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56026b07c7a0 0x56026b08d4d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x56026b0843a0 tx=0x7f958000b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e150c155745d568 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.106+0000 7f958cff9640 1 -- 172.21.15.100:0/759179928 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9580005000 con 0x56026b07c7a0 2024-05-10T11:20:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.106+0000 7f958cff9640 1 -- 172.21.15.100:0/759179928 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f95800051a0 con 0x56026b07c7a0 2024-05-10T11:20:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.106+0000 7f958cff9640 1 -- 172.21.15.100:0/759179928 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95800116a0 con 0x56026b07c7a0 2024-05-10T11:20:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.107+0000 7f95a01fc740 1 -- 172.21.15.100:0/759179928 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56026b07c7a0 msgr2=0x56026b08d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.107+0000 7f95a01fc740 1 --2- 172.21.15.100:0/759179928 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56026b07c7a0 0x56026b08d4d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x56026b0843a0 tx=0x7f958000b8a0 comp rx=0 tx=0).stop 2024-05-10T11:20:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.107+0000 7f95a01fc740 1 -- 172.21.15.100:0/759179928 shutdown_connections 2024-05-10T11:20:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.107+0000 7f95a01fc740 1 --2- 172.21.15.100:0/759179928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56026b081910 0x56026b07d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.107+0000 7f95a01fc740 1 --2- 172.21.15.100:0/759179928 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56026b07c7a0 0x56026b08d4d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.107+0000 7f95a01fc740 1 --2- 172.21.15.100:0/759179928 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56026b07bdb0 0x56026b0723e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.107+0000 7f95a01fc740 1 -- 172.21.15.100:0/759179928 >> 172.21.15.100:0/759179928 conn(0x56026b077420 msgr2=0x56026b080fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.107+0000 7f95a01fc740 1 -- 172.21.15.100:0/759179928 shutdown_connections 2024-05-10T11:20:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.107+0000 7f95a01fc740 1 -- 172.21.15.100:0/759179928 wait complete. 2024-05-10T11:20:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.107+0000 7f95a01fc740 1 -- start start 2024-05-10T11:20:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.108+0000 7f95a01fc740 1 Processor -- start 2024-05-10T11:20:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.108+0000 7f95a01fc740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56026b07bdb0 0x56026b08acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.108+0000 7f95a01fc740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56026b07c7a0 0x56026b08b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.108+0000 7f95a01fc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56026b081910 0x56026b08c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.108+0000 7f95a01fc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f95981a7720 con 0x56026b081910 2024-05-10T11:20:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.108+0000 7f95a01fc740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f95981a8b90 con 0x56026b07c7a0 2024-05-10T11:20:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.108+0000 7f95a01fc740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f95981a7df0 con 0x56026b07bdb0 2024-05-10T11:20:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.108+0000 7f95a01fc740 10 client.0 _ll_register_callbacks cb 0x7f9590031570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:20:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.108+0000 7f9596ffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56026b07bdb0 0x56026b08acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:47.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.108+0000 7f95977fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56026b081910 0x56026b08c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:47.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.108+0000 7f95977fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56026b081910 0x56026b08c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46400/0 (socket says 172.21.15.100:46400) 2024-05-10T11:20:47.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.108+0000 7f9596ffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56026b07bdb0 0x56026b08acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:49916/0 (socket says 172.21.15.100:49916) 2024-05-10T11:20:47.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.108+0000 7f95977fe640 1 -- 172.21.15.100:0/3316549935 learned_addr learned my addr 172.21.15.100:0/3316549935 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:20:47.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.108+0000 7f958ffff640 1 --2- 172.21.15.100:0/3316549935 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56026b07c7a0 0x56026b08b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:47.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.108+0000 7f9596ffd640 1 -- 172.21.15.100:0/3316549935 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56026b07c7a0 msgr2=0x56026b08b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:47.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.108+0000 7f9596ffd640 1 --2- 172.21.15.100:0/3316549935 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56026b07c7a0 0x56026b08b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:47.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.108+0000 7f9596ffd640 1 -- 172.21.15.100:0/3316549935 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56026b081910 msgr2=0x56026b08c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:47.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.108+0000 7f9596ffd640 1 --2- 172.21.15.100:0/3316549935 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56026b081910 0x56026b08c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:47.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.108+0000 7f9596ffd640 1 -- 172.21.15.100:0/3316549935 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56026b08c640 con 0x56026b07bdb0 2024-05-10T11:20:47.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.108+0000 7f95977fe640 1 --2- 172.21.15.100:0/3316549935 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56026b081910 0x56026b08c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:20:47.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.109+0000 7f9596ffd640 1 --2- 172.21.15.100:0/3316549935 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56026b07bdb0 0x56026b08acd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f95880e1430 tx=0x7f958800bf80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=522444704d82b4e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.109+0000 7f957e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:20:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.109+0000 7f957e7fc640 1 -- 172.21.15.100:0/3316549935 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95880b3d60 con 0x56026b07bdb0 2024-05-10T11:20:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.109+0000 7f957e7fc640 1 -- 172.21.15.100:0/3316549935 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f95880aa420 con 0x56026b07bdb0 2024-05-10T11:20:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.109+0000 7f95a01fc740 1 -- 172.21.15.100:0/3316549935 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95800051a0 con 0x56026b07bdb0 2024-05-10T11:20:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.109+0000 7f957e7fc640 1 -- 172.21.15.100:0/3316549935 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f95880a6920 con 0x56026b07bdb0 2024-05-10T11:20:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.109+0000 7f95a01fc740 10 client.25318 fetch_fsmap learned FSMap version 29 2024-05-10T11:20:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.109+0000 7f95a01fc740 1 -- 172.21.15.100:0/3316549935 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56026b0fc5a0 con 0x56026b07bdb0 2024-05-10T11:20:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.109+0000 7f957e7fc640 1 -- 172.21.15.100:0/3316549935 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f95880a6ac0 con 0x56026b07bdb0 2024-05-10T11:20:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.109+0000 7f957e7fc640 1 -- 172.21.15.100:0/3316549935 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f95880c5120 con 0x56026b07bdb0 2024-05-10T11:20:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.109+0000 7f95a01fc740 10 client.25318 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T11:20:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.109+0000 7f95a01fc740 10 client.25318 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:20:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.109+0000 7f95a01fc740 20 client.25318 populate_metadata read hostname 'smithi100' 2024-05-10T11:20:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.109+0000 7f95a01fc740 1 --2- 172.21.15.100:0/3316549935 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56026b0fcea0 0x56026b0ff290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.109+0000 7f95a01fc740 4 client.25318 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:20:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.109+0000 7f95a01fc740 1 -- 172.21.15.100:0/3316549935 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f95800051a0 con 0x56026b0fcea0 2024-05-10T11:20:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.109+0000 7f958ffff640 1 --2- 172.21.15.100:0/3316549935 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56026b0fcea0 0x56026b0ff290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.110+0000 7f957e7fc640 10 client.25318 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:20:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.110+0000 7f958ffff640 1 --2- 172.21.15.100:0/3316549935 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56026b0fcea0 0x56026b0ff290 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83c35e810fd7a6fd server_cookie=55073728c5514aab in_seq=0 out_seq=0 2024-05-10T11:20:47.129 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.128+0000 7f957e7fc640 1 -- 172.21.15.100:0/3316549935 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f95800051a0 con 0x56026b0fcea0 2024-05-10T11:20:47.130 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.128+0000 7f957e7fc640 10 client.25318 handle_command_reply: tid=0 2024-05-10T11:20:47.130 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.128+0000 7f95a01fc740 1 client.25318 shutdown 2024-05-10T11:20:47.130 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.129+0000 7f95a01fc740 1 -- 172.21.15.100:0/3316549935 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56026b07bdb0 msgr2=0x56026b08acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:47.131 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.129+0000 7f95a01fc740 1 --2- 172.21.15.100:0/3316549935 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56026b07bdb0 0x56026b08acd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f95880e1430 tx=0x7f958800bf80 comp rx=0 tx=0).stop 2024-05-10T11:20:47.131 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.129+0000 7f95a01fc740 1 -- 172.21.15.100:0/3316549935 shutdown_connections 2024-05-10T11:20:47.131 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.129+0000 7f95a01fc740 1 --2- 172.21.15.100:0/3316549935 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56026b0fcea0 0x56026b0ff290 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:47.131 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.129+0000 7f95a01fc740 1 --2- 172.21.15.100:0/3316549935 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56026b081910 0x56026b08c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:47.131 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.129+0000 7f95a01fc740 1 --2- 172.21.15.100:0/3316549935 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56026b07c7a0 0x56026b08b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:47.132 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.129+0000 7f95a01fc740 1 --2- 172.21.15.100:0/3316549935 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56026b07bdb0 0x56026b08acd0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:47.132 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.129+0000 7f957e7fc640 0 client.25318 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:20:47.132 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.129+0000 7f95a01fc740 1 -- 172.21.15.100:0/3316549935 >> 172.21.15.100:0/3316549935 conn(0x56026b077420 msgr2=0x56026b080bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:47.132 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.129+0000 7f95a01fc740 1 -- 172.21.15.100:0/3316549935 shutdown_connections 2024-05-10T11:20:47.132 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.129+0000 7f95a01fc740 1 -- 172.21.15.100:0/3316549935 wait complete. 2024-05-10T11:20:47.132 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.129+0000 7f95a01fc740 20 client.25318 trim_cache size 0 max 16384 2024-05-10T11:20:47.134 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:20:47.134 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:20:47.134 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:20:47.135 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:20:47.135 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.130+0000 7f959d4da640 1 -- 172.21.15.100:0/2923973766 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f95840b28a0 msgr2=0x7f95840b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.130+0000 7f959d4da640 1 --2- 172.21.15.100:0/2923973766 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f95840b28a0 0x7f95840b4d60 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9590006010 tx=0x7f95900319a0 comp rx=0 tx=0).stop 2024-05-10T11:20:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.130+0000 7f959d4da640 1 -- 172.21.15.100:0/2923973766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9598101910 msgr2=0x7f9598197990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.130+0000 7f959d4da640 1 --2- 172.21.15.100:0/2923973766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9598101910 0x7f9598197990 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f958800ead0 tx=0x7f958800ef90 comp rx=0 tx=0).stop 2024-05-10T11:20:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.130+0000 7f959d4da640 1 -- 172.21.15.100:0/2923973766 shutdown_connections 2024-05-10T11:20:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.130+0000 7f959d4da640 1 --2- 172.21.15.100:0/2923973766 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f95840b28a0 0x7f95840b4d60 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.130+0000 7f959d4da640 1 --2- 172.21.15.100:0/2923973766 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95981034c0 0x7f959819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.130+0000 7f959d4da640 1 --2- 172.21.15.100:0/2923973766 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9598102b00 0x7f9598197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.130+0000 7f959d4da640 1 --2- 172.21.15.100:0/2923973766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9598101910 0x7f9598197990 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.130+0000 7f959d4da640 1 -- 172.21.15.100:0/2923973766 >> 172.21.15.100:0/2923973766 conn(0x7f95980fce80 msgr2=0x7f9598105e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:47.137 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.130+0000 7f959d4da640 1 -- 172.21.15.100:0/2923973766 shutdown_connections 2024-05-10T11:20:47.137 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:20:47.130+0000 7f959d4da640 1 -- 172.21.15.100:0/2923973766 wait complete. 2024-05-10T11:20:47.245 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:20:47.932 INFO:tasks.workunit.client.0.smithi100.stdout: 1 23370 3.37 MB/sec warmup 42 sec latency 22.162 ms 2024-05-10T11:20:48.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:20:47 smithi100 ceph-mon[30296]: pgmap v492: 129 pgs: 129 active+clean; 29 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 1.0 MiB/s wr, 132 op/s 2024-05-10T11:20:48.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:20:47 smithi178 ceph-mon[33359]: pgmap v492: 129 pgs: 129 active+clean; 29 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 1.0 MiB/s wr, 132 op/s 2024-05-10T11:20:48.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:20:47 smithi188 ceph-mon[34075]: pgmap v492: 129 pgs: 129 active+clean; 29 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 1.0 MiB/s wr, 132 op/s 2024-05-10T11:20:48.932 INFO:tasks.workunit.client.0.smithi100.stdout: 1 23917 3.34 MB/sec warmup 43 sec latency 15.807 ms 2024-05-10T11:20:49.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:49.172+0000 7ff9d9ee8700 1 -- 172.21.15.100:0/2105518708 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9d4108400 msgr2=0x7ff9d41087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:49.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:49.172+0000 7ff9d9ee8700 1 --2- 172.21.15.100:0/2105518708 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9d4108400 0x7ff9d41087e0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4009990 tx=0x7ff9c4009ca0 comp rx=0 tx=0).stop 2024-05-10T11:20:49.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:49.173+0000 7ff9d9ee8700 1 -- 172.21.15.100:0/2105518708 shutdown_connections 2024-05-10T11:20:49.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:49.173+0000 7ff9d9ee8700 1 --2- 172.21.15.100:0/2105518708 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9d4102d50 0x7ff9d410f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:49.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:49.173+0000 7ff9d9ee8700 1 --2- 172.21.15.100:0/2105518708 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9d4108400 0x7ff9d41087e0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:49.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:49.173+0000 7ff9d9ee8700 1 --2- 172.21.15.100:0/2105518708 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9d4102390 0x7ff9d4102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:49.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:49.173+0000 7ff9d9ee8700 1 -- 172.21.15.100:0/2105518708 >> 172.21.15.100:0/2105518708 conn(0x7ff9d4079310 msgr2=0x7ff9d4079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:49.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:49.173+0000 7ff9d9ee8700 1 -- 172.21.15.100:0/2105518708 shutdown_connections 2024-05-10T11:20:49.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:49.173+0000 7ff9d9ee8700 1 -- 172.21.15.100:0/2105518708 wait complete. 2024-05-10T11:20:49.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:49.173+0000 7ff9d9ee8700 1 Processor -- start 2024-05-10T11:20:49.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:49.173+0000 7ff9d9ee8700 1 -- start start 2024-05-10T11:20:49.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:49.174+0000 7ff9d9ee8700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9d4102390 0x7ff9d419a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:49.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:49.174+0000 7ff9d9ee8700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9d4102d50 0x7ff9d419ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:49.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:49.174+0000 7ff9d9ee8700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9d4108400 0x7ff9d41a5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:49.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:49.174+0000 7ff9d9ee8700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d4112400 con 0x7ff9d4102d50 2024-05-10T11:20:49.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:49.174+0000 7ff9d9ee8700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d4112280 con 0x7ff9d4102390 2024-05-10T11:20:49.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:49.174+0000 7ff9d9ee8700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d4112580 con 0x7ff9d4108400 2024-05-10T11:20:49.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:49.174+0000 7ff9d2ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9d4102d50 0x7ff9d419ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:49.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:49.174+0000 7ff9d2ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9d4102d50 0x7ff9d419ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46416/0 (socket says 172.21.15.100:46416) 2024-05-10T11:20:49.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:49.174+0000 7ff9d2ffd700 1 -- 172.21.15.100:0/2507263554 learned_addr learned my addr 172.21.15.100:0/2507263554 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:20:49.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:49.174+0000 7ff9d3fff700 1 --2- 172.21.15.100:0/2507263554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9d4108400 0x7ff9d41a5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:49.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:49.174+0000 7ff9d37fe700 1 --2- 172.21.15.100:0/2507263554 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9d4102390 0x7ff9d419a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:49.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:49.174+0000 7ff9d3fff700 1 -- 172.21.15.100:0/2507263554 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9d4102390 msgr2=0x7ff9d419a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:49.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:49.174+0000 7ff9d3fff700 1 --2- 172.21.15.100:0/2507263554 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9d4102390 0x7ff9d419a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:49.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:49.174+0000 7ff9d3fff700 1 -- 172.21.15.100:0/2507263554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9d4102d50 msgr2=0x7ff9d419ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:49.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:49.174+0000 7ff9d3fff700 1 --2- 172.21.15.100:0/2507263554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9d4102d50 0x7ff9d419ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:49.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:49.174+0000 7ff9d3fff700 1 -- 172.21.15.100:0/2507263554 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9d41a5ea0 con 0x7ff9d4108400 2024-05-10T11:20:49.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:49.174+0000 7ff9d2ffd700 1 --2- 172.21.15.100:0/2507263554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9d4102d50 0x7ff9d419ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:20:49.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:49.174+0000 7ff9d3fff700 1 --2- 172.21.15.100:0/2507263554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9d4108400 0x7ff9d41a5960 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc00efc0 tx=0x7ff9cc00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:49.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:49.175+0000 7ff9d0ff9700 1 -- 172.21.15.100:0/2507263554 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc0092e0 con 0x7ff9d4108400 2024-05-10T11:20:49.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:49.175+0000 7ff9d9ee8700 1 -- 172.21.15.100:0/2507263554 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9d41a6070 con 0x7ff9d4108400 2024-05-10T11:20:49.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:49.175+0000 7ff9d0ff9700 1 -- 172.21.15.100:0/2507263554 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff9cc004bb0 con 0x7ff9d4108400 2024-05-10T11:20:49.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:49.175+0000 7ff9d0ff9700 1 -- 172.21.15.100:0/2507263554 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc004020 con 0x7ff9d4108400 2024-05-10T11:20:49.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:49.175+0000 7ff9d9ee8700 1 -- 172.21.15.100:0/2507263554 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9d41a65b0 con 0x7ff9d4108400 2024-05-10T11:20:49.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:49.176+0000 7ff9d9ee8700 1 -- 172.21.15.100:0/2507263554 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9d406a190 con 0x7ff9d4108400 2024-05-10T11:20:49.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:49.177+0000 7ff9d0ff9700 1 -- 172.21.15.100:0/2507263554 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7ff9cc01b030 con 0x7ff9d4108400 2024-05-10T11:20:49.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:49.178+0000 7ff9d0ff9700 1 --2- 172.21.15.100:0/2507263554 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff9bc0b2900 0x7ff9bc0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:49.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:49.178+0000 7ff9d37fe700 1 --2- 172.21.15.100:0/2507263554 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff9bc0b2900 0x7ff9bc0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:49.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:49.178+0000 7ff9d0ff9700 1 -- 172.21.15.100:0/2507263554 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7ff9cc0e1f60 con 0x7ff9d4108400 2024-05-10T11:20:49.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:49.179+0000 7ff9d37fe700 1 --2- 172.21.15.100:0/2507263554 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff9bc0b2900 0x7ff9bc0b4dc0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff9c400b2e0 tx=0x7ff9c4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:49.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:49.183+0000 7ff9d0ff9700 1 -- 172.21.15.100:0/2507263554 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7ff9cc022020 con 0x7ff9d4108400 2024-05-10T11:20:49.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:49.469+0000 7ff9d9ee8700 1 -- 172.21.15.100:0/2507263554 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff9d402e510 con 0x7ff9d4108400 2024-05-10T11:20:49.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:49.470+0000 7ff9d0ff9700 1 -- 172.21.15.100:0/2507263554 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11897 (secure 0 0 0) 0x7ff9cc010040 con 0x7ff9d4108400 2024-05-10T11:20:49.470 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:20:49.470 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2072225012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2072225012},{"type":"v1","addr":"172.21.15.100:6835","nonce":2072225012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/401939540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":401939540},{"type":"v1","addr":"172.21.15.100:6837","nonce":401939540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14571,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/3946311025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3946311025},{"type":"v1","addr":"172.21.15.100:6839","nonce":3946311025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3736371660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3736371660},{"type":"v1","addr":"172.21.15.100:6841","nonce":3736371660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/2741810791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2741810791},{"type":"v1","addr":"172.21.15.178:6833","nonce":2741810791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1704891041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1704891041},{"type":"v1","addr":"172.21.15.188:6833","nonce":1704891041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T11:14:27.242452+0000","modified":"2024-05-10T11:20:29.528470+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24439,"mds_1":24437,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/2124076312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2124076312},{"type":"v1","addr":"172.21.15.178:6835","nonce":2124076312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/3224358425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3224358425},{"type":"v1","addr":"172.21.15.178:6837","nonce":3224358425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/515547690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":515547690},{"type":"v1","addr":"172.21.15.188:6835","nonce":515547690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/1943342620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1943342620},{"type":"v1","addr":"172.21.15.188:6837","nonce":1943342620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.178:6839/3760985404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3760985404},{"type":"v1","addr":"172.21.15.178:6839","nonce":3760985404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.188:6839/3194917070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":3194917070},{"type":"v1","addr":"172.21.15.188:6839","nonce":3194917070}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24439,"qdb_cluster":[24437,24439,24427]},"id":1}]} 2024-05-10T11:20:49.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:49.472+0000 7ff9d9ee8700 1 -- 172.21.15.100:0/2507263554 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff9bc0b2900 msgr2=0x7ff9bc0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:49.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:49.472+0000 7ff9d9ee8700 1 --2- 172.21.15.100:0/2507263554 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff9bc0b2900 0x7ff9bc0b4dc0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff9c400b2e0 tx=0x7ff9c4005fb0 comp rx=0 tx=0).stop 2024-05-10T11:20:49.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:49.472+0000 7ff9d9ee8700 1 -- 172.21.15.100:0/2507263554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9d4108400 msgr2=0x7ff9d41a5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:49.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:49.472+0000 7ff9d9ee8700 1 --2- 172.21.15.100:0/2507263554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9d4108400 0x7ff9d41a5960 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc00efc0 tx=0x7ff9cc00c3a0 comp rx=0 tx=0).stop 2024-05-10T11:20:49.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:49.472+0000 7ff9d9ee8700 1 -- 172.21.15.100:0/2507263554 shutdown_connections 2024-05-10T11:20:49.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:49.472+0000 7ff9d9ee8700 1 --2- 172.21.15.100:0/2507263554 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff9bc0b2900 0x7ff9bc0b4dc0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:49.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:49.472+0000 7ff9d9ee8700 1 --2- 172.21.15.100:0/2507263554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9d4108400 0x7ff9d41a5960 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:49.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:49.472+0000 7ff9d9ee8700 1 --2- 172.21.15.100:0/2507263554 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9d4102390 0x7ff9d419a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:49.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:49.472+0000 7ff9d9ee8700 1 --2- 172.21.15.100:0/2507263554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9d4102d50 0x7ff9d419ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:49.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:49.472+0000 7ff9d9ee8700 1 -- 172.21.15.100:0/2507263554 >> 172.21.15.100:0/2507263554 conn(0x7ff9d4079310 msgr2=0x7ff9d4101280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:49.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:49.473+0000 7ff9d9ee8700 1 -- 172.21.15.100:0/2507263554 shutdown_connections 2024-05-10T11:20:49.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:49.473+0000 7ff9d9ee8700 1 -- 172.21.15.100:0/2507263554 wait complete. 2024-05-10T11:20:49.474 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 29 2024-05-10T11:20:49.933 INFO:tasks.workunit.client.0.smithi100.stdout: 1 24637 3.41 MB/sec warmup 44 sec latency 42.725 ms 2024-05-10T11:20:50.154 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:20:49 smithi100 ceph-mon[30296]: pgmap v493: 129 pgs: 129 active+clean; 29 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 869 KiB/s wr, 96 op/s 2024-05-10T11:20:50.154 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:20:49 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/2507263554' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:20:50.184 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-10T11:20:50.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:20:49 smithi178 ceph-mon[33359]: pgmap v493: 129 pgs: 129 active+clean; 29 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 869 KiB/s wr, 96 op/s 2024-05-10T11:20:50.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:20:49 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/2507263554' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:20:50.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:20:49 smithi188 ceph-mon[34075]: pgmap v493: 129 pgs: 129 active+clean; 29 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 869 KiB/s wr, 96 op/s 2024-05-10T11:20:50.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:20:49 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/2507263554' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:20:50.481 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:20:50.933 INFO:tasks.workunit.client.0.smithi100.stdout: 1 25112 3.43 MB/sec warmup 45 sec latency 14.253 ms 2024-05-10T11:20:51.932 INFO:tasks.workunit.client.0.smithi100.stdout: 1 25378 3.37 MB/sec warmup 46 sec latency 15.116 ms 2024-05-10T11:20:52.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:20:51 smithi100 ceph-mon[30296]: pgmap v494: 129 pgs: 129 active+clean; 29 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 1021 KiB/s wr, 132 op/s 2024-05-10T11:20:52.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:20:51 smithi178 ceph-mon[33359]: pgmap v494: 129 pgs: 129 active+clean; 29 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 1021 KiB/s wr, 132 op/s 2024-05-10T11:20:52.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:20:51 smithi188 ceph-mon[34075]: pgmap v494: 129 pgs: 129 active+clean; 29 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 1021 KiB/s wr, 132 op/s 2024-05-10T11:20:52.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.276+0000 7f3e8eabb700 1 -- 172.21.15.100:0/4194740679 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e88075c10 msgr2=0x7f3e88076090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:52.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.276+0000 7f3e8eabb700 1 --2- 172.21.15.100:0/4194740679 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e88075c10 0x7f3e88076090 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f3e78009990 tx=0x7f3e78009ca0 comp rx=0 tx=0).stop 2024-05-10T11:20:52.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.276+0000 7f3e8eabb700 1 -- 172.21.15.100:0/4194740679 shutdown_connections 2024-05-10T11:20:52.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.276+0000 7f3e8eabb700 1 --2- 172.21.15.100:0/4194740679 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e88114190 0x7f3e88116580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:52.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.276+0000 7f3e8eabb700 1 --2- 172.21.15.100:0/4194740679 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3e880774c0 0x7f3e880756d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:52.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.276+0000 7f3e8eabb700 1 --2- 172.21.15.100:0/4194740679 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e88075c10 0x7f3e88076090 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:52.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.276+0000 7f3e8eabb700 1 -- 172.21.15.100:0/4194740679 >> 172.21.15.100:0/4194740679 conn(0x7f3e8807a530 msgr2=0x7f3e8807a940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:52.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.276+0000 7f3e8eabb700 1 -- 172.21.15.100:0/4194740679 shutdown_connections 2024-05-10T11:20:52.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.276+0000 7f3e8eabb700 1 -- 172.21.15.100:0/4194740679 wait complete. 2024-05-10T11:20:52.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.277+0000 7f3e8eabb700 1 Processor -- start 2024-05-10T11:20:52.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.277+0000 7f3e8eabb700 1 -- start start 2024-05-10T11:20:52.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.277+0000 7f3e8eabb700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e88075c10 0x7f3e881a0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:52.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.277+0000 7f3e8eabb700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e880774c0 0x7f3e881a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:52.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.277+0000 7f3e8eabb700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3e88114190 0x7f3e881a49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:52.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.277+0000 7f3e8eabb700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3e88119b70 con 0x7f3e880774c0 2024-05-10T11:20:52.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.277+0000 7f3e8eabb700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3e881199f0 con 0x7f3e88114190 2024-05-10T11:20:52.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.277+0000 7f3e8eabb700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3e88119cf0 con 0x7f3e88075c10 2024-05-10T11:20:52.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.277+0000 7f3e7ffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e880774c0 0x7f3e881a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:52.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.277+0000 7f3e8c857700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e88075c10 0x7f3e881a0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:52.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.277+0000 7f3e7ffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e880774c0 0x7f3e881a0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46446/0 (socket says 172.21.15.100:46446) 2024-05-10T11:20:52.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.277+0000 7f3e8d058700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3e88114190 0x7f3e881a49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:52.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.277+0000 7f3e7ffff700 1 -- 172.21.15.100:0/753417383 learned_addr learned my addr 172.21.15.100:0/753417383 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:20:52.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.277+0000 7f3e8c857700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e88075c10 0x7f3e881a0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:49960/0 (socket says 172.21.15.100:49960) 2024-05-10T11:20:52.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.277+0000 7f3e8c857700 1 -- 172.21.15.100:0/753417383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3e88114190 msgr2=0x7f3e881a49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:52.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.277+0000 7f3e8c857700 1 --2- 172.21.15.100:0/753417383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3e88114190 0x7f3e881a49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:52.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.277+0000 7f3e8c857700 1 -- 172.21.15.100:0/753417383 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e880774c0 msgr2=0x7f3e881a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:52.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.277+0000 7f3e8c857700 1 --2- 172.21.15.100:0/753417383 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e880774c0 0x7f3e881a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:52.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.277+0000 7f3e8c857700 1 -- 172.21.15.100:0/753417383 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e881a5130 con 0x7f3e88075c10 2024-05-10T11:20:52.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.277+0000 7f3e7ffff700 1 --2- 172.21.15.100:0/753417383 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e880774c0 0x7f3e881a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:20:52.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.277+0000 7f3e8d058700 1 --2- 172.21.15.100:0/753417383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3e88114190 0x7f3e881a49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:20:52.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.278+0000 7f3e8c857700 1 --2- 172.21.15.100:0/753417383 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e88075c10 0x7f3e881a0120 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3e8400ef10 tx=0x7f3e8400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:52.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.278+0000 7f3e7dffb700 1 -- 172.21.15.100:0/753417383 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e8400c9c0 con 0x7f3e88075c10 2024-05-10T11:20:52.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.278+0000 7f3e7dffb700 1 -- 172.21.15.100:0/753417383 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3e84019070 con 0x7f3e88075c10 2024-05-10T11:20:52.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.278+0000 7f3e8eabb700 1 -- 172.21.15.100:0/753417383 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e881a53c0 con 0x7f3e88075c10 2024-05-10T11:20:52.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.278+0000 7f3e7dffb700 1 -- 172.21.15.100:0/753417383 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e84014480 con 0x7f3e88075c10 2024-05-10T11:20:52.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.278+0000 7f3e8eabb700 1 -- 172.21.15.100:0/753417383 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e881acca0 con 0x7f3e88075c10 2024-05-10T11:20:52.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.280+0000 7f3e7dffb700 1 -- 172.21.15.100:0/753417383 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f3e84012050 con 0x7f3e88075c10 2024-05-10T11:20:52.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.281+0000 7f3e7dffb700 1 --2- 172.21.15.100:0/753417383 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f3e700b2900 0x7f3e700b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:52.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.281+0000 7f3e7ffff700 1 --2- 172.21.15.100:0/753417383 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f3e700b2900 0x7f3e700b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:52.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.281+0000 7f3e7dffb700 1 -- 172.21.15.100:0/753417383 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f3e840e2470 con 0x7f3e88075c10 2024-05-10T11:20:52.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.282+0000 7f3e7ffff700 1 --2- 172.21.15.100:0/753417383 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f3e700b2900 0x7f3e700b4dc0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3e88113f90 tx=0x7f3e7801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:52.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.282+0000 7f3e9934eb80 1 Processor -- start 2024-05-10T11:20:52.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.282+0000 7f3e9934eb80 1 -- start start 2024-05-10T11:20:52.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.282+0000 7f3e9934eb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639f65b21b0 0x5639f65b2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:52.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.282+0000 7f3e9934eb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639f65b2ad0 0x5639f65ac810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:52.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.282+0000 7f3e9934eb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639f65acd50 0x5639f65ad190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:52.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.282+0000 7f3e9934eb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5639f64fe940 con 0x5639f65b2ad0 2024-05-10T11:20:52.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.282+0000 7f3e9934eb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5639f6463da0 con 0x5639f65b21b0 2024-05-10T11:20:52.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.282+0000 7f3e9934eb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5639f64fa900 con 0x5639f65acd50 2024-05-10T11:20:52.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.282+0000 7f3e7ffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639f65b2ad0 0x5639f65ac810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:52.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.282+0000 7f3e8c857700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639f65b21b0 0x5639f65b2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:52.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.282+0000 7f3e7ffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639f65b2ad0 0x5639f65ac810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46448/0 (socket says 172.21.15.100:46448) 2024-05-10T11:20:52.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.282+0000 7f3e7ffff700 1 -- 172.21.15.100:0/1889429009 learned_addr learned my addr 172.21.15.100:0/1889429009 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:20:52.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.282+0000 7f3e8d058700 1 --2- 172.21.15.100:0/1889429009 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639f65acd50 0x5639f65ad190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:52.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.282+0000 7f3e8c857700 1 -- 172.21.15.100:0/1889429009 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639f65acd50 msgr2=0x5639f65ad190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:52.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.282+0000 7f3e8c857700 1 --2- 172.21.15.100:0/1889429009 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639f65acd50 0x5639f65ad190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:52.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.282+0000 7f3e8c857700 1 -- 172.21.15.100:0/1889429009 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639f65b2ad0 msgr2=0x5639f65ac810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:52.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.282+0000 7f3e8c857700 1 --2- 172.21.15.100:0/1889429009 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639f65b2ad0 0x5639f65ac810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:52.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.282+0000 7f3e8c857700 1 -- 172.21.15.100:0/1889429009 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639f644b8d0 con 0x5639f65b21b0 2024-05-10T11:20:52.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.283+0000 7f3e8c857700 1 --2- 172.21.15.100:0/1889429009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639f65b21b0 0x5639f65b2590 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3e8400eee0 tx=0x7f3e840a8ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a5a7bc98b819a15 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:52.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.283+0000 7f3e6effd700 1 -- 172.21.15.100:0/1889429009 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e840e2070 con 0x5639f65b21b0 2024-05-10T11:20:52.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.283+0000 7f3e6effd700 1 -- 172.21.15.100:0/1889429009 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3e840a3e50 con 0x5639f65b21b0 2024-05-10T11:20:52.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.283+0000 7f3e6effd700 1 -- 172.21.15.100:0/1889429009 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e840b0e60 con 0x5639f65b21b0 2024-05-10T11:20:52.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.283+0000 7f3e9934eb80 1 -- 172.21.15.100:0/1889429009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639f65b21b0 msgr2=0x5639f65b2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:52.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.284+0000 7f3e9934eb80 1 --2- 172.21.15.100:0/1889429009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639f65b21b0 0x5639f65b2590 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3e8400eee0 tx=0x7f3e840a8ea0 comp rx=0 tx=0).stop 2024-05-10T11:20:52.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.284+0000 7f3e9934eb80 1 -- 172.21.15.100:0/1889429009 shutdown_connections 2024-05-10T11:20:52.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.284+0000 7f3e9934eb80 1 --2- 172.21.15.100:0/1889429009 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639f65acd50 0x5639f65ad190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:52.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.284+0000 7f3e9934eb80 1 --2- 172.21.15.100:0/1889429009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639f65b21b0 0x5639f65b2590 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:52.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.284+0000 7f3e9934eb80 1 --2- 172.21.15.100:0/1889429009 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639f65b2ad0 0x5639f65ac810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:52.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.284+0000 7f3e9934eb80 1 -- 172.21.15.100:0/1889429009 >> 172.21.15.100:0/1889429009 conn(0x5639f64d8b70 msgr2=0x5639f65a72a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:52.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.284+0000 7f3e9934eb80 1 -- 172.21.15.100:0/1889429009 shutdown_connections 2024-05-10T11:20:52.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.284+0000 7f3e9934eb80 1 -- 172.21.15.100:0/1889429009 wait complete. 2024-05-10T11:20:52.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.284+0000 7f3e9934eb80 1 -- start start 2024-05-10T11:20:52.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.284+0000 7f3e9934eb80 1 Processor -- start 2024-05-10T11:20:52.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.285+0000 7f3e9934eb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639f65acd50 0x5639f65b87c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:52.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.285+0000 7f3e9934eb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639f65b21b0 0x5639f65b5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:52.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.285+0000 7f3e9934eb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639f65b2ad0 0x5639f65b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:52.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.285+0000 7f3e9934eb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5639f65b9450 con 0x5639f65b21b0 2024-05-10T11:20:52.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.285+0000 7f3e9934eb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5639f64b7ed0 con 0x5639f65acd50 2024-05-10T11:20:52.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.285+0000 7f3e9934eb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5639f65b95d0 con 0x5639f65b2ad0 2024-05-10T11:20:52.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.285+0000 7f3e9934eb80 10 client.0 _ll_register_callbacks cb 0x5639f64925b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:20:52.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.285+0000 7f3e7ffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639f65b21b0 0x5639f65b5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:52.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.285+0000 7f3e7ffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639f65b21b0 0x5639f65b5b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46458/0 (socket says 172.21.15.100:46458) 2024-05-10T11:20:52.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.285+0000 7f3e7ffff700 1 -- 172.21.15.100:0/3289113496 learned_addr learned my addr 172.21.15.100:0/3289113496 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:20:52.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.285+0000 7f3e8c857700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639f65acd50 0x5639f65b87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:52.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.285+0000 7f3e8d058700 1 --2- 172.21.15.100:0/3289113496 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639f65b2ad0 0x5639f65b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:52.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.285+0000 7f3e8c857700 1 -- 172.21.15.100:0/3289113496 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639f65b2ad0 msgr2=0x5639f65b60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:52.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.285+0000 7f3e8c857700 1 --2- 172.21.15.100:0/3289113496 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639f65b2ad0 0x5639f65b60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:52.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.285+0000 7f3e8c857700 1 -- 172.21.15.100:0/3289113496 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639f65b21b0 msgr2=0x5639f65b5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:52.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.285+0000 7f3e8c857700 1 --2- 172.21.15.100:0/3289113496 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639f65b21b0 0x5639f65b5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:52.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.285+0000 7f3e8c857700 1 -- 172.21.15.100:0/3289113496 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639f64b5760 con 0x5639f65acd50 2024-05-10T11:20:52.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.285+0000 7f3e8c857700 1 --2- 172.21.15.100:0/3289113496 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639f65acd50 0x5639f65b87c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3e840a98f0 tx=0x7f3e840cc170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=322dec7981493e64 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:52.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.285+0000 7f3e67fff700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:20:52.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.285+0000 7f3e67fff700 1 -- 172.21.15.100:0/3289113496 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e840e2040 con 0x5639f65acd50 2024-05-10T11:20:52.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.285+0000 7f3e67fff700 1 -- 172.21.15.100:0/3289113496 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f3e840a9ba0 con 0x5639f65acd50 2024-05-10T11:20:52.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.285+0000 7f3e9934eb80 1 -- 172.21.15.100:0/3289113496 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e840a3e50 con 0x5639f65acd50 2024-05-10T11:20:52.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.286+0000 7f3e67fff700 1 -- 172.21.15.100:0/3289113496 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3e840a9730 con 0x5639f65acd50 2024-05-10T11:20:52.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.286+0000 7f3e9934eb80 10 client.25340 fetch_fsmap learned FSMap version 29 2024-05-10T11:20:52.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.286+0000 7f3e9934eb80 1 -- 172.21.15.100:0/3289113496 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5639f65c3d80 con 0x5639f65acd50 2024-05-10T11:20:52.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.286+0000 7f3e67fff700 1 -- 172.21.15.100:0/3289113496 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3e840a3e50 con 0x5639f65acd50 2024-05-10T11:20:52.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.286+0000 7f3e67fff700 1 -- 172.21.15.100:0/3289113496 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f3e840beb80 con 0x5639f65acd50 2024-05-10T11:20:52.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.286+0000 7f3e9934eb80 10 client.25340 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T11:20:52.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.286+0000 7f3e9934eb80 10 client.25340 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-10T11:20:52.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.286+0000 7f3e9934eb80 20 client.25340 populate_metadata read hostname 'smithi100' 2024-05-10T11:20:52.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.286+0000 7f3e9934eb80 1 --2- 172.21.15.100:0/3289113496 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x5639f65c6250 0x5639f65c8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:52.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.286+0000 7f3e9934eb80 4 client.25340 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:20:52.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.286+0000 7f3e9934eb80 1 -- 172.21.15.100:0/3289113496 --> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5639f650c320 con 0x5639f65c6250 2024-05-10T11:20:52.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.286+0000 7f3e7ffff700 1 --2- 172.21.15.100:0/3289113496 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x5639f65c6250 0x5639f65c8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:52.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.287+0000 7f3e7ffff700 1 --2- 172.21.15.100:0/3289113496 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x5639f65c6250 0x5639f65c8640 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bd814e2df264f2df server_cookie=f2421a5b5a9c3b95 in_seq=0 out_seq=0 2024-05-10T11:20:52.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.287+0000 7f3e67fff700 10 client.25340 ms_handle_connect on v2:172.21.15.188:6836/1943342620 2024-05-10T11:20:52.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.287+0000 7f3e67fff700 1 -- 172.21.15.100:0/3289113496 <== mds.2 v2:172.21.15.188:6836/1943342620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5639f650c320 con 0x5639f65c6250 2024-05-10T11:20:52.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.287+0000 7f3e67fff700 10 client.25340 handle_command_reply: tid=0 2024-05-10T11:20:52.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.288+0000 7f3e9934eb80 1 client.25340 shutdown 2024-05-10T11:20:52.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.288+0000 7f3e9934eb80 1 -- 172.21.15.100:0/3289113496 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639f65acd50 msgr2=0x5639f65b87c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:52.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.288+0000 7f3e9934eb80 1 --2- 172.21.15.100:0/3289113496 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639f65acd50 0x5639f65b87c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3e840a98f0 tx=0x7f3e840cc170 comp rx=0 tx=0).stop 2024-05-10T11:20:52.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.288+0000 7f3e9934eb80 1 -- 172.21.15.100:0/3289113496 shutdown_connections 2024-05-10T11:20:52.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.288+0000 7f3e9934eb80 1 --2- 172.21.15.100:0/3289113496 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x5639f65c6250 0x5639f65c8640 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:52.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.288+0000 7f3e9934eb80 1 --2- 172.21.15.100:0/3289113496 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639f65b2ad0 0x5639f65b60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:52.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.288+0000 7f3e9934eb80 1 --2- 172.21.15.100:0/3289113496 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639f65acd50 0x5639f65b87c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:52.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.288+0000 7f3e9934eb80 1 --2- 172.21.15.100:0/3289113496 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639f65b21b0 0x5639f65b5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:52.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.288+0000 7f3e9934eb80 1 -- 172.21.15.100:0/3289113496 >> 172.21.15.100:0/3289113496 conn(0x5639f64d8b70 msgr2=0x5639f65a1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:52.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.288+0000 7f3e67fff700 0 client.25340 ms_handle_reset on v2:172.21.15.188:6836/1943342620 2024-05-10T11:20:52.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.288+0000 7f3e9934eb80 1 -- 172.21.15.100:0/3289113496 shutdown_connections 2024-05-10T11:20:52.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.288+0000 7f3e9934eb80 1 -- 172.21.15.100:0/3289113496 wait complete. 2024-05-10T11:20:52.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.288+0000 7f3e9934eb80 20 client.25340 trim_cache size 0 max 16384 2024-05-10T11:20:52.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.304+0000 7f3e9934eb80 1 Processor -- start 2024-05-10T11:20:52.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.304+0000 7f3e9934eb80 1 -- start start 2024-05-10T11:20:52.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.304+0000 7f3e9934eb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639f65acd50 0x5639f65ac810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:52.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.304+0000 7f3e9934eb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639f65b21b0 0x5639f65a1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:52.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.304+0000 7f3e9934eb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639f65b2ad0 0x5639f65b4b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:52.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.304+0000 7f3e9934eb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3e881b7fd0 con 0x5639f65b2ad0 2024-05-10T11:20:52.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.304+0000 7f3e9934eb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3e881b5860 con 0x5639f65acd50 2024-05-10T11:20:52.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.304+0000 7f3e9934eb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3e881b36b0 con 0x5639f65b21b0 2024-05-10T11:20:52.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.304+0000 7f3e8d058700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639f65b2ad0 0x5639f65b4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:52.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.304+0000 7f3e8d058700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639f65b2ad0 0x5639f65b4b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46470/0 (socket says 172.21.15.100:46470) 2024-05-10T11:20:52.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.304+0000 7f3e8c857700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639f65acd50 0x5639f65ac810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:52.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.304+0000 7f3e8d058700 1 -- 172.21.15.100:0/794357405 learned_addr learned my addr 172.21.15.100:0/794357405 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:20:52.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.305+0000 7f3e7ffff700 1 --2- 172.21.15.100:0/794357405 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639f65b21b0 0x5639f65a1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:52.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.305+0000 7f3e8d058700 1 -- 172.21.15.100:0/794357405 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639f65b21b0 msgr2=0x5639f65a1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:52.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.305+0000 7f3e8d058700 1 --2- 172.21.15.100:0/794357405 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639f65b21b0 0x5639f65a1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:52.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.305+0000 7f3e8d058700 1 -- 172.21.15.100:0/794357405 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639f65acd50 msgr2=0x5639f65ac810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:52.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.305+0000 7f3e8d058700 1 --2- 172.21.15.100:0/794357405 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639f65acd50 0x5639f65ac810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:52.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.305+0000 7f3e8d058700 1 -- 172.21.15.100:0/794357405 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639f65b27e0 con 0x5639f65b2ad0 2024-05-10T11:20:52.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.305+0000 7f3e7ffff700 1 --2- 172.21.15.100:0/794357405 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639f65b21b0 0x5639f65a1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:20:52.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.305+0000 7f3e8c857700 1 --2- 172.21.15.100:0/794357405 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639f65acd50 0x5639f65ac810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:20:52.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.305+0000 7f3e8d058700 1 --2- 172.21.15.100:0/794357405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639f65b2ad0 0x5639f65b4b60 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f3e800116e0 tx=0x7f3e80009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa66fc9e150db2a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:52.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.305+0000 7f3e6e7fc700 1 -- 172.21.15.100:0/794357405 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e80004830 con 0x5639f65b2ad0 2024-05-10T11:20:52.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.305+0000 7f3e6e7fc700 1 -- 172.21.15.100:0/794357405 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3e80005020 con 0x5639f65b2ad0 2024-05-10T11:20:52.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.305+0000 7f3e6e7fc700 1 -- 172.21.15.100:0/794357405 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e8001c650 con 0x5639f65b2ad0 2024-05-10T11:20:52.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.306+0000 7f3e9934eb80 1 -- 172.21.15.100:0/794357405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639f65b2ad0 msgr2=0x5639f65b4b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:52.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.306+0000 7f3e9934eb80 1 --2- 172.21.15.100:0/794357405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639f65b2ad0 0x5639f65b4b60 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f3e800116e0 tx=0x7f3e80009260 comp rx=0 tx=0).stop 2024-05-10T11:20:52.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.306+0000 7f3e9934eb80 1 -- 172.21.15.100:0/794357405 shutdown_connections 2024-05-10T11:20:52.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.306+0000 7f3e9934eb80 1 --2- 172.21.15.100:0/794357405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639f65b2ad0 0x5639f65b4b60 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:52.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.306+0000 7f3e9934eb80 1 --2- 172.21.15.100:0/794357405 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639f65acd50 0x5639f65ac810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:52.347 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:20:52.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.306+0000 7f3e9934eb80 1 --2- 172.21.15.100:0/794357405 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639f65b21b0 0x5639f65a1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:52.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.306+0000 7f3e9934eb80 1 -- 172.21.15.100:0/794357405 >> 172.21.15.100:0/794357405 conn(0x5639f64d8b70 msgr2=0x5639f65b4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:52.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.306+0000 7f3e9934eb80 1 -- 172.21.15.100:0/794357405 shutdown_connections 2024-05-10T11:20:52.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.306+0000 7f3e9934eb80 1 -- 172.21.15.100:0/794357405 wait complete. 2024-05-10T11:20:52.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.306+0000 7f3e9934eb80 1 -- start start 2024-05-10T11:20:52.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.306+0000 7f3e9934eb80 1 Processor -- start 2024-05-10T11:20:52.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.307+0000 7f3e9934eb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639f65acd50 0x5639f65c8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:52.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.307+0000 7f3e9934eb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639f65b21b0 0x5639f65c8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:52.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.307+0000 7f3e9934eb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639f65b2ad0 0x5639f662e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:52.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.307+0000 7f3e9934eb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3e881b3d20 con 0x5639f65b21b0 2024-05-10T11:20:52.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.307+0000 7f3e9934eb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3e881b5190 con 0x5639f65acd50 2024-05-10T11:20:52.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.307+0000 7f3e9934eb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3e881b43f0 con 0x5639f65b2ad0 2024-05-10T11:20:52.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.307+0000 7f3e9934eb80 10 client.0 _ll_register_callbacks cb 0x5639f64925b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:20:52.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.307+0000 7f3e7ffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639f65b21b0 0x5639f65c8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:52.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.307+0000 7f3e8c857700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639f65acd50 0x5639f65c8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:52.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.307+0000 7f3e8d058700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639f65b2ad0 0x5639f662e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:52.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.307+0000 7f3e8c857700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639f65acd50 0x5639f65c8240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:58098/0 (socket says 172.21.15.100:58098) 2024-05-10T11:20:52.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.307+0000 7f3e8c857700 1 -- 172.21.15.100:0/1829822034 learned_addr learned my addr 172.21.15.100:0/1829822034 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:20:52.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.307+0000 7f3e8c857700 1 -- 172.21.15.100:0/1829822034 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639f65b2ad0 msgr2=0x5639f662e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:52.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.307+0000 7f3e8c857700 1 --2- 172.21.15.100:0/1829822034 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639f65b2ad0 0x5639f662e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:52.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.307+0000 7f3e8c857700 1 -- 172.21.15.100:0/1829822034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639f65b21b0 msgr2=0x5639f65c8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:52.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.307+0000 7f3e8c857700 1 --2- 172.21.15.100:0/1829822034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639f65b21b0 0x5639f65c8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:52.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.307+0000 7f3e8c857700 1 -- 172.21.15.100:0/1829822034 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639f662eca0 con 0x5639f65acd50 2024-05-10T11:20:52.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.307+0000 7f3e7ffff700 1 --2- 172.21.15.100:0/1829822034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639f65b21b0 0x5639f65c8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:20:52.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.307+0000 7f3e8c857700 1 --2- 172.21.15.100:0/1829822034 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639f65acd50 0x5639f65c8240 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3e84000f80 tx=0x7f3e840a8a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7f294f2ae5ae5a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:52.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.307+0000 7f3e67fff700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:20:52.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.307+0000 7f3e67fff700 1 -- 172.21.15.100:0/1829822034 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e840e2040 con 0x5639f65acd50 2024-05-10T11:20:52.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.307+0000 7f3e9934eb80 1 -- 172.21.15.100:0/1829822034 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e80005020 con 0x5639f65acd50 2024-05-10T11:20:52.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.308+0000 7f3e67fff700 1 -- 172.21.15.100:0/1829822034 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f3e840ca6a0 con 0x5639f65acd50 2024-05-10T11:20:52.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.308+0000 7f3e67fff700 1 -- 172.21.15.100:0/1829822034 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3e840a8ea0 con 0x5639f65acd50 2024-05-10T11:20:52.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.308+0000 7f3e9934eb80 10 client.25346 fetch_fsmap learned FSMap version 29 2024-05-10T11:20:52.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.308+0000 7f3e9934eb80 1 -- 172.21.15.100:0/1829822034 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5639f662fe80 con 0x5639f65acd50 2024-05-10T11:20:52.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.308+0000 7f3e67fff700 1 -- 172.21.15.100:0/1829822034 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3e80005020 con 0x5639f65acd50 2024-05-10T11:20:52.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.308+0000 7f3e67fff700 1 -- 172.21.15.100:0/1829822034 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f3e840ca2e0 con 0x5639f65acd50 2024-05-10T11:20:52.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.308+0000 7f3e9934eb80 10 client.25346 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T11:20:52.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.308+0000 7f3e9934eb80 10 client.25346 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-10T11:20:52.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.308+0000 7f3e9934eb80 20 client.25346 populate_metadata read hostname 'smithi100' 2024-05-10T11:20:52.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.308+0000 7f3e9934eb80 1 --2- 172.21.15.100:0/1829822034 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x5639f66302e0 0x5639f66326d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:52.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.308+0000 7f3e9934eb80 4 client.25346 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:20:52.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.308+0000 7f3e9934eb80 1 -- 172.21.15.100:0/1829822034 --> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5639f65063c0 con 0x5639f66302e0 2024-05-10T11:20:52.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.308+0000 7f3e7ffff700 1 --2- 172.21.15.100:0/1829822034 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x5639f66302e0 0x5639f66326d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:52.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.309+0000 7f3e67fff700 10 client.25346 ms_handle_connect on v2:172.21.15.188:6836/1943342620 2024-05-10T11:20:52.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.309+0000 7f3e7ffff700 1 --2- 172.21.15.100:0/1829822034 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x5639f66302e0 0x5639f66326d0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=60461a2033a3983f server_cookie=3625bda6f6f6021d in_seq=0 out_seq=0 2024-05-10T11:20:52.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.309+0000 7f3e67fff700 1 -- 172.21.15.100:0/1829822034 <== mds.2 v2:172.21.15.188:6836/1943342620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5639f65063c0 con 0x5639f66302e0 2024-05-10T11:20:52.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.309+0000 7f3e67fff700 10 client.25346 handle_command_reply: tid=0 2024-05-10T11:20:52.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.309+0000 7f3e9934eb80 1 client.25346 shutdown 2024-05-10T11:20:52.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.309+0000 7f3e9934eb80 1 -- 172.21.15.100:0/1829822034 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639f65acd50 msgr2=0x5639f65c8240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:52.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.309+0000 7f3e9934eb80 1 --2- 172.21.15.100:0/1829822034 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639f65acd50 0x5639f65c8240 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3e84000f80 tx=0x7f3e840a8a30 comp rx=0 tx=0).stop 2024-05-10T11:20:52.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.309+0000 7f3e9934eb80 1 -- 172.21.15.100:0/1829822034 shutdown_connections 2024-05-10T11:20:52.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.309+0000 7f3e9934eb80 1 --2- 172.21.15.100:0/1829822034 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x5639f66302e0 0x5639f66326d0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:52.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.309+0000 7f3e9934eb80 1 --2- 172.21.15.100:0/1829822034 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639f65b2ad0 0x5639f662e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:52.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.309+0000 7f3e67fff700 0 client.25346 ms_handle_reset on v2:172.21.15.188:6836/1943342620 2024-05-10T11:20:52.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.309+0000 7f3e9934eb80 1 --2- 172.21.15.100:0/1829822034 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639f65acd50 0x5639f65c8240 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:52.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.309+0000 7f3e9934eb80 1 --2- 172.21.15.100:0/1829822034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639f65b21b0 0x5639f65c8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:52.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.309+0000 7f3e9934eb80 1 -- 172.21.15.100:0/1829822034 >> 172.21.15.100:0/1829822034 conn(0x5639f64d8b70 msgr2=0x5639f65b1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:52.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.309+0000 7f3e9934eb80 1 -- 172.21.15.100:0/1829822034 shutdown_connections 2024-05-10T11:20:52.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.309+0000 7f3e9934eb80 1 -- 172.21.15.100:0/1829822034 wait complete. 2024-05-10T11:20:52.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.309+0000 7f3e9934eb80 20 client.25346 trim_cache size 0 max 16384 2024-05-10T11:20:52.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.310+0000 7f3e8eabb700 1 -- 172.21.15.100:0/753417383 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f3e700b2900 msgr2=0x7f3e700b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:52.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.310+0000 7f3e8eabb700 1 --2- 172.21.15.100:0/753417383 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f3e700b2900 0x7f3e700b4dc0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3e88113f90 tx=0x7f3e7801a040 comp rx=0 tx=0).stop 2024-05-10T11:20:52.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.310+0000 7f3e8eabb700 1 -- 172.21.15.100:0/753417383 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e88075c10 msgr2=0x7f3e881a0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:52.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.310+0000 7f3e8eabb700 1 --2- 172.21.15.100:0/753417383 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e88075c10 0x7f3e881a0120 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3e8400ef10 tx=0x7f3e8400c3a0 comp rx=0 tx=0).stop 2024-05-10T11:20:52.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.310+0000 7f3e8eabb700 1 -- 172.21.15.100:0/753417383 shutdown_connections 2024-05-10T11:20:52.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.310+0000 7f3e8eabb700 1 --2- 172.21.15.100:0/753417383 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f3e700b2900 0x7f3e700b4dc0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:52.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.310+0000 7f3e8eabb700 1 --2- 172.21.15.100:0/753417383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3e88114190 0x7f3e881a49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:52.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.310+0000 7f3e8eabb700 1 --2- 172.21.15.100:0/753417383 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e88075c10 0x7f3e881a0120 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:52.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.310+0000 7f3e8eabb700 1 --2- 172.21.15.100:0/753417383 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e880774c0 0x7f3e881a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:52.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.310+0000 7f3e8eabb700 1 -- 172.21.15.100:0/753417383 >> 172.21.15.100:0/753417383 conn(0x7f3e8807a530 msgr2=0x7f3e88112180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:52.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.310+0000 7f3e8eabb700 1 -- 172.21.15.100:0/753417383 shutdown_connections 2024-05-10T11:20:52.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:52.310+0000 7f3e8eabb700 1 -- 172.21.15.100:0/753417383 wait complete. 2024-05-10T11:20:52.932 INFO:tasks.workunit.client.0.smithi100.stdout: 1 25777 3.31 MB/sec warmup 47 sec latency 62.996 ms 2024-05-10T11:20:53.932 INFO:tasks.workunit.client.0.smithi100.stdout: 1 26262 3.33 MB/sec warmup 48 sec latency 96.456 ms 2024-05-10T11:20:54.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:20:53 smithi178 ceph-mon[33359]: pgmap v495: 129 pgs: 129 active+clean; 29 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 1.3 MiB/s wr, 133 op/s 2024-05-10T11:20:54.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:20:53 smithi188 ceph-mon[34075]: pgmap v495: 129 pgs: 129 active+clean; 29 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 1.3 MiB/s wr, 133 op/s 2024-05-10T11:20:54.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:20:53 smithi100 ceph-mon[30296]: pgmap v495: 129 pgs: 129 active+clean; 29 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 1.3 MiB/s wr, 133 op/s 2024-05-10T11:20:54.933 INFO:tasks.workunit.client.0.smithi100.stdout: 1 26750 3.27 MB/sec warmup 49 sec latency 79.274 ms 2024-05-10T11:20:55.933 INFO:tasks.workunit.client.0.smithi100.stdout: 1 27286 3.24 MB/sec warmup 50 sec latency 103.634 ms 2024-05-10T11:20:56.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:20:55 smithi178 ceph-mon[33359]: pgmap v496: 129 pgs: 129 active+clean; 30 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 1.4 MiB/s wr, 129 op/s 2024-05-10T11:20:56.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:20:55 smithi188 ceph-mon[34075]: pgmap v496: 129 pgs: 129 active+clean; 30 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 1.4 MiB/s wr, 129 op/s 2024-05-10T11:20:56.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:20:55 smithi100 ceph-mon[30296]: pgmap v496: 129 pgs: 129 active+clean; 30 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 1.4 MiB/s wr, 129 op/s 2024-05-10T11:20:56.598 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-10T11:20:56.862 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:20:56.933 INFO:tasks.workunit.client.0.smithi100.stdout: 1 27902 3.28 MB/sec warmup 51 sec latency 30.353 ms 2024-05-10T11:20:57.933 INFO:tasks.workunit.client.0.smithi100.stdout: 1 28435 3.31 MB/sec warmup 52 sec latency 15.300 ms 2024-05-10T11:20:58.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:20:57 smithi100 ceph-mon[30296]: pgmap v497: 129 pgs: 129 active+clean; 30 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 1.5 MiB/s wr, 176 op/s 2024-05-10T11:20:58.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:20:57 smithi178 ceph-mon[33359]: pgmap v497: 129 pgs: 129 active+clean; 30 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 1.5 MiB/s wr, 176 op/s 2024-05-10T11:20:58.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:20:57 smithi188 ceph-mon[34075]: pgmap v497: 129 pgs: 129 active+clean; 30 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 1.5 MiB/s wr, 176 op/s 2024-05-10T11:20:58.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.864+0000 7f64d310b700 1 -- 172.21.15.100:0/553677442 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64cc06ba40 msgr2=0x7f64cc06be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:58.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.864+0000 7f64d310b700 1 --2- 172.21.15.100:0/553677442 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64cc06ba40 0x7f64cc06be20 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f64c0009990 tx=0x7f64c0009ca0 comp rx=0 tx=0).stop 2024-05-10T11:20:58.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.864+0000 7f64d310b700 1 -- 172.21.15.100:0/553677442 shutdown_connections 2024-05-10T11:20:58.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.864+0000 7f64d310b700 1 --2- 172.21.15.100:0/553677442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64cc10ff40 0x7f64cc112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:58.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.864+0000 7f64d310b700 1 --2- 172.21.15.100:0/553677442 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64cc06ba40 0x7f64cc06be20 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:58.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.864+0000 7f64d310b700 1 --2- 172.21.15.100:0/553677442 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64cc06c360 0x7f64cc10fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:58.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.864+0000 7f64d310b700 1 -- 172.21.15.100:0/553677442 >> 172.21.15.100:0/553677442 conn(0x7f64cc079310 msgr2=0x7f64cc079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:58.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.864+0000 7f64d310b700 1 -- 172.21.15.100:0/553677442 shutdown_connections 2024-05-10T11:20:58.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.864+0000 7f64d310b700 1 -- 172.21.15.100:0/553677442 wait complete. 2024-05-10T11:20:58.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.865+0000 7f64d310b700 1 Processor -- start 2024-05-10T11:20:58.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.865+0000 7f64d310b700 1 -- start start 2024-05-10T11:20:58.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.865+0000 7f64d310b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64cc06ba40 0x7f64cc1a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:58.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.865+0000 7f64d310b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64cc06c360 0x7f64cc1a1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:58.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.865+0000 7f64d310b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64cc10ff40 0x7f64cc19baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:58.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.865+0000 7f64d310b700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f64cc115700 con 0x7f64cc10ff40 2024-05-10T11:20:58.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.865+0000 7f64d310b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f64cc115580 con 0x7f64cc06ba40 2024-05-10T11:20:58.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.865+0000 7f64d310b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f64cc115880 con 0x7f64cc06c360 2024-05-10T11:20:58.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.865+0000 7f64cbfff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64cc06c360 0x7f64cc1a1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:58.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.865+0000 7f64d0ea7700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64cc06ba40 0x7f64cc1a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:58.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.865+0000 7f64cbfff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64cc06c360 0x7f64cc1a1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:44752/0 (socket says 172.21.15.100:44752) 2024-05-10T11:20:58.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.866+0000 7f64cbfff700 1 -- 172.21.15.100:0/2722216553 learned_addr learned my addr 172.21.15.100:0/2722216553 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:20:58.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.866+0000 7f64d16a8700 1 --2- 172.21.15.100:0/2722216553 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64cc10ff40 0x7f64cc19baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:58.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.866+0000 7f64cbfff700 1 -- 172.21.15.100:0/2722216553 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64cc06ba40 msgr2=0x7f64cc1a18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:58.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.866+0000 7f64cbfff700 1 --2- 172.21.15.100:0/2722216553 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64cc06ba40 0x7f64cc1a18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:58.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.866+0000 7f64cbfff700 1 -- 172.21.15.100:0/2722216553 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64cc10ff40 msgr2=0x7f64cc19baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:58.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.866+0000 7f64cbfff700 1 --2- 172.21.15.100:0/2722216553 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64cc10ff40 0x7f64cc19baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:58.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.866+0000 7f64cbfff700 1 -- 172.21.15.100:0/2722216553 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64cc19c200 con 0x7f64cc06c360 2024-05-10T11:20:58.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.866+0000 7f64cbfff700 1 --2- 172.21.15.100:0/2722216553 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64cc06c360 0x7f64cc1a1e20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f64b800d980 tx=0x7f64b800dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:58.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.866+0000 7f64c9ffb700 1 -- 172.21.15.100:0/2722216553 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b8016070 con 0x7f64cc06c360 2024-05-10T11:20:58.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.866+0000 7f64c9ffb700 1 -- 172.21.15.100:0/2722216553 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f64b8004830 con 0x7f64cc06c360 2024-05-10T11:20:58.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.866+0000 7f64c9ffb700 1 -- 172.21.15.100:0/2722216553 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b80054c0 con 0x7f64cc06c360 2024-05-10T11:20:58.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.866+0000 7f64d310b700 1 -- 172.21.15.100:0/2722216553 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64cc19c4f0 con 0x7f64cc06c360 2024-05-10T11:20:58.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.867+0000 7f64d310b700 1 -- 172.21.15.100:0/2722216553 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64cc1a87b0 con 0x7f64cc06c360 2024-05-10T11:20:58.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.869+0000 7f64c9ffb700 1 -- 172.21.15.100:0/2722216553 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f64b8004220 con 0x7f64cc06c360 2024-05-10T11:20:58.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.869+0000 7f64c9ffb700 1 --2- 172.21.15.100:0/2722216553 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f64bc0b2880 0x7f64bc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:58.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.870+0000 7f64d0ea7700 1 --2- 172.21.15.100:0/2722216553 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f64bc0b2880 0x7f64bc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:58.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.870+0000 7f64c9ffb700 1 -- 172.21.15.100:0/2722216553 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f64b80e22f0 con 0x7f64cc06c360 2024-05-10T11:20:58.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.870+0000 7f64d0ea7700 1 --2- 172.21.15.100:0/2722216553 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f64bc0b2880 0x7f64bc0b4d40 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f64c000b2e0 tx=0x7f64c001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:58.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.871+0000 7f64dd99eb80 1 Processor -- start 2024-05-10T11:20:58.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.871+0000 7f64dd99eb80 1 -- start start 2024-05-10T11:20:58.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.871+0000 7f64dd99eb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc850a21b0 0x55dc850a2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:58.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.871+0000 7f64dd99eb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc850a2ad0 0x55dc8509c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:58.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.871+0000 7f64dd99eb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc8509cd50 0x55dc8509d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:58.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.871+0000 7f64dd99eb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55dc84fee940 con 0x55dc850a2ad0 2024-05-10T11:20:58.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.871+0000 7f64dd99eb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55dc84f53da0 con 0x55dc8509cd50 2024-05-10T11:20:58.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.871+0000 7f64dd99eb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55dc84fea900 con 0x55dc850a21b0 2024-05-10T11:20:58.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.871+0000 7f64cbfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc850a2ad0 0x55dc8509c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:58.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.871+0000 7f64d16a8700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc8509cd50 0x55dc8509d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:58.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.871+0000 7f64cbfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc850a2ad0 0x55dc8509c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51378/0 (socket says 172.21.15.100:51378) 2024-05-10T11:20:58.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.871+0000 7f64d16a8700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc8509cd50 0x55dc8509d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:36302/0 (socket says 172.21.15.100:36302) 2024-05-10T11:20:58.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.871+0000 7f64cbfff700 1 -- 172.21.15.100:0/792654087 learned_addr learned my addr 172.21.15.100:0/792654087 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:20:58.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.871+0000 7f64d0ea7700 1 --2- 172.21.15.100:0/792654087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc850a21b0 0x55dc850a2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:58.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.871+0000 7f64cbfff700 1 -- 172.21.15.100:0/792654087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc850a21b0 msgr2=0x55dc850a2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:58.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.871+0000 7f64cbfff700 1 --2- 172.21.15.100:0/792654087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc850a21b0 0x55dc850a2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:58.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.871+0000 7f64cbfff700 1 -- 172.21.15.100:0/792654087 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc8509cd50 msgr2=0x55dc8509d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:58.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.871+0000 7f64cbfff700 1 --2- 172.21.15.100:0/792654087 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc8509cd50 0x55dc8509d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:58.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.871+0000 7f64cbfff700 1 -- 172.21.15.100:0/792654087 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc84f3b8d0 con 0x55dc850a2ad0 2024-05-10T11:20:58.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.872+0000 7f64cbfff700 1 --2- 172.21.15.100:0/792654087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc850a2ad0 0x55dc8509c810 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f64b80a67c0 tx=0x7f64b80a9130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1adbc0e437791bb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:58.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.872+0000 7f64b6ffd700 1 -- 172.21.15.100:0/792654087 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b80a9550 con 0x55dc850a2ad0 2024-05-10T11:20:58.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.872+0000 7f64b6ffd700 1 -- 172.21.15.100:0/792654087 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f64b80ad900 con 0x55dc850a2ad0 2024-05-10T11:20:58.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.872+0000 7f64b6ffd700 1 -- 172.21.15.100:0/792654087 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b80a97b0 con 0x55dc850a2ad0 2024-05-10T11:20:58.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.873+0000 7f64dd99eb80 1 -- 172.21.15.100:0/792654087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc850a2ad0 msgr2=0x55dc8509c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:58.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.873+0000 7f64dd99eb80 1 --2- 172.21.15.100:0/792654087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc850a2ad0 0x55dc8509c810 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f64b80a67c0 tx=0x7f64b80a9130 comp rx=0 tx=0).stop 2024-05-10T11:20:58.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.873+0000 7f64dd99eb80 1 -- 172.21.15.100:0/792654087 shutdown_connections 2024-05-10T11:20:58.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.873+0000 7f64dd99eb80 1 --2- 172.21.15.100:0/792654087 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc8509cd50 0x55dc8509d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:58.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.873+0000 7f64dd99eb80 1 --2- 172.21.15.100:0/792654087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc850a21b0 0x55dc850a2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:58.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.873+0000 7f64dd99eb80 1 --2- 172.21.15.100:0/792654087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc850a2ad0 0x55dc8509c810 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:58.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.873+0000 7f64dd99eb80 1 -- 172.21.15.100:0/792654087 >> 172.21.15.100:0/792654087 conn(0x55dc84fc8b70 msgr2=0x55dc850972a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:58.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.873+0000 7f64dd99eb80 1 -- 172.21.15.100:0/792654087 shutdown_connections 2024-05-10T11:20:58.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.873+0000 7f64dd99eb80 1 -- 172.21.15.100:0/792654087 wait complete. 2024-05-10T11:20:58.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.873+0000 7f64dd99eb80 1 -- start start 2024-05-10T11:20:58.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.873+0000 7f64dd99eb80 1 Processor -- start 2024-05-10T11:20:58.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.874+0000 7f64dd99eb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc8509cd50 0x55dc850a87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:58.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.874+0000 7f64dd99eb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc850a21b0 0x55dc850a5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:58.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.874+0000 7f64dd99eb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc850a2ad0 0x55dc850a6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:58.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.874+0000 7f64dd99eb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55dc850a9450 con 0x55dc8509cd50 2024-05-10T11:20:58.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.874+0000 7f64dd99eb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55dc84fa7ed0 con 0x55dc850a2ad0 2024-05-10T11:20:58.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.874+0000 7f64dd99eb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55dc850a95d0 con 0x55dc850a21b0 2024-05-10T11:20:58.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.874+0000 7f64dd99eb80 10 client.0 _ll_register_callbacks cb 0x55dc84f825b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:20:58.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.874+0000 7f64d0ea7700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc8509cd50 0x55dc850a87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:58.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.874+0000 7f64cbfff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc850a21b0 0x55dc850a5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:58.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.874+0000 7f64d16a8700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc850a2ad0 0x55dc850a6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:58.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.874+0000 7f64d0ea7700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc8509cd50 0x55dc850a87a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51380/0 (socket says 172.21.15.100:51380) 2024-05-10T11:20:58.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.874+0000 7f64cbfff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc850a21b0 0x55dc850a5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:44772/0 (socket says 172.21.15.100:44772) 2024-05-10T11:20:58.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.874+0000 7f64d0ea7700 1 -- 172.21.15.100:0/349925322 learned_addr learned my addr 172.21.15.100:0/349925322 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:20:58.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.874+0000 7f64cbfff700 1 -- 172.21.15.100:0/349925322 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc850a2ad0 msgr2=0x55dc850a6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:58.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.874+0000 7f64cbfff700 1 --2- 172.21.15.100:0/349925322 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc850a2ad0 0x55dc850a6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:58.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.874+0000 7f64cbfff700 1 -- 172.21.15.100:0/349925322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc8509cd50 msgr2=0x55dc850a87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:58.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.874+0000 7f64cbfff700 1 --2- 172.21.15.100:0/349925322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc8509cd50 0x55dc850a87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:58.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.874+0000 7f64cbfff700 1 -- 172.21.15.100:0/349925322 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc84fa5760 con 0x55dc850a21b0 2024-05-10T11:20:58.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.874+0000 7f64d16a8700 1 --2- 172.21.15.100:0/349925322 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc850a2ad0 0x55dc850a6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:20:58.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.874+0000 7f64cbfff700 1 --2- 172.21.15.100:0/349925322 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc850a21b0 0x55dc850a5b40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x55dc850a2eb0 tx=0x7f64b8004ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12e7159736d91956 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:58.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.874+0000 7f64abfff700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:20:58.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.875+0000 7f64abfff700 1 -- 172.21.15.100:0/349925322 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b80c2020 con 0x55dc850a21b0 2024-05-10T11:20:58.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.875+0000 7f64dd99eb80 1 -- 172.21.15.100:0/349925322 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64b80ad900 con 0x55dc850a21b0 2024-05-10T11:20:58.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.875+0000 7f64abfff700 1 -- 172.21.15.100:0/349925322 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f64b80049e0 con 0x55dc850a21b0 2024-05-10T11:20:58.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.875+0000 7f64abfff700 1 -- 172.21.15.100:0/349925322 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f64b8005660 con 0x55dc850a21b0 2024-05-10T11:20:58.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.875+0000 7f64dd99eb80 10 client.25354 fetch_fsmap learned FSMap version 29 2024-05-10T11:20:58.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.875+0000 7f64dd99eb80 1 -- 172.21.15.100:0/349925322 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55dc850a94a0 con 0x55dc850a21b0 2024-05-10T11:20:58.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.875+0000 7f64abfff700 1 -- 172.21.15.100:0/349925322 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f64b80ad900 con 0x55dc850a21b0 2024-05-10T11:20:58.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.875+0000 7f64abfff700 1 -- 172.21.15.100:0/349925322 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f64b80b1f20 con 0x55dc850a21b0 2024-05-10T11:20:58.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.875+0000 7f64dd99eb80 10 client.25354 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T11:20:58.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.875+0000 7f64dd99eb80 10 client.25354 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-10T11:20:58.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.875+0000 7f64dd99eb80 20 client.25354 populate_metadata read hostname 'smithi100' 2024-05-10T11:20:58.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.875+0000 7f64dd99eb80 1 --2- 172.21.15.100:0/349925322 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55dc850b6060 0x55dc850b8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:58.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.875+0000 7f64dd99eb80 4 client.25354 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:20:58.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.875+0000 7f64dd99eb80 1 -- 172.21.15.100:0/349925322 --> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc84ffc320 con 0x55dc850b6060 2024-05-10T11:20:58.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.876+0000 7f64d0ea7700 1 --2- 172.21.15.100:0/349925322 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55dc850b6060 0x55dc850b8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:58.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.876+0000 7f64abfff700 10 client.25354 ms_handle_connect on v2:172.21.15.178:6838/3760985404 2024-05-10T11:20:58.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.876+0000 7f64d0ea7700 1 --2- 172.21.15.100:0/349925322 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55dc850b6060 0x55dc850b8450 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9033843e8b929506 server_cookie=abeb8d9abb8fbbb2 in_seq=0 out_seq=0 2024-05-10T11:20:58.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.877+0000 7f64abfff700 1 -- 172.21.15.100:0/349925322 <== mds.1 v2:172.21.15.178:6838/3760985404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dc84ffc320 con 0x55dc850b6060 2024-05-10T11:20:58.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.877+0000 7f64abfff700 10 client.25354 handle_command_reply: tid=0 2024-05-10T11:20:58.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.877+0000 7f64dd99eb80 1 client.25354 shutdown 2024-05-10T11:20:58.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.877+0000 7f64dd99eb80 1 -- 172.21.15.100:0/349925322 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc850a21b0 msgr2=0x55dc850a5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:58.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.877+0000 7f64dd99eb80 1 --2- 172.21.15.100:0/349925322 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc850a21b0 0x55dc850a5b40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x55dc850a2eb0 tx=0x7f64b8004ef0 comp rx=0 tx=0).stop 2024-05-10T11:20:58.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.877+0000 7f64dd99eb80 1 -- 172.21.15.100:0/349925322 shutdown_connections 2024-05-10T11:20:58.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.877+0000 7f64dd99eb80 1 --2- 172.21.15.100:0/349925322 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55dc850b6060 0x55dc850b8450 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:58.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.877+0000 7f64dd99eb80 1 --2- 172.21.15.100:0/349925322 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc850a2ad0 0x55dc850a6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:58.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.877+0000 7f64dd99eb80 1 --2- 172.21.15.100:0/349925322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc8509cd50 0x55dc850a87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:58.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.877+0000 7f64abfff700 0 client.25354 ms_handle_reset on v2:172.21.15.178:6838/3760985404 2024-05-10T11:20:58.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.877+0000 7f64dd99eb80 1 --2- 172.21.15.100:0/349925322 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc850a21b0 0x55dc850a5b40 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:58.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.877+0000 7f64dd99eb80 1 -- 172.21.15.100:0/349925322 >> 172.21.15.100:0/349925322 conn(0x55dc84fc8b70 msgr2=0x55dc85091b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:58.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.877+0000 7f64dd99eb80 1 -- 172.21.15.100:0/349925322 shutdown_connections 2024-05-10T11:20:58.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.877+0000 7f64dd99eb80 1 -- 172.21.15.100:0/349925322 wait complete. 2024-05-10T11:20:58.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.877+0000 7f64dd99eb80 20 client.25354 trim_cache size 0 max 16384 2024-05-10T11:20:58.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.893+0000 7f64dd99eb80 1 Processor -- start 2024-05-10T11:20:58.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.893+0000 7f64dd99eb80 1 -- start start 2024-05-10T11:20:58.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.893+0000 7f64dd99eb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc8509cd50 0x55dc8509c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:58.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.893+0000 7f64dd99eb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc850a21b0 0x55dc850b24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:58.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.893+0000 7f64dd99eb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc850a2ad0 0x55dc85091c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:58.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.893+0000 7f64dd99eb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f64cc1b3c10 con 0x55dc850a2ad0 2024-05-10T11:20:58.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.893+0000 7f64dd99eb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f64cc1b14a0 con 0x55dc8509cd50 2024-05-10T11:20:58.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.893+0000 7f64dd99eb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f64cc1af2f0 con 0x55dc850a21b0 2024-05-10T11:20:58.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.893+0000 7f64d16a8700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc850a2ad0 0x55dc85091c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:58.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.893+0000 7f64d16a8700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc850a2ad0 0x55dc85091c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51388/0 (socket says 172.21.15.100:51388) 2024-05-10T11:20:58.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.893+0000 7f64d0ea7700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc8509cd50 0x55dc8509c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:58.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.893+0000 7f64d16a8700 1 -- 172.21.15.100:0/349610564 learned_addr learned my addr 172.21.15.100:0/349610564 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:20:58.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.893+0000 7f64cbfff700 1 --2- 172.21.15.100:0/349610564 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc850a21b0 0x55dc850b24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:58.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.894+0000 7f64d16a8700 1 -- 172.21.15.100:0/349610564 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc850a21b0 msgr2=0x55dc850b24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:58.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.894+0000 7f64d16a8700 1 --2- 172.21.15.100:0/349610564 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc850a21b0 0x55dc850b24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:58.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.894+0000 7f64d16a8700 1 -- 172.21.15.100:0/349610564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc8509cd50 msgr2=0x55dc8509c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:58.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.894+0000 7f64d16a8700 1 --2- 172.21.15.100:0/349610564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc8509cd50 0x55dc8509c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:58.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.894+0000 7f64d16a8700 1 -- 172.21.15.100:0/349610564 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc850a6a80 con 0x55dc850a2ad0 2024-05-10T11:20:58.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.894+0000 7f64d0ea7700 1 --2- 172.21.15.100:0/349610564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc8509cd50 0x55dc8509c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:20:58.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.894+0000 7f64d16a8700 1 --2- 172.21.15.100:0/349610564 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc850a2ad0 0x55dc85091c40 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f64c4012ba0 tx=0x7f64c4009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1168b3af9c3712bd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:58.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.894+0000 7f64b67fc700 1 -- 172.21.15.100:0/349610564 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c40179a0 con 0x55dc850a2ad0 2024-05-10T11:20:58.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.894+0000 7f64b67fc700 1 -- 172.21.15.100:0/349610564 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f64c40093a0 con 0x55dc850a2ad0 2024-05-10T11:20:58.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.894+0000 7f64b67fc700 1 -- 172.21.15.100:0/349610564 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c401f3e0 con 0x55dc850a2ad0 2024-05-10T11:20:58.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.895+0000 7f64dd99eb80 1 -- 172.21.15.100:0/349610564 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc850a2ad0 msgr2=0x55dc85091c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:58.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.895+0000 7f64dd99eb80 1 --2- 172.21.15.100:0/349610564 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc850a2ad0 0x55dc85091c40 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f64c4012ba0 tx=0x7f64c4009920 comp rx=0 tx=0).stop 2024-05-10T11:20:58.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.895+0000 7f64dd99eb80 1 -- 172.21.15.100:0/349610564 shutdown_connections 2024-05-10T11:20:58.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.895+0000 7f64dd99eb80 1 --2- 172.21.15.100:0/349610564 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc850a2ad0 0x55dc85091c40 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:58.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.895+0000 7f64dd99eb80 1 --2- 172.21.15.100:0/349610564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc8509cd50 0x55dc8509c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:58.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.895+0000 7f64dd99eb80 1 --2- 172.21.15.100:0/349610564 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc850a21b0 0x55dc850b24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:58.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.895+0000 7f64dd99eb80 1 -- 172.21.15.100:0/349610564 >> 172.21.15.100:0/349610564 conn(0x55dc84fc8b70 msgr2=0x55dc850a4430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:58.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.895+0000 7f64dd99eb80 1 -- 172.21.15.100:0/349610564 shutdown_connections 2024-05-10T11:20:58.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.895+0000 7f64dd99eb80 1 -- 172.21.15.100:0/349610564 wait complete. 2024-05-10T11:20:58.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.895+0000 7f64dd99eb80 1 -- start start 2024-05-10T11:20:58.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.895+0000 7f64dd99eb80 1 Processor -- start 2024-05-10T11:20:58.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.896+0000 7f64dd99eb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc8509cd50 0x55dc850a9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:58.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.896+0000 7f64dd99eb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc850a21b0 0x55dc850aa000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:58.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.896+0000 7f64dd99eb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc850a2ad0 0x55dc8511e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:58.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.896+0000 7f64dd99eb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f64cc1af960 con 0x55dc8509cd50 2024-05-10T11:20:58.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.896+0000 7f64dd99eb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f64cc1b0dd0 con 0x55dc850a21b0 2024-05-10T11:20:58.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.896+0000 7f64dd99eb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f64cc1b0030 con 0x55dc850a2ad0 2024-05-10T11:20:58.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.896+0000 7f64dd99eb80 10 client.0 _ll_register_callbacks cb 0x55dc84f825b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:20:58.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.896+0000 7f64d0ea7700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc8509cd50 0x55dc850a9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:58.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.896+0000 7f64d0ea7700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc8509cd50 0x55dc850a9ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51396/0 (socket says 172.21.15.100:51396) 2024-05-10T11:20:58.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.896+0000 7f64d0ea7700 1 -- 172.21.15.100:0/2883633828 learned_addr learned my addr 172.21.15.100:0/2883633828 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:20:58.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.896+0000 7f64d16a8700 1 --2- 172.21.15.100:0/2883633828 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc850a2ad0 0x55dc8511e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:58.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.896+0000 7f64cbfff700 1 --2- 172.21.15.100:0/2883633828 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc850a21b0 0x55dc850aa000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:58.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.896+0000 7f64d0ea7700 1 -- 172.21.15.100:0/2883633828 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc850a2ad0 msgr2=0x55dc8511e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:58.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.896+0000 7f64d0ea7700 1 --2- 172.21.15.100:0/2883633828 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc850a2ad0 0x55dc8511e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:58.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.896+0000 7f64d0ea7700 1 -- 172.21.15.100:0/2883633828 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc850a21b0 msgr2=0x55dc850aa000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:20:58.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.896+0000 7f64d0ea7700 1 --2- 172.21.15.100:0/2883633828 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc850a21b0 0x55dc850aa000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:58.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.896+0000 7f64d0ea7700 1 -- 172.21.15.100:0/2883633828 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc8511eb60 con 0x55dc8509cd50 2024-05-10T11:20:58.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.896+0000 7f64d16a8700 1 --2- 172.21.15.100:0/2883633828 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc850a2ad0 0x55dc8511e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:20:58.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.896+0000 7f64d0ea7700 1 --2- 172.21.15.100:0/2883633828 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc8509cd50 0x55dc850a9ac0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f64c0000c00 tx=0x7f64c0009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac16e48f75963db1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:20:58.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.896+0000 7f64abfff700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:20:58.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.897+0000 7f64abfff700 1 -- 172.21.15.100:0/2883633828 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c0003880 con 0x55dc8509cd50 2024-05-10T11:20:58.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.897+0000 7f64abfff700 1 -- 172.21.15.100:0/2883633828 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f64c0003a20 con 0x55dc8509cd50 2024-05-10T11:20:58.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.897+0000 7f64dd99eb80 1 -- 172.21.15.100:0/2883633828 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64c40093a0 con 0x55dc8509cd50 2024-05-10T11:20:58.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.897+0000 7f64abfff700 1 -- 172.21.15.100:0/2883633828 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f64c0045050 con 0x55dc8509cd50 2024-05-10T11:20:58.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.897+0000 7f64dd99eb80 10 client.15858 fetch_fsmap learned FSMap version 29 2024-05-10T11:20:58.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.897+0000 7f64dd99eb80 1 -- 172.21.15.100:0/2883633828 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55dc8511fe10 con 0x55dc8509cd50 2024-05-10T11:20:58.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.897+0000 7f64abfff700 1 -- 172.21.15.100:0/2883633828 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f64c40093a0 con 0x55dc8509cd50 2024-05-10T11:20:58.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.897+0000 7f64abfff700 1 -- 172.21.15.100:0/2883633828 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f64c00360b0 con 0x55dc8509cd50 2024-05-10T11:20:58.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.897+0000 7f64dd99eb80 10 client.15858 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T11:20:58.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.897+0000 7f64dd99eb80 10 client.15858 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-10T11:20:58.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.897+0000 7f64dd99eb80 20 client.15858 populate_metadata read hostname 'smithi100' 2024-05-10T11:20:58.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.897+0000 7f64dd99eb80 1 --2- 172.21.15.100:0/2883633828 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55dc851204f0 0x55dc85122930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:20:58.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.897+0000 7f64dd99eb80 4 client.15858 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:20:58.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.897+0000 7f64dd99eb80 1 -- 172.21.15.100:0/2883633828 --> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dc84ff63c0 con 0x55dc851204f0 2024-05-10T11:20:58.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.898+0000 7f64cbfff700 1 --2- 172.21.15.100:0/2883633828 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55dc851204f0 0x55dc85122930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:20:58.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.898+0000 7f64abfff700 10 client.15858 ms_handle_connect on v2:172.21.15.178:6838/3760985404 2024-05-10T11:20:58.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.898+0000 7f64cbfff700 1 --2- 172.21.15.100:0/2883633828 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55dc851204f0 0x55dc85122930 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a856debf9d1b1bb8 server_cookie=6c06667b804c8940 in_seq=0 out_seq=0 2024-05-10T11:20:58.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.898+0000 7f64abfff700 1 -- 172.21.15.100:0/2883633828 <== mds.1 v2:172.21.15.178:6838/3760985404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dc84ff63c0 con 0x55dc851204f0 2024-05-10T11:20:58.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.898+0000 7f64abfff700 10 client.15858 handle_command_reply: tid=0 2024-05-10T11:20:58.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.898+0000 7f64dd99eb80 1 client.15858 shutdown 2024-05-10T11:20:58.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.898+0000 7f64dd99eb80 1 -- 172.21.15.100:0/2883633828 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc8509cd50 msgr2=0x55dc850a9ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:58.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.898+0000 7f64dd99eb80 1 --2- 172.21.15.100:0/2883633828 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc8509cd50 0x55dc850a9ac0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f64c0000c00 tx=0x7f64c0009310 comp rx=0 tx=0).stop 2024-05-10T11:20:58.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.898+0000 7f64dd99eb80 1 -- 172.21.15.100:0/2883633828 shutdown_connections 2024-05-10T11:20:58.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.898+0000 7f64dd99eb80 1 --2- 172.21.15.100:0/2883633828 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55dc851204f0 0x55dc85122930 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:58.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.898+0000 7f64abfff700 0 client.15858 ms_handle_reset on v2:172.21.15.178:6838/3760985404 2024-05-10T11:20:58.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.898+0000 7f64dd99eb80 1 --2- 172.21.15.100:0/2883633828 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc850a2ad0 0x55dc8511e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:58.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.898+0000 7f64dd99eb80 1 --2- 172.21.15.100:0/2883633828 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc8509cd50 0x55dc850a9ac0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:58.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.898+0000 7f64dd99eb80 1 --2- 172.21.15.100:0/2883633828 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc850a21b0 0x55dc850aa000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:58.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.898+0000 7f64dd99eb80 1 -- 172.21.15.100:0/2883633828 >> 172.21.15.100:0/2883633828 conn(0x55dc84fc8b70 msgr2=0x55dc850a1530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:58.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.898+0000 7f64dd99eb80 1 -- 172.21.15.100:0/2883633828 shutdown_connections 2024-05-10T11:20:58.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.899+0000 7f64dd99eb80 1 -- 172.21.15.100:0/2883633828 wait complete. 2024-05-10T11:20:58.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.899+0000 7f64dd99eb80 20 client.15858 trim_cache size 0 max 16384 2024-05-10T11:20:58.907 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:20:58.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.899+0000 7f64d310b700 1 -- 172.21.15.100:0/2722216553 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f64bc0b2880 msgr2=0x7f64bc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:58.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.899+0000 7f64d310b700 1 --2- 172.21.15.100:0/2722216553 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f64bc0b2880 0x7f64bc0b4d40 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f64c000b2e0 tx=0x7f64c001a040 comp rx=0 tx=0).stop 2024-05-10T11:20:58.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.899+0000 7f64d310b700 1 -- 172.21.15.100:0/2722216553 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64cc06c360 msgr2=0x7f64cc1a1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:20:58.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.899+0000 7f64d310b700 1 --2- 172.21.15.100:0/2722216553 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64cc06c360 0x7f64cc1a1e20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f64b800d980 tx=0x7f64b800dc90 comp rx=0 tx=0).stop 2024-05-10T11:20:58.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.899+0000 7f64d310b700 1 -- 172.21.15.100:0/2722216553 shutdown_connections 2024-05-10T11:20:58.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.899+0000 7f64d310b700 1 --2- 172.21.15.100:0/2722216553 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f64bc0b2880 0x7f64bc0b4d40 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:58.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.899+0000 7f64d310b700 1 --2- 172.21.15.100:0/2722216553 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64cc10ff40 0x7f64cc19baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:58.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.899+0000 7f64d310b700 1 --2- 172.21.15.100:0/2722216553 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64cc06ba40 0x7f64cc1a18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:58.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.899+0000 7f64d310b700 1 --2- 172.21.15.100:0/2722216553 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64cc06c360 0x7f64cc1a1e20 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:20:58.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.899+0000 7f64d310b700 1 -- 172.21.15.100:0/2722216553 >> 172.21.15.100:0/2722216553 conn(0x7f64cc079310 msgr2=0x7f64cc111490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:20:58.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.899+0000 7f64d310b700 1 -- 172.21.15.100:0/2722216553 shutdown_connections 2024-05-10T11:20:58.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:20:58.899+0000 7f64d310b700 1 -- 172.21.15.100:0/2722216553 wait complete. 2024-05-10T11:20:58.933 INFO:tasks.workunit.client.0.smithi100.stdout: 1 28801 3.28 MB/sec warmup 53 sec latency 15.510 ms 2024-05-10T11:20:59.769 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-10T11:20:59.933 INFO:tasks.workunit.client.0.smithi100.stdout: 1 29140 3.23 MB/sec warmup 54 sec latency 13.848 ms 2024-05-10T11:21:00.034 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:21:00.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:20:59 smithi100 ceph-mon[30296]: pgmap v498: 129 pgs: 129 active+clean; 30 MiB data, 527 MiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 1.3 MiB/s wr, 134 op/s 2024-05-10T11:21:00.229 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.227+0000 7f953f1f7640 1 -- 172.21.15.100:0/2130041743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9538103530 msgr2=0x7f9538105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:00.229 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.227+0000 7f953f1f7640 1 --2- 172.21.15.100:0/2130041743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9538103530 0x7f9538105920 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f9534009900 tx=0x7f953402a410 comp rx=0 tx=0).stop 2024-05-10T11:21:00.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.227+0000 7f953f1f7640 1 -- 172.21.15.100:0/2130041743 shutdown_connections 2024-05-10T11:21:00.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.227+0000 7f953f1f7640 1 --2- 172.21.15.100:0/2130041743 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9538108790 0x7f953810ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:00.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.227+0000 7f953f1f7640 1 --2- 172.21.15.100:0/2130041743 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9538105e60 0x7f9538108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:00.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.227+0000 7f953f1f7640 1 --2- 172.21.15.100:0/2130041743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9538103530 0x7f9538105920 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:00.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.227+0000 7f953f1f7640 1 -- 172.21.15.100:0/2130041743 >> 172.21.15.100:0/2130041743 conn(0x7f95380fce80 msgr2=0x7f95380ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:00.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.227+0000 7f953f1f7640 1 -- 172.21.15.100:0/2130041743 shutdown_connections 2024-05-10T11:21:00.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.227+0000 7f953f1f7640 1 -- 172.21.15.100:0/2130041743 wait complete. 2024-05-10T11:21:00.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.228+0000 7f953f1f7640 1 Processor -- start 2024-05-10T11:21:00.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.228+0000 7f953f1f7640 1 -- start start 2024-05-10T11:21:00.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.228+0000 7f953f1f7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9538103530 0x7f9538102ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:00.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.228+0000 7f953f1f7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9538105e60 0x7f95381014e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:00.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.228+0000 7f953f1f7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9538108790 0x7f9538101a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:00.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.228+0000 7f953f1f7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f953810e450 con 0x7f9538105e60 2024-05-10T11:21:00.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.228+0000 7f953f1f7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f953810e2d0 con 0x7f9538103530 2024-05-10T11:21:00.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.228+0000 7f953f1f7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f953810e5d0 con 0x7f9538108790 2024-05-10T11:21:00.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.228+0000 7f952ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9538105e60 0x7f95381014e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:00.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.228+0000 7f952ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9538105e60 0x7f95381014e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51402/0 (socket says 172.21.15.100:51402) 2024-05-10T11:21:00.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.228+0000 7f952ffff640 1 -- 172.21.15.100:0/2427240907 learned_addr learned my addr 172.21.15.100:0/2427240907 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:21:00.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.228+0000 7f953d794640 1 --2- 172.21.15.100:0/2427240907 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9538108790 0x7f9538101a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:00.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.228+0000 7f953cf93640 1 --2- 172.21.15.100:0/2427240907 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9538103530 0x7f9538102ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:00.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.228+0000 7f952ffff640 1 -- 172.21.15.100:0/2427240907 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9538108790 msgr2=0x7f9538101a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:00.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.228+0000 7f952ffff640 1 --2- 172.21.15.100:0/2427240907 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9538108790 0x7f9538101a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:00.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.229+0000 7f952ffff640 1 -- 172.21.15.100:0/2427240907 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9538103530 msgr2=0x7f9538102ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:00.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.229+0000 7f952ffff640 1 --2- 172.21.15.100:0/2427240907 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9538103530 0x7f9538102ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:00.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.229+0000 7f952ffff640 1 -- 172.21.15.100:0/2427240907 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95381022b0 con 0x7f9538105e60 2024-05-10T11:21:00.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.229+0000 7f953cf93640 1 --2- 172.21.15.100:0/2427240907 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9538103530 0x7f9538102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:21:00.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.229+0000 7f952ffff640 1 --2- 172.21.15.100:0/2427240907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9538105e60 0x7f95381014e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f952800d770 tx=0x7f952800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:00.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.229+0000 7f952dffb640 1 -- 172.21.15.100:0/2427240907 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9528017070 con 0x7f9538105e60 2024-05-10T11:21:00.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.229+0000 7f953f1f7640 1 -- 172.21.15.100:0/2427240907 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95381a2c20 con 0x7f9538105e60 2024-05-10T11:21:00.232 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.229+0000 7f952dffb640 1 -- 172.21.15.100:0/2427240907 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9528009c30 con 0x7f9538105e60 2024-05-10T11:21:00.232 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.229+0000 7f952dffb640 1 -- 172.21.15.100:0/2427240907 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952800f620 con 0x7f9538105e60 2024-05-10T11:21:00.232 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.229+0000 7f953f1f7640 1 -- 172.21.15.100:0/2427240907 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95381a3170 con 0x7f9538105e60 2024-05-10T11:21:00.238 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.232+0000 7f9541feb740 1 Processor -- start 2024-05-10T11:21:00.238 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.232+0000 7f952dffb640 1 -- 172.21.15.100:0/2427240907 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f952801a070 con 0x7f9538105e60 2024-05-10T11:21:00.238 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.232+0000 7f9541feb740 1 -- start start 2024-05-10T11:21:00.238 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.232+0000 7f9541feb740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c1afcebdb0 0x55c1afcec1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.232+0000 7f9541feb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1afcec7a0 0x55c1afcf12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.232+0000 7f9541feb740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c1afcf1910 0x55c1afcf3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.232+0000 7f9541feb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55c1afc505d0 con 0x55c1afcec7a0 2024-05-10T11:21:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.232+0000 7f9541feb740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c1afc09890 con 0x55c1afcebdb0 2024-05-10T11:21:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.232+0000 7f9541feb740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c1afc4c590 con 0x55c1afcf1910 2024-05-10T11:21:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.232+0000 7f953cf93640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1afcec7a0 0x55c1afcf12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.232+0000 7f953cf93640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1afcec7a0 0x55c1afcf12a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51418/0 (socket says 172.21.15.100:51418) 2024-05-10T11:21:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.232+0000 7f953cf93640 1 -- 172.21.15.100:0/647757491 learned_addr learned my addr 172.21.15.100:0/647757491 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:21:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.232+0000 7f953d794640 1 --2- 172.21.15.100:0/647757491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c1afcebdb0 0x55c1afcec1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.232+0000 7f952ffff640 1 --2- 172.21.15.100:0/647757491 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c1afcf1910 0x55c1afcf3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.233+0000 7f953cf93640 1 -- 172.21.15.100:0/647757491 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c1afcf1910 msgr2=0x55c1afcf3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.233+0000 7f953cf93640 1 --2- 172.21.15.100:0/647757491 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c1afcf1910 0x55c1afcf3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.233+0000 7f953cf93640 1 -- 172.21.15.100:0/647757491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c1afcebdb0 msgr2=0x55c1afcec1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.233+0000 7f953cf93640 1 --2- 172.21.15.100:0/647757491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c1afcebdb0 0x55c1afcec1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.233+0000 7f953cf93640 1 -- 172.21.15.100:0/647757491 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1af962210 con 0x55c1afcec7a0 2024-05-10T11:21:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.233+0000 7f953d794640 1 --2- 172.21.15.100:0/647757491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c1afcebdb0 0x55c1afcec1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:21:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.233+0000 7f953cf93640 1 --2- 172.21.15.100:0/647757491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1afcec7a0 0x55c1afcf12a0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f9534006010 tx=0x7f9534004150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=159c7b02522e2e0b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.233+0000 7f952dffb640 1 --2- 172.21.15.100:0/2427240907 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f95200b28a0 0x7f95200b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.233+0000 7f951effd640 1 -- 172.21.15.100:0/647757491 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9534040070 con 0x55c1afcec7a0 2024-05-10T11:21:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.233+0000 7f953d794640 1 --2- 172.21.15.100:0/2427240907 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f95200b28a0 0x7f95200b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.233+0000 7f952dffb640 1 -- 172.21.15.100:0/2427240907 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f95280e2070 con 0x7f9538105e60 2024-05-10T11:21:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.233+0000 7f951effd640 1 -- 172.21.15.100:0/647757491 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9534003a80 con 0x55c1afcec7a0 2024-05-10T11:21:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.233+0000 7f951effd640 1 -- 172.21.15.100:0/647757491 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9534035030 con 0x55c1afcec7a0 2024-05-10T11:21:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.234+0000 7f9541feb740 1 -- 172.21.15.100:0/647757491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1afcec7a0 msgr2=0x55c1afcf12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.234+0000 7f953d794640 1 --2- 172.21.15.100:0/2427240907 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f95200b28a0 0x7f95200b4d60 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x55c1afced000 tx=0x7f9530009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.234+0000 7f9541feb740 1 --2- 172.21.15.100:0/647757491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1afcec7a0 0x55c1afcf12a0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f9534006010 tx=0x7f9534004150 comp rx=0 tx=0).stop 2024-05-10T11:21:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.234+0000 7f9541feb740 1 -- 172.21.15.100:0/647757491 shutdown_connections 2024-05-10T11:21:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.234+0000 7f9541feb740 1 --2- 172.21.15.100:0/647757491 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c1afcf1910 0x55c1afcf3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.234+0000 7f9541feb740 1 --2- 172.21.15.100:0/647757491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1afcec7a0 0x55c1afcf12a0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.234+0000 7f9541feb740 1 --2- 172.21.15.100:0/647757491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c1afcebdb0 0x55c1afcec1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.234+0000 7f9541feb740 1 -- 172.21.15.100:0/647757491 >> 172.21.15.100:0/647757491 conn(0x55c1afce7420 msgr2=0x55c1afce7820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.234+0000 7f9541feb740 1 -- 172.21.15.100:0/647757491 shutdown_connections 2024-05-10T11:21:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.234+0000 7f9541feb740 1 -- 172.21.15.100:0/647757491 wait complete. 2024-05-10T11:21:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.234+0000 7f9541feb740 1 -- start start 2024-05-10T11:21:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.234+0000 7f9541feb740 1 Processor -- start 2024-05-10T11:21:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.235+0000 7f9541feb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1afcebdb0 0x55c1afcf5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.235+0000 7f9541feb740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c1afcec7a0 0x55c1afcf6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.235+0000 7f9541feb740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c1afcf1910 0x55c1afcfd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.235+0000 7f9541feb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55c1afcf4860 con 0x55c1afcebdb0 2024-05-10T11:21:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.235+0000 7f9541feb740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c1afc0d370 con 0x55c1afcf1910 2024-05-10T11:21:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.235+0000 7f9541feb740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c1afcf49e0 con 0x55c1afcec7a0 2024-05-10T11:21:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.235+0000 7f9541feb740 10 client.0 _ll_register_callbacks cb 0x55c1af961a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:21:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.235+0000 7f953cf93640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1afcebdb0 0x55c1afcf5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.235+0000 7f953cf93640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1afcebdb0 0x55c1afcf5fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51432/0 (socket says 172.21.15.100:51432) 2024-05-10T11:21:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.235+0000 7f953cf93640 1 -- 172.21.15.100:0/767127339 learned_addr learned my addr 172.21.15.100:0/767127339 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:21:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.235+0000 7f952ffff640 1 --2- 172.21.15.100:0/767127339 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c1afcec7a0 0x55c1afcf6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.235+0000 7f953d794640 1 --2- 172.21.15.100:0/767127339 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c1afcf1910 0x55c1afcfd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.235+0000 7f953cf93640 1 -- 172.21.15.100:0/767127339 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c1afcec7a0 msgr2=0x55c1afcf6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.235+0000 7f953cf93640 1 --2- 172.21.15.100:0/767127339 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c1afcec7a0 0x55c1afcf6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.235+0000 7f953cf93640 1 -- 172.21.15.100:0/767127339 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c1afcf1910 msgr2=0x55c1afcfd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.235+0000 7f953cf93640 1 --2- 172.21.15.100:0/767127339 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c1afcf1910 0x55c1afcfd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.235+0000 7f953cf93640 1 -- 172.21.15.100:0/767127339 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1afa2e620 con 0x55c1afcebdb0 2024-05-10T11:21:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.236+0000 7f953cf93640 1 --2- 172.21.15.100:0/767127339 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1afcebdb0 0x55c1afcf5fd0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f9534009900 tx=0x7f95340045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beff643be6207e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.236+0000 7f9517fff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:21:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.236+0000 7f9517fff640 1 -- 172.21.15.100:0/767127339 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953404d020 con 0x55c1afcebdb0 2024-05-10T11:21:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.236+0000 7f9517fff640 1 -- 172.21.15.100:0/767127339 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f95340037f0 con 0x55c1afcebdb0 2024-05-10T11:21:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.236+0000 7f9541feb740 1 -- 172.21.15.100:0/767127339 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9534003a80 con 0x55c1afcebdb0 2024-05-10T11:21:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.236+0000 7f9517fff640 1 -- 172.21.15.100:0/767127339 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9534058050 con 0x55c1afcebdb0 2024-05-10T11:21:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.236+0000 7f9541feb740 10 client.15870 fetch_fsmap learned FSMap version 29 2024-05-10T11:21:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.236+0000 7f9541feb740 1 -- 172.21.15.100:0/767127339 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c1afd00530 con 0x55c1afcebdb0 2024-05-10T11:21:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.236+0000 7f9517fff640 1 -- 172.21.15.100:0/767127339 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9534003a80 con 0x55c1afcebdb0 2024-05-10T11:21:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.236+0000 7f9517fff640 1 -- 172.21.15.100:0/767127339 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f953403a040 con 0x55c1afcebdb0 2024-05-10T11:21:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.236+0000 7f9541feb740 10 client.15870 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T11:21:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.236+0000 7f9541feb740 10 client.15870 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:21:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.236+0000 7f9541feb740 20 client.15870 populate_metadata read hostname 'smithi100' 2024-05-10T11:21:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.236+0000 7f9541feb740 1 --2- 172.21.15.100:0/767127339 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c1afd01880 0x55c1afd03c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.236+0000 7f9541feb740 4 client.15870 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:21:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.236+0000 7f9541feb740 1 -- 172.21.15.100:0/767127339 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c1afc5e1e0 con 0x55c1afd01880 2024-05-10T11:21:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.237+0000 7f952ffff640 1 --2- 172.21.15.100:0/767127339 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c1afd01880 0x55c1afd03c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.237+0000 7f952ffff640 1 --2- 172.21.15.100:0/767127339 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c1afd01880 0x55c1afd03c70 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=568d31f945647e00 server_cookie=3c2b13f4efaae4d6 in_seq=0 out_seq=0 2024-05-10T11:21:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.237+0000 7f9517fff640 10 client.15870 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:21:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.238+0000 7f9517fff640 1 -- 172.21.15.100:0/767127339 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c1afc5e1e0 con 0x55c1afd01880 2024-05-10T11:21:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.238+0000 7f9517fff640 10 client.15870 handle_command_reply: tid=0 2024-05-10T11:21:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.238+0000 7f9541feb740 1 client.15870 shutdown 2024-05-10T11:21:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.238+0000 7f9541feb740 1 -- 172.21.15.100:0/767127339 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1afcebdb0 msgr2=0x55c1afcf5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.238+0000 7f9541feb740 1 --2- 172.21.15.100:0/767127339 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1afcebdb0 0x55c1afcf5fd0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f9534009900 tx=0x7f95340045f0 comp rx=0 tx=0).stop 2024-05-10T11:21:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.238+0000 7f9541feb740 1 -- 172.21.15.100:0/767127339 shutdown_connections 2024-05-10T11:21:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.238+0000 7f9541feb740 1 --2- 172.21.15.100:0/767127339 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c1afd01880 0x55c1afd03c70 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.238+0000 7f9541feb740 1 --2- 172.21.15.100:0/767127339 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c1afcf1910 0x55c1afcfd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:00.245 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.238+0000 7f9517fff640 0 client.15870 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:21:00.245 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.238+0000 7f9541feb740 1 --2- 172.21.15.100:0/767127339 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c1afcec7a0 0x55c1afcf6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:00.245 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.238+0000 7f9541feb740 1 --2- 172.21.15.100:0/767127339 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1afcebdb0 0x55c1afcf5fd0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:00.245 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.238+0000 7f9541feb740 1 -- 172.21.15.100:0/767127339 >> 172.21.15.100:0/767127339 conn(0x55c1afce7420 msgr2=0x55c1afce6580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:00.245 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.238+0000 7f9541feb740 1 -- 172.21.15.100:0/767127339 shutdown_connections 2024-05-10T11:21:00.245 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.238+0000 7f9541feb740 1 -- 172.21.15.100:0/767127339 wait complete. 2024-05-10T11:21:00.245 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.238+0000 7f9541feb740 20 client.15870 trim_cache size 0 max 16384 2024-05-10T11:21:00.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:20:59 smithi178 ceph-mon[33359]: pgmap v498: 129 pgs: 129 active+clean; 30 MiB data, 527 MiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 1.3 MiB/s wr, 134 op/s 2024-05-10T11:21:00.258 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.254+0000 7f9541feb740 1 Processor -- start 2024-05-10T11:21:00.258 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.254+0000 7f9541feb740 1 -- start start 2024-05-10T11:21:00.258 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.254+0000 7f9541feb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1afcebdb0 0x55c1afcf1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:00.258 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.254+0000 7f9541feb740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c1afcec7a0 0x55c1afcea100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:00.258 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.254+0000 7f9541feb740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c1afcf1910 0x55c1afcea640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:00.258 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.254+0000 7f9541feb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f95381afe20 con 0x55c1afcebdb0 2024-05-10T11:21:00.258 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.254+0000 7f9541feb740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f95381ad6f0 con 0x55c1afcf1910 2024-05-10T11:21:00.259 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.254+0000 7f9541feb740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f95381ab540 con 0x55c1afcec7a0 2024-05-10T11:21:00.259 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.254+0000 7f953cf93640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1afcebdb0 0x55c1afcf1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:00.259 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.254+0000 7f953cf93640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1afcebdb0 0x55c1afcf1030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51434/0 (socket says 172.21.15.100:51434) 2024-05-10T11:21:00.259 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.254+0000 7f953cf93640 1 -- 172.21.15.100:0/1896007979 learned_addr learned my addr 172.21.15.100:0/1896007979 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:21:00.259 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.254+0000 7f952ffff640 1 --2- 172.21.15.100:0/1896007979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c1afcec7a0 0x55c1afcea100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:00.259 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.254+0000 7f953d794640 1 --2- 172.21.15.100:0/1896007979 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c1afcf1910 0x55c1afcea640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:00.259 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.254+0000 7f953cf93640 1 -- 172.21.15.100:0/1896007979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c1afcec7a0 msgr2=0x55c1afcea100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:00.259 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.254+0000 7f953cf93640 1 --2- 172.21.15.100:0/1896007979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c1afcec7a0 0x55c1afcea100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:00.259 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.254+0000 7f953cf93640 1 -- 172.21.15.100:0/1896007979 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c1afcf1910 msgr2=0x55c1afcea640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:00.259 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.254+0000 7f953cf93640 1 --2- 172.21.15.100:0/1896007979 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c1afcf1910 0x55c1afcea640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:00.259 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.254+0000 7f953cf93640 1 -- 172.21.15.100:0/1896007979 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1afce7250 con 0x55c1afcebdb0 2024-05-10T11:21:00.259 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.254+0000 7f952ffff640 1 --2- 172.21.15.100:0/1896007979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c1afcec7a0 0x55c1afcea100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:21:00.259 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.254+0000 7f953d794640 1 --2- 172.21.15.100:0/1896007979 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c1afcf1910 0x55c1afcea640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:21:00.259 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.255+0000 7f953cf93640 1 --2- 172.21.15.100:0/1896007979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1afcebdb0 0x55c1afcf1030 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f9534000c00 tx=0x7f9534031750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72ad512756a269cb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.255+0000 7f951e7fc640 1 -- 172.21.15.100:0/1896007979 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953404d040 con 0x55c1afcebdb0 2024-05-10T11:21:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.255+0000 7f951e7fc640 1 -- 172.21.15.100:0/1896007979 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f95340035f0 con 0x55c1afcebdb0 2024-05-10T11:21:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.255+0000 7f951e7fc640 1 -- 172.21.15.100:0/1896007979 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9534040070 con 0x55c1afcebdb0 2024-05-10T11:21:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.255+0000 7f9541feb740 1 -- 172.21.15.100:0/1896007979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1afcebdb0 msgr2=0x55c1afcf1030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.255+0000 7f9541feb740 1 --2- 172.21.15.100:0/1896007979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1afcebdb0 0x55c1afcf1030 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f9534000c00 tx=0x7f9534031750 comp rx=0 tx=0).stop 2024-05-10T11:21:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.255+0000 7f9541feb740 1 -- 172.21.15.100:0/1896007979 shutdown_connections 2024-05-10T11:21:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.255+0000 7f9541feb740 1 --2- 172.21.15.100:0/1896007979 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c1afcf1910 0x55c1afcea640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.255+0000 7f9541feb740 1 --2- 172.21.15.100:0/1896007979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c1afcec7a0 0x55c1afcea100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.255+0000 7f9541feb740 1 --2- 172.21.15.100:0/1896007979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1afcebdb0 0x55c1afcf1030 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.255+0000 7f9541feb740 1 -- 172.21.15.100:0/1896007979 >> 172.21.15.100:0/1896007979 conn(0x55c1afce7420 msgr2=0x55c1afce7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.256+0000 7f9541feb740 1 -- 172.21.15.100:0/1896007979 shutdown_connections 2024-05-10T11:21:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.256+0000 7f9541feb740 1 -- 172.21.15.100:0/1896007979 wait complete. 2024-05-10T11:21:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.256+0000 7f9541feb740 1 -- start start 2024-05-10T11:21:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.256+0000 7f9541feb740 1 Processor -- start 2024-05-10T11:21:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.256+0000 7f9541feb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1afcebdb0 0x55c1afcfa030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:00.261 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.256+0000 7f9541feb740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c1afcec7a0 0x55c1afcfa570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:00.261 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.256+0000 7f9541feb740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c1afcf1910 0x55c1afcfb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:00.261 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.256+0000 7f9541feb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f95381abbb0 con 0x55c1afcebdb0 2024-05-10T11:21:00.261 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.256+0000 7f9541feb740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f95381ad020 con 0x55c1afcec7a0 2024-05-10T11:21:00.261 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.256+0000 7f9541feb740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f95381ac280 con 0x55c1afcf1910 2024-05-10T11:21:00.261 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.256+0000 7f9541feb740 10 client.0 _ll_register_callbacks cb 0x7f9530009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:21:00.261 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.256+0000 7f953cf93640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1afcebdb0 0x55c1afcfa030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:00.261 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.256+0000 7f953cf93640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1afcebdb0 0x55c1afcfa030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51438/0 (socket says 172.21.15.100:51438) 2024-05-10T11:21:00.261 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.256+0000 7f953cf93640 1 -- 172.21.15.100:0/2665483149 learned_addr learned my addr 172.21.15.100:0/2665483149 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:21:00.261 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.256+0000 7f953d794640 1 --2- 172.21.15.100:0/2665483149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c1afcf1910 0x55c1afcfb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:00.261 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.256+0000 7f952ffff640 1 --2- 172.21.15.100:0/2665483149 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c1afcec7a0 0x55c1afcfa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:00.261 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.257+0000 7f953cf93640 1 -- 172.21.15.100:0/2665483149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c1afcf1910 msgr2=0x55c1afcfb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:00.261 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.257+0000 7f953cf93640 1 --2- 172.21.15.100:0/2665483149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c1afcf1910 0x55c1afcfb580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:00.261 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.257+0000 7f953cf93640 1 -- 172.21.15.100:0/2665483149 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c1afcec7a0 msgr2=0x55c1afcfa570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:00.261 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.257+0000 7f953cf93640 1 --2- 172.21.15.100:0/2665483149 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c1afcec7a0 0x55c1afcfa570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:00.262 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.257+0000 7f953cf93640 1 -- 172.21.15.100:0/2665483149 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1afcfbb70 con 0x55c1afcebdb0 2024-05-10T11:21:00.262 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.257+0000 7f953cf93640 1 --2- 172.21.15.100:0/2665483149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1afcebdb0 0x55c1afcfa030 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f9534009820 tx=0x7f953403c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=469d3ab15e7cd931 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:00.262 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.257+0000 7f9517fff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:21:00.262 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.257+0000 7f9517fff640 1 -- 172.21.15.100:0/2665483149 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953404d070 con 0x55c1afcebdb0 2024-05-10T11:21:00.262 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.257+0000 7f9517fff640 1 -- 172.21.15.100:0/2665483149 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f95340039b0 con 0x55c1afcebdb0 2024-05-10T11:21:00.262 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.257+0000 7f9541feb740 1 -- 172.21.15.100:0/2665483149 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95340035f0 con 0x55c1afcebdb0 2024-05-10T11:21:00.262 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.257+0000 7f9517fff640 1 -- 172.21.15.100:0/2665483149 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9534040070 con 0x55c1afcebdb0 2024-05-10T11:21:00.262 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.257+0000 7f9541feb740 10 client.15876 fetch_fsmap learned FSMap version 29 2024-05-10T11:21:00.262 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.257+0000 7f9541feb740 1 -- 172.21.15.100:0/2665483149 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c1afcfcdf0 con 0x55c1afcebdb0 2024-05-10T11:21:00.262 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.257+0000 7f9517fff640 1 -- 172.21.15.100:0/2665483149 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f95340035f0 con 0x55c1afcebdb0 2024-05-10T11:21:00.262 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.258+0000 7f9517fff640 1 -- 172.21.15.100:0/2665483149 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f95340454e0 con 0x55c1afcebdb0 2024-05-10T11:21:00.262 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.258+0000 7f9541feb740 10 client.15876 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T11:21:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.258+0000 7f9541feb740 10 client.15876 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:21:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.258+0000 7f9541feb740 20 client.15876 populate_metadata read hostname 'smithi100' 2024-05-10T11:21:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.258+0000 7f9541feb740 1 --2- 172.21.15.100:0/2665483149 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c1afd01880 0x55c1afd6e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.258+0000 7f9541feb740 4 client.15876 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:21:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.258+0000 7f9541feb740 1 -- 172.21.15.100:0/2665483149 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c1afc58050 con 0x55c1afd01880 2024-05-10T11:21:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.258+0000 7f952ffff640 1 --2- 172.21.15.100:0/2665483149 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c1afd01880 0x55c1afd6e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.258+0000 7f9517fff640 10 client.15876 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:21:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.258+0000 7f952ffff640 1 --2- 172.21.15.100:0/2665483149 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c1afd01880 0x55c1afd6e560 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbb7bcf288d02776 server_cookie=63493e442e0a79be in_seq=0 out_seq=0 2024-05-10T11:21:00.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:20:59 smithi188 ceph-mon[34075]: pgmap v498: 129 pgs: 129 active+clean; 30 MiB data, 527 MiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 1.3 MiB/s wr, 134 op/s 2024-05-10T11:21:00.282 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.281+0000 7f9517fff640 1 -- 172.21.15.100:0/2665483149 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c1afc58050 con 0x55c1afd01880 2024-05-10T11:21:00.282 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.281+0000 7f9517fff640 10 client.15876 handle_command_reply: tid=0 2024-05-10T11:21:00.282 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.281+0000 7f9541feb740 1 client.15876 shutdown 2024-05-10T11:21:00.282 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.281+0000 7f9541feb740 1 -- 172.21.15.100:0/2665483149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1afcebdb0 msgr2=0x55c1afcfa030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:00.282 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.281+0000 7f9541feb740 1 --2- 172.21.15.100:0/2665483149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1afcebdb0 0x55c1afcfa030 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f9534009820 tx=0x7f953403c6a0 comp rx=0 tx=0).stop 2024-05-10T11:21:00.282 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.281+0000 7f9541feb740 1 -- 172.21.15.100:0/2665483149 shutdown_connections 2024-05-10T11:21:00.282 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.281+0000 7f9541feb740 1 --2- 172.21.15.100:0/2665483149 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c1afd01880 0x55c1afd6e560 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:00.282 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.281+0000 7f9541feb740 1 --2- 172.21.15.100:0/2665483149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c1afcf1910 0x55c1afcfb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:00.282 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.281+0000 7f9517fff640 0 client.15876 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:21:00.282 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.281+0000 7f9541feb740 1 --2- 172.21.15.100:0/2665483149 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c1afcec7a0 0x55c1afcfa570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:00.282 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.281+0000 7f9541feb740 1 --2- 172.21.15.100:0/2665483149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1afcebdb0 0x55c1afcfa030 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:00.282 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.281+0000 7f9541feb740 1 -- 172.21.15.100:0/2665483149 >> 172.21.15.100:0/2665483149 conn(0x55c1afce7420 msgr2=0x55c1afced9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:00.282 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.281+0000 7f9541feb740 1 -- 172.21.15.100:0/2665483149 shutdown_connections 2024-05-10T11:21:00.283 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.281+0000 7f9541feb740 1 -- 172.21.15.100:0/2665483149 wait complete. 2024-05-10T11:21:00.283 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.282+0000 7f9541feb740 20 client.15876 trim_cache size 0 max 16384 2024-05-10T11:21:00.283 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:21:00.283 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:21:00.283 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:21:00.283 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:21:00.284 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.282+0000 7f953f1f7640 1 -- 172.21.15.100:0/2427240907 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f95200b28a0 msgr2=0x7f95200b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:00.284 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.282+0000 7f953f1f7640 1 --2- 172.21.15.100:0/2427240907 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f95200b28a0 0x7f95200b4d60 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x55c1afced000 tx=0x7f9530009500 comp rx=0 tx=0).stop 2024-05-10T11:21:00.284 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.282+0000 7f953f1f7640 1 -- 172.21.15.100:0/2427240907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9538105e60 msgr2=0x7f95381014e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:00.284 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.282+0000 7f953f1f7640 1 --2- 172.21.15.100:0/2427240907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9538105e60 0x7f95381014e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f952800d770 tx=0x7f952800dc30 comp rx=0 tx=0).stop 2024-05-10T11:21:00.284 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.282+0000 7f953f1f7640 1 -- 172.21.15.100:0/2427240907 shutdown_connections 2024-05-10T11:21:00.284 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.282+0000 7f953f1f7640 1 --2- 172.21.15.100:0/2427240907 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f95200b28a0 0x7f95200b4d60 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:00.284 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.282+0000 7f953f1f7640 1 --2- 172.21.15.100:0/2427240907 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9538108790 0x7f9538101a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:00.285 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.282+0000 7f953f1f7640 1 --2- 172.21.15.100:0/2427240907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9538105e60 0x7f95381014e0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:00.285 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.282+0000 7f953f1f7640 1 --2- 172.21.15.100:0/2427240907 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9538103530 0x7f9538102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:00.285 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.282+0000 7f953f1f7640 1 -- 172.21.15.100:0/2427240907 >> 172.21.15.100:0/2427240907 conn(0x7f95380fce80 msgr2=0x7f9538103f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:00.285 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.282+0000 7f953f1f7640 1 -- 172.21.15.100:0/2427240907 shutdown_connections 2024-05-10T11:21:00.285 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:00.282+0000 7f953f1f7640 1 -- 172.21.15.100:0/2427240907 wait complete. 2024-05-10T11:21:00.933 INFO:tasks.workunit.client.0.smithi100.stdout: 1 29634 3.20 MB/sec warmup 55 sec latency 57.275 ms 2024-05-10T11:21:01.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.855+0000 7fac0ec0f700 1 -- 172.21.15.100:0/2851726255 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac08108880 msgr2=0x7fac08110400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:01.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.855+0000 7fac0ec0f700 1 --2- 172.21.15.100:0/2851726255 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac08108880 0x7fac08110400 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fac0000b0d0 tx=0x7fac0000b3e0 comp rx=0 tx=0).stop 2024-05-10T11:21:01.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.855+0000 7fac0ec0f700 1 -- 172.21.15.100:0/2851726255 shutdown_connections 2024-05-10T11:21:01.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.855+0000 7fac0ec0f700 1 --2- 172.21.15.100:0/2851726255 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac08108880 0x7fac08110400 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:01.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.855+0000 7fac0ec0f700 1 --2- 172.21.15.100:0/2851726255 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac081037b0 0x7fac08103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:01.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.855+0000 7fac0ec0f700 1 --2- 172.21.15.100:0/2851726255 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac08104160 0x7fac08108150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:01.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.855+0000 7fac0ec0f700 1 -- 172.21.15.100:0/2851726255 >> 172.21.15.100:0/2851726255 conn(0x7fac0807a580 msgr2=0x7fac0807a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:01.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.855+0000 7fac0ec0f700 1 -- 172.21.15.100:0/2851726255 shutdown_connections 2024-05-10T11:21:01.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.855+0000 7fac0ec0f700 1 -- 172.21.15.100:0/2851726255 wait complete. 2024-05-10T11:21:01.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.856+0000 7fac0ec0f700 1 Processor -- start 2024-05-10T11:21:01.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.856+0000 7fac0ec0f700 1 -- start start 2024-05-10T11:21:01.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.856+0000 7fac0ec0f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac081037b0 0x7fac080757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:01.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.856+0000 7fac0ec0f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac08104160 0x7fac08075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:01.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.856+0000 7fac0ec0f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac08108880 0x7fac08076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:01.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.856+0000 7fac0ec0f700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fac08113520 con 0x7fac08108880 2024-05-10T11:21:01.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.856+0000 7fac0ec0f700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fac081133a0 con 0x7fac08104160 2024-05-10T11:21:01.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.856+0000 7fac0ec0f700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fac081136a0 con 0x7fac081037b0 2024-05-10T11:21:01.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.856+0000 7fac0d1ac700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac08108880 0x7fac08076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:01.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.856+0000 7fac0c9ab700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac081037b0 0x7fac080757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:01.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.856+0000 7fac0c9ab700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac081037b0 0x7fac080757a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:44860/0 (socket says 172.21.15.100:44860) 2024-05-10T11:21:01.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.856+0000 7fac0c9ab700 1 -- 172.21.15.100:0/927870208 learned_addr learned my addr 172.21.15.100:0/927870208 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:21:01.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.856+0000 7fac07fff700 1 --2- 172.21.15.100:0/927870208 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac08104160 0x7fac08075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:01.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.857+0000 7fac0c9ab700 1 -- 172.21.15.100:0/927870208 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac08104160 msgr2=0x7fac08075ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:01.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.857+0000 7fac0c9ab700 1 --2- 172.21.15.100:0/927870208 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac08104160 0x7fac08075ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:01.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.857+0000 7fac0c9ab700 1 -- 172.21.15.100:0/927870208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac08108880 msgr2=0x7fac08076220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:01.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.857+0000 7fac0c9ab700 1 --2- 172.21.15.100:0/927870208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac08108880 0x7fac08076220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:01.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.857+0000 7fac0c9ab700 1 -- 172.21.15.100:0/927870208 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac081a7ef0 con 0x7fac081037b0 2024-05-10T11:21:01.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.857+0000 7fac0d1ac700 1 --2- 172.21.15.100:0/927870208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac08108880 0x7fac08076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:21:01.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.857+0000 7fac0c9ab700 1 --2- 172.21.15.100:0/927870208 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac081037b0 0x7fac080757a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fabfc00d950 tx=0x7fabfc00dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:01.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.857+0000 7fac05ffb700 1 -- 172.21.15.100:0/927870208 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabfc016070 con 0x7fac081037b0 2024-05-10T11:21:01.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.857+0000 7fac05ffb700 1 -- 172.21.15.100:0/927870208 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fabfc00fd20 con 0x7fac081037b0 2024-05-10T11:21:01.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.857+0000 7fac0ec0f700 1 -- 172.21.15.100:0/927870208 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac081a81e0 con 0x7fac081037b0 2024-05-10T11:21:01.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.857+0000 7fac05ffb700 1 -- 172.21.15.100:0/927870208 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabfc00e3b0 con 0x7fac081037b0 2024-05-10T11:21:01.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.857+0000 7fac0ec0f700 1 -- 172.21.15.100:0/927870208 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac081a8720 con 0x7fac081037b0 2024-05-10T11:21:01.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.859+0000 7fac05ffb700 1 -- 172.21.15.100:0/927870208 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fabfc019070 con 0x7fac081037b0 2024-05-10T11:21:01.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.860+0000 7fac05ffb700 1 --2- 172.21.15.100:0/927870208 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fabf80b2880 0x7fabf80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:01.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.860+0000 7fac07fff700 1 --2- 172.21.15.100:0/927870208 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fabf80b2880 0x7fabf80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:01.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.860+0000 7fac05ffb700 1 -- 172.21.15.100:0/927870208 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fabfc0e2030 con 0x7fac081037b0 2024-05-10T11:21:01.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.861+0000 7fac07fff700 1 --2- 172.21.15.100:0/927870208 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fabf80b2880 0x7fabf80b4d40 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fac08108100 tx=0x7fabf4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:01.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.861+0000 7fac194a2b80 1 Processor -- start 2024-05-10T11:21:01.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.861+0000 7fac194a2b80 1 -- start start 2024-05-10T11:21:01.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.861+0000 7fac194a2b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561114e7a1b0 0x561114e7a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:01.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.861+0000 7fac194a2b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561114e7aad0 0x561114e74810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:01.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.861+0000 7fac194a2b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561114e74d50 0x561114e75190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:01.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.861+0000 7fac194a2b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x561114dc6940 con 0x561114e74d50 2024-05-10T11:21:01.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.861+0000 7fac194a2b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561114d2bda0 con 0x561114e7aad0 2024-05-10T11:21:01.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.861+0000 7fac194a2b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x561114dc2900 con 0x561114e7a1b0 2024-05-10T11:21:01.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.862+0000 7fac0c9ab700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561114e7a1b0 0x561114e7a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:01.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.862+0000 7fac0c9ab700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561114e7a1b0 0x561114e7a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:44870/0 (socket says 172.21.15.100:44870) 2024-05-10T11:21:01.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.862+0000 7fac0c9ab700 1 -- 172.21.15.100:0/3195290980 learned_addr learned my addr 172.21.15.100:0/3195290980 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:21:01.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.862+0000 7fac07fff700 1 --2- 172.21.15.100:0/3195290980 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561114e7aad0 0x561114e74810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:01.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.862+0000 7fac0d1ac700 1 --2- 172.21.15.100:0/3195290980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561114e74d50 0x561114e75190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:01.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.862+0000 7fac0c9ab700 1 -- 172.21.15.100:0/3195290980 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561114e7aad0 msgr2=0x561114e74810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:01.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.862+0000 7fac0c9ab700 1 --2- 172.21.15.100:0/3195290980 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561114e7aad0 0x561114e74810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:01.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.862+0000 7fac0c9ab700 1 -- 172.21.15.100:0/3195290980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561114e74d50 msgr2=0x561114e75190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:01.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.862+0000 7fac0c9ab700 1 --2- 172.21.15.100:0/3195290980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561114e74d50 0x561114e75190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:01.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.862+0000 7fac0c9ab700 1 -- 172.21.15.100:0/3195290980 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561114d138d0 con 0x561114e7a1b0 2024-05-10T11:21:01.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.862+0000 7fac07fff700 1 --2- 172.21.15.100:0/3195290980 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561114e7aad0 0x561114e74810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:21:01.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.862+0000 7fac0c9ab700 1 --2- 172.21.15.100:0/3195290980 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561114e7a1b0 0x561114e7a590 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fabfc00b2f0 tx=0x7fabfc0a8e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b223610318e157c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:01.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.862+0000 7fabf2ffd700 1 -- 172.21.15.100:0/3195290980 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabfc020070 con 0x561114e7a1b0 2024-05-10T11:21:01.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.862+0000 7fabf2ffd700 1 -- 172.21.15.100:0/3195290980 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fabfc0ad6a0 con 0x561114e7a1b0 2024-05-10T11:21:01.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.862+0000 7fabf2ffd700 1 -- 172.21.15.100:0/3195290980 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabfc0b1f20 con 0x561114e7a1b0 2024-05-10T11:21:01.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.863+0000 7fac194a2b80 1 -- 172.21.15.100:0/3195290980 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561114e7a1b0 msgr2=0x561114e7a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:01.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.863+0000 7fac194a2b80 1 --2- 172.21.15.100:0/3195290980 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561114e7a1b0 0x561114e7a590 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fabfc00b2f0 tx=0x7fabfc0a8e80 comp rx=0 tx=0).stop 2024-05-10T11:21:01.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.863+0000 7fac194a2b80 1 -- 172.21.15.100:0/3195290980 shutdown_connections 2024-05-10T11:21:01.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.863+0000 7fac194a2b80 1 --2- 172.21.15.100:0/3195290980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561114e74d50 0x561114e75190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:01.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.863+0000 7fac194a2b80 1 --2- 172.21.15.100:0/3195290980 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561114e7a1b0 0x561114e7a590 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:01.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.863+0000 7fac194a2b80 1 --2- 172.21.15.100:0/3195290980 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561114e7aad0 0x561114e74810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:01.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.863+0000 7fac194a2b80 1 -- 172.21.15.100:0/3195290980 >> 172.21.15.100:0/3195290980 conn(0x561114da0b70 msgr2=0x561114e6f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:01.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.863+0000 7fac194a2b80 1 -- 172.21.15.100:0/3195290980 shutdown_connections 2024-05-10T11:21:01.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.863+0000 7fac194a2b80 1 -- 172.21.15.100:0/3195290980 wait complete. 2024-05-10T11:21:01.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.864+0000 7fac194a2b80 1 -- start start 2024-05-10T11:21:01.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.864+0000 7fac194a2b80 1 Processor -- start 2024-05-10T11:21:01.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.864+0000 7fac194a2b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561114e74d50 0x561114e807a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:01.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.864+0000 7fac194a2b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561114e7a1b0 0x561114e7db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:01.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.864+0000 7fac194a2b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561114e7aad0 0x561114e7e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:01.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.864+0000 7fac194a2b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x561114e81390 con 0x561114e74d50 2024-05-10T11:21:01.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.864+0000 7fac194a2b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561114d7fed0 con 0x561114e7aad0 2024-05-10T11:21:01.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.864+0000 7fac194a2b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x561114e81510 con 0x561114e7a1b0 2024-05-10T11:21:01.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.864+0000 7fac194a2b80 10 client.0 _ll_register_callbacks cb 0x561114d5a5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:21:01.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.864+0000 7fac0c9ab700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561114e74d50 0x561114e807a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:01.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.864+0000 7fac0c9ab700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561114e74d50 0x561114e807a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51472/0 (socket says 172.21.15.100:51472) 2024-05-10T11:21:01.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.864+0000 7fac07fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561114e7a1b0 0x561114e7db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:01.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.864+0000 7fac0c9ab700 1 -- 172.21.15.100:0/2753467554 learned_addr learned my addr 172.21.15.100:0/2753467554 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:21:01.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.864+0000 7fac0d1ac700 1 --2- 172.21.15.100:0/2753467554 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561114e7aad0 0x561114e7e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:01.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.865+0000 7fac0c9ab700 1 -- 172.21.15.100:0/2753467554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561114e7a1b0 msgr2=0x561114e7db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:01.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.865+0000 7fac0c9ab700 1 --2- 172.21.15.100:0/2753467554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561114e7a1b0 0x561114e7db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:01.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.865+0000 7fac0c9ab700 1 -- 172.21.15.100:0/2753467554 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561114e7aad0 msgr2=0x561114e7e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:01.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.865+0000 7fac0c9ab700 1 --2- 172.21.15.100:0/2753467554 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561114e7aad0 0x561114e7e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:01.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.865+0000 7fac0c9ab700 1 -- 172.21.15.100:0/2753467554 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561114d7d760 con 0x561114e74d50 2024-05-10T11:21:01.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.865+0000 7fac0c9ab700 1 --2- 172.21.15.100:0/2753467554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561114e74d50 0x561114e807a0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fabfc0df820 tx=0x7fabfc010970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d6f0ad114a41df8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:01.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.865+0000 7fabe7fff700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:21:01.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.865+0000 7fabe7fff700 1 -- 172.21.15.100:0/2753467554 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabfc020070 con 0x561114e74d50 2024-05-10T11:21:01.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.865+0000 7fabe7fff700 1 -- 172.21.15.100:0/2753467554 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fabfc0b1cc0 con 0x561114e74d50 2024-05-10T11:21:01.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.865+0000 7fac194a2b80 1 -- 172.21.15.100:0/2753467554 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabfc0ad6a0 con 0x561114e74d50 2024-05-10T11:21:01.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.865+0000 7fabe7fff700 1 -- 172.21.15.100:0/2753467554 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fabfc0b1ec0 con 0x561114e74d50 2024-05-10T11:21:01.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.865+0000 7fac194a2b80 10 client.15888 fetch_fsmap learned FSMap version 29 2024-05-10T11:21:01.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.865+0000 7fac194a2b80 1 -- 172.21.15.100:0/2753467554 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561114e8bca0 con 0x561114e74d50 2024-05-10T11:21:01.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.865+0000 7fabe7fff700 1 -- 172.21.15.100:0/2753467554 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fabfc0ad6a0 con 0x561114e74d50 2024-05-10T11:21:01.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.865+0000 7fabe7fff700 1 -- 172.21.15.100:0/2753467554 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fabfc0ad240 con 0x561114e74d50 2024-05-10T11:21:01.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.865+0000 7fac194a2b80 10 client.15888 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T11:21:01.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.865+0000 7fac194a2b80 10 client.15888 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:21:01.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.865+0000 7fac194a2b80 20 client.15888 populate_metadata read hostname 'smithi100' 2024-05-10T11:21:01.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.865+0000 7fac194a2b80 1 --2- 172.21.15.100:0/2753467554 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x561114e8e170 0x561114e90560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:01.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.865+0000 7fac194a2b80 4 client.15888 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:21:01.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.866+0000 7fac194a2b80 1 -- 172.21.15.100:0/2753467554 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561114dd4320 con 0x561114e8e170 2024-05-10T11:21:01.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.866+0000 7fac07fff700 1 --2- 172.21.15.100:0/2753467554 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x561114e8e170 0x561114e90560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:01.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.866+0000 7fabe7fff700 10 client.15888 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:21:01.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.866+0000 7fac07fff700 1 --2- 172.21.15.100:0/2753467554 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x561114e8e170 0x561114e90560 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdc2b910ff0001c0 server_cookie=5e580f3ed5430c95 in_seq=0 out_seq=0 2024-05-10T11:21:01.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.867+0000 7fabe7fff700 1 -- 172.21.15.100:0/2753467554 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561114dd4320 con 0x561114e8e170 2024-05-10T11:21:01.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.867+0000 7fabe7fff700 10 client.15888 handle_command_reply: tid=0 2024-05-10T11:21:01.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.867+0000 7fac194a2b80 1 client.15888 shutdown 2024-05-10T11:21:01.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.867+0000 7fac194a2b80 1 -- 172.21.15.100:0/2753467554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561114e74d50 msgr2=0x561114e807a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:01.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.867+0000 7fac194a2b80 1 --2- 172.21.15.100:0/2753467554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561114e74d50 0x561114e807a0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fabfc0df820 tx=0x7fabfc010970 comp rx=0 tx=0).stop 2024-05-10T11:21:01.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.867+0000 7fac194a2b80 1 -- 172.21.15.100:0/2753467554 shutdown_connections 2024-05-10T11:21:01.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.867+0000 7fac194a2b80 1 --2- 172.21.15.100:0/2753467554 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x561114e8e170 0x561114e90560 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:01.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.867+0000 7fabe7fff700 0 client.15888 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:21:01.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.867+0000 7fac194a2b80 1 --2- 172.21.15.100:0/2753467554 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561114e7aad0 0x561114e7e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:01.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.867+0000 7fac194a2b80 1 --2- 172.21.15.100:0/2753467554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561114e74d50 0x561114e807a0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:01.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.867+0000 7fac194a2b80 1 --2- 172.21.15.100:0/2753467554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561114e7a1b0 0x561114e7db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:01.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.867+0000 7fac194a2b80 1 -- 172.21.15.100:0/2753467554 >> 172.21.15.100:0/2753467554 conn(0x561114da0b70 msgr2=0x561114e69b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:01.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.868+0000 7fac194a2b80 1 -- 172.21.15.100:0/2753467554 shutdown_connections 2024-05-10T11:21:01.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.868+0000 7fac194a2b80 1 -- 172.21.15.100:0/2753467554 wait complete. 2024-05-10T11:21:01.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.868+0000 7fac194a2b80 20 client.15888 trim_cache size 0 max 16384 2024-05-10T11:21:01.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.885+0000 7fac194a2b80 1 Processor -- start 2024-05-10T11:21:01.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.885+0000 7fac194a2b80 1 -- start start 2024-05-10T11:21:01.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.885+0000 7fac194a2b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561114e74d50 0x561114e74810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:01.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.885+0000 7fac194a2b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561114e7a1b0 0x561114e7e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:01.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.885+0000 7fac194a2b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561114e7aad0 0x561114e69c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:01.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.885+0000 7fac194a2b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fac081b3b40 con 0x561114e74d50 2024-05-10T11:21:01.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.885+0000 7fac194a2b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fac081b13d0 con 0x561114e7aad0 2024-05-10T11:21:01.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.885+0000 7fac194a2b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fac081af220 con 0x561114e7a1b0 2024-05-10T11:21:01.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.885+0000 7fac0c9ab700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561114e74d50 0x561114e74810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:01.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.885+0000 7fac0c9ab700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561114e74d50 0x561114e74810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51476/0 (socket says 172.21.15.100:51476) 2024-05-10T11:21:01.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.885+0000 7fac0c9ab700 1 -- 172.21.15.100:0/2109003983 learned_addr learned my addr 172.21.15.100:0/2109003983 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:21:01.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.885+0000 7fac07fff700 1 --2- 172.21.15.100:0/2109003983 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561114e7a1b0 0x561114e7e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:01.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.885+0000 7fac0d1ac700 1 --2- 172.21.15.100:0/2109003983 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561114e7aad0 0x561114e69c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:01.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.885+0000 7fac0c9ab700 1 -- 172.21.15.100:0/2109003983 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561114e7a1b0 msgr2=0x561114e7e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:01.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.885+0000 7fac0c9ab700 1 --2- 172.21.15.100:0/2109003983 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561114e7a1b0 0x561114e7e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:01.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.885+0000 7fac0c9ab700 1 -- 172.21.15.100:0/2109003983 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561114e7aad0 msgr2=0x561114e69c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:01.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.885+0000 7fac0c9ab700 1 --2- 172.21.15.100:0/2109003983 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561114e7aad0 0x561114e69c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:01.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.885+0000 7fac0c9ab700 1 -- 172.21.15.100:0/2109003983 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561114e75650 con 0x561114e74d50 2024-05-10T11:21:01.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.885+0000 7fac0d1ac700 1 --2- 172.21.15.100:0/2109003983 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561114e7aad0 0x561114e69c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:21:01.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.886+0000 7fac0c9ab700 1 --2- 172.21.15.100:0/2109003983 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561114e74d50 0x561114e74810 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fabfc005950 tx=0x7fabfc0a91f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9859400e35965782 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:01.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.886+0000 7fabf27fc700 1 -- 172.21.15.100:0/2109003983 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabfc020070 con 0x561114e74d50 2024-05-10T11:21:01.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.886+0000 7fabf27fc700 1 -- 172.21.15.100:0/2109003983 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fabfc0ad6a0 con 0x561114e74d50 2024-05-10T11:21:01.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.886+0000 7fabf27fc700 1 -- 172.21.15.100:0/2109003983 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabfc0a4e30 con 0x561114e74d50 2024-05-10T11:21:01.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.886+0000 7fac194a2b80 1 -- 172.21.15.100:0/2109003983 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561114e74d50 msgr2=0x561114e74810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:01.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.886+0000 7fac194a2b80 1 --2- 172.21.15.100:0/2109003983 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561114e74d50 0x561114e74810 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fabfc005950 tx=0x7fabfc0a91f0 comp rx=0 tx=0).stop 2024-05-10T11:21:01.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.887+0000 7fac194a2b80 1 -- 172.21.15.100:0/2109003983 shutdown_connections 2024-05-10T11:21:01.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.887+0000 7fac194a2b80 1 --2- 172.21.15.100:0/2109003983 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561114e7aad0 0x561114e69c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:01.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.887+0000 7fac194a2b80 1 --2- 172.21.15.100:0/2109003983 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561114e74d50 0x561114e74810 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:01.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.887+0000 7fac194a2b80 1 --2- 172.21.15.100:0/2109003983 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561114e7a1b0 0x561114e7e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:01.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.887+0000 7fac194a2b80 1 -- 172.21.15.100:0/2109003983 >> 172.21.15.100:0/2109003983 conn(0x561114da0b70 msgr2=0x561114e7b550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:01.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.887+0000 7fac194a2b80 1 -- 172.21.15.100:0/2109003983 shutdown_connections 2024-05-10T11:21:01.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.887+0000 7fac194a2b80 1 -- 172.21.15.100:0/2109003983 wait complete. 2024-05-10T11:21:01.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.887+0000 7fac194a2b80 1 -- start start 2024-05-10T11:21:01.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.887+0000 7fac194a2b80 1 Processor -- start 2024-05-10T11:21:01.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.887+0000 7fac194a2b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561114e74d50 0x561114e93850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:01.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.888+0000 7fac194a2b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561114e7a1b0 0x561114e90c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:01.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.888+0000 7fac194a2b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561114e7aad0 0x561114e91160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:01.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.888+0000 7fac194a2b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fac081af890 con 0x561114e7aad0 2024-05-10T11:21:01.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.888+0000 7fac194a2b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fac081b0d00 con 0x561114e7a1b0 2024-05-10T11:21:01.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.888+0000 7fac194a2b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fac081aff60 con 0x561114e74d50 2024-05-10T11:21:01.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.888+0000 7fac194a2b80 10 client.0 _ll_register_callbacks cb 0x561114d5a5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:21:01.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.888+0000 7fac0d1ac700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561114e7aad0 0x561114e91160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:01.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.888+0000 7fac0d1ac700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561114e7aad0 0x561114e91160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51488/0 (socket says 172.21.15.100:51488) 2024-05-10T11:21:01.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.888+0000 7fac0d1ac700 1 -- 172.21.15.100:0/3857065587 learned_addr learned my addr 172.21.15.100:0/3857065587 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:21:01.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.888+0000 7fac0c9ab700 1 --2- 172.21.15.100:0/3857065587 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561114e74d50 0x561114e93850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:01.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.888+0000 7fac07fff700 1 --2- 172.21.15.100:0/3857065587 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561114e7a1b0 0x561114e90c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:01.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.888+0000 7fac0d1ac700 1 -- 172.21.15.100:0/3857065587 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561114e74d50 msgr2=0x561114e93850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:01.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.888+0000 7fac0d1ac700 1 --2- 172.21.15.100:0/3857065587 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561114e74d50 0x561114e93850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:01.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.888+0000 7fac0d1ac700 1 -- 172.21.15.100:0/3857065587 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561114e7a1b0 msgr2=0x561114e90c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:01.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.888+0000 7fac0d1ac700 1 --2- 172.21.15.100:0/3857065587 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561114e7a1b0 0x561114e90c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:01.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.888+0000 7fac0d1ac700 1 -- 172.21.15.100:0/3857065587 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561114e91720 con 0x561114e7aad0 2024-05-10T11:21:01.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.888+0000 7fac0c9ab700 1 --2- 172.21.15.100:0/3857065587 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561114e74d50 0x561114e93850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:21:01.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.888+0000 7fac0d1ac700 1 --2- 172.21.15.100:0/3857065587 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561114e7aad0 0x561114e91160 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fac00003d70 tx=0x7fac00003da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efc62fc1b9bd9a84 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:01.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.888+0000 7fabe7fff700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:21:01.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.888+0000 7fabe7fff700 1 -- 172.21.15.100:0/3857065587 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac0002b060 con 0x561114e7aad0 2024-05-10T11:21:01.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.888+0000 7fabe7fff700 1 -- 172.21.15.100:0/3857065587 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fac000078f0 con 0x561114e7aad0 2024-05-10T11:21:01.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.888+0000 7fac194a2b80 1 -- 172.21.15.100:0/3857065587 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabfc0ad6a0 con 0x561114e7aad0 2024-05-10T11:21:01.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.889+0000 7fabe7fff700 1 -- 172.21.15.100:0/3857065587 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fac00032050 con 0x561114e7aad0 2024-05-10T11:21:01.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.889+0000 7fac194a2b80 10 client.15894 fetch_fsmap learned FSMap version 29 2024-05-10T11:21:01.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.889+0000 7fac194a2b80 1 -- 172.21.15.100:0/3857065587 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561114ef8440 con 0x561114e7aad0 2024-05-10T11:21:01.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.889+0000 7fabe7fff700 1 -- 172.21.15.100:0/3857065587 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fabfc0ad6a0 con 0x561114e7aad0 2024-05-10T11:21:01.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.889+0000 7fabe7fff700 1 -- 172.21.15.100:0/3857065587 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fac00011070 con 0x561114e7aad0 2024-05-10T11:21:01.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.889+0000 7fac194a2b80 10 client.15894 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T11:21:01.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.889+0000 7fac194a2b80 10 client.15894 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:21:01.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.889+0000 7fac194a2b80 20 client.15894 populate_metadata read hostname 'smithi100' 2024-05-10T11:21:01.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.889+0000 7fac194a2b80 1 --2- 172.21.15.100:0/3857065587 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x561114e8e170 0x561114efac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:01.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.889+0000 7fac194a2b80 4 client.15894 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:21:01.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.889+0000 7fac194a2b80 1 -- 172.21.15.100:0/3857065587 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561114dce3c0 con 0x561114e8e170 2024-05-10T11:21:01.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.889+0000 7fac0c9ab700 1 --2- 172.21.15.100:0/3857065587 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x561114e8e170 0x561114efac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:01.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.890+0000 7fabe7fff700 10 client.15894 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:21:01.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.890+0000 7fac0c9ab700 1 --2- 172.21.15.100:0/3857065587 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x561114e8e170 0x561114efac70 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=201110a800756cc1 server_cookie=b1a506d0cc33aed3 in_seq=0 out_seq=0 2024-05-10T11:21:01.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.890+0000 7fabe7fff700 1 -- 172.21.15.100:0/3857065587 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561114dce3c0 con 0x561114e8e170 2024-05-10T11:21:01.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.890+0000 7fabe7fff700 10 client.15894 handle_command_reply: tid=0 2024-05-10T11:21:01.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.890+0000 7fac194a2b80 1 client.15894 shutdown 2024-05-10T11:21:01.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.891+0000 7fac194a2b80 1 -- 172.21.15.100:0/3857065587 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561114e7aad0 msgr2=0x561114e91160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:01.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.891+0000 7fac194a2b80 1 --2- 172.21.15.100:0/3857065587 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561114e7aad0 0x561114e91160 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fac00003d70 tx=0x7fac00003da0 comp rx=0 tx=0).stop 2024-05-10T11:21:01.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.891+0000 7fac194a2b80 1 -- 172.21.15.100:0/3857065587 shutdown_connections 2024-05-10T11:21:01.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.891+0000 7fac194a2b80 1 --2- 172.21.15.100:0/3857065587 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x561114e8e170 0x561114efac70 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:01.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.891+0000 7fac194a2b80 1 --2- 172.21.15.100:0/3857065587 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561114e7aad0 0x561114e91160 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:01.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.891+0000 7fac194a2b80 1 --2- 172.21.15.100:0/3857065587 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561114e74d50 0x561114e93850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:01.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.891+0000 7fabe7fff700 0 client.15894 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:21:01.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.891+0000 7fac194a2b80 1 --2- 172.21.15.100:0/3857065587 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561114e7a1b0 0x561114e90c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:01.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.891+0000 7fac194a2b80 1 -- 172.21.15.100:0/3857065587 >> 172.21.15.100:0/3857065587 conn(0x561114da0b70 msgr2=0x561114e794c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:01.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.891+0000 7fac194a2b80 1 -- 172.21.15.100:0/3857065587 shutdown_connections 2024-05-10T11:21:01.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.891+0000 7fac194a2b80 1 -- 172.21.15.100:0/3857065587 wait complete. 2024-05-10T11:21:01.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.891+0000 7fac194a2b80 20 client.15894 trim_cache size 0 max 16384 2024-05-10T11:21:01.896 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:21:01.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.892+0000 7fac0ec0f700 1 -- 172.21.15.100:0/927870208 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fabf80b2880 msgr2=0x7fabf80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:01.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.892+0000 7fac0ec0f700 1 --2- 172.21.15.100:0/927870208 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fabf80b2880 0x7fabf80b4d40 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fac08108100 tx=0x7fabf4009260 comp rx=0 tx=0).stop 2024-05-10T11:21:01.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.892+0000 7fac0ec0f700 1 -- 172.21.15.100:0/927870208 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac081037b0 msgr2=0x7fac080757a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:01.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.892+0000 7fac0ec0f700 1 --2- 172.21.15.100:0/927870208 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac081037b0 0x7fac080757a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fabfc00d950 tx=0x7fabfc00dc60 comp rx=0 tx=0).stop 2024-05-10T11:21:01.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.892+0000 7fac0ec0f700 1 -- 172.21.15.100:0/927870208 shutdown_connections 2024-05-10T11:21:01.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.892+0000 7fac0ec0f700 1 --2- 172.21.15.100:0/927870208 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fabf80b2880 0x7fabf80b4d40 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:01.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.892+0000 7fac0ec0f700 1 --2- 172.21.15.100:0/927870208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac08108880 0x7fac08076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:01.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.892+0000 7fac0ec0f700 1 --2- 172.21.15.100:0/927870208 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac081037b0 0x7fac080757a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:01.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.892+0000 7fac0ec0f700 1 --2- 172.21.15.100:0/927870208 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac08104160 0x7fac08075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:01.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.892+0000 7fac0ec0f700 1 -- 172.21.15.100:0/927870208 >> 172.21.15.100:0/927870208 conn(0x7fac0807a580 msgr2=0x7fac08101620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:01.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.892+0000 7fac0ec0f700 1 -- 172.21.15.100:0/927870208 shutdown_connections 2024-05-10T11:21:01.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:01.892+0000 7fac0ec0f700 1 -- 172.21.15.100:0/927870208 wait complete. 2024-05-10T11:21:01.933 INFO:tasks.workunit.client.0.smithi100.stdout: 1 30266 3.20 MB/sec warmup 56 sec latency 16.297 ms 2024-05-10T11:21:02.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:21:01 smithi100 ceph-mon[30296]: pgmap v499: 129 pgs: 129 active+clean; 31 MiB data, 554 MiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 1.3 MiB/s wr, 167 op/s 2024-05-10T11:21:02.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:21:01 smithi178 ceph-mon[33359]: pgmap v499: 129 pgs: 129 active+clean; 31 MiB data, 554 MiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 1.3 MiB/s wr, 167 op/s 2024-05-10T11:21:02.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:21:01 smithi188 ceph-mon[34075]: pgmap v499: 129 pgs: 129 active+clean; 31 MiB data, 554 MiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 1.3 MiB/s wr, 167 op/s 2024-05-10T11:21:02.933 INFO:tasks.workunit.client.0.smithi100.stdout: 1 30858 3.18 MB/sec warmup 57 sec latency 15.401 ms 2024-05-10T11:21:03.808 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T11:21:03.808 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T11:21:03.933 INFO:tasks.workunit.client.0.smithi100.stdout: 1 31605 3.23 MB/sec warmup 58 sec latency 55.417 ms 2024-05-10T11:21:04.073 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:21:04.194 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:21:03 smithi100 ceph-mon[30296]: pgmap v500: 129 pgs: 129 active+clean; 32 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 1.4 MiB/s wr, 137 op/s 2024-05-10T11:21:04.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:21:03 smithi178 ceph-mon[33359]: pgmap v500: 129 pgs: 129 active+clean; 32 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 1.4 MiB/s wr, 137 op/s 2024-05-10T11:21:04.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:21:03 smithi188 ceph-mon[34075]: pgmap v500: 129 pgs: 129 active+clean; 32 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 1.4 MiB/s wr, 137 op/s 2024-05-10T11:21:04.933 INFO:tasks.workunit.client.0.smithi100.stdout: 1 32073 3.25 MB/sec warmup 59 sec latency 15.369 ms 2024-05-10T11:21:05.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.872+0000 7f07619f0700 1 -- 172.21.15.100:0/3488363916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f075c102390 msgr2=0x7f075c102810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:05.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.872+0000 7f07619f0700 1 --2- 172.21.15.100:0/3488363916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f075c102390 0x7f075c102810 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f0744009990 tx=0x7f0744009ca0 comp rx=0 tx=0).stop 2024-05-10T11:21:05.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.872+0000 7f07619f0700 1 -- 172.21.15.100:0/3488363916 shutdown_connections 2024-05-10T11:21:05.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.872+0000 7f07619f0700 1 --2- 172.21.15.100:0/3488363916 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f075c102d50 0x7f075c10f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:05.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.872+0000 7f07619f0700 1 --2- 172.21.15.100:0/3488363916 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f075c108400 0x7f075c1087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:05.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.872+0000 7f07619f0700 1 --2- 172.21.15.100:0/3488363916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f075c102390 0x7f075c102810 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:05.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.872+0000 7f07619f0700 1 -- 172.21.15.100:0/3488363916 >> 172.21.15.100:0/3488363916 conn(0x7f075c079310 msgr2=0x7f075c079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:05.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.872+0000 7f07619f0700 1 -- 172.21.15.100:0/3488363916 shutdown_connections 2024-05-10T11:21:05.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.872+0000 7f07619f0700 1 -- 172.21.15.100:0/3488363916 wait complete. 2024-05-10T11:21:05.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.873+0000 7f07619f0700 1 Processor -- start 2024-05-10T11:21:05.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.873+0000 7f07619f0700 1 -- start start 2024-05-10T11:21:05.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.873+0000 7f07619f0700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f075c102390 0x7f075c19a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:05.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.873+0000 7f07619f0700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f075c102d50 0x7f075c19add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:05.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.873+0000 7f07619f0700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f075c108400 0x7f075c1a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:05.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.873+0000 7f07619f0700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f075c1123d0 con 0x7f075c102d50 2024-05-10T11:21:05.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.873+0000 7f07619f0700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f075c112250 con 0x7f075c108400 2024-05-10T11:21:05.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.873+0000 7f07619f0700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f075c112550 con 0x7f075c102390 2024-05-10T11:21:05.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.873+0000 7f0753fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f075c102d50 0x7f075c19add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:05.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.873+0000 7f075affd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f075c102390 0x7f075c19a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:05.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.873+0000 7f0753fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f075c102d50 0x7f075c19add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51506/0 (socket says 172.21.15.100:51506) 2024-05-10T11:21:05.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.873+0000 7f0753fff700 1 -- 172.21.15.100:0/796822268 learned_addr learned my addr 172.21.15.100:0/796822268 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:21:05.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.873+0000 7f075b7fe700 1 --2- 172.21.15.100:0/796822268 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f075c108400 0x7f075c1a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:05.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.874+0000 7f0753fff700 1 -- 172.21.15.100:0/796822268 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f075c102390 msgr2=0x7f075c19a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:05.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.874+0000 7f0753fff700 1 --2- 172.21.15.100:0/796822268 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f075c102390 0x7f075c19a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:05.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.874+0000 7f0753fff700 1 -- 172.21.15.100:0/796822268 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f075c108400 msgr2=0x7f075c1a5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:05.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.874+0000 7f0753fff700 1 --2- 172.21.15.100:0/796822268 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f075c108400 0x7f075c1a5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:05.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.874+0000 7f0753fff700 1 -- 172.21.15.100:0/796822268 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f075c1a5e50 con 0x7f075c102d50 2024-05-10T11:21:05.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.874+0000 7f075affd700 1 --2- 172.21.15.100:0/796822268 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f075c102390 0x7f075c19a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:21:05.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.874+0000 7f0753fff700 1 --2- 172.21.15.100:0/796822268 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f075c102d50 0x7f075c19add0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f0744006010 tx=0x7f07440057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:05.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.877+0000 7f07609ee700 1 -- 172.21.15.100:0/796822268 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0744027070 con 0x7f075c102d50 2024-05-10T11:21:05.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.877+0000 7f07619f0700 1 -- 172.21.15.100:0/796822268 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f075c1a6080 con 0x7f075c102d50 2024-05-10T11:21:05.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.877+0000 7f07609ee700 1 -- 172.21.15.100:0/796822268 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0744022470 con 0x7f075c102d50 2024-05-10T11:21:05.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.877+0000 7f07609ee700 1 -- 172.21.15.100:0/796822268 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0744017650 con 0x7f075c102d50 2024-05-10T11:21:05.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.877+0000 7f07619f0700 1 -- 172.21.15.100:0/796822268 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f075c1a6530 con 0x7f075c102d50 2024-05-10T11:21:05.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.879+0000 7f07609ee700 1 -- 172.21.15.100:0/796822268 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f074402a070 con 0x7f075c102d50 2024-05-10T11:21:05.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.880+0000 7f07609ee700 1 --2- 172.21.15.100:0/796822268 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f07480b2880 0x7f07480b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:05.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.880+0000 7f075affd700 1 --2- 172.21.15.100:0/796822268 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f07480b2880 0x7f07480b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:05.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.880+0000 7f07609ee700 1 -- 172.21.15.100:0/796822268 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f07440e34c0 con 0x7f075c102d50 2024-05-10T11:21:05.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.881+0000 7f075affd700 1 --2- 172.21.15.100:0/796822268 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f07480b2880 0x7f07480b4d40 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f074c009fd0 tx=0x7f074c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:05.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.881+0000 7f076c283b80 1 Processor -- start 2024-05-10T11:21:05.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.881+0000 7f076c283b80 1 -- start start 2024-05-10T11:21:05.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.881+0000 7f076c283b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56336f4e2230 0x56336f4e2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:05.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.881+0000 7f076c283b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56336f4e2b50 0x56336f4dc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:05.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.881+0000 7f076c283b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56336f4dcdd0 0x56336f4dd210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:05.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.881+0000 7f076c283b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56336f42e9c0 con 0x56336f4e2230 2024-05-10T11:21:05.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.881+0000 7f076c283b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56336f393e20 con 0x56336f4dcdd0 2024-05-10T11:21:05.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.881+0000 7f076c283b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56336f42a980 con 0x56336f4e2b50 2024-05-10T11:21:05.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.881+0000 7f075affd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56336f4e2230 0x56336f4e2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:05.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.881+0000 7f075affd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56336f4e2230 0x56336f4e2610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51516/0 (socket says 172.21.15.100:51516) 2024-05-10T11:21:05.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.881+0000 7f075affd700 1 -- 172.21.15.100:0/3684272116 learned_addr learned my addr 172.21.15.100:0/3684272116 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:21:05.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.881+0000 7f075b7fe700 1 --2- 172.21.15.100:0/3684272116 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56336f4dcdd0 0x56336f4dd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:05.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.881+0000 7f0753fff700 1 --2- 172.21.15.100:0/3684272116 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56336f4e2b50 0x56336f4dc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:05.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.882+0000 7f075affd700 1 -- 172.21.15.100:0/3684272116 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56336f4e2b50 msgr2=0x56336f4dc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:05.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.882+0000 7f075affd700 1 --2- 172.21.15.100:0/3684272116 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56336f4e2b50 0x56336f4dc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:05.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.882+0000 7f075affd700 1 -- 172.21.15.100:0/3684272116 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56336f4dcdd0 msgr2=0x56336f4dd210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:05.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.882+0000 7f075affd700 1 --2- 172.21.15.100:0/3684272116 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56336f4dcdd0 0x56336f4dd210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:05.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.882+0000 7f075affd700 1 -- 172.21.15.100:0/3684272116 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56336f37b950 con 0x56336f4e2230 2024-05-10T11:21:05.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.882+0000 7f075b7fe700 1 --2- 172.21.15.100:0/3684272116 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56336f4dcdd0 0x56336f4dd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:21:05.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.882+0000 7f075affd700 1 --2- 172.21.15.100:0/3684272116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56336f4e2230 0x56336f4e2610 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f074c00e720 tx=0x7f074c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c062bb5e2277511 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:05.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.882+0000 7f0751ffb700 1 -- 172.21.15.100:0/3684272116 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074c026600 con 0x56336f4e2230 2024-05-10T11:21:05.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.882+0000 7f0751ffb700 1 -- 172.21.15.100:0/3684272116 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f074c026c30 con 0x56336f4e2230 2024-05-10T11:21:05.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.882+0000 7f0751ffb700 1 -- 172.21.15.100:0/3684272116 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074c01ec10 con 0x56336f4e2230 2024-05-10T11:21:05.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.883+0000 7f076c283b80 1 -- 172.21.15.100:0/3684272116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56336f4e2230 msgr2=0x56336f4e2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:05.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.883+0000 7f076c283b80 1 --2- 172.21.15.100:0/3684272116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56336f4e2230 0x56336f4e2610 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f074c00e720 tx=0x7f074c014000 comp rx=0 tx=0).stop 2024-05-10T11:21:05.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.883+0000 7f076c283b80 1 -- 172.21.15.100:0/3684272116 shutdown_connections 2024-05-10T11:21:05.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.883+0000 7f076c283b80 1 --2- 172.21.15.100:0/3684272116 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56336f4dcdd0 0x56336f4dd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:05.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.883+0000 7f076c283b80 1 --2- 172.21.15.100:0/3684272116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56336f4e2230 0x56336f4e2610 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:05.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.883+0000 7f076c283b80 1 --2- 172.21.15.100:0/3684272116 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56336f4e2b50 0x56336f4dc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:05.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.883+0000 7f076c283b80 1 -- 172.21.15.100:0/3684272116 >> 172.21.15.100:0/3684272116 conn(0x56336f408bf0 msgr2=0x56336f4d7320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:05.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.883+0000 7f076c283b80 1 -- 172.21.15.100:0/3684272116 shutdown_connections 2024-05-10T11:21:05.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.883+0000 7f076c283b80 1 -- 172.21.15.100:0/3684272116 wait complete. 2024-05-10T11:21:05.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.883+0000 7f076c283b80 1 -- start start 2024-05-10T11:21:05.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.884+0000 7f076c283b80 1 Processor -- start 2024-05-10T11:21:05.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.884+0000 7f076c283b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56336f4dcdd0 0x56336f4e8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:05.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.884+0000 7f076c283b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56336f4e2230 0x56336f4e5b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:05.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.884+0000 7f076c283b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56336f4e2b50 0x56336f4e6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:05.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.884+0000 7f076c283b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56336f4e94d0 con 0x56336f4e2b50 2024-05-10T11:21:05.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.884+0000 7f076c283b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56336f3e7f50 con 0x56336f4dcdd0 2024-05-10T11:21:05.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.884+0000 7f076c283b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56336f4e9650 con 0x56336f4e2230 2024-05-10T11:21:05.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.884+0000 7f076c283b80 10 client.0 _ll_register_callbacks cb 0x56336f3c2630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:21:05.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.884+0000 7f075b7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56336f4e2b50 0x56336f4e6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:05.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.884+0000 7f075b7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56336f4e2b50 0x56336f4e6060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51518/0 (socket says 172.21.15.100:51518) 2024-05-10T11:21:05.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.884+0000 7f075b7fe700 1 -- 172.21.15.100:0/2667036282 learned_addr learned my addr 172.21.15.100:0/2667036282 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:21:05.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.884+0000 7f075affd700 1 --2- 172.21.15.100:0/2667036282 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56336f4dcdd0 0x56336f4e8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:05.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.884+0000 7f0753fff700 1 --2- 172.21.15.100:0/2667036282 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56336f4e2230 0x56336f4e5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:05.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.884+0000 7f075b7fe700 1 -- 172.21.15.100:0/2667036282 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56336f4e2230 msgr2=0x56336f4e5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:05.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.884+0000 7f075b7fe700 1 --2- 172.21.15.100:0/2667036282 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56336f4e2230 0x56336f4e5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:05.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.884+0000 7f075b7fe700 1 -- 172.21.15.100:0/2667036282 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56336f4dcdd0 msgr2=0x56336f4e8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:05.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.884+0000 7f075b7fe700 1 --2- 172.21.15.100:0/2667036282 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56336f4dcdd0 0x56336f4e8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:05.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.884+0000 7f075b7fe700 1 -- 172.21.15.100:0/2667036282 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56336f3e57e0 con 0x56336f4e2b50 2024-05-10T11:21:05.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.884+0000 7f075affd700 1 --2- 172.21.15.100:0/2667036282 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56336f4dcdd0 0x56336f4e8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:21:05.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.884+0000 7f075b7fe700 1 --2- 172.21.15.100:0/2667036282 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56336f4e2b50 0x56336f4e6060 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f07540042f0 tx=0x7f0754004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ac0cce1c494f78d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:05.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.884+0000 7f073affd700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:21:05.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.885+0000 7f073affd700 1 -- 172.21.15.100:0/2667036282 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f075401a070 con 0x56336f4e2b50 2024-05-10T11:21:05.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.885+0000 7f073affd700 1 -- 172.21.15.100:0/2667036282 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f0754007700 con 0x56336f4e2b50 2024-05-10T11:21:05.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.885+0000 7f076c283b80 1 -- 172.21.15.100:0/2667036282 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f074c026c30 con 0x56336f4e2b50 2024-05-10T11:21:05.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.885+0000 7f073affd700 1 -- 172.21.15.100:0/2667036282 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0754026050 con 0x56336f4e2b50 2024-05-10T11:21:05.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.885+0000 7f076c283b80 10 client.15906 fetch_fsmap learned FSMap version 29 2024-05-10T11:21:05.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.885+0000 7f076c283b80 1 -- 172.21.15.100:0/2667036282 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56336f4f3da0 con 0x56336f4e2b50 2024-05-10T11:21:05.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.885+0000 7f073affd700 1 -- 172.21.15.100:0/2667036282 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f074c026c30 con 0x56336f4e2b50 2024-05-10T11:21:05.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.885+0000 7f073affd700 1 -- 172.21.15.100:0/2667036282 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f0754016a20 con 0x56336f4e2b50 2024-05-10T11:21:05.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.885+0000 7f076c283b80 10 client.15906 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T11:21:05.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.885+0000 7f076c283b80 10 client.15906 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:21:05.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.885+0000 7f076c283b80 20 client.15906 populate_metadata read hostname 'smithi100' 2024-05-10T11:21:05.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.885+0000 7f076c283b80 1 --2- 172.21.15.100:0/2667036282 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56336f4f6380 0x56336f4f8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:05.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.885+0000 7f076c283b80 4 client.15906 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:21:05.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.885+0000 7f076c283b80 1 -- 172.21.15.100:0/2667036282 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56336f43c3a0 con 0x56336f4f6380 2024-05-10T11:21:05.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.885+0000 7f075affd700 1 --2- 172.21.15.100:0/2667036282 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56336f4f6380 0x56336f4f8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:05.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.886+0000 7f075affd700 1 --2- 172.21.15.100:0/2667036282 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56336f4f6380 0x56336f4f8770 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79f9b20e084c6768 server_cookie=2f1966a59cdf0ff3 in_seq=0 out_seq=0 2024-05-10T11:21:05.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.886+0000 7f073affd700 10 client.15906 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:21:05.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.886+0000 7f073affd700 1 -- 172.21.15.100:0/2667036282 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56336f43c3a0 con 0x56336f4f6380 2024-05-10T11:21:05.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.886+0000 7f073affd700 10 client.15906 handle_command_reply: tid=0 2024-05-10T11:21:05.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.887+0000 7f076c283b80 1 client.15906 shutdown 2024-05-10T11:21:05.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.887+0000 7f076c283b80 1 -- 172.21.15.100:0/2667036282 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56336f4e2b50 msgr2=0x56336f4e6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:05.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.887+0000 7f076c283b80 1 --2- 172.21.15.100:0/2667036282 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56336f4e2b50 0x56336f4e6060 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f07540042f0 tx=0x7f0754004320 comp rx=0 tx=0).stop 2024-05-10T11:21:05.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.887+0000 7f076c283b80 1 -- 172.21.15.100:0/2667036282 shutdown_connections 2024-05-10T11:21:05.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.887+0000 7f076c283b80 1 --2- 172.21.15.100:0/2667036282 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56336f4f6380 0x56336f4f8770 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:05.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.887+0000 7f076c283b80 1 --2- 172.21.15.100:0/2667036282 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56336f4e2b50 0x56336f4e6060 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:05.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.887+0000 7f076c283b80 1 --2- 172.21.15.100:0/2667036282 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56336f4dcdd0 0x56336f4e8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:05.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.887+0000 7f073affd700 0 client.15906 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:21:05.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.887+0000 7f076c283b80 1 --2- 172.21.15.100:0/2667036282 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56336f4e2230 0x56336f4e5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:05.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.887+0000 7f076c283b80 1 -- 172.21.15.100:0/2667036282 >> 172.21.15.100:0/2667036282 conn(0x56336f408bf0 msgr2=0x56336f4d1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:05.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.887+0000 7f076c283b80 1 -- 172.21.15.100:0/2667036282 shutdown_connections 2024-05-10T11:21:05.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.887+0000 7f076c283b80 1 -- 172.21.15.100:0/2667036282 wait complete. 2024-05-10T11:21:05.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.887+0000 7f076c283b80 20 client.15906 trim_cache size 0 max 16384 2024-05-10T11:21:05.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.908+0000 7f076c283b80 1 Processor -- start 2024-05-10T11:21:05.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.908+0000 7f076c283b80 1 -- start start 2024-05-10T11:21:05.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.908+0000 7f076c283b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56336f4dcdd0 0x56336f4f2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:05.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.908+0000 7f076c283b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56336f4e2230 0x56336f4e5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:05.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.908+0000 7f076c283b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56336f4e2b50 0x56336f4d1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:05.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.908+0000 7f076c283b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f075c1b1910 con 0x56336f4dcdd0 2024-05-10T11:21:05.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.908+0000 7f076c283b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f075c1af1a0 con 0x56336f4e2230 2024-05-10T11:21:05.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.908+0000 7f076c283b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f075c1acff0 con 0x56336f4e2b50 2024-05-10T11:21:05.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.909+0000 7f075affd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56336f4dcdd0 0x56336f4f2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:05.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.909+0000 7f0753fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56336f4e2230 0x56336f4e5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:05.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.909+0000 7f075b7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56336f4e2b50 0x56336f4d1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:05.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.909+0000 7f0753fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56336f4e2230 0x56336f4e5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:36486/0 (socket says 172.21.15.100:36486) 2024-05-10T11:21:05.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.909+0000 7f075affd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56336f4dcdd0 0x56336f4f2560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51520/0 (socket says 172.21.15.100:51520) 2024-05-10T11:21:05.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.909+0000 7f0753fff700 1 -- 172.21.15.100:0/2531289425 learned_addr learned my addr 172.21.15.100:0/2531289425 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:21:05.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.909+0000 7f0753fff700 1 -- 172.21.15.100:0/2531289425 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56336f4e2b50 msgr2=0x56336f4d1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:05.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.909+0000 7f0753fff700 1 --2- 172.21.15.100:0/2531289425 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56336f4e2b50 0x56336f4d1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:05.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.909+0000 7f0753fff700 1 -- 172.21.15.100:0/2531289425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56336f4dcdd0 msgr2=0x56336f4f2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:05.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.909+0000 7f0753fff700 1 --2- 172.21.15.100:0/2531289425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56336f4dcdd0 0x56336f4f2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:05.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.909+0000 7f0753fff700 1 -- 172.21.15.100:0/2531289425 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56336f4f8df0 con 0x56336f4e2230 2024-05-10T11:21:05.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.909+0000 7f0753fff700 1 --2- 172.21.15.100:0/2531289425 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56336f4e2230 0x56336f4e5420 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f07440e0c60 tx=0x7f07440b1040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de1abbda61de90f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:05.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.909+0000 7f07517fa700 1 -- 172.21.15.100:0/2531289425 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07440aaf10 con 0x56336f4e2230 2024-05-10T11:21:05.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.909+0000 7f07517fa700 1 -- 172.21.15.100:0/2531289425 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f07440aa810 con 0x56336f4e2230 2024-05-10T11:21:05.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.909+0000 7f07517fa700 1 -- 172.21.15.100:0/2531289425 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07440aacd0 con 0x56336f4e2230 2024-05-10T11:21:05.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.910+0000 7f076c283b80 1 -- 172.21.15.100:0/2531289425 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56336f4e2230 msgr2=0x56336f4e5420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:05.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.910+0000 7f076c283b80 1 --2- 172.21.15.100:0/2531289425 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56336f4e2230 0x56336f4e5420 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f07440e0c60 tx=0x7f07440b1040 comp rx=0 tx=0).stop 2024-05-10T11:21:05.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.910+0000 7f076c283b80 1 -- 172.21.15.100:0/2531289425 shutdown_connections 2024-05-10T11:21:05.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.910+0000 7f076c283b80 1 --2- 172.21.15.100:0/2531289425 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56336f4e2b50 0x56336f4d1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:05.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.910+0000 7f076c283b80 1 --2- 172.21.15.100:0/2531289425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56336f4dcdd0 0x56336f4f2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:05.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.910+0000 7f076c283b80 1 --2- 172.21.15.100:0/2531289425 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56336f4e2230 0x56336f4e5420 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:05.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.910+0000 7f076c283b80 1 -- 172.21.15.100:0/2531289425 >> 172.21.15.100:0/2531289425 conn(0x56336f408bf0 msgr2=0x56336f4e7a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:05.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.910+0000 7f076c283b80 1 -- 172.21.15.100:0/2531289425 shutdown_connections 2024-05-10T11:21:05.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.910+0000 7f076c283b80 1 -- 172.21.15.100:0/2531289425 wait complete. 2024-05-10T11:21:05.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.910+0000 7f076c283b80 1 -- start start 2024-05-10T11:21:05.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.911+0000 7f076c283b80 1 Processor -- start 2024-05-10T11:21:05.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.911+0000 7f076c283b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56336f4dcdd0 0x56336f4eeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:05.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.911+0000 7f076c283b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56336f4e2230 0x56336f55bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:05.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.911+0000 7f076c283b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56336f4e2b50 0x56336f55c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:05.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.911+0000 7f076c283b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f075c1ad660 con 0x56336f4dcdd0 2024-05-10T11:21:05.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.911+0000 7f076c283b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f075c1aead0 con 0x56336f4e2230 2024-05-10T11:21:05.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.911+0000 7f075affd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56336f4dcdd0 0x56336f4eeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:05.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.911+0000 7f076c283b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f075c1add30 con 0x56336f4e2b50 2024-05-10T11:21:05.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.911+0000 7f0753fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56336f4e2230 0x56336f55bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:05.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.911+0000 7f075affd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56336f4dcdd0 0x56336f4eeb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51536/0 (socket says 172.21.15.100:51536) 2024-05-10T11:21:05.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.911+0000 7f075affd700 1 -- 172.21.15.100:0/4273389766 learned_addr learned my addr 172.21.15.100:0/4273389766 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:21:05.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.911+0000 7f075b7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56336f4e2b50 0x56336f55c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:05.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.911+0000 7f076c283b80 10 client.0 _ll_register_callbacks cb 0x56336f3c2630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:21:05.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.911+0000 7f0753fff700 1 -- 172.21.15.100:0/4273389766 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56336f4e2b50 msgr2=0x56336f55c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:05.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.911+0000 7f0753fff700 1 --2- 172.21.15.100:0/4273389766 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56336f4e2b50 0x56336f55c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:05.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.911+0000 7f0753fff700 1 -- 172.21.15.100:0/4273389766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56336f4dcdd0 msgr2=0x56336f4eeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:05.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.911+0000 7f0753fff700 1 --2- 172.21.15.100:0/4273389766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56336f4dcdd0 0x56336f4eeb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:05.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.911+0000 7f0753fff700 1 -- 172.21.15.100:0/4273389766 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56336f55c6c0 con 0x56336f4e2230 2024-05-10T11:21:05.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.911+0000 7f075b7fe700 1 --2- 172.21.15.100:0/4273389766 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56336f4e2b50 0x56336f55c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:21:05.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.911+0000 7f075affd700 1 --2- 172.21.15.100:0/4273389766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56336f4dcdd0 0x56336f4eeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:21:05.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.912+0000 7f0753fff700 1 --2- 172.21.15.100:0/4273389766 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56336f4e2230 0x56336f55bc40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x56336f4d3d70 tx=0x7f07440096f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2bdd79010326848 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:05.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.912+0000 7f073affd700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:21:05.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.912+0000 7f073affd700 1 -- 172.21.15.100:0/4273389766 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07440aaf10 con 0x56336f4e2230 2024-05-10T11:21:05.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.912+0000 7f073affd700 1 -- 172.21.15.100:0/4273389766 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f07440e3260 con 0x56336f4e2230 2024-05-10T11:21:05.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.912+0000 7f076c283b80 1 -- 172.21.15.100:0/4273389766 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07440aa810 con 0x56336f4e2230 2024-05-10T11:21:05.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.912+0000 7f073affd700 1 -- 172.21.15.100:0/4273389766 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f07440aebc0 con 0x56336f4e2230 2024-05-10T11:21:05.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.912+0000 7f076c283b80 10 client.25412 fetch_fsmap learned FSMap version 29 2024-05-10T11:21:05.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.912+0000 7f076c283b80 1 -- 172.21.15.100:0/4273389766 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56336f55b970 con 0x56336f4e2230 2024-05-10T11:21:05.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.912+0000 7f073affd700 1 -- 172.21.15.100:0/4273389766 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f07440aa810 con 0x56336f4e2230 2024-05-10T11:21:05.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.912+0000 7f073affd700 1 -- 172.21.15.100:0/4273389766 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f07440b3150 con 0x56336f4e2230 2024-05-10T11:21:05.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.912+0000 7f076c283b80 10 client.25412 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T11:21:05.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.912+0000 7f076c283b80 10 client.25412 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:21:05.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.912+0000 7f076c283b80 20 client.25412 populate_metadata read hostname 'smithi100' 2024-05-10T11:21:05.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.912+0000 7f076c283b80 1 --2- 172.21.15.100:0/4273389766 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56336f4edcb0 0x56336f4ee0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:05.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.912+0000 7f076c283b80 4 client.25412 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T11:21:05.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.912+0000 7f076c283b80 1 -- 172.21.15.100:0/4273389766 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56336f436440 con 0x56336f4edcb0 2024-05-10T11:21:05.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.913+0000 7f075affd700 1 --2- 172.21.15.100:0/4273389766 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56336f4edcb0 0x56336f4ee0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:05.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.913+0000 7f073affd700 10 client.25412 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:21:05.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.913+0000 7f075affd700 1 --2- 172.21.15.100:0/4273389766 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56336f4edcb0 0x56336f4ee0c0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12ada449c143b034 server_cookie=721c0406d22a7c51 in_seq=0 out_seq=0 2024-05-10T11:21:05.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.914+0000 7f073affd700 1 -- 172.21.15.100:0/4273389766 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56336f436440 con 0x56336f4edcb0 2024-05-10T11:21:05.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.914+0000 7f073affd700 10 client.25412 handle_command_reply: tid=0 2024-05-10T11:21:05.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.914+0000 7f076c283b80 1 client.25412 shutdown 2024-05-10T11:21:05.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.914+0000 7f076c283b80 1 -- 172.21.15.100:0/4273389766 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56336f4e2230 msgr2=0x56336f55bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:05.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.914+0000 7f076c283b80 1 --2- 172.21.15.100:0/4273389766 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56336f4e2230 0x56336f55bc40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x56336f4d3d70 tx=0x7f07440096f0 comp rx=0 tx=0).stop 2024-05-10T11:21:05.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.914+0000 7f076c283b80 1 -- 172.21.15.100:0/4273389766 shutdown_connections 2024-05-10T11:21:05.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.914+0000 7f076c283b80 1 --2- 172.21.15.100:0/4273389766 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56336f4edcb0 0x56336f4ee0c0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:05.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.914+0000 7f076c283b80 1 --2- 172.21.15.100:0/4273389766 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56336f4e2b50 0x56336f55c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:05.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.914+0000 7f076c283b80 1 --2- 172.21.15.100:0/4273389766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56336f4dcdd0 0x56336f4eeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:05.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.914+0000 7f076c283b80 1 --2- 172.21.15.100:0/4273389766 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56336f4e2230 0x56336f55bc40 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:05.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.914+0000 7f076c283b80 1 -- 172.21.15.100:0/4273389766 >> 172.21.15.100:0/4273389766 conn(0x56336f408bf0 msgr2=0x56336f4e0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:05.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.914+0000 7f073affd700 0 client.25412 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:21:05.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.914+0000 7f076c283b80 1 -- 172.21.15.100:0/4273389766 shutdown_connections 2024-05-10T11:21:05.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.914+0000 7f076c283b80 1 -- 172.21.15.100:0/4273389766 wait complete. 2024-05-10T11:21:05.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.914+0000 7f076c283b80 20 client.25412 trim_cache size 0 max 16384 2024-05-10T11:21:05.922 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:21:05.923 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-05-10T11:21:05.923 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "c2a03361-ab28-4939-9f7b-0e4c2a5b18d8", 2024-05-10T11:21:05.923 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-05-10T11:21:05.923 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:21:05.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.915+0000 7f07619f0700 1 -- 172.21.15.100:0/796822268 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f07480b2880 msgr2=0x7f07480b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:05.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.915+0000 7f07619f0700 1 --2- 172.21.15.100:0/796822268 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f07480b2880 0x7f07480b4d40 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f074c009fd0 tx=0x7f074c009260 comp rx=0 tx=0).stop 2024-05-10T11:21:05.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.915+0000 7f07619f0700 1 -- 172.21.15.100:0/796822268 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f075c102d50 msgr2=0x7f075c19add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:05.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.915+0000 7f07619f0700 1 --2- 172.21.15.100:0/796822268 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f075c102d50 0x7f075c19add0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f0744006010 tx=0x7f07440057c0 comp rx=0 tx=0).stop 2024-05-10T11:21:05.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.915+0000 7f07619f0700 1 -- 172.21.15.100:0/796822268 shutdown_connections 2024-05-10T11:21:05.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.915+0000 7f07619f0700 1 --2- 172.21.15.100:0/796822268 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f07480b2880 0x7f07480b4d40 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:05.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.915+0000 7f07619f0700 1 --2- 172.21.15.100:0/796822268 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f075c108400 0x7f075c1a5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:05.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.915+0000 7f07619f0700 1 --2- 172.21.15.100:0/796822268 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f075c102390 0x7f075c19a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:05.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.915+0000 7f07619f0700 1 --2- 172.21.15.100:0/796822268 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f075c102d50 0x7f075c19add0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:05.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.915+0000 7f07619f0700 1 -- 172.21.15.100:0/796822268 >> 172.21.15.100:0/796822268 conn(0x7f075c079310 msgr2=0x7f075c1010c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:05.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.915+0000 7f07619f0700 1 -- 172.21.15.100:0/796822268 shutdown_connections 2024-05-10T11:21:05.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:05.915+0000 7f07619f0700 1 -- 172.21.15.100:0/796822268 wait complete. 2024-05-10T11:21:05.933 INFO:tasks.workunit.client.0.smithi100.stdout: 1 32375 3.22 MB/sec warmup 60 sec latency 20.664 ms 2024-05-10T11:21:06.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:21:05 smithi100 ceph-mon[30296]: pgmap v501: 129 pgs: 129 active+clean; 32 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 1.2 MiB/s wr, 127 op/s 2024-05-10T11:21:06.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:21:05 smithi178 ceph-mon[33359]: pgmap v501: 129 pgs: 129 active+clean; 32 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 1.2 MiB/s wr, 127 op/s 2024-05-10T11:21:06.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:21:05 smithi188 ceph-mon[34075]: pgmap v501: 129 pgs: 129 active+clean; 32 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 1.2 MiB/s wr, 127 op/s 2024-05-10T11:21:06.693 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T11:21:06.933 INFO:tasks.workunit.client.0.smithi100.stdout: 1 32756 3.17 MB/sec warmup 61 sec latency 16.054 ms 2024-05-10T11:21:06.967 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:21:07.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:21:06 smithi100 ceph-mon[30296]: scrub queued for path: / 2024-05-10T11:21:07.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:21:06 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:21:07.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:21:06 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:21:07.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:21:06 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:21:07.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:21:06 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:21:07.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:21:06 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:21:07.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:21:06 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:21:07.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:21:06 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:21:07.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:21:06 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:21:07.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:21:06 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:21:07.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:21:06 smithi178 ceph-mon[33359]: scrub queued for path: / 2024-05-10T11:21:07.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:21:06 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:21:07.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:21:06 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:21:07.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:21:06 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:21:07.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:21:06 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:21:07.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:21:06 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:21:07.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:21:06 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:21:07.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:21:06 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:21:07.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:21:06 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:21:07.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:21:06 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:21:07.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:21:06 smithi188 ceph-mon[34075]: scrub queued for path: / 2024-05-10T11:21:07.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:21:06 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:21:07.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:21:06 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:21:07.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:21:06 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:21:07.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:21:06 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:21:07.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:21:06 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:21:07.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:21:06 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:21:07.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:21:06 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:21:07.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:21:06 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:21:07.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:21:06 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:21:07.933 INFO:tasks.workunit.client.0.smithi100.stdout: 1 33347 3.20 MB/sec warmup 62 sec latency 15.421 ms 2024-05-10T11:21:08.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:21:07 smithi100 ceph-mon[30296]: pgmap v502: 129 pgs: 129 active+clean; 32 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 1.4 MiB/s wr, 170 op/s 2024-05-10T11:21:08.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:21:07 smithi178 ceph-mon[33359]: pgmap v502: 129 pgs: 129 active+clean; 32 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 1.4 MiB/s wr, 170 op/s 2024-05-10T11:21:08.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:21:07 smithi188 ceph-mon[34075]: pgmap v502: 129 pgs: 129 active+clean; 32 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 1.4 MiB/s wr, 170 op/s 2024-05-10T11:21:08.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.777+0000 7f85b4807700 1 -- 172.21.15.100:0/1438140222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85b01023a0 msgr2=0x7f85b0102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:08.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.777+0000 7f85b4807700 1 --2- 172.21.15.100:0/1438140222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85b01023a0 0x7f85b0102820 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f85a4009990 tx=0x7f85a4009ca0 comp rx=0 tx=0).stop 2024-05-10T11:21:08.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.777+0000 7f85b4807700 1 -- 172.21.15.100:0/1438140222 shutdown_connections 2024-05-10T11:21:08.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.777+0000 7f85b4807700 1 --2- 172.21.15.100:0/1438140222 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85b0102d60 0x7f85b010f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:08.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.777+0000 7f85b4807700 1 --2- 172.21.15.100:0/1438140222 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85b0108410 0x7f85b01087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:08.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.777+0000 7f85b4807700 1 --2- 172.21.15.100:0/1438140222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85b01023a0 0x7f85b0102820 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:08.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.777+0000 7f85b4807700 1 -- 172.21.15.100:0/1438140222 >> 172.21.15.100:0/1438140222 conn(0x7f85b0079350 msgr2=0x7f85b0079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:08.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.777+0000 7f85b4807700 1 -- 172.21.15.100:0/1438140222 shutdown_connections 2024-05-10T11:21:08.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.777+0000 7f85b4807700 1 -- 172.21.15.100:0/1438140222 wait complete. 2024-05-10T11:21:08.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.778+0000 7f85b4807700 1 Processor -- start 2024-05-10T11:21:08.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.778+0000 7f85b4807700 1 -- start start 2024-05-10T11:21:08.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.778+0000 7f85b4807700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85b01023a0 0x7f85b0104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:08.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.778+0000 7f85b4807700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85b0102d60 0x7f85b01045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:08.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.778+0000 7f85b4807700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85b0108410 0x7f85b0104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:08.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.779+0000 7f85b4807700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f85b0112400 con 0x7f85b0108410 2024-05-10T11:21:08.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.779+0000 7f85b4807700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f85b0112280 con 0x7f85b01023a0 2024-05-10T11:21:08.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.779+0000 7f85b4807700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f85b0112580 con 0x7f85b0102d60 2024-05-10T11:21:08.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.779+0000 7f85add9b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85b0102d60 0x7f85b01045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:08.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.779+0000 7f85aed9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85b0108410 0x7f85b0104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:08.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.779+0000 7f85add9b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85b0102d60 0x7f85b01045c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:60792/0 (socket says 172.21.15.100:60792) 2024-05-10T11:21:08.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.779+0000 7f85aed9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85b0108410 0x7f85b0104b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53582/0 (socket says 172.21.15.100:53582) 2024-05-10T11:21:08.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.779+0000 7f85add9b700 1 -- 172.21.15.100:0/2684125628 learned_addr learned my addr 172.21.15.100:0/2684125628 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:21:08.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.779+0000 7f85ae59c700 1 --2- 172.21.15.100:0/2684125628 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85b01023a0 0x7f85b0104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:08.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.779+0000 7f85aed9d700 1 -- 172.21.15.100:0/2684125628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85b0102d60 msgr2=0x7f85b01045c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:08.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.779+0000 7f85aed9d700 1 --2- 172.21.15.100:0/2684125628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85b0102d60 0x7f85b01045c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:08.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.779+0000 7f85aed9d700 1 -- 172.21.15.100:0/2684125628 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85b01023a0 msgr2=0x7f85b0104080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:08.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.779+0000 7f85aed9d700 1 --2- 172.21.15.100:0/2684125628 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85b01023a0 0x7f85b0104080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:08.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.779+0000 7f85aed9d700 1 -- 172.21.15.100:0/2684125628 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85b01a7fd0 con 0x7f85b0108410 2024-05-10T11:21:08.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.779+0000 7f85add9b700 1 --2- 172.21.15.100:0/2684125628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85b0102d60 0x7f85b01045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:21:08.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.779+0000 7f85ae59c700 1 --2- 172.21.15.100:0/2684125628 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85b01023a0 0x7f85b0104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:21:08.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.779+0000 7f85aed9d700 1 --2- 172.21.15.100:0/2684125628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85b0108410 0x7f85b0104b00 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f85a800efc0 tx=0x7f85a800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:08.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.779+0000 7f859f7fe700 1 -- 172.21.15.100:0/2684125628 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a80092e0 con 0x7f85b0108410 2024-05-10T11:21:08.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.779+0000 7f859f7fe700 1 -- 172.21.15.100:0/2684125628 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f85a8004bb0 con 0x7f85b0108410 2024-05-10T11:21:08.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.779+0000 7f859f7fe700 1 -- 172.21.15.100:0/2684125628 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a8004020 con 0x7f85b0108410 2024-05-10T11:21:08.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.779+0000 7f85b4807700 1 -- 172.21.15.100:0/2684125628 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85b01a8260 con 0x7f85b0108410 2024-05-10T11:21:08.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.779+0000 7f85b4807700 1 -- 172.21.15.100:0/2684125628 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85b01a8770 con 0x7f85b0108410 2024-05-10T11:21:08.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.782+0000 7f859f7fe700 1 -- 172.21.15.100:0/2684125628 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f85a801b030 con 0x7f85b0108410 2024-05-10T11:21:08.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.783+0000 7f859f7fe700 1 --2- 172.21.15.100:0/2684125628 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f85980b2900 0x7f85980b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:08.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.783+0000 7f85ae59c700 1 --2- 172.21.15.100:0/2684125628 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f85980b2900 0x7f85980b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:08.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.783+0000 7f859f7fe700 1 -- 172.21.15.100:0/2684125628 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f85a80e1fd0 con 0x7f85b0108410 2024-05-10T11:21:08.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.783+0000 7f85ae59c700 1 --2- 172.21.15.100:0/2684125628 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f85980b2900 0x7f85980b4dc0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f85a0009fd0 tx=0x7f85a0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:08.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.784+0000 7f85bf09ab80 1 Processor -- start 2024-05-10T11:21:08.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.784+0000 7f85bf09ab80 1 -- start start 2024-05-10T11:21:08.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.784+0000 7f85bf09ab80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56223afc24d0 0x56223afc28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:08.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.784+0000 7f85bf09ab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56223afc2df0 0x56223afc66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:08.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.784+0000 7f85bf09ab80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56223afc6c80 0x56223afc7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:08.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.784+0000 7f85bf09ab80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56223af14860 con 0x56223afc6c80 2024-05-10T11:21:08.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.784+0000 7f85bf09ab80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56223aeb37c0 con 0x56223afc24d0 2024-05-10T11:21:08.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.784+0000 7f85bf09ab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56223af10820 con 0x56223afc2df0 2024-05-10T11:21:08.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.784+0000 7f85ae59c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56223afc24d0 0x56223afc28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:08.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.784+0000 7f85ae59c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56223afc24d0 0x56223afc28b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:55556/0 (socket says 172.21.15.100:55556) 2024-05-10T11:21:08.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.784+0000 7f85ae59c700 1 -- 172.21.15.100:0/942742403 learned_addr learned my addr 172.21.15.100:0/942742403 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:21:08.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.784+0000 7f85aed9d700 1 --2- 172.21.15.100:0/942742403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56223afc6c80 0x56223afc7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:08.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.784+0000 7f85add9b700 1 --2- 172.21.15.100:0/942742403 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56223afc2df0 0x56223afc66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:08.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.785+0000 7f85ae59c700 1 -- 172.21.15.100:0/942742403 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56223afc2df0 msgr2=0x56223afc66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:08.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.785+0000 7f85ae59c700 1 --2- 172.21.15.100:0/942742403 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56223afc2df0 0x56223afc66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:08.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.785+0000 7f85ae59c700 1 -- 172.21.15.100:0/942742403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56223afc6c80 msgr2=0x56223afc7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:08.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.785+0000 7f85ae59c700 1 --2- 172.21.15.100:0/942742403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56223afc6c80 0x56223afc7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:08.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.785+0000 7f85ae59c700 1 -- 172.21.15.100:0/942742403 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56223ae618d0 con 0x56223afc24d0 2024-05-10T11:21:08.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.785+0000 7f85ae59c700 1 --2- 172.21.15.100:0/942742403 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56223afc24d0 0x56223afc28b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f85a00043e0 tx=0x7f85a0014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3979758cc443e003 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:08.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.785+0000 7f859cff9700 1 -- 172.21.15.100:0/942742403 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a0025d90 con 0x56223afc24d0 2024-05-10T11:21:08.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.785+0000 7f859cff9700 1 -- 172.21.15.100:0/942742403 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f85a0029070 con 0x56223afc24d0 2024-05-10T11:21:08.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.785+0000 7f859cff9700 1 -- 172.21.15.100:0/942742403 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a000ed90 con 0x56223afc24d0 2024-05-10T11:21:08.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.786+0000 7f85bf09ab80 1 -- 172.21.15.100:0/942742403 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56223afc24d0 msgr2=0x56223afc28b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:08.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.786+0000 7f85bf09ab80 1 --2- 172.21.15.100:0/942742403 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56223afc24d0 0x56223afc28b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f85a00043e0 tx=0x7f85a0014000 comp rx=0 tx=0).stop 2024-05-10T11:21:08.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.786+0000 7f85bf09ab80 1 -- 172.21.15.100:0/942742403 shutdown_connections 2024-05-10T11:21:08.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.786+0000 7f85bf09ab80 1 --2- 172.21.15.100:0/942742403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56223afc6c80 0x56223afc7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:08.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.786+0000 7f85bf09ab80 1 --2- 172.21.15.100:0/942742403 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56223afc24d0 0x56223afc28b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:08.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.786+0000 7f85bf09ab80 1 --2- 172.21.15.100:0/942742403 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56223afc2df0 0x56223afc66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:08.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.786+0000 7f85bf09ab80 1 -- 172.21.15.100:0/942742403 >> 172.21.15.100:0/942742403 conn(0x56223aeee100 msgr2=0x56223afbcc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:08.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.786+0000 7f85bf09ab80 1 -- 172.21.15.100:0/942742403 shutdown_connections 2024-05-10T11:21:08.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.786+0000 7f85bf09ab80 1 -- 172.21.15.100:0/942742403 wait complete. 2024-05-10T11:21:08.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.786+0000 7f85bf09ab80 1 -- start start 2024-05-10T11:21:08.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.786+0000 7f85bf09ab80 1 Processor -- start 2024-05-10T11:21:08.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.787+0000 7f85bf09ab80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56223afc24d0 0x56223afce090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:08.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.787+0000 7f85bf09ab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56223afc2df0 0x56223afcb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:08.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.787+0000 7f85ae59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56223afc24d0 0x56223afce090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:08.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.787+0000 7f85ae59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56223afc24d0 0x56223afce090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53596/0 (socket says 172.21.15.100:53596) 2024-05-10T11:21:08.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.787+0000 7f85ae59c700 1 -- 172.21.15.100:0/3933449227 learned_addr learned my addr 172.21.15.100:0/3933449227 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:21:08.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.787+0000 7f85bf09ab80 1 --2- 172.21.15.100:0/3933449227 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56223afc6c80 0x56223afcb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:08.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.787+0000 7f85bf09ab80 1 -- 172.21.15.100:0/3933449227 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56223afcee90 con 0x56223afc24d0 2024-05-10T11:21:08.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.787+0000 7f85bf09ab80 1 -- 172.21.15.100:0/3933449227 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56223ae7ac30 con 0x56223afc6c80 2024-05-10T11:21:08.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.787+0000 7f85bf09ab80 1 -- 172.21.15.100:0/3933449227 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56223afcf010 con 0x56223afc2df0 2024-05-10T11:21:08.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.787+0000 7f85bf09ab80 10 client.0 _ll_register_callbacks cb 0x56223aeac920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:21:08.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.787+0000 7f85add9b700 1 --2- 172.21.15.100:0/3933449227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56223afc2df0 0x56223afcb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:08.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.787+0000 7f85aed9d700 1 --2- 172.21.15.100:0/3933449227 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56223afc6c80 0x56223afcb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:08.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.787+0000 7f85ae59c700 1 -- 172.21.15.100:0/3933449227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56223afc2df0 msgr2=0x56223afcb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:08.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.787+0000 7f85ae59c700 1 --2- 172.21.15.100:0/3933449227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56223afc2df0 0x56223afcb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:08.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.787+0000 7f85ae59c700 1 -- 172.21.15.100:0/3933449227 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56223afc6c80 msgr2=0x56223afcb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:08.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.787+0000 7f85ae59c700 1 --2- 172.21.15.100:0/3933449227 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56223afc6c80 0x56223afcb970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:08.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.787+0000 7f85ae59c700 1 -- 172.21.15.100:0/3933449227 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56223aeb0690 con 0x56223afc24d0 2024-05-10T11:21:08.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.787+0000 7f85ae59c700 1 --2- 172.21.15.100:0/3933449227 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56223afc24d0 0x56223afce090 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f85a00043b0 tx=0x7f85a0003d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ae86891d4b2a61c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:08.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.787+0000 7f8595ffb700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:21:08.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.788+0000 7f8595ffb700 1 -- 172.21.15.100:0/3933449227 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a000e7c0 con 0x56223afc24d0 2024-05-10T11:21:08.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.788+0000 7f85bf09ab80 1 -- 172.21.15.100:0/3933449227 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85a0029070 con 0x56223afc24d0 2024-05-10T11:21:08.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.788+0000 7f8595ffb700 1 -- 172.21.15.100:0/3933449227 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f85a000e990 con 0x56223afc24d0 2024-05-10T11:21:08.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.788+0000 7f8595ffb700 1 -- 172.21.15.100:0/3933449227 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f85a0043050 con 0x56223afc24d0 2024-05-10T11:21:08.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.788+0000 7f85bf09ab80 10 client.15924 fetch_fsmap learned FSMap version 29 2024-05-10T11:21:08.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.788+0000 7f85bf09ab80 1 -- 172.21.15.100:0/3933449227 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56223afd9670 con 0x56223afc24d0 2024-05-10T11:21:08.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.788+0000 7f8595ffb700 1 -- 172.21.15.100:0/3933449227 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f85a0029070 con 0x56223afc24d0 2024-05-10T11:21:08.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.788+0000 7f8595ffb700 1 -- 172.21.15.100:0/3933449227 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f85a002ec50 con 0x56223afc24d0 2024-05-10T11:21:08.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.788+0000 7f85bf09ab80 10 client.15924 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T11:21:08.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.788+0000 7f85bf09ab80 10 client.15924 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:21:08.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.788+0000 7f85bf09ab80 20 client.15924 populate_metadata read hostname 'smithi100' 2024-05-10T11:21:08.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.788+0000 7f85bf09ab80 1 --2- 172.21.15.100:0/3933449227 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56223afdbb40 0x56223afddf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:08.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.788+0000 7f85bf09ab80 4 client.15924 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:21:08.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.788+0000 7f85bf09ab80 1 -- 172.21.15.100:0/3933449227 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56223af8e650 con 0x56223afdbb40 2024-05-10T11:21:08.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.789+0000 7f85add9b700 1 --2- 172.21.15.100:0/3933449227 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56223afdbb40 0x56223afddf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:08.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.789+0000 7f8595ffb700 10 client.15924 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:21:08.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.789+0000 7f85add9b700 1 --2- 172.21.15.100:0/3933449227 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56223afdbb40 0x56223afddf30 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bf1aaabde33d5b server_cookie=15790f39a7c92acc in_seq=0 out_seq=0 2024-05-10T11:21:08.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.790+0000 7f8595ffb700 1 -- 172.21.15.100:0/3933449227 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56223af8e650 con 0x56223afdbb40 2024-05-10T11:21:08.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.790+0000 7f8595ffb700 10 client.15924 handle_command_reply: tid=0 2024-05-10T11:21:08.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.790+0000 7f85bf09ab80 1 client.15924 shutdown 2024-05-10T11:21:08.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.790+0000 7f85bf09ab80 1 -- 172.21.15.100:0/3933449227 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56223afc24d0 msgr2=0x56223afce090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:08.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.790+0000 7f85bf09ab80 1 --2- 172.21.15.100:0/3933449227 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56223afc24d0 0x56223afce090 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f85a00043b0 tx=0x7f85a0003d20 comp rx=0 tx=0).stop 2024-05-10T11:21:08.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.790+0000 7f85bf09ab80 1 -- 172.21.15.100:0/3933449227 shutdown_connections 2024-05-10T11:21:08.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.790+0000 7f85bf09ab80 1 --2- 172.21.15.100:0/3933449227 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56223afdbb40 0x56223afddf30 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:08.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.790+0000 7f85bf09ab80 1 --2- 172.21.15.100:0/3933449227 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56223afc6c80 0x56223afcb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:08.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.790+0000 7f85bf09ab80 1 --2- 172.21.15.100:0/3933449227 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56223afc24d0 0x56223afce090 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:08.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.790+0000 7f8595ffb700 0 client.15924 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:21:08.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.790+0000 7f85bf09ab80 1 --2- 172.21.15.100:0/3933449227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56223afc2df0 0x56223afcb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:08.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.790+0000 7f85bf09ab80 1 -- 172.21.15.100:0/3933449227 >> 172.21.15.100:0/3933449227 conn(0x56223aeee100 msgr2=0x56223afb7720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:08.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.790+0000 7f85bf09ab80 1 -- 172.21.15.100:0/3933449227 shutdown_connections 2024-05-10T11:21:08.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.790+0000 7f85bf09ab80 1 -- 172.21.15.100:0/3933449227 wait complete. 2024-05-10T11:21:08.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.790+0000 7f85bf09ab80 20 client.15924 trim_cache size 0 max 16384 2024-05-10T11:21:08.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.815+0000 7f85bf09ab80 1 Processor -- start 2024-05-10T11:21:08.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.815+0000 7f85bf09ab80 1 -- start start 2024-05-10T11:21:08.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.815+0000 7f85bf09ab80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56223aeee100 0x56223afd7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:08.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.815+0000 7f85bf09ab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56223afc24d0 0x56223afc19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:08.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.815+0000 7f85bf09ab80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56223afc2df0 0x56223afe35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:08.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.815+0000 7f85bf09ab80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f85b01af800 con 0x56223aeee100 2024-05-10T11:21:08.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.815+0000 7f85bf09ab80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f85b01ad090 con 0x56223afc2df0 2024-05-10T11:21:08.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.815+0000 7f85bf09ab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f85b01aaf00 con 0x56223afc24d0 2024-05-10T11:21:08.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.815+0000 7f85ae59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56223aeee100 0x56223afd7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:08.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.815+0000 7f85ae59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56223aeee100 0x56223afd7f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53604/0 (socket says 172.21.15.100:53604) 2024-05-10T11:21:08.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.815+0000 7f85ae59c700 1 -- 172.21.15.100:0/415496964 learned_addr learned my addr 172.21.15.100:0/415496964 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:21:08.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.815+0000 7f85add9b700 1 --2- 172.21.15.100:0/415496964 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56223afc24d0 0x56223afc19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:08.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.815+0000 7f85aed9d700 1 --2- 172.21.15.100:0/415496964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56223afc2df0 0x56223afe35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:08.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.815+0000 7f85ae59c700 1 -- 172.21.15.100:0/415496964 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56223afc24d0 msgr2=0x56223afc19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:08.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.815+0000 7f85ae59c700 1 --2- 172.21.15.100:0/415496964 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56223afc24d0 0x56223afc19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:08.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.815+0000 7f85ae59c700 1 -- 172.21.15.100:0/415496964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56223afc2df0 msgr2=0x56223afe35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:08.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.815+0000 7f85ae59c700 1 --2- 172.21.15.100:0/415496964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56223afc2df0 0x56223afe35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:08.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.815+0000 7f85ae59c700 1 -- 172.21.15.100:0/415496964 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56223afce9f0 con 0x56223aeee100 2024-05-10T11:21:08.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.815+0000 7f85aed9d700 1 --2- 172.21.15.100:0/415496964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56223afc2df0 0x56223afe35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:21:08.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.815+0000 7f85ae59c700 1 --2- 172.21.15.100:0/415496964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56223aeee100 0x56223afd7f20 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f85a0026fd0 tx=0x7f85a0003e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b783b97ef4026524 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:08.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.816+0000 7f8597fff700 1 -- 172.21.15.100:0/415496964 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a0003960 con 0x56223aeee100 2024-05-10T11:21:08.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.816+0000 7f8597fff700 1 -- 172.21.15.100:0/415496964 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f85a0029040 con 0x56223aeee100 2024-05-10T11:21:08.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.816+0000 7f8597fff700 1 -- 172.21.15.100:0/415496964 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a000eb20 con 0x56223aeee100 2024-05-10T11:21:08.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.816+0000 7f85bf09ab80 1 -- 172.21.15.100:0/415496964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56223aeee100 msgr2=0x56223afd7f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:08.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.816+0000 7f85bf09ab80 1 --2- 172.21.15.100:0/415496964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56223aeee100 0x56223afd7f20 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f85a0026fd0 tx=0x7f85a0003e40 comp rx=0 tx=0).stop 2024-05-10T11:21:08.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.816+0000 7f85bf09ab80 1 -- 172.21.15.100:0/415496964 shutdown_connections 2024-05-10T11:21:08.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.816+0000 7f85bf09ab80 1 --2- 172.21.15.100:0/415496964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56223afc2df0 0x56223afe35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:08.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.816+0000 7f85bf09ab80 1 --2- 172.21.15.100:0/415496964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56223aeee100 0x56223afd7f20 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:08.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.816+0000 7f85bf09ab80 1 --2- 172.21.15.100:0/415496964 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56223afc24d0 0x56223afc19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:08.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.816+0000 7f85bf09ab80 1 -- 172.21.15.100:0/415496964 >> 172.21.15.100:0/415496964 conn(0x56223afb7ab0 msgr2=0x56223afc12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:08.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.816+0000 7f85bf09ab80 1 -- 172.21.15.100:0/415496964 shutdown_connections 2024-05-10T11:21:08.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.816+0000 7f85bf09ab80 1 -- 172.21.15.100:0/415496964 wait complete. 2024-05-10T11:21:08.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.817+0000 7f85bf09ab80 1 -- start start 2024-05-10T11:21:08.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.817+0000 7f85bf09ab80 1 Processor -- start 2024-05-10T11:21:08.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.817+0000 7f85bf09ab80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56223aeee100 0x56223afd45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:08.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.817+0000 7f85bf09ab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56223afc24d0 0x56223afd0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:08.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.817+0000 7f85bf09ab80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56223afc2df0 0x56223afd0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:08.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.817+0000 7f85bf09ab80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f85b01ab550 con 0x56223aeee100 2024-05-10T11:21:08.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.817+0000 7f85bf09ab80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f85b01ac9c0 con 0x56223afc2df0 2024-05-10T11:21:08.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.817+0000 7f85bf09ab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f85b01abc20 con 0x56223afc24d0 2024-05-10T11:21:08.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.817+0000 7f85bf09ab80 10 client.0 _ll_register_callbacks cb 0x56223aeac920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:21:08.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.817+0000 7f85ae59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56223aeee100 0x56223afd45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:08.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.817+0000 7f85ae59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56223aeee100 0x56223afd45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53614/0 (socket says 172.21.15.100:53614) 2024-05-10T11:21:08.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.817+0000 7f85ae59c700 1 -- 172.21.15.100:0/3861658577 learned_addr learned my addr 172.21.15.100:0/3861658577 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:21:08.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.817+0000 7f85aed9d700 1 --2- 172.21.15.100:0/3861658577 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56223afc2df0 0x56223afd0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:08.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.817+0000 7f85add9b700 1 --2- 172.21.15.100:0/3861658577 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56223afc24d0 0x56223afd0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:08.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.818+0000 7f85ae59c700 1 -- 172.21.15.100:0/3861658577 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56223afc24d0 msgr2=0x56223afd0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:08.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.818+0000 7f85ae59c700 1 --2- 172.21.15.100:0/3861658577 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56223afc24d0 0x56223afd0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:08.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.818+0000 7f85ae59c700 1 -- 172.21.15.100:0/3861658577 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56223afc2df0 msgr2=0x56223afd0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:08.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.818+0000 7f85ae59c700 1 --2- 172.21.15.100:0/3861658577 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56223afc2df0 0x56223afd0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:08.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.818+0000 7f85ae59c700 1 -- 172.21.15.100:0/3861658577 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56223afd14f0 con 0x56223aeee100 2024-05-10T11:21:08.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.818+0000 7f85aed9d700 1 --2- 172.21.15.100:0/3861658577 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56223afc2df0 0x56223afd0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:21:08.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.818+0000 7f85ae59c700 1 --2- 172.21.15.100:0/3861658577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56223aeee100 0x56223afd45e0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f85a0003d60 tx=0x7f85a000e5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29f875cf461ba92f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:08.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.818+0000 7f8595ffb700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:21:08.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.818+0000 7f8595ffb700 1 -- 172.21.15.100:0/3861658577 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a004d070 con 0x56223aeee100 2024-05-10T11:21:08.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.818+0000 7f8595ffb700 1 -- 172.21.15.100:0/3861658577 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f85a000ec30 con 0x56223aeee100 2024-05-10T11:21:08.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.818+0000 7f85bf09ab80 1 -- 172.21.15.100:0/3861658577 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85a0029040 con 0x56223aeee100 2024-05-10T11:21:08.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.818+0000 7f8595ffb700 1 -- 172.21.15.100:0/3861658577 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f85a000ee30 con 0x56223aeee100 2024-05-10T11:21:08.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.818+0000 7f85bf09ab80 10 client.15930 fetch_fsmap learned FSMap version 29 2024-05-10T11:21:08.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.818+0000 7f85bf09ab80 1 -- 172.21.15.100:0/3861658577 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56223b046fe0 con 0x56223aeee100 2024-05-10T11:21:08.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.818+0000 7f8595ffb700 1 -- 172.21.15.100:0/3861658577 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f85a0029040 con 0x56223aeee100 2024-05-10T11:21:08.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.818+0000 7f8595ffb700 1 -- 172.21.15.100:0/3861658577 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f85a002e770 con 0x56223aeee100 2024-05-10T11:21:08.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.818+0000 7f85bf09ab80 10 client.15930 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T11:21:08.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.819+0000 7f85bf09ab80 10 client.15930 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:21:08.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.819+0000 7f85bf09ab80 20 client.15930 populate_metadata read hostname 'smithi100' 2024-05-10T11:21:08.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.819+0000 7f85bf09ab80 1 --2- 172.21.15.100:0/3861658577 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56223afc6c80 0x56223b0495d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:08.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.819+0000 7f85bf09ab80 4 client.15930 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T11:21:08.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.819+0000 7f85bf09ab80 1 -- 172.21.15.100:0/3861658577 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56223af22240 con 0x56223afc6c80 2024-05-10T11:21:08.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.819+0000 7f85add9b700 1 --2- 172.21.15.100:0/3861658577 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56223afc6c80 0x56223b0495d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:08.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.819+0000 7f85add9b700 1 --2- 172.21.15.100:0/3861658577 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56223afc6c80 0x56223b0495d0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8195b69c91bb2f34 server_cookie=3c887ba6d6baf4aa in_seq=0 out_seq=0 2024-05-10T11:21:08.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.819+0000 7f8595ffb700 10 client.15930 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:21:08.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.819+0000 7f8595ffb700 1 -- 172.21.15.100:0/3861658577 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56223af22240 con 0x56223afc6c80 2024-05-10T11:21:08.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.819+0000 7f8595ffb700 10 client.15930 handle_command_reply: tid=0 2024-05-10T11:21:08.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.819+0000 7f85bf09ab80 1 client.15930 shutdown 2024-05-10T11:21:08.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.819+0000 7f85bf09ab80 1 -- 172.21.15.100:0/3861658577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56223aeee100 msgr2=0x56223afd45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:08.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.819+0000 7f85bf09ab80 1 --2- 172.21.15.100:0/3861658577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56223aeee100 0x56223afd45e0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f85a0003d60 tx=0x7f85a000e5c0 comp rx=0 tx=0).stop 2024-05-10T11:21:08.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.819+0000 7f85bf09ab80 1 -- 172.21.15.100:0/3861658577 shutdown_connections 2024-05-10T11:21:08.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.819+0000 7f85bf09ab80 1 --2- 172.21.15.100:0/3861658577 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56223afc6c80 0x56223b0495d0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:08.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.819+0000 7f8595ffb700 0 client.15930 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:21:08.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.819+0000 7f85bf09ab80 1 --2- 172.21.15.100:0/3861658577 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56223afc2df0 0x56223afd0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:08.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.819+0000 7f85bf09ab80 1 --2- 172.21.15.100:0/3861658577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56223aeee100 0x56223afd45e0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:08.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.819+0000 7f85bf09ab80 1 --2- 172.21.15.100:0/3861658577 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56223afc24d0 0x56223afd0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:08.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.819+0000 7f85bf09ab80 1 -- 172.21.15.100:0/3861658577 >> 172.21.15.100:0/3861658577 conn(0x56223afb7ab0 msgr2=0x56223afc5520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:08.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.820+0000 7f85bf09ab80 1 -- 172.21.15.100:0/3861658577 shutdown_connections 2024-05-10T11:21:08.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.820+0000 7f85bf09ab80 1 -- 172.21.15.100:0/3861658577 wait complete. 2024-05-10T11:21:08.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.820+0000 7f85bf09ab80 20 client.15930 trim_cache size 0 max 16384 2024-05-10T11:21:08.826 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:21:08.826 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T11:21:08.826 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-05-10T11:21:08.826 INFO:teuthology.orchestra.run.smithi100.stdout: "c2a03361-ab28-4939-9f7b-0e4c2a5b18d8": { 2024-05-10T11:21:08.826 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-05-10T11:21:08.827 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "c2a03361-ab28-4939-9f7b-0e4c2a5b18d8", 2024-05-10T11:21:08.827 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-05-10T11:21:08.827 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:21:08.827 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:21:08.827 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:21:08.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.820+0000 7f85b4807700 1 -- 172.21.15.100:0/2684125628 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f85980b2900 msgr2=0x7f85980b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:08.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.820+0000 7f85b4807700 1 --2- 172.21.15.100:0/2684125628 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f85980b2900 0x7f85980b4dc0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f85a0009fd0 tx=0x7f85a0009260 comp rx=0 tx=0).stop 2024-05-10T11:21:08.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.820+0000 7f85b4807700 1 -- 172.21.15.100:0/2684125628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85b0108410 msgr2=0x7f85b0104b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:08.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.820+0000 7f85b4807700 1 --2- 172.21.15.100:0/2684125628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85b0108410 0x7f85b0104b00 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f85a800efc0 tx=0x7f85a800c3a0 comp rx=0 tx=0).stop 2024-05-10T11:21:08.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.820+0000 7f85b4807700 1 -- 172.21.15.100:0/2684125628 shutdown_connections 2024-05-10T11:21:08.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.820+0000 7f85b4807700 1 --2- 172.21.15.100:0/2684125628 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f85980b2900 0x7f85980b4dc0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:08.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.820+0000 7f85b4807700 1 --2- 172.21.15.100:0/2684125628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85b0108410 0x7f85b0104b00 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:08.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.820+0000 7f85b4807700 1 --2- 172.21.15.100:0/2684125628 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85b01023a0 0x7f85b0104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:08.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.820+0000 7f85b4807700 1 --2- 172.21.15.100:0/2684125628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85b0102d60 0x7f85b01045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:08.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.820+0000 7f85b4807700 1 -- 172.21.15.100:0/2684125628 >> 172.21.15.100:0/2684125628 conn(0x7f85b0079350 msgr2=0x7f85b0101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:08.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.821+0000 7f85b4807700 1 -- 172.21.15.100:0/2684125628 shutdown_connections 2024-05-10T11:21:08.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:08.821+0000 7f85b4807700 1 -- 172.21.15.100:0/2684125628 wait complete. 2024-05-10T11:21:08.933 INFO:tasks.workunit.client.0.smithi100.stdout: 1 33882 3.15 MB/sec warmup 63 sec latency 60.618 ms 2024-05-10T11:21:09.591 INFO:tasks.cephfs.filesystem:scrub status for tag:c2a03361-ab28-4939-9f7b-0e4c2a5b18d8 - {'path': '/', 'tag': 'c2a03361-ab28-4939-9f7b-0e4c2a5b18d8', 'options': 'recursive,force'} 2024-05-10T11:21:09.933 INFO:tasks.workunit.client.0.smithi100.stdout: 1 34456 3.14 MB/sec warmup 64 sec latency 18.904 ms 2024-05-10T11:21:10.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:21:09 smithi178 ceph-mon[33359]: pgmap v503: 129 pgs: 129 active+clean; 33 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 1.4 MiB/s wr, 147 op/s 2024-05-10T11:21:10.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:21:09 smithi188 ceph-mon[34075]: pgmap v503: 129 pgs: 129 active+clean; 33 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 1.4 MiB/s wr, 147 op/s 2024-05-10T11:21:10.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:21:09 smithi100 ceph-mon[30296]: pgmap v503: 129 pgs: 129 active+clean; 33 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 1.4 MiB/s wr, 147 op/s 2024-05-10T11:21:10.934 INFO:tasks.workunit.client.0.smithi100.stdout: 1 35254 3.19 MB/sec warmup 65 sec latency 15.492 ms 2024-05-10T11:21:11.934 INFO:tasks.workunit.client.0.smithi100.stdout: 1 35711 3.21 MB/sec warmup 66 sec latency 16.540 ms 2024-05-10T11:21:12.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:21:11 smithi100 ceph-mon[30296]: pgmap v504: 129 pgs: 129 active+clean; 33 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 1.3 MiB/s wr, 173 op/s 2024-05-10T11:21:12.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:21:11 smithi178 ceph-mon[33359]: pgmap v504: 129 pgs: 129 active+clean; 33 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 1.3 MiB/s wr, 173 op/s 2024-05-10T11:21:12.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:21:11 smithi188 ceph-mon[34075]: pgmap v504: 129 pgs: 129 active+clean; 33 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 1.3 MiB/s wr, 173 op/s 2024-05-10T11:21:12.933 INFO:tasks.workunit.client.0.smithi100.stdout: 1 35961 3.17 MB/sec warmup 67 sec latency 16.259 ms 2024-05-10T11:21:13.379 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.379+0000 7f938f704640 1 -- 172.21.15.100:0/182556856 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9388102a10 msgr2=0x7f9388102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:13.379 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.379+0000 7f938f704640 1 --2- 172.21.15.100:0/182556856 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9388102a10 0x7f9388102e30 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f9384009900 tx=0x7f938402a410 comp rx=0 tx=0).stop 2024-05-10T11:21:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.380+0000 7f938f704640 1 -- 172.21.15.100:0/182556856 shutdown_connections 2024-05-10T11:21:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.380+0000 7f938f704640 1 --2- 172.21.15.100:0/182556856 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93881045c0 0x7f938810ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.380+0000 7f938f704640 1 --2- 172.21.15.100:0/182556856 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9388103c00 0x7f9388104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.380+0000 7f938f704640 1 --2- 172.21.15.100:0/182556856 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9388102a10 0x7f9388102e30 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.380+0000 7f938f704640 1 -- 172.21.15.100:0/182556856 >> 172.21.15.100:0/182556856 conn(0x7f93880fdfa0 msgr2=0x7f93881003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.380+0000 7f938f704640 1 -- 172.21.15.100:0/182556856 shutdown_connections 2024-05-10T11:21:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.380+0000 7f938f704640 1 -- 172.21.15.100:0/182556856 wait complete. 2024-05-10T11:21:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.380+0000 7f938f704640 1 Processor -- start 2024-05-10T11:21:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.380+0000 7f938f704640 1 -- start start 2024-05-10T11:21:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.381+0000 7f938f704640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9388103c00 0x7f9388198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.381+0000 7f938f704640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93881045c0 0x7f93881991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.381+0000 7f938f704640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f938819e1d0 0x7f93881a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.381+0000 7f938f704640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f938810e3c0 con 0x7f9388103c00 2024-05-10T11:21:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.381+0000 7f938f704640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f938810e240 con 0x7f93881045c0 2024-05-10T11:21:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.381+0000 7f938f704640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f938810e540 con 0x7f938819e1d0 2024-05-10T11:21:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.381+0000 7f938d4a0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9388103c00 0x7f9388198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.381+0000 7f938cc9f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93881045c0 0x7f93881991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.381+0000 7f938d4a0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9388103c00 0x7f9388198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53636/0 (socket says 172.21.15.100:53636) 2024-05-10T11:21:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.381+0000 7f938d4a0640 1 -- 172.21.15.100:0/3142112936 learned_addr learned my addr 172.21.15.100:0/3142112936 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:21:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.381+0000 7f938dca1640 1 --2- 172.21.15.100:0/3142112936 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f938819e1d0 0x7f93881a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.381+0000 7f938cc9f640 1 -- 172.21.15.100:0/3142112936 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f938819e1d0 msgr2=0x7f93881a05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.381+0000 7f938cc9f640 1 --2- 172.21.15.100:0/3142112936 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f938819e1d0 0x7f93881a05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.381+0000 7f938cc9f640 1 -- 172.21.15.100:0/3142112936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9388103c00 msgr2=0x7f9388198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.381+0000 7f938cc9f640 1 --2- 172.21.15.100:0/3142112936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9388103c00 0x7f9388198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.381+0000 7f938cc9f640 1 -- 172.21.15.100:0/3142112936 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93881a0c60 con 0x7f93881045c0 2024-05-10T11:21:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.381+0000 7f938d4a0640 1 --2- 172.21.15.100:0/3142112936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9388103c00 0x7f9388198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:21:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.381+0000 7f938cc9f640 1 --2- 172.21.15.100:0/3142112936 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93881045c0 0x7f93881991b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f937800c970 tx=0x7f937800ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.382+0000 7f937e7fc640 1 -- 172.21.15.100:0/3142112936 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9378007bb0 con 0x7f93881045c0 2024-05-10T11:21:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.382+0000 7f937e7fc640 1 -- 172.21.15.100:0/3142112936 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9378007d50 con 0x7f93881045c0 2024-05-10T11:21:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.382+0000 7f938f704640 1 -- 172.21.15.100:0/3142112936 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93881a0f50 con 0x7f93881045c0 2024-05-10T11:21:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.382+0000 7f937e7fc640 1 -- 172.21.15.100:0/3142112936 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93780186f0 con 0x7f93881045c0 2024-05-10T11:21:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.382+0000 7f938f704640 1 -- 172.21.15.100:0/3142112936 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93881a1410 con 0x7f93881045c0 2024-05-10T11:21:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.384+0000 7f937e7fc640 1 -- 172.21.15.100:0/3142112936 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f937800f450 con 0x7f93881045c0 2024-05-10T11:21:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.385+0000 7f9392419740 1 Processor -- start 2024-05-10T11:21:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.385+0000 7f9392419740 1 -- start start 2024-05-10T11:21:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.385+0000 7f9392419740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564e9ccddb0 0x5564e9cce1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.385+0000 7f9392419740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564e9cce7a0 0x5564e9cd32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.385+0000 7f9392419740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564e9cd3910 0x5564e9cd5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.385+0000 7f9392419740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5564e9c325d0 con 0x5564e9ccddb0 2024-05-10T11:21:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.385+0000 7f9392419740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5564e9beb890 con 0x5564e9cd3910 2024-05-10T11:21:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.385+0000 7f9392419740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5564e9c2e590 con 0x5564e9cce7a0 2024-05-10T11:21:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.385+0000 7f938d4a0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564e9cce7a0 0x5564e9cd32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.385+0000 7f938dca1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564e9ccddb0 0x5564e9cce1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.385+0000 7f938cc9f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564e9cd3910 0x5564e9cd5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.385+0000 7f938d4a0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564e9cce7a0 0x5564e9cd32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:60860/0 (socket says 172.21.15.100:60860) 2024-05-10T11:21:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.385+0000 7f938d4a0640 1 -- 172.21.15.100:0/4092345595 learned_addr learned my addr 172.21.15.100:0/4092345595 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:21:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.385+0000 7f938dca1640 1 -- 172.21.15.100:0/4092345595 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564e9cce7a0 msgr2=0x5564e9cd32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.385+0000 7f938dca1640 1 --2- 172.21.15.100:0/4092345595 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564e9cce7a0 0x5564e9cd32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.385+0000 7f938dca1640 1 -- 172.21.15.100:0/4092345595 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564e9cd3910 msgr2=0x5564e9cd5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.385+0000 7f938dca1640 1 --2- 172.21.15.100:0/4092345595 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564e9cd3910 0x5564e9cd5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.385+0000 7f938dca1640 1 -- 172.21.15.100:0/4092345595 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564e9944210 con 0x5564e9ccddb0 2024-05-10T11:21:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.385+0000 7f937e7fc640 1 --2- 172.21.15.100:0/3142112936 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f93740b2920 0x7f93740b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.385+0000 7f938d4a0640 1 --2- 172.21.15.100:0/3142112936 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f93740b2920 0x7f93740b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.385+0000 7f938dca1640 1 --2- 172.21.15.100:0/4092345595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564e9ccddb0 0x5564e9cce1d0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f9380009f00 tx=0x7f9380011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=764c50adbf5a1de3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.386+0000 7f937e7fc640 1 -- 172.21.15.100:0/3142112936 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f93780e25b0 con 0x7f93881045c0 2024-05-10T11:21:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.386+0000 7f93737fe640 1 -- 172.21.15.100:0/4092345595 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9380004d20 con 0x5564e9ccddb0 2024-05-10T11:21:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.386+0000 7f93737fe640 1 -- 172.21.15.100:0/4092345595 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9380004030 con 0x5564e9ccddb0 2024-05-10T11:21:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.386+0000 7f93737fe640 1 -- 172.21.15.100:0/4092345595 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9380005260 con 0x5564e9ccddb0 2024-05-10T11:21:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.386+0000 7f9392419740 1 -- 172.21.15.100:0/4092345595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564e9ccddb0 msgr2=0x5564e9cce1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.386+0000 7f9392419740 1 --2- 172.21.15.100:0/4092345595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564e9ccddb0 0x5564e9cce1d0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f9380009f00 tx=0x7f9380011810 comp rx=0 tx=0).stop 2024-05-10T11:21:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.386+0000 7f938d4a0640 1 --2- 172.21.15.100:0/3142112936 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f93740b2920 0x7f93740b4de0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f93840096c0 tx=0x7f9384031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.387+0000 7f9392419740 1 -- 172.21.15.100:0/4092345595 shutdown_connections 2024-05-10T11:21:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.387+0000 7f9392419740 1 --2- 172.21.15.100:0/4092345595 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564e9cd3910 0x5564e9cd5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.387+0000 7f9392419740 1 --2- 172.21.15.100:0/4092345595 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564e9cce7a0 0x5564e9cd32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.387+0000 7f9392419740 1 --2- 172.21.15.100:0/4092345595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564e9ccddb0 0x5564e9cce1d0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.387+0000 7f9392419740 1 -- 172.21.15.100:0/4092345595 >> 172.21.15.100:0/4092345595 conn(0x5564e9cc9420 msgr2=0x5564e9cc9820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.387+0000 7f9392419740 1 -- 172.21.15.100:0/4092345595 shutdown_connections 2024-05-10T11:21:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.387+0000 7f9392419740 1 -- 172.21.15.100:0/4092345595 wait complete. 2024-05-10T11:21:13.393 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.387+0000 7f9392419740 1 -- start start 2024-05-10T11:21:13.394 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.387+0000 7f9392419740 1 Processor -- start 2024-05-10T11:21:13.394 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.387+0000 7f9392419740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564e9ccddb0 0x5564e9cd8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:13.394 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.388+0000 7f9392419740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564e9cce7a0 0x5564e9cd8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:13.394 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.388+0000 7f9392419740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564e9cd3910 0x5564e9cdf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:13.394 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.388+0000 7f9392419740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5564e9cd68c0 con 0x5564e9cd3910 2024-05-10T11:21:13.394 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.388+0000 7f9392419740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5564e9bef370 con 0x5564e9ccddb0 2024-05-10T11:21:13.394 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.388+0000 7f9392419740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5564e9cd6a40 con 0x5564e9cce7a0 2024-05-10T11:21:13.394 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.388+0000 7f9392419740 10 client.0 _ll_register_callbacks cb 0x5564e9943a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:21:13.394 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.388+0000 7f938d4a0640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564e9ccddb0 0x5564e9cd8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:13.394 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.388+0000 7f938cc9f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564e9cce7a0 0x5564e9cd8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.388+0000 7f938d4a0640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564e9ccddb0 0x5564e9cd8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:55610/0 (socket says 172.21.15.100:55610) 2024-05-10T11:21:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.388+0000 7f938d4a0640 1 -- 172.21.15.100:0/1714888919 learned_addr learned my addr 172.21.15.100:0/1714888919 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:21:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.388+0000 7f938dca1640 1 --2- 172.21.15.100:0/1714888919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564e9cd3910 0x5564e9cdf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.388+0000 7f938cc9f640 1 -- 172.21.15.100:0/1714888919 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564e9ccddb0 msgr2=0x5564e9cd8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.388+0000 7f938cc9f640 1 --2- 172.21.15.100:0/1714888919 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564e9ccddb0 0x5564e9cd8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.388+0000 7f938cc9f640 1 -- 172.21.15.100:0/1714888919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564e9cd3910 msgr2=0x5564e9cdf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.388+0000 7f938cc9f640 1 --2- 172.21.15.100:0/1714888919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564e9cd3910 0x5564e9cdf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.388+0000 7f938cc9f640 1 -- 172.21.15.100:0/1714888919 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564e9a10620 con 0x5564e9cce7a0 2024-05-10T11:21:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.388+0000 7f938d4a0640 1 --2- 172.21.15.100:0/1714888919 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564e9ccddb0 0x5564e9cd8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:21:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.388+0000 7f938cc9f640 1 --2- 172.21.15.100:0/1714888919 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564e9cce7a0 0x5564e9cd8560 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f93780dfda0 tx=0x7f937800fc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4be7d1f991c905 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.388+0000 7f9370ff9640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:21:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.388+0000 7f9370ff9640 1 -- 172.21.15.100:0/1714888919 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93780a9640 con 0x5564e9cce7a0 2024-05-10T11:21:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.388+0000 7f9392419740 1 -- 172.21.15.100:0/1714888919 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9380004030 con 0x5564e9cce7a0 2024-05-10T11:21:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.388+0000 7f9370ff9640 1 -- 172.21.15.100:0/1714888919 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f937800f740 con 0x5564e9cce7a0 2024-05-10T11:21:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.389+0000 7f9370ff9640 1 -- 172.21.15.100:0/1714888919 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f93780a8fe0 con 0x5564e9cce7a0 2024-05-10T11:21:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.389+0000 7f9392419740 10 client.25444 fetch_fsmap learned FSMap version 29 2024-05-10T11:21:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.389+0000 7f9392419740 1 -- 172.21.15.100:0/1714888919 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5564e9ce2650 con 0x5564e9cce7a0 2024-05-10T11:21:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.389+0000 7f9370ff9640 1 -- 172.21.15.100:0/1714888919 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9380004030 con 0x5564e9cce7a0 2024-05-10T11:21:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.389+0000 7f9370ff9640 1 -- 172.21.15.100:0/1714888919 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f93780b2240 con 0x5564e9cce7a0 2024-05-10T11:21:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.389+0000 7f9392419740 10 client.25444 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T11:21:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.389+0000 7f9392419740 10 client.25444 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:21:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.389+0000 7f9392419740 20 client.25444 populate_metadata read hostname 'smithi100' 2024-05-10T11:21:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.389+0000 7f9392419740 1 --2- 172.21.15.100:0/1714888919 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5564e9ce3d20 0x5564e9ce61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.389+0000 7f9392419740 4 client.25444 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:21:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.389+0000 7f9392419740 1 -- 172.21.15.100:0/1714888919 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5564e9c401e0 con 0x5564e9ce3d20 2024-05-10T11:21:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.389+0000 7f938d4a0640 1 --2- 172.21.15.100:0/1714888919 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5564e9ce3d20 0x5564e9ce61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.390+0000 7f938d4a0640 1 --2- 172.21.15.100:0/1714888919 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5564e9ce3d20 0x5564e9ce61c0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ad4f524db017aee server_cookie=559aeb02850b94a1 in_seq=0 out_seq=0 2024-05-10T11:21:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.390+0000 7f9370ff9640 10 client.25444 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:21:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.390+0000 7f9370ff9640 1 -- 172.21.15.100:0/1714888919 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5564e9c401e0 con 0x5564e9ce3d20 2024-05-10T11:21:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.390+0000 7f9370ff9640 10 client.25444 handle_command_reply: tid=0 2024-05-10T11:21:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.390+0000 7f9392419740 1 client.25444 shutdown 2024-05-10T11:21:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.391+0000 7f9392419740 1 -- 172.21.15.100:0/1714888919 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564e9cce7a0 msgr2=0x5564e9cd8560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.391+0000 7f9392419740 1 --2- 172.21.15.100:0/1714888919 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564e9cce7a0 0x5564e9cd8560 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f93780dfda0 tx=0x7f937800fc30 comp rx=0 tx=0).stop 2024-05-10T11:21:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.391+0000 7f9392419740 1 -- 172.21.15.100:0/1714888919 shutdown_connections 2024-05-10T11:21:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.391+0000 7f9392419740 1 --2- 172.21.15.100:0/1714888919 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5564e9ce3d20 0x5564e9ce61c0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.391+0000 7f9392419740 1 --2- 172.21.15.100:0/1714888919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564e9cd3910 0x5564e9cdf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.391+0000 7f9392419740 1 --2- 172.21.15.100:0/1714888919 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564e9cce7a0 0x5564e9cd8560 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.391+0000 7f9370ff9640 0 client.25444 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:21:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.391+0000 7f9392419740 1 --2- 172.21.15.100:0/1714888919 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564e9ccddb0 0x5564e9cd8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.391+0000 7f9392419740 1 -- 172.21.15.100:0/1714888919 >> 172.21.15.100:0/1714888919 conn(0x5564e9cc9420 msgr2=0x5564e9cc8580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.391+0000 7f9392419740 1 -- 172.21.15.100:0/1714888919 shutdown_connections 2024-05-10T11:21:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.391+0000 7f9392419740 1 -- 172.21.15.100:0/1714888919 wait complete. 2024-05-10T11:21:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.391+0000 7f9392419740 20 client.25444 trim_cache size 0 max 16384 2024-05-10T11:21:13.407 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.407+0000 7f9392419740 1 Processor -- start 2024-05-10T11:21:13.407 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.408+0000 7f9392419740 1 -- start start 2024-05-10T11:21:13.409 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.408+0000 7f9392419740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564e9ccddb0 0x5564e9cc43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:13.410 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.408+0000 7f9392419740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564e9cce7a0 0x5564e9ccfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:13.410 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.408+0000 7f9392419740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564e9cd3910 0x5564e9ccff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:13.410 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.408+0000 7f9392419740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f93881ae100 con 0x5564e9cce7a0 2024-05-10T11:21:13.410 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.408+0000 7f9392419740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f93881ab9d0 con 0x5564e9ccddb0 2024-05-10T11:21:13.410 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.408+0000 7f9392419740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f93881a9820 con 0x5564e9cd3910 2024-05-10T11:21:13.410 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.408+0000 7f938cc9f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564e9cce7a0 0x5564e9ccfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:13.410 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.408+0000 7f938d4a0640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564e9ccddb0 0x5564e9cc43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:13.410 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.408+0000 7f938cc9f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564e9cce7a0 0x5564e9ccfa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53660/0 (socket says 172.21.15.100:53660) 2024-05-10T11:21:13.410 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.408+0000 7f938cc9f640 1 -- 172.21.15.100:0/4086384945 learned_addr learned my addr 172.21.15.100:0/4086384945 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:21:13.410 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.408+0000 7f938dca1640 1 --2- 172.21.15.100:0/4086384945 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564e9cd3910 0x5564e9ccff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:13.410 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.408+0000 7f938d4a0640 1 -- 172.21.15.100:0/4086384945 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564e9cd3910 msgr2=0x5564e9ccff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:13.410 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.408+0000 7f938d4a0640 1 --2- 172.21.15.100:0/4086384945 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564e9cd3910 0x5564e9ccff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.408+0000 7f938d4a0640 1 -- 172.21.15.100:0/4086384945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564e9cce7a0 msgr2=0x5564e9ccfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.408+0000 7f938d4a0640 1 --2- 172.21.15.100:0/4086384945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564e9cce7a0 0x5564e9ccfa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.408+0000 7f938d4a0640 1 -- 172.21.15.100:0/4086384945 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564e9ce0690 con 0x5564e9ccddb0 2024-05-10T11:21:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.408+0000 7f938cc9f640 1 --2- 172.21.15.100:0/4086384945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564e9cce7a0 0x5564e9ccfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:21:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.408+0000 7f938d4a0640 1 --2- 172.21.15.100:0/4086384945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564e9ccddb0 0x5564e9cc43e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9384009230 tx=0x7f938403f040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f66c27de93c7d59 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.409+0000 7f9372ffd640 1 -- 172.21.15.100:0/4086384945 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9384059070 con 0x5564e9ccddb0 2024-05-10T11:21:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.409+0000 7f9372ffd640 1 -- 172.21.15.100:0/4086384945 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9384003c80 con 0x5564e9ccddb0 2024-05-10T11:21:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.409+0000 7f9372ffd640 1 -- 172.21.15.100:0/4086384945 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9384035040 con 0x5564e9ccddb0 2024-05-10T11:21:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.409+0000 7f9392419740 1 -- 172.21.15.100:0/4086384945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564e9ccddb0 msgr2=0x5564e9cc43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.409+0000 7f9392419740 1 --2- 172.21.15.100:0/4086384945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564e9ccddb0 0x5564e9cc43e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9384009230 tx=0x7f938403f040 comp rx=0 tx=0).stop 2024-05-10T11:21:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.409+0000 7f9392419740 1 -- 172.21.15.100:0/4086384945 shutdown_connections 2024-05-10T11:21:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.409+0000 7f9392419740 1 --2- 172.21.15.100:0/4086384945 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564e9cd3910 0x5564e9ccff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.409+0000 7f9392419740 1 --2- 172.21.15.100:0/4086384945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564e9cce7a0 0x5564e9ccfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.409+0000 7f9392419740 1 --2- 172.21.15.100:0/4086384945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564e9ccddb0 0x5564e9cc43e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.410+0000 7f9392419740 1 -- 172.21.15.100:0/4086384945 >> 172.21.15.100:0/4086384945 conn(0x5564e9cc9420 msgr2=0x5564e9cc9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.410+0000 7f9392419740 1 -- 172.21.15.100:0/4086384945 shutdown_connections 2024-05-10T11:21:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.410+0000 7f9392419740 1 -- 172.21.15.100:0/4086384945 wait complete. 2024-05-10T11:21:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.410+0000 7f9392419740 1 -- start start 2024-05-10T11:21:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.410+0000 7f9392419740 1 Processor -- start 2024-05-10T11:21:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.410+0000 7f9392419740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564e9ccddb0 0x5564e9cdbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.410+0000 7f9392419740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564e9cce7a0 0x5564e9cdc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.410+0000 7f9392419740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564e9cd3910 0x5564e9d50a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.410+0000 7f9392419740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f93881a9e90 con 0x5564e9cd3910 2024-05-10T11:21:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.410+0000 7f9392419740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f93881ab300 con 0x5564e9cce7a0 2024-05-10T11:21:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.410+0000 7f9392419740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f93881aa560 con 0x5564e9ccddb0 2024-05-10T11:21:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.411+0000 7f9392419740 10 client.0 _ll_register_callbacks cb 0x7f93780e5d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:21:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.411+0000 7f938dca1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564e9cd3910 0x5564e9d50a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.411+0000 7f938d4a0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564e9ccddb0 0x5564e9cdbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.411+0000 7f938cc9f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564e9cce7a0 0x5564e9cdc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.411+0000 7f938dca1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564e9cd3910 0x5564e9d50a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53664/0 (socket says 172.21.15.100:53664) 2024-05-10T11:21:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.411+0000 7f938dca1640 1 -- 172.21.15.100:0/3630151200 learned_addr learned my addr 172.21.15.100:0/3630151200 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:21:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.411+0000 7f938dca1640 1 -- 172.21.15.100:0/3630151200 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564e9ccddb0 msgr2=0x5564e9cdbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.411+0000 7f938dca1640 1 --2- 172.21.15.100:0/3630151200 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564e9ccddb0 0x5564e9cdbcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.411+0000 7f938dca1640 1 -- 172.21.15.100:0/3630151200 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564e9cce7a0 msgr2=0x5564e9cdc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.411+0000 7f938dca1640 1 --2- 172.21.15.100:0/3630151200 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564e9cce7a0 0x5564e9cdc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.411+0000 7f938dca1640 1 -- 172.21.15.100:0/3630151200 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564e9d51010 con 0x5564e9cd3910 2024-05-10T11:21:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.411+0000 7f938d4a0640 1 --2- 172.21.15.100:0/3630151200 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564e9ccddb0 0x5564e9cdbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:21:13.417 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.411+0000 7f938dca1640 1 --2- 172.21.15.100:0/3630151200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564e9cd3910 0x5564e9d50a80 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x5564e9ccf000 tx=0x7f93800097c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea354e927aa17039 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:13.417 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.411+0000 7f9370ff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:21:13.417 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.411+0000 7f9370ff9640 1 -- 172.21.15.100:0/3630151200 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9380005250 con 0x5564e9cd3910 2024-05-10T11:21:13.417 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.411+0000 7f9370ff9640 1 -- 172.21.15.100:0/3630151200 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f93800053f0 con 0x5564e9cd3910 2024-05-10T11:21:13.417 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.411+0000 7f9392419740 1 -- 172.21.15.100:0/3630151200 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9384003c80 con 0x5564e9cd3910 2024-05-10T11:21:13.417 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.411+0000 7f9370ff9640 1 -- 172.21.15.100:0/3630151200 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9380035050 con 0x5564e9cd3910 2024-05-10T11:21:13.417 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.411+0000 7f9392419740 10 client.15948 fetch_fsmap learned FSMap version 29 2024-05-10T11:21:13.418 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.411+0000 7f9392419740 1 -- 172.21.15.100:0/3630151200 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5564e9d52320 con 0x5564e9cd3910 2024-05-10T11:21:13.418 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.412+0000 7f9370ff9640 1 -- 172.21.15.100:0/3630151200 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9384003c80 con 0x5564e9cd3910 2024-05-10T11:21:13.418 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.412+0000 7f9370ff9640 1 -- 172.21.15.100:0/3630151200 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f9380018070 con 0x5564e9cd3910 2024-05-10T11:21:13.418 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.412+0000 7f9392419740 10 client.15948 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T11:21:13.418 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.412+0000 7f9392419740 10 client.15948 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:21:13.418 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.412+0000 7f9392419740 20 client.15948 populate_metadata read hostname 'smithi100' 2024-05-10T11:21:13.418 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.412+0000 7f9392419740 1 --2- 172.21.15.100:0/3630151200 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5564e9d52b30 0x5564e9d54f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:13.418 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.412+0000 7f9392419740 4 client.15948 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:21:13.418 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.412+0000 7f9392419740 1 -- 172.21.15.100:0/3630151200 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5564e9c3a050 con 0x5564e9d52b30 2024-05-10T11:21:13.418 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.412+0000 7f938d4a0640 1 --2- 172.21.15.100:0/3630151200 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5564e9d52b30 0x5564e9d54f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:13.418 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.412+0000 7f9370ff9640 10 client.15948 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:21:13.418 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.412+0000 7f938d4a0640 1 --2- 172.21.15.100:0/3630151200 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5564e9d52b30 0x5564e9d54f50 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d1ae831ce9bf69 server_cookie=92239ca77777605d in_seq=0 out_seq=0 2024-05-10T11:21:13.431 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.430+0000 7f9370ff9640 1 -- 172.21.15.100:0/3630151200 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5564e9c3a050 con 0x5564e9d52b30 2024-05-10T11:21:13.431 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.430+0000 7f9370ff9640 10 client.15948 handle_command_reply: tid=0 2024-05-10T11:21:13.431 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.430+0000 7f9392419740 1 client.15948 shutdown 2024-05-10T11:21:13.431 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.431+0000 7f9392419740 1 -- 172.21.15.100:0/3630151200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564e9cd3910 msgr2=0x5564e9d50a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:13.431 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.431+0000 7f9392419740 1 --2- 172.21.15.100:0/3630151200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564e9cd3910 0x5564e9d50a80 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x5564e9ccf000 tx=0x7f93800097c0 comp rx=0 tx=0).stop 2024-05-10T11:21:13.431 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.431+0000 7f9392419740 1 -- 172.21.15.100:0/3630151200 shutdown_connections 2024-05-10T11:21:13.431 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.431+0000 7f9392419740 1 --2- 172.21.15.100:0/3630151200 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5564e9d52b30 0x5564e9d54f50 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:13.431 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.431+0000 7f9392419740 1 --2- 172.21.15.100:0/3630151200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564e9cd3910 0x5564e9d50a80 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:13.431 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.431+0000 7f9370ff9640 0 client.15948 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:21:13.431 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.431+0000 7f9392419740 1 --2- 172.21.15.100:0/3630151200 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564e9cce7a0 0x5564e9cdc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:13.432 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.431+0000 7f9392419740 1 --2- 172.21.15.100:0/3630151200 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564e9ccddb0 0x5564e9cdbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:13.432 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.431+0000 7f9392419740 1 -- 172.21.15.100:0/3630151200 >> 172.21.15.100:0/3630151200 conn(0x5564e9cc9420 msgr2=0x5564e9cc43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:13.432 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.431+0000 7f9392419740 1 -- 172.21.15.100:0/3630151200 shutdown_connections 2024-05-10T11:21:13.432 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.431+0000 7f9392419740 1 -- 172.21.15.100:0/3630151200 wait complete. 2024-05-10T11:21:13.432 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.431+0000 7f9392419740 20 client.15948 trim_cache size 0 max 16384 2024-05-10T11:21:13.432 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:21:13.432 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:21:13.432 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:21:13.432 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:21:13.433 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.432+0000 7f938f704640 1 -- 172.21.15.100:0/3142112936 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f93740b2920 msgr2=0x7f93740b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:13.433 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.432+0000 7f938f704640 1 --2- 172.21.15.100:0/3142112936 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f93740b2920 0x7f93740b4de0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f93840096c0 tx=0x7f9384031400 comp rx=0 tx=0).stop 2024-05-10T11:21:13.433 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.432+0000 7f938f704640 1 -- 172.21.15.100:0/3142112936 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93881045c0 msgr2=0x7f93881991b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:13.433 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.432+0000 7f938f704640 1 --2- 172.21.15.100:0/3142112936 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93881045c0 0x7f93881991b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f937800c970 tx=0x7f937800ce30 comp rx=0 tx=0).stop 2024-05-10T11:21:13.433 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.432+0000 7f938f704640 1 -- 172.21.15.100:0/3142112936 shutdown_connections 2024-05-10T11:21:13.434 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.432+0000 7f938f704640 1 --2- 172.21.15.100:0/3142112936 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f93740b2920 0x7f93740b4de0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:13.434 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.432+0000 7f938f704640 1 --2- 172.21.15.100:0/3142112936 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f938819e1d0 0x7f93881a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:13.434 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.432+0000 7f938f704640 1 --2- 172.21.15.100:0/3142112936 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93881045c0 0x7f93881991b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:13.434 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.432+0000 7f938f704640 1 --2- 172.21.15.100:0/3142112936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9388103c00 0x7f9388198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:13.434 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.432+0000 7f938f704640 1 -- 172.21.15.100:0/3142112936 >> 172.21.15.100:0/3142112936 conn(0x7f93880fdfa0 msgr2=0x7f9388106f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:13.434 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.432+0000 7f938f704640 1 -- 172.21.15.100:0/3142112936 shutdown_connections 2024-05-10T11:21:13.434 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:13.432+0000 7f938f704640 1 -- 172.21.15.100:0/3142112936 wait complete. 2024-05-10T11:21:13.933 INFO:tasks.workunit.client.0.smithi100.stdout: 1 36369 3.13 MB/sec warmup 68 sec latency 16.479 ms 2024-05-10T11:21:14.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:21:13 smithi178 ceph-mon[33359]: pgmap v505: 129 pgs: 129 active+clean; 34 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-05-10T11:21:14.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:21:13 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:21:14.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:21:13 smithi188 ceph-mon[34075]: pgmap v505: 129 pgs: 129 active+clean; 34 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-05-10T11:21:14.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:21:13 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:21:14.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:21:13 smithi100 ceph-mon[30296]: pgmap v505: 129 pgs: 129 active+clean; 34 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-05-10T11:21:14.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:21:13 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:21:14.933 INFO:tasks.workunit.client.0.smithi100.stdout: 1 36920 3.15 MB/sec warmup 69 sec latency 14.540 ms 2024-05-10T11:21:15.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:21:14 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:21:15.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:21:14 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:21:15.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:21:14 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:21:15.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:21:14 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:21:15.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:21:14 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:21:15.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:21:14 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:21:15.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:21:14 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:21:15.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:21:14 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:21:15.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:21:14 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:21:15.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:21:14 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:21:15.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:21:14 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:21:15.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:21:14 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:21:15.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:21:14 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:21:15.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:21:14 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:21:15.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:21:14 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:21:15.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:21:14 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:21:15.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:21:14 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:21:15.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:21:14 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:21:15.933 INFO:tasks.workunit.client.0.smithi100.stdout: 1 37562 3.11 MB/sec warmup 70 sec latency 15.139 ms 2024-05-10T11:21:16.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:21:16 smithi178 ceph-mon[33359]: pgmap v506: 129 pgs: 129 active+clean; 35 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 1.4 MiB/s wr, 156 op/s 2024-05-10T11:21:16.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:21:16 smithi188 ceph-mon[34075]: pgmap v506: 129 pgs: 129 active+clean; 35 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 1.4 MiB/s wr, 156 op/s 2024-05-10T11:21:16.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:21:16 smithi100 ceph-mon[30296]: pgmap v506: 129 pgs: 129 active+clean; 35 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 1.4 MiB/s wr, 156 op/s 2024-05-10T11:21:16.933 INFO:tasks.workunit.client.0.smithi100.stdout: 1 38103 3.10 MB/sec warmup 71 sec latency 13.691 ms 2024-05-10T11:21:17.933 INFO:tasks.workunit.client.0.smithi100.stdout: 1 38826 3.15 MB/sec warmup 72 sec latency 14.707 ms 2024-05-10T11:21:18.304 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:21:18 smithi178 ceph-mon[33359]: pgmap v507: 129 pgs: 129 active+clean; 32 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 1.4 MiB/s wr, 183 op/s 2024-05-10T11:21:18.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:21:18 smithi188 ceph-mon[34075]: pgmap v507: 129 pgs: 129 active+clean; 32 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 1.4 MiB/s wr, 183 op/s 2024-05-10T11:21:18.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:21:18 smithi100 ceph-mon[30296]: pgmap v507: 129 pgs: 129 active+clean; 32 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 1.4 MiB/s wr, 183 op/s 2024-05-10T11:21:18.933 INFO:tasks.workunit.client.0.smithi100.stdout: 1 39299 3.16 MB/sec warmup 73 sec latency 16.456 ms 2024-05-10T11:21:19.933 INFO:tasks.workunit.client.0.smithi100.stdout: 1 39558 3.13 MB/sec warmup 74 sec latency 13.703 ms 2024-05-10T11:21:20.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:21:20 smithi178 ceph-mon[33359]: pgmap v508: 129 pgs: 129 active+clean; 32 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 1.4 MiB/s wr, 162 op/s 2024-05-10T11:21:20.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:21:20 smithi188 ceph-mon[34075]: pgmap v508: 129 pgs: 129 active+clean; 32 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 1.4 MiB/s wr, 162 op/s 2024-05-10T11:21:20.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:21:20 smithi100 ceph-mon[30296]: pgmap v508: 129 pgs: 129 active+clean; 32 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 1.4 MiB/s wr, 162 op/s 2024-05-10T11:21:20.933 INFO:tasks.workunit.client.0.smithi100.stdout: 1 40002 3.10 MB/sec warmup 75 sec latency 14.942 ms 2024-05-10T11:21:21.933 INFO:tasks.workunit.client.0.smithi100.stdout: 1 40593 3.11 MB/sec warmup 76 sec latency 12.996 ms 2024-05-10T11:21:22.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:21:22 smithi178 ceph-mon[33359]: pgmap v509: 129 pgs: 129 active+clean; 33 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 1.3 MiB/s wr, 159 op/s 2024-05-10T11:21:22.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:21:22 smithi188 ceph-mon[34075]: pgmap v509: 129 pgs: 129 active+clean; 33 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 1.3 MiB/s wr, 159 op/s 2024-05-10T11:21:22.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:21:22 smithi100 ceph-mon[30296]: pgmap v509: 129 pgs: 129 active+clean; 33 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 1.3 MiB/s wr, 159 op/s 2024-05-10T11:21:22.933 INFO:tasks.workunit.client.0.smithi100.stdout: 1 41318 3.08 MB/sec warmup 77 sec latency 13.131 ms 2024-05-10T11:21:23.933 INFO:tasks.workunit.client.0.smithi100.stdout: 1 41950 3.12 MB/sec warmup 78 sec latency 15.534 ms 2024-05-10T11:21:24.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:21:24 smithi178 ceph-mon[33359]: pgmap v510: 129 pgs: 129 active+clean; 33 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 1.3 MiB/s wr, 139 op/s 2024-05-10T11:21:24.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:21:24 smithi188 ceph-mon[34075]: pgmap v510: 129 pgs: 129 active+clean; 33 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 1.3 MiB/s wr, 139 op/s 2024-05-10T11:21:24.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:21:24 smithi100 ceph-mon[30296]: pgmap v510: 129 pgs: 129 active+clean; 33 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 1.3 MiB/s wr, 139 op/s 2024-05-10T11:21:24.933 INFO:tasks.workunit.client.0.smithi100.stdout: 1 42544 3.15 MB/sec warmup 79 sec latency 13.418 ms 2024-05-10T11:21:25.933 INFO:tasks.workunit.client.0.smithi100.stdout: 1 42924 3.13 MB/sec warmup 80 sec latency 20.480 ms 2024-05-10T11:21:26.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:21:26 smithi178 ceph-mon[33359]: pgmap v511: 129 pgs: 129 active+clean; 32 MiB data, 592 MiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 1.3 MiB/s wr, 146 op/s 2024-05-10T11:21:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.525+0000 7f37a2483640 1 -- 172.21.15.100:0/217963641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f379c103490 msgr2=0x7f379c105880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.525+0000 7f37a2483640 1 --2- 172.21.15.100:0/217963641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f379c103490 0x7f379c105880 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f378c009900 tx=0x7f378c02a410 comp rx=0 tx=0).stop 2024-05-10T11:21:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.525+0000 7f37a2483640 1 -- 172.21.15.100:0/217963641 shutdown_connections 2024-05-10T11:21:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.526+0000 7f37a2483640 1 --2- 172.21.15.100:0/217963641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f379c1086f0 0x7f379c10ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.526+0000 7f37a2483640 1 --2- 172.21.15.100:0/217963641 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f379c105dc0 0x7f379c1081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.526+0000 7f37a2483640 1 --2- 172.21.15.100:0/217963641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f379c103490 0x7f379c105880 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.526+0000 7f37a2483640 1 -- 172.21.15.100:0/217963641 >> 172.21.15.100:0/217963641 conn(0x7f379c0fce80 msgr2=0x7f379c0ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.526+0000 7f37a2483640 1 -- 172.21.15.100:0/217963641 shutdown_connections 2024-05-10T11:21:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.526+0000 7f37a2483640 1 -- 172.21.15.100:0/217963641 wait complete. 2024-05-10T11:21:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.526+0000 7f37a2483640 1 Processor -- start 2024-05-10T11:21:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.526+0000 7f37a2483640 1 -- start start 2024-05-10T11:21:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.526+0000 7f37a2483640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f379c103490 0x7f379c198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.526+0000 7f37a2483640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f379c105dc0 0x7f379c199010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.526+0000 7f37a2483640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f379c1086f0 0x7f379c1a0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.526+0000 7f37a2483640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f379c10e3f0 con 0x7f379c103490 2024-05-10T11:21:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.526+0000 7f37a2483640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f379c10e270 con 0x7f379c1086f0 2024-05-10T11:21:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.526+0000 7f37a2483640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f379c10e570 con 0x7f379c105dc0 2024-05-10T11:21:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.527+0000 7f379bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f379c103490 0x7f379c198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.527+0000 7f379bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f379c103490 0x7f379c198ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36178/0 (socket says 172.21.15.100:36178) 2024-05-10T11:21:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.527+0000 7f379bfff640 1 -- 172.21.15.100:0/1446108854 learned_addr learned my addr 172.21.15.100:0/1446108854 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:21:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.527+0000 7f37a0a20640 1 --2- 172.21.15.100:0/1446108854 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f379c1086f0 0x7f379c1a0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.527+0000 7f379b7fe640 1 --2- 172.21.15.100:0/1446108854 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f379c105dc0 0x7f379c199010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.527+0000 7f379bfff640 1 -- 172.21.15.100:0/1446108854 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f379c105dc0 msgr2=0x7f379c199010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.527+0000 7f379bfff640 1 --2- 172.21.15.100:0/1446108854 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f379c105dc0 0x7f379c199010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.527+0000 7f379bfff640 1 -- 172.21.15.100:0/1446108854 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f379c1086f0 msgr2=0x7f379c1a0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:26.530 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.527+0000 7f379bfff640 1 --2- 172.21.15.100:0/1446108854 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f379c1086f0 0x7f379c1a0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:26.530 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.527+0000 7f379bfff640 1 -- 172.21.15.100:0/1446108854 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f379c1a06f0 con 0x7f379c103490 2024-05-10T11:21:26.530 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.527+0000 7f379bfff640 1 --2- 172.21.15.100:0/1446108854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f379c103490 0x7f379c198ad0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f378c009a20 tx=0x7f378c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:26.530 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.527+0000 7f37997fa640 1 -- 172.21.15.100:0/1446108854 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378c03f030 con 0x7f379c103490 2024-05-10T11:21:26.530 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.527+0000 7f37997fa640 1 -- 172.21.15.100:0/1446108854 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f378c003a80 con 0x7f379c103490 2024-05-10T11:21:26.530 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.527+0000 7f37a2483640 1 -- 172.21.15.100:0/1446108854 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f379c1a0980 con 0x7f379c103490 2024-05-10T11:21:26.530 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.527+0000 7f37997fa640 1 -- 172.21.15.100:0/1446108854 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378c035070 con 0x7f379c103490 2024-05-10T11:21:26.530 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.527+0000 7f37a2483640 1 -- 172.21.15.100:0/1446108854 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f379c1a0e70 con 0x7f379c103490 2024-05-10T11:21:26.531 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:21:26 smithi188 ceph-mon[34075]: pgmap v511: 129 pgs: 129 active+clean; 32 MiB data, 592 MiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 1.3 MiB/s wr, 146 op/s 2024-05-10T11:21:26.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.530+0000 7f37997fa640 1 -- 172.21.15.100:0/1446108854 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f378c042030 con 0x7f379c103490 2024-05-10T11:21:26.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.530+0000 7f37a51a7740 1 Processor -- start 2024-05-10T11:21:26.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.530+0000 7f37a51a7740 1 -- start start 2024-05-10T11:21:26.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.530+0000 7f37a51a7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55936c491db0 0x55936c4921d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:26.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.530+0000 7f37a51a7740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55936c4927a0 0x55936c4972a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:26.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.530+0000 7f37a51a7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55936c497910 0x55936c499da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:26.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.530+0000 7f37a51a7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55936c3f65d0 con 0x55936c497910 2024-05-10T11:21:26.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.530+0000 7f37a51a7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55936c3af890 con 0x55936c491db0 2024-05-10T11:21:26.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.530+0000 7f37a51a7740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55936c3f2590 con 0x55936c4927a0 2024-05-10T11:21:26.538 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.530+0000 7f37a0a20640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55936c491db0 0x55936c4921d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:26.538 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.530+0000 7f379bfff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55936c4927a0 0x55936c4972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:26.538 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.530+0000 7f379b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55936c497910 0x55936c499da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:26.538 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.530+0000 7f37a0a20640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55936c491db0 0x55936c4921d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:36278/0 (socket says 172.21.15.100:36278) 2024-05-10T11:21:26.538 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.530+0000 7f37a0a20640 1 -- 172.21.15.100:0/226927989 learned_addr learned my addr 172.21.15.100:0/226927989 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:21:26.538 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.531+0000 7f37a0a20640 1 -- 172.21.15.100:0/226927989 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55936c4927a0 msgr2=0x55936c4972a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.531+0000 7f37a0a20640 1 --2- 172.21.15.100:0/226927989 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55936c4927a0 0x55936c4972a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.531+0000 7f37a0a20640 1 -- 172.21.15.100:0/226927989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55936c497910 msgr2=0x55936c499da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.531+0000 7f37997fa640 1 --2- 172.21.15.100:0/1446108854 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f37840b28a0 0x7f37840b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.531+0000 7f37a0a20640 1 --2- 172.21.15.100:0/226927989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55936c497910 0x55936c499da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.531+0000 7f37a0a20640 1 -- 172.21.15.100:0/226927989 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55936c108210 con 0x55936c491db0 2024-05-10T11:21:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.531+0000 7f37997fa640 1 -- 172.21.15.100:0/1446108854 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f378c102b10 con 0x7f379c103490 2024-05-10T11:21:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.531+0000 7f37a0a20640 1 --2- 172.21.15.100:0/1446108854 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f37840b28a0 0x7f37840b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.531+0000 7f37a0a20640 1 --2- 172.21.15.100:0/226927989 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55936c491db0 0x55936c4921d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3794009f50 tx=0x7f37940118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5175b101c59a8abc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.531+0000 7f37827fc640 1 -- 172.21.15.100:0/226927989 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3794004250 con 0x55936c491db0 2024-05-10T11:21:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.531+0000 7f37827fc640 1 -- 172.21.15.100:0/226927989 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f379400fe30 con 0x55936c491db0 2024-05-10T11:21:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.531+0000 7f37827fc640 1 -- 172.21.15.100:0/226927989 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f379401b9e0 con 0x55936c491db0 2024-05-10T11:21:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.532+0000 7f37a51a7740 1 -- 172.21.15.100:0/226927989 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55936c491db0 msgr2=0x55936c4921d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.532+0000 7f37a0a20640 1 --2- 172.21.15.100:0/1446108854 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f37840b28a0 0x7f37840b4d60 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3794012fd0 tx=0x7f3794021000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.532+0000 7f37a51a7740 1 --2- 172.21.15.100:0/226927989 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55936c491db0 0x55936c4921d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3794009f50 tx=0x7f37940118d0 comp rx=0 tx=0).stop 2024-05-10T11:21:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.532+0000 7f37a51a7740 1 -- 172.21.15.100:0/226927989 shutdown_connections 2024-05-10T11:21:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.532+0000 7f37a51a7740 1 --2- 172.21.15.100:0/226927989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55936c497910 0x55936c499da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.532+0000 7f37a51a7740 1 --2- 172.21.15.100:0/226927989 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55936c4927a0 0x55936c4972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.532+0000 7f37a51a7740 1 --2- 172.21.15.100:0/226927989 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55936c491db0 0x55936c4921d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.532+0000 7f37a51a7740 1 -- 172.21.15.100:0/226927989 >> 172.21.15.100:0/226927989 conn(0x55936c48d420 msgr2=0x55936c48d820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.532+0000 7f37a51a7740 1 -- 172.21.15.100:0/226927989 shutdown_connections 2024-05-10T11:21:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.532+0000 7f37a51a7740 1 -- 172.21.15.100:0/226927989 wait complete. 2024-05-10T11:21:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.532+0000 7f37a51a7740 1 -- start start 2024-05-10T11:21:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.533+0000 7f37a51a7740 1 Processor -- start 2024-05-10T11:21:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.533+0000 7f37a51a7740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55936c491db0 0x55936c49c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.533+0000 7f37a51a7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55936c4927a0 0x55936c49c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.533+0000 7f37a51a7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55936c497910 0x55936c4a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.533+0000 7f37a51a7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55936c49a8c0 con 0x55936c497910 2024-05-10T11:21:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.533+0000 7f37a51a7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55936c3b3370 con 0x55936c4927a0 2024-05-10T11:21:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.533+0000 7f37a51a7740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55936c49aa40 con 0x55936c491db0 2024-05-10T11:21:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.533+0000 7f37a51a7740 10 client.0 _ll_register_callbacks cb 0x55936c107a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:21:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.533+0000 7f37a0a20640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55936c497910 0x55936c4a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:26.541 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.533+0000 7f37a0a20640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55936c497910 0x55936c4a35d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36192/0 (socket says 172.21.15.100:36192) 2024-05-10T11:21:26.541 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.533+0000 7f37a0a20640 1 -- 172.21.15.100:0/999416022 learned_addr learned my addr 172.21.15.100:0/999416022 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:21:26.541 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.533+0000 7f379bfff640 1 --2- 172.21.15.100:0/999416022 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55936c491db0 0x55936c49c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:26.541 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.533+0000 7f379b7fe640 1 --2- 172.21.15.100:0/999416022 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55936c4927a0 0x55936c49c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:26.541 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.533+0000 7f37a0a20640 1 -- 172.21.15.100:0/999416022 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55936c491db0 msgr2=0x55936c49c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:26.541 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.533+0000 7f37a0a20640 1 --2- 172.21.15.100:0/999416022 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55936c491db0 0x55936c49c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:26.541 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.533+0000 7f37a0a20640 1 -- 172.21.15.100:0/999416022 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55936c4927a0 msgr2=0x55936c49c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:26.541 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.533+0000 7f37a0a20640 1 --2- 172.21.15.100:0/999416022 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55936c4927a0 0x55936c49c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:26.541 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.533+0000 7f37a0a20640 1 -- 172.21.15.100:0/999416022 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55936c1d4620 con 0x55936c497910 2024-05-10T11:21:26.541 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.533+0000 7f379b7fe640 1 --2- 172.21.15.100:0/999416022 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55936c4927a0 0x55936c49c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:21:26.541 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.533+0000 7f379bfff640 1 --2- 172.21.15.100:0/999416022 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55936c491db0 0x55936c49c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:21:26.541 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.534+0000 7f37a0a20640 1 --2- 172.21.15.100:0/999416022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55936c497910 0x55936c4a35d0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f3794003f70 tx=0x7f37940039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=849dce9496560800 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:26.541 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.534+0000 7f377b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:21:26.541 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.534+0000 7f377b7fe640 1 -- 172.21.15.100:0/999416022 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3794009be0 con 0x55936c497910 2024-05-10T11:21:26.542 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.534+0000 7f377b7fe640 1 -- 172.21.15.100:0/999416022 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f379400f920 con 0x55936c497910 2024-05-10T11:21:26.542 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.534+0000 7f37a51a7740 1 -- 172.21.15.100:0/999416022 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f379400fe30 con 0x55936c497910 2024-05-10T11:21:26.542 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.534+0000 7f377b7fe640 1 -- 172.21.15.100:0/999416022 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f379403d050 con 0x55936c497910 2024-05-10T11:21:26.542 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.534+0000 7f37a51a7740 10 client.15960 fetch_fsmap learned FSMap version 29 2024-05-10T11:21:26.542 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.534+0000 7f37a51a7740 1 -- 172.21.15.100:0/999416022 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55936c4a65b0 con 0x55936c497910 2024-05-10T11:21:26.542 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.534+0000 7f377b7fe640 1 -- 172.21.15.100:0/999416022 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f379400fe30 con 0x55936c497910 2024-05-10T11:21:26.542 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.534+0000 7f377b7fe640 1 -- 172.21.15.100:0/999416022 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f379401bce0 con 0x55936c497910 2024-05-10T11:21:26.542 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.534+0000 7f37a51a7740 10 client.15960 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T11:21:26.542 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.534+0000 7f37a51a7740 10 client.15960 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:21:26.542 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.534+0000 7f37a51a7740 20 client.15960 populate_metadata read hostname 'smithi100' 2024-05-10T11:21:26.542 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.534+0000 7f37a51a7740 1 --2- 172.21.15.100:0/999416022 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55936c4a7b60 0x55936c4aa000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:26.542 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.534+0000 7f37a51a7740 4 client.15960 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:21:26.542 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.534+0000 7f37a51a7740 1 -- 172.21.15.100:0/999416022 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55936c4041e0 con 0x55936c4a7b60 2024-05-10T11:21:26.542 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.535+0000 7f379bfff640 1 --2- 172.21.15.100:0/999416022 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55936c4a7b60 0x55936c4aa000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:26.542 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.535+0000 7f377b7fe640 10 client.15960 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:21:26.543 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.535+0000 7f379bfff640 1 --2- 172.21.15.100:0/999416022 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55936c4a7b60 0x55936c4aa000 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a61f85d7a69cae62 server_cookie=9a1c1ab0edcab3c9 in_seq=0 out_seq=0 2024-05-10T11:21:26.543 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.536+0000 7f377b7fe640 1 -- 172.21.15.100:0/999416022 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55936c4041e0 con 0x55936c4a7b60 2024-05-10T11:21:26.543 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.536+0000 7f377b7fe640 10 client.15960 handle_command_reply: tid=0 2024-05-10T11:21:26.543 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.536+0000 7f37a51a7740 1 client.15960 shutdown 2024-05-10T11:21:26.543 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.536+0000 7f37a51a7740 1 -- 172.21.15.100:0/999416022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55936c497910 msgr2=0x55936c4a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:26.543 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.536+0000 7f37a51a7740 1 --2- 172.21.15.100:0/999416022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55936c497910 0x55936c4a35d0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f3794003f70 tx=0x7f37940039d0 comp rx=0 tx=0).stop 2024-05-10T11:21:26.543 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.536+0000 7f37a51a7740 1 -- 172.21.15.100:0/999416022 shutdown_connections 2024-05-10T11:21:26.543 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.536+0000 7f37a51a7740 1 --2- 172.21.15.100:0/999416022 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55936c4a7b60 0x55936c4aa000 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:26.543 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.536+0000 7f37a51a7740 1 --2- 172.21.15.100:0/999416022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55936c497910 0x55936c4a35d0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:26.543 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.536+0000 7f37a51a7740 1 --2- 172.21.15.100:0/999416022 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55936c4927a0 0x55936c49c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:26.543 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.536+0000 7f377b7fe640 0 client.15960 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:21:26.543 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.536+0000 7f37a51a7740 1 --2- 172.21.15.100:0/999416022 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55936c491db0 0x55936c49c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:26.543 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.536+0000 7f37a51a7740 1 -- 172.21.15.100:0/999416022 >> 172.21.15.100:0/999416022 conn(0x55936c48d420 msgr2=0x55936c48c580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:26.543 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.536+0000 7f37a51a7740 1 -- 172.21.15.100:0/999416022 shutdown_connections 2024-05-10T11:21:26.543 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.536+0000 7f37a51a7740 1 -- 172.21.15.100:0/999416022 wait complete. 2024-05-10T11:21:26.543 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.536+0000 7f37a51a7740 20 client.15960 trim_cache size 0 max 16384 2024-05-10T11:21:26.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:21:26 smithi100 ceph-mon[30296]: pgmap v511: 129 pgs: 129 active+clean; 32 MiB data, 592 MiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 1.3 MiB/s wr, 146 op/s 2024-05-10T11:21:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.552+0000 7f37a51a7740 1 Processor -- start 2024-05-10T11:21:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.552+0000 7f37a51a7740 1 -- start start 2024-05-10T11:21:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.552+0000 7f37a51a7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55936c491db0 0x55936c490330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.552+0000 7f37a51a7740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55936c4927a0 0x55936c490870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.552+0000 7f37a51a7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55936c497910 0x55936c4ae910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.552+0000 7f37a51a7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f379c1adbc0 con 0x55936c497910 2024-05-10T11:21:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.552+0000 7f37a51a7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f379c1ab490 con 0x55936c491db0 2024-05-10T11:21:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.552+0000 7f37a51a7740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f379c1a92e0 con 0x55936c4927a0 2024-05-10T11:21:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.552+0000 7f37a0a20640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55936c497910 0x55936c4ae910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.552+0000 7f379b7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55936c4927a0 0x55936c490870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.552+0000 7f37a0a20640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55936c497910 0x55936c4ae910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36206/0 (socket says 172.21.15.100:36206) 2024-05-10T11:21:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.552+0000 7f37a0a20640 1 -- 172.21.15.100:0/3718827296 learned_addr learned my addr 172.21.15.100:0/3718827296 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:21:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.552+0000 7f379bfff640 1 --2- 172.21.15.100:0/3718827296 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55936c491db0 0x55936c490330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.552+0000 7f37a0a20640 1 -- 172.21.15.100:0/3718827296 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55936c4927a0 msgr2=0x55936c490870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.552+0000 7f37a0a20640 1 --2- 172.21.15.100:0/3718827296 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55936c4927a0 0x55936c490870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.552+0000 7f37a0a20640 1 -- 172.21.15.100:0/3718827296 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55936c491db0 msgr2=0x55936c490330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.552+0000 7f37a0a20640 1 --2- 172.21.15.100:0/3718827296 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55936c491db0 0x55936c490330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.552+0000 7f37a0a20640 1 -- 172.21.15.100:0/3718827296 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55936c48d250 con 0x55936c497910 2024-05-10T11:21:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.552+0000 7f379bfff640 1 --2- 172.21.15.100:0/3718827296 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55936c491db0 0x55936c490330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:21:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.552+0000 7f37a0a20640 1 --2- 172.21.15.100:0/3718827296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55936c497910 0x55936c4ae910 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f3794006010 tx=0x7f3794003a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51d0760052a5b82a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.553+0000 7f3781ffb640 1 -- 172.21.15.100:0/3718827296 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f379401b430 con 0x55936c497910 2024-05-10T11:21:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.553+0000 7f3781ffb640 1 -- 172.21.15.100:0/3718827296 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3794046070 con 0x55936c497910 2024-05-10T11:21:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.553+0000 7f3781ffb640 1 -- 172.21.15.100:0/3718827296 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f379400f850 con 0x55936c497910 2024-05-10T11:21:26.558 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.553+0000 7f37a51a7740 1 -- 172.21.15.100:0/3718827296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55936c497910 msgr2=0x55936c4ae910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:26.558 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.553+0000 7f37a51a7740 1 --2- 172.21.15.100:0/3718827296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55936c497910 0x55936c4ae910 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f3794006010 tx=0x7f3794003a40 comp rx=0 tx=0).stop 2024-05-10T11:21:26.558 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.553+0000 7f37a51a7740 1 -- 172.21.15.100:0/3718827296 shutdown_connections 2024-05-10T11:21:26.558 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.553+0000 7f37a51a7740 1 --2- 172.21.15.100:0/3718827296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55936c497910 0x55936c4ae910 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:26.558 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.553+0000 7f37a51a7740 1 --2- 172.21.15.100:0/3718827296 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55936c4927a0 0x55936c490870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:26.558 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.553+0000 7f37a51a7740 1 --2- 172.21.15.100:0/3718827296 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55936c491db0 0x55936c490330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:26.558 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.553+0000 7f37a51a7740 1 -- 172.21.15.100:0/3718827296 >> 172.21.15.100:0/3718827296 conn(0x55936c48d420 msgr2=0x55936c493a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:26.558 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.553+0000 7f37a51a7740 1 -- 172.21.15.100:0/3718827296 shutdown_connections 2024-05-10T11:21:26.558 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.554+0000 7f37a51a7740 1 -- 172.21.15.100:0/3718827296 wait complete. 2024-05-10T11:21:26.558 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.554+0000 7f37a51a7740 1 -- start start 2024-05-10T11:21:26.558 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.554+0000 7f37a51a7740 1 Processor -- start 2024-05-10T11:21:26.559 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.554+0000 7f37a51a7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55936c491db0 0x55936c4a0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:26.559 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.554+0000 7f37a51a7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55936c4927a0 0x55936c4a0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:26.559 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.554+0000 7f37a51a7740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55936c497910 0x55936c4a1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:26.559 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.554+0000 7f37a51a7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f379c1a9950 con 0x55936c4927a0 2024-05-10T11:21:26.559 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.554+0000 7f37a51a7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f379c1aadc0 con 0x55936c491db0 2024-05-10T11:21:26.559 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.554+0000 7f37a51a7740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f379c1aa020 con 0x55936c497910 2024-05-10T11:21:26.559 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.554+0000 7f37a51a7740 10 client.0 _ll_register_callbacks cb 0x7f379000b810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:21:26.559 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.554+0000 7f379b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55936c4927a0 0x55936c4a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:26.559 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.554+0000 7f379b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55936c4927a0 0x55936c4a0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36210/0 (socket says 172.21.15.100:36210) 2024-05-10T11:21:26.559 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.554+0000 7f379b7fe640 1 -- 172.21.15.100:0/1964836374 learned_addr learned my addr 172.21.15.100:0/1964836374 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:21:26.559 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.554+0000 7f37a0a20640 1 --2- 172.21.15.100:0/1964836374 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55936c497910 0x55936c4a1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:26.559 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.554+0000 7f379bfff640 1 --2- 172.21.15.100:0/1964836374 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55936c491db0 0x55936c4a0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:26.559 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.555+0000 7f379b7fe640 1 -- 172.21.15.100:0/1964836374 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55936c497910 msgr2=0x55936c4a1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.555+0000 7f379b7fe640 1 --2- 172.21.15.100:0/1964836374 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55936c497910 0x55936c4a1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.555+0000 7f379b7fe640 1 -- 172.21.15.100:0/1964836374 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55936c491db0 msgr2=0x55936c4a0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.555+0000 7f379b7fe640 1 --2- 172.21.15.100:0/1964836374 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55936c491db0 0x55936c4a0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.555+0000 7f379b7fe640 1 -- 172.21.15.100:0/1964836374 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55936c4a1b80 con 0x55936c4927a0 2024-05-10T11:21:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.555+0000 7f379bfff640 1 --2- 172.21.15.100:0/1964836374 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55936c491db0 0x55936c4a0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:21:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.555+0000 7f379b7fe640 1 --2- 172.21.15.100:0/1964836374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55936c4927a0 0x55936c4a0580 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f3790005fd0 tx=0x7f3790012d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2b48fcc4dca9f3c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.555+0000 7f377b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:21:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.555+0000 7f377b7fe640 1 -- 172.21.15.100:0/1964836374 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3790021030 con 0x55936c4927a0 2024-05-10T11:21:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.555+0000 7f377b7fe640 1 -- 172.21.15.100:0/1964836374 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f3790010bd0 con 0x55936c4927a0 2024-05-10T11:21:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.555+0000 7f37a51a7740 1 -- 172.21.15.100:0/1964836374 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3794046070 con 0x55936c4927a0 2024-05-10T11:21:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.555+0000 7f377b7fe640 1 -- 172.21.15.100:0/1964836374 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f379002b050 con 0x55936c4927a0 2024-05-10T11:21:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.555+0000 7f37a51a7740 10 client.15966 fetch_fsmap learned FSMap version 29 2024-05-10T11:21:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.555+0000 7f37a51a7740 1 -- 172.21.15.100:0/1964836374 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55936c4a2f00 con 0x55936c4927a0 2024-05-10T11:21:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.555+0000 7f377b7fe640 1 -- 172.21.15.100:0/1964836374 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3794046070 con 0x55936c4927a0 2024-05-10T11:21:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.556+0000 7f377b7fe640 1 -- 172.21.15.100:0/1964836374 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f3790026400 con 0x55936c4927a0 2024-05-10T11:21:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.556+0000 7f37a51a7740 10 client.15966 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T11:21:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.556+0000 7f37a51a7740 10 client.15966 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:21:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.556+0000 7f37a51a7740 20 client.15966 populate_metadata read hostname 'smithi100' 2024-05-10T11:21:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.556+0000 7f37a51a7740 1 --2- 172.21.15.100:0/1964836374 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55936c4a7b60 0x55936c514cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.556+0000 7f37a51a7740 4 client.15966 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:21:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.556+0000 7f37a51a7740 1 -- 172.21.15.100:0/1964836374 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55936c3fe050 con 0x55936c4a7b60 2024-05-10T11:21:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.556+0000 7f379bfff640 1 --2- 172.21.15.100:0/1964836374 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55936c4a7b60 0x55936c514cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.556+0000 7f377b7fe640 10 client.15966 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:21:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.556+0000 7f379bfff640 1 --2- 172.21.15.100:0/1964836374 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55936c4a7b60 0x55936c514cc0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da1b768f999f65d7 server_cookie=bf3da14c515d96f8 in_seq=0 out_seq=0 2024-05-10T11:21:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.575+0000 7f377b7fe640 1 -- 172.21.15.100:0/1964836374 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55936c3fe050 con 0x55936c4a7b60 2024-05-10T11:21:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.575+0000 7f377b7fe640 10 client.15966 handle_command_reply: tid=0 2024-05-10T11:21:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.575+0000 7f37a51a7740 1 client.15966 shutdown 2024-05-10T11:21:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.575+0000 7f37a51a7740 1 -- 172.21.15.100:0/1964836374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55936c4927a0 msgr2=0x55936c4a0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.575+0000 7f37a51a7740 1 --2- 172.21.15.100:0/1964836374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55936c4927a0 0x55936c4a0580 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f3790005fd0 tx=0x7f3790012d10 comp rx=0 tx=0).stop 2024-05-10T11:21:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.575+0000 7f37a51a7740 1 -- 172.21.15.100:0/1964836374 shutdown_connections 2024-05-10T11:21:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.575+0000 7f37a51a7740 1 --2- 172.21.15.100:0/1964836374 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55936c4a7b60 0x55936c514cc0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.575+0000 7f37a51a7740 1 --2- 172.21.15.100:0/1964836374 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55936c497910 0x55936c4a1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.575+0000 7f37a51a7740 1 --2- 172.21.15.100:0/1964836374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55936c4927a0 0x55936c4a0580 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.575+0000 7f377b7fe640 0 client.15966 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:21:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.575+0000 7f37a51a7740 1 --2- 172.21.15.100:0/1964836374 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55936c491db0 0x55936c4a0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.575+0000 7f37a51a7740 1 -- 172.21.15.100:0/1964836374 >> 172.21.15.100:0/1964836374 conn(0x55936c48d420 msgr2=0x55936c490330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.576+0000 7f37a51a7740 1 -- 172.21.15.100:0/1964836374 shutdown_connections 2024-05-10T11:21:26.577 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.576+0000 7f37a51a7740 1 -- 172.21.15.100:0/1964836374 wait complete. 2024-05-10T11:21:26.577 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.576+0000 7f37a51a7740 20 client.15966 trim_cache size 0 max 16384 2024-05-10T11:21:26.577 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:21:26.577 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:21:26.577 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:21:26.577 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:21:26.578 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.576+0000 7f37a2483640 1 -- 172.21.15.100:0/1446108854 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f37840b28a0 msgr2=0x7f37840b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:26.578 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.576+0000 7f37a2483640 1 --2- 172.21.15.100:0/1446108854 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f37840b28a0 0x7f37840b4d60 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3794012fd0 tx=0x7f3794021000 comp rx=0 tx=0).stop 2024-05-10T11:21:26.578 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.576+0000 7f37a2483640 1 -- 172.21.15.100:0/1446108854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f379c103490 msgr2=0x7f379c198ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:26.578 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.576+0000 7f37a2483640 1 --2- 172.21.15.100:0/1446108854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f379c103490 0x7f379c198ad0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f378c009a20 tx=0x7f378c0040c0 comp rx=0 tx=0).stop 2024-05-10T11:21:26.578 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.576+0000 7f37a2483640 1 -- 172.21.15.100:0/1446108854 shutdown_connections 2024-05-10T11:21:26.578 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.576+0000 7f37a2483640 1 --2- 172.21.15.100:0/1446108854 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f37840b28a0 0x7f37840b4d60 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:26.578 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.576+0000 7f37a2483640 1 --2- 172.21.15.100:0/1446108854 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f379c1086f0 0x7f379c1a0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:26.578 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.576+0000 7f37a2483640 1 --2- 172.21.15.100:0/1446108854 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f379c105dc0 0x7f379c199010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:26.578 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.576+0000 7f37a2483640 1 --2- 172.21.15.100:0/1446108854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f379c103490 0x7f379c198ad0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:26.578 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.576+0000 7f37a2483640 1 -- 172.21.15.100:0/1446108854 >> 172.21.15.100:0/1446108854 conn(0x7f379c0fce80 msgr2=0x7f379c104010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:26.579 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.576+0000 7f37a2483640 1 -- 172.21.15.100:0/1446108854 shutdown_connections 2024-05-10T11:21:26.579 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:26.576+0000 7f37a2483640 1 -- 172.21.15.100:0/1446108854 wait complete. 2024-05-10T11:21:26.934 INFO:tasks.workunit.client.0.smithi100.stdout: 1 43209 3.09 MB/sec warmup 81 sec latency 17.546 ms 2024-05-10T11:21:27.933 INFO:tasks.workunit.client.0.smithi100.stdout: 1 43772 3.08 MB/sec warmup 82 sec latency 15.469 ms 2024-05-10T11:21:28.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:21:28 smithi178 ceph-mon[33359]: pgmap v512: 129 pgs: 129 active+clean; 33 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 1.4 MiB/s wr, 156 op/s 2024-05-10T11:21:28.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:21:28 smithi188 ceph-mon[34075]: pgmap v512: 129 pgs: 129 active+clean; 33 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 1.4 MiB/s wr, 156 op/s 2024-05-10T11:21:28.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:21:28 smithi100 ceph-mon[30296]: pgmap v512: 129 pgs: 129 active+clean; 33 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 1.4 MiB/s wr, 156 op/s 2024-05-10T11:21:28.933 INFO:tasks.workunit.client.0.smithi100.stdout: 1 44408 3.08 MB/sec warmup 83 sec latency 32.236 ms 2024-05-10T11:21:29.933 INFO:tasks.workunit.client.0.smithi100.stdout: 1 45011 3.07 MB/sec warmup 84 sec latency 16.060 ms 2024-05-10T11:21:30.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:21:30 smithi178 ceph-mon[33359]: pgmap v513: 129 pgs: 129 active+clean; 34 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 1.4 MiB/s wr, 151 op/s 2024-05-10T11:21:30.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:21:30 smithi188 ceph-mon[34075]: pgmap v513: 129 pgs: 129 active+clean; 34 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 1.4 MiB/s wr, 151 op/s 2024-05-10T11:21:30.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:21:30 smithi100 ceph-mon[30296]: pgmap v513: 129 pgs: 129 active+clean; 34 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 1.4 MiB/s wr, 151 op/s 2024-05-10T11:21:30.934 INFO:tasks.workunit.client.0.smithi100.stdout: 1 45804 3.10 MB/sec warmup 85 sec latency 13.868 ms 2024-05-10T11:21:31.933 INFO:tasks.workunit.client.0.smithi100.stdout: 1 46263 3.12 MB/sec warmup 86 sec latency 15.464 ms 2024-05-10T11:21:32.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:21:32 smithi178 ceph-mon[33359]: pgmap v514: 129 pgs: 129 active+clean; 34 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 1.3 MiB/s wr, 158 op/s 2024-05-10T11:21:32.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:21:32 smithi188 ceph-mon[34075]: pgmap v514: 129 pgs: 129 active+clean; 34 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 1.3 MiB/s wr, 158 op/s 2024-05-10T11:21:32.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:21:32 smithi100 ceph-mon[30296]: pgmap v514: 129 pgs: 129 active+clean; 34 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 1.3 MiB/s wr, 158 op/s 2024-05-10T11:21:32.934 INFO:tasks.workunit.client.0.smithi100.stdout: 1 46587 3.10 MB/sec warmup 87 sec latency 11.955 ms 2024-05-10T11:21:33.934 INFO:tasks.workunit.client.0.smithi100.stdout: 1 46981 3.07 MB/sec warmup 88 sec latency 25.523 ms 2024-05-10T11:21:34.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:21:34 smithi178 ceph-mon[33359]: pgmap v515: 129 pgs: 129 active+clean; 34 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 1.3 MiB/s wr, 150 op/s 2024-05-10T11:21:34.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:21:34 smithi188 ceph-mon[34075]: pgmap v515: 129 pgs: 129 active+clean; 34 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 1.3 MiB/s wr, 150 op/s 2024-05-10T11:21:34.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:21:34 smithi100 ceph-mon[30296]: pgmap v515: 129 pgs: 129 active+clean; 34 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 1.3 MiB/s wr, 150 op/s 2024-05-10T11:21:34.934 INFO:tasks.workunit.client.0.smithi100.stdout: 1 47597 3.09 MB/sec warmup 89 sec latency 16.688 ms 2024-05-10T11:21:35.934 INFO:tasks.workunit.client.0.smithi100.stdout: 1 48213 3.06 MB/sec warmup 90 sec latency 17.987 ms 2024-05-10T11:21:36.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:21:36 smithi178 ceph-mon[33359]: pgmap v516: 129 pgs: 129 active+clean; 36 MiB data, 613 MiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 1.4 MiB/s wr, 159 op/s 2024-05-10T11:21:36.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:21:36 smithi188 ceph-mon[34075]: pgmap v516: 129 pgs: 129 active+clean; 36 MiB data, 613 MiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 1.4 MiB/s wr, 159 op/s 2024-05-10T11:21:36.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:21:36 smithi100 ceph-mon[30296]: pgmap v516: 129 pgs: 129 active+clean; 36 MiB data, 613 MiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 1.4 MiB/s wr, 159 op/s 2024-05-10T11:21:36.934 INFO:tasks.workunit.client.0.smithi100.stdout: 1 48761 3.05 MB/sec warmup 91 sec latency 16.103 ms 2024-05-10T11:21:37.934 INFO:tasks.workunit.client.0.smithi100.stdout: 1 49472 3.09 MB/sec warmup 92 sec latency 11.793 ms 2024-05-10T11:21:38.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:21:38 smithi178 ceph-mon[33359]: pgmap v517: 129 pgs: 129 active+clean; 37 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 1.5 MiB/s wr, 169 op/s 2024-05-10T11:21:38.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:21:38 smithi188 ceph-mon[34075]: pgmap v517: 129 pgs: 129 active+clean; 37 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 1.5 MiB/s wr, 169 op/s 2024-05-10T11:21:38.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:21:38 smithi100 ceph-mon[30296]: pgmap v517: 129 pgs: 129 active+clean; 37 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 1.5 MiB/s wr, 169 op/s 2024-05-10T11:21:38.934 INFO:tasks.workunit.client.0.smithi100.stdout: 1 49970 3.10 MB/sec warmup 93 sec latency 13.432 ms 2024-05-10T11:21:39.592 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T11:21:39.674 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.673+0000 7f6761693640 1 -- 172.21.15.100:0/822930247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f675c1022a0 msgr2=0x7f675c102720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:39.675 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.673+0000 7f6761693640 1 --2- 172.21.15.100:0/822930247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f675c1022a0 0x7f675c102720 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f6744009900 tx=0x7f674402a410 comp rx=0 tx=0).stop 2024-05-10T11:21:39.675 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.673+0000 7f6761693640 1 -- 172.21.15.100:0/822930247 shutdown_connections 2024-05-10T11:21:39.675 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.673+0000 7f6761693640 1 --2- 172.21.15.100:0/822930247 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f675c102c60 0x7f675c1094f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:39.675 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.673+0000 7f6761693640 1 --2- 172.21.15.100:0/822930247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f675c1022a0 0x7f675c102720 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:39.675 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.673+0000 7f6761693640 1 --2- 172.21.15.100:0/822930247 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f675c1010b0 0x7f675c1014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:39.675 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.673+0000 7f6761693640 1 -- 172.21.15.100:0/822930247 >> 172.21.15.100:0/822930247 conn(0x7f675c0fc620 msgr2=0x7f675c0fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:39.675 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.673+0000 7f6761693640 1 -- 172.21.15.100:0/822930247 shutdown_connections 2024-05-10T11:21:39.675 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.673+0000 7f6761693640 1 -- 172.21.15.100:0/822930247 wait complete. 2024-05-10T11:21:39.675 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.673+0000 7f6761693640 1 Processor -- start 2024-05-10T11:21:39.675 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.673+0000 7f6761693640 1 -- start start 2024-05-10T11:21:39.675 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.673+0000 7f6761693640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f675c1010b0 0x7f675c104ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:39.676 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.674+0000 7f6761693640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f675c1022a0 0x7f675c103680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:39.676 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.674+0000 7f6761693640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f675c102c60 0x7f675c103bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:39.676 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.674+0000 7f6761693640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f675c10c270 con 0x7f675c1010b0 2024-05-10T11:21:39.676 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.674+0000 7f6761693640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f675c10c0f0 con 0x7f675c102c60 2024-05-10T11:21:39.676 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.674+0000 7f6761693640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f675c10c3f0 con 0x7f675c1022a0 2024-05-10T11:21:39.676 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.674+0000 7f675affd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f675c1010b0 0x7f675c104ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:39.676 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.674+0000 7f6753fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f675c1022a0 0x7f675c103680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:39.676 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.674+0000 7f675affd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f675c1010b0 0x7f675c104ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33472/0 (socket says 172.21.15.100:33472) 2024-05-10T11:21:39.676 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.674+0000 7f675affd640 1 -- 172.21.15.100:0/1348674235 learned_addr learned my addr 172.21.15.100:0/1348674235 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:21:39.676 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.674+0000 7f675b7fe640 1 --2- 172.21.15.100:0/1348674235 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f675c102c60 0x7f675c103bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:39.676 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.674+0000 7f6753fff640 1 -- 172.21.15.100:0/1348674235 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f675c102c60 msgr2=0x7f675c103bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:39.676 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.674+0000 7f6753fff640 1 --2- 172.21.15.100:0/1348674235 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f675c102c60 0x7f675c103bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:39.676 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.674+0000 7f6753fff640 1 -- 172.21.15.100:0/1348674235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f675c1010b0 msgr2=0x7f675c104ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:39.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.674+0000 7f6753fff640 1 --2- 172.21.15.100:0/1348674235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f675c1010b0 0x7f675c104ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:39.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.674+0000 7f6753fff640 1 -- 172.21.15.100:0/1348674235 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f675c104480 con 0x7f675c1022a0 2024-05-10T11:21:39.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.674+0000 7f675affd640 1 --2- 172.21.15.100:0/1348674235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f675c1010b0 0x7f675c104ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:21:39.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.674+0000 7f675b7fe640 1 --2- 172.21.15.100:0/1348674235 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f675c102c60 0x7f675c103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:21:39.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.674+0000 7f6753fff640 1 --2- 172.21.15.100:0/1348674235 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f675c1022a0 0x7f675c103680 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f674402a930 tx=0x7f6744003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:39.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.674+0000 7f6758ff9640 1 -- 172.21.15.100:0/1348674235 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f674403f070 con 0x7f675c1022a0 2024-05-10T11:21:39.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.674+0000 7f6761693640 1 -- 172.21.15.100:0/1348674235 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f675c1a2c00 con 0x7f675c1022a0 2024-05-10T11:21:39.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.675+0000 7f6758ff9640 1 -- 172.21.15.100:0/1348674235 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6744031ba0 con 0x7f675c1022a0 2024-05-10T11:21:39.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.675+0000 7f6758ff9640 1 -- 172.21.15.100:0/1348674235 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6744035030 con 0x7f675c1022a0 2024-05-10T11:21:39.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.675+0000 7f6761693640 1 -- 172.21.15.100:0/1348674235 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f675c1a3110 con 0x7f675c1022a0 2024-05-10T11:21:39.684 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.677+0000 7f67643b4740 1 Processor -- start 2024-05-10T11:21:39.684 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.677+0000 7f67643b4740 1 -- start start 2024-05-10T11:21:39.684 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.678+0000 7f67643b4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560348781db0 0x5603487821d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:39.684 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.678+0000 7f67643b4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5603487827a0 0x5603487872a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:39.684 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.678+0000 7f67643b4740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560348787910 0x560348789da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:39.684 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.678+0000 7f67643b4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5603486e65d0 con 0x5603487827a0 2024-05-10T11:21:39.684 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.678+0000 7f67643b4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56034869f890 con 0x560348781db0 2024-05-10T11:21:39.685 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.678+0000 7f67643b4740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5603486e2590 con 0x560348787910 2024-05-10T11:21:39.685 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.678+0000 7f6758ff9640 1 -- 172.21.15.100:0/1348674235 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f6744042070 con 0x7f675c1022a0 2024-05-10T11:21:39.685 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.678+0000 7f675affd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5603487827a0 0x5603487872a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:39.685 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.678+0000 7f675affd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5603487827a0 0x5603487872a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33478/0 (socket says 172.21.15.100:33478) 2024-05-10T11:21:39.685 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.678+0000 7f675affd640 1 -- 172.21.15.100:0/3052485042 learned_addr learned my addr 172.21.15.100:0/3052485042 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:21:39.685 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.678+0000 7f675b7fe640 1 --2- 172.21.15.100:0/3052485042 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560348781db0 0x5603487821d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:39.685 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.678+0000 7f6753fff640 1 --2- 172.21.15.100:0/3052485042 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560348787910 0x560348789da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:39.685 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.678+0000 7f675affd640 1 -- 172.21.15.100:0/3052485042 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560348787910 msgr2=0x560348789da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:39.685 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.678+0000 7f675affd640 1 --2- 172.21.15.100:0/3052485042 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560348787910 0x560348789da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:39.685 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.678+0000 7f675affd640 1 -- 172.21.15.100:0/3052485042 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560348781db0 msgr2=0x5603487821d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:39.685 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.678+0000 7f675affd640 1 --2- 172.21.15.100:0/3052485042 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560348781db0 0x5603487821d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:39.685 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.678+0000 7f675affd640 1 -- 172.21.15.100:0/3052485042 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603483f8210 con 0x5603487827a0 2024-05-10T11:21:39.685 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.678+0000 7f6753fff640 1 --2- 172.21.15.100:0/3052485042 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560348787910 0x560348789da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:21:39.686 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.679+0000 7f675affd640 1 --2- 172.21.15.100:0/3052485042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5603487827a0 0x5603487872a0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f674c0115f0 tx=0x7f674c0119f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4e311edfbd2bf88 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:39.686 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.679+0000 7f67517fa640 1 -- 172.21.15.100:0/3052485042 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f674c01e070 con 0x5603487827a0 2024-05-10T11:21:39.686 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.679+0000 7f67517fa640 1 -- 172.21.15.100:0/3052485042 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f674c00fa70 con 0x5603487827a0 2024-05-10T11:21:39.686 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.679+0000 7f67517fa640 1 -- 172.21.15.100:0/3052485042 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f674c019a00 con 0x5603487827a0 2024-05-10T11:21:39.686 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.679+0000 7f6758ff9640 1 --2- 172.21.15.100:0/1348674235 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f67480b28a0 0x7f67480b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:39.686 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.679+0000 7f675b7fe640 1 --2- 172.21.15.100:0/1348674235 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f67480b28a0 0x7f67480b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:39.686 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.679+0000 7f6758ff9640 1 -- 172.21.15.100:0/1348674235 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f6744102ba0 con 0x7f675c1022a0 2024-05-10T11:21:39.686 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.680+0000 7f67643b4740 1 -- 172.21.15.100:0/3052485042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5603487827a0 msgr2=0x5603487872a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:39.686 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.680+0000 7f67643b4740 1 --2- 172.21.15.100:0/3052485042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5603487827a0 0x5603487872a0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f674c0115f0 tx=0x7f674c0119f0 comp rx=0 tx=0).stop 2024-05-10T11:21:39.686 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.680+0000 7f675b7fe640 1 --2- 172.21.15.100:0/1348674235 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f67480b28a0 0x7f67480b4d60 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x560348783000 tx=0x7f6754010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:39.686 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.680+0000 7f67643b4740 1 -- 172.21.15.100:0/3052485042 shutdown_connections 2024-05-10T11:21:39.687 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.680+0000 7f67643b4740 1 --2- 172.21.15.100:0/3052485042 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560348787910 0x560348789da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:39.687 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.680+0000 7f67643b4740 1 --2- 172.21.15.100:0/3052485042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5603487827a0 0x5603487872a0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:39.687 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.680+0000 7f67643b4740 1 --2- 172.21.15.100:0/3052485042 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560348781db0 0x5603487821d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:39.687 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.680+0000 7f67643b4740 1 -- 172.21.15.100:0/3052485042 >> 172.21.15.100:0/3052485042 conn(0x56034877d420 msgr2=0x56034877d820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:39.687 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.680+0000 7f67643b4740 1 -- 172.21.15.100:0/3052485042 shutdown_connections 2024-05-10T11:21:39.687 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.680+0000 7f67643b4740 1 -- 172.21.15.100:0/3052485042 wait complete. 2024-05-10T11:21:39.687 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.680+0000 7f67643b4740 1 -- start start 2024-05-10T11:21:39.687 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.680+0000 7f67643b4740 1 Processor -- start 2024-05-10T11:21:39.687 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.681+0000 7f67643b4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560348781db0 0x56034878c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:39.687 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.681+0000 7f67643b4740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5603487827a0 0x56034878c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:39.687 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.681+0000 7f67643b4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560348787910 0x560348793610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:39.687 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.681+0000 7f67643b4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56034878a8c0 con 0x560348781db0 2024-05-10T11:21:39.687 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.681+0000 7f67643b4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5603486a3370 con 0x560348787910 2024-05-10T11:21:39.688 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.681+0000 7f67643b4740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56034878aa40 con 0x5603487827a0 2024-05-10T11:21:39.688 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.681+0000 7f67643b4740 10 client.0 _ll_register_callbacks cb 0x5603483f7a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:21:39.688 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.681+0000 7f6753fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5603487827a0 0x56034878c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:39.688 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.681+0000 7f675affd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560348781db0 0x56034878c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:39.688 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.681+0000 7f6753fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5603487827a0 0x56034878c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:49432/0 (socket says 172.21.15.100:49432) 2024-05-10T11:21:39.688 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.681+0000 7f6753fff640 1 -- 172.21.15.100:0/3459782906 learned_addr learned my addr 172.21.15.100:0/3459782906 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:21:39.688 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.681+0000 7f675b7fe640 1 --2- 172.21.15.100:0/3459782906 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560348787910 0x560348793610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:39.688 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.681+0000 7f6753fff640 1 -- 172.21.15.100:0/3459782906 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560348787910 msgr2=0x560348793610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:39.688 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.681+0000 7f6753fff640 1 --2- 172.21.15.100:0/3459782906 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560348787910 0x560348793610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:39.688 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.681+0000 7f6753fff640 1 -- 172.21.15.100:0/3459782906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560348781db0 msgr2=0x56034878c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:39.688 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.681+0000 7f6753fff640 1 --2- 172.21.15.100:0/3459782906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560348781db0 0x56034878c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:39.688 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.681+0000 7f6753fff640 1 -- 172.21.15.100:0/3459782906 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603484c4620 con 0x5603487827a0 2024-05-10T11:21:39.688 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.681+0000 7f6753fff640 1 --2- 172.21.15.100:0/3459782906 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5603487827a0 0x56034878c5d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f67440c9c10 tx=0x7f67440c5e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21cbe0fa190f9c3b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:39.689 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.681+0000 7f67427fc640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:21:39.689 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.681+0000 7f67427fc640 1 -- 172.21.15.100:0/3459782906 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6744049070 con 0x5603487827a0 2024-05-10T11:21:39.689 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.681+0000 7f67427fc640 1 -- 172.21.15.100:0/3459782906 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f6744031d50 con 0x5603487827a0 2024-05-10T11:21:39.689 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.681+0000 7f67643b4740 1 -- 172.21.15.100:0/3459782906 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f674c00fa70 con 0x5603487827a0 2024-05-10T11:21:39.689 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.682+0000 7f67427fc640 1 -- 172.21.15.100:0/3459782906 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f67440e9050 con 0x5603487827a0 2024-05-10T11:21:39.689 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.682+0000 7f67643b4740 10 client.25480 fetch_fsmap learned FSMap version 29 2024-05-10T11:21:39.689 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.682+0000 7f67643b4740 1 -- 172.21.15.100:0/3459782906 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56034878a990 con 0x5603487827a0 2024-05-10T11:21:39.689 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.682+0000 7f67427fc640 1 -- 172.21.15.100:0/3459782906 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f674c00fa70 con 0x5603487827a0 2024-05-10T11:21:39.689 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.682+0000 7f67427fc640 1 -- 172.21.15.100:0/3459782906 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f67440c61a0 con 0x5603487827a0 2024-05-10T11:21:39.689 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.682+0000 7f67643b4740 10 client.25480 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T11:21:39.689 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.682+0000 7f67643b4740 10 client.25480 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:21:39.689 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.682+0000 7f67643b4740 20 client.25480 populate_metadata read hostname 'smithi100' 2024-05-10T11:21:39.689 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.682+0000 7f67643b4740 1 --2- 172.21.15.100:0/3459782906 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x560348797e30 0x56034879a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:39.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.682+0000 7f67643b4740 4 client.25480 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:21:39.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.682+0000 7f67643b4740 1 -- 172.21.15.100:0/3459782906 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603486f41e0 con 0x560348797e30 2024-05-10T11:21:39.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.682+0000 7f675affd640 1 --2- 172.21.15.100:0/3459782906 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x560348797e30 0x56034879a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:39.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.683+0000 7f67427fc640 10 client.25480 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:21:39.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.683+0000 7f675affd640 1 --2- 172.21.15.100:0/3459782906 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x560348797e30 0x56034879a2d0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89338c8409d0826f server_cookie=93e7fbd3f6eeaa8a in_seq=0 out_seq=0 2024-05-10T11:21:39.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.683+0000 7f67427fc640 1 -- 172.21.15.100:0/3459782906 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5603486f41e0 con 0x560348797e30 2024-05-10T11:21:39.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.683+0000 7f67427fc640 10 client.25480 handle_command_reply: tid=0 2024-05-10T11:21:39.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.683+0000 7f67643b4740 1 client.25480 shutdown 2024-05-10T11:21:39.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.683+0000 7f67643b4740 1 -- 172.21.15.100:0/3459782906 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5603487827a0 msgr2=0x56034878c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:39.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.683+0000 7f67643b4740 1 --2- 172.21.15.100:0/3459782906 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5603487827a0 0x56034878c5d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f67440c9c10 tx=0x7f67440c5e90 comp rx=0 tx=0).stop 2024-05-10T11:21:39.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.684+0000 7f67643b4740 1 -- 172.21.15.100:0/3459782906 shutdown_connections 2024-05-10T11:21:39.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.684+0000 7f67643b4740 1 --2- 172.21.15.100:0/3459782906 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x560348797e30 0x56034879a2d0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:39.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.684+0000 7f67643b4740 1 --2- 172.21.15.100:0/3459782906 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560348787910 0x560348793610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:39.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.684+0000 7f67643b4740 1 --2- 172.21.15.100:0/3459782906 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5603487827a0 0x56034878c5d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:39.691 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.684+0000 7f67427fc640 0 client.25480 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:21:39.691 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.684+0000 7f67643b4740 1 --2- 172.21.15.100:0/3459782906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560348781db0 0x56034878c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:39.691 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.684+0000 7f67643b4740 1 -- 172.21.15.100:0/3459782906 >> 172.21.15.100:0/3459782906 conn(0x56034877d420 msgr2=0x56034877c580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:39.691 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.684+0000 7f67643b4740 1 -- 172.21.15.100:0/3459782906 shutdown_connections 2024-05-10T11:21:39.691 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.684+0000 7f67643b4740 1 -- 172.21.15.100:0/3459782906 wait complete. 2024-05-10T11:21:39.691 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.684+0000 7f67643b4740 20 client.25480 trim_cache size 0 max 16384 2024-05-10T11:21:39.703 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.702+0000 7f67643b4740 1 Processor -- start 2024-05-10T11:21:39.703 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.702+0000 7f67643b4740 1 -- start start 2024-05-10T11:21:39.703 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.702+0000 7f67643b4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560348781db0 0x5603487783e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:39.703 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.702+0000 7f67643b4740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5603487827a0 0x5603487839c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:39.703 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.702+0000 7f67643b4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560348787910 0x560348783f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:39.703 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.702+0000 7f67643b4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f675c1afd40 con 0x560348781db0 2024-05-10T11:21:39.704 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.702+0000 7f67643b4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f675c1ad610 con 0x560348787910 2024-05-10T11:21:39.704 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.702+0000 7f67643b4740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f675c1ab460 con 0x5603487827a0 2024-05-10T11:21:39.704 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.702+0000 7f675affd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560348781db0 0x5603487783e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:39.704 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.702+0000 7f675affd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560348781db0 0x5603487783e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33490/0 (socket says 172.21.15.100:33490) 2024-05-10T11:21:39.704 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.703+0000 7f675affd640 1 -- 172.21.15.100:0/2841793476 learned_addr learned my addr 172.21.15.100:0/2841793476 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:21:39.704 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.703+0000 7f675b7fe640 1 --2- 172.21.15.100:0/2841793476 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560348787910 0x560348783f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:39.704 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.703+0000 7f6753fff640 1 --2- 172.21.15.100:0/2841793476 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5603487827a0 0x5603487839c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:39.704 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.703+0000 7f675affd640 1 -- 172.21.15.100:0/2841793476 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5603487827a0 msgr2=0x5603487839c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:39.704 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.703+0000 7f675affd640 1 --2- 172.21.15.100:0/2841793476 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5603487827a0 0x5603487839c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:39.704 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.703+0000 7f675affd640 1 -- 172.21.15.100:0/2841793476 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560348787910 msgr2=0x560348783f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:39.704 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.703+0000 7f675affd640 1 --2- 172.21.15.100:0/2841793476 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560348787910 0x560348783f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:39.704 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.703+0000 7f675affd640 1 -- 172.21.15.100:0/2841793476 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56034878a7a0 con 0x560348781db0 2024-05-10T11:21:39.704 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.703+0000 7f675affd640 1 --2- 172.21.15.100:0/2841793476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560348781db0 0x5603487783e0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f674c0058c0 tx=0x7f674c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d128cc7fd6740409 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:39.705 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.703+0000 7f6750ff9640 1 -- 172.21.15.100:0/2841793476 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f674c01e040 con 0x560348781db0 2024-05-10T11:21:39.705 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.703+0000 7f6750ff9640 1 -- 172.21.15.100:0/2841793476 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f674c00fe80 con 0x560348781db0 2024-05-10T11:21:39.705 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.703+0000 7f6750ff9640 1 -- 172.21.15.100:0/2841793476 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f674c005030 con 0x560348781db0 2024-05-10T11:21:39.708 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.704+0000 7f67643b4740 1 -- 172.21.15.100:0/2841793476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560348781db0 msgr2=0x5603487783e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:39.708 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.704+0000 7f67643b4740 1 --2- 172.21.15.100:0/2841793476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560348781db0 0x5603487783e0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f674c0058c0 tx=0x7f674c000bc0 comp rx=0 tx=0).stop 2024-05-10T11:21:39.708 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.704+0000 7f67643b4740 1 -- 172.21.15.100:0/2841793476 shutdown_connections 2024-05-10T11:21:39.708 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.704+0000 7f67643b4740 1 --2- 172.21.15.100:0/2841793476 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560348787910 0x560348783f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:39.708 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.704+0000 7f67643b4740 1 --2- 172.21.15.100:0/2841793476 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5603487827a0 0x5603487839c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.704+0000 7f67643b4740 1 --2- 172.21.15.100:0/2841793476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560348781db0 0x5603487783e0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.704+0000 7f67643b4740 1 -- 172.21.15.100:0/2841793476 >> 172.21.15.100:0/2841793476 conn(0x56034877d420 msgr2=0x560348787030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.704+0000 7f67643b4740 1 -- 172.21.15.100:0/2841793476 shutdown_connections 2024-05-10T11:21:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.704+0000 7f67643b4740 1 -- 172.21.15.100:0/2841793476 wait complete. 2024-05-10T11:21:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.704+0000 7f67643b4740 1 -- start start 2024-05-10T11:21:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.705+0000 7f67643b4740 1 Processor -- start 2024-05-10T11:21:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.705+0000 7f67643b4740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560348781db0 0x560348790100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.705+0000 7f67643b4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5603487827a0 0x560348790640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.705+0000 7f67643b4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560348787910 0x560348791650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.705+0000 7f67643b4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f675c1abad0 con 0x5603487827a0 2024-05-10T11:21:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.705+0000 7f67643b4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f675c1acf40 con 0x560348787910 2024-05-10T11:21:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.705+0000 7f67643b4740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f675c1ac1a0 con 0x560348781db0 2024-05-10T11:21:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.705+0000 7f67643b4740 10 client.0 _ll_register_callbacks cb 0x7f6744005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:21:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.705+0000 7f6753fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5603487827a0 0x560348790640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.705+0000 7f6753fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5603487827a0 0x560348790640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33500/0 (socket says 172.21.15.100:33500) 2024-05-10T11:21:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.705+0000 7f6753fff640 1 -- 172.21.15.100:0/3148745576 learned_addr learned my addr 172.21.15.100:0/3148745576 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:21:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.705+0000 7f675b7fe640 1 --2- 172.21.15.100:0/3148745576 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560348787910 0x560348791650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.705+0000 7f675affd640 1 --2- 172.21.15.100:0/3148745576 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560348781db0 0x560348790100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.705+0000 7f6753fff640 1 -- 172.21.15.100:0/3148745576 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560348781db0 msgr2=0x560348790100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.705+0000 7f6753fff640 1 --2- 172.21.15.100:0/3148745576 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560348781db0 0x560348790100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.705+0000 7f6753fff640 1 -- 172.21.15.100:0/3148745576 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560348787910 msgr2=0x560348791650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.705+0000 7f6753fff640 1 --2- 172.21.15.100:0/3148745576 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560348787910 0x560348791650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.705+0000 7f6753fff640 1 -- 172.21.15.100:0/3148745576 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560348791c40 con 0x5603487827a0 2024-05-10T11:21:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.705+0000 7f675b7fe640 1 --2- 172.21.15.100:0/3148745576 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560348787910 0x560348791650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:21:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.706+0000 7f6753fff640 1 --2- 172.21.15.100:0/3148745576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5603487827a0 0x560348790640 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f67440d2d60 tx=0x7f6744005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9caa809209164c72 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.706+0000 7f67427fc640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:21:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.706+0000 7f67427fc640 1 -- 172.21.15.100:0/3148745576 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67440a9060 con 0x5603487827a0 2024-05-10T11:21:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.706+0000 7f67427fc640 1 -- 172.21.15.100:0/3148745576 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f67440d2830 con 0x5603487827a0 2024-05-10T11:21:39.711 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.706+0000 7f67643b4740 1 -- 172.21.15.100:0/3148745576 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5603486ee050 con 0x5603487827a0 2024-05-10T11:21:39.711 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.706+0000 7f67427fc640 1 -- 172.21.15.100:0/3148745576 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f67440b0050 con 0x5603487827a0 2024-05-10T11:21:39.711 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.706+0000 7f67643b4740 10 client.15984 fetch_fsmap learned FSMap version 29 2024-05-10T11:21:39.711 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.706+0000 7f67643b4740 1 -- 172.21.15.100:0/3148745576 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560348792fc0 con 0x5603487827a0 2024-05-10T11:21:39.711 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.706+0000 7f67427fc640 1 -- 172.21.15.100:0/3148745576 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x5603486ee050 con 0x5603487827a0 2024-05-10T11:21:39.711 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.706+0000 7f67427fc640 1 -- 172.21.15.100:0/3148745576 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f67440c9170 con 0x5603487827a0 2024-05-10T11:21:39.711 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.706+0000 7f67643b4740 10 client.15984 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T11:21:39.711 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.706+0000 7f67643b4740 10 client.15984 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:21:39.711 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.706+0000 7f67643b4740 20 client.15984 populate_metadata read hostname 'smithi100' 2024-05-10T11:21:39.711 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.706+0000 7f67643b4740 1 --2- 172.21.15.100:0/3148745576 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x560348802e10 0x560348805230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:39.711 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.706+0000 7f67643b4740 4 client.15984 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:21:39.711 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.706+0000 7f67643b4740 1 -- 172.21.15.100:0/3148745576 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5603486695f0 con 0x560348802e10 2024-05-10T11:21:39.711 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.707+0000 7f675affd640 1 --2- 172.21.15.100:0/3148745576 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x560348802e10 0x560348805230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:39.711 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.707+0000 7f67427fc640 10 client.15984 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:21:39.712 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.707+0000 7f675affd640 1 --2- 172.21.15.100:0/3148745576 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x560348802e10 0x560348805230 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2af32a0dd8d00167 server_cookie=83a7fdee05828363 in_seq=0 out_seq=0 2024-05-10T11:21:39.726 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.725+0000 7f67427fc640 1 -- 172.21.15.100:0/3148745576 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5603486695f0 con 0x560348802e10 2024-05-10T11:21:39.726 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.725+0000 7f67427fc640 10 client.15984 handle_command_reply: tid=0 2024-05-10T11:21:39.726 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.725+0000 7f67643b4740 1 client.15984 shutdown 2024-05-10T11:21:39.726 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.725+0000 7f67643b4740 1 -- 172.21.15.100:0/3148745576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5603487827a0 msgr2=0x560348790640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:39.726 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.725+0000 7f67643b4740 1 --2- 172.21.15.100:0/3148745576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5603487827a0 0x560348790640 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f67440d2d60 tx=0x7f6744005fd0 comp rx=0 tx=0).stop 2024-05-10T11:21:39.726 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.725+0000 7f67643b4740 1 -- 172.21.15.100:0/3148745576 shutdown_connections 2024-05-10T11:21:39.726 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.725+0000 7f67643b4740 1 --2- 172.21.15.100:0/3148745576 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x560348802e10 0x560348805230 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:39.726 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.725+0000 7f67643b4740 1 --2- 172.21.15.100:0/3148745576 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560348787910 0x560348791650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:39.726 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.725+0000 7f67427fc640 0 client.15984 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:21:39.726 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.725+0000 7f67643b4740 1 --2- 172.21.15.100:0/3148745576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5603487827a0 0x560348790640 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:39.726 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.725+0000 7f67643b4740 1 --2- 172.21.15.100:0/3148745576 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560348781db0 0x560348790100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:39.726 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.725+0000 7f67643b4740 1 -- 172.21.15.100:0/3148745576 >> 172.21.15.100:0/3148745576 conn(0x56034877d420 msgr2=0x5603487839c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:39.727 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.726+0000 7f67643b4740 1 -- 172.21.15.100:0/3148745576 shutdown_connections 2024-05-10T11:21:39.727 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.726+0000 7f67643b4740 1 -- 172.21.15.100:0/3148745576 wait complete. 2024-05-10T11:21:39.727 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.726+0000 7f67643b4740 20 client.15984 trim_cache size 0 max 16384 2024-05-10T11:21:39.727 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:21:39.727 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:21:39.727 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:21:39.727 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:21:39.728 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.726+0000 7f6761693640 1 -- 172.21.15.100:0/1348674235 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f67480b28a0 msgr2=0x7f67480b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:39.728 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.726+0000 7f6761693640 1 --2- 172.21.15.100:0/1348674235 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f67480b28a0 0x7f67480b4d60 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x560348783000 tx=0x7f6754010040 comp rx=0 tx=0).stop 2024-05-10T11:21:39.728 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.726+0000 7f6761693640 1 -- 172.21.15.100:0/1348674235 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f675c1022a0 msgr2=0x7f675c103680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:39.728 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.726+0000 7f6761693640 1 --2- 172.21.15.100:0/1348674235 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f675c1022a0 0x7f675c103680 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f674402a930 tx=0x7f6744003c10 comp rx=0 tx=0).stop 2024-05-10T11:21:39.728 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.726+0000 7f6761693640 1 -- 172.21.15.100:0/1348674235 shutdown_connections 2024-05-10T11:21:39.728 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.726+0000 7f6761693640 1 --2- 172.21.15.100:0/1348674235 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f67480b28a0 0x7f67480b4d60 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:39.728 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.726+0000 7f6761693640 1 --2- 172.21.15.100:0/1348674235 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f675c102c60 0x7f675c103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:39.729 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.726+0000 7f6761693640 1 --2- 172.21.15.100:0/1348674235 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f675c1022a0 0x7f675c103680 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:39.729 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.726+0000 7f6761693640 1 --2- 172.21.15.100:0/1348674235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f675c1010b0 0x7f675c104ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:39.729 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.726+0000 7f6761693640 1 -- 172.21.15.100:0/1348674235 >> 172.21.15.100:0/1348674235 conn(0x7f675c0fc620 msgr2=0x7f675c1077b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:39.729 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.726+0000 7f6761693640 1 -- 172.21.15.100:0/1348674235 shutdown_connections 2024-05-10T11:21:39.729 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:39.726+0000 7f6761693640 1 -- 172.21.15.100:0/1348674235 wait complete. 2024-05-10T11:21:39.865 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:21:39.934 INFO:tasks.workunit.client.0.smithi100.stdout: 1 50252 3.07 MB/sec warmup 94 sec latency 19.760 ms 2024-05-10T11:21:40.241 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:21:40 smithi100 ceph-mon[30296]: pgmap v518: 129 pgs: 129 active+clean; 36 MiB data, 613 MiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 1.3 MiB/s wr, 159 op/s 2024-05-10T11:21:40.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:21:40 smithi178 ceph-mon[33359]: pgmap v518: 129 pgs: 129 active+clean; 36 MiB data, 613 MiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 1.3 MiB/s wr, 159 op/s 2024-05-10T11:21:40.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:21:40 smithi188 ceph-mon[34075]: pgmap v518: 129 pgs: 129 active+clean; 36 MiB data, 613 MiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 1.3 MiB/s wr, 159 op/s 2024-05-10T11:21:40.934 INFO:tasks.workunit.client.0.smithi100.stdout: 1 50750 3.06 MB/sec warmup 95 sec latency 11.609 ms 2024-05-10T11:21:41.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.684+0000 7f53e02f5700 1 -- 172.21.15.100:0/1424254379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53d8102fe0 msgr2=0x7f53d8106fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:41.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.684+0000 7f53e02f5700 1 --2- 172.21.15.100:0/1424254379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53d8102fe0 0x7f53d8106fe0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f53c80099c0 tx=0x7f53c8009cd0 comp rx=0 tx=0).stop 2024-05-10T11:21:41.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.684+0000 7f53e02f5700 1 -- 172.21.15.100:0/1424254379 shutdown_connections 2024-05-10T11:21:41.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.684+0000 7f53e02f5700 1 --2- 172.21.15.100:0/1424254379 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53d81076c0 0x7f53d810f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:41.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.684+0000 7f53e02f5700 1 --2- 172.21.15.100:0/1424254379 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53d8102630 0x7f53d8102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:41.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.684+0000 7f53e02f5700 1 --2- 172.21.15.100:0/1424254379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53d8102fe0 0x7f53d8106fe0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:41.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.684+0000 7f53e02f5700 1 -- 172.21.15.100:0/1424254379 >> 172.21.15.100:0/1424254379 conn(0x7f53d8079350 msgr2=0x7f53d8079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:41.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.684+0000 7f53e02f5700 1 -- 172.21.15.100:0/1424254379 shutdown_connections 2024-05-10T11:21:41.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.684+0000 7f53e02f5700 1 -- 172.21.15.100:0/1424254379 wait complete. 2024-05-10T11:21:41.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.685+0000 7f53e02f5700 1 Processor -- start 2024-05-10T11:21:41.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.685+0000 7f53e02f5700 1 -- start start 2024-05-10T11:21:41.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.685+0000 7f53e02f5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53d8102630 0x7f53d819e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:41.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.685+0000 7f53e02f5700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53d8102fe0 0x7f53d819ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:41.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.685+0000 7f53e02f5700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53d81076c0 0x7f53d81988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:41.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.685+0000 7f53e02f5700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f53d8112430 con 0x7f53d8102fe0 2024-05-10T11:21:41.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.685+0000 7f53e02f5700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f53d81122b0 con 0x7f53d8102630 2024-05-10T11:21:41.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.685+0000 7f53e02f5700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f53d81125b0 con 0x7f53d81076c0 2024-05-10T11:21:41.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.685+0000 7f53dd890700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53d8102fe0 0x7f53d819ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:41.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.685+0000 7f53dd890700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53d8102fe0 0x7f53d819ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33524/0 (socket says 172.21.15.100:33524) 2024-05-10T11:21:41.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.685+0000 7f53dd890700 1 -- 172.21.15.100:0/3527767414 learned_addr learned my addr 172.21.15.100:0/3527767414 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:21:41.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.685+0000 7f53de091700 1 --2- 172.21.15.100:0/3527767414 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53d8102630 0x7f53d819e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:41.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.685+0000 7f53de892700 1 --2- 172.21.15.100:0/3527767414 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53d81076c0 0x7f53d81988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:41.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.686+0000 7f53dd890700 1 -- 172.21.15.100:0/3527767414 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53d81076c0 msgr2=0x7f53d81988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:41.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.686+0000 7f53dd890700 1 --2- 172.21.15.100:0/3527767414 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53d81076c0 0x7f53d81988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:41.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.686+0000 7f53dd890700 1 -- 172.21.15.100:0/3527767414 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53d8102630 msgr2=0x7f53d819e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:41.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.686+0000 7f53dd890700 1 --2- 172.21.15.100:0/3527767414 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53d8102630 0x7f53d819e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:41.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.686+0000 7f53dd890700 1 -- 172.21.15.100:0/3527767414 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53d8199040 con 0x7f53d8102fe0 2024-05-10T11:21:41.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.686+0000 7f53dd890700 1 --2- 172.21.15.100:0/3527767414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53d8102fe0 0x7f53d819ec10 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f53c80055f0 tx=0x7f53c8005620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:41.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.686+0000 7f53cf7fe700 1 -- 172.21.15.100:0/3527767414 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c8026070 con 0x7f53d8102fe0 2024-05-10T11:21:41.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.686+0000 7f53cf7fe700 1 -- 172.21.15.100:0/3527767414 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f53c800f460 con 0x7f53d8102fe0 2024-05-10T11:21:41.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.686+0000 7f53e02f5700 1 -- 172.21.15.100:0/3527767414 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53d81992d0 con 0x7f53d8102fe0 2024-05-10T11:21:41.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.686+0000 7f53cf7fe700 1 -- 172.21.15.100:0/3527767414 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c8009e60 con 0x7f53d8102fe0 2024-05-10T11:21:41.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.686+0000 7f53e02f5700 1 -- 172.21.15.100:0/3527767414 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53d806bbd0 con 0x7f53d8102fe0 2024-05-10T11:21:41.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.689+0000 7f53cf7fe700 1 -- 172.21.15.100:0/3527767414 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f53c8029070 con 0x7f53d8102fe0 2024-05-10T11:21:41.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.690+0000 7f53cf7fe700 1 --2- 172.21.15.100:0/3527767414 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f53c40b2880 0x7f53c40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:41.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.690+0000 7f53eab88b80 1 Processor -- start 2024-05-10T11:21:41.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.690+0000 7f53de091700 1 --2- 172.21.15.100:0/3527767414 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f53c40b2880 0x7f53c40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:41.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.690+0000 7f53cf7fe700 1 -- 172.21.15.100:0/3527767414 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f53c80e2cb0 con 0x7f53d8102fe0 2024-05-10T11:21:41.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.690+0000 7f53eab88b80 1 -- start start 2024-05-10T11:21:41.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.691+0000 7f53eab88b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c86e9a52b0 0x55c86e9a5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:41.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.691+0000 7f53eab88b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c86e9a5bd0 0x55c86e99fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:41.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.691+0000 7f53eab88b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c86e9a0120 0x55c86e9a0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:41.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.691+0000 7f53eab88b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55c86e8e9860 con 0x55c86e9a52b0 2024-05-10T11:21:41.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.691+0000 7f53eab88b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c86e8887c0 con 0x55c86e9a0120 2024-05-10T11:21:41.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.691+0000 7f53eab88b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c86e8e5820 con 0x55c86e9a5bd0 2024-05-10T11:21:41.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.691+0000 7f53de091700 1 --2- 172.21.15.100:0/3527767414 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f53c40b2880 0x7f53c40b4d40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f53d81072a0 tx=0x7f53d400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:41.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.691+0000 7f53de091700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c86e9a52b0 0x55c86e9a5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:41.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.691+0000 7f53de091700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c86e9a52b0 0x55c86e9a5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33538/0 (socket says 172.21.15.100:33538) 2024-05-10T11:21:41.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.691+0000 7f53de091700 1 -- 172.21.15.100:0/4267693820 learned_addr learned my addr 172.21.15.100:0/4267693820 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:21:41.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.691+0000 7f53dd890700 1 --2- 172.21.15.100:0/4267693820 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c86e9a5bd0 0x55c86e99fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:41.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.691+0000 7f53de892700 1 --2- 172.21.15.100:0/4267693820 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c86e9a0120 0x55c86e9a0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:41.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.691+0000 7f53de091700 1 -- 172.21.15.100:0/4267693820 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c86e9a5bd0 msgr2=0x55c86e99fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:41.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.691+0000 7f53de091700 1 --2- 172.21.15.100:0/4267693820 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c86e9a5bd0 0x55c86e99fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:41.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.691+0000 7f53de091700 1 -- 172.21.15.100:0/4267693820 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c86e9a0120 msgr2=0x55c86e9a0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:41.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.691+0000 7f53de091700 1 --2- 172.21.15.100:0/4267693820 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c86e9a0120 0x55c86e9a0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:41.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.691+0000 7f53de091700 1 -- 172.21.15.100:0/4267693820 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c86e8368d0 con 0x55c86e9a52b0 2024-05-10T11:21:41.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.691+0000 7f53dd890700 1 --2- 172.21.15.100:0/4267693820 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c86e9a5bd0 0x55c86e99fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:21:41.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.691+0000 7f53de892700 1 --2- 172.21.15.100:0/4267693820 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c86e9a0120 0x55c86e9a0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:21:41.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.692+0000 7f53de091700 1 --2- 172.21.15.100:0/4267693820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c86e9a52b0 0x55c86e9a5690 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f53d4010f70 tx=0x7f53d4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcab1f5b5a8655ca server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:41.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.692+0000 7f53ccd70700 1 -- 172.21.15.100:0/4267693820 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d400be60 con 0x55c86e9a52b0 2024-05-10T11:21:41.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.692+0000 7f53ccd70700 1 -- 172.21.15.100:0/4267693820 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f53d4011db0 con 0x55c86e9a52b0 2024-05-10T11:21:41.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.692+0000 7f53ccd70700 1 -- 172.21.15.100:0/4267693820 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d401d630 con 0x55c86e9a52b0 2024-05-10T11:21:41.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.693+0000 7f53eab88b80 1 -- 172.21.15.100:0/4267693820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c86e9a52b0 msgr2=0x55c86e9a5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:41.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.693+0000 7f53eab88b80 1 --2- 172.21.15.100:0/4267693820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c86e9a52b0 0x55c86e9a5690 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f53d4010f70 tx=0x7f53d4013000 comp rx=0 tx=0).stop 2024-05-10T11:21:41.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.693+0000 7f53eab88b80 1 -- 172.21.15.100:0/4267693820 shutdown_connections 2024-05-10T11:21:41.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.693+0000 7f53eab88b80 1 --2- 172.21.15.100:0/4267693820 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c86e9a0120 0x55c86e9a0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:41.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.693+0000 7f53eab88b80 1 --2- 172.21.15.100:0/4267693820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c86e9a52b0 0x55c86e9a5690 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:41.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.693+0000 7f53eab88b80 1 --2- 172.21.15.100:0/4267693820 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c86e9a5bd0 0x55c86e99fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:41.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.693+0000 7f53eab88b80 1 -- 172.21.15.100:0/4267693820 >> 172.21.15.100:0/4267693820 conn(0x55c86e8c3100 msgr2=0x55c86e991c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:41.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.693+0000 7f53eab88b80 1 -- 172.21.15.100:0/4267693820 shutdown_connections 2024-05-10T11:21:41.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.693+0000 7f53eab88b80 1 -- 172.21.15.100:0/4267693820 wait complete. 2024-05-10T11:21:41.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.693+0000 7f53eab88b80 1 -- start start 2024-05-10T11:21:41.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.693+0000 7f53eab88b80 1 Processor -- start 2024-05-10T11:21:41.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.694+0000 7f53eab88b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c86e9a0120 0x55c86e9ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:41.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.694+0000 7f53eab88b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c86e9a52b0 0x55c86e9a8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:41.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.694+0000 7f53eab88b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c86e9a5bd0 0x55c86e9a91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:41.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.694+0000 7f53eab88b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55c86e9ac680 con 0x55c86e9a52b0 2024-05-10T11:21:41.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.694+0000 7f53eab88b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c86e84fc30 con 0x55c86e9a5bd0 2024-05-10T11:21:41.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.694+0000 7f53eab88b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c86e9ac800 con 0x55c86e9a0120 2024-05-10T11:21:41.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.694+0000 7f53de091700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c86e9a0120 0x55c86e9ab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:41.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.694+0000 7f53dd890700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c86e9a52b0 0x55c86e9a8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:41.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.694+0000 7f53de091700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c86e9a0120 0x55c86e9ab8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:49496/0 (socket says 172.21.15.100:49496) 2024-05-10T11:21:41.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.694+0000 7f53de091700 1 -- 172.21.15.100:0/1864941059 learned_addr learned my addr 172.21.15.100:0/1864941059 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:21:41.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.694+0000 7f53de892700 1 --2- 172.21.15.100:0/1864941059 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c86e9a5bd0 0x55c86e9a91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:41.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.694+0000 7f53eab88b80 10 client.0 _ll_register_callbacks cb 0x55c86e881920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:21:41.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.694+0000 7f53de091700 1 -- 172.21.15.100:0/1864941059 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c86e9a5bd0 msgr2=0x55c86e9a91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:41.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.694+0000 7f53de091700 1 --2- 172.21.15.100:0/1864941059 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c86e9a5bd0 0x55c86e9a91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:41.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.694+0000 7f53de091700 1 -- 172.21.15.100:0/1864941059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c86e9a52b0 msgr2=0x55c86e9a8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:41.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.694+0000 7f53de091700 1 --2- 172.21.15.100:0/1864941059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c86e9a52b0 0x55c86e9a8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:41.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.694+0000 7f53de091700 1 -- 172.21.15.100:0/1864941059 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c86e885690 con 0x55c86e9a0120 2024-05-10T11:21:41.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.694+0000 7f53eab88b80 1 -- 172.21.15.100:0/1864941059 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53d4011db0 con 0x55c86e9a0120 2024-05-10T11:21:41.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.694+0000 7f53dd890700 1 --2- 172.21.15.100:0/1864941059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c86e9a52b0 0x55c86e9a8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:21:41.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.694+0000 7f53de091700 1 --2- 172.21.15.100:0/1864941059 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c86e9a0120 0x55c86e9ab8c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f53d4005950 tx=0x7f53d401e650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=803dab4ca07df8c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:41.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.695+0000 7f53c1ffb700 10 client.25498 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:21:41.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.695+0000 7f53c1ffb700 1 -- 172.21.15.100:0/1864941059 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d400be60 con 0x55c86e9a0120 2024-05-10T11:21:41.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.695+0000 7f53c1ffb700 1 -- 172.21.15.100:0/1864941059 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f53d401ead0 con 0x55c86e9a0120 2024-05-10T11:21:41.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.695+0000 7f53c1ffb700 1 -- 172.21.15.100:0/1864941059 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f53d401ed10 con 0x55c86e9a0120 2024-05-10T11:21:41.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.695+0000 7f53eab88b80 10 client.25498 fetch_fsmap learned FSMap version 29 2024-05-10T11:21:41.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.695+0000 7f53eab88b80 1 -- 172.21.15.100:0/1864941059 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({fsmap=29}) -- 0x55c86e9b6d60 con 0x55c86e9a0120 2024-05-10T11:21:41.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.695+0000 7f53c1ffb700 1 -- 172.21.15.100:0/1864941059 <== mon.1 v2:172.21.15.188:3300/0 4 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f53d402f400 con 0x55c86e9a0120 2024-05-10T11:21:41.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.695+0000 7f53eab88b80 10 client.25498 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T11:21:41.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.695+0000 7f53eab88b80 10 client.25498 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:21:41.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.695+0000 7f53eab88b80 20 client.25498 populate_metadata read hostname 'smithi100' 2024-05-10T11:21:41.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.695+0000 7f53eab88b80 1 --2- 172.21.15.100:0/1864941059 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c86e9b9230 0x55c86e9bb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:41.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.695+0000 7f53eab88b80 4 client.25498 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:21:41.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.695+0000 7f53eab88b80 1 -- 172.21.15.100:0/1864941059 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c86e963650 con 0x55c86e9b9230 2024-05-10T11:21:41.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.696+0000 7f53dd890700 1 --2- 172.21.15.100:0/1864941059 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c86e9b9230 0x55c86e9bb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:41.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.696+0000 7f53c1ffb700 10 client.25498 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:21:41.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.696+0000 7f53dd890700 1 --2- 172.21.15.100:0/1864941059 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c86e9b9230 0x55c86e9bb620 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a58bf43f7873cfdb server_cookie=f321ee1a24a9ed0b in_seq=0 out_seq=0 2024-05-10T11:21:41.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.697+0000 7f53c1ffb700 1 -- 172.21.15.100:0/1864941059 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c86e963650 con 0x55c86e9b9230 2024-05-10T11:21:41.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.697+0000 7f53c1ffb700 10 client.25498 handle_command_reply: tid=0 2024-05-10T11:21:41.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.697+0000 7f53eab88b80 1 client.25498 shutdown 2024-05-10T11:21:41.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.697+0000 7f53eab88b80 1 -- 172.21.15.100:0/1864941059 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c86e9a0120 msgr2=0x55c86e9ab8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:41.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.697+0000 7f53eab88b80 1 --2- 172.21.15.100:0/1864941059 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c86e9a0120 0x55c86e9ab8c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f53d4005950 tx=0x7f53d401e650 comp rx=0 tx=0).stop 2024-05-10T11:21:41.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.697+0000 7f53eab88b80 1 -- 172.21.15.100:0/1864941059 shutdown_connections 2024-05-10T11:21:41.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.697+0000 7f53eab88b80 1 --2- 172.21.15.100:0/1864941059 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c86e9b9230 0x55c86e9bb620 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:41.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.697+0000 7f53eab88b80 1 --2- 172.21.15.100:0/1864941059 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c86e9a5bd0 0x55c86e9a91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:41.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.697+0000 7f53eab88b80 1 --2- 172.21.15.100:0/1864941059 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c86e9a0120 0x55c86e9ab8c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:41.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.697+0000 7f53eab88b80 1 --2- 172.21.15.100:0/1864941059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c86e9a52b0 0x55c86e9a8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:41.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.697+0000 7f53eab88b80 1 -- 172.21.15.100:0/1864941059 >> 172.21.15.100:0/1864941059 conn(0x55c86e8c3100 msgr2=0x55c86e98c720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:41.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.697+0000 7f53c1ffb700 0 client.25498 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:21:41.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.697+0000 7f53eab88b80 1 -- 172.21.15.100:0/1864941059 shutdown_connections 2024-05-10T11:21:41.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.697+0000 7f53eab88b80 1 -- 172.21.15.100:0/1864941059 wait complete. 2024-05-10T11:21:41.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.697+0000 7f53eab88b80 20 client.25498 trim_cache size 0 max 16384 2024-05-10T11:21:41.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.720+0000 7f53eab88b80 1 Processor -- start 2024-05-10T11:21:41.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.720+0000 7f53eab88b80 1 -- start start 2024-05-10T11:21:41.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.721+0000 7f53eab88b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c86e9a0120 0x55c86e9a7c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:41.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.721+0000 7f53eab88b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c86e9a52b0 0x55c86e9be640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:41.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.721+0000 7f53eab88b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c86e9a5bd0 0x55c86e9beb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:41.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.721+0000 7f53eab88b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f53d81af730 con 0x55c86e9a52b0 2024-05-10T11:21:41.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.721+0000 7f53eab88b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f53d81acfc0 con 0x55c86e9a5bd0 2024-05-10T11:21:41.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.721+0000 7f53eab88b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f53d81aae10 con 0x55c86e9a0120 2024-05-10T11:21:41.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.721+0000 7f53dd890700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c86e9a52b0 0x55c86e9be640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:41.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.721+0000 7f53dd890700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c86e9a52b0 0x55c86e9be640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33556/0 (socket says 172.21.15.100:33556) 2024-05-10T11:21:41.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.721+0000 7f53dd890700 1 -- 172.21.15.100:0/3111237658 learned_addr learned my addr 172.21.15.100:0/3111237658 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:21:41.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.721+0000 7f53de091700 1 --2- 172.21.15.100:0/3111237658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c86e9a0120 0x55c86e9a7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:41.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.721+0000 7f53de892700 1 --2- 172.21.15.100:0/3111237658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c86e9a5bd0 0x55c86e9beb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:41.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.721+0000 7f53dd890700 1 -- 172.21.15.100:0/3111237658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c86e9a0120 msgr2=0x55c86e9a7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:41.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.721+0000 7f53dd890700 1 --2- 172.21.15.100:0/3111237658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c86e9a0120 0x55c86e9a7c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:41.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.721+0000 7f53dd890700 1 -- 172.21.15.100:0/3111237658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c86e9a5bd0 msgr2=0x55c86e9beb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:41.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.721+0000 7f53dd890700 1 --2- 172.21.15.100:0/3111237658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c86e9a5bd0 0x55c86e9beb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:41.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.721+0000 7f53dd890700 1 -- 172.21.15.100:0/3111237658 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c86e99fe40 con 0x55c86e9a52b0 2024-05-10T11:21:41.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.721+0000 7f53dd890700 1 --2- 172.21.15.100:0/3111237658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c86e9a52b0 0x55c86e9be640 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f53c80e0450 tx=0x7f53c8009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=832ccaab56aa6886 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:41.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.722+0000 7f53c3fff700 1 -- 172.21.15.100:0/3111237658 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c80b2dd0 con 0x55c86e9a52b0 2024-05-10T11:21:41.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.722+0000 7f53c3fff700 1 -- 172.21.15.100:0/3111237658 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f53c800b730 con 0x55c86e9a52b0 2024-05-10T11:21:41.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.722+0000 7f53c3fff700 1 -- 172.21.15.100:0/3111237658 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c80b2940 con 0x55c86e9a52b0 2024-05-10T11:21:41.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.722+0000 7f53eab88b80 1 -- 172.21.15.100:0/3111237658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c86e9a52b0 msgr2=0x55c86e9be640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:41.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.722+0000 7f53eab88b80 1 --2- 172.21.15.100:0/3111237658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c86e9a52b0 0x55c86e9be640 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f53c80e0450 tx=0x7f53c8009310 comp rx=0 tx=0).stop 2024-05-10T11:21:41.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.722+0000 7f53eab88b80 1 -- 172.21.15.100:0/3111237658 shutdown_connections 2024-05-10T11:21:41.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.722+0000 7f53eab88b80 1 --2- 172.21.15.100:0/3111237658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c86e9a5bd0 0x55c86e9beb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:41.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.722+0000 7f53eab88b80 1 --2- 172.21.15.100:0/3111237658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c86e9a0120 0x55c86e9a7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:41.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.722+0000 7f53eab88b80 1 --2- 172.21.15.100:0/3111237658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c86e9a52b0 0x55c86e9be640 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:41.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.722+0000 7f53eab88b80 1 -- 172.21.15.100:0/3111237658 >> 172.21.15.100:0/3111237658 conn(0x55c86e8c3100 msgr2=0x55c86e9a7570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:41.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.722+0000 7f53eab88b80 1 -- 172.21.15.100:0/3111237658 shutdown_connections 2024-05-10T11:21:41.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.722+0000 7f53eab88b80 1 -- 172.21.15.100:0/3111237658 wait complete. 2024-05-10T11:21:41.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.723+0000 7f53eab88b80 1 -- start start 2024-05-10T11:21:41.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.723+0000 7f53eab88b80 1 Processor -- start 2024-05-10T11:21:41.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.723+0000 7f53eab88b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c86e9a0120 0x55c86e9bc980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:41.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.723+0000 7f53eab88b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c86e9a52b0 0x55c86e9bcec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:41.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.723+0000 7f53eab88b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c86e9a5bd0 0x55c86ea1c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:41.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.723+0000 7f53eab88b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f53d81ab480 con 0x55c86e9a0120 2024-05-10T11:21:41.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.723+0000 7f53eab88b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f53d81ac8f0 con 0x55c86e9a5bd0 2024-05-10T11:21:41.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.723+0000 7f53eab88b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f53d81abb50 con 0x55c86e9a52b0 2024-05-10T11:21:41.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.723+0000 7f53eab88b80 10 client.0 _ll_register_callbacks cb 0x55c86e881920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:21:41.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.723+0000 7f53de091700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c86e9a0120 0x55c86e9bc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:41.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.723+0000 7f53dd890700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c86e9a52b0 0x55c86e9bcec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:41.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.723+0000 7f53dd890700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c86e9a52b0 0x55c86e9bcec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:49516/0 (socket says 172.21.15.100:49516) 2024-05-10T11:21:41.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.723+0000 7f53dd890700 1 -- 172.21.15.100:0/3049289540 learned_addr learned my addr 172.21.15.100:0/3049289540 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:21:41.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.723+0000 7f53de091700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c86e9a0120 0x55c86e9bc980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33564/0 (socket says 172.21.15.100:33564) 2024-05-10T11:21:41.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.723+0000 7f53de892700 1 --2- 172.21.15.100:0/3049289540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c86e9a5bd0 0x55c86ea1c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:41.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.724+0000 7f53dd890700 1 -- 172.21.15.100:0/3049289540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c86e9a5bd0 msgr2=0x55c86ea1c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:41.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.724+0000 7f53dd890700 1 --2- 172.21.15.100:0/3049289540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c86e9a5bd0 0x55c86ea1c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:41.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.724+0000 7f53dd890700 1 -- 172.21.15.100:0/3049289540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c86e9a0120 msgr2=0x55c86e9bc980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:41.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.724+0000 7f53dd890700 1 --2- 172.21.15.100:0/3049289540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c86e9a0120 0x55c86e9bc980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:41.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.724+0000 7f53dd890700 1 -- 172.21.15.100:0/3049289540 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c86ea1cd50 con 0x55c86e9a52b0 2024-05-10T11:21:41.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.724+0000 7f53de091700 1 --2- 172.21.15.100:0/3049289540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c86e9a0120 0x55c86e9bc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:21:41.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.724+0000 7f53de892700 1 --2- 172.21.15.100:0/3049289540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c86e9a5bd0 0x55c86ea1c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:21:41.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.724+0000 7f53dd890700 1 --2- 172.21.15.100:0/3049289540 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c86e9a52b0 0x55c86e9bcec0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f53c80a9f90 tx=0x7f53c80b2e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5266c3f667fd23e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:41.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.724+0000 7f53c1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:21:41.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.724+0000 7f53c1ffb700 1 -- 172.21.15.100:0/3049289540 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c80aa460 con 0x55c86e9a52b0 2024-05-10T11:21:41.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.724+0000 7f53eab88b80 1 -- 172.21.15.100:0/3049289540 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53c800b730 con 0x55c86e9a52b0 2024-05-10T11:21:41.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.724+0000 7f53c1ffb700 1 -- 172.21.15.100:0/3049289540 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f53c80b2940 con 0x55c86e9a52b0 2024-05-10T11:21:41.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.724+0000 7f53c1ffb700 1 -- 172.21.15.100:0/3049289540 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f53c80b2b40 con 0x55c86e9a52b0 2024-05-10T11:21:41.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.724+0000 7f53eab88b80 10 client.25504 fetch_fsmap learned FSMap version 29 2024-05-10T11:21:41.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.724+0000 7f53eab88b80 1 -- 172.21.15.100:0/3049289540 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c86ea24160 con 0x55c86e9a52b0 2024-05-10T11:21:41.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.724+0000 7f53c1ffb700 1 -- 172.21.15.100:0/3049289540 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f53c800b730 con 0x55c86e9a52b0 2024-05-10T11:21:41.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.724+0000 7f53c1ffb700 1 -- 172.21.15.100:0/3049289540 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f53c80a8e90 con 0x55c86e9a52b0 2024-05-10T11:21:41.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.724+0000 7f53eab88b80 10 client.25504 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T11:21:41.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.724+0000 7f53eab88b80 10 client.25504 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:21:41.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.724+0000 7f53eab88b80 20 client.25504 populate_metadata read hostname 'smithi100' 2024-05-10T11:21:41.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.724+0000 7f53eab88b80 1 --2- 172.21.15.100:0/3049289540 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c86ea247f0 0x55c86ea26c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:41.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.724+0000 7f53eab88b80 4 client.25504 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T11:21:41.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.724+0000 7f53eab88b80 1 -- 172.21.15.100:0/3049289540 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c86e8f7240 con 0x55c86ea247f0 2024-05-10T11:21:41.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.725+0000 7f53de091700 1 --2- 172.21.15.100:0/3049289540 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c86ea247f0 0x55c86ea26c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:41.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.725+0000 7f53c1ffb700 10 client.25504 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:21:41.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.725+0000 7f53de091700 1 --2- 172.21.15.100:0/3049289540 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c86ea247f0 0x55c86ea26c10 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3940451f6ff3d7a1 server_cookie=ec036687f676ff8a in_seq=0 out_seq=0 2024-05-10T11:21:41.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.726+0000 7f53c1ffb700 1 -- 172.21.15.100:0/3049289540 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c86e8f7240 con 0x55c86ea247f0 2024-05-10T11:21:41.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.726+0000 7f53c1ffb700 10 client.25504 handle_command_reply: tid=0 2024-05-10T11:21:41.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.726+0000 7f53eab88b80 1 client.25504 shutdown 2024-05-10T11:21:41.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.726+0000 7f53eab88b80 1 -- 172.21.15.100:0/3049289540 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c86e9a52b0 msgr2=0x55c86e9bcec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:41.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.726+0000 7f53eab88b80 1 --2- 172.21.15.100:0/3049289540 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c86e9a52b0 0x55c86e9bcec0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f53c80a9f90 tx=0x7f53c80b2e80 comp rx=0 tx=0).stop 2024-05-10T11:21:41.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.726+0000 7f53eab88b80 1 -- 172.21.15.100:0/3049289540 shutdown_connections 2024-05-10T11:21:41.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.726+0000 7f53eab88b80 1 --2- 172.21.15.100:0/3049289540 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c86ea247f0 0x55c86ea26c10 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:41.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.726+0000 7f53eab88b80 1 --2- 172.21.15.100:0/3049289540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c86e9a5bd0 0x55c86ea1c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:41.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.726+0000 7f53c1ffb700 0 client.25504 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:21:41.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.726+0000 7f53eab88b80 1 --2- 172.21.15.100:0/3049289540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c86e9a0120 0x55c86e9bc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:41.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.726+0000 7f53eab88b80 1 --2- 172.21.15.100:0/3049289540 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c86e9a52b0 0x55c86e9bcec0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:41.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.726+0000 7f53eab88b80 1 -- 172.21.15.100:0/3049289540 >> 172.21.15.100:0/3049289540 conn(0x55c86e8c3100 msgr2=0x55c86e98c720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:41.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.726+0000 7f53eab88b80 1 -- 172.21.15.100:0/3049289540 shutdown_connections 2024-05-10T11:21:41.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.726+0000 7f53eab88b80 1 -- 172.21.15.100:0/3049289540 wait complete. 2024-05-10T11:21:41.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.726+0000 7f53eab88b80 20 client.25504 trim_cache size 0 max 16384 2024-05-10T11:21:41.738 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:21:41.738 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-05-10T11:21:41.738 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-05-10T11:21:41.738 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:21:41.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.726+0000 7f53e02f5700 1 -- 172.21.15.100:0/3527767414 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f53c40b2880 msgr2=0x7f53c40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:41.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.726+0000 7f53e02f5700 1 --2- 172.21.15.100:0/3527767414 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f53c40b2880 0x7f53c40b4d40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f53d81072a0 tx=0x7f53d400b410 comp rx=0 tx=0).stop 2024-05-10T11:21:41.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.726+0000 7f53e02f5700 1 -- 172.21.15.100:0/3527767414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53d8102fe0 msgr2=0x7f53d819ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:41.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.726+0000 7f53e02f5700 1 --2- 172.21.15.100:0/3527767414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53d8102fe0 0x7f53d819ec10 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f53c80055f0 tx=0x7f53c8005620 comp rx=0 tx=0).stop 2024-05-10T11:21:41.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.727+0000 7f53e02f5700 1 -- 172.21.15.100:0/3527767414 shutdown_connections 2024-05-10T11:21:41.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.727+0000 7f53e02f5700 1 --2- 172.21.15.100:0/3527767414 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f53c40b2880 0x7f53c40b4d40 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:41.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.727+0000 7f53e02f5700 1 --2- 172.21.15.100:0/3527767414 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53d81076c0 0x7f53d81988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:41.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.727+0000 7f53e02f5700 1 --2- 172.21.15.100:0/3527767414 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53d8102630 0x7f53d819e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:41.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.727+0000 7f53e02f5700 1 --2- 172.21.15.100:0/3527767414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53d8102fe0 0x7f53d819ec10 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:41.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.727+0000 7f53e02f5700 1 -- 172.21.15.100:0/3527767414 >> 172.21.15.100:0/3527767414 conn(0x7f53d8079350 msgr2=0x7f53d8100850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:41.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.727+0000 7f53e02f5700 1 -- 172.21.15.100:0/3527767414 shutdown_connections 2024-05-10T11:21:41.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:41.727+0000 7f53e02f5700 1 -- 172.21.15.100:0/3527767414 wait complete. 2024-05-10T11:21:41.934 INFO:tasks.workunit.client.0.smithi100.stdout: 1 51303 3.06 MB/sec warmup 96 sec latency 42.862 ms 2024-05-10T11:21:42.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:21:42 smithi178 ceph-mon[33359]: pgmap v519: 129 pgs: 129 active+clean; 36 MiB data, 613 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 1.2 MiB/s wr, 157 op/s 2024-05-10T11:21:42.521 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:21:42 smithi100 ceph-mon[30296]: pgmap v519: 129 pgs: 129 active+clean; 36 MiB data, 613 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 1.2 MiB/s wr, 157 op/s 2024-05-10T11:21:42.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:21:42 smithi188 ceph-mon[34075]: pgmap v519: 129 pgs: 129 active+clean; 36 MiB data, 613 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 1.2 MiB/s wr, 157 op/s 2024-05-10T11:21:42.540 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T11:21:42.541 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T11:21:42.820 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:21:42.934 INFO:tasks.workunit.client.0.smithi100.stdout: 1 51995 3.05 MB/sec warmup 97 sec latency 19.076 ms 2024-05-10T11:21:43.934 INFO:tasks.workunit.client.0.smithi100.stdout: 1 52561 3.06 MB/sec warmup 98 sec latency 22.381 ms 2024-05-10T11:21:44.382 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:21:44 smithi100 ceph-mon[30296]: pgmap v520: 129 pgs: 129 active+clean; 38 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 1.3 MiB/s wr, 140 op/s 2024-05-10T11:21:44.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:21:44 smithi178 ceph-mon[33359]: pgmap v520: 129 pgs: 129 active+clean; 38 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 1.3 MiB/s wr, 140 op/s 2024-05-10T11:21:44.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:21:44 smithi188 ceph-mon[34075]: pgmap v520: 129 pgs: 129 active+clean; 38 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 1.3 MiB/s wr, 140 op/s 2024-05-10T11:21:44.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:44.610+0000 7f808fe47700 1 -- 172.21.15.100:0/2307022980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80881037b0 msgr2=0x7f8088107030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:44.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:44.610+0000 7f808fe47700 1 --2- 172.21.15.100:0/2307022980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80881037b0 0x7f8088107030 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f80780099c0 tx=0x7f8078009cd0 comp rx=0 tx=0).stop 2024-05-10T11:21:44.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:44.610+0000 7f808fe47700 1 -- 172.21.15.100:0/2307022980 shutdown_connections 2024-05-10T11:21:44.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:44.610+0000 7f808fe47700 1 --2- 172.21.15.100:0/2307022980 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8088107760 0x7f808810f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:44.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:44.610+0000 7f808fe47700 1 --2- 172.21.15.100:0/2307022980 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8088102e00 0x7f80881031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:44.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:44.610+0000 7f808fe47700 1 --2- 172.21.15.100:0/2307022980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80881037b0 0x7f8088107030 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:44.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:44.610+0000 7f808fe47700 1 -- 172.21.15.100:0/2307022980 >> 172.21.15.100:0/2307022980 conn(0x7f8088078a10 msgr2=0x7f80880ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:44.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:44.611+0000 7f808fe47700 1 -- 172.21.15.100:0/2307022980 shutdown_connections 2024-05-10T11:21:44.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:44.611+0000 7f808fe47700 1 -- 172.21.15.100:0/2307022980 wait complete. 2024-05-10T11:21:44.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:44.611+0000 7f808fe47700 1 Processor -- start 2024-05-10T11:21:44.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:44.611+0000 7f808fe47700 1 -- start start 2024-05-10T11:21:44.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:44.612+0000 7f808fe47700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8088102e00 0x7f80880fde80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:44.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:44.612+0000 7f808fe47700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80881037b0 0x7f80880fe3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:44.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:44.612+0000 7f808fe47700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8088107760 0x7f80880fe900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:44.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:44.612+0000 7f808fe47700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8088112420 con 0x7f8088107760 2024-05-10T11:21:44.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:44.612+0000 7f808fe47700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f80881122a0 con 0x7f8088102e00 2024-05-10T11:21:44.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:44.612+0000 7f808fe47700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f80881125a0 con 0x7f80881037b0 2024-05-10T11:21:44.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:44.612+0000 7f808dbe3700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8088102e00 0x7f80880fde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:44.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:44.612+0000 7f808e3e4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8088107760 0x7f80880fe900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:44.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:44.612+0000 7f808e3e4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8088107760 0x7f80880fe900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33594/0 (socket says 172.21.15.100:33594) 2024-05-10T11:21:44.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:44.612+0000 7f808dbe3700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8088102e00 0x7f80880fde80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:50154/0 (socket says 172.21.15.100:50154) 2024-05-10T11:21:44.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:44.612+0000 7f808e3e4700 1 -- 172.21.15.100:0/1150559863 learned_addr learned my addr 172.21.15.100:0/1150559863 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:21:44.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:44.612+0000 7f808d3e2700 1 --2- 172.21.15.100:0/1150559863 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80881037b0 0x7f80880fe3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:44.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:44.612+0000 7f808e3e4700 1 -- 172.21.15.100:0/1150559863 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80881037b0 msgr2=0x7f80880fe3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:44.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:44.612+0000 7f808e3e4700 1 --2- 172.21.15.100:0/1150559863 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80881037b0 0x7f80880fe3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:44.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:44.612+0000 7f808e3e4700 1 -- 172.21.15.100:0/1150559863 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8088102e00 msgr2=0x7f80880fde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:44.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:44.612+0000 7f808e3e4700 1 --2- 172.21.15.100:0/1150559863 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8088102e00 0x7f80880fde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:44.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:44.612+0000 7f808e3e4700 1 -- 172.21.15.100:0/1150559863 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80881a7fa0 con 0x7f8088107760 2024-05-10T11:21:44.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:44.612+0000 7f808dbe3700 1 --2- 172.21.15.100:0/1150559863 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8088102e00 0x7f80880fde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:21:44.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:44.612+0000 7f808e3e4700 1 --2- 172.21.15.100:0/1150559863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8088107760 0x7f80880fe900 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f8080007ee0 tx=0x7f808000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:44.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:44.613+0000 7f807effd700 1 -- 172.21.15.100:0/1150559863 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f808000dd20 con 0x7f8088107760 2024-05-10T11:21:44.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:44.613+0000 7f808fe47700 1 -- 172.21.15.100:0/1150559863 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80881a8290 con 0x7f8088107760 2024-05-10T11:21:44.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:44.613+0000 7f807effd700 1 -- 172.21.15.100:0/1150559863 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8080004020 con 0x7f8088107760 2024-05-10T11:21:44.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:44.613+0000 7f807effd700 1 -- 172.21.15.100:0/1150559863 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8080015660 con 0x7f8088107760 2024-05-10T11:21:44.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:44.613+0000 7f808fe47700 1 -- 172.21.15.100:0/1150559863 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80881a87a0 con 0x7f8088107760 2024-05-10T11:21:44.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:44.614+0000 7f808fe47700 1 -- 172.21.15.100:0/1150559863 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80880ff3b0 con 0x7f8088107760 2024-05-10T11:21:44.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:44.615+0000 7f807effd700 1 -- 172.21.15.100:0/1150559863 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f80800041d0 con 0x7f8088107760 2024-05-10T11:21:44.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:44.616+0000 7f807effd700 1 --2- 172.21.15.100:0/1150559863 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f80740b2880 0x7f80740b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:44.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:44.616+0000 7f808dbe3700 1 --2- 172.21.15.100:0/1150559863 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f80740b2880 0x7f80740b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:44.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:44.616+0000 7f807effd700 1 -- 172.21.15.100:0/1150559863 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f80800e26a0 con 0x7f8088107760 2024-05-10T11:21:44.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:44.617+0000 7f808dbe3700 1 --2- 172.21.15.100:0/1150559863 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f80740b2880 0x7f80740b4d40 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8084011490 tx=0x7f8084009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:44.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:44.620+0000 7f807effd700 1 -- 172.21.15.100:0/1150559863 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f8080010070 con 0x7f8088107760 2024-05-10T11:21:44.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:44.925+0000 7f808fe47700 1 -- 172.21.15.100:0/1150559863 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f808802e630 con 0x7f8088107760 2024-05-10T11:21:44.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:44.926+0000 7f807effd700 1 -- 172.21.15.100:0/1150559863 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11897 (secure 0 0 0) 0x7f80800aa060 con 0x7f8088107760 2024-05-10T11:21:44.926 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:21:44.926 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2072225012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2072225012},{"type":"v1","addr":"172.21.15.100:6835","nonce":2072225012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/401939540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":401939540},{"type":"v1","addr":"172.21.15.100:6837","nonce":401939540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14571,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/3946311025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3946311025},{"type":"v1","addr":"172.21.15.100:6839","nonce":3946311025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3736371660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3736371660},{"type":"v1","addr":"172.21.15.100:6841","nonce":3736371660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/2741810791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2741810791},{"type":"v1","addr":"172.21.15.178:6833","nonce":2741810791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1704891041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1704891041},{"type":"v1","addr":"172.21.15.188:6833","nonce":1704891041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T11:14:27.242452+0000","modified":"2024-05-10T11:20:29.528470+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24439,"mds_1":24437,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/2124076312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2124076312},{"type":"v1","addr":"172.21.15.178:6835","nonce":2124076312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/3224358425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3224358425},{"type":"v1","addr":"172.21.15.178:6837","nonce":3224358425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/515547690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":515547690},{"type":"v1","addr":"172.21.15.188:6835","nonce":515547690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/1943342620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1943342620},{"type":"v1","addr":"172.21.15.188:6837","nonce":1943342620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.178:6839/3760985404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3760985404},{"type":"v1","addr":"172.21.15.178:6839","nonce":3760985404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.188:6839/3194917070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":3194917070},{"type":"v1","addr":"172.21.15.188:6839","nonce":3194917070}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24439,"qdb_cluster":[24437,24439,24427]},"id":1}]} 2024-05-10T11:21:44.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:44.929+0000 7f808fe47700 1 -- 172.21.15.100:0/1150559863 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f80740b2880 msgr2=0x7f80740b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:44.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:44.929+0000 7f808fe47700 1 --2- 172.21.15.100:0/1150559863 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f80740b2880 0x7f80740b4d40 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8084011490 tx=0x7f8084009260 comp rx=0 tx=0).stop 2024-05-10T11:21:44.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:44.929+0000 7f808fe47700 1 -- 172.21.15.100:0/1150559863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8088107760 msgr2=0x7f80880fe900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:44.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:44.929+0000 7f808fe47700 1 --2- 172.21.15.100:0/1150559863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8088107760 0x7f80880fe900 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f8080007ee0 tx=0x7f808000d3b0 comp rx=0 tx=0).stop 2024-05-10T11:21:44.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:44.929+0000 7f808fe47700 1 -- 172.21.15.100:0/1150559863 shutdown_connections 2024-05-10T11:21:44.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:44.929+0000 7f808fe47700 1 --2- 172.21.15.100:0/1150559863 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f80740b2880 0x7f80740b4d40 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:44.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:44.929+0000 7f808fe47700 1 --2- 172.21.15.100:0/1150559863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8088107760 0x7f80880fe900 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:44.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:44.929+0000 7f808fe47700 1 --2- 172.21.15.100:0/1150559863 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8088102e00 0x7f80880fde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:44.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:44.929+0000 7f808fe47700 1 --2- 172.21.15.100:0/1150559863 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80881037b0 0x7f80880fe3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:44.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:44.929+0000 7f808fe47700 1 -- 172.21.15.100:0/1150559863 >> 172.21.15.100:0/1150559863 conn(0x7f8088078a10 msgr2=0x7f8088101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:44.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:44.929+0000 7f808fe47700 1 -- 172.21.15.100:0/1150559863 shutdown_connections 2024-05-10T11:21:44.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:44.929+0000 7f808fe47700 1 -- 172.21.15.100:0/1150559863 wait complete. 2024-05-10T11:21:44.930 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 29 2024-05-10T11:21:44.934 INFO:tasks.workunit.client.0.smithi100.stdout: 1 53039 3.06 MB/sec warmup 99 sec latency 46.443 ms 2024-05-10T11:21:45.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:21:45 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/1150559863' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:21:45.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:21:45 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/1150559863' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:21:45.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:21:45 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/1150559863' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:21:45.721 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-10T11:21:45.934 INFO:tasks.workunit.client.0.smithi100.stdout: 1 53522 3.07 MB/sec warmup 100 sec latency 15.241 ms 2024-05-10T11:21:45.984 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:21:46.346 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:21:46 smithi100 ceph-mon[30296]: pgmap v521: 129 pgs: 129 active+clean; 38 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 1.4 MiB/s wr, 150 op/s 2024-05-10T11:21:46.429 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:21:46 smithi188 ceph-mon[34075]: pgmap v521: 129 pgs: 129 active+clean; 38 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 1.4 MiB/s wr, 150 op/s 2024-05-10T11:21:46.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:21:46 smithi178 ceph-mon[33359]: pgmap v521: 129 pgs: 129 active+clean; 38 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 1.4 MiB/s wr, 150 op/s 2024-05-10T11:21:46.934 INFO:tasks.workunit.client.0.smithi100.stdout: 1 53786 3.05 MB/sec warmup 101 sec latency 16.231 ms 2024-05-10T11:21:47.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.690+0000 7f4bdd096700 1 -- 172.21.15.100:0/2782811697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4bd8103f00 msgr2=0x7f4bd81103b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:47.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.690+0000 7f4bdd096700 1 --2- 172.21.15.100:0/2782811697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4bd8103f00 0x7f4bd81103b0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f4bd000b0d0 tx=0x7f4bd000b3e0 comp rx=0 tx=0).stop 2024-05-10T11:21:47.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.690+0000 7f4bdd096700 1 -- 172.21.15.100:0/2782811697 shutdown_connections 2024-05-10T11:21:47.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.690+0000 7f4bdd096700 1 --2- 172.21.15.100:0/2782811697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4bd8103f00 0x7f4bd81103b0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:47.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.690+0000 7f4bdd096700 1 --2- 172.21.15.100:0/2782811697 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4bd81095e0 0x7f4bd81099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:47.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.690+0000 7f4bdd096700 1 --2- 172.21.15.100:0/2782811697 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bd8103540 0x7f4bd81039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:47.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.690+0000 7f4bdd096700 1 -- 172.21.15.100:0/2782811697 >> 172.21.15.100:0/2782811697 conn(0x7f4bd807a650 msgr2=0x7f4bd807aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:47.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.690+0000 7f4bdd096700 1 -- 172.21.15.100:0/2782811697 shutdown_connections 2024-05-10T11:21:47.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.690+0000 7f4bdd096700 1 -- 172.21.15.100:0/2782811697 wait complete. 2024-05-10T11:21:47.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.691+0000 7f4bdd096700 1 Processor -- start 2024-05-10T11:21:47.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.691+0000 7f4bdd096700 1 -- start start 2024-05-10T11:21:47.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.691+0000 7f4bdd096700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4bd8103540 0x7f4bd810f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:47.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.691+0000 7f4bdd096700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bd8103f00 0x7f4bd810fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:47.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.691+0000 7f4bdd096700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4bd81095e0 0x7f4bd81a48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:47.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.691+0000 7f4bdd096700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd8113590 con 0x7f4bd81095e0 2024-05-10T11:21:47.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.691+0000 7f4bdd096700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd8113410 con 0x7f4bd8103540 2024-05-10T11:21:47.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.691+0000 7f4bdd096700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd8113710 con 0x7f4bd8103f00 2024-05-10T11:21:47.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.692+0000 7f4bd659c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bd8103f00 0x7f4bd810fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:47.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.692+0000 7f4bd759e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4bd81095e0 0x7f4bd81a48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:47.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.692+0000 7f4bd659c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bd8103f00 0x7f4bd810fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:53170/0 (socket says 172.21.15.100:53170) 2024-05-10T11:21:47.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.692+0000 7f4bd659c700 1 -- 172.21.15.100:0/3501590040 learned_addr learned my addr 172.21.15.100:0/3501590040 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:21:47.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.692+0000 7f4bd6d9d700 1 --2- 172.21.15.100:0/3501590040 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4bd8103540 0x7f4bd810f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:47.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.692+0000 7f4bd6d9d700 1 -- 172.21.15.100:0/3501590040 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bd8103f00 msgr2=0x7f4bd810fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:47.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.692+0000 7f4bd6d9d700 1 --2- 172.21.15.100:0/3501590040 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bd8103f00 0x7f4bd810fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:47.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.692+0000 7f4bd6d9d700 1 -- 172.21.15.100:0/3501590040 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4bd81095e0 msgr2=0x7f4bd81a48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:47.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.692+0000 7f4bd6d9d700 1 --2- 172.21.15.100:0/3501590040 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4bd81095e0 0x7f4bd81a48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:47.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.692+0000 7f4bd6d9d700 1 -- 172.21.15.100:0/3501590040 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bd81a4fe0 con 0x7f4bd8103540 2024-05-10T11:21:47.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.692+0000 7f4bd659c700 1 --2- 172.21.15.100:0/3501590040 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bd8103f00 0x7f4bd810fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:21:47.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.692+0000 7f4bd6d9d700 1 --2- 172.21.15.100:0/3501590040 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4bd8103540 0x7f4bd810f670 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4bc800e9f0 tx=0x7f4bc800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:47.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.692+0000 7f4bbffff700 1 -- 172.21.15.100:0/3501590040 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc800c9c0 con 0x7f4bd8103540 2024-05-10T11:21:47.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.692+0000 7f4bbffff700 1 -- 172.21.15.100:0/3501590040 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4bc8010430 con 0x7f4bd8103540 2024-05-10T11:21:47.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.692+0000 7f4bdd096700 1 -- 172.21.15.100:0/3501590040 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bd81a52d0 con 0x7f4bd8103540 2024-05-10T11:21:47.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.692+0000 7f4bbffff700 1 -- 172.21.15.100:0/3501590040 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc8018680 con 0x7f4bd8103540 2024-05-10T11:21:47.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.692+0000 7f4bdd096700 1 -- 172.21.15.100:0/3501590040 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bd81acb50 con 0x7f4bd8103540 2024-05-10T11:21:47.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.695+0000 7f4bbffff700 1 -- 172.21.15.100:0/3501590040 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f4bc8018820 con 0x7f4bd8103540 2024-05-10T11:21:47.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.696+0000 7f4bbffff700 1 --2- 172.21.15.100:0/3501590040 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4bc00b2880 0x7f4bc00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:47.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.696+0000 7f4bd659c700 1 --2- 172.21.15.100:0/3501590040 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4bc00b2880 0x7f4bc00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:47.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.696+0000 7f4bbffff700 1 -- 172.21.15.100:0/3501590040 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f4bc80e26d0 con 0x7f4bd8103540 2024-05-10T11:21:47.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.696+0000 7f4bd659c700 1 --2- 172.21.15.100:0/3501590040 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4bc00b2880 0x7f4bc00b4d40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4bd80777c0 tx=0x7f4bcc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:47.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.697+0000 7f4be7929b80 1 Processor -- start 2024-05-10T11:21:47.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.697+0000 7f4be7929b80 1 -- start start 2024-05-10T11:21:47.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.697+0000 7f4be7929b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d63e5534d0 0x55d63e5538b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:47.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.697+0000 7f4be7929b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d63e553df0 0x55d63e5576f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:47.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.697+0000 7f4be7929b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d63e557c80 0x55d63e558100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:47.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.697+0000 7f4be7929b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55d63e4a5860 con 0x55d63e557c80 2024-05-10T11:21:47.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.697+0000 7f4be7929b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d63e4447c0 con 0x55d63e5534d0 2024-05-10T11:21:47.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.697+0000 7f4be7929b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d63e4a1820 con 0x55d63e553df0 2024-05-10T11:21:47.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.697+0000 7f4bd759e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d63e557c80 0x55d63e558100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:47.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.697+0000 7f4bd659c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d63e553df0 0x55d63e5576f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:47.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.697+0000 7f4bd759e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d63e557c80 0x55d63e558100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42378/0 (socket says 172.21.15.100:42378) 2024-05-10T11:21:47.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.697+0000 7f4bd759e700 1 -- 172.21.15.100:0/1372900556 learned_addr learned my addr 172.21.15.100:0/1372900556 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:21:47.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.697+0000 7f4bd6d9d700 1 --2- 172.21.15.100:0/1372900556 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d63e5534d0 0x55d63e5538b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:47.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.698+0000 7f4bd759e700 1 -- 172.21.15.100:0/1372900556 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d63e553df0 msgr2=0x55d63e5576f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:47.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.698+0000 7f4bd759e700 1 --2- 172.21.15.100:0/1372900556 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d63e553df0 0x55d63e5576f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:47.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.698+0000 7f4bd759e700 1 -- 172.21.15.100:0/1372900556 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d63e5534d0 msgr2=0x55d63e5538b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:47.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.698+0000 7f4bd759e700 1 --2- 172.21.15.100:0/1372900556 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d63e5534d0 0x55d63e5538b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:47.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.698+0000 7f4bd759e700 1 -- 172.21.15.100:0/1372900556 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d63e3f28d0 con 0x55d63e557c80 2024-05-10T11:21:47.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.698+0000 7f4bd659c700 1 --2- 172.21.15.100:0/1372900556 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d63e553df0 0x55d63e5576f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:21:47.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.698+0000 7f4bd759e700 1 --2- 172.21.15.100:0/1372900556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d63e557c80 0x55d63e558100 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0009c30 tx=0x7f4bd0009c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba1c83b1e3a4f38c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:47.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.698+0000 7f4bbd7fa700 1 -- 172.21.15.100:0/1372900556 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd00092e0 con 0x55d63e557c80 2024-05-10T11:21:47.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.698+0000 7f4bbd7fa700 1 -- 172.21.15.100:0/1372900556 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4bd0020070 con 0x55d63e557c80 2024-05-10T11:21:47.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.698+0000 7f4bbd7fa700 1 -- 172.21.15.100:0/1372900556 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd0005410 con 0x55d63e557c80 2024-05-10T11:21:47.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.699+0000 7f4be7929b80 1 -- 172.21.15.100:0/1372900556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d63e557c80 msgr2=0x55d63e558100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:47.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.699+0000 7f4be7929b80 1 --2- 172.21.15.100:0/1372900556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d63e557c80 0x55d63e558100 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0009c30 tx=0x7f4bd0009c60 comp rx=0 tx=0).stop 2024-05-10T11:21:47.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.699+0000 7f4be7929b80 1 -- 172.21.15.100:0/1372900556 shutdown_connections 2024-05-10T11:21:47.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.699+0000 7f4be7929b80 1 --2- 172.21.15.100:0/1372900556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d63e557c80 0x55d63e558100 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:47.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.699+0000 7f4be7929b80 1 --2- 172.21.15.100:0/1372900556 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d63e5534d0 0x55d63e5538b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:47.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.699+0000 7f4be7929b80 1 --2- 172.21.15.100:0/1372900556 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d63e553df0 0x55d63e5576f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:47.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.699+0000 7f4be7929b80 1 -- 172.21.15.100:0/1372900556 >> 172.21.15.100:0/1372900556 conn(0x55d63e47f100 msgr2=0x55d63e54dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:47.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.699+0000 7f4be7929b80 1 -- 172.21.15.100:0/1372900556 shutdown_connections 2024-05-10T11:21:47.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.699+0000 7f4be7929b80 1 -- 172.21.15.100:0/1372900556 wait complete. 2024-05-10T11:21:47.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.699+0000 7f4be7929b80 1 -- start start 2024-05-10T11:21:47.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.700+0000 7f4be7929b80 1 Processor -- start 2024-05-10T11:21:47.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.700+0000 7f4be7929b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d63e5534d0 0x55d63e55eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:47.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.700+0000 7f4be7929b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d63e553df0 0x55d63e55c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:47.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.700+0000 7f4be7929b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d63e557c80 0x55d63e55c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:47.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.700+0000 7f4be7929b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55d63e55fdd0 con 0x55d63e5534d0 2024-05-10T11:21:47.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.700+0000 7f4be7929b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d63e40bc30 con 0x55d63e553df0 2024-05-10T11:21:47.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.700+0000 7f4be7929b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d63e55ff50 con 0x55d63e557c80 2024-05-10T11:21:47.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.700+0000 7f4be7929b80 10 client.0 _ll_register_callbacks cb 0x55d63e43d920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:21:47.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.700+0000 7f4bd6d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d63e5534d0 0x55d63e55eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:47.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.700+0000 7f4bd6d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d63e5534d0 0x55d63e55eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42382/0 (socket says 172.21.15.100:42382) 2024-05-10T11:21:47.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.700+0000 7f4bd6d9d700 1 -- 172.21.15.100:0/3010916035 learned_addr learned my addr 172.21.15.100:0/3010916035 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:21:47.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.700+0000 7f4bd759e700 1 --2- 172.21.15.100:0/3010916035 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d63e557c80 0x55d63e55c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:47.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.700+0000 7f4bd659c700 1 --2- 172.21.15.100:0/3010916035 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d63e553df0 0x55d63e55c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:47.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.700+0000 7f4bd6d9d700 1 -- 172.21.15.100:0/3010916035 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d63e557c80 msgr2=0x55d63e55c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:47.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.700+0000 7f4bd6d9d700 1 --2- 172.21.15.100:0/3010916035 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d63e557c80 0x55d63e55c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:47.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.700+0000 7f4bd6d9d700 1 -- 172.21.15.100:0/3010916035 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d63e553df0 msgr2=0x55d63e55c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:47.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.700+0000 7f4bd6d9d700 1 --2- 172.21.15.100:0/3010916035 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d63e553df0 0x55d63e55c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:47.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.700+0000 7f4bd6d9d700 1 -- 172.21.15.100:0/3010916035 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d63e441690 con 0x55d63e5534d0 2024-05-10T11:21:47.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.700+0000 7f4bd759e700 1 --2- 172.21.15.100:0/3010916035 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d63e557c80 0x55d63e55c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:21:47.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.701+0000 7f4bd6d9d700 1 --2- 172.21.15.100:0/3010916035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d63e5534d0 0x55d63e55eff0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f4bc80dfdf0 tx=0x7f4bc8010b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8af664a840f71d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:47.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.701+0000 7f4bbaff5700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:21:47.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.701+0000 7f4bbaff5700 1 -- 172.21.15.100:0/3010916035 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc80b2110 con 0x55d63e5534d0 2024-05-10T11:21:47.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.701+0000 7f4bbaff5700 1 -- 172.21.15.100:0/3010916035 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f4bc80cc0e0 con 0x55d63e5534d0 2024-05-10T11:21:47.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.701+0000 7f4be7929b80 1 -- 172.21.15.100:0/3010916035 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4bd0020070 con 0x55d63e5534d0 2024-05-10T11:21:47.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.701+0000 7f4bbaff5700 1 -- 172.21.15.100:0/3010916035 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4bc80a9480 con 0x55d63e5534d0 2024-05-10T11:21:47.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.701+0000 7f4be7929b80 10 client.16020 fetch_fsmap learned FSMap version 29 2024-05-10T11:21:47.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.701+0000 7f4be7929b80 1 -- 172.21.15.100:0/3010916035 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d63e56a600 con 0x55d63e5534d0 2024-05-10T11:21:47.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.701+0000 7f4bbaff5700 1 -- 172.21.15.100:0/3010916035 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4bd0020070 con 0x55d63e5534d0 2024-05-10T11:21:47.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.701+0000 7f4bbaff5700 1 -- 172.21.15.100:0/3010916035 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f4bc806c050 con 0x55d63e5534d0 2024-05-10T11:21:47.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.701+0000 7f4be7929b80 10 client.16020 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T11:21:47.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.701+0000 7f4be7929b80 10 client.16020 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-10T11:21:47.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.701+0000 7f4be7929b80 20 client.16020 populate_metadata read hostname 'smithi100' 2024-05-10T11:21:47.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.701+0000 7f4be7929b80 1 --2- 172.21.15.100:0/3010916035 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55d63e56cc40 0x55d63e56f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:47.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.701+0000 7f4be7929b80 4 client.16020 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:21:47.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.701+0000 7f4be7929b80 1 -- 172.21.15.100:0/3010916035 --> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d63e51f650 con 0x55d63e56cc40 2024-05-10T11:21:47.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.702+0000 7f4bd659c700 1 --2- 172.21.15.100:0/3010916035 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55d63e56cc40 0x55d63e56f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:47.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.702+0000 7f4bbaff5700 10 client.16020 ms_handle_connect on v2:172.21.15.188:6836/1943342620 2024-05-10T11:21:47.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.702+0000 7f4bd659c700 1 --2- 172.21.15.100:0/3010916035 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55d63e56cc40 0x55d63e56f030 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2f62222669d84688 server_cookie=7d6ca4b659b2c927 in_seq=0 out_seq=0 2024-05-10T11:21:47.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.703+0000 7f4bbaff5700 1 -- 172.21.15.100:0/3010916035 <== mds.2 v2:172.21.15.188:6836/1943342620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d63e51f650 con 0x55d63e56cc40 2024-05-10T11:21:47.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.703+0000 7f4bbaff5700 10 client.16020 handle_command_reply: tid=0 2024-05-10T11:21:47.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.703+0000 7f4be7929b80 1 client.16020 shutdown 2024-05-10T11:21:47.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.703+0000 7f4be7929b80 1 -- 172.21.15.100:0/3010916035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d63e5534d0 msgr2=0x55d63e55eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:47.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.703+0000 7f4be7929b80 1 --2- 172.21.15.100:0/3010916035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d63e5534d0 0x55d63e55eff0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f4bc80dfdf0 tx=0x7f4bc8010b10 comp rx=0 tx=0).stop 2024-05-10T11:21:47.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.703+0000 7f4be7929b80 1 -- 172.21.15.100:0/3010916035 shutdown_connections 2024-05-10T11:21:47.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.703+0000 7f4be7929b80 1 --2- 172.21.15.100:0/3010916035 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55d63e56cc40 0x55d63e56f030 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:47.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.703+0000 7f4be7929b80 1 --2- 172.21.15.100:0/3010916035 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d63e557c80 0x55d63e55c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:47.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.703+0000 7f4bbaff5700 0 client.16020 ms_handle_reset on v2:172.21.15.188:6836/1943342620 2024-05-10T11:21:47.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.703+0000 7f4be7929b80 1 --2- 172.21.15.100:0/3010916035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d63e5534d0 0x55d63e55eff0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:47.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.703+0000 7f4be7929b80 1 --2- 172.21.15.100:0/3010916035 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d63e553df0 0x55d63e55c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:47.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.703+0000 7f4be7929b80 1 -- 172.21.15.100:0/3010916035 >> 172.21.15.100:0/3010916035 conn(0x55d63e47f100 msgr2=0x55d63e558080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:47.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.703+0000 7f4be7929b80 1 -- 172.21.15.100:0/3010916035 shutdown_connections 2024-05-10T11:21:47.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.703+0000 7f4be7929b80 1 -- 172.21.15.100:0/3010916035 wait complete. 2024-05-10T11:21:47.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.704+0000 7f4be7929b80 20 client.16020 trim_cache size 0 max 16384 2024-05-10T11:21:47.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.722+0000 7f4be7929b80 1 Processor -- start 2024-05-10T11:21:47.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.722+0000 7f4be7929b80 1 -- start start 2024-05-10T11:21:47.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.722+0000 7f4be7929b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d63e5534d0 0x55d63e568e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:47.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.722+0000 7f4be7929b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d63e553df0 0x55d63e55cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:47.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.722+0000 7f4be7929b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d63e557c80 0x55d63e5485a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:47.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.722+0000 7f4be7929b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd81b7d70 con 0x55d63e557c80 2024-05-10T11:21:47.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.722+0000 7f4be7929b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd81b5600 con 0x55d63e5534d0 2024-05-10T11:21:47.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.722+0000 7f4be7929b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd81b3450 con 0x55d63e553df0 2024-05-10T11:21:47.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.722+0000 7f4bd6d9d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d63e5534d0 0x55d63e568e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:47.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.722+0000 7f4bd759e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d63e557c80 0x55d63e5485a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:47.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.722+0000 7f4bd6d9d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d63e5534d0 0x55d63e568e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:57366/0 (socket says 172.21.15.100:57366) 2024-05-10T11:21:47.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.722+0000 7f4bd659c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d63e553df0 0x55d63e55cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:47.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.722+0000 7f4bd759e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d63e557c80 0x55d63e5485a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42384/0 (socket says 172.21.15.100:42384) 2024-05-10T11:21:47.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.722+0000 7f4bd6d9d700 1 -- 172.21.15.100:0/3110953261 learned_addr learned my addr 172.21.15.100:0/3110953261 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:21:47.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.722+0000 7f4bd759e700 1 -- 172.21.15.100:0/3110953261 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d63e553df0 msgr2=0x55d63e55cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:47.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.722+0000 7f4bd759e700 1 --2- 172.21.15.100:0/3110953261 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d63e553df0 0x55d63e55cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:47.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.722+0000 7f4bd759e700 1 -- 172.21.15.100:0/3110953261 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d63e5534d0 msgr2=0x55d63e568e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:47.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.722+0000 7f4bd759e700 1 --2- 172.21.15.100:0/3110953261 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d63e5534d0 0x55d63e568e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:47.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.722+0000 7f4bd759e700 1 -- 172.21.15.100:0/3110953261 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d63e55a900 con 0x55d63e557c80 2024-05-10T11:21:47.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.722+0000 7f4bd6d9d700 1 --2- 172.21.15.100:0/3110953261 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d63e5534d0 0x55d63e568e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:21:47.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.722+0000 7f4bd759e700 1 --2- 172.21.15.100:0/3110953261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d63e557c80 0x55d63e5485a0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f4bd000b770 tx=0x7f4bd0003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=589fc4485fa137f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:47.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.723+0000 7f4bbcff9700 1 -- 172.21.15.100:0/3110953261 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd000baa0 con 0x55d63e557c80 2024-05-10T11:21:47.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.723+0000 7f4bbcff9700 1 -- 172.21.15.100:0/3110953261 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4bd0020040 con 0x55d63e557c80 2024-05-10T11:21:47.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.723+0000 7f4bbcff9700 1 -- 172.21.15.100:0/3110953261 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd00074a0 con 0x55d63e557c80 2024-05-10T11:21:47.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.723+0000 7f4be7929b80 1 -- 172.21.15.100:0/3110953261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d63e557c80 msgr2=0x55d63e5485a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:47.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.723+0000 7f4be7929b80 1 --2- 172.21.15.100:0/3110953261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d63e557c80 0x55d63e5485a0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f4bd000b770 tx=0x7f4bd0003f90 comp rx=0 tx=0).stop 2024-05-10T11:21:47.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.723+0000 7f4be7929b80 1 -- 172.21.15.100:0/3110953261 shutdown_connections 2024-05-10T11:21:47.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.723+0000 7f4be7929b80 1 --2- 172.21.15.100:0/3110953261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d63e557c80 0x55d63e5485a0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:47.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.723+0000 7f4be7929b80 1 --2- 172.21.15.100:0/3110953261 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d63e5534d0 0x55d63e568e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:47.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.723+0000 7f4be7929b80 1 --2- 172.21.15.100:0/3110953261 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d63e553df0 0x55d63e55cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:47.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.723+0000 7f4be7929b80 1 -- 172.21.15.100:0/3110953261 >> 172.21.15.100:0/3110953261 conn(0x55d63e47f100 msgr2=0x55d63e547e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:47.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.724+0000 7f4be7929b80 1 -- 172.21.15.100:0/3110953261 shutdown_connections 2024-05-10T11:21:47.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.724+0000 7f4be7929b80 1 -- 172.21.15.100:0/3110953261 wait complete. 2024-05-10T11:21:47.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.724+0000 7f4be7929b80 1 -- start start 2024-05-10T11:21:47.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.724+0000 7f4be7929b80 1 Processor -- start 2024-05-10T11:21:47.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.724+0000 7f4be7929b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d63e5534d0 0x55d63e5604a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:47.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.724+0000 7f4be7929b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d63e553df0 0x55d63e5609e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:47.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.724+0000 7f4be7929b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d63e557c80 0x55d63e5d5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:47.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.724+0000 7f4be7929b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd81b3ac0 con 0x55d63e557c80 2024-05-10T11:21:47.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.724+0000 7f4be7929b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd81b4f30 con 0x55d63e553df0 2024-05-10T11:21:47.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.724+0000 7f4be7929b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd81b4190 con 0x55d63e5534d0 2024-05-10T11:21:47.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.724+0000 7f4be7929b80 10 client.0 _ll_register_callbacks cb 0x55d63e43d920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:21:47.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.724+0000 7f4bd659c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d63e553df0 0x55d63e5609e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:47.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.724+0000 7f4bd6d9d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d63e5534d0 0x55d63e5604a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:47.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.724+0000 7f4bd759e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d63e557c80 0x55d63e5d5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:47.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.724+0000 7f4bd6d9d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d63e5534d0 0x55d63e5604a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:53218/0 (socket says 172.21.15.100:53218) 2024-05-10T11:21:47.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.724+0000 7f4bd759e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d63e557c80 0x55d63e5d5470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42394/0 (socket says 172.21.15.100:42394) 2024-05-10T11:21:47.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.724+0000 7f4bd6d9d700 1 -- 172.21.15.100:0/3388648380 learned_addr learned my addr 172.21.15.100:0/3388648380 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:21:47.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.725+0000 7f4bd6d9d700 1 -- 172.21.15.100:0/3388648380 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d63e553df0 msgr2=0x55d63e5609e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:47.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.725+0000 7f4bd6d9d700 1 --2- 172.21.15.100:0/3388648380 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d63e553df0 0x55d63e5609e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:47.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.725+0000 7f4bd6d9d700 1 -- 172.21.15.100:0/3388648380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d63e557c80 msgr2=0x55d63e5d5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:47.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.725+0000 7f4bd6d9d700 1 --2- 172.21.15.100:0/3388648380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d63e557c80 0x55d63e5d5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:47.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.725+0000 7f4bd6d9d700 1 -- 172.21.15.100:0/3388648380 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d63e5d5a60 con 0x55d63e5534d0 2024-05-10T11:21:47.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.725+0000 7f4bd759e700 1 --2- 172.21.15.100:0/3388648380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d63e557c80 0x55d63e5d5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:21:47.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.725+0000 7f4bd6d9d700 1 --2- 172.21.15.100:0/3388648380 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d63e5534d0 0x55d63e5604a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4bc80b2910 tx=0x7f4bc800c750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35b06293c76ef527 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:47.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.725+0000 7f4bbaff5700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:21:47.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.725+0000 7f4bbaff5700 1 -- 172.21.15.100:0/3388648380 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc80b2110 con 0x55d63e5534d0 2024-05-10T11:21:47.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.725+0000 7f4bbaff5700 1 -- 172.21.15.100:0/3388648380 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f4bc800cb60 con 0x55d63e5534d0 2024-05-10T11:21:47.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.725+0000 7f4be7929b80 1 -- 172.21.15.100:0/3388648380 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4bd0020040 con 0x55d63e5534d0 2024-05-10T11:21:47.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.725+0000 7f4bbaff5700 1 -- 172.21.15.100:0/3388648380 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4bc80aa070 con 0x55d63e5534d0 2024-05-10T11:21:47.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.725+0000 7f4be7929b80 10 client.25528 fetch_fsmap learned FSMap version 29 2024-05-10T11:21:47.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.725+0000 7f4be7929b80 1 -- 172.21.15.100:0/3388648380 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d63e5d6d40 con 0x55d63e5534d0 2024-05-10T11:21:47.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.726+0000 7f4bbaff5700 1 -- 172.21.15.100:0/3388648380 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4bd0020040 con 0x55d63e5534d0 2024-05-10T11:21:47.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.726+0000 7f4bbaff5700 1 -- 172.21.15.100:0/3388648380 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f4bc8089070 con 0x55d63e5534d0 2024-05-10T11:21:47.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.726+0000 7f4be7929b80 10 client.25528 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T11:21:47.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.726+0000 7f4be7929b80 10 client.25528 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-10T11:21:47.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.726+0000 7f4be7929b80 20 client.25528 populate_metadata read hostname 'smithi100' 2024-05-10T11:21:47.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.726+0000 7f4be7929b80 1 --2- 172.21.15.100:0/3388648380 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55d63e5d73f0 0x55d63e5d9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:47.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.726+0000 7f4be7929b80 4 client.25528 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:21:47.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.726+0000 7f4be7929b80 1 -- 172.21.15.100:0/3388648380 --> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d63e4b3240 con 0x55d63e5d73f0 2024-05-10T11:21:47.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.726+0000 7f4bd659c700 1 --2- 172.21.15.100:0/3388648380 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55d63e5d73f0 0x55d63e5d9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:47.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.726+0000 7f4bbaff5700 10 client.25528 ms_handle_connect on v2:172.21.15.188:6836/1943342620 2024-05-10T11:21:47.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.726+0000 7f4bd659c700 1 --2- 172.21.15.100:0/3388648380 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55d63e5d73f0 0x55d63e5d9810 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fba2a58d77bf17a7 server_cookie=ec8630dd09d1cdd in_seq=0 out_seq=0 2024-05-10T11:21:47.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.727+0000 7f4bbaff5700 1 -- 172.21.15.100:0/3388648380 <== mds.2 v2:172.21.15.188:6836/1943342620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d63e4b3240 con 0x55d63e5d73f0 2024-05-10T11:21:47.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.727+0000 7f4bbaff5700 10 client.25528 handle_command_reply: tid=0 2024-05-10T11:21:47.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.727+0000 7f4be7929b80 1 client.25528 shutdown 2024-05-10T11:21:47.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.727+0000 7f4be7929b80 1 -- 172.21.15.100:0/3388648380 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d63e5534d0 msgr2=0x55d63e5604a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:47.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.727+0000 7f4be7929b80 1 --2- 172.21.15.100:0/3388648380 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d63e5534d0 0x55d63e5604a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4bc80b2910 tx=0x7f4bc800c750 comp rx=0 tx=0).stop 2024-05-10T11:21:47.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.727+0000 7f4be7929b80 1 -- 172.21.15.100:0/3388648380 shutdown_connections 2024-05-10T11:21:47.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.727+0000 7f4be7929b80 1 --2- 172.21.15.100:0/3388648380 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55d63e5d73f0 0x55d63e5d9810 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:47.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.727+0000 7f4be7929b80 1 --2- 172.21.15.100:0/3388648380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d63e557c80 0x55d63e5d5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:47.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.727+0000 7f4be7929b80 1 --2- 172.21.15.100:0/3388648380 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d63e5534d0 0x55d63e5604a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:47.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.727+0000 7f4bbaff5700 0 client.25528 ms_handle_reset on v2:172.21.15.188:6836/1943342620 2024-05-10T11:21:47.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.727+0000 7f4be7929b80 1 --2- 172.21.15.100:0/3388648380 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d63e553df0 0x55d63e5609e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:47.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.727+0000 7f4be7929b80 1 -- 172.21.15.100:0/3388648380 >> 172.21.15.100:0/3388648380 conn(0x55d63e47f100 msgr2=0x55d63e5522e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:47.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.727+0000 7f4be7929b80 1 -- 172.21.15.100:0/3388648380 shutdown_connections 2024-05-10T11:21:47.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.727+0000 7f4be7929b80 1 -- 172.21.15.100:0/3388648380 wait complete. 2024-05-10T11:21:47.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.727+0000 7f4be7929b80 20 client.25528 trim_cache size 0 max 16384 2024-05-10T11:21:47.736 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:21:47.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.728+0000 7f4bdd096700 1 -- 172.21.15.100:0/3501590040 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4bc00b2880 msgr2=0x7f4bc00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:47.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.728+0000 7f4bdd096700 1 --2- 172.21.15.100:0/3501590040 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4bc00b2880 0x7f4bc00b4d40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4bd80777c0 tx=0x7f4bcc009260 comp rx=0 tx=0).stop 2024-05-10T11:21:47.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.728+0000 7f4bdd096700 1 -- 172.21.15.100:0/3501590040 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4bd8103540 msgr2=0x7f4bd810f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:47.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.728+0000 7f4bdd096700 1 --2- 172.21.15.100:0/3501590040 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4bd8103540 0x7f4bd810f670 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4bc800e9f0 tx=0x7f4bc800ed00 comp rx=0 tx=0).stop 2024-05-10T11:21:47.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.728+0000 7f4bdd096700 1 -- 172.21.15.100:0/3501590040 shutdown_connections 2024-05-10T11:21:47.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.728+0000 7f4bdd096700 1 --2- 172.21.15.100:0/3501590040 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4bc00b2880 0x7f4bc00b4d40 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:47.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.728+0000 7f4bdd096700 1 --2- 172.21.15.100:0/3501590040 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4bd81095e0 0x7f4bd81a48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:47.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.728+0000 7f4bdd096700 1 --2- 172.21.15.100:0/3501590040 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4bd8103540 0x7f4bd810f670 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:47.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.728+0000 7f4bdd096700 1 --2- 172.21.15.100:0/3501590040 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bd8103f00 0x7f4bd810fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:47.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.728+0000 7f4bdd096700 1 -- 172.21.15.100:0/3501590040 >> 172.21.15.100:0/3501590040 conn(0x7f4bd807a650 msgr2=0x7f4bd8101f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:47.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.728+0000 7f4bdd096700 1 -- 172.21.15.100:0/3501590040 shutdown_connections 2024-05-10T11:21:47.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:47.728+0000 7f4bdd096700 1 -- 172.21.15.100:0/3501590040 wait complete. 2024-05-10T11:21:47.934 INFO:tasks.workunit.client.0.smithi100.stdout: 1 54241 3.03 MB/sec warmup 102 sec latency 17.252 ms 2024-05-10T11:21:48.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:21:48 smithi100 ceph-mon[30296]: pgmap v522: 129 pgs: 129 active+clean; 38 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 1.5 MiB/s wr, 158 op/s 2024-05-10T11:21:48.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:21:48 smithi178 ceph-mon[33359]: pgmap v522: 129 pgs: 129 active+clean; 38 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 1.5 MiB/s wr, 158 op/s 2024-05-10T11:21:48.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:21:48 smithi188 ceph-mon[34075]: pgmap v522: 129 pgs: 129 active+clean; 38 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 1.5 MiB/s wr, 158 op/s 2024-05-10T11:21:48.544 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-10T11:21:48.803 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:21:48.934 INFO:tasks.workunit.client.0.smithi100.stdout: 1 54814 3.04 MB/sec warmup 103 sec latency 15.950 ms 2024-05-10T11:21:49.934 INFO:tasks.workunit.client.0.smithi100.stdout: 1 55485 3.02 MB/sec warmup 104 sec latency 15.752 ms 2024-05-10T11:21:50.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:21:50 smithi100 ceph-mon[30296]: pgmap v523: 129 pgs: 129 active+clean; 40 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 1.5 MiB/s wr, 150 op/s 2024-05-10T11:21:50.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:21:50 smithi178 ceph-mon[33359]: pgmap v523: 129 pgs: 129 active+clean; 40 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 1.5 MiB/s wr, 150 op/s 2024-05-10T11:21:50.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:21:50 smithi188 ceph-mon[34075]: pgmap v523: 129 pgs: 129 active+clean; 40 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 1.5 MiB/s wr, 150 op/s 2024-05-10T11:21:50.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.665+0000 7fe8d94aa700 1 -- 172.21.15.100:0/999292916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8d4102540 msgr2=0x7fe8d410e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:50.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.665+0000 7fe8d94aa700 1 --2- 172.21.15.100:0/999292916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8d4102540 0x7fe8d410e9b0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc00c240 tx=0x7fe8cc00c550 comp rx=0 tx=0).stop 2024-05-10T11:21:50.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.665+0000 7fe8d94aa700 1 -- 172.21.15.100:0/999292916 shutdown_connections 2024-05-10T11:21:50.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.665+0000 7fe8d94aa700 1 --2- 172.21.15.100:0/999292916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8d4102540 0x7fe8d410e9b0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:50.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.665+0000 7fe8d94aa700 1 --2- 172.21.15.100:0/999292916 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8d4107be0 0x7fe8d4107fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:50.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.665+0000 7fe8d94aa700 1 --2- 172.21.15.100:0/999292916 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8d4101b80 0x7fe8d4102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:50.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.665+0000 7fe8d94aa700 1 -- 172.21.15.100:0/999292916 >> 172.21.15.100:0/999292916 conn(0x7fe8d4078a10 msgr2=0x7fe8d40ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:50.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.665+0000 7fe8d94aa700 1 -- 172.21.15.100:0/999292916 shutdown_connections 2024-05-10T11:21:50.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.665+0000 7fe8d94aa700 1 -- 172.21.15.100:0/999292916 wait complete. 2024-05-10T11:21:50.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.666+0000 7fe8d94aa700 1 Processor -- start 2024-05-10T11:21:50.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.666+0000 7fe8d94aa700 1 -- start start 2024-05-10T11:21:50.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.667+0000 7fe8d94aa700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8d4101b80 0x7fe8d419f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:50.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.667+0000 7fe8d94aa700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8d4102540 0x7fe8d419fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:50.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.667+0000 7fe8d94aa700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8d4107be0 0x7fe8d4199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:50.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.667+0000 7fe8d94aa700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d4111360 con 0x7fe8d4102540 2024-05-10T11:21:50.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.667+0000 7fe8d94aa700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d41111e0 con 0x7fe8d4101b80 2024-05-10T11:21:50.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.667+0000 7fe8d94aa700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d41114e0 con 0x7fe8d4107be0 2024-05-10T11:21:50.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.667+0000 7fe8d27fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8d4102540 0x7fe8d419fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:50.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.667+0000 7fe8d27fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8d4102540 0x7fe8d419fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42406/0 (socket says 172.21.15.100:42406) 2024-05-10T11:21:50.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.667+0000 7fe8d27fc700 1 -- 172.21.15.100:0/965800140 learned_addr learned my addr 172.21.15.100:0/965800140 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:21:50.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.667+0000 7fe8d2ffd700 1 --2- 172.21.15.100:0/965800140 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8d4101b80 0x7fe8d419f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:50.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.667+0000 7fe8d37fe700 1 --2- 172.21.15.100:0/965800140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8d4107be0 0x7fe8d4199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:50.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.667+0000 7fe8d2ffd700 1 -- 172.21.15.100:0/965800140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8d4107be0 msgr2=0x7fe8d4199790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:50.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.667+0000 7fe8d2ffd700 1 --2- 172.21.15.100:0/965800140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8d4107be0 0x7fe8d4199790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:50.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.667+0000 7fe8d2ffd700 1 -- 172.21.15.100:0/965800140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8d4102540 msgr2=0x7fe8d419fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:50.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.667+0000 7fe8d2ffd700 1 --2- 172.21.15.100:0/965800140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8d4102540 0x7fe8d419fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:50.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.667+0000 7fe8d2ffd700 1 -- 172.21.15.100:0/965800140 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8d4199f70 con 0x7fe8d4101b80 2024-05-10T11:21:50.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.667+0000 7fe8d27fc700 1 --2- 172.21.15.100:0/965800140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8d4102540 0x7fe8d419fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:21:50.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.667+0000 7fe8d2ffd700 1 --2- 172.21.15.100:0/965800140 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8d4101b80 0x7fe8d419f610 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe8c400e9f0 tx=0x7fe8c400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:50.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.668+0000 7fe8bbfff700 1 -- 172.21.15.100:0/965800140 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c4004220 con 0x7fe8d4101b80 2024-05-10T11:21:50.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.668+0000 7fe8d94aa700 1 -- 172.21.15.100:0/965800140 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8d419a260 con 0x7fe8d4101b80 2024-05-10T11:21:50.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.668+0000 7fe8bbfff700 1 -- 172.21.15.100:0/965800140 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe8c40043c0 con 0x7fe8d4101b80 2024-05-10T11:21:50.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.668+0000 7fe8bbfff700 1 -- 172.21.15.100:0/965800140 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c400ce30 con 0x7fe8d4101b80 2024-05-10T11:21:50.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.668+0000 7fe8d94aa700 1 -- 172.21.15.100:0/965800140 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8d41a6470 con 0x7fe8d4101b80 2024-05-10T11:21:50.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.671+0000 7fe8bbfff700 1 -- 172.21.15.100:0/965800140 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fe8c4004750 con 0x7fe8d4101b80 2024-05-10T11:21:50.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.672+0000 7fe8bbfff700 1 --2- 172.21.15.100:0/965800140 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe8bc0b2880 0x7fe8bc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:50.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.672+0000 7fe8d27fc700 1 --2- 172.21.15.100:0/965800140 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe8bc0b2880 0x7fe8bc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:50.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.672+0000 7fe8bbfff700 1 -- 172.21.15.100:0/965800140 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fe8c40e2520 con 0x7fe8d4101b80 2024-05-10T11:21:50.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.672+0000 7fe8e3d3db80 1 Processor -- start 2024-05-10T11:21:50.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.672+0000 7fe8e3d3db80 1 -- start start 2024-05-10T11:21:50.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.673+0000 7fe8d27fc700 1 --2- 172.21.15.100:0/965800140 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe8bc0b2880 0x7fe8bc0b4d40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4107230 tx=0x7fe8c8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:50.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.673+0000 7fe8e3d3db80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56050ceca1f0 0x56050ceca5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:50.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.673+0000 7fe8e3d3db80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56050cec6810 0x56050cec6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:50.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.673+0000 7fe8e3d3db80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56050cec71d0 0x56050cedb480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:50.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.673+0000 7fe8e3d3db80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56050ce18940 con 0x56050cec71d0 2024-05-10T11:21:50.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.673+0000 7fe8e3d3db80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56050cd7dda0 con 0x56050ceca1f0 2024-05-10T11:21:50.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.673+0000 7fe8e3d3db80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56050ce14900 con 0x56050cec6810 2024-05-10T11:21:50.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.673+0000 7fe8d27fc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56050cec6810 0x56050cec6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:50.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.673+0000 7fe8d27fc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56050cec6810 0x56050cec6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:53250/0 (socket says 172.21.15.100:53250) 2024-05-10T11:21:50.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.673+0000 7fe8d27fc700 1 -- 172.21.15.100:0/2120041917 learned_addr learned my addr 172.21.15.100:0/2120041917 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:21:50.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.673+0000 7fe8d37fe700 1 --2- 172.21.15.100:0/2120041917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56050cec71d0 0x56050cedb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:50.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.673+0000 7fe8d2ffd700 1 --2- 172.21.15.100:0/2120041917 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56050ceca1f0 0x56050ceca5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:50.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.673+0000 7fe8d27fc700 1 -- 172.21.15.100:0/2120041917 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56050ceca1f0 msgr2=0x56050ceca5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:50.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.673+0000 7fe8d27fc700 1 --2- 172.21.15.100:0/2120041917 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56050ceca1f0 0x56050ceca5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:50.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.673+0000 7fe8d27fc700 1 -- 172.21.15.100:0/2120041917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56050cec71d0 msgr2=0x56050cedb480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:50.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.673+0000 7fe8d27fc700 1 --2- 172.21.15.100:0/2120041917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56050cec71d0 0x56050cedb480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:50.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.673+0000 7fe8d27fc700 1 -- 172.21.15.100:0/2120041917 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56050cd658d0 con 0x56050cec6810 2024-05-10T11:21:50.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.673+0000 7fe8d2ffd700 1 --2- 172.21.15.100:0/2120041917 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56050ceca1f0 0x56050ceca5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:21:50.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.673+0000 7fe8d27fc700 1 --2- 172.21.15.100:0/2120041917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56050cec6810 0x56050cec6c90 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8009880 tx=0x7fe8c8014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=781b153093adb4f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:50.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.673+0000 7fe8b97fa700 1 -- 172.21.15.100:0/2120041917 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c8026630 con 0x56050cec6810 2024-05-10T11:21:50.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.674+0000 7fe8b97fa700 1 -- 172.21.15.100:0/2120041917 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe8c800ee40 con 0x56050cec6810 2024-05-10T11:21:50.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.674+0000 7fe8b97fa700 1 -- 172.21.15.100:0/2120041917 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c801d770 con 0x56050cec6810 2024-05-10T11:21:50.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.674+0000 7fe8e3d3db80 1 -- 172.21.15.100:0/2120041917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56050cec6810 msgr2=0x56050cec6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:50.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.674+0000 7fe8e3d3db80 1 --2- 172.21.15.100:0/2120041917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56050cec6810 0x56050cec6c90 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8009880 tx=0x7fe8c8014000 comp rx=0 tx=0).stop 2024-05-10T11:21:50.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.674+0000 7fe8e3d3db80 1 -- 172.21.15.100:0/2120041917 shutdown_connections 2024-05-10T11:21:50.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.674+0000 7fe8e3d3db80 1 --2- 172.21.15.100:0/2120041917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56050cec71d0 0x56050cedb480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:50.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.674+0000 7fe8e3d3db80 1 --2- 172.21.15.100:0/2120041917 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56050ceca1f0 0x56050ceca5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:50.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.674+0000 7fe8e3d3db80 1 --2- 172.21.15.100:0/2120041917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56050cec6810 0x56050cec6c90 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:50.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.674+0000 7fe8e3d3db80 1 -- 172.21.15.100:0/2120041917 >> 172.21.15.100:0/2120041917 conn(0x56050cdf2b70 msgr2=0x56050cec12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:50.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.674+0000 7fe8e3d3db80 1 -- 172.21.15.100:0/2120041917 shutdown_connections 2024-05-10T11:21:50.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.674+0000 7fe8e3d3db80 1 -- 172.21.15.100:0/2120041917 wait complete. 2024-05-10T11:21:50.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.675+0000 7fe8e3d3db80 1 -- start start 2024-05-10T11:21:50.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.675+0000 7fe8e3d3db80 1 Processor -- start 2024-05-10T11:21:50.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.675+0000 7fe8e3d3db80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56050cec6810 0x56050cedb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:50.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.675+0000 7fe8e3d3db80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56050cec71d0 0x56050ced8420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:50.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.675+0000 7fe8e3d3db80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56050ceca1f0 0x56050ced8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:50.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.675+0000 7fe8e3d3db80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56050cedbc60 con 0x56050cec71d0 2024-05-10T11:21:50.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.675+0000 7fe8e3d3db80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56050cdd1ed0 con 0x56050ceca1f0 2024-05-10T11:21:50.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.675+0000 7fe8e3d3db80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56050cedbde0 con 0x56050cec6810 2024-05-10T11:21:50.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.675+0000 7fe8e3d3db80 10 client.0 _ll_register_callbacks cb 0x56050cdac5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:21:50.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.675+0000 7fe8d2ffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56050cec6810 0x56050cedb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:50.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.675+0000 7fe8d2ffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56050cec6810 0x56050cedb080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:53264/0 (socket says 172.21.15.100:53264) 2024-05-10T11:21:50.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.675+0000 7fe8d2ffd700 1 -- 172.21.15.100:0/2637386860 learned_addr learned my addr 172.21.15.100:0/2637386860 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:21:50.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.675+0000 7fe8d27fc700 1 --2- 172.21.15.100:0/2637386860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56050cec71d0 0x56050ced8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:50.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.675+0000 7fe8d37fe700 1 --2- 172.21.15.100:0/2637386860 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56050ceca1f0 0x56050ced8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:50.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.676+0000 7fe8d2ffd700 1 -- 172.21.15.100:0/2637386860 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56050ceca1f0 msgr2=0x56050ced8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:50.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.676+0000 7fe8d2ffd700 1 --2- 172.21.15.100:0/2637386860 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56050ceca1f0 0x56050ced8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:50.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.676+0000 7fe8d2ffd700 1 -- 172.21.15.100:0/2637386860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56050cec71d0 msgr2=0x56050ced8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:50.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.676+0000 7fe8d2ffd700 1 --2- 172.21.15.100:0/2637386860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56050cec71d0 0x56050ced8420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:50.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.676+0000 7fe8d2ffd700 1 -- 172.21.15.100:0/2637386860 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56050cdcf760 con 0x56050cec6810 2024-05-10T11:21:50.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.676+0000 7fe8d37fe700 1 --2- 172.21.15.100:0/2637386860 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56050ceca1f0 0x56050ced8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:21:50.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.676+0000 7fe8d2ffd700 1 --2- 172.21.15.100:0/2637386860 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56050cec6810 0x56050cedb080 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe8c40a9770 tx=0x7fe8c40a9370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1cd9971caf39e4a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:50.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.676+0000 7fe8b6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:21:50.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.676+0000 7fe8b6ff5700 1 -- 172.21.15.100:0/2637386860 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c401c070 con 0x56050cec6810 2024-05-10T11:21:50.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.676+0000 7fe8b6ff5700 1 -- 172.21.15.100:0/2637386860 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fe8c40cc0e0 con 0x56050cec6810 2024-05-10T11:21:50.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.676+0000 7fe8e3d3db80 1 -- 172.21.15.100:0/2637386860 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56050ce26320 con 0x56050cec6810 2024-05-10T11:21:50.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.676+0000 7fe8b6ff5700 1 -- 172.21.15.100:0/2637386860 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe8c40a97e0 con 0x56050cec6810 2024-05-10T11:21:50.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.676+0000 7fe8e3d3db80 10 client.25540 fetch_fsmap learned FSMap version 29 2024-05-10T11:21:50.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.676+0000 7fe8e3d3db80 1 -- 172.21.15.100:0/2637386860 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56050cee64c0 con 0x56050cec6810 2024-05-10T11:21:50.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.677+0000 7fe8b6ff5700 1 -- 172.21.15.100:0/2637386860 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe8c40e1070 con 0x56050cec6810 2024-05-10T11:21:50.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.677+0000 7fe8b6ff5700 1 -- 172.21.15.100:0/2637386860 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fe8c40aab10 con 0x56050cec6810 2024-05-10T11:21:50.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.677+0000 7fe8e3d3db80 10 client.25540 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T11:21:50.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.677+0000 7fe8e3d3db80 10 client.25540 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-10T11:21:50.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.677+0000 7fe8e3d3db80 20 client.25540 populate_metadata read hostname 'smithi100' 2024-05-10T11:21:50.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.677+0000 7fe8e3d3db80 1 --2- 172.21.15.100:0/2637386860 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x56050cee8cd0 0x56050ceeb100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:50.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.677+0000 7fe8e3d3db80 4 client.25540 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:21:50.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.677+0000 7fe8e3d3db80 1 -- 172.21.15.100:0/2637386860 --> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56050ce26320 con 0x56050cee8cd0 2024-05-10T11:21:50.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.677+0000 7fe8d27fc700 1 --2- 172.21.15.100:0/2637386860 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x56050cee8cd0 0x56050ceeb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:50.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.678+0000 7fe8b6ff5700 10 client.25540 ms_handle_connect on v2:172.21.15.178:6838/3760985404 2024-05-10T11:21:50.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.678+0000 7fe8d27fc700 1 --2- 172.21.15.100:0/2637386860 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x56050cee8cd0 0x56050ceeb100 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e30c87af8b11c243 server_cookie=33fb58b2250b7212 in_seq=0 out_seq=0 2024-05-10T11:21:50.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.678+0000 7fe8b6ff5700 1 -- 172.21.15.100:0/2637386860 <== mds.1 v2:172.21.15.178:6838/3760985404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56050ce26320 con 0x56050cee8cd0 2024-05-10T11:21:50.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.678+0000 7fe8b6ff5700 10 client.25540 handle_command_reply: tid=0 2024-05-10T11:21:50.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.678+0000 7fe8e3d3db80 1 client.25540 shutdown 2024-05-10T11:21:50.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.678+0000 7fe8e3d3db80 1 -- 172.21.15.100:0/2637386860 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56050cec6810 msgr2=0x56050cedb080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:50.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.678+0000 7fe8e3d3db80 1 --2- 172.21.15.100:0/2637386860 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56050cec6810 0x56050cedb080 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe8c40a9770 tx=0x7fe8c40a9370 comp rx=0 tx=0).stop 2024-05-10T11:21:50.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.678+0000 7fe8e3d3db80 1 -- 172.21.15.100:0/2637386860 shutdown_connections 2024-05-10T11:21:50.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.678+0000 7fe8e3d3db80 1 --2- 172.21.15.100:0/2637386860 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x56050cee8cd0 0x56050ceeb100 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:50.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.678+0000 7fe8e3d3db80 1 --2- 172.21.15.100:0/2637386860 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56050ceca1f0 0x56050ced8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:50.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.678+0000 7fe8b6ff5700 0 client.25540 ms_handle_reset on v2:172.21.15.178:6838/3760985404 2024-05-10T11:21:50.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.678+0000 7fe8e3d3db80 1 --2- 172.21.15.100:0/2637386860 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56050cec6810 0x56050cedb080 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:50.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.678+0000 7fe8e3d3db80 1 --2- 172.21.15.100:0/2637386860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56050cec71d0 0x56050ced8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:50.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.678+0000 7fe8e3d3db80 1 -- 172.21.15.100:0/2637386860 >> 172.21.15.100:0/2637386860 conn(0x56050cdf2b70 msgr2=0x56050cebbb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:50.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.679+0000 7fe8e3d3db80 1 -- 172.21.15.100:0/2637386860 shutdown_connections 2024-05-10T11:21:50.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.679+0000 7fe8e3d3db80 1 -- 172.21.15.100:0/2637386860 wait complete. 2024-05-10T11:21:50.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.679+0000 7fe8e3d3db80 20 client.25540 trim_cache size 0 max 16384 2024-05-10T11:21:50.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.696+0000 7fe8e3d3db80 1 Processor -- start 2024-05-10T11:21:50.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.696+0000 7fe8e3d3db80 1 -- start start 2024-05-10T11:21:50.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.696+0000 7fe8e3d3db80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56050cec6810 0x56050ced73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:50.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.696+0000 7fe8e3d3db80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56050cec71d0 0x56050cec2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:50.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.696+0000 7fe8e3d3db80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56050ceca1f0 0x56050ced3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:50.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.696+0000 7fe8e3d3db80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d41af640 con 0x56050cec6810 2024-05-10T11:21:50.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.696+0000 7fe8e3d3db80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d41aced0 con 0x56050cec71d0 2024-05-10T11:21:50.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.696+0000 7fe8e3d3db80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d41aad20 con 0x56050ceca1f0 2024-05-10T11:21:50.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.696+0000 7fe8d2ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56050cec6810 0x56050ced73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:50.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.696+0000 7fe8d2ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56050cec6810 0x56050ced73e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42424/0 (socket says 172.21.15.100:42424) 2024-05-10T11:21:50.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.696+0000 7fe8d2ffd700 1 -- 172.21.15.100:0/3895556253 learned_addr learned my addr 172.21.15.100:0/3895556253 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:21:50.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.696+0000 7fe8d27fc700 1 --2- 172.21.15.100:0/3895556253 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56050cec71d0 0x56050cec2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:50.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.696+0000 7fe8d37fe700 1 --2- 172.21.15.100:0/3895556253 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56050ceca1f0 0x56050ced3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:50.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.696+0000 7fe8d2ffd700 1 -- 172.21.15.100:0/3895556253 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56050ceca1f0 msgr2=0x56050ced3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:50.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.696+0000 7fe8d2ffd700 1 --2- 172.21.15.100:0/3895556253 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56050ceca1f0 0x56050ced3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:50.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.696+0000 7fe8d2ffd700 1 -- 172.21.15.100:0/3895556253 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56050cec71d0 msgr2=0x56050cec2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:50.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.696+0000 7fe8d2ffd700 1 --2- 172.21.15.100:0/3895556253 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56050cec71d0 0x56050cec2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:50.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.696+0000 7fe8d2ffd700 1 -- 172.21.15.100:0/3895556253 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56050cedbd90 con 0x56050cec6810 2024-05-10T11:21:50.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.696+0000 7fe8d2ffd700 1 --2- 172.21.15.100:0/3895556253 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56050cec6810 0x56050ced73e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fe8c40e0030 tx=0x7fe8c400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1faf529c5ff458e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:50.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.697+0000 7fe8b8ff9700 1 -- 172.21.15.100:0/3895556253 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c40c6020 con 0x56050cec6810 2024-05-10T11:21:50.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.697+0000 7fe8b8ff9700 1 -- 172.21.15.100:0/3895556253 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe8c40e1070 con 0x56050cec6810 2024-05-10T11:21:50.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.697+0000 7fe8b8ff9700 1 -- 172.21.15.100:0/3895556253 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c401c070 con 0x56050cec6810 2024-05-10T11:21:50.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.697+0000 7fe8e3d3db80 1 -- 172.21.15.100:0/3895556253 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56050cec6810 msgr2=0x56050ced73e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:50.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.697+0000 7fe8e3d3db80 1 --2- 172.21.15.100:0/3895556253 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56050cec6810 0x56050ced73e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fe8c40e0030 tx=0x7fe8c400c860 comp rx=0 tx=0).stop 2024-05-10T11:21:50.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.697+0000 7fe8e3d3db80 1 -- 172.21.15.100:0/3895556253 shutdown_connections 2024-05-10T11:21:50.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.697+0000 7fe8e3d3db80 1 --2- 172.21.15.100:0/3895556253 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56050ceca1f0 0x56050ced3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:50.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.697+0000 7fe8e3d3db80 1 --2- 172.21.15.100:0/3895556253 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56050cec6810 0x56050ced73e0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:50.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.697+0000 7fe8e3d3db80 1 --2- 172.21.15.100:0/3895556253 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56050cec71d0 0x56050cec2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:50.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.697+0000 7fe8e3d3db80 1 -- 172.21.15.100:0/3895556253 >> 172.21.15.100:0/3895556253 conn(0x56050cdf2b70 msgr2=0x56050ced9ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:50.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.698+0000 7fe8e3d3db80 1 -- 172.21.15.100:0/3895556253 shutdown_connections 2024-05-10T11:21:50.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.698+0000 7fe8e3d3db80 1 -- 172.21.15.100:0/3895556253 wait complete. 2024-05-10T11:21:50.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.698+0000 7fe8e3d3db80 1 -- start start 2024-05-10T11:21:50.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.698+0000 7fe8e3d3db80 1 Processor -- start 2024-05-10T11:21:50.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.698+0000 7fe8e3d3db80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56050cec6810 0x56050cee1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:50.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.698+0000 7fe8e3d3db80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56050cec71d0 0x56050ced4e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:50.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.698+0000 7fe8e3d3db80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56050ceca1f0 0x56050ced5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:50.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.698+0000 7fe8e3d3db80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d41ab390 con 0x56050cec71d0 2024-05-10T11:21:50.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.698+0000 7fe8e3d3db80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d41ac800 con 0x56050cec6810 2024-05-10T11:21:50.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.698+0000 7fe8e3d3db80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d41aba60 con 0x56050ceca1f0 2024-05-10T11:21:50.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.698+0000 7fe8e3d3db80 10 client.0 _ll_register_callbacks cb 0x56050cdac5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:21:50.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.698+0000 7fe8d27fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56050cec71d0 0x56050ced4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:50.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.698+0000 7fe8d2ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56050cec6810 0x56050cee1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:50.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.698+0000 7fe8d27fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56050cec71d0 0x56050ced4e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42434/0 (socket says 172.21.15.100:42434) 2024-05-10T11:21:50.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.698+0000 7fe8d27fc700 1 -- 172.21.15.100:0/784438906 learned_addr learned my addr 172.21.15.100:0/784438906 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:21:50.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.698+0000 7fe8d37fe700 1 --2- 172.21.15.100:0/784438906 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56050ceca1f0 0x56050ced5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:50.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.699+0000 7fe8d37fe700 1 -- 172.21.15.100:0/784438906 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56050cec6810 msgr2=0x56050cee1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:50.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.699+0000 7fe8d37fe700 1 --2- 172.21.15.100:0/784438906 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56050cec6810 0x56050cee1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:50.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.699+0000 7fe8d37fe700 1 -- 172.21.15.100:0/784438906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56050cec71d0 msgr2=0x56050ced4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:50.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.699+0000 7fe8d37fe700 1 --2- 172.21.15.100:0/784438906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56050cec71d0 0x56050ced4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:50.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.699+0000 7fe8d37fe700 1 -- 172.21.15.100:0/784438906 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56050ced5980 con 0x56050ceca1f0 2024-05-10T11:21:50.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.699+0000 7fe8d27fc700 1 --2- 172.21.15.100:0/784438906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56050cec71d0 0x56050ced4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:21:50.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.699+0000 7fe8d2ffd700 1 --2- 172.21.15.100:0/784438906 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56050cec6810 0x56050cee1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:21:50.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.699+0000 7fe8d37fe700 1 --2- 172.21.15.100:0/784438906 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56050ceca1f0 0x56050ced5340 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc00c240 tx=0x7fe8cc00a300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c5d50f6f36931bd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:50.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.699+0000 7fe8b6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:21:50.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.699+0000 7fe8b6ff5700 1 -- 172.21.15.100:0/784438906 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc0050b0 con 0x56050ceca1f0 2024-05-10T11:21:50.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.699+0000 7fe8e3d3db80 1 -- 172.21.15.100:0/784438906 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8c40e1070 con 0x56050ceca1f0 2024-05-10T11:21:50.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.699+0000 7fe8b6ff5700 1 -- 172.21.15.100:0/784438906 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fe8cc005250 con 0x56050ceca1f0 2024-05-10T11:21:50.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.699+0000 7fe8b6ff5700 1 -- 172.21.15.100:0/784438906 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe8cc005490 con 0x56050ceca1f0 2024-05-10T11:21:50.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.699+0000 7fe8e3d3db80 10 client.25546 fetch_fsmap learned FSMap version 29 2024-05-10T11:21:50.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.699+0000 7fe8e3d3db80 1 -- 172.21.15.100:0/784438906 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56050cf533f0 con 0x56050ceca1f0 2024-05-10T11:21:50.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.700+0000 7fe8b6ff5700 1 -- 172.21.15.100:0/784438906 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe8c40e1070 con 0x56050ceca1f0 2024-05-10T11:21:50.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.700+0000 7fe8b6ff5700 1 -- 172.21.15.100:0/784438906 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fe8cc012070 con 0x56050ceca1f0 2024-05-10T11:21:50.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.700+0000 7fe8e3d3db80 10 client.25546 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T11:21:50.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.700+0000 7fe8e3d3db80 10 client.25546 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-10T11:21:50.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.700+0000 7fe8e3d3db80 20 client.25546 populate_metadata read hostname 'smithi100' 2024-05-10T11:21:50.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.700+0000 7fe8e3d3db80 1 --2- 172.21.15.100:0/784438906 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x56050cf53b30 0x56050cf55f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:50.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.700+0000 7fe8e3d3db80 4 client.25546 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:21:50.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.700+0000 7fe8e3d3db80 1 -- 172.21.15.100:0/784438906 --> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56050ce203c0 con 0x56050cf53b30 2024-05-10T11:21:50.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.700+0000 7fe8d2ffd700 1 --2- 172.21.15.100:0/784438906 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x56050cf53b30 0x56050cf55f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:50.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.700+0000 7fe8d2ffd700 1 --2- 172.21.15.100:0/784438906 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x56050cf53b30 0x56050cf55f50 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2ff04918d0dda0b3 server_cookie=f3ffb7bbe2f41cc3 in_seq=0 out_seq=0 2024-05-10T11:21:50.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.700+0000 7fe8b6ff5700 10 client.25546 ms_handle_connect on v2:172.21.15.178:6838/3760985404 2024-05-10T11:21:50.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.700+0000 7fe8b6ff5700 1 -- 172.21.15.100:0/784438906 <== mds.1 v2:172.21.15.178:6838/3760985404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56050ce203c0 con 0x56050cf53b30 2024-05-10T11:21:50.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.701+0000 7fe8b6ff5700 10 client.25546 handle_command_reply: tid=0 2024-05-10T11:21:50.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.701+0000 7fe8e3d3db80 1 client.25546 shutdown 2024-05-10T11:21:50.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.701+0000 7fe8e3d3db80 1 -- 172.21.15.100:0/784438906 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56050ceca1f0 msgr2=0x56050ced5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:50.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.701+0000 7fe8e3d3db80 1 --2- 172.21.15.100:0/784438906 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56050ceca1f0 0x56050ced5340 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc00c240 tx=0x7fe8cc00a300 comp rx=0 tx=0).stop 2024-05-10T11:21:50.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.701+0000 7fe8e3d3db80 1 -- 172.21.15.100:0/784438906 shutdown_connections 2024-05-10T11:21:50.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.701+0000 7fe8e3d3db80 1 --2- 172.21.15.100:0/784438906 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x56050cf53b30 0x56050cf55f50 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:50.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.701+0000 7fe8e3d3db80 1 --2- 172.21.15.100:0/784438906 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56050ceca1f0 0x56050ced5340 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:50.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.701+0000 7fe8e3d3db80 1 --2- 172.21.15.100:0/784438906 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56050cec6810 0x56050cee1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:50.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.701+0000 7fe8e3d3db80 1 --2- 172.21.15.100:0/784438906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56050cec71d0 0x56050ced4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:50.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.701+0000 7fe8b6ff5700 0 client.25546 ms_handle_reset on v2:172.21.15.178:6838/3760985404 2024-05-10T11:21:50.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.701+0000 7fe8e3d3db80 1 -- 172.21.15.100:0/784438906 >> 172.21.15.100:0/784438906 conn(0x56050cdf2b70 msgr2=0x56050ced3b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:50.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.701+0000 7fe8e3d3db80 1 -- 172.21.15.100:0/784438906 shutdown_connections 2024-05-10T11:21:50.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.701+0000 7fe8e3d3db80 1 -- 172.21.15.100:0/784438906 wait complete. 2024-05-10T11:21:50.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.701+0000 7fe8e3d3db80 20 client.25546 trim_cache size 0 max 16384 2024-05-10T11:21:50.707 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:21:50.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.701+0000 7fe8d94aa700 1 -- 172.21.15.100:0/965800140 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe8bc0b2880 msgr2=0x7fe8bc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:50.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.701+0000 7fe8d94aa700 1 --2- 172.21.15.100:0/965800140 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe8bc0b2880 0x7fe8bc0b4d40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4107230 tx=0x7fe8c8009260 comp rx=0 tx=0).stop 2024-05-10T11:21:50.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.701+0000 7fe8d94aa700 1 -- 172.21.15.100:0/965800140 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8d4101b80 msgr2=0x7fe8d419f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:50.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.701+0000 7fe8d94aa700 1 --2- 172.21.15.100:0/965800140 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8d4101b80 0x7fe8d419f610 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe8c400e9f0 tx=0x7fe8c400ed00 comp rx=0 tx=0).stop 2024-05-10T11:21:50.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.701+0000 7fe8d94aa700 1 -- 172.21.15.100:0/965800140 shutdown_connections 2024-05-10T11:21:50.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.701+0000 7fe8d94aa700 1 --2- 172.21.15.100:0/965800140 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe8bc0b2880 0x7fe8bc0b4d40 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:50.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.701+0000 7fe8d94aa700 1 --2- 172.21.15.100:0/965800140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8d4107be0 0x7fe8d4199790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:50.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.701+0000 7fe8d94aa700 1 --2- 172.21.15.100:0/965800140 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8d4101b80 0x7fe8d419f610 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:50.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.701+0000 7fe8d94aa700 1 --2- 172.21.15.100:0/965800140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8d4102540 0x7fe8d419fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:50.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.701+0000 7fe8d94aa700 1 -- 172.21.15.100:0/965800140 >> 172.21.15.100:0/965800140 conn(0x7fe8d4078a10 msgr2=0x7fe8d4100130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:50.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.702+0000 7fe8d94aa700 1 -- 172.21.15.100:0/965800140 shutdown_connections 2024-05-10T11:21:50.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:50.702+0000 7fe8d94aa700 1 -- 172.21.15.100:0/965800140 wait complete. 2024-05-10T11:21:50.934 INFO:tasks.workunit.client.0.smithi100.stdout: 1 56012 3.04 MB/sec warmup 105 sec latency 16.956 ms 2024-05-10T11:21:51.590 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-10T11:21:51.855 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:21:51.934 INFO:tasks.workunit.client.0.smithi100.stdout: 1 56577 3.04 MB/sec warmup 106 sec latency 15.395 ms 2024-05-10T11:21:52.249 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:21:52 smithi100 ceph-mon[30296]: pgmap v524: 129 pgs: 129 active+clean; 40 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 1.4 MiB/s wr, 156 op/s 2024-05-10T11:21:52.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:21:52 smithi178 ceph-mon[33359]: pgmap v524: 129 pgs: 129 active+clean; 40 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 1.4 MiB/s wr, 156 op/s 2024-05-10T11:21:52.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:21:52 smithi188 ceph-mon[34075]: pgmap v524: 129 pgs: 129 active+clean; 40 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 1.4 MiB/s wr, 156 op/s 2024-05-10T11:21:52.828 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.825+0000 7f653e7af640 1 -- 172.21.15.100:0/3233849600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65381034c0 msgr2=0x7f6538109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:52.828 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.825+0000 7f653e7af640 1 --2- 172.21.15.100:0/3233849600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65381034c0 0x7f6538109d50 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f6530009ca0 tx=0x7f653002a8c0 comp rx=0 tx=0).stop 2024-05-10T11:21:52.828 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.825+0000 7f653e7af640 1 -- 172.21.15.100:0/3233849600 shutdown_connections 2024-05-10T11:21:52.828 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.825+0000 7f653e7af640 1 --2- 172.21.15.100:0/3233849600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65381034c0 0x7f6538109d50 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:52.828 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.825+0000 7f653e7af640 1 --2- 172.21.15.100:0/3233849600 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6538102b00 0x7f6538102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:52.828 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.825+0000 7f653e7af640 1 --2- 172.21.15.100:0/3233849600 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6538101910 0x7f6538101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:52.828 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.825+0000 7f653e7af640 1 -- 172.21.15.100:0/3233849600 >> 172.21.15.100:0/3233849600 conn(0x7f65380fce80 msgr2=0x7f65380ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:52.828 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.825+0000 7f653e7af640 1 -- 172.21.15.100:0/3233849600 shutdown_connections 2024-05-10T11:21:52.829 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.825+0000 7f653e7af640 1 -- 172.21.15.100:0/3233849600 wait complete. 2024-05-10T11:21:52.829 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.825+0000 7f653e7af640 1 Processor -- start 2024-05-10T11:21:52.829 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.826+0000 7f653e7af640 1 -- start start 2024-05-10T11:21:52.829 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.826+0000 7f653e7af640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6538101910 0x7f65381979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:52.829 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.826+0000 7f653e7af640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6538102b00 0x7f6538197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:52.829 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.826+0000 7f653e7af640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65381034c0 0x7f653819ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:52.829 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.826+0000 7f653e7af640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f653810d340 con 0x7f65381034c0 2024-05-10T11:21:52.829 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.826+0000 7f653e7af640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f653810d1c0 con 0x7f6538102b00 2024-05-10T11:21:52.829 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.826+0000 7f653e7af640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f653810d4c0 con 0x7f6538101910 2024-05-10T11:21:52.829 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.826+0000 7f653cd4c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65381034c0 0x7f653819ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:52.829 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.826+0000 7f652ffff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6538102b00 0x7f6538197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:52.829 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.826+0000 7f652ffff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6538102b00 0x7f6538197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:57452/0 (socket says 172.21.15.100:57452) 2024-05-10T11:21:52.829 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.826+0000 7f652ffff640 1 -- 172.21.15.100:0/4114273803 learned_addr learned my addr 172.21.15.100:0/4114273803 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:21:52.829 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.826+0000 7f6537fff640 1 --2- 172.21.15.100:0/4114273803 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6538101910 0x7f65381979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:52.829 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.826+0000 7f653cd4c640 1 -- 172.21.15.100:0/4114273803 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6538101910 msgr2=0x7f65381979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:52.830 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.826+0000 7f653cd4c640 1 --2- 172.21.15.100:0/4114273803 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6538101910 0x7f65381979b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:52.830 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.826+0000 7f653cd4c640 1 -- 172.21.15.100:0/4114273803 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6538102b00 msgr2=0x7f6538197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:52.830 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.826+0000 7f653cd4c640 1 --2- 172.21.15.100:0/4114273803 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6538102b00 0x7f6538197ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:52.830 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.826+0000 7f653cd4c640 1 -- 172.21.15.100:0/4114273803 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f653819f5d0 con 0x7f65381034c0 2024-05-10T11:21:52.830 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.826+0000 7f652ffff640 1 --2- 172.21.15.100:0/4114273803 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6538102b00 0x7f6538197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:21:52.830 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.826+0000 7f653cd4c640 1 --2- 172.21.15.100:0/4114273803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65381034c0 0x7f653819ef60 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f6530031040 tx=0x7f6530003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:52.830 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.827+0000 7f6535ffb640 1 -- 172.21.15.100:0/4114273803 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653004a070 con 0x7f65381034c0 2024-05-10T11:21:52.830 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.827+0000 7f6535ffb640 1 -- 172.21.15.100:0/4114273803 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6530003cf0 con 0x7f65381034c0 2024-05-10T11:21:52.830 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.827+0000 7f6535ffb640 1 -- 172.21.15.100:0/4114273803 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6530030a50 con 0x7f65381034c0 2024-05-10T11:21:52.830 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.827+0000 7f653e7af640 1 -- 172.21.15.100:0/4114273803 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f653806a060 con 0x7f65381034c0 2024-05-10T11:21:52.830 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.827+0000 7f653e7af640 1 -- 172.21.15.100:0/4114273803 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f653806a600 con 0x7f65381034c0 2024-05-10T11:21:52.837 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.829+0000 7f6535ffb640 1 -- 172.21.15.100:0/4114273803 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f653004d070 con 0x7f65381034c0 2024-05-10T11:21:52.837 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.829+0000 7f654159f740 1 Processor -- start 2024-05-10T11:21:52.837 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.829+0000 7f654159f740 1 -- start start 2024-05-10T11:21:52.837 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.829+0000 7f654159f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599b0266db0 0x5599b02671d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:52.837 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.829+0000 7f654159f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599b02677a0 0x5599b026c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:52.837 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.829+0000 7f654159f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599b026c910 0x5599b026eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:52.837 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.829+0000 7f654159f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5599b01cb5d0 con 0x5599b0266db0 2024-05-10T11:21:52.837 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.829+0000 7f654159f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5599b0184890 con 0x5599b02677a0 2024-05-10T11:21:52.837 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.829+0000 7f654159f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5599b01c7590 con 0x5599b026c910 2024-05-10T11:21:52.837 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.830+0000 7f652ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599b0266db0 0x5599b02671d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:52.837 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.830+0000 7f6537fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599b02677a0 0x5599b026c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:52.837 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.830+0000 7f652ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599b0266db0 0x5599b02671d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42450/0 (socket says 172.21.15.100:42450) 2024-05-10T11:21:52.838 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.830+0000 7f652ffff640 1 -- 172.21.15.100:0/1224443731 learned_addr learned my addr 172.21.15.100:0/1224443731 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:21:52.838 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.830+0000 7f652ffff640 1 --2- 172.21.15.100:0/1224443731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599b026c910 0x5599b026eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:52.838 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.830+0000 7f6537fff640 1 -- 172.21.15.100:0/1224443731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599b026c910 msgr2=0x5599b026eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:52.838 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.830+0000 7f6537fff640 1 --2- 172.21.15.100:0/1224443731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599b026c910 0x5599b026eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:52.838 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.830+0000 7f6537fff640 1 -- 172.21.15.100:0/1224443731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599b0266db0 msgr2=0x5599b02671d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:52.838 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.830+0000 7f6537fff640 1 --2- 172.21.15.100:0/1224443731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599b0266db0 0x5599b02671d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:52.838 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.830+0000 7f6537fff640 1 -- 172.21.15.100:0/1224443731 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599afedd210 con 0x5599b02677a0 2024-05-10T11:21:52.838 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.830+0000 7f652ffff640 1 --2- 172.21.15.100:0/1224443731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599b026c910 0x5599b026eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:21:52.838 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.830+0000 7f6535ffb640 1 --2- 172.21.15.100:0/4114273803 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f65240b2920 0x7f65240b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:52.838 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.830+0000 7f652ffff640 1 --2- 172.21.15.100:0/4114273803 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f65240b2920 0x7f65240b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:52.838 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.830+0000 7f6537fff640 1 --2- 172.21.15.100:0/1224443731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599b02677a0 0x5599b026c2a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6528009bd0 tx=0x7f6528011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13bdb0b6c6430fcd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:52.838 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.830+0000 7f6535ffb640 1 -- 172.21.15.100:0/4114273803 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f6530104aa0 con 0x7f65381034c0 2024-05-10T11:21:52.838 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.830+0000 7f652e7fc640 1 -- 172.21.15.100:0/1224443731 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6528004230 con 0x5599b02677a0 2024-05-10T11:21:52.839 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.830+0000 7f652e7fc640 1 -- 172.21.15.100:0/1224443731 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6528009e40 con 0x5599b02677a0 2024-05-10T11:21:52.839 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.830+0000 7f652e7fc640 1 -- 172.21.15.100:0/1224443731 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6528019660 con 0x5599b02677a0 2024-05-10T11:21:52.839 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.831+0000 7f654159f740 1 -- 172.21.15.100:0/1224443731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599b02677a0 msgr2=0x5599b026c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:52.839 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.831+0000 7f652ffff640 1 --2- 172.21.15.100:0/4114273803 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f65240b2920 0x7f65240b4de0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x5599b0268000 tx=0x7f6520015040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:52.839 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.831+0000 7f654159f740 1 --2- 172.21.15.100:0/1224443731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599b02677a0 0x5599b026c2a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6528009bd0 tx=0x7f6528011810 comp rx=0 tx=0).stop 2024-05-10T11:21:52.839 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.831+0000 7f654159f740 1 -- 172.21.15.100:0/1224443731 shutdown_connections 2024-05-10T11:21:52.839 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.831+0000 7f654159f740 1 --2- 172.21.15.100:0/1224443731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599b026c910 0x5599b026eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:52.839 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.831+0000 7f654159f740 1 --2- 172.21.15.100:0/1224443731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599b02677a0 0x5599b026c2a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:52.839 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.831+0000 7f654159f740 1 --2- 172.21.15.100:0/1224443731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599b0266db0 0x5599b02671d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:52.839 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.831+0000 7f654159f740 1 -- 172.21.15.100:0/1224443731 >> 172.21.15.100:0/1224443731 conn(0x5599b0262420 msgr2=0x5599b0262820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:52.839 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.831+0000 7f654159f740 1 -- 172.21.15.100:0/1224443731 shutdown_connections 2024-05-10T11:21:52.839 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.831+0000 7f654159f740 1 -- 172.21.15.100:0/1224443731 wait complete. 2024-05-10T11:21:52.839 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.831+0000 7f654159f740 1 -- start start 2024-05-10T11:21:52.839 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.832+0000 7f654159f740 1 Processor -- start 2024-05-10T11:21:52.840 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.832+0000 7f654159f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599b0266db0 0x5599b0271080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:52.840 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.832+0000 7f654159f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599b02677a0 0x5599b02715c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:52.840 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.832+0000 7f654159f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599b026c910 0x5599b0278590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:52.840 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.832+0000 7f654159f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5599b026f8c0 con 0x5599b026c910 2024-05-10T11:21:52.840 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.832+0000 7f654159f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5599b0188370 con 0x5599b0266db0 2024-05-10T11:21:52.840 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.832+0000 7f654159f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5599b026fa40 con 0x5599b02677a0 2024-05-10T11:21:52.840 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.832+0000 7f654159f740 10 client.0 _ll_register_callbacks cb 0x5599afedca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:21:52.840 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.832+0000 7f653cd4c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599b026c910 0x5599b0278590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:52.840 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.832+0000 7f6537fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599b0266db0 0x5599b0271080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:52.840 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.832+0000 7f653cd4c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599b026c910 0x5599b0278590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42454/0 (socket says 172.21.15.100:42454) 2024-05-10T11:21:52.840 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.832+0000 7f6537fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599b0266db0 0x5599b0271080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:57470/0 (socket says 172.21.15.100:57470) 2024-05-10T11:21:52.840 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.832+0000 7f653cd4c640 1 -- 172.21.15.100:0/249834547 learned_addr learned my addr 172.21.15.100:0/249834547 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:21:52.841 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.832+0000 7f652ffff640 1 --2- 172.21.15.100:0/249834547 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599b02677a0 0x5599b02715c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:52.841 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.833+0000 7f6537fff640 1 -- 172.21.15.100:0/249834547 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599b02677a0 msgr2=0x5599b02715c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:52.841 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.833+0000 7f6537fff640 1 --2- 172.21.15.100:0/249834547 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599b02677a0 0x5599b02715c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:52.841 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.833+0000 7f6537fff640 1 -- 172.21.15.100:0/249834547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599b026c910 msgr2=0x5599b0278590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:52.841 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.833+0000 7f6537fff640 1 --2- 172.21.15.100:0/249834547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599b026c910 0x5599b0278590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:52.841 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.833+0000 7f6537fff640 1 -- 172.21.15.100:0/249834547 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599affa9620 con 0x5599b0266db0 2024-05-10T11:21:52.841 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.833+0000 7f6537fff640 1 --2- 172.21.15.100:0/249834547 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599b0266db0 0x5599b0271080 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x5599b02632f0 tx=0x7f6528005c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87f80f30bd7c8c88 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:52.841 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.833+0000 7f65177fe640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:21:52.841 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.833+0000 7f65177fe640 1 -- 172.21.15.100:0/249834547 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6528028070 con 0x5599b0266db0 2024-05-10T11:21:52.841 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.833+0000 7f654159f740 1 -- 172.21.15.100:0/249834547 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5599b01d91e0 con 0x5599b0266db0 2024-05-10T11:21:52.841 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.833+0000 7f65177fe640 1 -- 172.21.15.100:0/249834547 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f6528003880 con 0x5599b0266db0 2024-05-10T11:21:52.841 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.833+0000 7f65177fe640 1 -- 172.21.15.100:0/249834547 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6528034050 con 0x5599b0266db0 2024-05-10T11:21:52.841 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.833+0000 7f654159f740 10 client.25547 fetch_fsmap learned FSMap version 29 2024-05-10T11:21:52.841 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.833+0000 7f654159f740 1 -- 172.21.15.100:0/249834547 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5599b027b570 con 0x5599b0266db0 2024-05-10T11:21:52.842 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.833+0000 7f65177fe640 1 -- 172.21.15.100:0/249834547 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x5599b01d91e0 con 0x5599b0266db0 2024-05-10T11:21:52.842 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.834+0000 7f65177fe640 1 -- 172.21.15.100:0/249834547 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f65280049c0 con 0x5599b0266db0 2024-05-10T11:21:52.842 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.834+0000 7f654159f740 10 client.25547 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T11:21:52.842 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.834+0000 7f654159f740 10 client.25547 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:21:52.842 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.834+0000 7f654159f740 20 client.25547 populate_metadata read hostname 'smithi100' 2024-05-10T11:21:52.842 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.834+0000 7f654159f740 1 --2- 172.21.15.100:0/249834547 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5599b027cab0 0x5599b027eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:52.842 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.834+0000 7f654159f740 4 client.25547 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:21:52.842 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.834+0000 7f654159f740 1 -- 172.21.15.100:0/249834547 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599b01d3050 con 0x5599b027cab0 2024-05-10T11:21:52.842 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.834+0000 7f652ffff640 1 --2- 172.21.15.100:0/249834547 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5599b027cab0 0x5599b027eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:52.842 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.835+0000 7f652ffff640 1 --2- 172.21.15.100:0/249834547 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5599b027cab0 0x5599b027eea0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=116d11ddd9ac17b3 server_cookie=a944b869183dc42b in_seq=0 out_seq=0 2024-05-10T11:21:52.842 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.835+0000 7f65177fe640 10 client.25547 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:21:52.842 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.835+0000 7f65177fe640 1 -- 172.21.15.100:0/249834547 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5599b01d3050 con 0x5599b027cab0 2024-05-10T11:21:52.842 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.835+0000 7f65177fe640 10 client.25547 handle_command_reply: tid=0 2024-05-10T11:21:52.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.835+0000 7f654159f740 1 client.25547 shutdown 2024-05-10T11:21:52.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.835+0000 7f654159f740 1 -- 172.21.15.100:0/249834547 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599b0266db0 msgr2=0x5599b0271080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:52.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.835+0000 7f654159f740 1 --2- 172.21.15.100:0/249834547 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599b0266db0 0x5599b0271080 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x5599b02632f0 tx=0x7f6528005c50 comp rx=0 tx=0).stop 2024-05-10T11:21:52.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.835+0000 7f654159f740 1 -- 172.21.15.100:0/249834547 shutdown_connections 2024-05-10T11:21:52.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.835+0000 7f654159f740 1 --2- 172.21.15.100:0/249834547 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5599b027cab0 0x5599b027eea0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:52.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.835+0000 7f654159f740 1 --2- 172.21.15.100:0/249834547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599b026c910 0x5599b0278590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:52.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.835+0000 7f654159f740 1 --2- 172.21.15.100:0/249834547 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599b02677a0 0x5599b02715c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:52.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.835+0000 7f654159f740 1 --2- 172.21.15.100:0/249834547 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599b0266db0 0x5599b0271080 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:52.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.835+0000 7f65177fe640 0 client.25547 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:21:52.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.835+0000 7f654159f740 1 -- 172.21.15.100:0/249834547 >> 172.21.15.100:0/249834547 conn(0x5599b0262420 msgr2=0x5599b0261580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:52.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.836+0000 7f654159f740 1 -- 172.21.15.100:0/249834547 shutdown_connections 2024-05-10T11:21:52.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.836+0000 7f654159f740 1 -- 172.21.15.100:0/249834547 wait complete. 2024-05-10T11:21:52.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.836+0000 7f654159f740 20 client.25547 trim_cache size 0 max 16384 2024-05-10T11:21:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.851+0000 7f654159f740 1 Processor -- start 2024-05-10T11:21:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.851+0000 7f654159f740 1 -- start start 2024-05-10T11:21:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.851+0000 7f654159f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599b0262420 0x5599b02706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.851+0000 7f654159f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599b0266db0 0x5599b0270be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.851+0000 7f654159f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599b02677a0 0x5599b0271120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.851+0000 7f654159f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f65381ab900 con 0x5599b02677a0 2024-05-10T11:21:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.851+0000 7f654159f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f65381a91d0 con 0x5599b0262420 2024-05-10T11:21:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.851+0000 7f654159f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f65381a7020 con 0x5599b0266db0 2024-05-10T11:21:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.852+0000 7f653cd4c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599b02677a0 0x5599b0271120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.852+0000 7f653cd4c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599b02677a0 0x5599b0271120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42458/0 (socket says 172.21.15.100:42458) 2024-05-10T11:21:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.852+0000 7f653cd4c640 1 -- 172.21.15.100:0/1772386869 learned_addr learned my addr 172.21.15.100:0/1772386869 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:21:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.852+0000 7f652ffff640 1 --2- 172.21.15.100:0/1772386869 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599b0266db0 0x5599b0270be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.852+0000 7f6537fff640 1 --2- 172.21.15.100:0/1772386869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599b0262420 0x5599b02706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.852+0000 7f653cd4c640 1 -- 172.21.15.100:0/1772386869 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599b0266db0 msgr2=0x5599b0270be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.852+0000 7f653cd4c640 1 --2- 172.21.15.100:0/1772386869 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599b0266db0 0x5599b0270be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.852+0000 7f653cd4c640 1 -- 172.21.15.100:0/1772386869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599b0262420 msgr2=0x5599b02706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.852+0000 7f653cd4c640 1 --2- 172.21.15.100:0/1772386869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599b0262420 0x5599b02706a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.852+0000 7f653cd4c640 1 -- 172.21.15.100:0/1772386869 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599b027f520 con 0x5599b02677a0 2024-05-10T11:21:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.852+0000 7f653cd4c640 1 --2- 172.21.15.100:0/1772386869 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599b02677a0 0x5599b0271120 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f65301025b0 tx=0x7f65300cb7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6e6ea4dc8a91179 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.852+0000 7f652dffb640 1 -- 172.21.15.100:0/1772386869 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65300cbe10 con 0x5599b02677a0 2024-05-10T11:21:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.852+0000 7f652dffb640 1 -- 172.21.15.100:0/1772386869 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6530036040 con 0x5599b02677a0 2024-05-10T11:21:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.852+0000 7f652dffb640 1 -- 172.21.15.100:0/1772386869 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65300d4730 con 0x5599b02677a0 2024-05-10T11:21:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.853+0000 7f654159f740 1 -- 172.21.15.100:0/1772386869 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599b02677a0 msgr2=0x5599b0271120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.853+0000 7f654159f740 1 --2- 172.21.15.100:0/1772386869 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599b02677a0 0x5599b0271120 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f65301025b0 tx=0x7f65300cb7e0 comp rx=0 tx=0).stop 2024-05-10T11:21:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.853+0000 7f654159f740 1 -- 172.21.15.100:0/1772386869 shutdown_connections 2024-05-10T11:21:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.853+0000 7f654159f740 1 --2- 172.21.15.100:0/1772386869 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599b02677a0 0x5599b0271120 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.853+0000 7f654159f740 1 --2- 172.21.15.100:0/1772386869 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599b0266db0 0x5599b0270be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.853+0000 7f654159f740 1 --2- 172.21.15.100:0/1772386869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599b0262420 0x5599b02706a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.853+0000 7f654159f740 1 -- 172.21.15.100:0/1772386869 >> 172.21.15.100:0/1772386869 conn(0x5599b026c3e0 msgr2=0x5599b0265390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.853+0000 7f654159f740 1 -- 172.21.15.100:0/1772386869 shutdown_connections 2024-05-10T11:21:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.853+0000 7f654159f740 1 -- 172.21.15.100:0/1772386869 wait complete. 2024-05-10T11:21:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.853+0000 7f654159f740 1 -- start start 2024-05-10T11:21:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.854+0000 7f654159f740 1 Processor -- start 2024-05-10T11:21:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.854+0000 7f654159f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599b0262420 0x5599b02755e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.854+0000 7f654159f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599b0266db0 0x5599b0275b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.854+0000 7f654159f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599b02677a0 0x5599b0276b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.854+0000 7f654159f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f65381a7690 con 0x5599b02677a0 2024-05-10T11:21:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.854+0000 7f654159f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f65381a8b00 con 0x5599b0266db0 2024-05-10T11:21:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.854+0000 7f654159f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f65381a7d60 con 0x5599b0262420 2024-05-10T11:21:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.854+0000 7f654159f740 10 client.0 _ll_register_callbacks cb 0x7f6520009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:21:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.854+0000 7f6537fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599b0262420 0x5599b02755e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.854+0000 7f653cd4c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599b02677a0 0x5599b0276b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.854+0000 7f6537fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599b0262420 0x5599b02755e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:53332/0 (socket says 172.21.15.100:53332) 2024-05-10T11:21:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.854+0000 7f652ffff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599b0266db0 0x5599b0275b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.854+0000 7f6537fff640 1 -- 172.21.15.100:0/2014715864 learned_addr learned my addr 172.21.15.100:0/2014715864 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:21:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.854+0000 7f653cd4c640 1 -- 172.21.15.100:0/2014715864 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599b0262420 msgr2=0x5599b02755e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.854+0000 7f653cd4c640 1 --2- 172.21.15.100:0/2014715864 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599b0262420 0x5599b02755e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.854+0000 7f653cd4c640 1 -- 172.21.15.100:0/2014715864 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599b0266db0 msgr2=0x5599b0275b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.854+0000 7f653cd4c640 1 --2- 172.21.15.100:0/2014715864 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599b0266db0 0x5599b0275b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.854+0000 7f653cd4c640 1 -- 172.21.15.100:0/2014715864 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599b0262250 con 0x5599b02677a0 2024-05-10T11:21:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.854+0000 7f6537fff640 1 --2- 172.21.15.100:0/2014715864 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599b0262420 0x5599b02755e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:21:52.861 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.854+0000 7f652ffff640 1 --2- 172.21.15.100:0/2014715864 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599b0266db0 0x5599b0275b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:21:52.861 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.854+0000 7f653cd4c640 1 --2- 172.21.15.100:0/2014715864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599b02677a0 0x5599b0276b30 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f65300d4d80 tx=0x7f65300d4960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f1c43e4d4ebebf server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:52.861 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.854+0000 7f65177fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:21:52.861 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.855+0000 7f65177fe640 1 -- 172.21.15.100:0/2014715864 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65300d0110 con 0x5599b02677a0 2024-05-10T11:21:52.861 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.855+0000 7f65177fe640 1 -- 172.21.15.100:0/2014715864 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f6530004d50 con 0x5599b02677a0 2024-05-10T11:21:52.861 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.855+0000 7f654159f740 1 -- 172.21.15.100:0/2014715864 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6530036040 con 0x5599b02677a0 2024-05-10T11:21:52.861 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.855+0000 7f65177fe640 1 -- 172.21.15.100:0/2014715864 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f65300d4bc0 con 0x5599b02677a0 2024-05-10T11:21:52.861 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.855+0000 7f654159f740 10 client.16062 fetch_fsmap learned FSMap version 29 2024-05-10T11:21:52.861 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.855+0000 7f654159f740 1 -- 172.21.15.100:0/2014715864 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5599b02e7420 con 0x5599b02677a0 2024-05-10T11:21:52.862 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.855+0000 7f65177fe640 1 -- 172.21.15.100:0/2014715864 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6530036040 con 0x5599b02677a0 2024-05-10T11:21:52.862 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.855+0000 7f65177fe640 1 -- 172.21.15.100:0/2014715864 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f65300caae0 con 0x5599b02677a0 2024-05-10T11:21:52.862 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.855+0000 7f654159f740 10 client.16062 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T11:21:52.862 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.855+0000 7f654159f740 10 client.16062 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:21:52.862 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.855+0000 7f654159f740 20 client.16062 populate_metadata read hostname 'smithi100' 2024-05-10T11:21:52.862 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.855+0000 7f654159f740 1 --2- 172.21.15.100:0/2014715864 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5599b026c910 0x5599b02e9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:52.862 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.855+0000 7f654159f740 4 client.16062 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:21:52.862 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.855+0000 7f654159f740 1 -- 172.21.15.100:0/2014715864 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5599b014e5f0 con 0x5599b026c910 2024-05-10T11:21:52.862 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.855+0000 7f6537fff640 1 --2- 172.21.15.100:0/2014715864 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5599b026c910 0x5599b02e9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:52.862 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.856+0000 7f6537fff640 1 --2- 172.21.15.100:0/2014715864 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5599b026c910 0x5599b02e9e30 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b9a42f44adb8aad server_cookie=afc6d34a0d9ab2e2 in_seq=0 out_seq=0 2024-05-10T11:21:52.862 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.856+0000 7f65177fe640 10 client.16062 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:21:52.872 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.872+0000 7f65177fe640 1 -- 172.21.15.100:0/2014715864 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5599b014e5f0 con 0x5599b026c910 2024-05-10T11:21:52.872 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.872+0000 7f65177fe640 10 client.16062 handle_command_reply: tid=0 2024-05-10T11:21:52.872 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.872+0000 7f654159f740 1 client.16062 shutdown 2024-05-10T11:21:52.873 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.872+0000 7f654159f740 1 -- 172.21.15.100:0/2014715864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599b02677a0 msgr2=0x5599b0276b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:52.873 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.872+0000 7f654159f740 1 --2- 172.21.15.100:0/2014715864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599b02677a0 0x5599b0276b30 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f65300d4d80 tx=0x7f65300d4960 comp rx=0 tx=0).stop 2024-05-10T11:21:52.873 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.872+0000 7f654159f740 1 -- 172.21.15.100:0/2014715864 shutdown_connections 2024-05-10T11:21:52.873 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.872+0000 7f654159f740 1 --2- 172.21.15.100:0/2014715864 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5599b026c910 0x5599b02e9e30 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:52.873 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.872+0000 7f654159f740 1 --2- 172.21.15.100:0/2014715864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599b02677a0 0x5599b0276b30 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:52.873 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.872+0000 7f65177fe640 0 client.16062 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:21:52.873 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.872+0000 7f654159f740 1 --2- 172.21.15.100:0/2014715864 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599b0266db0 0x5599b0275b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:52.873 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.872+0000 7f654159f740 1 --2- 172.21.15.100:0/2014715864 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599b0262420 0x5599b02755e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:52.873 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.872+0000 7f654159f740 1 -- 172.21.15.100:0/2014715864 >> 172.21.15.100:0/2014715864 conn(0x5599b026c3e0 msgr2=0x5599b0260cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:52.873 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.872+0000 7f654159f740 1 -- 172.21.15.100:0/2014715864 shutdown_connections 2024-05-10T11:21:52.873 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.872+0000 7f654159f740 1 -- 172.21.15.100:0/2014715864 wait complete. 2024-05-10T11:21:52.873 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.872+0000 7f654159f740 20 client.16062 trim_cache size 0 max 16384 2024-05-10T11:21:52.874 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:21:52.874 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:21:52.874 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:21:52.874 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:21:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.873+0000 7f653e7af640 1 -- 172.21.15.100:0/4114273803 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f65240b2920 msgr2=0x7f65240b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.873+0000 7f653e7af640 1 --2- 172.21.15.100:0/4114273803 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f65240b2920 0x7f65240b4de0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x5599b0268000 tx=0x7f6520015040 comp rx=0 tx=0).stop 2024-05-10T11:21:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.873+0000 7f653e7af640 1 -- 172.21.15.100:0/4114273803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65381034c0 msgr2=0x7f653819ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.873+0000 7f653e7af640 1 --2- 172.21.15.100:0/4114273803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65381034c0 0x7f653819ef60 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f6530031040 tx=0x7f6530003970 comp rx=0 tx=0).stop 2024-05-10T11:21:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.873+0000 7f653e7af640 1 -- 172.21.15.100:0/4114273803 shutdown_connections 2024-05-10T11:21:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.873+0000 7f653e7af640 1 --2- 172.21.15.100:0/4114273803 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f65240b2920 0x7f65240b4de0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.873+0000 7f653e7af640 1 --2- 172.21.15.100:0/4114273803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65381034c0 0x7f653819ef60 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.873+0000 7f653e7af640 1 --2- 172.21.15.100:0/4114273803 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6538102b00 0x7f6538197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.873+0000 7f653e7af640 1 --2- 172.21.15.100:0/4114273803 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6538101910 0x7f65381979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.873+0000 7f653e7af640 1 -- 172.21.15.100:0/4114273803 >> 172.21.15.100:0/4114273803 conn(0x7f65380fce80 msgr2=0x7f6538105e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.873+0000 7f653e7af640 1 -- 172.21.15.100:0/4114273803 shutdown_connections 2024-05-10T11:21:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:21:52.873+0000 7f653e7af640 1 -- 172.21.15.100:0/4114273803 wait complete. 2024-05-10T11:21:52.934 INFO:tasks.workunit.client.0.smithi100.stdout: 1 57074 3.05 MB/sec warmup 107 sec latency 12.460 ms 2024-05-10T11:21:53.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.685+0000 7f563ed76700 1 -- 172.21.15.100:0/3188500976 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56381023d0 msgr2=0x7f5638102850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:53.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.685+0000 7f563ed76700 1 --2- 172.21.15.100:0/3188500976 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56381023d0 0x7f5638102850 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f56240098f0 tx=0x7f5624009c00 comp rx=0 tx=0).stop 2024-05-10T11:21:53.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.685+0000 7f563ed76700 1 -- 172.21.15.100:0/3188500976 shutdown_connections 2024-05-10T11:21:53.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.685+0000 7f563ed76700 1 --2- 172.21.15.100:0/3188500976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5638102d90 0x7f563810f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:53.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.685+0000 7f563ed76700 1 --2- 172.21.15.100:0/3188500976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5638108440 0x7f5638108820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:53.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.685+0000 7f563ed76700 1 --2- 172.21.15.100:0/3188500976 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56381023d0 0x7f5638102850 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:53.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.685+0000 7f563ed76700 1 -- 172.21.15.100:0/3188500976 >> 172.21.15.100:0/3188500976 conn(0x7f56380793e0 msgr2=0x7f56380797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:53.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.685+0000 7f563ed76700 1 -- 172.21.15.100:0/3188500976 shutdown_connections 2024-05-10T11:21:53.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.685+0000 7f563ed76700 1 -- 172.21.15.100:0/3188500976 wait complete. 2024-05-10T11:21:53.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.686+0000 7f563ed76700 1 Processor -- start 2024-05-10T11:21:53.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.686+0000 7f563ed76700 1 -- start start 2024-05-10T11:21:53.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.686+0000 7f563ed76700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56381023d0 0x7f563819a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:53.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.686+0000 7f563ed76700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5638102d90 0x7f563819ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:53.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.686+0000 7f563ed76700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5638108440 0x7f56381a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:53.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.686+0000 7f563ed76700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5638112410 con 0x7f5638102d90 2024-05-10T11:21:53.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.686+0000 7f563ed76700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5638112290 con 0x7f5638108440 2024-05-10T11:21:53.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.686+0000 7f563ed76700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5638112590 con 0x7f56381023d0 2024-05-10T11:21:53.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.686+0000 7f5637fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5638102d90 0x7f563819ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:53.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.686+0000 7f5637fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5638102d90 0x7f563819ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42496/0 (socket says 172.21.15.100:42496) 2024-05-10T11:21:53.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.686+0000 7f5637fff700 1 -- 172.21.15.100:0/3722887726 learned_addr learned my addr 172.21.15.100:0/3722887726 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:21:53.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.686+0000 7f563cb12700 1 --2- 172.21.15.100:0/3722887726 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56381023d0 0x7f563819a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:53.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.686+0000 7f563d313700 1 --2- 172.21.15.100:0/3722887726 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5638108440 0x7f56381a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:53.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.687+0000 7f563cb12700 1 -- 172.21.15.100:0/3722887726 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5638108440 msgr2=0x7f56381a5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:53.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.687+0000 7f563cb12700 1 --2- 172.21.15.100:0/3722887726 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5638108440 0x7f56381a5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:53.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.687+0000 7f563cb12700 1 -- 172.21.15.100:0/3722887726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5638102d90 msgr2=0x7f563819ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:53.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.687+0000 7f563cb12700 1 --2- 172.21.15.100:0/3722887726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5638102d90 0x7f563819ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:53.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.687+0000 7f563cb12700 1 -- 172.21.15.100:0/3722887726 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56381a5ed0 con 0x7f56381023d0 2024-05-10T11:21:53.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.687+0000 7f563d313700 1 --2- 172.21.15.100:0/3722887726 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5638108440 0x7f56381a5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:21:53.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.687+0000 7f5637fff700 1 --2- 172.21.15.100:0/3722887726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5638102d90 0x7f563819ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:21:53.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.687+0000 7f563cb12700 1 --2- 172.21.15.100:0/3722887726 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56381023d0 0x7f563819a910 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f562c00eb90 tx=0x7f562c00eea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:53.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.687+0000 7f5635ffb700 1 -- 172.21.15.100:0/3722887726 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562c00cc90 con 0x7f56381023d0 2024-05-10T11:21:53.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.687+0000 7f5635ffb700 1 -- 172.21.15.100:0/3722887726 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f562c00ce30 con 0x7f56381023d0 2024-05-10T11:21:53.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.687+0000 7f563ed76700 1 -- 172.21.15.100:0/3722887726 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5638198760 con 0x7f56381023d0 2024-05-10T11:21:53.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.687+0000 7f5635ffb700 1 -- 172.21.15.100:0/3722887726 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562c003db0 con 0x7f56381023d0 2024-05-10T11:21:53.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.687+0000 7f563ed76700 1 -- 172.21.15.100:0/3722887726 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5638198ca0 con 0x7f56381023d0 2024-05-10T11:21:53.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.690+0000 7f5635ffb700 1 -- 172.21.15.100:0/3722887726 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f562c003f50 con 0x7f56381023d0 2024-05-10T11:21:53.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.691+0000 7f5635ffb700 1 --2- 172.21.15.100:0/3722887726 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f56280b2880 0x7f56280b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:53.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.691+0000 7f5637fff700 1 --2- 172.21.15.100:0/3722887726 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f56280b2880 0x7f56280b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:53.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.691+0000 7f5635ffb700 1 -- 172.21.15.100:0/3722887726 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f562c0e2760 con 0x7f56381023d0 2024-05-10T11:21:53.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.692+0000 7f5649609b80 1 Processor -- start 2024-05-10T11:21:53.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.692+0000 7f5649609b80 1 -- start start 2024-05-10T11:21:53.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.692+0000 7f5637fff700 1 --2- 172.21.15.100:0/3722887726 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f56280b2880 0x7f56280b4d40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5624009680 tx=0x7f5624019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:53.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.692+0000 7f5649609b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d647eab4d0 0x55d647eab8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:53.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.692+0000 7f5649609b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d647eabdf0 0x55d647eaf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:53.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.692+0000 7f5649609b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d647eafc80 0x55d647eb0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:53.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.692+0000 7f5649609b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55d647dfd860 con 0x55d647eab4d0 2024-05-10T11:21:53.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.692+0000 7f5649609b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d647d9c7c0 con 0x55d647eabdf0 2024-05-10T11:21:53.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.692+0000 7f5649609b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d647df9820 con 0x55d647eafc80 2024-05-10T11:21:53.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.692+0000 7f5637fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d647eabdf0 0x55d647eaf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:53.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.692+0000 7f563cb12700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d647eab4d0 0x55d647eab8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:53.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.692+0000 7f5637fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d647eabdf0 0x55d647eaf6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:57520/0 (socket says 172.21.15.100:57520) 2024-05-10T11:21:53.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.692+0000 7f563cb12700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d647eab4d0 0x55d647eab8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42510/0 (socket says 172.21.15.100:42510) 2024-05-10T11:21:53.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.692+0000 7f5637fff700 1 -- 172.21.15.100:0/750770811 learned_addr learned my addr 172.21.15.100:0/750770811 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:21:53.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.692+0000 7f563d313700 1 --2- 172.21.15.100:0/750770811 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d647eafc80 0x55d647eb0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:53.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.692+0000 7f5637fff700 1 -- 172.21.15.100:0/750770811 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d647eafc80 msgr2=0x55d647eb0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:53.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.692+0000 7f5637fff700 1 --2- 172.21.15.100:0/750770811 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d647eafc80 0x55d647eb0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:53.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.692+0000 7f5637fff700 1 -- 172.21.15.100:0/750770811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d647eab4d0 msgr2=0x55d647eab8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:53.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.692+0000 7f5637fff700 1 --2- 172.21.15.100:0/750770811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d647eab4d0 0x55d647eab8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:53.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.692+0000 7f5637fff700 1 -- 172.21.15.100:0/750770811 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d647d4a8d0 con 0x55d647eabdf0 2024-05-10T11:21:53.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.693+0000 7f5637fff700 1 --2- 172.21.15.100:0/750770811 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d647eabdf0 0x55d647eaf6f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5624009230 tx=0x7f562401f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e0fc2658108e4cc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:53.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.693+0000 7f5622ffd700 1 -- 172.21.15.100:0/750770811 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5624005450 con 0x55d647eabdf0 2024-05-10T11:21:53.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.693+0000 7f5622ffd700 1 -- 172.21.15.100:0/750770811 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f56240055f0 con 0x55d647eabdf0 2024-05-10T11:21:53.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.693+0000 7f5622ffd700 1 -- 172.21.15.100:0/750770811 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562400b7a0 con 0x55d647eabdf0 2024-05-10T11:21:53.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.694+0000 7f5649609b80 1 -- 172.21.15.100:0/750770811 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d647eabdf0 msgr2=0x55d647eaf6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:53.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.694+0000 7f5649609b80 1 --2- 172.21.15.100:0/750770811 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d647eabdf0 0x55d647eaf6f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5624009230 tx=0x7f562401f000 comp rx=0 tx=0).stop 2024-05-10T11:21:53.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.694+0000 7f5649609b80 1 -- 172.21.15.100:0/750770811 shutdown_connections 2024-05-10T11:21:53.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.694+0000 7f5649609b80 1 --2- 172.21.15.100:0/750770811 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d647eafc80 0x55d647eb0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:53.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.694+0000 7f5649609b80 1 --2- 172.21.15.100:0/750770811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d647eab4d0 0x55d647eab8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:53.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.694+0000 7f5649609b80 1 --2- 172.21.15.100:0/750770811 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d647eabdf0 0x55d647eaf6f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:53.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.694+0000 7f5649609b80 1 -- 172.21.15.100:0/750770811 >> 172.21.15.100:0/750770811 conn(0x55d647dd7100 msgr2=0x55d647ea5c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:53.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.694+0000 7f5649609b80 1 -- 172.21.15.100:0/750770811 shutdown_connections 2024-05-10T11:21:53.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.694+0000 7f5649609b80 1 -- 172.21.15.100:0/750770811 wait complete. 2024-05-10T11:21:53.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.694+0000 7f5649609b80 1 -- start start 2024-05-10T11:21:53.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.694+0000 7f5649609b80 1 Processor -- start 2024-05-10T11:21:53.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.695+0000 7f5649609b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d647eab4d0 0x55d647eb7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:53.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.695+0000 7f5649609b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d647eabdf0 0x55d647eb44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:53.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.695+0000 7f5649609b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d647eafc80 0x55d647eb49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:53.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.695+0000 7f5649609b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55d647eb7dd0 con 0x55d647eab4d0 2024-05-10T11:21:53.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.695+0000 7f5649609b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d647d63c30 con 0x55d647eabdf0 2024-05-10T11:21:53.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.695+0000 7f5649609b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d647eb7f50 con 0x55d647eafc80 2024-05-10T11:21:53.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.695+0000 7f5649609b80 10 client.0 _ll_register_callbacks cb 0x55d647d95920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:21:53.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.695+0000 7f563cb12700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d647eab4d0 0x55d647eb7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:53.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.695+0000 7f563cb12700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d647eab4d0 0x55d647eb7100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42514/0 (socket says 172.21.15.100:42514) 2024-05-10T11:21:53.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.695+0000 7f563cb12700 1 -- 172.21.15.100:0/3688114646 learned_addr learned my addr 172.21.15.100:0/3688114646 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:21:53.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.695+0000 7f5637fff700 1 --2- 172.21.15.100:0/3688114646 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d647eabdf0 0x55d647eb44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:53.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.695+0000 7f563d313700 1 --2- 172.21.15.100:0/3688114646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d647eafc80 0x55d647eb49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:53.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.695+0000 7f563cb12700 1 -- 172.21.15.100:0/3688114646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d647eafc80 msgr2=0x55d647eb49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:53.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.695+0000 7f563cb12700 1 --2- 172.21.15.100:0/3688114646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d647eafc80 0x55d647eb49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:53.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.695+0000 7f563cb12700 1 -- 172.21.15.100:0/3688114646 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d647eabdf0 msgr2=0x55d647eb44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:53.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.695+0000 7f563cb12700 1 --2- 172.21.15.100:0/3688114646 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d647eabdf0 0x55d647eb44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:53.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.695+0000 7f563cb12700 1 -- 172.21.15.100:0/3688114646 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d647d99690 con 0x55d647eab4d0 2024-05-10T11:21:53.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.695+0000 7f563d313700 1 --2- 172.21.15.100:0/3688114646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d647eafc80 0x55d647eb49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:21:53.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.695+0000 7f5637fff700 1 --2- 172.21.15.100:0/3688114646 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d647eabdf0 0x55d647eb44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:21:53.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.695+0000 7f563cb12700 1 --2- 172.21.15.100:0/3688114646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d647eab4d0 0x55d647eb7100 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f562c0dff50 tx=0x7f562c0a9460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4972f1e70d3f6ee server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:53.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.695+0000 7f5617fff700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:21:53.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.696+0000 7f5617fff700 1 -- 172.21.15.100:0/3688114646 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562c0c2070 con 0x55d647eab4d0 2024-05-10T11:21:53.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.696+0000 7f5617fff700 1 -- 172.21.15.100:0/3688114646 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f562c0a9c80 con 0x55d647eab4d0 2024-05-10T11:21:53.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.696+0000 7f5649609b80 1 -- 172.21.15.100:0/3688114646 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56240055f0 con 0x55d647eab4d0 2024-05-10T11:21:53.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.696+0000 7f5617fff700 1 -- 172.21.15.100:0/3688114646 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f562c0b21c0 con 0x55d647eab4d0 2024-05-10T11:21:53.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.696+0000 7f5649609b80 10 client.16074 fetch_fsmap learned FSMap version 29 2024-05-10T11:21:53.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.696+0000 7f5649609b80 1 -- 172.21.15.100:0/3688114646 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d647ec2710 con 0x55d647eab4d0 2024-05-10T11:21:53.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.696+0000 7f5617fff700 1 -- 172.21.15.100:0/3688114646 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f56240055f0 con 0x55d647eab4d0 2024-05-10T11:21:53.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.696+0000 7f5617fff700 1 -- 172.21.15.100:0/3688114646 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f562c0b23f0 con 0x55d647eab4d0 2024-05-10T11:21:53.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.696+0000 7f5649609b80 10 client.16074 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T11:21:53.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.696+0000 7f5649609b80 10 client.16074 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:21:53.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.696+0000 7f5649609b80 20 client.16074 populate_metadata read hostname 'smithi100' 2024-05-10T11:21:53.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.696+0000 7f5649609b80 1 --2- 172.21.15.100:0/3688114646 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d647ec4ca0 0x55d647ec70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:53.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.696+0000 7f5649609b80 4 client.16074 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:21:53.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.696+0000 7f5649609b80 1 -- 172.21.15.100:0/3688114646 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d647e77650 con 0x55d647ec4ca0 2024-05-10T11:21:53.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.696+0000 7f5637fff700 1 --2- 172.21.15.100:0/3688114646 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d647ec4ca0 0x55d647ec70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:53.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.697+0000 7f5637fff700 1 --2- 172.21.15.100:0/3688114646 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d647ec4ca0 0x55d647ec70d0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79d8ee67bab5e74b server_cookie=a714df337a4297f6 in_seq=0 out_seq=0 2024-05-10T11:21:53.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.697+0000 7f5617fff700 10 client.16074 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:21:53.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.698+0000 7f5617fff700 1 -- 172.21.15.100:0/3688114646 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d647e77650 con 0x55d647ec4ca0 2024-05-10T11:21:53.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.698+0000 7f5617fff700 10 client.16074 handle_command_reply: tid=0 2024-05-10T11:21:53.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.698+0000 7f5649609b80 1 client.16074 shutdown 2024-05-10T11:21:53.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.698+0000 7f5649609b80 1 -- 172.21.15.100:0/3688114646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d647eab4d0 msgr2=0x55d647eb7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:53.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.698+0000 7f5649609b80 1 --2- 172.21.15.100:0/3688114646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d647eab4d0 0x55d647eb7100 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f562c0dff50 tx=0x7f562c0a9460 comp rx=0 tx=0).stop 2024-05-10T11:21:53.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.698+0000 7f5649609b80 1 -- 172.21.15.100:0/3688114646 shutdown_connections 2024-05-10T11:21:53.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.698+0000 7f5649609b80 1 --2- 172.21.15.100:0/3688114646 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d647ec4ca0 0x55d647ec70d0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:53.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.698+0000 7f5649609b80 1 --2- 172.21.15.100:0/3688114646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d647eafc80 0x55d647eb49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:53.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.698+0000 7f5617fff700 0 client.16074 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:21:53.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.698+0000 7f5649609b80 1 --2- 172.21.15.100:0/3688114646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d647eab4d0 0x55d647eb7100 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:53.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.698+0000 7f5649609b80 1 --2- 172.21.15.100:0/3688114646 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d647eabdf0 0x55d647eb44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:53.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.698+0000 7f5649609b80 1 -- 172.21.15.100:0/3688114646 >> 172.21.15.100:0/3688114646 conn(0x55d647dd7100 msgr2=0x55d647ea0720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:53.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.698+0000 7f5649609b80 1 -- 172.21.15.100:0/3688114646 shutdown_connections 2024-05-10T11:21:53.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.698+0000 7f5649609b80 1 -- 172.21.15.100:0/3688114646 wait complete. 2024-05-10T11:21:53.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.698+0000 7f5649609b80 20 client.16074 trim_cache size 0 max 16384 2024-05-10T11:21:53.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.713+0000 7f5649609b80 1 Processor -- start 2024-05-10T11:21:53.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.713+0000 7f5649609b80 1 -- start start 2024-05-10T11:21:53.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.714+0000 7f5649609b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d647eab4d0 0x55d647ec0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:53.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.714+0000 7f5649609b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d647eabdf0 0x55d647ea05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:53.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.714+0000 7f5649609b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d647eafc80 0x55d647ecc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:53.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.714+0000 7f5649609b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f56381af760 con 0x55d647eafc80 2024-05-10T11:21:53.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.714+0000 7f5649609b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f56381acff0 con 0x55d647eabdf0 2024-05-10T11:21:53.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.714+0000 7f5649609b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f56381aae40 con 0x55d647eab4d0 2024-05-10T11:21:53.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.714+0000 7f563d313700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d647eafc80 0x55d647ecc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:53.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.714+0000 7f563d313700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d647eafc80 0x55d647ecc9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42522/0 (socket says 172.21.15.100:42522) 2024-05-10T11:21:53.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.714+0000 7f563d313700 1 -- 172.21.15.100:0/466587818 learned_addr learned my addr 172.21.15.100:0/466587818 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:21:53.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.714+0000 7f563cb12700 1 --2- 172.21.15.100:0/466587818 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d647eab4d0 0x55d647ec0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:53.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.714+0000 7f5637fff700 1 --2- 172.21.15.100:0/466587818 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d647eabdf0 0x55d647ea05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:53.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.714+0000 7f563d313700 1 -- 172.21.15.100:0/466587818 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d647eab4d0 msgr2=0x55d647ec0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:53.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.714+0000 7f563d313700 1 --2- 172.21.15.100:0/466587818 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d647eab4d0 0x55d647ec0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:53.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.714+0000 7f563d313700 1 -- 172.21.15.100:0/466587818 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d647eabdf0 msgr2=0x55d647ea05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:53.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.714+0000 7f563d313700 1 --2- 172.21.15.100:0/466587818 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d647eabdf0 0x55d647ea05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:53.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.714+0000 7f563d313700 1 -- 172.21.15.100:0/466587818 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d647ec7750 con 0x55d647eafc80 2024-05-10T11:21:53.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.714+0000 7f563d313700 1 --2- 172.21.15.100:0/466587818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d647eafc80 0x55d647ecc9a0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f5630010c80 tx=0x7f563000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e50b44e0f984c91e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:53.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.714+0000 7f56227fc700 1 -- 172.21.15.100:0/466587818 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5630021070 con 0x55d647eafc80 2024-05-10T11:21:53.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.715+0000 7f56227fc700 1 -- 172.21.15.100:0/466587818 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5630004bb0 con 0x55d647eafc80 2024-05-10T11:21:53.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.715+0000 7f56227fc700 1 -- 172.21.15.100:0/466587818 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56300042b0 con 0x55d647eafc80 2024-05-10T11:21:53.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.715+0000 7f5649609b80 1 -- 172.21.15.100:0/466587818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d647eafc80 msgr2=0x55d647ecc9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:53.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.715+0000 7f5649609b80 1 --2- 172.21.15.100:0/466587818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d647eafc80 0x55d647ecc9a0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f5630010c80 tx=0x7f563000b040 comp rx=0 tx=0).stop 2024-05-10T11:21:53.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.715+0000 7f5649609b80 1 -- 172.21.15.100:0/466587818 shutdown_connections 2024-05-10T11:21:53.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.715+0000 7f5649609b80 1 --2- 172.21.15.100:0/466587818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d647eafc80 0x55d647ecc9a0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:53.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.715+0000 7f5649609b80 1 --2- 172.21.15.100:0/466587818 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d647eab4d0 0x55d647ec0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:53.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.715+0000 7f5649609b80 1 --2- 172.21.15.100:0/466587818 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d647eabdf0 0x55d647ea05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:53.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.715+0000 7f5649609b80 1 -- 172.21.15.100:0/466587818 >> 172.21.15.100:0/466587818 conn(0x55d647dd7100 msgr2=0x55d647e9fe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:53.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.715+0000 7f5649609b80 1 -- 172.21.15.100:0/466587818 shutdown_connections 2024-05-10T11:21:53.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.715+0000 7f5649609b80 1 -- 172.21.15.100:0/466587818 wait complete. 2024-05-10T11:21:53.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.716+0000 7f5649609b80 1 -- start start 2024-05-10T11:21:53.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.716+0000 7f5649609b80 1 Processor -- start 2024-05-10T11:21:53.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.716+0000 7f5649609b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d647eab4d0 0x55d647ebd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:53.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.716+0000 7f5649609b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d647eabdf0 0x55d647ebd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:53.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.716+0000 7f5649609b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d647eafc80 0x55d647eb8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:53.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.716+0000 7f5649609b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f56381ab4b0 con 0x55d647eafc80 2024-05-10T11:21:53.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.716+0000 7f5649609b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f56381ac920 con 0x55d647eabdf0 2024-05-10T11:21:53.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.716+0000 7f5649609b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f56381abb80 con 0x55d647eab4d0 2024-05-10T11:21:53.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.716+0000 7f5649609b80 10 client.0 _ll_register_callbacks cb 0x55d647d95920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:21:53.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.716+0000 7f563d313700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d647eafc80 0x55d647eb8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:53.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.716+0000 7f563d313700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d647eafc80 0x55d647eb8800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42526/0 (socket says 172.21.15.100:42526) 2024-05-10T11:21:53.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.716+0000 7f563d313700 1 -- 172.21.15.100:0/3761789263 learned_addr learned my addr 172.21.15.100:0/3761789263 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:21:53.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.716+0000 7f563cb12700 1 --2- 172.21.15.100:0/3761789263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d647eab4d0 0x55d647ebd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:53.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.716+0000 7f5637fff700 1 --2- 172.21.15.100:0/3761789263 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d647eabdf0 0x55d647ebd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:53.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.717+0000 7f563cb12700 1 -- 172.21.15.100:0/3761789263 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d647eabdf0 msgr2=0x55d647ebd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:53.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.717+0000 7f563cb12700 1 --2- 172.21.15.100:0/3761789263 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d647eabdf0 0x55d647ebd790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:53.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.717+0000 7f563cb12700 1 -- 172.21.15.100:0/3761789263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d647eafc80 msgr2=0x55d647eb8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:53.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.717+0000 7f563cb12700 1 --2- 172.21.15.100:0/3761789263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d647eafc80 0x55d647eb8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:53.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.717+0000 7f563cb12700 1 -- 172.21.15.100:0/3761789263 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d647eb90c0 con 0x55d647eab4d0 2024-05-10T11:21:53.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.717+0000 7f563d313700 1 --2- 172.21.15.100:0/3761789263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d647eafc80 0x55d647eb8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:21:53.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.717+0000 7f563cb12700 1 --2- 172.21.15.100:0/3761789263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d647eab4d0 0x55d647ebd250 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f562c0e0270 tx=0x7f562c06f500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea05b9ab93ae3042 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:53.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.717+0000 7f5617fff700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:21:53.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.717+0000 7f5617fff700 1 -- 172.21.15.100:0/3761789263 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562c0c2070 con 0x55d647eab4d0 2024-05-10T11:21:53.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.717+0000 7f5617fff700 1 -- 172.21.15.100:0/3761789263 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f562c0cc0d0 con 0x55d647eab4d0 2024-05-10T11:21:53.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.717+0000 7f5649609b80 1 -- 172.21.15.100:0/3761789263 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5630004bb0 con 0x55d647eab4d0 2024-05-10T11:21:53.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.717+0000 7f5617fff700 1 -- 172.21.15.100:0/3761789263 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f562c0a9530 con 0x55d647eab4d0 2024-05-10T11:21:53.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.717+0000 7f5649609b80 10 client.25582 fetch_fsmap learned FSMap version 29 2024-05-10T11:21:53.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.717+0000 7f5649609b80 1 -- 172.21.15.100:0/3761789263 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d647f2f110 con 0x55d647eab4d0 2024-05-10T11:21:53.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.717+0000 7f5617fff700 1 -- 172.21.15.100:0/3761789263 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5630004bb0 con 0x55d647eab4d0 2024-05-10T11:21:53.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.718+0000 7f5617fff700 1 -- 172.21.15.100:0/3761789263 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f562c0a5390 con 0x55d647eab4d0 2024-05-10T11:21:53.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.718+0000 7f5649609b80 10 client.25582 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T11:21:53.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.718+0000 7f5649609b80 10 client.25582 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:21:53.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.718+0000 7f5649609b80 20 client.25582 populate_metadata read hostname 'smithi100' 2024-05-10T11:21:53.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.718+0000 7f5649609b80 1 --2- 172.21.15.100:0/3761789263 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d647f2f6c0 0x55d647f31ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:53.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.718+0000 7f5649609b80 4 client.25582 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:21:53.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.718+0000 7f5649609b80 1 -- 172.21.15.100:0/3761789263 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d647e0b240 con 0x55d647f2f6c0 2024-05-10T11:21:53.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.718+0000 7f5637fff700 1 --2- 172.21.15.100:0/3761789263 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d647f2f6c0 0x55d647f31ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:53.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.718+0000 7f5617fff700 10 client.25582 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:21:53.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.718+0000 7f5637fff700 1 --2- 172.21.15.100:0/3761789263 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d647f2f6c0 0x55d647f31ae0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a068b25cf18b95f7 server_cookie=c59668e6152b5fa6 in_seq=0 out_seq=0 2024-05-10T11:21:53.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.719+0000 7f5617fff700 1 -- 172.21.15.100:0/3761789263 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d647e0b240 con 0x55d647f2f6c0 2024-05-10T11:21:53.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.719+0000 7f5617fff700 10 client.25582 handle_command_reply: tid=0 2024-05-10T11:21:53.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.719+0000 7f5649609b80 1 client.25582 shutdown 2024-05-10T11:21:53.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.719+0000 7f5649609b80 1 -- 172.21.15.100:0/3761789263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d647eab4d0 msgr2=0x55d647ebd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:53.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.719+0000 7f5649609b80 1 --2- 172.21.15.100:0/3761789263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d647eab4d0 0x55d647ebd250 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f562c0e0270 tx=0x7f562c06f500 comp rx=0 tx=0).stop 2024-05-10T11:21:53.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.719+0000 7f5649609b80 1 -- 172.21.15.100:0/3761789263 shutdown_connections 2024-05-10T11:21:53.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.719+0000 7f5649609b80 1 --2- 172.21.15.100:0/3761789263 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d647f2f6c0 0x55d647f31ae0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:53.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.719+0000 7f5617fff700 0 client.25582 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:21:53.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.719+0000 7f5649609b80 1 --2- 172.21.15.100:0/3761789263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d647eafc80 0x55d647eb8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:53.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.719+0000 7f5649609b80 1 --2- 172.21.15.100:0/3761789263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d647eab4d0 0x55d647ebd250 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:53.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.719+0000 7f5649609b80 1 --2- 172.21.15.100:0/3761789263 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d647eabdf0 0x55d647ebd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:53.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.719+0000 7f5649609b80 1 -- 172.21.15.100:0/3761789263 >> 172.21.15.100:0/3761789263 conn(0x55d647dd7100 msgr2=0x55d647eb3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:53.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.719+0000 7f5649609b80 1 -- 172.21.15.100:0/3761789263 shutdown_connections 2024-05-10T11:21:53.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.719+0000 7f5649609b80 1 -- 172.21.15.100:0/3761789263 wait complete. 2024-05-10T11:21:53.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.719+0000 7f5649609b80 20 client.25582 trim_cache size 0 max 16384 2024-05-10T11:21:53.727 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:21:53.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.720+0000 7f563ed76700 1 -- 172.21.15.100:0/3722887726 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f56280b2880 msgr2=0x7f56280b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:53.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.720+0000 7f563ed76700 1 --2- 172.21.15.100:0/3722887726 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f56280b2880 0x7f56280b4d40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5624009680 tx=0x7f5624019040 comp rx=0 tx=0).stop 2024-05-10T11:21:53.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.720+0000 7f563ed76700 1 -- 172.21.15.100:0/3722887726 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56381023d0 msgr2=0x7f563819a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:53.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.720+0000 7f563ed76700 1 --2- 172.21.15.100:0/3722887726 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56381023d0 0x7f563819a910 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f562c00eb90 tx=0x7f562c00eea0 comp rx=0 tx=0).stop 2024-05-10T11:21:53.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.720+0000 7f563ed76700 1 -- 172.21.15.100:0/3722887726 shutdown_connections 2024-05-10T11:21:53.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.720+0000 7f563ed76700 1 --2- 172.21.15.100:0/3722887726 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f56280b2880 0x7f56280b4d40 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:53.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.720+0000 7f563ed76700 1 --2- 172.21.15.100:0/3722887726 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5638108440 0x7f56381a5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:53.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.720+0000 7f563ed76700 1 --2- 172.21.15.100:0/3722887726 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56381023d0 0x7f563819a910 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:53.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.720+0000 7f563ed76700 1 --2- 172.21.15.100:0/3722887726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5638102d90 0x7f563819ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:53.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.720+0000 7f563ed76700 1 -- 172.21.15.100:0/3722887726 >> 172.21.15.100:0/3722887726 conn(0x7f56380793e0 msgr2=0x7f5638101240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:53.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.720+0000 7f563ed76700 1 -- 172.21.15.100:0/3722887726 shutdown_connections 2024-05-10T11:21:53.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:53.720+0000 7f563ed76700 1 -- 172.21.15.100:0/3722887726 wait complete. 2024-05-10T11:21:53.934 INFO:tasks.workunit.client.0.smithi100.stdout: 1 57350 3.03 MB/sec warmup 108 sec latency 22.075 ms 2024-05-10T11:21:54.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:21:54 smithi100 ceph-mon[30296]: pgmap v525: 129 pgs: 129 active+clean; 39 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 1.4 MiB/s wr, 141 op/s 2024-05-10T11:21:54.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:21:54 smithi178 ceph-mon[33359]: pgmap v525: 129 pgs: 129 active+clean; 39 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 1.4 MiB/s wr, 141 op/s 2024-05-10T11:21:54.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:21:54 smithi188 ceph-mon[34075]: pgmap v525: 129 pgs: 129 active+clean; 39 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 1.4 MiB/s wr, 141 op/s 2024-05-10T11:21:54.934 INFO:tasks.workunit.client.0.smithi100.stdout: 1 57839 3.02 MB/sec warmup 109 sec latency 16.060 ms 2024-05-10T11:21:55.616 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T11:21:55.616 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T11:21:55.883 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:21:55.934 INFO:tasks.workunit.client.0.smithi100.stdout: 1 58430 3.02 MB/sec warmup 110 sec latency 15.509 ms 2024-05-10T11:21:56.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:21:56 smithi100 ceph-mon[30296]: pgmap v526: 129 pgs: 129 active+clean; 36 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 1.4 MiB/s wr, 142 op/s 2024-05-10T11:21:56.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:21:56 smithi178 ceph-mon[33359]: pgmap v526: 129 pgs: 129 active+clean; 36 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 1.4 MiB/s wr, 142 op/s 2024-05-10T11:21:56.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:21:56 smithi188 ceph-mon[34075]: pgmap v526: 129 pgs: 129 active+clean; 36 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 1.4 MiB/s wr, 142 op/s 2024-05-10T11:21:56.934 INFO:tasks.workunit.client.0.smithi100.stdout: 1 59102 3.01 MB/sec warmup 111 sec latency 20.747 ms 2024-05-10T11:21:57.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.927+0000 7f13eff4d700 1 -- 172.21.15.100:0/2564970740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13e8103ee0 msgr2=0x7f13e8110370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:57.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.927+0000 7f13eff4d700 1 --2- 172.21.15.100:0/2564970740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13e8103ee0 0x7f13e8110370 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f13e000b0a0 tx=0x7f13e000b3b0 comp rx=0 tx=0).stop 2024-05-10T11:21:57.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.927+0000 7f13eff4d700 1 -- 172.21.15.100:0/2564970740 shutdown_connections 2024-05-10T11:21:57.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.927+0000 7f13eff4d700 1 --2- 172.21.15.100:0/2564970740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13e8103ee0 0x7f13e8110370 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:57.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.927+0000 7f13eff4d700 1 --2- 172.21.15.100:0/2564970740 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13e8109530 0x7f13e8109910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:57.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.927+0000 7f13eff4d700 1 --2- 172.21.15.100:0/2564970740 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13e8103520 0x7f13e81039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:57.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.927+0000 7f13eff4d700 1 -- 172.21.15.100:0/2564970740 >> 172.21.15.100:0/2564970740 conn(0x7f13e807a570 msgr2=0x7f13e807a980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:57.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.928+0000 7f13eff4d700 1 -- 172.21.15.100:0/2564970740 shutdown_connections 2024-05-10T11:21:57.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.928+0000 7f13eff4d700 1 -- 172.21.15.100:0/2564970740 wait complete. 2024-05-10T11:21:57.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.928+0000 7f13eff4d700 1 Processor -- start 2024-05-10T11:21:57.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.928+0000 7f13eff4d700 1 -- start start 2024-05-10T11:21:57.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.928+0000 7f13eff4d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13e8103520 0x7f13e80757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:57.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.928+0000 7f13eff4d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13e8103ee0 0x7f13e8075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:57.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.928+0000 7f13eff4d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13e8109530 0x7f13e8076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:57.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.928+0000 7f13eff4d700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f13e8113530 con 0x7f13e8109530 2024-05-10T11:21:57.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.928+0000 7f13eff4d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f13e81133b0 con 0x7f13e8103520 2024-05-10T11:21:57.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.928+0000 7f13eff4d700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f13e81136b0 con 0x7f13e8103ee0 2024-05-10T11:21:57.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.929+0000 7f13ee4ea700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13e8109530 0x7f13e8076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:57.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.929+0000 7f13ee4ea700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13e8109530 0x7f13e8076220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56468/0 (socket says 172.21.15.100:56468) 2024-05-10T11:21:57.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.929+0000 7f13ee4ea700 1 -- 172.21.15.100:0/2473293083 learned_addr learned my addr 172.21.15.100:0/2473293083 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:21:57.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.929+0000 7f13edce9700 1 --2- 172.21.15.100:0/2473293083 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13e8103520 0x7f13e80757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:57.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.929+0000 7f13ed4e8700 1 --2- 172.21.15.100:0/2473293083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13e8103ee0 0x7f13e8075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:57.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.929+0000 7f13ee4ea700 1 -- 172.21.15.100:0/2473293083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13e8103ee0 msgr2=0x7f13e8075ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:57.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.929+0000 7f13ee4ea700 1 --2- 172.21.15.100:0/2473293083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13e8103ee0 0x7f13e8075ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:57.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.929+0000 7f13ee4ea700 1 -- 172.21.15.100:0/2473293083 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13e8103520 msgr2=0x7f13e80757a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:57.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.929+0000 7f13ee4ea700 1 --2- 172.21.15.100:0/2473293083 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13e8103520 0x7f13e80757a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:57.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.929+0000 7f13ee4ea700 1 -- 172.21.15.100:0/2473293083 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13e81a7f50 con 0x7f13e8109530 2024-05-10T11:21:57.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.929+0000 7f13edce9700 1 --2- 172.21.15.100:0/2473293083 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13e8103520 0x7f13e80757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:21:57.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.929+0000 7f13ee4ea700 1 --2- 172.21.15.100:0/2473293083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13e8109530 0x7f13e8076220 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f13e000b780 tx=0x7f13e0004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:57.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.929+0000 7f13deffd700 1 -- 172.21.15.100:0/2473293083 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e0021070 con 0x7f13e8109530 2024-05-10T11:21:57.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.929+0000 7f13eff4d700 1 -- 172.21.15.100:0/2473293083 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13e81a81e0 con 0x7f13e8109530 2024-05-10T11:21:57.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.929+0000 7f13deffd700 1 -- 172.21.15.100:0/2473293083 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f13e0007b80 con 0x7f13e8109530 2024-05-10T11:21:57.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.929+0000 7f13deffd700 1 -- 172.21.15.100:0/2473293083 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e0005040 con 0x7f13e8109530 2024-05-10T11:21:57.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.929+0000 7f13eff4d700 1 -- 172.21.15.100:0/2473293083 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13e81a8690 con 0x7f13e8109530 2024-05-10T11:21:57.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.932+0000 7f13deffd700 1 -- 172.21.15.100:0/2473293083 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f13e0024070 con 0x7f13e8109530 2024-05-10T11:21:57.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.932+0000 7f13deffd700 1 --2- 172.21.15.100:0/2473293083 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f13d40b2880 0x7f13d40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:57.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.933+0000 7f13edce9700 1 --2- 172.21.15.100:0/2473293083 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f13d40b2880 0x7f13d40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:57.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.933+0000 7f13deffd700 1 -- 172.21.15.100:0/2473293083 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f13e000f020 con 0x7f13e8109530 2024-05-10T11:21:57.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.933+0000 7f13edce9700 1 --2- 172.21.15.100:0/2473293083 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f13d40b2880 0x7f13d40b4d40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f13e4009fd0 tx=0x7f13e4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:57.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.933+0000 7f13fa7e0b80 1 Processor -- start 2024-05-10T11:21:57.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.934+0000 7f13fa7e0b80 1 -- start start 2024-05-10T11:21:57.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.934+0000 7f13fa7e0b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5654ae57e550 0x5654ae57e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:57.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.934+0000 7f13fa7e0b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654ae57ee70 0x5654ae582770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:57.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.934+0000 7f13fa7e0b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654ae582d00 0x5654ae583180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:57.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.934+0000 7f13fa7e0b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5654ae4d08e0 con 0x5654ae582d00 2024-05-10T11:21:57.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.934+0000 7f13fa7e0b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5654ae46f840 con 0x5654ae57e550 2024-05-10T11:21:57.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.934+0000 7f13fa7e0b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5654ae4cc8a0 con 0x5654ae57ee70 2024-05-10T11:21:57.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.934+0000 7f13ed4e8700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654ae57ee70 0x5654ae582770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:57.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.934+0000 7f13edce9700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5654ae57e550 0x5654ae57e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:57.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.934+0000 7f13ee4ea700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654ae582d00 0x5654ae583180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:57.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.934+0000 7f13ed4e8700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654ae57ee70 0x5654ae582770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:49740/0 (socket says 172.21.15.100:49740) 2024-05-10T11:21:57.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.934+0000 7f13edce9700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5654ae57e550 0x5654ae57e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:37200/0 (socket says 172.21.15.100:37200) 2024-05-10T11:21:57.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.934+0000 7f13ed4e8700 1 -- 172.21.15.100:0/286646731 learned_addr learned my addr 172.21.15.100:0/286646731 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:21:57.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.934+0000 7f13ee4ea700 1 -- 172.21.15.100:0/286646731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654ae57ee70 msgr2=0x5654ae582770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:57.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.934+0000 7f13ee4ea700 1 --2- 172.21.15.100:0/286646731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654ae57ee70 0x5654ae582770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:57.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.934+0000 7f13ee4ea700 1 -- 172.21.15.100:0/286646731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5654ae57e550 msgr2=0x5654ae57e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:57.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.934+0000 7f13ee4ea700 1 --2- 172.21.15.100:0/286646731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5654ae57e550 0x5654ae57e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:57.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.934+0000 7f13ee4ea700 1 -- 172.21.15.100:0/286646731 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654ae41d950 con 0x5654ae582d00 2024-05-10T11:21:57.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.934+0000 7f13ed4e8700 1 --2- 172.21.15.100:0/286646731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654ae57ee70 0x5654ae582770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:21:57.940 INFO:tasks.workunit.client.0.smithi100.stdout: 1 59694 3.03 MB/sec warmup 112 sec latency 15.926 ms 2024-05-10T11:21:57.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.935+0000 7f13ee4ea700 1 --2- 172.21.15.100:0/286646731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654ae582d00 0x5654ae583180 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f13e00e1170 tx=0x7f13e00095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ba20414e4ed51a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:57.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.935+0000 7f13d3fff700 1 -- 172.21.15.100:0/286646731 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e00b33f0 con 0x5654ae582d00 2024-05-10T11:21:57.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.935+0000 7f13d3fff700 1 -- 172.21.15.100:0/286646731 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f13e00aacd0 con 0x5654ae582d00 2024-05-10T11:21:57.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.935+0000 7f13d3fff700 1 -- 172.21.15.100:0/286646731 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e00ab190 con 0x5654ae582d00 2024-05-10T11:21:57.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.936+0000 7f13fa7e0b80 1 -- 172.21.15.100:0/286646731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654ae582d00 msgr2=0x5654ae583180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:57.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.936+0000 7f13fa7e0b80 1 --2- 172.21.15.100:0/286646731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654ae582d00 0x5654ae583180 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f13e00e1170 tx=0x7f13e00095a0 comp rx=0 tx=0).stop 2024-05-10T11:21:57.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.936+0000 7f13fa7e0b80 1 -- 172.21.15.100:0/286646731 shutdown_connections 2024-05-10T11:21:57.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.936+0000 7f13fa7e0b80 1 --2- 172.21.15.100:0/286646731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654ae582d00 0x5654ae583180 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:57.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.936+0000 7f13fa7e0b80 1 --2- 172.21.15.100:0/286646731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5654ae57e550 0x5654ae57e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:57.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.936+0000 7f13fa7e0b80 1 --2- 172.21.15.100:0/286646731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654ae57ee70 0x5654ae582770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:57.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.936+0000 7f13fa7e0b80 1 -- 172.21.15.100:0/286646731 >> 172.21.15.100:0/286646731 conn(0x5654ae4aa180 msgr2=0x5654ae578cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:57.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.936+0000 7f13fa7e0b80 1 -- 172.21.15.100:0/286646731 shutdown_connections 2024-05-10T11:21:57.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.936+0000 7f13fa7e0b80 1 -- 172.21.15.100:0/286646731 wait complete. 2024-05-10T11:21:57.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.936+0000 7f13fa7e0b80 1 -- start start 2024-05-10T11:21:57.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.936+0000 7f13fa7e0b80 1 Processor -- start 2024-05-10T11:21:57.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.937+0000 7f13fa7e0b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654ae57e550 0x5654ae58a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:57.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.937+0000 7f13fa7e0b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5654ae57ee70 0x5654ae587530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:57.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.937+0000 7f13fa7e0b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654ae582d00 0x5654ae587a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:57.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.937+0000 7f13fa7e0b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5654ae58acd0 con 0x5654ae582d00 2024-05-10T11:21:57.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.937+0000 7f13fa7e0b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5654ae436cb0 con 0x5654ae57ee70 2024-05-10T11:21:57.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.937+0000 7f13fa7e0b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5654ae58ae50 con 0x5654ae57e550 2024-05-10T11:21:57.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.937+0000 7f13fa7e0b80 10 client.0 _ll_register_callbacks cb 0x5654ae4689a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:21:57.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.937+0000 7f13ee4ea700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654ae582d00 0x5654ae587a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:57.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.937+0000 7f13ee4ea700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654ae582d00 0x5654ae587a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56478/0 (socket says 172.21.15.100:56478) 2024-05-10T11:21:57.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.937+0000 7f13ee4ea700 1 -- 172.21.15.100:0/2050704095 learned_addr learned my addr 172.21.15.100:0/2050704095 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:21:57.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.937+0000 7f13edce9700 1 --2- 172.21.15.100:0/2050704095 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654ae57e550 0x5654ae58a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:57.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.937+0000 7f13ed4e8700 1 --2- 172.21.15.100:0/2050704095 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5654ae57ee70 0x5654ae587530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:57.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.937+0000 7f13ee4ea700 1 -- 172.21.15.100:0/2050704095 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654ae57e550 msgr2=0x5654ae58a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:57.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.937+0000 7f13ee4ea700 1 --2- 172.21.15.100:0/2050704095 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654ae57e550 0x5654ae58a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:57.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.937+0000 7f13ee4ea700 1 -- 172.21.15.100:0/2050704095 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5654ae57ee70 msgr2=0x5654ae587530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:57.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.937+0000 7f13ee4ea700 1 --2- 172.21.15.100:0/2050704095 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5654ae57ee70 0x5654ae587530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:57.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.937+0000 7f13ee4ea700 1 -- 172.21.15.100:0/2050704095 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654ae46c710 con 0x5654ae582d00 2024-05-10T11:21:57.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.938+0000 7f13ee4ea700 1 --2- 172.21.15.100:0/2050704095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654ae582d00 0x5654ae587a70 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f13e00aac60 tx=0x7f13e00ab2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fea9210ca8d10c1d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:57.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.938+0000 7f13d17fa700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:21:57.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.938+0000 7f13d17fa700 1 -- 172.21.15.100:0/2050704095 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e00af080 con 0x5654ae582d00 2024-05-10T11:21:57.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.938+0000 7f13d17fa700 1 -- 172.21.15.100:0/2050704095 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f13e00a72c0 con 0x5654ae582d00 2024-05-10T11:21:57.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.938+0000 7f13fa7e0b80 1 -- 172.21.15.100:0/2050704095 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13e00aacd0 con 0x5654ae582d00 2024-05-10T11:21:57.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.938+0000 7f13d17fa700 1 -- 172.21.15.100:0/2050704095 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f13e00b3aa0 con 0x5654ae582d00 2024-05-10T11:21:57.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.938+0000 7f13fa7e0b80 10 client.16092 fetch_fsmap learned FSMap version 29 2024-05-10T11:21:57.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.938+0000 7f13fa7e0b80 1 -- 172.21.15.100:0/2050704095 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5654ae58acd0 con 0x5654ae582d00 2024-05-10T11:21:57.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.938+0000 7f13d17fa700 1 -- 172.21.15.100:0/2050704095 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f13e00aacd0 con 0x5654ae582d00 2024-05-10T11:21:57.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.939+0000 7f13d17fa700 1 -- 172.21.15.100:0/2050704095 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f13e0011070 con 0x5654ae582d00 2024-05-10T11:21:57.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.939+0000 7f13fa7e0b80 10 client.16092 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T11:21:57.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.939+0000 7f13fa7e0b80 10 client.16092 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:21:57.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.939+0000 7f13fa7e0b80 20 client.16092 populate_metadata read hostname 'smithi100' 2024-05-10T11:21:57.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.939+0000 7f13fa7e0b80 1 --2- 172.21.15.100:0/2050704095 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5654ae597b30 0x5654ae599f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:57.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.939+0000 7f13fa7e0b80 4 client.16092 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:21:57.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.939+0000 7f13fa7e0b80 1 -- 172.21.15.100:0/2050704095 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654ae54a6d0 con 0x5654ae597b30 2024-05-10T11:21:57.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.939+0000 7f13edce9700 1 --2- 172.21.15.100:0/2050704095 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5654ae597b30 0x5654ae599f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:57.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.939+0000 7f13d17fa700 10 client.16092 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:21:57.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.939+0000 7f13edce9700 1 --2- 172.21.15.100:0/2050704095 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5654ae597b30 0x5654ae599f50 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88c05064f0bd4403 server_cookie=404629bd7ba51028 in_seq=0 out_seq=0 2024-05-10T11:21:57.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.940+0000 7f13d17fa700 1 -- 172.21.15.100:0/2050704095 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5654ae54a6d0 con 0x5654ae597b30 2024-05-10T11:21:57.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.940+0000 7f13d17fa700 10 client.16092 handle_command_reply: tid=0 2024-05-10T11:21:57.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.940+0000 7f13fa7e0b80 1 client.16092 shutdown 2024-05-10T11:21:57.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.940+0000 7f13fa7e0b80 1 -- 172.21.15.100:0/2050704095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654ae582d00 msgr2=0x5654ae587a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:57.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.940+0000 7f13fa7e0b80 1 --2- 172.21.15.100:0/2050704095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654ae582d00 0x5654ae587a70 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f13e00aac60 tx=0x7f13e00ab2f0 comp rx=0 tx=0).stop 2024-05-10T11:21:57.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.940+0000 7f13fa7e0b80 1 -- 172.21.15.100:0/2050704095 shutdown_connections 2024-05-10T11:21:57.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.940+0000 7f13fa7e0b80 1 --2- 172.21.15.100:0/2050704095 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5654ae597b30 0x5654ae599f50 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:57.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.940+0000 7f13fa7e0b80 1 --2- 172.21.15.100:0/2050704095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654ae582d00 0x5654ae587a70 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:57.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.940+0000 7f13d17fa700 0 client.16092 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:21:57.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.940+0000 7f13fa7e0b80 1 --2- 172.21.15.100:0/2050704095 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654ae57e550 0x5654ae58a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:57.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.940+0000 7f13fa7e0b80 1 --2- 172.21.15.100:0/2050704095 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5654ae57ee70 0x5654ae587530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:57.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.940+0000 7f13fa7e0b80 1 -- 172.21.15.100:0/2050704095 >> 172.21.15.100:0/2050704095 conn(0x5654ae4aa180 msgr2=0x5654ae578c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:57.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.940+0000 7f13fa7e0b80 1 -- 172.21.15.100:0/2050704095 shutdown_connections 2024-05-10T11:21:57.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.940+0000 7f13fa7e0b80 1 -- 172.21.15.100:0/2050704095 wait complete. 2024-05-10T11:21:57.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.941+0000 7f13fa7e0b80 20 client.16092 trim_cache size 0 max 16384 2024-05-10T11:21:57.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.963+0000 7f13fa7e0b80 1 Processor -- start 2024-05-10T11:21:57.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.963+0000 7f13fa7e0b80 1 -- start start 2024-05-10T11:21:57.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.963+0000 7f13fa7e0b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654ae4aa180 0x5654ae587b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:57.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.963+0000 7f13fa7e0b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5654ae57e550 0x5654ae59f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:57.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.963+0000 7f13fa7e0b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654ae57ee70 0x5654ae59fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:57.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.963+0000 7f13fa7e0b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f13e81b3a60 con 0x5654ae4aa180 2024-05-10T11:21:57.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.963+0000 7f13fa7e0b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f13e81b12f0 con 0x5654ae57e550 2024-05-10T11:21:57.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.963+0000 7f13fa7e0b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f13e81af140 con 0x5654ae57ee70 2024-05-10T11:21:57.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.963+0000 7f13edce9700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654ae4aa180 0x5654ae587b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:57.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.963+0000 7f13ed4e8700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5654ae57e550 0x5654ae59f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:57.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.963+0000 7f13edce9700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654ae4aa180 0x5654ae587b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56488/0 (socket says 172.21.15.100:56488) 2024-05-10T11:21:57.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.963+0000 7f13edce9700 1 -- 172.21.15.100:0/1971257670 learned_addr learned my addr 172.21.15.100:0/1971257670 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:21:57.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.963+0000 7f13ee4ea700 1 --2- 172.21.15.100:0/1971257670 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654ae57ee70 0x5654ae59fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:57.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.963+0000 7f13edce9700 1 -- 172.21.15.100:0/1971257670 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654ae57ee70 msgr2=0x5654ae59fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:57.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.963+0000 7f13edce9700 1 --2- 172.21.15.100:0/1971257670 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654ae57ee70 0x5654ae59fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:57.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.963+0000 7f13edce9700 1 -- 172.21.15.100:0/1971257670 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5654ae57e550 msgr2=0x5654ae59f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:57.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.963+0000 7f13edce9700 1 --2- 172.21.15.100:0/1971257670 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5654ae57e550 0x5654ae59f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:57.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.963+0000 7f13edce9700 1 -- 172.21.15.100:0/1971257670 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654ae587300 con 0x5654ae4aa180 2024-05-10T11:21:57.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.963+0000 7f13ee4ea700 1 --2- 172.21.15.100:0/1971257670 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654ae57ee70 0x5654ae59fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:21:57.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.963+0000 7f13edce9700 1 --2- 172.21.15.100:0/1971257670 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654ae4aa180 0x5654ae587b60 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f13e4005d90 tx=0x7f13e4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=891c32f7903dba3a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:57.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.964+0000 7f13d37fe700 1 -- 172.21.15.100:0/1971257670 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e402f070 con 0x5654ae4aa180 2024-05-10T11:21:57.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.964+0000 7f13d37fe700 1 -- 172.21.15.100:0/1971257670 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f13e40041f0 con 0x5654ae4aa180 2024-05-10T11:21:57.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.964+0000 7f13d37fe700 1 -- 172.21.15.100:0/1971257670 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e401e6e0 con 0x5654ae4aa180 2024-05-10T11:21:57.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.964+0000 7f13fa7e0b80 1 -- 172.21.15.100:0/1971257670 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654ae4aa180 msgr2=0x5654ae587b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:57.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.964+0000 7f13fa7e0b80 1 --2- 172.21.15.100:0/1971257670 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654ae4aa180 0x5654ae587b60 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f13e4005d90 tx=0x7f13e4014000 comp rx=0 tx=0).stop 2024-05-10T11:21:57.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.964+0000 7f13fa7e0b80 1 -- 172.21.15.100:0/1971257670 shutdown_connections 2024-05-10T11:21:57.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.964+0000 7f13fa7e0b80 1 --2- 172.21.15.100:0/1971257670 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654ae57ee70 0x5654ae59fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:57.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.964+0000 7f13fa7e0b80 1 --2- 172.21.15.100:0/1971257670 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654ae4aa180 0x5654ae587b60 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:57.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.964+0000 7f13fa7e0b80 1 --2- 172.21.15.100:0/1971257670 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5654ae57e550 0x5654ae59f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:57.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.964+0000 7f13fa7e0b80 1 -- 172.21.15.100:0/1971257670 >> 172.21.15.100:0/1971257670 conn(0x5654ae573b30 msgr2=0x5654ae57dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:57.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.964+0000 7f13fa7e0b80 1 -- 172.21.15.100:0/1971257670 shutdown_connections 2024-05-10T11:21:57.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.964+0000 7f13fa7e0b80 1 -- 172.21.15.100:0/1971257670 wait complete. 2024-05-10T11:21:57.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.965+0000 7f13fa7e0b80 1 -- start start 2024-05-10T11:21:57.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.965+0000 7f13fa7e0b80 1 Processor -- start 2024-05-10T11:21:57.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.965+0000 7f13fa7e0b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5654ae4aa180 0x5654ae590460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:57.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.965+0000 7f13fa7e0b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654ae57e550 0x5654ae58e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:57.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.965+0000 7f13fa7e0b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654ae57ee70 0x5654ae58ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:57.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.965+0000 7f13fa7e0b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f13e81af7b0 con 0x5654ae57e550 2024-05-10T11:21:57.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.965+0000 7f13fa7e0b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f13e81b0c20 con 0x5654ae4aa180 2024-05-10T11:21:57.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.965+0000 7f13fa7e0b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f13e81afe80 con 0x5654ae57ee70 2024-05-10T11:21:57.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.965+0000 7f13ed4e8700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654ae57e550 0x5654ae58e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:57.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.965+0000 7f13ed4e8700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654ae57e550 0x5654ae58e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56496/0 (socket says 172.21.15.100:56496) 2024-05-10T11:21:57.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.965+0000 7f13fa7e0b80 10 client.0 _ll_register_callbacks cb 0x5654ae4689a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:21:57.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.965+0000 7f13ed4e8700 1 -- 172.21.15.100:0/867322029 learned_addr learned my addr 172.21.15.100:0/867322029 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:21:57.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.965+0000 7f13edce9700 1 --2- 172.21.15.100:0/867322029 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5654ae4aa180 0x5654ae590460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:57.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.965+0000 7f13ee4ea700 1 --2- 172.21.15.100:0/867322029 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654ae57ee70 0x5654ae58ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:57.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.966+0000 7f13ed4e8700 1 -- 172.21.15.100:0/867322029 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654ae57ee70 msgr2=0x5654ae58ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:57.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.966+0000 7f13ed4e8700 1 --2- 172.21.15.100:0/867322029 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654ae57ee70 0x5654ae58ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:57.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.966+0000 7f13ed4e8700 1 -- 172.21.15.100:0/867322029 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5654ae4aa180 msgr2=0x5654ae590460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:21:57.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.966+0000 7f13ed4e8700 1 --2- 172.21.15.100:0/867322029 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5654ae4aa180 0x5654ae590460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:57.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.966+0000 7f13ed4e8700 1 -- 172.21.15.100:0/867322029 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654ae58f450 con 0x5654ae57e550 2024-05-10T11:21:57.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.966+0000 7f13edce9700 1 --2- 172.21.15.100:0/867322029 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5654ae4aa180 0x5654ae590460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:21:57.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.966+0000 7f13ee4ea700 1 --2- 172.21.15.100:0/867322029 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654ae57ee70 0x5654ae58ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:21:57.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.966+0000 7f13ed4e8700 1 --2- 172.21.15.100:0/867322029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654ae57e550 0x5654ae58e810 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f13d8012cb0 tx=0x7f13d8009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e7bacfe19469dfd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:21:57.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.966+0000 7f13d17fa700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:21:57.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.966+0000 7f13d17fa700 1 -- 172.21.15.100:0/867322029 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d8022020 con 0x5654ae57e550 2024-05-10T11:21:57.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.966+0000 7f13d17fa700 1 -- 172.21.15.100:0/867322029 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f13d8004230 con 0x5654ae57e550 2024-05-10T11:21:57.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.966+0000 7f13fa7e0b80 1 -- 172.21.15.100:0/867322029 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13e40041f0 con 0x5654ae57e550 2024-05-10T11:21:57.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.966+0000 7f13d17fa700 1 -- 172.21.15.100:0/867322029 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f13d8004430 con 0x5654ae57e550 2024-05-10T11:21:57.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.966+0000 7f13fa7e0b80 10 client.16098 fetch_fsmap learned FSMap version 29 2024-05-10T11:21:57.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.966+0000 7f13fa7e0b80 1 -- 172.21.15.100:0/867322029 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5654ae603320 con 0x5654ae57e550 2024-05-10T11:21:57.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.967+0000 7f13d17fa700 1 -- 172.21.15.100:0/867322029 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f13e40041f0 con 0x5654ae57e550 2024-05-10T11:21:57.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.967+0000 7f13d17fa700 1 -- 172.21.15.100:0/867322029 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f13d8004cc0 con 0x5654ae57e550 2024-05-10T11:21:57.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.967+0000 7f13fa7e0b80 10 client.16098 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T11:21:57.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.967+0000 7f13fa7e0b80 10 client.16098 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:21:57.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.967+0000 7f13fa7e0b80 20 client.16098 populate_metadata read hostname 'smithi100' 2024-05-10T11:21:57.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.967+0000 7f13fa7e0b80 1 --2- 172.21.15.100:0/867322029 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5654ae582d00 0x5654ae605b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:21:57.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.967+0000 7f13fa7e0b80 4 client.16098 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T11:21:57.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.967+0000 7f13fa7e0b80 1 -- 172.21.15.100:0/867322029 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5654ae4de2c0 con 0x5654ae582d00 2024-05-10T11:21:57.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.967+0000 7f13edce9700 1 --2- 172.21.15.100:0/867322029 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5654ae582d00 0x5654ae605b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:21:57.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.967+0000 7f13d17fa700 10 client.16098 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:21:57.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.967+0000 7f13edce9700 1 --2- 172.21.15.100:0/867322029 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5654ae582d00 0x5654ae605b80 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=843b299d7f67f335 server_cookie=79f91515b7cbde2 in_seq=0 out_seq=0 2024-05-10T11:21:57.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.968+0000 7f13d17fa700 1 -- 172.21.15.100:0/867322029 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5654ae4de2c0 con 0x5654ae582d00 2024-05-10T11:21:57.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.969+0000 7f13d17fa700 10 client.16098 handle_command_reply: tid=0 2024-05-10T11:21:57.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.969+0000 7f13fa7e0b80 1 client.16098 shutdown 2024-05-10T11:21:57.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.969+0000 7f13fa7e0b80 1 -- 172.21.15.100:0/867322029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654ae57e550 msgr2=0x5654ae58e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:57.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.969+0000 7f13fa7e0b80 1 --2- 172.21.15.100:0/867322029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654ae57e550 0x5654ae58e810 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f13d8012cb0 tx=0x7f13d8009ed0 comp rx=0 tx=0).stop 2024-05-10T11:21:57.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.969+0000 7f13fa7e0b80 1 -- 172.21.15.100:0/867322029 shutdown_connections 2024-05-10T11:21:57.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.969+0000 7f13fa7e0b80 1 --2- 172.21.15.100:0/867322029 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5654ae582d00 0x5654ae605b80 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:57.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.969+0000 7f13fa7e0b80 1 --2- 172.21.15.100:0/867322029 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654ae57ee70 0x5654ae58ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:57.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.969+0000 7f13fa7e0b80 1 --2- 172.21.15.100:0/867322029 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5654ae4aa180 0x5654ae590460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:57.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.969+0000 7f13d17fa700 0 client.16098 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:21:57.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.969+0000 7f13fa7e0b80 1 --2- 172.21.15.100:0/867322029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654ae57e550 0x5654ae58e810 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:57.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.969+0000 7f13fa7e0b80 1 -- 172.21.15.100:0/867322029 >> 172.21.15.100:0/867322029 conn(0x5654ae573b30 msgr2=0x5654ae5815a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:57.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.969+0000 7f13fa7e0b80 1 -- 172.21.15.100:0/867322029 shutdown_connections 2024-05-10T11:21:57.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.969+0000 7f13fa7e0b80 1 -- 172.21.15.100:0/867322029 wait complete. 2024-05-10T11:21:57.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.969+0000 7f13fa7e0b80 20 client.16098 trim_cache size 0 max 16384 2024-05-10T11:21:57.975 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:21:57.975 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-05-10T11:21:57.975 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "8cb48f0f-f6c3-4445-8565-b0b128f9e20a", 2024-05-10T11:21:57.975 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-05-10T11:21:57.975 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:21:57.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.970+0000 7f13eff4d700 1 -- 172.21.15.100:0/2473293083 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f13d40b2880 msgr2=0x7f13d40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:57.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.970+0000 7f13eff4d700 1 --2- 172.21.15.100:0/2473293083 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f13d40b2880 0x7f13d40b4d40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f13e4009fd0 tx=0x7f13e4009260 comp rx=0 tx=0).stop 2024-05-10T11:21:57.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.970+0000 7f13eff4d700 1 -- 172.21.15.100:0/2473293083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13e8109530 msgr2=0x7f13e8076220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:21:57.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.970+0000 7f13eff4d700 1 --2- 172.21.15.100:0/2473293083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13e8109530 0x7f13e8076220 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f13e000b780 tx=0x7f13e0004d00 comp rx=0 tx=0).stop 2024-05-10T11:21:57.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.970+0000 7f13eff4d700 1 -- 172.21.15.100:0/2473293083 shutdown_connections 2024-05-10T11:21:57.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.970+0000 7f13eff4d700 1 --2- 172.21.15.100:0/2473293083 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f13d40b2880 0x7f13d40b4d40 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:57.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.970+0000 7f13eff4d700 1 --2- 172.21.15.100:0/2473293083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13e8109530 0x7f13e8076220 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:57.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.970+0000 7f13eff4d700 1 --2- 172.21.15.100:0/2473293083 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13e8103520 0x7f13e80757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:57.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.970+0000 7f13eff4d700 1 --2- 172.21.15.100:0/2473293083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13e8103ee0 0x7f13e8075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:21:57.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.970+0000 7f13eff4d700 1 -- 172.21.15.100:0/2473293083 >> 172.21.15.100:0/2473293083 conn(0x7f13e807a570 msgr2=0x7f13e8102200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:21:57.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.970+0000 7f13eff4d700 1 -- 172.21.15.100:0/2473293083 shutdown_connections 2024-05-10T11:21:57.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:21:57.970+0000 7f13eff4d700 1 -- 172.21.15.100:0/2473293083 wait complete. 2024-05-10T11:21:58.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:21:58 smithi100 ceph-mon[30296]: pgmap v527: 129 pgs: 129 active+clean; 38 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 1.5 MiB/s wr, 163 op/s 2024-05-10T11:21:58.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:21:58 smithi178 ceph-mon[33359]: pgmap v527: 129 pgs: 129 active+clean; 38 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 1.5 MiB/s wr, 163 op/s 2024-05-10T11:21:58.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:21:58 smithi188 ceph-mon[34075]: pgmap v527: 129 pgs: 129 active+clean; 38 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 1.5 MiB/s wr, 163 op/s 2024-05-10T11:21:58.643 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T11:21:58.902 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:21:58.934 INFO:tasks.workunit.client.0.smithi100.stdout: 1 60167 3.02 MB/sec warmup 113 sec latency 23.673 ms 2024-05-10T11:21:59.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:21:59 smithi100 ceph-mon[30296]: scrub queued for path: / 2024-05-10T11:21:59.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:21:59 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:21:59.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:21:59 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:21:59.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:21:59 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:21:59.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:21:59 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:21:59.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:21:59 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:21:59.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:21:59 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:21:59.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:21:59 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:21:59.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:21:59 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:21:59.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:21:59 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:21:59.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:21:59 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:21:59.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:21:59 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:21:59.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:21:59 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:21:59.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:21:59 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:21:59.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:21:59 smithi100 ceph-mon[30296]: pgmap v528: 129 pgs: 129 active+clean; 39 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 1.3 MiB/s wr, 153 op/s 2024-05-10T11:21:59.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:21:59 smithi178 ceph-mon[33359]: scrub queued for path: / 2024-05-10T11:21:59.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:21:59 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:21:59.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:21:59 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:21:59.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:21:59 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:21:59.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:21:59 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:21:59.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:21:59 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:21:59.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:21:59 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:21:59.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:21:59 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:21:59.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:21:59 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:21:59.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:21:59 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:21:59.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:21:59 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:21:59.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:21:59 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:21:59.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:21:59 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:21:59.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:21:59 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:21:59.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:21:59 smithi178 ceph-mon[33359]: pgmap v528: 129 pgs: 129 active+clean; 39 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 1.3 MiB/s wr, 153 op/s 2024-05-10T11:21:59.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:21:59 smithi188 ceph-mon[34075]: scrub queued for path: / 2024-05-10T11:21:59.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:21:59 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:21:59.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:21:59 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:21:59.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:21:59 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:21:59.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:21:59 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:21:59.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:21:59 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:21:59.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:21:59 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:21:59.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:21:59 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:21:59.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:21:59 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:21:59.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:21:59 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:21:59.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:21:59 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:21:59.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:21:59 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:21:59.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:21:59 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:21:59.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:21:59 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:21:59.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:21:59 smithi188 ceph-mon[34075]: pgmap v528: 129 pgs: 129 active+clean; 39 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 1.3 MiB/s wr, 153 op/s 2024-05-10T11:21:59.934 INFO:tasks.workunit.client.0.smithi100.stdout: 1 60653 3.03 MB/sec warmup 114 sec latency 15.471 ms 2024-05-10T11:22:00.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.768+0000 7fbd83247700 1 -- 172.21.15.100:0/1643274877 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd7c109500 msgr2=0x7fbd7c1098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:00.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.768+0000 7fbd83247700 1 --2- 172.21.15.100:0/1643274877 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd7c109500 0x7fbd7c1098e0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fbd70009990 tx=0x7fbd70009ca0 comp rx=0 tx=0).stop 2024-05-10T11:22:00.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.769+0000 7fbd83247700 1 -- 172.21.15.100:0/1643274877 shutdown_connections 2024-05-10T11:22:00.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.769+0000 7fbd83247700 1 --2- 172.21.15.100:0/1643274877 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd7c06c3d0 0x7fbd7c110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:00.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.769+0000 7fbd83247700 1 --2- 172.21.15.100:0/1643274877 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd7c109500 0x7fbd7c1098e0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:00.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.769+0000 7fbd83247700 1 --2- 172.21.15.100:0/1643274877 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbd7c06ba10 0x7fbd7c06be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:00.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.769+0000 7fbd83247700 1 -- 172.21.15.100:0/1643274877 >> 172.21.15.100:0/1643274877 conn(0x7fbd7c079310 msgr2=0x7fbd7c079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:00.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.769+0000 7fbd83247700 1 -- 172.21.15.100:0/1643274877 shutdown_connections 2024-05-10T11:22:00.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.769+0000 7fbd83247700 1 -- 172.21.15.100:0/1643274877 wait complete. 2024-05-10T11:22:00.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.769+0000 7fbd83247700 1 Processor -- start 2024-05-10T11:22:00.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.769+0000 7fbd83247700 1 -- start start 2024-05-10T11:22:00.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.769+0000 7fbd83247700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd7c06ba10 0x7fbd7c102460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:00.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.769+0000 7fbd83247700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbd7c06c3d0 0x7fbd7c1029a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:00.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.769+0000 7fbd83247700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd7c109500 0x7fbd7c102ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:00.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.769+0000 7fbd83247700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbd7c113530 con 0x7fbd7c06ba10 2024-05-10T11:22:00.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.769+0000 7fbd83247700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbd7c1133b0 con 0x7fbd7c06c3d0 2024-05-10T11:22:00.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.769+0000 7fbd83247700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbd7c1136b0 con 0x7fbd7c109500 2024-05-10T11:22:00.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.770+0000 7fbd80fe3700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd7c06ba10 0x7fbd7c102460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:00.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.770+0000 7fbd80fe3700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd7c06ba10 0x7fbd7c102460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56512/0 (socket says 172.21.15.100:56512) 2024-05-10T11:22:00.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.770+0000 7fbd80fe3700 1 -- 172.21.15.100:0/2817743554 learned_addr learned my addr 172.21.15.100:0/2817743554 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:22:00.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.770+0000 7fbd817e4700 1 --2- 172.21.15.100:0/2817743554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd7c109500 0x7fbd7c102ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:00.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.770+0000 7fbd7bfff700 1 --2- 172.21.15.100:0/2817743554 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbd7c06c3d0 0x7fbd7c1029a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:00.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.770+0000 7fbd80fe3700 1 -- 172.21.15.100:0/2817743554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd7c109500 msgr2=0x7fbd7c102ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:00.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.770+0000 7fbd80fe3700 1 --2- 172.21.15.100:0/2817743554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd7c109500 0x7fbd7c102ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:00.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.770+0000 7fbd80fe3700 1 -- 172.21.15.100:0/2817743554 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbd7c06c3d0 msgr2=0x7fbd7c1029a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:00.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.770+0000 7fbd80fe3700 1 --2- 172.21.15.100:0/2817743554 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbd7c06c3d0 0x7fbd7c1029a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:00.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.770+0000 7fbd80fe3700 1 -- 172.21.15.100:0/2817743554 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd7c1a7ff0 con 0x7fbd7c06ba10 2024-05-10T11:22:00.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.770+0000 7fbd80fe3700 1 --2- 172.21.15.100:0/2817743554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd7c06ba10 0x7fbd7c102460 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fbd7000b650 tx=0x7fbd7000bfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:00.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.770+0000 7fbd79ffb700 1 -- 172.21.15.100:0/2817743554 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd70027070 con 0x7fbd7c06ba10 2024-05-10T11:22:00.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.770+0000 7fbd83247700 1 -- 172.21.15.100:0/2817743554 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd7c1a8280 con 0x7fbd7c06ba10 2024-05-10T11:22:00.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.770+0000 7fbd79ffb700 1 -- 172.21.15.100:0/2817743554 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbd70022470 con 0x7fbd7c06ba10 2024-05-10T11:22:00.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.770+0000 7fbd79ffb700 1 -- 172.21.15.100:0/2817743554 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd70017650 con 0x7fbd7c06ba10 2024-05-10T11:22:00.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.771+0000 7fbd83247700 1 -- 172.21.15.100:0/2817743554 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd7c1a8730 con 0x7fbd7c06ba10 2024-05-10T11:22:00.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.773+0000 7fbd79ffb700 1 -- 172.21.15.100:0/2817743554 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fbd7002a070 con 0x7fbd7c06ba10 2024-05-10T11:22:00.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.774+0000 7fbd79ffb700 1 --2- 172.21.15.100:0/2817743554 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fbd6c0b2880 0x7fbd6c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:00.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.774+0000 7fbd7bfff700 1 --2- 172.21.15.100:0/2817743554 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fbd6c0b2880 0x7fbd6c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:00.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.774+0000 7fbd79ffb700 1 -- 172.21.15.100:0/2817743554 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fbd700e34a0 con 0x7fbd7c06ba10 2024-05-10T11:22:00.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.774+0000 7fbd7bfff700 1 --2- 172.21.15.100:0/2817743554 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fbd6c0b2880 0x7fbd6c0b4d40 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c079a50 tx=0x7fbd6800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:00.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.775+0000 7fbd8dadab80 1 Processor -- start 2024-05-10T11:22:00.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.775+0000 7fbd8dadab80 1 -- start start 2024-05-10T11:22:00.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.775+0000 7fbd8dadab80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d7d4151b0 0x563d7d415590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:00.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.775+0000 7fbd8dadab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d7d415ad0 0x563d7d40f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:00.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.775+0000 7fbd8dadab80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d7d40fd50 0x563d7d410190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:00.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.775+0000 7fbd8dadab80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x563d7d361940 con 0x563d7d40fd50 2024-05-10T11:22:00.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.775+0000 7fbd8dadab80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563d7d2c6da0 con 0x563d7d4151b0 2024-05-10T11:22:00.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.775+0000 7fbd8dadab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563d7d35d900 con 0x563d7d415ad0 2024-05-10T11:22:00.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.775+0000 7fbd7bfff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d7d415ad0 0x563d7d40f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:00.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.775+0000 7fbd7bfff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d7d415ad0 0x563d7d40f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:49816/0 (socket says 172.21.15.100:49816) 2024-05-10T11:22:00.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.775+0000 7fbd7bfff700 1 -- 172.21.15.100:0/2981328494 learned_addr learned my addr 172.21.15.100:0/2981328494 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:22:00.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.775+0000 7fbd817e4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d7d40fd50 0x563d7d410190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:00.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.775+0000 7fbd80fe3700 1 --2- 172.21.15.100:0/2981328494 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d7d4151b0 0x563d7d415590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:00.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.775+0000 7fbd7bfff700 1 -- 172.21.15.100:0/2981328494 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d7d4151b0 msgr2=0x563d7d415590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:00.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.775+0000 7fbd7bfff700 1 --2- 172.21.15.100:0/2981328494 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d7d4151b0 0x563d7d415590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:00.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.775+0000 7fbd7bfff700 1 -- 172.21.15.100:0/2981328494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d7d40fd50 msgr2=0x563d7d410190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:00.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.775+0000 7fbd7bfff700 1 --2- 172.21.15.100:0/2981328494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d7d40fd50 0x563d7d410190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:00.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.775+0000 7fbd7bfff700 1 -- 172.21.15.100:0/2981328494 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d7d2ae8d0 con 0x563d7d415ad0 2024-05-10T11:22:00.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.776+0000 7fbd7bfff700 1 --2- 172.21.15.100:0/2981328494 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d7d415ad0 0x563d7d40f810 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbd68010f40 tx=0x7fbd68013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=346358007ac06aae server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:00.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.776+0000 7fbd66ffd700 1 -- 172.21.15.100:0/2981328494 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6800be60 con 0x563d7d415ad0 2024-05-10T11:22:00.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.776+0000 7fbd66ffd700 1 -- 172.21.15.100:0/2981328494 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbd68011db0 con 0x563d7d415ad0 2024-05-10T11:22:00.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.776+0000 7fbd66ffd700 1 -- 172.21.15.100:0/2981328494 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6801c610 con 0x563d7d415ad0 2024-05-10T11:22:00.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.777+0000 7fbd8dadab80 1 -- 172.21.15.100:0/2981328494 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d7d415ad0 msgr2=0x563d7d40f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:00.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.777+0000 7fbd8dadab80 1 --2- 172.21.15.100:0/2981328494 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d7d415ad0 0x563d7d40f810 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbd68010f40 tx=0x7fbd68013000 comp rx=0 tx=0).stop 2024-05-10T11:22:00.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.777+0000 7fbd8dadab80 1 -- 172.21.15.100:0/2981328494 shutdown_connections 2024-05-10T11:22:00.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.777+0000 7fbd8dadab80 1 --2- 172.21.15.100:0/2981328494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d7d40fd50 0x563d7d410190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:00.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.777+0000 7fbd8dadab80 1 --2- 172.21.15.100:0/2981328494 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d7d4151b0 0x563d7d415590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:00.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.777+0000 7fbd8dadab80 1 --2- 172.21.15.100:0/2981328494 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d7d415ad0 0x563d7d40f810 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:00.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.777+0000 7fbd8dadab80 1 -- 172.21.15.100:0/2981328494 >> 172.21.15.100:0/2981328494 conn(0x563d7d33bb70 msgr2=0x563d7d40a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:00.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.777+0000 7fbd8dadab80 1 -- 172.21.15.100:0/2981328494 shutdown_connections 2024-05-10T11:22:00.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.777+0000 7fbd8dadab80 1 -- 172.21.15.100:0/2981328494 wait complete. 2024-05-10T11:22:00.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.777+0000 7fbd8dadab80 1 -- start start 2024-05-10T11:22:00.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.777+0000 7fbd8dadab80 1 Processor -- start 2024-05-10T11:22:00.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.778+0000 7fbd8dadab80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d7d40fd50 0x563d7d41b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:00.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.778+0000 7fbd8dadab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d7d4151b0 0x563d7d418a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:00.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.778+0000 7fbd8dadab80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d7d415ad0 0x563d7d418f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:00.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.778+0000 7fbd8dadab80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x563d7d41c450 con 0x563d7d40fd50 2024-05-10T11:22:00.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.778+0000 7fbd8dadab80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563d7d31aed0 con 0x563d7d415ad0 2024-05-10T11:22:00.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.778+0000 7fbd8dadab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563d7d41c5d0 con 0x563d7d4151b0 2024-05-10T11:22:00.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.778+0000 7fbd8dadab80 10 client.0 _ll_register_callbacks cb 0x563d7d2f55b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:22:00.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.778+0000 7fbd7bfff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d7d4151b0 0x563d7d418a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:00.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.778+0000 7fbd80fe3700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d7d40fd50 0x563d7d41b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:00.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.778+0000 7fbd7bfff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d7d4151b0 0x563d7d418a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:49830/0 (socket says 172.21.15.100:49830) 2024-05-10T11:22:00.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.778+0000 7fbd80fe3700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d7d40fd50 0x563d7d41b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56518/0 (socket says 172.21.15.100:56518) 2024-05-10T11:22:00.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.778+0000 7fbd7bfff700 1 -- 172.21.15.100:0/3698074143 learned_addr learned my addr 172.21.15.100:0/3698074143 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:22:00.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.778+0000 7fbd817e4700 1 --2- 172.21.15.100:0/3698074143 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d7d415ad0 0x563d7d418f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:00.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.778+0000 7fbd7bfff700 1 -- 172.21.15.100:0/3698074143 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d7d415ad0 msgr2=0x563d7d418f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:00.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.778+0000 7fbd7bfff700 1 --2- 172.21.15.100:0/3698074143 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d7d415ad0 0x563d7d418f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:00.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.778+0000 7fbd7bfff700 1 -- 172.21.15.100:0/3698074143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d7d40fd50 msgr2=0x563d7d41b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:00.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.778+0000 7fbd7bfff700 1 --2- 172.21.15.100:0/3698074143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d7d40fd50 0x563d7d41b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:00.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.778+0000 7fbd7bfff700 1 -- 172.21.15.100:0/3698074143 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d7d318760 con 0x563d7d4151b0 2024-05-10T11:22:00.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.778+0000 7fbd817e4700 1 --2- 172.21.15.100:0/3698074143 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d7d415ad0 0x563d7d418f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:22:00.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.778+0000 7fbd7bfff700 1 --2- 172.21.15.100:0/3698074143 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d7d4151b0 0x563d7d418a40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbd680059d0 tx=0x7fbd6801e630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56915bede810bc31 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:00.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.778+0000 7fbd5bfff700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:22:00.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.778+0000 7fbd5bfff700 1 -- 172.21.15.100:0/3698074143 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd68036060 con 0x563d7d4151b0 2024-05-10T11:22:00.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.778+0000 7fbd8dadab80 1 -- 172.21.15.100:0/3698074143 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd68011db0 con 0x563d7d4151b0 2024-05-10T11:22:00.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.778+0000 7fbd5bfff700 1 -- 172.21.15.100:0/3698074143 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fbd68011bb0 con 0x563d7d4151b0 2024-05-10T11:22:00.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.779+0000 7fbd5bfff700 1 -- 172.21.15.100:0/3698074143 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbd6803d050 con 0x563d7d4151b0 2024-05-10T11:22:00.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.779+0000 7fbd8dadab80 10 client.25609 fetch_fsmap learned FSMap version 29 2024-05-10T11:22:00.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.779+0000 7fbd8dadab80 1 -- 172.21.15.100:0/3698074143 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563d7d426c70 con 0x563d7d4151b0 2024-05-10T11:22:00.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.779+0000 7fbd5bfff700 1 -- 172.21.15.100:0/3698074143 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbd68011db0 con 0x563d7d4151b0 2024-05-10T11:22:00.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.779+0000 7fbd5bfff700 1 -- 172.21.15.100:0/3698074143 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fbd68026c20 con 0x563d7d4151b0 2024-05-10T11:22:00.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.779+0000 7fbd8dadab80 10 client.25609 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T11:22:00.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.779+0000 7fbd8dadab80 10 client.25609 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:22:00.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.779+0000 7fbd8dadab80 20 client.25609 populate_metadata read hostname 'smithi100' 2024-05-10T11:22:00.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.779+0000 7fbd8dadab80 1 --2- 172.21.15.100:0/3698074143 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x563d7d429250 0x563d7d42b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:00.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.779+0000 7fbd8dadab80 4 client.25609 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:22:00.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.779+0000 7fbd8dadab80 1 -- 172.21.15.100:0/3698074143 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d7d36f320 con 0x563d7d429250 2024-05-10T11:22:00.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.779+0000 7fbd80fe3700 1 --2- 172.21.15.100:0/3698074143 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x563d7d429250 0x563d7d42b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:00.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.780+0000 7fbd5bfff700 10 client.25609 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:22:00.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.780+0000 7fbd80fe3700 1 --2- 172.21.15.100:0/3698074143 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x563d7d429250 0x563d7d42b640 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf3e646fd571bd72 server_cookie=e2ea0d78dc904939 in_seq=0 out_seq=0 2024-05-10T11:22:00.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.781+0000 7fbd5bfff700 1 -- 172.21.15.100:0/3698074143 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563d7d36f320 con 0x563d7d429250 2024-05-10T11:22:00.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.781+0000 7fbd5bfff700 10 client.25609 handle_command_reply: tid=0 2024-05-10T11:22:00.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.781+0000 7fbd8dadab80 1 client.25609 shutdown 2024-05-10T11:22:00.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.781+0000 7fbd8dadab80 1 -- 172.21.15.100:0/3698074143 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d7d4151b0 msgr2=0x563d7d418a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:00.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.781+0000 7fbd8dadab80 1 --2- 172.21.15.100:0/3698074143 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d7d4151b0 0x563d7d418a40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbd680059d0 tx=0x7fbd6801e630 comp rx=0 tx=0).stop 2024-05-10T11:22:00.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.781+0000 7fbd8dadab80 1 -- 172.21.15.100:0/3698074143 shutdown_connections 2024-05-10T11:22:00.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.781+0000 7fbd8dadab80 1 --2- 172.21.15.100:0/3698074143 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x563d7d429250 0x563d7d42b640 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:00.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.781+0000 7fbd8dadab80 1 --2- 172.21.15.100:0/3698074143 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d7d415ad0 0x563d7d418f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:00.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.781+0000 7fbd8dadab80 1 --2- 172.21.15.100:0/3698074143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d7d40fd50 0x563d7d41b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:00.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.781+0000 7fbd8dadab80 1 --2- 172.21.15.100:0/3698074143 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d7d4151b0 0x563d7d418a40 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:00.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.781+0000 7fbd8dadab80 1 -- 172.21.15.100:0/3698074143 >> 172.21.15.100:0/3698074143 conn(0x563d7d33bb70 msgr2=0x563d7d404b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:00.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.781+0000 7fbd5bfff700 0 client.25609 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:22:00.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.781+0000 7fbd8dadab80 1 -- 172.21.15.100:0/3698074143 shutdown_connections 2024-05-10T11:22:00.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.781+0000 7fbd8dadab80 1 -- 172.21.15.100:0/3698074143 wait complete. 2024-05-10T11:22:00.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.781+0000 7fbd8dadab80 20 client.25609 trim_cache size 0 max 16384 2024-05-10T11:22:00.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.805+0000 7fbd8dadab80 1 Processor -- start 2024-05-10T11:22:00.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.805+0000 7fbd8dadab80 1 -- start start 2024-05-10T11:22:00.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.805+0000 7fbd8dadab80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d7d40fd50 0x563d7d40f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:00.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.805+0000 7fbd8dadab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d7d4151b0 0x563d7d4254e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:00.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.805+0000 7fbd8dadab80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d7d415ad0 0x563d7d417bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:00.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.805+0000 7fbd8dadab80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbd7c1b39d0 con 0x563d7d40fd50 2024-05-10T11:22:00.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.805+0000 7fbd8dadab80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbd7c1b1260 con 0x563d7d415ad0 2024-05-10T11:22:00.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.805+0000 7fbd8dadab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbd7c1af0b0 con 0x563d7d4151b0 2024-05-10T11:22:00.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.805+0000 7fbd80fe3700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d7d40fd50 0x563d7d40f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:00.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.805+0000 7fbd80fe3700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d7d40fd50 0x563d7d40f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56530/0 (socket says 172.21.15.100:56530) 2024-05-10T11:22:00.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.805+0000 7fbd80fe3700 1 -- 172.21.15.100:0/989640724 learned_addr learned my addr 172.21.15.100:0/989640724 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:22:00.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.805+0000 7fbd817e4700 1 --2- 172.21.15.100:0/989640724 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d7d415ad0 0x563d7d417bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:00.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.805+0000 7fbd7bfff700 1 --2- 172.21.15.100:0/989640724 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d7d4151b0 0x563d7d4254e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:00.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.805+0000 7fbd80fe3700 1 -- 172.21.15.100:0/989640724 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d7d4151b0 msgr2=0x563d7d4254e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:00.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.805+0000 7fbd80fe3700 1 --2- 172.21.15.100:0/989640724 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d7d4151b0 0x563d7d4254e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:00.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.805+0000 7fbd80fe3700 1 -- 172.21.15.100:0/989640724 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d7d415ad0 msgr2=0x563d7d417bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:00.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.805+0000 7fbd80fe3700 1 --2- 172.21.15.100:0/989640724 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d7d415ad0 0x563d7d417bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:00.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.805+0000 7fbd80fe3700 1 -- 172.21.15.100:0/989640724 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d7d406c80 con 0x563d7d40fd50 2024-05-10T11:22:00.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.805+0000 7fbd817e4700 1 --2- 172.21.15.100:0/989640724 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d7d415ad0 0x563d7d417bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:22:00.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.806+0000 7fbd80fe3700 1 --2- 172.21.15.100:0/989640724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d7d40fd50 0x563d7d40f810 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fbd700e0c90 tx=0x7fbd700b1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e68dc85f329b82c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:00.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.806+0000 7fbd667fc700 1 -- 172.21.15.100:0/989640724 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd700e3040 con 0x563d7d40fd50 2024-05-10T11:22:00.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.806+0000 7fbd667fc700 1 -- 172.21.15.100:0/989640724 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbd700aeba0 con 0x563d7d40fd50 2024-05-10T11:22:00.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.806+0000 7fbd667fc700 1 -- 172.21.15.100:0/989640724 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd700e3300 con 0x563d7d40fd50 2024-05-10T11:22:00.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.806+0000 7fbd8dadab80 1 -- 172.21.15.100:0/989640724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d7d40fd50 msgr2=0x563d7d40f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:00.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.806+0000 7fbd8dadab80 1 --2- 172.21.15.100:0/989640724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d7d40fd50 0x563d7d40f810 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fbd700e0c90 tx=0x7fbd700b1040 comp rx=0 tx=0).stop 2024-05-10T11:22:00.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.807+0000 7fbd8dadab80 1 -- 172.21.15.100:0/989640724 shutdown_connections 2024-05-10T11:22:00.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.807+0000 7fbd8dadab80 1 --2- 172.21.15.100:0/989640724 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d7d415ad0 0x563d7d417bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:00.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.807+0000 7fbd8dadab80 1 --2- 172.21.15.100:0/989640724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d7d40fd50 0x563d7d40f810 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:00.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.807+0000 7fbd8dadab80 1 --2- 172.21.15.100:0/989640724 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d7d4151b0 0x563d7d4254e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:00.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.807+0000 7fbd8dadab80 1 -- 172.21.15.100:0/989640724 >> 172.21.15.100:0/989640724 conn(0x563d7d33bb70 msgr2=0x563d7d416520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:00.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.807+0000 7fbd8dadab80 1 -- 172.21.15.100:0/989640724 shutdown_connections 2024-05-10T11:22:00.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.807+0000 7fbd8dadab80 1 -- 172.21.15.100:0/989640724 wait complete. 2024-05-10T11:22:00.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.807+0000 7fbd8dadab80 1 -- start start 2024-05-10T11:22:00.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.807+0000 7fbd8dadab80 1 Processor -- start 2024-05-10T11:22:00.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.807+0000 7fbd8dadab80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d7d40fd50 0x563d7d430440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:00.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.807+0000 7fbd8dadab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d7d4151b0 0x563d7d430980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:00.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.807+0000 7fbd8dadab80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d7d415ad0 0x563d7d430ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:00.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.807+0000 7fbd8dadab80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbd7c1af720 con 0x563d7d40fd50 2024-05-10T11:22:00.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.808+0000 7fbd8dadab80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbd7c1b0b90 con 0x563d7d415ad0 2024-05-10T11:22:00.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.808+0000 7fbd8dadab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbd7c1afdf0 con 0x563d7d4151b0 2024-05-10T11:22:00.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.808+0000 7fbd8dadab80 10 client.0 _ll_register_callbacks cb 0x563d7d2f55b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:22:00.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.808+0000 7fbd80fe3700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d7d40fd50 0x563d7d430440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:00.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.808+0000 7fbd80fe3700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d7d40fd50 0x563d7d430440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56534/0 (socket says 172.21.15.100:56534) 2024-05-10T11:22:00.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.808+0000 7fbd7bfff700 1 --2- 172.21.15.100:0/1249221354 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d7d4151b0 0x563d7d430980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:00.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.808+0000 7fbd80fe3700 1 -- 172.21.15.100:0/1249221354 learned_addr learned my addr 172.21.15.100:0/1249221354 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:22:00.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.808+0000 7fbd817e4700 1 --2- 172.21.15.100:0/1249221354 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d7d415ad0 0x563d7d430ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:00.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.808+0000 7fbd7bfff700 1 -- 172.21.15.100:0/1249221354 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d7d415ad0 msgr2=0x563d7d430ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:00.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.808+0000 7fbd7bfff700 1 --2- 172.21.15.100:0/1249221354 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d7d415ad0 0x563d7d430ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:00.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.808+0000 7fbd7bfff700 1 -- 172.21.15.100:0/1249221354 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d7d40fd50 msgr2=0x563d7d430440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:00.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.808+0000 7fbd7bfff700 1 --2- 172.21.15.100:0/1249221354 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d7d40fd50 0x563d7d430440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:00.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.808+0000 7fbd7bfff700 1 -- 172.21.15.100:0/1249221354 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d7d4927f0 con 0x563d7d4151b0 2024-05-10T11:22:00.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.808+0000 7fbd80fe3700 1 --2- 172.21.15.100:0/1249221354 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d7d40fd50 0x563d7d430440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:22:00.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.808+0000 7fbd7bfff700 1 --2- 172.21.15.100:0/1249221354 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d7d4151b0 0x563d7d430980 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbd6801a420 tx=0x7fbd6800bb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a46ea25ea51400ac server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:00.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.808+0000 7fbd5bfff700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:22:00.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.808+0000 7fbd5bfff700 1 -- 172.21.15.100:0/1249221354 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd68036040 con 0x563d7d4151b0 2024-05-10T11:22:00.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.808+0000 7fbd5bfff700 1 -- 172.21.15.100:0/1249221354 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fbd6801a610 con 0x563d7d4151b0 2024-05-10T11:22:00.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.808+0000 7fbd8dadab80 1 -- 172.21.15.100:0/1249221354 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd700aeba0 con 0x563d7d4151b0 2024-05-10T11:22:00.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.809+0000 7fbd5bfff700 1 -- 172.21.15.100:0/1249221354 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbd68011a00 con 0x563d7d4151b0 2024-05-10T11:22:00.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.809+0000 7fbd8dadab80 10 client.25615 fetch_fsmap learned FSMap version 29 2024-05-10T11:22:00.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.809+0000 7fbd8dadab80 1 -- 172.21.15.100:0/1249221354 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563d7d493ae0 con 0x563d7d4151b0 2024-05-10T11:22:00.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.809+0000 7fbd5bfff700 1 -- 172.21.15.100:0/1249221354 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbd700aeba0 con 0x563d7d4151b0 2024-05-10T11:22:00.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.809+0000 7fbd5bfff700 1 -- 172.21.15.100:0/1249221354 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fbd68031470 con 0x563d7d4151b0 2024-05-10T11:22:00.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.809+0000 7fbd8dadab80 10 client.25615 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T11:22:00.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.809+0000 7fbd8dadab80 10 client.25615 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:22:00.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.809+0000 7fbd8dadab80 20 client.25615 populate_metadata read hostname 'smithi100' 2024-05-10T11:22:00.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.809+0000 7fbd8dadab80 1 --2- 172.21.15.100:0/1249221354 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x563d7d4941c0 0x563d7d4965e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:00.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.809+0000 7fbd8dadab80 4 client.25615 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T11:22:00.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.809+0000 7fbd8dadab80 1 -- 172.21.15.100:0/1249221354 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563d7d3693c0 con 0x563d7d4941c0 2024-05-10T11:22:00.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.809+0000 7fbd80fe3700 1 --2- 172.21.15.100:0/1249221354 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x563d7d4941c0 0x563d7d4965e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:00.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.810+0000 7fbd80fe3700 1 --2- 172.21.15.100:0/1249221354 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x563d7d4941c0 0x563d7d4965e0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed9418bef57caa0b server_cookie=e9fd3ef13a6d46d0 in_seq=0 out_seq=0 2024-05-10T11:22:00.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.810+0000 7fbd5bfff700 10 client.25615 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:22:00.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.810+0000 7fbd5bfff700 1 -- 172.21.15.100:0/1249221354 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x563d7d3693c0 con 0x563d7d4941c0 2024-05-10T11:22:00.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.810+0000 7fbd5bfff700 10 client.25615 handle_command_reply: tid=0 2024-05-10T11:22:00.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.810+0000 7fbd8dadab80 1 client.25615 shutdown 2024-05-10T11:22:00.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.810+0000 7fbd8dadab80 1 -- 172.21.15.100:0/1249221354 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d7d4151b0 msgr2=0x563d7d430980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:00.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.810+0000 7fbd8dadab80 1 --2- 172.21.15.100:0/1249221354 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d7d4151b0 0x563d7d430980 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbd6801a420 tx=0x7fbd6800bb20 comp rx=0 tx=0).stop 2024-05-10T11:22:00.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.810+0000 7fbd8dadab80 1 -- 172.21.15.100:0/1249221354 shutdown_connections 2024-05-10T11:22:00.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.810+0000 7fbd8dadab80 1 --2- 172.21.15.100:0/1249221354 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x563d7d4941c0 0x563d7d4965e0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:00.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.810+0000 7fbd8dadab80 1 --2- 172.21.15.100:0/1249221354 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d7d415ad0 0x563d7d430ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:00.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.811+0000 7fbd8dadab80 1 --2- 172.21.15.100:0/1249221354 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d7d40fd50 0x563d7d430440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:00.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.811+0000 7fbd5bfff700 0 client.25615 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:22:00.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.811+0000 7fbd8dadab80 1 --2- 172.21.15.100:0/1249221354 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d7d4151b0 0x563d7d430980 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:00.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.811+0000 7fbd8dadab80 1 -- 172.21.15.100:0/1249221354 >> 172.21.15.100:0/1249221354 conn(0x563d7d33bb70 msgr2=0x563d7d413ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:00.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.811+0000 7fbd8dadab80 1 -- 172.21.15.100:0/1249221354 shutdown_connections 2024-05-10T11:22:00.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.811+0000 7fbd8dadab80 1 -- 172.21.15.100:0/1249221354 wait complete. 2024-05-10T11:22:00.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.811+0000 7fbd8dadab80 20 client.25615 trim_cache size 0 max 16384 2024-05-10T11:22:00.816 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:22:00.816 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T11:22:00.816 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-05-10T11:22:00.816 INFO:teuthology.orchestra.run.smithi100.stdout: "8cb48f0f-f6c3-4445-8565-b0b128f9e20a": { 2024-05-10T11:22:00.816 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-05-10T11:22:00.816 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "8cb48f0f-f6c3-4445-8565-b0b128f9e20a", 2024-05-10T11:22:00.816 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-05-10T11:22:00.816 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:22:00.817 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:22:00.817 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:22:00.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.811+0000 7fbd83247700 1 -- 172.21.15.100:0/2817743554 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fbd6c0b2880 msgr2=0x7fbd6c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:00.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.811+0000 7fbd83247700 1 --2- 172.21.15.100:0/2817743554 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fbd6c0b2880 0x7fbd6c0b4d40 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c079a50 tx=0x7fbd6800b410 comp rx=0 tx=0).stop 2024-05-10T11:22:00.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.811+0000 7fbd83247700 1 -- 172.21.15.100:0/2817743554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd7c06ba10 msgr2=0x7fbd7c102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:00.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.811+0000 7fbd83247700 1 --2- 172.21.15.100:0/2817743554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd7c06ba10 0x7fbd7c102460 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fbd7000b650 tx=0x7fbd7000bfc0 comp rx=0 tx=0).stop 2024-05-10T11:22:00.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.811+0000 7fbd83247700 1 -- 172.21.15.100:0/2817743554 shutdown_connections 2024-05-10T11:22:00.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.811+0000 7fbd83247700 1 --2- 172.21.15.100:0/2817743554 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fbd6c0b2880 0x7fbd6c0b4d40 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:00.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.811+0000 7fbd83247700 1 --2- 172.21.15.100:0/2817743554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd7c109500 0x7fbd7c102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:00.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.811+0000 7fbd83247700 1 --2- 172.21.15.100:0/2817743554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd7c06ba10 0x7fbd7c102460 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:00.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.811+0000 7fbd83247700 1 --2- 172.21.15.100:0/2817743554 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbd7c06c3d0 0x7fbd7c1029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:00.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.811+0000 7fbd83247700 1 -- 172.21.15.100:0/2817743554 >> 172.21.15.100:0/2817743554 conn(0x7fbd7c079310 msgr2=0x7fbd7c101230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:00.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.812+0000 7fbd83247700 1 -- 172.21.15.100:0/2817743554 shutdown_connections 2024-05-10T11:22:00.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:00.812+0000 7fbd83247700 1 -- 172.21.15.100:0/2817743554 wait complete. 2024-05-10T11:22:00.934 INFO:tasks.workunit.client.0.smithi100.stdout: 1 60955 3.01 MB/sec warmup 115 sec latency 12.859 ms 2024-05-10T11:22:01.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:22:01 smithi100 ceph-mon[30296]: pgmap v529: 129 pgs: 129 active+clean; 39 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 1.2 MiB/s wr, 134 op/s 2024-05-10T11:22:01.649 INFO:tasks.cephfs.filesystem:scrub status for tag:8cb48f0f-f6c3-4445-8565-b0b128f9e20a - {'path': '/', 'tag': '8cb48f0f-f6c3-4445-8565-b0b128f9e20a', 'options': 'recursive,force'} 2024-05-10T11:22:01.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:22:01 smithi178 ceph-mon[33359]: pgmap v529: 129 pgs: 129 active+clean; 39 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 1.2 MiB/s wr, 134 op/s 2024-05-10T11:22:01.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:22:01 smithi188 ceph-mon[34075]: pgmap v529: 129 pgs: 129 active+clean; 39 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 1.2 MiB/s wr, 134 op/s 2024-05-10T11:22:01.934 INFO:tasks.workunit.client.0.smithi100.stdout: 1 61508 3.00 MB/sec warmup 116 sec latency 11.613 ms 2024-05-10T11:22:02.935 INFO:tasks.workunit.client.0.smithi100.stdout: 1 62173 3.00 MB/sec warmup 117 sec latency 18.978 ms 2024-05-10T11:22:03.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:22:03 smithi178 ceph-mon[33359]: pgmap v530: 129 pgs: 129 active+clean; 40 MiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.4 MiB/s wr, 166 op/s 2024-05-10T11:22:03.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:22:03 smithi188 ceph-mon[34075]: pgmap v530: 129 pgs: 129 active+clean; 40 MiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.4 MiB/s wr, 166 op/s 2024-05-10T11:22:03.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:22:03 smithi100 ceph-mon[30296]: pgmap v530: 129 pgs: 129 active+clean; 40 MiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.4 MiB/s wr, 166 op/s 2024-05-10T11:22:03.934 INFO:tasks.workunit.client.0.smithi100.stdout: 1 62783 2.99 MB/sec warmup 118 sec latency 14.130 ms 2024-05-10T11:22:04.934 INFO:tasks.workunit.client.0.smithi100.stdout: 1 63607 3.03 MB/sec warmup 119 sec latency 15.445 ms 2024-05-10T11:22:05.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:22:05 smithi178 ceph-mon[33359]: pgmap v531: 129 pgs: 129 active+clean; 39 MiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 1.4 MiB/s wr, 159 op/s 2024-05-10T11:22:05.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:22:05 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:22:05.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:22:05 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:22:05.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:22:05 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:22:05.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:22:05 smithi188 ceph-mon[34075]: pgmap v531: 129 pgs: 129 active+clean; 39 MiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 1.4 MiB/s wr, 159 op/s 2024-05-10T11:22:05.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:22:05 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:22:05.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:22:05 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:22:05.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:22:05 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:22:05.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:22:05 smithi100 ceph-mon[30296]: pgmap v531: 129 pgs: 129 active+clean; 39 MiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 1.4 MiB/s wr, 159 op/s 2024-05-10T11:22:05.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:22:05 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:22:05.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:22:05 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:22:05.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:22:05 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:22:05.973 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.971+0000 7f58ac475640 1 -- 172.21.15.100:0/515290965 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58a4108790 msgr2=0x7f58a410ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:05.973 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.971+0000 7f58ac475640 1 --2- 172.21.15.100:0/515290965 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58a4108790 0x7f58a410ab80 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f589c009ca0 tx=0x7f589c02a8c0 comp rx=0 tx=0).stop 2024-05-10T11:22:05.973 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.971+0000 7f58ac475640 1 -- 172.21.15.100:0/515290965 shutdown_connections 2024-05-10T11:22:05.973 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.971+0000 7f58ac475640 1 --2- 172.21.15.100:0/515290965 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58a4108790 0x7f58a410ab80 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:05.973 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.971+0000 7f58ac475640 1 --2- 172.21.15.100:0/515290965 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58a4105e60 0x7f58a4108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:05.973 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.971+0000 7f58ac475640 1 --2- 172.21.15.100:0/515290965 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58a4103530 0x7f58a4105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.971+0000 7f58ac475640 1 -- 172.21.15.100:0/515290965 >> 172.21.15.100:0/515290965 conn(0x7f58a40fcec0 msgr2=0x7f58a40ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.971+0000 7f58ac475640 1 -- 172.21.15.100:0/515290965 shutdown_connections 2024-05-10T11:22:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.971+0000 7f58ac475640 1 -- 172.21.15.100:0/515290965 wait complete. 2024-05-10T11:22:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.972+0000 7f58ac475640 1 Processor -- start 2024-05-10T11:22:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.972+0000 7f58ac475640 1 -- start start 2024-05-10T11:22:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.972+0000 7f58ac475640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58a4103530 0x7f58a419ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.972+0000 7f58ac475640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58a4105e60 0x7f58a419b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.972+0000 7f58ac475640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58a4108790 0x7f58a41a2230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.972+0000 7f58ac475640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f58a4102530 con 0x7f58a4103530 2024-05-10T11:22:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.972+0000 7f58ac475640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f58a41023b0 con 0x7f58a4105e60 2024-05-10T11:22:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.972+0000 7f58ac475640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f58a41026b0 con 0x7f58a4108790 2024-05-10T11:22:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.972+0000 7f58aa211640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58a4103530 0x7f58a419ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.972+0000 7f58aaa12640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58a4108790 0x7f58a41a2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.972+0000 7f58aa211640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58a4103530 0x7f58a419ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56562/0 (socket says 172.21.15.100:56562) 2024-05-10T11:22:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.972+0000 7f58aa211640 1 -- 172.21.15.100:0/3935407006 learned_addr learned my addr 172.21.15.100:0/3935407006 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:22:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.972+0000 7f58a9a10640 1 --2- 172.21.15.100:0/3935407006 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58a4105e60 0x7f58a419b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.972+0000 7f58aa211640 1 -- 172.21.15.100:0/3935407006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58a4108790 msgr2=0x7f58a41a2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.972+0000 7f58aa211640 1 --2- 172.21.15.100:0/3935407006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58a4108790 0x7f58a41a2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.972+0000 7f58aa211640 1 -- 172.21.15.100:0/3935407006 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58a4105e60 msgr2=0x7f58a419b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.972+0000 7f58aa211640 1 --2- 172.21.15.100:0/3935407006 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58a4105e60 0x7f58a419b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.972+0000 7f58aa211640 1 -- 172.21.15.100:0/3935407006 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58a41a28d0 con 0x7f58a4103530 2024-05-10T11:22:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.972+0000 7f58aaa12640 1 --2- 172.21.15.100:0/3935407006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58a4108790 0x7f58a41a2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:22:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.972+0000 7f58a9a10640 1 --2- 172.21.15.100:0/3935407006 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58a4105e60 0x7f58a419b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:22:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.973+0000 7f58aa211640 1 --2- 172.21.15.100:0/3935407006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58a4103530 0x7f58a419ace0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f58a000e9f0 tx=0x7f58a000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.973+0000 7f589b7fe640 1 -- 172.21.15.100:0/3935407006 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a000cd80 con 0x7f58a4103530 2024-05-10T11:22:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.973+0000 7f58ac475640 1 -- 172.21.15.100:0/3935407006 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58a41a2bc0 con 0x7f58a4103530 2024-05-10T11:22:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.973+0000 7f589b7fe640 1 -- 172.21.15.100:0/3935407006 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f58a00054b0 con 0x7f58a4103530 2024-05-10T11:22:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.973+0000 7f589b7fe640 1 -- 172.21.15.100:0/3935407006 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a0010430 con 0x7f58a4103530 2024-05-10T11:22:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.973+0000 7f58ac475640 1 -- 172.21.15.100:0/3935407006 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58a41a3110 con 0x7f58a4103530 2024-05-10T11:22:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.976+0000 7f589b7fe640 1 -- 172.21.15.100:0/3935407006 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f58a00046c0 con 0x7f58a4103530 2024-05-10T11:22:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.976+0000 7f58af19a740 1 Processor -- start 2024-05-10T11:22:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.976+0000 7f58af19a740 1 -- start start 2024-05-10T11:22:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.976+0000 7f58af19a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c78754db0 0x556c787551d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.976+0000 7f58af19a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c787557a0 0x556c7875a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.976+0000 7f58af19a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556c7875a910 0x556c7875cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.976+0000 7f58af19a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x556c786b95d0 con 0x556c78754db0 2024-05-10T11:22:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.976+0000 7f58af19a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556c78672890 con 0x556c787557a0 2024-05-10T11:22:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.976+0000 7f58af19a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556c786b5590 con 0x556c7875a910 2024-05-10T11:22:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.977+0000 7f58aaa12640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c78754db0 0x556c787551d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.977+0000 7f58aa211640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c787557a0 0x556c7875a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.977+0000 7f58aaa12640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c78754db0 0x556c787551d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56570/0 (socket says 172.21.15.100:56570) 2024-05-10T11:22:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.977+0000 7f58a9a10640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556c7875a910 0x556c7875cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.977+0000 7f58aaa12640 1 -- 172.21.15.100:0/3012201082 learned_addr learned my addr 172.21.15.100:0/3012201082 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:22:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.977+0000 7f58aaa12640 1 -- 172.21.15.100:0/3012201082 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556c7875a910 msgr2=0x556c7875cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.977+0000 7f58aaa12640 1 --2- 172.21.15.100:0/3012201082 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556c7875a910 0x556c7875cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.977+0000 7f58aaa12640 1 -- 172.21.15.100:0/3012201082 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c787557a0 msgr2=0x556c7875a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.977+0000 7f58aaa12640 1 --2- 172.21.15.100:0/3012201082 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c787557a0 0x556c7875a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.977+0000 7f589b7fe640 1 --2- 172.21.15.100:0/3935407006 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f58900b2920 0x7f58900b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.977+0000 7f58aaa12640 1 -- 172.21.15.100:0/3012201082 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c783cb210 con 0x556c78754db0 2024-05-10T11:22:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.977+0000 7f58a9a10640 1 --2- 172.21.15.100:0/3012201082 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556c7875a910 0x556c7875cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:22:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.977+0000 7f58a9a10640 1 --2- 172.21.15.100:0/3935407006 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f58900b2920 0x7f58900b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.977+0000 7f589b7fe640 1 -- 172.21.15.100:0/3935407006 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f58a00e2830 con 0x7f58a4103530 2024-05-10T11:22:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.977+0000 7f58aaa12640 1 --2- 172.21.15.100:0/3012201082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c78754db0 0x556c787551d0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f589c009ca0 tx=0x7f589c009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=842bc7f19bfab36d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.978+0000 7f5898bfe640 1 -- 172.21.15.100:0/3012201082 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589c031d40 con 0x556c78754db0 2024-05-10T11:22:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.978+0000 7f5898bfe640 1 -- 172.21.15.100:0/3012201082 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f589c036030 con 0x556c78754db0 2024-05-10T11:22:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.978+0000 7f5898bfe640 1 -- 172.21.15.100:0/3012201082 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589c004750 con 0x556c78754db0 2024-05-10T11:22:05.980 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.978+0000 7f58af19a740 1 -- 172.21.15.100:0/3012201082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c78754db0 msgr2=0x556c787551d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:05.981 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.978+0000 7f58a9a10640 1 --2- 172.21.15.100:0/3935407006 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f58900b2920 0x7f58900b4de0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5894009fd0 tx=0x7f5894005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:05.981 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.978+0000 7f58af19a740 1 --2- 172.21.15.100:0/3012201082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c78754db0 0x556c787551d0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f589c009ca0 tx=0x7f589c009a20 comp rx=0 tx=0).stop 2024-05-10T11:22:05.981 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.979+0000 7f58af19a740 1 -- 172.21.15.100:0/3012201082 shutdown_connections 2024-05-10T11:22:05.981 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.979+0000 7f58af19a740 1 --2- 172.21.15.100:0/3012201082 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556c7875a910 0x556c7875cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:05.981 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.979+0000 7f58af19a740 1 --2- 172.21.15.100:0/3012201082 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c787557a0 0x556c7875a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:05.981 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.979+0000 7f58af19a740 1 --2- 172.21.15.100:0/3012201082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c78754db0 0x556c787551d0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:05.981 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.979+0000 7f58af19a740 1 -- 172.21.15.100:0/3012201082 >> 172.21.15.100:0/3012201082 conn(0x556c78750420 msgr2=0x556c78750820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:05.981 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.979+0000 7f58af19a740 1 -- 172.21.15.100:0/3012201082 shutdown_connections 2024-05-10T11:22:05.981 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.979+0000 7f58af19a740 1 -- 172.21.15.100:0/3012201082 wait complete. 2024-05-10T11:22:05.981 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.979+0000 7f58af19a740 1 -- start start 2024-05-10T11:22:05.981 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.980+0000 7f58af19a740 1 Processor -- start 2024-05-10T11:22:05.982 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.981+0000 7f58af19a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556c78754db0 0x556c7875ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:05.982 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.981+0000 7f58af19a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c787557a0 0x556c7875f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:05.982 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.981+0000 7f58af19a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c7875a910 0x556c78766300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.981+0000 7f58af19a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x556c7875d6e0 con 0x556c7875a910 2024-05-10T11:22:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.981+0000 7f58af19a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556c78676370 con 0x556c787557a0 2024-05-10T11:22:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.981+0000 7f58af19a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556c7875d860 con 0x556c78754db0 2024-05-10T11:22:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.981+0000 7f58aa211640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556c78754db0 0x556c7875ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.981+0000 7f58af19a740 10 client.0 _ll_register_callbacks cb 0x556c783caa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:22:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.981+0000 7f58aa211640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556c78754db0 0x556c7875ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:49902/0 (socket says 172.21.15.100:49902) 2024-05-10T11:22:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.982+0000 7f58aa211640 1 -- 172.21.15.100:0/1603650323 learned_addr learned my addr 172.21.15.100:0/1603650323 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:22:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.982+0000 7f58aaa12640 1 --2- 172.21.15.100:0/1603650323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c7875a910 0x556c78766300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.982+0000 7f58a9a10640 1 --2- 172.21.15.100:0/1603650323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c787557a0 0x556c7875f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.982+0000 7f58aaa12640 1 -- 172.21.15.100:0/1603650323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556c78754db0 msgr2=0x556c7875ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.982+0000 7f58aaa12640 1 --2- 172.21.15.100:0/1603650323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556c78754db0 0x556c7875ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.982+0000 7f58aaa12640 1 -- 172.21.15.100:0/1603650323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c787557a0 msgr2=0x556c7875f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.982+0000 7f58aaa12640 1 --2- 172.21.15.100:0/1603650323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c787557a0 0x556c7875f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.982+0000 7f58aaa12640 1 -- 172.21.15.100:0/1603650323 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c78497620 con 0x556c7875a910 2024-05-10T11:22:05.984 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.982+0000 7f58a9a10640 1 --2- 172.21.15.100:0/1603650323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c787557a0 0x556c7875f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:22:05.985 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.983+0000 7f58aaa12640 1 --2- 172.21.15.100:0/1603650323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c7875a910 0x556c78766300 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f589c005e30 tx=0x7f589c004780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2329811b6710ca2b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:05.985 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.983+0000 7f588dffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:22:05.985 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.983+0000 7f588dffb640 1 -- 172.21.15.100:0/1603650323 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589c04a040 con 0x556c7875a910 2024-05-10T11:22:05.985 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.983+0000 7f588dffb640 1 -- 172.21.15.100:0/1603650323 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f589c003970 con 0x556c7875a910 2024-05-10T11:22:05.985 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.983+0000 7f58af19a740 1 -- 172.21.15.100:0/1603650323 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f589c036030 con 0x556c7875a910 2024-05-10T11:22:05.986 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.984+0000 7f588dffb640 1 -- 172.21.15.100:0/1603650323 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f589c003b70 con 0x556c7875a910 2024-05-10T11:22:05.986 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.984+0000 7f58af19a740 10 client.16128 fetch_fsmap learned FSMap version 29 2024-05-10T11:22:05.986 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.984+0000 7f58af19a740 1 -- 172.21.15.100:0/1603650323 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556c7875d6e0 con 0x556c7875a910 2024-05-10T11:22:05.986 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.984+0000 7f588dffb640 1 -- 172.21.15.100:0/1603650323 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f589c036030 con 0x556c7875a910 2024-05-10T11:22:05.986 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.984+0000 7f588dffb640 1 -- 172.21.15.100:0/1603650323 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f589c04d030 con 0x556c7875a910 2024-05-10T11:22:05.986 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.984+0000 7f58af19a740 10 client.16128 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T11:22:05.986 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.984+0000 7f58af19a740 10 client.16128 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:22:05.986 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.984+0000 7f58af19a740 20 client.16128 populate_metadata read hostname 'smithi100' 2024-05-10T11:22:05.986 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.984+0000 7f58af19a740 1 --2- 172.21.15.100:0/1603650323 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556c7876a7f0 0x556c7876cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:05.986 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.984+0000 7f58af19a740 4 client.16128 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:22:05.986 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.984+0000 7f58af19a740 1 -- 172.21.15.100:0/1603650323 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c786c71e0 con 0x556c7876a7f0 2024-05-10T11:22:05.986 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.985+0000 7f58aa211640 1 --2- 172.21.15.100:0/1603650323 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556c7876a7f0 0x556c7876cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:05.986 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.986+0000 7f588dffb640 10 client.16128 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:22:05.986 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.986+0000 7f58aa211640 1 --2- 172.21.15.100:0/1603650323 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556c7876a7f0 0x556c7876cc10 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5eb520d09027d729 server_cookie=60fb3032924771d2 in_seq=0 out_seq=0 2024-05-10T11:22:05.988 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.986+0000 7f588dffb640 1 -- 172.21.15.100:0/1603650323 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556c786c71e0 con 0x556c7876a7f0 2024-05-10T11:22:05.988 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.986+0000 7f588dffb640 10 client.16128 handle_command_reply: tid=0 2024-05-10T11:22:05.988 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.986+0000 7f58af19a740 1 client.16128 shutdown 2024-05-10T11:22:05.988 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.986+0000 7f58af19a740 1 -- 172.21.15.100:0/1603650323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c7875a910 msgr2=0x556c78766300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:05.988 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.986+0000 7f58af19a740 1 --2- 172.21.15.100:0/1603650323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c7875a910 0x556c78766300 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f589c005e30 tx=0x7f589c004780 comp rx=0 tx=0).stop 2024-05-10T11:22:05.988 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.986+0000 7f58af19a740 1 -- 172.21.15.100:0/1603650323 shutdown_connections 2024-05-10T11:22:05.988 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.986+0000 7f58af19a740 1 --2- 172.21.15.100:0/1603650323 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556c7876a7f0 0x556c7876cc10 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:05.988 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.986+0000 7f58af19a740 1 --2- 172.21.15.100:0/1603650323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c7875a910 0x556c78766300 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:05.988 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.986+0000 7f58af19a740 1 --2- 172.21.15.100:0/1603650323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c787557a0 0x556c7875f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:05.988 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.986+0000 7f588dffb640 0 client.16128 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:22:05.988 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.986+0000 7f58af19a740 1 --2- 172.21.15.100:0/1603650323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556c78754db0 0x556c7875ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:05.988 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.986+0000 7f58af19a740 1 -- 172.21.15.100:0/1603650323 >> 172.21.15.100:0/1603650323 conn(0x556c78750420 msgr2=0x556c7874f580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:05.988 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.987+0000 7f58af19a740 1 -- 172.21.15.100:0/1603650323 shutdown_connections 2024-05-10T11:22:05.989 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.987+0000 7f58af19a740 1 -- 172.21.15.100:0/1603650323 wait complete. 2024-05-10T11:22:05.989 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:05.987+0000 7f58af19a740 20 client.16128 trim_cache size 0 max 16384 2024-05-10T11:22:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.004+0000 7f58af19a740 1 Processor -- start 2024-05-10T11:22:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.004+0000 7f58af19a740 1 -- start start 2024-05-10T11:22:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.004+0000 7f58af19a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c78754db0 0x556c7875e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.004+0000 7f58af19a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c787557a0 0x556c787662c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.004+0000 7f58af19a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556c7875a910 0x556c7875e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.004+0000 7f58af19a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f58a41afc90 con 0x556c787557a0 2024-05-10T11:22:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.004+0000 7f58af19a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f58a41ad560 con 0x556c78754db0 2024-05-10T11:22:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.004+0000 7f58af19a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f58a41ab3b0 con 0x556c7875a910 2024-05-10T11:22:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.005+0000 7f58a9a10640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c787557a0 0x556c787662c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.005+0000 7f58a9a10640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c787557a0 0x556c787662c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56586/0 (socket says 172.21.15.100:56586) 2024-05-10T11:22:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.005+0000 7f58a9a10640 1 -- 172.21.15.100:0/3622645056 learned_addr learned my addr 172.21.15.100:0/3622645056 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:22:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.005+0000 7f58aaa12640 1 --2- 172.21.15.100:0/3622645056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556c7875a910 0x556c7875e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.005+0000 7f58aa211640 1 --2- 172.21.15.100:0/3622645056 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c78754db0 0x556c7875e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.005+0000 7f58a9a10640 1 -- 172.21.15.100:0/3622645056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556c7875a910 msgr2=0x556c7875e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.005+0000 7f58a9a10640 1 --2- 172.21.15.100:0/3622645056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556c7875a910 0x556c7875e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.005+0000 7f58a9a10640 1 -- 172.21.15.100:0/3622645056 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c78754db0 msgr2=0x556c7875e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.005+0000 7f58a9a10640 1 --2- 172.21.15.100:0/3622645056 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c78754db0 0x556c7875e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.005+0000 7f58a9a10640 1 -- 172.21.15.100:0/3622645056 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c787549d0 con 0x556c787557a0 2024-05-10T11:22:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.005+0000 7f58a9a10640 1 --2- 172.21.15.100:0/3622645056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c787557a0 0x556c787662c0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f58940123e0 tx=0x7f5894017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42872f44e8450223 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.005+0000 7f588ffff640 1 -- 172.21.15.100:0/3622645056 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5894004110 con 0x556c787557a0 2024-05-10T11:22:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.005+0000 7f588ffff640 1 -- 172.21.15.100:0/3622645056 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f58940042b0 con 0x556c787557a0 2024-05-10T11:22:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.005+0000 7f588ffff640 1 -- 172.21.15.100:0/3622645056 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589402a620 con 0x556c787557a0 2024-05-10T11:22:06.010 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.006+0000 7f58af19a740 1 -- 172.21.15.100:0/3622645056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c787557a0 msgr2=0x556c787662c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:06.010 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.006+0000 7f58af19a740 1 --2- 172.21.15.100:0/3622645056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c787557a0 0x556c787662c0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f58940123e0 tx=0x7f5894017000 comp rx=0 tx=0).stop 2024-05-10T11:22:06.010 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.006+0000 7f58af19a740 1 -- 172.21.15.100:0/3622645056 shutdown_connections 2024-05-10T11:22:06.010 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.006+0000 7f58af19a740 1 --2- 172.21.15.100:0/3622645056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556c7875a910 0x556c7875e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:06.010 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.006+0000 7f58af19a740 1 --2- 172.21.15.100:0/3622645056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c787557a0 0x556c787662c0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:06.010 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.006+0000 7f58af19a740 1 --2- 172.21.15.100:0/3622645056 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c78754db0 0x556c7875e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:06.010 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.006+0000 7f58af19a740 1 -- 172.21.15.100:0/3622645056 >> 172.21.15.100:0/3622645056 conn(0x556c78750420 msgr2=0x556c7875ed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:06.011 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.006+0000 7f58af19a740 1 -- 172.21.15.100:0/3622645056 shutdown_connections 2024-05-10T11:22:06.011 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.006+0000 7f58af19a740 1 -- 172.21.15.100:0/3622645056 wait complete. 2024-05-10T11:22:06.011 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.006+0000 7f58af19a740 1 -- start start 2024-05-10T11:22:06.011 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.006+0000 7f58af19a740 1 Processor -- start 2024-05-10T11:22:06.011 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.007+0000 7f58af19a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c78754db0 0x556c78763560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:06.011 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.007+0000 7f58af19a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c787557a0 0x556c78763aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:06.011 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.007+0000 7f58af19a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556c7875a910 0x556c78764ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:06.011 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.007+0000 7f58af19a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f58a41aba20 con 0x556c78754db0 2024-05-10T11:22:06.011 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.007+0000 7f58af19a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f58a41ace90 con 0x556c787557a0 2024-05-10T11:22:06.011 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.007+0000 7f58af19a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f58a41ac0f0 con 0x556c7875a910 2024-05-10T11:22:06.011 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.007+0000 7f58af19a740 10 client.0 _ll_register_callbacks cb 0x7f58a00e8420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:22:06.011 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.007+0000 7f58aa211640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c78754db0 0x556c78763560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:06.011 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.007+0000 7f58aa211640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c78754db0 0x556c78763560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56600/0 (socket says 172.21.15.100:56600) 2024-05-10T11:22:06.011 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.007+0000 7f58aa211640 1 -- 172.21.15.100:0/1009979220 learned_addr learned my addr 172.21.15.100:0/1009979220 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:22:06.011 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.007+0000 7f58aaa12640 1 --2- 172.21.15.100:0/1009979220 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556c7875a910 0x556c78764ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:06.012 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.007+0000 7f58a9a10640 1 --2- 172.21.15.100:0/1009979220 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c787557a0 0x556c78763aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:06.012 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.007+0000 7f58aa211640 1 -- 172.21.15.100:0/1009979220 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556c7875a910 msgr2=0x556c78764ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:06.012 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.007+0000 7f58aa211640 1 --2- 172.21.15.100:0/1009979220 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556c7875a910 0x556c78764ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:06.012 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.007+0000 7f58aa211640 1 -- 172.21.15.100:0/1009979220 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c787557a0 msgr2=0x556c78763aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:06.012 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.007+0000 7f58aa211640 1 --2- 172.21.15.100:0/1009979220 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c787557a0 0x556c78763aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:06.012 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.007+0000 7f58aa211640 1 -- 172.21.15.100:0/1009979220 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c787650a0 con 0x556c78754db0 2024-05-10T11:22:06.012 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.007+0000 7f58aa211640 1 --2- 172.21.15.100:0/1009979220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c78754db0 0x556c78763560 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f58a00a97f0 tx=0x7f58a00e8550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=550cf518b72c0211 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:06.012 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.007+0000 7f588dffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:22:06.012 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.007+0000 7f588dffb640 1 -- 172.21.15.100:0/1009979220 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a00c2020 con 0x556c78754db0 2024-05-10T11:22:06.012 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.007+0000 7f588dffb640 1 -- 172.21.15.100:0/1009979220 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f58a00b24c0 con 0x556c78754db0 2024-05-10T11:22:06.012 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.007+0000 7f58af19a740 1 -- 172.21.15.100:0/1009979220 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58940042b0 con 0x556c78754db0 2024-05-10T11:22:06.012 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.008+0000 7f588dffb640 1 -- 172.21.15.100:0/1009979220 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f58a00a9260 con 0x556c78754db0 2024-05-10T11:22:06.012 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.008+0000 7f58af19a740 10 client.16134 fetch_fsmap learned FSMap version 29 2024-05-10T11:22:06.012 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.008+0000 7f58af19a740 1 -- 172.21.15.100:0/1009979220 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556c78766480 con 0x556c78754db0 2024-05-10T11:22:06.013 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.008+0000 7f588dffb640 1 -- 172.21.15.100:0/1009979220 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f58940042b0 con 0x556c78754db0 2024-05-10T11:22:06.013 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.008+0000 7f588dffb640 1 -- 172.21.15.100:0/1009979220 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f58a00e2ce0 con 0x556c78754db0 2024-05-10T11:22:06.013 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.008+0000 7f58af19a740 10 client.16134 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T11:22:06.013 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.008+0000 7f58af19a740 10 client.16134 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:22:06.013 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.008+0000 7f58af19a740 20 client.16134 populate_metadata read hostname 'smithi100' 2024-05-10T11:22:06.013 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.008+0000 7f58af19a740 1 --2- 172.21.15.100:0/1009979220 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556c787d5800 0x556c787d7c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:06.013 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.008+0000 7f58af19a740 4 client.16134 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:22:06.013 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.008+0000 7f58af19a740 1 -- 172.21.15.100:0/1009979220 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556c786c1050 con 0x556c787d5800 2024-05-10T11:22:06.013 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.008+0000 7f58a9a10640 1 --2- 172.21.15.100:0/1009979220 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556c787d5800 0x556c787d7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:06.013 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.009+0000 7f58a9a10640 1 --2- 172.21.15.100:0/1009979220 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556c787d5800 0x556c787d7c20 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbab7f13019d48c7 server_cookie=987a072584f0a949 in_seq=0 out_seq=0 2024-05-10T11:22:06.013 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.009+0000 7f588dffb640 10 client.16134 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:22:06.027 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.027+0000 7f588dffb640 1 -- 172.21.15.100:0/1009979220 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556c786c1050 con 0x556c787d5800 2024-05-10T11:22:06.027 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.027+0000 7f588dffb640 10 client.16134 handle_command_reply: tid=0 2024-05-10T11:22:06.027 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.027+0000 7f58af19a740 1 client.16134 shutdown 2024-05-10T11:22:06.028 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.027+0000 7f58af19a740 1 -- 172.21.15.100:0/1009979220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c78754db0 msgr2=0x556c78763560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:06.028 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.027+0000 7f58af19a740 1 --2- 172.21.15.100:0/1009979220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c78754db0 0x556c78763560 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f58a00a97f0 tx=0x7f58a00e8550 comp rx=0 tx=0).stop 2024-05-10T11:22:06.028 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.027+0000 7f58af19a740 1 -- 172.21.15.100:0/1009979220 shutdown_connections 2024-05-10T11:22:06.028 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.027+0000 7f58af19a740 1 --2- 172.21.15.100:0/1009979220 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556c787d5800 0x556c787d7c20 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:06.028 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.027+0000 7f58af19a740 1 --2- 172.21.15.100:0/1009979220 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556c7875a910 0x556c78764ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:06.028 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.027+0000 7f58af19a740 1 --2- 172.21.15.100:0/1009979220 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c787557a0 0x556c78763aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:06.028 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.027+0000 7f58af19a740 1 --2- 172.21.15.100:0/1009979220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c78754db0 0x556c78763560 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:06.028 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.027+0000 7f588dffb640 0 client.16134 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:22:06.028 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.027+0000 7f58af19a740 1 -- 172.21.15.100:0/1009979220 >> 172.21.15.100:0/1009979220 conn(0x556c78750420 msgr2=0x556c7874ecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:06.028 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.027+0000 7f58af19a740 1 -- 172.21.15.100:0/1009979220 shutdown_connections 2024-05-10T11:22:06.028 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.028+0000 7f58af19a740 1 -- 172.21.15.100:0/1009979220 wait complete. 2024-05-10T11:22:06.028 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.028+0000 7f58af19a740 20 client.16134 trim_cache size 0 max 16384 2024-05-10T11:22:06.029 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:22:06.029 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:22:06.029 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:22:06.029 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:22:06.030 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.028+0000 7f58ac475640 1 -- 172.21.15.100:0/3935407006 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f58900b2920 msgr2=0x7f58900b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:06.030 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.028+0000 7f58ac475640 1 --2- 172.21.15.100:0/3935407006 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f58900b2920 0x7f58900b4de0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5894009fd0 tx=0x7f5894005f90 comp rx=0 tx=0).stop 2024-05-10T11:22:06.030 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.028+0000 7f58ac475640 1 -- 172.21.15.100:0/3935407006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58a4103530 msgr2=0x7f58a419ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:06.030 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.028+0000 7f58ac475640 1 --2- 172.21.15.100:0/3935407006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58a4103530 0x7f58a419ace0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f58a000e9f0 tx=0x7f58a000edf0 comp rx=0 tx=0).stop 2024-05-10T11:22:06.030 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.028+0000 7f58ac475640 1 -- 172.21.15.100:0/3935407006 shutdown_connections 2024-05-10T11:22:06.030 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.028+0000 7f58ac475640 1 --2- 172.21.15.100:0/3935407006 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f58900b2920 0x7f58900b4de0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:06.030 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.028+0000 7f58ac475640 1 --2- 172.21.15.100:0/3935407006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58a4108790 0x7f58a41a2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:06.030 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.028+0000 7f58ac475640 1 --2- 172.21.15.100:0/3935407006 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58a4105e60 0x7f58a419b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:06.030 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.028+0000 7f58ac475640 1 --2- 172.21.15.100:0/3935407006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58a4103530 0x7f58a419ace0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:06.030 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.028+0000 7f58ac475640 1 -- 172.21.15.100:0/3935407006 >> 172.21.15.100:0/3935407006 conn(0x7f58a40fcec0 msgr2=0x7f58a4108b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:06.030 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.028+0000 7f58ac475640 1 -- 172.21.15.100:0/3935407006 shutdown_connections 2024-05-10T11:22:06.030 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:06.028+0000 7f58ac475640 1 -- 172.21.15.100:0/3935407006 wait complete. 2024-05-10T11:22:06.934 INFO:tasks.workunit.client.0.smithi100.stdout: 1 64328 0.83 MB/sec execute 1 sec latency 14.954 ms 2024-05-10T11:22:07.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:22:07 smithi178 ceph-mon[33359]: pgmap v532: 129 pgs: 129 active+clean; 40 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 1.4 MiB/s wr, 175 op/s 2024-05-10T11:22:07.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:22:07 smithi188 ceph-mon[34075]: pgmap v532: 129 pgs: 129 active+clean; 40 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 1.4 MiB/s wr, 175 op/s 2024-05-10T11:22:07.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:22:07 smithi100 ceph-mon[30296]: pgmap v532: 129 pgs: 129 active+clean; 40 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 1.4 MiB/s wr, 175 op/s 2024-05-10T11:22:07.934 INFO:tasks.workunit.client.0.smithi100.stdout: 1 64778 0.78 MB/sec execute 2 sec latency 12.716 ms 2024-05-10T11:22:08.934 INFO:tasks.workunit.client.0.smithi100.stdout: 1 65324 1.97 MB/sec execute 3 sec latency 17.046 ms 2024-05-10T11:22:09.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:22:09 smithi178 ceph-mon[33359]: pgmap v533: 129 pgs: 129 active+clean; 41 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 1.4 MiB/s wr, 153 op/s 2024-05-10T11:22:09.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:22:09 smithi188 ceph-mon[34075]: pgmap v533: 129 pgs: 129 active+clean; 41 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 1.4 MiB/s wr, 153 op/s 2024-05-10T11:22:09.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:22:09 smithi100 ceph-mon[30296]: pgmap v533: 129 pgs: 129 active+clean; 41 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 1.4 MiB/s wr, 153 op/s 2024-05-10T11:22:09.935 INFO:tasks.workunit.client.0.smithi100.stdout: 1 65969 1.63 MB/sec execute 4 sec latency 21.378 ms 2024-05-10T11:22:10.935 INFO:tasks.workunit.client.0.smithi100.stdout: 1 66491 1.79 MB/sec execute 5 sec latency 14.978 ms 2024-05-10T11:22:11.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:22:11 smithi178 ceph-mon[33359]: pgmap v534: 129 pgs: 129 active+clean; 41 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 1.2 MiB/s wr, 139 op/s 2024-05-10T11:22:11.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:22:11 smithi188 ceph-mon[34075]: pgmap v534: 129 pgs: 129 active+clean; 41 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 1.2 MiB/s wr, 139 op/s 2024-05-10T11:22:11.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:22:11 smithi100 ceph-mon[30296]: pgmap v534: 129 pgs: 129 active+clean; 41 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 1.2 MiB/s wr, 139 op/s 2024-05-10T11:22:11.935 INFO:tasks.workunit.client.0.smithi100.stdout: 1 67201 2.54 MB/sec execute 6 sec latency 14.211 ms 2024-05-10T11:22:12.935 INFO:tasks.workunit.client.0.smithi100.stdout: 1 67689 2.81 MB/sec execute 7 sec latency 11.570 ms 2024-05-10T11:22:13.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:22:13 smithi178 ceph-mon[33359]: pgmap v535: 129 pgs: 129 active+clean; 41 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 1.4 MiB/s wr, 182 op/s 2024-05-10T11:22:13.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:22:13 smithi188 ceph-mon[34075]: pgmap v535: 129 pgs: 129 active+clean; 41 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 1.4 MiB/s wr, 182 op/s 2024-05-10T11:22:13.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:22:13 smithi100 ceph-mon[30296]: pgmap v535: 129 pgs: 129 active+clean; 41 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 1.4 MiB/s wr, 182 op/s 2024-05-10T11:22:13.935 INFO:tasks.workunit.client.0.smithi100.stdout: 1 67964 2.50 MB/sec execute 8 sec latency 51.924 ms 2024-05-10T11:22:14.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:22:14 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:22:14.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:22:14 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:22:14.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:22:14 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:22:14.935 INFO:tasks.workunit.client.0.smithi100.stdout: 1 68364 2.31 MB/sec execute 9 sec latency 105.238 ms 2024-05-10T11:22:15.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:22:15 smithi178 ceph-mon[33359]: pgmap v536: 129 pgs: 129 active+clean; 41 MiB data, 718 MiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 1.3 MiB/s wr, 125 op/s 2024-05-10T11:22:15.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:22:15 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:22:15.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:22:15 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:22:15.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:22:15 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:22:15.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:22:15 smithi188 ceph-mon[34075]: pgmap v536: 129 pgs: 129 active+clean; 41 MiB data, 718 MiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 1.3 MiB/s wr, 125 op/s 2024-05-10T11:22:15.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:22:15 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:22:15.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:22:15 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:22:15.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:22:15 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:22:15.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:22:15 smithi100 ceph-mon[30296]: pgmap v536: 129 pgs: 129 active+clean; 41 MiB data, 718 MiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 1.3 MiB/s wr, 125 op/s 2024-05-10T11:22:15.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:22:15 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:22:15.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:22:15 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:22:15.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:22:15 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:22:15.935 INFO:tasks.workunit.client.0.smithi100.stdout: 1 68957 2.51 MB/sec execute 10 sec latency 12.010 ms 2024-05-10T11:22:16.935 INFO:tasks.workunit.client.0.smithi100.stdout: 1 69620 2.34 MB/sec execute 11 sec latency 14.452 ms 2024-05-10T11:22:17.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:22:17 smithi178 ceph-mon[33359]: pgmap v537: 129 pgs: 129 active+clean; 43 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 1.5 MiB/s wr, 156 op/s 2024-05-10T11:22:17.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:22:17 smithi188 ceph-mon[34075]: pgmap v537: 129 pgs: 129 active+clean; 43 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 1.5 MiB/s wr, 156 op/s 2024-05-10T11:22:17.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:22:17 smithi100 ceph-mon[30296]: pgmap v537: 129 pgs: 129 active+clean; 43 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 1.5 MiB/s wr, 156 op/s 2024-05-10T11:22:17.935 INFO:tasks.workunit.client.0.smithi100.stdout: 1 70192 2.58 MB/sec execute 12 sec latency 16.534 ms 2024-05-10T11:22:18.935 INFO:tasks.workunit.client.0.smithi100.stdout: 1 70761 2.65 MB/sec execute 13 sec latency 15.441 ms 2024-05-10T11:22:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.130+0000 7f077acd1640 1 -- 172.21.15.100:0/2289043361 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0774101910 msgr2=0x7f0774101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.130+0000 7f077acd1640 1 --2- 172.21.15.100:0/2289043361 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0774101910 0x7f0774101d30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0770009860 tx=0x7f077002a340 comp rx=0 tx=0).stop 2024-05-10T11:22:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.130+0000 7f077acd1640 1 -- 172.21.15.100:0/2289043361 shutdown_connections 2024-05-10T11:22:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.130+0000 7f077acd1640 1 --2- 172.21.15.100:0/2289043361 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07741034c0 0x7f0774109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:19.133 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.130+0000 7f077acd1640 1 --2- 172.21.15.100:0/2289043361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0774102b00 0x7f0774102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:19.133 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.130+0000 7f077acd1640 1 --2- 172.21.15.100:0/2289043361 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0774101910 0x7f0774101d30 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:19.133 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.130+0000 7f077acd1640 1 -- 172.21.15.100:0/2289043361 >> 172.21.15.100:0/2289043361 conn(0x7f07740fce80 msgr2=0x7f07740ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:19.133 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.130+0000 7f077acd1640 1 -- 172.21.15.100:0/2289043361 shutdown_connections 2024-05-10T11:22:19.133 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.130+0000 7f077acd1640 1 -- 172.21.15.100:0/2289043361 wait complete. 2024-05-10T11:22:19.133 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.131+0000 7f077acd1640 1 Processor -- start 2024-05-10T11:22:19.133 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.131+0000 7f077acd1640 1 -- start start 2024-05-10T11:22:19.133 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.131+0000 7f077acd1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0774101910 0x7f077406ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:19.133 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.131+0000 7f077acd1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0774102b00 0x7f077419d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:19.133 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.131+0000 7f077acd1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07741034c0 0x7f077419fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:19.133 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.131+0000 7f077acd1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f077410d2c0 con 0x7f0774102b00 2024-05-10T11:22:19.134 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.131+0000 7f077acd1640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f077410d140 con 0x7f0774101910 2024-05-10T11:22:19.134 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.131+0000 7f077acd1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f077410d440 con 0x7f07741034c0 2024-05-10T11:22:19.134 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.131+0000 7f076bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0774102b00 0x7f077419d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:19.134 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.131+0000 7f076bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0774102b00 0x7f077419d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46840/0 (socket says 172.21.15.100:46840) 2024-05-10T11:22:19.134 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.131+0000 7f076bfff640 1 -- 172.21.15.100:0/85879572 learned_addr learned my addr 172.21.15.100:0/85879572 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:22:19.134 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.131+0000 7f0778a6d640 1 --2- 172.21.15.100:0/85879572 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0774101910 0x7f077406ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:19.134 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.131+0000 7f077926e640 1 --2- 172.21.15.100:0/85879572 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07741034c0 0x7f077419fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:19.134 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.131+0000 7f076bfff640 1 -- 172.21.15.100:0/85879572 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07741034c0 msgr2=0x7f077419fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:19.134 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.131+0000 7f076bfff640 1 --2- 172.21.15.100:0/85879572 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07741034c0 0x7f077419fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:19.134 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.131+0000 7f076bfff640 1 -- 172.21.15.100:0/85879572 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0774101910 msgr2=0x7f077406ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:19.134 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.131+0000 7f076bfff640 1 --2- 172.21.15.100:0/85879572 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0774101910 0x7f077406ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:19.134 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.131+0000 7f076bfff640 1 -- 172.21.15.100:0/85879572 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07741a0710 con 0x7f0774102b00 2024-05-10T11:22:19.135 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.131+0000 7f0778a6d640 1 --2- 172.21.15.100:0/85879572 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0774101910 0x7f077406ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:22:19.135 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.132+0000 7f076bfff640 1 --2- 172.21.15.100:0/85879572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0774102b00 0x7f077419d7e0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f076400e9c0 tx=0x7f076400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:19.135 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.132+0000 7f0769ffb640 1 -- 172.21.15.100:0/85879572 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f076400ccc0 con 0x7f0774102b00 2024-05-10T11:22:19.135 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.132+0000 7f0769ffb640 1 -- 172.21.15.100:0/85879572 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f07640054b0 con 0x7f0774102b00 2024-05-10T11:22:19.135 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.132+0000 7f077acd1640 1 -- 172.21.15.100:0/85879572 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07741a0a00 con 0x7f0774102b00 2024-05-10T11:22:19.135 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.132+0000 7f0769ffb640 1 -- 172.21.15.100:0/85879572 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0764010430 con 0x7f0774102b00 2024-05-10T11:22:19.135 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.132+0000 7f077acd1640 1 -- 172.21.15.100:0/85879572 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07741a0f50 con 0x7f0774102b00 2024-05-10T11:22:19.142 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.135+0000 7f0769ffb640 1 -- 172.21.15.100:0/85879572 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f07640046c0 con 0x7f0774102b00 2024-05-10T11:22:19.143 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.135+0000 7f077d9ee740 1 Processor -- start 2024-05-10T11:22:19.143 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.135+0000 7f077d9ee740 1 -- start start 2024-05-10T11:22:19.143 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.135+0000 7f077d9ee740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c256591c0 0x557c256595e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:19.143 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.135+0000 7f077d9ee740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c25659bb0 0x557c2565e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:19.143 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.135+0000 7f077d9ee740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c2565ed20 0x557c256611b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:19.143 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.135+0000 7f077d9ee740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x557c255b95d0 con 0x557c25659bb0 2024-05-10T11:22:19.143 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.135+0000 7f077d9ee740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x557c25572890 con 0x557c256591c0 2024-05-10T11:22:19.143 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.135+0000 7f077d9ee740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x557c255b5590 con 0x557c2565ed20 2024-05-10T11:22:19.143 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.135+0000 7f0778a6d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c25659bb0 0x557c2565e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:19.143 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.135+0000 7f077926e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c256591c0 0x557c256595e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:19.143 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.135+0000 7f0778a6d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c25659bb0 0x557c2565e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46842/0 (socket says 172.21.15.100:46842) 2024-05-10T11:22:19.143 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.135+0000 7f0778a6d640 1 -- 172.21.15.100:0/1179106580 learned_addr learned my addr 172.21.15.100:0/1179106580 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:22:19.144 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.136+0000 7f0769ffb640 1 --2- 172.21.15.100:0/85879572 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f075c0b28a0 0x7f075c0b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:19.144 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.136+0000 7f076bfff640 1 --2- 172.21.15.100:0/1179106580 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c2565ed20 0x557c256611b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:19.144 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.136+0000 7f077926e640 1 -- 172.21.15.100:0/1179106580 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c2565ed20 msgr2=0x557c256611b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:19.144 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.136+0000 7f077926e640 1 --2- 172.21.15.100:0/1179106580 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c2565ed20 0x557c256611b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:19.144 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.136+0000 7f077926e640 1 -- 172.21.15.100:0/1179106580 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c25659bb0 msgr2=0x557c2565e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:19.144 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.136+0000 7f077926e640 1 --2- 172.21.15.100:0/1179106580 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c25659bb0 0x557c2565e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:19.144 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.136+0000 7f077926e640 1 -- 172.21.15.100:0/1179106580 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c252cb210 con 0x557c256591c0 2024-05-10T11:22:19.144 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.136+0000 7f077926e640 1 --2- 172.21.15.100:0/85879572 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f075c0b28a0 0x7f075c0b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:19.144 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.136+0000 7f0769ffb640 1 -- 172.21.15.100:0/85879572 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f07640e2960 con 0x7f0774102b00 2024-05-10T11:22:19.144 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.136+0000 7f077926e640 1 --2- 172.21.15.100:0/1179106580 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c256591c0 0x557c256595e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f076c009fd0 tx=0x7f076c011920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c782262ac52a24eb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:19.144 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.136+0000 7f077926e640 1 --2- 172.21.15.100:0/85879572 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f075c0b28a0 0x7f075c0b4d60 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f076c019000 tx=0x7f076c01b000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:19.145 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.136+0000 7f075affd640 1 -- 172.21.15.100:0/1179106580 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f076c017070 con 0x557c256591c0 2024-05-10T11:22:19.145 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.136+0000 7f075affd640 1 -- 172.21.15.100:0/1179106580 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f076c003e30 con 0x557c256591c0 2024-05-10T11:22:19.145 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.136+0000 7f075affd640 1 -- 172.21.15.100:0/1179106580 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f076c01bad0 con 0x557c256591c0 2024-05-10T11:22:19.145 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.137+0000 7f077d9ee740 1 -- 172.21.15.100:0/1179106580 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c256591c0 msgr2=0x557c256595e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:19.145 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.137+0000 7f077d9ee740 1 --2- 172.21.15.100:0/1179106580 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c256591c0 0x557c256595e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f076c009fd0 tx=0x7f076c011920 comp rx=0 tx=0).stop 2024-05-10T11:22:19.145 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.137+0000 7f077d9ee740 1 -- 172.21.15.100:0/1179106580 shutdown_connections 2024-05-10T11:22:19.145 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.137+0000 7f077d9ee740 1 --2- 172.21.15.100:0/1179106580 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c2565ed20 0x557c256611b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:19.145 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.137+0000 7f077d9ee740 1 --2- 172.21.15.100:0/1179106580 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c25659bb0 0x557c2565e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:19.145 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.137+0000 7f077d9ee740 1 --2- 172.21.15.100:0/1179106580 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c256591c0 0x557c256595e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:19.145 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.137+0000 7f077d9ee740 1 -- 172.21.15.100:0/1179106580 >> 172.21.15.100:0/1179106580 conn(0x557c25650420 msgr2=0x557c25650820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:19.145 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.137+0000 7f077d9ee740 1 -- 172.21.15.100:0/1179106580 shutdown_connections 2024-05-10T11:22:19.146 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.137+0000 7f077d9ee740 1 -- 172.21.15.100:0/1179106580 wait complete. 2024-05-10T11:22:19.146 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.137+0000 7f077d9ee740 1 -- start start 2024-05-10T11:22:19.146 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.138+0000 7f077d9ee740 1 Processor -- start 2024-05-10T11:22:19.146 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.138+0000 7f077d9ee740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c256591c0 0x557c25663470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:19.146 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.138+0000 7f077d9ee740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c25659bb0 0x557c256639b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:19.146 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.138+0000 7f077d9ee740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c2565ed20 0x557c2566a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:19.146 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.138+0000 7f077d9ee740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x557c25661cd0 con 0x557c256591c0 2024-05-10T11:22:19.146 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.138+0000 7f077d9ee740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x557c25576370 con 0x557c2565ed20 2024-05-10T11:22:19.146 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.138+0000 7f077d9ee740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x557c25661e50 con 0x557c25659bb0 2024-05-10T11:22:19.146 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.138+0000 7f077d9ee740 10 client.0 _ll_register_callbacks cb 0x557c252caa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:22:19.147 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.138+0000 7f0778a6d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c256591c0 0x557c25663470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:19.147 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.138+0000 7f0778a6d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c256591c0 0x557c25663470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46848/0 (socket says 172.21.15.100:46848) 2024-05-10T11:22:19.147 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.138+0000 7f0778a6d640 1 -- 172.21.15.100:0/1036667293 learned_addr learned my addr 172.21.15.100:0/1036667293 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:22:19.147 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.138+0000 7f077926e640 1 --2- 172.21.15.100:0/1036667293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c2565ed20 0x557c2566a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:19.147 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.138+0000 7f076bfff640 1 --2- 172.21.15.100:0/1036667293 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c25659bb0 0x557c256639b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:19.147 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.138+0000 7f0778a6d640 1 -- 172.21.15.100:0/1036667293 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c25659bb0 msgr2=0x557c256639b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:19.147 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.138+0000 7f0778a6d640 1 --2- 172.21.15.100:0/1036667293 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c25659bb0 0x557c256639b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:19.147 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.138+0000 7f0778a6d640 1 -- 172.21.15.100:0/1036667293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c2565ed20 msgr2=0x557c2566a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:19.147 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.138+0000 7f0778a6d640 1 --2- 172.21.15.100:0/1036667293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c2565ed20 0x557c2566a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:19.148 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.138+0000 7f0778a6d640 1 -- 172.21.15.100:0/1036667293 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c25397620 con 0x557c256591c0 2024-05-10T11:22:19.148 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.139+0000 7f0778a6d640 1 --2- 172.21.15.100:0/1036667293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c256591c0 0x557c25663470 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f0774102960 tx=0x7f0770005730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93a1d73e7def968b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:19.148 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.139+0000 7f0753fff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:22:19.148 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.139+0000 7f0753fff640 1 -- 172.21.15.100:0/1036667293 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0770045070 con 0x557c256591c0 2024-05-10T11:22:19.148 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.139+0000 7f0753fff640 1 -- 172.21.15.100:0/1036667293 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f0770004040 con 0x557c256591c0 2024-05-10T11:22:19.148 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.139+0000 7f077d9ee740 1 -- 172.21.15.100:0/1036667293 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f076c003e30 con 0x557c256591c0 2024-05-10T11:22:19.148 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.139+0000 7f0753fff640 1 -- 172.21.15.100:0/1036667293 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0770050050 con 0x557c256591c0 2024-05-10T11:22:19.148 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.139+0000 7f077d9ee740 10 client.16146 fetch_fsmap learned FSMap version 29 2024-05-10T11:22:19.148 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.139+0000 7f077d9ee740 1 -- 172.21.15.100:0/1036667293 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557c2566da40 con 0x557c256591c0 2024-05-10T11:22:19.149 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.139+0000 7f0753fff640 1 -- 172.21.15.100:0/1036667293 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f076c003e30 con 0x557c256591c0 2024-05-10T11:22:19.149 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.139+0000 7f0753fff640 1 -- 172.21.15.100:0/1036667293 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f077003d040 con 0x557c256591c0 2024-05-10T11:22:19.149 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.139+0000 7f077d9ee740 10 client.16146 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T11:22:19.149 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.139+0000 7f077d9ee740 10 client.16146 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:22:19.149 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.139+0000 7f077d9ee740 20 client.16146 populate_metadata read hostname 'smithi100' 2024-05-10T11:22:19.149 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.139+0000 7f077d9ee740 1 --2- 172.21.15.100:0/1036667293 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x557c2566f130 0x557c25671560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:19.149 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.139+0000 7f077d9ee740 4 client.16146 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:22:19.149 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.139+0000 7f077d9ee740 1 -- 172.21.15.100:0/1036667293 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c255c71e0 con 0x557c2566f130 2024-05-10T11:22:19.149 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.140+0000 7f076bfff640 1 --2- 172.21.15.100:0/1036667293 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x557c2566f130 0x557c25671560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:19.149 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.140+0000 7f0753fff640 10 client.16146 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:22:19.149 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.140+0000 7f076bfff640 1 --2- 172.21.15.100:0/1036667293 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x557c2566f130 0x557c25671560 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a27914876c83b7de server_cookie=6dc4ef7a017a601b in_seq=0 out_seq=0 2024-05-10T11:22:19.149 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.140+0000 7f0753fff640 1 -- 172.21.15.100:0/1036667293 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557c255c71e0 con 0x557c2566f130 2024-05-10T11:22:19.149 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.140+0000 7f0753fff640 10 client.16146 handle_command_reply: tid=0 2024-05-10T11:22:19.149 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.141+0000 7f077d9ee740 1 client.16146 shutdown 2024-05-10T11:22:19.150 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.141+0000 7f077d9ee740 1 -- 172.21.15.100:0/1036667293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c256591c0 msgr2=0x557c25663470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:19.150 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.141+0000 7f077d9ee740 1 --2- 172.21.15.100:0/1036667293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c256591c0 0x557c25663470 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f0774102960 tx=0x7f0770005730 comp rx=0 tx=0).stop 2024-05-10T11:22:19.150 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.141+0000 7f077d9ee740 1 -- 172.21.15.100:0/1036667293 shutdown_connections 2024-05-10T11:22:19.150 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.141+0000 7f077d9ee740 1 --2- 172.21.15.100:0/1036667293 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x557c2566f130 0x557c25671560 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:19.150 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.141+0000 7f077d9ee740 1 --2- 172.21.15.100:0/1036667293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c2565ed20 0x557c2566a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:19.150 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.141+0000 7f077d9ee740 1 --2- 172.21.15.100:0/1036667293 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c25659bb0 0x557c256639b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:19.150 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.141+0000 7f077d9ee740 1 --2- 172.21.15.100:0/1036667293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c256591c0 0x557c25663470 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:19.150 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.141+0000 7f0753fff640 0 client.16146 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:22:19.150 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.141+0000 7f077d9ee740 1 -- 172.21.15.100:0/1036667293 >> 172.21.15.100:0/1036667293 conn(0x557c25650420 msgr2=0x557c2564f580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:19.150 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.141+0000 7f077d9ee740 1 -- 172.21.15.100:0/1036667293 shutdown_connections 2024-05-10T11:22:19.150 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.141+0000 7f077d9ee740 1 -- 172.21.15.100:0/1036667293 wait complete. 2024-05-10T11:22:19.150 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.141+0000 7f077d9ee740 20 client.16146 trim_cache size 0 max 16384 2024-05-10T11:22:19.162 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.158+0000 7f077d9ee740 1 Processor -- start 2024-05-10T11:22:19.163 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.158+0000 7f077d9ee740 1 -- start start 2024-05-10T11:22:19.163 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.158+0000 7f077d9ee740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c256591c0 0x557c25653810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:19.163 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.158+0000 7f077d9ee740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c25659bb0 0x557c25653d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:19.163 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.158+0000 7f077d9ee740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c2565ed20 0x557c25662a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:19.163 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.158+0000 7f077d9ee740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f07741aba30 con 0x557c256591c0 2024-05-10T11:22:19.163 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.158+0000 7f077d9ee740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f07741a9300 con 0x557c25659bb0 2024-05-10T11:22:19.163 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.158+0000 7f077d9ee740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f07741a7150 con 0x557c2565ed20 2024-05-10T11:22:19.163 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.158+0000 7f0778a6d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c256591c0 0x557c25653810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:19.163 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.158+0000 7f0778a6d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c256591c0 0x557c25653810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46854/0 (socket says 172.21.15.100:46854) 2024-05-10T11:22:19.163 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.158+0000 7f0778a6d640 1 -- 172.21.15.100:0/816612212 learned_addr learned my addr 172.21.15.100:0/816612212 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:22:19.163 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.158+0000 7f077926e640 1 --2- 172.21.15.100:0/816612212 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c2565ed20 0x557c25662a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:19.164 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.158+0000 7f076bfff640 1 --2- 172.21.15.100:0/816612212 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c25659bb0 0x557c25653d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:19.164 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.159+0000 7f0778a6d640 1 -- 172.21.15.100:0/816612212 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c2565ed20 msgr2=0x557c25662a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:19.164 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.159+0000 7f0778a6d640 1 --2- 172.21.15.100:0/816612212 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c2565ed20 0x557c25662a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:19.164 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.159+0000 7f0778a6d640 1 -- 172.21.15.100:0/816612212 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c25659bb0 msgr2=0x557c25653d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:19.164 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.159+0000 7f0778a6d640 1 --2- 172.21.15.100:0/816612212 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c25659bb0 0x557c25653d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:19.164 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.159+0000 7f0778a6d640 1 -- 172.21.15.100:0/816612212 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c2565b6c0 con 0x557c256591c0 2024-05-10T11:22:19.164 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.159+0000 7f077926e640 1 --2- 172.21.15.100:0/816612212 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c2565ed20 0x557c25662a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:22:19.164 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.159+0000 7f0778a6d640 1 --2- 172.21.15.100:0/816612212 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c256591c0 0x557c25653810 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f0770004570 tx=0x7f07700308d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8967cd2a560024a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:19.164 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.159+0000 7f075a7fc640 1 -- 172.21.15.100:0/816612212 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0770045040 con 0x557c256591c0 2024-05-10T11:22:19.164 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.159+0000 7f075a7fc640 1 -- 172.21.15.100:0/816612212 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0770009450 con 0x557c256591c0 2024-05-10T11:22:19.164 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.159+0000 7f075a7fc640 1 -- 172.21.15.100:0/816612212 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0770037040 con 0x557c256591c0 2024-05-10T11:22:19.164 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.160+0000 7f077d9ee740 1 -- 172.21.15.100:0/816612212 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c256591c0 msgr2=0x557c25653810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:19.164 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.160+0000 7f077d9ee740 1 --2- 172.21.15.100:0/816612212 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c256591c0 0x557c25653810 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f0770004570 tx=0x7f07700308d0 comp rx=0 tx=0).stop 2024-05-10T11:22:19.164 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.160+0000 7f077d9ee740 1 -- 172.21.15.100:0/816612212 shutdown_connections 2024-05-10T11:22:19.164 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.160+0000 7f077d9ee740 1 --2- 172.21.15.100:0/816612212 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c2565ed20 0x557c25662a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:19.165 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.160+0000 7f077d9ee740 1 --2- 172.21.15.100:0/816612212 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c25659bb0 0x557c25653d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:19.165 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.160+0000 7f077d9ee740 1 --2- 172.21.15.100:0/816612212 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c256591c0 0x557c25653810 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:19.165 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.160+0000 7f077d9ee740 1 -- 172.21.15.100:0/816612212 >> 172.21.15.100:0/816612212 conn(0x557c25650420 msgr2=0x557c25653100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:19.165 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.160+0000 7f077d9ee740 1 -- 172.21.15.100:0/816612212 shutdown_connections 2024-05-10T11:22:19.165 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.160+0000 7f077d9ee740 1 -- 172.21.15.100:0/816612212 wait complete. 2024-05-10T11:22:19.165 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.160+0000 7f077d9ee740 1 -- start start 2024-05-10T11:22:19.165 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.160+0000 7f077d9ee740 1 Processor -- start 2024-05-10T11:22:19.165 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.161+0000 7f077d9ee740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c256591c0 0x557c256677f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:19.165 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.161+0000 7f077d9ee740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c25659bb0 0x557c25667d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:19.165 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.161+0000 7f077d9ee740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c2565ed20 0x557c25668d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:19.165 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.161+0000 7f077d9ee740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f07741a77c0 con 0x557c256591c0 2024-05-10T11:22:19.165 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.161+0000 7f077d9ee740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f07741a8c30 con 0x557c25659bb0 2024-05-10T11:22:19.165 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.161+0000 7f077d9ee740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f07741a7e90 con 0x557c2565ed20 2024-05-10T11:22:19.165 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.161+0000 7f077d9ee740 10 client.0 _ll_register_callbacks cb 0x7f076c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:22:19.165 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.161+0000 7f0778a6d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c256591c0 0x557c256677f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:19.166 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.161+0000 7f0778a6d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c256591c0 0x557c256677f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46858/0 (socket says 172.21.15.100:46858) 2024-05-10T11:22:19.166 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.161+0000 7f0778a6d640 1 -- 172.21.15.100:0/3819583502 learned_addr learned my addr 172.21.15.100:0/3819583502 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:22:19.166 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.161+0000 7f076bfff640 1 --2- 172.21.15.100:0/3819583502 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c25659bb0 0x557c25667d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:19.166 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.161+0000 7f077926e640 1 --2- 172.21.15.100:0/3819583502 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c2565ed20 0x557c25668d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:19.166 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.161+0000 7f0778a6d640 1 -- 172.21.15.100:0/3819583502 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c2565ed20 msgr2=0x557c25668d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:19.166 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.161+0000 7f0778a6d640 1 --2- 172.21.15.100:0/3819583502 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c2565ed20 0x557c25668d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:19.166 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.161+0000 7f0778a6d640 1 -- 172.21.15.100:0/3819583502 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c25659bb0 msgr2=0x557c25667d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:19.166 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.161+0000 7f0778a6d640 1 --2- 172.21.15.100:0/3819583502 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c25659bb0 0x557c25667d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:19.166 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.161+0000 7f0778a6d640 1 -- 172.21.15.100:0/3819583502 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c25669330 con 0x557c256591c0 2024-05-10T11:22:19.166 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.161+0000 7f077926e640 1 --2- 172.21.15.100:0/3819583502 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c2565ed20 0x557c25668d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:22:19.166 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.161+0000 7f076bfff640 1 --2- 172.21.15.100:0/3819583502 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c25659bb0 0x557c25667d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:22:19.166 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.161+0000 7f0778a6d640 1 --2- 172.21.15.100:0/3819583502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c256591c0 0x557c256677f0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f0770004650 tx=0x7f0770030f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4f81b22259cf43c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:19.166 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.161+0000 7f0753fff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:22:19.166 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.162+0000 7f0753fff640 1 -- 172.21.15.100:0/3819583502 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f077004d070 con 0x557c256591c0 2024-05-10T11:22:19.167 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.162+0000 7f0753fff640 1 -- 172.21.15.100:0/3819583502 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f0770005020 con 0x557c256591c0 2024-05-10T11:22:19.167 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.162+0000 7f077d9ee740 1 -- 172.21.15.100:0/3819583502 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0770009450 con 0x557c256591c0 2024-05-10T11:22:19.167 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.162+0000 7f0753fff640 1 -- 172.21.15.100:0/3819583502 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0770037040 con 0x557c256591c0 2024-05-10T11:22:19.167 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.162+0000 7f077d9ee740 10 client.16152 fetch_fsmap learned FSMap version 29 2024-05-10T11:22:19.167 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.162+0000 7f077d9ee740 1 -- 172.21.15.100:0/3819583502 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557c256d9a80 con 0x557c256591c0 2024-05-10T11:22:19.167 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.162+0000 7f0753fff640 1 -- 172.21.15.100:0/3819583502 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0770009450 con 0x557c256591c0 2024-05-10T11:22:19.167 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.163+0000 7f0753fff640 1 -- 172.21.15.100:0/3819583502 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f077003f470 con 0x557c256591c0 2024-05-10T11:22:19.167 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.163+0000 7f077d9ee740 10 client.16152 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T11:22:19.167 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.163+0000 7f077d9ee740 10 client.16152 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:22:19.167 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.163+0000 7f077d9ee740 20 client.16152 populate_metadata read hostname 'smithi100' 2024-05-10T11:22:19.167 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.163+0000 7f077d9ee740 1 --2- 172.21.15.100:0/3819583502 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x557c256da360 0x557c256dc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:19.167 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.163+0000 7f077d9ee740 4 client.16152 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:22:19.167 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.163+0000 7f077d9ee740 1 -- 172.21.15.100:0/3819583502 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557c255c1050 con 0x557c256da360 2024-05-10T11:22:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.163+0000 7f076bfff640 1 --2- 172.21.15.100:0/3819583502 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x557c256da360 0x557c256dc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.164+0000 7f076bfff640 1 --2- 172.21.15.100:0/3819583502 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x557c256da360 0x557c256dc780 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b957c2e07c1a01b3 server_cookie=38999f757901f5d2 in_seq=0 out_seq=0 2024-05-10T11:22:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.164+0000 7f0753fff640 10 client.16152 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:22:19.183 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.182+0000 7f0753fff640 1 -- 172.21.15.100:0/3819583502 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557c255c1050 con 0x557c256da360 2024-05-10T11:22:19.183 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.182+0000 7f0753fff640 10 client.16152 handle_command_reply: tid=0 2024-05-10T11:22:19.183 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.182+0000 7f077d9ee740 1 client.16152 shutdown 2024-05-10T11:22:19.183 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.182+0000 7f077d9ee740 1 -- 172.21.15.100:0/3819583502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c256591c0 msgr2=0x557c256677f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:19.183 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.182+0000 7f077d9ee740 1 --2- 172.21.15.100:0/3819583502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c256591c0 0x557c256677f0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f0770004650 tx=0x7f0770030f70 comp rx=0 tx=0).stop 2024-05-10T11:22:19.183 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.182+0000 7f077d9ee740 1 -- 172.21.15.100:0/3819583502 shutdown_connections 2024-05-10T11:22:19.183 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.182+0000 7f077d9ee740 1 --2- 172.21.15.100:0/3819583502 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x557c256da360 0x557c256dc780 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:19.184 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.182+0000 7f077d9ee740 1 --2- 172.21.15.100:0/3819583502 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c2565ed20 0x557c25668d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:19.184 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.182+0000 7f077d9ee740 1 --2- 172.21.15.100:0/3819583502 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c25659bb0 0x557c25667d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:19.184 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.182+0000 7f0753fff640 0 client.16152 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:22:19.184 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.182+0000 7f077d9ee740 1 --2- 172.21.15.100:0/3819583502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c256591c0 0x557c256677f0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:19.184 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.182+0000 7f077d9ee740 1 -- 172.21.15.100:0/3819583502 >> 172.21.15.100:0/3819583502 conn(0x557c25650420 msgr2=0x557c25653990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:19.184 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.182+0000 7f077d9ee740 1 -- 172.21.15.100:0/3819583502 shutdown_connections 2024-05-10T11:22:19.184 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.182+0000 7f077d9ee740 1 -- 172.21.15.100:0/3819583502 wait complete. 2024-05-10T11:22:19.184 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.182+0000 7f077d9ee740 20 client.16152 trim_cache size 0 max 16384 2024-05-10T11:22:19.184 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:22:19.184 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:22:19.184 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:22:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:22:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.183+0000 7f077acd1640 1 -- 172.21.15.100:0/85879572 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f075c0b28a0 msgr2=0x7f075c0b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:19.186 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.183+0000 7f077acd1640 1 --2- 172.21.15.100:0/85879572 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f075c0b28a0 0x7f075c0b4d60 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f076c019000 tx=0x7f076c01b000 comp rx=0 tx=0).stop 2024-05-10T11:22:19.186 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.183+0000 7f077acd1640 1 -- 172.21.15.100:0/85879572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0774102b00 msgr2=0x7f077419d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:19.186 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.183+0000 7f077acd1640 1 --2- 172.21.15.100:0/85879572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0774102b00 0x7f077419d7e0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f076400e9c0 tx=0x7f076400ee80 comp rx=0 tx=0).stop 2024-05-10T11:22:19.186 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.183+0000 7f077acd1640 1 -- 172.21.15.100:0/85879572 shutdown_connections 2024-05-10T11:22:19.186 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.183+0000 7f077acd1640 1 --2- 172.21.15.100:0/85879572 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f075c0b28a0 0x7f075c0b4d60 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:19.186 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.183+0000 7f077acd1640 1 --2- 172.21.15.100:0/85879572 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07741034c0 0x7f077419fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:19.186 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.183+0000 7f077acd1640 1 --2- 172.21.15.100:0/85879572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0774102b00 0x7f077419d7e0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:19.186 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.183+0000 7f077acd1640 1 --2- 172.21.15.100:0/85879572 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0774101910 0x7f077406ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:19.186 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.183+0000 7f077acd1640 1 -- 172.21.15.100:0/85879572 >> 172.21.15.100:0/85879572 conn(0x7f07740fce80 msgr2=0x7f0774105fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:19.186 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.183+0000 7f077acd1640 1 -- 172.21.15.100:0/85879572 shutdown_connections 2024-05-10T11:22:19.187 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:19.183+0000 7f077acd1640 1 -- 172.21.15.100:0/85879572 wait complete. 2024-05-10T11:22:19.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:22:19 smithi178 ceph-mon[33359]: pgmap v538: 129 pgs: 129 active+clean; 44 MiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-05-10T11:22:19.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:22:19 smithi188 ceph-mon[34075]: pgmap v538: 129 pgs: 129 active+clean; 44 MiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-05-10T11:22:19.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:22:19 smithi100 ceph-mon[30296]: pgmap v538: 129 pgs: 129 active+clean; 44 MiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-05-10T11:22:19.935 INFO:tasks.workunit.client.0.smithi100.stdout: 1 71208 2.77 MB/sec execute 14 sec latency 125.212 ms 2024-05-10T11:22:20.935 INFO:tasks.workunit.client.0.smithi100.stdout: 1 71490 2.61 MB/sec execute 15 sec latency 12.889 ms 2024-05-10T11:22:21.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:22:21 smithi178 ceph-mon[33359]: pgmap v539: 129 pgs: 129 active+clean; 43 MiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 1.2 MiB/s wr, 134 op/s 2024-05-10T11:22:21.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:22:21 smithi188 ceph-mon[34075]: pgmap v539: 129 pgs: 129 active+clean; 43 MiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 1.2 MiB/s wr, 134 op/s 2024-05-10T11:22:21.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:22:21 smithi100 ceph-mon[30296]: pgmap v539: 129 pgs: 129 active+clean; 43 MiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 1.2 MiB/s wr, 134 op/s 2024-05-10T11:22:21.935 INFO:tasks.workunit.client.0.smithi100.stdout: 1 71952 2.50 MB/sec execute 16 sec latency 14.544 ms 2024-05-10T11:22:22.935 INFO:tasks.workunit.client.0.smithi100.stdout: 1 72557 2.60 MB/sec execute 17 sec latency 20.639 ms 2024-05-10T11:22:23.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:22:23 smithi178 ceph-mon[33359]: pgmap v540: 129 pgs: 129 active+clean; 45 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 1.5 MiB/s wr, 180 op/s 2024-05-10T11:22:23.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:22:23 smithi188 ceph-mon[34075]: pgmap v540: 129 pgs: 129 active+clean; 45 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 1.5 MiB/s wr, 180 op/s 2024-05-10T11:22:23.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:22:23 smithi100 ceph-mon[30296]: pgmap v540: 129 pgs: 129 active+clean; 45 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 1.5 MiB/s wr, 180 op/s 2024-05-10T11:22:23.935 INFO:tasks.workunit.client.0.smithi100.stdout: 1 73304 2.55 MB/sec execute 18 sec latency 16.739 ms 2024-05-10T11:22:24.935 INFO:tasks.workunit.client.0.smithi100.stdout: 1 73921 2.72 MB/sec execute 19 sec latency 15.473 ms 2024-05-10T11:22:25.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:22:25 smithi178 ceph-mon[33359]: pgmap v541: 129 pgs: 129 active+clean; 44 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.4 MiB/s wr, 135 op/s 2024-05-10T11:22:25.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:22:25 smithi188 ceph-mon[34075]: pgmap v541: 129 pgs: 129 active+clean; 44 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.4 MiB/s wr, 135 op/s 2024-05-10T11:22:25.790 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:22:25 smithi100 ceph-mon[30296]: pgmap v541: 129 pgs: 129 active+clean; 44 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.4 MiB/s wr, 135 op/s 2024-05-10T11:22:25.935 INFO:tasks.workunit.client.0.smithi100.stdout: 1 74485 2.83 MB/sec execute 20 sec latency 16.112 ms 2024-05-10T11:22:26.935 INFO:tasks.workunit.client.0.smithi100.stdout: 1 74889 2.77 MB/sec execute 21 sec latency 14.445 ms 2024-05-10T11:22:27.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:22:27 smithi188 ceph-mon[34075]: pgmap v542: 129 pgs: 129 active+clean; 45 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 1.4 MiB/s wr, 173 op/s 2024-05-10T11:22:27.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:22:27 smithi100 ceph-mon[30296]: pgmap v542: 129 pgs: 129 active+clean; 45 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 1.4 MiB/s wr, 173 op/s 2024-05-10T11:22:27.935 INFO:tasks.workunit.client.0.smithi100.stdout: 1 75165 2.65 MB/sec execute 22 sec latency 14.732 ms 2024-05-10T11:22:28.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:22:27 smithi178 ceph-mon[33359]: pgmap v542: 129 pgs: 129 active+clean; 45 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 1.4 MiB/s wr, 173 op/s 2024-05-10T11:22:28.935 INFO:tasks.workunit.client.0.smithi100.stdout: 1 75720 2.62 MB/sec execute 23 sec latency 15.036 ms 2024-05-10T11:22:29.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:22:29 smithi188 ceph-mon[34075]: pgmap v543: 129 pgs: 129 active+clean; 45 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 1.3 MiB/s wr, 149 op/s 2024-05-10T11:22:29.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:22:29 smithi100 ceph-mon[30296]: pgmap v543: 129 pgs: 129 active+clean; 45 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 1.3 MiB/s wr, 149 op/s 2024-05-10T11:22:29.935 INFO:tasks.workunit.client.0.smithi100.stdout: 1 76371 2.65 MB/sec execute 24 sec latency 17.825 ms 2024-05-10T11:22:30.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:22:29 smithi178 ceph-mon[33359]: pgmap v543: 129 pgs: 129 active+clean; 45 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 1.3 MiB/s wr, 149 op/s 2024-05-10T11:22:30.935 INFO:tasks.workunit.client.0.smithi100.stdout: 1 76976 2.62 MB/sec execute 25 sec latency 15.514 ms 2024-05-10T11:22:31.650 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T11:22:31.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:22:31 smithi188 ceph-mon[34075]: pgmap v544: 129 pgs: 129 active+clean; 47 MiB data, 713 MiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 1.3 MiB/s wr, 146 op/s 2024-05-10T11:22:31.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:22:31 smithi100 ceph-mon[30296]: pgmap v544: 129 pgs: 129 active+clean; 47 MiB data, 713 MiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 1.3 MiB/s wr, 146 op/s 2024-05-10T11:22:31.919 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:22:31.935 INFO:tasks.workunit.client.0.smithi100.stdout: 1 77732 2.75 MB/sec execute 26 sec latency 15.281 ms 2024-05-10T11:22:32.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:22:31 smithi178 ceph-mon[33359]: pgmap v544: 129 pgs: 129 active+clean; 47 MiB data, 713 MiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 1.3 MiB/s wr, 146 op/s 2024-05-10T11:22:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.284+0000 7f9a0d4b4640 1 -- 172.21.15.100:0/3830339773 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a08108790 msgr2=0x7f9a0810ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.284+0000 7f9a0d4b4640 1 --2- 172.21.15.100:0/3830339773 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a08108790 0x7f9a0810ab80 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f9a00009ca0 tx=0x7f9a0002a8c0 comp rx=0 tx=0).stop 2024-05-10T11:22:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.285+0000 7f9a0d4b4640 1 -- 172.21.15.100:0/3830339773 shutdown_connections 2024-05-10T11:22:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.285+0000 7f9a0d4b4640 1 --2- 172.21.15.100:0/3830339773 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a08108790 0x7f9a0810ab80 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.285+0000 7f9a0d4b4640 1 --2- 172.21.15.100:0/3830339773 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a08105e60 0x7f9a08108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.285+0000 7f9a0d4b4640 1 --2- 172.21.15.100:0/3830339773 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a08103530 0x7f9a08105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.285+0000 7f9a0d4b4640 1 -- 172.21.15.100:0/3830339773 >> 172.21.15.100:0/3830339773 conn(0x7f9a080fce80 msgr2=0x7f9a080ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.285+0000 7f9a0d4b4640 1 -- 172.21.15.100:0/3830339773 shutdown_connections 2024-05-10T11:22:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.285+0000 7f9a0d4b4640 1 -- 172.21.15.100:0/3830339773 wait complete. 2024-05-10T11:22:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.285+0000 7f9a0d4b4640 1 Processor -- start 2024-05-10T11:22:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.285+0000 7f9a0d4b4640 1 -- start start 2024-05-10T11:22:32.289 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.286+0000 7f9a0d4b4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a08103530 0x7f9a08198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:32.289 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.286+0000 7f9a0d4b4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a08105e60 0x7f9a08199020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:32.289 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.286+0000 7f9a0d4b4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a08108790 0x7f9a081a0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:32.289 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.286+0000 7f9a0d4b4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0810e460 con 0x7f9a08108790 2024-05-10T11:22:32.290 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.286+0000 7f9a0d4b4640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0810e2e0 con 0x7f9a08105e60 2024-05-10T11:22:32.290 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.286+0000 7f9a0d4b4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0810e5e0 con 0x7f9a08103530 2024-05-10T11:22:32.290 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.286+0000 7f9a077fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a08108790 0x7f9a081a0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:32.290 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.286+0000 7f9a06ffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a08103530 0x7f9a08198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:32.290 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.286+0000 7f9a06ffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a08103530 0x7f9a08198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:51046/0 (socket says 172.21.15.100:51046) 2024-05-10T11:22:32.290 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.286+0000 7f9a077fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a08108790 0x7f9a081a0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40392/0 (socket says 172.21.15.100:40392) 2024-05-10T11:22:32.290 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.286+0000 7f9a06ffd640 1 -- 172.21.15.100:0/2061660755 learned_addr learned my addr 172.21.15.100:0/2061660755 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:22:32.290 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.286+0000 7f99fe7fc640 1 --2- 172.21.15.100:0/2061660755 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a08105e60 0x7f9a08199020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:32.290 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.286+0000 7f9a077fe640 1 -- 172.21.15.100:0/2061660755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a08103530 msgr2=0x7f9a08198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:32.290 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.286+0000 7f9a077fe640 1 --2- 172.21.15.100:0/2061660755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a08103530 0x7f9a08198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:32.291 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.286+0000 7f9a077fe640 1 -- 172.21.15.100:0/2061660755 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a08105e60 msgr2=0x7f9a08199020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:32.291 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.286+0000 7f9a077fe640 1 --2- 172.21.15.100:0/2061660755 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a08105e60 0x7f9a08199020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:32.291 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.286+0000 7f9a077fe640 1 -- 172.21.15.100:0/2061660755 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a081a0700 con 0x7f9a08108790 2024-05-10T11:22:32.291 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.286+0000 7f9a077fe640 1 --2- 172.21.15.100:0/2061660755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a08108790 0x7f9a081a0090 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f9a00031040 tx=0x7f9a00003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:32.291 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.286+0000 7f9a04ff9640 1 -- 172.21.15.100:0/2061660755 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a00032e90 con 0x7f9a08108790 2024-05-10T11:22:32.291 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.286+0000 7f9a04ff9640 1 -- 172.21.15.100:0/2061660755 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9a000044d0 con 0x7f9a08108790 2024-05-10T11:22:32.291 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.287+0000 7f9a0d4b4640 1 -- 172.21.15.100:0/2061660755 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a081a0990 con 0x7f9a08108790 2024-05-10T11:22:32.291 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.287+0000 7f9a04ff9640 1 -- 172.21.15.100:0/2061660755 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a00030ce0 con 0x7f9a08108790 2024-05-10T11:22:32.291 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.287+0000 7f9a0d4b4640 1 -- 172.21.15.100:0/2061660755 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a081a0eb0 con 0x7f9a08108790 2024-05-10T11:22:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.289+0000 7f9a04ff9640 1 -- 172.21.15.100:0/2061660755 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f9a00004040 con 0x7f9a08108790 2024-05-10T11:22:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.289+0000 7f9a101ce740 1 Processor -- start 2024-05-10T11:22:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.289+0000 7f9a101ce740 1 -- start start 2024-05-10T11:22:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.290+0000 7f9a101ce740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650a16d31c0 0x5650a16d35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.290+0000 7f9a101ce740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5650a16d3bb0 0x5650a16d86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.290+0000 7f9a101ce740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650a16d8d20 0x5650a16db1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.290+0000 7f9a101ce740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5650a16335d0 con 0x5650a16d8d20 2024-05-10T11:22:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.290+0000 7f9a101ce740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5650a15ec890 con 0x5650a16d3bb0 2024-05-10T11:22:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.290+0000 7f9a101ce740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5650a162f590 con 0x5650a16d31c0 2024-05-10T11:22:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.290+0000 7f99fe7fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650a16d31c0 0x5650a16d35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:32.300 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.290+0000 7f99fe7fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650a16d31c0 0x5650a16d35e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:51052/0 (socket says 172.21.15.100:51052) 2024-05-10T11:22:32.300 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.290+0000 7f99fe7fc640 1 -- 172.21.15.100:0/1075067818 learned_addr learned my addr 172.21.15.100:0/1075067818 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:22:32.300 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.290+0000 7f99fe7fc640 1 --2- 172.21.15.100:0/1075067818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650a16d8d20 0x5650a16db1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:32.300 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.290+0000 7f9a06ffd640 1 --2- 172.21.15.100:0/1075067818 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5650a16d3bb0 0x5650a16d86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:32.300 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.290+0000 7f99fe7fc640 1 -- 172.21.15.100:0/1075067818 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5650a16d3bb0 msgr2=0x5650a16d86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:32.300 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.290+0000 7f99fe7fc640 1 --2- 172.21.15.100:0/1075067818 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5650a16d3bb0 0x5650a16d86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:32.300 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.290+0000 7f99fe7fc640 1 -- 172.21.15.100:0/1075067818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650a16d8d20 msgr2=0x5650a16db1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:32.300 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.290+0000 7f99fe7fc640 1 --2- 172.21.15.100:0/1075067818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650a16d8d20 0x5650a16db1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:32.300 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.290+0000 7f99fe7fc640 1 -- 172.21.15.100:0/1075067818 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650a1345210 con 0x5650a16d31c0 2024-05-10T11:22:32.300 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.290+0000 7f9a06ffd640 1 --2- 172.21.15.100:0/1075067818 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5650a16d3bb0 0x5650a16d86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:22:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.290+0000 7f9a04ff9640 1 --2- 172.21.15.100:0/2061660755 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f99f40b28a0 0x7f99f40b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.290+0000 7f9a06ffd640 1 --2- 172.21.15.100:0/2061660755 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f99f40b28a0 0x7f99f40b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.290+0000 7f99fe7fc640 1 --2- 172.21.15.100:0/1075067818 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650a16d31c0 0x5650a16d35e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f99f0012770 tx=0x7f99f00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc3c743f58f8e721 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.290+0000 7f9a04ff9640 1 -- 172.21.15.100:0/2061660755 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f9a001039e0 con 0x7f9a08108790 2024-05-10T11:22:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.291+0000 7f99fd7fa640 1 -- 172.21.15.100:0/1075067818 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f0012cf0 con 0x5650a16d31c0 2024-05-10T11:22:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.291+0000 7f99fd7fa640 1 -- 172.21.15.100:0/1075067818 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f99f0010770 con 0x5650a16d31c0 2024-05-10T11:22:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.291+0000 7f99fd7fa640 1 -- 172.21.15.100:0/1075067818 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f00044e0 con 0x5650a16d31c0 2024-05-10T11:22:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.291+0000 7f9a101ce740 1 -- 172.21.15.100:0/1075067818 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650a16d31c0 msgr2=0x5650a16d35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.291+0000 7f9a06ffd640 1 --2- 172.21.15.100:0/2061660755 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f99f40b28a0 0x7f99f40b4d60 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x5650a16d3fb0 tx=0x7f99f8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.291+0000 7f9a101ce740 1 --2- 172.21.15.100:0/1075067818 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650a16d31c0 0x5650a16d35e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f99f0012770 tx=0x7f99f00046c0 comp rx=0 tx=0).stop 2024-05-10T11:22:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.291+0000 7f9a101ce740 1 -- 172.21.15.100:0/1075067818 shutdown_connections 2024-05-10T11:22:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.291+0000 7f9a101ce740 1 --2- 172.21.15.100:0/1075067818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650a16d8d20 0x5650a16db1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.291+0000 7f9a101ce740 1 --2- 172.21.15.100:0/1075067818 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5650a16d3bb0 0x5650a16d86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.291+0000 7f9a101ce740 1 --2- 172.21.15.100:0/1075067818 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650a16d31c0 0x5650a16d35e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.291+0000 7f9a101ce740 1 -- 172.21.15.100:0/1075067818 >> 172.21.15.100:0/1075067818 conn(0x5650a16ca420 msgr2=0x5650a16ca820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.292+0000 7f9a101ce740 1 -- 172.21.15.100:0/1075067818 shutdown_connections 2024-05-10T11:22:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.292+0000 7f9a101ce740 1 -- 172.21.15.100:0/1075067818 wait complete. 2024-05-10T11:22:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.292+0000 7f9a101ce740 1 -- start start 2024-05-10T11:22:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.292+0000 7f9a101ce740 1 Processor -- start 2024-05-10T11:22:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.292+0000 7f9a101ce740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650a16d31c0 0x5650a16dd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.292+0000 7f9a101ce740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650a16d3bb0 0x5650a16dd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.292+0000 7f9a101ce740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5650a16d8d20 0x5650a16e49b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.292+0000 7f9a101ce740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5650a16dbcd0 con 0x5650a16d31c0 2024-05-10T11:22:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.292+0000 7f9a101ce740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5650a15f0370 con 0x5650a16d8d20 2024-05-10T11:22:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.292+0000 7f9a101ce740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5650a16dbe50 con 0x5650a16d3bb0 2024-05-10T11:22:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.292+0000 7f9a101ce740 10 client.0 _ll_register_callbacks cb 0x5650a1344a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:22:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.292+0000 7f9a06ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650a16d31c0 0x5650a16dd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.292+0000 7f99fe7fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650a16d3bb0 0x5650a16dd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.292+0000 7f9a06ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650a16d31c0 0x5650a16dd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40410/0 (socket says 172.21.15.100:40410) 2024-05-10T11:22:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.292+0000 7f9a06ffd640 1 -- 172.21.15.100:0/3081811202 learned_addr learned my addr 172.21.15.100:0/3081811202 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:22:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.292+0000 7f9a077fe640 1 --2- 172.21.15.100:0/3081811202 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5650a16d8d20 0x5650a16e49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.293+0000 7f9a06ffd640 1 -- 172.21.15.100:0/3081811202 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650a16d3bb0 msgr2=0x5650a16dd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.293+0000 7f9a06ffd640 1 --2- 172.21.15.100:0/3081811202 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650a16d3bb0 0x5650a16dd970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.293+0000 7f9a06ffd640 1 -- 172.21.15.100:0/3081811202 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5650a16d8d20 msgr2=0x5650a16e49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.293+0000 7f9a06ffd640 1 --2- 172.21.15.100:0/3081811202 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5650a16d8d20 0x5650a16e49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.293+0000 7f9a06ffd640 1 -- 172.21.15.100:0/3081811202 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650a1411620 con 0x5650a16d31c0 2024-05-10T11:22:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.293+0000 7f99fe7fc640 1 --2- 172.21.15.100:0/3081811202 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650a16d3bb0 0x5650a16dd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:22:32.304 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.293+0000 7f9a06ffd640 1 --2- 172.21.15.100:0/3081811202 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650a16d31c0 0x5650a16dd430 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f99f8005e00 tx=0x7f99f8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c14396cff3b91d35 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:32.304 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.293+0000 7f99ee7fc640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:22:32.304 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.293+0000 7f99ee7fc640 1 -- 172.21.15.100:0/3081811202 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f802e060 con 0x5650a16d31c0 2024-05-10T11:22:32.304 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.293+0000 7f99ee7fc640 1 -- 172.21.15.100:0/3081811202 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f99f8018030 con 0x5650a16d31c0 2024-05-10T11:22:32.304 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.293+0000 7f9a101ce740 1 -- 172.21.15.100:0/3081811202 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99f0010770 con 0x5650a16d31c0 2024-05-10T11:22:32.304 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.293+0000 7f99ee7fc640 1 -- 172.21.15.100:0/3081811202 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f99f8035050 con 0x5650a16d31c0 2024-05-10T11:22:32.304 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.293+0000 7f9a101ce740 10 client.16164 fetch_fsmap learned FSMap version 29 2024-05-10T11:22:32.304 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.293+0000 7f9a101ce740 1 -- 172.21.15.100:0/3081811202 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5650a16e7a60 con 0x5650a16d31c0 2024-05-10T11:22:32.304 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.293+0000 7f99ee7fc640 1 -- 172.21.15.100:0/3081811202 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f99f0010770 con 0x5650a16d31c0 2024-05-10T11:22:32.304 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.293+0000 7f99ee7fc640 1 -- 172.21.15.100:0/3081811202 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f99f801c3a0 con 0x5650a16d31c0 2024-05-10T11:22:32.304 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.294+0000 7f9a101ce740 10 client.16164 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T11:22:32.304 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.294+0000 7f9a101ce740 10 client.16164 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:22:32.304 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.294+0000 7f9a101ce740 20 client.16164 populate_metadata read hostname 'smithi100' 2024-05-10T11:22:32.304 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.294+0000 7f9a101ce740 1 --2- 172.21.15.100:0/3081811202 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5650a16e9210 0x5650a16eb660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:32.305 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.294+0000 7f9a101ce740 4 client.16164 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:22:32.305 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.294+0000 7f9a101ce740 1 -- 172.21.15.100:0/3081811202 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650a16411e0 con 0x5650a16e9210 2024-05-10T11:22:32.305 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.294+0000 7f99fe7fc640 1 --2- 172.21.15.100:0/3081811202 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5650a16e9210 0x5650a16eb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:32.305 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.294+0000 7f99fe7fc640 1 --2- 172.21.15.100:0/3081811202 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5650a16e9210 0x5650a16eb660 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f089db94135a0bc server_cookie=49d5b919b580e034 in_seq=0 out_seq=0 2024-05-10T11:22:32.305 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.294+0000 7f99ee7fc640 10 client.16164 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:22:32.305 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.295+0000 7f99ee7fc640 1 -- 172.21.15.100:0/3081811202 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5650a16411e0 con 0x5650a16e9210 2024-05-10T11:22:32.305 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.295+0000 7f99ee7fc640 10 client.16164 handle_command_reply: tid=0 2024-05-10T11:22:32.305 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.295+0000 7f9a101ce740 1 client.16164 shutdown 2024-05-10T11:22:32.305 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.295+0000 7f9a101ce740 1 -- 172.21.15.100:0/3081811202 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650a16d31c0 msgr2=0x5650a16dd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:32.305 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.295+0000 7f9a101ce740 1 --2- 172.21.15.100:0/3081811202 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650a16d31c0 0x5650a16dd430 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f99f8005e00 tx=0x7f99f8004730 comp rx=0 tx=0).stop 2024-05-10T11:22:32.305 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.295+0000 7f9a101ce740 1 -- 172.21.15.100:0/3081811202 shutdown_connections 2024-05-10T11:22:32.305 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.295+0000 7f9a101ce740 1 --2- 172.21.15.100:0/3081811202 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5650a16e9210 0x5650a16eb660 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:32.305 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.295+0000 7f9a101ce740 1 --2- 172.21.15.100:0/3081811202 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5650a16d8d20 0x5650a16e49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:32.305 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.295+0000 7f9a101ce740 1 --2- 172.21.15.100:0/3081811202 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650a16d3bb0 0x5650a16dd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:32.305 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.295+0000 7f99ee7fc640 0 client.16164 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:22:32.306 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.295+0000 7f9a101ce740 1 --2- 172.21.15.100:0/3081811202 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650a16d31c0 0x5650a16dd430 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:32.306 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.295+0000 7f9a101ce740 1 -- 172.21.15.100:0/3081811202 >> 172.21.15.100:0/3081811202 conn(0x5650a16ca420 msgr2=0x5650a16c9580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:32.306 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.296+0000 7f9a101ce740 1 -- 172.21.15.100:0/3081811202 shutdown_connections 2024-05-10T11:22:32.306 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.296+0000 7f9a101ce740 1 -- 172.21.15.100:0/3081811202 wait complete. 2024-05-10T11:22:32.306 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.296+0000 7f9a101ce740 20 client.16164 trim_cache size 0 max 16384 2024-05-10T11:22:32.316 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.310+0000 7f9a101ce740 1 Processor -- start 2024-05-10T11:22:32.316 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.310+0000 7f9a101ce740 1 -- start start 2024-05-10T11:22:32.316 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.311+0000 7f9a101ce740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650a16d31c0 0x5650a16dd040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:32.316 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.311+0000 7f9a101ce740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650a16d3bb0 0x5650a16e4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:32.316 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.311+0000 7f9a101ce740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5650a16d8d20 0x5650a16dd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:32.316 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.311+0000 7f9a101ce740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9a081ab940 con 0x5650a16d31c0 2024-05-10T11:22:32.316 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.311+0000 7f9a101ce740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9a081a9210 con 0x5650a16d8d20 2024-05-10T11:22:32.316 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.311+0000 7f9a101ce740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9a081a7060 con 0x5650a16d3bb0 2024-05-10T11:22:32.316 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.311+0000 7f9a06ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650a16d31c0 0x5650a16dd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:32.316 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.311+0000 7f9a06ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650a16d31c0 0x5650a16dd040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40420/0 (socket says 172.21.15.100:40420) 2024-05-10T11:22:32.316 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.311+0000 7f9a06ffd640 1 -- 172.21.15.100:0/3646085143 learned_addr learned my addr 172.21.15.100:0/3646085143 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:22:32.316 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.311+0000 7f99fe7fc640 1 --2- 172.21.15.100:0/3646085143 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650a16d3bb0 0x5650a16e4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:32.316 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.311+0000 7f9a06ffd640 1 -- 172.21.15.100:0/3646085143 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650a16d3bb0 msgr2=0x5650a16e4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:32.316 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.311+0000 7f9a06ffd640 1 --2- 172.21.15.100:0/3646085143 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650a16d3bb0 0x5650a16e4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.311+0000 7f9a06ffd640 1 -- 172.21.15.100:0/3646085143 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5650a16d8d20 msgr2=0x5650a16dd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.311+0000 7f9a077fe640 1 --2- 172.21.15.100:0/3646085143 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5650a16d8d20 0x5650a16dd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.311+0000 7f9a06ffd640 1 --2- 172.21.15.100:0/3646085143 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5650a16d8d20 0x5650a16dd580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.311+0000 7f9a06ffd640 1 -- 172.21.15.100:0/3646085143 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650a16e5aa0 con 0x5650a16d31c0 2024-05-10T11:22:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.311+0000 7f9a077fe640 1 --2- 172.21.15.100:0/3646085143 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5650a16d8d20 0x5650a16dd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:22:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.311+0000 7f9a06ffd640 1 --2- 172.21.15.100:0/3646085143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650a16d31c0 0x5650a16dd040 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f99f8019040 tx=0x7f99f801ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33a6daaf5023bd83 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.311+0000 7f99fcff9640 1 -- 172.21.15.100:0/3646085143 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f803e070 con 0x5650a16d31c0 2024-05-10T11:22:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.311+0000 7f99fcff9640 1 -- 172.21.15.100:0/3646085143 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f99f8009b20 con 0x5650a16d31c0 2024-05-10T11:22:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.311+0000 7f99fcff9640 1 -- 172.21.15.100:0/3646085143 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f8029c30 con 0x5650a16d31c0 2024-05-10T11:22:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.312+0000 7f9a101ce740 1 -- 172.21.15.100:0/3646085143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650a16d31c0 msgr2=0x5650a16dd040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.312+0000 7f9a101ce740 1 --2- 172.21.15.100:0/3646085143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650a16d31c0 0x5650a16dd040 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f99f8019040 tx=0x7f99f801ce50 comp rx=0 tx=0).stop 2024-05-10T11:22:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.312+0000 7f9a101ce740 1 -- 172.21.15.100:0/3646085143 shutdown_connections 2024-05-10T11:22:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.312+0000 7f9a101ce740 1 --2- 172.21.15.100:0/3646085143 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5650a16d8d20 0x5650a16dd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.312+0000 7f9a101ce740 1 --2- 172.21.15.100:0/3646085143 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650a16d3bb0 0x5650a16e4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.312+0000 7f9a101ce740 1 --2- 172.21.15.100:0/3646085143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650a16d31c0 0x5650a16dd040 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.312+0000 7f9a101ce740 1 -- 172.21.15.100:0/3646085143 >> 172.21.15.100:0/3646085143 conn(0x5650a16ca420 msgr2=0x5650a16c53e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.312+0000 7f9a101ce740 1 -- 172.21.15.100:0/3646085143 shutdown_connections 2024-05-10T11:22:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.312+0000 7f9a101ce740 1 -- 172.21.15.100:0/3646085143 wait complete. 2024-05-10T11:22:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.313+0000 7f9a101ce740 1 -- start start 2024-05-10T11:22:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.313+0000 7f9a101ce740 1 Processor -- start 2024-05-10T11:22:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.313+0000 7f9a101ce740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650a16d31c0 0x5650a16e1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.313+0000 7f9a101ce740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650a16d3bb0 0x5650a16e2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.313+0000 7f9a101ce740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5650a16d8d20 0x5650a16e32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.313+0000 7f9a101ce740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9a081a76d0 con 0x5650a16d3bb0 2024-05-10T11:22:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.313+0000 7f9a101ce740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9a081a8b40 con 0x5650a16d8d20 2024-05-10T11:22:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.313+0000 7f9a101ce740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9a081a7da0 con 0x5650a16d31c0 2024-05-10T11:22:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.313+0000 7f9a101ce740 10 client.0 _ll_register_callbacks cb 0x7f99f8005d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:22:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.313+0000 7f99fe7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650a16d3bb0 0x5650a16e2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.313+0000 7f99fe7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650a16d3bb0 0x5650a16e2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40436/0 (socket says 172.21.15.100:40436) 2024-05-10T11:22:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.313+0000 7f99fe7fc640 1 -- 172.21.15.100:0/1917614789 learned_addr learned my addr 172.21.15.100:0/1917614789 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:22:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.313+0000 7f9a06ffd640 1 --2- 172.21.15.100:0/1917614789 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650a16d31c0 0x5650a16e1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.313+0000 7f99fe7fc640 1 -- 172.21.15.100:0/1917614789 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650a16d31c0 msgr2=0x5650a16e1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.313+0000 7f99fe7fc640 1 --2- 172.21.15.100:0/1917614789 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650a16d31c0 0x5650a16e1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.313+0000 7f99fe7fc640 1 -- 172.21.15.100:0/1917614789 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5650a16d8d20 msgr2=0x5650a16e32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.313+0000 7f99fe7fc640 1 --2- 172.21.15.100:0/1917614789 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5650a16d8d20 0x5650a16e32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.313+0000 7f99fe7fc640 1 -- 172.21.15.100:0/1917614789 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650a16e3890 con 0x5650a16d3bb0 2024-05-10T11:22:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.314+0000 7f99fe7fc640 1 --2- 172.21.15.100:0/1917614789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650a16d3bb0 0x5650a16e2290 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x5650a16d4410 tx=0x7f99f0005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=324decfaeacba874 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.314+0000 7f99ee7fc640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:22:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.314+0000 7f99ee7fc640 1 -- 172.21.15.100:0/1917614789 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f0010e60 con 0x5650a16d3bb0 2024-05-10T11:22:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.314+0000 7f9a101ce740 1 -- 172.21.15.100:0/1917614789 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99f8009b20 con 0x5650a16d3bb0 2024-05-10T11:22:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.314+0000 7f99ee7fc640 1 -- 172.21.15.100:0/1917614789 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f99f0004030 con 0x5650a16d3bb0 2024-05-10T11:22:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.314+0000 7f99ee7fc640 1 -- 172.21.15.100:0/1917614789 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f99f0038050 con 0x5650a16d3bb0 2024-05-10T11:22:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.314+0000 7f9a101ce740 10 client.16170 fetch_fsmap learned FSMap version 29 2024-05-10T11:22:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.314+0000 7f9a101ce740 1 -- 172.21.15.100:0/1917614789 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5650a16e4b40 con 0x5650a16d3bb0 2024-05-10T11:22:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.314+0000 7f99ee7fc640 1 -- 172.21.15.100:0/1917614789 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f99f8009b20 con 0x5650a16d3bb0 2024-05-10T11:22:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.314+0000 7f99ee7fc640 1 -- 172.21.15.100:0/1917614789 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f99f0029420 con 0x5650a16d3bb0 2024-05-10T11:22:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.314+0000 7f9a101ce740 10 client.16170 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T11:22:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.314+0000 7f9a101ce740 10 client.16170 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:22:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.314+0000 7f9a101ce740 20 client.16170 populate_metadata read hostname 'smithi100' 2024-05-10T11:22:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.314+0000 7f9a101ce740 1 --2- 172.21.15.100:0/1917614789 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5650a1753fb0 0x5650a17563d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.314+0000 7f9a101ce740 4 client.16170 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:22:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.314+0000 7f9a101ce740 1 -- 172.21.15.100:0/1917614789 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5650a163b050 con 0x5650a1753fb0 2024-05-10T11:22:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.314+0000 7f9a06ffd640 1 --2- 172.21.15.100:0/1917614789 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5650a1753fb0 0x5650a17563d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.315+0000 7f99ee7fc640 10 client.16170 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:22:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.315+0000 7f9a06ffd640 1 --2- 172.21.15.100:0/1917614789 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5650a1753fb0 0x5650a17563d0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa076ad071d04d5e server_cookie=c8fb20611a2c5daf in_seq=0 out_seq=0 2024-05-10T11:22:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.334+0000 7f99ee7fc640 1 -- 172.21.15.100:0/1917614789 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5650a163b050 con 0x5650a1753fb0 2024-05-10T11:22:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.334+0000 7f99ee7fc640 10 client.16170 handle_command_reply: tid=0 2024-05-10T11:22:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.334+0000 7f9a101ce740 1 client.16170 shutdown 2024-05-10T11:22:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.334+0000 7f9a101ce740 1 -- 172.21.15.100:0/1917614789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650a16d3bb0 msgr2=0x5650a16e2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.334+0000 7f9a101ce740 1 --2- 172.21.15.100:0/1917614789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650a16d3bb0 0x5650a16e2290 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x5650a16d4410 tx=0x7f99f0005850 comp rx=0 tx=0).stop 2024-05-10T11:22:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.334+0000 7f9a101ce740 1 -- 172.21.15.100:0/1917614789 shutdown_connections 2024-05-10T11:22:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.334+0000 7f9a101ce740 1 --2- 172.21.15.100:0/1917614789 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5650a1753fb0 0x5650a17563d0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.334+0000 7f9a101ce740 1 --2- 172.21.15.100:0/1917614789 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5650a16d8d20 0x5650a16e32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.334+0000 7f9a101ce740 1 --2- 172.21.15.100:0/1917614789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650a16d3bb0 0x5650a16e2290 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.334+0000 7f9a101ce740 1 --2- 172.21.15.100:0/1917614789 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650a16d31c0 0x5650a16e1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.334+0000 7f9a101ce740 1 -- 172.21.15.100:0/1917614789 >> 172.21.15.100:0/1917614789 conn(0x5650a16ca420 msgr2=0x5650a16cd370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.334+0000 7f99ee7fc640 0 client.16170 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:22:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.334+0000 7f9a101ce740 1 -- 172.21.15.100:0/1917614789 shutdown_connections 2024-05-10T11:22:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.334+0000 7f9a101ce740 1 -- 172.21.15.100:0/1917614789 wait complete. 2024-05-10T11:22:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.334+0000 7f9a101ce740 20 client.16170 trim_cache size 0 max 16384 2024-05-10T11:22:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:22:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:22:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:22:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:22:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.335+0000 7f9a0d4b4640 1 -- 172.21.15.100:0/2061660755 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f99f40b28a0 msgr2=0x7f99f40b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.335+0000 7f9a0d4b4640 1 --2- 172.21.15.100:0/2061660755 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f99f40b28a0 0x7f99f40b4d60 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x5650a16d3fb0 tx=0x7f99f8005f90 comp rx=0 tx=0).stop 2024-05-10T11:22:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.335+0000 7f9a0d4b4640 1 -- 172.21.15.100:0/2061660755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a08108790 msgr2=0x7f9a081a0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.335+0000 7f9a0d4b4640 1 --2- 172.21.15.100:0/2061660755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a08108790 0x7f9a081a0090 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f9a00031040 tx=0x7f9a00003ba0 comp rx=0 tx=0).stop 2024-05-10T11:22:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.335+0000 7f9a0d4b4640 1 -- 172.21.15.100:0/2061660755 shutdown_connections 2024-05-10T11:22:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.335+0000 7f9a0d4b4640 1 --2- 172.21.15.100:0/2061660755 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f99f40b28a0 0x7f99f40b4d60 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.335+0000 7f9a0d4b4640 1 --2- 172.21.15.100:0/2061660755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a08108790 0x7f9a081a0090 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.335+0000 7f9a0d4b4640 1 --2- 172.21.15.100:0/2061660755 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a08105e60 0x7f9a08199020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.335+0000 7f9a0d4b4640 1 --2- 172.21.15.100:0/2061660755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a08103530 0x7f9a08198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.335+0000 7f9a0d4b4640 1 -- 172.21.15.100:0/2061660755 >> 172.21.15.100:0/2061660755 conn(0x7f9a080fce80 msgr2=0x7f9a08103f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:32.339 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.335+0000 7f9a0d4b4640 1 -- 172.21.15.100:0/2061660755 shutdown_connections 2024-05-10T11:22:32.339 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:32.335+0000 7f9a0d4b4640 1 -- 172.21.15.100:0/2061660755 wait complete. 2024-05-10T11:22:32.939 INFO:tasks.workunit.client.0.smithi100.stdout: 1 78096 2.80 MB/sec execute 27 sec latency 91.086 ms 2024-05-10T11:22:33.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:22:33 smithi188 ceph-mon[34075]: pgmap v545: 129 pgs: 129 active+clean; 46 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 1.4 MiB/s wr, 172 op/s 2024-05-10T11:22:33.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:22:33 smithi100 ceph-mon[30296]: pgmap v545: 129 pgs: 129 active+clean; 46 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 1.4 MiB/s wr, 172 op/s 2024-05-10T11:22:33.935 INFO:tasks.workunit.client.0.smithi100.stdout: 1 78431 2.76 MB/sec execute 28 sec latency 78.955 ms 2024-05-10T11:22:34.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:22:33 smithi178 ceph-mon[33359]: pgmap v545: 129 pgs: 129 active+clean; 46 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 1.4 MiB/s wr, 172 op/s 2024-05-10T11:22:34.935 INFO:tasks.workunit.client.0.smithi100.stdout: 1 78657 2.67 MB/sec execute 29 sec latency 105.452 ms 2024-05-10T11:22:35.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:22:35 smithi188 ceph-mon[34075]: pgmap v546: 129 pgs: 129 active+clean; 46 MiB data, 763 MiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 1.3 MiB/s wr, 124 op/s 2024-05-10T11:22:35.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:22:35 smithi100 ceph-mon[30296]: pgmap v546: 129 pgs: 129 active+clean; 46 MiB data, 763 MiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 1.3 MiB/s wr, 124 op/s 2024-05-10T11:22:35.935 INFO:tasks.workunit.client.0.smithi100.stdout: 1 79142 2.64 MB/sec execute 30 sec latency 17.354 ms 2024-05-10T11:22:36.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:22:35 smithi178 ceph-mon[33359]: pgmap v546: 129 pgs: 129 active+clean; 46 MiB data, 763 MiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 1.3 MiB/s wr, 124 op/s 2024-05-10T11:22:36.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.800+0000 7f1416405700 1 -- 172.21.15.100:0/1786820017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14101023a0 msgr2=0x7f1410102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:36.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.800+0000 7f1416405700 1 --2- 172.21.15.100:0/1786820017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14101023a0 0x7f1410102820 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f1404009990 tx=0x7f1404009ca0 comp rx=0 tx=0).stop 2024-05-10T11:22:36.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.800+0000 7f1416405700 1 -- 172.21.15.100:0/1786820017 shutdown_connections 2024-05-10T11:22:36.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.800+0000 7f1416405700 1 --2- 172.21.15.100:0/1786820017 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1410102d60 0x7f141010f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:36.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.800+0000 7f1416405700 1 --2- 172.21.15.100:0/1786820017 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1410108410 0x7f14101087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:36.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.800+0000 7f1416405700 1 --2- 172.21.15.100:0/1786820017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14101023a0 0x7f1410102820 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:36.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.800+0000 7f1416405700 1 -- 172.21.15.100:0/1786820017 >> 172.21.15.100:0/1786820017 conn(0x7f1410079350 msgr2=0x7f1410079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:36.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.800+0000 7f1416405700 1 -- 172.21.15.100:0/1786820017 shutdown_connections 2024-05-10T11:22:36.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.800+0000 7f1416405700 1 -- 172.21.15.100:0/1786820017 wait complete. 2024-05-10T11:22:36.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.801+0000 7f1416405700 1 Processor -- start 2024-05-10T11:22:36.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.801+0000 7f1416405700 1 -- start start 2024-05-10T11:22:36.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.801+0000 7f1416405700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14101023a0 0x7f141019f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:36.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.802+0000 7f1416405700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1410102d60 0x7f141019fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:36.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.802+0000 7f1416405700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1410108410 0x7f14101998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:36.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.802+0000 7f1416405700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1410112410 con 0x7f1410108410 2024-05-10T11:22:36.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.802+0000 7f1416405700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1410112290 con 0x7f14101023a0 2024-05-10T11:22:36.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.802+0000 7f1416405700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1410112590 con 0x7f1410102d60 2024-05-10T11:22:36.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.802+0000 7f140f7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1410102d60 0x7f141019fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:36.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.802+0000 7f14149a2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1410108410 0x7f14101998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:36.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.802+0000 7f140f7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1410102d60 0x7f141019fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:57328/0 (socket says 172.21.15.100:57328) 2024-05-10T11:22:36.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.802+0000 7f140f7fe700 1 -- 172.21.15.100:0/750364364 learned_addr learned my addr 172.21.15.100:0/750364364 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:22:36.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.802+0000 7f140ffff700 1 --2- 172.21.15.100:0/750364364 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14101023a0 0x7f141019f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:36.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.802+0000 7f14149a2700 1 -- 172.21.15.100:0/750364364 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1410102d60 msgr2=0x7f141019fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:36.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.802+0000 7f14149a2700 1 --2- 172.21.15.100:0/750364364 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1410102d60 0x7f141019fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:36.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.802+0000 7f14149a2700 1 -- 172.21.15.100:0/750364364 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14101023a0 msgr2=0x7f141019f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:36.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.802+0000 7f14149a2700 1 --2- 172.21.15.100:0/750364364 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14101023a0 0x7f141019f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:36.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.802+0000 7f14149a2700 1 -- 172.21.15.100:0/750364364 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f141019a010 con 0x7f1410108410 2024-05-10T11:22:36.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.802+0000 7f140f7fe700 1 --2- 172.21.15.100:0/750364364 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1410102d60 0x7f141019fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:22:36.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.802+0000 7f14149a2700 1 --2- 172.21.15.100:0/750364364 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1410108410 0x7f14101998b0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f140800efc0 tx=0x7f140800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:36.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.802+0000 7f140d7fa700 1 -- 172.21.15.100:0/750364364 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14080092e0 con 0x7f1410108410 2024-05-10T11:22:36.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.803+0000 7f1416405700 1 -- 172.21.15.100:0/750364364 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f141019a2a0 con 0x7f1410108410 2024-05-10T11:22:36.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.802+0000 7f140d7fa700 1 -- 172.21.15.100:0/750364364 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1408004bb0 con 0x7f1410108410 2024-05-10T11:22:36.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.803+0000 7f1416405700 1 -- 172.21.15.100:0/750364364 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1410109220 con 0x7f1410108410 2024-05-10T11:22:36.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.803+0000 7f140d7fa700 1 -- 172.21.15.100:0/750364364 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1408004020 con 0x7f1410108410 2024-05-10T11:22:36.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.805+0000 7f140d7fa700 1 -- 172.21.15.100:0/750364364 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f140801b030 con 0x7f1410108410 2024-05-10T11:22:36.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.806+0000 7f140d7fa700 1 --2- 172.21.15.100:0/750364364 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f13f80b2900 0x7f13f80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:36.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.806+0000 7f140ffff700 1 --2- 172.21.15.100:0/750364364 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f13f80b2900 0x7f13f80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:36.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.807+0000 7f140d7fa700 1 -- 172.21.15.100:0/750364364 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f14080e1ff0 con 0x7f1410108410 2024-05-10T11:22:36.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.807+0000 7f1420c98b80 1 Processor -- start 2024-05-10T11:22:36.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.807+0000 7f1420c98b80 1 -- start start 2024-05-10T11:22:36.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.807+0000 7f140ffff700 1 --2- 172.21.15.100:0/750364364 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f13f80b2900 0x7f13f80b4dc0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f141019a9f0 tx=0x7f14000073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:36.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.807+0000 7f1420c98b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e6ab86200 0x556e6ab865e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:36.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.807+0000 7f1420c98b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e6ab86b20 0x556e6ab8a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:36.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.807+0000 7f1420c98b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556e6ab8aaa0 0x556e6ab8af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:36.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.807+0000 7f1420c98b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x556e6aacf940 con 0x556e6ab86200 2024-05-10T11:22:36.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.807+0000 7f1420c98b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556e6aa34da0 con 0x556e6ab86b20 2024-05-10T11:22:36.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.807+0000 7f1420c98b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556e6aacb900 con 0x556e6ab8aaa0 2024-05-10T11:22:36.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.807+0000 7f140ffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e6ab86200 0x556e6ab865e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:36.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.807+0000 7f140ffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e6ab86200 0x556e6ab865e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40278/0 (socket says 172.21.15.100:40278) 2024-05-10T11:22:36.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.807+0000 7f140ffff700 1 -- 172.21.15.100:0/3442291491 learned_addr learned my addr 172.21.15.100:0/3442291491 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:22:36.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.808+0000 7f14149a2700 1 --2- 172.21.15.100:0/3442291491 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556e6ab8aaa0 0x556e6ab8af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:36.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.808+0000 7f140f7fe700 1 --2- 172.21.15.100:0/3442291491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e6ab86b20 0x556e6ab8a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:36.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.808+0000 7f140ffff700 1 -- 172.21.15.100:0/3442291491 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556e6ab8aaa0 msgr2=0x556e6ab8af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:36.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.808+0000 7f140ffff700 1 --2- 172.21.15.100:0/3442291491 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556e6ab8aaa0 0x556e6ab8af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:36.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.808+0000 7f140ffff700 1 -- 172.21.15.100:0/3442291491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e6ab86b20 msgr2=0x556e6ab8a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:36.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.808+0000 7f140ffff700 1 --2- 172.21.15.100:0/3442291491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e6ab86b20 0x556e6ab8a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:36.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.808+0000 7f140ffff700 1 -- 172.21.15.100:0/3442291491 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e6aa1c8d0 con 0x556e6ab86200 2024-05-10T11:22:36.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.808+0000 7f140ffff700 1 --2- 172.21.15.100:0/3442291491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e6ab86200 0x556e6ab865e0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f14000076a0 tx=0x7f1400013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adea41f42579ac1d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:36.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.808+0000 7f13f67fc700 1 -- 172.21.15.100:0/3442291491 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1400011bb0 con 0x556e6ab86200 2024-05-10T11:22:36.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.808+0000 7f13f67fc700 1 -- 172.21.15.100:0/3442291491 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1400011d50 con 0x556e6ab86200 2024-05-10T11:22:36.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.808+0000 7f13f67fc700 1 -- 172.21.15.100:0/3442291491 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f140001c610 con 0x556e6ab86200 2024-05-10T11:22:36.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.809+0000 7f1420c98b80 1 -- 172.21.15.100:0/3442291491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e6ab86200 msgr2=0x556e6ab865e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:36.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.809+0000 7f1420c98b80 1 --2- 172.21.15.100:0/3442291491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e6ab86200 0x556e6ab865e0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f14000076a0 tx=0x7f1400013000 comp rx=0 tx=0).stop 2024-05-10T11:22:36.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.809+0000 7f1420c98b80 1 -- 172.21.15.100:0/3442291491 shutdown_connections 2024-05-10T11:22:36.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.809+0000 7f1420c98b80 1 --2- 172.21.15.100:0/3442291491 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556e6ab8aaa0 0x556e6ab8af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:36.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.809+0000 7f1420c98b80 1 --2- 172.21.15.100:0/3442291491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e6ab86200 0x556e6ab865e0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:36.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.809+0000 7f1420c98b80 1 --2- 172.21.15.100:0/3442291491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e6ab86b20 0x556e6ab8a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:36.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.809+0000 7f1420c98b80 1 -- 172.21.15.100:0/3442291491 >> 172.21.15.100:0/3442291491 conn(0x556e6aaa9b70 msgr2=0x556e6ab782a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:36.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.809+0000 7f1420c98b80 1 -- 172.21.15.100:0/3442291491 shutdown_connections 2024-05-10T11:22:36.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.809+0000 7f1420c98b80 1 -- 172.21.15.100:0/3442291491 wait complete. 2024-05-10T11:22:36.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.810+0000 7f1420c98b80 1 -- start start 2024-05-10T11:22:36.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.810+0000 7f1420c98b80 1 Processor -- start 2024-05-10T11:22:36.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.810+0000 7f1420c98b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556e6ab86200 0x556e6ab91ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:36.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.810+0000 7f1420c98b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e6ab86b20 0x556e6ab8f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:36.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.810+0000 7f1420c98b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e6ab8aaa0 0x556e6ab8f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:36.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.810+0000 7f1420c98b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x556e6ab92c00 con 0x556e6ab86b20 2024-05-10T11:22:36.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.810+0000 7f1420c98b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556e6aa88ed0 con 0x556e6ab8aaa0 2024-05-10T11:22:36.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.810+0000 7f1420c98b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556e6ab92d80 con 0x556e6ab86200 2024-05-10T11:22:36.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.810+0000 7f1420c98b80 10 client.0 _ll_register_callbacks cb 0x556e6aa635b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:22:36.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.810+0000 7f140f7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e6ab86b20 0x556e6ab8f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:36.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.810+0000 7f140f7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e6ab86b20 0x556e6ab8f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40284/0 (socket says 172.21.15.100:40284) 2024-05-10T11:22:36.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.810+0000 7f140f7fe700 1 -- 172.21.15.100:0/1744178689 learned_addr learned my addr 172.21.15.100:0/1744178689 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:22:36.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.811+0000 7f14149a2700 1 --2- 172.21.15.100:0/1744178689 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e6ab8aaa0 0x556e6ab8f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:36.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.811+0000 7f140f7fe700 1 -- 172.21.15.100:0/1744178689 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556e6ab86200 msgr2=0x556e6ab91ea0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T11:22:36.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.811+0000 7f140f7fe700 1 --2- 172.21.15.100:0/1744178689 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556e6ab86200 0x556e6ab91ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:36.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.811+0000 7f140f7fe700 1 -- 172.21.15.100:0/1744178689 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e6ab8aaa0 msgr2=0x556e6ab8f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:36.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.811+0000 7f140f7fe700 1 --2- 172.21.15.100:0/1744178689 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e6ab8aaa0 0x556e6ab8f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:36.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.811+0000 7f140f7fe700 1 -- 172.21.15.100:0/1744178689 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e6aa86760 con 0x556e6ab86b20 2024-05-10T11:22:36.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.811+0000 7f140f7fe700 1 --2- 172.21.15.100:0/1744178689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e6ab86b20 0x556e6ab8f290 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f140400b650 tx=0x7f14040052e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2f7d83c2f4e8dd2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:36.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.811+0000 7f13ef7fe700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:22:36.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.811+0000 7f13ef7fe700 1 -- 172.21.15.100:0/1744178689 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1404003880 con 0x556e6ab86b20 2024-05-10T11:22:36.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.811+0000 7f13ef7fe700 1 -- 172.21.15.100:0/1744178689 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f1404003a20 con 0x556e6ab86b20 2024-05-10T11:22:36.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.811+0000 7f1420c98b80 1 -- 172.21.15.100:0/1744178689 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1400011d50 con 0x556e6ab86b20 2024-05-10T11:22:36.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.811+0000 7f13ef7fe700 1 -- 172.21.15.100:0/1744178689 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1404031050 con 0x556e6ab86b20 2024-05-10T11:22:36.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.811+0000 7f1420c98b80 10 client.16182 fetch_fsmap learned FSMap version 29 2024-05-10T11:22:36.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.812+0000 7f1420c98b80 1 -- 172.21.15.100:0/1744178689 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556e6ab9d460 con 0x556e6ab86b20 2024-05-10T11:22:36.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.812+0000 7f13ef7fe700 1 -- 172.21.15.100:0/1744178689 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1400011d50 con 0x556e6ab86b20 2024-05-10T11:22:36.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.812+0000 7f13ef7fe700 1 -- 172.21.15.100:0/1744178689 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f140402c030 con 0x556e6ab86b20 2024-05-10T11:22:36.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.812+0000 7f1420c98b80 10 client.16182 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T11:22:36.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.812+0000 7f1420c98b80 10 client.16182 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:22:36.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.812+0000 7f1420c98b80 20 client.16182 populate_metadata read hostname 'smithi100' 2024-05-10T11:22:36.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.812+0000 7f1420c98b80 1 --2- 172.21.15.100:0/1744178689 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556e6ab9faa0 0x556e6aba1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:36.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.812+0000 7f1420c98b80 4 client.16182 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:22:36.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.812+0000 7f1420c98b80 1 -- 172.21.15.100:0/1744178689 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e6aadd320 con 0x556e6ab9faa0 2024-05-10T11:22:36.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.812+0000 7f140ffff700 1 --2- 172.21.15.100:0/1744178689 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556e6ab9faa0 0x556e6aba1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:36.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.813+0000 7f13ef7fe700 10 client.16182 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:22:36.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.813+0000 7f140ffff700 1 --2- 172.21.15.100:0/1744178689 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556e6ab9faa0 0x556e6aba1f10 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b95cc7560edfae2 server_cookie=d769bd2b60615e86 in_seq=0 out_seq=0 2024-05-10T11:22:36.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.813+0000 7f13ef7fe700 1 -- 172.21.15.100:0/1744178689 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556e6aadd320 con 0x556e6ab9faa0 2024-05-10T11:22:36.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.813+0000 7f13ef7fe700 10 client.16182 handle_command_reply: tid=0 2024-05-10T11:22:36.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.814+0000 7f1420c98b80 1 client.16182 shutdown 2024-05-10T11:22:36.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.814+0000 7f1420c98b80 1 -- 172.21.15.100:0/1744178689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e6ab86b20 msgr2=0x556e6ab8f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:36.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.814+0000 7f1420c98b80 1 --2- 172.21.15.100:0/1744178689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e6ab86b20 0x556e6ab8f290 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f140400b650 tx=0x7f14040052e0 comp rx=0 tx=0).stop 2024-05-10T11:22:36.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.814+0000 7f1420c98b80 1 -- 172.21.15.100:0/1744178689 shutdown_connections 2024-05-10T11:22:36.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.814+0000 7f1420c98b80 1 --2- 172.21.15.100:0/1744178689 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556e6ab9faa0 0x556e6aba1f10 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:36.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.814+0000 7f13ef7fe700 0 client.16182 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:22:36.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.814+0000 7f1420c98b80 1 --2- 172.21.15.100:0/1744178689 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e6ab8aaa0 0x556e6ab8f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:36.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.814+0000 7f1420c98b80 1 --2- 172.21.15.100:0/1744178689 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556e6ab86200 0x556e6ab91ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:36.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.814+0000 7f1420c98b80 1 --2- 172.21.15.100:0/1744178689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e6ab86b20 0x556e6ab8f290 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:36.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.814+0000 7f1420c98b80 1 -- 172.21.15.100:0/1744178689 >> 172.21.15.100:0/1744178689 conn(0x556e6aaa9b70 msgr2=0x556e6ab72b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:36.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.814+0000 7f1420c98b80 1 -- 172.21.15.100:0/1744178689 shutdown_connections 2024-05-10T11:22:36.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.814+0000 7f1420c98b80 1 -- 172.21.15.100:0/1744178689 wait complete. 2024-05-10T11:22:36.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.815+0000 7f1420c98b80 20 client.16182 trim_cache size 0 max 16384 2024-05-10T11:22:36.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.840+0000 7f1420c98b80 1 Processor -- start 2024-05-10T11:22:36.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.840+0000 7f1420c98b80 1 -- start start 2024-05-10T11:22:36.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.841+0000 7f1420c98b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556e6ab86200 0x556e6ab9bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:36.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.841+0000 7f1420c98b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e6ab86b20 0x556e6ab72c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:36.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.841+0000 7f1420c98b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e6ab8aaa0 0x556e6ab7cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:36.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.841+0000 7f1420c98b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f14101af740 con 0x556e6ab86b20 2024-05-10T11:22:36.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.841+0000 7f1420c98b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f14101acfd0 con 0x556e6ab8aaa0 2024-05-10T11:22:36.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.841+0000 7f1420c98b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f14101aae40 con 0x556e6ab86200 2024-05-10T11:22:36.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.841+0000 7f140ffff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556e6ab86200 0x556e6ab9bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:36.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.841+0000 7f14149a2700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e6ab8aaa0 0x556e6ab7cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:36.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.841+0000 7f140f7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e6ab86b20 0x556e6ab72c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:36.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.841+0000 7f140ffff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556e6ab86200 0x556e6ab9bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:57350/0 (socket says 172.21.15.100:57350) 2024-05-10T11:22:36.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.841+0000 7f140ffff700 1 -- 172.21.15.100:0/500294419 learned_addr learned my addr 172.21.15.100:0/500294419 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:22:36.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.841+0000 7f14149a2700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e6ab8aaa0 0x556e6ab7cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:44638/0 (socket says 172.21.15.100:44638) 2024-05-10T11:22:36.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.841+0000 7f140f7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e6ab86b20 0x556e6ab72c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40298/0 (socket says 172.21.15.100:40298) 2024-05-10T11:22:36.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.841+0000 7f140ffff700 1 -- 172.21.15.100:0/500294419 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e6ab8aaa0 msgr2=0x556e6ab7cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:36.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.841+0000 7f140ffff700 1 --2- 172.21.15.100:0/500294419 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e6ab8aaa0 0x556e6ab7cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:36.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.841+0000 7f140ffff700 1 -- 172.21.15.100:0/500294419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e6ab86b20 msgr2=0x556e6ab72c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:36.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.841+0000 7f140ffff700 1 --2- 172.21.15.100:0/500294419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e6ab86b20 0x556e6ab72c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:36.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.841+0000 7f140ffff700 1 -- 172.21.15.100:0/500294419 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e6ab790d0 con 0x556e6ab86200 2024-05-10T11:22:36.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.841+0000 7f140ffff700 1 --2- 172.21.15.100:0/500294419 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556e6ab86200 0x556e6ab9bcf0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f14000106b0 tx=0x7f1400007460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6af79b289ffa492c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:36.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.842+0000 7f13f5ffb700 1 -- 172.21.15.100:0/500294419 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1400011db0 con 0x556e6ab86200 2024-05-10T11:22:36.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.842+0000 7f13f5ffb700 1 -- 172.21.15.100:0/500294419 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1400005570 con 0x556e6ab86200 2024-05-10T11:22:36.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.842+0000 7f13f5ffb700 1 -- 172.21.15.100:0/500294419 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1400031aa0 con 0x556e6ab86200 2024-05-10T11:22:36.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.843+0000 7f1420c98b80 1 -- 172.21.15.100:0/500294419 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556e6ab86200 msgr2=0x556e6ab9bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:36.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.843+0000 7f1420c98b80 1 --2- 172.21.15.100:0/500294419 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556e6ab86200 0x556e6ab9bcf0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f14000106b0 tx=0x7f1400007460 comp rx=0 tx=0).stop 2024-05-10T11:22:36.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.843+0000 7f1420c98b80 1 -- 172.21.15.100:0/500294419 shutdown_connections 2024-05-10T11:22:36.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.843+0000 7f1420c98b80 1 --2- 172.21.15.100:0/500294419 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e6ab8aaa0 0x556e6ab7cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:36.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.843+0000 7f1420c98b80 1 --2- 172.21.15.100:0/500294419 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556e6ab86200 0x556e6ab9bcf0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:36.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.843+0000 7f1420c98b80 1 --2- 172.21.15.100:0/500294419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e6ab86b20 0x556e6ab72c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:36.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.843+0000 7f1420c98b80 1 -- 172.21.15.100:0/500294419 >> 172.21.15.100:0/500294419 conn(0x556e6aaa9b70 msgr2=0x556e6ab7c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:36.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.843+0000 7f1420c98b80 1 -- 172.21.15.100:0/500294419 shutdown_connections 2024-05-10T11:22:36.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.843+0000 7f1420c98b80 1 -- 172.21.15.100:0/500294419 wait complete. 2024-05-10T11:22:36.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.843+0000 7f1420c98b80 1 -- start start 2024-05-10T11:22:36.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.843+0000 7f1420c98b80 1 Processor -- start 2024-05-10T11:22:36.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.844+0000 7f1420c98b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e6ab86200 0x556e6ab8c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:36.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.844+0000 7f1420c98b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e6ab86b20 0x556e6ab8c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:36.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.844+0000 7f1420c98b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556e6ab8aaa0 0x556e6ab96760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:36.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.844+0000 7f1420c98b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f14101ab490 con 0x556e6ab86200 2024-05-10T11:22:36.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.844+0000 7f1420c98b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f14101ac900 con 0x556e6ab86b20 2024-05-10T11:22:36.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.844+0000 7f1420c98b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f14101abb60 con 0x556e6ab8aaa0 2024-05-10T11:22:36.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.844+0000 7f1420c98b80 10 client.0 _ll_register_callbacks cb 0x556e6aa635b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:22:36.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.844+0000 7f140ffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e6ab86200 0x556e6ab8c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:36.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.844+0000 7f140f7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e6ab86b20 0x556e6ab8c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:36.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.844+0000 7f14149a2700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556e6ab8aaa0 0x556e6ab96760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:36.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.844+0000 7f140f7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e6ab86b20 0x556e6ab8c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:44648/0 (socket says 172.21.15.100:44648) 2024-05-10T11:22:36.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.844+0000 7f140f7fe700 1 -- 172.21.15.100:0/2945563098 learned_addr learned my addr 172.21.15.100:0/2945563098 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:22:36.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.844+0000 7f14149a2700 1 -- 172.21.15.100:0/2945563098 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e6ab86b20 msgr2=0x556e6ab8c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:36.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.844+0000 7f14149a2700 1 --2- 172.21.15.100:0/2945563098 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e6ab86b20 0x556e6ab8c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:36.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.844+0000 7f14149a2700 1 -- 172.21.15.100:0/2945563098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e6ab86200 msgr2=0x556e6ab8c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:36.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.844+0000 7f14149a2700 1 --2- 172.21.15.100:0/2945563098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e6ab86200 0x556e6ab8c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:36.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.844+0000 7f14149a2700 1 -- 172.21.15.100:0/2945563098 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e6ab96ca0 con 0x556e6ab8aaa0 2024-05-10T11:22:36.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.844+0000 7f140f7fe700 1 --2- 172.21.15.100:0/2945563098 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e6ab86b20 0x556e6ab8c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:22:36.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.844+0000 7f14149a2700 1 --2- 172.21.15.100:0/2945563098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556e6ab8aaa0 0x556e6ab96760 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f14080df7e0 tx=0x7f14080b1f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49ae19efe39aa18d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:36.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.844+0000 7f13ef7fe700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:22:36.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.845+0000 7f13ef7fe700 1 -- 172.21.15.100:0/2945563098 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14080c2020 con 0x556e6ab8aaa0 2024-05-10T11:22:36.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.845+0000 7f1420c98b80 1 -- 172.21.15.100:0/2945563098 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1400005570 con 0x556e6ab8aaa0 2024-05-10T11:22:36.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.845+0000 7f13ef7fe700 1 -- 172.21.15.100:0/2945563098 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f14080a4ec0 con 0x556e6ab8aaa0 2024-05-10T11:22:36.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.845+0000 7f13ef7fe700 1 -- 172.21.15.100:0/2945563098 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f14080a8e50 con 0x556e6ab8aaa0 2024-05-10T11:22:36.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.845+0000 7f1420c98b80 10 client.25684 fetch_fsmap learned FSMap version 29 2024-05-10T11:22:36.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.845+0000 7f1420c98b80 1 -- 172.21.15.100:0/2945563098 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556e6ac0ab20 con 0x556e6ab8aaa0 2024-05-10T11:22:36.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.845+0000 7f13ef7fe700 1 -- 172.21.15.100:0/2945563098 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1400005570 con 0x556e6ab8aaa0 2024-05-10T11:22:36.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.845+0000 7f13ef7fe700 1 -- 172.21.15.100:0/2945563098 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f14080a8010 con 0x556e6ab8aaa0 2024-05-10T11:22:36.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.845+0000 7f1420c98b80 10 client.25684 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T11:22:36.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.845+0000 7f1420c98b80 10 client.25684 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:22:36.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.845+0000 7f1420c98b80 20 client.25684 populate_metadata read hostname 'smithi100' 2024-05-10T11:22:36.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.845+0000 7f1420c98b80 1 --2- 172.21.15.100:0/2945563098 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556e6ac0b240 0x556e6ac0d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:36.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.845+0000 7f1420c98b80 4 client.25684 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T11:22:36.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.845+0000 7f1420c98b80 1 -- 172.21.15.100:0/2945563098 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556e6aad73c0 con 0x556e6ac0b240 2024-05-10T11:22:36.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.845+0000 7f140ffff700 1 --2- 172.21.15.100:0/2945563098 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556e6ac0b240 0x556e6ac0d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:36.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.846+0000 7f140ffff700 1 --2- 172.21.15.100:0/2945563098 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556e6ac0b240 0x556e6ac0d660 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da01bbf2e7b11d5a server_cookie=342dde108de55b17 in_seq=0 out_seq=0 2024-05-10T11:22:36.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.846+0000 7f13ef7fe700 10 client.25684 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:22:36.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.846+0000 7f13ef7fe700 1 -- 172.21.15.100:0/2945563098 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556e6aad73c0 con 0x556e6ac0b240 2024-05-10T11:22:36.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.846+0000 7f13ef7fe700 10 client.25684 handle_command_reply: tid=0 2024-05-10T11:22:36.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.846+0000 7f1420c98b80 1 client.25684 shutdown 2024-05-10T11:22:36.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.847+0000 7f1420c98b80 1 -- 172.21.15.100:0/2945563098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556e6ab8aaa0 msgr2=0x556e6ab96760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:36.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.847+0000 7f1420c98b80 1 --2- 172.21.15.100:0/2945563098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556e6ab8aaa0 0x556e6ab96760 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f14080df7e0 tx=0x7f14080b1f60 comp rx=0 tx=0).stop 2024-05-10T11:22:36.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.847+0000 7f1420c98b80 1 -- 172.21.15.100:0/2945563098 shutdown_connections 2024-05-10T11:22:36.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.847+0000 7f1420c98b80 1 --2- 172.21.15.100:0/2945563098 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556e6ac0b240 0x556e6ac0d660 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:36.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.847+0000 7f1420c98b80 1 --2- 172.21.15.100:0/2945563098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556e6ab8aaa0 0x556e6ab96760 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:36.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.847+0000 7f13ef7fe700 0 client.25684 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:22:36.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.847+0000 7f1420c98b80 1 --2- 172.21.15.100:0/2945563098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e6ab86200 0x556e6ab8c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:36.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.847+0000 7f1420c98b80 1 --2- 172.21.15.100:0/2945563098 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e6ab86b20 0x556e6ab8c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:36.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.847+0000 7f1420c98b80 1 -- 172.21.15.100:0/2945563098 >> 172.21.15.100:0/2945563098 conn(0x556e6aaa9b70 msgr2=0x556e6ab7d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:36.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.847+0000 7f1420c98b80 1 -- 172.21.15.100:0/2945563098 shutdown_connections 2024-05-10T11:22:36.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.847+0000 7f1420c98b80 1 -- 172.21.15.100:0/2945563098 wait complete. 2024-05-10T11:22:36.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.847+0000 7f1420c98b80 20 client.25684 trim_cache size 0 max 16384 2024-05-10T11:22:36.858 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:22:36.858 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-05-10T11:22:36.859 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-05-10T11:22:36.859 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:22:36.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.847+0000 7f1416405700 1 -- 172.21.15.100:0/750364364 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f13f80b2900 msgr2=0x7f13f80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:36.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.847+0000 7f1416405700 1 --2- 172.21.15.100:0/750364364 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f13f80b2900 0x7f13f80b4dc0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f141019a9f0 tx=0x7f14000073d0 comp rx=0 tx=0).stop 2024-05-10T11:22:36.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.847+0000 7f1416405700 1 -- 172.21.15.100:0/750364364 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1410108410 msgr2=0x7f14101998b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:36.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.847+0000 7f1416405700 1 --2- 172.21.15.100:0/750364364 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1410108410 0x7f14101998b0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f140800efc0 tx=0x7f140800c3a0 comp rx=0 tx=0).stop 2024-05-10T11:22:36.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.848+0000 7f1416405700 1 -- 172.21.15.100:0/750364364 shutdown_connections 2024-05-10T11:22:36.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.848+0000 7f1416405700 1 --2- 172.21.15.100:0/750364364 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f13f80b2900 0x7f13f80b4dc0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:36.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.848+0000 7f1416405700 1 --2- 172.21.15.100:0/750364364 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1410108410 0x7f14101998b0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:36.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.848+0000 7f1416405700 1 --2- 172.21.15.100:0/750364364 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14101023a0 0x7f141019f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:36.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.848+0000 7f1416405700 1 --2- 172.21.15.100:0/750364364 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1410102d60 0x7f141019fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:36.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.848+0000 7f1416405700 1 -- 172.21.15.100:0/750364364 >> 172.21.15.100:0/750364364 conn(0x7f1410079350 msgr2=0x7f14100ffe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:36.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.848+0000 7f1416405700 1 -- 172.21.15.100:0/750364364 shutdown_connections 2024-05-10T11:22:36.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:36.848+0000 7f1416405700 1 -- 172.21.15.100:0/750364364 wait complete. 2024-05-10T11:22:36.936 INFO:tasks.workunit.client.0.smithi100.stdout: 1 79699 2.66 MB/sec execute 31 sec latency 18.166 ms 2024-05-10T11:22:37.758 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T11:22:37.759 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T11:22:37.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:22:37 smithi188 ceph-mon[34075]: pgmap v547: 129 pgs: 129 active+clean; 48 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 1.4 MiB/s wr, 169 op/s 2024-05-10T11:22:37.786 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:22:37 smithi100 ceph-mon[30296]: pgmap v547: 129 pgs: 129 active+clean; 48 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 1.4 MiB/s wr, 169 op/s 2024-05-10T11:22:37.936 INFO:tasks.workunit.client.0.smithi100.stdout: 1 80384 2.63 MB/sec execute 32 sec latency 15.021 ms 2024-05-10T11:22:38.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:22:37 smithi178 ceph-mon[33359]: pgmap v547: 129 pgs: 129 active+clean; 48 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 1.4 MiB/s wr, 169 op/s 2024-05-10T11:22:38.036 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:22:38.936 INFO:tasks.workunit.client.0.smithi100.stdout: 1 80984 2.70 MB/sec execute 33 sec latency 15.880 ms 2024-05-10T11:22:39.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:22:39 smithi100 ceph-mon[30296]: pgmap v548: 129 pgs: 129 active+clean; 48 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 1.3 MiB/s wr, 133 op/s 2024-05-10T11:22:39.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:22:39 smithi188 ceph-mon[34075]: pgmap v548: 129 pgs: 129 active+clean; 48 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 1.3 MiB/s wr, 133 op/s 2024-05-10T11:22:39.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:39.861+0000 7f6f7ae4b700 1 -- 172.21.15.100:0/4273449595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f74102d50 msgr2=0x7f6f7410f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:39.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:39.861+0000 7f6f7ae4b700 1 --2- 172.21.15.100:0/4273449595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f74102d50 0x7f6f7410f240 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c00b0a0 tx=0x7f6f6c00b3b0 comp rx=0 tx=0).stop 2024-05-10T11:22:39.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:39.861+0000 7f6f7ae4b700 1 -- 172.21.15.100:0/4273449595 shutdown_connections 2024-05-10T11:22:39.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:39.861+0000 7f6f7ae4b700 1 --2- 172.21.15.100:0/4273449595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f74102d50 0x7f6f7410f240 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:39.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:39.861+0000 7f6f7ae4b700 1 --2- 172.21.15.100:0/4273449595 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f74108400 0x7f6f741087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:39.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:39.861+0000 7f6f7ae4b700 1 --2- 172.21.15.100:0/4273449595 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f74102390 0x7f6f74102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:39.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:39.861+0000 7f6f7ae4b700 1 -- 172.21.15.100:0/4273449595 >> 172.21.15.100:0/4273449595 conn(0x7f6f74079310 msgr2=0x7f6f74079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:39.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:39.861+0000 7f6f7ae4b700 1 -- 172.21.15.100:0/4273449595 shutdown_connections 2024-05-10T11:22:39.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:39.861+0000 7f6f7ae4b700 1 -- 172.21.15.100:0/4273449595 wait complete. 2024-05-10T11:22:39.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:39.862+0000 7f6f7ae4b700 1 Processor -- start 2024-05-10T11:22:39.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:39.862+0000 7f6f7ae4b700 1 -- start start 2024-05-10T11:22:39.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:39.862+0000 7f6f7ae4b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f74102390 0x7f6f741986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:39.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:39.862+0000 7f6f7ae4b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f74102d50 0x7f6f74198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:39.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:39.862+0000 7f6f7ae4b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f74108400 0x7f6f74199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:39.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:39.862+0000 7f6f7ae4b700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6f741123d0 con 0x7f6f74108400 2024-05-10T11:22:39.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:39.862+0000 7f6f7ae4b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6f74112250 con 0x7f6f74102d50 2024-05-10T11:22:39.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:39.862+0000 7f6f7ae4b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6f74112550 con 0x7f6f74102390 2024-05-10T11:22:39.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:39.862+0000 7f6f78be7700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f74102390 0x7f6f741986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:39.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:39.862+0000 7f6f73fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f74102d50 0x7f6f74198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:39.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:39.862+0000 7f6f78be7700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f74102390 0x7f6f741986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:57370/0 (socket says 172.21.15.100:57370) 2024-05-10T11:22:39.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:39.862+0000 7f6f78be7700 1 -- 172.21.15.100:0/188686070 learned_addr learned my addr 172.21.15.100:0/188686070 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:22:39.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:39.862+0000 7f6f793e8700 1 --2- 172.21.15.100:0/188686070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f74108400 0x7f6f74199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:39.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:39.862+0000 7f6f73fff700 1 -- 172.21.15.100:0/188686070 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f74102390 msgr2=0x7f6f741986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:39.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:39.862+0000 7f6f73fff700 1 --2- 172.21.15.100:0/188686070 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f74102390 0x7f6f741986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:39.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:39.862+0000 7f6f73fff700 1 -- 172.21.15.100:0/188686070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f74108400 msgr2=0x7f6f74199140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:39.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:39.862+0000 7f6f73fff700 1 --2- 172.21.15.100:0/188686070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f74108400 0x7f6f74199140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:39.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:39.862+0000 7f6f73fff700 1 -- 172.21.15.100:0/188686070 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f741a7f30 con 0x7f6f74102d50 2024-05-10T11:22:39.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:39.862+0000 7f6f78be7700 1 --2- 172.21.15.100:0/188686070 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f74102390 0x7f6f741986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:22:39.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:39.863+0000 7f6f73fff700 1 --2- 172.21.15.100:0/188686070 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f74102d50 0x7f6f74198c00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6f6000e9f0 tx=0x7f6f6000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:39.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:39.863+0000 7f6f71ffb700 1 -- 172.21.15.100:0/188686070 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f6000c9c0 con 0x7f6f74102d50 2024-05-10T11:22:39.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:39.863+0000 7f6f71ffb700 1 -- 172.21.15.100:0/188686070 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6f60010430 con 0x7f6f74102d50 2024-05-10T11:22:39.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:39.863+0000 7f6f7ae4b700 1 -- 172.21.15.100:0/188686070 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f741a81c0 con 0x7f6f74102d50 2024-05-10T11:22:39.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:39.863+0000 7f6f71ffb700 1 -- 172.21.15.100:0/188686070 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f60018680 con 0x7f6f74102d50 2024-05-10T11:22:39.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:39.863+0000 7f6f7ae4b700 1 -- 172.21.15.100:0/188686070 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f741a8700 con 0x7f6f74102d50 2024-05-10T11:22:39.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:39.864+0000 7f6f7ae4b700 1 -- 172.21.15.100:0/188686070 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f7406a190 con 0x7f6f74102d50 2024-05-10T11:22:39.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:39.865+0000 7f6f71ffb700 1 -- 172.21.15.100:0/188686070 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f6f6000cb60 con 0x7f6f74102d50 2024-05-10T11:22:39.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:39.866+0000 7f6f71ffb700 1 --2- 172.21.15.100:0/188686070 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f6f640b2880 0x7f6f640b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:39.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:39.866+0000 7f6f78be7700 1 --2- 172.21.15.100:0/188686070 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f6f640b2880 0x7f6f640b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:39.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:39.866+0000 7f6f71ffb700 1 -- 172.21.15.100:0/188686070 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f6f600e2650 con 0x7f6f74102d50 2024-05-10T11:22:39.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:39.867+0000 7f6f78be7700 1 --2- 172.21.15.100:0/188686070 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f6f640b2880 0x7f6f640b4d40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6f68011550 tx=0x7f6f68009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:39.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:39.869+0000 7f6f71ffb700 1 -- 172.21.15.100:0/188686070 <== mon.2 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f6f600aa010 con 0x7f6f74102d50 2024-05-10T11:22:39.935 INFO:tasks.workunit.client.0.smithi100.stdout: 1 81476 2.70 MB/sec execute 34 sec latency 19.938 ms 2024-05-10T11:22:40.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:22:39 smithi178 ceph-mon[33359]: pgmap v548: 129 pgs: 129 active+clean; 48 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 1.3 MiB/s wr, 133 op/s 2024-05-10T11:22:40.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:40.169+0000 7f6f7ae4b700 1 -- 172.21.15.100:0/188686070 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6f7402e510 con 0x7f6f74102d50 2024-05-10T11:22:40.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:40.170+0000 7f6f71ffb700 1 -- 172.21.15.100:0/188686070 <== mon.2 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11897 (secure 0 0 0) 0x7f6f600b2090 con 0x7f6f74102d50 2024-05-10T11:22:40.170 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:22:40.170 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2072225012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2072225012},{"type":"v1","addr":"172.21.15.100:6835","nonce":2072225012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/401939540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":401939540},{"type":"v1","addr":"172.21.15.100:6837","nonce":401939540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14571,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/3946311025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3946311025},{"type":"v1","addr":"172.21.15.100:6839","nonce":3946311025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3736371660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3736371660},{"type":"v1","addr":"172.21.15.100:6841","nonce":3736371660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/2741810791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2741810791},{"type":"v1","addr":"172.21.15.178:6833","nonce":2741810791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1704891041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1704891041},{"type":"v1","addr":"172.21.15.188:6833","nonce":1704891041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T11:14:27.242452+0000","modified":"2024-05-10T11:20:29.528470+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24439,"mds_1":24437,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/2124076312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2124076312},{"type":"v1","addr":"172.21.15.178:6835","nonce":2124076312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/3224358425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3224358425},{"type":"v1","addr":"172.21.15.178:6837","nonce":3224358425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/515547690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":515547690},{"type":"v1","addr":"172.21.15.188:6835","nonce":515547690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/1943342620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1943342620},{"type":"v1","addr":"172.21.15.188:6837","nonce":1943342620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.178:6839/3760985404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3760985404},{"type":"v1","addr":"172.21.15.178:6839","nonce":3760985404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.188:6839/3194917070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":3194917070},{"type":"v1","addr":"172.21.15.188:6839","nonce":3194917070}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24439,"qdb_cluster":[24437,24439,24427]},"id":1}]} 2024-05-10T11:22:40.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:40.172+0000 7f6f7ae4b700 1 -- 172.21.15.100:0/188686070 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f6f640b2880 msgr2=0x7f6f640b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:40.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:40.172+0000 7f6f7ae4b700 1 --2- 172.21.15.100:0/188686070 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f6f640b2880 0x7f6f640b4d40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6f68011550 tx=0x7f6f68009260 comp rx=0 tx=0).stop 2024-05-10T11:22:40.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:40.172+0000 7f6f7ae4b700 1 -- 172.21.15.100:0/188686070 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f74102d50 msgr2=0x7f6f74198c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:40.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:40.172+0000 7f6f7ae4b700 1 --2- 172.21.15.100:0/188686070 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f74102d50 0x7f6f74198c00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6f6000e9f0 tx=0x7f6f6000ed00 comp rx=0 tx=0).stop 2024-05-10T11:22:40.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:40.172+0000 7f6f7ae4b700 1 -- 172.21.15.100:0/188686070 shutdown_connections 2024-05-10T11:22:40.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:40.172+0000 7f6f7ae4b700 1 --2- 172.21.15.100:0/188686070 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f6f640b2880 0x7f6f640b4d40 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:40.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:40.172+0000 7f6f7ae4b700 1 --2- 172.21.15.100:0/188686070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f74108400 0x7f6f74199140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:40.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:40.172+0000 7f6f7ae4b700 1 --2- 172.21.15.100:0/188686070 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f74102390 0x7f6f741986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:40.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:40.172+0000 7f6f7ae4b700 1 --2- 172.21.15.100:0/188686070 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f74102d50 0x7f6f74198c00 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:40.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:40.172+0000 7f6f7ae4b700 1 -- 172.21.15.100:0/188686070 >> 172.21.15.100:0/188686070 conn(0x7f6f74079310 msgr2=0x7f6f74101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:40.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:40.173+0000 7f6f7ae4b700 1 -- 172.21.15.100:0/188686070 shutdown_connections 2024-05-10T11:22:40.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:40.173+0000 7f6f7ae4b700 1 -- 172.21.15.100:0/188686070 wait complete. 2024-05-10T11:22:40.173 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 29 2024-05-10T11:22:40.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:22:40 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/188686070' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:22:40.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:22:40 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/188686070' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:22:40.912 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-10T11:22:40.936 INFO:tasks.workunit.client.0.smithi100.stdout: 1 81958 2.75 MB/sec execute 35 sec latency 14.791 ms 2024-05-10T11:22:41.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:22:40 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/188686070' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:22:41.194 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:22:41.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:22:41 smithi100 ceph-mon[30296]: pgmap v549: 129 pgs: 129 active+clean; 47 MiB data, 822 MiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 1.2 MiB/s wr, 146 op/s 2024-05-10T11:22:41.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:22:41 smithi188 ceph-mon[34075]: pgmap v549: 129 pgs: 129 active+clean; 47 MiB data, 822 MiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 1.2 MiB/s wr, 146 op/s 2024-05-10T11:22:41.936 INFO:tasks.workunit.client.0.smithi100.stdout: 1 82257 2.68 MB/sec execute 36 sec latency 13.317 ms 2024-05-10T11:22:42.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:22:41 smithi178 ceph-mon[33359]: pgmap v549: 129 pgs: 129 active+clean; 47 MiB data, 822 MiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 1.2 MiB/s wr, 146 op/s 2024-05-10T11:22:42.936 INFO:tasks.workunit.client.0.smithi100.stdout: 1 82811 2.66 MB/sec execute 37 sec latency 19.755 ms 2024-05-10T11:22:43.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.149+0000 7f2c62d36700 1 -- 172.21.15.100:0/2279559128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c5c109520 msgr2=0x7f2c5c109900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:43.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.149+0000 7f2c62d36700 1 --2- 172.21.15.100:0/2279559128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c5c109520 0x7f2c5c109900 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f2c50009990 tx=0x7f2c50009ca0 comp rx=0 tx=0).stop 2024-05-10T11:22:43.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.150+0000 7f2c62d36700 1 -- 172.21.15.100:0/2279559128 shutdown_connections 2024-05-10T11:22:43.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.150+0000 7f2c62d36700 1 --2- 172.21.15.100:0/2279559128 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c5c102450 0x7f2c5c110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:43.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.150+0000 7f2c62d36700 1 --2- 172.21.15.100:0/2279559128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c5c109520 0x7f2c5c109900 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:43.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.150+0000 7f2c62d36700 1 --2- 172.21.15.100:0/2279559128 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c5c101a90 0x7f2c5c101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:43.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.150+0000 7f2c62d36700 1 -- 172.21.15.100:0/2279559128 >> 172.21.15.100:0/2279559128 conn(0x7f2c5c078a10 msgr2=0x7f2c5c0ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:43.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.150+0000 7f2c62d36700 1 -- 172.21.15.100:0/2279559128 shutdown_connections 2024-05-10T11:22:43.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.150+0000 7f2c62d36700 1 -- 172.21.15.100:0/2279559128 wait complete. 2024-05-10T11:22:43.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.151+0000 7f2c62d36700 1 Processor -- start 2024-05-10T11:22:43.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.151+0000 7f2c62d36700 1 -- start start 2024-05-10T11:22:43.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.151+0000 7f2c62d36700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c5c101a90 0x7f2c5c19f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:43.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.151+0000 7f2c62d36700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c5c102450 0x7f2c5c19fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:43.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.152+0000 7f2c62d36700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c5c109520 0x7f2c5c199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:43.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.152+0000 7f2c62d36700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5c113500 con 0x7f2c5c109520 2024-05-10T11:22:43.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.152+0000 7f2c62d36700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5c113380 con 0x7f2c5c101a90 2024-05-10T11:22:43.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.152+0000 7f2c62d36700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5c113680 con 0x7f2c5c102450 2024-05-10T11:22:43.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.152+0000 7f2c5bfff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c5c102450 0x7f2c5c19fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:43.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.152+0000 7f2c60ad2700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c5c101a90 0x7f2c5c19f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:43.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.152+0000 7f2c60ad2700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c5c101a90 0x7f2c5c19f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:44682/0 (socket says 172.21.15.100:44682) 2024-05-10T11:22:43.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.152+0000 7f2c60ad2700 1 -- 172.21.15.100:0/612857130 learned_addr learned my addr 172.21.15.100:0/612857130 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:22:43.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.152+0000 7f2c5bfff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c5c102450 0x7f2c5c19fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:57398/0 (socket says 172.21.15.100:57398) 2024-05-10T11:22:43.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.152+0000 7f2c612d3700 1 --2- 172.21.15.100:0/612857130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c5c109520 0x7f2c5c199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:43.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.152+0000 7f2c60ad2700 1 -- 172.21.15.100:0/612857130 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c5c102450 msgr2=0x7f2c5c19fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:43.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.152+0000 7f2c60ad2700 1 --2- 172.21.15.100:0/612857130 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c5c102450 0x7f2c5c19fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:43.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.152+0000 7f2c60ad2700 1 -- 172.21.15.100:0/612857130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c5c109520 msgr2=0x7f2c5c199860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:43.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.152+0000 7f2c60ad2700 1 --2- 172.21.15.100:0/612857130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c5c109520 0x7f2c5c199860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:43.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.152+0000 7f2c60ad2700 1 -- 172.21.15.100:0/612857130 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c5c199fc0 con 0x7f2c5c101a90 2024-05-10T11:22:43.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.152+0000 7f2c5bfff700 1 --2- 172.21.15.100:0/612857130 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c5c102450 0x7f2c5c19fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:22:43.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.152+0000 7f2c60ad2700 1 --2- 172.21.15.100:0/612857130 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c5c101a90 0x7f2c5c19f6a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2c50006010 tx=0x7f2c5000bfc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:43.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.153+0000 7f2c59ffb700 1 -- 172.21.15.100:0/612857130 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c50027070 con 0x7f2c5c101a90 2024-05-10T11:22:43.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.153+0000 7f2c59ffb700 1 -- 172.21.15.100:0/612857130 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2c50022470 con 0x7f2c5c101a90 2024-05-10T11:22:43.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.153+0000 7f2c59ffb700 1 -- 172.21.15.100:0/612857130 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c50017650 con 0x7f2c5c101a90 2024-05-10T11:22:43.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.153+0000 7f2c62d36700 1 -- 172.21.15.100:0/612857130 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c5c19a250 con 0x7f2c5c101a90 2024-05-10T11:22:43.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.153+0000 7f2c62d36700 1 -- 172.21.15.100:0/612857130 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c5c1a6570 con 0x7f2c5c101a90 2024-05-10T11:22:43.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.155+0000 7f2c59ffb700 1 -- 172.21.15.100:0/612857130 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f2c50009d10 con 0x7f2c5c101a90 2024-05-10T11:22:43.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.156+0000 7f2c59ffb700 1 --2- 172.21.15.100:0/612857130 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2c4c0b2880 0x7f2c4c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:43.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.156+0000 7f2c5bfff700 1 --2- 172.21.15.100:0/612857130 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2c4c0b2880 0x7f2c4c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:43.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.156+0000 7f2c59ffb700 1 -- 172.21.15.100:0/612857130 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f2c500e3130 con 0x7f2c5c101a90 2024-05-10T11:22:43.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.157+0000 7f2c5bfff700 1 --2- 172.21.15.100:0/612857130 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2c4c0b2880 0x7f2c4c0b4d40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c0ffa70 tx=0x7f2c48009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:43.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.157+0000 7f2c6d5c9b80 1 Processor -- start 2024-05-10T11:22:43.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.157+0000 7f2c6d5c9b80 1 -- start start 2024-05-10T11:22:43.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.157+0000 7f2c6d5c9b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56326eb101b0 0x56326eb10590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:43.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.157+0000 7f2c6d5c9b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56326eb10ad0 0x56326eb0a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:43.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.157+0000 7f2c6d5c9b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56326eb0ad50 0x56326eb0b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:43.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.157+0000 7f2c6d5c9b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56326ea5c940 con 0x56326eb10ad0 2024-05-10T11:22:43.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.157+0000 7f2c6d5c9b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56326e9c1da0 con 0x56326eb0ad50 2024-05-10T11:22:43.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.157+0000 7f2c6d5c9b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56326ea58900 con 0x56326eb101b0 2024-05-10T11:22:43.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.157+0000 7f2c60ad2700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56326eb101b0 0x56326eb10590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:43.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.157+0000 7f2c5bfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56326eb10ad0 0x56326eb0a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:43.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.157+0000 7f2c60ad2700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56326eb101b0 0x56326eb10590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:57408/0 (socket says 172.21.15.100:57408) 2024-05-10T11:22:43.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.157+0000 7f2c60ad2700 1 -- 172.21.15.100:0/4278273627 learned_addr learned my addr 172.21.15.100:0/4278273627 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:22:43.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.157+0000 7f2c612d3700 1 --2- 172.21.15.100:0/4278273627 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56326eb0ad50 0x56326eb0b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:43.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.157+0000 7f2c60ad2700 1 -- 172.21.15.100:0/4278273627 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56326eb0ad50 msgr2=0x56326eb0b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:43.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.157+0000 7f2c60ad2700 1 --2- 172.21.15.100:0/4278273627 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56326eb0ad50 0x56326eb0b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:43.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.157+0000 7f2c60ad2700 1 -- 172.21.15.100:0/4278273627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56326eb10ad0 msgr2=0x56326eb0a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:43.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.157+0000 7f2c60ad2700 1 --2- 172.21.15.100:0/4278273627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56326eb10ad0 0x56326eb0a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:43.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.158+0000 7f2c60ad2700 1 -- 172.21.15.100:0/4278273627 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56326e9a98d0 con 0x56326eb101b0 2024-05-10T11:22:43.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.158+0000 7f2c60ad2700 1 --2- 172.21.15.100:0/4278273627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56326eb101b0 0x56326eb10590 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2c500a6450 tx=0x7f2c500a9fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5a6c950fb3dffa5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:43.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.158+0000 7f2c46ffd700 1 -- 172.21.15.100:0/4278273627 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c500ae7d0 con 0x56326eb101b0 2024-05-10T11:22:43.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.158+0000 7f2c46ffd700 1 -- 172.21.15.100:0/4278273627 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2c500b31f0 con 0x56326eb101b0 2024-05-10T11:22:43.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.158+0000 7f2c46ffd700 1 -- 172.21.15.100:0/4278273627 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c500b2d60 con 0x56326eb101b0 2024-05-10T11:22:43.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.159+0000 7f2c6d5c9b80 1 -- 172.21.15.100:0/4278273627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56326eb101b0 msgr2=0x56326eb10590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:43.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.159+0000 7f2c6d5c9b80 1 --2- 172.21.15.100:0/4278273627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56326eb101b0 0x56326eb10590 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2c500a6450 tx=0x7f2c500a9fb0 comp rx=0 tx=0).stop 2024-05-10T11:22:43.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.159+0000 7f2c6d5c9b80 1 -- 172.21.15.100:0/4278273627 shutdown_connections 2024-05-10T11:22:43.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.159+0000 7f2c6d5c9b80 1 --2- 172.21.15.100:0/4278273627 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56326eb0ad50 0x56326eb0b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:43.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.159+0000 7f2c6d5c9b80 1 --2- 172.21.15.100:0/4278273627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56326eb101b0 0x56326eb10590 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:43.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.159+0000 7f2c6d5c9b80 1 --2- 172.21.15.100:0/4278273627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56326eb10ad0 0x56326eb0a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:43.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.159+0000 7f2c6d5c9b80 1 -- 172.21.15.100:0/4278273627 >> 172.21.15.100:0/4278273627 conn(0x56326ea36b70 msgr2=0x56326eb052a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:43.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.159+0000 7f2c6d5c9b80 1 -- 172.21.15.100:0/4278273627 shutdown_connections 2024-05-10T11:22:43.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.159+0000 7f2c6d5c9b80 1 -- 172.21.15.100:0/4278273627 wait complete. 2024-05-10T11:22:43.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.159+0000 7f2c6d5c9b80 1 -- start start 2024-05-10T11:22:43.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.159+0000 7f2c6d5c9b80 1 Processor -- start 2024-05-10T11:22:43.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.160+0000 7f2c6d5c9b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56326eb0ad50 0x56326eb166c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:43.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.160+0000 7f2c6d5c9b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56326eb101b0 0x56326eb13a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:43.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.160+0000 7f2c6d5c9b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56326eb10ad0 0x56326eb13fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:43.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.160+0000 7f2c60ad2700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56326eb0ad50 0x56326eb166c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:43.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.160+0000 7f2c60ad2700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56326eb0ad50 0x56326eb166c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:57418/0 (socket says 172.21.15.100:57418) 2024-05-10T11:22:43.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.160+0000 7f2c60ad2700 1 -- 172.21.15.100:0/2744834694 learned_addr learned my addr 172.21.15.100:0/2744834694 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:22:43.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.160+0000 7f2c6d5c9b80 1 -- 172.21.15.100:0/2744834694 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56326eb1b620 con 0x56326eb10ad0 2024-05-10T11:22:43.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.160+0000 7f2c6d5c9b80 1 -- 172.21.15.100:0/2744834694 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56326ea15ed0 con 0x56326eb101b0 2024-05-10T11:22:43.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.160+0000 7f2c6d5c9b80 1 -- 172.21.15.100:0/2744834694 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56326eb1b7a0 con 0x56326eb0ad50 2024-05-10T11:22:43.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.160+0000 7f2c5bfff700 1 --2- 172.21.15.100:0/2744834694 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56326eb101b0 0x56326eb13a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:43.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.160+0000 7f2c6d5c9b80 10 client.0 _ll_register_callbacks cb 0x56326e9f05b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:22:43.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.160+0000 7f2c612d3700 1 --2- 172.21.15.100:0/2744834694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56326eb10ad0 0x56326eb13fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:43.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.160+0000 7f2c60ad2700 1 -- 172.21.15.100:0/2744834694 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56326eb101b0 msgr2=0x56326eb13a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:43.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.160+0000 7f2c60ad2700 1 --2- 172.21.15.100:0/2744834694 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56326eb101b0 0x56326eb13a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:43.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.160+0000 7f2c60ad2700 1 -- 172.21.15.100:0/2744834694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56326eb10ad0 msgr2=0x56326eb13fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:43.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.160+0000 7f2c60ad2700 1 --2- 172.21.15.100:0/2744834694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56326eb10ad0 0x56326eb13fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:43.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.160+0000 7f2c60ad2700 1 -- 172.21.15.100:0/2744834694 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56326ea13760 con 0x56326eb0ad50 2024-05-10T11:22:43.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.161+0000 7f2c60ad2700 1 --2- 172.21.15.100:0/2744834694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56326eb0ad50 0x56326eb166c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2c5000b2e0 tx=0x7f2c500aa960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd0b98db949592b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:43.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.161+0000 7f2c3bfff700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:22:43.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.161+0000 7f2c3bfff700 1 -- 172.21.15.100:0/2744834694 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c500c3020 con 0x56326eb0ad50 2024-05-10T11:22:43.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.161+0000 7f2c3bfff700 1 -- 172.21.15.100:0/2744834694 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f2c500b2f70 con 0x56326eb0ad50 2024-05-10T11:22:43.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.161+0000 7f2c6d5c9b80 1 -- 172.21.15.100:0/2744834694 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c500b31f0 con 0x56326eb0ad50 2024-05-10T11:22:43.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.161+0000 7f2c3bfff700 1 -- 172.21.15.100:0/2744834694 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2c500a4b40 con 0x56326eb0ad50 2024-05-10T11:22:43.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.161+0000 7f2c6d5c9b80 10 client.25702 fetch_fsmap learned FSMap version 29 2024-05-10T11:22:43.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.161+0000 7f2c6d5c9b80 1 -- 172.21.15.100:0/2744834694 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56326eb1b620 con 0x56326eb0ad50 2024-05-10T11:22:43.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.161+0000 7f2c3bfff700 1 -- 172.21.15.100:0/2744834694 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2c500b31f0 con 0x56326eb0ad50 2024-05-10T11:22:43.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.161+0000 7f2c3bfff700 1 -- 172.21.15.100:0/2744834694 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f2c500b1ac0 con 0x56326eb0ad50 2024-05-10T11:22:43.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.161+0000 7f2c6d5c9b80 10 client.25702 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T11:22:43.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.161+0000 7f2c6d5c9b80 10 client.25702 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-10T11:22:43.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.161+0000 7f2c6d5c9b80 20 client.25702 populate_metadata read hostname 'smithi100' 2024-05-10T11:22:43.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.161+0000 7f2c6d5c9b80 1 --2- 172.21.15.100:0/2744834694 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x56326eb280c0 0x56326eb2a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:43.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.162+0000 7f2c6d5c9b80 4 client.25702 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:22:43.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.162+0000 7f2c6d5c9b80 1 -- 172.21.15.100:0/2744834694 --> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56326ea6a320 con 0x56326eb280c0 2024-05-10T11:22:43.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.162+0000 7f2c5bfff700 1 --2- 172.21.15.100:0/2744834694 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x56326eb280c0 0x56326eb2a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:43.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.162+0000 7f2c3bfff700 10 client.25702 ms_handle_connect on v2:172.21.15.188:6836/1943342620 2024-05-10T11:22:43.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.162+0000 7f2c5bfff700 1 --2- 172.21.15.100:0/2744834694 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x56326eb280c0 0x56326eb2a4b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=63382f8aee934b8d server_cookie=34f2b608014f6fec in_seq=0 out_seq=0 2024-05-10T11:22:43.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.163+0000 7f2c3bfff700 1 -- 172.21.15.100:0/2744834694 <== mds.2 v2:172.21.15.188:6836/1943342620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56326ea6a320 con 0x56326eb280c0 2024-05-10T11:22:43.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.163+0000 7f2c3bfff700 10 client.25702 handle_command_reply: tid=0 2024-05-10T11:22:43.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.163+0000 7f2c6d5c9b80 1 client.25702 shutdown 2024-05-10T11:22:43.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.163+0000 7f2c6d5c9b80 1 -- 172.21.15.100:0/2744834694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56326eb0ad50 msgr2=0x56326eb166c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:43.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.163+0000 7f2c6d5c9b80 1 --2- 172.21.15.100:0/2744834694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56326eb0ad50 0x56326eb166c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2c5000b2e0 tx=0x7f2c500aa960 comp rx=0 tx=0).stop 2024-05-10T11:22:43.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.163+0000 7f2c6d5c9b80 1 -- 172.21.15.100:0/2744834694 shutdown_connections 2024-05-10T11:22:43.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.163+0000 7f2c6d5c9b80 1 --2- 172.21.15.100:0/2744834694 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x56326eb280c0 0x56326eb2a4b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:43.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.163+0000 7f2c6d5c9b80 1 --2- 172.21.15.100:0/2744834694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56326eb10ad0 0x56326eb13fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:43.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.163+0000 7f2c6d5c9b80 1 --2- 172.21.15.100:0/2744834694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56326eb0ad50 0x56326eb166c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:43.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.163+0000 7f2c6d5c9b80 1 --2- 172.21.15.100:0/2744834694 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56326eb101b0 0x56326eb13a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:43.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.163+0000 7f2c6d5c9b80 1 -- 172.21.15.100:0/2744834694 >> 172.21.15.100:0/2744834694 conn(0x56326ea36b70 msgr2=0x56326eaffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:43.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.163+0000 7f2c3bfff700 0 client.25702 ms_handle_reset on v2:172.21.15.188:6836/1943342620 2024-05-10T11:22:43.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.163+0000 7f2c6d5c9b80 1 -- 172.21.15.100:0/2744834694 shutdown_connections 2024-05-10T11:22:43.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.163+0000 7f2c6d5c9b80 1 -- 172.21.15.100:0/2744834694 wait complete. 2024-05-10T11:22:43.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.163+0000 7f2c6d5c9b80 20 client.25702 trim_cache size 0 max 16384 2024-05-10T11:22:43.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.178+0000 7f2c6d5c9b80 1 Processor -- start 2024-05-10T11:22:43.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.178+0000 7f2c6d5c9b80 1 -- start start 2024-05-10T11:22:43.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.179+0000 7f2c6d5c9b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56326eb0ad50 0x56326eb0a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:43.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.179+0000 7f2c6d5c9b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56326eb101b0 0x56326eb24670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:43.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.179+0000 7f2c6d5c9b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56326eb10ad0 0x56326eb115f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:43.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.179+0000 7f2c6d5c9b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5c1af610 con 0x56326eb0ad50 2024-05-10T11:22:43.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.179+0000 7f2c6d5c9b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5c1acea0 con 0x56326eb101b0 2024-05-10T11:22:43.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.179+0000 7f2c6d5c9b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5c1aad10 con 0x56326eb10ad0 2024-05-10T11:22:43.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.179+0000 7f2c60ad2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56326eb0ad50 0x56326eb0a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:43.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.179+0000 7f2c60ad2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56326eb0ad50 0x56326eb0a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40366/0 (socket says 172.21.15.100:40366) 2024-05-10T11:22:43.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.179+0000 7f2c60ad2700 1 -- 172.21.15.100:0/3765614570 learned_addr learned my addr 172.21.15.100:0/3765614570 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:22:43.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.179+0000 7f2c612d3700 1 --2- 172.21.15.100:0/3765614570 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56326eb10ad0 0x56326eb115f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:43.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.179+0000 7f2c5bfff700 1 --2- 172.21.15.100:0/3765614570 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56326eb101b0 0x56326eb24670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:43.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.179+0000 7f2c60ad2700 1 -- 172.21.15.100:0/3765614570 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56326eb10ad0 msgr2=0x56326eb115f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:43.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.179+0000 7f2c60ad2700 1 --2- 172.21.15.100:0/3765614570 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56326eb10ad0 0x56326eb115f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:43.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.179+0000 7f2c60ad2700 1 -- 172.21.15.100:0/3765614570 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56326eb101b0 msgr2=0x56326eb24670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:43.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.179+0000 7f2c60ad2700 1 --2- 172.21.15.100:0/3765614570 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56326eb101b0 0x56326eb24670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:43.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.179+0000 7f2c60ad2700 1 -- 172.21.15.100:0/3765614570 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56326eb0b650 con 0x56326eb0ad50 2024-05-10T11:22:43.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.179+0000 7f2c5bfff700 1 --2- 172.21.15.100:0/3765614570 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56326eb101b0 0x56326eb24670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:22:43.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.179+0000 7f2c60ad2700 1 --2- 172.21.15.100:0/3765614570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56326eb0ad50 0x56326eb0a810 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f2c500a6450 tx=0x7f2c500a6360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb5fb984059defba server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:43.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.179+0000 7f2c467fc700 1 -- 172.21.15.100:0/3765614570 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c500c3070 con 0x56326eb0ad50 2024-05-10T11:22:43.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.180+0000 7f2c467fc700 1 -- 172.21.15.100:0/3765614570 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2c500cb3e0 con 0x56326eb0ad50 2024-05-10T11:22:43.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.180+0000 7f2c467fc700 1 -- 172.21.15.100:0/3765614570 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c500b1ac0 con 0x56326eb0ad50 2024-05-10T11:22:43.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.180+0000 7f2c6d5c9b80 1 -- 172.21.15.100:0/3765614570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56326eb0ad50 msgr2=0x56326eb0a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:43.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.180+0000 7f2c6d5c9b80 1 --2- 172.21.15.100:0/3765614570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56326eb0ad50 0x56326eb0a810 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f2c500a6450 tx=0x7f2c500a6360 comp rx=0 tx=0).stop 2024-05-10T11:22:43.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.180+0000 7f2c6d5c9b80 1 -- 172.21.15.100:0/3765614570 shutdown_connections 2024-05-10T11:22:43.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.180+0000 7f2c6d5c9b80 1 --2- 172.21.15.100:0/3765614570 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56326eb10ad0 0x56326eb115f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:43.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.180+0000 7f2c6d5c9b80 1 --2- 172.21.15.100:0/3765614570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56326eb0ad50 0x56326eb0a810 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:43.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.180+0000 7f2c6d5c9b80 1 --2- 172.21.15.100:0/3765614570 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56326eb101b0 0x56326eb24670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:43.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.180+0000 7f2c6d5c9b80 1 -- 172.21.15.100:0/3765614570 >> 172.21.15.100:0/3765614570 conn(0x56326ea36b70 msgr2=0x56326eb10ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:43.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.180+0000 7f2c6d5c9b80 1 -- 172.21.15.100:0/3765614570 shutdown_connections 2024-05-10T11:22:43.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.180+0000 7f2c6d5c9b80 1 -- 172.21.15.100:0/3765614570 wait complete. 2024-05-10T11:22:43.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.181+0000 7f2c6d5c9b80 1 -- start start 2024-05-10T11:22:43.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.181+0000 7f2c6d5c9b80 1 Processor -- start 2024-05-10T11:22:43.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.181+0000 7f2c6d5c9b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56326eb0ad50 0x56326eb30290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:43.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.181+0000 7f2c6d5c9b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56326eb101b0 0x56326eb209f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:43.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.181+0000 7f2c6d5c9b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56326eb10ad0 0x56326eb20fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:43.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.181+0000 7f2c6d5c9b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5c1ab360 con 0x56326eb0ad50 2024-05-10T11:22:43.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.181+0000 7f2c6d5c9b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5c1ac7d0 con 0x56326eb101b0 2024-05-10T11:22:43.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.181+0000 7f2c6d5c9b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5c1aba30 con 0x56326eb10ad0 2024-05-10T11:22:43.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.181+0000 7f2c60ad2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56326eb0ad50 0x56326eb30290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:43.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.181+0000 7f2c6d5c9b80 10 client.0 _ll_register_callbacks cb 0x56326e9f05b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:22:43.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.181+0000 7f2c60ad2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56326eb0ad50 0x56326eb30290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40380/0 (socket says 172.21.15.100:40380) 2024-05-10T11:22:43.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.181+0000 7f2c5bfff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56326eb101b0 0x56326eb209f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:43.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.181+0000 7f2c60ad2700 1 -- 172.21.15.100:0/296190074 learned_addr learned my addr 172.21.15.100:0/296190074 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:22:43.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.181+0000 7f2c612d3700 1 --2- 172.21.15.100:0/296190074 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56326eb10ad0 0x56326eb20fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:43.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.182+0000 7f2c60ad2700 1 -- 172.21.15.100:0/296190074 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56326eb10ad0 msgr2=0x56326eb20fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:43.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.182+0000 7f2c60ad2700 1 --2- 172.21.15.100:0/296190074 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56326eb10ad0 0x56326eb20fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:43.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.182+0000 7f2c60ad2700 1 -- 172.21.15.100:0/296190074 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56326eb101b0 msgr2=0x56326eb209f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:43.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.182+0000 7f2c60ad2700 1 --2- 172.21.15.100:0/296190074 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56326eb101b0 0x56326eb209f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:43.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.182+0000 7f2c60ad2700 1 -- 172.21.15.100:0/296190074 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56326eb2ee20 con 0x56326eb0ad50 2024-05-10T11:22:43.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.182+0000 7f2c5bfff700 1 --2- 172.21.15.100:0/296190074 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56326eb101b0 0x56326eb209f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:22:43.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.182+0000 7f2c60ad2700 1 --2- 172.21.15.100:0/296190074 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56326eb0ad50 0x56326eb30290 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f2c500cb360 tx=0x7f2c500ae7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f48e834f9e7f8b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:43.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.182+0000 7f2c3bfff700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:22:43.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.182+0000 7f2c3bfff700 1 -- 172.21.15.100:0/296190074 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c500c3070 con 0x56326eb0ad50 2024-05-10T11:22:43.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.182+0000 7f2c3bfff700 1 -- 172.21.15.100:0/296190074 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f2c500a6a10 con 0x56326eb0ad50 2024-05-10T11:22:43.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.182+0000 7f2c6d5c9b80 1 -- 172.21.15.100:0/296190074 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c500cb3e0 con 0x56326eb0ad50 2024-05-10T11:22:43.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.182+0000 7f2c3bfff700 1 -- 172.21.15.100:0/296190074 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2c500b31f0 con 0x56326eb0ad50 2024-05-10T11:22:43.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.182+0000 7f2c6d5c9b80 10 client.16209 fetch_fsmap learned FSMap version 29 2024-05-10T11:22:43.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.182+0000 7f2c6d5c9b80 1 -- 172.21.15.100:0/296190074 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56326eb2ee20 con 0x56326eb0ad50 2024-05-10T11:22:43.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.183+0000 7f2c3bfff700 1 -- 172.21.15.100:0/296190074 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2c500cb3e0 con 0x56326eb0ad50 2024-05-10T11:22:43.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.183+0000 7f2c3bfff700 1 -- 172.21.15.100:0/296190074 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f2c500a4ce0 con 0x56326eb0ad50 2024-05-10T11:22:43.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.183+0000 7f2c6d5c9b80 10 client.16209 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T11:22:43.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.183+0000 7f2c6d5c9b80 10 client.16209 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-10T11:22:43.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.183+0000 7f2c6d5c9b80 20 client.16209 populate_metadata read hostname 'smithi100' 2024-05-10T11:22:43.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.183+0000 7f2c6d5c9b80 1 --2- 172.21.15.100:0/296190074 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x56326eb91e90 0x56326eb94280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:43.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.183+0000 7f2c6d5c9b80 4 client.16209 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:22:43.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.183+0000 7f2c6d5c9b80 1 -- 172.21.15.100:0/296190074 --> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56326ea643c0 con 0x56326eb91e90 2024-05-10T11:22:43.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.183+0000 7f2c5bfff700 1 --2- 172.21.15.100:0/296190074 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x56326eb91e90 0x56326eb94280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:43.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.183+0000 7f2c5bfff700 1 --2- 172.21.15.100:0/296190074 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x56326eb91e90 0x56326eb94280 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=95e76d1841bb77e4 server_cookie=5c7b5ca55c12ddcd in_seq=0 out_seq=0 2024-05-10T11:22:43.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.183+0000 7f2c3bfff700 10 client.16209 ms_handle_connect on v2:172.21.15.188:6836/1943342620 2024-05-10T11:22:43.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.183+0000 7f2c3bfff700 1 -- 172.21.15.100:0/296190074 <== mds.2 v2:172.21.15.188:6836/1943342620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56326ea643c0 con 0x56326eb91e90 2024-05-10T11:22:43.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.183+0000 7f2c3bfff700 10 client.16209 handle_command_reply: tid=0 2024-05-10T11:22:43.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.183+0000 7f2c6d5c9b80 1 client.16209 shutdown 2024-05-10T11:22:43.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.184+0000 7f2c6d5c9b80 1 -- 172.21.15.100:0/296190074 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56326eb0ad50 msgr2=0x56326eb30290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:43.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.184+0000 7f2c6d5c9b80 1 --2- 172.21.15.100:0/296190074 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56326eb0ad50 0x56326eb30290 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f2c500cb360 tx=0x7f2c500ae7d0 comp rx=0 tx=0).stop 2024-05-10T11:22:43.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.184+0000 7f2c6d5c9b80 1 -- 172.21.15.100:0/296190074 shutdown_connections 2024-05-10T11:22:43.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.184+0000 7f2c6d5c9b80 1 --2- 172.21.15.100:0/296190074 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x56326eb91e90 0x56326eb94280 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:43.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.184+0000 7f2c6d5c9b80 1 --2- 172.21.15.100:0/296190074 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56326eb10ad0 0x56326eb20fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:43.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.184+0000 7f2c3bfff700 0 client.16209 ms_handle_reset on v2:172.21.15.188:6836/1943342620 2024-05-10T11:22:43.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.184+0000 7f2c6d5c9b80 1 --2- 172.21.15.100:0/296190074 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56326eb0ad50 0x56326eb30290 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:43.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.184+0000 7f2c6d5c9b80 1 --2- 172.21.15.100:0/296190074 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56326eb101b0 0x56326eb209f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:43.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.184+0000 7f2c6d5c9b80 1 -- 172.21.15.100:0/296190074 >> 172.21.15.100:0/296190074 conn(0x56326ea36b70 msgr2=0x56326eb127e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:43.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.184+0000 7f2c6d5c9b80 1 -- 172.21.15.100:0/296190074 shutdown_connections 2024-05-10T11:22:43.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.184+0000 7f2c6d5c9b80 1 -- 172.21.15.100:0/296190074 wait complete. 2024-05-10T11:22:43.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.184+0000 7f2c6d5c9b80 20 client.16209 trim_cache size 0 max 16384 2024-05-10T11:22:43.189 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:22:43.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.185+0000 7f2c62d36700 1 -- 172.21.15.100:0/612857130 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2c4c0b2880 msgr2=0x7f2c4c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:43.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.185+0000 7f2c62d36700 1 --2- 172.21.15.100:0/612857130 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2c4c0b2880 0x7f2c4c0b4d40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c0ffa70 tx=0x7f2c48009260 comp rx=0 tx=0).stop 2024-05-10T11:22:43.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.185+0000 7f2c62d36700 1 -- 172.21.15.100:0/612857130 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c5c101a90 msgr2=0x7f2c5c19f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:43.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.185+0000 7f2c62d36700 1 --2- 172.21.15.100:0/612857130 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c5c101a90 0x7f2c5c19f6a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2c50006010 tx=0x7f2c5000bfc0 comp rx=0 tx=0).stop 2024-05-10T11:22:43.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.185+0000 7f2c62d36700 1 -- 172.21.15.100:0/612857130 shutdown_connections 2024-05-10T11:22:43.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.185+0000 7f2c62d36700 1 --2- 172.21.15.100:0/612857130 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2c4c0b2880 0x7f2c4c0b4d40 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:43.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.185+0000 7f2c62d36700 1 --2- 172.21.15.100:0/612857130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c5c109520 0x7f2c5c199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:43.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.185+0000 7f2c62d36700 1 --2- 172.21.15.100:0/612857130 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c5c101a90 0x7f2c5c19f6a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:43.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.185+0000 7f2c62d36700 1 --2- 172.21.15.100:0/612857130 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c5c102450 0x7f2c5c19fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:43.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.185+0000 7f2c62d36700 1 -- 172.21.15.100:0/612857130 >> 172.21.15.100:0/612857130 conn(0x7f2c5c078a10 msgr2=0x7f2c5c100250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:43.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.185+0000 7f2c62d36700 1 -- 172.21.15.100:0/612857130 shutdown_connections 2024-05-10T11:22:43.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:43.185+0000 7f2c62d36700 1 -- 172.21.15.100:0/612857130 wait complete. 2024-05-10T11:22:43.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:22:43 smithi188 ceph-mon[34075]: pgmap v550: 129 pgs: 129 active+clean; 48 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 1.3 MiB/s wr, 153 op/s 2024-05-10T11:22:43.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:22:43 smithi100 ceph-mon[30296]: pgmap v550: 129 pgs: 129 active+clean; 48 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 1.3 MiB/s wr, 153 op/s 2024-05-10T11:22:43.936 INFO:tasks.workunit.client.0.smithi100.stdout: 1 83444 2.68 MB/sec execute 38 sec latency 20.165 ms 2024-05-10T11:22:44.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:22:43 smithi178 ceph-mon[33359]: pgmap v550: 129 pgs: 129 active+clean; 48 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 1.3 MiB/s wr, 153 op/s 2024-05-10T11:22:44.086 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-10T11:22:44.360 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:22:44.936 INFO:tasks.workunit.client.0.smithi100.stdout: 1 84026 2.65 MB/sec execute 39 sec latency 70.881 ms 2024-05-10T11:22:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.429+0000 7fdd9b1b6640 1 -- 172.21.15.100:0/4165407812 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd94103490 msgr2=0x7fdd94105880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.429+0000 7fdd9b1b6640 1 --2- 172.21.15.100:0/4165407812 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd94103490 0x7fdd94105880 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fdd88009900 tx=0x7fdd8802a410 comp rx=0 tx=0).stop 2024-05-10T11:22:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.429+0000 7fdd9b1b6640 1 -- 172.21.15.100:0/4165407812 shutdown_connections 2024-05-10T11:22:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.429+0000 7fdd9b1b6640 1 --2- 172.21.15.100:0/4165407812 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd941086f0 0x7fdd9410ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.429+0000 7fdd9b1b6640 1 --2- 172.21.15.100:0/4165407812 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd94105dc0 0x7fdd941081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.429+0000 7fdd9b1b6640 1 --2- 172.21.15.100:0/4165407812 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd94103490 0x7fdd94105880 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.429+0000 7fdd9b1b6640 1 -- 172.21.15.100:0/4165407812 >> 172.21.15.100:0/4165407812 conn(0x7fdd940fce80 msgr2=0x7fdd940ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.429+0000 7fdd9b1b6640 1 -- 172.21.15.100:0/4165407812 shutdown_connections 2024-05-10T11:22:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.429+0000 7fdd9b1b6640 1 -- 172.21.15.100:0/4165407812 wait complete. 2024-05-10T11:22:45.432 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.429+0000 7fdd9b1b6640 1 Processor -- start 2024-05-10T11:22:45.432 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.430+0000 7fdd9b1b6640 1 -- start start 2024-05-10T11:22:45.432 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.430+0000 7fdd9b1b6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd94103490 0x7fdd94102e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:45.432 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.430+0000 7fdd9b1b6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd94105dc0 0x7fdd94101440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:45.432 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.430+0000 7fdd9b1b6640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd941086f0 0x7fdd94101980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:45.432 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.430+0000 7fdd9b1b6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdd9410e3f0 con 0x7fdd94103490 2024-05-10T11:22:45.432 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.430+0000 7fdd9b1b6640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdd9410e270 con 0x7fdd941086f0 2024-05-10T11:22:45.432 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.430+0000 7fdd9b1b6640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fdd9410e570 con 0x7fdd94105dc0 2024-05-10T11:22:45.432 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.430+0000 7fdd98f52640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd94103490 0x7fdd94102e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:45.432 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.430+0000 7fdd98f52640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd94103490 0x7fdd94102e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40402/0 (socket says 172.21.15.100:40402) 2024-05-10T11:22:45.432 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.430+0000 7fdd98f52640 1 -- 172.21.15.100:0/3946102367 learned_addr learned my addr 172.21.15.100:0/3946102367 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:22:45.432 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.430+0000 7fdd99753640 1 --2- 172.21.15.100:0/3946102367 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd941086f0 0x7fdd94101980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:45.432 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.430+0000 7fdd93fff640 1 --2- 172.21.15.100:0/3946102367 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd94105dc0 0x7fdd94101440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:45.432 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.430+0000 7fdd98f52640 1 -- 172.21.15.100:0/3946102367 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd94105dc0 msgr2=0x7fdd94101440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:45.433 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.430+0000 7fdd98f52640 1 --2- 172.21.15.100:0/3946102367 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd94105dc0 0x7fdd94101440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:45.433 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.430+0000 7fdd98f52640 1 -- 172.21.15.100:0/3946102367 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd941086f0 msgr2=0x7fdd94101980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:45.433 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.430+0000 7fdd98f52640 1 --2- 172.21.15.100:0/3946102367 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd941086f0 0x7fdd94101980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:45.433 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.430+0000 7fdd98f52640 1 -- 172.21.15.100:0/3946102367 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd94102210 con 0x7fdd94103490 2024-05-10T11:22:45.433 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.430+0000 7fdd98f52640 1 --2- 172.21.15.100:0/3946102367 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd94103490 0x7fdd94102e00 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fdd88009a20 tx=0x7fdd880040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:45.433 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.430+0000 7fdd91ffb640 1 -- 172.21.15.100:0/3946102367 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd8803f030 con 0x7fdd94103490 2024-05-10T11:22:45.433 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.430+0000 7fdd9b1b6640 1 -- 172.21.15.100:0/3946102367 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd941a2b90 con 0x7fdd94103490 2024-05-10T11:22:45.433 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.430+0000 7fdd91ffb640 1 -- 172.21.15.100:0/3946102367 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdd88003a80 con 0x7fdd94103490 2024-05-10T11:22:45.433 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.430+0000 7fdd91ffb640 1 -- 172.21.15.100:0/3946102367 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd88035070 con 0x7fdd94103490 2024-05-10T11:22:45.433 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.431+0000 7fdd9b1b6640 1 -- 172.21.15.100:0/3946102367 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd941a3130 con 0x7fdd94103490 2024-05-10T11:22:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.433+0000 7fdd9dfae740 1 Processor -- start 2024-05-10T11:22:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.433+0000 7fdd9dfae740 1 -- start start 2024-05-10T11:22:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.434+0000 7fdd91ffb640 1 -- 172.21.15.100:0/3946102367 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fdd88042030 con 0x7fdd94103490 2024-05-10T11:22:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.434+0000 7fdd9dfae740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56063f98edb0 0x56063f98f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.434+0000 7fdd9dfae740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56063f98f7a0 0x56063f9942a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.434+0000 7fdd9dfae740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56063f994910 0x56063f996da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.434+0000 7fdd9dfae740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56063f8f35d0 con 0x56063f994910 2024-05-10T11:22:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.434+0000 7fdd9dfae740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56063f8ac890 con 0x56063f98edb0 2024-05-10T11:22:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.434+0000 7fdd9dfae740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56063f8ef590 con 0x56063f98f7a0 2024-05-10T11:22:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.434+0000 7fdd93fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56063f994910 0x56063f996da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.434+0000 7fdd99753640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56063f98edb0 0x56063f98f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.434+0000 7fdd93fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56063f994910 0x56063f996da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40418/0 (socket says 172.21.15.100:40418) 2024-05-10T11:22:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.434+0000 7fdd93fff640 1 -- 172.21.15.100:0/1197126803 learned_addr learned my addr 172.21.15.100:0/1197126803 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:22:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.434+0000 7fdd98f52640 1 --2- 172.21.15.100:0/1197126803 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56063f98f7a0 0x56063f9942a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.434+0000 7fdd99753640 1 -- 172.21.15.100:0/1197126803 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56063f98f7a0 msgr2=0x56063f9942a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.434+0000 7fdd99753640 1 --2- 172.21.15.100:0/1197126803 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56063f98f7a0 0x56063f9942a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.434+0000 7fdd99753640 1 -- 172.21.15.100:0/1197126803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56063f994910 msgr2=0x56063f996da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.434+0000 7fdd99753640 1 --2- 172.21.15.100:0/1197126803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56063f994910 0x56063f996da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.434+0000 7fdd99753640 1 -- 172.21.15.100:0/1197126803 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56063f605210 con 0x56063f98edb0 2024-05-10T11:22:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.434+0000 7fdd99753640 1 --2- 172.21.15.100:0/1197126803 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56063f98edb0 0x56063f98f1d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c009f30 tx=0x7fdd8c010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4ab33c454fe5168 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.435+0000 7fdd91ffb640 1 --2- 172.21.15.100:0/3946102367 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fdd840b28a0 0x7fdd840b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.435+0000 7fdd7effd640 1 -- 172.21.15.100:0/1197126803 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd8c016070 con 0x56063f98edb0 2024-05-10T11:22:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.435+0000 7fdd98f52640 1 --2- 172.21.15.100:0/3946102367 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fdd840b28a0 0x7fdd840b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.435+0000 7fdd7effd640 1 -- 172.21.15.100:0/1197126803 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdd8c004d20 con 0x56063f98edb0 2024-05-10T11:22:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.435+0000 7fdd7effd640 1 -- 172.21.15.100:0/1197126803 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd8c0126b0 con 0x56063f98edb0 2024-05-10T11:22:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.435+0000 7fdd91ffb640 1 -- 172.21.15.100:0/3946102367 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fdd88102bb0 con 0x7fdd94103490 2024-05-10T11:22:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.436+0000 7fdd9dfae740 1 -- 172.21.15.100:0/1197126803 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56063f98edb0 msgr2=0x56063f98f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.436+0000 7fdd9dfae740 1 --2- 172.21.15.100:0/1197126803 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56063f98edb0 0x56063f98f1d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c009f30 tx=0x7fdd8c010800 comp rx=0 tx=0).stop 2024-05-10T11:22:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.436+0000 7fdd98f52640 1 --2- 172.21.15.100:0/3946102367 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fdd840b28a0 0x7fdd840b4d60 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fdd881006c0 tx=0x7fdd88106690 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.436+0000 7fdd9dfae740 1 -- 172.21.15.100:0/1197126803 shutdown_connections 2024-05-10T11:22:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.436+0000 7fdd9dfae740 1 --2- 172.21.15.100:0/1197126803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56063f994910 0x56063f996da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.436+0000 7fdd9dfae740 1 --2- 172.21.15.100:0/1197126803 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56063f98f7a0 0x56063f9942a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.436+0000 7fdd9dfae740 1 --2- 172.21.15.100:0/1197126803 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56063f98edb0 0x56063f98f1d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.436+0000 7fdd9dfae740 1 -- 172.21.15.100:0/1197126803 >> 172.21.15.100:0/1197126803 conn(0x56063f98a420 msgr2=0x56063f98a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.436+0000 7fdd9dfae740 1 -- 172.21.15.100:0/1197126803 shutdown_connections 2024-05-10T11:22:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.436+0000 7fdd9dfae740 1 -- 172.21.15.100:0/1197126803 wait complete. 2024-05-10T11:22:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.436+0000 7fdd9dfae740 1 -- start start 2024-05-10T11:22:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.436+0000 7fdd9dfae740 1 Processor -- start 2024-05-10T11:22:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.437+0000 7fdd9dfae740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56063f98edb0 0x56063f998ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.437+0000 7fdd9dfae740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56063f98f7a0 0x56063f999530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.437+0000 7fdd9dfae740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56063f994910 0x56063f9a0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.437+0000 7fdd9dfae740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56063f9978c0 con 0x56063f98f7a0 2024-05-10T11:22:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.437+0000 7fdd9dfae740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56063f8b0370 con 0x56063f98edb0 2024-05-10T11:22:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.437+0000 7fdd9dfae740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56063f997a40 con 0x56063f994910 2024-05-10T11:22:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.437+0000 7fdd9dfae740 10 client.0 _ll_register_callbacks cb 0x56063f604a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:22:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.437+0000 7fdd93fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56063f98f7a0 0x56063f999530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.437+0000 7fdd93fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56063f98f7a0 0x56063f999530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40428/0 (socket says 172.21.15.100:40428) 2024-05-10T11:22:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.437+0000 7fdd93fff640 1 -- 172.21.15.100:0/1860270341 learned_addr learned my addr 172.21.15.100:0/1860270341 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:22:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.437+0000 7fdd99753640 1 --2- 172.21.15.100:0/1860270341 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56063f98edb0 0x56063f998ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.437+0000 7fdd99753640 1 --2- 172.21.15.100:0/1860270341 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56063f994910 0x56063f9a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.437+0000 7fdd93fff640 1 -- 172.21.15.100:0/1860270341 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56063f994910 msgr2=0x56063f9a0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.437+0000 7fdd93fff640 1 --2- 172.21.15.100:0/1860270341 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56063f994910 0x56063f9a0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.437+0000 7fdd93fff640 1 -- 172.21.15.100:0/1860270341 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56063f98edb0 msgr2=0x56063f998ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.437+0000 7fdd93fff640 1 --2- 172.21.15.100:0/1860270341 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56063f98edb0 0x56063f998ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.437+0000 7fdd93fff640 1 -- 172.21.15.100:0/1860270341 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56063f6d1620 con 0x56063f98f7a0 2024-05-10T11:22:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.437+0000 7fdd99753640 1 --2- 172.21.15.100:0/1860270341 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56063f98edb0 0x56063f998ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:22:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.437+0000 7fdd93fff640 1 --2- 172.21.15.100:0/1860270341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56063f98f7a0 0x56063f999530 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fdd80012700 tx=0x7fdd80012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b1ba67ffc33050d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.437+0000 7fdd73fff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:22:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.437+0000 7fdd73fff640 1 -- 172.21.15.100:0/1860270341 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd80004260 con 0x56063f98f7a0 2024-05-10T11:22:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.437+0000 7fdd73fff640 1 -- 172.21.15.100:0/1860270341 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fdd80004400 con 0x56063f98f7a0 2024-05-10T11:22:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.437+0000 7fdd9dfae740 1 -- 172.21.15.100:0/1860270341 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd8c004d20 con 0x56063f98f7a0 2024-05-10T11:22:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.438+0000 7fdd73fff640 1 -- 172.21.15.100:0/1860270341 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdd80025050 con 0x56063f98f7a0 2024-05-10T11:22:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.438+0000 7fdd9dfae740 10 client.16221 fetch_fsmap learned FSMap version 29 2024-05-10T11:22:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.438+0000 7fdd9dfae740 1 -- 172.21.15.100:0/1860270341 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56063f9a3620 con 0x56063f98f7a0 2024-05-10T11:22:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.438+0000 7fdd73fff640 1 -- 172.21.15.100:0/1860270341 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdd8c004d20 con 0x56063f98f7a0 2024-05-10T11:22:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.438+0000 7fdd73fff640 1 -- 172.21.15.100:0/1860270341 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fdd80003bd0 con 0x56063f98f7a0 2024-05-10T11:22:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.438+0000 7fdd9dfae740 10 client.16221 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T11:22:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.438+0000 7fdd9dfae740 10 client.16221 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:22:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.438+0000 7fdd9dfae740 20 client.16221 populate_metadata read hostname 'smithi100' 2024-05-10T11:22:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.438+0000 7fdd9dfae740 1 --2- 172.21.15.100:0/1860270341 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56063f9a4d60 0x56063f9a71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.438+0000 7fdd9dfae740 4 client.16221 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:22:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.438+0000 7fdd9dfae740 1 -- 172.21.15.100:0/1860270341 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56063f9011e0 con 0x56063f9a4d60 2024-05-10T11:22:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.438+0000 7fdd99753640 1 --2- 172.21.15.100:0/1860270341 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56063f9a4d60 0x56063f9a71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.439+0000 7fdd73fff640 10 client.16221 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:22:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.439+0000 7fdd99753640 1 --2- 172.21.15.100:0/1860270341 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56063f9a4d60 0x56063f9a71a0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16f63533a0651497 server_cookie=204c56c066798f1 in_seq=0 out_seq=0 2024-05-10T11:22:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.440+0000 7fdd73fff640 1 -- 172.21.15.100:0/1860270341 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56063f9011e0 con 0x56063f9a4d60 2024-05-10T11:22:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.440+0000 7fdd73fff640 10 client.16221 handle_command_reply: tid=0 2024-05-10T11:22:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.440+0000 7fdd9dfae740 1 client.16221 shutdown 2024-05-10T11:22:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.440+0000 7fdd9dfae740 1 -- 172.21.15.100:0/1860270341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56063f98f7a0 msgr2=0x56063f999530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.440+0000 7fdd9dfae740 1 --2- 172.21.15.100:0/1860270341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56063f98f7a0 0x56063f999530 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fdd80012700 tx=0x7fdd80012bc0 comp rx=0 tx=0).stop 2024-05-10T11:22:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.440+0000 7fdd9dfae740 1 -- 172.21.15.100:0/1860270341 shutdown_connections 2024-05-10T11:22:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.440+0000 7fdd9dfae740 1 --2- 172.21.15.100:0/1860270341 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56063f9a4d60 0x56063f9a71a0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.440+0000 7fdd9dfae740 1 --2- 172.21.15.100:0/1860270341 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56063f994910 0x56063f9a0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.440+0000 7fdd9dfae740 1 --2- 172.21.15.100:0/1860270341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56063f98f7a0 0x56063f999530 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.440+0000 7fdd73fff640 0 client.16221 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:22:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.440+0000 7fdd9dfae740 1 --2- 172.21.15.100:0/1860270341 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56063f98edb0 0x56063f998ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.440+0000 7fdd9dfae740 1 -- 172.21.15.100:0/1860270341 >> 172.21.15.100:0/1860270341 conn(0x56063f98a420 msgr2=0x56063f989580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.440+0000 7fdd9dfae740 1 -- 172.21.15.100:0/1860270341 shutdown_connections 2024-05-10T11:22:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.440+0000 7fdd9dfae740 1 -- 172.21.15.100:0/1860270341 wait complete. 2024-05-10T11:22:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.440+0000 7fdd9dfae740 20 client.16221 trim_cache size 0 max 16384 2024-05-10T11:22:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.457+0000 7fdd9dfae740 1 Processor -- start 2024-05-10T11:22:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.457+0000 7fdd9dfae740 1 -- start start 2024-05-10T11:22:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.457+0000 7fdd9dfae740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56063f98edb0 0x56063f985430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.457+0000 7fdd9dfae740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56063f98f7a0 0x56063f9909c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.457+0000 7fdd9dfae740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56063f994910 0x56063f990f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.457+0000 7fdd9dfae740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdd941afd50 con 0x56063f98f7a0 2024-05-10T11:22:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.457+0000 7fdd9dfae740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdd941ad620 con 0x56063f994910 2024-05-10T11:22:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.457+0000 7fdd9dfae740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fdd941ab470 con 0x56063f98edb0 2024-05-10T11:22:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.457+0000 7fdd93fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56063f98f7a0 0x56063f9909c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.457+0000 7fdd99753640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56063f98edb0 0x56063f985430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.457+0000 7fdd93fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56063f98f7a0 0x56063f9909c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40440/0 (socket says 172.21.15.100:40440) 2024-05-10T11:22:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.457+0000 7fdd93fff640 1 -- 172.21.15.100:0/2896589349 learned_addr learned my addr 172.21.15.100:0/2896589349 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:22:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.458+0000 7fdd99753640 1 --2- 172.21.15.100:0/2896589349 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56063f994910 0x56063f990f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.458+0000 7fdd93fff640 1 -- 172.21.15.100:0/2896589349 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56063f98edb0 msgr2=0x56063f985430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.458+0000 7fdd93fff640 1 --2- 172.21.15.100:0/2896589349 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56063f98edb0 0x56063f985430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.458+0000 7fdd93fff640 1 -- 172.21.15.100:0/2896589349 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56063f994910 msgr2=0x56063f990f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.458+0000 7fdd93fff640 1 --2- 172.21.15.100:0/2896589349 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56063f994910 0x56063f990f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.458+0000 7fdd93fff640 1 -- 172.21.15.100:0/2896589349 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56063f9a7820 con 0x56063f98f7a0 2024-05-10T11:22:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.458+0000 7fdd99753640 1 --2- 172.21.15.100:0/2896589349 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56063f994910 0x56063f990f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:22:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.458+0000 7fdd93fff640 1 --2- 172.21.15.100:0/2896589349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56063f98f7a0 0x56063f9909c0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fdd8000f490 tx=0x7fdd8000f890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac63b6fc5b78d882 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.458+0000 7fdd7e7fc640 1 -- 172.21.15.100:0/2896589349 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd800052d0 con 0x56063f98f7a0 2024-05-10T11:22:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.458+0000 7fdd7e7fc640 1 -- 172.21.15.100:0/2896589349 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdd8002e070 con 0x56063f98f7a0 2024-05-10T11:22:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.458+0000 7fdd7e7fc640 1 -- 172.21.15.100:0/2896589349 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd80011c40 con 0x56063f98f7a0 2024-05-10T11:22:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.459+0000 7fdd9dfae740 1 -- 172.21.15.100:0/2896589349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56063f98f7a0 msgr2=0x56063f9909c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.459+0000 7fdd9dfae740 1 --2- 172.21.15.100:0/2896589349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56063f98f7a0 0x56063f9909c0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fdd8000f490 tx=0x7fdd8000f890 comp rx=0 tx=0).stop 2024-05-10T11:22:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.459+0000 7fdd9dfae740 1 -- 172.21.15.100:0/2896589349 shutdown_connections 2024-05-10T11:22:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.459+0000 7fdd9dfae740 1 --2- 172.21.15.100:0/2896589349 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56063f994910 0x56063f990f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.459+0000 7fdd9dfae740 1 --2- 172.21.15.100:0/2896589349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56063f98f7a0 0x56063f9909c0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.459+0000 7fdd9dfae740 1 --2- 172.21.15.100:0/2896589349 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56063f98edb0 0x56063f985430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.459+0000 7fdd9dfae740 1 -- 172.21.15.100:0/2896589349 >> 172.21.15.100:0/2896589349 conn(0x56063f98a420 msgr2=0x56063f997de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.459+0000 7fdd9dfae740 1 -- 172.21.15.100:0/2896589349 shutdown_connections 2024-05-10T11:22:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.459+0000 7fdd9dfae740 1 -- 172.21.15.100:0/2896589349 wait complete. 2024-05-10T11:22:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.459+0000 7fdd9dfae740 1 -- start start 2024-05-10T11:22:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.459+0000 7fdd9dfae740 1 Processor -- start 2024-05-10T11:22:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.460+0000 7fdd9dfae740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56063f98edb0 0x56063f99d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.460+0000 7fdd9dfae740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56063f98f7a0 0x56063f99d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.460+0000 7fdd9dfae740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56063f994910 0x56063f99e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.460+0000 7fdd9dfae740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdd941abae0 con 0x56063f98f7a0 2024-05-10T11:22:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.460+0000 7fdd9dfae740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdd941acf50 con 0x56063f994910 2024-05-10T11:22:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.460+0000 7fdd9dfae740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fdd941ac1b0 con 0x56063f98edb0 2024-05-10T11:22:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.460+0000 7fdd9dfae740 10 client.0 _ll_register_callbacks cb 0x7fdd80009740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:22:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.460+0000 7fdd93fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56063f98f7a0 0x56063f99d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.460+0000 7fdd93fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56063f98f7a0 0x56063f99d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40448/0 (socket says 172.21.15.100:40448) 2024-05-10T11:22:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.460+0000 7fdd99753640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56063f98edb0 0x56063f99d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.460+0000 7fdd93fff640 1 -- 172.21.15.100:0/4260106585 learned_addr learned my addr 172.21.15.100:0/4260106585 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:22:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.460+0000 7fdd99753640 1 --2- 172.21.15.100:0/4260106585 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56063f994910 0x56063f99e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.460+0000 7fdd93fff640 1 -- 172.21.15.100:0/4260106585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56063f98edb0 msgr2=0x56063f99d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.460+0000 7fdd93fff640 1 --2- 172.21.15.100:0/4260106585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56063f98edb0 0x56063f99d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.460+0000 7fdd93fff640 1 -- 172.21.15.100:0/4260106585 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56063f994910 msgr2=0x56063f99e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.460+0000 7fdd93fff640 1 --2- 172.21.15.100:0/4260106585 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56063f994910 0x56063f99e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.460+0000 7fdd93fff640 1 -- 172.21.15.100:0/4260106585 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56063f98a250 con 0x56063f98f7a0 2024-05-10T11:22:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.460+0000 7fdd99753640 1 --2- 172.21.15.100:0/4260106585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56063f98edb0 0x56063f99d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:22:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.460+0000 7fdd93fff640 1 --2- 172.21.15.100:0/4260106585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56063f98f7a0 0x56063f99d6d0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fdd80009e10 tx=0x7fdd800058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fd88dc98529c9e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.460+0000 7fdd73fff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:22:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.460+0000 7fdd73fff640 1 -- 172.21.15.100:0/4260106585 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd80036020 con 0x56063f98f7a0 2024-05-10T11:22:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.460+0000 7fdd73fff640 1 -- 172.21.15.100:0/4260106585 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fdd8000fd40 con 0x56063f98f7a0 2024-05-10T11:22:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.460+0000 7fdd9dfae740 1 -- 172.21.15.100:0/4260106585 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd8002e070 con 0x56063f98f7a0 2024-05-10T11:22:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.461+0000 7fdd73fff640 1 -- 172.21.15.100:0/4260106585 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdd80041050 con 0x56063f98f7a0 2024-05-10T11:22:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.461+0000 7fdd9dfae740 10 client.16227 fetch_fsmap learned FSMap version 29 2024-05-10T11:22:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.461+0000 7fdd9dfae740 1 -- 172.21.15.100:0/4260106585 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56063f99fd80 con 0x56063f98f7a0 2024-05-10T11:22:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.461+0000 7fdd73fff640 1 -- 172.21.15.100:0/4260106585 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdd8002e070 con 0x56063f98f7a0 2024-05-10T11:22:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.461+0000 7fdd73fff640 1 -- 172.21.15.100:0/4260106585 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fdd80028040 con 0x56063f98f7a0 2024-05-10T11:22:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.461+0000 7fdd9dfae740 10 client.16227 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T11:22:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.461+0000 7fdd9dfae740 10 client.16227 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:22:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.461+0000 7fdd9dfae740 20 client.16227 populate_metadata read hostname 'smithi100' 2024-05-10T11:22:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.461+0000 7fdd9dfae740 1 --2- 172.21.15.100:0/4260106585 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56063fa0fc60 0x56063fa12080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.461+0000 7fdd9dfae740 4 client.16227 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:22:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.461+0000 7fdd9dfae740 1 -- 172.21.15.100:0/4260106585 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56063f8fb050 con 0x56063fa0fc60 2024-05-10T11:22:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.461+0000 7fdd99753640 1 --2- 172.21.15.100:0/4260106585 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56063fa0fc60 0x56063fa12080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.462+0000 7fdd73fff640 10 client.16227 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:22:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.462+0000 7fdd99753640 1 --2- 172.21.15.100:0/4260106585 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56063fa0fc60 0x56063fa12080 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e938fcd193dc0a6c server_cookie=3cc6389480411a10 in_seq=0 out_seq=0 2024-05-10T11:22:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.480+0000 7fdd73fff640 1 -- 172.21.15.100:0/4260106585 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56063f8fb050 con 0x56063fa0fc60 2024-05-10T11:22:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.480+0000 7fdd73fff640 10 client.16227 handle_command_reply: tid=0 2024-05-10T11:22:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.480+0000 7fdd9dfae740 1 client.16227 shutdown 2024-05-10T11:22:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.480+0000 7fdd9dfae740 1 -- 172.21.15.100:0/4260106585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56063f98f7a0 msgr2=0x56063f99d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.480+0000 7fdd9dfae740 1 --2- 172.21.15.100:0/4260106585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56063f98f7a0 0x56063f99d6d0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fdd80009e10 tx=0x7fdd800058d0 comp rx=0 tx=0).stop 2024-05-10T11:22:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.480+0000 7fdd9dfae740 1 -- 172.21.15.100:0/4260106585 shutdown_connections 2024-05-10T11:22:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.480+0000 7fdd9dfae740 1 --2- 172.21.15.100:0/4260106585 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56063fa0fc60 0x56063fa12080 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.480+0000 7fdd9dfae740 1 --2- 172.21.15.100:0/4260106585 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56063f994910 0x56063f99e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.480+0000 7fdd9dfae740 1 --2- 172.21.15.100:0/4260106585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56063f98f7a0 0x56063f99d6d0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.480+0000 7fdd9dfae740 1 --2- 172.21.15.100:0/4260106585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56063f98edb0 0x56063f99d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.480+0000 7fdd9dfae740 1 -- 172.21.15.100:0/4260106585 >> 172.21.15.100:0/4260106585 conn(0x56063f98a420 msgr2=0x56063f997de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.480+0000 7fdd73fff640 0 client.16227 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:22:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.480+0000 7fdd9dfae740 1 -- 172.21.15.100:0/4260106585 shutdown_connections 2024-05-10T11:22:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.480+0000 7fdd9dfae740 1 -- 172.21.15.100:0/4260106585 wait complete. 2024-05-10T11:22:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.480+0000 7fdd9dfae740 20 client.16227 trim_cache size 0 max 16384 2024-05-10T11:22:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:22:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:22:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:22:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:22:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.481+0000 7fdd9b1b6640 1 -- 172.21.15.100:0/3946102367 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fdd840b28a0 msgr2=0x7fdd840b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.481+0000 7fdd9b1b6640 1 --2- 172.21.15.100:0/3946102367 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fdd840b28a0 0x7fdd840b4d60 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fdd881006c0 tx=0x7fdd88106690 comp rx=0 tx=0).stop 2024-05-10T11:22:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.481+0000 7fdd9b1b6640 1 -- 172.21.15.100:0/3946102367 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd94103490 msgr2=0x7fdd94102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.481+0000 7fdd9b1b6640 1 --2- 172.21.15.100:0/3946102367 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd94103490 0x7fdd94102e00 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fdd88009a20 tx=0x7fdd880040c0 comp rx=0 tx=0).stop 2024-05-10T11:22:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.481+0000 7fdd9b1b6640 1 -- 172.21.15.100:0/3946102367 shutdown_connections 2024-05-10T11:22:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.481+0000 7fdd9b1b6640 1 --2- 172.21.15.100:0/3946102367 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fdd840b28a0 0x7fdd840b4d60 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.481+0000 7fdd9b1b6640 1 --2- 172.21.15.100:0/3946102367 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd941086f0 0x7fdd94101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.481+0000 7fdd9b1b6640 1 --2- 172.21.15.100:0/3946102367 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd94105dc0 0x7fdd94101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.481+0000 7fdd9b1b6640 1 --2- 172.21.15.100:0/3946102367 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd94103490 0x7fdd94102e00 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.481+0000 7fdd9b1b6640 1 -- 172.21.15.100:0/3946102367 >> 172.21.15.100:0/3946102367 conn(0x7fdd940fce80 msgr2=0x7fdd94104010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.481+0000 7fdd9b1b6640 1 -- 172.21.15.100:0/3946102367 shutdown_connections 2024-05-10T11:22:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:45.481+0000 7fdd9b1b6640 1 -- 172.21.15.100:0/3946102367 wait complete. 2024-05-10T11:22:45.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:22:45 smithi188 ceph-mon[34075]: pgmap v551: 129 pgs: 129 active+clean; 49 MiB data, 831 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 1.3 MiB/s wr, 128 op/s 2024-05-10T11:22:45.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:22:45 smithi100 ceph-mon[30296]: pgmap v551: 129 pgs: 129 active+clean; 49 MiB data, 831 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 1.3 MiB/s wr, 128 op/s 2024-05-10T11:22:45.951 INFO:tasks.workunit.client.0.smithi100.stdout: 1 84657 2.73 MB/sec execute 40 sec latency 20.098 ms 2024-05-10T11:22:46.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:22:45 smithi178 ceph-mon[33359]: pgmap v551: 129 pgs: 129 active+clean; 49 MiB data, 831 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 1.3 MiB/s wr, 128 op/s 2024-05-10T11:22:46.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.190+0000 7f2d44337700 1 -- 172.21.15.100:0/2536120779 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d3c102d60 msgr2=0x7f2d3c10f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:46.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.190+0000 7f2d44337700 1 --2- 172.21.15.100:0/2536120779 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d3c102d60 0x7f2d3c10f250 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f2d3400b0a0 tx=0x7f2d3400b3b0 comp rx=0 tx=0).stop 2024-05-10T11:22:46.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.190+0000 7f2d44337700 1 -- 172.21.15.100:0/2536120779 shutdown_connections 2024-05-10T11:22:46.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.190+0000 7f2d44337700 1 --2- 172.21.15.100:0/2536120779 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d3c102d60 0x7f2d3c10f250 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:46.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.190+0000 7f2d44337700 1 --2- 172.21.15.100:0/2536120779 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d3c108410 0x7f2d3c1087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:46.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.190+0000 7f2d44337700 1 --2- 172.21.15.100:0/2536120779 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2d3c1023a0 0x7f2d3c102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:46.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.190+0000 7f2d44337700 1 -- 172.21.15.100:0/2536120779 >> 172.21.15.100:0/2536120779 conn(0x7f2d3c079350 msgr2=0x7f2d3c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:46.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.190+0000 7f2d44337700 1 -- 172.21.15.100:0/2536120779 shutdown_connections 2024-05-10T11:22:46.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.190+0000 7f2d44337700 1 -- 172.21.15.100:0/2536120779 wait complete. 2024-05-10T11:22:46.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.191+0000 7f2d44337700 1 Processor -- start 2024-05-10T11:22:46.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.191+0000 7f2d44337700 1 -- start start 2024-05-10T11:22:46.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.191+0000 7f2d44337700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d3c1023a0 0x7f2d3c19f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:46.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.191+0000 7f2d44337700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2d3c102d60 0x7f2d3c19fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:46.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.191+0000 7f2d44337700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d3c108410 0x7f2d3c1998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:46.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.191+0000 7f2d44337700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2d3c1123e0 con 0x7f2d3c1023a0 2024-05-10T11:22:46.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.191+0000 7f2d44337700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2d3c112260 con 0x7f2d3c108410 2024-05-10T11:22:46.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.191+0000 7f2d44337700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2d3c112560 con 0x7f2d3c102d60 2024-05-10T11:22:46.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.191+0000 7f2d420d3700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d3c1023a0 0x7f2d3c19f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:46.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.191+0000 7f2d420d3700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d3c1023a0 0x7f2d3c19f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40476/0 (socket says 172.21.15.100:40476) 2024-05-10T11:22:46.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.191+0000 7f2d420d3700 1 -- 172.21.15.100:0/589591202 learned_addr learned my addr 172.21.15.100:0/589591202 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:22:46.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.191+0000 7f2d418d2700 1 --2- 172.21.15.100:0/589591202 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2d3c102d60 0x7f2d3c19fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:46.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.192+0000 7f2d428d4700 1 --2- 172.21.15.100:0/589591202 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d3c108410 0x7f2d3c1998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:46.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.192+0000 7f2d420d3700 1 -- 172.21.15.100:0/589591202 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2d3c102d60 msgr2=0x7f2d3c19fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:46.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.192+0000 7f2d420d3700 1 --2- 172.21.15.100:0/589591202 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2d3c102d60 0x7f2d3c19fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:46.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.192+0000 7f2d420d3700 1 -- 172.21.15.100:0/589591202 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d3c108410 msgr2=0x7f2d3c1998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:46.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.192+0000 7f2d420d3700 1 --2- 172.21.15.100:0/589591202 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d3c108410 0x7f2d3c1998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:46.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.192+0000 7f2d420d3700 1 -- 172.21.15.100:0/589591202 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d3c19a030 con 0x7f2d3c1023a0 2024-05-10T11:22:46.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.192+0000 7f2d428d4700 1 --2- 172.21.15.100:0/589591202 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d3c108410 0x7f2d3c1998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:22:46.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.192+0000 7f2d420d3700 1 --2- 172.21.15.100:0/589591202 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d3c1023a0 0x7f2d3c19f710 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f2d3800d950 tx=0x7f2d3800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:46.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.192+0000 7f2d337fe700 1 -- 172.21.15.100:0/589591202 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d38016070 con 0x7f2d3c1023a0 2024-05-10T11:22:46.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.192+0000 7f2d337fe700 1 -- 172.21.15.100:0/589591202 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2d38009dd0 con 0x7f2d3c1023a0 2024-05-10T11:22:46.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.192+0000 7f2d44337700 1 -- 172.21.15.100:0/589591202 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d3c19a2c0 con 0x7f2d3c1023a0 2024-05-10T11:22:46.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.192+0000 7f2d337fe700 1 -- 172.21.15.100:0/589591202 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d3800f3c0 con 0x7f2d3c1023a0 2024-05-10T11:22:46.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.192+0000 7f2d44337700 1 -- 172.21.15.100:0/589591202 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d3c1a65e0 con 0x7f2d3c1023a0 2024-05-10T11:22:46.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.195+0000 7f2d337fe700 1 -- 172.21.15.100:0/589591202 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f2d38019070 con 0x7f2d3c1023a0 2024-05-10T11:22:46.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.196+0000 7f2d337fe700 1 --2- 172.21.15.100:0/589591202 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2d280b2880 0x7f2d280b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:46.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.196+0000 7f2d418d2700 1 --2- 172.21.15.100:0/589591202 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2d280b2880 0x7f2d280b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:46.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.196+0000 7f2d337fe700 1 -- 172.21.15.100:0/589591202 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f2d380e2160 con 0x7f2d3c1023a0 2024-05-10T11:22:46.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.196+0000 7f2d418d2700 1 --2- 172.21.15.100:0/589591202 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2d280b2880 0x7f2d280b4d40 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c06c1f0 tx=0x7f2d2c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:46.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.197+0000 7f2d4ebcab80 1 Processor -- start 2024-05-10T11:22:46.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.197+0000 7f2d4ebcab80 1 -- start start 2024-05-10T11:22:46.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.197+0000 7f2d4ebcab80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561b38b28f80 0x561b38b29360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:46.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.197+0000 7f2d4ebcab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561b38b298a0 0x561b38b23700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:46.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.197+0000 7f2d4ebcab80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b38b23c90 0x561b38b24110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:46.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.197+0000 7f2d4ebcab80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x561b38a71860 con 0x561b38b23c90 2024-05-10T11:22:46.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.197+0000 7f2d4ebcab80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561b38a107c0 con 0x561b38b28f80 2024-05-10T11:22:46.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.197+0000 7f2d4ebcab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x561b38a6d820 con 0x561b38b298a0 2024-05-10T11:22:46.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.197+0000 7f2d428d4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b38b23c90 0x561b38b24110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:46.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.197+0000 7f2d428d4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b38b23c90 0x561b38b24110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40490/0 (socket says 172.21.15.100:40490) 2024-05-10T11:22:46.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.197+0000 7f2d428d4700 1 -- 172.21.15.100:0/3974873824 learned_addr learned my addr 172.21.15.100:0/3974873824 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:22:46.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.197+0000 7f2d418d2700 1 --2- 172.21.15.100:0/3974873824 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561b38b298a0 0x561b38b23700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:46.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.198+0000 7f2d420d3700 1 --2- 172.21.15.100:0/3974873824 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561b38b28f80 0x561b38b29360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:46.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.198+0000 7f2d428d4700 1 -- 172.21.15.100:0/3974873824 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561b38b298a0 msgr2=0x561b38b23700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:46.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.198+0000 7f2d428d4700 1 --2- 172.21.15.100:0/3974873824 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561b38b298a0 0x561b38b23700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:46.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.198+0000 7f2d428d4700 1 -- 172.21.15.100:0/3974873824 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561b38b28f80 msgr2=0x561b38b29360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:46.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.198+0000 7f2d428d4700 1 --2- 172.21.15.100:0/3974873824 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561b38b28f80 0x561b38b29360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:46.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.198+0000 7f2d428d4700 1 -- 172.21.15.100:0/3974873824 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b389be8d0 con 0x561b38b23c90 2024-05-10T11:22:46.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.198+0000 7f2d428d4700 1 --2- 172.21.15.100:0/3974873824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b38b23c90 0x561b38b24110 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f2d34006130 tx=0x7f2d340092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74e60cab02fcea95 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:46.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.198+0000 7f2d30d70700 1 -- 172.21.15.100:0/3974873824 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d3400bcb0 con 0x561b38b23c90 2024-05-10T11:22:46.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.198+0000 7f2d30d70700 1 -- 172.21.15.100:0/3974873824 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2d3400be50 con 0x561b38b23c90 2024-05-10T11:22:46.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.198+0000 7f2d30d70700 1 -- 172.21.15.100:0/3974873824 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d34007da0 con 0x561b38b23c90 2024-05-10T11:22:46.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.199+0000 7f2d4ebcab80 1 -- 172.21.15.100:0/3974873824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b38b23c90 msgr2=0x561b38b24110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:46.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.199+0000 7f2d4ebcab80 1 --2- 172.21.15.100:0/3974873824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b38b23c90 0x561b38b24110 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f2d34006130 tx=0x7f2d340092e0 comp rx=0 tx=0).stop 2024-05-10T11:22:46.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.199+0000 7f2d4ebcab80 1 -- 172.21.15.100:0/3974873824 shutdown_connections 2024-05-10T11:22:46.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.199+0000 7f2d4ebcab80 1 --2- 172.21.15.100:0/3974873824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b38b23c90 0x561b38b24110 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:46.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.199+0000 7f2d4ebcab80 1 --2- 172.21.15.100:0/3974873824 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561b38b28f80 0x561b38b29360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:46.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.199+0000 7f2d4ebcab80 1 --2- 172.21.15.100:0/3974873824 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561b38b298a0 0x561b38b23700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:46.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.199+0000 7f2d4ebcab80 1 -- 172.21.15.100:0/3974873824 >> 172.21.15.100:0/3974873824 conn(0x561b38a4b100 msgr2=0x561b38b19c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:46.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.199+0000 7f2d4ebcab80 1 -- 172.21.15.100:0/3974873824 shutdown_connections 2024-05-10T11:22:46.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.199+0000 7f2d4ebcab80 1 -- 172.21.15.100:0/3974873824 wait complete. 2024-05-10T11:22:46.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.199+0000 7f2d4ebcab80 1 -- start start 2024-05-10T11:22:46.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.200+0000 7f2d4ebcab80 1 Processor -- start 2024-05-10T11:22:46.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.200+0000 7f2d4ebcab80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561b38b23c90 0x561b38b2f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:46.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.200+0000 7f2d4ebcab80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b38b28f80 0x561b38b2c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:46.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.200+0000 7f2d4ebcab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561b38b298a0 0x561b38b2ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:46.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.200+0000 7f2d4ebcab80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x561b38b30260 con 0x561b38b28f80 2024-05-10T11:22:46.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.200+0000 7f2d4ebcab80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561b389d7c30 con 0x561b38b23c90 2024-05-10T11:22:46.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.200+0000 7f2d4ebcab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x561b38b303e0 con 0x561b38b298a0 2024-05-10T11:22:46.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.200+0000 7f2d4ebcab80 10 client.0 _ll_register_callbacks cb 0x561b38a09920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:22:46.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.200+0000 7f2d418d2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b38b28f80 0x561b38b2c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:46.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.200+0000 7f2d428d4700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561b38b298a0 0x561b38b2ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:46.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.200+0000 7f2d418d2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b38b28f80 0x561b38b2c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40494/0 (socket says 172.21.15.100:40494) 2024-05-10T11:22:46.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.200+0000 7f2d418d2700 1 -- 172.21.15.100:0/307854645 learned_addr learned my addr 172.21.15.100:0/307854645 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:22:46.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.200+0000 7f2d420d3700 1 --2- 172.21.15.100:0/307854645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561b38b23c90 0x561b38b2f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:46.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.200+0000 7f2d428d4700 1 -- 172.21.15.100:0/307854645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561b38b23c90 msgr2=0x561b38b2f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:46.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.200+0000 7f2d428d4700 1 --2- 172.21.15.100:0/307854645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561b38b23c90 0x561b38b2f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:46.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.200+0000 7f2d428d4700 1 -- 172.21.15.100:0/307854645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b38b28f80 msgr2=0x561b38b2c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:46.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.200+0000 7f2d428d4700 1 --2- 172.21.15.100:0/307854645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b38b28f80 0x561b38b2c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:46.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.200+0000 7f2d428d4700 1 -- 172.21.15.100:0/307854645 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b38a0d690 con 0x561b38b298a0 2024-05-10T11:22:46.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.200+0000 7f2d418d2700 1 --2- 172.21.15.100:0/307854645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b38b28f80 0x561b38b2c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:22:46.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.201+0000 7f2d428d4700 1 --2- 172.21.15.100:0/307854645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561b38b298a0 0x561b38b2ce90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2d34009a20 tx=0x7f2d34007d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a2f3be63d4fa73c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:46.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.201+0000 7f2d25ffb700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:22:46.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.201+0000 7f2d25ffb700 1 -- 172.21.15.100:0/307854645 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d34004a50 con 0x561b38b298a0 2024-05-10T11:22:46.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.201+0000 7f2d4ebcab80 1 -- 172.21.15.100:0/307854645 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d3400be50 con 0x561b38b298a0 2024-05-10T11:22:46.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.201+0000 7f2d25ffb700 1 -- 172.21.15.100:0/307854645 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f2d34004bf0 con 0x561b38b298a0 2024-05-10T11:22:46.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.201+0000 7f2d25ffb700 1 -- 172.21.15.100:0/307854645 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2d34038050 con 0x561b38b298a0 2024-05-10T11:22:46.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.201+0000 7f2d4ebcab80 10 client.25738 fetch_fsmap learned FSMap version 30 2024-05-10T11:22:46.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.201+0000 7f2d4ebcab80 1 -- 172.21.15.100:0/307854645 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561b38b30260 con 0x561b38b298a0 2024-05-10T11:22:46.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.201+0000 7f2d25ffb700 1 -- 172.21.15.100:0/307854645 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2d3400be50 con 0x561b38b298a0 2024-05-10T11:22:46.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.202+0000 7f2d25ffb700 1 -- 172.21.15.100:0/307854645 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7f2d34011070 con 0x561b38b298a0 2024-05-10T11:22:46.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.202+0000 7f2d4ebcab80 10 client.25738 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T11:22:46.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.202+0000 7f2d4ebcab80 10 client.25738 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-10T11:22:46.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.202+0000 7f2d4ebcab80 20 client.25738 populate_metadata read hostname 'smithi100' 2024-05-10T11:22:46.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.202+0000 7f2d4ebcab80 1 --2- 172.21.15.100:0/307854645 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x561b38b3cec0 0x561b38b3f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:46.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.202+0000 7f2d4ebcab80 4 client.25738 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:22:46.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.202+0000 7f2d4ebcab80 1 -- 172.21.15.100:0/307854645 --> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b38aeb650 con 0x561b38b3cec0 2024-05-10T11:22:46.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.202+0000 7f2d420d3700 1 --2- 172.21.15.100:0/307854645 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x561b38b3cec0 0x561b38b3f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:46.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.202+0000 7f2d420d3700 1 --2- 172.21.15.100:0/307854645 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x561b38b3cec0 0x561b38b3f2b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e66120084b7b18c2 server_cookie=62f27526ac0a3e5e in_seq=0 out_seq=0 2024-05-10T11:22:46.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.202+0000 7f2d25ffb700 10 client.25738 ms_handle_connect on v2:172.21.15.178:6838/3760985404 2024-05-10T11:22:46.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.203+0000 7f2d25ffb700 1 -- 172.21.15.100:0/307854645 <== mds.1 v2:172.21.15.178:6838/3760985404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561b38aeb650 con 0x561b38b3cec0 2024-05-10T11:22:46.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.203+0000 7f2d25ffb700 10 client.25738 handle_command_reply: tid=0 2024-05-10T11:22:46.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.203+0000 7f2d4ebcab80 1 client.25738 shutdown 2024-05-10T11:22:46.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.203+0000 7f2d4ebcab80 1 -- 172.21.15.100:0/307854645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561b38b298a0 msgr2=0x561b38b2ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:46.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.203+0000 7f2d4ebcab80 1 --2- 172.21.15.100:0/307854645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561b38b298a0 0x561b38b2ce90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2d34009a20 tx=0x7f2d34007d00 comp rx=0 tx=0).stop 2024-05-10T11:22:46.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.203+0000 7f2d4ebcab80 1 -- 172.21.15.100:0/307854645 shutdown_connections 2024-05-10T11:22:46.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.203+0000 7f2d4ebcab80 1 --2- 172.21.15.100:0/307854645 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x561b38b3cec0 0x561b38b3f2b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:46.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.203+0000 7f2d4ebcab80 1 --2- 172.21.15.100:0/307854645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561b38b298a0 0x561b38b2ce90 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:46.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.203+0000 7f2d4ebcab80 1 --2- 172.21.15.100:0/307854645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561b38b23c90 0x561b38b2f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:46.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.203+0000 7f2d4ebcab80 1 --2- 172.21.15.100:0/307854645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b38b28f80 0x561b38b2c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:46.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.203+0000 7f2d4ebcab80 1 -- 172.21.15.100:0/307854645 >> 172.21.15.100:0/307854645 conn(0x561b38a4b100 msgr2=0x561b38b14720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:46.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.203+0000 7f2d25ffb700 0 client.25738 ms_handle_reset on v2:172.21.15.178:6838/3760985404 2024-05-10T11:22:46.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.203+0000 7f2d4ebcab80 1 -- 172.21.15.100:0/307854645 shutdown_connections 2024-05-10T11:22:46.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.203+0000 7f2d4ebcab80 1 -- 172.21.15.100:0/307854645 wait complete. 2024-05-10T11:22:46.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.203+0000 7f2d4ebcab80 20 client.25738 trim_cache size 0 max 16384 2024-05-10T11:22:46.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.218+0000 7f2d4ebcab80 1 Processor -- start 2024-05-10T11:22:46.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.218+0000 7f2d4ebcab80 1 -- start start 2024-05-10T11:22:46.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.218+0000 7f2d4ebcab80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561b38b23c90 0x561b38b27070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:46.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.218+0000 7f2d4ebcab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561b38b28f80 0x561b38b2a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:46.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.218+0000 7f2d4ebcab80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b38b298a0 0x561b38b392f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:46.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.219+0000 7f2d4ebcab80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2d3c1af620 con 0x561b38b298a0 2024-05-10T11:22:46.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.219+0000 7f2d4ebcab80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2d3c1aceb0 con 0x561b38b23c90 2024-05-10T11:22:46.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.219+0000 7f2d4ebcab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2d3c1aad20 con 0x561b38b28f80 2024-05-10T11:22:46.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.219+0000 7f2d418d2700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561b38b28f80 0x561b38b2a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:46.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.219+0000 7f2d428d4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b38b298a0 0x561b38b392f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:46.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.219+0000 7f2d418d2700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561b38b28f80 0x561b38b2a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:57518/0 (socket says 172.21.15.100:57518) 2024-05-10T11:22:46.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.219+0000 7f2d418d2700 1 -- 172.21.15.100:0/3009898883 learned_addr learned my addr 172.21.15.100:0/3009898883 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:22:46.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.219+0000 7f2d420d3700 1 --2- 172.21.15.100:0/3009898883 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561b38b23c90 0x561b38b27070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:46.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.219+0000 7f2d418d2700 1 -- 172.21.15.100:0/3009898883 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561b38b23c90 msgr2=0x561b38b27070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:46.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.219+0000 7f2d418d2700 1 --2- 172.21.15.100:0/3009898883 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561b38b23c90 0x561b38b27070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:46.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.219+0000 7f2d418d2700 1 -- 172.21.15.100:0/3009898883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b38b298a0 msgr2=0x561b38b392f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:46.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.219+0000 7f2d418d2700 1 --2- 172.21.15.100:0/3009898883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b38b298a0 0x561b38b392f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:46.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.219+0000 7f2d418d2700 1 -- 172.21.15.100:0/3009898883 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b38b1ae20 con 0x561b38b28f80 2024-05-10T11:22:46.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.219+0000 7f2d420d3700 1 --2- 172.21.15.100:0/3009898883 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561b38b23c90 0x561b38b27070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:22:46.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.219+0000 7f2d428d4700 1 --2- 172.21.15.100:0/3009898883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b38b298a0 0x561b38b392f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:22:46.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.219+0000 7f2d418d2700 1 --2- 172.21.15.100:0/3009898883 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561b38b28f80 0x561b38b2a510 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c0045f0 tx=0x7f2d2c014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bc22517bf1c1a31 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:46.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.219+0000 7f2d27fff700 1 -- 172.21.15.100:0/3009898883 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2c003c90 con 0x561b38b28f80 2024-05-10T11:22:46.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.219+0000 7f2d27fff700 1 -- 172.21.15.100:0/3009898883 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2d2c02c070 con 0x561b38b28f80 2024-05-10T11:22:46.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.219+0000 7f2d27fff700 1 -- 172.21.15.100:0/3009898883 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2c027400 con 0x561b38b28f80 2024-05-10T11:22:46.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.220+0000 7f2d4ebcab80 1 -- 172.21.15.100:0/3009898883 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561b38b28f80 msgr2=0x561b38b2a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:46.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.220+0000 7f2d4ebcab80 1 --2- 172.21.15.100:0/3009898883 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561b38b28f80 0x561b38b2a510 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c0045f0 tx=0x7f2d2c014000 comp rx=0 tx=0).stop 2024-05-10T11:22:46.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.220+0000 7f2d4ebcab80 1 -- 172.21.15.100:0/3009898883 shutdown_connections 2024-05-10T11:22:46.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.220+0000 7f2d4ebcab80 1 --2- 172.21.15.100:0/3009898883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b38b298a0 0x561b38b392f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:46.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.220+0000 7f2d4ebcab80 1 --2- 172.21.15.100:0/3009898883 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561b38b23c90 0x561b38b27070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:46.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.220+0000 7f2d4ebcab80 1 --2- 172.21.15.100:0/3009898883 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561b38b28f80 0x561b38b2a510 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:46.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.220+0000 7f2d4ebcab80 1 -- 172.21.15.100:0/3009898883 >> 172.21.15.100:0/3009898883 conn(0x561b38a4b100 msgr2=0x561b38b2b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:46.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.220+0000 7f2d4ebcab80 1 -- 172.21.15.100:0/3009898883 shutdown_connections 2024-05-10T11:22:46.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.220+0000 7f2d4ebcab80 1 -- 172.21.15.100:0/3009898883 wait complete. 2024-05-10T11:22:46.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.221+0000 7f2d4ebcab80 1 -- start start 2024-05-10T11:22:46.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.221+0000 7f2d4ebcab80 1 Processor -- start 2024-05-10T11:22:46.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.221+0000 7f2d4ebcab80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b38b23c90 0x561b38b35d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:46.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.221+0000 7f2d4ebcab80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561b38b28f80 0x561b38b30630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:46.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.221+0000 7f2d4ebcab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561b38b298a0 0x561b38b30d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:46.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.221+0000 7f2d4ebcab80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2d3c1ab370 con 0x561b38b23c90 2024-05-10T11:22:46.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.221+0000 7f2d420d3700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b38b23c90 0x561b38b35d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:46.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.221+0000 7f2d4ebcab80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2d3c1ac7e0 con 0x561b38b28f80 2024-05-10T11:22:46.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.221+0000 7f2d420d3700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b38b23c90 0x561b38b35d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40518/0 (socket says 172.21.15.100:40518) 2024-05-10T11:22:46.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.221+0000 7f2d420d3700 1 -- 172.21.15.100:0/1567778364 learned_addr learned my addr 172.21.15.100:0/1567778364 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:22:46.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.221+0000 7f2d4ebcab80 1 -- 172.21.15.100:0/1567778364 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2d3c1aba40 con 0x561b38b298a0 2024-05-10T11:22:46.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.221+0000 7f2d428d4700 1 --2- 172.21.15.100:0/1567778364 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561b38b298a0 0x561b38b30d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:46.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.221+0000 7f2d4ebcab80 10 client.0 _ll_register_callbacks cb 0x561b38a09920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:22:46.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.221+0000 7f2d418d2700 1 --2- 172.21.15.100:0/1567778364 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561b38b28f80 0x561b38b30630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:46.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.221+0000 7f2d420d3700 1 -- 172.21.15.100:0/1567778364 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561b38b298a0 msgr2=0x561b38b30d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:46.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.221+0000 7f2d420d3700 1 --2- 172.21.15.100:0/1567778364 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561b38b298a0 0x561b38b30d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:46.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.221+0000 7f2d420d3700 1 -- 172.21.15.100:0/1567778364 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561b38b28f80 msgr2=0x561b38b30630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:46.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.221+0000 7f2d420d3700 1 --2- 172.21.15.100:0/1567778364 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561b38b28f80 0x561b38b30630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:46.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.221+0000 7f2d420d3700 1 -- 172.21.15.100:0/1567778364 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b38ba54e0 con 0x561b38b23c90 2024-05-10T11:22:46.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.222+0000 7f2d418d2700 1 --2- 172.21.15.100:0/1567778364 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561b38b28f80 0x561b38b30630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:22:46.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.222+0000 7f2d420d3700 1 --2- 172.21.15.100:0/1567778364 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b38b23c90 0x561b38b35d30 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f2d380dfc70 tx=0x7f2d38091040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=729d5102a1ef7008 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:46.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.222+0000 7f2d25ffb700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:22:46.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.222+0000 7f2d25ffb700 1 -- 172.21.15.100:0/1567778364 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d38020040 con 0x561b38b23c90 2024-05-10T11:22:46.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.222+0000 7f2d25ffb700 1 -- 172.21.15.100:0/1567778364 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f2d380106a0 con 0x561b38b23c90 2024-05-10T11:22:46.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.222+0000 7f2d4ebcab80 1 -- 172.21.15.100:0/1567778364 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d2c02c070 con 0x561b38b23c90 2024-05-10T11:22:46.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.222+0000 7f2d25ffb700 1 -- 172.21.15.100:0/1567778364 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2d380a9420 con 0x561b38b23c90 2024-05-10T11:22:46.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.222+0000 7f2d4ebcab80 10 client.16245 fetch_fsmap learned FSMap version 30 2024-05-10T11:22:46.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.222+0000 7f2d4ebcab80 1 -- 172.21.15.100:0/1567778364 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561b38ba6720 con 0x561b38b23c90 2024-05-10T11:22:46.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.222+0000 7f2d25ffb700 1 -- 172.21.15.100:0/1567778364 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2d2c02c070 con 0x561b38b23c90 2024-05-10T11:22:46.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.222+0000 7f2d25ffb700 1 -- 172.21.15.100:0/1567778364 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7f2d380a3b40 con 0x561b38b23c90 2024-05-10T11:22:46.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.222+0000 7f2d4ebcab80 10 client.16245 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T11:22:46.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.222+0000 7f2d4ebcab80 10 client.16245 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-10T11:22:46.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.222+0000 7f2d4ebcab80 20 client.16245 populate_metadata read hostname 'smithi100' 2024-05-10T11:22:46.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.222+0000 7f2d4ebcab80 1 --2- 172.21.15.100:0/1567778364 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x561b38ba6eb0 0x561b38ba92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:46.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.222+0000 7f2d4ebcab80 4 client.16245 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:22:46.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.222+0000 7f2d4ebcab80 1 -- 172.21.15.100:0/1567778364 --> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561b38a7f240 con 0x561b38ba6eb0 2024-05-10T11:22:46.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.223+0000 7f2d418d2700 1 --2- 172.21.15.100:0/1567778364 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x561b38ba6eb0 0x561b38ba92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:46.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.223+0000 7f2d418d2700 1 --2- 172.21.15.100:0/1567778364 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x561b38ba6eb0 0x561b38ba92a0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7866bfff61f5021a server_cookie=b14fafdc08092ca9 in_seq=0 out_seq=0 2024-05-10T11:22:46.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.223+0000 7f2d25ffb700 10 client.16245 ms_handle_connect on v2:172.21.15.178:6838/3760985404 2024-05-10T11:22:46.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.223+0000 7f2d25ffb700 1 -- 172.21.15.100:0/1567778364 <== mds.1 v2:172.21.15.178:6838/3760985404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561b38a7f240 con 0x561b38ba6eb0 2024-05-10T11:22:46.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.223+0000 7f2d25ffb700 10 client.16245 handle_command_reply: tid=0 2024-05-10T11:22:46.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.223+0000 7f2d4ebcab80 1 client.16245 shutdown 2024-05-10T11:22:46.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.223+0000 7f2d4ebcab80 1 -- 172.21.15.100:0/1567778364 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b38b23c90 msgr2=0x561b38b35d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:46.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.223+0000 7f2d4ebcab80 1 --2- 172.21.15.100:0/1567778364 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b38b23c90 0x561b38b35d30 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f2d380dfc70 tx=0x7f2d38091040 comp rx=0 tx=0).stop 2024-05-10T11:22:46.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.224+0000 7f2d4ebcab80 1 -- 172.21.15.100:0/1567778364 shutdown_connections 2024-05-10T11:22:46.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.224+0000 7f2d4ebcab80 1 --2- 172.21.15.100:0/1567778364 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x561b38ba6eb0 0x561b38ba92a0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:46.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.224+0000 7f2d4ebcab80 1 --2- 172.21.15.100:0/1567778364 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561b38b298a0 0x561b38b30d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:46.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.224+0000 7f2d4ebcab80 1 --2- 172.21.15.100:0/1567778364 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b38b23c90 0x561b38b35d30 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:46.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.224+0000 7f2d25ffb700 0 client.16245 ms_handle_reset on v2:172.21.15.178:6838/3760985404 2024-05-10T11:22:46.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.224+0000 7f2d4ebcab80 1 --2- 172.21.15.100:0/1567778364 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561b38b28f80 0x561b38b30630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:46.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.224+0000 7f2d4ebcab80 1 -- 172.21.15.100:0/1567778364 >> 172.21.15.100:0/1567778364 conn(0x561b38a4b100 msgr2=0x561b38b14720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:46.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.224+0000 7f2d4ebcab80 1 -- 172.21.15.100:0/1567778364 shutdown_connections 2024-05-10T11:22:46.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.224+0000 7f2d4ebcab80 1 -- 172.21.15.100:0/1567778364 wait complete. 2024-05-10T11:22:46.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.224+0000 7f2d4ebcab80 20 client.16245 trim_cache size 0 max 16384 2024-05-10T11:22:46.232 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:22:46.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.224+0000 7f2d44337700 1 -- 172.21.15.100:0/589591202 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2d280b2880 msgr2=0x7f2d280b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:46.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.224+0000 7f2d44337700 1 --2- 172.21.15.100:0/589591202 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2d280b2880 0x7f2d280b4d40 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c06c1f0 tx=0x7f2d2c009260 comp rx=0 tx=0).stop 2024-05-10T11:22:46.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.224+0000 7f2d44337700 1 -- 172.21.15.100:0/589591202 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d3c1023a0 msgr2=0x7f2d3c19f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:46.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.224+0000 7f2d44337700 1 --2- 172.21.15.100:0/589591202 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d3c1023a0 0x7f2d3c19f710 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f2d3800d950 tx=0x7f2d3800dc60 comp rx=0 tx=0).stop 2024-05-10T11:22:46.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.225+0000 7f2d44337700 1 -- 172.21.15.100:0/589591202 shutdown_connections 2024-05-10T11:22:46.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.225+0000 7f2d44337700 1 --2- 172.21.15.100:0/589591202 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2d280b2880 0x7f2d280b4d40 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:46.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.225+0000 7f2d44337700 1 --2- 172.21.15.100:0/589591202 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d3c108410 0x7f2d3c1998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:46.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.225+0000 7f2d44337700 1 --2- 172.21.15.100:0/589591202 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d3c1023a0 0x7f2d3c19f710 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:46.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.225+0000 7f2d44337700 1 --2- 172.21.15.100:0/589591202 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2d3c102d60 0x7f2d3c19fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:46.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.225+0000 7f2d44337700 1 -- 172.21.15.100:0/589591202 >> 172.21.15.100:0/589591202 conn(0x7f2d3c079350 msgr2=0x7f2d3c106620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:46.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.225+0000 7f2d44337700 1 -- 172.21.15.100:0/589591202 shutdown_connections 2024-05-10T11:22:46.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:46.225+0000 7f2d44337700 1 -- 172.21.15.100:0/589591202 wait complete. 2024-05-10T11:22:46.951 INFO:tasks.workunit.client.0.smithi100.stdout: 1 85250 2.78 MB/sec execute 41 sec latency 16.009 ms 2024-05-10T11:22:46.988 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:22:46 smithi100 ceph-mon[30296]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T11:22:47.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:22:46 smithi178 ceph-mon[33359]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T11:22:47.019 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-10T11:22:47.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:22:46 smithi188 ceph-mon[34075]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T11:22:47.284 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:22:47.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:22:47 smithi100 ceph-mon[30296]: pgmap v552: 129 pgs: 129 active+clean; 45 MiB data, 811 MiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 1.4 MiB/s wr, 180 op/s 2024-05-10T11:22:47.929 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:22:47 smithi188 ceph-mon[34075]: pgmap v552: 129 pgs: 129 active+clean; 45 MiB data, 811 MiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 1.4 MiB/s wr, 180 op/s 2024-05-10T11:22:47.950 INFO:tasks.workunit.client.0.smithi100.stdout: 1 85579 2.75 MB/sec execute 42 sec latency 12.921 ms 2024-05-10T11:22:48.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:22:47 smithi178 ceph-mon[33359]: pgmap v552: 129 pgs: 129 active+clean; 45 MiB data, 811 MiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 1.4 MiB/s wr, 180 op/s 2024-05-10T11:22:48.969 INFO:tasks.workunit.client.0.smithi100.stdout: 1 85914 2.69 MB/sec execute 43 sec latency 37.924 ms 2024-05-10T11:22:49.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.154+0000 7fddbd78b700 1 -- 172.21.15.100:0/2044025821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fddb8108400 msgr2=0x7fddb81087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:49.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.154+0000 7fddbd78b700 1 --2- 172.21.15.100:0/2044025821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fddb8108400 0x7fddb81087e0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fdda8009990 tx=0x7fdda8009ca0 comp rx=0 tx=0).stop 2024-05-10T11:22:49.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.154+0000 7fddbd78b700 1 -- 172.21.15.100:0/2044025821 shutdown_connections 2024-05-10T11:22:49.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.154+0000 7fddbd78b700 1 --2- 172.21.15.100:0/2044025821 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fddb8102d50 0x7fddb810f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:49.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.154+0000 7fddbd78b700 1 --2- 172.21.15.100:0/2044025821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fddb8108400 0x7fddb81087e0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:49.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.154+0000 7fddbd78b700 1 --2- 172.21.15.100:0/2044025821 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fddb8102390 0x7fddb8102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:49.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.154+0000 7fddbd78b700 1 -- 172.21.15.100:0/2044025821 >> 172.21.15.100:0/2044025821 conn(0x7fddb8079310 msgr2=0x7fddb8079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:49.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.154+0000 7fddbd78b700 1 -- 172.21.15.100:0/2044025821 shutdown_connections 2024-05-10T11:22:49.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.155+0000 7fddbd78b700 1 -- 172.21.15.100:0/2044025821 wait complete. 2024-05-10T11:22:49.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.155+0000 7fddbd78b700 1 Processor -- start 2024-05-10T11:22:49.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.156+0000 7fddbd78b700 1 -- start start 2024-05-10T11:22:49.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.156+0000 7fddbd78b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fddb8102390 0x7fddb8199bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:49.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.156+0000 7fddbd78b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fddb8102d50 0x7fddb819a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:49.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.156+0000 7fddbd78b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fddb8108400 0x7fddb819e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:49.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.156+0000 7fddbd78b700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fddb8112450 con 0x7fddb8102d50 2024-05-10T11:22:49.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.156+0000 7fddbd78b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fddb81122d0 con 0x7fddb8102390 2024-05-10T11:22:49.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.156+0000 7fddbd78b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fddb81125d0 con 0x7fddb8108400 2024-05-10T11:22:49.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.156+0000 7fddb67fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fddb8102d50 0x7fddb819a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:49.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.156+0000 7fddb6ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fddb8102390 0x7fddb8199bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:49.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.156+0000 7fddb6ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fddb8102390 0x7fddb8199bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:55808/0 (socket says 172.21.15.100:55808) 2024-05-10T11:22:49.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.156+0000 7fddb67fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fddb8102d50 0x7fddb819a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33438/0 (socket says 172.21.15.100:33438) 2024-05-10T11:22:49.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.156+0000 7fddb6ffd700 1 -- 172.21.15.100:0/1016184560 learned_addr learned my addr 172.21.15.100:0/1016184560 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:22:49.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.156+0000 7fddb77fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fddb8108400 0x7fddb819e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:49.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.156+0000 7fddb67fc700 1 -- 172.21.15.100:0/1016184560 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fddb8108400 msgr2=0x7fddb819e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:49.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.156+0000 7fddb67fc700 1 --2- 172.21.15.100:0/1016184560 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fddb8108400 0x7fddb819e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:49.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.156+0000 7fddb67fc700 1 -- 172.21.15.100:0/1016184560 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fddb8102390 msgr2=0x7fddb8199bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:49.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.156+0000 7fddb67fc700 1 --2- 172.21.15.100:0/1016184560 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fddb8102390 0x7fddb8199bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:49.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.156+0000 7fddb67fc700 1 -- 172.21.15.100:0/1016184560 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddb819eba0 con 0x7fddb8102d50 2024-05-10T11:22:49.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.157+0000 7fddb6ffd700 1 --2- 172.21.15.100:0/1016184560 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fddb8102390 0x7fddb8199bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:22:49.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.157+0000 7fddb67fc700 1 --2- 172.21.15.100:0/1016184560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fddb8102d50 0x7fddb819a130 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fddac00d750 tx=0x7fddac00da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:49.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.157+0000 7fdd9ffff700 1 -- 172.21.15.100:0/1016184560 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddac014030 con 0x7fddb8102d50 2024-05-10T11:22:49.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.157+0000 7fddbd78b700 1 -- 172.21.15.100:0/1016184560 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddb819ee90 con 0x7fddb8102d50 2024-05-10T11:22:49.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.157+0000 7fdd9ffff700 1 -- 172.21.15.100:0/1016184560 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fddac0044b0 con 0x7fddb8102d50 2024-05-10T11:22:49.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.157+0000 7fdd9ffff700 1 -- 172.21.15.100:0/1016184560 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddac005280 con 0x7fddb8102d50 2024-05-10T11:22:49.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.157+0000 7fddbd78b700 1 -- 172.21.15.100:0/1016184560 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddb81a6750 con 0x7fddb8102d50 2024-05-10T11:22:49.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.160+0000 7fdd9ffff700 1 -- 172.21.15.100:0/1016184560 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fddac0049e0 con 0x7fddb8102d50 2024-05-10T11:22:49.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.161+0000 7fdd9ffff700 1 --2- 172.21.15.100:0/1016184560 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fdda00b2880 0x7fdda00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:49.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.161+0000 7fddb6ffd700 1 --2- 172.21.15.100:0/1016184560 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fdda00b2880 0x7fdda00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:49.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.161+0000 7fdd9ffff700 1 -- 172.21.15.100:0/1016184560 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fddac0e1e10 con 0x7fddb8102d50 2024-05-10T11:22:49.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.161+0000 7fddc801eb80 1 Processor -- start 2024-05-10T11:22:49.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.161+0000 7fddc801eb80 1 -- start start 2024-05-10T11:22:49.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.161+0000 7fddc801eb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631325174d0 0x5631325178b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:49.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.161+0000 7fddb6ffd700 1 --2- 172.21.15.100:0/1016184560 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fdda00b2880 0x7fdda00b4d40 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fdda800b5c0 tx=0x7fdda801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:49.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.161+0000 7fddc801eb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563132517df0 0x56313251b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:49.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.161+0000 7fddc801eb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56313251bc80 0x56313251c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:49.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.161+0000 7fddc801eb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x563132469860 con 0x56313251bc80 2024-05-10T11:22:49.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.161+0000 7fddc801eb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5631324087c0 con 0x563132517df0 2024-05-10T11:22:49.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.161+0000 7fddc801eb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563132465820 con 0x5631325174d0 2024-05-10T11:22:49.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.161+0000 7fddb77fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56313251bc80 0x56313251c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:49.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.161+0000 7fddb6ffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631325174d0 0x5631325178b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:49.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.161+0000 7fddb77fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56313251bc80 0x56313251c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33446/0 (socket says 172.21.15.100:33446) 2024-05-10T11:22:49.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.161+0000 7fddb77fe700 1 -- 172.21.15.100:0/2238327832 learned_addr learned my addr 172.21.15.100:0/2238327832 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:22:49.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.161+0000 7fddb6ffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631325174d0 0x5631325178b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:46834/0 (socket says 172.21.15.100:46834) 2024-05-10T11:22:49.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.162+0000 7fddb67fc700 1 --2- 172.21.15.100:0/2238327832 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563132517df0 0x56313251b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:49.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.162+0000 7fddb77fe700 1 -- 172.21.15.100:0/2238327832 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631325174d0 msgr2=0x5631325178b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:49.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.162+0000 7fddb77fe700 1 --2- 172.21.15.100:0/2238327832 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631325174d0 0x5631325178b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:49.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.162+0000 7fddb77fe700 1 -- 172.21.15.100:0/2238327832 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563132517df0 msgr2=0x56313251b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:49.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.162+0000 7fddb77fe700 1 --2- 172.21.15.100:0/2238327832 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563132517df0 0x56313251b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:49.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.162+0000 7fddb77fe700 1 -- 172.21.15.100:0/2238327832 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631323b68d0 con 0x56313251bc80 2024-05-10T11:22:49.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.162+0000 7fddb67fc700 1 --2- 172.21.15.100:0/2238327832 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563132517df0 0x56313251b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:22:49.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.162+0000 7fddb77fe700 1 --2- 172.21.15.100:0/2238327832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56313251bc80 0x56313251c100 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fddb000a430 tx=0x7fddb0009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8c1ad7f83c7300f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:49.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.162+0000 7fdd9d7fa700 1 -- 172.21.15.100:0/2238327832 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb0014620 con 0x56313251bc80 2024-05-10T11:22:49.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.162+0000 7fdd9d7fa700 1 -- 172.21.15.100:0/2238327832 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fddb0019070 con 0x56313251bc80 2024-05-10T11:22:49.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.162+0000 7fdd9d7fa700 1 -- 172.21.15.100:0/2238327832 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb0013a20 con 0x56313251bc80 2024-05-10T11:22:49.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.163+0000 7fddc801eb80 1 -- 172.21.15.100:0/2238327832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56313251bc80 msgr2=0x56313251c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:49.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.163+0000 7fddc801eb80 1 --2- 172.21.15.100:0/2238327832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56313251bc80 0x56313251c100 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fddb000a430 tx=0x7fddb0009650 comp rx=0 tx=0).stop 2024-05-10T11:22:49.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.163+0000 7fddc801eb80 1 -- 172.21.15.100:0/2238327832 shutdown_connections 2024-05-10T11:22:49.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.163+0000 7fddc801eb80 1 --2- 172.21.15.100:0/2238327832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56313251bc80 0x56313251c100 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:49.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.163+0000 7fddc801eb80 1 --2- 172.21.15.100:0/2238327832 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631325174d0 0x5631325178b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:49.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.163+0000 7fddc801eb80 1 --2- 172.21.15.100:0/2238327832 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563132517df0 0x56313251b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:49.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.163+0000 7fddc801eb80 1 -- 172.21.15.100:0/2238327832 >> 172.21.15.100:0/2238327832 conn(0x563132443100 msgr2=0x563132511c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:49.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.163+0000 7fddc801eb80 1 -- 172.21.15.100:0/2238327832 shutdown_connections 2024-05-10T11:22:49.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.163+0000 7fddc801eb80 1 -- 172.21.15.100:0/2238327832 wait complete. 2024-05-10T11:22:49.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.163+0000 7fddc801eb80 1 -- start start 2024-05-10T11:22:49.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.163+0000 7fddc801eb80 1 Processor -- start 2024-05-10T11:22:49.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.164+0000 7fddc801eb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631325174d0 0x563132523030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:49.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.164+0000 7fddc801eb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563132517df0 0x563132520420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:49.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.164+0000 7fddc801eb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56313251bc80 0x563132520960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:49.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.164+0000 7fddc801eb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x563132523e90 con 0x5631325174d0 2024-05-10T11:22:49.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.164+0000 7fddc801eb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5631323cfc30 con 0x56313251bc80 2024-05-10T11:22:49.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.164+0000 7fddc801eb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563132524010 con 0x563132517df0 2024-05-10T11:22:49.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.164+0000 7fddc801eb80 10 client.0 _ll_register_callbacks cb 0x563132401920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:22:49.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.164+0000 7fddb6ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631325174d0 0x563132523030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:49.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.164+0000 7fddb67fc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563132517df0 0x563132520420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:49.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.164+0000 7fddb67fc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563132517df0 0x563132520420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:46838/0 (socket says 172.21.15.100:46838) 2024-05-10T11:22:49.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.164+0000 7fddb67fc700 1 -- 172.21.15.100:0/354892216 learned_addr learned my addr 172.21.15.100:0/354892216 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:22:49.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.164+0000 7fddb6ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631325174d0 0x563132523030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33448/0 (socket says 172.21.15.100:33448) 2024-05-10T11:22:49.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.164+0000 7fddb77fe700 1 --2- 172.21.15.100:0/354892216 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56313251bc80 0x563132520960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:49.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.164+0000 7fddb67fc700 1 -- 172.21.15.100:0/354892216 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56313251bc80 msgr2=0x563132520960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:49.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.164+0000 7fddb67fc700 1 --2- 172.21.15.100:0/354892216 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56313251bc80 0x563132520960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:49.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.164+0000 7fddb67fc700 1 -- 172.21.15.100:0/354892216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631325174d0 msgr2=0x563132523030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:49.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.164+0000 7fddb67fc700 1 --2- 172.21.15.100:0/354892216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631325174d0 0x563132523030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:49.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.164+0000 7fddb67fc700 1 -- 172.21.15.100:0/354892216 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563132405690 con 0x563132517df0 2024-05-10T11:22:49.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.164+0000 7fddb77fe700 1 --2- 172.21.15.100:0/354892216 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56313251bc80 0x563132520960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:22:49.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.164+0000 7fddb67fc700 1 --2- 172.21.15.100:0/354892216 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563132517df0 0x563132520420 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fddac005cb0 tx=0x7fddac0a9850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2e146c59c58ddfa server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:49.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.164+0000 7fdd9aff5700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:22:49.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.165+0000 7fdd9aff5700 1 -- 172.21.15.100:0/354892216 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddac01d030 con 0x563132517df0 2024-05-10T11:22:49.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.165+0000 7fdd9aff5700 1 -- 172.21.15.100:0/354892216 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fddac0b1cb0 con 0x563132517df0 2024-05-10T11:22:49.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.165+0000 7fddc801eb80 1 -- 172.21.15.100:0/354892216 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fddb0019070 con 0x563132517df0 2024-05-10T11:22:49.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.165+0000 7fdd9aff5700 1 -- 172.21.15.100:0/354892216 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fddac0ad480 con 0x563132517df0 2024-05-10T11:22:49.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.165+0000 7fddc801eb80 10 client.25753 fetch_fsmap learned FSMap version 30 2024-05-10T11:22:49.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.165+0000 7fddc801eb80 1 -- 172.21.15.100:0/354892216 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56313252e670 con 0x563132517df0 2024-05-10T11:22:49.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.165+0000 7fdd9aff5700 1 -- 172.21.15.100:0/354892216 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fddb0019070 con 0x563132517df0 2024-05-10T11:22:49.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.165+0000 7fdd9aff5700 1 -- 172.21.15.100:0/354892216 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7fddac0a8330 con 0x563132517df0 2024-05-10T11:22:49.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.165+0000 7fddc801eb80 10 client.25753 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T11:22:49.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.165+0000 7fddc801eb80 10 client.25753 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:22:49.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.165+0000 7fddc801eb80 20 client.25753 populate_metadata read hostname 'smithi100' 2024-05-10T11:22:49.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.165+0000 7fddc801eb80 1 --2- 172.21.15.100:0/354892216 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x563132530cb0 0x5631325330d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:49.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.165+0000 7fddc801eb80 4 client.25753 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:22:49.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.165+0000 7fddc801eb80 1 -- 172.21.15.100:0/354892216 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5631324e3650 con 0x563132530cb0 2024-05-10T11:22:49.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.166+0000 7fddb6ffd700 1 --2- 172.21.15.100:0/354892216 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x563132530cb0 0x5631325330d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:49.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.166+0000 7fddb6ffd700 1 --2- 172.21.15.100:0/354892216 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x563132530cb0 0x5631325330d0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dec4d4701d9e6f04 server_cookie=22d4c89f742c01fc in_seq=0 out_seq=0 2024-05-10T11:22:49.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.166+0000 7fdd9aff5700 10 client.25753 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:22:49.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.166+0000 7fdd9aff5700 1 -- 172.21.15.100:0/354892216 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5631324e3650 con 0x563132530cb0 2024-05-10T11:22:49.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.167+0000 7fdd9aff5700 10 client.25753 handle_command_reply: tid=0 2024-05-10T11:22:49.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.167+0000 7fddc801eb80 1 client.25753 shutdown 2024-05-10T11:22:49.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.167+0000 7fddc801eb80 1 -- 172.21.15.100:0/354892216 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563132517df0 msgr2=0x563132520420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:49.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.167+0000 7fddc801eb80 1 --2- 172.21.15.100:0/354892216 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563132517df0 0x563132520420 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fddac005cb0 tx=0x7fddac0a9850 comp rx=0 tx=0).stop 2024-05-10T11:22:49.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.167+0000 7fddc801eb80 1 -- 172.21.15.100:0/354892216 shutdown_connections 2024-05-10T11:22:49.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.167+0000 7fddc801eb80 1 --2- 172.21.15.100:0/354892216 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x563132530cb0 0x5631325330d0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:49.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.167+0000 7fddc801eb80 1 --2- 172.21.15.100:0/354892216 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56313251bc80 0x563132520960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:49.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.167+0000 7fddc801eb80 1 --2- 172.21.15.100:0/354892216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631325174d0 0x563132523030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:49.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.167+0000 7fdd9aff5700 0 client.25753 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:22:49.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.167+0000 7fddc801eb80 1 --2- 172.21.15.100:0/354892216 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563132517df0 0x563132520420 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:49.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.167+0000 7fddc801eb80 1 -- 172.21.15.100:0/354892216 >> 172.21.15.100:0/354892216 conn(0x563132443100 msgr2=0x56313251c080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:49.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.167+0000 7fddc801eb80 1 -- 172.21.15.100:0/354892216 shutdown_connections 2024-05-10T11:22:49.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.167+0000 7fddc801eb80 1 -- 172.21.15.100:0/354892216 wait complete. 2024-05-10T11:22:49.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.167+0000 7fddc801eb80 20 client.25753 trim_cache size 0 max 16384 2024-05-10T11:22:49.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.184+0000 7fddc801eb80 1 Processor -- start 2024-05-10T11:22:49.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.184+0000 7fddc801eb80 1 -- start start 2024-05-10T11:22:49.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.184+0000 7fddc801eb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631325174d0 0x56313252cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:49.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.184+0000 7fddc801eb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563132517df0 0x5631325330a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:49.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.184+0000 7fddc801eb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56313251bc80 0x56313250c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:49.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.184+0000 7fddc801eb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fddb81b1960 con 0x5631325174d0 2024-05-10T11:22:49.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.184+0000 7fddc801eb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fddb81af1f0 con 0x563132517df0 2024-05-10T11:22:49.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.184+0000 7fddc801eb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fddb81ad040 con 0x56313251bc80 2024-05-10T11:22:49.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.184+0000 7fddb6ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631325174d0 0x56313252cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:49.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.184+0000 7fddb67fc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563132517df0 0x5631325330a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:49.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.184+0000 7fddb67fc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563132517df0 0x5631325330a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:55838/0 (socket says 172.21.15.100:55838) 2024-05-10T11:22:49.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.184+0000 7fddb67fc700 1 -- 172.21.15.100:0/2989984735 learned_addr learned my addr 172.21.15.100:0/2989984735 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:22:49.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.184+0000 7fddb77fe700 1 --2- 172.21.15.100:0/2989984735 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56313251bc80 0x56313250c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:49.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.185+0000 7fddb6ffd700 1 -- 172.21.15.100:0/2989984735 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56313251bc80 msgr2=0x56313250c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:49.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.185+0000 7fddb6ffd700 1 --2- 172.21.15.100:0/2989984735 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56313251bc80 0x56313250c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:49.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.185+0000 7fddb6ffd700 1 -- 172.21.15.100:0/2989984735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563132517df0 msgr2=0x5631325330a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:49.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.185+0000 7fddb6ffd700 1 --2- 172.21.15.100:0/2989984735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563132517df0 0x5631325330a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:49.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.185+0000 7fddb6ffd700 1 -- 172.21.15.100:0/2989984735 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56313250e810 con 0x5631325174d0 2024-05-10T11:22:49.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.185+0000 7fddb77fe700 1 --2- 172.21.15.100:0/2989984735 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56313251bc80 0x56313250c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:22:49.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.185+0000 7fddb67fc700 1 --2- 172.21.15.100:0/2989984735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563132517df0 0x5631325330a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:22:49.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.185+0000 7fddb6ffd700 1 --2- 172.21.15.100:0/2989984735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631325174d0 0x56313252cf20 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x563132512c80 tx=0x7fdda8022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec3a073bfa7d8fca server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:49.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.185+0000 7fdd9cff9700 1 -- 172.21.15.100:0/2989984735 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda803a070 con 0x5631325174d0 2024-05-10T11:22:49.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.185+0000 7fdd9cff9700 1 -- 172.21.15.100:0/2989984735 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdda80056d0 con 0x5631325174d0 2024-05-10T11:22:49.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.185+0000 7fdd9cff9700 1 -- 172.21.15.100:0/2989984735 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda8033c00 con 0x5631325174d0 2024-05-10T11:22:49.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.186+0000 7fddc801eb80 1 -- 172.21.15.100:0/2989984735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631325174d0 msgr2=0x56313252cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:49.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.186+0000 7fddc801eb80 1 --2- 172.21.15.100:0/2989984735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631325174d0 0x56313252cf20 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x563132512c80 tx=0x7fdda8022000 comp rx=0 tx=0).stop 2024-05-10T11:22:49.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.186+0000 7fddc801eb80 1 -- 172.21.15.100:0/2989984735 shutdown_connections 2024-05-10T11:22:49.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.186+0000 7fddc801eb80 1 --2- 172.21.15.100:0/2989984735 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56313251bc80 0x56313250c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:49.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.186+0000 7fddc801eb80 1 --2- 172.21.15.100:0/2989984735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631325174d0 0x56313252cf20 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:49.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.186+0000 7fddc801eb80 1 --2- 172.21.15.100:0/2989984735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563132517df0 0x5631325330a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:49.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.186+0000 7fddc801eb80 1 -- 172.21.15.100:0/2989984735 >> 172.21.15.100:0/2989984735 conn(0x563132443100 msgr2=0x56313250be90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:49.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.186+0000 7fddc801eb80 1 -- 172.21.15.100:0/2989984735 shutdown_connections 2024-05-10T11:22:49.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.186+0000 7fddc801eb80 1 -- 172.21.15.100:0/2989984735 wait complete. 2024-05-10T11:22:49.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.186+0000 7fddc801eb80 1 -- start start 2024-05-10T11:22:49.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.187+0000 7fddc801eb80 1 Processor -- start 2024-05-10T11:22:49.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.187+0000 7fddc801eb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631325174d0 0x563132529250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:49.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.187+0000 7fddc801eb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563132517df0 0x563132529790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:49.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.187+0000 7fddc801eb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56313251bc80 0x563132524660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:49.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.187+0000 7fddc801eb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fddb81ad6b0 con 0x56313251bc80 2024-05-10T11:22:49.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.187+0000 7fddc801eb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fddb81aeb20 con 0x5631325174d0 2024-05-10T11:22:49.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.187+0000 7fddc801eb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fddb81add80 con 0x563132517df0 2024-05-10T11:22:49.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.187+0000 7fddc801eb80 10 client.0 _ll_register_callbacks cb 0x563132401920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:22:49.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.187+0000 7fddb6ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631325174d0 0x563132529250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:49.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.187+0000 7fddb67fc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563132517df0 0x563132529790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:49.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.187+0000 7fddb77fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56313251bc80 0x563132524660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:49.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.187+0000 7fddb6ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631325174d0 0x563132529250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:55844/0 (socket says 172.21.15.100:55844) 2024-05-10T11:22:49.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.187+0000 7fddb6ffd700 1 -- 172.21.15.100:0/4098191175 learned_addr learned my addr 172.21.15.100:0/4098191175 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:22:49.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.187+0000 7fddb77fe700 1 -- 172.21.15.100:0/4098191175 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563132517df0 msgr2=0x563132529790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:49.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.187+0000 7fddb77fe700 1 --2- 172.21.15.100:0/4098191175 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563132517df0 0x563132529790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:49.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.187+0000 7fddb77fe700 1 -- 172.21.15.100:0/4098191175 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631325174d0 msgr2=0x563132529250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:49.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.187+0000 7fddb77fe700 1 --2- 172.21.15.100:0/4098191175 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631325174d0 0x563132529250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:49.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.187+0000 7fddb77fe700 1 -- 172.21.15.100:0/4098191175 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563132524ea0 con 0x56313251bc80 2024-05-10T11:22:49.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.187+0000 7fddb6ffd700 1 --2- 172.21.15.100:0/4098191175 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631325174d0 0x563132529250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:22:49.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.187+0000 7fddb67fc700 1 --2- 172.21.15.100:0/4098191175 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563132517df0 0x563132529790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:22:49.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.188+0000 7fddb77fe700 1 --2- 172.21.15.100:0/4098191175 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56313251bc80 0x563132524660 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fddb000a400 tx=0x7fddb0014650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7df32667d4f4edb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:49.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.188+0000 7fdd9aff5700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:22:49.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.188+0000 7fdd9aff5700 1 -- 172.21.15.100:0/4098191175 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb0014bd0 con 0x56313251bc80 2024-05-10T11:22:49.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.188+0000 7fdd9aff5700 1 -- 172.21.15.100:0/4098191175 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fddb0014d70 con 0x56313251bc80 2024-05-10T11:22:49.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.188+0000 7fddc801eb80 1 -- 172.21.15.100:0/4098191175 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdda80056d0 con 0x56313251bc80 2024-05-10T11:22:49.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.188+0000 7fdd9aff5700 1 -- 172.21.15.100:0/4098191175 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fddb0031050 con 0x56313251bc80 2024-05-10T11:22:49.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.188+0000 7fddc801eb80 10 client.16263 fetch_fsmap learned FSMap version 30 2024-05-10T11:22:49.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.188+0000 7fddc801eb80 1 -- 172.21.15.100:0/4098191175 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56313259f4b0 con 0x56313251bc80 2024-05-10T11:22:49.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.188+0000 7fdd9aff5700 1 -- 172.21.15.100:0/4098191175 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdda80056d0 con 0x56313251bc80 2024-05-10T11:22:49.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.188+0000 7fdd9aff5700 1 -- 172.21.15.100:0/4098191175 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7fddb001d400 con 0x56313251bc80 2024-05-10T11:22:49.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.188+0000 7fddc801eb80 10 client.16263 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T11:22:49.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.188+0000 7fddc801eb80 10 client.16263 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:22:49.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.188+0000 7fddc801eb80 20 client.16263 populate_metadata read hostname 'smithi100' 2024-05-10T11:22:49.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.188+0000 7fddc801eb80 1 --2- 172.21.15.100:0/4098191175 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56313259f9d0 0x5631325a1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:49.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.189+0000 7fddc801eb80 4 client.16263 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:22:49.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.189+0000 7fddc801eb80 1 -- 172.21.15.100:0/4098191175 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563132477240 con 0x56313259f9d0 2024-05-10T11:22:49.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.189+0000 7fddb6ffd700 1 --2- 172.21.15.100:0/4098191175 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56313259f9d0 0x5631325a1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:49.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.189+0000 7fdd9aff5700 10 client.16263 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:22:49.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.189+0000 7fddb6ffd700 1 --2- 172.21.15.100:0/4098191175 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56313259f9d0 0x5631325a1df0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3c274f1b811910 server_cookie=44a6d337364f7fea in_seq=0 out_seq=0 2024-05-10T11:22:49.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.190+0000 7fdd9aff5700 1 -- 172.21.15.100:0/4098191175 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563132477240 con 0x56313259f9d0 2024-05-10T11:22:49.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.190+0000 7fdd9aff5700 10 client.16263 handle_command_reply: tid=0 2024-05-10T11:22:49.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.190+0000 7fddc801eb80 1 client.16263 shutdown 2024-05-10T11:22:49.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.190+0000 7fddc801eb80 1 -- 172.21.15.100:0/4098191175 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56313251bc80 msgr2=0x563132524660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:49.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.190+0000 7fddc801eb80 1 --2- 172.21.15.100:0/4098191175 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56313251bc80 0x563132524660 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fddb000a400 tx=0x7fddb0014650 comp rx=0 tx=0).stop 2024-05-10T11:22:49.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.190+0000 7fddc801eb80 1 -- 172.21.15.100:0/4098191175 shutdown_connections 2024-05-10T11:22:49.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.190+0000 7fddc801eb80 1 --2- 172.21.15.100:0/4098191175 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56313259f9d0 0x5631325a1df0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:49.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.190+0000 7fdd9aff5700 0 client.16263 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:22:49.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.190+0000 7fddc801eb80 1 --2- 172.21.15.100:0/4098191175 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56313251bc80 0x563132524660 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:49.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.190+0000 7fddc801eb80 1 --2- 172.21.15.100:0/4098191175 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631325174d0 0x563132529250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:49.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.190+0000 7fddc801eb80 1 --2- 172.21.15.100:0/4098191175 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563132517df0 0x563132529790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:49.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.190+0000 7fddc801eb80 1 -- 172.21.15.100:0/4098191175 >> 172.21.15.100:0/4098191175 conn(0x563132443100 msgr2=0x5631325162e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:49.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.190+0000 7fddc801eb80 1 -- 172.21.15.100:0/4098191175 shutdown_connections 2024-05-10T11:22:49.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.190+0000 7fddc801eb80 1 -- 172.21.15.100:0/4098191175 wait complete. 2024-05-10T11:22:49.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.190+0000 7fddc801eb80 20 client.16263 trim_cache size 0 max 16384 2024-05-10T11:22:49.196 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:22:49.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.190+0000 7fddbd78b700 1 -- 172.21.15.100:0/1016184560 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fdda00b2880 msgr2=0x7fdda00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:49.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.190+0000 7fddbd78b700 1 --2- 172.21.15.100:0/1016184560 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fdda00b2880 0x7fdda00b4d40 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fdda800b5c0 tx=0x7fdda801a040 comp rx=0 tx=0).stop 2024-05-10T11:22:49.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.190+0000 7fddbd78b700 1 -- 172.21.15.100:0/1016184560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fddb8102d50 msgr2=0x7fddb819a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:49.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.190+0000 7fddbd78b700 1 --2- 172.21.15.100:0/1016184560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fddb8102d50 0x7fddb819a130 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fddac00d750 tx=0x7fddac00da60 comp rx=0 tx=0).stop 2024-05-10T11:22:49.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.191+0000 7fddbd78b700 1 -- 172.21.15.100:0/1016184560 shutdown_connections 2024-05-10T11:22:49.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.191+0000 7fddbd78b700 1 --2- 172.21.15.100:0/1016184560 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fdda00b2880 0x7fdda00b4d40 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:49.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.191+0000 7fddbd78b700 1 --2- 172.21.15.100:0/1016184560 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fddb8108400 0x7fddb819e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:49.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.191+0000 7fddbd78b700 1 --2- 172.21.15.100:0/1016184560 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fddb8102390 0x7fddb8199bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:49.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.191+0000 7fddbd78b700 1 --2- 172.21.15.100:0/1016184560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fddb8102d50 0x7fddb819a130 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:49.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.191+0000 7fddbd78b700 1 -- 172.21.15.100:0/1016184560 >> 172.21.15.100:0/1016184560 conn(0x7fddb8079310 msgr2=0x7fddb806bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:49.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.191+0000 7fddbd78b700 1 -- 172.21.15.100:0/1016184560 shutdown_connections 2024-05-10T11:22:49.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:49.191+0000 7fddbd78b700 1 -- 172.21.15.100:0/1016184560 wait complete. 2024-05-10T11:22:49.804 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:22:49 smithi178 ceph-mon[33359]: pgmap v553: 129 pgs: 129 active+clean; 45 MiB data, 814 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 1.3 MiB/s wr, 135 op/s 2024-05-10T11:22:49.969 INFO:tasks.workunit.client.0.smithi100.stdout: 1 86437 2.67 MB/sec execute 44 sec latency 38.401 ms 2024-05-10T11:22:49.971 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:22:49 smithi100 ceph-mon[30296]: pgmap v553: 129 pgs: 129 active+clean; 45 MiB data, 814 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 1.3 MiB/s wr, 135 op/s 2024-05-10T11:22:50.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:22:49 smithi188 ceph-mon[34075]: pgmap v553: 129 pgs: 129 active+clean; 45 MiB data, 814 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 1.3 MiB/s wr, 135 op/s 2024-05-10T11:22:50.969 INFO:tasks.workunit.client.0.smithi100.stdout: 1 87133 2.69 MB/sec execute 45 sec latency 20.604 ms 2024-05-10T11:22:51.002 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T11:22:51.002 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T11:22:51.281 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:22:51.654 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:22:51 smithi100 ceph-mon[30296]: pgmap v554: 129 pgs: 129 active+clean; 47 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 1.3 MiB/s wr, 158 op/s 2024-05-10T11:22:51.969 INFO:tasks.workunit.client.0.smithi100.stdout: 1 87725 2.68 MB/sec execute 46 sec latency 14.517 ms 2024-05-10T11:22:52.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:22:51 smithi178 ceph-mon[33359]: pgmap v554: 129 pgs: 129 active+clean; 47 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 1.3 MiB/s wr, 158 op/s 2024-05-10T11:22:52.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:22:51 smithi188 ceph-mon[34075]: pgmap v554: 129 pgs: 129 active+clean; 47 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 1.3 MiB/s wr, 158 op/s 2024-05-10T11:22:52.969 INFO:tasks.workunit.client.0.smithi100.stdout: 1 88473 2.76 MB/sec execute 47 sec latency 14.965 ms 2024-05-10T11:22:53.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.107+0000 7fca377e7700 1 -- 172.21.15.100:0/822015853 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca30109530 msgr2=0x7fca30109910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:53.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.107+0000 7fca377e7700 1 --2- 172.21.15.100:0/822015853 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca30109530 0x7fca30109910 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fca2c009990 tx=0x7fca2c009ca0 comp rx=0 tx=0).stop 2024-05-10T11:22:53.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.107+0000 7fca377e7700 1 -- 172.21.15.100:0/822015853 shutdown_connections 2024-05-10T11:22:53.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.107+0000 7fca377e7700 1 --2- 172.21.15.100:0/822015853 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca30103ee0 0x7fca30110370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:53.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.107+0000 7fca377e7700 1 --2- 172.21.15.100:0/822015853 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca30109530 0x7fca30109910 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:53.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.107+0000 7fca377e7700 1 --2- 172.21.15.100:0/822015853 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca30103520 0x7fca301039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:53.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.107+0000 7fca377e7700 1 -- 172.21.15.100:0/822015853 >> 172.21.15.100:0/822015853 conn(0x7fca3007a650 msgr2=0x7fca3007aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:53.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.107+0000 7fca377e7700 1 -- 172.21.15.100:0/822015853 shutdown_connections 2024-05-10T11:22:53.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.107+0000 7fca377e7700 1 -- 172.21.15.100:0/822015853 wait complete. 2024-05-10T11:22:53.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.108+0000 7fca377e7700 1 Processor -- start 2024-05-10T11:22:53.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.108+0000 7fca377e7700 1 -- start start 2024-05-10T11:22:53.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.108+0000 7fca377e7700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca30103520 0x7fca3019f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:53.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.108+0000 7fca377e7700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca30103ee0 0x7fca3019fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:53.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.108+0000 7fca377e7700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca30109530 0x7fca30199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:53.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.108+0000 7fca377e7700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fca30113520 con 0x7fca30103ee0 2024-05-10T11:22:53.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.108+0000 7fca377e7700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fca301133a0 con 0x7fca30109530 2024-05-10T11:22:53.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.108+0000 7fca377e7700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fca301136a0 con 0x7fca30103520 2024-05-10T11:22:53.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.108+0000 7fca35583700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca30103520 0x7fca3019f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:53.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.108+0000 7fca35583700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca30103520 0x7fca3019f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:46870/0 (socket says 172.21.15.100:46870) 2024-05-10T11:22:53.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.108+0000 7fca35583700 1 -- 172.21.15.100:0/308875861 learned_addr learned my addr 172.21.15.100:0/308875861 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:22:53.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.108+0000 7fca34d82700 1 --2- 172.21.15.100:0/308875861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca30103ee0 0x7fca3019fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:53.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.108+0000 7fca35d84700 1 --2- 172.21.15.100:0/308875861 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca30109530 0x7fca30199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:53.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.108+0000 7fca35583700 1 -- 172.21.15.100:0/308875861 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca30109530 msgr2=0x7fca30199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:53.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.108+0000 7fca35583700 1 --2- 172.21.15.100:0/308875861 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca30109530 0x7fca30199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:53.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.108+0000 7fca35583700 1 -- 172.21.15.100:0/308875861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca30103ee0 msgr2=0x7fca3019fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:53.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.108+0000 7fca35583700 1 --2- 172.21.15.100:0/308875861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca30103ee0 0x7fca3019fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:53.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.108+0000 7fca35583700 1 -- 172.21.15.100:0/308875861 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca3019a000 con 0x7fca30103520 2024-05-10T11:22:53.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.109+0000 7fca35583700 1 --2- 172.21.15.100:0/308875861 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca30103520 0x7fca3019f6a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fca2c006010 tx=0x7fca2c00bfc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:53.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.109+0000 7fca267fc700 1 -- 172.21.15.100:0/308875861 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2c027070 con 0x7fca30103520 2024-05-10T11:22:53.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.109+0000 7fca377e7700 1 -- 172.21.15.100:0/308875861 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca3019a290 con 0x7fca30103520 2024-05-10T11:22:53.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.109+0000 7fca267fc700 1 -- 172.21.15.100:0/308875861 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fca2c022470 con 0x7fca30103520 2024-05-10T11:22:53.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.109+0000 7fca267fc700 1 -- 172.21.15.100:0/308875861 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2c017650 con 0x7fca30103520 2024-05-10T11:22:53.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.109+0000 7fca377e7700 1 -- 172.21.15.100:0/308875861 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca301a6500 con 0x7fca30103520 2024-05-10T11:22:53.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.111+0000 7fca267fc700 1 -- 172.21.15.100:0/308875861 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fca2c009d10 con 0x7fca30103520 2024-05-10T11:22:53.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.112+0000 7fca267fc700 1 --2- 172.21.15.100:0/308875861 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fca1c0b2880 0x7fca1c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:53.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.112+0000 7fca34d82700 1 --2- 172.21.15.100:0/308875861 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fca1c0b2880 0x7fca1c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:53.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.112+0000 7fca267fc700 1 -- 172.21.15.100:0/308875861 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fca2c0e3130 con 0x7fca30103520 2024-05-10T11:22:53.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.113+0000 7fca34d82700 1 --2- 172.21.15.100:0/308875861 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fca1c0b2880 0x7fca1c0b4d40 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fca3007ad30 tx=0x7fca2000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:53.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.113+0000 7fca4207ab80 1 Processor -- start 2024-05-10T11:22:53.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.113+0000 7fca4207ab80 1 -- start start 2024-05-10T11:22:53.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.113+0000 7fca4207ab80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb41242550 0x55eb41242930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:53.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.113+0000 7fca4207ab80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eb41242e70 0x55eb41246770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:53.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.113+0000 7fca4207ab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb41246d00 0x55eb41247180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:53.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.113+0000 7fca4207ab80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55eb411948e0 con 0x55eb41242550 2024-05-10T11:22:53.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.113+0000 7fca4207ab80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55eb41133840 con 0x55eb41242e70 2024-05-10T11:22:53.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.113+0000 7fca4207ab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55eb411908a0 con 0x55eb41246d00 2024-05-10T11:22:53.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.114+0000 7fca35583700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb41242550 0x55eb41242930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:53.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.114+0000 7fca35d84700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb41246d00 0x55eb41247180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:53.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.114+0000 7fca35583700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb41242550 0x55eb41242930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33490/0 (socket says 172.21.15.100:33490) 2024-05-10T11:22:53.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.114+0000 7fca35583700 1 -- 172.21.15.100:0/1488624987 learned_addr learned my addr 172.21.15.100:0/1488624987 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:22:53.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.114+0000 7fca34d82700 1 --2- 172.21.15.100:0/1488624987 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eb41242e70 0x55eb41246770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:53.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.114+0000 7fca34d82700 1 -- 172.21.15.100:0/1488624987 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb41246d00 msgr2=0x55eb41247180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:53.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.114+0000 7fca34d82700 1 --2- 172.21.15.100:0/1488624987 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb41246d00 0x55eb41247180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:53.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.114+0000 7fca34d82700 1 -- 172.21.15.100:0/1488624987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb41242550 msgr2=0x55eb41242930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:53.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.114+0000 7fca34d82700 1 --2- 172.21.15.100:0/1488624987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb41242550 0x55eb41242930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:53.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.114+0000 7fca34d82700 1 -- 172.21.15.100:0/1488624987 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb410e1950 con 0x55eb41242e70 2024-05-10T11:22:53.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.114+0000 7fca35583700 1 --2- 172.21.15.100:0/1488624987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb41242550 0x55eb41242930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:22:53.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.114+0000 7fca35d84700 1 --2- 172.21.15.100:0/1488624987 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb41246d00 0x55eb41247180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:22:53.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.114+0000 7fca34d82700 1 --2- 172.21.15.100:0/1488624987 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eb41242e70 0x55eb41246770 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fca20005d90 tx=0x7fca20013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c993503726a4b427 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:53.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.114+0000 7fca1b7fe700 1 -- 172.21.15.100:0/1488624987 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca20011b80 con 0x55eb41242e70 2024-05-10T11:22:53.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.114+0000 7fca1b7fe700 1 -- 172.21.15.100:0/1488624987 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fca20011d20 con 0x55eb41242e70 2024-05-10T11:22:53.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.114+0000 7fca1b7fe700 1 -- 172.21.15.100:0/1488624987 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2001e420 con 0x55eb41242e70 2024-05-10T11:22:53.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.115+0000 7fca4207ab80 1 -- 172.21.15.100:0/1488624987 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eb41242e70 msgr2=0x55eb41246770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:53.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.115+0000 7fca4207ab80 1 --2- 172.21.15.100:0/1488624987 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eb41242e70 0x55eb41246770 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fca20005d90 tx=0x7fca20013000 comp rx=0 tx=0).stop 2024-05-10T11:22:53.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.115+0000 7fca4207ab80 1 -- 172.21.15.100:0/1488624987 shutdown_connections 2024-05-10T11:22:53.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.115+0000 7fca4207ab80 1 --2- 172.21.15.100:0/1488624987 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb41246d00 0x55eb41247180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:53.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.115+0000 7fca4207ab80 1 --2- 172.21.15.100:0/1488624987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb41242550 0x55eb41242930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:53.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.115+0000 7fca4207ab80 1 --2- 172.21.15.100:0/1488624987 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eb41242e70 0x55eb41246770 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:53.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.115+0000 7fca4207ab80 1 -- 172.21.15.100:0/1488624987 >> 172.21.15.100:0/1488624987 conn(0x55eb4116e180 msgr2=0x55eb4123ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:53.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.115+0000 7fca4207ab80 1 -- 172.21.15.100:0/1488624987 shutdown_connections 2024-05-10T11:22:53.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.115+0000 7fca4207ab80 1 -- 172.21.15.100:0/1488624987 wait complete. 2024-05-10T11:22:53.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.115+0000 7fca4207ab80 1 -- start start 2024-05-10T11:22:53.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.116+0000 7fca4207ab80 1 Processor -- start 2024-05-10T11:22:53.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.116+0000 7fca4207ab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb41242550 0x55eb4124e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:53.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.116+0000 7fca4207ab80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eb41242e70 0x55eb4124b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:53.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.116+0000 7fca4207ab80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb41246d00 0x55eb4124ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:53.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.116+0000 7fca4207ab80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55eb4124ef10 con 0x55eb41246d00 2024-05-10T11:22:53.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.116+0000 7fca4207ab80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55eb410facb0 con 0x55eb41242e70 2024-05-10T11:22:53.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.116+0000 7fca4207ab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55eb4124f090 con 0x55eb41242550 2024-05-10T11:22:53.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.116+0000 7fca4207ab80 10 client.0 _ll_register_callbacks cb 0x55eb4112c9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:22:53.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.116+0000 7fca35583700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb41242550 0x55eb4124e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:53.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.116+0000 7fca35583700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb41242550 0x55eb4124e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:46890/0 (socket says 172.21.15.100:46890) 2024-05-10T11:22:53.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.116+0000 7fca35583700 1 -- 172.21.15.100:0/2275421176 learned_addr learned my addr 172.21.15.100:0/2275421176 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:22:53.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.116+0000 7fca35d84700 1 --2- 172.21.15.100:0/2275421176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb41246d00 0x55eb4124ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:53.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.116+0000 7fca34d82700 1 --2- 172.21.15.100:0/2275421176 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eb41242e70 0x55eb4124b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:53.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.117+0000 7fca35583700 1 -- 172.21.15.100:0/2275421176 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eb41242e70 msgr2=0x55eb4124b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:53.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.117+0000 7fca35583700 1 --2- 172.21.15.100:0/2275421176 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eb41242e70 0x55eb4124b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:53.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.117+0000 7fca35583700 1 -- 172.21.15.100:0/2275421176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb41246d00 msgr2=0x55eb4124ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:53.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.117+0000 7fca35583700 1 --2- 172.21.15.100:0/2275421176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb41246d00 0x55eb4124ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:53.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.117+0000 7fca35583700 1 -- 172.21.15.100:0/2275421176 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb41130710 con 0x55eb41242550 2024-05-10T11:22:53.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.117+0000 7fca35583700 1 --2- 172.21.15.100:0/2275421176 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb41242550 0x55eb4124e140 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fca2c00b2e0 tx=0x7fca2c022c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36e611a21987ba70 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:53.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.117+0000 7fca18ff9700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:22:53.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.117+0000 7fca18ff9700 1 -- 172.21.15.100:0/2275421176 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2c0aa420 con 0x55eb41242550 2024-05-10T11:22:53.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.117+0000 7fca18ff9700 1 -- 172.21.15.100:0/2275421176 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fca2c0b2f70 con 0x55eb41242550 2024-05-10T11:22:53.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.117+0000 7fca4207ab80 1 -- 172.21.15.100:0/2275421176 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca20011d20 con 0x55eb41242550 2024-05-10T11:22:53.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.117+0000 7fca18ff9700 1 -- 172.21.15.100:0/2275421176 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fca2c0ae7d0 con 0x55eb41242550 2024-05-10T11:22:53.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.117+0000 7fca4207ab80 10 client.25771 fetch_fsmap learned FSMap version 30 2024-05-10T11:22:53.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.117+0000 7fca4207ab80 1 -- 172.21.15.100:0/2275421176 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55eb4124ef70 con 0x55eb41242550 2024-05-10T11:22:53.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.117+0000 7fca18ff9700 1 -- 172.21.15.100:0/2275421176 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fca20011d20 con 0x55eb41242550 2024-05-10T11:22:53.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.117+0000 7fca18ff9700 1 -- 172.21.15.100:0/2275421176 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7fca2c0a5eb0 con 0x55eb41242550 2024-05-10T11:22:53.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.118+0000 7fca4207ab80 10 client.25771 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T11:22:53.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.118+0000 7fca4207ab80 10 client.25771 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:22:53.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.118+0000 7fca4207ab80 20 client.25771 populate_metadata read hostname 'smithi100' 2024-05-10T11:22:53.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.118+0000 7fca4207ab80 1 --2- 172.21.15.100:0/2275421176 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55eb4125bec0 0x55eb4125e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:53.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.118+0000 7fca4207ab80 4 client.25771 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:22:53.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.118+0000 7fca4207ab80 1 -- 172.21.15.100:0/2275421176 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb4120e6d0 con 0x55eb4125bec0 2024-05-10T11:22:53.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.118+0000 7fca34d82700 1 --2- 172.21.15.100:0/2275421176 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55eb4125bec0 0x55eb4125e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:53.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.118+0000 7fca34d82700 1 --2- 172.21.15.100:0/2275421176 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55eb4125bec0 0x55eb4125e2b0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4afb0fd825af6ff2 server_cookie=58f1f8f008562735 in_seq=0 out_seq=0 2024-05-10T11:22:53.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.118+0000 7fca18ff9700 10 client.25771 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:22:53.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.119+0000 7fca18ff9700 1 -- 172.21.15.100:0/2275421176 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55eb4120e6d0 con 0x55eb4125bec0 2024-05-10T11:22:53.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.119+0000 7fca18ff9700 10 client.25771 handle_command_reply: tid=0 2024-05-10T11:22:53.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.119+0000 7fca4207ab80 1 client.25771 shutdown 2024-05-10T11:22:53.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.119+0000 7fca4207ab80 1 -- 172.21.15.100:0/2275421176 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb41242550 msgr2=0x55eb4124e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:53.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.119+0000 7fca4207ab80 1 --2- 172.21.15.100:0/2275421176 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb41242550 0x55eb4124e140 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fca2c00b2e0 tx=0x7fca2c022c00 comp rx=0 tx=0).stop 2024-05-10T11:22:53.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.119+0000 7fca4207ab80 1 -- 172.21.15.100:0/2275421176 shutdown_connections 2024-05-10T11:22:53.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.119+0000 7fca4207ab80 1 --2- 172.21.15.100:0/2275421176 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55eb4125bec0 0x55eb4125e2b0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:53.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.119+0000 7fca4207ab80 1 --2- 172.21.15.100:0/2275421176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb41246d00 0x55eb4124ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:53.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.119+0000 7fca4207ab80 1 --2- 172.21.15.100:0/2275421176 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb41242550 0x55eb4124e140 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:53.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.119+0000 7fca4207ab80 1 --2- 172.21.15.100:0/2275421176 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eb41242e70 0x55eb4124b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:53.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.119+0000 7fca4207ab80 1 -- 172.21.15.100:0/2275421176 >> 172.21.15.100:0/2275421176 conn(0x55eb4116e180 msgr2=0x55eb412377a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:53.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.119+0000 7fca18ff9700 0 client.25771 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:22:53.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.119+0000 7fca4207ab80 1 -- 172.21.15.100:0/2275421176 shutdown_connections 2024-05-10T11:22:53.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.119+0000 7fca4207ab80 1 -- 172.21.15.100:0/2275421176 wait complete. 2024-05-10T11:22:53.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.120+0000 7fca4207ab80 20 client.25771 trim_cache size 0 max 16384 2024-05-10T11:22:53.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.145+0000 7fca4207ab80 1 Processor -- start 2024-05-10T11:22:53.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.145+0000 7fca4207ab80 1 -- start start 2024-05-10T11:22:53.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.145+0000 7fca4207ab80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb41242550 0x55eb41257fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:53.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.145+0000 7fca4207ab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb41242e70 0x55eb41263ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:53.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.145+0000 7fca4207ab80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eb41246d00 0x55eb41264000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:53.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.145+0000 7fca4207ab80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fca301b1750 con 0x55eb41242550 2024-05-10T11:22:53.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.145+0000 7fca4207ab80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fca301aefe0 con 0x55eb41246d00 2024-05-10T11:22:53.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.145+0000 7fca4207ab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fca301ace30 con 0x55eb41242e70 2024-05-10T11:22:53.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.146+0000 7fca35583700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb41242550 0x55eb41257fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:53.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.146+0000 7fca35583700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb41242550 0x55eb41257fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33518/0 (socket says 172.21.15.100:33518) 2024-05-10T11:22:53.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.146+0000 7fca35583700 1 -- 172.21.15.100:0/2716197854 learned_addr learned my addr 172.21.15.100:0/2716197854 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:22:53.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.146+0000 7fca34d82700 1 --2- 172.21.15.100:0/2716197854 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb41242e70 0x55eb41263ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:53.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.146+0000 7fca35583700 1 -- 172.21.15.100:0/2716197854 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb41242e70 msgr2=0x55eb41263ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:53.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.146+0000 7fca35583700 1 --2- 172.21.15.100:0/2716197854 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb41242e70 0x55eb41263ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:53.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.146+0000 7fca35583700 1 -- 172.21.15.100:0/2716197854 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eb41246d00 msgr2=0x55eb41264000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:53.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.146+0000 7fca35583700 1 --2- 172.21.15.100:0/2716197854 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eb41246d00 0x55eb41264000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:53.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.146+0000 7fca35583700 1 -- 172.21.15.100:0/2716197854 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb41246ac0 con 0x55eb41242550 2024-05-10T11:22:53.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.146+0000 7fca35583700 1 --2- 172.21.15.100:0/2716197854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb41242550 0x55eb41257fa0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fca2c0059d0 tx=0x7fca2c022830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9899d9cd56ef85fa server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:53.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.146+0000 7fca1affd700 1 -- 172.21.15.100:0/2716197854 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2c0b31f0 con 0x55eb41242550 2024-05-10T11:22:53.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.146+0000 7fca1affd700 1 -- 172.21.15.100:0/2716197854 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fca2c0a5d90 con 0x55eb41242550 2024-05-10T11:22:53.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.146+0000 7fca1affd700 1 -- 172.21.15.100:0/2716197854 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2c0a98e0 con 0x55eb41242550 2024-05-10T11:22:53.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.147+0000 7fca4207ab80 1 -- 172.21.15.100:0/2716197854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb41242550 msgr2=0x55eb41257fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:53.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.147+0000 7fca4207ab80 1 --2- 172.21.15.100:0/2716197854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb41242550 0x55eb41257fa0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fca2c0059d0 tx=0x7fca2c022830 comp rx=0 tx=0).stop 2024-05-10T11:22:53.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.147+0000 7fca4207ab80 1 -- 172.21.15.100:0/2716197854 shutdown_connections 2024-05-10T11:22:53.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.147+0000 7fca4207ab80 1 --2- 172.21.15.100:0/2716197854 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eb41246d00 0x55eb41264000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:53.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.147+0000 7fca4207ab80 1 --2- 172.21.15.100:0/2716197854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb41242550 0x55eb41257fa0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:53.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.147+0000 7fca4207ab80 1 --2- 172.21.15.100:0/2716197854 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb41242e70 0x55eb41263ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:53.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.147+0000 7fca4207ab80 1 -- 172.21.15.100:0/2716197854 >> 172.21.15.100:0/2716197854 conn(0x55eb4116e180 msgr2=0x55eb412377a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:53.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.147+0000 7fca4207ab80 1 -- 172.21.15.100:0/2716197854 shutdown_connections 2024-05-10T11:22:53.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.147+0000 7fca4207ab80 1 -- 172.21.15.100:0/2716197854 wait complete. 2024-05-10T11:22:53.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.147+0000 7fca4207ab80 1 -- start start 2024-05-10T11:22:53.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.148+0000 7fca4207ab80 1 Processor -- start 2024-05-10T11:22:53.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.148+0000 7fca4207ab80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eb41242550 0x55eb41250150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:53.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.148+0000 7fca4207ab80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb41242e70 0x55eb41250690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:53.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.148+0000 7fca4207ab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb41246d00 0x55eb41250bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:53.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.148+0000 7fca4207ab80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fca301ad4a0 con 0x55eb41242e70 2024-05-10T11:22:53.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.148+0000 7fca4207ab80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fca301ae910 con 0x55eb41242550 2024-05-10T11:22:53.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.148+0000 7fca4207ab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fca301adb70 con 0x55eb41246d00 2024-05-10T11:22:53.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.148+0000 7fca4207ab80 10 client.0 _ll_register_callbacks cb 0x55eb4112c9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:22:53.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.148+0000 7fca34d82700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb41242e70 0x55eb41250690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:53.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.148+0000 7fca35d84700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb41246d00 0x55eb41250bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:53.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.148+0000 7fca34d82700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb41242e70 0x55eb41250690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33534/0 (socket says 172.21.15.100:33534) 2024-05-10T11:22:53.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.148+0000 7fca34d82700 1 -- 172.21.15.100:0/131156705 learned_addr learned my addr 172.21.15.100:0/131156705 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:22:53.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.148+0000 7fca35d84700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb41246d00 0x55eb41250bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:46906/0 (socket says 172.21.15.100:46906) 2024-05-10T11:22:53.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.148+0000 7fca35583700 1 --2- 172.21.15.100:0/131156705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eb41242550 0x55eb41250150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:53.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.148+0000 7fca35d84700 1 -- 172.21.15.100:0/131156705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eb41242550 msgr2=0x55eb41250150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:53.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.149+0000 7fca35d84700 1 --2- 172.21.15.100:0/131156705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eb41242550 0x55eb41250150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:53.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.149+0000 7fca35d84700 1 -- 172.21.15.100:0/131156705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb41242e70 msgr2=0x55eb41250690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:53.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.149+0000 7fca35d84700 1 --2- 172.21.15.100:0/131156705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb41242e70 0x55eb41250690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:53.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.149+0000 7fca35d84700 1 -- 172.21.15.100:0/131156705 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb412c5920 con 0x55eb41246d00 2024-05-10T11:22:53.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.149+0000 7fca34d82700 1 --2- 172.21.15.100:0/131156705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb41242e70 0x55eb41250690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:22:53.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.149+0000 7fca35583700 1 --2- 172.21.15.100:0/131156705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eb41242550 0x55eb41250150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:22:53.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.149+0000 7fca35d84700 1 --2- 172.21.15.100:0/131156705 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb41246d00 0x55eb41250bd0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x55eb4123e790 tx=0x7fca280045a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6a9bf738a0cad6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:53.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.149+0000 7fca18ff9700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:22:53.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.149+0000 7fca18ff9700 1 -- 172.21.15.100:0/131156705 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca28007910 con 0x55eb41246d00 2024-05-10T11:22:53.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.149+0000 7fca4207ab80 1 -- 172.21.15.100:0/131156705 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca2c0a5d90 con 0x55eb41246d00 2024-05-10T11:22:53.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.149+0000 7fca18ff9700 1 -- 172.21.15.100:0/131156705 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fca28007ab0 con 0x55eb41246d00 2024-05-10T11:22:53.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.149+0000 7fca18ff9700 1 -- 172.21.15.100:0/131156705 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fca28007cb0 con 0x55eb41246d00 2024-05-10T11:22:53.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.149+0000 7fca4207ab80 10 client.25777 fetch_fsmap learned FSMap version 30 2024-05-10T11:22:53.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.149+0000 7fca4207ab80 1 -- 172.21.15.100:0/131156705 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55eb412c6cc0 con 0x55eb41246d00 2024-05-10T11:22:53.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.149+0000 7fca18ff9700 1 -- 172.21.15.100:0/131156705 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fca2c0a5d90 con 0x55eb41246d00 2024-05-10T11:22:53.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.149+0000 7fca18ff9700 1 -- 172.21.15.100:0/131156705 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7fca2801e070 con 0x55eb41246d00 2024-05-10T11:22:53.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.149+0000 7fca4207ab80 10 client.25777 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T11:22:53.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.150+0000 7fca4207ab80 10 client.25777 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:22:53.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.150+0000 7fca4207ab80 20 client.25777 populate_metadata read hostname 'smithi100' 2024-05-10T11:22:53.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.150+0000 7fca4207ab80 1 --2- 172.21.15.100:0/131156705 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55eb412c7480 0x55eb412c98a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:53.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.150+0000 7fca4207ab80 4 client.25777 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T11:22:53.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.150+0000 7fca4207ab80 1 -- 172.21.15.100:0/131156705 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55eb411a22c0 con 0x55eb412c7480 2024-05-10T11:22:53.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.150+0000 7fca35583700 1 --2- 172.21.15.100:0/131156705 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55eb412c7480 0x55eb412c98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:53.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.150+0000 7fca35583700 1 --2- 172.21.15.100:0/131156705 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55eb412c7480 0x55eb412c98a0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6dbe3e0012bc770 server_cookie=7091b9e29c405eaf in_seq=0 out_seq=0 2024-05-10T11:22:53.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.150+0000 7fca18ff9700 10 client.25777 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:22:53.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.151+0000 7fca18ff9700 1 -- 172.21.15.100:0/131156705 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55eb411a22c0 con 0x55eb412c7480 2024-05-10T11:22:53.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.151+0000 7fca18ff9700 10 client.25777 handle_command_reply: tid=0 2024-05-10T11:22:53.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.151+0000 7fca4207ab80 1 client.25777 shutdown 2024-05-10T11:22:53.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.151+0000 7fca4207ab80 1 -- 172.21.15.100:0/131156705 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb41246d00 msgr2=0x55eb41250bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:53.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.151+0000 7fca4207ab80 1 --2- 172.21.15.100:0/131156705 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb41246d00 0x55eb41250bd0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x55eb4123e790 tx=0x7fca280045a0 comp rx=0 tx=0).stop 2024-05-10T11:22:53.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.151+0000 7fca4207ab80 1 -- 172.21.15.100:0/131156705 shutdown_connections 2024-05-10T11:22:53.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.151+0000 7fca4207ab80 1 --2- 172.21.15.100:0/131156705 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55eb412c7480 0x55eb412c98a0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:53.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.151+0000 7fca4207ab80 1 --2- 172.21.15.100:0/131156705 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb41246d00 0x55eb41250bd0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:53.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.151+0000 7fca18ff9700 0 client.25777 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:22:53.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.151+0000 7fca4207ab80 1 --2- 172.21.15.100:0/131156705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eb41242550 0x55eb41250150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:53.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.151+0000 7fca4207ab80 1 --2- 172.21.15.100:0/131156705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb41242e70 0x55eb41250690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:53.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.151+0000 7fca4207ab80 1 -- 172.21.15.100:0/131156705 >> 172.21.15.100:0/131156705 conn(0x55eb4116e180 msgr2=0x55eb41241360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:53.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.152+0000 7fca4207ab80 1 -- 172.21.15.100:0/131156705 shutdown_connections 2024-05-10T11:22:53.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.152+0000 7fca4207ab80 1 -- 172.21.15.100:0/131156705 wait complete. 2024-05-10T11:22:53.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.152+0000 7fca4207ab80 20 client.25777 trim_cache size 0 max 16384 2024-05-10T11:22:53.157 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:22:53.157 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-05-10T11:22:53.158 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "bbb8fab3-fdcf-4321-9986-0d3fe39e79ec", 2024-05-10T11:22:53.158 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-05-10T11:22:53.158 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:22:53.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.152+0000 7fca377e7700 1 -- 172.21.15.100:0/308875861 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fca1c0b2880 msgr2=0x7fca1c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:53.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.152+0000 7fca377e7700 1 --2- 172.21.15.100:0/308875861 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fca1c0b2880 0x7fca1c0b4d40 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fca3007ad30 tx=0x7fca2000b410 comp rx=0 tx=0).stop 2024-05-10T11:22:53.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.152+0000 7fca377e7700 1 -- 172.21.15.100:0/308875861 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca30103520 msgr2=0x7fca3019f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:53.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.152+0000 7fca377e7700 1 --2- 172.21.15.100:0/308875861 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca30103520 0x7fca3019f6a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fca2c006010 tx=0x7fca2c00bfc0 comp rx=0 tx=0).stop 2024-05-10T11:22:53.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.153+0000 7fca377e7700 1 -- 172.21.15.100:0/308875861 shutdown_connections 2024-05-10T11:22:53.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.153+0000 7fca377e7700 1 --2- 172.21.15.100:0/308875861 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fca1c0b2880 0x7fca1c0b4d40 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:53.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.153+0000 7fca377e7700 1 --2- 172.21.15.100:0/308875861 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca30109530 0x7fca30199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:53.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.153+0000 7fca377e7700 1 --2- 172.21.15.100:0/308875861 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca30103520 0x7fca3019f6a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:53.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.153+0000 7fca377e7700 1 --2- 172.21.15.100:0/308875861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca30103ee0 0x7fca3019fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:53.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.153+0000 7fca377e7700 1 -- 172.21.15.100:0/308875861 >> 172.21.15.100:0/308875861 conn(0x7fca3007a650 msgr2=0x7fca301023c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:53.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.153+0000 7fca377e7700 1 -- 172.21.15.100:0/308875861 shutdown_connections 2024-05-10T11:22:53.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:53.153+0000 7fca377e7700 1 -- 172.21.15.100:0/308875861 wait complete. 2024-05-10T11:22:53.969 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:22:53 smithi100 ceph-mon[30296]: pgmap v555: 129 pgs: 129 active+clean; 47 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 1.5 MiB/s wr, 167 op/s 2024-05-10T11:22:53.970 INFO:tasks.workunit.client.0.smithi100.stdout: 1 88937 2.80 MB/sec execute 48 sec latency 27.586 ms 2024-05-10T11:22:53.998 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T11:22:54.001 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:22:53 smithi178 ceph-mon[33359]: pgmap v555: 129 pgs: 129 active+clean; 47 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 1.5 MiB/s wr, 167 op/s 2024-05-10T11:22:54.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:22:53 smithi188 ceph-mon[34075]: pgmap v555: 129 pgs: 129 active+clean; 47 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 1.5 MiB/s wr, 167 op/s 2024-05-10T11:22:54.256 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:22:54.724 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:22:54 smithi100 ceph-mon[30296]: scrub queued for path: / 2024-05-10T11:22:54.725 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:22:54 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:22:54.725 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:22:54 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:22:54.725 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:22:54 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:22:54.725 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:22:54 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:22:54.725 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:22:54 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:22:54.725 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:22:54 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:22:54.725 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:22:54 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:22:54.725 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:22:54 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:22:54.725 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:22:54 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:22:54.725 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:22:54 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:22:54.725 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:22:54 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:22:54.725 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:22:54 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:22:54.725 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:22:54 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:22:54.969 INFO:tasks.workunit.client.0.smithi100.stdout: 1 89189 2.75 MB/sec execute 49 sec latency 13.201 ms 2024-05-10T11:22:55.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:22:54 smithi178 ceph-mon[33359]: scrub queued for path: / 2024-05-10T11:22:55.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:22:54 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:22:55.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:22:54 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:22:55.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:22:54 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:22:55.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:22:54 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:22:55.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:22:54 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:22:55.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:22:54 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:22:55.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:22:54 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:22:55.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:22:54 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:22:55.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:22:54 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:22:55.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:22:54 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:22:55.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:22:54 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:22:55.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:22:54 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:22:55.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:22:54 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:22:55.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:22:54 smithi188 ceph-mon[34075]: scrub queued for path: / 2024-05-10T11:22:55.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:22:54 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:22:55.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:22:54 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:22:55.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:22:54 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:22:55.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:22:54 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:22:55.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:22:54 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:22:55.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:22:54 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:22:55.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:22:54 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:22:55.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:22:54 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:22:55.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:22:54 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:22:55.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:22:54 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:22:55.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:22:54 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:22:55.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:22:54 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:22:55.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:22:54 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:22:55.781 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:22:55 smithi100 ceph-mon[30296]: pgmap v556: 129 pgs: 129 active+clean; 47 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 1.4 MiB/s wr, 141 op/s 2024-05-10T11:22:55.969 INFO:tasks.workunit.client.0.smithi100.stdout: 1 89589 2.71 MB/sec execute 50 sec latency 41.424 ms 2024-05-10T11:22:56.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:22:55 smithi178 ceph-mon[33359]: pgmap v556: 129 pgs: 129 active+clean; 47 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 1.4 MiB/s wr, 141 op/s 2024-05-10T11:22:56.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:22:55 smithi188 ceph-mon[34075]: pgmap v556: 129 pgs: 129 active+clean; 47 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 1.4 MiB/s wr, 141 op/s 2024-05-10T11:22:56.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.174+0000 7f35982e7700 1 -- 172.21.15.100:0/3573186945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f359006b180 msgr2=0x7f359006b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:56.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.174+0000 7f35982e7700 1 --2- 172.21.15.100:0/3573186945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f359006b180 0x7f359006b600 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f3580009990 tx=0x7f3580009ca0 comp rx=0 tx=0).stop 2024-05-10T11:22:56.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.174+0000 7f35982e7700 1 -- 172.21.15.100:0/3573186945 shutdown_connections 2024-05-10T11:22:56.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.174+0000 7f35982e7700 1 --2- 172.21.15.100:0/3573186945 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f359006bb40 0x7f3590110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:56.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.174+0000 7f35982e7700 1 --2- 172.21.15.100:0/3573186945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3590109510 0x7f35901098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:56.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.174+0000 7f35982e7700 1 --2- 172.21.15.100:0/3573186945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f359006b180 0x7f359006b600 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:56.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.174+0000 7f35982e7700 1 -- 172.21.15.100:0/3573186945 >> 172.21.15.100:0/3573186945 conn(0x7f3590079350 msgr2=0x7f3590079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:56.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.174+0000 7f35982e7700 1 -- 172.21.15.100:0/3573186945 shutdown_connections 2024-05-10T11:22:56.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.174+0000 7f35982e7700 1 -- 172.21.15.100:0/3573186945 wait complete. 2024-05-10T11:22:56.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.175+0000 7f35982e7700 1 Processor -- start 2024-05-10T11:22:56.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.175+0000 7f35982e7700 1 -- start start 2024-05-10T11:22:56.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.175+0000 7f35982e7700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f359006b180 0x7f359019f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:56.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.175+0000 7f35982e7700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f359006bb40 0x7f359019fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:56.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.175+0000 7f35982e7700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3590109510 0x7f35901998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:56.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.175+0000 7f35982e7700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f35901134f0 con 0x7f359006b180 2024-05-10T11:22:56.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.175+0000 7f35982e7700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3590113370 con 0x7f359006bb40 2024-05-10T11:22:56.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.175+0000 7f35982e7700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3590113670 con 0x7f3590109510 2024-05-10T11:22:56.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.175+0000 7f3596083700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f359006b180 0x7f359019f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:56.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.175+0000 7f3595882700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f359006bb40 0x7f359019fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:56.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.175+0000 7f3596083700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f359006b180 0x7f359019f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33552/0 (socket says 172.21.15.100:33552) 2024-05-10T11:22:56.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.175+0000 7f3596083700 1 -- 172.21.15.100:0/3902815699 learned_addr learned my addr 172.21.15.100:0/3902815699 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:22:56.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.175+0000 7f3596884700 1 --2- 172.21.15.100:0/3902815699 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3590109510 0x7f35901998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:56.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.176+0000 7f3596083700 1 -- 172.21.15.100:0/3902815699 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3590109510 msgr2=0x7f35901998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:56.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.176+0000 7f3596083700 1 --2- 172.21.15.100:0/3902815699 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3590109510 0x7f35901998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:56.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.176+0000 7f3596083700 1 -- 172.21.15.100:0/3902815699 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f359006bb40 msgr2=0x7f359019fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:56.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.176+0000 7f3596083700 1 --2- 172.21.15.100:0/3902815699 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f359006bb40 0x7f359019fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:56.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.176+0000 7f3596083700 1 -- 172.21.15.100:0/3902815699 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f359019a050 con 0x7f359006b180 2024-05-10T11:22:56.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.176+0000 7f3596083700 1 --2- 172.21.15.100:0/3902815699 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f359006b180 0x7f359019f6e0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f358c00d950 tx=0x7f358c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:56.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.176+0000 7f35877fe700 1 -- 172.21.15.100:0/3902815699 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358c016070 con 0x7f359006b180 2024-05-10T11:22:56.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.176+0000 7f35982e7700 1 -- 172.21.15.100:0/3902815699 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f359019a340 con 0x7f359006b180 2024-05-10T11:22:56.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.176+0000 7f35877fe700 1 -- 172.21.15.100:0/3902815699 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f358c00fd20 con 0x7f359006b180 2024-05-10T11:22:56.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.176+0000 7f35877fe700 1 -- 172.21.15.100:0/3902815699 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358c00e3b0 con 0x7f359006b180 2024-05-10T11:22:56.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.176+0000 7f35982e7700 1 -- 172.21.15.100:0/3902815699 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35901a65b0 con 0x7f359006b180 2024-05-10T11:22:56.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.179+0000 7f35877fe700 1 -- 172.21.15.100:0/3902815699 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f358c019070 con 0x7f359006b180 2024-05-10T11:22:56.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.179+0000 7f35877fe700 1 --2- 172.21.15.100:0/3902815699 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f357c0b2880 0x7f357c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:56.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.179+0000 7f3595882700 1 --2- 172.21.15.100:0/3902815699 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f357c0b2880 0x7f357c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:56.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.180+0000 7f35877fe700 1 -- 172.21.15.100:0/3902815699 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f358c0e2000 con 0x7f359006b180 2024-05-10T11:22:56.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.180+0000 7f3595882700 1 --2- 172.21.15.100:0/3902815699 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f357c0b2880 0x7f357c0b4d40 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3590079b20 tx=0x7f3580005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:56.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.180+0000 7f35a2b7ab80 1 Processor -- start 2024-05-10T11:22:56.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.180+0000 7f35a2b7ab80 1 -- start start 2024-05-10T11:22:56.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.180+0000 7f35a2b7ab80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56183fddc4d0 0x56183fddc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:56.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.180+0000 7f35a2b7ab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56183fddcdf0 0x56183fde06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:56.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.180+0000 7f35a2b7ab80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56183fde0c80 0x56183fde1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:56.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.180+0000 7f35a2b7ab80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56183fd2e860 con 0x56183fddc4d0 2024-05-10T11:22:56.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.180+0000 7f35a2b7ab80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56183fccd7c0 con 0x56183fde0c80 2024-05-10T11:22:56.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.180+0000 7f35a2b7ab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56183fd2a820 con 0x56183fddcdf0 2024-05-10T11:22:56.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.181+0000 7f3596083700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56183fddc4d0 0x56183fddc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:56.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.181+0000 7f3596083700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56183fddc4d0 0x56183fddc8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33554/0 (socket says 172.21.15.100:33554) 2024-05-10T11:22:56.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.181+0000 7f3596083700 1 -- 172.21.15.100:0/2981215225 learned_addr learned my addr 172.21.15.100:0/2981215225 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:22:56.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.181+0000 7f3595882700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56183fddcdf0 0x56183fde06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:56.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.181+0000 7f3596884700 1 --2- 172.21.15.100:0/2981215225 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56183fde0c80 0x56183fde1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:56.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.181+0000 7f3596083700 1 -- 172.21.15.100:0/2981215225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56183fddcdf0 msgr2=0x56183fde06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:56.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.181+0000 7f3596083700 1 --2- 172.21.15.100:0/2981215225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56183fddcdf0 0x56183fde06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:56.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.181+0000 7f3596083700 1 -- 172.21.15.100:0/2981215225 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56183fde0c80 msgr2=0x56183fde1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:56.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.181+0000 7f3596083700 1 --2- 172.21.15.100:0/2981215225 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56183fde0c80 0x56183fde1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:56.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.181+0000 7f3596083700 1 -- 172.21.15.100:0/2981215225 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56183fc7b8d0 con 0x56183fddc4d0 2024-05-10T11:22:56.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.181+0000 7f3596083700 1 --2- 172.21.15.100:0/2981215225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56183fddc4d0 0x56183fddc8b0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f358c0a9250 tx=0x7f358c0a8b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4df9a05a4737be98 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:56.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.181+0000 7f3584d70700 1 -- 172.21.15.100:0/2981215225 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358c020040 con 0x56183fddc4d0 2024-05-10T11:22:56.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.181+0000 7f3584d70700 1 -- 172.21.15.100:0/2981215225 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f358c0a92c0 con 0x56183fddc4d0 2024-05-10T11:22:56.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.181+0000 7f3584d70700 1 -- 172.21.15.100:0/2981215225 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358c0a9780 con 0x56183fddc4d0 2024-05-10T11:22:56.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.182+0000 7f35a2b7ab80 1 -- 172.21.15.100:0/2981215225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56183fddc4d0 msgr2=0x56183fddc8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:56.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.182+0000 7f35a2b7ab80 1 --2- 172.21.15.100:0/2981215225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56183fddc4d0 0x56183fddc8b0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f358c0a9250 tx=0x7f358c0a8b30 comp rx=0 tx=0).stop 2024-05-10T11:22:56.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.182+0000 7f35a2b7ab80 1 -- 172.21.15.100:0/2981215225 shutdown_connections 2024-05-10T11:22:56.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.182+0000 7f35a2b7ab80 1 --2- 172.21.15.100:0/2981215225 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56183fde0c80 0x56183fde1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:56.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.182+0000 7f35a2b7ab80 1 --2- 172.21.15.100:0/2981215225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56183fddc4d0 0x56183fddc8b0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:56.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.182+0000 7f35a2b7ab80 1 --2- 172.21.15.100:0/2981215225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56183fddcdf0 0x56183fde06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:56.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.182+0000 7f35a2b7ab80 1 -- 172.21.15.100:0/2981215225 >> 172.21.15.100:0/2981215225 conn(0x56183fd08100 msgr2=0x56183fdd6c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:56.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.182+0000 7f35a2b7ab80 1 -- 172.21.15.100:0/2981215225 shutdown_connections 2024-05-10T11:22:56.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.182+0000 7f35a2b7ab80 1 -- 172.21.15.100:0/2981215225 wait complete. 2024-05-10T11:22:56.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.183+0000 7f35a2b7ab80 1 -- start start 2024-05-10T11:22:56.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.183+0000 7f35a2b7ab80 1 Processor -- start 2024-05-10T11:22:56.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.183+0000 7f35a2b7ab80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56183fddc4d0 0x56183fde8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:56.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.183+0000 7f35a2b7ab80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56183fddcdf0 0x56183fde5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:56.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.183+0000 7f35a2b7ab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56183fde0c80 0x56183fde5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:56.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.183+0000 7f35a2b7ab80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56183fde8dd0 con 0x56183fddc4d0 2024-05-10T11:22:56.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.183+0000 7f35a2b7ab80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56183fc94c30 con 0x56183fddcdf0 2024-05-10T11:22:56.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.183+0000 7f3596083700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56183fddc4d0 0x56183fde8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:56.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.183+0000 7f3596083700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56183fddc4d0 0x56183fde8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33568/0 (socket says 172.21.15.100:33568) 2024-05-10T11:22:56.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.183+0000 7f3596083700 1 -- 172.21.15.100:0/2234218292 learned_addr learned my addr 172.21.15.100:0/2234218292 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:22:56.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.183+0000 7f3596884700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56183fde0c80 0x56183fde5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:56.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.183+0000 7f35a2b7ab80 1 -- 172.21.15.100:0/2234218292 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56183fde8f50 con 0x56183fde0c80 2024-05-10T11:22:56.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.183+0000 7f35a2b7ab80 10 client.0 _ll_register_callbacks cb 0x56183fcc6920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:22:56.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.183+0000 7f3595882700 1 --2- 172.21.15.100:0/2234218292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56183fddcdf0 0x56183fde5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:56.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.184+0000 7f3596083700 1 -- 172.21.15.100:0/2234218292 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56183fde0c80 msgr2=0x56183fde5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:56.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.184+0000 7f3596083700 1 --2- 172.21.15.100:0/2234218292 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56183fde0c80 0x56183fde5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:56.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.184+0000 7f3596083700 1 -- 172.21.15.100:0/2234218292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56183fddcdf0 msgr2=0x56183fde5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:56.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.184+0000 7f3596083700 1 --2- 172.21.15.100:0/2234218292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56183fddcdf0 0x56183fde5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:56.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.184+0000 7f3596083700 1 -- 172.21.15.100:0/2234218292 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56183fcca690 con 0x56183fddc4d0 2024-05-10T11:22:56.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.184+0000 7f3595882700 1 --2- 172.21.15.100:0/2234218292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56183fddcdf0 0x56183fde5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:22:56.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.184+0000 7f3596884700 1 --2- 172.21.15.100:0/2234218292 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56183fde0c80 0x56183fde5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:22:56.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.184+0000 7f3596083700 1 --2- 172.21.15.100:0/2234218292 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56183fddc4d0 0x56183fde8090 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f358c0a6530 tx=0x7f358c010970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60ce67eb9ad9381e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:56.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.184+0000 7f3579ffb700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:22:56.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.184+0000 7f3579ffb700 1 -- 172.21.15.100:0/2234218292 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358c020050 con 0x56183fddc4d0 2024-05-10T11:22:56.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.184+0000 7f3579ffb700 1 -- 172.21.15.100:0/2234218292 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f358c0a9520 con 0x56183fddc4d0 2024-05-10T11:22:56.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.184+0000 7f35a2b7ab80 1 -- 172.21.15.100:0/2234218292 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f358c0a92c0 con 0x56183fddc4d0 2024-05-10T11:22:56.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.184+0000 7f3579ffb700 1 -- 172.21.15.100:0/2234218292 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f358c0a9720 con 0x56183fddc4d0 2024-05-10T11:22:56.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.184+0000 7f35a2b7ab80 10 client.16293 fetch_fsmap learned FSMap version 30 2024-05-10T11:22:56.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.184+0000 7f35a2b7ab80 1 -- 172.21.15.100:0/2234218292 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56183fdf79e0 con 0x56183fddc4d0 2024-05-10T11:22:56.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.184+0000 7f3579ffb700 1 -- 172.21.15.100:0/2234218292 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f358c0a92c0 con 0x56183fddc4d0 2024-05-10T11:22:56.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.185+0000 7f3579ffb700 1 -- 172.21.15.100:0/2234218292 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7f358c0ad120 con 0x56183fddc4d0 2024-05-10T11:22:56.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.185+0000 7f35a2b7ab80 10 client.16293 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T11:22:56.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.185+0000 7f35a2b7ab80 10 client.16293 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:22:56.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.185+0000 7f35a2b7ab80 20 client.16293 populate_metadata read hostname 'smithi100' 2024-05-10T11:22:56.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.185+0000 7f35a2b7ab80 1 --2- 172.21.15.100:0/2234218292 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56183fdf9eb0 0x56183fdfc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:56.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.185+0000 7f35a2b7ab80 4 client.16293 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:22:56.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.185+0000 7f35a2b7ab80 1 -- 172.21.15.100:0/2234218292 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56183fda8650 con 0x56183fdf9eb0 2024-05-10T11:22:56.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.185+0000 7f3595882700 1 --2- 172.21.15.100:0/2234218292 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56183fdf9eb0 0x56183fdfc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:56.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.185+0000 7f3579ffb700 10 client.16293 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:22:56.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.185+0000 7f3595882700 1 --2- 172.21.15.100:0/2234218292 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56183fdf9eb0 0x56183fdfc2a0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a291613f62ce9db server_cookie=2c2e42e27199ed97 in_seq=0 out_seq=0 2024-05-10T11:22:56.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.186+0000 7f3579ffb700 1 -- 172.21.15.100:0/2234218292 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56183fda8650 con 0x56183fdf9eb0 2024-05-10T11:22:56.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.186+0000 7f3579ffb700 10 client.16293 handle_command_reply: tid=0 2024-05-10T11:22:56.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.186+0000 7f35a2b7ab80 1 client.16293 shutdown 2024-05-10T11:22:56.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.187+0000 7f35a2b7ab80 1 -- 172.21.15.100:0/2234218292 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56183fddc4d0 msgr2=0x56183fde8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:56.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.187+0000 7f35a2b7ab80 1 --2- 172.21.15.100:0/2234218292 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56183fddc4d0 0x56183fde8090 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f358c0a6530 tx=0x7f358c010970 comp rx=0 tx=0).stop 2024-05-10T11:22:56.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.187+0000 7f35a2b7ab80 1 -- 172.21.15.100:0/2234218292 shutdown_connections 2024-05-10T11:22:56.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.187+0000 7f35a2b7ab80 1 --2- 172.21.15.100:0/2234218292 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56183fdf9eb0 0x56183fdfc2a0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:56.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.187+0000 7f35a2b7ab80 1 --2- 172.21.15.100:0/2234218292 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56183fde0c80 0x56183fde5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:56.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.187+0000 7f35a2b7ab80 1 --2- 172.21.15.100:0/2234218292 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56183fddc4d0 0x56183fde8090 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:56.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.187+0000 7f35a2b7ab80 1 --2- 172.21.15.100:0/2234218292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56183fddcdf0 0x56183fde5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:56.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.187+0000 7f35a2b7ab80 1 -- 172.21.15.100:0/2234218292 >> 172.21.15.100:0/2234218292 conn(0x56183fd08100 msgr2=0x56183fdd1720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:56.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.187+0000 7f3579ffb700 0 client.16293 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:22:56.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.187+0000 7f35a2b7ab80 1 -- 172.21.15.100:0/2234218292 shutdown_connections 2024-05-10T11:22:56.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.187+0000 7f35a2b7ab80 1 -- 172.21.15.100:0/2234218292 wait complete. 2024-05-10T11:22:56.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.187+0000 7f35a2b7ab80 20 client.16293 trim_cache size 0 max 16384 2024-05-10T11:22:56.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.211+0000 7f35a2b7ab80 1 Processor -- start 2024-05-10T11:22:56.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.211+0000 7f35a2b7ab80 1 -- start start 2024-05-10T11:22:56.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.211+0000 7f35a2b7ab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56183fddc4d0 0x56183fdde5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:56.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.211+0000 7f35a2b7ab80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56183fddcdf0 0x56183fddeb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:56.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.211+0000 7f35a2b7ab80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56183fde0c80 0x56183fddb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:56.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.211+0000 7f35a2b7ab80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f35901af780 con 0x56183fde0c80 2024-05-10T11:22:56.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.211+0000 7f35a2b7ab80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f35901ad010 con 0x56183fddcdf0 2024-05-10T11:22:56.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.211+0000 7f35a2b7ab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f35901aae80 con 0x56183fddc4d0 2024-05-10T11:22:56.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.211+0000 7f3596884700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56183fde0c80 0x56183fddb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:56.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.211+0000 7f3596884700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56183fde0c80 0x56183fddb2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33582/0 (socket says 172.21.15.100:33582) 2024-05-10T11:22:56.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.211+0000 7f3596884700 1 -- 172.21.15.100:0/4014691018 learned_addr learned my addr 172.21.15.100:0/4014691018 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:22:56.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.211+0000 7f3596083700 1 --2- 172.21.15.100:0/4014691018 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56183fddc4d0 0x56183fdde5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:56.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.211+0000 7f3595882700 1 --2- 172.21.15.100:0/4014691018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56183fddcdf0 0x56183fddeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:56.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.211+0000 7f3596884700 1 -- 172.21.15.100:0/4014691018 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56183fddc4d0 msgr2=0x56183fdde5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:56.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.211+0000 7f3596884700 1 --2- 172.21.15.100:0/4014691018 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56183fddc4d0 0x56183fdde5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:56.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.211+0000 7f3596884700 1 -- 172.21.15.100:0/4014691018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56183fddcdf0 msgr2=0x56183fddeb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:56.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.211+0000 7f3596884700 1 --2- 172.21.15.100:0/4014691018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56183fddcdf0 0x56183fddeb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:56.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.211+0000 7f3596884700 1 -- 172.21.15.100:0/4014691018 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56183fde6890 con 0x56183fde0c80 2024-05-10T11:22:56.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.212+0000 7f3596884700 1 --2- 172.21.15.100:0/4014691018 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56183fde0c80 0x56183fddb2e0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f3588011b90 tx=0x7f3588009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9dcbf7defd04462 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:56.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.212+0000 7f357bfff700 1 -- 172.21.15.100:0/4014691018 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3588017a80 con 0x56183fde0c80 2024-05-10T11:22:56.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.212+0000 7f357bfff700 1 -- 172.21.15.100:0/4014691018 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f35880093a0 con 0x56183fde0c80 2024-05-10T11:22:56.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.212+0000 7f357bfff700 1 -- 172.21.15.100:0/4014691018 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358801f3e0 con 0x56183fde0c80 2024-05-10T11:22:56.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.212+0000 7f35a2b7ab80 1 -- 172.21.15.100:0/4014691018 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56183fde0c80 msgr2=0x56183fddb2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:56.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.212+0000 7f35a2b7ab80 1 --2- 172.21.15.100:0/4014691018 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56183fde0c80 0x56183fddb2e0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f3588011b90 tx=0x7f3588009920 comp rx=0 tx=0).stop 2024-05-10T11:22:56.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.213+0000 7f35a2b7ab80 1 -- 172.21.15.100:0/4014691018 shutdown_connections 2024-05-10T11:22:56.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.213+0000 7f35a2b7ab80 1 --2- 172.21.15.100:0/4014691018 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56183fde0c80 0x56183fddb2e0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:56.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.213+0000 7f35a2b7ab80 1 --2- 172.21.15.100:0/4014691018 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56183fddc4d0 0x56183fdde5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:56.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.213+0000 7f35a2b7ab80 1 --2- 172.21.15.100:0/4014691018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56183fddcdf0 0x56183fddeb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:56.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.213+0000 7f35a2b7ab80 1 -- 172.21.15.100:0/4014691018 >> 172.21.15.100:0/4014691018 conn(0x56183fd08100 msgr2=0x56183fdd1720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:56.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.213+0000 7f35a2b7ab80 1 -- 172.21.15.100:0/4014691018 shutdown_connections 2024-05-10T11:22:56.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.213+0000 7f35a2b7ab80 1 -- 172.21.15.100:0/4014691018 wait complete. 2024-05-10T11:22:56.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.213+0000 7f35a2b7ab80 1 -- start start 2024-05-10T11:22:56.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.213+0000 7f35a2b7ab80 1 Processor -- start 2024-05-10T11:22:56.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.213+0000 7f35a2b7ab80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56183fddc4d0 0x56183fdff0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:56.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.213+0000 7f35a2b7ab80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56183fddcdf0 0x56183fdff610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:56.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.213+0000 7f35a2b7ab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56183fde0c80 0x56183fdfc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:56.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.213+0000 7f35a2b7ab80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f35901ab4d0 con 0x56183fddcdf0 2024-05-10T11:22:56.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.213+0000 7f35a2b7ab80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f35901ac940 con 0x56183fddc4d0 2024-05-10T11:22:56.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.213+0000 7f35a2b7ab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f35901abba0 con 0x56183fde0c80 2024-05-10T11:22:56.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.213+0000 7f35a2b7ab80 10 client.0 _ll_register_callbacks cb 0x56183fcc6920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:22:56.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.213+0000 7f3595882700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56183fddcdf0 0x56183fdff610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:56.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.213+0000 7f3595882700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56183fddcdf0 0x56183fdff610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33584/0 (socket says 172.21.15.100:33584) 2024-05-10T11:22:56.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.213+0000 7f3595882700 1 -- 172.21.15.100:0/3989460689 learned_addr learned my addr 172.21.15.100:0/3989460689 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:22:56.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.213+0000 7f3596884700 1 --2- 172.21.15.100:0/3989460689 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56183fde0c80 0x56183fdfc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:56.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.213+0000 7f3596083700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56183fddc4d0 0x56183fdff0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:56.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.214+0000 7f3595882700 1 -- 172.21.15.100:0/3989460689 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56183fde0c80 msgr2=0x56183fdfc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:56.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.214+0000 7f3595882700 1 --2- 172.21.15.100:0/3989460689 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56183fde0c80 0x56183fdfc670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:56.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.214+0000 7f3595882700 1 -- 172.21.15.100:0/3989460689 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56183fddc4d0 msgr2=0x56183fdff0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:56.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.214+0000 7f3595882700 1 --2- 172.21.15.100:0/3989460689 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56183fddc4d0 0x56183fdff0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:56.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.214+0000 7f3595882700 1 -- 172.21.15.100:0/3989460689 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56183fdfceb0 con 0x56183fddcdf0 2024-05-10T11:22:56.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.214+0000 7f3596884700 1 --2- 172.21.15.100:0/3989460689 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56183fde0c80 0x56183fdfc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:22:56.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.214+0000 7f3596083700 1 --2- 172.21.15.100:0/3989460689 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56183fddc4d0 0x56183fdff0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:22:56.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.214+0000 7f3595882700 1 --2- 172.21.15.100:0/3989460689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56183fddcdf0 0x56183fdff610 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f3580000c00 tx=0x7f3580009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3a9cdc5f365266d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:56.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.214+0000 7f3579ffb700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:22:56.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.214+0000 7f3579ffb700 1 -- 172.21.15.100:0/3989460689 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3580034900 con 0x56183fddcdf0 2024-05-10T11:22:56.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.214+0000 7f3579ffb700 1 -- 172.21.15.100:0/3989460689 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f358000b7b0 con 0x56183fddcdf0 2024-05-10T11:22:56.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.214+0000 7f35a2b7ab80 1 -- 172.21.15.100:0/3989460689 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35880093a0 con 0x56183fddcdf0 2024-05-10T11:22:56.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.214+0000 7f3579ffb700 1 -- 172.21.15.100:0/3989460689 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3580043050 con 0x56183fddcdf0 2024-05-10T11:22:56.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.214+0000 7f35a2b7ab80 10 client.16299 fetch_fsmap learned FSMap version 30 2024-05-10T11:22:56.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.214+0000 7f35a2b7ab80 1 -- 172.21.15.100:0/3989460689 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56183fe64140 con 0x56183fddcdf0 2024-05-10T11:22:56.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.214+0000 7f3579ffb700 1 -- 172.21.15.100:0/3989460689 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f35880093a0 con 0x56183fddcdf0 2024-05-10T11:22:56.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.215+0000 7f3579ffb700 1 -- 172.21.15.100:0/3989460689 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7f3580021070 con 0x56183fddcdf0 2024-05-10T11:22:56.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.215+0000 7f35a2b7ab80 10 client.16299 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T11:22:56.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.215+0000 7f35a2b7ab80 10 client.16299 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:22:56.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.215+0000 7f35a2b7ab80 20 client.16299 populate_metadata read hostname 'smithi100' 2024-05-10T11:22:56.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.215+0000 7f35a2b7ab80 1 --2- 172.21.15.100:0/3989460689 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56183fe648b0 0x56183fe66cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:56.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.215+0000 7f35a2b7ab80 4 client.16299 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T11:22:56.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.215+0000 7f35a2b7ab80 1 -- 172.21.15.100:0/3989460689 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56183fd3c240 con 0x56183fe648b0 2024-05-10T11:22:56.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.215+0000 7f3596083700 1 --2- 172.21.15.100:0/3989460689 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56183fe648b0 0x56183fe66cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:56.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.215+0000 7f3579ffb700 10 client.16299 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:22:56.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.215+0000 7f3596083700 1 --2- 172.21.15.100:0/3989460689 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56183fe648b0 0x56183fe66cd0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9d71ff42d8cda87 server_cookie=5463d499faa79fad in_seq=0 out_seq=0 2024-05-10T11:22:56.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.215+0000 7f3579ffb700 1 -- 172.21.15.100:0/3989460689 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56183fd3c240 con 0x56183fe648b0 2024-05-10T11:22:56.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.215+0000 7f3579ffb700 10 client.16299 handle_command_reply: tid=0 2024-05-10T11:22:56.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.215+0000 7f35a2b7ab80 1 client.16299 shutdown 2024-05-10T11:22:56.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.215+0000 7f35a2b7ab80 1 -- 172.21.15.100:0/3989460689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56183fddcdf0 msgr2=0x56183fdff610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:56.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.215+0000 7f35a2b7ab80 1 --2- 172.21.15.100:0/3989460689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56183fddcdf0 0x56183fdff610 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f3580000c00 tx=0x7f3580009ca0 comp rx=0 tx=0).stop 2024-05-10T11:22:56.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.215+0000 7f35a2b7ab80 1 -- 172.21.15.100:0/3989460689 shutdown_connections 2024-05-10T11:22:56.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.215+0000 7f35a2b7ab80 1 --2- 172.21.15.100:0/3989460689 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56183fe648b0 0x56183fe66cd0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:56.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.216+0000 7f35a2b7ab80 1 --2- 172.21.15.100:0/3989460689 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56183fde0c80 0x56183fdfc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:56.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.216+0000 7f35a2b7ab80 1 --2- 172.21.15.100:0/3989460689 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56183fddc4d0 0x56183fdff0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:56.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.216+0000 7f35a2b7ab80 1 --2- 172.21.15.100:0/3989460689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56183fddcdf0 0x56183fdff610 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:56.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.216+0000 7f35a2b7ab80 1 -- 172.21.15.100:0/3989460689 >> 172.21.15.100:0/3989460689 conn(0x56183fd08100 msgr2=0x56183fddb2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:56.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.216+0000 7f3579ffb700 0 client.16299 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:22:56.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.216+0000 7f35a2b7ab80 1 -- 172.21.15.100:0/3989460689 shutdown_connections 2024-05-10T11:22:56.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.216+0000 7f35a2b7ab80 1 -- 172.21.15.100:0/3989460689 wait complete. 2024-05-10T11:22:56.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.216+0000 7f35a2b7ab80 20 client.16299 trim_cache size 0 max 16384 2024-05-10T11:22:56.221 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:22:56.221 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T11:22:56.221 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-05-10T11:22:56.221 INFO:teuthology.orchestra.run.smithi100.stdout: "bbb8fab3-fdcf-4321-9986-0d3fe39e79ec": { 2024-05-10T11:22:56.221 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-05-10T11:22:56.221 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "bbb8fab3-fdcf-4321-9986-0d3fe39e79ec", 2024-05-10T11:22:56.221 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-05-10T11:22:56.221 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:22:56.221 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:22:56.221 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:22:56.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.216+0000 7f35982e7700 1 -- 172.21.15.100:0/3902815699 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f357c0b2880 msgr2=0x7f357c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:56.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.216+0000 7f35982e7700 1 --2- 172.21.15.100:0/3902815699 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f357c0b2880 0x7f357c0b4d40 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3590079b20 tx=0x7f3580005fb0 comp rx=0 tx=0).stop 2024-05-10T11:22:56.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.216+0000 7f35982e7700 1 -- 172.21.15.100:0/3902815699 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f359006b180 msgr2=0x7f359019f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:56.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.216+0000 7f35982e7700 1 --2- 172.21.15.100:0/3902815699 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f359006b180 0x7f359019f6e0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f358c00d950 tx=0x7f358c00dc60 comp rx=0 tx=0).stop 2024-05-10T11:22:56.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.216+0000 7f35982e7700 1 -- 172.21.15.100:0/3902815699 shutdown_connections 2024-05-10T11:22:56.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.216+0000 7f35982e7700 1 --2- 172.21.15.100:0/3902815699 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f357c0b2880 0x7f357c0b4d40 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:56.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.216+0000 7f35982e7700 1 --2- 172.21.15.100:0/3902815699 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3590109510 0x7f35901998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:56.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.216+0000 7f35982e7700 1 --2- 172.21.15.100:0/3902815699 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f359006b180 0x7f359019f6e0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:56.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.216+0000 7f35982e7700 1 --2- 172.21.15.100:0/3902815699 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f359006bb40 0x7f359019fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:56.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.216+0000 7f35982e7700 1 -- 172.21.15.100:0/3902815699 >> 172.21.15.100:0/3902815699 conn(0x7f3590079350 msgr2=0x7f3590101330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:56.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.216+0000 7f35982e7700 1 -- 172.21.15.100:0/3902815699 shutdown_connections 2024-05-10T11:22:56.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:22:56.216+0000 7f35982e7700 1 -- 172.21.15.100:0/3902815699 wait complete. 2024-05-10T11:22:56.969 INFO:tasks.workunit.client.0.smithi100.stdout: 1 90154 2.74 MB/sec execute 51 sec latency 42.669 ms 2024-05-10T11:22:57.222 INFO:tasks.cephfs.filesystem:scrub status for tag:bbb8fab3-fdcf-4321-9986-0d3fe39e79ec - {'path': '/', 'tag': 'bbb8fab3-fdcf-4321-9986-0d3fe39e79ec', 'options': 'recursive,force'} 2024-05-10T11:22:57.970 INFO:tasks.workunit.client.0.smithi100.stdout: 1 90810 2.70 MB/sec execute 52 sec latency 21.768 ms 2024-05-10T11:22:58.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:22:57 smithi178 ceph-mon[33359]: pgmap v557: 129 pgs: 129 active+clean; 49 MiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 1.4 MiB/s wr, 181 op/s 2024-05-10T11:22:58.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:22:57 smithi188 ceph-mon[34075]: pgmap v557: 129 pgs: 129 active+clean; 49 MiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 1.4 MiB/s wr, 181 op/s 2024-05-10T11:22:58.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:22:57 smithi100 ceph-mon[30296]: pgmap v557: 129 pgs: 129 active+clean; 49 MiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 1.4 MiB/s wr, 181 op/s 2024-05-10T11:22:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.578+0000 7f159e7f9640 1 -- 172.21.15.100:0/3062623583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15981019c0 msgr2=0x7f1598101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.578+0000 7f159e7f9640 1 --2- 172.21.15.100:0/3062623583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15981019c0 0x7f1598101de0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f1588009900 tx=0x7f158802a410 comp rx=0 tx=0).stop 2024-05-10T11:22:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.578+0000 7f159e7f9640 1 -- 172.21.15.100:0/3062623583 shutdown_connections 2024-05-10T11:22:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.578+0000 7f159e7f9640 1 --2- 172.21.15.100:0/3062623583 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1598103570 0x7f1598109e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.578+0000 7f159e7f9640 1 --2- 172.21.15.100:0/3062623583 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1598102bb0 0x7f1598103030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.578+0000 7f159e7f9640 1 --2- 172.21.15.100:0/3062623583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15981019c0 0x7f1598101de0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.578+0000 7f159e7f9640 1 -- 172.21.15.100:0/3062623583 >> 172.21.15.100:0/3062623583 conn(0x7f15980fcf30 msgr2=0x7f15980ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.578+0000 7f159e7f9640 1 -- 172.21.15.100:0/3062623583 shutdown_connections 2024-05-10T11:22:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.578+0000 7f159e7f9640 1 -- 172.21.15.100:0/3062623583 wait complete. 2024-05-10T11:22:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.579+0000 7f159e7f9640 1 Processor -- start 2024-05-10T11:22:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.579+0000 7f159e7f9640 1 -- start start 2024-05-10T11:22:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.579+0000 7f159e7f9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15981019c0 0x7f159806bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.579+0000 7f159e7f9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1598102bb0 0x7f159819d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.579+0000 7f159e7f9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1598103570 0x7f159819fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.579+0000 7f159e7f9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f159810d320 con 0x7f15981019c0 2024-05-10T11:22:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.579+0000 7f159e7f9640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f159810d1a0 con 0x7f1598102bb0 2024-05-10T11:22:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.579+0000 7f159e7f9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f159810d4a0 con 0x7f1598103570 2024-05-10T11:22:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.579+0000 7f1597fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15981019c0 0x7f159806bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.579+0000 7f1597fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15981019c0 0x7f159806bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42864/0 (socket says 172.21.15.100:42864) 2024-05-10T11:22:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.579+0000 7f1597fff640 1 -- 172.21.15.100:0/225779843 learned_addr learned my addr 172.21.15.100:0/225779843 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:22:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.579+0000 7f15977fe640 1 --2- 172.21.15.100:0/225779843 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1598102bb0 0x7f159819d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.579+0000 7f159cd96640 1 --2- 172.21.15.100:0/225779843 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1598103570 0x7f159819fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.580+0000 7f1597fff640 1 -- 172.21.15.100:0/225779843 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1598103570 msgr2=0x7f159819fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.580+0000 7f1597fff640 1 --2- 172.21.15.100:0/225779843 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1598103570 0x7f159819fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.580+0000 7f1597fff640 1 -- 172.21.15.100:0/225779843 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1598102bb0 msgr2=0x7f159819d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.580+0000 7f1597fff640 1 --2- 172.21.15.100:0/225779843 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1598102bb0 0x7f159819d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.580+0000 7f1597fff640 1 -- 172.21.15.100:0/225779843 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15981a0720 con 0x7f15981019c0 2024-05-10T11:22:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.580+0000 7f15977fe640 1 --2- 172.21.15.100:0/225779843 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1598102bb0 0x7f159819d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:22:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.580+0000 7f1597fff640 1 --2- 172.21.15.100:0/225779843 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15981019c0 0x7f159806bd70 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f1588009230 tx=0x7f15880040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.580+0000 7f15957fa640 1 -- 172.21.15.100:0/225779843 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f158803f030 con 0x7f15981019c0 2024-05-10T11:22:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.580+0000 7f15957fa640 1 -- 172.21.15.100:0/225779843 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f15880044c0 con 0x7f15981019c0 2024-05-10T11:22:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.580+0000 7f159e7f9640 1 -- 172.21.15.100:0/225779843 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15981a09b0 con 0x7f15981019c0 2024-05-10T11:22:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.580+0000 7f15957fa640 1 -- 172.21.15.100:0/225779843 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1588035070 con 0x7f15981019c0 2024-05-10T11:22:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.580+0000 7f159e7f9640 1 -- 172.21.15.100:0/225779843 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15981a0ea0 con 0x7f15981019c0 2024-05-10T11:22:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.582+0000 7f15957fa640 1 -- 172.21.15.100:0/225779843 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f1588042030 con 0x7f15981019c0 2024-05-10T11:22:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.583+0000 7f15a15ea740 1 Processor -- start 2024-05-10T11:22:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.583+0000 7f15a15ea740 1 -- start start 2024-05-10T11:22:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.583+0000 7f15a15ea740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639e61661c0 0x5639e61665e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.583+0000 7f15a15ea740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639e6166bb0 0x5639e616b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.583+0000 7f15a15ea740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639e616bd20 0x5639e616e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.583+0000 7f15a15ea740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5639e60c65d0 con 0x5639e61661c0 2024-05-10T11:22:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.583+0000 7f15a15ea740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5639e607f890 con 0x5639e6166bb0 2024-05-10T11:22:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.583+0000 7f15a15ea740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5639e60c2590 con 0x5639e616bd20 2024-05-10T11:22:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.583+0000 7f159cd96640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639e61661c0 0x5639e61665e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.583+0000 7f159cd96640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639e61661c0 0x5639e61665e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42880/0 (socket says 172.21.15.100:42880) 2024-05-10T11:22:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.583+0000 7f159cd96640 1 -- 172.21.15.100:0/1932368213 learned_addr learned my addr 172.21.15.100:0/1932368213 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:22:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.583+0000 7f15977fe640 1 --2- 172.21.15.100:0/1932368213 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639e616bd20 0x5639e616e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.583+0000 7f15957fa640 1 --2- 172.21.15.100:0/225779843 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f15800b28a0 0x7f15800b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.583+0000 7f1597fff640 1 --2- 172.21.15.100:0/1932368213 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639e6166bb0 0x5639e616b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.583+0000 7f159cd96640 1 --2- 172.21.15.100:0/225779843 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f15800b28a0 0x7f15800b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.584+0000 7f15977fe640 1 -- 172.21.15.100:0/1932368213 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639e6166bb0 msgr2=0x5639e616b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.584+0000 7f15977fe640 1 --2- 172.21.15.100:0/1932368213 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639e6166bb0 0x5639e616b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.584+0000 7f15977fe640 1 -- 172.21.15.100:0/1932368213 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639e61661c0 msgr2=0x5639e61665e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.584+0000 7f15977fe640 1 --2- 172.21.15.100:0/1932368213 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639e61661c0 0x5639e61665e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.584+0000 7f15977fe640 1 -- 172.21.15.100:0/1932368213 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639e5dd8210 con 0x5639e616bd20 2024-05-10T11:22:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.584+0000 7f159cd96640 1 --2- 172.21.15.100:0/1932368213 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639e61661c0 0x5639e61665e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:22:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.584+0000 7f15957fa640 1 -- 172.21.15.100:0/225779843 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f1588102a20 con 0x7f15981019c0 2024-05-10T11:22:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.584+0000 7f15977fe640 1 --2- 172.21.15.100:0/1932368213 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639e616bd20 0x5639e616e1b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f158c009fa0 tx=0x7f158c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b374dcbf2cc0517a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.584+0000 7f159cd96640 1 --2- 172.21.15.100:0/225779843 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f15800b28a0 0x7f15800b4d60 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x5639e6167410 tx=0x7f1590004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.584+0000 7f157e7fc640 1 -- 172.21.15.100:0/1932368213 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f158c004310 con 0x5639e616bd20 2024-05-10T11:22:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.584+0000 7f157e7fc640 1 -- 172.21.15.100:0/1932368213 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f158c0044b0 con 0x5639e616bd20 2024-05-10T11:22:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.584+0000 7f157e7fc640 1 -- 172.21.15.100:0/1932368213 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f158c004e30 con 0x5639e616bd20 2024-05-10T11:22:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.585+0000 7f15a15ea740 1 -- 172.21.15.100:0/1932368213 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639e616bd20 msgr2=0x5639e616e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.585+0000 7f15a15ea740 1 --2- 172.21.15.100:0/1932368213 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639e616bd20 0x5639e616e1b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f158c009fa0 tx=0x7f158c0118d0 comp rx=0 tx=0).stop 2024-05-10T11:22:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.585+0000 7f15a15ea740 1 -- 172.21.15.100:0/1932368213 shutdown_connections 2024-05-10T11:22:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.585+0000 7f15a15ea740 1 --2- 172.21.15.100:0/1932368213 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639e616bd20 0x5639e616e1b0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.585+0000 7f15a15ea740 1 --2- 172.21.15.100:0/1932368213 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639e6166bb0 0x5639e616b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.585+0000 7f15a15ea740 1 --2- 172.21.15.100:0/1932368213 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639e61661c0 0x5639e61665e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.585+0000 7f15a15ea740 1 -- 172.21.15.100:0/1932368213 >> 172.21.15.100:0/1932368213 conn(0x5639e615d420 msgr2=0x5639e615d820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.585+0000 7f15a15ea740 1 -- 172.21.15.100:0/1932368213 shutdown_connections 2024-05-10T11:22:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.585+0000 7f15a15ea740 1 -- 172.21.15.100:0/1932368213 wait complete. 2024-05-10T11:22:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.585+0000 7f15a15ea740 1 -- start start 2024-05-10T11:22:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.585+0000 7f15a15ea740 1 Processor -- start 2024-05-10T11:22:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.586+0000 7f15a15ea740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639e61661c0 0x5639e6170430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.586+0000 7f15a15ea740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639e6166bb0 0x5639e6170970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.586+0000 7f15a15ea740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639e616bd20 0x5639e61779b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.586+0000 7f15a15ea740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5639e616ecd0 con 0x5639e61661c0 2024-05-10T11:22:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.586+0000 7f15a15ea740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5639e6083370 con 0x5639e616bd20 2024-05-10T11:22:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.586+0000 7f15a15ea740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5639e616ee50 con 0x5639e6166bb0 2024-05-10T11:22:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.586+0000 7f15a15ea740 10 client.0 _ll_register_callbacks cb 0x5639e5dd7a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:22:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.586+0000 7f1597fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639e61661c0 0x5639e6170430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.586+0000 7f1597fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639e61661c0 0x5639e6170430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42882/0 (socket says 172.21.15.100:42882) 2024-05-10T11:22:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.586+0000 7f1597fff640 1 -- 172.21.15.100:0/3446348521 learned_addr learned my addr 172.21.15.100:0/3446348521 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:22:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.586+0000 7f15977fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639e6166bb0 0x5639e6170970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.586+0000 7f159cd96640 1 --2- 172.21.15.100:0/3446348521 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639e616bd20 0x5639e61779b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.586+0000 7f1597fff640 1 -- 172.21.15.100:0/3446348521 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639e6166bb0 msgr2=0x5639e6170970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.586+0000 7f1597fff640 1 --2- 172.21.15.100:0/3446348521 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639e6166bb0 0x5639e6170970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.586+0000 7f1597fff640 1 -- 172.21.15.100:0/3446348521 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639e616bd20 msgr2=0x5639e61779b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.586+0000 7f1597fff640 1 --2- 172.21.15.100:0/3446348521 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639e616bd20 0x5639e61779b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.586+0000 7f1597fff640 1 -- 172.21.15.100:0/3446348521 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639e5ea4620 con 0x5639e61661c0 2024-05-10T11:22:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.586+0000 7f15977fe640 1 --2- 172.21.15.100:0/3446348521 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639e6166bb0 0x5639e6170970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:22:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.586+0000 7f1597fff640 1 --2- 172.21.15.100:0/3446348521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639e61661c0 0x5639e6170430 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f15881001c0 tx=0x7f15880c6250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eeeb6fb58a561a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.586+0000 7f15777fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:22:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.586+0000 7f15777fe640 1 -- 172.21.15.100:0/3446348521 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1588049040 con 0x5639e61661c0 2024-05-10T11:22:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.587+0000 7f15777fe640 1 -- 172.21.15.100:0/3446348521 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f158808c150 con 0x5639e61661c0 2024-05-10T11:22:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.587+0000 7f15a15ea740 1 -- 172.21.15.100:0/3446348521 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f158c0044b0 con 0x5639e61661c0 2024-05-10T11:22:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.587+0000 7f15777fe640 1 -- 172.21.15.100:0/3446348521 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f15880c9500 con 0x5639e61661c0 2024-05-10T11:22:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.587+0000 7f15a15ea740 10 client.16311 fetch_fsmap learned FSMap version 30 2024-05-10T11:22:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.587+0000 7f15a15ea740 1 -- 172.21.15.100:0/3446348521 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5639e617a990 con 0x5639e61661c0 2024-05-10T11:22:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.587+0000 7f15777fe640 1 -- 172.21.15.100:0/3446348521 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f158c0044b0 con 0x5639e61661c0 2024-05-10T11:22:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.587+0000 7f15777fe640 1 -- 172.21.15.100:0/3446348521 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7f15880d26b0 con 0x5639e61661c0 2024-05-10T11:22:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.587+0000 7f15a15ea740 10 client.16311 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T11:22:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.587+0000 7f15a15ea740 10 client.16311 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:22:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.587+0000 7f15a15ea740 20 client.16311 populate_metadata read hostname 'smithi100' 2024-05-10T11:22:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.587+0000 7f15a15ea740 1 --2- 172.21.15.100:0/3446348521 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5639e617c0a0 0x5639e617e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.587+0000 7f15a15ea740 4 client.16311 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:22:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.587+0000 7f15a15ea740 1 -- 172.21.15.100:0/3446348521 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5639e60d41e0 con 0x5639e617c0a0 2024-05-10T11:22:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.587+0000 7f15977fe640 1 --2- 172.21.15.100:0/3446348521 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5639e617c0a0 0x5639e617e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.588+0000 7f15977fe640 1 --2- 172.21.15.100:0/3446348521 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5639e617c0a0 0x5639e617e540 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dff885dc6a59fd77 server_cookie=56c0005dd3b194c1 in_seq=0 out_seq=0 2024-05-10T11:22:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.588+0000 7f15777fe640 10 client.16311 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:22:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.588+0000 7f15777fe640 1 -- 172.21.15.100:0/3446348521 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5639e60d41e0 con 0x5639e617c0a0 2024-05-10T11:22:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.588+0000 7f15777fe640 10 client.16311 handle_command_reply: tid=0 2024-05-10T11:22:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.588+0000 7f15a15ea740 1 client.16311 shutdown 2024-05-10T11:22:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.589+0000 7f15a15ea740 1 -- 172.21.15.100:0/3446348521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639e61661c0 msgr2=0x5639e6170430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.589+0000 7f15a15ea740 1 --2- 172.21.15.100:0/3446348521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639e61661c0 0x5639e6170430 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f15881001c0 tx=0x7f15880c6250 comp rx=0 tx=0).stop 2024-05-10T11:22:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.589+0000 7f15a15ea740 1 -- 172.21.15.100:0/3446348521 shutdown_connections 2024-05-10T11:22:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.589+0000 7f15a15ea740 1 --2- 172.21.15.100:0/3446348521 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5639e617c0a0 0x5639e617e540 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.589+0000 7f15a15ea740 1 --2- 172.21.15.100:0/3446348521 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639e616bd20 0x5639e61779b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.589+0000 7f15a15ea740 1 --2- 172.21.15.100:0/3446348521 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639e6166bb0 0x5639e6170970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.589+0000 7f15777fe640 0 client.16311 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:22:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.589+0000 7f15a15ea740 1 --2- 172.21.15.100:0/3446348521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639e61661c0 0x5639e6170430 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.589+0000 7f15a15ea740 1 -- 172.21.15.100:0/3446348521 >> 172.21.15.100:0/3446348521 conn(0x5639e615d420 msgr2=0x5639e615c580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.589+0000 7f15a15ea740 1 -- 172.21.15.100:0/3446348521 shutdown_connections 2024-05-10T11:22:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.589+0000 7f15a15ea740 1 -- 172.21.15.100:0/3446348521 wait complete. 2024-05-10T11:22:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.589+0000 7f15a15ea740 20 client.16311 trim_cache size 0 max 16384 2024-05-10T11:22:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.605+0000 7f15a15ea740 1 Processor -- start 2024-05-10T11:22:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.605+0000 7f15a15ea740 1 -- start start 2024-05-10T11:22:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.605+0000 7f15a15ea740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639e61661c0 0x5639e6158560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.605+0000 7f15a15ea740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639e6166bb0 0x5639e6160a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.605+0000 7f15a15ea740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639e616bd20 0x5639e616fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.605+0000 7f15a15ea740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f15981ab8f0 con 0x5639e61661c0 2024-05-10T11:22:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.605+0000 7f15a15ea740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f15981a91c0 con 0x5639e6166bb0 2024-05-10T11:22:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.605+0000 7f15a15ea740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f15981a7010 con 0x5639e616bd20 2024-05-10T11:22:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.605+0000 7f1597fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639e61661c0 0x5639e6158560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:58.612 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.605+0000 7f1597fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639e61661c0 0x5639e6158560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42890/0 (socket says 172.21.15.100:42890) 2024-05-10T11:22:58.612 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.605+0000 7f1597fff640 1 -- 172.21.15.100:0/3305658886 learned_addr learned my addr 172.21.15.100:0/3305658886 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:22:58.612 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.605+0000 7f15977fe640 1 --2- 172.21.15.100:0/3305658886 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639e6166bb0 0x5639e6160a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:58.612 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.605+0000 7f159cd96640 1 --2- 172.21.15.100:0/3305658886 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639e616bd20 0x5639e616fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:58.612 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.606+0000 7f1597fff640 1 -- 172.21.15.100:0/3305658886 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639e616bd20 msgr2=0x5639e616fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:58.612 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.606+0000 7f1597fff640 1 --2- 172.21.15.100:0/3305658886 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639e616bd20 0x5639e616fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:58.612 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.606+0000 7f1597fff640 1 -- 172.21.15.100:0/3305658886 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639e6166bb0 msgr2=0x5639e6160a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:58.612 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.606+0000 7f1597fff640 1 --2- 172.21.15.100:0/3305658886 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639e6166bb0 0x5639e6160a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:58.612 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.606+0000 7f1597fff640 1 -- 172.21.15.100:0/3305658886 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639e615d250 con 0x5639e61661c0 2024-05-10T11:22:58.613 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.606+0000 7f1597fff640 1 --2- 172.21.15.100:0/3305658886 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639e61661c0 0x5639e6158560 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f15880c5e10 tx=0x7f158802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70aa01f010241b2f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:58.613 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.606+0000 7f157dffb640 1 -- 172.21.15.100:0/3305658886 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1588049070 con 0x5639e61661c0 2024-05-10T11:22:58.613 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.606+0000 7f157dffb640 1 -- 172.21.15.100:0/3305658886 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f15880c9d90 con 0x5639e61661c0 2024-05-10T11:22:58.613 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.606+0000 7f157dffb640 1 -- 172.21.15.100:0/3305658886 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15880c5750 con 0x5639e61661c0 2024-05-10T11:22:58.613 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.607+0000 7f15a15ea740 1 -- 172.21.15.100:0/3305658886 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639e61661c0 msgr2=0x5639e6158560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:58.613 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.607+0000 7f15a15ea740 1 --2- 172.21.15.100:0/3305658886 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639e61661c0 0x5639e6158560 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f15880c5e10 tx=0x7f158802a410 comp rx=0 tx=0).stop 2024-05-10T11:22:58.613 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.607+0000 7f15a15ea740 1 -- 172.21.15.100:0/3305658886 shutdown_connections 2024-05-10T11:22:58.613 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.607+0000 7f15a15ea740 1 --2- 172.21.15.100:0/3305658886 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639e616bd20 0x5639e616fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:58.613 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.607+0000 7f15a15ea740 1 --2- 172.21.15.100:0/3305658886 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639e6166bb0 0x5639e6160a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:58.613 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.607+0000 7f15a15ea740 1 --2- 172.21.15.100:0/3305658886 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639e61661c0 0x5639e6158560 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:58.613 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.607+0000 7f15a15ea740 1 -- 172.21.15.100:0/3305658886 >> 172.21.15.100:0/3305658886 conn(0x5639e615d420 msgr2=0x5639e6160330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:58.614 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.607+0000 7f15a15ea740 1 -- 172.21.15.100:0/3305658886 shutdown_connections 2024-05-10T11:22:58.614 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.607+0000 7f15a15ea740 1 -- 172.21.15.100:0/3305658886 wait complete. 2024-05-10T11:22:58.614 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.607+0000 7f15a15ea740 1 -- start start 2024-05-10T11:22:58.614 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.607+0000 7f15a15ea740 1 Processor -- start 2024-05-10T11:22:58.614 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.607+0000 7f15a15ea740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639e61661c0 0x5639e61743f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:58.614 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.607+0000 7f15a15ea740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639e6166bb0 0x5639e6174930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:58.614 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.607+0000 7f15a15ea740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639e616bd20 0x5639e61758a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:58.614 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.607+0000 7f15a15ea740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f15981a7680 con 0x5639e61661c0 2024-05-10T11:22:58.614 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.607+0000 7f15a15ea740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f15981a8af0 con 0x5639e6166bb0 2024-05-10T11:22:58.614 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.607+0000 7f15a15ea740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f15981a7d50 con 0x5639e616bd20 2024-05-10T11:22:58.615 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.607+0000 7f15a15ea740 10 client.0 _ll_register_callbacks cb 0x7f1588009260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:22:58.615 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.608+0000 7f1597fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639e61661c0 0x5639e61743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:58.615 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.608+0000 7f1597fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639e61661c0 0x5639e61743f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42896/0 (socket says 172.21.15.100:42896) 2024-05-10T11:22:58.615 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.608+0000 7f1597fff640 1 -- 172.21.15.100:0/716545199 learned_addr learned my addr 172.21.15.100:0/716545199 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:22:58.615 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.608+0000 7f159cd96640 1 --2- 172.21.15.100:0/716545199 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639e616bd20 0x5639e61758a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:58.615 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.608+0000 7f15977fe640 1 --2- 172.21.15.100:0/716545199 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639e6166bb0 0x5639e6174930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:58.615 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.608+0000 7f1597fff640 1 -- 172.21.15.100:0/716545199 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639e616bd20 msgr2=0x5639e61758a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:58.615 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.608+0000 7f1597fff640 1 --2- 172.21.15.100:0/716545199 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639e616bd20 0x5639e61758a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:58.615 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.608+0000 7f1597fff640 1 -- 172.21.15.100:0/716545199 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639e6166bb0 msgr2=0x5639e6174930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:22:58.615 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.608+0000 7f1597fff640 1 --2- 172.21.15.100:0/716545199 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639e6166bb0 0x5639e6174930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:58.615 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.608+0000 7f1597fff640 1 -- 172.21.15.100:0/716545199 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639e6175e90 con 0x5639e61661c0 2024-05-10T11:22:58.615 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.608+0000 7f1597fff640 1 --2- 172.21.15.100:0/716545199 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639e61661c0 0x5639e61743f0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f1588005ab0 tx=0x7f15880c6230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34d43f6961ce0a7c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:22:58.615 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.608+0000 7f15777fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:22:58.615 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.608+0000 7f15777fe640 1 -- 172.21.15.100:0/716545199 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1588049050 con 0x5639e61661c0 2024-05-10T11:22:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.608+0000 7f15777fe640 1 -- 172.21.15.100:0/716545199 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f15880c5950 con 0x5639e61661c0 2024-05-10T11:22:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.608+0000 7f15a15ea740 1 -- 172.21.15.100:0/716545199 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15880c9d90 con 0x5639e61661c0 2024-05-10T11:22:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.608+0000 7f15777fe640 1 -- 172.21.15.100:0/716545199 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f15880e43d0 con 0x5639e61661c0 2024-05-10T11:22:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.608+0000 7f15a15ea740 10 client.16317 fetch_fsmap learned FSMap version 30 2024-05-10T11:22:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.608+0000 7f15a15ea740 1 -- 172.21.15.100:0/716545199 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5639e6175e90 con 0x5639e61661c0 2024-05-10T11:22:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.608+0000 7f15777fe640 1 -- 172.21.15.100:0/716545199 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f15880c9d90 con 0x5639e61661c0 2024-05-10T11:22:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.609+0000 7f15777fe640 1 -- 172.21.15.100:0/716545199 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7f15880cf340 con 0x5639e61661c0 2024-05-10T11:22:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.609+0000 7f15a15ea740 10 client.16317 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T11:22:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.609+0000 7f15a15ea740 10 client.16317 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:22:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.609+0000 7f15a15ea740 20 client.16317 populate_metadata read hostname 'smithi100' 2024-05-10T11:22:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.609+0000 7f15a15ea740 1 --2- 172.21.15.100:0/716545199 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5639e61e6c10 0x5639e61e9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:22:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.609+0000 7f15a15ea740 4 client.16317 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:22:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.609+0000 7f15a15ea740 1 -- 172.21.15.100:0/716545199 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5639e60ce050 con 0x5639e61e6c10 2024-05-10T11:22:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.609+0000 7f15977fe640 1 --2- 172.21.15.100:0/716545199 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5639e61e6c10 0x5639e61e9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:22:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.609+0000 7f15777fe640 10 client.16317 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:22:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.609+0000 7f15977fe640 1 --2- 172.21.15.100:0/716545199 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5639e61e6c10 0x5639e61e9000 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cd06ef69d09ee9d server_cookie=f2d4d4d91a7521f in_seq=0 out_seq=0 2024-05-10T11:22:58.626 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.625+0000 7f15777fe640 1 -- 172.21.15.100:0/716545199 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5639e60ce050 con 0x5639e61e6c10 2024-05-10T11:22:58.626 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.625+0000 7f15777fe640 10 client.16317 handle_command_reply: tid=0 2024-05-10T11:22:58.626 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.625+0000 7f15a15ea740 1 client.16317 shutdown 2024-05-10T11:22:58.626 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.625+0000 7f15a15ea740 1 -- 172.21.15.100:0/716545199 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639e61661c0 msgr2=0x5639e61743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:58.626 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.625+0000 7f15a15ea740 1 --2- 172.21.15.100:0/716545199 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639e61661c0 0x5639e61743f0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f1588005ab0 tx=0x7f15880c6230 comp rx=0 tx=0).stop 2024-05-10T11:22:58.626 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.625+0000 7f15a15ea740 1 -- 172.21.15.100:0/716545199 shutdown_connections 2024-05-10T11:22:58.627 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.625+0000 7f15a15ea740 1 --2- 172.21.15.100:0/716545199 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5639e61e6c10 0x5639e61e9000 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:58.627 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.625+0000 7f15a15ea740 1 --2- 172.21.15.100:0/716545199 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639e616bd20 0x5639e61758a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:58.627 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.625+0000 7f15a15ea740 1 --2- 172.21.15.100:0/716545199 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639e6166bb0 0x5639e6174930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:58.627 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.625+0000 7f15777fe640 0 client.16317 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:22:58.627 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.625+0000 7f15a15ea740 1 --2- 172.21.15.100:0/716545199 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639e61661c0 0x5639e61743f0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:58.627 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.625+0000 7f15a15ea740 1 -- 172.21.15.100:0/716545199 >> 172.21.15.100:0/716545199 conn(0x5639e615d420 msgr2=0x5639e6160330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:58.627 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.626+0000 7f15a15ea740 1 -- 172.21.15.100:0/716545199 shutdown_connections 2024-05-10T11:22:58.627 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.626+0000 7f15a15ea740 1 -- 172.21.15.100:0/716545199 wait complete. 2024-05-10T11:22:58.627 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.626+0000 7f15a15ea740 20 client.16317 trim_cache size 0 max 16384 2024-05-10T11:22:58.627 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:22:58.627 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:22:58.627 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:22:58.628 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:22:58.628 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.626+0000 7f159e7f9640 1 -- 172.21.15.100:0/225779843 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f15800b28a0 msgr2=0x7f15800b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:58.628 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.626+0000 7f159e7f9640 1 --2- 172.21.15.100:0/225779843 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f15800b28a0 0x7f15800b4d60 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x5639e6167410 tx=0x7f1590004ab0 comp rx=0 tx=0).stop 2024-05-10T11:22:58.629 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.626+0000 7f159e7f9640 1 -- 172.21.15.100:0/225779843 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15981019c0 msgr2=0x7f159806bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:22:58.629 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.626+0000 7f159e7f9640 1 --2- 172.21.15.100:0/225779843 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15981019c0 0x7f159806bd70 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f1588009230 tx=0x7f15880040c0 comp rx=0 tx=0).stop 2024-05-10T11:22:58.629 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.626+0000 7f159e7f9640 1 -- 172.21.15.100:0/225779843 shutdown_connections 2024-05-10T11:22:58.629 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.627+0000 7f159e7f9640 1 --2- 172.21.15.100:0/225779843 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f15800b28a0 0x7f15800b4d60 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:58.629 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.627+0000 7f159e7f9640 1 --2- 172.21.15.100:0/225779843 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1598103570 0x7f159819fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:58.629 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.627+0000 7f159e7f9640 1 --2- 172.21.15.100:0/225779843 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1598102bb0 0x7f159819d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:58.629 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.627+0000 7f159e7f9640 1 --2- 172.21.15.100:0/225779843 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15981019c0 0x7f159806bd70 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:22:58.629 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.627+0000 7f159e7f9640 1 -- 172.21.15.100:0/225779843 >> 172.21.15.100:0/225779843 conn(0x7f15980fcf30 msgr2=0x7f1598105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:22:58.629 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.627+0000 7f159e7f9640 1 -- 172.21.15.100:0/225779843 shutdown_connections 2024-05-10T11:22:58.629 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:22:58.627+0000 7f159e7f9640 1 -- 172.21.15.100:0/225779843 wait complete. 2024-05-10T11:22:58.970 INFO:tasks.workunit.client.0.smithi100.stdout: 1 91341 2.69 MB/sec execute 53 sec latency 16.401 ms 2024-05-10T11:22:59.970 INFO:tasks.workunit.client.0.smithi100.stdout: 1 92049 2.76 MB/sec execute 54 sec latency 14.924 ms 2024-05-10T11:23:00.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:22:59 smithi178 ceph-mon[33359]: pgmap v558: 129 pgs: 129 active+clean; 50 MiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-05-10T11:23:00.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:22:59 smithi188 ceph-mon[34075]: pgmap v558: 129 pgs: 129 active+clean; 50 MiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-05-10T11:23:00.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:22:59 smithi100 ceph-mon[30296]: pgmap v558: 129 pgs: 129 active+clean; 50 MiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-05-10T11:23:00.970 INFO:tasks.workunit.client.0.smithi100.stdout: 1 92543 2.79 MB/sec execute 55 sec latency 15.102 ms 2024-05-10T11:23:01.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:00 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:23:01.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:00 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:23:01.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:00 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:23:01.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:00 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:23:01.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:00 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:23:01.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:00 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:23:01.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:00 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:23:01.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:00 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:23:01.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:00 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:23:01.970 INFO:tasks.workunit.client.0.smithi100.stdout: 1 92830 2.75 MB/sec execute 56 sec latency 15.291 ms 2024-05-10T11:23:02.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:01 smithi178 ceph-mon[33359]: pgmap v559: 129 pgs: 129 active+clean; 49 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 1.3 MiB/s wr, 157 op/s 2024-05-10T11:23:02.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:01 smithi188 ceph-mon[34075]: pgmap v559: 129 pgs: 129 active+clean; 49 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 1.3 MiB/s wr, 157 op/s 2024-05-10T11:23:02.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:01 smithi100 ceph-mon[30296]: pgmap v559: 129 pgs: 129 active+clean; 49 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 1.3 MiB/s wr, 157 op/s 2024-05-10T11:23:02.970 INFO:tasks.workunit.client.0.smithi100.stdout: 1 93337 2.73 MB/sec execute 57 sec latency 15.003 ms 2024-05-10T11:23:03.970 INFO:tasks.workunit.client.0.smithi100.stdout: 1 93953 2.74 MB/sec execute 58 sec latency 14.517 ms 2024-05-10T11:23:04.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:03 smithi178 ceph-mon[33359]: pgmap v560: 129 pgs: 129 active+clean; 50 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 1.4 MiB/s wr, 162 op/s 2024-05-10T11:23:04.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:03 smithi188 ceph-mon[34075]: pgmap v560: 129 pgs: 129 active+clean; 50 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 1.4 MiB/s wr, 162 op/s 2024-05-10T11:23:04.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:03 smithi100 ceph-mon[30296]: pgmap v560: 129 pgs: 129 active+clean; 50 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 1.4 MiB/s wr, 162 op/s 2024-05-10T11:23:04.970 INFO:tasks.workunit.client.0.smithi100.stdout: 1 94633 2.72 MB/sec execute 59 sec latency 28.290 ms 2024-05-10T11:23:05.970 INFO:tasks.workunit.client.0.smithi100.stdout: 1 95256 2.77 MB/sec execute 60 sec latency 16.381 ms 2024-05-10T11:23:06.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:05 smithi178 ceph-mon[33359]: pgmap v561: 129 pgs: 129 active+clean; 52 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 1.5 MiB/s wr, 158 op/s 2024-05-10T11:23:06.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:05 smithi188 ceph-mon[34075]: pgmap v561: 129 pgs: 129 active+clean; 52 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 1.5 MiB/s wr, 158 op/s 2024-05-10T11:23:06.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:05 smithi100 ceph-mon[30296]: pgmap v561: 129 pgs: 129 active+clean; 52 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 1.5 MiB/s wr, 158 op/s 2024-05-10T11:23:06.970 INFO:tasks.workunit.client.0.smithi100.stdout: 1 95795 2.81 MB/sec execute 61 sec latency 80.178 ms 2024-05-10T11:23:07.970 INFO:tasks.workunit.client.0.smithi100.stdout: 1 96172 2.79 MB/sec execute 62 sec latency 33.206 ms 2024-05-10T11:23:08.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:07 smithi178 ceph-mon[33359]: pgmap v562: 129 pgs: 129 active+clean; 51 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 1.5 MiB/s wr, 177 op/s 2024-05-10T11:23:08.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:07 smithi188 ceph-mon[34075]: pgmap v562: 129 pgs: 129 active+clean; 51 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 1.5 MiB/s wr, 177 op/s 2024-05-10T11:23:08.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:07 smithi100 ceph-mon[30296]: pgmap v562: 129 pgs: 129 active+clean; 51 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 1.5 MiB/s wr, 177 op/s 2024-05-10T11:23:08.970 INFO:tasks.workunit.client.0.smithi100.stdout: 1 96443 2.74 MB/sec execute 63 sec latency 15.130 ms 2024-05-10T11:23:09.970 INFO:tasks.workunit.client.0.smithi100.stdout: 1 96986 2.73 MB/sec execute 64 sec latency 16.520 ms 2024-05-10T11:23:10.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:09 smithi178 ceph-mon[33359]: pgmap v563: 129 pgs: 129 active+clean; 52 MiB data, 916 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 1.5 MiB/s wr, 158 op/s 2024-05-10T11:23:10.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:09 smithi188 ceph-mon[34075]: pgmap v563: 129 pgs: 129 active+clean; 52 MiB data, 916 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 1.5 MiB/s wr, 158 op/s 2024-05-10T11:23:10.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:09 smithi100 ceph-mon[30296]: pgmap v563: 129 pgs: 129 active+clean; 52 MiB data, 916 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 1.5 MiB/s wr, 158 op/s 2024-05-10T11:23:10.970 INFO:tasks.workunit.client.0.smithi100.stdout: 1 97551 2.74 MB/sec execute 65 sec latency 18.453 ms 2024-05-10T11:23:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.724+0000 7f58b6d0a640 1 -- 172.21.15.100:0/2865257173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58b01045d0 msgr2=0x7f58b010ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:11.728 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.724+0000 7f58b6d0a640 1 --2- 172.21.15.100:0/2865257173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58b01045d0 0x7f58b010ae60 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f58a8009ca0 tx=0x7f58a802a8c0 comp rx=0 tx=0).stop 2024-05-10T11:23:11.728 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.724+0000 7f58b6d0a640 1 -- 172.21.15.100:0/2865257173 shutdown_connections 2024-05-10T11:23:11.728 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.724+0000 7f58b6d0a640 1 --2- 172.21.15.100:0/2865257173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58b01045d0 0x7f58b010ae60 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:11.728 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.724+0000 7f58b6d0a640 1 --2- 172.21.15.100:0/2865257173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58b0103c10 0x7f58b0104090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:11.728 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.724+0000 7f58b6d0a640 1 --2- 172.21.15.100:0/2865257173 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58b0102a20 0x7f58b0102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:11.728 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.724+0000 7f58b6d0a640 1 -- 172.21.15.100:0/2865257173 >> 172.21.15.100:0/2865257173 conn(0x7f58b00fdf70 msgr2=0x7f58b01003d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:23:11.728 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.724+0000 7f58b6d0a640 1 -- 172.21.15.100:0/2865257173 shutdown_connections 2024-05-10T11:23:11.728 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.724+0000 7f58b6d0a640 1 -- 172.21.15.100:0/2865257173 wait complete. 2024-05-10T11:23:11.728 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.725+0000 7f58b6d0a640 1 Processor -- start 2024-05-10T11:23:11.728 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.725+0000 7f58b6d0a640 1 -- start start 2024-05-10T11:23:11.728 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.725+0000 7f58b6d0a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58b0102a20 0x7f58b0076760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:11.728 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.725+0000 7f58b6d0a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58b0103c10 0x7f58b0074da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:11.728 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.725+0000 7f58b6d0a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58b01045d0 0x7f58b00752e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.725+0000 7f58b6d0a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f58b010e430 con 0x7f58b0103c10 2024-05-10T11:23:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.725+0000 7f58b6d0a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f58b010e2b0 con 0x7f58b01045d0 2024-05-10T11:23:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.725+0000 7f58b6d0a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f58b010e5b0 con 0x7f58b0102a20 2024-05-10T11:23:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.725+0000 7f58affff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58b0103c10 0x7f58b0074da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.725+0000 7f58affff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58b0103c10 0x7f58b0074da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47974/0 (socket says 172.21.15.100:47974) 2024-05-10T11:23:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.725+0000 7f58affff640 1 -- 172.21.15.100:0/2334438859 learned_addr learned my addr 172.21.15.100:0/2334438859 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:23:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.725+0000 7f58b4aa6640 1 --2- 172.21.15.100:0/2334438859 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58b0102a20 0x7f58b0076760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.725+0000 7f58b52a7640 1 --2- 172.21.15.100:0/2334438859 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58b01045d0 0x7f58b00752e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.725+0000 7f58affff640 1 -- 172.21.15.100:0/2334438859 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58b0102a20 msgr2=0x7f58b0076760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.725+0000 7f58affff640 1 --2- 172.21.15.100:0/2334438859 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58b0102a20 0x7f58b0076760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.725+0000 7f58affff640 1 -- 172.21.15.100:0/2334438859 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58b01045d0 msgr2=0x7f58b00752e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.725+0000 7f58affff640 1 --2- 172.21.15.100:0/2334438859 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58b01045d0 0x7f58b00752e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.725+0000 7f58affff640 1 -- 172.21.15.100:0/2334438859 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58b0075b70 con 0x7f58b0103c10 2024-05-10T11:23:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.725+0000 7f58b52a7640 1 --2- 172.21.15.100:0/2334438859 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58b01045d0 0x7f58b00752e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:23:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.726+0000 7f58affff640 1 --2- 172.21.15.100:0/2334438859 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58b0103c10 0x7f58b0074da0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f589c00d950 tx=0x7f589c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:23:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.726+0000 7f58adffb640 1 -- 172.21.15.100:0/2334438859 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589c017030 con 0x7f58b0103c10 2024-05-10T11:23:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.726+0000 7f58adffb640 1 -- 172.21.15.100:0/2334438859 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f589c004020 con 0x7f58b0103c10 2024-05-10T11:23:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.726+0000 7f58b6d0a640 1 -- 172.21.15.100:0/2334438859 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58b01a2b30 con 0x7f58b0103c10 2024-05-10T11:23:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.726+0000 7f58adffb640 1 -- 172.21.15.100:0/2334438859 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589c0055d0 con 0x7f58b0103c10 2024-05-10T11:23:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.726+0000 7f58b6d0a640 1 -- 172.21.15.100:0/2334438859 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58b01a3080 con 0x7f58b0103c10 2024-05-10T11:23:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.729+0000 7f58adffb640 1 -- 172.21.15.100:0/2334438859 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f589c009a30 con 0x7f58b0103c10 2024-05-10T11:23:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.729+0000 7f58b9a23740 1 Processor -- start 2024-05-10T11:23:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.729+0000 7f58b9a23740 1 -- start start 2024-05-10T11:23:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.729+0000 7f58b9a23740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cae7da5db0 0x55cae7da61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.729+0000 7f58b9a23740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cae7da67a0 0x55cae7dab2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.729+0000 7f58b9a23740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cae7dab910 0x55cae7dadda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.729+0000 7f58b9a23740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55cae7d0a5d0 con 0x55cae7dab910 2024-05-10T11:23:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.729+0000 7f58b9a23740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55cae7cc3890 con 0x55cae7da5db0 2024-05-10T11:23:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.729+0000 7f58b9a23740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55cae7d06590 con 0x55cae7da67a0 2024-05-10T11:23:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.729+0000 7f58affff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cae7dab910 0x55cae7dadda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.729+0000 7f58affff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cae7dab910 0x55cae7dadda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47986/0 (socket says 172.21.15.100:47986) 2024-05-10T11:23:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.729+0000 7f58affff640 1 -- 172.21.15.100:0/2423352173 learned_addr learned my addr 172.21.15.100:0/2423352173 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:23:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.729+0000 7f58b52a7640 1 --2- 172.21.15.100:0/2423352173 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cae7da5db0 0x55cae7da61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.729+0000 7f58b4aa6640 1 --2- 172.21.15.100:0/2423352173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cae7da67a0 0x55cae7dab2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.729+0000 7f58adffb640 1 --2- 172.21.15.100:0/2334438859 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f58a00b28a0 0x7f58a00b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.729+0000 7f58affff640 1 -- 172.21.15.100:0/2423352173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cae7da67a0 msgr2=0x55cae7dab2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.729+0000 7f58affff640 1 --2- 172.21.15.100:0/2423352173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cae7da67a0 0x55cae7dab2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.729+0000 7f58affff640 1 -- 172.21.15.100:0/2423352173 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cae7da5db0 msgr2=0x55cae7da61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.729+0000 7f58affff640 1 --2- 172.21.15.100:0/2423352173 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cae7da5db0 0x55cae7da61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.729+0000 7f58affff640 1 -- 172.21.15.100:0/2423352173 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cae7a1c210 con 0x55cae7dab910 2024-05-10T11:23:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.729+0000 7f58b4aa6640 1 --2- 172.21.15.100:0/2423352173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cae7da67a0 0x55cae7dab2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:23:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.730+0000 7f58b52a7640 1 --2- 172.21.15.100:0/2334438859 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f58a00b28a0 0x7f58a00b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.730+0000 7f58adffb640 1 -- 172.21.15.100:0/2334438859 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f589c0e1fe0 con 0x7f58b0103c10 2024-05-10T11:23:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.730+0000 7f58affff640 1 --2- 172.21.15.100:0/2423352173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cae7dab910 0x55cae7dadda0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f589c0c35a0 tx=0x7f589c0e5f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7117745cb154b47c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:23:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.730+0000 7f589affd640 1 -- 172.21.15.100:0/2423352173 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589c0a9070 con 0x55cae7dab910 2024-05-10T11:23:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.730+0000 7f589affd640 1 -- 172.21.15.100:0/2423352173 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f589c0a8db0 con 0x55cae7dab910 2024-05-10T11:23:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.730+0000 7f589affd640 1 -- 172.21.15.100:0/2423352173 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589c0a8a10 con 0x55cae7dab910 2024-05-10T11:23:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.731+0000 7f58b52a7640 1 --2- 172.21.15.100:0/2334438859 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f58a00b28a0 0x7f58a00b4d60 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x55cae7da7000 tx=0x7f58a80095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:23:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.731+0000 7f58b9a23740 1 -- 172.21.15.100:0/2423352173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cae7dab910 msgr2=0x55cae7dadda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.731+0000 7f58b9a23740 1 --2- 172.21.15.100:0/2423352173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cae7dab910 0x55cae7dadda0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f589c0c35a0 tx=0x7f589c0e5f60 comp rx=0 tx=0).stop 2024-05-10T11:23:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.731+0000 7f58b9a23740 1 -- 172.21.15.100:0/2423352173 shutdown_connections 2024-05-10T11:23:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.731+0000 7f58b9a23740 1 --2- 172.21.15.100:0/2423352173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cae7dab910 0x55cae7dadda0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.731+0000 7f58b9a23740 1 --2- 172.21.15.100:0/2423352173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cae7da67a0 0x55cae7dab2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.731+0000 7f58b9a23740 1 --2- 172.21.15.100:0/2423352173 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cae7da5db0 0x55cae7da61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.731+0000 7f58b9a23740 1 -- 172.21.15.100:0/2423352173 >> 172.21.15.100:0/2423352173 conn(0x55cae7da1420 msgr2=0x55cae7da1820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:23:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.731+0000 7f58b9a23740 1 -- 172.21.15.100:0/2423352173 shutdown_connections 2024-05-10T11:23:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.731+0000 7f58b9a23740 1 -- 172.21.15.100:0/2423352173 wait complete. 2024-05-10T11:23:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.731+0000 7f58b9a23740 1 -- start start 2024-05-10T11:23:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.731+0000 7f58b9a23740 1 Processor -- start 2024-05-10T11:23:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.732+0000 7f58b9a23740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cae7da5db0 0x55cae7dafff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.732+0000 7f58b9a23740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cae7da67a0 0x55cae7db0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.732+0000 7f58b9a23740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cae7dab910 0x55cae7db7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.732+0000 7f58b9a23740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55cae7dae8c0 con 0x55cae7da5db0 2024-05-10T11:23:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.732+0000 7f58b9a23740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55cae7cc7370 con 0x55cae7da67a0 2024-05-10T11:23:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.732+0000 7f58b9a23740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55cae7daea40 con 0x55cae7dab910 2024-05-10T11:23:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.732+0000 7f58b9a23740 10 client.0 _ll_register_callbacks cb 0x55cae7a1ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:23:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.732+0000 7f58b4aa6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cae7da5db0 0x55cae7dafff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.732+0000 7f58b4aa6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cae7da5db0 0x55cae7dafff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47994/0 (socket says 172.21.15.100:47994) 2024-05-10T11:23:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.732+0000 7f58b4aa6640 1 -- 172.21.15.100:0/4210116486 learned_addr learned my addr 172.21.15.100:0/4210116486 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:23:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.732+0000 7f58affff640 1 --2- 172.21.15.100:0/4210116486 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cae7da67a0 0x55cae7db0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.732+0000 7f58b52a7640 1 --2- 172.21.15.100:0/4210116486 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cae7dab910 0x55cae7db7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.732+0000 7f58b4aa6640 1 -- 172.21.15.100:0/4210116486 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cae7dab910 msgr2=0x55cae7db7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.732+0000 7f58b4aa6640 1 --2- 172.21.15.100:0/4210116486 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cae7dab910 0x55cae7db7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.732+0000 7f58b4aa6640 1 -- 172.21.15.100:0/4210116486 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cae7da67a0 msgr2=0x55cae7db0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.732+0000 7f58b4aa6640 1 --2- 172.21.15.100:0/4210116486 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cae7da67a0 0x55cae7db0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.732+0000 7f58b4aa6640 1 -- 172.21.15.100:0/4210116486 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cae7ae8620 con 0x55cae7da5db0 2024-05-10T11:23:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.732+0000 7f58b4aa6640 1 --2- 172.21.15.100:0/4210116486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cae7da5db0 0x55cae7dafff0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f58a4012780 tx=0x7f58a4012c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cac68d1cb79dc039 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:23:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.732+0000 7f588ffff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:23:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.732+0000 7f588ffff640 1 -- 172.21.15.100:0/4210116486 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a4010d60 con 0x55cae7da5db0 2024-05-10T11:23:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.732+0000 7f588ffff640 1 -- 172.21.15.100:0/4210116486 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f58a4005020 con 0x55cae7da5db0 2024-05-10T11:23:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.732+0000 7f58b9a23740 1 -- 172.21.15.100:0/4210116486 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f589c0a8db0 con 0x55cae7da5db0 2024-05-10T11:23:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.733+0000 7f588ffff640 1 -- 172.21.15.100:0/4210116486 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f58a4025050 con 0x55cae7da5db0 2024-05-10T11:23:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.733+0000 7f58b9a23740 10 client.16329 fetch_fsmap learned FSMap version 30 2024-05-10T11:23:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.733+0000 7f58b9a23740 1 -- 172.21.15.100:0/4210116486 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cae7dba620 con 0x55cae7da5db0 2024-05-10T11:23:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.733+0000 7f588ffff640 1 -- 172.21.15.100:0/4210116486 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f589c0a8db0 con 0x55cae7da5db0 2024-05-10T11:23:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.733+0000 7f588ffff640 1 -- 172.21.15.100:0/4210116486 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7f58a401a430 con 0x55cae7da5db0 2024-05-10T11:23:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.733+0000 7f58b9a23740 10 client.16329 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T11:23:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.733+0000 7f58b9a23740 10 client.16329 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:23:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.733+0000 7f58b9a23740 20 client.16329 populate_metadata read hostname 'smithi100' 2024-05-10T11:23:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.733+0000 7f58b9a23740 1 --2- 172.21.15.100:0/4210116486 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55cae7dbbd60 0x55cae7dbe200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.733+0000 7f58b9a23740 4 client.16329 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:23:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.733+0000 7f58b9a23740 1 -- 172.21.15.100:0/4210116486 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cae7d181e0 con 0x55cae7dbbd60 2024-05-10T11:23:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.733+0000 7f58affff640 1 --2- 172.21.15.100:0/4210116486 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55cae7dbbd60 0x55cae7dbe200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.734+0000 7f58affff640 1 --2- 172.21.15.100:0/4210116486 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55cae7dbbd60 0x55cae7dbe200 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca69000cbd63c939 server_cookie=5b90b0590211e93f in_seq=0 out_seq=0 2024-05-10T11:23:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.734+0000 7f588ffff640 10 client.16329 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:23:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.734+0000 7f588ffff640 1 -- 172.21.15.100:0/4210116486 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55cae7d181e0 con 0x55cae7dbbd60 2024-05-10T11:23:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.734+0000 7f588ffff640 10 client.16329 handle_command_reply: tid=0 2024-05-10T11:23:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.734+0000 7f58b9a23740 1 client.16329 shutdown 2024-05-10T11:23:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.735+0000 7f58b9a23740 1 -- 172.21.15.100:0/4210116486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cae7da5db0 msgr2=0x55cae7dafff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.735+0000 7f58b9a23740 1 --2- 172.21.15.100:0/4210116486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cae7da5db0 0x55cae7dafff0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f58a4012780 tx=0x7f58a4012c40 comp rx=0 tx=0).stop 2024-05-10T11:23:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.735+0000 7f58b9a23740 1 -- 172.21.15.100:0/4210116486 shutdown_connections 2024-05-10T11:23:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.735+0000 7f58b9a23740 1 --2- 172.21.15.100:0/4210116486 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55cae7dbbd60 0x55cae7dbe200 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.735+0000 7f58b9a23740 1 --2- 172.21.15.100:0/4210116486 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cae7dab910 0x55cae7db7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.735+0000 7f58b9a23740 1 --2- 172.21.15.100:0/4210116486 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cae7da67a0 0x55cae7db0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.735+0000 7f58b9a23740 1 --2- 172.21.15.100:0/4210116486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cae7da5db0 0x55cae7dafff0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.735+0000 7f58b9a23740 1 -- 172.21.15.100:0/4210116486 >> 172.21.15.100:0/4210116486 conn(0x55cae7da1420 msgr2=0x55cae7da0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:23:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.735+0000 7f588ffff640 0 client.16329 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:23:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.735+0000 7f58b9a23740 1 -- 172.21.15.100:0/4210116486 shutdown_connections 2024-05-10T11:23:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.735+0000 7f58b9a23740 1 -- 172.21.15.100:0/4210116486 wait complete. 2024-05-10T11:23:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.735+0000 7f58b9a23740 20 client.16329 trim_cache size 0 max 16384 2024-05-10T11:23:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.753+0000 7f58b9a23740 1 Processor -- start 2024-05-10T11:23:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.753+0000 7f58b9a23740 1 -- start start 2024-05-10T11:23:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.753+0000 7f58b9a23740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cae7da5db0 0x55cae7db02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.753+0000 7f58b9a23740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cae7da67a0 0x55cae7da0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.753+0000 7f58b9a23740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cae7dab910 0x55cae7da0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.753+0000 7f58b9a23740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f58b01afde0 con 0x55cae7da5db0 2024-05-10T11:23:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.753+0000 7f58b9a23740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f58b01ad6b0 con 0x55cae7dab910 2024-05-10T11:23:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.753+0000 7f58b9a23740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f58b01ab500 con 0x55cae7da67a0 2024-05-10T11:23:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.753+0000 7f58b4aa6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cae7da5db0 0x55cae7db02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.753+0000 7f58b4aa6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cae7da5db0 0x55cae7db02d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48002/0 (socket says 172.21.15.100:48002) 2024-05-10T11:23:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.753+0000 7f58b4aa6640 1 -- 172.21.15.100:0/940900748 learned_addr learned my addr 172.21.15.100:0/940900748 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:23:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.753+0000 7f58affff640 1 --2- 172.21.15.100:0/940900748 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cae7da67a0 0x55cae7da0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.753+0000 7f58b52a7640 1 --2- 172.21.15.100:0/940900748 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cae7dab910 0x55cae7da0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.753+0000 7f58b4aa6640 1 -- 172.21.15.100:0/940900748 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cae7da67a0 msgr2=0x55cae7da0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.753+0000 7f58b4aa6640 1 --2- 172.21.15.100:0/940900748 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cae7da67a0 0x55cae7da0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.753+0000 7f58b4aa6640 1 -- 172.21.15.100:0/940900748 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cae7dab910 msgr2=0x55cae7da0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.753+0000 7f58b4aa6640 1 --2- 172.21.15.100:0/940900748 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cae7dab910 0x55cae7da0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.753+0000 7f58b4aa6640 1 -- 172.21.15.100:0/940900748 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cae7da1250 con 0x55cae7da5db0 2024-05-10T11:23:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.753+0000 7f58b4aa6640 1 --2- 172.21.15.100:0/940900748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cae7da5db0 0x55cae7db02d0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f58a4000c00 tx=0x7f58a4005e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7595f299b87eaf60 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:23:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.753+0000 7f589a7fc640 1 -- 172.21.15.100:0/940900748 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a402d070 con 0x55cae7da5db0 2024-05-10T11:23:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.754+0000 7f589a7fc640 1 -- 172.21.15.100:0/940900748 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f58a401a430 con 0x55cae7da5db0 2024-05-10T11:23:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.754+0000 7f589a7fc640 1 -- 172.21.15.100:0/940900748 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a40035f0 con 0x55cae7da5db0 2024-05-10T11:23:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.754+0000 7f58b9a23740 1 -- 172.21.15.100:0/940900748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cae7da5db0 msgr2=0x55cae7db02d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.754+0000 7f58b9a23740 1 --2- 172.21.15.100:0/940900748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cae7da5db0 0x55cae7db02d0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f58a4000c00 tx=0x7f58a4005e60 comp rx=0 tx=0).stop 2024-05-10T11:23:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.754+0000 7f58b9a23740 1 -- 172.21.15.100:0/940900748 shutdown_connections 2024-05-10T11:23:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.754+0000 7f58b9a23740 1 --2- 172.21.15.100:0/940900748 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cae7dab910 0x55cae7da0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:11.760 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.754+0000 7f58b9a23740 1 --2- 172.21.15.100:0/940900748 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cae7da67a0 0x55cae7da0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:11.760 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.754+0000 7f58b9a23740 1 --2- 172.21.15.100:0/940900748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cae7da5db0 0x55cae7db02d0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:11.760 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.754+0000 7f58b9a23740 1 -- 172.21.15.100:0/940900748 >> 172.21.15.100:0/940900748 conn(0x55cae7da1420 msgr2=0x55cae7d9fcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:23:11.760 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.754+0000 7f58b9a23740 1 -- 172.21.15.100:0/940900748 shutdown_connections 2024-05-10T11:23:11.760 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.754+0000 7f58b9a23740 1 -- 172.21.15.100:0/940900748 wait complete. 2024-05-10T11:23:11.760 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.754+0000 7f58b9a23740 1 -- start start 2024-05-10T11:23:11.760 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.755+0000 7f58b9a23740 1 Processor -- start 2024-05-10T11:23:11.760 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.755+0000 7f58b9a23740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cae7da5db0 0x55cae7db45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:11.760 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.755+0000 7f58b9a23740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cae7da67a0 0x55cae7db4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:11.760 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.755+0000 7f58b9a23740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cae7dab910 0x55cae7db5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:11.760 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.755+0000 7f58b9a23740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f58b01abb70 con 0x55cae7dab910 2024-05-10T11:23:11.760 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.755+0000 7f58b9a23740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f58b01acfe0 con 0x55cae7da67a0 2024-05-10T11:23:11.760 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.755+0000 7f58b9a23740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f58b01ac240 con 0x55cae7da5db0 2024-05-10T11:23:11.760 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.755+0000 7f58b9a23740 10 client.0 _ll_register_callbacks cb 0x7f58a400a630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:23:11.760 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.755+0000 7f58b52a7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cae7dab910 0x55cae7db5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:11.761 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.755+0000 7f58b52a7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cae7dab910 0x55cae7db5b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48010/0 (socket says 172.21.15.100:48010) 2024-05-10T11:23:11.761 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.755+0000 7f58b4aa6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cae7da5db0 0x55cae7db45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:11.761 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.755+0000 7f58b52a7640 1 -- 172.21.15.100:0/3279567066 learned_addr learned my addr 172.21.15.100:0/3279567066 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:23:11.761 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.755+0000 7f58affff640 1 --2- 172.21.15.100:0/3279567066 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cae7da67a0 0x55cae7db4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:11.761 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.755+0000 7f58b52a7640 1 -- 172.21.15.100:0/3279567066 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cae7da5db0 msgr2=0x55cae7db45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:11.761 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.755+0000 7f58b52a7640 1 --2- 172.21.15.100:0/3279567066 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cae7da5db0 0x55cae7db45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:11.761 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.755+0000 7f58b52a7640 1 -- 172.21.15.100:0/3279567066 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cae7da67a0 msgr2=0x55cae7db4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:11.761 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.755+0000 7f58b52a7640 1 --2- 172.21.15.100:0/3279567066 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cae7da67a0 0x55cae7db4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:11.761 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.755+0000 7f58b52a7640 1 -- 172.21.15.100:0/3279567066 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cae7db6100 con 0x55cae7dab910 2024-05-10T11:23:11.761 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.756+0000 7f58b52a7640 1 --2- 172.21.15.100:0/3279567066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cae7dab910 0x55cae7db5b10 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f58a8009ca0 tx=0x7f58a802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91c70f790077c50d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:23:11.761 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.756+0000 7f588ffff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:23:11.761 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.756+0000 7f588ffff640 1 -- 172.21.15.100:0/3279567066 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a8059060 con 0x55cae7dab910 2024-05-10T11:23:11.761 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.756+0000 7f588ffff640 1 -- 172.21.15.100:0/3279567066 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f58a80305c0 con 0x55cae7dab910 2024-05-10T11:23:11.761 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.756+0000 7f58b9a23740 1 -- 172.21.15.100:0/3279567066 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58a401a430 con 0x55cae7dab910 2024-05-10T11:23:11.761 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.756+0000 7f588ffff640 1 -- 172.21.15.100:0/3279567066 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f58a8003e30 con 0x55cae7dab910 2024-05-10T11:23:11.762 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.756+0000 7f58b9a23740 10 client.16335 fetch_fsmap learned FSMap version 30 2024-05-10T11:23:11.762 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.756+0000 7f58b9a23740 1 -- 172.21.15.100:0/3279567066 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cae7e269a0 con 0x55cae7dab910 2024-05-10T11:23:11.762 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.756+0000 7f588ffff640 1 -- 172.21.15.100:0/3279567066 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f58a401a430 con 0x55cae7dab910 2024-05-10T11:23:11.762 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.756+0000 7f588ffff640 1 -- 172.21.15.100:0/3279567066 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7f58a8053030 con 0x55cae7dab910 2024-05-10T11:23:11.762 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.756+0000 7f58b9a23740 10 client.16335 fetch_fsmap finished waiting for FSMap version 30 2024-05-10T11:23:11.762 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.756+0000 7f58b9a23740 10 client.16335 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:23:11.762 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.756+0000 7f58b9a23740 20 client.16335 populate_metadata read hostname 'smithi100' 2024-05-10T11:23:11.762 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.756+0000 7f58b9a23740 1 --2- 172.21.15.100:0/3279567066 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55cae7e27430 0x55cae7e29850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:11.762 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.756+0000 7f58b9a23740 4 client.16335 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:23:11.762 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.756+0000 7f58b9a23740 1 -- 172.21.15.100:0/3279567066 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cae7d12050 con 0x55cae7e27430 2024-05-10T11:23:11.762 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.757+0000 7f58b4aa6640 1 --2- 172.21.15.100:0/3279567066 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55cae7e27430 0x55cae7e29850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:11.762 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.757+0000 7f58b4aa6640 1 --2- 172.21.15.100:0/3279567066 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55cae7e27430 0x55cae7e29850 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f528e8514cb8a9ab server_cookie=8a5f89e4e33064ad in_seq=0 out_seq=0 2024-05-10T11:23:11.762 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.757+0000 7f588ffff640 10 client.16335 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:23:11.778 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.776+0000 7f588ffff640 1 -- 172.21.15.100:0/3279567066 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cae7d12050 con 0x55cae7e27430 2024-05-10T11:23:11.778 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.776+0000 7f588ffff640 10 client.16335 handle_command_reply: tid=0 2024-05-10T11:23:11.778 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.776+0000 7f58b9a23740 1 client.16335 shutdown 2024-05-10T11:23:11.778 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.776+0000 7f58b9a23740 1 -- 172.21.15.100:0/3279567066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cae7dab910 msgr2=0x55cae7db5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:11.778 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.776+0000 7f58b9a23740 1 --2- 172.21.15.100:0/3279567066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cae7dab910 0x55cae7db5b10 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f58a8009ca0 tx=0x7f58a802a8c0 comp rx=0 tx=0).stop 2024-05-10T11:23:11.778 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.776+0000 7f58b9a23740 1 -- 172.21.15.100:0/3279567066 shutdown_connections 2024-05-10T11:23:11.778 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.776+0000 7f58b9a23740 1 --2- 172.21.15.100:0/3279567066 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55cae7e27430 0x55cae7e29850 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:11.778 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.776+0000 7f58b9a23740 1 --2- 172.21.15.100:0/3279567066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cae7dab910 0x55cae7db5b10 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:11.778 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.776+0000 7f58b9a23740 1 --2- 172.21.15.100:0/3279567066 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cae7da67a0 0x55cae7db4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:11.778 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.776+0000 7f58b9a23740 1 --2- 172.21.15.100:0/3279567066 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cae7da5db0 0x55cae7db45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:11.779 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.777+0000 7f58b9a23740 1 -- 172.21.15.100:0/3279567066 >> 172.21.15.100:0/3279567066 conn(0x55cae7da1420 msgr2=0x55cae7da0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:23:11.779 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.777+0000 7f588ffff640 0 client.16335 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:23:11.779 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.777+0000 7f58b9a23740 1 -- 172.21.15.100:0/3279567066 shutdown_connections 2024-05-10T11:23:11.779 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.777+0000 7f58b9a23740 1 -- 172.21.15.100:0/3279567066 wait complete. 2024-05-10T11:23:11.779 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.777+0000 7f58b9a23740 20 client.16335 trim_cache size 0 max 16384 2024-05-10T11:23:11.779 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:23:11.779 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:23:11.779 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:23:11.779 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:23:11.780 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.777+0000 7f58b6d0a640 1 -- 172.21.15.100:0/2334438859 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f58a00b28a0 msgr2=0x7f58a00b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:11.780 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.777+0000 7f58b6d0a640 1 --2- 172.21.15.100:0/2334438859 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f58a00b28a0 0x7f58a00b4d60 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x55cae7da7000 tx=0x7f58a80095c0 comp rx=0 tx=0).stop 2024-05-10T11:23:11.780 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.777+0000 7f58b6d0a640 1 -- 172.21.15.100:0/2334438859 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58b0103c10 msgr2=0x7f58b0074da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:11.780 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.777+0000 7f58b6d0a640 1 --2- 172.21.15.100:0/2334438859 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58b0103c10 0x7f58b0074da0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f589c00d950 tx=0x7f589c00de10 comp rx=0 tx=0).stop 2024-05-10T11:23:11.780 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.777+0000 7f58b6d0a640 1 -- 172.21.15.100:0/2334438859 shutdown_connections 2024-05-10T11:23:11.780 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.777+0000 7f58b6d0a640 1 --2- 172.21.15.100:0/2334438859 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f58a00b28a0 0x7f58a00b4d60 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:11.781 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.777+0000 7f58b6d0a640 1 --2- 172.21.15.100:0/2334438859 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58b01045d0 0x7f58b00752e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:11.781 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.777+0000 7f58b6d0a640 1 --2- 172.21.15.100:0/2334438859 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58b0103c10 0x7f58b0074da0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:11.781 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.777+0000 7f58b6d0a640 1 --2- 172.21.15.100:0/2334438859 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58b0102a20 0x7f58b0076760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:11.781 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.777+0000 7f58b6d0a640 1 -- 172.21.15.100:0/2334438859 >> 172.21.15.100:0/2334438859 conn(0x7f58b00fdf70 msgr2=0x7f58b0106e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:23:11.781 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.777+0000 7f58b6d0a640 1 -- 172.21.15.100:0/2334438859 shutdown_connections 2024-05-10T11:23:11.781 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:11.778+0000 7f58b6d0a640 1 -- 172.21.15.100:0/2334438859 wait complete. 2024-05-10T11:23:11.838 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:11 smithi100 ceph-mon[30296]: pgmap v564: 129 pgs: 129 active+clean; 52 MiB data, 940 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 1.3 MiB/s wr, 134 op/s 2024-05-10T11:23:11.970 INFO:tasks.workunit.client.0.smithi100.stdout: 1 98206 2.73 MB/sec execute 66 sec latency 18.976 ms 2024-05-10T11:23:12.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:11 smithi178 ceph-mon[33359]: pgmap v564: 129 pgs: 129 active+clean; 52 MiB data, 940 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 1.3 MiB/s wr, 134 op/s 2024-05-10T11:23:12.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:11 smithi188 ceph-mon[34075]: pgmap v564: 129 pgs: 129 active+clean; 52 MiB data, 940 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 1.3 MiB/s wr, 134 op/s 2024-05-10T11:23:12.970 INFO:tasks.workunit.client.0.smithi100.stdout: 1 98841 2.77 MB/sec execute 67 sec latency 12.711 ms 2024-05-10T11:23:13.970 INFO:tasks.workunit.client.0.smithi100.stdout: 1 99392 2.80 MB/sec execute 68 sec latency 14.911 ms 2024-05-10T11:23:14.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:13 smithi178 ceph-mon[33359]: pgmap v565: 129 pgs: 129 active+clean; 51 MiB data, 958 MiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 1.4 MiB/s wr, 154 op/s 2024-05-10T11:23:14.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:13 smithi188 ceph-mon[34075]: pgmap v565: 129 pgs: 129 active+clean; 51 MiB data, 958 MiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 1.4 MiB/s wr, 154 op/s 2024-05-10T11:23:14.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:13 smithi100 ceph-mon[30296]: pgmap v565: 129 pgs: 129 active+clean; 51 MiB data, 958 MiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 1.4 MiB/s wr, 154 op/s 2024-05-10T11:23:14.970 INFO:tasks.workunit.client.0.smithi100.stdout: 1 99744 2.78 MB/sec execute 69 sec latency 15.476 ms 2024-05-10T11:23:15.970 INFO:tasks.workunit.client.0.smithi100.stdout: 1 100047 2.75 MB/sec execute 70 sec latency 15.105 ms 2024-05-10T11:23:16.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:15 smithi178 ceph-mon[33359]: pgmap v566: 129 pgs: 129 active+clean; 52 MiB data, 951 MiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 1.4 MiB/s wr, 153 op/s 2024-05-10T11:23:16.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:15 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:23:16.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:15 smithi188 ceph-mon[34075]: pgmap v566: 129 pgs: 129 active+clean; 52 MiB data, 951 MiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 1.4 MiB/s wr, 153 op/s 2024-05-10T11:23:16.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:15 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:23:16.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:15 smithi100 ceph-mon[30296]: pgmap v566: 129 pgs: 129 active+clean; 52 MiB data, 951 MiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 1.4 MiB/s wr, 153 op/s 2024-05-10T11:23:16.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:15 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:23:16.971 INFO:tasks.workunit.client.0.smithi100.stdout: 1 100568 2.73 MB/sec execute 71 sec latency 15.165 ms 2024-05-10T11:23:17.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:16 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:23:17.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:16 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:23:17.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:16 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:23:17.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:16 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:23:17.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:16 smithi178 ceph-mon[33359]: Adjusting osd_memory_target on smithi178 to 176.1M 2024-05-10T11:23:17.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:16 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:23:17.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:16 smithi178 ceph-mon[33359]: Unable to set osd_memory_target on smithi178 to 184736563: error parsing value: Value '184736563' is below minimum 939524096 2024-05-10T11:23:17.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:16 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:23:17.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:16 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:23:17.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:16 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:23:17.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:16 smithi178 ceph-mon[33359]: Adjusting osd_memory_target on smithi100 to 176.1M 2024-05-10T11:23:17.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:16 smithi178 ceph-mon[33359]: Unable to set osd_memory_target on smithi100 to 184709324: error parsing value: Value '184709324' is below minimum 939524096 2024-05-10T11:23:17.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:16 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:23:17.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:16 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:23:17.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:16 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:23:17.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:16 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:23:17.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:16 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:23:17.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:16 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:23:17.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:16 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:23:17.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:16 smithi188 ceph-mon[34075]: Adjusting osd_memory_target on smithi178 to 176.1M 2024-05-10T11:23:17.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:16 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:23:17.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:16 smithi188 ceph-mon[34075]: Unable to set osd_memory_target on smithi178 to 184736563: error parsing value: Value '184736563' is below minimum 939524096 2024-05-10T11:23:17.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:16 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:23:17.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:16 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:23:17.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:16 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:23:17.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:16 smithi188 ceph-mon[34075]: Adjusting osd_memory_target on smithi100 to 176.1M 2024-05-10T11:23:17.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:16 smithi188 ceph-mon[34075]: Unable to set osd_memory_target on smithi100 to 184709324: error parsing value: Value '184709324' is below minimum 939524096 2024-05-10T11:23:17.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:16 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:23:17.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:16 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:23:17.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:16 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:23:17.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:16 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:23:17.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:16 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:23:17.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:16 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:23:17.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:16 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:23:17.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:16 smithi100 ceph-mon[30296]: Adjusting osd_memory_target on smithi178 to 176.1M 2024-05-10T11:23:17.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:16 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:23:17.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:16 smithi100 ceph-mon[30296]: Unable to set osd_memory_target on smithi178 to 184736563: error parsing value: Value '184736563' is below minimum 939524096 2024-05-10T11:23:17.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:16 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:23:17.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:16 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:23:17.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:16 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:23:17.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:16 smithi100 ceph-mon[30296]: Adjusting osd_memory_target on smithi100 to 176.1M 2024-05-10T11:23:17.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:16 smithi100 ceph-mon[30296]: Unable to set osd_memory_target on smithi100 to 184709324: error parsing value: Value '184709324' is below minimum 939524096 2024-05-10T11:23:17.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:16 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:23:17.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:16 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:23:17.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:16 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:23:17.970 INFO:tasks.workunit.client.0.smithi100.stdout: 1 101208 2.74 MB/sec execute 72 sec latency 15.865 ms 2024-05-10T11:23:18.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:17 smithi178 ceph-mon[33359]: pgmap v567: 129 pgs: 129 active+clean; 54 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 1.3 MiB/s wr, 154 op/s 2024-05-10T11:23:18.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:17 smithi188 ceph-mon[34075]: pgmap v567: 129 pgs: 129 active+clean; 54 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 1.3 MiB/s wr, 154 op/s 2024-05-10T11:23:18.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:17 smithi100 ceph-mon[30296]: pgmap v567: 129 pgs: 129 active+clean; 54 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 1.3 MiB/s wr, 154 op/s 2024-05-10T11:23:18.970 INFO:tasks.workunit.client.0.smithi100.stdout: 1 101803 2.73 MB/sec execute 73 sec latency 15.545 ms 2024-05-10T11:23:19.970 INFO:tasks.workunit.client.0.smithi100.stdout: 1 102623 2.79 MB/sec execute 74 sec latency 14.652 ms 2024-05-10T11:23:20.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:19 smithi178 ceph-mon[33359]: pgmap v568: 129 pgs: 129 active+clean; 55 MiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 1.3 MiB/s wr, 153 op/s 2024-05-10T11:23:20.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:19 smithi188 ceph-mon[34075]: pgmap v568: 129 pgs: 129 active+clean; 55 MiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 1.3 MiB/s wr, 153 op/s 2024-05-10T11:23:20.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:19 smithi100 ceph-mon[30296]: pgmap v568: 129 pgs: 129 active+clean; 55 MiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 1.3 MiB/s wr, 153 op/s 2024-05-10T11:23:20.971 INFO:tasks.workunit.client.0.smithi100.stdout: 1 103026 2.80 MB/sec execute 75 sec latency 17.310 ms 2024-05-10T11:23:21.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:20 smithi178 ceph-mon[33359]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T11:23:21.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:20 smithi188 ceph-mon[34075]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T11:23:21.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:20 smithi100 ceph-mon[30296]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T11:23:21.970 INFO:tasks.workunit.client.0.smithi100.stdout: 1 103335 2.78 MB/sec execute 76 sec latency 16.056 ms 2024-05-10T11:23:22.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:21 smithi178 ceph-mon[33359]: pgmap v569: 129 pgs: 129 active+clean; 54 MiB data, 977 MiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 1.1 MiB/s wr, 135 op/s 2024-05-10T11:23:22.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:21 smithi188 ceph-mon[34075]: pgmap v569: 129 pgs: 129 active+clean; 54 MiB data, 977 MiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 1.1 MiB/s wr, 135 op/s 2024-05-10T11:23:22.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:21 smithi100 ceph-mon[30296]: pgmap v569: 129 pgs: 129 active+clean; 54 MiB data, 977 MiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 1.1 MiB/s wr, 135 op/s 2024-05-10T11:23:22.970 INFO:tasks.workunit.client.0.smithi100.stdout: 1 103719 2.75 MB/sec execute 77 sec latency 15.466 ms 2024-05-10T11:23:23.970 INFO:tasks.workunit.client.0.smithi100.stdout: 1 104302 2.77 MB/sec execute 78 sec latency 15.495 ms 2024-05-10T11:23:24.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:23 smithi178 ceph-mon[33359]: pgmap v570: 129 pgs: 129 active+clean; 55 MiB data, 986 MiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 1.4 MiB/s wr, 158 op/s 2024-05-10T11:23:24.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:23 smithi188 ceph-mon[34075]: pgmap v570: 129 pgs: 129 active+clean; 55 MiB data, 986 MiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 1.4 MiB/s wr, 158 op/s 2024-05-10T11:23:24.283 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:23 smithi100 ceph-mon[30296]: pgmap v570: 129 pgs: 129 active+clean; 55 MiB data, 986 MiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 1.4 MiB/s wr, 158 op/s 2024-05-10T11:23:24.882 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.879+0000 7fa4b952b640 1 -- 172.21.15.100:0/2762238392 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4b4103c30 msgr2=0x7fa4b41040b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:24.882 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.879+0000 7fa4b952b640 1 --2- 172.21.15.100:0/2762238392 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4b4103c30 0x7fa4b41040b0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8009900 tx=0x7fa4a802a410 comp rx=0 tx=0).stop 2024-05-10T11:23:24.882 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.879+0000 7fa4b952b640 1 -- 172.21.15.100:0/2762238392 shutdown_connections 2024-05-10T11:23:24.882 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.879+0000 7fa4b952b640 1 --2- 172.21.15.100:0/2762238392 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4b41045f0 0x7fa4b410ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:24.882 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.879+0000 7fa4b952b640 1 --2- 172.21.15.100:0/2762238392 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4b4103c30 0x7fa4b41040b0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:24.882 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.879+0000 7fa4b952b640 1 --2- 172.21.15.100:0/2762238392 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4b4102a40 0x7fa4b4102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:24.882 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.879+0000 7fa4b952b640 1 -- 172.21.15.100:0/2762238392 >> 172.21.15.100:0/2762238392 conn(0x7fa4b40fdfd0 msgr2=0x7fa4b41003f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:23:24.882 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.879+0000 7fa4b952b640 1 -- 172.21.15.100:0/2762238392 shutdown_connections 2024-05-10T11:23:24.882 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.879+0000 7fa4b952b640 1 -- 172.21.15.100:0/2762238392 wait complete. 2024-05-10T11:23:24.883 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.880+0000 7fa4b952b640 1 Processor -- start 2024-05-10T11:23:24.883 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.880+0000 7fa4b952b640 1 -- start start 2024-05-10T11:23:24.883 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.880+0000 7fa4b952b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4b4102a40 0x7fa4b4198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:24.883 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.880+0000 7fa4b952b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4b4103c30 0x7fa4b4199040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:24.883 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.880+0000 7fa4b952b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4b41045f0 0x7fa4b41a0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:24.883 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.880+0000 7fa4b952b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b410e400 con 0x7fa4b4102a40 2024-05-10T11:23:24.883 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.880+0000 7fa4b952b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b410e280 con 0x7fa4b41045f0 2024-05-10T11:23:24.883 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.880+0000 7fa4b952b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b410e580 con 0x7fa4b4103c30 2024-05-10T11:23:24.883 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.880+0000 7fa4b2ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4b4102a40 0x7fa4b4198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:24.883 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.880+0000 7fa4b2ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4b4102a40 0x7fa4b4198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58354/0 (socket says 172.21.15.100:58354) 2024-05-10T11:23:24.883 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.880+0000 7fa4b2ffd640 1 -- 172.21.15.100:0/1787931881 learned_addr learned my addr 172.21.15.100:0/1787931881 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:23:24.883 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.881+0000 7fa4b37fe640 1 --2- 172.21.15.100:0/1787931881 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4b41045f0 0x7fa4b41a0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:24.883 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.881+0000 7fa4b27fc640 1 --2- 172.21.15.100:0/1787931881 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4b4103c30 0x7fa4b4199040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:24.883 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.881+0000 7fa4b2ffd640 1 -- 172.21.15.100:0/1787931881 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4b4103c30 msgr2=0x7fa4b4199040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:24.883 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.881+0000 7fa4b2ffd640 1 --2- 172.21.15.100:0/1787931881 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4b4103c30 0x7fa4b4199040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:24.884 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.881+0000 7fa4b2ffd640 1 -- 172.21.15.100:0/1787931881 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4b41045f0 msgr2=0x7fa4b41a0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:24.884 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.881+0000 7fa4b2ffd640 1 --2- 172.21.15.100:0/1787931881 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4b41045f0 0x7fa4b41a0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:24.884 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.881+0000 7fa4b2ffd640 1 -- 172.21.15.100:0/1787931881 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4b41a06d0 con 0x7fa4b4102a40 2024-05-10T11:23:24.884 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.881+0000 7fa4b37fe640 1 --2- 172.21.15.100:0/1787931881 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4b41045f0 0x7fa4b41a0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:23:24.884 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.881+0000 7fa4b2ffd640 1 --2- 172.21.15.100:0/1787931881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4b4102a40 0x7fa4b4198b00 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fa4a400e9c0 tx=0x7fa4a400edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:23:24.884 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.881+0000 7fa49bfff640 1 -- 172.21.15.100:0/1787931881 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a400ccc0 con 0x7fa4b4102a40 2024-05-10T11:23:24.884 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.881+0000 7fa4b952b640 1 -- 172.21.15.100:0/1787931881 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4b41a09c0 con 0x7fa4b4102a40 2024-05-10T11:23:24.884 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.881+0000 7fa49bfff640 1 -- 172.21.15.100:0/1787931881 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa4a40044b0 con 0x7fa4b4102a40 2024-05-10T11:23:24.884 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.881+0000 7fa49bfff640 1 -- 172.21.15.100:0/1787931881 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a4003e50 con 0x7fa4b4102a40 2024-05-10T11:23:24.884 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.881+0000 7fa4b952b640 1 -- 172.21.15.100:0/1787931881 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4b41a0f30 con 0x7fa4b4102a40 2024-05-10T11:23:24.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.884+0000 7fa4bc24a740 1 Processor -- start 2024-05-10T11:23:24.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.884+0000 7fa4bc24a740 1 -- start start 2024-05-10T11:23:24.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.884+0000 7fa49bfff640 1 -- 172.21.15.100:0/1787931881 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fa4a4004000 con 0x7fa4b4102a40 2024-05-10T11:23:24.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.884+0000 7fa4bc24a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e496a3db0 0x562e496a41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:24.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.884+0000 7fa4bc24a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e496a47a0 0x562e496a92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:24.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.884+0000 7fa4bc24a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e496a9910 0x562e496abda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:24.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.884+0000 7fa4bc24a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x562e496085d0 con 0x562e496a9910 2024-05-10T11:23:24.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.884+0000 7fa4bc24a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562e495c1890 con 0x562e496a3db0 2024-05-10T11:23:24.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.884+0000 7fa4bc24a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x562e49604590 con 0x562e496a47a0 2024-05-10T11:23:24.892 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.885+0000 7fa4b27fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e496a9910 0x562e496abda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:24.892 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.885+0000 7fa4b27fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e496a9910 0x562e496abda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58370/0 (socket says 172.21.15.100:58370) 2024-05-10T11:23:24.892 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.885+0000 7fa4b27fc640 1 -- 172.21.15.100:0/4022996874 learned_addr learned my addr 172.21.15.100:0/4022996874 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:23:24.892 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.885+0000 7fa4b37fe640 1 --2- 172.21.15.100:0/4022996874 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e496a3db0 0x562e496a41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:24.892 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.885+0000 7fa4b2ffd640 1 --2- 172.21.15.100:0/4022996874 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e496a47a0 0x562e496a92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:24.892 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.885+0000 7fa4b27fc640 1 -- 172.21.15.100:0/4022996874 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e496a47a0 msgr2=0x562e496a92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:24.892 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.885+0000 7fa4b27fc640 1 --2- 172.21.15.100:0/4022996874 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e496a47a0 0x562e496a92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:24.892 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.885+0000 7fa4b27fc640 1 -- 172.21.15.100:0/4022996874 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e496a3db0 msgr2=0x562e496a41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:24.892 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.885+0000 7fa4b27fc640 1 --2- 172.21.15.100:0/4022996874 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e496a3db0 0x562e496a41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:24.892 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.885+0000 7fa4b27fc640 1 -- 172.21.15.100:0/4022996874 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e4931a210 con 0x562e496a9910 2024-05-10T11:23:24.892 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.885+0000 7fa4b27fc640 1 --2- 172.21.15.100:0/4022996874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e496a9910 0x562e496abda0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fa4a802a930 tx=0x7fa4a8005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c2cb2e18477856c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:23:24.892 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.885+0000 7fa49bfff640 1 --2- 172.21.15.100:0/1787931881 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa49c0b2a40 0x7fa49c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:24.892 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.885+0000 7fa4997fa640 1 -- 172.21.15.100:0/4022996874 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a8035030 con 0x562e496a9910 2024-05-10T11:23:24.892 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.885+0000 7fa4997fa640 1 -- 172.21.15.100:0/4022996874 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa4a8040070 con 0x562e496a9910 2024-05-10T11:23:24.893 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.885+0000 7fa4997fa640 1 -- 172.21.15.100:0/4022996874 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a8003850 con 0x562e496a9910 2024-05-10T11:23:24.893 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.885+0000 7fa4b37fe640 1 --2- 172.21.15.100:0/1787931881 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa49c0b2a40 0x7fa49c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:24.893 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.885+0000 7fa49bfff640 1 -- 172.21.15.100:0/1787931881 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fa4a40e29e0 con 0x7fa4b4102a40 2024-05-10T11:23:24.893 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.886+0000 7fa4b37fe640 1 --2- 172.21.15.100:0/1787931881 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa49c0b2a40 0x7fa49c0b4f00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x562e496a5000 tx=0x7fa4ac009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:23:24.893 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.886+0000 7fa4bc24a740 1 -- 172.21.15.100:0/4022996874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e496a9910 msgr2=0x562e496abda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:24.893 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.886+0000 7fa4bc24a740 1 --2- 172.21.15.100:0/4022996874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e496a9910 0x562e496abda0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fa4a802a930 tx=0x7fa4a8005de0 comp rx=0 tx=0).stop 2024-05-10T11:23:24.893 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.886+0000 7fa4bc24a740 1 -- 172.21.15.100:0/4022996874 shutdown_connections 2024-05-10T11:23:24.893 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.886+0000 7fa4bc24a740 1 --2- 172.21.15.100:0/4022996874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e496a9910 0x562e496abda0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:24.893 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.886+0000 7fa4bc24a740 1 --2- 172.21.15.100:0/4022996874 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e496a47a0 0x562e496a92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:24.893 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.886+0000 7fa4bc24a740 1 --2- 172.21.15.100:0/4022996874 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e496a3db0 0x562e496a41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:24.893 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.886+0000 7fa4bc24a740 1 -- 172.21.15.100:0/4022996874 >> 172.21.15.100:0/4022996874 conn(0x562e4969f420 msgr2=0x562e4969f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:23:24.893 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.886+0000 7fa4bc24a740 1 -- 172.21.15.100:0/4022996874 shutdown_connections 2024-05-10T11:23:24.893 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.886+0000 7fa4bc24a740 1 -- 172.21.15.100:0/4022996874 wait complete. 2024-05-10T11:23:24.893 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.886+0000 7fa4bc24a740 1 -- start start 2024-05-10T11:23:24.894 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.886+0000 7fa4bc24a740 1 Processor -- start 2024-05-10T11:23:24.894 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.887+0000 7fa4bc24a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e496a3db0 0x562e496adf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:24.894 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.887+0000 7fa4bc24a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e496a47a0 0x562e496ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:24.894 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.887+0000 7fa4bc24a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e496a9910 0x562e496b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:24.894 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.887+0000 7fa4bc24a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x562e496ac860 con 0x562e496a47a0 2024-05-10T11:23:24.894 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.887+0000 7fa4bc24a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562e495c5370 con 0x562e496a3db0 2024-05-10T11:23:24.894 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.887+0000 7fa4bc24a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x562e496ac9e0 con 0x562e496a9910 2024-05-10T11:23:24.894 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.887+0000 7fa4bc24a740 10 client.0 _ll_register_callbacks cb 0x562e49319a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:23:24.894 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.887+0000 7fa4b27fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e496a47a0 0x562e496ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:24.894 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.887+0000 7fa4b2ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e496a3db0 0x562e496adf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:24.894 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.887+0000 7fa4b27fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e496a47a0 0x562e496ae4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58386/0 (socket says 172.21.15.100:58386) 2024-05-10T11:23:24.894 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.887+0000 7fa4b2ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e496a3db0 0x562e496adf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:44948/0 (socket says 172.21.15.100:44948) 2024-05-10T11:23:24.894 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.887+0000 7fa4b27fc640 1 -- 172.21.15.100:0/1839798521 learned_addr learned my addr 172.21.15.100:0/1839798521 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:23:24.895 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.887+0000 7fa4b37fe640 1 --2- 172.21.15.100:0/1839798521 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e496a9910 0x562e496b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:24.895 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.887+0000 7fa4b27fc640 1 -- 172.21.15.100:0/1839798521 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e496a9910 msgr2=0x562e496b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:24.895 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.887+0000 7fa4b27fc640 1 --2- 172.21.15.100:0/1839798521 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e496a9910 0x562e496b54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:24.895 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.887+0000 7fa4b27fc640 1 -- 172.21.15.100:0/1839798521 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e496a3db0 msgr2=0x562e496adf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:24.895 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.887+0000 7fa4b27fc640 1 --2- 172.21.15.100:0/1839798521 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e496a3db0 0x562e496adf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:24.895 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.887+0000 7fa4b27fc640 1 -- 172.21.15.100:0/1839798521 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e493e6620 con 0x562e496a47a0 2024-05-10T11:23:24.895 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.887+0000 7fa4b2ffd640 1 --2- 172.21.15.100:0/1839798521 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e496a3db0 0x562e496adf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:23:24.895 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.887+0000 7fa4b27fc640 1 --2- 172.21.15.100:0/1839798521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e496a47a0 0x562e496ae4b0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8005ab0 tx=0x7fa4a802abc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2d2e90b9e3c0b09 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:23:24.895 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.887+0000 7fa496ff5640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:23:24.895 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.887+0000 7fa496ff5640 1 -- 172.21.15.100:0/1839798521 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a8051060 con 0x562e496a47a0 2024-05-10T11:23:24.895 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.887+0000 7fa496ff5640 1 -- 172.21.15.100:0/1839798521 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fa4a803c440 con 0x562e496a47a0 2024-05-10T11:23:24.895 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.888+0000 7fa4bc24a740 1 -- 172.21.15.100:0/1839798521 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4a8040070 con 0x562e496a47a0 2024-05-10T11:23:24.895 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.888+0000 7fa496ff5640 1 -- 172.21.15.100:0/1839798521 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa4a8058050 con 0x562e496a47a0 2024-05-10T11:23:24.895 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.888+0000 7fa4bc24a740 10 client.16347 fetch_fsmap learned FSMap version 31 2024-05-10T11:23:24.896 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.888+0000 7fa4bc24a740 1 -- 172.21.15.100:0/1839798521 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562e496b84d0 con 0x562e496a47a0 2024-05-10T11:23:24.896 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.888+0000 7fa496ff5640 1 -- 172.21.15.100:0/1839798521 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa4a8040070 con 0x562e496a47a0 2024-05-10T11:23:24.896 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.888+0000 7fa496ff5640 1 -- 172.21.15.100:0/1839798521 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fa4a8044530 con 0x562e496a47a0 2024-05-10T11:23:24.896 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.888+0000 7fa4bc24a740 10 client.16347 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:23:24.896 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.888+0000 7fa4bc24a740 10 client.16347 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:23:24.896 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.888+0000 7fa4bc24a740 20 client.16347 populate_metadata read hostname 'smithi100' 2024-05-10T11:23:24.896 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.888+0000 7fa4bc24a740 1 --2- 172.21.15.100:0/1839798521 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562e496b9820 0x562e496bbc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:24.896 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.888+0000 7fa4bc24a740 4 client.16347 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:23:24.896 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.888+0000 7fa4bc24a740 1 -- 172.21.15.100:0/1839798521 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e496161e0 con 0x562e496b9820 2024-05-10T11:23:24.896 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.888+0000 7fa4b2ffd640 1 --2- 172.21.15.100:0/1839798521 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562e496b9820 0x562e496bbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:24.896 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.889+0000 7fa496ff5640 10 client.16347 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:23:24.896 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.889+0000 7fa4b2ffd640 1 --2- 172.21.15.100:0/1839798521 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562e496b9820 0x562e496bbc10 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4045af703d21a5d server_cookie=2965d2f49262c5f0 in_seq=0 out_seq=0 2024-05-10T11:23:24.896 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.890+0000 7fa496ff5640 1 -- 172.21.15.100:0/1839798521 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562e496161e0 con 0x562e496b9820 2024-05-10T11:23:24.896 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.890+0000 7fa496ff5640 10 client.16347 handle_command_reply: tid=0 2024-05-10T11:23:24.896 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.890+0000 7fa4bc24a740 1 client.16347 shutdown 2024-05-10T11:23:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.890+0000 7fa4bc24a740 1 -- 172.21.15.100:0/1839798521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e496a47a0 msgr2=0x562e496ae4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.890+0000 7fa4bc24a740 1 --2- 172.21.15.100:0/1839798521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e496a47a0 0x562e496ae4b0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8005ab0 tx=0x7fa4a802abc0 comp rx=0 tx=0).stop 2024-05-10T11:23:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.890+0000 7fa4bc24a740 1 -- 172.21.15.100:0/1839798521 shutdown_connections 2024-05-10T11:23:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.890+0000 7fa4bc24a740 1 --2- 172.21.15.100:0/1839798521 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562e496b9820 0x562e496bbc10 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.890+0000 7fa4bc24a740 1 --2- 172.21.15.100:0/1839798521 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e496a9910 0x562e496b54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.890+0000 7fa4bc24a740 1 --2- 172.21.15.100:0/1839798521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e496a47a0 0x562e496ae4b0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.890+0000 7fa4bc24a740 1 --2- 172.21.15.100:0/1839798521 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e496a3db0 0x562e496adf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.890+0000 7fa4bc24a740 1 -- 172.21.15.100:0/1839798521 >> 172.21.15.100:0/1839798521 conn(0x562e4969f420 msgr2=0x562e4969e580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:23:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.890+0000 7fa496ff5640 0 client.16347 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:23:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.890+0000 7fa4bc24a740 1 -- 172.21.15.100:0/1839798521 shutdown_connections 2024-05-10T11:23:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.890+0000 7fa4bc24a740 1 -- 172.21.15.100:0/1839798521 wait complete. 2024-05-10T11:23:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.890+0000 7fa4bc24a740 20 client.16347 trim_cache size 0 max 16384 2024-05-10T11:23:24.913 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.908+0000 7fa4bc24a740 1 Processor -- start 2024-05-10T11:23:24.913 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.908+0000 7fa4bc24a740 1 -- start start 2024-05-10T11:23:24.913 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.908+0000 7fa4bc24a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e496a3db0 0x562e496bb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:24.913 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.908+0000 7fa4bc24a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e496a47a0 0x562e496bbd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:24.913 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.908+0000 7fa4bc24a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e496a9910 0x562e496a23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:24.913 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.908+0000 7fa4bc24a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b41adb90 con 0x562e496a47a0 2024-05-10T11:23:24.913 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.908+0000 7fa4bc24a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b41ab460 con 0x562e496a3db0 2024-05-10T11:23:24.913 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.908+0000 7fa4bc24a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b41a92b0 con 0x562e496a9910 2024-05-10T11:23:24.913 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.908+0000 7fa4b27fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e496a47a0 0x562e496bbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:24.913 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.908+0000 7fa4b27fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e496a47a0 0x562e496bbd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58396/0 (socket says 172.21.15.100:58396) 2024-05-10T11:23:24.913 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.908+0000 7fa4b27fc640 1 -- 172.21.15.100:0/1552501754 learned_addr learned my addr 172.21.15.100:0/1552501754 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:23:24.913 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.908+0000 7fa4b2ffd640 1 --2- 172.21.15.100:0/1552501754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e496a3db0 0x562e496bb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:24.913 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.909+0000 7fa4b37fe640 1 --2- 172.21.15.100:0/1552501754 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e496a9910 0x562e496a23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:24.913 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.909+0000 7fa4b27fc640 1 -- 172.21.15.100:0/1552501754 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e496a9910 msgr2=0x562e496a23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:24.914 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.909+0000 7fa4b27fc640 1 --2- 172.21.15.100:0/1552501754 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e496a9910 0x562e496a23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:24.914 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.909+0000 7fa4b27fc640 1 -- 172.21.15.100:0/1552501754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e496a3db0 msgr2=0x562e496bb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:24.914 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.909+0000 7fa4b27fc640 1 --2- 172.21.15.100:0/1552501754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e496a3db0 0x562e496bb830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:24.914 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.909+0000 7fa4b27fc640 1 -- 172.21.15.100:0/1552501754 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e4969f250 con 0x562e496a47a0 2024-05-10T11:23:24.914 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.909+0000 7fa4b27fc640 1 --2- 172.21.15.100:0/1552501754 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e496a47a0 0x562e496bbd70 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8006010 tx=0x7fa4a8031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae820cbafe1e1f3f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:23:24.914 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.909+0000 7fa498ff9640 1 -- 172.21.15.100:0/1552501754 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a8061070 con 0x562e496a47a0 2024-05-10T11:23:24.914 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.909+0000 7fa498ff9640 1 -- 172.21.15.100:0/1552501754 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa4a8040070 con 0x562e496a47a0 2024-05-10T11:23:24.914 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.909+0000 7fa498ff9640 1 -- 172.21.15.100:0/1552501754 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a8051040 con 0x562e496a47a0 2024-05-10T11:23:24.914 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.910+0000 7fa4bc24a740 1 -- 172.21.15.100:0/1552501754 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e496a47a0 msgr2=0x562e496bbd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:24.914 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.910+0000 7fa4bc24a740 1 --2- 172.21.15.100:0/1552501754 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e496a47a0 0x562e496bbd70 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8006010 tx=0x7fa4a8031400 comp rx=0 tx=0).stop 2024-05-10T11:23:24.914 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.910+0000 7fa4bc24a740 1 -- 172.21.15.100:0/1552501754 shutdown_connections 2024-05-10T11:23:24.914 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.910+0000 7fa4bc24a740 1 --2- 172.21.15.100:0/1552501754 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e496a9910 0x562e496a23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:24.915 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.910+0000 7fa4bc24a740 1 --2- 172.21.15.100:0/1552501754 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e496a47a0 0x562e496bbd70 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:24.915 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.910+0000 7fa4bc24a740 1 --2- 172.21.15.100:0/1552501754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e496a3db0 0x562e496bb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:24.915 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.910+0000 7fa4bc24a740 1 -- 172.21.15.100:0/1552501754 >> 172.21.15.100:0/1552501754 conn(0x562e4969f420 msgr2=0x562e496a5980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:23:24.915 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.910+0000 7fa4bc24a740 1 -- 172.21.15.100:0/1552501754 shutdown_connections 2024-05-10T11:23:24.915 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.910+0000 7fa4bc24a740 1 -- 172.21.15.100:0/1552501754 wait complete. 2024-05-10T11:23:24.915 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.910+0000 7fa4bc24a740 1 -- start start 2024-05-10T11:23:24.915 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.910+0000 7fa4bc24a740 1 Processor -- start 2024-05-10T11:23:24.915 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.910+0000 7fa4bc24a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e496a3db0 0x562e496b20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:24.915 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.911+0000 7fa4bc24a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e496a47a0 0x562e496b2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:24.915 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.911+0000 7fa4bc24a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e496a9910 0x562e496b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:24.915 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.911+0000 7fa4bc24a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b41a9920 con 0x562e496a47a0 2024-05-10T11:23:24.915 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.911+0000 7fa4bc24a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b41aad90 con 0x562e496a3db0 2024-05-10T11:23:24.915 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.911+0000 7fa4bc24a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b41a9ff0 con 0x562e496a9910 2024-05-10T11:23:24.915 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.911+0000 7fa4bc24a740 10 client.0 _ll_register_callbacks cb 0x7fa4ac009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:23:24.916 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.911+0000 7fa4b27fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e496a47a0 0x562e496b2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:24.916 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.911+0000 7fa4b2ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e496a3db0 0x562e496b20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:24.916 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.911+0000 7fa4b37fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e496a9910 0x562e496b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:24.916 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.911+0000 7fa4b2ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e496a3db0 0x562e496b20c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:44962/0 (socket says 172.21.15.100:44962) 2024-05-10T11:23:24.916 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.911+0000 7fa4b27fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e496a47a0 0x562e496b2600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58406/0 (socket says 172.21.15.100:58406) 2024-05-10T11:23:24.916 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.911+0000 7fa4b2ffd640 1 -- 172.21.15.100:0/3710563284 learned_addr learned my addr 172.21.15.100:0/3710563284 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:23:24.916 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.911+0000 7fa4b2ffd640 1 -- 172.21.15.100:0/3710563284 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e496a9910 msgr2=0x562e496b3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:24.916 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.911+0000 7fa4b2ffd640 1 --2- 172.21.15.100:0/3710563284 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e496a9910 0x562e496b3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:24.916 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.911+0000 7fa4b2ffd640 1 -- 172.21.15.100:0/3710563284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e496a47a0 msgr2=0x562e496b2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:24.916 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.911+0000 7fa4b2ffd640 1 --2- 172.21.15.100:0/3710563284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e496a47a0 0x562e496b2600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:24.916 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.911+0000 7fa4b2ffd640 1 -- 172.21.15.100:0/3710563284 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e496b3c00 con 0x562e496a3db0 2024-05-10T11:23:24.916 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.911+0000 7fa4b27fc640 1 --2- 172.21.15.100:0/3710563284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e496a47a0 0x562e496b2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:23:24.916 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.911+0000 7fa4b2ffd640 1 --2- 172.21.15.100:0/3710563284 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e496a3db0 0x562e496b20c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa4a40a6f10 tx=0x7fa4a40e0050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1792bdf4e79fa1a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:23:24.916 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.911+0000 7fa496ff5640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:23:24.917 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.911+0000 7fa496ff5640 1 -- 172.21.15.100:0/3710563284 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a408f070 con 0x562e496a3db0 2024-05-10T11:23:24.917 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.911+0000 7fa496ff5640 1 -- 172.21.15.100:0/3710563284 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fa4a40a8f00 con 0x562e496a3db0 2024-05-10T11:23:24.917 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.911+0000 7fa4bc24a740 1 -- 172.21.15.100:0/3710563284 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4a8040070 con 0x562e496a3db0 2024-05-10T11:23:24.917 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.911+0000 7fa496ff5640 1 -- 172.21.15.100:0/3710563284 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa4a406e050 con 0x562e496a3db0 2024-05-10T11:23:24.917 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.911+0000 7fa4bc24a740 10 client.25817 fetch_fsmap learned FSMap version 31 2024-05-10T11:23:24.917 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.911+0000 7fa4bc24a740 1 -- 172.21.15.100:0/3710563284 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562e49723fe0 con 0x562e496a3db0 2024-05-10T11:23:24.917 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.912+0000 7fa496ff5640 1 -- 172.21.15.100:0/3710563284 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa4a8040070 con 0x562e496a3db0 2024-05-10T11:23:24.917 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.912+0000 7fa496ff5640 1 -- 172.21.15.100:0/3710563284 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fa4a40a8970 con 0x562e496a3db0 2024-05-10T11:23:24.917 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.912+0000 7fa4bc24a740 10 client.25817 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:23:24.917 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.912+0000 7fa4bc24a740 10 client.25817 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:23:24.917 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.912+0000 7fa4bc24a740 20 client.25817 populate_metadata read hostname 'smithi100' 2024-05-10T11:23:24.917 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.912+0000 7fa4bc24a740 1 --2- 172.21.15.100:0/3710563284 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562e497247c0 0x562e49726bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:24.917 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.912+0000 7fa4bc24a740 4 client.25817 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:23:24.917 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.912+0000 7fa4bc24a740 1 -- 172.21.15.100:0/3710563284 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562e49610050 con 0x562e497247c0 2024-05-10T11:23:24.917 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.912+0000 7fa4b27fc640 1 --2- 172.21.15.100:0/3710563284 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562e497247c0 0x562e49726bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:24.918 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.912+0000 7fa4b27fc640 1 --2- 172.21.15.100:0/3710563284 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562e497247c0 0x562e49726bb0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab24665b98e925b0 server_cookie=3dd309cadd047865 in_seq=0 out_seq=0 2024-05-10T11:23:24.918 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.912+0000 7fa496ff5640 10 client.25817 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:23:24.932 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.932+0000 7fa496ff5640 1 -- 172.21.15.100:0/3710563284 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562e49610050 con 0x562e497247c0 2024-05-10T11:23:24.932 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.932+0000 7fa496ff5640 10 client.25817 handle_command_reply: tid=0 2024-05-10T11:23:24.932 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.932+0000 7fa4bc24a740 1 client.25817 shutdown 2024-05-10T11:23:24.932 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.932+0000 7fa4bc24a740 1 -- 172.21.15.100:0/3710563284 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e496a3db0 msgr2=0x562e496b20c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.932+0000 7fa4bc24a740 1 --2- 172.21.15.100:0/3710563284 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e496a3db0 0x562e496b20c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa4a40a6f10 tx=0x7fa4a40e0050 comp rx=0 tx=0).stop 2024-05-10T11:23:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.932+0000 7fa4bc24a740 1 -- 172.21.15.100:0/3710563284 shutdown_connections 2024-05-10T11:23:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.932+0000 7fa4bc24a740 1 --2- 172.21.15.100:0/3710563284 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562e497247c0 0x562e49726bb0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.932+0000 7fa4bc24a740 1 --2- 172.21.15.100:0/3710563284 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e496a9910 0x562e496b3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.932+0000 7fa4bc24a740 1 --2- 172.21.15.100:0/3710563284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e496a47a0 0x562e496b2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.932+0000 7fa4bc24a740 1 --2- 172.21.15.100:0/3710563284 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e496a3db0 0x562e496b20c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.932+0000 7fa496ff5640 0 client.25817 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:23:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.932+0000 7fa4bc24a740 1 -- 172.21.15.100:0/3710563284 >> 172.21.15.100:0/3710563284 conn(0x562e4969f420 msgr2=0x562e496adf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:23:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.932+0000 7fa4bc24a740 1 -- 172.21.15.100:0/3710563284 shutdown_connections 2024-05-10T11:23:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.932+0000 7fa4bc24a740 1 -- 172.21.15.100:0/3710563284 wait complete. 2024-05-10T11:23:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.932+0000 7fa4bc24a740 20 client.25817 trim_cache size 0 max 16384 2024-05-10T11:23:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:23:24.934 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:23:24.934 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:23:24.934 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:23:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.933+0000 7fa4b952b640 1 -- 172.21.15.100:0/1787931881 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa49c0b2a40 msgr2=0x7fa49c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.933+0000 7fa4b952b640 1 --2- 172.21.15.100:0/1787931881 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa49c0b2a40 0x7fa49c0b4f00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x562e496a5000 tx=0x7fa4ac009500 comp rx=0 tx=0).stop 2024-05-10T11:23:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.933+0000 7fa4b952b640 1 -- 172.21.15.100:0/1787931881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4b4102a40 msgr2=0x7fa4b4198b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.933+0000 7fa4b952b640 1 --2- 172.21.15.100:0/1787931881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4b4102a40 0x7fa4b4198b00 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fa4a400e9c0 tx=0x7fa4a400edc0 comp rx=0 tx=0).stop 2024-05-10T11:23:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.933+0000 7fa4b952b640 1 -- 172.21.15.100:0/1787931881 shutdown_connections 2024-05-10T11:23:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.933+0000 7fa4b952b640 1 --2- 172.21.15.100:0/1787931881 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa49c0b2a40 0x7fa49c0b4f00 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.933+0000 7fa4b952b640 1 --2- 172.21.15.100:0/1787931881 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4b41045f0 0x7fa4b41a0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.933+0000 7fa4b952b640 1 --2- 172.21.15.100:0/1787931881 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4b4103c30 0x7fa4b4199040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.933+0000 7fa4b952b640 1 --2- 172.21.15.100:0/1787931881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4b4102a40 0x7fa4b4198b00 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.933+0000 7fa4b952b640 1 -- 172.21.15.100:0/1787931881 >> 172.21.15.100:0/1787931881 conn(0x7fa4b40fdfd0 msgr2=0x7fa4b4107000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:23:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.933+0000 7fa4b952b640 1 -- 172.21.15.100:0/1787931881 shutdown_connections 2024-05-10T11:23:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:24.933+0000 7fa4b952b640 1 -- 172.21.15.100:0/1787931881 wait complete. 2024-05-10T11:23:24.970 INFO:tasks.workunit.client.0.smithi100.stdout: 1 104946 2.74 MB/sec execute 79 sec latency 20.350 ms 2024-05-10T11:23:25.970 INFO:tasks.workunit.client.0.smithi100.stdout: 1 105487 2.74 MB/sec execute 80 sec latency 15.804 ms 2024-05-10T11:23:26.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:25 smithi178 ceph-mon[33359]: pgmap v571: 129 pgs: 129 active+clean; 56 MiB data, 998 MiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 1.4 MiB/s wr, 145 op/s 2024-05-10T11:23:26.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:25 smithi188 ceph-mon[34075]: pgmap v571: 129 pgs: 129 active+clean; 56 MiB data, 998 MiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 1.4 MiB/s wr, 145 op/s 2024-05-10T11:23:26.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:25 smithi100 ceph-mon[30296]: pgmap v571: 129 pgs: 129 active+clean; 56 MiB data, 998 MiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 1.4 MiB/s wr, 145 op/s 2024-05-10T11:23:26.971 INFO:tasks.workunit.client.0.smithi100.stdout: 1 106223 2.78 MB/sec execute 81 sec latency 17.421 ms 2024-05-10T11:23:27.223 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T11:23:27.489 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:23:27.970 INFO:tasks.workunit.client.0.smithi100.stdout: 1 106699 2.80 MB/sec execute 82 sec latency 13.485 ms 2024-05-10T11:23:28.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:27 smithi178 ceph-mon[33359]: pgmap v572: 129 pgs: 129 active+clean; 56 MiB data, 1003 MiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 1.4 MiB/s wr, 165 op/s 2024-05-10T11:23:28.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:27 smithi188 ceph-mon[34075]: pgmap v572: 129 pgs: 129 active+clean; 56 MiB data, 1003 MiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 1.4 MiB/s wr, 165 op/s 2024-05-10T11:23:28.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:27 smithi100 ceph-mon[30296]: pgmap v572: 129 pgs: 129 active+clean; 56 MiB data, 1003 MiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 1.4 MiB/s wr, 165 op/s 2024-05-10T11:23:28.970 INFO:tasks.workunit.client.0.smithi100.stdout: 1 106977 2.77 MB/sec execute 83 sec latency 11.380 ms 2024-05-10T11:23:29.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.800+0000 7f0dce830700 1 -- 172.21.15.100:0/60475864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0dc8108410 msgr2=0x7f0dc81087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:29.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.800+0000 7f0dce830700 1 --2- 172.21.15.100:0/60475864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0dc8108410 0x7f0dc81087f0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f0db8009990 tx=0x7f0db8009ca0 comp rx=0 tx=0).stop 2024-05-10T11:23:29.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.800+0000 7f0dce830700 1 -- 172.21.15.100:0/60475864 shutdown_connections 2024-05-10T11:23:29.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.800+0000 7f0dce830700 1 --2- 172.21.15.100:0/60475864 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0dc8102d60 0x7f0dc810f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:29.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.800+0000 7f0dce830700 1 --2- 172.21.15.100:0/60475864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0dc8108410 0x7f0dc81087f0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:29.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.800+0000 7f0dce830700 1 --2- 172.21.15.100:0/60475864 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0dc81023a0 0x7f0dc8102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:29.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.800+0000 7f0dce830700 1 -- 172.21.15.100:0/60475864 >> 172.21.15.100:0/60475864 conn(0x7f0dc8079350 msgr2=0x7f0dc8079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:23:29.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.800+0000 7f0dce830700 1 -- 172.21.15.100:0/60475864 shutdown_connections 2024-05-10T11:23:29.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.800+0000 7f0dce830700 1 -- 172.21.15.100:0/60475864 wait complete. 2024-05-10T11:23:29.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.801+0000 7f0dce830700 1 Processor -- start 2024-05-10T11:23:29.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.801+0000 7f0dce830700 1 -- start start 2024-05-10T11:23:29.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.801+0000 7f0dce830700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0dc81023a0 0x7f0dc8199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:29.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.801+0000 7f0dce830700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0dc8102d60 0x7f0dc819a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:29.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.801+0000 7f0dce830700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0dc8108410 0x7f0dc819e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:29.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.801+0000 7f0dce830700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc81123e0 con 0x7f0dc8102d60 2024-05-10T11:23:29.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.801+0000 7f0dce830700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc8112260 con 0x7f0dc81023a0 2024-05-10T11:23:29.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.801+0000 7f0dce830700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc8112560 con 0x7f0dc8108410 2024-05-10T11:23:29.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.801+0000 7f0dc77fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0dc8102d60 0x7f0dc819a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:29.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.801+0000 7f0dc7fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0dc81023a0 0x7f0dc8199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:29.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.801+0000 7f0dc77fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0dc8102d60 0x7f0dc819a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59114/0 (socket says 172.21.15.100:59114) 2024-05-10T11:23:29.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.801+0000 7f0dc77fe700 1 -- 172.21.15.100:0/4085147646 learned_addr learned my addr 172.21.15.100:0/4085147646 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:23:29.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.801+0000 7f0dccdcd700 1 --2- 172.21.15.100:0/4085147646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0dc8108410 0x7f0dc819e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:29.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.802+0000 7f0dc7fff700 1 -- 172.21.15.100:0/4085147646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0dc8108410 msgr2=0x7f0dc819e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:29.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.802+0000 7f0dc7fff700 1 --2- 172.21.15.100:0/4085147646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0dc8108410 0x7f0dc819e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:29.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.802+0000 7f0dc7fff700 1 -- 172.21.15.100:0/4085147646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0dc8102d60 msgr2=0x7f0dc819a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:29.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.802+0000 7f0dc7fff700 1 --2- 172.21.15.100:0/4085147646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0dc8102d60 0x7f0dc819a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:29.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.802+0000 7f0dc7fff700 1 -- 172.21.15.100:0/4085147646 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dc819eac0 con 0x7f0dc81023a0 2024-05-10T11:23:29.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.802+0000 7f0dc77fe700 1 --2- 172.21.15.100:0/4085147646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0dc8102d60 0x7f0dc819a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:23:29.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.802+0000 7f0dccdcd700 1 --2- 172.21.15.100:0/4085147646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0dc8108410 0x7f0dc819e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:23:29.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.802+0000 7f0dc7fff700 1 --2- 172.21.15.100:0/4085147646 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0dc81023a0 0x7f0dc8199ac0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0db8000c00 tx=0x7f0db80047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:23:29.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.802+0000 7f0dc57fa700 1 -- 172.21.15.100:0/4085147646 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db8026070 con 0x7f0dc81023a0 2024-05-10T11:23:29.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.802+0000 7f0dc57fa700 1 -- 172.21.15.100:0/4085147646 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0db8021410 con 0x7f0dc81023a0 2024-05-10T11:23:29.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.802+0000 7f0dce830700 1 -- 172.21.15.100:0/4085147646 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dc819ed50 con 0x7f0dc81023a0 2024-05-10T11:23:29.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.802+0000 7f0dc57fa700 1 -- 172.21.15.100:0/4085147646 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db8017650 con 0x7f0dc81023a0 2024-05-10T11:23:29.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.802+0000 7f0dce830700 1 -- 172.21.15.100:0/4085147646 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dc81a6670 con 0x7f0dc81023a0 2024-05-10T11:23:29.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.804+0000 7f0dc57fa700 1 -- 172.21.15.100:0/4085147646 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f0db8029070 con 0x7f0dc81023a0 2024-05-10T11:23:29.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.805+0000 7f0dc57fa700 1 --2- 172.21.15.100:0/4085147646 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0db00b2880 0x7f0db00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:29.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.806+0000 7f0dc77fe700 1 --2- 172.21.15.100:0/4085147646 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0db00b2880 0x7f0db00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:29.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.806+0000 7f0dc57fa700 1 -- 172.21.15.100:0/4085147646 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f0db80e3340 con 0x7f0dc81023a0 2024-05-10T11:23:29.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.806+0000 7f0dd90c3b80 1 Processor -- start 2024-05-10T11:23:29.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.806+0000 7f0dc77fe700 1 --2- 172.21.15.100:0/4085147646 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0db00b2880 0x7f0db00b4d40 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8102350 tx=0x7f0dbc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:23:29.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.806+0000 7f0dd90c3b80 1 -- start start 2024-05-10T11:23:29.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.806+0000 7f0dd90c3b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560998ec4590 0x560998ec4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:29.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.806+0000 7f0dd90c3b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560998ec4eb0 0x560998ebec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:29.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.806+0000 7f0dd90c3b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560998ebf1b0 0x560998ebf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:29.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.806+0000 7f0dd90c3b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x560998e0c940 con 0x560998ec4590 2024-05-10T11:23:29.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.806+0000 7f0dd90c3b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560998d71da0 con 0x560998ec4eb0 2024-05-10T11:23:29.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.806+0000 7f0dd90c3b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x560998e08900 con 0x560998ebf1b0 2024-05-10T11:23:29.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.806+0000 7f0dc7fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560998ec4590 0x560998ec4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:29.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.806+0000 7f0dc7fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560998ec4590 0x560998ec4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59118/0 (socket says 172.21.15.100:59118) 2024-05-10T11:23:29.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.806+0000 7f0dc7fff700 1 -- 172.21.15.100:0/4087407660 learned_addr learned my addr 172.21.15.100:0/4087407660 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:23:29.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.806+0000 7f0dccdcd700 1 --2- 172.21.15.100:0/4087407660 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560998ebf1b0 0x560998ebf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:29.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.807+0000 7f0dc77fe700 1 --2- 172.21.15.100:0/4087407660 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560998ec4eb0 0x560998ebec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:29.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.807+0000 7f0dc7fff700 1 -- 172.21.15.100:0/4087407660 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560998ebf1b0 msgr2=0x560998ebf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:29.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.807+0000 7f0dc7fff700 1 --2- 172.21.15.100:0/4087407660 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560998ebf1b0 0x560998ebf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:29.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.807+0000 7f0dc7fff700 1 -- 172.21.15.100:0/4087407660 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560998ec4eb0 msgr2=0x560998ebec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:29.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.807+0000 7f0dc7fff700 1 --2- 172.21.15.100:0/4087407660 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560998ec4eb0 0x560998ebec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:29.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.807+0000 7f0dc7fff700 1 -- 172.21.15.100:0/4087407660 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560998d598d0 con 0x560998ec4590 2024-05-10T11:23:29.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.807+0000 7f0dc7fff700 1 --2- 172.21.15.100:0/4087407660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560998ec4590 0x560998ec4970 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f0db800b2e0 tx=0x7f0db80a9da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16ce5fc232a88ec5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:23:29.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.807+0000 7f0dae7fc700 1 -- 172.21.15.100:0/4087407660 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db80aa630 con 0x560998ec4590 2024-05-10T11:23:29.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.807+0000 7f0dae7fc700 1 -- 172.21.15.100:0/4087407660 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0db80ae9e0 con 0x560998ec4590 2024-05-10T11:23:29.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.807+0000 7f0dae7fc700 1 -- 172.21.15.100:0/4087407660 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db80aa890 con 0x560998ec4590 2024-05-10T11:23:29.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.808+0000 7f0dd90c3b80 1 -- 172.21.15.100:0/4087407660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560998ec4590 msgr2=0x560998ec4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:29.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.808+0000 7f0dd90c3b80 1 --2- 172.21.15.100:0/4087407660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560998ec4590 0x560998ec4970 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f0db800b2e0 tx=0x7f0db80a9da0 comp rx=0 tx=0).stop 2024-05-10T11:23:29.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.808+0000 7f0dd90c3b80 1 -- 172.21.15.100:0/4087407660 shutdown_connections 2024-05-10T11:23:29.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.808+0000 7f0dd90c3b80 1 --2- 172.21.15.100:0/4087407660 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560998ebf1b0 0x560998ebf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:29.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.808+0000 7f0dd90c3b80 1 --2- 172.21.15.100:0/4087407660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560998ec4590 0x560998ec4970 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:29.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.808+0000 7f0dd90c3b80 1 --2- 172.21.15.100:0/4087407660 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560998ec4eb0 0x560998ebec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:29.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.808+0000 7f0dd90c3b80 1 -- 172.21.15.100:0/4087407660 >> 172.21.15.100:0/4087407660 conn(0x560998de6b70 msgr2=0x560998eb52a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:23:29.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.808+0000 7f0dd90c3b80 1 -- 172.21.15.100:0/4087407660 shutdown_connections 2024-05-10T11:23:29.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.808+0000 7f0dd90c3b80 1 -- 172.21.15.100:0/4087407660 wait complete. 2024-05-10T11:23:29.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.808+0000 7f0dd90c3b80 1 -- start start 2024-05-10T11:23:29.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.809+0000 7f0dd90c3b80 1 Processor -- start 2024-05-10T11:23:29.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.809+0000 7f0dd90c3b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560998ebf1b0 0x560998ecac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:29.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.809+0000 7f0dd90c3b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560998ec4590 0x560998ec7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:29.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.809+0000 7f0dd90c3b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560998ec4eb0 0x560998ec8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:29.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.809+0000 7f0dd90c3b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x560998ecb6a0 con 0x560998ec4eb0 2024-05-10T11:23:29.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.809+0000 7f0dd90c3b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560998dc5ed0 con 0x560998ec4590 2024-05-10T11:23:29.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.809+0000 7f0dd90c3b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x560998ecb820 con 0x560998ebf1b0 2024-05-10T11:23:29.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.809+0000 7f0dd90c3b80 10 client.0 _ll_register_callbacks cb 0x560998da05b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:23:29.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.809+0000 7f0dc7fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560998ebf1b0 0x560998ecac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:29.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.809+0000 7f0dc7fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560998ebf1b0 0x560998ecac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:57904/0 (socket says 172.21.15.100:57904) 2024-05-10T11:23:29.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.809+0000 7f0dc7fff700 1 -- 172.21.15.100:0/1165368504 learned_addr learned my addr 172.21.15.100:0/1165368504 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:23:29.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.809+0000 7f0dccdcd700 1 --2- 172.21.15.100:0/1165368504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560998ec4eb0 0x560998ec8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:29.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.809+0000 7f0dc77fe700 1 --2- 172.21.15.100:0/1165368504 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560998ec4590 0x560998ec7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:29.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.809+0000 7f0dc7fff700 1 -- 172.21.15.100:0/1165368504 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560998ec4590 msgr2=0x560998ec7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:29.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.809+0000 7f0dc7fff700 1 --2- 172.21.15.100:0/1165368504 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560998ec4590 0x560998ec7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:29.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.809+0000 7f0dc7fff700 1 -- 172.21.15.100:0/1165368504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560998ec4eb0 msgr2=0x560998ec8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:29.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.809+0000 7f0dc7fff700 1 --2- 172.21.15.100:0/1165368504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560998ec4eb0 0x560998ec8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:29.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.809+0000 7f0dc7fff700 1 -- 172.21.15.100:0/1165368504 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560998dc3760 con 0x560998ebf1b0 2024-05-10T11:23:29.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.810+0000 7f0dc7fff700 1 --2- 172.21.15.100:0/1165368504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560998ebf1b0 0x560998ecac30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0db80cd070 tx=0x7f0db8092340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd0b55c932305bf9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:23:29.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.810+0000 7f0da77fe700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:23:29.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.810+0000 7f0da77fe700 1 -- 172.21.15.100:0/1165368504 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db80b3400 con 0x560998ebf1b0 2024-05-10T11:23:29.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.810+0000 7f0dd90c3b80 1 -- 172.21.15.100:0/1165368504 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0db80ae9e0 con 0x560998ebf1b0 2024-05-10T11:23:29.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.810+0000 7f0da77fe700 1 -- 172.21.15.100:0/1165368504 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f0db80aaaa0 con 0x560998ebf1b0 2024-05-10T11:23:29.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.810+0000 7f0da77fe700 1 -- 172.21.15.100:0/1165368504 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0db80aa630 con 0x560998ebf1b0 2024-05-10T11:23:29.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.810+0000 7f0dd90c3b80 10 client.25861 fetch_fsmap learned FSMap version 31 2024-05-10T11:23:29.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.810+0000 7f0dd90c3b80 1 -- 172.21.15.100:0/1165368504 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560998ecb6a0 con 0x560998ebf1b0 2024-05-10T11:23:29.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.810+0000 7f0da77fe700 1 -- 172.21.15.100:0/1165368504 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0db80ae9e0 con 0x560998ebf1b0 2024-05-10T11:23:29.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.810+0000 7f0da77fe700 1 -- 172.21.15.100:0/1165368504 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f0db8090ba0 con 0x560998ebf1b0 2024-05-10T11:23:29.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.810+0000 7f0dd90c3b80 10 client.25861 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:23:29.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.811+0000 7f0dd90c3b80 10 client.25861 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:23:29.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.811+0000 7f0dd90c3b80 20 client.25861 populate_metadata read hostname 'smithi100' 2024-05-10T11:23:29.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.811+0000 7f0dd90c3b80 1 --2- 172.21.15.100:0/1165368504 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x560998ed8270 0x560998eda660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:29.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.811+0000 7f0dd90c3b80 4 client.25861 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:23:29.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.811+0000 7f0dd90c3b80 1 -- 172.21.15.100:0/1165368504 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560998e1a320 con 0x560998ed8270 2024-05-10T11:23:29.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.811+0000 7f0dc77fe700 1 --2- 172.21.15.100:0/1165368504 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x560998ed8270 0x560998eda660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:29.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.811+0000 7f0dc77fe700 1 --2- 172.21.15.100:0/1165368504 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x560998ed8270 0x560998eda660 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89169ecbb3ce617c server_cookie=3227ee376e71f94a in_seq=0 out_seq=0 2024-05-10T11:23:29.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.811+0000 7f0da77fe700 10 client.25861 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:23:29.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.812+0000 7f0da77fe700 1 -- 172.21.15.100:0/1165368504 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560998e1a320 con 0x560998ed8270 2024-05-10T11:23:29.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.812+0000 7f0da77fe700 10 client.25861 handle_command_reply: tid=0 2024-05-10T11:23:29.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.812+0000 7f0dd90c3b80 1 client.25861 shutdown 2024-05-10T11:23:29.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.812+0000 7f0dd90c3b80 1 -- 172.21.15.100:0/1165368504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560998ebf1b0 msgr2=0x560998ecac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:29.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.812+0000 7f0dd90c3b80 1 --2- 172.21.15.100:0/1165368504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560998ebf1b0 0x560998ecac30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0db80cd070 tx=0x7f0db8092340 comp rx=0 tx=0).stop 2024-05-10T11:23:29.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.812+0000 7f0dd90c3b80 1 -- 172.21.15.100:0/1165368504 shutdown_connections 2024-05-10T11:23:29.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.812+0000 7f0dd90c3b80 1 --2- 172.21.15.100:0/1165368504 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x560998ed8270 0x560998eda660 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:29.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.813+0000 7f0dd90c3b80 1 --2- 172.21.15.100:0/1165368504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560998ec4eb0 0x560998ec8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:29.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.813+0000 7f0da77fe700 0 client.25861 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:23:29.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.813+0000 7f0dd90c3b80 1 --2- 172.21.15.100:0/1165368504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560998ebf1b0 0x560998ecac30 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:29.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.813+0000 7f0dd90c3b80 1 --2- 172.21.15.100:0/1165368504 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560998ec4590 0x560998ec7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:29.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.813+0000 7f0dd90c3b80 1 -- 172.21.15.100:0/1165368504 >> 172.21.15.100:0/1165368504 conn(0x560998de6b70 msgr2=0x560998eafb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:23:29.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.813+0000 7f0dd90c3b80 1 -- 172.21.15.100:0/1165368504 shutdown_connections 2024-05-10T11:23:29.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.813+0000 7f0dd90c3b80 1 -- 172.21.15.100:0/1165368504 wait complete. 2024-05-10T11:23:29.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.813+0000 7f0dd90c3b80 20 client.25861 trim_cache size 0 max 16384 2024-05-10T11:23:29.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.833+0000 7f0dd90c3b80 1 Processor -- start 2024-05-10T11:23:29.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.833+0000 7f0dd90c3b80 1 -- start start 2024-05-10T11:23:29.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.834+0000 7f0dd90c3b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560998ebf1b0 0x560998eba1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:29.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.834+0000 7f0dd90c3b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560998ec4590 0x560998ed46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:29.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.834+0000 7f0dd90c3b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560998ec4eb0 0x560998eafc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:29.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.834+0000 7f0dd90c3b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc81af680 con 0x560998ec4590 2024-05-10T11:23:29.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.834+0000 7f0dd90c3b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc81acf10 con 0x560998ebf1b0 2024-05-10T11:23:29.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.834+0000 7f0dd90c3b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc81aad60 con 0x560998ec4eb0 2024-05-10T11:23:29.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.834+0000 7f0dc77fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560998ec4590 0x560998ed46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:29.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.834+0000 7f0dc77fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560998ec4590 0x560998ed46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59144/0 (socket says 172.21.15.100:59144) 2024-05-10T11:23:29.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.834+0000 7f0dc77fe700 1 -- 172.21.15.100:0/1279316779 learned_addr learned my addr 172.21.15.100:0/1279316779 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:23:29.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.834+0000 7f0dc7fff700 1 --2- 172.21.15.100:0/1279316779 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560998ebf1b0 0x560998eba1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:29.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.834+0000 7f0dccdcd700 1 --2- 172.21.15.100:0/1279316779 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560998ec4eb0 0x560998eafc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:29.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.834+0000 7f0dc77fe700 1 -- 172.21.15.100:0/1279316779 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560998ec4eb0 msgr2=0x560998eafc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:29.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.834+0000 7f0dc77fe700 1 --2- 172.21.15.100:0/1279316779 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560998ec4eb0 0x560998eafc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:29.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.834+0000 7f0dc77fe700 1 -- 172.21.15.100:0/1279316779 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560998ebf1b0 msgr2=0x560998eba1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:29.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.834+0000 7f0dc77fe700 1 --2- 172.21.15.100:0/1279316779 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560998ebf1b0 0x560998eba1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:29.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.834+0000 7f0dc77fe700 1 -- 172.21.15.100:0/1279316779 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560998eb6070 con 0x560998ec4590 2024-05-10T11:23:29.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.834+0000 7f0dc77fe700 1 --2- 172.21.15.100:0/1279316779 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560998ec4590 0x560998ed46f0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x560998eb6d60 tx=0x7f0dbc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c642583b42570de8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:23:29.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.835+0000 7f0dadffb700 1 -- 172.21.15.100:0/1279316779 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc004830 con 0x560998ec4590 2024-05-10T11:23:29.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.835+0000 7f0dadffb700 1 -- 172.21.15.100:0/1279316779 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0dbc005020 con 0x560998ec4590 2024-05-10T11:23:29.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.835+0000 7f0dadffb700 1 -- 172.21.15.100:0/1279316779 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc004e60 con 0x560998ec4590 2024-05-10T11:23:29.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.835+0000 7f0dd90c3b80 1 -- 172.21.15.100:0/1279316779 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560998ec4590 msgr2=0x560998ed46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:29.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.835+0000 7f0dd90c3b80 1 --2- 172.21.15.100:0/1279316779 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560998ec4590 0x560998ed46f0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x560998eb6d60 tx=0x7f0dbc014000 comp rx=0 tx=0).stop 2024-05-10T11:23:29.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.835+0000 7f0dd90c3b80 1 -- 172.21.15.100:0/1279316779 shutdown_connections 2024-05-10T11:23:29.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.835+0000 7f0dd90c3b80 1 --2- 172.21.15.100:0/1279316779 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560998ec4eb0 0x560998eafc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:29.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.835+0000 7f0dd90c3b80 1 --2- 172.21.15.100:0/1279316779 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560998ebf1b0 0x560998eba1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:29.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.835+0000 7f0dd90c3b80 1 --2- 172.21.15.100:0/1279316779 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560998ec4590 0x560998ed46f0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:29.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.835+0000 7f0dd90c3b80 1 -- 172.21.15.100:0/1279316779 >> 172.21.15.100:0/1279316779 conn(0x560998de6b70 msgr2=0x560998ec7060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:23:29.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.836+0000 7f0dd90c3b80 1 -- 172.21.15.100:0/1279316779 shutdown_connections 2024-05-10T11:23:29.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.836+0000 7f0dd90c3b80 1 -- 172.21.15.100:0/1279316779 wait complete. 2024-05-10T11:23:29.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.836+0000 7f0dd90c3b80 1 -- start start 2024-05-10T11:23:29.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.836+0000 7f0dd90c3b80 1 Processor -- start 2024-05-10T11:23:29.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.836+0000 7f0dd90c3b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560998ebf1b0 0x560998ed0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:29.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.836+0000 7f0dd90c3b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560998ec4590 0x560998ed0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:29.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.836+0000 7f0dd90c3b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560998ec4eb0 0x560998ecba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:29.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.836+0000 7f0dd90c3b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc81ab3d0 con 0x560998ec4590 2024-05-10T11:23:29.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.836+0000 7f0dd90c3b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc81ac840 con 0x560998ebf1b0 2024-05-10T11:23:29.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.836+0000 7f0dd90c3b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc81abaa0 con 0x560998ec4eb0 2024-05-10T11:23:29.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.836+0000 7f0dd90c3b80 10 client.0 _ll_register_callbacks cb 0x560998da05b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:23:29.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.836+0000 7f0dc77fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560998ec4590 0x560998ed0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:29.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.837+0000 7f0dc77fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560998ec4590 0x560998ed0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59150/0 (socket says 172.21.15.100:59150) 2024-05-10T11:23:29.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.837+0000 7f0dc77fe700 1 -- 172.21.15.100:0/3070373121 learned_addr learned my addr 172.21.15.100:0/3070373121 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:23:29.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.837+0000 7f0dccdcd700 1 --2- 172.21.15.100:0/3070373121 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560998ec4eb0 0x560998ecba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:29.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.837+0000 7f0dc7fff700 1 --2- 172.21.15.100:0/3070373121 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560998ebf1b0 0x560998ed0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:29.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.837+0000 7f0dc77fe700 1 -- 172.21.15.100:0/3070373121 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560998ec4eb0 msgr2=0x560998ecba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:29.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.837+0000 7f0dc77fe700 1 --2- 172.21.15.100:0/3070373121 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560998ec4eb0 0x560998ecba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:29.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.837+0000 7f0dc77fe700 1 -- 172.21.15.100:0/3070373121 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560998ebf1b0 msgr2=0x560998ed0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:29.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.837+0000 7f0dc77fe700 1 --2- 172.21.15.100:0/3070373121 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560998ebf1b0 0x560998ed0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:29.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.837+0000 7f0dc77fe700 1 -- 172.21.15.100:0/3070373121 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560998ecc040 con 0x560998ec4590 2024-05-10T11:23:29.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.837+0000 7f0dc7fff700 1 --2- 172.21.15.100:0/3070373121 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560998ebf1b0 0x560998ed0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:23:29.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.837+0000 7f0dc77fe700 1 --2- 172.21.15.100:0/3070373121 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560998ec4590 0x560998ed0fc0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x560998ed5f40 tx=0x7f0dbc009490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfe493422f710edf server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:23:29.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.837+0000 7f0da77fe700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:23:29.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.837+0000 7f0da77fe700 1 -- 172.21.15.100:0/3070373121 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc03b060 con 0x560998ec4590 2024-05-10T11:23:29.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.837+0000 7f0da77fe700 1 -- 172.21.15.100:0/3070373121 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f0dbc004750 con 0x560998ec4590 2024-05-10T11:23:29.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.837+0000 7f0dd90c3b80 1 -- 172.21.15.100:0/3070373121 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0dbc005020 con 0x560998ec4590 2024-05-10T11:23:29.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.838+0000 7f0da77fe700 1 -- 172.21.15.100:0/3070373121 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0dbc042050 con 0x560998ec4590 2024-05-10T11:23:29.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.838+0000 7f0dd90c3b80 10 client.16371 fetch_fsmap learned FSMap version 31 2024-05-10T11:23:29.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.838+0000 7f0dd90c3b80 1 -- 172.21.15.100:0/3070373121 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560998f43240 con 0x560998ec4590 2024-05-10T11:23:29.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.838+0000 7f0da77fe700 1 -- 172.21.15.100:0/3070373121 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0dbc005020 con 0x560998ec4590 2024-05-10T11:23:29.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.838+0000 7f0da77fe700 1 -- 172.21.15.100:0/3070373121 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f0dbc02c5e0 con 0x560998ec4590 2024-05-10T11:23:29.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.838+0000 7f0dd90c3b80 10 client.16371 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:23:29.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.838+0000 7f0dd90c3b80 10 client.16371 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:23:29.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.838+0000 7f0dd90c3b80 20 client.16371 populate_metadata read hostname 'smithi100' 2024-05-10T11:23:29.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.838+0000 7f0dd90c3b80 1 --2- 172.21.15.100:0/3070373121 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x560998f43920 0x560998f45d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:29.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.838+0000 7f0dd90c3b80 4 client.16371 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T11:23:29.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.838+0000 7f0dd90c3b80 1 -- 172.21.15.100:0/3070373121 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560998e143c0 con 0x560998f43920 2024-05-10T11:23:29.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.838+0000 7f0dc7fff700 1 --2- 172.21.15.100:0/3070373121 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x560998f43920 0x560998f45d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:29.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.839+0000 7f0da77fe700 10 client.16371 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:23:29.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.839+0000 7f0dc7fff700 1 --2- 172.21.15.100:0/3070373121 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x560998f43920 0x560998f45d40 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=612a336f2e8656ff server_cookie=f91a8de9184f1201 in_seq=0 out_seq=0 2024-05-10T11:23:29.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.839+0000 7f0da77fe700 1 -- 172.21.15.100:0/3070373121 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560998e143c0 con 0x560998f43920 2024-05-10T11:23:29.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.839+0000 7f0da77fe700 10 client.16371 handle_command_reply: tid=0 2024-05-10T11:23:29.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.839+0000 7f0dd90c3b80 1 client.16371 shutdown 2024-05-10T11:23:29.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.839+0000 7f0dd90c3b80 1 -- 172.21.15.100:0/3070373121 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560998ec4590 msgr2=0x560998ed0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:29.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.839+0000 7f0dd90c3b80 1 --2- 172.21.15.100:0/3070373121 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560998ec4590 0x560998ed0fc0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x560998ed5f40 tx=0x7f0dbc009490 comp rx=0 tx=0).stop 2024-05-10T11:23:29.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.839+0000 7f0dd90c3b80 1 -- 172.21.15.100:0/3070373121 shutdown_connections 2024-05-10T11:23:29.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.839+0000 7f0dd90c3b80 1 --2- 172.21.15.100:0/3070373121 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x560998f43920 0x560998f45d40 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:29.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.839+0000 7f0dd90c3b80 1 --2- 172.21.15.100:0/3070373121 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560998ec4eb0 0x560998ecba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:29.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.839+0000 7f0dd90c3b80 1 --2- 172.21.15.100:0/3070373121 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560998ebf1b0 0x560998ed0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:29.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.839+0000 7f0dd90c3b80 1 --2- 172.21.15.100:0/3070373121 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560998ec4590 0x560998ed0fc0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:29.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.839+0000 7f0dd90c3b80 1 -- 172.21.15.100:0/3070373121 >> 172.21.15.100:0/3070373121 conn(0x560998de6b70 msgr2=0x560998ec2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:23:29.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.839+0000 7f0da77fe700 0 client.16371 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:23:29.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.839+0000 7f0dd90c3b80 1 -- 172.21.15.100:0/3070373121 shutdown_connections 2024-05-10T11:23:29.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.839+0000 7f0dd90c3b80 1 -- 172.21.15.100:0/3070373121 wait complete. 2024-05-10T11:23:29.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.840+0000 7f0dd90c3b80 20 client.16371 trim_cache size 0 max 16384 2024-05-10T11:23:29.844 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:23:29.844 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-05-10T11:23:29.844 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-05-10T11:23:29.845 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:23:29.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.840+0000 7f0dce830700 1 -- 172.21.15.100:0/4085147646 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0db00b2880 msgr2=0x7f0db00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:29.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.840+0000 7f0dce830700 1 --2- 172.21.15.100:0/4085147646 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0db00b2880 0x7f0db00b4d40 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8102350 tx=0x7f0dbc009260 comp rx=0 tx=0).stop 2024-05-10T11:23:29.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.840+0000 7f0dce830700 1 -- 172.21.15.100:0/4085147646 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0dc81023a0 msgr2=0x7f0dc8199ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:29.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.840+0000 7f0dce830700 1 --2- 172.21.15.100:0/4085147646 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0dc81023a0 0x7f0dc8199ac0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0db8000c00 tx=0x7f0db80047b0 comp rx=0 tx=0).stop 2024-05-10T11:23:29.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.840+0000 7f0dce830700 1 -- 172.21.15.100:0/4085147646 shutdown_connections 2024-05-10T11:23:29.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.840+0000 7f0dce830700 1 --2- 172.21.15.100:0/4085147646 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0db00b2880 0x7f0db00b4d40 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:29.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.840+0000 7f0dce830700 1 --2- 172.21.15.100:0/4085147646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0dc8108410 0x7f0dc819e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:29.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.840+0000 7f0dce830700 1 --2- 172.21.15.100:0/4085147646 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0dc81023a0 0x7f0dc8199ac0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:29.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.840+0000 7f0dce830700 1 --2- 172.21.15.100:0/4085147646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0dc8102d60 0x7f0dc819a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:29.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.840+0000 7f0dce830700 1 -- 172.21.15.100:0/4085147646 >> 172.21.15.100:0/4085147646 conn(0x7f0dc8079350 msgr2=0x7f0dc806b360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:23:29.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.840+0000 7f0dce830700 1 -- 172.21.15.100:0/4085147646 shutdown_connections 2024-05-10T11:23:29.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:29.840+0000 7f0dce830700 1 -- 172.21.15.100:0/4085147646 wait complete. 2024-05-10T11:23:29.970 INFO:tasks.workunit.client.0.smithi100.stdout: 1 107392 2.75 MB/sec execute 84 sec latency 15.158 ms 2024-05-10T11:23:30.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:29 smithi100 ceph-mon[30296]: pgmap v573: 129 pgs: 129 active+clean; 57 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 1.5 MiB/s wr, 143 op/s 2024-05-10T11:23:30.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:29 smithi178 ceph-mon[33359]: pgmap v573: 129 pgs: 129 active+clean; 57 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 1.5 MiB/s wr, 143 op/s 2024-05-10T11:23:30.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:29 smithi188 ceph-mon[34075]: pgmap v573: 129 pgs: 129 active+clean; 57 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 1.5 MiB/s wr, 143 op/s 2024-05-10T11:23:30.638 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T11:23:30.638 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T11:23:30.913 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:23:30.970 INFO:tasks.workunit.client.0.smithi100.stdout: 1 107938 2.77 MB/sec execute 85 sec latency 19.496 ms 2024-05-10T11:23:31.970 INFO:tasks.workunit.client.0.smithi100.stdout: 1 108591 2.74 MB/sec execute 86 sec latency 20.624 ms 2024-05-10T11:23:32.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:31 smithi178 ceph-mon[33359]: pgmap v574: 129 pgs: 129 active+clean; 57 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 1.3 MiB/s wr, 140 op/s 2024-05-10T11:23:32.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:31 smithi188 ceph-mon[34075]: pgmap v574: 129 pgs: 129 active+clean; 57 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 1.3 MiB/s wr, 140 op/s 2024-05-10T11:23:32.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:31 smithi100 ceph-mon[30296]: pgmap v574: 129 pgs: 129 active+clean; 57 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 1.3 MiB/s wr, 140 op/s 2024-05-10T11:23:32.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:32.873+0000 7f9e20908700 1 -- 172.21.15.100:0/1235487381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e1c108a80 msgr2=0x7f9e1c10f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:32.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:32.873+0000 7f9e20908700 1 --2- 172.21.15.100:0/1235487381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e1c108a80 0x7f9e1c10f240 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f9e1400b0d0 tx=0x7f9e1400b3e0 comp rx=0 tx=0).stop 2024-05-10T11:23:32.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:32.873+0000 7f9e20908700 1 -- 172.21.15.100:0/1235487381 shutdown_connections 2024-05-10T11:23:32.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:32.873+0000 7f9e20908700 1 --2- 172.21.15.100:0/1235487381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e1c108a80 0x7f9e1c10f240 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:32.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:32.873+0000 7f9e20908700 1 --2- 172.21.15.100:0/1235487381 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e1c1073d0 0x7f9e1c1077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:32.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:32.873+0000 7f9e20908700 1 --2- 172.21.15.100:0/1235487381 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9e1c1080c0 0x7f9e1c108540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:32.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:32.873+0000 7f9e20908700 1 -- 172.21.15.100:0/1235487381 >> 172.21.15.100:0/1235487381 conn(0x7f9e1c077740 msgr2=0x7f9e1c077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:23:32.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:32.873+0000 7f9e20908700 1 -- 172.21.15.100:0/1235487381 shutdown_connections 2024-05-10T11:23:32.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:32.873+0000 7f9e20908700 1 -- 172.21.15.100:0/1235487381 wait complete. 2024-05-10T11:23:32.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:32.874+0000 7f9e20908700 1 Processor -- start 2024-05-10T11:23:32.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:32.874+0000 7f9e20908700 1 -- start start 2024-05-10T11:23:32.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:32.875+0000 7f9e20908700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e1c1073d0 0x7f9e1c19a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:32.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:32.875+0000 7f9e20908700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9e1c1080c0 0x7f9e1c19ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:32.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:32.875+0000 7f9e20908700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e1c108a80 0x7f9e1c1a5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:32.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:32.875+0000 7f9e20908700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9e1c112400 con 0x7f9e1c108a80 2024-05-10T11:23:32.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:32.875+0000 7f9e20908700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9e1c112280 con 0x7f9e1c1073d0 2024-05-10T11:23:32.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:32.875+0000 7f9e20908700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9e1c112580 con 0x7f9e1c1080c0 2024-05-10T11:23:32.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:32.875+0000 7f9e1a59c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e1c1073d0 0x7f9e1c19a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:32.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:32.875+0000 7f9e1ad9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e1c108a80 0x7f9e1c1a5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:32.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:32.875+0000 7f9e1a59c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e1c1073d0 0x7f9e1c19a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:37524/0 (socket says 172.21.15.100:37524) 2024-05-10T11:23:32.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:32.875+0000 7f9e1a59c700 1 -- 172.21.15.100:0/1366578588 learned_addr learned my addr 172.21.15.100:0/1366578588 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:23:32.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:32.875+0000 7f9e19d9b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9e1c1080c0 0x7f9e1c19ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:32.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:32.875+0000 7f9e1a59c700 1 -- 172.21.15.100:0/1366578588 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9e1c1080c0 msgr2=0x7f9e1c19ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:32.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:32.875+0000 7f9e1a59c700 1 --2- 172.21.15.100:0/1366578588 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9e1c1080c0 0x7f9e1c19ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:32.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:32.875+0000 7f9e1a59c700 1 -- 172.21.15.100:0/1366578588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e1c108a80 msgr2=0x7f9e1c1a5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:32.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:32.875+0000 7f9e1a59c700 1 --2- 172.21.15.100:0/1366578588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e1c108a80 0x7f9e1c1a5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:32.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:32.875+0000 7f9e1a59c700 1 -- 172.21.15.100:0/1366578588 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e1c1a5d60 con 0x7f9e1c1073d0 2024-05-10T11:23:32.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:32.875+0000 7f9e1ad9d700 1 --2- 172.21.15.100:0/1366578588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e1c108a80 0x7f9e1c1a5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:23:32.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:32.875+0000 7f9e1a59c700 1 --2- 172.21.15.100:0/1366578588 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e1c1073d0 0x7f9e1c19a7a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c00e9a0 tx=0x7f9e0c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:23:32.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:32.876+0000 7f9e0b7fe700 1 -- 172.21.15.100:0/1366578588 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c00ca10 con 0x7f9e1c1073d0 2024-05-10T11:23:32.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:32.876+0000 7f9e20908700 1 -- 172.21.15.100:0/1366578588 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e1c198590 con 0x7f9e1c1073d0 2024-05-10T11:23:32.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:32.876+0000 7f9e20908700 1 -- 172.21.15.100:0/1366578588 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e1c198b20 con 0x7f9e1c1073d0 2024-05-10T11:23:32.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:32.876+0000 7f9e0b7fe700 1 -- 172.21.15.100:0/1366578588 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9e0c00cbb0 con 0x7f9e1c1073d0 2024-05-10T11:23:32.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:32.876+0000 7f9e0b7fe700 1 -- 172.21.15.100:0/1366578588 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c018680 con 0x7f9e1c1073d0 2024-05-10T11:23:32.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:32.877+0000 7f9e20908700 1 -- 172.21.15.100:0/1366578588 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dfc005330 con 0x7f9e1c1073d0 2024-05-10T11:23:32.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:32.878+0000 7f9e0b7fe700 1 -- 172.21.15.100:0/1366578588 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f9e0c010430 con 0x7f9e1c1073d0 2024-05-10T11:23:32.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:32.879+0000 7f9e0b7fe700 1 --2- 172.21.15.100:0/1366578588 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f9e040b2880 0x7f9e040b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:32.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:32.879+0000 7f9e19d9b700 1 --2- 172.21.15.100:0/1366578588 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f9e040b2880 0x7f9e040b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:32.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:32.879+0000 7f9e0b7fe700 1 -- 172.21.15.100:0/1366578588 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f9e0c0e2a40 con 0x7f9e1c1073d0 2024-05-10T11:23:32.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:32.879+0000 7f9e19d9b700 1 --2- 172.21.15.100:0/1366578588 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f9e040b2880 0x7f9e040b4d40 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c077c40 tx=0x7f9e1000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:23:32.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:32.883+0000 7f9e0b7fe700 1 -- 172.21.15.100:0/1366578588 <== mon.2 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f9e0c0aa400 con 0x7f9e1c1073d0 2024-05-10T11:23:32.970 INFO:tasks.workunit.client.0.smithi100.stdout: 1 109054 2.74 MB/sec execute 87 sec latency 65.496 ms 2024-05-10T11:23:33.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:33.187+0000 7f9e20908700 1 -- 172.21.15.100:0/1366578588 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9dfc005620 con 0x7f9e1c1073d0 2024-05-10T11:23:33.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:33.188+0000 7f9e0b7fe700 1 -- 172.21.15.100:0/1366578588 <== mon.2 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11894 (secure 0 0 0) 0x7f9e0c0a9d00 con 0x7f9e1c1073d0 2024-05-10T11:23:33.188 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:23:33.188 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2072225012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2072225012},{"type":"v1","addr":"172.21.15.100:6835","nonce":2072225012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/401939540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":401939540},{"type":"v1","addr":"172.21.15.100:6837","nonce":401939540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14571,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/3946311025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3946311025},{"type":"v1","addr":"172.21.15.100:6839","nonce":3946311025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3736371660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3736371660},{"type":"v1","addr":"172.21.15.100:6841","nonce":3736371660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/2741810791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2741810791},{"type":"v1","addr":"172.21.15.178:6833","nonce":2741810791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1704891041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1704891041},{"type":"v1","addr":"172.21.15.188:6833","nonce":1704891041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T11:14:27.242452+0000","modified":"2024-05-10T11:23:19.557113+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24439,"mds_1":24437,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/2124076312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2124076312},{"type":"v1","addr":"172.21.15.178:6835","nonce":2124076312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/3224358425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3224358425},{"type":"v1","addr":"172.21.15.178:6837","nonce":3224358425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/515547690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":515547690},{"type":"v1","addr":"172.21.15.188:6835","nonce":515547690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/1943342620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1943342620},{"type":"v1","addr":"172.21.15.188:6837","nonce":1943342620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.178:6839/3760985404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3760985404},{"type":"v1","addr":"172.21.15.178:6839","nonce":3760985404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.188:6839/3194917070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":3194917070},{"type":"v1","addr":"172.21.15.188:6839","nonce":3194917070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24439,"qdb_cluster":[24439,24427,24437]},"id":1}]} 2024-05-10T11:23:33.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:33.191+0000 7f9e20908700 1 -- 172.21.15.100:0/1366578588 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f9e040b2880 msgr2=0x7f9e040b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:33.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:33.191+0000 7f9e20908700 1 --2- 172.21.15.100:0/1366578588 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f9e040b2880 0x7f9e040b4d40 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c077c40 tx=0x7f9e1000a380 comp rx=0 tx=0).stop 2024-05-10T11:23:33.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:33.191+0000 7f9e20908700 1 -- 172.21.15.100:0/1366578588 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e1c1073d0 msgr2=0x7f9e1c19a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:33.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:33.191+0000 7f9e20908700 1 --2- 172.21.15.100:0/1366578588 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e1c1073d0 0x7f9e1c19a7a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c00e9a0 tx=0x7f9e0c00ecb0 comp rx=0 tx=0).stop 2024-05-10T11:23:33.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:33.191+0000 7f9e20908700 1 -- 172.21.15.100:0/1366578588 shutdown_connections 2024-05-10T11:23:33.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:33.191+0000 7f9e20908700 1 --2- 172.21.15.100:0/1366578588 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f9e040b2880 0x7f9e040b4d40 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:33.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:33.191+0000 7f9e20908700 1 --2- 172.21.15.100:0/1366578588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e1c108a80 0x7f9e1c1a5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:33.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:33.191+0000 7f9e20908700 1 --2- 172.21.15.100:0/1366578588 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e1c1073d0 0x7f9e1c19a7a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:33.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:33.191+0000 7f9e20908700 1 --2- 172.21.15.100:0/1366578588 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9e1c1080c0 0x7f9e1c19ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:33.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:33.191+0000 7f9e20908700 1 -- 172.21.15.100:0/1366578588 >> 172.21.15.100:0/1366578588 conn(0x7f9e1c077740 msgr2=0x7f9e1c100a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:23:33.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:33.191+0000 7f9e20908700 1 -- 172.21.15.100:0/1366578588 shutdown_connections 2024-05-10T11:23:33.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:33.191+0000 7f9e20908700 1 -- 172.21.15.100:0/1366578588 wait complete. 2024-05-10T11:23:33.192 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 31 2024-05-10T11:23:33.971 INFO:tasks.workunit.client.0.smithi100.stdout: 1 109772 2.78 MB/sec execute 88 sec latency 16.797 ms 2024-05-10T11:23:34.082 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-10T11:23:34.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:33 smithi178 ceph-mon[33359]: pgmap v575: 129 pgs: 129 active+clean; 58 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.5 MiB/s wr, 162 op/s 2024-05-10T11:23:34.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:33 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/1366578588' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:23:34.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:33 smithi188 ceph-mon[34075]: pgmap v575: 129 pgs: 129 active+clean; 58 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.5 MiB/s wr, 162 op/s 2024-05-10T11:23:34.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:33 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/1366578588' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:23:34.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:33 smithi100 ceph-mon[30296]: pgmap v575: 129 pgs: 129 active+clean; 58 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.5 MiB/s wr, 162 op/s 2024-05-10T11:23:34.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:33 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/1366578588' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:23:34.354 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:23:34.971 INFO:tasks.workunit.client.0.smithi100.stdout: 1 110201 2.80 MB/sec execute 89 sec latency 67.570 ms 2024-05-10T11:23:35.971 INFO:tasks.workunit.client.0.smithi100.stdout: 1 110423 2.77 MB/sec execute 90 sec latency 67.326 ms 2024-05-10T11:23:36.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:35 smithi100 ceph-mon[30296]: pgmap v576: 129 pgs: 129 active+clean; 59 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 1.5 MiB/s wr, 158 op/s 2024-05-10T11:23:36.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:35 smithi178 ceph-mon[33359]: pgmap v576: 129 pgs: 129 active+clean; 59 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 1.5 MiB/s wr, 158 op/s 2024-05-10T11:23:36.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:35 smithi188 ceph-mon[34075]: pgmap v576: 129 pgs: 129 active+clean; 59 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 1.5 MiB/s wr, 158 op/s 2024-05-10T11:23:36.971 INFO:tasks.workunit.client.0.smithi100.stdout: 1 110718 2.74 MB/sec execute 91 sec latency 96.989 ms 2024-05-10T11:23:37.971 INFO:tasks.workunit.client.0.smithi100.stdout: 1 111184 2.73 MB/sec execute 92 sec latency 79.772 ms 2024-05-10T11:23:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.032+0000 7f1338966640 1 -- 172.21.15.100:0/1910602823 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1334102b00 msgr2=0x7f1334102f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.032+0000 7f1338966640 1 --2- 172.21.15.100:0/1910602823 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1334102b00 0x7f1334102f80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1328009860 tx=0x7f132802a360 comp rx=0 tx=0).stop 2024-05-10T11:23:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.032+0000 7f1338966640 1 -- 172.21.15.100:0/1910602823 shutdown_connections 2024-05-10T11:23:38.033 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.032+0000 7f1338966640 1 --2- 172.21.15.100:0/1910602823 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13341034c0 0x7f1334109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:38.033 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.032+0000 7f1338966640 1 --2- 172.21.15.100:0/1910602823 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1334102b00 0x7f1334102f80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:38.033 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.032+0000 7f1338966640 1 --2- 172.21.15.100:0/1910602823 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1334101910 0x7f1334101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:38.033 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.032+0000 7f1338966640 1 -- 172.21.15.100:0/1910602823 >> 172.21.15.100:0/1910602823 conn(0x7f13340fce80 msgr2=0x7f13340ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:23:38.033 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.033+0000 7f1338966640 1 -- 172.21.15.100:0/1910602823 shutdown_connections 2024-05-10T11:23:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.033+0000 7f1338966640 1 -- 172.21.15.100:0/1910602823 wait complete. 2024-05-10T11:23:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.034+0000 7f1338966640 1 Processor -- start 2024-05-10T11:23:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.034+0000 7f1338966640 1 -- start start 2024-05-10T11:23:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.034+0000 7f1338966640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1334101910 0x7f1334196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.034+0000 7f1338966640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1334102b00 0x7f1334196f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.034+0000 7f1338966640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13341034c0 0x7f13341af1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.034+0000 7f1338966640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f133410c2a0 con 0x7f13341034c0 2024-05-10T11:23:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.034+0000 7f1338966640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f133410c120 con 0x7f1334101910 2024-05-10T11:23:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.034+0000 7f1338966640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f133410c420 con 0x7f1334102b00 2024-05-10T11:23:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.034+0000 7f1331d9b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1334102b00 0x7f1334196f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.034+0000 7f1331d9b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1334102b00 0x7f1334196f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:47320/0 (socket says 172.21.15.100:47320) 2024-05-10T11:23:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.034+0000 7f1331d9b640 1 -- 172.21.15.100:0/795466138 learned_addr learned my addr 172.21.15.100:0/795466138 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:23:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.034+0000 7f1332d9d640 1 --2- 172.21.15.100:0/795466138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13341034c0 0x7f13341af1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.034+0000 7f133259c640 1 --2- 172.21.15.100:0/795466138 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1334101910 0x7f1334196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.035+0000 7f1331d9b640 1 -- 172.21.15.100:0/795466138 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1334101910 msgr2=0x7f1334196a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.035+0000 7f1331d9b640 1 --2- 172.21.15.100:0/795466138 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1334101910 0x7f1334196a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.035+0000 7f1331d9b640 1 -- 172.21.15.100:0/795466138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13341034c0 msgr2=0x7f13341af1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.035+0000 7f1331d9b640 1 --2- 172.21.15.100:0/795466138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13341034c0 0x7f13341af1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.035+0000 7f1331d9b640 1 -- 172.21.15.100:0/795466138 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13341af700 con 0x7f1334102b00 2024-05-10T11:23:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.035+0000 7f133259c640 1 --2- 172.21.15.100:0/795466138 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1334101910 0x7f1334196a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:23:38.037 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.035+0000 7f1331d9b640 1 --2- 172.21.15.100:0/795466138 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1334102b00 0x7f1334196f50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1328009980 tx=0x7f1328032cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:23:38.037 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.035+0000 7f13237fe640 1 -- 172.21.15.100:0/795466138 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1328037050 con 0x7f1334102b00 2024-05-10T11:23:38.037 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.035+0000 7f13237fe640 1 -- 172.21.15.100:0/795466138 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1328038040 con 0x7f1334102b00 2024-05-10T11:23:38.037 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.035+0000 7f1338966640 1 -- 172.21.15.100:0/795466138 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13341af930 con 0x7f1334102b00 2024-05-10T11:23:38.037 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.035+0000 7f13237fe640 1 -- 172.21.15.100:0/795466138 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13280046c0 con 0x7f1334102b00 2024-05-10T11:23:38.037 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.035+0000 7f1338966640 1 -- 172.21.15.100:0/795466138 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13341afe70 con 0x7f1334102b00 2024-05-10T11:23:38.037 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.038+0000 7f13237fe640 1 -- 172.21.15.100:0/795466138 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f1328004040 con 0x7f1334102b00 2024-05-10T11:23:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.038+0000 7f133b67e740 1 Processor -- start 2024-05-10T11:23:38.039 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.038+0000 7f133b67e740 1 -- start start 2024-05-10T11:23:38.039 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.038+0000 7f133b67e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc859ebdb0 0x55bc859ec1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:38.039 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.038+0000 7f133b67e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc859ec7a0 0x55bc859f12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:38.040 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.038+0000 7f133b67e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bc859f1910 0x55bc859f3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:38.040 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.038+0000 7f133b67e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55bc859505d0 con 0x55bc859ebdb0 2024-05-10T11:23:38.040 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.038+0000 7f133b67e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55bc85909890 con 0x55bc859ec7a0 2024-05-10T11:23:38.040 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.038+0000 7f133b67e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55bc8594c590 con 0x55bc859f1910 2024-05-10T11:23:38.040 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.039+0000 7f1332d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc859ebdb0 0x55bc859ec1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:38.040 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.039+0000 7f133259c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc859ec7a0 0x55bc859f12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:38.040 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.039+0000 7f1332d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc859ebdb0 0x55bc859ec1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47302/0 (socket says 172.21.15.100:47302) 2024-05-10T11:23:38.040 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.039+0000 7f1332d9d640 1 -- 172.21.15.100:0/2266125250 learned_addr learned my addr 172.21.15.100:0/2266125250 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:23:38.040 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.039+0000 7f1331d9b640 1 --2- 172.21.15.100:0/2266125250 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bc859f1910 0x55bc859f3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:38.040 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.039+0000 7f1332d9d640 1 -- 172.21.15.100:0/2266125250 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bc859f1910 msgr2=0x55bc859f3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:38.040 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.039+0000 7f1332d9d640 1 --2- 172.21.15.100:0/2266125250 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bc859f1910 0x55bc859f3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:38.040 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.039+0000 7f1332d9d640 1 -- 172.21.15.100:0/2266125250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc859ec7a0 msgr2=0x55bc859f12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:38.040 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.039+0000 7f1332d9d640 1 --2- 172.21.15.100:0/2266125250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc859ec7a0 0x55bc859f12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:38.041 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.039+0000 7f1332d9d640 1 -- 172.21.15.100:0/2266125250 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc85662210 con 0x55bc859ebdb0 2024-05-10T11:23:38.041 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.039+0000 7f13237fe640 1 --2- 172.21.15.100:0/795466138 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f131c0b28e0 0x7f131c0b4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:38.041 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.039+0000 7f1331d9b640 1 --2- 172.21.15.100:0/2266125250 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bc859f1910 0x55bc859f3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:23:38.041 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.039+0000 7f133259c640 1 --2- 172.21.15.100:0/2266125250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc859ec7a0 0x55bc859f12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:23:38.041 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.039+0000 7f1332d9d640 1 --2- 172.21.15.100:0/2266125250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc859ebdb0 0x55bc859ec1d0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f132c009ee0 tx=0x7f132c00f7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3378bad428813560 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:23:38.041 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.039+0000 7f133259c640 1 --2- 172.21.15.100:0/795466138 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f131c0b28e0 0x7f131c0b4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:38.041 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.039+0000 7f13237fe640 1 -- 172.21.15.100:0/795466138 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f1328102a80 con 0x7f1334102b00 2024-05-10T11:23:38.041 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.039+0000 7f1320ff9640 1 -- 172.21.15.100:0/2266125250 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f132c003970 con 0x55bc859ebdb0 2024-05-10T11:23:38.041 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.039+0000 7f1320ff9640 1 -- 172.21.15.100:0/2266125250 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f132c004030 con 0x55bc859ebdb0 2024-05-10T11:23:38.041 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.039+0000 7f1320ff9640 1 -- 172.21.15.100:0/2266125250 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f132c00c990 con 0x55bc859ebdb0 2024-05-10T11:23:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.040+0000 7f133b67e740 1 -- 172.21.15.100:0/2266125250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc859ebdb0 msgr2=0x55bc859ec1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.040+0000 7f133259c640 1 --2- 172.21.15.100:0/795466138 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f131c0b28e0 0x7f131c0b4da0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x55bc859ecba0 tx=0x7f1324008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:23:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.040+0000 7f133b67e740 1 --2- 172.21.15.100:0/2266125250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc859ebdb0 0x55bc859ec1d0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f132c009ee0 tx=0x7f132c00f7d0 comp rx=0 tx=0).stop 2024-05-10T11:23:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.040+0000 7f133b67e740 1 -- 172.21.15.100:0/2266125250 shutdown_connections 2024-05-10T11:23:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.040+0000 7f133b67e740 1 --2- 172.21.15.100:0/2266125250 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bc859f1910 0x55bc859f3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.040+0000 7f133b67e740 1 --2- 172.21.15.100:0/2266125250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc859ec7a0 0x55bc859f12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.040+0000 7f133b67e740 1 --2- 172.21.15.100:0/2266125250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc859ebdb0 0x55bc859ec1d0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.040+0000 7f133b67e740 1 -- 172.21.15.100:0/2266125250 >> 172.21.15.100:0/2266125250 conn(0x55bc859e7420 msgr2=0x55bc859e7820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:23:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.040+0000 7f133b67e740 1 -- 172.21.15.100:0/2266125250 shutdown_connections 2024-05-10T11:23:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.040+0000 7f133b67e740 1 -- 172.21.15.100:0/2266125250 wait complete. 2024-05-10T11:23:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.041+0000 7f133b67e740 1 -- start start 2024-05-10T11:23:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.041+0000 7f133b67e740 1 Processor -- start 2024-05-10T11:23:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.041+0000 7f133b67e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc859ebdb0 0x55bc859f6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.041+0000 7f133b67e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc859ec7a0 0x55bc859f6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.041+0000 7f133b67e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bc859f1910 0x55bc859fd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.041+0000 7f133b67e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55bc859f48c0 con 0x55bc859ebdb0 2024-05-10T11:23:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.041+0000 7f133b67e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55bc8590d370 con 0x55bc859ec7a0 2024-05-10T11:23:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.041+0000 7f133b67e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55bc859f4a40 con 0x55bc859f1910 2024-05-10T11:23:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.041+0000 7f133b67e740 10 client.0 _ll_register_callbacks cb 0x55bc85661a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:23:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.041+0000 7f133259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc859ebdb0 0x55bc859f6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.041+0000 7f133259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc859ebdb0 0x55bc859f6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47316/0 (socket says 172.21.15.100:47316) 2024-05-10T11:23:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.041+0000 7f133259c640 1 -- 172.21.15.100:0/2323739902 learned_addr learned my addr 172.21.15.100:0/2323739902 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:23:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.041+0000 7f1331d9b640 1 --2- 172.21.15.100:0/2323739902 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc859ec7a0 0x55bc859f6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.041+0000 7f1332d9d640 1 --2- 172.21.15.100:0/2323739902 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bc859f1910 0x55bc859fd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.041+0000 7f133259c640 1 -- 172.21.15.100:0/2323739902 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bc859f1910 msgr2=0x55bc859fd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.041+0000 7f133259c640 1 --2- 172.21.15.100:0/2323739902 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bc859f1910 0x55bc859fd5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.041+0000 7f133259c640 1 -- 172.21.15.100:0/2323739902 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc859ec7a0 msgr2=0x55bc859f6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.041+0000 7f133259c640 1 --2- 172.21.15.100:0/2323739902 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc859ec7a0 0x55bc859f6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.041+0000 7f133259c640 1 -- 172.21.15.100:0/2323739902 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc8572e620 con 0x55bc859ebdb0 2024-05-10T11:23:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.041+0000 7f133259c640 1 --2- 172.21.15.100:0/2323739902 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc859ebdb0 0x55bc859f6050 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f13240167d0 tx=0x7f13240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81c451b4f8914c53 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:23:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.041+0000 7f1319ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:23:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.042+0000 7f1319ffb640 1 -- 172.21.15.100:0/2323739902 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1324014840 con 0x55bc859ebdb0 2024-05-10T11:23:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.042+0000 7f1319ffb640 1 -- 172.21.15.100:0/2323739902 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f13240149e0 con 0x55bc859ebdb0 2024-05-10T11:23:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.042+0000 7f133b67e740 1 -- 172.21.15.100:0/2323739902 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f132c004030 con 0x55bc859ebdb0 2024-05-10T11:23:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.042+0000 7f1319ffb640 1 -- 172.21.15.100:0/2323739902 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f1324014be0 con 0x55bc859ebdb0 2024-05-10T11:23:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.042+0000 7f133b67e740 10 client.16386 fetch_fsmap learned FSMap version 31 2024-05-10T11:23:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.042+0000 7f133b67e740 1 -- 172.21.15.100:0/2323739902 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55bc85a005f0 con 0x55bc859ebdb0 2024-05-10T11:23:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.042+0000 7f1319ffb640 1 -- 172.21.15.100:0/2323739902 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f132c004030 con 0x55bc859ebdb0 2024-05-10T11:23:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.042+0000 7f1319ffb640 1 -- 172.21.15.100:0/2323739902 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f1324029030 con 0x55bc859ebdb0 2024-05-10T11:23:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.042+0000 7f133b67e740 10 client.16386 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:23:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.042+0000 7f133b67e740 10 client.16386 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:23:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.042+0000 7f133b67e740 20 client.16386 populate_metadata read hostname 'smithi100' 2024-05-10T11:23:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.042+0000 7f133b67e740 1 --2- 172.21.15.100:0/2323739902 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55bc85a01d30 0x55bc85a04140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.042+0000 7f133b67e740 4 client.16386 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:23:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.042+0000 7f133b67e740 1 -- 172.21.15.100:0/2323739902 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc8595e1e0 con 0x55bc85a01d30 2024-05-10T11:23:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.042+0000 7f1331d9b640 1 --2- 172.21.15.100:0/2323739902 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55bc85a01d30 0x55bc85a04140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.043+0000 7f1319ffb640 10 client.16386 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:23:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.043+0000 7f1331d9b640 1 --2- 172.21.15.100:0/2323739902 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55bc85a01d30 0x55bc85a04140 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d151a2c1c7e5b947 server_cookie=c44eed49a2408d99 in_seq=0 out_seq=0 2024-05-10T11:23:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.043+0000 7f1319ffb640 1 -- 172.21.15.100:0/2323739902 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bc8595e1e0 con 0x55bc85a01d30 2024-05-10T11:23:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.043+0000 7f1319ffb640 10 client.16386 handle_command_reply: tid=0 2024-05-10T11:23:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.043+0000 7f133b67e740 1 client.16386 shutdown 2024-05-10T11:23:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.044+0000 7f133b67e740 1 -- 172.21.15.100:0/2323739902 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc859ebdb0 msgr2=0x55bc859f6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.044+0000 7f133b67e740 1 --2- 172.21.15.100:0/2323739902 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc859ebdb0 0x55bc859f6050 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f13240167d0 tx=0x7f13240046c0 comp rx=0 tx=0).stop 2024-05-10T11:23:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.044+0000 7f133b67e740 1 -- 172.21.15.100:0/2323739902 shutdown_connections 2024-05-10T11:23:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.044+0000 7f133b67e740 1 --2- 172.21.15.100:0/2323739902 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55bc85a01d30 0x55bc85a04140 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.044+0000 7f133b67e740 1 --2- 172.21.15.100:0/2323739902 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bc859f1910 0x55bc859fd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.044+0000 7f133b67e740 1 --2- 172.21.15.100:0/2323739902 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc859ec7a0 0x55bc859f6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:38.050 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.044+0000 7f133b67e740 1 --2- 172.21.15.100:0/2323739902 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc859ebdb0 0x55bc859f6050 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:38.050 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.044+0000 7f133b67e740 1 -- 172.21.15.100:0/2323739902 >> 172.21.15.100:0/2323739902 conn(0x55bc859e7420 msgr2=0x55bc859e6580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:23:38.050 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.044+0000 7f1319ffb640 0 client.16386 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:23:38.050 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.044+0000 7f133b67e740 1 -- 172.21.15.100:0/2323739902 shutdown_connections 2024-05-10T11:23:38.050 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.044+0000 7f133b67e740 1 -- 172.21.15.100:0/2323739902 wait complete. 2024-05-10T11:23:38.050 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.044+0000 7f133b67e740 20 client.16386 trim_cache size 0 max 16384 2024-05-10T11:23:38.062 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.061+0000 7f133b67e740 1 Processor -- start 2024-05-10T11:23:38.062 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.061+0000 7f133b67e740 1 -- start start 2024-05-10T11:23:38.062 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.061+0000 7f133b67e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bc859e7420 0x55bc859f5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:38.062 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.061+0000 7f133b67e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc859ebdb0 0x55bc859fd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:38.062 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.061+0000 7f133b67e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc859ec7a0 0x55bc859f61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:38.062 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.061+0000 7f133b67e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f13341bcb70 con 0x55bc859ebdb0 2024-05-10T11:23:38.062 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.061+0000 7f133b67e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f13341ba440 con 0x55bc859ec7a0 2024-05-10T11:23:38.062 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.061+0000 7f133b67e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f13341b8290 con 0x55bc859e7420 2024-05-10T11:23:38.062 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.061+0000 7f1331d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc859ebdb0 0x55bc859fd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:38.062 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.061+0000 7f1331d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc859ebdb0 0x55bc859fd590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47322/0 (socket says 172.21.15.100:47322) 2024-05-10T11:23:38.062 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.061+0000 7f133259c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bc859e7420 0x55bc859f5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:38.062 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.061+0000 7f1331d9b640 1 -- 172.21.15.100:0/3848530616 learned_addr learned my addr 172.21.15.100:0/3848530616 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:23:38.063 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.061+0000 7f1332d9d640 1 --2- 172.21.15.100:0/3848530616 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc859ec7a0 0x55bc859f61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:38.063 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.061+0000 7f1331d9b640 1 -- 172.21.15.100:0/3848530616 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bc859e7420 msgr2=0x55bc859f5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:38.063 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.061+0000 7f1331d9b640 1 --2- 172.21.15.100:0/3848530616 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bc859e7420 0x55bc859f5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:38.063 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.061+0000 7f1331d9b640 1 -- 172.21.15.100:0/3848530616 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc859ec7a0 msgr2=0x55bc859f61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:38.063 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.061+0000 7f1331d9b640 1 --2- 172.21.15.100:0/3848530616 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc859ec7a0 0x55bc859f61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:38.063 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.061+0000 7f1331d9b640 1 -- 172.21.15.100:0/3848530616 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc859e80f0 con 0x55bc859ebdb0 2024-05-10T11:23:38.063 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.062+0000 7f1331d9b640 1 --2- 172.21.15.100:0/3848530616 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc859ebdb0 0x55bc859fd590 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f13280c9d60 tx=0x7f1328032f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ee3b2a8d2407df7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:23:38.063 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.062+0000 7f131bfff640 1 -- 172.21.15.100:0/3848530616 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13280e4420 con 0x55bc859ebdb0 2024-05-10T11:23:38.063 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.062+0000 7f131bfff640 1 -- 172.21.15.100:0/3848530616 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f13280cd070 con 0x55bc859ebdb0 2024-05-10T11:23:38.063 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.062+0000 7f131bfff640 1 -- 172.21.15.100:0/3848530616 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13280ce180 con 0x55bc859ebdb0 2024-05-10T11:23:38.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.063+0000 7f133b67e740 1 -- 172.21.15.100:0/3848530616 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc859ebdb0 msgr2=0x55bc859fd590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:38.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.063+0000 7f133b67e740 1 --2- 172.21.15.100:0/3848530616 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc859ebdb0 0x55bc859fd590 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f13280c9d60 tx=0x7f1328032f60 comp rx=0 tx=0).stop 2024-05-10T11:23:38.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.063+0000 7f133b67e740 1 -- 172.21.15.100:0/3848530616 shutdown_connections 2024-05-10T11:23:38.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.063+0000 7f133b67e740 1 --2- 172.21.15.100:0/3848530616 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc859ec7a0 0x55bc859f61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:38.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.063+0000 7f133b67e740 1 --2- 172.21.15.100:0/3848530616 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc859ebdb0 0x55bc859fd590 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:38.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.063+0000 7f133b67e740 1 --2- 172.21.15.100:0/3848530616 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bc859e7420 0x55bc859f5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:38.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.063+0000 7f133b67e740 1 -- 172.21.15.100:0/3848530616 >> 172.21.15.100:0/3848530616 conn(0x55bc859f13e0 msgr2=0x55bc859eda20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:23:38.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.063+0000 7f133b67e740 1 -- 172.21.15.100:0/3848530616 shutdown_connections 2024-05-10T11:23:38.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.063+0000 7f133b67e740 1 -- 172.21.15.100:0/3848530616 wait complete. 2024-05-10T11:23:38.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.063+0000 7f133b67e740 1 -- start start 2024-05-10T11:23:38.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.063+0000 7f133b67e740 1 Processor -- start 2024-05-10T11:23:38.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.063+0000 7f133b67e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bc859e7420 0x55bc859fa720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:38.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.063+0000 7f133b67e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc859ebdb0 0x55bc859fac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:38.068 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.063+0000 7f133b67e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc859ec7a0 0x55bc859fbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:38.068 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.063+0000 7f133b67e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f13341b8900 con 0x55bc859ebdb0 2024-05-10T11:23:38.068 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.063+0000 7f133b67e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f13341b9d70 con 0x55bc859ec7a0 2024-05-10T11:23:38.068 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.063+0000 7f133b67e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f13341b8fd0 con 0x55bc859e7420 2024-05-10T11:23:38.068 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.063+0000 7f133b67e740 10 client.0 _ll_register_callbacks cb 0x7f1324007800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:23:38.068 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.063+0000 7f1331d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc859ebdb0 0x55bc859fac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:38.068 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.063+0000 7f133259c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bc859e7420 0x55bc859fa720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:38.068 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.063+0000 7f1331d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc859ebdb0 0x55bc859fac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47326/0 (socket says 172.21.15.100:47326) 2024-05-10T11:23:38.068 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.064+0000 7f1331d9b640 1 -- 172.21.15.100:0/4274447113 learned_addr learned my addr 172.21.15.100:0/4274447113 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:23:38.068 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.064+0000 7f1332d9d640 1 --2- 172.21.15.100:0/4274447113 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc859ec7a0 0x55bc859fbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:38.068 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.064+0000 7f133259c640 1 -- 172.21.15.100:0/4274447113 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc859ec7a0 msgr2=0x55bc859fbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:38.068 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.064+0000 7f133259c640 1 --2- 172.21.15.100:0/4274447113 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc859ec7a0 0x55bc859fbc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:38.068 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.064+0000 7f133259c640 1 -- 172.21.15.100:0/4274447113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc859ebdb0 msgr2=0x55bc859fac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:38.068 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.064+0000 7f133259c640 1 --2- 172.21.15.100:0/4274447113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc859ebdb0 0x55bc859fac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:38.068 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.064+0000 7f133259c640 1 -- 172.21.15.100:0/4274447113 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc859fc260 con 0x55bc859e7420 2024-05-10T11:23:38.068 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.064+0000 7f1331d9b640 1 --2- 172.21.15.100:0/4274447113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc859ebdb0 0x55bc859fac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:23:38.069 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.064+0000 7f133259c640 1 --2- 172.21.15.100:0/4274447113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bc859e7420 0x55bc859fa720 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x55bc859e7240 tx=0x7f1324006c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb309ca5fc3a0dc7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:23:38.069 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.064+0000 7f1319ffb640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:23:38.069 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.064+0000 7f1319ffb640 1 -- 172.21.15.100:0/4274447113 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1324037020 con 0x55bc859e7420 2024-05-10T11:23:38.069 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.064+0000 7f1319ffb640 1 -- 172.21.15.100:0/4274447113 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f1324005f10 con 0x55bc859e7420 2024-05-10T11:23:38.069 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.064+0000 7f133b67e740 1 -- 172.21.15.100:0/4274447113 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13280cd070 con 0x55bc859e7420 2024-05-10T11:23:38.069 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.064+0000 7f1319ffb640 1 -- 172.21.15.100:0/4274447113 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f1324042050 con 0x55bc859e7420 2024-05-10T11:23:38.069 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.064+0000 7f133b67e740 10 client.25885 fetch_fsmap learned FSMap version 31 2024-05-10T11:23:38.069 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.064+0000 7f133b67e740 1 -- 172.21.15.100:0/4274447113 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55bc859fd5e0 con 0x55bc859e7420 2024-05-10T11:23:38.069 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.065+0000 7f1319ffb640 1 -- 172.21.15.100:0/4274447113 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f13280cd070 con 0x55bc859e7420 2024-05-10T11:23:38.069 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.065+0000 7f1319ffb640 1 -- 172.21.15.100:0/4274447113 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f1324029040 con 0x55bc859e7420 2024-05-10T11:23:38.069 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.065+0000 7f133b67e740 10 client.25885 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:23:38.069 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.065+0000 7f133b67e740 10 client.25885 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:23:38.069 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.065+0000 7f133b67e740 20 client.25885 populate_metadata read hostname 'smithi100' 2024-05-10T11:23:38.069 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.065+0000 7f133b67e740 1 --2- 172.21.15.100:0/4274447113 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55bc859f1910 0x55bc85a6ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:38.069 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.065+0000 7f133b67e740 4 client.25885 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:23:38.070 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.065+0000 7f133b67e740 1 -- 172.21.15.100:0/4274447113 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bc859e7270 con 0x55bc859f1910 2024-05-10T11:23:38.070 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.065+0000 7f1331d9b640 1 --2- 172.21.15.100:0/4274447113 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55bc859f1910 0x55bc85a6ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:38.070 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.065+0000 7f1331d9b640 1 --2- 172.21.15.100:0/4274447113 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55bc859f1910 0x55bc85a6ead0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b55ef1bc0327db93 server_cookie=55ca4971d00fef8d in_seq=0 out_seq=0 2024-05-10T11:23:38.070 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.065+0000 7f1319ffb640 10 client.25885 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:23:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.081+0000 7f1319ffb640 1 -- 172.21.15.100:0/4274447113 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bc859e7270 con 0x55bc859f1910 2024-05-10T11:23:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.081+0000 7f1319ffb640 10 client.25885 handle_command_reply: tid=0 2024-05-10T11:23:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.081+0000 7f133b67e740 1 client.25885 shutdown 2024-05-10T11:23:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.081+0000 7f133b67e740 1 -- 172.21.15.100:0/4274447113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bc859e7420 msgr2=0x55bc859fa720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.081+0000 7f133b67e740 1 --2- 172.21.15.100:0/4274447113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bc859e7420 0x55bc859fa720 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x55bc859e7240 tx=0x7f1324006c10 comp rx=0 tx=0).stop 2024-05-10T11:23:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.081+0000 7f133b67e740 1 -- 172.21.15.100:0/4274447113 shutdown_connections 2024-05-10T11:23:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.081+0000 7f133b67e740 1 --2- 172.21.15.100:0/4274447113 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55bc859f1910 0x55bc85a6ead0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.081+0000 7f133b67e740 1 --2- 172.21.15.100:0/4274447113 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc859ec7a0 0x55bc859fbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.081+0000 7f1319ffb640 0 client.25885 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:23:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.081+0000 7f133b67e740 1 --2- 172.21.15.100:0/4274447113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc859ebdb0 0x55bc859fac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.081+0000 7f133b67e740 1 --2- 172.21.15.100:0/4274447113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bc859e7420 0x55bc859fa720 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:38.083 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.082+0000 7f133b67e740 1 -- 172.21.15.100:0/4274447113 >> 172.21.15.100:0/4274447113 conn(0x55bc859f13e0 msgr2=0x55bc859f5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:23:38.083 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.082+0000 7f133b67e740 1 -- 172.21.15.100:0/4274447113 shutdown_connections 2024-05-10T11:23:38.083 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.082+0000 7f133b67e740 1 -- 172.21.15.100:0/4274447113 wait complete. 2024-05-10T11:23:38.083 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.082+0000 7f133b67e740 20 client.25885 trim_cache size 0 max 16384 2024-05-10T11:23:38.083 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:23:38.083 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:23:38.083 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:23:38.083 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:23:38.084 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.082+0000 7f1338966640 1 -- 172.21.15.100:0/795466138 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f131c0b28e0 msgr2=0x7f131c0b4da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:38.084 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.082+0000 7f1338966640 1 --2- 172.21.15.100:0/795466138 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f131c0b28e0 0x7f131c0b4da0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x55bc859ecba0 tx=0x7f1324008040 comp rx=0 tx=0).stop 2024-05-10T11:23:38.084 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.082+0000 7f1338966640 1 -- 172.21.15.100:0/795466138 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1334102b00 msgr2=0x7f1334196f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:38.084 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.082+0000 7f1338966640 1 --2- 172.21.15.100:0/795466138 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1334102b00 0x7f1334196f50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1328009980 tx=0x7f1328032cb0 comp rx=0 tx=0).stop 2024-05-10T11:23:38.084 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.082+0000 7f1338966640 1 -- 172.21.15.100:0/795466138 shutdown_connections 2024-05-10T11:23:38.085 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.082+0000 7f1338966640 1 --2- 172.21.15.100:0/795466138 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f131c0b28e0 0x7f131c0b4da0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:38.085 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.082+0000 7f1338966640 1 --2- 172.21.15.100:0/795466138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13341034c0 0x7f13341af1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:38.085 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.082+0000 7f1338966640 1 --2- 172.21.15.100:0/795466138 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1334102b00 0x7f1334196f50 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:38.085 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.082+0000 7f1338966640 1 --2- 172.21.15.100:0/795466138 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1334101910 0x7f1334196a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:38.085 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.082+0000 7f1338966640 1 -- 172.21.15.100:0/795466138 >> 172.21.15.100:0/795466138 conn(0x7f13340fce80 msgr2=0x7f1334105e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:23:38.085 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.082+0000 7f1338966640 1 -- 172.21.15.100:0/795466138 shutdown_connections 2024-05-10T11:23:38.085 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:38.082+0000 7f1338966640 1 -- 172.21.15.100:0/795466138 wait complete. 2024-05-10T11:23:38.241 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:37 smithi100 ceph-mon[30296]: pgmap v577: 129 pgs: 129 active+clean; 59 MiB data, 999 MiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 1.5 MiB/s wr, 168 op/s 2024-05-10T11:23:38.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:37 smithi178 ceph-mon[33359]: pgmap v577: 129 pgs: 129 active+clean; 59 MiB data, 999 MiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 1.5 MiB/s wr, 168 op/s 2024-05-10T11:23:38.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:37 smithi188 ceph-mon[34075]: pgmap v577: 129 pgs: 129 active+clean; 59 MiB data, 999 MiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 1.5 MiB/s wr, 168 op/s 2024-05-10T11:23:38.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.822+0000 7f740e59b700 1 -- 172.21.15.100:0/2215432442 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7408109520 msgr2=0x7f7408109900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:38.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.822+0000 7f740e59b700 1 --2- 172.21.15.100:0/2215432442 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7408109520 0x7f7408109900 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f73f8009990 tx=0x7f73f8009ca0 comp rx=0 tx=0).stop 2024-05-10T11:23:38.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.822+0000 7f740e59b700 1 -- 172.21.15.100:0/2215432442 shutdown_connections 2024-05-10T11:23:38.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.822+0000 7f740e59b700 1 --2- 172.21.15.100:0/2215432442 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7408102450 0x7f7408110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:38.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.822+0000 7f740e59b700 1 --2- 172.21.15.100:0/2215432442 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7408109520 0x7f7408109900 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:38.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.822+0000 7f740e59b700 1 --2- 172.21.15.100:0/2215432442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7408101a90 0x7f7408101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:38.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.822+0000 7f740e59b700 1 -- 172.21.15.100:0/2215432442 >> 172.21.15.100:0/2215432442 conn(0x7f7408078a10 msgr2=0x7f74080ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:23:38.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.822+0000 7f740e59b700 1 -- 172.21.15.100:0/2215432442 shutdown_connections 2024-05-10T11:23:38.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.822+0000 7f740e59b700 1 -- 172.21.15.100:0/2215432442 wait complete. 2024-05-10T11:23:38.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.823+0000 7f740e59b700 1 Processor -- start 2024-05-10T11:23:38.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.823+0000 7f740e59b700 1 -- start start 2024-05-10T11:23:38.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.824+0000 7f740e59b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7408101a90 0x7f740810c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:38.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.824+0000 7f740e59b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7408102450 0x7f740810c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:38.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.824+0000 7f740e59b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7408109520 0x7f740810ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:38.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.824+0000 7f740e59b700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7408103700 con 0x7f7408101a90 2024-05-10T11:23:38.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.824+0000 7f740e59b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7408103580 con 0x7f7408102450 2024-05-10T11:23:38.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.824+0000 7f740e59b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7408103880 con 0x7f7408109520 2024-05-10T11:23:38.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.824+0000 7f7407fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7408101a90 0x7f740810c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:38.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.824+0000 7f7407fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7408101a90 0x7f740810c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47342/0 (socket says 172.21.15.100:47342) 2024-05-10T11:23:38.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.824+0000 7f7407fff700 1 -- 172.21.15.100:0/3923625972 learned_addr learned my addr 172.21.15.100:0/3923625972 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:23:38.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.824+0000 7f74077fe700 1 --2- 172.21.15.100:0/3923625972 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7408102450 0x7f740810c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:38.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.824+0000 7f740cb38700 1 --2- 172.21.15.100:0/3923625972 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7408109520 0x7f740810ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:38.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.824+0000 7f74077fe700 1 -- 172.21.15.100:0/3923625972 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7408109520 msgr2=0x7f740810ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:38.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.824+0000 7f74077fe700 1 --2- 172.21.15.100:0/3923625972 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7408109520 0x7f740810ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:38.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.824+0000 7f74077fe700 1 -- 172.21.15.100:0/3923625972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7408101a90 msgr2=0x7f740810c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:38.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.824+0000 7f74077fe700 1 --2- 172.21.15.100:0/3923625972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7408101a90 0x7f740810c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:38.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.824+0000 7f74077fe700 1 -- 172.21.15.100:0/3923625972 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74081ac300 con 0x7f7408102450 2024-05-10T11:23:38.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.824+0000 7f7407fff700 1 --2- 172.21.15.100:0/3923625972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7408101a90 0x7f740810c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:23:38.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.824+0000 7f74077fe700 1 --2- 172.21.15.100:0/3923625972 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7408102450 0x7f740810c950 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f73fc00d980 tx=0x7f73fc00dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:23:38.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.825+0000 7f74057fa700 1 -- 172.21.15.100:0/3923625972 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73fc017070 con 0x7f7408102450 2024-05-10T11:23:38.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.825+0000 7f74057fa700 1 -- 172.21.15.100:0/3923625972 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f73fc00fd20 con 0x7f7408102450 2024-05-10T11:23:38.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.825+0000 7f740e59b700 1 -- 172.21.15.100:0/3923625972 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74081ac5f0 con 0x7f7408102450 2024-05-10T11:23:38.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.825+0000 7f74057fa700 1 -- 172.21.15.100:0/3923625972 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73fc00e3b0 con 0x7f7408102450 2024-05-10T11:23:38.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.825+0000 7f740e59b700 1 -- 172.21.15.100:0/3923625972 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74081acb80 con 0x7f7408102450 2024-05-10T11:23:38.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.827+0000 7f74057fa700 1 -- 172.21.15.100:0/3923625972 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f73fc01a070 con 0x7f7408102450 2024-05-10T11:23:38.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.828+0000 7f74057fa700 1 --2- 172.21.15.100:0/3923625972 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f73f00b2900 0x7f73f00b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:38.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.828+0000 7f7407fff700 1 --2- 172.21.15.100:0/3923625972 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f73f00b2900 0x7f73f00b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:38.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.828+0000 7f74057fa700 1 -- 172.21.15.100:0/3923625972 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f73fc0e1ca0 con 0x7f7408102450 2024-05-10T11:23:38.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.829+0000 7f7407fff700 1 --2- 172.21.15.100:0/3923625972 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f73f00b2900 0x7f73f00b4dc0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f73f800b5c0 tx=0x7f73f801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:23:38.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.829+0000 7f7418e2eb80 1 Processor -- start 2024-05-10T11:23:38.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.829+0000 7f7418e2eb80 1 -- start start 2024-05-10T11:23:38.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.829+0000 7f7418e2eb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55614f06b1b0 0x55614f06b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:38.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.829+0000 7f7418e2eb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55614f06bad0 0x55614f065810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:38.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.829+0000 7f7418e2eb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55614f065d50 0x55614f066190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:38.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.829+0000 7f7418e2eb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55614efb7940 con 0x55614f06bad0 2024-05-10T11:23:38.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.829+0000 7f7418e2eb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55614ef1cda0 con 0x55614f06b1b0 2024-05-10T11:23:38.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.829+0000 7f7418e2eb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55614efb3900 con 0x55614f065d50 2024-05-10T11:23:38.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.829+0000 7f74077fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55614f06bad0 0x55614f065810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:38.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.829+0000 7f74077fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55614f06bad0 0x55614f065810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47358/0 (socket says 172.21.15.100:47358) 2024-05-10T11:23:38.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.829+0000 7f74077fe700 1 -- 172.21.15.100:0/1100474162 learned_addr learned my addr 172.21.15.100:0/1100474162 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:23:38.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.829+0000 7f7407fff700 1 --2- 172.21.15.100:0/1100474162 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55614f06b1b0 0x55614f06b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:38.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.830+0000 7f740cb38700 1 --2- 172.21.15.100:0/1100474162 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55614f065d50 0x55614f066190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:38.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.830+0000 7f74077fe700 1 -- 172.21.15.100:0/1100474162 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55614f065d50 msgr2=0x55614f066190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:38.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.830+0000 7f74077fe700 1 --2- 172.21.15.100:0/1100474162 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55614f065d50 0x55614f066190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:38.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.830+0000 7f74077fe700 1 -- 172.21.15.100:0/1100474162 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55614f06b1b0 msgr2=0x55614f06b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:38.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.830+0000 7f74077fe700 1 --2- 172.21.15.100:0/1100474162 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55614f06b1b0 0x55614f06b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:38.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.830+0000 7f74077fe700 1 -- 172.21.15.100:0/1100474162 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55614ef048d0 con 0x55614f06bad0 2024-05-10T11:23:38.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.830+0000 7f74077fe700 1 --2- 172.21.15.100:0/1100474162 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55614f06bad0 0x55614f065810 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f73fc0a8ef0 tx=0x7f73fc0a87a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=215283745b1f0102 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:23:38.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.830+0000 7f73ee7fc700 1 -- 172.21.15.100:0/1100474162 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73fc021070 con 0x55614f06bad0 2024-05-10T11:23:38.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.830+0000 7f73ee7fc700 1 -- 172.21.15.100:0/1100474162 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f73fc0a9660 con 0x55614f06bad0 2024-05-10T11:23:38.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.830+0000 7f73ee7fc700 1 -- 172.21.15.100:0/1100474162 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73fc0b1b90 con 0x55614f06bad0 2024-05-10T11:23:38.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.831+0000 7f7418e2eb80 1 -- 172.21.15.100:0/1100474162 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55614f06bad0 msgr2=0x55614f065810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:38.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.831+0000 7f7418e2eb80 1 --2- 172.21.15.100:0/1100474162 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55614f06bad0 0x55614f065810 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f73fc0a8ef0 tx=0x7f73fc0a87a0 comp rx=0 tx=0).stop 2024-05-10T11:23:38.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.831+0000 7f7418e2eb80 1 -- 172.21.15.100:0/1100474162 shutdown_connections 2024-05-10T11:23:38.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.831+0000 7f7418e2eb80 1 --2- 172.21.15.100:0/1100474162 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55614f065d50 0x55614f066190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:38.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.831+0000 7f7418e2eb80 1 --2- 172.21.15.100:0/1100474162 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55614f06b1b0 0x55614f06b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:38.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.831+0000 7f7418e2eb80 1 --2- 172.21.15.100:0/1100474162 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55614f06bad0 0x55614f065810 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:38.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.831+0000 7f7418e2eb80 1 -- 172.21.15.100:0/1100474162 >> 172.21.15.100:0/1100474162 conn(0x55614ef91b70 msgr2=0x55614f0602a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:23:38.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.831+0000 7f7418e2eb80 1 -- 172.21.15.100:0/1100474162 shutdown_connections 2024-05-10T11:23:38.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.831+0000 7f7418e2eb80 1 -- 172.21.15.100:0/1100474162 wait complete. 2024-05-10T11:23:38.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.832+0000 7f7418e2eb80 1 -- start start 2024-05-10T11:23:38.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.832+0000 7f7418e2eb80 1 Processor -- start 2024-05-10T11:23:38.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.832+0000 7f7418e2eb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55614f065d50 0x55614f0716b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:38.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.832+0000 7f7418e2eb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55614f06b1b0 0x55614f06ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:38.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.832+0000 7f7418e2eb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55614f06bad0 0x55614f06ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:38.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.832+0000 7f7418e2eb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55614f0723f0 con 0x55614f06b1b0 2024-05-10T11:23:38.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.832+0000 7f7418e2eb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55614ef70ed0 con 0x55614f065d50 2024-05-10T11:23:38.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.832+0000 7f7418e2eb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55614f072570 con 0x55614f06bad0 2024-05-10T11:23:38.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.832+0000 7f7418e2eb80 10 client.0 _ll_register_callbacks cb 0x55614ef4b5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:23:38.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.832+0000 7f74077fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55614f06b1b0 0x55614f06ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:38.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.832+0000 7f74077fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55614f06b1b0 0x55614f06ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47370/0 (socket says 172.21.15.100:47370) 2024-05-10T11:23:38.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.832+0000 7f74077fe700 1 -- 172.21.15.100:0/1980946307 learned_addr learned my addr 172.21.15.100:0/1980946307 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:23:38.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.832+0000 7f740cb38700 1 --2- 172.21.15.100:0/1980946307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55614f06bad0 0x55614f06ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:38.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.833+0000 7f7407fff700 1 --2- 172.21.15.100:0/1980946307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55614f065d50 0x55614f0716b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:38.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.833+0000 7f74077fe700 1 -- 172.21.15.100:0/1980946307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55614f06bad0 msgr2=0x55614f06ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:38.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.833+0000 7f74077fe700 1 --2- 172.21.15.100:0/1980946307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55614f06bad0 0x55614f06ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:38.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.833+0000 7f74077fe700 1 -- 172.21.15.100:0/1980946307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55614f065d50 msgr2=0x55614f0716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:38.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.833+0000 7f74077fe700 1 --2- 172.21.15.100:0/1980946307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55614f065d50 0x55614f0716b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:38.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.833+0000 7f74077fe700 1 -- 172.21.15.100:0/1980946307 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55614ef6e760 con 0x55614f06b1b0 2024-05-10T11:23:38.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.833+0000 7f74077fe700 1 --2- 172.21.15.100:0/1980946307 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55614f06b1b0 0x55614f06ea50 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f73fc0df7b0 tx=0x7f73fc0ad380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f8c73e65dede6c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:23:38.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.833+0000 7f73e77fe700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:23:38.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.833+0000 7f73e77fe700 1 -- 172.21.15.100:0/1980946307 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73fc0c3030 con 0x55614f06b1b0 2024-05-10T11:23:38.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.833+0000 7f73e77fe700 1 -- 172.21.15.100:0/1980946307 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f73fc0b1b40 con 0x55614f06b1b0 2024-05-10T11:23:38.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.833+0000 7f7418e2eb80 1 -- 172.21.15.100:0/1980946307 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73fc0a9660 con 0x55614f06b1b0 2024-05-10T11:23:38.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.833+0000 7f73e77fe700 1 -- 172.21.15.100:0/1980946307 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f73fc021040 con 0x55614f06b1b0 2024-05-10T11:23:38.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.833+0000 7f7418e2eb80 10 client.16404 fetch_fsmap learned FSMap version 31 2024-05-10T11:23:38.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.833+0000 7f7418e2eb80 1 -- 172.21.15.100:0/1980946307 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55614f07cc10 con 0x55614f06b1b0 2024-05-10T11:23:38.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.833+0000 7f73e77fe700 1 -- 172.21.15.100:0/1980946307 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f73fc0a9660 con 0x55614f06b1b0 2024-05-10T11:23:38.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.834+0000 7f73e77fe700 1 -- 172.21.15.100:0/1980946307 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f73fc0ac070 con 0x55614f06b1b0 2024-05-10T11:23:38.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.834+0000 7f7418e2eb80 10 client.16404 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:23:38.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.834+0000 7f7418e2eb80 10 client.16404 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-10T11:23:38.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.834+0000 7f7418e2eb80 20 client.16404 populate_metadata read hostname 'smithi100' 2024-05-10T11:23:38.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.834+0000 7f7418e2eb80 1 --2- 172.21.15.100:0/1980946307 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55614f07f0e0 0x55614f0814d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:38.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.834+0000 7f7418e2eb80 4 client.16404 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:23:38.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.834+0000 7f7418e2eb80 1 -- 172.21.15.100:0/1980946307 --> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55614efc5320 con 0x55614f07f0e0 2024-05-10T11:23:38.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.834+0000 7f7407fff700 1 --2- 172.21.15.100:0/1980946307 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55614f07f0e0 0x55614f0814d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:38.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.834+0000 7f7407fff700 1 --2- 172.21.15.100:0/1980946307 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55614f07f0e0 0x55614f0814d0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a0f5d8d2d50716a4 server_cookie=5135be957a10f19d in_seq=0 out_seq=0 2024-05-10T11:23:38.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.834+0000 7f73e77fe700 10 client.16404 ms_handle_connect on v2:172.21.15.188:6836/1943342620 2024-05-10T11:23:38.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.835+0000 7f73e77fe700 1 -- 172.21.15.100:0/1980946307 <== mds.2 v2:172.21.15.188:6836/1943342620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55614efc5320 con 0x55614f07f0e0 2024-05-10T11:23:38.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.835+0000 7f73e77fe700 10 client.16404 handle_command_reply: tid=0 2024-05-10T11:23:38.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.835+0000 7f7418e2eb80 1 client.16404 shutdown 2024-05-10T11:23:38.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.835+0000 7f7418e2eb80 1 -- 172.21.15.100:0/1980946307 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55614f06b1b0 msgr2=0x55614f06ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:38.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.835+0000 7f7418e2eb80 1 --2- 172.21.15.100:0/1980946307 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55614f06b1b0 0x55614f06ea50 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f73fc0df7b0 tx=0x7f73fc0ad380 comp rx=0 tx=0).stop 2024-05-10T11:23:38.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.835+0000 7f7418e2eb80 1 -- 172.21.15.100:0/1980946307 shutdown_connections 2024-05-10T11:23:38.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.835+0000 7f7418e2eb80 1 --2- 172.21.15.100:0/1980946307 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55614f07f0e0 0x55614f0814d0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:38.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.835+0000 7f7418e2eb80 1 --2- 172.21.15.100:0/1980946307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55614f06bad0 0x55614f06ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:38.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.835+0000 7f7418e2eb80 1 --2- 172.21.15.100:0/1980946307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55614f065d50 0x55614f0716b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:38.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.835+0000 7f7418e2eb80 1 --2- 172.21.15.100:0/1980946307 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55614f06b1b0 0x55614f06ea50 secure :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f73fc0df7b0 tx=0x7f73fc0ad380 comp rx=0 tx=0).stop 2024-05-10T11:23:38.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.835+0000 7f7418e2eb80 1 -- 172.21.15.100:0/1980946307 >> 172.21.15.100:0/1980946307 conn(0x55614ef91b70 msgr2=0x55614f05ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:23:38.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.836+0000 7f73e77fe700 0 client.16404 ms_handle_reset on v2:172.21.15.188:6836/1943342620 2024-05-10T11:23:38.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.836+0000 7f7418e2eb80 1 -- 172.21.15.100:0/1980946307 shutdown_connections 2024-05-10T11:23:38.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.836+0000 7f7418e2eb80 1 -- 172.21.15.100:0/1980946307 wait complete. 2024-05-10T11:23:38.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.836+0000 7f7418e2eb80 20 client.16404 trim_cache size 0 max 16384 2024-05-10T11:23:38.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.852+0000 7f7418e2eb80 1 Processor -- start 2024-05-10T11:23:38.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.852+0000 7f7418e2eb80 1 -- start start 2024-05-10T11:23:38.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.852+0000 7f7418e2eb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55614f065d50 0x55614f065810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:38.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.852+0000 7f7418e2eb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55614f06bad0 0x55614f05ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:38.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.853+0000 7f7418e2eb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55614f071060 0x55614f084110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:38.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.853+0000 7f7418e2eb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f74081b7fc0 con 0x55614f065d50 2024-05-10T11:23:38.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.853+0000 7f7418e2eb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f74081b5850 con 0x55614f06bad0 2024-05-10T11:23:38.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.853+0000 7f7418e2eb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f74081b36a0 con 0x55614f071060 2024-05-10T11:23:38.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.853+0000 7f7407fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55614f065d50 0x55614f065810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:38.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.853+0000 7f74077fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55614f06bad0 0x55614f05ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:38.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.853+0000 7f740cb38700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55614f071060 0x55614f084110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:38.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.853+0000 7f74077fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55614f06bad0 0x55614f05ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:45928/0 (socket says 172.21.15.100:45928) 2024-05-10T11:23:38.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.853+0000 7f74077fe700 1 -- 172.21.15.100:0/3396866547 learned_addr learned my addr 172.21.15.100:0/3396866547 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:23:38.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.853+0000 7f740cb38700 1 -- 172.21.15.100:0/3396866547 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55614f06bad0 msgr2=0x55614f05ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:38.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.853+0000 7f740cb38700 1 --2- 172.21.15.100:0/3396866547 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55614f06bad0 0x55614f05ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:38.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.853+0000 7f740cb38700 1 -- 172.21.15.100:0/3396866547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55614f065d50 msgr2=0x55614f065810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:38.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.853+0000 7f740cb38700 1 --2- 172.21.15.100:0/3396866547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55614f065d50 0x55614f065810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:38.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.853+0000 7f740cb38700 1 -- 172.21.15.100:0/3396866547 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55614f05cc80 con 0x55614f071060 2024-05-10T11:23:38.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.853+0000 7f7407fff700 1 --2- 172.21.15.100:0/3396866547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55614f065d50 0x55614f065810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:23:38.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.853+0000 7f740cb38700 1 --2- 172.21.15.100:0/3396866547 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55614f071060 0x55614f084110 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7400009e80 tx=0x7f7400009eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc1cc0bc16f400bb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:23:38.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.853+0000 7f73edffb700 1 -- 172.21.15.100:0/3396866547 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7400007c60 con 0x55614f071060 2024-05-10T11:23:38.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.853+0000 7f73edffb700 1 -- 172.21.15.100:0/3396866547 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7400007e00 con 0x55614f071060 2024-05-10T11:23:38.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.853+0000 7f73edffb700 1 -- 172.21.15.100:0/3396866547 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f740001f780 con 0x55614f071060 2024-05-10T11:23:38.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.854+0000 7f7418e2eb80 1 -- 172.21.15.100:0/3396866547 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55614f071060 msgr2=0x55614f084110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:38.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.854+0000 7f7418e2eb80 1 --2- 172.21.15.100:0/3396866547 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55614f071060 0x55614f084110 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7400009e80 tx=0x7f7400009eb0 comp rx=0 tx=0).stop 2024-05-10T11:23:38.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.854+0000 7f7418e2eb80 1 -- 172.21.15.100:0/3396866547 shutdown_connections 2024-05-10T11:23:38.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.854+0000 7f7418e2eb80 1 --2- 172.21.15.100:0/3396866547 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55614f071060 0x55614f084110 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:38.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.854+0000 7f7418e2eb80 1 --2- 172.21.15.100:0/3396866547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55614f065d50 0x55614f065810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:38.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.854+0000 7f7418e2eb80 1 --2- 172.21.15.100:0/3396866547 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55614f06bad0 0x55614f05ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:38.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.854+0000 7f7418e2eb80 1 -- 172.21.15.100:0/3396866547 >> 172.21.15.100:0/3396866547 conn(0x55614ef91b70 msgr2=0x55614f06d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:23:38.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.854+0000 7f7418e2eb80 1 -- 172.21.15.100:0/3396866547 shutdown_connections 2024-05-10T11:23:38.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.854+0000 7f7418e2eb80 1 -- 172.21.15.100:0/3396866547 wait complete. 2024-05-10T11:23:38.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.855+0000 7f7418e2eb80 1 -- start start 2024-05-10T11:23:38.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.855+0000 7f7418e2eb80 1 Processor -- start 2024-05-10T11:23:38.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.855+0000 7f7418e2eb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55614f065d50 0x55614f084540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:38.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.855+0000 7f7418e2eb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55614f06bad0 0x55614f0818e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:38.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.855+0000 7f7418e2eb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55614f071060 0x55614f081e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:38.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.855+0000 7f7418e2eb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f74081b3d10 con 0x55614f065d50 2024-05-10T11:23:38.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.855+0000 7f7418e2eb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f74081b5180 con 0x55614f06bad0 2024-05-10T11:23:38.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.855+0000 7f7418e2eb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f74081b43e0 con 0x55614f071060 2024-05-10T11:23:38.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.855+0000 7f7418e2eb80 10 client.0 _ll_register_callbacks cb 0x55614ef4b5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:23:38.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.855+0000 7f7407fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55614f065d50 0x55614f084540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:38.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.855+0000 7f7407fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55614f065d50 0x55614f084540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47376/0 (socket says 172.21.15.100:47376) 2024-05-10T11:23:38.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.855+0000 7f7407fff700 1 -- 172.21.15.100:0/784658628 learned_addr learned my addr 172.21.15.100:0/784658628 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:23:38.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.856+0000 7f74077fe700 1 --2- 172.21.15.100:0/784658628 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55614f06bad0 0x55614f0818e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:38.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.856+0000 7f740cb38700 1 --2- 172.21.15.100:0/784658628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55614f071060 0x55614f081e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:38.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.856+0000 7f7407fff700 1 -- 172.21.15.100:0/784658628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55614f071060 msgr2=0x55614f081e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:38.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.856+0000 7f7407fff700 1 --2- 172.21.15.100:0/784658628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55614f071060 0x55614f081e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:38.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.856+0000 7f7407fff700 1 -- 172.21.15.100:0/784658628 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55614f06bad0 msgr2=0x55614f0818e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:38.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.856+0000 7f7407fff700 1 --2- 172.21.15.100:0/784658628 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55614f06bad0 0x55614f0818e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:38.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.856+0000 7f7407fff700 1 -- 172.21.15.100:0/784658628 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55614f082520 con 0x55614f065d50 2024-05-10T11:23:38.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.856+0000 7f740cb38700 1 --2- 172.21.15.100:0/784658628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55614f071060 0x55614f081e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:23:38.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.856+0000 7f7407fff700 1 --2- 172.21.15.100:0/784658628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55614f065d50 0x55614f084540 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f73f80098e0 tx=0x7f73f8009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f38ae8f67b6353f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:23:38.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.856+0000 7f73e77fe700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:23:38.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.856+0000 7f73e77fe700 1 -- 172.21.15.100:0/784658628 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f8026e10 con 0x55614f065d50 2024-05-10T11:23:38.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.856+0000 7f73e77fe700 1 -- 172.21.15.100:0/784658628 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f73f8034400 con 0x55614f065d50 2024-05-10T11:23:38.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.857+0000 7f7418e2eb80 1 -- 172.21.15.100:0/784658628 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7400007e00 con 0x55614f065d50 2024-05-10T11:23:38.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.857+0000 7f73e77fe700 1 -- 172.21.15.100:0/784658628 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f73f8034600 con 0x55614f065d50 2024-05-10T11:23:38.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.857+0000 7f7418e2eb80 10 client.16410 fetch_fsmap learned FSMap version 31 2024-05-10T11:23:38.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.857+0000 7f7418e2eb80 1 -- 172.21.15.100:0/784658628 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55614f0e9570 con 0x55614f065d50 2024-05-10T11:23:38.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.857+0000 7f73e77fe700 1 -- 172.21.15.100:0/784658628 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7400007e00 con 0x55614f065d50 2024-05-10T11:23:38.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.857+0000 7f73e77fe700 1 -- 172.21.15.100:0/784658628 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f73f8025ad0 con 0x55614f065d50 2024-05-10T11:23:38.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.857+0000 7f7418e2eb80 10 client.16410 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:23:38.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.857+0000 7f7418e2eb80 10 client.16410 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-10T11:23:38.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.857+0000 7f7418e2eb80 20 client.16410 populate_metadata read hostname 'smithi100' 2024-05-10T11:23:38.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.857+0000 7f7418e2eb80 1 --2- 172.21.15.100:0/784658628 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55614f0e9c60 0x55614f0ec080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:38.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.857+0000 7f7418e2eb80 4 client.16410 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:23:38.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.857+0000 7f7418e2eb80 1 -- 172.21.15.100:0/784658628 --> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55614efbf3c0 con 0x55614f0e9c60 2024-05-10T11:23:38.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.858+0000 7f74077fe700 1 --2- 172.21.15.100:0/784658628 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55614f0e9c60 0x55614f0ec080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:38.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.858+0000 7f73e77fe700 10 client.16410 ms_handle_connect on v2:172.21.15.188:6836/1943342620 2024-05-10T11:23:38.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.858+0000 7f74077fe700 1 --2- 172.21.15.100:0/784658628 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55614f0e9c60 0x55614f0ec080 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5695e7f2bd07c4a8 server_cookie=4cccadad700b5207 in_seq=0 out_seq=0 2024-05-10T11:23:38.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.858+0000 7f73e77fe700 1 -- 172.21.15.100:0/784658628 <== mds.2 v2:172.21.15.188:6836/1943342620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55614efbf3c0 con 0x55614f0e9c60 2024-05-10T11:23:38.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.858+0000 7f73e77fe700 10 client.16410 handle_command_reply: tid=0 2024-05-10T11:23:38.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.858+0000 7f7418e2eb80 1 client.16410 shutdown 2024-05-10T11:23:38.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.858+0000 7f7418e2eb80 1 -- 172.21.15.100:0/784658628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55614f065d50 msgr2=0x55614f084540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:38.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.858+0000 7f7418e2eb80 1 --2- 172.21.15.100:0/784658628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55614f065d50 0x55614f084540 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f73f80098e0 tx=0x7f73f8009ca0 comp rx=0 tx=0).stop 2024-05-10T11:23:38.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.859+0000 7f7418e2eb80 1 -- 172.21.15.100:0/784658628 shutdown_connections 2024-05-10T11:23:38.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.859+0000 7f7418e2eb80 1 --2- 172.21.15.100:0/784658628 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55614f0e9c60 0x55614f0ec080 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:38.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.859+0000 7f7418e2eb80 1 --2- 172.21.15.100:0/784658628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55614f071060 0x55614f081e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:38.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.859+0000 7f73e77fe700 0 client.16410 ms_handle_reset on v2:172.21.15.188:6836/1943342620 2024-05-10T11:23:38.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.859+0000 7f7418e2eb80 1 --2- 172.21.15.100:0/784658628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55614f065d50 0x55614f084540 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:38.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.859+0000 7f7418e2eb80 1 --2- 172.21.15.100:0/784658628 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55614f06bad0 0x55614f0818e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:38.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.859+0000 7f7418e2eb80 1 -- 172.21.15.100:0/784658628 >> 172.21.15.100:0/784658628 conn(0x55614ef91b70 msgr2=0x55614f070050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:23:38.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.859+0000 7f7418e2eb80 1 -- 172.21.15.100:0/784658628 shutdown_connections 2024-05-10T11:23:38.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.859+0000 7f7418e2eb80 1 -- 172.21.15.100:0/784658628 wait complete. 2024-05-10T11:23:38.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.859+0000 7f7418e2eb80 20 client.16410 trim_cache size 0 max 16384 2024-05-10T11:23:38.863 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:23:38.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.860+0000 7f740e59b700 1 -- 172.21.15.100:0/3923625972 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f73f00b2900 msgr2=0x7f73f00b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:38.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.860+0000 7f740e59b700 1 --2- 172.21.15.100:0/3923625972 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f73f00b2900 0x7f73f00b4dc0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f73f800b5c0 tx=0x7f73f801a040 comp rx=0 tx=0).stop 2024-05-10T11:23:38.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.860+0000 7f740e59b700 1 -- 172.21.15.100:0/3923625972 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7408102450 msgr2=0x7f740810c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:38.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.860+0000 7f740e59b700 1 --2- 172.21.15.100:0/3923625972 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7408102450 0x7f740810c950 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f73fc00d980 tx=0x7f73fc00dc90 comp rx=0 tx=0).stop 2024-05-10T11:23:38.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.860+0000 7f740e59b700 1 -- 172.21.15.100:0/3923625972 shutdown_connections 2024-05-10T11:23:38.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.860+0000 7f740e59b700 1 --2- 172.21.15.100:0/3923625972 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f73f00b2900 0x7f73f00b4dc0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:38.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.860+0000 7f740e59b700 1 --2- 172.21.15.100:0/3923625972 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7408109520 0x7f740810ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:38.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.860+0000 7f740e59b700 1 --2- 172.21.15.100:0/3923625972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7408101a90 0x7f740810c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:38.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.860+0000 7f740e59b700 1 --2- 172.21.15.100:0/3923625972 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7408102450 0x7f740810c950 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:38.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.860+0000 7f740e59b700 1 -- 172.21.15.100:0/3923625972 >> 172.21.15.100:0/3923625972 conn(0x7f7408078a10 msgr2=0x7f74080ff580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:23:38.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.860+0000 7f740e59b700 1 -- 172.21.15.100:0/3923625972 shutdown_connections 2024-05-10T11:23:38.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:38.860+0000 7f740e59b700 1 -- 172.21.15.100:0/3923625972 wait complete. 2024-05-10T11:23:38.971 INFO:tasks.workunit.client.0.smithi100.stdout: 1 111795 2.74 MB/sec execute 93 sec latency 16.975 ms 2024-05-10T11:23:39.699 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-10T11:23:39.967 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:23:39.971 INFO:tasks.workunit.client.0.smithi100.stdout: 1 112372 2.73 MB/sec execute 94 sec latency 16.198 ms 2024-05-10T11:23:40.254 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:39 smithi178 ceph-mon[33359]: pgmap v578: 129 pgs: 129 active+clean; 60 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 1.4 MiB/s wr, 122 op/s 2024-05-10T11:23:40.261 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:39 smithi100 ceph-mon[30296]: pgmap v578: 129 pgs: 129 active+clean; 60 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 1.4 MiB/s wr, 122 op/s 2024-05-10T11:23:40.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:39 smithi188 ceph-mon[34075]: pgmap v578: 129 pgs: 129 active+clean; 60 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 1.4 MiB/s wr, 122 op/s 2024-05-10T11:23:40.971 INFO:tasks.workunit.client.0.smithi100.stdout: 1 112990 2.76 MB/sec execute 95 sec latency 14.948 ms 2024-05-10T11:23:41.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.853+0000 7f8ef7c33700 1 -- 172.21.15.100:0/750495616 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ef0102e00 msgr2=0x7f8ef01031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:41.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.853+0000 7f8ef7c33700 1 --2- 172.21.15.100:0/750495616 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ef0102e00 0x7f8ef01031e0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f8eec0099c0 tx=0x7f8eec009cd0 comp rx=0 tx=0).stop 2024-05-10T11:23:41.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.853+0000 7f8ef7c33700 1 -- 172.21.15.100:0/750495616 shutdown_connections 2024-05-10T11:23:41.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.853+0000 7f8ef7c33700 1 --2- 172.21.15.100:0/750495616 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ef0107760 0x7f8ef010f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:41.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.853+0000 7f8ef7c33700 1 --2- 172.21.15.100:0/750495616 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ef0102e00 0x7f8ef01031e0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:41.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.853+0000 7f8ef7c33700 1 --2- 172.21.15.100:0/750495616 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ef01037b0 0x7f8ef0107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:41.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.853+0000 7f8ef7c33700 1 -- 172.21.15.100:0/750495616 >> 172.21.15.100:0/750495616 conn(0x7f8ef0078a10 msgr2=0x7f8ef00ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:23:41.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.853+0000 7f8ef7c33700 1 -- 172.21.15.100:0/750495616 shutdown_connections 2024-05-10T11:23:41.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.853+0000 7f8ef7c33700 1 -- 172.21.15.100:0/750495616 wait complete. 2024-05-10T11:23:41.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.853+0000 7f8ef7c33700 1 Processor -- start 2024-05-10T11:23:41.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.854+0000 7f8ef7c33700 1 -- start start 2024-05-10T11:23:41.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.854+0000 7f8ef7c33700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ef0102e00 0x7f8ef019e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:41.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.854+0000 7f8ef7c33700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ef01037b0 0x7f8ef019ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:41.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.854+0000 7f8ef7c33700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ef0107760 0x7f8ef0198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:41.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.854+0000 7f8ef7c33700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef0112480 con 0x7f8ef0102e00 2024-05-10T11:23:41.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.854+0000 7f8ef7c33700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef0112300 con 0x7f8ef0107760 2024-05-10T11:23:41.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.854+0000 7f8ef7c33700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef0112600 con 0x7f8ef01037b0 2024-05-10T11:23:41.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.854+0000 7f8ef59cf700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ef0102e00 0x7f8ef019e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:41.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.854+0000 7f8ef59cf700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ef0102e00 0x7f8ef019e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47402/0 (socket says 172.21.15.100:47402) 2024-05-10T11:23:41.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.854+0000 7f8ef59cf700 1 -- 172.21.15.100:0/3101831135 learned_addr learned my addr 172.21.15.100:0/3101831135 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:23:41.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.854+0000 7f8ef51ce700 1 --2- 172.21.15.100:0/3101831135 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ef01037b0 0x7f8ef019ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:41.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.854+0000 7f8ef61d0700 1 --2- 172.21.15.100:0/3101831135 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ef0107760 0x7f8ef0198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:41.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.854+0000 7f8ef59cf700 1 -- 172.21.15.100:0/3101831135 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ef01037b0 msgr2=0x7f8ef019ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:41.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.854+0000 7f8ef59cf700 1 --2- 172.21.15.100:0/3101831135 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ef01037b0 0x7f8ef019ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:41.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.854+0000 7f8ef59cf700 1 -- 172.21.15.100:0/3101831135 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ef0107760 msgr2=0x7f8ef0198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:41.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.854+0000 7f8ef59cf700 1 --2- 172.21.15.100:0/3101831135 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ef0107760 0x7f8ef0198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:41.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.855+0000 7f8ef59cf700 1 -- 172.21.15.100:0/3101831135 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ef0198fe0 con 0x7f8ef0102e00 2024-05-10T11:23:41.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.855+0000 7f8ef61d0700 1 --2- 172.21.15.100:0/3101831135 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ef0107760 0x7f8ef0198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:23:41.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.855+0000 7f8ef59cf700 1 --2- 172.21.15.100:0/3101831135 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ef0102e00 0x7f8ef019e680 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f8eec006010 tx=0x7f8eec0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:23:41.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.855+0000 7f8ee6ffd700 1 -- 172.21.15.100:0/3101831135 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eec028070 con 0x7f8ef0102e00 2024-05-10T11:23:41.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.855+0000 7f8ee6ffd700 1 -- 172.21.15.100:0/3101831135 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8eec019600 con 0x7f8ef0102e00 2024-05-10T11:23:41.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.855+0000 7f8ee6ffd700 1 -- 172.21.15.100:0/3101831135 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eec009e60 con 0x7f8ef0102e00 2024-05-10T11:23:41.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.855+0000 7f8ef7c33700 1 -- 172.21.15.100:0/3101831135 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ef0199270 con 0x7f8ef0102e00 2024-05-10T11:23:41.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.855+0000 7f8ef7c33700 1 -- 172.21.15.100:0/3101831135 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ef0101a90 con 0x7f8ef0102e00 2024-05-10T11:23:41.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.858+0000 7f8ee6ffd700 1 -- 172.21.15.100:0/3101831135 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f8eec02b070 con 0x7f8ef0102e00 2024-05-10T11:23:41.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.859+0000 7f8ee6ffd700 1 --2- 172.21.15.100:0/3101831135 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8edc0b2900 0x7f8edc0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:41.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.859+0000 7f8ef51ce700 1 --2- 172.21.15.100:0/3101831135 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8edc0b2900 0x7f8edc0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:41.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.859+0000 7f8ee6ffd700 1 -- 172.21.15.100:0/3101831135 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f8eec0e4f80 con 0x7f8ef0102e00 2024-05-10T11:23:41.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.859+0000 7f8f024c6b80 1 Processor -- start 2024-05-10T11:23:41.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.859+0000 7f8ef51ce700 1 --2- 172.21.15.100:0/3101831135 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8edc0b2900 0x7f8edc0b4dc0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0104ff0 tx=0x7f8ee0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:23:41.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.859+0000 7f8f024c6b80 1 -- start start 2024-05-10T11:23:41.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.860+0000 7f8f024c6b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56205814d4d0 0x56205814d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:41.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.860+0000 7f8f024c6b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56205814ddf0 0x5620581516f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:41.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.860+0000 7f8f024c6b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562058151c80 0x562058152100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:41.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.860+0000 7f8f024c6b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56205809f860 con 0x56205814d4d0 2024-05-10T11:23:41.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.860+0000 7f8f024c6b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56205803e7c0 con 0x562058151c80 2024-05-10T11:23:41.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.860+0000 7f8f024c6b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56205809b820 con 0x56205814ddf0 2024-05-10T11:23:41.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.860+0000 7f8ef51ce700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56205814ddf0 0x5620581516f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:41.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.860+0000 7f8ef59cf700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56205814d4d0 0x56205814d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:41.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.860+0000 7f8ef51ce700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56205814ddf0 0x5620581516f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:47412/0 (socket says 172.21.15.100:47412) 2024-05-10T11:23:41.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.860+0000 7f8ef51ce700 1 -- 172.21.15.100:0/286762516 learned_addr learned my addr 172.21.15.100:0/286762516 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:23:41.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.860+0000 7f8ef61d0700 1 --2- 172.21.15.100:0/286762516 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562058151c80 0x562058152100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:41.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.860+0000 7f8ef51ce700 1 -- 172.21.15.100:0/286762516 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562058151c80 msgr2=0x562058152100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:41.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.860+0000 7f8ef51ce700 1 --2- 172.21.15.100:0/286762516 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562058151c80 0x562058152100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:41.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.860+0000 7f8ef51ce700 1 -- 172.21.15.100:0/286762516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56205814d4d0 msgr2=0x56205814d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:41.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.860+0000 7f8ef51ce700 1 --2- 172.21.15.100:0/286762516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56205814d4d0 0x56205814d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:41.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.860+0000 7f8ef51ce700 1 -- 172.21.15.100:0/286762516 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562057fec8d0 con 0x56205814ddf0 2024-05-10T11:23:41.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.860+0000 7f8ef51ce700 1 --2- 172.21.15.100:0/286762516 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56205814ddf0 0x5620581516f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0011da0 tx=0x7f8ee0014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a6ab0a0bb39ce4a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:23:41.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.861+0000 7f8edbfff700 1 -- 172.21.15.100:0/286762516 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee0004a30 con 0x56205814ddf0 2024-05-10T11:23:41.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.861+0000 7f8edbfff700 1 -- 172.21.15.100:0/286762516 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8ee00054b0 con 0x56205814ddf0 2024-05-10T11:23:41.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.861+0000 7f8edbfff700 1 -- 172.21.15.100:0/286762516 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee0003ae0 con 0x56205814ddf0 2024-05-10T11:23:41.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.861+0000 7f8f024c6b80 1 -- 172.21.15.100:0/286762516 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56205814ddf0 msgr2=0x5620581516f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:41.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.861+0000 7f8f024c6b80 1 --2- 172.21.15.100:0/286762516 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56205814ddf0 0x5620581516f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0011da0 tx=0x7f8ee0014000 comp rx=0 tx=0).stop 2024-05-10T11:23:41.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.861+0000 7f8f024c6b80 1 -- 172.21.15.100:0/286762516 shutdown_connections 2024-05-10T11:23:41.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.861+0000 7f8f024c6b80 1 --2- 172.21.15.100:0/286762516 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562058151c80 0x562058152100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:41.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.861+0000 7f8f024c6b80 1 --2- 172.21.15.100:0/286762516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56205814d4d0 0x56205814d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:41.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.861+0000 7f8f024c6b80 1 --2- 172.21.15.100:0/286762516 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56205814ddf0 0x5620581516f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:41.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.861+0000 7f8f024c6b80 1 -- 172.21.15.100:0/286762516 >> 172.21.15.100:0/286762516 conn(0x562058079100 msgr2=0x562058147c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:23:41.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.862+0000 7f8f024c6b80 1 -- 172.21.15.100:0/286762516 shutdown_connections 2024-05-10T11:23:41.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.862+0000 7f8f024c6b80 1 -- 172.21.15.100:0/286762516 wait complete. 2024-05-10T11:23:41.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.862+0000 7f8f024c6b80 1 -- start start 2024-05-10T11:23:41.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.862+0000 7f8f024c6b80 1 Processor -- start 2024-05-10T11:23:41.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.863+0000 7f8f024c6b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56205814d4d0 0x562058158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:41.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.863+0000 7f8f024c6b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56205814ddf0 0x562058156360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:41.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.863+0000 7f8f024c6b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562058151c80 0x5620581568a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:41.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.863+0000 7f8f024c6b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56205815e2a0 con 0x56205814ddf0 2024-05-10T11:23:41.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.863+0000 7f8f024c6b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562058005c30 con 0x562058151c80 2024-05-10T11:23:41.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.863+0000 7f8f024c6b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56205815e420 con 0x56205814d4d0 2024-05-10T11:23:41.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.863+0000 7f8f024c6b80 10 client.0 _ll_register_callbacks cb 0x562058037920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:23:41.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.863+0000 7f8ef59cf700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56205814d4d0 0x562058158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:41.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.863+0000 7f8ef59cf700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56205814d4d0 0x562058158fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:47414/0 (socket says 172.21.15.100:47414) 2024-05-10T11:23:41.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.863+0000 7f8ef59cf700 1 -- 172.21.15.100:0/3307195790 learned_addr learned my addr 172.21.15.100:0/3307195790 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:23:41.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.863+0000 7f8ef61d0700 1 --2- 172.21.15.100:0/3307195790 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562058151c80 0x5620581568a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:41.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.863+0000 7f8ef51ce700 1 --2- 172.21.15.100:0/3307195790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56205814ddf0 0x562058156360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:41.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.864+0000 7f8ef59cf700 1 -- 172.21.15.100:0/3307195790 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562058151c80 msgr2=0x5620581568a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:41.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.864+0000 7f8ef59cf700 1 --2- 172.21.15.100:0/3307195790 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562058151c80 0x5620581568a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:41.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.864+0000 7f8ef59cf700 1 -- 172.21.15.100:0/3307195790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56205814ddf0 msgr2=0x562058156360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:41.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.864+0000 7f8ef59cf700 1 --2- 172.21.15.100:0/3307195790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56205814ddf0 0x562058156360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:41.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.864+0000 7f8ef59cf700 1 -- 172.21.15.100:0/3307195790 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56205803b690 con 0x56205814d4d0 2024-05-10T11:23:41.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.864+0000 7f8ef59cf700 1 --2- 172.21.15.100:0/3307195790 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56205814d4d0 0x562058158fc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8eec0e2a90 tx=0x7f8eec019d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1ddcb78093e9009 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:23:41.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.864+0000 7f8ed97fa700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:23:41.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.864+0000 7f8ed97fa700 1 -- 172.21.15.100:0/3307195790 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eec0ac2d0 con 0x56205814d4d0 2024-05-10T11:23:41.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.864+0000 7f8ed97fa700 1 -- 172.21.15.100:0/3307195790 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f8eec0e2be0 con 0x56205814d4d0 2024-05-10T11:23:41.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.864+0000 7f8f024c6b80 1 -- 172.21.15.100:0/3307195790 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ee00054b0 con 0x56205814d4d0 2024-05-10T11:23:41.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.864+0000 7f8ed97fa700 1 -- 172.21.15.100:0/3307195790 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8eec0b0680 con 0x56205814d4d0 2024-05-10T11:23:41.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.864+0000 7f8f024c6b80 10 client.25912 fetch_fsmap learned FSMap version 31 2024-05-10T11:23:41.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.864+0000 7f8f024c6b80 1 -- 172.21.15.100:0/3307195790 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56205815e2a0 con 0x56205814d4d0 2024-05-10T11:23:41.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.864+0000 7f8ed97fa700 1 -- 172.21.15.100:0/3307195790 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8ee00054b0 con 0x56205814d4d0 2024-05-10T11:23:41.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.864+0000 7f8ed97fa700 1 -- 172.21.15.100:0/3307195790 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f8eec0ab530 con 0x56205814d4d0 2024-05-10T11:23:41.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.865+0000 7f8f024c6b80 10 client.25912 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:23:41.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.865+0000 7f8f024c6b80 10 client.25912 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-10T11:23:41.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.865+0000 7f8f024c6b80 20 client.25912 populate_metadata read hostname 'smithi100' 2024-05-10T11:23:41.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.865+0000 7f8f024c6b80 1 --2- 172.21.15.100:0/3307195790 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x56205816aee0 0x56205816d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:41.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.865+0000 7f8f024c6b80 4 client.25912 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:23:41.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.865+0000 7f8f024c6b80 1 -- 172.21.15.100:0/3307195790 --> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562058119650 con 0x56205816aee0 2024-05-10T11:23:41.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.865+0000 7f8ef51ce700 1 --2- 172.21.15.100:0/3307195790 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x56205816aee0 0x56205816d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:41.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.865+0000 7f8ed97fa700 10 client.25912 ms_handle_connect on v2:172.21.15.178:6838/3760985404 2024-05-10T11:23:41.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.865+0000 7f8ef51ce700 1 --2- 172.21.15.100:0/3307195790 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x56205816aee0 0x56205816d310 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d15545f025736e96 server_cookie=55d0c823a4c56362 in_seq=0 out_seq=0 2024-05-10T11:23:41.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.866+0000 7f8ed97fa700 1 -- 172.21.15.100:0/3307195790 <== mds.1 v2:172.21.15.178:6838/3760985404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562058119650 con 0x56205816aee0 2024-05-10T11:23:41.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.866+0000 7f8ed97fa700 10 client.25912 handle_command_reply: tid=0 2024-05-10T11:23:41.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.866+0000 7f8f024c6b80 1 client.25912 shutdown 2024-05-10T11:23:41.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.866+0000 7f8f024c6b80 1 -- 172.21.15.100:0/3307195790 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56205814d4d0 msgr2=0x562058158fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:41.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.866+0000 7f8f024c6b80 1 --2- 172.21.15.100:0/3307195790 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56205814d4d0 0x562058158fc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8eec0e2a90 tx=0x7f8eec019d40 comp rx=0 tx=0).stop 2024-05-10T11:23:41.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.866+0000 7f8f024c6b80 1 -- 172.21.15.100:0/3307195790 shutdown_connections 2024-05-10T11:23:41.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.866+0000 7f8f024c6b80 1 --2- 172.21.15.100:0/3307195790 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x56205816aee0 0x56205816d310 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:41.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.866+0000 7f8f024c6b80 1 --2- 172.21.15.100:0/3307195790 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562058151c80 0x5620581568a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:41.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.866+0000 7f8ed97fa700 0 client.25912 ms_handle_reset on v2:172.21.15.178:6838/3760985404 2024-05-10T11:23:41.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.866+0000 7f8f024c6b80 1 --2- 172.21.15.100:0/3307195790 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56205814d4d0 0x562058158fc0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:41.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.866+0000 7f8f024c6b80 1 --2- 172.21.15.100:0/3307195790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56205814ddf0 0x562058156360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:41.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.866+0000 7f8f024c6b80 1 -- 172.21.15.100:0/3307195790 >> 172.21.15.100:0/3307195790 conn(0x562058079100 msgr2=0x562058142720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:23:41.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.867+0000 7f8f024c6b80 1 -- 172.21.15.100:0/3307195790 shutdown_connections 2024-05-10T11:23:41.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.867+0000 7f8f024c6b80 1 -- 172.21.15.100:0/3307195790 wait complete. 2024-05-10T11:23:41.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.867+0000 7f8f024c6b80 20 client.25912 trim_cache size 0 max 16384 2024-05-10T11:23:41.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.883+0000 7f8f024c6b80 1 Processor -- start 2024-05-10T11:23:41.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.883+0000 7f8f024c6b80 1 -- start start 2024-05-10T11:23:41.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.883+0000 7f8f024c6b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56205814d4d0 0x5620581552c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:41.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.883+0000 7f8f024c6b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56205814ddf0 0x56205814f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:41.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.883+0000 7f8f024c6b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562058151c80 0x56205814fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:41.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.884+0000 7f8f024c6b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef01af6f0 con 0x56205814ddf0 2024-05-10T11:23:41.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.884+0000 7f8f024c6b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef01acf80 con 0x562058151c80 2024-05-10T11:23:41.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.884+0000 7f8f024c6b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef01aadd0 con 0x56205814d4d0 2024-05-10T11:23:41.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.884+0000 7f8ef59cf700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56205814d4d0 0x5620581552c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:41.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.884+0000 7f8ef61d0700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562058151c80 0x56205814fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:41.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.884+0000 7f8ef59cf700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56205814d4d0 0x5620581552c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:47424/0 (socket says 172.21.15.100:47424) 2024-05-10T11:23:41.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.884+0000 7f8ef59cf700 1 -- 172.21.15.100:0/2048263052 learned_addr learned my addr 172.21.15.100:0/2048263052 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:23:41.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.884+0000 7f8ef51ce700 1 --2- 172.21.15.100:0/2048263052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56205814ddf0 0x56205814f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:41.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.884+0000 7f8ef59cf700 1 -- 172.21.15.100:0/2048263052 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562058151c80 msgr2=0x56205814fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:41.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.884+0000 7f8ef59cf700 1 --2- 172.21.15.100:0/2048263052 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562058151c80 0x56205814fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:41.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.884+0000 7f8ef59cf700 1 -- 172.21.15.100:0/2048263052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56205814ddf0 msgr2=0x56205814f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:41.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.884+0000 7f8ef59cf700 1 --2- 172.21.15.100:0/2048263052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56205814ddf0 0x56205814f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:41.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.884+0000 7f8ef59cf700 1 -- 172.21.15.100:0/2048263052 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562058148e20 con 0x56205814d4d0 2024-05-10T11:23:41.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.884+0000 7f8ef59cf700 1 --2- 172.21.15.100:0/2048263052 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56205814d4d0 0x5620581552c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8eec0ac260 tx=0x7f8eec0ac100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ff051ea5564685a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:23:41.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.884+0000 7f8edb7fe700 1 -- 172.21.15.100:0/2048263052 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eec0b50a0 con 0x56205814d4d0 2024-05-10T11:23:41.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.885+0000 7f8edb7fe700 1 -- 172.21.15.100:0/2048263052 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8eec0ab530 con 0x56205814d4d0 2024-05-10T11:23:41.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.885+0000 7f8edb7fe700 1 -- 172.21.15.100:0/2048263052 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eec0b4c10 con 0x56205814d4d0 2024-05-10T11:23:41.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.885+0000 7f8f024c6b80 1 -- 172.21.15.100:0/2048263052 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56205814d4d0 msgr2=0x5620581552c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:41.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.885+0000 7f8f024c6b80 1 --2- 172.21.15.100:0/2048263052 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56205814d4d0 0x5620581552c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8eec0ac260 tx=0x7f8eec0ac100 comp rx=0 tx=0).stop 2024-05-10T11:23:41.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.885+0000 7f8f024c6b80 1 -- 172.21.15.100:0/2048263052 shutdown_connections 2024-05-10T11:23:41.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.885+0000 7f8f024c6b80 1 --2- 172.21.15.100:0/2048263052 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562058151c80 0x56205814fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:41.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.885+0000 7f8f024c6b80 1 --2- 172.21.15.100:0/2048263052 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56205814d4d0 0x5620581552c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:41.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.885+0000 7f8f024c6b80 1 --2- 172.21.15.100:0/2048263052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56205814ddf0 0x56205814f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:41.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.885+0000 7f8f024c6b80 1 -- 172.21.15.100:0/2048263052 >> 172.21.15.100:0/2048263052 conn(0x562058079100 msgr2=0x562058142720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:23:41.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.885+0000 7f8f024c6b80 1 -- 172.21.15.100:0/2048263052 shutdown_connections 2024-05-10T11:23:41.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.885+0000 7f8f024c6b80 1 -- 172.21.15.100:0/2048263052 wait complete. 2024-05-10T11:23:41.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.886+0000 7f8f024c6b80 1 -- start start 2024-05-10T11:23:41.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.886+0000 7f8f024c6b80 1 Processor -- start 2024-05-10T11:23:41.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.886+0000 7f8f024c6b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56205814d4d0 0x562058163a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:41.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.886+0000 7f8f024c6b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56205814ddf0 0x562058172010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:41.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.886+0000 7f8f024c6b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562058151c80 0x562058172550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:41.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.886+0000 7f8f024c6b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef01ab440 con 0x56205814ddf0 2024-05-10T11:23:41.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.886+0000 7f8f024c6b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef01ac8b0 con 0x56205814d4d0 2024-05-10T11:23:41.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.886+0000 7f8f024c6b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef01abb10 con 0x562058151c80 2024-05-10T11:23:41.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.886+0000 7f8f024c6b80 10 client.0 _ll_register_callbacks cb 0x562058037920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:23:41.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.887+0000 7f8ef51ce700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56205814ddf0 0x562058172010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:41.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.887+0000 7f8ef51ce700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56205814ddf0 0x562058172010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47438/0 (socket says 172.21.15.100:47438) 2024-05-10T11:23:41.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.887+0000 7f8ef51ce700 1 -- 172.21.15.100:0/3797110980 learned_addr learned my addr 172.21.15.100:0/3797110980 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:23:41.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.887+0000 7f8ef61d0700 1 --2- 172.21.15.100:0/3797110980 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562058151c80 0x562058172550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:41.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.887+0000 7f8ef59cf700 1 --2- 172.21.15.100:0/3797110980 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56205814d4d0 0x562058163a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:41.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.887+0000 7f8ef51ce700 1 -- 172.21.15.100:0/3797110980 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562058151c80 msgr2=0x562058172550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:41.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.887+0000 7f8ef51ce700 1 --2- 172.21.15.100:0/3797110980 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562058151c80 0x562058172550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:41.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.887+0000 7f8ef51ce700 1 -- 172.21.15.100:0/3797110980 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56205814d4d0 msgr2=0x562058163a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:41.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.887+0000 7f8ef51ce700 1 --2- 172.21.15.100:0/3797110980 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56205814d4d0 0x562058163a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:41.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.887+0000 7f8ef51ce700 1 -- 172.21.15.100:0/3797110980 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562058172bd0 con 0x56205814ddf0 2024-05-10T11:23:41.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.887+0000 7f8ef59cf700 1 --2- 172.21.15.100:0/3797110980 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56205814d4d0 0x562058163a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:23:41.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.887+0000 7f8ef61d0700 1 --2- 172.21.15.100:0/3797110980 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562058151c80 0x562058172550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:23:41.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.887+0000 7f8ef51ce700 1 --2- 172.21.15.100:0/3797110980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56205814ddf0 0x562058172010 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0004600 tx=0x7f8ee0005ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4eb4a48b3099adf server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:23:41.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.887+0000 7f8ed97fa700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:23:41.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.887+0000 7f8ed97fa700 1 -- 172.21.15.100:0/3797110980 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee0037070 con 0x56205814ddf0 2024-05-10T11:23:41.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.887+0000 7f8ed97fa700 1 -- 172.21.15.100:0/3797110980 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f8ee0004b00 con 0x56205814ddf0 2024-05-10T11:23:41.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.887+0000 7f8f024c6b80 1 -- 172.21.15.100:0/3797110980 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8eec0ab530 con 0x56205814ddf0 2024-05-10T11:23:41.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.887+0000 7f8ed97fa700 1 -- 172.21.15.100:0/3797110980 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8ee0043050 con 0x56205814ddf0 2024-05-10T11:23:41.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.887+0000 7f8f024c6b80 10 client.16428 fetch_fsmap learned FSMap version 31 2024-05-10T11:23:41.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.887+0000 7f8f024c6b80 1 -- 172.21.15.100:0/3797110980 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5620581d5660 con 0x56205814ddf0 2024-05-10T11:23:41.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.888+0000 7f8ed97fa700 1 -- 172.21.15.100:0/3797110980 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8eec0ab530 con 0x56205814ddf0 2024-05-10T11:23:41.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.888+0000 7f8ed97fa700 1 -- 172.21.15.100:0/3797110980 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f8ee0005140 con 0x56205814ddf0 2024-05-10T11:23:41.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.888+0000 7f8f024c6b80 10 client.16428 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:23:41.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.888+0000 7f8f024c6b80 10 client.16428 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-10T11:23:41.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.888+0000 7f8f024c6b80 20 client.16428 populate_metadata read hostname 'smithi100' 2024-05-10T11:23:41.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.888+0000 7f8f024c6b80 1 --2- 172.21.15.100:0/3797110980 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x5620581d5a90 0x5620581d7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:41.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.888+0000 7f8f024c6b80 4 client.16428 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:23:41.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.888+0000 7f8f024c6b80 1 -- 172.21.15.100:0/3797110980 --> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5620580ad240 con 0x5620581d5a90 2024-05-10T11:23:41.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.888+0000 7f8ef59cf700 1 --2- 172.21.15.100:0/3797110980 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x5620581d5a90 0x5620581d7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:41.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.889+0000 7f8ef59cf700 1 --2- 172.21.15.100:0/3797110980 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x5620581d5a90 0x5620581d7e80 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dec08c0e6b575cab server_cookie=28191444ec9fd760 in_seq=0 out_seq=0 2024-05-10T11:23:41.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.889+0000 7f8ed97fa700 10 client.16428 ms_handle_connect on v2:172.21.15.178:6838/3760985404 2024-05-10T11:23:41.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.889+0000 7f8ed97fa700 1 -- 172.21.15.100:0/3797110980 <== mds.1 v2:172.21.15.178:6838/3760985404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5620580ad240 con 0x5620581d5a90 2024-05-10T11:23:41.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.889+0000 7f8ed97fa700 10 client.16428 handle_command_reply: tid=0 2024-05-10T11:23:41.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.889+0000 7f8f024c6b80 1 client.16428 shutdown 2024-05-10T11:23:41.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.889+0000 7f8f024c6b80 1 -- 172.21.15.100:0/3797110980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56205814ddf0 msgr2=0x562058172010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:41.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.889+0000 7f8f024c6b80 1 --2- 172.21.15.100:0/3797110980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56205814ddf0 0x562058172010 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0004600 tx=0x7f8ee0005ca0 comp rx=0 tx=0).stop 2024-05-10T11:23:41.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.889+0000 7f8f024c6b80 1 -- 172.21.15.100:0/3797110980 shutdown_connections 2024-05-10T11:23:41.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.889+0000 7f8f024c6b80 1 --2- 172.21.15.100:0/3797110980 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x5620581d5a90 0x5620581d7e80 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:41.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.889+0000 7f8f024c6b80 1 --2- 172.21.15.100:0/3797110980 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562058151c80 0x562058172550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:41.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.889+0000 7f8f024c6b80 1 --2- 172.21.15.100:0/3797110980 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56205814d4d0 0x562058163a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:41.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.889+0000 7f8f024c6b80 1 --2- 172.21.15.100:0/3797110980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56205814ddf0 0x562058172010 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:41.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.889+0000 7f8f024c6b80 1 -- 172.21.15.100:0/3797110980 >> 172.21.15.100:0/3797110980 conn(0x562058079100 msgr2=0x56205814c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:23:41.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.889+0000 7f8ed97fa700 0 client.16428 ms_handle_reset on v2:172.21.15.178:6838/3760985404 2024-05-10T11:23:41.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.889+0000 7f8f024c6b80 1 -- 172.21.15.100:0/3797110980 shutdown_connections 2024-05-10T11:23:41.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.889+0000 7f8f024c6b80 1 -- 172.21.15.100:0/3797110980 wait complete. 2024-05-10T11:23:41.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.890+0000 7f8f024c6b80 20 client.16428 trim_cache size 0 max 16384 2024-05-10T11:23:41.896 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:23:41.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.890+0000 7f8ef7c33700 1 -- 172.21.15.100:0/3101831135 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8edc0b2900 msgr2=0x7f8edc0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:41.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.890+0000 7f8ef7c33700 1 --2- 172.21.15.100:0/3101831135 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8edc0b2900 0x7f8edc0b4dc0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0104ff0 tx=0x7f8ee0009260 comp rx=0 tx=0).stop 2024-05-10T11:23:41.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.890+0000 7f8ef7c33700 1 -- 172.21.15.100:0/3101831135 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ef0102e00 msgr2=0x7f8ef019e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:41.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.890+0000 7f8ef7c33700 1 --2- 172.21.15.100:0/3101831135 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ef0102e00 0x7f8ef019e680 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f8eec006010 tx=0x7f8eec0045e0 comp rx=0 tx=0).stop 2024-05-10T11:23:41.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.890+0000 7f8ef7c33700 1 -- 172.21.15.100:0/3101831135 shutdown_connections 2024-05-10T11:23:41.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.890+0000 7f8ef7c33700 1 --2- 172.21.15.100:0/3101831135 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8edc0b2900 0x7f8edc0b4dc0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:41.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.890+0000 7f8ef7c33700 1 --2- 172.21.15.100:0/3101831135 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ef0107760 0x7f8ef0198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:41.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.890+0000 7f8ef7c33700 1 --2- 172.21.15.100:0/3101831135 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ef0102e00 0x7f8ef019e680 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:41.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.890+0000 7f8ef7c33700 1 --2- 172.21.15.100:0/3101831135 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ef01037b0 0x7f8ef019ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:41.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.890+0000 7f8ef7c33700 1 -- 172.21.15.100:0/3101831135 >> 172.21.15.100:0/3101831135 conn(0x7f8ef0078a10 msgr2=0x7f8ef00ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:23:41.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.890+0000 7f8ef7c33700 1 -- 172.21.15.100:0/3101831135 shutdown_connections 2024-05-10T11:23:41.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:41.890+0000 7f8ef7c33700 1 -- 172.21.15.100:0/3101831135 wait complete. 2024-05-10T11:23:41.971 INFO:tasks.workunit.client.0.smithi100.stdout: 1 113524 2.78 MB/sec execute 96 sec latency 19.671 ms 2024-05-10T11:23:42.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:41 smithi178 ceph-mon[33359]: pgmap v579: 129 pgs: 129 active+clean; 59 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 1.1 MiB/s wr, 141 op/s 2024-05-10T11:23:42.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:41 smithi188 ceph-mon[34075]: pgmap v579: 129 pgs: 129 active+clean; 59 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 1.1 MiB/s wr, 141 op/s 2024-05-10T11:23:42.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:41 smithi100 ceph-mon[30296]: pgmap v579: 129 pgs: 129 active+clean; 59 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 1.1 MiB/s wr, 141 op/s 2024-05-10T11:23:42.639 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-10T11:23:42.920 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:23:42.971 INFO:tasks.workunit.client.0.smithi100.stdout: 1 113882 2.77 MB/sec execute 97 sec latency 13.280 ms 2024-05-10T11:23:43.971 INFO:tasks.workunit.client.0.smithi100.stdout: 1 114153 2.74 MB/sec execute 98 sec latency 13.496 ms 2024-05-10T11:23:44.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:43 smithi100 ceph-mon[30296]: pgmap v580: 129 pgs: 129 active+clean; 55 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 1.2 MiB/s wr, 146 op/s 2024-05-10T11:23:44.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:43 smithi178 ceph-mon[33359]: pgmap v580: 129 pgs: 129 active+clean; 55 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 1.2 MiB/s wr, 146 op/s 2024-05-10T11:23:44.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:43 smithi188 ceph-mon[34075]: pgmap v580: 129 pgs: 129 active+clean; 55 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 1.2 MiB/s wr, 146 op/s 2024-05-10T11:23:44.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.689+0000 7f538252b700 1 -- 172.21.15.100:0/1222619993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f537c108400 msgr2=0x7f537c1087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:44.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.689+0000 7f538252b700 1 --2- 172.21.15.100:0/1222619993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f537c108400 0x7f537c1087e0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f536c009990 tx=0x7f536c009ca0 comp rx=0 tx=0).stop 2024-05-10T11:23:44.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.689+0000 7f538252b700 1 -- 172.21.15.100:0/1222619993 shutdown_connections 2024-05-10T11:23:44.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.689+0000 7f538252b700 1 --2- 172.21.15.100:0/1222619993 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f537c102d50 0x7f537c10f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:44.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.689+0000 7f538252b700 1 --2- 172.21.15.100:0/1222619993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f537c108400 0x7f537c1087e0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:44.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.689+0000 7f538252b700 1 --2- 172.21.15.100:0/1222619993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f537c102390 0x7f537c102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:44.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.689+0000 7f538252b700 1 -- 172.21.15.100:0/1222619993 >> 172.21.15.100:0/1222619993 conn(0x7f537c079310 msgr2=0x7f537c079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:23:44.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.689+0000 7f538252b700 1 -- 172.21.15.100:0/1222619993 shutdown_connections 2024-05-10T11:23:44.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.689+0000 7f538252b700 1 -- 172.21.15.100:0/1222619993 wait complete. 2024-05-10T11:23:44.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.690+0000 7f538252b700 1 Processor -- start 2024-05-10T11:23:44.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.690+0000 7f538252b700 1 -- start start 2024-05-10T11:23:44.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.690+0000 7f538252b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f537c102390 0x7f537c1a5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:44.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.690+0000 7f538252b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f537c102d50 0x7f537c1a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:44.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.690+0000 7f538252b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f537c108400 0x7f537c1a00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:44.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.690+0000 7f538252b700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f537c1123d0 con 0x7f537c102d50 2024-05-10T11:23:44.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.690+0000 7f538252b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f537c112250 con 0x7f537c102390 2024-05-10T11:23:44.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.690+0000 7f538252b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f537c112550 con 0x7f537c108400 2024-05-10T11:23:44.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.690+0000 7f5373fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f537c102d50 0x7f537c1a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:44.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.690+0000 7f5373fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f537c102d50 0x7f537c1a6250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47450/0 (socket says 172.21.15.100:47450) 2024-05-10T11:23:44.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.690+0000 7f5373fff700 1 -- 172.21.15.100:0/321095158 learned_addr learned my addr 172.21.15.100:0/321095158 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:23:44.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.690+0000 7f537bfff700 1 --2- 172.21.15.100:0/321095158 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f537c102390 0x7f537c1a5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:44.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.690+0000 7f5380ac8700 1 --2- 172.21.15.100:0/321095158 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f537c108400 0x7f537c1a00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:44.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.690+0000 7f5373fff700 1 -- 172.21.15.100:0/321095158 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f537c108400 msgr2=0x7f537c1a00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:44.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.690+0000 7f5373fff700 1 --2- 172.21.15.100:0/321095158 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f537c108400 0x7f537c1a00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:44.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.690+0000 7f5373fff700 1 -- 172.21.15.100:0/321095158 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f537c102390 msgr2=0x7f537c1a5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:44.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.690+0000 7f5373fff700 1 --2- 172.21.15.100:0/321095158 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f537c102390 0x7f537c1a5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:44.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.691+0000 7f5373fff700 1 -- 172.21.15.100:0/321095158 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f537c1a0880 con 0x7f537c102d50 2024-05-10T11:23:44.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.691+0000 7f5373fff700 1 --2- 172.21.15.100:0/321095158 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f537c102d50 0x7f537c1a6250 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f536400e9a0 tx=0x7f536400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:23:44.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.691+0000 7f5379ffb700 1 -- 172.21.15.100:0/321095158 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536400cbd0 con 0x7f537c102d50 2024-05-10T11:23:44.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.691+0000 7f538252b700 1 -- 172.21.15.100:0/321095158 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f537c1a0b70 con 0x7f537c102d50 2024-05-10T11:23:44.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.691+0000 7f5379ffb700 1 -- 172.21.15.100:0/321095158 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f536400cd70 con 0x7f537c102d50 2024-05-10T11:23:44.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.691+0000 7f5379ffb700 1 -- 172.21.15.100:0/321095158 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5364018680 con 0x7f537c102d50 2024-05-10T11:23:44.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.691+0000 7f538252b700 1 -- 172.21.15.100:0/321095158 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f537c1acb70 con 0x7f537c102d50 2024-05-10T11:23:44.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.694+0000 7f5379ffb700 1 -- 172.21.15.100:0/321095158 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f5364010950 con 0x7f537c102d50 2024-05-10T11:23:44.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.694+0000 7f5379ffb700 1 --2- 172.21.15.100:0/321095158 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f53680b2880 0x7f53680b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:44.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.695+0000 7f537bfff700 1 --2- 172.21.15.100:0/321095158 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f53680b2880 0x7f53680b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:44.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.695+0000 7f5379ffb700 1 -- 172.21.15.100:0/321095158 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f53640e28a0 con 0x7f537c102d50 2024-05-10T11:23:44.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.695+0000 7f537bfff700 1 --2- 172.21.15.100:0/321095158 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f53680b2880 0x7f53680b4d40 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f536c00b2e0 tx=0x7f536c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:23:44.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.695+0000 7f538cdbeb80 1 Processor -- start 2024-05-10T11:23:44.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.696+0000 7f538cdbeb80 1 -- start start 2024-05-10T11:23:44.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.696+0000 7f538cdbeb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56018c9824d0 0x56018c9828b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:44.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.696+0000 7f538cdbeb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56018c982df0 0x56018c9866f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:44.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.696+0000 7f538cdbeb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56018c986c80 0x56018c987100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:44.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.696+0000 7f538cdbeb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56018c8d4860 con 0x56018c986c80 2024-05-10T11:23:44.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.696+0000 7f538cdbeb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56018c8737c0 con 0x56018c9824d0 2024-05-10T11:23:44.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.696+0000 7f538cdbeb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56018c8d0820 con 0x56018c982df0 2024-05-10T11:23:44.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.696+0000 7f5380ac8700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56018c986c80 0x56018c987100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:44.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.696+0000 7f5380ac8700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56018c986c80 0x56018c987100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47452/0 (socket says 172.21.15.100:47452) 2024-05-10T11:23:44.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.696+0000 7f5380ac8700 1 -- 172.21.15.100:0/3465231730 learned_addr learned my addr 172.21.15.100:0/3465231730 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:23:44.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.696+0000 7f5373fff700 1 --2- 172.21.15.100:0/3465231730 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56018c982df0 0x56018c9866f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:44.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.696+0000 7f537bfff700 1 --2- 172.21.15.100:0/3465231730 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56018c9824d0 0x56018c9828b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:44.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.696+0000 7f5380ac8700 1 -- 172.21.15.100:0/3465231730 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56018c982df0 msgr2=0x56018c9866f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:44.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.696+0000 7f5380ac8700 1 --2- 172.21.15.100:0/3465231730 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56018c982df0 0x56018c9866f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:44.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.696+0000 7f5380ac8700 1 -- 172.21.15.100:0/3465231730 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56018c9824d0 msgr2=0x56018c9828b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:44.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.696+0000 7f5380ac8700 1 --2- 172.21.15.100:0/3465231730 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56018c9824d0 0x56018c9828b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:44.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.696+0000 7f5380ac8700 1 -- 172.21.15.100:0/3465231730 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56018c8218d0 con 0x56018c986c80 2024-05-10T11:23:44.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.696+0000 7f537bfff700 1 --2- 172.21.15.100:0/3465231730 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56018c9824d0 0x56018c9828b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:23:44.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.696+0000 7f5380ac8700 1 --2- 172.21.15.100:0/3465231730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56018c986c80 0x56018c987100 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f5374009cf0 tx=0x7f5374007500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63a29a90821a4db7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:23:44.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.697+0000 7f53727fc700 1 -- 172.21.15.100:0/3465231730 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5374007d90 con 0x56018c986c80 2024-05-10T11:23:44.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.697+0000 7f53727fc700 1 -- 172.21.15.100:0/3465231730 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5374004bb0 con 0x56018c986c80 2024-05-10T11:23:44.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.697+0000 7f53727fc700 1 -- 172.21.15.100:0/3465231730 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5374014670 con 0x56018c986c80 2024-05-10T11:23:44.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.698+0000 7f538cdbeb80 1 -- 172.21.15.100:0/3465231730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56018c986c80 msgr2=0x56018c987100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:44.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.698+0000 7f538cdbeb80 1 --2- 172.21.15.100:0/3465231730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56018c986c80 0x56018c987100 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f5374009cf0 tx=0x7f5374007500 comp rx=0 tx=0).stop 2024-05-10T11:23:44.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.698+0000 7f538cdbeb80 1 -- 172.21.15.100:0/3465231730 shutdown_connections 2024-05-10T11:23:44.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.698+0000 7f538cdbeb80 1 --2- 172.21.15.100:0/3465231730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56018c986c80 0x56018c987100 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:44.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.698+0000 7f538cdbeb80 1 --2- 172.21.15.100:0/3465231730 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56018c9824d0 0x56018c9828b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:44.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.698+0000 7f538cdbeb80 1 --2- 172.21.15.100:0/3465231730 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56018c982df0 0x56018c9866f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:44.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.698+0000 7f538cdbeb80 1 -- 172.21.15.100:0/3465231730 >> 172.21.15.100:0/3465231730 conn(0x56018c8ae100 msgr2=0x56018c97cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:23:44.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.698+0000 7f538cdbeb80 1 -- 172.21.15.100:0/3465231730 shutdown_connections 2024-05-10T11:23:44.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.698+0000 7f538cdbeb80 1 -- 172.21.15.100:0/3465231730 wait complete. 2024-05-10T11:23:44.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.698+0000 7f538cdbeb80 1 -- start start 2024-05-10T11:23:44.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.698+0000 7f538cdbeb80 1 Processor -- start 2024-05-10T11:23:44.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.698+0000 7f538cdbeb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56018c9824d0 0x56018c98e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:44.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.699+0000 7f538cdbeb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56018c982df0 0x56018c98b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:44.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.699+0000 7f538cdbeb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56018c986c80 0x56018c98b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:44.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.699+0000 7f538cdbeb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56018c98ee90 con 0x56018c982df0 2024-05-10T11:23:44.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.699+0000 7f538cdbeb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56018c83ac30 con 0x56018c9824d0 2024-05-10T11:23:44.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.699+0000 7f538cdbeb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56018c98f010 con 0x56018c986c80 2024-05-10T11:23:44.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.699+0000 7f538cdbeb80 10 client.0 _ll_register_callbacks cb 0x56018c86c920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:23:44.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.699+0000 7f537bfff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56018c9824d0 0x56018c98e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:44.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.699+0000 7f5380ac8700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56018c986c80 0x56018c98b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:44.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.699+0000 7f537bfff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56018c9824d0 0x56018c98e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:46014/0 (socket says 172.21.15.100:46014) 2024-05-10T11:23:44.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.699+0000 7f537bfff700 1 -- 172.21.15.100:0/3206241522 learned_addr learned my addr 172.21.15.100:0/3206241522 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:23:44.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.699+0000 7f5373fff700 1 --2- 172.21.15.100:0/3206241522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56018c982df0 0x56018c98b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:44.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.699+0000 7f537bfff700 1 -- 172.21.15.100:0/3206241522 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56018c986c80 msgr2=0x56018c98b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:44.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.699+0000 7f537bfff700 1 --2- 172.21.15.100:0/3206241522 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56018c986c80 0x56018c98b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:44.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.699+0000 7f537bfff700 1 -- 172.21.15.100:0/3206241522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56018c982df0 msgr2=0x56018c98b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:44.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.699+0000 7f537bfff700 1 --2- 172.21.15.100:0/3206241522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56018c982df0 0x56018c98b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:44.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.699+0000 7f537bfff700 1 -- 172.21.15.100:0/3206241522 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56018c870690 con 0x56018c9824d0 2024-05-10T11:23:44.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.699+0000 7f5373fff700 1 --2- 172.21.15.100:0/3206241522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56018c982df0 0x56018c98b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:23:44.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.699+0000 7f537bfff700 1 --2- 172.21.15.100:0/3206241522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56018c9824d0 0x56018c98e060 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f536c009990 tx=0x7f536c009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be6f9254a4249c1a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:23:44.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.699+0000 7f535b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:23:44.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.699+0000 7f535b7fe700 1 -- 172.21.15.100:0/3206241522 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536c034060 con 0x56018c9824d0 2024-05-10T11:23:44.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.699+0000 7f535b7fe700 1 -- 172.21.15.100:0/3206241522 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f536c023820 con 0x56018c9824d0 2024-05-10T11:23:44.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.699+0000 7f538cdbeb80 1 -- 172.21.15.100:0/3206241522 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5374004bb0 con 0x56018c9824d0 2024-05-10T11:23:44.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.700+0000 7f535b7fe700 1 -- 172.21.15.100:0/3206241522 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f536c03b050 con 0x56018c9824d0 2024-05-10T11:23:44.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.700+0000 7f538cdbeb80 10 client.25901 fetch_fsmap learned FSMap version 31 2024-05-10T11:23:44.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.700+0000 7f538cdbeb80 1 -- 172.21.15.100:0/3206241522 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56018c9996f0 con 0x56018c9824d0 2024-05-10T11:23:44.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.700+0000 7f535b7fe700 1 -- 172.21.15.100:0/3206241522 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5374004bb0 con 0x56018c9824d0 2024-05-10T11:23:44.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.700+0000 7f535b7fe700 1 -- 172.21.15.100:0/3206241522 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f536c02fa40 con 0x56018c9824d0 2024-05-10T11:23:44.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.700+0000 7f538cdbeb80 10 client.25901 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:23:44.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.700+0000 7f538cdbeb80 10 client.25901 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:23:44.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.700+0000 7f538cdbeb80 20 client.25901 populate_metadata read hostname 'smithi100' 2024-05-10T11:23:44.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.700+0000 7f538cdbeb80 1 --2- 172.21.15.100:0/3206241522 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56018c99bd30 0x56018c99e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:44.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.700+0000 7f538cdbeb80 4 client.25901 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:23:44.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.700+0000 7f538cdbeb80 1 -- 172.21.15.100:0/3206241522 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56018c94e650 con 0x56018c99bd30 2024-05-10T11:23:44.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.700+0000 7f5373fff700 1 --2- 172.21.15.100:0/3206241522 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56018c99bd30 0x56018c99e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:44.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.701+0000 7f535b7fe700 10 client.25901 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:23:44.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.701+0000 7f5373fff700 1 --2- 172.21.15.100:0/3206241522 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56018c99bd30 0x56018c99e160 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38de351525661659 server_cookie=24910578c7b8e723 in_seq=0 out_seq=0 2024-05-10T11:23:44.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.702+0000 7f535b7fe700 1 -- 172.21.15.100:0/3206241522 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56018c94e650 con 0x56018c99bd30 2024-05-10T11:23:44.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.702+0000 7f535b7fe700 10 client.25901 handle_command_reply: tid=0 2024-05-10T11:23:44.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.702+0000 7f538cdbeb80 1 client.25901 shutdown 2024-05-10T11:23:44.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.702+0000 7f538cdbeb80 1 -- 172.21.15.100:0/3206241522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56018c9824d0 msgr2=0x56018c98e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:44.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.702+0000 7f538cdbeb80 1 --2- 172.21.15.100:0/3206241522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56018c9824d0 0x56018c98e060 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f536c009990 tx=0x7f536c009ca0 comp rx=0 tx=0).stop 2024-05-10T11:23:44.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.702+0000 7f538cdbeb80 1 -- 172.21.15.100:0/3206241522 shutdown_connections 2024-05-10T11:23:44.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.702+0000 7f538cdbeb80 1 --2- 172.21.15.100:0/3206241522 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56018c99bd30 0x56018c99e160 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:44.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.702+0000 7f538cdbeb80 1 --2- 172.21.15.100:0/3206241522 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56018c986c80 0x56018c98b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:44.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.702+0000 7f538cdbeb80 1 --2- 172.21.15.100:0/3206241522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56018c9824d0 0x56018c98e060 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:44.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.702+0000 7f538cdbeb80 1 --2- 172.21.15.100:0/3206241522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56018c982df0 0x56018c98b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:44.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.702+0000 7f538cdbeb80 1 -- 172.21.15.100:0/3206241522 >> 172.21.15.100:0/3206241522 conn(0x56018c8ae100 msgr2=0x56018c987080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:23:44.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.702+0000 7f535b7fe700 0 client.25901 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:23:44.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.703+0000 7f538cdbeb80 1 -- 172.21.15.100:0/3206241522 shutdown_connections 2024-05-10T11:23:44.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.703+0000 7f538cdbeb80 1 -- 172.21.15.100:0/3206241522 wait complete. 2024-05-10T11:23:44.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.703+0000 7f538cdbeb80 20 client.25901 trim_cache size 0 max 16384 2024-05-10T11:23:44.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.720+0000 7f538cdbeb80 1 Processor -- start 2024-05-10T11:23:44.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.720+0000 7f538cdbeb80 1 -- start start 2024-05-10T11:23:44.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.720+0000 7f538cdbeb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56018c9824d0 0x56018c997f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:44.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.720+0000 7f538cdbeb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56018c982df0 0x56018c98a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:44.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.720+0000 7f538cdbeb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56018c986c80 0x56018c9775a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:44.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.720+0000 7f538cdbeb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f537c1b80b0 con 0x56018c982df0 2024-05-10T11:23:44.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.720+0000 7f538cdbeb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f537c1b5940 con 0x56018c986c80 2024-05-10T11:23:44.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.720+0000 7f538cdbeb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f537c1b3790 con 0x56018c9824d0 2024-05-10T11:23:44.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.721+0000 7f5373fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56018c982df0 0x56018c98a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:44.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.721+0000 7f5373fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56018c982df0 0x56018c98a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47472/0 (socket says 172.21.15.100:47472) 2024-05-10T11:23:44.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.721+0000 7f5373fff700 1 -- 172.21.15.100:0/1620691031 learned_addr learned my addr 172.21.15.100:0/1620691031 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:23:44.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.721+0000 7f537bfff700 1 --2- 172.21.15.100:0/1620691031 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56018c9824d0 0x56018c997f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:44.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.721+0000 7f5380ac8700 1 --2- 172.21.15.100:0/1620691031 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56018c986c80 0x56018c9775a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:44.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.721+0000 7f5373fff700 1 -- 172.21.15.100:0/1620691031 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56018c9824d0 msgr2=0x56018c997f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:44.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.721+0000 7f5373fff700 1 --2- 172.21.15.100:0/1620691031 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56018c9824d0 0x56018c997f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:44.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.721+0000 7f5373fff700 1 -- 172.21.15.100:0/1620691031 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56018c986c80 msgr2=0x56018c9775a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:44.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.721+0000 7f5373fff700 1 --2- 172.21.15.100:0/1620691031 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56018c986c80 0x56018c9775a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:44.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.721+0000 7f5373fff700 1 -- 172.21.15.100:0/1620691031 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56018c98e3d0 con 0x56018c982df0 2024-05-10T11:23:44.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.721+0000 7f5373fff700 1 --2- 172.21.15.100:0/1620691031 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56018c982df0 0x56018c98a5a0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f53640e0090 tx=0x7f53640dff10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f120476537fa7ee server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:23:44.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.721+0000 7f5371ffb700 1 -- 172.21.15.100:0/1620691031 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53640a52f0 con 0x56018c982df0 2024-05-10T11:23:44.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.721+0000 7f5371ffb700 1 -- 172.21.15.100:0/1620691031 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f53640a9670 con 0x56018c982df0 2024-05-10T11:23:44.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.721+0000 7f5371ffb700 1 -- 172.21.15.100:0/1620691031 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5364010430 con 0x56018c982df0 2024-05-10T11:23:44.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.722+0000 7f538cdbeb80 1 -- 172.21.15.100:0/1620691031 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56018c982df0 msgr2=0x56018c98a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:44.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.722+0000 7f538cdbeb80 1 --2- 172.21.15.100:0/1620691031 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56018c982df0 0x56018c98a5a0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f53640e0090 tx=0x7f53640dff10 comp rx=0 tx=0).stop 2024-05-10T11:23:44.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.722+0000 7f538cdbeb80 1 -- 172.21.15.100:0/1620691031 shutdown_connections 2024-05-10T11:23:44.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.722+0000 7f538cdbeb80 1 --2- 172.21.15.100:0/1620691031 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56018c986c80 0x56018c9775a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:44.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.722+0000 7f538cdbeb80 1 --2- 172.21.15.100:0/1620691031 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56018c9824d0 0x56018c997f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:44.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.722+0000 7f538cdbeb80 1 --2- 172.21.15.100:0/1620691031 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56018c982df0 0x56018c98a5a0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:44.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.722+0000 7f538cdbeb80 1 -- 172.21.15.100:0/1620691031 >> 172.21.15.100:0/1620691031 conn(0x56018c8ae100 msgr2=0x56018c976e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:23:44.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.722+0000 7f538cdbeb80 1 -- 172.21.15.100:0/1620691031 shutdown_connections 2024-05-10T11:23:44.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.722+0000 7f538cdbeb80 1 -- 172.21.15.100:0/1620691031 wait complete. 2024-05-10T11:23:44.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.722+0000 7f538cdbeb80 1 -- start start 2024-05-10T11:23:44.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.722+0000 7f538cdbeb80 1 Processor -- start 2024-05-10T11:23:44.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.723+0000 7f538cdbeb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56018c9824d0 0x56018c98f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:44.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.723+0000 7f538cdbeb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56018c982df0 0x56018c98f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:44.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.723+0000 7f538cdbeb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56018c986c80 0x56018c98fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:44.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.723+0000 7f538cdbeb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f537c1b3e00 con 0x56018c982df0 2024-05-10T11:23:44.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.723+0000 7f538cdbeb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f537c1b5270 con 0x56018c986c80 2024-05-10T11:23:44.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.723+0000 7f538cdbeb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f537c1b44d0 con 0x56018c9824d0 2024-05-10T11:23:44.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.723+0000 7f538cdbeb80 10 client.0 _ll_register_callbacks cb 0x56018c86c920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:23:44.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.723+0000 7f5373fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56018c982df0 0x56018c98f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:44.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.723+0000 7f537bfff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56018c9824d0 0x56018c98f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:44.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.723+0000 7f5373fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56018c982df0 0x56018c98f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47488/0 (socket says 172.21.15.100:47488) 2024-05-10T11:23:44.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.723+0000 7f5373fff700 1 -- 172.21.15.100:0/2986161096 learned_addr learned my addr 172.21.15.100:0/2986161096 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:23:44.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.723+0000 7f5380ac8700 1 --2- 172.21.15.100:0/2986161096 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56018c986c80 0x56018c98fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:44.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.723+0000 7f537bfff700 1 -- 172.21.15.100:0/2986161096 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56018c986c80 msgr2=0x56018c98fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:44.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.723+0000 7f537bfff700 1 --2- 172.21.15.100:0/2986161096 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56018c986c80 0x56018c98fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:44.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.723+0000 7f537bfff700 1 -- 172.21.15.100:0/2986161096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56018c982df0 msgr2=0x56018c98f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:44.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.723+0000 7f537bfff700 1 --2- 172.21.15.100:0/2986161096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56018c982df0 0x56018c98f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:44.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.723+0000 7f537bfff700 1 -- 172.21.15.100:0/2986161096 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56018ca048b0 con 0x56018c9824d0 2024-05-10T11:23:44.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.723+0000 7f5373fff700 1 --2- 172.21.15.100:0/2986161096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56018c982df0 0x56018c98f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:23:44.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.723+0000 7f5380ac8700 1 --2- 172.21.15.100:0/2986161096 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56018c986c80 0x56018c98fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:23:44.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.723+0000 7f537bfff700 1 --2- 172.21.15.100:0/2986161096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56018c9824d0 0x56018c98f2c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f536c005a80 tx=0x7f536c0225a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4e968b91c12ad6b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:23:44.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.723+0000 7f535b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:23:44.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.724+0000 7f535b7fe700 1 -- 172.21.15.100:0/2986161096 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536c034040 con 0x56018c9824d0 2024-05-10T11:23:44.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.724+0000 7f535b7fe700 1 -- 172.21.15.100:0/2986161096 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f536c02f880 con 0x56018c9824d0 2024-05-10T11:23:44.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.724+0000 7f538cdbeb80 1 -- 172.21.15.100:0/2986161096 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53640a9670 con 0x56018c9824d0 2024-05-10T11:23:44.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.724+0000 7f535b7fe700 1 -- 172.21.15.100:0/2986161096 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f536c02fa80 con 0x56018c9824d0 2024-05-10T11:23:44.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.724+0000 7f538cdbeb80 10 client.25936 fetch_fsmap learned FSMap version 31 2024-05-10T11:23:44.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.724+0000 7f538cdbeb80 1 -- 172.21.15.100:0/2986161096 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56018ca05b90 con 0x56018c9824d0 2024-05-10T11:23:44.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.724+0000 7f535b7fe700 1 -- 172.21.15.100:0/2986161096 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f53640a9670 con 0x56018c9824d0 2024-05-10T11:23:44.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.724+0000 7f535b7fe700 1 -- 172.21.15.100:0/2986161096 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f536c023bd0 con 0x56018c9824d0 2024-05-10T11:23:44.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.724+0000 7f538cdbeb80 10 client.25936 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:23:44.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.724+0000 7f538cdbeb80 10 client.25936 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:23:44.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.724+0000 7f538cdbeb80 20 client.25936 populate_metadata read hostname 'smithi100' 2024-05-10T11:23:44.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.724+0000 7f538cdbeb80 1 --2- 172.21.15.100:0/2986161096 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56018ca061e0 0x56018ca08600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:44.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.724+0000 7f538cdbeb80 4 client.25936 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:23:44.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.724+0000 7f538cdbeb80 1 -- 172.21.15.100:0/2986161096 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56018c8e2240 con 0x56018ca061e0 2024-05-10T11:23:44.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.725+0000 7f5373fff700 1 --2- 172.21.15.100:0/2986161096 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56018ca061e0 0x56018ca08600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:44.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.725+0000 7f535b7fe700 10 client.25936 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:23:44.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.725+0000 7f5373fff700 1 --2- 172.21.15.100:0/2986161096 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56018ca061e0 0x56018ca08600 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0a6c329a616d20 server_cookie=224b610afc7354a8 in_seq=0 out_seq=0 2024-05-10T11:23:44.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.726+0000 7f535b7fe700 1 -- 172.21.15.100:0/2986161096 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56018c8e2240 con 0x56018ca061e0 2024-05-10T11:23:44.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.726+0000 7f535b7fe700 10 client.25936 handle_command_reply: tid=0 2024-05-10T11:23:44.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.726+0000 7f538cdbeb80 1 client.25936 shutdown 2024-05-10T11:23:44.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.726+0000 7f538cdbeb80 1 -- 172.21.15.100:0/2986161096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56018c9824d0 msgr2=0x56018c98f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:44.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.726+0000 7f538cdbeb80 1 --2- 172.21.15.100:0/2986161096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56018c9824d0 0x56018c98f2c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f536c005a80 tx=0x7f536c0225a0 comp rx=0 tx=0).stop 2024-05-10T11:23:44.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.726+0000 7f538cdbeb80 1 -- 172.21.15.100:0/2986161096 shutdown_connections 2024-05-10T11:23:44.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.726+0000 7f538cdbeb80 1 --2- 172.21.15.100:0/2986161096 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56018ca061e0 0x56018ca08600 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:44.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.726+0000 7f538cdbeb80 1 --2- 172.21.15.100:0/2986161096 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56018c986c80 0x56018c98fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:44.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.726+0000 7f535b7fe700 0 client.25936 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:23:44.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.726+0000 7f538cdbeb80 1 --2- 172.21.15.100:0/2986161096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56018c9824d0 0x56018c98f2c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:44.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.726+0000 7f538cdbeb80 1 --2- 172.21.15.100:0/2986161096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56018c982df0 0x56018c98f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:44.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.726+0000 7f538cdbeb80 1 -- 172.21.15.100:0/2986161096 >> 172.21.15.100:0/2986161096 conn(0x56018c8ae100 msgr2=0x56018c98a420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:23:44.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.726+0000 7f538cdbeb80 1 -- 172.21.15.100:0/2986161096 shutdown_connections 2024-05-10T11:23:44.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.726+0000 7f538cdbeb80 1 -- 172.21.15.100:0/2986161096 wait complete. 2024-05-10T11:23:44.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.726+0000 7f538cdbeb80 20 client.25936 trim_cache size 0 max 16384 2024-05-10T11:23:44.731 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:23:44.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.727+0000 7f538252b700 1 -- 172.21.15.100:0/321095158 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f53680b2880 msgr2=0x7f53680b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:44.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.727+0000 7f538252b700 1 --2- 172.21.15.100:0/321095158 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f53680b2880 0x7f53680b4d40 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f536c00b2e0 tx=0x7f536c01a040 comp rx=0 tx=0).stop 2024-05-10T11:23:44.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.727+0000 7f538252b700 1 -- 172.21.15.100:0/321095158 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f537c102d50 msgr2=0x7f537c1a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:44.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.727+0000 7f538252b700 1 --2- 172.21.15.100:0/321095158 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f537c102d50 0x7f537c1a6250 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f536400e9a0 tx=0x7f536400ed60 comp rx=0 tx=0).stop 2024-05-10T11:23:44.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.727+0000 7f538252b700 1 -- 172.21.15.100:0/321095158 shutdown_connections 2024-05-10T11:23:44.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.727+0000 7f538252b700 1 --2- 172.21.15.100:0/321095158 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f53680b2880 0x7f53680b4d40 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:44.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.727+0000 7f538252b700 1 --2- 172.21.15.100:0/321095158 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f537c108400 0x7f537c1a00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:44.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.727+0000 7f538252b700 1 --2- 172.21.15.100:0/321095158 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f537c102390 0x7f537c1a5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:44.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.727+0000 7f538252b700 1 --2- 172.21.15.100:0/321095158 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f537c102d50 0x7f537c1a6250 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:44.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.727+0000 7f538252b700 1 -- 172.21.15.100:0/321095158 >> 172.21.15.100:0/321095158 conn(0x7f537c079310 msgr2=0x7f537c06bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:23:44.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.727+0000 7f538252b700 1 -- 172.21.15.100:0/321095158 shutdown_connections 2024-05-10T11:23:44.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:44.727+0000 7f538252b700 1 -- 172.21.15.100:0/321095158 wait complete. 2024-05-10T11:23:44.971 INFO:tasks.workunit.client.0.smithi100.stdout: 1 114647 2.73 MB/sec execute 99 sec latency 48.910 ms 2024-05-10T11:23:45.971 INFO:tasks.workunit.client.0.smithi100.stdout: 1 115195 2.74 MB/sec execute 100 sec latency 21.886 ms 2024-05-10T11:23:46.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:45 smithi178 ceph-mon[33359]: pgmap v581: 129 pgs: 129 active+clean; 56 MiB data, 1024 MiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 1.2 MiB/s wr, 147 op/s 2024-05-10T11:23:46.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:45 smithi188 ceph-mon[34075]: pgmap v581: 129 pgs: 129 active+clean; 56 MiB data, 1024 MiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 1.2 MiB/s wr, 147 op/s 2024-05-10T11:23:46.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:45 smithi100 ceph-mon[30296]: pgmap v581: 129 pgs: 129 active+clean; 56 MiB data, 1024 MiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 1.2 MiB/s wr, 147 op/s 2024-05-10T11:23:46.444 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T11:23:46.444 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T11:23:46.718 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:23:46.971 INFO:tasks.workunit.client.0.smithi100.stdout: 1 115903 2.73 MB/sec execute 101 sec latency 12.740 ms 2024-05-10T11:23:47.971 INFO:tasks.workunit.client.0.smithi100.stdout: 1 116516 2.76 MB/sec execute 102 sec latency 15.210 ms 2024-05-10T11:23:48.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:47 smithi100 ceph-mon[30296]: pgmap v582: 129 pgs: 129 active+clean; 58 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 1.3 MiB/s wr, 151 op/s 2024-05-10T11:23:48.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:47 smithi178 ceph-mon[33359]: pgmap v582: 129 pgs: 129 active+clean; 58 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 1.3 MiB/s wr, 151 op/s 2024-05-10T11:23:48.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:47 smithi188 ceph-mon[34075]: pgmap v582: 129 pgs: 129 active+clean; 58 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 1.3 MiB/s wr, 151 op/s 2024-05-10T11:23:48.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.369+0000 7fcc2dd1d700 1 -- 172.21.15.100:0/1561865970 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc28102390 msgr2=0x7fcc28102810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:48.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.369+0000 7fcc2dd1d700 1 --2- 172.21.15.100:0/1561865970 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc28102390 0x7fcc28102810 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c009990 tx=0x7fcc1c009ca0 comp rx=0 tx=0).stop 2024-05-10T11:23:48.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.369+0000 7fcc2dd1d700 1 -- 172.21.15.100:0/1561865970 shutdown_connections 2024-05-10T11:23:48.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.369+0000 7fcc2dd1d700 1 --2- 172.21.15.100:0/1561865970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcc28102d50 0x7fcc2810f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:48.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.369+0000 7fcc2dd1d700 1 --2- 172.21.15.100:0/1561865970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc28108400 0x7fcc281087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:48.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.369+0000 7fcc2dd1d700 1 --2- 172.21.15.100:0/1561865970 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc28102390 0x7fcc28102810 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:48.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.369+0000 7fcc2dd1d700 1 -- 172.21.15.100:0/1561865970 >> 172.21.15.100:0/1561865970 conn(0x7fcc28079310 msgr2=0x7fcc28079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:23:48.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.369+0000 7fcc2dd1d700 1 -- 172.21.15.100:0/1561865970 shutdown_connections 2024-05-10T11:23:48.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.369+0000 7fcc2dd1d700 1 -- 172.21.15.100:0/1561865970 wait complete. 2024-05-10T11:23:48.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.369+0000 7fcc2dd1d700 1 Processor -- start 2024-05-10T11:23:48.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.369+0000 7fcc2dd1d700 1 -- start start 2024-05-10T11:23:48.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.370+0000 7fcc2dd1d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc28102390 0x7fcc2810fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:48.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.370+0000 7fcc2dd1d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc28102d50 0x7fcc28110210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:48.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.370+0000 7fcc2dd1d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcc28108400 0x7fcc28110750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:48.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.370+0000 7fcc2dd1d700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fcc28113520 con 0x7fcc28102d50 2024-05-10T11:23:48.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.370+0000 7fcc2dd1d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcc281133a0 con 0x7fcc28108400 2024-05-10T11:23:48.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.370+0000 7fcc2dd1d700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcc281136a0 con 0x7fcc28102390 2024-05-10T11:23:48.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.370+0000 7fcc277fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc28102390 0x7fcc2810fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:48.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.370+0000 7fcc26ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc28102d50 0x7fcc28110210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:48.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.370+0000 7fcc277fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc28102390 0x7fcc2810fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:49236/0 (socket says 172.21.15.100:49236) 2024-05-10T11:23:48.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.370+0000 7fcc277fe700 1 -- 172.21.15.100:0/1951705279 learned_addr learned my addr 172.21.15.100:0/1951705279 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:23:48.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.370+0000 7fcc27fff700 1 --2- 172.21.15.100:0/1951705279 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcc28108400 0x7fcc28110750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:48.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.370+0000 7fcc277fe700 1 -- 172.21.15.100:0/1951705279 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcc28108400 msgr2=0x7fcc28110750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:48.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.370+0000 7fcc277fe700 1 --2- 172.21.15.100:0/1951705279 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcc28108400 0x7fcc28110750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:48.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.370+0000 7fcc277fe700 1 -- 172.21.15.100:0/1951705279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc28102d50 msgr2=0x7fcc28110210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:48.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.370+0000 7fcc277fe700 1 --2- 172.21.15.100:0/1951705279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc28102d50 0x7fcc28110210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:48.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.370+0000 7fcc277fe700 1 -- 172.21.15.100:0/1951705279 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc281a7fb0 con 0x7fcc28102390 2024-05-10T11:23:48.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.370+0000 7fcc26ffd700 1 --2- 172.21.15.100:0/1951705279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc28102d50 0x7fcc28110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:23:48.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.370+0000 7fcc27fff700 1 --2- 172.21.15.100:0/1951705279 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcc28108400 0x7fcc28110750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:23:48.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.370+0000 7fcc277fe700 1 --2- 172.21.15.100:0/1951705279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc28102390 0x7fcc2810fcd0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcc1800e9c0 tx=0x7fcc1800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:23:48.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.371+0000 7fcc24ff9700 1 -- 172.21.15.100:0/1951705279 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc1800cbd0 con 0x7fcc28102390 2024-05-10T11:23:48.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.371+0000 7fcc24ff9700 1 -- 172.21.15.100:0/1951705279 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcc1800cd70 con 0x7fcc28102390 2024-05-10T11:23:48.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.371+0000 7fcc2dd1d700 1 -- 172.21.15.100:0/1951705279 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc281a82a0 con 0x7fcc28102390 2024-05-10T11:23:48.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.371+0000 7fcc24ff9700 1 -- 172.21.15.100:0/1951705279 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc18005540 con 0x7fcc28102390 2024-05-10T11:23:48.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.371+0000 7fcc2dd1d700 1 -- 172.21.15.100:0/1951705279 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc281a8790 con 0x7fcc28102390 2024-05-10T11:23:48.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.373+0000 7fcc24ff9700 1 -- 172.21.15.100:0/1951705279 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fcc18004750 con 0x7fcc28102390 2024-05-10T11:23:48.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.374+0000 7fcc24ff9700 1 --2- 172.21.15.100:0/1951705279 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fcc100b2880 0x7fcc100b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:48.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.374+0000 7fcc26ffd700 1 --2- 172.21.15.100:0/1951705279 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fcc100b2880 0x7fcc100b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:48.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.374+0000 7fcc24ff9700 1 -- 172.21.15.100:0/1951705279 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fcc180e27c0 con 0x7fcc28102390 2024-05-10T11:23:48.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.374+0000 7fcc26ffd700 1 --2- 172.21.15.100:0/1951705279 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fcc100b2880 0x7fcc100b4d40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fcc28079980 tx=0x7fcc1c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:23:48.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.375+0000 7fcc385b0b80 1 Processor -- start 2024-05-10T11:23:48.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.375+0000 7fcc385b0b80 1 -- start start 2024-05-10T11:23:48.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.375+0000 7fcc385b0b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae53615550 0x55ae53615930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:48.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.375+0000 7fcc385b0b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ae53615e70 0x55ae53619770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:48.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.375+0000 7fcc385b0b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ae53619d00 0x55ae5361a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:48.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.375+0000 7fcc385b0b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55ae535678e0 con 0x55ae53615550 2024-05-10T11:23:48.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.375+0000 7fcc385b0b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ae53506840 con 0x55ae53615e70 2024-05-10T11:23:48.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.375+0000 7fcc385b0b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55ae535638a0 con 0x55ae53619d00 2024-05-10T11:23:48.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.375+0000 7fcc277fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae53615550 0x55ae53615930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:48.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.375+0000 7fcc27fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ae53619d00 0x55ae5361a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:48.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.375+0000 7fcc277fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae53615550 0x55ae53615930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47466/0 (socket says 172.21.15.100:47466) 2024-05-10T11:23:48.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.375+0000 7fcc277fe700 1 -- 172.21.15.100:0/20692790 learned_addr learned my addr 172.21.15.100:0/20692790 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:23:48.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.375+0000 7fcc26ffd700 1 --2- 172.21.15.100:0/20692790 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ae53615e70 0x55ae53619770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:48.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.376+0000 7fcc27fff700 1 -- 172.21.15.100:0/20692790 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ae53615e70 msgr2=0x55ae53619770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:48.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.376+0000 7fcc27fff700 1 --2- 172.21.15.100:0/20692790 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ae53615e70 0x55ae53619770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:48.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.376+0000 7fcc27fff700 1 -- 172.21.15.100:0/20692790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae53615550 msgr2=0x55ae53615930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:48.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.376+0000 7fcc27fff700 1 --2- 172.21.15.100:0/20692790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae53615550 0x55ae53615930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:48.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.376+0000 7fcc27fff700 1 -- 172.21.15.100:0/20692790 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae534b4950 con 0x55ae53619d00 2024-05-10T11:23:48.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.376+0000 7fcc277fe700 1 --2- 172.21.15.100:0/20692790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae53615550 0x55ae53615930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:23:48.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.376+0000 7fcc27fff700 1 --2- 172.21.15.100:0/20692790 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ae53619d00 0x55ae5361a180 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fcc20009d90 tx=0x7fcc200077e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=207e9d2aaf06d26a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:23:48.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.376+0000 7fcc0dffb700 1 -- 172.21.15.100:0/20692790 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc20014610 con 0x55ae53619d00 2024-05-10T11:23:48.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.376+0000 7fcc0dffb700 1 -- 172.21.15.100:0/20692790 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcc20019070 con 0x55ae53619d00 2024-05-10T11:23:48.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.376+0000 7fcc0dffb700 1 -- 172.21.15.100:0/20692790 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc20013830 con 0x55ae53619d00 2024-05-10T11:23:48.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.377+0000 7fcc385b0b80 1 -- 172.21.15.100:0/20692790 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ae53619d00 msgr2=0x55ae5361a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:48.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.377+0000 7fcc385b0b80 1 --2- 172.21.15.100:0/20692790 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ae53619d00 0x55ae5361a180 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fcc20009d90 tx=0x7fcc200077e0 comp rx=0 tx=0).stop 2024-05-10T11:23:48.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.377+0000 7fcc385b0b80 1 -- 172.21.15.100:0/20692790 shutdown_connections 2024-05-10T11:23:48.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.377+0000 7fcc385b0b80 1 --2- 172.21.15.100:0/20692790 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ae53619d00 0x55ae5361a180 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:48.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.377+0000 7fcc385b0b80 1 --2- 172.21.15.100:0/20692790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae53615550 0x55ae53615930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:48.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.377+0000 7fcc385b0b80 1 --2- 172.21.15.100:0/20692790 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ae53615e70 0x55ae53619770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:48.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.377+0000 7fcc385b0b80 1 -- 172.21.15.100:0/20692790 >> 172.21.15.100:0/20692790 conn(0x55ae53541180 msgr2=0x55ae5360fcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:23:48.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.377+0000 7fcc385b0b80 1 -- 172.21.15.100:0/20692790 shutdown_connections 2024-05-10T11:23:48.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.377+0000 7fcc385b0b80 1 -- 172.21.15.100:0/20692790 wait complete. 2024-05-10T11:23:48.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.377+0000 7fcc385b0b80 1 -- start start 2024-05-10T11:23:48.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.378+0000 7fcc385b0b80 1 Processor -- start 2024-05-10T11:23:48.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.378+0000 7fcc385b0b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ae53615550 0x55ae53621120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:48.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.378+0000 7fcc385b0b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ae53615e70 0x55ae5361e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:48.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.378+0000 7fcc385b0b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae53619d00 0x55ae5361ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:48.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.378+0000 7fcc385b0b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55ae53621f10 con 0x55ae53619d00 2024-05-10T11:23:48.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.378+0000 7fcc385b0b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ae534cdcb0 con 0x55ae53615550 2024-05-10T11:23:48.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.378+0000 7fcc385b0b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55ae53622090 con 0x55ae53615e70 2024-05-10T11:23:48.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.378+0000 7fcc385b0b80 10 client.0 _ll_register_callbacks cb 0x55ae534ff9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:23:48.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.378+0000 7fcc26ffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ae53615e70 0x55ae5361e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:48.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.378+0000 7fcc26ffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ae53615e70 0x55ae5361e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:49246/0 (socket says 172.21.15.100:49246) 2024-05-10T11:23:48.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.378+0000 7fcc26ffd700 1 -- 172.21.15.100:0/3648598220 learned_addr learned my addr 172.21.15.100:0/3648598220 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:23:48.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.378+0000 7fcc27fff700 1 --2- 172.21.15.100:0/3648598220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae53619d00 0x55ae5361ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:48.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.378+0000 7fcc277fe700 1 --2- 172.21.15.100:0/3648598220 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ae53615550 0x55ae53621120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:48.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.378+0000 7fcc26ffd700 1 -- 172.21.15.100:0/3648598220 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ae53615550 msgr2=0x55ae53621120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:48.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.378+0000 7fcc26ffd700 1 --2- 172.21.15.100:0/3648598220 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ae53615550 0x55ae53621120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:48.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.378+0000 7fcc26ffd700 1 -- 172.21.15.100:0/3648598220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae53619d00 msgr2=0x55ae5361ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:48.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.378+0000 7fcc26ffd700 1 --2- 172.21.15.100:0/3648598220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae53619d00 0x55ae5361ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:48.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.379+0000 7fcc26ffd700 1 -- 172.21.15.100:0/3648598220 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae53503710 con 0x55ae53615e70 2024-05-10T11:23:48.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.379+0000 7fcc27fff700 1 --2- 172.21.15.100:0/3648598220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae53619d00 0x55ae5361ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:23:48.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.379+0000 7fcc26ffd700 1 --2- 172.21.15.100:0/3648598220 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ae53615e70 0x55ae5361e510 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c009990 tx=0x7fcc1c009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d09805a916da844e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:23:48.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.379+0000 7fcc06ffd700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:23:48.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.379+0000 7fcc06ffd700 1 -- 172.21.15.100:0/3648598220 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c0056a0 con 0x55ae53615e70 2024-05-10T11:23:48.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.379+0000 7fcc06ffd700 1 -- 172.21.15.100:0/3648598220 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fcc1c031460 con 0x55ae53615e70 2024-05-10T11:23:48.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.379+0000 7fcc385b0b80 1 -- 172.21.15.100:0/3648598220 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc20019070 con 0x55ae53615e70 2024-05-10T11:23:48.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.379+0000 7fcc06ffd700 1 -- 172.21.15.100:0/3648598220 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fcc1c031660 con 0x55ae53615e70 2024-05-10T11:23:48.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.379+0000 7fcc385b0b80 10 client.25948 fetch_fsmap learned FSMap version 31 2024-05-10T11:23:48.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.379+0000 7fcc385b0b80 1 -- 172.21.15.100:0/3648598220 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55ae5362c750 con 0x55ae53615e70 2024-05-10T11:23:48.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.379+0000 7fcc06ffd700 1 -- 172.21.15.100:0/3648598220 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcc20019070 con 0x55ae53615e70 2024-05-10T11:23:48.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.379+0000 7fcc06ffd700 1 -- 172.21.15.100:0/3648598220 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fcc1c00f880 con 0x55ae53615e70 2024-05-10T11:23:48.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.380+0000 7fcc385b0b80 10 client.25948 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:23:48.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.380+0000 7fcc385b0b80 10 client.25948 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:23:48.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.380+0000 7fcc385b0b80 20 client.25948 populate_metadata read hostname 'smithi100' 2024-05-10T11:23:48.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.380+0000 7fcc385b0b80 1 --2- 172.21.15.100:0/3648598220 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ae5362ed90 0x55ae53631180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:48.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.380+0000 7fcc385b0b80 4 client.25948 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:23:48.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.380+0000 7fcc385b0b80 1 -- 172.21.15.100:0/3648598220 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae535e16d0 con 0x55ae5362ed90 2024-05-10T11:23:48.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.380+0000 7fcc277fe700 1 --2- 172.21.15.100:0/3648598220 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ae5362ed90 0x55ae53631180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:48.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.380+0000 7fcc277fe700 1 --2- 172.21.15.100:0/3648598220 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ae5362ed90 0x55ae53631180 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a980cfe369a03d52 server_cookie=30971207d9e60374 in_seq=0 out_seq=0 2024-05-10T11:23:48.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.380+0000 7fcc06ffd700 10 client.25948 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:23:48.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.381+0000 7fcc06ffd700 1 -- 172.21.15.100:0/3648598220 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ae535e16d0 con 0x55ae5362ed90 2024-05-10T11:23:48.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.381+0000 7fcc06ffd700 10 client.25948 handle_command_reply: tid=0 2024-05-10T11:23:48.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.381+0000 7fcc385b0b80 1 client.25948 shutdown 2024-05-10T11:23:48.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.381+0000 7fcc385b0b80 1 -- 172.21.15.100:0/3648598220 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ae53615e70 msgr2=0x55ae5361e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:48.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.381+0000 7fcc385b0b80 1 --2- 172.21.15.100:0/3648598220 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ae53615e70 0x55ae5361e510 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c009990 tx=0x7fcc1c009310 comp rx=0 tx=0).stop 2024-05-10T11:23:48.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.381+0000 7fcc385b0b80 1 -- 172.21.15.100:0/3648598220 shutdown_connections 2024-05-10T11:23:48.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.381+0000 7fcc385b0b80 1 --2- 172.21.15.100:0/3648598220 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ae5362ed90 0x55ae53631180 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:48.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.381+0000 7fcc385b0b80 1 --2- 172.21.15.100:0/3648598220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae53619d00 0x55ae5361ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:48.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.381+0000 7fcc385b0b80 1 --2- 172.21.15.100:0/3648598220 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ae53615550 0x55ae53621120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:48.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.381+0000 7fcc385b0b80 1 --2- 172.21.15.100:0/3648598220 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ae53615e70 0x55ae5361e510 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:48.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.381+0000 7fcc06ffd700 0 client.25948 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:23:48.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.381+0000 7fcc385b0b80 1 -- 172.21.15.100:0/3648598220 >> 172.21.15.100:0/3648598220 conn(0x55ae53541180 msgr2=0x55ae5361a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:23:48.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.381+0000 7fcc385b0b80 1 -- 172.21.15.100:0/3648598220 shutdown_connections 2024-05-10T11:23:48.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.381+0000 7fcc385b0b80 1 -- 172.21.15.100:0/3648598220 wait complete. 2024-05-10T11:23:48.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.381+0000 7fcc385b0b80 20 client.25948 trim_cache size 0 max 16384 2024-05-10T11:23:48.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.404+0000 7fcc385b0b80 1 Processor -- start 2024-05-10T11:23:48.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.404+0000 7fcc385b0b80 1 -- start start 2024-05-10T11:23:48.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.404+0000 7fcc385b0b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ae53615550 0x55ae5361eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:48.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.404+0000 7fcc385b0b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ae53615e70 0x55ae5362afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:48.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.404+0000 7fcc385b0b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae53619d00 0x55ae53617670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:48.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.404+0000 7fcc385b0b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fcc281b3b10 con 0x55ae53619d00 2024-05-10T11:23:48.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.404+0000 7fcc385b0b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcc281b13a0 con 0x55ae53615e70 2024-05-10T11:23:48.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.404+0000 7fcc385b0b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcc281af1f0 con 0x55ae53615550 2024-05-10T11:23:48.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.404+0000 7fcc27fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae53619d00 0x55ae53617670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:48.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.405+0000 7fcc27fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae53619d00 0x55ae53617670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47476/0 (socket says 172.21.15.100:47476) 2024-05-10T11:23:48.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.405+0000 7fcc27fff700 1 -- 172.21.15.100:0/2730882332 learned_addr learned my addr 172.21.15.100:0/2730882332 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:23:48.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.405+0000 7fcc277fe700 1 --2- 172.21.15.100:0/2730882332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ae53615550 0x55ae5361eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:48.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.405+0000 7fcc26ffd700 1 --2- 172.21.15.100:0/2730882332 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ae53615e70 0x55ae5362afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:48.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.405+0000 7fcc27fff700 1 -- 172.21.15.100:0/2730882332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ae53615550 msgr2=0x55ae5361eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:48.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.405+0000 7fcc27fff700 1 --2- 172.21.15.100:0/2730882332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ae53615550 0x55ae5361eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:48.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.405+0000 7fcc27fff700 1 -- 172.21.15.100:0/2730882332 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ae53615e70 msgr2=0x55ae5362afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:48.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.405+0000 7fcc27fff700 1 --2- 172.21.15.100:0/2730882332 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ae53615e70 0x55ae5362afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:48.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.405+0000 7fcc27fff700 1 -- 172.21.15.100:0/2730882332 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae53610810 con 0x55ae53619d00 2024-05-10T11:23:48.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.405+0000 7fcc26ffd700 1 --2- 172.21.15.100:0/2730882332 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ae53615e70 0x55ae5362afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:23:48.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.405+0000 7fcc27fff700 1 --2- 172.21.15.100:0/2730882332 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae53619d00 0x55ae53617670 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fcc20009d90 tx=0x7fcc20009270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a318fda0ef052661 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:23:48.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.405+0000 7fcc0d7fa700 1 -- 172.21.15.100:0/2730882332 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc20014610 con 0x55ae53619d00 2024-05-10T11:23:48.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.405+0000 7fcc0d7fa700 1 -- 172.21.15.100:0/2730882332 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcc20019050 con 0x55ae53619d00 2024-05-10T11:23:48.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.405+0000 7fcc0d7fa700 1 -- 172.21.15.100:0/2730882332 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc20005520 con 0x55ae53619d00 2024-05-10T11:23:48.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.406+0000 7fcc385b0b80 1 -- 172.21.15.100:0/2730882332 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae53619d00 msgr2=0x55ae53617670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:48.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.406+0000 7fcc385b0b80 1 --2- 172.21.15.100:0/2730882332 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae53619d00 0x55ae53617670 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fcc20009d90 tx=0x7fcc20009270 comp rx=0 tx=0).stop 2024-05-10T11:23:48.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.406+0000 7fcc385b0b80 1 -- 172.21.15.100:0/2730882332 shutdown_connections 2024-05-10T11:23:48.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.406+0000 7fcc385b0b80 1 --2- 172.21.15.100:0/2730882332 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae53619d00 0x55ae53617670 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:48.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.406+0000 7fcc385b0b80 1 --2- 172.21.15.100:0/2730882332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ae53615550 0x55ae5361eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:48.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.406+0000 7fcc385b0b80 1 --2- 172.21.15.100:0/2730882332 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ae53615e70 0x55ae5362afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:48.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.406+0000 7fcc385b0b80 1 -- 172.21.15.100:0/2730882332 >> 172.21.15.100:0/2730882332 conn(0x55ae53541180 msgr2=0x55ae5360a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:23:48.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.406+0000 7fcc385b0b80 1 -- 172.21.15.100:0/2730882332 shutdown_connections 2024-05-10T11:23:48.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.406+0000 7fcc385b0b80 1 -- 172.21.15.100:0/2730882332 wait complete. 2024-05-10T11:23:48.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.407+0000 7fcc385b0b80 1 -- start start 2024-05-10T11:23:48.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.407+0000 7fcc385b0b80 1 Processor -- start 2024-05-10T11:23:48.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.407+0000 7fcc385b0b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ae53615550 0x55ae53627920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:48.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.407+0000 7fcc385b0b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ae53615e70 0x55ae53622ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:48.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.407+0000 7fcc385b0b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae53619d00 0x55ae53623260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:48.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.407+0000 7fcc385b0b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fcc281af860 con 0x55ae53619d00 2024-05-10T11:23:48.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.407+0000 7fcc385b0b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcc281b0cd0 con 0x55ae53615550 2024-05-10T11:23:48.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.407+0000 7fcc385b0b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcc281aff30 con 0x55ae53615e70 2024-05-10T11:23:48.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.407+0000 7fcc385b0b80 10 client.0 _ll_register_callbacks cb 0x55ae534ff9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:23:48.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.407+0000 7fcc27fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae53619d00 0x55ae53623260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:48.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.407+0000 7fcc26ffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ae53615e70 0x55ae53622ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:48.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.407+0000 7fcc27fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae53619d00 0x55ae53623260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47486/0 (socket says 172.21.15.100:47486) 2024-05-10T11:23:48.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.407+0000 7fcc27fff700 1 -- 172.21.15.100:0/2320931045 learned_addr learned my addr 172.21.15.100:0/2320931045 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:23:48.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.408+0000 7fcc277fe700 1 --2- 172.21.15.100:0/2320931045 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ae53615550 0x55ae53627920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:48.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.408+0000 7fcc27fff700 1 -- 172.21.15.100:0/2320931045 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ae53615e70 msgr2=0x55ae53622ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:48.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.408+0000 7fcc27fff700 1 --2- 172.21.15.100:0/2320931045 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ae53615e70 0x55ae53622ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:48.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.408+0000 7fcc27fff700 1 -- 172.21.15.100:0/2320931045 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ae53615550 msgr2=0x55ae53627920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:48.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.408+0000 7fcc27fff700 1 --2- 172.21.15.100:0/2320931045 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ae53615550 0x55ae53627920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:48.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.408+0000 7fcc27fff700 1 -- 172.21.15.100:0/2320931045 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae53623aa0 con 0x55ae53619d00 2024-05-10T11:23:48.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.408+0000 7fcc26ffd700 1 --2- 172.21.15.100:0/2320931045 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ae53615e70 0x55ae53622ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:23:48.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.408+0000 7fcc27fff700 1 --2- 172.21.15.100:0/2320931045 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae53619d00 0x55ae53623260 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fcc2000c010 tx=0x7fcc20014ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71570f104fe4195 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:23:48.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.408+0000 7fcc06ffd700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:23:48.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.408+0000 7fcc06ffd700 1 -- 172.21.15.100:0/2320931045 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc20013680 con 0x55ae53619d00 2024-05-10T11:23:48.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.408+0000 7fcc06ffd700 1 -- 172.21.15.100:0/2320931045 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fcc20013820 con 0x55ae53619d00 2024-05-10T11:23:48.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.408+0000 7fcc385b0b80 1 -- 172.21.15.100:0/2320931045 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc20019050 con 0x55ae53619d00 2024-05-10T11:23:48.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.408+0000 7fcc06ffd700 1 -- 172.21.15.100:0/2320931045 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fcc2003a050 con 0x55ae53619d00 2024-05-10T11:23:48.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.408+0000 7fcc385b0b80 10 client.16464 fetch_fsmap learned FSMap version 31 2024-05-10T11:23:48.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.408+0000 7fcc385b0b80 1 -- 172.21.15.100:0/2320931045 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55ae536997b0 con 0x55ae53619d00 2024-05-10T11:23:48.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.409+0000 7fcc06ffd700 1 -- 172.21.15.100:0/2320931045 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcc20019050 con 0x55ae53619d00 2024-05-10T11:23:48.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.409+0000 7fcc06ffd700 1 -- 172.21.15.100:0/2320931045 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fcc20026030 con 0x55ae53619d00 2024-05-10T11:23:48.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.409+0000 7fcc385b0b80 10 client.16464 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:23:48.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.409+0000 7fcc385b0b80 10 client.16464 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:23:48.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.409+0000 7fcc385b0b80 20 client.16464 populate_metadata read hostname 'smithi100' 2024-05-10T11:23:48.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.409+0000 7fcc385b0b80 1 --2- 172.21.15.100:0/2320931045 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ae53699a80 0x55ae5369be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:48.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.409+0000 7fcc385b0b80 4 client.16464 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T11:23:48.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.409+0000 7fcc385b0b80 1 -- 172.21.15.100:0/2320931045 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ae535752c0 con 0x55ae53699a80 2024-05-10T11:23:48.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.409+0000 7fcc277fe700 1 --2- 172.21.15.100:0/2320931045 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ae53699a80 0x55ae5369be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:48.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.409+0000 7fcc06ffd700 10 client.16464 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:23:48.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.409+0000 7fcc277fe700 1 --2- 172.21.15.100:0/2320931045 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ae53699a80 0x55ae5369be70 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3c4fee1056239c0 server_cookie=99525cbc7745accd in_seq=0 out_seq=0 2024-05-10T11:23:48.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.411+0000 7fcc06ffd700 1 -- 172.21.15.100:0/2320931045 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ae535752c0 con 0x55ae53699a80 2024-05-10T11:23:48.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.411+0000 7fcc06ffd700 10 client.16464 handle_command_reply: tid=0 2024-05-10T11:23:48.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.411+0000 7fcc385b0b80 1 client.16464 shutdown 2024-05-10T11:23:48.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.411+0000 7fcc385b0b80 1 -- 172.21.15.100:0/2320931045 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae53619d00 msgr2=0x55ae53623260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:48.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.411+0000 7fcc385b0b80 1 --2- 172.21.15.100:0/2320931045 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae53619d00 0x55ae53623260 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fcc2000c010 tx=0x7fcc20014ae0 comp rx=0 tx=0).stop 2024-05-10T11:23:48.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.411+0000 7fcc385b0b80 1 -- 172.21.15.100:0/2320931045 shutdown_connections 2024-05-10T11:23:48.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.411+0000 7fcc385b0b80 1 --2- 172.21.15.100:0/2320931045 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ae53699a80 0x55ae5369be70 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:48.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.411+0000 7fcc06ffd700 0 client.16464 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:23:48.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.411+0000 7fcc385b0b80 1 --2- 172.21.15.100:0/2320931045 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae53619d00 0x55ae53623260 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:48.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.411+0000 7fcc385b0b80 1 --2- 172.21.15.100:0/2320931045 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ae53615550 0x55ae53627920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:48.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.411+0000 7fcc385b0b80 1 --2- 172.21.15.100:0/2320931045 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ae53615e70 0x55ae53622ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:48.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.411+0000 7fcc385b0b80 1 -- 172.21.15.100:0/2320931045 >> 172.21.15.100:0/2320931045 conn(0x55ae53541180 msgr2=0x55ae53614360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:23:48.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.412+0000 7fcc385b0b80 1 -- 172.21.15.100:0/2320931045 shutdown_connections 2024-05-10T11:23:48.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.412+0000 7fcc385b0b80 1 -- 172.21.15.100:0/2320931045 wait complete. 2024-05-10T11:23:48.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.412+0000 7fcc385b0b80 20 client.16464 trim_cache size 0 max 16384 2024-05-10T11:23:48.417 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:23:48.417 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-05-10T11:23:48.417 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "b6218ccb-7442-4715-a85d-c45a328d77ee", 2024-05-10T11:23:48.417 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-05-10T11:23:48.417 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:23:48.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.412+0000 7fcc2dd1d700 1 -- 172.21.15.100:0/1951705279 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fcc100b2880 msgr2=0x7fcc100b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:48.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.412+0000 7fcc2dd1d700 1 --2- 172.21.15.100:0/1951705279 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fcc100b2880 0x7fcc100b4d40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fcc28079980 tx=0x7fcc1c01a040 comp rx=0 tx=0).stop 2024-05-10T11:23:48.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.412+0000 7fcc2dd1d700 1 -- 172.21.15.100:0/1951705279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc28102390 msgr2=0x7fcc2810fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:48.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.412+0000 7fcc2dd1d700 1 --2- 172.21.15.100:0/1951705279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc28102390 0x7fcc2810fcd0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcc1800e9c0 tx=0x7fcc1800ecd0 comp rx=0 tx=0).stop 2024-05-10T11:23:48.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.413+0000 7fcc2dd1d700 1 -- 172.21.15.100:0/1951705279 shutdown_connections 2024-05-10T11:23:48.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.413+0000 7fcc2dd1d700 1 --2- 172.21.15.100:0/1951705279 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fcc100b2880 0x7fcc100b4d40 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:48.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.413+0000 7fcc2dd1d700 1 --2- 172.21.15.100:0/1951705279 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcc28108400 0x7fcc28110750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:48.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.413+0000 7fcc2dd1d700 1 --2- 172.21.15.100:0/1951705279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc28102390 0x7fcc2810fcd0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:48.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.413+0000 7fcc2dd1d700 1 --2- 172.21.15.100:0/1951705279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc28102d50 0x7fcc28110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:48.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.413+0000 7fcc2dd1d700 1 -- 172.21.15.100:0/1951705279 >> 172.21.15.100:0/1951705279 conn(0x7fcc28079310 msgr2=0x7fcc281010c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:23:48.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.413+0000 7fcc2dd1d700 1 -- 172.21.15.100:0/1951705279 shutdown_connections 2024-05-10T11:23:48.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:48.413+0000 7fcc2dd1d700 1 -- 172.21.15.100:0/1951705279 wait complete. 2024-05-10T11:23:48.971 INFO:tasks.workunit.client.0.smithi100.stdout: 1 117068 2.78 MB/sec execute 103 sec latency 24.607 ms 2024-05-10T11:23:49.138 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:48 smithi100 ceph-mon[30296]: scrub queued for path: / 2024-05-10T11:23:49.138 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:48 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:23:49.139 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:48 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:23:49.139 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:48 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:23:49.139 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:48 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:23:49.139 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:48 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:23:49.139 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:48 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:23:49.139 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:48 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:23:49.139 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:48 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:23:49.139 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:48 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:23:49.139 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:48 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:23:49.139 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:48 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:23:49.139 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:48 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:23:49.139 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:48 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:23:49.139 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:48 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:23:49.139 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:48 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:23:49.167 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T11:23:49.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:48 smithi178 ceph-mon[33359]: scrub queued for path: / 2024-05-10T11:23:49.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:48 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:23:49.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:48 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:23:49.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:48 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:23:49.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:48 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:23:49.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:48 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:23:49.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:48 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:23:49.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:48 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:23:49.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:48 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:23:49.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:48 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:23:49.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:48 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:23:49.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:48 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:23:49.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:48 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:23:49.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:48 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:23:49.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:48 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:23:49.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:48 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:23:49.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:48 smithi188 ceph-mon[34075]: scrub queued for path: / 2024-05-10T11:23:49.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:48 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:23:49.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:48 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:23:49.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:48 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:23:49.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:48 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:23:49.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:48 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:23:49.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:48 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:23:49.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:48 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:23:49.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:48 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:23:49.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:48 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:23:49.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:48 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:23:49.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:48 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:23:49.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:48 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:23:49.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:48 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:23:49.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:48 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:23:49.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:48 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:23:49.436 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:23:49.971 INFO:tasks.workunit.client.0.smithi100.stdout: 1 117429 2.77 MB/sec execute 104 sec latency 15.157 ms 2024-05-10T11:23:50.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:49 smithi178 ceph-mon[33359]: pgmap v583: 129 pgs: 129 active+clean; 57 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.2 MiB/s wr, 132 op/s 2024-05-10T11:23:50.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:49 smithi188 ceph-mon[34075]: pgmap v583: 129 pgs: 129 active+clean; 57 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.2 MiB/s wr, 132 op/s 2024-05-10T11:23:50.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:49 smithi100 ceph-mon[30296]: pgmap v583: 129 pgs: 129 active+clean; 57 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.2 MiB/s wr, 132 op/s 2024-05-10T11:23:50.971 INFO:tasks.workunit.client.0.smithi100.stdout: 1 117746 2.74 MB/sec execute 105 sec latency 21.288 ms 2024-05-10T11:23:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.189+0000 7fa0ed862640 1 -- 172.21.15.100:0/779169520 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0e8102b20 msgr2=0x7fa0e8102fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.189+0000 7fa0ed862640 1 --2- 172.21.15.100:0/779169520 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0e8102b20 0x7fa0e8102fa0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fa0d0009ca0 tx=0x7fa0d002a8c0 comp rx=0 tx=0).stop 2024-05-10T11:23:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.189+0000 7fa0ed862640 1 -- 172.21.15.100:0/779169520 shutdown_connections 2024-05-10T11:23:51.190 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.189+0000 7fa0ed862640 1 --2- 172.21.15.100:0/779169520 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0e81034e0 0x7fa0e8109d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:51.190 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.189+0000 7fa0ed862640 1 --2- 172.21.15.100:0/779169520 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0e8102b20 0x7fa0e8102fa0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:51.190 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.189+0000 7fa0ed862640 1 --2- 172.21.15.100:0/779169520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0e8101930 0x7fa0e8101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:51.190 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.189+0000 7fa0ed862640 1 -- 172.21.15.100:0/779169520 >> 172.21.15.100:0/779169520 conn(0x7fa0e80fcea0 msgr2=0x7fa0e80ff2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:23:51.190 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.189+0000 7fa0ed862640 1 -- 172.21.15.100:0/779169520 shutdown_connections 2024-05-10T11:23:51.190 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.189+0000 7fa0ed862640 1 -- 172.21.15.100:0/779169520 wait complete. 2024-05-10T11:23:51.190 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.189+0000 7fa0ed862640 1 Processor -- start 2024-05-10T11:23:51.192 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.190+0000 7fa0ed862640 1 -- start start 2024-05-10T11:23:51.192 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.190+0000 7fa0ed862640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0e8101930 0x7fa0e8198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:51.192 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.190+0000 7fa0ed862640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0e8102b20 0x7fa0e8199000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:51.192 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.190+0000 7fa0ed862640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0e81034e0 0x7fa0e81a0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:51.192 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.190+0000 7fa0ed862640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e810d360 con 0x7fa0e8101930 2024-05-10T11:23:51.192 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.190+0000 7fa0ed862640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e810d1e0 con 0x7fa0e81034e0 2024-05-10T11:23:51.192 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.190+0000 7fa0ed862640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e810d4e0 con 0x7fa0e8102b20 2024-05-10T11:23:51.192 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.190+0000 7fa0e6ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0e8101930 0x7fa0e8198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:51.192 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.190+0000 7fa0e77fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0e81034e0 0x7fa0e81a0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:51.192 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.190+0000 7fa0e6ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0e8101930 0x7fa0e8198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47512/0 (socket says 172.21.15.100:47512) 2024-05-10T11:23:51.192 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.190+0000 7fa0e77fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0e81034e0 0x7fa0e81a0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:48636/0 (socket says 172.21.15.100:48636) 2024-05-10T11:23:51.193 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.190+0000 7fa0e6ffd640 1 -- 172.21.15.100:0/1439925267 learned_addr learned my addr 172.21.15.100:0/1439925267 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:23:51.193 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.190+0000 7fa0e67fc640 1 --2- 172.21.15.100:0/1439925267 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0e8102b20 0x7fa0e8199000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:51.193 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.191+0000 7fa0e6ffd640 1 -- 172.21.15.100:0/1439925267 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0e8102b20 msgr2=0x7fa0e8199000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:51.193 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.191+0000 7fa0e6ffd640 1 --2- 172.21.15.100:0/1439925267 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0e8102b20 0x7fa0e8199000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:51.193 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.191+0000 7fa0e6ffd640 1 -- 172.21.15.100:0/1439925267 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0e81034e0 msgr2=0x7fa0e81a0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:51.193 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.191+0000 7fa0e6ffd640 1 --2- 172.21.15.100:0/1439925267 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0e81034e0 0x7fa0e81a0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:51.193 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.191+0000 7fa0e6ffd640 1 -- 172.21.15.100:0/1439925267 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0e81a06e0 con 0x7fa0e8101930 2024-05-10T11:23:51.193 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.191+0000 7fa0e77fe640 1 --2- 172.21.15.100:0/1439925267 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0e81034e0 0x7fa0e81a0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:23:51.193 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.191+0000 7fa0e6ffd640 1 --2- 172.21.15.100:0/1439925267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0e8101930 0x7fa0e8198ac0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fa0e000e9c0 tx=0x7fa0e000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:23:51.193 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.191+0000 7fa0ec860640 1 -- 172.21.15.100:0/1439925267 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e000cd80 con 0x7fa0e8101930 2024-05-10T11:23:51.193 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.191+0000 7fa0ec860640 1 -- 172.21.15.100:0/1439925267 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa0e00044b0 con 0x7fa0e8101930 2024-05-10T11:23:51.193 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.191+0000 7fa0ed862640 1 -- 172.21.15.100:0/1439925267 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0e81a09d0 con 0x7fa0e8101930 2024-05-10T11:23:51.193 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.191+0000 7fa0ec860640 1 -- 172.21.15.100:0/1439925267 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e0010430 con 0x7fa0e8101930 2024-05-10T11:23:51.193 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.191+0000 7fa0ed862640 1 -- 172.21.15.100:0/1439925267 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0e81a0f70 con 0x7fa0e8101930 2024-05-10T11:23:51.196 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.194+0000 7fa0f0577740 1 Processor -- start 2024-05-10T11:23:51.197 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.194+0000 7fa0f0577740 1 -- start start 2024-05-10T11:23:51.197 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.195+0000 7fa0f0577740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55947ed74db0 0x55947ed751d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:51.197 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.195+0000 7fa0f0577740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55947ed757a0 0x55947ed7a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:51.197 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.195+0000 7fa0f0577740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55947ed7a910 0x55947ed7cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:51.197 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.195+0000 7fa0f0577740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55947ecd95d0 con 0x55947ed74db0 2024-05-10T11:23:51.197 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.195+0000 7fa0f0577740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55947ec92890 con 0x55947ed7a910 2024-05-10T11:23:51.197 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.195+0000 7fa0f0577740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55947ecd5590 con 0x55947ed757a0 2024-05-10T11:23:51.197 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.195+0000 7fa0e77fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55947ed74db0 0x55947ed751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:51.197 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.195+0000 7fa0e77fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55947ed74db0 0x55947ed751d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47518/0 (socket says 172.21.15.100:47518) 2024-05-10T11:23:51.197 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.195+0000 7fa0e77fe640 1 -- 172.21.15.100:0/2135991662 learned_addr learned my addr 172.21.15.100:0/2135991662 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:23:51.197 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.195+0000 7fa0e67fc640 1 --2- 172.21.15.100:0/2135991662 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55947ed7a910 0x55947ed7cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:51.197 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.195+0000 7fa0ec860640 1 -- 172.21.15.100:0/1439925267 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fa0e0003f00 con 0x7fa0e8101930 2024-05-10T11:23:51.198 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.195+0000 7fa0e77fe640 1 -- 172.21.15.100:0/2135991662 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55947ed757a0 msgr2=0x55947ed7a2a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T11:23:51.198 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.195+0000 7fa0e77fe640 1 --2- 172.21.15.100:0/2135991662 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55947ed757a0 0x55947ed7a2a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:51.198 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.195+0000 7fa0e77fe640 1 -- 172.21.15.100:0/2135991662 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55947ed7a910 msgr2=0x55947ed7cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:51.198 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.195+0000 7fa0e77fe640 1 --2- 172.21.15.100:0/2135991662 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55947ed7a910 0x55947ed7cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:51.198 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.195+0000 7fa0e77fe640 1 -- 172.21.15.100:0/2135991662 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55947e9eb210 con 0x55947ed74db0 2024-05-10T11:23:51.198 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.195+0000 7fa0e67fc640 1 --2- 172.21.15.100:0/2135991662 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55947ed7a910 0x55947ed7cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:23:51.198 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.195+0000 7fa0e77fe640 1 --2- 172.21.15.100:0/2135991662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55947ed74db0 0x55947ed751d0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fa0d8011540 tx=0x7fa0d8011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c54a7a24240b1544 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:23:51.198 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.196+0000 7fa0ddffb640 1 -- 172.21.15.100:0/2135991662 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d800f5c0 con 0x55947ed74db0 2024-05-10T11:23:51.198 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.196+0000 7fa0ddffb640 1 -- 172.21.15.100:0/2135991662 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa0d800fbf0 con 0x55947ed74db0 2024-05-10T11:23:51.198 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.196+0000 7fa0ddffb640 1 -- 172.21.15.100:0/2135991662 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d8019b60 con 0x55947ed74db0 2024-05-10T11:23:51.198 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.196+0000 7fa0ec860640 1 --2- 172.21.15.100:0/1439925267 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa0d40b2920 0x7fa0d40b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:51.198 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.196+0000 7fa0e6ffd640 1 --2- 172.21.15.100:0/1439925267 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa0d40b2920 0x7fa0d40b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:51.198 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.196+0000 7fa0f0577740 1 -- 172.21.15.100:0/2135991662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55947ed74db0 msgr2=0x55947ed751d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.196+0000 7fa0f0577740 1 --2- 172.21.15.100:0/2135991662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55947ed74db0 0x55947ed751d0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fa0d8011540 tx=0x7fa0d8011a00 comp rx=0 tx=0).stop 2024-05-10T11:23:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.197+0000 7fa0ec860640 1 -- 172.21.15.100:0/1439925267 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fa0e00e28e0 con 0x7fa0e8101930 2024-05-10T11:23:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.197+0000 7fa0f0577740 1 -- 172.21.15.100:0/2135991662 shutdown_connections 2024-05-10T11:23:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.197+0000 7fa0f0577740 1 --2- 172.21.15.100:0/2135991662 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55947ed7a910 0x55947ed7cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.197+0000 7fa0f0577740 1 --2- 172.21.15.100:0/2135991662 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55947ed757a0 0x55947ed7a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.197+0000 7fa0f0577740 1 --2- 172.21.15.100:0/2135991662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55947ed74db0 0x55947ed751d0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.197+0000 7fa0f0577740 1 -- 172.21.15.100:0/2135991662 >> 172.21.15.100:0/2135991662 conn(0x55947ed70420 msgr2=0x55947ed70820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:23:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.197+0000 7fa0f0577740 1 -- 172.21.15.100:0/2135991662 shutdown_connections 2024-05-10T11:23:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.197+0000 7fa0f0577740 1 -- 172.21.15.100:0/2135991662 wait complete. 2024-05-10T11:23:51.203 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.197+0000 7fa0f0577740 1 -- start start 2024-05-10T11:23:51.203 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.197+0000 7fa0e6ffd640 1 --2- 172.21.15.100:0/1439925267 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa0d40b2920 0x7fa0d40b4de0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa0e00e00d0 tx=0x7fa0e0005880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:23:51.203 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.197+0000 7fa0f0577740 1 Processor -- start 2024-05-10T11:23:51.203 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.198+0000 7fa0f0577740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55947ed74db0 0x55947ed7f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:51.203 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.198+0000 7fa0f0577740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55947ed757a0 0x55947ed7f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:51.203 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.198+0000 7fa0f0577740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55947ed7a910 0x55947ed865a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:51.203 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.198+0000 7fa0f0577740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55947ed7d8c0 con 0x55947ed74db0 2024-05-10T11:23:51.203 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.198+0000 7fa0f0577740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55947ec96370 con 0x55947ed7a910 2024-05-10T11:23:51.204 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.198+0000 7fa0f0577740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55947ed7da40 con 0x55947ed757a0 2024-05-10T11:23:51.204 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.198+0000 7fa0e77fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55947ed74db0 0x55947ed7f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:51.204 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.198+0000 7fa0e77fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55947ed74db0 0x55947ed7f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47522/0 (socket says 172.21.15.100:47522) 2024-05-10T11:23:51.204 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.198+0000 7fa0e77fe640 1 -- 172.21.15.100:0/3659581976 learned_addr learned my addr 172.21.15.100:0/3659581976 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:23:51.204 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.198+0000 7fa0f0577740 10 client.0 _ll_register_callbacks cb 0x55947e9eaa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:23:51.204 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.198+0000 7fa0e67fc640 1 --2- 172.21.15.100:0/3659581976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55947ed757a0 0x55947ed7f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:51.204 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.198+0000 7fa0e77fe640 1 --2- 172.21.15.100:0/3659581976 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55947ed7a910 0x55947ed865a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:51.204 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.198+0000 7fa0e77fe640 1 -- 172.21.15.100:0/3659581976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55947ed757a0 msgr2=0x55947ed7f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:51.204 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.198+0000 7fa0e77fe640 1 --2- 172.21.15.100:0/3659581976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55947ed757a0 0x55947ed7f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:51.204 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.198+0000 7fa0e77fe640 1 -- 172.21.15.100:0/3659581976 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55947ed7a910 msgr2=0x55947ed865a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:51.204 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.198+0000 7fa0e77fe640 1 --2- 172.21.15.100:0/3659581976 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55947ed7a910 0x55947ed865a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:51.204 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.198+0000 7fa0e77fe640 1 -- 172.21.15.100:0/3659581976 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55947eab7620 con 0x55947ed74db0 2024-05-10T11:23:51.204 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.198+0000 7fa0e67fc640 1 --2- 172.21.15.100:0/3659581976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55947ed757a0 0x55947ed7f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:23:51.204 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.198+0000 7fa0e77fe640 1 --2- 172.21.15.100:0/3659581976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55947ed74db0 0x55947ed7f020 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fa0d8009e30 tx=0x7fa0d8009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=925f357187e0e8fe server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:23:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.198+0000 7fa0c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:23:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.199+0000 7fa0c6ffd640 1 -- 172.21.15.100:0/3659581976 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d8004040 con 0x55947ed74db0 2024-05-10T11:23:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.199+0000 7fa0c6ffd640 1 -- 172.21.15.100:0/3659581976 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fa0d80041e0 con 0x55947ed74db0 2024-05-10T11:23:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.199+0000 7fa0f0577740 1 -- 172.21.15.100:0/3659581976 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0d800fbf0 con 0x55947ed74db0 2024-05-10T11:23:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.199+0000 7fa0c6ffd640 1 -- 172.21.15.100:0/3659581976 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa0d8033050 con 0x55947ed74db0 2024-05-10T11:23:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.199+0000 7fa0f0577740 10 client.16476 fetch_fsmap learned FSMap version 31 2024-05-10T11:23:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.199+0000 7fa0f0577740 1 -- 172.21.15.100:0/3659581976 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55947ed89580 con 0x55947ed74db0 2024-05-10T11:23:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.199+0000 7fa0c6ffd640 1 -- 172.21.15.100:0/3659581976 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa0d800fbf0 con 0x55947ed74db0 2024-05-10T11:23:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.199+0000 7fa0c6ffd640 1 -- 172.21.15.100:0/3659581976 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fa0d80035f0 con 0x55947ed74db0 2024-05-10T11:23:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.199+0000 7fa0f0577740 10 client.16476 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:23:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.199+0000 7fa0f0577740 10 client.16476 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:23:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.199+0000 7fa0f0577740 20 client.16476 populate_metadata read hostname 'smithi100' 2024-05-10T11:23:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.199+0000 7fa0f0577740 1 --2- 172.21.15.100:0/3659581976 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55947ed8aa50 0x55947ed8ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.199+0000 7fa0f0577740 4 client.16476 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:23:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.199+0000 7fa0f0577740 1 -- 172.21.15.100:0/3659581976 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55947ece71e0 con 0x55947ed8aa50 2024-05-10T11:23:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.200+0000 7fa0e67fc640 1 --2- 172.21.15.100:0/3659581976 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55947ed8aa50 0x55947ed8ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.200+0000 7fa0c6ffd640 10 client.16476 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:23:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.200+0000 7fa0e67fc640 1 --2- 172.21.15.100:0/3659581976 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55947ed8aa50 0x55947ed8ce40 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb258f90b156349b server_cookie=8e3b671782ef30bb in_seq=0 out_seq=0 2024-05-10T11:23:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.201+0000 7fa0c6ffd640 1 -- 172.21.15.100:0/3659581976 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55947ece71e0 con 0x55947ed8aa50 2024-05-10T11:23:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.201+0000 7fa0c6ffd640 10 client.16476 handle_command_reply: tid=0 2024-05-10T11:23:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.201+0000 7fa0f0577740 1 client.16476 shutdown 2024-05-10T11:23:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.201+0000 7fa0f0577740 1 -- 172.21.15.100:0/3659581976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55947ed74db0 msgr2=0x55947ed7f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.201+0000 7fa0f0577740 1 --2- 172.21.15.100:0/3659581976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55947ed74db0 0x55947ed7f020 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fa0d8009e30 tx=0x7fa0d8009310 comp rx=0 tx=0).stop 2024-05-10T11:23:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.201+0000 7fa0f0577740 1 -- 172.21.15.100:0/3659581976 shutdown_connections 2024-05-10T11:23:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.201+0000 7fa0f0577740 1 --2- 172.21.15.100:0/3659581976 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55947ed8aa50 0x55947ed8ce40 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.201+0000 7fa0f0577740 1 --2- 172.21.15.100:0/3659581976 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55947ed7a910 0x55947ed865a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.201+0000 7fa0c6ffd640 0 client.16476 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:23:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.201+0000 7fa0f0577740 1 --2- 172.21.15.100:0/3659581976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55947ed757a0 0x55947ed7f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.201+0000 7fa0f0577740 1 --2- 172.21.15.100:0/3659581976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55947ed74db0 0x55947ed7f020 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.201+0000 7fa0f0577740 1 -- 172.21.15.100:0/3659581976 >> 172.21.15.100:0/3659581976 conn(0x55947ed70420 msgr2=0x55947ed6f580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:23:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.201+0000 7fa0f0577740 1 -- 172.21.15.100:0/3659581976 shutdown_connections 2024-05-10T11:23:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.201+0000 7fa0f0577740 1 -- 172.21.15.100:0/3659581976 wait complete. 2024-05-10T11:23:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.201+0000 7fa0f0577740 20 client.16476 trim_cache size 0 max 16384 2024-05-10T11:23:51.222 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.221+0000 7fa0f0577740 1 Processor -- start 2024-05-10T11:23:51.222 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.221+0000 7fa0f0577740 1 -- start start 2024-05-10T11:23:51.222 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.221+0000 7fa0f0577740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55947ed74db0 0x55947ed7f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:51.222 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.221+0000 7fa0f0577740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55947ed757a0 0x55947ed73810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:51.222 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.221+0000 7fa0f0577740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55947ed7a910 0x55947ed73d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:51.222 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.221+0000 7fa0f0577740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e81afe00 con 0x55947ed7a910 2024-05-10T11:23:51.222 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.221+0000 7fa0f0577740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e81ad6d0 con 0x55947ed757a0 2024-05-10T11:23:51.222 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.221+0000 7fa0f0577740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e81ab520 con 0x55947ed74db0 2024-05-10T11:23:51.222 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.222+0000 7fa0e77fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55947ed7a910 0x55947ed73d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:51.222 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.222+0000 7fa0e77fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55947ed7a910 0x55947ed73d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47524/0 (socket says 172.21.15.100:47524) 2024-05-10T11:23:51.222 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.222+0000 7fa0e77fe640 1 -- 172.21.15.100:0/112088060 learned_addr learned my addr 172.21.15.100:0/112088060 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:23:51.222 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.222+0000 7fa0e67fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55947ed757a0 0x55947ed73810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:51.222 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.222+0000 7fa0e77fe640 1 --2- 172.21.15.100:0/112088060 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55947ed74db0 0x55947ed7f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:51.225 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.222+0000 7fa0e67fc640 1 -- 172.21.15.100:0/112088060 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55947ed74db0 msgr2=0x55947ed7f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:51.225 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.222+0000 7fa0e67fc640 1 --2- 172.21.15.100:0/112088060 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55947ed74db0 0x55947ed7f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:51.225 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.222+0000 7fa0e67fc640 1 -- 172.21.15.100:0/112088060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55947ed7a910 msgr2=0x55947ed73d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:51.225 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.222+0000 7fa0e67fc640 1 --2- 172.21.15.100:0/112088060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55947ed7a910 0x55947ed73d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:51.225 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.222+0000 7fa0e67fc640 1 -- 172.21.15.100:0/112088060 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55947ed8d1b0 con 0x55947ed757a0 2024-05-10T11:23:51.225 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.222+0000 7fa0e77fe640 1 --2- 172.21.15.100:0/112088060 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55947ed74db0 0x55947ed7f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:23:51.226 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.222+0000 7fa0e67fc640 1 --2- 172.21.15.100:0/112088060 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55947ed757a0 0x55947ed73810 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa0d002aeb0 tx=0x7fa0d0031000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a9eb8440a2dc544 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:23:51.226 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.223+0000 7fa0dd7fa640 1 -- 172.21.15.100:0/112088060 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d0031e40 con 0x55947ed757a0 2024-05-10T11:23:51.226 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.223+0000 7fa0dd7fa640 1 -- 172.21.15.100:0/112088060 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa0d004e070 con 0x55947ed757a0 2024-05-10T11:23:51.226 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.223+0000 7fa0dd7fa640 1 -- 172.21.15.100:0/112088060 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d0004840 con 0x55947ed757a0 2024-05-10T11:23:51.226 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.223+0000 7fa0f0577740 1 -- 172.21.15.100:0/112088060 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55947ed757a0 msgr2=0x55947ed73810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:51.226 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.223+0000 7fa0f0577740 1 --2- 172.21.15.100:0/112088060 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55947ed757a0 0x55947ed73810 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa0d002aeb0 tx=0x7fa0d0031000 comp rx=0 tx=0).stop 2024-05-10T11:23:51.226 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.223+0000 7fa0f0577740 1 -- 172.21.15.100:0/112088060 shutdown_connections 2024-05-10T11:23:51.226 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.223+0000 7fa0f0577740 1 --2- 172.21.15.100:0/112088060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55947ed7a910 0x55947ed73d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:51.226 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.223+0000 7fa0f0577740 1 --2- 172.21.15.100:0/112088060 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55947ed757a0 0x55947ed73810 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:51.226 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.223+0000 7fa0f0577740 1 --2- 172.21.15.100:0/112088060 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55947ed74db0 0x55947ed7f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:51.226 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.223+0000 7fa0f0577740 1 -- 172.21.15.100:0/112088060 >> 172.21.15.100:0/112088060 conn(0x55947ed70420 msgr2=0x55947ed73100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:23:51.226 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.224+0000 7fa0f0577740 1 -- 172.21.15.100:0/112088060 shutdown_connections 2024-05-10T11:23:51.226 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.224+0000 7fa0f0577740 1 -- 172.21.15.100:0/112088060 wait complete. 2024-05-10T11:23:51.227 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.224+0000 7fa0f0577740 1 -- start start 2024-05-10T11:23:51.227 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.224+0000 7fa0f0577740 1 Processor -- start 2024-05-10T11:23:51.227 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.225+0000 7fa0f0577740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55947ed74db0 0x55947edf1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:51.227 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.225+0000 7fa0f0577740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55947ed757a0 0x55947edf23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:51.227 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.225+0000 7fa0f0577740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55947ed7a910 0x55947ed845e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:51.227 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.225+0000 7fa0f0577740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e81abb90 con 0x55947ed7a910 2024-05-10T11:23:51.227 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.225+0000 7fa0f0577740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e81ad000 con 0x55947ed74db0 2024-05-10T11:23:51.227 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.225+0000 7fa0f0577740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e81ac260 con 0x55947ed757a0 2024-05-10T11:23:51.227 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.225+0000 7fa0f0577740 10 client.0 _ll_register_callbacks cb 0x7fa0d8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:23:51.227 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.225+0000 7fa0e77fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55947ed74db0 0x55947edf1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:51.227 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.225+0000 7fa0e77fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55947ed74db0 0x55947edf1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:48664/0 (socket says 172.21.15.100:48664) 2024-05-10T11:23:51.227 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.225+0000 7fa0e77fe640 1 -- 172.21.15.100:0/1911407164 learned_addr learned my addr 172.21.15.100:0/1911407164 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:23:51.227 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.225+0000 7fa0e77fe640 1 --2- 172.21.15.100:0/1911407164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55947ed7a910 0x55947ed845e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:51.227 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.225+0000 7fa0e67fc640 1 --2- 172.21.15.100:0/1911407164 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55947ed757a0 0x55947edf23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:51.229 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.225+0000 7fa0e77fe640 1 -- 172.21.15.100:0/1911407164 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55947ed757a0 msgr2=0x55947edf23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:51.229 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.225+0000 7fa0e77fe640 1 --2- 172.21.15.100:0/1911407164 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55947ed757a0 0x55947edf23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:51.229 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.225+0000 7fa0e77fe640 1 -- 172.21.15.100:0/1911407164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55947ed7a910 msgr2=0x55947ed845e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:51.229 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.225+0000 7fa0e77fe640 1 --2- 172.21.15.100:0/1911407164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55947ed7a910 0x55947ed845e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:51.229 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.225+0000 7fa0e77fe640 1 -- 172.21.15.100:0/1911407164 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55947ed84ba0 con 0x55947ed74db0 2024-05-10T11:23:51.229 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.226+0000 7fa0e77fe640 1 --2- 172.21.15.100:0/1911407164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55947ed74db0 0x55947edf1e70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa0d8013010 tx=0x7fa0d8019610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84d7549f168e26c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:23:51.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.226+0000 7fa0c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:23:51.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.226+0000 7fa0c6ffd640 1 -- 172.21.15.100:0/1911407164 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d8003820 con 0x55947ed74db0 2024-05-10T11:23:51.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.226+0000 7fa0c6ffd640 1 -- 172.21.15.100:0/1911407164 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fa0d8005030 con 0x55947ed74db0 2024-05-10T11:23:51.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.226+0000 7fa0f0577740 1 -- 172.21.15.100:0/1911407164 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0d004e070 con 0x55947ed74db0 2024-05-10T11:23:51.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.226+0000 7fa0c6ffd640 1 -- 172.21.15.100:0/1911407164 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa0d8005230 con 0x55947ed74db0 2024-05-10T11:23:51.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.226+0000 7fa0f0577740 10 client.25943 fetch_fsmap learned FSMap version 31 2024-05-10T11:23:51.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.226+0000 7fa0f0577740 1 -- 172.21.15.100:0/1911407164 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55947ed85eb0 con 0x55947ed74db0 2024-05-10T11:23:51.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.226+0000 7fa0c6ffd640 1 -- 172.21.15.100:0/1911407164 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa0d004e070 con 0x55947ed74db0 2024-05-10T11:23:51.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.227+0000 7fa0c6ffd640 1 -- 172.21.15.100:0/1911407164 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fa0d8019680 con 0x55947ed74db0 2024-05-10T11:23:51.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.227+0000 7fa0f0577740 10 client.25943 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:23:51.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.227+0000 7fa0f0577740 10 client.25943 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:23:51.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.227+0000 7fa0f0577740 20 client.25943 populate_metadata read hostname 'smithi100' 2024-05-10T11:23:51.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.227+0000 7fa0f0577740 1 --2- 172.21.15.100:0/1911407164 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55947ed8aa50 0x55947ee00160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:51.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.227+0000 7fa0f0577740 4 client.25943 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:23:51.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.227+0000 7fa0f0577740 1 -- 172.21.15.100:0/1911407164 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55947ece1050 con 0x55947ed8aa50 2024-05-10T11:23:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.227+0000 7fa0e67fc640 1 --2- 172.21.15.100:0/1911407164 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55947ed8aa50 0x55947ee00160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.227+0000 7fa0c6ffd640 10 client.25943 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:23:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.227+0000 7fa0e67fc640 1 --2- 172.21.15.100:0/1911407164 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55947ed8aa50 0x55947ee00160 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53eb6e0e3b60985f server_cookie=79663032e235cc83 in_seq=0 out_seq=0 2024-05-10T11:23:51.247 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.246+0000 7fa0c6ffd640 1 -- 172.21.15.100:0/1911407164 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55947ece1050 con 0x55947ed8aa50 2024-05-10T11:23:51.247 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.246+0000 7fa0c6ffd640 10 client.25943 handle_command_reply: tid=0 2024-05-10T11:23:51.247 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.246+0000 7fa0f0577740 1 client.25943 shutdown 2024-05-10T11:23:51.247 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.246+0000 7fa0f0577740 1 -- 172.21.15.100:0/1911407164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55947ed74db0 msgr2=0x55947edf1e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:51.247 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.246+0000 7fa0f0577740 1 --2- 172.21.15.100:0/1911407164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55947ed74db0 0x55947edf1e70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa0d8013010 tx=0x7fa0d8019610 comp rx=0 tx=0).stop 2024-05-10T11:23:51.247 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.246+0000 7fa0f0577740 1 -- 172.21.15.100:0/1911407164 shutdown_connections 2024-05-10T11:23:51.247 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.246+0000 7fa0f0577740 1 --2- 172.21.15.100:0/1911407164 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55947ed8aa50 0x55947ee00160 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:51.247 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.246+0000 7fa0f0577740 1 --2- 172.21.15.100:0/1911407164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55947ed7a910 0x55947ed845e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:51.247 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.246+0000 7fa0c6ffd640 0 client.25943 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:23:51.247 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.246+0000 7fa0f0577740 1 --2- 172.21.15.100:0/1911407164 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55947ed757a0 0x55947edf23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:51.247 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.246+0000 7fa0f0577740 1 --2- 172.21.15.100:0/1911407164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55947ed74db0 0x55947edf1e70 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:51.247 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.246+0000 7fa0f0577740 1 -- 172.21.15.100:0/1911407164 >> 172.21.15.100:0/1911407164 conn(0x55947ed70420 msgr2=0x55947ed73100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:23:51.248 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.247+0000 7fa0f0577740 1 -- 172.21.15.100:0/1911407164 shutdown_connections 2024-05-10T11:23:51.248 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.247+0000 7fa0f0577740 1 -- 172.21.15.100:0/1911407164 wait complete. 2024-05-10T11:23:51.248 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.247+0000 7fa0f0577740 20 client.25943 trim_cache size 0 max 16384 2024-05-10T11:23:51.248 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:23:51.248 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:23:51.248 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:23:51.248 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:23:51.249 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.247+0000 7fa0ed862640 1 -- 172.21.15.100:0/1439925267 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa0d40b2920 msgr2=0x7fa0d40b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:51.249 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.247+0000 7fa0ed862640 1 --2- 172.21.15.100:0/1439925267 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa0d40b2920 0x7fa0d40b4de0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa0e00e00d0 tx=0x7fa0e0005880 comp rx=0 tx=0).stop 2024-05-10T11:23:51.249 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.247+0000 7fa0ed862640 1 -- 172.21.15.100:0/1439925267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0e8101930 msgr2=0x7fa0e8198ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:51.249 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.247+0000 7fa0ed862640 1 --2- 172.21.15.100:0/1439925267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0e8101930 0x7fa0e8198ac0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fa0e000e9c0 tx=0x7fa0e000ee80 comp rx=0 tx=0).stop 2024-05-10T11:23:51.249 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.247+0000 7fa0ed862640 1 -- 172.21.15.100:0/1439925267 shutdown_connections 2024-05-10T11:23:51.249 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.247+0000 7fa0ed862640 1 --2- 172.21.15.100:0/1439925267 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa0d40b2920 0x7fa0d40b4de0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:51.249 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.247+0000 7fa0ed862640 1 --2- 172.21.15.100:0/1439925267 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0e81034e0 0x7fa0e81a0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:51.249 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.247+0000 7fa0ed862640 1 --2- 172.21.15.100:0/1439925267 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0e8102b20 0x7fa0e8199000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:51.250 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.247+0000 7fa0ed862640 1 --2- 172.21.15.100:0/1439925267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0e8101930 0x7fa0e8198ac0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:51.250 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.247+0000 7fa0ed862640 1 -- 172.21.15.100:0/1439925267 >> 172.21.15.100:0/1439925267 conn(0x7fa0e80fcea0 msgr2=0x7fa0e80fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:23:51.250 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.247+0000 7fa0ed862640 1 -- 172.21.15.100:0/1439925267 shutdown_connections 2024-05-10T11:23:51.250 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:23:51.248+0000 7fa0ed862640 1 -- 172.21.15.100:0/1439925267 wait complete. 2024-05-10T11:23:51.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.263+0000 7fa8646a5700 1 -- 172.21.15.100:0/987105817 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa85c102390 msgr2=0x7fa85c102810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:51.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.263+0000 7fa8646a5700 1 --2- 172.21.15.100:0/987105817 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa85c102390 0x7fa85c102810 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fa850009990 tx=0x7fa850009ca0 comp rx=0 tx=0).stop 2024-05-10T11:23:51.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.263+0000 7fa8646a5700 1 -- 172.21.15.100:0/987105817 shutdown_connections 2024-05-10T11:23:51.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.263+0000 7fa8646a5700 1 --2- 172.21.15.100:0/987105817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa85c102d50 0x7fa85c10f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:51.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.263+0000 7fa8646a5700 1 --2- 172.21.15.100:0/987105817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa85c108400 0x7fa85c1087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:51.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.263+0000 7fa8646a5700 1 --2- 172.21.15.100:0/987105817 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa85c102390 0x7fa85c102810 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:51.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.263+0000 7fa8646a5700 1 -- 172.21.15.100:0/987105817 >> 172.21.15.100:0/987105817 conn(0x7fa85c079310 msgr2=0x7fa85c079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:23:51.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.263+0000 7fa8646a5700 1 -- 172.21.15.100:0/987105817 shutdown_connections 2024-05-10T11:23:51.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.263+0000 7fa8646a5700 1 -- 172.21.15.100:0/987105817 wait complete. 2024-05-10T11:23:51.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.264+0000 7fa8646a5700 1 Processor -- start 2024-05-10T11:23:51.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.264+0000 7fa8646a5700 1 -- start start 2024-05-10T11:23:51.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.265+0000 7fa8646a5700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa85c102390 0x7fa85c19f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:51.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.265+0000 7fa8646a5700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa85c102d50 0x7fa85c19fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:51.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.265+0000 7fa8646a5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa85c108400 0x7fa85c1998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:51.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.265+0000 7fa8646a5700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa85c112400 con 0x7fa85c102d50 2024-05-10T11:23:51.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.265+0000 7fa8646a5700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa85c112280 con 0x7fa85c108400 2024-05-10T11:23:51.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.265+0000 7fa8646a5700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa85c112580 con 0x7fa85c102390 2024-05-10T11:23:51.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.265+0000 7fa861c40700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa85c102d50 0x7fa85c19fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:51.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.265+0000 7fa862441700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa85c102390 0x7fa85c19f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:51.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.265+0000 7fa861c40700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa85c102d50 0x7fa85c19fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47546/0 (socket says 172.21.15.100:47546) 2024-05-10T11:23:51.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.265+0000 7fa861c40700 1 -- 172.21.15.100:0/2826951767 learned_addr learned my addr 172.21.15.100:0/2826951767 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:23:51.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.265+0000 7fa862c42700 1 --2- 172.21.15.100:0/2826951767 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa85c108400 0x7fa85c1998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:51.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.265+0000 7fa861c40700 1 -- 172.21.15.100:0/2826951767 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa85c102390 msgr2=0x7fa85c19f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:51.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.265+0000 7fa861c40700 1 --2- 172.21.15.100:0/2826951767 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa85c102390 0x7fa85c19f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:51.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.265+0000 7fa861c40700 1 -- 172.21.15.100:0/2826951767 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa85c108400 msgr2=0x7fa85c1998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:51.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.265+0000 7fa861c40700 1 --2- 172.21.15.100:0/2826951767 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa85c108400 0x7fa85c1998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:51.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.265+0000 7fa861c40700 1 -- 172.21.15.100:0/2826951767 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa85c19a050 con 0x7fa85c102d50 2024-05-10T11:23:51.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.265+0000 7fa862c42700 1 --2- 172.21.15.100:0/2826951767 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa85c108400 0x7fa85c1998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:23:51.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.265+0000 7fa861c40700 1 --2- 172.21.15.100:0/2826951767 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa85c102d50 0x7fa85c19fc20 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fa850006010 tx=0x7fa85000bfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:23:51.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.266+0000 7fa85b7fe700 1 -- 172.21.15.100:0/2826951767 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa850027070 con 0x7fa85c102d50 2024-05-10T11:23:51.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.266+0000 7fa85b7fe700 1 -- 172.21.15.100:0/2826951767 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa850022470 con 0x7fa85c102d50 2024-05-10T11:23:51.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.266+0000 7fa85b7fe700 1 -- 172.21.15.100:0/2826951767 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa850017650 con 0x7fa85c102d50 2024-05-10T11:23:51.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.266+0000 7fa8646a5700 1 -- 172.21.15.100:0/2826951767 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa85c19a2e0 con 0x7fa85c102d50 2024-05-10T11:23:51.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.266+0000 7fa8646a5700 1 -- 172.21.15.100:0/2826951767 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa85c1a65b0 con 0x7fa85c102d50 2024-05-10T11:23:51.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.269+0000 7fa85b7fe700 1 -- 172.21.15.100:0/2826951767 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fa850009d10 con 0x7fa85c102d50 2024-05-10T11:23:51.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.269+0000 7fa85b7fe700 1 --2- 172.21.15.100:0/2826951767 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa8440b2900 0x7fa8440b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:51.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.270+0000 7fa862441700 1 --2- 172.21.15.100:0/2826951767 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa8440b2900 0x7fa8440b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:51.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.270+0000 7fa85b7fe700 1 -- 172.21.15.100:0/2826951767 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fa8500e2fc0 con 0x7fa85c102d50 2024-05-10T11:23:51.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.270+0000 7fa862441700 1 --2- 172.21.15.100:0/2826951767 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa8440b2900 0x7fa8440b4dc0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa84c009550 tx=0x7fa84c0091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:23:51.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.270+0000 7fa86ef38b80 1 Processor -- start 2024-05-10T11:23:51.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.271+0000 7fa86ef38b80 1 -- start start 2024-05-10T11:23:51.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.271+0000 7fa86ef38b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55577ac41590 0x55577ac41970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:51.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.271+0000 7fa86ef38b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55577ac41eb0 0x55577ac3bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:51.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.271+0000 7fa86ef38b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55577ac3c1b0 0x55577ac3c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:51.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.271+0000 7fa86ef38b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55577ab89940 con 0x55577ac41590 2024-05-10T11:23:51.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.271+0000 7fa86ef38b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55577aaeeda0 con 0x55577ac3c1b0 2024-05-10T11:23:51.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.271+0000 7fa86ef38b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55577ab85900 con 0x55577ac41eb0 2024-05-10T11:23:51.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.271+0000 7fa862441700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55577ac41590 0x55577ac41970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:51.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.271+0000 7fa861c40700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55577ac41eb0 0x55577ac3bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:51.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.271+0000 7fa862441700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55577ac41590 0x55577ac41970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47550/0 (socket says 172.21.15.100:47550) 2024-05-10T11:23:51.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.271+0000 7fa862441700 1 -- 172.21.15.100:0/3469010389 learned_addr learned my addr 172.21.15.100:0/3469010389 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:23:51.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.271+0000 7fa862c42700 1 --2- 172.21.15.100:0/3469010389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55577ac3c1b0 0x55577ac3c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:51.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.271+0000 7fa862441700 1 -- 172.21.15.100:0/3469010389 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55577ac41eb0 msgr2=0x55577ac3bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:51.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.271+0000 7fa862441700 1 --2- 172.21.15.100:0/3469010389 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55577ac41eb0 0x55577ac3bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:51.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.271+0000 7fa862441700 1 -- 172.21.15.100:0/3469010389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55577ac3c1b0 msgr2=0x55577ac3c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:51.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.271+0000 7fa862441700 1 --2- 172.21.15.100:0/3469010389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55577ac3c1b0 0x55577ac3c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:51.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.271+0000 7fa862441700 1 -- 172.21.15.100:0/3469010389 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55577aad68d0 con 0x55577ac41590 2024-05-10T11:23:51.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.272+0000 7fa862441700 1 --2- 172.21.15.100:0/3469010389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55577ac41590 0x55577ac41970 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fa84c004610 tx=0x7fa84c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8554dae5f68d4ccb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:23:51.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.272+0000 7fa858ff9700 1 -- 172.21.15.100:0/3469010389 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84c01c050 con 0x55577ac41590 2024-05-10T11:23:51.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.272+0000 7fa858ff9700 1 -- 172.21.15.100:0/3469010389 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa84c026070 con 0x55577ac41590 2024-05-10T11:23:51.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.272+0000 7fa858ff9700 1 -- 172.21.15.100:0/3469010389 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84c015440 con 0x55577ac41590 2024-05-10T11:23:51.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.273+0000 7fa86ef38b80 1 -- 172.21.15.100:0/3469010389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55577ac41590 msgr2=0x55577ac41970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:51.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.273+0000 7fa86ef38b80 1 --2- 172.21.15.100:0/3469010389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55577ac41590 0x55577ac41970 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fa84c004610 tx=0x7fa84c013000 comp rx=0 tx=0).stop 2024-05-10T11:23:51.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.273+0000 7fa86ef38b80 1 -- 172.21.15.100:0/3469010389 shutdown_connections 2024-05-10T11:23:51.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.273+0000 7fa86ef38b80 1 --2- 172.21.15.100:0/3469010389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55577ac3c1b0 0x55577ac3c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:51.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.273+0000 7fa86ef38b80 1 --2- 172.21.15.100:0/3469010389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55577ac41590 0x55577ac41970 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:51.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.273+0000 7fa86ef38b80 1 --2- 172.21.15.100:0/3469010389 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55577ac41eb0 0x55577ac3bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:51.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.273+0000 7fa86ef38b80 1 -- 172.21.15.100:0/3469010389 >> 172.21.15.100:0/3469010389 conn(0x55577ab63b70 msgr2=0x55577ac322a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:23:51.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.273+0000 7fa86ef38b80 1 -- 172.21.15.100:0/3469010389 shutdown_connections 2024-05-10T11:23:51.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.273+0000 7fa86ef38b80 1 -- 172.21.15.100:0/3469010389 wait complete. 2024-05-10T11:23:51.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.273+0000 7fa86ef38b80 1 -- start start 2024-05-10T11:23:51.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.273+0000 7fa86ef38b80 1 Processor -- start 2024-05-10T11:23:51.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.273+0000 7fa86ef38b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55577ac3c1b0 0x55577ac47b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:51.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.273+0000 7fa86ef38b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55577ac41590 0x55577ac44f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:51.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.273+0000 7fa86ef38b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55577ac41eb0 0x55577ac45440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:51.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.273+0000 7fa86ef38b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55577ac488e0 con 0x55577ac41eb0 2024-05-10T11:23:51.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.273+0000 7fa86ef38b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55577ab42ed0 con 0x55577ac3c1b0 2024-05-10T11:23:51.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.273+0000 7fa86ef38b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55577ac48a60 con 0x55577ac41590 2024-05-10T11:23:51.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.274+0000 7fa86ef38b80 10 client.0 _ll_register_callbacks cb 0x55577ab1d5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:23:51.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.274+0000 7fa862c42700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55577ac41eb0 0x55577ac45440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:51.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.274+0000 7fa862441700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55577ac3c1b0 0x55577ac47b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:51.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.274+0000 7fa862c42700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55577ac41eb0 0x55577ac45440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47556/0 (socket says 172.21.15.100:47556) 2024-05-10T11:23:51.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.274+0000 7fa862441700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55577ac3c1b0 0x55577ac47b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:48698/0 (socket says 172.21.15.100:48698) 2024-05-10T11:23:51.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.274+0000 7fa862c42700 1 -- 172.21.15.100:0/2409926189 learned_addr learned my addr 172.21.15.100:0/2409926189 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:23:51.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.274+0000 7fa861c40700 1 --2- 172.21.15.100:0/2409926189 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55577ac41590 0x55577ac44f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:51.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.274+0000 7fa862441700 1 -- 172.21.15.100:0/2409926189 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55577ac41590 msgr2=0x55577ac44f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:51.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.274+0000 7fa862441700 1 --2- 172.21.15.100:0/2409926189 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55577ac41590 0x55577ac44f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:51.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.274+0000 7fa862441700 1 -- 172.21.15.100:0/2409926189 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55577ac41eb0 msgr2=0x55577ac45440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:51.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.274+0000 7fa862441700 1 --2- 172.21.15.100:0/2409926189 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55577ac41eb0 0x55577ac45440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:51.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.274+0000 7fa862441700 1 -- 172.21.15.100:0/2409926189 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55577ab40760 con 0x55577ac3c1b0 2024-05-10T11:23:51.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.274+0000 7fa862441700 1 --2- 172.21.15.100:0/2409926189 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55577ac3c1b0 0x55577ac47b60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa84c0077e0 tx=0x7fa84c007e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7560912ad834b813 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:23:51.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.274+0000 7fa841ffb700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:23:51.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.274+0000 7fa841ffb700 1 -- 172.21.15.100:0/2409926189 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84c02f070 con 0x55577ac3c1b0 2024-05-10T11:23:51.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.274+0000 7fa841ffb700 1 -- 172.21.15.100:0/2409926189 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fa84c021910 con 0x55577ac3c1b0 2024-05-10T11:23:51.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.274+0000 7fa86ef38b80 1 -- 172.21.15.100:0/2409926189 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa84c026070 con 0x55577ac3c1b0 2024-05-10T11:23:51.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.275+0000 7fa841ffb700 1 -- 172.21.15.100:0/2409926189 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa84c01c020 con 0x55577ac3c1b0 2024-05-10T11:23:51.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.275+0000 7fa86ef38b80 10 client.25955 fetch_fsmap learned FSMap version 31 2024-05-10T11:23:51.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.275+0000 7fa86ef38b80 1 -- 172.21.15.100:0/2409926189 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55577ac53190 con 0x55577ac3c1b0 2024-05-10T11:23:51.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.275+0000 7fa841ffb700 1 -- 172.21.15.100:0/2409926189 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa84c026070 con 0x55577ac3c1b0 2024-05-10T11:23:51.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.275+0000 7fa841ffb700 1 -- 172.21.15.100:0/2409926189 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fa84c015440 con 0x55577ac3c1b0 2024-05-10T11:23:51.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.275+0000 7fa86ef38b80 10 client.25955 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:23:51.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.275+0000 7fa86ef38b80 10 client.25955 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:23:51.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.275+0000 7fa86ef38b80 20 client.25955 populate_metadata read hostname 'smithi100' 2024-05-10T11:23:51.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.275+0000 7fa86ef38b80 1 --2- 172.21.15.100:0/2409926189 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55577ac55770 0x55577ac57b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:51.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.275+0000 7fa86ef38b80 4 client.25955 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:23:51.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.275+0000 7fa86ef38b80 1 -- 172.21.15.100:0/2409926189 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55577ab97320 con 0x55577ac55770 2024-05-10T11:23:51.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.275+0000 7fa861c40700 1 --2- 172.21.15.100:0/2409926189 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55577ac55770 0x55577ac57b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:51.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.276+0000 7fa841ffb700 10 client.25955 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:23:51.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.276+0000 7fa861c40700 1 --2- 172.21.15.100:0/2409926189 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55577ac55770 0x55577ac57b60 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=651fb006723de55a server_cookie=b1fc2784fb1f3f34 in_seq=0 out_seq=0 2024-05-10T11:23:51.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.276+0000 7fa841ffb700 1 -- 172.21.15.100:0/2409926189 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55577ab97320 con 0x55577ac55770 2024-05-10T11:23:51.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.276+0000 7fa841ffb700 10 client.25955 handle_command_reply: tid=0 2024-05-10T11:23:51.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.276+0000 7fa86ef38b80 1 client.25955 shutdown 2024-05-10T11:23:51.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.276+0000 7fa86ef38b80 1 -- 172.21.15.100:0/2409926189 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55577ac3c1b0 msgr2=0x55577ac47b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:51.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.276+0000 7fa86ef38b80 1 --2- 172.21.15.100:0/2409926189 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55577ac3c1b0 0x55577ac47b60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa84c0077e0 tx=0x7fa84c007e50 comp rx=0 tx=0).stop 2024-05-10T11:23:51.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.276+0000 7fa86ef38b80 1 -- 172.21.15.100:0/2409926189 shutdown_connections 2024-05-10T11:23:51.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.276+0000 7fa86ef38b80 1 --2- 172.21.15.100:0/2409926189 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55577ac55770 0x55577ac57b60 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:51.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.277+0000 7fa841ffb700 0 client.25955 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:23:51.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.277+0000 7fa86ef38b80 1 --2- 172.21.15.100:0/2409926189 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55577ac41eb0 0x55577ac45440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:51.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.277+0000 7fa86ef38b80 1 --2- 172.21.15.100:0/2409926189 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55577ac3c1b0 0x55577ac47b60 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:51.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.277+0000 7fa86ef38b80 1 --2- 172.21.15.100:0/2409926189 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55577ac41590 0x55577ac44f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:51.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.277+0000 7fa86ef38b80 1 -- 172.21.15.100:0/2409926189 >> 172.21.15.100:0/2409926189 conn(0x55577ab63b70 msgr2=0x55577ac2cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:23:51.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.277+0000 7fa86ef38b80 1 -- 172.21.15.100:0/2409926189 shutdown_connections 2024-05-10T11:23:51.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.277+0000 7fa86ef38b80 1 -- 172.21.15.100:0/2409926189 wait complete. 2024-05-10T11:23:51.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.277+0000 7fa86ef38b80 20 client.25955 trim_cache size 0 max 16384 2024-05-10T11:23:51.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.299+0000 7fa86ef38b80 1 Processor -- start 2024-05-10T11:23:51.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.299+0000 7fa86ef38b80 1 -- start start 2024-05-10T11:23:51.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.299+0000 7fa86ef38b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55577ac3c1b0 0x55577ac42b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:51.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.299+0000 7fa86ef38b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55577ac41590 0x55577ac2cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:51.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.299+0000 7fa86ef38b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55577ac41eb0 0x55577ac46cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:51.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.299+0000 7fa86ef38b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa85c1af640 con 0x55577ac3c1b0 2024-05-10T11:23:51.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.299+0000 7fa86ef38b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa85c1aced0 con 0x55577ac41eb0 2024-05-10T11:23:51.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.299+0000 7fa86ef38b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa85c1aad20 con 0x55577ac41590 2024-05-10T11:23:51.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.299+0000 7fa862441700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55577ac3c1b0 0x55577ac42b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:51.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.299+0000 7fa862441700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55577ac3c1b0 0x55577ac42b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47558/0 (socket says 172.21.15.100:47558) 2024-05-10T11:23:51.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.299+0000 7fa862441700 1 -- 172.21.15.100:0/2039122307 learned_addr learned my addr 172.21.15.100:0/2039122307 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:23:51.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.299+0000 7fa862c42700 1 --2- 172.21.15.100:0/2039122307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55577ac41eb0 0x55577ac46cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:51.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.300+0000 7fa861c40700 1 --2- 172.21.15.100:0/2039122307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55577ac41590 0x55577ac2cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:51.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.300+0000 7fa862441700 1 -- 172.21.15.100:0/2039122307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55577ac41590 msgr2=0x55577ac2cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:51.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.300+0000 7fa862441700 1 --2- 172.21.15.100:0/2039122307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55577ac41590 0x55577ac2cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:51.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.300+0000 7fa862441700 1 -- 172.21.15.100:0/2039122307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55577ac41eb0 msgr2=0x55577ac46cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:51.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.300+0000 7fa862441700 1 --2- 172.21.15.100:0/2039122307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55577ac41eb0 0x55577ac46cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:51.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.300+0000 7fa862441700 1 -- 172.21.15.100:0/2039122307 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55577ac434f0 con 0x55577ac3c1b0 2024-05-10T11:23:51.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.300+0000 7fa861c40700 1 --2- 172.21.15.100:0/2039122307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55577ac41590 0x55577ac2cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:23:51.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.300+0000 7fa862441700 1 --2- 172.21.15.100:0/2039122307 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55577ac3c1b0 0x55577ac42b10 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fa84c009450 tx=0x7fa84c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29d69373657b681a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:23:51.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.300+0000 7fa843fff700 1 -- 172.21.15.100:0/2039122307 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84c02f040 con 0x55577ac3c1b0 2024-05-10T11:23:51.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.300+0000 7fa843fff700 1 -- 172.21.15.100:0/2039122307 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa84c026070 con 0x55577ac3c1b0 2024-05-10T11:23:51.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.300+0000 7fa843fff700 1 -- 172.21.15.100:0/2039122307 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84c015cd0 con 0x55577ac3c1b0 2024-05-10T11:23:51.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.301+0000 7fa86ef38b80 1 -- 172.21.15.100:0/2039122307 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55577ac3c1b0 msgr2=0x55577ac42b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:51.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.301+0000 7fa86ef38b80 1 --2- 172.21.15.100:0/2039122307 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55577ac3c1b0 0x55577ac42b10 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fa84c009450 tx=0x7fa84c0040c0 comp rx=0 tx=0).stop 2024-05-10T11:23:51.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.301+0000 7fa86ef38b80 1 -- 172.21.15.100:0/2039122307 shutdown_connections 2024-05-10T11:23:51.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.301+0000 7fa86ef38b80 1 --2- 172.21.15.100:0/2039122307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55577ac41eb0 0x55577ac46cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:51.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.301+0000 7fa86ef38b80 1 --2- 172.21.15.100:0/2039122307 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55577ac3c1b0 0x55577ac42b10 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:51.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.301+0000 7fa86ef38b80 1 --2- 172.21.15.100:0/2039122307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55577ac41590 0x55577ac2cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:51.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.301+0000 7fa86ef38b80 1 -- 172.21.15.100:0/2039122307 >> 172.21.15.100:0/2039122307 conn(0x55577ab63b70 msgr2=0x55577ac465b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:23:51.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.302+0000 7fa86ef38b80 1 -- 172.21.15.100:0/2039122307 shutdown_connections 2024-05-10T11:23:51.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.302+0000 7fa86ef38b80 1 -- 172.21.15.100:0/2039122307 wait complete. 2024-05-10T11:23:51.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.302+0000 7fa86ef38b80 1 -- start start 2024-05-10T11:23:51.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.302+0000 7fa86ef38b80 1 Processor -- start 2024-05-10T11:23:51.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.302+0000 7fa86ef38b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55577ac3c1b0 0x55577ac5adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:51.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.302+0000 7fa86ef38b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55577ac41590 0x55577ac59160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:51.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.302+0000 7fa86ef38b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55577ac41eb0 0x55577ac596a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:51.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.302+0000 7fa86ef38b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa85c1ab390 con 0x55577ac41eb0 2024-05-10T11:23:51.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.302+0000 7fa86ef38b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa85c1ac800 con 0x55577ac3c1b0 2024-05-10T11:23:51.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.302+0000 7fa86ef38b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa85c1aba60 con 0x55577ac41590 2024-05-10T11:23:51.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.302+0000 7fa86ef38b80 10 client.0 _ll_register_callbacks cb 0x55577ab1d5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:23:51.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.302+0000 7fa862c42700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55577ac41eb0 0x55577ac596a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:51.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.302+0000 7fa862441700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55577ac3c1b0 0x55577ac5adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:51.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.302+0000 7fa861c40700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55577ac41590 0x55577ac59160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:51.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.302+0000 7fa862c42700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55577ac41eb0 0x55577ac596a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47574/0 (socket says 172.21.15.100:47574) 2024-05-10T11:23:51.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.302+0000 7fa862441700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55577ac3c1b0 0x55577ac5adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:48722/0 (socket says 172.21.15.100:48722) 2024-05-10T11:23:51.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.302+0000 7fa862c42700 1 -- 172.21.15.100:0/387373280 learned_addr learned my addr 172.21.15.100:0/387373280 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:23:51.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.303+0000 7fa861c40700 1 -- 172.21.15.100:0/387373280 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55577ac3c1b0 msgr2=0x55577ac5adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:51.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.303+0000 7fa861c40700 1 --2- 172.21.15.100:0/387373280 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55577ac3c1b0 0x55577ac5adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:51.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.303+0000 7fa861c40700 1 -- 172.21.15.100:0/387373280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55577ac41eb0 msgr2=0x55577ac596a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:23:51.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.303+0000 7fa861c40700 1 --2- 172.21.15.100:0/387373280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55577ac41eb0 0x55577ac596a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:51.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.303+0000 7fa861c40700 1 -- 172.21.15.100:0/387373280 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55577ac59be0 con 0x55577ac41590 2024-05-10T11:23:51.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.303+0000 7fa862c42700 1 --2- 172.21.15.100:0/387373280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55577ac41eb0 0x55577ac596a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:23:51.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.303+0000 7fa861c40700 1 --2- 172.21.15.100:0/387373280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55577ac41590 0x55577ac59160 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa85000b2e0 tx=0x7fa850092040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=901ac6075b18049 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:23:51.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.303+0000 7fa841ffb700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:23:51.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.303+0000 7fa841ffb700 1 -- 172.21.15.100:0/387373280 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8500ae6c0 con 0x55577ac41590 2024-05-10T11:23:51.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.303+0000 7fa841ffb700 1 -- 172.21.15.100:0/387373280 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fa8500aa570 con 0x55577ac41590 2024-05-10T11:23:51.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.303+0000 7fa86ef38b80 1 -- 172.21.15.100:0/387373280 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa84c026070 con 0x55577ac41590 2024-05-10T11:23:51.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.303+0000 7fa841ffb700 1 -- 172.21.15.100:0/387373280 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa8500b30e0 con 0x55577ac41590 2024-05-10T11:23:51.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.303+0000 7fa86ef38b80 10 client.25987 fetch_fsmap learned FSMap version 31 2024-05-10T11:23:51.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.303+0000 7fa86ef38b80 1 -- 172.21.15.100:0/387373280 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55577acc0a00 con 0x55577ac41590 2024-05-10T11:23:51.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.304+0000 7fa841ffb700 1 -- 172.21.15.100:0/387373280 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa84c026070 con 0x55577ac41590 2024-05-10T11:23:51.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.304+0000 7fa841ffb700 1 -- 172.21.15.100:0/387373280 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fa8500a9570 con 0x55577ac41590 2024-05-10T11:23:51.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.304+0000 7fa86ef38b80 10 client.25987 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:23:51.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.304+0000 7fa86ef38b80 10 client.25987 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:23:51.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.304+0000 7fa86ef38b80 20 client.25987 populate_metadata read hostname 'smithi100' 2024-05-10T11:23:51.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.304+0000 7fa86ef38b80 1 --2- 172.21.15.100:0/387373280 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55577ac55770 0x55577acc3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:23:51.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.304+0000 7fa86ef38b80 4 client.25987 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T11:23:51.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.304+0000 7fa86ef38b80 1 -- 172.21.15.100:0/387373280 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55577ab913c0 con 0x55577ac55770 2024-05-10T11:23:51.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.304+0000 7fa862441700 1 --2- 172.21.15.100:0/387373280 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55577ac55770 0x55577acc3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:23:51.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.304+0000 7fa841ffb700 10 client.25987 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:23:51.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.304+0000 7fa862441700 1 --2- 172.21.15.100:0/387373280 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55577ac55770 0x55577acc3100 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a51895d24dc87b3d server_cookie=b2b9139c5e8b8541 in_seq=0 out_seq=0 2024-05-10T11:23:51.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.305+0000 7fa841ffb700 1 -- 172.21.15.100:0/387373280 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55577ab913c0 con 0x55577ac55770 2024-05-10T11:23:51.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.305+0000 7fa841ffb700 10 client.25987 handle_command_reply: tid=0 2024-05-10T11:23:51.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.305+0000 7fa86ef38b80 1 client.25987 shutdown 2024-05-10T11:23:51.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.305+0000 7fa86ef38b80 1 -- 172.21.15.100:0/387373280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55577ac41590 msgr2=0x55577ac59160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:51.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.305+0000 7fa86ef38b80 1 --2- 172.21.15.100:0/387373280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55577ac41590 0x55577ac59160 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa85000b2e0 tx=0x7fa850092040 comp rx=0 tx=0).stop 2024-05-10T11:23:51.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.305+0000 7fa86ef38b80 1 -- 172.21.15.100:0/387373280 shutdown_connections 2024-05-10T11:23:51.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.305+0000 7fa86ef38b80 1 --2- 172.21.15.100:0/387373280 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55577ac55770 0x55577acc3100 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:51.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.305+0000 7fa86ef38b80 1 --2- 172.21.15.100:0/387373280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55577ac41eb0 0x55577ac596a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:51.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.305+0000 7fa86ef38b80 1 --2- 172.21.15.100:0/387373280 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55577ac3c1b0 0x55577ac5adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:51.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.305+0000 7fa86ef38b80 1 --2- 172.21.15.100:0/387373280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55577ac41590 0x55577ac59160 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:51.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.305+0000 7fa86ef38b80 1 -- 172.21.15.100:0/387373280 >> 172.21.15.100:0/387373280 conn(0x55577ab63b70 msgr2=0x55577ac43fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:23:51.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.305+0000 7fa841ffb700 0 client.25987 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:23:51.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.305+0000 7fa86ef38b80 1 -- 172.21.15.100:0/387373280 shutdown_connections 2024-05-10T11:23:51.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.305+0000 7fa86ef38b80 1 -- 172.21.15.100:0/387373280 wait complete. 2024-05-10T11:23:51.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.305+0000 7fa86ef38b80 20 client.25987 trim_cache size 0 max 16384 2024-05-10T11:23:51.311 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:23:51.311 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T11:23:51.311 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-05-10T11:23:51.311 INFO:teuthology.orchestra.run.smithi100.stdout: "b6218ccb-7442-4715-a85d-c45a328d77ee": { 2024-05-10T11:23:51.311 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-05-10T11:23:51.311 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "b6218ccb-7442-4715-a85d-c45a328d77ee", 2024-05-10T11:23:51.311 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-05-10T11:23:51.311 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:23:51.312 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:23:51.312 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:23:51.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.306+0000 7fa8646a5700 1 -- 172.21.15.100:0/2826951767 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa8440b2900 msgr2=0x7fa8440b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:51.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.306+0000 7fa8646a5700 1 --2- 172.21.15.100:0/2826951767 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa8440b2900 0x7fa8440b4dc0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa84c009550 tx=0x7fa84c0091b0 comp rx=0 tx=0).stop 2024-05-10T11:23:51.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.306+0000 7fa8646a5700 1 -- 172.21.15.100:0/2826951767 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa85c102d50 msgr2=0x7fa85c19fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:23:51.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.306+0000 7fa8646a5700 1 --2- 172.21.15.100:0/2826951767 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa85c102d50 0x7fa85c19fc20 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fa850006010 tx=0x7fa85000bfc0 comp rx=0 tx=0).stop 2024-05-10T11:23:51.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.306+0000 7fa8646a5700 1 -- 172.21.15.100:0/2826951767 shutdown_connections 2024-05-10T11:23:51.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.306+0000 7fa8646a5700 1 --2- 172.21.15.100:0/2826951767 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa8440b2900 0x7fa8440b4dc0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:51.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.306+0000 7fa8646a5700 1 --2- 172.21.15.100:0/2826951767 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa85c108400 0x7fa85c1998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:51.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.306+0000 7fa8646a5700 1 --2- 172.21.15.100:0/2826951767 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa85c102390 0x7fa85c19f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:51.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.306+0000 7fa8646a5700 1 --2- 172.21.15.100:0/2826951767 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa85c102d50 0x7fa85c19fc20 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:23:51.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.306+0000 7fa8646a5700 1 -- 172.21.15.100:0/2826951767 >> 172.21.15.100:0/2826951767 conn(0x7fa85c079310 msgr2=0x7fa85c100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:23:51.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.306+0000 7fa8646a5700 1 -- 172.21.15.100:0/2826951767 shutdown_connections 2024-05-10T11:23:51.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:23:51.306+0000 7fa8646a5700 1 -- 172.21.15.100:0/2826951767 wait complete. 2024-05-10T11:23:51.971 INFO:tasks.workunit.client.0.smithi100.stdout: 1 118226 2.73 MB/sec execute 106 sec latency 66.591 ms 2024-05-10T11:23:52.014 INFO:tasks.cephfs.filesystem:scrub status for tag:b6218ccb-7442-4715-a85d-c45a328d77ee - {'path': '/', 'tag': 'b6218ccb-7442-4715-a85d-c45a328d77ee', 'options': 'recursive,force'} 2024-05-10T11:23:52.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:51 smithi178 ceph-mon[33359]: pgmap v584: 129 pgs: 129 active+clean; 57 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 1.1 MiB/s wr, 154 op/s 2024-05-10T11:23:52.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:51 smithi188 ceph-mon[34075]: pgmap v584: 129 pgs: 129 active+clean; 57 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 1.1 MiB/s wr, 154 op/s 2024-05-10T11:23:52.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:51 smithi100 ceph-mon[30296]: pgmap v584: 129 pgs: 129 active+clean; 57 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 1.1 MiB/s wr, 154 op/s 2024-05-10T11:23:52.971 INFO:tasks.workunit.client.0.smithi100.stdout: 1 118805 2.74 MB/sec execute 107 sec latency 21.160 ms 2024-05-10T11:23:53.971 INFO:tasks.workunit.client.0.smithi100.stdout: 1 119488 2.73 MB/sec execute 108 sec latency 11.816 ms 2024-05-10T11:23:54.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:53 smithi178 ceph-mon[33359]: pgmap v585: 129 pgs: 129 active+clean; 59 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 1.4 MiB/s wr, 161 op/s 2024-05-10T11:23:54.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:53 smithi188 ceph-mon[34075]: pgmap v585: 129 pgs: 129 active+clean; 59 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 1.4 MiB/s wr, 161 op/s 2024-05-10T11:23:54.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:53 smithi100 ceph-mon[30296]: pgmap v585: 129 pgs: 129 active+clean; 59 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 1.4 MiB/s wr, 161 op/s 2024-05-10T11:23:54.971 INFO:tasks.workunit.client.0.smithi100.stdout: 1 120120 2.76 MB/sec execute 109 sec latency 15.290 ms 2024-05-10T11:23:55.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:54 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:23:55.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:54 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:23:55.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:54 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:23:55.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:54 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:23:55.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:54 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:23:55.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:54 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:23:55.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:54 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:23:55.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:54 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:23:55.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:54 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:23:55.971 INFO:tasks.workunit.client.0.smithi100.stdout: 1 120720 2.78 MB/sec execute 110 sec latency 14.864 ms 2024-05-10T11:23:56.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:55 smithi178 ceph-mon[33359]: pgmap v586: 129 pgs: 129 active+clean; 59 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 1.4 MiB/s wr, 157 op/s 2024-05-10T11:23:56.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:55 smithi188 ceph-mon[34075]: pgmap v586: 129 pgs: 129 active+clean; 59 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 1.4 MiB/s wr, 157 op/s 2024-05-10T11:23:56.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:55 smithi100 ceph-mon[30296]: pgmap v586: 129 pgs: 129 active+clean; 59 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 1.4 MiB/s wr, 157 op/s 2024-05-10T11:23:56.971 INFO:tasks.workunit.client.0.smithi100.stdout: 1 121035 2.76 MB/sec execute 111 sec latency 16.451 ms 2024-05-10T11:23:57.971 INFO:tasks.workunit.client.0.smithi100.stdout: 1 121392 2.74 MB/sec execute 112 sec latency 10.682 ms 2024-05-10T11:23:58.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:57 smithi178 ceph-mon[33359]: pgmap v587: 129 pgs: 129 active+clean; 59 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 1.3 MiB/s wr, 157 op/s 2024-05-10T11:23:58.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:57 smithi188 ceph-mon[34075]: pgmap v587: 129 pgs: 129 active+clean; 59 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 1.3 MiB/s wr, 157 op/s 2024-05-10T11:23:58.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:57 smithi100 ceph-mon[30296]: pgmap v587: 129 pgs: 129 active+clean; 59 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 1.3 MiB/s wr, 157 op/s 2024-05-10T11:23:58.971 INFO:tasks.workunit.client.0.smithi100.stdout: 1 121914 2.73 MB/sec execute 113 sec latency 12.733 ms 2024-05-10T11:23:59.971 INFO:tasks.workunit.client.0.smithi100.stdout: 1 122559 2.74 MB/sec execute 114 sec latency 16.936 ms 2024-05-10T11:24:00.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:23:59 smithi178 ceph-mon[33359]: pgmap v588: 129 pgs: 129 active+clean; 59 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 1.2 MiB/s wr, 134 op/s 2024-05-10T11:24:00.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:23:59 smithi188 ceph-mon[34075]: pgmap v588: 129 pgs: 129 active+clean; 59 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 1.2 MiB/s wr, 134 op/s 2024-05-10T11:24:00.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:23:59 smithi100 ceph-mon[30296]: pgmap v588: 129 pgs: 129 active+clean; 59 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 1.2 MiB/s wr, 134 op/s 2024-05-10T11:24:00.971 INFO:tasks.workunit.client.0.smithi100.stdout: 1 123119 2.73 MB/sec execute 115 sec latency 15.865 ms 2024-05-10T11:24:01.971 INFO:tasks.workunit.client.0.smithi100.stdout: 1 123912 2.77 MB/sec execute 116 sec latency 14.752 ms 2024-05-10T11:24:02.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:24:01 smithi178 ceph-mon[33359]: pgmap v589: 129 pgs: 129 active+clean; 61 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 1.3 MiB/s wr, 160 op/s 2024-05-10T11:24:02.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:24:01 smithi188 ceph-mon[34075]: pgmap v589: 129 pgs: 129 active+clean; 61 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 1.3 MiB/s wr, 160 op/s 2024-05-10T11:24:02.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:24:01 smithi100 ceph-mon[30296]: pgmap v589: 129 pgs: 129 active+clean; 61 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 1.3 MiB/s wr, 160 op/s 2024-05-10T11:24:02.972 INFO:tasks.workunit.client.0.smithi100.stdout: 1 124329 2.77 MB/sec execute 117 sec latency 13.624 ms 2024-05-10T11:24:03.971 INFO:tasks.workunit.client.0.smithi100.stdout: 1 124618 2.76 MB/sec execute 118 sec latency 12.828 ms 2024-05-10T11:24:04.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:24:03 smithi178 ceph-mon[33359]: pgmap v590: 129 pgs: 129 active+clean; 61 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 1.4 MiB/s wr, 166 op/s 2024-05-10T11:24:04.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:24:03 smithi188 ceph-mon[34075]: pgmap v590: 129 pgs: 129 active+clean; 61 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 1.4 MiB/s wr, 166 op/s 2024-05-10T11:24:04.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:24:03 smithi100 ceph-mon[30296]: pgmap v590: 129 pgs: 129 active+clean; 61 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 1.4 MiB/s wr, 166 op/s 2024-05-10T11:24:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.344+0000 7fe6c0020640 1 -- 172.21.15.100:0/680903140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6b81045e0 msgr2=0x7fe6b810ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.344+0000 7fe6c0020640 1 --2- 172.21.15.100:0/680903140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6b81045e0 0x7fe6b810ae70 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0009ca0 tx=0x7fe6b002a8c0 comp rx=0 tx=0).stop 2024-05-10T11:24:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.344+0000 7fe6c0020640 1 -- 172.21.15.100:0/680903140 shutdown_connections 2024-05-10T11:24:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.344+0000 7fe6c0020640 1 --2- 172.21.15.100:0/680903140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6b81045e0 0x7fe6b810ae70 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.344+0000 7fe6c0020640 1 --2- 172.21.15.100:0/680903140 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6b8103c20 0x7fe6b81040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.344+0000 7fe6c0020640 1 --2- 172.21.15.100:0/680903140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6b8102a30 0x7fe6b8102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.344+0000 7fe6c0020640 1 -- 172.21.15.100:0/680903140 >> 172.21.15.100:0/680903140 conn(0x7fe6b80fdfa0 msgr2=0x7fe6b81003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:24:04.347 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.344+0000 7fe6c0020640 1 -- 172.21.15.100:0/680903140 shutdown_connections 2024-05-10T11:24:04.347 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.344+0000 7fe6c0020640 1 -- 172.21.15.100:0/680903140 wait complete. 2024-05-10T11:24:04.347 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.345+0000 7fe6c0020640 1 Processor -- start 2024-05-10T11:24:04.347 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.345+0000 7fe6c0020640 1 -- start start 2024-05-10T11:24:04.347 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.345+0000 7fe6c0020640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6b8102a30 0x7fe6b8198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:04.347 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.345+0000 7fe6c0020640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6b8103c20 0x7fe6b8199020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:04.347 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.345+0000 7fe6c0020640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6b81045e0 0x7fe6b81a0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:04.347 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.345+0000 7fe6c0020640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b810e4d0 con 0x7fe6b8103c20 2024-05-10T11:24:04.347 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.345+0000 7fe6c0020640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b810e350 con 0x7fe6b8102a30 2024-05-10T11:24:04.347 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.345+0000 7fe6c0020640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b810e650 con 0x7fe6b81045e0 2024-05-10T11:24:04.347 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.345+0000 7fe6bd5bb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6b8103c20 0x7fe6b8199020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:04.347 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.345+0000 7fe6bd5bb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6b8103c20 0x7fe6b8199020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57436/0 (socket says 172.21.15.100:57436) 2024-05-10T11:24:04.347 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.345+0000 7fe6bd5bb640 1 -- 172.21.15.100:0/1991268377 learned_addr learned my addr 172.21.15.100:0/1991268377 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:24:04.347 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.345+0000 7fe6bd5bb640 1 -- 172.21.15.100:0/1991268377 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6b81045e0 msgr2=0x7fe6b81a0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:04.347 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.345+0000 7fe6bd5bb640 1 --2- 172.21.15.100:0/1991268377 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6b81045e0 0x7fe6b81a0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:04.348 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.345+0000 7fe6bd5bb640 1 -- 172.21.15.100:0/1991268377 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6b8102a30 msgr2=0x7fe6b8198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:04.348 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.345+0000 7fe6bddbc640 1 --2- 172.21.15.100:0/1991268377 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6b8102a30 0x7fe6b8198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:04.348 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.345+0000 7fe6bd5bb640 1 --2- 172.21.15.100:0/1991268377 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6b8102a30 0x7fe6b8198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:04.348 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.345+0000 7fe6bd5bb640 1 -- 172.21.15.100:0/1991268377 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6b81a0700 con 0x7fe6b8103c20 2024-05-10T11:24:04.348 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.345+0000 7fe6bddbc640 1 --2- 172.21.15.100:0/1991268377 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6b8102a30 0x7fe6b8198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:24:04.348 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.345+0000 7fe6bd5bb640 1 --2- 172.21.15.100:0/1991268377 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6b8103c20 0x7fe6b8199020 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fe6a800d950 tx=0x7fe6a800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:24:04.348 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.346+0000 7fe6aeffd640 1 -- 172.21.15.100:0/1991268377 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a8017070 con 0x7fe6b8103c20 2024-05-10T11:24:04.348 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.346+0000 7fe6aeffd640 1 -- 172.21.15.100:0/1991268377 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe6a8009c30 con 0x7fe6b8103c20 2024-05-10T11:24:04.348 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.346+0000 7fe6c0020640 1 -- 172.21.15.100:0/1991268377 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6b81a09f0 con 0x7fe6b8103c20 2024-05-10T11:24:04.348 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.346+0000 7fe6aeffd640 1 -- 172.21.15.100:0/1991268377 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a800f620 con 0x7fe6b8103c20 2024-05-10T11:24:04.348 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.346+0000 7fe6c0020640 1 -- 172.21.15.100:0/1991268377 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6b81a0f40 con 0x7fe6b8103c20 2024-05-10T11:24:04.351 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.348+0000 7fe6aeffd640 1 -- 172.21.15.100:0/1991268377 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fe6a8010460 con 0x7fe6b8103c20 2024-05-10T11:24:04.351 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.348+0000 7fe6c2e12740 1 Processor -- start 2024-05-10T11:24:04.351 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.349+0000 7fe6c2e12740 1 -- start start 2024-05-10T11:24:04.351 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.349+0000 7fe6c2e12740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e8c92edb0 0x558e8c92f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:04.351 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.349+0000 7fe6c2e12740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e8c92f7a0 0x558e8c9342a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:04.351 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.349+0000 7fe6c2e12740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558e8c934910 0x558e8c936da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:04.351 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.349+0000 7fe6c2e12740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x558e8c8935d0 con 0x558e8c92edb0 2024-05-10T11:24:04.351 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.349+0000 7fe6c2e12740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558e8c84c890 con 0x558e8c92f7a0 2024-05-10T11:24:04.352 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.349+0000 7fe6c2e12740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558e8c88f590 con 0x558e8c934910 2024-05-10T11:24:04.352 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.349+0000 7fe6be5bd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e8c92edb0 0x558e8c92f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:04.352 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.349+0000 7fe6bddbc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e8c92f7a0 0x558e8c9342a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:04.352 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.349+0000 7fe6bd5bb640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558e8c934910 0x558e8c936da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:04.352 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.349+0000 7fe6bddbc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e8c92f7a0 0x558e8c9342a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:35706/0 (socket says 172.21.15.100:35706) 2024-05-10T11:24:04.352 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.349+0000 7fe6be5bd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e8c92edb0 0x558e8c92f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57438/0 (socket says 172.21.15.100:57438) 2024-05-10T11:24:04.352 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.349+0000 7fe6bddbc640 1 -- 172.21.15.100:0/4212184109 learned_addr learned my addr 172.21.15.100:0/4212184109 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:24:04.352 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.349+0000 7fe6aeffd640 1 --2- 172.21.15.100:0/1991268377 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe6a40b28a0 0x7fe6a40b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:04.352 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.349+0000 7fe6bddbc640 1 -- 172.21.15.100:0/4212184109 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558e8c934910 msgr2=0x558e8c936da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:04.352 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.349+0000 7fe6bddbc640 1 --2- 172.21.15.100:0/4212184109 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558e8c934910 0x558e8c936da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:04.352 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.349+0000 7fe6be5bd640 1 --2- 172.21.15.100:0/1991268377 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe6a40b28a0 0x7fe6a40b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:04.352 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.349+0000 7fe6bddbc640 1 -- 172.21.15.100:0/4212184109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e8c92edb0 msgr2=0x558e8c92f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:04.352 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.349+0000 7fe6bddbc640 1 --2- 172.21.15.100:0/4212184109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e8c92edb0 0x558e8c92f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:04.352 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.350+0000 7fe6bddbc640 1 -- 172.21.15.100:0/4212184109 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e8c5a5210 con 0x558e8c92f7a0 2024-05-10T11:24:04.353 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.350+0000 7fe6aeffd640 1 -- 172.21.15.100:0/1991268377 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fe6a80e22b0 con 0x7fe6b8103c20 2024-05-10T11:24:04.353 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.350+0000 7fe6bddbc640 1 --2- 172.21.15.100:0/4212184109 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e8c92f7a0 0x558e8c9342a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe6b400a9d0 tx=0x7fe6b400ae90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab13a65d01d26a8e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:24:04.353 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.350+0000 7fe6be5bd640 1 --2- 172.21.15.100:0/1991268377 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe6a40b28a0 0x7fe6a40b4d60 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0006010 tx=0x7fe6b0031000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:24:04.353 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.350+0000 7fe6a3fff640 1 -- 172.21.15.100:0/4212184109 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b4015070 con 0x558e8c92f7a0 2024-05-10T11:24:04.353 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.350+0000 7fe6a3fff640 1 -- 172.21.15.100:0/4212184109 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe6b400cc60 con 0x558e8c92f7a0 2024-05-10T11:24:04.353 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.350+0000 7fe6a3fff640 1 -- 172.21.15.100:0/4212184109 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b4018560 con 0x558e8c92f7a0 2024-05-10T11:24:04.353 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.351+0000 7fe6c2e12740 1 -- 172.21.15.100:0/4212184109 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e8c92f7a0 msgr2=0x558e8c9342a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:04.353 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.351+0000 7fe6c2e12740 1 --2- 172.21.15.100:0/4212184109 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e8c92f7a0 0x558e8c9342a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe6b400a9d0 tx=0x7fe6b400ae90 comp rx=0 tx=0).stop 2024-05-10T11:24:04.353 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.351+0000 7fe6c2e12740 1 -- 172.21.15.100:0/4212184109 shutdown_connections 2024-05-10T11:24:04.353 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.351+0000 7fe6c2e12740 1 --2- 172.21.15.100:0/4212184109 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558e8c934910 0x558e8c936da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:04.353 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.351+0000 7fe6c2e12740 1 --2- 172.21.15.100:0/4212184109 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e8c92f7a0 0x558e8c9342a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:04.353 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.351+0000 7fe6c2e12740 1 --2- 172.21.15.100:0/4212184109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e8c92edb0 0x558e8c92f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:04.353 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.351+0000 7fe6c2e12740 1 -- 172.21.15.100:0/4212184109 >> 172.21.15.100:0/4212184109 conn(0x558e8c92a420 msgr2=0x558e8c92a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:24:04.353 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.351+0000 7fe6c2e12740 1 -- 172.21.15.100:0/4212184109 shutdown_connections 2024-05-10T11:24:04.353 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.351+0000 7fe6c2e12740 1 -- 172.21.15.100:0/4212184109 wait complete. 2024-05-10T11:24:04.354 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.351+0000 7fe6c2e12740 1 -- start start 2024-05-10T11:24:04.354 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.351+0000 7fe6c2e12740 1 Processor -- start 2024-05-10T11:24:04.358 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.352+0000 7fe6c2e12740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e8c92edb0 0x558e8c938f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:04.358 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.352+0000 7fe6c2e12740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e8c92f7a0 0x558e8c9394b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:04.358 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.352+0000 7fe6c2e12740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558e8c934910 0x558e8c9404f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:04.358 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.352+0000 7fe6c2e12740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x558e8c9378c0 con 0x558e8c92f7a0 2024-05-10T11:24:04.358 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.352+0000 7fe6c2e12740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558e8c850370 con 0x558e8c92edb0 2024-05-10T11:24:04.358 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.352+0000 7fe6c2e12740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558e8c937a40 con 0x558e8c934910 2024-05-10T11:24:04.358 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.352+0000 7fe6c2e12740 10 client.0 _ll_register_callbacks cb 0x558e8c5a4a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:24:04.359 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.352+0000 7fe6bddbc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e8c92edb0 0x558e8c938f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:04.359 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.352+0000 7fe6bddbc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e8c92edb0 0x558e8c938f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:35718/0 (socket says 172.21.15.100:35718) 2024-05-10T11:24:04.359 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.352+0000 7fe6bddbc640 1 -- 172.21.15.100:0/1055840680 learned_addr learned my addr 172.21.15.100:0/1055840680 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:24:04.359 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.352+0000 7fe6bd5bb640 1 --2- 172.21.15.100:0/1055840680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e8c92f7a0 0x558e8c9394b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:04.359 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.352+0000 7fe6be5bd640 1 --2- 172.21.15.100:0/1055840680 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558e8c934910 0x558e8c9404f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:04.359 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.352+0000 7fe6bddbc640 1 -- 172.21.15.100:0/1055840680 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558e8c934910 msgr2=0x558e8c9404f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:04.359 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.352+0000 7fe6bddbc640 1 --2- 172.21.15.100:0/1055840680 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558e8c934910 0x558e8c9404f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:04.359 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.352+0000 7fe6bddbc640 1 -- 172.21.15.100:0/1055840680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e8c92f7a0 msgr2=0x558e8c9394b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:04.359 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.352+0000 7fe6bddbc640 1 --2- 172.21.15.100:0/1055840680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e8c92f7a0 0x558e8c9394b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:04.359 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.352+0000 7fe6bddbc640 1 -- 172.21.15.100:0/1055840680 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e8c671620 con 0x558e8c92edb0 2024-05-10T11:24:04.359 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.352+0000 7fe6bd5bb640 1 --2- 172.21.15.100:0/1055840680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e8c92f7a0 0x558e8c9394b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:24:04.359 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.352+0000 7fe6bddbc640 1 --2- 172.21.15.100:0/1055840680 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e8c92edb0 0x558e8c938f70 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4000c00 tx=0x7fe6b4006cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d40cf87feefeb27 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:24:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.352+0000 7fe6a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:24:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.353+0000 7fe6a17fa640 1 -- 172.21.15.100:0/1055840680 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b4020070 con 0x558e8c92edb0 2024-05-10T11:24:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.353+0000 7fe6a17fa640 1 -- 172.21.15.100:0/1055840680 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fe6b401a450 con 0x558e8c92edb0 2024-05-10T11:24:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.353+0000 7fe6c2e12740 1 -- 172.21.15.100:0/1055840680 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6b400cc60 con 0x558e8c92edb0 2024-05-10T11:24:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.353+0000 7fe6a17fa640 1 -- 172.21.15.100:0/1055840680 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe6b4015040 con 0x558e8c92edb0 2024-05-10T11:24:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.353+0000 7fe6c2e12740 10 client.25970 fetch_fsmap learned FSMap version 31 2024-05-10T11:24:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.353+0000 7fe6c2e12740 1 -- 172.21.15.100:0/1055840680 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558e8c9478e0 con 0x558e8c92edb0 2024-05-10T11:24:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.353+0000 7fe6a17fa640 1 -- 172.21.15.100:0/1055840680 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe6b400cc60 con 0x558e8c92edb0 2024-05-10T11:24:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.353+0000 7fe6a17fa640 1 -- 172.21.15.100:0/1055840680 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fe6b4024600 con 0x558e8c92edb0 2024-05-10T11:24:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.353+0000 7fe6c2e12740 10 client.25970 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:24:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.353+0000 7fe6c2e12740 10 client.25970 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:24:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.353+0000 7fe6c2e12740 20 client.25970 populate_metadata read hostname 'smithi100' 2024-05-10T11:24:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.353+0000 7fe6c2e12740 1 --2- 172.21.15.100:0/1055840680 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558e8c948ca0 0x558e8c94b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.353+0000 7fe6c2e12740 4 client.25970 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:24:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.353+0000 7fe6c2e12740 1 -- 172.21.15.100:0/1055840680 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e8c8a11e0 con 0x558e8c948ca0 2024-05-10T11:24:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.354+0000 7fe6bd5bb640 1 --2- 172.21.15.100:0/1055840680 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558e8c948ca0 0x558e8c94b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.354+0000 7fe6bd5bb640 1 --2- 172.21.15.100:0/1055840680 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558e8c948ca0 0x558e8c94b090 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc77317778c84825 server_cookie=51f3f08004965fa9 in_seq=0 out_seq=0 2024-05-10T11:24:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.354+0000 7fe6a17fa640 10 client.25970 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:24:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.355+0000 7fe6a17fa640 1 -- 172.21.15.100:0/1055840680 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558e8c8a11e0 con 0x558e8c948ca0 2024-05-10T11:24:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.355+0000 7fe6a17fa640 10 client.25970 handle_command_reply: tid=0 2024-05-10T11:24:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.355+0000 7fe6c2e12740 1 client.25970 shutdown 2024-05-10T11:24:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.355+0000 7fe6c2e12740 1 -- 172.21.15.100:0/1055840680 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e8c92edb0 msgr2=0x558e8c938f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.355+0000 7fe6c2e12740 1 --2- 172.21.15.100:0/1055840680 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e8c92edb0 0x558e8c938f70 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4000c00 tx=0x7fe6b4006cb0 comp rx=0 tx=0).stop 2024-05-10T11:24:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.355+0000 7fe6c2e12740 1 -- 172.21.15.100:0/1055840680 shutdown_connections 2024-05-10T11:24:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.355+0000 7fe6c2e12740 1 --2- 172.21.15.100:0/1055840680 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558e8c948ca0 0x558e8c94b090 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.355+0000 7fe6c2e12740 1 --2- 172.21.15.100:0/1055840680 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558e8c934910 0x558e8c9404f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.355+0000 7fe6c2e12740 1 --2- 172.21.15.100:0/1055840680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e8c92f7a0 0x558e8c9394b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.355+0000 7fe6c2e12740 1 --2- 172.21.15.100:0/1055840680 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e8c92edb0 0x558e8c938f70 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.355+0000 7fe6a17fa640 0 client.25970 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:24:04.362 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.355+0000 7fe6c2e12740 1 -- 172.21.15.100:0/1055840680 >> 172.21.15.100:0/1055840680 conn(0x558e8c92a420 msgr2=0x558e8c929580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:24:04.362 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.355+0000 7fe6c2e12740 1 -- 172.21.15.100:0/1055840680 shutdown_connections 2024-05-10T11:24:04.362 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.355+0000 7fe6c2e12740 1 -- 172.21.15.100:0/1055840680 wait complete. 2024-05-10T11:24:04.362 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.355+0000 7fe6c2e12740 20 client.25970 trim_cache size 0 max 16384 2024-05-10T11:24:04.373 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.372+0000 7fe6c2e12740 1 Processor -- start 2024-05-10T11:24:04.373 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.372+0000 7fe6c2e12740 1 -- start start 2024-05-10T11:24:04.373 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.372+0000 7fe6c2e12740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558e8c92edb0 0x558e8c929400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:04.373 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.372+0000 7fe6c2e12740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e8c92f7a0 0x558e8c929940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:04.373 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.372+0000 7fe6c2e12740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e8c934910 0x558e8c940410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:04.373 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.372+0000 7fe6c2e12740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b81ab9c0 con 0x558e8c92f7a0 2024-05-10T11:24:04.373 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.372+0000 7fe6c2e12740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b81a9290 con 0x558e8c934910 2024-05-10T11:24:04.373 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.372+0000 7fe6c2e12740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b81a70e0 con 0x558e8c92edb0 2024-05-10T11:24:04.373 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.372+0000 7fe6bd5bb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e8c92f7a0 0x558e8c929940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:04.374 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.372+0000 7fe6bd5bb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e8c92f7a0 0x558e8c929940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57460/0 (socket says 172.21.15.100:57460) 2024-05-10T11:24:04.374 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.372+0000 7fe6bd5bb640 1 -- 172.21.15.100:0/1903453131 learned_addr learned my addr 172.21.15.100:0/1903453131 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:24:04.374 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.372+0000 7fe6be5bd640 1 --2- 172.21.15.100:0/1903453131 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e8c934910 0x558e8c940410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:04.374 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.372+0000 7fe6bddbc640 1 --2- 172.21.15.100:0/1903453131 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558e8c92edb0 0x558e8c929400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:04.374 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.372+0000 7fe6bd5bb640 1 -- 172.21.15.100:0/1903453131 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558e8c92edb0 msgr2=0x558e8c929400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:04.374 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.372+0000 7fe6bd5bb640 1 --2- 172.21.15.100:0/1903453131 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558e8c92edb0 0x558e8c929400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:04.374 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.372+0000 7fe6bd5bb640 1 -- 172.21.15.100:0/1903453131 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e8c934910 msgr2=0x558e8c940410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:04.374 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.372+0000 7fe6bd5bb640 1 --2- 172.21.15.100:0/1903453131 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e8c934910 0x558e8c940410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:04.374 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.372+0000 7fe6bd5bb640 1 -- 172.21.15.100:0/1903453131 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e8c92b0f0 con 0x558e8c92f7a0 2024-05-10T11:24:04.374 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.373+0000 7fe6bd5bb640 1 --2- 172.21.15.100:0/1903453131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e8c92f7a0 0x558e8c929940 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fe6a80058c0 tx=0x7fe6a8091040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d3555383c07f20a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:24:04.374 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.373+0000 7fe6a37fe640 1 -- 172.21.15.100:0/1903453131 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a8020040 con 0x558e8c92f7a0 2024-05-10T11:24:04.374 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.373+0000 7fe6a37fe640 1 -- 172.21.15.100:0/1903453131 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe6a8009de0 con 0x558e8c92f7a0 2024-05-10T11:24:04.374 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.373+0000 7fe6a37fe640 1 -- 172.21.15.100:0/1903453131 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a80b1ee0 con 0x558e8c92f7a0 2024-05-10T11:24:04.378 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.374+0000 7fe6c2e12740 1 -- 172.21.15.100:0/1903453131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e8c92f7a0 msgr2=0x558e8c929940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:04.378 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.374+0000 7fe6c2e12740 1 --2- 172.21.15.100:0/1903453131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e8c92f7a0 0x558e8c929940 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fe6a80058c0 tx=0x7fe6a8091040 comp rx=0 tx=0).stop 2024-05-10T11:24:04.378 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.374+0000 7fe6c2e12740 1 -- 172.21.15.100:0/1903453131 shutdown_connections 2024-05-10T11:24:04.378 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.374+0000 7fe6c2e12740 1 --2- 172.21.15.100:0/1903453131 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e8c934910 0x558e8c940410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:04.378 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.374+0000 7fe6c2e12740 1 --2- 172.21.15.100:0/1903453131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e8c92f7a0 0x558e8c929940 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:04.378 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.374+0000 7fe6c2e12740 1 --2- 172.21.15.100:0/1903453131 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558e8c92edb0 0x558e8c929400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:04.378 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.374+0000 7fe6c2e12740 1 -- 172.21.15.100:0/1903453131 >> 172.21.15.100:0/1903453131 conn(0x558e8c92a420 msgr2=0x558e8c928cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:24:04.378 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.374+0000 7fe6c2e12740 1 -- 172.21.15.100:0/1903453131 shutdown_connections 2024-05-10T11:24:04.378 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.374+0000 7fe6c2e12740 1 -- 172.21.15.100:0/1903453131 wait complete. 2024-05-10T11:24:04.379 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.374+0000 7fe6c2e12740 1 -- start start 2024-05-10T11:24:04.379 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.374+0000 7fe6c2e12740 1 Processor -- start 2024-05-10T11:24:04.379 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.374+0000 7fe6c2e12740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e8c92edb0 0x558e8c93d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:04.379 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.374+0000 7fe6c2e12740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558e8c92f7a0 0x558e8c93d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:04.379 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.374+0000 7fe6c2e12740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e8c934910 0x558e8c93e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:04.379 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.374+0000 7fe6c2e12740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b81a7750 con 0x558e8c934910 2024-05-10T11:24:04.379 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.374+0000 7fe6c2e12740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b81a8bc0 con 0x558e8c92edb0 2024-05-10T11:24:04.379 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.374+0000 7fe6c2e12740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b81a7e20 con 0x558e8c92f7a0 2024-05-10T11:24:04.379 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.374+0000 7fe6c2e12740 10 client.0 _ll_register_callbacks cb 0x7fe6b0009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:24:04.379 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.374+0000 7fe6bddbc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e8c92edb0 0x558e8c93d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:04.379 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.374+0000 7fe6be5bd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e8c934910 0x558e8c93e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:04.379 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.375+0000 7fe6bddbc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e8c92edb0 0x558e8c93d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:35726/0 (socket says 172.21.15.100:35726) 2024-05-10T11:24:04.379 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.375+0000 7fe6bddbc640 1 -- 172.21.15.100:0/1667191559 learned_addr learned my addr 172.21.15.100:0/1667191559 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:24:04.379 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.375+0000 7fe6bd5bb640 1 --2- 172.21.15.100:0/1667191559 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558e8c92f7a0 0x558e8c93d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:04.379 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.375+0000 7fe6bddbc640 1 -- 172.21.15.100:0/1667191559 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558e8c92f7a0 msgr2=0x558e8c93d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.375+0000 7fe6bddbc640 1 --2- 172.21.15.100:0/1667191559 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558e8c92f7a0 0x558e8c93d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.375+0000 7fe6bddbc640 1 -- 172.21.15.100:0/1667191559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e8c934910 msgr2=0x558e8c93e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.375+0000 7fe6bddbc640 1 --2- 172.21.15.100:0/1667191559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e8c934910 0x558e8c93e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.375+0000 7fe6bddbc640 1 -- 172.21.15.100:0/1667191559 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e8c93ef90 con 0x558e8c92edb0 2024-05-10T11:24:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.375+0000 7fe6be5bd640 1 --2- 172.21.15.100:0/1667191559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e8c934910 0x558e8c93e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:24:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.375+0000 7fe6bddbc640 1 --2- 172.21.15.100:0/1667191559 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e8c92edb0 0x558e8c93d450 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4011810 tx=0x7fe6b400a820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9183f846983438af server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:24:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.375+0000 7fe6a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:24:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.375+0000 7fe6a17fa640 1 -- 172.21.15.100:0/1667191559 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b4020040 con 0x558e8c92edb0 2024-05-10T11:24:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.375+0000 7fe6c2e12740 1 -- 172.21.15.100:0/1667191559 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6a8009de0 con 0x558e8c92edb0 2024-05-10T11:24:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.375+0000 7fe6a17fa640 1 -- 172.21.15.100:0/1667191559 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fe6b401a450 con 0x558e8c92edb0 2024-05-10T11:24:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.375+0000 7fe6a17fa640 1 -- 172.21.15.100:0/1667191559 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe6b403e050 con 0x558e8c92edb0 2024-05-10T11:24:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.375+0000 7fe6c2e12740 10 client.25976 fetch_fsmap learned FSMap version 31 2024-05-10T11:24:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.375+0000 7fe6c2e12740 1 -- 172.21.15.100:0/1667191559 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558e8c9402a0 con 0x558e8c92edb0 2024-05-10T11:24:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.376+0000 7fe6a17fa640 1 -- 172.21.15.100:0/1667191559 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe6a8009de0 con 0x558e8c92edb0 2024-05-10T11:24:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.376+0000 7fe6a17fa640 1 -- 172.21.15.100:0/1667191559 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fe6b401ace0 con 0x558e8c92edb0 2024-05-10T11:24:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.376+0000 7fe6c2e12740 10 client.25976 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:24:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.376+0000 7fe6c2e12740 10 client.25976 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:24:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.376+0000 7fe6c2e12740 20 client.25976 populate_metadata read hostname 'smithi100' 2024-05-10T11:24:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.376+0000 7fe6c2e12740 1 --2- 172.21.15.100:0/1667191559 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558e8c9b3760 0x558e8c9b5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.376+0000 7fe6c2e12740 4 client.25976 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:24:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.376+0000 7fe6c2e12740 1 -- 172.21.15.100:0/1667191559 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558e8c89b050 con 0x558e8c9b3760 2024-05-10T11:24:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.376+0000 7fe6bd5bb640 1 --2- 172.21.15.100:0/1667191559 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558e8c9b3760 0x558e8c9b5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.376+0000 7fe6a17fa640 10 client.25976 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:24:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.376+0000 7fe6bd5bb640 1 --2- 172.21.15.100:0/1667191559 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558e8c9b3760 0x558e8c9b5b80 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36060f27b7829447 server_cookie=8838c3fbecb4f21e in_seq=0 out_seq=0 2024-05-10T11:24:04.395 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.394+0000 7fe6a17fa640 1 -- 172.21.15.100:0/1667191559 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558e8c89b050 con 0x558e8c9b3760 2024-05-10T11:24:04.395 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.394+0000 7fe6a17fa640 10 client.25976 handle_command_reply: tid=0 2024-05-10T11:24:04.395 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.394+0000 7fe6c2e12740 1 client.25976 shutdown 2024-05-10T11:24:04.395 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.395+0000 7fe6c2e12740 1 -- 172.21.15.100:0/1667191559 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e8c92edb0 msgr2=0x558e8c93d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:04.395 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.395+0000 7fe6c2e12740 1 --2- 172.21.15.100:0/1667191559 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e8c92edb0 0x558e8c93d450 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4011810 tx=0x7fe6b400a820 comp rx=0 tx=0).stop 2024-05-10T11:24:04.395 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.395+0000 7fe6c2e12740 1 -- 172.21.15.100:0/1667191559 shutdown_connections 2024-05-10T11:24:04.395 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.395+0000 7fe6c2e12740 1 --2- 172.21.15.100:0/1667191559 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558e8c9b3760 0x558e8c9b5b80 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:04.395 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.395+0000 7fe6c2e12740 1 --2- 172.21.15.100:0/1667191559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e8c934910 0x558e8c93e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:04.395 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.395+0000 7fe6c2e12740 1 --2- 172.21.15.100:0/1667191559 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558e8c92f7a0 0x558e8c93d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:04.396 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.395+0000 7fe6a17fa640 0 client.25976 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:24:04.396 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.395+0000 7fe6c2e12740 1 --2- 172.21.15.100:0/1667191559 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e8c92edb0 0x558e8c93d450 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:04.396 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.395+0000 7fe6c2e12740 1 -- 172.21.15.100:0/1667191559 >> 172.21.15.100:0/1667191559 conn(0x558e8c92a420 msgr2=0x558e8c929580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:24:04.396 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.395+0000 7fe6c2e12740 1 -- 172.21.15.100:0/1667191559 shutdown_connections 2024-05-10T11:24:04.396 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.395+0000 7fe6c2e12740 1 -- 172.21.15.100:0/1667191559 wait complete. 2024-05-10T11:24:04.396 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.395+0000 7fe6c2e12740 20 client.25976 trim_cache size 0 max 16384 2024-05-10T11:24:04.396 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:24:04.396 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:24:04.396 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:24:04.396 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:24:04.397 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.396+0000 7fe6c0020640 1 -- 172.21.15.100:0/1991268377 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe6a40b28a0 msgr2=0x7fe6a40b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:04.397 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.396+0000 7fe6c0020640 1 --2- 172.21.15.100:0/1991268377 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe6a40b28a0 0x7fe6a40b4d60 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0006010 tx=0x7fe6b0031000 comp rx=0 tx=0).stop 2024-05-10T11:24:04.397 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.396+0000 7fe6c0020640 1 -- 172.21.15.100:0/1991268377 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6b8103c20 msgr2=0x7fe6b8199020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:04.397 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.396+0000 7fe6c0020640 1 --2- 172.21.15.100:0/1991268377 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6b8103c20 0x7fe6b8199020 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fe6a800d950 tx=0x7fe6a800de10 comp rx=0 tx=0).stop 2024-05-10T11:24:04.397 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.396+0000 7fe6c0020640 1 -- 172.21.15.100:0/1991268377 shutdown_connections 2024-05-10T11:24:04.397 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.396+0000 7fe6c0020640 1 --2- 172.21.15.100:0/1991268377 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe6a40b28a0 0x7fe6a40b4d60 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:04.398 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.396+0000 7fe6c0020640 1 --2- 172.21.15.100:0/1991268377 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6b81045e0 0x7fe6b81a0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:04.398 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.396+0000 7fe6c0020640 1 --2- 172.21.15.100:0/1991268377 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6b8103c20 0x7fe6b8199020 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:04.398 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.396+0000 7fe6c0020640 1 --2- 172.21.15.100:0/1991268377 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6b8102a30 0x7fe6b8198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:04.398 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.396+0000 7fe6c0020640 1 -- 172.21.15.100:0/1991268377 >> 172.21.15.100:0/1991268377 conn(0x7fe6b80fdfa0 msgr2=0x7fe6b8106fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:24:04.398 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.396+0000 7fe6c0020640 1 -- 172.21.15.100:0/1991268377 shutdown_connections 2024-05-10T11:24:04.398 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:04.396+0000 7fe6c0020640 1 -- 172.21.15.100:0/1991268377 wait complete. 2024-05-10T11:24:04.971 INFO:tasks.workunit.client.0.smithi100.stdout: 1 124990 2.74 MB/sec execute 119 sec latency 15.264 ms 2024-05-10T11:24:05.972 INFO:tasks.workunit.client.0.smithi100.stdout: 1 125473 2.73 MB/sec execute 120 sec latency 14.791 ms 2024-05-10T11:24:06.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:24:05 smithi178 ceph-mon[33359]: pgmap v591: 129 pgs: 129 active+clean; 61 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 1.4 MiB/s wr, 160 op/s 2024-05-10T11:24:06.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:24:05 smithi188 ceph-mon[34075]: pgmap v591: 129 pgs: 129 active+clean; 61 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 1.4 MiB/s wr, 160 op/s 2024-05-10T11:24:06.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:24:05 smithi100 ceph-mon[30296]: pgmap v591: 129 pgs: 129 active+clean; 61 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 1.4 MiB/s wr, 160 op/s 2024-05-10T11:24:06.972 INFO:tasks.workunit.client.0.smithi100.stdout: 1 126117 2.74 MB/sec execute 121 sec latency 16.014 ms 2024-05-10T11:24:07.971 INFO:tasks.workunit.client.0.smithi100.stdout: 1 126658 2.73 MB/sec execute 122 sec latency 16.747 ms 2024-05-10T11:24:08.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:24:07 smithi178 ceph-mon[33359]: pgmap v592: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 1.5 MiB/s wr, 164 op/s 2024-05-10T11:24:08.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:24:07 smithi188 ceph-mon[34075]: pgmap v592: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 1.5 MiB/s wr, 164 op/s 2024-05-10T11:24:08.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:24:07 smithi100 ceph-mon[30296]: pgmap v592: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 1.5 MiB/s wr, 164 op/s 2024-05-10T11:24:08.972 INFO:tasks.workunit.client.0.smithi100.stdout: 1 127464 2.77 MB/sec execute 123 sec latency 15.013 ms 2024-05-10T11:24:09.972 INFO:tasks.workunit.client.0.smithi100.stdout: 1 127861 2.77 MB/sec execute 124 sec latency 14.217 ms 2024-05-10T11:24:10.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:24:09 smithi178 ceph-mon[33359]: pgmap v593: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 1.4 MiB/s wr, 141 op/s 2024-05-10T11:24:10.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:24:09 smithi188 ceph-mon[34075]: pgmap v593: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 1.4 MiB/s wr, 141 op/s 2024-05-10T11:24:10.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:24:09 smithi100 ceph-mon[30296]: pgmap v593: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 1.4 MiB/s wr, 141 op/s 2024-05-10T11:24:10.972 INFO:tasks.workunit.client.0.smithi100.stdout: 1 128136 2.76 MB/sec execute 125 sec latency 15.101 ms 2024-05-10T11:24:11.972 INFO:tasks.workunit.client.0.smithi100.stdout: 1 128456 2.74 MB/sec execute 126 sec latency 15.553 ms 2024-05-10T11:24:12.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:24:11 smithi178 ceph-mon[33359]: pgmap v594: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 1.4 MiB/s wr, 161 op/s 2024-05-10T11:24:12.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:24:11 smithi188 ceph-mon[34075]: pgmap v594: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 1.4 MiB/s wr, 161 op/s 2024-05-10T11:24:12.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:24:11 smithi100 ceph-mon[30296]: pgmap v594: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 1.4 MiB/s wr, 161 op/s 2024-05-10T11:24:12.972 INFO:tasks.workunit.client.0.smithi100.stdout: 1 128938 2.73 MB/sec execute 127 sec latency 17.905 ms 2024-05-10T11:24:13.972 INFO:tasks.workunit.client.0.smithi100.stdout: 1 129559 2.74 MB/sec execute 128 sec latency 18.259 ms 2024-05-10T11:24:14.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:24:13 smithi178 ceph-mon[33359]: pgmap v595: 129 pgs: 129 active+clean; 64 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.4 MiB/s wr, 156 op/s 2024-05-10T11:24:14.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:24:13 smithi188 ceph-mon[34075]: pgmap v595: 129 pgs: 129 active+clean; 64 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.4 MiB/s wr, 156 op/s 2024-05-10T11:24:14.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:24:13 smithi100 ceph-mon[30296]: pgmap v595: 129 pgs: 129 active+clean; 64 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.4 MiB/s wr, 156 op/s 2024-05-10T11:24:14.972 INFO:tasks.workunit.client.0.smithi100.stdout: 1 130153 2.73 MB/sec execute 129 sec latency 15.795 ms 2024-05-10T11:24:15.972 INFO:tasks.workunit.client.0.smithi100.stdout: 1 130903 2.76 MB/sec execute 130 sec latency 15.811 ms 2024-05-10T11:24:16.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:24:15 smithi178 ceph-mon[33359]: pgmap v596: 129 pgs: 129 active+clean; 64 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 1.4 MiB/s wr, 145 op/s 2024-05-10T11:24:16.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:24:15 smithi188 ceph-mon[34075]: pgmap v596: 129 pgs: 129 active+clean; 64 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 1.4 MiB/s wr, 145 op/s 2024-05-10T11:24:16.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:24:15 smithi100 ceph-mon[30296]: pgmap v596: 129 pgs: 129 active+clean; 64 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 1.4 MiB/s wr, 145 op/s 2024-05-10T11:24:16.972 INFO:tasks.workunit.client.0.smithi100.stdout: 1 131359 2.77 MB/sec execute 131 sec latency 15.562 ms 2024-05-10T11:24:17.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:24:16 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:24:17.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:24:16 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:24:17.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:24:16 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:24:17.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:24:16 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:24:17.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:24:16 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:24:17.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:24:16 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:24:17.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:24:16 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:24:17.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:24:16 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:24:17.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:24:16 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:24:17.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:24:16 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:24:17.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:24:16 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:24:17.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:24:16 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:24:17.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:24:16 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:24:17.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:24:16 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:24:17.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:24:16 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:24:17.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:24:16 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:24:17.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:24:16 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:24:17.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:24:16 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:24:17.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:24:16 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:24:17.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:24:16 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:24:17.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:24:16 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:24:17.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:24:16 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:24:17.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:24:16 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:24:17.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:24:16 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:24:17.489 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.487+0000 7fc5148a0640 1 -- 172.21.15.100:0/4235826592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc510101910 msgr2=0x7fc510101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:17.490 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.487+0000 7fc5148a0640 1 --2- 172.21.15.100:0/4235826592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc510101910 0x7fc510101d30 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fc500009900 tx=0x7fc50002a410 comp rx=0 tx=0).stop 2024-05-10T11:24:17.490 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.487+0000 7fc5148a0640 1 -- 172.21.15.100:0/4235826592 shutdown_connections 2024-05-10T11:24:17.490 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.487+0000 7fc5148a0640 1 --2- 172.21.15.100:0/4235826592 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc5101034c0 0x7fc510109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:17.490 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.487+0000 7fc5148a0640 1 --2- 172.21.15.100:0/4235826592 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc510102b00 0x7fc510102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:17.490 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.487+0000 7fc5148a0640 1 --2- 172.21.15.100:0/4235826592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc510101910 0x7fc510101d30 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:17.490 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.487+0000 7fc5148a0640 1 -- 172.21.15.100:0/4235826592 >> 172.21.15.100:0/4235826592 conn(0x7fc5100fce80 msgr2=0x7fc5100ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:24:17.490 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.487+0000 7fc5148a0640 1 -- 172.21.15.100:0/4235826592 shutdown_connections 2024-05-10T11:24:17.490 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.487+0000 7fc5148a0640 1 -- 172.21.15.100:0/4235826592 wait complete. 2024-05-10T11:24:17.490 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.488+0000 7fc5148a0640 1 Processor -- start 2024-05-10T11:24:17.490 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.488+0000 7fc5148a0640 1 -- start start 2024-05-10T11:24:17.490 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.488+0000 7fc5148a0640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc510101910 0x7fc51006aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:17.491 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.488+0000 7fc5148a0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc510102b00 0x7fc51019d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:17.491 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.488+0000 7fc5148a0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5101034c0 0x7fc51019fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:17.491 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.488+0000 7fc5148a0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc51010d2e0 con 0x7fc5101034c0 2024-05-10T11:24:17.491 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.488+0000 7fc5148a0640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc51010d160 con 0x7fc510101910 2024-05-10T11:24:17.491 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.488+0000 7fc5148a0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc51010d460 con 0x7fc510102b00 2024-05-10T11:24:17.491 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.488+0000 7fc50ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5101034c0 0x7fc51019fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:17.491 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.488+0000 7fc50ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5101034c0 0x7fc51019fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33320/0 (socket says 172.21.15.100:33320) 2024-05-10T11:24:17.491 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.488+0000 7fc507fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc510102b00 0x7fc51019d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:17.491 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.488+0000 7fc50ed9d640 1 -- 172.21.15.100:0/3218629199 learned_addr learned my addr 172.21.15.100:0/3218629199 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:24:17.491 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.488+0000 7fc50e59c640 1 --2- 172.21.15.100:0/3218629199 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc510101910 0x7fc51006aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:17.491 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.489+0000 7fc50ed9d640 1 -- 172.21.15.100:0/3218629199 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc510102b00 msgr2=0x7fc51019d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:17.491 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.489+0000 7fc50ed9d640 1 --2- 172.21.15.100:0/3218629199 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc510102b00 0x7fc51019d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:17.491 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.489+0000 7fc50ed9d640 1 -- 172.21.15.100:0/3218629199 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc510101910 msgr2=0x7fc51006aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:17.491 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.489+0000 7fc50ed9d640 1 --2- 172.21.15.100:0/3218629199 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc510101910 0x7fc51006aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:17.492 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.489+0000 7fc50ed9d640 1 -- 172.21.15.100:0/3218629199 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5101a0760 con 0x7fc5101034c0 2024-05-10T11:24:17.492 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.489+0000 7fc507fff640 1 --2- 172.21.15.100:0/3218629199 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc510102b00 0x7fc51019d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:24:17.492 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.489+0000 7fc50e59c640 1 --2- 172.21.15.100:0/3218629199 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc510101910 0x7fc51006aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:24:17.492 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.489+0000 7fc50ed9d640 1 --2- 172.21.15.100:0/3218629199 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5101034c0 0x7fc51019fea0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fc50800eb10 tx=0x7fc50800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:24:17.492 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.489+0000 7fc5077fe640 1 -- 172.21.15.100:0/3218629199 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc508004b20 con 0x7fc5101034c0 2024-05-10T11:24:17.492 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.489+0000 7fc5077fe640 1 -- 172.21.15.100:0/3218629199 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc508004cc0 con 0x7fc5101034c0 2024-05-10T11:24:17.492 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.489+0000 7fc5148a0640 1 -- 172.21.15.100:0/3218629199 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5101a0a50 con 0x7fc5101034c0 2024-05-10T11:24:17.492 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.489+0000 7fc5077fe640 1 -- 172.21.15.100:0/3218629199 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc508010690 con 0x7fc5101034c0 2024-05-10T11:24:17.492 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.489+0000 7fc5148a0640 1 -- 172.21.15.100:0/3218629199 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5101a0f50 con 0x7fc5101034c0 2024-05-10T11:24:17.498 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.492+0000 7fc5077fe640 1 -- 172.21.15.100:0/3218629199 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fc508005020 con 0x7fc5101034c0 2024-05-10T11:24:17.498 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.492+0000 7fc5175bf740 1 Processor -- start 2024-05-10T11:24:17.498 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.492+0000 7fc5175bf740 1 -- start start 2024-05-10T11:24:17.498 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.492+0000 7fc5175bf740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560fd91d0db0 0x560fd91d11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:17.498 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.492+0000 7fc5175bf740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560fd91d17a0 0x560fd91d62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:17.498 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.492+0000 7fc5175bf740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560fd91d6910 0x560fd91d8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:17.498 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.492+0000 7fc5175bf740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x560fd91355d0 con 0x560fd91d0db0 2024-05-10T11:24:17.498 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.492+0000 7fc5175bf740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560fd90ee890 con 0x560fd91d6910 2024-05-10T11:24:17.498 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.492+0000 7fc5175bf740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x560fd9131590 con 0x560fd91d17a0 2024-05-10T11:24:17.499 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.492+0000 7fc507fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560fd91d0db0 0x560fd91d11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:17.499 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.492+0000 7fc507fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560fd91d0db0 0x560fd91d11d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33328/0 (socket says 172.21.15.100:33328) 2024-05-10T11:24:17.499 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.492+0000 7fc50e59c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560fd91d17a0 0x560fd91d62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:17.499 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.492+0000 7fc507fff640 1 -- 172.21.15.100:0/2977715314 learned_addr learned my addr 172.21.15.100:0/2977715314 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:24:17.499 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.492+0000 7fc507fff640 1 --2- 172.21.15.100:0/2977715314 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560fd91d6910 0x560fd91d8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:17.499 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.493+0000 7fc5077fe640 1 --2- 172.21.15.100:0/3218629199 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc4fc0b28a0 0x7fc4fc0b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:17.499 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.493+0000 7fc50e59c640 1 -- 172.21.15.100:0/2977715314 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560fd91d6910 msgr2=0x560fd91d8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:17.499 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.493+0000 7fc50e59c640 1 --2- 172.21.15.100:0/2977715314 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560fd91d6910 0x560fd91d8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:17.499 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.493+0000 7fc50e59c640 1 -- 172.21.15.100:0/2977715314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560fd91d0db0 msgr2=0x560fd91d11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:17.499 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.493+0000 7fc50e59c640 1 --2- 172.21.15.100:0/2977715314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560fd91d0db0 0x560fd91d11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:17.499 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.493+0000 7fc50e59c640 1 -- 172.21.15.100:0/2977715314 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fd8e47210 con 0x560fd91d17a0 2024-05-10T11:24:17.499 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.493+0000 7fc507fff640 1 --2- 172.21.15.100:0/2977715314 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560fd91d6910 0x560fd91d8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:24:17.499 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.493+0000 7fc50ed9d640 1 --2- 172.21.15.100:0/3218629199 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc4fc0b28a0 0x7fc4fc0b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:17.499 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.493+0000 7fc50e59c640 1 --2- 172.21.15.100:0/2977715314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560fd91d17a0 0x560fd91d62a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc500009230 tx=0x7fc500031940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ccfb602de2ee78b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:24:17.499 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.493+0000 7fc5077fe640 1 -- 172.21.15.100:0/3218629199 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fc5080e2b20 con 0x7fc5101034c0 2024-05-10T11:24:17.499 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.493+0000 7fc504ff9640 1 -- 172.21.15.100:0/2977715314 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc500035030 con 0x560fd91d17a0 2024-05-10T11:24:17.500 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.493+0000 7fc504ff9640 1 -- 172.21.15.100:0/2977715314 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc500040070 con 0x560fd91d17a0 2024-05-10T11:24:17.500 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.493+0000 7fc504ff9640 1 -- 172.21.15.100:0/2977715314 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc500003a50 con 0x560fd91d17a0 2024-05-10T11:24:17.500 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.494+0000 7fc5175bf740 1 -- 172.21.15.100:0/2977715314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560fd91d17a0 msgr2=0x560fd91d62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:17.500 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.494+0000 7fc5175bf740 1 --2- 172.21.15.100:0/2977715314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560fd91d17a0 0x560fd91d62a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc500009230 tx=0x7fc500031940 comp rx=0 tx=0).stop 2024-05-10T11:24:17.500 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.494+0000 7fc50ed9d640 1 --2- 172.21.15.100:0/3218629199 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc4fc0b28a0 0x7fc4fc0b4d60 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc50808e660 tx=0x7fc50806f870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:24:17.500 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.494+0000 7fc5175bf740 1 -- 172.21.15.100:0/2977715314 shutdown_connections 2024-05-10T11:24:17.500 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.494+0000 7fc5175bf740 1 --2- 172.21.15.100:0/2977715314 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560fd91d6910 0x560fd91d8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:17.500 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.494+0000 7fc5175bf740 1 --2- 172.21.15.100:0/2977715314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560fd91d17a0 0x560fd91d62a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:17.500 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.494+0000 7fc5175bf740 1 --2- 172.21.15.100:0/2977715314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560fd91d0db0 0x560fd91d11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:17.500 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.494+0000 7fc5175bf740 1 -- 172.21.15.100:0/2977715314 >> 172.21.15.100:0/2977715314 conn(0x560fd91cc420 msgr2=0x560fd91cc820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:24:17.500 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.494+0000 7fc5175bf740 1 -- 172.21.15.100:0/2977715314 shutdown_connections 2024-05-10T11:24:17.500 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.494+0000 7fc5175bf740 1 -- 172.21.15.100:0/2977715314 wait complete. 2024-05-10T11:24:17.500 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.494+0000 7fc5175bf740 1 -- start start 2024-05-10T11:24:17.500 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.494+0000 7fc5175bf740 1 Processor -- start 2024-05-10T11:24:17.500 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.495+0000 7fc5175bf740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560fd91d0db0 0x560fd91dafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:17.500 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.495+0000 7fc5175bf740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560fd91d17a0 0x560fd91db510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:17.501 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.495+0000 7fc5175bf740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560fd91d6910 0x560fd91e2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:17.501 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.495+0000 7fc5175bf740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x560fd91d9860 con 0x560fd91d17a0 2024-05-10T11:24:17.501 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.495+0000 7fc5175bf740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560fd90f2370 con 0x560fd91d0db0 2024-05-10T11:24:17.501 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.495+0000 7fc5175bf740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x560fd91d99e0 con 0x560fd91d6910 2024-05-10T11:24:17.501 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.495+0000 7fc5175bf740 10 client.0 _ll_register_callbacks cb 0x560fd8e46a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:24:17.501 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.495+0000 7fc50e59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560fd91d17a0 0x560fd91db510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:17.501 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.495+0000 7fc50e59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560fd91d17a0 0x560fd91db510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33330/0 (socket says 172.21.15.100:33330) 2024-05-10T11:24:17.501 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.495+0000 7fc50e59c640 1 -- 172.21.15.100:0/2470399434 learned_addr learned my addr 172.21.15.100:0/2470399434 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:24:17.501 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.495+0000 7fc507fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560fd91d0db0 0x560fd91dafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:17.501 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.495+0000 7fc507fff640 1 --2- 172.21.15.100:0/2470399434 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560fd91d6910 0x560fd91e2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:17.501 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.495+0000 7fc50e59c640 1 -- 172.21.15.100:0/2470399434 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560fd91d6910 msgr2=0x560fd91e2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:17.501 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.495+0000 7fc50e59c640 1 --2- 172.21.15.100:0/2470399434 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560fd91d6910 0x560fd91e2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:17.501 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.495+0000 7fc50e59c640 1 -- 172.21.15.100:0/2470399434 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560fd91d0db0 msgr2=0x560fd91dafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:17.501 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.495+0000 7fc50e59c640 1 --2- 172.21.15.100:0/2470399434 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560fd91d0db0 0x560fd91dafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:17.501 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.495+0000 7fc50e59c640 1 -- 172.21.15.100:0/2470399434 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fd8f13620 con 0x560fd91d17a0 2024-05-10T11:24:17.502 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.495+0000 7fc50e59c640 1 --2- 172.21.15.100:0/2470399434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560fd91d17a0 0x560fd91db510 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fc5000057c0 tx=0x7fc500031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6591a964e6081d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:24:17.502 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.495+0000 7fc4f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:24:17.502 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.495+0000 7fc4f5ffb640 1 -- 172.21.15.100:0/2470399434 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc500052060 con 0x560fd91d17a0 2024-05-10T11:24:17.502 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.495+0000 7fc5175bf740 1 -- 172.21.15.100:0/2470399434 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc500040070 con 0x560fd91d17a0 2024-05-10T11:24:17.502 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.495+0000 7fc4f5ffb640 1 -- 172.21.15.100:0/2470399434 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fc50003c440 con 0x560fd91d17a0 2024-05-10T11:24:17.502 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.496+0000 7fc4f5ffb640 1 -- 172.21.15.100:0/2470399434 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc500059050 con 0x560fd91d17a0 2024-05-10T11:24:17.502 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.496+0000 7fc5175bf740 10 client.16530 fetch_fsmap learned FSMap version 31 2024-05-10T11:24:17.502 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.496+0000 7fc5175bf740 1 -- 172.21.15.100:0/2470399434 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560fd91e5510 con 0x560fd91d17a0 2024-05-10T11:24:17.502 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.496+0000 7fc4f5ffb640 1 -- 172.21.15.100:0/2470399434 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc500040070 con 0x560fd91d17a0 2024-05-10T11:24:17.502 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.496+0000 7fc4f5ffb640 1 -- 172.21.15.100:0/2470399434 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fc50004d6b0 con 0x560fd91d17a0 2024-05-10T11:24:17.502 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.496+0000 7fc5175bf740 10 client.16530 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:24:17.502 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.496+0000 7fc5175bf740 10 client.16530 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:24:17.502 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.496+0000 7fc5175bf740 20 client.16530 populate_metadata read hostname 'smithi100' 2024-05-10T11:24:17.502 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.496+0000 7fc5175bf740 1 --2- 172.21.15.100:0/2470399434 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x560fd91e6860 0x560fd91e8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:17.503 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.496+0000 7fc5175bf740 4 client.16530 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:24:17.503 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.496+0000 7fc5175bf740 1 -- 172.21.15.100:0/2470399434 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560fd91431e0 con 0x560fd91e6860 2024-05-10T11:24:17.503 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.496+0000 7fc507fff640 1 --2- 172.21.15.100:0/2470399434 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x560fd91e6860 0x560fd91e8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:17.503 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.497+0000 7fc507fff640 1 --2- 172.21.15.100:0/2470399434 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x560fd91e6860 0x560fd91e8c50 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de9fdda3a8a47241 server_cookie=c4032bfa486efdd0 in_seq=0 out_seq=0 2024-05-10T11:24:17.503 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.497+0000 7fc4f5ffb640 10 client.16530 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:24:17.503 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.497+0000 7fc4f5ffb640 1 -- 172.21.15.100:0/2470399434 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560fd91431e0 con 0x560fd91e6860 2024-05-10T11:24:17.503 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.497+0000 7fc4f5ffb640 10 client.16530 handle_command_reply: tid=0 2024-05-10T11:24:17.503 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.497+0000 7fc5175bf740 1 client.16530 shutdown 2024-05-10T11:24:17.503 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.498+0000 7fc5175bf740 1 -- 172.21.15.100:0/2470399434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560fd91d17a0 msgr2=0x560fd91db510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:17.503 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.498+0000 7fc5175bf740 1 --2- 172.21.15.100:0/2470399434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560fd91d17a0 0x560fd91db510 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fc5000057c0 tx=0x7fc500031ca0 comp rx=0 tx=0).stop 2024-05-10T11:24:17.503 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.498+0000 7fc5175bf740 1 -- 172.21.15.100:0/2470399434 shutdown_connections 2024-05-10T11:24:17.503 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.498+0000 7fc5175bf740 1 --2- 172.21.15.100:0/2470399434 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x560fd91e6860 0x560fd91e8c50 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:17.503 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.498+0000 7fc5175bf740 1 --2- 172.21.15.100:0/2470399434 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560fd91d6910 0x560fd91e2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:17.503 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.498+0000 7fc5175bf740 1 --2- 172.21.15.100:0/2470399434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560fd91d17a0 0x560fd91db510 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:17.504 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.498+0000 7fc5175bf740 1 --2- 172.21.15.100:0/2470399434 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560fd91d0db0 0x560fd91dafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:17.504 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.498+0000 7fc5175bf740 1 -- 172.21.15.100:0/2470399434 >> 172.21.15.100:0/2470399434 conn(0x560fd91cc420 msgr2=0x560fd91cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:24:17.504 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.498+0000 7fc4f5ffb640 0 client.16530 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:24:17.504 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.498+0000 7fc5175bf740 1 -- 172.21.15.100:0/2470399434 shutdown_connections 2024-05-10T11:24:17.504 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.498+0000 7fc5175bf740 1 -- 172.21.15.100:0/2470399434 wait complete. 2024-05-10T11:24:17.504 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.498+0000 7fc5175bf740 20 client.16530 trim_cache size 0 max 16384 2024-05-10T11:24:17.519 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.514+0000 7fc5175bf740 1 Processor -- start 2024-05-10T11:24:17.519 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.514+0000 7fc5175bf740 1 -- start start 2024-05-10T11:24:17.519 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.514+0000 7fc5175bf740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560fd91d0db0 0x560fd91d2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:17.519 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.514+0000 7fc5175bf740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560fd91d17a0 0x560fd91d2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:17.519 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.514+0000 7fc5175bf740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560fd91d6910 0x560fd91e8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:17.519 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.514+0000 7fc5175bf740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc5101afdc0 con 0x560fd91d0db0 2024-05-10T11:24:17.519 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.514+0000 7fc5175bf740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc5101ad690 con 0x560fd91d6910 2024-05-10T11:24:17.519 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.514+0000 7fc5175bf740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc5101ab4e0 con 0x560fd91d17a0 2024-05-10T11:24:17.519 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.514+0000 7fc507fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560fd91d0db0 0x560fd91d2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:17.519 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.515+0000 7fc507fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560fd91d0db0 0x560fd91d2a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33340/0 (socket says 172.21.15.100:33340) 2024-05-10T11:24:17.519 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.515+0000 7fc507fff640 1 -- 172.21.15.100:0/1646662724 learned_addr learned my addr 172.21.15.100:0/1646662724 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:24:17.519 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.515+0000 7fc50e59c640 1 --2- 172.21.15.100:0/1646662724 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560fd91d17a0 0x560fd91d2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:17.520 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.515+0000 7fc507fff640 1 --2- 172.21.15.100:0/1646662724 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560fd91d6910 0x560fd91e8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:17.520 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.515+0000 7fc507fff640 1 -- 172.21.15.100:0/1646662724 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560fd91d17a0 msgr2=0x560fd91d2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:17.520 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.515+0000 7fc507fff640 1 --2- 172.21.15.100:0/1646662724 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560fd91d17a0 0x560fd91d2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:17.520 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.515+0000 7fc507fff640 1 -- 172.21.15.100:0/1646662724 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560fd91d6910 msgr2=0x560fd91e8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:17.520 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.515+0000 7fc507fff640 1 --2- 172.21.15.100:0/1646662724 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560fd91d6910 0x560fd91e8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:17.520 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.515+0000 7fc507fff640 1 -- 172.21.15.100:0/1646662724 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fd91cc250 con 0x560fd91d0db0 2024-05-10T11:24:17.520 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.515+0000 7fc507fff640 1 --2- 172.21.15.100:0/1646662724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560fd91d0db0 0x560fd91d2a20 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8013430 tx=0x7fc4f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae5bfa8277c9f143 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:24:17.520 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.515+0000 7fc4f7fff640 1 -- 172.21.15.100:0/1646662724 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f80036b0 con 0x560fd91d0db0 2024-05-10T11:24:17.520 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.515+0000 7fc4f7fff640 1 -- 172.21.15.100:0/1646662724 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc4f8003ce0 con 0x560fd91d0db0 2024-05-10T11:24:17.520 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.515+0000 7fc4f7fff640 1 -- 172.21.15.100:0/1646662724 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f8024ab0 con 0x560fd91d0db0 2024-05-10T11:24:17.520 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.516+0000 7fc5175bf740 1 -- 172.21.15.100:0/1646662724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560fd91d0db0 msgr2=0x560fd91d2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:17.520 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.516+0000 7fc5175bf740 1 --2- 172.21.15.100:0/1646662724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560fd91d0db0 0x560fd91d2a20 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8013430 tx=0x7fc4f80046c0 comp rx=0 tx=0).stop 2024-05-10T11:24:17.520 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.516+0000 7fc5175bf740 1 -- 172.21.15.100:0/1646662724 shutdown_connections 2024-05-10T11:24:17.520 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.516+0000 7fc5175bf740 1 --2- 172.21.15.100:0/1646662724 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560fd91d6910 0x560fd91e8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:17.520 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.516+0000 7fc5175bf740 1 --2- 172.21.15.100:0/1646662724 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560fd91d17a0 0x560fd91d2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:17.521 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.516+0000 7fc5175bf740 1 --2- 172.21.15.100:0/1646662724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560fd91d0db0 0x560fd91d2a20 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:17.521 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.516+0000 7fc5175bf740 1 -- 172.21.15.100:0/1646662724 >> 172.21.15.100:0/1646662724 conn(0x560fd91cc420 msgr2=0x560fd91d6060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:24:17.521 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.516+0000 7fc5175bf740 1 -- 172.21.15.100:0/1646662724 shutdown_connections 2024-05-10T11:24:17.521 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.516+0000 7fc5175bf740 1 -- 172.21.15.100:0/1646662724 wait complete. 2024-05-10T11:24:17.521 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.516+0000 7fc5175bf740 1 -- start start 2024-05-10T11:24:17.521 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.516+0000 7fc5175bf740 1 Processor -- start 2024-05-10T11:24:17.521 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.516+0000 7fc5175bf740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560fd91d0db0 0x560fd91df390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:17.521 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.516+0000 7fc5175bf740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560fd91d17a0 0x560fd91df8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:17.521 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.516+0000 7fc5175bf740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560fd91d6910 0x560fd91e0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:17.521 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.516+0000 7fc5175bf740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc5101abb50 con 0x560fd91d0db0 2024-05-10T11:24:17.521 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.516+0000 7fc5175bf740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc5101acfc0 con 0x560fd91d6910 2024-05-10T11:24:17.521 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.516+0000 7fc5175bf740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc5101ac220 con 0x560fd91d17a0 2024-05-10T11:24:17.521 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.516+0000 7fc5175bf740 10 client.0 _ll_register_callbacks cb 0x7fc4f8009be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:24:17.521 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.517+0000 7fc507fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560fd91d0db0 0x560fd91df390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:17.521 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.517+0000 7fc50e59c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560fd91d17a0 0x560fd91df8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:17.521 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.517+0000 7fc507fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560fd91d0db0 0x560fd91df390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33348/0 (socket says 172.21.15.100:33348) 2024-05-10T11:24:17.522 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.517+0000 7fc507fff640 1 -- 172.21.15.100:0/2053715836 learned_addr learned my addr 172.21.15.100:0/2053715836 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:24:17.522 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.517+0000 7fc507fff640 1 --2- 172.21.15.100:0/2053715836 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560fd91d6910 0x560fd91e0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:17.522 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.517+0000 7fc507fff640 1 -- 172.21.15.100:0/2053715836 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560fd91d17a0 msgr2=0x560fd91df8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:17.522 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.517+0000 7fc507fff640 1 --2- 172.21.15.100:0/2053715836 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560fd91d17a0 0x560fd91df8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:17.522 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.517+0000 7fc507fff640 1 -- 172.21.15.100:0/2053715836 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560fd91d6910 msgr2=0x560fd91e0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:17.522 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.517+0000 7fc507fff640 1 --2- 172.21.15.100:0/2053715836 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560fd91d6910 0x560fd91e0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:17.522 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.517+0000 7fc507fff640 1 -- 172.21.15.100:0/2053715836 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fd91da0b0 con 0x560fd91d0db0 2024-05-10T11:24:17.522 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.517+0000 7fc50e59c640 1 --2- 172.21.15.100:0/2053715836 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560fd91d17a0 0x560fd91df8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:24:17.522 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.517+0000 7fc507fff640 1 --2- 172.21.15.100:0/2053715836 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560fd91d0db0 0x560fd91df390 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fc4f80057c0 tx=0x7fc4f80045d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4686b73452b1dff8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:24:17.522 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.517+0000 7fc4f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:24:17.522 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.517+0000 7fc4f5ffb640 1 -- 172.21.15.100:0/2053715836 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f80036b0 con 0x560fd91d0db0 2024-05-10T11:24:17.522 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.517+0000 7fc5175bf740 1 -- 172.21.15.100:0/2053715836 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4f8003ce0 con 0x560fd91d0db0 2024-05-10T11:24:17.522 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.517+0000 7fc4f5ffb640 1 -- 172.21.15.100:0/2053715836 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fc4f8003850 con 0x560fd91d0db0 2024-05-10T11:24:17.522 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.517+0000 7fc4f5ffb640 1 -- 172.21.15.100:0/2053715836 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc4f8003a50 con 0x560fd91d0db0 2024-05-10T11:24:17.523 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.517+0000 7fc5175bf740 10 client.16536 fetch_fsmap learned FSMap version 31 2024-05-10T11:24:17.523 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.517+0000 7fc5175bf740 1 -- 172.21.15.100:0/2053715836 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560fd91e1fa0 con 0x560fd91d0db0 2024-05-10T11:24:17.523 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.517+0000 7fc4f5ffb640 1 -- 172.21.15.100:0/2053715836 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc4f8003ce0 con 0x560fd91d0db0 2024-05-10T11:24:17.523 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.518+0000 7fc4f5ffb640 1 -- 172.21.15.100:0/2053715836 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fc4f8014510 con 0x560fd91d0db0 2024-05-10T11:24:17.523 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.518+0000 7fc5175bf740 10 client.16536 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:24:17.523 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.518+0000 7fc5175bf740 10 client.16536 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:24:17.523 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.518+0000 7fc5175bf740 20 client.16536 populate_metadata read hostname 'smithi100' 2024-05-10T11:24:17.523 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.518+0000 7fc5175bf740 1 --2- 172.21.15.100:0/2053715836 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x560fd92516d0 0x560fd9253af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:17.523 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.518+0000 7fc5175bf740 4 client.16536 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:24:17.523 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.518+0000 7fc5175bf740 1 -- 172.21.15.100:0/2053715836 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560fd913d050 con 0x560fd92516d0 2024-05-10T11:24:17.523 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.518+0000 7fc50e59c640 1 --2- 172.21.15.100:0/2053715836 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x560fd92516d0 0x560fd9253af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:17.523 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.518+0000 7fc4f5ffb640 10 client.16536 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:24:17.523 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.518+0000 7fc50e59c640 1 --2- 172.21.15.100:0/2053715836 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x560fd92516d0 0x560fd9253af0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfa6fc99d98baaf server_cookie=6fa4455d54fca04a in_seq=0 out_seq=0 2024-05-10T11:24:17.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.536+0000 7fc4f5ffb640 1 -- 172.21.15.100:0/2053715836 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fc500040040 con 0x560fd92516d0 2024-05-10T11:24:17.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.536+0000 7fc4f5ffb640 10 client.16536 handle_command_reply: tid=0 2024-05-10T11:24:17.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.536+0000 7fc5175bf740 1 client.16536 shutdown 2024-05-10T11:24:17.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.536+0000 7fc5175bf740 1 -- 172.21.15.100:0/2053715836 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560fd91d0db0 msgr2=0x560fd91df390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:17.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.536+0000 7fc5175bf740 1 --2- 172.21.15.100:0/2053715836 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560fd91d0db0 0x560fd91df390 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fc4f80057c0 tx=0x7fc4f80045d0 comp rx=0 tx=0).stop 2024-05-10T11:24:17.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.536+0000 7fc5175bf740 1 -- 172.21.15.100:0/2053715836 shutdown_connections 2024-05-10T11:24:17.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.536+0000 7fc5175bf740 1 --2- 172.21.15.100:0/2053715836 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x560fd92516d0 0x560fd9253af0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:17.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.536+0000 7fc5175bf740 1 --2- 172.21.15.100:0/2053715836 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560fd91d6910 0x560fd91e0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:17.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.536+0000 7fc5175bf740 1 --2- 172.21.15.100:0/2053715836 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560fd91d17a0 0x560fd91df8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:17.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.536+0000 7fc5175bf740 1 --2- 172.21.15.100:0/2053715836 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560fd91d0db0 0x560fd91df390 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:17.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.536+0000 7fc5175bf740 1 -- 172.21.15.100:0/2053715836 >> 172.21.15.100:0/2053715836 conn(0x560fd91cc420 msgr2=0x560fd91dafb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:24:17.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.536+0000 7fc4f5ffb640 0 client.16536 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:24:17.538 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.536+0000 7fc5175bf740 1 -- 172.21.15.100:0/2053715836 shutdown_connections 2024-05-10T11:24:17.538 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.537+0000 7fc5175bf740 1 -- 172.21.15.100:0/2053715836 wait complete. 2024-05-10T11:24:17.538 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.537+0000 7fc5175bf740 20 client.16536 trim_cache size 0 max 16384 2024-05-10T11:24:17.538 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:24:17.538 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:24:17.538 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:24:17.538 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:24:17.539 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.537+0000 7fc5148a0640 1 -- 172.21.15.100:0/3218629199 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc4fc0b28a0 msgr2=0x7fc4fc0b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:17.539 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.537+0000 7fc5148a0640 1 --2- 172.21.15.100:0/3218629199 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc4fc0b28a0 0x7fc4fc0b4d60 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc50808e660 tx=0x7fc50806f870 comp rx=0 tx=0).stop 2024-05-10T11:24:17.539 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.537+0000 7fc5148a0640 1 -- 172.21.15.100:0/3218629199 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5101034c0 msgr2=0x7fc51019fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:17.539 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.537+0000 7fc5148a0640 1 --2- 172.21.15.100:0/3218629199 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5101034c0 0x7fc51019fea0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fc50800eb10 tx=0x7fc50800c2d0 comp rx=0 tx=0).stop 2024-05-10T11:24:17.539 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.537+0000 7fc5148a0640 1 -- 172.21.15.100:0/3218629199 shutdown_connections 2024-05-10T11:24:17.539 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.537+0000 7fc5148a0640 1 --2- 172.21.15.100:0/3218629199 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc4fc0b28a0 0x7fc4fc0b4d60 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:17.539 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.537+0000 7fc5148a0640 1 --2- 172.21.15.100:0/3218629199 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5101034c0 0x7fc51019fea0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:17.539 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.537+0000 7fc5148a0640 1 --2- 172.21.15.100:0/3218629199 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc510102b00 0x7fc51019d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:17.540 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.537+0000 7fc5148a0640 1 --2- 172.21.15.100:0/3218629199 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc510101910 0x7fc51006aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:17.540 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.537+0000 7fc5148a0640 1 -- 172.21.15.100:0/3218629199 >> 172.21.15.100:0/3218629199 conn(0x7fc5100fce80 msgr2=0x7fc510105f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:24:17.540 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.537+0000 7fc5148a0640 1 -- 172.21.15.100:0/3218629199 shutdown_connections 2024-05-10T11:24:17.540 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:17.537+0000 7fc5148a0640 1 -- 172.21.15.100:0/3218629199 wait complete. 2024-05-10T11:24:17.972 INFO:tasks.workunit.client.0.smithi100.stdout: 1 131670 2.76 MB/sec execute 132 sec latency 14.466 ms 2024-05-10T11:24:18.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:24:17 smithi178 ceph-mon[33359]: pgmap v597: 129 pgs: 129 active+clean; 64 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-05-10T11:24:18.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:24:17 smithi178 ceph-mon[33359]: Adjusting osd_memory_target on smithi188 to 176.1M 2024-05-10T11:24:18.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:24:17 smithi178 ceph-mon[33359]: Unable to set osd_memory_target on smithi188 to 184753049: error parsing value: Value '184753049' is below minimum 939524096 2024-05-10T11:24:18.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:24:17 smithi188 ceph-mon[34075]: pgmap v597: 129 pgs: 129 active+clean; 64 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-05-10T11:24:18.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:24:17 smithi188 ceph-mon[34075]: Adjusting osd_memory_target on smithi188 to 176.1M 2024-05-10T11:24:18.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:24:17 smithi188 ceph-mon[34075]: Unable to set osd_memory_target on smithi188 to 184753049: error parsing value: Value '184753049' is below minimum 939524096 2024-05-10T11:24:18.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:24:17 smithi100 ceph-mon[30296]: pgmap v597: 129 pgs: 129 active+clean; 64 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-05-10T11:24:18.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:24:17 smithi100 ceph-mon[30296]: Adjusting osd_memory_target on smithi188 to 176.1M 2024-05-10T11:24:18.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:24:17 smithi100 ceph-mon[30296]: Unable to set osd_memory_target on smithi188 to 184753049: error parsing value: Value '184753049' is below minimum 939524096 2024-05-10T11:24:18.972 INFO:tasks.workunit.client.0.smithi100.stdout: 1 132045 2.74 MB/sec execute 133 sec latency 12.289 ms 2024-05-10T11:24:19.972 INFO:tasks.workunit.client.0.smithi100.stdout: 1 132661 2.75 MB/sec execute 134 sec latency 16.916 ms 2024-05-10T11:24:20.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:24:19 smithi178 ceph-mon[33359]: pgmap v598: 129 pgs: 129 active+clean; 65 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 1.3 MiB/s wr, 139 op/s 2024-05-10T11:24:20.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:24:19 smithi188 ceph-mon[34075]: pgmap v598: 129 pgs: 129 active+clean; 65 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 1.3 MiB/s wr, 139 op/s 2024-05-10T11:24:20.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:24:19 smithi100 ceph-mon[30296]: pgmap v598: 129 pgs: 129 active+clean; 65 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 1.3 MiB/s wr, 139 op/s 2024-05-10T11:24:20.972 INFO:tasks.workunit.client.0.smithi100.stdout: 1 133312 2.74 MB/sec execute 135 sec latency 16.931 ms 2024-05-10T11:24:21.972 INFO:tasks.workunit.client.0.smithi100.stdout: 1 133837 2.74 MB/sec execute 136 sec latency 48.531 ms 2024-05-10T11:24:22.016 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T11:24:22.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:24:21 smithi178 ceph-mon[33359]: pgmap v599: 129 pgs: 129 active+clean; 61 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 1.2 MiB/s wr, 148 op/s 2024-05-10T11:24:22.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:24:21 smithi188 ceph-mon[34075]: pgmap v599: 129 pgs: 129 active+clean; 61 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 1.2 MiB/s wr, 148 op/s 2024-05-10T11:24:22.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:24:21 smithi100 ceph-mon[30296]: pgmap v599: 129 pgs: 129 active+clean; 61 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 1.2 MiB/s wr, 148 op/s 2024-05-10T11:24:22.292 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:24:22.972 INFO:tasks.workunit.client.0.smithi100.stdout: 1 134580 2.76 MB/sec execute 137 sec latency 13.692 ms 2024-05-10T11:24:23.966 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:24:23 smithi100 ceph-mon[30296]: pgmap v600: 129 pgs: 129 active+clean; 61 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 1.3 MiB/s wr, 140 op/s 2024-05-10T11:24:23.972 INFO:tasks.workunit.client.0.smithi100.stdout: 1 135033 2.77 MB/sec execute 138 sec latency 21.740 ms 2024-05-10T11:24:24.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.067+0000 7fd5fedf6700 1 -- 172.21.15.100:0/431888380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5f81037b0 msgr2=0x7fd5f8103b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:24.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.067+0000 7fd5fedf6700 1 --2- 172.21.15.100:0/431888380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5f81037b0 0x7fd5f8103b90 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec0099c0 tx=0x7fd5ec009cd0 comp rx=0 tx=0).stop 2024-05-10T11:24:24.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.067+0000 7fd5fedf6700 1 -- 172.21.15.100:0/431888380 shutdown_connections 2024-05-10T11:24:24.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.067+0000 7fd5fedf6700 1 --2- 172.21.15.100:0/431888380 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5f8108880 0x7fd5f8110400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:24.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.067+0000 7fd5fedf6700 1 --2- 172.21.15.100:0/431888380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5f81037b0 0x7fd5f8103b90 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:24.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.067+0000 7fd5fedf6700 1 --2- 172.21.15.100:0/431888380 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5f8104160 0x7fd5f8108150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:24.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.067+0000 7fd5fedf6700 1 -- 172.21.15.100:0/431888380 >> 172.21.15.100:0/431888380 conn(0x7fd5f807a530 msgr2=0x7fd5f807a940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:24:24.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.067+0000 7fd5fedf6700 1 -- 172.21.15.100:0/431888380 shutdown_connections 2024-05-10T11:24:24.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.067+0000 7fd5fedf6700 1 -- 172.21.15.100:0/431888380 wait complete. 2024-05-10T11:24:24.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.067+0000 7fd5fedf6700 1 Processor -- start 2024-05-10T11:24:24.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.067+0000 7fd5fedf6700 1 -- start start 2024-05-10T11:24:24.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.069+0000 7fd5fedf6700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5f81037b0 0x7fd5f819f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:24.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.069+0000 7fd5fedf6700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5f8104160 0x7fd5f819fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:24.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.069+0000 7fd5fedf6700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5f8108880 0x7fd5f8199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:24.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.069+0000 7fd5fedf6700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f8113500 con 0x7fd5f8108880 2024-05-10T11:24:24.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.069+0000 7fd5fedf6700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f8113380 con 0x7fd5f81037b0 2024-05-10T11:24:24.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.069+0000 7fd5fedf6700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f8113680 con 0x7fd5f8104160 2024-05-10T11:24:24.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.069+0000 7fd5fcb92700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5f81037b0 0x7fd5f819f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:24.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.069+0000 7fd5fcb92700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5f81037b0 0x7fd5f819f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:43284/0 (socket says 172.21.15.100:43284) 2024-05-10T11:24:24.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.069+0000 7fd5fcb92700 1 -- 172.21.15.100:0/1340622145 learned_addr learned my addr 172.21.15.100:0/1340622145 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:24:24.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.069+0000 7fd5fd393700 1 --2- 172.21.15.100:0/1340622145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5f8108880 0x7fd5f8199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:24.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.069+0000 7fd5f7fff700 1 --2- 172.21.15.100:0/1340622145 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5f8104160 0x7fd5f819fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:24.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.069+0000 7fd5fcb92700 1 -- 172.21.15.100:0/1340622145 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5f8104160 msgr2=0x7fd5f819fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:24.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.069+0000 7fd5fcb92700 1 --2- 172.21.15.100:0/1340622145 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5f8104160 0x7fd5f819fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:24.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.069+0000 7fd5fcb92700 1 -- 172.21.15.100:0/1340622145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5f8108880 msgr2=0x7fd5f8199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:24.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.069+0000 7fd5fcb92700 1 --2- 172.21.15.100:0/1340622145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5f8108880 0x7fd5f8199840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:24.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.069+0000 7fd5fcb92700 1 -- 172.21.15.100:0/1340622145 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5f819a020 con 0x7fd5f81037b0 2024-05-10T11:24:24.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.069+0000 7fd5f7fff700 1 --2- 172.21.15.100:0/1340622145 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5f8104160 0x7fd5f819fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:24:24.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.069+0000 7fd5fcb92700 1 --2- 172.21.15.100:0/1340622145 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5f81037b0 0x7fd5f819f6c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec006010 tx=0x7fd5ec004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:24:24.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.069+0000 7fd5f5ffb700 1 -- 172.21.15.100:0/1340622145 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec026070 con 0x7fd5f81037b0 2024-05-10T11:24:24.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.069+0000 7fd5f5ffb700 1 -- 172.21.15.100:0/1340622145 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd5ec004630 con 0x7fd5f81037b0 2024-05-10T11:24:24.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.069+0000 7fd5fedf6700 1 -- 172.21.15.100:0/1340622145 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5f819a2b0 con 0x7fd5f81037b0 2024-05-10T11:24:24.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.069+0000 7fd5f5ffb700 1 -- 172.21.15.100:0/1340622145 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec00faa0 con 0x7fd5f81037b0 2024-05-10T11:24:24.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.069+0000 7fd5fedf6700 1 -- 172.21.15.100:0/1340622145 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5f81a6520 con 0x7fd5f81037b0 2024-05-10T11:24:24.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.072+0000 7fd5f5ffb700 1 -- 172.21.15.100:0/1340622145 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fd5ec029070 con 0x7fd5f81037b0 2024-05-10T11:24:24.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.073+0000 7fd5f5ffb700 1 --2- 172.21.15.100:0/1340622145 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fd5e80b2880 0x7fd5e80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:24.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.073+0000 7fd5f7fff700 1 --2- 172.21.15.100:0/1340622145 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fd5e80b2880 0x7fd5e80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:24.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.073+0000 7fd5f5ffb700 1 -- 172.21.15.100:0/1340622145 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fd5ec0e2ca0 con 0x7fd5f81037b0 2024-05-10T11:24:24.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.073+0000 7fd609689b80 1 Processor -- start 2024-05-10T11:24:24.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.073+0000 7fd609689b80 1 -- start start 2024-05-10T11:24:24.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.073+0000 7fd609689b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5600c1720590 0x5600c1720970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:24.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.073+0000 7fd609689b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5600c1720eb0 0x5600c171ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:24.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.073+0000 7fd609689b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5600c171b1b0 0x5600c171b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:24.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.073+0000 7fd609689b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5600c1668940 con 0x5600c1720eb0 2024-05-10T11:24:24.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.073+0000 7fd609689b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5600c15cdda0 con 0x5600c1720590 2024-05-10T11:24:24.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.073+0000 7fd609689b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5600c1664900 con 0x5600c171b1b0 2024-05-10T11:24:24.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.073+0000 7fd5f7fff700 1 --2- 172.21.15.100:0/1340622145 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fd5e80b2880 0x7fd5e80b4d40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd5f81030b0 tx=0x7fd5e4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:24:24.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.074+0000 7fd5f7fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5600c1720eb0 0x5600c171ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:24.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.074+0000 7fd5fcb92700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5600c1720590 0x5600c1720970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:24.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.074+0000 7fd5f7fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5600c1720eb0 0x5600c171ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33370/0 (socket says 172.21.15.100:33370) 2024-05-10T11:24:24.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.074+0000 7fd5f7fff700 1 -- 172.21.15.100:0/2108509234 learned_addr learned my addr 172.21.15.100:0/2108509234 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:24:24.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.074+0000 7fd5fd393700 1 --2- 172.21.15.100:0/2108509234 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5600c171b1b0 0x5600c171b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:24.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.074+0000 7fd5f7fff700 1 -- 172.21.15.100:0/2108509234 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5600c171b1b0 msgr2=0x5600c171b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:24.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.074+0000 7fd5f7fff700 1 --2- 172.21.15.100:0/2108509234 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5600c171b1b0 0x5600c171b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:24.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.074+0000 7fd5f7fff700 1 -- 172.21.15.100:0/2108509234 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5600c1720590 msgr2=0x5600c1720970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:24.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.074+0000 7fd5f7fff700 1 --2- 172.21.15.100:0/2108509234 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5600c1720590 0x5600c1720970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:24.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.074+0000 7fd5f7fff700 1 -- 172.21.15.100:0/2108509234 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600c15b58d0 con 0x5600c1720eb0 2024-05-10T11:24:24.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.074+0000 7fd5f7fff700 1 --2- 172.21.15.100:0/2108509234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5600c1720eb0 0x5600c171ac20 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fd5e40043c0 tx=0x7fd5e4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=714128101457810a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:24:24.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.074+0000 7fd5e2ffd700 1 -- 172.21.15.100:0/2108509234 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e4003dc0 con 0x5600c1720eb0 2024-05-10T11:24:24.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.074+0000 7fd5e2ffd700 1 -- 172.21.15.100:0/2108509234 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd5e4026890 con 0x5600c1720eb0 2024-05-10T11:24:24.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.074+0000 7fd5e2ffd700 1 -- 172.21.15.100:0/2108509234 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e401e7f0 con 0x5600c1720eb0 2024-05-10T11:24:24.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.075+0000 7fd609689b80 1 -- 172.21.15.100:0/2108509234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5600c1720eb0 msgr2=0x5600c171ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:24.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.075+0000 7fd609689b80 1 --2- 172.21.15.100:0/2108509234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5600c1720eb0 0x5600c171ac20 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fd5e40043c0 tx=0x7fd5e4014000 comp rx=0 tx=0).stop 2024-05-10T11:24:24.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.075+0000 7fd609689b80 1 -- 172.21.15.100:0/2108509234 shutdown_connections 2024-05-10T11:24:24.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.075+0000 7fd609689b80 1 --2- 172.21.15.100:0/2108509234 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5600c171b1b0 0x5600c171b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:24.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.075+0000 7fd609689b80 1 --2- 172.21.15.100:0/2108509234 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5600c1720590 0x5600c1720970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:24.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.075+0000 7fd609689b80 1 --2- 172.21.15.100:0/2108509234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5600c1720eb0 0x5600c171ac20 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:24.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.075+0000 7fd609689b80 1 -- 172.21.15.100:0/2108509234 >> 172.21.15.100:0/2108509234 conn(0x5600c1642b70 msgr2=0x5600c17112a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:24:24.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.075+0000 7fd609689b80 1 -- 172.21.15.100:0/2108509234 shutdown_connections 2024-05-10T11:24:24.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.075+0000 7fd609689b80 1 -- 172.21.15.100:0/2108509234 wait complete. 2024-05-10T11:24:24.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.076+0000 7fd609689b80 1 -- start start 2024-05-10T11:24:24.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.076+0000 7fd609689b80 1 Processor -- start 2024-05-10T11:24:24.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.076+0000 7fd609689b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5600c171b1b0 0x5600c1726c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:24.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.076+0000 7fd609689b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5600c1720590 0x5600c1724010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:24.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.076+0000 7fd609689b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5600c1720eb0 0x5600c1724550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:24.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.076+0000 7fd609689b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5600c17278e0 con 0x5600c1720eb0 2024-05-10T11:24:24.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.076+0000 7fd609689b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5600c1621ed0 con 0x5600c1720590 2024-05-10T11:24:24.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.076+0000 7fd609689b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5600c1727a60 con 0x5600c171b1b0 2024-05-10T11:24:24.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.076+0000 7fd609689b80 10 client.0 _ll_register_callbacks cb 0x5600c15fc5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:24:24.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.076+0000 7fd5fcb92700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5600c171b1b0 0x5600c1726c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:24.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.076+0000 7fd5fd393700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5600c1720eb0 0x5600c1724550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:24.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.076+0000 7fd5fcb92700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5600c171b1b0 0x5600c1726c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:42002/0 (socket says 172.21.15.100:42002) 2024-05-10T11:24:24.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.076+0000 7fd5fcb92700 1 -- 172.21.15.100:0/1281107278 learned_addr learned my addr 172.21.15.100:0/1281107278 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:24:24.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.076+0000 7fd5f7fff700 1 --2- 172.21.15.100:0/1281107278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5600c1720590 0x5600c1724010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:24.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.077+0000 7fd5fcb92700 1 -- 172.21.15.100:0/1281107278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5600c1720590 msgr2=0x5600c1724010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:24.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.077+0000 7fd5fcb92700 1 --2- 172.21.15.100:0/1281107278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5600c1720590 0x5600c1724010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:24.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.077+0000 7fd5fcb92700 1 -- 172.21.15.100:0/1281107278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5600c1720eb0 msgr2=0x5600c1724550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:24.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.077+0000 7fd5fcb92700 1 --2- 172.21.15.100:0/1281107278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5600c1720eb0 0x5600c1724550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:24.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.077+0000 7fd5fcb92700 1 -- 172.21.15.100:0/1281107278 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600c161f760 con 0x5600c171b1b0 2024-05-10T11:24:24.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.077+0000 7fd5fcb92700 1 --2- 172.21.15.100:0/1281107278 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5600c171b1b0 0x5600c1726c70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec0e0440 tx=0x7fd5ec010f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d730e30c38547456 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:24:24.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.077+0000 7fd5d7fff700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:24:24.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.077+0000 7fd5d7fff700 1 -- 172.21.15.100:0/1281107278 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec003e00 con 0x5600c171b1b0 2024-05-10T11:24:24.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.077+0000 7fd609689b80 1 -- 172.21.15.100:0/1281107278 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5e4026890 con 0x5600c171b1b0 2024-05-10T11:24:24.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.077+0000 7fd5d7fff700 1 -- 172.21.15.100:0/1281107278 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fd5ec0b2b40 con 0x5600c171b1b0 2024-05-10T11:24:24.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.077+0000 7fd5d7fff700 1 -- 172.21.15.100:0/1281107278 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fd5ec00f460 con 0x5600c171b1b0 2024-05-10T11:24:24.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.077+0000 7fd609689b80 10 client.26029 fetch_fsmap learned FSMap version 31 2024-05-10T11:24:24.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.077+0000 7fd609689b80 1 -- 172.21.15.100:0/1281107278 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5600c1727940 con 0x5600c171b1b0 2024-05-10T11:24:24.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.077+0000 7fd5d7fff700 1 -- 172.21.15.100:0/1281107278 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd5e4026890 con 0x5600c171b1b0 2024-05-10T11:24:24.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.078+0000 7fd5d7fff700 1 -- 172.21.15.100:0/1281107278 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fd5ec0a5960 con 0x5600c171b1b0 2024-05-10T11:24:24.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.078+0000 7fd609689b80 10 client.26029 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:24:24.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.078+0000 7fd609689b80 10 client.26029 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:24:24.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.078+0000 7fd609689b80 20 client.26029 populate_metadata read hostname 'smithi100' 2024-05-10T11:24:24.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.078+0000 7fd609689b80 1 --2- 172.21.15.100:0/1281107278 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5600c17349f0 0x5600c1736de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:24.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.078+0000 7fd609689b80 4 client.26029 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:24:24.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.078+0000 7fd609689b80 1 -- 172.21.15.100:0/1281107278 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600c1676320 con 0x5600c17349f0 2024-05-10T11:24:24.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.078+0000 7fd5f7fff700 1 --2- 172.21.15.100:0/1281107278 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5600c17349f0 0x5600c1736de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:24.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.078+0000 7fd5f7fff700 1 --2- 172.21.15.100:0/1281107278 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5600c17349f0 0x5600c1736de0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c64260d3c6bf937d server_cookie=acd6b60055359c20 in_seq=0 out_seq=0 2024-05-10T11:24:24.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.078+0000 7fd5d7fff700 10 client.26029 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:24:24.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.079+0000 7fd5d7fff700 1 -- 172.21.15.100:0/1281107278 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5600c1676320 con 0x5600c17349f0 2024-05-10T11:24:24.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.079+0000 7fd5d7fff700 10 client.26029 handle_command_reply: tid=0 2024-05-10T11:24:24.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.079+0000 7fd609689b80 1 client.26029 shutdown 2024-05-10T11:24:24.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.079+0000 7fd609689b80 1 -- 172.21.15.100:0/1281107278 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5600c171b1b0 msgr2=0x5600c1726c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:24.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.079+0000 7fd609689b80 1 --2- 172.21.15.100:0/1281107278 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5600c171b1b0 0x5600c1726c70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec0e0440 tx=0x7fd5ec010f30 comp rx=0 tx=0).stop 2024-05-10T11:24:24.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.079+0000 7fd609689b80 1 -- 172.21.15.100:0/1281107278 shutdown_connections 2024-05-10T11:24:24.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.079+0000 7fd609689b80 1 --2- 172.21.15.100:0/1281107278 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5600c17349f0 0x5600c1736de0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:24.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.079+0000 7fd609689b80 1 --2- 172.21.15.100:0/1281107278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5600c1720eb0 0x5600c1724550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:24.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.079+0000 7fd609689b80 1 --2- 172.21.15.100:0/1281107278 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5600c171b1b0 0x5600c1726c70 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:24.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.079+0000 7fd609689b80 1 --2- 172.21.15.100:0/1281107278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5600c1720590 0x5600c1724010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:24.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.079+0000 7fd609689b80 1 -- 172.21.15.100:0/1281107278 >> 172.21.15.100:0/1281107278 conn(0x5600c1642b70 msgr2=0x5600c170bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:24:24.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.079+0000 7fd5d7fff700 0 client.26029 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:24:24.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.080+0000 7fd609689b80 1 -- 172.21.15.100:0/1281107278 shutdown_connections 2024-05-10T11:24:24.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.080+0000 7fd609689b80 1 -- 172.21.15.100:0/1281107278 wait complete. 2024-05-10T11:24:24.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.080+0000 7fd609689b80 20 client.26029 trim_cache size 0 max 16384 2024-05-10T11:24:24.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.101+0000 7fd609689b80 1 Processor -- start 2024-05-10T11:24:24.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.101+0000 7fd609689b80 1 -- start start 2024-05-10T11:24:24.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.101+0000 7fd609689b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5600c171b1b0 0x5600c171e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:24.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.102+0000 7fd609689b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5600c1720590 0x5600c1721b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:24.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.102+0000 7fd609689b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5600c1720eb0 0x5600c17309a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:24.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.102+0000 7fd609689b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f81af600 con 0x5600c1720eb0 2024-05-10T11:24:24.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.102+0000 7fd609689b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f81ace90 con 0x5600c171b1b0 2024-05-10T11:24:24.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.102+0000 7fd609689b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f81aace0 con 0x5600c1720590 2024-05-10T11:24:24.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.102+0000 7fd5fd393700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5600c1720eb0 0x5600c17309a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:24.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.102+0000 7fd5fcb92700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5600c171b1b0 0x5600c171e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:24.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.102+0000 7fd5f7fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5600c1720590 0x5600c1721b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:24.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.102+0000 7fd5fd393700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5600c1720eb0 0x5600c17309a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33380/0 (socket says 172.21.15.100:33380) 2024-05-10T11:24:24.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.102+0000 7fd5fcb92700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5600c171b1b0 0x5600c171e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:43320/0 (socket says 172.21.15.100:43320) 2024-05-10T11:24:24.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.102+0000 7fd5fd393700 1 -- 172.21.15.100:0/1726548264 learned_addr learned my addr 172.21.15.100:0/1726548264 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:24:24.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.102+0000 7fd5f7fff700 1 -- 172.21.15.100:0/1726548264 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5600c171b1b0 msgr2=0x5600c171e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:24.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.102+0000 7fd5f7fff700 1 --2- 172.21.15.100:0/1726548264 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5600c171b1b0 0x5600c171e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:24.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.102+0000 7fd5f7fff700 1 -- 172.21.15.100:0/1726548264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5600c1720eb0 msgr2=0x5600c17309a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:24.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.102+0000 7fd5f7fff700 1 --2- 172.21.15.100:0/1726548264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5600c1720eb0 0x5600c17309a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:24.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.102+0000 7fd5f7fff700 1 -- 172.21.15.100:0/1726548264 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600c1711f80 con 0x5600c1720590 2024-05-10T11:24:24.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.102+0000 7fd5fcb92700 1 --2- 172.21.15.100:0/1726548264 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5600c171b1b0 0x5600c171e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:24:24.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.102+0000 7fd5f7fff700 1 --2- 172.21.15.100:0/1726548264 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5600c1720590 0x5600c1721b20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x5600c1713310 tx=0x7fd5e40058e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48d0294c914f66e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:24:24.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.102+0000 7fd5e27fc700 1 -- 172.21.15.100:0/1726548264 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e4005240 con 0x5600c1720590 2024-05-10T11:24:24.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.102+0000 7fd5e27fc700 1 -- 172.21.15.100:0/1726548264 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd5e40053e0 con 0x5600c1720590 2024-05-10T11:24:24.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.102+0000 7fd5e27fc700 1 -- 172.21.15.100:0/1726548264 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e4033380 con 0x5600c1720590 2024-05-10T11:24:24.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.103+0000 7fd609689b80 1 -- 172.21.15.100:0/1726548264 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5600c1720590 msgr2=0x5600c1721b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:24.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.103+0000 7fd609689b80 1 --2- 172.21.15.100:0/1726548264 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5600c1720590 0x5600c1721b20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x5600c1713310 tx=0x7fd5e40058e0 comp rx=0 tx=0).stop 2024-05-10T11:24:24.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.103+0000 7fd609689b80 1 -- 172.21.15.100:0/1726548264 shutdown_connections 2024-05-10T11:24:24.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.103+0000 7fd609689b80 1 --2- 172.21.15.100:0/1726548264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5600c1720eb0 0x5600c17309a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:24.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.103+0000 7fd609689b80 1 --2- 172.21.15.100:0/1726548264 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5600c171b1b0 0x5600c171e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:24.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.103+0000 7fd609689b80 1 --2- 172.21.15.100:0/1726548264 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5600c1720590 0x5600c1721b20 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:24.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.103+0000 7fd609689b80 1 -- 172.21.15.100:0/1726548264 >> 172.21.15.100:0/1726548264 conn(0x5600c1642b70 msgr2=0x5600c1715800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:24:24.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.103+0000 7fd609689b80 1 -- 172.21.15.100:0/1726548264 shutdown_connections 2024-05-10T11:24:24.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.103+0000 7fd609689b80 1 -- 172.21.15.100:0/1726548264 wait complete. 2024-05-10T11:24:24.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.104+0000 7fd609689b80 1 -- start start 2024-05-10T11:24:24.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.104+0000 7fd609689b80 1 Processor -- start 2024-05-10T11:24:24.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.104+0000 7fd609689b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5600c171b1b0 0x5600c172cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:24.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.104+0000 7fd609689b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5600c1720590 0x5600c172d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:24.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.104+0000 7fd609689b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5600c1720eb0 0x5600c173b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:24.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.104+0000 7fd609689b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f81ab350 con 0x5600c1720590 2024-05-10T11:24:24.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.104+0000 7fd609689b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f81ac7c0 con 0x5600c1720eb0 2024-05-10T11:24:24.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.104+0000 7fd609689b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f81aba20 con 0x5600c171b1b0 2024-05-10T11:24:24.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.104+0000 7fd609689b80 10 client.0 _ll_register_callbacks cb 0x5600c15fc5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:24:24.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.104+0000 7fd5f7fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5600c1720590 0x5600c172d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:24.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.104+0000 7fd5f7fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5600c1720590 0x5600c172d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33394/0 (socket says 172.21.15.100:33394) 2024-05-10T11:24:24.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.104+0000 7fd5fcb92700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5600c171b1b0 0x5600c172cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:24.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.104+0000 7fd5f7fff700 1 -- 172.21.15.100:0/1886667605 learned_addr learned my addr 172.21.15.100:0/1886667605 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:24:24.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.104+0000 7fd5fd393700 1 --2- 172.21.15.100:0/1886667605 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5600c1720eb0 0x5600c173b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:24.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.105+0000 7fd5f7fff700 1 -- 172.21.15.100:0/1886667605 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5600c171b1b0 msgr2=0x5600c172cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:24.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.105+0000 7fd5f7fff700 1 --2- 172.21.15.100:0/1886667605 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5600c171b1b0 0x5600c172cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:24.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.105+0000 7fd5f7fff700 1 -- 172.21.15.100:0/1886667605 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5600c1720eb0 msgr2=0x5600c173b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:24.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.105+0000 7fd5f7fff700 1 --2- 172.21.15.100:0/1886667605 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5600c1720eb0 0x5600c173b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:24.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.105+0000 7fd5f7fff700 1 -- 172.21.15.100:0/1886667605 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600c173c0a0 con 0x5600c1720590 2024-05-10T11:24:24.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.105+0000 7fd5fd393700 1 --2- 172.21.15.100:0/1886667605 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5600c1720eb0 0x5600c173b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:24:24.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.105+0000 7fd5f7fff700 1 --2- 172.21.15.100:0/1886667605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5600c1720590 0x5600c172d440 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x5600c1713310 tx=0x7fd5e4005590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3958d8d10ad6ade server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:24:24.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.105+0000 7fd5d7fff700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:24:24.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.105+0000 7fd5d7fff700 1 -- 172.21.15.100:0/1886667605 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e4033320 con 0x5600c1720590 2024-05-10T11:24:24.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.105+0000 7fd609689b80 1 -- 172.21.15.100:0/1886667605 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5e40053e0 con 0x5600c1720590 2024-05-10T11:24:24.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.105+0000 7fd5d7fff700 1 -- 172.21.15.100:0/1886667605 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fd5e40334c0 con 0x5600c1720590 2024-05-10T11:24:24.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.105+0000 7fd5d7fff700 1 -- 172.21.15.100:0/1886667605 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fd5e40336c0 con 0x5600c1720590 2024-05-10T11:24:24.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.105+0000 7fd609689b80 10 client.16554 fetch_fsmap learned FSMap version 31 2024-05-10T11:24:24.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.105+0000 7fd609689b80 1 -- 172.21.15.100:0/1886667605 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5600c179ed40 con 0x5600c1720590 2024-05-10T11:24:24.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.106+0000 7fd5d7fff700 1 -- 172.21.15.100:0/1886667605 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd5e40053e0 con 0x5600c1720590 2024-05-10T11:24:24.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.106+0000 7fd5d7fff700 1 -- 172.21.15.100:0/1886667605 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fd5e4030cb0 con 0x5600c1720590 2024-05-10T11:24:24.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.106+0000 7fd609689b80 10 client.16554 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:24:24.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.106+0000 7fd609689b80 10 client.16554 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:24:24.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.106+0000 7fd609689b80 20 client.16554 populate_metadata read hostname 'smithi100' 2024-05-10T11:24:24.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.106+0000 7fd609689b80 1 --2- 172.21.15.100:0/1886667605 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5600c179f050 0x5600c17a1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:24.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.106+0000 7fd609689b80 4 client.16554 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T11:24:24.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.106+0000 7fd609689b80 1 -- 172.21.15.100:0/1886667605 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5600c16703c0 con 0x5600c179f050 2024-05-10T11:24:24.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.106+0000 7fd5fcb92700 1 --2- 172.21.15.100:0/1886667605 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5600c179f050 0x5600c17a1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:24.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.106+0000 7fd5d7fff700 10 client.16554 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:24:24.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.106+0000 7fd5fcb92700 1 --2- 172.21.15.100:0/1886667605 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5600c179f050 0x5600c17a1440 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=845d2e89f1c3ad8 server_cookie=941bdcae11fd37c3 in_seq=0 out_seq=0 2024-05-10T11:24:24.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.107+0000 7fd5d7fff700 1 -- 172.21.15.100:0/1886667605 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5600c16703c0 con 0x5600c179f050 2024-05-10T11:24:24.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.107+0000 7fd5d7fff700 10 client.16554 handle_command_reply: tid=0 2024-05-10T11:24:24.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.107+0000 7fd609689b80 1 client.16554 shutdown 2024-05-10T11:24:24.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.107+0000 7fd609689b80 1 -- 172.21.15.100:0/1886667605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5600c1720590 msgr2=0x5600c172d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:24.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.107+0000 7fd609689b80 1 --2- 172.21.15.100:0/1886667605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5600c1720590 0x5600c172d440 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x5600c1713310 tx=0x7fd5e4005590 comp rx=0 tx=0).stop 2024-05-10T11:24:24.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.107+0000 7fd609689b80 1 -- 172.21.15.100:0/1886667605 shutdown_connections 2024-05-10T11:24:24.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.107+0000 7fd609689b80 1 --2- 172.21.15.100:0/1886667605 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5600c179f050 0x5600c17a1440 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:24.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.107+0000 7fd609689b80 1 --2- 172.21.15.100:0/1886667605 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5600c1720eb0 0x5600c173b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:24.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.107+0000 7fd609689b80 1 --2- 172.21.15.100:0/1886667605 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5600c171b1b0 0x5600c172cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:24.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.107+0000 7fd5d7fff700 0 client.16554 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:24:24.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.107+0000 7fd609689b80 1 --2- 172.21.15.100:0/1886667605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5600c1720590 0x5600c172d440 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:24.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.107+0000 7fd609689b80 1 -- 172.21.15.100:0/1886667605 >> 172.21.15.100:0/1886667605 conn(0x5600c1642b70 msgr2=0x5600c1716090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:24:24.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.107+0000 7fd609689b80 1 -- 172.21.15.100:0/1886667605 shutdown_connections 2024-05-10T11:24:24.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.107+0000 7fd609689b80 1 -- 172.21.15.100:0/1886667605 wait complete. 2024-05-10T11:24:24.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.107+0000 7fd609689b80 20 client.16554 trim_cache size 0 max 16384 2024-05-10T11:24:24.113 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:24:24.113 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-05-10T11:24:24.113 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-05-10T11:24:24.114 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:24:24.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.108+0000 7fd5fedf6700 1 -- 172.21.15.100:0/1340622145 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fd5e80b2880 msgr2=0x7fd5e80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:24.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.108+0000 7fd5fedf6700 1 --2- 172.21.15.100:0/1340622145 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fd5e80b2880 0x7fd5e80b4d40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd5f81030b0 tx=0x7fd5e4009260 comp rx=0 tx=0).stop 2024-05-10T11:24:24.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.108+0000 7fd5fedf6700 1 -- 172.21.15.100:0/1340622145 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5f81037b0 msgr2=0x7fd5f819f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:24.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.108+0000 7fd5fedf6700 1 --2- 172.21.15.100:0/1340622145 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5f81037b0 0x7fd5f819f6c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec006010 tx=0x7fd5ec004270 comp rx=0 tx=0).stop 2024-05-10T11:24:24.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.108+0000 7fd5fedf6700 1 -- 172.21.15.100:0/1340622145 shutdown_connections 2024-05-10T11:24:24.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.108+0000 7fd5fedf6700 1 --2- 172.21.15.100:0/1340622145 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fd5e80b2880 0x7fd5e80b4d40 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:24.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.108+0000 7fd5fedf6700 1 --2- 172.21.15.100:0/1340622145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5f8108880 0x7fd5f8199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:24.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.108+0000 7fd5fedf6700 1 --2- 172.21.15.100:0/1340622145 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5f81037b0 0x7fd5f819f6c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:24.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.108+0000 7fd5fedf6700 1 --2- 172.21.15.100:0/1340622145 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5f8104160 0x7fd5f819fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:24.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.108+0000 7fd5fedf6700 1 -- 172.21.15.100:0/1340622145 >> 172.21.15.100:0/1340622145 conn(0x7fd5f807a530 msgr2=0x7fd5f8101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:24:24.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.108+0000 7fd5fedf6700 1 -- 172.21.15.100:0/1340622145 shutdown_connections 2024-05-10T11:24:24.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:24.108+0000 7fd5fedf6700 1 -- 172.21.15.100:0/1340622145 wait complete. 2024-05-10T11:24:24.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:24:23 smithi178 ceph-mon[33359]: pgmap v600: 129 pgs: 129 active+clean; 61 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 1.3 MiB/s wr, 140 op/s 2024-05-10T11:24:24.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:24:23 smithi188 ceph-mon[34075]: pgmap v600: 129 pgs: 129 active+clean; 61 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 1.3 MiB/s wr, 140 op/s 2024-05-10T11:24:24.972 INFO:tasks.workunit.client.0.smithi100.stdout: 1 135289 2.76 MB/sec execute 139 sec latency 11.875 ms 2024-05-10T11:24:24.981 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T11:24:24.981 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T11:24:25.254 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:24:25.972 INFO:tasks.workunit.client.0.smithi100.stdout: 1 135725 2.74 MB/sec execute 140 sec latency 12.759 ms 2024-05-10T11:24:26.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:24:25 smithi178 ceph-mon[33359]: pgmap v601: 129 pgs: 129 active+clean; 62 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 1.3 MiB/s wr, 159 op/s 2024-05-10T11:24:26.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:24:25 smithi188 ceph-mon[34075]: pgmap v601: 129 pgs: 129 active+clean; 62 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 1.3 MiB/s wr, 159 op/s 2024-05-10T11:24:26.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:24:25 smithi100 ceph-mon[30296]: pgmap v601: 129 pgs: 129 active+clean; 62 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 1.3 MiB/s wr, 159 op/s 2024-05-10T11:24:26.972 INFO:tasks.workunit.client.0.smithi100.stdout: 1 136203 2.75 MB/sec execute 141 sec latency 83.589 ms 2024-05-10T11:24:27.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:27.186+0000 7efd97844700 1 -- 172.21.15.100:0/2949327777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd9006b180 msgr2=0x7efd9006b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:27.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:27.186+0000 7efd97844700 1 --2- 172.21.15.100:0/2949327777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd9006b180 0x7efd9006b600 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7efd80009990 tx=0x7efd80009ca0 comp rx=0 tx=0).stop 2024-05-10T11:24:27.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:27.187+0000 7efd97844700 1 -- 172.21.15.100:0/2949327777 shutdown_connections 2024-05-10T11:24:27.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:27.187+0000 7efd97844700 1 --2- 172.21.15.100:0/2949327777 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd9006bb40 0x7efd90110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:27.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:27.187+0000 7efd97844700 1 --2- 172.21.15.100:0/2949327777 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd90109510 0x7efd901098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:27.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:27.187+0000 7efd97844700 1 --2- 172.21.15.100:0/2949327777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd9006b180 0x7efd9006b600 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:27.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:27.187+0000 7efd97844700 1 -- 172.21.15.100:0/2949327777 >> 172.21.15.100:0/2949327777 conn(0x7efd90079350 msgr2=0x7efd90079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:24:27.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:27.187+0000 7efd97844700 1 -- 172.21.15.100:0/2949327777 shutdown_connections 2024-05-10T11:24:27.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:27.187+0000 7efd97844700 1 -- 172.21.15.100:0/2949327777 wait complete. 2024-05-10T11:24:27.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:27.187+0000 7efd97844700 1 Processor -- start 2024-05-10T11:24:27.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:27.187+0000 7efd97844700 1 -- start start 2024-05-10T11:24:27.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:27.188+0000 7efd97844700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd9006b180 0x7efd9019f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:27.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:27.188+0000 7efd97844700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd9006bb40 0x7efd9019fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:27.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:27.188+0000 7efd97844700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd90109510 0x7efd901998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:27.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:27.188+0000 7efd97844700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7efd901134d0 con 0x7efd90109510 2024-05-10T11:24:27.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:27.188+0000 7efd97844700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efd90113350 con 0x7efd9006b180 2024-05-10T11:24:27.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:27.188+0000 7efd97844700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7efd90113650 con 0x7efd9006bb40 2024-05-10T11:24:27.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:27.188+0000 7efd95de1700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd90109510 0x7efd901998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:27.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:27.188+0000 7efd955e0700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd9006b180 0x7efd9019f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:27.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:27.188+0000 7efd94ddf700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd9006bb40 0x7efd9019fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:27.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:27.188+0000 7efd955e0700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd9006b180 0x7efd9019f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:38532/0 (socket says 172.21.15.100:38532) 2024-05-10T11:24:27.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:27.188+0000 7efd95de1700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd90109510 0x7efd901998b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57964/0 (socket says 172.21.15.100:57964) 2024-05-10T11:24:27.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:27.188+0000 7efd955e0700 1 -- 172.21.15.100:0/282570726 learned_addr learned my addr 172.21.15.100:0/282570726 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:24:27.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:27.188+0000 7efd955e0700 1 -- 172.21.15.100:0/282570726 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd9006bb40 msgr2=0x7efd9019fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:27.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:27.188+0000 7efd955e0700 1 --2- 172.21.15.100:0/282570726 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd9006bb40 0x7efd9019fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:27.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:27.188+0000 7efd955e0700 1 -- 172.21.15.100:0/282570726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd90109510 msgr2=0x7efd901998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:27.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:27.188+0000 7efd955e0700 1 --2- 172.21.15.100:0/282570726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd90109510 0x7efd901998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:27.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:27.188+0000 7efd955e0700 1 -- 172.21.15.100:0/282570726 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd9019a010 con 0x7efd9006b180 2024-05-10T11:24:27.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:27.188+0000 7efd95de1700 1 --2- 172.21.15.100:0/282570726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd90109510 0x7efd901998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:24:27.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:27.188+0000 7efd955e0700 1 --2- 172.21.15.100:0/282570726 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd9006b180 0x7efd9019f6f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7efd8c00e9a0 tx=0x7efd8c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:24:27.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:27.188+0000 7efd867fc700 1 -- 172.21.15.100:0/282570726 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8c00ca10 con 0x7efd9006b180 2024-05-10T11:24:27.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:27.189+0000 7efd97844700 1 -- 172.21.15.100:0/282570726 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd9019a300 con 0x7efd9006b180 2024-05-10T11:24:27.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:27.189+0000 7efd867fc700 1 -- 172.21.15.100:0/282570726 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efd8c00cbb0 con 0x7efd9006b180 2024-05-10T11:24:27.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:27.189+0000 7efd867fc700 1 -- 172.21.15.100:0/282570726 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8c018680 con 0x7efd9006b180 2024-05-10T11:24:27.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:27.189+0000 7efd97844700 1 -- 172.21.15.100:0/282570726 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd901023a0 con 0x7efd9006b180 2024-05-10T11:24:27.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:27.190+0000 7efd97844700 1 -- 172.21.15.100:0/282570726 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd9006a190 con 0x7efd9006b180 2024-05-10T11:24:27.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:27.192+0000 7efd867fc700 1 -- 172.21.15.100:0/282570726 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7efd8c010430 con 0x7efd9006b180 2024-05-10T11:24:27.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:27.193+0000 7efd867fc700 1 --2- 172.21.15.100:0/282570726 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7efd7c0b2880 0x7efd7c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:27.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:27.193+0000 7efd867fc700 1 -- 172.21.15.100:0/282570726 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7efd8c0e2a40 con 0x7efd9006b180 2024-05-10T11:24:27.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:27.193+0000 7efd94ddf700 1 --2- 172.21.15.100:0/282570726 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7efd7c0b2880 0x7efd7c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:27.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:27.193+0000 7efd94ddf700 1 --2- 172.21.15.100:0/282570726 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7efd7c0b2880 0x7efd7c0b4d40 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7efd90079a90 tx=0x7efd80005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:24:27.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:27.196+0000 7efd867fc700 1 -- 172.21.15.100:0/282570726 <== mon.2 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7efd8c0aa400 con 0x7efd9006b180 2024-05-10T11:24:27.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:27.518+0000 7efd97844700 1 -- 172.21.15.100:0/282570726 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efd9002e510 con 0x7efd9006b180 2024-05-10T11:24:27.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:27.519+0000 7efd867fc700 1 -- 172.21.15.100:0/282570726 <== mon.2 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11894 (secure 0 0 0) 0x7efd8c0a9d00 con 0x7efd9006b180 2024-05-10T11:24:27.520 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:24:27.520 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2072225012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2072225012},{"type":"v1","addr":"172.21.15.100:6835","nonce":2072225012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/401939540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":401939540},{"type":"v1","addr":"172.21.15.100:6837","nonce":401939540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14571,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/3946311025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3946311025},{"type":"v1","addr":"172.21.15.100:6839","nonce":3946311025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3736371660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3736371660},{"type":"v1","addr":"172.21.15.100:6841","nonce":3736371660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/2741810791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2741810791},{"type":"v1","addr":"172.21.15.178:6833","nonce":2741810791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1704891041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1704891041},{"type":"v1","addr":"172.21.15.188:6833","nonce":1704891041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T11:14:27.242452+0000","modified":"2024-05-10T11:23:19.557113+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24439,"mds_1":24437,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/2124076312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2124076312},{"type":"v1","addr":"172.21.15.178:6835","nonce":2124076312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/3224358425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3224358425},{"type":"v1","addr":"172.21.15.178:6837","nonce":3224358425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/515547690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":515547690},{"type":"v1","addr":"172.21.15.188:6835","nonce":515547690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/1943342620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1943342620},{"type":"v1","addr":"172.21.15.188:6837","nonce":1943342620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.178:6839/3760985404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3760985404},{"type":"v1","addr":"172.21.15.178:6839","nonce":3760985404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.188:6839/3194917070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":3194917070},{"type":"v1","addr":"172.21.15.188:6839","nonce":3194917070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24439,"qdb_cluster":[24439,24427,24437]},"id":1}]} 2024-05-10T11:24:27.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:27.522+0000 7efd97844700 1 -- 172.21.15.100:0/282570726 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7efd7c0b2880 msgr2=0x7efd7c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:27.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:27.522+0000 7efd97844700 1 --2- 172.21.15.100:0/282570726 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7efd7c0b2880 0x7efd7c0b4d40 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7efd90079a90 tx=0x7efd80005fb0 comp rx=0 tx=0).stop 2024-05-10T11:24:27.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:27.522+0000 7efd97844700 1 -- 172.21.15.100:0/282570726 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd9006b180 msgr2=0x7efd9019f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:27.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:27.522+0000 7efd97844700 1 --2- 172.21.15.100:0/282570726 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd9006b180 0x7efd9019f6f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7efd8c00e9a0 tx=0x7efd8c00ecb0 comp rx=0 tx=0).stop 2024-05-10T11:24:27.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:27.522+0000 7efd97844700 1 -- 172.21.15.100:0/282570726 shutdown_connections 2024-05-10T11:24:27.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:27.522+0000 7efd97844700 1 --2- 172.21.15.100:0/282570726 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7efd7c0b2880 0x7efd7c0b4d40 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:27.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:27.522+0000 7efd97844700 1 --2- 172.21.15.100:0/282570726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd90109510 0x7efd901998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:27.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:27.522+0000 7efd97844700 1 --2- 172.21.15.100:0/282570726 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd9006b180 0x7efd9019f6f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:27.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:27.522+0000 7efd97844700 1 --2- 172.21.15.100:0/282570726 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd9006bb40 0x7efd9019fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:27.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:27.522+0000 7efd97844700 1 -- 172.21.15.100:0/282570726 >> 172.21.15.100:0/282570726 conn(0x7efd90079350 msgr2=0x7efd90101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:24:27.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:27.522+0000 7efd97844700 1 -- 172.21.15.100:0/282570726 shutdown_connections 2024-05-10T11:24:27.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:27.522+0000 7efd97844700 1 -- 172.21.15.100:0/282570726 wait complete. 2024-05-10T11:24:27.524 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 31 2024-05-10T11:24:27.972 INFO:tasks.workunit.client.0.smithi100.stdout: 1 136800 2.74 MB/sec execute 142 sec latency 20.074 ms 2024-05-10T11:24:28.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:24:27 smithi178 ceph-mon[33359]: pgmap v602: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 1.4 MiB/s wr, 144 op/s 2024-05-10T11:24:28.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:24:27 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/282570726' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:24:28.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:24:27 smithi188 ceph-mon[34075]: pgmap v602: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 1.4 MiB/s wr, 144 op/s 2024-05-10T11:24:28.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:24:27 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/282570726' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:24:28.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:24:27 smithi100 ceph-mon[30296]: pgmap v602: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 1.4 MiB/s wr, 144 op/s 2024-05-10T11:24:28.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:24:27 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/282570726' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:24:28.321 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-10T11:24:28.592 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:24:28.972 INFO:tasks.workunit.client.0.smithi100.stdout: 1 137376 2.74 MB/sec execute 143 sec latency 15.970 ms 2024-05-10T11:24:29.972 INFO:tasks.workunit.client.0.smithi100.stdout: 1 138116 2.76 MB/sec execute 144 sec latency 14.784 ms 2024-05-10T11:24:30.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:24:29 smithi178 ceph-mon[33359]: pgmap v603: 129 pgs: 129 active+clean; 64 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 1.3 MiB/s wr, 140 op/s 2024-05-10T11:24:30.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:24:29 smithi188 ceph-mon[34075]: pgmap v603: 129 pgs: 129 active+clean; 64 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 1.3 MiB/s wr, 140 op/s 2024-05-10T11:24:30.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:24:29 smithi100 ceph-mon[30296]: pgmap v603: 129 pgs: 129 active+clean; 64 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 1.3 MiB/s wr, 140 op/s 2024-05-10T11:24:30.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.447+0000 7f3267234700 1 -- 172.21.15.100:0/831011618 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3260107760 msgr2=0x7f326010f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:30.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.447+0000 7f3267234700 1 --2- 172.21.15.100:0/831011618 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3260107760 0x7f326010f2e0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f325800b0d0 tx=0x7f325800b3e0 comp rx=0 tx=0).stop 2024-05-10T11:24:30.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.447+0000 7f3267234700 1 -- 172.21.15.100:0/831011618 shutdown_connections 2024-05-10T11:24:30.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.447+0000 7f3267234700 1 --2- 172.21.15.100:0/831011618 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3260107760 0x7f326010f2e0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:30.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.447+0000 7f3267234700 1 --2- 172.21.15.100:0/831011618 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3260102e00 0x7f32601031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:30.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.447+0000 7f3267234700 1 --2- 172.21.15.100:0/831011618 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32601037b0 0x7f3260107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:30.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.447+0000 7f3267234700 1 -- 172.21.15.100:0/831011618 >> 172.21.15.100:0/831011618 conn(0x7f3260078a10 msgr2=0x7f32600ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:24:30.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.447+0000 7f3267234700 1 -- 172.21.15.100:0/831011618 shutdown_connections 2024-05-10T11:24:30.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.447+0000 7f3267234700 1 -- 172.21.15.100:0/831011618 wait complete. 2024-05-10T11:24:30.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.447+0000 7f3267234700 1 Processor -- start 2024-05-10T11:24:30.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.447+0000 7f3267234700 1 -- start start 2024-05-10T11:24:30.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.448+0000 7f3267234700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3260102e00 0x7f32600fde80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:30.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.448+0000 7f3267234700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32601037b0 0x7f32600fe3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:30.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.448+0000 7f3267234700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3260107760 0x7f32600fe900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:30.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.448+0000 7f3267234700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f32601123e0 con 0x7f3260107760 2024-05-10T11:24:30.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.448+0000 7f3267234700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3260112260 con 0x7f3260102e00 2024-05-10T11:24:30.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.448+0000 7f3267234700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3260112560 con 0x7f32601037b0 2024-05-10T11:24:30.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.448+0000 7f32657d1700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3260107760 0x7f32600fe900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:30.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.448+0000 7f32657d1700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3260107760 0x7f32600fe900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57984/0 (socket says 172.21.15.100:57984) 2024-05-10T11:24:30.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.448+0000 7f3264fd0700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3260102e00 0x7f32600fde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:30.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.448+0000 7f32657d1700 1 -- 172.21.15.100:0/3840197617 learned_addr learned my addr 172.21.15.100:0/3840197617 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:24:30.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.448+0000 7f325ffff700 1 --2- 172.21.15.100:0/3840197617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32601037b0 0x7f32600fe3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:30.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.448+0000 7f32657d1700 1 -- 172.21.15.100:0/3840197617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32601037b0 msgr2=0x7f32600fe3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:30.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.448+0000 7f32657d1700 1 --2- 172.21.15.100:0/3840197617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32601037b0 0x7f32600fe3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:30.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.448+0000 7f32657d1700 1 -- 172.21.15.100:0/3840197617 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3260102e00 msgr2=0x7f32600fde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:30.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.448+0000 7f32657d1700 1 --2- 172.21.15.100:0/3840197617 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3260102e00 0x7f32600fde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:30.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.448+0000 7f32657d1700 1 -- 172.21.15.100:0/3840197617 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32601a7f30 con 0x7f3260107760 2024-05-10T11:24:30.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.448+0000 7f32657d1700 1 --2- 172.21.15.100:0/3840197617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3260107760 0x7f32600fe900 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f3258009850 tx=0x7f3258007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:24:30.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.449+0000 7f325dffb700 1 -- 172.21.15.100:0/3840197617 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f325801f070 con 0x7f3260107760 2024-05-10T11:24:30.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.449+0000 7f325dffb700 1 -- 172.21.15.100:0/3840197617 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3258004c50 con 0x7f3260107760 2024-05-10T11:24:30.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.449+0000 7f3267234700 1 -- 172.21.15.100:0/3840197617 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32601a81c0 con 0x7f3260107760 2024-05-10T11:24:30.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.449+0000 7f325dffb700 1 -- 172.21.15.100:0/3840197617 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f325801b690 con 0x7f3260107760 2024-05-10T11:24:30.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.449+0000 7f3267234700 1 -- 172.21.15.100:0/3840197617 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32601a85c0 con 0x7f3260107760 2024-05-10T11:24:30.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.452+0000 7f325dffb700 1 -- 172.21.15.100:0/3840197617 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f3258022070 con 0x7f3260107760 2024-05-10T11:24:30.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.453+0000 7f325dffb700 1 --2- 172.21.15.100:0/3840197617 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f32500b2880 0x7f32500b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:30.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.453+0000 7f3264fd0700 1 --2- 172.21.15.100:0/3840197617 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f32500b2880 0x7f32500b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:30.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.453+0000 7f325dffb700 1 -- 172.21.15.100:0/3840197617 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f32580e3480 con 0x7f3260107760 2024-05-10T11:24:30.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.453+0000 7f3271ac7b80 1 Processor -- start 2024-05-10T11:24:30.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.453+0000 7f3271ac7b80 1 -- start start 2024-05-10T11:24:30.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.453+0000 7f3271ac7b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e002f24d0 0x563e002f28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:30.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.453+0000 7f3271ac7b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e002f2df0 0x563e002f66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:30.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.453+0000 7f3271ac7b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563e002f6c80 0x563e002f7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:30.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.453+0000 7f3271ac7b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x563e00244860 con 0x563e002f24d0 2024-05-10T11:24:30.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.453+0000 7f3271ac7b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563e001e37c0 con 0x563e002f6c80 2024-05-10T11:24:30.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.453+0000 7f3271ac7b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563e00240820 con 0x563e002f2df0 2024-05-10T11:24:30.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.453+0000 7f3264fd0700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e002f24d0 0x563e002f28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:30.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.453+0000 7f3264fd0700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e002f24d0 0x563e002f28b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57994/0 (socket says 172.21.15.100:57994) 2024-05-10T11:24:30.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.453+0000 7f3264fd0700 1 -- 172.21.15.100:0/3387275240 learned_addr learned my addr 172.21.15.100:0/3387275240 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:24:30.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.453+0000 7f325ffff700 1 --2- 172.21.15.100:0/3387275240 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e002f2df0 0x563e002f66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:30.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.453+0000 7f3264fd0700 1 --2- 172.21.15.100:0/3840197617 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f32500b2880 0x7f32500b4d40 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3254009c60 tx=0x7f3254009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:24:30.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.453+0000 7f32657d1700 1 --2- 172.21.15.100:0/3387275240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563e002f6c80 0x563e002f7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:30.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.454+0000 7f3264fd0700 1 -- 172.21.15.100:0/3387275240 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e002f2df0 msgr2=0x563e002f66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:30.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.454+0000 7f3264fd0700 1 --2- 172.21.15.100:0/3387275240 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e002f2df0 0x563e002f66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:30.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.454+0000 7f3264fd0700 1 -- 172.21.15.100:0/3387275240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563e002f6c80 msgr2=0x563e002f7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:30.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.454+0000 7f3264fd0700 1 --2- 172.21.15.100:0/3387275240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563e002f6c80 0x563e002f7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:30.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.454+0000 7f3264fd0700 1 -- 172.21.15.100:0/3387275240 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e001918d0 con 0x563e002f24d0 2024-05-10T11:24:30.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.454+0000 7f32657d1700 1 --2- 172.21.15.100:0/3387275240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563e002f6c80 0x563e002f7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:24:30.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.454+0000 7f3264fd0700 1 --2- 172.21.15.100:0/3387275240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e002f24d0 0x563e002f28b0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f325400e430 tx=0x7f3254014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13e009b4e3bac35c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:24:30.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.454+0000 7f324affd700 1 -- 172.21.15.100:0/3387275240 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3254004830 con 0x563e002f24d0 2024-05-10T11:24:30.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.454+0000 7f324affd700 1 -- 172.21.15.100:0/3387275240 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3254005020 con 0x563e002f24d0 2024-05-10T11:24:30.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.454+0000 7f324affd700 1 -- 172.21.15.100:0/3387275240 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3254004e60 con 0x563e002f24d0 2024-05-10T11:24:30.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.455+0000 7f3271ac7b80 1 -- 172.21.15.100:0/3387275240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e002f24d0 msgr2=0x563e002f28b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:30.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.455+0000 7f3271ac7b80 1 --2- 172.21.15.100:0/3387275240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e002f24d0 0x563e002f28b0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f325400e430 tx=0x7f3254014000 comp rx=0 tx=0).stop 2024-05-10T11:24:30.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.455+0000 7f3271ac7b80 1 -- 172.21.15.100:0/3387275240 shutdown_connections 2024-05-10T11:24:30.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.455+0000 7f3271ac7b80 1 --2- 172.21.15.100:0/3387275240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563e002f6c80 0x563e002f7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:30.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.455+0000 7f3271ac7b80 1 --2- 172.21.15.100:0/3387275240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e002f24d0 0x563e002f28b0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:30.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.455+0000 7f3271ac7b80 1 --2- 172.21.15.100:0/3387275240 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e002f2df0 0x563e002f66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:30.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.455+0000 7f3271ac7b80 1 -- 172.21.15.100:0/3387275240 >> 172.21.15.100:0/3387275240 conn(0x563e0021e100 msgr2=0x563e002ecc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:24:30.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.455+0000 7f3271ac7b80 1 -- 172.21.15.100:0/3387275240 shutdown_connections 2024-05-10T11:24:30.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.455+0000 7f3271ac7b80 1 -- 172.21.15.100:0/3387275240 wait complete. 2024-05-10T11:24:30.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.455+0000 7f3271ac7b80 1 -- start start 2024-05-10T11:24:30.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.455+0000 7f3271ac7b80 1 Processor -- start 2024-05-10T11:24:30.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.456+0000 7f3271ac7b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e002f24d0 0x563e002fe080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:30.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.456+0000 7f3271ac7b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563e002f2df0 0x563e002fb420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:30.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.456+0000 7f3271ac7b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e002f6c80 0x563e002fb960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:30.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.456+0000 7f3271ac7b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x563e002fee90 con 0x563e002f6c80 2024-05-10T11:24:30.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.456+0000 7f3271ac7b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563e001aac30 con 0x563e002f2df0 2024-05-10T11:24:30.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.456+0000 7f3271ac7b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563e002ff010 con 0x563e002f24d0 2024-05-10T11:24:30.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.456+0000 7f3271ac7b80 10 client.0 _ll_register_callbacks cb 0x563e001dc920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:24:30.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.456+0000 7f32657d1700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e002f6c80 0x563e002fb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:30.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.456+0000 7f32657d1700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e002f6c80 0x563e002fb960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58004/0 (socket says 172.21.15.100:58004) 2024-05-10T11:24:30.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.456+0000 7f32657d1700 1 -- 172.21.15.100:0/3756762091 learned_addr learned my addr 172.21.15.100:0/3756762091 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:24:30.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.456+0000 7f325ffff700 1 --2- 172.21.15.100:0/3756762091 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563e002f2df0 0x563e002fb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:30.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.456+0000 7f3264fd0700 1 --2- 172.21.15.100:0/3756762091 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e002f24d0 0x563e002fe080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:30.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.456+0000 7f325ffff700 1 -- 172.21.15.100:0/3756762091 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e002f24d0 msgr2=0x563e002fe080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:30.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.456+0000 7f325ffff700 1 --2- 172.21.15.100:0/3756762091 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e002f24d0 0x563e002fe080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:30.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.456+0000 7f325ffff700 1 -- 172.21.15.100:0/3756762091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e002f6c80 msgr2=0x563e002fb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:30.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.456+0000 7f325ffff700 1 --2- 172.21.15.100:0/3756762091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e002f6c80 0x563e002fb960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:30.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.456+0000 7f325ffff700 1 -- 172.21.15.100:0/3756762091 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e001e0690 con 0x563e002f2df0 2024-05-10T11:24:30.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.456+0000 7f32657d1700 1 --2- 172.21.15.100:0/3756762091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e002f6c80 0x563e002fb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:24:30.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.457+0000 7f325ffff700 1 --2- 172.21.15.100:0/3756762091 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563e002f2df0 0x563e002fb420 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f32600ff910 tx=0x7f324c011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=feea49791356413d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:24:30.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.457+0000 7f323ffff700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:24:30.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.457+0000 7f323ffff700 1 -- 172.21.15.100:0/3756762091 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f324c004830 con 0x563e002f2df0 2024-05-10T11:24:30.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.457+0000 7f3271ac7b80 1 -- 172.21.15.100:0/3756762091 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3254005020 con 0x563e002f2df0 2024-05-10T11:24:30.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.457+0000 7f323ffff700 1 -- 172.21.15.100:0/3756762091 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f324c0049d0 con 0x563e002f2df0 2024-05-10T11:24:30.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.457+0000 7f323ffff700 1 -- 172.21.15.100:0/3756762091 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f324c025050 con 0x563e002f2df0 2024-05-10T11:24:30.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.457+0000 7f3271ac7b80 10 client.26030 fetch_fsmap learned FSMap version 31 2024-05-10T11:24:30.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.457+0000 7f3271ac7b80 1 -- 172.21.15.100:0/3756762091 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563e003096b0 con 0x563e002f2df0 2024-05-10T11:24:30.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.457+0000 7f323ffff700 1 -- 172.21.15.100:0/3756762091 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3254005020 con 0x563e002f2df0 2024-05-10T11:24:30.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.457+0000 7f323ffff700 1 -- 172.21.15.100:0/3756762091 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f324c01a710 con 0x563e002f2df0 2024-05-10T11:24:30.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.458+0000 7f3271ac7b80 10 client.26030 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:24:30.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.458+0000 7f3271ac7b80 10 client.26030 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-10T11:24:30.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.458+0000 7f3271ac7b80 20 client.26030 populate_metadata read hostname 'smithi100' 2024-05-10T11:24:30.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.458+0000 7f3271ac7b80 1 --2- 172.21.15.100:0/3756762091 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x563e0030bd40 0x563e0030e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:30.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.458+0000 7f3271ac7b80 4 client.26030 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:24:30.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.458+0000 7f3271ac7b80 1 -- 172.21.15.100:0/3756762091 --> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e002be650 con 0x563e0030bd40 2024-05-10T11:24:30.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.458+0000 7f3264fd0700 1 --2- 172.21.15.100:0/3756762091 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x563e0030bd40 0x563e0030e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:30.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.458+0000 7f3264fd0700 1 --2- 172.21.15.100:0/3756762091 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x563e0030bd40 0x563e0030e160 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a7e7375fda2e9985 server_cookie=90ad5a588759c707 in_seq=0 out_seq=0 2024-05-10T11:24:30.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.458+0000 7f323ffff700 10 client.26030 ms_handle_connect on v2:172.21.15.188:6836/1943342620 2024-05-10T11:24:30.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.459+0000 7f323ffff700 1 -- 172.21.15.100:0/3756762091 <== mds.2 v2:172.21.15.188:6836/1943342620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563e002be650 con 0x563e0030bd40 2024-05-10T11:24:30.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.459+0000 7f323ffff700 10 client.26030 handle_command_reply: tid=0 2024-05-10T11:24:30.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.459+0000 7f3271ac7b80 1 client.26030 shutdown 2024-05-10T11:24:30.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.459+0000 7f3271ac7b80 1 -- 172.21.15.100:0/3756762091 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563e002f2df0 msgr2=0x563e002fb420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:30.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.459+0000 7f3271ac7b80 1 --2- 172.21.15.100:0/3756762091 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563e002f2df0 0x563e002fb420 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f32600ff910 tx=0x7f324c011720 comp rx=0 tx=0).stop 2024-05-10T11:24:30.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.459+0000 7f3271ac7b80 1 -- 172.21.15.100:0/3756762091 shutdown_connections 2024-05-10T11:24:30.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.459+0000 7f3271ac7b80 1 --2- 172.21.15.100:0/3756762091 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x563e0030bd40 0x563e0030e160 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:30.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.459+0000 7f3271ac7b80 1 --2- 172.21.15.100:0/3756762091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e002f6c80 0x563e002fb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:30.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.459+0000 7f3271ac7b80 1 --2- 172.21.15.100:0/3756762091 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e002f24d0 0x563e002fe080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:30.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.459+0000 7f3271ac7b80 1 --2- 172.21.15.100:0/3756762091 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563e002f2df0 0x563e002fb420 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:30.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.459+0000 7f323ffff700 0 client.26030 ms_handle_reset on v2:172.21.15.188:6836/1943342620 2024-05-10T11:24:30.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.459+0000 7f3271ac7b80 1 -- 172.21.15.100:0/3756762091 >> 172.21.15.100:0/3756762091 conn(0x563e0021e100 msgr2=0x563e002e7720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:24:30.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.459+0000 7f3271ac7b80 1 -- 172.21.15.100:0/3756762091 shutdown_connections 2024-05-10T11:24:30.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.459+0000 7f3271ac7b80 1 -- 172.21.15.100:0/3756762091 wait complete. 2024-05-10T11:24:30.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.459+0000 7f3271ac7b80 20 client.26030 trim_cache size 0 max 16384 2024-05-10T11:24:30.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.476+0000 7f3271ac7b80 1 Processor -- start 2024-05-10T11:24:30.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.477+0000 7f3271ac7b80 1 -- start start 2024-05-10T11:24:30.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.477+0000 7f3271ac7b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563e002f24d0 0x563e00307f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:30.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.477+0000 7f3271ac7b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e002f2df0 0x563e00313a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:30.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.477+0000 7f3271ac7b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e002f6c80 0x563e00313f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:30.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.477+0000 7f3271ac7b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f32601b3b00 con 0x563e002f2df0 2024-05-10T11:24:30.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.477+0000 7f3271ac7b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f32601b1390 con 0x563e002f24d0 2024-05-10T11:24:30.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.477+0000 7f3271ac7b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f32601af1e0 con 0x563e002f6c80 2024-05-10T11:24:30.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.477+0000 7f325ffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e002f2df0 0x563e00313a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:30.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.477+0000 7f325ffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e002f2df0 0x563e00313a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58006/0 (socket says 172.21.15.100:58006) 2024-05-10T11:24:30.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.477+0000 7f3264fd0700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563e002f24d0 0x563e00307f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:30.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.477+0000 7f325ffff700 1 -- 172.21.15.100:0/1213681558 learned_addr learned my addr 172.21.15.100:0/1213681558 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:24:30.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.477+0000 7f32657d1700 1 --2- 172.21.15.100:0/1213681558 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e002f6c80 0x563e00313f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:30.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.477+0000 7f3264fd0700 1 -- 172.21.15.100:0/1213681558 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e002f6c80 msgr2=0x563e00313f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:30.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.477+0000 7f3264fd0700 1 --2- 172.21.15.100:0/1213681558 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e002f6c80 0x563e00313f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:30.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.477+0000 7f3264fd0700 1 -- 172.21.15.100:0/1213681558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e002f2df0 msgr2=0x563e00313a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:30.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.477+0000 7f3264fd0700 1 --2- 172.21.15.100:0/1213681558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e002f2df0 0x563e00313a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:30.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.477+0000 7f3264fd0700 1 -- 172.21.15.100:0/1213681558 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e002fe3f0 con 0x563e002f24d0 2024-05-10T11:24:30.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.478+0000 7f3264fd0700 1 --2- 172.21.15.100:0/1213681558 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563e002f24d0 0x563e00307f20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f325400e400 tx=0x7f3254005ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4e692a5a7b4d145 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:24:30.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.478+0000 7f324a7fc700 1 -- 172.21.15.100:0/1213681558 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3254005550 con 0x563e002f24d0 2024-05-10T11:24:30.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.478+0000 7f324a7fc700 1 -- 172.21.15.100:0/1213681558 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3254003b10 con 0x563e002f24d0 2024-05-10T11:24:30.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.478+0000 7f324a7fc700 1 -- 172.21.15.100:0/1213681558 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3254026770 con 0x563e002f24d0 2024-05-10T11:24:30.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.479+0000 7f3271ac7b80 1 -- 172.21.15.100:0/1213681558 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563e002f24d0 msgr2=0x563e00307f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:30.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.479+0000 7f3271ac7b80 1 --2- 172.21.15.100:0/1213681558 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563e002f24d0 0x563e00307f20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f325400e400 tx=0x7f3254005ce0 comp rx=0 tx=0).stop 2024-05-10T11:24:30.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.479+0000 7f3271ac7b80 1 -- 172.21.15.100:0/1213681558 shutdown_connections 2024-05-10T11:24:30.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.479+0000 7f3271ac7b80 1 --2- 172.21.15.100:0/1213681558 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e002f6c80 0x563e00313f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:30.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.479+0000 7f3271ac7b80 1 --2- 172.21.15.100:0/1213681558 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563e002f24d0 0x563e00307f20 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:30.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.479+0000 7f3271ac7b80 1 --2- 172.21.15.100:0/1213681558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e002f2df0 0x563e00313a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:30.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.479+0000 7f3271ac7b80 1 -- 172.21.15.100:0/1213681558 >> 172.21.15.100:0/1213681558 conn(0x563e0021e100 msgr2=0x563e002e7720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:24:30.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.479+0000 7f3271ac7b80 1 -- 172.21.15.100:0/1213681558 shutdown_connections 2024-05-10T11:24:30.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.479+0000 7f3271ac7b80 1 -- 172.21.15.100:0/1213681558 wait complete. 2024-05-10T11:24:30.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.479+0000 7f3271ac7b80 1 -- start start 2024-05-10T11:24:30.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.479+0000 7f3271ac7b80 1 Processor -- start 2024-05-10T11:24:30.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.480+0000 7f3271ac7b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e002f24d0 0x563e00301180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:30.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.480+0000 7f3271ac7b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563e002f2df0 0x563e003016c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:30.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.480+0000 7f3271ac7b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e002f6c80 0x563e00301c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:30.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.480+0000 7f3271ac7b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f32601af850 con 0x563e002f24d0 2024-05-10T11:24:30.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.480+0000 7f3271ac7b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f32601b0cc0 con 0x563e002f2df0 2024-05-10T11:24:30.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.480+0000 7f3271ac7b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f32601aff20 con 0x563e002f6c80 2024-05-10T11:24:30.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.480+0000 7f3264fd0700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e002f24d0 0x563e00301180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:30.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.480+0000 7f325ffff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563e002f2df0 0x563e003016c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:30.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.480+0000 7f3264fd0700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e002f24d0 0x563e00301180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58014/0 (socket says 172.21.15.100:58014) 2024-05-10T11:24:30.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.480+0000 7f3271ac7b80 10 client.0 _ll_register_callbacks cb 0x563e001dc920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:24:30.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.480+0000 7f3264fd0700 1 -- 172.21.15.100:0/4206603269 learned_addr learned my addr 172.21.15.100:0/4206603269 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:24:30.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.480+0000 7f32657d1700 1 --2- 172.21.15.100:0/4206603269 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e002f6c80 0x563e00301c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:30.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.480+0000 7f325ffff700 1 -- 172.21.15.100:0/4206603269 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e002f6c80 msgr2=0x563e00301c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:30.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.480+0000 7f325ffff700 1 --2- 172.21.15.100:0/4206603269 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e002f6c80 0x563e00301c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:30.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.480+0000 7f325ffff700 1 -- 172.21.15.100:0/4206603269 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e002f24d0 msgr2=0x563e00301180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:30.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.480+0000 7f325ffff700 1 --2- 172.21.15.100:0/4206603269 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e002f24d0 0x563e00301180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:30.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.480+0000 7f325ffff700 1 -- 172.21.15.100:0/4206603269 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e003748e0 con 0x563e002f2df0 2024-05-10T11:24:30.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.480+0000 7f32657d1700 1 --2- 172.21.15.100:0/4206603269 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e002f6c80 0x563e00301c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:24:30.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.480+0000 7f3264fd0700 1 --2- 172.21.15.100:0/4206603269 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e002f24d0 0x563e00301180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:24:30.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.480+0000 7f325ffff700 1 --2- 172.21.15.100:0/4206603269 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563e002f2df0 0x563e003016c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x563e002ee680 tx=0x7f324c011b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b86bbbaf7031d997 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:24:30.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.480+0000 7f323ffff700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:24:30.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.481+0000 7f323ffff700 1 -- 172.21.15.100:0/4206603269 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f324c005c40 con 0x563e002f2df0 2024-05-10T11:24:30.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.481+0000 7f323ffff700 1 -- 172.21.15.100:0/4206603269 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f324c005030 con 0x563e002f2df0 2024-05-10T11:24:30.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.481+0000 7f3271ac7b80 1 -- 172.21.15.100:0/4206603269 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3254003b10 con 0x563e002f2df0 2024-05-10T11:24:30.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.481+0000 7f323ffff700 1 -- 172.21.15.100:0/4206603269 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f324c037050 con 0x563e002f2df0 2024-05-10T11:24:30.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.481+0000 7f3271ac7b80 10 client.26036 fetch_fsmap learned FSMap version 31 2024-05-10T11:24:30.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.481+0000 7f3271ac7b80 1 -- 172.21.15.100:0/4206603269 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563e00375be0 con 0x563e002f2df0 2024-05-10T11:24:30.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.481+0000 7f323ffff700 1 -- 172.21.15.100:0/4206603269 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3254003b10 con 0x563e002f2df0 2024-05-10T11:24:30.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.481+0000 7f323ffff700 1 -- 172.21.15.100:0/4206603269 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f324c010b40 con 0x563e002f2df0 2024-05-10T11:24:30.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.481+0000 7f3271ac7b80 10 client.26036 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:24:30.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.481+0000 7f3271ac7b80 10 client.26036 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-10T11:24:30.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.481+0000 7f3271ac7b80 20 client.26036 populate_metadata read hostname 'smithi100' 2024-05-10T11:24:30.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.481+0000 7f3271ac7b80 1 --2- 172.21.15.100:0/4206603269 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x563e00376280 0x563e003786a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:30.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.481+0000 7f3271ac7b80 4 client.26036 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:24:30.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.481+0000 7f3271ac7b80 1 -- 172.21.15.100:0/4206603269 --> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563e00252240 con 0x563e00376280 2024-05-10T11:24:30.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.482+0000 7f3264fd0700 1 --2- 172.21.15.100:0/4206603269 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x563e00376280 0x563e003786a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:30.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.482+0000 7f3264fd0700 1 --2- 172.21.15.100:0/4206603269 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x563e00376280 0x563e003786a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=328e5bad7611c4d server_cookie=60bd06f431348a01 in_seq=0 out_seq=0 2024-05-10T11:24:30.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.482+0000 7f323ffff700 10 client.26036 ms_handle_connect on v2:172.21.15.188:6836/1943342620 2024-05-10T11:24:30.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.482+0000 7f323ffff700 1 -- 172.21.15.100:0/4206603269 <== mds.2 v2:172.21.15.188:6836/1943342620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563e00252240 con 0x563e00376280 2024-05-10T11:24:30.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.482+0000 7f323ffff700 10 client.26036 handle_command_reply: tid=0 2024-05-10T11:24:30.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.482+0000 7f3271ac7b80 1 client.26036 shutdown 2024-05-10T11:24:30.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.482+0000 7f3271ac7b80 1 -- 172.21.15.100:0/4206603269 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563e002f2df0 msgr2=0x563e003016c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:30.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.482+0000 7f3271ac7b80 1 --2- 172.21.15.100:0/4206603269 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563e002f2df0 0x563e003016c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x563e002ee680 tx=0x7f324c011b00 comp rx=0 tx=0).stop 2024-05-10T11:24:30.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.482+0000 7f3271ac7b80 1 -- 172.21.15.100:0/4206603269 shutdown_connections 2024-05-10T11:24:30.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.482+0000 7f3271ac7b80 1 --2- 172.21.15.100:0/4206603269 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x563e00376280 0x563e003786a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:30.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.482+0000 7f3271ac7b80 1 --2- 172.21.15.100:0/4206603269 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e002f6c80 0x563e00301c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:30.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.482+0000 7f3271ac7b80 1 --2- 172.21.15.100:0/4206603269 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e002f24d0 0x563e00301180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:30.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.482+0000 7f3271ac7b80 1 --2- 172.21.15.100:0/4206603269 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563e002f2df0 0x563e003016c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:30.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.482+0000 7f323ffff700 0 client.26036 ms_handle_reset on v2:172.21.15.188:6836/1943342620 2024-05-10T11:24:30.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.482+0000 7f3271ac7b80 1 -- 172.21.15.100:0/4206603269 >> 172.21.15.100:0/4206603269 conn(0x563e0021e100 msgr2=0x563e002f12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:24:30.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.483+0000 7f3271ac7b80 1 -- 172.21.15.100:0/4206603269 shutdown_connections 2024-05-10T11:24:30.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.483+0000 7f3271ac7b80 1 -- 172.21.15.100:0/4206603269 wait complete. 2024-05-10T11:24:30.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.483+0000 7f3271ac7b80 20 client.26036 trim_cache size 0 max 16384 2024-05-10T11:24:30.489 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:24:30.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.483+0000 7f3267234700 1 -- 172.21.15.100:0/3840197617 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f32500b2880 msgr2=0x7f32500b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:30.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.483+0000 7f3267234700 1 --2- 172.21.15.100:0/3840197617 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f32500b2880 0x7f32500b4d40 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3254009c60 tx=0x7f3254009260 comp rx=0 tx=0).stop 2024-05-10T11:24:30.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.483+0000 7f3267234700 1 -- 172.21.15.100:0/3840197617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3260107760 msgr2=0x7f32600fe900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:30.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.483+0000 7f3267234700 1 --2- 172.21.15.100:0/3840197617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3260107760 0x7f32600fe900 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f3258009850 tx=0x7f3258007a20 comp rx=0 tx=0).stop 2024-05-10T11:24:30.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.483+0000 7f3267234700 1 -- 172.21.15.100:0/3840197617 shutdown_connections 2024-05-10T11:24:30.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.483+0000 7f3267234700 1 --2- 172.21.15.100:0/3840197617 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f32500b2880 0x7f32500b4d40 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:30.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.483+0000 7f3267234700 1 --2- 172.21.15.100:0/3840197617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3260107760 0x7f32600fe900 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:30.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.483+0000 7f3267234700 1 --2- 172.21.15.100:0/3840197617 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3260102e00 0x7f32600fde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:30.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.483+0000 7f3267234700 1 --2- 172.21.15.100:0/3840197617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32601037b0 0x7f32600fe3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:30.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.483+0000 7f3267234700 1 -- 172.21.15.100:0/3840197617 >> 172.21.15.100:0/3840197617 conn(0x7f3260078a10 msgr2=0x7f3260101d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:24:30.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.484+0000 7f3267234700 1 -- 172.21.15.100:0/3840197617 shutdown_connections 2024-05-10T11:24:30.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:30.484+0000 7f3267234700 1 -- 172.21.15.100:0/3840197617 wait complete. 2024-05-10T11:24:30.631 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.630+0000 7fda45c93640 1 -- 172.21.15.100:0/2840857415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda40105e20 msgr2=0x7fda40108210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:30.631 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.630+0000 7fda45c93640 1 --2- 172.21.15.100:0/2840857415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda40105e20 0x7fda40108210 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fda34009900 tx=0x7fda3402a410 comp rx=0 tx=0).stop 2024-05-10T11:24:30.631 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.631+0000 7fda45c93640 1 -- 172.21.15.100:0/2840857415 shutdown_connections 2024-05-10T11:24:30.631 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.631+0000 7fda45c93640 1 --2- 172.21.15.100:0/2840857415 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda40108750 0x7fda4010ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:30.631 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.631+0000 7fda45c93640 1 --2- 172.21.15.100:0/2840857415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda40105e20 0x7fda40108210 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:30.631 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.631+0000 7fda45c93640 1 --2- 172.21.15.100:0/2840857415 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda401034f0 0x7fda401058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:30.631 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.631+0000 7fda45c93640 1 -- 172.21.15.100:0/2840857415 >> 172.21.15.100:0/2840857415 conn(0x7fda400fce80 msgr2=0x7fda400ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:24:30.631 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.631+0000 7fda45c93640 1 -- 172.21.15.100:0/2840857415 shutdown_connections 2024-05-10T11:24:30.631 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.631+0000 7fda45c93640 1 -- 172.21.15.100:0/2840857415 wait complete. 2024-05-10T11:24:30.633 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.631+0000 7fda45c93640 1 Processor -- start 2024-05-10T11:24:30.633 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.632+0000 7fda45c93640 1 -- start start 2024-05-10T11:24:30.633 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.632+0000 7fda45c93640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda401034f0 0x7fda40198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:30.634 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.632+0000 7fda45c93640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda40105e20 0x7fda40199000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:30.634 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.632+0000 7fda45c93640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda40108750 0x7fda401a0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:30.634 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.632+0000 7fda45c93640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fda4010e3c0 con 0x7fda40108750 2024-05-10T11:24:30.634 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.632+0000 7fda45c93640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fda4010e240 con 0x7fda40105e20 2024-05-10T11:24:30.634 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.632+0000 7fda45c93640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fda4010e540 con 0x7fda401034f0 2024-05-10T11:24:30.634 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.632+0000 7fda3ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda40108750 0x7fda401a0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:30.634 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.632+0000 7fda3ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda40108750 0x7fda401a0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58030/0 (socket says 172.21.15.100:58030) 2024-05-10T11:24:30.634 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.632+0000 7fda3ffff640 1 -- 172.21.15.100:0/3946199388 learned_addr learned my addr 172.21.15.100:0/3946199388 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:24:30.634 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.632+0000 7fda3effd640 1 --2- 172.21.15.100:0/3946199388 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda40105e20 0x7fda40199000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:30.634 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.632+0000 7fda3f7fe640 1 --2- 172.21.15.100:0/3946199388 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda401034f0 0x7fda40198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:30.634 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.632+0000 7fda3ffff640 1 -- 172.21.15.100:0/3946199388 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda401034f0 msgr2=0x7fda40198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:30.634 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.632+0000 7fda3ffff640 1 --2- 172.21.15.100:0/3946199388 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda401034f0 0x7fda40198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:30.634 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.632+0000 7fda3ffff640 1 -- 172.21.15.100:0/3946199388 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda40105e20 msgr2=0x7fda40199000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:30.634 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.632+0000 7fda3ffff640 1 --2- 172.21.15.100:0/3946199388 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda40105e20 0x7fda40199000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.632+0000 7fda3ffff640 1 -- 172.21.15.100:0/3946199388 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda401a06e0 con 0x7fda40108750 2024-05-10T11:24:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.632+0000 7fda3ffff640 1 --2- 172.21.15.100:0/3946199388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda40108750 0x7fda401a0070 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fda3800eb10 tx=0x7fda3800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:24:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.633+0000 7fda3cff9640 1 -- 172.21.15.100:0/3946199388 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda38004b20 con 0x7fda40108750 2024-05-10T11:24:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.633+0000 7fda3cff9640 1 -- 172.21.15.100:0/3946199388 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fda38004cc0 con 0x7fda40108750 2024-05-10T11:24:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.633+0000 7fda45c93640 1 -- 172.21.15.100:0/3946199388 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda401a09d0 con 0x7fda40108750 2024-05-10T11:24:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.633+0000 7fda3cff9640 1 -- 172.21.15.100:0/3946199388 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda38010690 con 0x7fda40108750 2024-05-10T11:24:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.633+0000 7fda45c93640 1 -- 172.21.15.100:0/3946199388 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda401a0f40 con 0x7fda40108750 2024-05-10T11:24:30.638 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.635+0000 7fda489b2740 1 Processor -- start 2024-05-10T11:24:30.638 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.635+0000 7fda3cff9640 1 -- 172.21.15.100:0/3946199388 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fda38010830 con 0x7fda40108750 2024-05-10T11:24:30.638 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.635+0000 7fda489b2740 1 -- start start 2024-05-10T11:24:30.638 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.636+0000 7fda489b2740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f877c0cdb0 0x55f877c0d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:30.638 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.636+0000 7fda489b2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f877c0d7a0 0x55f877c122a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:30.638 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.636+0000 7fda489b2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f877c12910 0x55f877c14da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:30.638 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.636+0000 7fda489b2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55f877b715d0 con 0x55f877c12910 2024-05-10T11:24:30.638 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.636+0000 7fda489b2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f877b2a890 con 0x55f877c0d7a0 2024-05-10T11:24:30.638 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.636+0000 7fda489b2740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55f877b6d590 con 0x55f877c0cdb0 2024-05-10T11:24:30.638 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.636+0000 7fda3effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f877c12910 0x55f877c14da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:30.638 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.636+0000 7fda3f7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f877c0d7a0 0x55f877c122a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:30.638 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.636+0000 7fda3effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f877c12910 0x55f877c14da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58034/0 (socket says 172.21.15.100:58034) 2024-05-10T11:24:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.636+0000 7fda3f7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f877c0d7a0 0x55f877c122a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:38618/0 (socket says 172.21.15.100:38618) 2024-05-10T11:24:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.636+0000 7fda3effd640 1 -- 172.21.15.100:0/1654314591 learned_addr learned my addr 172.21.15.100:0/1654314591 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:24:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.636+0000 7fda3effd640 1 --2- 172.21.15.100:0/1654314591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f877c0cdb0 0x55f877c0d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.636+0000 7fda3f7fe640 1 -- 172.21.15.100:0/1654314591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f877c0cdb0 msgr2=0x55f877c0d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.636+0000 7fda3f7fe640 1 --2- 172.21.15.100:0/1654314591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f877c0cdb0 0x55f877c0d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.636+0000 7fda3f7fe640 1 -- 172.21.15.100:0/1654314591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f877c12910 msgr2=0x55f877c14da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.636+0000 7fda3f7fe640 1 --2- 172.21.15.100:0/1654314591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f877c12910 0x55f877c14da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.636+0000 7fda3f7fe640 1 -- 172.21.15.100:0/1654314591 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f877883210 con 0x55f877c0d7a0 2024-05-10T11:24:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.636+0000 7fda3effd640 1 --2- 172.21.15.100:0/1654314591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f877c12910 0x55f877c14da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:24:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.636+0000 7fda3cff9640 1 --2- 172.21.15.100:0/3946199388 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fda280b2920 0x7fda280b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.636+0000 7fda3effd640 1 --2- 172.21.15.100:0/3946199388 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fda280b2920 0x7fda280b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.636+0000 7fda3f7fe640 1 --2- 172.21.15.100:0/1654314591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f877c0d7a0 0x55f877c122a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fda30009c20 tx=0x7fda30011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c40ade133acb4412 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:24:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.636+0000 7fda3cff9640 1 -- 172.21.15.100:0/3946199388 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fda380e2a40 con 0x7fda40108750 2024-05-10T11:24:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.637+0000 7fda25ffb640 1 -- 172.21.15.100:0/1654314591 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda30004050 con 0x55f877c0d7a0 2024-05-10T11:24:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.637+0000 7fda25ffb640 1 -- 172.21.15.100:0/1654314591 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fda30005020 con 0x55f877c0d7a0 2024-05-10T11:24:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.637+0000 7fda25ffb640 1 -- 172.21.15.100:0/1654314591 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda30004bc0 con 0x55f877c0d7a0 2024-05-10T11:24:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.638+0000 7fda489b2740 1 -- 172.21.15.100:0/1654314591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f877c0d7a0 msgr2=0x55f877c122a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.638+0000 7fda3effd640 1 --2- 172.21.15.100:0/3946199388 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fda280b2920 0x7fda280b4de0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fda34000c00 tx=0x7fda340046c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:24:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.638+0000 7fda489b2740 1 --2- 172.21.15.100:0/1654314591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f877c0d7a0 0x55f877c122a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fda30009c20 tx=0x7fda30011810 comp rx=0 tx=0).stop 2024-05-10T11:24:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.638+0000 7fda489b2740 1 -- 172.21.15.100:0/1654314591 shutdown_connections 2024-05-10T11:24:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.638+0000 7fda489b2740 1 --2- 172.21.15.100:0/1654314591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f877c12910 0x55f877c14da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.638+0000 7fda489b2740 1 --2- 172.21.15.100:0/1654314591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f877c0d7a0 0x55f877c122a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.638+0000 7fda489b2740 1 --2- 172.21.15.100:0/1654314591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f877c0cdb0 0x55f877c0d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.638+0000 7fda489b2740 1 -- 172.21.15.100:0/1654314591 >> 172.21.15.100:0/1654314591 conn(0x55f877c08420 msgr2=0x55f877c08820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:24:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.638+0000 7fda489b2740 1 -- 172.21.15.100:0/1654314591 shutdown_connections 2024-05-10T11:24:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.638+0000 7fda489b2740 1 -- 172.21.15.100:0/1654314591 wait complete. 2024-05-10T11:24:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.638+0000 7fda489b2740 1 -- start start 2024-05-10T11:24:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.639+0000 7fda489b2740 1 Processor -- start 2024-05-10T11:24:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.639+0000 7fda489b2740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f877c0cdb0 0x55f877c17110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.639+0000 7fda489b2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f877c0d7a0 0x55f877c17650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.639+0000 7fda489b2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f877c12910 0x55f877c1e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.639+0000 7fda489b2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55f877c158c0 con 0x55f877c12910 2024-05-10T11:24:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.639+0000 7fda489b2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f877b2e370 con 0x55f877c0d7a0 2024-05-10T11:24:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.639+0000 7fda489b2740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55f877c15a40 con 0x55f877c0cdb0 2024-05-10T11:24:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.639+0000 7fda489b2740 10 client.0 _ll_register_callbacks cb 0x55f877882a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:24:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.639+0000 7fda3ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f877c12910 0x55f877c1e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.639+0000 7fda3f7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f877c0cdb0 0x55f877c17110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.639+0000 7fda3effd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f877c0d7a0 0x55f877c17650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.639+0000 7fda3effd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f877c0d7a0 0x55f877c17650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:38630/0 (socket says 172.21.15.100:38630) 2024-05-10T11:24:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.639+0000 7fda3f7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f877c0cdb0 0x55f877c17110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:39808/0 (socket says 172.21.15.100:39808) 2024-05-10T11:24:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.639+0000 7fda3effd640 1 -- 172.21.15.100:0/2074523125 learned_addr learned my addr 172.21.15.100:0/2074523125 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:24:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.639+0000 7fda3effd640 1 -- 172.21.15.100:0/2074523125 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f877c0cdb0 msgr2=0x55f877c17110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.639+0000 7fda3effd640 1 --2- 172.21.15.100:0/2074523125 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f877c0cdb0 0x55f877c17110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.639+0000 7fda3effd640 1 -- 172.21.15.100:0/2074523125 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f877c12910 msgr2=0x55f877c1e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.639+0000 7fda3effd640 1 --2- 172.21.15.100:0/2074523125 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f877c12910 0x55f877c1e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.639+0000 7fda3effd640 1 -- 172.21.15.100:0/2074523125 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f87794f620 con 0x55f877c0d7a0 2024-05-10T11:24:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.639+0000 7fda3f7fe640 1 --2- 172.21.15.100:0/2074523125 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f877c0cdb0 0x55f877c17110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:24:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.640+0000 7fda3effd640 1 --2- 172.21.15.100:0/2074523125 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f877c0d7a0 0x55f877c17650 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fda34009230 tx=0x7fda3402a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=feaf59a55996024 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:24:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.640+0000 7fda1effd640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:24:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.640+0000 7fda1effd640 1 -- 172.21.15.100:0/2074523125 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda34035040 con 0x55f877c0d7a0 2024-05-10T11:24:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.640+0000 7fda1effd640 1 -- 172.21.15.100:0/2074523125 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fda340035f0 con 0x55f877c0d7a0 2024-05-10T11:24:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.640+0000 7fda489b2740 1 -- 172.21.15.100:0/2074523125 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda30005020 con 0x55f877c0d7a0 2024-05-10T11:24:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.640+0000 7fda1effd640 1 -- 172.21.15.100:0/2074523125 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fda34004490 con 0x55f877c0d7a0 2024-05-10T11:24:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.640+0000 7fda489b2740 10 client.26048 fetch_fsmap learned FSMap version 31 2024-05-10T11:24:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.640+0000 7fda489b2740 1 -- 172.21.15.100:0/2074523125 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55f877c216e0 con 0x55f877c0d7a0 2024-05-10T11:24:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.640+0000 7fda1effd640 1 -- 172.21.15.100:0/2074523125 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fda30005020 con 0x55f877c0d7a0 2024-05-10T11:24:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.640+0000 7fda1effd640 1 -- 172.21.15.100:0/2074523125 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fda3403d940 con 0x55f877c0d7a0 2024-05-10T11:24:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.641+0000 7fda489b2740 10 client.26048 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:24:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.641+0000 7fda489b2740 10 client.26048 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:24:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.641+0000 7fda489b2740 20 client.26048 populate_metadata read hostname 'smithi100' 2024-05-10T11:24:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.641+0000 7fda489b2740 1 --2- 172.21.15.100:0/2074523125 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55f877c22d30 0x55f877c251c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.641+0000 7fda489b2740 4 client.26048 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:24:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.641+0000 7fda489b2740 1 -- 172.21.15.100:0/2074523125 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f877b7f1e0 con 0x55f877c22d30 2024-05-10T11:24:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.641+0000 7fda3f7fe640 1 --2- 172.21.15.100:0/2074523125 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55f877c22d30 0x55f877c251c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.641+0000 7fda1effd640 10 client.26048 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:24:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.641+0000 7fda3f7fe640 1 --2- 172.21.15.100:0/2074523125 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55f877c22d30 0x55f877c251c0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a91ad2e79bb02811 server_cookie=86b141cd23eb41f6 in_seq=0 out_seq=0 2024-05-10T11:24:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.642+0000 7fda1effd640 1 -- 172.21.15.100:0/2074523125 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f877b7f1e0 con 0x55f877c22d30 2024-05-10T11:24:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.642+0000 7fda1effd640 10 client.26048 handle_command_reply: tid=0 2024-05-10T11:24:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.642+0000 7fda489b2740 1 client.26048 shutdown 2024-05-10T11:24:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.642+0000 7fda489b2740 1 -- 172.21.15.100:0/2074523125 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f877c0d7a0 msgr2=0x55f877c17650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.642+0000 7fda489b2740 1 --2- 172.21.15.100:0/2074523125 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f877c0d7a0 0x55f877c17650 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fda34009230 tx=0x7fda3402a410 comp rx=0 tx=0).stop 2024-05-10T11:24:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.643+0000 7fda489b2740 1 -- 172.21.15.100:0/2074523125 shutdown_connections 2024-05-10T11:24:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.643+0000 7fda489b2740 1 --2- 172.21.15.100:0/2074523125 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55f877c22d30 0x55f877c251c0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.643+0000 7fda489b2740 1 --2- 172.21.15.100:0/2074523125 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f877c12910 0x55f877c1e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.643+0000 7fda489b2740 1 --2- 172.21.15.100:0/2074523125 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f877c0d7a0 0x55f877c17650 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.643+0000 7fda489b2740 1 --2- 172.21.15.100:0/2074523125 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f877c0cdb0 0x55f877c17110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.643+0000 7fda1effd640 0 client.26048 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:24:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.643+0000 7fda489b2740 1 -- 172.21.15.100:0/2074523125 >> 172.21.15.100:0/2074523125 conn(0x55f877c08420 msgr2=0x55f877c07580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:24:30.648 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.643+0000 7fda489b2740 1 -- 172.21.15.100:0/2074523125 shutdown_connections 2024-05-10T11:24:30.648 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.643+0000 7fda489b2740 1 -- 172.21.15.100:0/2074523125 wait complete. 2024-05-10T11:24:30.648 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.643+0000 7fda489b2740 20 client.26048 trim_cache size 0 max 16384 2024-05-10T11:24:30.663 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.661+0000 7fda489b2740 1 Processor -- start 2024-05-10T11:24:30.663 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.662+0000 7fda489b2740 1 -- start start 2024-05-10T11:24:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.662+0000 7fda489b2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f877c0cdb0 0x55f877c0b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.662+0000 7fda489b2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f877c0d7a0 0x55f877c0b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.662+0000 7fda489b2740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f877c12910 0x55f877c29ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.662+0000 7fda489b2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fda401afd80 con 0x55f877c0d7a0 2024-05-10T11:24:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.662+0000 7fda489b2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fda401ad650 con 0x55f877c0cdb0 2024-05-10T11:24:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.662+0000 7fda489b2740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fda401ab4a0 con 0x55f877c12910 2024-05-10T11:24:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.662+0000 7fda3effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f877c0d7a0 0x55f877c0b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.662+0000 7fda3effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f877c0d7a0 0x55f877c0b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58060/0 (socket says 172.21.15.100:58060) 2024-05-10T11:24:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.662+0000 7fda3ffff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f877c12910 0x55f877c29ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.662+0000 7fda3effd640 1 -- 172.21.15.100:0/1526042627 learned_addr learned my addr 172.21.15.100:0/1526042627 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:24:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.662+0000 7fda3f7fe640 1 --2- 172.21.15.100:0/1526042627 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f877c0cdb0 0x55f877c0b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.662+0000 7fda3effd640 1 -- 172.21.15.100:0/1526042627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f877c12910 msgr2=0x55f877c29ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.662+0000 7fda3effd640 1 --2- 172.21.15.100:0/1526042627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f877c12910 0x55f877c29ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.662+0000 7fda3effd640 1 -- 172.21.15.100:0/1526042627 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f877c0cdb0 msgr2=0x55f877c0b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.662+0000 7fda3effd640 1 --2- 172.21.15.100:0/1526042627 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f877c0cdb0 0x55f877c0b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.662+0000 7fda3effd640 1 -- 172.21.15.100:0/1526042627 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f877c0d190 con 0x55f877c0d7a0 2024-05-10T11:24:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.662+0000 7fda3ffff640 1 --2- 172.21.15.100:0/1526042627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f877c12910 0x55f877c29ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:24:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.662+0000 7fda3effd640 1 --2- 172.21.15.100:0/1526042627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f877c0d7a0 0x55f877c0b870 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fda34004650 tx=0x7fda34009830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de377b60a02fc178 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:24:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.663+0000 7fda257fa640 1 -- 172.21.15.100:0/1526042627 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda34035070 con 0x55f877c0d7a0 2024-05-10T11:24:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.663+0000 7fda257fa640 1 -- 172.21.15.100:0/1526042627 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fda3404f070 con 0x55f877c0d7a0 2024-05-10T11:24:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.663+0000 7fda257fa640 1 -- 172.21.15.100:0/1526042627 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda34003b50 con 0x55f877c0d7a0 2024-05-10T11:24:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.663+0000 7fda489b2740 1 -- 172.21.15.100:0/1526042627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f877c0d7a0 msgr2=0x55f877c0b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.663+0000 7fda489b2740 1 --2- 172.21.15.100:0/1526042627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f877c0d7a0 0x55f877c0b870 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fda34004650 tx=0x7fda34009830 comp rx=0 tx=0).stop 2024-05-10T11:24:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.663+0000 7fda489b2740 1 -- 172.21.15.100:0/1526042627 shutdown_connections 2024-05-10T11:24:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.663+0000 7fda489b2740 1 --2- 172.21.15.100:0/1526042627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f877c12910 0x55f877c29ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.663+0000 7fda489b2740 1 --2- 172.21.15.100:0/1526042627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f877c0d7a0 0x55f877c0b870 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.663+0000 7fda489b2740 1 --2- 172.21.15.100:0/1526042627 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f877c0cdb0 0x55f877c0b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.663+0000 7fda489b2740 1 -- 172.21.15.100:0/1526042627 >> 172.21.15.100:0/1526042627 conn(0x55f877c08420 msgr2=0x55f877c0eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:24:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.663+0000 7fda489b2740 1 -- 172.21.15.100:0/1526042627 shutdown_connections 2024-05-10T11:24:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.663+0000 7fda489b2740 1 -- 172.21.15.100:0/1526042627 wait complete. 2024-05-10T11:24:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.664+0000 7fda489b2740 1 -- start start 2024-05-10T11:24:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.664+0000 7fda489b2740 1 Processor -- start 2024-05-10T11:24:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.664+0000 7fda489b2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f877c0cdb0 0x55f877c1b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.664+0000 7fda489b2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f877c0d7a0 0x55f877c1bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.664+0000 7fda489b2740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f877c12910 0x55f877c1cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.664+0000 7fda489b2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fda401abb10 con 0x55f877c0d7a0 2024-05-10T11:24:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.664+0000 7fda489b2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fda401acf80 con 0x55f877c0cdb0 2024-05-10T11:24:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.664+0000 7fda489b2740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fda401ac1e0 con 0x55f877c12910 2024-05-10T11:24:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.664+0000 7fda489b2740 10 client.0 _ll_register_callbacks cb 0x7fda34032b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:24:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.664+0000 7fda3f7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f877c0cdb0 0x55f877c1b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:30.669 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.664+0000 7fda3f7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f877c0cdb0 0x55f877c1b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:38648/0 (socket says 172.21.15.100:38648) 2024-05-10T11:24:30.669 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.664+0000 7fda3f7fe640 1 -- 172.21.15.100:0/2266747550 learned_addr learned my addr 172.21.15.100:0/2266747550 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:24:30.669 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.664+0000 7fda3effd640 1 --2- 172.21.15.100:0/2266747550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f877c0d7a0 0x55f877c1bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:30.669 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.665+0000 7fda3ffff640 1 --2- 172.21.15.100:0/2266747550 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f877c12910 0x55f877c1cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:30.669 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.665+0000 7fda3f7fe640 1 -- 172.21.15.100:0/2266747550 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f877c12910 msgr2=0x55f877c1cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:30.669 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.665+0000 7fda3f7fe640 1 --2- 172.21.15.100:0/2266747550 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f877c12910 0x55f877c1cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:30.669 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.665+0000 7fda3f7fe640 1 -- 172.21.15.100:0/2266747550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f877c0d7a0 msgr2=0x55f877c1bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:30.669 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.665+0000 7fda3f7fe640 1 --2- 172.21.15.100:0/2266747550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f877c0d7a0 0x55f877c1bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:30.669 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.665+0000 7fda3f7fe640 1 -- 172.21.15.100:0/2266747550 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f877c1d520 con 0x55f877c0cdb0 2024-05-10T11:24:30.669 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.665+0000 7fda3f7fe640 1 --2- 172.21.15.100:0/2266747550 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f877c0cdb0 0x55f877c1b9e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fda30009fd0 tx=0x7fda30005c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc92946da456527b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:24:30.669 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.665+0000 7fda1effd640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:24:30.669 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.665+0000 7fda1effd640 1 -- 172.21.15.100:0/2266747550 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda3002e410 con 0x55f877c0cdb0 2024-05-10T11:24:30.669 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.665+0000 7fda1effd640 1 -- 172.21.15.100:0/2266747550 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fda3002e5b0 con 0x55f877c0cdb0 2024-05-10T11:24:30.669 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.665+0000 7fda489b2740 1 -- 172.21.15.100:0/2266747550 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda3404f070 con 0x55f877c0cdb0 2024-05-10T11:24:30.669 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.665+0000 7fda1effd640 1 -- 172.21.15.100:0/2266747550 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fda30037050 con 0x55f877c0cdb0 2024-05-10T11:24:30.670 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.665+0000 7fda489b2740 10 client.26054 fetch_fsmap learned FSMap version 31 2024-05-10T11:24:30.670 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.665+0000 7fda489b2740 1 -- 172.21.15.100:0/2266747550 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55f877c8dc70 con 0x55f877c0cdb0 2024-05-10T11:24:30.670 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.666+0000 7fda1effd640 1 -- 172.21.15.100:0/2266747550 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fda3404f070 con 0x55f877c0cdb0 2024-05-10T11:24:30.670 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.666+0000 7fda1effd640 1 -- 172.21.15.100:0/2266747550 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fda30003c20 con 0x55f877c0cdb0 2024-05-10T11:24:30.670 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.666+0000 7fda489b2740 10 client.26054 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:24:30.670 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.666+0000 7fda489b2740 10 client.26054 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:24:30.670 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.666+0000 7fda489b2740 20 client.26054 populate_metadata read hostname 'smithi100' 2024-05-10T11:24:30.670 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.666+0000 7fda489b2740 1 --2- 172.21.15.100:0/2266747550 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55f877c8e3f0 0x55f877c90810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:30.670 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.666+0000 7fda489b2740 4 client.26054 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:24:30.670 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.666+0000 7fda489b2740 1 -- 172.21.15.100:0/2266747550 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f877b79050 con 0x55f877c8e3f0 2024-05-10T11:24:30.670 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.666+0000 7fda3effd640 1 --2- 172.21.15.100:0/2266747550 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55f877c8e3f0 0x55f877c90810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:30.670 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.667+0000 7fda3effd640 1 --2- 172.21.15.100:0/2266747550 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55f877c8e3f0 0x55f877c90810 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7794d135b84afa6 server_cookie=664a98653d8d6d5d in_seq=0 out_seq=0 2024-05-10T11:24:30.670 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.667+0000 7fda1effd640 10 client.26054 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:24:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.688+0000 7fda1effd640 1 -- 172.21.15.100:0/2266747550 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f877b79050 con 0x55f877c8e3f0 2024-05-10T11:24:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.688+0000 7fda1effd640 10 client.26054 handle_command_reply: tid=0 2024-05-10T11:24:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.688+0000 7fda489b2740 1 client.26054 shutdown 2024-05-10T11:24:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.688+0000 7fda489b2740 1 -- 172.21.15.100:0/2266747550 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f877c0cdb0 msgr2=0x55f877c1b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.688+0000 7fda489b2740 1 --2- 172.21.15.100:0/2266747550 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f877c0cdb0 0x55f877c1b9e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fda30009fd0 tx=0x7fda30005c90 comp rx=0 tx=0).stop 2024-05-10T11:24:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.688+0000 7fda489b2740 1 -- 172.21.15.100:0/2266747550 shutdown_connections 2024-05-10T11:24:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.688+0000 7fda489b2740 1 --2- 172.21.15.100:0/2266747550 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55f877c8e3f0 0x55f877c90810 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.688+0000 7fda489b2740 1 --2- 172.21.15.100:0/2266747550 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f877c12910 0x55f877c1cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.688+0000 7fda489b2740 1 --2- 172.21.15.100:0/2266747550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f877c0d7a0 0x55f877c1bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.688+0000 7fda489b2740 1 --2- 172.21.15.100:0/2266747550 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f877c0cdb0 0x55f877c1b9e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.688+0000 7fda1effd640 0 client.26054 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:24:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.688+0000 7fda489b2740 1 -- 172.21.15.100:0/2266747550 >> 172.21.15.100:0/2266747550 conn(0x55f877c08420 msgr2=0x55f877c170f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:24:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.688+0000 7fda489b2740 1 -- 172.21.15.100:0/2266747550 shutdown_connections 2024-05-10T11:24:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.688+0000 7fda489b2740 1 -- 172.21.15.100:0/2266747550 wait complete. 2024-05-10T11:24:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.688+0000 7fda489b2740 20 client.26054 trim_cache size 0 max 16384 2024-05-10T11:24:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:24:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:24:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:24:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:24:30.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.688+0000 7fda45c93640 1 -- 172.21.15.100:0/3946199388 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fda280b2920 msgr2=0x7fda280b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:30.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.688+0000 7fda45c93640 1 --2- 172.21.15.100:0/3946199388 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fda280b2920 0x7fda280b4de0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fda34000c00 tx=0x7fda340046c0 comp rx=0 tx=0).stop 2024-05-10T11:24:30.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.689+0000 7fda45c93640 1 -- 172.21.15.100:0/3946199388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda40108750 msgr2=0x7fda401a0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:30.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.689+0000 7fda45c93640 1 --2- 172.21.15.100:0/3946199388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda40108750 0x7fda401a0070 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fda3800eb10 tx=0x7fda3800c2d0 comp rx=0 tx=0).stop 2024-05-10T11:24:30.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.689+0000 7fda45c93640 1 -- 172.21.15.100:0/3946199388 shutdown_connections 2024-05-10T11:24:30.691 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.689+0000 7fda45c93640 1 --2- 172.21.15.100:0/3946199388 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fda280b2920 0x7fda280b4de0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:30.691 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.689+0000 7fda45c93640 1 --2- 172.21.15.100:0/3946199388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda40108750 0x7fda401a0070 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:30.691 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.689+0000 7fda45c93640 1 --2- 172.21.15.100:0/3946199388 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda40105e20 0x7fda40199000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:30.691 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.689+0000 7fda45c93640 1 --2- 172.21.15.100:0/3946199388 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda401034f0 0x7fda40198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:30.691 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.689+0000 7fda45c93640 1 -- 172.21.15.100:0/3946199388 >> 172.21.15.100:0/3946199388 conn(0x7fda400fce80 msgr2=0x7fda40104130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:24:30.691 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.689+0000 7fda45c93640 1 -- 172.21.15.100:0/3946199388 shutdown_connections 2024-05-10T11:24:30.691 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:30.689+0000 7fda45c93640 1 -- 172.21.15.100:0/3946199388 wait complete. 2024-05-10T11:24:30.972 INFO:tasks.workunit.client.0.smithi100.stdout: 1 138577 2.77 MB/sec execute 145 sec latency 30.272 ms 2024-05-10T11:24:31.283 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-10T11:24:31.537 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:24:31.972 INFO:tasks.workunit.client.0.smithi100.stdout: 1 138837 2.76 MB/sec execute 146 sec latency 12.085 ms 2024-05-10T11:24:32.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:24:31 smithi178 ceph-mon[33359]: pgmap v604: 129 pgs: 129 active+clean; 65 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 1.3 MiB/s wr, 148 op/s 2024-05-10T11:24:32.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:24:31 smithi188 ceph-mon[34075]: pgmap v604: 129 pgs: 129 active+clean; 65 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 1.3 MiB/s wr, 148 op/s 2024-05-10T11:24:32.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:24:31 smithi100 ceph-mon[30296]: pgmap v604: 129 pgs: 129 active+clean; 65 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 1.3 MiB/s wr, 148 op/s 2024-05-10T11:24:32.972 INFO:tasks.workunit.client.0.smithi100.stdout: 1 139282 2.74 MB/sec execute 147 sec latency 14.957 ms 2024-05-10T11:24:33.972 INFO:tasks.workunit.client.0.smithi100.stdout: 1 139769 2.75 MB/sec execute 148 sec latency 61.999 ms 2024-05-10T11:24:34.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:24:33 smithi178 ceph-mon[33359]: pgmap v605: 129 pgs: 129 active+clean; 64 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 1.3 MiB/s wr, 142 op/s 2024-05-10T11:24:34.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:24:33 smithi188 ceph-mon[34075]: pgmap v605: 129 pgs: 129 active+clean; 64 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 1.3 MiB/s wr, 142 op/s 2024-05-10T11:24:34.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:24:33 smithi100 ceph-mon[30296]: pgmap v605: 129 pgs: 129 active+clean; 64 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 1.3 MiB/s wr, 142 op/s 2024-05-10T11:24:34.972 INFO:tasks.workunit.client.0.smithi100.stdout: 1 140335 2.74 MB/sec execute 149 sec latency 71.764 ms 2024-05-10T11:24:35.972 INFO:tasks.workunit.client.0.smithi100.stdout: 1 140775 2.73 MB/sec execute 150 sec latency 209.072 ms 2024-05-10T11:24:36.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:24:35 smithi100 ceph-mon[30296]: pgmap v606: 129 pgs: 129 active+clean; 66 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 1.5 MiB/s wr, 175 op/s 2024-05-10T11:24:36.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:24:35 smithi178 ceph-mon[33359]: pgmap v606: 129 pgs: 129 active+clean; 66 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 1.5 MiB/s wr, 175 op/s 2024-05-10T11:24:36.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:24:35 smithi188 ceph-mon[34075]: pgmap v606: 129 pgs: 129 active+clean; 66 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 1.5 MiB/s wr, 175 op/s 2024-05-10T11:24:36.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.725+0000 7f43d5957700 1 -- 172.21.15.100:0/2702991784 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43d0101a90 msgr2=0x7f43d0101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:36.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.726+0000 7f43d5957700 1 --2- 172.21.15.100:0/2702991784 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43d0101a90 0x7f43d0101f10 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f43c4009990 tx=0x7f43c4009ca0 comp rx=0 tx=0).stop 2024-05-10T11:24:36.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.726+0000 7f43d5957700 1 -- 172.21.15.100:0/2702991784 shutdown_connections 2024-05-10T11:24:36.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.726+0000 7f43d5957700 1 --2- 172.21.15.100:0/2702991784 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43d0102450 0x7f43d0110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:36.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.726+0000 7f43d5957700 1 --2- 172.21.15.100:0/2702991784 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43d0109520 0x7f43d0109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:36.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.726+0000 7f43d5957700 1 --2- 172.21.15.100:0/2702991784 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43d0101a90 0x7f43d0101f10 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:36.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.726+0000 7f43d5957700 1 -- 172.21.15.100:0/2702991784 >> 172.21.15.100:0/2702991784 conn(0x7f43d0078a10 msgr2=0x7f43d00ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:24:36.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.726+0000 7f43d5957700 1 -- 172.21.15.100:0/2702991784 shutdown_connections 2024-05-10T11:24:36.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.726+0000 7f43d5957700 1 -- 172.21.15.100:0/2702991784 wait complete. 2024-05-10T11:24:36.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.726+0000 7f43d5957700 1 Processor -- start 2024-05-10T11:24:36.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.727+0000 7f43d5957700 1 -- start start 2024-05-10T11:24:36.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.727+0000 7f43d5957700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43d0101a90 0x7f43d0102c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:36.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.727+0000 7f43d5957700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43d0102450 0x7f43d01031b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:36.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.727+0000 7f43d5957700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43d0109520 0x7f43d01036f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:36.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.727+0000 7f43d5957700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f43d01134e0 con 0x7f43d0109520 2024-05-10T11:24:36.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.727+0000 7f43d5957700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f43d0113360 con 0x7f43d0102450 2024-05-10T11:24:36.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.727+0000 7f43d5957700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f43d0113660 con 0x7f43d0101a90 2024-05-10T11:24:36.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.727+0000 7f43ceffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43d0101a90 0x7f43d0102c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:36.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.727+0000 7f43cf7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43d0109520 0x7f43d01036f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:36.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.727+0000 7f43ceffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43d0101a90 0x7f43d0102c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:56086/0 (socket says 172.21.15.100:56086) 2024-05-10T11:24:36.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.727+0000 7f43ceffd700 1 -- 172.21.15.100:0/1134277488 learned_addr learned my addr 172.21.15.100:0/1134277488 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:24:36.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.727+0000 7f43ce7fc700 1 --2- 172.21.15.100:0/1134277488 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43d0102450 0x7f43d01031b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:36.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.727+0000 7f43ceffd700 1 -- 172.21.15.100:0/1134277488 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43d0102450 msgr2=0x7f43d01031b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:36.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.727+0000 7f43ceffd700 1 --2- 172.21.15.100:0/1134277488 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43d0102450 0x7f43d01031b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:36.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.727+0000 7f43ceffd700 1 -- 172.21.15.100:0/1134277488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43d0109520 msgr2=0x7f43d01036f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:36.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.727+0000 7f43ceffd700 1 --2- 172.21.15.100:0/1134277488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43d0109520 0x7f43d01036f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:36.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.727+0000 7f43ceffd700 1 -- 172.21.15.100:0/1134277488 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43d01a7fa0 con 0x7f43d0101a90 2024-05-10T11:24:36.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.727+0000 7f43cf7fe700 1 --2- 172.21.15.100:0/1134277488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43d0109520 0x7f43d01036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:24:36.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.728+0000 7f43ceffd700 1 --2- 172.21.15.100:0/1134277488 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43d0101a90 0x7f43d0102c70 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f43c000e9a0 tx=0x7f43c000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:24:36.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.728+0000 7f43d4955700 1 -- 172.21.15.100:0/1134277488 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c000cbd0 con 0x7f43d0101a90 2024-05-10T11:24:36.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.728+0000 7f43d4955700 1 -- 172.21.15.100:0/1134277488 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f43c000cd70 con 0x7f43d0101a90 2024-05-10T11:24:36.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.728+0000 7f43d5957700 1 -- 172.21.15.100:0/1134277488 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43d01a8290 con 0x7f43d0101a90 2024-05-10T11:24:36.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.728+0000 7f43d4955700 1 -- 172.21.15.100:0/1134277488 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c0018680 con 0x7f43d0101a90 2024-05-10T11:24:36.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.728+0000 7f43d5957700 1 -- 172.21.15.100:0/1134277488 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43d01a87d0 con 0x7f43d0101a90 2024-05-10T11:24:36.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.730+0000 7f43d4955700 1 -- 172.21.15.100:0/1134277488 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f43c0010430 con 0x7f43d0101a90 2024-05-10T11:24:36.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.731+0000 7f43d4955700 1 --2- 172.21.15.100:0/1134277488 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f43b80b2880 0x7f43b80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:36.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.732+0000 7f43ce7fc700 1 --2- 172.21.15.100:0/1134277488 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f43b80b2880 0x7f43b80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:36.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.732+0000 7f43d4955700 1 -- 172.21.15.100:0/1134277488 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f43c00e27a0 con 0x7f43d0101a90 2024-05-10T11:24:36.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.732+0000 7f43e01eab80 1 Processor -- start 2024-05-10T11:24:36.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.732+0000 7f43e01eab80 1 -- start start 2024-05-10T11:24:36.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.732+0000 7f43ce7fc700 1 --2- 172.21.15.100:0/1134277488 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f43b80b2880 0x7f43b80b4d40 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f43d00ffab0 tx=0x7f43c401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:24:36.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.732+0000 7f43e01eab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566a8c701b0 0x5566a8c70590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:36.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.732+0000 7f43e01eab80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566a8c70ad0 0x5566a8c6a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:36.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.732+0000 7f43e01eab80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5566a8c6ad50 0x5566a8c6b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:36.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.732+0000 7f43e01eab80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5566a8bbc940 con 0x5566a8c70ad0 2024-05-10T11:24:36.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.732+0000 7f43e01eab80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5566a8b21da0 con 0x5566a8c6ad50 2024-05-10T11:24:36.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.732+0000 7f43e01eab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5566a8bb8900 con 0x5566a8c701b0 2024-05-10T11:24:36.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.732+0000 7f43ce7fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566a8c70ad0 0x5566a8c6a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:36.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.732+0000 7f43ceffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566a8c701b0 0x5566a8c70590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:36.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.732+0000 7f43ce7fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566a8c70ad0 0x5566a8c6a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39918/0 (socket says 172.21.15.100:39918) 2024-05-10T11:24:36.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.732+0000 7f43ce7fc700 1 -- 172.21.15.100:0/2329443698 learned_addr learned my addr 172.21.15.100:0/2329443698 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:24:36.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.732+0000 7f43cf7fe700 1 --2- 172.21.15.100:0/2329443698 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5566a8c6ad50 0x5566a8c6b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:36.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.733+0000 7f43ce7fc700 1 -- 172.21.15.100:0/2329443698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566a8c701b0 msgr2=0x5566a8c70590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:36.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.733+0000 7f43ce7fc700 1 --2- 172.21.15.100:0/2329443698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566a8c701b0 0x5566a8c70590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:36.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.733+0000 7f43ce7fc700 1 -- 172.21.15.100:0/2329443698 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5566a8c6ad50 msgr2=0x5566a8c6b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:36.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.733+0000 7f43ce7fc700 1 --2- 172.21.15.100:0/2329443698 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5566a8c6ad50 0x5566a8c6b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:36.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.733+0000 7f43ce7fc700 1 -- 172.21.15.100:0/2329443698 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566a8b098d0 con 0x5566a8c70ad0 2024-05-10T11:24:36.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.733+0000 7f43ceffd700 1 --2- 172.21.15.100:0/2329443698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566a8c701b0 0x5566a8c70590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:24:36.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.733+0000 7f43ce7fc700 1 --2- 172.21.15.100:0/2329443698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566a8c70ad0 0x5566a8c6a810 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f43c4009990 tx=0x7f43c401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=985440072520c077 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:24:36.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.733+0000 7f43b5ffb700 1 -- 172.21.15.100:0/2329443698 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c400bda0 con 0x5566a8c70ad0 2024-05-10T11:24:36.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.733+0000 7f43b5ffb700 1 -- 172.21.15.100:0/2329443698 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f43c4023410 con 0x5566a8c70ad0 2024-05-10T11:24:36.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.733+0000 7f43b5ffb700 1 -- 172.21.15.100:0/2329443698 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c402d610 con 0x5566a8c70ad0 2024-05-10T11:24:36.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.734+0000 7f43e01eab80 1 -- 172.21.15.100:0/2329443698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566a8c70ad0 msgr2=0x5566a8c6a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:36.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.734+0000 7f43e01eab80 1 --2- 172.21.15.100:0/2329443698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566a8c70ad0 0x5566a8c6a810 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f43c4009990 tx=0x7f43c401e000 comp rx=0 tx=0).stop 2024-05-10T11:24:36.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.734+0000 7f43e01eab80 1 -- 172.21.15.100:0/2329443698 shutdown_connections 2024-05-10T11:24:36.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.734+0000 7f43e01eab80 1 --2- 172.21.15.100:0/2329443698 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5566a8c6ad50 0x5566a8c6b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:36.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.734+0000 7f43e01eab80 1 --2- 172.21.15.100:0/2329443698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566a8c701b0 0x5566a8c70590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:36.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.734+0000 7f43e01eab80 1 --2- 172.21.15.100:0/2329443698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566a8c70ad0 0x5566a8c6a810 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:36.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.734+0000 7f43e01eab80 1 -- 172.21.15.100:0/2329443698 >> 172.21.15.100:0/2329443698 conn(0x5566a8b96b70 msgr2=0x5566a8c652a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:24:36.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.734+0000 7f43e01eab80 1 -- 172.21.15.100:0/2329443698 shutdown_connections 2024-05-10T11:24:36.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.734+0000 7f43e01eab80 1 -- 172.21.15.100:0/2329443698 wait complete. 2024-05-10T11:24:36.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.735+0000 7f43e01eab80 1 -- start start 2024-05-10T11:24:36.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.735+0000 7f43e01eab80 1 Processor -- start 2024-05-10T11:24:36.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.735+0000 7f43e01eab80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5566a8c6ad50 0x5566a8c76860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:36.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.735+0000 7f43e01eab80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566a8c701b0 0x5566a8c73b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:36.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.735+0000 7f43e01eab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566a8c70ad0 0x5566a8c740a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:36.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.735+0000 7f43e01eab80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5566a8c773f0 con 0x5566a8c701b0 2024-05-10T11:24:36.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.735+0000 7f43e01eab80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5566a8b75ed0 con 0x5566a8c6ad50 2024-05-10T11:24:36.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.735+0000 7f43e01eab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5566a8c77570 con 0x5566a8c70ad0 2024-05-10T11:24:36.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.735+0000 7f43ce7fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566a8c701b0 0x5566a8c73b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:36.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.735+0000 7f43ce7fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566a8c701b0 0x5566a8c73b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39920/0 (socket says 172.21.15.100:39920) 2024-05-10T11:24:36.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.735+0000 7f43ce7fc700 1 -- 172.21.15.100:0/1782857706 learned_addr learned my addr 172.21.15.100:0/1782857706 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:24:36.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.735+0000 7f43ceffd700 1 --2- 172.21.15.100:0/1782857706 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5566a8c6ad50 0x5566a8c76860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:36.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.735+0000 7f43e01eab80 10 client.0 _ll_register_callbacks cb 0x5566a8b505b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:24:36.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.736+0000 7f43cf7fe700 1 --2- 172.21.15.100:0/1782857706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566a8c70ad0 0x5566a8c740a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:36.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.736+0000 7f43ce7fc700 1 -- 172.21.15.100:0/1782857706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566a8c70ad0 msgr2=0x5566a8c740a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:36.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.736+0000 7f43ce7fc700 1 --2- 172.21.15.100:0/1782857706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566a8c70ad0 0x5566a8c740a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:36.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.736+0000 7f43ce7fc700 1 -- 172.21.15.100:0/1782857706 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5566a8c6ad50 msgr2=0x5566a8c76860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:36.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.736+0000 7f43ce7fc700 1 --2- 172.21.15.100:0/1782857706 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5566a8c6ad50 0x5566a8c76860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:36.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.736+0000 7f43ce7fc700 1 -- 172.21.15.100:0/1782857706 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566a8b73760 con 0x5566a8c701b0 2024-05-10T11:24:36.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.736+0000 7f43ceffd700 1 --2- 172.21.15.100:0/1782857706 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5566a8c6ad50 0x5566a8c76860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:24:36.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.736+0000 7f43ce7fc700 1 --2- 172.21.15.100:0/1782857706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566a8c701b0 0x5566a8c73b60 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f43c4009350 tx=0x7f43c4005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55f7fce8401b737d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:24:36.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.736+0000 7f43aeffd700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:24:36.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.736+0000 7f43aeffd700 1 -- 172.21.15.100:0/1782857706 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c403c070 con 0x5566a8c701b0 2024-05-10T11:24:36.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.736+0000 7f43aeffd700 1 -- 172.21.15.100:0/1782857706 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f43c40235c0 con 0x5566a8c701b0 2024-05-10T11:24:36.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.736+0000 7f43e01eab80 1 -- 172.21.15.100:0/1782857706 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43c4023410 con 0x5566a8c701b0 2024-05-10T11:24:36.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.736+0000 7f43aeffd700 1 -- 172.21.15.100:0/1782857706 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f43c4044050 con 0x5566a8c701b0 2024-05-10T11:24:36.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.737+0000 7f43e01eab80 10 client.16608 fetch_fsmap learned FSMap version 31 2024-05-10T11:24:36.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.737+0000 7f43e01eab80 1 -- 172.21.15.100:0/1782857706 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5566a8c81dd0 con 0x5566a8c701b0 2024-05-10T11:24:36.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.737+0000 7f43aeffd700 1 -- 172.21.15.100:0/1782857706 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f43c4023410 con 0x5566a8c701b0 2024-05-10T11:24:36.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.737+0000 7f43aeffd700 1 -- 172.21.15.100:0/1782857706 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f43c403f030 con 0x5566a8c701b0 2024-05-10T11:24:36.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.737+0000 7f43e01eab80 10 client.16608 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:24:36.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.737+0000 7f43e01eab80 10 client.16608 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-10T11:24:36.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.737+0000 7f43e01eab80 20 client.16608 populate_metadata read hostname 'smithi100' 2024-05-10T11:24:36.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.737+0000 7f43e01eab80 1 --2- 172.21.15.100:0/1782857706 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x5566a8c842a0 0x5566a8c86690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:36.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.737+0000 7f43e01eab80 4 client.16608 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:24:36.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.737+0000 7f43e01eab80 1 -- 172.21.15.100:0/1782857706 --> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566a8bca320 con 0x5566a8c842a0 2024-05-10T11:24:36.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.737+0000 7f43ceffd700 1 --2- 172.21.15.100:0/1782857706 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x5566a8c842a0 0x5566a8c86690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:36.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.738+0000 7f43ceffd700 1 --2- 172.21.15.100:0/1782857706 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x5566a8c842a0 0x5566a8c86690 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b6f5849e85d83a2d server_cookie=33379089bd071dba in_seq=0 out_seq=0 2024-05-10T11:24:36.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.738+0000 7f43aeffd700 10 client.16608 ms_handle_connect on v2:172.21.15.178:6838/3760985404 2024-05-10T11:24:36.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.739+0000 7f43aeffd700 1 -- 172.21.15.100:0/1782857706 <== mds.1 v2:172.21.15.178:6838/3760985404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5566a8bca320 con 0x5566a8c842a0 2024-05-10T11:24:36.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.739+0000 7f43aeffd700 10 client.16608 handle_command_reply: tid=0 2024-05-10T11:24:36.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.739+0000 7f43e01eab80 1 client.16608 shutdown 2024-05-10T11:24:36.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.739+0000 7f43e01eab80 1 -- 172.21.15.100:0/1782857706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566a8c701b0 msgr2=0x5566a8c73b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:36.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.739+0000 7f43e01eab80 1 --2- 172.21.15.100:0/1782857706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566a8c701b0 0x5566a8c73b60 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f43c4009350 tx=0x7f43c4005c00 comp rx=0 tx=0).stop 2024-05-10T11:24:36.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.739+0000 7f43e01eab80 1 -- 172.21.15.100:0/1782857706 shutdown_connections 2024-05-10T11:24:36.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.739+0000 7f43e01eab80 1 --2- 172.21.15.100:0/1782857706 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x5566a8c842a0 0x5566a8c86690 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:36.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.739+0000 7f43e01eab80 1 --2- 172.21.15.100:0/1782857706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566a8c70ad0 0x5566a8c740a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:36.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.739+0000 7f43e01eab80 1 --2- 172.21.15.100:0/1782857706 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5566a8c6ad50 0x5566a8c76860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:36.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.739+0000 7f43e01eab80 1 --2- 172.21.15.100:0/1782857706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566a8c701b0 0x5566a8c73b60 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:36.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.739+0000 7f43e01eab80 1 -- 172.21.15.100:0/1782857706 >> 172.21.15.100:0/1782857706 conn(0x5566a8b96b70 msgr2=0x5566a8c5fb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:24:36.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.739+0000 7f43aeffd700 0 client.16608 ms_handle_reset on v2:172.21.15.178:6838/3760985404 2024-05-10T11:24:36.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.739+0000 7f43e01eab80 1 -- 172.21.15.100:0/1782857706 shutdown_connections 2024-05-10T11:24:36.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.739+0000 7f43e01eab80 1 -- 172.21.15.100:0/1782857706 wait complete. 2024-05-10T11:24:36.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.739+0000 7f43e01eab80 20 client.16608 trim_cache size 0 max 16384 2024-05-10T11:24:36.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.756+0000 7f43e01eab80 1 Processor -- start 2024-05-10T11:24:36.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.756+0000 7f43e01eab80 1 -- start start 2024-05-10T11:24:36.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.756+0000 7f43e01eab80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566a8c6ad50 0x5566a8c6a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:36.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.756+0000 7f43e01eab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566a8c701b0 0x5566a8c5fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:36.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.756+0000 7f43e01eab80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5566a8c70ad0 0x5566a8c715c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:36.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.756+0000 7f43e01eab80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f43d01af7a0 con 0x5566a8c6ad50 2024-05-10T11:24:36.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.756+0000 7f43e01eab80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f43d01ad030 con 0x5566a8c70ad0 2024-05-10T11:24:36.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.756+0000 7f43e01eab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f43d01aaea0 con 0x5566a8c701b0 2024-05-10T11:24:36.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.757+0000 7f43ceffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566a8c6ad50 0x5566a8c6a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:36.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.757+0000 7f43ceffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566a8c6ad50 0x5566a8c6a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39936/0 (socket says 172.21.15.100:39936) 2024-05-10T11:24:36.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.757+0000 7f43ceffd700 1 -- 172.21.15.100:0/77197732 learned_addr learned my addr 172.21.15.100:0/77197732 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:24:36.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.757+0000 7f43ce7fc700 1 --2- 172.21.15.100:0/77197732 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566a8c701b0 0x5566a8c5fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:36.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.757+0000 7f43cf7fe700 1 --2- 172.21.15.100:0/77197732 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5566a8c70ad0 0x5566a8c715c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:36.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.757+0000 7f43ceffd700 1 -- 172.21.15.100:0/77197732 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566a8c701b0 msgr2=0x5566a8c5fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:36.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.757+0000 7f43ceffd700 1 --2- 172.21.15.100:0/77197732 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566a8c701b0 0x5566a8c5fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:36.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.757+0000 7f43ceffd700 1 -- 172.21.15.100:0/77197732 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5566a8c70ad0 msgr2=0x5566a8c715c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:36.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.757+0000 7f43ceffd700 1 --2- 172.21.15.100:0/77197732 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5566a8c70ad0 0x5566a8c715c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:36.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.757+0000 7f43ceffd700 1 -- 172.21.15.100:0/77197732 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566a8c707e0 con 0x5566a8c6ad50 2024-05-10T11:24:36.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.757+0000 7f43ceffd700 1 --2- 172.21.15.100:0/77197732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566a8c6ad50 0x5566a8c6a810 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f43c0009860 tx=0x7f43c00cc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca8820e234308790 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:24:36.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.757+0000 7f43b57fa700 1 -- 172.21.15.100:0/77197732 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c00a9510 con 0x5566a8c6ad50 2024-05-10T11:24:36.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.757+0000 7f43b57fa700 1 -- 172.21.15.100:0/77197732 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f43c00bee50 con 0x5566a8c6ad50 2024-05-10T11:24:36.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.757+0000 7f43b57fa700 1 -- 172.21.15.100:0/77197732 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c0010720 con 0x5566a8c6ad50 2024-05-10T11:24:36.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.758+0000 7f43e01eab80 1 -- 172.21.15.100:0/77197732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566a8c6ad50 msgr2=0x5566a8c6a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:36.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.758+0000 7f43e01eab80 1 --2- 172.21.15.100:0/77197732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566a8c6ad50 0x5566a8c6a810 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f43c0009860 tx=0x7f43c00cc040 comp rx=0 tx=0).stop 2024-05-10T11:24:36.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.758+0000 7f43e01eab80 1 -- 172.21.15.100:0/77197732 shutdown_connections 2024-05-10T11:24:36.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.758+0000 7f43e01eab80 1 --2- 172.21.15.100:0/77197732 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5566a8c70ad0 0x5566a8c715c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:36.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.758+0000 7f43e01eab80 1 --2- 172.21.15.100:0/77197732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566a8c6ad50 0x5566a8c6a810 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:36.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.758+0000 7f43e01eab80 1 --2- 172.21.15.100:0/77197732 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566a8c701b0 0x5566a8c5fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:36.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.758+0000 7f43e01eab80 1 -- 172.21.15.100:0/77197732 >> 172.21.15.100:0/77197732 conn(0x5566a8b96b70 msgr2=0x5566a8c70eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:24:36.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.758+0000 7f43e01eab80 1 -- 172.21.15.100:0/77197732 shutdown_connections 2024-05-10T11:24:36.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.758+0000 7f43e01eab80 1 -- 172.21.15.100:0/77197732 wait complete. 2024-05-10T11:24:36.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.759+0000 7f43e01eab80 1 -- start start 2024-05-10T11:24:36.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.759+0000 7f43e01eab80 1 Processor -- start 2024-05-10T11:24:36.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.759+0000 7f43e01eab80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5566a8c6ad50 0x5566a8c862b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:36.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.759+0000 7f43e01eab80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566a8c701b0 0x5566a8c867f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:36.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.759+0000 7f43e01eab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566a8c70ad0 0x5566a8c86d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:36.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.759+0000 7f43e01eab80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f43d01ab4f0 con 0x5566a8c701b0 2024-05-10T11:24:36.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.759+0000 7f43e01eab80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f43d01ac960 con 0x5566a8c6ad50 2024-05-10T11:24:36.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.759+0000 7f43e01eab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f43d01abbc0 con 0x5566a8c70ad0 2024-05-10T11:24:36.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.759+0000 7f43ce7fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566a8c701b0 0x5566a8c867f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:36.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.759+0000 7f43e01eab80 10 client.0 _ll_register_callbacks cb 0x5566a8b505b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:24:36.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.759+0000 7f43cf7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566a8c70ad0 0x5566a8c86d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:36.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.759+0000 7f43ceffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5566a8c6ad50 0x5566a8c862b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:36.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.759+0000 7f43ce7fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566a8c701b0 0x5566a8c867f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39948/0 (socket says 172.21.15.100:39948) 2024-05-10T11:24:36.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.759+0000 7f43ce7fc700 1 -- 172.21.15.100:0/3472953240 learned_addr learned my addr 172.21.15.100:0/3472953240 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:24:36.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.760+0000 7f43ce7fc700 1 -- 172.21.15.100:0/3472953240 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566a8c70ad0 msgr2=0x5566a8c86d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:36.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.760+0000 7f43ce7fc700 1 --2- 172.21.15.100:0/3472953240 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566a8c70ad0 0x5566a8c86d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:36.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.760+0000 7f43ce7fc700 1 -- 172.21.15.100:0/3472953240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5566a8c6ad50 msgr2=0x5566a8c862b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:36.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.760+0000 7f43ce7fc700 1 --2- 172.21.15.100:0/3472953240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5566a8c6ad50 0x5566a8c862b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:36.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.760+0000 7f43ce7fc700 1 -- 172.21.15.100:0/3472953240 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566a8cec7b0 con 0x5566a8c701b0 2024-05-10T11:24:36.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.760+0000 7f43ce7fc700 1 --2- 172.21.15.100:0/3472953240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566a8c701b0 0x5566a8c867f0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f43c400b650 tx=0x7f43c4036660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=490883277794716e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:24:36.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.760+0000 7f43aeffd700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:24:36.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.760+0000 7f43aeffd700 1 -- 172.21.15.100:0/3472953240 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c403c070 con 0x5566a8c701b0 2024-05-10T11:24:36.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.760+0000 7f43aeffd700 1 -- 172.21.15.100:0/3472953240 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f43c40387f0 con 0x5566a8c701b0 2024-05-10T11:24:36.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.760+0000 7f43e01eab80 1 -- 172.21.15.100:0/3472953240 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43c00bee50 con 0x5566a8c701b0 2024-05-10T11:24:36.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.760+0000 7f43aeffd700 1 -- 172.21.15.100:0/3472953240 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f43c4038a10 con 0x5566a8c701b0 2024-05-10T11:24:36.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.760+0000 7f43e01eab80 10 client.16614 fetch_fsmap learned FSMap version 31 2024-05-10T11:24:36.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.760+0000 7f43e01eab80 1 -- 172.21.15.100:0/3472953240 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5566a8ceda90 con 0x5566a8c701b0 2024-05-10T11:24:36.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.761+0000 7f43aeffd700 1 -- 172.21.15.100:0/3472953240 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f43c00bee50 con 0x5566a8c701b0 2024-05-10T11:24:36.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.761+0000 7f43aeffd700 1 -- 172.21.15.100:0/3472953240 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f43c403f070 con 0x5566a8c701b0 2024-05-10T11:24:36.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.761+0000 7f43e01eab80 10 client.16614 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:24:36.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.761+0000 7f43e01eab80 10 client.16614 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-10T11:24:36.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.761+0000 7f43e01eab80 20 client.16614 populate_metadata read hostname 'smithi100' 2024-05-10T11:24:36.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.761+0000 7f43e01eab80 1 --2- 172.21.15.100:0/3472953240 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x5566a8cee090 0x5566a8cf04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:36.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.761+0000 7f43e01eab80 4 client.16614 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:24:36.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.761+0000 7f43e01eab80 1 -- 172.21.15.100:0/3472953240 --> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5566a8bc43c0 con 0x5566a8cee090 2024-05-10T11:24:36.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.761+0000 7f43ceffd700 1 --2- 172.21.15.100:0/3472953240 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x5566a8cee090 0x5566a8cf04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:36.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.762+0000 7f43ceffd700 1 --2- 172.21.15.100:0/3472953240 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x5566a8cee090 0x5566a8cf04b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bdbf9545e2c6c80f server_cookie=594714d2ae35602b in_seq=0 out_seq=0 2024-05-10T11:24:36.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.762+0000 7f43aeffd700 10 client.16614 ms_handle_connect on v2:172.21.15.178:6838/3760985404 2024-05-10T11:24:36.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.762+0000 7f43aeffd700 1 -- 172.21.15.100:0/3472953240 <== mds.1 v2:172.21.15.178:6838/3760985404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5566a8bc43c0 con 0x5566a8cee090 2024-05-10T11:24:36.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.762+0000 7f43aeffd700 10 client.16614 handle_command_reply: tid=0 2024-05-10T11:24:36.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.762+0000 7f43e01eab80 1 client.16614 shutdown 2024-05-10T11:24:36.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.762+0000 7f43e01eab80 1 -- 172.21.15.100:0/3472953240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566a8c701b0 msgr2=0x5566a8c867f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:36.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.762+0000 7f43e01eab80 1 --2- 172.21.15.100:0/3472953240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566a8c701b0 0x5566a8c867f0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f43c400b650 tx=0x7f43c4036660 comp rx=0 tx=0).stop 2024-05-10T11:24:36.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.762+0000 7f43e01eab80 1 -- 172.21.15.100:0/3472953240 shutdown_connections 2024-05-10T11:24:36.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.762+0000 7f43e01eab80 1 --2- 172.21.15.100:0/3472953240 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x5566a8cee090 0x5566a8cf04b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:36.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.762+0000 7f43e01eab80 1 --2- 172.21.15.100:0/3472953240 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566a8c70ad0 0x5566a8c86d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:36.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.762+0000 7f43e01eab80 1 --2- 172.21.15.100:0/3472953240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5566a8c6ad50 0x5566a8c862b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:36.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.762+0000 7f43e01eab80 1 --2- 172.21.15.100:0/3472953240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566a8c701b0 0x5566a8c867f0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:36.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.762+0000 7f43e01eab80 1 -- 172.21.15.100:0/3472953240 >> 172.21.15.100:0/3472953240 conn(0x5566a8b96b70 msgr2=0x5566a8c72450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:24:36.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.762+0000 7f43aeffd700 0 client.16614 ms_handle_reset on v2:172.21.15.178:6838/3760985404 2024-05-10T11:24:36.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.762+0000 7f43e01eab80 1 -- 172.21.15.100:0/3472953240 shutdown_connections 2024-05-10T11:24:36.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.762+0000 7f43e01eab80 1 -- 172.21.15.100:0/3472953240 wait complete. 2024-05-10T11:24:36.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.762+0000 7f43e01eab80 20 client.16614 trim_cache size 0 max 16384 2024-05-10T11:24:36.767 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:24:36.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.763+0000 7f43d5957700 1 -- 172.21.15.100:0/1134277488 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f43b80b2880 msgr2=0x7f43b80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:36.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.763+0000 7f43d5957700 1 --2- 172.21.15.100:0/1134277488 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f43b80b2880 0x7f43b80b4d40 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f43d00ffab0 tx=0x7f43c401a040 comp rx=0 tx=0).stop 2024-05-10T11:24:36.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.763+0000 7f43d5957700 1 -- 172.21.15.100:0/1134277488 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43d0101a90 msgr2=0x7f43d0102c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:36.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.763+0000 7f43d5957700 1 --2- 172.21.15.100:0/1134277488 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43d0101a90 0x7f43d0102c70 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f43c000e9a0 tx=0x7f43c000ecb0 comp rx=0 tx=0).stop 2024-05-10T11:24:36.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.763+0000 7f43d5957700 1 -- 172.21.15.100:0/1134277488 shutdown_connections 2024-05-10T11:24:36.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.763+0000 7f43d5957700 1 --2- 172.21.15.100:0/1134277488 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f43b80b2880 0x7f43b80b4d40 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:36.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.763+0000 7f43d5957700 1 --2- 172.21.15.100:0/1134277488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43d0109520 0x7f43d01036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:36.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.763+0000 7f43d5957700 1 --2- 172.21.15.100:0/1134277488 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43d0101a90 0x7f43d0102c70 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:36.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.763+0000 7f43d5957700 1 --2- 172.21.15.100:0/1134277488 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43d0102450 0x7f43d01031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:36.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.763+0000 7f43d5957700 1 -- 172.21.15.100:0/1134277488 >> 172.21.15.100:0/1134277488 conn(0x7f43d0078a10 msgr2=0x7f43d0100400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:24:36.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.763+0000 7f43d5957700 1 -- 172.21.15.100:0/1134277488 shutdown_connections 2024-05-10T11:24:36.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:36.763+0000 7f43d5957700 1 -- 172.21.15.100:0/1134277488 wait complete. 2024-05-10T11:24:36.972 INFO:tasks.workunit.client.0.smithi100.stdout: 1 141539 2.75 MB/sec execute 151 sec latency 13.468 ms 2024-05-10T11:24:37.586 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-10T11:24:37.849 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:24:37.972 INFO:tasks.workunit.client.0.smithi100.stdout: 1 141997 2.76 MB/sec execute 152 sec latency 15.301 ms 2024-05-10T11:24:38.225 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:24:37 smithi100 ceph-mon[30296]: pgmap v607: 129 pgs: 129 active+clean; 65 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1.3 MiB/s wr, 140 op/s 2024-05-10T11:24:38.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:24:37 smithi178 ceph-mon[33359]: pgmap v607: 129 pgs: 129 active+clean; 65 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1.3 MiB/s wr, 140 op/s 2024-05-10T11:24:38.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:24:37 smithi188 ceph-mon[34075]: pgmap v607: 129 pgs: 129 active+clean; 65 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1.3 MiB/s wr, 140 op/s 2024-05-10T11:24:38.972 INFO:tasks.workunit.client.0.smithi100.stdout: 1 142317 2.75 MB/sec execute 153 sec latency 15.336 ms 2024-05-10T11:24:39.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.479+0000 7fb2a0262700 1 -- 172.21.15.100:0/1762810420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb298103750 msgr2=0x7fb298103b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:39.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.479+0000 7fb2a0262700 1 --2- 172.21.15.100:0/1762810420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb298103750 0x7fb298103b30 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fb2940099c0 tx=0x7fb294009cd0 comp rx=0 tx=0).stop 2024-05-10T11:24:39.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.479+0000 7fb2a0262700 1 -- 172.21.15.100:0/1762810420 shutdown_connections 2024-05-10T11:24:39.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.479+0000 7fb2a0262700 1 --2- 172.21.15.100:0/1762810420 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb298108850 0x7fb2981103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:39.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.479+0000 7fb2a0262700 1 --2- 172.21.15.100:0/1762810420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb298103750 0x7fb298103b30 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:39.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.479+0000 7fb2a0262700 1 --2- 172.21.15.100:0/1762810420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb298104100 0x7fb298108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:39.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.479+0000 7fb2a0262700 1 -- 172.21.15.100:0/1762810420 >> 172.21.15.100:0/1762810420 conn(0x7fb29807a600 msgr2=0x7fb29807aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:24:39.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.479+0000 7fb2a0262700 1 -- 172.21.15.100:0/1762810420 shutdown_connections 2024-05-10T11:24:39.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.479+0000 7fb2a0262700 1 -- 172.21.15.100:0/1762810420 wait complete. 2024-05-10T11:24:39.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.480+0000 7fb2a0262700 1 Processor -- start 2024-05-10T11:24:39.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.480+0000 7fb2a0262700 1 -- start start 2024-05-10T11:24:39.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.480+0000 7fb2a0262700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb298103750 0x7fb29819f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:39.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.480+0000 7fb2a0262700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb298104100 0x7fb29819fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:39.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.480+0000 7fb2a0262700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb298108850 0x7fb298199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:39.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.480+0000 7fb2a0262700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb2981134f0 con 0x7fb298108850 2024-05-10T11:24:39.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.480+0000 7fb2a0262700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb298113370 con 0x7fb298104100 2024-05-10T11:24:39.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.480+0000 7fb2a0262700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb298113670 con 0x7fb298103750 2024-05-10T11:24:39.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.480+0000 7fb29dffe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb298103750 0x7fb29819f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:39.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.480+0000 7fb29dffe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb298103750 0x7fb29819f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:56132/0 (socket says 172.21.15.100:56132) 2024-05-10T11:24:39.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.480+0000 7fb29d7fd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb298104100 0x7fb29819fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:39.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.480+0000 7fb29e7ff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb298108850 0x7fb298199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:39.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.480+0000 7fb29dffe700 1 -- 172.21.15.100:0/2230767398 learned_addr learned my addr 172.21.15.100:0/2230767398 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:24:39.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.481+0000 7fb29d7fd700 1 -- 172.21.15.100:0/2230767398 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb298103750 msgr2=0x7fb29819f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:39.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.481+0000 7fb29d7fd700 1 --2- 172.21.15.100:0/2230767398 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb298103750 0x7fb29819f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:39.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.481+0000 7fb29d7fd700 1 -- 172.21.15.100:0/2230767398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb298108850 msgr2=0x7fb298199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:39.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.481+0000 7fb29d7fd700 1 --2- 172.21.15.100:0/2230767398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb298108850 0x7fb298199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:39.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.481+0000 7fb29d7fd700 1 -- 172.21.15.100:0/2230767398 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb29819a010 con 0x7fb298104100 2024-05-10T11:24:39.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.481+0000 7fb29e7ff700 1 --2- 172.21.15.100:0/2230767398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb298108850 0x7fb298199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:24:39.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.481+0000 7fb29d7fd700 1 --2- 172.21.15.100:0/2230767398 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb298104100 0x7fb29819fbf0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb28800e9f0 tx=0x7fb28800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:24:39.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.481+0000 7fb28effd700 1 -- 172.21.15.100:0/2230767398 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28800c9c0 con 0x7fb298104100 2024-05-10T11:24:39.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.481+0000 7fb2a0262700 1 -- 172.21.15.100:0/2230767398 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb29819a300 con 0x7fb298104100 2024-05-10T11:24:39.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.481+0000 7fb28effd700 1 -- 172.21.15.100:0/2230767398 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb288010430 con 0x7fb298104100 2024-05-10T11:24:39.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.481+0000 7fb28effd700 1 -- 172.21.15.100:0/2230767398 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb288018680 con 0x7fb298104100 2024-05-10T11:24:39.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.481+0000 7fb2a0262700 1 -- 172.21.15.100:0/2230767398 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2981a6510 con 0x7fb298104100 2024-05-10T11:24:39.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.484+0000 7fb28effd700 1 -- 172.21.15.100:0/2230767398 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fb28800cb60 con 0x7fb298104100 2024-05-10T11:24:39.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.485+0000 7fb28effd700 1 --2- 172.21.15.100:0/2230767398 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fb2840b2900 0x7fb2840b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:39.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.485+0000 7fb28effd700 1 -- 172.21.15.100:0/2230767398 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fb2880e2600 con 0x7fb298104100 2024-05-10T11:24:39.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.485+0000 7fb29dffe700 1 --2- 172.21.15.100:0/2230767398 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fb2840b2900 0x7fb2840b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:39.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.485+0000 7fb2aaaf5b80 1 Processor -- start 2024-05-10T11:24:39.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.485+0000 7fb2aaaf5b80 1 -- start start 2024-05-10T11:24:39.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.485+0000 7fb2aaaf5b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56371d7dc4d0 0x56371d7dc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:39.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.485+0000 7fb2aaaf5b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56371d7dcdf0 0x56371d7e06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:39.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.485+0000 7fb2aaaf5b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56371d7e0c80 0x56371d7e1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:39.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.485+0000 7fb29dffe700 1 --2- 172.21.15.100:0/2230767398 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fb2840b2900 0x7fb2840b4dc0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb294009990 tx=0x7fb294005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:24:39.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.486+0000 7fb2aaaf5b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56371d72e860 con 0x56371d7e0c80 2024-05-10T11:24:39.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.486+0000 7fb2aaaf5b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56371d6cd7c0 con 0x56371d7dcdf0 2024-05-10T11:24:39.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.486+0000 7fb2aaaf5b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56371d72a820 con 0x56371d7dc4d0 2024-05-10T11:24:39.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.486+0000 7fb29dffe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56371d7dc4d0 0x56371d7dc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:39.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.486+0000 7fb29d7fd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56371d7dcdf0 0x56371d7e06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:39.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.486+0000 7fb29dffe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56371d7dc4d0 0x56371d7dc8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:56140/0 (socket says 172.21.15.100:56140) 2024-05-10T11:24:39.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.486+0000 7fb29dffe700 1 -- 172.21.15.100:0/1962887204 learned_addr learned my addr 172.21.15.100:0/1962887204 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:24:39.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.486+0000 7fb29e7ff700 1 --2- 172.21.15.100:0/1962887204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56371d7e0c80 0x56371d7e1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:39.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.486+0000 7fb29dffe700 1 -- 172.21.15.100:0/1962887204 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56371d7dcdf0 msgr2=0x56371d7e06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:39.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.486+0000 7fb29dffe700 1 --2- 172.21.15.100:0/1962887204 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56371d7dcdf0 0x56371d7e06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:39.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.486+0000 7fb29dffe700 1 -- 172.21.15.100:0/1962887204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56371d7e0c80 msgr2=0x56371d7e1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:39.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.486+0000 7fb29dffe700 1 --2- 172.21.15.100:0/1962887204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56371d7e0c80 0x56371d7e1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:39.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.486+0000 7fb29dffe700 1 -- 172.21.15.100:0/1962887204 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56371d67b8d0 con 0x56371d7dc4d0 2024-05-10T11:24:39.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.486+0000 7fb29dffe700 1 --2- 172.21.15.100:0/1962887204 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56371d7dc4d0 0x56371d7dc8b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb294003730 tx=0x7fb29401a070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4408f1e03e5cd722 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:24:39.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.486+0000 7fb283fff700 1 -- 172.21.15.100:0/1962887204 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb294004e30 con 0x56371d7dc4d0 2024-05-10T11:24:39.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.486+0000 7fb283fff700 1 -- 172.21.15.100:0/1962887204 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb29400bc20 con 0x56371d7dc4d0 2024-05-10T11:24:39.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.486+0000 7fb283fff700 1 -- 172.21.15.100:0/1962887204 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb294023b10 con 0x56371d7dc4d0 2024-05-10T11:24:39.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.487+0000 7fb2aaaf5b80 1 -- 172.21.15.100:0/1962887204 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56371d7dc4d0 msgr2=0x56371d7dc8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:39.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.487+0000 7fb2aaaf5b80 1 --2- 172.21.15.100:0/1962887204 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56371d7dc4d0 0x56371d7dc8b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb294003730 tx=0x7fb29401a070 comp rx=0 tx=0).stop 2024-05-10T11:24:39.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.487+0000 7fb2aaaf5b80 1 -- 172.21.15.100:0/1962887204 shutdown_connections 2024-05-10T11:24:39.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.487+0000 7fb2aaaf5b80 1 --2- 172.21.15.100:0/1962887204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56371d7e0c80 0x56371d7e1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:39.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.487+0000 7fb2aaaf5b80 1 --2- 172.21.15.100:0/1962887204 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56371d7dc4d0 0x56371d7dc8b0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:39.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.487+0000 7fb2aaaf5b80 1 --2- 172.21.15.100:0/1962887204 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56371d7dcdf0 0x56371d7e06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:39.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.487+0000 7fb2aaaf5b80 1 -- 172.21.15.100:0/1962887204 >> 172.21.15.100:0/1962887204 conn(0x56371d708100 msgr2=0x56371d7d6c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:24:39.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.487+0000 7fb2aaaf5b80 1 -- 172.21.15.100:0/1962887204 shutdown_connections 2024-05-10T11:24:39.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.487+0000 7fb2aaaf5b80 1 -- 172.21.15.100:0/1962887204 wait complete. 2024-05-10T11:24:39.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.488+0000 7fb2aaaf5b80 1 -- start start 2024-05-10T11:24:39.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.488+0000 7fb2aaaf5b80 1 Processor -- start 2024-05-10T11:24:39.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.488+0000 7fb2aaaf5b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56371d7dc4d0 0x56371d7e8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:39.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.488+0000 7fb2aaaf5b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56371d7dcdf0 0x56371d7e53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:39.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.488+0000 7fb2aaaf5b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56371d7e0c80 0x56371d7e5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:39.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.488+0000 7fb2aaaf5b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56371d7e8dd0 con 0x56371d7dcdf0 2024-05-10T11:24:39.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.488+0000 7fb2aaaf5b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56371d694c30 con 0x56371d7dc4d0 2024-05-10T11:24:39.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.488+0000 7fb2aaaf5b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56371d7e8f50 con 0x56371d7e0c80 2024-05-10T11:24:39.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.488+0000 7fb2aaaf5b80 10 client.0 _ll_register_callbacks cb 0x56371d6c6920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:24:39.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.488+0000 7fb29d7fd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56371d7dcdf0 0x56371d7e53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:39.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.488+0000 7fb29dffe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56371d7dc4d0 0x56371d7e8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:39.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.488+0000 7fb29d7fd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56371d7dcdf0 0x56371d7e53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39970/0 (socket says 172.21.15.100:39970) 2024-05-10T11:24:39.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.488+0000 7fb29d7fd700 1 -- 172.21.15.100:0/3650115048 learned_addr learned my addr 172.21.15.100:0/3650115048 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:24:39.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.489+0000 7fb29e7ff700 1 --2- 172.21.15.100:0/3650115048 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56371d7e0c80 0x56371d7e5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:39.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.489+0000 7fb29d7fd700 1 -- 172.21.15.100:0/3650115048 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56371d7e0c80 msgr2=0x56371d7e5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:39.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.489+0000 7fb29d7fd700 1 --2- 172.21.15.100:0/3650115048 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56371d7e0c80 0x56371d7e5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:39.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.489+0000 7fb29d7fd700 1 -- 172.21.15.100:0/3650115048 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56371d7dc4d0 msgr2=0x56371d7e8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:39.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.489+0000 7fb29d7fd700 1 --2- 172.21.15.100:0/3650115048 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56371d7dc4d0 0x56371d7e8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:39.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.489+0000 7fb29d7fd700 1 -- 172.21.15.100:0/3650115048 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56371d6ca690 con 0x56371d7dcdf0 2024-05-10T11:24:39.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.489+0000 7fb29dffe700 1 --2- 172.21.15.100:0/3650115048 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56371d7dc4d0 0x56371d7e8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:24:39.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.489+0000 7fb29d7fd700 1 --2- 172.21.15.100:0/3650115048 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56371d7dcdf0 0x56371d7e53e0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fb2880dfdf0 tx=0x7fb288010b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f67fa0060fcb22b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:24:39.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.489+0000 7fb2817fa700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:24:39.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.489+0000 7fb2817fa700 1 -- 172.21.15.100:0/3650115048 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2880a93d0 con 0x56371d7dcdf0 2024-05-10T11:24:39.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.489+0000 7fb2817fa700 1 -- 172.21.15.100:0/3650115048 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fb2880a9b20 con 0x56371d7dcdf0 2024-05-10T11:24:39.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.489+0000 7fb2aaaf5b80 1 -- 172.21.15.100:0/3650115048 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb29400bc20 con 0x56371d7dcdf0 2024-05-10T11:24:39.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.489+0000 7fb2817fa700 1 -- 172.21.15.100:0/3650115048 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb2880a9fc0 con 0x56371d7dcdf0 2024-05-10T11:24:39.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.489+0000 7fb2aaaf5b80 10 client.16626 fetch_fsmap learned FSMap version 31 2024-05-10T11:24:39.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.489+0000 7fb2aaaf5b80 1 -- 172.21.15.100:0/3650115048 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56371d7f35e0 con 0x56371d7dcdf0 2024-05-10T11:24:39.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.490+0000 7fb2817fa700 1 -- 172.21.15.100:0/3650115048 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb29400bc20 con 0x56371d7dcdf0 2024-05-10T11:24:39.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.490+0000 7fb2817fa700 1 -- 172.21.15.100:0/3650115048 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fb2880b2290 con 0x56371d7dcdf0 2024-05-10T11:24:39.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.490+0000 7fb2aaaf5b80 10 client.16626 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:24:39.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.490+0000 7fb2aaaf5b80 10 client.16626 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:24:39.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.490+0000 7fb2aaaf5b80 20 client.16626 populate_metadata read hostname 'smithi100' 2024-05-10T11:24:39.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.490+0000 7fb2aaaf5b80 1 --2- 172.21.15.100:0/3650115048 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56371d7f5c20 0x56371d7f8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:39.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.490+0000 7fb2aaaf5b80 4 client.16626 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:24:39.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.490+0000 7fb2aaaf5b80 1 -- 172.21.15.100:0/3650115048 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56371d7a8650 con 0x56371d7f5c20 2024-05-10T11:24:39.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.490+0000 7fb29dffe700 1 --2- 172.21.15.100:0/3650115048 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56371d7f5c20 0x56371d7f8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:39.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.490+0000 7fb29dffe700 1 --2- 172.21.15.100:0/3650115048 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56371d7f5c20 0x56371d7f8040 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7290626f0058bef server_cookie=a2e7f4cc4953db42 in_seq=0 out_seq=0 2024-05-10T11:24:39.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.490+0000 7fb2817fa700 10 client.16626 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:24:39.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.491+0000 7fb2817fa700 1 -- 172.21.15.100:0/3650115048 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56371d7a8650 con 0x56371d7f5c20 2024-05-10T11:24:39.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.491+0000 7fb2817fa700 10 client.16626 handle_command_reply: tid=0 2024-05-10T11:24:39.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.491+0000 7fb2aaaf5b80 1 client.16626 shutdown 2024-05-10T11:24:39.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.491+0000 7fb2aaaf5b80 1 -- 172.21.15.100:0/3650115048 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56371d7dcdf0 msgr2=0x56371d7e53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:39.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.491+0000 7fb2aaaf5b80 1 --2- 172.21.15.100:0/3650115048 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56371d7dcdf0 0x56371d7e53e0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fb2880dfdf0 tx=0x7fb288010b10 comp rx=0 tx=0).stop 2024-05-10T11:24:39.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.492+0000 7fb2aaaf5b80 1 -- 172.21.15.100:0/3650115048 shutdown_connections 2024-05-10T11:24:39.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.492+0000 7fb2aaaf5b80 1 --2- 172.21.15.100:0/3650115048 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56371d7f5c20 0x56371d7f8040 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:39.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.492+0000 7fb2aaaf5b80 1 --2- 172.21.15.100:0/3650115048 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56371d7e0c80 0x56371d7e5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:39.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.492+0000 7fb2aaaf5b80 1 --2- 172.21.15.100:0/3650115048 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56371d7dc4d0 0x56371d7e8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:39.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.492+0000 7fb2817fa700 0 client.16626 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:24:39.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.492+0000 7fb2aaaf5b80 1 --2- 172.21.15.100:0/3650115048 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56371d7dcdf0 0x56371d7e53e0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:39.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.492+0000 7fb2aaaf5b80 1 -- 172.21.15.100:0/3650115048 >> 172.21.15.100:0/3650115048 conn(0x56371d708100 msgr2=0x56371d7d1720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:24:39.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.492+0000 7fb2aaaf5b80 1 -- 172.21.15.100:0/3650115048 shutdown_connections 2024-05-10T11:24:39.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.492+0000 7fb2aaaf5b80 1 -- 172.21.15.100:0/3650115048 wait complete. 2024-05-10T11:24:39.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.492+0000 7fb2aaaf5b80 20 client.16626 trim_cache size 0 max 16384 2024-05-10T11:24:39.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.508+0000 7fb2aaaf5b80 1 Processor -- start 2024-05-10T11:24:39.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.508+0000 7fb2aaaf5b80 1 -- start start 2024-05-10T11:24:39.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.509+0000 7fb2aaaf5b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56371d7dc4d0 0x56371d7e5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:39.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.509+0000 7fb2aaaf5b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56371d7dcdf0 0x56371d7f1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:39.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.509+0000 7fb2aaaf5b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56371d7e0c80 0x56371d7e43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:39.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.509+0000 7fb2aaaf5b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb2981b3b40 con 0x56371d7e0c80 2024-05-10T11:24:39.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.509+0000 7fb2aaaf5b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb2981b13d0 con 0x56371d7dc4d0 2024-05-10T11:24:39.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.509+0000 7fb2aaaf5b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb2981af220 con 0x56371d7dcdf0 2024-05-10T11:24:39.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.509+0000 7fb29dffe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56371d7dc4d0 0x56371d7e5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:39.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.509+0000 7fb29e7ff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56371d7e0c80 0x56371d7e43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:39.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.509+0000 7fb29d7fd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56371d7dcdf0 0x56371d7f1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:39.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.509+0000 7fb29dffe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56371d7dc4d0 0x56371d7e5fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:40358/0 (socket says 172.21.15.100:40358) 2024-05-10T11:24:39.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.509+0000 7fb29dffe700 1 -- 172.21.15.100:0/1296525909 learned_addr learned my addr 172.21.15.100:0/1296525909 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:24:39.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.509+0000 7fb29d7fd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56371d7dcdf0 0x56371d7f1e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:56162/0 (socket says 172.21.15.100:56162) 2024-05-10T11:24:39.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.509+0000 7fb29d7fd700 1 -- 172.21.15.100:0/1296525909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56371d7dc4d0 msgr2=0x56371d7e5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:39.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.509+0000 7fb29d7fd700 1 --2- 172.21.15.100:0/1296525909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56371d7dc4d0 0x56371d7e5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:39.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.509+0000 7fb29d7fd700 1 -- 172.21.15.100:0/1296525909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56371d7e0c80 msgr2=0x56371d7e43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:39.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.509+0000 7fb29d7fd700 1 --2- 172.21.15.100:0/1296525909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56371d7e0c80 0x56371d7e43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:39.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.509+0000 7fb29d7fd700 1 -- 172.21.15.100:0/1296525909 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56371d7e3910 con 0x56371d7dcdf0 2024-05-10T11:24:39.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.509+0000 7fb29dffe700 1 --2- 172.21.15.100:0/1296525909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56371d7dc4d0 0x56371d7e5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:24:39.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.509+0000 7fb29d7fd700 1 --2- 172.21.15.100:0/1296525909 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56371d7dcdf0 0x56371d7f1e90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb28800e9c0 tx=0x7fb2880e2590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba3cf247859b6b64 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:24:39.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.510+0000 7fb2837fe700 1 -- 172.21.15.100:0/1296525909 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2880adc70 con 0x56371d7dcdf0 2024-05-10T11:24:39.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.510+0000 7fb2837fe700 1 -- 172.21.15.100:0/1296525909 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb2880b2720 con 0x56371d7dcdf0 2024-05-10T11:24:39.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.510+0000 7fb2837fe700 1 -- 172.21.15.100:0/1296525909 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2880b2290 con 0x56371d7dcdf0 2024-05-10T11:24:39.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.510+0000 7fb2aaaf5b80 1 -- 172.21.15.100:0/1296525909 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56371d7dcdf0 msgr2=0x56371d7f1e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:39.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.510+0000 7fb2aaaf5b80 1 --2- 172.21.15.100:0/1296525909 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56371d7dcdf0 0x56371d7f1e90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb28800e9c0 tx=0x7fb2880e2590 comp rx=0 tx=0).stop 2024-05-10T11:24:39.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.510+0000 7fb2aaaf5b80 1 -- 172.21.15.100:0/1296525909 shutdown_connections 2024-05-10T11:24:39.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.510+0000 7fb2aaaf5b80 1 --2- 172.21.15.100:0/1296525909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56371d7e0c80 0x56371d7e43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:39.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.510+0000 7fb2aaaf5b80 1 --2- 172.21.15.100:0/1296525909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56371d7dc4d0 0x56371d7e5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:39.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.510+0000 7fb2aaaf5b80 1 --2- 172.21.15.100:0/1296525909 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56371d7dcdf0 0x56371d7f1e90 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:39.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.510+0000 7fb2aaaf5b80 1 -- 172.21.15.100:0/1296525909 >> 172.21.15.100:0/1296525909 conn(0x56371d708100 msgr2=0x56371d7e3cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:24:39.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.511+0000 7fb2aaaf5b80 1 -- 172.21.15.100:0/1296525909 shutdown_connections 2024-05-10T11:24:39.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.511+0000 7fb2aaaf5b80 1 -- 172.21.15.100:0/1296525909 wait complete. 2024-05-10T11:24:39.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.511+0000 7fb2aaaf5b80 1 -- start start 2024-05-10T11:24:39.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.511+0000 7fb2aaaf5b80 1 Processor -- start 2024-05-10T11:24:39.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.511+0000 7fb2aaaf5b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56371d7dc4d0 0x56371d7e91d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:39.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.511+0000 7fb2aaaf5b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56371d7dcdf0 0x56371d7e9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:39.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.511+0000 7fb2aaaf5b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56371d7e0c80 0x56371d866c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:39.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.511+0000 7fb2aaaf5b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb2981af890 con 0x56371d7dcdf0 2024-05-10T11:24:39.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.511+0000 7fb2aaaf5b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb2981b0d00 con 0x56371d7e0c80 2024-05-10T11:24:39.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.511+0000 7fb2aaaf5b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb2981aff60 con 0x56371d7dc4d0 2024-05-10T11:24:39.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.511+0000 7fb2aaaf5b80 10 client.0 _ll_register_callbacks cb 0x56371d6c6920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:24:39.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.512+0000 7fb29dffe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56371d7dc4d0 0x56371d7e91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:39.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.512+0000 7fb29d7fd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56371d7dcdf0 0x56371d7e9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:39.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.512+0000 7fb29dffe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56371d7dc4d0 0x56371d7e91d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:56172/0 (socket says 172.21.15.100:56172) 2024-05-10T11:24:39.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.512+0000 7fb29dffe700 1 -- 172.21.15.100:0/818801461 learned_addr learned my addr 172.21.15.100:0/818801461 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:24:39.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.512+0000 7fb29e7ff700 1 --2- 172.21.15.100:0/818801461 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56371d7e0c80 0x56371d866c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:39.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.512+0000 7fb29dffe700 1 -- 172.21.15.100:0/818801461 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56371d7e0c80 msgr2=0x56371d866c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:39.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.512+0000 7fb29dffe700 1 --2- 172.21.15.100:0/818801461 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56371d7e0c80 0x56371d866c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:39.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.512+0000 7fb29dffe700 1 -- 172.21.15.100:0/818801461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56371d7dcdf0 msgr2=0x56371d7e9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:39.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.512+0000 7fb29dffe700 1 --2- 172.21.15.100:0/818801461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56371d7dcdf0 0x56371d7e9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:39.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.512+0000 7fb29dffe700 1 -- 172.21.15.100:0/818801461 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56371d8671c0 con 0x56371d7dc4d0 2024-05-10T11:24:39.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.512+0000 7fb29dffe700 1 --2- 172.21.15.100:0/818801461 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56371d7dc4d0 0x56371d7e91d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb29400b580 tx=0x7fb294004f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f016c6f50e71a3f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:24:39.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.512+0000 7fb2817fa700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:24:39.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.512+0000 7fb2817fa700 1 -- 172.21.15.100:0/818801461 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb29400bb40 con 0x56371d7dc4d0 2024-05-10T11:24:39.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.512+0000 7fb2aaaf5b80 1 -- 172.21.15.100:0/818801461 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2880b2720 con 0x56371d7dc4d0 2024-05-10T11:24:39.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.512+0000 7fb2817fa700 1 -- 172.21.15.100:0/818801461 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fb29403c1d0 con 0x56371d7dc4d0 2024-05-10T11:24:39.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.513+0000 7fb2817fa700 1 -- 172.21.15.100:0/818801461 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb29404d050 con 0x56371d7dc4d0 2024-05-10T11:24:39.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.513+0000 7fb2aaaf5b80 10 client.26110 fetch_fsmap learned FSMap version 31 2024-05-10T11:24:39.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.513+0000 7fb2aaaf5b80 1 -- 172.21.15.100:0/818801461 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56371d8682f0 con 0x56371d7dc4d0 2024-05-10T11:24:39.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.513+0000 7fb2817fa700 1 -- 172.21.15.100:0/818801461 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb2880b2720 con 0x56371d7dc4d0 2024-05-10T11:24:39.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.513+0000 7fb2817fa700 1 -- 172.21.15.100:0/818801461 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fb2940234f0 con 0x56371d7dc4d0 2024-05-10T11:24:39.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.513+0000 7fb2aaaf5b80 10 client.26110 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:24:39.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.513+0000 7fb2aaaf5b80 10 client.26110 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:24:39.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.513+0000 7fb2aaaf5b80 20 client.26110 populate_metadata read hostname 'smithi100' 2024-05-10T11:24:39.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.513+0000 7fb2aaaf5b80 1 --2- 172.21.15.100:0/818801461 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56371d7f5c20 0x56371d86a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:39.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.513+0000 7fb2aaaf5b80 4 client.26110 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:24:39.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.513+0000 7fb2aaaf5b80 1 -- 172.21.15.100:0/818801461 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56371d73c240 con 0x56371d7f5c20 2024-05-10T11:24:39.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.513+0000 7fb29d7fd700 1 --2- 172.21.15.100:0/818801461 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56371d7f5c20 0x56371d86a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:39.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.513+0000 7fb2817fa700 10 client.26110 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:24:39.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.513+0000 7fb29d7fd700 1 --2- 172.21.15.100:0/818801461 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56371d7f5c20 0x56371d86a4d0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=370f5c4fc6ae5ae7 server_cookie=aae1ff838b16e2fc in_seq=0 out_seq=0 2024-05-10T11:24:39.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.514+0000 7fb2817fa700 1 -- 172.21.15.100:0/818801461 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56371d73c240 con 0x56371d7f5c20 2024-05-10T11:24:39.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.514+0000 7fb2817fa700 10 client.26110 handle_command_reply: tid=0 2024-05-10T11:24:39.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.514+0000 7fb2aaaf5b80 1 client.26110 shutdown 2024-05-10T11:24:39.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.514+0000 7fb2aaaf5b80 1 -- 172.21.15.100:0/818801461 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56371d7dc4d0 msgr2=0x56371d7e91d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:39.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.514+0000 7fb2aaaf5b80 1 --2- 172.21.15.100:0/818801461 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56371d7dc4d0 0x56371d7e91d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb29400b580 tx=0x7fb294004f40 comp rx=0 tx=0).stop 2024-05-10T11:24:39.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.514+0000 7fb2aaaf5b80 1 -- 172.21.15.100:0/818801461 shutdown_connections 2024-05-10T11:24:39.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.514+0000 7fb2aaaf5b80 1 --2- 172.21.15.100:0/818801461 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56371d7f5c20 0x56371d86a4d0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:39.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.514+0000 7fb2aaaf5b80 1 --2- 172.21.15.100:0/818801461 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56371d7e0c80 0x56371d866c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:39.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.514+0000 7fb2817fa700 0 client.26110 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:24:39.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.514+0000 7fb2aaaf5b80 1 --2- 172.21.15.100:0/818801461 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56371d7dc4d0 0x56371d7e91d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:39.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.514+0000 7fb2aaaf5b80 1 --2- 172.21.15.100:0/818801461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56371d7dcdf0 0x56371d7e9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:39.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.514+0000 7fb2aaaf5b80 1 -- 172.21.15.100:0/818801461 >> 172.21.15.100:0/818801461 conn(0x56371d708100 msgr2=0x56371d7d0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:24:39.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.514+0000 7fb2aaaf5b80 1 -- 172.21.15.100:0/818801461 shutdown_connections 2024-05-10T11:24:39.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.514+0000 7fb2aaaf5b80 1 -- 172.21.15.100:0/818801461 wait complete. 2024-05-10T11:24:39.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.514+0000 7fb2aaaf5b80 20 client.26110 trim_cache size 0 max 16384 2024-05-10T11:24:39.519 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:24:39.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.515+0000 7fb2a0262700 1 -- 172.21.15.100:0/2230767398 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fb2840b2900 msgr2=0x7fb2840b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:39.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.515+0000 7fb2a0262700 1 --2- 172.21.15.100:0/2230767398 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fb2840b2900 0x7fb2840b4dc0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb294009990 tx=0x7fb294005960 comp rx=0 tx=0).stop 2024-05-10T11:24:39.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.515+0000 7fb2a0262700 1 -- 172.21.15.100:0/2230767398 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb298104100 msgr2=0x7fb29819fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:39.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.515+0000 7fb2a0262700 1 --2- 172.21.15.100:0/2230767398 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb298104100 0x7fb29819fbf0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb28800e9f0 tx=0x7fb28800ed00 comp rx=0 tx=0).stop 2024-05-10T11:24:39.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.515+0000 7fb2a0262700 1 -- 172.21.15.100:0/2230767398 shutdown_connections 2024-05-10T11:24:39.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.515+0000 7fb2a0262700 1 --2- 172.21.15.100:0/2230767398 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fb2840b2900 0x7fb2840b4dc0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:39.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.515+0000 7fb2a0262700 1 --2- 172.21.15.100:0/2230767398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb298108850 0x7fb298199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:39.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.515+0000 7fb2a0262700 1 --2- 172.21.15.100:0/2230767398 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb298103750 0x7fb29819f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:39.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.515+0000 7fb2a0262700 1 --2- 172.21.15.100:0/2230767398 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb298104100 0x7fb29819fbf0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:39.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.515+0000 7fb2a0262700 1 -- 172.21.15.100:0/2230767398 >> 172.21.15.100:0/2230767398 conn(0x7fb29807a600 msgr2=0x7fb2981017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:24:39.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.515+0000 7fb2a0262700 1 -- 172.21.15.100:0/2230767398 shutdown_connections 2024-05-10T11:24:39.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:39.515+0000 7fb2a0262700 1 -- 172.21.15.100:0/2230767398 wait complete. 2024-05-10T11:24:39.972 INFO:tasks.workunit.client.0.smithi100.stdout: 1 142660 2.74 MB/sec execute 154 sec latency 34.420 ms 2024-05-10T11:24:40.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:24:39 smithi178 ceph-mon[33359]: pgmap v608: 129 pgs: 129 active+clean; 65 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 1.3 MiB/s wr, 151 op/s 2024-05-10T11:24:40.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:24:39 smithi188 ceph-mon[34075]: pgmap v608: 129 pgs: 129 active+clean; 65 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 1.3 MiB/s wr, 151 op/s 2024-05-10T11:24:40.284 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:24:39 smithi100 ceph-mon[30296]: pgmap v608: 129 pgs: 129 active+clean; 65 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 1.3 MiB/s wr, 151 op/s 2024-05-10T11:24:40.972 INFO:tasks.workunit.client.0.smithi100.stdout: 1 143166 2.73 MB/sec execute 155 sec latency 15.938 ms 2024-05-10T11:24:41.314 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T11:24:41.314 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T11:24:41.587 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:24:41.972 INFO:tasks.workunit.client.0.smithi100.stdout: 1 143772 2.74 MB/sec execute 156 sec latency 52.650 ms 2024-05-10T11:24:42.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:24:41 smithi178 ceph-mon[33359]: pgmap v609: 129 pgs: 129 active+clean; 66 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 1.3 MiB/s wr, 152 op/s 2024-05-10T11:24:42.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:24:41 smithi188 ceph-mon[34075]: pgmap v609: 129 pgs: 129 active+clean; 66 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 1.3 MiB/s wr, 152 op/s 2024-05-10T11:24:42.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:24:41 smithi100 ceph-mon[30296]: pgmap v609: 129 pgs: 129 active+clean; 66 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 1.3 MiB/s wr, 152 op/s 2024-05-10T11:24:42.972 INFO:tasks.workunit.client.0.smithi100.stdout: 1 144365 2.73 MB/sec execute 157 sec latency 14.992 ms 2024-05-10T11:24:43.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.433+0000 7f2efbdaa700 1 -- 172.21.15.100:0/1381864675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ef4108400 msgr2=0x7f2ef41087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:43.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.433+0000 7f2efbdaa700 1 --2- 172.21.15.100:0/1381864675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ef4108400 0x7f2ef41087e0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f2ef0009990 tx=0x7f2ef0009ca0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.433+0000 7f2efbdaa700 1 -- 172.21.15.100:0/1381864675 shutdown_connections 2024-05-10T11:24:43.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.433+0000 7f2efbdaa700 1 --2- 172.21.15.100:0/1381864675 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ef4102d50 0x7f2ef410f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.433+0000 7f2efbdaa700 1 --2- 172.21.15.100:0/1381864675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ef4108400 0x7f2ef41087e0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.433+0000 7f2efbdaa700 1 --2- 172.21.15.100:0/1381864675 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ef4102390 0x7f2ef4102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.433+0000 7f2efbdaa700 1 -- 172.21.15.100:0/1381864675 >> 172.21.15.100:0/1381864675 conn(0x7f2ef4079310 msgr2=0x7f2ef4079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:24:43.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.434+0000 7f2efbdaa700 1 -- 172.21.15.100:0/1381864675 shutdown_connections 2024-05-10T11:24:43.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.434+0000 7f2efbdaa700 1 -- 172.21.15.100:0/1381864675 wait complete. 2024-05-10T11:24:43.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.435+0000 7f2efbdaa700 1 Processor -- start 2024-05-10T11:24:43.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.435+0000 7f2efbdaa700 1 -- start start 2024-05-10T11:24:43.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.435+0000 7f2efbdaa700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ef4102390 0x7f2ef419a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:43.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.435+0000 7f2efbdaa700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ef4102d50 0x7f2ef419ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:43.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.435+0000 7f2efbdaa700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ef4108400 0x7f2ef41a5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:43.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.435+0000 7f2efbdaa700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef41123d0 con 0x7f2ef4108400 2024-05-10T11:24:43.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.435+0000 7f2efbdaa700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef4112250 con 0x7f2ef4102390 2024-05-10T11:24:43.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.435+0000 7f2efbdaa700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef4112550 con 0x7f2ef4102d50 2024-05-10T11:24:43.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.435+0000 7f2ef9345700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ef4102d50 0x7f2ef419ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:43.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.435+0000 7f2ef9345700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ef4102d50 0x7f2ef419ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:56202/0 (socket says 172.21.15.100:56202) 2024-05-10T11:24:43.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.435+0000 7f2ef9345700 1 -- 172.21.15.100:0/1442360257 learned_addr learned my addr 172.21.15.100:0/1442360257 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:24:43.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.435+0000 7f2ef9b46700 1 --2- 172.21.15.100:0/1442360257 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ef4102390 0x7f2ef419a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:43.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.435+0000 7f2efa347700 1 --2- 172.21.15.100:0/1442360257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ef4108400 0x7f2ef41a5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:43.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.436+0000 7f2ef9345700 1 -- 172.21.15.100:0/1442360257 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ef4102390 msgr2=0x7f2ef419a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:43.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.436+0000 7f2ef9345700 1 --2- 172.21.15.100:0/1442360257 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ef4102390 0x7f2ef419a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.436+0000 7f2ef9345700 1 -- 172.21.15.100:0/1442360257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ef4108400 msgr2=0x7f2ef41a5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:43.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.436+0000 7f2ef9345700 1 --2- 172.21.15.100:0/1442360257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ef4108400 0x7f2ef41a5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.436+0000 7f2ef9345700 1 -- 172.21.15.100:0/1442360257 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ef41a5e90 con 0x7f2ef4102d50 2024-05-10T11:24:43.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.436+0000 7f2ef9345700 1 --2- 172.21.15.100:0/1442360257 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ef4102d50 0x7f2ef419ae10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4012db0 tx=0x7f2ee400c5b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:24:43.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.436+0000 7f2eeaffd700 1 -- 172.21.15.100:0/1442360257 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee401f070 con 0x7f2ef4102d50 2024-05-10T11:24:43.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.436+0000 7f2eeaffd700 1 -- 172.21.15.100:0/1442360257 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2ee401a8c0 con 0x7f2ef4102d50 2024-05-10T11:24:43.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.436+0000 7f2eeaffd700 1 -- 172.21.15.100:0/1442360257 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee40187f0 con 0x7f2ef4102d50 2024-05-10T11:24:43.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.436+0000 7f2efbdaa700 1 -- 172.21.15.100:0/1442360257 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ef4198720 con 0x7f2ef4102d50 2024-05-10T11:24:43.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.436+0000 7f2efbdaa700 1 -- 172.21.15.100:0/1442360257 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ef4198c10 con 0x7f2ef4102d50 2024-05-10T11:24:43.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.439+0000 7f2eeaffd700 1 -- 172.21.15.100:0/1442360257 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f2ee4022070 con 0x7f2ef4102d50 2024-05-10T11:24:43.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.440+0000 7f2f0663db80 1 Processor -- start 2024-05-10T11:24:43.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.440+0000 7f2eeaffd700 1 --2- 172.21.15.100:0/1442360257 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2ee00b29d0 0x7f2ee00b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:43.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.440+0000 7f2efa347700 1 --2- 172.21.15.100:0/1442360257 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2ee00b29d0 0x7f2ee00b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:43.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.441+0000 7f2eeaffd700 1 -- 172.21.15.100:0/1442360257 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f2ee40e6090 con 0x7f2ef4102d50 2024-05-10T11:24:43.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.441+0000 7f2f0663db80 1 -- start start 2024-05-10T11:24:43.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.441+0000 7f2f0663db80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce96ce2230 0x55ce96ce2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:43.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.441+0000 7f2f0663db80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce96ce2b50 0x55ce96cdc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:43.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.441+0000 7f2f0663db80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce96cdcdd0 0x55ce96cdd210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:43.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.441+0000 7f2f0663db80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55ce96c2e9c0 con 0x55ce96ce2230 2024-05-10T11:24:43.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.441+0000 7f2f0663db80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ce96b93e20 con 0x55ce96cdcdd0 2024-05-10T11:24:43.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.441+0000 7f2f0663db80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55ce96c2a980 con 0x55ce96ce2b50 2024-05-10T11:24:43.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.441+0000 7f2ef9b46700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce96ce2230 0x55ce96ce2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:43.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.441+0000 7f2ef9b46700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce96ce2230 0x55ce96ce2610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40012/0 (socket says 172.21.15.100:40012) 2024-05-10T11:24:43.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.441+0000 7f2ef9b46700 1 -- 172.21.15.100:0/354043319 learned_addr learned my addr 172.21.15.100:0/354043319 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:24:43.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.441+0000 7f2efa347700 1 --2- 172.21.15.100:0/1442360257 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2ee00b29d0 0x7f2ee00b4e90 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2eec003e90 tx=0x7f2eec009100 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:24:43.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.441+0000 7f2ef9345700 1 --2- 172.21.15.100:0/354043319 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce96ce2b50 0x55ce96cdc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:43.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.441+0000 7f2efa347700 1 --2- 172.21.15.100:0/354043319 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce96cdcdd0 0x55ce96cdd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:43.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.442+0000 7f2ef9b46700 1 -- 172.21.15.100:0/354043319 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce96ce2b50 msgr2=0x55ce96cdc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:43.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.442+0000 7f2ef9b46700 1 --2- 172.21.15.100:0/354043319 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce96ce2b50 0x55ce96cdc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.442+0000 7f2ef9b46700 1 -- 172.21.15.100:0/354043319 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce96cdcdd0 msgr2=0x55ce96cdd210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:43.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.442+0000 7f2ef9b46700 1 --2- 172.21.15.100:0/354043319 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce96cdcdd0 0x55ce96cdd210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.442+0000 7f2ef9b46700 1 -- 172.21.15.100:0/354043319 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce96b7b950 con 0x55ce96ce2230 2024-05-10T11:24:43.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.442+0000 7f2ef9b46700 1 --2- 172.21.15.100:0/354043319 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce96ce2230 0x55ce96ce2610 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f2ef0015000 tx=0x7f2ef0005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6e5d1437887d767 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:24:43.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.442+0000 7f2edffff700 1 -- 172.21.15.100:0/354043319 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef0027070 con 0x55ce96ce2230 2024-05-10T11:24:43.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.442+0000 7f2edffff700 1 -- 172.21.15.100:0/354043319 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2ef00056a0 con 0x55ce96ce2230 2024-05-10T11:24:43.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.442+0000 7f2edffff700 1 -- 172.21.15.100:0/354043319 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef000bb50 con 0x55ce96ce2230 2024-05-10T11:24:43.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.443+0000 7f2f0663db80 1 -- 172.21.15.100:0/354043319 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce96ce2230 msgr2=0x55ce96ce2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:43.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.443+0000 7f2f0663db80 1 --2- 172.21.15.100:0/354043319 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce96ce2230 0x55ce96ce2610 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f2ef0015000 tx=0x7f2ef0005bc0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.443+0000 7f2f0663db80 1 -- 172.21.15.100:0/354043319 shutdown_connections 2024-05-10T11:24:43.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.443+0000 7f2f0663db80 1 --2- 172.21.15.100:0/354043319 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce96cdcdd0 0x55ce96cdd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.443+0000 7f2f0663db80 1 --2- 172.21.15.100:0/354043319 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce96ce2230 0x55ce96ce2610 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.443+0000 7f2f0663db80 1 --2- 172.21.15.100:0/354043319 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce96ce2b50 0x55ce96cdc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.443+0000 7f2f0663db80 1 -- 172.21.15.100:0/354043319 >> 172.21.15.100:0/354043319 conn(0x55ce96c08bf0 msgr2=0x55ce96cd7320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:24:43.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.443+0000 7f2f0663db80 1 -- 172.21.15.100:0/354043319 shutdown_connections 2024-05-10T11:24:43.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.443+0000 7f2f0663db80 1 -- 172.21.15.100:0/354043319 wait complete. 2024-05-10T11:24:43.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.444+0000 7f2f0663db80 1 -- start start 2024-05-10T11:24:43.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.444+0000 7f2f0663db80 1 Processor -- start 2024-05-10T11:24:43.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.444+0000 7f2f0663db80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce96cdcdd0 0x55ce96ce8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:43.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.444+0000 7f2f0663db80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce96ce2230 0x55ce96ce5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:43.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.444+0000 7f2f0663db80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce96ce2b50 0x55ce96ce6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:43.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.444+0000 7f2f0663db80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55ce96ce9470 con 0x55ce96cdcdd0 2024-05-10T11:24:43.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.444+0000 7f2f0663db80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ce96be7f50 con 0x55ce96ce2230 2024-05-10T11:24:43.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.444+0000 7f2f0663db80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55ce96ce95f0 con 0x55ce96ce2b50 2024-05-10T11:24:43.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.444+0000 7f2f0663db80 10 client.0 _ll_register_callbacks cb 0x55ce96bc2630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:24:43.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.444+0000 7f2ef9b46700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce96cdcdd0 0x55ce96ce8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:43.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.444+0000 7f2ef9b46700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce96cdcdd0 0x55ce96ce8720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40024/0 (socket says 172.21.15.100:40024) 2024-05-10T11:24:43.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.444+0000 7f2ef9b46700 1 -- 172.21.15.100:0/1315942096 learned_addr learned my addr 172.21.15.100:0/1315942096 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:24:43.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.445+0000 7f2efa347700 1 --2- 172.21.15.100:0/1315942096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce96ce2b50 0x55ce96ce6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:43.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.445+0000 7f2ef9345700 1 --2- 172.21.15.100:0/1315942096 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce96ce2230 0x55ce96ce5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:43.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.445+0000 7f2ef9b46700 1 -- 172.21.15.100:0/1315942096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce96ce2b50 msgr2=0x55ce96ce6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:43.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.445+0000 7f2ef9b46700 1 --2- 172.21.15.100:0/1315942096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce96ce2b50 0x55ce96ce6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.445+0000 7f2ef9b46700 1 -- 172.21.15.100:0/1315942096 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce96ce2230 msgr2=0x55ce96ce5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:43.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.445+0000 7f2ef9b46700 1 --2- 172.21.15.100:0/1315942096 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce96ce2230 0x55ce96ce5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.445+0000 7f2ef9b46700 1 -- 172.21.15.100:0/1315942096 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce96be57e0 con 0x55ce96cdcdd0 2024-05-10T11:24:43.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.445+0000 7f2efa347700 1 --2- 172.21.15.100:0/1315942096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce96ce2b50 0x55ce96ce6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:24:43.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.445+0000 7f2ef9345700 1 --2- 172.21.15.100:0/1315942096 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce96ce2230 0x55ce96ce5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:24:43.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.445+0000 7f2ef9b46700 1 --2- 172.21.15.100:0/1315942096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce96cdcdd0 0x55ce96ce8720 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f2ef000b2e0 tx=0x7f2ef000bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b189299201ee7d80 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:24:43.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.445+0000 7f2edd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:24:43.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.445+0000 7f2edd7fa700 1 -- 172.21.15.100:0/1315942096 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef0034060 con 0x55ce96cdcdd0 2024-05-10T11:24:43.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.445+0000 7f2edd7fa700 1 -- 172.21.15.100:0/1315942096 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f2ef0003890 con 0x55ce96cdcdd0 2024-05-10T11:24:43.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.445+0000 7f2f0663db80 1 -- 172.21.15.100:0/1315942096 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ef00056a0 con 0x55ce96cdcdd0 2024-05-10T11:24:43.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.446+0000 7f2edd7fa700 1 -- 172.21.15.100:0/1315942096 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2ef0027070 con 0x55ce96cdcdd0 2024-05-10T11:24:43.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.446+0000 7f2f0663db80 10 client.16644 fetch_fsmap learned FSMap version 31 2024-05-10T11:24:43.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.446+0000 7f2f0663db80 1 -- 172.21.15.100:0/1315942096 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55ce96cf3d10 con 0x55ce96cdcdd0 2024-05-10T11:24:43.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.446+0000 7f2edd7fa700 1 -- 172.21.15.100:0/1315942096 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2ef0028040 con 0x55ce96cdcdd0 2024-05-10T11:24:43.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.446+0000 7f2edd7fa700 1 -- 172.21.15.100:0/1315942096 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f2ef0021070 con 0x55ce96cdcdd0 2024-05-10T11:24:43.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.446+0000 7f2f0663db80 10 client.16644 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:24:43.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.446+0000 7f2f0663db80 10 client.16644 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:24:43.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.446+0000 7f2f0663db80 20 client.16644 populate_metadata read hostname 'smithi100' 2024-05-10T11:24:43.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.446+0000 7f2f0663db80 1 --2- 172.21.15.100:0/1315942096 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ce96cf61e0 0x55ce96cf85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:43.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.446+0000 7f2f0663db80 4 client.16644 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:24:43.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.446+0000 7f2f0663db80 1 -- 172.21.15.100:0/1315942096 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2ef00056a0 con 0x55ce96cf61e0 2024-05-10T11:24:43.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.447+0000 7f2ef9345700 1 --2- 172.21.15.100:0/1315942096 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ce96cf61e0 0x55ce96cf85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:43.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.447+0000 7f2edd7fa700 10 client.16644 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:24:43.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.447+0000 7f2ef9345700 1 --2- 172.21.15.100:0/1315942096 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ce96cf61e0 0x55ce96cf85d0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5d79d9045f7f985 server_cookie=56678e8d77e2fca1 in_seq=0 out_seq=0 2024-05-10T11:24:43.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.448+0000 7f2edd7fa700 1 -- 172.21.15.100:0/1315942096 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f2ef00056a0 con 0x55ce96cf61e0 2024-05-10T11:24:43.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.448+0000 7f2edd7fa700 10 client.16644 handle_command_reply: tid=0 2024-05-10T11:24:43.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.448+0000 7f2f0663db80 1 client.16644 shutdown 2024-05-10T11:24:43.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.448+0000 7f2f0663db80 1 -- 172.21.15.100:0/1315942096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce96cdcdd0 msgr2=0x55ce96ce8720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:43.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.448+0000 7f2f0663db80 1 --2- 172.21.15.100:0/1315942096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce96cdcdd0 0x55ce96ce8720 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f2ef000b2e0 tx=0x7f2ef000bdf0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.448+0000 7f2f0663db80 1 -- 172.21.15.100:0/1315942096 shutdown_connections 2024-05-10T11:24:43.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.448+0000 7f2f0663db80 1 --2- 172.21.15.100:0/1315942096 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ce96cf61e0 0x55ce96cf85d0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.448+0000 7f2f0663db80 1 --2- 172.21.15.100:0/1315942096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce96ce2b50 0x55ce96ce6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.448+0000 7f2f0663db80 1 --2- 172.21.15.100:0/1315942096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce96cdcdd0 0x55ce96ce8720 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.448+0000 7f2f0663db80 1 --2- 172.21.15.100:0/1315942096 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce96ce2230 0x55ce96ce5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.448+0000 7f2f0663db80 1 -- 172.21.15.100:0/1315942096 >> 172.21.15.100:0/1315942096 conn(0x55ce96c08bf0 msgr2=0x55ce96cd1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:24:43.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.448+0000 7f2edd7fa700 0 client.16644 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:24:43.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.448+0000 7f2f0663db80 1 -- 172.21.15.100:0/1315942096 shutdown_connections 2024-05-10T11:24:43.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.448+0000 7f2f0663db80 1 -- 172.21.15.100:0/1315942096 wait complete. 2024-05-10T11:24:43.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.448+0000 7f2f0663db80 20 client.16644 trim_cache size 0 max 16384 2024-05-10T11:24:43.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.484+0000 7f2f0663db80 1 Processor -- start 2024-05-10T11:24:43.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.484+0000 7f2f0663db80 1 -- start start 2024-05-10T11:24:43.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.484+0000 7f2f0663db80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce96cdcdd0 0x55ce96cd1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:43.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.484+0000 7f2f0663db80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce96ce2230 0x55ce96ce7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:43.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.484+0000 7f2f0663db80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce96ce2b50 0x55ce96cfb210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:43.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.484+0000 7f2f0663db80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef41af6e0 con 0x55ce96cdcdd0 2024-05-10T11:24:43.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.484+0000 7f2f0663db80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef41acf70 con 0x55ce96ce2b50 2024-05-10T11:24:43.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.484+0000 7f2f0663db80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef41aadc0 con 0x55ce96ce2230 2024-05-10T11:24:43.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.485+0000 7f2ef9b46700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce96cdcdd0 0x55ce96cd1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:43.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.485+0000 7f2ef9b46700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce96cdcdd0 0x55ce96cd1cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40030/0 (socket says 172.21.15.100:40030) 2024-05-10T11:24:43.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.485+0000 7f2ef9b46700 1 -- 172.21.15.100:0/1321170945 learned_addr learned my addr 172.21.15.100:0/1321170945 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:24:43.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.485+0000 7f2ef9345700 1 --2- 172.21.15.100:0/1321170945 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce96ce2230 0x55ce96ce7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:43.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.485+0000 7f2efa347700 1 --2- 172.21.15.100:0/1321170945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce96ce2b50 0x55ce96cfb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:43.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.485+0000 7f2ef9b46700 1 -- 172.21.15.100:0/1321170945 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce96ce2230 msgr2=0x55ce96ce7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:43.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.485+0000 7f2ef9b46700 1 --2- 172.21.15.100:0/1321170945 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce96ce2230 0x55ce96ce7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.485+0000 7f2ef9b46700 1 -- 172.21.15.100:0/1321170945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce96ce2b50 msgr2=0x55ce96cfb210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:43.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.485+0000 7f2ef9b46700 1 --2- 172.21.15.100:0/1321170945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce96ce2b50 0x55ce96cfb210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.485+0000 7f2ef9b46700 1 -- 172.21.15.100:0/1321170945 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce96cdd6d0 con 0x55ce96cdcdd0 2024-05-10T11:24:43.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.485+0000 7f2efa347700 1 --2- 172.21.15.100:0/1321170945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce96ce2b50 0x55ce96cfb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:24:43.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.485+0000 7f2ef9b46700 1 --2- 172.21.15.100:0/1321170945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce96cdcdd0 0x55ce96cd1cc0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f2ef000b650 tx=0x7f2ef0005ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88f5bc25d651eb5e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:24:43.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.485+0000 7f2edf7fe700 1 -- 172.21.15.100:0/1321170945 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef0034020 con 0x55ce96cdcdd0 2024-05-10T11:24:43.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.485+0000 7f2edf7fe700 1 -- 172.21.15.100:0/1321170945 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2ef0003d20 con 0x55ce96cdcdd0 2024-05-10T11:24:43.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.485+0000 7f2edf7fe700 1 -- 172.21.15.100:0/1321170945 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef002e920 con 0x55ce96cdcdd0 2024-05-10T11:24:43.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.486+0000 7f2f0663db80 1 -- 172.21.15.100:0/1321170945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce96cdcdd0 msgr2=0x55ce96cd1cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:43.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.486+0000 7f2f0663db80 1 --2- 172.21.15.100:0/1321170945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce96cdcdd0 0x55ce96cd1cc0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f2ef000b650 tx=0x7f2ef0005ee0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.486+0000 7f2f0663db80 1 -- 172.21.15.100:0/1321170945 shutdown_connections 2024-05-10T11:24:43.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.486+0000 7f2f0663db80 1 --2- 172.21.15.100:0/1321170945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce96ce2b50 0x55ce96cfb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.486+0000 7f2f0663db80 1 --2- 172.21.15.100:0/1321170945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce96cdcdd0 0x55ce96cd1cc0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.486+0000 7f2f0663db80 1 --2- 172.21.15.100:0/1321170945 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce96ce2230 0x55ce96ce7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.486+0000 7f2f0663db80 1 -- 172.21.15.100:0/1321170945 >> 172.21.15.100:0/1321170945 conn(0x55ce96c08bf0 msgr2=0x55ce96ce35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:24:43.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.486+0000 7f2f0663db80 1 -- 172.21.15.100:0/1321170945 shutdown_connections 2024-05-10T11:24:43.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.486+0000 7f2f0663db80 1 -- 172.21.15.100:0/1321170945 wait complete. 2024-05-10T11:24:43.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.487+0000 7f2f0663db80 1 -- start start 2024-05-10T11:24:43.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.487+0000 7f2f0663db80 1 Processor -- start 2024-05-10T11:24:43.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.488+0000 7f2f0663db80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce96cdcdd0 0x55ce96cf9990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:43.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.488+0000 7f2f0663db80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce96ce2230 0x55ce96cf9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:43.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.488+0000 7f2f0663db80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce96ce2b50 0x55ce96cfae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:43.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.488+0000 7f2ef9b46700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce96cdcdd0 0x55ce96cf9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:43.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.488+0000 7f2ef9b46700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce96cdcdd0 0x55ce96cf9990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:56236/0 (socket says 172.21.15.100:56236) 2024-05-10T11:24:43.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.488+0000 7f2ef9345700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce96ce2230 0x55ce96cf9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:43.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.488+0000 7f2ef9345700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce96ce2230 0x55ce96cf9ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:40432/0 (socket says 172.21.15.100:40432) 2024-05-10T11:24:43.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.488+0000 7f2efa347700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce96ce2b50 0x55ce96cfae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:43.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.488+0000 7f2efa347700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce96ce2b50 0x55ce96cfae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40036/0 (socket says 172.21.15.100:40036) 2024-05-10T11:24:43.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.488+0000 7f2ef9b46700 1 -- 172.21.15.100:0/3755030509 learned_addr learned my addr 172.21.15.100:0/3755030509 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:24:43.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.488+0000 7f2f0663db80 1 -- 172.21.15.100:0/3755030509 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef41ab430 con 0x55ce96ce2b50 2024-05-10T11:24:43.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.488+0000 7f2f0663db80 1 -- 172.21.15.100:0/3755030509 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef41ac8a0 con 0x55ce96ce2230 2024-05-10T11:24:43.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.488+0000 7f2f0663db80 1 -- 172.21.15.100:0/3755030509 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef41abb00 con 0x55ce96cdcdd0 2024-05-10T11:24:43.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.488+0000 7f2ef9b46700 1 -- 172.21.15.100:0/3755030509 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce96ce2230 msgr2=0x55ce96cf9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:43.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.488+0000 7f2ef9b46700 1 --2- 172.21.15.100:0/3755030509 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce96ce2230 0x55ce96cf9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.488+0000 7f2ef9b46700 1 -- 172.21.15.100:0/3755030509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce96ce2b50 msgr2=0x55ce96cfae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:43.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.488+0000 7f2ef9b46700 1 --2- 172.21.15.100:0/3755030509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce96ce2b50 0x55ce96cfae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.489+0000 7f2f0663db80 10 client.0 _ll_register_callbacks cb 0x55ce96bc2630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:24:43.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.489+0000 7f2ef9b46700 1 -- 172.21.15.100:0/3755030509 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce96cfb560 con 0x55ce96cdcdd0 2024-05-10T11:24:43.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.489+0000 7f2efa347700 1 --2- 172.21.15.100:0/3755030509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce96ce2b50 0x55ce96cfae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:24:43.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.489+0000 7f2f0663db80 1 -- 172.21.15.100:0/3755030509 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ef0003d20 con 0x55ce96cdcdd0 2024-05-10T11:24:43.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.489+0000 7f2ef9b46700 1 --2- 172.21.15.100:0/3755030509 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce96cdcdd0 0x55ce96cf9990 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2ef000be30 tx=0x7f2ef00056b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c52900c7f943ec8a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:24:43.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.489+0000 7f2edd7fa700 10 client.26128 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:24:43.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.489+0000 7f2edd7fa700 1 -- 172.21.15.100:0/3755030509 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef0034030 con 0x55ce96cdcdd0 2024-05-10T11:24:43.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.489+0000 7f2edd7fa700 1 -- 172.21.15.100:0/3755030509 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f2ef000f880 con 0x55ce96cdcdd0 2024-05-10T11:24:43.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.489+0000 7f2edd7fa700 1 -- 172.21.15.100:0/3755030509 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2ef000fac0 con 0x55ce96cdcdd0 2024-05-10T11:24:43.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.489+0000 7f2f0663db80 10 client.26128 fetch_fsmap learned FSMap version 31 2024-05-10T11:24:43.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.489+0000 7f2f0663db80 1 -- 172.21.15.100:0/3755030509 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({fsmap=31}) -- 0x55ce96d610e0 con 0x55ce96cdcdd0 2024-05-10T11:24:43.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.489+0000 7f2edd7fa700 1 -- 172.21.15.100:0/3755030509 <== mon.1 v2:172.21.15.188:3300/0 4 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f2ef0021070 con 0x55ce96cdcdd0 2024-05-10T11:24:43.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.490+0000 7f2f0663db80 10 client.26128 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:24:43.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.490+0000 7f2f0663db80 10 client.26128 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:24:43.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.490+0000 7f2f0663db80 20 client.26128 populate_metadata read hostname 'smithi100' 2024-05-10T11:24:43.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.490+0000 7f2f0663db80 1 --2- 172.21.15.100:0/3755030509 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ce96d61720 0x55ce96d63b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:43.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.490+0000 7f2f0663db80 4 client.26128 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T11:24:43.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.490+0000 7f2f0663db80 1 -- 172.21.15.100:0/3755030509 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ce96c3c3a0 con 0x55ce96d61720 2024-05-10T11:24:43.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.490+0000 7f2ef9345700 1 --2- 172.21.15.100:0/3755030509 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ce96d61720 0x55ce96d63b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:43.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.490+0000 7f2ef9345700 1 --2- 172.21.15.100:0/3755030509 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ce96d61720 0x55ce96d63b60 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec3c3f609ddc77d9 server_cookie=251fd3aa0486671 in_seq=0 out_seq=0 2024-05-10T11:24:43.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.490+0000 7f2edd7fa700 10 client.26128 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:24:43.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.491+0000 7f2edd7fa700 1 -- 172.21.15.100:0/3755030509 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ce96c3c3a0 con 0x55ce96d61720 2024-05-10T11:24:43.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.491+0000 7f2edd7fa700 10 client.26128 handle_command_reply: tid=0 2024-05-10T11:24:43.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.491+0000 7f2f0663db80 1 client.26128 shutdown 2024-05-10T11:24:43.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.492+0000 7f2f0663db80 1 -- 172.21.15.100:0/3755030509 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce96cdcdd0 msgr2=0x55ce96cf9990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:43.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.492+0000 7f2f0663db80 1 --2- 172.21.15.100:0/3755030509 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce96cdcdd0 0x55ce96cf9990 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2ef000be30 tx=0x7f2ef00056b0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.492+0000 7f2f0663db80 1 -- 172.21.15.100:0/3755030509 shutdown_connections 2024-05-10T11:24:43.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.492+0000 7f2f0663db80 1 --2- 172.21.15.100:0/3755030509 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ce96d61720 0x55ce96d63b60 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.492+0000 7f2f0663db80 1 --2- 172.21.15.100:0/3755030509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce96ce2b50 0x55ce96cfae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.492+0000 7f2f0663db80 1 --2- 172.21.15.100:0/3755030509 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce96cdcdd0 0x55ce96cf9990 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.492+0000 7f2f0663db80 1 --2- 172.21.15.100:0/3755030509 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce96ce2230 0x55ce96cf9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.492+0000 7f2f0663db80 1 -- 172.21.15.100:0/3755030509 >> 172.21.15.100:0/3755030509 conn(0x55ce96c08bf0 msgr2=0x55ce96ce1520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:24:43.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.493+0000 7f2edd7fa700 0 client.26128 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:24:43.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.493+0000 7f2f0663db80 1 -- 172.21.15.100:0/3755030509 shutdown_connections 2024-05-10T11:24:43.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.493+0000 7f2f0663db80 1 -- 172.21.15.100:0/3755030509 wait complete. 2024-05-10T11:24:43.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.493+0000 7f2f0663db80 20 client.26128 trim_cache size 0 max 16384 2024-05-10T11:24:43.497 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:24:43.497 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-05-10T11:24:43.498 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "51a6d3a5-affd-4d91-8070-9fc824841270", 2024-05-10T11:24:43.498 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-05-10T11:24:43.498 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:24:43.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.494+0000 7f2efbdaa700 1 -- 172.21.15.100:0/1442360257 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2ee00b29d0 msgr2=0x7f2ee00b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:43.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.494+0000 7f2efbdaa700 1 --2- 172.21.15.100:0/1442360257 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2ee00b29d0 0x7f2ee00b4e90 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2eec003e90 tx=0x7f2eec009100 comp rx=0 tx=0).stop 2024-05-10T11:24:43.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.494+0000 7f2efbdaa700 1 -- 172.21.15.100:0/1442360257 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ef4102d50 msgr2=0x7f2ef419ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:43.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.494+0000 7f2efbdaa700 1 --2- 172.21.15.100:0/1442360257 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ef4102d50 0x7f2ef419ae10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4012db0 tx=0x7f2ee400c5b0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.494+0000 7f2efbdaa700 1 -- 172.21.15.100:0/1442360257 shutdown_connections 2024-05-10T11:24:43.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.494+0000 7f2efbdaa700 1 --2- 172.21.15.100:0/1442360257 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2ee00b29d0 0x7f2ee00b4e90 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.494+0000 7f2efbdaa700 1 --2- 172.21.15.100:0/1442360257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ef4108400 0x7f2ef41a5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.494+0000 7f2efbdaa700 1 --2- 172.21.15.100:0/1442360257 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ef4102390 0x7f2ef419a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.494+0000 7f2efbdaa700 1 --2- 172.21.15.100:0/1442360257 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ef4102d50 0x7f2ef419ae10 secure :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4012db0 tx=0x7f2ee400c5b0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.494+0000 7f2efbdaa700 1 -- 172.21.15.100:0/1442360257 >> 172.21.15.100:0/1442360257 conn(0x7f2ef4079310 msgr2=0x7f2ef4101280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:24:43.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.494+0000 7f2efbdaa700 1 -- 172.21.15.100:0/1442360257 shutdown_connections 2024-05-10T11:24:43.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:43.494+0000 7f2efbdaa700 1 -- 172.21.15.100:0/1442360257 wait complete. 2024-05-10T11:24:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.793+0000 7f99e7e33640 1 -- 172.21.15.100:0/1470938181 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99e0108750 msgr2=0x7f99e010ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.793+0000 7f99e7e33640 1 --2- 172.21.15.100:0/1470938181 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99e0108750 0x7f99e010ab40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f99dc009860 tx=0x7f99dc02a360 comp rx=0 tx=0).stop 2024-05-10T11:24:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.793+0000 7f99e7e33640 1 -- 172.21.15.100:0/1470938181 shutdown_connections 2024-05-10T11:24:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.793+0000 7f99e7e33640 1 --2- 172.21.15.100:0/1470938181 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99e0108750 0x7f99e010ab40 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.793+0000 7f99e7e33640 1 --2- 172.21.15.100:0/1470938181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99e0105e20 0x7f99e0108210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.793+0000 7f99e7e33640 1 --2- 172.21.15.100:0/1470938181 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99e01034f0 0x7f99e01058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.793+0000 7f99e7e33640 1 -- 172.21.15.100:0/1470938181 >> 172.21.15.100:0/1470938181 conn(0x7f99e00fce80 msgr2=0x7f99e00ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:24:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.793+0000 7f99e7e33640 1 -- 172.21.15.100:0/1470938181 shutdown_connections 2024-05-10T11:24:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.793+0000 7f99e7e33640 1 -- 172.21.15.100:0/1470938181 wait complete. 2024-05-10T11:24:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.794+0000 7f99e7e33640 1 Processor -- start 2024-05-10T11:24:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.794+0000 7f99e7e33640 1 -- start start 2024-05-10T11:24:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.794+0000 7f99e7e33640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99e01034f0 0x7f99e0102e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.794+0000 7f99e7e33640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99e0105e20 0x7f99e01014a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.794+0000 7f99e7e33640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99e0108750 0x7f99e01019e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.794+0000 7f99e7e33640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f99e010e480 con 0x7f99e0108750 2024-05-10T11:24:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.794+0000 7f99e7e33640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f99e010e300 con 0x7f99e01034f0 2024-05-10T11:24:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.794+0000 7f99e7e33640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f99e010e600 con 0x7f99e0105e20 2024-05-10T11:24:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.794+0000 7f99e63d0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99e0108750 0x7f99e01019e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.794+0000 7f99e53ce640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99e0105e20 0x7f99e01014a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.794+0000 7f99e5bcf640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99e01034f0 0x7f99e0102e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.794+0000 7f99e63d0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99e0108750 0x7f99e01019e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40054/0 (socket says 172.21.15.100:40054) 2024-05-10T11:24:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.794+0000 7f99e5bcf640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99e01034f0 0x7f99e0102e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:40442/0 (socket says 172.21.15.100:40442) 2024-05-10T11:24:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.794+0000 7f99e53ce640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99e0105e20 0x7f99e01014a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:56256/0 (socket says 172.21.15.100:56256) 2024-05-10T11:24:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.794+0000 7f99e63d0640 1 -- 172.21.15.100:0/3543024114 learned_addr learned my addr 172.21.15.100:0/3543024114 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:24:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.794+0000 7f99e63d0640 1 -- 172.21.15.100:0/3543024114 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99e0105e20 msgr2=0x7f99e01014a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.794+0000 7f99e63d0640 1 --2- 172.21.15.100:0/3543024114 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99e0105e20 0x7f99e01014a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.794+0000 7f99e63d0640 1 -- 172.21.15.100:0/3543024114 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99e01034f0 msgr2=0x7f99e0102e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.794+0000 7f99e63d0640 1 --2- 172.21.15.100:0/3543024114 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99e01034f0 0x7f99e0102e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.794+0000 7f99e63d0640 1 -- 172.21.15.100:0/3543024114 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99e01022a0 con 0x7f99e0108750 2024-05-10T11:24:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.794+0000 7f99e5bcf640 1 --2- 172.21.15.100:0/3543024114 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99e01034f0 0x7f99e0102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:24:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.795+0000 7f99e63d0640 1 --2- 172.21.15.100:0/3543024114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99e0108750 0x7f99e01019e0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f99dc02a950 tx=0x7f99dc032d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:24:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.795+0000 7f99d6ffd640 1 -- 172.21.15.100:0/3543024114 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99dc037050 con 0x7f99e0108750 2024-05-10T11:24:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.795+0000 7f99e7e33640 1 -- 172.21.15.100:0/3543024114 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99e01a2c50 con 0x7f99e0108750 2024-05-10T11:24:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.795+0000 7f99d6ffd640 1 -- 172.21.15.100:0/3543024114 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f99dc038040 con 0x7f99e0108750 2024-05-10T11:24:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.795+0000 7f99d6ffd640 1 -- 172.21.15.100:0/3543024114 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99dc0046c0 con 0x7f99e0108750 2024-05-10T11:24:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.795+0000 7f99e7e33640 1 -- 172.21.15.100:0/3543024114 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99e01a3110 con 0x7f99e0108750 2024-05-10T11:24:43.800 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.798+0000 7f99d6ffd640 1 -- 172.21.15.100:0/3543024114 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f99dc004040 con 0x7f99e0108750 2024-05-10T11:24:43.800 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.798+0000 7f99eac2a740 1 Processor -- start 2024-05-10T11:24:43.800 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.798+0000 7f99eac2a740 1 -- start start 2024-05-10T11:24:43.800 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.798+0000 7f99eac2a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bf6c35db0 0x561bf6c361d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:43.800 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.798+0000 7f99eac2a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561bf6c367a0 0x561bf6c3b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:43.800 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.798+0000 7f99eac2a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561bf6c3b910 0x561bf6c3dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:43.800 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.798+0000 7f99eac2a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x561bf6b9a5d0 con 0x561bf6c35db0 2024-05-10T11:24:43.800 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.798+0000 7f99eac2a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561bf6b53890 con 0x561bf6c367a0 2024-05-10T11:24:43.800 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.798+0000 7f99eac2a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x561bf6b96590 con 0x561bf6c3b910 2024-05-10T11:24:43.800 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.798+0000 7f99e53ce640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bf6c35db0 0x561bf6c361d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:43.800 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.798+0000 7f99e53ce640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bf6c35db0 0x561bf6c361d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40060/0 (socket says 172.21.15.100:40060) 2024-05-10T11:24:43.800 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.798+0000 7f99e53ce640 1 -- 172.21.15.100:0/3269793400 learned_addr learned my addr 172.21.15.100:0/3269793400 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:24:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.798+0000 7f99e5bcf640 1 --2- 172.21.15.100:0/3269793400 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561bf6c367a0 0x561bf6c3b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.798+0000 7f99e53ce640 1 --2- 172.21.15.100:0/3269793400 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561bf6c3b910 0x561bf6c3dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.799+0000 7f99e5bcf640 1 -- 172.21.15.100:0/3269793400 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561bf6c3b910 msgr2=0x561bf6c3dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.799+0000 7f99e5bcf640 1 --2- 172.21.15.100:0/3269793400 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561bf6c3b910 0x561bf6c3dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.799+0000 7f99e5bcf640 1 -- 172.21.15.100:0/3269793400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bf6c35db0 msgr2=0x561bf6c361d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.799+0000 7f99e5bcf640 1 --2- 172.21.15.100:0/3269793400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bf6c35db0 0x561bf6c361d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.799+0000 7f99e5bcf640 1 -- 172.21.15.100:0/3269793400 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bf68ac210 con 0x561bf6c367a0 2024-05-10T11:24:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.799+0000 7f99d6ffd640 1 --2- 172.21.15.100:0/3543024114 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f99cc0b28a0 0x7f99cc0b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.799+0000 7f99e53ce640 1 --2- 172.21.15.100:0/3269793400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bf6c35db0 0x561bf6c361d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:24:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.799+0000 7f99e53ce640 1 --2- 172.21.15.100:0/3543024114 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f99cc0b28a0 0x7f99cc0b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.799+0000 7f99e5bcf640 1 --2- 172.21.15.100:0/3269793400 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561bf6c367a0 0x561bf6c3b2a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f99d800a810 tx=0x7f99d80096d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b818c8b365e0a86a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:24:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.799+0000 7f99d6ffd640 1 -- 172.21.15.100:0/3543024114 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f99dc102990 con 0x7f99e0108750 2024-05-10T11:24:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.799+0000 7f99cbfff640 1 -- 172.21.15.100:0/3269793400 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d8018020 con 0x561bf6c367a0 2024-05-10T11:24:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.799+0000 7f99cbfff640 1 -- 172.21.15.100:0/3269793400 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f99d8014400 con 0x561bf6c367a0 2024-05-10T11:24:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.799+0000 7f99cbfff640 1 -- 172.21.15.100:0/3269793400 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d80135c0 con 0x561bf6c367a0 2024-05-10T11:24:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.800+0000 7f99eac2a740 1 -- 172.21.15.100:0/3269793400 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561bf6c367a0 msgr2=0x561bf6c3b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.800+0000 7f99e53ce640 1 --2- 172.21.15.100:0/3543024114 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f99cc0b28a0 0x7f99cc0b4d60 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x561bf6c37000 tx=0x7f99d0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:24:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.800+0000 7f99eac2a740 1 --2- 172.21.15.100:0/3269793400 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561bf6c367a0 0x561bf6c3b2a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f99d800a810 tx=0x7f99d80096d0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.800+0000 7f99eac2a740 1 -- 172.21.15.100:0/3269793400 shutdown_connections 2024-05-10T11:24:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.800+0000 7f99eac2a740 1 --2- 172.21.15.100:0/3269793400 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561bf6c3b910 0x561bf6c3dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.800+0000 7f99eac2a740 1 --2- 172.21.15.100:0/3269793400 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561bf6c367a0 0x561bf6c3b2a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.800+0000 7f99eac2a740 1 --2- 172.21.15.100:0/3269793400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bf6c35db0 0x561bf6c361d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.800+0000 7f99eac2a740 1 -- 172.21.15.100:0/3269793400 >> 172.21.15.100:0/3269793400 conn(0x561bf6c31420 msgr2=0x561bf6c31820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:24:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.801+0000 7f99eac2a740 1 -- 172.21.15.100:0/3269793400 shutdown_connections 2024-05-10T11:24:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.801+0000 7f99eac2a740 1 -- 172.21.15.100:0/3269793400 wait complete. 2024-05-10T11:24:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.801+0000 7f99eac2a740 1 -- start start 2024-05-10T11:24:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.801+0000 7f99eac2a740 1 Processor -- start 2024-05-10T11:24:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.802+0000 7f99eac2a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bf6c35db0 0x561bf6c40080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.802+0000 7f99eac2a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561bf6c367a0 0x561bf6c405c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.802+0000 7f99eac2a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561bf6c3b910 0x561bf6c47590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.802+0000 7f99eac2a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x561bf6c3e8c0 con 0x561bf6c35db0 2024-05-10T11:24:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.802+0000 7f99eac2a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561bf6b57370 con 0x561bf6c367a0 2024-05-10T11:24:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.802+0000 7f99eac2a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x561bf6c3ea40 con 0x561bf6c3b910 2024-05-10T11:24:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.802+0000 7f99eac2a740 10 client.0 _ll_register_callbacks cb 0x561bf68aba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:24:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.802+0000 7f99e5bcf640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bf6c35db0 0x561bf6c40080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.802+0000 7f99e5bcf640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bf6c35db0 0x561bf6c40080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40074/0 (socket says 172.21.15.100:40074) 2024-05-10T11:24:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.802+0000 7f99e5bcf640 1 -- 172.21.15.100:0/2687312072 learned_addr learned my addr 172.21.15.100:0/2687312072 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:24:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.802+0000 7f99e63d0640 1 --2- 172.21.15.100:0/2687312072 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561bf6c3b910 0x561bf6c47590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.802+0000 7f99e53ce640 1 --2- 172.21.15.100:0/2687312072 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561bf6c367a0 0x561bf6c405c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.802+0000 7f99e5bcf640 1 -- 172.21.15.100:0/2687312072 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561bf6c3b910 msgr2=0x561bf6c47590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.802+0000 7f99e5bcf640 1 --2- 172.21.15.100:0/2687312072 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561bf6c3b910 0x561bf6c47590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.802+0000 7f99e5bcf640 1 -- 172.21.15.100:0/2687312072 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561bf6c367a0 msgr2=0x561bf6c405c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.802+0000 7f99e5bcf640 1 --2- 172.21.15.100:0/2687312072 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561bf6c367a0 0x561bf6c405c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.802+0000 7f99e5bcf640 1 -- 172.21.15.100:0/2687312072 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bf6978620 con 0x561bf6c35db0 2024-05-10T11:24:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.802+0000 7f99e5bcf640 1 --2- 172.21.15.100:0/2687312072 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bf6c35db0 0x561bf6c40080 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f99d8019000 tx=0x7f99d800a440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75c52b1ff011984c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:24:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.802+0000 7f99c97fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:24:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.803+0000 7f99c97fa640 1 -- 172.21.15.100:0/2687312072 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d8018040 con 0x561bf6c35db0 2024-05-10T11:24:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.803+0000 7f99eac2a740 1 -- 172.21.15.100:0/2687312072 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99d8014400 con 0x561bf6c35db0 2024-05-10T11:24:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.803+0000 7f99c97fa640 1 -- 172.21.15.100:0/2687312072 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f99d800ecc0 con 0x561bf6c35db0 2024-05-10T11:24:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.803+0000 7f99c97fa640 1 -- 172.21.15.100:0/2687312072 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f99d8031050 con 0x561bf6c35db0 2024-05-10T11:24:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.803+0000 7f99eac2a740 10 client.16662 fetch_fsmap learned FSMap version 31 2024-05-10T11:24:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.803+0000 7f99eac2a740 1 -- 172.21.15.100:0/2687312072 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561bf6c4a5c0 con 0x561bf6c35db0 2024-05-10T11:24:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.803+0000 7f99c97fa640 1 -- 172.21.15.100:0/2687312072 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f99d8014400 con 0x561bf6c35db0 2024-05-10T11:24:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.803+0000 7f99c97fa640 1 -- 172.21.15.100:0/2687312072 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f99d801d420 con 0x561bf6c35db0 2024-05-10T11:24:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.803+0000 7f99eac2a740 10 client.16662 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:24:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.803+0000 7f99eac2a740 10 client.16662 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:24:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.803+0000 7f99eac2a740 20 client.16662 populate_metadata read hostname 'smithi100' 2024-05-10T11:24:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.803+0000 7f99eac2a740 1 --2- 172.21.15.100:0/2687312072 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x561bf6c4bfe0 0x561bf6c4e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.803+0000 7f99eac2a740 4 client.16662 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:24:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.803+0000 7f99eac2a740 1 -- 172.21.15.100:0/2687312072 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561bf6ba81e0 con 0x561bf6c4bfe0 2024-05-10T11:24:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.804+0000 7f99e53ce640 1 --2- 172.21.15.100:0/2687312072 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x561bf6c4bfe0 0x561bf6c4e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.804+0000 7f99c97fa640 10 client.16662 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:24:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.804+0000 7f99e53ce640 1 --2- 172.21.15.100:0/2687312072 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x561bf6c4bfe0 0x561bf6c4e480 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34a0a245697931df server_cookie=b4911d43f3ef0810 in_seq=0 out_seq=0 2024-05-10T11:24:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.805+0000 7f99c97fa640 1 -- 172.21.15.100:0/2687312072 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561bf6ba81e0 con 0x561bf6c4bfe0 2024-05-10T11:24:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.805+0000 7f99c97fa640 10 client.16662 handle_command_reply: tid=0 2024-05-10T11:24:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.805+0000 7f99eac2a740 1 client.16662 shutdown 2024-05-10T11:24:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.806+0000 7f99eac2a740 1 -- 172.21.15.100:0/2687312072 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bf6c35db0 msgr2=0x561bf6c40080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.806+0000 7f99eac2a740 1 --2- 172.21.15.100:0/2687312072 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bf6c35db0 0x561bf6c40080 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f99d8019000 tx=0x7f99d800a440 comp rx=0 tx=0).stop 2024-05-10T11:24:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.806+0000 7f99eac2a740 1 -- 172.21.15.100:0/2687312072 shutdown_connections 2024-05-10T11:24:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.806+0000 7f99eac2a740 1 --2- 172.21.15.100:0/2687312072 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x561bf6c4bfe0 0x561bf6c4e480 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.806+0000 7f99eac2a740 1 --2- 172.21.15.100:0/2687312072 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561bf6c3b910 0x561bf6c47590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.806+0000 7f99eac2a740 1 --2- 172.21.15.100:0/2687312072 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561bf6c367a0 0x561bf6c405c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.806+0000 7f99c97fa640 0 client.16662 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:24:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.806+0000 7f99eac2a740 1 --2- 172.21.15.100:0/2687312072 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bf6c35db0 0x561bf6c40080 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.806+0000 7f99eac2a740 1 -- 172.21.15.100:0/2687312072 >> 172.21.15.100:0/2687312072 conn(0x561bf6c31420 msgr2=0x561bf6c30580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:24:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.806+0000 7f99eac2a740 1 -- 172.21.15.100:0/2687312072 shutdown_connections 2024-05-10T11:24:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.806+0000 7f99eac2a740 1 -- 172.21.15.100:0/2687312072 wait complete. 2024-05-10T11:24:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.806+0000 7f99eac2a740 20 client.16662 trim_cache size 0 max 16384 2024-05-10T11:24:43.839 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.837+0000 7f99eac2a740 1 Processor -- start 2024-05-10T11:24:43.839 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.837+0000 7f99eac2a740 1 -- start start 2024-05-10T11:24:43.839 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.837+0000 7f99eac2a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561bf6c35db0 0x561bf6c40360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:43.839 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.837+0000 7f99eac2a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561bf6c367a0 0x561bf6c30400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:43.839 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.837+0000 7f99eac2a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bf6c3b910 0x561bf6c30940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:43.839 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.837+0000 7f99eac2a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f99e01afe60 con 0x561bf6c3b910 2024-05-10T11:24:43.839 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.837+0000 7f99eac2a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f99e01ad730 con 0x561bf6c367a0 2024-05-10T11:24:43.839 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.837+0000 7f99eac2a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f99e01ab580 con 0x561bf6c35db0 2024-05-10T11:24:43.839 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.838+0000 7f99e63d0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bf6c3b910 0x561bf6c30940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:43.839 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.838+0000 7f99e5bcf640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561bf6c35db0 0x561bf6c40360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:43.839 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.838+0000 7f99e5bcf640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561bf6c35db0 0x561bf6c40360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:56284/0 (socket says 172.21.15.100:56284) 2024-05-10T11:24:43.839 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.838+0000 7f99e5bcf640 1 -- 172.21.15.100:0/1434696247 learned_addr learned my addr 172.21.15.100:0/1434696247 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:24:43.839 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.838+0000 7f99e53ce640 1 --2- 172.21.15.100:0/1434696247 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561bf6c367a0 0x561bf6c30400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.838+0000 7f99e5bcf640 1 -- 172.21.15.100:0/1434696247 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561bf6c367a0 msgr2=0x561bf6c30400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.838+0000 7f99e5bcf640 1 --2- 172.21.15.100:0/1434696247 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561bf6c367a0 0x561bf6c30400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.838+0000 7f99e5bcf640 1 -- 172.21.15.100:0/1434696247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bf6c3b910 msgr2=0x561bf6c30940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.838+0000 7f99e5bcf640 1 --2- 172.21.15.100:0/1434696247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bf6c3b910 0x561bf6c30940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.838+0000 7f99e5bcf640 1 -- 172.21.15.100:0/1434696247 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bf6c4eb00 con 0x561bf6c35db0 2024-05-10T11:24:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.838+0000 7f99e63d0640 1 --2- 172.21.15.100:0/1434696247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bf6c3b910 0x561bf6c30940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:24:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.838+0000 7f99e5bcf640 1 --2- 172.21.15.100:0/1434696247 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561bf6c35db0 0x561bf6c40360 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f99d801d8c0 tx=0x7f99d8014900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb5d83e7238ab6dd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:24:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.838+0000 7f99cb7fe640 1 -- 172.21.15.100:0/1434696247 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d8018070 con 0x561bf6c35db0 2024-05-10T11:24:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.838+0000 7f99cb7fe640 1 -- 172.21.15.100:0/1434696247 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f99d8014400 con 0x561bf6c35db0 2024-05-10T11:24:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.838+0000 7f99cb7fe640 1 -- 172.21.15.100:0/1434696247 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d8003a10 con 0x561bf6c35db0 2024-05-10T11:24:43.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.839+0000 7f99eac2a740 1 -- 172.21.15.100:0/1434696247 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561bf6c35db0 msgr2=0x561bf6c40360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:43.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.839+0000 7f99eac2a740 1 --2- 172.21.15.100:0/1434696247 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561bf6c35db0 0x561bf6c40360 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f99d801d8c0 tx=0x7f99d8014900 comp rx=0 tx=0).stop 2024-05-10T11:24:43.844 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.839+0000 7f99eac2a740 1 -- 172.21.15.100:0/1434696247 shutdown_connections 2024-05-10T11:24:43.844 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.839+0000 7f99eac2a740 1 --2- 172.21.15.100:0/1434696247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bf6c3b910 0x561bf6c30940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.844 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.839+0000 7f99eac2a740 1 --2- 172.21.15.100:0/1434696247 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561bf6c367a0 0x561bf6c30400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.844 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.839+0000 7f99eac2a740 1 --2- 172.21.15.100:0/1434696247 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561bf6c35db0 0x561bf6c40360 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.844 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.839+0000 7f99eac2a740 1 -- 172.21.15.100:0/1434696247 >> 172.21.15.100:0/1434696247 conn(0x561bf6c31420 msgr2=0x561bf6c2fcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:24:43.844 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.839+0000 7f99eac2a740 1 -- 172.21.15.100:0/1434696247 shutdown_connections 2024-05-10T11:24:43.844 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.840+0000 7f99eac2a740 1 -- 172.21.15.100:0/1434696247 wait complete. 2024-05-10T11:24:43.844 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.840+0000 7f99eac2a740 1 -- start start 2024-05-10T11:24:43.844 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.840+0000 7f99eac2a740 1 Processor -- start 2024-05-10T11:24:43.844 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.840+0000 7f99eac2a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561bf6c35db0 0x561bf6c43f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:43.844 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.840+0000 7f99eac2a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561bf6c367a0 0x561bf6c444c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:43.844 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.840+0000 7f99eac2a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bf6c3b910 0x561bf6c45480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:43.844 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.840+0000 7f99eac2a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f99e01abbf0 con 0x561bf6c3b910 2024-05-10T11:24:43.844 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.840+0000 7f99eac2a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f99e01ad060 con 0x561bf6c35db0 2024-05-10T11:24:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.840+0000 7f99eac2a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f99e01ac2c0 con 0x561bf6c367a0 2024-05-10T11:24:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.840+0000 7f99eac2a740 10 client.0 _ll_register_callbacks cb 0x7f99d0009800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:24:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.841+0000 7f99e53ce640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561bf6c367a0 0x561bf6c444c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.841+0000 7f99e53ce640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561bf6c367a0 0x561bf6c444c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:56288/0 (socket says 172.21.15.100:56288) 2024-05-10T11:24:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.841+0000 7f99e53ce640 1 -- 172.21.15.100:0/3028782495 learned_addr learned my addr 172.21.15.100:0/3028782495 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:24:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.841+0000 7f99e63d0640 1 --2- 172.21.15.100:0/3028782495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bf6c3b910 0x561bf6c45480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.841+0000 7f99e53ce640 1 -- 172.21.15.100:0/3028782495 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561bf6c35db0 msgr2=0x561bf6c43f80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T11:24:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.841+0000 7f99e53ce640 1 --2- 172.21.15.100:0/3028782495 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561bf6c35db0 0x561bf6c43f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.841+0000 7f99e53ce640 1 -- 172.21.15.100:0/3028782495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bf6c3b910 msgr2=0x561bf6c45480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.841+0000 7f99e53ce640 1 --2- 172.21.15.100:0/3028782495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bf6c3b910 0x561bf6c45480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.841+0000 7f99e53ce640 1 -- 172.21.15.100:0/3028782495 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bf6c31250 con 0x561bf6c367a0 2024-05-10T11:24:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.841+0000 7f99e53ce640 1 --2- 172.21.15.100:0/3028782495 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561bf6c367a0 0x561bf6c444c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f99d00097d0 tx=0x7f99d0009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f80d35a876611de server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:24:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.841+0000 7f99c97fa640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:24:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.842+0000 7f99c97fa640 1 -- 172.21.15.100:0/3028782495 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d0031070 con 0x561bf6c367a0 2024-05-10T11:24:43.846 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.842+0000 7f99c97fa640 1 -- 172.21.15.100:0/3028782495 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f99d002d620 con 0x561bf6c367a0 2024-05-10T11:24:43.846 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.842+0000 7f99eac2a740 1 -- 172.21.15.100:0/3028782495 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99d8014400 con 0x561bf6c367a0 2024-05-10T11:24:43.846 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.842+0000 7f99c97fa640 1 -- 172.21.15.100:0/3028782495 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f99d003b050 con 0x561bf6c367a0 2024-05-10T11:24:43.846 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.842+0000 7f99eac2a740 10 client.26143 fetch_fsmap learned FSMap version 31 2024-05-10T11:24:43.846 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.842+0000 7f99eac2a740 1 -- 172.21.15.100:0/3028782495 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561bf6c46c50 con 0x561bf6c367a0 2024-05-10T11:24:43.846 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.842+0000 7f99c97fa640 1 -- 172.21.15.100:0/3028782495 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f99d8014400 con 0x561bf6c367a0 2024-05-10T11:24:43.846 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.842+0000 7f99c97fa640 1 -- 172.21.15.100:0/3028782495 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f99d0003b10 con 0x561bf6c367a0 2024-05-10T11:24:43.846 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.842+0000 7f99eac2a740 10 client.26143 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:24:43.846 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.842+0000 7f99eac2a740 10 client.26143 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:24:43.846 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.843+0000 7f99eac2a740 20 client.26143 populate_metadata read hostname 'smithi100' 2024-05-10T11:24:43.846 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.843+0000 7f99eac2a740 1 --2- 172.21.15.100:0/3028782495 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x561bf6cbf6b0 0x561bf6cc1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:43.846 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.843+0000 7f99eac2a740 4 client.26143 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:24:43.846 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.843+0000 7f99eac2a740 1 -- 172.21.15.100:0/3028782495 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561bf6c3b610 con 0x561bf6cbf6b0 2024-05-10T11:24:43.846 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.843+0000 7f99e5bcf640 1 --2- 172.21.15.100:0/3028782495 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x561bf6cbf6b0 0x561bf6cc1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:43.847 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.843+0000 7f99c97fa640 10 client.26143 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:24:43.847 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.843+0000 7f99e5bcf640 1 --2- 172.21.15.100:0/3028782495 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x561bf6cbf6b0 0x561bf6cc1ad0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e45584e3c3c12e5 server_cookie=c596234571761dc1 in_seq=0 out_seq=0 2024-05-10T11:24:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.860+0000 7f99c97fa640 1 -- 172.21.15.100:0/3028782495 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561bf6c3b610 con 0x561bf6cbf6b0 2024-05-10T11:24:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.860+0000 7f99c97fa640 10 client.26143 handle_command_reply: tid=0 2024-05-10T11:24:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.860+0000 7f99eac2a740 1 client.26143 shutdown 2024-05-10T11:24:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.860+0000 7f99eac2a740 1 -- 172.21.15.100:0/3028782495 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561bf6c367a0 msgr2=0x561bf6c444c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.860+0000 7f99eac2a740 1 --2- 172.21.15.100:0/3028782495 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561bf6c367a0 0x561bf6c444c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f99d00097d0 tx=0x7f99d0009260 comp rx=0 tx=0).stop 2024-05-10T11:24:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.860+0000 7f99eac2a740 1 -- 172.21.15.100:0/3028782495 shutdown_connections 2024-05-10T11:24:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.860+0000 7f99eac2a740 1 --2- 172.21.15.100:0/3028782495 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x561bf6cbf6b0 0x561bf6cc1ad0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.860+0000 7f99eac2a740 1 --2- 172.21.15.100:0/3028782495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bf6c3b910 0x561bf6c45480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.860+0000 7f99eac2a740 1 --2- 172.21.15.100:0/3028782495 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561bf6c367a0 0x561bf6c444c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.860+0000 7f99c97fa640 0 client.26143 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:24:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.860+0000 7f99eac2a740 1 --2- 172.21.15.100:0/3028782495 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561bf6c35db0 0x561bf6c43f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.860+0000 7f99eac2a740 1 -- 172.21.15.100:0/3028782495 >> 172.21.15.100:0/3028782495 conn(0x561bf6c31420 msgr2=0x561bf6c30580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:24:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.860+0000 7f99eac2a740 1 -- 172.21.15.100:0/3028782495 shutdown_connections 2024-05-10T11:24:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.860+0000 7f99eac2a740 1 -- 172.21.15.100:0/3028782495 wait complete. 2024-05-10T11:24:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.860+0000 7f99eac2a740 20 client.26143 trim_cache size 0 max 16384 2024-05-10T11:24:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:24:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:24:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:24:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:24:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.861+0000 7f99e7e33640 1 -- 172.21.15.100:0/3543024114 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f99cc0b28a0 msgr2=0x7f99cc0b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.861+0000 7f99e7e33640 1 --2- 172.21.15.100:0/3543024114 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f99cc0b28a0 0x7f99cc0b4d60 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x561bf6c37000 tx=0x7f99d0005850 comp rx=0 tx=0).stop 2024-05-10T11:24:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.861+0000 7f99e7e33640 1 -- 172.21.15.100:0/3543024114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99e0108750 msgr2=0x7f99e01019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.861+0000 7f99e7e33640 1 --2- 172.21.15.100:0/3543024114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99e0108750 0x7f99e01019e0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f99dc02a950 tx=0x7f99dc032d30 comp rx=0 tx=0).stop 2024-05-10T11:24:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.861+0000 7f99e7e33640 1 -- 172.21.15.100:0/3543024114 shutdown_connections 2024-05-10T11:24:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.861+0000 7f99e7e33640 1 --2- 172.21.15.100:0/3543024114 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f99cc0b28a0 0x7f99cc0b4d60 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.861+0000 7f99e7e33640 1 --2- 172.21.15.100:0/3543024114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99e0108750 0x7f99e01019e0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.861+0000 7f99e7e33640 1 --2- 172.21.15.100:0/3543024114 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99e0105e20 0x7f99e01014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.861+0000 7f99e7e33640 1 --2- 172.21.15.100:0/3543024114 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99e01034f0 0x7f99e0102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.861+0000 7f99e7e33640 1 -- 172.21.15.100:0/3543024114 >> 172.21.15.100:0/3543024114 conn(0x7f99e00fce80 msgr2=0x7f99e0103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:24:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.861+0000 7f99e7e33640 1 -- 172.21.15.100:0/3543024114 shutdown_connections 2024-05-10T11:24:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:43.861+0000 7f99e7e33640 1 -- 172.21.15.100:0/3543024114 wait complete. 2024-05-10T11:24:43.972 INFO:tasks.workunit.client.0.smithi100.stdout: 1 145083 2.75 MB/sec execute 158 sec latency 49.656 ms 2024-05-10T11:24:44.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:24:43 smithi178 ceph-mon[33359]: pgmap v610: 129 pgs: 129 active+clean; 66 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 1.2 MiB/s wr, 130 op/s 2024-05-10T11:24:44.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:24:43 smithi178 ceph-mon[33359]: scrub queued for path: / 2024-05-10T11:24:44.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:24:43 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:24:44.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:24:43 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:24:44.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:24:43 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:24:44.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:24:43 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:24:44.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:24:43 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:24:44.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:24:43 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:24:44.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:24:43 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:24:44.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:24:43 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:24:44.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:24:43 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:24:44.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:24:43 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:24:44.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:24:43 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:24:44.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:24:43 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:24:44.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:24:43 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:24:44.260 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:24:43 smithi100 ceph-mon[30296]: pgmap v610: 129 pgs: 129 active+clean; 66 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 1.2 MiB/s wr, 130 op/s 2024-05-10T11:24:44.260 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:24:43 smithi100 ceph-mon[30296]: scrub queued for path: / 2024-05-10T11:24:44.261 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:24:43 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:24:44.261 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:24:43 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:24:44.261 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:24:43 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:24:44.261 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:24:43 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:24:44.261 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:24:43 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:24:44.261 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:24:43 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:24:44.261 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:24:43 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:24:44.261 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:24:43 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:24:44.261 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:24:43 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:24:44.261 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:24:43 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:24:44.261 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:24:43 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:24:44.261 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:24:43 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:24:44.261 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:24:43 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:24:44.262 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T11:24:44.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:24:43 smithi188 ceph-mon[34075]: pgmap v610: 129 pgs: 129 active+clean; 66 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 1.2 MiB/s wr, 130 op/s 2024-05-10T11:24:44.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:24:43 smithi188 ceph-mon[34075]: scrub queued for path: / 2024-05-10T11:24:44.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:24:43 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:24:44.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:24:43 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:24:44.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:24:43 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:24:44.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:24:43 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:24:44.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:24:43 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:24:44.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:24:43 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:24:44.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:24:43 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:24:44.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:24:43 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:24:44.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:24:43 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:24:44.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:24:43 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:24:44.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:24:43 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:24:44.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:24:43 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:24:44.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:24:43 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:24:44.538 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:24:44.973 INFO:tasks.workunit.client.0.smithi100.stdout: 1 145488 2.76 MB/sec execute 159 sec latency 17.015 ms 2024-05-10T11:24:45.972 INFO:tasks.workunit.client.0.smithi100.stdout: 1 145845 2.75 MB/sec execute 160 sec latency 16.219 ms 2024-05-10T11:24:46.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:24:45 smithi178 ceph-mon[33359]: pgmap v611: 129 pgs: 129 active+clean; 67 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 1.4 MiB/s wr, 167 op/s 2024-05-10T11:24:46.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:24:45 smithi188 ceph-mon[34075]: pgmap v611: 129 pgs: 129 active+clean; 67 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 1.4 MiB/s wr, 167 op/s 2024-05-10T11:24:46.279 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:24:45 smithi100 ceph-mon[30296]: pgmap v611: 129 pgs: 129 active+clean; 67 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 1.4 MiB/s wr, 167 op/s 2024-05-10T11:24:46.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.305+0000 7fcb081dc700 1 -- 172.21.15.100:0/3526262388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb001037b0 msgr2=0x7fcb00107030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:46.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.305+0000 7fcb081dc700 1 --2- 172.21.15.100:0/3526262388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb001037b0 0x7fcb00107030 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fcaf00099c0 tx=0x7fcaf0009cd0 comp rx=0 tx=0).stop 2024-05-10T11:24:46.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.305+0000 7fcb081dc700 1 -- 172.21.15.100:0/3526262388 shutdown_connections 2024-05-10T11:24:46.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.305+0000 7fcb081dc700 1 --2- 172.21.15.100:0/3526262388 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb00107760 0x7fcb0010f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:46.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.305+0000 7fcb081dc700 1 --2- 172.21.15.100:0/3526262388 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb00102e00 0x7fcb001031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:46.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.305+0000 7fcb081dc700 1 --2- 172.21.15.100:0/3526262388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb001037b0 0x7fcb00107030 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:46.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.305+0000 7fcb081dc700 1 -- 172.21.15.100:0/3526262388 >> 172.21.15.100:0/3526262388 conn(0x7fcb00078a10 msgr2=0x7fcb000ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:24:46.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.305+0000 7fcb081dc700 1 -- 172.21.15.100:0/3526262388 shutdown_connections 2024-05-10T11:24:46.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.305+0000 7fcb081dc700 1 -- 172.21.15.100:0/3526262388 wait complete. 2024-05-10T11:24:46.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.306+0000 7fcb081dc700 1 Processor -- start 2024-05-10T11:24:46.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.306+0000 7fcb081dc700 1 -- start start 2024-05-10T11:24:46.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.306+0000 7fcb081dc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb00102e00 0x7fcb0019a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:46.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.306+0000 7fcb081dc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb001037b0 0x7fcb0019ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:46.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.306+0000 7fcb081dc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb00107760 0x7fcb001a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:46.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.306+0000 7fcb081dc700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fcb001123f0 con 0x7fcb001037b0 2024-05-10T11:24:46.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.306+0000 7fcb081dc700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcb00112270 con 0x7fcb00102e00 2024-05-10T11:24:46.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.306+0000 7fcb081dc700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcb00112570 con 0x7fcb00107760 2024-05-10T11:24:46.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.306+0000 7fcb05777700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb001037b0 0x7fcb0019ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:46.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.306+0000 7fcb05777700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb001037b0 0x7fcb0019ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40120/0 (socket says 172.21.15.100:40120) 2024-05-10T11:24:46.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.306+0000 7fcb05777700 1 -- 172.21.15.100:0/1062762686 learned_addr learned my addr 172.21.15.100:0/1062762686 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:24:46.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.306+0000 7fcb05f78700 1 --2- 172.21.15.100:0/1062762686 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb00102e00 0x7fcb0019a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:46.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.306+0000 7fcb06779700 1 --2- 172.21.15.100:0/1062762686 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb00107760 0x7fcb001a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:46.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.306+0000 7fcb05777700 1 -- 172.21.15.100:0/1062762686 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb00107760 msgr2=0x7fcb001a5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:46.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.306+0000 7fcb05777700 1 --2- 172.21.15.100:0/1062762686 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb00107760 0x7fcb001a5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:46.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.306+0000 7fcb05777700 1 -- 172.21.15.100:0/1062762686 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb00102e00 msgr2=0x7fcb0019a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:46.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.306+0000 7fcb05777700 1 --2- 172.21.15.100:0/1062762686 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb00102e00 0x7fcb0019a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:46.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.306+0000 7fcb05777700 1 -- 172.21.15.100:0/1062762686 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb001986e0 con 0x7fcb001037b0 2024-05-10T11:24:46.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.307+0000 7fcb05777700 1 --2- 172.21.15.100:0/1062762686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb001037b0 0x7fcb0019ae30 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0006010 tx=0x7fcaf0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:24:46.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.307+0000 7fcaf6ffd700 1 -- 172.21.15.100:0/1062762686 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf0026070 con 0x7fcb001037b0 2024-05-10T11:24:46.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.307+0000 7fcaf6ffd700 1 -- 172.21.15.100:0/1062762686 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcaf0004630 con 0x7fcb001037b0 2024-05-10T11:24:46.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.307+0000 7fcb081dc700 1 -- 172.21.15.100:0/1062762686 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb00198970 con 0x7fcb001037b0 2024-05-10T11:24:46.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.307+0000 7fcaf6ffd700 1 -- 172.21.15.100:0/1062762686 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf000faa0 con 0x7fcb001037b0 2024-05-10T11:24:46.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.307+0000 7fcb081dc700 1 -- 172.21.15.100:0/1062762686 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb00198ea0 con 0x7fcb001037b0 2024-05-10T11:24:46.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.309+0000 7fcaf6ffd700 1 -- 172.21.15.100:0/1062762686 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fcaf0029070 con 0x7fcb001037b0 2024-05-10T11:24:46.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.310+0000 7fcaf6ffd700 1 --2- 172.21.15.100:0/1062762686 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fcaec0b2900 0x7fcaec0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:46.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.310+0000 7fcb05f78700 1 --2- 172.21.15.100:0/1062762686 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fcaec0b2900 0x7fcaec0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:46.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.310+0000 7fcaf6ffd700 1 -- 172.21.15.100:0/1062762686 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fcaf00e2c00 con 0x7fcb001037b0 2024-05-10T11:24:46.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.311+0000 7fcb12a6fb80 1 Processor -- start 2024-05-10T11:24:46.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.311+0000 7fcb05f78700 1 --2- 172.21.15.100:0/1062762686 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fcaec0b2900 0x7fcaec0b4dc0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fcafc009c60 tx=0x7fcafc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:24:46.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.311+0000 7fcb12a6fb80 1 -- start start 2024-05-10T11:24:46.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.311+0000 7fcb12a6fb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631c97d01b0 0x5631c97d0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:46.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.311+0000 7fcb12a6fb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631c97d0ad0 0x5631c97ca810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:46.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.311+0000 7fcb12a6fb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631c97cad50 0x5631c97cb190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:46.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.311+0000 7fcb12a6fb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5631c971c940 con 0x5631c97d0ad0 2024-05-10T11:24:46.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.311+0000 7fcb12a6fb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5631c9681da0 con 0x5631c97d01b0 2024-05-10T11:24:46.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.311+0000 7fcb12a6fb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5631c9718900 con 0x5631c97cad50 2024-05-10T11:24:46.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.311+0000 7fcb05777700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631c97d0ad0 0x5631c97ca810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:46.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.311+0000 7fcb05f78700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631c97d01b0 0x5631c97d0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:46.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.311+0000 7fcb05777700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631c97d0ad0 0x5631c97ca810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40130/0 (socket says 172.21.15.100:40130) 2024-05-10T11:24:46.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.311+0000 7fcb05777700 1 -- 172.21.15.100:0/2746400986 learned_addr learned my addr 172.21.15.100:0/2746400986 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:24:46.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.312+0000 7fcb06779700 1 --2- 172.21.15.100:0/2746400986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631c97cad50 0x5631c97cb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:46.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.312+0000 7fcb05f78700 1 -- 172.21.15.100:0/2746400986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631c97cad50 msgr2=0x5631c97cb190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:46.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.312+0000 7fcb05f78700 1 --2- 172.21.15.100:0/2746400986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631c97cad50 0x5631c97cb190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:46.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.312+0000 7fcb05f78700 1 -- 172.21.15.100:0/2746400986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631c97d0ad0 msgr2=0x5631c97ca810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:46.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.312+0000 7fcb05f78700 1 --2- 172.21.15.100:0/2746400986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631c97d0ad0 0x5631c97ca810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:46.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.312+0000 7fcb05f78700 1 -- 172.21.15.100:0/2746400986 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631c96698d0 con 0x5631c97d01b0 2024-05-10T11:24:46.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.312+0000 7fcb05777700 1 --2- 172.21.15.100:0/2746400986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631c97d0ad0 0x5631c97ca810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:24:46.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.312+0000 7fcb05f78700 1 --2- 172.21.15.100:0/2746400986 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631c97d01b0 0x5631c97d0590 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fcafc00e3b0 tx=0x7fcafc014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=319f54f5e089274f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:24:46.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.312+0000 7fcaebfff700 1 -- 172.21.15.100:0/2746400986 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcafc00ed50 con 0x5631c97d01b0 2024-05-10T11:24:46.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.312+0000 7fcaebfff700 1 -- 172.21.15.100:0/2746400986 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcafc0044c0 con 0x5631c97d01b0 2024-05-10T11:24:46.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.312+0000 7fcaebfff700 1 -- 172.21.15.100:0/2746400986 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcafc026470 con 0x5631c97d01b0 2024-05-10T11:24:46.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.313+0000 7fcb12a6fb80 1 -- 172.21.15.100:0/2746400986 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631c97d01b0 msgr2=0x5631c97d0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:46.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.313+0000 7fcb12a6fb80 1 --2- 172.21.15.100:0/2746400986 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631c97d01b0 0x5631c97d0590 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fcafc00e3b0 tx=0x7fcafc014000 comp rx=0 tx=0).stop 2024-05-10T11:24:46.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.313+0000 7fcb12a6fb80 1 -- 172.21.15.100:0/2746400986 shutdown_connections 2024-05-10T11:24:46.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.313+0000 7fcb12a6fb80 1 --2- 172.21.15.100:0/2746400986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631c97cad50 0x5631c97cb190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:46.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.313+0000 7fcb12a6fb80 1 --2- 172.21.15.100:0/2746400986 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631c97d01b0 0x5631c97d0590 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:46.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.313+0000 7fcb12a6fb80 1 --2- 172.21.15.100:0/2746400986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631c97d0ad0 0x5631c97ca810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:46.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.313+0000 7fcb12a6fb80 1 -- 172.21.15.100:0/2746400986 >> 172.21.15.100:0/2746400986 conn(0x5631c96f6b70 msgr2=0x5631c97c52a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:24:46.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.313+0000 7fcb12a6fb80 1 -- 172.21.15.100:0/2746400986 shutdown_connections 2024-05-10T11:24:46.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.313+0000 7fcb12a6fb80 1 -- 172.21.15.100:0/2746400986 wait complete. 2024-05-10T11:24:46.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.313+0000 7fcb12a6fb80 1 -- start start 2024-05-10T11:24:46.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.313+0000 7fcb12a6fb80 1 Processor -- start 2024-05-10T11:24:46.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.314+0000 7fcb12a6fb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631c97cad50 0x5631c97d6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:46.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.314+0000 7fcb12a6fb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631c97d01b0 0x5631c97d3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:46.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.314+0000 7fcb12a6fb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631c97d0ad0 0x5631c97d4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:46.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.314+0000 7fcb12a6fb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5631c97d7450 con 0x5631c97d01b0 2024-05-10T11:24:46.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.314+0000 7fcb12a6fb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5631c96d5ed0 con 0x5631c97d0ad0 2024-05-10T11:24:46.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.314+0000 7fcb12a6fb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5631c97d75d0 con 0x5631c97cad50 2024-05-10T11:24:46.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.314+0000 7fcb12a6fb80 10 client.0 _ll_register_callbacks cb 0x5631c96b05b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:24:46.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.314+0000 7fcb05777700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631c97d01b0 0x5631c97d3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:46.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.314+0000 7fcb05777700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631c97d01b0 0x5631c97d3b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40134/0 (socket says 172.21.15.100:40134) 2024-05-10T11:24:46.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.314+0000 7fcb05777700 1 -- 172.21.15.100:0/841977433 learned_addr learned my addr 172.21.15.100:0/841977433 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:24:46.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.314+0000 7fcb06779700 1 --2- 172.21.15.100:0/841977433 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631c97d0ad0 0x5631c97d4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:46.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.314+0000 7fcb05777700 1 -- 172.21.15.100:0/841977433 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631c97cad50 msgr2=0x5631c97d6720 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T11:24:46.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.315+0000 7fcb05777700 1 --2- 172.21.15.100:0/841977433 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631c97cad50 0x5631c97d6720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:46.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.315+0000 7fcb05777700 1 -- 172.21.15.100:0/841977433 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631c97d0ad0 msgr2=0x5631c97d4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:46.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.315+0000 7fcb05777700 1 --2- 172.21.15.100:0/841977433 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631c97d0ad0 0x5631c97d4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:46.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.315+0000 7fcb05777700 1 -- 172.21.15.100:0/841977433 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631c96d3760 con 0x5631c97d01b0 2024-05-10T11:24:46.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.315+0000 7fcb06779700 1 --2- 172.21.15.100:0/841977433 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631c97d0ad0 0x5631c97d4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:24:46.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.315+0000 7fcb05777700 1 --2- 172.21.15.100:0/841977433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631c97d01b0 0x5631c97d3b10 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fcaf00e03f0 tx=0x7fcaf0010f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=163e4d764fa66039 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:24:46.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.315+0000 7fcae97fa700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:24:46.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.315+0000 7fcae97fa700 1 -- 172.21.15.100:0/841977433 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf00cc070 con 0x5631c97d01b0 2024-05-10T11:24:46.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.315+0000 7fcae97fa700 1 -- 172.21.15.100:0/841977433 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fcaf00b2aa0 con 0x5631c97d01b0 2024-05-10T11:24:46.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.316+0000 7fcb12a6fb80 1 -- 172.21.15.100:0/841977433 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcafc0044c0 con 0x5631c97d01b0 2024-05-10T11:24:46.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.316+0000 7fcae97fa700 1 -- 172.21.15.100:0/841977433 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fcaf00aa650 con 0x5631c97d01b0 2024-05-10T11:24:46.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.316+0000 7fcb12a6fb80 10 client.16680 fetch_fsmap learned FSMap version 31 2024-05-10T11:24:46.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.316+0000 7fcb12a6fb80 1 -- 172.21.15.100:0/841977433 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5631c97e1ce0 con 0x5631c97d01b0 2024-05-10T11:24:46.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.316+0000 7fcae97fa700 1 -- 172.21.15.100:0/841977433 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcafc0044c0 con 0x5631c97d01b0 2024-05-10T11:24:46.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.316+0000 7fcae97fa700 1 -- 172.21.15.100:0/841977433 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fcaf00a5a50 con 0x5631c97d01b0 2024-05-10T11:24:46.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.316+0000 7fcb12a6fb80 10 client.16680 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:24:46.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.316+0000 7fcb12a6fb80 10 client.16680 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:24:46.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.316+0000 7fcb12a6fb80 20 client.16680 populate_metadata read hostname 'smithi100' 2024-05-10T11:24:46.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.316+0000 7fcb12a6fb80 1 --2- 172.21.15.100:0/841977433 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5631c97e42a0 0x5631c97e6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:46.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.316+0000 7fcb12a6fb80 4 client.16680 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:24:46.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.316+0000 7fcb12a6fb80 1 -- 172.21.15.100:0/841977433 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5631c972a320 con 0x5631c97e42a0 2024-05-10T11:24:46.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.316+0000 7fcb05f78700 1 --2- 172.21.15.100:0/841977433 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5631c97e42a0 0x5631c97e6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:46.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.317+0000 7fcae97fa700 10 client.16680 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:24:46.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.317+0000 7fcb05f78700 1 --2- 172.21.15.100:0/841977433 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5631c97e42a0 0x5631c97e6710 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f34ade37a371684 server_cookie=1b879a15d90c2a40 in_seq=0 out_seq=0 2024-05-10T11:24:46.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.317+0000 7fcae97fa700 1 -- 172.21.15.100:0/841977433 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5631c972a320 con 0x5631c97e42a0 2024-05-10T11:24:46.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.317+0000 7fcae97fa700 10 client.16680 handle_command_reply: tid=0 2024-05-10T11:24:46.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.317+0000 7fcb12a6fb80 1 client.16680 shutdown 2024-05-10T11:24:46.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.318+0000 7fcb12a6fb80 1 -- 172.21.15.100:0/841977433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631c97d01b0 msgr2=0x5631c97d3b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:46.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.318+0000 7fcb12a6fb80 1 --2- 172.21.15.100:0/841977433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631c97d01b0 0x5631c97d3b10 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fcaf00e03f0 tx=0x7fcaf0010f30 comp rx=0 tx=0).stop 2024-05-10T11:24:46.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.318+0000 7fcb12a6fb80 1 -- 172.21.15.100:0/841977433 shutdown_connections 2024-05-10T11:24:46.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.318+0000 7fcb12a6fb80 1 --2- 172.21.15.100:0/841977433 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5631c97e42a0 0x5631c97e6710 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:46.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.318+0000 7fcb12a6fb80 1 --2- 172.21.15.100:0/841977433 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631c97d0ad0 0x5631c97d4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:46.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.318+0000 7fcb12a6fb80 1 --2- 172.21.15.100:0/841977433 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631c97cad50 0x5631c97d6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:46.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.318+0000 7fcae97fa700 0 client.16680 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:24:46.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.318+0000 7fcb12a6fb80 1 --2- 172.21.15.100:0/841977433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631c97d01b0 0x5631c97d3b10 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:46.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.318+0000 7fcb12a6fb80 1 -- 172.21.15.100:0/841977433 >> 172.21.15.100:0/841977433 conn(0x5631c96f6b70 msgr2=0x5631c97bfb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:24:46.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.318+0000 7fcb12a6fb80 1 -- 172.21.15.100:0/841977433 shutdown_connections 2024-05-10T11:24:46.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.318+0000 7fcb12a6fb80 1 -- 172.21.15.100:0/841977433 wait complete. 2024-05-10T11:24:46.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.318+0000 7fcb12a6fb80 20 client.16680 trim_cache size 0 max 16384 2024-05-10T11:24:46.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.339+0000 7fcb12a6fb80 1 Processor -- start 2024-05-10T11:24:46.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.339+0000 7fcb12a6fb80 1 -- start start 2024-05-10T11:24:46.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.339+0000 7fcb12a6fb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631c97cad50 0x5631c97ca810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:46.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.339+0000 7fcb12a6fb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631c97d01b0 0x5631c97e04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:46.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.339+0000 7fcb12a6fb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631c97d0ad0 0x5631c97d15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:46.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.339+0000 7fcb12a6fb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fcb001b3ad0 con 0x5631c97d01b0 2024-05-10T11:24:46.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.339+0000 7fcb12a6fb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcb001b1360 con 0x5631c97cad50 2024-05-10T11:24:46.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.339+0000 7fcb12a6fb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcb001af1b0 con 0x5631c97d0ad0 2024-05-10T11:24:46.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.339+0000 7fcb05777700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631c97d01b0 0x5631c97e04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:46.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.339+0000 7fcb05777700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631c97d01b0 0x5631c97e04e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40138/0 (socket says 172.21.15.100:40138) 2024-05-10T11:24:46.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.339+0000 7fcb05777700 1 -- 172.21.15.100:0/3838211870 learned_addr learned my addr 172.21.15.100:0/3838211870 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:24:46.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.340+0000 7fcb06779700 1 --2- 172.21.15.100:0/3838211870 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631c97d0ad0 0x5631c97d15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:46.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.340+0000 7fcb05f78700 1 --2- 172.21.15.100:0/3838211870 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631c97cad50 0x5631c97ca810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:46.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.340+0000 7fcb05777700 1 -- 172.21.15.100:0/3838211870 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631c97d0ad0 msgr2=0x5631c97d15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:46.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.340+0000 7fcb05777700 1 --2- 172.21.15.100:0/3838211870 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631c97d0ad0 0x5631c97d15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:46.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.340+0000 7fcb05777700 1 -- 172.21.15.100:0/3838211870 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631c97cad50 msgr2=0x5631c97ca810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:46.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.340+0000 7fcb05777700 1 --2- 172.21.15.100:0/3838211870 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631c97cad50 0x5631c97ca810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:46.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.340+0000 7fcb05777700 1 -- 172.21.15.100:0/3838211870 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631c97e6d90 con 0x5631c97d01b0 2024-05-10T11:24:46.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.340+0000 7fcb05777700 1 --2- 172.21.15.100:0/3838211870 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631c97d01b0 0x5631c97e04e0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fcaf00a9c50 tx=0x7fcaf0009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=484cacd8229dcf32 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:24:46.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.340+0000 7fcaeb7fe700 1 -- 172.21.15.100:0/3838211870 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf00cc040 con 0x5631c97d01b0 2024-05-10T11:24:46.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.340+0000 7fcaeb7fe700 1 -- 172.21.15.100:0/3838211870 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcaf00ae300 con 0x5631c97d01b0 2024-05-10T11:24:46.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.340+0000 7fcaeb7fe700 1 -- 172.21.15.100:0/3838211870 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf00a5b20 con 0x5631c97d01b0 2024-05-10T11:24:46.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.341+0000 7fcb12a6fb80 1 -- 172.21.15.100:0/3838211870 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631c97d01b0 msgr2=0x5631c97e04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:46.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.341+0000 7fcb12a6fb80 1 --2- 172.21.15.100:0/3838211870 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631c97d01b0 0x5631c97e04e0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fcaf00a9c50 tx=0x7fcaf0009710 comp rx=0 tx=0).stop 2024-05-10T11:24:46.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.341+0000 7fcb12a6fb80 1 -- 172.21.15.100:0/3838211870 shutdown_connections 2024-05-10T11:24:46.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.341+0000 7fcb12a6fb80 1 --2- 172.21.15.100:0/3838211870 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631c97d0ad0 0x5631c97d15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:46.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.341+0000 7fcb12a6fb80 1 --2- 172.21.15.100:0/3838211870 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631c97cad50 0x5631c97ca810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:46.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.341+0000 7fcb12a6fb80 1 --2- 172.21.15.100:0/3838211870 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631c97d01b0 0x5631c97e04e0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:46.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.341+0000 7fcb12a6fb80 1 -- 172.21.15.100:0/3838211870 >> 172.21.15.100:0/3838211870 conn(0x5631c96f6b70 msgr2=0x5631c97d0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:24:46.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.341+0000 7fcb12a6fb80 1 -- 172.21.15.100:0/3838211870 shutdown_connections 2024-05-10T11:24:46.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.341+0000 7fcb12a6fb80 1 -- 172.21.15.100:0/3838211870 wait complete. 2024-05-10T11:24:46.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.341+0000 7fcb12a6fb80 1 -- start start 2024-05-10T11:24:46.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.341+0000 7fcb12a6fb80 1 Processor -- start 2024-05-10T11:24:46.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.342+0000 7fcb12a6fb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631c97cad50 0x5631c97dc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:46.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.342+0000 7fcb12a6fb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631c97d01b0 0x5631c97dcda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:46.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.342+0000 7fcb12a6fb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631c97d0ad0 0x5631c97d9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:46.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.342+0000 7fcb12a6fb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fcb001af820 con 0x5631c97cad50 2024-05-10T11:24:46.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.342+0000 7fcb12a6fb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcb001b0c90 con 0x5631c97d0ad0 2024-05-10T11:24:46.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.342+0000 7fcb05f78700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631c97cad50 0x5631c97dc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:46.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.342+0000 7fcb05f78700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631c97cad50 0x5631c97dc860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40142/0 (socket says 172.21.15.100:40142) 2024-05-10T11:24:46.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.342+0000 7fcb05f78700 1 -- 172.21.15.100:0/3043703795 learned_addr learned my addr 172.21.15.100:0/3043703795 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:24:46.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.342+0000 7fcb12a6fb80 1 -- 172.21.15.100:0/3043703795 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcb001afef0 con 0x5631c97d01b0 2024-05-10T11:24:46.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.342+0000 7fcb06779700 1 --2- 172.21.15.100:0/3043703795 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631c97d0ad0 0x5631c97d9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:46.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.342+0000 7fcb12a6fb80 10 client.0 _ll_register_callbacks cb 0x5631c96b05b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:24:46.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.342+0000 7fcb05777700 1 --2- 172.21.15.100:0/3043703795 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631c97d01b0 0x5631c97dcda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:46.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.342+0000 7fcb05f78700 1 -- 172.21.15.100:0/3043703795 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631c97d01b0 msgr2=0x5631c97dcda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:46.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.342+0000 7fcb05f78700 1 --2- 172.21.15.100:0/3043703795 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631c97d01b0 0x5631c97dcda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:46.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.342+0000 7fcb05f78700 1 -- 172.21.15.100:0/3043703795 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631c97d0ad0 msgr2=0x5631c97d9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:46.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.342+0000 7fcb05f78700 1 --2- 172.21.15.100:0/3043703795 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631c97d0ad0 0x5631c97d9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:46.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.342+0000 7fcb05f78700 1 -- 172.21.15.100:0/3043703795 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631c97d9d70 con 0x5631c97cad50 2024-05-10T11:24:46.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.342+0000 7fcb06779700 1 --2- 172.21.15.100:0/3043703795 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631c97d0ad0 0x5631c97d9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:24:46.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.343+0000 7fcb05f78700 1 --2- 172.21.15.100:0/3043703795 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631c97cad50 0x5631c97dc860 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fcafc00e3b0 tx=0x7fcafc0096f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92437d5ed0ac3c09 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:24:46.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.343+0000 7fcae97fa700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:24:46.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.343+0000 7fcae97fa700 1 -- 172.21.15.100:0/3043703795 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcafc035060 con 0x5631c97cad50 2024-05-10T11:24:46.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.343+0000 7fcae97fa700 1 -- 172.21.15.100:0/3043703795 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fcafc0050c0 con 0x5631c97cad50 2024-05-10T11:24:46.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.343+0000 7fcb12a6fb80 1 -- 172.21.15.100:0/3043703795 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcaf00ae300 con 0x5631c97cad50 2024-05-10T11:24:46.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.343+0000 7fcae97fa700 1 -- 172.21.15.100:0/3043703795 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fcafc03c050 con 0x5631c97cad50 2024-05-10T11:24:46.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.343+0000 7fcb12a6fb80 10 client.16686 fetch_fsmap learned FSMap version 31 2024-05-10T11:24:46.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.343+0000 7fcb12a6fb80 1 -- 172.21.15.100:0/3043703795 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5631c984f280 con 0x5631c97cad50 2024-05-10T11:24:46.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.343+0000 7fcae97fa700 1 -- 172.21.15.100:0/3043703795 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcaf00ae300 con 0x5631c97cad50 2024-05-10T11:24:46.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.343+0000 7fcae97fa700 1 -- 172.21.15.100:0/3043703795 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fcafc003680 con 0x5631c97cad50 2024-05-10T11:24:46.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.343+0000 7fcb12a6fb80 10 client.16686 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:24:46.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.343+0000 7fcb12a6fb80 10 client.16686 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:24:46.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.343+0000 7fcb12a6fb80 20 client.16686 populate_metadata read hostname 'smithi100' 2024-05-10T11:24:46.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.343+0000 7fcb12a6fb80 1 --2- 172.21.15.100:0/3043703795 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5631c984f620 0x5631c9851a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:46.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.343+0000 7fcb12a6fb80 4 client.16686 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T11:24:46.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.343+0000 7fcb12a6fb80 1 -- 172.21.15.100:0/3043703795 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5631c97243c0 con 0x5631c984f620 2024-05-10T11:24:46.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.344+0000 7fcb05777700 1 --2- 172.21.15.100:0/3043703795 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5631c984f620 0x5631c9851a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:46.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.344+0000 7fcb05777700 1 --2- 172.21.15.100:0/3043703795 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5631c984f620 0x5631c9851a40 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc095343a051b252 server_cookie=9c5b6fa9fc94b111 in_seq=0 out_seq=0 2024-05-10T11:24:46.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.344+0000 7fcae97fa700 10 client.16686 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:24:46.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.345+0000 7fcae97fa700 1 -- 172.21.15.100:0/3043703795 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5631c97243c0 con 0x5631c984f620 2024-05-10T11:24:46.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.345+0000 7fcae97fa700 10 client.16686 handle_command_reply: tid=0 2024-05-10T11:24:46.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.345+0000 7fcb12a6fb80 1 client.16686 shutdown 2024-05-10T11:24:46.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.345+0000 7fcb12a6fb80 1 -- 172.21.15.100:0/3043703795 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631c97cad50 msgr2=0x5631c97dc860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:46.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.345+0000 7fcb12a6fb80 1 --2- 172.21.15.100:0/3043703795 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631c97cad50 0x5631c97dc860 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fcafc00e3b0 tx=0x7fcafc0096f0 comp rx=0 tx=0).stop 2024-05-10T11:24:46.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.345+0000 7fcb12a6fb80 1 -- 172.21.15.100:0/3043703795 shutdown_connections 2024-05-10T11:24:46.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.345+0000 7fcb12a6fb80 1 --2- 172.21.15.100:0/3043703795 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5631c984f620 0x5631c9851a40 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:46.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.345+0000 7fcb12a6fb80 1 --2- 172.21.15.100:0/3043703795 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631c97d0ad0 0x5631c97d9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:46.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.345+0000 7fcb12a6fb80 1 --2- 172.21.15.100:0/3043703795 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631c97cad50 0x5631c97dc860 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:46.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.345+0000 7fcb12a6fb80 1 --2- 172.21.15.100:0/3043703795 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631c97d01b0 0x5631c97dcda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:46.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.345+0000 7fcb12a6fb80 1 -- 172.21.15.100:0/3043703795 >> 172.21.15.100:0/3043703795 conn(0x5631c96f6b70 msgr2=0x5631c97cec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:24:46.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.345+0000 7fcae97fa700 0 client.16686 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:24:46.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.345+0000 7fcb12a6fb80 1 -- 172.21.15.100:0/3043703795 shutdown_connections 2024-05-10T11:24:46.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.345+0000 7fcb12a6fb80 1 -- 172.21.15.100:0/3043703795 wait complete. 2024-05-10T11:24:46.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.345+0000 7fcb12a6fb80 20 client.16686 trim_cache size 0 max 16384 2024-05-10T11:24:46.350 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:24:46.350 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T11:24:46.350 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-05-10T11:24:46.350 INFO:teuthology.orchestra.run.smithi100.stdout: "51a6d3a5-affd-4d91-8070-9fc824841270": { 2024-05-10T11:24:46.350 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-05-10T11:24:46.350 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "51a6d3a5-affd-4d91-8070-9fc824841270", 2024-05-10T11:24:46.350 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-05-10T11:24:46.350 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:24:46.350 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:24:46.350 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:24:46.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.346+0000 7fcb081dc700 1 -- 172.21.15.100:0/1062762686 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fcaec0b2900 msgr2=0x7fcaec0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:46.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.346+0000 7fcb081dc700 1 --2- 172.21.15.100:0/1062762686 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fcaec0b2900 0x7fcaec0b4dc0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fcafc009c60 tx=0x7fcafc009260 comp rx=0 tx=0).stop 2024-05-10T11:24:46.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.346+0000 7fcb081dc700 1 -- 172.21.15.100:0/1062762686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb001037b0 msgr2=0x7fcb0019ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:46.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.346+0000 7fcb081dc700 1 --2- 172.21.15.100:0/1062762686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb001037b0 0x7fcb0019ae30 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0006010 tx=0x7fcaf0004270 comp rx=0 tx=0).stop 2024-05-10T11:24:46.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.346+0000 7fcb081dc700 1 -- 172.21.15.100:0/1062762686 shutdown_connections 2024-05-10T11:24:46.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.346+0000 7fcb081dc700 1 --2- 172.21.15.100:0/1062762686 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fcaec0b2900 0x7fcaec0b4dc0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:46.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.346+0000 7fcb081dc700 1 --2- 172.21.15.100:0/1062762686 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb00107760 0x7fcb001a5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:46.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.346+0000 7fcb081dc700 1 --2- 172.21.15.100:0/1062762686 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb00102e00 0x7fcb0019a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:46.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.346+0000 7fcb081dc700 1 --2- 172.21.15.100:0/1062762686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb001037b0 0x7fcb0019ae30 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:46.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.346+0000 7fcb081dc700 1 -- 172.21.15.100:0/1062762686 >> 172.21.15.100:0/1062762686 conn(0x7fcb00078a10 msgr2=0x7fcb000ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:24:46.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.346+0000 7fcb081dc700 1 -- 172.21.15.100:0/1062762686 shutdown_connections 2024-05-10T11:24:46.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:24:46.346+0000 7fcb081dc700 1 -- 172.21.15.100:0/1062762686 wait complete. 2024-05-10T11:24:46.973 INFO:tasks.workunit.client.0.smithi100.stdout: 1 146201 2.74 MB/sec execute 161 sec latency 12.048 ms 2024-05-10T11:24:47.138 INFO:tasks.cephfs.filesystem:scrub status for tag:51a6d3a5-affd-4d91-8070-9fc824841270 - {'path': '/', 'tag': '51a6d3a5-affd-4d91-8070-9fc824841270', 'options': 'recursive,force'} 2024-05-10T11:24:47.973 INFO:tasks.workunit.client.0.smithi100.stdout: 1 146714 2.73 MB/sec execute 162 sec latency 15.623 ms 2024-05-10T11:24:48.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:24:47 smithi178 ceph-mon[33359]: pgmap v612: 129 pgs: 129 active+clean; 68 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 1.3 MiB/s wr, 132 op/s 2024-05-10T11:24:48.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:24:47 smithi188 ceph-mon[34075]: pgmap v612: 129 pgs: 129 active+clean; 68 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 1.3 MiB/s wr, 132 op/s 2024-05-10T11:24:48.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:24:47 smithi100 ceph-mon[30296]: pgmap v612: 129 pgs: 129 active+clean; 68 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 1.3 MiB/s wr, 132 op/s 2024-05-10T11:24:48.973 INFO:tasks.workunit.client.0.smithi100.stdout: 1 147364 2.74 MB/sec execute 163 sec latency 15.369 ms 2024-05-10T11:24:49.973 INFO:tasks.workunit.client.0.smithi100.stdout: 1 147947 2.73 MB/sec execute 164 sec latency 15.571 ms 2024-05-10T11:24:50.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:24:49 smithi178 ceph-mon[33359]: pgmap v613: 129 pgs: 129 active+clean; 69 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 1.4 MiB/s wr, 151 op/s 2024-05-10T11:24:50.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:24:49 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:24:50.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:24:49 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:24:50.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:24:49 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:24:50.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:24:49 smithi188 ceph-mon[34075]: pgmap v613: 129 pgs: 129 active+clean; 69 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 1.4 MiB/s wr, 151 op/s 2024-05-10T11:24:50.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:24:49 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:24:50.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:24:49 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:24:50.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:24:49 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:24:50.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:24:49 smithi100 ceph-mon[30296]: pgmap v613: 129 pgs: 129 active+clean; 69 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 1.4 MiB/s wr, 151 op/s 2024-05-10T11:24:50.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:24:49 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:24:50.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:24:49 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:24:50.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:24:49 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:24:50.973 INFO:tasks.workunit.client.0.smithi100.stdout: 1 148738 2.76 MB/sec execute 165 sec latency 15.472 ms 2024-05-10T11:24:51.973 INFO:tasks.workunit.client.0.smithi100.stdout: 1 149154 2.76 MB/sec execute 166 sec latency 12.345 ms 2024-05-10T11:24:52.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:24:51 smithi178 ceph-mon[33359]: pgmap v614: 129 pgs: 129 active+clean; 69 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 1.3 MiB/s wr, 154 op/s 2024-05-10T11:24:52.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:24:51 smithi188 ceph-mon[34075]: pgmap v614: 129 pgs: 129 active+clean; 69 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 1.3 MiB/s wr, 154 op/s 2024-05-10T11:24:52.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:24:51 smithi100 ceph-mon[30296]: pgmap v614: 129 pgs: 129 active+clean; 69 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 1.3 MiB/s wr, 154 op/s 2024-05-10T11:24:52.973 INFO:tasks.workunit.client.0.smithi100.stdout: 1 149438 2.75 MB/sec execute 167 sec latency 15.385 ms 2024-05-10T11:24:53.973 INFO:tasks.workunit.client.0.smithi100.stdout: 1 149790 2.74 MB/sec execute 168 sec latency 16.041 ms 2024-05-10T11:24:54.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:24:53 smithi178 ceph-mon[33359]: pgmap v615: 129 pgs: 129 active+clean; 69 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 1.3 MiB/s wr, 151 op/s 2024-05-10T11:24:54.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:24:53 smithi188 ceph-mon[34075]: pgmap v615: 129 pgs: 129 active+clean; 69 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 1.3 MiB/s wr, 151 op/s 2024-05-10T11:24:54.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:24:53 smithi100 ceph-mon[30296]: pgmap v615: 129 pgs: 129 active+clean; 69 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 1.3 MiB/s wr, 151 op/s 2024-05-10T11:24:54.973 INFO:tasks.workunit.client.0.smithi100.stdout: 1 150285 2.73 MB/sec execute 169 sec latency 15.553 ms 2024-05-10T11:24:55.973 INFO:tasks.workunit.client.0.smithi100.stdout: 1 150968 2.74 MB/sec execute 170 sec latency 19.378 ms 2024-05-10T11:24:56.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:24:55 smithi178 ceph-mon[33359]: pgmap v616: 129 pgs: 129 active+clean; 70 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 1.4 MiB/s wr, 171 op/s 2024-05-10T11:24:56.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:24:55 smithi188 ceph-mon[34075]: pgmap v616: 129 pgs: 129 active+clean; 70 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 1.4 MiB/s wr, 171 op/s 2024-05-10T11:24:56.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:24:55 smithi100 ceph-mon[30296]: pgmap v616: 129 pgs: 129 active+clean; 70 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 1.4 MiB/s wr, 171 op/s 2024-05-10T11:24:56.957 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.956+0000 7fd38861c640 1 -- 172.21.15.100:0/811613096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd380101910 msgr2=0x7fd380101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:56.958 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.956+0000 7fd38861c640 1 --2- 172.21.15.100:0/811613096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd380101910 0x7fd380101d30 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fd37c009900 tx=0x7fd37c02a410 comp rx=0 tx=0).stop 2024-05-10T11:24:56.958 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.956+0000 7fd38861c640 1 -- 172.21.15.100:0/811613096 shutdown_connections 2024-05-10T11:24:56.958 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.956+0000 7fd38861c640 1 --2- 172.21.15.100:0/811613096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3801034c0 0x7fd380109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:56.958 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.956+0000 7fd38861c640 1 --2- 172.21.15.100:0/811613096 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd380102b00 0x7fd380102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:56.958 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.956+0000 7fd38861c640 1 --2- 172.21.15.100:0/811613096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd380101910 0x7fd380101d30 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:56.958 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.956+0000 7fd38861c640 1 -- 172.21.15.100:0/811613096 >> 172.21.15.100:0/811613096 conn(0x7fd3800fce80 msgr2=0x7fd3800ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:24:56.958 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.956+0000 7fd38861c640 1 -- 172.21.15.100:0/811613096 shutdown_connections 2024-05-10T11:24:56.958 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.956+0000 7fd38861c640 1 -- 172.21.15.100:0/811613096 wait complete. 2024-05-10T11:24:56.958 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.956+0000 7fd38861c640 1 Processor -- start 2024-05-10T11:24:56.958 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.956+0000 7fd38861c640 1 -- start start 2024-05-10T11:24:56.958 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.957+0000 7fd38861c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd380101910 0x7fd380197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:56.959 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.957+0000 7fd38861c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd380102b00 0x7fd380197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:56.959 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.957+0000 7fd38861c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3801034c0 0x7fd38019eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:56.959 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.957+0000 7fd38861c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd38010d310 con 0x7fd380102b00 2024-05-10T11:24:56.959 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.957+0000 7fd38861c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd38010d190 con 0x7fd3801034c0 2024-05-10T11:24:56.959 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.957+0000 7fd38861c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd38010d490 con 0x7fd380101910 2024-05-10T11:24:56.959 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.957+0000 7fd385bb7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd380102b00 0x7fd380197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:56.959 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.957+0000 7fd3863b8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd380101910 0x7fd380197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:56.959 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.957+0000 7fd385bb7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd380102b00 0x7fd380197f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38178/0 (socket says 172.21.15.100:38178) 2024-05-10T11:24:56.959 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.957+0000 7fd3863b8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd380101910 0x7fd380197a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:34148/0 (socket says 172.21.15.100:34148) 2024-05-10T11:24:56.959 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.957+0000 7fd386bb9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3801034c0 0x7fd38019eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:56.959 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.957+0000 7fd385bb7640 1 -- 172.21.15.100:0/3460779807 learned_addr learned my addr 172.21.15.100:0/3460779807 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:24:56.959 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.957+0000 7fd385bb7640 1 -- 172.21.15.100:0/3460779807 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd380101910 msgr2=0x7fd380197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:56.959 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.957+0000 7fd385bb7640 1 --2- 172.21.15.100:0/3460779807 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd380101910 0x7fd380197a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:56.959 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.957+0000 7fd385bb7640 1 -- 172.21.15.100:0/3460779807 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3801034c0 msgr2=0x7fd38019eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:56.959 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.957+0000 7fd385bb7640 1 --2- 172.21.15.100:0/3460779807 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3801034c0 0x7fd38019eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:56.960 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.957+0000 7fd385bb7640 1 -- 172.21.15.100:0/3460779807 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd38019f660 con 0x7fd380102b00 2024-05-10T11:24:56.960 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.957+0000 7fd385bb7640 1 --2- 172.21.15.100:0/3460779807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd380102b00 0x7fd380197f80 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fd37000c890 tx=0x7fd37000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:24:56.960 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.957+0000 7fd3777fe640 1 -- 172.21.15.100:0/3460779807 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd370004220 con 0x7fd380102b00 2024-05-10T11:24:56.960 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.958+0000 7fd38861c640 1 -- 172.21.15.100:0/3460779807 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd38019f950 con 0x7fd380102b00 2024-05-10T11:24:56.960 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.958+0000 7fd3777fe640 1 -- 172.21.15.100:0/3460779807 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd370007e40 con 0x7fd380102b00 2024-05-10T11:24:56.960 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.958+0000 7fd3777fe640 1 -- 172.21.15.100:0/3460779807 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd370004a00 con 0x7fd380102b00 2024-05-10T11:24:56.960 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.958+0000 7fd38861c640 1 -- 172.21.15.100:0/3460779807 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd38019fea0 con 0x7fd380102b00 2024-05-10T11:24:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.960+0000 7fd38b41a740 1 Processor -- start 2024-05-10T11:24:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.961+0000 7fd38b41a740 1 -- start start 2024-05-10T11:24:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.961+0000 7fd3777fe640 1 -- 172.21.15.100:0/3460779807 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fd3700079b0 con 0x7fd380102b00 2024-05-10T11:24:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.961+0000 7fd38b41a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588261b9db0 0x5588261ba1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.961+0000 7fd38b41a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5588261ba7a0 0x5588261bf2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.961+0000 7fd38b41a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5588261bf910 0x5588261c1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.961+0000 7fd38b41a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55882611e5d0 con 0x5588261b9db0 2024-05-10T11:24:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.961+0000 7fd38b41a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5588260d7890 con 0x5588261bf910 2024-05-10T11:24:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.961+0000 7fd38b41a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55882611a590 con 0x5588261ba7a0 2024-05-10T11:24:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.961+0000 7fd386bb9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588261b9db0 0x5588261ba1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.961+0000 7fd386bb9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588261b9db0 0x5588261ba1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38186/0 (socket says 172.21.15.100:38186) 2024-05-10T11:24:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.961+0000 7fd3863b8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5588261ba7a0 0x5588261bf2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.961+0000 7fd386bb9640 1 -- 172.21.15.100:0/2736431944 learned_addr learned my addr 172.21.15.100:0/2736431944 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:24:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.961+0000 7fd385bb7640 1 --2- 172.21.15.100:0/2736431944 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5588261bf910 0x5588261c1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.961+0000 7fd386bb9640 1 -- 172.21.15.100:0/2736431944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5588261ba7a0 msgr2=0x5588261bf2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.961+0000 7fd386bb9640 1 --2- 172.21.15.100:0/2736431944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5588261ba7a0 0x5588261bf2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.961+0000 7fd386bb9640 1 -- 172.21.15.100:0/2736431944 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5588261bf910 msgr2=0x5588261c1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.961+0000 7fd386bb9640 1 --2- 172.21.15.100:0/2736431944 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5588261bf910 0x5588261c1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.961+0000 7fd386bb9640 1 -- 172.21.15.100:0/2736431944 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558825e30210 con 0x5588261b9db0 2024-05-10T11:24:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.961+0000 7fd386bb9640 1 --2- 172.21.15.100:0/2736431944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588261b9db0 0x5588261ba1d0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fd3780039f0 tx=0x7fd378003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5500c027f4c1b1b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:24:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.962+0000 7fd374bfe640 1 -- 172.21.15.100:0/2736431944 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37801a470 con 0x5588261b9db0 2024-05-10T11:24:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.962+0000 7fd3777fe640 1 --2- 172.21.15.100:0/3460779807 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fd36c0b28a0 0x7fd36c0b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.962+0000 7fd374bfe640 1 -- 172.21.15.100:0/2736431944 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd378015070 con 0x5588261b9db0 2024-05-10T11:24:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.962+0000 7fd374bfe640 1 -- 172.21.15.100:0/2736431944 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd378010770 con 0x5588261b9db0 2024-05-10T11:24:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.962+0000 7fd3863b8640 1 --2- 172.21.15.100:0/3460779807 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fd36c0b28a0 0x7fd36c0b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.962+0000 7fd3777fe640 1 -- 172.21.15.100:0/3460779807 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fd3700e2200 con 0x7fd380102b00 2024-05-10T11:24:56.968 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.962+0000 7fd38b41a740 1 -- 172.21.15.100:0/2736431944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588261b9db0 msgr2=0x5588261ba1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:56.968 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.962+0000 7fd3863b8640 1 --2- 172.21.15.100:0/3460779807 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fd36c0b28a0 0x7fd36c0b4d60 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd37c0057c0 tx=0x7fd37c005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:24:56.968 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.962+0000 7fd38b41a740 1 --2- 172.21.15.100:0/2736431944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588261b9db0 0x5588261ba1d0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fd3780039f0 tx=0x7fd378003eb0 comp rx=0 tx=0).stop 2024-05-10T11:24:56.968 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.963+0000 7fd38b41a740 1 -- 172.21.15.100:0/2736431944 shutdown_connections 2024-05-10T11:24:56.968 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.963+0000 7fd38b41a740 1 --2- 172.21.15.100:0/2736431944 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5588261bf910 0x5588261c1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:56.968 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.963+0000 7fd38b41a740 1 --2- 172.21.15.100:0/2736431944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5588261ba7a0 0x5588261bf2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:56.968 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.963+0000 7fd38b41a740 1 --2- 172.21.15.100:0/2736431944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588261b9db0 0x5588261ba1d0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:56.968 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.963+0000 7fd38b41a740 1 -- 172.21.15.100:0/2736431944 >> 172.21.15.100:0/2736431944 conn(0x5588261b5420 msgr2=0x5588261b5820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:24:56.968 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.963+0000 7fd38b41a740 1 -- 172.21.15.100:0/2736431944 shutdown_connections 2024-05-10T11:24:56.968 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.963+0000 7fd38b41a740 1 -- 172.21.15.100:0/2736431944 wait complete. 2024-05-10T11:24:56.968 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.963+0000 7fd38b41a740 1 -- start start 2024-05-10T11:24:56.968 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.963+0000 7fd38b41a740 1 Processor -- start 2024-05-10T11:24:56.968 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.963+0000 7fd38b41a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588261b9db0 0x5588261c4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:56.968 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.963+0000 7fd38b41a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5588261ba7a0 0x5588261c45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.963+0000 7fd38b41a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5588261bf910 0x5588261cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.963+0000 7fd38b41a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5588261c28c0 con 0x5588261b9db0 2024-05-10T11:24:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.963+0000 7fd38b41a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5588260db370 con 0x5588261ba7a0 2024-05-10T11:24:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.963+0000 7fd38b41a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5588261c2a40 con 0x5588261bf910 2024-05-10T11:24:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.963+0000 7fd3863b8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588261b9db0 0x5588261c4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.963+0000 7fd3863b8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588261b9db0 0x5588261c4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38190/0 (socket says 172.21.15.100:38190) 2024-05-10T11:24:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.963+0000 7fd3863b8640 1 -- 172.21.15.100:0/905516263 learned_addr learned my addr 172.21.15.100:0/905516263 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:24:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.963+0000 7fd38b41a740 10 client.0 _ll_register_callbacks cb 0x558825e2fa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:24:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.963+0000 7fd385bb7640 1 --2- 172.21.15.100:0/905516263 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5588261ba7a0 0x5588261c45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.963+0000 7fd386bb9640 1 --2- 172.21.15.100:0/905516263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5588261bf910 0x5588261cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.964+0000 7fd3863b8640 1 -- 172.21.15.100:0/905516263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5588261bf910 msgr2=0x5588261cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.964+0000 7fd3863b8640 1 --2- 172.21.15.100:0/905516263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5588261bf910 0x5588261cb5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.964+0000 7fd3863b8640 1 -- 172.21.15.100:0/905516263 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5588261ba7a0 msgr2=0x5588261c45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.964+0000 7fd3863b8640 1 --2- 172.21.15.100:0/905516263 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5588261ba7a0 0x5588261c45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.964+0000 7fd3863b8640 1 -- 172.21.15.100:0/905516263 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558825efc620 con 0x5588261b9db0 2024-05-10T11:24:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.964+0000 7fd3863b8640 1 --2- 172.21.15.100:0/905516263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588261b9db0 0x5588261c4060 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fd37c005ab0 tx=0x7fd37c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bd15d1c6d2b4065 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:24:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.964+0000 7fd369ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:24:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.964+0000 7fd369ffb640 1 -- 172.21.15.100:0/905516263 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37c035040 con 0x5588261b9db0 2024-05-10T11:24:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.964+0000 7fd369ffb640 1 -- 172.21.15.100:0/905516263 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fd37c031700 con 0x5588261b9db0 2024-05-10T11:24:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.964+0000 7fd38b41a740 1 -- 172.21.15.100:0/905516263 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd378015070 con 0x5588261b9db0 2024-05-10T11:24:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.964+0000 7fd369ffb640 1 -- 172.21.15.100:0/905516263 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fd37c05a050 con 0x5588261b9db0 2024-05-10T11:24:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.964+0000 7fd38b41a740 10 client.16698 fetch_fsmap learned FSMap version 31 2024-05-10T11:24:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.964+0000 7fd38b41a740 1 -- 172.21.15.100:0/905516263 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5588261ce610 con 0x5588261b9db0 2024-05-10T11:24:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.964+0000 7fd369ffb640 1 -- 172.21.15.100:0/905516263 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd378015070 con 0x5588261b9db0 2024-05-10T11:24:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.965+0000 7fd369ffb640 1 -- 172.21.15.100:0/905516263 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fd37c044030 con 0x5588261b9db0 2024-05-10T11:24:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.965+0000 7fd38b41a740 10 client.16698 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:24:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.965+0000 7fd38b41a740 10 client.16698 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:24:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.965+0000 7fd38b41a740 20 client.16698 populate_metadata read hostname 'smithi100' 2024-05-10T11:24:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.965+0000 7fd38b41a740 1 --2- 172.21.15.100:0/905516263 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5588261cfce0 0x5588261d2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.965+0000 7fd38b41a740 4 client.16698 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:24:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.965+0000 7fd38b41a740 1 -- 172.21.15.100:0/905516263 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55882612c1e0 con 0x5588261cfce0 2024-05-10T11:24:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.965+0000 7fd385bb7640 1 --2- 172.21.15.100:0/905516263 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5588261cfce0 0x5588261d2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.965+0000 7fd385bb7640 1 --2- 172.21.15.100:0/905516263 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5588261cfce0 0x5588261d2130 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ce12f227438a386 server_cookie=d3b5adabaa665fb9 in_seq=0 out_seq=0 2024-05-10T11:24:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.965+0000 7fd369ffb640 10 client.16698 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:24:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.966+0000 7fd369ffb640 1 -- 172.21.15.100:0/905516263 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55882612c1e0 con 0x5588261cfce0 2024-05-10T11:24:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.966+0000 7fd369ffb640 10 client.16698 handle_command_reply: tid=0 2024-05-10T11:24:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.966+0000 7fd38b41a740 1 client.16698 shutdown 2024-05-10T11:24:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.966+0000 7fd38b41a740 1 -- 172.21.15.100:0/905516263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588261b9db0 msgr2=0x5588261c4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.966+0000 7fd38b41a740 1 --2- 172.21.15.100:0/905516263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588261b9db0 0x5588261c4060 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fd37c005ab0 tx=0x7fd37c009260 comp rx=0 tx=0).stop 2024-05-10T11:24:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.967+0000 7fd38b41a740 1 -- 172.21.15.100:0/905516263 shutdown_connections 2024-05-10T11:24:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.967+0000 7fd38b41a740 1 --2- 172.21.15.100:0/905516263 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5588261cfce0 0x5588261d2130 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.967+0000 7fd38b41a740 1 --2- 172.21.15.100:0/905516263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5588261bf910 0x5588261cb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.967+0000 7fd369ffb640 0 client.16698 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:24:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.967+0000 7fd38b41a740 1 --2- 172.21.15.100:0/905516263 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5588261ba7a0 0x5588261c45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.967+0000 7fd38b41a740 1 --2- 172.21.15.100:0/905516263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588261b9db0 0x5588261c4060 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.967+0000 7fd38b41a740 1 -- 172.21.15.100:0/905516263 >> 172.21.15.100:0/905516263 conn(0x5588261b5420 msgr2=0x5588261b4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:24:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.967+0000 7fd38b41a740 1 -- 172.21.15.100:0/905516263 shutdown_connections 2024-05-10T11:24:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.967+0000 7fd38b41a740 1 -- 172.21.15.100:0/905516263 wait complete. 2024-05-10T11:24:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.967+0000 7fd38b41a740 20 client.16698 trim_cache size 0 max 16384 2024-05-10T11:24:56.973 INFO:tasks.workunit.client.0.smithi100.stdout: 1 151499 2.73 MB/sec execute 171 sec latency 46.386 ms 2024-05-10T11:24:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.983+0000 7fd38b41a740 1 Processor -- start 2024-05-10T11:24:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.983+0000 7fd38b41a740 1 -- start start 2024-05-10T11:24:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.983+0000 7fd38b41a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5588261b9db0 0x5588261b3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.983+0000 7fd38b41a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588261ba7a0 0x5588261cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.983+0000 7fd38b41a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5588261bf910 0x5588261b4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.983+0000 7fd38b41a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd3801aca70 con 0x5588261ba7a0 2024-05-10T11:24:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.983+0000 7fd38b41a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd3801aa340 con 0x5588261b9db0 2024-05-10T11:24:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.983+0000 7fd38b41a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd3801a8190 con 0x5588261bf910 2024-05-10T11:24:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.984+0000 7fd385bb7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588261ba7a0 0x5588261cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.984+0000 7fd3863b8640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5588261b9db0 0x5588261b3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.984+0000 7fd385bb7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588261ba7a0 0x5588261cb5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38192/0 (socket says 172.21.15.100:38192) 2024-05-10T11:24:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.984+0000 7fd385bb7640 1 -- 172.21.15.100:0/789673104 learned_addr learned my addr 172.21.15.100:0/789673104 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:24:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.984+0000 7fd386bb9640 1 --2- 172.21.15.100:0/789673104 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5588261bf910 0x5588261b4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.984+0000 7fd3863b8640 1 -- 172.21.15.100:0/789673104 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5588261bf910 msgr2=0x5588261b4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.984+0000 7fd3863b8640 1 --2- 172.21.15.100:0/789673104 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5588261bf910 0x5588261b4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.984+0000 7fd3863b8640 1 -- 172.21.15.100:0/789673104 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588261ba7a0 msgr2=0x5588261cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.984+0000 7fd3863b8640 1 --2- 172.21.15.100:0/789673104 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588261ba7a0 0x5588261cb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.984+0000 7fd3863b8640 1 -- 172.21.15.100:0/789673104 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588261b99f0 con 0x5588261b9db0 2024-05-10T11:24:56.986 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.984+0000 7fd385bb7640 1 --2- 172.21.15.100:0/789673104 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588261ba7a0 0x5588261cb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:24:56.986 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.984+0000 7fd3863b8640 1 --2- 172.21.15.100:0/789673104 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5588261b9db0 0x5588261b3cf0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd37c0518e0 tx=0x7fd37c02a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92ffa08c4d0eb071 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:24:56.986 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.984+0000 7fd36bfff640 1 -- 172.21.15.100:0/789673104 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37c035070 con 0x5588261b9db0 2024-05-10T11:24:56.986 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.984+0000 7fd36bfff640 1 -- 172.21.15.100:0/789673104 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd37c04c070 con 0x5588261b9db0 2024-05-10T11:24:56.986 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.984+0000 7fd36bfff640 1 -- 172.21.15.100:0/789673104 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37c031bf0 con 0x5588261b9db0 2024-05-10T11:24:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.985+0000 7fd38b41a740 1 -- 172.21.15.100:0/789673104 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5588261b9db0 msgr2=0x5588261b3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.985+0000 7fd38b41a740 1 --2- 172.21.15.100:0/789673104 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5588261b9db0 0x5588261b3cf0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd37c0518e0 tx=0x7fd37c02a410 comp rx=0 tx=0).stop 2024-05-10T11:24:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.985+0000 7fd38b41a740 1 -- 172.21.15.100:0/789673104 shutdown_connections 2024-05-10T11:24:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.985+0000 7fd38b41a740 1 --2- 172.21.15.100:0/789673104 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5588261bf910 0x5588261b4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.985+0000 7fd38b41a740 1 --2- 172.21.15.100:0/789673104 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588261ba7a0 0x5588261cb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.985+0000 7fd38b41a740 1 --2- 172.21.15.100:0/789673104 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5588261b9db0 0x5588261b3cf0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.985+0000 7fd38b41a740 1 -- 172.21.15.100:0/789673104 >> 172.21.15.100:0/789673104 conn(0x5588261b5420 msgr2=0x5588261c3c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:24:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.985+0000 7fd38b41a740 1 -- 172.21.15.100:0/789673104 shutdown_connections 2024-05-10T11:24:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.985+0000 7fd38b41a740 1 -- 172.21.15.100:0/789673104 wait complete. 2024-05-10T11:24:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.985+0000 7fd38b41a740 1 -- start start 2024-05-10T11:24:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.986+0000 7fd38b41a740 1 Processor -- start 2024-05-10T11:24:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.986+0000 7fd38b41a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5588261b9db0 0x5588261c8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.986+0000 7fd38b41a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588261ba7a0 0x5588261c9370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.986+0000 7fd38b41a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5588261bf910 0x5588261ca330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.986+0000 7fd38b41a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd3801a8800 con 0x5588261ba7a0 2024-05-10T11:24:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.986+0000 7fd38b41a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd3801a9c70 con 0x5588261bf910 2024-05-10T11:24:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.986+0000 7fd38b41a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd3801a8ed0 con 0x5588261b9db0 2024-05-10T11:24:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.986+0000 7fd38b41a740 10 client.0 _ll_register_callbacks cb 0x7fd37c0045b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:24:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.986+0000 7fd385bb7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588261ba7a0 0x5588261c9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.987+0000 7fd3863b8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5588261b9db0 0x5588261c8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.987+0000 7fd3863b8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5588261b9db0 0x5588261c8e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:34186/0 (socket says 172.21.15.100:34186) 2024-05-10T11:24:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.987+0000 7fd386bb9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5588261bf910 0x5588261ca330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.987+0000 7fd385bb7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588261ba7a0 0x5588261c9370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38196/0 (socket says 172.21.15.100:38196) 2024-05-10T11:24:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.987+0000 7fd3863b8640 1 -- 172.21.15.100:0/1332768644 learned_addr learned my addr 172.21.15.100:0/1332768644 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:24:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.987+0000 7fd3863b8640 1 -- 172.21.15.100:0/1332768644 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5588261bf910 msgr2=0x5588261ca330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.987+0000 7fd3863b8640 1 --2- 172.21.15.100:0/1332768644 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5588261bf910 0x5588261ca330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.987+0000 7fd3863b8640 1 -- 172.21.15.100:0/1332768644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588261ba7a0 msgr2=0x5588261c9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:24:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.987+0000 7fd3863b8640 1 --2- 172.21.15.100:0/1332768644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588261ba7a0 0x5588261c9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.987+0000 7fd3863b8640 1 -- 172.21.15.100:0/1332768644 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588261ca920 con 0x5588261b9db0 2024-05-10T11:24:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.987+0000 7fd3863b8640 1 --2- 172.21.15.100:0/1332768644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5588261b9db0 0x5588261c8e30 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd37c02a930 tx=0x7fd37c051750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9ffdad510f402bb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:24:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.987+0000 7fd369ffb640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:24:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.987+0000 7fd369ffb640 1 -- 172.21.15.100:0/1332768644 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37c035040 con 0x5588261b9db0 2024-05-10T11:24:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.987+0000 7fd369ffb640 1 -- 172.21.15.100:0/1332768644 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fd37c031500 con 0x5588261b9db0 2024-05-10T11:24:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.987+0000 7fd38b41a740 1 -- 172.21.15.100:0/1332768644 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd37c04c070 con 0x5588261b9db0 2024-05-10T11:24:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.988+0000 7fd369ffb640 1 -- 172.21.15.100:0/1332768644 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fd37c031700 con 0x5588261b9db0 2024-05-10T11:24:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.988+0000 7fd38b41a740 10 client.26176 fetch_fsmap learned FSMap version 31 2024-05-10T11:24:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.988+0000 7fd38b41a740 1 -- 172.21.15.100:0/1332768644 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55882623a6b0 con 0x5588261b9db0 2024-05-10T11:24:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.988+0000 7fd369ffb640 1 -- 172.21.15.100:0/1332768644 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd37c04c070 con 0x5588261b9db0 2024-05-10T11:24:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.988+0000 7fd369ffb640 1 -- 172.21.15.100:0/1332768644 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fd37c044040 con 0x5588261b9db0 2024-05-10T11:24:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.988+0000 7fd38b41a740 10 client.26176 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:24:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.988+0000 7fd38b41a740 10 client.26176 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:24:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.988+0000 7fd38b41a740 20 client.26176 populate_metadata read hostname 'smithi100' 2024-05-10T11:24:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.988+0000 7fd38b41a740 1 --2- 172.21.15.100:0/1332768644 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55882623af70 0x55882623d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:24:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.988+0000 7fd38b41a740 4 client.26176 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:24:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.988+0000 7fd38b41a740 1 -- 172.21.15.100:0/1332768644 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558826126050 con 0x55882623af70 2024-05-10T11:24:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.988+0000 7fd385bb7640 1 --2- 172.21.15.100:0/1332768644 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55882623af70 0x55882623d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:24:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.989+0000 7fd369ffb640 10 client.26176 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:24:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:56.989+0000 7fd385bb7640 1 --2- 172.21.15.100:0/1332768644 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55882623af70 0x55882623d390 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b5c44231f8a3a7b server_cookie=3ed65e81dc549b55 in_seq=0 out_seq=0 2024-05-10T11:24:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:57.008+0000 7fd369ffb640 1 -- 172.21.15.100:0/1332768644 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558826126050 con 0x55882623af70 2024-05-10T11:24:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:57.008+0000 7fd369ffb640 10 client.26176 handle_command_reply: tid=0 2024-05-10T11:24:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:57.008+0000 7fd38b41a740 1 client.26176 shutdown 2024-05-10T11:24:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:57.008+0000 7fd38b41a740 1 -- 172.21.15.100:0/1332768644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5588261b9db0 msgr2=0x5588261c8e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:57.008+0000 7fd38b41a740 1 --2- 172.21.15.100:0/1332768644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5588261b9db0 0x5588261c8e30 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd37c02a930 tx=0x7fd37c051750 comp rx=0 tx=0).stop 2024-05-10T11:24:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:57.008+0000 7fd38b41a740 1 -- 172.21.15.100:0/1332768644 shutdown_connections 2024-05-10T11:24:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:57.008+0000 7fd38b41a740 1 --2- 172.21.15.100:0/1332768644 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55882623af70 0x55882623d390 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:57.008+0000 7fd38b41a740 1 --2- 172.21.15.100:0/1332768644 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5588261bf910 0x5588261ca330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:57.008+0000 7fd369ffb640 0 client.26176 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:24:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:57.008+0000 7fd38b41a740 1 --2- 172.21.15.100:0/1332768644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588261ba7a0 0x5588261c9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:57.008+0000 7fd38b41a740 1 --2- 172.21.15.100:0/1332768644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5588261b9db0 0x5588261c8e30 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:57.008+0000 7fd38b41a740 1 -- 172.21.15.100:0/1332768644 >> 172.21.15.100:0/1332768644 conn(0x5588261b5420 msgr2=0x5588261b8100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:24:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:57.008+0000 7fd38b41a740 1 -- 172.21.15.100:0/1332768644 shutdown_connections 2024-05-10T11:24:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:57.008+0000 7fd38b41a740 1 -- 172.21.15.100:0/1332768644 wait complete. 2024-05-10T11:24:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:57.008+0000 7fd38b41a740 20 client.26176 trim_cache size 0 max 16384 2024-05-10T11:24:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:24:57.010 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:24:57.010 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:24:57.010 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:24:57.010 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:57.009+0000 7fd38861c640 1 -- 172.21.15.100:0/3460779807 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fd36c0b28a0 msgr2=0x7fd36c0b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:57.011 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:57.009+0000 7fd38861c640 1 --2- 172.21.15.100:0/3460779807 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fd36c0b28a0 0x7fd36c0b4d60 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd37c0057c0 tx=0x7fd37c005a20 comp rx=0 tx=0).stop 2024-05-10T11:24:57.011 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:57.009+0000 7fd38861c640 1 -- 172.21.15.100:0/3460779807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd380102b00 msgr2=0x7fd380197f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:24:57.011 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:57.009+0000 7fd38861c640 1 --2- 172.21.15.100:0/3460779807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd380102b00 0x7fd380197f80 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fd37000c890 tx=0x7fd37000cd50 comp rx=0 tx=0).stop 2024-05-10T11:24:57.011 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:57.009+0000 7fd38861c640 1 -- 172.21.15.100:0/3460779807 shutdown_connections 2024-05-10T11:24:57.011 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:57.009+0000 7fd38861c640 1 --2- 172.21.15.100:0/3460779807 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fd36c0b28a0 0x7fd36c0b4d60 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:57.011 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:57.009+0000 7fd38861c640 1 --2- 172.21.15.100:0/3460779807 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3801034c0 0x7fd38019eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:57.011 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:57.009+0000 7fd38861c640 1 --2- 172.21.15.100:0/3460779807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd380102b00 0x7fd380197f80 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:57.011 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:57.009+0000 7fd38861c640 1 --2- 172.21.15.100:0/3460779807 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd380101910 0x7fd380197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:24:57.011 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:57.009+0000 7fd38861c640 1 -- 172.21.15.100:0/3460779807 >> 172.21.15.100:0/3460779807 conn(0x7fd3800fce80 msgr2=0x7fd380105f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:24:57.011 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:57.009+0000 7fd38861c640 1 -- 172.21.15.100:0/3460779807 shutdown_connections 2024-05-10T11:24:57.011 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:24:57.009+0000 7fd38861c640 1 -- 172.21.15.100:0/3460779807 wait complete. 2024-05-10T11:24:57.973 INFO:tasks.workunit.client.0.smithi100.stdout: 1 152292 2.76 MB/sec execute 172 sec latency 20.923 ms 2024-05-10T11:24:58.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:24:58 smithi178 ceph-mon[33359]: pgmap v617: 129 pgs: 129 active+clean; 70 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 1.4 MiB/s wr, 154 op/s 2024-05-10T11:24:58.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:24:58 smithi188 ceph-mon[34075]: pgmap v617: 129 pgs: 129 active+clean; 70 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 1.4 MiB/s wr, 154 op/s 2024-05-10T11:24:58.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:24:58 smithi100 ceph-mon[30296]: pgmap v617: 129 pgs: 129 active+clean; 70 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 1.4 MiB/s wr, 154 op/s 2024-05-10T11:24:58.973 INFO:tasks.workunit.client.0.smithi100.stdout: 1 152751 2.77 MB/sec execute 173 sec latency 12.036 ms 2024-05-10T11:24:59.973 INFO:tasks.workunit.client.0.smithi100.stdout: 1 152988 2.75 MB/sec execute 174 sec latency 44.400 ms 2024-05-10T11:25:00.254 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:25:00 smithi178 ceph-mon[33359]: pgmap v618: 129 pgs: 129 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 1.3 MiB/s wr, 162 op/s 2024-05-10T11:25:00.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:25:00 smithi188 ceph-mon[34075]: pgmap v618: 129 pgs: 129 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 1.3 MiB/s wr, 162 op/s 2024-05-10T11:25:00.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:25:00 smithi100 ceph-mon[30296]: pgmap v618: 129 pgs: 129 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 1.3 MiB/s wr, 162 op/s 2024-05-10T11:25:00.973 INFO:tasks.workunit.client.0.smithi100.stdout: 1 153375 2.74 MB/sec execute 175 sec latency 12.847 ms 2024-05-10T11:25:01.973 INFO:tasks.workunit.client.0.smithi100.stdout: 1 153838 2.73 MB/sec execute 176 sec latency 72.525 ms 2024-05-10T11:25:02.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:25:02 smithi188 ceph-mon[34075]: pgmap v619: 129 pgs: 129 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 1.2 MiB/s wr, 145 op/s 2024-05-10T11:25:02.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:25:02 smithi100 ceph-mon[30296]: pgmap v619: 129 pgs: 129 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 1.2 MiB/s wr, 145 op/s 2024-05-10T11:25:02.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:25:02 smithi178 ceph-mon[33359]: pgmap v619: 129 pgs: 129 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 1.2 MiB/s wr, 145 op/s 2024-05-10T11:25:02.973 INFO:tasks.workunit.client.0.smithi100.stdout: 1 154465 2.74 MB/sec execute 177 sec latency 21.208 ms 2024-05-10T11:25:03.973 INFO:tasks.workunit.client.0.smithi100.stdout: 1 155017 2.73 MB/sec execute 178 sec latency 18.261 ms 2024-05-10T11:25:04.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:25:04 smithi188 ceph-mon[34075]: pgmap v620: 129 pgs: 129 active+clean; 73 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 1.4 MiB/s wr, 151 op/s 2024-05-10T11:25:04.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:25:04 smithi100 ceph-mon[30296]: pgmap v620: 129 pgs: 129 active+clean; 73 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 1.4 MiB/s wr, 151 op/s 2024-05-10T11:25:04.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:25:04 smithi178 ceph-mon[33359]: pgmap v620: 129 pgs: 129 active+clean; 73 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 1.4 MiB/s wr, 151 op/s 2024-05-10T11:25:04.973 INFO:tasks.workunit.client.0.smithi100.stdout: 1 155825 2.76 MB/sec execute 179 sec latency 14.229 ms 2024-05-10T11:25:05.973 INFO:tasks.workunit.client.0.smithi100.stdout: 1 156209 2.76 MB/sec execute 180 sec latency 15.934 ms 2024-05-10T11:25:06.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:25:06 smithi188 ceph-mon[34075]: pgmap v621: 129 pgs: 129 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 1.3 MiB/s wr, 151 op/s 2024-05-10T11:25:06.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:25:06 smithi100 ceph-mon[30296]: pgmap v621: 129 pgs: 129 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 1.3 MiB/s wr, 151 op/s 2024-05-10T11:25:06.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:25:06 smithi178 ceph-mon[33359]: pgmap v621: 129 pgs: 129 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 1.3 MiB/s wr, 151 op/s 2024-05-10T11:25:06.973 INFO:tasks.workunit.client.0.smithi100.stdout: 1 156481 2.75 MB/sec execute 181 sec latency 16.415 ms 2024-05-10T11:25:07.973 INFO:tasks.workunit.client.0.smithi100.stdout: 1 156771 2.74 MB/sec execute 182 sec latency 13.243 ms 2024-05-10T11:25:08.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:25:08 smithi188 ceph-mon[34075]: pgmap v622: 129 pgs: 129 active+clean; 73 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 1.4 MiB/s wr, 149 op/s 2024-05-10T11:25:08.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:25:08 smithi100 ceph-mon[30296]: pgmap v622: 129 pgs: 129 active+clean; 73 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 1.4 MiB/s wr, 149 op/s 2024-05-10T11:25:08.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:25:08 smithi178 ceph-mon[33359]: pgmap v622: 129 pgs: 129 active+clean; 73 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 1.4 MiB/s wr, 149 op/s 2024-05-10T11:25:08.973 INFO:tasks.workunit.client.0.smithi100.stdout: 1 157276 2.73 MB/sec execute 183 sec latency 15.906 ms 2024-05-10T11:25:09.973 INFO:tasks.workunit.client.0.smithi100.stdout: 1 157907 2.74 MB/sec execute 184 sec latency 16.757 ms 2024-05-10T11:25:10.107 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.107+0000 7f7336615640 1 -- 172.21.15.100:0/3420441286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7330105dc0 msgr2=0x7f73301081b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:10.107 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.107+0000 7f7336615640 1 --2- 172.21.15.100:0/3420441286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7330105dc0 0x7f73301081b0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f7324009900 tx=0x7f732402a410 comp rx=0 tx=0).stop 2024-05-10T11:25:10.107 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.107+0000 7f7336615640 1 -- 172.21.15.100:0/3420441286 shutdown_connections 2024-05-10T11:25:10.108 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.107+0000 7f7336615640 1 --2- 172.21.15.100:0/3420441286 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73301086f0 0x7f733010ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:10.108 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.107+0000 7f7336615640 1 --2- 172.21.15.100:0/3420441286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7330105dc0 0x7f73301081b0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:10.108 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.107+0000 7f7336615640 1 --2- 172.21.15.100:0/3420441286 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7330103490 0x7f7330105880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:10.108 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.107+0000 7f7336615640 1 -- 172.21.15.100:0/3420441286 >> 172.21.15.100:0/3420441286 conn(0x7f73300fce80 msgr2=0x7f73300ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:25:10.108 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.107+0000 7f7336615640 1 -- 172.21.15.100:0/3420441286 shutdown_connections 2024-05-10T11:25:10.108 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.107+0000 7f7336615640 1 -- 172.21.15.100:0/3420441286 wait complete. 2024-05-10T11:25:10.108 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.107+0000 7f7336615640 1 Processor -- start 2024-05-10T11:25:10.108 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.108+0000 7f7336615640 1 -- start start 2024-05-10T11:25:10.110 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.108+0000 7f7336615640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7330103490 0x7f7330198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:10.110 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.108+0000 7f7336615640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7330105dc0 0x7f7330199020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:10.110 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.108+0000 7f7336615640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73301086f0 0x7f73301a0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:10.110 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.108+0000 7f7336615640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f733010e3e0 con 0x7f7330103490 2024-05-10T11:25:10.110 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.108+0000 7f7336615640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f733010e260 con 0x7f7330105dc0 2024-05-10T11:25:10.110 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.108+0000 7f7336615640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f733010e560 con 0x7f73301086f0 2024-05-10T11:25:10.110 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.108+0000 7f732ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7330103490 0x7f7330198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:10.110 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.108+0000 7f732ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7330103490 0x7f7330198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53918/0 (socket says 172.21.15.100:53918) 2024-05-10T11:25:10.110 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.108+0000 7f732ffff640 1 -- 172.21.15.100:0/3581973819 learned_addr learned my addr 172.21.15.100:0/3581973819 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:25:10.110 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.108+0000 7f7334bb2640 1 --2- 172.21.15.100:0/3581973819 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73301086f0 0x7f73301a0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:10.111 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.108+0000 7f732f7fe640 1 --2- 172.21.15.100:0/3581973819 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7330105dc0 0x7f7330199020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:10.111 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.108+0000 7f732ffff640 1 -- 172.21.15.100:0/3581973819 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73301086f0 msgr2=0x7f73301a0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:10.111 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.108+0000 7f732ffff640 1 --2- 172.21.15.100:0/3581973819 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73301086f0 0x7f73301a0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:10.111 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.108+0000 7f732ffff640 1 -- 172.21.15.100:0/3581973819 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7330105dc0 msgr2=0x7f7330199020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:10.111 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.108+0000 7f732ffff640 1 --2- 172.21.15.100:0/3581973819 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7330105dc0 0x7f7330199020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:10.111 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.108+0000 7f732ffff640 1 -- 172.21.15.100:0/3581973819 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73301a0700 con 0x7f7330103490 2024-05-10T11:25:10.111 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.108+0000 7f732ffff640 1 --2- 172.21.15.100:0/3581973819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7330103490 0x7f7330198ae0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f732000d950 tx=0x7f732000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:25:10.111 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.109+0000 7f732d7fa640 1 -- 172.21.15.100:0/3581973819 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7320017070 con 0x7f7330103490 2024-05-10T11:25:10.111 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.109+0000 7f732d7fa640 1 -- 172.21.15.100:0/3581973819 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7320009c30 con 0x7f7330103490 2024-05-10T11:25:10.111 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.109+0000 7f7336615640 1 -- 172.21.15.100:0/3581973819 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73301a09f0 con 0x7f7330103490 2024-05-10T11:25:10.111 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.109+0000 7f732d7fa640 1 -- 172.21.15.100:0/3581973819 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f732000f620 con 0x7f7330103490 2024-05-10T11:25:10.111 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.109+0000 7f7336615640 1 -- 172.21.15.100:0/3581973819 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73301a0f10 con 0x7f7330103490 2024-05-10T11:25:10.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.111+0000 7f732d7fa640 1 -- 172.21.15.100:0/3581973819 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f7320010460 con 0x7f7330103490 2024-05-10T11:25:10.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.111+0000 7f7339409740 1 Processor -- start 2024-05-10T11:25:10.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.111+0000 7f7339409740 1 -- start start 2024-05-10T11:25:10.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.112+0000 7f7339409740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd5a1be1c0 0x55dd5a1be5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:10.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.112+0000 7f7339409740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd5a1bebb0 0x55dd5a1c36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:10.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.112+0000 7f7339409740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd5a1c3d20 0x55dd5a1c61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:10.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.112+0000 7f7339409740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55dd5a11e5d0 con 0x55dd5a1bebb0 2024-05-10T11:25:10.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.112+0000 7f7339409740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55dd5a0d7890 con 0x55dd5a1c3d20 2024-05-10T11:25:10.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.112+0000 7f7339409740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55dd5a11a590 con 0x55dd5a1be1c0 2024-05-10T11:25:10.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.112+0000 7f732ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd5a1bebb0 0x55dd5a1c36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:10.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.112+0000 7f7334bb2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd5a1be1c0 0x55dd5a1be5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:10.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.112+0000 7f732ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd5a1bebb0 0x55dd5a1c36b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53920/0 (socket says 172.21.15.100:53920) 2024-05-10T11:25:10.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.112+0000 7f732ffff640 1 -- 172.21.15.100:0/2996605271 learned_addr learned my addr 172.21.15.100:0/2996605271 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:25:10.118 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.112+0000 7f732f7fe640 1 --2- 172.21.15.100:0/2996605271 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd5a1c3d20 0x55dd5a1c61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:10.118 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.112+0000 7f732ffff640 1 -- 172.21.15.100:0/2996605271 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd5a1be1c0 msgr2=0x55dd5a1be5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:10.118 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.112+0000 7f732ffff640 1 --2- 172.21.15.100:0/2996605271 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd5a1be1c0 0x55dd5a1be5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:10.118 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.112+0000 7f732d7fa640 1 --2- 172.21.15.100:0/3581973819 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f73180b28a0 0x7f73180b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:10.118 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.112+0000 7f732ffff640 1 -- 172.21.15.100:0/2996605271 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd5a1c3d20 msgr2=0x55dd5a1c61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:10.118 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.112+0000 7f732ffff640 1 --2- 172.21.15.100:0/2996605271 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd5a1c3d20 0x55dd5a1c61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:10.118 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.112+0000 7f732ffff640 1 -- 172.21.15.100:0/2996605271 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd59e30210 con 0x55dd5a1bebb0 2024-05-10T11:25:10.118 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.112+0000 7f732f7fe640 1 --2- 172.21.15.100:0/2996605271 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd5a1c3d20 0x55dd5a1c61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:25:10.118 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.112+0000 7f7334bb2640 1 --2- 172.21.15.100:0/3581973819 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f73180b28a0 0x7f73180b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:10.118 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.112+0000 7f732d7fa640 1 -- 172.21.15.100:0/3581973819 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f73200e20f0 con 0x7f7330103490 2024-05-10T11:25:10.118 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.112+0000 7f732ffff640 1 --2- 172.21.15.100:0/2996605271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd5a1bebb0 0x55dd5a1c36b0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f73200c36b0 tx=0x7f73200baa60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b94388cf1ca365a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:25:10.118 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.113+0000 7f73167fc640 1 -- 172.21.15.100:0/2996605271 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7320020030 con 0x55dd5a1bebb0 2024-05-10T11:25:10.118 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.113+0000 7f73167fc640 1 -- 172.21.15.100:0/2996605271 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f73200a8f70 con 0x55dd5a1bebb0 2024-05-10T11:25:10.118 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.113+0000 7f73167fc640 1 -- 172.21.15.100:0/2996605271 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73200a8bd0 con 0x55dd5a1bebb0 2024-05-10T11:25:10.119 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.113+0000 7f7339409740 1 -- 172.21.15.100:0/2996605271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd5a1bebb0 msgr2=0x55dd5a1c36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:10.119 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.113+0000 7f7339409740 1 --2- 172.21.15.100:0/2996605271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd5a1bebb0 0x55dd5a1c36b0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f73200c36b0 tx=0x7f73200baa60 comp rx=0 tx=0).stop 2024-05-10T11:25:10.119 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.113+0000 7f7334bb2640 1 --2- 172.21.15.100:0/3581973819 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f73180b28a0 0x7f73180b4d60 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x55dd5a1bf410 tx=0x7f7328009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:25:10.119 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.113+0000 7f7339409740 1 -- 172.21.15.100:0/2996605271 shutdown_connections 2024-05-10T11:25:10.119 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.113+0000 7f7339409740 1 --2- 172.21.15.100:0/2996605271 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd5a1c3d20 0x55dd5a1c61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:10.119 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.113+0000 7f7339409740 1 --2- 172.21.15.100:0/2996605271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd5a1bebb0 0x55dd5a1c36b0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:10.119 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.113+0000 7f7339409740 1 --2- 172.21.15.100:0/2996605271 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd5a1be1c0 0x55dd5a1be5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:10.119 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.113+0000 7f7339409740 1 -- 172.21.15.100:0/2996605271 >> 172.21.15.100:0/2996605271 conn(0x55dd5a1b9830 msgr2=0x55dd5a1b9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:25:10.119 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.113+0000 7f7339409740 1 -- 172.21.15.100:0/2996605271 shutdown_connections 2024-05-10T11:25:10.119 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.113+0000 7f7339409740 1 -- 172.21.15.100:0/2996605271 wait complete. 2024-05-10T11:25:10.120 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.114+0000 7f7339409740 1 -- start start 2024-05-10T11:25:10.120 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.114+0000 7f7339409740 1 Processor -- start 2024-05-10T11:25:10.120 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.114+0000 7f7339409740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd5a1be1c0 0x55dd5a1c8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:10.120 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.114+0000 7f7339409740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd5a1bebb0 0x55dd5a1c8970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:10.120 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.114+0000 7f7339409740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd5a1c3d20 0x55dd5a1cf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:10.120 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.114+0000 7f7339409740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55dd5a1c6cd0 con 0x55dd5a1c3d20 2024-05-10T11:25:10.120 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.114+0000 7f7339409740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55dd5a0db370 con 0x55dd5a1bebb0 2024-05-10T11:25:10.120 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.114+0000 7f7339409740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55dd5a1c6e50 con 0x55dd5a1be1c0 2024-05-10T11:25:10.120 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.114+0000 7f7339409740 10 client.0 _ll_register_callbacks cb 0x55dd59e2fa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:25:10.120 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.114+0000 7f7334bb2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd5a1c3d20 0x55dd5a1cf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:10.120 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.114+0000 7f7334bb2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd5a1c3d20 0x55dd5a1cf9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53924/0 (socket says 172.21.15.100:53924) 2024-05-10T11:25:10.120 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.114+0000 7f7334bb2640 1 -- 172.21.15.100:0/1566023762 learned_addr learned my addr 172.21.15.100:0/1566023762 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:25:10.120 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.114+0000 7f732ffff640 1 --2- 172.21.15.100:0/1566023762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd5a1be1c0 0x55dd5a1c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:10.120 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.114+0000 7f732f7fe640 1 --2- 172.21.15.100:0/1566023762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd5a1bebb0 0x55dd5a1c8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:10.121 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.115+0000 7f7334bb2640 1 -- 172.21.15.100:0/1566023762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd5a1be1c0 msgr2=0x55dd5a1c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:10.121 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.115+0000 7f7334bb2640 1 --2- 172.21.15.100:0/1566023762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd5a1be1c0 0x55dd5a1c8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:10.121 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.115+0000 7f7334bb2640 1 -- 172.21.15.100:0/1566023762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd5a1bebb0 msgr2=0x55dd5a1c8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:10.121 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.115+0000 7f7334bb2640 1 --2- 172.21.15.100:0/1566023762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd5a1bebb0 0x55dd5a1c8970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:10.121 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.115+0000 7f7334bb2640 1 -- 172.21.15.100:0/1566023762 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd59efc620 con 0x55dd5a1c3d20 2024-05-10T11:25:10.121 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.115+0000 7f732ffff640 1 --2- 172.21.15.100:0/1566023762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd5a1be1c0 0x55dd5a1c8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:25:10.121 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.115+0000 7f732f7fe640 1 --2- 172.21.15.100:0/1566023762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd5a1bebb0 0x55dd5a1c8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:25:10.121 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.115+0000 7f7334bb2640 1 --2- 172.21.15.100:0/1566023762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd5a1c3d20 0x55dd5a1cf9b0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f7328004070 tx=0x7f7328004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55f0c7570d9c82e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:25:10.121 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.115+0000 7f730f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:25:10.121 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.115+0000 7f730f7fe640 1 -- 172.21.15.100:0/1566023762 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7328004b20 con 0x55dd5a1c3d20 2024-05-10T11:25:10.121 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.115+0000 7f730f7fe640 1 -- 172.21.15.100:0/1566023762 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f7328004cc0 con 0x55dd5a1c3d20 2024-05-10T11:25:10.121 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.115+0000 7f7339409740 1 -- 172.21.15.100:0/1566023762 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73200a8f70 con 0x55dd5a1c3d20 2024-05-10T11:25:10.121 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.115+0000 7f730f7fe640 1 -- 172.21.15.100:0/1566023762 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f732800f8b0 con 0x55dd5a1c3d20 2024-05-10T11:25:10.121 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.115+0000 7f7339409740 10 client.16716 fetch_fsmap learned FSMap version 31 2024-05-10T11:25:10.121 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.115+0000 7f7339409740 1 -- 172.21.15.100:0/1566023762 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55dd5a1d2a60 con 0x55dd5a1c3d20 2024-05-10T11:25:10.122 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.115+0000 7f730f7fe640 1 -- 172.21.15.100:0/1566023762 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f73200a8f70 con 0x55dd5a1c3d20 2024-05-10T11:25:10.122 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.116+0000 7f730f7fe640 1 -- 172.21.15.100:0/1566023762 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f7328026070 con 0x55dd5a1c3d20 2024-05-10T11:25:10.122 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.116+0000 7f7339409740 10 client.16716 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:25:10.122 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.116+0000 7f7339409740 10 client.16716 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:25:10.122 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.116+0000 7f7339409740 20 client.16716 populate_metadata read hostname 'smithi100' 2024-05-10T11:25:10.122 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.116+0000 7f7339409740 1 --2- 172.21.15.100:0/1566023762 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55dd5a1d41a0 0x55dd5a1d65b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:10.122 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.116+0000 7f7339409740 4 client.16716 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:25:10.122 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.116+0000 7f7339409740 1 -- 172.21.15.100:0/1566023762 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd5a12c1e0 con 0x55dd5a1d41a0 2024-05-10T11:25:10.122 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.116+0000 7f732ffff640 1 --2- 172.21.15.100:0/1566023762 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55dd5a1d41a0 0x55dd5a1d65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:10.122 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.116+0000 7f730f7fe640 10 client.16716 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:25:10.122 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.116+0000 7f732ffff640 1 --2- 172.21.15.100:0/1566023762 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55dd5a1d41a0 0x55dd5a1d65b0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20e1924def1297ed server_cookie=ea56133c48bd05d9 in_seq=0 out_seq=0 2024-05-10T11:25:10.123 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.117+0000 7f730f7fe640 1 -- 172.21.15.100:0/1566023762 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dd5a12c1e0 con 0x55dd5a1d41a0 2024-05-10T11:25:10.124 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.117+0000 7f730f7fe640 10 client.16716 handle_command_reply: tid=0 2024-05-10T11:25:10.124 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.117+0000 7f7339409740 1 client.16716 shutdown 2024-05-10T11:25:10.124 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.118+0000 7f7339409740 1 -- 172.21.15.100:0/1566023762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd5a1c3d20 msgr2=0x55dd5a1cf9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:10.124 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.118+0000 7f7339409740 1 --2- 172.21.15.100:0/1566023762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd5a1c3d20 0x55dd5a1cf9b0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f7328004070 tx=0x7f7328004ab0 comp rx=0 tx=0).stop 2024-05-10T11:25:10.124 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.118+0000 7f7339409740 1 -- 172.21.15.100:0/1566023762 shutdown_connections 2024-05-10T11:25:10.124 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.118+0000 7f7339409740 1 --2- 172.21.15.100:0/1566023762 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55dd5a1d41a0 0x55dd5a1d65b0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:10.124 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.118+0000 7f7339409740 1 --2- 172.21.15.100:0/1566023762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd5a1c3d20 0x55dd5a1cf9b0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:10.124 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.118+0000 7f7339409740 1 --2- 172.21.15.100:0/1566023762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd5a1bebb0 0x55dd5a1c8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:10.124 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.118+0000 7f7339409740 1 --2- 172.21.15.100:0/1566023762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd5a1be1c0 0x55dd5a1c8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:10.124 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.118+0000 7f7339409740 1 -- 172.21.15.100:0/1566023762 >> 172.21.15.100:0/1566023762 conn(0x55dd5a1b9830 msgr2=0x55dd5a1b4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:25:10.124 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.118+0000 7f730f7fe640 0 client.16716 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:25:10.124 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.118+0000 7f7339409740 1 -- 172.21.15.100:0/1566023762 shutdown_connections 2024-05-10T11:25:10.124 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.118+0000 7f7339409740 1 -- 172.21.15.100:0/1566023762 wait complete. 2024-05-10T11:25:10.124 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.118+0000 7f7339409740 20 client.16716 trim_cache size 0 max 16384 2024-05-10T11:25:10.135 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.134+0000 7f7339409740 1 Processor -- start 2024-05-10T11:25:10.135 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.134+0000 7f7339409740 1 -- start start 2024-05-10T11:25:10.135 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.134+0000 7f7339409740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd5a1be1c0 0x55dd5a1b03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:10.135 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.134+0000 7f7339409740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd5a1bebb0 0x55dd5a1cf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.135+0000 7f7339409740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd5a1c3d20 0x55dd5a1c8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.135+0000 7f7339409740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f73301aba60 con 0x55dd5a1c3d20 2024-05-10T11:25:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.135+0000 7f7339409740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f73301a9330 con 0x55dd5a1bebb0 2024-05-10T11:25:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.135+0000 7f7339409740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f73301a7180 con 0x55dd5a1be1c0 2024-05-10T11:25:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.135+0000 7f7334bb2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd5a1c3d20 0x55dd5a1c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.135+0000 7f732ffff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd5a1be1c0 0x55dd5a1b03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.135+0000 7f7334bb2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd5a1c3d20 0x55dd5a1c8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53932/0 (socket says 172.21.15.100:53932) 2024-05-10T11:25:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.135+0000 7f732f7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd5a1bebb0 0x55dd5a1cf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.135+0000 7f7334bb2640 1 -- 172.21.15.100:0/3640059543 learned_addr learned my addr 172.21.15.100:0/3640059543 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:25:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.135+0000 7f732ffff640 1 -- 172.21.15.100:0/3640059543 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd5a1bebb0 msgr2=0x55dd5a1cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.135+0000 7f732ffff640 1 --2- 172.21.15.100:0/3640059543 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd5a1bebb0 0x55dd5a1cf970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.135+0000 7f732ffff640 1 -- 172.21.15.100:0/3640059543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd5a1c3d20 msgr2=0x55dd5a1c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.135+0000 7f732ffff640 1 --2- 172.21.15.100:0/3640059543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd5a1c3d20 0x55dd5a1c8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.135+0000 7f732ffff640 1 -- 172.21.15.100:0/3640059543 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd5a1b0920 con 0x55dd5a1be1c0 2024-05-10T11:25:10.137 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.135+0000 7f7334bb2640 1 --2- 172.21.15.100:0/3640059543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd5a1c3d20 0x55dd5a1c8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:25:10.137 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.135+0000 7f732ffff640 1 --2- 172.21.15.100:0/3640059543 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd5a1be1c0 0x55dd5a1b03e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x55dd5a1befb0 tx=0x7f73200e5bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91db4116dbb1f6ce server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:25:10.137 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.135+0000 7f7315ffb640 1 -- 172.21.15.100:0/3640059543 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7320020040 con 0x55dd5a1be1c0 2024-05-10T11:25:10.137 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.135+0000 7f7315ffb640 1 -- 172.21.15.100:0/3640059543 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f73200a8f70 con 0x55dd5a1be1c0 2024-05-10T11:25:10.137 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.135+0000 7f7315ffb640 1 -- 172.21.15.100:0/3640059543 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73200ad760 con 0x55dd5a1be1c0 2024-05-10T11:25:10.140 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.136+0000 7f7339409740 1 -- 172.21.15.100:0/3640059543 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd5a1be1c0 msgr2=0x55dd5a1b03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:10.140 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.136+0000 7f7339409740 1 --2- 172.21.15.100:0/3640059543 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd5a1be1c0 0x55dd5a1b03e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x55dd5a1befb0 tx=0x7f73200e5bd0 comp rx=0 tx=0).stop 2024-05-10T11:25:10.140 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.136+0000 7f7339409740 1 -- 172.21.15.100:0/3640059543 shutdown_connections 2024-05-10T11:25:10.140 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.136+0000 7f7339409740 1 --2- 172.21.15.100:0/3640059543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd5a1c3d20 0x55dd5a1c8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:10.140 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.136+0000 7f7339409740 1 --2- 172.21.15.100:0/3640059543 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd5a1bebb0 0x55dd5a1cf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:10.140 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.136+0000 7f7339409740 1 --2- 172.21.15.100:0/3640059543 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd5a1be1c0 0x55dd5a1b03e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.136+0000 7f7339409740 1 -- 172.21.15.100:0/3640059543 >> 172.21.15.100:0/3640059543 conn(0x55dd5a1b9830 msgr2=0x55dd5a1b9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:25:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.136+0000 7f7339409740 1 -- 172.21.15.100:0/3640059543 shutdown_connections 2024-05-10T11:25:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.136+0000 7f7339409740 1 -- 172.21.15.100:0/3640059543 wait complete. 2024-05-10T11:25:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.137+0000 7f7339409740 1 -- start start 2024-05-10T11:25:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.137+0000 7f7339409740 1 Processor -- start 2024-05-10T11:25:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.137+0000 7f7339409740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd5a1be1c0 0x55dd5a1cf650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.137+0000 7f7339409740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd5a1bebb0 0x55dd5a1cc140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.138+0000 7f732f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd5a1bebb0 0x55dd5a1cc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.137+0000 7f7339409740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd5a1c3d20 0x55dd5a1cc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.138+0000 7f732f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd5a1bebb0 0x55dd5a1cc140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53944/0 (socket says 172.21.15.100:53944) 2024-05-10T11:25:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.138+0000 7f732f7fe640 1 -- 172.21.15.100:0/3239228164 learned_addr learned my addr 172.21.15.100:0/3239228164 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:25:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.138+0000 7f7339409740 1 -- 172.21.15.100:0/3239228164 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f73301a77f0 con 0x55dd5a1bebb0 2024-05-10T11:25:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.138+0000 7f7339409740 1 -- 172.21.15.100:0/3239228164 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f73301a8c60 con 0x55dd5a1be1c0 2024-05-10T11:25:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.138+0000 7f7339409740 1 -- 172.21.15.100:0/3239228164 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f73301a7ec0 con 0x55dd5a1c3d20 2024-05-10T11:25:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.138+0000 7f7339409740 10 client.0 _ll_register_callbacks cb 0x7f7328009e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:25:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.138+0000 7f732ffff640 1 --2- 172.21.15.100:0/3239228164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd5a1be1c0 0x55dd5a1cf650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:10.142 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.138+0000 7f7334bb2640 1 --2- 172.21.15.100:0/3239228164 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd5a1c3d20 0x55dd5a1cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:10.142 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.138+0000 7f732f7fe640 1 -- 172.21.15.100:0/3239228164 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd5a1c3d20 msgr2=0x55dd5a1cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:10.142 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.138+0000 7f732f7fe640 1 --2- 172.21.15.100:0/3239228164 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd5a1c3d20 0x55dd5a1cc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:10.142 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.138+0000 7f732f7fe640 1 -- 172.21.15.100:0/3239228164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd5a1be1c0 msgr2=0x55dd5a1cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:10.142 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.138+0000 7f732f7fe640 1 --2- 172.21.15.100:0/3239228164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd5a1be1c0 0x55dd5a1cf650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:10.142 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.138+0000 7f732f7fe640 1 -- 172.21.15.100:0/3239228164 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd5a1cccc0 con 0x55dd5a1bebb0 2024-05-10T11:25:10.142 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.138+0000 7f732f7fe640 1 --2- 172.21.15.100:0/3239228164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd5a1bebb0 0x55dd5a1cc140 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f7324000c00 tx=0x7f7324031940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=835b4d5e30fdc974 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:25:10.142 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.138+0000 7f730f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:25:10.142 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.139+0000 7f730f7fe640 1 -- 172.21.15.100:0/3239228164 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7324049060 con 0x55dd5a1bebb0 2024-05-10T11:25:10.142 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.139+0000 7f730f7fe640 1 -- 172.21.15.100:0/3239228164 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f7324004240 con 0x55dd5a1bebb0 2024-05-10T11:25:10.142 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.139+0000 7f7339409740 1 -- 172.21.15.100:0/3239228164 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73200a8f70 con 0x55dd5a1bebb0 2024-05-10T11:25:10.142 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.139+0000 7f730f7fe640 1 -- 172.21.15.100:0/3239228164 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7324050050 con 0x55dd5a1bebb0 2024-05-10T11:25:10.142 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.139+0000 7f7339409740 10 client.16722 fetch_fsmap learned FSMap version 31 2024-05-10T11:25:10.142 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.139+0000 7f7339409740 1 -- 172.21.15.100:0/3239228164 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55dd5a23dba0 con 0x55dd5a1bebb0 2024-05-10T11:25:10.143 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.139+0000 7f730f7fe640 1 -- 172.21.15.100:0/3239228164 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f73200a8f70 con 0x55dd5a1bebb0 2024-05-10T11:25:10.143 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.139+0000 7f730f7fe640 1 -- 172.21.15.100:0/3239228164 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f732403b430 con 0x55dd5a1bebb0 2024-05-10T11:25:10.143 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.139+0000 7f7339409740 10 client.16722 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:25:10.143 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.139+0000 7f7339409740 10 client.16722 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:25:10.143 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.139+0000 7f7339409740 20 client.16722 populate_metadata read hostname 'smithi100' 2024-05-10T11:25:10.143 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.139+0000 7f7339409740 1 --2- 172.21.15.100:0/3239228164 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55dd5a23e580 0x55dd5a2409a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:10.143 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.139+0000 7f7339409740 4 client.16722 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:25:10.143 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.139+0000 7f7339409740 1 -- 172.21.15.100:0/3239228164 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dd5a126050 con 0x55dd5a23e580 2024-05-10T11:25:10.143 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.139+0000 7f732ffff640 1 --2- 172.21.15.100:0/3239228164 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55dd5a23e580 0x55dd5a2409a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:10.143 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.140+0000 7f730f7fe640 10 client.16722 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:25:10.143 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.140+0000 7f732ffff640 1 --2- 172.21.15.100:0/3239228164 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55dd5a23e580 0x55dd5a2409a0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b54a3380dfd0ad70 server_cookie=fcc9a670c279c519 in_seq=0 out_seq=0 2024-05-10T11:25:10.161 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.161+0000 7f730f7fe640 1 -- 172.21.15.100:0/3239228164 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dd5a126050 con 0x55dd5a23e580 2024-05-10T11:25:10.161 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.161+0000 7f730f7fe640 10 client.16722 handle_command_reply: tid=0 2024-05-10T11:25:10.162 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.161+0000 7f7339409740 1 client.16722 shutdown 2024-05-10T11:25:10.162 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.161+0000 7f7339409740 1 -- 172.21.15.100:0/3239228164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd5a1bebb0 msgr2=0x55dd5a1cc140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:10.162 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.161+0000 7f7339409740 1 --2- 172.21.15.100:0/3239228164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd5a1bebb0 0x55dd5a1cc140 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f7324000c00 tx=0x7f7324031940 comp rx=0 tx=0).stop 2024-05-10T11:25:10.162 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.161+0000 7f7339409740 1 -- 172.21.15.100:0/3239228164 shutdown_connections 2024-05-10T11:25:10.162 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.161+0000 7f7339409740 1 --2- 172.21.15.100:0/3239228164 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55dd5a23e580 0x55dd5a2409a0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:10.162 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.161+0000 7f7339409740 1 --2- 172.21.15.100:0/3239228164 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd5a1c3d20 0x55dd5a1cc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:10.162 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.161+0000 7f7339409740 1 --2- 172.21.15.100:0/3239228164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd5a1bebb0 0x55dd5a1cc140 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:10.162 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.161+0000 7f7339409740 1 --2- 172.21.15.100:0/3239228164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd5a1be1c0 0x55dd5a1cf650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:10.162 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.161+0000 7f7339409740 1 -- 172.21.15.100:0/3239228164 >> 172.21.15.100:0/3239228164 conn(0x55dd5a1b9830 msgr2=0x55dd5a1c8430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:25:10.162 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.161+0000 7f730f7fe640 0 client.16722 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:25:10.162 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.161+0000 7f7339409740 1 -- 172.21.15.100:0/3239228164 shutdown_connections 2024-05-10T11:25:10.162 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.161+0000 7f7339409740 1 -- 172.21.15.100:0/3239228164 wait complete. 2024-05-10T11:25:10.162 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.161+0000 7f7339409740 20 client.16722 trim_cache size 0 max 16384 2024-05-10T11:25:10.163 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:25:10.163 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:25:10.163 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:25:10.163 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:25:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.162+0000 7f7336615640 1 -- 172.21.15.100:0/3581973819 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f73180b28a0 msgr2=0x7f73180b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.162+0000 7f7336615640 1 --2- 172.21.15.100:0/3581973819 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f73180b28a0 0x7f73180b4d60 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x55dd5a1bf410 tx=0x7f7328009500 comp rx=0 tx=0).stop 2024-05-10T11:25:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.162+0000 7f7336615640 1 -- 172.21.15.100:0/3581973819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7330103490 msgr2=0x7f7330198ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.162+0000 7f7336615640 1 --2- 172.21.15.100:0/3581973819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7330103490 0x7f7330198ae0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f732000d950 tx=0x7f732000de10 comp rx=0 tx=0).stop 2024-05-10T11:25:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.162+0000 7f7336615640 1 -- 172.21.15.100:0/3581973819 shutdown_connections 2024-05-10T11:25:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.162+0000 7f7336615640 1 --2- 172.21.15.100:0/3581973819 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f73180b28a0 0x7f73180b4d60 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.162+0000 7f7336615640 1 --2- 172.21.15.100:0/3581973819 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73301086f0 0x7f73301a0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.162+0000 7f7336615640 1 --2- 172.21.15.100:0/3581973819 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7330105dc0 0x7f7330199020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.162+0000 7f7336615640 1 --2- 172.21.15.100:0/3581973819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7330103490 0x7f7330198ae0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.162+0000 7f7336615640 1 -- 172.21.15.100:0/3581973819 >> 172.21.15.100:0/3581973819 conn(0x7f73300fce80 msgr2=0x7f73301052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:25:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.162+0000 7f7336615640 1 -- 172.21.15.100:0/3581973819 shutdown_connections 2024-05-10T11:25:10.165 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:10.162+0000 7f7336615640 1 -- 172.21.15.100:0/3581973819 wait complete. 2024-05-10T11:25:10.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:25:10 smithi188 ceph-mon[34075]: pgmap v623: 129 pgs: 129 active+clean; 73 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 1.3 MiB/s wr, 144 op/s 2024-05-10T11:25:10.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:25:10 smithi100 ceph-mon[30296]: pgmap v623: 129 pgs: 129 active+clean; 73 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 1.3 MiB/s wr, 144 op/s 2024-05-10T11:25:10.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:25:10 smithi178 ceph-mon[33359]: pgmap v623: 129 pgs: 129 active+clean; 73 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 1.3 MiB/s wr, 144 op/s 2024-05-10T11:25:10.973 INFO:tasks.workunit.client.0.smithi100.stdout: 1 158505 2.73 MB/sec execute 185 sec latency 19.331 ms 2024-05-10T11:25:11.973 INFO:tasks.workunit.client.0.smithi100.stdout: 1 159098 2.74 MB/sec execute 186 sec latency 78.418 ms 2024-05-10T11:25:12.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:25:12 smithi188 ceph-mon[34075]: pgmap v624: 129 pgs: 129 active+clean; 74 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 1.2 MiB/s wr, 130 op/s 2024-05-10T11:25:12.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:25:12 smithi100 ceph-mon[30296]: pgmap v624: 129 pgs: 129 active+clean; 74 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 1.2 MiB/s wr, 130 op/s 2024-05-10T11:25:12.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:25:12 smithi178 ceph-mon[33359]: pgmap v624: 129 pgs: 129 active+clean; 74 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 1.2 MiB/s wr, 130 op/s 2024-05-10T11:25:12.973 INFO:tasks.workunit.client.0.smithi100.stdout: 1 159643 2.76 MB/sec execute 187 sec latency 13.928 ms 2024-05-10T11:25:13.973 INFO:tasks.workunit.client.0.smithi100.stdout: 1 159978 2.75 MB/sec execute 188 sec latency 14.840 ms 2024-05-10T11:25:14.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:25:14 smithi188 ceph-mon[34075]: pgmap v625: 129 pgs: 129 active+clean; 74 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 1.3 MiB/s wr, 150 op/s 2024-05-10T11:25:14.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:25:14 smithi100 ceph-mon[30296]: pgmap v625: 129 pgs: 129 active+clean; 74 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 1.3 MiB/s wr, 150 op/s 2024-05-10T11:25:14.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:25:14 smithi178 ceph-mon[33359]: pgmap v625: 129 pgs: 129 active+clean; 74 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 1.3 MiB/s wr, 150 op/s 2024-05-10T11:25:14.973 INFO:tasks.workunit.client.0.smithi100.stdout: 1 160238 2.74 MB/sec execute 189 sec latency 15.716 ms 2024-05-10T11:25:15.973 INFO:tasks.workunit.client.0.smithi100.stdout: 1 160711 2.73 MB/sec execute 190 sec latency 16.264 ms 2024-05-10T11:25:16.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:25:16 smithi188 ceph-mon[34075]: pgmap v626: 129 pgs: 129 active+clean; 70 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 1.3 MiB/s wr, 150 op/s 2024-05-10T11:25:16.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:25:16 smithi100 ceph-mon[30296]: pgmap v626: 129 pgs: 129 active+clean; 70 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 1.3 MiB/s wr, 150 op/s 2024-05-10T11:25:16.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:25:16 smithi178 ceph-mon[33359]: pgmap v626: 129 pgs: 129 active+clean; 70 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 1.3 MiB/s wr, 150 op/s 2024-05-10T11:25:16.973 INFO:tasks.workunit.client.0.smithi100.stdout: 1 161261 2.73 MB/sec execute 191 sec latency 17.231 ms 2024-05-10T11:25:17.139 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T11:25:17.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:25:17 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:25:17.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:25:17 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:25:17.418 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:25:17.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:25:17 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:25:17.974 INFO:tasks.workunit.client.0.smithi100.stdout: 1 161931 2.73 MB/sec execute 192 sec latency 15.216 ms 2024-05-10T11:25:18.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:25:18 smithi188 ceph-mon[34075]: pgmap v627: 129 pgs: 129 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 1.4 MiB/s wr, 153 op/s 2024-05-10T11:25:18.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:25:18 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:25:18.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:25:18 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:25:18.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:25:18 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:25:18.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:25:18 smithi100 ceph-mon[30296]: pgmap v627: 129 pgs: 129 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 1.4 MiB/s wr, 153 op/s 2024-05-10T11:25:18.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:25:18 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:25:18.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:25:18 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:25:18.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:25:18 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:25:18.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:25:18 smithi178 ceph-mon[33359]: pgmap v627: 129 pgs: 129 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 1.4 MiB/s wr, 153 op/s 2024-05-10T11:25:18.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:25:18 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:25:18.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:25:18 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:25:18.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:25:18 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:25:18.974 INFO:tasks.workunit.client.0.smithi100.stdout: 1 162536 2.74 MB/sec execute 193 sec latency 15.270 ms 2024-05-10T11:25:19.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.409+0000 7f59a285e700 1 -- 172.21.15.100:0/3835869349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f599c103550 msgr2=0x7f599c1039b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:19.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.409+0000 7f59a285e700 1 --2- 172.21.15.100:0/3835869349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f599c103550 0x7f599c1039b0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f5984009990 tx=0x7f5984009ca0 comp rx=0 tx=0).stop 2024-05-10T11:25:19.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.409+0000 7f59a285e700 1 -- 172.21.15.100:0/3835869349 shutdown_connections 2024-05-10T11:25:19.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.409+0000 7f59a285e700 1 --2- 172.21.15.100:0/3835869349 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f599c103ef0 0x7f599c1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:19.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.409+0000 7f59a285e700 1 --2- 172.21.15.100:0/3835869349 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f599c108460 0x7f599c108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:19.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.409+0000 7f59a285e700 1 --2- 172.21.15.100:0/3835869349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f599c103550 0x7f599c1039b0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:19.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.409+0000 7f59a285e700 1 -- 172.21.15.100:0/3835869349 >> 172.21.15.100:0/3835869349 conn(0x7f599c07a600 msgr2=0x7f599c07aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:25:19.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.409+0000 7f59a285e700 1 -- 172.21.15.100:0/3835869349 shutdown_connections 2024-05-10T11:25:19.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.409+0000 7f59a285e700 1 -- 172.21.15.100:0/3835869349 wait complete. 2024-05-10T11:25:19.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.410+0000 7f59a285e700 1 Processor -- start 2024-05-10T11:25:19.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.410+0000 7f59a285e700 1 -- start start 2024-05-10T11:25:19.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.410+0000 7f59a285e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f599c103550 0x7f599c1a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:19.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.410+0000 7f59a285e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f599c103ef0 0x7f599c1a1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:19.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.410+0000 7f59a285e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f599c108460 0x7f599c19bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:19.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.410+0000 7f59a285e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f599c113550 con 0x7f599c108460 2024-05-10T11:25:19.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.410+0000 7f59a285e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f599c1133d0 con 0x7f599c103ef0 2024-05-10T11:25:19.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.410+0000 7f59a285e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f599c1136d0 con 0x7f599c103550 2024-05-10T11:25:19.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.411+0000 7f59a0dfb700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f599c108460 0x7f599c19bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:19.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.411+0000 7f5993fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f599c103ef0 0x7f599c1a1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:19.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.411+0000 7f5993fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f599c103ef0 0x7f599c1a1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:54596/0 (socket says 172.21.15.100:54596) 2024-05-10T11:25:19.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.411+0000 7f5993fff700 1 -- 172.21.15.100:0/2324396344 learned_addr learned my addr 172.21.15.100:0/2324396344 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:25:19.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.411+0000 7f599bfff700 1 --2- 172.21.15.100:0/2324396344 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f599c103550 0x7f599c1a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:19.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.411+0000 7f5993fff700 1 -- 172.21.15.100:0/2324396344 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f599c103550 msgr2=0x7f599c1a1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:19.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.411+0000 7f5993fff700 1 --2- 172.21.15.100:0/2324396344 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f599c103550 0x7f599c1a1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:19.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.411+0000 7f5993fff700 1 -- 172.21.15.100:0/2324396344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f599c108460 msgr2=0x7f599c19bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:19.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.411+0000 7f5993fff700 1 --2- 172.21.15.100:0/2324396344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f599c108460 0x7f599c19bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:19.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.411+0000 7f5993fff700 1 -- 172.21.15.100:0/2324396344 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f599c19c230 con 0x7f599c103ef0 2024-05-10T11:25:19.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.411+0000 7f599bfff700 1 --2- 172.21.15.100:0/2324396344 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f599c103550 0x7f599c1a1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:25:19.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.411+0000 7f5993fff700 1 --2- 172.21.15.100:0/2324396344 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f599c103ef0 0x7f599c1a1e50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5984006010 tx=0x7f598400bfc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:25:19.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.412+0000 7f5999ffb700 1 -- 172.21.15.100:0/2324396344 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5984027070 con 0x7f599c103ef0 2024-05-10T11:25:19.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.412+0000 7f5999ffb700 1 -- 172.21.15.100:0/2324396344 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5984022470 con 0x7f599c103ef0 2024-05-10T11:25:19.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.412+0000 7f59a285e700 1 -- 172.21.15.100:0/2324396344 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f599c19c4c0 con 0x7f599c103ef0 2024-05-10T11:25:19.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.412+0000 7f5999ffb700 1 -- 172.21.15.100:0/2324396344 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5984017650 con 0x7f599c103ef0 2024-05-10T11:25:19.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.412+0000 7f59a285e700 1 -- 172.21.15.100:0/2324396344 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f599c1a87e0 con 0x7f599c103ef0 2024-05-10T11:25:19.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.415+0000 7f5999ffb700 1 -- 172.21.15.100:0/2324396344 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f5984009d10 con 0x7f599c103ef0 2024-05-10T11:25:19.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.416+0000 7f5999ffb700 1 --2- 172.21.15.100:0/2324396344 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f59880b2980 0x7f59880b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:19.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.416+0000 7f599bfff700 1 --2- 172.21.15.100:0/2324396344 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f59880b2980 0x7f59880b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:19.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.416+0000 7f5999ffb700 1 -- 172.21.15.100:0/2324396344 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f59840e3020 con 0x7f599c103ef0 2024-05-10T11:25:19.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.416+0000 7f599bfff700 1 --2- 172.21.15.100:0/2324396344 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f59880b2980 0x7f59880b4e40 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f599c104730 tx=0x7f598c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:25:19.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.417+0000 7f59ad0f1b80 1 Processor -- start 2024-05-10T11:25:19.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.417+0000 7f59ad0f1b80 1 -- start start 2024-05-10T11:25:19.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.417+0000 7f59ad0f1b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f3196f4d0 0x562f3196f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:19.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.417+0000 7f59ad0f1b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562f3196fdf0 0x562f319736f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:19.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.417+0000 7f59ad0f1b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f31973c80 0x562f31974100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:19.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.417+0000 7f59ad0f1b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x562f318c1860 con 0x562f3196f4d0 2024-05-10T11:25:19.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.417+0000 7f59ad0f1b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562f318607c0 con 0x562f31973c80 2024-05-10T11:25:19.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.417+0000 7f59ad0f1b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x562f318bd820 con 0x562f3196fdf0 2024-05-10T11:25:19.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.417+0000 7f599bfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f3196f4d0 0x562f3196f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:19.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.417+0000 7f599bfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f3196f4d0 0x562f3196f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53290/0 (socket says 172.21.15.100:53290) 2024-05-10T11:25:19.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.417+0000 7f599bfff700 1 -- 172.21.15.100:0/2284974252 learned_addr learned my addr 172.21.15.100:0/2284974252 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:25:19.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.417+0000 7f59a0dfb700 1 --2- 172.21.15.100:0/2284974252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f31973c80 0x562f31974100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:19.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.417+0000 7f5993fff700 1 --2- 172.21.15.100:0/2284974252 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562f3196fdf0 0x562f319736f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:19.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.417+0000 7f59a0dfb700 1 -- 172.21.15.100:0/2284974252 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562f3196fdf0 msgr2=0x562f319736f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:19.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.417+0000 7f59a0dfb700 1 --2- 172.21.15.100:0/2284974252 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562f3196fdf0 0x562f319736f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:19.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.418+0000 7f59a0dfb700 1 -- 172.21.15.100:0/2284974252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f3196f4d0 msgr2=0x562f3196f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:19.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.418+0000 7f59a0dfb700 1 --2- 172.21.15.100:0/2284974252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f3196f4d0 0x562f3196f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:19.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.418+0000 7f59a0dfb700 1 -- 172.21.15.100:0/2284974252 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f3180e8d0 con 0x562f31973c80 2024-05-10T11:25:19.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.418+0000 7f599bfff700 1 --2- 172.21.15.100:0/2284974252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f3196f4d0 0x562f3196f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:25:19.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.418+0000 7f59a0dfb700 1 --2- 172.21.15.100:0/2284974252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f31973c80 0x562f31974100 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5994003e80 tx=0x7f5994009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c904ca210ed4af39 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:25:19.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.418+0000 7f59927fc700 1 -- 172.21.15.100:0/2284974252 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5994004bb0 con 0x562f31973c80 2024-05-10T11:25:19.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.418+0000 7f59927fc700 1 -- 172.21.15.100:0/2284974252 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5994004880 con 0x562f31973c80 2024-05-10T11:25:19.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.418+0000 7f59927fc700 1 -- 172.21.15.100:0/2284974252 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5994007ca0 con 0x562f31973c80 2024-05-10T11:25:19.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.419+0000 7f59ad0f1b80 1 -- 172.21.15.100:0/2284974252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f31973c80 msgr2=0x562f31974100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:19.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.419+0000 7f59ad0f1b80 1 --2- 172.21.15.100:0/2284974252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f31973c80 0x562f31974100 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5994003e80 tx=0x7f5994009c20 comp rx=0 tx=0).stop 2024-05-10T11:25:19.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.419+0000 7f59ad0f1b80 1 -- 172.21.15.100:0/2284974252 shutdown_connections 2024-05-10T11:25:19.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.419+0000 7f59ad0f1b80 1 --2- 172.21.15.100:0/2284974252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f31973c80 0x562f31974100 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:19.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.419+0000 7f59ad0f1b80 1 --2- 172.21.15.100:0/2284974252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f3196f4d0 0x562f3196f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:19.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.419+0000 7f59ad0f1b80 1 --2- 172.21.15.100:0/2284974252 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562f3196fdf0 0x562f319736f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:19.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.419+0000 7f59ad0f1b80 1 -- 172.21.15.100:0/2284974252 >> 172.21.15.100:0/2284974252 conn(0x562f3189b100 msgr2=0x562f31969c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:25:19.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.419+0000 7f59ad0f1b80 1 -- 172.21.15.100:0/2284974252 shutdown_connections 2024-05-10T11:25:19.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.419+0000 7f59ad0f1b80 1 -- 172.21.15.100:0/2284974252 wait complete. 2024-05-10T11:25:19.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.420+0000 7f59ad0f1b80 1 -- start start 2024-05-10T11:25:19.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.420+0000 7f59ad0f1b80 1 Processor -- start 2024-05-10T11:25:19.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.420+0000 7f59ad0f1b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f3196f4d0 0x562f3197afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:19.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.420+0000 7f59ad0f1b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f3196fdf0 0x562f31978380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:19.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.420+0000 7f59ad0f1b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562f31973c80 0x562f319788c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:19.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.420+0000 7f59ad0f1b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x562f3197be90 con 0x562f3196fdf0 2024-05-10T11:25:19.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.420+0000 7f59ad0f1b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562f31827c30 con 0x562f3196f4d0 2024-05-10T11:25:19.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.420+0000 7f5993fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f3196fdf0 0x562f31978380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:19.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.421+0000 7f59ad0f1b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x562f3197c010 con 0x562f31973c80 2024-05-10T11:25:19.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.421+0000 7f5993fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f3196fdf0 0x562f31978380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53300/0 (socket says 172.21.15.100:53300) 2024-05-10T11:25:19.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.421+0000 7f5993fff700 1 -- 172.21.15.100:0/3485363998 learned_addr learned my addr 172.21.15.100:0/3485363998 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:25:19.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.421+0000 7f59ad0f1b80 10 client.0 _ll_register_callbacks cb 0x562f31859920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:25:19.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.421+0000 7f599bfff700 1 --2- 172.21.15.100:0/3485363998 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f3196f4d0 0x562f3197afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:19.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.421+0000 7f59a0dfb700 1 --2- 172.21.15.100:0/3485363998 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562f31973c80 0x562f319788c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:19.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.421+0000 7f5993fff700 1 -- 172.21.15.100:0/3485363998 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562f31973c80 msgr2=0x562f319788c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:19.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.421+0000 7f5993fff700 1 --2- 172.21.15.100:0/3485363998 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562f31973c80 0x562f319788c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:19.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.421+0000 7f5993fff700 1 -- 172.21.15.100:0/3485363998 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f3196f4d0 msgr2=0x562f3197afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:19.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.421+0000 7f5993fff700 1 --2- 172.21.15.100:0/3485363998 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f3196f4d0 0x562f3197afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:19.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.421+0000 7f5993fff700 1 -- 172.21.15.100:0/3485363998 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f3185d690 con 0x562f3196fdf0 2024-05-10T11:25:19.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.421+0000 7f59a0dfb700 1 --2- 172.21.15.100:0/3485363998 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562f31973c80 0x562f319788c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:25:19.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.421+0000 7f5993fff700 1 --2- 172.21.15.100:0/3485363998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f3196fdf0 0x562f31978380 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f59840059d0 tx=0x7f5984022c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd429899e95a4d8d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:25:19.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.421+0000 7f597b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:25:19.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.422+0000 7f597b7fe700 1 -- 172.21.15.100:0/3485363998 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59840aa310 con 0x562f3196fdf0 2024-05-10T11:25:19.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.422+0000 7f597b7fe700 1 -- 172.21.15.100:0/3485363998 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f59840b2e60 con 0x562f3196fdf0 2024-05-10T11:25:19.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.422+0000 7f59ad0f1b80 1 -- 172.21.15.100:0/3485363998 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5994004880 con 0x562f3196fdf0 2024-05-10T11:25:19.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.422+0000 7f597b7fe700 1 -- 172.21.15.100:0/3485363998 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f59840ae6c0 con 0x562f3196fdf0 2024-05-10T11:25:19.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.422+0000 7f59ad0f1b80 10 client.16734 fetch_fsmap learned FSMap version 31 2024-05-10T11:25:19.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.422+0000 7f59ad0f1b80 1 -- 172.21.15.100:0/3485363998 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562f31986670 con 0x562f3196fdf0 2024-05-10T11:25:19.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.422+0000 7f597b7fe700 1 -- 172.21.15.100:0/3485363998 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f59840b30e0 con 0x562f3196fdf0 2024-05-10T11:25:19.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.422+0000 7f597b7fe700 1 -- 172.21.15.100:0/3485363998 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f59840ae240 con 0x562f3196fdf0 2024-05-10T11:25:19.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.422+0000 7f59ad0f1b80 10 client.16734 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:25:19.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.422+0000 7f59ad0f1b80 10 client.16734 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:25:19.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.422+0000 7f59ad0f1b80 20 client.16734 populate_metadata read hostname 'smithi100' 2024-05-10T11:25:19.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.422+0000 7f59ad0f1b80 1 --2- 172.21.15.100:0/3485363998 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562f31988b90 0x562f3198afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:19.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.422+0000 7f59ad0f1b80 4 client.16734 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:25:19.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.422+0000 7f59ad0f1b80 1 -- 172.21.15.100:0/3485363998 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5994004880 con 0x562f31988b90 2024-05-10T11:25:19.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.423+0000 7f599bfff700 1 --2- 172.21.15.100:0/3485363998 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562f31988b90 0x562f3198afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:19.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.423+0000 7f597b7fe700 10 client.16734 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:25:19.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.423+0000 7f599bfff700 1 --2- 172.21.15.100:0/3485363998 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562f31988b90 0x562f3198afd0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11353aceb67afe7e server_cookie=b07d32ef1543c84f in_seq=0 out_seq=0 2024-05-10T11:25:19.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.424+0000 7f597b7fe700 1 -- 172.21.15.100:0/3485363998 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f5994004880 con 0x562f31988b90 2024-05-10T11:25:19.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.424+0000 7f597b7fe700 10 client.16734 handle_command_reply: tid=0 2024-05-10T11:25:19.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.424+0000 7f59ad0f1b80 1 client.16734 shutdown 2024-05-10T11:25:19.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.424+0000 7f59ad0f1b80 1 -- 172.21.15.100:0/3485363998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f3196fdf0 msgr2=0x562f31978380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:19.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.424+0000 7f59ad0f1b80 1 --2- 172.21.15.100:0/3485363998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f3196fdf0 0x562f31978380 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f59840059d0 tx=0x7f5984022c00 comp rx=0 tx=0).stop 2024-05-10T11:25:19.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.424+0000 7f59ad0f1b80 1 -- 172.21.15.100:0/3485363998 shutdown_connections 2024-05-10T11:25:19.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.424+0000 7f59ad0f1b80 1 --2- 172.21.15.100:0/3485363998 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562f31988b90 0x562f3198afd0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:19.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.424+0000 7f59ad0f1b80 1 --2- 172.21.15.100:0/3485363998 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562f31973c80 0x562f319788c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:19.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.424+0000 7f59ad0f1b80 1 --2- 172.21.15.100:0/3485363998 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f3196f4d0 0x562f3197afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:19.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.424+0000 7f59ad0f1b80 1 --2- 172.21.15.100:0/3485363998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f3196fdf0 0x562f31978380 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:19.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.424+0000 7f597b7fe700 0 client.16734 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:25:19.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.424+0000 7f59ad0f1b80 1 -- 172.21.15.100:0/3485363998 >> 172.21.15.100:0/3485363998 conn(0x562f3189b100 msgr2=0x562f31974080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:25:19.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.425+0000 7f59ad0f1b80 1 -- 172.21.15.100:0/3485363998 shutdown_connections 2024-05-10T11:25:19.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.425+0000 7f59ad0f1b80 1 -- 172.21.15.100:0/3485363998 wait complete. 2024-05-10T11:25:19.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.425+0000 7f59ad0f1b80 20 client.16734 trim_cache size 0 max 16384 2024-05-10T11:25:19.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.447+0000 7f59ad0f1b80 1 Processor -- start 2024-05-10T11:25:19.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.447+0000 7f59ad0f1b80 1 -- start start 2024-05-10T11:25:19.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.447+0000 7f59ad0f1b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f3196f4d0 0x562f319715f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:19.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.448+0000 7f59ad0f1b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f3196fdf0 0x562f31971b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:19.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.448+0000 7f59ad0f1b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562f31973c80 0x562f319908e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:19.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.448+0000 7f59ad0f1b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f599c1b3a30 con 0x562f3196fdf0 2024-05-10T11:25:19.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.448+0000 7f59ad0f1b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f599c1b12c0 con 0x562f3196f4d0 2024-05-10T11:25:19.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.448+0000 7f59ad0f1b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f599c1af110 con 0x562f31973c80 2024-05-10T11:25:19.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.448+0000 7f5993fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f3196fdf0 0x562f31971b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:19.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.448+0000 7f5993fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f3196fdf0 0x562f31971b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53314/0 (socket says 172.21.15.100:53314) 2024-05-10T11:25:19.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.448+0000 7f5993fff700 1 -- 172.21.15.100:0/3433364061 learned_addr learned my addr 172.21.15.100:0/3433364061 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:25:19.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.448+0000 7f599bfff700 1 --2- 172.21.15.100:0/3433364061 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f3196f4d0 0x562f319715f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:19.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.448+0000 7f59a0dfb700 1 --2- 172.21.15.100:0/3433364061 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562f31973c80 0x562f319908e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:19.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.448+0000 7f599bfff700 1 -- 172.21.15.100:0/3433364061 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562f31973c80 msgr2=0x562f319908e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:19.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.448+0000 7f599bfff700 1 --2- 172.21.15.100:0/3433364061 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562f31973c80 0x562f319908e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:19.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.448+0000 7f599bfff700 1 -- 172.21.15.100:0/3433364061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f3196fdf0 msgr2=0x562f31971b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:19.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.448+0000 7f599bfff700 1 --2- 172.21.15.100:0/3433364061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f3196fdf0 0x562f31971b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:19.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.448+0000 7f599bfff700 1 -- 172.21.15.100:0/3433364061 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f31978bd0 con 0x562f3196f4d0 2024-05-10T11:25:19.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.448+0000 7f599bfff700 1 --2- 172.21.15.100:0/3433364061 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f3196f4d0 0x562f319715f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x562f3198b880 tx=0x7f598c014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e48c87adfa23b74 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:25:19.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.448+0000 7f5991ffb700 1 -- 172.21.15.100:0/3433364061 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f598c02a470 con 0x562f3196f4d0 2024-05-10T11:25:19.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.448+0000 7f5991ffb700 1 -- 172.21.15.100:0/3433364061 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f598c02f070 con 0x562f3196f4d0 2024-05-10T11:25:19.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.448+0000 7f5991ffb700 1 -- 172.21.15.100:0/3433364061 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f598c01ec80 con 0x562f3196f4d0 2024-05-10T11:25:19.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.449+0000 7f59ad0f1b80 1 -- 172.21.15.100:0/3433364061 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f3196f4d0 msgr2=0x562f319715f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:19.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.449+0000 7f59ad0f1b80 1 --2- 172.21.15.100:0/3433364061 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f3196f4d0 0x562f319715f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x562f3198b880 tx=0x7f598c014000 comp rx=0 tx=0).stop 2024-05-10T11:25:19.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.449+0000 7f59ad0f1b80 1 -- 172.21.15.100:0/3433364061 shutdown_connections 2024-05-10T11:25:19.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.449+0000 7f59ad0f1b80 1 --2- 172.21.15.100:0/3433364061 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562f31973c80 0x562f319908e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:19.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.449+0000 7f59ad0f1b80 1 --2- 172.21.15.100:0/3433364061 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f3196f4d0 0x562f319715f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:19.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.449+0000 7f59ad0f1b80 1 --2- 172.21.15.100:0/3433364061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f3196fdf0 0x562f31971b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:19.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.449+0000 7f59ad0f1b80 1 -- 172.21.15.100:0/3433364061 >> 172.21.15.100:0/3433364061 conn(0x562f3189b100 msgr2=0x562f31976cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:25:19.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.449+0000 7f59ad0f1b80 1 -- 172.21.15.100:0/3433364061 shutdown_connections 2024-05-10T11:25:19.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.449+0000 7f59ad0f1b80 1 -- 172.21.15.100:0/3433364061 wait complete. 2024-05-10T11:25:19.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.450+0000 7f59ad0f1b80 1 -- start start 2024-05-10T11:25:19.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.450+0000 7f59ad0f1b80 1 Processor -- start 2024-05-10T11:25:19.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.450+0000 7f59ad0f1b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f3196f4d0 0x562f31981300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:19.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.450+0000 7f59ad0f1b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f3196fdf0 0x562f31981840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:19.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.450+0000 7f59ad0f1b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562f31973c80 0x562f3197e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:19.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.450+0000 7f59ad0f1b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f599c1af780 con 0x562f3196fdf0 2024-05-10T11:25:19.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.450+0000 7f59ad0f1b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f599c1b0bf0 con 0x562f3196f4d0 2024-05-10T11:25:19.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.450+0000 7f59ad0f1b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f599c1afe50 con 0x562f31973c80 2024-05-10T11:25:19.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.450+0000 7f5993fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f3196fdf0 0x562f31981840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:19.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.450+0000 7f59ad0f1b80 10 client.0 _ll_register_callbacks cb 0x562f31859920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:25:19.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.450+0000 7f5993fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f3196fdf0 0x562f31981840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53322/0 (socket says 172.21.15.100:53322) 2024-05-10T11:25:19.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.450+0000 7f5993fff700 1 -- 172.21.15.100:0/1078748524 learned_addr learned my addr 172.21.15.100:0/1078748524 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:25:19.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.451+0000 7f599bfff700 1 --2- 172.21.15.100:0/1078748524 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f3196f4d0 0x562f31981300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:19.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.451+0000 7f59a0dfb700 1 --2- 172.21.15.100:0/1078748524 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562f31973c80 0x562f3197e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:19.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.451+0000 7f5993fff700 1 -- 172.21.15.100:0/1078748524 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562f31973c80 msgr2=0x562f3197e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:19.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.451+0000 7f5993fff700 1 --2- 172.21.15.100:0/1078748524 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562f31973c80 0x562f3197e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:19.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.451+0000 7f5993fff700 1 -- 172.21.15.100:0/1078748524 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f3196f4d0 msgr2=0x562f31981300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:19.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.451+0000 7f5993fff700 1 --2- 172.21.15.100:0/1078748524 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f3196f4d0 0x562f31981300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:19.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.451+0000 7f5993fff700 1 -- 172.21.15.100:0/1078748524 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f3197e810 con 0x562f3196fdf0 2024-05-10T11:25:19.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.451+0000 7f599bfff700 1 --2- 172.21.15.100:0/1078748524 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f3196f4d0 0x562f31981300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:25:19.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.451+0000 7f5993fff700 1 --2- 172.21.15.100:0/1078748524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f3196fdf0 0x562f31981840 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f59840a6390 tx=0x7f59840a9b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=617a0b76a1269ffc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:25:19.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.451+0000 7f597b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:25:19.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.451+0000 7f597b7fe700 1 -- 172.21.15.100:0/1078748524 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59840aa310 con 0x562f3196fdf0 2024-05-10T11:25:19.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.451+0000 7f59ad0f1b80 1 -- 172.21.15.100:0/1078748524 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f598c02f070 con 0x562f3196fdf0 2024-05-10T11:25:19.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.451+0000 7f597b7fe700 1 -- 172.21.15.100:0/1078748524 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f59840ae330 con 0x562f3196fdf0 2024-05-10T11:25:19.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.451+0000 7f597b7fe700 1 -- 172.21.15.100:0/1078748524 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f59840b30e0 con 0x562f3196fdf0 2024-05-10T11:25:19.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.451+0000 7f59ad0f1b80 10 client.16740 fetch_fsmap learned FSMap version 31 2024-05-10T11:25:19.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.451+0000 7f59ad0f1b80 1 -- 172.21.15.100:0/1078748524 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562f319f3ce0 con 0x562f3196fdf0 2024-05-10T11:25:19.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.452+0000 7f597b7fe700 1 -- 172.21.15.100:0/1078748524 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f598c02f070 con 0x562f3196fdf0 2024-05-10T11:25:19.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.452+0000 7f597b7fe700 1 -- 172.21.15.100:0/1078748524 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f59840a5e10 con 0x562f3196fdf0 2024-05-10T11:25:19.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.452+0000 7f59ad0f1b80 10 client.16740 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:25:19.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.452+0000 7f59ad0f1b80 10 client.16740 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:25:19.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.452+0000 7f59ad0f1b80 20 client.16740 populate_metadata read hostname 'smithi100' 2024-05-10T11:25:19.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.452+0000 7f59ad0f1b80 1 --2- 172.21.15.100:0/1078748524 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562f319f4260 0x562f319f6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:19.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.452+0000 7f59ad0f1b80 4 client.16740 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T11:25:19.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.452+0000 7f59ad0f1b80 1 -- 172.21.15.100:0/1078748524 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562f3198b360 con 0x562f319f4260 2024-05-10T11:25:19.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.452+0000 7f599bfff700 1 --2- 172.21.15.100:0/1078748524 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562f319f4260 0x562f319f6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:19.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.452+0000 7f599bfff700 1 --2- 172.21.15.100:0/1078748524 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562f319f4260 0x562f319f6680 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aee7da13339f8317 server_cookie=896b9b8049afcb80 in_seq=0 out_seq=0 2024-05-10T11:25:19.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.452+0000 7f597b7fe700 10 client.16740 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:25:19.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.453+0000 7f597b7fe700 1 -- 172.21.15.100:0/1078748524 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562f3198b360 con 0x562f319f4260 2024-05-10T11:25:19.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.453+0000 7f597b7fe700 10 client.16740 handle_command_reply: tid=0 2024-05-10T11:25:19.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.453+0000 7f59ad0f1b80 1 client.16740 shutdown 2024-05-10T11:25:19.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.453+0000 7f59ad0f1b80 1 -- 172.21.15.100:0/1078748524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f3196fdf0 msgr2=0x562f31981840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:19.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.453+0000 7f59ad0f1b80 1 --2- 172.21.15.100:0/1078748524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f3196fdf0 0x562f31981840 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f59840a6390 tx=0x7f59840a9b30 comp rx=0 tx=0).stop 2024-05-10T11:25:19.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.453+0000 7f59ad0f1b80 1 -- 172.21.15.100:0/1078748524 shutdown_connections 2024-05-10T11:25:19.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.453+0000 7f59ad0f1b80 1 --2- 172.21.15.100:0/1078748524 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562f319f4260 0x562f319f6680 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:19.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.453+0000 7f59ad0f1b80 1 --2- 172.21.15.100:0/1078748524 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562f31973c80 0x562f3197e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:19.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.453+0000 7f59ad0f1b80 1 --2- 172.21.15.100:0/1078748524 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f3196f4d0 0x562f31981300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:19.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.453+0000 7f59ad0f1b80 1 --2- 172.21.15.100:0/1078748524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f3196fdf0 0x562f31981840 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:19.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.453+0000 7f59ad0f1b80 1 -- 172.21.15.100:0/1078748524 >> 172.21.15.100:0/1078748524 conn(0x562f3189b100 msgr2=0x562f31963e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:25:19.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.453+0000 7f597b7fe700 0 client.16740 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:25:19.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.453+0000 7f59ad0f1b80 1 -- 172.21.15.100:0/1078748524 shutdown_connections 2024-05-10T11:25:19.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.453+0000 7f59ad0f1b80 1 -- 172.21.15.100:0/1078748524 wait complete. 2024-05-10T11:25:19.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.453+0000 7f59ad0f1b80 20 client.16740 trim_cache size 0 max 16384 2024-05-10T11:25:19.457 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:25:19.458 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-05-10T11:25:19.458 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-05-10T11:25:19.458 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:25:19.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.454+0000 7f59a285e700 1 -- 172.21.15.100:0/2324396344 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f59880b2980 msgr2=0x7f59880b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:19.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.454+0000 7f59a285e700 1 --2- 172.21.15.100:0/2324396344 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f59880b2980 0x7f59880b4e40 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f599c104730 tx=0x7f598c009260 comp rx=0 tx=0).stop 2024-05-10T11:25:19.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.454+0000 7f59a285e700 1 -- 172.21.15.100:0/2324396344 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f599c103ef0 msgr2=0x7f599c1a1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:19.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.454+0000 7f59a285e700 1 --2- 172.21.15.100:0/2324396344 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f599c103ef0 0x7f599c1a1e50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5984006010 tx=0x7f598400bfc0 comp rx=0 tx=0).stop 2024-05-10T11:25:19.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.454+0000 7f59a285e700 1 -- 172.21.15.100:0/2324396344 shutdown_connections 2024-05-10T11:25:19.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.454+0000 7f59a285e700 1 --2- 172.21.15.100:0/2324396344 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f59880b2980 0x7f59880b4e40 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:19.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.454+0000 7f59a285e700 1 --2- 172.21.15.100:0/2324396344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f599c108460 0x7f599c19bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:19.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.454+0000 7f59a285e700 1 --2- 172.21.15.100:0/2324396344 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f599c103550 0x7f599c1a1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:19.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.454+0000 7f59a285e700 1 --2- 172.21.15.100:0/2324396344 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f599c103ef0 0x7f599c1a1e50 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:19.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.454+0000 7f59a285e700 1 -- 172.21.15.100:0/2324396344 >> 172.21.15.100:0/2324396344 conn(0x7f599c07a600 msgr2=0x7f599c1013c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:25:19.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.454+0000 7f59a285e700 1 -- 172.21.15.100:0/2324396344 shutdown_connections 2024-05-10T11:25:19.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:19.454+0000 7f59a285e700 1 -- 172.21.15.100:0/2324396344 wait complete. 2024-05-10T11:25:19.973 INFO:tasks.workunit.client.0.smithi100.stdout: 1 163003 2.74 MB/sec execute 194 sec latency 15.535 ms 2024-05-10T11:25:20.025 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:25:20 smithi100 ceph-mon[30296]: pgmap v628: 129 pgs: 129 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 1.3 MiB/s wr, 154 op/s 2024-05-10T11:25:20.161 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T11:25:20.162 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T11:25:20.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:25:20 smithi188 ceph-mon[34075]: pgmap v628: 129 pgs: 129 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 1.3 MiB/s wr, 154 op/s 2024-05-10T11:25:20.304 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:25:20 smithi178 ceph-mon[33359]: pgmap v628: 129 pgs: 129 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 1.3 MiB/s wr, 154 op/s 2024-05-10T11:25:20.431 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:25:20.974 INFO:tasks.workunit.client.0.smithi100.stdout: 1 163485 2.75 MB/sec execute 195 sec latency 14.979 ms 2024-05-10T11:25:21.974 INFO:tasks.workunit.client.0.smithi100.stdout: 1 163758 2.74 MB/sec execute 196 sec latency 21.005 ms 2024-05-10T11:25:22.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:22.165+0000 7f6554a3b700 1 -- 172.21.15.100:0/579755019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6550108880 msgr2=0x7f6550110400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:22.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:22.165+0000 7f6554a3b700 1 --2- 172.21.15.100:0/579755019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6550108880 0x7f6550110400 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f654800b0d0 tx=0x7f654800b3e0 comp rx=0 tx=0).stop 2024-05-10T11:25:22.166 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:25:22 smithi100 ceph-mon[30296]: pgmap v629: 129 pgs: 129 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.2 MiB/s wr, 135 op/s 2024-05-10T11:25:22.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:22.165+0000 7f6554a3b700 1 -- 172.21.15.100:0/579755019 shutdown_connections 2024-05-10T11:25:22.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:22.165+0000 7f6554a3b700 1 --2- 172.21.15.100:0/579755019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6550108880 0x7f6550110400 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:22.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:22.165+0000 7f6554a3b700 1 --2- 172.21.15.100:0/579755019 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65501037b0 0x7f6550103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:22.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:22.165+0000 7f6554a3b700 1 --2- 172.21.15.100:0/579755019 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6550104160 0x7f6550108150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:22.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:22.165+0000 7f6554a3b700 1 -- 172.21.15.100:0/579755019 >> 172.21.15.100:0/579755019 conn(0x7f655007a530 msgr2=0x7f655007a940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:25:22.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:22.165+0000 7f6554a3b700 1 -- 172.21.15.100:0/579755019 shutdown_connections 2024-05-10T11:25:22.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:22.165+0000 7f6554a3b700 1 -- 172.21.15.100:0/579755019 wait complete. 2024-05-10T11:25:22.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:22.166+0000 7f6554a3b700 1 Processor -- start 2024-05-10T11:25:22.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:22.166+0000 7f6554a3b700 1 -- start start 2024-05-10T11:25:22.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:22.166+0000 7f6554a3b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65501037b0 0x7f655019f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:22.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:22.166+0000 7f6554a3b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6550104160 0x7f655019fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:22.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:22.166+0000 7f6554a3b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6550108880 0x7f65501997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:22.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:22.166+0000 7f6554a3b700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6550113500 con 0x7f65501037b0 2024-05-10T11:25:22.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:22.166+0000 7f6554a3b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6550113380 con 0x7f6550104160 2024-05-10T11:25:22.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:22.166+0000 7f6554a3b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6550113680 con 0x7f6550108880 2024-05-10T11:25:22.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:22.166+0000 7f654e59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65501037b0 0x7f655019f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:22.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:22.167+0000 7f654e59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65501037b0 0x7f655019f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53338/0 (socket says 172.21.15.100:53338) 2024-05-10T11:25:22.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:22.167+0000 7f654e59c700 1 -- 172.21.15.100:0/890071779 learned_addr learned my addr 172.21.15.100:0/890071779 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:25:22.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:22.167+0000 7f654dd9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6550104160 0x7f655019fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:22.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:22.167+0000 7f654ed9d700 1 --2- 172.21.15.100:0/890071779 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6550108880 0x7f65501997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:22.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:22.167+0000 7f654dd9b700 1 -- 172.21.15.100:0/890071779 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6550108880 msgr2=0x7f65501997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:22.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:22.167+0000 7f654dd9b700 1 --2- 172.21.15.100:0/890071779 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6550108880 0x7f65501997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:22.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:22.167+0000 7f654dd9b700 1 -- 172.21.15.100:0/890071779 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65501037b0 msgr2=0x7f655019f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:22.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:22.167+0000 7f654dd9b700 1 --2- 172.21.15.100:0/890071779 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65501037b0 0x7f655019f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:22.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:22.167+0000 7f654dd9b700 1 -- 172.21.15.100:0/890071779 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f655019a0a0 con 0x7f6550104160 2024-05-10T11:25:22.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:22.167+0000 7f654e59c700 1 --2- 172.21.15.100:0/890071779 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65501037b0 0x7f655019f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:25:22.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:22.167+0000 7f654dd9b700 1 --2- 172.21.15.100:0/890071779 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6550104160 0x7f655019fbe0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f654400e9f0 tx=0x7f654400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:25:22.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:22.167+0000 7f653f7fe700 1 -- 172.21.15.100:0/890071779 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f654400cbd0 con 0x7f6550104160 2024-05-10T11:25:22.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:22.168+0000 7f653f7fe700 1 -- 172.21.15.100:0/890071779 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f654400cd70 con 0x7f6550104160 2024-05-10T11:25:22.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:22.168+0000 7f6554a3b700 1 -- 172.21.15.100:0/890071779 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f655019a390 con 0x7f6550104160 2024-05-10T11:25:22.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:22.168+0000 7f653f7fe700 1 -- 172.21.15.100:0/890071779 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6544018680 con 0x7f6550104160 2024-05-10T11:25:22.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:22.168+0000 7f6554a3b700 1 -- 172.21.15.100:0/890071779 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65501a6620 con 0x7f6550104160 2024-05-10T11:25:22.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:22.169+0000 7f6554a3b700 1 -- 172.21.15.100:0/890071779 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6550104c10 con 0x7f6550104160 2024-05-10T11:25:22.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:22.170+0000 7f653f7fe700 1 -- 172.21.15.100:0/890071779 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f6544010430 con 0x7f6550104160 2024-05-10T11:25:22.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:22.171+0000 7f653f7fe700 1 --2- 172.21.15.100:0/890071779 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f65380b2900 0x7f65380b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:22.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:22.171+0000 7f654e59c700 1 --2- 172.21.15.100:0/890071779 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f65380b2900 0x7f65380b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:22.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:22.171+0000 7f653f7fe700 1 -- 172.21.15.100:0/890071779 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f65440e2660 con 0x7f6550104160 2024-05-10T11:25:22.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:22.171+0000 7f654e59c700 1 --2- 172.21.15.100:0/890071779 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f65380b2900 0x7f65380b4dc0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6540011550 tx=0x7f6540009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:25:22.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:22.175+0000 7f653f7fe700 1 -- 172.21.15.100:0/890071779 <== mon.2 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f65440aa020 con 0x7f6550104160 2024-05-10T11:25:22.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:25:22 smithi188 ceph-mon[34075]: pgmap v629: 129 pgs: 129 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.2 MiB/s wr, 135 op/s 2024-05-10T11:25:22.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:22.486+0000 7f6554a3b700 1 -- 172.21.15.100:0/890071779 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f655019aca0 con 0x7f6550104160 2024-05-10T11:25:22.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:22.487+0000 7f653f7fe700 1 -- 172.21.15.100:0/890071779 <== mon.2 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11894 (secure 0 0 0) 0x7f65440b20a0 con 0x7f6550104160 2024-05-10T11:25:22.487 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:25:22.487 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2072225012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2072225012},{"type":"v1","addr":"172.21.15.100:6835","nonce":2072225012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/401939540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":401939540},{"type":"v1","addr":"172.21.15.100:6837","nonce":401939540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14571,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/3946311025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3946311025},{"type":"v1","addr":"172.21.15.100:6839","nonce":3946311025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3736371660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3736371660},{"type":"v1","addr":"172.21.15.100:6841","nonce":3736371660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/2741810791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2741810791},{"type":"v1","addr":"172.21.15.178:6833","nonce":2741810791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1704891041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1704891041},{"type":"v1","addr":"172.21.15.188:6833","nonce":1704891041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T11:14:27.242452+0000","modified":"2024-05-10T11:23:19.557113+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24439,"mds_1":24437,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/2124076312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2124076312},{"type":"v1","addr":"172.21.15.178:6835","nonce":2124076312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/3224358425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3224358425},{"type":"v1","addr":"172.21.15.178:6837","nonce":3224358425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/515547690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":515547690},{"type":"v1","addr":"172.21.15.188:6835","nonce":515547690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/1943342620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1943342620},{"type":"v1","addr":"172.21.15.188:6837","nonce":1943342620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.178:6839/3760985404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3760985404},{"type":"v1","addr":"172.21.15.178:6839","nonce":3760985404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.188:6839/3194917070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":3194917070},{"type":"v1","addr":"172.21.15.188:6839","nonce":3194917070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24439,"qdb_cluster":[24439,24427,24437]},"id":1}]} 2024-05-10T11:25:22.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:22.489+0000 7f6554a3b700 1 -- 172.21.15.100:0/890071779 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f65380b2900 msgr2=0x7f65380b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:22.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:22.489+0000 7f6554a3b700 1 --2- 172.21.15.100:0/890071779 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f65380b2900 0x7f65380b4dc0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6540011550 tx=0x7f6540009260 comp rx=0 tx=0).stop 2024-05-10T11:25:22.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:22.489+0000 7f6554a3b700 1 -- 172.21.15.100:0/890071779 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6550104160 msgr2=0x7f655019fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:22.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:22.489+0000 7f6554a3b700 1 --2- 172.21.15.100:0/890071779 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6550104160 0x7f655019fbe0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f654400e9f0 tx=0x7f654400ed00 comp rx=0 tx=0).stop 2024-05-10T11:25:22.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:22.490+0000 7f6554a3b700 1 -- 172.21.15.100:0/890071779 shutdown_connections 2024-05-10T11:25:22.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:22.490+0000 7f6554a3b700 1 --2- 172.21.15.100:0/890071779 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f65380b2900 0x7f65380b4dc0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:22.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:22.490+0000 7f6554a3b700 1 --2- 172.21.15.100:0/890071779 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6550108880 0x7f65501997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:22.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:22.490+0000 7f6554a3b700 1 --2- 172.21.15.100:0/890071779 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65501037b0 0x7f655019f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:22.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:22.490+0000 7f6554a3b700 1 --2- 172.21.15.100:0/890071779 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6550104160 0x7f655019fbe0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:22.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:22.490+0000 7f6554a3b700 1 -- 172.21.15.100:0/890071779 >> 172.21.15.100:0/890071779 conn(0x7f655007a530 msgr2=0x7f6550101790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:25:22.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:22.490+0000 7f6554a3b700 1 -- 172.21.15.100:0/890071779 shutdown_connections 2024-05-10T11:25:22.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:22.490+0000 7f6554a3b700 1 -- 172.21.15.100:0/890071779 wait complete. 2024-05-10T11:25:22.491 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 31 2024-05-10T11:25:22.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:25:22 smithi178 ceph-mon[33359]: pgmap v629: 129 pgs: 129 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.2 MiB/s wr, 135 op/s 2024-05-10T11:25:22.974 INFO:tasks.workunit.client.0.smithi100.stdout: 1 164240 2.73 MB/sec execute 197 sec latency 27.371 ms 2024-05-10T11:25:23.267 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.265+0000 7f684e654640 1 -- 172.21.15.100:0/1790287944 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6848102400 msgr2=0x7f6848106ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:23.267 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.265+0000 7f684e654640 1 --2- 172.21.15.100:0/1790287944 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6848102400 0x7f6848106ca0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6840009be0 tx=0x7f684002a6d0 comp rx=0 tx=0).stop 2024-05-10T11:25:23.267 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.266+0000 7f684e654640 1 -- 172.21.15.100:0/1790287944 shutdown_connections 2024-05-10T11:25:23.267 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.266+0000 7f684e654640 1 --2- 172.21.15.100:0/1790287944 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6848102400 0x7f6848106ca0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:23.267 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.266+0000 7f684e654640 1 --2- 172.21.15.100:0/1790287944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6848101a40 0x7f6848101ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:23.267 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.266+0000 7f684e654640 1 --2- 172.21.15.100:0/1790287944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6848100ce0 0x7f6848101100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:23.267 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.266+0000 7f684e654640 1 -- 172.21.15.100:0/1790287944 >> 172.21.15.100:0/1790287944 conn(0x7f68480fc610 msgr2=0x7f68480fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:25:23.267 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.266+0000 7f684e654640 1 -- 172.21.15.100:0/1790287944 shutdown_connections 2024-05-10T11:25:23.268 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.267+0000 7f684e654640 1 -- 172.21.15.100:0/1790287944 wait complete. 2024-05-10T11:25:23.268 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.267+0000 7f684e654640 1 Processor -- start 2024-05-10T11:25:23.268 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.268+0000 7f684e654640 1 -- start start 2024-05-10T11:25:23.268 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.268+0000 7f684e654640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6848100ce0 0x7f6848071330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:23.268 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.268+0000 7f684e654640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6848101a40 0x7f6848071870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:23.268 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.268+0000 7f684e654640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6848102400 0x7f68481af120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:23.268 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.268+0000 7f684e654640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6848114a70 con 0x7f6848101a40 2024-05-10T11:25:23.268 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.268+0000 7f684e654640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f68481148f0 con 0x7f6848100ce0 2024-05-10T11:25:23.268 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.268+0000 7f684e654640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6848114bf0 con 0x7f6848102400 2024-05-10T11:25:23.270 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.268+0000 7f6847fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6848100ce0 0x7f6848071330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:23.270 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.268+0000 7f6847fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6848100ce0 0x7f6848071330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:54668/0 (socket says 172.21.15.100:54668) 2024-05-10T11:25:23.270 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.268+0000 7f6847fff640 1 -- 172.21.15.100:0/3084527541 learned_addr learned my addr 172.21.15.100:0/3084527541 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:25:23.270 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.269+0000 7f684cbf1640 1 --2- 172.21.15.100:0/3084527541 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6848102400 0x7f68481af120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:23.270 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.269+0000 7f6847fff640 1 -- 172.21.15.100:0/3084527541 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6848102400 msgr2=0x7f68481af120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:23.270 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.269+0000 7f6847fff640 1 --2- 172.21.15.100:0/3084527541 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6848102400 0x7f68481af120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:23.270 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.269+0000 7f6847fff640 1 -- 172.21.15.100:0/3084527541 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6848101a40 msgr2=0x7f6848071870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:23.270 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.269+0000 7f6847fff640 1 --2- 172.21.15.100:0/3084527541 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6848101a40 0x7f6848071870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:23.270 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.269+0000 7f6847fff640 1 -- 172.21.15.100:0/3084527541 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6848071f40 con 0x7f6848100ce0 2024-05-10T11:25:23.270 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.269+0000 7f6847fff640 1 --2- 172.21.15.100:0/3084527541 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6848100ce0 0x7f6848071330 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f683800d980 tx=0x7f683800de40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:25:23.270 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.269+0000 7f6845ffb640 1 -- 172.21.15.100:0/3084527541 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6838017030 con 0x7f6848100ce0 2024-05-10T11:25:23.271 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.269+0000 7f6845ffb640 1 -- 172.21.15.100:0/3084527541 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6838004020 con 0x7f6848100ce0 2024-05-10T11:25:23.271 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.269+0000 7f6845ffb640 1 -- 172.21.15.100:0/3084527541 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6838003b30 con 0x7f6848100ce0 2024-05-10T11:25:23.271 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.269+0000 7f684e654640 1 -- 172.21.15.100:0/3084527541 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68481081e0 con 0x7f6848100ce0 2024-05-10T11:25:23.271 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.269+0000 7f684e654640 1 -- 172.21.15.100:0/3084527541 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68481086e0 con 0x7f6848100ce0 2024-05-10T11:25:23.271 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:25:23 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/890071779' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:25:23.275 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.273+0000 7f685136b740 1 Processor -- start 2024-05-10T11:25:23.275 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.273+0000 7f685136b740 1 -- start start 2024-05-10T11:25:23.275 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.273+0000 7f685136b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d5733afdb0 0x55d5733b01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:23.275 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.273+0000 7f685136b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d5733b07a0 0x55d5733b52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:23.275 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.274+0000 7f6845ffb640 1 -- 172.21.15.100:0/3084527541 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f683801a070 con 0x7f6848100ce0 2024-05-10T11:25:23.275 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.274+0000 7f685136b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5733b5910 0x55d5733b7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:23.275 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.274+0000 7f685136b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55d5733145d0 con 0x55d5733b5910 2024-05-10T11:25:23.275 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.274+0000 7f685136b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d5732cd890 con 0x55d5733b07a0 2024-05-10T11:25:23.275 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.274+0000 7f685136b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d573310590 con 0x55d5733afdb0 2024-05-10T11:25:23.275 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.274+0000 7f684cbf1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d5733afdb0 0x55d5733b01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.274+0000 7f684cbf1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d5733afdb0 0x55d5733b01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:50424/0 (socket says 172.21.15.100:50424) 2024-05-10T11:25:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.274+0000 7f684cbf1640 1 -- 172.21.15.100:0/1388917323 learned_addr learned my addr 172.21.15.100:0/1388917323 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:25:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.274+0000 7f6847fff640 1 --2- 172.21.15.100:0/1388917323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d5733b07a0 0x55d5733b52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.274+0000 7f683ffff640 1 --2- 172.21.15.100:0/1388917323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5733b5910 0x55d5733b7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.274+0000 7f684cbf1640 1 -- 172.21.15.100:0/1388917323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d5733b07a0 msgr2=0x55d5733b52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.274+0000 7f684cbf1640 1 --2- 172.21.15.100:0/1388917323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d5733b07a0 0x55d5733b52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.274+0000 7f684cbf1640 1 -- 172.21.15.100:0/1388917323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5733b5910 msgr2=0x55d5733b7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.274+0000 7f684cbf1640 1 --2- 172.21.15.100:0/1388917323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5733b5910 0x55d5733b7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.274+0000 7f684cbf1640 1 -- 172.21.15.100:0/1388917323 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d573026210 con 0x55d5733afdb0 2024-05-10T11:25:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.274+0000 7f683ffff640 1 --2- 172.21.15.100:0/1388917323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5733b5910 0x55d5733b7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:25:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.274+0000 7f684cbf1640 1 --2- 172.21.15.100:0/1388917323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d5733afdb0 0x55d5733b01d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6840009be0 tx=0x7f6840009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b4d05e8ce4c5b32 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:25:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.275+0000 7f683e7fc640 1 -- 172.21.15.100:0/1388917323 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6840040030 con 0x55d5733afdb0 2024-05-10T11:25:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.275+0000 7f683e7fc640 1 -- 172.21.15.100:0/1388917323 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6840037040 con 0x55d5733afdb0 2024-05-10T11:25:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.275+0000 7f683e7fc640 1 -- 172.21.15.100:0/1388917323 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6840036070 con 0x55d5733afdb0 2024-05-10T11:25:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.275+0000 7f6845ffb640 1 --2- 172.21.15.100:0/3084527541 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f68340b28f0 0x7f68340b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.275+0000 7f6847fff640 1 --2- 172.21.15.100:0/3084527541 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f68340b28f0 0x7f68340b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.275+0000 7f6845ffb640 1 -- 172.21.15.100:0/3084527541 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f68380e1bc0 con 0x7f6848100ce0 2024-05-10T11:25:23.281 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.275+0000 7f6847fff640 1 --2- 172.21.15.100:0/3084527541 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f68340b28f0 0x7f68340b4db0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f683800d950 tx=0x7f68380e5600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:25:23.281 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.275+0000 7f685136b740 1 -- 172.21.15.100:0/1388917323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d5733afdb0 msgr2=0x55d5733b01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:23.281 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.275+0000 7f685136b740 1 --2- 172.21.15.100:0/1388917323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d5733afdb0 0x55d5733b01d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6840009be0 tx=0x7f6840009840 comp rx=0 tx=0).stop 2024-05-10T11:25:23.281 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.275+0000 7f685136b740 1 -- 172.21.15.100:0/1388917323 shutdown_connections 2024-05-10T11:25:23.281 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.275+0000 7f685136b740 1 --2- 172.21.15.100:0/1388917323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5733b5910 0x55d5733b7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:23.281 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.275+0000 7f685136b740 1 --2- 172.21.15.100:0/1388917323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d5733b07a0 0x55d5733b52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:23.282 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.275+0000 7f685136b740 1 --2- 172.21.15.100:0/1388917323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d5733afdb0 0x55d5733b01d0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:23.282 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.275+0000 7f685136b740 1 -- 172.21.15.100:0/1388917323 >> 172.21.15.100:0/1388917323 conn(0x55d5733ab420 msgr2=0x55d5733ab820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:25:23.282 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.276+0000 7f685136b740 1 -- 172.21.15.100:0/1388917323 shutdown_connections 2024-05-10T11:25:23.282 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.276+0000 7f685136b740 1 -- 172.21.15.100:0/1388917323 wait complete. 2024-05-10T11:25:23.282 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.276+0000 7f685136b740 1 -- start start 2024-05-10T11:25:23.282 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.276+0000 7f685136b740 1 Processor -- start 2024-05-10T11:25:23.282 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.276+0000 7f685136b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d5733afdb0 0x55d5733b9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:23.282 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.276+0000 7f685136b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d5733b07a0 0x55d5733ba4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:23.282 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.276+0000 7f685136b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5733b5910 0x55d5733c14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:23.282 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.276+0000 7f685136b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55d5733b8860 con 0x55d5733b5910 2024-05-10T11:25:23.282 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.276+0000 7f685136b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d5732d1370 con 0x55d5733b07a0 2024-05-10T11:25:23.282 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.276+0000 7f685136b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d5733b89e0 con 0x55d5733afdb0 2024-05-10T11:25:23.282 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.277+0000 7f685136b740 10 client.0 _ll_register_callbacks cb 0x55d573025a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:25:23.282 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.277+0000 7f683ffff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d5733b07a0 0x55d5733ba4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:23.283 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.277+0000 7f684cbf1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d5733afdb0 0x55d5733b9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:23.283 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.277+0000 7f683ffff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d5733b07a0 0x55d5733ba4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:54676/0 (socket says 172.21.15.100:54676) 2024-05-10T11:25:23.283 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.277+0000 7f684cbf1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d5733afdb0 0x55d5733b9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:50426/0 (socket says 172.21.15.100:50426) 2024-05-10T11:25:23.283 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.277+0000 7f683ffff640 1 -- 172.21.15.100:0/3943783690 learned_addr learned my addr 172.21.15.100:0/3943783690 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:25:23.283 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.277+0000 7f684cbf1640 1 --2- 172.21.15.100:0/3943783690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5733b5910 0x55d5733c14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:23.283 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.277+0000 7f683ffff640 1 -- 172.21.15.100:0/3943783690 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d5733afdb0 msgr2=0x55d5733b9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:23.283 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.277+0000 7f683ffff640 1 --2- 172.21.15.100:0/3943783690 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d5733afdb0 0x55d5733b9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:23.283 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.277+0000 7f683ffff640 1 -- 172.21.15.100:0/3943783690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5733b5910 msgr2=0x55d5733c14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:23.283 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.277+0000 7f683ffff640 1 --2- 172.21.15.100:0/3943783690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5733b5910 0x55d5733c14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:23.283 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.277+0000 7f683ffff640 1 -- 172.21.15.100:0/3943783690 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5730f2620 con 0x55d5733b07a0 2024-05-10T11:25:23.283 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.277+0000 7f683ffff640 1 --2- 172.21.15.100:0/3943783690 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d5733b07a0 0x55d5733ba4b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f683000dc90 tx=0x7f683000f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cad9a1cf80b9550b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:25:23.283 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.277+0000 7f68277fe640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:25:23.283 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.277+0000 7f68277fe640 1 -- 172.21.15.100:0/3943783690 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6830011640 con 0x55d5733b07a0 2024-05-10T11:25:23.283 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.277+0000 7f685136b740 1 -- 172.21.15.100:0/3943783690 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6840037040 con 0x55d5733b07a0 2024-05-10T11:25:23.283 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.277+0000 7f68277fe640 1 -- 172.21.15.100:0/3943783690 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f68300117e0 con 0x55d5733b07a0 2024-05-10T11:25:23.283 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.277+0000 7f68277fe640 1 -- 172.21.15.100:0/3943783690 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6830024050 con 0x55d5733b07a0 2024-05-10T11:25:23.284 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.277+0000 7f685136b740 10 client.26201 fetch_fsmap learned FSMap version 31 2024-05-10T11:25:23.284 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.277+0000 7f685136b740 1 -- 172.21.15.100:0/3943783690 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d5733c4570 con 0x55d5733b07a0 2024-05-10T11:25:23.284 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.278+0000 7f68277fe640 1 -- 172.21.15.100:0/3943783690 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6840037040 con 0x55d5733b07a0 2024-05-10T11:25:23.284 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.278+0000 7f68277fe640 1 -- 172.21.15.100:0/3943783690 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f683001b980 con 0x55d5733b07a0 2024-05-10T11:25:23.284 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.278+0000 7f685136b740 10 client.26201 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:25:23.284 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.278+0000 7f685136b740 10 client.26201 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:25:23.284 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.278+0000 7f685136b740 20 client.26201 populate_metadata read hostname 'smithi100' 2024-05-10T11:25:23.284 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.278+0000 7f685136b740 1 --2- 172.21.15.100:0/3943783690 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d5733c5cb0 0x55d5733c80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:23.284 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.278+0000 7f685136b740 4 client.26201 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:25:23.284 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.278+0000 7f685136b740 1 -- 172.21.15.100:0/3943783690 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5733221e0 con 0x55d5733c5cb0 2024-05-10T11:25:23.284 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.278+0000 7f684cbf1640 1 --2- 172.21.15.100:0/3943783690 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d5733c5cb0 0x55d5733c80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:23.284 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.279+0000 7f68277fe640 10 client.26201 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:25:23.284 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.279+0000 7f684cbf1640 1 --2- 172.21.15.100:0/3943783690 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d5733c5cb0 0x55d5733c80f0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d8113b4b465c1a6 server_cookie=2b8992837bbe5478 in_seq=0 out_seq=0 2024-05-10T11:25:23.284 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.279+0000 7f68277fe640 1 -- 172.21.15.100:0/3943783690 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d5733221e0 con 0x55d5733c5cb0 2024-05-10T11:25:23.284 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.279+0000 7f68277fe640 10 client.26201 handle_command_reply: tid=0 2024-05-10T11:25:23.285 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.279+0000 7f685136b740 1 client.26201 shutdown 2024-05-10T11:25:23.285 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.280+0000 7f685136b740 1 -- 172.21.15.100:0/3943783690 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d5733b07a0 msgr2=0x55d5733ba4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:23.285 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.280+0000 7f685136b740 1 --2- 172.21.15.100:0/3943783690 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d5733b07a0 0x55d5733ba4b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f683000dc90 tx=0x7f683000f730 comp rx=0 tx=0).stop 2024-05-10T11:25:23.285 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.280+0000 7f685136b740 1 -- 172.21.15.100:0/3943783690 shutdown_connections 2024-05-10T11:25:23.285 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.280+0000 7f685136b740 1 --2- 172.21.15.100:0/3943783690 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d5733c5cb0 0x55d5733c80f0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:23.285 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.280+0000 7f685136b740 1 --2- 172.21.15.100:0/3943783690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5733b5910 0x55d5733c14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:23.285 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.280+0000 7f68277fe640 0 client.26201 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:25:23.285 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.280+0000 7f685136b740 1 --2- 172.21.15.100:0/3943783690 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d5733b07a0 0x55d5733ba4b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:23.285 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.280+0000 7f685136b740 1 --2- 172.21.15.100:0/3943783690 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d5733afdb0 0x55d5733b9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:23.285 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.280+0000 7f685136b740 1 -- 172.21.15.100:0/3943783690 >> 172.21.15.100:0/3943783690 conn(0x55d5733ab420 msgr2=0x55d5733aa580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:25:23.285 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.280+0000 7f685136b740 1 -- 172.21.15.100:0/3943783690 shutdown_connections 2024-05-10T11:25:23.285 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.280+0000 7f685136b740 1 -- 172.21.15.100:0/3943783690 wait complete. 2024-05-10T11:25:23.285 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.280+0000 7f685136b740 20 client.26201 trim_cache size 0 max 16384 2024-05-10T11:25:23.300 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.296+0000 7f685136b740 1 Processor -- start 2024-05-10T11:25:23.300 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.297+0000 7f685136b740 1 -- start start 2024-05-10T11:25:23.300 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.297+0000 7f685136b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d5733afdb0 0x55d5733b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:23.300 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.297+0000 7f685136b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5733b07a0 0x55d5733b9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.297+0000 7f685136b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d5733b5910 0x55d5733ba4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.297+0000 7f685136b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f68481c4950 con 0x55d5733b07a0 2024-05-10T11:25:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.297+0000 7f685136b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f68481c5020 con 0x55d5733b5910 2024-05-10T11:25:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.297+0000 7f685136b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f68481c28f0 con 0x55d5733afdb0 2024-05-10T11:25:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.297+0000 7f683ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5733b07a0 0x55d5733b9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.297+0000 7f684cbf1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d5733afdb0 0x55d5733b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.297+0000 7f683ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5733b07a0 0x55d5733b9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53392/0 (socket says 172.21.15.100:53392) 2024-05-10T11:25:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.297+0000 7f684cbf1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d5733afdb0 0x55d5733b4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:50442/0 (socket says 172.21.15.100:50442) 2024-05-10T11:25:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.297+0000 7f683ffff640 1 -- 172.21.15.100:0/1037559032 learned_addr learned my addr 172.21.15.100:0/1037559032 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:25:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.297+0000 7f684cbf1640 1 --2- 172.21.15.100:0/1037559032 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d5733b5910 0x55d5733ba4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.297+0000 7f683ffff640 1 -- 172.21.15.100:0/1037559032 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d5733afdb0 msgr2=0x55d5733b4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.297+0000 7f683ffff640 1 --2- 172.21.15.100:0/1037559032 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d5733afdb0 0x55d5733b4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.297+0000 7f683ffff640 1 -- 172.21.15.100:0/1037559032 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d5733b5910 msgr2=0x55d5733ba4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.297+0000 7f683ffff640 1 --2- 172.21.15.100:0/1037559032 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d5733b5910 0x55d5733ba4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.297+0000 7f683ffff640 1 -- 172.21.15.100:0/1037559032 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5733b4c50 con 0x55d5733b07a0 2024-05-10T11:25:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.297+0000 7f684cbf1640 1 --2- 172.21.15.100:0/1037559032 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d5733afdb0 0x55d5733b4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:25:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.297+0000 7f683ffff640 1 --2- 172.21.15.100:0/1037559032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5733b07a0 0x55d5733b9f70 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f6830000c00 tx=0x7f683000d800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c33adb4f8d406c3a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:25:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.297+0000 7f683dffb640 1 -- 172.21.15.100:0/1037559032 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f683002c030 con 0x55d5733b07a0 2024-05-10T11:25:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.298+0000 7f683dffb640 1 -- 172.21.15.100:0/1037559032 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f68300103d0 con 0x55d5733b07a0 2024-05-10T11:25:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.298+0000 7f683dffb640 1 -- 172.21.15.100:0/1037559032 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68300047a0 con 0x55d5733b07a0 2024-05-10T11:25:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.298+0000 7f685136b740 1 -- 172.21.15.100:0/1037559032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5733b07a0 msgr2=0x55d5733b9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.298+0000 7f685136b740 1 --2- 172.21.15.100:0/1037559032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5733b07a0 0x55d5733b9f70 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f6830000c00 tx=0x7f683000d800 comp rx=0 tx=0).stop 2024-05-10T11:25:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.298+0000 7f685136b740 1 -- 172.21.15.100:0/1037559032 shutdown_connections 2024-05-10T11:25:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.298+0000 7f685136b740 1 --2- 172.21.15.100:0/1037559032 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d5733b5910 0x55d5733ba4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.298+0000 7f685136b740 1 --2- 172.21.15.100:0/1037559032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5733b07a0 0x55d5733b9f70 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.298+0000 7f685136b740 1 --2- 172.21.15.100:0/1037559032 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d5733afdb0 0x55d5733b4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.298+0000 7f685136b740 1 -- 172.21.15.100:0/1037559032 >> 172.21.15.100:0/1037559032 conn(0x55d5733ab420 msgr2=0x55d5733a63e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:25:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.298+0000 7f685136b740 1 -- 172.21.15.100:0/1037559032 shutdown_connections 2024-05-10T11:25:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.298+0000 7f685136b740 1 -- 172.21.15.100:0/1037559032 wait complete. 2024-05-10T11:25:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.299+0000 7f685136b740 1 -- start start 2024-05-10T11:25:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.299+0000 7f685136b740 1 Processor -- start 2024-05-10T11:25:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.299+0000 7f685136b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5733afdb0 0x55d5733be030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.299+0000 7f685136b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d5733b07a0 0x55d5733be570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.299+0000 7f685136b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d5733b5910 0x55d5733bf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.299+0000 7f685136b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f68481106f0 con 0x55d5733afdb0 2024-05-10T11:25:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.299+0000 7f685136b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f68481c2220 con 0x55d5733b07a0 2024-05-10T11:25:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.299+0000 7f685136b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f68481c1480 con 0x55d5733b5910 2024-05-10T11:25:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.299+0000 7f685136b740 10 client.0 _ll_register_callbacks cb 0x7f6830005eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:25:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.299+0000 7f684cbf1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5733afdb0 0x55d5733be030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.299+0000 7f684cbf1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5733afdb0 0x55d5733be030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53406/0 (socket says 172.21.15.100:53406) 2024-05-10T11:25:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.299+0000 7f684cbf1640 1 -- 172.21.15.100:0/2490041964 learned_addr learned my addr 172.21.15.100:0/2490041964 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:25:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.299+0000 7f683ffff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d5733b07a0 0x55d5733be570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.299+0000 7f684cbf1640 1 --2- 172.21.15.100:0/2490041964 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d5733b5910 0x55d5733bf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.300+0000 7f684cbf1640 1 -- 172.21.15.100:0/2490041964 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d5733b5910 msgr2=0x55d5733bf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.300+0000 7f684cbf1640 1 --2- 172.21.15.100:0/2490041964 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d5733b5910 0x55d5733bf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.300+0000 7f684cbf1640 1 -- 172.21.15.100:0/2490041964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d5733b07a0 msgr2=0x55d5733be570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.300+0000 7f684cbf1640 1 --2- 172.21.15.100:0/2490041964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d5733b07a0 0x55d5733be570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.300+0000 7f684cbf1640 1 -- 172.21.15.100:0/2490041964 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5733bfb00 con 0x55d5733afdb0 2024-05-10T11:25:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.300+0000 7f684cbf1640 1 --2- 172.21.15.100:0/2490041964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5733afdb0 0x55d5733be030 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f6840003fd0 tx=0x7f683000d790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc11e47a451a2d23 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:25:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.300+0000 7f68277fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:25:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.300+0000 7f68277fe640 1 -- 172.21.15.100:0/2490041964 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6840036040 con 0x55d5733afdb0 2024-05-10T11:25:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.300+0000 7f68277fe640 1 -- 172.21.15.100:0/2490041964 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f6840053c70 con 0x55d5733afdb0 2024-05-10T11:25:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.300+0000 7f685136b740 1 -- 172.21.15.100:0/2490041964 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68300103d0 con 0x55d5733afdb0 2024-05-10T11:25:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.300+0000 7f68277fe640 1 -- 172.21.15.100:0/2490041964 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6840062050 con 0x55d5733afdb0 2024-05-10T11:25:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.300+0000 7f685136b740 10 client.16761 fetch_fsmap learned FSMap version 31 2024-05-10T11:25:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.300+0000 7f685136b740 1 -- 172.21.15.100:0/2490041964 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d5733c0e10 con 0x55d5733afdb0 2024-05-10T11:25:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.300+0000 7f68277fe640 1 -- 172.21.15.100:0/2490041964 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f68300103d0 con 0x55d5733afdb0 2024-05-10T11:25:23.305 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-10T11:25:23.307 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.301+0000 7f68277fe640 1 -- 172.21.15.100:0/2490041964 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f6840004040 con 0x55d5733afdb0 2024-05-10T11:25:23.307 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.301+0000 7f685136b740 10 client.16761 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:25:23.307 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.301+0000 7f685136b740 10 client.16761 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:25:23.307 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.301+0000 7f685136b740 20 client.16761 populate_metadata read hostname 'smithi100' 2024-05-10T11:25:23.307 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.301+0000 7f685136b740 1 --2- 172.21.15.100:0/2490041964 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d5733c5cb0 0x55d5734328d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:23.307 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.301+0000 7f685136b740 4 client.16761 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:25:23.307 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.301+0000 7f685136b740 1 -- 172.21.15.100:0/2490041964 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d57331c050 con 0x55d5733c5cb0 2024-05-10T11:25:23.307 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.301+0000 7f683ffff640 1 --2- 172.21.15.100:0/2490041964 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d5733c5cb0 0x55d5734328d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:23.307 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.301+0000 7f68277fe640 10 client.16761 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:25:23.307 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.301+0000 7f683ffff640 1 --2- 172.21.15.100:0/2490041964 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d5733c5cb0 0x55d5734328d0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=317ba3711260c7b8 server_cookie=e3ab5fa7017fd6f1 in_seq=0 out_seq=0 2024-05-10T11:25:23.324 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.323+0000 7f68277fe640 1 -- 172.21.15.100:0/2490041964 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d57331c050 con 0x55d5733c5cb0 2024-05-10T11:25:23.324 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.323+0000 7f68277fe640 10 client.16761 handle_command_reply: tid=0 2024-05-10T11:25:23.324 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.323+0000 7f685136b740 1 client.16761 shutdown 2024-05-10T11:25:23.324 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.323+0000 7f685136b740 1 -- 172.21.15.100:0/2490041964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5733afdb0 msgr2=0x55d5733be030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:23.324 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.323+0000 7f685136b740 1 --2- 172.21.15.100:0/2490041964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5733afdb0 0x55d5733be030 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f6840003fd0 tx=0x7f683000d790 comp rx=0 tx=0).stop 2024-05-10T11:25:23.324 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.324+0000 7f685136b740 1 -- 172.21.15.100:0/2490041964 shutdown_connections 2024-05-10T11:25:23.324 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.324+0000 7f685136b740 1 --2- 172.21.15.100:0/2490041964 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d5733c5cb0 0x55d5734328d0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:23.324 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.324+0000 7f685136b740 1 --2- 172.21.15.100:0/2490041964 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d5733b5910 0x55d5733bf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:23.324 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.324+0000 7f68277fe640 0 client.16761 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:25:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.324+0000 7f685136b740 1 --2- 172.21.15.100:0/2490041964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d5733b07a0 0x55d5733be570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.324+0000 7f685136b740 1 --2- 172.21.15.100:0/2490041964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5733afdb0 0x55d5733be030 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.324+0000 7f685136b740 1 -- 172.21.15.100:0/2490041964 >> 172.21.15.100:0/2490041964 conn(0x55d5733ab420 msgr2=0x55d5733b1980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:25:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.324+0000 7f685136b740 1 -- 172.21.15.100:0/2490041964 shutdown_connections 2024-05-10T11:25:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.324+0000 7f685136b740 1 -- 172.21.15.100:0/2490041964 wait complete. 2024-05-10T11:25:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.324+0000 7f685136b740 20 client.16761 trim_cache size 0 max 16384 2024-05-10T11:25:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:25:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:25:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:25:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:25:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.324+0000 7f684e654640 1 -- 172.21.15.100:0/3084527541 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f68340b28f0 msgr2=0x7f68340b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.324+0000 7f684e654640 1 --2- 172.21.15.100:0/3084527541 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f68340b28f0 0x7f68340b4db0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f683800d950 tx=0x7f68380e5600 comp rx=0 tx=0).stop 2024-05-10T11:25:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.324+0000 7f684e654640 1 -- 172.21.15.100:0/3084527541 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6848100ce0 msgr2=0x7f6848071330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.324+0000 7f684e654640 1 --2- 172.21.15.100:0/3084527541 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6848100ce0 0x7f6848071330 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f683800d980 tx=0x7f683800de40 comp rx=0 tx=0).stop 2024-05-10T11:25:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.324+0000 7f684e654640 1 -- 172.21.15.100:0/3084527541 shutdown_connections 2024-05-10T11:25:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.324+0000 7f684e654640 1 --2- 172.21.15.100:0/3084527541 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f68340b28f0 0x7f68340b4db0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.324+0000 7f684e654640 1 --2- 172.21.15.100:0/3084527541 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6848102400 0x7f68481af120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.324+0000 7f684e654640 1 --2- 172.21.15.100:0/3084527541 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6848101a40 0x7f6848071870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.324+0000 7f684e654640 1 --2- 172.21.15.100:0/3084527541 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6848100ce0 0x7f6848071330 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.324+0000 7f684e654640 1 -- 172.21.15.100:0/3084527541 >> 172.21.15.100:0/3084527541 conn(0x7f68480fc610 msgr2=0x7f68480fd280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:25:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.325+0000 7f684e654640 1 -- 172.21.15.100:0/3084527541 shutdown_connections 2024-05-10T11:25:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:23.325+0000 7f684e654640 1 -- 172.21.15.100:0/3084527541 wait complete. 2024-05-10T11:25:23.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:25:23 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/890071779' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:25:23.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:25:23 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/890071779' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:25:23.577 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:25:23.974 INFO:tasks.workunit.client.0.smithi100.stdout: 1 164805 2.73 MB/sec execute 198 sec latency 15.482 ms 2024-05-10T11:25:24.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:25:24 smithi100 ceph-mon[30296]: pgmap v630: 129 pgs: 129 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 1.3 MiB/s wr, 153 op/s 2024-05-10T11:25:24.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:25:24 smithi178 ceph-mon[33359]: pgmap v630: 129 pgs: 129 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 1.3 MiB/s wr, 153 op/s 2024-05-10T11:25:24.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:25:24 smithi188 ceph-mon[34075]: pgmap v630: 129 pgs: 129 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 1.3 MiB/s wr, 153 op/s 2024-05-10T11:25:24.974 INFO:tasks.workunit.client.0.smithi100.stdout: 1 165517 2.73 MB/sec execute 199 sec latency 16.966 ms 2024-05-10T11:25:25.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.367+0000 7fda5ac31700 1 -- 172.21.15.100:0/921778223 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda54102d50 msgr2=0x7fda5410f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:25.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.367+0000 7fda5ac31700 1 --2- 172.21.15.100:0/921778223 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda54102d50 0x7fda5410f240 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fda4c00b0a0 tx=0x7fda4c00b3b0 comp rx=0 tx=0).stop 2024-05-10T11:25:25.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.367+0000 7fda5ac31700 1 -- 172.21.15.100:0/921778223 shutdown_connections 2024-05-10T11:25:25.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.367+0000 7fda5ac31700 1 --2- 172.21.15.100:0/921778223 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda54102d50 0x7fda5410f240 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:25.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.367+0000 7fda5ac31700 1 --2- 172.21.15.100:0/921778223 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda54108400 0x7fda541087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:25.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.367+0000 7fda5ac31700 1 --2- 172.21.15.100:0/921778223 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda54102390 0x7fda54102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:25.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.367+0000 7fda5ac31700 1 -- 172.21.15.100:0/921778223 >> 172.21.15.100:0/921778223 conn(0x7fda54079310 msgr2=0x7fda54079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:25:25.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.367+0000 7fda5ac31700 1 -- 172.21.15.100:0/921778223 shutdown_connections 2024-05-10T11:25:25.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.367+0000 7fda5ac31700 1 -- 172.21.15.100:0/921778223 wait complete. 2024-05-10T11:25:25.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.368+0000 7fda5ac31700 1 Processor -- start 2024-05-10T11:25:25.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.368+0000 7fda5ac31700 1 -- start start 2024-05-10T11:25:25.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.368+0000 7fda5ac31700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda54102390 0x7fda54104070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:25.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.368+0000 7fda5ac31700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda54102d50 0x7fda541045b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:25.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.368+0000 7fda5ac31700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda54108400 0x7fda54104af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:25.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.368+0000 7fda5ac31700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fda54112450 con 0x7fda54102390 2024-05-10T11:25:25.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.368+0000 7fda5ac31700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fda541122d0 con 0x7fda54108400 2024-05-10T11:25:25.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.368+0000 7fda5ac31700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fda541125d0 con 0x7fda54102d50 2024-05-10T11:25:25.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.368+0000 7fda589cd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda54102390 0x7fda54104070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:25.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.368+0000 7fda589cd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda54102390 0x7fda54104070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53432/0 (socket says 172.21.15.100:53432) 2024-05-10T11:25:25.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.368+0000 7fda589cd700 1 -- 172.21.15.100:0/1392961530 learned_addr learned my addr 172.21.15.100:0/1392961530 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:25:25.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.368+0000 7fda53fff700 1 --2- 172.21.15.100:0/1392961530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda54102d50 0x7fda541045b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:25.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.368+0000 7fda591ce700 1 --2- 172.21.15.100:0/1392961530 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda54108400 0x7fda54104af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:25.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.369+0000 7fda53fff700 1 -- 172.21.15.100:0/1392961530 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda54108400 msgr2=0x7fda54104af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:25.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.369+0000 7fda53fff700 1 --2- 172.21.15.100:0/1392961530 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda54108400 0x7fda54104af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:25.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.369+0000 7fda53fff700 1 -- 172.21.15.100:0/1392961530 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda54102390 msgr2=0x7fda54104070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:25.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.369+0000 7fda53fff700 1 --2- 172.21.15.100:0/1392961530 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda54102390 0x7fda54104070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:25.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.369+0000 7fda53fff700 1 -- 172.21.15.100:0/1392961530 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda541a8060 con 0x7fda54102d50 2024-05-10T11:25:25.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.369+0000 7fda589cd700 1 --2- 172.21.15.100:0/1392961530 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda54102390 0x7fda54104070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:25:25.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.369+0000 7fda591ce700 1 --2- 172.21.15.100:0/1392961530 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda54108400 0x7fda54104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:25:25.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.369+0000 7fda53fff700 1 --2- 172.21.15.100:0/1392961530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda54102d50 0x7fda541045b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fda4000e9f0 tx=0x7fda4000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:25:25.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.369+0000 7fda51ffb700 1 -- 172.21.15.100:0/1392961530 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda40004220 con 0x7fda54102d50 2024-05-10T11:25:25.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.369+0000 7fda5ac31700 1 -- 172.21.15.100:0/1392961530 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda541a82f0 con 0x7fda54102d50 2024-05-10T11:25:25.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.369+0000 7fda51ffb700 1 -- 172.21.15.100:0/1392961530 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fda400043c0 con 0x7fda54102d50 2024-05-10T11:25:25.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.369+0000 7fda51ffb700 1 -- 172.21.15.100:0/1392961530 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda40005020 con 0x7fda54102d50 2024-05-10T11:25:25.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.369+0000 7fda5ac31700 1 -- 172.21.15.100:0/1392961530 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda541a8830 con 0x7fda54102d50 2024-05-10T11:25:25.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.372+0000 7fda51ffb700 1 -- 172.21.15.100:0/1392961530 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fda40004750 con 0x7fda54102d50 2024-05-10T11:25:25.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.372+0000 7fda51ffb700 1 --2- 172.21.15.100:0/1392961530 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fda440b2880 0x7fda440b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:25.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.373+0000 7fda589cd700 1 --2- 172.21.15.100:0/1392961530 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fda440b2880 0x7fda440b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:25.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.373+0000 7fda51ffb700 1 -- 172.21.15.100:0/1392961530 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fda400e24c0 con 0x7fda54102d50 2024-05-10T11:25:25.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.373+0000 7fda589cd700 1 --2- 172.21.15.100:0/1392961530 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fda440b2880 0x7fda440b4d40 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fda48011550 tx=0x7fda48009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:25:25.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.373+0000 7fda654c4b80 1 Processor -- start 2024-05-10T11:25:25.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.373+0000 7fda654c4b80 1 -- start start 2024-05-10T11:25:25.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.374+0000 7fda654c4b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565254fea4d0 0x565254fea8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:25.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.374+0000 7fda654c4b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565254feadf0 0x565254fee6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:25.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.374+0000 7fda654c4b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565254feec80 0x565254fef100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:25.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.374+0000 7fda654c4b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x565254f3c860 con 0x565254feec80 2024-05-10T11:25:25.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.374+0000 7fda654c4b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x565254edb7c0 con 0x565254fea4d0 2024-05-10T11:25:25.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.374+0000 7fda654c4b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x565254f38820 con 0x565254feadf0 2024-05-10T11:25:25.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.374+0000 7fda53fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565254feadf0 0x565254fee6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:25.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.374+0000 7fda589cd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565254fea4d0 0x565254fea8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:25.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.374+0000 7fda591ce700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565254feec80 0x565254fef100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:25.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.374+0000 7fda53fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565254feadf0 0x565254fee6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:50470/0 (socket says 172.21.15.100:50470) 2024-05-10T11:25:25.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.374+0000 7fda53fff700 1 -- 172.21.15.100:0/866574656 learned_addr learned my addr 172.21.15.100:0/866574656 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:25:25.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.374+0000 7fda53fff700 1 -- 172.21.15.100:0/866574656 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565254fea4d0 msgr2=0x565254fea8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:25.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.374+0000 7fda53fff700 1 --2- 172.21.15.100:0/866574656 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565254fea4d0 0x565254fea8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:25.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.374+0000 7fda53fff700 1 -- 172.21.15.100:0/866574656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565254feec80 msgr2=0x565254fef100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:25.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.374+0000 7fda53fff700 1 --2- 172.21.15.100:0/866574656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565254feec80 0x565254fef100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:25.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.374+0000 7fda53fff700 1 -- 172.21.15.100:0/866574656 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565254e898d0 con 0x565254feadf0 2024-05-10T11:25:25.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.374+0000 7fda589cd700 1 --2- 172.21.15.100:0/866574656 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565254fea4d0 0x565254fea8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:25:25.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.374+0000 7fda53fff700 1 --2- 172.21.15.100:0/866574656 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565254feadf0 0x565254fee6f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fda400a5800 tx=0x7fda40004de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf728c4114e53311 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:25:25.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.374+0000 7fda3effd700 1 -- 172.21.15.100:0/866574656 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda4001c070 con 0x565254feadf0 2024-05-10T11:25:25.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.374+0000 7fda3effd700 1 -- 172.21.15.100:0/866574656 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fda400e2070 con 0x565254feadf0 2024-05-10T11:25:25.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.374+0000 7fda3effd700 1 -- 172.21.15.100:0/866574656 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda400a99e0 con 0x565254feadf0 2024-05-10T11:25:25.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.375+0000 7fda654c4b80 1 -- 172.21.15.100:0/866574656 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565254feadf0 msgr2=0x565254fee6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:25.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.375+0000 7fda654c4b80 1 --2- 172.21.15.100:0/866574656 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565254feadf0 0x565254fee6f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fda400a5800 tx=0x7fda40004de0 comp rx=0 tx=0).stop 2024-05-10T11:25:25.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.375+0000 7fda654c4b80 1 -- 172.21.15.100:0/866574656 shutdown_connections 2024-05-10T11:25:25.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.375+0000 7fda654c4b80 1 --2- 172.21.15.100:0/866574656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565254feec80 0x565254fef100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:25.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.375+0000 7fda654c4b80 1 --2- 172.21.15.100:0/866574656 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565254fea4d0 0x565254fea8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:25.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.375+0000 7fda654c4b80 1 --2- 172.21.15.100:0/866574656 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565254feadf0 0x565254fee6f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:25.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.375+0000 7fda654c4b80 1 -- 172.21.15.100:0/866574656 >> 172.21.15.100:0/866574656 conn(0x565254f16100 msgr2=0x565254fe4c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:25:25.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.375+0000 7fda654c4b80 1 -- 172.21.15.100:0/866574656 shutdown_connections 2024-05-10T11:25:25.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.375+0000 7fda654c4b80 1 -- 172.21.15.100:0/866574656 wait complete. 2024-05-10T11:25:25.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.376+0000 7fda654c4b80 1 -- start start 2024-05-10T11:25:25.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.376+0000 7fda654c4b80 1 Processor -- start 2024-05-10T11:25:25.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.376+0000 7fda654c4b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565254fea4d0 0x565254ff6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:25.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.376+0000 7fda654c4b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565254feadf0 0x565254ff34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:25.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.376+0000 7fda654c4b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565254feec80 0x565254ff39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:25.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.376+0000 7fda654c4b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x565254ff6e90 con 0x565254fea4d0 2024-05-10T11:25:25.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.376+0000 7fda654c4b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x565254ea2c30 con 0x565254feec80 2024-05-10T11:25:25.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.376+0000 7fda654c4b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x565254ff7010 con 0x565254feadf0 2024-05-10T11:25:25.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.376+0000 7fda654c4b80 10 client.0 _ll_register_callbacks cb 0x565254ed4920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:25:25.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.376+0000 7fda591ce700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565254feec80 0x565254ff39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:25.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.376+0000 7fda589cd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565254fea4d0 0x565254ff6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:25.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.376+0000 7fda591ce700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565254feec80 0x565254ff39f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:54710/0 (socket says 172.21.15.100:54710) 2024-05-10T11:25:25.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.376+0000 7fda591ce700 1 -- 172.21.15.100:0/1208060039 learned_addr learned my addr 172.21.15.100:0/1208060039 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:25:25.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.377+0000 7fda53fff700 1 --2- 172.21.15.100:0/1208060039 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565254feadf0 0x565254ff34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:25.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.377+0000 7fda591ce700 1 -- 172.21.15.100:0/1208060039 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565254feadf0 msgr2=0x565254ff34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:25.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.377+0000 7fda591ce700 1 --2- 172.21.15.100:0/1208060039 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565254feadf0 0x565254ff34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:25.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.377+0000 7fda591ce700 1 -- 172.21.15.100:0/1208060039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565254fea4d0 msgr2=0x565254ff6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:25.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.377+0000 7fda591ce700 1 --2- 172.21.15.100:0/1208060039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565254fea4d0 0x565254ff6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:25.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.377+0000 7fda591ce700 1 -- 172.21.15.100:0/1208060039 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565254ed8690 con 0x565254feec80 2024-05-10T11:25:25.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.377+0000 7fda589cd700 1 --2- 172.21.15.100:0/1208060039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565254fea4d0 0x565254ff6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:25:25.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.377+0000 7fda591ce700 1 --2- 172.21.15.100:0/1208060039 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565254feec80 0x565254ff39f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fda4c00b070 tx=0x7fda4c003c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c45ba017a5739a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:25:25.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.377+0000 7fda33fff700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:25:25.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.377+0000 7fda33fff700 1 -- 172.21.15.100:0/1208060039 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda4c0078f0 con 0x565254feec80 2024-05-10T11:25:25.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.377+0000 7fda654c4b80 1 -- 172.21.15.100:0/1208060039 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda400e2070 con 0x565254feec80 2024-05-10T11:25:25.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.377+0000 7fda33fff700 1 -- 172.21.15.100:0/1208060039 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fda4c007a90 con 0x565254feec80 2024-05-10T11:25:25.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.377+0000 7fda33fff700 1 -- 172.21.15.100:0/1208060039 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fda4c007c90 con 0x565254feec80 2024-05-10T11:25:25.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.377+0000 7fda654c4b80 10 client.26219 fetch_fsmap learned FSMap version 31 2024-05-10T11:25:25.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.377+0000 7fda654c4b80 1 -- 172.21.15.100:0/1208060039 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x565255001790 con 0x565254feec80 2024-05-10T11:25:25.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.378+0000 7fda33fff700 1 -- 172.21.15.100:0/1208060039 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fda400e2070 con 0x565254feec80 2024-05-10T11:25:25.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.378+0000 7fda33fff700 1 -- 172.21.15.100:0/1208060039 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fda4c023070 con 0x565254feec80 2024-05-10T11:25:25.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.378+0000 7fda654c4b80 10 client.26219 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:25:25.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.378+0000 7fda654c4b80 10 client.26219 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-10T11:25:25.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.378+0000 7fda654c4b80 20 client.26219 populate_metadata read hostname 'smithi100' 2024-05-10T11:25:25.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.378+0000 7fda654c4b80 1 --2- 172.21.15.100:0/1208060039 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x565255003d20 0x565255006110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:25.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.378+0000 7fda654c4b80 4 client.26219 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:25:25.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.378+0000 7fda654c4b80 1 -- 172.21.15.100:0/1208060039 --> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565254fb6650 con 0x565255003d20 2024-05-10T11:25:25.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.378+0000 7fda589cd700 1 --2- 172.21.15.100:0/1208060039 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x565255003d20 0x565255006110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:25.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.379+0000 7fda33fff700 10 client.26219 ms_handle_connect on v2:172.21.15.188:6836/1943342620 2024-05-10T11:25:25.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.379+0000 7fda589cd700 1 --2- 172.21.15.100:0/1208060039 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x565255003d20 0x565255006110 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a6401f83deac1e3b server_cookie=f4369b068920f004 in_seq=0 out_seq=0 2024-05-10T11:25:25.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.379+0000 7fda33fff700 1 -- 172.21.15.100:0/1208060039 <== mds.2 v2:172.21.15.188:6836/1943342620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x565254fb6650 con 0x565255003d20 2024-05-10T11:25:25.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.379+0000 7fda33fff700 10 client.26219 handle_command_reply: tid=0 2024-05-10T11:25:25.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.379+0000 7fda654c4b80 1 client.26219 shutdown 2024-05-10T11:25:25.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.379+0000 7fda654c4b80 1 -- 172.21.15.100:0/1208060039 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565254feec80 msgr2=0x565254ff39f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:25.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.379+0000 7fda654c4b80 1 --2- 172.21.15.100:0/1208060039 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565254feec80 0x565254ff39f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fda4c00b070 tx=0x7fda4c003c30 comp rx=0 tx=0).stop 2024-05-10T11:25:25.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.379+0000 7fda654c4b80 1 -- 172.21.15.100:0/1208060039 shutdown_connections 2024-05-10T11:25:25.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.379+0000 7fda654c4b80 1 --2- 172.21.15.100:0/1208060039 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x565255003d20 0x565255006110 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:25.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.379+0000 7fda654c4b80 1 --2- 172.21.15.100:0/1208060039 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565254feec80 0x565254ff39f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:25.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.379+0000 7fda33fff700 0 client.26219 ms_handle_reset on v2:172.21.15.188:6836/1943342620 2024-05-10T11:25:25.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.379+0000 7fda654c4b80 1 --2- 172.21.15.100:0/1208060039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565254fea4d0 0x565254ff6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:25.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.379+0000 7fda654c4b80 1 --2- 172.21.15.100:0/1208060039 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565254feadf0 0x565254ff34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:25.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.379+0000 7fda654c4b80 1 -- 172.21.15.100:0/1208060039 >> 172.21.15.100:0/1208060039 conn(0x565254f16100 msgr2=0x565254fdf720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:25:25.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.380+0000 7fda654c4b80 1 -- 172.21.15.100:0/1208060039 shutdown_connections 2024-05-10T11:25:25.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.380+0000 7fda654c4b80 1 -- 172.21.15.100:0/1208060039 wait complete. 2024-05-10T11:25:25.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.380+0000 7fda654c4b80 20 client.26219 trim_cache size 0 max 16384 2024-05-10T11:25:25.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.395+0000 7fda654c4b80 1 Processor -- start 2024-05-10T11:25:25.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.395+0000 7fda654c4b80 1 -- start start 2024-05-10T11:25:25.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.395+0000 7fda654c4b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565254fea4d0 0x565254ffff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:25.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.395+0000 7fda654c4b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565254feadf0 0x565254fe4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:25.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.395+0000 7fda654c4b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565254feec80 0x56525500b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:25.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.395+0000 7fda654c4b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fda541b3a70 con 0x565254feadf0 2024-05-10T11:25:25.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.395+0000 7fda654c4b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fda541b1300 con 0x565254feec80 2024-05-10T11:25:25.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.395+0000 7fda654c4b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fda541af150 con 0x565254fea4d0 2024-05-10T11:25:25.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.395+0000 7fda53fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565254feadf0 0x565254fe4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:25.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.395+0000 7fda53fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565254feadf0 0x565254fe4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53464/0 (socket says 172.21.15.100:53464) 2024-05-10T11:25:25.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.395+0000 7fda53fff700 1 -- 172.21.15.100:0/3576547204 learned_addr learned my addr 172.21.15.100:0/3576547204 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:25:25.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.395+0000 7fda591ce700 1 --2- 172.21.15.100:0/3576547204 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565254feec80 0x56525500b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:25.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.395+0000 7fda589cd700 1 --2- 172.21.15.100:0/3576547204 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565254fea4d0 0x565254ffff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:25.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.396+0000 7fda53fff700 1 -- 172.21.15.100:0/3576547204 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565254fea4d0 msgr2=0x565254ffff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:25.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.396+0000 7fda53fff700 1 --2- 172.21.15.100:0/3576547204 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565254fea4d0 0x565254ffff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:25.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.396+0000 7fda53fff700 1 -- 172.21.15.100:0/3576547204 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565254feec80 msgr2=0x56525500b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:25.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.396+0000 7fda53fff700 1 --2- 172.21.15.100:0/3576547204 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565254feec80 0x56525500b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:25.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.396+0000 7fda53fff700 1 -- 172.21.15.100:0/3576547204 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565254ff4390 con 0x565254feadf0 2024-05-10T11:25:25.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.396+0000 7fda591ce700 1 --2- 172.21.15.100:0/3576547204 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565254feec80 0x56525500b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:25:25.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.396+0000 7fda53fff700 1 --2- 172.21.15.100:0/3576547204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565254feadf0 0x565254fe4bf0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fda400a9480 tx=0x7fda400e26c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e5155b762621b7b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:25:25.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.396+0000 7fda3e7fc700 1 -- 172.21.15.100:0/3576547204 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda4001c040 con 0x565254feadf0 2024-05-10T11:25:25.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.396+0000 7fda3e7fc700 1 -- 172.21.15.100:0/3576547204 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fda400e2040 con 0x565254feadf0 2024-05-10T11:25:25.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.396+0000 7fda3e7fc700 1 -- 172.21.15.100:0/3576547204 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda400a9c40 con 0x565254feadf0 2024-05-10T11:25:25.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.397+0000 7fda654c4b80 1 -- 172.21.15.100:0/3576547204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565254feadf0 msgr2=0x565254fe4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:25.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.397+0000 7fda654c4b80 1 --2- 172.21.15.100:0/3576547204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565254feadf0 0x565254fe4bf0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fda400a9480 tx=0x7fda400e26c0 comp rx=0 tx=0).stop 2024-05-10T11:25:25.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.397+0000 7fda654c4b80 1 -- 172.21.15.100:0/3576547204 shutdown_connections 2024-05-10T11:25:25.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.397+0000 7fda654c4b80 1 --2- 172.21.15.100:0/3576547204 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565254feec80 0x56525500b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:25.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.397+0000 7fda654c4b80 1 --2- 172.21.15.100:0/3576547204 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565254fea4d0 0x565254ffff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:25.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.397+0000 7fda654c4b80 1 --2- 172.21.15.100:0/3576547204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565254feadf0 0x565254fe4bf0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:25.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.397+0000 7fda654c4b80 1 -- 172.21.15.100:0/3576547204 >> 172.21.15.100:0/3576547204 conn(0x565254f16100 msgr2=0x565254fde710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:25:25.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.397+0000 7fda654c4b80 1 -- 172.21.15.100:0/3576547204 shutdown_connections 2024-05-10T11:25:25.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.397+0000 7fda654c4b80 1 -- 172.21.15.100:0/3576547204 wait complete. 2024-05-10T11:25:25.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.397+0000 7fda654c4b80 1 -- start start 2024-05-10T11:25:25.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.397+0000 7fda654c4b80 1 Processor -- start 2024-05-10T11:25:25.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.397+0000 7fda654c4b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565254fea4d0 0x565254ff7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:25.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.397+0000 7fda654c4b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565254feadf0 0x565254ff7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:25.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.397+0000 7fda654c4b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565254feec80 0x565255066530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:25.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.397+0000 7fda654c4b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fda541af7c0 con 0x565254fea4d0 2024-05-10T11:25:25.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.397+0000 7fda654c4b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fda541b0c30 con 0x565254feadf0 2024-05-10T11:25:25.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.397+0000 7fda654c4b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fda541afe90 con 0x565254feec80 2024-05-10T11:25:25.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.397+0000 7fda654c4b80 10 client.0 _ll_register_callbacks cb 0x565254ed4920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:25:25.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.397+0000 7fda589cd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565254fea4d0 0x565254ff7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:25.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.398+0000 7fda53fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565254feadf0 0x565254ff7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:25.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.398+0000 7fda589cd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565254fea4d0 0x565254ff7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53476/0 (socket says 172.21.15.100:53476) 2024-05-10T11:25:25.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.398+0000 7fda589cd700 1 -- 172.21.15.100:0/4059889344 learned_addr learned my addr 172.21.15.100:0/4059889344 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:25:25.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.398+0000 7fda591ce700 1 --2- 172.21.15.100:0/4059889344 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565254feec80 0x565255066530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:25.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.398+0000 7fda53fff700 1 -- 172.21.15.100:0/4059889344 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565254feec80 msgr2=0x565255066530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:25.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.398+0000 7fda53fff700 1 --2- 172.21.15.100:0/4059889344 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565254feec80 0x565255066530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:25.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.398+0000 7fda53fff700 1 -- 172.21.15.100:0/4059889344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565254fea4d0 msgr2=0x565254ff7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:25.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.398+0000 7fda53fff700 1 --2- 172.21.15.100:0/4059889344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565254fea4d0 0x565254ff7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:25.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.398+0000 7fda53fff700 1 -- 172.21.15.100:0/4059889344 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565255066af0 con 0x565254feadf0 2024-05-10T11:25:25.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.398+0000 7fda589cd700 1 --2- 172.21.15.100:0/4059889344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565254fea4d0 0x565254ff7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:25:25.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.398+0000 7fda53fff700 1 --2- 172.21.15.100:0/4059889344 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565254feadf0 0x565254ff7b80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fda400dfcb0 tx=0x7fda400e2420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb987d3c622d8a51 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:25:25.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.398+0000 7fda33fff700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:25:25.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.398+0000 7fda33fff700 1 -- 172.21.15.100:0/4059889344 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda4001c050 con 0x565254feadf0 2024-05-10T11:25:25.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.398+0000 7fda654c4b80 1 -- 172.21.15.100:0/4059889344 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda400e2040 con 0x565254feadf0 2024-05-10T11:25:25.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.398+0000 7fda33fff700 1 -- 172.21.15.100:0/4059889344 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fda400a9bf0 con 0x565254feadf0 2024-05-10T11:25:25.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.398+0000 7fda33fff700 1 -- 172.21.15.100:0/4059889344 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fda400adb30 con 0x565254feadf0 2024-05-10T11:25:25.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.398+0000 7fda654c4b80 10 client.26225 fetch_fsmap learned FSMap version 31 2024-05-10T11:25:25.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.398+0000 7fda654c4b80 1 -- 172.21.15.100:0/4059889344 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56525506de80 con 0x565254feadf0 2024-05-10T11:25:25.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.399+0000 7fda33fff700 1 -- 172.21.15.100:0/4059889344 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fda400e2040 con 0x565254feadf0 2024-05-10T11:25:25.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.399+0000 7fda33fff700 1 -- 172.21.15.100:0/4059889344 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fda400b2150 con 0x565254feadf0 2024-05-10T11:25:25.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.399+0000 7fda654c4b80 10 client.26225 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:25:25.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.399+0000 7fda654c4b80 10 client.26225 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-10T11:25:25.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.399+0000 7fda654c4b80 20 client.26225 populate_metadata read hostname 'smithi100' 2024-05-10T11:25:25.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.399+0000 7fda654c4b80 1 --2- 172.21.15.100:0/4059889344 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x56525506e100 0x5652550704f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:25.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.399+0000 7fda654c4b80 4 client.26225 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:25:25.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.399+0000 7fda654c4b80 1 -- 172.21.15.100:0/4059889344 --> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] -- command(tid 0: {"prefix": "damage ls"}) -- 0x565254f4a240 con 0x56525506e100 2024-05-10T11:25:25.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.399+0000 7fda589cd700 1 --2- 172.21.15.100:0/4059889344 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x56525506e100 0x5652550704f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:25.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.399+0000 7fda589cd700 1 --2- 172.21.15.100:0/4059889344 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x56525506e100 0x5652550704f0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ac1f913f6876389a server_cookie=90e14f4d2cd011 in_seq=0 out_seq=0 2024-05-10T11:25:25.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.399+0000 7fda33fff700 10 client.26225 ms_handle_connect on v2:172.21.15.188:6836/1943342620 2024-05-10T11:25:25.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.400+0000 7fda33fff700 1 -- 172.21.15.100:0/4059889344 <== mds.2 v2:172.21.15.188:6836/1943342620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x565254f4a240 con 0x56525506e100 2024-05-10T11:25:25.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.400+0000 7fda33fff700 10 client.26225 handle_command_reply: tid=0 2024-05-10T11:25:25.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.400+0000 7fda654c4b80 1 client.26225 shutdown 2024-05-10T11:25:25.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.400+0000 7fda654c4b80 1 -- 172.21.15.100:0/4059889344 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565254feadf0 msgr2=0x565254ff7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:25.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.400+0000 7fda654c4b80 1 --2- 172.21.15.100:0/4059889344 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565254feadf0 0x565254ff7b80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fda400dfcb0 tx=0x7fda400e2420 comp rx=0 tx=0).stop 2024-05-10T11:25:25.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.400+0000 7fda654c4b80 1 -- 172.21.15.100:0/4059889344 shutdown_connections 2024-05-10T11:25:25.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.400+0000 7fda654c4b80 1 --2- 172.21.15.100:0/4059889344 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x56525506e100 0x5652550704f0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:25.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.400+0000 7fda654c4b80 1 --2- 172.21.15.100:0/4059889344 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565254feec80 0x565255066530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:25.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.400+0000 7fda33fff700 0 client.26225 ms_handle_reset on v2:172.21.15.188:6836/1943342620 2024-05-10T11:25:25.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.400+0000 7fda654c4b80 1 --2- 172.21.15.100:0/4059889344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565254fea4d0 0x565254ff7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:25.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.400+0000 7fda654c4b80 1 --2- 172.21.15.100:0/4059889344 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565254feadf0 0x565254ff7b80 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:25.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.400+0000 7fda654c4b80 1 -- 172.21.15.100:0/4059889344 >> 172.21.15.100:0/4059889344 conn(0x565254f16100 msgr2=0x565254fe4b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:25:25.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.400+0000 7fda654c4b80 1 -- 172.21.15.100:0/4059889344 shutdown_connections 2024-05-10T11:25:25.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.400+0000 7fda654c4b80 1 -- 172.21.15.100:0/4059889344 wait complete. 2024-05-10T11:25:25.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.400+0000 7fda654c4b80 20 client.26225 trim_cache size 0 max 16384 2024-05-10T11:25:25.406 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:25:25.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.401+0000 7fda5ac31700 1 -- 172.21.15.100:0/1392961530 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fda440b2880 msgr2=0x7fda440b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:25.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.401+0000 7fda5ac31700 1 --2- 172.21.15.100:0/1392961530 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fda440b2880 0x7fda440b4d40 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fda48011550 tx=0x7fda48009260 comp rx=0 tx=0).stop 2024-05-10T11:25:25.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.401+0000 7fda5ac31700 1 -- 172.21.15.100:0/1392961530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda54102d50 msgr2=0x7fda541045b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:25.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.401+0000 7fda5ac31700 1 --2- 172.21.15.100:0/1392961530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda54102d50 0x7fda541045b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fda4000e9f0 tx=0x7fda4000ed00 comp rx=0 tx=0).stop 2024-05-10T11:25:25.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.401+0000 7fda5ac31700 1 -- 172.21.15.100:0/1392961530 shutdown_connections 2024-05-10T11:25:25.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.401+0000 7fda5ac31700 1 --2- 172.21.15.100:0/1392961530 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fda440b2880 0x7fda440b4d40 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:25.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.401+0000 7fda5ac31700 1 --2- 172.21.15.100:0/1392961530 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda54108400 0x7fda54104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:25.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.401+0000 7fda5ac31700 1 --2- 172.21.15.100:0/1392961530 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda54102390 0x7fda54104070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:25.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.401+0000 7fda5ac31700 1 --2- 172.21.15.100:0/1392961530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda54102d50 0x7fda541045b0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:25.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.401+0000 7fda5ac31700 1 -- 172.21.15.100:0/1392961530 >> 172.21.15.100:0/1392961530 conn(0x7fda54079310 msgr2=0x7fda541016c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:25:25.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.401+0000 7fda5ac31700 1 -- 172.21.15.100:0/1392961530 shutdown_connections 2024-05-10T11:25:25.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:25.401+0000 7fda5ac31700 1 -- 172.21.15.100:0/1392961530 wait complete. 2024-05-10T11:25:25.974 INFO:tasks.workunit.client.0.smithi100.stdout: 1 166082 2.74 MB/sec execute 200 sec latency 15.208 ms 2024-05-10T11:25:26.166 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:25:26 smithi100 ceph-mon[30296]: pgmap v631: 129 pgs: 129 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 1.4 MiB/s wr, 149 op/s 2024-05-10T11:25:26.195 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-10T11:25:26.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:25:26 smithi178 ceph-mon[33359]: pgmap v631: 129 pgs: 129 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 1.4 MiB/s wr, 149 op/s 2024-05-10T11:25:26.507 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:25:26.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:25:26 smithi188 ceph-mon[34075]: pgmap v631: 129 pgs: 129 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 1.4 MiB/s wr, 149 op/s 2024-05-10T11:25:26.974 INFO:tasks.workunit.client.0.smithi100.stdout: 1 166551 2.74 MB/sec execute 201 sec latency 15.922 ms 2024-05-10T11:25:27.974 INFO:tasks.workunit.client.0.smithi100.stdout: 1 167032 2.75 MB/sec execute 202 sec latency 15.632 ms 2024-05-10T11:25:28.230 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:25:28 smithi100 ceph-mon[30296]: pgmap v632: 129 pgs: 129 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 1.3 MiB/s wr, 134 op/s 2024-05-10T11:25:28.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.342+0000 7f066f645700 1 -- 172.21.15.100:0/1770128614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0668102390 msgr2=0x7f0668102810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:28.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.342+0000 7f066f645700 1 --2- 172.21.15.100:0/1770128614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0668102390 0x7f0668102810 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f0658009990 tx=0x7f0658009ca0 comp rx=0 tx=0).stop 2024-05-10T11:25:28.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.342+0000 7f066f645700 1 -- 172.21.15.100:0/1770128614 shutdown_connections 2024-05-10T11:25:28.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.342+0000 7f066f645700 1 --2- 172.21.15.100:0/1770128614 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0668102d50 0x7f066810f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:28.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.342+0000 7f066f645700 1 --2- 172.21.15.100:0/1770128614 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0668108400 0x7f06681087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:28.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.342+0000 7f066f645700 1 --2- 172.21.15.100:0/1770128614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0668102390 0x7f0668102810 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:28.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.342+0000 7f066f645700 1 -- 172.21.15.100:0/1770128614 >> 172.21.15.100:0/1770128614 conn(0x7f0668079310 msgr2=0x7f0668079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:25:28.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.342+0000 7f066f645700 1 -- 172.21.15.100:0/1770128614 shutdown_connections 2024-05-10T11:25:28.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.342+0000 7f066f645700 1 -- 172.21.15.100:0/1770128614 wait complete. 2024-05-10T11:25:28.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.343+0000 7f066f645700 1 Processor -- start 2024-05-10T11:25:28.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.343+0000 7f066f645700 1 -- start start 2024-05-10T11:25:28.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.343+0000 7f066f645700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0668102390 0x7f06681a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:28.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.343+0000 7f066f645700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0668102d50 0x7f06681a1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:28.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.343+0000 7f066f645700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0668108400 0x7f066819bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:28.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.343+0000 7f066f645700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f06681156f0 con 0x7f0668108400 2024-05-10T11:25:28.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.343+0000 7f066f645700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0668115570 con 0x7f0668102390 2024-05-10T11:25:28.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.343+0000 7f066f645700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0668115870 con 0x7f0668102d50 2024-05-10T11:25:28.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.343+0000 7f066dbe2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0668108400 0x7f066819bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:28.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.343+0000 7f066cbe0700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0668102d50 0x7f06681a1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:28.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.343+0000 7f066dbe2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0668108400 0x7f066819bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51490/0 (socket says 172.21.15.100:51490) 2024-05-10T11:25:28.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.343+0000 7f066dbe2700 1 -- 172.21.15.100:0/374705138 learned_addr learned my addr 172.21.15.100:0/374705138 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:25:28.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.343+0000 7f066d3e1700 1 --2- 172.21.15.100:0/374705138 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0668102390 0x7f06681a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:28.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.344+0000 7f066dbe2700 1 -- 172.21.15.100:0/374705138 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0668102d50 msgr2=0x7f06681a1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:28.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.344+0000 7f066dbe2700 1 --2- 172.21.15.100:0/374705138 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0668102d50 0x7f06681a1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:28.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.344+0000 7f066dbe2700 1 -- 172.21.15.100:0/374705138 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0668102390 msgr2=0x7f06681a18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:28.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.344+0000 7f066dbe2700 1 --2- 172.21.15.100:0/374705138 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0668102390 0x7f06681a18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:28.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.344+0000 7f066dbe2700 1 -- 172.21.15.100:0/374705138 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f066819c220 con 0x7f0668108400 2024-05-10T11:25:28.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.344+0000 7f066dbe2700 1 --2- 172.21.15.100:0/374705138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0668108400 0x7f066819bac0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f066000ef30 tx=0x7f066000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:25:28.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.344+0000 7f065e7fc700 1 -- 172.21.15.100:0/374705138 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06600092e0 con 0x7f0668108400 2024-05-10T11:25:28.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.344+0000 7f065e7fc700 1 -- 172.21.15.100:0/374705138 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0660004bb0 con 0x7f0668108400 2024-05-10T11:25:28.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.344+0000 7f066f645700 1 -- 172.21.15.100:0/374705138 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f066819c4b0 con 0x7f0668108400 2024-05-10T11:25:28.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.344+0000 7f065e7fc700 1 -- 172.21.15.100:0/374705138 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0660004020 con 0x7f0668108400 2024-05-10T11:25:28.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.344+0000 7f066f645700 1 -- 172.21.15.100:0/374705138 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06681a8780 con 0x7f0668108400 2024-05-10T11:25:28.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.347+0000 7f065e7fc700 1 -- 172.21.15.100:0/374705138 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f066001b030 con 0x7f0668108400 2024-05-10T11:25:28.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.347+0000 7f065e7fc700 1 --2- 172.21.15.100:0/374705138 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f06540b2860 0x7f06540b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:28.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.348+0000 7f066d3e1700 1 --2- 172.21.15.100:0/374705138 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f06540b2860 0x7f06540b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:28.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.348+0000 7f065e7fc700 1 -- 172.21.15.100:0/374705138 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f06600e1fd0 con 0x7f0668108400 2024-05-10T11:25:28.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.348+0000 7f0679ed8b80 1 Processor -- start 2024-05-10T11:25:28.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.348+0000 7f0679ed8b80 1 -- start start 2024-05-10T11:25:28.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.348+0000 7f066d3e1700 1 --2- 172.21.15.100:0/374705138 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f06540b2860 0x7f06540b4d20 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f0668107f60 tx=0x7f0664009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:25:28.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.348+0000 7f0679ed8b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb3ef9c4d0 0x55bb3ef9c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:28.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.348+0000 7f0679ed8b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb3ef9cdf0 0x55bb3efa06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:28.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.348+0000 7f0679ed8b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb3efa0c80 0x55bb3efa1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:28.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.348+0000 7f0679ed8b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55bb3eeee860 con 0x55bb3ef9c4d0 2024-05-10T11:25:28.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.348+0000 7f0679ed8b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55bb3ee8d7c0 con 0x55bb3ef9cdf0 2024-05-10T11:25:28.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.348+0000 7f0679ed8b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55bb3eeea820 con 0x55bb3efa0c80 2024-05-10T11:25:28.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.348+0000 7f066d3e1700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb3ef9c4d0 0x55bb3ef9c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:28.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.348+0000 7f066d3e1700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb3ef9c4d0 0x55bb3ef9c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51502/0 (socket says 172.21.15.100:51502) 2024-05-10T11:25:28.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.348+0000 7f066d3e1700 1 -- 172.21.15.100:0/1424443529 learned_addr learned my addr 172.21.15.100:0/1424443529 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:25:28.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.348+0000 7f066cbe0700 1 --2- 172.21.15.100:0/1424443529 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb3ef9cdf0 0x55bb3efa06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:28.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.348+0000 7f066dbe2700 1 --2- 172.21.15.100:0/1424443529 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb3efa0c80 0x55bb3efa1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:28.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.349+0000 7f066d3e1700 1 -- 172.21.15.100:0/1424443529 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb3efa0c80 msgr2=0x55bb3efa1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:28.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.349+0000 7f066d3e1700 1 --2- 172.21.15.100:0/1424443529 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb3efa0c80 0x55bb3efa1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:28.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.349+0000 7f066d3e1700 1 -- 172.21.15.100:0/1424443529 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb3ef9cdf0 msgr2=0x55bb3efa06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:28.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.349+0000 7f066d3e1700 1 --2- 172.21.15.100:0/1424443529 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb3ef9cdf0 0x55bb3efa06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:28.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.349+0000 7f066d3e1700 1 -- 172.21.15.100:0/1424443529 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb3ee3b8d0 con 0x55bb3ef9c4d0 2024-05-10T11:25:28.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.349+0000 7f066d3e1700 1 --2- 172.21.15.100:0/1424443529 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb3ef9c4d0 0x55bb3ef9c8b0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f0664011d60 tx=0x7f0664014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32f2c58375401a12 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:25:28.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.349+0000 7f06537fe700 1 -- 172.21.15.100:0/1424443529 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0664011d90 con 0x55bb3ef9c4d0 2024-05-10T11:25:28.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.349+0000 7f06537fe700 1 -- 172.21.15.100:0/1424443529 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0664004030 con 0x55bb3ef9c4d0 2024-05-10T11:25:28.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.349+0000 7f06537fe700 1 -- 172.21.15.100:0/1424443529 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0664003b80 con 0x55bb3ef9c4d0 2024-05-10T11:25:28.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.350+0000 7f0679ed8b80 1 -- 172.21.15.100:0/1424443529 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb3ef9c4d0 msgr2=0x55bb3ef9c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:28.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.350+0000 7f0679ed8b80 1 --2- 172.21.15.100:0/1424443529 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb3ef9c4d0 0x55bb3ef9c8b0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f0664011d60 tx=0x7f0664014000 comp rx=0 tx=0).stop 2024-05-10T11:25:28.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.350+0000 7f0679ed8b80 1 -- 172.21.15.100:0/1424443529 shutdown_connections 2024-05-10T11:25:28.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.350+0000 7f0679ed8b80 1 --2- 172.21.15.100:0/1424443529 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb3efa0c80 0x55bb3efa1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:28.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.350+0000 7f0679ed8b80 1 --2- 172.21.15.100:0/1424443529 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb3ef9c4d0 0x55bb3ef9c8b0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:28.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.350+0000 7f0679ed8b80 1 --2- 172.21.15.100:0/1424443529 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb3ef9cdf0 0x55bb3efa06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:28.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.350+0000 7f0679ed8b80 1 -- 172.21.15.100:0/1424443529 >> 172.21.15.100:0/1424443529 conn(0x55bb3eec8100 msgr2=0x55bb3ef96c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:25:28.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.350+0000 7f0679ed8b80 1 -- 172.21.15.100:0/1424443529 shutdown_connections 2024-05-10T11:25:28.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.350+0000 7f0679ed8b80 1 -- 172.21.15.100:0/1424443529 wait complete. 2024-05-10T11:25:28.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.350+0000 7f0679ed8b80 1 -- start start 2024-05-10T11:25:28.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.350+0000 7f0679ed8b80 1 Processor -- start 2024-05-10T11:25:28.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.351+0000 7f0679ed8b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb3ef9c4d0 0x55bb3efa8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:28.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.351+0000 7f0679ed8b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb3ef9cdf0 0x55bb3efa5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:28.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.351+0000 7f0679ed8b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb3efa0c80 0x55bb3efa5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:28.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.351+0000 7f0679ed8b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55bb3efa8e90 con 0x55bb3ef9cdf0 2024-05-10T11:25:28.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.351+0000 7f0679ed8b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55bb3ee54c30 con 0x55bb3efa0c80 2024-05-10T11:25:28.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.351+0000 7f0679ed8b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55bb3efa9010 con 0x55bb3ef9c4d0 2024-05-10T11:25:28.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.351+0000 7f0679ed8b80 10 client.0 _ll_register_callbacks cb 0x55bb3ee86920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:25:28.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.351+0000 7f066cbe0700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb3ef9cdf0 0x55bb3efa5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:28.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.351+0000 7f066d3e1700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb3ef9c4d0 0x55bb3efa8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:28.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.351+0000 7f066cbe0700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb3ef9cdf0 0x55bb3efa5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51506/0 (socket says 172.21.15.100:51506) 2024-05-10T11:25:28.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.351+0000 7f066cbe0700 1 -- 172.21.15.100:0/215179257 learned_addr learned my addr 172.21.15.100:0/215179257 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:25:28.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.351+0000 7f066dbe2700 1 --2- 172.21.15.100:0/215179257 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb3efa0c80 0x55bb3efa5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:28.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.351+0000 7f066d3e1700 1 -- 172.21.15.100:0/215179257 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb3efa0c80 msgr2=0x55bb3efa5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:28.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.351+0000 7f066d3e1700 1 --2- 172.21.15.100:0/215179257 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb3efa0c80 0x55bb3efa5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:28.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.351+0000 7f066d3e1700 1 -- 172.21.15.100:0/215179257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb3ef9cdf0 msgr2=0x55bb3efa5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:28.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.351+0000 7f066d3e1700 1 --2- 172.21.15.100:0/215179257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb3ef9cdf0 0x55bb3efa5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:28.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.351+0000 7f066d3e1700 1 -- 172.21.15.100:0/215179257 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb3ee8a690 con 0x55bb3ef9c4d0 2024-05-10T11:25:28.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.351+0000 7f066cbe0700 1 --2- 172.21.15.100:0/215179257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb3ef9cdf0 0x55bb3efa5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:25:28.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.352+0000 7f066d3e1700 1 --2- 172.21.15.100:0/215179257 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb3ef9c4d0 0x55bb3efa8030 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0664011d60 tx=0x7f0664005420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=341ffa2dee449410 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:25:28.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.352+0000 7f0650ff9700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:25:28.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.352+0000 7f0650ff9700 1 -- 172.21.15.100:0/215179257 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0664034060 con 0x55bb3ef9c4d0 2024-05-10T11:25:28.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.352+0000 7f0650ff9700 1 -- 172.21.15.100:0/215179257 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f0664003820 con 0x55bb3ef9c4d0 2024-05-10T11:25:28.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.352+0000 7f0679ed8b80 1 -- 172.21.15.100:0/215179257 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0664004030 con 0x55bb3ef9c4d0 2024-05-10T11:25:28.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.352+0000 7f0650ff9700 1 -- 172.21.15.100:0/215179257 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f066403b050 con 0x55bb3ef9c4d0 2024-05-10T11:25:28.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.352+0000 7f0679ed8b80 10 client.26266 fetch_fsmap learned FSMap version 31 2024-05-10T11:25:28.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.352+0000 7f0679ed8b80 1 -- 172.21.15.100:0/215179257 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55bb3efb3650 con 0x55bb3ef9c4d0 2024-05-10T11:25:28.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.352+0000 7f0650ff9700 1 -- 172.21.15.100:0/215179257 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0664004030 con 0x55bb3ef9c4d0 2024-05-10T11:25:28.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.352+0000 7f0650ff9700 1 -- 172.21.15.100:0/215179257 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f066401e930 con 0x55bb3ef9c4d0 2024-05-10T11:25:28.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.352+0000 7f0679ed8b80 10 client.26266 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:25:28.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.352+0000 7f0679ed8b80 10 client.26266 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-10T11:25:28.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.353+0000 7f0679ed8b80 20 client.26266 populate_metadata read hostname 'smithi100' 2024-05-10T11:25:28.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.353+0000 7f0679ed8b80 1 --2- 172.21.15.100:0/215179257 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55bb3efb5b20 0x55bb3efb7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:28.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.353+0000 7f0679ed8b80 4 client.26266 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:25:28.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.353+0000 7f0679ed8b80 1 -- 172.21.15.100:0/215179257 --> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb3ef68650 con 0x55bb3efb5b20 2024-05-10T11:25:28.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.353+0000 7f066cbe0700 1 --2- 172.21.15.100:0/215179257 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55bb3efb5b20 0x55bb3efb7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:28.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.353+0000 7f0650ff9700 10 client.26266 ms_handle_connect on v2:172.21.15.178:6838/3760985404 2024-05-10T11:25:28.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.353+0000 7f066cbe0700 1 --2- 172.21.15.100:0/215179257 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55bb3efb5b20 0x55bb3efb7f10 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f9e06e729368570b server_cookie=f6f83054647d0564 in_seq=0 out_seq=0 2024-05-10T11:25:28.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.354+0000 7f0650ff9700 1 -- 172.21.15.100:0/215179257 <== mds.1 v2:172.21.15.178:6838/3760985404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bb3ef68650 con 0x55bb3efb5b20 2024-05-10T11:25:28.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.354+0000 7f0650ff9700 10 client.26266 handle_command_reply: tid=0 2024-05-10T11:25:28.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.354+0000 7f0679ed8b80 1 client.26266 shutdown 2024-05-10T11:25:28.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.354+0000 7f0679ed8b80 1 -- 172.21.15.100:0/215179257 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb3ef9c4d0 msgr2=0x55bb3efa8030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:28.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.354+0000 7f0679ed8b80 1 --2- 172.21.15.100:0/215179257 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb3ef9c4d0 0x55bb3efa8030 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0664011d60 tx=0x7f0664005420 comp rx=0 tx=0).stop 2024-05-10T11:25:28.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.355+0000 7f0679ed8b80 1 -- 172.21.15.100:0/215179257 shutdown_connections 2024-05-10T11:25:28.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.355+0000 7f0679ed8b80 1 --2- 172.21.15.100:0/215179257 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55bb3efb5b20 0x55bb3efb7f10 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:28.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.355+0000 7f0679ed8b80 1 --2- 172.21.15.100:0/215179257 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb3efa0c80 0x55bb3efa5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:28.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.355+0000 7f0679ed8b80 1 --2- 172.21.15.100:0/215179257 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb3ef9c4d0 0x55bb3efa8030 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:28.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.355+0000 7f0679ed8b80 1 --2- 172.21.15.100:0/215179257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb3ef9cdf0 0x55bb3efa5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:28.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.355+0000 7f0650ff9700 0 client.26266 ms_handle_reset on v2:172.21.15.178:6838/3760985404 2024-05-10T11:25:28.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.355+0000 7f0679ed8b80 1 -- 172.21.15.100:0/215179257 >> 172.21.15.100:0/215179257 conn(0x55bb3eec8100 msgr2=0x55bb3ef91720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:25:28.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.355+0000 7f0679ed8b80 1 -- 172.21.15.100:0/215179257 shutdown_connections 2024-05-10T11:25:28.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.355+0000 7f0679ed8b80 1 -- 172.21.15.100:0/215179257 wait complete. 2024-05-10T11:25:28.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.355+0000 7f0679ed8b80 20 client.26266 trim_cache size 0 max 16384 2024-05-10T11:25:28.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.372+0000 7f0679ed8b80 1 Processor -- start 2024-05-10T11:25:28.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.372+0000 7f0679ed8b80 1 -- start start 2024-05-10T11:25:28.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.372+0000 7f0679ed8b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb3ef9c4d0 0x55bb3efb1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:28.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.372+0000 7f0679ed8b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb3ef9cdf0 0x55bb3ef915a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:28.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.372+0000 7f0679ed8b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb3efa0c80 0x55bb3ef9e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:28.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.373+0000 7f0679ed8b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f06681b3ba0 con 0x55bb3ef9cdf0 2024-05-10T11:25:28.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.373+0000 7f0679ed8b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f06681b1430 con 0x55bb3ef9c4d0 2024-05-10T11:25:28.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.373+0000 7f0679ed8b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f06681af280 con 0x55bb3efa0c80 2024-05-10T11:25:28.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.373+0000 7f066cbe0700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb3ef9cdf0 0x55bb3ef915a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:28.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.373+0000 7f066d3e1700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb3ef9c4d0 0x55bb3efb1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:28.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.373+0000 7f066cbe0700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb3ef9cdf0 0x55bb3ef915a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51522/0 (socket says 172.21.15.100:51522) 2024-05-10T11:25:28.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.373+0000 7f066cbe0700 1 -- 172.21.15.100:0/3232796207 learned_addr learned my addr 172.21.15.100:0/3232796207 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:25:28.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.373+0000 7f066dbe2700 1 --2- 172.21.15.100:0/3232796207 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb3efa0c80 0x55bb3ef9e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:28.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.373+0000 7f066cbe0700 1 -- 172.21.15.100:0/3232796207 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb3efa0c80 msgr2=0x55bb3ef9e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:28.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.373+0000 7f066cbe0700 1 --2- 172.21.15.100:0/3232796207 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb3efa0c80 0x55bb3ef9e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:28.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.373+0000 7f066cbe0700 1 -- 172.21.15.100:0/3232796207 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb3ef9c4d0 msgr2=0x55bb3efb1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:28.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.373+0000 7f066cbe0700 1 --2- 172.21.15.100:0/3232796207 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb3ef9c4d0 0x55bb3efb1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:28.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.373+0000 7f066cbe0700 1 -- 172.21.15.100:0/3232796207 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb3ef986d0 con 0x55bb3ef9cdf0 2024-05-10T11:25:28.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.373+0000 7f066cbe0700 1 --2- 172.21.15.100:0/3232796207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb3ef9cdf0 0x55bb3ef915a0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f0658000c00 tx=0x7f0658005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82eef78421be6a01 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:25:28.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.373+0000 7f0652ffd700 1 -- 172.21.15.100:0/3232796207 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0658003880 con 0x55bb3ef9cdf0 2024-05-10T11:25:28.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.373+0000 7f0652ffd700 1 -- 172.21.15.100:0/3232796207 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f065802e070 con 0x55bb3ef9cdf0 2024-05-10T11:25:28.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.373+0000 7f0652ffd700 1 -- 172.21.15.100:0/3232796207 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065800be40 con 0x55bb3ef9cdf0 2024-05-10T11:25:28.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.374+0000 7f0679ed8b80 1 -- 172.21.15.100:0/3232796207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb3ef9cdf0 msgr2=0x55bb3ef915a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:28.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.374+0000 7f0679ed8b80 1 --2- 172.21.15.100:0/3232796207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb3ef9cdf0 0x55bb3ef915a0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f0658000c00 tx=0x7f0658005fb0 comp rx=0 tx=0).stop 2024-05-10T11:25:28.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.374+0000 7f0679ed8b80 1 -- 172.21.15.100:0/3232796207 shutdown_connections 2024-05-10T11:25:28.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.374+0000 7f0679ed8b80 1 --2- 172.21.15.100:0/3232796207 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb3efa0c80 0x55bb3ef9e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:28.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.374+0000 7f0679ed8b80 1 --2- 172.21.15.100:0/3232796207 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb3ef9c4d0 0x55bb3efb1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:28.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.374+0000 7f0679ed8b80 1 --2- 172.21.15.100:0/3232796207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb3ef9cdf0 0x55bb3ef915a0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:28.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.374+0000 7f0679ed8b80 1 -- 172.21.15.100:0/3232796207 >> 172.21.15.100:0/3232796207 conn(0x55bb3eec8100 msgr2=0x55bb3ef90e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:25:28.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.374+0000 7f0679ed8b80 1 -- 172.21.15.100:0/3232796207 shutdown_connections 2024-05-10T11:25:28.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.374+0000 7f0679ed8b80 1 -- 172.21.15.100:0/3232796207 wait complete. 2024-05-10T11:25:28.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.374+0000 7f0679ed8b80 1 -- start start 2024-05-10T11:25:28.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.374+0000 7f0679ed8b80 1 Processor -- start 2024-05-10T11:25:28.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.375+0000 7f0679ed8b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb3ef9c4d0 0x55bb3efa94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:28.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.375+0000 7f0679ed8b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb3ef9cdf0 0x55bb3efa99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:28.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.375+0000 7f0679ed8b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb3efa0c80 0x55bb3f01e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:28.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.375+0000 7f0679ed8b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f06681af8f0 con 0x55bb3efa0c80 2024-05-10T11:25:28.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.375+0000 7f0679ed8b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f06681b0d60 con 0x55bb3ef9cdf0 2024-05-10T11:25:28.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.375+0000 7f0679ed8b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f06681affc0 con 0x55bb3ef9c4d0 2024-05-10T11:25:28.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.375+0000 7f0679ed8b80 10 client.0 _ll_register_callbacks cb 0x55bb3ee86920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:25:28.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.375+0000 7f066dbe2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb3efa0c80 0x55bb3f01e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:28.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.375+0000 7f066cbe0700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb3ef9cdf0 0x55bb3efa99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:28.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.375+0000 7f066d3e1700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb3ef9c4d0 0x55bb3efa94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:28.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.375+0000 7f066cbe0700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb3ef9cdf0 0x55bb3efa99e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:45516/0 (socket says 172.21.15.100:45516) 2024-05-10T11:25:28.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.375+0000 7f066dbe2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb3efa0c80 0x55bb3f01e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51532/0 (socket says 172.21.15.100:51532) 2024-05-10T11:25:28.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.375+0000 7f066cbe0700 1 -- 172.21.15.100:0/574054376 learned_addr learned my addr 172.21.15.100:0/574054376 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:25:28.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.375+0000 7f066dbe2700 1 -- 172.21.15.100:0/574054376 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb3ef9c4d0 msgr2=0x55bb3efa94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:28.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.375+0000 7f066dbe2700 1 --2- 172.21.15.100:0/574054376 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb3ef9c4d0 0x55bb3efa94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:28.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.375+0000 7f066dbe2700 1 -- 172.21.15.100:0/574054376 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb3ef9cdf0 msgr2=0x55bb3efa99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:28.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.375+0000 7f066dbe2700 1 --2- 172.21.15.100:0/574054376 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb3ef9cdf0 0x55bb3efa99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:28.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.375+0000 7f066dbe2700 1 -- 172.21.15.100:0/574054376 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb3f01e710 con 0x55bb3efa0c80 2024-05-10T11:25:28.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.375+0000 7f066cbe0700 1 --2- 172.21.15.100:0/574054376 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb3ef9cdf0 0x55bb3efa99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:25:28.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.375+0000 7f066d3e1700 1 --2- 172.21.15.100:0/574054376 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb3ef9c4d0 0x55bb3efa94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:25:28.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.376+0000 7f066dbe2700 1 --2- 172.21.15.100:0/574054376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb3efa0c80 0x55bb3f01e1d0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f066000ef00 tx=0x7f0660004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e588b2c74d15753 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:25:28.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.376+0000 7f0650ff9700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:25:28.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.376+0000 7f0650ff9700 1 -- 172.21.15.100:0/574054376 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06600c2070 con 0x55bb3efa0c80 2024-05-10T11:25:28.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.376+0000 7f0650ff9700 1 -- 172.21.15.100:0/574054376 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f06600a8790 con 0x55bb3efa0c80 2024-05-10T11:25:28.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.376+0000 7f0679ed8b80 1 -- 172.21.15.100:0/574054376 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f065802e070 con 0x55bb3efa0c80 2024-05-10T11:25:28.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.376+0000 7f0650ff9700 1 -- 172.21.15.100:0/574054376 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f06600a9320 con 0x55bb3efa0c80 2024-05-10T11:25:28.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.376+0000 7f0679ed8b80 10 client.16797 fetch_fsmap learned FSMap version 31 2024-05-10T11:25:28.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.376+0000 7f0679ed8b80 1 -- 172.21.15.100:0/574054376 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55bb3f01f9d0 con 0x55bb3efa0c80 2024-05-10T11:25:28.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.376+0000 7f0650ff9700 1 -- 172.21.15.100:0/574054376 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f065802e070 con 0x55bb3efa0c80 2024-05-10T11:25:28.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.376+0000 7f0650ff9700 1 -- 172.21.15.100:0/574054376 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f06600a4c90 con 0x55bb3efa0c80 2024-05-10T11:25:28.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.376+0000 7f0679ed8b80 10 client.16797 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:25:28.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.376+0000 7f0679ed8b80 10 client.16797 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-10T11:25:28.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.376+0000 7f0679ed8b80 20 client.16797 populate_metadata read hostname 'smithi100' 2024-05-10T11:25:28.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.376+0000 7f0679ed8b80 1 --2- 172.21.15.100:0/574054376 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55bb3f0200b0 0x55bb3f0224d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:28.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.376+0000 7f0679ed8b80 4 client.16797 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:25:28.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.376+0000 7f0679ed8b80 1 -- 172.21.15.100:0/574054376 --> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bb3eefc240 con 0x55bb3f0200b0 2024-05-10T11:25:28.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.377+0000 7f066d3e1700 1 --2- 172.21.15.100:0/574054376 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55bb3f0200b0 0x55bb3f0224d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:28.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.377+0000 7f066d3e1700 1 --2- 172.21.15.100:0/574054376 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55bb3f0200b0 0x55bb3f0224d0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6b529f55cd6c858b server_cookie=a3cffcdf4887eaae in_seq=0 out_seq=0 2024-05-10T11:25:28.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.377+0000 7f0650ff9700 10 client.16797 ms_handle_connect on v2:172.21.15.178:6838/3760985404 2024-05-10T11:25:28.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.377+0000 7f0650ff9700 1 -- 172.21.15.100:0/574054376 <== mds.1 v2:172.21.15.178:6838/3760985404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bb3eefc240 con 0x55bb3f0200b0 2024-05-10T11:25:28.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.377+0000 7f0650ff9700 10 client.16797 handle_command_reply: tid=0 2024-05-10T11:25:28.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.377+0000 7f0679ed8b80 1 client.16797 shutdown 2024-05-10T11:25:28.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.377+0000 7f0679ed8b80 1 -- 172.21.15.100:0/574054376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb3efa0c80 msgr2=0x55bb3f01e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:28.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.377+0000 7f0679ed8b80 1 --2- 172.21.15.100:0/574054376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb3efa0c80 0x55bb3f01e1d0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f066000ef00 tx=0x7f0660004f90 comp rx=0 tx=0).stop 2024-05-10T11:25:28.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.377+0000 7f0679ed8b80 1 -- 172.21.15.100:0/574054376 shutdown_connections 2024-05-10T11:25:28.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.377+0000 7f0679ed8b80 1 --2- 172.21.15.100:0/574054376 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55bb3f0200b0 0x55bb3f0224d0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:28.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.377+0000 7f0679ed8b80 1 --2- 172.21.15.100:0/574054376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb3efa0c80 0x55bb3f01e1d0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:28.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.377+0000 7f0679ed8b80 1 --2- 172.21.15.100:0/574054376 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb3ef9c4d0 0x55bb3efa94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:28.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.377+0000 7f0679ed8b80 1 --2- 172.21.15.100:0/574054376 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb3ef9cdf0 0x55bb3efa99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:28.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.377+0000 7f0679ed8b80 1 -- 172.21.15.100:0/574054376 >> 172.21.15.100:0/574054376 conn(0x55bb3eec8100 msgr2=0x55bb3ef91720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:25:28.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.377+0000 7f0650ff9700 0 client.16797 ms_handle_reset on v2:172.21.15.178:6838/3760985404 2024-05-10T11:25:28.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.378+0000 7f0679ed8b80 1 -- 172.21.15.100:0/574054376 shutdown_connections 2024-05-10T11:25:28.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.378+0000 7f0679ed8b80 1 -- 172.21.15.100:0/574054376 wait complete. 2024-05-10T11:25:28.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.378+0000 7f0679ed8b80 20 client.16797 trim_cache size 0 max 16384 2024-05-10T11:25:28.382 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:25:28.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.378+0000 7f066f645700 1 -- 172.21.15.100:0/374705138 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f06540b2860 msgr2=0x7f06540b4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:28.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.378+0000 7f066f645700 1 --2- 172.21.15.100:0/374705138 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f06540b2860 0x7f06540b4d20 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f0668107f60 tx=0x7f0664009260 comp rx=0 tx=0).stop 2024-05-10T11:25:28.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.378+0000 7f066f645700 1 -- 172.21.15.100:0/374705138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0668108400 msgr2=0x7f066819bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:28.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.378+0000 7f066f645700 1 --2- 172.21.15.100:0/374705138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0668108400 0x7f066819bac0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f066000ef30 tx=0x7f066000c3a0 comp rx=0 tx=0).stop 2024-05-10T11:25:28.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.378+0000 7f066f645700 1 -- 172.21.15.100:0/374705138 shutdown_connections 2024-05-10T11:25:28.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.378+0000 7f066f645700 1 --2- 172.21.15.100:0/374705138 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f06540b2860 0x7f06540b4d20 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:28.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.378+0000 7f066f645700 1 --2- 172.21.15.100:0/374705138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0668108400 0x7f066819bac0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:28.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.378+0000 7f066f645700 1 --2- 172.21.15.100:0/374705138 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0668102390 0x7f06681a18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:28.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.378+0000 7f066f645700 1 --2- 172.21.15.100:0/374705138 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0668102d50 0x7f06681a1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:28.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.378+0000 7f066f645700 1 -- 172.21.15.100:0/374705138 >> 172.21.15.100:0/374705138 conn(0x7f0668079310 msgr2=0x7f0668101040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:25:28.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.379+0000 7f066f645700 1 -- 172.21.15.100:0/374705138 shutdown_connections 2024-05-10T11:25:28.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:28.379+0000 7f066f645700 1 -- 172.21.15.100:0/374705138 wait complete. 2024-05-10T11:25:28.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:25:28 smithi178 ceph-mon[33359]: pgmap v632: 129 pgs: 129 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 1.3 MiB/s wr, 134 op/s 2024-05-10T11:25:28.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:25:28 smithi188 ceph-mon[34075]: pgmap v632: 129 pgs: 129 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 1.3 MiB/s wr, 134 op/s 2024-05-10T11:25:28.974 INFO:tasks.workunit.client.0.smithi100.stdout: 1 167307 2.74 MB/sec execute 203 sec latency 29.097 ms 2024-05-10T11:25:29.064 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-10T11:25:29.332 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:25:29.974 INFO:tasks.workunit.client.0.smithi100.stdout: 1 167711 2.73 MB/sec execute 204 sec latency 108.898 ms 2024-05-10T11:25:30.396 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:25:30 smithi100 ceph-mon[30296]: pgmap v633: 129 pgs: 129 active+clean; 73 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 1.3 MiB/s wr, 150 op/s 2024-05-10T11:25:30.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:25:30 smithi178 ceph-mon[33359]: pgmap v633: 129 pgs: 129 active+clean; 73 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 1.3 MiB/s wr, 150 op/s 2024-05-10T11:25:30.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:25:30 smithi188 ceph-mon[34075]: pgmap v633: 129 pgs: 129 active+clean; 73 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 1.3 MiB/s wr, 150 op/s 2024-05-10T11:25:30.974 INFO:tasks.workunit.client.0.smithi100.stdout: 1 168293 2.73 MB/sec execute 205 sec latency 15.586 ms 2024-05-10T11:25:31.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.281+0000 7f577fa3d700 1 -- 172.21.15.100:0/2466007827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5778108410 msgr2=0x7f57781087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:31.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.281+0000 7f577fa3d700 1 --2- 172.21.15.100:0/2466007827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5778108410 0x7f57781087f0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f5774009990 tx=0x7f5774009ca0 comp rx=0 tx=0).stop 2024-05-10T11:25:31.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.281+0000 7f577fa3d700 1 -- 172.21.15.100:0/2466007827 shutdown_connections 2024-05-10T11:25:31.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.281+0000 7f577fa3d700 1 --2- 172.21.15.100:0/2466007827 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5778102d60 0x7f577810f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:31.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.281+0000 7f577fa3d700 1 --2- 172.21.15.100:0/2466007827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5778108410 0x7f57781087f0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:31.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.281+0000 7f577fa3d700 1 --2- 172.21.15.100:0/2466007827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57781023a0 0x7f5778102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:31.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.281+0000 7f577fa3d700 1 -- 172.21.15.100:0/2466007827 >> 172.21.15.100:0/2466007827 conn(0x7f5778079350 msgr2=0x7f5778079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:25:31.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.281+0000 7f577fa3d700 1 -- 172.21.15.100:0/2466007827 shutdown_connections 2024-05-10T11:25:31.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.281+0000 7f577fa3d700 1 -- 172.21.15.100:0/2466007827 wait complete. 2024-05-10T11:25:31.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.282+0000 7f577fa3d700 1 Processor -- start 2024-05-10T11:25:31.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.282+0000 7f577fa3d700 1 -- start start 2024-05-10T11:25:31.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.282+0000 7f577fa3d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57781023a0 0x7f577819e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:31.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.282+0000 7f577fa3d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5778102d60 0x7f577819eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:31.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.282+0000 7f577fa3d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5778108410 0x7f5778198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:31.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.282+0000 7f577fa3d700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5778112410 con 0x7f5778108410 2024-05-10T11:25:31.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.282+0000 7f577fa3d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5778112290 con 0x7f5778102d60 2024-05-10T11:25:31.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.282+0000 7f577fa3d700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5778112590 con 0x7f57781023a0 2024-05-10T11:25:31.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.282+0000 7f577dfda700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5778108410 0x7f5778198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:31.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.282+0000 7f577d7d9700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57781023a0 0x7f577819e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:31.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.282+0000 7f577dfda700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5778108410 0x7f5778198790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51556/0 (socket says 172.21.15.100:51556) 2024-05-10T11:25:31.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.282+0000 7f577dfda700 1 -- 172.21.15.100:0/3411374183 learned_addr learned my addr 172.21.15.100:0/3411374183 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:25:31.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.282+0000 7f577cfd8700 1 --2- 172.21.15.100:0/3411374183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5778102d60 0x7f577819eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:31.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.283+0000 7f577dfda700 1 -- 172.21.15.100:0/3411374183 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57781023a0 msgr2=0x7f577819e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:31.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.283+0000 7f577dfda700 1 --2- 172.21.15.100:0/3411374183 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57781023a0 0x7f577819e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:31.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.283+0000 7f577dfda700 1 -- 172.21.15.100:0/3411374183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5778102d60 msgr2=0x7f577819eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:31.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.283+0000 7f577dfda700 1 --2- 172.21.15.100:0/3411374183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5778102d60 0x7f577819eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:31.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.283+0000 7f577dfda700 1 -- 172.21.15.100:0/3411374183 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5778198f70 con 0x7f5778108410 2024-05-10T11:25:31.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.283+0000 7f577d7d9700 1 --2- 172.21.15.100:0/3411374183 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57781023a0 0x7f577819e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:25:31.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.283+0000 7f577cfd8700 1 --2- 172.21.15.100:0/3411374183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5778102d60 0x7f577819eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:25:31.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.283+0000 7f577dfda700 1 --2- 172.21.15.100:0/3411374183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5778108410 0x7f5778198790 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f577000e580 tx=0x7f577000e890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:25:31.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.283+0000 7f576e7fc700 1 -- 172.21.15.100:0/3411374183 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5770004db0 con 0x7f5778108410 2024-05-10T11:25:31.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.283+0000 7f577fa3d700 1 -- 172.21.15.100:0/3411374183 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5778199260 con 0x7f5778108410 2024-05-10T11:25:31.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.283+0000 7f577fa3d700 1 -- 172.21.15.100:0/3411374183 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f577806b180 con 0x7f5778108410 2024-05-10T11:25:31.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.283+0000 7f576e7fc700 1 -- 172.21.15.100:0/3411374183 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5770003a00 con 0x7f5778108410 2024-05-10T11:25:31.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.283+0000 7f576e7fc700 1 -- 172.21.15.100:0/3411374183 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577001f060 con 0x7f5778108410 2024-05-10T11:25:31.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.286+0000 7f576e7fc700 1 -- 172.21.15.100:0/3411374183 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f57700044a0 con 0x7f5778108410 2024-05-10T11:25:31.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.286+0000 7f576e7fc700 1 --2- 172.21.15.100:0/3411374183 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f57640b2880 0x7f57640b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:31.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.287+0000 7f576e7fc700 1 -- 172.21.15.100:0/3411374183 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f57700e3f20 con 0x7f5778108410 2024-05-10T11:25:31.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.287+0000 7f577d7d9700 1 --2- 172.21.15.100:0/3411374183 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f57640b2880 0x7f57640b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:31.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.287+0000 7f578a2d0b80 1 Processor -- start 2024-05-10T11:25:31.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.287+0000 7f578a2d0b80 1 -- start start 2024-05-10T11:25:31.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.287+0000 7f577d7d9700 1 --2- 172.21.15.100:0/3411374183 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f57640b2880 0x7f57640b4d40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f577400b2e0 tx=0x7f577401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:25:31.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.288+0000 7f578a2d0b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563dd4d114d0 0x563dd4d118b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:31.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.288+0000 7f578a2d0b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563dd4d11df0 0x563dd4d156f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:31.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.288+0000 7f578a2d0b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563dd4d15c80 0x563dd4d16100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:31.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.288+0000 7f578a2d0b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x563dd4c63860 con 0x563dd4d11df0 2024-05-10T11:25:31.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.288+0000 7f578a2d0b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563dd4c027c0 con 0x563dd4d15c80 2024-05-10T11:25:31.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.288+0000 7f578a2d0b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563dd4c5f820 con 0x563dd4d114d0 2024-05-10T11:25:31.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.288+0000 7f577cfd8700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563dd4d11df0 0x563dd4d156f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:31.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.288+0000 7f577d7d9700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563dd4d114d0 0x563dd4d118b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:31.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.288+0000 7f577cfd8700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563dd4d11df0 0x563dd4d156f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51560/0 (socket says 172.21.15.100:51560) 2024-05-10T11:25:31.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.288+0000 7f577cfd8700 1 -- 172.21.15.100:0/2710688246 learned_addr learned my addr 172.21.15.100:0/2710688246 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:25:31.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.288+0000 7f577dfda700 1 --2- 172.21.15.100:0/2710688246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563dd4d15c80 0x563dd4d16100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:31.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.289+0000 7f577cfd8700 1 -- 172.21.15.100:0/2710688246 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563dd4d114d0 msgr2=0x563dd4d118b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:31.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.289+0000 7f577cfd8700 1 --2- 172.21.15.100:0/2710688246 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563dd4d114d0 0x563dd4d118b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:31.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.289+0000 7f577cfd8700 1 -- 172.21.15.100:0/2710688246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563dd4d15c80 msgr2=0x563dd4d16100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:31.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.289+0000 7f577cfd8700 1 --2- 172.21.15.100:0/2710688246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563dd4d15c80 0x563dd4d16100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:31.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.289+0000 7f577cfd8700 1 -- 172.21.15.100:0/2710688246 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563dd4bb08d0 con 0x563dd4d11df0 2024-05-10T11:25:31.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.289+0000 7f577d7d9700 1 --2- 172.21.15.100:0/2710688246 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563dd4d114d0 0x563dd4d118b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:25:31.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.289+0000 7f577cfd8700 1 --2- 172.21.15.100:0/2710688246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563dd4d11df0 0x563dd4d156f0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f5768010580 tx=0x7f5768010890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f888f536ed15d7f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:25:31.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.289+0000 7f57637fe700 1 -- 172.21.15.100:0/2710688246 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576801c680 con 0x563dd4d11df0 2024-05-10T11:25:31.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.289+0000 7f57637fe700 1 -- 172.21.15.100:0/2710688246 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f576801ccb0 con 0x563dd4d11df0 2024-05-10T11:25:31.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.289+0000 7f57637fe700 1 -- 172.21.15.100:0/2710688246 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5768011be0 con 0x563dd4d11df0 2024-05-10T11:25:31.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.290+0000 7f578a2d0b80 1 -- 172.21.15.100:0/2710688246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563dd4d11df0 msgr2=0x563dd4d156f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:31.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.290+0000 7f578a2d0b80 1 --2- 172.21.15.100:0/2710688246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563dd4d11df0 0x563dd4d156f0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f5768010580 tx=0x7f5768010890 comp rx=0 tx=0).stop 2024-05-10T11:25:31.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.290+0000 7f578a2d0b80 1 -- 172.21.15.100:0/2710688246 shutdown_connections 2024-05-10T11:25:31.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.290+0000 7f578a2d0b80 1 --2- 172.21.15.100:0/2710688246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563dd4d15c80 0x563dd4d16100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:31.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.290+0000 7f578a2d0b80 1 --2- 172.21.15.100:0/2710688246 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563dd4d114d0 0x563dd4d118b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:31.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.290+0000 7f578a2d0b80 1 --2- 172.21.15.100:0/2710688246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563dd4d11df0 0x563dd4d156f0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:31.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.290+0000 7f578a2d0b80 1 -- 172.21.15.100:0/2710688246 >> 172.21.15.100:0/2710688246 conn(0x563dd4c3d100 msgr2=0x563dd4d0bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:25:31.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.290+0000 7f578a2d0b80 1 -- 172.21.15.100:0/2710688246 shutdown_connections 2024-05-10T11:25:31.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.290+0000 7f578a2d0b80 1 -- 172.21.15.100:0/2710688246 wait complete. 2024-05-10T11:25:31.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.291+0000 7f578a2d0b80 1 -- start start 2024-05-10T11:25:31.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.291+0000 7f578a2d0b80 1 Processor -- start 2024-05-10T11:25:31.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.291+0000 7f578a2d0b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563dd4d114d0 0x563dd4d1cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:31.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.291+0000 7f578a2d0b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563dd4d11df0 0x563dd4d1a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:31.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.291+0000 7f578a2d0b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563dd4d15c80 0x563dd4d1a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:31.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.291+0000 7f578a2d0b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x563dd4d1de90 con 0x563dd4d11df0 2024-05-10T11:25:31.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.291+0000 7f578a2d0b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563dd4bc9c30 con 0x563dd4d15c80 2024-05-10T11:25:31.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.291+0000 7f578a2d0b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563dd4d1e010 con 0x563dd4d114d0 2024-05-10T11:25:31.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.291+0000 7f578a2d0b80 10 client.0 _ll_register_callbacks cb 0x563dd4bfb920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:25:31.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.291+0000 7f577cfd8700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563dd4d11df0 0x563dd4d1a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:31.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.291+0000 7f577cfd8700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563dd4d11df0 0x563dd4d1a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51566/0 (socket says 172.21.15.100:51566) 2024-05-10T11:25:31.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.291+0000 7f577cfd8700 1 -- 172.21.15.100:0/1821747410 learned_addr learned my addr 172.21.15.100:0/1821747410 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:25:31.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.292+0000 7f577d7d9700 1 --2- 172.21.15.100:0/1821747410 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563dd4d114d0 0x563dd4d1cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:31.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.292+0000 7f577dfda700 1 --2- 172.21.15.100:0/1821747410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563dd4d15c80 0x563dd4d1a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:31.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.292+0000 7f577d7d9700 1 -- 172.21.15.100:0/1821747410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563dd4d15c80 msgr2=0x563dd4d1a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:31.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.292+0000 7f577d7d9700 1 --2- 172.21.15.100:0/1821747410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563dd4d15c80 0x563dd4d1a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:31.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.292+0000 7f577d7d9700 1 -- 172.21.15.100:0/1821747410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563dd4d11df0 msgr2=0x563dd4d1a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:31.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.292+0000 7f577d7d9700 1 --2- 172.21.15.100:0/1821747410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563dd4d11df0 0x563dd4d1a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:31.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.292+0000 7f577d7d9700 1 -- 172.21.15.100:0/1821747410 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563dd4bff690 con 0x563dd4d114d0 2024-05-10T11:25:31.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.292+0000 7f577dfda700 1 --2- 172.21.15.100:0/1821747410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563dd4d15c80 0x563dd4d1a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:25:31.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.292+0000 7f577d7d9700 1 --2- 172.21.15.100:0/1821747410 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563dd4d114d0 0x563dd4d1cf70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f577400b650 tx=0x7f5774009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=282caa5b7816bd75 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:25:31.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.292+0000 7f5760ff9700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:25:31.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.292+0000 7f5760ff9700 1 -- 172.21.15.100:0/1821747410 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577400f460 con 0x563dd4d114d0 2024-05-10T11:25:31.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.292+0000 7f5760ff9700 1 -- 172.21.15.100:0/1821747410 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f577400f600 con 0x563dd4d114d0 2024-05-10T11:25:31.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.292+0000 7f578a2d0b80 1 -- 172.21.15.100:0/1821747410 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f576801ccb0 con 0x563dd4d114d0 2024-05-10T11:25:31.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.293+0000 7f5760ff9700 1 -- 172.21.15.100:0/1821747410 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f577400f800 con 0x563dd4d114d0 2024-05-10T11:25:31.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.293+0000 7f578a2d0b80 10 client.26284 fetch_fsmap learned FSMap version 31 2024-05-10T11:25:31.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.293+0000 7f578a2d0b80 1 -- 172.21.15.100:0/1821747410 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563dd4d285c0 con 0x563dd4d114d0 2024-05-10T11:25:31.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.293+0000 7f5760ff9700 1 -- 172.21.15.100:0/1821747410 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f576801ccb0 con 0x563dd4d114d0 2024-05-10T11:25:31.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.293+0000 7f5760ff9700 1 -- 172.21.15.100:0/1821747410 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f577402c880 con 0x563dd4d114d0 2024-05-10T11:25:31.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.293+0000 7f578a2d0b80 10 client.26284 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:25:31.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.293+0000 7f578a2d0b80 10 client.26284 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:25:31.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.293+0000 7f578a2d0b80 20 client.26284 populate_metadata read hostname 'smithi100' 2024-05-10T11:25:31.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.293+0000 7f578a2d0b80 1 --2- 172.21.15.100:0/1821747410 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x563dd4d2aca0 0x563dd4d2d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:31.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.293+0000 7f578a2d0b80 4 client.26284 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:25:31.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.293+0000 7f578a2d0b80 1 -- 172.21.15.100:0/1821747410 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563dd4cdd650 con 0x563dd4d2aca0 2024-05-10T11:25:31.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.293+0000 7f577cfd8700 1 --2- 172.21.15.100:0/1821747410 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x563dd4d2aca0 0x563dd4d2d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:31.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.294+0000 7f577cfd8700 1 --2- 172.21.15.100:0/1821747410 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x563dd4d2aca0 0x563dd4d2d090 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fab611dda3edbff server_cookie=20e2b96d066d058a in_seq=0 out_seq=0 2024-05-10T11:25:31.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.294+0000 7f5760ff9700 10 client.26284 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:25:31.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.294+0000 7f5760ff9700 1 -- 172.21.15.100:0/1821747410 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563dd4cdd650 con 0x563dd4d2aca0 2024-05-10T11:25:31.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.294+0000 7f5760ff9700 10 client.26284 handle_command_reply: tid=0 2024-05-10T11:25:31.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.294+0000 7f578a2d0b80 1 client.26284 shutdown 2024-05-10T11:25:31.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.295+0000 7f578a2d0b80 1 -- 172.21.15.100:0/1821747410 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563dd4d114d0 msgr2=0x563dd4d1cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:31.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.295+0000 7f578a2d0b80 1 --2- 172.21.15.100:0/1821747410 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563dd4d114d0 0x563dd4d1cf70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f577400b650 tx=0x7f5774009310 comp rx=0 tx=0).stop 2024-05-10T11:25:31.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.295+0000 7f578a2d0b80 1 -- 172.21.15.100:0/1821747410 shutdown_connections 2024-05-10T11:25:31.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.295+0000 7f578a2d0b80 1 --2- 172.21.15.100:0/1821747410 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x563dd4d2aca0 0x563dd4d2d090 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:31.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.295+0000 7f578a2d0b80 1 --2- 172.21.15.100:0/1821747410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563dd4d15c80 0x563dd4d1a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:31.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.295+0000 7f578a2d0b80 1 --2- 172.21.15.100:0/1821747410 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563dd4d114d0 0x563dd4d1cf70 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:31.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.295+0000 7f578a2d0b80 1 --2- 172.21.15.100:0/1821747410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563dd4d11df0 0x563dd4d1a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:31.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.295+0000 7f578a2d0b80 1 -- 172.21.15.100:0/1821747410 >> 172.21.15.100:0/1821747410 conn(0x563dd4c3d100 msgr2=0x563dd4d06720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:25:31.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.295+0000 7f5760ff9700 0 client.26284 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:25:31.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.295+0000 7f578a2d0b80 1 -- 172.21.15.100:0/1821747410 shutdown_connections 2024-05-10T11:25:31.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.295+0000 7f578a2d0b80 1 -- 172.21.15.100:0/1821747410 wait complete. 2024-05-10T11:25:31.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.295+0000 7f578a2d0b80 20 client.26284 trim_cache size 0 max 16384 2024-05-10T11:25:31.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.314+0000 7f578a2d0b80 1 Processor -- start 2024-05-10T11:25:31.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.314+0000 7f578a2d0b80 1 -- start start 2024-05-10T11:25:31.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.314+0000 7f578a2d0b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563dd4d114d0 0x563dd4d065a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:31.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.314+0000 7f578a2d0b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563dd4d11df0 0x563dd4d135f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:31.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.314+0000 7f578a2d0b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563dd4d15c80 0x563dd4d13b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:31.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.314+0000 7f578a2d0b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f575c011290 con 0x563dd4d114d0 2024-05-10T11:25:31.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.314+0000 7f578a2d0b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f575c00eb20 con 0x563dd4d15c80 2024-05-10T11:25:31.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.314+0000 7f578a2d0b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f575c00c970 con 0x563dd4d11df0 2024-05-10T11:25:31.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.314+0000 7f577d7d9700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563dd4d114d0 0x563dd4d065a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:31.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.314+0000 7f577cfd8700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563dd4d11df0 0x563dd4d135f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:31.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.314+0000 7f577d7d9700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563dd4d114d0 0x563dd4d065a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51574/0 (socket says 172.21.15.100:51574) 2024-05-10T11:25:31.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.314+0000 7f577d7d9700 1 -- 172.21.15.100:0/289833157 learned_addr learned my addr 172.21.15.100:0/289833157 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:25:31.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.315+0000 7f577dfda700 1 --2- 172.21.15.100:0/289833157 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563dd4d15c80 0x563dd4d13b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:31.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.315+0000 7f577cfd8700 1 -- 172.21.15.100:0/289833157 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563dd4d15c80 msgr2=0x563dd4d13b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:31.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.315+0000 7f577cfd8700 1 --2- 172.21.15.100:0/289833157 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563dd4d15c80 0x563dd4d13b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:31.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.315+0000 7f577cfd8700 1 -- 172.21.15.100:0/289833157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563dd4d114d0 msgr2=0x563dd4d065a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:31.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.315+0000 7f577cfd8700 1 --2- 172.21.15.100:0/289833157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563dd4d114d0 0x563dd4d065a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:31.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.315+0000 7f577cfd8700 1 -- 172.21.15.100:0/289833157 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563dd4d1d2e0 con 0x563dd4d11df0 2024-05-10T11:25:31.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.315+0000 7f577d7d9700 1 --2- 172.21.15.100:0/289833157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563dd4d114d0 0x563dd4d065a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:25:31.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.315+0000 7f577cfd8700 1 --2- 172.21.15.100:0/289833157 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563dd4d11df0 0x563dd4d135f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x563dd4d1b8e0 tx=0x7f576800bcb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c259735773e433fd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:25:31.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.315+0000 7f5762ffd700 1 -- 172.21.15.100:0/289833157 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576802c020 con 0x563dd4d11df0 2024-05-10T11:25:31.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.315+0000 7f5762ffd700 1 -- 172.21.15.100:0/289833157 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5768028430 con 0x563dd4d11df0 2024-05-10T11:25:31.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.315+0000 7f5762ffd700 1 -- 172.21.15.100:0/289833157 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57680052a0 con 0x563dd4d11df0 2024-05-10T11:25:31.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.316+0000 7f578a2d0b80 1 -- 172.21.15.100:0/289833157 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563dd4d11df0 msgr2=0x563dd4d135f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:31.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.316+0000 7f578a2d0b80 1 --2- 172.21.15.100:0/289833157 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563dd4d11df0 0x563dd4d135f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x563dd4d1b8e0 tx=0x7f576800bcb0 comp rx=0 tx=0).stop 2024-05-10T11:25:31.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.316+0000 7f578a2d0b80 1 -- 172.21.15.100:0/289833157 shutdown_connections 2024-05-10T11:25:31.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.316+0000 7f578a2d0b80 1 --2- 172.21.15.100:0/289833157 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563dd4d15c80 0x563dd4d13b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:31.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.316+0000 7f578a2d0b80 1 --2- 172.21.15.100:0/289833157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563dd4d114d0 0x563dd4d065a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:31.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.316+0000 7f578a2d0b80 1 --2- 172.21.15.100:0/289833157 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563dd4d11df0 0x563dd4d135f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:31.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.316+0000 7f578a2d0b80 1 -- 172.21.15.100:0/289833157 >> 172.21.15.100:0/289833157 conn(0x563dd4c3d100 msgr2=0x563dd4d05e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:25:31.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.316+0000 7f578a2d0b80 1 -- 172.21.15.100:0/289833157 shutdown_connections 2024-05-10T11:25:31.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.316+0000 7f578a2d0b80 1 -- 172.21.15.100:0/289833157 wait complete. 2024-05-10T11:25:31.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.316+0000 7f578a2d0b80 1 -- start start 2024-05-10T11:25:31.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.316+0000 7f578a2d0b80 1 Processor -- start 2024-05-10T11:25:31.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.317+0000 7f578a2d0b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563dd4d114d0 0x563dd4d1e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:31.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.317+0000 7f578a2d0b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563dd4d11df0 0x563dd4d1e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:31.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.317+0000 7f578a2d0b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563dd4d15c80 0x563dd4d93540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:31.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.317+0000 7f578a2d0b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f575c00cfe0 con 0x563dd4d114d0 2024-05-10T11:25:31.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.317+0000 7f578a2d0b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f575c00e450 con 0x563dd4d11df0 2024-05-10T11:25:31.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.317+0000 7f578a2d0b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f575c00d6b0 con 0x563dd4d15c80 2024-05-10T11:25:31.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.317+0000 7f578a2d0b80 10 client.0 _ll_register_callbacks cb 0x563dd4bfb920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:25:31.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.317+0000 7f577d7d9700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563dd4d114d0 0x563dd4d1e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:31.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.317+0000 7f577d7d9700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563dd4d114d0 0x563dd4d1e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51578/0 (socket says 172.21.15.100:51578) 2024-05-10T11:25:31.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.317+0000 7f577d7d9700 1 -- 172.21.15.100:0/857520587 learned_addr learned my addr 172.21.15.100:0/857520587 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:25:31.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.317+0000 7f577cfd8700 1 --2- 172.21.15.100:0/857520587 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563dd4d11df0 0x563dd4d1e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:31.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.317+0000 7f577dfda700 1 --2- 172.21.15.100:0/857520587 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563dd4d15c80 0x563dd4d93540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:31.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.317+0000 7f577d7d9700 1 -- 172.21.15.100:0/857520587 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563dd4d15c80 msgr2=0x563dd4d93540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:31.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.317+0000 7f577d7d9700 1 --2- 172.21.15.100:0/857520587 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563dd4d15c80 0x563dd4d93540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:31.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.317+0000 7f577d7d9700 1 -- 172.21.15.100:0/857520587 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563dd4d11df0 msgr2=0x563dd4d1e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:31.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.317+0000 7f577d7d9700 1 --2- 172.21.15.100:0/857520587 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563dd4d11df0 0x563dd4d1e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:31.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.317+0000 7f577d7d9700 1 -- 172.21.15.100:0/857520587 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563dd4d1f070 con 0x563dd4d114d0 2024-05-10T11:25:31.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.317+0000 7f577cfd8700 1 --2- 172.21.15.100:0/857520587 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563dd4d11df0 0x563dd4d1e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:25:31.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.318+0000 7f577d7d9700 1 --2- 172.21.15.100:0/857520587 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563dd4d114d0 0x563dd4d1e290 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f577402cee0 tx=0x7f577400fbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b0f67cbb491c261 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:25:31.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.318+0000 7f5760ff9700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:25:31.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.318+0000 7f5760ff9700 1 -- 172.21.15.100:0/857520587 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5774005be0 con 0x563dd4d114d0 2024-05-10T11:25:31.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.318+0000 7f578a2d0b80 1 -- 172.21.15.100:0/857520587 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5768028430 con 0x563dd4d114d0 2024-05-10T11:25:31.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.318+0000 7f5760ff9700 1 -- 172.21.15.100:0/857520587 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f577402c450 con 0x563dd4d114d0 2024-05-10T11:25:31.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.318+0000 7f5760ff9700 1 -- 172.21.15.100:0/857520587 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f577402c650 con 0x563dd4d114d0 2024-05-10T11:25:31.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.318+0000 7f578a2d0b80 10 client.16815 fetch_fsmap learned FSMap version 31 2024-05-10T11:25:31.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.318+0000 7f578a2d0b80 1 -- 172.21.15.100:0/857520587 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563dd4d94b50 con 0x563dd4d114d0 2024-05-10T11:25:31.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.318+0000 7f5760ff9700 1 -- 172.21.15.100:0/857520587 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5768028430 con 0x563dd4d114d0 2024-05-10T11:25:31.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.318+0000 7f5760ff9700 1 -- 172.21.15.100:0/857520587 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f5774042070 con 0x563dd4d114d0 2024-05-10T11:25:31.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.318+0000 7f578a2d0b80 10 client.16815 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:25:31.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.318+0000 7f578a2d0b80 10 client.16815 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:25:31.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.318+0000 7f578a2d0b80 20 client.16815 populate_metadata read hostname 'smithi100' 2024-05-10T11:25:31.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.318+0000 7f578a2d0b80 1 --2- 172.21.15.100:0/857520587 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x563dd4d951c0 0x563dd4d975e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:31.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.318+0000 7f578a2d0b80 4 client.16815 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:25:31.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.318+0000 7f578a2d0b80 1 -- 172.21.15.100:0/857520587 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563dd4c71240 con 0x563dd4d951c0 2024-05-10T11:25:31.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.319+0000 7f577cfd8700 1 --2- 172.21.15.100:0/857520587 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x563dd4d951c0 0x563dd4d975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:31.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.319+0000 7f5760ff9700 10 client.16815 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:25:31.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.319+0000 7f577cfd8700 1 --2- 172.21.15.100:0/857520587 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x563dd4d951c0 0x563dd4d975e0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70d4717a537c38e4 server_cookie=8d43044aa093b97a in_seq=0 out_seq=0 2024-05-10T11:25:31.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.320+0000 7f5760ff9700 1 -- 172.21.15.100:0/857520587 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563dd4c71240 con 0x563dd4d951c0 2024-05-10T11:25:31.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.320+0000 7f5760ff9700 10 client.16815 handle_command_reply: tid=0 2024-05-10T11:25:31.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.320+0000 7f578a2d0b80 1 client.16815 shutdown 2024-05-10T11:25:31.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.320+0000 7f578a2d0b80 1 -- 172.21.15.100:0/857520587 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563dd4d114d0 msgr2=0x563dd4d1e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:31.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.320+0000 7f578a2d0b80 1 --2- 172.21.15.100:0/857520587 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563dd4d114d0 0x563dd4d1e290 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f577402cee0 tx=0x7f577400fbb0 comp rx=0 tx=0).stop 2024-05-10T11:25:31.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.320+0000 7f578a2d0b80 1 -- 172.21.15.100:0/857520587 shutdown_connections 2024-05-10T11:25:31.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.320+0000 7f578a2d0b80 1 --2- 172.21.15.100:0/857520587 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x563dd4d951c0 0x563dd4d975e0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:31.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.320+0000 7f578a2d0b80 1 --2- 172.21.15.100:0/857520587 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563dd4d15c80 0x563dd4d93540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:31.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.320+0000 7f5760ff9700 0 client.16815 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:25:31.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.320+0000 7f578a2d0b80 1 --2- 172.21.15.100:0/857520587 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563dd4d114d0 0x563dd4d1e290 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:31.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.320+0000 7f578a2d0b80 1 --2- 172.21.15.100:0/857520587 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563dd4d11df0 0x563dd4d1e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:31.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.320+0000 7f578a2d0b80 1 -- 172.21.15.100:0/857520587 >> 172.21.15.100:0/857520587 conn(0x563dd4c3d100 msgr2=0x563dd4d06720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:25:31.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.320+0000 7f578a2d0b80 1 -- 172.21.15.100:0/857520587 shutdown_connections 2024-05-10T11:25:31.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.320+0000 7f578a2d0b80 1 -- 172.21.15.100:0/857520587 wait complete. 2024-05-10T11:25:31.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.320+0000 7f578a2d0b80 20 client.16815 trim_cache size 0 max 16384 2024-05-10T11:25:31.324 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:25:31.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.321+0000 7f577fa3d700 1 -- 172.21.15.100:0/3411374183 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f57640b2880 msgr2=0x7f57640b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:31.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.321+0000 7f577fa3d700 1 --2- 172.21.15.100:0/3411374183 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f57640b2880 0x7f57640b4d40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f577400b2e0 tx=0x7f577401a040 comp rx=0 tx=0).stop 2024-05-10T11:25:31.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.321+0000 7f577fa3d700 1 -- 172.21.15.100:0/3411374183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5778108410 msgr2=0x7f5778198790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:31.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.321+0000 7f577fa3d700 1 --2- 172.21.15.100:0/3411374183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5778108410 0x7f5778198790 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f577000e580 tx=0x7f577000e890 comp rx=0 tx=0).stop 2024-05-10T11:25:31.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.321+0000 7f577fa3d700 1 -- 172.21.15.100:0/3411374183 shutdown_connections 2024-05-10T11:25:31.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.321+0000 7f577fa3d700 1 --2- 172.21.15.100:0/3411374183 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f57640b2880 0x7f57640b4d40 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:31.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.321+0000 7f577fa3d700 1 --2- 172.21.15.100:0/3411374183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5778108410 0x7f5778198790 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:31.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.321+0000 7f577fa3d700 1 --2- 172.21.15.100:0/3411374183 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57781023a0 0x7f577819e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:31.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.321+0000 7f577fa3d700 1 --2- 172.21.15.100:0/3411374183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5778102d60 0x7f577819eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:31.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.321+0000 7f577fa3d700 1 -- 172.21.15.100:0/3411374183 >> 172.21.15.100:0/3411374183 conn(0x7f5778079350 msgr2=0x7f5778101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:25:31.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.321+0000 7f577fa3d700 1 -- 172.21.15.100:0/3411374183 shutdown_connections 2024-05-10T11:25:31.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:31.321+0000 7f577fa3d700 1 -- 172.21.15.100:0/3411374183 wait complete. 2024-05-10T11:25:31.974 INFO:tasks.workunit.client.0.smithi100.stdout: 1 168939 2.72 MB/sec execute 206 sec latency 36.679 ms 2024-05-10T11:25:32.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:25:32 smithi100 ceph-mon[30296]: pgmap v634: 129 pgs: 129 active+clean; 74 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 1.2 MiB/s wr, 133 op/s 2024-05-10T11:25:32.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:25:32 smithi178 ceph-mon[33359]: pgmap v634: 129 pgs: 129 active+clean; 74 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 1.2 MiB/s wr, 133 op/s 2024-05-10T11:25:32.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:25:32 smithi188 ceph-mon[34075]: pgmap v634: 129 pgs: 129 active+clean; 74 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 1.2 MiB/s wr, 133 op/s 2024-05-10T11:25:32.970 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T11:25:32.971 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T11:25:32.974 INFO:tasks.workunit.client.0.smithi100.stdout: 1 169525 2.74 MB/sec execute 207 sec latency 13.361 ms 2024-05-10T11:25:33.236 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:25:33.974 INFO:tasks.workunit.client.0.smithi100.stdout: 1 170079 2.74 MB/sec execute 208 sec latency 14.506 ms 2024-05-10T11:25:34.394 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:25:34 smithi100 ceph-mon[30296]: pgmap v635: 129 pgs: 129 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 1.4 MiB/s wr, 159 op/s 2024-05-10T11:25:34.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:25:34 smithi178 ceph-mon[33359]: pgmap v635: 129 pgs: 129 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 1.4 MiB/s wr, 159 op/s 2024-05-10T11:25:34.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:25:34 smithi188 ceph-mon[34075]: pgmap v635: 129 pgs: 129 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 1.4 MiB/s wr, 159 op/s 2024-05-10T11:25:34.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.958+0000 7f881a8c4700 1 -- 172.21.15.100:0/3121329065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8814102d60 msgr2=0x7f881410f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:34.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.958+0000 7f881a8c4700 1 --2- 172.21.15.100:0/3121329065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8814102d60 0x7f881410f250 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f880c00b0a0 tx=0x7f880c00b3b0 comp rx=0 tx=0).stop 2024-05-10T11:25:34.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.958+0000 7f881a8c4700 1 -- 172.21.15.100:0/3121329065 shutdown_connections 2024-05-10T11:25:34.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.958+0000 7f881a8c4700 1 --2- 172.21.15.100:0/3121329065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8814102d60 0x7f881410f250 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:34.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.958+0000 7f881a8c4700 1 --2- 172.21.15.100:0/3121329065 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8814108410 0x7f88141087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:34.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.958+0000 7f881a8c4700 1 --2- 172.21.15.100:0/3121329065 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88141023a0 0x7f8814102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:34.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.958+0000 7f881a8c4700 1 -- 172.21.15.100:0/3121329065 >> 172.21.15.100:0/3121329065 conn(0x7f8814079350 msgr2=0x7f8814079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:25:34.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.958+0000 7f881a8c4700 1 -- 172.21.15.100:0/3121329065 shutdown_connections 2024-05-10T11:25:34.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.958+0000 7f881a8c4700 1 -- 172.21.15.100:0/3121329065 wait complete. 2024-05-10T11:25:34.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.958+0000 7f881a8c4700 1 Processor -- start 2024-05-10T11:25:34.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.958+0000 7f881a8c4700 1 -- start start 2024-05-10T11:25:34.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.959+0000 7f881a8c4700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88141023a0 0x7f8814104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:34.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.959+0000 7f881a8c4700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8814102d60 0x7f88141045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:34.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.959+0000 7f881a8c4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8814108410 0x7f8814104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:34.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.959+0000 7f881a8c4700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8814112460 con 0x7f8814108410 2024-05-10T11:25:34.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.959+0000 7f881a8c4700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f88141122e0 con 0x7f8814102d60 2024-05-10T11:25:34.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.959+0000 7f881a8c4700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f88141125e0 con 0x7f88141023a0 2024-05-10T11:25:34.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.959+0000 7f8818e61700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8814108410 0x7f8814104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:34.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.959+0000 7f88137fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8814102d60 0x7f88141045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:34.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.959+0000 7f88137fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8814102d60 0x7f88141045c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:45574/0 (socket says 172.21.15.100:45574) 2024-05-10T11:25:34.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.959+0000 7f8813fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88141023a0 0x7f8814104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:34.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.959+0000 7f8818e61700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8814108410 0x7f8814104b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51604/0 (socket says 172.21.15.100:51604) 2024-05-10T11:25:34.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.959+0000 7f88137fe700 1 -- 172.21.15.100:0/3750283105 learned_addr learned my addr 172.21.15.100:0/3750283105 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:25:34.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.959+0000 7f88137fe700 1 -- 172.21.15.100:0/3750283105 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88141023a0 msgr2=0x7f8814104080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:34.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.959+0000 7f88137fe700 1 --2- 172.21.15.100:0/3750283105 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88141023a0 0x7f8814104080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:34.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.959+0000 7f88137fe700 1 -- 172.21.15.100:0/3750283105 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8814108410 msgr2=0x7f8814104b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:34.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.959+0000 7f88137fe700 1 --2- 172.21.15.100:0/3750283105 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8814108410 0x7f8814104b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:34.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.959+0000 7f88137fe700 1 -- 172.21.15.100:0/3750283105 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88141a7f70 con 0x7f8814102d60 2024-05-10T11:25:34.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.959+0000 7f8818e61700 1 --2- 172.21.15.100:0/3750283105 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8814108410 0x7f8814104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:25:34.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.959+0000 7f88137fe700 1 --2- 172.21.15.100:0/3750283105 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8814102d60 0x7f88141045c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f880800e9c0 tx=0x7f880800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:25:34.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.960+0000 7f88117fa700 1 -- 172.21.15.100:0/3750283105 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880800cbd0 con 0x7f8814102d60 2024-05-10T11:25:34.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.960+0000 7f881a8c4700 1 -- 172.21.15.100:0/3750283105 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88141a8200 con 0x7f8814102d60 2024-05-10T11:25:34.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.960+0000 7f88117fa700 1 -- 172.21.15.100:0/3750283105 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f880800ee60 con 0x7f8814102d60 2024-05-10T11:25:34.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.960+0000 7f88117fa700 1 -- 172.21.15.100:0/3750283105 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88080054b0 con 0x7f8814102d60 2024-05-10T11:25:34.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.960+0000 7f881a8c4700 1 -- 172.21.15.100:0/3750283105 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88141a8740 con 0x7f8814102d60 2024-05-10T11:25:34.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.962+0000 7f88117fa700 1 -- 172.21.15.100:0/3750283105 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f8808004750 con 0x7f8814102d60 2024-05-10T11:25:34.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.963+0000 7f88117fa700 1 --2- 172.21.15.100:0/3750283105 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f87fc0b2900 0x7f87fc0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:34.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.963+0000 7f8825157b80 1 Processor -- start 2024-05-10T11:25:34.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.963+0000 7f8813fff700 1 --2- 172.21.15.100:0/3750283105 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f87fc0b2900 0x7f87fc0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:34.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.964+0000 7f8825157b80 1 -- start start 2024-05-10T11:25:34.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.964+0000 7f88117fa700 1 -- 172.21.15.100:0/3750283105 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f88080e27f0 con 0x7f8814102d60 2024-05-10T11:25:34.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.964+0000 7f8825157b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564fc4e60280 0x564fc4e60660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:34.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.964+0000 7f8825157b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564fc4e60ba0 0x564fc4e645e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:34.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.964+0000 7f8825157b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564fc4e64b20 0x564fc4e64f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:34.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.964+0000 7f8825157b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x564fc4da99c0 con 0x564fc4e64b20 2024-05-10T11:25:34.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.964+0000 7f8825157b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x564fc4d0ee20 con 0x564fc4e60ba0 2024-05-10T11:25:34.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.964+0000 7f8825157b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x564fc4da5980 con 0x564fc4e60280 2024-05-10T11:25:34.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.964+0000 7f88137fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564fc4e60ba0 0x564fc4e645e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:34.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.964+0000 7f8813fff700 1 --2- 172.21.15.100:0/3750283105 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f87fc0b2900 0x7f87fc0b4dc0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8804009be0 tx=0x7f8804009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:25:34.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.964+0000 7f8813fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564fc4e60280 0x564fc4e60660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:34.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.964+0000 7f88137fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564fc4e60ba0 0x564fc4e645e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:45576/0 (socket says 172.21.15.100:45576) 2024-05-10T11:25:34.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.964+0000 7f88137fe700 1 -- 172.21.15.100:0/2706120857 learned_addr learned my addr 172.21.15.100:0/2706120857 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:25:34.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.964+0000 7f8813fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564fc4e60280 0x564fc4e60660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:36986/0 (socket says 172.21.15.100:36986) 2024-05-10T11:25:34.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.964+0000 7f8818e61700 1 --2- 172.21.15.100:0/2706120857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564fc4e64b20 0x564fc4e64f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:34.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.964+0000 7f88137fe700 1 -- 172.21.15.100:0/2706120857 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564fc4e60280 msgr2=0x564fc4e60660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:34.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.964+0000 7f88137fe700 1 --2- 172.21.15.100:0/2706120857 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564fc4e60280 0x564fc4e60660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:34.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.964+0000 7f88137fe700 1 -- 172.21.15.100:0/2706120857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564fc4e64b20 msgr2=0x564fc4e64f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:34.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.964+0000 7f88137fe700 1 --2- 172.21.15.100:0/2706120857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564fc4e64b20 0x564fc4e64f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:34.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.964+0000 7f88137fe700 1 -- 172.21.15.100:0/2706120857 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fc4cf6950 con 0x564fc4e60ba0 2024-05-10T11:25:34.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.964+0000 7f8813fff700 1 --2- 172.21.15.100:0/2706120857 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564fc4e60280 0x564fc4e60660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:25:34.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.965+0000 7f88137fe700 1 --2- 172.21.15.100:0/2706120857 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564fc4e60ba0 0x564fc4e645e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f88080a5b30 tx=0x7f88080a9320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=844695e55b1dd66b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:25:34.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.965+0000 7f87fa7fc700 1 -- 172.21.15.100:0/2706120857 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88080c3070 con 0x564fc4e60ba0 2024-05-10T11:25:34.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.965+0000 7f87fa7fc700 1 -- 172.21.15.100:0/2706120857 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f88080b2260 con 0x564fc4e60ba0 2024-05-10T11:25:34.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.965+0000 7f87fa7fc700 1 -- 172.21.15.100:0/2706120857 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88080e00a0 con 0x564fc4e60ba0 2024-05-10T11:25:34.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.965+0000 7f8825157b80 1 -- 172.21.15.100:0/2706120857 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564fc4e60ba0 msgr2=0x564fc4e645e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:34.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.965+0000 7f8825157b80 1 --2- 172.21.15.100:0/2706120857 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564fc4e60ba0 0x564fc4e645e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f88080a5b30 tx=0x7f88080a9320 comp rx=0 tx=0).stop 2024-05-10T11:25:34.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.966+0000 7f8825157b80 1 -- 172.21.15.100:0/2706120857 shutdown_connections 2024-05-10T11:25:34.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.966+0000 7f8825157b80 1 --2- 172.21.15.100:0/2706120857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564fc4e64b20 0x564fc4e64f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:34.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.966+0000 7f8825157b80 1 --2- 172.21.15.100:0/2706120857 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564fc4e60280 0x564fc4e60660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:34.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.966+0000 7f8825157b80 1 --2- 172.21.15.100:0/2706120857 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564fc4e60ba0 0x564fc4e645e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:34.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.966+0000 7f8825157b80 1 -- 172.21.15.100:0/2706120857 >> 172.21.15.100:0/2706120857 conn(0x564fc4d83bf0 msgr2=0x564fc4e52320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:25:34.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.966+0000 7f8825157b80 1 -- 172.21.15.100:0/2706120857 shutdown_connections 2024-05-10T11:25:34.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.966+0000 7f8825157b80 1 -- 172.21.15.100:0/2706120857 wait complete. 2024-05-10T11:25:34.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.966+0000 7f8825157b80 1 -- start start 2024-05-10T11:25:34.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.966+0000 7f8825157b80 1 Processor -- start 2024-05-10T11:25:34.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.967+0000 7f8825157b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564fc4e60280 0x564fc4e6c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:34.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.967+0000 7f8825157b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564fc4e60ba0 0x564fc4e693a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:34.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.967+0000 7f8825157b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564fc4e64b20 0x564fc4e698e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:34.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.967+0000 7f8825157b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x564fc4e6ccc0 con 0x564fc4e60280 2024-05-10T11:25:34.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.967+0000 7f8825157b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x564fc4d62f50 con 0x564fc4e60ba0 2024-05-10T11:25:34.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.967+0000 7f8825157b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x564fc4e6ce40 con 0x564fc4e64b20 2024-05-10T11:25:34.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.967+0000 7f8825157b80 10 client.0 _ll_register_callbacks cb 0x564fc4d3d630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:25:34.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.967+0000 7f8813fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564fc4e60280 0x564fc4e6c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:34.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.967+0000 7f8813fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564fc4e60280 0x564fc4e6c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51628/0 (socket says 172.21.15.100:51628) 2024-05-10T11:25:34.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.967+0000 7f8813fff700 1 -- 172.21.15.100:0/63915885 learned_addr learned my addr 172.21.15.100:0/63915885 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:25:34.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.967+0000 7f8818e61700 1 --2- 172.21.15.100:0/63915885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564fc4e64b20 0x564fc4e698e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:34.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.967+0000 7f88137fe700 1 --2- 172.21.15.100:0/63915885 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564fc4e60ba0 0x564fc4e693a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:34.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.967+0000 7f8813fff700 1 -- 172.21.15.100:0/63915885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564fc4e64b20 msgr2=0x564fc4e698e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:34.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.967+0000 7f8813fff700 1 --2- 172.21.15.100:0/63915885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564fc4e64b20 0x564fc4e698e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:34.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.967+0000 7f8813fff700 1 -- 172.21.15.100:0/63915885 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564fc4e60ba0 msgr2=0x564fc4e693a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:34.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.967+0000 7f8813fff700 1 --2- 172.21.15.100:0/63915885 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564fc4e60ba0 0x564fc4e693a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:34.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.967+0000 7f8813fff700 1 -- 172.21.15.100:0/63915885 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fc4d607e0 con 0x564fc4e60280 2024-05-10T11:25:34.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.967+0000 7f8813fff700 1 --2- 172.21.15.100:0/63915885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564fc4e60280 0x564fc4e6c000 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f880401f880 tx=0x7f88040047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1076927bef3c1e62 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:25:34.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.967+0000 7f87f37fe700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:25:34.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.967+0000 7f87f37fe700 1 -- 172.21.15.100:0/63915885 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8804004030 con 0x564fc4e60280 2024-05-10T11:25:34.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.967+0000 7f87f37fe700 1 -- 172.21.15.100:0/63915885 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f8804009040 con 0x564fc4e60280 2024-05-10T11:25:34.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.967+0000 7f8825157b80 1 -- 172.21.15.100:0/63915885 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88080b2260 con 0x564fc4e60280 2024-05-10T11:25:34.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.968+0000 7f87f37fe700 1 -- 172.21.15.100:0/63915885 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8804030050 con 0x564fc4e60280 2024-05-10T11:25:34.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.968+0000 7f8825157b80 10 client.16827 fetch_fsmap learned FSMap version 31 2024-05-10T11:25:34.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.968+0000 7f8825157b80 1 -- 172.21.15.100:0/63915885 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564fc4e775c0 con 0x564fc4e60280 2024-05-10T11:25:34.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.968+0000 7f87f37fe700 1 -- 172.21.15.100:0/63915885 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8804004830 con 0x564fc4e60280 2024-05-10T11:25:34.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.968+0000 7f87f37fe700 1 -- 172.21.15.100:0/63915885 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f8804005230 con 0x564fc4e60280 2024-05-10T11:25:34.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.968+0000 7f8825157b80 10 client.16827 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:25:34.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.968+0000 7f8825157b80 10 client.16827 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:25:34.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.968+0000 7f8825157b80 20 client.16827 populate_metadata read hostname 'smithi100' 2024-05-10T11:25:34.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.968+0000 7f8825157b80 1 --2- 172.21.15.100:0/63915885 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x564fc4e79c00 0x564fc4e7bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:34.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.968+0000 7f8825157b80 4 client.16827 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:25:34.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.968+0000 7f8825157b80 1 -- 172.21.15.100:0/63915885 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f88080b2260 con 0x564fc4e79c00 2024-05-10T11:25:34.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.969+0000 7f88137fe700 1 --2- 172.21.15.100:0/63915885 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x564fc4e79c00 0x564fc4e7bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:34.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.969+0000 7f87f37fe700 10 client.16827 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:25:34.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.969+0000 7f88137fe700 1 --2- 172.21.15.100:0/63915885 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x564fc4e79c00 0x564fc4e7bff0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8136a0d0ce177dda server_cookie=3a4c6a11745e5ca8 in_seq=0 out_seq=0 2024-05-10T11:25:34.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.970+0000 7f87f37fe700 1 -- 172.21.15.100:0/63915885 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f88080b2260 con 0x564fc4e79c00 2024-05-10T11:25:34.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.970+0000 7f87f37fe700 10 client.16827 handle_command_reply: tid=0 2024-05-10T11:25:34.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.970+0000 7f8825157b80 1 client.16827 shutdown 2024-05-10T11:25:34.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.970+0000 7f8825157b80 1 -- 172.21.15.100:0/63915885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564fc4e60280 msgr2=0x564fc4e6c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:34.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.970+0000 7f8825157b80 1 --2- 172.21.15.100:0/63915885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564fc4e60280 0x564fc4e6c000 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f880401f880 tx=0x7f88040047c0 comp rx=0 tx=0).stop 2024-05-10T11:25:34.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.970+0000 7f8825157b80 1 -- 172.21.15.100:0/63915885 shutdown_connections 2024-05-10T11:25:34.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.970+0000 7f8825157b80 1 --2- 172.21.15.100:0/63915885 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x564fc4e79c00 0x564fc4e7bff0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:34.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.970+0000 7f8825157b80 1 --2- 172.21.15.100:0/63915885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564fc4e64b20 0x564fc4e698e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:34.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.970+0000 7f8825157b80 1 --2- 172.21.15.100:0/63915885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564fc4e60280 0x564fc4e6c000 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:34.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.970+0000 7f8825157b80 1 --2- 172.21.15.100:0/63915885 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564fc4e60ba0 0x564fc4e693a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:34.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.970+0000 7f8825157b80 1 -- 172.21.15.100:0/63915885 >> 172.21.15.100:0/63915885 conn(0x564fc4d83bf0 msgr2=0x564fc4e65ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:25:34.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.970+0000 7f87f37fe700 0 client.16827 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:25:34.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.970+0000 7f8825157b80 1 -- 172.21.15.100:0/63915885 shutdown_connections 2024-05-10T11:25:34.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.970+0000 7f8825157b80 1 -- 172.21.15.100:0/63915885 wait complete. 2024-05-10T11:25:34.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.970+0000 7f8825157b80 20 client.16827 trim_cache size 0 max 16384 2024-05-10T11:25:34.974 INFO:tasks.workunit.client.0.smithi100.stdout: 1 170551 2.75 MB/sec execute 209 sec latency 15.904 ms 2024-05-10T11:25:34.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.992+0000 7f8825157b80 1 Processor -- start 2024-05-10T11:25:34.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.992+0000 7f8825157b80 1 -- start start 2024-05-10T11:25:34.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.992+0000 7f8825157b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564fc4e60ba0 0x564fc4e75d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:34.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.992+0000 7f8825157b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564fc4e64b20 0x564fc4e56f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:34.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.992+0000 7f8825157b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564fc4e818a0 0x564fc4e81ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:34.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.992+0000 7f8825157b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f88141af6e0 con 0x564fc4e818a0 2024-05-10T11:25:34.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.992+0000 7f8825157b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f88141acf70 con 0x564fc4e60ba0 2024-05-10T11:25:34.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.992+0000 7f8825157b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f88141aae30 con 0x564fc4e64b20 2024-05-10T11:25:34.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.992+0000 7f8818e61700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564fc4e818a0 0x564fc4e81ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:34.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.992+0000 7f8818e61700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564fc4e818a0 0x564fc4e81ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51630/0 (socket says 172.21.15.100:51630) 2024-05-10T11:25:34.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.992+0000 7f8818e61700 1 -- 172.21.15.100:0/1467372939 learned_addr learned my addr 172.21.15.100:0/1467372939 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:25:34.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.992+0000 7f88137fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564fc4e64b20 0x564fc4e56f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:34.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.992+0000 7f8813fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564fc4e60ba0 0x564fc4e75d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:34.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.992+0000 7f8818e61700 1 -- 172.21.15.100:0/1467372939 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564fc4e64b20 msgr2=0x564fc4e56f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:34.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.992+0000 7f8818e61700 1 --2- 172.21.15.100:0/1467372939 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564fc4e64b20 0x564fc4e56f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:34.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.992+0000 7f8818e61700 1 -- 172.21.15.100:0/1467372939 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564fc4e60ba0 msgr2=0x564fc4e75d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:34.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.992+0000 7f8818e61700 1 --2- 172.21.15.100:0/1467372939 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564fc4e60ba0 0x564fc4e75d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:34.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.992+0000 7f8818e61700 1 -- 172.21.15.100:0/1467372939 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fc4e69c90 con 0x564fc4e818a0 2024-05-10T11:25:34.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.992+0000 7f8813fff700 1 --2- 172.21.15.100:0/1467372939 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564fc4e60ba0 0x564fc4e75d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:25:34.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.993+0000 7f8818e61700 1 --2- 172.21.15.100:0/1467372939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564fc4e818a0 0x564fc4e81ce0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f880c00b070 tx=0x7f880c006210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2ac2c4f2074f67b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:25:34.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.993+0000 7f87f9ffb700 1 -- 172.21.15.100:0/1467372939 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880c02b070 con 0x564fc4e818a0 2024-05-10T11:25:34.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.993+0000 7f87f9ffb700 1 -- 172.21.15.100:0/1467372939 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f880c003c00 con 0x564fc4e818a0 2024-05-10T11:25:34.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.993+0000 7f87f9ffb700 1 -- 172.21.15.100:0/1467372939 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880c0052a0 con 0x564fc4e818a0 2024-05-10T11:25:34.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.994+0000 7f8825157b80 1 -- 172.21.15.100:0/1467372939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564fc4e818a0 msgr2=0x564fc4e81ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:34.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.994+0000 7f8825157b80 1 --2- 172.21.15.100:0/1467372939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564fc4e818a0 0x564fc4e81ce0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f880c00b070 tx=0x7f880c006210 comp rx=0 tx=0).stop 2024-05-10T11:25:34.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.994+0000 7f8825157b80 1 -- 172.21.15.100:0/1467372939 shutdown_connections 2024-05-10T11:25:34.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.994+0000 7f8825157b80 1 --2- 172.21.15.100:0/1467372939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564fc4e818a0 0x564fc4e81ce0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:34.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.994+0000 7f8825157b80 1 --2- 172.21.15.100:0/1467372939 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564fc4e60ba0 0x564fc4e75d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:34.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.994+0000 7f8825157b80 1 --2- 172.21.15.100:0/1467372939 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564fc4e64b20 0x564fc4e56f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:34.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.994+0000 7f8825157b80 1 -- 172.21.15.100:0/1467372939 >> 172.21.15.100:0/1467372939 conn(0x564fc4d83bf0 msgr2=0x564fc4e56880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:25:34.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.994+0000 7f8825157b80 1 -- 172.21.15.100:0/1467372939 shutdown_connections 2024-05-10T11:25:34.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.994+0000 7f8825157b80 1 -- 172.21.15.100:0/1467372939 wait complete. 2024-05-10T11:25:34.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.994+0000 7f8825157b80 1 -- start start 2024-05-10T11:25:34.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.994+0000 7f8825157b80 1 Processor -- start 2024-05-10T11:25:34.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.995+0000 7f8825157b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564fc4e60ba0 0x564fc4e70fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:34.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.995+0000 7f8825157b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564fc4e64b20 0x564fc4e71500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:34.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.995+0000 7f8825157b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564fc4e818a0 0x564fc4e71a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:34.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.995+0000 7f8825157b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f88141ab430 con 0x564fc4e818a0 2024-05-10T11:25:34.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.995+0000 7f8825157b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f88141ac8a0 con 0x564fc4e64b20 2024-05-10T11:25:34.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.995+0000 7f8825157b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f88141abb00 con 0x564fc4e60ba0 2024-05-10T11:25:34.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.995+0000 7f8825157b80 10 client.0 _ll_register_callbacks cb 0x564fc4d3d630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:25:34.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.995+0000 7f8818e61700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564fc4e818a0 0x564fc4e71a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:34.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.995+0000 7f8818e61700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564fc4e818a0 0x564fc4e71a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51636/0 (socket says 172.21.15.100:51636) 2024-05-10T11:25:34.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.995+0000 7f8818e61700 1 -- 172.21.15.100:0/2285203030 learned_addr learned my addr 172.21.15.100:0/2285203030 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:25:34.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.995+0000 7f88137fe700 1 --2- 172.21.15.100:0/2285203030 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564fc4e64b20 0x564fc4e71500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:34.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.995+0000 7f8813fff700 1 --2- 172.21.15.100:0/2285203030 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564fc4e60ba0 0x564fc4e70fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:34.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.995+0000 7f8818e61700 1 -- 172.21.15.100:0/2285203030 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564fc4e60ba0 msgr2=0x564fc4e70fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:34.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.995+0000 7f8818e61700 1 --2- 172.21.15.100:0/2285203030 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564fc4e60ba0 0x564fc4e70fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:34.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.995+0000 7f8818e61700 1 -- 172.21.15.100:0/2285203030 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564fc4e64b20 msgr2=0x564fc4e71500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:34.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.995+0000 7f8818e61700 1 --2- 172.21.15.100:0/2285203030 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564fc4e64b20 0x564fc4e71500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:34.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.995+0000 7f8818e61700 1 -- 172.21.15.100:0/2285203030 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fc4ee4770 con 0x564fc4e818a0 2024-05-10T11:25:34.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.995+0000 7f8813fff700 1 --2- 172.21.15.100:0/2285203030 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564fc4e60ba0 0x564fc4e70fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:25:34.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.995+0000 7f88137fe700 1 --2- 172.21.15.100:0/2285203030 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564fc4e64b20 0x564fc4e71500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:25:34.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.995+0000 7f87f37fe700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:25:34.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.995+0000 7f8818e61700 1 --2- 172.21.15.100:0/2285203030 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564fc4e818a0 0x564fc4e71a80 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f880c029040 tx=0x7f880c007b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ddf643512dbcf44 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:25:34.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.996+0000 7f87f37fe700 1 -- 172.21.15.100:0/2285203030 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880c036060 con 0x564fc4e818a0 2024-05-10T11:25:34.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.996+0000 7f8825157b80 1 -- 172.21.15.100:0/2285203030 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f880c003c00 con 0x564fc4e818a0 2024-05-10T11:25:34.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.996+0000 7f87f37fe700 1 -- 172.21.15.100:0/2285203030 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f880c007d00 con 0x564fc4e818a0 2024-05-10T11:25:34.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.996+0000 7f87f37fe700 1 -- 172.21.15.100:0/2285203030 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f880c03d050 con 0x564fc4e818a0 2024-05-10T11:25:34.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.996+0000 7f8825157b80 10 client.16833 fetch_fsmap learned FSMap version 31 2024-05-10T11:25:34.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.996+0000 7f8825157b80 1 -- 172.21.15.100:0/2285203030 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564fc4ee5ab0 con 0x564fc4e818a0 2024-05-10T11:25:34.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.996+0000 7f87f37fe700 1 -- 172.21.15.100:0/2285203030 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f880c003c00 con 0x564fc4e818a0 2024-05-10T11:25:34.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.996+0000 7f87f37fe700 1 -- 172.21.15.100:0/2285203030 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f880c025070 con 0x564fc4e818a0 2024-05-10T11:25:35.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.996+0000 7f8825157b80 10 client.16833 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:25:35.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.996+0000 7f8825157b80 10 client.16833 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:25:35.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.996+0000 7f8825157b80 20 client.16833 populate_metadata read hostname 'smithi100' 2024-05-10T11:25:35.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.996+0000 7f8825157b80 1 --2- 172.21.15.100:0/2285203030 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x564fc4ee6050 0x564fc4ee8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:35.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.996+0000 7f8825157b80 4 client.16833 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T11:25:35.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.996+0000 7f8825157b80 1 -- 172.21.15.100:0/2285203030 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564fc4db73a0 con 0x564fc4ee6050 2024-05-10T11:25:35.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.996+0000 7f8813fff700 1 --2- 172.21.15.100:0/2285203030 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x564fc4ee6050 0x564fc4ee8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:35.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.997+0000 7f87f37fe700 10 client.16833 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:25:35.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.997+0000 7f8813fff700 1 --2- 172.21.15.100:0/2285203030 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x564fc4ee6050 0x564fc4ee8470 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25e64800cb69bbd1 server_cookie=7150cf329b647893 in_seq=0 out_seq=0 2024-05-10T11:25:35.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.998+0000 7f87f37fe700 1 -- 172.21.15.100:0/2285203030 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564fc4db73a0 con 0x564fc4ee6050 2024-05-10T11:25:35.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.998+0000 7f87f37fe700 10 client.16833 handle_command_reply: tid=0 2024-05-10T11:25:35.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.998+0000 7f8825157b80 1 client.16833 shutdown 2024-05-10T11:25:35.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.998+0000 7f8825157b80 1 -- 172.21.15.100:0/2285203030 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564fc4e818a0 msgr2=0x564fc4e71a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:35.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.998+0000 7f8825157b80 1 --2- 172.21.15.100:0/2285203030 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564fc4e818a0 0x564fc4e71a80 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f880c029040 tx=0x7f880c007b80 comp rx=0 tx=0).stop 2024-05-10T11:25:35.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.998+0000 7f8825157b80 1 -- 172.21.15.100:0/2285203030 shutdown_connections 2024-05-10T11:25:35.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.998+0000 7f8825157b80 1 --2- 172.21.15.100:0/2285203030 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x564fc4ee6050 0x564fc4ee8470 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:35.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.998+0000 7f8825157b80 1 --2- 172.21.15.100:0/2285203030 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564fc4e818a0 0x564fc4e71a80 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:35.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.998+0000 7f87f37fe700 0 client.16833 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:25:35.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.998+0000 7f8825157b80 1 --2- 172.21.15.100:0/2285203030 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564fc4e60ba0 0x564fc4e70fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:35.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.998+0000 7f8825157b80 1 --2- 172.21.15.100:0/2285203030 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564fc4e64b20 0x564fc4e71500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:35.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.998+0000 7f8825157b80 1 -- 172.21.15.100:0/2285203030 >> 172.21.15.100:0/2285203030 conn(0x564fc4d83bf0 msgr2=0x564fc4e57110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:25:35.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.998+0000 7f8825157b80 1 -- 172.21.15.100:0/2285203030 shutdown_connections 2024-05-10T11:25:35.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.998+0000 7f8825157b80 1 -- 172.21.15.100:0/2285203030 wait complete. 2024-05-10T11:25:35.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.998+0000 7f8825157b80 20 client.16833 trim_cache size 0 max 16384 2024-05-10T11:25:35.003 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:25:35.003 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-05-10T11:25:35.003 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "6cfe3177-791e-478a-886f-bf8119258da8", 2024-05-10T11:25:35.003 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-05-10T11:25:35.003 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:25:35.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.999+0000 7f881a8c4700 1 -- 172.21.15.100:0/3750283105 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f87fc0b2900 msgr2=0x7f87fc0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:35.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.999+0000 7f881a8c4700 1 --2- 172.21.15.100:0/3750283105 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f87fc0b2900 0x7f87fc0b4dc0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8804009be0 tx=0x7f8804009260 comp rx=0 tx=0).stop 2024-05-10T11:25:35.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.999+0000 7f881a8c4700 1 -- 172.21.15.100:0/3750283105 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8814102d60 msgr2=0x7f88141045c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:35.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.999+0000 7f881a8c4700 1 --2- 172.21.15.100:0/3750283105 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8814102d60 0x7f88141045c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f880800e9c0 tx=0x7f880800ecd0 comp rx=0 tx=0).stop 2024-05-10T11:25:35.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.999+0000 7f881a8c4700 1 -- 172.21.15.100:0/3750283105 shutdown_connections 2024-05-10T11:25:35.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.999+0000 7f881a8c4700 1 --2- 172.21.15.100:0/3750283105 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f87fc0b2900 0x7f87fc0b4dc0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:35.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.999+0000 7f881a8c4700 1 --2- 172.21.15.100:0/3750283105 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8814108410 0x7f8814104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:35.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.999+0000 7f881a8c4700 1 --2- 172.21.15.100:0/3750283105 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88141023a0 0x7f8814104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:35.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.999+0000 7f881a8c4700 1 --2- 172.21.15.100:0/3750283105 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8814102d60 0x7f88141045c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:35.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.999+0000 7f881a8c4700 1 -- 172.21.15.100:0/3750283105 >> 172.21.15.100:0/3750283105 conn(0x7f8814079350 msgr2=0x7f8814101690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:25:35.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.999+0000 7f881a8c4700 1 -- 172.21.15.100:0/3750283105 shutdown_connections 2024-05-10T11:25:35.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:34.999+0000 7f881a8c4700 1 -- 172.21.15.100:0/3750283105 wait complete. 2024-05-10T11:25:35.974 INFO:tasks.workunit.client.0.smithi100.stdout: 1 170825 2.74 MB/sec execute 210 sec latency 15.374 ms 2024-05-10T11:25:36.419 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.418+0000 7fc66a215640 1 -- 172.21.15.100:0/2441907742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc664103500 msgr2=0x7fc664109d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:36.419 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.418+0000 7fc66a215640 1 --2- 172.21.15.100:0/2441907742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc664103500 0x7fc664109d90 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fc65c009ca0 tx=0x7fc65c02a8c0 comp rx=0 tx=0).stop 2024-05-10T11:25:36.419 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.418+0000 7fc66a215640 1 -- 172.21.15.100:0/2441907742 shutdown_connections 2024-05-10T11:25:36.419 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.418+0000 7fc66a215640 1 --2- 172.21.15.100:0/2441907742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc664103500 0x7fc664109d90 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:36.419 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.418+0000 7fc66a215640 1 --2- 172.21.15.100:0/2441907742 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc664102b40 0x7fc664102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:36.419 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.418+0000 7fc66a215640 1 --2- 172.21.15.100:0/2441907742 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc664101950 0x7fc664101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:36.419 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.418+0000 7fc66a215640 1 -- 172.21.15.100:0/2441907742 >> 172.21.15.100:0/2441907742 conn(0x7fc6640fcec0 msgr2=0x7fc6640ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:25:36.419 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.418+0000 7fc66a215640 1 -- 172.21.15.100:0/2441907742 shutdown_connections 2024-05-10T11:25:36.419 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.418+0000 7fc66a215640 1 -- 172.21.15.100:0/2441907742 wait complete. 2024-05-10T11:25:36.419 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.419+0000 7fc66a215640 1 Processor -- start 2024-05-10T11:25:36.419 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.419+0000 7fc66a215640 1 -- start start 2024-05-10T11:25:36.421 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.419+0000 7fc66a215640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc664101950 0x7fc66406bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:36.421 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.419+0000 7fc66a215640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc664102b40 0x7fc66419d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:36.421 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.419+0000 7fc66a215640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc664103500 0x7fc66419fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:36.421 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.419+0000 7fc66a215640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc66410d380 con 0x7fc664101950 2024-05-10T11:25:36.421 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.419+0000 7fc66a215640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc66410d200 con 0x7fc664102b40 2024-05-10T11:25:36.421 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.419+0000 7fc66a215640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc66410d500 con 0x7fc664103500 2024-05-10T11:25:36.421 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.419+0000 7fc6637fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc664101950 0x7fc66406bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:36.421 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.419+0000 7fc6637fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc664101950 0x7fc66406bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51644/0 (socket says 172.21.15.100:51644) 2024-05-10T11:25:36.421 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.419+0000 7fc6637fe640 1 -- 172.21.15.100:0/1831368755 learned_addr learned my addr 172.21.15.100:0/1831368755 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:25:36.421 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.419+0000 7fc663fff640 1 --2- 172.21.15.100:0/1831368755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc664103500 0x7fc66419fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:36.421 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.419+0000 7fc662ffd640 1 --2- 172.21.15.100:0/1831368755 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc664102b40 0x7fc66419d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:36.422 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.420+0000 7fc6637fe640 1 -- 172.21.15.100:0/1831368755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc664103500 msgr2=0x7fc66419fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:36.422 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.420+0000 7fc6637fe640 1 --2- 172.21.15.100:0/1831368755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc664103500 0x7fc66419fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:36.422 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.420+0000 7fc6637fe640 1 -- 172.21.15.100:0/1831368755 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc664102b40 msgr2=0x7fc66419d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:36.422 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.420+0000 7fc6637fe640 1 --2- 172.21.15.100:0/1831368755 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc664102b40 0x7fc66419d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:36.422 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.420+0000 7fc6637fe640 1 -- 172.21.15.100:0/1831368755 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6641a0660 con 0x7fc664101950 2024-05-10T11:25:36.422 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.420+0000 7fc6637fe640 1 --2- 172.21.15.100:0/1831368755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc664101950 0x7fc66406bd50 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fc65400e9c0 tx=0x7fc65400edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:25:36.422 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.420+0000 7fc660ff9640 1 -- 172.21.15.100:0/1831368755 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc65400ccc0 con 0x7fc664101950 2024-05-10T11:25:36.422 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.420+0000 7fc66a215640 1 -- 172.21.15.100:0/1831368755 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6641a0950 con 0x7fc664101950 2024-05-10T11:25:36.422 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.420+0000 7fc660ff9640 1 -- 172.21.15.100:0/1831368755 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc6540035f0 con 0x7fc664101950 2024-05-10T11:25:36.422 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.420+0000 7fc660ff9640 1 -- 172.21.15.100:0/1831368755 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc65400ce60 con 0x7fc664101950 2024-05-10T11:25:36.422 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.420+0000 7fc66a215640 1 -- 172.21.15.100:0/1831368755 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6641a0e50 con 0x7fc664101950 2024-05-10T11:25:36.424 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.423+0000 7fc660ff9640 1 -- 172.21.15.100:0/1831368755 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fc654004000 con 0x7fc664101950 2024-05-10T11:25:36.425 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.423+0000 7fc66d007740 1 Processor -- start 2024-05-10T11:25:36.425 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.423+0000 7fc66d007740 1 -- start start 2024-05-10T11:25:36.425 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.423+0000 7fc66d007740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc143f61c0 0x55cc143f65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:36.425 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.423+0000 7fc66d007740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc143f6bb0 0x55cc143fb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:36.425 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.423+0000 7fc66d007740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc143fbd20 0x55cc143fe1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:36.425 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.423+0000 7fc66d007740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55cc143565d0 con 0x55cc143f61c0 2024-05-10T11:25:36.425 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.423+0000 7fc66d007740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55cc1430f890 con 0x55cc143f6bb0 2024-05-10T11:25:36.425 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.423+0000 7fc66d007740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55cc14352590 con 0x55cc143fbd20 2024-05-10T11:25:36.425 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.423+0000 7fc663fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc143f61c0 0x55cc143f65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:36.425 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.423+0000 7fc6637fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc143f6bb0 0x55cc143fb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:36.425 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.423+0000 7fc663fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc143f61c0 0x55cc143f65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51652/0 (socket says 172.21.15.100:51652) 2024-05-10T11:25:36.425 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.423+0000 7fc663fff640 1 -- 172.21.15.100:0/280179361 learned_addr learned my addr 172.21.15.100:0/280179361 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:25:36.425 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.424+0000 7fc662ffd640 1 --2- 172.21.15.100:0/280179361 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc143fbd20 0x55cc143fe1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:36.425 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.424+0000 7fc6637fe640 1 -- 172.21.15.100:0/280179361 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc143fbd20 msgr2=0x55cc143fe1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:36.426 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.424+0000 7fc6637fe640 1 --2- 172.21.15.100:0/280179361 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc143fbd20 0x55cc143fe1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:36.426 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.424+0000 7fc6637fe640 1 -- 172.21.15.100:0/280179361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc143f61c0 msgr2=0x55cc143f65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:36.426 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.424+0000 7fc6637fe640 1 --2- 172.21.15.100:0/280179361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc143f61c0 0x55cc143f65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:36.426 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.424+0000 7fc6637fe640 1 -- 172.21.15.100:0/280179361 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc14068210 con 0x55cc143f6bb0 2024-05-10T11:25:36.426 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.424+0000 7fc663fff640 1 --2- 172.21.15.100:0/280179361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc143f61c0 0x55cc143f65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:25:36.426 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.424+0000 7fc662ffd640 1 --2- 172.21.15.100:0/280179361 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc143fbd20 0x55cc143fe1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:25:36.426 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.424+0000 7fc660ff9640 1 --2- 172.21.15.100:0/1831368755 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc64c0b28a0 0x7fc64c0b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:36.426 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.424+0000 7fc6637fe640 1 --2- 172.21.15.100:0/280179361 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc143f6bb0 0x55cc143fb6b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc6540e0070 tx=0x7fc6540e8b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ff5a439d00d85fa server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:25:36.426 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.424+0000 7fc663fff640 1 --2- 172.21.15.100:0/1831368755 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc64c0b28a0 0x7fc64c0b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:36.426 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.424+0000 7fc649ffb640 1 -- 172.21.15.100:0/280179361 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6540a5a80 con 0x55cc143f6bb0 2024-05-10T11:25:36.426 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.424+0000 7fc660ff9640 1 -- 172.21.15.100:0/1831368755 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fc6540e2900 con 0x7fc664101950 2024-05-10T11:25:36.426 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.424+0000 7fc649ffb640 1 -- 172.21.15.100:0/280179361 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc6540a96d0 con 0x55cc143f6bb0 2024-05-10T11:25:36.426 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.424+0000 7fc649ffb640 1 -- 172.21.15.100:0/280179361 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6540a9330 con 0x55cc143f6bb0 2024-05-10T11:25:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.425+0000 7fc663fff640 1 --2- 172.21.15.100:0/1831368755 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc64c0b28a0 0x7fc64c0b4d60 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc65c005fe0 tx=0x7fc65c005f70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:25:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.425+0000 7fc66d007740 1 -- 172.21.15.100:0/280179361 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc143f6bb0 msgr2=0x55cc143fb6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.425+0000 7fc66d007740 1 --2- 172.21.15.100:0/280179361 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc143f6bb0 0x55cc143fb6b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc6540e0070 tx=0x7fc6540e8b40 comp rx=0 tx=0).stop 2024-05-10T11:25:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.425+0000 7fc66d007740 1 -- 172.21.15.100:0/280179361 shutdown_connections 2024-05-10T11:25:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.425+0000 7fc66d007740 1 --2- 172.21.15.100:0/280179361 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc143fbd20 0x55cc143fe1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.425+0000 7fc66d007740 1 --2- 172.21.15.100:0/280179361 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc143f6bb0 0x55cc143fb6b0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.425+0000 7fc66d007740 1 --2- 172.21.15.100:0/280179361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc143f61c0 0x55cc143f65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.425+0000 7fc66d007740 1 -- 172.21.15.100:0/280179361 >> 172.21.15.100:0/280179361 conn(0x55cc143ed420 msgr2=0x55cc143ed820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:25:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.425+0000 7fc66d007740 1 -- 172.21.15.100:0/280179361 shutdown_connections 2024-05-10T11:25:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.425+0000 7fc66d007740 1 -- 172.21.15.100:0/280179361 wait complete. 2024-05-10T11:25:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.426+0000 7fc66d007740 1 -- start start 2024-05-10T11:25:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.426+0000 7fc66d007740 1 Processor -- start 2024-05-10T11:25:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.426+0000 7fc66d007740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc143f61c0 0x55cc14400350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.426+0000 7fc66d007740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc143f6bb0 0x55cc14400890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.426+0000 7fc66d007740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc143fbd20 0x55cc144078d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.426+0000 7fc66d007740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55cc143fecd0 con 0x55cc143fbd20 2024-05-10T11:25:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.426+0000 7fc66d007740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55cc14313370 con 0x55cc143f6bb0 2024-05-10T11:25:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.426+0000 7fc66d007740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55cc143fee50 con 0x55cc143f61c0 2024-05-10T11:25:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.426+0000 7fc66d007740 10 client.0 _ll_register_callbacks cb 0x55cc14067a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:25:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.426+0000 7fc663fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc143fbd20 0x55cc144078d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.426+0000 7fc663fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc143fbd20 0x55cc144078d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51664/0 (socket says 172.21.15.100:51664) 2024-05-10T11:25:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.426+0000 7fc663fff640 1 -- 172.21.15.100:0/1637089107 learned_addr learned my addr 172.21.15.100:0/1637089107 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:25:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.426+0000 7fc6637fe640 1 --2- 172.21.15.100:0/1637089107 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc143f61c0 0x55cc14400350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.426+0000 7fc662ffd640 1 --2- 172.21.15.100:0/1637089107 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc143f6bb0 0x55cc14400890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.427+0000 7fc663fff640 1 -- 172.21.15.100:0/1637089107 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc143f61c0 msgr2=0x55cc14400350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.427+0000 7fc663fff640 1 --2- 172.21.15.100:0/1637089107 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc143f61c0 0x55cc14400350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.427+0000 7fc663fff640 1 -- 172.21.15.100:0/1637089107 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc143f6bb0 msgr2=0x55cc14400890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:36.433 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.427+0000 7fc663fff640 1 --2- 172.21.15.100:0/1637089107 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc143f6bb0 0x55cc14400890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:36.433 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.427+0000 7fc663fff640 1 -- 172.21.15.100:0/1637089107 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc14134620 con 0x55cc143fbd20 2024-05-10T11:25:36.433 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.427+0000 7fc663fff640 1 --2- 172.21.15.100:0/1637089107 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc143fbd20 0x55cc144078d0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fc65c02aeb0 tx=0x7fc65c02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce32c884974d27cc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:25:36.433 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.427+0000 7fc642ffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:25:36.433 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.427+0000 7fc642ffd640 1 -- 172.21.15.100:0/1637089107 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc65c003e40 con 0x55cc143fbd20 2024-05-10T11:25:36.433 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.427+0000 7fc642ffd640 1 -- 172.21.15.100:0/1637089107 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fc65c004250 con 0x55cc143fbd20 2024-05-10T11:25:36.433 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.427+0000 7fc66d007740 1 -- 172.21.15.100:0/1637089107 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6540a96d0 con 0x55cc143fbd20 2024-05-10T11:25:36.433 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.427+0000 7fc642ffd640 1 -- 172.21.15.100:0/1637089107 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc65c004450 con 0x55cc143fbd20 2024-05-10T11:25:36.433 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.427+0000 7fc66d007740 10 client.16845 fetch_fsmap learned FSMap version 31 2024-05-10T11:25:36.433 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.427+0000 7fc66d007740 1 -- 172.21.15.100:0/1637089107 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55cc1440a8b0 con 0x55cc143fbd20 2024-05-10T11:25:36.433 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.427+0000 7fc642ffd640 1 -- 172.21.15.100:0/1637089107 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc6540a96d0 con 0x55cc143fbd20 2024-05-10T11:25:36.433 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.428+0000 7fc642ffd640 1 -- 172.21.15.100:0/1637089107 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fc65c051070 con 0x55cc143fbd20 2024-05-10T11:25:36.433 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.428+0000 7fc66d007740 10 client.16845 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:25:36.433 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.428+0000 7fc66d007740 10 client.16845 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:25:36.433 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.428+0000 7fc66d007740 20 client.16845 populate_metadata read hostname 'smithi100' 2024-05-10T11:25:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.428+0000 7fc66d007740 1 --2- 172.21.15.100:0/1637089107 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55cc1440c0a0 0x55cc1440e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.428+0000 7fc66d007740 4 client.16845 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:25:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.428+0000 7fc66d007740 1 -- 172.21.15.100:0/1637089107 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc143641e0 con 0x55cc1440c0a0 2024-05-10T11:25:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.428+0000 7fc6637fe640 1 --2- 172.21.15.100:0/1637089107 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55cc1440c0a0 0x55cc1440e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.428+0000 7fc642ffd640 10 client.16845 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:25:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.428+0000 7fc6637fe640 1 --2- 172.21.15.100:0/1637089107 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55cc1440c0a0 0x55cc1440e4c0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18bb208be7bf677d server_cookie=6d3975def9eee7be in_seq=0 out_seq=0 2024-05-10T11:25:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.429+0000 7fc642ffd640 1 -- 172.21.15.100:0/1637089107 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55cc143641e0 con 0x55cc1440c0a0 2024-05-10T11:25:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.429+0000 7fc642ffd640 10 client.16845 handle_command_reply: tid=0 2024-05-10T11:25:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.429+0000 7fc66d007740 1 client.16845 shutdown 2024-05-10T11:25:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.429+0000 7fc66d007740 1 -- 172.21.15.100:0/1637089107 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc143fbd20 msgr2=0x55cc144078d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.429+0000 7fc66d007740 1 --2- 172.21.15.100:0/1637089107 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc143fbd20 0x55cc144078d0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fc65c02aeb0 tx=0x7fc65c02a8c0 comp rx=0 tx=0).stop 2024-05-10T11:25:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.429+0000 7fc66d007740 1 -- 172.21.15.100:0/1637089107 shutdown_connections 2024-05-10T11:25:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.429+0000 7fc66d007740 1 --2- 172.21.15.100:0/1637089107 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55cc1440c0a0 0x55cc1440e4c0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.429+0000 7fc66d007740 1 --2- 172.21.15.100:0/1637089107 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc143fbd20 0x55cc144078d0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:36.435 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.429+0000 7fc66d007740 1 --2- 172.21.15.100:0/1637089107 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc143f6bb0 0x55cc14400890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:36.435 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.429+0000 7fc642ffd640 0 client.16845 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:25:36.435 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.429+0000 7fc66d007740 1 --2- 172.21.15.100:0/1637089107 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc143f61c0 0x55cc14400350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:36.435 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.429+0000 7fc66d007740 1 -- 172.21.15.100:0/1637089107 >> 172.21.15.100:0/1637089107 conn(0x55cc143ed420 msgr2=0x55cc143ec580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:25:36.435 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.430+0000 7fc66d007740 1 -- 172.21.15.100:0/1637089107 shutdown_connections 2024-05-10T11:25:36.435 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.430+0000 7fc66d007740 1 -- 172.21.15.100:0/1637089107 wait complete. 2024-05-10T11:25:36.435 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.430+0000 7fc66d007740 20 client.16845 trim_cache size 0 max 16384 2024-05-10T11:25:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.446+0000 7fc66d007740 1 Processor -- start 2024-05-10T11:25:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.446+0000 7fc66d007740 1 -- start start 2024-05-10T11:25:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.446+0000 7fc66d007740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc143f61c0 0x55cc143ffdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.446+0000 7fc66d007740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc143f6bb0 0x55cc14407890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.446+0000 7fc66d007740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc143fbd20 0x55cc143f0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.446+0000 7fc66d007740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc6641ab990 con 0x55cc143f61c0 2024-05-10T11:25:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.446+0000 7fc66d007740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc6641a9260 con 0x55cc143fbd20 2024-05-10T11:25:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.446+0000 7fc66d007740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc6641a70b0 con 0x55cc143f6bb0 2024-05-10T11:25:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.446+0000 7fc6637fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc143f61c0 0x55cc143ffdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.446+0000 7fc6637fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc143f61c0 0x55cc143ffdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51672/0 (socket says 172.21.15.100:51672) 2024-05-10T11:25:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.446+0000 7fc6637fe640 1 -- 172.21.15.100:0/1814743271 learned_addr learned my addr 172.21.15.100:0/1814743271 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:25:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.446+0000 7fc662ffd640 1 --2- 172.21.15.100:0/1814743271 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc143f6bb0 0x55cc14407890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.446+0000 7fc663fff640 1 --2- 172.21.15.100:0/1814743271 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc143fbd20 0x55cc143f0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.447+0000 7fc6637fe640 1 -- 172.21.15.100:0/1814743271 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc143f6bb0 msgr2=0x55cc14407890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.447+0000 7fc6637fe640 1 --2- 172.21.15.100:0/1814743271 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc143f6bb0 0x55cc14407890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.447+0000 7fc6637fe640 1 -- 172.21.15.100:0/1814743271 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc143fbd20 msgr2=0x55cc143f0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.447+0000 7fc6637fe640 1 --2- 172.21.15.100:0/1814743271 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc143fbd20 0x55cc143f0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:36.452 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.447+0000 7fc6637fe640 1 -- 172.21.15.100:0/1814743271 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc143f6f90 con 0x55cc143f61c0 2024-05-10T11:25:36.452 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.447+0000 7fc662ffd640 1 --2- 172.21.15.100:0/1814743271 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc143f6bb0 0x55cc14407890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:25:36.452 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.447+0000 7fc6637fe640 1 --2- 172.21.15.100:0/1814743271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc143f61c0 0x55cc143ffdd0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fc6540e81b0 tx=0x7fc654003a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d29e54e6e73a284e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:25:36.452 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.447+0000 7fc6497fa640 1 -- 172.21.15.100:0/1814743271 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6540a9510 con 0x55cc143f61c0 2024-05-10T11:25:36.452 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.447+0000 7fc6497fa640 1 -- 172.21.15.100:0/1814743271 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc6540a96d0 con 0x55cc143f61c0 2024-05-10T11:25:36.452 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.447+0000 7fc6497fa640 1 -- 172.21.15.100:0/1814743271 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6540a9990 con 0x55cc143f61c0 2024-05-10T11:25:36.452 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.448+0000 7fc66d007740 1 -- 172.21.15.100:0/1814743271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc143f61c0 msgr2=0x55cc143ffdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:36.452 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.448+0000 7fc66d007740 1 --2- 172.21.15.100:0/1814743271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc143f61c0 0x55cc143ffdd0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fc6540e81b0 tx=0x7fc654003a10 comp rx=0 tx=0).stop 2024-05-10T11:25:36.452 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.448+0000 7fc66d007740 1 -- 172.21.15.100:0/1814743271 shutdown_connections 2024-05-10T11:25:36.452 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.448+0000 7fc66d007740 1 --2- 172.21.15.100:0/1814743271 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc143fbd20 0x55cc143f0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:36.452 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.448+0000 7fc66d007740 1 --2- 172.21.15.100:0/1814743271 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc143f6bb0 0x55cc14407890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:36.452 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.448+0000 7fc66d007740 1 --2- 172.21.15.100:0/1814743271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc143f61c0 0x55cc143ffdd0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:36.452 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.448+0000 7fc66d007740 1 -- 172.21.15.100:0/1814743271 >> 172.21.15.100:0/1814743271 conn(0x55cc143ed420 msgr2=0x55cc143f0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:25:36.452 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.448+0000 7fc66d007740 1 -- 172.21.15.100:0/1814743271 shutdown_connections 2024-05-10T11:25:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.448+0000 7fc66d007740 1 -- 172.21.15.100:0/1814743271 wait complete. 2024-05-10T11:25:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.448+0000 7fc66d007740 1 -- start start 2024-05-10T11:25:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.448+0000 7fc66d007740 1 Processor -- start 2024-05-10T11:25:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.448+0000 7fc66d007740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc143f61c0 0x55cc144050a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.448+0000 7fc66d007740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc143f6bb0 0x55cc144055e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.448+0000 7fc66d007740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc143fbd20 0x55cc144065f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.448+0000 7fc66d007740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc6641a7720 con 0x55cc143f6bb0 2024-05-10T11:25:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.448+0000 7fc66d007740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc6641a8b90 con 0x55cc143f61c0 2024-05-10T11:25:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.448+0000 7fc66d007740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc6641a7df0 con 0x55cc143fbd20 2024-05-10T11:25:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.449+0000 7fc66d007740 10 client.0 _ll_register_callbacks cb 0x7fc65c0316d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:25:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.449+0000 7fc662ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc143f6bb0 0x55cc144055e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.449+0000 7fc662ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc143f6bb0 0x55cc144055e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51684/0 (socket says 172.21.15.100:51684) 2024-05-10T11:25:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.449+0000 7fc662ffd640 1 -- 172.21.15.100:0/3020970457 learned_addr learned my addr 172.21.15.100:0/3020970457 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:25:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.449+0000 7fc6637fe640 1 --2- 172.21.15.100:0/3020970457 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc143f61c0 0x55cc144050a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.449+0000 7fc663fff640 1 --2- 172.21.15.100:0/3020970457 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc143fbd20 0x55cc144065f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.449+0000 7fc662ffd640 1 -- 172.21.15.100:0/3020970457 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc143fbd20 msgr2=0x55cc144065f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.449+0000 7fc662ffd640 1 --2- 172.21.15.100:0/3020970457 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc143fbd20 0x55cc144065f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.449+0000 7fc662ffd640 1 -- 172.21.15.100:0/3020970457 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc143f61c0 msgr2=0x55cc144050a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.449+0000 7fc662ffd640 1 --2- 172.21.15.100:0/3020970457 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc143f61c0 0x55cc144050a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.449+0000 7fc662ffd640 1 -- 172.21.15.100:0/3020970457 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc14406be0 con 0x55cc143f6bb0 2024-05-10T11:25:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.449+0000 7fc662ffd640 1 --2- 172.21.15.100:0/3020970457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc143f6bb0 0x55cc144055e0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fc658010a20 tx=0x7fc658010ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7badc909ae5731a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:25:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.449+0000 7fc642ffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:25:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.449+0000 7fc642ffd640 1 -- 172.21.15.100:0/3020970457 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc65801f070 con 0x55cc143f6bb0 2024-05-10T11:25:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.449+0000 7fc642ffd640 1 -- 172.21.15.100:0/3020970457 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fc65800fdf0 con 0x55cc143f6bb0 2024-05-10T11:25:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.449+0000 7fc66d007740 1 -- 172.21.15.100:0/3020970457 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6540a96d0 con 0x55cc143f6bb0 2024-05-10T11:25:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.449+0000 7fc642ffd640 1 -- 172.21.15.100:0/3020970457 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc65802a050 con 0x55cc143f6bb0 2024-05-10T11:25:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.449+0000 7fc66d007740 10 client.16851 fetch_fsmap learned FSMap version 31 2024-05-10T11:25:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.449+0000 7fc66d007740 1 -- 172.21.15.100:0/3020970457 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55cc14476ad0 con 0x55cc143f6bb0 2024-05-10T11:25:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.449+0000 7fc642ffd640 1 -- 172.21.15.100:0/3020970457 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc6540a96d0 con 0x55cc143f6bb0 2024-05-10T11:25:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.450+0000 7fc642ffd640 1 -- 172.21.15.100:0/3020970457 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fc65801a4c0 con 0x55cc143f6bb0 2024-05-10T11:25:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.450+0000 7fc66d007740 10 client.16851 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:25:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.450+0000 7fc66d007740 10 client.16851 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:25:36.455 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.450+0000 7fc66d007740 20 client.16851 populate_metadata read hostname 'smithi100' 2024-05-10T11:25:36.455 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.450+0000 7fc66d007740 1 --2- 172.21.15.100:0/3020970457 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55cc14477530 0x55cc14479950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:36.455 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.450+0000 7fc66d007740 4 client.16851 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:25:36.455 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.450+0000 7fc66d007740 1 -- 172.21.15.100:0/3020970457 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cc1435e050 con 0x55cc14477530 2024-05-10T11:25:36.455 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.450+0000 7fc6637fe640 1 --2- 172.21.15.100:0/3020970457 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55cc14477530 0x55cc14479950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:36.455 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.450+0000 7fc6637fe640 1 --2- 172.21.15.100:0/3020970457 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55cc14477530 0x55cc14479950 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=182bceebe78ceeaa server_cookie=17a2c2339e6cd748 in_seq=0 out_seq=0 2024-05-10T11:25:36.455 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.450+0000 7fc642ffd640 10 client.16851 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:25:36.471 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.471+0000 7fc642ffd640 1 -- 172.21.15.100:0/3020970457 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cc1435e050 con 0x55cc14477530 2024-05-10T11:25:36.471 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.471+0000 7fc642ffd640 10 client.16851 handle_command_reply: tid=0 2024-05-10T11:25:36.471 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.471+0000 7fc66d007740 1 client.16851 shutdown 2024-05-10T11:25:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.471+0000 7fc66d007740 1 -- 172.21.15.100:0/3020970457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc143f6bb0 msgr2=0x55cc144055e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.471+0000 7fc66d007740 1 --2- 172.21.15.100:0/3020970457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc143f6bb0 0x55cc144055e0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fc658010a20 tx=0x7fc658010ee0 comp rx=0 tx=0).stop 2024-05-10T11:25:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.471+0000 7fc66d007740 1 -- 172.21.15.100:0/3020970457 shutdown_connections 2024-05-10T11:25:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.471+0000 7fc66d007740 1 --2- 172.21.15.100:0/3020970457 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55cc14477530 0x55cc14479950 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.471+0000 7fc66d007740 1 --2- 172.21.15.100:0/3020970457 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc143fbd20 0x55cc144065f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.471+0000 7fc642ffd640 0 client.16851 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:25:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.471+0000 7fc66d007740 1 --2- 172.21.15.100:0/3020970457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc143f6bb0 0x55cc144055e0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.471+0000 7fc66d007740 1 --2- 172.21.15.100:0/3020970457 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc143f61c0 0x55cc144050a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.471+0000 7fc66d007740 1 -- 172.21.15.100:0/3020970457 >> 172.21.15.100:0/3020970457 conn(0x55cc143ed420 msgr2=0x55cc143f0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:25:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.471+0000 7fc66d007740 1 -- 172.21.15.100:0/3020970457 shutdown_connections 2024-05-10T11:25:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.471+0000 7fc66d007740 1 -- 172.21.15.100:0/3020970457 wait complete. 2024-05-10T11:25:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.471+0000 7fc66d007740 20 client.16851 trim_cache size 0 max 16384 2024-05-10T11:25:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:25:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:25:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:25:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:25:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.472+0000 7fc66a215640 1 -- 172.21.15.100:0/1831368755 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc64c0b28a0 msgr2=0x7fc64c0b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.472+0000 7fc66a215640 1 --2- 172.21.15.100:0/1831368755 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc64c0b28a0 0x7fc64c0b4d60 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc65c005fe0 tx=0x7fc65c005f70 comp rx=0 tx=0).stop 2024-05-10T11:25:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.472+0000 7fc66a215640 1 -- 172.21.15.100:0/1831368755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc664101950 msgr2=0x7fc66406bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.472+0000 7fc66a215640 1 --2- 172.21.15.100:0/1831368755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc664101950 0x7fc66406bd50 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fc65400e9c0 tx=0x7fc65400edc0 comp rx=0 tx=0).stop 2024-05-10T11:25:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.472+0000 7fc66a215640 1 -- 172.21.15.100:0/1831368755 shutdown_connections 2024-05-10T11:25:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.472+0000 7fc66a215640 1 --2- 172.21.15.100:0/1831368755 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc64c0b28a0 0x7fc64c0b4d60 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.472+0000 7fc66a215640 1 --2- 172.21.15.100:0/1831368755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc664103500 0x7fc66419fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.472+0000 7fc66a215640 1 --2- 172.21.15.100:0/1831368755 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc664102b40 0x7fc66419d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.472+0000 7fc66a215640 1 --2- 172.21.15.100:0/1831368755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc664101950 0x7fc66406bd50 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.472+0000 7fc66a215640 1 -- 172.21.15.100:0/1831368755 >> 172.21.15.100:0/1831368755 conn(0x7fc6640fcec0 msgr2=0x7fc664105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:25:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.472+0000 7fc66a215640 1 -- 172.21.15.100:0/1831368755 shutdown_connections 2024-05-10T11:25:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:36.472+0000 7fc66a215640 1 -- 172.21.15.100:0/1831368755 wait complete. 2024-05-10T11:25:36.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:25:36 smithi178 ceph-mon[33359]: pgmap v636: 129 pgs: 129 active+clean; 75 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 1.3 MiB/s wr, 157 op/s 2024-05-10T11:25:36.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:25:36 smithi178 ceph-mon[33359]: scrub queued for path: / 2024-05-10T11:25:36.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:25:36 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:25:36.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:25:36 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:25:36.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:25:36 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:25:36.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:25:36 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:25:36.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:25:36 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:25:36.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:25:36 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:25:36.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:25:36 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:25:36.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:25:36 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:25:36.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:25:36 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:25:36.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:25:36 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:25:36.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:25:36 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:25:36.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:25:36 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:25:36.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:25:36 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:25:36.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:25:36 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:25:36.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:25:36 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:25:36.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:25:36 smithi188 ceph-mon[34075]: pgmap v636: 129 pgs: 129 active+clean; 75 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 1.3 MiB/s wr, 157 op/s 2024-05-10T11:25:36.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:25:36 smithi188 ceph-mon[34075]: scrub queued for path: / 2024-05-10T11:25:36.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:25:36 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:25:36.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:25:36 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:25:36.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:25:36 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:25:36.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:25:36 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:25:36.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:25:36 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:25:36.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:25:36 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:25:36.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:25:36 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:25:36.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:25:36 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:25:36.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:25:36 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:25:36.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:25:36 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:25:36.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:25:36 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:25:36.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:25:36 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:25:36.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:25:36 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:25:36.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:25:36 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:25:36.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:25:36 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:25:36.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:25:36 smithi100 ceph-mon[30296]: pgmap v636: 129 pgs: 129 active+clean; 75 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 1.3 MiB/s wr, 157 op/s 2024-05-10T11:25:36.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:25:36 smithi100 ceph-mon[30296]: scrub queued for path: / 2024-05-10T11:25:36.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:25:36 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:25:36.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:25:36 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:25:36.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:25:36 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:25:36.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:25:36 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:25:36.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:25:36 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:25:36.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:25:36 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:25:36.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:25:36 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:25:36.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:25:36 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:25:36.539 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:25:36 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:25:36.539 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:25:36 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:25:36.539 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:25:36 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:25:36.539 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:25:36 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:25:36.539 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:25:36 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:25:36.539 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:25:36 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:25:36.539 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:25:36 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:25:36.974 INFO:tasks.workunit.client.0.smithi100.stdout: 1 171259 2.73 MB/sec execute 211 sec latency 62.498 ms 2024-05-10T11:25:37.797 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T11:25:37.974 INFO:tasks.workunit.client.0.smithi100.stdout: 1 171777 2.73 MB/sec execute 212 sec latency 95.520 ms 2024-05-10T11:25:38.107 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:25:38.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:25:38 smithi178 ceph-mon[33359]: pgmap v637: 129 pgs: 129 active+clean; 75 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 1.2 MiB/s wr, 137 op/s 2024-05-10T11:25:38.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:25:38 smithi188 ceph-mon[34075]: pgmap v637: 129 pgs: 129 active+clean; 75 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 1.2 MiB/s wr, 137 op/s 2024-05-10T11:25:38.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:25:38 smithi100 ceph-mon[30296]: pgmap v637: 129 pgs: 129 active+clean; 75 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 1.2 MiB/s wr, 137 op/s 2024-05-10T11:25:38.974 INFO:tasks.workunit.client.0.smithi100.stdout: 1 172311 2.72 MB/sec execute 213 sec latency 91.206 ms 2024-05-10T11:25:39.974 INFO:tasks.workunit.client.0.smithi100.stdout: 1 172832 2.72 MB/sec execute 214 sec latency 15.967 ms 2024-05-10T11:25:40.484 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:25:40 smithi100 ceph-mon[30296]: pgmap v638: 129 pgs: 129 active+clean; 77 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 1.4 MiB/s wr, 173 op/s 2024-05-10T11:25:40.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:25:40 smithi178 ceph-mon[33359]: pgmap v638: 129 pgs: 129 active+clean; 77 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 1.4 MiB/s wr, 173 op/s 2024-05-10T11:25:40.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:25:40 smithi188 ceph-mon[34075]: pgmap v638: 129 pgs: 129 active+clean; 77 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 1.4 MiB/s wr, 173 op/s 2024-05-10T11:25:40.974 INFO:tasks.workunit.client.0.smithi100.stdout: 1 173564 2.74 MB/sec execute 215 sec latency 36.585 ms 2024-05-10T11:25:41.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.036+0000 7fa259fa0700 1 -- 172.21.15.100:0/878697615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa254108700 msgr2=0x7fa2541102d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:41.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.036+0000 7fa259fa0700 1 --2- 172.21.15.100:0/878697615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa254108700 0x7fa2541102d0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fa24c00b0d0 tx=0x7fa24c00b3e0 comp rx=0 tx=0).stop 2024-05-10T11:25:41.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.037+0000 7fa259fa0700 1 -- 172.21.15.100:0/878697615 shutdown_connections 2024-05-10T11:25:41.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.037+0000 7fa259fa0700 1 --2- 172.21.15.100:0/878697615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa254108700 0x7fa2541102d0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:41.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.037+0000 7fa259fa0700 1 --2- 172.21.15.100:0/878697615 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa254103620 0x7fa254103a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:41.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.037+0000 7fa259fa0700 1 --2- 172.21.15.100:0/878697615 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa254103fd0 0x7fa254108020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:41.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.037+0000 7fa259fa0700 1 -- 172.21.15.100:0/878697615 >> 172.21.15.100:0/878697615 conn(0x7fa2540feef0 msgr2=0x7fa254101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:25:41.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.037+0000 7fa259fa0700 1 -- 172.21.15.100:0/878697615 shutdown_connections 2024-05-10T11:25:41.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.037+0000 7fa259fa0700 1 -- 172.21.15.100:0/878697615 wait complete. 2024-05-10T11:25:41.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.037+0000 7fa259fa0700 1 Processor -- start 2024-05-10T11:25:41.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.037+0000 7fa259fa0700 1 -- start start 2024-05-10T11:25:41.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.038+0000 7fa259fa0700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa254103620 0x7fa25419f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:41.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.038+0000 7fa259fa0700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa254103fd0 0x7fa25419fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:41.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.038+0000 7fa259fa0700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa254108700 0x7fa2541997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:41.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.038+0000 7fa259fa0700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa254113470 con 0x7fa254103620 2024-05-10T11:25:41.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.038+0000 7fa259fa0700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa2541132f0 con 0x7fa254103fd0 2024-05-10T11:25:41.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.038+0000 7fa259fa0700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa2541135f0 con 0x7fa254108700 2024-05-10T11:25:41.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.038+0000 7fa258f9e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa254103620 0x7fa25419f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:41.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.038+0000 7fa258f9e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa254103620 0x7fa25419f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50324/0 (socket says 172.21.15.100:50324) 2024-05-10T11:25:41.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.038+0000 7fa258f9e700 1 -- 172.21.15.100:0/439901311 learned_addr learned my addr 172.21.15.100:0/439901311 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:25:41.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.038+0000 7fa253fff700 1 --2- 172.21.15.100:0/439901311 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa254103fd0 0x7fa25419fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:41.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.038+0000 7fa25979f700 1 --2- 172.21.15.100:0/439901311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa254108700 0x7fa2541997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:41.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.038+0000 7fa258f9e700 1 -- 172.21.15.100:0/439901311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa254108700 msgr2=0x7fa2541997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:41.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.038+0000 7fa258f9e700 1 --2- 172.21.15.100:0/439901311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa254108700 0x7fa2541997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:41.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.038+0000 7fa258f9e700 1 -- 172.21.15.100:0/439901311 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa254103fd0 msgr2=0x7fa25419fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:41.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.038+0000 7fa258f9e700 1 --2- 172.21.15.100:0/439901311 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa254103fd0 0x7fa25419fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:41.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.038+0000 7fa258f9e700 1 -- 172.21.15.100:0/439901311 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa254199fb0 con 0x7fa254103620 2024-05-10T11:25:41.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.038+0000 7fa253fff700 1 --2- 172.21.15.100:0/439901311 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa254103fd0 0x7fa25419fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:25:41.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.038+0000 7fa258f9e700 1 --2- 172.21.15.100:0/439901311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa254103620 0x7fa25419f650 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fa24800c920 tx=0x7fa24800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:25:41.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.039+0000 7fa251ffb700 1 -- 172.21.15.100:0/439901311 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa248004a30 con 0x7fa254103620 2024-05-10T11:25:41.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.039+0000 7fa259fa0700 1 -- 172.21.15.100:0/439901311 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa25419a2a0 con 0x7fa254103620 2024-05-10T11:25:41.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.039+0000 7fa251ffb700 1 -- 172.21.15.100:0/439901311 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa2480054b0 con 0x7fa254103620 2024-05-10T11:25:41.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.039+0000 7fa251ffb700 1 -- 172.21.15.100:0/439901311 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa248007b70 con 0x7fa254103620 2024-05-10T11:25:41.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.039+0000 7fa259fa0700 1 -- 172.21.15.100:0/439901311 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2541a64b0 con 0x7fa254103620 2024-05-10T11:25:41.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.041+0000 7fa251ffb700 1 -- 172.21.15.100:0/439901311 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fa248007d10 con 0x7fa254103620 2024-05-10T11:25:41.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.042+0000 7fa251ffb700 1 --2- 172.21.15.100:0/439901311 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa2440b2900 0x7fa2440b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:41.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.042+0000 7fa253fff700 1 --2- 172.21.15.100:0/439901311 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa2440b2900 0x7fa2440b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:41.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.042+0000 7fa251ffb700 1 -- 172.21.15.100:0/439901311 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fa2480e2020 con 0x7fa254103620 2024-05-10T11:25:41.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.043+0000 7fa253fff700 1 --2- 172.21.15.100:0/439901311 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa2440b2900 0x7fa2440b4dc0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa254109cb0 tx=0x7fa240009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:25:41.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.043+0000 7fa265a95b80 1 Processor -- start 2024-05-10T11:25:41.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.043+0000 7fa265a95b80 1 -- start start 2024-05-10T11:25:41.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.043+0000 7fa265a95b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e5fb627c0 0x555e5fb62ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:41.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.043+0000 7fa265a95b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e5fb630e0 0x555e5fb5cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:41.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.043+0000 7fa265a95b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e5fb5d160 0x555e5fb5d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:41.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.044+0000 7fa25979f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e5fb5d160 0x555e5fb5d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:41.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.044+0000 7fa25979f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e5fb5d160 0x555e5fb5d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50328/0 (socket says 172.21.15.100:50328) 2024-05-10T11:25:41.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.044+0000 7fa25979f700 1 -- 172.21.15.100:0/286808467 learned_addr learned my addr 172.21.15.100:0/286808467 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:25:41.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.044+0000 7fa253fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e5fb630e0 0x555e5fb5cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:41.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.044+0000 7fa265a95b80 1 -- 172.21.15.100:0/286808467 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x555e5faaa940 con 0x555e5fb5d160 2024-05-10T11:25:41.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.044+0000 7fa265a95b80 1 -- 172.21.15.100:0/286808467 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x555e5fa0fda0 con 0x555e5fb630e0 2024-05-10T11:25:41.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.044+0000 7fa265a95b80 1 -- 172.21.15.100:0/286808467 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x555e5faa6900 con 0x555e5fb627c0 2024-05-10T11:25:41.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.044+0000 7fa258f9e700 1 --2- 172.21.15.100:0/286808467 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e5fb627c0 0x555e5fb62ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:41.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.044+0000 7fa253fff700 1 -- 172.21.15.100:0/286808467 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e5fb627c0 msgr2=0x555e5fb62ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:41.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.044+0000 7fa253fff700 1 --2- 172.21.15.100:0/286808467 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e5fb627c0 0x555e5fb62ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:41.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.044+0000 7fa253fff700 1 -- 172.21.15.100:0/286808467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e5fb5d160 msgr2=0x555e5fb5d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:41.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.044+0000 7fa253fff700 1 --2- 172.21.15.100:0/286808467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e5fb5d160 0x555e5fb5d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:41.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.044+0000 7fa253fff700 1 -- 172.21.15.100:0/286808467 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e5f9f78d0 con 0x555e5fb630e0 2024-05-10T11:25:41.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.044+0000 7fa253fff700 1 --2- 172.21.15.100:0/286808467 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e5fb630e0 0x555e5fb5cc20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa2400042d0 tx=0x7fa240014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad48eef66962c7e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:25:41.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.044+0000 7fa23effd700 1 -- 172.21.15.100:0/286808467 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa240003b10 con 0x555e5fb630e0 2024-05-10T11:25:41.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.044+0000 7fa23effd700 1 -- 172.21.15.100:0/286808467 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa240003cb0 con 0x555e5fb630e0 2024-05-10T11:25:41.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.044+0000 7fa23effd700 1 -- 172.21.15.100:0/286808467 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa240025610 con 0x555e5fb630e0 2024-05-10T11:25:41.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.045+0000 7fa265a95b80 1 -- 172.21.15.100:0/286808467 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e5fb630e0 msgr2=0x555e5fb5cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:41.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.045+0000 7fa265a95b80 1 --2- 172.21.15.100:0/286808467 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e5fb630e0 0x555e5fb5cc20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa2400042d0 tx=0x7fa240014000 comp rx=0 tx=0).stop 2024-05-10T11:25:41.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.045+0000 7fa265a95b80 1 -- 172.21.15.100:0/286808467 shutdown_connections 2024-05-10T11:25:41.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.045+0000 7fa265a95b80 1 --2- 172.21.15.100:0/286808467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e5fb5d160 0x555e5fb5d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:41.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.045+0000 7fa265a95b80 1 --2- 172.21.15.100:0/286808467 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e5fb627c0 0x555e5fb62ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:41.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.045+0000 7fa265a95b80 1 --2- 172.21.15.100:0/286808467 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e5fb630e0 0x555e5fb5cc20 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:41.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.045+0000 7fa265a95b80 1 -- 172.21.15.100:0/286808467 >> 172.21.15.100:0/286808467 conn(0x555e5fa84b70 msgr2=0x555e5fb532d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:25:41.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.046+0000 7fa265a95b80 1 -- 172.21.15.100:0/286808467 shutdown_connections 2024-05-10T11:25:41.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.046+0000 7fa265a95b80 1 -- 172.21.15.100:0/286808467 wait complete. 2024-05-10T11:25:41.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.046+0000 7fa265a95b80 1 -- start start 2024-05-10T11:25:41.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.046+0000 7fa265a95b80 1 Processor -- start 2024-05-10T11:25:41.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.047+0000 7fa265a95b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e5fb5d160 0x555e5fb68d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:41.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.047+0000 7fa265a95b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e5fb627c0 0x555e5fb660d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:41.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.047+0000 7fa265a95b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e5fb630e0 0x555e5fb66610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:41.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.047+0000 7fa265a95b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x555e5fb69a40 con 0x555e5fb627c0 2024-05-10T11:25:41.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.047+0000 7fa265a95b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x555e5fa63ed0 con 0x555e5fb630e0 2024-05-10T11:25:41.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.047+0000 7fa265a95b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x555e5fb69bc0 con 0x555e5fb5d160 2024-05-10T11:25:41.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.047+0000 7fa265a95b80 10 client.0 _ll_register_callbacks cb 0x555e5fa3e5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:25:41.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.047+0000 7fa258f9e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e5fb5d160 0x555e5fb68d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:41.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.047+0000 7fa258f9e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e5fb5d160 0x555e5fb68d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:39518/0 (socket says 172.21.15.100:39518) 2024-05-10T11:25:41.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.047+0000 7fa258f9e700 1 -- 172.21.15.100:0/1511904118 learned_addr learned my addr 172.21.15.100:0/1511904118 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:25:41.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.047+0000 7fa253fff700 1 --2- 172.21.15.100:0/1511904118 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e5fb627c0 0x555e5fb660d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:41.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.047+0000 7fa258f9e700 1 -- 172.21.15.100:0/1511904118 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e5fb630e0 msgr2=0x555e5fb66610 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T11:25:41.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.047+0000 7fa258f9e700 1 --2- 172.21.15.100:0/1511904118 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e5fb630e0 0x555e5fb66610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:41.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.047+0000 7fa258f9e700 1 -- 172.21.15.100:0/1511904118 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e5fb627c0 msgr2=0x555e5fb660d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:41.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.047+0000 7fa258f9e700 1 --2- 172.21.15.100:0/1511904118 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e5fb627c0 0x555e5fb660d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:41.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.047+0000 7fa258f9e700 1 -- 172.21.15.100:0/1511904118 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e5fa61760 con 0x555e5fb5d160 2024-05-10T11:25:41.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.047+0000 7fa258f9e700 1 --2- 172.21.15.100:0/1511904118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e5fb5d160 0x555e5fb68d30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa2480a64f0 tx=0x7fa2480a5790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=146da4fe03200207 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:25:41.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.047+0000 7fa233fff700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:25:41.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.048+0000 7fa233fff700 1 -- 172.21.15.100:0/1511904118 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2480c3060 con 0x555e5fb5d160 2024-05-10T11:25:41.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.048+0000 7fa233fff700 1 -- 172.21.15.100:0/1511904118 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fa2480b1e60 con 0x555e5fb5d160 2024-05-10T11:25:41.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.048+0000 7fa265a95b80 1 -- 172.21.15.100:0/1511904118 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa240003cb0 con 0x555e5fb5d160 2024-05-10T11:25:41.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.048+0000 7fa233fff700 1 -- 172.21.15.100:0/1511904118 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa2480ca050 con 0x555e5fb5d160 2024-05-10T11:25:41.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.048+0000 7fa265a95b80 10 client.26338 fetch_fsmap learned FSMap version 31 2024-05-10T11:25:41.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.048+0000 7fa265a95b80 1 -- 172.21.15.100:0/1511904118 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x555e5fb58870 con 0x555e5fb5d160 2024-05-10T11:25:41.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.048+0000 7fa233fff700 1 -- 172.21.15.100:0/1511904118 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa240003cb0 con 0x555e5fb5d160 2024-05-10T11:25:41.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.048+0000 7fa233fff700 1 -- 172.21.15.100:0/1511904118 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fa2480ad170 con 0x555e5fb5d160 2024-05-10T11:25:41.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.048+0000 7fa265a95b80 10 client.26338 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:25:41.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.048+0000 7fa265a95b80 10 client.26338 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:25:41.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.048+0000 7fa265a95b80 20 client.26338 populate_metadata read hostname 'smithi100' 2024-05-10T11:25:41.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.049+0000 7fa265a95b80 1 --2- 172.21.15.100:0/1511904118 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x555e5fb5ae00 0x555e5fb5b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:41.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.049+0000 7fa265a95b80 4 client.26338 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:25:41.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.049+0000 7fa265a95b80 1 -- 172.21.15.100:0/1511904118 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e5fab8320 con 0x555e5fb5ae00 2024-05-10T11:25:41.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.049+0000 7fa253fff700 1 --2- 172.21.15.100:0/1511904118 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x555e5fb5ae00 0x555e5fb5b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:41.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.049+0000 7fa233fff700 10 client.26338 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:25:41.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.049+0000 7fa253fff700 1 --2- 172.21.15.100:0/1511904118 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x555e5fb5ae00 0x555e5fb5b270 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53c71f3b9c8bd63b server_cookie=a2056fbf9dc3ff3d in_seq=0 out_seq=0 2024-05-10T11:25:41.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.050+0000 7fa233fff700 1 -- 172.21.15.100:0/1511904118 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555e5fab8320 con 0x555e5fb5ae00 2024-05-10T11:25:41.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.050+0000 7fa233fff700 10 client.26338 handle_command_reply: tid=0 2024-05-10T11:25:41.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.050+0000 7fa265a95b80 1 client.26338 shutdown 2024-05-10T11:25:41.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.050+0000 7fa265a95b80 1 -- 172.21.15.100:0/1511904118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e5fb5d160 msgr2=0x555e5fb68d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:41.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.050+0000 7fa265a95b80 1 --2- 172.21.15.100:0/1511904118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e5fb5d160 0x555e5fb68d30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa2480a64f0 tx=0x7fa2480a5790 comp rx=0 tx=0).stop 2024-05-10T11:25:41.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.050+0000 7fa265a95b80 1 -- 172.21.15.100:0/1511904118 shutdown_connections 2024-05-10T11:25:41.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.050+0000 7fa265a95b80 1 --2- 172.21.15.100:0/1511904118 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x555e5fb5ae00 0x555e5fb5b270 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:41.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.050+0000 7fa265a95b80 1 --2- 172.21.15.100:0/1511904118 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e5fb630e0 0x555e5fb66610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:41.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.050+0000 7fa265a95b80 1 --2- 172.21.15.100:0/1511904118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e5fb5d160 0x555e5fb68d30 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:41.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.050+0000 7fa265a95b80 1 --2- 172.21.15.100:0/1511904118 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e5fb627c0 0x555e5fb660d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:41.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.050+0000 7fa233fff700 0 client.26338 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:25:41.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.050+0000 7fa265a95b80 1 -- 172.21.15.100:0/1511904118 >> 172.21.15.100:0/1511904118 conn(0x555e5fa84b70 msgr2=0x555e5fb52960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:25:41.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.050+0000 7fa265a95b80 1 -- 172.21.15.100:0/1511904118 shutdown_connections 2024-05-10T11:25:41.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.050+0000 7fa265a95b80 1 -- 172.21.15.100:0/1511904118 wait complete. 2024-05-10T11:25:41.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.051+0000 7fa265a95b80 20 client.26338 trim_cache size 0 max 16384 2024-05-10T11:25:41.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.076+0000 7fa265a95b80 1 Processor -- start 2024-05-10T11:25:41.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.076+0000 7fa265a95b80 1 -- start start 2024-05-10T11:25:41.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.076+0000 7fa265a95b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e5fb5d160 0x555e5fb5ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:41.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.076+0000 7fa265a95b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e5fb627c0 0x555e5fb530e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:41.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.076+0000 7fa265a95b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e5fb630e0 0x555e5fb64fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:41.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.076+0000 7fa265a95b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa2541af5c0 con 0x555e5fb630e0 2024-05-10T11:25:41.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.076+0000 7fa265a95b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa2541ace50 con 0x555e5fb627c0 2024-05-10T11:25:41.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.076+0000 7fa265a95b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa2541aacc0 con 0x555e5fb5d160 2024-05-10T11:25:41.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.076+0000 7fa25979f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e5fb630e0 0x555e5fb64fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:41.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.076+0000 7fa253fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e5fb627c0 0x555e5fb530e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:41.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.076+0000 7fa258f9e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e5fb5d160 0x555e5fb5ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:41.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.076+0000 7fa25979f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e5fb630e0 0x555e5fb64fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50336/0 (socket says 172.21.15.100:50336) 2024-05-10T11:25:41.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.076+0000 7fa25979f700 1 -- 172.21.15.100:0/3112931417 learned_addr learned my addr 172.21.15.100:0/3112931417 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:25:41.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.077+0000 7fa253fff700 1 -- 172.21.15.100:0/3112931417 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e5fb5d160 msgr2=0x555e5fb5ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:41.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.077+0000 7fa253fff700 1 --2- 172.21.15.100:0/3112931417 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e5fb5d160 0x555e5fb5ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:41.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.077+0000 7fa253fff700 1 -- 172.21.15.100:0/3112931417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e5fb630e0 msgr2=0x555e5fb64fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:41.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.077+0000 7fa253fff700 1 --2- 172.21.15.100:0/3112931417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e5fb630e0 0x555e5fb64fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:41.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.077+0000 7fa253fff700 1 -- 172.21.15.100:0/3112931417 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e5fb5b850 con 0x555e5fb627c0 2024-05-10T11:25:41.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.077+0000 7fa253fff700 1 --2- 172.21.15.100:0/3112931417 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e5fb627c0 0x555e5fb530e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x555e5fb63500 tx=0x7fa2400092d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3795d47a3d0ad73a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:25:41.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.077+0000 7fa23e7fc700 1 -- 172.21.15.100:0/3112931417 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa240003d70 con 0x555e5fb627c0 2024-05-10T11:25:41.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.077+0000 7fa23e7fc700 1 -- 172.21.15.100:0/3112931417 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa24001e880 con 0x555e5fb627c0 2024-05-10T11:25:41.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.077+0000 7fa23e7fc700 1 -- 172.21.15.100:0/3112931417 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2400054a0 con 0x555e5fb627c0 2024-05-10T11:25:41.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.078+0000 7fa265a95b80 1 -- 172.21.15.100:0/3112931417 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e5fb627c0 msgr2=0x555e5fb530e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:41.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.078+0000 7fa265a95b80 1 --2- 172.21.15.100:0/3112931417 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e5fb627c0 0x555e5fb530e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x555e5fb63500 tx=0x7fa2400092d0 comp rx=0 tx=0).stop 2024-05-10T11:25:41.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.078+0000 7fa265a95b80 1 -- 172.21.15.100:0/3112931417 shutdown_connections 2024-05-10T11:25:41.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.078+0000 7fa265a95b80 1 --2- 172.21.15.100:0/3112931417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e5fb630e0 0x555e5fb64fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:41.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.078+0000 7fa265a95b80 1 --2- 172.21.15.100:0/3112931417 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e5fb5d160 0x555e5fb5ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:41.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.078+0000 7fa265a95b80 1 --2- 172.21.15.100:0/3112931417 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e5fb627c0 0x555e5fb530e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:41.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.078+0000 7fa265a95b80 1 -- 172.21.15.100:0/3112931417 >> 172.21.15.100:0/3112931417 conn(0x555e5fa84b70 msgr2=0x555e5fb64890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:25:41.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.078+0000 7fa265a95b80 1 -- 172.21.15.100:0/3112931417 shutdown_connections 2024-05-10T11:25:41.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.078+0000 7fa265a95b80 1 -- 172.21.15.100:0/3112931417 wait complete. 2024-05-10T11:25:41.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.078+0000 7fa265a95b80 1 -- start start 2024-05-10T11:25:41.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.078+0000 7fa265a95b80 1 Processor -- start 2024-05-10T11:25:41.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.079+0000 7fa265a95b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e5fb5d160 0x555e5fb761f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:41.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.079+0000 7fa265a95b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e5fb627c0 0x555e5fb76730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:41.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.079+0000 7fa265a95b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e5fb630e0 0x555e5fb76c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:41.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.079+0000 7fa265a95b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa2541ab310 con 0x555e5fb5d160 2024-05-10T11:25:41.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.079+0000 7fa265a95b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa2541ac780 con 0x555e5fb630e0 2024-05-10T11:25:41.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.079+0000 7fa265a95b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa2541ab9e0 con 0x555e5fb627c0 2024-05-10T11:25:41.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.079+0000 7fa265a95b80 10 client.0 _ll_register_callbacks cb 0x555e5fa3e5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:25:41.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.079+0000 7fa25979f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e5fb630e0 0x555e5fb76c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:41.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.079+0000 7fa253fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e5fb627c0 0x555e5fb76730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:41.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.079+0000 7fa253fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e5fb627c0 0x555e5fb76730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:39538/0 (socket says 172.21.15.100:39538) 2024-05-10T11:25:41.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.079+0000 7fa25979f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e5fb630e0 0x555e5fb76c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:43142/0 (socket says 172.21.15.100:43142) 2024-05-10T11:25:41.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.079+0000 7fa258f9e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e5fb5d160 0x555e5fb761f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:41.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.079+0000 7fa253fff700 1 -- 172.21.15.100:0/2675898787 learned_addr learned my addr 172.21.15.100:0/2675898787 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:25:41.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.079+0000 7fa25979f700 1 -- 172.21.15.100:0/2675898787 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e5fb627c0 msgr2=0x555e5fb76730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:41.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.079+0000 7fa25979f700 1 --2- 172.21.15.100:0/2675898787 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e5fb627c0 0x555e5fb76730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:41.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.080+0000 7fa25979f700 1 -- 172.21.15.100:0/2675898787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e5fb5d160 msgr2=0x555e5fb761f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:41.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.080+0000 7fa25979f700 1 --2- 172.21.15.100:0/2675898787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e5fb5d160 0x555e5fb761f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:41.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.080+0000 7fa25979f700 1 -- 172.21.15.100:0/2675898787 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e5fbdc0d0 con 0x555e5fb630e0 2024-05-10T11:25:41.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.080+0000 7fa253fff700 1 --2- 172.21.15.100:0/2675898787 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e5fb627c0 0x555e5fb76730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:25:41.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.080+0000 7fa258f9e700 1 --2- 172.21.15.100:0/2675898787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e5fb5d160 0x555e5fb761f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:25:41.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.080+0000 7fa25979f700 1 --2- 172.21.15.100:0/2675898787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e5fb630e0 0x555e5fb76c70 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa24c006070 tx=0x7fa24c004c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0d1cbef5802f23c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:25:41.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.080+0000 7fa233fff700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:25:41.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.080+0000 7fa233fff700 1 -- 172.21.15.100:0/2675898787 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24c0078f0 con 0x555e5fb630e0 2024-05-10T11:25:41.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.080+0000 7fa265a95b80 1 -- 172.21.15.100:0/2675898787 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa24001e880 con 0x555e5fb630e0 2024-05-10T11:25:41.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.080+0000 7fa233fff700 1 -- 172.21.15.100:0/2675898787 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fa24c007a90 con 0x555e5fb630e0 2024-05-10T11:25:41.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.080+0000 7fa233fff700 1 -- 172.21.15.100:0/2675898787 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa24c007c90 con 0x555e5fb630e0 2024-05-10T11:25:41.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.080+0000 7fa265a95b80 10 client.26306 fetch_fsmap learned FSMap version 31 2024-05-10T11:25:41.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.080+0000 7fa265a95b80 1 -- 172.21.15.100:0/2675898787 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x555e5fbdd3f0 con 0x555e5fb630e0 2024-05-10T11:25:41.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.081+0000 7fa233fff700 1 -- 172.21.15.100:0/2675898787 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa24001e880 con 0x555e5fb630e0 2024-05-10T11:25:41.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.081+0000 7fa233fff700 1 -- 172.21.15.100:0/2675898787 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fa24c024070 con 0x555e5fb630e0 2024-05-10T11:25:41.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.081+0000 7fa265a95b80 10 client.26306 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:25:41.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.081+0000 7fa265a95b80 10 client.26306 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:25:41.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.081+0000 7fa265a95b80 20 client.26306 populate_metadata read hostname 'smithi100' 2024-05-10T11:25:41.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.081+0000 7fa265a95b80 1 --2- 172.21.15.100:0/2675898787 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x555e5fbddae0 0x555e5fbdff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:41.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.081+0000 7fa265a95b80 4 client.26306 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T11:25:41.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.081+0000 7fa265a95b80 1 -- 172.21.15.100:0/2675898787 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555e5fab23c0 con 0x555e5fbddae0 2024-05-10T11:25:41.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.081+0000 7fa258f9e700 1 --2- 172.21.15.100:0/2675898787 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x555e5fbddae0 0x555e5fbdff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:41.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.081+0000 7fa233fff700 10 client.26306 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:25:41.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.082+0000 7fa258f9e700 1 --2- 172.21.15.100:0/2675898787 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x555e5fbddae0 0x555e5fbdff00 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43fe476e460f3c23 server_cookie=d264d37de96204b9 in_seq=0 out_seq=0 2024-05-10T11:25:41.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.082+0000 7fa233fff700 1 -- 172.21.15.100:0/2675898787 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x555e5fab23c0 con 0x555e5fbddae0 2024-05-10T11:25:41.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.082+0000 7fa233fff700 10 client.26306 handle_command_reply: tid=0 2024-05-10T11:25:41.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.082+0000 7fa265a95b80 1 client.26306 shutdown 2024-05-10T11:25:41.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.082+0000 7fa265a95b80 1 -- 172.21.15.100:0/2675898787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e5fb630e0 msgr2=0x555e5fb76c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:41.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.082+0000 7fa265a95b80 1 --2- 172.21.15.100:0/2675898787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e5fb630e0 0x555e5fb76c70 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa24c006070 tx=0x7fa24c004c40 comp rx=0 tx=0).stop 2024-05-10T11:25:41.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.082+0000 7fa265a95b80 1 -- 172.21.15.100:0/2675898787 shutdown_connections 2024-05-10T11:25:41.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.082+0000 7fa265a95b80 1 --2- 172.21.15.100:0/2675898787 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x555e5fbddae0 0x555e5fbdff00 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:41.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.082+0000 7fa233fff700 0 client.26306 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:25:41.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.082+0000 7fa265a95b80 1 --2- 172.21.15.100:0/2675898787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e5fb630e0 0x555e5fb76c70 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:41.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.082+0000 7fa265a95b80 1 --2- 172.21.15.100:0/2675898787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e5fb5d160 0x555e5fb761f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:41.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.082+0000 7fa265a95b80 1 --2- 172.21.15.100:0/2675898787 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e5fb627c0 0x555e5fb76730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:41.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.082+0000 7fa265a95b80 1 -- 172.21.15.100:0/2675898787 >> 172.21.15.100:0/2675898787 conn(0x555e5fa84b70 msgr2=0x555e5fb65120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:25:41.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.083+0000 7fa265a95b80 1 -- 172.21.15.100:0/2675898787 shutdown_connections 2024-05-10T11:25:41.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.083+0000 7fa265a95b80 1 -- 172.21.15.100:0/2675898787 wait complete. 2024-05-10T11:25:41.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.083+0000 7fa265a95b80 20 client.26306 trim_cache size 0 max 16384 2024-05-10T11:25:41.087 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:25:41.087 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T11:25:41.087 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-05-10T11:25:41.087 INFO:teuthology.orchestra.run.smithi100.stdout: "6cfe3177-791e-478a-886f-bf8119258da8": { 2024-05-10T11:25:41.087 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-05-10T11:25:41.087 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "6cfe3177-791e-478a-886f-bf8119258da8", 2024-05-10T11:25:41.087 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-05-10T11:25:41.087 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:25:41.087 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:25:41.087 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:25:41.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.083+0000 7fa259fa0700 1 -- 172.21.15.100:0/439901311 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa2440b2900 msgr2=0x7fa2440b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:41.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.083+0000 7fa259fa0700 1 --2- 172.21.15.100:0/439901311 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa2440b2900 0x7fa2440b4dc0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa254109cb0 tx=0x7fa240009260 comp rx=0 tx=0).stop 2024-05-10T11:25:41.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.083+0000 7fa259fa0700 1 -- 172.21.15.100:0/439901311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa254103620 msgr2=0x7fa25419f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:41.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.083+0000 7fa259fa0700 1 --2- 172.21.15.100:0/439901311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa254103620 0x7fa25419f650 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fa24800c920 tx=0x7fa24800cc30 comp rx=0 tx=0).stop 2024-05-10T11:25:41.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.083+0000 7fa259fa0700 1 -- 172.21.15.100:0/439901311 shutdown_connections 2024-05-10T11:25:41.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.083+0000 7fa259fa0700 1 --2- 172.21.15.100:0/439901311 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa2440b2900 0x7fa2440b4dc0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:41.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.083+0000 7fa259fa0700 1 --2- 172.21.15.100:0/439901311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa254108700 0x7fa2541997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:41.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.083+0000 7fa259fa0700 1 --2- 172.21.15.100:0/439901311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa254103620 0x7fa25419f650 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:41.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.083+0000 7fa259fa0700 1 --2- 172.21.15.100:0/439901311 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa254103fd0 0x7fa25419fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:41.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.083+0000 7fa259fa0700 1 -- 172.21.15.100:0/439901311 >> 172.21.15.100:0/439901311 conn(0x7fa2540feef0 msgr2=0x7fa254101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:25:41.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.084+0000 7fa259fa0700 1 -- 172.21.15.100:0/439901311 shutdown_connections 2024-05-10T11:25:41.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:25:41.084+0000 7fa259fa0700 1 -- 172.21.15.100:0/439901311 wait complete. 2024-05-10T11:25:41.706 INFO:tasks.cephfs.filesystem:scrub status for tag:6cfe3177-791e-478a-886f-bf8119258da8 - {'path': '/', 'tag': '6cfe3177-791e-478a-886f-bf8119258da8', 'options': 'recursive,force'} 2024-05-10T11:25:41.974 INFO:tasks.workunit.client.0.smithi100.stdout: 1 173978 2.74 MB/sec execute 216 sec latency 31.931 ms 2024-05-10T11:25:42.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:25:42 smithi178 ceph-mon[33359]: pgmap v639: 129 pgs: 129 active+clean; 77 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 1.2 MiB/s wr, 142 op/s 2024-05-10T11:25:42.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:25:42 smithi188 ceph-mon[34075]: pgmap v639: 129 pgs: 129 active+clean; 77 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 1.2 MiB/s wr, 142 op/s 2024-05-10T11:25:42.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:25:42 smithi100 ceph-mon[30296]: pgmap v639: 129 pgs: 129 active+clean; 77 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 1.2 MiB/s wr, 142 op/s 2024-05-10T11:25:42.974 INFO:tasks.workunit.client.0.smithi100.stdout: 1 174265 2.74 MB/sec execute 217 sec latency 14.142 ms 2024-05-10T11:25:43.974 INFO:tasks.workunit.client.0.smithi100.stdout: 1 174632 2.73 MB/sec execute 218 sec latency 15.140 ms 2024-05-10T11:25:44.378 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:25:44 smithi100 ceph-mon[30296]: pgmap v640: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 1.4 MiB/s wr, 156 op/s 2024-05-10T11:25:44.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:25:44 smithi178 ceph-mon[33359]: pgmap v640: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 1.4 MiB/s wr, 156 op/s 2024-05-10T11:25:44.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:25:44 smithi188 ceph-mon[34075]: pgmap v640: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 1.4 MiB/s wr, 156 op/s 2024-05-10T11:25:44.974 INFO:tasks.workunit.client.0.smithi100.stdout: 1 175205 2.73 MB/sec execute 219 sec latency 17.609 ms 2024-05-10T11:25:45.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:25:45 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:25:45.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:25:45 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:25:45.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:25:45 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:25:45.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:25:45 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:25:45.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:25:45 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:25:45.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:25:45 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:25:45.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:25:45 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:25:45.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:25:45 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:25:45.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:25:45 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:25:45.974 INFO:tasks.workunit.client.0.smithi100.stdout: 1 175778 2.72 MB/sec execute 220 sec latency 16.228 ms 2024-05-10T11:25:46.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:25:46 smithi178 ceph-mon[33359]: pgmap v641: 129 pgs: 129 active+clean; 79 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 1.3 MiB/s wr, 149 op/s 2024-05-10T11:25:46.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:25:46 smithi188 ceph-mon[34075]: pgmap v641: 129 pgs: 129 active+clean; 79 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 1.3 MiB/s wr, 149 op/s 2024-05-10T11:25:46.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:25:46 smithi100 ceph-mon[30296]: pgmap v641: 129 pgs: 129 active+clean; 79 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 1.3 MiB/s wr, 149 op/s 2024-05-10T11:25:46.974 INFO:tasks.workunit.client.0.smithi100.stdout: 1 176328 2.72 MB/sec execute 221 sec latency 15.242 ms 2024-05-10T11:25:47.974 INFO:tasks.workunit.client.0.smithi100.stdout: 1 177111 2.74 MB/sec execute 222 sec latency 15.416 ms 2024-05-10T11:25:48.330 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:25:48 smithi188 ceph-mon[34075]: pgmap v642: 129 pgs: 129 active+clean; 79 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 1.3 MiB/s wr, 132 op/s 2024-05-10T11:25:48.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:25:48 smithi178 ceph-mon[33359]: pgmap v642: 129 pgs: 129 active+clean; 79 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 1.3 MiB/s wr, 132 op/s 2024-05-10T11:25:48.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:25:48 smithi100 ceph-mon[30296]: pgmap v642: 129 pgs: 129 active+clean; 79 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 1.3 MiB/s wr, 132 op/s 2024-05-10T11:25:48.974 INFO:tasks.workunit.client.0.smithi100.stdout: 1 177580 2.75 MB/sec execute 223 sec latency 16.434 ms 2024-05-10T11:25:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.563+0000 7f5fd6b89640 1 -- 172.21.15.100:0/1470430989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fd0102a40 msgr2=0x7f5fd0102e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.563+0000 7f5fd6b89640 1 --2- 172.21.15.100:0/1470430989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fd0102a40 0x7f5fd0102e60 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc009900 tx=0x7f5fcc02a410 comp rx=0 tx=0).stop 2024-05-10T11:25:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.564+0000 7f5fd6b89640 1 -- 172.21.15.100:0/1470430989 shutdown_connections 2024-05-10T11:25:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.564+0000 7f5fd6b89640 1 --2- 172.21.15.100:0/1470430989 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5fd01045f0 0x7f5fd010ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.564+0000 7f5fd6b89640 1 --2- 172.21.15.100:0/1470430989 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fd0103c30 0x7f5fd01040b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.564+0000 7f5fd6b89640 1 --2- 172.21.15.100:0/1470430989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fd0102a40 0x7f5fd0102e60 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.564+0000 7f5fd6b89640 1 -- 172.21.15.100:0/1470430989 >> 172.21.15.100:0/1470430989 conn(0x7f5fd00fdfd0 msgr2=0x7f5fd01003f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:25:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.564+0000 7f5fd6b89640 1 -- 172.21.15.100:0/1470430989 shutdown_connections 2024-05-10T11:25:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.564+0000 7f5fd6b89640 1 -- 172.21.15.100:0/1470430989 wait complete. 2024-05-10T11:25:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.564+0000 7f5fd6b89640 1 Processor -- start 2024-05-10T11:25:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.564+0000 7f5fd6b89640 1 -- start start 2024-05-10T11:25:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.564+0000 7f5fd6b89640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fd0102a40 0x7f5fd0076710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.564+0000 7f5fd6b89640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fd0103c30 0x7f5fd0074da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:49.567 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.564+0000 7f5fd6b89640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5fd01045f0 0x7f5fd00752e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:49.567 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.564+0000 7f5fd6b89640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd010e410 con 0x7f5fd0103c30 2024-05-10T11:25:49.567 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.564+0000 7f5fd6b89640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd010e290 con 0x7f5fd01045f0 2024-05-10T11:25:49.567 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.564+0000 7f5fd6b89640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd010e590 con 0x7f5fd0102a40 2024-05-10T11:25:49.567 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.564+0000 7f5fc7fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fd0103c30 0x7f5fd0074da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:49.567 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.564+0000 7f5fc7fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fd0103c30 0x7f5fd0074da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54510/0 (socket says 172.21.15.100:54510) 2024-05-10T11:25:49.567 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.564+0000 7f5fc7fff640 1 -- 172.21.15.100:0/3597624442 learned_addr learned my addr 172.21.15.100:0/3597624442 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:25:49.567 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.564+0000 7f5fd5126640 1 --2- 172.21.15.100:0/3597624442 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5fd01045f0 0x7f5fd00752e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:49.567 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.564+0000 7f5fd4925640 1 --2- 172.21.15.100:0/3597624442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fd0102a40 0x7f5fd0076710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:49.567 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.565+0000 7f5fc7fff640 1 -- 172.21.15.100:0/3597624442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fd0102a40 msgr2=0x7f5fd0076710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:49.567 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.565+0000 7f5fc7fff640 1 --2- 172.21.15.100:0/3597624442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fd0102a40 0x7f5fd0076710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:49.568 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.565+0000 7f5fc7fff640 1 -- 172.21.15.100:0/3597624442 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5fd01045f0 msgr2=0x7f5fd00752e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:49.568 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.565+0000 7f5fc7fff640 1 --2- 172.21.15.100:0/3597624442 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5fd01045f0 0x7f5fd00752e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:49.568 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.565+0000 7f5fc7fff640 1 -- 172.21.15.100:0/3597624442 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fd0075ba0 con 0x7f5fd0103c30 2024-05-10T11:25:49.568 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.565+0000 7f5fc7fff640 1 --2- 172.21.15.100:0/3597624442 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fd0103c30 0x7f5fd0074da0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f5fc000e9f0 tx=0x7f5fc000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:25:49.568 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.565+0000 7f5fc5ffb640 1 -- 172.21.15.100:0/3597624442 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc000cd80 con 0x7f5fd0103c30 2024-05-10T11:25:49.568 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.565+0000 7f5fc5ffb640 1 -- 172.21.15.100:0/3597624442 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5fc00054b0 con 0x7f5fd0103c30 2024-05-10T11:25:49.568 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.565+0000 7f5fc5ffb640 1 -- 172.21.15.100:0/3597624442 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc0010430 con 0x7f5fd0103c30 2024-05-10T11:25:49.568 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.565+0000 7f5fd6b89640 1 -- 172.21.15.100:0/3597624442 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fd01a2c90 con 0x7f5fd0103c30 2024-05-10T11:25:49.568 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.565+0000 7f5fd6b89640 1 -- 172.21.15.100:0/3597624442 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fd01a3160 con 0x7f5fd0103c30 2024-05-10T11:25:49.576 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.567+0000 7f5fc5ffb640 1 -- 172.21.15.100:0/3597624442 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f5fc00046c0 con 0x7f5fd0103c30 2024-05-10T11:25:49.576 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.568+0000 7f5fd9979740 1 Processor -- start 2024-05-10T11:25:49.576 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.568+0000 7f5fd9979740 1 -- start start 2024-05-10T11:25:49.576 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.568+0000 7f5fd9979740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5582ef249db0 0x5582ef24a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:49.576 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.568+0000 7f5fd9979740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5582ef24a7a0 0x5582ef24f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:49.576 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.568+0000 7f5fd9979740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582ef24f910 0x5582ef251da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:49.576 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.568+0000 7f5fd9979740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5582ef1ae5d0 con 0x5582ef24a7a0 2024-05-10T11:25:49.577 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.568+0000 7f5fd9979740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5582ef167890 con 0x5582ef24f910 2024-05-10T11:25:49.577 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.568+0000 7f5fd9979740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5582ef1aa590 con 0x5582ef249db0 2024-05-10T11:25:49.577 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.568+0000 7f5fd4925640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5582ef24a7a0 0x5582ef24f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:49.577 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.568+0000 7f5fd5126640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5582ef249db0 0x5582ef24a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:49.577 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.568+0000 7f5fd4925640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5582ef24a7a0 0x5582ef24f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54522/0 (socket says 172.21.15.100:54522) 2024-05-10T11:25:49.577 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.568+0000 7f5fc7fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582ef24f910 0x5582ef251da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:49.577 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.568+0000 7f5fd4925640 1 -- 172.21.15.100:0/139880129 learned_addr learned my addr 172.21.15.100:0/139880129 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:25:49.577 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.568+0000 7f5fd5126640 1 -- 172.21.15.100:0/139880129 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582ef24f910 msgr2=0x5582ef251da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:49.577 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.568+0000 7f5fd5126640 1 --2- 172.21.15.100:0/139880129 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582ef24f910 0x5582ef251da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:49.577 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.568+0000 7f5fc5ffb640 1 --2- 172.21.15.100:0/3597624442 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f5fb80b28a0 0x7f5fb80b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:49.577 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.568+0000 7f5fd5126640 1 -- 172.21.15.100:0/139880129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5582ef24a7a0 msgr2=0x5582ef24f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:49.577 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.569+0000 7f5fd5126640 1 --2- 172.21.15.100:0/139880129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5582ef24a7a0 0x5582ef24f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:49.577 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.569+0000 7f5fd5126640 1 -- 172.21.15.100:0/139880129 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582eeec0210 con 0x5582ef249db0 2024-05-10T11:25:49.577 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.569+0000 7f5fd4925640 1 --2- 172.21.15.100:0/139880129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5582ef24a7a0 0x5582ef24f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:25:49.577 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.569+0000 7f5fc7fff640 1 --2- 172.21.15.100:0/3597624442 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f5fb80b28a0 0x7f5fb80b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:49.578 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.569+0000 7f5fd5126640 1 --2- 172.21.15.100:0/139880129 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5582ef249db0 0x5582ef24a1d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5fc8009f00 tx=0x7f5fc8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2791d4200fb45d23 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:25:49.578 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.569+0000 7f5fc5ffb640 1 -- 172.21.15.100:0/3597624442 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f5fc00e28b0 con 0x7f5fd0103c30 2024-05-10T11:25:49.578 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.569+0000 7f5fb6ffd640 1 -- 172.21.15.100:0/139880129 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc8004d20 con 0x5582ef249db0 2024-05-10T11:25:49.580 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.569+0000 7f5fb6ffd640 1 -- 172.21.15.100:0/139880129 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5fc8004030 con 0x5582ef249db0 2024-05-10T11:25:49.580 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.569+0000 7f5fb6ffd640 1 -- 172.21.15.100:0/139880129 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc8005280 con 0x5582ef249db0 2024-05-10T11:25:49.580 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.570+0000 7f5fd9979740 1 -- 172.21.15.100:0/139880129 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5582ef249db0 msgr2=0x5582ef24a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:49.580 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.570+0000 7f5fd9979740 1 --2- 172.21.15.100:0/139880129 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5582ef249db0 0x5582ef24a1d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5fc8009f00 tx=0x7f5fc8011810 comp rx=0 tx=0).stop 2024-05-10T11:25:49.580 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.570+0000 7f5fc7fff640 1 --2- 172.21.15.100:0/3597624442 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f5fb80b28a0 0x7f5fb80b4d60 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5fc00a9870 tx=0x7f5fc00e8550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:25:49.580 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.570+0000 7f5fd9979740 1 -- 172.21.15.100:0/139880129 shutdown_connections 2024-05-10T11:25:49.580 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.570+0000 7f5fd9979740 1 --2- 172.21.15.100:0/139880129 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582ef24f910 0x5582ef251da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:49.580 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.570+0000 7f5fd9979740 1 --2- 172.21.15.100:0/139880129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5582ef24a7a0 0x5582ef24f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:49.580 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.570+0000 7f5fd9979740 1 --2- 172.21.15.100:0/139880129 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5582ef249db0 0x5582ef24a1d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:49.580 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.570+0000 7f5fd9979740 1 -- 172.21.15.100:0/139880129 >> 172.21.15.100:0/139880129 conn(0x5582ef245420 msgr2=0x5582ef245820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:25:49.581 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.570+0000 7f5fd9979740 1 -- 172.21.15.100:0/139880129 shutdown_connections 2024-05-10T11:25:49.581 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.570+0000 7f5fd9979740 1 -- 172.21.15.100:0/139880129 wait complete. 2024-05-10T11:25:49.581 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.570+0000 7f5fd9979740 1 -- start start 2024-05-10T11:25:49.581 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.570+0000 7f5fd9979740 1 Processor -- start 2024-05-10T11:25:49.581 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.571+0000 7f5fd9979740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5582ef249db0 0x5582ef253f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:49.581 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.571+0000 7f5fd9979740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582ef24a7a0 0x5582ef2544d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:49.581 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.571+0000 7f5fd9979740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5582ef24f910 0x5582ef25b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:49.581 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.571+0000 7f5fd9979740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5582ef2528c0 con 0x5582ef24f910 2024-05-10T11:25:49.581 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.571+0000 7f5fd9979740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5582ef16b370 con 0x5582ef24a7a0 2024-05-10T11:25:49.581 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.571+0000 7f5fd9979740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5582ef252a40 con 0x5582ef249db0 2024-05-10T11:25:49.581 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.571+0000 7f5fd9979740 10 client.0 _ll_register_callbacks cb 0x5582eeebfa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:25:49.581 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.571+0000 7f5fd5126640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5582ef24f910 0x5582ef25b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:49.581 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.571+0000 7f5fd4925640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582ef24a7a0 0x5582ef2544d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:49.581 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.571+0000 7f5fd5126640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5582ef24f910 0x5582ef25b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54530/0 (socket says 172.21.15.100:54530) 2024-05-10T11:25:49.582 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.571+0000 7f5fd5126640 1 -- 172.21.15.100:0/2975713531 learned_addr learned my addr 172.21.15.100:0/2975713531 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:25:49.582 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.571+0000 7f5fd5126640 1 --2- 172.21.15.100:0/2975713531 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5582ef249db0 0x5582ef253f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:49.582 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.571+0000 7f5fd4925640 1 -- 172.21.15.100:0/2975713531 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5582ef249db0 msgr2=0x5582ef253f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:49.582 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.571+0000 7f5fd4925640 1 --2- 172.21.15.100:0/2975713531 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5582ef249db0 0x5582ef253f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:49.582 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.571+0000 7f5fd4925640 1 -- 172.21.15.100:0/2975713531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5582ef24f910 msgr2=0x5582ef25b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:49.582 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.571+0000 7f5fd4925640 1 --2- 172.21.15.100:0/2975713531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5582ef24f910 0x5582ef25b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:49.582 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.571+0000 7f5fd4925640 1 -- 172.21.15.100:0/2975713531 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582eef8c620 con 0x5582ef24a7a0 2024-05-10T11:25:49.582 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.571+0000 7f5fd4925640 1 --2- 172.21.15.100:0/2975713531 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582ef24a7a0 0x5582ef2544d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x5582ef24b210 tx=0x7f5fcc031870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=673936759d699097 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:25:49.582 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.571+0000 7f5faffff640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:25:49.582 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.571+0000 7f5faffff640 1 -- 172.21.15.100:0/2975713531 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc035040 con 0x5582ef24a7a0 2024-05-10T11:25:49.582 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.572+0000 7f5fd9979740 1 -- 172.21.15.100:0/2975713531 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fc8004030 con 0x5582ef24a7a0 2024-05-10T11:25:49.582 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.572+0000 7f5faffff640 1 -- 172.21.15.100:0/2975713531 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f5fcc004240 con 0x5582ef24a7a0 2024-05-10T11:25:49.582 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.572+0000 7f5faffff640 1 -- 172.21.15.100:0/2975713531 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5fcc04e050 con 0x5582ef24a7a0 2024-05-10T11:25:49.582 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.572+0000 7f5fd9979740 10 client.26318 fetch_fsmap learned FSMap version 31 2024-05-10T11:25:49.583 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.572+0000 7f5fd9979740 1 -- 172.21.15.100:0/2975713531 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5582ef25e560 con 0x5582ef24a7a0 2024-05-10T11:25:49.583 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.572+0000 7f5faffff640 1 -- 172.21.15.100:0/2975713531 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5fc8004030 con 0x5582ef24a7a0 2024-05-10T11:25:49.583 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.572+0000 7f5faffff640 1 -- 172.21.15.100:0/2975713531 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f5fcc03c440 con 0x5582ef24a7a0 2024-05-10T11:25:49.583 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.572+0000 7f5fd9979740 10 client.26318 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:25:49.583 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.572+0000 7f5fd9979740 10 client.26318 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:25:49.583 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.572+0000 7f5fd9979740 20 client.26318 populate_metadata read hostname 'smithi100' 2024-05-10T11:25:49.583 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.572+0000 7f5fd9979740 1 --2- 172.21.15.100:0/2975713531 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5582ef25fc90 0x5582ef262080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:49.583 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.572+0000 7f5fd9979740 4 client.26318 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:25:49.583 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.572+0000 7f5fd9979740 1 -- 172.21.15.100:0/2975713531 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582ef1bc1e0 con 0x5582ef25fc90 2024-05-10T11:25:49.583 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.572+0000 7f5fd5126640 1 --2- 172.21.15.100:0/2975713531 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5582ef25fc90 0x5582ef262080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:49.583 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.573+0000 7f5faffff640 10 client.26318 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:25:49.583 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.573+0000 7f5fd5126640 1 --2- 172.21.15.100:0/2975713531 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5582ef25fc90 0x5582ef262080 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d50bb9ae958442e server_cookie=731d4f0945ee80d in_seq=0 out_seq=0 2024-05-10T11:25:49.583 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.573+0000 7f5faffff640 1 -- 172.21.15.100:0/2975713531 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5582ef1bc1e0 con 0x5582ef25fc90 2024-05-10T11:25:49.583 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.573+0000 7f5faffff640 10 client.26318 handle_command_reply: tid=0 2024-05-10T11:25:49.583 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.573+0000 7f5fd9979740 1 client.26318 shutdown 2024-05-10T11:25:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.573+0000 7f5fd9979740 1 -- 172.21.15.100:0/2975713531 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582ef24a7a0 msgr2=0x5582ef2544d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.573+0000 7f5fd9979740 1 --2- 172.21.15.100:0/2975713531 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582ef24a7a0 0x5582ef2544d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x5582ef24b210 tx=0x7f5fcc031870 comp rx=0 tx=0).stop 2024-05-10T11:25:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.573+0000 7f5fd9979740 1 -- 172.21.15.100:0/2975713531 shutdown_connections 2024-05-10T11:25:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.573+0000 7f5fd9979740 1 --2- 172.21.15.100:0/2975713531 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5582ef25fc90 0x5582ef262080 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.573+0000 7f5fd9979740 1 --2- 172.21.15.100:0/2975713531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5582ef24f910 0x5582ef25b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.573+0000 7f5fd9979740 1 --2- 172.21.15.100:0/2975713531 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582ef24a7a0 0x5582ef2544d0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.573+0000 7f5fd9979740 1 --2- 172.21.15.100:0/2975713531 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5582ef249db0 0x5582ef253f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.573+0000 7f5faffff640 0 client.26318 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:25:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.573+0000 7f5fd9979740 1 -- 172.21.15.100:0/2975713531 >> 172.21.15.100:0/2975713531 conn(0x5582ef245420 msgr2=0x5582ef244580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:25:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.574+0000 7f5fd9979740 1 -- 172.21.15.100:0/2975713531 shutdown_connections 2024-05-10T11:25:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.574+0000 7f5fd9979740 1 -- 172.21.15.100:0/2975713531 wait complete. 2024-05-10T11:25:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.574+0000 7f5fd9979740 20 client.26318 trim_cache size 0 max 16384 2024-05-10T11:25:49.594 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.590+0000 7f5fd9979740 1 Processor -- start 2024-05-10T11:25:49.594 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.590+0000 7f5fd9979740 1 -- start start 2024-05-10T11:25:49.594 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.590+0000 7f5fd9979740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5582ef249db0 0x5582ef248810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:49.594 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.590+0000 7f5fd9979740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582ef24a7a0 0x5582ef25b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:49.594 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.590+0000 7f5fd9979740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5582ef24f910 0x5582ef266960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:49.594 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.590+0000 7f5fd9979740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd01afdc0 con 0x5582ef249db0 2024-05-10T11:25:49.594 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.590+0000 7f5fd9979740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd01ad690 con 0x5582ef24a7a0 2024-05-10T11:25:49.595 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.590+0000 7f5fd9979740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd01ab4e0 con 0x5582ef24f910 2024-05-10T11:25:49.595 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.590+0000 7f5fd5126640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5582ef249db0 0x5582ef248810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:49.595 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.590+0000 7f5fd5126640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5582ef249db0 0x5582ef248810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54534/0 (socket says 172.21.15.100:54534) 2024-05-10T11:25:49.595 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.590+0000 7f5fd5126640 1 -- 172.21.15.100:0/2232774090 learned_addr learned my addr 172.21.15.100:0/2232774090 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:25:49.595 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.590+0000 7f5fd5126640 1 --2- 172.21.15.100:0/2232774090 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5582ef24f910 0x5582ef266960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:49.595 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.590+0000 7f5fd4925640 1 --2- 172.21.15.100:0/2232774090 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582ef24a7a0 0x5582ef25b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:49.595 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.590+0000 7f5fd5126640 1 -- 172.21.15.100:0/2232774090 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5582ef24f910 msgr2=0x5582ef266960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:49.595 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.590+0000 7f5fd5126640 1 --2- 172.21.15.100:0/2232774090 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5582ef24f910 0x5582ef266960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:49.595 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.590+0000 7f5fd5126640 1 -- 172.21.15.100:0/2232774090 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582ef24a7a0 msgr2=0x5582ef25b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:49.595 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.590+0000 7f5fd5126640 1 --2- 172.21.15.100:0/2232774090 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582ef24a7a0 0x5582ef25b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:49.595 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.590+0000 7f5fd5126640 1 -- 172.21.15.100:0/2232774090 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582ef2533b0 con 0x5582ef249db0 2024-05-10T11:25:49.595 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.590+0000 7f5fd4925640 1 --2- 172.21.15.100:0/2232774090 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582ef24a7a0 0x5582ef25b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:25:49.595 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.591+0000 7f5fd5126640 1 --2- 172.21.15.100:0/2232774090 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5582ef249db0 0x5582ef248810 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f5fc8014040 tx=0x7f5fc8009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78557264823e423a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:25:49.595 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.591+0000 7f5fb67fc640 1 -- 172.21.15.100:0/2232774090 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc801e040 con 0x5582ef249db0 2024-05-10T11:25:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.591+0000 7f5fb67fc640 1 -- 172.21.15.100:0/2232774090 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5fc8003ec0 con 0x5582ef249db0 2024-05-10T11:25:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.591+0000 7f5fb67fc640 1 -- 172.21.15.100:0/2232774090 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc800f9e0 con 0x5582ef249db0 2024-05-10T11:25:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.591+0000 7f5fd9979740 1 -- 172.21.15.100:0/2232774090 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5582ef249db0 msgr2=0x5582ef248810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.591+0000 7f5fd9979740 1 --2- 172.21.15.100:0/2232774090 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5582ef249db0 0x5582ef248810 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f5fc8014040 tx=0x7f5fc8009500 comp rx=0 tx=0).stop 2024-05-10T11:25:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.591+0000 7f5fd9979740 1 -- 172.21.15.100:0/2232774090 shutdown_connections 2024-05-10T11:25:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.591+0000 7f5fd9979740 1 --2- 172.21.15.100:0/2232774090 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5582ef24f910 0x5582ef266960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.591+0000 7f5fd9979740 1 --2- 172.21.15.100:0/2232774090 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582ef24a7a0 0x5582ef25b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.591+0000 7f5fd9979740 1 --2- 172.21.15.100:0/2232774090 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5582ef249db0 0x5582ef248810 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.591+0000 7f5fd9979740 1 -- 172.21.15.100:0/2232774090 >> 172.21.15.100:0/2232774090 conn(0x5582ef245420 msgr2=0x5582ef248100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:25:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.592+0000 7f5fd9979740 1 -- 172.21.15.100:0/2232774090 shutdown_connections 2024-05-10T11:25:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.592+0000 7f5fd9979740 1 -- 172.21.15.100:0/2232774090 wait complete. 2024-05-10T11:25:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.592+0000 7f5fd9979740 1 -- start start 2024-05-10T11:25:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.592+0000 7f5fd9979740 1 Processor -- start 2024-05-10T11:25:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.592+0000 7f5fd9979740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582ef249db0 0x5582ef258f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.592+0000 7f5fd9979740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5582ef24a7a0 0x5582ef2594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.592+0000 7f5fd9979740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5582ef24f910 0x5582ef25a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.592+0000 7f5fd9979740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd01abb50 con 0x5582ef24f910 2024-05-10T11:25:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.592+0000 7f5fd9979740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd01acfc0 con 0x5582ef249db0 2024-05-10T11:25:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.592+0000 7f5fd9979740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd01ac220 con 0x5582ef24a7a0 2024-05-10T11:25:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.592+0000 7f5fd9979740 10 client.0 _ll_register_callbacks cb 0x7f5fcc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:25:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.592+0000 7f5fd5126640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5582ef24f910 0x5582ef25a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.592+0000 7f5fd4925640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5582ef24a7a0 0x5582ef2594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.592+0000 7f5fd4925640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5582ef24a7a0 0x5582ef2594a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:53380/0 (socket says 172.21.15.100:53380) 2024-05-10T11:25:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.592+0000 7f5fd4925640 1 -- 172.21.15.100:0/2329370373 learned_addr learned my addr 172.21.15.100:0/2329370373 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:25:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.592+0000 7f5fd5126640 1 --2- 172.21.15.100:0/2329370373 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582ef249db0 0x5582ef258f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.593+0000 7f5fd4925640 1 -- 172.21.15.100:0/2329370373 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582ef249db0 msgr2=0x5582ef258f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.593+0000 7f5fd4925640 1 --2- 172.21.15.100:0/2329370373 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582ef249db0 0x5582ef258f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.593+0000 7f5fd4925640 1 -- 172.21.15.100:0/2329370373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5582ef24f910 msgr2=0x5582ef25a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:25:49.598 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.593+0000 7f5fd4925640 1 --2- 172.21.15.100:0/2329370373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5582ef24f910 0x5582ef25a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:49.598 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.593+0000 7f5fd4925640 1 -- 172.21.15.100:0/2329370373 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582ef25aa30 con 0x5582ef24a7a0 2024-05-10T11:25:49.598 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.593+0000 7f5fd4925640 1 --2- 172.21.15.100:0/2329370373 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5582ef24a7a0 0x5582ef2594a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc031a40 tx=0x7f5fcc03cd00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e6ab77e26c08ffc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:25:49.598 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.593+0000 7f5faffff640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:25:49.598 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.593+0000 7f5faffff640 1 -- 172.21.15.100:0/2329370373 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc057060 con 0x5582ef24a7a0 2024-05-10T11:25:49.598 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.593+0000 7f5faffff640 1 -- 172.21.15.100:0/2329370373 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f5fcc004240 con 0x5582ef24a7a0 2024-05-10T11:25:49.598 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.593+0000 7f5fd9979740 1 -- 172.21.15.100:0/2329370373 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fc8003ec0 con 0x5582ef24a7a0 2024-05-10T11:25:49.598 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.593+0000 7f5faffff640 1 -- 172.21.15.100:0/2329370373 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5fcc05e050 con 0x5582ef24a7a0 2024-05-10T11:25:49.598 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.593+0000 7f5fd9979740 10 client.26362 fetch_fsmap learned FSMap version 31 2024-05-10T11:25:49.598 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.593+0000 7f5fd9979740 1 -- 172.21.15.100:0/2329370373 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5582ef2ca800 con 0x5582ef24a7a0 2024-05-10T11:25:49.598 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.593+0000 7f5faffff640 1 -- 172.21.15.100:0/2329370373 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5fc8003ec0 con 0x5582ef24a7a0 2024-05-10T11:25:49.598 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.594+0000 7f5faffff640 1 -- 172.21.15.100:0/2329370373 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f5fcc003a80 con 0x5582ef24a7a0 2024-05-10T11:25:49.598 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.594+0000 7f5fd9979740 10 client.26362 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:25:49.598 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.594+0000 7f5fd9979740 10 client.26362 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:25:49.598 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.594+0000 7f5fd9979740 20 client.26362 populate_metadata read hostname 'smithi100' 2024-05-10T11:25:49.599 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.594+0000 7f5fd9979740 1 --2- 172.21.15.100:0/2329370373 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5582ef2cb230 0x5582ef2cd650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:25:49.599 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.594+0000 7f5fd9979740 4 client.26362 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:25:49.599 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.594+0000 7f5fd9979740 1 -- 172.21.15.100:0/2329370373 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5582ef1b6050 con 0x5582ef2cb230 2024-05-10T11:25:49.599 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.594+0000 7f5fd5126640 1 --2- 172.21.15.100:0/2329370373 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5582ef2cb230 0x5582ef2cd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:25:49.599 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.594+0000 7f5faffff640 10 client.26362 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:25:49.599 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.594+0000 7f5fd5126640 1 --2- 172.21.15.100:0/2329370373 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5582ef2cb230 0x5582ef2cd650 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1e0e170fe61c7e8 server_cookie=b2061eee179b7e98 in_seq=0 out_seq=0 2024-05-10T11:25:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.619+0000 7f5faffff640 1 -- 172.21.15.100:0/2329370373 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5582ef1b6050 con 0x5582ef2cb230 2024-05-10T11:25:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.619+0000 7f5faffff640 10 client.26362 handle_command_reply: tid=0 2024-05-10T11:25:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.619+0000 7f5fd9979740 1 client.26362 shutdown 2024-05-10T11:25:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.619+0000 7f5fd9979740 1 -- 172.21.15.100:0/2329370373 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5582ef24a7a0 msgr2=0x5582ef2594a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.619+0000 7f5fd9979740 1 --2- 172.21.15.100:0/2329370373 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5582ef24a7a0 0x5582ef2594a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc031a40 tx=0x7f5fcc03cd00 comp rx=0 tx=0).stop 2024-05-10T11:25:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.619+0000 7f5fd9979740 1 -- 172.21.15.100:0/2329370373 shutdown_connections 2024-05-10T11:25:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.619+0000 7f5fd9979740 1 --2- 172.21.15.100:0/2329370373 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5582ef2cb230 0x5582ef2cd650 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.619+0000 7f5fd9979740 1 --2- 172.21.15.100:0/2329370373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5582ef24f910 0x5582ef25a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.619+0000 7f5fd9979740 1 --2- 172.21.15.100:0/2329370373 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5582ef24a7a0 0x5582ef2594a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.619+0000 7f5fd9979740 1 --2- 172.21.15.100:0/2329370373 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582ef249db0 0x5582ef258f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.619+0000 7f5faffff640 0 client.26362 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:25:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.619+0000 7f5fd9979740 1 -- 172.21.15.100:0/2329370373 >> 172.21.15.100:0/2329370373 conn(0x5582ef245420 msgr2=0x5582ef243cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:25:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.619+0000 7f5fd9979740 1 -- 172.21.15.100:0/2329370373 shutdown_connections 2024-05-10T11:25:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.619+0000 7f5fd9979740 1 -- 172.21.15.100:0/2329370373 wait complete. 2024-05-10T11:25:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.619+0000 7f5fd9979740 20 client.26362 trim_cache size 0 max 16384 2024-05-10T11:25:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:25:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:25:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:25:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:25:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.620+0000 7f5fd6b89640 1 -- 172.21.15.100:0/3597624442 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f5fb80b28a0 msgr2=0x7f5fb80b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.620+0000 7f5fd6b89640 1 --2- 172.21.15.100:0/3597624442 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f5fb80b28a0 0x7f5fb80b4d60 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5fc00a9870 tx=0x7f5fc00e8550 comp rx=0 tx=0).stop 2024-05-10T11:25:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.620+0000 7f5fd6b89640 1 -- 172.21.15.100:0/3597624442 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fd0103c30 msgr2=0x7f5fd0074da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:25:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.620+0000 7f5fd6b89640 1 --2- 172.21.15.100:0/3597624442 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fd0103c30 0x7f5fd0074da0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f5fc000e9f0 tx=0x7f5fc000edf0 comp rx=0 tx=0).stop 2024-05-10T11:25:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.620+0000 7f5fd6b89640 1 -- 172.21.15.100:0/3597624442 shutdown_connections 2024-05-10T11:25:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.620+0000 7f5fd6b89640 1 --2- 172.21.15.100:0/3597624442 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f5fb80b28a0 0x7f5fb80b4d60 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.620+0000 7f5fd6b89640 1 --2- 172.21.15.100:0/3597624442 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5fd01045f0 0x7f5fd00752e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.620+0000 7f5fd6b89640 1 --2- 172.21.15.100:0/3597624442 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fd0103c30 0x7f5fd0074da0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.620+0000 7f5fd6b89640 1 --2- 172.21.15.100:0/3597624442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fd0102a40 0x7f5fd0076710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:25:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.620+0000 7f5fd6b89640 1 -- 172.21.15.100:0/3597624442 >> 172.21.15.100:0/3597624442 conn(0x7f5fd00fdfd0 msgr2=0x7f5fd0107050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:25:49.623 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.620+0000 7f5fd6b89640 1 -- 172.21.15.100:0/3597624442 shutdown_connections 2024-05-10T11:25:49.623 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:25:49.620+0000 7f5fd6b89640 1 -- 172.21.15.100:0/3597624442 wait complete. 2024-05-10T11:25:49.974 INFO:tasks.workunit.client.0.smithi100.stdout: 1 177833 2.74 MB/sec execute 224 sec latency 13.611 ms 2024-05-10T11:25:50.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:25:50 smithi178 ceph-mon[33359]: pgmap v643: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 1.3 MiB/s wr, 176 op/s 2024-05-10T11:25:50.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:25:50 smithi188 ceph-mon[34075]: pgmap v643: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 1.3 MiB/s wr, 176 op/s 2024-05-10T11:25:50.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:25:50 smithi100 ceph-mon[30296]: pgmap v643: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 1.3 MiB/s wr, 176 op/s 2024-05-10T11:25:50.974 INFO:tasks.workunit.client.0.smithi100.stdout: 1 178252 2.73 MB/sec execute 225 sec latency 15.401 ms 2024-05-10T11:25:51.975 INFO:tasks.workunit.client.0.smithi100.stdout: 1 178838 2.73 MB/sec execute 226 sec latency 15.460 ms 2024-05-10T11:25:52.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:25:52 smithi178 ceph-mon[33359]: pgmap v644: 129 pgs: 129 active+clean; 79 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 1.1 MiB/s wr, 135 op/s 2024-05-10T11:25:52.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:25:52 smithi188 ceph-mon[34075]: pgmap v644: 129 pgs: 129 active+clean; 79 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 1.1 MiB/s wr, 135 op/s 2024-05-10T11:25:52.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:25:52 smithi100 ceph-mon[30296]: pgmap v644: 129 pgs: 129 active+clean; 79 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 1.1 MiB/s wr, 135 op/s 2024-05-10T11:25:52.974 INFO:tasks.workunit.client.0.smithi100.stdout: 1 179465 2.72 MB/sec execute 227 sec latency 15.578 ms 2024-05-10T11:25:53.974 INFO:tasks.workunit.client.0.smithi100.stdout: 1 180082 2.74 MB/sec execute 228 sec latency 17.592 ms 2024-05-10T11:25:54.444 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:25:54 smithi100 ceph-mon[30296]: pgmap v645: 129 pgs: 129 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 1.3 MiB/s wr, 133 op/s 2024-05-10T11:25:54.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:25:54 smithi178 ceph-mon[33359]: pgmap v645: 129 pgs: 129 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 1.3 MiB/s wr, 133 op/s 2024-05-10T11:25:54.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:25:54 smithi188 ceph-mon[34075]: pgmap v645: 129 pgs: 129 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 1.3 MiB/s wr, 133 op/s 2024-05-10T11:25:54.974 INFO:tasks.workunit.client.0.smithi100.stdout: 1 180696 2.74 MB/sec execute 229 sec latency 15.753 ms 2024-05-10T11:25:55.975 INFO:tasks.workunit.client.0.smithi100.stdout: 1 181198 2.75 MB/sec execute 230 sec latency 14.810 ms 2024-05-10T11:25:56.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:25:56 smithi178 ceph-mon[33359]: pgmap v646: 129 pgs: 129 active+clean; 81 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 1.3 MiB/s wr, 161 op/s 2024-05-10T11:25:56.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:25:56 smithi188 ceph-mon[34075]: pgmap v646: 129 pgs: 129 active+clean; 81 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 1.3 MiB/s wr, 161 op/s 2024-05-10T11:25:56.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:25:56 smithi100 ceph-mon[30296]: pgmap v646: 129 pgs: 129 active+clean; 81 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 1.3 MiB/s wr, 161 op/s 2024-05-10T11:25:56.975 INFO:tasks.workunit.client.0.smithi100.stdout: 1 181469 2.74 MB/sec execute 231 sec latency 17.170 ms 2024-05-10T11:25:57.975 INFO:tasks.workunit.client.0.smithi100.stdout: 1 181962 2.73 MB/sec execute 232 sec latency 15.942 ms 2024-05-10T11:25:58.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:25:58 smithi178 ceph-mon[33359]: pgmap v647: 129 pgs: 129 active+clean; 81 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 1.4 MiB/s wr, 145 op/s 2024-05-10T11:25:58.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:25:58 smithi188 ceph-mon[34075]: pgmap v647: 129 pgs: 129 active+clean; 81 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 1.4 MiB/s wr, 145 op/s 2024-05-10T11:25:58.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:25:58 smithi100 ceph-mon[30296]: pgmap v647: 129 pgs: 129 active+clean; 81 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 1.4 MiB/s wr, 145 op/s 2024-05-10T11:25:58.975 INFO:tasks.workunit.client.0.smithi100.stdout: 1 182556 2.73 MB/sec execute 233 sec latency 15.361 ms 2024-05-10T11:25:59.975 INFO:tasks.workunit.client.0.smithi100.stdout: 1 183272 2.73 MB/sec execute 234 sec latency 14.525 ms 2024-05-10T11:26:00.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:26:00 smithi178 ceph-mon[33359]: pgmap v648: 129 pgs: 129 active+clean; 83 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 1.6 MiB/s wr, 190 op/s 2024-05-10T11:26:00.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:26:00 smithi188 ceph-mon[34075]: pgmap v648: 129 pgs: 129 active+clean; 83 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 1.6 MiB/s wr, 190 op/s 2024-05-10T11:26:00.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:26:00 smithi100 ceph-mon[30296]: pgmap v648: 129 pgs: 129 active+clean; 83 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 1.6 MiB/s wr, 190 op/s 2024-05-10T11:26:00.975 INFO:tasks.workunit.client.0.smithi100.stdout: 1 183900 2.74 MB/sec execute 235 sec latency 15.570 ms 2024-05-10T11:26:01.975 INFO:tasks.workunit.client.0.smithi100.stdout: 1 184457 2.75 MB/sec execute 236 sec latency 16.298 ms 2024-05-10T11:26:02.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:26:02 smithi178 ceph-mon[33359]: pgmap v649: 129 pgs: 129 active+clean; 81 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 1.4 MiB/s wr, 142 op/s 2024-05-10T11:26:02.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:26:02 smithi188 ceph-mon[34075]: pgmap v649: 129 pgs: 129 active+clean; 81 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 1.4 MiB/s wr, 142 op/s 2024-05-10T11:26:02.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:26:02 smithi100 ceph-mon[30296]: pgmap v649: 129 pgs: 129 active+clean; 81 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 1.4 MiB/s wr, 142 op/s 2024-05-10T11:26:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.714+0000 7f25fc39d640 1 -- 172.21.15.100:0/2303141251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25f4102b00 msgr2=0x7f25f4102f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.714+0000 7f25fc39d640 1 --2- 172.21.15.100:0/2303141251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25f4102b00 0x7f25f4102f80 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f25e4009900 tx=0x7f25e402a410 comp rx=0 tx=0).stop 2024-05-10T11:26:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.714+0000 7f25fc39d640 1 -- 172.21.15.100:0/2303141251 shutdown_connections 2024-05-10T11:26:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.714+0000 7f25fc39d640 1 --2- 172.21.15.100:0/2303141251 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25f41034c0 0x7f25f4109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.714+0000 7f25fc39d640 1 --2- 172.21.15.100:0/2303141251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25f4102b00 0x7f25f4102f80 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.714+0000 7f25fc39d640 1 --2- 172.21.15.100:0/2303141251 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25f4101910 0x7f25f4101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.714+0000 7f25fc39d640 1 -- 172.21.15.100:0/2303141251 >> 172.21.15.100:0/2303141251 conn(0x7f25f40fce80 msgr2=0x7f25f40ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:26:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.714+0000 7f25fc39d640 1 -- 172.21.15.100:0/2303141251 shutdown_connections 2024-05-10T11:26:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.714+0000 7f25fc39d640 1 -- 172.21.15.100:0/2303141251 wait complete. 2024-05-10T11:26:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.714+0000 7f25fc39d640 1 Processor -- start 2024-05-10T11:26:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.715+0000 7f25fc39d640 1 -- start start 2024-05-10T11:26:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.715+0000 7f25fc39d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25f4101910 0x7f25f4197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.715+0000 7f25fc39d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25f4102b00 0x7f25f4197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.715+0000 7f25fc39d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25f41034c0 0x7f25f419ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.715+0000 7f25fc39d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f25f410d2f0 con 0x7f25f41034c0 2024-05-10T11:26:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.715+0000 7f25fc39d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f25f410d170 con 0x7f25f4101910 2024-05-10T11:26:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.715+0000 7f25fc39d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f25f410d470 con 0x7f25f4102b00 2024-05-10T11:26:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.715+0000 7f25fa93a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25f41034c0 0x7f25f419ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.715+0000 7f25fa139640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25f4101910 0x7f25f4197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.715+0000 7f25f9938640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25f4102b00 0x7f25f4197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.715+0000 7f25fa93a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25f41034c0 0x7f25f419ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46428/0 (socket says 172.21.15.100:46428) 2024-05-10T11:26:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.715+0000 7f25fa93a640 1 -- 172.21.15.100:0/725001324 learned_addr learned my addr 172.21.15.100:0/725001324 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:26:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.715+0000 7f25fa93a640 1 -- 172.21.15.100:0/725001324 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25f4102b00 msgr2=0x7f25f4197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.715+0000 7f25fa93a640 1 --2- 172.21.15.100:0/725001324 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25f4102b00 0x7f25f4197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.715+0000 7f25fa93a640 1 -- 172.21.15.100:0/725001324 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25f4101910 msgr2=0x7f25f4197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.715+0000 7f25fa93a640 1 --2- 172.21.15.100:0/725001324 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25f4101910 0x7f25f4197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.715+0000 7f25fa93a640 1 -- 172.21.15.100:0/725001324 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25f419f5f0 con 0x7f25f41034c0 2024-05-10T11:26:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.715+0000 7f25fa93a640 1 --2- 172.21.15.100:0/725001324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25f41034c0 0x7f25f419ef80 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f25ec00eb10 tx=0x7f25ec00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:26:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.715+0000 7f25eb7fe640 1 -- 172.21.15.100:0/725001324 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25ec004b20 con 0x7f25f41034c0 2024-05-10T11:26:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.715+0000 7f25eb7fe640 1 -- 172.21.15.100:0/725001324 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f25ec004cc0 con 0x7f25f41034c0 2024-05-10T11:26:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.715+0000 7f25fc39d640 1 -- 172.21.15.100:0/725001324 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25f419f8e0 con 0x7f25f41034c0 2024-05-10T11:26:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.715+0000 7f25eb7fe640 1 -- 172.21.15.100:0/725001324 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25ec010690 con 0x7f25f41034c0 2024-05-10T11:26:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.716+0000 7f25fc39d640 1 -- 172.21.15.100:0/725001324 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25f419fdb0 con 0x7f25f41034c0 2024-05-10T11:26:02.723 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.718+0000 7f25eb7fe640 1 -- 172.21.15.100:0/725001324 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f25ec005020 con 0x7f25f41034c0 2024-05-10T11:26:02.723 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.718+0000 7f25ff18c740 1 Processor -- start 2024-05-10T11:26:02.724 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.718+0000 7f25ff18c740 1 -- start start 2024-05-10T11:26:02.724 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.719+0000 7f25ff18c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55648feb8fc0 0x55648feb93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:02.724 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.719+0000 7f25ff18c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55648feb99b0 0x55648febe4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:02.724 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.719+0000 7f25ff18c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55648febeb20 0x55648fec0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:02.724 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.719+0000 7f25ff18c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55648fe1b5d0 con 0x55648feb99b0 2024-05-10T11:26:02.724 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.719+0000 7f25ff18c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55648fdd4890 con 0x55648febeb20 2024-05-10T11:26:02.724 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.719+0000 7f25ff18c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55648fe17590 con 0x55648feb8fc0 2024-05-10T11:26:02.724 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.719+0000 7f25f9938640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55648feb8fc0 0x55648feb93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:02.724 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.719+0000 7f25f9938640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55648feb8fc0 0x55648feb93e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:59764/0 (socket says 172.21.15.100:59764) 2024-05-10T11:26:02.724 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.719+0000 7f25f9938640 1 -- 172.21.15.100:0/453386239 learned_addr learned my addr 172.21.15.100:0/453386239 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:26:02.724 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.719+0000 7f25fa139640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55648feb99b0 0x55648febe4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:02.724 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.719+0000 7f25f9938640 1 --2- 172.21.15.100:0/453386239 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55648febeb20 0x55648fec0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:02.724 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.719+0000 7f25f9938640 1 -- 172.21.15.100:0/453386239 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55648febeb20 msgr2=0x55648fec0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:02.724 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.719+0000 7f25f9938640 1 --2- 172.21.15.100:0/453386239 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55648febeb20 0x55648fec0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:02.725 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.719+0000 7f25f9938640 1 -- 172.21.15.100:0/453386239 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55648feb99b0 msgr2=0x55648febe4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:02.725 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.719+0000 7f25f9938640 1 --2- 172.21.15.100:0/453386239 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55648feb99b0 0x55648febe4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:02.725 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.719+0000 7f25f9938640 1 -- 172.21.15.100:0/453386239 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55648fb2d210 con 0x55648feb8fc0 2024-05-10T11:26:02.725 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.719+0000 7f25fa139640 1 --2- 172.21.15.100:0/453386239 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55648feb99b0 0x55648febe4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:26:02.725 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.719+0000 7f25eb7fe640 1 --2- 172.21.15.100:0/725001324 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f25e00b28a0 0x7f25e00b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:02.725 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.719+0000 7f25f9938640 1 --2- 172.21.15.100:0/453386239 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55648feb8fc0 0x55648feb93e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f25e402a930 tx=0x7f25e402a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b2361b8d33e0318 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:26:02.725 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.719+0000 7f25fa139640 1 --2- 172.21.15.100:0/725001324 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f25e00b28a0 0x7f25e00b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:02.725 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.719+0000 7f25eb7fe640 1 -- 172.21.15.100:0/725001324 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f25ec0e2b20 con 0x7f25f41034c0 2024-05-10T11:26:02.725 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.720+0000 7f25e8bfe640 1 -- 172.21.15.100:0/453386239 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e4035040 con 0x55648feb8fc0 2024-05-10T11:26:02.725 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.720+0000 7f25e8bfe640 1 -- 172.21.15.100:0/453386239 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f25e4004030 con 0x55648feb8fc0 2024-05-10T11:26:02.725 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.720+0000 7f25e8bfe640 1 -- 172.21.15.100:0/453386239 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e4003850 con 0x55648feb8fc0 2024-05-10T11:26:02.725 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.720+0000 7f25ff18c740 1 -- 172.21.15.100:0/453386239 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55648feb8fc0 msgr2=0x55648feb93e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:02.725 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.720+0000 7f25fa139640 1 --2- 172.21.15.100:0/725001324 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f25e00b28a0 0x7f25e00b4d60 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x55648feb9db0 tx=0x7f25f0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:26:02.725 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.720+0000 7f25ff18c740 1 --2- 172.21.15.100:0/453386239 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55648feb8fc0 0x55648feb93e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f25e402a930 tx=0x7f25e402a410 comp rx=0 tx=0).stop 2024-05-10T11:26:02.726 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.721+0000 7f25ff18c740 1 -- 172.21.15.100:0/453386239 shutdown_connections 2024-05-10T11:26:02.726 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.721+0000 7f25ff18c740 1 --2- 172.21.15.100:0/453386239 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55648febeb20 0x55648fec0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:02.726 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.721+0000 7f25ff18c740 1 --2- 172.21.15.100:0/453386239 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55648feb99b0 0x55648febe4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:02.726 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.721+0000 7f25ff18c740 1 --2- 172.21.15.100:0/453386239 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55648feb8fc0 0x55648feb93e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:02.726 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.721+0000 7f25ff18c740 1 -- 172.21.15.100:0/453386239 >> 172.21.15.100:0/453386239 conn(0x55648feb2420 msgr2=0x55648feb2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:26:02.726 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.721+0000 7f25ff18c740 1 -- 172.21.15.100:0/453386239 shutdown_connections 2024-05-10T11:26:02.726 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.721+0000 7f25ff18c740 1 -- 172.21.15.100:0/453386239 wait complete. 2024-05-10T11:26:02.726 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.721+0000 7f25ff18c740 1 -- start start 2024-05-10T11:26:02.726 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.721+0000 7f25ff18c740 1 Processor -- start 2024-05-10T11:26:02.726 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.721+0000 7f25ff18c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55648feb8fc0 0x55648fec39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:02.726 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.721+0000 7f25ff18c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55648feb99b0 0x55648fec5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:02.726 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.721+0000 7f25ff18c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55648febeb20 0x55648fec8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:02.726 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.721+0000 7f25ff18c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55648feb6ad0 con 0x55648feb8fc0 2024-05-10T11:26:02.726 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.721+0000 7f25ff18c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55648fdd8370 con 0x55648febeb20 2024-05-10T11:26:02.727 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.721+0000 7f25ff18c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55648feb6c50 con 0x55648feb99b0 2024-05-10T11:26:02.727 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.721+0000 7f25ff18c740 10 client.0 _ll_register_callbacks cb 0x55648fb2ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:26:02.727 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.721+0000 7f25fa139640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55648feb8fc0 0x55648fec39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:02.727 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.721+0000 7f25fa139640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55648feb8fc0 0x55648fec39c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46444/0 (socket says 172.21.15.100:46444) 2024-05-10T11:26:02.727 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.721+0000 7f25fa139640 1 -- 172.21.15.100:0/2000538595 learned_addr learned my addr 172.21.15.100:0/2000538595 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:26:02.727 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.721+0000 7f25f9938640 1 --2- 172.21.15.100:0/2000538595 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55648feb99b0 0x55648fec5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:02.727 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.721+0000 7f25fa93a640 1 --2- 172.21.15.100:0/2000538595 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55648febeb20 0x55648fec8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:02.727 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.722+0000 7f25fa139640 1 -- 172.21.15.100:0/2000538595 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55648feb99b0 msgr2=0x55648fec5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:02.727 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.722+0000 7f25fa139640 1 --2- 172.21.15.100:0/2000538595 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55648feb99b0 0x55648fec5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:02.727 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.722+0000 7f25fa139640 1 -- 172.21.15.100:0/2000538595 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55648febeb20 msgr2=0x55648fec8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:02.727 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.722+0000 7f25fa139640 1 --2- 172.21.15.100:0/2000538595 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55648febeb20 0x55648fec8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:02.727 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.722+0000 7f25fa139640 1 -- 172.21.15.100:0/2000538595 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55648fbf9620 con 0x55648feb8fc0 2024-05-10T11:26:02.727 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.722+0000 7f25f9938640 1 --2- 172.21.15.100:0/2000538595 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55648feb99b0 0x55648fec5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:26:02.727 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.722+0000 7f25fa139640 1 --2- 172.21.15.100:0/2000538595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55648feb8fc0 0x55648fec39c0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f25f0009560 tx=0x7f25f0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfd8ebc1955b901f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:26:02.727 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.722+0000 7f25ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:26:02.727 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.722+0000 7f25ddffb640 1 -- 172.21.15.100:0/2000538595 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f0009830 con 0x55648feb8fc0 2024-05-10T11:26:02.728 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.722+0000 7f25ddffb640 1 -- 172.21.15.100:0/2000538595 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f25f0018030 con 0x55648feb8fc0 2024-05-10T11:26:02.728 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.722+0000 7f25ff18c740 1 -- 172.21.15.100:0/2000538595 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25e4004030 con 0x55648feb8fc0 2024-05-10T11:26:02.728 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.722+0000 7f25ddffb640 1 -- 172.21.15.100:0/2000538595 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f25f0034050 con 0x55648feb8fc0 2024-05-10T11:26:02.728 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.722+0000 7f25ff18c740 10 client.16899 fetch_fsmap learned FSMap version 31 2024-05-10T11:26:02.728 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.722+0000 7f25ff18c740 1 -- 172.21.15.100:0/2000538595 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55648fecb630 con 0x55648feb8fc0 2024-05-10T11:26:02.728 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.722+0000 7f25ddffb640 1 -- 172.21.15.100:0/2000538595 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f25e4004030 con 0x55648feb8fc0 2024-05-10T11:26:02.728 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.723+0000 7f25ddffb640 1 -- 172.21.15.100:0/2000538595 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f25f000f450 con 0x55648feb8fc0 2024-05-10T11:26:02.728 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.723+0000 7f25ff18c740 10 client.16899 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:26:02.728 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.723+0000 7f25ff18c740 10 client.16899 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:26:02.728 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.723+0000 7f25ff18c740 20 client.16899 populate_metadata read hostname 'smithi100' 2024-05-10T11:26:02.728 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.723+0000 7f25ff18c740 1 --2- 172.21.15.100:0/2000538595 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55648feccd20 0x55648fecf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:02.728 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.723+0000 7f25ff18c740 4 client.16899 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:26:02.728 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.723+0000 7f25ff18c740 1 -- 172.21.15.100:0/2000538595 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55648fe291e0 con 0x55648feccd20 2024-05-10T11:26:02.728 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.723+0000 7f25f9938640 1 --2- 172.21.15.100:0/2000538595 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55648feccd20 0x55648fecf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:02.728 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.723+0000 7f25ddffb640 10 client.16899 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:26:02.728 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.723+0000 7f25f9938640 1 --2- 172.21.15.100:0/2000538595 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55648feccd20 0x55648fecf1c0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8dbda3b7039a54d5 server_cookie=697cdf7b647680a6 in_seq=0 out_seq=0 2024-05-10T11:26:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.724+0000 7f25ddffb640 1 -- 172.21.15.100:0/2000538595 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55648fe291e0 con 0x55648feccd20 2024-05-10T11:26:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.724+0000 7f25ddffb640 10 client.16899 handle_command_reply: tid=0 2024-05-10T11:26:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.724+0000 7f25ff18c740 1 client.16899 shutdown 2024-05-10T11:26:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.724+0000 7f25ff18c740 1 -- 172.21.15.100:0/2000538595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55648feb8fc0 msgr2=0x55648fec39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.724+0000 7f25ff18c740 1 --2- 172.21.15.100:0/2000538595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55648feb8fc0 0x55648fec39c0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f25f0009560 tx=0x7f25f0004730 comp rx=0 tx=0).stop 2024-05-10T11:26:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.724+0000 7f25ff18c740 1 -- 172.21.15.100:0/2000538595 shutdown_connections 2024-05-10T11:26:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.724+0000 7f25ff18c740 1 --2- 172.21.15.100:0/2000538595 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55648feccd20 0x55648fecf1c0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.724+0000 7f25ff18c740 1 --2- 172.21.15.100:0/2000538595 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55648febeb20 0x55648fec8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.724+0000 7f25ddffb640 0 client.16899 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:26:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.724+0000 7f25ff18c740 1 --2- 172.21.15.100:0/2000538595 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55648feb99b0 0x55648fec5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.724+0000 7f25ff18c740 1 --2- 172.21.15.100:0/2000538595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55648feb8fc0 0x55648fec39c0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.724+0000 7f25ff18c740 1 -- 172.21.15.100:0/2000538595 >> 172.21.15.100:0/2000538595 conn(0x55648feb2420 msgr2=0x55648feb1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:26:02.731 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.724+0000 7f25ff18c740 1 -- 172.21.15.100:0/2000538595 shutdown_connections 2024-05-10T11:26:02.731 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.724+0000 7f25ff18c740 1 -- 172.21.15.100:0/2000538595 wait complete. 2024-05-10T11:26:02.731 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.724+0000 7f25ff18c740 20 client.16899 trim_cache size 0 max 16384 2024-05-10T11:26:02.746 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.741+0000 7f25ff18c740 1 Processor -- start 2024-05-10T11:26:02.746 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.741+0000 7f25ff18c740 1 -- start start 2024-05-10T11:26:02.746 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.741+0000 7f25ff18c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55648feb8fc0 0x55648fed3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:02.746 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.741+0000 7f25ff18c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55648feb99b0 0x55648fec79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:02.746 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.741+0000 7f25ff18c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55648febeb20 0x55648fec7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:02.746 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.741+0000 7f25ff18c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f25f41afd70 con 0x55648feb99b0 2024-05-10T11:26:02.746 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.741+0000 7f25ff18c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f25f41ad640 con 0x55648feb8fc0 2024-05-10T11:26:02.746 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.741+0000 7f25ff18c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f25f41ab490 con 0x55648febeb20 2024-05-10T11:26:02.746 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.742+0000 7f25f9938640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55648feb99b0 0x55648fec79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:02.746 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.742+0000 7f25f9938640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55648feb99b0 0x55648fec79e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46446/0 (socket says 172.21.15.100:46446) 2024-05-10T11:26:02.746 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.742+0000 7f25f9938640 1 -- 172.21.15.100:0/2849529440 learned_addr learned my addr 172.21.15.100:0/2849529440 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:26:02.746 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.742+0000 7f25fa139640 1 --2- 172.21.15.100:0/2849529440 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55648feb8fc0 0x55648fed3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:02.746 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.742+0000 7f25fa93a640 1 --2- 172.21.15.100:0/2849529440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55648febeb20 0x55648fec7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:02.747 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.742+0000 7f25f9938640 1 -- 172.21.15.100:0/2849529440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55648febeb20 msgr2=0x55648fec7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:02.747 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.742+0000 7f25f9938640 1 --2- 172.21.15.100:0/2849529440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55648febeb20 0x55648fec7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:02.747 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.742+0000 7f25f9938640 1 -- 172.21.15.100:0/2849529440 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55648feb8fc0 msgr2=0x55648fed3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:02.747 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.742+0000 7f25f9938640 1 --2- 172.21.15.100:0/2849529440 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55648feb8fc0 0x55648fed3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:02.747 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.742+0000 7f25f9938640 1 -- 172.21.15.100:0/2849529440 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55648feb1ec0 con 0x55648feb99b0 2024-05-10T11:26:02.747 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.742+0000 7f25f9938640 1 --2- 172.21.15.100:0/2849529440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55648feb99b0 0x55648fec79e0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f25e4009900 tx=0x7f25e4005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89f78be00e2467d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:26:02.747 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.742+0000 7f25dffff640 1 -- 172.21.15.100:0/2849529440 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e4035040 con 0x55648feb99b0 2024-05-10T11:26:02.747 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.742+0000 7f25dffff640 1 -- 172.21.15.100:0/2849529440 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f25e4004240 con 0x55648feb99b0 2024-05-10T11:26:02.747 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.742+0000 7f25dffff640 1 -- 172.21.15.100:0/2849529440 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e40035f0 con 0x55648feb99b0 2024-05-10T11:26:02.747 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.743+0000 7f25ff18c740 1 -- 172.21.15.100:0/2849529440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55648feb99b0 msgr2=0x55648fec79e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:02.747 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.743+0000 7f25ff18c740 1 --2- 172.21.15.100:0/2849529440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55648feb99b0 0x55648fec79e0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f25e4009900 tx=0x7f25e4005fd0 comp rx=0 tx=0).stop 2024-05-10T11:26:02.747 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.743+0000 7f25ff18c740 1 -- 172.21.15.100:0/2849529440 shutdown_connections 2024-05-10T11:26:02.747 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.743+0000 7f25ff18c740 1 --2- 172.21.15.100:0/2849529440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55648febeb20 0x55648fec7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:02.747 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.743+0000 7f25ff18c740 1 --2- 172.21.15.100:0/2849529440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55648feb99b0 0x55648fec79e0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:02.748 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.743+0000 7f25ff18c740 1 --2- 172.21.15.100:0/2849529440 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55648feb8fc0 0x55648fed3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:02.748 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.743+0000 7f25ff18c740 1 -- 172.21.15.100:0/2849529440 >> 172.21.15.100:0/2849529440 conn(0x55648feb2420 msgr2=0x55648feb7050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:26:02.748 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.743+0000 7f25ff18c740 1 -- 172.21.15.100:0/2849529440 shutdown_connections 2024-05-10T11:26:02.748 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.743+0000 7f25ff18c740 1 -- 172.21.15.100:0/2849529440 wait complete. 2024-05-10T11:26:02.748 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.743+0000 7f25ff18c740 1 -- start start 2024-05-10T11:26:02.748 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.743+0000 7f25ff18c740 1 Processor -- start 2024-05-10T11:26:02.748 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.743+0000 7f25ff18c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55648feb8fc0 0x55648fec43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:02.748 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.744+0000 7f25ff18c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55648feb99b0 0x55648fec4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:02.748 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.744+0000 7f25ff18c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55648febeb20 0x55648fec7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:02.748 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.744+0000 7f25ff18c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f25f41abb00 con 0x55648feb8fc0 2024-05-10T11:26:02.748 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.744+0000 7f25ff18c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f25f41acf70 con 0x55648feb99b0 2024-05-10T11:26:02.748 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.744+0000 7f25ff18c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f25f41ac1d0 con 0x55648febeb20 2024-05-10T11:26:02.748 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.744+0000 7f25ff18c740 10 client.0 _ll_register_callbacks cb 0x7f25f0009660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:26:02.748 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.744+0000 7f25fa139640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55648feb8fc0 0x55648fec43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:02.748 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.744+0000 7f25f9938640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55648feb99b0 0x55648fec4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:02.749 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.744+0000 7f25fa139640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55648feb8fc0 0x55648fec43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46462/0 (socket says 172.21.15.100:46462) 2024-05-10T11:26:02.749 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.744+0000 7f25fa139640 1 -- 172.21.15.100:0/3372816409 learned_addr learned my addr 172.21.15.100:0/3372816409 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:26:02.749 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.744+0000 7f25fa93a640 1 --2- 172.21.15.100:0/3372816409 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55648febeb20 0x55648fec7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:02.749 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.744+0000 7f25fa139640 1 -- 172.21.15.100:0/3372816409 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55648febeb20 msgr2=0x55648fec7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:02.749 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.744+0000 7f25fa139640 1 --2- 172.21.15.100:0/3372816409 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55648febeb20 0x55648fec7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:02.749 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.744+0000 7f25fa139640 1 -- 172.21.15.100:0/3372816409 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55648feb99b0 msgr2=0x55648fec4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:02.749 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.744+0000 7f25fa139640 1 --2- 172.21.15.100:0/3372816409 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55648feb99b0 0x55648fec4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:02.749 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.744+0000 7f25fa139640 1 -- 172.21.15.100:0/3372816409 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55648fec7f10 con 0x55648feb8fc0 2024-05-10T11:26:02.749 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.744+0000 7f25f9938640 1 --2- 172.21.15.100:0/3372816409 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55648feb99b0 0x55648fec4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:26:02.749 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.744+0000 7f25fa93a640 1 --2- 172.21.15.100:0/3372816409 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55648febeb20 0x55648fec7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:26:02.749 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.744+0000 7f25fa139640 1 --2- 172.21.15.100:0/3372816409 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55648feb8fc0 0x55648fec43c0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f25f0019040 tx=0x7f25f0012d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0082a8190205648 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:26:02.749 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.744+0000 7f25ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:26:02.749 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.744+0000 7f25ddffb640 1 -- 172.21.15.100:0/3372816409 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f003e030 con 0x55648feb8fc0 2024-05-10T11:26:02.749 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.744+0000 7f25ddffb640 1 -- 172.21.15.100:0/3372816409 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f25f0024040 con 0x55648feb8fc0 2024-05-10T11:26:02.749 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.744+0000 7f25ff18c740 1 -- 172.21.15.100:0/3372816409 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25e4004240 con 0x55648feb8fc0 2024-05-10T11:26:02.749 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.744+0000 7f25ddffb640 1 -- 172.21.15.100:0/3372816409 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f25f0048050 con 0x55648feb8fc0 2024-05-10T11:26:02.750 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.744+0000 7f25ff18c740 10 client.16905 fetch_fsmap learned FSMap version 31 2024-05-10T11:26:02.750 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.744+0000 7f25ff18c740 1 -- 172.21.15.100:0/3372816409 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55648ff37da0 con 0x55648feb8fc0 2024-05-10T11:26:02.750 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.745+0000 7f25ddffb640 1 -- 172.21.15.100:0/3372816409 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f25e4004240 con 0x55648feb8fc0 2024-05-10T11:26:02.750 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.745+0000 7f25ddffb640 1 -- 172.21.15.100:0/3372816409 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f25f0005020 con 0x55648feb8fc0 2024-05-10T11:26:02.750 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.745+0000 7f25ff18c740 10 client.16905 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:26:02.750 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.745+0000 7f25ff18c740 10 client.16905 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:26:02.750 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.745+0000 7f25ff18c740 20 client.16905 populate_metadata read hostname 'smithi100' 2024-05-10T11:26:02.750 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.745+0000 7f25ff18c740 1 --2- 172.21.15.100:0/3372816409 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55648ff38770 0x55648ff3ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:02.750 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.745+0000 7f25ff18c740 4 client.16905 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:26:02.750 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.745+0000 7f25ff18c740 1 -- 172.21.15.100:0/3372816409 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55648fe23050 con 0x55648ff38770 2024-05-10T11:26:02.750 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.745+0000 7f25f9938640 1 --2- 172.21.15.100:0/3372816409 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55648ff38770 0x55648ff3ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:02.750 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.745+0000 7f25ddffb640 10 client.16905 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:26:02.750 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.745+0000 7f25f9938640 1 --2- 172.21.15.100:0/3372816409 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55648ff38770 0x55648ff3ab90 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e0f8ec0a68c6747 server_cookie=a3c9206243b83785 in_seq=0 out_seq=0 2024-05-10T11:26:02.764 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.763+0000 7f25ddffb640 1 -- 172.21.15.100:0/3372816409 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55648fe23050 con 0x55648ff38770 2024-05-10T11:26:02.764 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.763+0000 7f25ddffb640 10 client.16905 handle_command_reply: tid=0 2024-05-10T11:26:02.764 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.763+0000 7f25ff18c740 1 client.16905 shutdown 2024-05-10T11:26:02.764 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.764+0000 7f25ff18c740 1 -- 172.21.15.100:0/3372816409 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55648feb8fc0 msgr2=0x55648fec43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:02.764 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.764+0000 7f25ff18c740 1 --2- 172.21.15.100:0/3372816409 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55648feb8fc0 0x55648fec43c0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f25f0019040 tx=0x7f25f0012d80 comp rx=0 tx=0).stop 2024-05-10T11:26:02.765 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.764+0000 7f25ff18c740 1 -- 172.21.15.100:0/3372816409 shutdown_connections 2024-05-10T11:26:02.765 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.764+0000 7f25ff18c740 1 --2- 172.21.15.100:0/3372816409 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55648ff38770 0x55648ff3ab90 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:02.765 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.764+0000 7f25ff18c740 1 --2- 172.21.15.100:0/3372816409 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55648febeb20 0x55648fec7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:02.765 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.764+0000 7f25ddffb640 0 client.16905 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:26:02.765 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.764+0000 7f25ff18c740 1 --2- 172.21.15.100:0/3372816409 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55648feb99b0 0x55648fec4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:02.765 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.764+0000 7f25ff18c740 1 --2- 172.21.15.100:0/3372816409 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55648feb8fc0 0x55648fec43c0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:02.765 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.764+0000 7f25ff18c740 1 -- 172.21.15.100:0/3372816409 >> 172.21.15.100:0/3372816409 conn(0x55648feb2420 msgr2=0x55648feb7050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:26:02.765 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.764+0000 7f25ff18c740 1 -- 172.21.15.100:0/3372816409 shutdown_connections 2024-05-10T11:26:02.765 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.764+0000 7f25ff18c740 1 -- 172.21.15.100:0/3372816409 wait complete. 2024-05-10T11:26:02.765 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.764+0000 7f25ff18c740 20 client.16905 trim_cache size 0 max 16384 2024-05-10T11:26:02.765 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:26:02.766 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:26:02.766 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:26:02.766 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:26:02.766 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.764+0000 7f25fc39d640 1 -- 172.21.15.100:0/725001324 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f25e00b28a0 msgr2=0x7f25e00b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:02.767 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.764+0000 7f25fc39d640 1 --2- 172.21.15.100:0/725001324 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f25e00b28a0 0x7f25e00b4d60 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x55648feb9db0 tx=0x7f25f0005f90 comp rx=0 tx=0).stop 2024-05-10T11:26:02.767 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.764+0000 7f25fc39d640 1 -- 172.21.15.100:0/725001324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25f41034c0 msgr2=0x7f25f419ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:02.767 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.764+0000 7f25fc39d640 1 --2- 172.21.15.100:0/725001324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25f41034c0 0x7f25f419ef80 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f25ec00eb10 tx=0x7f25ec00c2d0 comp rx=0 tx=0).stop 2024-05-10T11:26:02.767 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.764+0000 7f25fc39d640 1 -- 172.21.15.100:0/725001324 shutdown_connections 2024-05-10T11:26:02.767 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.764+0000 7f25fc39d640 1 --2- 172.21.15.100:0/725001324 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f25e00b28a0 0x7f25e00b4d60 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:02.767 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.764+0000 7f25fc39d640 1 --2- 172.21.15.100:0/725001324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25f41034c0 0x7f25f419ef80 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:02.767 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.764+0000 7f25fc39d640 1 --2- 172.21.15.100:0/725001324 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25f4102b00 0x7f25f4197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:02.767 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.764+0000 7f25fc39d640 1 --2- 172.21.15.100:0/725001324 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25f4101910 0x7f25f4197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:02.767 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.764+0000 7f25fc39d640 1 -- 172.21.15.100:0/725001324 >> 172.21.15.100:0/725001324 conn(0x7f25f40fce80 msgr2=0x7f25f4105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:26:02.767 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.764+0000 7f25fc39d640 1 -- 172.21.15.100:0/725001324 shutdown_connections 2024-05-10T11:26:02.767 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:02.764+0000 7f25fc39d640 1 -- 172.21.15.100:0/725001324 wait complete. 2024-05-10T11:26:02.975 INFO:tasks.workunit.client.0.smithi100.stdout: 1 184831 2.75 MB/sec execute 237 sec latency 14.230 ms 2024-05-10T11:26:03.975 INFO:tasks.workunit.client.0.smithi100.stdout: 1 185144 2.74 MB/sec execute 238 sec latency 13.537 ms 2024-05-10T11:26:04.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:26:04 smithi178 ceph-mon[33359]: pgmap v650: 129 pgs: 129 active+clean; 81 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-05-10T11:26:04.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:26:04 smithi188 ceph-mon[34075]: pgmap v650: 129 pgs: 129 active+clean; 81 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-05-10T11:26:04.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:26:04 smithi100 ceph-mon[30296]: pgmap v650: 129 pgs: 129 active+clean; 81 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-05-10T11:26:04.975 INFO:tasks.workunit.client.0.smithi100.stdout: 1 185674 2.73 MB/sec execute 239 sec latency 18.054 ms 2024-05-10T11:26:05.975 INFO:tasks.workunit.client.0.smithi100.stdout: 1 186311 2.73 MB/sec execute 240 sec latency 20.697 ms 2024-05-10T11:26:06.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:26:06 smithi178 ceph-mon[33359]: pgmap v651: 129 pgs: 129 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 1.5 MiB/s wr, 184 op/s 2024-05-10T11:26:06.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:26:06 smithi188 ceph-mon[34075]: pgmap v651: 129 pgs: 129 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 1.5 MiB/s wr, 184 op/s 2024-05-10T11:26:06.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:26:06 smithi100 ceph-mon[30296]: pgmap v651: 129 pgs: 129 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 1.5 MiB/s wr, 184 op/s 2024-05-10T11:26:06.975 INFO:tasks.workunit.client.0.smithi100.stdout: 1 186886 2.73 MB/sec execute 241 sec latency 43.425 ms 2024-05-10T11:26:07.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:26:07 smithi178 ceph-mon[33359]: pgmap v652: 129 pgs: 129 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 1.4 MiB/s wr, 141 op/s 2024-05-10T11:26:07.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:26:07 smithi188 ceph-mon[34075]: pgmap v652: 129 pgs: 129 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 1.4 MiB/s wr, 141 op/s 2024-05-10T11:26:07.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:26:07 smithi100 ceph-mon[30296]: pgmap v652: 129 pgs: 129 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 1.4 MiB/s wr, 141 op/s 2024-05-10T11:26:07.975 INFO:tasks.workunit.client.0.smithi100.stdout: 1 187658 2.74 MB/sec execute 242 sec latency 13.807 ms 2024-05-10T11:26:08.975 INFO:tasks.workunit.client.0.smithi100.stdout: 1 188122 2.75 MB/sec execute 243 sec latency 15.742 ms 2024-05-10T11:26:09.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:26:09 smithi178 ceph-mon[33359]: pgmap v653: 129 pgs: 129 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 1.3 MiB/s wr, 178 op/s 2024-05-10T11:26:09.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:26:09 smithi188 ceph-mon[34075]: pgmap v653: 129 pgs: 129 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 1.3 MiB/s wr, 178 op/s 2024-05-10T11:26:09.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:26:09 smithi100 ceph-mon[30296]: pgmap v653: 129 pgs: 129 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 1.3 MiB/s wr, 178 op/s 2024-05-10T11:26:09.975 INFO:tasks.workunit.client.0.smithi100.stdout: 1 188426 2.75 MB/sec execute 244 sec latency 13.410 ms 2024-05-10T11:26:10.975 INFO:tasks.workunit.client.0.smithi100.stdout: 1 188790 2.74 MB/sec execute 245 sec latency 16.152 ms 2024-05-10T11:26:11.707 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T11:26:11.733 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:26:11 smithi100 ceph-mon[30296]: pgmap v654: 129 pgs: 129 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 1.1 MiB/s wr, 130 op/s 2024-05-10T11:26:11.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:26:11 smithi178 ceph-mon[33359]: pgmap v654: 129 pgs: 129 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 1.1 MiB/s wr, 130 op/s 2024-05-10T11:26:11.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:26:11 smithi188 ceph-mon[34075]: pgmap v654: 129 pgs: 129 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 1.1 MiB/s wr, 130 op/s 2024-05-10T11:26:11.971 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:26:11.975 INFO:tasks.workunit.client.0.smithi100.stdout: 1 189268 2.73 MB/sec execute 246 sec latency 26.374 ms 2024-05-10T11:26:12.975 INFO:tasks.workunit.client.0.smithi100.stdout: 1 189881 2.73 MB/sec execute 247 sec latency 16.049 ms 2024-05-10T11:26:13.677 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:26:13 smithi100 ceph-mon[30296]: pgmap v655: 129 pgs: 129 active+clean; 81 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 1.3 MiB/s wr, 133 op/s 2024-05-10T11:26:13.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:26:13 smithi178 ceph-mon[33359]: pgmap v655: 129 pgs: 129 active+clean; 81 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 1.3 MiB/s wr, 133 op/s 2024-05-10T11:26:13.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:26:13 smithi188 ceph-mon[34075]: pgmap v655: 129 pgs: 129 active+clean; 81 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 1.3 MiB/s wr, 133 op/s 2024-05-10T11:26:13.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.791+0000 7f559a60c700 1 -- 172.21.15.100:0/3604188646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f559406bb40 msgr2=0x7f5594110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:13.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.791+0000 7f559a60c700 1 --2- 172.21.15.100:0/3604188646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f559406bb40 0x7f5594110330 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f558c00b0a0 tx=0x7f558c00b3b0 comp rx=0 tx=0).stop 2024-05-10T11:26:13.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.791+0000 7f559a60c700 1 -- 172.21.15.100:0/3604188646 shutdown_connections 2024-05-10T11:26:13.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.791+0000 7f559a60c700 1 --2- 172.21.15.100:0/3604188646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f559406bb40 0x7f5594110330 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:13.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.791+0000 7f559a60c700 1 --2- 172.21.15.100:0/3604188646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5594109510 0x7f55941098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:13.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.791+0000 7f559a60c700 1 --2- 172.21.15.100:0/3604188646 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f559406b180 0x7f559406b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:13.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.791+0000 7f559a60c700 1 -- 172.21.15.100:0/3604188646 >> 172.21.15.100:0/3604188646 conn(0x7f5594079350 msgr2=0x7f5594079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:26:13.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.791+0000 7f559a60c700 1 -- 172.21.15.100:0/3604188646 shutdown_connections 2024-05-10T11:26:13.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.791+0000 7f559a60c700 1 -- 172.21.15.100:0/3604188646 wait complete. 2024-05-10T11:26:13.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.792+0000 7f559a60c700 1 Processor -- start 2024-05-10T11:26:13.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.792+0000 7f559a60c700 1 -- start start 2024-05-10T11:26:13.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.792+0000 7f559a60c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f559406b180 0x7f5594102470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:13.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.792+0000 7f559a60c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f559406bb40 0x7f55941029b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:13.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.792+0000 7f559a60c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5594109510 0x7f5594102ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:13.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.792+0000 7f559a60c700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f55941134f0 con 0x7f5594109510 2024-05-10T11:26:13.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.792+0000 7f559a60c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5594113370 con 0x7f559406bb40 2024-05-10T11:26:13.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.792+0000 7f559a60c700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5594113670 con 0x7f559406b180 2024-05-10T11:26:13.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.792+0000 7f5598ba9700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5594109510 0x7f5594102ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:13.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.792+0000 7f5593fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f559406b180 0x7f5594102470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:13.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.792+0000 7f5593fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f559406b180 0x7f5594102470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:60252/0 (socket says 172.21.15.100:60252) 2024-05-10T11:26:13.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.792+0000 7f5593fff700 1 -- 172.21.15.100:0/1767965963 learned_addr learned my addr 172.21.15.100:0/1767965963 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:26:13.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.792+0000 7f55937fe700 1 --2- 172.21.15.100:0/1767965963 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f559406bb40 0x7f55941029b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:13.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.792+0000 7f5593fff700 1 -- 172.21.15.100:0/1767965963 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f559406bb40 msgr2=0x7f55941029b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:13.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.793+0000 7f5593fff700 1 --2- 172.21.15.100:0/1767965963 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f559406bb40 0x7f55941029b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:13.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.793+0000 7f5593fff700 1 -- 172.21.15.100:0/1767965963 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5594109510 msgr2=0x7f5594102ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:13.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.793+0000 7f5593fff700 1 --2- 172.21.15.100:0/1767965963 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5594109510 0x7f5594102ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:13.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.793+0000 7f5593fff700 1 -- 172.21.15.100:0/1767965963 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55941a7f50 con 0x7f559406b180 2024-05-10T11:26:13.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.793+0000 7f5598ba9700 1 --2- 172.21.15.100:0/1767965963 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5594109510 0x7f5594102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:26:13.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.793+0000 7f5593fff700 1 --2- 172.21.15.100:0/1767965963 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f559406b180 0x7f5594102470 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f558400e9c0 tx=0x7f558400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:26:13.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.793+0000 7f55917fa700 1 -- 172.21.15.100:0/1767965963 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558400cbd0 con 0x7f559406b180 2024-05-10T11:26:13.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.793+0000 7f55917fa700 1 -- 172.21.15.100:0/1767965963 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f558400ee60 con 0x7f559406b180 2024-05-10T11:26:13.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.793+0000 7f559a60c700 1 -- 172.21.15.100:0/1767965963 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55941a81e0 con 0x7f559406b180 2024-05-10T11:26:13.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.793+0000 7f55917fa700 1 -- 172.21.15.100:0/1767965963 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55840054b0 con 0x7f559406b180 2024-05-10T11:26:13.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.793+0000 7f559a60c700 1 -- 172.21.15.100:0/1767965963 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55941a86d0 con 0x7f559406b180 2024-05-10T11:26:13.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.795+0000 7f55917fa700 1 -- 172.21.15.100:0/1767965963 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f5584004750 con 0x7f559406b180 2024-05-10T11:26:13.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.796+0000 7f55917fa700 1 --2- 172.21.15.100:0/1767965963 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f557c0b2880 0x7f557c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:13.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.796+0000 7f55937fe700 1 --2- 172.21.15.100:0/1767965963 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f557c0b2880 0x7f557c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:13.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.796+0000 7f55917fa700 1 -- 172.21.15.100:0/1767965963 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f55840e2880 con 0x7f559406b180 2024-05-10T11:26:13.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.797+0000 7f55a4e9fb80 1 Processor -- start 2024-05-10T11:26:13.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.797+0000 7f55a4e9fb80 1 -- start start 2024-05-10T11:26:13.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.797+0000 7f55937fe700 1 --2- 172.21.15.100:0/1767965963 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f557c0b2880 0x7f557c0b4d40 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5594079a30 tx=0x7f558800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:26:13.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.797+0000 7f55a4e9fb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c743d24d0 0x562c743d28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:13.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.797+0000 7f55a4e9fb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c743d2df0 0x562c743d66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:13.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.797+0000 7f55a4e9fb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c743d6c80 0x562c743d7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:13.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.797+0000 7f55a4e9fb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x562c74324860 con 0x562c743d6c80 2024-05-10T11:26:13.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.797+0000 7f55a4e9fb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562c742c37c0 con 0x562c743d2df0 2024-05-10T11:26:13.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.797+0000 7f55a4e9fb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x562c74320820 con 0x562c743d24d0 2024-05-10T11:26:13.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.797+0000 7f5598ba9700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c743d6c80 0x562c743d7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:13.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.797+0000 7f5598ba9700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c743d6c80 0x562c743d7100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49274/0 (socket says 172.21.15.100:49274) 2024-05-10T11:26:13.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.797+0000 7f5598ba9700 1 -- 172.21.15.100:0/2905099738 learned_addr learned my addr 172.21.15.100:0/2905099738 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:26:13.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.797+0000 7f5593fff700 1 --2- 172.21.15.100:0/2905099738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c743d24d0 0x562c743d28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:13.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.797+0000 7f55937fe700 1 --2- 172.21.15.100:0/2905099738 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c743d2df0 0x562c743d66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:13.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.798+0000 7f5598ba9700 1 -- 172.21.15.100:0/2905099738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c743d24d0 msgr2=0x562c743d28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:13.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.798+0000 7f5598ba9700 1 --2- 172.21.15.100:0/2905099738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c743d24d0 0x562c743d28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:13.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.798+0000 7f5598ba9700 1 -- 172.21.15.100:0/2905099738 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c743d2df0 msgr2=0x562c743d66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:13.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.798+0000 7f5598ba9700 1 --2- 172.21.15.100:0/2905099738 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c743d2df0 0x562c743d66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:13.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.798+0000 7f5598ba9700 1 -- 172.21.15.100:0/2905099738 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c742718d0 con 0x562c743d6c80 2024-05-10T11:26:13.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.798+0000 7f5598ba9700 1 --2- 172.21.15.100:0/2905099738 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c743d6c80 0x562c743d7100 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f558c00cef0 tx=0x7f558c00cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0c8378e2e539fd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:26:13.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.798+0000 7f557a7fc700 1 -- 172.21.15.100:0/2905099738 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558c021070 con 0x562c743d6c80 2024-05-10T11:26:13.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.798+0000 7f557a7fc700 1 -- 172.21.15.100:0/2905099738 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f558c0074e0 con 0x562c743d6c80 2024-05-10T11:26:13.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.798+0000 7f557a7fc700 1 -- 172.21.15.100:0/2905099738 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558c0052a0 con 0x562c743d6c80 2024-05-10T11:26:13.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.799+0000 7f55a4e9fb80 1 -- 172.21.15.100:0/2905099738 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c743d6c80 msgr2=0x562c743d7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:13.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.799+0000 7f55a4e9fb80 1 --2- 172.21.15.100:0/2905099738 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c743d6c80 0x562c743d7100 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f558c00cef0 tx=0x7f558c00cf20 comp rx=0 tx=0).stop 2024-05-10T11:26:13.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.799+0000 7f55a4e9fb80 1 -- 172.21.15.100:0/2905099738 shutdown_connections 2024-05-10T11:26:13.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.799+0000 7f55a4e9fb80 1 --2- 172.21.15.100:0/2905099738 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c743d6c80 0x562c743d7100 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:13.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.799+0000 7f55a4e9fb80 1 --2- 172.21.15.100:0/2905099738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c743d24d0 0x562c743d28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:13.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.799+0000 7f55a4e9fb80 1 --2- 172.21.15.100:0/2905099738 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c743d2df0 0x562c743d66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:13.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.799+0000 7f55a4e9fb80 1 -- 172.21.15.100:0/2905099738 >> 172.21.15.100:0/2905099738 conn(0x562c742fe100 msgr2=0x562c743ccc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:26:13.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.799+0000 7f55a4e9fb80 1 -- 172.21.15.100:0/2905099738 shutdown_connections 2024-05-10T11:26:13.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.799+0000 7f55a4e9fb80 1 -- 172.21.15.100:0/2905099738 wait complete. 2024-05-10T11:26:13.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.799+0000 7f55a4e9fb80 1 -- start start 2024-05-10T11:26:13.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.800+0000 7f55a4e9fb80 1 Processor -- start 2024-05-10T11:26:13.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.800+0000 7f55a4e9fb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c743d24d0 0x562c743de020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:13.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.800+0000 7f55a4e9fb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c743d2df0 0x562c743db3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:13.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.800+0000 7f55a4e9fb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c743d6c80 0x562c743db900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:13.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.800+0000 7f55a4e9fb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x562c743dee30 con 0x562c743d2df0 2024-05-10T11:26:13.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.800+0000 7f55a4e9fb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562c7428ac30 con 0x562c743d6c80 2024-05-10T11:26:13.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.800+0000 7f55a4e9fb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x562c743defb0 con 0x562c743d24d0 2024-05-10T11:26:13.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.800+0000 7f55a4e9fb80 10 client.0 _ll_register_callbacks cb 0x562c742bc920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:26:13.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.800+0000 7f5593fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c743d24d0 0x562c743de020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:13.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.800+0000 7f5593fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c743d24d0 0x562c743de020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:60276/0 (socket says 172.21.15.100:60276) 2024-05-10T11:26:13.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.800+0000 7f5593fff700 1 -- 172.21.15.100:0/4212766146 learned_addr learned my addr 172.21.15.100:0/4212766146 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:26:13.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.800+0000 7f55937fe700 1 --2- 172.21.15.100:0/4212766146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c743d2df0 0x562c743db3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:13.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.800+0000 7f5598ba9700 1 --2- 172.21.15.100:0/4212766146 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c743d6c80 0x562c743db900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:13.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.801+0000 7f5593fff700 1 -- 172.21.15.100:0/4212766146 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c743d6c80 msgr2=0x562c743db900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:13.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.801+0000 7f5593fff700 1 --2- 172.21.15.100:0/4212766146 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c743d6c80 0x562c743db900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:13.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.801+0000 7f5593fff700 1 -- 172.21.15.100:0/4212766146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c743d2df0 msgr2=0x562c743db3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:13.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.801+0000 7f5593fff700 1 --2- 172.21.15.100:0/4212766146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c743d2df0 0x562c743db3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:13.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.801+0000 7f5593fff700 1 -- 172.21.15.100:0/4212766146 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c742c0690 con 0x562c743d24d0 2024-05-10T11:26:13.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.801+0000 7f55937fe700 1 --2- 172.21.15.100:0/4212766146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c743d2df0 0x562c743db3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:26:13.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.801+0000 7f5593fff700 1 --2- 172.21.15.100:0/4212766146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c743d24d0 0x562c743de020 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f55840dffc0 tx=0x7f55840165b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f8b518c29013c98 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:26:13.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.801+0000 7f55737fe700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:26:13.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.801+0000 7f55737fe700 1 -- 172.21.15.100:0/4212766146 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55840c3060 con 0x562c743d24d0 2024-05-10T11:26:13.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.801+0000 7f55a4e9fb80 1 -- 172.21.15.100:0/4212766146 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f558c0074e0 con 0x562c743d24d0 2024-05-10T11:26:13.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.801+0000 7f55737fe700 1 -- 172.21.15.100:0/4212766146 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f55840b26c0 con 0x562c743d24d0 2024-05-10T11:26:13.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.801+0000 7f55737fe700 1 -- 172.21.15.100:0/4212766146 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f55840b2280 con 0x562c743d24d0 2024-05-10T11:26:13.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.801+0000 7f55a4e9fb80 10 client.26392 fetch_fsmap learned FSMap version 31 2024-05-10T11:26:13.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.801+0000 7f55a4e9fb80 1 -- 172.21.15.100:0/4212766146 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562c743e9650 con 0x562c743d24d0 2024-05-10T11:26:13.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.801+0000 7f55737fe700 1 -- 172.21.15.100:0/4212766146 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f558c0074e0 con 0x562c743d24d0 2024-05-10T11:26:13.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.802+0000 7f55737fe700 1 -- 172.21.15.100:0/4212766146 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f55840a8d40 con 0x562c743d24d0 2024-05-10T11:26:13.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.802+0000 7f55a4e9fb80 10 client.26392 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:26:13.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.802+0000 7f55a4e9fb80 10 client.26392 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:26:13.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.802+0000 7f55a4e9fb80 20 client.26392 populate_metadata read hostname 'smithi100' 2024-05-10T11:26:13.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.802+0000 7f55a4e9fb80 1 --2- 172.21.15.100:0/4212766146 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562c743ebb20 0x562c743edf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:13.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.802+0000 7f55a4e9fb80 4 client.26392 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:26:13.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.802+0000 7f55a4e9fb80 1 -- 172.21.15.100:0/4212766146 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c7439e650 con 0x562c743ebb20 2024-05-10T11:26:13.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.802+0000 7f55937fe700 1 --2- 172.21.15.100:0/4212766146 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562c743ebb20 0x562c743edf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:13.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.802+0000 7f55737fe700 10 client.26392 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:26:13.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.802+0000 7f55937fe700 1 --2- 172.21.15.100:0/4212766146 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562c743ebb20 0x562c743edf70 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a398a81a6840323 server_cookie=1becdd7ac7ce54da in_seq=0 out_seq=0 2024-05-10T11:26:13.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.803+0000 7f55737fe700 1 -- 172.21.15.100:0/4212766146 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562c7439e650 con 0x562c743ebb20 2024-05-10T11:26:13.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.803+0000 7f55737fe700 10 client.26392 handle_command_reply: tid=0 2024-05-10T11:26:13.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.803+0000 7f55a4e9fb80 1 client.26392 shutdown 2024-05-10T11:26:13.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.803+0000 7f55a4e9fb80 1 -- 172.21.15.100:0/4212766146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c743d24d0 msgr2=0x562c743de020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:13.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.803+0000 7f55a4e9fb80 1 --2- 172.21.15.100:0/4212766146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c743d24d0 0x562c743de020 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f55840dffc0 tx=0x7f55840165b0 comp rx=0 tx=0).stop 2024-05-10T11:26:13.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.803+0000 7f55a4e9fb80 1 -- 172.21.15.100:0/4212766146 shutdown_connections 2024-05-10T11:26:13.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.803+0000 7f55a4e9fb80 1 --2- 172.21.15.100:0/4212766146 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562c743ebb20 0x562c743edf70 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:13.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.803+0000 7f55a4e9fb80 1 --2- 172.21.15.100:0/4212766146 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c743d6c80 0x562c743db900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:13.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.803+0000 7f55a4e9fb80 1 --2- 172.21.15.100:0/4212766146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c743d24d0 0x562c743de020 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:13.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.803+0000 7f55a4e9fb80 1 --2- 172.21.15.100:0/4212766146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c743d2df0 0x562c743db3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:13.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.803+0000 7f55737fe700 0 client.26392 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:26:13.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.803+0000 7f55a4e9fb80 1 -- 172.21.15.100:0/4212766146 >> 172.21.15.100:0/4212766146 conn(0x562c742fe100 msgr2=0x562c743d7080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:26:13.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.803+0000 7f55a4e9fb80 1 -- 172.21.15.100:0/4212766146 shutdown_connections 2024-05-10T11:26:13.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.803+0000 7f55a4e9fb80 1 -- 172.21.15.100:0/4212766146 wait complete. 2024-05-10T11:26:13.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.803+0000 7f55a4e9fb80 20 client.26392 trim_cache size 0 max 16384 2024-05-10T11:26:13.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.825+0000 7f55a4e9fb80 1 Processor -- start 2024-05-10T11:26:13.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.826+0000 7f55a4e9fb80 1 -- start start 2024-05-10T11:26:13.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.826+0000 7f55a4e9fb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c742fe100 0x562c743e7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:13.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.826+0000 7f55a4e9fb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c743d24d0 0x562c743dd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:13.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.826+0000 7f55a4e9fb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c743d2df0 0x562c743d45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:13.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.826+0000 7f55a4e9fb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f55941b3a10 con 0x562c743d24d0 2024-05-10T11:26:13.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.826+0000 7f55a4e9fb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f55941b12a0 con 0x562c743d2df0 2024-05-10T11:26:13.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.826+0000 7f55a4e9fb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f55941af0f0 con 0x562c742fe100 2024-05-10T11:26:13.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.826+0000 7f55937fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c743d24d0 0x562c743dd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:13.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.826+0000 7f5593fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c742fe100 0x562c743e7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:13.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.826+0000 7f55937fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c743d24d0 0x562c743dd180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49294/0 (socket says 172.21.15.100:49294) 2024-05-10T11:26:13.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.826+0000 7f5593fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c742fe100 0x562c743e7ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:60280/0 (socket says 172.21.15.100:60280) 2024-05-10T11:26:13.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.826+0000 7f55937fe700 1 -- 172.21.15.100:0/3798898242 learned_addr learned my addr 172.21.15.100:0/3798898242 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:26:13.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.826+0000 7f5598ba9700 1 --2- 172.21.15.100:0/3798898242 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c743d2df0 0x562c743d45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:13.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.826+0000 7f55937fe700 1 -- 172.21.15.100:0/3798898242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c742fe100 msgr2=0x562c743e7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:13.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.826+0000 7f55937fe700 1 --2- 172.21.15.100:0/3798898242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c742fe100 0x562c743e7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:13.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.826+0000 7f55937fe700 1 -- 172.21.15.100:0/3798898242 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c743d2df0 msgr2=0x562c743d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:13.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.826+0000 7f55937fe700 1 --2- 172.21.15.100:0/3798898242 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c743d2df0 0x562c743d45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:13.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.826+0000 7f55937fe700 1 -- 172.21.15.100:0/3798898242 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c743de390 con 0x562c743d24d0 2024-05-10T11:26:13.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.826+0000 7f55937fe700 1 --2- 172.21.15.100:0/3798898242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c743d24d0 0x562c743dd180 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f5588005d90 tx=0x7f5588013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8342e5ab2cdf369d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:26:13.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.827+0000 7f5579ffb700 1 -- 172.21.15.100:0/3798898242 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558800bdb0 con 0x562c743d24d0 2024-05-10T11:26:13.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.827+0000 7f5579ffb700 1 -- 172.21.15.100:0/3798898242 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5588004830 con 0x562c743d24d0 2024-05-10T11:26:13.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.827+0000 7f5579ffb700 1 -- 172.21.15.100:0/3798898242 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558802a7a0 con 0x562c743d24d0 2024-05-10T11:26:13.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.827+0000 7f55a4e9fb80 1 -- 172.21.15.100:0/3798898242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c743d24d0 msgr2=0x562c743dd180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:13.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.827+0000 7f55a4e9fb80 1 --2- 172.21.15.100:0/3798898242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c743d24d0 0x562c743dd180 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f5588005d90 tx=0x7f5588013000 comp rx=0 tx=0).stop 2024-05-10T11:26:13.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.827+0000 7f55a4e9fb80 1 -- 172.21.15.100:0/3798898242 shutdown_connections 2024-05-10T11:26:13.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.827+0000 7f55a4e9fb80 1 --2- 172.21.15.100:0/3798898242 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c743d2df0 0x562c743d45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:13.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.827+0000 7f55a4e9fb80 1 --2- 172.21.15.100:0/3798898242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c742fe100 0x562c743e7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:13.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.827+0000 7f55a4e9fb80 1 --2- 172.21.15.100:0/3798898242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c743d24d0 0x562c743dd180 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:13.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.827+0000 7f55a4e9fb80 1 -- 172.21.15.100:0/3798898242 >> 172.21.15.100:0/3798898242 conn(0x562c743c7ab0 msgr2=0x562c743dca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:26:13.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.827+0000 7f55a4e9fb80 1 -- 172.21.15.100:0/3798898242 shutdown_connections 2024-05-10T11:26:13.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.828+0000 7f55a4e9fb80 1 -- 172.21.15.100:0/3798898242 wait complete. 2024-05-10T11:26:13.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.828+0000 7f55a4e9fb80 1 -- start start 2024-05-10T11:26:13.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.828+0000 7f55a4e9fb80 1 Processor -- start 2024-05-10T11:26:13.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.828+0000 7f55a4e9fb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c742fe100 0x562c743e05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:13.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.828+0000 7f55a4e9fb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c743d24d0 0x562c743e0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:13.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.828+0000 7f55a4e9fb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c743d2df0 0x562c744553f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:13.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.828+0000 7f55a4e9fb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f55941af760 con 0x562c742fe100 2024-05-10T11:26:13.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.828+0000 7f55a4e9fb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f55941b0bd0 con 0x562c743d2df0 2024-05-10T11:26:13.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.828+0000 7f55a4e9fb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f55941afe30 con 0x562c743d24d0 2024-05-10T11:26:13.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.828+0000 7f55a4e9fb80 10 client.0 _ll_register_callbacks cb 0x562c742bc920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:26:13.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.828+0000 7f5593fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c742fe100 0x562c743e05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:13.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.828+0000 7f55937fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c743d24d0 0x562c743e0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:13.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.829+0000 7f55937fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c743d24d0 0x562c743e0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:60286/0 (socket says 172.21.15.100:60286) 2024-05-10T11:26:13.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.829+0000 7f5598ba9700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c743d2df0 0x562c744553f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:13.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.829+0000 7f55937fe700 1 -- 172.21.15.100:0/236511018 learned_addr learned my addr 172.21.15.100:0/236511018 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:26:13.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.829+0000 7f5593fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c742fe100 0x562c743e05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49310/0 (socket says 172.21.15.100:49310) 2024-05-10T11:26:13.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.829+0000 7f55937fe700 1 -- 172.21.15.100:0/236511018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c743d2df0 msgr2=0x562c744553f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:13.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.829+0000 7f55937fe700 1 --2- 172.21.15.100:0/236511018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c743d2df0 0x562c744553f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:13.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.829+0000 7f55937fe700 1 -- 172.21.15.100:0/236511018 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c742fe100 msgr2=0x562c743e05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:13.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.829+0000 7f55937fe700 1 --2- 172.21.15.100:0/236511018 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c742fe100 0x562c743e05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:13.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.829+0000 7f55937fe700 1 -- 172.21.15.100:0/236511018 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c74455a30 con 0x562c743d24d0 2024-05-10T11:26:13.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.829+0000 7f5593fff700 1 --2- 172.21.15.100:0/236511018 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c742fe100 0x562c743e05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:26:13.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.829+0000 7f55937fe700 1 --2- 172.21.15.100:0/236511018 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c743d24d0 0x562c743e0b10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5588005f90 tx=0x7f558800bde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58aeb30d88210bf7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:26:13.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.829+0000 7f55737fe700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:26:13.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.829+0000 7f55737fe700 1 -- 172.21.15.100:0/236511018 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5588038070 con 0x562c743d24d0 2024-05-10T11:26:13.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.829+0000 7f55a4e9fb80 1 -- 172.21.15.100:0/236511018 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5588004830 con 0x562c743d24d0 2024-05-10T11:26:13.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.829+0000 7f55737fe700 1 -- 172.21.15.100:0/236511018 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f55880049e0 con 0x562c743d24d0 2024-05-10T11:26:13.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.829+0000 7f55737fe700 1 -- 172.21.15.100:0/236511018 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5588004be0 con 0x562c743d24d0 2024-05-10T11:26:13.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.829+0000 7f55a4e9fb80 10 client.26398 fetch_fsmap learned FSMap version 31 2024-05-10T11:26:13.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.829+0000 7f55a4e9fb80 1 -- 172.21.15.100:0/236511018 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562c74456d20 con 0x562c743d24d0 2024-05-10T11:26:13.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.830+0000 7f55737fe700 1 -- 172.21.15.100:0/236511018 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5588004830 con 0x562c743d24d0 2024-05-10T11:26:13.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.830+0000 7f55737fe700 1 -- 172.21.15.100:0/236511018 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f5588005200 con 0x562c743d24d0 2024-05-10T11:26:13.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.830+0000 7f55a4e9fb80 10 client.26398 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:26:13.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.830+0000 7f55a4e9fb80 10 client.26398 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:26:13.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.830+0000 7f55a4e9fb80 20 client.26398 populate_metadata read hostname 'smithi100' 2024-05-10T11:26:13.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.830+0000 7f55a4e9fb80 1 --2- 172.21.15.100:0/236511018 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562c743d6c80 0x562c74459480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:13.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.830+0000 7f55a4e9fb80 4 client.26398 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T11:26:13.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.830+0000 7f55a4e9fb80 1 -- 172.21.15.100:0/236511018 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562c74332240 con 0x562c743d6c80 2024-05-10T11:26:13.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.830+0000 7f5593fff700 1 --2- 172.21.15.100:0/236511018 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562c743d6c80 0x562c74459480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:13.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.830+0000 7f55737fe700 10 client.26398 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:26:13.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.830+0000 7f5593fff700 1 --2- 172.21.15.100:0/236511018 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562c743d6c80 0x562c74459480 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92ea274249c8c2f7 server_cookie=48f8f023fad927db in_seq=0 out_seq=0 2024-05-10T11:26:13.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.831+0000 7f55737fe700 1 -- 172.21.15.100:0/236511018 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562c74332240 con 0x562c743d6c80 2024-05-10T11:26:13.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.831+0000 7f55737fe700 10 client.26398 handle_command_reply: tid=0 2024-05-10T11:26:13.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.831+0000 7f55a4e9fb80 1 client.26398 shutdown 2024-05-10T11:26:13.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.831+0000 7f55a4e9fb80 1 -- 172.21.15.100:0/236511018 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c743d24d0 msgr2=0x562c743e0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:13.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.831+0000 7f55a4e9fb80 1 --2- 172.21.15.100:0/236511018 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c743d24d0 0x562c743e0b10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5588005f90 tx=0x7f558800bde0 comp rx=0 tx=0).stop 2024-05-10T11:26:13.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.831+0000 7f55a4e9fb80 1 -- 172.21.15.100:0/236511018 shutdown_connections 2024-05-10T11:26:13.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.831+0000 7f55a4e9fb80 1 --2- 172.21.15.100:0/236511018 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562c743d6c80 0x562c74459480 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:13.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.832+0000 7f55a4e9fb80 1 --2- 172.21.15.100:0/236511018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c743d2df0 0x562c744553f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:13.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.832+0000 7f55a4e9fb80 1 --2- 172.21.15.100:0/236511018 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c742fe100 0x562c743e05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:13.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.832+0000 7f55a4e9fb80 1 --2- 172.21.15.100:0/236511018 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c743d24d0 0x562c743e0b10 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:13.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.832+0000 7f55a4e9fb80 1 -- 172.21.15.100:0/236511018 >> 172.21.15.100:0/236511018 conn(0x562c743c7ab0 msgr2=0x562c743d1b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:26:13.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.832+0000 7f55737fe700 0 client.26398 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:26:13.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.832+0000 7f55a4e9fb80 1 -- 172.21.15.100:0/236511018 shutdown_connections 2024-05-10T11:26:13.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.832+0000 7f55a4e9fb80 1 -- 172.21.15.100:0/236511018 wait complete. 2024-05-10T11:26:13.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.832+0000 7f55a4e9fb80 20 client.26398 trim_cache size 0 max 16384 2024-05-10T11:26:13.836 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:26:13.836 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-05-10T11:26:13.836 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-05-10T11:26:13.836 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:26:13.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.832+0000 7f559a60c700 1 -- 172.21.15.100:0/1767965963 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f557c0b2880 msgr2=0x7f557c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:13.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.832+0000 7f559a60c700 1 --2- 172.21.15.100:0/1767965963 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f557c0b2880 0x7f557c0b4d40 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5594079a30 tx=0x7f558800b410 comp rx=0 tx=0).stop 2024-05-10T11:26:13.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.832+0000 7f559a60c700 1 -- 172.21.15.100:0/1767965963 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f559406b180 msgr2=0x7f5594102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:13.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.832+0000 7f559a60c700 1 --2- 172.21.15.100:0/1767965963 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f559406b180 0x7f5594102470 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f558400e9c0 tx=0x7f558400ecd0 comp rx=0 tx=0).stop 2024-05-10T11:26:13.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.833+0000 7f559a60c700 1 -- 172.21.15.100:0/1767965963 shutdown_connections 2024-05-10T11:26:13.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.833+0000 7f559a60c700 1 --2- 172.21.15.100:0/1767965963 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f557c0b2880 0x7f557c0b4d40 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:13.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.833+0000 7f559a60c700 1 --2- 172.21.15.100:0/1767965963 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5594109510 0x7f5594102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:13.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.833+0000 7f559a60c700 1 --2- 172.21.15.100:0/1767965963 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f559406b180 0x7f5594102470 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:13.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.833+0000 7f559a60c700 1 --2- 172.21.15.100:0/1767965963 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f559406bb40 0x7f55941029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:13.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.833+0000 7f559a60c700 1 -- 172.21.15.100:0/1767965963 >> 172.21.15.100:0/1767965963 conn(0x7f5594079350 msgr2=0x7f55941011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:26:13.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.833+0000 7f559a60c700 1 -- 172.21.15.100:0/1767965963 shutdown_connections 2024-05-10T11:26:13.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:13.833+0000 7f559a60c700 1 -- 172.21.15.100:0/1767965963 wait complete. 2024-05-10T11:26:13.975 INFO:tasks.workunit.client.0.smithi100.stdout: 1 190440 2.73 MB/sec execute 248 sec latency 35.819 ms 2024-05-10T11:26:14.519 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T11:26:14.519 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T11:26:14.827 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:26:14.975 INFO:tasks.workunit.client.0.smithi100.stdout: 1 191212 2.74 MB/sec execute 249 sec latency 11.700 ms 2024-05-10T11:26:15.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:26:15 smithi100 ceph-mon[30296]: pgmap v656: 129 pgs: 129 active+clean; 82 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 1.4 MiB/s wr, 171 op/s 2024-05-10T11:26:15.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:26:15 smithi178 ceph-mon[33359]: pgmap v656: 129 pgs: 129 active+clean; 82 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 1.4 MiB/s wr, 171 op/s 2024-05-10T11:26:15.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:26:15 smithi188 ceph-mon[34075]: pgmap v656: 129 pgs: 129 active+clean; 82 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 1.4 MiB/s wr, 171 op/s 2024-05-10T11:26:15.861 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.861+0000 7f9f410d1640 1 -- 172.21.15.100:0/612822924 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f3c105e20 msgr2=0x7f9f3c108210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:15.861 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.861+0000 7f9f410d1640 1 --2- 172.21.15.100:0/612822924 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f3c105e20 0x7f9f3c108210 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f9f24009900 tx=0x7f9f2402a410 comp rx=0 tx=0).stop 2024-05-10T11:26:15.861 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.861+0000 7f9f410d1640 1 -- 172.21.15.100:0/612822924 shutdown_connections 2024-05-10T11:26:15.861 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.861+0000 7f9f410d1640 1 --2- 172.21.15.100:0/612822924 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f3c108750 0x7f9f3c10ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:15.861 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.861+0000 7f9f410d1640 1 --2- 172.21.15.100:0/612822924 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f3c105e20 0x7f9f3c108210 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:15.861 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.861+0000 7f9f410d1640 1 --2- 172.21.15.100:0/612822924 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f3c1034f0 0x7f9f3c1058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:15.861 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.861+0000 7f9f410d1640 1 -- 172.21.15.100:0/612822924 >> 172.21.15.100:0/612822924 conn(0x7f9f3c0fce80 msgr2=0x7f9f3c0ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:26:15.862 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.861+0000 7f9f410d1640 1 -- 172.21.15.100:0/612822924 shutdown_connections 2024-05-10T11:26:15.862 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.861+0000 7f9f410d1640 1 -- 172.21.15.100:0/612822924 wait complete. 2024-05-10T11:26:15.864 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.862+0000 7f9f410d1640 1 Processor -- start 2024-05-10T11:26:15.864 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.862+0000 7f9f410d1640 1 -- start start 2024-05-10T11:26:15.864 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.862+0000 7f9f410d1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f3c1034f0 0x7f9f3c198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:15.864 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.862+0000 7f9f410d1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f3c105e20 0x7f9f3c199000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:15.864 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.862+0000 7f9f410d1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f3c108750 0x7f9f3c1a0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:15.864 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.862+0000 7f9f410d1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3c10e3f0 con 0x7f9f3c108750 2024-05-10T11:26:15.864 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.862+0000 7f9f410d1640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3c10e270 con 0x7f9f3c1034f0 2024-05-10T11:26:15.864 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.862+0000 7f9f410d1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3c10e570 con 0x7f9f3c105e20 2024-05-10T11:26:15.864 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.862+0000 7f9f3b59e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f3c108750 0x7f9f3c1a0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:15.864 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.862+0000 7f9f3b59e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f3c108750 0x7f9f3c1a0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49320/0 (socket says 172.21.15.100:49320) 2024-05-10T11:26:15.864 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.862+0000 7f9f3b59e640 1 -- 172.21.15.100:0/2489998172 learned_addr learned my addr 172.21.15.100:0/2489998172 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:26:15.864 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.862+0000 7f9f33fff640 1 --2- 172.21.15.100:0/2489998172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f3c105e20 0x7f9f3c199000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:15.864 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.862+0000 7f9f3ad9d640 1 --2- 172.21.15.100:0/2489998172 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f3c1034f0 0x7f9f3c198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.862+0000 7f9f3b59e640 1 -- 172.21.15.100:0/2489998172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f3c105e20 msgr2=0x7f9f3c199000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.862+0000 7f9f3b59e640 1 --2- 172.21.15.100:0/2489998172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f3c105e20 0x7f9f3c199000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.862+0000 7f9f3b59e640 1 -- 172.21.15.100:0/2489998172 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f3c1034f0 msgr2=0x7f9f3c198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.862+0000 7f9f3b59e640 1 --2- 172.21.15.100:0/2489998172 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f3c1034f0 0x7f9f3c198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.862+0000 7f9f3b59e640 1 -- 172.21.15.100:0/2489998172 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f3c1a06e0 con 0x7f9f3c108750 2024-05-10T11:26:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.862+0000 7f9f3ad9d640 1 --2- 172.21.15.100:0/2489998172 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f3c1034f0 0x7f9f3c198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:26:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.863+0000 7f9f3b59e640 1 --2- 172.21.15.100:0/2489998172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f3c108750 0x7f9f3c1a0070 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f9f3400eb10 tx=0x7f9f3400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:26:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.863+0000 7f9f38d99640 1 -- 172.21.15.100:0/2489998172 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f34004b20 con 0x7f9f3c108750 2024-05-10T11:26:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.863+0000 7f9f410d1640 1 -- 172.21.15.100:0/2489998172 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f3c1a09d0 con 0x7f9f3c108750 2024-05-10T11:26:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.863+0000 7f9f38d99640 1 -- 172.21.15.100:0/2489998172 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9f34004cc0 con 0x7f9f3c108750 2024-05-10T11:26:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.863+0000 7f9f38d99640 1 -- 172.21.15.100:0/2489998172 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f34010690 con 0x7f9f3c108750 2024-05-10T11:26:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.863+0000 7f9f410d1640 1 -- 172.21.15.100:0/2489998172 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f3c1a0ef0 con 0x7f9f3c108750 2024-05-10T11:26:15.872 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.866+0000 7f9f43deb740 1 Processor -- start 2024-05-10T11:26:15.872 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.866+0000 7f9f38d99640 1 -- 172.21.15.100:0/2489998172 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f9f34005020 con 0x7f9f3c108750 2024-05-10T11:26:15.872 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.866+0000 7f9f43deb740 1 -- start start 2024-05-10T11:26:15.872 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.866+0000 7f9f43deb740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bfbb9c91c0 0x55bfbb9c95e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:15.872 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.866+0000 7f9f43deb740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bfbb9c9bb0 0x55bfbb9ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:15.872 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.866+0000 7f9f43deb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfbb9ced20 0x55bfbb9d11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:15.872 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.866+0000 7f9f43deb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55bfbb9295d0 con 0x55bfbb9ced20 2024-05-10T11:26:15.872 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.866+0000 7f9f43deb740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55bfbb8e2890 con 0x55bfbb9c91c0 2024-05-10T11:26:15.872 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.866+0000 7f9f43deb740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55bfbb925590 con 0x55bfbb9c9bb0 2024-05-10T11:26:15.872 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.866+0000 7f9f33fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfbb9ced20 0x55bfbb9d11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:15.872 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.866+0000 7f9f33fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bfbb9c91c0 0x55bfbb9c95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:15.873 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.866+0000 7f9f33fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bfbb9c91c0 0x55bfbb9c95e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:33310/0 (socket says 172.21.15.100:33310) 2024-05-10T11:26:15.873 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.866+0000 7f9f33fff640 1 -- 172.21.15.100:0/3465932900 learned_addr learned my addr 172.21.15.100:0/3465932900 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:26:15.873 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.866+0000 7f9f3ad9d640 1 --2- 172.21.15.100:0/3465932900 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bfbb9c9bb0 0x55bfbb9ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:15.873 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.867+0000 7f9f33fff640 1 -- 172.21.15.100:0/3465932900 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bfbb9c9bb0 msgr2=0x55bfbb9ce6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:15.873 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.867+0000 7f9f33fff640 1 --2- 172.21.15.100:0/3465932900 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bfbb9c9bb0 0x55bfbb9ce6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:15.873 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.867+0000 7f9f33fff640 1 -- 172.21.15.100:0/3465932900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfbb9ced20 msgr2=0x55bfbb9d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:15.873 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.867+0000 7f9f33fff640 1 --2- 172.21.15.100:0/3465932900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfbb9ced20 0x55bfbb9d11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:15.873 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.867+0000 7f9f38d99640 1 --2- 172.21.15.100:0/2489998172 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f9f280b28a0 0x7f9f280b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:15.873 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.867+0000 7f9f33fff640 1 -- 172.21.15.100:0/3465932900 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfbb63b210 con 0x55bfbb9c91c0 2024-05-10T11:26:15.873 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.867+0000 7f9f3ad9d640 1 --2- 172.21.15.100:0/2489998172 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f9f280b28a0 0x7f9f280b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:15.873 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.867+0000 7f9f33fff640 1 --2- 172.21.15.100:0/3465932900 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bfbb9c91c0 0x55bfbb9c95e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c06ab60 tx=0x7f9f24031400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8646b0667cb5c668 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:26:15.873 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.867+0000 7f9f38d99640 1 -- 172.21.15.100:0/2489998172 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f9f340e2b80 con 0x7f9f3c108750 2024-05-10T11:26:15.873 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.867+0000 7f9f317fa640 1 -- 172.21.15.100:0/3465932900 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f24035040 con 0x55bfbb9c91c0 2024-05-10T11:26:15.874 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.867+0000 7f9f317fa640 1 -- 172.21.15.100:0/3465932900 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9f240044c0 con 0x55bfbb9c91c0 2024-05-10T11:26:15.874 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.867+0000 7f9f317fa640 1 -- 172.21.15.100:0/3465932900 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f24003d70 con 0x55bfbb9c91c0 2024-05-10T11:26:15.874 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.868+0000 7f9f43deb740 1 -- 172.21.15.100:0/3465932900 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bfbb9c91c0 msgr2=0x55bfbb9c95e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:15.874 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.868+0000 7f9f3ad9d640 1 --2- 172.21.15.100:0/2489998172 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f9f280b28a0 0x7f9f280b4d60 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x55bfbb9c9fb0 tx=0x7f9f2c00b5a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:26:15.874 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.868+0000 7f9f43deb740 1 --2- 172.21.15.100:0/3465932900 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bfbb9c91c0 0x55bfbb9c95e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c06ab60 tx=0x7f9f24031400 comp rx=0 tx=0).stop 2024-05-10T11:26:15.874 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.868+0000 7f9f43deb740 1 -- 172.21.15.100:0/3465932900 shutdown_connections 2024-05-10T11:26:15.874 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.868+0000 7f9f43deb740 1 --2- 172.21.15.100:0/3465932900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfbb9ced20 0x55bfbb9d11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:15.874 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.868+0000 7f9f43deb740 1 --2- 172.21.15.100:0/3465932900 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bfbb9c9bb0 0x55bfbb9ce6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:15.874 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.868+0000 7f9f43deb740 1 --2- 172.21.15.100:0/3465932900 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bfbb9c91c0 0x55bfbb9c95e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:15.874 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.868+0000 7f9f43deb740 1 -- 172.21.15.100:0/3465932900 >> 172.21.15.100:0/3465932900 conn(0x55bfbb9c0420 msgr2=0x55bfbb9c0820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:26:15.874 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.868+0000 7f9f43deb740 1 -- 172.21.15.100:0/3465932900 shutdown_connections 2024-05-10T11:26:15.874 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.868+0000 7f9f43deb740 1 -- 172.21.15.100:0/3465932900 wait complete. 2024-05-10T11:26:15.874 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.868+0000 7f9f43deb740 1 -- start start 2024-05-10T11:26:15.874 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.869+0000 7f9f43deb740 1 Processor -- start 2024-05-10T11:26:15.874 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.869+0000 7f9f43deb740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bfbb9c91c0 0x55bfbb9d3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:15.875 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.869+0000 7f9f43deb740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bfbb9c9bb0 0x55bfbb9d3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:15.875 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.869+0000 7f9f43deb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfbb9ced20 0x55bfbb9da9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:15.875 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.869+0000 7f9f43deb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55bfbb9d1c70 con 0x55bfbb9ced20 2024-05-10T11:26:15.875 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.869+0000 7f9f43deb740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55bfbb8e6370 con 0x55bfbb9c91c0 2024-05-10T11:26:15.875 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.869+0000 7f9f43deb740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55bfbb9d1df0 con 0x55bfbb9c9bb0 2024-05-10T11:26:15.875 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.869+0000 7f9f43deb740 10 client.0 _ll_register_callbacks cb 0x55bfbb63aa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:26:15.875 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.869+0000 7f9f3b59e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfbb9ced20 0x55bfbb9da9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:15.875 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.869+0000 7f9f3ad9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bfbb9c91c0 0x55bfbb9d3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:15.875 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.869+0000 7f9f3b59e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfbb9ced20 0x55bfbb9da9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49334/0 (socket says 172.21.15.100:49334) 2024-05-10T11:26:15.875 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.869+0000 7f9f3ad9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bfbb9c91c0 0x55bfbb9d3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:33314/0 (socket says 172.21.15.100:33314) 2024-05-10T11:26:15.875 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.869+0000 7f9f3b59e640 1 -- 172.21.15.100:0/1263675148 learned_addr learned my addr 172.21.15.100:0/1263675148 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:26:15.875 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.869+0000 7f9f33fff640 1 --2- 172.21.15.100:0/1263675148 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bfbb9c9bb0 0x55bfbb9d3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:15.875 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.869+0000 7f9f3b59e640 1 -- 172.21.15.100:0/1263675148 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bfbb9c9bb0 msgr2=0x55bfbb9d3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:15.875 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.869+0000 7f9f3b59e640 1 --2- 172.21.15.100:0/1263675148 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bfbb9c9bb0 0x55bfbb9d3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:15.875 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.869+0000 7f9f3b59e640 1 -- 172.21.15.100:0/1263675148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bfbb9c91c0 msgr2=0x55bfbb9d3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:15.876 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.869+0000 7f9f3b59e640 1 --2- 172.21.15.100:0/1263675148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bfbb9c91c0 0x55bfbb9d3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:15.876 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.869+0000 7f9f3b59e640 1 -- 172.21.15.100:0/1263675148 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfbb707620 con 0x55bfbb9ced20 2024-05-10T11:26:15.876 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.869+0000 7f9f3b59e640 1 --2- 172.21.15.100:0/1263675148 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfbb9ced20 0x55bfbb9da9b0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f9f340a5ec0 tx=0x7f9f340a9790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=224d7aef9052970 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:26:15.876 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.869+0000 7f9f227fc640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:26:15.876 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.869+0000 7f9f227fc640 1 -- 172.21.15.100:0/1263675148 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f340ac030 con 0x55bfbb9ced20 2024-05-10T11:26:15.876 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.870+0000 7f9f227fc640 1 -- 172.21.15.100:0/1263675148 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f9f340a6430 con 0x55bfbb9ced20 2024-05-10T11:26:15.876 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.870+0000 7f9f43deb740 1 -- 172.21.15.100:0/1263675148 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bfbb9371e0 con 0x55bfbb9ced20 2024-05-10T11:26:15.876 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.870+0000 7f9f227fc640 1 -- 172.21.15.100:0/1263675148 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9f340aa540 con 0x55bfbb9ced20 2024-05-10T11:26:15.876 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.870+0000 7f9f43deb740 10 client.16935 fetch_fsmap learned FSMap version 31 2024-05-10T11:26:15.876 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.870+0000 7f9f43deb740 1 -- 172.21.15.100:0/1263675148 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55bfbb9ddab0 con 0x55bfbb9ced20 2024-05-10T11:26:15.876 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.870+0000 7f9f227fc640 1 -- 172.21.15.100:0/1263675148 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9f340a9e40 con 0x55bfbb9ced20 2024-05-10T11:26:15.876 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.870+0000 7f9f227fc640 1 -- 172.21.15.100:0/1263675148 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f9f340a57b0 con 0x55bfbb9ced20 2024-05-10T11:26:15.876 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.870+0000 7f9f43deb740 10 client.16935 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:26:15.876 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.870+0000 7f9f43deb740 10 client.16935 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:26:15.876 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.870+0000 7f9f43deb740 20 client.16935 populate_metadata read hostname 'smithi100' 2024-05-10T11:26:15.876 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.870+0000 7f9f43deb740 1 --2- 172.21.15.100:0/1263675148 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55bfbb9df180 0x55bfbb9e15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:15.877 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.870+0000 7f9f43deb740 4 client.16935 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:26:15.877 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.870+0000 7f9f43deb740 1 -- 172.21.15.100:0/1263675148 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bfbb9371e0 con 0x55bfbb9df180 2024-05-10T11:26:15.877 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.870+0000 7f9f3ad9d640 1 --2- 172.21.15.100:0/1263675148 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55bfbb9df180 0x55bfbb9e15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:15.877 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.871+0000 7f9f227fc640 10 client.16935 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:26:15.877 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.871+0000 7f9f3ad9d640 1 --2- 172.21.15.100:0/1263675148 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55bfbb9df180 0x55bfbb9e15d0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bf6f38a54039a0e server_cookie=3eaa82c2de3458f2 in_seq=0 out_seq=0 2024-05-10T11:26:15.877 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.871+0000 7f9f227fc640 1 -- 172.21.15.100:0/1263675148 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bfbb9371e0 con 0x55bfbb9df180 2024-05-10T11:26:15.877 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.871+0000 7f9f227fc640 10 client.16935 handle_command_reply: tid=0 2024-05-10T11:26:15.877 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.871+0000 7f9f43deb740 1 client.16935 shutdown 2024-05-10T11:26:15.877 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.871+0000 7f9f43deb740 1 -- 172.21.15.100:0/1263675148 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfbb9ced20 msgr2=0x55bfbb9da9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:15.877 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.871+0000 7f9f43deb740 1 --2- 172.21.15.100:0/1263675148 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfbb9ced20 0x55bfbb9da9b0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f9f340a5ec0 tx=0x7f9f340a9790 comp rx=0 tx=0).stop 2024-05-10T11:26:15.877 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.871+0000 7f9f43deb740 1 -- 172.21.15.100:0/1263675148 shutdown_connections 2024-05-10T11:26:15.877 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.871+0000 7f9f43deb740 1 --2- 172.21.15.100:0/1263675148 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55bfbb9df180 0x55bfbb9e15d0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:15.877 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.871+0000 7f9f43deb740 1 --2- 172.21.15.100:0/1263675148 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfbb9ced20 0x55bfbb9da9b0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:15.877 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.872+0000 7f9f43deb740 1 --2- 172.21.15.100:0/1263675148 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bfbb9c9bb0 0x55bfbb9d3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:15.877 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.872+0000 7f9f227fc640 0 client.16935 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:26:15.878 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.872+0000 7f9f43deb740 1 --2- 172.21.15.100:0/1263675148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bfbb9c91c0 0x55bfbb9d3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:15.878 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.872+0000 7f9f43deb740 1 -- 172.21.15.100:0/1263675148 >> 172.21.15.100:0/1263675148 conn(0x55bfbb9c0420 msgr2=0x55bfbb9bf580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:26:15.878 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.872+0000 7f9f43deb740 1 -- 172.21.15.100:0/1263675148 shutdown_connections 2024-05-10T11:26:15.878 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.872+0000 7f9f43deb740 1 -- 172.21.15.100:0/1263675148 wait complete. 2024-05-10T11:26:15.878 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.872+0000 7f9f43deb740 20 client.16935 trim_cache size 0 max 16384 2024-05-10T11:26:15.889 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.889+0000 7f9f43deb740 1 Processor -- start 2024-05-10T11:26:15.889 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.889+0000 7f9f43deb740 1 -- start start 2024-05-10T11:26:15.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.889+0000 7f9f43deb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfbb9c91c0 0x55bfbb9bb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:15.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.889+0000 7f9f43deb740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bfbb9c9bb0 0x55bfbb9cae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:15.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.889+0000 7f9f43deb740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bfbb9ced20 0x55bfbb9cb370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:15.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.889+0000 7f9f43deb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3c1ab940 con 0x55bfbb9c91c0 2024-05-10T11:26:15.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.889+0000 7f9f43deb740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3c1a9210 con 0x55bfbb9c9bb0 2024-05-10T11:26:15.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.889+0000 7f9f43deb740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3c1a7060 con 0x55bfbb9ced20 2024-05-10T11:26:15.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.889+0000 7f9f3ad9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfbb9c91c0 0x55bfbb9bb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:15.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.889+0000 7f9f3ad9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfbb9c91c0 0x55bfbb9bb3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49346/0 (socket says 172.21.15.100:49346) 2024-05-10T11:26:15.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.889+0000 7f9f3ad9d640 1 -- 172.21.15.100:0/3813763012 learned_addr learned my addr 172.21.15.100:0/3813763012 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:26:15.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.889+0000 7f9f3b59e640 1 --2- 172.21.15.100:0/3813763012 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bfbb9ced20 0x55bfbb9cb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:15.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.889+0000 7f9f33fff640 1 --2- 172.21.15.100:0/3813763012 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bfbb9c9bb0 0x55bfbb9cae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:15.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.890+0000 7f9f3ad9d640 1 -- 172.21.15.100:0/3813763012 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bfbb9ced20 msgr2=0x55bfbb9cb370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:15.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.890+0000 7f9f3ad9d640 1 --2- 172.21.15.100:0/3813763012 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bfbb9ced20 0x55bfbb9cb370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:15.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.890+0000 7f9f3ad9d640 1 -- 172.21.15.100:0/3813763012 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bfbb9c9bb0 msgr2=0x55bfbb9cae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:15.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.890+0000 7f9f3ad9d640 1 --2- 172.21.15.100:0/3813763012 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bfbb9c9bb0 0x55bfbb9cae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:15.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.890+0000 7f9f3ad9d640 1 -- 172.21.15.100:0/3813763012 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfbb9dd8d0 con 0x55bfbb9c91c0 2024-05-10T11:26:15.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.890+0000 7f9f3ad9d640 1 --2- 172.21.15.100:0/3813763012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfbb9c91c0 0x55bfbb9bb3e0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c005fd0 tx=0x7f9f2c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90d5ef42d748d27a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:26:15.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.890+0000 7f9f30ff9640 1 -- 172.21.15.100:0/3813763012 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c021ce0 con 0x55bfbb9c91c0 2024-05-10T11:26:15.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.890+0000 7f9f30ff9640 1 -- 172.21.15.100:0/3813763012 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9f2c02e070 con 0x55bfbb9c91c0 2024-05-10T11:26:15.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.890+0000 7f9f30ff9640 1 -- 172.21.15.100:0/3813763012 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c01f650 con 0x55bfbb9c91c0 2024-05-10T11:26:15.894 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.891+0000 7f9f43deb740 1 -- 172.21.15.100:0/3813763012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfbb9c91c0 msgr2=0x55bfbb9bb3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:15.895 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.891+0000 7f9f43deb740 1 --2- 172.21.15.100:0/3813763012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfbb9c91c0 0x55bfbb9bb3e0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c005fd0 tx=0x7f9f2c015000 comp rx=0 tx=0).stop 2024-05-10T11:26:15.895 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.891+0000 7f9f43deb740 1 -- 172.21.15.100:0/3813763012 shutdown_connections 2024-05-10T11:26:15.895 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.891+0000 7f9f43deb740 1 --2- 172.21.15.100:0/3813763012 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bfbb9ced20 0x55bfbb9cb370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:15.895 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.891+0000 7f9f43deb740 1 --2- 172.21.15.100:0/3813763012 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bfbb9c9bb0 0x55bfbb9cae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:15.895 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.891+0000 7f9f43deb740 1 --2- 172.21.15.100:0/3813763012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfbb9c91c0 0x55bfbb9bb3e0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:15.895 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.891+0000 7f9f43deb740 1 -- 172.21.15.100:0/3813763012 >> 172.21.15.100:0/3813763012 conn(0x55bfbb9c0420 msgr2=0x55bfbb9c0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:26:15.895 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.891+0000 7f9f43deb740 1 -- 172.21.15.100:0/3813763012 shutdown_connections 2024-05-10T11:26:15.895 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.891+0000 7f9f43deb740 1 -- 172.21.15.100:0/3813763012 wait complete. 2024-05-10T11:26:15.895 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.891+0000 7f9f43deb740 1 -- start start 2024-05-10T11:26:15.895 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.891+0000 7f9f43deb740 1 Processor -- start 2024-05-10T11:26:15.895 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.892+0000 7f9f43deb740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bfbb9c91c0 0x55bfbb9d73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:15.895 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.892+0000 7f9f43deb740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bfbb9c9bb0 0x55bfbb9d7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:15.895 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.892+0000 7f9f43deb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfbb9ced20 0x55bfbb9d8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:15.895 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.892+0000 7f9f43deb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3c1a76d0 con 0x55bfbb9ced20 2024-05-10T11:26:15.895 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.892+0000 7f9f43deb740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3c1a8b40 con 0x55bfbb9c9bb0 2024-05-10T11:26:15.896 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.892+0000 7f9f43deb740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3c1a7da0 con 0x55bfbb9c91c0 2024-05-10T11:26:15.896 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.892+0000 7f9f43deb740 10 client.0 _ll_register_callbacks cb 0x7f9f34004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:26:15.896 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.892+0000 7f9f3b59e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfbb9ced20 0x55bfbb9d8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:15.896 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.892+0000 7f9f3ad9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bfbb9c91c0 0x55bfbb9d73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:15.896 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.892+0000 7f9f3ad9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bfbb9c91c0 0x55bfbb9d73f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:60322/0 (socket says 172.21.15.100:60322) 2024-05-10T11:26:15.896 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.892+0000 7f9f3b59e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfbb9ced20 0x55bfbb9d8940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49360/0 (socket says 172.21.15.100:49360) 2024-05-10T11:26:15.896 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.892+0000 7f9f3ad9d640 1 -- 172.21.15.100:0/2413085597 learned_addr learned my addr 172.21.15.100:0/2413085597 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:26:15.896 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.892+0000 7f9f33fff640 1 --2- 172.21.15.100:0/2413085597 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bfbb9c9bb0 0x55bfbb9d7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:15.896 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.892+0000 7f9f3b59e640 1 -- 172.21.15.100:0/2413085597 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bfbb9c91c0 msgr2=0x55bfbb9d73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:15.896 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.892+0000 7f9f3b59e640 1 --2- 172.21.15.100:0/2413085597 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bfbb9c91c0 0x55bfbb9d73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:15.896 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.892+0000 7f9f3b59e640 1 -- 172.21.15.100:0/2413085597 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bfbb9c9bb0 msgr2=0x55bfbb9d7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:15.896 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.892+0000 7f9f3b59e640 1 --2- 172.21.15.100:0/2413085597 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bfbb9c9bb0 0x55bfbb9d7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:15.896 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.892+0000 7f9f3b59e640 1 -- 172.21.15.100:0/2413085597 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfbb9d8f30 con 0x55bfbb9ced20 2024-05-10T11:26:15.896 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.892+0000 7f9f3ad9d640 1 --2- 172.21.15.100:0/2413085597 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bfbb9c91c0 0x55bfbb9d73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:26:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.892+0000 7f9f3b59e640 1 --2- 172.21.15.100:0/2413085597 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfbb9ced20 0x55bfbb9d8940 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f9f340e0690 tx=0x7f9f3400c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4548ff1261a13ede server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:26:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.892+0000 7f9f227fc640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:26:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.892+0000 7f9f227fc640 1 -- 172.21.15.100:0/2413085597 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f340ac020 con 0x55bfbb9ced20 2024-05-10T11:26:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.892+0000 7f9f227fc640 1 -- 172.21.15.100:0/2413085597 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f9f340a8e60 con 0x55bfbb9ced20 2024-05-10T11:26:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.892+0000 7f9f43deb740 1 -- 172.21.15.100:0/2413085597 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f2c02e070 con 0x55bfbb9ced20 2024-05-10T11:26:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.892+0000 7f9f227fc640 1 -- 172.21.15.100:0/2413085597 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9f340a5d00 con 0x55bfbb9ced20 2024-05-10T11:26:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.892+0000 7f9f43deb740 10 client.16941 fetch_fsmap learned FSMap version 31 2024-05-10T11:26:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.892+0000 7f9f43deb740 1 -- 172.21.15.100:0/2413085597 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55bfbb9da2b0 con 0x55bfbb9ced20 2024-05-10T11:26:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.893+0000 7f9f227fc640 1 -- 172.21.15.100:0/2413085597 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9f2c02e070 con 0x55bfbb9ced20 2024-05-10T11:26:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.893+0000 7f9f227fc640 1 -- 172.21.15.100:0/2413085597 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f9f340af380 con 0x55bfbb9ced20 2024-05-10T11:26:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.893+0000 7f9f43deb740 10 client.16941 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:26:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.893+0000 7f9f43deb740 10 client.16941 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:26:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.893+0000 7f9f43deb740 20 client.16941 populate_metadata read hostname 'smithi100' 2024-05-10T11:26:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.893+0000 7f9f43deb740 1 --2- 172.21.15.100:0/2413085597 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55bfbb9df180 0x55bfbba4c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.893+0000 7f9f43deb740 4 client.16941 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:26:15.898 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.893+0000 7f9f43deb740 1 -- 172.21.15.100:0/2413085597 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bfbb931050 con 0x55bfbb9df180 2024-05-10T11:26:15.898 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.893+0000 7f9f3ad9d640 1 --2- 172.21.15.100:0/2413085597 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55bfbb9df180 0x55bfbba4c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:15.898 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.893+0000 7f9f227fc640 10 client.16941 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:26:15.898 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.893+0000 7f9f3ad9d640 1 --2- 172.21.15.100:0/2413085597 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55bfbb9df180 0x55bfbba4c000 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=464285ff59cd8760 server_cookie=d73fd3f8e1ef1e42 in_seq=0 out_seq=0 2024-05-10T11:26:15.914 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.913+0000 7f9f227fc640 1 -- 172.21.15.100:0/2413085597 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bfbb931050 con 0x55bfbb9df180 2024-05-10T11:26:15.914 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.913+0000 7f9f227fc640 10 client.16941 handle_command_reply: tid=0 2024-05-10T11:26:15.914 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.913+0000 7f9f43deb740 1 client.16941 shutdown 2024-05-10T11:26:15.914 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.913+0000 7f9f43deb740 1 -- 172.21.15.100:0/2413085597 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfbb9ced20 msgr2=0x55bfbb9d8940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:15.914 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.913+0000 7f9f43deb740 1 --2- 172.21.15.100:0/2413085597 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfbb9ced20 0x55bfbb9d8940 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f9f340e0690 tx=0x7f9f3400c9a0 comp rx=0 tx=0).stop 2024-05-10T11:26:15.914 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.913+0000 7f9f43deb740 1 -- 172.21.15.100:0/2413085597 shutdown_connections 2024-05-10T11:26:15.914 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.913+0000 7f9f43deb740 1 --2- 172.21.15.100:0/2413085597 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55bfbb9df180 0x55bfbba4c000 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:15.914 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.913+0000 7f9f43deb740 1 --2- 172.21.15.100:0/2413085597 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfbb9ced20 0x55bfbb9d8940 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:15.914 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.913+0000 7f9f227fc640 0 client.16941 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:26:15.914 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.913+0000 7f9f43deb740 1 --2- 172.21.15.100:0/2413085597 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bfbb9c9bb0 0x55bfbb9d7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:15.914 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.913+0000 7f9f43deb740 1 --2- 172.21.15.100:0/2413085597 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bfbb9c91c0 0x55bfbb9d73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:15.915 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.914+0000 7f9f43deb740 1 -- 172.21.15.100:0/2413085597 >> 172.21.15.100:0/2413085597 conn(0x55bfbb9c0420 msgr2=0x55bfbb9bb3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:26:15.915 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.914+0000 7f9f43deb740 1 -- 172.21.15.100:0/2413085597 shutdown_connections 2024-05-10T11:26:15.915 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.914+0000 7f9f43deb740 1 -- 172.21.15.100:0/2413085597 wait complete. 2024-05-10T11:26:15.915 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.914+0000 7f9f43deb740 20 client.16941 trim_cache size 0 max 16384 2024-05-10T11:26:15.915 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:26:15.915 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:26:15.915 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:26:15.915 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:26:15.916 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.914+0000 7f9f410d1640 1 -- 172.21.15.100:0/2489998172 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f9f280b28a0 msgr2=0x7f9f280b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:15.916 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.914+0000 7f9f410d1640 1 --2- 172.21.15.100:0/2489998172 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f9f280b28a0 0x7f9f280b4d60 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x55bfbb9c9fb0 tx=0x7f9f2c00b5a0 comp rx=0 tx=0).stop 2024-05-10T11:26:15.916 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.914+0000 7f9f410d1640 1 -- 172.21.15.100:0/2489998172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f3c108750 msgr2=0x7f9f3c1a0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:15.916 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.914+0000 7f9f410d1640 1 --2- 172.21.15.100:0/2489998172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f3c108750 0x7f9f3c1a0070 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f9f3400eb10 tx=0x7f9f3400c2d0 comp rx=0 tx=0).stop 2024-05-10T11:26:15.916 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.914+0000 7f9f410d1640 1 -- 172.21.15.100:0/2489998172 shutdown_connections 2024-05-10T11:26:15.916 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.914+0000 7f9f410d1640 1 --2- 172.21.15.100:0/2489998172 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f9f280b28a0 0x7f9f280b4d60 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:15.916 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.914+0000 7f9f410d1640 1 --2- 172.21.15.100:0/2489998172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f3c108750 0x7f9f3c1a0070 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:15.916 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.914+0000 7f9f410d1640 1 --2- 172.21.15.100:0/2489998172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f3c105e20 0x7f9f3c199000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:15.917 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.914+0000 7f9f410d1640 1 --2- 172.21.15.100:0/2489998172 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f3c1034f0 0x7f9f3c198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:15.917 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.914+0000 7f9f410d1640 1 -- 172.21.15.100:0/2489998172 >> 172.21.15.100:0/2489998172 conn(0x7f9f3c0fce80 msgr2=0x7f9f3c104050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:26:15.917 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.914+0000 7f9f410d1640 1 -- 172.21.15.100:0/2489998172 shutdown_connections 2024-05-10T11:26:15.917 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:15.914+0000 7f9f410d1640 1 -- 172.21.15.100:0/2489998172 wait complete. 2024-05-10T11:26:15.975 INFO:tasks.workunit.client.0.smithi100.stdout: 1 191669 2.75 MB/sec execute 250 sec latency 14.799 ms 2024-05-10T11:26:16.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:16.567+0000 7f2c2b8c9700 1 -- 172.21.15.100:0/1683429321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c24107760 msgr2=0x7f2c2410f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:16.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:16.567+0000 7f2c2b8c9700 1 --2- 172.21.15.100:0/1683429321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c24107760 0x7f2c2410f2e0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c00b0d0 tx=0x7f2c1c00b3e0 comp rx=0 tx=0).stop 2024-05-10T11:26:16.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:16.567+0000 7f2c2b8c9700 1 -- 172.21.15.100:0/1683429321 shutdown_connections 2024-05-10T11:26:16.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:16.567+0000 7f2c2b8c9700 1 --2- 172.21.15.100:0/1683429321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c24107760 0x7f2c2410f2e0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:16.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:16.567+0000 7f2c2b8c9700 1 --2- 172.21.15.100:0/1683429321 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c24102e00 0x7f2c241031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:16.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:16.567+0000 7f2c2b8c9700 1 --2- 172.21.15.100:0/1683429321 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c241037b0 0x7f2c24107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:16.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:16.567+0000 7f2c2b8c9700 1 -- 172.21.15.100:0/1683429321 >> 172.21.15.100:0/1683429321 conn(0x7f2c24078a10 msgr2=0x7f2c240ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:26:16.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:16.568+0000 7f2c2b8c9700 1 -- 172.21.15.100:0/1683429321 shutdown_connections 2024-05-10T11:26:16.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:16.568+0000 7f2c2b8c9700 1 -- 172.21.15.100:0/1683429321 wait complete. 2024-05-10T11:26:16.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:16.568+0000 7f2c2b8c9700 1 Processor -- start 2024-05-10T11:26:16.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:16.568+0000 7f2c2b8c9700 1 -- start start 2024-05-10T11:26:16.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:16.568+0000 7f2c2b8c9700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c24102e00 0x7f2c241a1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:16.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:16.568+0000 7f2c2b8c9700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c241037b0 0x7f2c241a1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:16.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:16.568+0000 7f2c2b8c9700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c24107760 0x7f2c2419b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:16.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:16.568+0000 7f2c2b8c9700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2c241123e0 con 0x7f2c24107760 2024-05-10T11:26:16.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:16.568+0000 7f2c2b8c9700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2c24112260 con 0x7f2c24102e00 2024-05-10T11:26:16.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:16.568+0000 7f2c2b8c9700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2c24112560 con 0x7f2c241037b0 2024-05-10T11:26:16.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:16.569+0000 7f2c28e64700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c241037b0 0x7f2c241a1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:16.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:16.569+0000 7f2c28e64700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c241037b0 0x7f2c241a1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:43366/0 (socket says 172.21.15.100:43366) 2024-05-10T11:26:16.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:16.569+0000 7f2c28e64700 1 -- 172.21.15.100:0/4194334254 learned_addr learned my addr 172.21.15.100:0/4194334254 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:26:16.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:16.569+0000 7f2c29665700 1 --2- 172.21.15.100:0/4194334254 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c24102e00 0x7f2c241a1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:16.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:16.569+0000 7f2c29e66700 1 --2- 172.21.15.100:0/4194334254 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c24107760 0x7f2c2419b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:16.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:16.569+0000 7f2c28e64700 1 -- 172.21.15.100:0/4194334254 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c24102e00 msgr2=0x7f2c241a1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:16.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:16.569+0000 7f2c28e64700 1 --2- 172.21.15.100:0/4194334254 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c24102e00 0x7f2c241a1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:16.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:16.569+0000 7f2c28e64700 1 -- 172.21.15.100:0/4194334254 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c24107760 msgr2=0x7f2c2419b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:16.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:16.569+0000 7f2c28e64700 1 --2- 172.21.15.100:0/4194334254 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c24107760 0x7f2c2419b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:16.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:16.569+0000 7f2c28e64700 1 -- 172.21.15.100:0/4194334254 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c2419c140 con 0x7f2c241037b0 2024-05-10T11:26:16.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:16.569+0000 7f2c29665700 1 --2- 172.21.15.100:0/4194334254 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c24102e00 0x7f2c241a1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:26:16.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:16.569+0000 7f2c28e64700 1 --2- 172.21.15.100:0/4194334254 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c241037b0 0x7f2c241a1d60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2c1400d980 tx=0x7f2c1400dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:26:16.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:16.569+0000 7f2c1a7fc700 1 -- 172.21.15.100:0/4194334254 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c14016070 con 0x7f2c241037b0 2024-05-10T11:26:16.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:16.569+0000 7f2c2b8c9700 1 -- 172.21.15.100:0/4194334254 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c2419c430 con 0x7f2c241037b0 2024-05-10T11:26:16.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:16.569+0000 7f2c1a7fc700 1 -- 172.21.15.100:0/4194334254 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2c14004830 con 0x7f2c241037b0 2024-05-10T11:26:16.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:16.569+0000 7f2c1a7fc700 1 -- 172.21.15.100:0/4194334254 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c14005440 con 0x7f2c241037b0 2024-05-10T11:26:16.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:16.569+0000 7f2c2b8c9700 1 -- 172.21.15.100:0/4194334254 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c241a86f0 con 0x7f2c241037b0 2024-05-10T11:26:16.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:16.570+0000 7f2c2b8c9700 1 -- 172.21.15.100:0/4194334254 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c2406a190 con 0x7f2c241037b0 2024-05-10T11:26:16.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:16.572+0000 7f2c1a7fc700 1 -- 172.21.15.100:0/4194334254 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f2c140055e0 con 0x7f2c241037b0 2024-05-10T11:26:16.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:16.572+0000 7f2c1a7fc700 1 --2- 172.21.15.100:0/4194334254 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2c100b2880 0x7f2c100b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:16.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:16.573+0000 7f2c29665700 1 --2- 172.21.15.100:0/4194334254 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2c100b2880 0x7f2c100b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:16.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:16.573+0000 7f2c1a7fc700 1 -- 172.21.15.100:0/4194334254 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f2c140e2990 con 0x7f2c241037b0 2024-05-10T11:26:16.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:16.573+0000 7f2c29665700 1 --2- 172.21.15.100:0/4194334254 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2c100b2880 0x7f2c100b4d40 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f2c2419cb20 tx=0x7f2c20009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:26:16.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:16.576+0000 7f2c1a7fc700 1 -- 172.21.15.100:0/4194334254 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f2c1401f020 con 0x7f2c241037b0 2024-05-10T11:26:16.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:16.872+0000 7f2c2b8c9700 1 -- 172.21.15.100:0/4194334254 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2c2402e510 con 0x7f2c241037b0 2024-05-10T11:26:16.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:16.873+0000 7f2c1a7fc700 1 -- 172.21.15.100:0/4194334254 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11894 (secure 0 0 0) 0x7f2c140aa350 con 0x7f2c241037b0 2024-05-10T11:26:16.873 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:26:16.873 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2072225012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2072225012},{"type":"v1","addr":"172.21.15.100:6835","nonce":2072225012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/401939540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":401939540},{"type":"v1","addr":"172.21.15.100:6837","nonce":401939540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14571,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/3946311025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3946311025},{"type":"v1","addr":"172.21.15.100:6839","nonce":3946311025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3736371660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3736371660},{"type":"v1","addr":"172.21.15.100:6841","nonce":3736371660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/2741810791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2741810791},{"type":"v1","addr":"172.21.15.178:6833","nonce":2741810791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1704891041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1704891041},{"type":"v1","addr":"172.21.15.188:6833","nonce":1704891041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T11:14:27.242452+0000","modified":"2024-05-10T11:23:19.557113+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24439,"mds_1":24437,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/2124076312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2124076312},{"type":"v1","addr":"172.21.15.178:6835","nonce":2124076312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/3224358425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3224358425},{"type":"v1","addr":"172.21.15.178:6837","nonce":3224358425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/515547690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":515547690},{"type":"v1","addr":"172.21.15.188:6835","nonce":515547690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/1943342620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1943342620},{"type":"v1","addr":"172.21.15.188:6837","nonce":1943342620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.178:6839/3760985404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3760985404},{"type":"v1","addr":"172.21.15.178:6839","nonce":3760985404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.188:6839/3194917070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":3194917070},{"type":"v1","addr":"172.21.15.188:6839","nonce":3194917070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24439,"qdb_cluster":[24439,24427,24437]},"id":1}]} 2024-05-10T11:26:16.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:16.875+0000 7f2c2b8c9700 1 -- 172.21.15.100:0/4194334254 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2c100b2880 msgr2=0x7f2c100b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:16.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:16.875+0000 7f2c2b8c9700 1 --2- 172.21.15.100:0/4194334254 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2c100b2880 0x7f2c100b4d40 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f2c2419cb20 tx=0x7f2c20009260 comp rx=0 tx=0).stop 2024-05-10T11:26:16.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:16.875+0000 7f2c2b8c9700 1 -- 172.21.15.100:0/4194334254 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c241037b0 msgr2=0x7f2c241a1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:16.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:16.875+0000 7f2c2b8c9700 1 --2- 172.21.15.100:0/4194334254 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c241037b0 0x7f2c241a1d60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2c1400d980 tx=0x7f2c1400dc90 comp rx=0 tx=0).stop 2024-05-10T11:26:16.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:16.876+0000 7f2c2b8c9700 1 -- 172.21.15.100:0/4194334254 shutdown_connections 2024-05-10T11:26:16.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:16.876+0000 7f2c2b8c9700 1 --2- 172.21.15.100:0/4194334254 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2c100b2880 0x7f2c100b4d40 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:16.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:16.876+0000 7f2c2b8c9700 1 --2- 172.21.15.100:0/4194334254 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c24107760 0x7f2c2419b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:16.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:16.876+0000 7f2c2b8c9700 1 --2- 172.21.15.100:0/4194334254 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c24102e00 0x7f2c241a1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:16.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:16.876+0000 7f2c2b8c9700 1 --2- 172.21.15.100:0/4194334254 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c241037b0 0x7f2c241a1d60 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:16.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:16.876+0000 7f2c2b8c9700 1 -- 172.21.15.100:0/4194334254 >> 172.21.15.100:0/4194334254 conn(0x7f2c24078a10 msgr2=0x7f2c2410da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:26:16.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:16.876+0000 7f2c2b8c9700 1 -- 172.21.15.100:0/4194334254 shutdown_connections 2024-05-10T11:26:16.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:16.876+0000 7f2c2b8c9700 1 -- 172.21.15.100:0/4194334254 wait complete. 2024-05-10T11:26:16.877 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 31 2024-05-10T11:26:16.975 INFO:tasks.workunit.client.0.smithi100.stdout: 1 191966 2.75 MB/sec execute 251 sec latency 16.144 ms 2024-05-10T11:26:17.536 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:26:17 smithi100 ceph-mon[30296]: pgmap v657: 129 pgs: 129 active+clean; 82 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1.4 MiB/s wr, 139 op/s 2024-05-10T11:26:17.536 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:26:17 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/4194334254' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:26:17.563 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-10T11:26:17.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:26:17 smithi178 ceph-mon[33359]: pgmap v657: 129 pgs: 129 active+clean; 82 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1.4 MiB/s wr, 139 op/s 2024-05-10T11:26:17.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:26:17 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/4194334254' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:26:17.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:26:17 smithi188 ceph-mon[34075]: pgmap v657: 129 pgs: 129 active+clean; 82 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1.4 MiB/s wr, 139 op/s 2024-05-10T11:26:17.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:26:17 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/4194334254' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:26:17.819 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:26:17.975 INFO:tasks.workunit.client.0.smithi100.stdout: 1 192305 2.74 MB/sec execute 252 sec latency 15.925 ms 2024-05-10T11:26:18.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:26:18 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:26:18.679 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:26:18 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:26:18.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:26:18 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:26:18.975 INFO:tasks.workunit.client.0.smithi100.stdout: 1 192802 2.73 MB/sec execute 253 sec latency 16.409 ms 2024-05-10T11:26:19.589 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:26:19 smithi100 ceph-mon[30296]: pgmap v658: 129 pgs: 129 active+clean; 83 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 1.4 MiB/s wr, 164 op/s 2024-05-10T11:26:19.590 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:26:19 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:26:19.590 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:26:19 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:26:19.590 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:26:19 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:26:19.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.699+0000 7f82f8a35700 1 -- 172.21.15.100:0/707351772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82f4103040 msgr2=0x7f82f4107000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:19.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.699+0000 7f82f8a35700 1 --2- 172.21.15.100:0/707351772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82f4103040 0x7f82f4107000 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f82e80099c0 tx=0x7f82e8009cd0 comp rx=0 tx=0).stop 2024-05-10T11:26:19.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.699+0000 7f82f8a35700 1 -- 172.21.15.100:0/707351772 shutdown_connections 2024-05-10T11:26:19.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.699+0000 7f82f8a35700 1 --2- 172.21.15.100:0/707351772 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82f41076e0 0x7f82f410f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:19.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.699+0000 7f82f8a35700 1 --2- 172.21.15.100:0/707351772 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82f4102690 0x7f82f4102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:19.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.699+0000 7f82f8a35700 1 --2- 172.21.15.100:0/707351772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82f4103040 0x7f82f4107000 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:19.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.699+0000 7f82f8a35700 1 -- 172.21.15.100:0/707351772 >> 172.21.15.100:0/707351772 conn(0x7f82f40783f0 msgr2=0x7f82f4078800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:26:19.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.699+0000 7f82f8a35700 1 -- 172.21.15.100:0/707351772 shutdown_connections 2024-05-10T11:26:19.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.699+0000 7f82f8a35700 1 -- 172.21.15.100:0/707351772 wait complete. 2024-05-10T11:26:19.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.700+0000 7f82f8a35700 1 Processor -- start 2024-05-10T11:26:19.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.700+0000 7f82f8a35700 1 -- start start 2024-05-10T11:26:19.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.700+0000 7f82f8a35700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82f4102690 0x7f82f419a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:19.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.700+0000 7f82f8a35700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82f4103040 0x7f82f419aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:19.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.700+0000 7f82f8a35700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82f41076e0 0x7f82f41a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:19.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.700+0000 7f82f8a35700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f82f4112400 con 0x7f82f41076e0 2024-05-10T11:26:19.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.700+0000 7f82f8a35700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f82f4112280 con 0x7f82f4103040 2024-05-10T11:26:19.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.700+0000 7f82f8a35700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f82f4112580 con 0x7f82f4102690 2024-05-10T11:26:19.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.700+0000 7f82f1d9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82f4103040 0x7f82f419aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:19.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.700+0000 7f82f1d9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82f4103040 0x7f82f419aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:54306/0 (socket says 172.21.15.100:54306) 2024-05-10T11:26:19.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.700+0000 7f82f1d9b700 1 -- 172.21.15.100:0/1118986109 learned_addr learned my addr 172.21.15.100:0/1118986109 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:26:19.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.700+0000 7f82f2d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82f41076e0 0x7f82f41a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:19.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.700+0000 7f82f259c700 1 --2- 172.21.15.100:0/1118986109 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82f4102690 0x7f82f419a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:19.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.701+0000 7f82f1d9b700 1 -- 172.21.15.100:0/1118986109 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82f4102690 msgr2=0x7f82f419a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:19.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.701+0000 7f82f1d9b700 1 --2- 172.21.15.100:0/1118986109 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82f4102690 0x7f82f419a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:19.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.701+0000 7f82f1d9b700 1 -- 172.21.15.100:0/1118986109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82f41076e0 msgr2=0x7f82f41a5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:19.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.701+0000 7f82f1d9b700 1 --2- 172.21.15.100:0/1118986109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82f41076e0 0x7f82f41a5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:19.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.701+0000 7f82f1d9b700 1 -- 172.21.15.100:0/1118986109 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82f41a5f50 con 0x7f82f4103040 2024-05-10T11:26:19.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.701+0000 7f82f2d9d700 1 --2- 172.21.15.100:0/1118986109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82f41076e0 0x7f82f41a5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:26:19.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.701+0000 7f82f259c700 1 --2- 172.21.15.100:0/1118986109 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82f4102690 0x7f82f419a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:26:19.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.701+0000 7f82f1d9b700 1 --2- 172.21.15.100:0/1118986109 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82f4103040 0x7f82f419aed0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f82e8005f50 tx=0x7f82e8004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:26:19.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.701+0000 7f82e37fe700 1 -- 172.21.15.100:0/1118986109 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e8026070 con 0x7f82f4103040 2024-05-10T11:26:19.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.701+0000 7f82f8a35700 1 -- 172.21.15.100:0/1118986109 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82f4198780 con 0x7f82f4103040 2024-05-10T11:26:19.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.701+0000 7f82e37fe700 1 -- 172.21.15.100:0/1118986109 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f82e8003c50 con 0x7f82f4103040 2024-05-10T11:26:19.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.701+0000 7f82e37fe700 1 -- 172.21.15.100:0/1118986109 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e800f950 con 0x7f82f4103040 2024-05-10T11:26:19.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.701+0000 7f82f8a35700 1 -- 172.21.15.100:0/1118986109 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82f4198cb0 con 0x7f82f4103040 2024-05-10T11:26:19.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.704+0000 7f82e37fe700 1 -- 172.21.15.100:0/1118986109 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f82e8029070 con 0x7f82f4103040 2024-05-10T11:26:19.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.705+0000 7f82e37fe700 1 --2- 172.21.15.100:0/1118986109 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f82dc0b2900 0x7f82dc0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:19.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.705+0000 7f82f259c700 1 --2- 172.21.15.100:0/1118986109 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f82dc0b2900 0x7f82dc0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:19.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.705+0000 7f82e37fe700 1 -- 172.21.15.100:0/1118986109 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f82e80e2d40 con 0x7f82f4103040 2024-05-10T11:26:19.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.705+0000 7f82f259c700 1 --2- 172.21.15.100:0/1118986109 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f82dc0b2900 0x7f82dc0b4dc0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f82e4009db0 tx=0x7f82e401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:26:19.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.705+0000 7f83032c8b80 1 Processor -- start 2024-05-10T11:26:19.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.706+0000 7f83032c8b80 1 -- start start 2024-05-10T11:26:19.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.706+0000 7f83032c8b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612dd8151b0 0x5612dd815590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:19.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.706+0000 7f83032c8b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612dd815ad0 0x5612dd80f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:19.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.706+0000 7f83032c8b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612dd80fd50 0x5612dd810190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:19.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.706+0000 7f83032c8b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5612dd761940 con 0x5612dd815ad0 2024-05-10T11:26:19.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.706+0000 7f83032c8b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5612dd6c6da0 con 0x5612dd80fd50 2024-05-10T11:26:19.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.706+0000 7f83032c8b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5612dd75d900 con 0x5612dd8151b0 2024-05-10T11:26:19.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.706+0000 7f82f1d9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612dd815ad0 0x5612dd80f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:19.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.706+0000 7f82f1d9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612dd815ad0 0x5612dd80f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49048/0 (socket says 172.21.15.100:49048) 2024-05-10T11:26:19.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.706+0000 7f82f1d9b700 1 -- 172.21.15.100:0/3868419125 learned_addr learned my addr 172.21.15.100:0/3868419125 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:26:19.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.706+0000 7f82f259c700 1 --2- 172.21.15.100:0/3868419125 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612dd8151b0 0x5612dd815590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:19.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.706+0000 7f82f2d9d700 1 --2- 172.21.15.100:0/3868419125 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612dd80fd50 0x5612dd810190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:19.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.707+0000 7f82f1d9b700 1 -- 172.21.15.100:0/3868419125 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612dd8151b0 msgr2=0x5612dd815590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:19.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.707+0000 7f82f1d9b700 1 --2- 172.21.15.100:0/3868419125 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612dd8151b0 0x5612dd815590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:19.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.707+0000 7f82f1d9b700 1 -- 172.21.15.100:0/3868419125 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612dd80fd50 msgr2=0x5612dd810190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:19.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.707+0000 7f82f1d9b700 1 --2- 172.21.15.100:0/3868419125 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612dd80fd50 0x5612dd810190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:19.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.707+0000 7f82f1d9b700 1 -- 172.21.15.100:0/3868419125 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612dd6ae8d0 con 0x5612dd815ad0 2024-05-10T11:26:19.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.707+0000 7f82f259c700 1 --2- 172.21.15.100:0/3868419125 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612dd8151b0 0x5612dd815590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:26:19.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.707+0000 7f82f2d9d700 1 --2- 172.21.15.100:0/3868419125 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612dd80fd50 0x5612dd810190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:26:19.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.707+0000 7f82f1d9b700 1 --2- 172.21.15.100:0/3868419125 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612dd815ad0 0x5612dd80f810 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f82e80a60b0 tx=0x7f82e800f460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=368a9ab0b13a0569 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:26:19.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.707+0000 7f82e0ff9700 1 -- 172.21.15.100:0/3868419125 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e80aa730 con 0x5612dd815ad0 2024-05-10T11:26:19.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.707+0000 7f82e0ff9700 1 -- 172.21.15.100:0/3868419125 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f82e80aa030 con 0x5612dd815ad0 2024-05-10T11:26:19.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.707+0000 7f82e0ff9700 1 -- 172.21.15.100:0/3868419125 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e80a94f0 con 0x5612dd815ad0 2024-05-10T11:26:19.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.708+0000 7f83032c8b80 1 -- 172.21.15.100:0/3868419125 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612dd815ad0 msgr2=0x5612dd80f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:19.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.708+0000 7f83032c8b80 1 --2- 172.21.15.100:0/3868419125 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612dd815ad0 0x5612dd80f810 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f82e80a60b0 tx=0x7f82e800f460 comp rx=0 tx=0).stop 2024-05-10T11:26:19.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.708+0000 7f83032c8b80 1 -- 172.21.15.100:0/3868419125 shutdown_connections 2024-05-10T11:26:19.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.708+0000 7f83032c8b80 1 --2- 172.21.15.100:0/3868419125 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612dd80fd50 0x5612dd810190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:19.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.708+0000 7f83032c8b80 1 --2- 172.21.15.100:0/3868419125 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612dd8151b0 0x5612dd815590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:19.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.708+0000 7f83032c8b80 1 --2- 172.21.15.100:0/3868419125 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612dd815ad0 0x5612dd80f810 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:19.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.708+0000 7f83032c8b80 1 -- 172.21.15.100:0/3868419125 >> 172.21.15.100:0/3868419125 conn(0x5612dd73bb70 msgr2=0x5612dd80a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:26:19.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.708+0000 7f83032c8b80 1 -- 172.21.15.100:0/3868419125 shutdown_connections 2024-05-10T11:26:19.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.708+0000 7f83032c8b80 1 -- 172.21.15.100:0/3868419125 wait complete. 2024-05-10T11:26:19.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.708+0000 7f83032c8b80 1 -- start start 2024-05-10T11:26:19.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.709+0000 7f83032c8b80 1 Processor -- start 2024-05-10T11:26:19.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.709+0000 7f83032c8b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612dd80fd50 0x5612dd81b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:19.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.709+0000 7f83032c8b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612dd8151b0 0x5612dd818af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:19.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.709+0000 7f83032c8b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612dd815ad0 0x5612dd819030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:19.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.709+0000 7f83032c8b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5612dd81c210 con 0x5612dd815ad0 2024-05-10T11:26:19.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.709+0000 7f83032c8b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5612dd71aed0 con 0x5612dd80fd50 2024-05-10T11:26:19.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.709+0000 7f83032c8b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5612dd81c390 con 0x5612dd8151b0 2024-05-10T11:26:19.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.709+0000 7f82f2d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612dd815ad0 0x5612dd819030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:19.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.709+0000 7f82f1d9b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612dd8151b0 0x5612dd818af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:19.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.709+0000 7f82f1d9b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612dd8151b0 0x5612dd818af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:43406/0 (socket says 172.21.15.100:43406) 2024-05-10T11:26:19.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.709+0000 7f82f2d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612dd815ad0 0x5612dd819030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49054/0 (socket says 172.21.15.100:49054) 2024-05-10T11:26:19.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.709+0000 7f82f1d9b700 1 -- 172.21.15.100:0/3859933468 learned_addr learned my addr 172.21.15.100:0/3859933468 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:26:19.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.709+0000 7f83032c8b80 10 client.0 _ll_register_callbacks cb 0x5612dd6f55b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:26:19.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.709+0000 7f82f259c700 1 --2- 172.21.15.100:0/3859933468 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612dd80fd50 0x5612dd81b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:19.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.709+0000 7f82f1d9b700 1 -- 172.21.15.100:0/3859933468 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612dd80fd50 msgr2=0x5612dd81b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:19.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.709+0000 7f82f1d9b700 1 --2- 172.21.15.100:0/3859933468 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612dd80fd50 0x5612dd81b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:19.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.709+0000 7f82f1d9b700 1 -- 172.21.15.100:0/3859933468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612dd815ad0 msgr2=0x5612dd819030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:19.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.709+0000 7f82f1d9b700 1 --2- 172.21.15.100:0/3859933468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612dd815ad0 0x5612dd819030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:19.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.709+0000 7f82f1d9b700 1 -- 172.21.15.100:0/3859933468 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612dd718760 con 0x5612dd8151b0 2024-05-10T11:26:19.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.710+0000 7f82f1d9b700 1 --2- 172.21.15.100:0/3859933468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612dd8151b0 0x5612dd818af0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f82e80a9d30 tx=0x7f82e8009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d1e0ec64c403866 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:26:19.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.710+0000 7f82d9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:26:19.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.710+0000 7f82d9ffb700 1 -- 172.21.15.100:0/3859933468 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e80c3070 con 0x5612dd8151b0 2024-05-10T11:26:19.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.710+0000 7f82d9ffb700 1 -- 172.21.15.100:0/3859933468 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f82e80039d0 con 0x5612dd8151b0 2024-05-10T11:26:19.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.710+0000 7f83032c8b80 1 -- 172.21.15.100:0/3859933468 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82e80aa030 con 0x5612dd8151b0 2024-05-10T11:26:19.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.710+0000 7f82d9ffb700 1 -- 172.21.15.100:0/3859933468 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f82e80ae3e0 con 0x5612dd8151b0 2024-05-10T11:26:19.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.710+0000 7f83032c8b80 10 client.26434 fetch_fsmap learned FSMap version 31 2024-05-10T11:26:19.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.710+0000 7f83032c8b80 1 -- 172.21.15.100:0/3859933468 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5612dd81c210 con 0x5612dd8151b0 2024-05-10T11:26:19.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.710+0000 7f82d9ffb700 1 -- 172.21.15.100:0/3859933468 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f82e80aa030 con 0x5612dd8151b0 2024-05-10T11:26:19.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.710+0000 7f82d9ffb700 1 -- 172.21.15.100:0/3859933468 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f82e80a59a0 con 0x5612dd8151b0 2024-05-10T11:26:19.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.711+0000 7f83032c8b80 10 client.26434 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:26:19.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.711+0000 7f83032c8b80 10 client.26434 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-10T11:26:19.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.711+0000 7f83032c8b80 20 client.26434 populate_metadata read hostname 'smithi100' 2024-05-10T11:26:19.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.711+0000 7f83032c8b80 1 --2- 172.21.15.100:0/3859933468 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x5612dd828d80 0x5612dd82b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:19.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.711+0000 7f83032c8b80 4 client.26434 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:26:19.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.711+0000 7f83032c8b80 1 -- 172.21.15.100:0/3859933468 --> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612dd76f320 con 0x5612dd828d80 2024-05-10T11:26:19.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.711+0000 7f82f259c700 1 --2- 172.21.15.100:0/3859933468 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x5612dd828d80 0x5612dd82b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:19.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.711+0000 7f82d9ffb700 10 client.26434 ms_handle_connect on v2:172.21.15.188:6836/1943342620 2024-05-10T11:26:19.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.711+0000 7f82f259c700 1 --2- 172.21.15.100:0/3859933468 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x5612dd828d80 0x5612dd82b170 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4846b3d792789387 server_cookie=f2725f3d9bcfcb1a in_seq=0 out_seq=0 2024-05-10T11:26:19.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.712+0000 7f82d9ffb700 1 -- 172.21.15.100:0/3859933468 <== mds.2 v2:172.21.15.188:6836/1943342620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5612dd76f320 con 0x5612dd828d80 2024-05-10T11:26:19.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.712+0000 7f82d9ffb700 10 client.26434 handle_command_reply: tid=0 2024-05-10T11:26:19.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.712+0000 7f83032c8b80 1 client.26434 shutdown 2024-05-10T11:26:19.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.712+0000 7f83032c8b80 1 -- 172.21.15.100:0/3859933468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612dd8151b0 msgr2=0x5612dd818af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:19.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.712+0000 7f83032c8b80 1 --2- 172.21.15.100:0/3859933468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612dd8151b0 0x5612dd818af0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f82e80a9d30 tx=0x7f82e8009310 comp rx=0 tx=0).stop 2024-05-10T11:26:19.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.713+0000 7f83032c8b80 1 -- 172.21.15.100:0/3859933468 shutdown_connections 2024-05-10T11:26:19.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.713+0000 7f83032c8b80 1 --2- 172.21.15.100:0/3859933468 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x5612dd828d80 0x5612dd82b170 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:19.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.713+0000 7f83032c8b80 1 --2- 172.21.15.100:0/3859933468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612dd815ad0 0x5612dd819030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:19.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.713+0000 7f82d9ffb700 0 client.26434 ms_handle_reset on v2:172.21.15.188:6836/1943342620 2024-05-10T11:26:19.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.713+0000 7f83032c8b80 1 --2- 172.21.15.100:0/3859933468 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612dd80fd50 0x5612dd81b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:19.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.713+0000 7f83032c8b80 1 --2- 172.21.15.100:0/3859933468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612dd8151b0 0x5612dd818af0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:19.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.713+0000 7f83032c8b80 1 -- 172.21.15.100:0/3859933468 >> 172.21.15.100:0/3859933468 conn(0x5612dd73bb70 msgr2=0x5612dd804b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:26:19.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.713+0000 7f83032c8b80 1 -- 172.21.15.100:0/3859933468 shutdown_connections 2024-05-10T11:26:19.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.713+0000 7f83032c8b80 1 -- 172.21.15.100:0/3859933468 wait complete. 2024-05-10T11:26:19.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.713+0000 7f83032c8b80 20 client.26434 trim_cache size 0 max 16384 2024-05-10T11:26:19.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.731+0000 7f83032c8b80 1 Processor -- start 2024-05-10T11:26:19.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.731+0000 7f83032c8b80 1 -- start start 2024-05-10T11:26:19.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.731+0000 7f83032c8b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612dd80fd50 0x5612dd816740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:19.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.731+0000 7f83032c8b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612dd8151b0 0x5612dd8251b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:19.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.731+0000 7f83032c8b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612dd815ad0 0x5612dd804c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:19.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.731+0000 7f83032c8b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f82f41af700 con 0x5612dd80fd50 2024-05-10T11:26:19.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.731+0000 7f83032c8b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f82f41acf90 con 0x5612dd8151b0 2024-05-10T11:26:19.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.731+0000 7f83032c8b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f82f41aade0 con 0x5612dd815ad0 2024-05-10T11:26:19.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.731+0000 7f82f259c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612dd80fd50 0x5612dd816740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:19.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.731+0000 7f82f259c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612dd80fd50 0x5612dd816740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49058/0 (socket says 172.21.15.100:49058) 2024-05-10T11:26:19.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.731+0000 7f82f259c700 1 -- 172.21.15.100:0/1443684781 learned_addr learned my addr 172.21.15.100:0/1443684781 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:26:19.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.731+0000 7f82f1d9b700 1 --2- 172.21.15.100:0/1443684781 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612dd8151b0 0x5612dd8251b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:19.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.731+0000 7f82f2d9d700 1 --2- 172.21.15.100:0/1443684781 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612dd815ad0 0x5612dd804c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:19.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.731+0000 7f82f259c700 1 -- 172.21.15.100:0/1443684781 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612dd815ad0 msgr2=0x5612dd804c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:19.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.731+0000 7f82f259c700 1 --2- 172.21.15.100:0/1443684781 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612dd815ad0 0x5612dd804c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:19.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.731+0000 7f82f259c700 1 -- 172.21.15.100:0/1443684781 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612dd8151b0 msgr2=0x5612dd8251b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:19.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.731+0000 7f82f259c700 1 --2- 172.21.15.100:0/1443684781 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612dd8151b0 0x5612dd8251b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:19.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.731+0000 7f82f259c700 1 -- 172.21.15.100:0/1443684781 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612dd810650 con 0x5612dd80fd50 2024-05-10T11:26:19.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.732+0000 7f82f259c700 1 --2- 172.21.15.100:0/1443684781 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612dd80fd50 0x5612dd816740 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f82e40106b0 tx=0x7f82e401c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d500f19b0e0744d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:26:19.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.732+0000 7f82dbfff700 1 -- 172.21.15.100:0/1443684781 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e4035030 con 0x5612dd80fd50 2024-05-10T11:26:19.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.732+0000 7f82dbfff700 1 -- 172.21.15.100:0/1443684781 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f82e4031980 con 0x5612dd80fd50 2024-05-10T11:26:19.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.732+0000 7f82dbfff700 1 -- 172.21.15.100:0/1443684781 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e4003df0 con 0x5612dd80fd50 2024-05-10T11:26:19.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.732+0000 7f83032c8b80 1 -- 172.21.15.100:0/1443684781 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612dd80fd50 msgr2=0x5612dd816740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:19.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.732+0000 7f83032c8b80 1 --2- 172.21.15.100:0/1443684781 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612dd80fd50 0x5612dd816740 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f82e40106b0 tx=0x7f82e401c000 comp rx=0 tx=0).stop 2024-05-10T11:26:19.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.732+0000 7f83032c8b80 1 -- 172.21.15.100:0/1443684781 shutdown_connections 2024-05-10T11:26:19.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.732+0000 7f83032c8b80 1 --2- 172.21.15.100:0/1443684781 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612dd815ad0 0x5612dd804c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:19.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.732+0000 7f83032c8b80 1 --2- 172.21.15.100:0/1443684781 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612dd80fd50 0x5612dd816740 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:19.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.732+0000 7f83032c8b80 1 --2- 172.21.15.100:0/1443684781 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612dd8151b0 0x5612dd8251b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:19.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.732+0000 7f83032c8b80 1 -- 172.21.15.100:0/1443684781 >> 172.21.15.100:0/1443684781 conn(0x5612dd73bb70 msgr2=0x5612dd817400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:26:19.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.733+0000 7f83032c8b80 1 -- 172.21.15.100:0/1443684781 shutdown_connections 2024-05-10T11:26:19.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.733+0000 7f83032c8b80 1 -- 172.21.15.100:0/1443684781 wait complete. 2024-05-10T11:26:19.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.733+0000 7f83032c8b80 1 -- start start 2024-05-10T11:26:19.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.733+0000 7f83032c8b80 1 Processor -- start 2024-05-10T11:26:19.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.733+0000 7f83032c8b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612dd80fd50 0x5612dd88b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:19.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.733+0000 7f83032c8b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612dd8151b0 0x5612dd88be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:19.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.733+0000 7f83032c8b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612dd815ad0 0x5612dd81c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:19.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.733+0000 7f83032c8b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f82f41ab450 con 0x5612dd8151b0 2024-05-10T11:26:19.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.733+0000 7f83032c8b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f82f41ac8c0 con 0x5612dd80fd50 2024-05-10T11:26:19.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.733+0000 7f83032c8b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f82f41abb20 con 0x5612dd815ad0 2024-05-10T11:26:19.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.733+0000 7f83032c8b80 10 client.0 _ll_register_callbacks cb 0x5612dd6f55b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:26:19.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.733+0000 7f82f1d9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612dd8151b0 0x5612dd88be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:19.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.733+0000 7f82f1d9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612dd8151b0 0x5612dd88be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49060/0 (socket says 172.21.15.100:49060) 2024-05-10T11:26:19.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.733+0000 7f82f1d9b700 1 -- 172.21.15.100:0/1731588505 learned_addr learned my addr 172.21.15.100:0/1731588505 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:26:19.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.733+0000 7f82f259c700 1 --2- 172.21.15.100:0/1731588505 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612dd80fd50 0x5612dd88b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:19.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.733+0000 7f82f2d9d700 1 --2- 172.21.15.100:0/1731588505 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612dd815ad0 0x5612dd81c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:19.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.734+0000 7f82f1d9b700 1 -- 172.21.15.100:0/1731588505 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612dd815ad0 msgr2=0x5612dd81c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:19.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.734+0000 7f82f1d9b700 1 --2- 172.21.15.100:0/1731588505 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612dd815ad0 0x5612dd81c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:19.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.734+0000 7f82f1d9b700 1 -- 172.21.15.100:0/1731588505 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612dd80fd50 msgr2=0x5612dd88b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:19.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.734+0000 7f82f1d9b700 1 --2- 172.21.15.100:0/1731588505 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612dd80fd50 0x5612dd88b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:19.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.734+0000 7f82f1d9b700 1 -- 172.21.15.100:0/1731588505 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612dd81d0c0 con 0x5612dd8151b0 2024-05-10T11:26:19.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.734+0000 7f82f1d9b700 1 --2- 172.21.15.100:0/1731588505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612dd8151b0 0x5612dd88be20 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f82e8009f80 tx=0x7f82e8003f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5840ead03aedf2e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:26:19.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.734+0000 7f82d9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:26:19.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.734+0000 7f82d9ffb700 1 -- 172.21.15.100:0/1731588505 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e80c3060 con 0x5612dd8151b0 2024-05-10T11:26:19.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.734+0000 7f82d9ffb700 1 -- 172.21.15.100:0/1731588505 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f82e80a8d00 con 0x5612dd8151b0 2024-05-10T11:26:19.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.734+0000 7f83032c8b80 1 -- 172.21.15.100:0/1731588505 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82e4031980 con 0x5612dd8151b0 2024-05-10T11:26:19.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.734+0000 7f82d9ffb700 1 -- 172.21.15.100:0/1731588505 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f82e80aa730 con 0x5612dd8151b0 2024-05-10T11:26:19.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.734+0000 7f83032c8b80 10 client.16965 fetch_fsmap learned FSMap version 31 2024-05-10T11:26:19.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.734+0000 7f83032c8b80 1 -- 172.21.15.100:0/1731588505 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5612dd81d0c0 con 0x5612dd8151b0 2024-05-10T11:26:19.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.734+0000 7f82d9ffb700 1 -- 172.21.15.100:0/1731588505 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f82e4031980 con 0x5612dd8151b0 2024-05-10T11:26:19.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.734+0000 7f82d9ffb700 1 -- 172.21.15.100:0/1731588505 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f82e80c4260 con 0x5612dd8151b0 2024-05-10T11:26:19.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.734+0000 7f83032c8b80 10 client.16965 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:26:19.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.734+0000 7f83032c8b80 10 client.16965 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-10T11:26:19.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.735+0000 7f83032c8b80 20 client.16965 populate_metadata read hostname 'smithi100' 2024-05-10T11:26:19.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.735+0000 7f83032c8b80 1 --2- 172.21.15.100:0/1731588505 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x5612dd828d80 0x5612dd895450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:19.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.735+0000 7f83032c8b80 4 client.16965 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:26:19.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.735+0000 7f83032c8b80 1 -- 172.21.15.100:0/1731588505 --> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5612dd7693c0 con 0x5612dd828d80 2024-05-10T11:26:19.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.735+0000 7f82f259c700 1 --2- 172.21.15.100:0/1731588505 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x5612dd828d80 0x5612dd895450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:19.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.735+0000 7f82d9ffb700 10 client.16965 ms_handle_connect on v2:172.21.15.188:6836/1943342620 2024-05-10T11:26:19.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.735+0000 7f82f259c700 1 --2- 172.21.15.100:0/1731588505 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x5612dd828d80 0x5612dd895450 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=67854c8022af4f6e server_cookie=4b0a7225096f4144 in_seq=0 out_seq=0 2024-05-10T11:26:19.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.735+0000 7f82d9ffb700 1 -- 172.21.15.100:0/1731588505 <== mds.2 v2:172.21.15.188:6836/1943342620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5612dd7693c0 con 0x5612dd828d80 2024-05-10T11:26:19.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.735+0000 7f82d9ffb700 10 client.16965 handle_command_reply: tid=0 2024-05-10T11:26:19.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.735+0000 7f83032c8b80 1 client.16965 shutdown 2024-05-10T11:26:19.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.735+0000 7f83032c8b80 1 -- 172.21.15.100:0/1731588505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612dd8151b0 msgr2=0x5612dd88be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:19.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.735+0000 7f83032c8b80 1 --2- 172.21.15.100:0/1731588505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612dd8151b0 0x5612dd88be20 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f82e8009f80 tx=0x7f82e8003f20 comp rx=0 tx=0).stop 2024-05-10T11:26:19.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.735+0000 7f83032c8b80 1 -- 172.21.15.100:0/1731588505 shutdown_connections 2024-05-10T11:26:19.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.735+0000 7f83032c8b80 1 --2- 172.21.15.100:0/1731588505 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x5612dd828d80 0x5612dd895450 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:19.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.735+0000 7f83032c8b80 1 --2- 172.21.15.100:0/1731588505 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612dd815ad0 0x5612dd81c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:19.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.735+0000 7f83032c8b80 1 --2- 172.21.15.100:0/1731588505 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612dd80fd50 0x5612dd88b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:19.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.735+0000 7f83032c8b80 1 --2- 172.21.15.100:0/1731588505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612dd8151b0 0x5612dd88be20 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:19.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.735+0000 7f83032c8b80 1 -- 172.21.15.100:0/1731588505 >> 172.21.15.100:0/1731588505 conn(0x5612dd73bb70 msgr2=0x5612dd80f090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:26:19.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.735+0000 7f82d9ffb700 0 client.16965 ms_handle_reset on v2:172.21.15.188:6836/1943342620 2024-05-10T11:26:19.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.736+0000 7f83032c8b80 1 -- 172.21.15.100:0/1731588505 shutdown_connections 2024-05-10T11:26:19.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.736+0000 7f83032c8b80 1 -- 172.21.15.100:0/1731588505 wait complete. 2024-05-10T11:26:19.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.736+0000 7f83032c8b80 20 client.16965 trim_cache size 0 max 16384 2024-05-10T11:26:19.742 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:26:19.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.736+0000 7f82f8a35700 1 -- 172.21.15.100:0/1118986109 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f82dc0b2900 msgr2=0x7f82dc0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:19.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.736+0000 7f82f8a35700 1 --2- 172.21.15.100:0/1118986109 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f82dc0b2900 0x7f82dc0b4dc0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f82e4009db0 tx=0x7f82e401a040 comp rx=0 tx=0).stop 2024-05-10T11:26:19.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.736+0000 7f82f8a35700 1 -- 172.21.15.100:0/1118986109 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82f4103040 msgr2=0x7f82f419aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:19.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.736+0000 7f82f8a35700 1 --2- 172.21.15.100:0/1118986109 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82f4103040 0x7f82f419aed0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f82e8005f50 tx=0x7f82e8004270 comp rx=0 tx=0).stop 2024-05-10T11:26:19.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.736+0000 7f82f8a35700 1 -- 172.21.15.100:0/1118986109 shutdown_connections 2024-05-10T11:26:19.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.736+0000 7f82f8a35700 1 --2- 172.21.15.100:0/1118986109 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f82dc0b2900 0x7f82dc0b4dc0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:19.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.736+0000 7f82f8a35700 1 --2- 172.21.15.100:0/1118986109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82f41076e0 0x7f82f41a5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:19.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.736+0000 7f82f8a35700 1 --2- 172.21.15.100:0/1118986109 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82f4102690 0x7f82f419a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:19.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.736+0000 7f82f8a35700 1 --2- 172.21.15.100:0/1118986109 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82f4103040 0x7f82f419aed0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:19.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.736+0000 7f82f8a35700 1 -- 172.21.15.100:0/1118986109 >> 172.21.15.100:0/1118986109 conn(0x7f82f40783f0 msgr2=0x7f82f4100830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:26:19.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.737+0000 7f82f8a35700 1 -- 172.21.15.100:0/1118986109 shutdown_connections 2024-05-10T11:26:19.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:19.737+0000 7f82f8a35700 1 -- 172.21.15.100:0/1118986109 wait complete. 2024-05-10T11:26:19.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:26:19 smithi178 ceph-mon[33359]: pgmap v658: 129 pgs: 129 active+clean; 83 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 1.4 MiB/s wr, 164 op/s 2024-05-10T11:26:19.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:26:19 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:26:19.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:26:19 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:26:19.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:26:19 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:26:19.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:26:19 smithi188 ceph-mon[34075]: pgmap v658: 129 pgs: 129 active+clean; 83 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 1.4 MiB/s wr, 164 op/s 2024-05-10T11:26:19.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:26:19 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:26:19.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:26:19 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:26:19.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:26:19 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:26:19.975 INFO:tasks.workunit.client.0.smithi100.stdout: 1 193406 2.73 MB/sec execute 254 sec latency 63.076 ms 2024-05-10T11:26:20.498 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-10T11:26:20.765 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:26:20.975 INFO:tasks.workunit.client.0.smithi100.stdout: 1 193989 2.73 MB/sec execute 255 sec latency 15.535 ms 2024-05-10T11:26:21.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:26:21 smithi178 ceph-mon[33359]: pgmap v659: 129 pgs: 129 active+clean; 83 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 1.2 MiB/s wr, 123 op/s 2024-05-10T11:26:21.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:26:21 smithi188 ceph-mon[34075]: pgmap v659: 129 pgs: 129 active+clean; 83 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 1.2 MiB/s wr, 123 op/s 2024-05-10T11:26:21.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:26:21 smithi100 ceph-mon[30296]: pgmap v659: 129 pgs: 129 active+clean; 83 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 1.2 MiB/s wr, 123 op/s 2024-05-10T11:26:21.975 INFO:tasks.workunit.client.0.smithi100.stdout: 1 194755 2.74 MB/sec execute 256 sec latency 15.805 ms 2024-05-10T11:26:22.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.436+0000 7f5c1fa19700 1 -- 172.21.15.100:0/2283801354 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c18102540 msgr2=0x7f5c1810e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:22.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.436+0000 7f5c1fa19700 1 --2- 172.21.15.100:0/2283801354 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c18102540 0x7f5c1810e9b0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f5c1000c240 tx=0x7f5c1000c550 comp rx=0 tx=0).stop 2024-05-10T11:26:22.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.436+0000 7f5c1fa19700 1 -- 172.21.15.100:0/2283801354 shutdown_connections 2024-05-10T11:26:22.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.436+0000 7f5c1fa19700 1 --2- 172.21.15.100:0/2283801354 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c18102540 0x7f5c1810e9b0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:22.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.436+0000 7f5c1fa19700 1 --2- 172.21.15.100:0/2283801354 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c18107be0 0x7f5c18107fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:22.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.436+0000 7f5c1fa19700 1 --2- 172.21.15.100:0/2283801354 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c18101b80 0x7f5c18102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:22.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.436+0000 7f5c1fa19700 1 -- 172.21.15.100:0/2283801354 >> 172.21.15.100:0/2283801354 conn(0x7f5c18078a10 msgr2=0x7f5c180ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:26:22.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.436+0000 7f5c1fa19700 1 -- 172.21.15.100:0/2283801354 shutdown_connections 2024-05-10T11:26:22.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.436+0000 7f5c1fa19700 1 -- 172.21.15.100:0/2283801354 wait complete. 2024-05-10T11:26:22.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.436+0000 7f5c1fa19700 1 Processor -- start 2024-05-10T11:26:22.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.436+0000 7f5c1fa19700 1 -- start start 2024-05-10T11:26:22.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.437+0000 7f5c1fa19700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c18101b80 0x7f5c1819f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:22.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.437+0000 7f5c1fa19700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c18102540 0x7f5c1819fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:22.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.437+0000 7f5c1fa19700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c18107be0 0x7f5c18199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:22.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.437+0000 7f5c1fa19700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5c18111330 con 0x7f5c18101b80 2024-05-10T11:26:22.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.437+0000 7f5c1fa19700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5c181111b0 con 0x7f5c18107be0 2024-05-10T11:26:22.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.437+0000 7f5c1fa19700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5c181114b0 con 0x7f5c18102540 2024-05-10T11:26:22.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.437+0000 7f5c1d7b5700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c18101b80 0x7f5c1819f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:22.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.437+0000 7f5c1d7b5700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c18101b80 0x7f5c1819f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49086/0 (socket says 172.21.15.100:49086) 2024-05-10T11:26:22.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.437+0000 7f5c1d7b5700 1 -- 172.21.15.100:0/1255297195 learned_addr learned my addr 172.21.15.100:0/1255297195 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:26:22.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.437+0000 7f5c1cfb4700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c18102540 0x7f5c1819fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:22.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.437+0000 7f5c1dfb6700 1 --2- 172.21.15.100:0/1255297195 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c18107be0 0x7f5c18199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:22.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.437+0000 7f5c1d7b5700 1 -- 172.21.15.100:0/1255297195 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c18102540 msgr2=0x7f5c1819fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:22.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.437+0000 7f5c1d7b5700 1 --2- 172.21.15.100:0/1255297195 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c18102540 0x7f5c1819fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:22.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.437+0000 7f5c1d7b5700 1 -- 172.21.15.100:0/1255297195 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c18107be0 msgr2=0x7f5c18199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:22.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.437+0000 7f5c1d7b5700 1 --2- 172.21.15.100:0/1255297195 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c18107be0 0x7f5c18199770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:22.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.437+0000 7f5c1d7b5700 1 -- 172.21.15.100:0/1255297195 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c1819a050 con 0x7f5c18101b80 2024-05-10T11:26:22.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.437+0000 7f5c1dfb6700 1 --2- 172.21.15.100:0/1255297195 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c18107be0 0x7f5c18199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:26:22.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.437+0000 7f5c1cfb4700 1 --2- 172.21.15.100:0/1255297195 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c18102540 0x7f5c1819fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:26:22.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.438+0000 7f5c1d7b5700 1 --2- 172.21.15.100:0/1255297195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c18101b80 0x7f5c1819f6a0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f5c1400c9b0 tx=0x7f5c1400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:26:22.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.440+0000 7f5c0e7fc700 1 -- 172.21.15.100:0/1255297195 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c14007b00 con 0x7f5c18101b80 2024-05-10T11:26:22.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.440+0000 7f5c0e7fc700 1 -- 172.21.15.100:0/1255297195 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5c14007ca0 con 0x7f5c18101b80 2024-05-10T11:26:22.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.440+0000 7f5c1fa19700 1 -- 172.21.15.100:0/1255297195 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c1819a2e0 con 0x7f5c18101b80 2024-05-10T11:26:22.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.440+0000 7f5c0e7fc700 1 -- 172.21.15.100:0/1255297195 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c14018660 con 0x7f5c18101b80 2024-05-10T11:26:22.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.440+0000 7f5c1fa19700 1 -- 172.21.15.100:0/1255297195 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c18103ad0 con 0x7f5c18101b80 2024-05-10T11:26:22.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.444+0000 7f5c0e7fc700 1 -- 172.21.15.100:0/1255297195 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f5c1400f450 con 0x7f5c18101b80 2024-05-10T11:26:22.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.444+0000 7f5c0e7fc700 1 --2- 172.21.15.100:0/1255297195 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f5c040b2900 0x7f5c040b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:22.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.444+0000 7f5c1cfb4700 1 --2- 172.21.15.100:0/1255297195 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f5c040b2900 0x7f5c040b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:22.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.444+0000 7f5c0e7fc700 1 -- 172.21.15.100:0/1255297195 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f5c140e2500 con 0x7f5c18101b80 2024-05-10T11:26:22.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.445+0000 7f5c1cfb4700 1 --2- 172.21.15.100:0/1255297195 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f5c040b2900 0x7f5c040b4dc0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f5c1819af30 tx=0x7f5c08009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:26:22.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.445+0000 7f5c2a2acb80 1 Processor -- start 2024-05-10T11:26:22.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.445+0000 7f5c2a2acb80 1 -- start start 2024-05-10T11:26:22.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.446+0000 7f5c2a2acb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56435647e4d0 0x56435647e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:22.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.446+0000 7f5c2a2acb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56435647edf0 0x5643564826f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:22.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.446+0000 7f5c1d7b5700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56435647e4d0 0x56435647e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:22.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.446+0000 7f5c2a2acb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564356482c80 0x564356483100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:22.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.446+0000 7f5c1d7b5700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56435647e4d0 0x56435647e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49094/0 (socket says 172.21.15.100:49094) 2024-05-10T11:26:22.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.446+0000 7f5c2a2acb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5643563d0860 con 0x56435647e4d0 2024-05-10T11:26:22.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.446+0000 7f5c1d7b5700 1 -- 172.21.15.100:0/84565096 learned_addr learned my addr 172.21.15.100:0/84565096 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:26:22.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.446+0000 7f5c2a2acb80 1 -- 172.21.15.100:0/84565096 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56435636f7c0 con 0x56435647edf0 2024-05-10T11:26:22.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.446+0000 7f5c2a2acb80 1 -- 172.21.15.100:0/84565096 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5643563cc820 con 0x564356482c80 2024-05-10T11:26:22.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.446+0000 7f5c1cfb4700 1 --2- 172.21.15.100:0/84565096 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56435647edf0 0x5643564826f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:22.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.446+0000 7f5c1dfb6700 1 --2- 172.21.15.100:0/84565096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564356482c80 0x564356483100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:22.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.446+0000 7f5c1d7b5700 1 -- 172.21.15.100:0/84565096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564356482c80 msgr2=0x564356483100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:22.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.446+0000 7f5c1d7b5700 1 --2- 172.21.15.100:0/84565096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564356482c80 0x564356483100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:22.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.446+0000 7f5c1d7b5700 1 -- 172.21.15.100:0/84565096 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56435647edf0 msgr2=0x5643564826f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:22.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.446+0000 7f5c1d7b5700 1 --2- 172.21.15.100:0/84565096 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56435647edf0 0x5643564826f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:22.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.446+0000 7f5c1d7b5700 1 -- 172.21.15.100:0/84565096 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56435631d8d0 con 0x56435647e4d0 2024-05-10T11:26:22.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.446+0000 7f5c1dfb6700 1 --2- 172.21.15.100:0/84565096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564356482c80 0x564356483100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:26:22.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.446+0000 7f5c1d7b5700 1 --2- 172.21.15.100:0/84565096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56435647e4d0 0x56435647e8b0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f5c140a6a30 tx=0x7f5c140a91e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8722171c01b05840 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:26:22.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.447+0000 7f5c037fe700 1 -- 172.21.15.100:0/84565096 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c140a9ec0 con 0x56435647e4d0 2024-05-10T11:26:22.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.447+0000 7f5c037fe700 1 -- 172.21.15.100:0/84565096 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5c140a97c0 con 0x56435647e4d0 2024-05-10T11:26:22.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.447+0000 7f5c037fe700 1 -- 172.21.15.100:0/84565096 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c140a9c80 con 0x56435647e4d0 2024-05-10T11:26:22.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.447+0000 7f5c2a2acb80 1 -- 172.21.15.100:0/84565096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56435647e4d0 msgr2=0x56435647e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:22.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.447+0000 7f5c2a2acb80 1 --2- 172.21.15.100:0/84565096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56435647e4d0 0x56435647e8b0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f5c140a6a30 tx=0x7f5c140a91e0 comp rx=0 tx=0).stop 2024-05-10T11:26:22.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.447+0000 7f5c2a2acb80 1 -- 172.21.15.100:0/84565096 shutdown_connections 2024-05-10T11:26:22.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.447+0000 7f5c2a2acb80 1 --2- 172.21.15.100:0/84565096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564356482c80 0x564356483100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:22.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.447+0000 7f5c2a2acb80 1 --2- 172.21.15.100:0/84565096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56435647e4d0 0x56435647e8b0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:22.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.447+0000 7f5c2a2acb80 1 --2- 172.21.15.100:0/84565096 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56435647edf0 0x5643564826f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:22.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.447+0000 7f5c2a2acb80 1 -- 172.21.15.100:0/84565096 >> 172.21.15.100:0/84565096 conn(0x5643563aa100 msgr2=0x564356478c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:26:22.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.447+0000 7f5c2a2acb80 1 -- 172.21.15.100:0/84565096 shutdown_connections 2024-05-10T11:26:22.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.448+0000 7f5c2a2acb80 1 -- 172.21.15.100:0/84565096 wait complete. 2024-05-10T11:26:22.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.448+0000 7f5c2a2acb80 1 -- start start 2024-05-10T11:26:22.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.448+0000 7f5c2a2acb80 1 Processor -- start 2024-05-10T11:26:22.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.448+0000 7f5c2a2acb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56435647e4d0 0x56435648a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:22.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.448+0000 7f5c2a2acb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56435647edf0 0x5643564873a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:22.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.448+0000 7f5c2a2acb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564356482c80 0x5643564878e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:22.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.448+0000 7f5c2a2acb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56435648ae90 con 0x56435647e4d0 2024-05-10T11:26:22.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.448+0000 7f5c2a2acb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x564356336c30 con 0x564356482c80 2024-05-10T11:26:22.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.448+0000 7f5c2a2acb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56435648b010 con 0x56435647edf0 2024-05-10T11:26:22.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.448+0000 7f5c2a2acb80 10 client.0 _ll_register_callbacks cb 0x564356368920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:26:22.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.448+0000 7f5c1d7b5700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56435647e4d0 0x56435648a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:22.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.448+0000 7f5c1d7b5700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56435647e4d0 0x56435648a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49104/0 (socket says 172.21.15.100:49104) 2024-05-10T11:26:22.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.448+0000 7f5c1d7b5700 1 -- 172.21.15.100:0/1689349644 learned_addr learned my addr 172.21.15.100:0/1689349644 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:26:22.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.449+0000 7f5c1cfb4700 1 --2- 172.21.15.100:0/1689349644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56435647edf0 0x5643564873a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:22.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.449+0000 7f5c1dfb6700 1 --2- 172.21.15.100:0/1689349644 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564356482c80 0x5643564878e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:22.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.449+0000 7f5c1d7b5700 1 -- 172.21.15.100:0/1689349644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56435647edf0 msgr2=0x5643564873a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:22.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.449+0000 7f5c1d7b5700 1 --2- 172.21.15.100:0/1689349644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56435647edf0 0x5643564873a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:22.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.449+0000 7f5c1d7b5700 1 -- 172.21.15.100:0/1689349644 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564356482c80 msgr2=0x5643564878e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:22.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.449+0000 7f5c1d7b5700 1 --2- 172.21.15.100:0/1689349644 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564356482c80 0x5643564878e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:22.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.449+0000 7f5c1d7b5700 1 -- 172.21.15.100:0/1689349644 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56435636c690 con 0x56435647e4d0 2024-05-10T11:26:22.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.449+0000 7f5c1cfb4700 1 --2- 172.21.15.100:0/1689349644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56435647edf0 0x5643564873a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:26:22.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.449+0000 7f5c1d7b5700 1 --2- 172.21.15.100:0/1689349644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56435647e4d0 0x56435648a000 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f5c140e0010 tx=0x7f5c140adb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74a69539f68590aa server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:26:22.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.449+0000 7f5c00ff9700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:26:22.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.449+0000 7f5c00ff9700 1 -- 172.21.15.100:0/1689349644 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c140ca060 con 0x56435647e4d0 2024-05-10T11:26:22.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.449+0000 7f5c00ff9700 1 -- 172.21.15.100:0/1689349644 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f5c140a9a20 con 0x56435647e4d0 2024-05-10T11:26:22.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.449+0000 7f5c2a2acb80 1 -- 172.21.15.100:0/1689349644 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c140a97c0 con 0x56435647e4d0 2024-05-10T11:26:22.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.450+0000 7f5c00ff9700 1 -- 172.21.15.100:0/1689349644 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5c140a9ec0 con 0x56435647e4d0 2024-05-10T11:26:22.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.450+0000 7f5c2a2acb80 10 client.16977 fetch_fsmap learned FSMap version 31 2024-05-10T11:26:22.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.450+0000 7f5c2a2acb80 1 -- 172.21.15.100:0/1689349644 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564356495630 con 0x56435647e4d0 2024-05-10T11:26:22.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.450+0000 7f5c00ff9700 1 -- 172.21.15.100:0/1689349644 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5c140a97c0 con 0x56435647e4d0 2024-05-10T11:26:22.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.450+0000 7f5c00ff9700 1 -- 172.21.15.100:0/1689349644 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f5c14091230 con 0x56435647e4d0 2024-05-10T11:26:22.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.450+0000 7f5c2a2acb80 10 client.16977 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:26:22.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.450+0000 7f5c2a2acb80 10 client.16977 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-10T11:26:22.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.450+0000 7f5c2a2acb80 20 client.16977 populate_metadata read hostname 'smithi100' 2024-05-10T11:26:22.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.450+0000 7f5c2a2acb80 1 --2- 172.21.15.100:0/1689349644 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x564356497b00 0x564356499ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:22.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.450+0000 7f5c2a2acb80 4 client.16977 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:26:22.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.450+0000 7f5c2a2acb80 1 -- 172.21.15.100:0/1689349644 --> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56435644a650 con 0x564356497b00 2024-05-10T11:26:22.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.450+0000 7f5c1cfb4700 1 --2- 172.21.15.100:0/1689349644 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x564356497b00 0x564356499ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:22.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.451+0000 7f5c00ff9700 10 client.16977 ms_handle_connect on v2:172.21.15.178:6838/3760985404 2024-05-10T11:26:22.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.451+0000 7f5c1cfb4700 1 --2- 172.21.15.100:0/1689349644 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x564356497b00 0x564356499ef0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=45bef2d45f75fb6b server_cookie=3b41eea55e2e6c6c in_seq=0 out_seq=0 2024-05-10T11:26:22.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.452+0000 7f5c00ff9700 1 -- 172.21.15.100:0/1689349644 <== mds.1 v2:172.21.15.178:6838/3760985404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56435644a650 con 0x564356497b00 2024-05-10T11:26:22.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.452+0000 7f5c00ff9700 10 client.16977 handle_command_reply: tid=0 2024-05-10T11:26:22.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.452+0000 7f5c2a2acb80 1 client.16977 shutdown 2024-05-10T11:26:22.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.452+0000 7f5c2a2acb80 1 -- 172.21.15.100:0/1689349644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56435647e4d0 msgr2=0x56435648a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:22.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.452+0000 7f5c2a2acb80 1 --2- 172.21.15.100:0/1689349644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56435647e4d0 0x56435648a000 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f5c140e0010 tx=0x7f5c140adb70 comp rx=0 tx=0).stop 2024-05-10T11:26:22.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.452+0000 7f5c2a2acb80 1 -- 172.21.15.100:0/1689349644 shutdown_connections 2024-05-10T11:26:22.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.452+0000 7f5c2a2acb80 1 --2- 172.21.15.100:0/1689349644 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x564356497b00 0x564356499ef0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:22.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.452+0000 7f5c2a2acb80 1 --2- 172.21.15.100:0/1689349644 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564356482c80 0x5643564878e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:22.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.452+0000 7f5c2a2acb80 1 --2- 172.21.15.100:0/1689349644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56435647e4d0 0x56435648a000 secure :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f5c140e0010 tx=0x7f5c140adb70 comp rx=0 tx=0).stop 2024-05-10T11:26:22.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.452+0000 7f5c2a2acb80 1 --2- 172.21.15.100:0/1689349644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56435647edf0 0x5643564873a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:22.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.452+0000 7f5c00ff9700 0 client.16977 ms_handle_reset on v2:172.21.15.178:6838/3760985404 2024-05-10T11:26:22.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.452+0000 7f5c2a2acb80 1 -- 172.21.15.100:0/1689349644 >> 172.21.15.100:0/1689349644 conn(0x5643563aa100 msgr2=0x564356473720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:26:22.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.452+0000 7f5c2a2acb80 1 -- 172.21.15.100:0/1689349644 shutdown_connections 2024-05-10T11:26:22.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.452+0000 7f5c2a2acb80 1 -- 172.21.15.100:0/1689349644 wait complete. 2024-05-10T11:26:22.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.453+0000 7f5c2a2acb80 20 client.16977 trim_cache size 0 max 16384 2024-05-10T11:26:22.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.470+0000 7f5c2a2acb80 1 Processor -- start 2024-05-10T11:26:22.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.470+0000 7f5c2a2acb80 1 -- start start 2024-05-10T11:26:22.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.470+0000 7f5c2a2acb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56435647edf0 0x564356493f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:22.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.470+0000 7f5c2a2acb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564356482c80 0x5643564735a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:22.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.470+0000 7f5c2a2acb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5643564805f0 0x564356480a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:22.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.470+0000 7f5c2a2acb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5c181b3b20 con 0x56435647edf0 2024-05-10T11:26:22.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.470+0000 7f5c2a2acb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5c181b13b0 con 0x564356482c80 2024-05-10T11:26:22.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.470+0000 7f5c2a2acb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5c181af200 con 0x5643564805f0 2024-05-10T11:26:22.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.470+0000 7f5c1d7b5700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56435647edf0 0x564356493f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:22.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.470+0000 7f5c1d7b5700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56435647edf0 0x564356493f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49118/0 (socket says 172.21.15.100:49118) 2024-05-10T11:26:22.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.470+0000 7f5c1d7b5700 1 -- 172.21.15.100:0/1643785832 learned_addr learned my addr 172.21.15.100:0/1643785832 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:26:22.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.470+0000 7f5c1dfb6700 1 --2- 172.21.15.100:0/1643785832 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5643564805f0 0x564356480a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:22.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.470+0000 7f5c1cfb4700 1 --2- 172.21.15.100:0/1643785832 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564356482c80 0x5643564735a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:22.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.470+0000 7f5c1d7b5700 1 -- 172.21.15.100:0/1643785832 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5643564805f0 msgr2=0x564356480a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:22.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.470+0000 7f5c1d7b5700 1 --2- 172.21.15.100:0/1643785832 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5643564805f0 0x564356480a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:22.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.470+0000 7f5c1d7b5700 1 -- 172.21.15.100:0/1643785832 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564356482c80 msgr2=0x5643564735a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:22.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.470+0000 7f5c1d7b5700 1 --2- 172.21.15.100:0/1643785832 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564356482c80 0x5643564735a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:22.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.470+0000 7f5c1d7b5700 1 -- 172.21.15.100:0/1643785832 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56435648a370 con 0x56435647edf0 2024-05-10T11:26:22.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.471+0000 7f5c1dfb6700 1 --2- 172.21.15.100:0/1643785832 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5643564805f0 0x564356480a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:26:22.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.471+0000 7f5c1d7b5700 1 --2- 172.21.15.100:0/1643785832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56435647edf0 0x564356493f20 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f5c140e24d0 tx=0x7f5c140a51b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f42fcc130b42831a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:26:22.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.471+0000 7f5c02ffd700 1 -- 172.21.15.100:0/1643785832 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c140c2070 con 0x56435647edf0 2024-05-10T11:26:22.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.471+0000 7f5c02ffd700 1 -- 172.21.15.100:0/1643785832 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5c140a97c0 con 0x56435647edf0 2024-05-10T11:26:22.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.471+0000 7f5c02ffd700 1 -- 172.21.15.100:0/1643785832 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c140ca070 con 0x56435647edf0 2024-05-10T11:26:22.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.472+0000 7f5c2a2acb80 1 -- 172.21.15.100:0/1643785832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56435647edf0 msgr2=0x564356493f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:22.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.472+0000 7f5c2a2acb80 1 --2- 172.21.15.100:0/1643785832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56435647edf0 0x564356493f20 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f5c140e24d0 tx=0x7f5c140a51b0 comp rx=0 tx=0).stop 2024-05-10T11:26:22.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.472+0000 7f5c2a2acb80 1 -- 172.21.15.100:0/1643785832 shutdown_connections 2024-05-10T11:26:22.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.472+0000 7f5c2a2acb80 1 --2- 172.21.15.100:0/1643785832 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5643564805f0 0x564356480a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:22.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.472+0000 7f5c2a2acb80 1 --2- 172.21.15.100:0/1643785832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56435647edf0 0x564356493f20 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:22.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.472+0000 7f5c2a2acb80 1 --2- 172.21.15.100:0/1643785832 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564356482c80 0x5643564735a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:22.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.472+0000 7f5c2a2acb80 1 -- 172.21.15.100:0/1643785832 >> 172.21.15.100:0/1643785832 conn(0x5643563aa100 msgr2=0x564356472e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:26:22.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.472+0000 7f5c2a2acb80 1 -- 172.21.15.100:0/1643785832 shutdown_connections 2024-05-10T11:26:22.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.472+0000 7f5c2a2acb80 1 -- 172.21.15.100:0/1643785832 wait complete. 2024-05-10T11:26:22.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.472+0000 7f5c2a2acb80 1 -- start start 2024-05-10T11:26:22.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.472+0000 7f5c2a2acb80 1 Processor -- start 2024-05-10T11:26:22.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.473+0000 7f5c2a2acb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56435647edf0 0x564356490800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:22.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.473+0000 7f5c2a2acb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5643564805f0 0x56435649eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:22.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.473+0000 7f5c2a2acb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564356482c80 0x56435649f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:22.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.473+0000 7f5c2a2acb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5c181af870 con 0x56435647edf0 2024-05-10T11:26:22.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.473+0000 7f5c2a2acb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5c181b0ce0 con 0x5643564805f0 2024-05-10T11:26:22.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.473+0000 7f5c2a2acb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5c181aff40 con 0x564356482c80 2024-05-10T11:26:22.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.473+0000 7f5c2a2acb80 10 client.0 _ll_register_callbacks cb 0x564356368920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:26:22.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.473+0000 7f5c1dfb6700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564356482c80 0x56435649f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:22.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.473+0000 7f5c1d7b5700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56435647edf0 0x564356490800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:22.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.473+0000 7f5c1dfb6700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564356482c80 0x56435649f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:43468/0 (socket says 172.21.15.100:43468) 2024-05-10T11:26:22.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.473+0000 7f5c1dfb6700 1 -- 172.21.15.100:0/2021157491 learned_addr learned my addr 172.21.15.100:0/2021157491 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:26:22.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.473+0000 7f5c1cfb4700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5643564805f0 0x56435649eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:22.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.473+0000 7f5c1dfb6700 1 -- 172.21.15.100:0/2021157491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5643564805f0 msgr2=0x56435649eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:22.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.473+0000 7f5c1dfb6700 1 --2- 172.21.15.100:0/2021157491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5643564805f0 0x56435649eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:22.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.473+0000 7f5c1dfb6700 1 -- 172.21.15.100:0/2021157491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56435647edf0 msgr2=0x564356490800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:22.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.473+0000 7f5c1dfb6700 1 --2- 172.21.15.100:0/2021157491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56435647edf0 0x564356490800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:22.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.473+0000 7f5c1dfb6700 1 -- 172.21.15.100:0/2021157491 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56435649f8c0 con 0x564356482c80 2024-05-10T11:26:22.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.473+0000 7f5c1dfb6700 1 --2- 172.21.15.100:0/2021157491 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564356482c80 0x56435649f050 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5c10016000 tx=0x7f5c1000a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84a7542cf873af9f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:26:22.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.473+0000 7f5c00ff9700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:26:22.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.473+0000 7f5c00ff9700 1 -- 172.21.15.100:0/2021157491 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c10007870 con 0x564356482c80 2024-05-10T11:26:22.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.473+0000 7f5c2a2acb80 1 -- 172.21.15.100:0/2021157491 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c140a97c0 con 0x564356482c80 2024-05-10T11:26:22.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.474+0000 7f5c00ff9700 1 -- 172.21.15.100:0/2021157491 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f5c10007a10 con 0x564356482c80 2024-05-10T11:26:22.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.474+0000 7f5c00ff9700 1 -- 172.21.15.100:0/2021157491 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5c10007c10 con 0x564356482c80 2024-05-10T11:26:22.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.474+0000 7f5c2a2acb80 10 client.26458 fetch_fsmap learned FSMap version 31 2024-05-10T11:26:22.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.474+0000 7f5c2a2acb80 1 -- 172.21.15.100:0/2021157491 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5643565021a0 con 0x564356482c80 2024-05-10T11:26:22.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.474+0000 7f5c00ff9700 1 -- 172.21.15.100:0/2021157491 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5c140a97c0 con 0x564356482c80 2024-05-10T11:26:22.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.474+0000 7f5c00ff9700 1 -- 172.21.15.100:0/2021157491 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f5c10012070 con 0x564356482c80 2024-05-10T11:26:22.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.474+0000 7f5c2a2acb80 10 client.26458 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:26:22.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.474+0000 7f5c2a2acb80 10 client.26458 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-10T11:26:22.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.474+0000 7f5c2a2acb80 20 client.26458 populate_metadata read hostname 'smithi100' 2024-05-10T11:26:22.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.474+0000 7f5c2a2acb80 1 --2- 172.21.15.100:0/2021157491 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x564356502890 0x564356504cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:22.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.474+0000 7f5c2a2acb80 4 client.26458 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:26:22.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.474+0000 7f5c2a2acb80 1 -- 172.21.15.100:0/2021157491 --> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5643563de240 con 0x564356502890 2024-05-10T11:26:22.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.474+0000 7f5c1d7b5700 1 --2- 172.21.15.100:0/2021157491 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x564356502890 0x564356504cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:22.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.475+0000 7f5c00ff9700 10 client.26458 ms_handle_connect on v2:172.21.15.178:6838/3760985404 2024-05-10T11:26:22.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.475+0000 7f5c1d7b5700 1 --2- 172.21.15.100:0/2021157491 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x564356502890 0x564356504cb0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e848ea7c75153de4 server_cookie=fd0724604bb1bb13 in_seq=0 out_seq=0 2024-05-10T11:26:22.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.475+0000 7f5c00ff9700 1 -- 172.21.15.100:0/2021157491 <== mds.1 v2:172.21.15.178:6838/3760985404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5643563de240 con 0x564356502890 2024-05-10T11:26:22.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.475+0000 7f5c00ff9700 10 client.26458 handle_command_reply: tid=0 2024-05-10T11:26:22.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.475+0000 7f5c2a2acb80 1 client.26458 shutdown 2024-05-10T11:26:22.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.475+0000 7f5c2a2acb80 1 -- 172.21.15.100:0/2021157491 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564356482c80 msgr2=0x56435649f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:22.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.475+0000 7f5c2a2acb80 1 --2- 172.21.15.100:0/2021157491 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564356482c80 0x56435649f050 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5c10016000 tx=0x7f5c1000a660 comp rx=0 tx=0).stop 2024-05-10T11:26:22.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.475+0000 7f5c2a2acb80 1 -- 172.21.15.100:0/2021157491 shutdown_connections 2024-05-10T11:26:22.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.475+0000 7f5c2a2acb80 1 --2- 172.21.15.100:0/2021157491 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x564356502890 0x564356504cb0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:22.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.475+0000 7f5c2a2acb80 1 --2- 172.21.15.100:0/2021157491 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564356482c80 0x56435649f050 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:22.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.475+0000 7f5c2a2acb80 1 --2- 172.21.15.100:0/2021157491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56435647edf0 0x564356490800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:22.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.475+0000 7f5c2a2acb80 1 --2- 172.21.15.100:0/2021157491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5643564805f0 0x56435649eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:22.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.475+0000 7f5c2a2acb80 1 -- 172.21.15.100:0/2021157491 >> 172.21.15.100:0/2021157491 conn(0x5643563aa100 msgr2=0x564356473720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:26:22.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.475+0000 7f5c00ff9700 0 client.26458 ms_handle_reset on v2:172.21.15.178:6838/3760985404 2024-05-10T11:26:22.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.475+0000 7f5c2a2acb80 1 -- 172.21.15.100:0/2021157491 shutdown_connections 2024-05-10T11:26:22.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.475+0000 7f5c2a2acb80 1 -- 172.21.15.100:0/2021157491 wait complete. 2024-05-10T11:26:22.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.475+0000 7f5c2a2acb80 20 client.26458 trim_cache size 0 max 16384 2024-05-10T11:26:22.482 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:26:22.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.476+0000 7f5c1fa19700 1 -- 172.21.15.100:0/1255297195 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f5c040b2900 msgr2=0x7f5c040b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:22.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.476+0000 7f5c1fa19700 1 --2- 172.21.15.100:0/1255297195 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f5c040b2900 0x7f5c040b4dc0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f5c1819af30 tx=0x7f5c08009260 comp rx=0 tx=0).stop 2024-05-10T11:26:22.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.476+0000 7f5c1fa19700 1 -- 172.21.15.100:0/1255297195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c18101b80 msgr2=0x7f5c1819f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:22.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.476+0000 7f5c1fa19700 1 --2- 172.21.15.100:0/1255297195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c18101b80 0x7f5c1819f6a0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f5c1400c9b0 tx=0x7f5c1400ccc0 comp rx=0 tx=0).stop 2024-05-10T11:26:22.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.476+0000 7f5c1fa19700 1 -- 172.21.15.100:0/1255297195 shutdown_connections 2024-05-10T11:26:22.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.476+0000 7f5c1fa19700 1 --2- 172.21.15.100:0/1255297195 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f5c040b2900 0x7f5c040b4dc0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:22.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.476+0000 7f5c1fa19700 1 --2- 172.21.15.100:0/1255297195 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c18107be0 0x7f5c18199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:22.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.476+0000 7f5c1fa19700 1 --2- 172.21.15.100:0/1255297195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c18101b80 0x7f5c1819f6a0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:22.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.476+0000 7f5c1fa19700 1 --2- 172.21.15.100:0/1255297195 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c18102540 0x7f5c1819fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:22.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.476+0000 7f5c1fa19700 1 -- 172.21.15.100:0/1255297195 >> 172.21.15.100:0/1255297195 conn(0x7f5c18078a10 msgr2=0x7f5c18105c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:26:22.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.476+0000 7f5c1fa19700 1 -- 172.21.15.100:0/1255297195 shutdown_connections 2024-05-10T11:26:22.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:22.476+0000 7f5c1fa19700 1 -- 172.21.15.100:0/1255297195 wait complete. 2024-05-10T11:26:22.975 INFO:tasks.workunit.client.0.smithi100.stdout: 1 195189 2.75 MB/sec execute 257 sec latency 15.245 ms 2024-05-10T11:26:23.177 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-10T11:26:23.435 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:26:23.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:26:23 smithi100 ceph-mon[30296]: pgmap v660: 129 pgs: 129 active+clean; 83 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 1.4 MiB/s wr, 151 op/s 2024-05-10T11:26:23.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:26:23 smithi178 ceph-mon[33359]: pgmap v660: 129 pgs: 129 active+clean; 83 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 1.4 MiB/s wr, 151 op/s 2024-05-10T11:26:23.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:26:23 smithi188 ceph-mon[34075]: pgmap v660: 129 pgs: 129 active+clean; 83 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 1.4 MiB/s wr, 151 op/s 2024-05-10T11:26:23.975 INFO:tasks.workunit.client.0.smithi100.stdout: 1 195505 2.75 MB/sec execute 258 sec latency 12.808 ms 2024-05-10T11:26:24.976 INFO:tasks.workunit.client.0.smithi100.stdout: 1 195836 2.74 MB/sec execute 259 sec latency 24.672 ms 2024-05-10T11:26:25.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.171+0000 7f8cb1165700 1 -- 172.21.15.100:0/1700904696 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8cac1023a0 msgr2=0x7f8cac102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:25.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.171+0000 7f8cb1165700 1 --2- 172.21.15.100:0/1700904696 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8cac1023a0 0x7f8cac102820 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f8ca0009990 tx=0x7f8ca0009ca0 comp rx=0 tx=0).stop 2024-05-10T11:26:25.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.171+0000 7f8cb1165700 1 -- 172.21.15.100:0/1700904696 shutdown_connections 2024-05-10T11:26:25.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.171+0000 7f8cb1165700 1 --2- 172.21.15.100:0/1700904696 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8cac102d60 0x7f8cac10f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:25.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.171+0000 7f8cb1165700 1 --2- 172.21.15.100:0/1700904696 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8cac108410 0x7f8cac1087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:25.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.171+0000 7f8cb1165700 1 --2- 172.21.15.100:0/1700904696 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8cac1023a0 0x7f8cac102820 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:25.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.171+0000 7f8cb1165700 1 -- 172.21.15.100:0/1700904696 >> 172.21.15.100:0/1700904696 conn(0x7f8cac079350 msgr2=0x7f8cac079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:26:25.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.171+0000 7f8cb1165700 1 -- 172.21.15.100:0/1700904696 shutdown_connections 2024-05-10T11:26:25.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.171+0000 7f8cb1165700 1 -- 172.21.15.100:0/1700904696 wait complete. 2024-05-10T11:26:25.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.172+0000 7f8cb1165700 1 Processor -- start 2024-05-10T11:26:25.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.172+0000 7f8cb1165700 1 -- start start 2024-05-10T11:26:25.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.172+0000 7f8cb1165700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8cac1023a0 0x7f8cac19a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:25.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.172+0000 7f8cb1165700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8cac102d60 0x7f8cac19aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:25.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.172+0000 7f8cb1165700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8cac108410 0x7f8cac1a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:25.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.173+0000 7f8cb1165700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8cac112480 con 0x7f8cac1023a0 2024-05-10T11:26:25.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.173+0000 7f8cb1165700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8cac112300 con 0x7f8cac102d60 2024-05-10T11:26:25.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.173+0000 7f8cb1165700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8cac112600 con 0x7f8cac108410 2024-05-10T11:26:25.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.173+0000 7f8caad9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8cac1023a0 0x7f8cac19a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:25.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.173+0000 7f8cab59e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8cac108410 0x7f8cac1a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:25.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.173+0000 7f8caad9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8cac1023a0 0x7f8cac19a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49150/0 (socket says 172.21.15.100:49150) 2024-05-10T11:26:25.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.173+0000 7f8caad9d700 1 -- 172.21.15.100:0/3290946755 learned_addr learned my addr 172.21.15.100:0/3290946755 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:26:25.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.173+0000 7f8caa59c700 1 --2- 172.21.15.100:0/3290946755 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8cac102d60 0x7f8cac19aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:25.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.173+0000 7f8cab59e700 1 -- 172.21.15.100:0/3290946755 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8cac102d60 msgr2=0x7f8cac19aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:25.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.173+0000 7f8cab59e700 1 --2- 172.21.15.100:0/3290946755 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8cac102d60 0x7f8cac19aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:25.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.173+0000 7f8cab59e700 1 -- 172.21.15.100:0/3290946755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8cac1023a0 msgr2=0x7f8cac19a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:25.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.173+0000 7f8cab59e700 1 --2- 172.21.15.100:0/3290946755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8cac1023a0 0x7f8cac19a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:25.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.173+0000 7f8cab59e700 1 -- 172.21.15.100:0/3290946755 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cac1a5f60 con 0x7f8cac108410 2024-05-10T11:26:25.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.173+0000 7f8cab59e700 1 --2- 172.21.15.100:0/3290946755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8cac108410 0x7f8cac1a5a20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8ca400be20 tx=0x7f8ca4009ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:26:25.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.173+0000 7f8c93fff700 1 -- 172.21.15.100:0/3290946755 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca4019070 con 0x7f8cac108410 2024-05-10T11:26:25.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.173+0000 7f8cb1165700 1 -- 172.21.15.100:0/3290946755 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cac1a6130 con 0x7f8cac108410 2024-05-10T11:26:25.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.173+0000 7f8c93fff700 1 -- 172.21.15.100:0/3290946755 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8ca400cb90 con 0x7f8cac108410 2024-05-10T11:26:25.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.174+0000 7f8c93fff700 1 -- 172.21.15.100:0/3290946755 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca40125a0 con 0x7f8cac108410 2024-05-10T11:26:25.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.174+0000 7f8cb1165700 1 -- 172.21.15.100:0/3290946755 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cac1a65f0 con 0x7f8cac108410 2024-05-10T11:26:25.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.176+0000 7f8c93fff700 1 -- 172.21.15.100:0/3290946755 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f8ca401c070 con 0x7f8cac108410 2024-05-10T11:26:25.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.177+0000 7f8c93fff700 1 --2- 172.21.15.100:0/3290946755 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8c940b2880 0x7f8c940b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:25.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.177+0000 7f8caad9d700 1 --2- 172.21.15.100:0/3290946755 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8c940b2880 0x7f8c940b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:25.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.177+0000 7f8c93fff700 1 -- 172.21.15.100:0/3290946755 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f8ca40e1d00 con 0x7f8cac108410 2024-05-10T11:26:25.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.178+0000 7f8cbb9f8b80 1 Processor -- start 2024-05-10T11:26:25.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.178+0000 7f8cbb9f8b80 1 -- start start 2024-05-10T11:26:25.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.178+0000 7f8caad9d700 1 --2- 172.21.15.100:0/3290946755 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8c940b2880 0x7f8c940b4d40 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c009d60 tx=0x7f8c9c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:26:25.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.178+0000 7f8cbb9f8b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ddb70c8e0 0x563ddb70ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:25.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.178+0000 7f8cbb9f8b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ddb70d200 0x563ddb710b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:25.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.178+0000 7f8cbb9f8b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563ddb711040 0x563ddb711480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:25.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.178+0000 7f8cbb9f8b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x563ddb65a860 con 0x563ddb70c8e0 2024-05-10T11:26:25.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.178+0000 7f8cbb9f8b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563ddb5f97c0 con 0x563ddb711040 2024-05-10T11:26:25.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.178+0000 7f8cbb9f8b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563ddb656820 con 0x563ddb70d200 2024-05-10T11:26:25.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.178+0000 7f8caad9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ddb70c8e0 0x563ddb70ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:25.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.178+0000 7f8caad9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ddb70c8e0 0x563ddb70ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49158/0 (socket says 172.21.15.100:49158) 2024-05-10T11:26:25.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.178+0000 7f8caad9d700 1 -- 172.21.15.100:0/905982007 learned_addr learned my addr 172.21.15.100:0/905982007 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:26:25.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.178+0000 7f8caa59c700 1 --2- 172.21.15.100:0/905982007 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ddb70d200 0x563ddb710b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:25.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.178+0000 7f8cab59e700 1 --2- 172.21.15.100:0/905982007 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563ddb711040 0x563ddb711480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:25.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.178+0000 7f8caa59c700 1 -- 172.21.15.100:0/905982007 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563ddb711040 msgr2=0x563ddb711480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:25.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.178+0000 7f8caa59c700 1 --2- 172.21.15.100:0/905982007 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563ddb711040 0x563ddb711480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:25.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.178+0000 7f8caa59c700 1 -- 172.21.15.100:0/905982007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ddb70c8e0 msgr2=0x563ddb70ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:25.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.178+0000 7f8caa59c700 1 --2- 172.21.15.100:0/905982007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ddb70c8e0 0x563ddb70ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:25.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.178+0000 7f8caa59c700 1 -- 172.21.15.100:0/905982007 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ddb5a78d0 con 0x563ddb70d200 2024-05-10T11:26:25.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.178+0000 7f8cab59e700 1 --2- 172.21.15.100:0/905982007 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563ddb711040 0x563ddb711480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:26:25.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.179+0000 7f8caa59c700 1 --2- 172.21.15.100:0/905982007 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ddb70d200 0x563ddb710b00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8ca000b650 tx=0x7f8ca0005c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45825264c7d68c72 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:26:25.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.179+0000 7f8c917fa700 1 -- 172.21.15.100:0/905982007 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca0027070 con 0x563ddb70d200 2024-05-10T11:26:25.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.179+0000 7f8c917fa700 1 -- 172.21.15.100:0/905982007 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8ca00056a0 con 0x563ddb70d200 2024-05-10T11:26:25.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.179+0000 7f8c917fa700 1 -- 172.21.15.100:0/905982007 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca000bb10 con 0x563ddb70d200 2024-05-10T11:26:25.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.180+0000 7f8cbb9f8b80 1 -- 172.21.15.100:0/905982007 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ddb70d200 msgr2=0x563ddb710b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:25.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.180+0000 7f8cbb9f8b80 1 --2- 172.21.15.100:0/905982007 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ddb70d200 0x563ddb710b00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8ca000b650 tx=0x7f8ca0005c80 comp rx=0 tx=0).stop 2024-05-10T11:26:25.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.180+0000 7f8cbb9f8b80 1 -- 172.21.15.100:0/905982007 shutdown_connections 2024-05-10T11:26:25.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.180+0000 7f8cbb9f8b80 1 --2- 172.21.15.100:0/905982007 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563ddb711040 0x563ddb711480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:25.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.180+0000 7f8cbb9f8b80 1 --2- 172.21.15.100:0/905982007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ddb70c8e0 0x563ddb70ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:25.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.180+0000 7f8cbb9f8b80 1 --2- 172.21.15.100:0/905982007 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ddb70d200 0x563ddb710b00 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:25.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.180+0000 7f8cbb9f8b80 1 -- 172.21.15.100:0/905982007 >> 172.21.15.100:0/905982007 conn(0x563ddb634100 msgr2=0x563ddb702c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:26:25.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.180+0000 7f8cbb9f8b80 1 -- 172.21.15.100:0/905982007 shutdown_connections 2024-05-10T11:26:25.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.180+0000 7f8cbb9f8b80 1 -- 172.21.15.100:0/905982007 wait complete. 2024-05-10T11:26:25.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.180+0000 7f8cbb9f8b80 1 -- start start 2024-05-10T11:26:25.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.180+0000 7f8cbb9f8b80 1 Processor -- start 2024-05-10T11:26:25.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.180+0000 7f8cbb9f8b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563ddb70c8e0 0x563ddb718470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:25.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.180+0000 7f8cbb9f8b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ddb70d200 0x563ddb715810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:25.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.180+0000 7f8cbb9f8b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ddb711040 0x563ddb715d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:25.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.180+0000 7f8cbb9f8b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x563ddb719210 con 0x563ddb711040 2024-05-10T11:26:25.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.180+0000 7f8cbb9f8b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563ddb5c0c30 con 0x563ddb70c8e0 2024-05-10T11:26:25.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.180+0000 7f8cbb9f8b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563ddb719390 con 0x563ddb70d200 2024-05-10T11:26:25.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.181+0000 7f8cbb9f8b80 10 client.0 _ll_register_callbacks cb 0x563ddb5f2920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:26:25.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.181+0000 7f8cab59e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ddb711040 0x563ddb715d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:25.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.181+0000 7f8cab59e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ddb711040 0x563ddb715d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49166/0 (socket says 172.21.15.100:49166) 2024-05-10T11:26:25.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.181+0000 7f8caa59c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ddb70d200 0x563ddb715810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:25.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.181+0000 7f8cab59e700 1 -- 172.21.15.100:0/4185420046 learned_addr learned my addr 172.21.15.100:0/4185420046 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:26:25.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.181+0000 7f8caad9d700 1 --2- 172.21.15.100:0/4185420046 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563ddb70c8e0 0x563ddb718470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:25.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.181+0000 7f8cab59e700 1 -- 172.21.15.100:0/4185420046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ddb70d200 msgr2=0x563ddb715810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:25.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.181+0000 7f8cab59e700 1 --2- 172.21.15.100:0/4185420046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ddb70d200 0x563ddb715810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:25.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.181+0000 7f8cab59e700 1 -- 172.21.15.100:0/4185420046 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563ddb70c8e0 msgr2=0x563ddb718470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:25.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.181+0000 7f8cab59e700 1 --2- 172.21.15.100:0/4185420046 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563ddb70c8e0 0x563ddb718470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:25.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.181+0000 7f8cab59e700 1 -- 172.21.15.100:0/4185420046 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ddb5f6690 con 0x563ddb711040 2024-05-10T11:26:25.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.181+0000 7f8caa59c700 1 --2- 172.21.15.100:0/4185420046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ddb70d200 0x563ddb715810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:26:25.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.181+0000 7f8cab59e700 1 --2- 172.21.15.100:0/4185420046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ddb711040 0x563ddb715d50 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f8ca40a8f50 tx=0x7f8ca40a9740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d64f5d0a8f96138 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:26:25.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.181+0000 7f8c8eff5700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:26:25.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.181+0000 7f8c8eff5700 1 -- 172.21.15.100:0/4185420046 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca40b1e20 con 0x563ddb711040 2024-05-10T11:26:25.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.181+0000 7f8c8eff5700 1 -- 172.21.15.100:0/4185420046 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f8ca40a55b0 con 0x563ddb711040 2024-05-10T11:26:25.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.181+0000 7f8cbb9f8b80 1 -- 172.21.15.100:0/4185420046 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ca00056a0 con 0x563ddb711040 2024-05-10T11:26:25.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.181+0000 7f8c8eff5700 1 -- 172.21.15.100:0/4185420046 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8ca40b1ba0 con 0x563ddb711040 2024-05-10T11:26:25.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.182+0000 7f8cbb9f8b80 10 client.16995 fetch_fsmap learned FSMap version 31 2024-05-10T11:26:25.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.182+0000 7f8cbb9f8b80 1 -- 172.21.15.100:0/4185420046 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563ddb723a70 con 0x563ddb711040 2024-05-10T11:26:25.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.182+0000 7f8c8eff5700 1 -- 172.21.15.100:0/4185420046 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8ca00056a0 con 0x563ddb711040 2024-05-10T11:26:25.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.182+0000 7f8c8eff5700 1 -- 172.21.15.100:0/4185420046 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f8ca40ab070 con 0x563ddb711040 2024-05-10T11:26:25.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.182+0000 7f8cbb9f8b80 10 client.16995 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:26:25.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.182+0000 7f8cbb9f8b80 10 client.16995 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:26:25.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.182+0000 7f8cbb9f8b80 20 client.16995 populate_metadata read hostname 'smithi100' 2024-05-10T11:26:25.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.182+0000 7f8cbb9f8b80 1 --2- 172.21.15.100:0/4185420046 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x563ddb726000 0x563ddb7283f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:25.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.182+0000 7f8cbb9f8b80 4 client.16995 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:26:25.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.182+0000 7f8cbb9f8b80 1 -- 172.21.15.100:0/4185420046 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ddb6d4650 con 0x563ddb726000 2024-05-10T11:26:25.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.182+0000 7f8caad9d700 1 --2- 172.21.15.100:0/4185420046 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x563ddb726000 0x563ddb7283f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:25.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.183+0000 7f8c8eff5700 10 client.16995 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:26:25.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.183+0000 7f8caad9d700 1 --2- 172.21.15.100:0/4185420046 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x563ddb726000 0x563ddb7283f0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7927d136e593636b server_cookie=4c019392c6fbc00 in_seq=0 out_seq=0 2024-05-10T11:26:25.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.183+0000 7f8c8eff5700 1 -- 172.21.15.100:0/4185420046 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563ddb6d4650 con 0x563ddb726000 2024-05-10T11:26:25.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.183+0000 7f8c8eff5700 10 client.16995 handle_command_reply: tid=0 2024-05-10T11:26:25.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.183+0000 7f8cbb9f8b80 1 client.16995 shutdown 2024-05-10T11:26:25.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.184+0000 7f8cbb9f8b80 1 -- 172.21.15.100:0/4185420046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ddb711040 msgr2=0x563ddb715d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:25.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.184+0000 7f8cbb9f8b80 1 --2- 172.21.15.100:0/4185420046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ddb711040 0x563ddb715d50 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f8ca40a8f50 tx=0x7f8ca40a9740 comp rx=0 tx=0).stop 2024-05-10T11:26:25.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.184+0000 7f8cbb9f8b80 1 -- 172.21.15.100:0/4185420046 shutdown_connections 2024-05-10T11:26:25.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.184+0000 7f8cbb9f8b80 1 --2- 172.21.15.100:0/4185420046 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x563ddb726000 0x563ddb7283f0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:25.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.184+0000 7f8cbb9f8b80 1 --2- 172.21.15.100:0/4185420046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ddb711040 0x563ddb715d50 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:25.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.184+0000 7f8c8eff5700 0 client.16995 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:26:25.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.184+0000 7f8cbb9f8b80 1 --2- 172.21.15.100:0/4185420046 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563ddb70c8e0 0x563ddb718470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:25.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.184+0000 7f8cbb9f8b80 1 --2- 172.21.15.100:0/4185420046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ddb70d200 0x563ddb715810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:25.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.184+0000 7f8cbb9f8b80 1 -- 172.21.15.100:0/4185420046 >> 172.21.15.100:0/4185420046 conn(0x563ddb634100 msgr2=0x563ddb6fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:26:25.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.184+0000 7f8cbb9f8b80 1 -- 172.21.15.100:0/4185420046 shutdown_connections 2024-05-10T11:26:25.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.184+0000 7f8cbb9f8b80 1 -- 172.21.15.100:0/4185420046 wait complete. 2024-05-10T11:26:25.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.184+0000 7f8cbb9f8b80 20 client.16995 trim_cache size 0 max 16384 2024-05-10T11:26:25.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.202+0000 7f8cbb9f8b80 1 Processor -- start 2024-05-10T11:26:25.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.203+0000 7f8cbb9f8b80 1 -- start start 2024-05-10T11:26:25.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.203+0000 7f8cbb9f8b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563ddb70c8e0 0x563ddb714790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:25.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.203+0000 7f8cbb9f8b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ddb70d200 0x563ddb712330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:25.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.203+0000 7f8cbb9f8b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ddb711040 0x563ddb713150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:25.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.203+0000 7f8cbb9f8b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8cac1af6f0 con 0x563ddb70d200 2024-05-10T11:26:25.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.203+0000 7f8cbb9f8b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8cac1acf80 con 0x563ddb70c8e0 2024-05-10T11:26:25.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.203+0000 7f8cbb9f8b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8cac1aadf0 con 0x563ddb711040 2024-05-10T11:26:25.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.203+0000 7f8caa59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ddb70d200 0x563ddb712330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:25.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.203+0000 7f8caa59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ddb70d200 0x563ddb712330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49176/0 (socket says 172.21.15.100:49176) 2024-05-10T11:26:25.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.203+0000 7f8caa59c700 1 -- 172.21.15.100:0/670266001 learned_addr learned my addr 172.21.15.100:0/670266001 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:26:25.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.203+0000 7f8cab59e700 1 --2- 172.21.15.100:0/670266001 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ddb711040 0x563ddb713150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:25.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.203+0000 7f8caad9d700 1 --2- 172.21.15.100:0/670266001 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563ddb70c8e0 0x563ddb714790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:25.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.203+0000 7f8caa59c700 1 -- 172.21.15.100:0/670266001 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ddb711040 msgr2=0x563ddb713150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:25.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.203+0000 7f8caa59c700 1 --2- 172.21.15.100:0/670266001 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ddb711040 0x563ddb713150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:25.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.203+0000 7f8caa59c700 1 -- 172.21.15.100:0/670266001 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563ddb70c8e0 msgr2=0x563ddb714790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:25.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.203+0000 7f8caa59c700 1 --2- 172.21.15.100:0/670266001 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563ddb70c8e0 0x563ddb714790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:25.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.203+0000 7f8caa59c700 1 -- 172.21.15.100:0/670266001 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ddb710e70 con 0x563ddb70d200 2024-05-10T11:26:25.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.203+0000 7f8caa59c700 1 --2- 172.21.15.100:0/670266001 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ddb70d200 0x563ddb712330 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f8ca0009750 tx=0x7f8ca00059e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3bf80bdf43f367d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:26:25.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.204+0000 7f8c90ff9700 1 -- 172.21.15.100:0/670266001 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca0027040 con 0x563ddb70d200 2024-05-10T11:26:25.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.204+0000 7f8c90ff9700 1 -- 172.21.15.100:0/670266001 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8ca0003b90 con 0x563ddb70d200 2024-05-10T11:26:25.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.204+0000 7f8c90ff9700 1 -- 172.21.15.100:0/670266001 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca000f460 con 0x563ddb70d200 2024-05-10T11:26:25.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.204+0000 7f8cbb9f8b80 1 -- 172.21.15.100:0/670266001 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ddb70d200 msgr2=0x563ddb712330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:25.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.204+0000 7f8cbb9f8b80 1 --2- 172.21.15.100:0/670266001 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ddb70d200 0x563ddb712330 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f8ca0009750 tx=0x7f8ca00059e0 comp rx=0 tx=0).stop 2024-05-10T11:26:25.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.204+0000 7f8cbb9f8b80 1 -- 172.21.15.100:0/670266001 shutdown_connections 2024-05-10T11:26:25.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.204+0000 7f8cbb9f8b80 1 --2- 172.21.15.100:0/670266001 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ddb711040 0x563ddb713150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:25.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.204+0000 7f8cbb9f8b80 1 --2- 172.21.15.100:0/670266001 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563ddb70c8e0 0x563ddb714790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:25.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.204+0000 7f8cbb9f8b80 1 --2- 172.21.15.100:0/670266001 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ddb70d200 0x563ddb712330 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:25.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.204+0000 7f8cbb9f8b80 1 -- 172.21.15.100:0/670266001 >> 172.21.15.100:0/670266001 conn(0x563ddb634100 msgr2=0x563ddb712a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:26:25.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.204+0000 7f8cbb9f8b80 1 -- 172.21.15.100:0/670266001 shutdown_connections 2024-05-10T11:26:25.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.205+0000 7f8cbb9f8b80 1 -- 172.21.15.100:0/670266001 wait complete. 2024-05-10T11:26:25.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.205+0000 7f8cbb9f8b80 1 -- start start 2024-05-10T11:26:25.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.205+0000 7f8cbb9f8b80 1 Processor -- start 2024-05-10T11:26:25.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.205+0000 7f8cbb9f8b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ddb70c8e0 0x563ddb71e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:25.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.205+0000 7f8cbb9f8b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563ddb70d200 0x563ddb71ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:25.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.205+0000 7f8cbb9f8b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ddb711040 0x563ddb715090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:25.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.205+0000 7f8cbb9f8b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8cac1ab440 con 0x563ddb70c8e0 2024-05-10T11:26:25.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.205+0000 7f8cbb9f8b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8cac1ac8b0 con 0x563ddb70d200 2024-05-10T11:26:25.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.205+0000 7f8cbb9f8b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8cac1abb10 con 0x563ddb711040 2024-05-10T11:26:25.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.205+0000 7f8cbb9f8b80 10 client.0 _ll_register_callbacks cb 0x563ddb5f2920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:26:25.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.205+0000 7f8caad9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ddb70c8e0 0x563ddb71e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:25.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.205+0000 7f8caad9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ddb70c8e0 0x563ddb71e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49186/0 (socket says 172.21.15.100:49186) 2024-05-10T11:26:25.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.205+0000 7f8caad9d700 1 -- 172.21.15.100:0/2332013154 learned_addr learned my addr 172.21.15.100:0/2332013154 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:26:25.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.205+0000 7f8caa59c700 1 --2- 172.21.15.100:0/2332013154 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563ddb70d200 0x563ddb71ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:25.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.205+0000 7f8cab59e700 1 --2- 172.21.15.100:0/2332013154 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ddb711040 0x563ddb715090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:25.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.206+0000 7f8caad9d700 1 -- 172.21.15.100:0/2332013154 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ddb711040 msgr2=0x563ddb715090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:25.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.206+0000 7f8caad9d700 1 --2- 172.21.15.100:0/2332013154 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ddb711040 0x563ddb715090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:25.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.206+0000 7f8caad9d700 1 -- 172.21.15.100:0/2332013154 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563ddb70d200 msgr2=0x563ddb71ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:25.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.206+0000 7f8caad9d700 1 --2- 172.21.15.100:0/2332013154 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563ddb70d200 0x563ddb71ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:25.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.206+0000 7f8caad9d700 1 -- 172.21.15.100:0/2332013154 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ddb7158d0 con 0x563ddb70c8e0 2024-05-10T11:26:25.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.206+0000 7f8cab59e700 1 --2- 172.21.15.100:0/2332013154 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ddb711040 0x563ddb715090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:26:25.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.206+0000 7f8caad9d700 1 --2- 172.21.15.100:0/2332013154 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ddb70c8e0 0x563ddb71e800 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c011ab0 tx=0x7f8c9c013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1b6bf6c82cb320e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:26:25.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.206+0000 7f8c8eff5700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:26:25.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.206+0000 7f8c8eff5700 1 -- 172.21.15.100:0/2332013154 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c01cea0 con 0x563ddb70c8e0 2024-05-10T11:26:25.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.206+0000 7f8c8eff5700 1 -- 172.21.15.100:0/2332013154 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f8c9c02c500 con 0x563ddb70c8e0 2024-05-10T11:26:25.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.206+0000 7f8cbb9f8b80 1 -- 172.21.15.100:0/2332013154 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ca0003b90 con 0x563ddb70c8e0 2024-05-10T11:26:25.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.206+0000 7f8c8eff5700 1 -- 172.21.15.100:0/2332013154 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8c9c03e050 con 0x563ddb70c8e0 2024-05-10T11:26:25.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.206+0000 7f8cbb9f8b80 10 client.17001 fetch_fsmap learned FSMap version 31 2024-05-10T11:26:25.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.206+0000 7f8cbb9f8b80 1 -- 172.21.15.100:0/2332013154 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563ddb7902f0 con 0x563ddb70c8e0 2024-05-10T11:26:25.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.207+0000 7f8c8eff5700 1 -- 172.21.15.100:0/2332013154 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8ca0003b90 con 0x563ddb70c8e0 2024-05-10T11:26:25.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.207+0000 7f8c8eff5700 1 -- 172.21.15.100:0/2332013154 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f8c9c01dc10 con 0x563ddb70c8e0 2024-05-10T11:26:25.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.207+0000 7f8cbb9f8b80 10 client.17001 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:26:25.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.207+0000 7f8cbb9f8b80 10 client.17001 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:26:25.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.207+0000 7f8cbb9f8b80 20 client.17001 populate_metadata read hostname 'smithi100' 2024-05-10T11:26:25.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.207+0000 7f8cbb9f8b80 1 --2- 172.21.15.100:0/2332013154 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x563ddb7909a0 0x563ddb792dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:25.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.207+0000 7f8cbb9f8b80 4 client.17001 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:26:25.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.207+0000 7f8cbb9f8b80 1 -- 172.21.15.100:0/2332013154 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563ddb668240 con 0x563ddb7909a0 2024-05-10T11:26:25.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.207+0000 7f8caa59c700 1 --2- 172.21.15.100:0/2332013154 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x563ddb7909a0 0x563ddb792dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:25.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.208+0000 7f8caa59c700 1 --2- 172.21.15.100:0/2332013154 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x563ddb7909a0 0x563ddb792dc0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60e71f73c7e8dd87 server_cookie=f37967af812a931c in_seq=0 out_seq=0 2024-05-10T11:26:25.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.208+0000 7f8c8eff5700 10 client.17001 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:26:25.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.208+0000 7f8c8eff5700 1 -- 172.21.15.100:0/2332013154 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563ddb668240 con 0x563ddb7909a0 2024-05-10T11:26:25.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.208+0000 7f8c8eff5700 10 client.17001 handle_command_reply: tid=0 2024-05-10T11:26:25.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.208+0000 7f8cbb9f8b80 1 client.17001 shutdown 2024-05-10T11:26:25.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.208+0000 7f8cbb9f8b80 1 -- 172.21.15.100:0/2332013154 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ddb70c8e0 msgr2=0x563ddb71e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:25.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.208+0000 7f8cbb9f8b80 1 --2- 172.21.15.100:0/2332013154 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ddb70c8e0 0x563ddb71e800 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c011ab0 tx=0x7f8c9c013040 comp rx=0 tx=0).stop 2024-05-10T11:26:25.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.208+0000 7f8cbb9f8b80 1 -- 172.21.15.100:0/2332013154 shutdown_connections 2024-05-10T11:26:25.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.208+0000 7f8cbb9f8b80 1 --2- 172.21.15.100:0/2332013154 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x563ddb7909a0 0x563ddb792dc0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:25.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.208+0000 7f8cbb9f8b80 1 --2- 172.21.15.100:0/2332013154 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ddb711040 0x563ddb715090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:25.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.208+0000 7f8cbb9f8b80 1 --2- 172.21.15.100:0/2332013154 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ddb70c8e0 0x563ddb71e800 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:25.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.208+0000 7f8cbb9f8b80 1 --2- 172.21.15.100:0/2332013154 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563ddb70d200 0x563ddb71ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:25.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.208+0000 7f8cbb9f8b80 1 -- 172.21.15.100:0/2332013154 >> 172.21.15.100:0/2332013154 conn(0x563ddb634100 msgr2=0x563ddb6fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:26:25.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.209+0000 7f8c8eff5700 0 client.17001 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:26:25.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.209+0000 7f8cbb9f8b80 1 -- 172.21.15.100:0/2332013154 shutdown_connections 2024-05-10T11:26:25.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.209+0000 7f8cbb9f8b80 1 -- 172.21.15.100:0/2332013154 wait complete. 2024-05-10T11:26:25.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.209+0000 7f8cbb9f8b80 20 client.17001 trim_cache size 0 max 16384 2024-05-10T11:26:25.214 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:26:25.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.209+0000 7f8cb1165700 1 -- 172.21.15.100:0/3290946755 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8c940b2880 msgr2=0x7f8c940b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:25.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.209+0000 7f8cb1165700 1 --2- 172.21.15.100:0/3290946755 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8c940b2880 0x7f8c940b4d40 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c009d60 tx=0x7f8c9c009260 comp rx=0 tx=0).stop 2024-05-10T11:26:25.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.209+0000 7f8cb1165700 1 -- 172.21.15.100:0/3290946755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8cac108410 msgr2=0x7f8cac1a5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:25.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.209+0000 7f8cb1165700 1 --2- 172.21.15.100:0/3290946755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8cac108410 0x7f8cac1a5a20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8ca400be20 tx=0x7f8ca4009ed0 comp rx=0 tx=0).stop 2024-05-10T11:26:25.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.210+0000 7f8cb1165700 1 -- 172.21.15.100:0/3290946755 shutdown_connections 2024-05-10T11:26:25.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.210+0000 7f8cb1165700 1 --2- 172.21.15.100:0/3290946755 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8c940b2880 0x7f8c940b4d40 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:25.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.210+0000 7f8cb1165700 1 --2- 172.21.15.100:0/3290946755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8cac108410 0x7f8cac1a5a20 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:25.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.210+0000 7f8cb1165700 1 --2- 172.21.15.100:0/3290946755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8cac1023a0 0x7f8cac19a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:25.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.210+0000 7f8cb1165700 1 --2- 172.21.15.100:0/3290946755 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8cac102d60 0x7f8cac19aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:25.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.210+0000 7f8cb1165700 1 -- 172.21.15.100:0/3290946755 >> 172.21.15.100:0/3290946755 conn(0x7f8cac079350 msgr2=0x7f8cac1010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:26:25.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.210+0000 7f8cb1165700 1 -- 172.21.15.100:0/3290946755 shutdown_connections 2024-05-10T11:26:25.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:25.210+0000 7f8cb1165700 1 -- 172.21.15.100:0/3290946755 wait complete. 2024-05-10T11:26:25.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:26:25 smithi178 ceph-mon[33359]: pgmap v661: 129 pgs: 129 active+clean; 84 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 1.5 MiB/s wr, 167 op/s 2024-05-10T11:26:25.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:26:25 smithi188 ceph-mon[34075]: pgmap v661: 129 pgs: 129 active+clean; 84 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 1.5 MiB/s wr, 167 op/s 2024-05-10T11:26:25.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:26:25 smithi100 ceph-mon[30296]: pgmap v661: 129 pgs: 129 active+clean; 84 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 1.5 MiB/s wr, 167 op/s 2024-05-10T11:26:25.975 INFO:tasks.workunit.client.0.smithi100.stdout: 1 196330 2.73 MB/sec execute 260 sec latency 18.376 ms 2024-05-10T11:26:26.916 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T11:26:26.917 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T11:26:26.975 INFO:tasks.workunit.client.0.smithi100.stdout: 1 196964 2.73 MB/sec execute 261 sec latency 15.561 ms 2024-05-10T11:26:27.182 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:26:27.555 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:26:27 smithi100 ceph-mon[30296]: pgmap v662: 129 pgs: 129 active+clean; 86 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 1.5 MiB/s wr, 147 op/s 2024-05-10T11:26:27.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:26:27 smithi178 ceph-mon[33359]: pgmap v662: 129 pgs: 129 active+clean; 86 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 1.5 MiB/s wr, 147 op/s 2024-05-10T11:26:27.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:26:27 smithi188 ceph-mon[34075]: pgmap v662: 129 pgs: 129 active+clean; 86 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 1.5 MiB/s wr, 147 op/s 2024-05-10T11:26:27.976 INFO:tasks.workunit.client.0.smithi100.stdout: 1 197550 2.73 MB/sec execute 262 sec latency 14.747 ms 2024-05-10T11:26:28.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.867+0000 7efd8c881700 1 -- 172.21.15.100:0/1906673701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd88108410 msgr2=0x7efd881087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:28.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.867+0000 7efd8c881700 1 --2- 172.21.15.100:0/1906673701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd88108410 0x7efd881087f0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7efd78009990 tx=0x7efd78009ca0 comp rx=0 tx=0).stop 2024-05-10T11:26:28.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.867+0000 7efd8c881700 1 -- 172.21.15.100:0/1906673701 shutdown_connections 2024-05-10T11:26:28.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.867+0000 7efd8c881700 1 --2- 172.21.15.100:0/1906673701 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd88102d60 0x7efd8810f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:28.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.867+0000 7efd8c881700 1 --2- 172.21.15.100:0/1906673701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd88108410 0x7efd881087f0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:28.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.867+0000 7efd8c881700 1 --2- 172.21.15.100:0/1906673701 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd881023a0 0x7efd88102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:28.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.867+0000 7efd8c881700 1 -- 172.21.15.100:0/1906673701 >> 172.21.15.100:0/1906673701 conn(0x7efd88079350 msgr2=0x7efd88079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:26:28.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.867+0000 7efd8c881700 1 -- 172.21.15.100:0/1906673701 shutdown_connections 2024-05-10T11:26:28.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.867+0000 7efd8c881700 1 -- 172.21.15.100:0/1906673701 wait complete. 2024-05-10T11:26:28.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.868+0000 7efd8c881700 1 Processor -- start 2024-05-10T11:26:28.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.868+0000 7efd8c881700 1 -- start start 2024-05-10T11:26:28.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.868+0000 7efd8c881700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd881023a0 0x7efd88104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:28.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.868+0000 7efd8c881700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd88102d60 0x7efd881045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:28.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.868+0000 7efd8c881700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd88108410 0x7efd88104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:28.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.868+0000 7efd8c881700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7efd88112460 con 0x7efd88108410 2024-05-10T11:26:28.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.868+0000 7efd8c881700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efd881122e0 con 0x7efd88102d60 2024-05-10T11:26:28.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.868+0000 7efd8c881700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7efd881125e0 con 0x7efd881023a0 2024-05-10T11:26:28.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.868+0000 7efd86d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd88108410 0x7efd88104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:28.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.868+0000 7efd8659c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd881023a0 0x7efd88104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:28.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.868+0000 7efd8659c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd881023a0 0x7efd88104080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:49968/0 (socket says 172.21.15.100:49968) 2024-05-10T11:26:28.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.868+0000 7efd86d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd88108410 0x7efd88104b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49254/0 (socket says 172.21.15.100:49254) 2024-05-10T11:26:28.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.868+0000 7efd8659c700 1 -- 172.21.15.100:0/4027958186 learned_addr learned my addr 172.21.15.100:0/4027958186 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:26:28.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.868+0000 7efd85d9b700 1 --2- 172.21.15.100:0/4027958186 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd88102d60 0x7efd881045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:28.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.869+0000 7efd8659c700 1 -- 172.21.15.100:0/4027958186 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd88102d60 msgr2=0x7efd881045c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:28.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.869+0000 7efd8659c700 1 --2- 172.21.15.100:0/4027958186 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd88102d60 0x7efd881045c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:28.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.869+0000 7efd8659c700 1 -- 172.21.15.100:0/4027958186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd88108410 msgr2=0x7efd88104b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:28.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.869+0000 7efd8659c700 1 --2- 172.21.15.100:0/4027958186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd88108410 0x7efd88104b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:28.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.869+0000 7efd8659c700 1 -- 172.21.15.100:0/4027958186 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd881a80e0 con 0x7efd881023a0 2024-05-10T11:26:28.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.869+0000 7efd85d9b700 1 --2- 172.21.15.100:0/4027958186 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd88102d60 0x7efd881045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:26:28.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.869+0000 7efd86d9d700 1 --2- 172.21.15.100:0/4027958186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd88108410 0x7efd88104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:26:28.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.869+0000 7efd8659c700 1 --2- 172.21.15.100:0/4027958186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd881023a0 0x7efd88104080 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7efd7800b650 tx=0x7efd7800bfc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:26:28.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.869+0000 7efd777fe700 1 -- 172.21.15.100:0/4027958186 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd78027070 con 0x7efd881023a0 2024-05-10T11:26:28.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.869+0000 7efd8c881700 1 -- 172.21.15.100:0/4027958186 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd881a8370 con 0x7efd881023a0 2024-05-10T11:26:28.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.869+0000 7efd777fe700 1 -- 172.21.15.100:0/4027958186 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efd78022470 con 0x7efd881023a0 2024-05-10T11:26:28.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.869+0000 7efd777fe700 1 -- 172.21.15.100:0/4027958186 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd78017650 con 0x7efd881023a0 2024-05-10T11:26:28.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.869+0000 7efd8c881700 1 -- 172.21.15.100:0/4027958186 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd881a8820 con 0x7efd881023a0 2024-05-10T11:26:28.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.872+0000 7efd777fe700 1 -- 172.21.15.100:0/4027958186 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7efd7802a070 con 0x7efd881023a0 2024-05-10T11:26:28.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.873+0000 7efd777fe700 1 --2- 172.21.15.100:0/4027958186 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7efd700b2880 0x7efd700b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:28.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.873+0000 7efd97114b80 1 Processor -- start 2024-05-10T11:26:28.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.873+0000 7efd85d9b700 1 --2- 172.21.15.100:0/4027958186 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7efd700b2880 0x7efd700b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:28.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.873+0000 7efd777fe700 1 -- 172.21.15.100:0/4027958186 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7efd780e34a0 con 0x7efd881023a0 2024-05-10T11:26:28.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.873+0000 7efd97114b80 1 -- start start 2024-05-10T11:26:28.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.873+0000 7efd97114b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c9bf435550 0x55c9bf435930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:28.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.873+0000 7efd97114b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c9bf435e70 0x55c9bf439770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:28.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.874+0000 7efd97114b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c9bf439d00 0x55c9bf43a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:28.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.874+0000 7efd85d9b700 1 --2- 172.21.15.100:0/4027958186 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7efd700b2880 0x7efd700b4d40 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7efd88079a00 tx=0x7efd7c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:26:28.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.874+0000 7efd97114b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55c9bf3878e0 con 0x55c9bf439d00 2024-05-10T11:26:28.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.874+0000 7efd97114b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c9bf326840 con 0x55c9bf435e70 2024-05-10T11:26:28.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.874+0000 7efd97114b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c9bf3838a0 con 0x55c9bf435550 2024-05-10T11:26:28.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.874+0000 7efd86d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c9bf439d00 0x55c9bf43a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:28.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.874+0000 7efd86d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c9bf439d00 0x55c9bf43a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49262/0 (socket says 172.21.15.100:49262) 2024-05-10T11:26:28.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.874+0000 7efd86d9d700 1 -- 172.21.15.100:0/2981328390 learned_addr learned my addr 172.21.15.100:0/2981328390 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:26:28.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.874+0000 7efd8659c700 1 --2- 172.21.15.100:0/2981328390 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c9bf435550 0x55c9bf435930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:28.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.874+0000 7efd85d9b700 1 --2- 172.21.15.100:0/2981328390 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c9bf435e70 0x55c9bf439770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:28.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.874+0000 7efd86d9d700 1 -- 172.21.15.100:0/2981328390 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c9bf435550 msgr2=0x55c9bf435930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:28.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.874+0000 7efd86d9d700 1 --2- 172.21.15.100:0/2981328390 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c9bf435550 0x55c9bf435930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:28.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.874+0000 7efd86d9d700 1 -- 172.21.15.100:0/2981328390 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c9bf435e70 msgr2=0x55c9bf439770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:28.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.874+0000 7efd86d9d700 1 --2- 172.21.15.100:0/2981328390 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c9bf435e70 0x55c9bf439770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:28.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.874+0000 7efd86d9d700 1 -- 172.21.15.100:0/2981328390 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9bf2d4950 con 0x55c9bf439d00 2024-05-10T11:26:28.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.874+0000 7efd86d9d700 1 --2- 172.21.15.100:0/2981328390 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c9bf439d00 0x55c9bf43a180 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7efd80007740 tx=0x7efd80009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1914a708d1acf88 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:26:28.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.874+0000 7efd74ff9700 1 -- 172.21.15.100:0/2981328390 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd80019020 con 0x55c9bf439d00 2024-05-10T11:26:28.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.874+0000 7efd74ff9700 1 -- 172.21.15.100:0/2981328390 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efd80015ce0 con 0x55c9bf439d00 2024-05-10T11:26:28.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.874+0000 7efd74ff9700 1 -- 172.21.15.100:0/2981328390 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd80014cc0 con 0x55c9bf439d00 2024-05-10T11:26:28.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.875+0000 7efd97114b80 1 -- 172.21.15.100:0/2981328390 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c9bf439d00 msgr2=0x55c9bf43a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:28.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.875+0000 7efd97114b80 1 --2- 172.21.15.100:0/2981328390 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c9bf439d00 0x55c9bf43a180 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7efd80007740 tx=0x7efd80009f90 comp rx=0 tx=0).stop 2024-05-10T11:26:28.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.875+0000 7efd97114b80 1 -- 172.21.15.100:0/2981328390 shutdown_connections 2024-05-10T11:26:28.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.875+0000 7efd97114b80 1 --2- 172.21.15.100:0/2981328390 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c9bf439d00 0x55c9bf43a180 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:28.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.875+0000 7efd97114b80 1 --2- 172.21.15.100:0/2981328390 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c9bf435550 0x55c9bf435930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:28.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.875+0000 7efd97114b80 1 --2- 172.21.15.100:0/2981328390 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c9bf435e70 0x55c9bf439770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:28.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.875+0000 7efd97114b80 1 -- 172.21.15.100:0/2981328390 >> 172.21.15.100:0/2981328390 conn(0x55c9bf361180 msgr2=0x55c9bf42fcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:26:28.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.875+0000 7efd97114b80 1 -- 172.21.15.100:0/2981328390 shutdown_connections 2024-05-10T11:26:28.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.875+0000 7efd97114b80 1 -- 172.21.15.100:0/2981328390 wait complete. 2024-05-10T11:26:28.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.876+0000 7efd97114b80 1 -- start start 2024-05-10T11:26:28.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.876+0000 7efd97114b80 1 Processor -- start 2024-05-10T11:26:28.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.876+0000 7efd97114b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c9bf435550 0x55c9bf441070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:28.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.876+0000 7efd97114b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c9bf435e70 0x55c9bf43e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:28.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.876+0000 7efd97114b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c9bf439d00 0x55c9bf43e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:28.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.876+0000 7efd97114b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55c9bf441f10 con 0x55c9bf435550 2024-05-10T11:26:28.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.876+0000 7efd97114b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c9bf2edcb0 con 0x55c9bf439d00 2024-05-10T11:26:28.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.876+0000 7efd97114b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c9bf442090 con 0x55c9bf435e70 2024-05-10T11:26:28.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.876+0000 7efd97114b80 10 client.0 _ll_register_callbacks cb 0x55c9bf31f9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:26:28.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.876+0000 7efd8659c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c9bf435550 0x55c9bf441070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:28.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.876+0000 7efd8659c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c9bf435550 0x55c9bf441070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49278/0 (socket says 172.21.15.100:49278) 2024-05-10T11:26:28.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.876+0000 7efd8659c700 1 -- 172.21.15.100:0/3544007519 learned_addr learned my addr 172.21.15.100:0/3544007519 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:26:28.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.876+0000 7efd85d9b700 1 --2- 172.21.15.100:0/3544007519 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c9bf435e70 0x55c9bf43e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:28.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.876+0000 7efd86d9d700 1 --2- 172.21.15.100:0/3544007519 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c9bf439d00 0x55c9bf43e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:28.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.877+0000 7efd8659c700 1 -- 172.21.15.100:0/3544007519 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c9bf435e70 msgr2=0x55c9bf43e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:28.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.877+0000 7efd8659c700 1 --2- 172.21.15.100:0/3544007519 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c9bf435e70 0x55c9bf43e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:28.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.877+0000 7efd8659c700 1 -- 172.21.15.100:0/3544007519 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c9bf439d00 msgr2=0x55c9bf43e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:28.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.877+0000 7efd8659c700 1 --2- 172.21.15.100:0/3544007519 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c9bf439d00 0x55c9bf43e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:28.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.877+0000 7efd8659c700 1 -- 172.21.15.100:0/3544007519 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9bf323710 con 0x55c9bf435550 2024-05-10T11:26:28.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.877+0000 7efd86d9d700 1 --2- 172.21.15.100:0/3544007519 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c9bf439d00 0x55c9bf43e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:26:28.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.877+0000 7efd8659c700 1 --2- 172.21.15.100:0/3544007519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c9bf435550 0x55c9bf441070 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7efd780e0c40 tx=0x7efd780aa3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f31e5492cc155cf server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:26:28.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.877+0000 7efd6dffb700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:26:28.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.877+0000 7efd6dffb700 1 -- 172.21.15.100:0/3544007519 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd780aa7f0 con 0x55c9bf435550 2024-05-10T11:26:28.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.877+0000 7efd6dffb700 1 -- 172.21.15.100:0/3544007519 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7efd780e3200 con 0x55c9bf435550 2024-05-10T11:26:28.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.877+0000 7efd97114b80 1 -- 172.21.15.100:0/3544007519 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd80015ce0 con 0x55c9bf435550 2024-05-10T11:26:28.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.877+0000 7efd6dffb700 1 -- 172.21.15.100:0/3544007519 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7efd780aeba0 con 0x55c9bf435550 2024-05-10T11:26:28.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.877+0000 7efd97114b80 10 client.17013 fetch_fsmap learned FSMap version 31 2024-05-10T11:26:28.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.877+0000 7efd97114b80 1 -- 172.21.15.100:0/3544007519 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c9bf44c710 con 0x55c9bf435550 2024-05-10T11:26:28.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.877+0000 7efd6dffb700 1 -- 172.21.15.100:0/3544007519 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efd80015ce0 con 0x55c9bf435550 2024-05-10T11:26:28.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.878+0000 7efd6dffb700 1 -- 172.21.15.100:0/3544007519 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7efd780b3130 con 0x55c9bf435550 2024-05-10T11:26:28.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.878+0000 7efd97114b80 10 client.17013 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:26:28.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.878+0000 7efd97114b80 10 client.17013 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:26:28.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.878+0000 7efd97114b80 20 client.17013 populate_metadata read hostname 'smithi100' 2024-05-10T11:26:28.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.878+0000 7efd97114b80 1 --2- 172.21.15.100:0/3544007519 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c9bf44ebe0 0x55c9bf450fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:28.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.878+0000 7efd97114b80 4 client.17013 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:26:28.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.878+0000 7efd97114b80 1 -- 172.21.15.100:0/3544007519 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9bf4016d0 con 0x55c9bf44ebe0 2024-05-10T11:26:28.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.878+0000 7efd85d9b700 1 --2- 172.21.15.100:0/3544007519 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c9bf44ebe0 0x55c9bf450fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:28.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.878+0000 7efd6dffb700 10 client.17013 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:26:28.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.878+0000 7efd85d9b700 1 --2- 172.21.15.100:0/3544007519 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c9bf44ebe0 0x55c9bf450fd0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f186d324cf08c194 server_cookie=6df1f28d170ffcc7 in_seq=0 out_seq=0 2024-05-10T11:26:28.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.879+0000 7efd6dffb700 1 -- 172.21.15.100:0/3544007519 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c9bf4016d0 con 0x55c9bf44ebe0 2024-05-10T11:26:28.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.879+0000 7efd6dffb700 10 client.17013 handle_command_reply: tid=0 2024-05-10T11:26:28.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.879+0000 7efd97114b80 1 client.17013 shutdown 2024-05-10T11:26:28.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.879+0000 7efd97114b80 1 -- 172.21.15.100:0/3544007519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c9bf435550 msgr2=0x55c9bf441070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:28.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.879+0000 7efd97114b80 1 --2- 172.21.15.100:0/3544007519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c9bf435550 0x55c9bf441070 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7efd780e0c40 tx=0x7efd780aa3d0 comp rx=0 tx=0).stop 2024-05-10T11:26:28.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.879+0000 7efd97114b80 1 -- 172.21.15.100:0/3544007519 shutdown_connections 2024-05-10T11:26:28.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.879+0000 7efd97114b80 1 --2- 172.21.15.100:0/3544007519 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c9bf44ebe0 0x55c9bf450fd0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:28.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.879+0000 7efd97114b80 1 --2- 172.21.15.100:0/3544007519 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c9bf439d00 0x55c9bf43e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:28.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.879+0000 7efd97114b80 1 --2- 172.21.15.100:0/3544007519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c9bf435550 0x55c9bf441070 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:28.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.879+0000 7efd97114b80 1 --2- 172.21.15.100:0/3544007519 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c9bf435e70 0x55c9bf43e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:28.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.879+0000 7efd97114b80 1 -- 172.21.15.100:0/3544007519 >> 172.21.15.100:0/3544007519 conn(0x55c9bf361180 msgr2=0x55c9bf43a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:26:28.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.879+0000 7efd6dffb700 0 client.17013 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:26:28.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.879+0000 7efd97114b80 1 -- 172.21.15.100:0/3544007519 shutdown_connections 2024-05-10T11:26:28.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.879+0000 7efd97114b80 1 -- 172.21.15.100:0/3544007519 wait complete. 2024-05-10T11:26:28.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.880+0000 7efd97114b80 20 client.17013 trim_cache size 0 max 16384 2024-05-10T11:26:28.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.904+0000 7efd97114b80 1 Processor -- start 2024-05-10T11:26:28.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.904+0000 7efd97114b80 1 -- start start 2024-05-10T11:26:28.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.904+0000 7efd97114b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c9bf361180 0x55c9bf43ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:28.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.904+0000 7efd97114b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c9bf435550 0x55c9bf44afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:28.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.904+0000 7efd97114b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c9bf435e70 0x55c9bf437670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:28.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.904+0000 7efd97114b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7efd881b3af0 con 0x55c9bf435e70 2024-05-10T11:26:28.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.904+0000 7efd97114b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efd881b1380 con 0x55c9bf361180 2024-05-10T11:26:28.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.904+0000 7efd97114b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7efd881af1d0 con 0x55c9bf435550 2024-05-10T11:26:28.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.905+0000 7efd86d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c9bf435e70 0x55c9bf437670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:28.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.905+0000 7efd86d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c9bf435e70 0x55c9bf437670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49288/0 (socket says 172.21.15.100:49288) 2024-05-10T11:26:28.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.905+0000 7efd86d9d700 1 -- 172.21.15.100:0/1502298218 learned_addr learned my addr 172.21.15.100:0/1502298218 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:26:28.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.905+0000 7efd8659c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c9bf361180 0x55c9bf43ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:28.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.905+0000 7efd85d9b700 1 --2- 172.21.15.100:0/1502298218 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c9bf435550 0x55c9bf44afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:28.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.905+0000 7efd86d9d700 1 -- 172.21.15.100:0/1502298218 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c9bf435550 msgr2=0x55c9bf44afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:28.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.905+0000 7efd86d9d700 1 --2- 172.21.15.100:0/1502298218 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c9bf435550 0x55c9bf44afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:28.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.905+0000 7efd86d9d700 1 -- 172.21.15.100:0/1502298218 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c9bf361180 msgr2=0x55c9bf43ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:28.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.905+0000 7efd86d9d700 1 --2- 172.21.15.100:0/1502298218 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c9bf361180 0x55c9bf43ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:28.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.905+0000 7efd86d9d700 1 -- 172.21.15.100:0/1502298218 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9bf42a580 con 0x55c9bf435e70 2024-05-10T11:26:28.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.905+0000 7efd8659c700 1 --2- 172.21.15.100:0/1502298218 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c9bf361180 0x55c9bf43ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:26:28.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.905+0000 7efd86d9d700 1 --2- 172.21.15.100:0/1502298218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c9bf435e70 0x55c9bf437670 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7efd80007660 tx=0x7efd80009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6ff5a6d1118b368 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:26:28.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.905+0000 7efd6ffff700 1 -- 172.21.15.100:0/1502298218 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd80019040 con 0x55c9bf435e70 2024-05-10T11:26:28.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.905+0000 7efd6ffff700 1 -- 172.21.15.100:0/1502298218 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efd80022070 con 0x55c9bf435e70 2024-05-10T11:26:28.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.905+0000 7efd6ffff700 1 -- 172.21.15.100:0/1502298218 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd80015940 con 0x55c9bf435e70 2024-05-10T11:26:28.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.906+0000 7efd97114b80 1 -- 172.21.15.100:0/1502298218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c9bf435e70 msgr2=0x55c9bf437670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:28.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.906+0000 7efd97114b80 1 --2- 172.21.15.100:0/1502298218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c9bf435e70 0x55c9bf437670 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7efd80007660 tx=0x7efd80009820 comp rx=0 tx=0).stop 2024-05-10T11:26:28.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.906+0000 7efd97114b80 1 -- 172.21.15.100:0/1502298218 shutdown_connections 2024-05-10T11:26:28.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.906+0000 7efd97114b80 1 --2- 172.21.15.100:0/1502298218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c9bf435e70 0x55c9bf437670 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:28.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.906+0000 7efd97114b80 1 --2- 172.21.15.100:0/1502298218 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c9bf361180 0x55c9bf43ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:28.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.906+0000 7efd97114b80 1 --2- 172.21.15.100:0/1502298218 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c9bf435550 0x55c9bf44afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:28.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.906+0000 7efd97114b80 1 -- 172.21.15.100:0/1502298218 >> 172.21.15.100:0/1502298218 conn(0x55c9bf42ab30 msgr2=0x55c9bf434bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:26:28.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.906+0000 7efd97114b80 1 -- 172.21.15.100:0/1502298218 shutdown_connections 2024-05-10T11:26:28.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.906+0000 7efd97114b80 1 -- 172.21.15.100:0/1502298218 wait complete. 2024-05-10T11:26:28.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.906+0000 7efd97114b80 1 -- start start 2024-05-10T11:26:28.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.907+0000 7efd97114b80 1 Processor -- start 2024-05-10T11:26:28.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.907+0000 7efd97114b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c9bf361180 0x55c9bf447a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:28.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.907+0000 7efd97114b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c9bf435550 0x55c9bf443d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:28.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.907+0000 7efd97114b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c9bf435e70 0x55c9bf444480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:28.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.907+0000 7efd97114b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7efd881af840 con 0x55c9bf361180 2024-05-10T11:26:28.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.907+0000 7efd97114b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efd881b0cb0 con 0x55c9bf435e70 2024-05-10T11:26:28.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.907+0000 7efd97114b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7efd881aff10 con 0x55c9bf435550 2024-05-10T11:26:28.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.907+0000 7efd97114b80 10 client.0 _ll_register_callbacks cb 0x55c9bf31f9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:26:28.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.907+0000 7efd8659c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c9bf361180 0x55c9bf447a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:28.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.907+0000 7efd8659c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c9bf361180 0x55c9bf447a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49302/0 (socket says 172.21.15.100:49302) 2024-05-10T11:26:28.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.907+0000 7efd85d9b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c9bf435550 0x55c9bf443d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:28.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.907+0000 7efd8659c700 1 -- 172.21.15.100:0/1097970457 learned_addr learned my addr 172.21.15.100:0/1097970457 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:26:28.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.907+0000 7efd86d9d700 1 --2- 172.21.15.100:0/1097970457 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c9bf435e70 0x55c9bf444480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:28.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.907+0000 7efd8659c700 1 -- 172.21.15.100:0/1097970457 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c9bf435550 msgr2=0x55c9bf443d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:28.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.907+0000 7efd8659c700 1 --2- 172.21.15.100:0/1097970457 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c9bf435550 0x55c9bf443d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:28.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.907+0000 7efd8659c700 1 -- 172.21.15.100:0/1097970457 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c9bf435e70 msgr2=0x55c9bf444480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:28.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.907+0000 7efd8659c700 1 --2- 172.21.15.100:0/1097970457 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c9bf435e70 0x55c9bf444480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:28.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.907+0000 7efd8659c700 1 -- 172.21.15.100:0/1097970457 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9bf4b94a0 con 0x55c9bf361180 2024-05-10T11:26:28.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.907+0000 7efd85d9b700 1 --2- 172.21.15.100:0/1097970457 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c9bf435550 0x55c9bf443d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:26:28.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.907+0000 7efd86d9d700 1 --2- 172.21.15.100:0/1097970457 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c9bf435e70 0x55c9bf444480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:26:28.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.907+0000 7efd8659c700 1 --2- 172.21.15.100:0/1097970457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c9bf361180 0x55c9bf447a00 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7efd780a60f0 tx=0x7efd780e3430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e474c88b4c64d5a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:26:28.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.907+0000 7efd6dffb700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:26:28.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.908+0000 7efd6dffb700 1 -- 172.21.15.100:0/1097970457 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd780b35c0 con 0x55c9bf361180 2024-05-10T11:26:28.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.908+0000 7efd6dffb700 1 -- 172.21.15.100:0/1097970457 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7efd780b3130 con 0x55c9bf361180 2024-05-10T11:26:28.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.908+0000 7efd97114b80 1 -- 172.21.15.100:0/1097970457 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd80022070 con 0x55c9bf361180 2024-05-10T11:26:28.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.908+0000 7efd6dffb700 1 -- 172.21.15.100:0/1097970457 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7efd780aa7f0 con 0x55c9bf361180 2024-05-10T11:26:28.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.908+0000 7efd97114b80 10 client.17019 fetch_fsmap learned FSMap version 31 2024-05-10T11:26:28.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.908+0000 7efd97114b80 1 -- 172.21.15.100:0/1097970457 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c9bf4b94a0 con 0x55c9bf361180 2024-05-10T11:26:28.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.908+0000 7efd6dffb700 1 -- 172.21.15.100:0/1097970457 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efd80022070 con 0x55c9bf361180 2024-05-10T11:26:28.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.908+0000 7efd6dffb700 1 -- 172.21.15.100:0/1097970457 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7efd780bfbb0 con 0x55c9bf361180 2024-05-10T11:26:28.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.908+0000 7efd97114b80 10 client.17019 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:26:28.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.908+0000 7efd97114b80 10 client.17019 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:26:28.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.908+0000 7efd97114b80 20 client.17019 populate_metadata read hostname 'smithi100' 2024-05-10T11:26:28.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.908+0000 7efd97114b80 1 --2- 172.21.15.100:0/1097970457 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c9bf439d00 0x55c9bf4bcae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:28.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.908+0000 7efd97114b80 4 client.17019 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T11:26:28.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.908+0000 7efd97114b80 1 -- 172.21.15.100:0/1097970457 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c9bf3952c0 con 0x55c9bf439d00 2024-05-10T11:26:28.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.909+0000 7efd85d9b700 1 --2- 172.21.15.100:0/1097970457 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c9bf439d00 0x55c9bf4bcae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:28.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.909+0000 7efd6dffb700 10 client.17019 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:26:28.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.909+0000 7efd85d9b700 1 --2- 172.21.15.100:0/1097970457 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c9bf439d00 0x55c9bf4bcae0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28b83e120d05d445 server_cookie=88b6bfeb890b545a in_seq=0 out_seq=0 2024-05-10T11:26:28.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.910+0000 7efd6dffb700 1 -- 172.21.15.100:0/1097970457 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c9bf3952c0 con 0x55c9bf439d00 2024-05-10T11:26:28.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.910+0000 7efd6dffb700 10 client.17019 handle_command_reply: tid=0 2024-05-10T11:26:28.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.910+0000 7efd97114b80 1 client.17019 shutdown 2024-05-10T11:26:28.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.910+0000 7efd97114b80 1 -- 172.21.15.100:0/1097970457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c9bf361180 msgr2=0x55c9bf447a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:28.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.910+0000 7efd97114b80 1 --2- 172.21.15.100:0/1097970457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c9bf361180 0x55c9bf447a00 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7efd780a60f0 tx=0x7efd780e3430 comp rx=0 tx=0).stop 2024-05-10T11:26:28.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.910+0000 7efd97114b80 1 -- 172.21.15.100:0/1097970457 shutdown_connections 2024-05-10T11:26:28.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.910+0000 7efd97114b80 1 --2- 172.21.15.100:0/1097970457 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c9bf439d00 0x55c9bf4bcae0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:28.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.910+0000 7efd97114b80 1 --2- 172.21.15.100:0/1097970457 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c9bf435e70 0x55c9bf444480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:28.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.910+0000 7efd97114b80 1 --2- 172.21.15.100:0/1097970457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c9bf361180 0x55c9bf447a00 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:28.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.910+0000 7efd97114b80 1 --2- 172.21.15.100:0/1097970457 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c9bf435550 0x55c9bf443d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:28.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.910+0000 7efd97114b80 1 -- 172.21.15.100:0/1097970457 >> 172.21.15.100:0/1097970457 conn(0x55c9bf42ab30 msgr2=0x55c9bf4385a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:26:28.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.910+0000 7efd6dffb700 0 client.17019 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:26:28.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.910+0000 7efd97114b80 1 -- 172.21.15.100:0/1097970457 shutdown_connections 2024-05-10T11:26:28.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.910+0000 7efd97114b80 1 -- 172.21.15.100:0/1097970457 wait complete. 2024-05-10T11:26:28.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.911+0000 7efd97114b80 20 client.17019 trim_cache size 0 max 16384 2024-05-10T11:26:28.915 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:26:28.916 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-05-10T11:26:28.916 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "a000f920-7e29-4990-ae30-67c162903c02", 2024-05-10T11:26:28.916 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-05-10T11:26:28.916 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:26:28.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.911+0000 7efd8c881700 1 -- 172.21.15.100:0/4027958186 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7efd700b2880 msgr2=0x7efd700b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:28.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.911+0000 7efd8c881700 1 --2- 172.21.15.100:0/4027958186 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7efd700b2880 0x7efd700b4d40 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7efd88079a00 tx=0x7efd7c009260 comp rx=0 tx=0).stop 2024-05-10T11:26:28.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.911+0000 7efd8c881700 1 -- 172.21.15.100:0/4027958186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd881023a0 msgr2=0x7efd88104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:28.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.911+0000 7efd8c881700 1 --2- 172.21.15.100:0/4027958186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd881023a0 0x7efd88104080 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7efd7800b650 tx=0x7efd7800bfc0 comp rx=0 tx=0).stop 2024-05-10T11:26:28.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.911+0000 7efd8c881700 1 -- 172.21.15.100:0/4027958186 shutdown_connections 2024-05-10T11:26:28.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.911+0000 7efd8c881700 1 --2- 172.21.15.100:0/4027958186 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7efd700b2880 0x7efd700b4d40 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:28.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.911+0000 7efd8c881700 1 --2- 172.21.15.100:0/4027958186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd88108410 0x7efd88104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:28.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.911+0000 7efd8c881700 1 --2- 172.21.15.100:0/4027958186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd881023a0 0x7efd88104080 secure :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7efd7800b650 tx=0x7efd7800bfc0 comp rx=0 tx=0).stop 2024-05-10T11:26:28.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.911+0000 7efd8c881700 1 --2- 172.21.15.100:0/4027958186 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd88102d60 0x7efd881045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:28.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.911+0000 7efd8c881700 1 -- 172.21.15.100:0/4027958186 >> 172.21.15.100:0/4027958186 conn(0x7efd88079350 msgr2=0x7efd8806b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:26:28.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.911+0000 7efd8c881700 1 -- 172.21.15.100:0/4027958186 shutdown_connections 2024-05-10T11:26:28.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:28.912+0000 7efd8c881700 1 -- 172.21.15.100:0/4027958186 wait complete. 2024-05-10T11:26:28.976 INFO:tasks.workunit.client.0.smithi100.stdout: 1 198297 2.74 MB/sec execute 263 sec latency 31.304 ms 2024-05-10T11:26:29.013 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.011+0000 7f04224a7640 1 -- 172.21.15.100:0/1870258909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f041c101910 msgr2=0x7f041c101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:29.013 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.011+0000 7f04224a7640 1 --2- 172.21.15.100:0/1870258909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f041c101910 0x7f041c101d30 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f040c009ca0 tx=0x7f040c02a8c0 comp rx=0 tx=0).stop 2024-05-10T11:26:29.013 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.011+0000 7f04224a7640 1 -- 172.21.15.100:0/1870258909 shutdown_connections 2024-05-10T11:26:29.013 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.011+0000 7f04224a7640 1 --2- 172.21.15.100:0/1870258909 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f041c1034c0 0x7f041c109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:29.013 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.011+0000 7f04224a7640 1 --2- 172.21.15.100:0/1870258909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f041c102b00 0x7f041c102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:29.013 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.011+0000 7f04224a7640 1 --2- 172.21.15.100:0/1870258909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f041c101910 0x7f041c101d30 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:29.013 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.011+0000 7f04224a7640 1 -- 172.21.15.100:0/1870258909 >> 172.21.15.100:0/1870258909 conn(0x7f041c0fce80 msgr2=0x7f041c0ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:26:29.013 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.011+0000 7f04224a7640 1 -- 172.21.15.100:0/1870258909 shutdown_connections 2024-05-10T11:26:29.013 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.011+0000 7f04224a7640 1 -- 172.21.15.100:0/1870258909 wait complete. 2024-05-10T11:26:29.013 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.012+0000 7f04224a7640 1 Processor -- start 2024-05-10T11:26:29.013 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.012+0000 7f04224a7640 1 -- start start 2024-05-10T11:26:29.013 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.012+0000 7f04224a7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f041c101910 0x7f041c06abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:29.014 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.012+0000 7f04224a7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f041c102b00 0x7f041c19d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:29.014 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.012+0000 7f04224a7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f041c1034c0 0x7f041c19fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:29.014 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.012+0000 7f04224a7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f041c10d340 con 0x7f041c102b00 2024-05-10T11:26:29.014 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.012+0000 7f04224a7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f041c10d1c0 con 0x7f041c101910 2024-05-10T11:26:29.014 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.012+0000 7f04224a7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f041c10d4c0 con 0x7f041c1034c0 2024-05-10T11:26:29.014 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.012+0000 7f040bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f041c102b00 0x7f041c19d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:29.014 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.012+0000 7f040bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f041c102b00 0x7f041c19d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49324/0 (socket says 172.21.15.100:49324) 2024-05-10T11:26:29.014 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.012+0000 7f040bfff640 1 -- 172.21.15.100:0/2553350533 learned_addr learned my addr 172.21.15.100:0/2553350533 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:26:29.014 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.012+0000 7f041bfff640 1 --2- 172.21.15.100:0/2553350533 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f041c101910 0x7f041c06abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:29.014 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.012+0000 7f0420a44640 1 --2- 172.21.15.100:0/2553350533 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f041c1034c0 0x7f041c19fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:29.014 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.012+0000 7f040bfff640 1 -- 172.21.15.100:0/2553350533 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f041c1034c0 msgr2=0x7f041c19fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:29.014 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.012+0000 7f040bfff640 1 --2- 172.21.15.100:0/2553350533 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f041c1034c0 0x7f041c19fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:29.014 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.012+0000 7f040bfff640 1 -- 172.21.15.100:0/2553350533 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f041c101910 msgr2=0x7f041c06abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:29.014 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.012+0000 7f040bfff640 1 --2- 172.21.15.100:0/2553350533 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f041c101910 0x7f041c06abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:29.015 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.012+0000 7f040bfff640 1 -- 172.21.15.100:0/2553350533 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f041c1a0760 con 0x7f041c102b00 2024-05-10T11:26:29.015 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.012+0000 7f040bfff640 1 --2- 172.21.15.100:0/2553350533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f041c102b00 0x7f041c19d830 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f041000e9f0 tx=0x7f041000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:26:29.015 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.013+0000 7f0419ffb640 1 -- 172.21.15.100:0/2553350533 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041000cd80 con 0x7f041c102b00 2024-05-10T11:26:29.015 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.013+0000 7f04224a7640 1 -- 172.21.15.100:0/2553350533 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f041c1a09f0 con 0x7f041c102b00 2024-05-10T11:26:29.015 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.013+0000 7f0419ffb640 1 -- 172.21.15.100:0/2553350533 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f04100108c0 con 0x7f041c102b00 2024-05-10T11:26:29.015 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.013+0000 7f0419ffb640 1 -- 172.21.15.100:0/2553350533 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04100187c0 con 0x7f041c102b00 2024-05-10T11:26:29.015 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.013+0000 7f04224a7640 1 -- 172.21.15.100:0/2553350533 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f041c1a0f90 con 0x7f041c102b00 2024-05-10T11:26:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.015+0000 7f04251c7740 1 Processor -- start 2024-05-10T11:26:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.015+0000 7f0419ffb640 1 -- 172.21.15.100:0/2553350533 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f0410010430 con 0x7f041c102b00 2024-05-10T11:26:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.016+0000 7f04251c7740 1 -- start start 2024-05-10T11:26:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.016+0000 7f04251c7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c816eddb0 0x562c816ee1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.016+0000 7f04251c7740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c816ee7a0 0x562c816f32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.016+0000 7f04251c7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c816f3910 0x562c816f5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.016+0000 7f04251c7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x562c816525d0 con 0x562c816f3910 2024-05-10T11:26:29.018 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.016+0000 7f04251c7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562c8160b890 con 0x562c816eddb0 2024-05-10T11:26:29.018 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.016+0000 7f04251c7740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x562c8164e590 con 0x562c816ee7a0 2024-05-10T11:26:29.018 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.016+0000 7f041bfff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c816ee7a0 0x562c816f32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:29.018 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.016+0000 7f040bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c816f3910 0x562c816f5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:29.018 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.016+0000 7f041bfff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c816ee7a0 0x562c816f32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:50054/0 (socket says 172.21.15.100:50054) 2024-05-10T11:26:29.018 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.016+0000 7f041bfff640 1 -- 172.21.15.100:0/2870424775 learned_addr learned my addr 172.21.15.100:0/2870424775 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:26:29.018 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.016+0000 7f0420a44640 1 --2- 172.21.15.100:0/2870424775 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c816eddb0 0x562c816ee1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:29.018 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.016+0000 7f041bfff640 1 -- 172.21.15.100:0/2870424775 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c816eddb0 msgr2=0x562c816ee1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:29.018 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.016+0000 7f041bfff640 1 --2- 172.21.15.100:0/2870424775 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c816eddb0 0x562c816ee1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:29.018 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.016+0000 7f041bfff640 1 -- 172.21.15.100:0/2870424775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c816f3910 msgr2=0x562c816f5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:29.018 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.016+0000 7f041bfff640 1 --2- 172.21.15.100:0/2870424775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c816f3910 0x562c816f5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:29.019 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.016+0000 7f041bfff640 1 -- 172.21.15.100:0/2870424775 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c81364210 con 0x562c816ee7a0 2024-05-10T11:26:29.019 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.016+0000 7f0419ffb640 1 --2- 172.21.15.100:0/2553350533 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f04040b28a0 0x7f04040b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:29.019 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.016+0000 7f041bfff640 1 --2- 172.21.15.100:0/2870424775 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c816ee7a0 0x562c816f32a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f041c102960 tx=0x7f040c009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d25d80ba27e9cc09 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:26:29.019 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.017+0000 7f0420a44640 1 --2- 172.21.15.100:0/2553350533 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f04040b28a0 0x7f04040b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:29.019 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.017+0000 7f040a7fc640 1 -- 172.21.15.100:0/2870424775 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c031650 con 0x562c816ee7a0 2024-05-10T11:26:29.019 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.017+0000 7f0419ffb640 1 -- 172.21.15.100:0/2553350533 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f04100e2900 con 0x7f041c102b00 2024-05-10T11:26:29.019 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.017+0000 7f040a7fc640 1 -- 172.21.15.100:0/2870424775 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f040c036030 con 0x562c816ee7a0 2024-05-10T11:26:29.019 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.017+0000 7f040a7fc640 1 -- 172.21.15.100:0/2870424775 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c031650 con 0x562c816ee7a0 2024-05-10T11:26:29.023 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.017+0000 7f0420a44640 1 --2- 172.21.15.100:0/2553350533 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f04040b28a0 0x7f04040b4d60 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x562c816ef000 tx=0x7f0414005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:26:29.023 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.017+0000 7f04251c7740 1 -- 172.21.15.100:0/2870424775 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c816ee7a0 msgr2=0x562c816f32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:29.023 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.018+0000 7f04251c7740 1 --2- 172.21.15.100:0/2870424775 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c816ee7a0 0x562c816f32a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f041c102960 tx=0x7f040c009a20 comp rx=0 tx=0).stop 2024-05-10T11:26:29.023 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.018+0000 7f04251c7740 1 -- 172.21.15.100:0/2870424775 shutdown_connections 2024-05-10T11:26:29.023 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.018+0000 7f04251c7740 1 --2- 172.21.15.100:0/2870424775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c816f3910 0x562c816f5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:29.024 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.018+0000 7f04251c7740 1 --2- 172.21.15.100:0/2870424775 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c816ee7a0 0x562c816f32a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:29.024 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.018+0000 7f04251c7740 1 --2- 172.21.15.100:0/2870424775 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c816eddb0 0x562c816ee1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:29.024 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.018+0000 7f04251c7740 1 -- 172.21.15.100:0/2870424775 >> 172.21.15.100:0/2870424775 conn(0x562c816e9420 msgr2=0x562c816e9820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:26:29.024 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.018+0000 7f04251c7740 1 -- 172.21.15.100:0/2870424775 shutdown_connections 2024-05-10T11:26:29.024 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.018+0000 7f04251c7740 1 -- 172.21.15.100:0/2870424775 wait complete. 2024-05-10T11:26:29.024 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.018+0000 7f04251c7740 1 -- start start 2024-05-10T11:26:29.024 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.018+0000 7f04251c7740 1 Processor -- start 2024-05-10T11:26:29.024 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.018+0000 7f04251c7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c816eddb0 0x562c816f7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:29.024 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.018+0000 7f04251c7740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c816ee7a0 0x562c816f82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:29.024 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.018+0000 7f04251c7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c816f3910 0x562c816ff330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:29.024 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.018+0000 7f04251c7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x562c816f66e0 con 0x562c816f3910 2024-05-10T11:26:29.024 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.019+0000 7f0420a44640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c816f3910 0x562c816ff330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:29.024 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.019+0000 7f040bfff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c816ee7a0 0x562c816f82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:29.024 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.019+0000 7f0420a44640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c816f3910 0x562c816ff330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49336/0 (socket says 172.21.15.100:49336) 2024-05-10T11:26:29.024 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.019+0000 7f0420a44640 1 -- 172.21.15.100:0/2946360988 learned_addr learned my addr 172.21.15.100:0/2946360988 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:26:29.025 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.019+0000 7f04251c7740 1 -- 172.21.15.100:0/2946360988 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562c8160f370 con 0x562c816eddb0 2024-05-10T11:26:29.025 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.019+0000 7f04251c7740 1 -- 172.21.15.100:0/2946360988 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x562c816f6860 con 0x562c816ee7a0 2024-05-10T11:26:29.025 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.019+0000 7f041bfff640 1 --2- 172.21.15.100:0/2946360988 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c816eddb0 0x562c816f7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:29.025 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.019+0000 7f04251c7740 10 client.0 _ll_register_callbacks cb 0x562c81363a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:26:29.025 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.019+0000 7f0420a44640 1 -- 172.21.15.100:0/2946360988 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c816ee7a0 msgr2=0x562c816f82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:29.025 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.019+0000 7f0420a44640 1 --2- 172.21.15.100:0/2946360988 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c816ee7a0 0x562c816f82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:29.025 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.019+0000 7f0420a44640 1 -- 172.21.15.100:0/2946360988 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c816eddb0 msgr2=0x562c816f7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:29.025 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.019+0000 7f0420a44640 1 --2- 172.21.15.100:0/2946360988 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c816eddb0 0x562c816f7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:29.025 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.019+0000 7f0420a44640 1 -- 172.21.15.100:0/2946360988 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c81430620 con 0x562c816f3910 2024-05-10T11:26:29.025 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.019+0000 7f040bfff640 1 --2- 172.21.15.100:0/2946360988 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c816ee7a0 0x562c816f82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:26:29.025 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.019+0000 7f0420a44640 1 --2- 172.21.15.100:0/2946360988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c816f3910 0x562c816ff330 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f041401e610 tx=0x7f0414004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=373f7ca07520e3dd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:26:29.025 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.019+0000 7f03fb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:26:29.025 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.019+0000 7f03fb7fe640 1 -- 172.21.15.100:0/2946360988 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041402e060 con 0x562c816f3910 2024-05-10T11:26:29.025 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.019+0000 7f03fb7fe640 1 -- 172.21.15.100:0/2946360988 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f0414018030 con 0x562c816f3910 2024-05-10T11:26:29.025 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.020+0000 7f04251c7740 1 -- 172.21.15.100:0/2946360988 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f040c036030 con 0x562c816f3910 2024-05-10T11:26:29.025 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.020+0000 7f03fb7fe640 1 -- 172.21.15.100:0/2946360988 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f041401ec00 con 0x562c816f3910 2024-05-10T11:26:29.026 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.020+0000 7f04251c7740 10 client.17031 fetch_fsmap learned FSMap version 31 2024-05-10T11:26:29.026 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.020+0000 7f04251c7740 1 -- 172.21.15.100:0/2946360988 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562c817023e0 con 0x562c816f3910 2024-05-10T11:26:29.026 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.020+0000 7f03fb7fe640 1 -- 172.21.15.100:0/2946360988 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f040c036030 con 0x562c816f3910 2024-05-10T11:26:29.026 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.020+0000 7f03fb7fe640 1 -- 172.21.15.100:0/2946360988 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f0414005330 con 0x562c816f3910 2024-05-10T11:26:29.026 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.020+0000 7f04251c7740 10 client.17031 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:26:29.026 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.020+0000 7f04251c7740 10 client.17031 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:26:29.026 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.020+0000 7f04251c7740 20 client.17031 populate_metadata read hostname 'smithi100' 2024-05-10T11:26:29.026 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.020+0000 7f04251c7740 1 --2- 172.21.15.100:0/2946360988 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562c81703b20 0x562c81705f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:29.026 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.020+0000 7f04251c7740 4 client.17031 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:26:29.026 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.020+0000 7f04251c7740 1 -- 172.21.15.100:0/2946360988 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c816601e0 con 0x562c81703b20 2024-05-10T11:26:29.026 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.020+0000 7f041bfff640 1 --2- 172.21.15.100:0/2946360988 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562c81703b20 0x562c81705f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:29.026 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.021+0000 7f03fb7fe640 10 client.17031 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:26:29.026 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.021+0000 7f041bfff640 1 --2- 172.21.15.100:0/2946360988 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562c81703b20 0x562c81705f10 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5f6b2b2b6c3b686 server_cookie=6703836b9345b17c in_seq=0 out_seq=0 2024-05-10T11:26:29.026 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.021+0000 7f03fb7fe640 1 -- 172.21.15.100:0/2946360988 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562c816601e0 con 0x562c81703b20 2024-05-10T11:26:29.027 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.021+0000 7f03fb7fe640 10 client.17031 handle_command_reply: tid=0 2024-05-10T11:26:29.027 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.021+0000 7f04251c7740 1 client.17031 shutdown 2024-05-10T11:26:29.027 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.022+0000 7f04251c7740 1 -- 172.21.15.100:0/2946360988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c816f3910 msgr2=0x562c816ff330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:29.027 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.022+0000 7f04251c7740 1 --2- 172.21.15.100:0/2946360988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c816f3910 0x562c816ff330 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f041401e610 tx=0x7f0414004730 comp rx=0 tx=0).stop 2024-05-10T11:26:29.027 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.022+0000 7f04251c7740 1 -- 172.21.15.100:0/2946360988 shutdown_connections 2024-05-10T11:26:29.027 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.022+0000 7f04251c7740 1 --2- 172.21.15.100:0/2946360988 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562c81703b20 0x562c81705f10 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:29.027 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.022+0000 7f04251c7740 1 --2- 172.21.15.100:0/2946360988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c816f3910 0x562c816ff330 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:29.027 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.022+0000 7f04251c7740 1 --2- 172.21.15.100:0/2946360988 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c816ee7a0 0x562c816f82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:29.027 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.022+0000 7f03fb7fe640 0 client.17031 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:26:29.027 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.022+0000 7f04251c7740 1 --2- 172.21.15.100:0/2946360988 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c816eddb0 0x562c816f7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:29.027 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.022+0000 7f04251c7740 1 -- 172.21.15.100:0/2946360988 >> 172.21.15.100:0/2946360988 conn(0x562c816e9420 msgr2=0x562c816e8580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:26:29.027 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.022+0000 7f04251c7740 1 -- 172.21.15.100:0/2946360988 shutdown_connections 2024-05-10T11:26:29.027 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.022+0000 7f04251c7740 1 -- 172.21.15.100:0/2946360988 wait complete. 2024-05-10T11:26:29.027 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.022+0000 7f04251c7740 20 client.17031 trim_cache size 0 max 16384 2024-05-10T11:26:29.042 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.038+0000 7f04251c7740 1 Processor -- start 2024-05-10T11:26:29.042 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.038+0000 7f04251c7740 1 -- start start 2024-05-10T11:26:29.042 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.038+0000 7f04251c7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c816eddb0 0x562c816ef980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:29.042 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.038+0000 7f04251c7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c816ee7a0 0x562c816efec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:29.042 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.038+0000 7f04251c7740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c816f3910 0x562c816ff2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:29.042 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.038+0000 7f04251c7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f041c1adc20 con 0x562c816ee7a0 2024-05-10T11:26:29.042 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.038+0000 7f04251c7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f041c1ab4f0 con 0x562c816eddb0 2024-05-10T11:26:29.042 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.038+0000 7f04251c7740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f041c1a9340 con 0x562c816f3910 2024-05-10T11:26:29.042 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.038+0000 7f040bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c816ee7a0 0x562c816efec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:29.043 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.038+0000 7f040bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c816ee7a0 0x562c816efec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49342/0 (socket says 172.21.15.100:49342) 2024-05-10T11:26:29.043 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.038+0000 7f040bfff640 1 -- 172.21.15.100:0/2329081764 learned_addr learned my addr 172.21.15.100:0/2329081764 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:26:29.043 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.038+0000 7f041bfff640 1 --2- 172.21.15.100:0/2329081764 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c816eddb0 0x562c816ef980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:29.043 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.038+0000 7f0420a44640 1 --2- 172.21.15.100:0/2329081764 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c816f3910 0x562c816ff2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:29.043 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.038+0000 7f040bfff640 1 -- 172.21.15.100:0/2329081764 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c816f3910 msgr2=0x562c816ff2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:29.043 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.038+0000 7f040bfff640 1 --2- 172.21.15.100:0/2329081764 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c816f3910 0x562c816ff2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:29.043 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.038+0000 7f040bfff640 1 -- 172.21.15.100:0/2329081764 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c816eddb0 msgr2=0x562c816ef980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:29.043 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.038+0000 7f040bfff640 1 --2- 172.21.15.100:0/2329081764 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c816eddb0 0x562c816ef980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:29.043 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.038+0000 7f040bfff640 1 -- 172.21.15.100:0/2329081764 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c816f71a0 con 0x562c816ee7a0 2024-05-10T11:26:29.043 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.038+0000 7f041bfff640 1 --2- 172.21.15.100:0/2329081764 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c816eddb0 0x562c816ef980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:26:29.043 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.038+0000 7f040bfff640 1 --2- 172.21.15.100:0/2329081764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c816ee7a0 0x562c816efec0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f04100a6210 tx=0x7f04100a9a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=823e455a037f6b28 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:26:29.043 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.038+0000 7f0409ffb640 1 -- 172.21.15.100:0/2329081764 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0410010720 con 0x562c816ee7a0 2024-05-10T11:26:29.043 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.038+0000 7f0409ffb640 1 -- 172.21.15.100:0/2329081764 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f04100b2360 con 0x562c816ee7a0 2024-05-10T11:26:29.043 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.038+0000 7f0409ffb640 1 -- 172.21.15.100:0/2329081764 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04100a5400 con 0x562c816ee7a0 2024-05-10T11:26:29.043 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.039+0000 7f04251c7740 1 -- 172.21.15.100:0/2329081764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c816ee7a0 msgr2=0x562c816efec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:29.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.039+0000 7f04251c7740 1 --2- 172.21.15.100:0/2329081764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c816ee7a0 0x562c816efec0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f04100a6210 tx=0x7f04100a9a40 comp rx=0 tx=0).stop 2024-05-10T11:26:29.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.039+0000 7f04251c7740 1 -- 172.21.15.100:0/2329081764 shutdown_connections 2024-05-10T11:26:29.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.039+0000 7f04251c7740 1 --2- 172.21.15.100:0/2329081764 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c816f3910 0x562c816ff2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:29.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.039+0000 7f04251c7740 1 --2- 172.21.15.100:0/2329081764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c816ee7a0 0x562c816efec0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:29.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.039+0000 7f04251c7740 1 --2- 172.21.15.100:0/2329081764 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c816eddb0 0x562c816ef980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:29.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.039+0000 7f04251c7740 1 -- 172.21.15.100:0/2329081764 >> 172.21.15.100:0/2329081764 conn(0x562c816e9420 msgr2=0x562c816e9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:26:29.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.039+0000 7f04251c7740 1 -- 172.21.15.100:0/2329081764 shutdown_connections 2024-05-10T11:26:29.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.039+0000 7f04251c7740 1 -- 172.21.15.100:0/2329081764 wait complete. 2024-05-10T11:26:29.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.039+0000 7f04251c7740 1 -- start start 2024-05-10T11:26:29.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.040+0000 7f04251c7740 1 Processor -- start 2024-05-10T11:26:29.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.040+0000 7f04251c7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c816eddb0 0x562c816fc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:29.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.040+0000 7f04251c7740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c816ee7a0 0x562c816fcd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:29.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.040+0000 7f04251c7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c816f3910 0x562c816fdd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:29.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.040+0000 7f04251c7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f041c1a99b0 con 0x562c816eddb0 2024-05-10T11:26:29.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.040+0000 7f04251c7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f041c1aae20 con 0x562c816f3910 2024-05-10T11:26:29.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.040+0000 7f04251c7740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f041c1aa080 con 0x562c816ee7a0 2024-05-10T11:26:29.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.040+0000 7f04251c7740 10 client.0 _ll_register_callbacks cb 0x7f0414005a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:26:29.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.040+0000 7f041bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c816eddb0 0x562c816fc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:29.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.040+0000 7f041bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c816eddb0 0x562c816fc7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49352/0 (socket says 172.21.15.100:49352) 2024-05-10T11:26:29.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.040+0000 7f041bfff640 1 -- 172.21.15.100:0/798255127 learned_addr learned my addr 172.21.15.100:0/798255127 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:26:29.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.040+0000 7f040bfff640 1 --2- 172.21.15.100:0/798255127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c816ee7a0 0x562c816fcd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:29.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.040+0000 7f0420a44640 1 --2- 172.21.15.100:0/798255127 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c816f3910 0x562c816fdd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:29.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.040+0000 7f041bfff640 1 -- 172.21.15.100:0/798255127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c816ee7a0 msgr2=0x562c816fcd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:29.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.040+0000 7f041bfff640 1 --2- 172.21.15.100:0/798255127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c816ee7a0 0x562c816fcd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:29.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.040+0000 7f041bfff640 1 -- 172.21.15.100:0/798255127 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c816f3910 msgr2=0x562c816fdd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:29.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.040+0000 7f041bfff640 1 --2- 172.21.15.100:0/798255127 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c816f3910 0x562c816fdd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:29.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.040+0000 7f041bfff640 1 -- 172.21.15.100:0/798255127 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c816fe300 con 0x562c816eddb0 2024-05-10T11:26:29.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.040+0000 7f040bfff640 1 --2- 172.21.15.100:0/798255127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c816ee7a0 0x562c816fcd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:26:29.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.041+0000 7f041bfff640 1 --2- 172.21.15.100:0/798255127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c816eddb0 0x562c816fc7c0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f040c02ade0 tx=0x7f040c005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2b00a131c0d568f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:26:29.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.041+0000 7f03fb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:26:29.046 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.041+0000 7f03fb7fe640 1 -- 172.21.15.100:0/798255127 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c04a040 con 0x562c816eddb0 2024-05-10T11:26:29.046 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.041+0000 7f04251c7740 1 -- 172.21.15.100:0/798255127 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562c8165a050 con 0x562c816eddb0 2024-05-10T11:26:29.046 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.041+0000 7f03fb7fe640 1 -- 172.21.15.100:0/798255127 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f040c003bb0 con 0x562c816eddb0 2024-05-10T11:26:29.046 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.041+0000 7f03fb7fe640 1 -- 172.21.15.100:0/798255127 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f040c031650 con 0x562c816eddb0 2024-05-10T11:26:29.046 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.041+0000 7f04251c7740 10 client.17037 fetch_fsmap learned FSMap version 31 2024-05-10T11:26:29.046 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.041+0000 7f04251c7740 1 -- 172.21.15.100:0/798255127 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562c816ff680 con 0x562c816eddb0 2024-05-10T11:26:29.046 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.041+0000 7f03fb7fe640 1 -- 172.21.15.100:0/798255127 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x562c8165a050 con 0x562c816eddb0 2024-05-10T11:26:29.046 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.041+0000 7f03fb7fe640 1 -- 172.21.15.100:0/798255127 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f040c043070 con 0x562c816eddb0 2024-05-10T11:26:29.046 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.041+0000 7f04251c7740 10 client.17037 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:26:29.046 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.041+0000 7f04251c7740 10 client.17037 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:26:29.046 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.041+0000 7f04251c7740 20 client.17037 populate_metadata read hostname 'smithi100' 2024-05-10T11:26:29.046 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.041+0000 7f04251c7740 1 --2- 172.21.15.100:0/798255127 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562c8176e7e0 0x562c81770c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:29.046 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.041+0000 7f04251c7740 4 client.17037 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:26:29.046 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.041+0000 7f04251c7740 1 -- 172.21.15.100:0/798255127 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562c815d55f0 con 0x562c8176e7e0 2024-05-10T11:26:29.046 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.042+0000 7f040bfff640 1 --2- 172.21.15.100:0/798255127 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562c8176e7e0 0x562c81770c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:29.047 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.042+0000 7f03fb7fe640 10 client.17037 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:26:29.047 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.042+0000 7f040bfff640 1 --2- 172.21.15.100:0/798255127 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562c8176e7e0 0x562c81770c00 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e6d68bf883b4ad6 server_cookie=6fc0c4eef280f4f6 in_seq=0 out_seq=0 2024-05-10T11:26:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.064+0000 7f03fb7fe640 1 -- 172.21.15.100:0/798255127 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562c815d55f0 con 0x562c8176e7e0 2024-05-10T11:26:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.064+0000 7f03fb7fe640 10 client.17037 handle_command_reply: tid=0 2024-05-10T11:26:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.064+0000 7f04251c7740 1 client.17037 shutdown 2024-05-10T11:26:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.064+0000 7f04251c7740 1 -- 172.21.15.100:0/798255127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c816eddb0 msgr2=0x562c816fc7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.064+0000 7f04251c7740 1 --2- 172.21.15.100:0/798255127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c816eddb0 0x562c816fc7c0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f040c02ade0 tx=0x7f040c005ef0 comp rx=0 tx=0).stop 2024-05-10T11:26:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.064+0000 7f04251c7740 1 -- 172.21.15.100:0/798255127 shutdown_connections 2024-05-10T11:26:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.064+0000 7f04251c7740 1 --2- 172.21.15.100:0/798255127 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562c8176e7e0 0x562c81770c00 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.064+0000 7f04251c7740 1 --2- 172.21.15.100:0/798255127 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c816f3910 0x562c816fdd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.064+0000 7f04251c7740 1 --2- 172.21.15.100:0/798255127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c816ee7a0 0x562c816fcd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.064+0000 7f03fb7fe640 0 client.17037 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:26:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.064+0000 7f04251c7740 1 --2- 172.21.15.100:0/798255127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c816eddb0 0x562c816fc7c0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.064+0000 7f04251c7740 1 -- 172.21.15.100:0/798255127 >> 172.21.15.100:0/798255127 conn(0x562c816e9420 msgr2=0x562c816f7db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:26:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.064+0000 7f04251c7740 1 -- 172.21.15.100:0/798255127 shutdown_connections 2024-05-10T11:26:29.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.064+0000 7f04251c7740 1 -- 172.21.15.100:0/798255127 wait complete. 2024-05-10T11:26:29.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.064+0000 7f04251c7740 20 client.17037 trim_cache size 0 max 16384 2024-05-10T11:26:29.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:26:29.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:26:29.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:26:29.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:26:29.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.065+0000 7f04224a7640 1 -- 172.21.15.100:0/2553350533 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f04040b28a0 msgr2=0x7f04040b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:29.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.065+0000 7f04224a7640 1 --2- 172.21.15.100:0/2553350533 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f04040b28a0 0x7f04040b4d60 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x562c816ef000 tx=0x7f0414005f90 comp rx=0 tx=0).stop 2024-05-10T11:26:29.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.065+0000 7f04224a7640 1 -- 172.21.15.100:0/2553350533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f041c102b00 msgr2=0x7f041c19d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:29.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.065+0000 7f04224a7640 1 --2- 172.21.15.100:0/2553350533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f041c102b00 0x7f041c19d830 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f041000e9f0 tx=0x7f041000eeb0 comp rx=0 tx=0).stop 2024-05-10T11:26:29.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.065+0000 7f04224a7640 1 -- 172.21.15.100:0/2553350533 shutdown_connections 2024-05-10T11:26:29.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.065+0000 7f04224a7640 1 --2- 172.21.15.100:0/2553350533 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f04040b28a0 0x7f04040b4d60 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:29.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.065+0000 7f04224a7640 1 --2- 172.21.15.100:0/2553350533 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f041c1034c0 0x7f041c19fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:29.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.065+0000 7f04224a7640 1 --2- 172.21.15.100:0/2553350533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f041c102b00 0x7f041c19d830 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:29.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.065+0000 7f04224a7640 1 --2- 172.21.15.100:0/2553350533 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f041c101910 0x7f041c06abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:29.068 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.065+0000 7f04224a7640 1 -- 172.21.15.100:0/2553350533 >> 172.21.15.100:0/2553350533 conn(0x7f041c0fce80 msgr2=0x7f041c105ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:26:29.068 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.065+0000 7f04224a7640 1 -- 172.21.15.100:0/2553350533 shutdown_connections 2024-05-10T11:26:29.068 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:29.065+0000 7f04224a7640 1 -- 172.21.15.100:0/2553350533 wait complete. 2024-05-10T11:26:29.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:26:29 smithi178 ceph-mon[33359]: pgmap v663: 129 pgs: 129 active+clean; 85 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 1.4 MiB/s wr, 158 op/s 2024-05-10T11:26:29.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:26:29 smithi188 ceph-mon[34075]: pgmap v663: 129 pgs: 129 active+clean; 85 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 1.4 MiB/s wr, 158 op/s 2024-05-10T11:26:29.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:26:29 smithi100 ceph-mon[30296]: pgmap v663: 129 pgs: 129 active+clean; 85 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 1.4 MiB/s wr, 158 op/s 2024-05-10T11:26:29.904 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T11:26:29.975 INFO:tasks.workunit.client.0.smithi100.stdout: 1 198717 2.75 MB/sec execute 264 sec latency 73.318 ms 2024-05-10T11:26:30.165 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:26:30.535 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:26:30 smithi100 ceph-mon[30296]: scrub queued for path: / 2024-05-10T11:26:30.535 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:26:30 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:26:30.535 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:26:30 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:26:30.535 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:26:30 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:26:30.535 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:26:30 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:26:30.536 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:26:30 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:26:30.536 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:26:30 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:26:30.536 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:26:30 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:26:30.536 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:26:30 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:26:30.536 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:26:30 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:26:30.536 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:26:30 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:26:30.536 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:26:30 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:26:30.536 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:26:30 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:26:30.536 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:26:30 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:26:30.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:26:30 smithi178 ceph-mon[33359]: scrub queued for path: / 2024-05-10T11:26:30.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:26:30 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:26:30.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:26:30 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:26:30.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:26:30 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:26:30.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:26:30 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:26:30.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:26:30 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:26:30.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:26:30 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:26:30.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:26:30 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:26:30.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:26:30 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:26:30.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:26:30 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:26:30.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:26:30 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:26:30.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:26:30 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:26:30.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:26:30 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:26:30.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:26:30 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:26:30.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:26:30 smithi188 ceph-mon[34075]: scrub queued for path: / 2024-05-10T11:26:30.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:26:30 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:26:30.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:26:30 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:26:30.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:26:30 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:26:30.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:26:30 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:26:30.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:26:30 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:26:30.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:26:30 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:26:30.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:26:30 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:26:30.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:26:30 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:26:30.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:26:30 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:26:30.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:26:30 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:26:30.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:26:30 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:26:30.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:26:30 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:26:30.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:26:30 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:26:30.976 INFO:tasks.workunit.client.0.smithi100.stdout: 1 199033 2.74 MB/sec execute 265 sec latency 12.475 ms 2024-05-10T11:26:31.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:26:31 smithi178 ceph-mon[33359]: pgmap v664: 129 pgs: 129 active+clean; 85 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 1.2 MiB/s wr, 129 op/s 2024-05-10T11:26:31.765 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:26:31 smithi100 ceph-mon[30296]: pgmap v664: 129 pgs: 129 active+clean; 85 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 1.2 MiB/s wr, 129 op/s 2024-05-10T11:26:31.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:26:31 smithi188 ceph-mon[34075]: pgmap v664: 129 pgs: 129 active+clean; 85 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 1.2 MiB/s wr, 129 op/s 2024-05-10T11:26:31.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.969+0000 7f2b20cc8700 1 -- 172.21.15.100:0/4052250302 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b1c101a90 msgr2=0x7f2b1c101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:31.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.969+0000 7f2b20cc8700 1 --2- 172.21.15.100:0/4052250302 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b1c101a90 0x7f2b1c101f10 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f2b10009990 tx=0x7f2b10009ca0 comp rx=0 tx=0).stop 2024-05-10T11:26:31.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.969+0000 7f2b20cc8700 1 -- 172.21.15.100:0/4052250302 shutdown_connections 2024-05-10T11:26:31.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.969+0000 7f2b20cc8700 1 --2- 172.21.15.100:0/4052250302 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b1c102450 0x7f2b1c110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:31.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.969+0000 7f2b20cc8700 1 --2- 172.21.15.100:0/4052250302 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b1c109520 0x7f2b1c109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:31.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.969+0000 7f2b20cc8700 1 --2- 172.21.15.100:0/4052250302 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b1c101a90 0x7f2b1c101f10 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:31.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.969+0000 7f2b20cc8700 1 -- 172.21.15.100:0/4052250302 >> 172.21.15.100:0/4052250302 conn(0x7f2b1c078a10 msgr2=0x7f2b1c0ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:26:31.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.969+0000 7f2b20cc8700 1 -- 172.21.15.100:0/4052250302 shutdown_connections 2024-05-10T11:26:31.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.969+0000 7f2b20cc8700 1 -- 172.21.15.100:0/4052250302 wait complete. 2024-05-10T11:26:31.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.970+0000 7f2b20cc8700 1 Processor -- start 2024-05-10T11:26:31.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.970+0000 7f2b20cc8700 1 -- start start 2024-05-10T11:26:31.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.970+0000 7f2b20cc8700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b1c101a90 0x7f2b1c19f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:31.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.970+0000 7f2b20cc8700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b1c102450 0x7f2b1c19fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:31.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.970+0000 7f2b20cc8700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b1c109520 0x7f2b1c199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:31.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.970+0000 7f2b20cc8700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1c113500 con 0x7f2b1c102450 2024-05-10T11:26:31.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.970+0000 7f2b20cc8700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1c113380 con 0x7f2b1c101a90 2024-05-10T11:26:31.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.970+0000 7f2b20cc8700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1c113680 con 0x7f2b1c109520 2024-05-10T11:26:31.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.971+0000 7f2b19d9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b1c102450 0x7f2b1c19fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:31.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.971+0000 7f2b19d9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b1c102450 0x7f2b1c19fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49372/0 (socket says 172.21.15.100:49372) 2024-05-10T11:26:31.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.971+0000 7f2b19d9b700 1 -- 172.21.15.100:0/4226130887 learned_addr learned my addr 172.21.15.100:0/4226130887 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:26:31.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.971+0000 7f2b1ad9d700 1 --2- 172.21.15.100:0/4226130887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b1c109520 0x7f2b1c199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:31.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.971+0000 7f2b1a59c700 1 --2- 172.21.15.100:0/4226130887 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b1c101a90 0x7f2b1c19f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:31.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.971+0000 7f2b19d9b700 1 -- 172.21.15.100:0/4226130887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b1c109520 msgr2=0x7f2b1c199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:31.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.971+0000 7f2b19d9b700 1 --2- 172.21.15.100:0/4226130887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b1c109520 0x7f2b1c199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:31.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.971+0000 7f2b19d9b700 1 -- 172.21.15.100:0/4226130887 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b1c101a90 msgr2=0x7f2b1c19f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:31.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.971+0000 7f2b19d9b700 1 --2- 172.21.15.100:0/4226130887 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b1c101a90 0x7f2b1c19f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:31.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.971+0000 7f2b19d9b700 1 -- 172.21.15.100:0/4226130887 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b1c19a060 con 0x7f2b1c102450 2024-05-10T11:26:31.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.971+0000 7f2b1ad9d700 1 --2- 172.21.15.100:0/4226130887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b1c109520 0x7f2b1c199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:26:31.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.971+0000 7f2b19d9b700 1 --2- 172.21.15.100:0/4226130887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b1c102450 0x7f2b1c19fc80 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f2b10006010 tx=0x7f2b100047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:26:31.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.971+0000 7f2b0b7fe700 1 -- 172.21.15.100:0/4226130887 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b10027070 con 0x7f2b1c102450 2024-05-10T11:26:31.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.971+0000 7f2b20cc8700 1 -- 172.21.15.100:0/4226130887 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b1c19a2f0 con 0x7f2b1c102450 2024-05-10T11:26:31.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.971+0000 7f2b0b7fe700 1 -- 172.21.15.100:0/4226130887 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2b10022470 con 0x7f2b1c102450 2024-05-10T11:26:31.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.971+0000 7f2b0b7fe700 1 -- 172.21.15.100:0/4226130887 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b10017650 con 0x7f2b1c102450 2024-05-10T11:26:31.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.971+0000 7f2b20cc8700 1 -- 172.21.15.100:0/4226130887 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b1c1a6610 con 0x7f2b1c102450 2024-05-10T11:26:31.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.974+0000 7f2b0b7fe700 1 -- 172.21.15.100:0/4226130887 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f2b10009d10 con 0x7f2b1c102450 2024-05-10T11:26:31.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.975+0000 7f2b0b7fe700 1 --2- 172.21.15.100:0/4226130887 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2b040b2880 0x7f2b040b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:31.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.975+0000 7f2b1a59c700 1 --2- 172.21.15.100:0/4226130887 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2b040b2880 0x7f2b040b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:31.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.976+0000 7f2b0b7fe700 1 -- 172.21.15.100:0/4226130887 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f2b100e2fe0 con 0x7f2b1c102450 2024-05-10T11:26:31.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.976+0000 7f2b2b55bb80 1 Processor -- start 2024-05-10T11:26:31.976 INFO:tasks.workunit.client.0.smithi100.stdout: 1 199345 2.74 MB/sec execute 266 sec latency 34.746 ms 2024-05-10T11:26:31.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.976+0000 7f2b2b55bb80 1 -- start start 2024-05-10T11:26:31.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.976+0000 7f2b1a59c700 1 --2- 172.21.15.100:0/4226130887 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2b040b2880 0x7f2b040b4d40 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c19aa40 tx=0x7f2b0c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:26:31.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.976+0000 7f2b2b55bb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559192f4c8e0 0x559192f4ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:31.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.976+0000 7f2b2b55bb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559192f4d200 0x559192f50b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:31.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.976+0000 7f2b2b55bb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559192f51040 0x559192f514a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:31.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.976+0000 7f2b2b55bb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x559192e9a860 con 0x559192f4c8e0 2024-05-10T11:26:31.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.976+0000 7f2b2b55bb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559192e397c0 con 0x559192f51040 2024-05-10T11:26:31.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.976+0000 7f2b2b55bb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x559192e96820 con 0x559192f4d200 2024-05-10T11:26:31.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.976+0000 7f2b1a59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559192f4c8e0 0x559192f4ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:31.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.976+0000 7f2b1a59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559192f4c8e0 0x559192f4ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49378/0 (socket says 172.21.15.100:49378) 2024-05-10T11:26:31.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.976+0000 7f2b1a59c700 1 -- 172.21.15.100:0/3013986950 learned_addr learned my addr 172.21.15.100:0/3013986950 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:26:31.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.977+0000 7f2b1ad9d700 1 --2- 172.21.15.100:0/3013986950 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559192f51040 0x559192f514a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:31.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.977+0000 7f2b19d9b700 1 --2- 172.21.15.100:0/3013986950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559192f4d200 0x559192f50b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:31.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.977+0000 7f2b1a59c700 1 -- 172.21.15.100:0/3013986950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559192f4d200 msgr2=0x559192f50b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:31.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.977+0000 7f2b1a59c700 1 --2- 172.21.15.100:0/3013986950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559192f4d200 0x559192f50b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:31.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.977+0000 7f2b1a59c700 1 -- 172.21.15.100:0/3013986950 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559192f51040 msgr2=0x559192f514a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:31.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.977+0000 7f2b1a59c700 1 --2- 172.21.15.100:0/3013986950 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559192f51040 0x559192f514a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:31.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.977+0000 7f2b1a59c700 1 -- 172.21.15.100:0/3013986950 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559192de78d0 con 0x559192f4c8e0 2024-05-10T11:26:31.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.977+0000 7f2b1a59c700 1 --2- 172.21.15.100:0/3013986950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559192f4c8e0 0x559192f4ccc0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c0046c0 tx=0x7f2b0c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32fdc7879a76bb45 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:26:31.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.977+0000 7f2b08ff9700 1 -- 172.21.15.100:0/3013986950 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c00eb20 con 0x559192f4c8e0 2024-05-10T11:26:31.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.977+0000 7f2b08ff9700 1 -- 172.21.15.100:0/3013986950 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2b0c00ecc0 con 0x559192f4c8e0 2024-05-10T11:26:31.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.977+0000 7f2b08ff9700 1 -- 172.21.15.100:0/3013986950 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c01d5e0 con 0x559192f4c8e0 2024-05-10T11:26:31.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.978+0000 7f2b2b55bb80 1 -- 172.21.15.100:0/3013986950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559192f4c8e0 msgr2=0x559192f4ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:31.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.978+0000 7f2b2b55bb80 1 --2- 172.21.15.100:0/3013986950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559192f4c8e0 0x559192f4ccc0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c0046c0 tx=0x7f2b0c014000 comp rx=0 tx=0).stop 2024-05-10T11:26:31.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.978+0000 7f2b2b55bb80 1 -- 172.21.15.100:0/3013986950 shutdown_connections 2024-05-10T11:26:31.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.978+0000 7f2b2b55bb80 1 --2- 172.21.15.100:0/3013986950 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559192f51040 0x559192f514a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:31.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.978+0000 7f2b2b55bb80 1 --2- 172.21.15.100:0/3013986950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559192f4c8e0 0x559192f4ccc0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:31.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.978+0000 7f2b2b55bb80 1 --2- 172.21.15.100:0/3013986950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559192f4d200 0x559192f50b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:31.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.978+0000 7f2b2b55bb80 1 -- 172.21.15.100:0/3013986950 >> 172.21.15.100:0/3013986950 conn(0x559192e74100 msgr2=0x559192f42c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:26:31.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.978+0000 7f2b2b55bb80 1 -- 172.21.15.100:0/3013986950 shutdown_connections 2024-05-10T11:26:31.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.978+0000 7f2b2b55bb80 1 -- 172.21.15.100:0/3013986950 wait complete. 2024-05-10T11:26:31.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.978+0000 7f2b2b55bb80 1 -- start start 2024-05-10T11:26:31.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.979+0000 7f2b2b55bb80 1 Processor -- start 2024-05-10T11:26:31.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.979+0000 7f2b2b55bb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559192f4c8e0 0x559192f58500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:31.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.979+0000 7f2b2b55bb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559192f4d200 0x559192f558a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:31.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.979+0000 7f2b2b55bb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559192f51040 0x559192f55de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:31.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.979+0000 7f2b2b55bb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x559192f59250 con 0x559192f4d200 2024-05-10T11:26:31.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.979+0000 7f2b2b55bb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559192e00c30 con 0x559192f51040 2024-05-10T11:26:31.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.979+0000 7f2b2b55bb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x559192f593d0 con 0x559192f4c8e0 2024-05-10T11:26:31.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.979+0000 7f2b2b55bb80 10 client.0 _ll_register_callbacks cb 0x559192e32920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:26:31.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.979+0000 7f2b19d9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559192f4d200 0x559192f558a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:31.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.979+0000 7f2b1a59c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559192f4c8e0 0x559192f58500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:31.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.979+0000 7f2b19d9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559192f4d200 0x559192f558a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49384/0 (socket says 172.21.15.100:49384) 2024-05-10T11:26:31.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.979+0000 7f2b19d9b700 1 -- 172.21.15.100:0/289302465 learned_addr learned my addr 172.21.15.100:0/289302465 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:26:31.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.979+0000 7f2b1a59c700 1 --2- 172.21.15.100:0/289302465 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559192f4c8e0 0x559192f58500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:50136/0 (socket says 172.21.15.100:50136) 2024-05-10T11:26:31.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.979+0000 7f2b1ad9d700 1 --2- 172.21.15.100:0/289302465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559192f51040 0x559192f55de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:31.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.979+0000 7f2b1a59c700 1 -- 172.21.15.100:0/289302465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559192f51040 msgr2=0x559192f55de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:31.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.979+0000 7f2b1a59c700 1 --2- 172.21.15.100:0/289302465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559192f51040 0x559192f55de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:31.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.979+0000 7f2b1a59c700 1 -- 172.21.15.100:0/289302465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559192f4d200 msgr2=0x559192f558a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:31.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.979+0000 7f2b1a59c700 1 --2- 172.21.15.100:0/289302465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559192f4d200 0x559192f558a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:31.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.979+0000 7f2b1a59c700 1 -- 172.21.15.100:0/289302465 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559192e36690 con 0x559192f4c8e0 2024-05-10T11:26:31.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.979+0000 7f2b19d9b700 1 --2- 172.21.15.100:0/289302465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559192f4d200 0x559192f558a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:26:31.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.980+0000 7f2b1a59c700 1 --2- 172.21.15.100:0/289302465 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559192f4c8e0 0x559192f58500 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c0117d0 tx=0x7f2b0c01e980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=988b5e52698fe35d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:26:31.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.980+0000 7f2b01ffb700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:26:31.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.980+0000 7f2b01ffb700 1 -- 172.21.15.100:0/289302465 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c00eb20 con 0x559192f4c8e0 2024-05-10T11:26:31.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.980+0000 7f2b2b55bb80 1 -- 172.21.15.100:0/289302465 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b0c00ecc0 con 0x559192f4c8e0 2024-05-10T11:26:31.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.980+0000 7f2b01ffb700 1 -- 172.21.15.100:0/289302465 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f2b0c005460 con 0x559192f4c8e0 2024-05-10T11:26:31.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.980+0000 7f2b01ffb700 1 -- 172.21.15.100:0/289302465 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2b0c005660 con 0x559192f4c8e0 2024-05-10T11:26:31.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.980+0000 7f2b2b55bb80 10 client.26524 fetch_fsmap learned FSMap version 31 2024-05-10T11:26:31.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.980+0000 7f2b2b55bb80 1 -- 172.21.15.100:0/289302465 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559192f63b00 con 0x559192f4c8e0 2024-05-10T11:26:31.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.980+0000 7f2b01ffb700 1 -- 172.21.15.100:0/289302465 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2b0c00ecc0 con 0x559192f4c8e0 2024-05-10T11:26:31.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.980+0000 7f2b01ffb700 1 -- 172.21.15.100:0/289302465 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f2b0c02e070 con 0x559192f4c8e0 2024-05-10T11:26:31.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.980+0000 7f2b2b55bb80 10 client.26524 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:26:31.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.980+0000 7f2b2b55bb80 10 client.26524 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:26:31.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.980+0000 7f2b2b55bb80 20 client.26524 populate_metadata read hostname 'smithi100' 2024-05-10T11:26:31.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.980+0000 7f2b2b55bb80 1 --2- 172.21.15.100:0/289302465 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x559192f65fd0 0x559192f683c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:31.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.980+0000 7f2b2b55bb80 4 client.26524 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:26:31.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.980+0000 7f2b2b55bb80 1 -- 172.21.15.100:0/289302465 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559192f14650 con 0x559192f65fd0 2024-05-10T11:26:31.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.981+0000 7f2b19d9b700 1 --2- 172.21.15.100:0/289302465 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x559192f65fd0 0x559192f683c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:31.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.981+0000 7f2b01ffb700 10 client.26524 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:26:31.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.981+0000 7f2b19d9b700 1 --2- 172.21.15.100:0/289302465 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x559192f65fd0 0x559192f683c0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a05f491d1e2b79 server_cookie=ebcca2e5e0868c7e in_seq=0 out_seq=0 2024-05-10T11:26:31.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.982+0000 7f2b01ffb700 1 -- 172.21.15.100:0/289302465 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559192f14650 con 0x559192f65fd0 2024-05-10T11:26:31.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.982+0000 7f2b01ffb700 10 client.26524 handle_command_reply: tid=0 2024-05-10T11:26:31.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.982+0000 7f2b2b55bb80 1 client.26524 shutdown 2024-05-10T11:26:31.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.982+0000 7f2b2b55bb80 1 -- 172.21.15.100:0/289302465 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559192f4c8e0 msgr2=0x559192f58500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:31.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.982+0000 7f2b2b55bb80 1 --2- 172.21.15.100:0/289302465 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559192f4c8e0 0x559192f58500 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c0117d0 tx=0x7f2b0c01e980 comp rx=0 tx=0).stop 2024-05-10T11:26:31.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.982+0000 7f2b2b55bb80 1 -- 172.21.15.100:0/289302465 shutdown_connections 2024-05-10T11:26:31.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.982+0000 7f2b2b55bb80 1 --2- 172.21.15.100:0/289302465 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x559192f65fd0 0x559192f683c0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:31.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.982+0000 7f2b2b55bb80 1 --2- 172.21.15.100:0/289302465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559192f51040 0x559192f55de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:31.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.982+0000 7f2b2b55bb80 1 --2- 172.21.15.100:0/289302465 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559192f4c8e0 0x559192f58500 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:31.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.982+0000 7f2b2b55bb80 1 --2- 172.21.15.100:0/289302465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559192f4d200 0x559192f558a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:31.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.982+0000 7f2b01ffb700 0 client.26524 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:26:31.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.982+0000 7f2b2b55bb80 1 -- 172.21.15.100:0/289302465 >> 172.21.15.100:0/289302465 conn(0x559192e74100 msgr2=0x559192f42c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:26:31.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.982+0000 7f2b2b55bb80 1 -- 172.21.15.100:0/289302465 shutdown_connections 2024-05-10T11:26:31.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.982+0000 7f2b2b55bb80 1 -- 172.21.15.100:0/289302465 wait complete. 2024-05-10T11:26:31.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:31.982+0000 7f2b2b55bb80 20 client.26524 trim_cache size 0 max 16384 2024-05-10T11:26:32.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.004+0000 7f2b2b55bb80 1 Processor -- start 2024-05-10T11:26:32.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.004+0000 7f2b2b55bb80 1 -- start start 2024-05-10T11:26:32.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.004+0000 7f2b2b55bb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559192e74100 0x559192f4be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:32.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.004+0000 7f2b2b55bb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559192f4c8e0 0x559192f4fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:32.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.004+0000 7f2b2b55bb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559192f4d200 0x559192f50420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:32.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.004+0000 7f2b2b55bb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1c1af5a0 con 0x559192f4d200 2024-05-10T11:26:32.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.004+0000 7f2b2b55bb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1c1ace30 con 0x559192f4c8e0 2024-05-10T11:26:32.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.004+0000 7f2b2b55bb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1c1aac80 con 0x559192e74100 2024-05-10T11:26:32.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.004+0000 7f2b1ad9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559192f4d200 0x559192f50420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:32.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.004+0000 7f2b1ad9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559192f4d200 0x559192f50420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49394/0 (socket says 172.21.15.100:49394) 2024-05-10T11:26:32.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.004+0000 7f2b1ad9d700 1 -- 172.21.15.100:0/3794456896 learned_addr learned my addr 172.21.15.100:0/3794456896 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:26:32.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.004+0000 7f2b19d9b700 1 --2- 172.21.15.100:0/3794456896 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559192f4c8e0 0x559192f4fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:32.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.004+0000 7f2b1a59c700 1 --2- 172.21.15.100:0/3794456896 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559192e74100 0x559192f4be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:32.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.005+0000 7f2b1ad9d700 1 -- 172.21.15.100:0/3794456896 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559192e74100 msgr2=0x559192f4be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:32.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.005+0000 7f2b1ad9d700 1 --2- 172.21.15.100:0/3794456896 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559192e74100 0x559192f4be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:32.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.005+0000 7f2b1ad9d700 1 -- 172.21.15.100:0/3794456896 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559192f4c8e0 msgr2=0x559192f4fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:32.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.005+0000 7f2b1ad9d700 1 --2- 172.21.15.100:0/3794456896 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559192f4c8e0 0x559192f4fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:32.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.005+0000 7f2b1ad9d700 1 -- 172.21.15.100:0/3794456896 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559192f3f810 con 0x559192f4d200 2024-05-10T11:26:32.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.005+0000 7f2b1ad9d700 1 --2- 172.21.15.100:0/3794456896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559192f4d200 0x559192f50420 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f2b14007630 tx=0x7f2b14009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b73372a648b1cc8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:26:32.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.005+0000 7f2b03fff700 1 -- 172.21.15.100:0/3794456896 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1400e900 con 0x559192f4d200 2024-05-10T11:26:32.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.005+0000 7f2b03fff700 1 -- 172.21.15.100:0/3794456896 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2b1401b030 con 0x559192f4d200 2024-05-10T11:26:32.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.005+0000 7f2b03fff700 1 -- 172.21.15.100:0/3794456896 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b14017d60 con 0x559192f4d200 2024-05-10T11:26:32.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.006+0000 7f2b2b55bb80 1 -- 172.21.15.100:0/3794456896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559192f4d200 msgr2=0x559192f50420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:32.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.006+0000 7f2b2b55bb80 1 --2- 172.21.15.100:0/3794456896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559192f4d200 0x559192f50420 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f2b14007630 tx=0x7f2b14009920 comp rx=0 tx=0).stop 2024-05-10T11:26:32.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.006+0000 7f2b2b55bb80 1 -- 172.21.15.100:0/3794456896 shutdown_connections 2024-05-10T11:26:32.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.006+0000 7f2b2b55bb80 1 --2- 172.21.15.100:0/3794456896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559192f4d200 0x559192f50420 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:32.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.006+0000 7f2b2b55bb80 1 --2- 172.21.15.100:0/3794456896 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559192e74100 0x559192f4be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:32.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.006+0000 7f2b2b55bb80 1 --2- 172.21.15.100:0/3794456896 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559192f4c8e0 0x559192f4fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:32.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.006+0000 7f2b2b55bb80 1 -- 172.21.15.100:0/3794456896 >> 172.21.15.100:0/3794456896 conn(0x559192f3dab0 msgr2=0x559192f4b6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:26:32.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.006+0000 7f2b2b55bb80 1 -- 172.21.15.100:0/3794456896 shutdown_connections 2024-05-10T11:26:32.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.006+0000 7f2b2b55bb80 1 -- 172.21.15.100:0/3794456896 wait complete. 2024-05-10T11:26:32.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.006+0000 7f2b2b55bb80 1 -- start start 2024-05-10T11:26:32.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.006+0000 7f2b2b55bb80 1 Processor -- start 2024-05-10T11:26:32.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.007+0000 7f2b2b55bb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559192e74100 0x559192fc4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:32.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.007+0000 7f2b2b55bb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559192f4c8e0 0x559192fc49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:32.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.007+0000 7f2b2b55bb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559192f4d200 0x559192fc4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:32.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.007+0000 7f2b2b55bb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1c1ab2f0 con 0x559192e74100 2024-05-10T11:26:32.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.007+0000 7f2b2b55bb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1c1ac760 con 0x559192f4d200 2024-05-10T11:26:32.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.007+0000 7f2b2b55bb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1c1ab9c0 con 0x559192f4c8e0 2024-05-10T11:26:32.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.007+0000 7f2b2b55bb80 10 client.0 _ll_register_callbacks cb 0x559192e32920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:26:32.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.007+0000 7f2b1a59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559192e74100 0x559192fc4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:32.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.007+0000 7f2b1a59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559192e74100 0x559192fc4480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49402/0 (socket says 172.21.15.100:49402) 2024-05-10T11:26:32.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.007+0000 7f2b1a59c700 1 -- 172.21.15.100:0/4082779741 learned_addr learned my addr 172.21.15.100:0/4082779741 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:26:32.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.007+0000 7f2b19d9b700 1 --2- 172.21.15.100:0/4082779741 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559192f4c8e0 0x559192fc49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:32.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.007+0000 7f2b1ad9d700 1 --2- 172.21.15.100:0/4082779741 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559192f4d200 0x559192fc4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:32.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.007+0000 7f2b1a59c700 1 -- 172.21.15.100:0/4082779741 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559192f4c8e0 msgr2=0x559192fc49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:32.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.007+0000 7f2b1a59c700 1 --2- 172.21.15.100:0/4082779741 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559192f4c8e0 0x559192fc49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:32.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.007+0000 7f2b1a59c700 1 -- 172.21.15.100:0/4082779741 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559192f4d200 msgr2=0x559192fc4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:32.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.007+0000 7f2b1a59c700 1 --2- 172.21.15.100:0/4082779741 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559192f4d200 0x559192fc4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:32.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.007+0000 7f2b1a59c700 1 -- 172.21.15.100:0/4082779741 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559192f4cf10 con 0x559192e74100 2024-05-10T11:26:32.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.007+0000 7f2b1ad9d700 1 --2- 172.21.15.100:0/4082779741 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559192f4d200 0x559192fc4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:26:32.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.007+0000 7f2b1a59c700 1 --2- 172.21.15.100:0/4082779741 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559192e74100 0x559192fc4480 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c030590 tx=0x7f2b0c01ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ea75afb7fef604b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:26:32.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.007+0000 7f2b01ffb700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:26:32.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.007+0000 7f2b01ffb700 1 -- 172.21.15.100:0/4082779741 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c00eb70 con 0x559192e74100 2024-05-10T11:26:32.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.008+0000 7f2b01ffb700 1 -- 172.21.15.100:0/4082779741 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f2b0c00ed10 con 0x559192e74100 2024-05-10T11:26:32.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.008+0000 7f2b2b55bb80 1 -- 172.21.15.100:0/4082779741 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b1401b030 con 0x559192e74100 2024-05-10T11:26:32.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.008+0000 7f2b01ffb700 1 -- 172.21.15.100:0/4082779741 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2b0c005470 con 0x559192e74100 2024-05-10T11:26:32.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.008+0000 7f2b2b55bb80 10 client.17055 fetch_fsmap learned FSMap version 31 2024-05-10T11:26:32.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.008+0000 7f2b2b55bb80 1 -- 172.21.15.100:0/4082779741 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559192fd0920 con 0x559192e74100 2024-05-10T11:26:32.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.008+0000 7f2b01ffb700 1 -- 172.21.15.100:0/4082779741 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2b1401b030 con 0x559192e74100 2024-05-10T11:26:32.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.008+0000 7f2b01ffb700 1 -- 172.21.15.100:0/4082779741 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f2b0c02e040 con 0x559192e74100 2024-05-10T11:26:32.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.008+0000 7f2b2b55bb80 10 client.17055 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:26:32.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.008+0000 7f2b2b55bb80 10 client.17055 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:26:32.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.008+0000 7f2b2b55bb80 20 client.17055 populate_metadata read hostname 'smithi100' 2024-05-10T11:26:32.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.008+0000 7f2b2b55bb80 1 --2- 172.21.15.100:0/4082779741 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x559192f51040 0x559192fd3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:32.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.008+0000 7f2b2b55bb80 4 client.17055 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T11:26:32.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.008+0000 7f2b2b55bb80 1 -- 172.21.15.100:0/4082779741 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559192ea8240 con 0x559192f51040 2024-05-10T11:26:32.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.008+0000 7f2b19d9b700 1 --2- 172.21.15.100:0/4082779741 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x559192f51040 0x559192fd3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:32.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.009+0000 7f2b19d9b700 1 --2- 172.21.15.100:0/4082779741 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x559192f51040 0x559192fd3030 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63013ffe421d2738 server_cookie=9670aac8a3f85d50 in_seq=0 out_seq=0 2024-05-10T11:26:32.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.009+0000 7f2b01ffb700 10 client.17055 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:26:32.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.010+0000 7f2b01ffb700 1 -- 172.21.15.100:0/4082779741 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x559192ea8240 con 0x559192f51040 2024-05-10T11:26:32.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.010+0000 7f2b01ffb700 10 client.17055 handle_command_reply: tid=0 2024-05-10T11:26:32.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.010+0000 7f2b2b55bb80 1 client.17055 shutdown 2024-05-10T11:26:32.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.010+0000 7f2b2b55bb80 1 -- 172.21.15.100:0/4082779741 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559192e74100 msgr2=0x559192fc4480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:32.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.010+0000 7f2b2b55bb80 1 --2- 172.21.15.100:0/4082779741 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559192e74100 0x559192fc4480 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c030590 tx=0x7f2b0c01ed70 comp rx=0 tx=0).stop 2024-05-10T11:26:32.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.010+0000 7f2b2b55bb80 1 -- 172.21.15.100:0/4082779741 shutdown_connections 2024-05-10T11:26:32.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.010+0000 7f2b2b55bb80 1 --2- 172.21.15.100:0/4082779741 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x559192f51040 0x559192fd3030 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:32.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.010+0000 7f2b2b55bb80 1 --2- 172.21.15.100:0/4082779741 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559192f4d200 0x559192fc4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:32.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.010+0000 7f2b2b55bb80 1 --2- 172.21.15.100:0/4082779741 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559192e74100 0x559192fc4480 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:32.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.010+0000 7f2b2b55bb80 1 --2- 172.21.15.100:0/4082779741 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559192f4c8e0 0x559192fc49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:32.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.010+0000 7f2b2b55bb80 1 -- 172.21.15.100:0/4082779741 >> 172.21.15.100:0/4082779741 conn(0x559192f3dab0 msgr2=0x559192f4bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:26:32.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.010+0000 7f2b01ffb700 0 client.17055 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:26:32.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.010+0000 7f2b2b55bb80 1 -- 172.21.15.100:0/4082779741 shutdown_connections 2024-05-10T11:26:32.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.010+0000 7f2b2b55bb80 1 -- 172.21.15.100:0/4082779741 wait complete. 2024-05-10T11:26:32.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.010+0000 7f2b2b55bb80 20 client.17055 trim_cache size 0 max 16384 2024-05-10T11:26:32.015 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:26:32.015 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T11:26:32.015 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-05-10T11:26:32.015 INFO:teuthology.orchestra.run.smithi100.stdout: "a000f920-7e29-4990-ae30-67c162903c02": { 2024-05-10T11:26:32.015 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-05-10T11:26:32.015 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "a000f920-7e29-4990-ae30-67c162903c02", 2024-05-10T11:26:32.015 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-05-10T11:26:32.015 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:26:32.015 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:26:32.015 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:26:32.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.011+0000 7f2b20cc8700 1 -- 172.21.15.100:0/4226130887 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2b040b2880 msgr2=0x7f2b040b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:32.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.011+0000 7f2b20cc8700 1 --2- 172.21.15.100:0/4226130887 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2b040b2880 0x7f2b040b4d40 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c19aa40 tx=0x7f2b0c009260 comp rx=0 tx=0).stop 2024-05-10T11:26:32.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.011+0000 7f2b20cc8700 1 -- 172.21.15.100:0/4226130887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b1c102450 msgr2=0x7f2b1c19fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:32.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.011+0000 7f2b20cc8700 1 --2- 172.21.15.100:0/4226130887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b1c102450 0x7f2b1c19fc80 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f2b10006010 tx=0x7f2b100047b0 comp rx=0 tx=0).stop 2024-05-10T11:26:32.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.011+0000 7f2b20cc8700 1 -- 172.21.15.100:0/4226130887 shutdown_connections 2024-05-10T11:26:32.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.011+0000 7f2b20cc8700 1 --2- 172.21.15.100:0/4226130887 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2b040b2880 0x7f2b040b4d40 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:32.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.011+0000 7f2b20cc8700 1 --2- 172.21.15.100:0/4226130887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b1c109520 0x7f2b1c199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:32.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.011+0000 7f2b20cc8700 1 --2- 172.21.15.100:0/4226130887 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b1c101a90 0x7f2b1c19f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:32.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.011+0000 7f2b20cc8700 1 --2- 172.21.15.100:0/4226130887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b1c102450 0x7f2b1c19fc80 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:32.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.011+0000 7f2b20cc8700 1 -- 172.21.15.100:0/4226130887 >> 172.21.15.100:0/4226130887 conn(0x7f2b1c078a10 msgr2=0x7f2b1c1002b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:26:32.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.011+0000 7f2b20cc8700 1 -- 172.21.15.100:0/4226130887 shutdown_connections 2024-05-10T11:26:32.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:26:32.011+0000 7f2b20cc8700 1 -- 172.21.15.100:0/4226130887 wait complete. 2024-05-10T11:26:32.849 INFO:tasks.cephfs.filesystem:scrub status for tag:a000f920-7e29-4990-ae30-67c162903c02 - {'path': '/', 'tag': 'a000f920-7e29-4990-ae30-67c162903c02', 'options': 'recursive,force'} 2024-05-10T11:26:32.976 INFO:tasks.workunit.client.0.smithi100.stdout: 1 199846 2.73 MB/sec execute 267 sec latency 15.481 ms 2024-05-10T11:26:33.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:26:33 smithi178 ceph-mon[33359]: pgmap v665: 129 pgs: 129 active+clean; 86 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 1.4 MiB/s wr, 156 op/s 2024-05-10T11:26:33.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:26:33 smithi188 ceph-mon[34075]: pgmap v665: 129 pgs: 129 active+clean; 86 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 1.4 MiB/s wr, 156 op/s 2024-05-10T11:26:33.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:26:33 smithi100 ceph-mon[30296]: pgmap v665: 129 pgs: 129 active+clean; 86 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 1.4 MiB/s wr, 156 op/s 2024-05-10T11:26:33.976 INFO:tasks.workunit.client.0.smithi100.stdout: 1 200486 2.73 MB/sec execute 268 sec latency 18.690 ms 2024-05-10T11:26:34.976 INFO:tasks.workunit.client.0.smithi100.stdout: 1 201047 2.73 MB/sec execute 269 sec latency 18.938 ms 2024-05-10T11:26:35.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:26:35 smithi178 ceph-mon[33359]: pgmap v666: 129 pgs: 129 active+clean; 87 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-05-10T11:26:35.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:26:35 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:26:35.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:26:35 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:26:35.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:26:35 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:26:35.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:26:35 smithi188 ceph-mon[34075]: pgmap v666: 129 pgs: 129 active+clean; 87 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-05-10T11:26:35.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:26:35 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:26:35.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:26:35 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:26:35.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:26:35 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:26:35.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:26:35 smithi100 ceph-mon[30296]: pgmap v666: 129 pgs: 129 active+clean; 87 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-05-10T11:26:35.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:26:35 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:26:35.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:26:35 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:26:35.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:26:35 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:26:35.976 INFO:tasks.workunit.client.0.smithi100.stdout: 1 201667 2.74 MB/sec execute 270 sec latency 15.367 ms 2024-05-10T11:26:36.976 INFO:tasks.workunit.client.0.smithi100.stdout: 1 202219 2.75 MB/sec execute 271 sec latency 15.070 ms 2024-05-10T11:26:37.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:26:37 smithi178 ceph-mon[33359]: pgmap v667: 129 pgs: 129 active+clean; 86 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 1.2 MiB/s wr, 159 op/s 2024-05-10T11:26:37.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:26:37 smithi188 ceph-mon[34075]: pgmap v667: 129 pgs: 129 active+clean; 86 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 1.2 MiB/s wr, 159 op/s 2024-05-10T11:26:37.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:26:37 smithi100 ceph-mon[30296]: pgmap v667: 129 pgs: 129 active+clean; 86 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 1.2 MiB/s wr, 159 op/s 2024-05-10T11:26:37.976 INFO:tasks.workunit.client.0.smithi100.stdout: 1 202571 2.74 MB/sec execute 272 sec latency 13.530 ms 2024-05-10T11:26:38.976 INFO:tasks.workunit.client.0.smithi100.stdout: 1 202880 2.74 MB/sec execute 273 sec latency 17.684 ms 2024-05-10T11:26:39.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:26:39 smithi178 ceph-mon[33359]: pgmap v668: 129 pgs: 129 active+clean; 87 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 1.3 MiB/s wr, 161 op/s 2024-05-10T11:26:39.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:26:39 smithi188 ceph-mon[34075]: pgmap v668: 129 pgs: 129 active+clean; 87 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 1.3 MiB/s wr, 161 op/s 2024-05-10T11:26:39.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:26:39 smithi100 ceph-mon[30296]: pgmap v668: 129 pgs: 129 active+clean; 87 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 1.3 MiB/s wr, 161 op/s 2024-05-10T11:26:39.976 INFO:tasks.workunit.client.0.smithi100.stdout: 1 203423 2.73 MB/sec execute 274 sec latency 15.485 ms 2024-05-10T11:26:40.976 INFO:tasks.workunit.client.0.smithi100.stdout: 1 204054 2.73 MB/sec execute 275 sec latency 18.130 ms 2024-05-10T11:26:41.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:26:41 smithi178 ceph-mon[33359]: pgmap v669: 129 pgs: 129 active+clean; 88 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 1.2 MiB/s wr, 143 op/s 2024-05-10T11:26:41.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:26:41 smithi188 ceph-mon[34075]: pgmap v669: 129 pgs: 129 active+clean; 88 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 1.2 MiB/s wr, 143 op/s 2024-05-10T11:26:41.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:26:41 smithi100 ceph-mon[30296]: pgmap v669: 129 pgs: 129 active+clean; 88 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 1.2 MiB/s wr, 143 op/s 2024-05-10T11:26:41.976 INFO:tasks.workunit.client.0.smithi100.stdout: 1 204662 2.73 MB/sec execute 276 sec latency 15.385 ms 2024-05-10T11:26:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.163+0000 7fe0f2ef1640 1 -- 172.21.15.100:0/4079331597 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0ec1034c0 msgr2=0x7fe0ec109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.163+0000 7fe0f2ef1640 1 --2- 172.21.15.100:0/4079331597 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0ec1034c0 0x7fe0ec109d50 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4009ca0 tx=0x7fe0e402a8c0 comp rx=0 tx=0).stop 2024-05-10T11:26:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.163+0000 7fe0f2ef1640 1 -- 172.21.15.100:0/4079331597 shutdown_connections 2024-05-10T11:26:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.163+0000 7fe0f2ef1640 1 --2- 172.21.15.100:0/4079331597 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0ec1034c0 0x7fe0ec109d50 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.163+0000 7fe0f2ef1640 1 --2- 172.21.15.100:0/4079331597 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0ec102b00 0x7fe0ec102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.163+0000 7fe0f2ef1640 1 --2- 172.21.15.100:0/4079331597 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0ec101910 0x7fe0ec101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.163+0000 7fe0f2ef1640 1 -- 172.21.15.100:0/4079331597 >> 172.21.15.100:0/4079331597 conn(0x7fe0ec0fce80 msgr2=0x7fe0ec0ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:26:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.163+0000 7fe0f2ef1640 1 -- 172.21.15.100:0/4079331597 shutdown_connections 2024-05-10T11:26:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.163+0000 7fe0f2ef1640 1 -- 172.21.15.100:0/4079331597 wait complete. 2024-05-10T11:26:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.164+0000 7fe0f2ef1640 1 Processor -- start 2024-05-10T11:26:42.167 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.164+0000 7fe0f2ef1640 1 -- start start 2024-05-10T11:26:42.167 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.164+0000 7fe0f2ef1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0ec101910 0x7fe0ec198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:42.167 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.164+0000 7fe0f2ef1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0ec102b00 0x7fe0ec198fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:42.167 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.164+0000 7fe0f2ef1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0ec1034c0 0x7fe0ec1a0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:42.167 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.164+0000 7fe0f2ef1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ec10d340 con 0x7fe0ec102b00 2024-05-10T11:26:42.167 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.164+0000 7fe0f2ef1640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ec10d1c0 con 0x7fe0ec101910 2024-05-10T11:26:42.167 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.164+0000 7fe0f2ef1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ec10d4c0 con 0x7fe0ec1034c0 2024-05-10T11:26:42.167 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.164+0000 7fe0ebfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0ec102b00 0x7fe0ec198fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:42.167 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.164+0000 7fe0ebfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0ec102b00 0x7fe0ec198fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41006/0 (socket says 172.21.15.100:41006) 2024-05-10T11:26:42.167 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.164+0000 7fe0ebfff640 1 -- 172.21.15.100:0/3634598362 learned_addr learned my addr 172.21.15.100:0/3634598362 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:26:42.167 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.164+0000 7fe0f148e640 1 --2- 172.21.15.100:0/3634598362 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0ec1034c0 0x7fe0ec1a0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:42.167 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.165+0000 7fe0f0c8d640 1 --2- 172.21.15.100:0/3634598362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0ec101910 0x7fe0ec198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:42.167 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.165+0000 7fe0ebfff640 1 -- 172.21.15.100:0/3634598362 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0ec1034c0 msgr2=0x7fe0ec1a0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:42.167 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.165+0000 7fe0ebfff640 1 --2- 172.21.15.100:0/3634598362 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0ec1034c0 0x7fe0ec1a0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:42.167 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.165+0000 7fe0ebfff640 1 -- 172.21.15.100:0/3634598362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0ec101910 msgr2=0x7fe0ec198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:42.168 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.165+0000 7fe0ebfff640 1 --2- 172.21.15.100:0/3634598362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0ec101910 0x7fe0ec198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:42.168 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.165+0000 7fe0ebfff640 1 -- 172.21.15.100:0/3634598362 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0ec1a06c0 con 0x7fe0ec102b00 2024-05-10T11:26:42.168 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.165+0000 7fe0f0c8d640 1 --2- 172.21.15.100:0/3634598362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0ec101910 0x7fe0ec198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:26:42.168 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.165+0000 7fe0ebfff640 1 --2- 172.21.15.100:0/3634598362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0ec102b00 0x7fe0ec198fe0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fe0d800c890 tx=0x7fe0d800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:26:42.168 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.165+0000 7fe0e9ffb640 1 -- 172.21.15.100:0/3634598362 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d8004220 con 0x7fe0ec102b00 2024-05-10T11:26:42.168 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.165+0000 7fe0e9ffb640 1 -- 172.21.15.100:0/3634598362 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe0d8007e40 con 0x7fe0ec102b00 2024-05-10T11:26:42.168 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.165+0000 7fe0f2ef1640 1 -- 172.21.15.100:0/3634598362 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0ec1a09b0 con 0x7fe0ec102b00 2024-05-10T11:26:42.168 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.165+0000 7fe0e9ffb640 1 -- 172.21.15.100:0/3634598362 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d8004a00 con 0x7fe0ec102b00 2024-05-10T11:26:42.168 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.165+0000 7fe0f2ef1640 1 -- 172.21.15.100:0/3634598362 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0ec1a0f00 con 0x7fe0ec102b00 2024-05-10T11:26:42.174 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.168+0000 7fe0f5c0c740 1 Processor -- start 2024-05-10T11:26:42.175 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.168+0000 7fe0e9ffb640 1 -- 172.21.15.100:0/3634598362 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fe0d80079b0 con 0x7fe0ec102b00 2024-05-10T11:26:42.175 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.168+0000 7fe0f5c0c740 1 -- start start 2024-05-10T11:26:42.175 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.168+0000 7fe0f5c0c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c514b301c0 0x55c514b305e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:42.175 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.168+0000 7fe0f5c0c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c514b30bb0 0x55c514b356b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:42.175 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.168+0000 7fe0f5c0c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c514b35d20 0x55c514b381b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:42.175 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.168+0000 7fe0f5c0c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55c514a905d0 con 0x55c514b30bb0 2024-05-10T11:26:42.175 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.168+0000 7fe0f5c0c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c514a49890 con 0x55c514b301c0 2024-05-10T11:26:42.175 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.168+0000 7fe0f5c0c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c514a8c590 con 0x55c514b35d20 2024-05-10T11:26:42.175 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.168+0000 7fe0f0c8d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c514b30bb0 0x55c514b356b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:42.175 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.168+0000 7fe0f0c8d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c514b30bb0 0x55c514b356b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41012/0 (socket says 172.21.15.100:41012) 2024-05-10T11:26:42.175 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.168+0000 7fe0f0c8d640 1 -- 172.21.15.100:0/223733420 learned_addr learned my addr 172.21.15.100:0/223733420 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:26:42.175 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.168+0000 7fe0f148e640 1 --2- 172.21.15.100:0/223733420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c514b301c0 0x55c514b305e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:42.175 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.168+0000 7fe0ebfff640 1 --2- 172.21.15.100:0/223733420 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c514b35d20 0x55c514b381b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:42.176 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.169+0000 7fe0f0c8d640 1 -- 172.21.15.100:0/223733420 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c514b35d20 msgr2=0x55c514b381b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:42.176 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.169+0000 7fe0f0c8d640 1 --2- 172.21.15.100:0/223733420 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c514b35d20 0x55c514b381b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:42.176 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.169+0000 7fe0f0c8d640 1 -- 172.21.15.100:0/223733420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c514b301c0 msgr2=0x55c514b305e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:42.176 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.169+0000 7fe0f0c8d640 1 --2- 172.21.15.100:0/223733420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c514b301c0 0x55c514b305e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:42.176 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.169+0000 7fe0f0c8d640 1 -- 172.21.15.100:0/223733420 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5147a2210 con 0x55c514b30bb0 2024-05-10T11:26:42.176 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.169+0000 7fe0e9ffb640 1 --2- 172.21.15.100:0/3634598362 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe0dc0b28a0 0x7fe0dc0b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:42.176 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.169+0000 7fe0f0c8d640 1 --2- 172.21.15.100:0/223733420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c514b30bb0 0x55c514b356b0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0009c20 tx=0x7fe0e0010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c87d48fa1092e7c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:26:42.176 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.169+0000 7fe0f148e640 1 --2- 172.21.15.100:0/3634598362 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe0dc0b28a0 0x7fe0dc0b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:42.176 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.169+0000 7fe0e9ffb640 1 -- 172.21.15.100:0/3634598362 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fe0d80e22d0 con 0x7fe0ec102b00 2024-05-10T11:26:42.176 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.169+0000 7fe0d6ffd640 1 -- 172.21.15.100:0/223733420 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e001b480 con 0x55c514b30bb0 2024-05-10T11:26:42.176 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.169+0000 7fe0d6ffd640 1 -- 172.21.15.100:0/223733420 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe0e001bab0 con 0x55c514b30bb0 2024-05-10T11:26:42.176 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.169+0000 7fe0d6ffd640 1 -- 172.21.15.100:0/223733420 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e0012a20 con 0x55c514b30bb0 2024-05-10T11:26:42.176 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.170+0000 7fe0f5c0c740 1 -- 172.21.15.100:0/223733420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c514b30bb0 msgr2=0x55c514b356b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:42.176 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.170+0000 7fe0f148e640 1 --2- 172.21.15.100:0/3634598362 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe0dc0b28a0 0x7fe0dc0b4d60 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4006010 tx=0x7fe0e4005ef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:26:42.176 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.170+0000 7fe0f5c0c740 1 --2- 172.21.15.100:0/223733420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c514b30bb0 0x55c514b356b0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0009c20 tx=0x7fe0e0010740 comp rx=0 tx=0).stop 2024-05-10T11:26:42.177 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.170+0000 7fe0f5c0c740 1 -- 172.21.15.100:0/223733420 shutdown_connections 2024-05-10T11:26:42.177 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.170+0000 7fe0f5c0c740 1 --2- 172.21.15.100:0/223733420 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c514b35d20 0x55c514b381b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:42.177 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.170+0000 7fe0f5c0c740 1 --2- 172.21.15.100:0/223733420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c514b30bb0 0x55c514b356b0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:42.177 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.170+0000 7fe0f5c0c740 1 --2- 172.21.15.100:0/223733420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c514b301c0 0x55c514b305e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:42.177 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.170+0000 7fe0f5c0c740 1 -- 172.21.15.100:0/223733420 >> 172.21.15.100:0/223733420 conn(0x55c514b27420 msgr2=0x55c514b27820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:26:42.177 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.170+0000 7fe0f5c0c740 1 -- 172.21.15.100:0/223733420 shutdown_connections 2024-05-10T11:26:42.177 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.170+0000 7fe0f5c0c740 1 -- 172.21.15.100:0/223733420 wait complete. 2024-05-10T11:26:42.177 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.170+0000 7fe0f5c0c740 1 -- start start 2024-05-10T11:26:42.177 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.171+0000 7fe0f5c0c740 1 Processor -- start 2024-05-10T11:26:42.177 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.171+0000 7fe0f5c0c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c514b301c0 0x55c514b3a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:42.177 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.171+0000 7fe0f5c0c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c514b30bb0 0x55c514b3a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:42.177 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.171+0000 7fe0f5c0c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c514b35d20 0x55c514b41950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:42.177 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.171+0000 7fe0f5c0c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55c514b38cd0 con 0x55c514b301c0 2024-05-10T11:26:42.177 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.171+0000 7fe0f5c0c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c514a4d370 con 0x55c514b30bb0 2024-05-10T11:26:42.178 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.171+0000 7fe0f5c0c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c514b38e50 con 0x55c514b35d20 2024-05-10T11:26:42.178 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.171+0000 7fe0f5c0c740 10 client.0 _ll_register_callbacks cb 0x55c5147a1a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:26:42.178 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.171+0000 7fe0f0c8d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c514b301c0 0x55c514b3a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:42.178 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.171+0000 7fe0f0c8d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c514b301c0 0x55c514b3a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41016/0 (socket says 172.21.15.100:41016) 2024-05-10T11:26:42.178 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.171+0000 7fe0f0c8d640 1 -- 172.21.15.100:0/2944768654 learned_addr learned my addr 172.21.15.100:0/2944768654 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:26:42.178 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.171+0000 7fe0f148e640 1 --2- 172.21.15.100:0/2944768654 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c514b35d20 0x55c514b41950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:42.178 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.171+0000 7fe0ebfff640 1 --2- 172.21.15.100:0/2944768654 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c514b30bb0 0x55c514b3a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:42.178 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.171+0000 7fe0f0c8d640 1 -- 172.21.15.100:0/2944768654 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c514b35d20 msgr2=0x55c514b41950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:42.178 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.171+0000 7fe0f0c8d640 1 --2- 172.21.15.100:0/2944768654 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c514b35d20 0x55c514b41950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:42.178 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.171+0000 7fe0f0c8d640 1 -- 172.21.15.100:0/2944768654 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c514b30bb0 msgr2=0x55c514b3a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:42.178 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.171+0000 7fe0f0c8d640 1 --2- 172.21.15.100:0/2944768654 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c514b30bb0 0x55c514b3a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:42.178 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.171+0000 7fe0f0c8d640 1 -- 172.21.15.100:0/2944768654 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c51486e620 con 0x55c514b301c0 2024-05-10T11:26:42.178 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.171+0000 7fe0ebfff640 1 --2- 172.21.15.100:0/2944768654 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c514b30bb0 0x55c514b3a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:26:42.178 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.172+0000 7fe0f0c8d640 1 --2- 172.21.15.100:0/2944768654 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c514b301c0 0x55c514b3a3d0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0009570 tx=0x7fe0e0005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f33f12c116d7fa6f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:26:42.179 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.172+0000 7fe0cbfff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:26:42.179 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.172+0000 7fe0cbfff640 1 -- 172.21.15.100:0/2944768654 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e001b890 con 0x55c514b301c0 2024-05-10T11:26:42.179 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.172+0000 7fe0cbfff640 1 -- 172.21.15.100:0/2944768654 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fe0e0012760 con 0x55c514b301c0 2024-05-10T11:26:42.179 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.172+0000 7fe0f5c0c740 1 -- 172.21.15.100:0/2944768654 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0e001bab0 con 0x55c514b301c0 2024-05-10T11:26:42.179 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.172+0000 7fe0cbfff640 1 -- 172.21.15.100:0/2944768654 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe0e0032050 con 0x55c514b301c0 2024-05-10T11:26:42.179 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.172+0000 7fe0f5c0c740 10 client.17067 fetch_fsmap learned FSMap version 31 2024-05-10T11:26:42.179 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.172+0000 7fe0f5c0c740 1 -- 172.21.15.100:0/2944768654 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c514b44930 con 0x55c514b301c0 2024-05-10T11:26:42.179 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.172+0000 7fe0cbfff640 1 -- 172.21.15.100:0/2944768654 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe0e001bab0 con 0x55c514b301c0 2024-05-10T11:26:42.179 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.172+0000 7fe0cbfff640 1 -- 172.21.15.100:0/2944768654 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fe0e000f040 con 0x55c514b301c0 2024-05-10T11:26:42.179 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.172+0000 7fe0f5c0c740 10 client.17067 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:26:42.179 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.172+0000 7fe0f5c0c740 10 client.17067 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:26:42.179 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.172+0000 7fe0f5c0c740 20 client.17067 populate_metadata read hostname 'smithi100' 2024-05-10T11:26:42.179 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.172+0000 7fe0f5c0c740 1 --2- 172.21.15.100:0/2944768654 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c514b45d30 0x55c514b48120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:42.179 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.172+0000 7fe0f5c0c740 4 client.17067 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:26:42.179 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.172+0000 7fe0f5c0c740 1 -- 172.21.15.100:0/2944768654 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c514a9e1e0 con 0x55c514b45d30 2024-05-10T11:26:42.180 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.173+0000 7fe0ebfff640 1 --2- 172.21.15.100:0/2944768654 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c514b45d30 0x55c514b48120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:42.180 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.173+0000 7fe0ebfff640 1 --2- 172.21.15.100:0/2944768654 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c514b45d30 0x55c514b48120 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2299b036a5caf9 server_cookie=1c237406469fbf4 in_seq=0 out_seq=0 2024-05-10T11:26:42.180 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.173+0000 7fe0cbfff640 10 client.17067 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:26:42.180 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.174+0000 7fe0cbfff640 1 -- 172.21.15.100:0/2944768654 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c514a9e1e0 con 0x55c514b45d30 2024-05-10T11:26:42.180 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.174+0000 7fe0cbfff640 10 client.17067 handle_command_reply: tid=0 2024-05-10T11:26:42.180 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.174+0000 7fe0f5c0c740 1 client.17067 shutdown 2024-05-10T11:26:42.180 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.174+0000 7fe0f5c0c740 1 -- 172.21.15.100:0/2944768654 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c514b301c0 msgr2=0x55c514b3a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:42.180 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.174+0000 7fe0f5c0c740 1 --2- 172.21.15.100:0/2944768654 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c514b301c0 0x55c514b3a3d0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0009570 tx=0x7fe0e0005c50 comp rx=0 tx=0).stop 2024-05-10T11:26:42.180 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.174+0000 7fe0f5c0c740 1 -- 172.21.15.100:0/2944768654 shutdown_connections 2024-05-10T11:26:42.180 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.174+0000 7fe0f5c0c740 1 --2- 172.21.15.100:0/2944768654 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c514b45d30 0x55c514b48120 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:42.180 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.174+0000 7fe0f5c0c740 1 --2- 172.21.15.100:0/2944768654 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c514b35d20 0x55c514b41950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:42.180 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.174+0000 7fe0f5c0c740 1 --2- 172.21.15.100:0/2944768654 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c514b30bb0 0x55c514b3a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:42.180 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.174+0000 7fe0cbfff640 0 client.17067 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:26:42.180 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.174+0000 7fe0f5c0c740 1 --2- 172.21.15.100:0/2944768654 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c514b301c0 0x55c514b3a3d0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:42.181 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.174+0000 7fe0f5c0c740 1 -- 172.21.15.100:0/2944768654 >> 172.21.15.100:0/2944768654 conn(0x55c514b27420 msgr2=0x55c514b26580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:26:42.181 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.174+0000 7fe0f5c0c740 1 -- 172.21.15.100:0/2944768654 shutdown_connections 2024-05-10T11:26:42.181 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.174+0000 7fe0f5c0c740 1 -- 172.21.15.100:0/2944768654 wait complete. 2024-05-10T11:26:42.181 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.174+0000 7fe0f5c0c740 20 client.17067 trim_cache size 0 max 16384 2024-05-10T11:26:42.192 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.191+0000 7fe0f5c0c740 1 Processor -- start 2024-05-10T11:26:42.192 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.191+0000 7fe0f5c0c740 1 -- start start 2024-05-10T11:26:42.192 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.191+0000 7fe0f5c0c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c514b301c0 0x55c514b2a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:42.192 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.191+0000 7fe0f5c0c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c514b30bb0 0x55c514b41870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:42.193 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.191+0000 7fe0f5c0c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c514b35d20 0x55c514b2a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:42.193 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.191+0000 7fe0f5c0c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ec1ab9f0 con 0x55c514b35d20 2024-05-10T11:26:42.193 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.191+0000 7fe0f5c0c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ec1a92c0 con 0x55c514b301c0 2024-05-10T11:26:42.193 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.191+0000 7fe0f5c0c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ec1a7110 con 0x55c514b30bb0 2024-05-10T11:26:42.193 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.191+0000 7fe0f148e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c514b35d20 0x55c514b2a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:42.193 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.191+0000 7fe0f148e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c514b35d20 0x55c514b2a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41018/0 (socket says 172.21.15.100:41018) 2024-05-10T11:26:42.193 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.191+0000 7fe0f148e640 1 -- 172.21.15.100:0/609554180 learned_addr learned my addr 172.21.15.100:0/609554180 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:26:42.193 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.191+0000 7fe0f0c8d640 1 --2- 172.21.15.100:0/609554180 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c514b301c0 0x55c514b2a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:42.193 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.191+0000 7fe0ebfff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c514b30bb0 0x55c514b41870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:42.193 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.192+0000 7fe0f148e640 1 -- 172.21.15.100:0/609554180 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c514b30bb0 msgr2=0x55c514b41870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:42.193 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.192+0000 7fe0f148e640 1 --2- 172.21.15.100:0/609554180 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c514b30bb0 0x55c514b41870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:42.193 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.192+0000 7fe0f148e640 1 -- 172.21.15.100:0/609554180 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c514b301c0 msgr2=0x55c514b2a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:42.193 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.192+0000 7fe0f148e640 1 --2- 172.21.15.100:0/609554180 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c514b301c0 0x55c514b2a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:42.193 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.192+0000 7fe0f148e640 1 -- 172.21.15.100:0/609554180 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c514b48490 con 0x55c514b35d20 2024-05-10T11:26:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.192+0000 7fe0f0c8d640 1 --2- 172.21.15.100:0/609554180 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c514b301c0 0x55c514b2a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:26:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.192+0000 7fe0f148e640 1 --2- 172.21.15.100:0/609554180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c514b35d20 0x55c514b2a950 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4009ca0 tx=0x7fe0e4044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91f86db951f80565 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:26:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.192+0000 7fe0d67fc640 1 -- 172.21.15.100:0/609554180 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e4030b10 con 0x55c514b35d20 2024-05-10T11:26:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.192+0000 7fe0d67fc640 1 -- 172.21.15.100:0/609554180 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe0e4036050 con 0x55c514b35d20 2024-05-10T11:26:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.192+0000 7fe0d67fc640 1 -- 172.21.15.100:0/609554180 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e4004480 con 0x55c514b35d20 2024-05-10T11:26:42.197 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.193+0000 7fe0f5c0c740 1 -- 172.21.15.100:0/609554180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c514b35d20 msgr2=0x55c514b2a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:42.197 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.193+0000 7fe0f5c0c740 1 --2- 172.21.15.100:0/609554180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c514b35d20 0x55c514b2a950 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4009ca0 tx=0x7fe0e4044000 comp rx=0 tx=0).stop 2024-05-10T11:26:42.197 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.193+0000 7fe0f5c0c740 1 -- 172.21.15.100:0/609554180 shutdown_connections 2024-05-10T11:26:42.198 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.193+0000 7fe0f5c0c740 1 --2- 172.21.15.100:0/609554180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c514b35d20 0x55c514b2a950 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:42.198 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.193+0000 7fe0f5c0c740 1 --2- 172.21.15.100:0/609554180 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c514b30bb0 0x55c514b41870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:42.198 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.193+0000 7fe0f5c0c740 1 --2- 172.21.15.100:0/609554180 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c514b301c0 0x55c514b2a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:42.198 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.193+0000 7fe0f5c0c740 1 -- 172.21.15.100:0/609554180 >> 172.21.15.100:0/609554180 conn(0x55c514b27420 msgr2=0x55c514b223e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:26:42.198 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.193+0000 7fe0f5c0c740 1 -- 172.21.15.100:0/609554180 shutdown_connections 2024-05-10T11:26:42.198 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.193+0000 7fe0f5c0c740 1 -- 172.21.15.100:0/609554180 wait complete. 2024-05-10T11:26:42.198 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.193+0000 7fe0f5c0c740 1 -- start start 2024-05-10T11:26:42.198 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.193+0000 7fe0f5c0c740 1 Processor -- start 2024-05-10T11:26:42.198 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.194+0000 7fe0f5c0c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c514b301c0 0x55c514b3e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:42.198 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.194+0000 7fe0f5c0c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c514b30bb0 0x55c514b3e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:42.198 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.194+0000 7fe0f5c0c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c514b35d20 0x55c514bae540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:42.198 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.194+0000 7fe0f5c0c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ec1a7780 con 0x55c514b35d20 2024-05-10T11:26:42.199 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.194+0000 7fe0f5c0c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ec1a8bf0 con 0x55c514b301c0 2024-05-10T11:26:42.199 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.194+0000 7fe0f5c0c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ec1a7e50 con 0x55c514b30bb0 2024-05-10T11:26:42.199 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.194+0000 7fe0f5c0c740 10 client.0 _ll_register_callbacks cb 0x7fe0e4009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:26:42.199 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.194+0000 7fe0f148e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c514b35d20 0x55c514bae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:42.199 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.194+0000 7fe0f148e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c514b35d20 0x55c514bae540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41022/0 (socket says 172.21.15.100:41022) 2024-05-10T11:26:42.199 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.194+0000 7fe0f148e640 1 -- 172.21.15.100:0/156549002 learned_addr learned my addr 172.21.15.100:0/156549002 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:26:42.199 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.194+0000 7fe0f0c8d640 1 --2- 172.21.15.100:0/156549002 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c514b301c0 0x55c514b3e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:42.199 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.194+0000 7fe0ebfff640 1 --2- 172.21.15.100:0/156549002 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c514b30bb0 0x55c514b3e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:42.199 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.194+0000 7fe0f148e640 1 -- 172.21.15.100:0/156549002 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c514b30bb0 msgr2=0x55c514b3e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:42.199 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.194+0000 7fe0f148e640 1 --2- 172.21.15.100:0/156549002 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c514b30bb0 0x55c514b3e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:42.199 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.194+0000 7fe0f148e640 1 -- 172.21.15.100:0/156549002 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c514b301c0 msgr2=0x55c514b3e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:42.199 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.194+0000 7fe0f148e640 1 --2- 172.21.15.100:0/156549002 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c514b301c0 0x55c514b3e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:42.199 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.194+0000 7fe0f148e640 1 -- 172.21.15.100:0/156549002 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c514b419d0 con 0x55c514b35d20 2024-05-10T11:26:42.199 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.195+0000 7fe0f148e640 1 --2- 172.21.15.100:0/156549002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c514b35d20 0x55c514bae540 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4005e30 tx=0x7fe0e4009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b81a218496cc90b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:26:42.200 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.195+0000 7fe0cbfff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:26:42.200 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.195+0000 7fe0cbfff640 1 -- 172.21.15.100:0/156549002 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e4003c40 con 0x55c514b35d20 2024-05-10T11:26:42.200 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.195+0000 7fe0cbfff640 1 -- 172.21.15.100:0/156549002 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fe0e4003de0 con 0x55c514b35d20 2024-05-10T11:26:42.200 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.195+0000 7fe0f5c0c740 1 -- 172.21.15.100:0/156549002 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0e4036050 con 0x55c514b35d20 2024-05-10T11:26:42.200 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.195+0000 7fe0cbfff640 1 -- 172.21.15.100:0/156549002 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe0e4069050 con 0x55c514b35d20 2024-05-10T11:26:42.200 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.195+0000 7fe0f5c0c740 10 client.17073 fetch_fsmap learned FSMap version 31 2024-05-10T11:26:42.200 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.195+0000 7fe0f5c0c740 1 -- 172.21.15.100:0/156549002 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c514baf9b0 con 0x55c514b35d20 2024-05-10T11:26:42.200 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.195+0000 7fe0cbfff640 1 -- 172.21.15.100:0/156549002 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe0e4036050 con 0x55c514b35d20 2024-05-10T11:26:42.200 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.195+0000 7fe0cbfff640 1 -- 172.21.15.100:0/156549002 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fe0e405b7e0 con 0x55c514b35d20 2024-05-10T11:26:42.200 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.195+0000 7fe0f5c0c740 10 client.17073 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:26:42.200 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.195+0000 7fe0f5c0c740 10 client.17073 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:26:42.200 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.195+0000 7fe0f5c0c740 20 client.17073 populate_metadata read hostname 'smithi100' 2024-05-10T11:26:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.195+0000 7fe0f5c0c740 1 --2- 172.21.15.100:0/156549002 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c514bb0430 0x55c514bb2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.195+0000 7fe0f5c0c740 4 client.17073 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:26:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.195+0000 7fe0f5c0c740 1 -- 172.21.15.100:0/156549002 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c514a98050 con 0x55c514bb0430 2024-05-10T11:26:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.196+0000 7fe0f0c8d640 1 --2- 172.21.15.100:0/156549002 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c514bb0430 0x55c514bb2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.196+0000 7fe0cbfff640 10 client.17073 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:26:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.196+0000 7fe0f0c8d640 1 --2- 172.21.15.100:0/156549002 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c514bb0430 0x55c514bb2850 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e49c5f991a22a9dc server_cookie=ab2fdb2278fabc42 in_seq=0 out_seq=0 2024-05-10T11:26:42.212 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.211+0000 7fe0cbfff640 1 -- 172.21.15.100:0/156549002 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c514a98050 con 0x55c514bb0430 2024-05-10T11:26:42.212 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.211+0000 7fe0cbfff640 10 client.17073 handle_command_reply: tid=0 2024-05-10T11:26:42.212 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.211+0000 7fe0f5c0c740 1 client.17073 shutdown 2024-05-10T11:26:42.212 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.211+0000 7fe0f5c0c740 1 -- 172.21.15.100:0/156549002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c514b35d20 msgr2=0x55c514bae540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:42.212 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.211+0000 7fe0f5c0c740 1 --2- 172.21.15.100:0/156549002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c514b35d20 0x55c514bae540 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4005e30 tx=0x7fe0e4009360 comp rx=0 tx=0).stop 2024-05-10T11:26:42.212 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.211+0000 7fe0f5c0c740 1 -- 172.21.15.100:0/156549002 shutdown_connections 2024-05-10T11:26:42.212 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.211+0000 7fe0f5c0c740 1 --2- 172.21.15.100:0/156549002 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c514bb0430 0x55c514bb2850 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:42.212 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.211+0000 7fe0f5c0c740 1 --2- 172.21.15.100:0/156549002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c514b35d20 0x55c514bae540 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:42.212 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.211+0000 7fe0cbfff640 0 client.17073 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:26:42.212 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.211+0000 7fe0f5c0c740 1 --2- 172.21.15.100:0/156549002 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c514b30bb0 0x55c514b3e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:42.212 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.211+0000 7fe0f5c0c740 1 --2- 172.21.15.100:0/156549002 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c514b301c0 0x55c514b3e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:42.212 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.211+0000 7fe0f5c0c740 1 -- 172.21.15.100:0/156549002 >> 172.21.15.100:0/156549002 conn(0x55c514b27420 msgr2=0x55c514b2a410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:26:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.212+0000 7fe0f5c0c740 1 -- 172.21.15.100:0/156549002 shutdown_connections 2024-05-10T11:26:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.212+0000 7fe0f5c0c740 1 -- 172.21.15.100:0/156549002 wait complete. 2024-05-10T11:26:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.212+0000 7fe0f5c0c740 20 client.17073 trim_cache size 0 max 16384 2024-05-10T11:26:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:26:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:26:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:26:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:26:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.212+0000 7fe0f2ef1640 1 -- 172.21.15.100:0/3634598362 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe0dc0b28a0 msgr2=0x7fe0dc0b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.212+0000 7fe0f2ef1640 1 --2- 172.21.15.100:0/3634598362 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe0dc0b28a0 0x7fe0dc0b4d60 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4006010 tx=0x7fe0e4005ef0 comp rx=0 tx=0).stop 2024-05-10T11:26:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.212+0000 7fe0f2ef1640 1 -- 172.21.15.100:0/3634598362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0ec102b00 msgr2=0x7fe0ec198fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.212+0000 7fe0f2ef1640 1 --2- 172.21.15.100:0/3634598362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0ec102b00 0x7fe0ec198fe0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fe0d800c890 tx=0x7fe0d800cd50 comp rx=0 tx=0).stop 2024-05-10T11:26:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.212+0000 7fe0f2ef1640 1 -- 172.21.15.100:0/3634598362 shutdown_connections 2024-05-10T11:26:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.212+0000 7fe0f2ef1640 1 --2- 172.21.15.100:0/3634598362 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe0dc0b28a0 0x7fe0dc0b4d60 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.212+0000 7fe0f2ef1640 1 --2- 172.21.15.100:0/3634598362 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0ec1034c0 0x7fe0ec1a0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.212+0000 7fe0f2ef1640 1 --2- 172.21.15.100:0/3634598362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0ec102b00 0x7fe0ec198fe0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.212+0000 7fe0f2ef1640 1 --2- 172.21.15.100:0/3634598362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0ec101910 0x7fe0ec198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.212+0000 7fe0f2ef1640 1 -- 172.21.15.100:0/3634598362 >> 172.21.15.100:0/3634598362 conn(0x7fe0ec0fce80 msgr2=0x7fe0ec0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:26:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.213+0000 7fe0f2ef1640 1 -- 172.21.15.100:0/3634598362 shutdown_connections 2024-05-10T11:26:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:42.213+0000 7fe0f2ef1640 1 -- 172.21.15.100:0/3634598362 wait complete. 2024-05-10T11:26:42.976 INFO:tasks.workunit.client.0.smithi100.stdout: 1 205474 2.75 MB/sec execute 277 sec latency 14.598 ms 2024-05-10T11:26:43.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:26:43 smithi178 ceph-mon[33359]: pgmap v670: 129 pgs: 129 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 1.3 MiB/s wr, 168 op/s 2024-05-10T11:26:43.881 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:26:43 smithi100 ceph-mon[30296]: pgmap v670: 129 pgs: 129 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 1.3 MiB/s wr, 168 op/s 2024-05-10T11:26:43.881 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:26:43 smithi188 ceph-mon[34075]: pgmap v670: 129 pgs: 129 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 1.3 MiB/s wr, 168 op/s 2024-05-10T11:26:43.976 INFO:tasks.workunit.client.0.smithi100.stdout: 1 205941 2.75 MB/sec execute 278 sec latency 14.313 ms 2024-05-10T11:26:44.976 INFO:tasks.workunit.client.0.smithi100.stdout: 1 206199 2.74 MB/sec execute 279 sec latency 15.109 ms 2024-05-10T11:26:45.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:26:45 smithi178 ceph-mon[33359]: pgmap v671: 129 pgs: 129 active+clean; 84 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 1.2 MiB/s wr, 151 op/s 2024-05-10T11:26:45.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:26:45 smithi188 ceph-mon[34075]: pgmap v671: 129 pgs: 129 active+clean; 84 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 1.2 MiB/s wr, 151 op/s 2024-05-10T11:26:45.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:26:45 smithi100 ceph-mon[30296]: pgmap v671: 129 pgs: 129 active+clean; 84 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 1.2 MiB/s wr, 151 op/s 2024-05-10T11:26:45.976 INFO:tasks.workunit.client.0.smithi100.stdout: 1 206640 2.74 MB/sec execute 280 sec latency 14.643 ms 2024-05-10T11:26:46.976 INFO:tasks.workunit.client.0.smithi100.stdout: 1 207228 2.74 MB/sec execute 281 sec latency 18.536 ms 2024-05-10T11:26:47.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:26:47 smithi178 ceph-mon[33359]: pgmap v672: 129 pgs: 129 active+clean; 86 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 1.4 MiB/s wr, 164 op/s 2024-05-10T11:26:47.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:26:47 smithi188 ceph-mon[34075]: pgmap v672: 129 pgs: 129 active+clean; 86 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 1.4 MiB/s wr, 164 op/s 2024-05-10T11:26:47.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:26:47 smithi100 ceph-mon[30296]: pgmap v672: 129 pgs: 129 active+clean; 86 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 1.4 MiB/s wr, 164 op/s 2024-05-10T11:26:47.976 INFO:tasks.workunit.client.0.smithi100.stdout: 1 207914 2.74 MB/sec execute 282 sec latency 17.548 ms 2024-05-10T11:26:48.976 INFO:tasks.workunit.client.0.smithi100.stdout: 1 208506 2.74 MB/sec execute 283 sec latency 15.340 ms 2024-05-10T11:26:49.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:26:49 smithi178 ceph-mon[33359]: pgmap v673: 129 pgs: 129 active+clean; 87 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 1.4 MiB/s wr, 142 op/s 2024-05-10T11:26:49.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:26:49 smithi188 ceph-mon[34075]: pgmap v673: 129 pgs: 129 active+clean; 87 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 1.4 MiB/s wr, 142 op/s 2024-05-10T11:26:49.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:26:49 smithi100 ceph-mon[30296]: pgmap v673: 129 pgs: 129 active+clean; 87 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 1.4 MiB/s wr, 142 op/s 2024-05-10T11:26:49.976 INFO:tasks.workunit.client.0.smithi100.stdout: 1 209103 2.75 MB/sec execute 284 sec latency 13.688 ms 2024-05-10T11:26:50.976 INFO:tasks.workunit.client.0.smithi100.stdout: 1 209594 2.75 MB/sec execute 285 sec latency 27.891 ms 2024-05-10T11:26:51.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:26:51 smithi178 ceph-mon[33359]: pgmap v674: 129 pgs: 129 active+clean; 86 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 1.2 MiB/s wr, 135 op/s 2024-05-10T11:26:51.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:26:51 smithi188 ceph-mon[34075]: pgmap v674: 129 pgs: 129 active+clean; 86 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 1.2 MiB/s wr, 135 op/s 2024-05-10T11:26:51.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:26:51 smithi100 ceph-mon[30296]: pgmap v674: 129 pgs: 129 active+clean; 86 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 1.2 MiB/s wr, 135 op/s 2024-05-10T11:26:51.976 INFO:tasks.workunit.client.0.smithi100.stdout: 1 209892 2.74 MB/sec execute 286 sec latency 15.048 ms 2024-05-10T11:26:52.976 INFO:tasks.workunit.client.0.smithi100.stdout: 1 210454 2.74 MB/sec execute 287 sec latency 12.476 ms 2024-05-10T11:26:53.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:26:53 smithi178 ceph-mon[33359]: pgmap v675: 129 pgs: 129 active+clean; 87 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-05-10T11:26:53.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:26:53 smithi188 ceph-mon[34075]: pgmap v675: 129 pgs: 129 active+clean; 87 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-05-10T11:26:53.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:26:53 smithi100 ceph-mon[30296]: pgmap v675: 129 pgs: 129 active+clean; 87 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-05-10T11:26:53.976 INFO:tasks.workunit.client.0.smithi100.stdout: 1 211106 2.74 MB/sec execute 288 sec latency 13.744 ms 2024-05-10T11:26:54.976 INFO:tasks.workunit.client.0.smithi100.stdout: 1 211704 2.74 MB/sec execute 289 sec latency 15.819 ms 2024-05-10T11:26:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.307+0000 7fa7bda59640 1 -- 172.21.15.100:0/2283344512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7b8103c00 msgr2=0x7fa7b8104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.307+0000 7fa7bda59640 1 --2- 172.21.15.100:0/2283344512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7b8103c00 0x7fa7b8104080 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac009900 tx=0x7fa7ac02a410 comp rx=0 tx=0).stop 2024-05-10T11:26:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.307+0000 7fa7bda59640 1 -- 172.21.15.100:0/2283344512 shutdown_connections 2024-05-10T11:26:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.307+0000 7fa7bda59640 1 --2- 172.21.15.100:0/2283344512 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7b81045c0 0x7fa7b810ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.307+0000 7fa7bda59640 1 --2- 172.21.15.100:0/2283344512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7b8103c00 0x7fa7b8104080 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.307+0000 7fa7bda59640 1 --2- 172.21.15.100:0/2283344512 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7b8102a10 0x7fa7b8102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.307+0000 7fa7bda59640 1 -- 172.21.15.100:0/2283344512 >> 172.21.15.100:0/2283344512 conn(0x7fa7b80fdfa0 msgr2=0x7fa7b81003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:26:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.307+0000 7fa7bda59640 1 -- 172.21.15.100:0/2283344512 shutdown_connections 2024-05-10T11:26:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.307+0000 7fa7bda59640 1 -- 172.21.15.100:0/2283344512 wait complete. 2024-05-10T11:26:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.308+0000 7fa7bda59640 1 Processor -- start 2024-05-10T11:26:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.308+0000 7fa7bda59640 1 -- start start 2024-05-10T11:26:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.308+0000 7fa7bda59640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7b8102a10 0x7fa7b8198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.308+0000 7fa7bda59640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7b8103c00 0x7fa7b8198fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.308+0000 7fa7bda59640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7b81045c0 0x7fa7b81a0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.308+0000 7fa7bda59640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b810e3f0 con 0x7fa7b81045c0 2024-05-10T11:26:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.308+0000 7fa7bda59640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b810e270 con 0x7fa7b8103c00 2024-05-10T11:26:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.308+0000 7fa7bda59640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b810e570 con 0x7fa7b8102a10 2024-05-10T11:26:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.308+0000 7fa7b77fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7b81045c0 0x7fa7b81a0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.308+0000 7fa7b6ffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7b8102a10 0x7fa7b8198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.308+0000 7fa7b67fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7b8103c00 0x7fa7b8198fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.308+0000 7fa7b77fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7b81045c0 0x7fa7b81a0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39070/0 (socket says 172.21.15.100:39070) 2024-05-10T11:26:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.308+0000 7fa7b77fe640 1 -- 172.21.15.100:0/2348373027 learned_addr learned my addr 172.21.15.100:0/2348373027 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:26:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.308+0000 7fa7b77fe640 1 -- 172.21.15.100:0/2348373027 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7b8102a10 msgr2=0x7fa7b8198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.308+0000 7fa7b77fe640 1 --2- 172.21.15.100:0/2348373027 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7b8102a10 0x7fa7b8198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.308+0000 7fa7b77fe640 1 -- 172.21.15.100:0/2348373027 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7b8103c00 msgr2=0x7fa7b8198fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.308+0000 7fa7b77fe640 1 --2- 172.21.15.100:0/2348373027 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7b8103c00 0x7fa7b8198fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.308+0000 7fa7b77fe640 1 -- 172.21.15.100:0/2348373027 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7b81a0680 con 0x7fa7b81045c0 2024-05-10T11:26:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.308+0000 7fa7b6ffd640 1 --2- 172.21.15.100:0/2348373027 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7b8102a10 0x7fa7b8198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:26:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.308+0000 7fa7b67fc640 1 --2- 172.21.15.100:0/2348373027 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7b8103c00 0x7fa7b8198fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:26:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.309+0000 7fa7b77fe640 1 --2- 172.21.15.100:0/2348373027 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7b81045c0 0x7fa7b81a0010 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fa7b000eb10 tx=0x7fa7b000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:26:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.309+0000 7fa7bca57640 1 -- 172.21.15.100:0/2348373027 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b0004b20 con 0x7fa7b81045c0 2024-05-10T11:26:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.309+0000 7fa7bca57640 1 -- 172.21.15.100:0/2348373027 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa7b0004cc0 con 0x7fa7b81045c0 2024-05-10T11:26:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.309+0000 7fa7bda59640 1 -- 172.21.15.100:0/2348373027 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7b81a0970 con 0x7fa7b81045c0 2024-05-10T11:26:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.309+0000 7fa7bca57640 1 -- 172.21.15.100:0/2348373027 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b0010690 con 0x7fa7b81045c0 2024-05-10T11:26:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.309+0000 7fa7bda59640 1 -- 172.21.15.100:0/2348373027 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7b81a0e90 con 0x7fa7b81045c0 2024-05-10T11:26:55.319 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.312+0000 7fa7bca57640 1 -- 172.21.15.100:0/2348373027 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fa7b0005020 con 0x7fa7b81045c0 2024-05-10T11:26:55.319 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.312+0000 7fa7c0772740 1 Processor -- start 2024-05-10T11:26:55.320 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.312+0000 7fa7c0772740 1 -- start start 2024-05-10T11:26:55.320 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.312+0000 7fa7c0772740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd615991c0 0x55dd615995e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:55.320 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.312+0000 7fa7c0772740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd61599bb0 0x55dd6159e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:55.320 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.312+0000 7fa7c0772740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd6159ed20 0x55dd615a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:55.320 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.312+0000 7fa7c0772740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55dd614f95d0 con 0x55dd6159ed20 2024-05-10T11:26:55.320 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.312+0000 7fa7c0772740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55dd614b2890 con 0x55dd615991c0 2024-05-10T11:26:55.320 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.312+0000 7fa7c0772740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55dd614f5590 con 0x55dd61599bb0 2024-05-10T11:26:55.320 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.312+0000 7fa7b67fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd6159ed20 0x55dd615a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:55.320 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.312+0000 7fa7b67fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd6159ed20 0x55dd615a11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39084/0 (socket says 172.21.15.100:39084) 2024-05-10T11:26:55.320 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.312+0000 7fa7b67fc640 1 -- 172.21.15.100:0/2021108197 learned_addr learned my addr 172.21.15.100:0/2021108197 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:26:55.320 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.312+0000 7fa7b6ffd640 1 --2- 172.21.15.100:0/2021108197 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd61599bb0 0x55dd6159e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:55.320 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.312+0000 7fa7b67fc640 1 --2- 172.21.15.100:0/2021108197 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd615991c0 0x55dd615995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:55.320 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.312+0000 7fa7b67fc640 1 -- 172.21.15.100:0/2021108197 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd61599bb0 msgr2=0x55dd6159e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:55.320 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.312+0000 7fa7b67fc640 1 --2- 172.21.15.100:0/2021108197 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd61599bb0 0x55dd6159e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:55.321 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.312+0000 7fa7b67fc640 1 -- 172.21.15.100:0/2021108197 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd615991c0 msgr2=0x55dd615995e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:55.321 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.312+0000 7fa7b67fc640 1 --2- 172.21.15.100:0/2021108197 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd615991c0 0x55dd615995e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:55.321 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.312+0000 7fa7b67fc640 1 -- 172.21.15.100:0/2021108197 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd6120b210 con 0x55dd6159ed20 2024-05-10T11:26:55.321 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.313+0000 7fa7bca57640 1 --2- 172.21.15.100:0/2348373027 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa7a00b28a0 0x7fa7a00b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:55.321 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.313+0000 7fa7b67fc640 1 --2- 172.21.15.100:0/2021108197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd6159ed20 0x55dd615a11b0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x55dd6159a410 tx=0x7fa7ac005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab5a2288a7be5d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:26:55.321 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.313+0000 7fa7b6ffd640 1 --2- 172.21.15.100:0/2348373027 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa7a00b28a0 0x7fa7a00b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:55.321 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.313+0000 7fa79dffb640 1 -- 172.21.15.100:0/2021108197 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac035040 con 0x55dd6159ed20 2024-05-10T11:26:55.321 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.313+0000 7fa79dffb640 1 -- 172.21.15.100:0/2021108197 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa7ac0044c0 con 0x55dd6159ed20 2024-05-10T11:26:55.321 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.313+0000 7fa7bca57640 1 -- 172.21.15.100:0/2348373027 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fa7b00e2ad0 con 0x7fa7b81045c0 2024-05-10T11:26:55.321 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.313+0000 7fa79dffb640 1 -- 172.21.15.100:0/2021108197 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac003d70 con 0x55dd6159ed20 2024-05-10T11:26:55.321 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.314+0000 7fa7c0772740 1 -- 172.21.15.100:0/2021108197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd6159ed20 msgr2=0x55dd615a11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:55.321 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.314+0000 7fa7b6ffd640 1 --2- 172.21.15.100:0/2348373027 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa7a00b28a0 0x7fa7a00b4d60 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x55dd61599fb0 tx=0x7fa7a8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:26:55.321 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.314+0000 7fa7c0772740 1 --2- 172.21.15.100:0/2021108197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd6159ed20 0x55dd615a11b0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x55dd6159a410 tx=0x7fa7ac005b50 comp rx=0 tx=0).stop 2024-05-10T11:26:55.321 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.314+0000 7fa7c0772740 1 -- 172.21.15.100:0/2021108197 shutdown_connections 2024-05-10T11:26:55.321 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.314+0000 7fa7c0772740 1 --2- 172.21.15.100:0/2021108197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd6159ed20 0x55dd615a11b0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:55.321 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.314+0000 7fa7c0772740 1 --2- 172.21.15.100:0/2021108197 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd61599bb0 0x55dd6159e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:55.322 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.314+0000 7fa7c0772740 1 --2- 172.21.15.100:0/2021108197 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd615991c0 0x55dd615995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:55.322 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.314+0000 7fa7c0772740 1 -- 172.21.15.100:0/2021108197 >> 172.21.15.100:0/2021108197 conn(0x55dd61590420 msgr2=0x55dd61590820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:26:55.322 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.314+0000 7fa7c0772740 1 -- 172.21.15.100:0/2021108197 shutdown_connections 2024-05-10T11:26:55.322 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.314+0000 7fa7c0772740 1 -- 172.21.15.100:0/2021108197 wait complete. 2024-05-10T11:26:55.322 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.314+0000 7fa7c0772740 1 -- start start 2024-05-10T11:26:55.322 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.314+0000 7fa7c0772740 1 Processor -- start 2024-05-10T11:26:55.322 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.314+0000 7fa7c0772740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd615991c0 0x55dd615a3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:55.322 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.314+0000 7fa7c0772740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd61599bb0 0x55dd615a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:55.322 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.314+0000 7fa7c0772740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd6159ed20 0x55dd615aa980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:55.322 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.315+0000 7fa7c0772740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55dd615a1c70 con 0x55dd61599bb0 2024-05-10T11:26:55.322 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.315+0000 7fa7c0772740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55dd614b6370 con 0x55dd6159ed20 2024-05-10T11:26:55.322 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.315+0000 7fa7c0772740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55dd615a1df0 con 0x55dd615991c0 2024-05-10T11:26:55.322 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.315+0000 7fa7c0772740 10 client.0 _ll_register_callbacks cb 0x55dd6120aa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:26:55.322 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.315+0000 7fa7b67fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd61599bb0 0x55dd615a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.315+0000 7fa7b67fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd61599bb0 0x55dd615a39b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39092/0 (socket says 172.21.15.100:39092) 2024-05-10T11:26:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.315+0000 7fa7b67fc640 1 -- 172.21.15.100:0/2257279883 learned_addr learned my addr 172.21.15.100:0/2257279883 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:26:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.315+0000 7fa7b77fe640 1 --2- 172.21.15.100:0/2257279883 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd6159ed20 0x55dd615aa980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.315+0000 7fa7b6ffd640 1 --2- 172.21.15.100:0/2257279883 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd615991c0 0x55dd615a3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.315+0000 7fa7b67fc640 1 -- 172.21.15.100:0/2257279883 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd615991c0 msgr2=0x55dd615a3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.315+0000 7fa7b67fc640 1 --2- 172.21.15.100:0/2257279883 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd615991c0 0x55dd615a3470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.315+0000 7fa7b67fc640 1 -- 172.21.15.100:0/2257279883 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd6159ed20 msgr2=0x55dd615aa980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.315+0000 7fa7b67fc640 1 --2- 172.21.15.100:0/2257279883 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd6159ed20 0x55dd615aa980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.315+0000 7fa7b67fc640 1 -- 172.21.15.100:0/2257279883 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd612d7620 con 0x55dd61599bb0 2024-05-10T11:26:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.315+0000 7fa7b67fc640 1 --2- 172.21.15.100:0/2257279883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd61599bb0 0x55dd615a39b0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac003830 tx=0x7fa7ac005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dedc8864c58e9149 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:26:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.315+0000 7fa796ffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:26:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.315+0000 7fa796ffd640 1 -- 172.21.15.100:0/2257279883 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac04d070 con 0x55dd61599bb0 2024-05-10T11:26:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.315+0000 7fa796ffd640 1 -- 172.21.15.100:0/2257279883 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fa7ac003f40 con 0x55dd61599bb0 2024-05-10T11:26:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.315+0000 7fa7c0772740 1 -- 172.21.15.100:0/2257279883 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55dd615071e0 con 0x55dd61599bb0 2024-05-10T11:26:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.315+0000 7fa796ffd640 1 -- 172.21.15.100:0/2257279883 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa7ac056050 con 0x55dd61599bb0 2024-05-10T11:26:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.315+0000 7fa7c0772740 10 client.17085 fetch_fsmap learned FSMap version 31 2024-05-10T11:26:55.324 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.315+0000 7fa7c0772740 1 -- 172.21.15.100:0/2257279883 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55dd615a1c70 con 0x55dd61599bb0 2024-05-10T11:26:55.324 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.316+0000 7fa796ffd640 1 -- 172.21.15.100:0/2257279883 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55dd615071e0 con 0x55dd61599bb0 2024-05-10T11:26:55.324 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.316+0000 7fa796ffd640 1 -- 172.21.15.100:0/2257279883 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fa7ac050020 con 0x55dd61599bb0 2024-05-10T11:26:55.324 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.316+0000 7fa7c0772740 10 client.17085 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:26:55.324 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.316+0000 7fa7c0772740 10 client.17085 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:26:55.324 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.316+0000 7fa7c0772740 20 client.17085 populate_metadata read hostname 'smithi100' 2024-05-10T11:26:55.324 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.316+0000 7fa7c0772740 1 --2- 172.21.15.100:0/2257279883 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55dd615aec40 0x55dd615b1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:55.324 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.316+0000 7fa7c0772740 4 client.17085 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:26:55.324 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.316+0000 7fa7c0772740 1 -- 172.21.15.100:0/2257279883 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd61501050 con 0x55dd615aec40 2024-05-10T11:26:55.324 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.316+0000 7fa7b6ffd640 1 --2- 172.21.15.100:0/2257279883 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55dd615aec40 0x55dd615b1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:55.324 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.316+0000 7fa796ffd640 10 client.17085 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:26:55.324 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.316+0000 7fa7b6ffd640 1 --2- 172.21.15.100:0/2257279883 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55dd615aec40 0x55dd615b1060 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af0e9632911602da server_cookie=dcadab8f290178d5 in_seq=0 out_seq=0 2024-05-10T11:26:55.324 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.317+0000 7fa796ffd640 1 -- 172.21.15.100:0/2257279883 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dd61501050 con 0x55dd615aec40 2024-05-10T11:26:55.324 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.317+0000 7fa796ffd640 10 client.17085 handle_command_reply: tid=0 2024-05-10T11:26:55.324 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.317+0000 7fa7c0772740 1 client.17085 shutdown 2024-05-10T11:26:55.325 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.317+0000 7fa7c0772740 1 -- 172.21.15.100:0/2257279883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd61599bb0 msgr2=0x55dd615a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:55.325 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.317+0000 7fa7c0772740 1 --2- 172.21.15.100:0/2257279883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd61599bb0 0x55dd615a39b0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac003830 tx=0x7fa7ac005fd0 comp rx=0 tx=0).stop 2024-05-10T11:26:55.325 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.317+0000 7fa7c0772740 1 -- 172.21.15.100:0/2257279883 shutdown_connections 2024-05-10T11:26:55.325 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.317+0000 7fa7c0772740 1 --2- 172.21.15.100:0/2257279883 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55dd615aec40 0x55dd615b1060 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:55.325 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.317+0000 7fa7c0772740 1 --2- 172.21.15.100:0/2257279883 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd6159ed20 0x55dd615aa980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:55.325 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.317+0000 7fa7c0772740 1 --2- 172.21.15.100:0/2257279883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd61599bb0 0x55dd615a39b0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:55.325 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.317+0000 7fa7c0772740 1 --2- 172.21.15.100:0/2257279883 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd615991c0 0x55dd615a3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:55.325 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.317+0000 7fa796ffd640 0 client.17085 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:26:55.325 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.318+0000 7fa7c0772740 1 -- 172.21.15.100:0/2257279883 >> 172.21.15.100:0/2257279883 conn(0x55dd61590420 msgr2=0x55dd6158f580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:26:55.325 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.318+0000 7fa7c0772740 1 -- 172.21.15.100:0/2257279883 shutdown_connections 2024-05-10T11:26:55.325 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.318+0000 7fa7c0772740 1 -- 172.21.15.100:0/2257279883 wait complete. 2024-05-10T11:26:55.325 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.318+0000 7fa7c0772740 20 client.17085 trim_cache size 0 max 16384 2024-05-10T11:26:55.337 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.336+0000 7fa7c0772740 1 Processor -- start 2024-05-10T11:26:55.337 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.336+0000 7fa7c0772740 1 -- start start 2024-05-10T11:26:55.337 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.336+0000 7fa7c0772740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd615991c0 0x55dd615a3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:55.337 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.336+0000 7fa7c0772740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd61599bb0 0x55dd615a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:55.337 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.336+0000 7fa7c0772740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd6159ed20 0x55dd615aa940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:55.337 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.336+0000 7fa7c0772740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b81ab900 con 0x55dd6159ed20 2024-05-10T11:26:55.338 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.336+0000 7fa7c0772740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b81a91d0 con 0x55dd61599bb0 2024-05-10T11:26:55.338 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.336+0000 7fa7c0772740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b81a7020 con 0x55dd615991c0 2024-05-10T11:26:55.338 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.336+0000 7fa7b77fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd6159ed20 0x55dd615aa940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:55.338 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.336+0000 7fa7b77fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd6159ed20 0x55dd615aa940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39098/0 (socket says 172.21.15.100:39098) 2024-05-10T11:26:55.338 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.336+0000 7fa7b77fe640 1 -- 172.21.15.100:0/25267165 learned_addr learned my addr 172.21.15.100:0/25267165 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:26:55.338 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.336+0000 7fa7b67fc640 1 --2- 172.21.15.100:0/25267165 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd61599bb0 0x55dd615a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:55.338 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.336+0000 7fa7b6ffd640 1 --2- 172.21.15.100:0/25267165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd615991c0 0x55dd615a3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.337+0000 7fa7b77fe640 1 -- 172.21.15.100:0/25267165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd615991c0 msgr2=0x55dd615a3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.337+0000 7fa7b77fe640 1 --2- 172.21.15.100:0/25267165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd615991c0 0x55dd615a3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.337+0000 7fa7b77fe640 1 -- 172.21.15.100:0/25267165 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd61599bb0 msgr2=0x55dd615a39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.337+0000 7fa7b77fe640 1 --2- 172.21.15.100:0/25267165 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd61599bb0 0x55dd615a39b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.337+0000 7fa7b77fe640 1 -- 172.21.15.100:0/25267165 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd61590250 con 0x55dd6159ed20 2024-05-10T11:26:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.337+0000 7fa7b77fe640 1 --2- 172.21.15.100:0/25267165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd6159ed20 0x55dd615aa940 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fa7b00e05e0 tx=0x7fa7b00cc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13d74f03e9ac3929 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:26:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.338+0000 7fa79d7fa640 1 -- 172.21.15.100:0/25267165 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b00a9500 con 0x55dd6159ed20 2024-05-10T11:26:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.338+0000 7fa79d7fa640 1 -- 172.21.15.100:0/25267165 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa7b00a98a0 con 0x55dd6159ed20 2024-05-10T11:26:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.338+0000 7fa79d7fa640 1 -- 172.21.15.100:0/25267165 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b00a9ff0 con 0x55dd6159ed20 2024-05-10T11:26:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.339+0000 7fa7c0772740 1 -- 172.21.15.100:0/25267165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd6159ed20 msgr2=0x55dd615aa940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.339+0000 7fa7c0772740 1 --2- 172.21.15.100:0/25267165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd6159ed20 0x55dd615aa940 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fa7b00e05e0 tx=0x7fa7b00cc040 comp rx=0 tx=0).stop 2024-05-10T11:26:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.339+0000 7fa7c0772740 1 -- 172.21.15.100:0/25267165 shutdown_connections 2024-05-10T11:26:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.339+0000 7fa7c0772740 1 --2- 172.21.15.100:0/25267165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd6159ed20 0x55dd615aa940 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.339+0000 7fa7c0772740 1 --2- 172.21.15.100:0/25267165 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd61599bb0 0x55dd615a39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.339+0000 7fa7c0772740 1 --2- 172.21.15.100:0/25267165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd615991c0 0x55dd615a3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.339+0000 7fa7c0772740 1 -- 172.21.15.100:0/25267165 >> 172.21.15.100:0/25267165 conn(0x55dd61590420 msgr2=0x55dd61590800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:26:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.340+0000 7fa7c0772740 1 -- 172.21.15.100:0/25267165 shutdown_connections 2024-05-10T11:26:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.340+0000 7fa7c0772740 1 -- 172.21.15.100:0/25267165 wait complete. 2024-05-10T11:26:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.340+0000 7fa7c0772740 1 -- start start 2024-05-10T11:26:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.340+0000 7fa7c0772740 1 Processor -- start 2024-05-10T11:26:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.341+0000 7fa7c0772740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd615991c0 0x55dd615a8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.341+0000 7fa7c0772740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd61599bb0 0x55dd615a8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.341+0000 7fa7c0772740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd6159ed20 0x55dd615a9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.341+0000 7fa7c0772740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b81a7690 con 0x55dd615991c0 2024-05-10T11:26:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.341+0000 7fa7c0772740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b81a8b00 con 0x55dd6159ed20 2024-05-10T11:26:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.341+0000 7fa7c0772740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b81a7d60 con 0x55dd61599bb0 2024-05-10T11:26:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.341+0000 7fa7c0772740 10 client.0 _ll_register_callbacks cb 0x7fa7ac009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:26:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.341+0000 7fa7b6ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd615991c0 0x55dd615a8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.341+0000 7fa7b6ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd615991c0 0x55dd615a8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39102/0 (socket says 172.21.15.100:39102) 2024-05-10T11:26:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.341+0000 7fa7b6ffd640 1 -- 172.21.15.100:0/1350236741 learned_addr learned my addr 172.21.15.100:0/1350236741 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:26:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.341+0000 7fa7b67fc640 1 --2- 172.21.15.100:0/1350236741 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd61599bb0 0x55dd615a8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.342+0000 7fa7b77fe640 1 --2- 172.21.15.100:0/1350236741 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd6159ed20 0x55dd615a9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.342+0000 7fa7b6ffd640 1 -- 172.21.15.100:0/1350236741 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd61599bb0 msgr2=0x55dd615a8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.342+0000 7fa7b6ffd640 1 --2- 172.21.15.100:0/1350236741 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd61599bb0 0x55dd615a8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.342+0000 7fa7b6ffd640 1 -- 172.21.15.100:0/1350236741 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd6159ed20 msgr2=0x55dd615a9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:26:55.345 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.342+0000 7fa7b6ffd640 1 --2- 172.21.15.100:0/1350236741 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd6159ed20 0x55dd615a9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:55.345 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.342+0000 7fa7b6ffd640 1 -- 172.21.15.100:0/1350236741 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd615a9a70 con 0x55dd615991c0 2024-05-10T11:26:55.345 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.343+0000 7fa7b6ffd640 1 --2- 172.21.15.100:0/1350236741 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd615991c0 0x55dd615a8040 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8009f10 tx=0x7fa7a8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a0b3502435efd2f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:26:55.345 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.343+0000 7fa796ffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:26:55.345 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.343+0000 7fa796ffd640 1 -- 172.21.15.100:0/1350236741 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a802e070 con 0x55dd615991c0 2024-05-10T11:26:55.345 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.343+0000 7fa7c0772740 1 -- 172.21.15.100:0/1350236741 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7b00a98a0 con 0x55dd615991c0 2024-05-10T11:26:55.345 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.343+0000 7fa796ffd640 1 -- 172.21.15.100:0/1350236741 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fa7a8004470 con 0x55dd615991c0 2024-05-10T11:26:55.345 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.343+0000 7fa796ffd640 1 -- 172.21.15.100:0/1350236741 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa7a803c050 con 0x55dd615991c0 2024-05-10T11:26:55.345 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.343+0000 7fa7c0772740 10 client.17091 fetch_fsmap learned FSMap version 31 2024-05-10T11:26:55.345 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.343+0000 7fa7c0772740 1 -- 172.21.15.100:0/1350236741 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55dd6162a4d0 con 0x55dd615991c0 2024-05-10T11:26:55.346 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.344+0000 7fa796ffd640 1 -- 172.21.15.100:0/1350236741 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa7b00a98a0 con 0x55dd615991c0 2024-05-10T11:26:55.346 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.344+0000 7fa796ffd640 1 -- 172.21.15.100:0/1350236741 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fa7a800f900 con 0x55dd615991c0 2024-05-10T11:26:55.346 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.344+0000 7fa7c0772740 10 client.17091 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:26:55.346 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.344+0000 7fa7c0772740 10 client.17091 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:26:55.347 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.344+0000 7fa7c0772740 20 client.17091 populate_metadata read hostname 'smithi100' 2024-05-10T11:26:55.347 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.344+0000 7fa7c0772740 1 --2- 172.21.15.100:0/1350236741 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55dd615aec40 0x55dd6162ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:26:55.347 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.344+0000 7fa7c0772740 4 client.17091 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:26:55.347 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.344+0000 7fa7c0772740 1 -- 172.21.15.100:0/1350236741 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dd6147c5f0 con 0x55dd615aec40 2024-05-10T11:26:55.347 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.344+0000 7fa7b67fc640 1 --2- 172.21.15.100:0/1350236741 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55dd615aec40 0x55dd6162ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:26:55.347 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.345+0000 7fa796ffd640 10 client.17091 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:26:55.347 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.345+0000 7fa7b67fc640 1 --2- 172.21.15.100:0/1350236741 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55dd615aec40 0x55dd6162ce70 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23bd03e88da3c2aa server_cookie=fc64a1d473ea59f4 in_seq=0 out_seq=0 2024-05-10T11:26:55.363 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.363+0000 7fa796ffd640 1 -- 172.21.15.100:0/1350236741 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dd6147c5f0 con 0x55dd615aec40 2024-05-10T11:26:55.363 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.363+0000 7fa796ffd640 10 client.17091 handle_command_reply: tid=0 2024-05-10T11:26:55.363 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.363+0000 7fa7c0772740 1 client.17091 shutdown 2024-05-10T11:26:55.363 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.363+0000 7fa7c0772740 1 -- 172.21.15.100:0/1350236741 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd615991c0 msgr2=0x55dd615a8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:55.363 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.363+0000 7fa7c0772740 1 --2- 172.21.15.100:0/1350236741 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd615991c0 0x55dd615a8040 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8009f10 tx=0x7fa7a8004730 comp rx=0 tx=0).stop 2024-05-10T11:26:55.363 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.363+0000 7fa7c0772740 1 -- 172.21.15.100:0/1350236741 shutdown_connections 2024-05-10T11:26:55.363 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.363+0000 7fa7c0772740 1 --2- 172.21.15.100:0/1350236741 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55dd615aec40 0x55dd6162ce70 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.363+0000 7fa7c0772740 1 --2- 172.21.15.100:0/1350236741 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd6159ed20 0x55dd615a9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.363+0000 7fa7c0772740 1 --2- 172.21.15.100:0/1350236741 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd61599bb0 0x55dd615a8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.363+0000 7fa796ffd640 0 client.17091 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:26:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.363+0000 7fa7c0772740 1 --2- 172.21.15.100:0/1350236741 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd615991c0 0x55dd615a8040 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.363+0000 7fa7c0772740 1 -- 172.21.15.100:0/1350236741 >> 172.21.15.100:0/1350236741 conn(0x55dd61590420 msgr2=0x55dd615a33a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:26:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.363+0000 7fa7c0772740 1 -- 172.21.15.100:0/1350236741 shutdown_connections 2024-05-10T11:26:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.363+0000 7fa7c0772740 1 -- 172.21.15.100:0/1350236741 wait complete. 2024-05-10T11:26:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.363+0000 7fa7c0772740 20 client.17091 trim_cache size 0 max 16384 2024-05-10T11:26:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:26:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:26:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:26:55.365 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:26:55.365 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.364+0000 7fa7bda59640 1 -- 172.21.15.100:0/2348373027 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa7a00b28a0 msgr2=0x7fa7a00b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:55.365 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.364+0000 7fa7bda59640 1 --2- 172.21.15.100:0/2348373027 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa7a00b28a0 0x7fa7a00b4d60 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x55dd61599fb0 tx=0x7fa7a8005f90 comp rx=0 tx=0).stop 2024-05-10T11:26:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.364+0000 7fa7bda59640 1 -- 172.21.15.100:0/2348373027 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7b81045c0 msgr2=0x7fa7b81a0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:26:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.364+0000 7fa7bda59640 1 --2- 172.21.15.100:0/2348373027 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7b81045c0 0x7fa7b81a0010 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fa7b000eb10 tx=0x7fa7b000c2d0 comp rx=0 tx=0).stop 2024-05-10T11:26:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.364+0000 7fa7bda59640 1 -- 172.21.15.100:0/2348373027 shutdown_connections 2024-05-10T11:26:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.364+0000 7fa7bda59640 1 --2- 172.21.15.100:0/2348373027 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa7a00b28a0 0x7fa7a00b4d60 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.364+0000 7fa7bda59640 1 --2- 172.21.15.100:0/2348373027 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7b81045c0 0x7fa7b81a0010 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.364+0000 7fa7bda59640 1 --2- 172.21.15.100:0/2348373027 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7b8103c00 0x7fa7b8198fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.364+0000 7fa7bda59640 1 --2- 172.21.15.100:0/2348373027 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7b8102a10 0x7fa7b8198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:26:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.364+0000 7fa7bda59640 1 -- 172.21.15.100:0/2348373027 >> 172.21.15.100:0/2348373027 conn(0x7fa7b80fdfa0 msgr2=0x7fa7b8106f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:26:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.364+0000 7fa7bda59640 1 -- 172.21.15.100:0/2348373027 shutdown_connections 2024-05-10T11:26:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:26:55.364+0000 7fa7bda59640 1 -- 172.21.15.100:0/2348373027 wait complete. 2024-05-10T11:26:55.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:26:55 smithi178 ceph-mon[33359]: pgmap v676: 129 pgs: 129 active+clean; 87 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1.4 MiB/s wr, 135 op/s 2024-05-10T11:26:55.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:26:55 smithi188 ceph-mon[34075]: pgmap v676: 129 pgs: 129 active+clean; 87 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1.4 MiB/s wr, 135 op/s 2024-05-10T11:26:55.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:26:55 smithi100 ceph-mon[30296]: pgmap v676: 129 pgs: 129 active+clean; 87 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1.4 MiB/s wr, 135 op/s 2024-05-10T11:26:55.976 INFO:tasks.workunit.client.0.smithi100.stdout: 1 212479 2.75 MB/sec execute 290 sec latency 19.966 ms 2024-05-10T11:26:56.976 INFO:tasks.workunit.client.0.smithi100.stdout: 1 212939 2.76 MB/sec execute 291 sec latency 14.736 ms 2024-05-10T11:26:57.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:26:57 smithi178 ceph-mon[33359]: pgmap v677: 129 pgs: 129 active+clean; 87 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 1.5 MiB/s wr, 173 op/s 2024-05-10T11:26:57.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:26:57 smithi188 ceph-mon[34075]: pgmap v677: 129 pgs: 129 active+clean; 87 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 1.5 MiB/s wr, 173 op/s 2024-05-10T11:26:57.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:26:57 smithi100 ceph-mon[30296]: pgmap v677: 129 pgs: 129 active+clean; 87 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 1.5 MiB/s wr, 173 op/s 2024-05-10T11:26:57.976 INFO:tasks.workunit.client.0.smithi100.stdout: 1 213253 2.75 MB/sec execute 292 sec latency 14.652 ms 2024-05-10T11:26:58.976 INFO:tasks.workunit.client.0.smithi100.stdout: 1 213683 2.75 MB/sec execute 293 sec latency 12.505 ms 2024-05-10T11:26:59.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:26:59 smithi178 ceph-mon[33359]: pgmap v678: 129 pgs: 129 active+clean; 88 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 1.4 MiB/s wr, 140 op/s 2024-05-10T11:26:59.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:26:59 smithi188 ceph-mon[34075]: pgmap v678: 129 pgs: 129 active+clean; 88 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 1.4 MiB/s wr, 140 op/s 2024-05-10T11:26:59.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:26:59 smithi100 ceph-mon[30296]: pgmap v678: 129 pgs: 129 active+clean; 88 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 1.4 MiB/s wr, 140 op/s 2024-05-10T11:26:59.976 INFO:tasks.workunit.client.0.smithi100.stdout: 1 214268 2.75 MB/sec execute 294 sec latency 16.701 ms 2024-05-10T11:27:00.976 INFO:tasks.workunit.client.0.smithi100.stdout: 1 214929 2.74 MB/sec execute 295 sec latency 18.274 ms 2024-05-10T11:27:01.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:27:01 smithi178 ceph-mon[33359]: pgmap v679: 129 pgs: 129 active+clean; 90 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 1.3 MiB/s wr, 159 op/s 2024-05-10T11:27:01.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:27:01 smithi188 ceph-mon[34075]: pgmap v679: 129 pgs: 129 active+clean; 90 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 1.3 MiB/s wr, 159 op/s 2024-05-10T11:27:01.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:27:01 smithi100 ceph-mon[30296]: pgmap v679: 129 pgs: 129 active+clean; 90 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 1.3 MiB/s wr, 159 op/s 2024-05-10T11:27:01.976 INFO:tasks.workunit.client.0.smithi100.stdout: 1 215471 2.74 MB/sec execute 296 sec latency 14.814 ms 2024-05-10T11:27:02.850 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T11:27:02.977 INFO:tasks.workunit.client.0.smithi100.stdout: 1 216186 2.76 MB/sec execute 297 sec latency 15.494 ms 2024-05-10T11:27:03.124 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:27:03.498 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:27:03 smithi100 ceph-mon[30296]: pgmap v680: 129 pgs: 129 active+clean; 89 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 1.5 MiB/s wr, 168 op/s 2024-05-10T11:27:03.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:27:03 smithi178 ceph-mon[33359]: pgmap v680: 129 pgs: 129 active+clean; 89 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 1.5 MiB/s wr, 168 op/s 2024-05-10T11:27:03.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:27:03 smithi188 ceph-mon[34075]: pgmap v680: 129 pgs: 129 active+clean; 89 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 1.5 MiB/s wr, 168 op/s 2024-05-10T11:27:03.977 INFO:tasks.workunit.client.0.smithi100.stdout: 1 216652 2.76 MB/sec execute 298 sec latency 15.536 ms 2024-05-10T11:27:04.977 INFO:tasks.workunit.client.0.smithi100.stdout: 1 216898 2.75 MB/sec execute 299 sec latency 19.791 ms 2024-05-10T11:27:04.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.985+0000 7f4387449700 1 -- 172.21.15.100:0/26166329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43801037b0 msgr2=0x7f4380103b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:04.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.985+0000 7f4387449700 1 --2- 172.21.15.100:0/26166329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43801037b0 0x7f4380103b90 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f437c0099c0 tx=0x7f437c009cd0 comp rx=0 tx=0).stop 2024-05-10T11:27:04.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.985+0000 7f4387449700 1 -- 172.21.15.100:0/26166329 shutdown_connections 2024-05-10T11:27:04.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.985+0000 7f4387449700 1 --2- 172.21.15.100:0/26166329 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4380108880 0x7f4380110400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:04.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.985+0000 7f4387449700 1 --2- 172.21.15.100:0/26166329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43801037b0 0x7f4380103b90 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:04.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.985+0000 7f4387449700 1 --2- 172.21.15.100:0/26166329 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4380104160 0x7f4380108150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:04.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.985+0000 7f4387449700 1 -- 172.21.15.100:0/26166329 >> 172.21.15.100:0/26166329 conn(0x7f438007a530 msgr2=0x7f438007a940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:27:04.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.986+0000 7f4387449700 1 -- 172.21.15.100:0/26166329 shutdown_connections 2024-05-10T11:27:04.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.986+0000 7f4387449700 1 -- 172.21.15.100:0/26166329 wait complete. 2024-05-10T11:27:04.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.986+0000 7f4387449700 1 Processor -- start 2024-05-10T11:27:04.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.986+0000 7f4387449700 1 -- start start 2024-05-10T11:27:04.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.986+0000 7f4387449700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43801037b0 0x7f438019f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:04.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.986+0000 7f4387449700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4380104160 0x7f438019fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:04.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.986+0000 7f4387449700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4380108880 0x7f4380199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:04.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.986+0000 7f4387449700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4380113550 con 0x7f43801037b0 2024-05-10T11:27:04.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.986+0000 7f4387449700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f43801133d0 con 0x7f4380108880 2024-05-10T11:27:04.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.986+0000 7f4387449700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f43801136d0 con 0x7f4380104160 2024-05-10T11:27:04.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.987+0000 7f43851e5700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43801037b0 0x7f438019f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:04.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.987+0000 7f43851e5700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43801037b0 0x7f438019f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40722/0 (socket says 172.21.15.100:40722) 2024-05-10T11:27:04.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.987+0000 7f43851e5700 1 -- 172.21.15.100:0/1929842675 learned_addr learned my addr 172.21.15.100:0/1929842675 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:27:04.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.987+0000 7f43849e4700 1 --2- 172.21.15.100:0/1929842675 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4380104160 0x7f438019fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:04.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.987+0000 7f43859e6700 1 --2- 172.21.15.100:0/1929842675 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4380108880 0x7f4380199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:04.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.987+0000 7f43851e5700 1 -- 172.21.15.100:0/1929842675 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4380104160 msgr2=0x7f438019fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:04.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.987+0000 7f43851e5700 1 --2- 172.21.15.100:0/1929842675 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4380104160 0x7f438019fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:04.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.987+0000 7f43851e5700 1 -- 172.21.15.100:0/1929842675 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4380108880 msgr2=0x7f4380199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:04.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.987+0000 7f43851e5700 1 --2- 172.21.15.100:0/1929842675 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4380108880 0x7f4380199890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:04.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.987+0000 7f43851e5700 1 -- 172.21.15.100:0/1929842675 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f438019a070 con 0x7f43801037b0 2024-05-10T11:27:04.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.987+0000 7f43851e5700 1 --2- 172.21.15.100:0/1929842675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43801037b0 0x7f438019f6c0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f437c000c00 tx=0x7f437c00f460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:27:04.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.987+0000 7f43767fc700 1 -- 172.21.15.100:0/1929842675 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f437c027070 con 0x7f43801037b0 2024-05-10T11:27:04.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.987+0000 7f43767fc700 1 -- 172.21.15.100:0/1929842675 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f437c00fcf0 con 0x7f43801037b0 2024-05-10T11:27:04.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.987+0000 7f4387449700 1 -- 172.21.15.100:0/1929842675 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f438019a300 con 0x7f43801037b0 2024-05-10T11:27:04.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.987+0000 7f4387449700 1 -- 172.21.15.100:0/1929842675 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43801a6520 con 0x7f43801037b0 2024-05-10T11:27:04.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.987+0000 7f43767fc700 1 -- 172.21.15.100:0/1929842675 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f437c022dc0 con 0x7f43801037b0 2024-05-10T11:27:04.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.990+0000 7f43767fc700 1 -- 172.21.15.100:0/1929842675 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f437c02a070 con 0x7f43801037b0 2024-05-10T11:27:04.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.991+0000 7f43767fc700 1 --2- 172.21.15.100:0/1929842675 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f436c0b2900 0x7f436c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:04.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.991+0000 7f43849e4700 1 --2- 172.21.15.100:0/1929842675 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f436c0b2900 0x7f436c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:04.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.991+0000 7f43767fc700 1 -- 172.21.15.100:0/1929842675 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f437c0e3050 con 0x7f43801037b0 2024-05-10T11:27:04.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.991+0000 7f4391cdcb80 1 Processor -- start 2024-05-10T11:27:04.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.991+0000 7f43849e4700 1 --2- 172.21.15.100:0/1929842675 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f436c0b2900 0x7f436c0b4dc0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4380103060 tx=0x7f43700073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:27:04.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.991+0000 7f4391cdcb80 1 -- start start 2024-05-10T11:27:04.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.992+0000 7f4391cdcb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c53076a4d0 0x55c53076a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:04.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.992+0000 7f4391cdcb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c53076adf0 0x55c53076e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:04.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.992+0000 7f4391cdcb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c53076ec80 0x55c53076f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:04.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.992+0000 7f4391cdcb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55c5306bc860 con 0x55c53076ec80 2024-05-10T11:27:04.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.992+0000 7f4391cdcb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c53065b7c0 con 0x55c53076adf0 2024-05-10T11:27:04.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.992+0000 7f4391cdcb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c5306b8820 con 0x55c53076a4d0 2024-05-10T11:27:04.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.992+0000 7f43859e6700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c53076ec80 0x55c53076f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:04.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.992+0000 7f43851e5700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c53076a4d0 0x55c53076a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:04.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.992+0000 7f43859e6700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c53076ec80 0x55c53076f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40736/0 (socket says 172.21.15.100:40736) 2024-05-10T11:27:04.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.992+0000 7f43859e6700 1 -- 172.21.15.100:0/1861778731 learned_addr learned my addr 172.21.15.100:0/1861778731 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:27:04.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.992+0000 7f43849e4700 1 --2- 172.21.15.100:0/1861778731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c53076adf0 0x55c53076e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:04.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.992+0000 7f43859e6700 1 -- 172.21.15.100:0/1861778731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c53076a4d0 msgr2=0x55c53076a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:04.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.992+0000 7f43859e6700 1 --2- 172.21.15.100:0/1861778731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c53076a4d0 0x55c53076a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:04.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.992+0000 7f43859e6700 1 -- 172.21.15.100:0/1861778731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c53076adf0 msgr2=0x55c53076e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:04.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.992+0000 7f43859e6700 1 --2- 172.21.15.100:0/1861778731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c53076adf0 0x55c53076e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:04.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.992+0000 7f43859e6700 1 -- 172.21.15.100:0/1861778731 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5306098d0 con 0x55c53076ec80 2024-05-10T11:27:04.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.992+0000 7f43849e4700 1 --2- 172.21.15.100:0/1861778731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c53076adf0 0x55c53076e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:27:04.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.992+0000 7f43859e6700 1 --2- 172.21.15.100:0/1861778731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c53076ec80 0x55c53076f100 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f4378003fa0 tx=0x7f4378009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68caf37c25448994 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:27:04.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.992+0000 7f436b7fe700 1 -- 172.21.15.100:0/1861778731 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4378004bb0 con 0x55c53076ec80 2024-05-10T11:27:04.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.992+0000 7f436b7fe700 1 -- 172.21.15.100:0/1861778731 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4378004820 con 0x55c53076ec80 2024-05-10T11:27:04.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.992+0000 7f436b7fe700 1 -- 172.21.15.100:0/1861778731 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4378007cb0 con 0x55c53076ec80 2024-05-10T11:27:04.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.993+0000 7f4391cdcb80 1 -- 172.21.15.100:0/1861778731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c53076ec80 msgr2=0x55c53076f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:04.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.993+0000 7f4391cdcb80 1 --2- 172.21.15.100:0/1861778731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c53076ec80 0x55c53076f100 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f4378003fa0 tx=0x7f4378009ba0 comp rx=0 tx=0).stop 2024-05-10T11:27:04.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.993+0000 7f4391cdcb80 1 -- 172.21.15.100:0/1861778731 shutdown_connections 2024-05-10T11:27:04.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.993+0000 7f4391cdcb80 1 --2- 172.21.15.100:0/1861778731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c53076ec80 0x55c53076f100 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:04.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.993+0000 7f4391cdcb80 1 --2- 172.21.15.100:0/1861778731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c53076a4d0 0x55c53076a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:04.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.993+0000 7f4391cdcb80 1 --2- 172.21.15.100:0/1861778731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c53076adf0 0x55c53076e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:04.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.993+0000 7f4391cdcb80 1 -- 172.21.15.100:0/1861778731 >> 172.21.15.100:0/1861778731 conn(0x55c530696100 msgr2=0x55c530764c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:27:04.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.993+0000 7f4391cdcb80 1 -- 172.21.15.100:0/1861778731 shutdown_connections 2024-05-10T11:27:04.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.993+0000 7f4391cdcb80 1 -- 172.21.15.100:0/1861778731 wait complete. 2024-05-10T11:27:04.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.994+0000 7f4391cdcb80 1 -- start start 2024-05-10T11:27:04.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.994+0000 7f4391cdcb80 1 Processor -- start 2024-05-10T11:27:04.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.994+0000 7f4391cdcb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c53076a4d0 0x55c530775ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:04.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.994+0000 7f4391cdcb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c53076adf0 0x55c530773400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:04.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.994+0000 7f4391cdcb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c53076ec80 0x55c530773940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:04.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.994+0000 7f4391cdcb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55c530776e90 con 0x55c53076ec80 2024-05-10T11:27:04.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.994+0000 7f43849e4700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c53076adf0 0x55c530773400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:04.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.994+0000 7f4391cdcb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c530622c30 con 0x55c53076a4d0 2024-05-10T11:27:04.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.994+0000 7f4391cdcb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c530777010 con 0x55c53076adf0 2024-05-10T11:27:04.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.994+0000 7f43849e4700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c53076adf0 0x55c530773400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:58950/0 (socket says 172.21.15.100:58950) 2024-05-10T11:27:04.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.994+0000 7f43849e4700 1 -- 172.21.15.100:0/3444301441 learned_addr learned my addr 172.21.15.100:0/3444301441 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:27:04.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.994+0000 7f43851e5700 1 --2- 172.21.15.100:0/3444301441 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c53076a4d0 0x55c530775ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:04.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.994+0000 7f43859e6700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c53076ec80 0x55c530773940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:04.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.995+0000 7f4391cdcb80 10 client.0 _ll_register_callbacks cb 0x55c530654920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:27:04.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.995+0000 7f43849e4700 1 -- 172.21.15.100:0/3444301441 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c53076a4d0 msgr2=0x55c530775ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:04.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.995+0000 7f43849e4700 1 --2- 172.21.15.100:0/3444301441 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c53076a4d0 0x55c530775ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:04.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.995+0000 7f43849e4700 1 -- 172.21.15.100:0/3444301441 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c53076ec80 msgr2=0x55c530773940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:04.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.995+0000 7f43849e4700 1 --2- 172.21.15.100:0/3444301441 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c53076ec80 0x55c530773940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:04.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.995+0000 7f43849e4700 1 -- 172.21.15.100:0/3444301441 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c530658690 con 0x55c53076adf0 2024-05-10T11:27:04.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.995+0000 7f43851e5700 1 --2- 172.21.15.100:0/3444301441 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c53076a4d0 0x55c530775ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:27:04.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.995+0000 7f43859e6700 1 --2- 172.21.15.100:0/3444301441 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c53076ec80 0x55c530773940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:27:04.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.995+0000 7f43849e4700 1 --2- 172.21.15.100:0/3444301441 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c53076adf0 0x55c530773400 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f43700117c0 tx=0x7f43700047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b315af2fdcff6f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:27:04.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.995+0000 7f4368ff9700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:27:04.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.995+0000 7f4368ff9700 1 -- 172.21.15.100:0/3444301441 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4370004250 con 0x55c53076adf0 2024-05-10T11:27:04.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.995+0000 7f4368ff9700 1 -- 172.21.15.100:0/3444301441 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f43700043f0 con 0x55c53076adf0 2024-05-10T11:27:04.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.995+0000 7f4391cdcb80 1 -- 172.21.15.100:0/3444301441 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4378004820 con 0x55c53076adf0 2024-05-10T11:27:04.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.996+0000 7f4368ff9700 1 -- 172.21.15.100:0/3444301441 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f4370030050 con 0x55c53076adf0 2024-05-10T11:27:04.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.996+0000 7f4391cdcb80 10 client.26575 fetch_fsmap learned FSMap version 31 2024-05-10T11:27:04.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.996+0000 7f4391cdcb80 1 -- 172.21.15.100:0/3444301441 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c5307816b0 con 0x55c53076adf0 2024-05-10T11:27:04.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.996+0000 7f4368ff9700 1 -- 172.21.15.100:0/3444301441 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4378004820 con 0x55c53076adf0 2024-05-10T11:27:04.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.996+0000 7f4368ff9700 1 -- 172.21.15.100:0/3444301441 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f437001f550 con 0x55c53076adf0 2024-05-10T11:27:04.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.996+0000 7f4391cdcb80 10 client.26575 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:27:04.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.996+0000 7f4391cdcb80 10 client.26575 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:27:04.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.996+0000 7f4391cdcb80 20 client.26575 populate_metadata read hostname 'smithi100' 2024-05-10T11:27:04.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.996+0000 7f4391cdcb80 1 --2- 172.21.15.100:0/3444301441 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c530783d40 0x55c530786150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:05.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.996+0000 7f4391cdcb80 4 client.26575 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:27:05.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.996+0000 7f4391cdcb80 1 -- 172.21.15.100:0/3444301441 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c530736650 con 0x55c530783d40 2024-05-10T11:27:05.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.996+0000 7f43851e5700 1 --2- 172.21.15.100:0/3444301441 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c530783d40 0x55c530786150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:05.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.997+0000 7f4368ff9700 10 client.26575 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:27:05.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.997+0000 7f43851e5700 1 --2- 172.21.15.100:0/3444301441 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c530783d40 0x55c530786150 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e34774af4362ce7 server_cookie=c3fe8508e5a9277c in_seq=0 out_seq=0 2024-05-10T11:27:05.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.998+0000 7f4368ff9700 1 -- 172.21.15.100:0/3444301441 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c530736650 con 0x55c530783d40 2024-05-10T11:27:05.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.998+0000 7f4368ff9700 10 client.26575 handle_command_reply: tid=0 2024-05-10T11:27:05.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.998+0000 7f4391cdcb80 1 client.26575 shutdown 2024-05-10T11:27:05.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.998+0000 7f4391cdcb80 1 -- 172.21.15.100:0/3444301441 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c53076adf0 msgr2=0x55c530773400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:05.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.998+0000 7f4391cdcb80 1 --2- 172.21.15.100:0/3444301441 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c53076adf0 0x55c530773400 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f43700117c0 tx=0x7f43700047c0 comp rx=0 tx=0).stop 2024-05-10T11:27:05.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.998+0000 7f4391cdcb80 1 -- 172.21.15.100:0/3444301441 shutdown_connections 2024-05-10T11:27:05.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.998+0000 7f4391cdcb80 1 --2- 172.21.15.100:0/3444301441 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c530783d40 0x55c530786150 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:05.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.998+0000 7f4391cdcb80 1 --2- 172.21.15.100:0/3444301441 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c53076ec80 0x55c530773940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:05.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.998+0000 7f4391cdcb80 1 --2- 172.21.15.100:0/3444301441 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c53076a4d0 0x55c530775ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:05.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.998+0000 7f4391cdcb80 1 --2- 172.21.15.100:0/3444301441 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c53076adf0 0x55c530773400 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:05.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.998+0000 7f4368ff9700 0 client.26575 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:27:05.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.998+0000 7f4391cdcb80 1 -- 172.21.15.100:0/3444301441 >> 172.21.15.100:0/3444301441 conn(0x55c530696100 msgr2=0x55c53076f080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:27:05.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.998+0000 7f4391cdcb80 1 -- 172.21.15.100:0/3444301441 shutdown_connections 2024-05-10T11:27:05.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.998+0000 7f4391cdcb80 1 -- 172.21.15.100:0/3444301441 wait complete. 2024-05-10T11:27:05.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:04.998+0000 7f4391cdcb80 20 client.26575 trim_cache size 0 max 16384 2024-05-10T11:27:05.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.024+0000 7f4391cdcb80 1 Processor -- start 2024-05-10T11:27:05.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.024+0000 7f4391cdcb80 1 -- start start 2024-05-10T11:27:05.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.024+0000 7f4391cdcb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c530696100 0x55c530773990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:05.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.024+0000 7f4391cdcb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c53076a4d0 0x55c53077ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:05.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.024+0000 7f4391cdcb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c53076adf0 0x55c5307699f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:05.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.024+0000 7f4391cdcb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f43801af660 con 0x55c530696100 2024-05-10T11:27:05.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.024+0000 7f4391cdcb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f43801acef0 con 0x55c53076adf0 2024-05-10T11:27:05.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.024+0000 7f4391cdcb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f43801aad40 con 0x55c53076a4d0 2024-05-10T11:27:05.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.024+0000 7f43851e5700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c530696100 0x55c530773990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:05.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.024+0000 7f43851e5700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c530696100 0x55c530773990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40752/0 (socket says 172.21.15.100:40752) 2024-05-10T11:27:05.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.024+0000 7f43851e5700 1 -- 172.21.15.100:0/2877027080 learned_addr learned my addr 172.21.15.100:0/2877027080 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:27:05.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.024+0000 7f43849e4700 1 --2- 172.21.15.100:0/2877027080 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c53076a4d0 0x55c53077ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:05.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.024+0000 7f43859e6700 1 --2- 172.21.15.100:0/2877027080 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c53076adf0 0x55c5307699f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:05.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.024+0000 7f43851e5700 1 -- 172.21.15.100:0/2877027080 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c53076a4d0 msgr2=0x55c53077ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:05.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.024+0000 7f43851e5700 1 --2- 172.21.15.100:0/2877027080 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c53076a4d0 0x55c53077ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:05.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.024+0000 7f43851e5700 1 -- 172.21.15.100:0/2877027080 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c53076adf0 msgr2=0x55c5307699f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:05.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.024+0000 7f43851e5700 1 --2- 172.21.15.100:0/2877027080 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c53076adf0 0x55c5307699f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:05.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.025+0000 7f43851e5700 1 -- 172.21.15.100:0/2877027080 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c530775ca0 con 0x55c530696100 2024-05-10T11:27:05.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.025+0000 7f43851e5700 1 --2- 172.21.15.100:0/2877027080 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c530696100 0x55c530773990 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f437c0aa0a0 tx=0x7f437c092040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd6e2f82c42ce66e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:27:05.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.025+0000 7f436affd700 1 -- 172.21.15.100:0/2877027080 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f437c0aa3a0 con 0x55c530696100 2024-05-10T11:27:05.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.025+0000 7f436affd700 1 -- 172.21.15.100:0/2877027080 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f437c0ae750 con 0x55c530696100 2024-05-10T11:27:05.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.025+0000 7f436affd700 1 -- 172.21.15.100:0/2877027080 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f437c0050d0 con 0x55c530696100 2024-05-10T11:27:05.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.026+0000 7f4391cdcb80 1 -- 172.21.15.100:0/2877027080 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c530696100 msgr2=0x55c530773990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:05.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.026+0000 7f4391cdcb80 1 --2- 172.21.15.100:0/2877027080 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c530696100 0x55c530773990 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f437c0aa0a0 tx=0x7f437c092040 comp rx=0 tx=0).stop 2024-05-10T11:27:05.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.026+0000 7f4391cdcb80 1 -- 172.21.15.100:0/2877027080 shutdown_connections 2024-05-10T11:27:05.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.026+0000 7f4391cdcb80 1 --2- 172.21.15.100:0/2877027080 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c53076adf0 0x55c5307699f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:05.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.026+0000 7f4391cdcb80 1 --2- 172.21.15.100:0/2877027080 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c530696100 0x55c530773990 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:05.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.026+0000 7f4391cdcb80 1 --2- 172.21.15.100:0/2877027080 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c53076a4d0 0x55c53077ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:05.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.026+0000 7f4391cdcb80 1 -- 172.21.15.100:0/2877027080 >> 172.21.15.100:0/2877027080 conn(0x55c53075fab0 msgr2=0x55c5307692e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:27:05.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.026+0000 7f4391cdcb80 1 -- 172.21.15.100:0/2877027080 shutdown_connections 2024-05-10T11:27:05.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.026+0000 7f4391cdcb80 1 -- 172.21.15.100:0/2877027080 wait complete. 2024-05-10T11:27:05.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.026+0000 7f4391cdcb80 1 -- start start 2024-05-10T11:27:05.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.026+0000 7f4391cdcb80 1 Processor -- start 2024-05-10T11:27:05.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.027+0000 7f4391cdcb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c530696100 0x55c53077c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:05.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.027+0000 7f4391cdcb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c53076a4d0 0x55c53077c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:05.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.027+0000 7f4391cdcb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c53076adf0 0x55c530778550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:05.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.027+0000 7f4391cdcb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f43801ab3b0 con 0x55c530696100 2024-05-10T11:27:05.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.027+0000 7f4391cdcb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f43801ac820 con 0x55c53076adf0 2024-05-10T11:27:05.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.027+0000 7f4391cdcb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f43801aba80 con 0x55c53076a4d0 2024-05-10T11:27:05.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.027+0000 7f43851e5700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c530696100 0x55c53077c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:05.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.027+0000 7f43849e4700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c53076a4d0 0x55c53077c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:05.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.027+0000 7f43851e5700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c530696100 0x55c53077c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40758/0 (socket says 172.21.15.100:40758) 2024-05-10T11:27:05.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.027+0000 7f43851e5700 1 -- 172.21.15.100:0/3179177041 learned_addr learned my addr 172.21.15.100:0/3179177041 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:27:05.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.027+0000 7f4391cdcb80 10 client.0 _ll_register_callbacks cb 0x55c530654920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:27:05.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.027+0000 7f43859e6700 1 --2- 172.21.15.100:0/3179177041 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c53076adf0 0x55c530778550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:05.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.027+0000 7f43851e5700 1 -- 172.21.15.100:0/3179177041 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c53076a4d0 msgr2=0x55c53077c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:05.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.027+0000 7f43851e5700 1 --2- 172.21.15.100:0/3179177041 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c53076a4d0 0x55c53077c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:05.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.027+0000 7f43851e5700 1 -- 172.21.15.100:0/3179177041 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c53076adf0 msgr2=0x55c530778550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:05.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.027+0000 7f43851e5700 1 --2- 172.21.15.100:0/3179177041 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c53076adf0 0x55c530778550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:05.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.027+0000 7f43851e5700 1 -- 172.21.15.100:0/3179177041 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c530778c70 con 0x55c530696100 2024-05-10T11:27:05.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.027+0000 7f43851e5700 1 --2- 172.21.15.100:0/3179177041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c530696100 0x55c53077c050 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f437c005cb0 tx=0x7f437c0aa920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeb69f87746f3412 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:27:05.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.027+0000 7f4368ff9700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:27:05.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.028+0000 7f4368ff9700 1 -- 172.21.15.100:0/3179177041 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f437c0ca060 con 0x55c530696100 2024-05-10T11:27:05.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.028+0000 7f4368ff9700 1 -- 172.21.15.100:0/3179177041 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f437c0b2ce0 con 0x55c530696100 2024-05-10T11:27:05.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.028+0000 7f4391cdcb80 1 -- 172.21.15.100:0/3179177041 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f437c0ae750 con 0x55c530696100 2024-05-10T11:27:05.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.028+0000 7f4368ff9700 1 -- 172.21.15.100:0/3179177041 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f437c0b3170 con 0x55c530696100 2024-05-10T11:27:05.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.028+0000 7f4391cdcb80 10 client.17109 fetch_fsmap learned FSMap version 31 2024-05-10T11:27:05.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.028+0000 7f4391cdcb80 1 -- 172.21.15.100:0/3179177041 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c5307ef050 con 0x55c530696100 2024-05-10T11:27:05.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.028+0000 7f4368ff9700 1 -- 172.21.15.100:0/3179177041 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f437c0ae750 con 0x55c530696100 2024-05-10T11:27:05.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.028+0000 7f4368ff9700 1 -- 172.21.15.100:0/3179177041 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f437c0a9600 con 0x55c530696100 2024-05-10T11:27:05.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.028+0000 7f4391cdcb80 10 client.17109 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:27:05.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.028+0000 7f4391cdcb80 10 client.17109 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:27:05.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.028+0000 7f4391cdcb80 20 client.17109 populate_metadata read hostname 'smithi100' 2024-05-10T11:27:05.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.028+0000 7f4391cdcb80 1 --2- 172.21.15.100:0/3179177041 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c53076ec80 0x55c5307f16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:05.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.028+0000 7f4391cdcb80 4 client.17109 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T11:27:05.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.028+0000 7f4391cdcb80 1 -- 172.21.15.100:0/3179177041 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c5306ca240 con 0x55c53076ec80 2024-05-10T11:27:05.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.028+0000 7f43849e4700 1 --2- 172.21.15.100:0/3179177041 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c53076ec80 0x55c5307f16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:05.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.029+0000 7f4368ff9700 10 client.17109 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:27:05.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.029+0000 7f43849e4700 1 --2- 172.21.15.100:0/3179177041 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c53076ec80 0x55c5307f16a0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4633a467380f8961 server_cookie=64e27cc60f5312f6 in_seq=0 out_seq=0 2024-05-10T11:27:05.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.029+0000 7f4368ff9700 1 -- 172.21.15.100:0/3179177041 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c5306ca240 con 0x55c53076ec80 2024-05-10T11:27:05.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.029+0000 7f4368ff9700 10 client.17109 handle_command_reply: tid=0 2024-05-10T11:27:05.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.029+0000 7f4391cdcb80 1 client.17109 shutdown 2024-05-10T11:27:05.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.029+0000 7f4391cdcb80 1 -- 172.21.15.100:0/3179177041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c530696100 msgr2=0x55c53077c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:05.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.029+0000 7f4391cdcb80 1 --2- 172.21.15.100:0/3179177041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c530696100 0x55c53077c050 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f437c005cb0 tx=0x7f437c0aa920 comp rx=0 tx=0).stop 2024-05-10T11:27:05.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.030+0000 7f4391cdcb80 1 -- 172.21.15.100:0/3179177041 shutdown_connections 2024-05-10T11:27:05.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.030+0000 7f4391cdcb80 1 --2- 172.21.15.100:0/3179177041 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c53076ec80 0x55c5307f16a0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:05.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.030+0000 7f4391cdcb80 1 --2- 172.21.15.100:0/3179177041 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c53076adf0 0x55c530778550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:05.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.030+0000 7f4391cdcb80 1 --2- 172.21.15.100:0/3179177041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c530696100 0x55c53077c050 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:05.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.030+0000 7f4391cdcb80 1 --2- 172.21.15.100:0/3179177041 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c53076a4d0 0x55c53077c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:05.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.030+0000 7f4391cdcb80 1 -- 172.21.15.100:0/3179177041 >> 172.21.15.100:0/3179177041 conn(0x55c53075fab0 msgr2=0x55c530769b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:27:05.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.030+0000 7f4368ff9700 0 client.17109 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:27:05.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.030+0000 7f4391cdcb80 1 -- 172.21.15.100:0/3179177041 shutdown_connections 2024-05-10T11:27:05.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.030+0000 7f4391cdcb80 1 -- 172.21.15.100:0/3179177041 wait complete. 2024-05-10T11:27:05.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.030+0000 7f4391cdcb80 20 client.17109 trim_cache size 0 max 16384 2024-05-10T11:27:05.034 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:27:05.035 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-05-10T11:27:05.035 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-05-10T11:27:05.035 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:27:05.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.030+0000 7f4387449700 1 -- 172.21.15.100:0/1929842675 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f436c0b2900 msgr2=0x7f436c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:05.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.030+0000 7f4387449700 1 --2- 172.21.15.100:0/1929842675 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f436c0b2900 0x7f436c0b4dc0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4380103060 tx=0x7f43700073d0 comp rx=0 tx=0).stop 2024-05-10T11:27:05.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.030+0000 7f4387449700 1 -- 172.21.15.100:0/1929842675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43801037b0 msgr2=0x7f438019f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:05.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.030+0000 7f4387449700 1 --2- 172.21.15.100:0/1929842675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43801037b0 0x7f438019f6c0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f437c000c00 tx=0x7f437c00f460 comp rx=0 tx=0).stop 2024-05-10T11:27:05.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.030+0000 7f4387449700 1 -- 172.21.15.100:0/1929842675 shutdown_connections 2024-05-10T11:27:05.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.031+0000 7f4387449700 1 --2- 172.21.15.100:0/1929842675 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f436c0b2900 0x7f436c0b4dc0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:05.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.031+0000 7f4387449700 1 --2- 172.21.15.100:0/1929842675 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4380108880 0x7f4380199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:05.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.031+0000 7f4387449700 1 --2- 172.21.15.100:0/1929842675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43801037b0 0x7f438019f6c0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:05.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.031+0000 7f4387449700 1 --2- 172.21.15.100:0/1929842675 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4380104160 0x7f438019fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:05.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.031+0000 7f4387449700 1 -- 172.21.15.100:0/1929842675 >> 172.21.15.100:0/1929842675 conn(0x7f438007a530 msgr2=0x7f43801017b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:27:05.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.031+0000 7f4387449700 1 -- 172.21.15.100:0/1929842675 shutdown_connections 2024-05-10T11:27:05.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:05.031+0000 7f4387449700 1 -- 172.21.15.100:0/1929842675 wait complete. 2024-05-10T11:27:05.720 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:27:05 smithi100 ceph-mon[30296]: pgmap v681: 129 pgs: 129 active+clean; 90 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 1.4 MiB/s wr, 150 op/s 2024-05-10T11:27:05.752 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T11:27:05.752 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T11:27:05.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:27:05 smithi178 ceph-mon[33359]: pgmap v681: 129 pgs: 129 active+clean; 90 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 1.4 MiB/s wr, 150 op/s 2024-05-10T11:27:05.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:27:05 smithi188 ceph-mon[34075]: pgmap v681: 129 pgs: 129 active+clean; 90 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 1.4 MiB/s wr, 150 op/s 2024-05-10T11:27:05.977 INFO:tasks.workunit.client.0.smithi100.stdout: 1 217313 2.75 MB/sec execute 300 sec latency 15.957 ms 2024-05-10T11:27:06.035 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:27:06.977 INFO:tasks.workunit.client.0.smithi100.stdout: 1 217846 2.75 MB/sec execute 301 sec latency 15.721 ms 2024-05-10T11:27:07.710 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:27:07 smithi100 ceph-mon[30296]: pgmap v682: 129 pgs: 129 active+clean; 91 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 1.4 MiB/s wr, 182 op/s 2024-05-10T11:27:07.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:27:07 smithi178 ceph-mon[33359]: pgmap v682: 129 pgs: 129 active+clean; 91 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 1.4 MiB/s wr, 182 op/s 2024-05-10T11:27:07.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:27:07 smithi188 ceph-mon[34075]: pgmap v682: 129 pgs: 129 active+clean; 91 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 1.4 MiB/s wr, 182 op/s 2024-05-10T11:27:07.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:07.967+0000 7ff22ea65700 1 -- 172.21.15.100:0/1762605569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff228102fd0 msgr2=0x7ff228107020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:07.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:07.967+0000 7ff22ea65700 1 --2- 172.21.15.100:0/1762605569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff228102fd0 0x7ff228107020 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7ff2140099c0 tx=0x7ff214009cd0 comp rx=0 tx=0).stop 2024-05-10T11:27:07.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:07.967+0000 7ff22ea65700 1 -- 172.21.15.100:0/1762605569 shutdown_connections 2024-05-10T11:27:07.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:07.967+0000 7ff22ea65700 1 --2- 172.21.15.100:0/1762605569 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff228107700 0x7ff22810f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:07.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:07.967+0000 7ff22ea65700 1 --2- 172.21.15.100:0/1762605569 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff228102620 0x7ff228102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:07.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:07.967+0000 7ff22ea65700 1 --2- 172.21.15.100:0/1762605569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff228102fd0 0x7ff228107020 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:07.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:07.967+0000 7ff22ea65700 1 -- 172.21.15.100:0/1762605569 >> 172.21.15.100:0/1762605569 conn(0x7ff228079440 msgr2=0x7ff228079850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:27:07.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:07.967+0000 7ff22ea65700 1 -- 172.21.15.100:0/1762605569 shutdown_connections 2024-05-10T11:27:07.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:07.967+0000 7ff22ea65700 1 -- 172.21.15.100:0/1762605569 wait complete. 2024-05-10T11:27:07.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:07.968+0000 7ff22ea65700 1 Processor -- start 2024-05-10T11:27:07.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:07.968+0000 7ff22ea65700 1 -- start start 2024-05-10T11:27:07.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:07.968+0000 7ff22ea65700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff228102620 0x7ff22819e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:07.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:07.968+0000 7ff22ea65700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff228102fd0 0x7ff22819ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:07.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:07.968+0000 7ff22ea65700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff228107700 0x7ff228198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:07.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:07.968+0000 7ff22ea65700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff228112470 con 0x7ff228102620 2024-05-10T11:27:07.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:07.968+0000 7ff22ea65700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff2281122f0 con 0x7ff228102fd0 2024-05-10T11:27:07.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:07.968+0000 7ff22ea65700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff2281125f0 con 0x7ff228107700 2024-05-10T11:27:07.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:07.968+0000 7ff22c801700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff228102620 0x7ff22819e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:07.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:07.968+0000 7ff22c801700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff228102620 0x7ff22819e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60858/0 (socket says 172.21.15.100:60858) 2024-05-10T11:27:07.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:07.968+0000 7ff22c801700 1 -- 172.21.15.100:0/3291931452 learned_addr learned my addr 172.21.15.100:0/3291931452 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:27:07.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:07.968+0000 7ff22d002700 1 --2- 172.21.15.100:0/3291931452 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff228107700 0x7ff228198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:07.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:07.968+0000 7ff227fff700 1 --2- 172.21.15.100:0/3291931452 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff228102fd0 0x7ff22819ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:07.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:07.969+0000 7ff22c801700 1 -- 172.21.15.100:0/3291931452 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff228107700 msgr2=0x7ff228198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:07.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:07.969+0000 7ff22c801700 1 --2- 172.21.15.100:0/3291931452 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff228107700 0x7ff228198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:07.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:07.969+0000 7ff22c801700 1 -- 172.21.15.100:0/3291931452 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff228102fd0 msgr2=0x7ff22819ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:07.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:07.969+0000 7ff22c801700 1 --2- 172.21.15.100:0/3291931452 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff228102fd0 0x7ff22819ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:07.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:07.969+0000 7ff22c801700 1 -- 172.21.15.100:0/3291931452 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff228199010 con 0x7ff228102620 2024-05-10T11:27:07.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:07.969+0000 7ff22d002700 1 --2- 172.21.15.100:0/3291931452 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff228107700 0x7ff228198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:27:07.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:07.969+0000 7ff22c801700 1 --2- 172.21.15.100:0/3291931452 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff228102620 0x7ff22819e6b0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7ff21c00d950 tx=0x7ff21c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:27:07.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:07.969+0000 7ff225ffb700 1 -- 172.21.15.100:0/3291931452 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff21c016070 con 0x7ff228102620 2024-05-10T11:27:07.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:07.969+0000 7ff22ea65700 1 -- 172.21.15.100:0/3291931452 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff228199300 con 0x7ff228102620 2024-05-10T11:27:07.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:07.969+0000 7ff225ffb700 1 -- 172.21.15.100:0/3291931452 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff21c00fd20 con 0x7ff228102620 2024-05-10T11:27:07.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:07.969+0000 7ff225ffb700 1 -- 172.21.15.100:0/3291931452 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff21c00e3b0 con 0x7ff228102620 2024-05-10T11:27:07.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:07.969+0000 7ff22ea65700 1 -- 172.21.15.100:0/3291931452 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2281a5510 con 0x7ff228102620 2024-05-10T11:27:07.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:07.970+0000 7ff22ea65700 1 -- 172.21.15.100:0/3291931452 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff228103b00 con 0x7ff228102620 2024-05-10T11:27:07.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:07.972+0000 7ff225ffb700 1 -- 172.21.15.100:0/3291931452 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7ff21c019070 con 0x7ff228102620 2024-05-10T11:27:07.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:07.972+0000 7ff225ffb700 1 --2- 172.21.15.100:0/3291931452 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff2180b2880 0x7ff2180b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:07.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:07.972+0000 7ff227fff700 1 --2- 172.21.15.100:0/3291931452 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff2180b2880 0x7ff2180b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:07.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:07.973+0000 7ff225ffb700 1 -- 172.21.15.100:0/3291931452 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7ff21c0e2030 con 0x7ff228102620 2024-05-10T11:27:07.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:07.973+0000 7ff227fff700 1 --2- 172.21.15.100:0/3291931452 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff2180b2880 0x7ff2180b4d40 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff214000c00 tx=0x7ff214005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:27:07.976 INFO:tasks.workunit.client.0.smithi100.stdout: 1 218476 2.74 MB/sec execute 302 sec latency 14.740 ms 2024-05-10T11:27:07.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:07.977+0000 7ff225ffb700 1 -- 172.21.15.100:0/3291931452 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7ff21c0a99f0 con 0x7ff228102620 2024-05-10T11:27:08.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:08.292+0000 7ff22ea65700 1 -- 172.21.15.100:0/3291931452 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff22802e510 con 0x7ff228102620 2024-05-10T11:27:08.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:08.293+0000 7ff225ffb700 1 -- 172.21.15.100:0/3291931452 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11894 (secure 0 0 0) 0x7ff21c0a92f0 con 0x7ff228102620 2024-05-10T11:27:08.293 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:27:08.293 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2072225012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2072225012},{"type":"v1","addr":"172.21.15.100:6835","nonce":2072225012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/401939540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":401939540},{"type":"v1","addr":"172.21.15.100:6837","nonce":401939540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14571,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/3946311025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3946311025},{"type":"v1","addr":"172.21.15.100:6839","nonce":3946311025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3736371660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3736371660},{"type":"v1","addr":"172.21.15.100:6841","nonce":3736371660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/2741810791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2741810791},{"type":"v1","addr":"172.21.15.178:6833","nonce":2741810791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1704891041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1704891041},{"type":"v1","addr":"172.21.15.188:6833","nonce":1704891041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T11:14:27.242452+0000","modified":"2024-05-10T11:23:19.557113+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24439,"mds_1":24437,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/2124076312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2124076312},{"type":"v1","addr":"172.21.15.178:6835","nonce":2124076312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/3224358425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3224358425},{"type":"v1","addr":"172.21.15.178:6837","nonce":3224358425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/515547690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":515547690},{"type":"v1","addr":"172.21.15.188:6835","nonce":515547690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/1943342620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1943342620},{"type":"v1","addr":"172.21.15.188:6837","nonce":1943342620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.178:6839/3760985404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3760985404},{"type":"v1","addr":"172.21.15.178:6839","nonce":3760985404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.188:6839/3194917070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":3194917070},{"type":"v1","addr":"172.21.15.188:6839","nonce":3194917070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24439,"qdb_cluster":[24439,24427,24437]},"id":1}]} 2024-05-10T11:27:08.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:08.295+0000 7ff22ea65700 1 -- 172.21.15.100:0/3291931452 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff2180b2880 msgr2=0x7ff2180b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:08.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:08.295+0000 7ff22ea65700 1 --2- 172.21.15.100:0/3291931452 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff2180b2880 0x7ff2180b4d40 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff214000c00 tx=0x7ff214005fb0 comp rx=0 tx=0).stop 2024-05-10T11:27:08.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:08.295+0000 7ff22ea65700 1 -- 172.21.15.100:0/3291931452 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff228102620 msgr2=0x7ff22819e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:08.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:08.295+0000 7ff22ea65700 1 --2- 172.21.15.100:0/3291931452 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff228102620 0x7ff22819e6b0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7ff21c00d950 tx=0x7ff21c00dc60 comp rx=0 tx=0).stop 2024-05-10T11:27:08.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:08.295+0000 7ff22ea65700 1 -- 172.21.15.100:0/3291931452 shutdown_connections 2024-05-10T11:27:08.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:08.295+0000 7ff22ea65700 1 --2- 172.21.15.100:0/3291931452 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff2180b2880 0x7ff2180b4d40 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:08.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:08.295+0000 7ff22ea65700 1 --2- 172.21.15.100:0/3291931452 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff228107700 0x7ff228198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:08.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:08.295+0000 7ff22ea65700 1 --2- 172.21.15.100:0/3291931452 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff228102620 0x7ff22819e6b0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:08.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:08.295+0000 7ff22ea65700 1 --2- 172.21.15.100:0/3291931452 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff228102fd0 0x7ff22819ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:08.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:08.295+0000 7ff22ea65700 1 -- 172.21.15.100:0/3291931452 >> 172.21.15.100:0/3291931452 conn(0x7ff228079440 msgr2=0x7ff228100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:27:08.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:08.296+0000 7ff22ea65700 1 -- 172.21.15.100:0/3291931452 shutdown_connections 2024-05-10T11:27:08.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:08.296+0000 7ff22ea65700 1 -- 172.21.15.100:0/3291931452 wait complete. 2024-05-10T11:27:08.297 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 31 2024-05-10T11:27:08.465 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.463+0000 7fa5af314640 1 -- 172.21.15.100:0/4056759428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5a8103490 msgr2=0x7fa5a8105880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:08.465 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.463+0000 7fa5af314640 1 --2- 172.21.15.100:0/4056759428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5a8103490 0x7fa5a8105880 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fa5a4009900 tx=0x7fa5a402a410 comp rx=0 tx=0).stop 2024-05-10T11:27:08.465 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.463+0000 7fa5af314640 1 -- 172.21.15.100:0/4056759428 shutdown_connections 2024-05-10T11:27:08.465 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.463+0000 7fa5af314640 1 --2- 172.21.15.100:0/4056759428 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5a81086f0 0x7fa5a810ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:08.465 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.463+0000 7fa5af314640 1 --2- 172.21.15.100:0/4056759428 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa5a8105dc0 0x7fa5a81081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:08.465 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.463+0000 7fa5af314640 1 --2- 172.21.15.100:0/4056759428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5a8103490 0x7fa5a8105880 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:08.466 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.463+0000 7fa5af314640 1 -- 172.21.15.100:0/4056759428 >> 172.21.15.100:0/4056759428 conn(0x7fa5a80fce80 msgr2=0x7fa5a80ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:27:08.466 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.463+0000 7fa5af314640 1 -- 172.21.15.100:0/4056759428 shutdown_connections 2024-05-10T11:27:08.466 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.463+0000 7fa5af314640 1 -- 172.21.15.100:0/4056759428 wait complete. 2024-05-10T11:27:08.466 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.464+0000 7fa5af314640 1 Processor -- start 2024-05-10T11:27:08.466 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.464+0000 7fa5af314640 1 -- start start 2024-05-10T11:27:08.466 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.464+0000 7fa5af314640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa5a8103490 0x7fa5a8198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:08.466 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.464+0000 7fa5af314640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5a8105dc0 0x7fa5a8199020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:08.466 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.464+0000 7fa5af314640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5a81086f0 0x7fa5a81a0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:08.466 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.464+0000 7fa5af314640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a810e410 con 0x7fa5a8105dc0 2024-05-10T11:27:08.466 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.464+0000 7fa5af314640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a810e290 con 0x7fa5a8103490 2024-05-10T11:27:08.466 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.464+0000 7fa5af314640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a810e590 con 0x7fa5a81086f0 2024-05-10T11:27:08.466 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.464+0000 7fa5ac8af640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5a8105dc0 0x7fa5a8199020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:08.466 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.464+0000 7fa5ac8af640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5a8105dc0 0x7fa5a8199020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60876/0 (socket says 172.21.15.100:60876) 2024-05-10T11:27:08.466 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.464+0000 7fa5ac8af640 1 -- 172.21.15.100:0/847961272 learned_addr learned my addr 172.21.15.100:0/847961272 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:27:08.467 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.464+0000 7fa5ad8b1640 1 --2- 172.21.15.100:0/847961272 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5a81086f0 0x7fa5a81a0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:08.467 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.464+0000 7fa5ad0b0640 1 --2- 172.21.15.100:0/847961272 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa5a8103490 0x7fa5a8198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:08.467 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.464+0000 7fa5ac8af640 1 -- 172.21.15.100:0/847961272 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5a81086f0 msgr2=0x7fa5a81a0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:08.467 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.464+0000 7fa5ac8af640 1 --2- 172.21.15.100:0/847961272 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5a81086f0 0x7fa5a81a0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:08.467 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.464+0000 7fa5ac8af640 1 -- 172.21.15.100:0/847961272 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa5a8103490 msgr2=0x7fa5a8198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:08.467 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.464+0000 7fa5ac8af640 1 --2- 172.21.15.100:0/847961272 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa5a8103490 0x7fa5a8198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:08.467 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.464+0000 7fa5ac8af640 1 -- 172.21.15.100:0/847961272 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5a81a0700 con 0x7fa5a8105dc0 2024-05-10T11:27:08.467 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.464+0000 7fa5ad8b1640 1 --2- 172.21.15.100:0/847961272 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5a81086f0 0x7fa5a81a0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:27:08.467 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.465+0000 7fa5ac8af640 1 --2- 172.21.15.100:0/847961272 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5a8105dc0 0x7fa5a8199020 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fa59800e9c0 tx=0x7fa59800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:27:08.467 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.465+0000 7fa59e7fc640 1 -- 172.21.15.100:0/847961272 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59800cd80 con 0x7fa5a8105dc0 2024-05-10T11:27:08.467 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.465+0000 7fa59e7fc640 1 -- 172.21.15.100:0/847961272 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa5980054b0 con 0x7fa5a8105dc0 2024-05-10T11:27:08.467 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.465+0000 7fa5af314640 1 -- 172.21.15.100:0/847961272 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5a81a09f0 con 0x7fa5a8105dc0 2024-05-10T11:27:08.467 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.465+0000 7fa59e7fc640 1 -- 172.21.15.100:0/847961272 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa598010430 con 0x7fa5a8105dc0 2024-05-10T11:27:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.465+0000 7fa5af314640 1 -- 172.21.15.100:0/847961272 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5a81a0f90 con 0x7fa5a8105dc0 2024-05-10T11:27:08.469 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.468+0000 7fa5b202d740 1 Processor -- start 2024-05-10T11:27:08.469 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.468+0000 7fa5b202d740 1 -- start start 2024-05-10T11:27:08.469 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.468+0000 7fa5b202d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9b1e4edb0 0x55a9b1e4f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.468+0000 7fa5b202d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9b1e4f7a0 0x55a9b1e542a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.468+0000 7fa5b202d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a9b1e54910 0x55a9b1e56da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.468+0000 7fa5b202d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55a9b1db35d0 con 0x55a9b1e4edb0 2024-05-10T11:27:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.468+0000 7fa5b202d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a9b1d6c890 con 0x55a9b1e4f7a0 2024-05-10T11:27:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.468+0000 7fa5b202d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55a9b1daf590 con 0x55a9b1e54910 2024-05-10T11:27:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.468+0000 7fa5ad8b1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9b1e4edb0 0x55a9b1e4f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.468+0000 7fa5ad8b1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9b1e4edb0 0x55a9b1e4f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60886/0 (socket says 172.21.15.100:60886) 2024-05-10T11:27:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.468+0000 7fa59e7fc640 1 -- 172.21.15.100:0/847961272 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fa598004f20 con 0x7fa5a8105dc0 2024-05-10T11:27:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.468+0000 7fa5ad8b1640 1 -- 172.21.15.100:0/352846622 learned_addr learned my addr 172.21.15.100:0/352846622 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:27:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.468+0000 7fa5ad0b0640 1 --2- 172.21.15.100:0/352846622 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9b1e4f7a0 0x55a9b1e542a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.468+0000 7fa5ad8b1640 1 -- 172.21.15.100:0/352846622 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a9b1e54910 msgr2=0x55a9b1e56da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.468+0000 7fa5ad8b1640 1 --2- 172.21.15.100:0/352846622 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a9b1e54910 0x55a9b1e56da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.468+0000 7fa5ad8b1640 1 -- 172.21.15.100:0/352846622 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9b1e4f7a0 msgr2=0x55a9b1e542a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.468+0000 7fa5ad8b1640 1 --2- 172.21.15.100:0/352846622 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9b1e4f7a0 0x55a9b1e542a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.468+0000 7fa5ad8b1640 1 -- 172.21.15.100:0/352846622 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9b1ac5210 con 0x55a9b1e4edb0 2024-05-10T11:27:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.469+0000 7fa5ad8b1640 1 --2- 172.21.15.100:0/352846622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9b1e4edb0 0x55a9b1e4f1d0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fa5a00117e0 tx=0x7fa5a0011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c6dfa9b9f3bff server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:27:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.469+0000 7fa59e7fc640 1 --2- 172.21.15.100:0/847961272 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa5940b2920 0x7fa5940b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.469+0000 7fa5ad0b0640 1 --2- 172.21.15.100:0/847961272 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa5940b2920 0x7fa5940b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.469+0000 7fa5937fe640 1 -- 172.21.15.100:0/352846622 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a0011310 con 0x55a9b1e4edb0 2024-05-10T11:27:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.469+0000 7fa5937fe640 1 -- 172.21.15.100:0/352846622 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa5a0003e00 con 0x55a9b1e4edb0 2024-05-10T11:27:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.469+0000 7fa5937fe640 1 -- 172.21.15.100:0/352846622 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a0004430 con 0x55a9b1e4edb0 2024-05-10T11:27:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.469+0000 7fa59e7fc640 1 -- 172.21.15.100:0/847961272 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fa5980e2a80 con 0x7fa5a8105dc0 2024-05-10T11:27:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.470+0000 7fa5b202d740 1 -- 172.21.15.100:0/352846622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9b1e4edb0 msgr2=0x55a9b1e4f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:08.476 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.470+0000 7fa5b202d740 1 --2- 172.21.15.100:0/352846622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9b1e4edb0 0x55a9b1e4f1d0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fa5a00117e0 tx=0x7fa5a0011ca0 comp rx=0 tx=0).stop 2024-05-10T11:27:08.476 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.470+0000 7fa5ad0b0640 1 --2- 172.21.15.100:0/847961272 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa5940b2920 0x7fa5940b4de0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa5a4006010 tx=0x7fa5a4031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:27:08.476 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.470+0000 7fa5b202d740 1 -- 172.21.15.100:0/352846622 shutdown_connections 2024-05-10T11:27:08.476 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.470+0000 7fa5b202d740 1 --2- 172.21.15.100:0/352846622 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a9b1e54910 0x55a9b1e56da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:08.476 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.470+0000 7fa5b202d740 1 --2- 172.21.15.100:0/352846622 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9b1e4f7a0 0x55a9b1e542a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:08.476 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.470+0000 7fa5b202d740 1 --2- 172.21.15.100:0/352846622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9b1e4edb0 0x55a9b1e4f1d0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:08.476 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.470+0000 7fa5b202d740 1 -- 172.21.15.100:0/352846622 >> 172.21.15.100:0/352846622 conn(0x55a9b1e4a420 msgr2=0x55a9b1e4a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:27:08.476 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.470+0000 7fa5b202d740 1 -- 172.21.15.100:0/352846622 shutdown_connections 2024-05-10T11:27:08.476 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.470+0000 7fa5b202d740 1 -- 172.21.15.100:0/352846622 wait complete. 2024-05-10T11:27:08.476 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.470+0000 7fa5b202d740 1 -- start start 2024-05-10T11:27:08.476 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.470+0000 7fa5b202d740 1 Processor -- start 2024-05-10T11:27:08.476 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.471+0000 7fa5b202d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9b1e4edb0 0x55a9b1e59030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:08.477 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.471+0000 7fa5b202d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a9b1e4f7a0 0x55a9b1e59570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:08.477 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.471+0000 7fa5b202d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9b1e54910 0x55a9b1e605b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:08.477 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.471+0000 7fa5b202d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55a9b1e578c0 con 0x55a9b1e4edb0 2024-05-10T11:27:08.477 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.471+0000 7fa5b202d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a9b1d70370 con 0x55a9b1e54910 2024-05-10T11:27:08.477 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.471+0000 7fa5b202d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55a9b1e57a40 con 0x55a9b1e4f7a0 2024-05-10T11:27:08.477 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.471+0000 7fa5b202d740 10 client.0 _ll_register_callbacks cb 0x55a9b1ac4a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:27:08.477 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.471+0000 7fa5ad0b0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9b1e4edb0 0x55a9b1e59030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:08.477 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.471+0000 7fa5ad0b0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9b1e4edb0 0x55a9b1e59030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60898/0 (socket says 172.21.15.100:60898) 2024-05-10T11:27:08.477 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.471+0000 7fa5ad0b0640 1 -- 172.21.15.100:0/2030844967 learned_addr learned my addr 172.21.15.100:0/2030844967 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:27:08.477 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.471+0000 7fa5ac8af640 1 --2- 172.21.15.100:0/2030844967 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a9b1e4f7a0 0x55a9b1e59570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:08.477 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.471+0000 7fa5ad8b1640 1 --2- 172.21.15.100:0/2030844967 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9b1e54910 0x55a9b1e605b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:08.477 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.471+0000 7fa5ac8af640 1 -- 172.21.15.100:0/2030844967 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9b1e54910 msgr2=0x55a9b1e605b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:08.477 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.471+0000 7fa5ac8af640 1 --2- 172.21.15.100:0/2030844967 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9b1e54910 0x55a9b1e605b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:08.477 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.471+0000 7fa5ac8af640 1 -- 172.21.15.100:0/2030844967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9b1e4edb0 msgr2=0x55a9b1e59030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:08.477 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.471+0000 7fa5ac8af640 1 --2- 172.21.15.100:0/2030844967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9b1e4edb0 0x55a9b1e59030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:08.478 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.471+0000 7fa5ac8af640 1 -- 172.21.15.100:0/2030844967 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9b1b91620 con 0x55a9b1e4f7a0 2024-05-10T11:27:08.478 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.471+0000 7fa5ad0b0640 1 --2- 172.21.15.100:0/2030844967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9b1e4edb0 0x55a9b1e59030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:27:08.478 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.471+0000 7fa5ad8b1640 1 --2- 172.21.15.100:0/2030844967 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9b1e54910 0x55a9b1e605b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:27:08.478 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.472+0000 7fa5ac8af640 1 --2- 172.21.15.100:0/2030844967 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a9b1e4f7a0 0x55a9b1e59570 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa5980e0590 tx=0x7fa5980a9790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2316d046d15e6d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:27:08.478 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.472+0000 7fa590ff9640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:27:08.478 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.472+0000 7fa590ff9640 1 -- 172.21.15.100:0/2030844967 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5980a9d40 con 0x55a9b1e4f7a0 2024-05-10T11:27:08.478 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.472+0000 7fa5b202d740 1 -- 172.21.15.100:0/2030844967 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5a0003e00 con 0x55a9b1e4f7a0 2024-05-10T11:27:08.478 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.472+0000 7fa590ff9640 1 -- 172.21.15.100:0/2030844967 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fa5980c43c0 con 0x55a9b1e4f7a0 2024-05-10T11:27:08.478 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.472+0000 7fa590ff9640 1 -- 172.21.15.100:0/2030844967 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa5980ae0f0 con 0x55a9b1e4f7a0 2024-05-10T11:27:08.478 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.472+0000 7fa5b202d740 10 client.26596 fetch_fsmap learned FSMap version 31 2024-05-10T11:27:08.478 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.472+0000 7fa5b202d740 1 -- 172.21.15.100:0/2030844967 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a9b1e636b0 con 0x55a9b1e4f7a0 2024-05-10T11:27:08.478 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.472+0000 7fa590ff9640 1 -- 172.21.15.100:0/2030844967 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa5a0003e00 con 0x55a9b1e4f7a0 2024-05-10T11:27:08.478 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.472+0000 7fa590ff9640 1 -- 172.21.15.100:0/2030844967 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fa5980a8a10 con 0x55a9b1e4f7a0 2024-05-10T11:27:08.479 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.472+0000 7fa5b202d740 10 client.26596 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:27:08.479 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.472+0000 7fa5b202d740 10 client.26596 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:27:08.479 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.472+0000 7fa5b202d740 20 client.26596 populate_metadata read hostname 'smithi100' 2024-05-10T11:27:08.479 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.472+0000 7fa5b202d740 1 --2- 172.21.15.100:0/2030844967 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55a9b1e64d80 0x55a9b1e67170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:08.479 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.472+0000 7fa5b202d740 4 client.26596 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:27:08.479 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.472+0000 7fa5b202d740 1 -- 172.21.15.100:0/2030844967 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a9b1dc11e0 con 0x55a9b1e64d80 2024-05-10T11:27:08.479 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.473+0000 7fa5ad0b0640 1 --2- 172.21.15.100:0/2030844967 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55a9b1e64d80 0x55a9b1e67170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:08.479 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.473+0000 7fa590ff9640 10 client.26596 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:27:08.479 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.473+0000 7fa5ad0b0640 1 --2- 172.21.15.100:0/2030844967 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55a9b1e64d80 0x55a9b1e67170 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c9d4623205b844d server_cookie=df609d8e0290ec50 in_seq=0 out_seq=0 2024-05-10T11:27:08.479 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.474+0000 7fa590ff9640 1 -- 172.21.15.100:0/2030844967 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a9b1dc11e0 con 0x55a9b1e64d80 2024-05-10T11:27:08.479 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.474+0000 7fa590ff9640 10 client.26596 handle_command_reply: tid=0 2024-05-10T11:27:08.479 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.474+0000 7fa5b202d740 1 client.26596 shutdown 2024-05-10T11:27:08.479 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.474+0000 7fa5b202d740 1 -- 172.21.15.100:0/2030844967 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a9b1e4f7a0 msgr2=0x55a9b1e59570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:08.479 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.474+0000 7fa5b202d740 1 --2- 172.21.15.100:0/2030844967 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a9b1e4f7a0 0x55a9b1e59570 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa5980e0590 tx=0x7fa5980a9790 comp rx=0 tx=0).stop 2024-05-10T11:27:08.480 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.474+0000 7fa5b202d740 1 -- 172.21.15.100:0/2030844967 shutdown_connections 2024-05-10T11:27:08.480 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.474+0000 7fa5b202d740 1 --2- 172.21.15.100:0/2030844967 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55a9b1e64d80 0x55a9b1e67170 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:08.480 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.474+0000 7fa5b202d740 1 --2- 172.21.15.100:0/2030844967 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9b1e54910 0x55a9b1e605b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:08.480 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.474+0000 7fa590ff9640 0 client.26596 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:27:08.480 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.474+0000 7fa5b202d740 1 --2- 172.21.15.100:0/2030844967 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a9b1e4f7a0 0x55a9b1e59570 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:08.480 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.474+0000 7fa5b202d740 1 --2- 172.21.15.100:0/2030844967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9b1e4edb0 0x55a9b1e59030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:08.480 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.474+0000 7fa5b202d740 1 -- 172.21.15.100:0/2030844967 >> 172.21.15.100:0/2030844967 conn(0x55a9b1e4a420 msgr2=0x55a9b1e49580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:27:08.480 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.474+0000 7fa5b202d740 1 -- 172.21.15.100:0/2030844967 shutdown_connections 2024-05-10T11:27:08.480 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.474+0000 7fa5b202d740 1 -- 172.21.15.100:0/2030844967 wait complete. 2024-05-10T11:27:08.480 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.474+0000 7fa5b202d740 20 client.26596 trim_cache size 0 max 16384 2024-05-10T11:27:08.493 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.491+0000 7fa5b202d740 1 Processor -- start 2024-05-10T11:27:08.493 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.491+0000 7fa5b202d740 1 -- start start 2024-05-10T11:27:08.493 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.492+0000 7fa5b202d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9b1e4edb0 0x55a9b1e61350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:08.493 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.492+0000 7fa5b202d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a9b1e4f7a0 0x55a9b1e60570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:08.493 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.492+0000 7fa5b202d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9b1e54910 0x55a9b1e4d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:08.493 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.492+0000 7fa5b202d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a81adbd0 con 0x55a9b1e54910 2024-05-10T11:27:08.493 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.492+0000 7fa5b202d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a81ab4a0 con 0x55a9b1e4edb0 2024-05-10T11:27:08.493 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.492+0000 7fa5b202d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a81a92f0 con 0x55a9b1e4f7a0 2024-05-10T11:27:08.494 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.492+0000 7fa5ad0b0640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9b1e4edb0 0x55a9b1e61350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:08.494 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.492+0000 7fa5ad8b1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9b1e54910 0x55a9b1e4d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:08.494 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.492+0000 7fa5ac8af640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a9b1e4f7a0 0x55a9b1e60570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:08.494 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.492+0000 7fa5ad8b1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9b1e54910 0x55a9b1e4d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60904/0 (socket says 172.21.15.100:60904) 2024-05-10T11:27:08.494 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.492+0000 7fa5ad8b1640 1 -- 172.21.15.100:0/133451940 learned_addr learned my addr 172.21.15.100:0/133451940 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:27:08.494 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.492+0000 7fa5ad0b0640 1 -- 172.21.15.100:0/133451940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a9b1e4f7a0 msgr2=0x55a9b1e60570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:08.494 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.492+0000 7fa5ad0b0640 1 --2- 172.21.15.100:0/133451940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a9b1e4f7a0 0x55a9b1e60570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:08.494 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.492+0000 7fa5ad0b0640 1 -- 172.21.15.100:0/133451940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9b1e54910 msgr2=0x55a9b1e4d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:08.494 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.492+0000 7fa5ad0b0640 1 --2- 172.21.15.100:0/133451940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9b1e54910 0x55a9b1e4d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:08.494 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.492+0000 7fa5ad0b0640 1 -- 172.21.15.100:0/133451940 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9b1e4e9f0 con 0x55a9b1e4edb0 2024-05-10T11:27:08.495 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.492+0000 7fa5ad8b1640 1 --2- 172.21.15.100:0/133451940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9b1e54910 0x55a9b1e4d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:27:08.495 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.492+0000 7fa5ad0b0640 1 --2- 172.21.15.100:0/133451940 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9b1e4edb0 0x55a9b1e61350 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x55a9b1e679e0 tx=0x7fa5a4040000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac9ea82c95ee209e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:27:08.495 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.493+0000 7fa592ffd640 1 -- 172.21.15.100:0/133451940 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a4035040 con 0x55a9b1e4edb0 2024-05-10T11:27:08.495 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.493+0000 7fa592ffd640 1 -- 172.21.15.100:0/133451940 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa5a4004400 con 0x55a9b1e4edb0 2024-05-10T11:27:08.495 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.493+0000 7fa592ffd640 1 -- 172.21.15.100:0/133451940 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a4003ae0 con 0x55a9b1e4edb0 2024-05-10T11:27:08.498 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.493+0000 7fa5b202d740 1 -- 172.21.15.100:0/133451940 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9b1e4edb0 msgr2=0x55a9b1e61350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:08.498 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.493+0000 7fa5b202d740 1 --2- 172.21.15.100:0/133451940 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9b1e4edb0 0x55a9b1e61350 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x55a9b1e679e0 tx=0x7fa5a4040000 comp rx=0 tx=0).stop 2024-05-10T11:27:08.499 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.493+0000 7fa5b202d740 1 -- 172.21.15.100:0/133451940 shutdown_connections 2024-05-10T11:27:08.499 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.493+0000 7fa5b202d740 1 --2- 172.21.15.100:0/133451940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9b1e54910 0x55a9b1e4d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:08.499 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.493+0000 7fa5b202d740 1 --2- 172.21.15.100:0/133451940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a9b1e4f7a0 0x55a9b1e60570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:08.499 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.493+0000 7fa5b202d740 1 --2- 172.21.15.100:0/133451940 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9b1e4edb0 0x55a9b1e61350 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:08.499 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.493+0000 7fa5b202d740 1 -- 172.21.15.100:0/133451940 >> 172.21.15.100:0/133451940 conn(0x55a9b1e4a420 msgr2=0x55a9b1e60c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:27:08.499 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.494+0000 7fa5b202d740 1 -- 172.21.15.100:0/133451940 shutdown_connections 2024-05-10T11:27:08.499 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.494+0000 7fa5b202d740 1 -- 172.21.15.100:0/133451940 wait complete. 2024-05-10T11:27:08.499 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.494+0000 7fa5b202d740 1 -- start start 2024-05-10T11:27:08.499 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.494+0000 7fa5b202d740 1 Processor -- start 2024-05-10T11:27:08.499 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.494+0000 7fa5b202d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9b1e4edb0 0x55a9b1e5d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:08.499 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.494+0000 7fa5b202d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a9b1e4f7a0 0x55a9b1e5dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:08.499 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.494+0000 7fa5b202d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9b1e54910 0x55a9b1e5ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:08.499 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.494+0000 7fa5b202d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a81a9960 con 0x55a9b1e4edb0 2024-05-10T11:27:08.499 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.494+0000 7fa5b202d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a81aadd0 con 0x55a9b1e54910 2024-05-10T11:27:08.500 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.494+0000 7fa5b202d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a81aa030 con 0x55a9b1e4f7a0 2024-05-10T11:27:08.500 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.494+0000 7fa5b202d740 10 client.0 _ll_register_callbacks cb 0x7fa598004920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:27:08.500 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.494+0000 7fa5ad0b0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9b1e4edb0 0x55a9b1e5d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:08.500 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.494+0000 7fa5ad0b0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9b1e4edb0 0x55a9b1e5d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60918/0 (socket says 172.21.15.100:60918) 2024-05-10T11:27:08.500 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.494+0000 7fa5ad0b0640 1 -- 172.21.15.100:0/1583250307 learned_addr learned my addr 172.21.15.100:0/1583250307 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:27:08.500 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.494+0000 7fa5ac8af640 1 --2- 172.21.15.100:0/1583250307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a9b1e4f7a0 0x55a9b1e5dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:08.500 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.495+0000 7fa5ad8b1640 1 --2- 172.21.15.100:0/1583250307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9b1e54910 0x55a9b1e5ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:08.500 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.495+0000 7fa5ad0b0640 1 -- 172.21.15.100:0/1583250307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a9b1e4f7a0 msgr2=0x55a9b1e5dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:08.500 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.495+0000 7fa5ad0b0640 1 --2- 172.21.15.100:0/1583250307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a9b1e4f7a0 0x55a9b1e5dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:08.500 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.495+0000 7fa5ad0b0640 1 -- 172.21.15.100:0/1583250307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9b1e54910 msgr2=0x55a9b1e5ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:08.500 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.495+0000 7fa5ad0b0640 1 --2- 172.21.15.100:0/1583250307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9b1e54910 0x55a9b1e5ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:08.500 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.495+0000 7fa5ad0b0640 1 -- 172.21.15.100:0/1583250307 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9b1e5f240 con 0x55a9b1e4edb0 2024-05-10T11:27:08.500 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.495+0000 7fa5ad8b1640 1 --2- 172.21.15.100:0/1583250307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9b1e54910 0x55a9b1e5ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:27:08.500 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.495+0000 7fa5ad0b0640 1 --2- 172.21.15.100:0/1583250307 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9b1e4edb0 0x55a9b1e5d700 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fa5a402a450 tx=0x7fa5a40045b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c1ba629c605e4a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:27:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.495+0000 7fa590ff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:27:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.496+0000 7fa590ff9640 1 -- 172.21.15.100:0/1583250307 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a4035070 con 0x55a9b1e4edb0 2024-05-10T11:27:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.496+0000 7fa590ff9640 1 -- 172.21.15.100:0/1583250307 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fa5a40528c0 con 0x55a9b1e4edb0 2024-05-10T11:27:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.496+0000 7fa5b202d740 1 -- 172.21.15.100:0/1583250307 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5a4004400 con 0x55a9b1e4edb0 2024-05-10T11:27:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.496+0000 7fa590ff9640 1 -- 172.21.15.100:0/1583250307 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa5a4052ac0 con 0x55a9b1e4edb0 2024-05-10T11:27:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.496+0000 7fa5b202d740 10 client.17133 fetch_fsmap learned FSMap version 31 2024-05-10T11:27:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.496+0000 7fa5b202d740 1 -- 172.21.15.100:0/1583250307 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a9b1e60550 con 0x55a9b1e4edb0 2024-05-10T11:27:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.497+0000 7fa590ff9640 1 -- 172.21.15.100:0/1583250307 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa5a4004400 con 0x55a9b1e4edb0 2024-05-10T11:27:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.497+0000 7fa590ff9640 1 -- 172.21.15.100:0/1583250307 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fa5a4064030 con 0x55a9b1e4edb0 2024-05-10T11:27:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.497+0000 7fa5b202d740 10 client.17133 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:27:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.497+0000 7fa5b202d740 10 client.17133 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:27:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.497+0000 7fa5b202d740 20 client.17133 populate_metadata read hostname 'smithi100' 2024-05-10T11:27:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.497+0000 7fa5b202d740 1 --2- 172.21.15.100:0/1583250307 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55a9b1ecf9d0 0x55a9b1ed1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.497+0000 7fa5b202d740 4 client.17133 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:27:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.497+0000 7fa5b202d740 1 -- 172.21.15.100:0/1583250307 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a9b1dbb050 con 0x55a9b1ecf9d0 2024-05-10T11:27:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.497+0000 7fa5ac8af640 1 --2- 172.21.15.100:0/1583250307 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55a9b1ecf9d0 0x55a9b1ed1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:08.502 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.497+0000 7fa590ff9640 10 client.17133 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:27:08.502 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.497+0000 7fa5ac8af640 1 --2- 172.21.15.100:0/1583250307 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55a9b1ecf9d0 0x55a9b1ed1df0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c855490f8d0d6af6 server_cookie=cdf20c52e4496cbb in_seq=0 out_seq=0 2024-05-10T11:27:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.516+0000 7fa590ff9640 1 -- 172.21.15.100:0/1583250307 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a9b1dbb050 con 0x55a9b1ecf9d0 2024-05-10T11:27:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.516+0000 7fa590ff9640 10 client.17133 handle_command_reply: tid=0 2024-05-10T11:27:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.516+0000 7fa5b202d740 1 client.17133 shutdown 2024-05-10T11:27:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.517+0000 7fa5b202d740 1 -- 172.21.15.100:0/1583250307 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9b1e4edb0 msgr2=0x55a9b1e5d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.517+0000 7fa5b202d740 1 --2- 172.21.15.100:0/1583250307 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9b1e4edb0 0x55a9b1e5d700 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fa5a402a450 tx=0x7fa5a40045b0 comp rx=0 tx=0).stop 2024-05-10T11:27:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.517+0000 7fa5b202d740 1 -- 172.21.15.100:0/1583250307 shutdown_connections 2024-05-10T11:27:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.517+0000 7fa5b202d740 1 --2- 172.21.15.100:0/1583250307 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55a9b1ecf9d0 0x55a9b1ed1df0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.517+0000 7fa5b202d740 1 --2- 172.21.15.100:0/1583250307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9b1e54910 0x55a9b1e5ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.517+0000 7fa5b202d740 1 --2- 172.21.15.100:0/1583250307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a9b1e4f7a0 0x55a9b1e5dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.517+0000 7fa5b202d740 1 --2- 172.21.15.100:0/1583250307 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9b1e4edb0 0x55a9b1e5d700 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.517+0000 7fa5b202d740 1 -- 172.21.15.100:0/1583250307 >> 172.21.15.100:0/1583250307 conn(0x55a9b1e4a420 msgr2=0x55a9b1e4d300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:27:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.517+0000 7fa590ff9640 0 client.17133 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:27:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.517+0000 7fa5b202d740 1 -- 172.21.15.100:0/1583250307 shutdown_connections 2024-05-10T11:27:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.517+0000 7fa5b202d740 1 -- 172.21.15.100:0/1583250307 wait complete. 2024-05-10T11:27:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.517+0000 7fa5b202d740 20 client.17133 trim_cache size 0 max 16384 2024-05-10T11:27:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:27:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:27:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:27:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:27:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.517+0000 7fa5af314640 1 -- 172.21.15.100:0/847961272 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa5940b2920 msgr2=0x7fa5940b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.517+0000 7fa5af314640 1 --2- 172.21.15.100:0/847961272 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa5940b2920 0x7fa5940b4de0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa5a4006010 tx=0x7fa5a4031400 comp rx=0 tx=0).stop 2024-05-10T11:27:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.517+0000 7fa5af314640 1 -- 172.21.15.100:0/847961272 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5a8105dc0 msgr2=0x7fa5a8199020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.517+0000 7fa5af314640 1 --2- 172.21.15.100:0/847961272 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5a8105dc0 0x7fa5a8199020 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fa59800e9c0 tx=0x7fa59800ee80 comp rx=0 tx=0).stop 2024-05-10T11:27:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.518+0000 7fa5af314640 1 -- 172.21.15.100:0/847961272 shutdown_connections 2024-05-10T11:27:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.518+0000 7fa5af314640 1 --2- 172.21.15.100:0/847961272 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa5940b2920 0x7fa5940b4de0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.518+0000 7fa5af314640 1 --2- 172.21.15.100:0/847961272 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5a81086f0 0x7fa5a81a0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.518+0000 7fa5af314640 1 --2- 172.21.15.100:0/847961272 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5a8105dc0 0x7fa5a8199020 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.518+0000 7fa5af314640 1 --2- 172.21.15.100:0/847961272 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa5a8103490 0x7fa5a8198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.518+0000 7fa5af314640 1 -- 172.21.15.100:0/847961272 >> 172.21.15.100:0/847961272 conn(0x7fa5a80fce80 msgr2=0x7fa5a8103f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:27:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.518+0000 7fa5af314640 1 -- 172.21.15.100:0/847961272 shutdown_connections 2024-05-10T11:27:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:08.518+0000 7fa5af314640 1 -- 172.21.15.100:0/847961272 wait complete. 2024-05-10T11:27:08.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:27:08 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/3291931452' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:27:08.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:27:08 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/3291931452' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:27:08.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:27:08 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/3291931452' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:27:08.977 INFO:tasks.workunit.client.0.smithi100.stdout: 1 218982 2.74 MB/sec execute 303 sec latency 60.159 ms 2024-05-10T11:27:09.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:27:09 smithi178 ceph-mon[33359]: pgmap v683: 129 pgs: 129 active+clean; 92 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 1.4 MiB/s wr, 138 op/s 2024-05-10T11:27:09.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:27:09 smithi188 ceph-mon[34075]: pgmap v683: 129 pgs: 129 active+clean; 92 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 1.4 MiB/s wr, 138 op/s 2024-05-10T11:27:09.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:27:09 smithi100 ceph-mon[30296]: pgmap v683: 129 pgs: 129 active+clean; 92 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 1.4 MiB/s wr, 138 op/s 2024-05-10T11:27:09.978 INFO:tasks.workunit.client.0.smithi100.stdout: 1 219700 2.75 MB/sec execute 304 sec latency 86.700 ms 2024-05-10T11:27:10.977 INFO:tasks.workunit.client.0.smithi100.stdout: 1 220037 2.76 MB/sec execute 305 sec latency 97.825 ms 2024-05-10T11:27:11.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:27:11 smithi178 ceph-mon[33359]: pgmap v684: 129 pgs: 129 active+clean; 91 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 1.2 MiB/s wr, 147 op/s 2024-05-10T11:27:11.759 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:27:11 smithi100 ceph-mon[30296]: pgmap v684: 129 pgs: 129 active+clean; 91 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 1.2 MiB/s wr, 147 op/s 2024-05-10T11:27:11.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:27:11 smithi188 ceph-mon[34075]: pgmap v684: 129 pgs: 129 active+clean; 91 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 1.2 MiB/s wr, 147 op/s 2024-05-10T11:27:11.789 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-10T11:27:11.977 INFO:tasks.workunit.client.0.smithi100.stdout: 1 220355 2.75 MB/sec execute 306 sec latency 33.438 ms 2024-05-10T11:27:12.050 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:27:12.977 INFO:tasks.workunit.client.0.smithi100.stdout: 1 220688 2.74 MB/sec execute 307 sec latency 12.311 ms 2024-05-10T11:27:13.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:27:13 smithi100 ceph-mon[30296]: pgmap v685: 129 pgs: 129 active+clean; 92 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 1.3 MiB/s wr, 147 op/s 2024-05-10T11:27:13.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:27:13 smithi178 ceph-mon[33359]: pgmap v685: 129 pgs: 129 active+clean; 92 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 1.3 MiB/s wr, 147 op/s 2024-05-10T11:27:13.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:27:13 smithi188 ceph-mon[34075]: pgmap v685: 129 pgs: 129 active+clean; 92 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 1.3 MiB/s wr, 147 op/s 2024-05-10T11:27:13.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.841+0000 7fd0c3f68700 1 -- 172.21.15.100:0/1551916033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0bc06c4c0 msgr2=0x7fd0bc10fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:13.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.841+0000 7fd0c3f68700 1 --2- 172.21.15.100:0/1551916033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0bc06c4c0 0x7fd0bc10fa40 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac009990 tx=0x7fd0ac009ca0 comp rx=0 tx=0).stop 2024-05-10T11:27:13.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.841+0000 7fd0c3f68700 1 -- 172.21.15.100:0/1551916033 shutdown_connections 2024-05-10T11:27:13.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.841+0000 7fd0c3f68700 1 --2- 172.21.15.100:0/1551916033 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd0bc10ff80 0x7fd0bc112370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:13.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.841+0000 7fd0c3f68700 1 --2- 172.21.15.100:0/1551916033 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0bc06bba0 0x7fd0bc06bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:13.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.841+0000 7fd0c3f68700 1 --2- 172.21.15.100:0/1551916033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0bc06c4c0 0x7fd0bc10fa40 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:13.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.841+0000 7fd0c3f68700 1 -- 172.21.15.100:0/1551916033 >> 172.21.15.100:0/1551916033 conn(0x7fd0bc0793e0 msgr2=0x7fd0bc0797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:27:13.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.841+0000 7fd0c3f68700 1 -- 172.21.15.100:0/1551916033 shutdown_connections 2024-05-10T11:27:13.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.841+0000 7fd0c3f68700 1 -- 172.21.15.100:0/1551916033 wait complete. 2024-05-10T11:27:13.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.842+0000 7fd0c3f68700 1 Processor -- start 2024-05-10T11:27:13.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.842+0000 7fd0c3f68700 1 -- start start 2024-05-10T11:27:13.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.842+0000 7fd0c3f68700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0bc06bba0 0x7fd0bc19bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:13.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.842+0000 7fd0c3f68700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0bc06c4c0 0x7fd0bc19c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:13.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.842+0000 7fd0c3f68700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd0bc10ff80 0x7fd0bc1a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:13.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.842+0000 7fd0c3f68700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd0bc1157b0 con 0x7fd0bc06bba0 2024-05-10T11:27:13.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.842+0000 7fd0c3f68700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd0bc115630 con 0x7fd0bc10ff80 2024-05-10T11:27:13.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.842+0000 7fd0c3f68700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd0bc115930 con 0x7fd0bc06c4c0 2024-05-10T11:27:13.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.842+0000 7fd0c1d04700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0bc06bba0 0x7fd0bc19bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:13.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.842+0000 7fd0c1d04700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0bc06bba0 0x7fd0bc19bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60936/0 (socket says 172.21.15.100:60936) 2024-05-10T11:27:13.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.842+0000 7fd0c1d04700 1 -- 172.21.15.100:0/138073762 learned_addr learned my addr 172.21.15.100:0/138073762 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:27:13.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.842+0000 7fd0c1503700 1 --2- 172.21.15.100:0/138073762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0bc06c4c0 0x7fd0bc19c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:13.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.842+0000 7fd0c2505700 1 --2- 172.21.15.100:0/138073762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd0bc10ff80 0x7fd0bc1a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:13.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.843+0000 7fd0c1d04700 1 -- 172.21.15.100:0/138073762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0bc06c4c0 msgr2=0x7fd0bc19c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:13.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.843+0000 7fd0c1d04700 1 --2- 172.21.15.100:0/138073762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0bc06c4c0 0x7fd0bc19c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:13.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.843+0000 7fd0c1d04700 1 -- 172.21.15.100:0/138073762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd0bc10ff80 msgr2=0x7fd0bc1a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:13.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.843+0000 7fd0c1d04700 1 --2- 172.21.15.100:0/138073762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd0bc10ff80 0x7fd0bc1a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:13.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.843+0000 7fd0c1d04700 1 -- 172.21.15.100:0/138073762 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0bc1a0d90 con 0x7fd0bc06bba0 2024-05-10T11:27:13.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.843+0000 7fd0c2505700 1 --2- 172.21.15.100:0/138073762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd0bc10ff80 0x7fd0bc1a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:27:13.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.843+0000 7fd0c1d04700 1 --2- 172.21.15.100:0/138073762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0bc06bba0 0x7fd0bc19bd80 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fd0b800e9a0 tx=0x7fd0b800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:27:13.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.843+0000 7fd0b2ffd700 1 -- 172.21.15.100:0/138073762 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b800cbd0 con 0x7fd0bc06bba0 2024-05-10T11:27:13.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.843+0000 7fd0c3f68700 1 -- 172.21.15.100:0/138073762 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0bc1a1080 con 0x7fd0bc06bba0 2024-05-10T11:27:13.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.843+0000 7fd0c3f68700 1 -- 172.21.15.100:0/138073762 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0bc1a8900 con 0x7fd0bc06bba0 2024-05-10T11:27:13.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.843+0000 7fd0b2ffd700 1 -- 172.21.15.100:0/138073762 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd0b800cd70 con 0x7fd0bc06bba0 2024-05-10T11:27:13.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.843+0000 7fd0b2ffd700 1 -- 172.21.15.100:0/138073762 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b8018680 con 0x7fd0bc06bba0 2024-05-10T11:27:13.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.845+0000 7fd0b2ffd700 1 -- 172.21.15.100:0/138073762 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fd0b8010430 con 0x7fd0bc06bba0 2024-05-10T11:27:13.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.846+0000 7fd0b2ffd700 1 --2- 172.21.15.100:0/138073762 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fd0a80b2900 0x7fd0a80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:13.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.846+0000 7fd0c1503700 1 --2- 172.21.15.100:0/138073762 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fd0a80b2900 0x7fd0a80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:13.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.846+0000 7fd0b2ffd700 1 -- 172.21.15.100:0/138073762 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fd0b80e2760 con 0x7fd0bc06bba0 2024-05-10T11:27:13.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.847+0000 7fd0c1503700 1 --2- 172.21.15.100:0/138073762 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fd0a80b2900 0x7fd0a80b4dc0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc103a00 tx=0x7fd0ac005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:27:13.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.847+0000 7fd0ce7fbb80 1 Processor -- start 2024-05-10T11:27:13.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.848+0000 7fd0ce7fbb80 1 -- start start 2024-05-10T11:27:13.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.848+0000 7fd0ce7fbb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55761da9c4d0 0x55761da9c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:13.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.848+0000 7fd0ce7fbb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55761da9cdf0 0x55761daa06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:13.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.848+0000 7fd0ce7fbb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55761daa0c80 0x55761daa1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:13.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.848+0000 7fd0ce7fbb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55761d9ee860 con 0x55761da9cdf0 2024-05-10T11:27:13.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.848+0000 7fd0ce7fbb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55761d98d7c0 con 0x55761daa0c80 2024-05-10T11:27:13.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.848+0000 7fd0ce7fbb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55761d9ea820 con 0x55761da9c4d0 2024-05-10T11:27:13.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.848+0000 7fd0c1503700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55761da9cdf0 0x55761daa06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:13.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.848+0000 7fd0c1503700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55761da9cdf0 0x55761daa06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60938/0 (socket says 172.21.15.100:60938) 2024-05-10T11:27:13.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.848+0000 7fd0c1503700 1 -- 172.21.15.100:0/3763607137 learned_addr learned my addr 172.21.15.100:0/3763607137 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:27:13.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.848+0000 7fd0c2505700 1 --2- 172.21.15.100:0/3763607137 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55761daa0c80 0x55761daa1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:13.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.848+0000 7fd0c1d04700 1 --2- 172.21.15.100:0/3763607137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55761da9c4d0 0x55761da9c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:13.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.848+0000 7fd0c1503700 1 -- 172.21.15.100:0/3763607137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55761da9c4d0 msgr2=0x55761da9c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:13.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.848+0000 7fd0c1503700 1 --2- 172.21.15.100:0/3763607137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55761da9c4d0 0x55761da9c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:13.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.848+0000 7fd0c1503700 1 -- 172.21.15.100:0/3763607137 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55761daa0c80 msgr2=0x55761daa1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:13.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.848+0000 7fd0c1503700 1 --2- 172.21.15.100:0/3763607137 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55761daa0c80 0x55761daa1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:13.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.848+0000 7fd0c1503700 1 -- 172.21.15.100:0/3763607137 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55761d93b8d0 con 0x55761da9cdf0 2024-05-10T11:27:13.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.848+0000 7fd0c1d04700 1 --2- 172.21.15.100:0/3763607137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55761da9c4d0 0x55761da9c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:27:13.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.849+0000 7fd0c1503700 1 --2- 172.21.15.100:0/3763607137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55761da9cdf0 0x55761daa06f0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac015000 tx=0x7fd0ac01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0d08894322eef85 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:27:13.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.849+0000 7fd0a7fff700 1 -- 172.21.15.100:0/3763607137 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac00b8b0 con 0x55761da9cdf0 2024-05-10T11:27:13.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.849+0000 7fd0a7fff700 1 -- 172.21.15.100:0/3763607137 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd0ac025470 con 0x55761da9cdf0 2024-05-10T11:27:13.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.849+0000 7fd0a7fff700 1 -- 172.21.15.100:0/3763607137 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac00f6c0 con 0x55761da9cdf0 2024-05-10T11:27:13.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.850+0000 7fd0ce7fbb80 1 -- 172.21.15.100:0/3763607137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55761da9cdf0 msgr2=0x55761daa06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:13.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.850+0000 7fd0ce7fbb80 1 --2- 172.21.15.100:0/3763607137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55761da9cdf0 0x55761daa06f0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac015000 tx=0x7fd0ac01f000 comp rx=0 tx=0).stop 2024-05-10T11:27:13.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.850+0000 7fd0ce7fbb80 1 -- 172.21.15.100:0/3763607137 shutdown_connections 2024-05-10T11:27:13.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.850+0000 7fd0ce7fbb80 1 --2- 172.21.15.100:0/3763607137 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55761daa0c80 0x55761daa1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:13.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.850+0000 7fd0ce7fbb80 1 --2- 172.21.15.100:0/3763607137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55761da9c4d0 0x55761da9c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:13.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.850+0000 7fd0ce7fbb80 1 --2- 172.21.15.100:0/3763607137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55761da9cdf0 0x55761daa06f0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:13.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.850+0000 7fd0ce7fbb80 1 -- 172.21.15.100:0/3763607137 >> 172.21.15.100:0/3763607137 conn(0x55761d9c8100 msgr2=0x55761da96c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:27:13.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.850+0000 7fd0ce7fbb80 1 -- 172.21.15.100:0/3763607137 shutdown_connections 2024-05-10T11:27:13.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.850+0000 7fd0ce7fbb80 1 -- 172.21.15.100:0/3763607137 wait complete. 2024-05-10T11:27:13.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.850+0000 7fd0ce7fbb80 1 -- start start 2024-05-10T11:27:13.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.850+0000 7fd0ce7fbb80 1 Processor -- start 2024-05-10T11:27:13.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.851+0000 7fd0ce7fbb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55761da9c4d0 0x55761daa8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:13.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.851+0000 7fd0ce7fbb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55761da9cdf0 0x55761daa5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:13.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.851+0000 7fd0ce7fbb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55761daa0c80 0x55761daa5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:13.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.851+0000 7fd0ce7fbb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55761daa8e30 con 0x55761daa0c80 2024-05-10T11:27:13.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.851+0000 7fd0ce7fbb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55761d954c30 con 0x55761da9c4d0 2024-05-10T11:27:13.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.851+0000 7fd0ce7fbb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55761daa8fb0 con 0x55761da9cdf0 2024-05-10T11:27:13.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.851+0000 7fd0ce7fbb80 10 client.0 _ll_register_callbacks cb 0x55761d986920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:27:13.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.851+0000 7fd0c1d04700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55761da9c4d0 0x55761daa8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:13.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.851+0000 7fd0c1503700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55761da9cdf0 0x55761daa5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:13.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.851+0000 7fd0c1d04700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55761da9c4d0 0x55761daa8030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:47542/0 (socket says 172.21.15.100:47542) 2024-05-10T11:27:13.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.851+0000 7fd0c1d04700 1 -- 172.21.15.100:0/2794193397 learned_addr learned my addr 172.21.15.100:0/2794193397 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:27:13.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.851+0000 7fd0c2505700 1 --2- 172.21.15.100:0/2794193397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55761daa0c80 0x55761daa5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:13.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.851+0000 7fd0c1d04700 1 -- 172.21.15.100:0/2794193397 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55761da9cdf0 msgr2=0x55761daa5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:13.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.851+0000 7fd0c1d04700 1 --2- 172.21.15.100:0/2794193397 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55761da9cdf0 0x55761daa5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:13.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.851+0000 7fd0c1d04700 1 -- 172.21.15.100:0/2794193397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55761daa0c80 msgr2=0x55761daa5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:13.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.851+0000 7fd0c1d04700 1 --2- 172.21.15.100:0/2794193397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55761daa0c80 0x55761daa5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:13.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.851+0000 7fd0c1d04700 1 -- 172.21.15.100:0/2794193397 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55761d98a690 con 0x55761da9c4d0 2024-05-10T11:27:13.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.851+0000 7fd0c1503700 1 --2- 172.21.15.100:0/2794193397 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55761da9cdf0 0x55761daa5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:27:13.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.851+0000 7fd0c2505700 1 --2- 172.21.15.100:0/2794193397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55761daa0c80 0x55761daa5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:27:13.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.851+0000 7fd0c1d04700 1 --2- 172.21.15.100:0/2794193397 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55761da9c4d0 0x55761daa8030 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd0b80a9720 tx=0x7fd0b80b26d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50c3bc3901e6eceb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:27:13.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.851+0000 7fd0a57fa700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:27:13.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.852+0000 7fd0a57fa700 1 -- 172.21.15.100:0/2794193397 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b80b21c0 con 0x55761da9c4d0 2024-05-10T11:27:13.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.852+0000 7fd0a57fa700 1 -- 172.21.15.100:0/2794193397 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fd0b80cc2b0 con 0x55761da9c4d0 2024-05-10T11:27:13.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.852+0000 7fd0ce7fbb80 1 -- 172.21.15.100:0/2794193397 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0ac025470 con 0x55761da9c4d0 2024-05-10T11:27:13.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.852+0000 7fd0a57fa700 1 -- 172.21.15.100:0/2794193397 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fd0b80a9530 con 0x55761da9c4d0 2024-05-10T11:27:13.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.852+0000 7fd0ce7fbb80 10 client.26558 fetch_fsmap learned FSMap version 31 2024-05-10T11:27:13.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.852+0000 7fd0ce7fbb80 1 -- 172.21.15.100:0/2794193397 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55761dab35f0 con 0x55761da9c4d0 2024-05-10T11:27:13.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.852+0000 7fd0a57fa700 1 -- 172.21.15.100:0/2794193397 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd0b80aa120 con 0x55761da9c4d0 2024-05-10T11:27:13.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.852+0000 7fd0a57fa700 1 -- 172.21.15.100:0/2794193397 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fd0b80a5390 con 0x55761da9c4d0 2024-05-10T11:27:13.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.852+0000 7fd0ce7fbb80 10 client.26558 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:27:13.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.852+0000 7fd0ce7fbb80 10 client.26558 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-10T11:27:13.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.852+0000 7fd0ce7fbb80 20 client.26558 populate_metadata read hostname 'smithi100' 2024-05-10T11:27:13.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.852+0000 7fd0ce7fbb80 1 --2- 172.21.15.100:0/2794193397 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55761dab5b80 0x55761dab7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:13.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.852+0000 7fd0ce7fbb80 4 client.26558 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:27:13.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.852+0000 7fd0ce7fbb80 1 -- 172.21.15.100:0/2794193397 --> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd0ac025470 con 0x55761dab5b80 2024-05-10T11:27:13.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.852+0000 7fd0c1503700 1 --2- 172.21.15.100:0/2794193397 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55761dab5b80 0x55761dab7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:13.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.853+0000 7fd0a57fa700 10 client.26558 ms_handle_connect on v2:172.21.15.188:6836/1943342620 2024-05-10T11:27:13.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.853+0000 7fd0c1503700 1 --2- 172.21.15.100:0/2794193397 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55761dab5b80 0x55761dab7f70 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a61016d1bc568496 server_cookie=d3217aeaa996f057 in_seq=0 out_seq=0 2024-05-10T11:27:13.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.853+0000 7fd0a57fa700 1 -- 172.21.15.100:0/2794193397 <== mds.2 v2:172.21.15.188:6836/1943342620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fd0ac025470 con 0x55761dab5b80 2024-05-10T11:27:13.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.853+0000 7fd0a57fa700 10 client.26558 handle_command_reply: tid=0 2024-05-10T11:27:13.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.854+0000 7fd0ce7fbb80 1 client.26558 shutdown 2024-05-10T11:27:13.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.854+0000 7fd0ce7fbb80 1 -- 172.21.15.100:0/2794193397 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55761da9c4d0 msgr2=0x55761daa8030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:13.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.854+0000 7fd0ce7fbb80 1 --2- 172.21.15.100:0/2794193397 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55761da9c4d0 0x55761daa8030 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd0b80a9720 tx=0x7fd0b80b26d0 comp rx=0 tx=0).stop 2024-05-10T11:27:13.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.854+0000 7fd0ce7fbb80 1 -- 172.21.15.100:0/2794193397 shutdown_connections 2024-05-10T11:27:13.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.854+0000 7fd0ce7fbb80 1 --2- 172.21.15.100:0/2794193397 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55761dab5b80 0x55761dab7f70 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:13.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.854+0000 7fd0ce7fbb80 1 --2- 172.21.15.100:0/2794193397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55761daa0c80 0x55761daa5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:13.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.854+0000 7fd0ce7fbb80 1 --2- 172.21.15.100:0/2794193397 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55761da9c4d0 0x55761daa8030 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:13.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.854+0000 7fd0a57fa700 0 client.26558 ms_handle_reset on v2:172.21.15.188:6836/1943342620 2024-05-10T11:27:13.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.854+0000 7fd0ce7fbb80 1 --2- 172.21.15.100:0/2794193397 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55761da9cdf0 0x55761daa5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:13.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.854+0000 7fd0ce7fbb80 1 -- 172.21.15.100:0/2794193397 >> 172.21.15.100:0/2794193397 conn(0x55761d9c8100 msgr2=0x55761da91720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:27:13.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.854+0000 7fd0ce7fbb80 1 -- 172.21.15.100:0/2794193397 shutdown_connections 2024-05-10T11:27:13.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.854+0000 7fd0ce7fbb80 1 -- 172.21.15.100:0/2794193397 wait complete. 2024-05-10T11:27:13.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.854+0000 7fd0ce7fbb80 20 client.26558 trim_cache size 0 max 16384 2024-05-10T11:27:13.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.871+0000 7fd0ce7fbb80 1 Processor -- start 2024-05-10T11:27:13.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.871+0000 7fd0ce7fbb80 1 -- start start 2024-05-10T11:27:13.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.872+0000 7fd0ce7fbb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55761da9c4d0 0x55761dab1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:13.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.872+0000 7fd0ce7fbb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55761da9cdf0 0x55761da91500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:13.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.872+0000 7fd0ce7fbb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55761daa0c80 0x55761da9e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:13.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.872+0000 7fd0ce7fbb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd0bc1b3d10 con 0x55761da9cdf0 2024-05-10T11:27:13.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.872+0000 7fd0ce7fbb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd0bc1b15a0 con 0x55761daa0c80 2024-05-10T11:27:13.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.872+0000 7fd0ce7fbb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd0bc1af3f0 con 0x55761da9c4d0 2024-05-10T11:27:13.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.872+0000 7fd0c1503700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55761da9cdf0 0x55761da91500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:13.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.872+0000 7fd0c1503700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55761da9cdf0 0x55761da91500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60954/0 (socket says 172.21.15.100:60954) 2024-05-10T11:27:13.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.872+0000 7fd0c1503700 1 -- 172.21.15.100:0/2827619406 learned_addr learned my addr 172.21.15.100:0/2827619406 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:27:13.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.872+0000 7fd0c1d04700 1 --2- 172.21.15.100:0/2827619406 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55761da9c4d0 0x55761dab1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:13.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.872+0000 7fd0c2505700 1 --2- 172.21.15.100:0/2827619406 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55761daa0c80 0x55761da9e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:13.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.872+0000 7fd0c1503700 1 -- 172.21.15.100:0/2827619406 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55761da9c4d0 msgr2=0x55761dab1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:13.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.872+0000 7fd0c1503700 1 --2- 172.21.15.100:0/2827619406 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55761da9c4d0 0x55761dab1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:13.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.872+0000 7fd0c1503700 1 -- 172.21.15.100:0/2827619406 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55761daa0c80 msgr2=0x55761da9e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:13.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.872+0000 7fd0c1503700 1 --2- 172.21.15.100:0/2827619406 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55761daa0c80 0x55761da9e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:13.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.872+0000 7fd0c1503700 1 -- 172.21.15.100:0/2827619406 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55761dab8550 con 0x55761da9cdf0 2024-05-10T11:27:13.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.872+0000 7fd0c1503700 1 --2- 172.21.15.100:0/2827619406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55761da9cdf0 0x55761da91500 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac009ce0 tx=0x7fd0ac00b300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b7b9e6b53cd7d7a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:27:13.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.872+0000 7fd0a77fe700 1 -- 172.21.15.100:0/2827619406 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac025b30 con 0x55761da9cdf0 2024-05-10T11:27:13.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.873+0000 7fd0a77fe700 1 -- 172.21.15.100:0/2827619406 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd0ac00bb40 con 0x55761da9cdf0 2024-05-10T11:27:13.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.873+0000 7fd0a77fe700 1 -- 172.21.15.100:0/2827619406 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac00f7a0 con 0x55761da9cdf0 2024-05-10T11:27:13.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.873+0000 7fd0ce7fbb80 1 -- 172.21.15.100:0/2827619406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55761da9cdf0 msgr2=0x55761da91500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:13.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.873+0000 7fd0ce7fbb80 1 --2- 172.21.15.100:0/2827619406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55761da9cdf0 0x55761da91500 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac009ce0 tx=0x7fd0ac00b300 comp rx=0 tx=0).stop 2024-05-10T11:27:13.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.873+0000 7fd0ce7fbb80 1 -- 172.21.15.100:0/2827619406 shutdown_connections 2024-05-10T11:27:13.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.873+0000 7fd0ce7fbb80 1 --2- 172.21.15.100:0/2827619406 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55761daa0c80 0x55761da9e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:13.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.873+0000 7fd0ce7fbb80 1 --2- 172.21.15.100:0/2827619406 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55761da9c4d0 0x55761dab1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:13.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.873+0000 7fd0ce7fbb80 1 --2- 172.21.15.100:0/2827619406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55761da9cdf0 0x55761da91500 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:13.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.873+0000 7fd0ce7fbb80 1 -- 172.21.15.100:0/2827619406 >> 172.21.15.100:0/2827619406 conn(0x55761d9c8100 msgr2=0x55761da9bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:27:13.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.873+0000 7fd0ce7fbb80 1 -- 172.21.15.100:0/2827619406 shutdown_connections 2024-05-10T11:27:13.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.873+0000 7fd0ce7fbb80 1 -- 172.21.15.100:0/2827619406 wait complete. 2024-05-10T11:27:13.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.874+0000 7fd0ce7fbb80 1 -- start start 2024-05-10T11:27:13.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.874+0000 7fd0ce7fbb80 1 Processor -- start 2024-05-10T11:27:13.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.874+0000 7fd0ce7fbb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55761da9c4d0 0x55761daae3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:13.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.874+0000 7fd0ce7fbb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55761da9cdf0 0x55761daae920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:13.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.874+0000 7fd0ce7fbb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55761daa0c80 0x55761daab980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:13.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.874+0000 7fd0ce7fbb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd0bc1afa60 con 0x55761da9c4d0 2024-05-10T11:27:13.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.874+0000 7fd0ce7fbb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd0bc1b0ed0 con 0x55761da9cdf0 2024-05-10T11:27:13.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.874+0000 7fd0ce7fbb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd0bc1b0130 con 0x55761daa0c80 2024-05-10T11:27:13.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.874+0000 7fd0ce7fbb80 10 client.0 _ll_register_callbacks cb 0x55761d986920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:27:13.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.874+0000 7fd0c1d04700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55761da9c4d0 0x55761daae3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:13.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.874+0000 7fd0c1503700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55761da9cdf0 0x55761daae920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:13.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.874+0000 7fd0c1d04700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55761da9c4d0 0x55761daae3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60958/0 (socket says 172.21.15.100:60958) 2024-05-10T11:27:13.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.874+0000 7fd0c1d04700 1 -- 172.21.15.100:0/629507306 learned_addr learned my addr 172.21.15.100:0/629507306 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:27:13.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.874+0000 7fd0c2505700 1 --2- 172.21.15.100:0/629507306 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55761daa0c80 0x55761daab980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:13.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.875+0000 7fd0c1503700 1 -- 172.21.15.100:0/629507306 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55761daa0c80 msgr2=0x55761daab980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:13.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.875+0000 7fd0c1503700 1 --2- 172.21.15.100:0/629507306 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55761daa0c80 0x55761daab980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:13.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.875+0000 7fd0c1503700 1 -- 172.21.15.100:0/629507306 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55761da9c4d0 msgr2=0x55761daae3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:13.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.875+0000 7fd0c1503700 1 --2- 172.21.15.100:0/629507306 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55761da9c4d0 0x55761daae3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:13.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.875+0000 7fd0c1503700 1 -- 172.21.15.100:0/629507306 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55761daac180 con 0x55761da9cdf0 2024-05-10T11:27:13.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.875+0000 7fd0c1d04700 1 --2- 172.21.15.100:0/629507306 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55761da9c4d0 0x55761daae3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:27:13.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.875+0000 7fd0c1503700 1 --2- 172.21.15.100:0/629507306 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55761da9cdf0 0x55761daae920 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x55761daa3c20 tx=0x7fd0ac005c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db8c8882e07df08f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:27:13.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.875+0000 7fd0a57fa700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:27:13.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.875+0000 7fd0a57fa700 1 -- 172.21.15.100:0/629507306 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac025930 con 0x55761da9cdf0 2024-05-10T11:27:13.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.875+0000 7fd0ce7fbb80 1 -- 172.21.15.100:0/629507306 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0ac00bb40 con 0x55761da9cdf0 2024-05-10T11:27:13.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.875+0000 7fd0a57fa700 1 -- 172.21.15.100:0/629507306 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fd0ac025ad0 con 0x55761da9cdf0 2024-05-10T11:27:13.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.875+0000 7fd0a57fa700 1 -- 172.21.15.100:0/629507306 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fd0ac00f460 con 0x55761da9cdf0 2024-05-10T11:27:13.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.875+0000 7fd0ce7fbb80 10 client.26564 fetch_fsmap learned FSMap version 31 2024-05-10T11:27:13.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.875+0000 7fd0ce7fbb80 1 -- 172.21.15.100:0/629507306 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55761db1fff0 con 0x55761da9cdf0 2024-05-10T11:27:13.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.875+0000 7fd0a57fa700 1 -- 172.21.15.100:0/629507306 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd0ac00bb40 con 0x55761da9cdf0 2024-05-10T11:27:13.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.876+0000 7fd0a57fa700 1 -- 172.21.15.100:0/629507306 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fd0ac02e020 con 0x55761da9cdf0 2024-05-10T11:27:13.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.876+0000 7fd0ce7fbb80 10 client.26564 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:27:13.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.876+0000 7fd0ce7fbb80 10 client.26564 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-10T11:27:13.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.876+0000 7fd0ce7fbb80 20 client.26564 populate_metadata read hostname 'smithi100' 2024-05-10T11:27:13.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.876+0000 7fd0ce7fbb80 1 --2- 172.21.15.100:0/629507306 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55761db20280 0x55761db22670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:13.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.876+0000 7fd0ce7fbb80 4 client.26564 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:27:13.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.876+0000 7fd0ce7fbb80 1 -- 172.21.15.100:0/629507306 --> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55761da68650 con 0x55761db20280 2024-05-10T11:27:13.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.876+0000 7fd0c1d04700 1 --2- 172.21.15.100:0/629507306 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55761db20280 0x55761db22670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:13.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.876+0000 7fd0a57fa700 10 client.26564 ms_handle_connect on v2:172.21.15.188:6836/1943342620 2024-05-10T11:27:13.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.877+0000 7fd0c1d04700 1 --2- 172.21.15.100:0/629507306 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55761db20280 0x55761db22670 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c305fafd6f2d5caa server_cookie=c6b6b22a6dfe9c1b in_seq=0 out_seq=0 2024-05-10T11:27:13.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.877+0000 7fd0a57fa700 1 -- 172.21.15.100:0/629507306 <== mds.2 v2:172.21.15.188:6836/1943342620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55761da68650 con 0x55761db20280 2024-05-10T11:27:13.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.877+0000 7fd0a57fa700 10 client.26564 handle_command_reply: tid=0 2024-05-10T11:27:13.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.877+0000 7fd0ce7fbb80 1 client.26564 shutdown 2024-05-10T11:27:13.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.877+0000 7fd0ce7fbb80 1 -- 172.21.15.100:0/629507306 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55761da9cdf0 msgr2=0x55761daae920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:13.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.877+0000 7fd0ce7fbb80 1 --2- 172.21.15.100:0/629507306 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55761da9cdf0 0x55761daae920 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x55761daa3c20 tx=0x7fd0ac005c50 comp rx=0 tx=0).stop 2024-05-10T11:27:13.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.877+0000 7fd0ce7fbb80 1 -- 172.21.15.100:0/629507306 shutdown_connections 2024-05-10T11:27:13.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.877+0000 7fd0ce7fbb80 1 --2- 172.21.15.100:0/629507306 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55761db20280 0x55761db22670 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:13.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.877+0000 7fd0ce7fbb80 1 --2- 172.21.15.100:0/629507306 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55761daa0c80 0x55761daab980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:13.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.877+0000 7fd0a57fa700 0 client.26564 ms_handle_reset on v2:172.21.15.188:6836/1943342620 2024-05-10T11:27:13.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.877+0000 7fd0ce7fbb80 1 --2- 172.21.15.100:0/629507306 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55761da9c4d0 0x55761daae3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:13.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.877+0000 7fd0ce7fbb80 1 --2- 172.21.15.100:0/629507306 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55761da9cdf0 0x55761daae920 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:13.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.877+0000 7fd0ce7fbb80 1 -- 172.21.15.100:0/629507306 >> 172.21.15.100:0/629507306 conn(0x55761d9c8100 msgr2=0x55761da9f470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:27:13.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.877+0000 7fd0ce7fbb80 1 -- 172.21.15.100:0/629507306 shutdown_connections 2024-05-10T11:27:13.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.877+0000 7fd0ce7fbb80 1 -- 172.21.15.100:0/629507306 wait complete. 2024-05-10T11:27:13.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.877+0000 7fd0ce7fbb80 20 client.26564 trim_cache size 0 max 16384 2024-05-10T11:27:13.882 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:27:13.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.878+0000 7fd0c3f68700 1 -- 172.21.15.100:0/138073762 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fd0a80b2900 msgr2=0x7fd0a80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:13.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.878+0000 7fd0c3f68700 1 --2- 172.21.15.100:0/138073762 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fd0a80b2900 0x7fd0a80b4dc0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc103a00 tx=0x7fd0ac005fb0 comp rx=0 tx=0).stop 2024-05-10T11:27:13.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.878+0000 7fd0c3f68700 1 -- 172.21.15.100:0/138073762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0bc06bba0 msgr2=0x7fd0bc19bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:13.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.878+0000 7fd0c3f68700 1 --2- 172.21.15.100:0/138073762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0bc06bba0 0x7fd0bc19bd80 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fd0b800e9a0 tx=0x7fd0b800ecb0 comp rx=0 tx=0).stop 2024-05-10T11:27:13.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.878+0000 7fd0c3f68700 1 -- 172.21.15.100:0/138073762 shutdown_connections 2024-05-10T11:27:13.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.878+0000 7fd0c3f68700 1 --2- 172.21.15.100:0/138073762 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fd0a80b2900 0x7fd0a80b4dc0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:13.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.878+0000 7fd0c3f68700 1 --2- 172.21.15.100:0/138073762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd0bc10ff80 0x7fd0bc1a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:13.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.878+0000 7fd0c3f68700 1 --2- 172.21.15.100:0/138073762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0bc06bba0 0x7fd0bc19bd80 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:13.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.878+0000 7fd0c3f68700 1 --2- 172.21.15.100:0/138073762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0bc06c4c0 0x7fd0bc19c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:13.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.878+0000 7fd0c3f68700 1 -- 172.21.15.100:0/138073762 >> 172.21.15.100:0/138073762 conn(0x7fd0bc0793e0 msgr2=0x7fd0bc1115d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:27:13.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.878+0000 7fd0c3f68700 1 -- 172.21.15.100:0/138073762 shutdown_connections 2024-05-10T11:27:13.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:13.878+0000 7fd0c3f68700 1 -- 172.21.15.100:0/138073762 wait complete. 2024-05-10T11:27:13.977 INFO:tasks.workunit.client.0.smithi100.stdout: 1 221153 2.74 MB/sec execute 308 sec latency 69.871 ms 2024-05-10T11:27:14.606 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-10T11:27:14.872 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:27:14.977 INFO:tasks.workunit.client.0.smithi100.stdout: 1 221762 2.74 MB/sec execute 309 sec latency 16.953 ms 2024-05-10T11:27:15.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:27:15 smithi178 ceph-mon[33359]: pgmap v686: 129 pgs: 129 active+clean; 93 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 1.3 MiB/s wr, 124 op/s 2024-05-10T11:27:15.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:27:15 smithi188 ceph-mon[34075]: pgmap v686: 129 pgs: 129 active+clean; 93 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 1.3 MiB/s wr, 124 op/s 2024-05-10T11:27:15.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:27:15 smithi100 ceph-mon[30296]: pgmap v686: 129 pgs: 129 active+clean; 93 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 1.3 MiB/s wr, 124 op/s 2024-05-10T11:27:15.977 INFO:tasks.workunit.client.0.smithi100.stdout: 1 222374 2.74 MB/sec execute 310 sec latency 15.563 ms 2024-05-10T11:27:16.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.718+0000 7f9334ce2700 1 -- 172.21.15.100:0/2971087602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93301037b0 msgr2=0x7f9330107030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:16.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.718+0000 7f9334ce2700 1 --2- 172.21.15.100:0/2971087602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93301037b0 0x7f9330107030 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f93240099c0 tx=0x7f9324009cd0 comp rx=0 tx=0).stop 2024-05-10T11:27:16.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.718+0000 7f9334ce2700 1 -- 172.21.15.100:0/2971087602 shutdown_connections 2024-05-10T11:27:16.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.718+0000 7f9334ce2700 1 --2- 172.21.15.100:0/2971087602 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9330107760 0x7f933010f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:16.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.718+0000 7f9334ce2700 1 --2- 172.21.15.100:0/2971087602 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9330102e00 0x7f93301031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:16.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.718+0000 7f9334ce2700 1 --2- 172.21.15.100:0/2971087602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93301037b0 0x7f9330107030 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:16.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.718+0000 7f9334ce2700 1 -- 172.21.15.100:0/2971087602 >> 172.21.15.100:0/2971087602 conn(0x7f9330078a10 msgr2=0x7f93300ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:27:16.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.718+0000 7f9334ce2700 1 -- 172.21.15.100:0/2971087602 shutdown_connections 2024-05-10T11:27:16.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.718+0000 7f9334ce2700 1 -- 172.21.15.100:0/2971087602 wait complete. 2024-05-10T11:27:16.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.719+0000 7f9334ce2700 1 Processor -- start 2024-05-10T11:27:16.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.719+0000 7f9334ce2700 1 -- start start 2024-05-10T11:27:16.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.720+0000 7f9334ce2700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9330102e00 0x7f933019f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:16.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.720+0000 7f9334ce2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93301037b0 0x7f933019fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:16.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.720+0000 7f9334ce2700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9330107760 0x7f93301997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:16.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.720+0000 7f9334ce2700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f93301123f0 con 0x7f93301037b0 2024-05-10T11:27:16.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.720+0000 7f9334ce2700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9330112270 con 0x7f9330102e00 2024-05-10T11:27:16.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.720+0000 7f9334ce2700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9330112570 con 0x7f9330107760 2024-05-10T11:27:16.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.720+0000 7f932ed9d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9330107760 0x7f93301997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:16.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.720+0000 7f932ed9d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9330107760 0x7f93301997b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:52844/0 (socket says 172.21.15.100:52844) 2024-05-10T11:27:16.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.720+0000 7f932ed9d700 1 -- 172.21.15.100:0/2577848259 learned_addr learned my addr 172.21.15.100:0/2577848259 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:27:16.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.720+0000 7f932dd9b700 1 --2- 172.21.15.100:0/2577848259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93301037b0 0x7f933019fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:16.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.720+0000 7f932e59c700 1 --2- 172.21.15.100:0/2577848259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9330102e00 0x7f933019f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:16.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.720+0000 7f932ed9d700 1 -- 172.21.15.100:0/2577848259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9330102e00 msgr2=0x7f933019f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:16.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.720+0000 7f932ed9d700 1 --2- 172.21.15.100:0/2577848259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9330102e00 0x7f933019f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:16.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.720+0000 7f932ed9d700 1 -- 172.21.15.100:0/2577848259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93301037b0 msgr2=0x7f933019fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:16.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.720+0000 7f932ed9d700 1 --2- 172.21.15.100:0/2577848259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93301037b0 0x7f933019fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:16.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.720+0000 7f932ed9d700 1 -- 172.21.15.100:0/2577848259 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9330199f90 con 0x7f9330107760 2024-05-10T11:27:16.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.720+0000 7f932ed9d700 1 --2- 172.21.15.100:0/2577848259 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9330107760 0x7f93301997b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f932800ef30 tx=0x7f932800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:27:16.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.721+0000 7f931f7fe700 1 -- 172.21.15.100:0/2577848259 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93280092e0 con 0x7f9330107760 2024-05-10T11:27:16.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.721+0000 7f931f7fe700 1 -- 172.21.15.100:0/2577848259 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9328004bb0 con 0x7f9330107760 2024-05-10T11:27:16.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.721+0000 7f9334ce2700 1 -- 172.21.15.100:0/2577848259 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f933019a220 con 0x7f9330107760 2024-05-10T11:27:16.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.721+0000 7f931f7fe700 1 -- 172.21.15.100:0/2577848259 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9328004020 con 0x7f9330107760 2024-05-10T11:27:16.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.721+0000 7f9334ce2700 1 -- 172.21.15.100:0/2577848259 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93300fddb0 con 0x7f9330107760 2024-05-10T11:27:16.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.724+0000 7f931f7fe700 1 -- 172.21.15.100:0/2577848259 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f932801b030 con 0x7f9330107760 2024-05-10T11:27:16.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.725+0000 7f931f7fe700 1 --2- 172.21.15.100:0/2577848259 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f93180b2880 0x7f93180b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:16.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.725+0000 7f932e59c700 1 --2- 172.21.15.100:0/2577848259 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f93180b2880 0x7f93180b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:16.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.725+0000 7f931f7fe700 1 -- 172.21.15.100:0/2577848259 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f93280e2070 con 0x7f9330107760 2024-05-10T11:27:16.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.725+0000 7f933f575b80 1 Processor -- start 2024-05-10T11:27:16.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.725+0000 7f932e59c700 1 --2- 172.21.15.100:0/2577848259 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f93180b2880 0x7f93180b4d40 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9320005950 tx=0x7f932000f820 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:27:16.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.725+0000 7f933f575b80 1 -- start start 2024-05-10T11:27:16.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.726+0000 7f933f575b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5597924b41b0 0x5597924b4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:16.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.726+0000 7f933f575b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5597924b4ad0 0x5597924ae810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:16.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.726+0000 7f933f575b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5597924aed50 0x5597924af190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:16.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.726+0000 7f933f575b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x559792400940 con 0x5597924b4ad0 2024-05-10T11:27:16.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.726+0000 7f933f575b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559792365da0 con 0x5597924b41b0 2024-05-10T11:27:16.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.726+0000 7f933f575b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5597923fc900 con 0x5597924aed50 2024-05-10T11:27:16.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.726+0000 7f932dd9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5597924b4ad0 0x5597924ae810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:16.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.726+0000 7f932dd9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5597924b4ad0 0x5597924ae810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45674/0 (socket says 172.21.15.100:45674) 2024-05-10T11:27:16.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.726+0000 7f932dd9b700 1 -- 172.21.15.100:0/4200018776 learned_addr learned my addr 172.21.15.100:0/4200018776 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:27:16.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.726+0000 7f932ed9d700 1 --2- 172.21.15.100:0/4200018776 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5597924aed50 0x5597924af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:16.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.726+0000 7f932e59c700 1 --2- 172.21.15.100:0/4200018776 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5597924b41b0 0x5597924b4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:16.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.726+0000 7f932dd9b700 1 -- 172.21.15.100:0/4200018776 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5597924aed50 msgr2=0x5597924af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:16.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.726+0000 7f932dd9b700 1 --2- 172.21.15.100:0/4200018776 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5597924aed50 0x5597924af190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:16.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.726+0000 7f932dd9b700 1 -- 172.21.15.100:0/4200018776 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5597924b41b0 msgr2=0x5597924b4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:16.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.726+0000 7f932dd9b700 1 --2- 172.21.15.100:0/4200018776 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5597924b41b0 0x5597924b4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:16.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.726+0000 7f932dd9b700 1 -- 172.21.15.100:0/4200018776 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55979234d8d0 con 0x5597924b4ad0 2024-05-10T11:27:16.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.726+0000 7f932e59c700 1 --2- 172.21.15.100:0/4200018776 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5597924b41b0 0x5597924b4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:27:16.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.726+0000 7f932dd9b700 1 --2- 172.21.15.100:0/4200018776 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5597924b4ad0 0x5597924ae810 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f93240099c0 tx=0x7f9324005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85be6c529a7bdc14 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:27:16.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.727+0000 7f931cff9700 1 -- 172.21.15.100:0/4200018776 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9324028070 con 0x5597924b4ad0 2024-05-10T11:27:16.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.727+0000 7f931cff9700 1 -- 172.21.15.100:0/4200018776 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f93240037c0 con 0x5597924b4ad0 2024-05-10T11:27:16.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.727+0000 7f931cff9700 1 -- 172.21.15.100:0/4200018776 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f932400b9f0 con 0x5597924b4ad0 2024-05-10T11:27:16.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.727+0000 7f933f575b80 1 -- 172.21.15.100:0/4200018776 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5597924b4ad0 msgr2=0x5597924ae810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:16.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.727+0000 7f933f575b80 1 --2- 172.21.15.100:0/4200018776 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5597924b4ad0 0x5597924ae810 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f93240099c0 tx=0x7f9324005bc0 comp rx=0 tx=0).stop 2024-05-10T11:27:16.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.727+0000 7f933f575b80 1 -- 172.21.15.100:0/4200018776 shutdown_connections 2024-05-10T11:27:16.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.727+0000 7f933f575b80 1 --2- 172.21.15.100:0/4200018776 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5597924aed50 0x5597924af190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:16.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.727+0000 7f933f575b80 1 --2- 172.21.15.100:0/4200018776 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5597924b41b0 0x5597924b4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:16.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.727+0000 7f933f575b80 1 --2- 172.21.15.100:0/4200018776 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5597924b4ad0 0x5597924ae810 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:16.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.727+0000 7f933f575b80 1 -- 172.21.15.100:0/4200018776 >> 172.21.15.100:0/4200018776 conn(0x5597923dab70 msgr2=0x5597924a92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:27:16.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.728+0000 7f933f575b80 1 -- 172.21.15.100:0/4200018776 shutdown_connections 2024-05-10T11:27:16.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.728+0000 7f933f575b80 1 -- 172.21.15.100:0/4200018776 wait complete. 2024-05-10T11:27:16.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.728+0000 7f933f575b80 1 -- start start 2024-05-10T11:27:16.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.728+0000 7f933f575b80 1 Processor -- start 2024-05-10T11:27:16.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.728+0000 7f933f575b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5597924aed50 0x5597924ba740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:16.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.728+0000 7f933f575b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5597924b41b0 0x5597924b7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:16.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.728+0000 7f933f575b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5597924b4ad0 0x5597924b8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:16.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.728+0000 7f933f575b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5597924bb3f0 con 0x5597924aed50 2024-05-10T11:27:16.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.728+0000 7f933f575b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5597923b9ed0 con 0x5597924b41b0 2024-05-10T11:27:16.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.728+0000 7f933f575b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5597924bb570 con 0x5597924b4ad0 2024-05-10T11:27:16.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.728+0000 7f933f575b80 10 client.0 _ll_register_callbacks cb 0x5597923945b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:27:16.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.729+0000 7f932ed9d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5597924b4ad0 0x5597924b8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:16.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.729+0000 7f932ed9d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5597924b4ad0 0x5597924b8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:52852/0 (socket says 172.21.15.100:52852) 2024-05-10T11:27:16.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.729+0000 7f932dd9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5597924b41b0 0x5597924b7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:16.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.729+0000 7f932ed9d700 1 -- 172.21.15.100:0/280438304 learned_addr learned my addr 172.21.15.100:0/280438304 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:27:16.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.729+0000 7f932e59c700 1 --2- 172.21.15.100:0/280438304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5597924aed50 0x5597924ba740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:16.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.729+0000 7f932ed9d700 1 -- 172.21.15.100:0/280438304 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5597924b41b0 msgr2=0x5597924b7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:16.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.729+0000 7f932ed9d700 1 --2- 172.21.15.100:0/280438304 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5597924b41b0 0x5597924b7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:16.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.729+0000 7f932ed9d700 1 -- 172.21.15.100:0/280438304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5597924aed50 msgr2=0x5597924ba740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:16.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.729+0000 7f932ed9d700 1 --2- 172.21.15.100:0/280438304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5597924aed50 0x5597924ba740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:16.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.729+0000 7f932ed9d700 1 -- 172.21.15.100:0/280438304 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597923b7760 con 0x5597924b4ad0 2024-05-10T11:27:16.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.729+0000 7f932ed9d700 1 --2- 172.21.15.100:0/280438304 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5597924b4ad0 0x5597924b8020 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f93280dfb80 tx=0x7f93280a9900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e1973239947c978 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:27:16.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.729+0000 7f9315ffb700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:27:16.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.729+0000 7f9315ffb700 1 -- 172.21.15.100:0/280438304 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93280a93c0 con 0x5597924b4ad0 2024-05-10T11:27:16.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.729+0000 7f933f575b80 1 -- 172.21.15.100:0/280438304 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93240037c0 con 0x5597924b4ad0 2024-05-10T11:27:16.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.729+0000 7f9315ffb700 1 -- 172.21.15.100:0/280438304 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f93280a8830 con 0x5597924b4ad0 2024-05-10T11:27:16.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.729+0000 7f9315ffb700 1 -- 172.21.15.100:0/280438304 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f93280ad770 con 0x5597924b4ad0 2024-05-10T11:27:16.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.729+0000 7f933f575b80 10 client.26632 fetch_fsmap learned FSMap version 31 2024-05-10T11:27:16.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.729+0000 7f933f575b80 1 -- 172.21.15.100:0/280438304 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5597924c5d50 con 0x5597924b4ad0 2024-05-10T11:27:16.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.730+0000 7f9315ffb700 1 -- 172.21.15.100:0/280438304 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f93240037c0 con 0x5597924b4ad0 2024-05-10T11:27:16.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.730+0000 7f9315ffb700 1 -- 172.21.15.100:0/280438304 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f93280a8090 con 0x5597924b4ad0 2024-05-10T11:27:16.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.730+0000 7f933f575b80 10 client.26632 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:27:16.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.730+0000 7f933f575b80 10 client.26632 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-10T11:27:16.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.730+0000 7f933f575b80 20 client.26632 populate_metadata read hostname 'smithi100' 2024-05-10T11:27:16.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.730+0000 7f933f575b80 1 --2- 172.21.15.100:0/280438304 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x5597924c8330 0x5597924ca720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:16.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.730+0000 7f933f575b80 4 client.26632 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:27:16.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.730+0000 7f933f575b80 1 -- 172.21.15.100:0/280438304 --> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55979240e320 con 0x5597924c8330 2024-05-10T11:27:16.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.730+0000 7f932e59c700 1 --2- 172.21.15.100:0/280438304 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x5597924c8330 0x5597924ca720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:16.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.731+0000 7f9315ffb700 10 client.26632 ms_handle_connect on v2:172.21.15.178:6838/3760985404 2024-05-10T11:27:16.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.731+0000 7f932e59c700 1 --2- 172.21.15.100:0/280438304 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x5597924c8330 0x5597924ca720 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6d7688bccc3df629 server_cookie=b9cd3a81ba47420 in_seq=0 out_seq=0 2024-05-10T11:27:16.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.731+0000 7f9315ffb700 1 -- 172.21.15.100:0/280438304 <== mds.1 v2:172.21.15.178:6838/3760985404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55979240e320 con 0x5597924c8330 2024-05-10T11:27:16.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.731+0000 7f9315ffb700 10 client.26632 handle_command_reply: tid=0 2024-05-10T11:27:16.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.731+0000 7f933f575b80 1 client.26632 shutdown 2024-05-10T11:27:16.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.732+0000 7f933f575b80 1 -- 172.21.15.100:0/280438304 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5597924b4ad0 msgr2=0x5597924b8020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:16.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.732+0000 7f933f575b80 1 --2- 172.21.15.100:0/280438304 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5597924b4ad0 0x5597924b8020 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f93280dfb80 tx=0x7f93280a9900 comp rx=0 tx=0).stop 2024-05-10T11:27:16.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.732+0000 7f933f575b80 1 -- 172.21.15.100:0/280438304 shutdown_connections 2024-05-10T11:27:16.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.732+0000 7f933f575b80 1 --2- 172.21.15.100:0/280438304 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x5597924c8330 0x5597924ca720 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:16.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.732+0000 7f933f575b80 1 --2- 172.21.15.100:0/280438304 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5597924b4ad0 0x5597924b8020 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:16.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.732+0000 7f933f575b80 1 --2- 172.21.15.100:0/280438304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5597924aed50 0x5597924ba740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:16.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.732+0000 7f933f575b80 1 --2- 172.21.15.100:0/280438304 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5597924b41b0 0x5597924b7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:16.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.732+0000 7f933f575b80 1 -- 172.21.15.100:0/280438304 >> 172.21.15.100:0/280438304 conn(0x5597923dab70 msgr2=0x5597924a3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:27:16.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.732+0000 7f9315ffb700 0 client.26632 ms_handle_reset on v2:172.21.15.178:6838/3760985404 2024-05-10T11:27:16.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.732+0000 7f933f575b80 1 -- 172.21.15.100:0/280438304 shutdown_connections 2024-05-10T11:27:16.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.732+0000 7f933f575b80 1 -- 172.21.15.100:0/280438304 wait complete. 2024-05-10T11:27:16.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.732+0000 7f933f575b80 20 client.26632 trim_cache size 0 max 16384 2024-05-10T11:27:16.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.747+0000 7f933f575b80 1 Processor -- start 2024-05-10T11:27:16.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.747+0000 7f933f575b80 1 -- start start 2024-05-10T11:27:16.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.748+0000 7f933f575b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5597924aed50 0x5597924ae810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:16.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.748+0000 7f933f575b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5597924b41b0 0x5597924ba740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:16.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.748+0000 7f933f575b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5597924b4ad0 0x5597924c4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:16.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.748+0000 7f933f575b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f93301af4b0 con 0x5597924b4ad0 2024-05-10T11:27:16.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.748+0000 7f933f575b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f93301acd40 con 0x5597924aed50 2024-05-10T11:27:16.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.748+0000 7f933f575b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f93301aab90 con 0x5597924b41b0 2024-05-10T11:27:16.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.748+0000 7f932dd9b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5597924b41b0 0x5597924ba740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:16.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.748+0000 7f932ed9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5597924b4ad0 0x5597924c4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:16.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.748+0000 7f932dd9b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5597924b41b0 0x5597924ba740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:52862/0 (socket says 172.21.15.100:52862) 2024-05-10T11:27:16.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.748+0000 7f932dd9b700 1 -- 172.21.15.100:0/1012053567 learned_addr learned my addr 172.21.15.100:0/1012053567 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:27:16.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.748+0000 7f932e59c700 1 --2- 172.21.15.100:0/1012053567 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5597924aed50 0x5597924ae810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:16.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.748+0000 7f932dd9b700 1 -- 172.21.15.100:0/1012053567 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5597924aed50 msgr2=0x5597924ae810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:16.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.748+0000 7f932dd9b700 1 --2- 172.21.15.100:0/1012053567 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5597924aed50 0x5597924ae810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:16.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.748+0000 7f932dd9b700 1 -- 172.21.15.100:0/1012053567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5597924b4ad0 msgr2=0x5597924c4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:16.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.748+0000 7f932dd9b700 1 --2- 172.21.15.100:0/1012053567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5597924b4ad0 0x5597924c4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:16.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.748+0000 7f932dd9b700 1 -- 172.21.15.100:0/1012053567 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597924aad00 con 0x5597924b41b0 2024-05-10T11:27:16.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.748+0000 7f932ed9d700 1 --2- 172.21.15.100:0/1012053567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5597924b4ad0 0x5597924c4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:27:16.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.748+0000 7f932dd9b700 1 --2- 172.21.15.100:0/1012053567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5597924b41b0 0x5597924ba740 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9324009d10 tx=0x7f9324005900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b362e2f774bf562 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:27:16.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.749+0000 7f9317fff700 1 -- 172.21.15.100:0/1012053567 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9324028040 con 0x5597924b41b0 2024-05-10T11:27:16.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.749+0000 7f9317fff700 1 -- 172.21.15.100:0/1012053567 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9324029040 con 0x5597924b41b0 2024-05-10T11:27:16.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.749+0000 7f9317fff700 1 -- 172.21.15.100:0/1012053567 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f932400b790 con 0x5597924b41b0 2024-05-10T11:27:16.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.749+0000 7f933f575b80 1 -- 172.21.15.100:0/1012053567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5597924b41b0 msgr2=0x5597924ba740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:16.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.749+0000 7f933f575b80 1 --2- 172.21.15.100:0/1012053567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5597924b41b0 0x5597924ba740 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9324009d10 tx=0x7f9324005900 comp rx=0 tx=0).stop 2024-05-10T11:27:16.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.749+0000 7f933f575b80 1 -- 172.21.15.100:0/1012053567 shutdown_connections 2024-05-10T11:27:16.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.749+0000 7f933f575b80 1 --2- 172.21.15.100:0/1012053567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5597924b4ad0 0x5597924c4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:16.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.749+0000 7f933f575b80 1 --2- 172.21.15.100:0/1012053567 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5597924aed50 0x5597924ae810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:16.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.749+0000 7f933f575b80 1 --2- 172.21.15.100:0/1012053567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5597924b41b0 0x5597924ba740 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:16.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.749+0000 7f933f575b80 1 -- 172.21.15.100:0/1012053567 >> 172.21.15.100:0/1012053567 conn(0x5597923dab70 msgr2=0x5597924b63d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:27:16.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.750+0000 7f933f575b80 1 -- 172.21.15.100:0/1012053567 shutdown_connections 2024-05-10T11:27:16.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.750+0000 7f933f575b80 1 -- 172.21.15.100:0/1012053567 wait complete. 2024-05-10T11:27:16.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.750+0000 7f933f575b80 1 -- start start 2024-05-10T11:27:16.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.750+0000 7f933f575b80 1 Processor -- start 2024-05-10T11:27:16.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.750+0000 7f933f575b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5597924aed50 0x5597924d0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:16.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.750+0000 7f933f575b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5597924b41b0 0x5597924c0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:16.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.750+0000 7f933f575b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5597924b4ad0 0x5597924c0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:16.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.750+0000 7f933f575b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f93301ab200 con 0x5597924b41b0 2024-05-10T11:27:16.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.750+0000 7f933f575b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f93301ac670 con 0x5597924aed50 2024-05-10T11:27:16.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.750+0000 7f933f575b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f93301ab8d0 con 0x5597924b4ad0 2024-05-10T11:27:16.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.750+0000 7f933f575b80 10 client.0 _ll_register_callbacks cb 0x5597923945b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:27:16.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.750+0000 7f932dd9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5597924b41b0 0x5597924c0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:16.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.750+0000 7f932dd9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5597924b41b0 0x5597924c0800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45698/0 (socket says 172.21.15.100:45698) 2024-05-10T11:27:16.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.750+0000 7f932dd9b700 1 -- 172.21.15.100:0/3851814801 learned_addr learned my addr 172.21.15.100:0/3851814801 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:27:16.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.751+0000 7f932ed9d700 1 --2- 172.21.15.100:0/3851814801 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5597924b4ad0 0x5597924c0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:16.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.751+0000 7f932dd9b700 1 -- 172.21.15.100:0/3851814801 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5597924b4ad0 msgr2=0x5597924c0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:16.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.751+0000 7f932dd9b700 1 --2- 172.21.15.100:0/3851814801 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5597924b4ad0 0x5597924c0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:16.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.751+0000 7f932dd9b700 1 -- 172.21.15.100:0/3851814801 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5597924aed50 msgr2=0x5597924d0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:16.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.751+0000 7f932dd9b700 1 --2- 172.21.15.100:0/3851814801 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5597924aed50 0x5597924d0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:16.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.751+0000 7f932dd9b700 1 -- 172.21.15.100:0/3851814801 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597924cf0a0 con 0x5597924b41b0 2024-05-10T11:27:16.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.751+0000 7f932ed9d700 1 --2- 172.21.15.100:0/3851814801 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5597924b4ad0 0x5597924c0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:27:16.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.751+0000 7f932dd9b700 1 --2- 172.21.15.100:0/3851814801 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5597924b41b0 0x5597924c0800 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f9324000c00 tx=0x7f9324005330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d3d694ca0a39960 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:27:16.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.751+0000 7f9315ffb700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:27:16.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.751+0000 7f9315ffb700 1 -- 172.21.15.100:0/3851814801 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9324028040 con 0x5597924b41b0 2024-05-10T11:27:16.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.751+0000 7f9315ffb700 1 -- 172.21.15.100:0/3851814801 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f932400b540 con 0x5597924b41b0 2024-05-10T11:27:16.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.751+0000 7f933f575b80 1 -- 172.21.15.100:0/3851814801 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9324029040 con 0x5597924b41b0 2024-05-10T11:27:16.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.751+0000 7f9315ffb700 1 -- 172.21.15.100:0/3851814801 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9324044050 con 0x5597924b41b0 2024-05-10T11:27:16.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.751+0000 7f933f575b80 10 client.17169 fetch_fsmap learned FSMap version 31 2024-05-10T11:27:16.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.751+0000 7f933f575b80 1 -- 172.21.15.100:0/3851814801 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559792531f50 con 0x5597924b41b0 2024-05-10T11:27:16.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.751+0000 7f9315ffb700 1 -- 172.21.15.100:0/3851814801 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9324029040 con 0x5597924b41b0 2024-05-10T11:27:16.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.752+0000 7f9315ffb700 1 -- 172.21.15.100:0/3851814801 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f932400f8b0 con 0x5597924b41b0 2024-05-10T11:27:16.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.752+0000 7f933f575b80 10 client.17169 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:27:16.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.752+0000 7f933f575b80 10 client.17169 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-10T11:27:16.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.752+0000 7f933f575b80 20 client.17169 populate_metadata read hostname 'smithi100' 2024-05-10T11:27:16.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.752+0000 7f933f575b80 1 --2- 172.21.15.100:0/3851814801 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x559792532120 0x559792534510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:16.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.752+0000 7f933f575b80 4 client.17169 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:27:16.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.752+0000 7f933f575b80 1 -- 172.21.15.100:0/3851814801 --> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5597924083c0 con 0x559792532120 2024-05-10T11:27:16.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.752+0000 7f932e59c700 1 --2- 172.21.15.100:0/3851814801 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x559792532120 0x559792534510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:16.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.753+0000 7f9315ffb700 10 client.17169 ms_handle_connect on v2:172.21.15.178:6838/3760985404 2024-05-10T11:27:16.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.753+0000 7f932e59c700 1 --2- 172.21.15.100:0/3851814801 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x559792532120 0x559792534510 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=39076bfcc8686bee server_cookie=fc5e687042bf9706 in_seq=0 out_seq=0 2024-05-10T11:27:16.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.753+0000 7f9315ffb700 1 -- 172.21.15.100:0/3851814801 <== mds.1 v2:172.21.15.178:6838/3760985404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5597924083c0 con 0x559792532120 2024-05-10T11:27:16.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.753+0000 7f9315ffb700 10 client.17169 handle_command_reply: tid=0 2024-05-10T11:27:16.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.753+0000 7f933f575b80 1 client.17169 shutdown 2024-05-10T11:27:16.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.753+0000 7f933f575b80 1 -- 172.21.15.100:0/3851814801 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5597924b41b0 msgr2=0x5597924c0800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:16.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.753+0000 7f933f575b80 1 --2- 172.21.15.100:0/3851814801 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5597924b41b0 0x5597924c0800 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f9324000c00 tx=0x7f9324005330 comp rx=0 tx=0).stop 2024-05-10T11:27:16.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.753+0000 7f933f575b80 1 -- 172.21.15.100:0/3851814801 shutdown_connections 2024-05-10T11:27:16.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.753+0000 7f933f575b80 1 --2- 172.21.15.100:0/3851814801 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x559792532120 0x559792534510 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:16.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.753+0000 7f933f575b80 1 --2- 172.21.15.100:0/3851814801 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5597924b4ad0 0x5597924c0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:16.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.753+0000 7f933f575b80 1 --2- 172.21.15.100:0/3851814801 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5597924aed50 0x5597924d0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:16.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.753+0000 7f933f575b80 1 --2- 172.21.15.100:0/3851814801 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5597924b41b0 0x5597924c0800 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:16.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.753+0000 7f9315ffb700 0 client.17169 ms_handle_reset on v2:172.21.15.178:6838/3760985404 2024-05-10T11:27:16.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.753+0000 7f933f575b80 1 -- 172.21.15.100:0/3851814801 >> 172.21.15.100:0/3851814801 conn(0x5597923dab70 msgr2=0x5597924b2c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:27:16.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.753+0000 7f933f575b80 1 -- 172.21.15.100:0/3851814801 shutdown_connections 2024-05-10T11:27:16.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.753+0000 7f933f575b80 1 -- 172.21.15.100:0/3851814801 wait complete. 2024-05-10T11:27:16.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.753+0000 7f933f575b80 20 client.17169 trim_cache size 0 max 16384 2024-05-10T11:27:16.758 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:27:16.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.754+0000 7f9334ce2700 1 -- 172.21.15.100:0/2577848259 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f93180b2880 msgr2=0x7f93180b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:16.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.754+0000 7f9334ce2700 1 --2- 172.21.15.100:0/2577848259 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f93180b2880 0x7f93180b4d40 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9320005950 tx=0x7f932000f820 comp rx=0 tx=0).stop 2024-05-10T11:27:16.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.754+0000 7f9334ce2700 1 -- 172.21.15.100:0/2577848259 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9330107760 msgr2=0x7f93301997b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:16.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.754+0000 7f9334ce2700 1 --2- 172.21.15.100:0/2577848259 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9330107760 0x7f93301997b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f932800ef30 tx=0x7f932800c3a0 comp rx=0 tx=0).stop 2024-05-10T11:27:16.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.754+0000 7f9334ce2700 1 -- 172.21.15.100:0/2577848259 shutdown_connections 2024-05-10T11:27:16.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.754+0000 7f9334ce2700 1 --2- 172.21.15.100:0/2577848259 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f93180b2880 0x7f93180b4d40 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:16.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.754+0000 7f9334ce2700 1 --2- 172.21.15.100:0/2577848259 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9330107760 0x7f93301997b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:16.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.754+0000 7f9334ce2700 1 --2- 172.21.15.100:0/2577848259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9330102e00 0x7f933019f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:16.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.754+0000 7f9334ce2700 1 --2- 172.21.15.100:0/2577848259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93301037b0 0x7f933019fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:16.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.754+0000 7f9334ce2700 1 -- 172.21.15.100:0/2577848259 >> 172.21.15.100:0/2577848259 conn(0x7f9330078a10 msgr2=0x7f9330101c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:27:16.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.754+0000 7f9334ce2700 1 -- 172.21.15.100:0/2577848259 shutdown_connections 2024-05-10T11:27:16.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:16.754+0000 7f9334ce2700 1 -- 172.21.15.100:0/2577848259 wait complete. 2024-05-10T11:27:16.977 INFO:tasks.workunit.client.0.smithi100.stdout: 1 222986 2.75 MB/sec execute 311 sec latency 15.715 ms 2024-05-10T11:27:17.517 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-10T11:27:17.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:27:17 smithi178 ceph-mon[33359]: pgmap v687: 129 pgs: 129 active+clean; 93 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 1.4 MiB/s wr, 167 op/s 2024-05-10T11:27:17.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:27:17 smithi188 ceph-mon[34075]: pgmap v687: 129 pgs: 129 active+clean; 93 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 1.4 MiB/s wr, 167 op/s 2024-05-10T11:27:17.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:27:17 smithi100 ceph-mon[30296]: pgmap v687: 129 pgs: 129 active+clean; 93 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 1.4 MiB/s wr, 167 op/s 2024-05-10T11:27:17.788 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:27:17.977 INFO:tasks.workunit.client.0.smithi100.stdout: 1 223582 2.76 MB/sec execute 312 sec latency 17.729 ms 2024-05-10T11:27:18.977 INFO:tasks.workunit.client.0.smithi100.stdout: 1 223871 2.75 MB/sec execute 313 sec latency 105.889 ms 2024-05-10T11:27:19.664 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:27:19 smithi100 ceph-mon[30296]: pgmap v688: 129 pgs: 129 active+clean; 92 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 1.3 MiB/s wr, 136 op/s 2024-05-10T11:27:19.664 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:27:19 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:27:19.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.675+0000 7fc5cae3b700 1 -- 172.21.15.100:0/2872294295 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5c4102d50 msgr2=0x7fc5c410f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:19.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.675+0000 7fc5cae3b700 1 --2- 172.21.15.100:0/2872294295 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5c4102d50 0x7fc5c410f240 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc00b0a0 tx=0x7fc5bc00b3b0 comp rx=0 tx=0).stop 2024-05-10T11:27:19.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.675+0000 7fc5cae3b700 1 -- 172.21.15.100:0/2872294295 shutdown_connections 2024-05-10T11:27:19.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.675+0000 7fc5cae3b700 1 --2- 172.21.15.100:0/2872294295 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5c4102d50 0x7fc5c410f240 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:19.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.675+0000 7fc5cae3b700 1 --2- 172.21.15.100:0/2872294295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5c4108400 0x7fc5c41087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:19.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.675+0000 7fc5cae3b700 1 --2- 172.21.15.100:0/2872294295 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc5c4102390 0x7fc5c4102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:19.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.675+0000 7fc5cae3b700 1 -- 172.21.15.100:0/2872294295 >> 172.21.15.100:0/2872294295 conn(0x7fc5c4079310 msgr2=0x7fc5c4079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:27:19.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.675+0000 7fc5cae3b700 1 -- 172.21.15.100:0/2872294295 shutdown_connections 2024-05-10T11:27:19.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.675+0000 7fc5cae3b700 1 -- 172.21.15.100:0/2872294295 wait complete. 2024-05-10T11:27:19.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.676+0000 7fc5cae3b700 1 Processor -- start 2024-05-10T11:27:19.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.676+0000 7fc5cae3b700 1 -- start start 2024-05-10T11:27:19.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.676+0000 7fc5cae3b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc5c4102390 0x7fc5c4104070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:19.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.676+0000 7fc5cae3b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5c4102d50 0x7fc5c41045b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:19.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.676+0000 7fc5cae3b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5c4108400 0x7fc5c4104af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:19.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.676+0000 7fc5cae3b700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c41123d0 con 0x7fc5c4108400 2024-05-10T11:27:19.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.676+0000 7fc5cae3b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c4112250 con 0x7fc5c4102390 2024-05-10T11:27:19.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.676+0000 7fc5cae3b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c4112550 con 0x7fc5c4102d50 2024-05-10T11:27:19.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.676+0000 7fc5c93d8700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5c4108400 0x7fc5c4104af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:19.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.676+0000 7fc5c3fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5c4102d50 0x7fc5c41045b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:19.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.676+0000 7fc5c93d8700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5c4108400 0x7fc5c4104af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45728/0 (socket says 172.21.15.100:45728) 2024-05-10T11:27:19.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.676+0000 7fc5c93d8700 1 -- 172.21.15.100:0/4178176083 learned_addr learned my addr 172.21.15.100:0/4178176083 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:27:19.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.676+0000 7fc5c8bd7700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc5c4102390 0x7fc5c4104070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:19.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.677+0000 7fc5c8bd7700 1 -- 172.21.15.100:0/4178176083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5c4102d50 msgr2=0x7fc5c41045b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:19.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.677+0000 7fc5c8bd7700 1 --2- 172.21.15.100:0/4178176083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5c4102d50 0x7fc5c41045b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:19.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.677+0000 7fc5c8bd7700 1 -- 172.21.15.100:0/4178176083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5c4108400 msgr2=0x7fc5c4104af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:19.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.677+0000 7fc5c8bd7700 1 --2- 172.21.15.100:0/4178176083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5c4108400 0x7fc5c4104af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:19.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.677+0000 7fc5c8bd7700 1 -- 172.21.15.100:0/4178176083 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5c41a7f40 con 0x7fc5c4102390 2024-05-10T11:27:19.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.677+0000 7fc5c8bd7700 1 --2- 172.21.15.100:0/4178176083 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc5c4102390 0x7fc5c4104070 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc5b800e9f0 tx=0x7fc5b800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:27:19.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.677+0000 7fc5c93d8700 1 --2- 172.21.15.100:0/4178176083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5c4108400 0x7fc5c4104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:27:19.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.677+0000 7fc5c1ffb700 1 -- 172.21.15.100:0/4178176083 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b800cb70 con 0x7fc5c4102390 2024-05-10T11:27:19.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.677+0000 7fc5cae3b700 1 -- 172.21.15.100:0/4178176083 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5c41a81d0 con 0x7fc5c4102390 2024-05-10T11:27:19.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.677+0000 7fc5c1ffb700 1 -- 172.21.15.100:0/4178176083 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc5b800ee90 con 0x7fc5c4102390 2024-05-10T11:27:19.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.678+0000 7fc5c1ffb700 1 -- 172.21.15.100:0/4178176083 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b8003c30 con 0x7fc5c4102390 2024-05-10T11:27:19.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.678+0000 7fc5cae3b700 1 -- 172.21.15.100:0/4178176083 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5c41a8760 con 0x7fc5c4102390 2024-05-10T11:27:19.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.680+0000 7fc5c1ffb700 1 -- 172.21.15.100:0/4178176083 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fc5b8003ed0 con 0x7fc5c4102390 2024-05-10T11:27:19.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.681+0000 7fc5c1ffb700 1 --2- 172.21.15.100:0/4178176083 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc5b40b2880 0x7fc5b40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:19.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.681+0000 7fc5c3fff700 1 --2- 172.21.15.100:0/4178176083 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc5b40b2880 0x7fc5b40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:19.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.682+0000 7fc5c1ffb700 1 -- 172.21.15.100:0/4178176083 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fc5b80e2600 con 0x7fc5c4102390 2024-05-10T11:27:19.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.682+0000 7fc5d56ceb80 1 Processor -- start 2024-05-10T11:27:19.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.682+0000 7fc5c3fff700 1 --2- 172.21.15.100:0/4178176083 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc5b40b2880 0x7fc5b40b4d40 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc5c40798a0 tx=0x7fc5b000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:27:19.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.682+0000 7fc5d56ceb80 1 -- start start 2024-05-10T11:27:19.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.682+0000 7fc5d56ceb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599b691d1b0 0x5599b691d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:19.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.682+0000 7fc5d56ceb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599b691dad0 0x5599b6917810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:19.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.682+0000 7fc5d56ceb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599b6917d50 0x5599b6918190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:19.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.682+0000 7fc5d56ceb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5599b6869940 con 0x5599b691dad0 2024-05-10T11:27:19.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.682+0000 7fc5d56ceb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5599b67ceda0 con 0x5599b6917d50 2024-05-10T11:27:19.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.682+0000 7fc5d56ceb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5599b6865900 con 0x5599b691d1b0 2024-05-10T11:27:19.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.682+0000 7fc5c3fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599b691dad0 0x5599b6917810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:19.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.682+0000 7fc5c8bd7700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599b691d1b0 0x5599b691d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:19.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.682+0000 7fc5c3fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599b691dad0 0x5599b6917810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45730/0 (socket says 172.21.15.100:45730) 2024-05-10T11:27:19.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.682+0000 7fc5c3fff700 1 -- 172.21.15.100:0/705844801 learned_addr learned my addr 172.21.15.100:0/705844801 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:27:19.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.682+0000 7fc5c93d8700 1 --2- 172.21.15.100:0/705844801 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599b6917d50 0x5599b6918190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:19.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.683+0000 7fc5c8bd7700 1 -- 172.21.15.100:0/705844801 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599b6917d50 msgr2=0x5599b6918190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:19.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.683+0000 7fc5c8bd7700 1 --2- 172.21.15.100:0/705844801 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599b6917d50 0x5599b6918190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:19.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.683+0000 7fc5c8bd7700 1 -- 172.21.15.100:0/705844801 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599b691dad0 msgr2=0x5599b6917810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:19.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.683+0000 7fc5c8bd7700 1 --2- 172.21.15.100:0/705844801 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599b691dad0 0x5599b6917810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:19.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.683+0000 7fc5c8bd7700 1 -- 172.21.15.100:0/705844801 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599b67b68d0 con 0x5599b691d1b0 2024-05-10T11:27:19.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.683+0000 7fc5c3fff700 1 --2- 172.21.15.100:0/705844801 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599b691dad0 0x5599b6917810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:27:19.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.683+0000 7fc5c8bd7700 1 --2- 172.21.15.100:0/705844801 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599b691d1b0 0x5599b691d590 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc5b80a6b30 tx=0x7fc5b8004660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e218a8c779829a39 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:27:19.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.683+0000 7fc5aeffd700 1 -- 172.21.15.100:0/705844801 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b80b2050 con 0x5599b691d1b0 2024-05-10T11:27:19.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.683+0000 7fc5aeffd700 1 -- 172.21.15.100:0/705844801 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc5b80a9fc0 con 0x5599b691d1b0 2024-05-10T11:27:19.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.683+0000 7fc5aeffd700 1 -- 172.21.15.100:0/705844801 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b80cc0e0 con 0x5599b691d1b0 2024-05-10T11:27:19.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.684+0000 7fc5d56ceb80 1 -- 172.21.15.100:0/705844801 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599b691d1b0 msgr2=0x5599b691d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:19.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.684+0000 7fc5d56ceb80 1 --2- 172.21.15.100:0/705844801 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599b691d1b0 0x5599b691d590 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc5b80a6b30 tx=0x7fc5b8004660 comp rx=0 tx=0).stop 2024-05-10T11:27:19.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.684+0000 7fc5d56ceb80 1 -- 172.21.15.100:0/705844801 shutdown_connections 2024-05-10T11:27:19.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.684+0000 7fc5d56ceb80 1 --2- 172.21.15.100:0/705844801 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599b6917d50 0x5599b6918190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:19.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.684+0000 7fc5d56ceb80 1 --2- 172.21.15.100:0/705844801 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599b691d1b0 0x5599b691d590 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:19.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.684+0000 7fc5d56ceb80 1 --2- 172.21.15.100:0/705844801 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599b691dad0 0x5599b6917810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:19.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.684+0000 7fc5d56ceb80 1 -- 172.21.15.100:0/705844801 >> 172.21.15.100:0/705844801 conn(0x5599b6843b70 msgr2=0x5599b69122a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:27:19.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.684+0000 7fc5d56ceb80 1 -- 172.21.15.100:0/705844801 shutdown_connections 2024-05-10T11:27:19.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.684+0000 7fc5d56ceb80 1 -- 172.21.15.100:0/705844801 wait complete. 2024-05-10T11:27:19.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.685+0000 7fc5d56ceb80 1 -- start start 2024-05-10T11:27:19.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.685+0000 7fc5d56ceb80 1 Processor -- start 2024-05-10T11:27:19.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.685+0000 7fc5d56ceb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599b6917d50 0x5599b69237c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:19.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.685+0000 7fc5d56ceb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599b691d1b0 0x5599b6920b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:19.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.685+0000 7fc5d56ceb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599b691dad0 0x5599b69210a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:19.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.685+0000 7fc5d56ceb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5599b6924450 con 0x5599b6917d50 2024-05-10T11:27:19.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.685+0000 7fc5d56ceb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5599b6822ed0 con 0x5599b691d1b0 2024-05-10T11:27:19.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.685+0000 7fc5d56ceb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5599b69245d0 con 0x5599b691dad0 2024-05-10T11:27:19.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.685+0000 7fc5d56ceb80 10 client.0 _ll_register_callbacks cb 0x5599b67fd5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:27:19.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.685+0000 7fc5c8bd7700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599b6917d50 0x5599b69237c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:19.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.685+0000 7fc5c8bd7700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599b6917d50 0x5599b69237c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45738/0 (socket says 172.21.15.100:45738) 2024-05-10T11:27:19.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.685+0000 7fc5c8bd7700 1 -- 172.21.15.100:0/3175798060 learned_addr learned my addr 172.21.15.100:0/3175798060 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:27:19.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.685+0000 7fc5c93d8700 1 --2- 172.21.15.100:0/3175798060 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599b691dad0 0x5599b69210a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:19.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.686+0000 7fc5c3fff700 1 --2- 172.21.15.100:0/3175798060 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599b691d1b0 0x5599b6920b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:19.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.686+0000 7fc5c93d8700 1 -- 172.21.15.100:0/3175798060 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599b691d1b0 msgr2=0x5599b6920b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:19.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.686+0000 7fc5c93d8700 1 --2- 172.21.15.100:0/3175798060 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599b691d1b0 0x5599b6920b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:19.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.686+0000 7fc5c93d8700 1 -- 172.21.15.100:0/3175798060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599b6917d50 msgr2=0x5599b69237c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:19.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.686+0000 7fc5c93d8700 1 --2- 172.21.15.100:0/3175798060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599b6917d50 0x5599b69237c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:19.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.686+0000 7fc5c93d8700 1 -- 172.21.15.100:0/3175798060 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599b6820760 con 0x5599b691dad0 2024-05-10T11:27:19.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.686+0000 7fc5c8bd7700 1 --2- 172.21.15.100:0/3175798060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599b6917d50 0x5599b69237c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:27:19.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.686+0000 7fc5c3fff700 1 --2- 172.21.15.100:0/3175798060 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599b691d1b0 0x5599b6920b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:27:19.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.686+0000 7fc5c93d8700 1 --2- 172.21.15.100:0/3175798060 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599b691dad0 0x5599b69210a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc0062a0 tx=0x7fc5bc003c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d5fd63108516a22 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:27:19.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.686+0000 7fc5a3fff700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:27:19.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.686+0000 7fc5a3fff700 1 -- 172.21.15.100:0/3175798060 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc0079b0 con 0x5599b691dad0 2024-05-10T11:27:19.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.686+0000 7fc5d56ceb80 1 -- 172.21.15.100:0/3175798060 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5b80a9fc0 con 0x5599b691dad0 2024-05-10T11:27:19.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.686+0000 7fc5a3fff700 1 -- 172.21.15.100:0/3175798060 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fc5bc007b50 con 0x5599b691dad0 2024-05-10T11:27:19.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.686+0000 7fc5a3fff700 1 -- 172.21.15.100:0/3175798060 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc5bc007d90 con 0x5599b691dad0 2024-05-10T11:27:19.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.686+0000 7fc5d56ceb80 10 client.26647 fetch_fsmap learned FSMap version 31 2024-05-10T11:27:19.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.686+0000 7fc5d56ceb80 1 -- 172.21.15.100:0/3175798060 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5599b692ed80 con 0x5599b691dad0 2024-05-10T11:27:19.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.687+0000 7fc5a3fff700 1 -- 172.21.15.100:0/3175798060 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc5b80a9fc0 con 0x5599b691dad0 2024-05-10T11:27:19.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.687+0000 7fc5a3fff700 1 -- 172.21.15.100:0/3175798060 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fc5bc023070 con 0x5599b691dad0 2024-05-10T11:27:19.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.687+0000 7fc5d56ceb80 10 client.26647 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:27:19.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.687+0000 7fc5d56ceb80 10 client.26647 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:27:19.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.687+0000 7fc5d56ceb80 20 client.26647 populate_metadata read hostname 'smithi100' 2024-05-10T11:27:19.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.687+0000 7fc5d56ceb80 1 --2- 172.21.15.100:0/3175798060 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5599b6931310 0x5599b6933730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:19.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.687+0000 7fc5d56ceb80 4 client.26647 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:27:19.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.687+0000 7fc5d56ceb80 1 -- 172.21.15.100:0/3175798060 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599b6877320 con 0x5599b6931310 2024-05-10T11:27:19.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.687+0000 7fc5c8bd7700 1 --2- 172.21.15.100:0/3175798060 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5599b6931310 0x5599b6933730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:19.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.687+0000 7fc5a3fff700 10 client.26647 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:27:19.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.688+0000 7fc5c8bd7700 1 --2- 172.21.15.100:0/3175798060 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5599b6931310 0x5599b6933730 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6944d392262a926d server_cookie=a0fca5fe51fba7be in_seq=0 out_seq=0 2024-05-10T11:27:19.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.688+0000 7fc5a3fff700 1 -- 172.21.15.100:0/3175798060 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5599b6877320 con 0x5599b6931310 2024-05-10T11:27:19.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.688+0000 7fc5a3fff700 10 client.26647 handle_command_reply: tid=0 2024-05-10T11:27:19.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.688+0000 7fc5d56ceb80 1 client.26647 shutdown 2024-05-10T11:27:19.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.689+0000 7fc5d56ceb80 1 -- 172.21.15.100:0/3175798060 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599b691dad0 msgr2=0x5599b69210a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:19.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.689+0000 7fc5d56ceb80 1 --2- 172.21.15.100:0/3175798060 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599b691dad0 0x5599b69210a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc0062a0 tx=0x7fc5bc003c80 comp rx=0 tx=0).stop 2024-05-10T11:27:19.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.689+0000 7fc5d56ceb80 1 -- 172.21.15.100:0/3175798060 shutdown_connections 2024-05-10T11:27:19.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.689+0000 7fc5d56ceb80 1 --2- 172.21.15.100:0/3175798060 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5599b6931310 0x5599b6933730 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:19.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.689+0000 7fc5a3fff700 0 client.26647 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:27:19.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.689+0000 7fc5d56ceb80 1 --2- 172.21.15.100:0/3175798060 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599b691dad0 0x5599b69210a0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:19.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.689+0000 7fc5d56ceb80 1 --2- 172.21.15.100:0/3175798060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599b6917d50 0x5599b69237c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:19.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.689+0000 7fc5d56ceb80 1 --2- 172.21.15.100:0/3175798060 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599b691d1b0 0x5599b6920b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:19.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.689+0000 7fc5d56ceb80 1 -- 172.21.15.100:0/3175798060 >> 172.21.15.100:0/3175798060 conn(0x5599b6843b70 msgr2=0x5599b690cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:27:19.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.689+0000 7fc5d56ceb80 1 -- 172.21.15.100:0/3175798060 shutdown_connections 2024-05-10T11:27:19.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.689+0000 7fc5d56ceb80 1 -- 172.21.15.100:0/3175798060 wait complete. 2024-05-10T11:27:19.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.689+0000 7fc5d56ceb80 20 client.26647 trim_cache size 0 max 16384 2024-05-10T11:27:19.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.707+0000 7fc5d56ceb80 1 Processor -- start 2024-05-10T11:27:19.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.707+0000 7fc5d56ceb80 1 -- start start 2024-05-10T11:27:19.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.707+0000 7fc5d56ceb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599b6917d50 0x5599b6917810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:19.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.707+0000 7fc5d56ceb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599b691d1b0 0x5599b6921230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:19.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.707+0000 7fc5d56ceb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599b691dad0 0x5599b692d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:19.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.707+0000 7fc5d56ceb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c41b3a10 con 0x5599b691dad0 2024-05-10T11:27:19.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.707+0000 7fc5d56ceb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c41b12a0 con 0x5599b691d1b0 2024-05-10T11:27:19.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.707+0000 7fc5d56ceb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c41af0f0 con 0x5599b6917d50 2024-05-10T11:27:19.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.707+0000 7fc5c93d8700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599b691dad0 0x5599b692d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:19.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.707+0000 7fc5c3fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599b691d1b0 0x5599b6921230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:19.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.707+0000 7fc5c3fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599b691d1b0 0x5599b6921230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:48412/0 (socket says 172.21.15.100:48412) 2024-05-10T11:27:19.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.707+0000 7fc5c3fff700 1 -- 172.21.15.100:0/2856669036 learned_addr learned my addr 172.21.15.100:0/2856669036 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:27:19.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.707+0000 7fc5c8bd7700 1 --2- 172.21.15.100:0/2856669036 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599b6917d50 0x5599b6917810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:19.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.708+0000 7fc5c3fff700 1 -- 172.21.15.100:0/2856669036 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599b6917d50 msgr2=0x5599b6917810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:19.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.708+0000 7fc5c3fff700 1 --2- 172.21.15.100:0/2856669036 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599b6917d50 0x5599b6917810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:19.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.708+0000 7fc5c3fff700 1 -- 172.21.15.100:0/2856669036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599b691dad0 msgr2=0x5599b692d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:19.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.708+0000 7fc5c3fff700 1 --2- 172.21.15.100:0/2856669036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599b691dad0 0x5599b692d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:19.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.708+0000 7fc5c3fff700 1 -- 172.21.15.100:0/2856669036 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599b691d7e0 con 0x5599b691d1b0 2024-05-10T11:27:19.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.708+0000 7fc5c93d8700 1 --2- 172.21.15.100:0/2856669036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599b691dad0 0x5599b692d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:27:19.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.708+0000 7fc5c3fff700 1 --2- 172.21.15.100:0/2856669036 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599b691d1b0 0x5599b6921230 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x5599b6913040 tx=0x7fc5b0013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73ef969d1d55e14 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:27:19.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.708+0000 7fc5ae7fc700 1 -- 172.21.15.100:0/2856669036 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b0020be0 con 0x5599b691d1b0 2024-05-10T11:27:19.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.708+0000 7fc5ae7fc700 1 -- 172.21.15.100:0/2856669036 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc5b0020d80 con 0x5599b691d1b0 2024-05-10T11:27:19.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.708+0000 7fc5ae7fc700 1 -- 172.21.15.100:0/2856669036 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b00116a0 con 0x5599b691d1b0 2024-05-10T11:27:19.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.709+0000 7fc5d56ceb80 1 -- 172.21.15.100:0/2856669036 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599b691d1b0 msgr2=0x5599b6921230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:19.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.709+0000 7fc5d56ceb80 1 --2- 172.21.15.100:0/2856669036 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599b691d1b0 0x5599b6921230 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x5599b6913040 tx=0x7fc5b0013000 comp rx=0 tx=0).stop 2024-05-10T11:27:19.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.709+0000 7fc5d56ceb80 1 -- 172.21.15.100:0/2856669036 shutdown_connections 2024-05-10T11:27:19.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.709+0000 7fc5d56ceb80 1 --2- 172.21.15.100:0/2856669036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599b691dad0 0x5599b692d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:19.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.709+0000 7fc5d56ceb80 1 --2- 172.21.15.100:0/2856669036 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599b6917d50 0x5599b6917810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:19.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.709+0000 7fc5d56ceb80 1 --2- 172.21.15.100:0/2856669036 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599b691d1b0 0x5599b6921230 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:19.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.709+0000 7fc5d56ceb80 1 -- 172.21.15.100:0/2856669036 >> 172.21.15.100:0/2856669036 conn(0x5599b6843b70 msgr2=0x5599b6920b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:27:19.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.709+0000 7fc5d56ceb80 1 -- 172.21.15.100:0/2856669036 shutdown_connections 2024-05-10T11:27:19.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.709+0000 7fc5d56ceb80 1 -- 172.21.15.100:0/2856669036 wait complete. 2024-05-10T11:27:19.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.709+0000 7fc5d56ceb80 1 -- start start 2024-05-10T11:27:19.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.709+0000 7fc5d56ceb80 1 Processor -- start 2024-05-10T11:27:19.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.710+0000 7fc5d56ceb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599b6917d50 0x5599b6929ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:19.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.710+0000 7fc5d56ceb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599b691d1b0 0x5599b69381f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:19.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.710+0000 7fc5d56ceb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599b691dad0 0x5599b6938730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:19.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.710+0000 7fc5d56ceb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c41af760 con 0x5599b691d1b0 2024-05-10T11:27:19.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.710+0000 7fc5d56ceb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c41b0bd0 con 0x5599b6917d50 2024-05-10T11:27:19.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.710+0000 7fc5d56ceb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c41afe30 con 0x5599b691dad0 2024-05-10T11:27:19.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.710+0000 7fc5d56ceb80 10 client.0 _ll_register_callbacks cb 0x5599b67fd5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:27:19.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.710+0000 7fc5c8bd7700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599b6917d50 0x5599b6929ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:19.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.710+0000 7fc5c8bd7700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599b6917d50 0x5599b6929ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:48416/0 (socket says 172.21.15.100:48416) 2024-05-10T11:27:19.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.710+0000 7fc5c8bd7700 1 -- 172.21.15.100:0/2399593373 learned_addr learned my addr 172.21.15.100:0/2399593373 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:27:19.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.710+0000 7fc5c3fff700 1 --2- 172.21.15.100:0/2399593373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599b691d1b0 0x5599b69381f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:19.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.710+0000 7fc5c93d8700 1 --2- 172.21.15.100:0/2399593373 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599b691dad0 0x5599b6938730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:19.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.710+0000 7fc5c8bd7700 1 -- 172.21.15.100:0/2399593373 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599b691dad0 msgr2=0x5599b6938730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:19.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.710+0000 7fc5c8bd7700 1 --2- 172.21.15.100:0/2399593373 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599b691dad0 0x5599b6938730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:19.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.710+0000 7fc5c8bd7700 1 -- 172.21.15.100:0/2399593373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599b691d1b0 msgr2=0x5599b69381f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:19.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.710+0000 7fc5c8bd7700 1 --2- 172.21.15.100:0/2399593373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599b691d1b0 0x5599b69381f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:19.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.710+0000 7fc5c8bd7700 1 -- 172.21.15.100:0/2399593373 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599b6938c70 con 0x5599b6917d50 2024-05-10T11:27:19.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.710+0000 7fc5c3fff700 1 --2- 172.21.15.100:0/2399593373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599b691d1b0 0x5599b69381f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:27:19.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.711+0000 7fc5c8bd7700 1 --2- 172.21.15.100:0/2399593373 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599b6917d50 0x5599b6929ab0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc5b80a5940 tx=0x7fc5b8003680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3053610ca2c427fb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:27:19.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.711+0000 7fc5a3fff700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:27:19.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.711+0000 7fc5a3fff700 1 -- 172.21.15.100:0/2399593373 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b80a9fc0 con 0x5599b6917d50 2024-05-10T11:27:19.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.711+0000 7fc5a3fff700 1 -- 172.21.15.100:0/2399593373 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fc5b80e2350 con 0x5599b6917d50 2024-05-10T11:27:19.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.711+0000 7fc5d56ceb80 1 -- 172.21.15.100:0/2399593373 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5b0020d80 con 0x5599b6917d50 2024-05-10T11:27:19.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.711+0000 7fc5a3fff700 1 -- 172.21.15.100:0/2399593373 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc5b80b2720 con 0x5599b6917d50 2024-05-10T11:27:19.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.711+0000 7fc5d56ceb80 10 client.26597 fetch_fsmap learned FSMap version 31 2024-05-10T11:27:19.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.711+0000 7fc5d56ceb80 1 -- 172.21.15.100:0/2399593373 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5599b699b8d0 con 0x5599b6917d50 2024-05-10T11:27:19.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.711+0000 7fc5a3fff700 1 -- 172.21.15.100:0/2399593373 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc5b0020d80 con 0x5599b6917d50 2024-05-10T11:27:19.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.711+0000 7fc5a3fff700 1 -- 172.21.15.100:0/2399593373 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fc5b80a9b20 con 0x5599b6917d50 2024-05-10T11:27:19.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.711+0000 7fc5d56ceb80 10 client.26597 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:27:19.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.711+0000 7fc5d56ceb80 10 client.26597 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:27:19.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.711+0000 7fc5d56ceb80 20 client.26597 populate_metadata read hostname 'smithi100' 2024-05-10T11:27:19.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.712+0000 7fc5d56ceb80 1 --2- 172.21.15.100:0/2399593373 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5599b699bb60 0x5599b699df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:19.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.712+0000 7fc5d56ceb80 4 client.26597 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:27:19.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.712+0000 7fc5d56ceb80 1 -- 172.21.15.100:0/2399593373 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5599b68713c0 con 0x5599b699bb60 2024-05-10T11:27:19.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.712+0000 7fc5c3fff700 1 --2- 172.21.15.100:0/2399593373 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5599b699bb60 0x5599b699df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:19.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.713+0000 7fc5a3fff700 10 client.26597 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:27:19.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.713+0000 7fc5c3fff700 1 --2- 172.21.15.100:0/2399593373 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5599b699bb60 0x5599b699df50 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b6bb2d289a3c620 server_cookie=1ac48ecd51e4cd03 in_seq=0 out_seq=0 2024-05-10T11:27:19.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.713+0000 7fc5a3fff700 1 -- 172.21.15.100:0/2399593373 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5599b68713c0 con 0x5599b699bb60 2024-05-10T11:27:19.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.713+0000 7fc5a3fff700 10 client.26597 handle_command_reply: tid=0 2024-05-10T11:27:19.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.713+0000 7fc5d56ceb80 1 client.26597 shutdown 2024-05-10T11:27:19.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.713+0000 7fc5d56ceb80 1 -- 172.21.15.100:0/2399593373 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599b6917d50 msgr2=0x5599b6929ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:19.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.713+0000 7fc5d56ceb80 1 --2- 172.21.15.100:0/2399593373 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599b6917d50 0x5599b6929ab0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc5b80a5940 tx=0x7fc5b8003680 comp rx=0 tx=0).stop 2024-05-10T11:27:19.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.713+0000 7fc5d56ceb80 1 -- 172.21.15.100:0/2399593373 shutdown_connections 2024-05-10T11:27:19.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.713+0000 7fc5d56ceb80 1 --2- 172.21.15.100:0/2399593373 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5599b699bb60 0x5599b699df50 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:19.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.713+0000 7fc5a3fff700 0 client.26597 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:27:19.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.713+0000 7fc5d56ceb80 1 --2- 172.21.15.100:0/2399593373 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599b691dad0 0x5599b6938730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:19.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.713+0000 7fc5d56ceb80 1 --2- 172.21.15.100:0/2399593373 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599b6917d50 0x5599b6929ab0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:19.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.713+0000 7fc5d56ceb80 1 --2- 172.21.15.100:0/2399593373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599b691d1b0 0x5599b69381f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:19.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.713+0000 7fc5d56ceb80 1 -- 172.21.15.100:0/2399593373 >> 172.21.15.100:0/2399593373 conn(0x5599b6843b70 msgr2=0x5599b691deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:27:19.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.714+0000 7fc5d56ceb80 1 -- 172.21.15.100:0/2399593373 shutdown_connections 2024-05-10T11:27:19.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.714+0000 7fc5d56ceb80 1 -- 172.21.15.100:0/2399593373 wait complete. 2024-05-10T11:27:19.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.714+0000 7fc5d56ceb80 20 client.26597 trim_cache size 0 max 16384 2024-05-10T11:27:19.719 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:27:19.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.714+0000 7fc5cae3b700 1 -- 172.21.15.100:0/4178176083 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc5b40b2880 msgr2=0x7fc5b40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:19.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.715+0000 7fc5cae3b700 1 --2- 172.21.15.100:0/4178176083 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc5b40b2880 0x7fc5b40b4d40 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc5c40798a0 tx=0x7fc5b000b410 comp rx=0 tx=0).stop 2024-05-10T11:27:19.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.715+0000 7fc5cae3b700 1 -- 172.21.15.100:0/4178176083 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc5c4102390 msgr2=0x7fc5c4104070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:19.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.715+0000 7fc5cae3b700 1 --2- 172.21.15.100:0/4178176083 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc5c4102390 0x7fc5c4104070 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc5b800e9f0 tx=0x7fc5b800ed00 comp rx=0 tx=0).stop 2024-05-10T11:27:19.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.715+0000 7fc5cae3b700 1 -- 172.21.15.100:0/4178176083 shutdown_connections 2024-05-10T11:27:19.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.715+0000 7fc5cae3b700 1 --2- 172.21.15.100:0/4178176083 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc5b40b2880 0x7fc5b40b4d40 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:19.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.715+0000 7fc5cae3b700 1 --2- 172.21.15.100:0/4178176083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5c4108400 0x7fc5c4104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:19.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.715+0000 7fc5cae3b700 1 --2- 172.21.15.100:0/4178176083 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc5c4102390 0x7fc5c4104070 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:19.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.715+0000 7fc5cae3b700 1 --2- 172.21.15.100:0/4178176083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5c4102d50 0x7fc5c41045b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:19.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.715+0000 7fc5cae3b700 1 -- 172.21.15.100:0/4178176083 >> 172.21.15.100:0/4178176083 conn(0x7fc5c4079310 msgr2=0x7fc5c41016c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:27:19.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.715+0000 7fc5cae3b700 1 -- 172.21.15.100:0/4178176083 shutdown_connections 2024-05-10T11:27:19.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:19.715+0000 7fc5cae3b700 1 -- 172.21.15.100:0/4178176083 wait complete. 2024-05-10T11:27:19.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:27:19 smithi188 ceph-mon[34075]: pgmap v688: 129 pgs: 129 active+clean; 92 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 1.3 MiB/s wr, 136 op/s 2024-05-10T11:27:19.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:27:19 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:27:19.948 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:27:19 smithi178 ceph-mon[33359]: pgmap v688: 129 pgs: 129 active+clean; 92 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 1.3 MiB/s wr, 136 op/s 2024-05-10T11:27:19.948 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:27:19 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:27:19.977 INFO:tasks.workunit.client.0.smithi100.stdout: 1 224110 2.74 MB/sec execute 314 sec latency 35.064 ms 2024-05-10T11:27:20.977 INFO:tasks.workunit.client.0.smithi100.stdout: 1 224573 2.74 MB/sec execute 315 sec latency 56.168 ms 2024-05-10T11:27:21.495 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T11:27:21.496 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T11:27:21.617 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.617+0000 7fbe3ce93640 1 -- 172.21.15.100:0/3491759399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe38102b00 msgr2=0x7fbe38102f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:21.617 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.617+0000 7fbe3ce93640 1 --2- 172.21.15.100:0/3491759399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe38102b00 0x7fbe38102f80 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c009900 tx=0x7fbe2c02a410 comp rx=0 tx=0).stop 2024-05-10T11:27:21.617 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.617+0000 7fbe3ce93640 1 -- 172.21.15.100:0/3491759399 shutdown_connections 2024-05-10T11:27:21.618 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.617+0000 7fbe3ce93640 1 --2- 172.21.15.100:0/3491759399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe381034c0 0x7fbe38109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:21.618 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.617+0000 7fbe3ce93640 1 --2- 172.21.15.100:0/3491759399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe38102b00 0x7fbe38102f80 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:21.618 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.617+0000 7fbe3ce93640 1 --2- 172.21.15.100:0/3491759399 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe38101910 0x7fbe38101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:21.618 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.617+0000 7fbe3ce93640 1 -- 172.21.15.100:0/3491759399 >> 172.21.15.100:0/3491759399 conn(0x7fbe380fce80 msgr2=0x7fbe380ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:27:21.618 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.617+0000 7fbe3ce93640 1 -- 172.21.15.100:0/3491759399 shutdown_connections 2024-05-10T11:27:21.618 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.617+0000 7fbe3ce93640 1 -- 172.21.15.100:0/3491759399 wait complete. 2024-05-10T11:27:21.620 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.618+0000 7fbe3ce93640 1 Processor -- start 2024-05-10T11:27:21.620 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.618+0000 7fbe3ce93640 1 -- start start 2024-05-10T11:27:21.620 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.618+0000 7fbe3ce93640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe38101910 0x7fbe38197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:21.620 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.618+0000 7fbe3ce93640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe38102b00 0x7fbe38197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:21.620 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.618+0000 7fbe3ce93640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe381034c0 0x7fbe3819efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:21.620 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.618+0000 7fbe3ce93640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbe3810d2d0 con 0x7fbe38102b00 2024-05-10T11:27:21.620 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.618+0000 7fbe3ce93640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbe3810d150 con 0x7fbe381034c0 2024-05-10T11:27:21.620 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.618+0000 7fbe3ce93640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbe3810d450 con 0x7fbe38101910 2024-05-10T11:27:21.620 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.619+0000 7fbe3659c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe38101910 0x7fbe38197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:21.621 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.619+0000 7fbe3659c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe38101910 0x7fbe38197a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:52944/0 (socket says 172.21.15.100:52944) 2024-05-10T11:27:21.621 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.619+0000 7fbe3659c640 1 -- 172.21.15.100:0/901284570 learned_addr learned my addr 172.21.15.100:0/901284570 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:27:21.621 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.619+0000 7fbe35d9b640 1 --2- 172.21.15.100:0/901284570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe38102b00 0x7fbe38197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:21.621 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.619+0000 7fbe36d9d640 1 --2- 172.21.15.100:0/901284570 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe381034c0 0x7fbe3819efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:21.621 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.619+0000 7fbe3659c640 1 -- 172.21.15.100:0/901284570 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe381034c0 msgr2=0x7fbe3819efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:21.621 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.619+0000 7fbe3659c640 1 --2- 172.21.15.100:0/901284570 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe381034c0 0x7fbe3819efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:21.621 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.619+0000 7fbe3659c640 1 -- 172.21.15.100:0/901284570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe38102b00 msgr2=0x7fbe38197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:21.621 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.619+0000 7fbe3659c640 1 --2- 172.21.15.100:0/901284570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe38102b00 0x7fbe38197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:21.621 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.619+0000 7fbe3659c640 1 -- 172.21.15.100:0/901284570 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe3819f620 con 0x7fbe38101910 2024-05-10T11:27:21.621 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.619+0000 7fbe36d9d640 1 --2- 172.21.15.100:0/901284570 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe381034c0 0x7fbe3819efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:27:21.621 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.619+0000 7fbe3659c640 1 --2- 172.21.15.100:0/901284570 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe38101910 0x7fbe38197a50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbe2800e9c0 tx=0x7fbe2800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:27:21.621 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.620+0000 7fbe277fe640 1 -- 172.21.15.100:0/901284570 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe2800cde0 con 0x7fbe38101910 2024-05-10T11:27:21.621 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.620+0000 7fbe277fe640 1 -- 172.21.15.100:0/901284570 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbe280108c0 con 0x7fbe38101910 2024-05-10T11:27:21.621 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.620+0000 7fbe3ce93640 1 -- 172.21.15.100:0/901284570 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe3819f910 con 0x7fbe38101910 2024-05-10T11:27:21.621 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.620+0000 7fbe277fe640 1 -- 172.21.15.100:0/901284570 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe28018820 con 0x7fbe38101910 2024-05-10T11:27:21.622 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.620+0000 7fbe3ce93640 1 -- 172.21.15.100:0/901284570 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe3806a060 con 0x7fbe38101910 2024-05-10T11:27:21.623 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.623+0000 7fbe277fe640 1 -- 172.21.15.100:0/901284570 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fbe2801f020 con 0x7fbe38101910 2024-05-10T11:27:21.623 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.623+0000 7fbe3fbb2740 1 Processor -- start 2024-05-10T11:27:21.623 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.623+0000 7fbe3fbb2740 1 -- start start 2024-05-10T11:27:21.623 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.624+0000 7fbe3fbb2740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556604de8db0 0x556604de91d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:21.626 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.624+0000 7fbe3fbb2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556604de97a0 0x556604dee2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:21.626 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.624+0000 7fbe3fbb2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556604dee910 0x556604df0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:21.626 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.624+0000 7fbe3fbb2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x556604d4d5d0 con 0x556604de97a0 2024-05-10T11:27:21.626 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.624+0000 7fbe3fbb2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556604d06890 con 0x556604dee910 2024-05-10T11:27:21.626 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.625+0000 7fbe3fbb2740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556604d49590 con 0x556604de8db0 2024-05-10T11:27:21.626 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.625+0000 7fbe3659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556604de97a0 0x556604dee2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:21.626 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.625+0000 7fbe3659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556604de97a0 0x556604dee2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45782/0 (socket says 172.21.15.100:45782) 2024-05-10T11:27:21.626 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.625+0000 7fbe3659c640 1 -- 172.21.15.100:0/3387000070 learned_addr learned my addr 172.21.15.100:0/3387000070 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:27:21.626 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.625+0000 7fbe3659c640 1 -- 172.21.15.100:0/3387000070 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556604de8db0 msgr2=0x556604de91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:21.626 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.624+0000 7fbe35d9b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556604dee910 0x556604df0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:21.626 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.624+0000 7fbe36d9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556604de8db0 0x556604de91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:21.626 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.626+0000 7fbe3659c640 1 --2- 172.21.15.100:0/3387000070 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556604de8db0 0x556604de91d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:21.626 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.626+0000 7fbe3659c640 1 -- 172.21.15.100:0/3387000070 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556604dee910 msgr2=0x556604df0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:21.627 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.626+0000 7fbe3659c640 1 --2- 172.21.15.100:0/3387000070 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556604dee910 0x556604df0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:21.627 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.626+0000 7fbe3659c640 1 -- 172.21.15.100:0/3387000070 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556604a5f210 con 0x556604de97a0 2024-05-10T11:27:21.627 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.626+0000 7fbe35d9b640 1 --2- 172.21.15.100:0/3387000070 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556604dee910 0x556604df0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:27:21.627 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.626+0000 7fbe36d9d640 1 --2- 172.21.15.100:0/3387000070 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556604de8db0 0x556604de91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:27:21.627 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.626+0000 7fbe3659c640 1 --2- 172.21.15.100:0/3387000070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556604de97a0 0x556604dee2a0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fbe280f2650 tx=0x7fbe280f2740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c0ffa3ca9ada64 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:27:21.627 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.626+0000 7fbe277fe640 1 --2- 172.21.15.100:0/901284570 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fbe20004aa0 0x7fbe20004f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:21.630 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.626+0000 7fbe36d9d640 1 --2- 172.21.15.100:0/901284570 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fbe20004aa0 0x7fbe20004f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:21.631 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.626+0000 7fbe277fe640 1 -- 172.21.15.100:0/901284570 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fbe280e27a0 con 0x7fbe38101910 2024-05-10T11:27:21.631 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.626+0000 7fbe24ff9640 1 -- 172.21.15.100:0/3387000070 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe280a91d0 con 0x556604de97a0 2024-05-10T11:27:21.631 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.627+0000 7fbe24ff9640 1 -- 172.21.15.100:0/3387000070 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbe280113c0 con 0x556604de97a0 2024-05-10T11:27:21.631 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.627+0000 7fbe24ff9640 1 -- 172.21.15.100:0/3387000070 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe280a93b0 con 0x556604de97a0 2024-05-10T11:27:21.631 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.628+0000 7fbe3fbb2740 1 -- 172.21.15.100:0/3387000070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556604de97a0 msgr2=0x556604dee2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:21.631 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.628+0000 7fbe3fbb2740 1 --2- 172.21.15.100:0/3387000070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556604de97a0 0x556604dee2a0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fbe280f2650 tx=0x7fbe280f2740 comp rx=0 tx=0).stop 2024-05-10T11:27:21.631 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.628+0000 7fbe36d9d640 1 --2- 172.21.15.100:0/901284570 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fbe20004aa0 0x7fbe20004f50 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x556604dea000 tx=0x7fbe30009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:27:21.631 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.628+0000 7fbe3fbb2740 1 -- 172.21.15.100:0/3387000070 shutdown_connections 2024-05-10T11:27:21.631 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.628+0000 7fbe3fbb2740 1 --2- 172.21.15.100:0/3387000070 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556604dee910 0x556604df0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:21.631 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.628+0000 7fbe3fbb2740 1 --2- 172.21.15.100:0/3387000070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556604de97a0 0x556604dee2a0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:21.631 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.628+0000 7fbe3fbb2740 1 --2- 172.21.15.100:0/3387000070 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556604de8db0 0x556604de91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:21.631 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.628+0000 7fbe3fbb2740 1 -- 172.21.15.100:0/3387000070 >> 172.21.15.100:0/3387000070 conn(0x556604de4420 msgr2=0x556604de4820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:27:21.631 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.628+0000 7fbe3fbb2740 1 -- 172.21.15.100:0/3387000070 shutdown_connections 2024-05-10T11:27:21.631 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.628+0000 7fbe3fbb2740 1 -- 172.21.15.100:0/3387000070 wait complete. 2024-05-10T11:27:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.628+0000 7fbe3fbb2740 1 -- start start 2024-05-10T11:27:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.628+0000 7fbe3fbb2740 1 Processor -- start 2024-05-10T11:27:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.629+0000 7fbe3fbb2740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556604de8db0 0x556604debf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.629+0000 7fbe3fbb2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556604dee910 0x556604dec4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.629+0000 7fbe3fbb2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556604ded460 0x556604ded8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.629+0000 7fbe3fbb2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x556604df1780 con 0x556604dee910 2024-05-10T11:27:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.629+0000 7fbe3fbb2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556604d0a370 con 0x556604ded460 2024-05-10T11:27:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.629+0000 7fbe3fbb2740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556604df1900 con 0x556604de8db0 2024-05-10T11:27:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.629+0000 7fbe3fbb2740 10 client.0 _ll_register_callbacks cb 0x556604a5ea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:27:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.629+0000 7fbe3659c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556604de8db0 0x556604debf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.629+0000 7fbe36d9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556604ded460 0x556604ded8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.629+0000 7fbe3659c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556604de8db0 0x556604debf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:52958/0 (socket says 172.21.15.100:52958) 2024-05-10T11:27:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.629+0000 7fbe3659c640 1 -- 172.21.15.100:0/1884873974 learned_addr learned my addr 172.21.15.100:0/1884873974 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:27:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.629+0000 7fbe35d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556604dee910 0x556604dec4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.629+0000 7fbe3659c640 1 -- 172.21.15.100:0/1884873974 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556604ded460 msgr2=0x556604ded8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:21.633 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.629+0000 7fbe3659c640 1 --2- 172.21.15.100:0/1884873974 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556604ded460 0x556604ded8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:21.633 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.629+0000 7fbe3659c640 1 -- 172.21.15.100:0/1884873974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556604dee910 msgr2=0x556604dec4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:21.633 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.629+0000 7fbe3659c640 1 --2- 172.21.15.100:0/1884873974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556604dee910 0x556604dec4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:21.633 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.629+0000 7fbe3659c640 1 -- 172.21.15.100:0/1884873974 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556604b2b620 con 0x556604de8db0 2024-05-10T11:27:21.633 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.629+0000 7fbe3659c640 1 --2- 172.21.15.100:0/1884873974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556604de8db0 0x556604debf60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbe280058c0 tx=0x7fbe280f1fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e03c10070d93ef2b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:27:21.633 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.629+0000 7fbe1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:27:21.633 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.630+0000 7fbe1dffb640 1 -- 172.21.15.100:0/1884873974 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe280a95a0 con 0x556604de8db0 2024-05-10T11:27:21.633 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.630+0000 7fbe1dffb640 1 -- 172.21.15.100:0/1884873974 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fbe28010a70 con 0x556604de8db0 2024-05-10T11:27:21.633 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.630+0000 7fbe3fbb2740 1 -- 172.21.15.100:0/1884873974 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe280113c0 con 0x556604de8db0 2024-05-10T11:27:21.633 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.630+0000 7fbe1dffb640 1 -- 172.21.15.100:0/1884873974 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fbe280a9830 con 0x556604de8db0 2024-05-10T11:27:21.633 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.630+0000 7fbe3fbb2740 10 client.26662 fetch_fsmap learned FSMap version 31 2024-05-10T11:27:21.633 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.630+0000 7fbe3fbb2740 1 -- 172.21.15.100:0/1884873974 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556604df17f0 con 0x556604de8db0 2024-05-10T11:27:21.633 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.630+0000 7fbe1dffb640 1 -- 172.21.15.100:0/1884873974 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbe280113c0 con 0x556604de8db0 2024-05-10T11:27:21.633 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.630+0000 7fbe1dffb640 1 -- 172.21.15.100:0/1884873974 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fbe280a5cf0 con 0x556604de8db0 2024-05-10T11:27:21.634 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.630+0000 7fbe3fbb2740 10 client.26662 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:27:21.634 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.630+0000 7fbe3fbb2740 10 client.26662 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:27:21.634 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.630+0000 7fbe3fbb2740 20 client.26662 populate_metadata read hostname 'smithi100' 2024-05-10T11:27:21.634 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.630+0000 7fbe3fbb2740 1 --2- 172.21.15.100:0/1884873974 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556604dfec50 0x556604e01040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:21.634 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.630+0000 7fbe3fbb2740 4 client.26662 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:27:21.634 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.630+0000 7fbe3fbb2740 1 -- 172.21.15.100:0/1884873974 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556604d5b1e0 con 0x556604dfec50 2024-05-10T11:27:21.634 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.631+0000 7fbe35d9b640 1 --2- 172.21.15.100:0/1884873974 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556604dfec50 0x556604e01040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:21.635 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.631+0000 7fbe1dffb640 10 client.26662 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:27:21.635 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.631+0000 7fbe35d9b640 1 --2- 172.21.15.100:0/1884873974 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556604dfec50 0x556604e01040 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ff845f229916d2c server_cookie=359698170d5c75ea in_seq=0 out_seq=0 2024-05-10T11:27:21.635 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.632+0000 7fbe1dffb640 1 -- 172.21.15.100:0/1884873974 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556604d5b1e0 con 0x556604dfec50 2024-05-10T11:27:21.635 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.632+0000 7fbe1dffb640 10 client.26662 handle_command_reply: tid=0 2024-05-10T11:27:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.632+0000 7fbe3fbb2740 1 client.26662 shutdown 2024-05-10T11:27:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.632+0000 7fbe3fbb2740 1 -- 172.21.15.100:0/1884873974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556604de8db0 msgr2=0x556604debf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.632+0000 7fbe3fbb2740 1 --2- 172.21.15.100:0/1884873974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556604de8db0 0x556604debf60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbe280058c0 tx=0x7fbe280f1fd0 comp rx=0 tx=0).stop 2024-05-10T11:27:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.632+0000 7fbe3fbb2740 1 -- 172.21.15.100:0/1884873974 shutdown_connections 2024-05-10T11:27:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.632+0000 7fbe3fbb2740 1 --2- 172.21.15.100:0/1884873974 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556604dfec50 0x556604e01040 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.632+0000 7fbe3fbb2740 1 --2- 172.21.15.100:0/1884873974 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556604ded460 0x556604ded8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.632+0000 7fbe1dffb640 0 client.26662 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:27:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.632+0000 7fbe3fbb2740 1 --2- 172.21.15.100:0/1884873974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556604dee910 0x556604dec4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.632+0000 7fbe3fbb2740 1 --2- 172.21.15.100:0/1884873974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556604de8db0 0x556604debf60 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.632+0000 7fbe3fbb2740 1 -- 172.21.15.100:0/1884873974 >> 172.21.15.100:0/1884873974 conn(0x556604de4420 msgr2=0x556604de3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:27:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.632+0000 7fbe3fbb2740 1 -- 172.21.15.100:0/1884873974 shutdown_connections 2024-05-10T11:27:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.632+0000 7fbe3fbb2740 1 -- 172.21.15.100:0/1884873974 wait complete. 2024-05-10T11:27:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.632+0000 7fbe3fbb2740 20 client.26662 trim_cache size 0 max 16384 2024-05-10T11:27:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.650+0000 7fbe3fbb2740 1 Processor -- start 2024-05-10T11:27:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.651+0000 7fbe3fbb2740 1 -- start start 2024-05-10T11:27:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.651+0000 7fbe3fbb2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556604de8db0 0x556604de73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.651+0000 7fbe3fbb2740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556604ded460 0x556604de7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.651+0000 7fbe3fbb2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556604dee910 0x556604debb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.651+0000 7fbe3fbb2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbe381ab9c0 con 0x556604dee910 2024-05-10T11:27:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.651+0000 7fbe3fbb2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbe381a9290 con 0x556604de8db0 2024-05-10T11:27:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.651+0000 7fbe3fbb2740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbe381a70e0 con 0x556604ded460 2024-05-10T11:27:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.651+0000 7fbe36d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556604dee910 0x556604debb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.651+0000 7fbe35d9b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556604ded460 0x556604de7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.651+0000 7fbe35d9b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556604ded460 0x556604de7920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:52970/0 (socket says 172.21.15.100:52970) 2024-05-10T11:27:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.651+0000 7fbe36d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556604dee910 0x556604debb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45806/0 (socket says 172.21.15.100:45806) 2024-05-10T11:27:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.651+0000 7fbe35d9b640 1 -- 172.21.15.100:0/3868634790 learned_addr learned my addr 172.21.15.100:0/3868634790 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:27:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.651+0000 7fbe3659c640 1 --2- 172.21.15.100:0/3868634790 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556604de8db0 0x556604de73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.651+0000 7fbe35d9b640 1 -- 172.21.15.100:0/3868634790 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556604de8db0 msgr2=0x556604de73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.651+0000 7fbe35d9b640 1 --2- 172.21.15.100:0/3868634790 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556604de8db0 0x556604de73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.651+0000 7fbe35d9b640 1 -- 172.21.15.100:0/3868634790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556604dee910 msgr2=0x556604debb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.651+0000 7fbe35d9b640 1 --2- 172.21.15.100:0/3868634790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556604dee910 0x556604debb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.651+0000 7fbe35d9b640 1 -- 172.21.15.100:0/3868634790 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556604de4250 con 0x556604ded460 2024-05-10T11:27:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.651+0000 7fbe36d9d640 1 --2- 172.21.15.100:0/3868634790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556604dee910 0x556604debb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:27:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.652+0000 7fbe35d9b640 1 --2- 172.21.15.100:0/3868634790 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556604ded460 0x556604de7920 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c006010 tx=0x7fbe2c004280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4222f77e82c29081 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:27:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.652+0000 7fbe1ffff640 1 -- 172.21.15.100:0/3868634790 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c04a070 con 0x556604ded460 2024-05-10T11:27:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.652+0000 7fbe1ffff640 1 -- 172.21.15.100:0/3868634790 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbe2c031bf0 con 0x556604ded460 2024-05-10T11:27:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.652+0000 7fbe1ffff640 1 -- 172.21.15.100:0/3868634790 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c035040 con 0x556604ded460 2024-05-10T11:27:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.653+0000 7fbe3fbb2740 1 -- 172.21.15.100:0/3868634790 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556604ded460 msgr2=0x556604de7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.653+0000 7fbe3fbb2740 1 --2- 172.21.15.100:0/3868634790 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556604ded460 0x556604de7920 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c006010 tx=0x7fbe2c004280 comp rx=0 tx=0).stop 2024-05-10T11:27:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.653+0000 7fbe3fbb2740 1 -- 172.21.15.100:0/3868634790 shutdown_connections 2024-05-10T11:27:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.653+0000 7fbe3fbb2740 1 --2- 172.21.15.100:0/3868634790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556604dee910 0x556604debb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.653+0000 7fbe3fbb2740 1 --2- 172.21.15.100:0/3868634790 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556604ded460 0x556604de7920 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.653+0000 7fbe3fbb2740 1 --2- 172.21.15.100:0/3868634790 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556604de8db0 0x556604de73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.653+0000 7fbe3fbb2740 1 -- 172.21.15.100:0/3868634790 >> 172.21.15.100:0/3868634790 conn(0x556604de4420 msgr2=0x556604de4800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:27:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.653+0000 7fbe3fbb2740 1 -- 172.21.15.100:0/3868634790 shutdown_connections 2024-05-10T11:27:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.653+0000 7fbe3fbb2740 1 -- 172.21.15.100:0/3868634790 wait complete. 2024-05-10T11:27:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.653+0000 7fbe3fbb2740 1 -- start start 2024-05-10T11:27:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.653+0000 7fbe3fbb2740 1 Processor -- start 2024-05-10T11:27:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.653+0000 7fbe3fbb2740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556604de8db0 0x556604df8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.654+0000 7fbe3fbb2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556604ded460 0x556604df92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.654+0000 7fbe3659c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556604de8db0 0x556604df8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:21.658 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.654+0000 7fbe3659c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556604de8db0 0x556604df8d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:52972/0 (socket says 172.21.15.100:52972) 2024-05-10T11:27:21.658 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.654+0000 7fbe35d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556604ded460 0x556604df92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:21.658 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.654+0000 7fbe35d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556604ded460 0x556604df92c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45818/0 (socket says 172.21.15.100:45818) 2024-05-10T11:27:21.658 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.654+0000 7fbe35d9b640 1 -- 172.21.15.100:0/3476314055 learned_addr learned my addr 172.21.15.100:0/3476314055 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:27:21.658 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.654+0000 7fbe3fbb2740 1 --2- 172.21.15.100:0/3476314055 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556604dee910 0x556604dfa2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:21.658 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.654+0000 7fbe3fbb2740 1 -- 172.21.15.100:0/3476314055 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbe381a7750 con 0x556604ded460 2024-05-10T11:27:21.658 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.654+0000 7fbe3fbb2740 1 -- 172.21.15.100:0/3476314055 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbe381a8bc0 con 0x556604dee910 2024-05-10T11:27:21.658 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.654+0000 7fbe3fbb2740 1 -- 172.21.15.100:0/3476314055 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbe381a7e20 con 0x556604de8db0 2024-05-10T11:27:21.658 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.654+0000 7fbe3fbb2740 10 client.0 _ll_register_callbacks cb 0x7fbe280f2500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:27:21.658 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.654+0000 7fbe3659c640 1 -- 172.21.15.100:0/3476314055 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556604dee910 msgr2=0x556604dfa2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:21.658 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.654+0000 7fbe3659c640 1 --2- 172.21.15.100:0/3476314055 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556604dee910 0x556604dfa2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:21.658 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.654+0000 7fbe3659c640 1 -- 172.21.15.100:0/3476314055 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556604ded460 msgr2=0x556604df92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:21.658 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.654+0000 7fbe3659c640 1 --2- 172.21.15.100:0/3476314055 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556604ded460 0x556604df92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:21.658 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.654+0000 7fbe3659c640 1 -- 172.21.15.100:0/3476314055 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556604dfa8c0 con 0x556604de8db0 2024-05-10T11:27:21.659 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.654+0000 7fbe3659c640 1 --2- 172.21.15.100:0/3476314055 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556604de8db0 0x556604df8d80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fbe280a5ae0 tx=0x7fbe280f2d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3fb545642d1bc60 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:27:21.659 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.654+0000 7fbe1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:27:21.659 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.655+0000 7fbe1dffb640 1 -- 172.21.15.100:0/3476314055 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe280a53d0 con 0x556604de8db0 2024-05-10T11:27:21.659 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.655+0000 7fbe3fbb2740 1 -- 172.21.15.100:0/3476314055 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe2c031bf0 con 0x556604de8db0 2024-05-10T11:27:21.659 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.655+0000 7fbe1dffb640 1 -- 172.21.15.100:0/3476314055 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fbe280120f0 con 0x556604de8db0 2024-05-10T11:27:21.659 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.655+0000 7fbe1dffb640 1 -- 172.21.15.100:0/3476314055 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fbe280a5750 con 0x556604de8db0 2024-05-10T11:27:21.659 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.655+0000 7fbe3fbb2740 10 client.26668 fetch_fsmap learned FSMap version 31 2024-05-10T11:27:21.659 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.655+0000 7fbe3fbb2740 1 -- 172.21.15.100:0/3476314055 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556604e69580 con 0x556604de8db0 2024-05-10T11:27:21.659 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.655+0000 7fbe1dffb640 1 -- 172.21.15.100:0/3476314055 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbe2c031bf0 con 0x556604de8db0 2024-05-10T11:27:21.659 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.655+0000 7fbe1dffb640 1 -- 172.21.15.100:0/3476314055 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fbe280b2430 con 0x556604de8db0 2024-05-10T11:27:21.659 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.655+0000 7fbe3fbb2740 10 client.26668 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:27:21.659 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.655+0000 7fbe3fbb2740 10 client.26668 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:27:21.659 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.656+0000 7fbe3fbb2740 20 client.26668 populate_metadata read hostname 'smithi100' 2024-05-10T11:27:21.659 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.656+0000 7fbe3fbb2740 1 --2- 172.21.15.100:0/3476314055 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556604e69cd0 0x556604e6c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:21.659 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.656+0000 7fbe3fbb2740 4 client.26668 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:27:21.660 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.656+0000 7fbe3fbb2740 1 -- 172.21.15.100:0/3476314055 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556604d55050 con 0x556604e69cd0 2024-05-10T11:27:21.660 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.656+0000 7fbe35d9b640 1 --2- 172.21.15.100:0/3476314055 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556604e69cd0 0x556604e6c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:21.660 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.656+0000 7fbe1dffb640 10 client.26668 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:27:21.660 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.656+0000 7fbe35d9b640 1 --2- 172.21.15.100:0/3476314055 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556604e69cd0 0x556604e6c0c0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=573d6bff26065671 server_cookie=f657356f55914f8d in_seq=0 out_seq=0 2024-05-10T11:27:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.679+0000 7fbe1dffb640 1 -- 172.21.15.100:0/3476314055 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556604d55050 con 0x556604e69cd0 2024-05-10T11:27:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.679+0000 7fbe1dffb640 10 client.26668 handle_command_reply: tid=0 2024-05-10T11:27:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.679+0000 7fbe3fbb2740 1 client.26668 shutdown 2024-05-10T11:27:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.679+0000 7fbe3fbb2740 1 -- 172.21.15.100:0/3476314055 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556604de8db0 msgr2=0x556604df8d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.679+0000 7fbe3fbb2740 1 --2- 172.21.15.100:0/3476314055 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556604de8db0 0x556604df8d80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fbe280a5ae0 tx=0x7fbe280f2d90 comp rx=0 tx=0).stop 2024-05-10T11:27:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.679+0000 7fbe3fbb2740 1 -- 172.21.15.100:0/3476314055 shutdown_connections 2024-05-10T11:27:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.679+0000 7fbe3fbb2740 1 --2- 172.21.15.100:0/3476314055 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556604e69cd0 0x556604e6c0c0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.679+0000 7fbe3fbb2740 1 --2- 172.21.15.100:0/3476314055 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556604dee910 0x556604dfa2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.679+0000 7fbe3fbb2740 1 --2- 172.21.15.100:0/3476314055 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556604ded460 0x556604df92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.679+0000 7fbe1dffb640 0 client.26668 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:27:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.679+0000 7fbe3fbb2740 1 --2- 172.21.15.100:0/3476314055 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556604de8db0 0x556604df8d80 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.679+0000 7fbe3fbb2740 1 -- 172.21.15.100:0/3476314055 >> 172.21.15.100:0/3476314055 conn(0x556604de4420 msgr2=0x556604de73e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:27:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.679+0000 7fbe3fbb2740 1 -- 172.21.15.100:0/3476314055 shutdown_connections 2024-05-10T11:27:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.679+0000 7fbe3fbb2740 1 -- 172.21.15.100:0/3476314055 wait complete. 2024-05-10T11:27:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.679+0000 7fbe3fbb2740 20 client.26668 trim_cache size 0 max 16384 2024-05-10T11:27:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:27:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:27:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:27:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:27:21.681 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.680+0000 7fbe3ce93640 1 -- 172.21.15.100:0/901284570 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fbe20004aa0 msgr2=0x7fbe20004f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:21.681 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.680+0000 7fbe3ce93640 1 --2- 172.21.15.100:0/901284570 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fbe20004aa0 0x7fbe20004f50 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x556604dea000 tx=0x7fbe30009500 comp rx=0 tx=0).stop 2024-05-10T11:27:21.681 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.680+0000 7fbe3ce93640 1 -- 172.21.15.100:0/901284570 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe38101910 msgr2=0x7fbe38197a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:21.681 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.680+0000 7fbe3ce93640 1 --2- 172.21.15.100:0/901284570 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe38101910 0x7fbe38197a50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbe2800e9c0 tx=0x7fbe2800ee80 comp rx=0 tx=0).stop 2024-05-10T11:27:21.681 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.680+0000 7fbe3ce93640 1 -- 172.21.15.100:0/901284570 shutdown_connections 2024-05-10T11:27:21.681 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.680+0000 7fbe3ce93640 1 --2- 172.21.15.100:0/901284570 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fbe20004aa0 0x7fbe20004f50 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:21.682 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.680+0000 7fbe3ce93640 1 --2- 172.21.15.100:0/901284570 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe381034c0 0x7fbe3819efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:21.682 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.680+0000 7fbe3ce93640 1 --2- 172.21.15.100:0/901284570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe38102b00 0x7fbe38197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:21.682 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.680+0000 7fbe3ce93640 1 --2- 172.21.15.100:0/901284570 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe38101910 0x7fbe38197a50 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:21.682 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.680+0000 7fbe3ce93640 1 -- 172.21.15.100:0/901284570 >> 172.21.15.100:0/901284570 conn(0x7fbe380fce80 msgr2=0x7fbe38105ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:27:21.682 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.680+0000 7fbe3ce93640 1 -- 172.21.15.100:0/901284570 shutdown_connections 2024-05-10T11:27:21.682 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:21.680+0000 7fbe3ce93640 1 -- 172.21.15.100:0/901284570 wait complete. 2024-05-10T11:27:21.796 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:27:21.977 INFO:tasks.workunit.client.0.smithi100.stdout: 1 225010 2.74 MB/sec execute 316 sec latency 77.791 ms 2024-05-10T11:27:22.198 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:27:21 smithi100 ceph-mon[30296]: pgmap v689: 129 pgs: 129 active+clean; 93 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 1.1 MiB/s wr, 151 op/s 2024-05-10T11:27:22.198 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:27:21 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:27:22.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:27:21 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:27:22.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:27:21 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:27:22.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:27:21 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:27:22.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:27:21 smithi178 ceph-mon[33359]: pgmap v689: 129 pgs: 129 active+clean; 93 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 1.1 MiB/s wr, 151 op/s 2024-05-10T11:27:22.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:27:21 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:27:22.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:27:21 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:27:22.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:27:21 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:27:22.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:27:21 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:27:22.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:27:21 smithi188 ceph-mon[34075]: pgmap v689: 129 pgs: 129 active+clean; 93 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 1.1 MiB/s wr, 151 op/s 2024-05-10T11:27:22.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:27:21 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:27:22.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:27:21 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:27:22.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:27:21 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:27:22.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:27:21 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:27:22.977 INFO:tasks.workunit.client.0.smithi100.stdout: 1 225508 2.73 MB/sec execute 317 sec latency 91.854 ms 2024-05-10T11:27:23.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.612+0000 7f68b6dd4700 1 -- 172.21.15.100:0/1976837277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68b01023a0 msgr2=0x7f68b0102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:23.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.612+0000 7f68b6dd4700 1 --2- 172.21.15.100:0/1976837277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68b01023a0 0x7f68b0102820 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f689c009990 tx=0x7f689c009ca0 comp rx=0 tx=0).stop 2024-05-10T11:27:23.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.612+0000 7f68b6dd4700 1 -- 172.21.15.100:0/1976837277 shutdown_connections 2024-05-10T11:27:23.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.612+0000 7f68b6dd4700 1 --2- 172.21.15.100:0/1976837277 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f68b0102d60 0x7f68b010f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:23.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.612+0000 7f68b6dd4700 1 --2- 172.21.15.100:0/1976837277 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f68b0108410 0x7f68b01087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:23.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.612+0000 7f68b6dd4700 1 --2- 172.21.15.100:0/1976837277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68b01023a0 0x7f68b0102820 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:23.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.612+0000 7f68b6dd4700 1 -- 172.21.15.100:0/1976837277 >> 172.21.15.100:0/1976837277 conn(0x7f68b0079350 msgr2=0x7f68b0079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:27:23.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.613+0000 7f68b6dd4700 1 -- 172.21.15.100:0/1976837277 shutdown_connections 2024-05-10T11:27:23.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.613+0000 7f68b6dd4700 1 -- 172.21.15.100:0/1976837277 wait complete. 2024-05-10T11:27:23.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.613+0000 7f68b6dd4700 1 Processor -- start 2024-05-10T11:27:23.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.614+0000 7f68b6dd4700 1 -- start start 2024-05-10T11:27:23.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.614+0000 7f68b6dd4700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f68b01023a0 0x7f68b019a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:23.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.614+0000 7f68b6dd4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68b0102d60 0x7f68b019ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:23.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.614+0000 7f68b6dd4700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f68b0108410 0x7f68b01a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:23.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.614+0000 7f68b6dd4700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f68b0112410 con 0x7f68b0102d60 2024-05-10T11:27:23.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.614+0000 7f68b6dd4700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f68b0112290 con 0x7f68b01023a0 2024-05-10T11:27:23.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.614+0000 7f68b6dd4700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f68b0112590 con 0x7f68b0108410 2024-05-10T11:27:23.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.614+0000 7f68affff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68b0102d60 0x7f68b019ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:23.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.614+0000 7f68affff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68b0102d60 0x7f68b019ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45838/0 (socket says 172.21.15.100:45838) 2024-05-10T11:27:23.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.614+0000 7f68affff700 1 -- 172.21.15.100:0/2273506429 learned_addr learned my addr 172.21.15.100:0/2273506429 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:27:23.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.614+0000 7f68b5371700 1 --2- 172.21.15.100:0/2273506429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f68b0108410 0x7f68b01a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:23.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.614+0000 7f68b4b70700 1 --2- 172.21.15.100:0/2273506429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f68b01023a0 0x7f68b019a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:23.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.614+0000 7f68affff700 1 -- 172.21.15.100:0/2273506429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f68b0108410 msgr2=0x7f68b01a5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:23.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.614+0000 7f68affff700 1 --2- 172.21.15.100:0/2273506429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f68b0108410 0x7f68b01a5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:23.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.614+0000 7f68affff700 1 -- 172.21.15.100:0/2273506429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f68b01023a0 msgr2=0x7f68b019a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:23.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.614+0000 7f68affff700 1 --2- 172.21.15.100:0/2273506429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f68b01023a0 0x7f68b019a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:23.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.614+0000 7f68affff700 1 -- 172.21.15.100:0/2273506429 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68b01a5ed0 con 0x7f68b0102d60 2024-05-10T11:27:23.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.615+0000 7f68affff700 1 --2- 172.21.15.100:0/2273506429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68b0102d60 0x7f68b019ae50 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f689c000c00 tx=0x7f689c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:27:23.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.615+0000 7f68adffb700 1 -- 172.21.15.100:0/2273506429 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f689c027070 con 0x7f68b0102d60 2024-05-10T11:27:23.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.615+0000 7f68adffb700 1 -- 172.21.15.100:0/2273506429 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f689c00fb00 con 0x7f68b0102d60 2024-05-10T11:27:23.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.615+0000 7f68adffb700 1 -- 172.21.15.100:0/2273506429 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f689c022c40 con 0x7f68b0102d60 2024-05-10T11:27:23.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.615+0000 7f68b6dd4700 1 -- 172.21.15.100:0/2273506429 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68b01a60a0 con 0x7f68b0102d60 2024-05-10T11:27:23.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.615+0000 7f68b6dd4700 1 -- 172.21.15.100:0/2273506429 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68b01a64a0 con 0x7f68b0102d60 2024-05-10T11:27:23.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.618+0000 7f68adffb700 1 -- 172.21.15.100:0/2273506429 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f689c02a070 con 0x7f68b0102d60 2024-05-10T11:27:23.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.618+0000 7f68adffb700 1 --2- 172.21.15.100:0/2273506429 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f68a00b2880 0x7f68a00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:23.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.619+0000 7f68b4b70700 1 --2- 172.21.15.100:0/2273506429 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f68a00b2880 0x7f68a00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:23.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.619+0000 7f68adffb700 1 -- 172.21.15.100:0/2273506429 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f689c0e3430 con 0x7f68b0102d60 2024-05-10T11:27:23.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.619+0000 7f68b4b70700 1 --2- 172.21.15.100:0/2273506429 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f68a00b2880 0x7f68a00b4d40 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f68a4005950 tx=0x7f68a400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:27:23.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.619+0000 7f68c1667b80 1 Processor -- start 2024-05-10T11:27:23.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.619+0000 7f68c1667b80 1 -- start start 2024-05-10T11:27:23.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.619+0000 7f68c1667b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56328e613230 0x56328e613610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:23.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.619+0000 7f68c1667b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56328e613b50 0x56328e60d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:23.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.619+0000 7f68c1667b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56328e60ddd0 0x56328e60e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:23.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.619+0000 7f68c1667b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56328e55f9c0 con 0x56328e60ddd0 2024-05-10T11:27:23.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.619+0000 7f68c1667b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56328e4c4e20 con 0x56328e613b50 2024-05-10T11:27:23.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.619+0000 7f68c1667b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56328e55b980 con 0x56328e613230 2024-05-10T11:27:23.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.619+0000 7f68b5371700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56328e60ddd0 0x56328e60e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:23.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.619+0000 7f68b4b70700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56328e613230 0x56328e613610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:23.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.619+0000 7f68b5371700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56328e60ddd0 0x56328e60e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45850/0 (socket says 172.21.15.100:45850) 2024-05-10T11:27:23.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.620+0000 7f68b5371700 1 -- 172.21.15.100:0/1646495995 learned_addr learned my addr 172.21.15.100:0/1646495995 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:27:23.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.620+0000 7f68affff700 1 --2- 172.21.15.100:0/1646495995 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56328e613b50 0x56328e60d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:23.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.620+0000 7f68b4b70700 1 -- 172.21.15.100:0/1646495995 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56328e613b50 msgr2=0x56328e60d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:23.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.620+0000 7f68b4b70700 1 --2- 172.21.15.100:0/1646495995 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56328e613b50 0x56328e60d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:23.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.620+0000 7f68b4b70700 1 -- 172.21.15.100:0/1646495995 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56328e60ddd0 msgr2=0x56328e60e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:23.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.620+0000 7f68b4b70700 1 --2- 172.21.15.100:0/1646495995 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56328e60ddd0 0x56328e60e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:23.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.620+0000 7f68b4b70700 1 -- 172.21.15.100:0/1646495995 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56328e4ac950 con 0x56328e613230 2024-05-10T11:27:23.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.620+0000 7f68b5371700 1 --2- 172.21.15.100:0/1646495995 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56328e60ddd0 0x56328e60e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:27:23.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.620+0000 7f68b4b70700 1 --2- 172.21.15.100:0/1646495995 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56328e613230 0x56328e613610 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f68a400fc70 tx=0x7f68a4012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a84551351c9a521d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:27:23.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.620+0000 7f689affd700 1 -- 172.21.15.100:0/1646495995 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a4016070 con 0x56328e613230 2024-05-10T11:27:23.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.620+0000 7f689affd700 1 -- 172.21.15.100:0/1646495995 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f68a4010bc0 con 0x56328e613230 2024-05-10T11:27:23.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.620+0000 7f689affd700 1 -- 172.21.15.100:0/1646495995 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a401bc20 con 0x56328e613230 2024-05-10T11:27:23.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.621+0000 7f68c1667b80 1 -- 172.21.15.100:0/1646495995 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56328e613230 msgr2=0x56328e613610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:23.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.621+0000 7f68c1667b80 1 --2- 172.21.15.100:0/1646495995 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56328e613230 0x56328e613610 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f68a400fc70 tx=0x7f68a4012000 comp rx=0 tx=0).stop 2024-05-10T11:27:23.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.621+0000 7f68c1667b80 1 -- 172.21.15.100:0/1646495995 shutdown_connections 2024-05-10T11:27:23.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.621+0000 7f68c1667b80 1 --2- 172.21.15.100:0/1646495995 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56328e60ddd0 0x56328e60e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:23.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.621+0000 7f68c1667b80 1 --2- 172.21.15.100:0/1646495995 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56328e613230 0x56328e613610 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:23.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.621+0000 7f68c1667b80 1 --2- 172.21.15.100:0/1646495995 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56328e613b50 0x56328e60d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:23.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.621+0000 7f68c1667b80 1 -- 172.21.15.100:0/1646495995 >> 172.21.15.100:0/1646495995 conn(0x56328e539bf0 msgr2=0x56328e608320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:27:23.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.621+0000 7f68c1667b80 1 -- 172.21.15.100:0/1646495995 shutdown_connections 2024-05-10T11:27:23.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.621+0000 7f68c1667b80 1 -- 172.21.15.100:0/1646495995 wait complete. 2024-05-10T11:27:23.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.621+0000 7f68c1667b80 1 -- start start 2024-05-10T11:27:23.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.622+0000 7f68c1667b80 1 Processor -- start 2024-05-10T11:27:23.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.622+0000 7f68c1667b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56328e60ddd0 0x56328e619760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:23.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.622+0000 7f68c1667b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56328e613230 0x56328e616b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:23.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.622+0000 7f68c1667b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56328e613b50 0x56328e617040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:23.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.622+0000 7f68c1667b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56328e61a4d0 con 0x56328e613b50 2024-05-10T11:27:23.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.622+0000 7f68c1667b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56328e518f50 con 0x56328e613230 2024-05-10T11:27:23.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.622+0000 7f68c1667b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56328e61a650 con 0x56328e60ddd0 2024-05-10T11:27:23.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.622+0000 7f68c1667b80 10 client.0 _ll_register_callbacks cb 0x56328e4f3630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:27:23.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.622+0000 7f68affff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56328e613230 0x56328e616b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:23.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.622+0000 7f68b4b70700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56328e60ddd0 0x56328e619760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:23.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.622+0000 7f68affff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56328e613230 0x56328e616b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:48510/0 (socket says 172.21.15.100:48510) 2024-05-10T11:27:23.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.622+0000 7f68affff700 1 -- 172.21.15.100:0/2756501806 learned_addr learned my addr 172.21.15.100:0/2756501806 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:27:23.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.622+0000 7f68b4b70700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56328e60ddd0 0x56328e619760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:53002/0 (socket says 172.21.15.100:53002) 2024-05-10T11:27:23.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.622+0000 7f68b5371700 1 --2- 172.21.15.100:0/2756501806 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56328e613b50 0x56328e617040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:23.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.622+0000 7f68b4b70700 1 -- 172.21.15.100:0/2756501806 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56328e613230 msgr2=0x56328e616b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:23.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.622+0000 7f68b4b70700 1 --2- 172.21.15.100:0/2756501806 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56328e613230 0x56328e616b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:23.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.622+0000 7f68b4b70700 1 -- 172.21.15.100:0/2756501806 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56328e613b50 msgr2=0x56328e617040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:23.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.622+0000 7f68b4b70700 1 --2- 172.21.15.100:0/2756501806 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56328e613b50 0x56328e617040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:23.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.622+0000 7f68b4b70700 1 -- 172.21.15.100:0/2756501806 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56328e5167e0 con 0x56328e60ddd0 2024-05-10T11:27:23.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.622+0000 7f68affff700 1 --2- 172.21.15.100:0/2756501806 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56328e613230 0x56328e616b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:27:23.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.623+0000 7f68b4b70700 1 --2- 172.21.15.100:0/2756501806 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56328e60ddd0 0x56328e619760 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f68a400a950 tx=0x7f68a40107e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5802b292d22bc25 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:27:23.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.623+0000 7f688ffff700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:27:23.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.623+0000 7f688ffff700 1 -- 172.21.15.100:0/2756501806 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a401bc70 con 0x56328e60ddd0 2024-05-10T11:27:23.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.623+0000 7f688ffff700 1 -- 172.21.15.100:0/2756501806 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f68a401be10 con 0x56328e60ddd0 2024-05-10T11:27:23.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.623+0000 7f68c1667b80 1 -- 172.21.15.100:0/2756501806 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68a4010bc0 con 0x56328e60ddd0 2024-05-10T11:27:23.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.623+0000 7f688ffff700 1 -- 172.21.15.100:0/2756501806 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f68a40109d0 con 0x56328e60ddd0 2024-05-10T11:27:23.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.623+0000 7f68c1667b80 10 client.26680 fetch_fsmap learned FSMap version 31 2024-05-10T11:27:23.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.623+0000 7f68c1667b80 1 -- 172.21.15.100:0/2756501806 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56328e629140 con 0x56328e60ddd0 2024-05-10T11:27:23.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.623+0000 7f688ffff700 1 -- 172.21.15.100:0/2756501806 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f68a4010bc0 con 0x56328e60ddd0 2024-05-10T11:27:23.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.623+0000 7f688ffff700 1 -- 172.21.15.100:0/2756501806 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f68a402d020 con 0x56328e60ddd0 2024-05-10T11:27:23.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.623+0000 7f68c1667b80 10 client.26680 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:27:23.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.623+0000 7f68c1667b80 10 client.26680 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:27:23.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.623+0000 7f68c1667b80 20 client.26680 populate_metadata read hostname 'smithi100' 2024-05-10T11:27:23.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.623+0000 7f68c1667b80 1 --2- 172.21.15.100:0/2756501806 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56328e62b720 0x56328e62db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:23.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.623+0000 7f68c1667b80 4 client.26680 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:27:23.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.623+0000 7f68c1667b80 1 -- 172.21.15.100:0/2756501806 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56328e56d3a0 con 0x56328e62b720 2024-05-10T11:27:23.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.624+0000 7f68affff700 1 --2- 172.21.15.100:0/2756501806 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56328e62b720 0x56328e62db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:23.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.624+0000 7f68affff700 1 --2- 172.21.15.100:0/2756501806 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56328e62b720 0x56328e62db10 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59d2464690bfc69c server_cookie=389cd19a1c9721e2 in_seq=0 out_seq=0 2024-05-10T11:27:23.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.624+0000 7f688ffff700 10 client.26680 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:27:23.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.624+0000 7f688ffff700 1 -- 172.21.15.100:0/2756501806 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56328e56d3a0 con 0x56328e62b720 2024-05-10T11:27:23.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.625+0000 7f688ffff700 10 client.26680 handle_command_reply: tid=0 2024-05-10T11:27:23.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.625+0000 7f68c1667b80 1 client.26680 shutdown 2024-05-10T11:27:23.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.625+0000 7f68c1667b80 1 -- 172.21.15.100:0/2756501806 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56328e60ddd0 msgr2=0x56328e619760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:23.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.625+0000 7f68c1667b80 1 --2- 172.21.15.100:0/2756501806 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56328e60ddd0 0x56328e619760 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f68a400a950 tx=0x7f68a40107e0 comp rx=0 tx=0).stop 2024-05-10T11:27:23.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.625+0000 7f68c1667b80 1 -- 172.21.15.100:0/2756501806 shutdown_connections 2024-05-10T11:27:23.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.625+0000 7f68c1667b80 1 --2- 172.21.15.100:0/2756501806 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56328e62b720 0x56328e62db10 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:23.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.625+0000 7f68c1667b80 1 --2- 172.21.15.100:0/2756501806 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56328e613b50 0x56328e617040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:23.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.625+0000 7f68c1667b80 1 --2- 172.21.15.100:0/2756501806 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56328e60ddd0 0x56328e619760 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:23.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.625+0000 7f688ffff700 0 client.26680 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:27:23.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.625+0000 7f68c1667b80 1 --2- 172.21.15.100:0/2756501806 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56328e613230 0x56328e616b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:23.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.625+0000 7f68c1667b80 1 -- 172.21.15.100:0/2756501806 >> 172.21.15.100:0/2756501806 conn(0x56328e539bf0 msgr2=0x56328e602bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:27:23.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.625+0000 7f68c1667b80 1 -- 172.21.15.100:0/2756501806 shutdown_connections 2024-05-10T11:27:23.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.625+0000 7f68c1667b80 1 -- 172.21.15.100:0/2756501806 wait complete. 2024-05-10T11:27:23.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.625+0000 7f68c1667b80 20 client.26680 trim_cache size 0 max 16384 2024-05-10T11:27:23.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.649+0000 7f68c1667b80 1 Processor -- start 2024-05-10T11:27:23.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.649+0000 7f68c1667b80 1 -- start start 2024-05-10T11:27:23.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.649+0000 7f68c1667b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56328e60ddd0 0x56328e602cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:23.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.649+0000 7f68c1667b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56328e613230 0x56328e612380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:23.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.649+0000 7f68c1667b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56328e613b50 0x56328e6128c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:23.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.649+0000 7f68c1667b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f68b01af540 con 0x56328e613b50 2024-05-10T11:27:23.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.649+0000 7f68c1667b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f68b01acdd0 con 0x56328e60ddd0 2024-05-10T11:27:23.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.649+0000 7f68c1667b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f68b01aac20 con 0x56328e613230 2024-05-10T11:27:23.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.649+0000 7f68b5371700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56328e613b50 0x56328e6128c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:23.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.650+0000 7f68b5371700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56328e613b50 0x56328e6128c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45860/0 (socket says 172.21.15.100:45860) 2024-05-10T11:27:23.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.650+0000 7f68b5371700 1 -- 172.21.15.100:0/4122612721 learned_addr learned my addr 172.21.15.100:0/4122612721 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:27:23.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.650+0000 7f68affff700 1 --2- 172.21.15.100:0/4122612721 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56328e613230 0x56328e612380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:23.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.650+0000 7f68b4b70700 1 --2- 172.21.15.100:0/4122612721 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56328e60ddd0 0x56328e602cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:23.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.650+0000 7f68b5371700 1 -- 172.21.15.100:0/4122612721 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56328e613230 msgr2=0x56328e612380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:23.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.650+0000 7f68b5371700 1 --2- 172.21.15.100:0/4122612721 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56328e613230 0x56328e612380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:23.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.650+0000 7f68b5371700 1 -- 172.21.15.100:0/4122612721 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56328e60ddd0 msgr2=0x56328e602cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:23.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.650+0000 7f68b5371700 1 --2- 172.21.15.100:0/4122612721 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56328e60ddd0 0x56328e602cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:23.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.650+0000 7f68b5371700 1 -- 172.21.15.100:0/4122612721 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56328e60dbd0 con 0x56328e613b50 2024-05-10T11:27:23.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.650+0000 7f68affff700 1 --2- 172.21.15.100:0/4122612721 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56328e613230 0x56328e612380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:27:23.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.650+0000 7f68b5371700 1 --2- 172.21.15.100:0/4122612721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56328e613b50 0x56328e6128c0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f68a8009e80 tx=0x7f68a8009eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8385a8f56a1a8845 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:27:23.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.650+0000 7f689a7fc700 1 -- 172.21.15.100:0/4122612721 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a8004bb0 con 0x56328e613b50 2024-05-10T11:27:23.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.650+0000 7f689a7fc700 1 -- 172.21.15.100:0/4122612721 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f68a801b070 con 0x56328e613b50 2024-05-10T11:27:23.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.650+0000 7f689a7fc700 1 -- 172.21.15.100:0/4122612721 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a8005020 con 0x56328e613b50 2024-05-10T11:27:23.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.651+0000 7f68c1667b80 1 -- 172.21.15.100:0/4122612721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56328e613b50 msgr2=0x56328e6128c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:23.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.651+0000 7f68c1667b80 1 --2- 172.21.15.100:0/4122612721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56328e613b50 0x56328e6128c0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f68a8009e80 tx=0x7f68a8009eb0 comp rx=0 tx=0).stop 2024-05-10T11:27:23.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.651+0000 7f68c1667b80 1 -- 172.21.15.100:0/4122612721 shutdown_connections 2024-05-10T11:27:23.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.651+0000 7f68c1667b80 1 --2- 172.21.15.100:0/4122612721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56328e613b50 0x56328e6128c0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:23.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.651+0000 7f68c1667b80 1 --2- 172.21.15.100:0/4122612721 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56328e60ddd0 0x56328e602cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:23.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.651+0000 7f68c1667b80 1 --2- 172.21.15.100:0/4122612721 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56328e613230 0x56328e612380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:23.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.651+0000 7f68c1667b80 1 -- 172.21.15.100:0/4122612721 >> 172.21.15.100:0/4122612721 conn(0x56328e539bf0 msgr2=0x56328e611c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:27:23.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.651+0000 7f68c1667b80 1 -- 172.21.15.100:0/4122612721 shutdown_connections 2024-05-10T11:27:23.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.651+0000 7f68c1667b80 1 -- 172.21.15.100:0/4122612721 wait complete. 2024-05-10T11:27:23.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.651+0000 7f68c1667b80 1 -- start start 2024-05-10T11:27:23.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.651+0000 7f68c1667b80 1 Processor -- start 2024-05-10T11:27:23.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.652+0000 7f68c1667b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56328e60ddd0 0x56328e61fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:23.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.652+0000 7f68c1667b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56328e613230 0x56328e61def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:23.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.652+0000 7f68c1667b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56328e613b50 0x56328e61e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:23.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.652+0000 7f68c1667b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f68b01ab290 con 0x56328e613230 2024-05-10T11:27:23.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.652+0000 7f68c1667b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f68b01ac700 con 0x56328e613b50 2024-05-10T11:27:23.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.652+0000 7f68c1667b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f68b01ab960 con 0x56328e60ddd0 2024-05-10T11:27:23.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.652+0000 7f68c1667b80 10 client.0 _ll_register_callbacks cb 0x56328e4f3630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:27:23.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.652+0000 7f68affff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56328e613230 0x56328e61def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:23.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.652+0000 7f68affff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56328e613230 0x56328e61def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45862/0 (socket says 172.21.15.100:45862) 2024-05-10T11:27:23.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.652+0000 7f68affff700 1 -- 172.21.15.100:0/2725766118 learned_addr learned my addr 172.21.15.100:0/2725766118 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:27:23.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.652+0000 7f68b4b70700 1 --2- 172.21.15.100:0/2725766118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56328e60ddd0 0x56328e61fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:23.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.652+0000 7f68b5371700 1 --2- 172.21.15.100:0/2725766118 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56328e613b50 0x56328e61e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:23.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.652+0000 7f68affff700 1 -- 172.21.15.100:0/2725766118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56328e60ddd0 msgr2=0x56328e61fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:23.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.652+0000 7f68affff700 1 --2- 172.21.15.100:0/2725766118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56328e60ddd0 0x56328e61fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:23.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.652+0000 7f68affff700 1 -- 172.21.15.100:0/2725766118 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56328e613b50 msgr2=0x56328e61e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:23.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.652+0000 7f68affff700 1 --2- 172.21.15.100:0/2725766118 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56328e613b50 0x56328e61e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:23.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.652+0000 7f68affff700 1 -- 172.21.15.100:0/2725766118 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56328e61e970 con 0x56328e613230 2024-05-10T11:27:23.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.652+0000 7f68affff700 1 --2- 172.21.15.100:0/2725766118 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56328e613230 0x56328e61def0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f689c0e0c20 tx=0x7f689c0b1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=637e7462781dba46 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:27:23.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.652+0000 7f688ffff700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:27:23.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.653+0000 7f688ffff700 1 -- 172.21.15.100:0/2725766118 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f689c0aeb30 con 0x56328e613230 2024-05-10T11:27:23.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.653+0000 7f688ffff700 1 -- 172.21.15.100:0/2725766118 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f689c0aa9e0 con 0x56328e613230 2024-05-10T11:27:23.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.653+0000 7f68c1667b80 1 -- 172.21.15.100:0/2725766118 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68a801b070 con 0x56328e613230 2024-05-10T11:27:23.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.653+0000 7f688ffff700 1 -- 172.21.15.100:0/2725766118 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f689c0b3550 con 0x56328e613230 2024-05-10T11:27:23.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.653+0000 7f68c1667b80 10 client.17223 fetch_fsmap learned FSMap version 31 2024-05-10T11:27:23.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.653+0000 7f68c1667b80 1 -- 172.21.15.100:0/2725766118 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56328e696b00 con 0x56328e613230 2024-05-10T11:27:23.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.653+0000 7f688ffff700 1 -- 172.21.15.100:0/2725766118 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f68a801b070 con 0x56328e613230 2024-05-10T11:27:23.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.653+0000 7f688ffff700 1 -- 172.21.15.100:0/2725766118 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f689c00f7e0 con 0x56328e613230 2024-05-10T11:27:23.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.653+0000 7f68c1667b80 10 client.17223 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:27:23.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.653+0000 7f68c1667b80 10 client.17223 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:27:23.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.653+0000 7f68c1667b80 20 client.17223 populate_metadata read hostname 'smithi100' 2024-05-10T11:27:23.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.653+0000 7f68c1667b80 1 --2- 172.21.15.100:0/2725766118 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56328e697260 0x56328e699680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:23.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.653+0000 7f68c1667b80 4 client.17223 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T11:27:23.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.653+0000 7f68c1667b80 1 -- 172.21.15.100:0/2725766118 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56328e567440 con 0x56328e697260 2024-05-10T11:27:23.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.675+0000 7f68b4b70700 1 --2- 172.21.15.100:0/2725766118 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56328e697260 0x56328e699680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:23.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.676+0000 7f68b4b70700 1 --2- 172.21.15.100:0/2725766118 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56328e697260 0x56328e699680 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=478abd65ad06ffd2 server_cookie=3a55aa3c32e0ea20 in_seq=0 out_seq=0 2024-05-10T11:27:23.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.676+0000 7f688ffff700 10 client.17223 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:27:23.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.677+0000 7f688ffff700 1 -- 172.21.15.100:0/2725766118 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56328e567440 con 0x56328e697260 2024-05-10T11:27:23.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.677+0000 7f688ffff700 10 client.17223 handle_command_reply: tid=0 2024-05-10T11:27:23.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.677+0000 7f68c1667b80 1 client.17223 shutdown 2024-05-10T11:27:23.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.677+0000 7f68c1667b80 1 -- 172.21.15.100:0/2725766118 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56328e613230 msgr2=0x56328e61def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:23.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.677+0000 7f68c1667b80 1 --2- 172.21.15.100:0/2725766118 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56328e613230 0x56328e61def0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f689c0e0c20 tx=0x7f689c0b1040 comp rx=0 tx=0).stop 2024-05-10T11:27:23.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.678+0000 7f68c1667b80 1 -- 172.21.15.100:0/2725766118 shutdown_connections 2024-05-10T11:27:23.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.678+0000 7f68c1667b80 1 --2- 172.21.15.100:0/2725766118 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56328e697260 0x56328e699680 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:23.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.678+0000 7f68c1667b80 1 --2- 172.21.15.100:0/2725766118 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56328e613b50 0x56328e61e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:23.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.678+0000 7f68c1667b80 1 --2- 172.21.15.100:0/2725766118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56328e60ddd0 0x56328e61fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:23.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.678+0000 7f68c1667b80 1 --2- 172.21.15.100:0/2725766118 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56328e613230 0x56328e61def0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:23.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.678+0000 7f68c1667b80 1 -- 172.21.15.100:0/2725766118 >> 172.21.15.100:0/2725766118 conn(0x56328e539bf0 msgr2=0x56328e614640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:27:23.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.678+0000 7f688ffff700 0 client.17223 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:27:23.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.678+0000 7f68c1667b80 1 -- 172.21.15.100:0/2725766118 shutdown_connections 2024-05-10T11:27:23.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.678+0000 7f68c1667b80 1 -- 172.21.15.100:0/2725766118 wait complete. 2024-05-10T11:27:23.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.678+0000 7f68c1667b80 20 client.17223 trim_cache size 0 max 16384 2024-05-10T11:27:23.679 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:27:23.679 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-05-10T11:27:23.679 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "da27c588-5931-485f-927f-46f6838a17c9", 2024-05-10T11:27:23.679 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-05-10T11:27:23.679 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:27:23.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.679+0000 7f68b6dd4700 1 -- 172.21.15.100:0/2273506429 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f68a00b2880 msgr2=0x7f68a00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:23.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.679+0000 7f68b6dd4700 1 --2- 172.21.15.100:0/2273506429 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f68a00b2880 0x7f68a00b4d40 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f68a4005950 tx=0x7f68a400a300 comp rx=0 tx=0).stop 2024-05-10T11:27:23.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.679+0000 7f68b6dd4700 1 -- 172.21.15.100:0/2273506429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68b0102d60 msgr2=0x7f68b019ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:23.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.679+0000 7f68b6dd4700 1 --2- 172.21.15.100:0/2273506429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68b0102d60 0x7f68b019ae50 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f689c000c00 tx=0x7f689c0047b0 comp rx=0 tx=0).stop 2024-05-10T11:27:23.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.679+0000 7f68b6dd4700 1 -- 172.21.15.100:0/2273506429 shutdown_connections 2024-05-10T11:27:23.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.679+0000 7f68b6dd4700 1 --2- 172.21.15.100:0/2273506429 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f68a00b2880 0x7f68a00b4d40 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:23.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.679+0000 7f68b6dd4700 1 --2- 172.21.15.100:0/2273506429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f68b0108410 0x7f68b01a5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:23.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.679+0000 7f68b6dd4700 1 --2- 172.21.15.100:0/2273506429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f68b01023a0 0x7f68b019a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:23.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.679+0000 7f68b6dd4700 1 --2- 172.21.15.100:0/2273506429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68b0102d60 0x7f68b019ae50 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:23.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.679+0000 7f68b6dd4700 1 -- 172.21.15.100:0/2273506429 >> 172.21.15.100:0/2273506429 conn(0x7f68b0079350 msgr2=0x7f68b0101110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:27:23.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.679+0000 7f68b6dd4700 1 -- 172.21.15.100:0/2273506429 shutdown_connections 2024-05-10T11:27:23.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:23.679+0000 7f68b6dd4700 1 -- 172.21.15.100:0/2273506429 wait complete. 2024-05-10T11:27:23.977 INFO:tasks.workunit.client.0.smithi100.stdout: 1 225946 2.73 MB/sec execute 318 sec latency 81.008 ms 2024-05-10T11:27:24.173 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:27:23 smithi100 ceph-mon[30296]: pgmap v690: 129 pgs: 129 active+clean; 95 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 1.4 MiB/s wr, 157 op/s 2024-05-10T11:27:24.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:27:23 smithi178 ceph-mon[33359]: pgmap v690: 129 pgs: 129 active+clean; 95 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 1.4 MiB/s wr, 157 op/s 2024-05-10T11:27:24.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:27:23 smithi188 ceph-mon[34075]: pgmap v690: 129 pgs: 129 active+clean; 95 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 1.4 MiB/s wr, 157 op/s 2024-05-10T11:27:24.503 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T11:27:24.796 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:27:24.977 INFO:tasks.workunit.client.0.smithi100.stdout: 1 226684 2.74 MB/sec execute 319 sec latency 14.439 ms 2024-05-10T11:27:25.213 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:27:24 smithi100 ceph-mon[30296]: scrub queued for path: / 2024-05-10T11:27:25.213 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:27:24 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:27:25.213 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:27:24 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:27:25.213 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:27:24 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:27:25.213 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:27:24 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:27:25.213 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:27:24 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:27:25.213 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:27:24 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:27:25.213 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:27:24 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:27:25.213 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:27:24 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:27:25.213 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:27:24 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:27:25.214 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:27:24 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:27:25.214 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:27:24 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:27:25.214 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:27:24 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:27:25.214 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:27:24 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:27:25.214 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:27:24 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:27:25.214 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:27:24 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:27:25.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:27:24 smithi178 ceph-mon[33359]: scrub queued for path: / 2024-05-10T11:27:25.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:27:24 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:27:25.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:27:24 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:27:25.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:27:24 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:27:25.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:27:24 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:27:25.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:27:24 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:27:25.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:27:24 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:27:25.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:27:24 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:27:25.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:27:24 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:27:25.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:27:24 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:27:25.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:27:24 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:27:25.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:27:24 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:27:25.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:27:24 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:27:25.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:27:24 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:27:25.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:27:24 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:27:25.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:27:24 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:27:25.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:27:24 smithi188 ceph-mon[34075]: scrub queued for path: / 2024-05-10T11:27:25.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:27:24 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:27:25.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:27:24 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:27:25.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:27:24 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:27:25.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:27:24 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:27:25.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:27:24 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:27:25.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:27:24 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:27:25.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:27:24 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:27:25.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:27:24 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:27:25.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:27:24 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:27:25.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:27:24 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:27:25.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:27:24 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:27:25.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:27:24 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:27:25.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:27:24 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:27:25.280 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:27:24 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:27:25.280 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:27:24 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:27:25.977 INFO:tasks.workunit.client.0.smithi100.stdout: 1 227140 2.75 MB/sec execute 320 sec latency 15.303 ms 2024-05-10T11:27:26.153 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:27:25 smithi100 ceph-mon[30296]: pgmap v691: 129 pgs: 129 active+clean; 95 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 1.2 MiB/s wr, 137 op/s 2024-05-10T11:27:26.154 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:27:25 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:27:26.154 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:27:25 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:27:26.154 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:27:25 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:27:26.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:27:25 smithi178 ceph-mon[33359]: pgmap v691: 129 pgs: 129 active+clean; 95 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 1.2 MiB/s wr, 137 op/s 2024-05-10T11:27:26.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:27:25 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:27:26.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:27:25 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:27:26.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:27:25 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:27:26.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:27:25 smithi188 ceph-mon[34075]: pgmap v691: 129 pgs: 129 active+clean; 95 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 1.2 MiB/s wr, 137 op/s 2024-05-10T11:27:26.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:27:25 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:27:26.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:27:25 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:27:26.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:27:25 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:27:26.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.501+0000 7f59bef09700 1 -- 172.21.15.100:0/750204809 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59b8102390 msgr2=0x7f59b8102810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:26.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.501+0000 7f59bef09700 1 --2- 172.21.15.100:0/750204809 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59b8102390 0x7f59b8102810 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f59a4009990 tx=0x7f59a4009ca0 comp rx=0 tx=0).stop 2024-05-10T11:27:26.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.501+0000 7f59bef09700 1 -- 172.21.15.100:0/750204809 shutdown_connections 2024-05-10T11:27:26.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.501+0000 7f59bef09700 1 --2- 172.21.15.100:0/750204809 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59b8102d50 0x7f59b810f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:26.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.501+0000 7f59bef09700 1 --2- 172.21.15.100:0/750204809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f59b8108400 0x7f59b81087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:26.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.501+0000 7f59bef09700 1 --2- 172.21.15.100:0/750204809 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59b8102390 0x7f59b8102810 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:26.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.501+0000 7f59bef09700 1 -- 172.21.15.100:0/750204809 >> 172.21.15.100:0/750204809 conn(0x7f59b8079310 msgr2=0x7f59b8079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:27:26.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.501+0000 7f59bef09700 1 -- 172.21.15.100:0/750204809 shutdown_connections 2024-05-10T11:27:26.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.501+0000 7f59bef09700 1 -- 172.21.15.100:0/750204809 wait complete. 2024-05-10T11:27:26.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.502+0000 7f59bef09700 1 Processor -- start 2024-05-10T11:27:26.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.502+0000 7f59bef09700 1 -- start start 2024-05-10T11:27:26.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.502+0000 7f59bef09700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59b8102390 0x7f59b819a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:26.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.502+0000 7f59bef09700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59b8102d50 0x7f59b819ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:26.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.502+0000 7f59bef09700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f59b8108400 0x7f59b81a5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:26.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.502+0000 7f59bef09700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f59b8112400 con 0x7f59b8102d50 2024-05-10T11:27:26.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.502+0000 7f59bef09700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f59b8112280 con 0x7f59b8108400 2024-05-10T11:27:26.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.502+0000 7f59bef09700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f59b8112580 con 0x7f59b8102390 2024-05-10T11:27:26.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.502+0000 7f59b7fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59b8102d50 0x7f59b819ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:26.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.502+0000 7f59b7fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59b8102d50 0x7f59b819ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46658/0 (socket says 172.21.15.100:46658) 2024-05-10T11:27:26.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.502+0000 7f59bcca5700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59b8102390 0x7f59b819a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:26.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.502+0000 7f59b7fff700 1 -- 172.21.15.100:0/3261425495 learned_addr learned my addr 172.21.15.100:0/3261425495 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:27:26.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.502+0000 7f59bd4a6700 1 --2- 172.21.15.100:0/3261425495 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f59b8108400 0x7f59b81a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:26.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.503+0000 7f59b7fff700 1 -- 172.21.15.100:0/3261425495 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59b8102390 msgr2=0x7f59b819a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:26.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.503+0000 7f59b7fff700 1 --2- 172.21.15.100:0/3261425495 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59b8102390 0x7f59b819a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:26.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.503+0000 7f59b7fff700 1 -- 172.21.15.100:0/3261425495 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f59b8108400 msgr2=0x7f59b81a5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:26.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.503+0000 7f59b7fff700 1 --2- 172.21.15.100:0/3261425495 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f59b8108400 0x7f59b81a5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:26.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.503+0000 7f59b7fff700 1 -- 172.21.15.100:0/3261425495 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59b81a5ec0 con 0x7f59b8102d50 2024-05-10T11:27:26.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.503+0000 7f59bcca5700 1 --2- 172.21.15.100:0/3261425495 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59b8102390 0x7f59b819a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:27:26.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.503+0000 7f59b7fff700 1 --2- 172.21.15.100:0/3261425495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59b8102d50 0x7f59b819ae40 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f59a4006010 tx=0x7f59a400bfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:27:26.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.503+0000 7f59b5ffb700 1 -- 172.21.15.100:0/3261425495 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a4027070 con 0x7f59b8102d50 2024-05-10T11:27:26.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.503+0000 7f59bef09700 1 -- 172.21.15.100:0/3261425495 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59b81a6090 con 0x7f59b8102d50 2024-05-10T11:27:26.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.503+0000 7f59b5ffb700 1 -- 172.21.15.100:0/3261425495 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f59a4022470 con 0x7f59b8102d50 2024-05-10T11:27:26.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.503+0000 7f59b5ffb700 1 -- 172.21.15.100:0/3261425495 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a4017650 con 0x7f59b8102d50 2024-05-10T11:27:26.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.503+0000 7f59bef09700 1 -- 172.21.15.100:0/3261425495 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59b81a6490 con 0x7f59b8102d50 2024-05-10T11:27:26.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.506+0000 7f59b5ffb700 1 -- 172.21.15.100:0/3261425495 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f59a4009d10 con 0x7f59b8102d50 2024-05-10T11:27:26.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.507+0000 7f59b5ffb700 1 --2- 172.21.15.100:0/3261425495 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f59a80b2880 0x7f59a80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:26.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.507+0000 7f59bcca5700 1 --2- 172.21.15.100:0/3261425495 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f59a80b2880 0x7f59a80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:26.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.507+0000 7f59b5ffb700 1 -- 172.21.15.100:0/3261425495 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f59a40e30d0 con 0x7f59b8102d50 2024-05-10T11:27:26.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.507+0000 7f59c979cb80 1 Processor -- start 2024-05-10T11:27:26.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.507+0000 7f59c979cb80 1 -- start start 2024-05-10T11:27:26.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.507+0000 7f59bcca5700 1 --2- 172.21.15.100:0/3261425495 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f59a80b2880 0x7f59a80b4d40 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f59ac009fd0 tx=0x7f59ac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:27:26.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.507+0000 7f59c979cb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55815be2a8e0 0x55815be2acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:26.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.507+0000 7f59c979cb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55815be2b200 0x55815be2eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:26.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.507+0000 7f59c979cb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55815be2f040 0x55815be2f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:26.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.507+0000 7f59c979cb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55815bd78860 con 0x55815be2f040 2024-05-10T11:27:26.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.507+0000 7f59c979cb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55815bd177c0 con 0x55815be2b200 2024-05-10T11:27:26.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.507+0000 7f59c979cb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55815bd74820 con 0x55815be2a8e0 2024-05-10T11:27:26.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.508+0000 7f59bd4a6700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55815be2f040 0x55815be2f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:26.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.508+0000 7f59b7fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55815be2b200 0x55815be2eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:26.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.508+0000 7f59bcca5700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55815be2a8e0 0x55815be2acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:26.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.508+0000 7f59b7fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55815be2b200 0x55815be2eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:52020/0 (socket says 172.21.15.100:52020) 2024-05-10T11:27:26.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.508+0000 7f59b7fff700 1 -- 172.21.15.100:0/1089181279 learned_addr learned my addr 172.21.15.100:0/1089181279 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:27:26.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.508+0000 7f59bcca5700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55815be2a8e0 0x55815be2acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:42026/0 (socket says 172.21.15.100:42026) 2024-05-10T11:27:26.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.508+0000 7f59b7fff700 1 -- 172.21.15.100:0/1089181279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55815be2a8e0 msgr2=0x55815be2acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:26.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.508+0000 7f59b7fff700 1 --2- 172.21.15.100:0/1089181279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55815be2a8e0 0x55815be2acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:26.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.508+0000 7f59b7fff700 1 -- 172.21.15.100:0/1089181279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55815be2f040 msgr2=0x55815be2f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:26.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.508+0000 7f59b7fff700 1 --2- 172.21.15.100:0/1089181279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55815be2f040 0x55815be2f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:26.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.508+0000 7f59b7fff700 1 -- 172.21.15.100:0/1089181279 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55815bcc58d0 con 0x55815be2b200 2024-05-10T11:27:26.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.508+0000 7f59bd4a6700 1 --2- 172.21.15.100:0/1089181279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55815be2f040 0x55815be2f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:27:26.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.508+0000 7f59b7fff700 1 --2- 172.21.15.100:0/1089181279 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55815be2b200 0x55815be2eb00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f59a40a64a0 tx=0x7f59a40a9fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d9649a1751f4d2f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:27:26.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.508+0000 7f59a2ffd700 1 -- 172.21.15.100:0/1089181279 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a40ae7d0 con 0x55815be2b200 2024-05-10T11:27:26.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.508+0000 7f59a2ffd700 1 -- 172.21.15.100:0/1089181279 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f59a40b31f0 con 0x55815be2b200 2024-05-10T11:27:26.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.508+0000 7f59a2ffd700 1 -- 172.21.15.100:0/1089181279 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a40b2d60 con 0x55815be2b200 2024-05-10T11:27:26.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.509+0000 7f59c979cb80 1 -- 172.21.15.100:0/1089181279 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55815be2b200 msgr2=0x55815be2eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:26.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.509+0000 7f59c979cb80 1 --2- 172.21.15.100:0/1089181279 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55815be2b200 0x55815be2eb00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f59a40a64a0 tx=0x7f59a40a9fb0 comp rx=0 tx=0).stop 2024-05-10T11:27:26.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.509+0000 7f59c979cb80 1 -- 172.21.15.100:0/1089181279 shutdown_connections 2024-05-10T11:27:26.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.509+0000 7f59c979cb80 1 --2- 172.21.15.100:0/1089181279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55815be2f040 0x55815be2f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:26.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.509+0000 7f59c979cb80 1 --2- 172.21.15.100:0/1089181279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55815be2a8e0 0x55815be2acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:26.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.509+0000 7f59c979cb80 1 --2- 172.21.15.100:0/1089181279 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55815be2b200 0x55815be2eb00 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:26.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.509+0000 7f59c979cb80 1 -- 172.21.15.100:0/1089181279 >> 172.21.15.100:0/1089181279 conn(0x55815bd52100 msgr2=0x55815be20c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:27:26.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.509+0000 7f59c979cb80 1 -- 172.21.15.100:0/1089181279 shutdown_connections 2024-05-10T11:27:26.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.509+0000 7f59c979cb80 1 -- 172.21.15.100:0/1089181279 wait complete. 2024-05-10T11:27:26.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.509+0000 7f59c979cb80 1 -- start start 2024-05-10T11:27:26.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.510+0000 7f59c979cb80 1 Processor -- start 2024-05-10T11:27:26.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.510+0000 7f59c979cb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55815be2a8e0 0x55815be36460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:26.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.510+0000 7f59c979cb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55815be2b200 0x55815be33800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:26.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.510+0000 7f59c979cb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55815be2f040 0x55815be33d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:26.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.510+0000 7f59c979cb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55815be37030 con 0x55815be2f040 2024-05-10T11:27:26.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.510+0000 7f59c979cb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55815bcdec30 con 0x55815be2a8e0 2024-05-10T11:27:26.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.510+0000 7f59c979cb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55815be371b0 con 0x55815be2b200 2024-05-10T11:27:26.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.510+0000 7f59c979cb80 10 client.0 _ll_register_callbacks cb 0x55815bd10920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:27:26.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.510+0000 7f59bd4a6700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55815be2f040 0x55815be33d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:26.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.510+0000 7f59b7fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55815be2b200 0x55815be33800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:26.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.510+0000 7f59bd4a6700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55815be2f040 0x55815be33d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46668/0 (socket says 172.21.15.100:46668) 2024-05-10T11:27:26.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.510+0000 7f59bcca5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55815be2a8e0 0x55815be36460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:26.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.510+0000 7f59bd4a6700 1 -- 172.21.15.100:0/2058054111 learned_addr learned my addr 172.21.15.100:0/2058054111 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:27:26.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.510+0000 7f59b7fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55815be2b200 0x55815be33800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:42028/0 (socket says 172.21.15.100:42028) 2024-05-10T11:27:26.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.511+0000 7f59b7fff700 1 -- 172.21.15.100:0/2058054111 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55815be2a8e0 msgr2=0x55815be36460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:26.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.511+0000 7f59b7fff700 1 --2- 172.21.15.100:0/2058054111 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55815be2a8e0 0x55815be36460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:26.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.511+0000 7f59b7fff700 1 -- 172.21.15.100:0/2058054111 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55815be2f040 msgr2=0x55815be33d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:26.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.511+0000 7f59b7fff700 1 --2- 172.21.15.100:0/2058054111 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55815be2f040 0x55815be33d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:26.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.511+0000 7f59b7fff700 1 -- 172.21.15.100:0/2058054111 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55815bd14690 con 0x55815be2b200 2024-05-10T11:27:26.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.511+0000 7f59b7fff700 1 --2- 172.21.15.100:0/2058054111 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55815be2b200 0x55815be33800 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f59a40a64a0 tx=0x7f59a40aa8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9063f04460d8e0f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:27:26.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.511+0000 7f5997fff700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:27:26.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.511+0000 7f5997fff700 1 -- 172.21.15.100:0/2058054111 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a40ae7d0 con 0x55815be2b200 2024-05-10T11:27:26.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.511+0000 7f59c979cb80 1 -- 172.21.15.100:0/2058054111 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59a40b31f0 con 0x55815be2b200 2024-05-10T11:27:26.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.511+0000 7f5997fff700 1 -- 172.21.15.100:0/2058054111 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f59a40b2f70 con 0x55815be2b200 2024-05-10T11:27:26.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.511+0000 7f5997fff700 1 -- 172.21.15.100:0/2058054111 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f59a40a4b40 con 0x55815be2b200 2024-05-10T11:27:26.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.511+0000 7f59c979cb80 10 client.26698 fetch_fsmap learned FSMap version 31 2024-05-10T11:27:26.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.511+0000 7f59c979cb80 1 -- 172.21.15.100:0/2058054111 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55815be37030 con 0x55815be2b200 2024-05-10T11:27:26.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.511+0000 7f5997fff700 1 -- 172.21.15.100:0/2058054111 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f59a40b31f0 con 0x55815be2b200 2024-05-10T11:27:26.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.511+0000 7f5997fff700 1 -- 172.21.15.100:0/2058054111 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f59a40b1ac0 con 0x55815be2b200 2024-05-10T11:27:26.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.512+0000 7f59c979cb80 10 client.26698 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:27:26.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.512+0000 7f59c979cb80 10 client.26698 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:27:26.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.512+0000 7f59c979cb80 20 client.26698 populate_metadata read hostname 'smithi100' 2024-05-10T11:27:26.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.512+0000 7f59c979cb80 1 --2- 172.21.15.100:0/2058054111 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55815be43bf0 0x55815be46010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:26.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.512+0000 7f59c979cb80 4 client.26698 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:27:26.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.512+0000 7f59c979cb80 1 -- 172.21.15.100:0/2058054111 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55815bdf2650 con 0x55815be43bf0 2024-05-10T11:27:26.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.512+0000 7f59bcca5700 1 --2- 172.21.15.100:0/2058054111 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55815be43bf0 0x55815be46010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:26.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.512+0000 7f5997fff700 10 client.26698 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:27:26.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.512+0000 7f59bcca5700 1 --2- 172.21.15.100:0/2058054111 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55815be43bf0 0x55815be46010 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b85d488b0724af5 server_cookie=2f09150e3e66f397 in_seq=0 out_seq=0 2024-05-10T11:27:26.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.513+0000 7f5997fff700 1 -- 172.21.15.100:0/2058054111 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55815bdf2650 con 0x55815be43bf0 2024-05-10T11:27:26.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.513+0000 7f5997fff700 10 client.26698 handle_command_reply: tid=0 2024-05-10T11:27:26.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.513+0000 7f59c979cb80 1 client.26698 shutdown 2024-05-10T11:27:26.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.513+0000 7f59c979cb80 1 -- 172.21.15.100:0/2058054111 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55815be2b200 msgr2=0x55815be33800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:26.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.513+0000 7f59c979cb80 1 --2- 172.21.15.100:0/2058054111 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55815be2b200 0x55815be33800 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f59a40a64a0 tx=0x7f59a40aa8b0 comp rx=0 tx=0).stop 2024-05-10T11:27:26.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.513+0000 7f59c979cb80 1 -- 172.21.15.100:0/2058054111 shutdown_connections 2024-05-10T11:27:26.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.513+0000 7f59c979cb80 1 --2- 172.21.15.100:0/2058054111 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55815be43bf0 0x55815be46010 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:26.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.513+0000 7f59c979cb80 1 --2- 172.21.15.100:0/2058054111 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55815be2f040 0x55815be33d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:26.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.513+0000 7f59c979cb80 1 --2- 172.21.15.100:0/2058054111 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55815be2a8e0 0x55815be36460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:26.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.513+0000 7f5997fff700 0 client.26698 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:27:26.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.513+0000 7f59c979cb80 1 --2- 172.21.15.100:0/2058054111 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55815be2b200 0x55815be33800 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:26.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.513+0000 7f59c979cb80 1 -- 172.21.15.100:0/2058054111 >> 172.21.15.100:0/2058054111 conn(0x55815bd52100 msgr2=0x55815be1b720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:27:26.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.513+0000 7f59c979cb80 1 -- 172.21.15.100:0/2058054111 shutdown_connections 2024-05-10T11:27:26.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.513+0000 7f59c979cb80 1 -- 172.21.15.100:0/2058054111 wait complete. 2024-05-10T11:27:26.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.513+0000 7f59c979cb80 20 client.26698 trim_cache size 0 max 16384 2024-05-10T11:27:26.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.536+0000 7f59c979cb80 1 Processor -- start 2024-05-10T11:27:26.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.536+0000 7f59c979cb80 1 -- start start 2024-05-10T11:27:26.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.536+0000 7f59c979cb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55815be2a8e0 0x55815be30fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:26.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.536+0000 7f59c979cb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55815be2b200 0x55815be30430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:26.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.536+0000 7f59c979cb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55815be2f040 0x55815be40080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:26.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.536+0000 7f59c979cb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f59b81af680 con 0x55815be2f040 2024-05-10T11:27:26.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.536+0000 7f59c979cb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f59b81acf10 con 0x55815be2a8e0 2024-05-10T11:27:26.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.536+0000 7f59c979cb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f59b81aad60 con 0x55815be2b200 2024-05-10T11:27:26.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.536+0000 7f59bcca5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55815be2a8e0 0x55815be30fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:26.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.536+0000 7f59bd4a6700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55815be2f040 0x55815be40080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:26.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.536+0000 7f59b7fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55815be2b200 0x55815be30430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:26.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.536+0000 7f59bd4a6700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55815be2f040 0x55815be40080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46684/0 (socket says 172.21.15.100:46684) 2024-05-10T11:27:26.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.536+0000 7f59bcca5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55815be2a8e0 0x55815be30fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:52040/0 (socket says 172.21.15.100:52040) 2024-05-10T11:27:26.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.536+0000 7f59bd4a6700 1 -- 172.21.15.100:0/173762470 learned_addr learned my addr 172.21.15.100:0/173762470 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:27:26.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.536+0000 7f59b7fff700 1 -- 172.21.15.100:0/173762470 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55815be2a8e0 msgr2=0x55815be30fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:26.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.536+0000 7f59b7fff700 1 --2- 172.21.15.100:0/173762470 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55815be2a8e0 0x55815be30fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:26.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.536+0000 7f59b7fff700 1 -- 172.21.15.100:0/173762470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55815be2f040 msgr2=0x55815be40080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:26.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.536+0000 7f59b7fff700 1 --2- 172.21.15.100:0/173762470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55815be2f040 0x55815be40080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:26.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.536+0000 7f59b7fff700 1 -- 172.21.15.100:0/173762470 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55815be335d0 con 0x55815be2b200 2024-05-10T11:27:26.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.536+0000 7f59bd4a6700 1 --2- 172.21.15.100:0/173762470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55815be2f040 0x55815be40080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:27:26.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.536+0000 7f59b7fff700 1 --2- 172.21.15.100:0/173762470 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55815be2b200 0x55815be30430 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f59a40059d0 tx=0x7f59a40e08c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9fea044afa0c0fc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:27:26.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.537+0000 7f59a27fc700 1 -- 172.21.15.100:0/173762470 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a40ae7d0 con 0x55815be2b200 2024-05-10T11:27:26.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.537+0000 7f59a27fc700 1 -- 172.21.15.100:0/173762470 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f59a40b31f0 con 0x55815be2b200 2024-05-10T11:27:26.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.537+0000 7f59a27fc700 1 -- 172.21.15.100:0/173762470 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a40a62e0 con 0x55815be2b200 2024-05-10T11:27:26.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.537+0000 7f59c979cb80 1 -- 172.21.15.100:0/173762470 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55815be2b200 msgr2=0x55815be30430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:26.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.537+0000 7f59c979cb80 1 --2- 172.21.15.100:0/173762470 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55815be2b200 0x55815be30430 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f59a40059d0 tx=0x7f59a40e08c0 comp rx=0 tx=0).stop 2024-05-10T11:27:26.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.537+0000 7f59c979cb80 1 -- 172.21.15.100:0/173762470 shutdown_connections 2024-05-10T11:27:26.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.537+0000 7f59c979cb80 1 --2- 172.21.15.100:0/173762470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55815be2f040 0x55815be40080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:26.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.537+0000 7f59c979cb80 1 --2- 172.21.15.100:0/173762470 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55815be2a8e0 0x55815be30fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:26.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.537+0000 7f59c979cb80 1 --2- 172.21.15.100:0/173762470 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55815be2b200 0x55815be30430 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:26.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.537+0000 7f59c979cb80 1 -- 172.21.15.100:0/173762470 >> 172.21.15.100:0/173762470 conn(0x55815bd52100 msgr2=0x55815be1ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:27:26.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.537+0000 7f59c979cb80 1 -- 172.21.15.100:0/173762470 shutdown_connections 2024-05-10T11:27:26.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.537+0000 7f59c979cb80 1 -- 172.21.15.100:0/173762470 wait complete. 2024-05-10T11:27:26.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.538+0000 7f59c979cb80 1 -- start start 2024-05-10T11:27:26.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.538+0000 7f59c979cb80 1 Processor -- start 2024-05-10T11:27:26.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.538+0000 7f59c979cb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55815be2a8e0 0x55815be4b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:26.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.538+0000 7f59c979cb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55815be2b200 0x55815be4b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:26.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.538+0000 7f59c979cb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55815be2f040 0x55815be3aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:26.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.538+0000 7f59c979cb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f59b81ab3d0 con 0x55815be2b200 2024-05-10T11:27:26.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.538+0000 7f59c979cb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f59b81ac840 con 0x55815be2f040 2024-05-10T11:27:26.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.538+0000 7f59c979cb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f59b81abaa0 con 0x55815be2a8e0 2024-05-10T11:27:26.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.538+0000 7f59c979cb80 10 client.0 _ll_register_callbacks cb 0x55815bd10920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:27:26.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.538+0000 7f59b7fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55815be2b200 0x55815be4b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:26.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.538+0000 7f59bd4a6700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55815be2f040 0x55815be3aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:26.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.538+0000 7f59b7fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55815be2b200 0x55815be4b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46692/0 (socket says 172.21.15.100:46692) 2024-05-10T11:27:26.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.539+0000 7f59b7fff700 1 -- 172.21.15.100:0/1234905534 learned_addr learned my addr 172.21.15.100:0/1234905534 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:27:26.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.538+0000 7f59bcca5700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55815be2a8e0 0x55815be4b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:26.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.539+0000 7f59b7fff700 1 -- 172.21.15.100:0/1234905534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55815be2a8e0 msgr2=0x55815be4b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:26.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.539+0000 7f59b7fff700 1 --2- 172.21.15.100:0/1234905534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55815be2a8e0 0x55815be4b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:26.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.539+0000 7f59b7fff700 1 -- 172.21.15.100:0/1234905534 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55815be2f040 msgr2=0x55815be3aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:26.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.539+0000 7f59b7fff700 1 --2- 172.21.15.100:0/1234905534 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55815be2f040 0x55815be3aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:26.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.539+0000 7f59b7fff700 1 -- 172.21.15.100:0/1234905534 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55815be3b000 con 0x55815be2b200 2024-05-10T11:27:26.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.539+0000 7f59bd4a6700 1 --2- 172.21.15.100:0/1234905534 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55815be2f040 0x55815be3aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:27:26.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.539+0000 7f59bcca5700 1 --2- 172.21.15.100:0/1234905534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55815be2a8e0 0x55815be4b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:27:26.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.539+0000 7f59b7fff700 1 --2- 172.21.15.100:0/1234905534 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55815be2b200 0x55815be4b950 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f59a40b1f10 tx=0x7f59a4005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a25a36b9183051c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:27:26.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.539+0000 7f5997fff700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:27:26.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.539+0000 7f5997fff700 1 -- 172.21.15.100:0/1234905534 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a40ae7d0 con 0x55815be2b200 2024-05-10T11:27:26.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.539+0000 7f5997fff700 1 -- 172.21.15.100:0/1234905534 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f59a40a5d90 con 0x55815be2b200 2024-05-10T11:27:26.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.539+0000 7f59c979cb80 1 -- 172.21.15.100:0/1234905534 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59a40b31f0 con 0x55815be2b200 2024-05-10T11:27:26.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.539+0000 7f5997fff700 1 -- 172.21.15.100:0/1234905534 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f59a40a9470 con 0x55815be2b200 2024-05-10T11:27:26.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.539+0000 7f59c979cb80 10 client.17241 fetch_fsmap learned FSMap version 31 2024-05-10T11:27:26.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.539+0000 7f59c979cb80 1 -- 172.21.15.100:0/1234905534 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55815be3b000 con 0x55815be2b200 2024-05-10T11:27:26.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.540+0000 7f5997fff700 1 -- 172.21.15.100:0/1234905534 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f59a40b31f0 con 0x55815be2b200 2024-05-10T11:27:26.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.540+0000 7f5997fff700 1 -- 172.21.15.100:0/1234905534 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f59a40a4ce0 con 0x55815be2b200 2024-05-10T11:27:26.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.540+0000 7f59c979cb80 10 client.17241 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:27:26.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.540+0000 7f59c979cb80 10 client.17241 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:27:26.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.540+0000 7f59c979cb80 20 client.17241 populate_metadata read hostname 'smithi100' 2024-05-10T11:27:26.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.540+0000 7f59c979cb80 1 --2- 172.21.15.100:0/1234905534 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55815beaed20 0x55815beb1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:26.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.540+0000 7f59c979cb80 4 client.17241 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T11:27:26.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.540+0000 7f59c979cb80 1 -- 172.21.15.100:0/1234905534 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55815bd86240 con 0x55815beaed20 2024-05-10T11:27:26.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.540+0000 7f59bcca5700 1 --2- 172.21.15.100:0/1234905534 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55815beaed20 0x55815beb1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:26.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.540+0000 7f5997fff700 10 client.17241 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:27:26.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.540+0000 7f59bcca5700 1 --2- 172.21.15.100:0/1234905534 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55815beaed20 0x55815beb1140 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e978f6ef6357b15a server_cookie=ca1c3bfbdedeaaa6 in_seq=0 out_seq=0 2024-05-10T11:27:26.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.541+0000 7f5997fff700 1 -- 172.21.15.100:0/1234905534 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55815bd86240 con 0x55815beaed20 2024-05-10T11:27:26.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.541+0000 7f5997fff700 10 client.17241 handle_command_reply: tid=0 2024-05-10T11:27:26.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.541+0000 7f59c979cb80 1 client.17241 shutdown 2024-05-10T11:27:26.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.541+0000 7f59c979cb80 1 -- 172.21.15.100:0/1234905534 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55815be2b200 msgr2=0x55815be4b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:26.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.541+0000 7f59c979cb80 1 --2- 172.21.15.100:0/1234905534 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55815be2b200 0x55815be4b950 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f59a40b1f10 tx=0x7f59a4005c00 comp rx=0 tx=0).stop 2024-05-10T11:27:26.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.541+0000 7f59c979cb80 1 -- 172.21.15.100:0/1234905534 shutdown_connections 2024-05-10T11:27:26.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.541+0000 7f59c979cb80 1 --2- 172.21.15.100:0/1234905534 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55815beaed20 0x55815beb1140 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:26.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.541+0000 7f59c979cb80 1 --2- 172.21.15.100:0/1234905534 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55815be2f040 0x55815be3aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:26.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.541+0000 7f59c979cb80 1 --2- 172.21.15.100:0/1234905534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55815be2a8e0 0x55815be4b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:26.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.541+0000 7f5997fff700 0 client.17241 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:27:26.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.541+0000 7f59c979cb80 1 --2- 172.21.15.100:0/1234905534 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55815be2b200 0x55815be4b950 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:26.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.541+0000 7f59c979cb80 1 -- 172.21.15.100:0/1234905534 >> 172.21.15.100:0/1234905534 conn(0x55815bd52100 msgr2=0x55815be1ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:27:26.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.541+0000 7f59c979cb80 1 -- 172.21.15.100:0/1234905534 shutdown_connections 2024-05-10T11:27:26.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.541+0000 7f59c979cb80 1 -- 172.21.15.100:0/1234905534 wait complete. 2024-05-10T11:27:26.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.541+0000 7f59c979cb80 20 client.17241 trim_cache size 0 max 16384 2024-05-10T11:27:26.547 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:27:26.547 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T11:27:26.547 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-05-10T11:27:26.547 INFO:teuthology.orchestra.run.smithi100.stdout: "da27c588-5931-485f-927f-46f6838a17c9": { 2024-05-10T11:27:26.547 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-05-10T11:27:26.547 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "da27c588-5931-485f-927f-46f6838a17c9", 2024-05-10T11:27:26.547 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-05-10T11:27:26.547 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:27:26.547 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:27:26.547 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:27:26.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.542+0000 7f59bef09700 1 -- 172.21.15.100:0/3261425495 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f59a80b2880 msgr2=0x7f59a80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:26.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.542+0000 7f59bef09700 1 --2- 172.21.15.100:0/3261425495 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f59a80b2880 0x7f59a80b4d40 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f59ac009fd0 tx=0x7f59ac009260 comp rx=0 tx=0).stop 2024-05-10T11:27:26.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.542+0000 7f59bef09700 1 -- 172.21.15.100:0/3261425495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59b8102d50 msgr2=0x7f59b819ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:26.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.542+0000 7f59bef09700 1 --2- 172.21.15.100:0/3261425495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59b8102d50 0x7f59b819ae40 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f59a4006010 tx=0x7f59a400bfc0 comp rx=0 tx=0).stop 2024-05-10T11:27:26.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.542+0000 7f59bef09700 1 -- 172.21.15.100:0/3261425495 shutdown_connections 2024-05-10T11:27:26.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.542+0000 7f59bef09700 1 --2- 172.21.15.100:0/3261425495 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f59a80b2880 0x7f59a80b4d40 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:26.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.542+0000 7f59bef09700 1 --2- 172.21.15.100:0/3261425495 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f59b8108400 0x7f59b81a5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:26.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.542+0000 7f59bef09700 1 --2- 172.21.15.100:0/3261425495 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59b8102390 0x7f59b819a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:26.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.542+0000 7f59bef09700 1 --2- 172.21.15.100:0/3261425495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59b8102d50 0x7f59b819ae40 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:26.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.542+0000 7f59bef09700 1 -- 172.21.15.100:0/3261425495 >> 172.21.15.100:0/3261425495 conn(0x7f59b8079310 msgr2=0x7f59b81010e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:27:26.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.542+0000 7f59bef09700 1 -- 172.21.15.100:0/3261425495 shutdown_connections 2024-05-10T11:27:26.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:26.542+0000 7f59bef09700 1 -- 172.21.15.100:0/3261425495 wait complete. 2024-05-10T11:27:26.977 INFO:tasks.workunit.client.0.smithi100.stdout: 1 227435 2.74 MB/sec execute 321 sec latency 25.335 ms 2024-05-10T11:27:27.278 INFO:tasks.cephfs.filesystem:scrub status for tag:da27c588-5931-485f-927f-46f6838a17c9 - {'path': '/', 'tag': 'da27c588-5931-485f-927f-46f6838a17c9', 'options': 'recursive,force'} 2024-05-10T11:27:27.977 INFO:tasks.workunit.client.0.smithi100.stdout: 1 227729 2.73 MB/sec execute 322 sec latency 15.682 ms 2024-05-10T11:27:28.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:27:27 smithi178 ceph-mon[33359]: pgmap v692: 129 pgs: 129 active+clean; 96 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 1.3 MiB/s wr, 170 op/s 2024-05-10T11:27:28.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:27:27 smithi188 ceph-mon[34075]: pgmap v692: 129 pgs: 129 active+clean; 96 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 1.3 MiB/s wr, 170 op/s 2024-05-10T11:27:28.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:27:27 smithi100 ceph-mon[30296]: pgmap v692: 129 pgs: 129 active+clean; 96 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 1.3 MiB/s wr, 170 op/s 2024-05-10T11:27:28.978 INFO:tasks.workunit.client.0.smithi100.stdout: 1 228239 2.73 MB/sec execute 323 sec latency 15.088 ms 2024-05-10T11:27:29.977 INFO:tasks.workunit.client.0.smithi100.stdout: 1 228867 2.73 MB/sec execute 324 sec latency 22.180 ms 2024-05-10T11:27:30.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:27:29 smithi178 ceph-mon[33359]: pgmap v693: 129 pgs: 129 active+clean; 96 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 1.3 MiB/s wr, 135 op/s 2024-05-10T11:27:30.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:27:29 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:27:30.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:27:29 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:27:30.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:27:29 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:27:30.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:27:29 smithi188 ceph-mon[34075]: pgmap v693: 129 pgs: 129 active+clean; 96 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 1.3 MiB/s wr, 135 op/s 2024-05-10T11:27:30.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:27:29 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:27:30.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:27:29 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:27:30.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:27:29 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:27:30.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:27:29 smithi100 ceph-mon[30296]: pgmap v693: 129 pgs: 129 active+clean; 96 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 1.3 MiB/s wr, 135 op/s 2024-05-10T11:27:30.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:27:29 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:27:30.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:27:29 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:27:30.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:27:29 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:27:30.977 INFO:tasks.workunit.client.0.smithi100.stdout: 1 229452 2.73 MB/sec execute 325 sec latency 15.412 ms 2024-05-10T11:27:31.977 INFO:tasks.workunit.client.0.smithi100.stdout: 1 230078 2.74 MB/sec execute 326 sec latency 15.815 ms 2024-05-10T11:27:32.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:27:31 smithi178 ceph-mon[33359]: pgmap v694: 129 pgs: 129 active+clean; 97 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 1.1 MiB/s wr, 136 op/s 2024-05-10T11:27:32.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:27:31 smithi188 ceph-mon[34075]: pgmap v694: 129 pgs: 129 active+clean; 97 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 1.1 MiB/s wr, 136 op/s 2024-05-10T11:27:32.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:27:31 smithi100 ceph-mon[30296]: pgmap v694: 129 pgs: 129 active+clean; 97 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 1.1 MiB/s wr, 136 op/s 2024-05-10T11:27:32.978 INFO:tasks.workunit.client.0.smithi100.stdout: 1 230679 2.75 MB/sec execute 327 sec latency 14.692 ms 2024-05-10T11:27:33.978 INFO:tasks.workunit.client.0.smithi100.stdout: 1 230992 2.74 MB/sec execute 328 sec latency 13.717 ms 2024-05-10T11:27:34.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:27:33 smithi178 ceph-mon[33359]: pgmap v695: 129 pgs: 129 active+clean; 96 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 1.3 MiB/s wr, 145 op/s 2024-05-10T11:27:34.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:27:33 smithi188 ceph-mon[34075]: pgmap v695: 129 pgs: 129 active+clean; 96 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 1.3 MiB/s wr, 145 op/s 2024-05-10T11:27:34.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:27:33 smithi100 ceph-mon[30296]: pgmap v695: 129 pgs: 129 active+clean; 96 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 1.3 MiB/s wr, 145 op/s 2024-05-10T11:27:34.779 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.777+0000 7f6b5476a640 1 -- 172.21.15.100:0/1267094436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b4c103490 msgr2=0x7f6b4c105880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:34.779 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.777+0000 7f6b5476a640 1 --2- 172.21.15.100:0/1267094436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b4c103490 0x7f6b4c105880 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c009900 tx=0x7f6b3c02a410 comp rx=0 tx=0).stop 2024-05-10T11:27:34.779 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.777+0000 7f6b5476a640 1 -- 172.21.15.100:0/1267094436 shutdown_connections 2024-05-10T11:27:34.779 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.777+0000 7f6b5476a640 1 --2- 172.21.15.100:0/1267094436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b4c1086f0 0x7f6b4c10ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:34.779 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.777+0000 7f6b5476a640 1 --2- 172.21.15.100:0/1267094436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b4c105dc0 0x7f6b4c1081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:34.779 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.777+0000 7f6b5476a640 1 --2- 172.21.15.100:0/1267094436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b4c103490 0x7f6b4c105880 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:34.780 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.777+0000 7f6b5476a640 1 -- 172.21.15.100:0/1267094436 >> 172.21.15.100:0/1267094436 conn(0x7f6b4c0fce80 msgr2=0x7f6b4c0ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:27:34.780 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.777+0000 7f6b5476a640 1 -- 172.21.15.100:0/1267094436 shutdown_connections 2024-05-10T11:27:34.780 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.777+0000 7f6b5476a640 1 -- 172.21.15.100:0/1267094436 wait complete. 2024-05-10T11:27:34.780 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.777+0000 7f6b5476a640 1 Processor -- start 2024-05-10T11:27:34.780 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.777+0000 7f6b5476a640 1 -- start start 2024-05-10T11:27:34.780 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.778+0000 7f6b5476a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b4c103490 0x7f6b4c198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:34.780 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.778+0000 7f6b5476a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b4c105dc0 0x7f6b4c199020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:34.780 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.778+0000 7f6b5476a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b4c1086f0 0x7f6b4c1a0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:34.780 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.778+0000 7f6b5476a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6b4c10e410 con 0x7f6b4c105dc0 2024-05-10T11:27:34.780 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.778+0000 7f6b5476a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6b4c10e290 con 0x7f6b4c1086f0 2024-05-10T11:27:34.780 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.778+0000 7f6b5476a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6b4c10e590 con 0x7f6b4c103490 2024-05-10T11:27:34.780 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.778+0000 7f6b51d05640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b4c105dc0 0x7f6b4c199020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:34.780 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.778+0000 7f6b51d05640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b4c105dc0 0x7f6b4c199020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46700/0 (socket says 172.21.15.100:46700) 2024-05-10T11:27:34.780 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.778+0000 7f6b51d05640 1 -- 172.21.15.100:0/68036530 learned_addr learned my addr 172.21.15.100:0/68036530 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:27:34.780 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.778+0000 7f6b52d07640 1 --2- 172.21.15.100:0/68036530 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b4c1086f0 0x7f6b4c1a0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:34.780 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.778+0000 7f6b52506640 1 --2- 172.21.15.100:0/68036530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b4c103490 0x7f6b4c198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.778+0000 7f6b51d05640 1 -- 172.21.15.100:0/68036530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b4c103490 msgr2=0x7f6b4c198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.778+0000 7f6b51d05640 1 --2- 172.21.15.100:0/68036530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b4c103490 0x7f6b4c198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.778+0000 7f6b51d05640 1 -- 172.21.15.100:0/68036530 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b4c1086f0 msgr2=0x7f6b4c1a0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.778+0000 7f6b51d05640 1 --2- 172.21.15.100:0/68036530 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b4c1086f0 0x7f6b4c1a0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.778+0000 7f6b51d05640 1 -- 172.21.15.100:0/68036530 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b4c1a0700 con 0x7f6b4c105dc0 2024-05-10T11:27:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.778+0000 7f6b51d05640 1 --2- 172.21.15.100:0/68036530 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b4c105dc0 0x7f6b4c199020 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f6b48005fd0 tx=0x7f6b4800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:27:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.778+0000 7f6b437fe640 1 -- 172.21.15.100:0/68036530 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b48017070 con 0x7f6b4c105dc0 2024-05-10T11:27:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.778+0000 7f6b5476a640 1 -- 172.21.15.100:0/68036530 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b4c1a09f0 con 0x7f6b4c105dc0 2024-05-10T11:27:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.778+0000 7f6b437fe640 1 -- 172.21.15.100:0/68036530 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6b48009c30 con 0x7f6b4c105dc0 2024-05-10T11:27:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.778+0000 7f6b437fe640 1 -- 172.21.15.100:0/68036530 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b48003c70 con 0x7f6b4c105dc0 2024-05-10T11:27:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.779+0000 7f6b5476a640 1 -- 172.21.15.100:0/68036530 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b4c1a0f90 con 0x7f6b4c105dc0 2024-05-10T11:27:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.781+0000 7f6b437fe640 1 -- 172.21.15.100:0/68036530 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f6b48003e10 con 0x7f6b4c105dc0 2024-05-10T11:27:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.781+0000 7f6b57480740 1 Processor -- start 2024-05-10T11:27:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.781+0000 7f6b57480740 1 -- start start 2024-05-10T11:27:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.782+0000 7f6b57480740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc4accadb0 0x55cc4accb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.782+0000 7f6b57480740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc4accb7a0 0x55cc4acd02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.782+0000 7f6b57480740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc4acd0910 0x55cc4acd2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.782+0000 7f6b57480740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55cc4ac2f5d0 con 0x55cc4acd0910 2024-05-10T11:27:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.782+0000 7f6b57480740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55cc4abe8890 con 0x55cc4accadb0 2024-05-10T11:27:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.782+0000 7f6b57480740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55cc4ac2b590 con 0x55cc4accb7a0 2024-05-10T11:27:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.782+0000 7f6b51d05640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc4acd0910 0x55cc4acd2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.782+0000 7f6b52506640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc4accb7a0 0x55cc4acd02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.782+0000 7f6b51d05640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc4acd0910 0x55cc4acd2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46706/0 (socket says 172.21.15.100:46706) 2024-05-10T11:27:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.782+0000 7f6b51d05640 1 -- 172.21.15.100:0/522096981 learned_addr learned my addr 172.21.15.100:0/522096981 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:27:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.782+0000 7f6b52d07640 1 --2- 172.21.15.100:0/522096981 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc4accadb0 0x55cc4accb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.782+0000 7f6b437fe640 1 --2- 172.21.15.100:0/68036530 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f6b380b28a0 0x7f6b380b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.782+0000 7f6b52d07640 1 --2- 172.21.15.100:0/68036530 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f6b380b28a0 0x7f6b380b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.782+0000 7f6b51d05640 1 -- 172.21.15.100:0/522096981 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc4accb7a0 msgr2=0x55cc4acd02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.782+0000 7f6b51d05640 1 --2- 172.21.15.100:0/522096981 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc4accb7a0 0x55cc4acd02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.782+0000 7f6b51d05640 1 -- 172.21.15.100:0/522096981 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc4accadb0 msgr2=0x55cc4accb1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.782+0000 7f6b51d05640 1 --2- 172.21.15.100:0/522096981 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc4accadb0 0x55cc4accb1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.782+0000 7f6b51d05640 1 -- 172.21.15.100:0/522096981 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc4a941210 con 0x55cc4acd0910 2024-05-10T11:27:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.782+0000 7f6b52506640 1 --2- 172.21.15.100:0/522096981 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc4accb7a0 0x55cc4acd02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:27:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.782+0000 7f6b437fe640 1 -- 172.21.15.100:0/68036530 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f6b480e21b0 con 0x7f6b4c105dc0 2024-05-10T11:27:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.782+0000 7f6b51d05640 1 --2- 172.21.15.100:0/522096981 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc4acd0910 0x55cc4acd2da0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f6b480a66e0 tx=0x7f6b48021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19a7442d023d17f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:27:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.782+0000 7f6b52d07640 1 --2- 172.21.15.100:0/68036530 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f6b380b28a0 0x7f6b380b4d60 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x55cc4accc000 tx=0x7f6b44004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:27:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.783+0000 7f6b40ff9640 1 -- 172.21.15.100:0/522096981 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b480035f0 con 0x55cc4acd0910 2024-05-10T11:27:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.783+0000 7f6b40ff9640 1 -- 172.21.15.100:0/522096981 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6b480a8f80 con 0x55cc4acd0910 2024-05-10T11:27:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.783+0000 7f6b40ff9640 1 -- 172.21.15.100:0/522096981 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b480a9470 con 0x55cc4acd0910 2024-05-10T11:27:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.783+0000 7f6b57480740 1 -- 172.21.15.100:0/522096981 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc4acd0910 msgr2=0x55cc4acd2da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.783+0000 7f6b57480740 1 --2- 172.21.15.100:0/522096981 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc4acd0910 0x55cc4acd2da0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f6b480a66e0 tx=0x7f6b48021000 comp rx=0 tx=0).stop 2024-05-10T11:27:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.783+0000 7f6b57480740 1 -- 172.21.15.100:0/522096981 shutdown_connections 2024-05-10T11:27:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.783+0000 7f6b57480740 1 --2- 172.21.15.100:0/522096981 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc4acd0910 0x55cc4acd2da0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.783+0000 7f6b57480740 1 --2- 172.21.15.100:0/522096981 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc4accb7a0 0x55cc4acd02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:34.790 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.783+0000 7f6b57480740 1 --2- 172.21.15.100:0/522096981 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc4accadb0 0x55cc4accb1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:34.790 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.783+0000 7f6b57480740 1 -- 172.21.15.100:0/522096981 >> 172.21.15.100:0/522096981 conn(0x55cc4acc6420 msgr2=0x55cc4acc6820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:27:34.790 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.784+0000 7f6b57480740 1 -- 172.21.15.100:0/522096981 shutdown_connections 2024-05-10T11:27:34.790 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.784+0000 7f6b57480740 1 -- 172.21.15.100:0/522096981 wait complete. 2024-05-10T11:27:34.790 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.784+0000 7f6b57480740 1 -- start start 2024-05-10T11:27:34.790 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.784+0000 7f6b57480740 1 Processor -- start 2024-05-10T11:27:34.790 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.784+0000 7f6b57480740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc4accadb0 0x55cc4acd4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:34.790 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.784+0000 7f6b57480740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc4accb7a0 0x55cc4acd5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:34.790 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.784+0000 7f6b57480740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc4acd0910 0x55cc4acdc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:34.790 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.784+0000 7f6b57480740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55cc4acd3800 con 0x55cc4acd0910 2024-05-10T11:27:34.790 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.784+0000 7f6b57480740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55cc4abec370 con 0x55cc4accb7a0 2024-05-10T11:27:34.790 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.784+0000 7f6b57480740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55cc4acd3980 con 0x55cc4accadb0 2024-05-10T11:27:34.790 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.784+0000 7f6b57480740 10 client.0 _ll_register_callbacks cb 0x55cc4a940a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:27:34.790 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.784+0000 7f6b52d07640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc4acd0910 0x55cc4acdc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:34.790 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.784+0000 7f6b52d07640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc4acd0910 0x55cc4acdc3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46722/0 (socket says 172.21.15.100:46722) 2024-05-10T11:27:34.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.784+0000 7f6b52d07640 1 -- 172.21.15.100:0/2173097133 learned_addr learned my addr 172.21.15.100:0/2173097133 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:27:34.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.784+0000 7f6b51d05640 1 --2- 172.21.15.100:0/2173097133 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc4accb7a0 0x55cc4acd5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:34.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.784+0000 7f6b52506640 1 --2- 172.21.15.100:0/2173097133 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc4accadb0 0x55cc4acd4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:34.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.785+0000 7f6b52d07640 1 -- 172.21.15.100:0/2173097133 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc4accadb0 msgr2=0x55cc4acd4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:34.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.785+0000 7f6b52d07640 1 --2- 172.21.15.100:0/2173097133 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc4accadb0 0x55cc4acd4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:34.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.785+0000 7f6b52d07640 1 -- 172.21.15.100:0/2173097133 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc4accb7a0 msgr2=0x55cc4acd5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:34.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.785+0000 7f6b52d07640 1 --2- 172.21.15.100:0/2173097133 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc4accb7a0 0x55cc4acd5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:34.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.785+0000 7f6b52d07640 1 -- 172.21.15.100:0/2173097133 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc4aa0d620 con 0x55cc4acd0910 2024-05-10T11:27:34.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.785+0000 7f6b52d07640 1 --2- 172.21.15.100:0/2173097133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc4acd0910 0x55cc4acdc3c0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f6b4400f830 tx=0x7f6b44004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3365ec9583879bbc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:27:34.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.785+0000 7f6b35ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:27:34.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.785+0000 7f6b35ffb640 1 -- 172.21.15.100:0/2173097133 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b44025070 con 0x55cc4acd0910 2024-05-10T11:27:34.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.785+0000 7f6b35ffb640 1 -- 172.21.15.100:0/2173097133 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f6b44010dc0 con 0x55cc4acd0910 2024-05-10T11:27:34.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.785+0000 7f6b57480740 1 -- 172.21.15.100:0/2173097133 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b480a8f80 con 0x55cc4acd0910 2024-05-10T11:27:34.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.785+0000 7f6b35ffb640 1 -- 172.21.15.100:0/2173097133 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6b44033050 con 0x55cc4acd0910 2024-05-10T11:27:34.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.785+0000 7f6b57480740 10 client.17253 fetch_fsmap learned FSMap version 31 2024-05-10T11:27:34.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.785+0000 7f6b57480740 1 -- 172.21.15.100:0/2173097133 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55cc4acdf410 con 0x55cc4acd0910 2024-05-10T11:27:34.792 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.785+0000 7f6b35ffb640 1 -- 172.21.15.100:0/2173097133 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6b480a8f80 con 0x55cc4acd0910 2024-05-10T11:27:34.792 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.786+0000 7f6b35ffb640 1 -- 172.21.15.100:0/2173097133 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f6b4401f3d0 con 0x55cc4acd0910 2024-05-10T11:27:34.792 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.786+0000 7f6b57480740 10 client.17253 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:27:34.792 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.786+0000 7f6b57480740 10 client.17253 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:27:34.792 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.786+0000 7f6b57480740 20 client.17253 populate_metadata read hostname 'smithi100' 2024-05-10T11:27:34.792 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.786+0000 7f6b57480740 1 --2- 172.21.15.100:0/2173097133 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55cc4ace0b50 0x55cc4ace2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:34.792 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.786+0000 7f6b57480740 4 client.17253 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:27:34.792 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.786+0000 7f6b57480740 1 -- 172.21.15.100:0/2173097133 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc4ac3d1e0 con 0x55cc4ace0b50 2024-05-10T11:27:34.792 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.786+0000 7f6b52506640 1 --2- 172.21.15.100:0/2173097133 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55cc4ace0b50 0x55cc4ace2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:34.792 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.786+0000 7f6b52506640 1 --2- 172.21.15.100:0/2173097133 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55cc4ace0b50 0x55cc4ace2f40 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7503e0fccf424068 server_cookie=aca2484971e94004 in_seq=0 out_seq=0 2024-05-10T11:27:34.792 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.786+0000 7f6b35ffb640 10 client.17253 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:27:34.792 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.787+0000 7f6b35ffb640 1 -- 172.21.15.100:0/2173097133 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55cc4ac3d1e0 con 0x55cc4ace0b50 2024-05-10T11:27:34.792 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.787+0000 7f6b35ffb640 10 client.17253 handle_command_reply: tid=0 2024-05-10T11:27:34.792 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.787+0000 7f6b57480740 1 client.17253 shutdown 2024-05-10T11:27:34.792 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.787+0000 7f6b57480740 1 -- 172.21.15.100:0/2173097133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc4acd0910 msgr2=0x55cc4acdc3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:34.793 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.787+0000 7f6b57480740 1 --2- 172.21.15.100:0/2173097133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc4acd0910 0x55cc4acdc3c0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f6b4400f830 tx=0x7f6b44004a40 comp rx=0 tx=0).stop 2024-05-10T11:27:34.793 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.787+0000 7f6b57480740 1 -- 172.21.15.100:0/2173097133 shutdown_connections 2024-05-10T11:27:34.793 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.787+0000 7f6b57480740 1 --2- 172.21.15.100:0/2173097133 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55cc4ace0b50 0x55cc4ace2f40 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:34.793 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.787+0000 7f6b57480740 1 --2- 172.21.15.100:0/2173097133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc4acd0910 0x55cc4acdc3c0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:34.793 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.787+0000 7f6b57480740 1 --2- 172.21.15.100:0/2173097133 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc4accb7a0 0x55cc4acd5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:34.793 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.787+0000 7f6b57480740 1 --2- 172.21.15.100:0/2173097133 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc4accadb0 0x55cc4acd4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:34.793 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.787+0000 7f6b57480740 1 -- 172.21.15.100:0/2173097133 >> 172.21.15.100:0/2173097133 conn(0x55cc4acc6420 msgr2=0x55cc4acc5580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:27:34.793 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.787+0000 7f6b35ffb640 0 client.17253 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:27:34.793 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.787+0000 7f6b57480740 1 -- 172.21.15.100:0/2173097133 shutdown_connections 2024-05-10T11:27:34.793 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.787+0000 7f6b57480740 1 -- 172.21.15.100:0/2173097133 wait complete. 2024-05-10T11:27:34.793 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.787+0000 7f6b57480740 20 client.17253 trim_cache size 0 max 16384 2024-05-10T11:27:34.804 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.803+0000 7f6b57480740 1 Processor -- start 2024-05-10T11:27:34.804 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.803+0000 7f6b57480740 1 -- start start 2024-05-10T11:27:34.804 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.803+0000 7f6b57480740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc4accadb0 0x55cc4acd4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:34.804 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.803+0000 7f6b57480740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc4accb7a0 0x55cc4acdc380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.803+0000 7f6b57480740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc4acd0910 0x55cc4acc9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.803+0000 7f6b57480740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6b4c1adc00 con 0x55cc4accadb0 2024-05-10T11:27:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.803+0000 7f6b57480740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6b4c1ab4d0 con 0x55cc4accb7a0 2024-05-10T11:27:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.803+0000 7f6b57480740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6b4c1a9320 con 0x55cc4acd0910 2024-05-10T11:27:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.803+0000 7f6b52506640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc4accadb0 0x55cc4acd4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.803+0000 7f6b52506640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc4accadb0 0x55cc4acd4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46730/0 (socket says 172.21.15.100:46730) 2024-05-10T11:27:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.804+0000 7f6b52506640 1 -- 172.21.15.100:0/3930662434 learned_addr learned my addr 172.21.15.100:0/3930662434 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:27:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.804+0000 7f6b52d07640 1 --2- 172.21.15.100:0/3930662434 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc4acd0910 0x55cc4acc9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.804+0000 7f6b51d05640 1 --2- 172.21.15.100:0/3930662434 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc4accb7a0 0x55cc4acdc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.804+0000 7f6b52506640 1 -- 172.21.15.100:0/3930662434 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc4acd0910 msgr2=0x55cc4acc9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.804+0000 7f6b52506640 1 --2- 172.21.15.100:0/3930662434 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc4acd0910 0x55cc4acc9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.804+0000 7f6b52506640 1 -- 172.21.15.100:0/3930662434 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc4accb7a0 msgr2=0x55cc4acdc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.804+0000 7f6b52506640 1 --2- 172.21.15.100:0/3930662434 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc4accb7a0 0x55cc4acdc380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.804+0000 7f6b52506640 1 -- 172.21.15.100:0/3930662434 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc4acc6250 con 0x55cc4accadb0 2024-05-10T11:27:34.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.804+0000 7f6b52506640 1 --2- 172.21.15.100:0/3930662434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc4accadb0 0x55cc4acd4460 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c009900 tx=0x7f6b3c004310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a3d77ff6a2ec7b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:27:34.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.804+0000 7f6b37fff640 1 -- 172.21.15.100:0/3930662434 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b3c035040 con 0x55cc4accadb0 2024-05-10T11:27:34.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.804+0000 7f6b37fff640 1 -- 172.21.15.100:0/3930662434 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6b3c031c50 con 0x55cc4accadb0 2024-05-10T11:27:34.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.804+0000 7f6b37fff640 1 -- 172.21.15.100:0/3930662434 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b3c046a10 con 0x55cc4accadb0 2024-05-10T11:27:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.805+0000 7f6b57480740 1 -- 172.21.15.100:0/3930662434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc4accadb0 msgr2=0x55cc4acd4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.805+0000 7f6b57480740 1 --2- 172.21.15.100:0/3930662434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc4accadb0 0x55cc4acd4460 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c009900 tx=0x7f6b3c004310 comp rx=0 tx=0).stop 2024-05-10T11:27:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.805+0000 7f6b57480740 1 -- 172.21.15.100:0/3930662434 shutdown_connections 2024-05-10T11:27:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.805+0000 7f6b57480740 1 --2- 172.21.15.100:0/3930662434 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc4acd0910 0x55cc4acc9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.805+0000 7f6b57480740 1 --2- 172.21.15.100:0/3930662434 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc4accb7a0 0x55cc4acdc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.805+0000 7f6b57480740 1 --2- 172.21.15.100:0/3930662434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc4accadb0 0x55cc4acd4460 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.805+0000 7f6b57480740 1 -- 172.21.15.100:0/3930662434 >> 172.21.15.100:0/3930662434 conn(0x55cc4acc6420 msgr2=0x55cc4acc9100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:27:34.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.805+0000 7f6b57480740 1 -- 172.21.15.100:0/3930662434 shutdown_connections 2024-05-10T11:27:34.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.805+0000 7f6b57480740 1 -- 172.21.15.100:0/3930662434 wait complete. 2024-05-10T11:27:34.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.805+0000 7f6b57480740 1 -- start start 2024-05-10T11:27:34.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.805+0000 7f6b57480740 1 Processor -- start 2024-05-10T11:27:34.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.806+0000 7f6b57480740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc4accadb0 0x55cc4acda090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:34.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.806+0000 7f6b57480740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc4accb7a0 0x55cc4acda5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:34.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.806+0000 7f6b57480740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc4acd0910 0x55cc4acdb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:34.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.806+0000 7f6b57480740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6b4c1a9990 con 0x55cc4accadb0 2024-05-10T11:27:34.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.806+0000 7f6b57480740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6b4c1aae00 con 0x55cc4acd0910 2024-05-10T11:27:34.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.806+0000 7f6b57480740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6b4c1aa060 con 0x55cc4accb7a0 2024-05-10T11:27:34.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.806+0000 7f6b57480740 10 client.0 _ll_register_callbacks cb 0x7f6b44009ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:27:34.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.806+0000 7f6b52506640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc4accadb0 0x55cc4acda090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:34.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.806+0000 7f6b52506640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc4accadb0 0x55cc4acda090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46744/0 (socket says 172.21.15.100:46744) 2024-05-10T11:27:34.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.806+0000 7f6b52506640 1 -- 172.21.15.100:0/3469619676 learned_addr learned my addr 172.21.15.100:0/3469619676 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:27:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.806+0000 7f6b52d07640 1 --2- 172.21.15.100:0/3469619676 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc4acd0910 0x55cc4acdb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.806+0000 7f6b51d05640 1 --2- 172.21.15.100:0/3469619676 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc4accb7a0 0x55cc4acda5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.806+0000 7f6b52506640 1 -- 172.21.15.100:0/3469619676 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc4accb7a0 msgr2=0x55cc4acda5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.806+0000 7f6b52506640 1 --2- 172.21.15.100:0/3469619676 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc4accb7a0 0x55cc4acda5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.806+0000 7f6b52506640 1 -- 172.21.15.100:0/3469619676 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc4acd0910 msgr2=0x55cc4acdb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.806+0000 7f6b52506640 1 --2- 172.21.15.100:0/3469619676 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc4acd0910 0x55cc4acdb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.806+0000 7f6b52506640 1 -- 172.21.15.100:0/3469619676 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc4acdbb80 con 0x55cc4accadb0 2024-05-10T11:27:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.806+0000 7f6b52d07640 1 --2- 172.21.15.100:0/3469619676 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc4acd0910 0x55cc4acdb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:27:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.806+0000 7f6b52506640 1 --2- 172.21.15.100:0/3469619676 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc4accadb0 0x55cc4acda090 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c02a930 tx=0x7f6b3c02ab10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da8f29bc6fc0f6aa server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:27:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.806+0000 7f6b35ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:27:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.806+0000 7f6b35ffb640 1 -- 172.21.15.100:0/3469619676 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b3c054070 con 0x55cc4accadb0 2024-05-10T11:27:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.806+0000 7f6b35ffb640 1 -- 172.21.15.100:0/3469619676 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f6b3c004400 con 0x55cc4accadb0 2024-05-10T11:27:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.806+0000 7f6b57480740 1 -- 172.21.15.100:0/3469619676 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b3c031c50 con 0x55cc4accadb0 2024-05-10T11:27:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.807+0000 7f6b35ffb640 1 -- 172.21.15.100:0/3469619676 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6b3c060050 con 0x55cc4accadb0 2024-05-10T11:27:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.807+0000 7f6b57480740 10 client.17259 fetch_fsmap learned FSMap version 31 2024-05-10T11:27:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.807+0000 7f6b57480740 1 -- 172.21.15.100:0/3469619676 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55cc4ad4b940 con 0x55cc4accadb0 2024-05-10T11:27:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.807+0000 7f6b35ffb640 1 -- 172.21.15.100:0/3469619676 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6b3c031c50 con 0x55cc4accadb0 2024-05-10T11:27:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.807+0000 7f6b35ffb640 1 -- 172.21.15.100:0/3469619676 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f6b3c0509f0 con 0x55cc4accadb0 2024-05-10T11:27:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.807+0000 7f6b57480740 10 client.17259 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:27:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.807+0000 7f6b57480740 10 client.17259 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:27:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.807+0000 7f6b57480740 20 client.17259 populate_metadata read hostname 'smithi100' 2024-05-10T11:27:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.807+0000 7f6b57480740 1 --2- 172.21.15.100:0/3469619676 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55cc4ad4c340 0x55cc4ad4e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.807+0000 7f6b57480740 4 client.17259 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:27:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.807+0000 7f6b57480740 1 -- 172.21.15.100:0/3469619676 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cc4ac37050 con 0x55cc4ad4c340 2024-05-10T11:27:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.807+0000 7f6b51d05640 1 --2- 172.21.15.100:0/3469619676 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55cc4ad4c340 0x55cc4ad4e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.808+0000 7f6b35ffb640 10 client.17259 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:27:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.808+0000 7f6b51d05640 1 --2- 172.21.15.100:0/3469619676 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55cc4ad4c340 0x55cc4ad4e760 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6e1725e5ab485a9 server_cookie=dffb4b07d60c3e8d in_seq=0 out_seq=0 2024-05-10T11:27:34.824 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.824+0000 7f6b35ffb640 1 -- 172.21.15.100:0/3469619676 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cc4ac37050 con 0x55cc4ad4c340 2024-05-10T11:27:34.824 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.824+0000 7f6b35ffb640 10 client.17259 handle_command_reply: tid=0 2024-05-10T11:27:34.825 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.824+0000 7f6b57480740 1 client.17259 shutdown 2024-05-10T11:27:34.825 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.824+0000 7f6b57480740 1 -- 172.21.15.100:0/3469619676 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc4accadb0 msgr2=0x55cc4acda090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:34.825 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.824+0000 7f6b57480740 1 --2- 172.21.15.100:0/3469619676 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc4accadb0 0x55cc4acda090 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c02a930 tx=0x7f6b3c02ab10 comp rx=0 tx=0).stop 2024-05-10T11:27:34.825 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.824+0000 7f6b57480740 1 -- 172.21.15.100:0/3469619676 shutdown_connections 2024-05-10T11:27:34.825 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.824+0000 7f6b57480740 1 --2- 172.21.15.100:0/3469619676 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55cc4ad4c340 0x55cc4ad4e760 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:34.825 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.824+0000 7f6b57480740 1 --2- 172.21.15.100:0/3469619676 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc4acd0910 0x55cc4acdb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:34.825 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.824+0000 7f6b57480740 1 --2- 172.21.15.100:0/3469619676 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc4accb7a0 0x55cc4acda5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:34.825 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.824+0000 7f6b35ffb640 0 client.17259 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:27:34.825 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.824+0000 7f6b57480740 1 --2- 172.21.15.100:0/3469619676 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc4accadb0 0x55cc4acda090 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:34.825 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.824+0000 7f6b57480740 1 -- 172.21.15.100:0/3469619676 >> 172.21.15.100:0/3469619676 conn(0x55cc4acc6420 msgr2=0x55cc4acc9100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:27:34.825 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.824+0000 7f6b57480740 1 -- 172.21.15.100:0/3469619676 shutdown_connections 2024-05-10T11:27:34.825 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.824+0000 7f6b57480740 1 -- 172.21.15.100:0/3469619676 wait complete. 2024-05-10T11:27:34.825 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.824+0000 7f6b57480740 20 client.17259 trim_cache size 0 max 16384 2024-05-10T11:27:34.826 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:27:34.826 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:27:34.826 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:27:34.826 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:27:34.827 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.825+0000 7f6b5476a640 1 -- 172.21.15.100:0/68036530 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f6b380b28a0 msgr2=0x7f6b380b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:34.827 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.825+0000 7f6b5476a640 1 --2- 172.21.15.100:0/68036530 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f6b380b28a0 0x7f6b380b4d60 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x55cc4accc000 tx=0x7f6b44004ab0 comp rx=0 tx=0).stop 2024-05-10T11:27:34.827 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.825+0000 7f6b5476a640 1 -- 172.21.15.100:0/68036530 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b4c105dc0 msgr2=0x7f6b4c199020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:34.827 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.825+0000 7f6b5476a640 1 --2- 172.21.15.100:0/68036530 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b4c105dc0 0x7f6b4c199020 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f6b48005fd0 tx=0x7f6b4800dde0 comp rx=0 tx=0).stop 2024-05-10T11:27:34.827 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.825+0000 7f6b5476a640 1 -- 172.21.15.100:0/68036530 shutdown_connections 2024-05-10T11:27:34.827 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.825+0000 7f6b5476a640 1 --2- 172.21.15.100:0/68036530 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f6b380b28a0 0x7f6b380b4d60 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:34.827 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.825+0000 7f6b5476a640 1 --2- 172.21.15.100:0/68036530 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b4c1086f0 0x7f6b4c1a0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:34.827 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.825+0000 7f6b5476a640 1 --2- 172.21.15.100:0/68036530 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b4c105dc0 0x7f6b4c199020 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:34.827 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.825+0000 7f6b5476a640 1 --2- 172.21.15.100:0/68036530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b4c103490 0x7f6b4c198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:34.827 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.825+0000 7f6b5476a640 1 -- 172.21.15.100:0/68036530 >> 172.21.15.100:0/68036530 conn(0x7f6b4c0fce80 msgr2=0x7f6b4c103f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:27:34.827 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.825+0000 7f6b5476a640 1 -- 172.21.15.100:0/68036530 shutdown_connections 2024-05-10T11:27:34.827 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:34.825+0000 7f6b5476a640 1 -- 172.21.15.100:0/68036530 wait complete. 2024-05-10T11:27:34.978 INFO:tasks.workunit.client.0.smithi100.stdout: 1 231326 2.73 MB/sec execute 329 sec latency 18.877 ms 2024-05-10T11:27:35.978 INFO:tasks.workunit.client.0.smithi100.stdout: 1 231788 2.73 MB/sec execute 330 sec latency 85.225 ms 2024-05-10T11:27:36.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:27:35 smithi178 ceph-mon[33359]: pgmap v696: 129 pgs: 129 active+clean; 96 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 1.2 MiB/s wr, 143 op/s 2024-05-10T11:27:36.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:27:35 smithi188 ceph-mon[34075]: pgmap v696: 129 pgs: 129 active+clean; 96 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 1.2 MiB/s wr, 143 op/s 2024-05-10T11:27:36.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:27:35 smithi100 ceph-mon[30296]: pgmap v696: 129 pgs: 129 active+clean; 96 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 1.2 MiB/s wr, 143 op/s 2024-05-10T11:27:36.978 INFO:tasks.workunit.client.0.smithi100.stdout: 1 232380 2.73 MB/sec execute 331 sec latency 18.720 ms 2024-05-10T11:27:37.978 INFO:tasks.workunit.client.0.smithi100.stdout: 1 232994 2.73 MB/sec execute 332 sec latency 16.038 ms 2024-05-10T11:27:38.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:27:37 smithi178 ceph-mon[33359]: pgmap v697: 129 pgs: 129 active+clean; 95 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 1.4 MiB/s wr, 166 op/s 2024-05-10T11:27:38.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:27:37 smithi188 ceph-mon[34075]: pgmap v697: 129 pgs: 129 active+clean; 95 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 1.4 MiB/s wr, 166 op/s 2024-05-10T11:27:38.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:27:37 smithi100 ceph-mon[30296]: pgmap v697: 129 pgs: 129 active+clean; 95 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 1.4 MiB/s wr, 166 op/s 2024-05-10T11:27:38.978 INFO:tasks.workunit.client.0.smithi100.stdout: 1 233610 2.74 MB/sec execute 333 sec latency 14.889 ms 2024-05-10T11:27:39.978 INFO:tasks.workunit.client.0.smithi100.stdout: 1 234160 2.75 MB/sec execute 334 sec latency 15.026 ms 2024-05-10T11:27:40.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:27:39 smithi178 ceph-mon[33359]: pgmap v698: 129 pgs: 129 active+clean; 95 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 1.3 MiB/s wr, 151 op/s 2024-05-10T11:27:40.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:27:39 smithi188 ceph-mon[34075]: pgmap v698: 129 pgs: 129 active+clean; 95 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 1.3 MiB/s wr, 151 op/s 2024-05-10T11:27:40.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:27:39 smithi100 ceph-mon[30296]: pgmap v698: 129 pgs: 129 active+clean; 95 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 1.3 MiB/s wr, 151 op/s 2024-05-10T11:27:40.978 INFO:tasks.workunit.client.0.smithi100.stdout: 1 234526 2.74 MB/sec execute 335 sec latency 13.777 ms 2024-05-10T11:27:41.978 INFO:tasks.workunit.client.0.smithi100.stdout: 1 234840 2.73 MB/sec execute 336 sec latency 13.165 ms 2024-05-10T11:27:42.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:27:41 smithi178 ceph-mon[33359]: pgmap v699: 129 pgs: 129 active+clean; 96 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 1.2 MiB/s wr, 138 op/s 2024-05-10T11:27:42.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:27:41 smithi188 ceph-mon[34075]: pgmap v699: 129 pgs: 129 active+clean; 96 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 1.2 MiB/s wr, 138 op/s 2024-05-10T11:27:42.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:27:41 smithi100 ceph-mon[30296]: pgmap v699: 129 pgs: 129 active+clean; 96 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 1.2 MiB/s wr, 138 op/s 2024-05-10T11:27:42.978 INFO:tasks.workunit.client.0.smithi100.stdout: 1 235378 2.73 MB/sec execute 337 sec latency 13.672 ms 2024-05-10T11:27:43.978 INFO:tasks.workunit.client.0.smithi100.stdout: 1 236051 2.73 MB/sec execute 338 sec latency 14.595 ms 2024-05-10T11:27:44.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:27:43 smithi178 ceph-mon[33359]: pgmap v700: 129 pgs: 129 active+clean; 95 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 1.3 MiB/s wr, 152 op/s 2024-05-10T11:27:44.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:27:43 smithi188 ceph-mon[34075]: pgmap v700: 129 pgs: 129 active+clean; 95 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 1.3 MiB/s wr, 152 op/s 2024-05-10T11:27:44.291 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:27:43 smithi100 ceph-mon[30296]: pgmap v700: 129 pgs: 129 active+clean; 95 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 1.3 MiB/s wr, 152 op/s 2024-05-10T11:27:44.978 INFO:tasks.workunit.client.0.smithi100.stdout: 1 236622 2.73 MB/sec execute 339 sec latency 15.345 ms 2024-05-10T11:27:45.978 INFO:tasks.workunit.client.0.smithi100.stdout: 1 237428 2.74 MB/sec execute 340 sec latency 12.355 ms 2024-05-10T11:27:46.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:27:45 smithi178 ceph-mon[33359]: pgmap v701: 129 pgs: 129 active+clean; 97 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 1.4 MiB/s wr, 150 op/s 2024-05-10T11:27:46.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:27:45 smithi188 ceph-mon[34075]: pgmap v701: 129 pgs: 129 active+clean; 97 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 1.4 MiB/s wr, 150 op/s 2024-05-10T11:27:46.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:27:45 smithi100 ceph-mon[30296]: pgmap v701: 129 pgs: 129 active+clean; 97 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 1.4 MiB/s wr, 150 op/s 2024-05-10T11:27:46.978 INFO:tasks.workunit.client.0.smithi100.stdout: 1 237856 2.75 MB/sec execute 341 sec latency 11.534 ms 2024-05-10T11:27:47.922 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.920+0000 7f4224171640 1 -- 172.21.15.100:0/4232516936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f421c102a40 msgr2=0x7f421c102e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:47.922 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.920+0000 7f4224171640 1 --2- 172.21.15.100:0/4232516936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f421c102a40 0x7f421c102e60 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f4218009900 tx=0x7f421802a410 comp rx=0 tx=0).stop 2024-05-10T11:27:47.922 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.920+0000 7f4224171640 1 -- 172.21.15.100:0/4232516936 shutdown_connections 2024-05-10T11:27:47.922 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.920+0000 7f4224171640 1 --2- 172.21.15.100:0/4232516936 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f421c1045f0 0x7f421c10ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:47.922 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.920+0000 7f4224171640 1 --2- 172.21.15.100:0/4232516936 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f421c103c30 0x7f421c1040b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:47.922 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.920+0000 7f4224171640 1 --2- 172.21.15.100:0/4232516936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f421c102a40 0x7f421c102e60 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:47.922 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.920+0000 7f4224171640 1 -- 172.21.15.100:0/4232516936 >> 172.21.15.100:0/4232516936 conn(0x7f421c0fdfd0 msgr2=0x7f421c1003f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:27:47.922 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.920+0000 7f4224171640 1 -- 172.21.15.100:0/4232516936 shutdown_connections 2024-05-10T11:27:47.922 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.920+0000 7f4224171640 1 -- 172.21.15.100:0/4232516936 wait complete. 2024-05-10T11:27:47.922 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.921+0000 7f4224171640 1 Processor -- start 2024-05-10T11:27:47.923 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.921+0000 7f4224171640 1 -- start start 2024-05-10T11:27:47.923 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.921+0000 7f4224171640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f421c102a40 0x7f421c198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:47.923 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.921+0000 7f4224171640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f421c103c30 0x7f421c199090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:47.923 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.921+0000 7f4224171640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f421c1045f0 0x7f421c1a00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:47.923 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.921+0000 7f4224171640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f421c10e420 con 0x7f421c102a40 2024-05-10T11:27:47.923 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.921+0000 7f4224171640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f421c10e2a0 con 0x7f421c1045f0 2024-05-10T11:27:47.923 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.921+0000 7f4224171640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f421c10e5a0 con 0x7f421c103c30 2024-05-10T11:27:47.923 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.921+0000 7f4221f0d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f421c102a40 0x7f421c198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:47.923 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.921+0000 7f4221f0d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f421c102a40 0x7f421c198b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48444/0 (socket says 172.21.15.100:48444) 2024-05-10T11:27:47.923 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.921+0000 7f4221f0d640 1 -- 172.21.15.100:0/166155646 learned_addr learned my addr 172.21.15.100:0/166155646 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:27:47.923 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.921+0000 7f422170c640 1 --2- 172.21.15.100:0/166155646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f421c103c30 0x7f421c199090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:47.923 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.921+0000 7f422270e640 1 --2- 172.21.15.100:0/166155646 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f421c1045f0 0x7f421c1a00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:47.924 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.922+0000 7f4221f0d640 1 -- 172.21.15.100:0/166155646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f421c103c30 msgr2=0x7f421c199090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:47.925 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.922+0000 7f4221f0d640 1 --2- 172.21.15.100:0/166155646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f421c103c30 0x7f421c199090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:47.925 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.922+0000 7f4221f0d640 1 -- 172.21.15.100:0/166155646 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f421c1045f0 msgr2=0x7f421c1a00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:47.925 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.922+0000 7f4221f0d640 1 --2- 172.21.15.100:0/166155646 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f421c1045f0 0x7f421c1a00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:47.925 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.922+0000 7f4221f0d640 1 -- 172.21.15.100:0/166155646 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f421c1a0720 con 0x7f421c102a40 2024-05-10T11:27:47.925 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.922+0000 7f4221f0d640 1 --2- 172.21.15.100:0/166155646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f421c102a40 0x7f421c198b50 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f4218005ab0 tx=0x7f42180042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:27:47.925 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.924+0000 7f4212ffd640 1 -- 172.21.15.100:0/166155646 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421803f030 con 0x7f421c102a40 2024-05-10T11:27:47.925 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.924+0000 7f4224171640 1 -- 172.21.15.100:0/166155646 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f421c1a09b0 con 0x7f421c102a40 2024-05-10T11:27:47.925 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.924+0000 7f4212ffd640 1 -- 172.21.15.100:0/166155646 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4218032e10 con 0x7f421c102a40 2024-05-10T11:27:47.925 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.924+0000 7f4212ffd640 1 -- 172.21.15.100:0/166155646 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4218035070 con 0x7f421c102a40 2024-05-10T11:27:47.925 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.924+0000 7f4224171640 1 -- 172.21.15.100:0/166155646 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f421c1a0ef0 con 0x7f421c102a40 2024-05-10T11:27:47.927 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.927+0000 7f4212ffd640 1 -- 172.21.15.100:0/166155646 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f4218042030 con 0x7f421c102a40 2024-05-10T11:27:47.927 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.927+0000 7f4226f66740 1 Processor -- start 2024-05-10T11:27:47.928 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.927+0000 7f4226f66740 1 -- start start 2024-05-10T11:27:47.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.927+0000 7f4226f66740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601218a3db0 0x5601218a41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:47.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.927+0000 7f4226f66740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601218a47a0 0x5601218a92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:47.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.927+0000 7f4226f66740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5601218a9910 0x5601218abda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:47.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.927+0000 7f4226f66740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5601218085d0 con 0x5601218a3db0 2024-05-10T11:27:47.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.927+0000 7f4226f66740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5601217c1890 con 0x5601218a47a0 2024-05-10T11:27:47.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.927+0000 7f4226f66740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x560121804590 con 0x5601218a9910 2024-05-10T11:27:47.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.928+0000 7f4221f0d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601218a47a0 0x5601218a92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:47.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.928+0000 7f422270e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601218a3db0 0x5601218a41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:47.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.928+0000 7f422270e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601218a3db0 0x5601218a41d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48454/0 (socket says 172.21.15.100:48454) 2024-05-10T11:27:47.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.928+0000 7f4221f0d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601218a47a0 0x5601218a92a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:55408/0 (socket says 172.21.15.100:55408) 2024-05-10T11:27:47.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.928+0000 7f422170c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5601218a9910 0x5601218abda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:47.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.928+0000 7f422270e640 1 -- 172.21.15.100:0/3313952763 learned_addr learned my addr 172.21.15.100:0/3313952763 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:27:47.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.928+0000 7f4212ffd640 1 --2- 172.21.15.100:0/166155646 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f42080b28a0 0x7f42080b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:47.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.928+0000 7f422270e640 1 -- 172.21.15.100:0/3313952763 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5601218a9910 msgr2=0x5601218abda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:47.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.928+0000 7f422270e640 1 --2- 172.21.15.100:0/3313952763 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5601218a9910 0x5601218abda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:47.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.928+0000 7f422270e640 1 -- 172.21.15.100:0/3313952763 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601218a47a0 msgr2=0x5601218a92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:47.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.928+0000 7f422270e640 1 --2- 172.21.15.100:0/3313952763 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601218a47a0 0x5601218a92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:47.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.928+0000 7f422270e640 1 -- 172.21.15.100:0/3313952763 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56012151a210 con 0x5601218a3db0 2024-05-10T11:27:47.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.928+0000 7f422270e640 1 --2- 172.21.15.100:0/3313952763 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601218a3db0 0x5601218a41d0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f4214009fd0 tx=0x7f4214013c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6b7de31a7dd3d31 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:27:47.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.928+0000 7f4212ffd640 1 -- 172.21.15.100:0/166155646 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f4218102970 con 0x7f421c102a40 2024-05-10T11:27:47.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.928+0000 7f422270e640 1 --2- 172.21.15.100:0/166155646 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f42080b28a0 0x7f42080b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:47.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.928+0000 7f4207fff640 1 -- 172.21.15.100:0/3313952763 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4214022070 con 0x5601218a3db0 2024-05-10T11:27:47.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.928+0000 7f4207fff640 1 -- 172.21.15.100:0/3313952763 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f421400ec10 con 0x5601218a3db0 2024-05-10T11:27:47.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.928+0000 7f4207fff640 1 -- 172.21.15.100:0/3313952763 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421401b690 con 0x5601218a3db0 2024-05-10T11:27:47.935 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.929+0000 7f4226f66740 1 -- 172.21.15.100:0/3313952763 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601218a3db0 msgr2=0x5601218a41d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:47.935 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.929+0000 7f422270e640 1 --2- 172.21.15.100:0/166155646 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f42080b28a0 0x7f42080b4d60 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f421401f000 tx=0x7f421401bdb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:27:47.935 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.929+0000 7f4226f66740 1 --2- 172.21.15.100:0/3313952763 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601218a3db0 0x5601218a41d0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f4214009fd0 tx=0x7f4214013c60 comp rx=0 tx=0).stop 2024-05-10T11:27:47.935 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.929+0000 7f4226f66740 1 -- 172.21.15.100:0/3313952763 shutdown_connections 2024-05-10T11:27:47.935 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.929+0000 7f4226f66740 1 --2- 172.21.15.100:0/3313952763 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5601218a9910 0x5601218abda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:47.935 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.929+0000 7f4226f66740 1 --2- 172.21.15.100:0/3313952763 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601218a47a0 0x5601218a92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:47.935 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.929+0000 7f4226f66740 1 --2- 172.21.15.100:0/3313952763 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601218a3db0 0x5601218a41d0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:47.935 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.929+0000 7f4226f66740 1 -- 172.21.15.100:0/3313952763 >> 172.21.15.100:0/3313952763 conn(0x56012189f420 msgr2=0x56012189f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:27:47.935 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.929+0000 7f4226f66740 1 -- 172.21.15.100:0/3313952763 shutdown_connections 2024-05-10T11:27:47.935 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.929+0000 7f4226f66740 1 -- 172.21.15.100:0/3313952763 wait complete. 2024-05-10T11:27:47.936 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.930+0000 7f4226f66740 1 -- start start 2024-05-10T11:27:47.936 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.930+0000 7f4226f66740 1 Processor -- start 2024-05-10T11:27:47.936 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.930+0000 7f4226f66740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5601218a3db0 0x5601218adf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:47.936 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.930+0000 7f4226f66740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601218a47a0 0x5601218ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:47.936 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.930+0000 7f4226f66740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601218a9910 0x5601218b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:47.936 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.930+0000 7f4226f66740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5601218ac8c0 con 0x5601218a47a0 2024-05-10T11:27:47.936 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.930+0000 7f4226f66740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5601217c5370 con 0x5601218a9910 2024-05-10T11:27:47.936 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.930+0000 7f4226f66740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5601218aca40 con 0x5601218a3db0 2024-05-10T11:27:47.936 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.930+0000 7f4226f66740 10 client.0 _ll_register_callbacks cb 0x560121519a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:27:47.936 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.930+0000 7f422170c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601218a47a0 0x5601218ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:47.936 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.930+0000 7f4221f0d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5601218a3db0 0x5601218adf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:47.936 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.930+0000 7f422170c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601218a47a0 0x5601218ae4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48462/0 (socket says 172.21.15.100:48462) 2024-05-10T11:27:47.936 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.930+0000 7f422170c640 1 -- 172.21.15.100:0/1438131932 learned_addr learned my addr 172.21.15.100:0/1438131932 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:27:47.936 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.930+0000 7f422270e640 1 --2- 172.21.15.100:0/1438131932 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601218a9910 0x5601218b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.931+0000 7f4221f0d640 1 -- 172.21.15.100:0/1438131932 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601218a9910 msgr2=0x5601218b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.931+0000 7f4221f0d640 1 --2- 172.21.15.100:0/1438131932 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601218a9910 0x5601218b54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.931+0000 7f4221f0d640 1 -- 172.21.15.100:0/1438131932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601218a47a0 msgr2=0x5601218ae4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.931+0000 7f4221f0d640 1 --2- 172.21.15.100:0/1438131932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601218a47a0 0x5601218ae4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.931+0000 7f4221f0d640 1 -- 172.21.15.100:0/1438131932 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601215e6620 con 0x5601218a3db0 2024-05-10T11:27:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.931+0000 7f422170c640 1 --2- 172.21.15.100:0/1438131932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601218a47a0 0x5601218ae4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:27:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.931+0000 7f4221f0d640 1 --2- 172.21.15.100:0/1438131932 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5601218a3db0 0x5601218adf70 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4218100160 tx=0x7f4218005c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca21bf31329ab6fd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:27:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.931+0000 7f42057fa640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:27:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.931+0000 7f42057fa640 1 -- 172.21.15.100:0/1438131932 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4218049070 con 0x5601218a3db0 2024-05-10T11:27:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.931+0000 7f4226f66740 1 -- 172.21.15.100:0/1438131932 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f421400ec10 con 0x5601218a3db0 2024-05-10T11:27:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.931+0000 7f42057fa640 1 -- 172.21.15.100:0/1438131932 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f42180c9f40 con 0x5601218a3db0 2024-05-10T11:27:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.931+0000 7f42057fa640 1 -- 172.21.15.100:0/1438131932 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f42180c9ab0 con 0x5601218a3db0 2024-05-10T11:27:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.931+0000 7f4226f66740 10 client.26734 fetch_fsmap learned FSMap version 31 2024-05-10T11:27:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.931+0000 7f4226f66740 1 -- 172.21.15.100:0/1438131932 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5601218ac980 con 0x5601218a3db0 2024-05-10T11:27:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.931+0000 7f42057fa640 1 -- 172.21.15.100:0/1438131932 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f421400ec10 con 0x5601218a3db0 2024-05-10T11:27:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.931+0000 7f42057fa640 1 -- 172.21.15.100:0/1438131932 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f42180d2600 con 0x5601218a3db0 2024-05-10T11:27:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.932+0000 7f4226f66740 10 client.26734 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:27:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.932+0000 7f4226f66740 10 client.26734 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:27:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.932+0000 7f4226f66740 20 client.26734 populate_metadata read hostname 'smithi100' 2024-05-10T11:27:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.932+0000 7f4226f66740 1 --2- 172.21.15.100:0/1438131932 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5601218b9be0 0x5601218bc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.932+0000 7f4226f66740 4 client.26734 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:27:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.932+0000 7f4226f66740 1 -- 172.21.15.100:0/1438131932 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5601218161e0 con 0x5601218b9be0 2024-05-10T11:27:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.932+0000 7f422170c640 1 --2- 172.21.15.100:0/1438131932 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5601218b9be0 0x5601218bc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.932+0000 7f422170c640 1 --2- 172.21.15.100:0/1438131932 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5601218b9be0 0x5601218bc050 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2630bbe2fa3c2317 server_cookie=e913e6c20ed9bb4b in_seq=0 out_seq=0 2024-05-10T11:27:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.932+0000 7f42057fa640 10 client.26734 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:27:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.933+0000 7f42057fa640 1 -- 172.21.15.100:0/1438131932 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5601218161e0 con 0x5601218b9be0 2024-05-10T11:27:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.933+0000 7f42057fa640 10 client.26734 handle_command_reply: tid=0 2024-05-10T11:27:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.933+0000 7f4226f66740 1 client.26734 shutdown 2024-05-10T11:27:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.933+0000 7f4226f66740 1 -- 172.21.15.100:0/1438131932 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5601218a3db0 msgr2=0x5601218adf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.933+0000 7f4226f66740 1 --2- 172.21.15.100:0/1438131932 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5601218a3db0 0x5601218adf70 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4218100160 tx=0x7f4218005c90 comp rx=0 tx=0).stop 2024-05-10T11:27:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.933+0000 7f4226f66740 1 -- 172.21.15.100:0/1438131932 shutdown_connections 2024-05-10T11:27:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.933+0000 7f4226f66740 1 --2- 172.21.15.100:0/1438131932 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5601218b9be0 0x5601218bc050 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.933+0000 7f4226f66740 1 --2- 172.21.15.100:0/1438131932 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601218a9910 0x5601218b54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.933+0000 7f4226f66740 1 --2- 172.21.15.100:0/1438131932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601218a47a0 0x5601218ae4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.933+0000 7f4226f66740 1 --2- 172.21.15.100:0/1438131932 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5601218a3db0 0x5601218adf70 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.933+0000 7f42057fa640 0 client.26734 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:27:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.933+0000 7f4226f66740 1 -- 172.21.15.100:0/1438131932 >> 172.21.15.100:0/1438131932 conn(0x56012189f420 msgr2=0x56012189e580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:27:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.933+0000 7f4226f66740 1 -- 172.21.15.100:0/1438131932 shutdown_connections 2024-05-10T11:27:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.933+0000 7f4226f66740 1 -- 172.21.15.100:0/1438131932 wait complete. 2024-05-10T11:27:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.933+0000 7f4226f66740 20 client.26734 trim_cache size 0 max 16384 2024-05-10T11:27:47.950 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.948+0000 7f4226f66740 1 Processor -- start 2024-05-10T11:27:47.950 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.948+0000 7f4226f66740 1 -- start start 2024-05-10T11:27:47.950 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.948+0000 7f4226f66740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601218a3db0 0x5601218adb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:47.950 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.949+0000 7f4226f66740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5601218a47a0 0x5601218ae0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:47.951 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.949+0000 7f4226f66740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601218a9910 0x5601218a2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:47.951 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.949+0000 7f4226f66740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f421c1ab9e0 con 0x5601218a9910 2024-05-10T11:27:47.951 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.949+0000 7f4226f66740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f421c1a92b0 con 0x5601218a3db0 2024-05-10T11:27:47.951 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.949+0000 7f4226f66740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f421c1a7100 con 0x5601218a47a0 2024-05-10T11:27:47.951 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.949+0000 7f422170c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5601218a47a0 0x5601218ae0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:47.951 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.949+0000 7f422270e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601218a9910 0x5601218a2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:47.951 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.949+0000 7f422170c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5601218a47a0 0x5601218ae0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:36398/0 (socket says 172.21.15.100:36398) 2024-05-10T11:27:47.951 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.949+0000 7f4221f0d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601218a3db0 0x5601218adb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:47.951 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.949+0000 7f422170c640 1 -- 172.21.15.100:0/701400324 learned_addr learned my addr 172.21.15.100:0/701400324 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:27:47.951 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.949+0000 7f422170c640 1 -- 172.21.15.100:0/701400324 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601218a3db0 msgr2=0x5601218adb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:47.951 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.949+0000 7f422170c640 1 --2- 172.21.15.100:0/701400324 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601218a3db0 0x5601218adb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:47.951 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.949+0000 7f422170c640 1 -- 172.21.15.100:0/701400324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601218a9910 msgr2=0x5601218a2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:47.951 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.949+0000 7f422170c640 1 --2- 172.21.15.100:0/701400324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601218a9910 0x5601218a2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:47.951 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.949+0000 7f422170c640 1 -- 172.21.15.100:0/701400324 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56012189f250 con 0x5601218a47a0 2024-05-10T11:27:47.952 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.949+0000 7f4221f0d640 1 --2- 172.21.15.100:0/701400324 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601218a3db0 0x5601218adb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:27:47.952 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.949+0000 7f422170c640 1 --2- 172.21.15.100:0/701400324 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5601218a47a0 0x5601218ae0c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f420c012680 tx=0x7f420c005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75c7a6c5b60f484b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:27:47.952 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.949+0000 7f42077fe640 1 -- 172.21.15.100:0/701400324 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420c024070 con 0x5601218a47a0 2024-05-10T11:27:47.952 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.949+0000 7f42077fe640 1 -- 172.21.15.100:0/701400324 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f420c0054b0 con 0x5601218a47a0 2024-05-10T11:27:47.952 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.949+0000 7f42077fe640 1 -- 172.21.15.100:0/701400324 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420c004ca0 con 0x5601218a47a0 2024-05-10T11:27:47.952 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.950+0000 7f4226f66740 1 -- 172.21.15.100:0/701400324 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5601218a47a0 msgr2=0x5601218ae0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:47.952 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.950+0000 7f4226f66740 1 --2- 172.21.15.100:0/701400324 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5601218a47a0 0x5601218ae0c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f420c012680 tx=0x7f420c005f90 comp rx=0 tx=0).stop 2024-05-10T11:27:47.952 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.950+0000 7f4226f66740 1 -- 172.21.15.100:0/701400324 shutdown_connections 2024-05-10T11:27:47.952 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.950+0000 7f4226f66740 1 --2- 172.21.15.100:0/701400324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601218a9910 0x5601218a2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:47.952 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.950+0000 7f4226f66740 1 --2- 172.21.15.100:0/701400324 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5601218a47a0 0x5601218ae0c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:47.952 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.950+0000 7f4226f66740 1 --2- 172.21.15.100:0/701400324 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601218a3db0 0x5601218adb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:47.952 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.950+0000 7f4226f66740 1 -- 172.21.15.100:0/701400324 >> 172.21.15.100:0/701400324 conn(0x56012189f420 msgr2=0x5601218a5980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:27:47.953 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.950+0000 7f4226f66740 1 -- 172.21.15.100:0/701400324 shutdown_connections 2024-05-10T11:27:47.953 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.950+0000 7f4226f66740 1 -- 172.21.15.100:0/701400324 wait complete. 2024-05-10T11:27:47.953 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.950+0000 7f4226f66740 1 -- start start 2024-05-10T11:27:47.953 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.951+0000 7f4226f66740 1 Processor -- start 2024-05-10T11:27:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.951+0000 7f4226f66740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601218a3db0 0x5601218b2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.951+0000 7f4226f66740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5601218a47a0 0x5601218b2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.951+0000 7f4226f66740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601218a9910 0x5601218b3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.951+0000 7f4226f66740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f421c1a7770 con 0x5601218a3db0 2024-05-10T11:27:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.951+0000 7f4226f66740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f421c1a8be0 con 0x5601218a9910 2024-05-10T11:27:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.951+0000 7f4226f66740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f421c1a7e40 con 0x5601218a47a0 2024-05-10T11:27:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.951+0000 7f4226f66740 10 client.0 _ll_register_callbacks cb 0x7f4218108240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:27:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.951+0000 7f4221f0d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601218a3db0 0x5601218b2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.951+0000 7f4221f0d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601218a3db0 0x5601218b2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48478/0 (socket says 172.21.15.100:48478) 2024-05-10T11:27:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.951+0000 7f4221f0d640 1 -- 172.21.15.100:0/483712515 learned_addr learned my addr 172.21.15.100:0/483712515 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:27:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.951+0000 7f422170c640 1 --2- 172.21.15.100:0/483712515 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5601218a47a0 0x5601218b2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.951+0000 7f422270e640 1 --2- 172.21.15.100:0/483712515 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601218a9910 0x5601218b3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.951+0000 7f4221f0d640 1 -- 172.21.15.100:0/483712515 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5601218a47a0 msgr2=0x5601218b2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.951+0000 7f4221f0d640 1 --2- 172.21.15.100:0/483712515 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5601218a47a0 0x5601218b2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.951+0000 7f4221f0d640 1 -- 172.21.15.100:0/483712515 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601218a9910 msgr2=0x5601218b3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.951+0000 7f4221f0d640 1 --2- 172.21.15.100:0/483712515 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601218a9910 0x5601218b3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.951+0000 7f4221f0d640 1 -- 172.21.15.100:0/483712515 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601218a5470 con 0x5601218a3db0 2024-05-10T11:27:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.952+0000 7f4221f0d640 1 --2- 172.21.15.100:0/483712515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601218a3db0 0x5601218b2a00 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f421802a450 tx=0x7f42180095d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfba3201736caee9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:27:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.952+0000 7f42057fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:27:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.952+0000 7f42057fa640 1 -- 172.21.15.100:0/483712515 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4218049040 con 0x5601218a3db0 2024-05-10T11:27:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.952+0000 7f42057fa640 1 -- 172.21.15.100:0/483712515 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f42180c62d0 con 0x5601218a3db0 2024-05-10T11:27:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.952+0000 7f4226f66740 1 -- 172.21.15.100:0/483712515 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f420c0054b0 con 0x5601218a3db0 2024-05-10T11:27:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.952+0000 7f42057fa640 1 -- 172.21.15.100:0/483712515 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f42180c9450 con 0x5601218a3db0 2024-05-10T11:27:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.952+0000 7f4226f66740 10 client.17277 fetch_fsmap learned FSMap version 31 2024-05-10T11:27:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.952+0000 7f4226f66740 1 -- 172.21.15.100:0/483712515 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560121924970 con 0x5601218a3db0 2024-05-10T11:27:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.952+0000 7f42057fa640 1 -- 172.21.15.100:0/483712515 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f420c0054b0 con 0x5601218a3db0 2024-05-10T11:27:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.952+0000 7f42057fa640 1 -- 172.21.15.100:0/483712515 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f42180c89b0 con 0x5601218a3db0 2024-05-10T11:27:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.952+0000 7f4226f66740 10 client.17277 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:27:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.952+0000 7f4226f66740 10 client.17277 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:27:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.952+0000 7f4226f66740 20 client.17277 populate_metadata read hostname 'smithi100' 2024-05-10T11:27:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.952+0000 7f4226f66740 1 --2- 172.21.15.100:0/483712515 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x560121924fe0 0x560121927400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.952+0000 7f4226f66740 4 client.17277 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:27:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.952+0000 7f4226f66740 1 -- 172.21.15.100:0/483712515 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560121810050 con 0x560121924fe0 2024-05-10T11:27:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.952+0000 7f422170c640 1 --2- 172.21.15.100:0/483712515 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x560121924fe0 0x560121927400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.953+0000 7f422170c640 1 --2- 172.21.15.100:0/483712515 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x560121924fe0 0x560121927400 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d43ebdcf7560d9e server_cookie=e9a68b5d4378a1af in_seq=0 out_seq=0 2024-05-10T11:27:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.953+0000 7f42057fa640 10 client.17277 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:27:47.970 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.969+0000 7f42057fa640 1 -- 172.21.15.100:0/483712515 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560121810050 con 0x560121924fe0 2024-05-10T11:27:47.970 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.969+0000 7f42057fa640 10 client.17277 handle_command_reply: tid=0 2024-05-10T11:27:47.970 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.969+0000 7f4226f66740 1 client.17277 shutdown 2024-05-10T11:27:47.970 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.969+0000 7f4226f66740 1 -- 172.21.15.100:0/483712515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601218a3db0 msgr2=0x5601218b2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:47.970 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.969+0000 7f4226f66740 1 --2- 172.21.15.100:0/483712515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601218a3db0 0x5601218b2a00 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f421802a450 tx=0x7f42180095d0 comp rx=0 tx=0).stop 2024-05-10T11:27:47.970 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.969+0000 7f4226f66740 1 -- 172.21.15.100:0/483712515 shutdown_connections 2024-05-10T11:27:47.970 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.969+0000 7f4226f66740 1 --2- 172.21.15.100:0/483712515 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x560121924fe0 0x560121927400 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:47.970 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.969+0000 7f4226f66740 1 --2- 172.21.15.100:0/483712515 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601218a9910 0x5601218b3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:47.970 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.969+0000 7f4226f66740 1 --2- 172.21.15.100:0/483712515 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5601218a47a0 0x5601218b2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:47.971 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.969+0000 7f4226f66740 1 --2- 172.21.15.100:0/483712515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601218a3db0 0x5601218b2a00 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:47.971 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.969+0000 7f4226f66740 1 -- 172.21.15.100:0/483712515 >> 172.21.15.100:0/483712515 conn(0x56012189f420 msgr2=0x5601218adb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:27:47.971 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.969+0000 7f42057fa640 0 client.17277 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:27:47.971 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.969+0000 7f4226f66740 1 -- 172.21.15.100:0/483712515 shutdown_connections 2024-05-10T11:27:47.971 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.969+0000 7f4226f66740 1 -- 172.21.15.100:0/483712515 wait complete. 2024-05-10T11:27:47.971 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.970+0000 7f4226f66740 20 client.17277 trim_cache size 0 max 16384 2024-05-10T11:27:47.971 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:27:47.971 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:27:47.971 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:27:47.971 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:27:47.972 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.970+0000 7f4224171640 1 -- 172.21.15.100:0/166155646 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f42080b28a0 msgr2=0x7f42080b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:47.972 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.970+0000 7f4224171640 1 --2- 172.21.15.100:0/166155646 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f42080b28a0 0x7f42080b4d60 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f421401f000 tx=0x7f421401bdb0 comp rx=0 tx=0).stop 2024-05-10T11:27:47.972 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.970+0000 7f4224171640 1 -- 172.21.15.100:0/166155646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f421c102a40 msgr2=0x7f421c198b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:47.972 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.970+0000 7f4224171640 1 --2- 172.21.15.100:0/166155646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f421c102a40 0x7f421c198b50 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f4218005ab0 tx=0x7f42180042f0 comp rx=0 tx=0).stop 2024-05-10T11:27:47.973 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.970+0000 7f4224171640 1 -- 172.21.15.100:0/166155646 shutdown_connections 2024-05-10T11:27:47.973 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.970+0000 7f4224171640 1 --2- 172.21.15.100:0/166155646 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f42080b28a0 0x7f42080b4d60 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:47.973 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.970+0000 7f4224171640 1 --2- 172.21.15.100:0/166155646 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f421c1045f0 0x7f421c1a00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:47.973 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.970+0000 7f4224171640 1 --2- 172.21.15.100:0/166155646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f421c103c30 0x7f421c199090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:47.973 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.970+0000 7f4224171640 1 --2- 172.21.15.100:0/166155646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f421c102a40 0x7f421c198b50 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:47.973 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.970+0000 7f4224171640 1 -- 172.21.15.100:0/166155646 >> 172.21.15.100:0/166155646 conn(0x7f421c0fdfd0 msgr2=0x7f421c106f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:27:47.973 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.970+0000 7f4224171640 1 -- 172.21.15.100:0/166155646 shutdown_connections 2024-05-10T11:27:47.973 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:27:47.970+0000 7f4224171640 1 -- 172.21.15.100:0/166155646 wait complete. 2024-05-10T11:27:47.978 INFO:tasks.workunit.client.0.smithi100.stdout: 1 238144 2.74 MB/sec execute 342 sec latency 13.257 ms 2024-05-10T11:27:48.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:27:47 smithi178 ceph-mon[33359]: pgmap v702: 129 pgs: 129 active+clean; 96 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 1.4 MiB/s wr, 135 op/s 2024-05-10T11:27:48.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:27:47 smithi188 ceph-mon[34075]: pgmap v702: 129 pgs: 129 active+clean; 96 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 1.4 MiB/s wr, 135 op/s 2024-05-10T11:27:48.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:27:47 smithi100 ceph-mon[30296]: pgmap v702: 129 pgs: 129 active+clean; 96 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 1.4 MiB/s wr, 135 op/s 2024-05-10T11:27:48.978 INFO:tasks.workunit.client.0.smithi100.stdout: 1 238566 2.74 MB/sec execute 343 sec latency 15.291 ms 2024-05-10T11:27:49.978 INFO:tasks.workunit.client.0.smithi100.stdout: 1 239121 2.74 MB/sec execute 344 sec latency 15.510 ms 2024-05-10T11:27:50.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:27:49 smithi178 ceph-mon[33359]: pgmap v703: 129 pgs: 129 active+clean; 98 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 1.4 MiB/s wr, 154 op/s 2024-05-10T11:27:50.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:27:49 smithi188 ceph-mon[34075]: pgmap v703: 129 pgs: 129 active+clean; 98 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 1.4 MiB/s wr, 154 op/s 2024-05-10T11:27:50.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:27:49 smithi100 ceph-mon[30296]: pgmap v703: 129 pgs: 129 active+clean; 98 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 1.4 MiB/s wr, 154 op/s 2024-05-10T11:27:50.982 INFO:tasks.workunit.client.0.smithi100.stdout: 1 239757 2.73 MB/sec execute 345 sec latency 15.364 ms 2024-05-10T11:27:51.978 INFO:tasks.workunit.client.0.smithi100.stdout: 1 240291 2.73 MB/sec execute 346 sec latency 14.276 ms 2024-05-10T11:27:52.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:27:51 smithi178 ceph-mon[33359]: pgmap v704: 129 pgs: 129 active+clean; 97 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 1.2 MiB/s wr, 132 op/s 2024-05-10T11:27:52.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:27:51 smithi188 ceph-mon[34075]: pgmap v704: 129 pgs: 129 active+clean; 97 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 1.2 MiB/s wr, 132 op/s 2024-05-10T11:27:52.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:27:51 smithi100 ceph-mon[30296]: pgmap v704: 129 pgs: 129 active+clean; 97 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 1.2 MiB/s wr, 132 op/s 2024-05-10T11:27:52.978 INFO:tasks.workunit.client.0.smithi100.stdout: 1 241005 2.74 MB/sec execute 347 sec latency 15.363 ms 2024-05-10T11:27:53.978 INFO:tasks.workunit.client.0.smithi100.stdout: 1 241466 2.75 MB/sec execute 348 sec latency 16.048 ms 2024-05-10T11:27:54.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:27:53 smithi178 ceph-mon[33359]: pgmap v705: 129 pgs: 129 active+clean; 99 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 1.4 MiB/s wr, 161 op/s 2024-05-10T11:27:54.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:27:53 smithi188 ceph-mon[34075]: pgmap v705: 129 pgs: 129 active+clean; 99 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 1.4 MiB/s wr, 161 op/s 2024-05-10T11:27:54.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:27:53 smithi100 ceph-mon[30296]: pgmap v705: 129 pgs: 129 active+clean; 99 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 1.4 MiB/s wr, 161 op/s 2024-05-10T11:27:54.978 INFO:tasks.workunit.client.0.smithi100.stdout: 1 241718 2.74 MB/sec execute 349 sec latency 16.336 ms 2024-05-10T11:27:55.978 INFO:tasks.workunit.client.0.smithi100.stdout: 1 242159 2.74 MB/sec execute 350 sec latency 17.533 ms 2024-05-10T11:27:56.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:27:55 smithi178 ceph-mon[33359]: pgmap v706: 129 pgs: 129 active+clean; 98 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 1.4 MiB/s wr, 160 op/s 2024-05-10T11:27:56.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:27:55 smithi188 ceph-mon[34075]: pgmap v706: 129 pgs: 129 active+clean; 98 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 1.4 MiB/s wr, 160 op/s 2024-05-10T11:27:56.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:27:55 smithi100 ceph-mon[30296]: pgmap v706: 129 pgs: 129 active+clean; 98 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 1.4 MiB/s wr, 160 op/s 2024-05-10T11:27:56.978 INFO:tasks.workunit.client.0.smithi100.stdout: 1 242736 2.74 MB/sec execute 351 sec latency 15.894 ms 2024-05-10T11:27:57.279 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T11:27:57.552 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:27:57.978 INFO:tasks.workunit.client.0.smithi100.stdout: 1 243384 2.73 MB/sec execute 352 sec latency 21.591 ms 2024-05-10T11:27:58.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:27:57 smithi178 ceph-mon[33359]: pgmap v707: 129 pgs: 129 active+clean; 99 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 1.3 MiB/s wr, 138 op/s 2024-05-10T11:27:58.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:27:57 smithi188 ceph-mon[34075]: pgmap v707: 129 pgs: 129 active+clean; 99 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 1.3 MiB/s wr, 138 op/s 2024-05-10T11:27:58.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:27:57 smithi100 ceph-mon[30296]: pgmap v707: 129 pgs: 129 active+clean; 99 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 1.3 MiB/s wr, 138 op/s 2024-05-10T11:27:58.978 INFO:tasks.workunit.client.0.smithi100.stdout: 1 243860 2.73 MB/sec execute 353 sec latency 15.545 ms 2024-05-10T11:27:59.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.366+0000 7fad827fd700 1 -- 172.21.15.100:0/386353866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad7c109520 msgr2=0x7fad7c109900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:59.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.366+0000 7fad827fd700 1 --2- 172.21.15.100:0/386353866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad7c109520 0x7fad7c109900 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fad6c009990 tx=0x7fad6c009ca0 comp rx=0 tx=0).stop 2024-05-10T11:27:59.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.367+0000 7fad827fd700 1 -- 172.21.15.100:0/386353866 shutdown_connections 2024-05-10T11:27:59.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.367+0000 7fad827fd700 1 --2- 172.21.15.100:0/386353866 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad7c102450 0x7fad7c110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:59.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.367+0000 7fad827fd700 1 --2- 172.21.15.100:0/386353866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad7c109520 0x7fad7c109900 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:59.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.367+0000 7fad827fd700 1 --2- 172.21.15.100:0/386353866 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad7c101a90 0x7fad7c101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:59.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.367+0000 7fad827fd700 1 -- 172.21.15.100:0/386353866 >> 172.21.15.100:0/386353866 conn(0x7fad7c078a10 msgr2=0x7fad7c0ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:27:59.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.367+0000 7fad827fd700 1 -- 172.21.15.100:0/386353866 shutdown_connections 2024-05-10T11:27:59.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.367+0000 7fad827fd700 1 -- 172.21.15.100:0/386353866 wait complete. 2024-05-10T11:27:59.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.367+0000 7fad827fd700 1 Processor -- start 2024-05-10T11:27:59.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.367+0000 7fad827fd700 1 -- start start 2024-05-10T11:27:59.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.368+0000 7fad827fd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad7c101a90 0x7fad7c1a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:59.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.368+0000 7fad827fd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad7c102450 0x7fad7c1a1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:59.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.368+0000 7fad827fd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad7c109520 0x7fad7c19bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:59.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.368+0000 7fad827fd700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fad7c113530 con 0x7fad7c109520 2024-05-10T11:27:59.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.368+0000 7fad827fd700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fad7c1133b0 con 0x7fad7c101a90 2024-05-10T11:27:59.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.368+0000 7fad827fd700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fad7c1136b0 con 0x7fad7c102450 2024-05-10T11:27:59.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.368+0000 7fad80d9a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad7c109520 0x7fad7c19bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:59.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.368+0000 7fad73fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad7c102450 0x7fad7c1a1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:59.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.368+0000 7fad80d9a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad7c109520 0x7fad7c19bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60346/0 (socket says 172.21.15.100:60346) 2024-05-10T11:27:59.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.368+0000 7fad80d9a700 1 -- 172.21.15.100:0/1384313859 learned_addr learned my addr 172.21.15.100:0/1384313859 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:27:59.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.368+0000 7fad7bfff700 1 --2- 172.21.15.100:0/1384313859 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad7c101a90 0x7fad7c1a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:59.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.368+0000 7fad73fff700 1 -- 172.21.15.100:0/1384313859 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad7c101a90 msgr2=0x7fad7c1a1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:59.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.368+0000 7fad73fff700 1 --2- 172.21.15.100:0/1384313859 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad7c101a90 0x7fad7c1a1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:59.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.368+0000 7fad73fff700 1 -- 172.21.15.100:0/1384313859 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad7c109520 msgr2=0x7fad7c19bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:59.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.368+0000 7fad73fff700 1 --2- 172.21.15.100:0/1384313859 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad7c109520 0x7fad7c19bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:59.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.368+0000 7fad73fff700 1 -- 172.21.15.100:0/1384313859 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad7c19c220 con 0x7fad7c102450 2024-05-10T11:27:59.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.368+0000 7fad73fff700 1 --2- 172.21.15.100:0/1384313859 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad7c102450 0x7fad7c1a1e40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fad6400e9f0 tx=0x7fad6400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:27:59.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.369+0000 7fad79ffb700 1 -- 172.21.15.100:0/1384313859 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6400cbd0 con 0x7fad7c102450 2024-05-10T11:27:59.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.369+0000 7fad827fd700 1 -- 172.21.15.100:0/1384313859 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad7c19c510 con 0x7fad7c102450 2024-05-10T11:27:59.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.369+0000 7fad79ffb700 1 -- 172.21.15.100:0/1384313859 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fad6400cd70 con 0x7fad7c102450 2024-05-10T11:27:59.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.369+0000 7fad79ffb700 1 -- 172.21.15.100:0/1384313859 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad64018680 con 0x7fad7c102450 2024-05-10T11:27:59.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.369+0000 7fad827fd700 1 -- 172.21.15.100:0/1384313859 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad7c1a87d0 con 0x7fad7c102450 2024-05-10T11:27:59.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.372+0000 7fad79ffb700 1 -- 172.21.15.100:0/1384313859 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fad64010430 con 0x7fad7c102450 2024-05-10T11:27:59.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.373+0000 7fad8d090b80 1 Processor -- start 2024-05-10T11:27:59.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.373+0000 7fad79ffb700 1 --2- 172.21.15.100:0/1384313859 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fad680b2880 0x7fad680b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:59.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.373+0000 7fad8d090b80 1 -- start start 2024-05-10T11:27:59.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.373+0000 7fad79ffb700 1 -- 172.21.15.100:0/1384313859 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fad640e27a0 con 0x7fad7c102450 2024-05-10T11:27:59.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.373+0000 7fad7bfff700 1 --2- 172.21.15.100:0/1384313859 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fad680b2880 0x7fad680b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:59.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.373+0000 7fad8d090b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555dfb7b34d0 0x555dfb7b38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:59.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.373+0000 7fad8d090b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555dfb7b3df0 0x555dfb7b76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:59.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.373+0000 7fad8d090b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555dfb7b7c80 0x555dfb7b8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:59.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.373+0000 7fad8d090b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x555dfb705860 con 0x555dfb7b3df0 2024-05-10T11:27:59.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.373+0000 7fad8d090b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x555dfb6a47c0 con 0x555dfb7b7c80 2024-05-10T11:27:59.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.373+0000 7fad8d090b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x555dfb701820 con 0x555dfb7b34d0 2024-05-10T11:27:59.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.374+0000 7fad73fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555dfb7b3df0 0x555dfb7b76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:59.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.374+0000 7fad73fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555dfb7b3df0 0x555dfb7b76f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60354/0 (socket says 172.21.15.100:60354) 2024-05-10T11:27:59.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.374+0000 7fad73fff700 1 -- 172.21.15.100:0/3558350461 learned_addr learned my addr 172.21.15.100:0/3558350461 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:27:59.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.374+0000 7fad80d9a700 1 --2- 172.21.15.100:0/3558350461 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555dfb7b7c80 0x555dfb7b8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:59.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.374+0000 7fad7bfff700 1 --2- 172.21.15.100:0/3558350461 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555dfb7b34d0 0x555dfb7b38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:59.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.374+0000 7fad73fff700 1 -- 172.21.15.100:0/3558350461 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555dfb7b34d0 msgr2=0x555dfb7b38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:59.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.374+0000 7fad73fff700 1 --2- 172.21.15.100:0/3558350461 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555dfb7b34d0 0x555dfb7b38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:59.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.374+0000 7fad7bfff700 1 --2- 172.21.15.100:0/1384313859 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fad680b2880 0x7fad680b4d40 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fad6c00b2e0 tx=0x7fad6c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:27:59.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.374+0000 7fad73fff700 1 -- 172.21.15.100:0/3558350461 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555dfb7b7c80 msgr2=0x555dfb7b8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:59.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.374+0000 7fad73fff700 1 --2- 172.21.15.100:0/3558350461 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555dfb7b7c80 0x555dfb7b8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:59.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.374+0000 7fad73fff700 1 -- 172.21.15.100:0/3558350461 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555dfb6528d0 con 0x555dfb7b3df0 2024-05-10T11:27:59.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.374+0000 7fad80d9a700 1 --2- 172.21.15.100:0/3558350461 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555dfb7b7c80 0x555dfb7b8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:27:59.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.374+0000 7fad73fff700 1 --2- 172.21.15.100:0/3558350461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555dfb7b3df0 0x555dfb7b76f0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fad6400e9c0 tx=0x7fad640a59f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8beebbad1d0e2216 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:27:59.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.375+0000 7fad727fc700 1 -- 172.21.15.100:0/3558350461 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad640a9510 con 0x555dfb7b3df0 2024-05-10T11:27:59.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.375+0000 7fad727fc700 1 -- 172.21.15.100:0/3558350461 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fad640aa100 con 0x555dfb7b3df0 2024-05-10T11:27:59.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.375+0000 7fad727fc700 1 -- 172.21.15.100:0/3558350461 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad640b23d0 con 0x555dfb7b3df0 2024-05-10T11:27:59.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.375+0000 7fad8d090b80 1 -- 172.21.15.100:0/3558350461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555dfb7b3df0 msgr2=0x555dfb7b76f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:59.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.375+0000 7fad8d090b80 1 --2- 172.21.15.100:0/3558350461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555dfb7b3df0 0x555dfb7b76f0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fad6400e9c0 tx=0x7fad640a59f0 comp rx=0 tx=0).stop 2024-05-10T11:27:59.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.375+0000 7fad8d090b80 1 -- 172.21.15.100:0/3558350461 shutdown_connections 2024-05-10T11:27:59.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.375+0000 7fad8d090b80 1 --2- 172.21.15.100:0/3558350461 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555dfb7b7c80 0x555dfb7b8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:59.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.375+0000 7fad8d090b80 1 --2- 172.21.15.100:0/3558350461 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555dfb7b34d0 0x555dfb7b38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:59.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.375+0000 7fad8d090b80 1 --2- 172.21.15.100:0/3558350461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555dfb7b3df0 0x555dfb7b76f0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:59.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.375+0000 7fad8d090b80 1 -- 172.21.15.100:0/3558350461 >> 172.21.15.100:0/3558350461 conn(0x555dfb6df100 msgr2=0x555dfb7adc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:27:59.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.376+0000 7fad8d090b80 1 -- 172.21.15.100:0/3558350461 shutdown_connections 2024-05-10T11:27:59.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.376+0000 7fad8d090b80 1 -- 172.21.15.100:0/3558350461 wait complete. 2024-05-10T11:27:59.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.376+0000 7fad8d090b80 1 -- start start 2024-05-10T11:27:59.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.376+0000 7fad8d090b80 1 Processor -- start 2024-05-10T11:27:59.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.376+0000 7fad8d090b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555dfb7b34d0 0x555dfb7bf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:59.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.376+0000 7fad8d090b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555dfb7b3df0 0x555dfb7bc490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:59.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.376+0000 7fad8d090b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555dfb7b7c80 0x555dfb7bc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:59.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.376+0000 7fad8d090b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x555dfb7bfe30 con 0x555dfb7b3df0 2024-05-10T11:27:59.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.376+0000 7fad8d090b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x555dfb66bc30 con 0x555dfb7b34d0 2024-05-10T11:27:59.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.376+0000 7fad8d090b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x555dfb7bffb0 con 0x555dfb7b7c80 2024-05-10T11:27:59.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.376+0000 7fad8d090b80 10 client.0 _ll_register_callbacks cb 0x555dfb69d920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:27:59.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.377+0000 7fad73fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555dfb7b3df0 0x555dfb7bc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:59.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.377+0000 7fad80d9a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555dfb7b7c80 0x555dfb7bc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:59.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.377+0000 7fad73fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555dfb7b3df0 0x555dfb7bc490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60364/0 (socket says 172.21.15.100:60364) 2024-05-10T11:27:59.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.377+0000 7fad80d9a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555dfb7b7c80 0x555dfb7bc9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:50884/0 (socket says 172.21.15.100:50884) 2024-05-10T11:27:59.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.377+0000 7fad73fff700 1 -- 172.21.15.100:0/499655427 learned_addr learned my addr 172.21.15.100:0/499655427 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:27:59.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.377+0000 7fad7bfff700 1 --2- 172.21.15.100:0/499655427 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555dfb7b34d0 0x555dfb7bf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:59.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.377+0000 7fad73fff700 1 -- 172.21.15.100:0/499655427 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555dfb7b7c80 msgr2=0x555dfb7bc9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:59.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.377+0000 7fad73fff700 1 --2- 172.21.15.100:0/499655427 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555dfb7b7c80 0x555dfb7bc9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:59.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.377+0000 7fad73fff700 1 -- 172.21.15.100:0/499655427 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555dfb7b34d0 msgr2=0x555dfb7bf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:59.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.377+0000 7fad73fff700 1 --2- 172.21.15.100:0/499655427 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555dfb7b34d0 0x555dfb7bf080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:59.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.377+0000 7fad73fff700 1 -- 172.21.15.100:0/499655427 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555dfb6a1690 con 0x555dfb7b3df0 2024-05-10T11:27:59.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.377+0000 7fad73fff700 1 --2- 172.21.15.100:0/499655427 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555dfb7b3df0 0x555dfb7bc490 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fad64000c00 tx=0x7fad640b2600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=711da64d5ead772d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:27:59.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.377+0000 7fad5b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:27:59.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.377+0000 7fad5b7fe700 1 -- 172.21.15.100:0/499655427 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad640c2070 con 0x555dfb7b3df0 2024-05-10T11:27:59.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.378+0000 7fad5b7fe700 1 -- 172.21.15.100:0/499655427 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fad64010930 con 0x555dfb7b3df0 2024-05-10T11:27:59.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.378+0000 7fad8d090b80 1 -- 172.21.15.100:0/499655427 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad640aa100 con 0x555dfb7b3df0 2024-05-10T11:27:59.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.378+0000 7fad5b7fe700 1 -- 172.21.15.100:0/499655427 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fad640a9510 con 0x555dfb7b3df0 2024-05-10T11:27:59.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.378+0000 7fad8d090b80 10 client.17289 fetch_fsmap learned FSMap version 31 2024-05-10T11:27:59.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.378+0000 7fad8d090b80 1 -- 172.21.15.100:0/499655427 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x555dfb7ca6d0 con 0x555dfb7b3df0 2024-05-10T11:27:59.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.378+0000 7fad5b7fe700 1 -- 172.21.15.100:0/499655427 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fad640aa100 con 0x555dfb7b3df0 2024-05-10T11:27:59.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.378+0000 7fad5b7fe700 1 -- 172.21.15.100:0/499655427 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fad640c5050 con 0x555dfb7b3df0 2024-05-10T11:27:59.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.378+0000 7fad8d090b80 10 client.17289 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:27:59.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.378+0000 7fad8d090b80 10 client.17289 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:27:59.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.378+0000 7fad8d090b80 20 client.17289 populate_metadata read hostname 'smithi100' 2024-05-10T11:27:59.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.378+0000 7fad8d090b80 1 --2- 172.21.15.100:0/499655427 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x555dfb7ccba0 0x555dfb7cef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:59.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.378+0000 7fad8d090b80 4 client.17289 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:27:59.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.378+0000 7fad8d090b80 1 -- 172.21.15.100:0/499655427 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555dfb77f650 con 0x555dfb7ccba0 2024-05-10T11:27:59.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.378+0000 7fad7bfff700 1 --2- 172.21.15.100:0/499655427 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x555dfb7ccba0 0x555dfb7cef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:59.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.379+0000 7fad5b7fe700 10 client.17289 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:27:59.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.379+0000 7fad7bfff700 1 --2- 172.21.15.100:0/499655427 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x555dfb7ccba0 0x555dfb7cef90 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98d34fc54ee987e5 server_cookie=c88bec46925dbe14 in_seq=0 out_seq=0 2024-05-10T11:27:59.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.380+0000 7fad5b7fe700 1 -- 172.21.15.100:0/499655427 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555dfb77f650 con 0x555dfb7ccba0 2024-05-10T11:27:59.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.380+0000 7fad5b7fe700 10 client.17289 handle_command_reply: tid=0 2024-05-10T11:27:59.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.380+0000 7fad8d090b80 1 client.17289 shutdown 2024-05-10T11:27:59.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.380+0000 7fad8d090b80 1 -- 172.21.15.100:0/499655427 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555dfb7b3df0 msgr2=0x555dfb7bc490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:59.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.380+0000 7fad8d090b80 1 --2- 172.21.15.100:0/499655427 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555dfb7b3df0 0x555dfb7bc490 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fad64000c00 tx=0x7fad640b2600 comp rx=0 tx=0).stop 2024-05-10T11:27:59.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.380+0000 7fad8d090b80 1 -- 172.21.15.100:0/499655427 shutdown_connections 2024-05-10T11:27:59.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.380+0000 7fad8d090b80 1 --2- 172.21.15.100:0/499655427 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x555dfb7ccba0 0x555dfb7cef90 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:59.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.380+0000 7fad5b7fe700 0 client.17289 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:27:59.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.380+0000 7fad8d090b80 1 --2- 172.21.15.100:0/499655427 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555dfb7b7c80 0x555dfb7bc9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:59.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.380+0000 7fad8d090b80 1 --2- 172.21.15.100:0/499655427 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555dfb7b34d0 0x555dfb7bf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:59.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.380+0000 7fad8d090b80 1 --2- 172.21.15.100:0/499655427 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555dfb7b3df0 0x555dfb7bc490 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:59.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.380+0000 7fad8d090b80 1 -- 172.21.15.100:0/499655427 >> 172.21.15.100:0/499655427 conn(0x555dfb6df100 msgr2=0x555dfb7a8720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:27:59.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.380+0000 7fad8d090b80 1 -- 172.21.15.100:0/499655427 shutdown_connections 2024-05-10T11:27:59.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.380+0000 7fad8d090b80 1 -- 172.21.15.100:0/499655427 wait complete. 2024-05-10T11:27:59.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.380+0000 7fad8d090b80 20 client.17289 trim_cache size 0 max 16384 2024-05-10T11:27:59.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.407+0000 7fad8d090b80 1 Processor -- start 2024-05-10T11:27:59.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.407+0000 7fad8d090b80 1 -- start start 2024-05-10T11:27:59.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.408+0000 7fad8d090b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555dfb7b34d0 0x555dfb7a8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:59.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.408+0000 7fad8d090b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555dfb7b3df0 0x555dfb7adbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:59.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.408+0000 7fad8d090b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555dfb7b7c80 0x555dfb7d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:59.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.408+0000 7fad8d090b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fad7c1b3c90 con 0x555dfb7b3df0 2024-05-10T11:27:59.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.408+0000 7fad8d090b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fad7c1b1520 con 0x555dfb7b7c80 2024-05-10T11:27:59.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.408+0000 7fad8d090b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fad7c1af370 con 0x555dfb7b34d0 2024-05-10T11:27:59.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.408+0000 7fad73fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555dfb7b3df0 0x555dfb7adbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:59.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.408+0000 7fad7bfff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555dfb7b34d0 0x555dfb7a8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:59.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.408+0000 7fad73fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555dfb7b3df0 0x555dfb7adbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60372/0 (socket says 172.21.15.100:60372) 2024-05-10T11:27:59.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.408+0000 7fad73fff700 1 -- 172.21.15.100:0/1963989186 learned_addr learned my addr 172.21.15.100:0/1963989186 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:27:59.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.408+0000 7fad80d9a700 1 --2- 172.21.15.100:0/1963989186 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555dfb7b7c80 0x555dfb7d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:59.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.408+0000 7fad7bfff700 1 -- 172.21.15.100:0/1963989186 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555dfb7b7c80 msgr2=0x555dfb7d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:59.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.408+0000 7fad7bfff700 1 --2- 172.21.15.100:0/1963989186 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555dfb7b7c80 0x555dfb7d1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:59.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.408+0000 7fad7bfff700 1 -- 172.21.15.100:0/1963989186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555dfb7b3df0 msgr2=0x555dfb7adbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:59.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.408+0000 7fad7bfff700 1 --2- 172.21.15.100:0/1963989186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555dfb7b3df0 0x555dfb7adbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:59.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.408+0000 7fad7bfff700 1 -- 172.21.15.100:0/1963989186 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555dfb7baa20 con 0x555dfb7b34d0 2024-05-10T11:27:59.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.409+0000 7fad7bfff700 1 --2- 172.21.15.100:0/1963989186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555dfb7b34d0 0x555dfb7a8870 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fad7c19cc00 tx=0x7fad6c021000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9333e7961c8515e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:27:59.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.409+0000 7fad71ffb700 1 -- 172.21.15.100:0/1963989186 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6c010850 con 0x555dfb7b34d0 2024-05-10T11:27:59.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.409+0000 7fad71ffb700 1 -- 172.21.15.100:0/1963989186 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fad6c010e80 con 0x555dfb7b34d0 2024-05-10T11:27:59.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.409+0000 7fad71ffb700 1 -- 172.21.15.100:0/1963989186 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6c004a10 con 0x555dfb7b34d0 2024-05-10T11:27:59.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.410+0000 7fad8d090b80 1 -- 172.21.15.100:0/1963989186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555dfb7b34d0 msgr2=0x555dfb7a8870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:59.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.410+0000 7fad8d090b80 1 --2- 172.21.15.100:0/1963989186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555dfb7b34d0 0x555dfb7a8870 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fad7c19cc00 tx=0x7fad6c021000 comp rx=0 tx=0).stop 2024-05-10T11:27:59.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.410+0000 7fad8d090b80 1 -- 172.21.15.100:0/1963989186 shutdown_connections 2024-05-10T11:27:59.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.410+0000 7fad8d090b80 1 --2- 172.21.15.100:0/1963989186 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555dfb7b7c80 0x555dfb7d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:59.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.410+0000 7fad8d090b80 1 --2- 172.21.15.100:0/1963989186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555dfb7b34d0 0x555dfb7a8870 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:59.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.410+0000 7fad8d090b80 1 --2- 172.21.15.100:0/1963989186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555dfb7b3df0 0x555dfb7adbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:59.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.410+0000 7fad8d090b80 1 -- 172.21.15.100:0/1963989186 >> 172.21.15.100:0/1963989186 conn(0x555dfb6df100 msgr2=0x555dfb7b2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:27:59.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.410+0000 7fad8d090b80 1 -- 172.21.15.100:0/1963989186 shutdown_connections 2024-05-10T11:27:59.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.410+0000 7fad8d090b80 1 -- 172.21.15.100:0/1963989186 wait complete. 2024-05-10T11:27:59.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.410+0000 7fad8d090b80 1 -- start start 2024-05-10T11:27:59.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.410+0000 7fad8d090b80 1 Processor -- start 2024-05-10T11:27:59.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.411+0000 7fad8d090b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555dfb7b34d0 0x555dfb7cfef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:59.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.411+0000 7fad8d090b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555dfb7b3df0 0x555dfb7d0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:59.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.411+0000 7fad8d090b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555dfb7b7c80 0x555dfb8360c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:59.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.411+0000 7fad8d090b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fad7c1af9e0 con 0x555dfb7b7c80 2024-05-10T11:27:59.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.411+0000 7fad8d090b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fad7c1b0e50 con 0x555dfb7b3df0 2024-05-10T11:27:59.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.411+0000 7fad8d090b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fad7c1b00b0 con 0x555dfb7b34d0 2024-05-10T11:27:59.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.411+0000 7fad8d090b80 10 client.0 _ll_register_callbacks cb 0x555dfb69d920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:27:59.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.411+0000 7fad7bfff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555dfb7b34d0 0x555dfb7cfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:59.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.411+0000 7fad7bfff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555dfb7b34d0 0x555dfb7cfef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:50898/0 (socket says 172.21.15.100:50898) 2024-05-10T11:27:59.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.411+0000 7fad80d9a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555dfb7b7c80 0x555dfb8360c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:59.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.411+0000 7fad7bfff700 1 -- 172.21.15.100:0/3029416050 learned_addr learned my addr 172.21.15.100:0/3029416050 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:27:59.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.411+0000 7fad73fff700 1 --2- 172.21.15.100:0/3029416050 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555dfb7b3df0 0x555dfb7d0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:59.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.411+0000 7fad7bfff700 1 -- 172.21.15.100:0/3029416050 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555dfb7b3df0 msgr2=0x555dfb7d0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:59.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.411+0000 7fad7bfff700 1 --2- 172.21.15.100:0/3029416050 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555dfb7b3df0 0x555dfb7d0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:59.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.411+0000 7fad7bfff700 1 -- 172.21.15.100:0/3029416050 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555dfb7b7c80 msgr2=0x555dfb8360c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:27:59.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.411+0000 7fad7bfff700 1 --2- 172.21.15.100:0/3029416050 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555dfb7b7c80 0x555dfb8360c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:59.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.411+0000 7fad7bfff700 1 -- 172.21.15.100:0/3029416050 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555dfb836600 con 0x555dfb7b34d0 2024-05-10T11:27:59.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.411+0000 7fad7bfff700 1 --2- 172.21.15.100:0/3029416050 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555dfb7b34d0 0x555dfb7cfef0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fad6c0107d0 tx=0x7fad6c010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b99e39e99f634de server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:27:59.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.411+0000 7fad5b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:27:59.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.412+0000 7fad5b7fe700 1 -- 172.21.15.100:0/3029416050 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6c010bb0 con 0x555dfb7b34d0 2024-05-10T11:27:59.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.412+0000 7fad8d090b80 1 -- 172.21.15.100:0/3029416050 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad6c010e80 con 0x555dfb7b34d0 2024-05-10T11:27:59.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.412+0000 7fad5b7fe700 1 -- 172.21.15.100:0/3029416050 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fad6c005050 con 0x555dfb7b34d0 2024-05-10T11:27:59.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.412+0000 7fad5b7fe700 1 -- 172.21.15.100:0/3029416050 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fad6c049050 con 0x555dfb7b34d0 2024-05-10T11:27:59.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.412+0000 7fad8d090b80 10 client.26755 fetch_fsmap learned FSMap version 31 2024-05-10T11:27:59.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.412+0000 7fad8d090b80 1 -- 172.21.15.100:0/3029416050 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x555dfb837750 con 0x555dfb7b34d0 2024-05-10T11:27:59.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.412+0000 7fad5b7fe700 1 -- 172.21.15.100:0/3029416050 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fad6c010e80 con 0x555dfb7b34d0 2024-05-10T11:27:59.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.412+0000 7fad5b7fe700 1 -- 172.21.15.100:0/3029416050 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fad6c022070 con 0x555dfb7b34d0 2024-05-10T11:27:59.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.412+0000 7fad8d090b80 10 client.26755 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:27:59.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.412+0000 7fad8d090b80 10 client.26755 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:27:59.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.412+0000 7fad8d090b80 20 client.26755 populate_metadata read hostname 'smithi100' 2024-05-10T11:27:59.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.412+0000 7fad8d090b80 1 --2- 172.21.15.100:0/3029416050 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x555dfb837d50 0x555dfb83a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:27:59.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.412+0000 7fad8d090b80 4 client.26755 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T11:27:59.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.412+0000 7fad8d090b80 1 -- 172.21.15.100:0/3029416050 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555dfb713240 con 0x555dfb837d50 2024-05-10T11:27:59.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.412+0000 7fad73fff700 1 --2- 172.21.15.100:0/3029416050 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x555dfb837d50 0x555dfb83a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:27:59.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.413+0000 7fad5b7fe700 10 client.26755 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:27:59.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.413+0000 7fad73fff700 1 --2- 172.21.15.100:0/3029416050 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x555dfb837d50 0x555dfb83a140 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a9b9118ac39d44d server_cookie=52df3e12b14cc473 in_seq=0 out_seq=0 2024-05-10T11:27:59.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.414+0000 7fad5b7fe700 1 -- 172.21.15.100:0/3029416050 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555dfb713240 con 0x555dfb837d50 2024-05-10T11:27:59.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.414+0000 7fad5b7fe700 10 client.26755 handle_command_reply: tid=0 2024-05-10T11:27:59.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.414+0000 7fad8d090b80 1 client.26755 shutdown 2024-05-10T11:27:59.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.414+0000 7fad8d090b80 1 -- 172.21.15.100:0/3029416050 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555dfb7b34d0 msgr2=0x555dfb7cfef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:59.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.414+0000 7fad8d090b80 1 --2- 172.21.15.100:0/3029416050 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555dfb7b34d0 0x555dfb7cfef0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fad6c0107d0 tx=0x7fad6c010800 comp rx=0 tx=0).stop 2024-05-10T11:27:59.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.414+0000 7fad8d090b80 1 -- 172.21.15.100:0/3029416050 shutdown_connections 2024-05-10T11:27:59.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.414+0000 7fad8d090b80 1 --2- 172.21.15.100:0/3029416050 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x555dfb837d50 0x555dfb83a140 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:59.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.414+0000 7fad8d090b80 1 --2- 172.21.15.100:0/3029416050 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555dfb7b7c80 0x555dfb8360c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:59.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.414+0000 7fad8d090b80 1 --2- 172.21.15.100:0/3029416050 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555dfb7b34d0 0x555dfb7cfef0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:59.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.414+0000 7fad8d090b80 1 --2- 172.21.15.100:0/3029416050 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555dfb7b3df0 0x555dfb7d0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:59.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.414+0000 7fad8d090b80 1 -- 172.21.15.100:0/3029416050 >> 172.21.15.100:0/3029416050 conn(0x555dfb6df100 msgr2=0x555dfb7ada70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:27:59.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.414+0000 7fad5b7fe700 0 client.26755 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:27:59.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.414+0000 7fad8d090b80 1 -- 172.21.15.100:0/3029416050 shutdown_connections 2024-05-10T11:27:59.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.414+0000 7fad8d090b80 1 -- 172.21.15.100:0/3029416050 wait complete. 2024-05-10T11:27:59.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.414+0000 7fad8d090b80 20 client.26755 trim_cache size 0 max 16384 2024-05-10T11:27:59.420 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:27:59.420 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-05-10T11:27:59.420 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-05-10T11:27:59.420 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:27:59.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.415+0000 7fad827fd700 1 -- 172.21.15.100:0/1384313859 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fad680b2880 msgr2=0x7fad680b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:59.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.415+0000 7fad827fd700 1 --2- 172.21.15.100:0/1384313859 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fad680b2880 0x7fad680b4d40 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fad6c00b2e0 tx=0x7fad6c01a040 comp rx=0 tx=0).stop 2024-05-10T11:27:59.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.415+0000 7fad827fd700 1 -- 172.21.15.100:0/1384313859 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad7c102450 msgr2=0x7fad7c1a1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:27:59.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.415+0000 7fad827fd700 1 --2- 172.21.15.100:0/1384313859 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad7c102450 0x7fad7c1a1e40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fad6400e9f0 tx=0x7fad6400ed00 comp rx=0 tx=0).stop 2024-05-10T11:27:59.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.415+0000 7fad827fd700 1 -- 172.21.15.100:0/1384313859 shutdown_connections 2024-05-10T11:27:59.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.415+0000 7fad827fd700 1 --2- 172.21.15.100:0/1384313859 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fad680b2880 0x7fad680b4d40 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:59.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.415+0000 7fad827fd700 1 --2- 172.21.15.100:0/1384313859 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad7c109520 0x7fad7c19bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:59.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.415+0000 7fad827fd700 1 --2- 172.21.15.100:0/1384313859 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad7c101a90 0x7fad7c1a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:59.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.415+0000 7fad827fd700 1 --2- 172.21.15.100:0/1384313859 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad7c102450 0x7fad7c1a1e40 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:27:59.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.415+0000 7fad827fd700 1 -- 172.21.15.100:0/1384313859 >> 172.21.15.100:0/1384313859 conn(0x7fad7c078a10 msgr2=0x7fad7c10f070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:27:59.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.415+0000 7fad827fd700 1 -- 172.21.15.100:0/1384313859 shutdown_connections 2024-05-10T11:27:59.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:27:59.415+0000 7fad827fd700 1 -- 172.21.15.100:0/1384313859 wait complete. 2024-05-10T11:27:59.978 INFO:tasks.workunit.client.0.smithi100.stdout: 1 244553 2.74 MB/sec execute 354 sec latency 17.007 ms 2024-05-10T11:28:00.092 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T11:28:00.093 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T11:28:00.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:27:59 smithi178 ceph-mon[33359]: pgmap v708: 129 pgs: 129 active+clean; 101 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 1.5 MiB/s wr, 176 op/s 2024-05-10T11:28:00.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:27:59 smithi188 ceph-mon[34075]: pgmap v708: 129 pgs: 129 active+clean; 101 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 1.5 MiB/s wr, 176 op/s 2024-05-10T11:28:00.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:27:59 smithi100 ceph-mon[30296]: pgmap v708: 129 pgs: 129 active+clean; 101 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 1.5 MiB/s wr, 176 op/s 2024-05-10T11:28:00.370 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:28:00.978 INFO:tasks.workunit.client.0.smithi100.stdout: 1 245017 2.75 MB/sec execute 355 sec latency 14.367 ms 2024-05-10T11:28:01.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.063+0000 7fadcb5f0640 1 -- 172.21.15.100:0/1817257599 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fadc4103490 msgr2=0x7fadc4105880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:01.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.063+0000 7fadcb5f0640 1 --2- 172.21.15.100:0/1817257599 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fadc4103490 0x7fadc4105880 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fadc0009900 tx=0x7fadc002a3f0 comp rx=0 tx=0).stop 2024-05-10T11:28:01.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.063+0000 7fadcb5f0640 1 -- 172.21.15.100:0/1817257599 shutdown_connections 2024-05-10T11:28:01.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.063+0000 7fadcb5f0640 1 --2- 172.21.15.100:0/1817257599 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fadc41086f0 0x7fadc410ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:01.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.063+0000 7fadcb5f0640 1 --2- 172.21.15.100:0/1817257599 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadc4105dc0 0x7fadc41081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:01.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.063+0000 7fadcb5f0640 1 --2- 172.21.15.100:0/1817257599 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fadc4103490 0x7fadc4105880 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:01.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.063+0000 7fadcb5f0640 1 -- 172.21.15.100:0/1817257599 >> 172.21.15.100:0/1817257599 conn(0x7fadc40fce60 msgr2=0x7fadc40ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:01.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.063+0000 7fadcb5f0640 1 -- 172.21.15.100:0/1817257599 shutdown_connections 2024-05-10T11:28:01.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.063+0000 7fadcb5f0640 1 -- 172.21.15.100:0/1817257599 wait complete. 2024-05-10T11:28:01.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.064+0000 7fadcb5f0640 1 Processor -- start 2024-05-10T11:28:01.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.064+0000 7fadcb5f0640 1 -- start start 2024-05-10T11:28:01.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.064+0000 7fadcb5f0640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fadc4103490 0x7fadc4198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:01.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.064+0000 7fadcb5f0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fadc4105dc0 0x7fadc4199020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:01.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.064+0000 7fadcb5f0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadc41086f0 0x7fadc41a0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:01.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.064+0000 7fadcb5f0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fadc410e410 con 0x7fadc41086f0 2024-05-10T11:28:01.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.064+0000 7fadcb5f0640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fadc410e290 con 0x7fadc4103490 2024-05-10T11:28:01.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.064+0000 7fadcb5f0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fadc410e590 con 0x7fadc4105dc0 2024-05-10T11:28:01.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.064+0000 7fadc938c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fadc4103490 0x7fadc4198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:01.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.065+0000 7fadc8b8b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fadc4105dc0 0x7fadc4199020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:01.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.065+0000 7fadc938c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fadc4103490 0x7fadc4198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:38324/0 (socket says 172.21.15.100:38324) 2024-05-10T11:28:01.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.065+0000 7fadc9b8d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadc41086f0 0x7fadc41a0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:01.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.065+0000 7fadc938c640 1 -- 172.21.15.100:0/4200759233 learned_addr learned my addr 172.21.15.100:0/4200759233 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:28:01.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.065+0000 7fadc8b8b640 1 -- 172.21.15.100:0/4200759233 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fadc4103490 msgr2=0x7fadc4198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:01.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.065+0000 7fadc8b8b640 1 --2- 172.21.15.100:0/4200759233 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fadc4103490 0x7fadc4198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:01.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.065+0000 7fadc8b8b640 1 -- 172.21.15.100:0/4200759233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadc41086f0 msgr2=0x7fadc41a0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:01.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.065+0000 7fadc8b8b640 1 --2- 172.21.15.100:0/4200759233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadc41086f0 0x7fadc41a0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:01.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.065+0000 7fadc8b8b640 1 -- 172.21.15.100:0/4200759233 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadc41a0700 con 0x7fadc4105dc0 2024-05-10T11:28:01.068 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.065+0000 7fadc9b8d640 1 --2- 172.21.15.100:0/4200759233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadc41086f0 0x7fadc41a0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:28:01.068 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.065+0000 7fadc8b8b640 1 --2- 172.21.15.100:0/4200759233 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fadc4105dc0 0x7fadc4199020 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fadb400e9c0 tx=0x7fadb400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:01.068 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.065+0000 7fadba7fc640 1 -- 172.21.15.100:0/4200759233 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb400cd80 con 0x7fadc4105dc0 2024-05-10T11:28:01.068 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.065+0000 7fadba7fc640 1 -- 172.21.15.100:0/4200759233 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fadb40035f0 con 0x7fadc4105dc0 2024-05-10T11:28:01.068 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.065+0000 7fadcb5f0640 1 -- 172.21.15.100:0/4200759233 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadc41a09f0 con 0x7fadc4105dc0 2024-05-10T11:28:01.068 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.065+0000 7fadba7fc640 1 -- 172.21.15.100:0/4200759233 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb4010690 con 0x7fadc4105dc0 2024-05-10T11:28:01.068 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.065+0000 7fadcb5f0640 1 -- 172.21.15.100:0/4200759233 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadc41a0f40 con 0x7fadc4105dc0 2024-05-10T11:28:01.075 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.068+0000 7fadce3e5740 1 Processor -- start 2024-05-10T11:28:01.075 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.068+0000 7fadce3e5740 1 -- start start 2024-05-10T11:28:01.075 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.068+0000 7fadba7fc640 1 -- 172.21.15.100:0/4200759233 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fadb4004000 con 0x7fadc4105dc0 2024-05-10T11:28:01.075 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.068+0000 7fadce3e5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601c8fb01c0 0x5601c8fb05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:01.075 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.068+0000 7fadce3e5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601c8fb0bb0 0x5601c8fb56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:01.075 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.068+0000 7fadce3e5740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5601c8fb5d20 0x5601c8fb81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:01.075 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.068+0000 7fadce3e5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5601c8f105d0 con 0x5601c8fb0bb0 2024-05-10T11:28:01.075 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.068+0000 7fadce3e5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5601c8ec9890 con 0x5601c8fb01c0 2024-05-10T11:28:01.075 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.068+0000 7fadce3e5740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5601c8f0c590 con 0x5601c8fb5d20 2024-05-10T11:28:01.075 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.068+0000 7fadc938c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601c8fb0bb0 0x5601c8fb56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:01.076 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.068+0000 7fadc938c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601c8fb0bb0 0x5601c8fb56b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60406/0 (socket says 172.21.15.100:60406) 2024-05-10T11:28:01.076 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.068+0000 7fadc938c640 1 -- 172.21.15.100:0/2882848588 learned_addr learned my addr 172.21.15.100:0/2882848588 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:28:01.076 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.068+0000 7fadc9b8d640 1 --2- 172.21.15.100:0/2882848588 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601c8fb01c0 0x5601c8fb05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:01.076 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.068+0000 7fadc8b8b640 1 --2- 172.21.15.100:0/2882848588 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5601c8fb5d20 0x5601c8fb81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:01.076 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.069+0000 7fadc938c640 1 -- 172.21.15.100:0/2882848588 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5601c8fb5d20 msgr2=0x5601c8fb81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:01.076 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.069+0000 7fadc938c640 1 --2- 172.21.15.100:0/2882848588 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5601c8fb5d20 0x5601c8fb81b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:01.076 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.069+0000 7fadc938c640 1 -- 172.21.15.100:0/2882848588 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601c8fb01c0 msgr2=0x5601c8fb05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:01.076 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.069+0000 7fadc938c640 1 --2- 172.21.15.100:0/2882848588 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601c8fb01c0 0x5601c8fb05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:01.076 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.069+0000 7fadc938c640 1 -- 172.21.15.100:0/2882848588 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601c8c22210 con 0x5601c8fb0bb0 2024-05-10T11:28:01.076 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.069+0000 7fadc938c640 1 --2- 172.21.15.100:0/2882848588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601c8fb0bb0 0x5601c8fb56b0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fadc0006010 tx=0x7fadc0005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3dead68711d5f66 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:01.076 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.069+0000 7fadaf7fe640 1 -- 172.21.15.100:0/2882848588 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc0035070 con 0x5601c8fb0bb0 2024-05-10T11:28:01.077 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.069+0000 7fadba7fc640 1 --2- 172.21.15.100:0/4200759233 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fadb00b28a0 0x7fadb00b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:01.077 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.069+0000 7fadaf7fe640 1 -- 172.21.15.100:0/2882848588 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fadc0040070 con 0x5601c8fb0bb0 2024-05-10T11:28:01.077 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.069+0000 7fadaf7fe640 1 -- 172.21.15.100:0/2882848588 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc0003830 con 0x5601c8fb0bb0 2024-05-10T11:28:01.077 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.069+0000 7fadc9b8d640 1 --2- 172.21.15.100:0/4200759233 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fadb00b28a0 0x7fadb00b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:01.077 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.069+0000 7fadba7fc640 1 -- 172.21.15.100:0/4200759233 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fadb40e27b0 con 0x7fadc4105dc0 2024-05-10T11:28:01.077 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.070+0000 7fadce3e5740 1 -- 172.21.15.100:0/2882848588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601c8fb0bb0 msgr2=0x5601c8fb56b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:01.077 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.070+0000 7fadc9b8d640 1 --2- 172.21.15.100:0/4200759233 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fadb00b28a0 0x7fadb00b4d60 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x5601c8fb1410 tx=0x7fadbc009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:01.077 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.070+0000 7fadce3e5740 1 --2- 172.21.15.100:0/2882848588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601c8fb0bb0 0x5601c8fb56b0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fadc0006010 tx=0x7fadc0005e40 comp rx=0 tx=0).stop 2024-05-10T11:28:01.077 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.070+0000 7fadce3e5740 1 -- 172.21.15.100:0/2882848588 shutdown_connections 2024-05-10T11:28:01.077 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.070+0000 7fadce3e5740 1 --2- 172.21.15.100:0/2882848588 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5601c8fb5d20 0x5601c8fb81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:01.078 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.070+0000 7fadce3e5740 1 --2- 172.21.15.100:0/2882848588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601c8fb0bb0 0x5601c8fb56b0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:01.078 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.070+0000 7fadce3e5740 1 --2- 172.21.15.100:0/2882848588 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601c8fb01c0 0x5601c8fb05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:01.078 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.070+0000 7fadce3e5740 1 -- 172.21.15.100:0/2882848588 >> 172.21.15.100:0/2882848588 conn(0x5601c8fa7420 msgr2=0x5601c8fa7820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:01.078 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.070+0000 7fadce3e5740 1 -- 172.21.15.100:0/2882848588 shutdown_connections 2024-05-10T11:28:01.078 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.070+0000 7fadce3e5740 1 -- 172.21.15.100:0/2882848588 wait complete. 2024-05-10T11:28:01.078 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.070+0000 7fadce3e5740 1 -- start start 2024-05-10T11:28:01.078 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.070+0000 7fadce3e5740 1 Processor -- start 2024-05-10T11:28:01.079 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.071+0000 7fadce3e5740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5601c8fb01c0 0x5601c8fba3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:01.079 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.071+0000 7fadce3e5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601c8fb0bb0 0x5601c8fba8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:01.079 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.071+0000 7fadce3e5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601c8fb5d20 0x5601c8fc1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:01.079 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.071+0000 7fadce3e5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5601c8fb8c70 con 0x5601c8fb5d20 2024-05-10T11:28:01.079 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.071+0000 7fadce3e5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5601c8ecd370 con 0x5601c8fb0bb0 2024-05-10T11:28:01.079 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.071+0000 7fadce3e5740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5601c8fb8df0 con 0x5601c8fb01c0 2024-05-10T11:28:01.079 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.071+0000 7fadce3e5740 10 client.0 _ll_register_callbacks cb 0x5601c8c21a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:28:01.079 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.071+0000 7fadc9b8d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601c8fb5d20 0x5601c8fc1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:01.080 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.071+0000 7fadc8b8b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601c8fb0bb0 0x5601c8fba8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:01.080 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.071+0000 7fadc9b8d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601c8fb5d20 0x5601c8fc1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60422/0 (socket says 172.21.15.100:60422) 2024-05-10T11:28:01.080 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.071+0000 7fadc8b8b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601c8fb0bb0 0x5601c8fba8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:38342/0 (socket says 172.21.15.100:38342) 2024-05-10T11:28:01.080 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.071+0000 7fadc9b8d640 1 -- 172.21.15.100:0/2394066862 learned_addr learned my addr 172.21.15.100:0/2394066862 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:28:01.080 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.071+0000 7fadc938c640 1 --2- 172.21.15.100:0/2394066862 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5601c8fb01c0 0x5601c8fba3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:01.080 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.071+0000 7fadc9b8d640 1 -- 172.21.15.100:0/2394066862 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5601c8fb01c0 msgr2=0x5601c8fba3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:01.080 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.071+0000 7fadc9b8d640 1 --2- 172.21.15.100:0/2394066862 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5601c8fb01c0 0x5601c8fba3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:01.080 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.071+0000 7fadc9b8d640 1 -- 172.21.15.100:0/2394066862 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601c8fb0bb0 msgr2=0x5601c8fba8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:01.080 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.071+0000 7fadc9b8d640 1 --2- 172.21.15.100:0/2394066862 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601c8fb0bb0 0x5601c8fba8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:01.080 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.071+0000 7fadc9b8d640 1 -- 172.21.15.100:0/2394066862 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601c8cee620 con 0x5601c8fb5d20 2024-05-10T11:28:01.080 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.071+0000 7fadc8b8b640 1 --2- 172.21.15.100:0/2394066862 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601c8fb0bb0 0x5601c8fba8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:28:01.081 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.071+0000 7fadc9b8d640 1 --2- 172.21.15.100:0/2394066862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601c8fb5d20 0x5601c8fc1930 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fadbc0099e0 tx=0x7fadbc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec378b76346f768b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:01.081 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.071+0000 7fadacff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:28:01.081 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.072+0000 7fadacff9640 1 -- 172.21.15.100:0/2394066862 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadbc011bd0 con 0x5601c8fb5d20 2024-05-10T11:28:01.081 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.072+0000 7fadce3e5740 1 -- 172.21.15.100:0/2394066862 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadc0040070 con 0x5601c8fb5d20 2024-05-10T11:28:01.081 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.072+0000 7fadacff9640 1 -- 172.21.15.100:0/2394066862 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fadbc011d70 con 0x5601c8fb5d20 2024-05-10T11:28:01.081 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.072+0000 7fadacff9640 1 -- 172.21.15.100:0/2394066862 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fadbc029440 con 0x5601c8fb5d20 2024-05-10T11:28:01.081 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.072+0000 7fadce3e5740 10 client.17307 fetch_fsmap learned FSMap version 31 2024-05-10T11:28:01.082 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.072+0000 7fadce3e5740 1 -- 172.21.15.100:0/2394066862 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5601c8fc4a30 con 0x5601c8fb5d20 2024-05-10T11:28:01.082 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.072+0000 7fadacff9640 1 -- 172.21.15.100:0/2394066862 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fadc0040070 con 0x5601c8fb5d20 2024-05-10T11:28:01.082 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.072+0000 7fadacff9640 1 -- 172.21.15.100:0/2394066862 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fadbc027070 con 0x5601c8fb5d20 2024-05-10T11:28:01.082 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.072+0000 7fadce3e5740 10 client.17307 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:28:01.082 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.072+0000 7fadce3e5740 10 client.17307 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:28:01.082 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.072+0000 7fadce3e5740 20 client.17307 populate_metadata read hostname 'smithi100' 2024-05-10T11:28:01.083 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.072+0000 7fadce3e5740 1 --2- 172.21.15.100:0/2394066862 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5601c8fc6170 0x5601c8fc85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:01.083 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.072+0000 7fadce3e5740 4 client.17307 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:28:01.084 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.072+0000 7fadce3e5740 1 -- 172.21.15.100:0/2394066862 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5601c8f1e1e0 con 0x5601c8fc6170 2024-05-10T11:28:01.084 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.072+0000 7fadc938c640 1 --2- 172.21.15.100:0/2394066862 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5601c8fc6170 0x5601c8fc85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:01.084 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.073+0000 7fadacff9640 10 client.17307 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:28:01.084 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.073+0000 7fadc938c640 1 --2- 172.21.15.100:0/2394066862 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5601c8fc6170 0x5601c8fc85d0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb93398d0f059df2 server_cookie=4c373f014d370e10 in_seq=0 out_seq=0 2024-05-10T11:28:01.085 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.073+0000 7fadacff9640 1 -- 172.21.15.100:0/2394066862 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5601c8f1e1e0 con 0x5601c8fc6170 2024-05-10T11:28:01.085 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.073+0000 7fadacff9640 10 client.17307 handle_command_reply: tid=0 2024-05-10T11:28:01.085 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.073+0000 7fadce3e5740 1 client.17307 shutdown 2024-05-10T11:28:01.085 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.074+0000 7fadce3e5740 1 -- 172.21.15.100:0/2394066862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601c8fb5d20 msgr2=0x5601c8fc1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:01.086 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.074+0000 7fadce3e5740 1 --2- 172.21.15.100:0/2394066862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601c8fb5d20 0x5601c8fc1930 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fadbc0099e0 tx=0x7fadbc004ab0 comp rx=0 tx=0).stop 2024-05-10T11:28:01.086 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.074+0000 7fadce3e5740 1 -- 172.21.15.100:0/2394066862 shutdown_connections 2024-05-10T11:28:01.086 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.074+0000 7fadce3e5740 1 --2- 172.21.15.100:0/2394066862 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5601c8fc6170 0x5601c8fc85d0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:01.086 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.074+0000 7fadce3e5740 1 --2- 172.21.15.100:0/2394066862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601c8fb5d20 0x5601c8fc1930 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:01.086 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.074+0000 7fadce3e5740 1 --2- 172.21.15.100:0/2394066862 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601c8fb0bb0 0x5601c8fba8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:01.086 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.074+0000 7fadce3e5740 1 --2- 172.21.15.100:0/2394066862 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5601c8fb01c0 0x5601c8fba3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:01.086 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.074+0000 7fadce3e5740 1 -- 172.21.15.100:0/2394066862 >> 172.21.15.100:0/2394066862 conn(0x5601c8fa7420 msgr2=0x5601c8fa6580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:01.086 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.074+0000 7fadacff9640 0 client.17307 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:28:01.086 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.074+0000 7fadce3e5740 1 -- 172.21.15.100:0/2394066862 shutdown_connections 2024-05-10T11:28:01.086 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.074+0000 7fadce3e5740 1 -- 172.21.15.100:0/2394066862 wait complete. 2024-05-10T11:28:01.087 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.074+0000 7fadce3e5740 20 client.17307 trim_cache size 0 max 16384 2024-05-10T11:28:01.097 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.090+0000 7fadce3e5740 1 Processor -- start 2024-05-10T11:28:01.097 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.090+0000 7fadce3e5740 1 -- start start 2024-05-10T11:28:01.098 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.090+0000 7fadce3e5740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5601c8fb01c0 0x5601c8faaa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:01.098 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.090+0000 7fadce3e5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601c8fb0bb0 0x5601c8fc18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:01.098 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.090+0000 7fadce3e5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601c8fb5d20 0x5601c8fcced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:01.098 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.090+0000 7fadce3e5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fadc41ab950 con 0x5601c8fb5d20 2024-05-10T11:28:01.098 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.090+0000 7fadce3e5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fadc41a9220 con 0x5601c8fb0bb0 2024-05-10T11:28:01.098 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.090+0000 7fadce3e5740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fadc41a7070 con 0x5601c8fb01c0 2024-05-10T11:28:01.098 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.090+0000 7fadc9b8d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601c8fb5d20 0x5601c8fcced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:01.099 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.090+0000 7fadc938c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5601c8fb01c0 0x5601c8faaa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:01.099 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.090+0000 7fadc8b8b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601c8fb0bb0 0x5601c8fc18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:01.099 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.090+0000 7fadc938c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5601c8fb01c0 0x5601c8faaa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:50944/0 (socket says 172.21.15.100:50944) 2024-05-10T11:28:01.099 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.090+0000 7fadc8b8b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601c8fb0bb0 0x5601c8fc18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:38358/0 (socket says 172.21.15.100:38358) 2024-05-10T11:28:01.099 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.090+0000 7fadc938c640 1 -- 172.21.15.100:0/2757116295 learned_addr learned my addr 172.21.15.100:0/2757116295 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:28:01.099 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.090+0000 7fadc9b8d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601c8fb5d20 0x5601c8fcced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60434/0 (socket says 172.21.15.100:60434) 2024-05-10T11:28:01.099 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.091+0000 7fadc8b8b640 1 -- 172.21.15.100:0/2757116295 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5601c8fb01c0 msgr2=0x5601c8faaa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:01.099 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.091+0000 7fadc8b8b640 1 --2- 172.21.15.100:0/2757116295 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5601c8fb01c0 0x5601c8faaa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:01.100 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.091+0000 7fadc8b8b640 1 -- 172.21.15.100:0/2757116295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601c8fb5d20 msgr2=0x5601c8fcced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:01.100 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.091+0000 7fadc8b8b640 1 --2- 172.21.15.100:0/2757116295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601c8fb5d20 0x5601c8fcced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:01.100 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.091+0000 7fadc8b8b640 1 -- 172.21.15.100:0/2757116295 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601c8fa7250 con 0x5601c8fb0bb0 2024-05-10T11:28:01.100 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.091+0000 7fadc9b8d640 1 --2- 172.21.15.100:0/2757116295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601c8fb5d20 0x5601c8fcced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:28:01.100 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.091+0000 7fadc8b8b640 1 --2- 172.21.15.100:0/2757116295 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601c8fb0bb0 0x5601c8fc18f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fadb40dffa0 tx=0x7fadb4021000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5311367cd1dd12f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:01.100 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.091+0000 7fadaeffd640 1 -- 172.21.15.100:0/2757116295 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb40b2210 con 0x5601c8fb0bb0 2024-05-10T11:28:01.100 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.091+0000 7fadaeffd640 1 -- 172.21.15.100:0/2757116295 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fadb40a9580 con 0x5601c8fb0bb0 2024-05-10T11:28:01.100 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.091+0000 7fadaeffd640 1 -- 172.21.15.100:0/2757116295 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb4003ac0 con 0x5601c8fb0bb0 2024-05-10T11:28:01.100 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.092+0000 7fadce3e5740 1 -- 172.21.15.100:0/2757116295 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601c8fb0bb0 msgr2=0x5601c8fc18f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:01.100 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.092+0000 7fadce3e5740 1 --2- 172.21.15.100:0/2757116295 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601c8fb0bb0 0x5601c8fc18f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fadb40dffa0 tx=0x7fadb4021000 comp rx=0 tx=0).stop 2024-05-10T11:28:01.100 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.092+0000 7fadce3e5740 1 -- 172.21.15.100:0/2757116295 shutdown_connections 2024-05-10T11:28:01.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.092+0000 7fadce3e5740 1 --2- 172.21.15.100:0/2757116295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601c8fb5d20 0x5601c8fcced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:01.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.092+0000 7fadce3e5740 1 --2- 172.21.15.100:0/2757116295 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601c8fb0bb0 0x5601c8fc18f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:01.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.092+0000 7fadce3e5740 1 --2- 172.21.15.100:0/2757116295 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5601c8fb01c0 0x5601c8faaa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:01.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.092+0000 7fadce3e5740 1 -- 172.21.15.100:0/2757116295 >> 172.21.15.100:0/2757116295 conn(0x5601c8fa7420 msgr2=0x5601c8faa330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:01.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.092+0000 7fadce3e5740 1 -- 172.21.15.100:0/2757116295 shutdown_connections 2024-05-10T11:28:01.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.092+0000 7fadce3e5740 1 -- 172.21.15.100:0/2757116295 wait complete. 2024-05-10T11:28:01.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.092+0000 7fadce3e5740 1 -- start start 2024-05-10T11:28:01.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.092+0000 7fadce3e5740 1 Processor -- start 2024-05-10T11:28:01.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.092+0000 7fadce3e5740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5601c8fb01c0 0x5601c8fbf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:01.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.092+0000 7fadce3e5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601c8fb0bb0 0x5601c8fbfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:01.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.092+0000 7fadce3e5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601c8fb5d20 0x5601c8fc0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:01.102 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.092+0000 7fadce3e5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fadc41a76e0 con 0x5601c8fb5d20 2024-05-10T11:28:01.102 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.092+0000 7fadce3e5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fadc41a8b50 con 0x5601c8fb0bb0 2024-05-10T11:28:01.102 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.092+0000 7fadce3e5740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fadc41a7db0 con 0x5601c8fb01c0 2024-05-10T11:28:01.102 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.093+0000 7fadce3e5740 10 client.0 _ll_register_callbacks cb 0x7fadbc012710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:28:01.102 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.093+0000 7fadc9b8d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601c8fb5d20 0x5601c8fc0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:01.102 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.093+0000 7fadc9b8d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601c8fb5d20 0x5601c8fc0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60442/0 (socket says 172.21.15.100:60442) 2024-05-10T11:28:01.102 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.093+0000 7fadc9b8d640 1 -- 172.21.15.100:0/3938864626 learned_addr learned my addr 172.21.15.100:0/3938864626 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:28:01.102 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.093+0000 7fadc9b8d640 1 -- 172.21.15.100:0/3938864626 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5601c8fb01c0 msgr2=0x5601c8fbf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:01.102 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.093+0000 7fadc9b8d640 1 --2- 172.21.15.100:0/3938864626 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5601c8fb01c0 0x5601c8fbf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:01.102 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.093+0000 7fadc9b8d640 1 -- 172.21.15.100:0/3938864626 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601c8fb0bb0 msgr2=0x5601c8fbfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:01.103 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.093+0000 7fadc9b8d640 1 --2- 172.21.15.100:0/3938864626 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601c8fb0bb0 0x5601c8fbfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:01.103 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.093+0000 7fadc9b8d640 1 -- 172.21.15.100:0/3938864626 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601c8fc10d0 con 0x5601c8fb5d20 2024-05-10T11:28:01.103 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.093+0000 7fadc9b8d640 1 --2- 172.21.15.100:0/3938864626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601c8fb5d20 0x5601c8fc0ae0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fadbc01f720 tx=0x7fadbc01f650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36801bdecf4be6aa server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:01.103 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.093+0000 7fadacff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:28:01.103 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.093+0000 7fadacff9640 1 -- 172.21.15.100:0/3938864626 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadbc03b070 con 0x5601c8fb5d20 2024-05-10T11:28:01.103 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.093+0000 7fadacff9640 1 -- 172.21.15.100:0/3938864626 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fadbc0295e0 con 0x5601c8fb5d20 2024-05-10T11:28:01.103 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.093+0000 7fadce3e5740 1 -- 172.21.15.100:0/3938864626 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadb40a9580 con 0x5601c8fb5d20 2024-05-10T11:28:01.103 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.093+0000 7fadacff9640 1 -- 172.21.15.100:0/3938864626 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fadbc011810 con 0x5601c8fb5d20 2024-05-10T11:28:01.103 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.093+0000 7fadce3e5740 10 client.17313 fetch_fsmap learned FSMap version 31 2024-05-10T11:28:01.103 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.093+0000 7fadce3e5740 1 -- 172.21.15.100:0/3938864626 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5601c9030fc0 con 0x5601c8fb5d20 2024-05-10T11:28:01.103 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.094+0000 7fadacff9640 1 -- 172.21.15.100:0/3938864626 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fadb40a9580 con 0x5601c8fb5d20 2024-05-10T11:28:01.104 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.094+0000 7fadacff9640 1 -- 172.21.15.100:0/3938864626 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fadbc027040 con 0x5601c8fb5d20 2024-05-10T11:28:01.104 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.094+0000 7fadce3e5740 10 client.17313 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:28:01.104 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.094+0000 7fadce3e5740 10 client.17313 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:28:01.104 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.094+0000 7fadce3e5740 20 client.17313 populate_metadata read hostname 'smithi100' 2024-05-10T11:28:01.104 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.094+0000 7fadce3e5740 1 --2- 172.21.15.100:0/3938864626 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5601c90319f0 0x5601c9033e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:01.104 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.094+0000 7fadce3e5740 4 client.17313 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:28:01.104 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.094+0000 7fadce3e5740 1 -- 172.21.15.100:0/3938864626 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5601c8f18050 con 0x5601c90319f0 2024-05-10T11:28:01.104 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.094+0000 7fadc938c640 1 --2- 172.21.15.100:0/3938864626 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5601c90319f0 0x5601c9033e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:01.104 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.094+0000 7fadacff9640 10 client.17313 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:28:01.104 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.094+0000 7fadc938c640 1 --2- 172.21.15.100:0/3938864626 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5601c90319f0 0x5601c9033e50 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2aeed9922261498a server_cookie=dde63aae7497b60b in_seq=0 out_seq=0 2024-05-10T11:28:01.113 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.113+0000 7fadacff9640 1 -- 172.21.15.100:0/3938864626 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5601c8f18050 con 0x5601c90319f0 2024-05-10T11:28:01.114 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.113+0000 7fadacff9640 10 client.17313 handle_command_reply: tid=0 2024-05-10T11:28:01.114 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.113+0000 7fadce3e5740 1 client.17313 shutdown 2024-05-10T11:28:01.114 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.113+0000 7fadce3e5740 1 -- 172.21.15.100:0/3938864626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601c8fb5d20 msgr2=0x5601c8fc0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:01.114 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.113+0000 7fadce3e5740 1 --2- 172.21.15.100:0/3938864626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601c8fb5d20 0x5601c8fc0ae0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fadbc01f720 tx=0x7fadbc01f650 comp rx=0 tx=0).stop 2024-05-10T11:28:01.114 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.113+0000 7fadce3e5740 1 -- 172.21.15.100:0/3938864626 shutdown_connections 2024-05-10T11:28:01.114 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.113+0000 7fadce3e5740 1 --2- 172.21.15.100:0/3938864626 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5601c90319f0 0x5601c9033e50 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:01.114 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.113+0000 7fadce3e5740 1 --2- 172.21.15.100:0/3938864626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601c8fb5d20 0x5601c8fc0ae0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:01.114 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.113+0000 7fadacff9640 0 client.17313 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:28:01.114 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.113+0000 7fadce3e5740 1 --2- 172.21.15.100:0/3938864626 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601c8fb0bb0 0x5601c8fbfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:01.114 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.113+0000 7fadce3e5740 1 --2- 172.21.15.100:0/3938864626 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5601c8fb01c0 0x5601c8fbf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:01.115 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.113+0000 7fadce3e5740 1 -- 172.21.15.100:0/3938864626 >> 172.21.15.100:0/3938864626 conn(0x5601c8fa7420 msgr2=0x5601c8fa5cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:01.115 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.113+0000 7fadce3e5740 1 -- 172.21.15.100:0/3938864626 shutdown_connections 2024-05-10T11:28:01.115 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.113+0000 7fadce3e5740 1 -- 172.21.15.100:0/3938864626 wait complete. 2024-05-10T11:28:01.115 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.113+0000 7fadce3e5740 20 client.17313 trim_cache size 0 max 16384 2024-05-10T11:28:01.115 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:28:01.115 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:28:01.115 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:28:01.116 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:28:01.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.114+0000 7fadcb5f0640 1 -- 172.21.15.100:0/4200759233 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fadb00b28a0 msgr2=0x7fadb00b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:01.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.114+0000 7fadcb5f0640 1 --2- 172.21.15.100:0/4200759233 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fadb00b28a0 0x7fadb00b4d60 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x5601c8fb1410 tx=0x7fadbc009500 comp rx=0 tx=0).stop 2024-05-10T11:28:01.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.114+0000 7fadcb5f0640 1 -- 172.21.15.100:0/4200759233 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fadc4105dc0 msgr2=0x7fadc4199020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:01.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.114+0000 7fadcb5f0640 1 --2- 172.21.15.100:0/4200759233 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fadc4105dc0 0x7fadc4199020 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fadb400e9c0 tx=0x7fadb400ee80 comp rx=0 tx=0).stop 2024-05-10T11:28:01.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.114+0000 7fadcb5f0640 1 -- 172.21.15.100:0/4200759233 shutdown_connections 2024-05-10T11:28:01.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.114+0000 7fadcb5f0640 1 --2- 172.21.15.100:0/4200759233 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fadb00b28a0 0x7fadb00b4d60 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:01.118 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.114+0000 7fadcb5f0640 1 --2- 172.21.15.100:0/4200759233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadc41086f0 0x7fadc41a0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:01.118 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.114+0000 7fadcb5f0640 1 --2- 172.21.15.100:0/4200759233 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fadc4105dc0 0x7fadc4199020 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:01.118 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.114+0000 7fadcb5f0640 1 --2- 172.21.15.100:0/4200759233 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fadc4103490 0x7fadc4198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:01.118 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.114+0000 7fadcb5f0640 1 -- 172.21.15.100:0/4200759233 >> 172.21.15.100:0/4200759233 conn(0x7fadc40fce60 msgr2=0x7fadc4103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:01.118 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.114+0000 7fadcb5f0640 1 -- 172.21.15.100:0/4200759233 shutdown_connections 2024-05-10T11:28:01.118 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:01.114+0000 7fadcb5f0640 1 -- 172.21.15.100:0/4200759233 wait complete. 2024-05-10T11:28:01.978 INFO:tasks.workunit.client.0.smithi100.stdout: 1 245252 2.74 MB/sec execute 356 sec latency 79.104 ms 2024-05-10T11:28:02.026 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:28:01 smithi100 ceph-mon[30296]: pgmap v709: 129 pgs: 129 active+clean; 101 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 1.1 MiB/s wr, 130 op/s 2024-05-10T11:28:02.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:28:01 smithi178 ceph-mon[33359]: pgmap v709: 129 pgs: 129 active+clean; 101 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 1.1 MiB/s wr, 130 op/s 2024-05-10T11:28:02.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:28:01 smithi188 ceph-mon[34075]: pgmap v709: 129 pgs: 129 active+clean; 101 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 1.1 MiB/s wr, 130 op/s 2024-05-10T11:28:02.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:02.280+0000 7fefea662700 1 -- 172.21.15.100:0/272044400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fefe4101e40 msgr2=0x7fefe41022c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:02.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:02.280+0000 7fefea662700 1 --2- 172.21.15.100:0/272044400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fefe4101e40 0x7fefe41022c0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fefd80099c0 tx=0x7fefd8009cd0 comp rx=0 tx=0).stop 2024-05-10T11:28:02.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:02.280+0000 7fefea662700 1 -- 172.21.15.100:0/272044400 shutdown_connections 2024-05-10T11:28:02.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:02.280+0000 7fefea662700 1 --2- 172.21.15.100:0/272044400 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fefe4102800 0x7fefe4110390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:02.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:02.280+0000 7fefea662700 1 --2- 172.21.15.100:0/272044400 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fefe4108510 0x7fefe41088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:02.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:02.280+0000 7fefea662700 1 --2- 172.21.15.100:0/272044400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fefe4101e40 0x7fefe41022c0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:02.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:02.280+0000 7fefea662700 1 -- 172.21.15.100:0/272044400 >> 172.21.15.100:0/272044400 conn(0x7fefe4077740 msgr2=0x7fefe4077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:02.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:02.280+0000 7fefea662700 1 -- 172.21.15.100:0/272044400 shutdown_connections 2024-05-10T11:28:02.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:02.280+0000 7fefea662700 1 -- 172.21.15.100:0/272044400 wait complete. 2024-05-10T11:28:02.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:02.281+0000 7fefea662700 1 Processor -- start 2024-05-10T11:28:02.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:02.281+0000 7fefea662700 1 -- start start 2024-05-10T11:28:02.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:02.282+0000 7fefea662700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fefe4101e40 0x7fefe419f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:02.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:02.282+0000 7fefea662700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fefe4102800 0x7fefe419fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:02.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:02.282+0000 7fefea662700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fefe4108510 0x7fefe4199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:02.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:02.282+0000 7fefea662700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fefe4113530 con 0x7fefe4102800 2024-05-10T11:28:02.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:02.282+0000 7fefea662700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fefe41133b0 con 0x7fefe4101e40 2024-05-10T11:28:02.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:02.282+0000 7fefea662700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fefe41136b0 con 0x7fefe4108510 2024-05-10T11:28:02.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:02.282+0000 7fefe3fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fefe4101e40 0x7fefe419f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:02.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:02.282+0000 7fefe3fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fefe4101e40 0x7fefe419f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:38382/0 (socket says 172.21.15.100:38382) 2024-05-10T11:28:02.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:02.282+0000 7fefe3fff700 1 -- 172.21.15.100:0/3678438911 learned_addr learned my addr 172.21.15.100:0/3678438911 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:28:02.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:02.282+0000 7fefe8bff700 1 --2- 172.21.15.100:0/3678438911 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fefe4108510 0x7fefe4199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:02.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:02.282+0000 7fefe37fe700 1 --2- 172.21.15.100:0/3678438911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fefe4102800 0x7fefe419fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:02.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:02.282+0000 7fefe3fff700 1 -- 172.21.15.100:0/3678438911 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fefe4108510 msgr2=0x7fefe4199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:02.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:02.282+0000 7fefe3fff700 1 --2- 172.21.15.100:0/3678438911 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fefe4108510 0x7fefe4199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:02.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:02.282+0000 7fefe3fff700 1 -- 172.21.15.100:0/3678438911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fefe4102800 msgr2=0x7fefe419fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:02.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:02.282+0000 7fefe3fff700 1 --2- 172.21.15.100:0/3678438911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fefe4102800 0x7fefe419fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:02.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:02.282+0000 7fefe3fff700 1 -- 172.21.15.100:0/3678438911 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefe419a010 con 0x7fefe4101e40 2024-05-10T11:28:02.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:02.282+0000 7fefe8bff700 1 --2- 172.21.15.100:0/3678438911 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fefe4108510 0x7fefe4199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:28:02.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:02.282+0000 7fefe3fff700 1 --2- 172.21.15.100:0/3678438911 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fefe4101e40 0x7fefe419f6b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fefd400e9a0 tx=0x7fefd400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:02.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:02.283+0000 7fefe17fa700 1 -- 172.21.15.100:0/3678438911 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd400ca10 con 0x7fefe4101e40 2024-05-10T11:28:02.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:02.283+0000 7fefea662700 1 -- 172.21.15.100:0/3678438911 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefe419a300 con 0x7fefe4101e40 2024-05-10T11:28:02.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:02.283+0000 7fefe17fa700 1 -- 172.21.15.100:0/3678438911 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fefd400cbb0 con 0x7fefe4101e40 2024-05-10T11:28:02.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:02.283+0000 7fefe17fa700 1 -- 172.21.15.100:0/3678438911 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd4018680 con 0x7fefe4101e40 2024-05-10T11:28:02.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:02.283+0000 7fefea662700 1 -- 172.21.15.100:0/3678438911 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefe41a6510 con 0x7fefe4101e40 2024-05-10T11:28:02.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:02.284+0000 7fefea662700 1 -- 172.21.15.100:0/3678438911 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefd0005330 con 0x7fefe4101e40 2024-05-10T11:28:02.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:02.285+0000 7fefe17fa700 1 -- 172.21.15.100:0/3678438911 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fefd4010430 con 0x7fefe4101e40 2024-05-10T11:28:02.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:02.286+0000 7fefe17fa700 1 --2- 172.21.15.100:0/3678438911 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fefcc0b2880 0x7fefcc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:02.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:02.286+0000 7fefe37fe700 1 --2- 172.21.15.100:0/3678438911 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fefcc0b2880 0x7fefcc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:02.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:02.286+0000 7fefe17fa700 1 -- 172.21.15.100:0/3678438911 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fefd40e2ab0 con 0x7fefe4101e40 2024-05-10T11:28:02.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:02.286+0000 7fefe37fe700 1 --2- 172.21.15.100:0/3678438911 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fefcc0b2880 0x7fefcc0b4d40 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fefe4077e90 tx=0x7fefd8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:02.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:02.290+0000 7fefe17fa700 1 -- 172.21.15.100:0/3678438911 <== mon.2 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fefd40aa410 con 0x7fefe4101e40 2024-05-10T11:28:02.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:02.582+0000 7fefea662700 1 -- 172.21.15.100:0/3678438911 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fefd0005620 con 0x7fefe4101e40 2024-05-10T11:28:02.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:02.583+0000 7fefe17fa700 1 -- 172.21.15.100:0/3678438911 <== mon.2 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11894 (secure 0 0 0) 0x7fefd40a9d10 con 0x7fefe4101e40 2024-05-10T11:28:02.583 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:28:02.583 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2072225012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2072225012},{"type":"v1","addr":"172.21.15.100:6835","nonce":2072225012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/401939540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":401939540},{"type":"v1","addr":"172.21.15.100:6837","nonce":401939540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14571,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/3946311025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3946311025},{"type":"v1","addr":"172.21.15.100:6839","nonce":3946311025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3736371660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3736371660},{"type":"v1","addr":"172.21.15.100:6841","nonce":3736371660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/2741810791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2741810791},{"type":"v1","addr":"172.21.15.178:6833","nonce":2741810791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1704891041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1704891041},{"type":"v1","addr":"172.21.15.188:6833","nonce":1704891041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T11:14:27.242452+0000","modified":"2024-05-10T11:23:19.557113+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24439,"mds_1":24437,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/2124076312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2124076312},{"type":"v1","addr":"172.21.15.178:6835","nonce":2124076312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/3224358425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3224358425},{"type":"v1","addr":"172.21.15.178:6837","nonce":3224358425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/515547690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":515547690},{"type":"v1","addr":"172.21.15.188:6835","nonce":515547690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/1943342620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1943342620},{"type":"v1","addr":"172.21.15.188:6837","nonce":1943342620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.178:6839/3760985404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3760985404},{"type":"v1","addr":"172.21.15.178:6839","nonce":3760985404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.188:6839/3194917070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":3194917070},{"type":"v1","addr":"172.21.15.188:6839","nonce":3194917070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24439,"qdb_cluster":[24439,24427,24437]},"id":1}]} 2024-05-10T11:28:02.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:02.585+0000 7fefea662700 1 -- 172.21.15.100:0/3678438911 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fefcc0b2880 msgr2=0x7fefcc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:02.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:02.585+0000 7fefea662700 1 --2- 172.21.15.100:0/3678438911 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fefcc0b2880 0x7fefcc0b4d40 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fefe4077e90 tx=0x7fefd8005fb0 comp rx=0 tx=0).stop 2024-05-10T11:28:02.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:02.585+0000 7fefea662700 1 -- 172.21.15.100:0/3678438911 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fefe4101e40 msgr2=0x7fefe419f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:02.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:02.585+0000 7fefea662700 1 --2- 172.21.15.100:0/3678438911 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fefe4101e40 0x7fefe419f6b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fefd400e9a0 tx=0x7fefd400ecb0 comp rx=0 tx=0).stop 2024-05-10T11:28:02.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:02.585+0000 7fefea662700 1 -- 172.21.15.100:0/3678438911 shutdown_connections 2024-05-10T11:28:02.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:02.585+0000 7fefea662700 1 --2- 172.21.15.100:0/3678438911 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fefcc0b2880 0x7fefcc0b4d40 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:02.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:02.585+0000 7fefea662700 1 --2- 172.21.15.100:0/3678438911 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fefe4108510 0x7fefe4199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:02.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:02.585+0000 7fefea662700 1 --2- 172.21.15.100:0/3678438911 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fefe4101e40 0x7fefe419f6b0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:02.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:02.585+0000 7fefea662700 1 --2- 172.21.15.100:0/3678438911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fefe4102800 0x7fefe419fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:02.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:02.585+0000 7fefea662700 1 -- 172.21.15.100:0/3678438911 >> 172.21.15.100:0/3678438911 conn(0x7fefe4077740 msgr2=0x7fefe40fff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:02.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:02.585+0000 7fefea662700 1 -- 172.21.15.100:0/3678438911 shutdown_connections 2024-05-10T11:28:02.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:02.585+0000 7fefea662700 1 -- 172.21.15.100:0/3678438911 wait complete. 2024-05-10T11:28:02.586 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 31 2024-05-10T11:28:02.978 INFO:tasks.workunit.client.0.smithi100.stdout: 1 245615 2.74 MB/sec execute 357 sec latency 45.459 ms 2024-05-10T11:28:03.192 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:28:02 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/3678438911' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:28:03.221 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-10T11:28:03.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:28:02 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/3678438911' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:28:03.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:28:02 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/3678438911' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:28:03.537 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:28:03.979 INFO:tasks.workunit.client.0.smithi100.stdout: 1 246180 2.74 MB/sec execute 358 sec latency 19.761 ms 2024-05-10T11:28:04.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:28:03 smithi178 ceph-mon[33359]: pgmap v710: 129 pgs: 129 active+clean; 100 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 1.4 MiB/s wr, 151 op/s 2024-05-10T11:28:04.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:28:03 smithi188 ceph-mon[34075]: pgmap v710: 129 pgs: 129 active+clean; 100 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 1.4 MiB/s wr, 151 op/s 2024-05-10T11:28:04.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:28:03 smithi100 ceph-mon[30296]: pgmap v710: 129 pgs: 129 active+clean; 100 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 1.4 MiB/s wr, 151 op/s 2024-05-10T11:28:04.979 INFO:tasks.workunit.client.0.smithi100.stdout: 1 246823 2.73 MB/sec execute 359 sec latency 16.860 ms 2024-05-10T11:28:05.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.237+0000 7ff31f1b9700 1 -- 172.21.15.100:0/2460565028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff318109530 msgr2=0x7ff318109910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:05.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.237+0000 7ff31f1b9700 1 --2- 172.21.15.100:0/2460565028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff318109530 0x7ff318109910 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7ff30c009990 tx=0x7ff30c009ca0 comp rx=0 tx=0).stop 2024-05-10T11:28:05.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.237+0000 7ff31f1b9700 1 -- 172.21.15.100:0/2460565028 shutdown_connections 2024-05-10T11:28:05.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.237+0000 7ff31f1b9700 1 --2- 172.21.15.100:0/2460565028 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff318103ee0 0x7ff318110370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:05.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.237+0000 7ff31f1b9700 1 --2- 172.21.15.100:0/2460565028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff318109530 0x7ff318109910 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:05.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.237+0000 7ff31f1b9700 1 --2- 172.21.15.100:0/2460565028 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff318103520 0x7ff3181039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:05.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.237+0000 7ff31f1b9700 1 -- 172.21.15.100:0/2460565028 >> 172.21.15.100:0/2460565028 conn(0x7ff31807a570 msgr2=0x7ff31807a980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:05.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.237+0000 7ff31f1b9700 1 -- 172.21.15.100:0/2460565028 shutdown_connections 2024-05-10T11:28:05.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.237+0000 7ff31f1b9700 1 -- 172.21.15.100:0/2460565028 wait complete. 2024-05-10T11:28:05.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.238+0000 7ff31f1b9700 1 Processor -- start 2024-05-10T11:28:05.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.238+0000 7ff31f1b9700 1 -- start start 2024-05-10T11:28:05.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.239+0000 7ff31f1b9700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff318103520 0x7ff3181a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:05.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.239+0000 7ff31f1b9700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff318103ee0 0x7ff3181a1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:05.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.239+0000 7ff31f1b9700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff318109530 0x7ff31819bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:05.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.239+0000 7ff31f1b9700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff318113520 con 0x7ff318103520 2024-05-10T11:28:05.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.239+0000 7ff31f1b9700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff3181133a0 con 0x7ff318103ee0 2024-05-10T11:28:05.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.239+0000 7ff31f1b9700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff3181136a0 con 0x7ff318109530 2024-05-10T11:28:05.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.239+0000 7ff31cf55700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff318103520 0x7ff3181a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:05.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.239+0000 7ff317fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff318103ee0 0x7ff3181a1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:05.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.239+0000 7ff31cf55700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff318103520 0x7ff3181a1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60472/0 (socket says 172.21.15.100:60472) 2024-05-10T11:28:05.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.239+0000 7ff31cf55700 1 -- 172.21.15.100:0/1543886937 learned_addr learned my addr 172.21.15.100:0/1543886937 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:28:05.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.239+0000 7ff31d756700 1 --2- 172.21.15.100:0/1543886937 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff318109530 0x7ff31819bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:05.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.239+0000 7ff317fff700 1 -- 172.21.15.100:0/1543886937 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff318109530 msgr2=0x7ff31819bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:05.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.239+0000 7ff317fff700 1 --2- 172.21.15.100:0/1543886937 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff318109530 0x7ff31819bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:05.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.239+0000 7ff317fff700 1 -- 172.21.15.100:0/1543886937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff318103520 msgr2=0x7ff3181a1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:05.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.239+0000 7ff317fff700 1 --2- 172.21.15.100:0/1543886937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff318103520 0x7ff3181a1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:05.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.239+0000 7ff317fff700 1 -- 172.21.15.100:0/1543886937 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff31819c220 con 0x7ff318103ee0 2024-05-10T11:28:05.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.240+0000 7ff31cf55700 1 --2- 172.21.15.100:0/1543886937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff318103520 0x7ff3181a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:28:05.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.240+0000 7ff317fff700 1 --2- 172.21.15.100:0/1543886937 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff318103ee0 0x7ff3181a1e40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff30400e9f0 tx=0x7ff30400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:05.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.240+0000 7ff315ffb700 1 -- 172.21.15.100:0/1543886937 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30400cbd0 con 0x7ff318103ee0 2024-05-10T11:28:05.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.240+0000 7ff315ffb700 1 -- 172.21.15.100:0/1543886937 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff3040044b0 con 0x7ff318103ee0 2024-05-10T11:28:05.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.240+0000 7ff315ffb700 1 -- 172.21.15.100:0/1543886937 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff304005020 con 0x7ff318103ee0 2024-05-10T11:28:05.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.240+0000 7ff31f1b9700 1 -- 172.21.15.100:0/1543886937 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff31819c510 con 0x7ff318103ee0 2024-05-10T11:28:05.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.240+0000 7ff31f1b9700 1 -- 172.21.15.100:0/1543886937 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3181a87d0 con 0x7ff318103ee0 2024-05-10T11:28:05.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.243+0000 7ff315ffb700 1 -- 172.21.15.100:0/1543886937 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7ff3040051c0 con 0x7ff318103ee0 2024-05-10T11:28:05.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.243+0000 7ff315ffb700 1 --2- 172.21.15.100:0/1543886937 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff3080b2880 0x7ff3080b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:05.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.243+0000 7ff31cf55700 1 --2- 172.21.15.100:0/1543886937 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff3080b2880 0x7ff3080b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:05.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.243+0000 7ff315ffb700 1 -- 172.21.15.100:0/1543886937 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7ff3040e24f0 con 0x7ff318103ee0 2024-05-10T11:28:05.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.244+0000 7ff31cf55700 1 --2- 172.21.15.100:0/1543886937 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff3080b2880 0x7ff3080b4d40 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff30c009750 tx=0x7ff30c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:05.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.244+0000 7ff329a4cb80 1 Processor -- start 2024-05-10T11:28:05.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.245+0000 7ff329a4cb80 1 -- start start 2024-05-10T11:28:05.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.245+0000 7ff329a4cb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5653085b84d0 0x5653085b88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:05.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.245+0000 7ff329a4cb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5653085b8df0 0x5653085bc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:05.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.245+0000 7ff329a4cb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653085bcc80 0x5653085bd100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:05.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.245+0000 7ff329a4cb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56530850a860 con 0x5653085bcc80 2024-05-10T11:28:05.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.245+0000 7ff329a4cb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5653084a97c0 con 0x5653085b8df0 2024-05-10T11:28:05.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.245+0000 7ff329a4cb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x565308506820 con 0x5653085b84d0 2024-05-10T11:28:05.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.245+0000 7ff31d756700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653085bcc80 0x5653085bd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:05.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.245+0000 7ff31cf55700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5653085b84d0 0x5653085b88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:05.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.245+0000 7ff31d756700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653085bcc80 0x5653085bd100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60484/0 (socket says 172.21.15.100:60484) 2024-05-10T11:28:05.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.245+0000 7ff31d756700 1 -- 172.21.15.100:0/3686209162 learned_addr learned my addr 172.21.15.100:0/3686209162 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:28:05.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.245+0000 7ff317fff700 1 --2- 172.21.15.100:0/3686209162 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5653085b8df0 0x5653085bc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:05.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.245+0000 7ff31d756700 1 -- 172.21.15.100:0/3686209162 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5653085b84d0 msgr2=0x5653085b88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:05.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.245+0000 7ff31d756700 1 --2- 172.21.15.100:0/3686209162 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5653085b84d0 0x5653085b88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:05.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.245+0000 7ff31d756700 1 -- 172.21.15.100:0/3686209162 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5653085b8df0 msgr2=0x5653085bc6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:05.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.245+0000 7ff31d756700 1 --2- 172.21.15.100:0/3686209162 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5653085b8df0 0x5653085bc6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:05.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.245+0000 7ff31d756700 1 -- 172.21.15.100:0/3686209162 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653084578d0 con 0x5653085bcc80 2024-05-10T11:28:05.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.245+0000 7ff31cf55700 1 --2- 172.21.15.100:0/3686209162 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5653085b84d0 0x5653085b88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:28:05.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.246+0000 7ff31d756700 1 --2- 172.21.15.100:0/3686209162 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653085bcc80 0x5653085bd100 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7ff310009460 tx=0x7ff310009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d10076bca6e82edc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:05.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.246+0000 7ff302ffd700 1 -- 172.21.15.100:0/3686209162 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff310013810 con 0x5653085bcc80 2024-05-10T11:28:05.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.246+0000 7ff302ffd700 1 -- 172.21.15.100:0/3686209162 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff310013e40 con 0x5653085bcc80 2024-05-10T11:28:05.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.246+0000 7ff302ffd700 1 -- 172.21.15.100:0/3686209162 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff310015d70 con 0x5653085bcc80 2024-05-10T11:28:05.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.247+0000 7ff329a4cb80 1 -- 172.21.15.100:0/3686209162 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653085bcc80 msgr2=0x5653085bd100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:05.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.247+0000 7ff329a4cb80 1 --2- 172.21.15.100:0/3686209162 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653085bcc80 0x5653085bd100 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7ff310009460 tx=0x7ff310009920 comp rx=0 tx=0).stop 2024-05-10T11:28:05.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.247+0000 7ff329a4cb80 1 -- 172.21.15.100:0/3686209162 shutdown_connections 2024-05-10T11:28:05.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.247+0000 7ff329a4cb80 1 --2- 172.21.15.100:0/3686209162 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653085bcc80 0x5653085bd100 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:05.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.247+0000 7ff329a4cb80 1 --2- 172.21.15.100:0/3686209162 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5653085b84d0 0x5653085b88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:05.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.247+0000 7ff329a4cb80 1 --2- 172.21.15.100:0/3686209162 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5653085b8df0 0x5653085bc6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:05.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.247+0000 7ff329a4cb80 1 -- 172.21.15.100:0/3686209162 >> 172.21.15.100:0/3686209162 conn(0x5653084e4100 msgr2=0x5653085b2c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:05.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.247+0000 7ff329a4cb80 1 -- 172.21.15.100:0/3686209162 shutdown_connections 2024-05-10T11:28:05.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.247+0000 7ff329a4cb80 1 -- 172.21.15.100:0/3686209162 wait complete. 2024-05-10T11:28:05.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.247+0000 7ff329a4cb80 1 -- start start 2024-05-10T11:28:05.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.247+0000 7ff329a4cb80 1 Processor -- start 2024-05-10T11:28:05.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.248+0000 7ff329a4cb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5653085b84d0 0x5653085c4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:05.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.248+0000 7ff329a4cb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653085b8df0 0x5653085c1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:05.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.248+0000 7ff329a4cb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5653085bcc80 0x5653085c1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:05.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.248+0000 7ff329a4cb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5653085c4e90 con 0x5653085b8df0 2024-05-10T11:28:05.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.248+0000 7ff329a4cb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x565308470c30 con 0x5653085bcc80 2024-05-10T11:28:05.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.248+0000 7ff329a4cb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5653085c5010 con 0x5653085b84d0 2024-05-10T11:28:05.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.248+0000 7ff329a4cb80 10 client.0 _ll_register_callbacks cb 0x5653084a2920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:28:05.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.248+0000 7ff317fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653085b8df0 0x5653085c1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:05.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.248+0000 7ff31d756700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5653085bcc80 0x5653085c1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:05.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.248+0000 7ff317fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653085b8df0 0x5653085c1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60496/0 (socket says 172.21.15.100:60496) 2024-05-10T11:28:05.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.248+0000 7ff317fff700 1 -- 172.21.15.100:0/423512986 learned_addr learned my addr 172.21.15.100:0/423512986 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:28:05.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.248+0000 7ff31cf55700 1 --2- 172.21.15.100:0/423512986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5653085b84d0 0x5653085c4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:05.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.248+0000 7ff317fff700 1 -- 172.21.15.100:0/423512986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5653085b84d0 msgr2=0x5653085c4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:05.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.248+0000 7ff317fff700 1 --2- 172.21.15.100:0/423512986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5653085b84d0 0x5653085c4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:05.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.248+0000 7ff317fff700 1 -- 172.21.15.100:0/423512986 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5653085bcc80 msgr2=0x5653085c1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:05.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.248+0000 7ff317fff700 1 --2- 172.21.15.100:0/423512986 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5653085bcc80 0x5653085c1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:05.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.248+0000 7ff317fff700 1 -- 172.21.15.100:0/423512986 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653084a6690 con 0x5653085b8df0 2024-05-10T11:28:05.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.248+0000 7ff31d756700 1 --2- 172.21.15.100:0/423512986 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5653085bcc80 0x5653085c1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:28:05.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.248+0000 7ff317fff700 1 --2- 172.21.15.100:0/423512986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653085b8df0 0x5653085c1420 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7ff3040dfce0 tx=0x7ff3040057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8cecf1edfc81a6e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:05.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.248+0000 7ff2f7fff700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:28:05.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.249+0000 7ff2f7fff700 1 -- 172.21.15.100:0/423512986 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3040c3060 con 0x5653085b8df0 2024-05-10T11:28:05.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.249+0000 7ff2f7fff700 1 -- 172.21.15.100:0/423512986 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7ff3040b2180 con 0x5653085b8df0 2024-05-10T11:28:05.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.249+0000 7ff329a4cb80 1 -- 172.21.15.100:0/423512986 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x565308584650 con 0x5653085b8df0 2024-05-10T11:28:05.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.249+0000 7ff2f7fff700 1 -- 172.21.15.100:0/423512986 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff30401c040 con 0x5653085b8df0 2024-05-10T11:28:05.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.249+0000 7ff329a4cb80 10 client.17331 fetch_fsmap learned FSMap version 31 2024-05-10T11:28:05.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.249+0000 7ff329a4cb80 1 -- 172.21.15.100:0/423512986 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5653085cf6b0 con 0x5653085b8df0 2024-05-10T11:28:05.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.249+0000 7ff2f7fff700 1 -- 172.21.15.100:0/423512986 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x565308584650 con 0x5653085b8df0 2024-05-10T11:28:05.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.249+0000 7ff2f7fff700 1 -- 172.21.15.100:0/423512986 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7ff3040a8a10 con 0x5653085b8df0 2024-05-10T11:28:05.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.249+0000 7ff329a4cb80 10 client.17331 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:28:05.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.249+0000 7ff329a4cb80 10 client.17331 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-10T11:28:05.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.249+0000 7ff329a4cb80 20 client.17331 populate_metadata read hostname 'smithi100' 2024-05-10T11:28:05.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.249+0000 7ff329a4cb80 1 --2- 172.21.15.100:0/423512986 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x5653085d1cf0 0x5653085d40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:05.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.249+0000 7ff329a4cb80 4 client.17331 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:28:05.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.249+0000 7ff329a4cb80 1 -- 172.21.15.100:0/423512986 --> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565308518240 con 0x5653085d1cf0 2024-05-10T11:28:05.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.250+0000 7ff31cf55700 1 --2- 172.21.15.100:0/423512986 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x5653085d1cf0 0x5653085d40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:05.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.250+0000 7ff31cf55700 1 --2- 172.21.15.100:0/423512986 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x5653085d1cf0 0x5653085d40e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bad625d99749528 server_cookie=251c2e1b25bdce2b in_seq=0 out_seq=0 2024-05-10T11:28:05.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.250+0000 7ff2f7fff700 10 client.17331 ms_handle_connect on v2:172.21.15.188:6836/1943342620 2024-05-10T11:28:05.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.251+0000 7ff2f7fff700 1 -- 172.21.15.100:0/423512986 <== mds.2 v2:172.21.15.188:6836/1943342620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x565308518240 con 0x5653085d1cf0 2024-05-10T11:28:05.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.251+0000 7ff2f7fff700 10 client.17331 handle_command_reply: tid=0 2024-05-10T11:28:05.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.251+0000 7ff329a4cb80 1 client.17331 shutdown 2024-05-10T11:28:05.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.251+0000 7ff329a4cb80 1 -- 172.21.15.100:0/423512986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653085b8df0 msgr2=0x5653085c1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:05.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.251+0000 7ff329a4cb80 1 --2- 172.21.15.100:0/423512986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653085b8df0 0x5653085c1420 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7ff3040dfce0 tx=0x7ff3040057b0 comp rx=0 tx=0).stop 2024-05-10T11:28:05.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.251+0000 7ff329a4cb80 1 -- 172.21.15.100:0/423512986 shutdown_connections 2024-05-10T11:28:05.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.251+0000 7ff329a4cb80 1 --2- 172.21.15.100:0/423512986 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x5653085d1cf0 0x5653085d40e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:05.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.251+0000 7ff2f7fff700 0 client.17331 ms_handle_reset on v2:172.21.15.188:6836/1943342620 2024-05-10T11:28:05.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.251+0000 7ff329a4cb80 1 --2- 172.21.15.100:0/423512986 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5653085bcc80 0x5653085c1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:05.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.251+0000 7ff329a4cb80 1 --2- 172.21.15.100:0/423512986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5653085b84d0 0x5653085c4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:05.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.251+0000 7ff329a4cb80 1 --2- 172.21.15.100:0/423512986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653085b8df0 0x5653085c1420 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:05.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.251+0000 7ff329a4cb80 1 -- 172.21.15.100:0/423512986 >> 172.21.15.100:0/423512986 conn(0x5653084e4100 msgr2=0x5653085bd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:05.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.251+0000 7ff329a4cb80 1 -- 172.21.15.100:0/423512986 shutdown_connections 2024-05-10T11:28:05.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.251+0000 7ff329a4cb80 1 -- 172.21.15.100:0/423512986 wait complete. 2024-05-10T11:28:05.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.252+0000 7ff329a4cb80 20 client.17331 trim_cache size 0 max 16384 2024-05-10T11:28:05.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.269+0000 7ff329a4cb80 1 Processor -- start 2024-05-10T11:28:05.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.269+0000 7ff329a4cb80 1 -- start start 2024-05-10T11:28:05.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.269+0000 7ff329a4cb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5653085b84d0 0x5653085cdf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:05.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.269+0000 7ff329a4cb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653085b8df0 0x5653085ad5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:05.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.269+0000 7ff329a4cb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5653085bcc80 0x5653085ba5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:05.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.269+0000 7ff329a4cb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff3181b3b60 con 0x5653085b8df0 2024-05-10T11:28:05.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.269+0000 7ff329a4cb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff3181b13f0 con 0x5653085bcc80 2024-05-10T11:28:05.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.269+0000 7ff329a4cb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff3181af240 con 0x5653085b84d0 2024-05-10T11:28:05.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.269+0000 7ff317fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653085b8df0 0x5653085ad5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:05.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.269+0000 7ff31cf55700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5653085b84d0 0x5653085cdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:05.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.269+0000 7ff317fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653085b8df0 0x5653085ad5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60508/0 (socket says 172.21.15.100:60508) 2024-05-10T11:28:05.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.269+0000 7ff317fff700 1 -- 172.21.15.100:0/992747309 learned_addr learned my addr 172.21.15.100:0/992747309 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:28:05.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.270+0000 7ff31d756700 1 --2- 172.21.15.100:0/992747309 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5653085bcc80 0x5653085ba5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:05.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.270+0000 7ff317fff700 1 -- 172.21.15.100:0/992747309 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5653085b84d0 msgr2=0x5653085cdf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:05.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.270+0000 7ff317fff700 1 --2- 172.21.15.100:0/992747309 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5653085b84d0 0x5653085cdf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:05.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.270+0000 7ff317fff700 1 -- 172.21.15.100:0/992747309 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5653085bcc80 msgr2=0x5653085ba5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:05.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.270+0000 7ff317fff700 1 --2- 172.21.15.100:0/992747309 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5653085bcc80 0x5653085ba5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:05.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.270+0000 7ff317fff700 1 -- 172.21.15.100:0/992747309 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653085ac710 con 0x5653085b8df0 2024-05-10T11:28:05.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.270+0000 7ff31d756700 1 --2- 172.21.15.100:0/992747309 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5653085bcc80 0x5653085ba5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:28:05.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.270+0000 7ff317fff700 1 --2- 172.21.15.100:0/992747309 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653085b8df0 0x5653085ad5a0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7ff3040a94b0 tx=0x7ff3040e2480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4c67a5451fc6724 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:05.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.270+0000 7ff3027fc700 1 -- 172.21.15.100:0/992747309 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff304004750 con 0x5653085b8df0 2024-05-10T11:28:05.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.270+0000 7ff3027fc700 1 -- 172.21.15.100:0/992747309 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff3040a92c0 con 0x5653085b8df0 2024-05-10T11:28:05.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.270+0000 7ff3027fc700 1 -- 172.21.15.100:0/992747309 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3040b2610 con 0x5653085b8df0 2024-05-10T11:28:05.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.271+0000 7ff329a4cb80 1 -- 172.21.15.100:0/992747309 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653085b8df0 msgr2=0x5653085ad5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:05.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.271+0000 7ff329a4cb80 1 --2- 172.21.15.100:0/992747309 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653085b8df0 0x5653085ad5a0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7ff3040a94b0 tx=0x7ff3040e2480 comp rx=0 tx=0).stop 2024-05-10T11:28:05.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.271+0000 7ff329a4cb80 1 -- 172.21.15.100:0/992747309 shutdown_connections 2024-05-10T11:28:05.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.271+0000 7ff329a4cb80 1 --2- 172.21.15.100:0/992747309 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5653085bcc80 0x5653085ba5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:05.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.271+0000 7ff329a4cb80 1 --2- 172.21.15.100:0/992747309 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5653085b84d0 0x5653085cdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:05.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.271+0000 7ff329a4cb80 1 --2- 172.21.15.100:0/992747309 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653085b8df0 0x5653085ad5a0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:05.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.271+0000 7ff329a4cb80 1 -- 172.21.15.100:0/992747309 >> 172.21.15.100:0/992747309 conn(0x5653084e4100 msgr2=0x5653085ace90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:05.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.271+0000 7ff329a4cb80 1 -- 172.21.15.100:0/992747309 shutdown_connections 2024-05-10T11:28:05.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.271+0000 7ff329a4cb80 1 -- 172.21.15.100:0/992747309 wait complete. 2024-05-10T11:28:05.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.271+0000 7ff329a4cb80 1 -- start start 2024-05-10T11:28:05.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.272+0000 7ff329a4cb80 1 Processor -- start 2024-05-10T11:28:05.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.272+0000 7ff329a4cb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5653085b84d0 0x5653085ca790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:05.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.272+0000 7ff329a4cb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5653085b8df0 0x5653085c5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:05.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.272+0000 7ff329a4cb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653085bcc80 0x5653085c60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:05.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.272+0000 7ff329a4cb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff3181af8b0 con 0x5653085bcc80 2024-05-10T11:28:05.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.272+0000 7ff329a4cb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff3181b0d20 con 0x5653085b84d0 2024-05-10T11:28:05.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.272+0000 7ff329a4cb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff3181aff80 con 0x5653085b8df0 2024-05-10T11:28:05.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.272+0000 7ff329a4cb80 10 client.0 _ll_register_callbacks cb 0x5653084a2920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:28:05.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.272+0000 7ff31d756700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653085bcc80 0x5653085c60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:05.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.272+0000 7ff31d756700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653085bcc80 0x5653085c60f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60514/0 (socket says 172.21.15.100:60514) 2024-05-10T11:28:05.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.272+0000 7ff31d756700 1 -- 172.21.15.100:0/3691833603 learned_addr learned my addr 172.21.15.100:0/3691833603 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:28:05.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.272+0000 7ff31cf55700 1 --2- 172.21.15.100:0/3691833603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5653085b84d0 0x5653085ca790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:05.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.272+0000 7ff317fff700 1 --2- 172.21.15.100:0/3691833603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5653085b8df0 0x5653085c5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:05.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.273+0000 7ff31d756700 1 -- 172.21.15.100:0/3691833603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5653085b8df0 msgr2=0x5653085c5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:05.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.273+0000 7ff31d756700 1 --2- 172.21.15.100:0/3691833603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5653085b8df0 0x5653085c5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:05.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.273+0000 7ff31d756700 1 -- 172.21.15.100:0/3691833603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5653085b84d0 msgr2=0x5653085ca790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:05.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.273+0000 7ff31d756700 1 --2- 172.21.15.100:0/3691833603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5653085b84d0 0x5653085ca790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:05.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.273+0000 7ff31d756700 1 -- 172.21.15.100:0/3691833603 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653085c6900 con 0x5653085bcc80 2024-05-10T11:28:05.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.273+0000 7ff31d756700 1 --2- 172.21.15.100:0/3691833603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653085bcc80 0x5653085c60f0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7ff310009460 tx=0x7ff310004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1927ef60b53fc91 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:05.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.273+0000 7ff2f7fff700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:28:05.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.273+0000 7ff2f7fff700 1 -- 172.21.15.100:0/3691833603 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff310013a20 con 0x5653085bcc80 2024-05-10T11:28:05.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.273+0000 7ff2f7fff700 1 -- 172.21.15.100:0/3691833603 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7ff310013bc0 con 0x5653085bcc80 2024-05-10T11:28:05.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.273+0000 7ff329a4cb80 1 -- 172.21.15.100:0/3691833603 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3040a92c0 con 0x5653085bcc80 2024-05-10T11:28:05.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.273+0000 7ff2f7fff700 1 -- 172.21.15.100:0/3691833603 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff310031050 con 0x5653085bcc80 2024-05-10T11:28:05.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.273+0000 7ff329a4cb80 10 client.17337 fetch_fsmap learned FSMap version 31 2024-05-10T11:28:05.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.273+0000 7ff329a4cb80 1 -- 172.21.15.100:0/3691833603 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56530863c4e0 con 0x5653085bcc80 2024-05-10T11:28:05.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.273+0000 7ff2f7fff700 1 -- 172.21.15.100:0/3691833603 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff3040a92c0 con 0x5653085bcc80 2024-05-10T11:28:05.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.274+0000 7ff2f7fff700 1 -- 172.21.15.100:0/3691833603 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7ff31001f8b0 con 0x5653085bcc80 2024-05-10T11:28:05.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.274+0000 7ff329a4cb80 10 client.17337 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:28:05.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.274+0000 7ff329a4cb80 10 client.17337 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-10T11:28:05.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.274+0000 7ff329a4cb80 20 client.17337 populate_metadata read hostname 'smithi100' 2024-05-10T11:28:05.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.274+0000 7ff329a4cb80 1 --2- 172.21.15.100:0/3691833603 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x5653085d1cf0 0x56530863ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:05.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.274+0000 7ff329a4cb80 4 client.17337 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:28:05.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.274+0000 7ff329a4cb80 1 -- 172.21.15.100:0/3691833603 --> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5653085122e0 con 0x5653085d1cf0 2024-05-10T11:28:05.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.274+0000 7ff31cf55700 1 --2- 172.21.15.100:0/3691833603 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x5653085d1cf0 0x56530863ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:05.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.274+0000 7ff31cf55700 1 --2- 172.21.15.100:0/3691833603 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x5653085d1cf0 0x56530863ea60 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1e98ae28f941b3e3 server_cookie=b5ec48b27f8322e5 in_seq=0 out_seq=0 2024-05-10T11:28:05.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.274+0000 7ff2f7fff700 10 client.17337 ms_handle_connect on v2:172.21.15.188:6836/1943342620 2024-05-10T11:28:05.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.275+0000 7ff2f7fff700 1 -- 172.21.15.100:0/3691833603 <== mds.2 v2:172.21.15.188:6836/1943342620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5653085122e0 con 0x5653085d1cf0 2024-05-10T11:28:05.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.275+0000 7ff2f7fff700 10 client.17337 handle_command_reply: tid=0 2024-05-10T11:28:05.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.275+0000 7ff329a4cb80 1 client.17337 shutdown 2024-05-10T11:28:05.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.275+0000 7ff329a4cb80 1 -- 172.21.15.100:0/3691833603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653085bcc80 msgr2=0x5653085c60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:05.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.275+0000 7ff329a4cb80 1 --2- 172.21.15.100:0/3691833603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653085bcc80 0x5653085c60f0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7ff310009460 tx=0x7ff310004f90 comp rx=0 tx=0).stop 2024-05-10T11:28:05.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.275+0000 7ff329a4cb80 1 -- 172.21.15.100:0/3691833603 shutdown_connections 2024-05-10T11:28:05.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.275+0000 7ff329a4cb80 1 --2- 172.21.15.100:0/3691833603 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x5653085d1cf0 0x56530863ea60 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:05.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.275+0000 7ff2f7fff700 0 client.17337 ms_handle_reset on v2:172.21.15.188:6836/1943342620 2024-05-10T11:28:05.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.275+0000 7ff329a4cb80 1 --2- 172.21.15.100:0/3691833603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653085bcc80 0x5653085c60f0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:05.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.275+0000 7ff329a4cb80 1 --2- 172.21.15.100:0/3691833603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5653085b84d0 0x5653085ca790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:05.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.275+0000 7ff329a4cb80 1 --2- 172.21.15.100:0/3691833603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5653085b8df0 0x5653085c5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:05.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.275+0000 7ff329a4cb80 1 -- 172.21.15.100:0/3691833603 >> 172.21.15.100:0/3691833603 conn(0x5653084e4100 msgr2=0x5653085ad720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:05.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.275+0000 7ff329a4cb80 1 -- 172.21.15.100:0/3691833603 shutdown_connections 2024-05-10T11:28:05.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.275+0000 7ff329a4cb80 1 -- 172.21.15.100:0/3691833603 wait complete. 2024-05-10T11:28:05.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.275+0000 7ff329a4cb80 20 client.17337 trim_cache size 0 max 16384 2024-05-10T11:28:05.280 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:28:05.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.276+0000 7ff31f1b9700 1 -- 172.21.15.100:0/1543886937 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff3080b2880 msgr2=0x7ff3080b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:05.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.276+0000 7ff31f1b9700 1 --2- 172.21.15.100:0/1543886937 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff3080b2880 0x7ff3080b4d40 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff30c009750 tx=0x7ff30c005960 comp rx=0 tx=0).stop 2024-05-10T11:28:05.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.276+0000 7ff31f1b9700 1 -- 172.21.15.100:0/1543886937 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff318103ee0 msgr2=0x7ff3181a1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:05.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.276+0000 7ff31f1b9700 1 --2- 172.21.15.100:0/1543886937 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff318103ee0 0x7ff3181a1e40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff30400e9f0 tx=0x7ff30400ed00 comp rx=0 tx=0).stop 2024-05-10T11:28:05.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.276+0000 7ff31f1b9700 1 -- 172.21.15.100:0/1543886937 shutdown_connections 2024-05-10T11:28:05.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.276+0000 7ff31f1b9700 1 --2- 172.21.15.100:0/1543886937 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff3080b2880 0x7ff3080b4d40 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:05.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.276+0000 7ff31f1b9700 1 --2- 172.21.15.100:0/1543886937 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff318109530 0x7ff31819bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:05.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.276+0000 7ff31f1b9700 1 --2- 172.21.15.100:0/1543886937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff318103520 0x7ff3181a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:05.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.276+0000 7ff31f1b9700 1 --2- 172.21.15.100:0/1543886937 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff318103ee0 0x7ff3181a1e40 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:05.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.276+0000 7ff31f1b9700 1 -- 172.21.15.100:0/1543886937 >> 172.21.15.100:0/1543886937 conn(0x7ff31807a570 msgr2=0x7ff31810e950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:05.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.276+0000 7ff31f1b9700 1 -- 172.21.15.100:0/1543886937 shutdown_connections 2024-05-10T11:28:05.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:05.276+0000 7ff31f1b9700 1 -- 172.21.15.100:0/1543886937 wait complete. 2024-05-10T11:28:05.978 INFO:tasks.workunit.client.0.smithi100.stdout: 1 247292 2.73 MB/sec execute 360 sec latency 65.309 ms 2024-05-10T11:28:06.110 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-10T11:28:06.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:28:05 smithi178 ceph-mon[33359]: pgmap v711: 129 pgs: 129 active+clean; 102 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 1.4 MiB/s wr, 149 op/s 2024-05-10T11:28:06.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:28:05 smithi188 ceph-mon[34075]: pgmap v711: 129 pgs: 129 active+clean; 102 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 1.4 MiB/s wr, 149 op/s 2024-05-10T11:28:06.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:28:05 smithi100 ceph-mon[30296]: pgmap v711: 129 pgs: 129 active+clean; 102 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 1.4 MiB/s wr, 149 op/s 2024-05-10T11:28:06.381 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:28:06.979 INFO:tasks.workunit.client.0.smithi100.stdout: 1 248076 2.74 MB/sec execute 361 sec latency 12.951 ms 2024-05-10T11:28:07.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:28:07 smithi178 ceph-mon[33359]: pgmap v712: 129 pgs: 129 active+clean; 100 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 1.3 MiB/s wr, 133 op/s 2024-05-10T11:28:07.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:28:07 smithi188 ceph-mon[34075]: pgmap v712: 129 pgs: 129 active+clean; 100 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 1.3 MiB/s wr, 133 op/s 2024-05-10T11:28:07.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:28:07 smithi100 ceph-mon[30296]: pgmap v712: 129 pgs: 129 active+clean; 100 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 1.3 MiB/s wr, 133 op/s 2024-05-10T11:28:07.978 INFO:tasks.workunit.client.0.smithi100.stdout: 1 248497 2.75 MB/sec execute 362 sec latency 13.814 ms 2024-05-10T11:28:08.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.526+0000 7ff47b7cb700 1 -- 172.21.15.100:0/3431924893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff474107760 msgr2=0x7ff47410f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:08.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.526+0000 7ff47b7cb700 1 --2- 172.21.15.100:0/3431924893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff474107760 0x7ff47410f2e0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7ff46c0099c0 tx=0x7ff46c009cd0 comp rx=0 tx=0).stop 2024-05-10T11:28:08.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.526+0000 7ff47b7cb700 1 -- 172.21.15.100:0/3431924893 shutdown_connections 2024-05-10T11:28:08.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.526+0000 7ff47b7cb700 1 --2- 172.21.15.100:0/3431924893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff474107760 0x7ff47410f2e0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:08.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.526+0000 7ff47b7cb700 1 --2- 172.21.15.100:0/3431924893 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff474102e00 0x7ff4741031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:08.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.526+0000 7ff47b7cb700 1 --2- 172.21.15.100:0/3431924893 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4741037b0 0x7ff474107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:08.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.526+0000 7ff47b7cb700 1 -- 172.21.15.100:0/3431924893 >> 172.21.15.100:0/3431924893 conn(0x7ff474078a10 msgr2=0x7ff4740ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:08.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.526+0000 7ff47b7cb700 1 -- 172.21.15.100:0/3431924893 shutdown_connections 2024-05-10T11:28:08.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.526+0000 7ff47b7cb700 1 -- 172.21.15.100:0/3431924893 wait complete. 2024-05-10T11:28:08.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.527+0000 7ff47b7cb700 1 Processor -- start 2024-05-10T11:28:08.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.527+0000 7ff47b7cb700 1 -- start start 2024-05-10T11:28:08.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.527+0000 7ff47b7cb700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff474102e00 0x7ff47419a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:08.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.528+0000 7ff47b7cb700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4741037b0 0x7ff47419ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:08.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.528+0000 7ff47b7cb700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff474107760 0x7ff4741a5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:08.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.528+0000 7ff47b7cb700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff4741123e0 con 0x7ff474107760 2024-05-10T11:28:08.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.528+0000 7ff47b7cb700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff474112260 con 0x7ff4741037b0 2024-05-10T11:28:08.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.528+0000 7ff47b7cb700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff474112560 con 0x7ff474102e00 2024-05-10T11:28:08.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.528+0000 7ff479d68700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff474107760 0x7ff4741a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:08.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.528+0000 7ff479567700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff474102e00 0x7ff47419a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:08.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.528+0000 7ff479d68700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff474107760 0x7ff4741a5840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44756/0 (socket says 172.21.15.100:44756) 2024-05-10T11:28:08.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.528+0000 7ff479567700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff474102e00 0x7ff47419a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:43598/0 (socket says 172.21.15.100:43598) 2024-05-10T11:28:08.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.528+0000 7ff479d68700 1 -- 172.21.15.100:0/456901610 learned_addr learned my addr 172.21.15.100:0/456901610 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:28:08.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.528+0000 7ff478d66700 1 --2- 172.21.15.100:0/456901610 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4741037b0 0x7ff47419ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:08.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.528+0000 7ff479d68700 1 -- 172.21.15.100:0/456901610 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff474102e00 msgr2=0x7ff47419a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:08.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.528+0000 7ff479d68700 1 --2- 172.21.15.100:0/456901610 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff474102e00 0x7ff47419a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:08.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.528+0000 7ff479d68700 1 -- 172.21.15.100:0/456901610 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4741037b0 msgr2=0x7ff47419ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:08.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.528+0000 7ff479d68700 1 --2- 172.21.15.100:0/456901610 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4741037b0 0x7ff47419ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:08.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.528+0000 7ff479d68700 1 -- 172.21.15.100:0/456901610 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4741a5d80 con 0x7ff474107760 2024-05-10T11:28:08.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.528+0000 7ff479d68700 1 --2- 172.21.15.100:0/456901610 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff474107760 0x7ff4741a5840 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7ff46c005910 tx=0x7ff46c0055f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:08.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.528+0000 7ff4727fc700 1 -- 172.21.15.100:0/456901610 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff46c026070 con 0x7ff474107760 2024-05-10T11:28:08.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.529+0000 7ff47b7cb700 1 -- 172.21.15.100:0/456901610 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4741a5f50 con 0x7ff474107760 2024-05-10T11:28:08.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.529+0000 7ff4727fc700 1 -- 172.21.15.100:0/456901610 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff46c00f460 con 0x7ff474107760 2024-05-10T11:28:08.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.529+0000 7ff4727fc700 1 -- 172.21.15.100:0/456901610 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff46c009e60 con 0x7ff474107760 2024-05-10T11:28:08.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.529+0000 7ff47b7cb700 1 -- 172.21.15.100:0/456901610 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4741a63a0 con 0x7ff474107760 2024-05-10T11:28:08.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.531+0000 7ff4727fc700 1 -- 172.21.15.100:0/456901610 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7ff46c029070 con 0x7ff474107760 2024-05-10T11:28:08.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.532+0000 7ff4727fc700 1 --2- 172.21.15.100:0/456901610 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff45c0b2880 0x7ff45c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:08.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.532+0000 7ff479567700 1 --2- 172.21.15.100:0/456901610 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff45c0b2880 0x7ff45c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:08.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.532+0000 7ff4727fc700 1 -- 172.21.15.100:0/456901610 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7ff46c0e2d00 con 0x7ff474107760 2024-05-10T11:28:08.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.532+0000 7ff479567700 1 --2- 172.21.15.100:0/456901610 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff45c0b2880 0x7ff45c0b4d40 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff4640060b0 tx=0x7ff46400c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:08.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.533+0000 7ff48605eb80 1 Processor -- start 2024-05-10T11:28:08.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.533+0000 7ff48605eb80 1 -- start start 2024-05-10T11:28:08.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.533+0000 7ff48605eb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c10c358590 0x55c10c358970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:08.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.533+0000 7ff48605eb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c10c358eb0 0x55c10c352c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:08.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.533+0000 7ff48605eb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c10c3531b0 0x55c10c353610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:08.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.533+0000 7ff48605eb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55c10c2a0940 con 0x55c10c358eb0 2024-05-10T11:28:08.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.533+0000 7ff48605eb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c10c205da0 con 0x55c10c3531b0 2024-05-10T11:28:08.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.533+0000 7ff48605eb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c10c29c900 con 0x55c10c358590 2024-05-10T11:28:08.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.533+0000 7ff478d66700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c10c358eb0 0x55c10c352c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:08.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.533+0000 7ff478d66700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c10c358eb0 0x55c10c352c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44764/0 (socket says 172.21.15.100:44764) 2024-05-10T11:28:08.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.533+0000 7ff478d66700 1 -- 172.21.15.100:0/2871589868 learned_addr learned my addr 172.21.15.100:0/2871589868 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:28:08.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.533+0000 7ff479567700 1 --2- 172.21.15.100:0/2871589868 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c10c358590 0x55c10c358970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:08.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.533+0000 7ff479d68700 1 --2- 172.21.15.100:0/2871589868 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c10c3531b0 0x55c10c353610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:08.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.533+0000 7ff478d66700 1 -- 172.21.15.100:0/2871589868 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c10c358590 msgr2=0x55c10c358970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:08.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.534+0000 7ff478d66700 1 --2- 172.21.15.100:0/2871589868 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c10c358590 0x55c10c358970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:08.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.534+0000 7ff478d66700 1 -- 172.21.15.100:0/2871589868 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c10c3531b0 msgr2=0x55c10c353610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:08.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.534+0000 7ff478d66700 1 --2- 172.21.15.100:0/2871589868 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c10c3531b0 0x55c10c353610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:08.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.534+0000 7ff478d66700 1 -- 172.21.15.100:0/2871589868 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c10c1ed8d0 con 0x55c10c358eb0 2024-05-10T11:28:08.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.534+0000 7ff479567700 1 --2- 172.21.15.100:0/2871589868 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c10c358590 0x55c10c358970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:28:08.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.534+0000 7ff479d68700 1 --2- 172.21.15.100:0/2871589868 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c10c3531b0 0x55c10c353610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:28:08.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.534+0000 7ff478d66700 1 --2- 172.21.15.100:0/2871589868 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c10c358eb0 0x55c10c352c20 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7ff468009cb0 tx=0x7ff468011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f80a9cc5214e3228 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:08.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.534+0000 7ff45b7fe700 1 -- 172.21.15.100:0/2871589868 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff468011d20 con 0x55c10c358eb0 2024-05-10T11:28:08.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.534+0000 7ff45b7fe700 1 -- 172.21.15.100:0/2871589868 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff4680044e0 con 0x55c10c358eb0 2024-05-10T11:28:08.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.534+0000 7ff45b7fe700 1 -- 172.21.15.100:0/2871589868 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff468005280 con 0x55c10c358eb0 2024-05-10T11:28:08.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.535+0000 7ff48605eb80 1 -- 172.21.15.100:0/2871589868 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c10c358eb0 msgr2=0x55c10c352c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:08.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.535+0000 7ff48605eb80 1 --2- 172.21.15.100:0/2871589868 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c10c358eb0 0x55c10c352c20 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7ff468009cb0 tx=0x7ff468011440 comp rx=0 tx=0).stop 2024-05-10T11:28:08.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.535+0000 7ff48605eb80 1 -- 172.21.15.100:0/2871589868 shutdown_connections 2024-05-10T11:28:08.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.535+0000 7ff48605eb80 1 --2- 172.21.15.100:0/2871589868 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c10c3531b0 0x55c10c353610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:08.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.535+0000 7ff48605eb80 1 --2- 172.21.15.100:0/2871589868 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c10c358590 0x55c10c358970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:08.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.535+0000 7ff48605eb80 1 --2- 172.21.15.100:0/2871589868 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c10c358eb0 0x55c10c352c20 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:08.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.535+0000 7ff48605eb80 1 -- 172.21.15.100:0/2871589868 >> 172.21.15.100:0/2871589868 conn(0x55c10c27ab70 msgr2=0x55c10c3492a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:08.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.535+0000 7ff48605eb80 1 -- 172.21.15.100:0/2871589868 shutdown_connections 2024-05-10T11:28:08.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.535+0000 7ff48605eb80 1 -- 172.21.15.100:0/2871589868 wait complete. 2024-05-10T11:28:08.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.535+0000 7ff48605eb80 1 -- start start 2024-05-10T11:28:08.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.535+0000 7ff48605eb80 1 Processor -- start 2024-05-10T11:28:08.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.536+0000 7ff48605eb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c10c3531b0 0x55c10c35eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:08.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.536+0000 7ff48605eb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c10c358590 0x55c10c35bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:08.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.536+0000 7ff479567700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c10c3531b0 0x55c10c35eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:08.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.536+0000 7ff479567700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c10c3531b0 0x55c10c35eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44774/0 (socket says 172.21.15.100:44774) 2024-05-10T11:28:08.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.536+0000 7ff48605eb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c10c358eb0 0x55c10c35c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:08.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.536+0000 7ff48605eb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55c10c35f8e0 con 0x55c10c3531b0 2024-05-10T11:28:08.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.536+0000 7ff479567700 1 -- 172.21.15.100:0/333540656 learned_addr learned my addr 172.21.15.100:0/333540656 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:28:08.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.536+0000 7ff48605eb80 1 -- 172.21.15.100:0/333540656 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c10c259ed0 con 0x55c10c358eb0 2024-05-10T11:28:08.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.536+0000 7ff48605eb80 1 -- 172.21.15.100:0/333540656 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c10c35fa60 con 0x55c10c358590 2024-05-10T11:28:08.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.536+0000 7ff48605eb80 10 client.0 _ll_register_callbacks cb 0x55c10c2345b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:28:08.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.536+0000 7ff478d66700 1 --2- 172.21.15.100:0/333540656 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c10c358590 0x55c10c35bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:08.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.536+0000 7ff479d68700 1 --2- 172.21.15.100:0/333540656 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c10c358eb0 0x55c10c35c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:08.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.536+0000 7ff479567700 1 -- 172.21.15.100:0/333540656 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c10c358590 msgr2=0x55c10c35bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:08.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.536+0000 7ff479567700 1 --2- 172.21.15.100:0/333540656 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c10c358590 0x55c10c35bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:08.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.536+0000 7ff479567700 1 -- 172.21.15.100:0/333540656 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c10c358eb0 msgr2=0x55c10c35c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:08.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.536+0000 7ff479567700 1 --2- 172.21.15.100:0/333540656 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c10c358eb0 0x55c10c35c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:08.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.536+0000 7ff479567700 1 -- 172.21.15.100:0/333540656 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c10c257760 con 0x55c10c3531b0 2024-05-10T11:28:08.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.536+0000 7ff479d68700 1 --2- 172.21.15.100:0/333540656 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c10c358eb0 0x55c10c35c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:28:08.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.536+0000 7ff479567700 1 --2- 172.21.15.100:0/333540656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c10c3531b0 0x55c10c35eb60 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7ff464016de0 tx=0x7ff464004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1491efdea025696 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:08.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.536+0000 7ff458ff9700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:28:08.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.537+0000 7ff458ff9700 1 -- 172.21.15.100:0/333540656 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff464009e30 con 0x55c10c3531b0 2024-05-10T11:28:08.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.537+0000 7ff458ff9700 1 -- 172.21.15.100:0/333540656 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7ff4640223f0 con 0x55c10c3531b0 2024-05-10T11:28:08.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.537+0000 7ff48605eb80 1 -- 172.21.15.100:0/333540656 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4680044e0 con 0x55c10c3531b0 2024-05-10T11:28:08.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.537+0000 7ff458ff9700 1 -- 172.21.15.100:0/333540656 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff464030050 con 0x55c10c3531b0 2024-05-10T11:28:08.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.537+0000 7ff48605eb80 10 client.17349 fetch_fsmap learned FSMap version 31 2024-05-10T11:28:08.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.537+0000 7ff48605eb80 1 -- 172.21.15.100:0/333540656 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c10c36a1a0 con 0x55c10c3531b0 2024-05-10T11:28:08.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.537+0000 7ff458ff9700 1 -- 172.21.15.100:0/333540656 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff4680044e0 con 0x55c10c3531b0 2024-05-10T11:28:08.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.537+0000 7ff458ff9700 1 -- 172.21.15.100:0/333540656 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7ff4640144e0 con 0x55c10c3531b0 2024-05-10T11:28:08.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.537+0000 7ff48605eb80 10 client.17349 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:28:08.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.537+0000 7ff48605eb80 10 client.17349 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-10T11:28:08.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.537+0000 7ff48605eb80 20 client.17349 populate_metadata read hostname 'smithi100' 2024-05-10T11:28:08.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.537+0000 7ff48605eb80 1 --2- 172.21.15.100:0/333540656 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55c10c36c7e0 0x55c10c36ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:08.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.537+0000 7ff48605eb80 4 client.17349 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:28:08.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.537+0000 7ff48605eb80 1 -- 172.21.15.100:0/333540656 --> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c10c2ae320 con 0x55c10c36c7e0 2024-05-10T11:28:08.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.537+0000 7ff478d66700 1 --2- 172.21.15.100:0/333540656 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55c10c36c7e0 0x55c10c36ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:08.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.538+0000 7ff478d66700 1 --2- 172.21.15.100:0/333540656 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55c10c36c7e0 0x55c10c36ec30 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1604ac1c416cadfa server_cookie=540c623aa1db7cd in_seq=0 out_seq=0 2024-05-10T11:28:08.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.538+0000 7ff458ff9700 10 client.17349 ms_handle_connect on v2:172.21.15.178:6838/3760985404 2024-05-10T11:28:08.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.539+0000 7ff458ff9700 1 -- 172.21.15.100:0/333540656 <== mds.1 v2:172.21.15.178:6838/3760985404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c10c2ae320 con 0x55c10c36c7e0 2024-05-10T11:28:08.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.539+0000 7ff458ff9700 10 client.17349 handle_command_reply: tid=0 2024-05-10T11:28:08.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.539+0000 7ff48605eb80 1 client.17349 shutdown 2024-05-10T11:28:08.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.539+0000 7ff48605eb80 1 -- 172.21.15.100:0/333540656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c10c3531b0 msgr2=0x55c10c35eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:08.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.539+0000 7ff48605eb80 1 --2- 172.21.15.100:0/333540656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c10c3531b0 0x55c10c35eb60 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7ff464016de0 tx=0x7ff464004ad0 comp rx=0 tx=0).stop 2024-05-10T11:28:08.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.539+0000 7ff48605eb80 1 -- 172.21.15.100:0/333540656 shutdown_connections 2024-05-10T11:28:08.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.539+0000 7ff48605eb80 1 --2- 172.21.15.100:0/333540656 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55c10c36c7e0 0x55c10c36ec30 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:08.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.539+0000 7ff48605eb80 1 --2- 172.21.15.100:0/333540656 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c10c358eb0 0x55c10c35c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:08.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.539+0000 7ff48605eb80 1 --2- 172.21.15.100:0/333540656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c10c3531b0 0x55c10c35eb60 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:08.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.539+0000 7ff48605eb80 1 --2- 172.21.15.100:0/333540656 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c10c358590 0x55c10c35bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:08.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.539+0000 7ff48605eb80 1 -- 172.21.15.100:0/333540656 >> 172.21.15.100:0/333540656 conn(0x55c10c27ab70 msgr2=0x55c10c343b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:08.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.539+0000 7ff458ff9700 0 client.17349 ms_handle_reset on v2:172.21.15.178:6838/3760985404 2024-05-10T11:28:08.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.539+0000 7ff48605eb80 1 -- 172.21.15.100:0/333540656 shutdown_connections 2024-05-10T11:28:08.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.539+0000 7ff48605eb80 1 -- 172.21.15.100:0/333540656 wait complete. 2024-05-10T11:28:08.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.539+0000 7ff48605eb80 20 client.17349 trim_cache size 0 max 16384 2024-05-10T11:28:08.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.555+0000 7ff48605eb80 1 Processor -- start 2024-05-10T11:28:08.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.555+0000 7ff48605eb80 1 -- start start 2024-05-10T11:28:08.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.555+0000 7ff48605eb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c10c3531b0 0x55c10c3689a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:08.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.555+0000 7ff48605eb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c10c358590 0x55c10c353590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:08.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.555+0000 7ff48605eb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c10c358eb0 0x55c10c3599d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:08.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.555+0000 7ff48605eb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff4741af670 con 0x55c10c358eb0 2024-05-10T11:28:08.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.555+0000 7ff48605eb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff4741acf00 con 0x55c10c358590 2024-05-10T11:28:08.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.555+0000 7ff48605eb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff4741aad50 con 0x55c10c3531b0 2024-05-10T11:28:08.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.555+0000 7ff479d68700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c10c358eb0 0x55c10c3599d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:08.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.555+0000 7ff479d68700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c10c358eb0 0x55c10c3599d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44790/0 (socket says 172.21.15.100:44790) 2024-05-10T11:28:08.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.555+0000 7ff479d68700 1 -- 172.21.15.100:0/1428418600 learned_addr learned my addr 172.21.15.100:0/1428418600 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:28:08.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.555+0000 7ff479567700 1 --2- 172.21.15.100:0/1428418600 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c10c3531b0 0x55c10c3689a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:08.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.555+0000 7ff478d66700 1 --2- 172.21.15.100:0/1428418600 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c10c358590 0x55c10c353590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:08.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.555+0000 7ff479d68700 1 -- 172.21.15.100:0/1428418600 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c10c3531b0 msgr2=0x55c10c3689a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:08.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.555+0000 7ff479d68700 1 --2- 172.21.15.100:0/1428418600 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c10c3531b0 0x55c10c3689a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:08.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.556+0000 7ff479d68700 1 -- 172.21.15.100:0/1428418600 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c10c358590 msgr2=0x55c10c353590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:08.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.556+0000 7ff479d68700 1 --2- 172.21.15.100:0/1428418600 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c10c358590 0x55c10c353590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:08.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.556+0000 7ff479d68700 1 -- 172.21.15.100:0/1428418600 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c10c34ab70 con 0x55c10c358eb0 2024-05-10T11:28:08.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.556+0000 7ff479567700 1 --2- 172.21.15.100:0/1428418600 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c10c3531b0 0x55c10c3689a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:28:08.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.556+0000 7ff479d68700 1 --2- 172.21.15.100:0/1428418600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c10c358eb0 0x55c10c3599d0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7ff46c0a72c0 tx=0x7ff46c0a98c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8c97481c0dda270 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:08.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.556+0000 7ff45affd700 1 -- 172.21.15.100:0/1428418600 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff46c0aa050 con 0x55c10c358eb0 2024-05-10T11:28:08.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.556+0000 7ff45affd700 1 -- 172.21.15.100:0/1428418600 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff46c0ae400 con 0x55c10c358eb0 2024-05-10T11:28:08.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.556+0000 7ff45affd700 1 -- 172.21.15.100:0/1428418600 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff46c0aa510 con 0x55c10c358eb0 2024-05-10T11:28:08.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.557+0000 7ff48605eb80 1 -- 172.21.15.100:0/1428418600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c10c358eb0 msgr2=0x55c10c3599d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:08.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.557+0000 7ff48605eb80 1 --2- 172.21.15.100:0/1428418600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c10c358eb0 0x55c10c3599d0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7ff46c0a72c0 tx=0x7ff46c0a98c0 comp rx=0 tx=0).stop 2024-05-10T11:28:08.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.557+0000 7ff48605eb80 1 -- 172.21.15.100:0/1428418600 shutdown_connections 2024-05-10T11:28:08.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.557+0000 7ff48605eb80 1 --2- 172.21.15.100:0/1428418600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c10c358eb0 0x55c10c3599d0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:08.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.557+0000 7ff48605eb80 1 --2- 172.21.15.100:0/1428418600 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c10c3531b0 0x55c10c3689a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:08.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.557+0000 7ff48605eb80 1 --2- 172.21.15.100:0/1428418600 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c10c358590 0x55c10c353590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:08.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.557+0000 7ff48605eb80 1 -- 172.21.15.100:0/1428418600 >> 172.21.15.100:0/1428418600 conn(0x55c10c27ab70 msgr2=0x55c10c3592c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:08.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.557+0000 7ff48605eb80 1 -- 172.21.15.100:0/1428418600 shutdown_connections 2024-05-10T11:28:08.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.557+0000 7ff48605eb80 1 -- 172.21.15.100:0/1428418600 wait complete. 2024-05-10T11:28:08.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.557+0000 7ff48605eb80 1 -- start start 2024-05-10T11:28:08.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.557+0000 7ff48605eb80 1 Processor -- start 2024-05-10T11:28:08.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.558+0000 7ff48605eb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c10c3531b0 0x55c10c364d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:08.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.558+0000 7ff48605eb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c10c358590 0x55c10c365260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:08.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.558+0000 7ff48605eb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c10c358eb0 0x55c10c3734f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:08.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.558+0000 7ff48605eb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff4741ab3c0 con 0x55c10c358eb0 2024-05-10T11:28:08.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.558+0000 7ff48605eb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff4741ac830 con 0x55c10c3531b0 2024-05-10T11:28:08.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.558+0000 7ff48605eb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff4741aba90 con 0x55c10c358590 2024-05-10T11:28:08.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.558+0000 7ff48605eb80 10 client.0 _ll_register_callbacks cb 0x55c10c2345b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:28:08.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.558+0000 7ff479d68700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c10c358eb0 0x55c10c3734f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:08.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.558+0000 7ff479d68700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c10c358eb0 0x55c10c3734f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44792/0 (socket says 172.21.15.100:44792) 2024-05-10T11:28:08.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.558+0000 7ff479d68700 1 -- 172.21.15.100:0/4136751593 learned_addr learned my addr 172.21.15.100:0/4136751593 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:28:08.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.558+0000 7ff479567700 1 --2- 172.21.15.100:0/4136751593 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c10c3531b0 0x55c10c364d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:08.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.558+0000 7ff478d66700 1 --2- 172.21.15.100:0/4136751593 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c10c358590 0x55c10c365260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:08.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.558+0000 7ff479d68700 1 -- 172.21.15.100:0/4136751593 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c10c358590 msgr2=0x55c10c365260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:08.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.558+0000 7ff479d68700 1 --2- 172.21.15.100:0/4136751593 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c10c358590 0x55c10c365260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:08.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.558+0000 7ff479d68700 1 -- 172.21.15.100:0/4136751593 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c10c3531b0 msgr2=0x55c10c364d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:08.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.558+0000 7ff479d68700 1 --2- 172.21.15.100:0/4136751593 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c10c3531b0 0x55c10c364d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:08.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.558+0000 7ff479d68700 1 -- 172.21.15.100:0/4136751593 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c10c373c40 con 0x55c10c358eb0 2024-05-10T11:28:08.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.558+0000 7ff479d68700 1 --2- 172.21.15.100:0/4136751593 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c10c358eb0 0x55c10c3734f0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7ff46c0a9fe0 tx=0x7ff46c005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96e8a08fa96a7ff7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:08.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.558+0000 7ff458ff9700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:28:08.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.559+0000 7ff458ff9700 1 -- 172.21.15.100:0/4136751593 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff46c0aa050 con 0x55c10c358eb0 2024-05-10T11:28:08.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.559+0000 7ff458ff9700 1 -- 172.21.15.100:0/4136751593 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7ff46c0aa2b0 con 0x55c10c358eb0 2024-05-10T11:28:08.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.559+0000 7ff48605eb80 1 -- 172.21.15.100:0/4136751593 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff46c0ae400 con 0x55c10c358eb0 2024-05-10T11:28:08.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.559+0000 7ff458ff9700 1 -- 172.21.15.100:0/4136751593 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff46c06be60 con 0x55c10c358eb0 2024-05-10T11:28:08.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.559+0000 7ff48605eb80 10 client.17355 fetch_fsmap learned FSMap version 31 2024-05-10T11:28:08.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.559+0000 7ff48605eb80 1 -- 172.21.15.100:0/4136751593 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c10c352270 con 0x55c10c358eb0 2024-05-10T11:28:08.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.559+0000 7ff458ff9700 1 -- 172.21.15.100:0/4136751593 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff46c0ae400 con 0x55c10c358eb0 2024-05-10T11:28:08.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.559+0000 7ff458ff9700 1 -- 172.21.15.100:0/4136751593 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7ff46c0a8d20 con 0x55c10c358eb0 2024-05-10T11:28:08.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.559+0000 7ff48605eb80 10 client.17355 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:28:08.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.559+0000 7ff48605eb80 10 client.17355 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-10T11:28:08.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.559+0000 7ff48605eb80 20 client.17355 populate_metadata read hostname 'smithi100' 2024-05-10T11:28:08.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.559+0000 7ff48605eb80 1 --2- 172.21.15.100:0/4136751593 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55c10c3db8a0 0x55c10c3ddc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:08.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.559+0000 7ff48605eb80 4 client.17355 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:28:08.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.559+0000 7ff48605eb80 1 -- 172.21.15.100:0/4136751593 --> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c10c2a83c0 con 0x55c10c3db8a0 2024-05-10T11:28:08.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.560+0000 7ff479567700 1 --2- 172.21.15.100:0/4136751593 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55c10c3db8a0 0x55c10c3ddc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:08.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.560+0000 7ff479567700 1 --2- 172.21.15.100:0/4136751593 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55c10c3db8a0 0x55c10c3ddc90 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bb5921b2c405241b server_cookie=20bc42d14c9ce84a in_seq=0 out_seq=0 2024-05-10T11:28:08.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.560+0000 7ff458ff9700 10 client.17355 ms_handle_connect on v2:172.21.15.178:6838/3760985404 2024-05-10T11:28:08.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.560+0000 7ff458ff9700 1 -- 172.21.15.100:0/4136751593 <== mds.1 v2:172.21.15.178:6838/3760985404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c10c2a83c0 con 0x55c10c3db8a0 2024-05-10T11:28:08.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.560+0000 7ff458ff9700 10 client.17355 handle_command_reply: tid=0 2024-05-10T11:28:08.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.560+0000 7ff48605eb80 1 client.17355 shutdown 2024-05-10T11:28:08.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.560+0000 7ff48605eb80 1 -- 172.21.15.100:0/4136751593 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c10c358eb0 msgr2=0x55c10c3734f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:08.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.560+0000 7ff48605eb80 1 --2- 172.21.15.100:0/4136751593 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c10c358eb0 0x55c10c3734f0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7ff46c0a9fe0 tx=0x7ff46c005750 comp rx=0 tx=0).stop 2024-05-10T11:28:08.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.560+0000 7ff48605eb80 1 -- 172.21.15.100:0/4136751593 shutdown_connections 2024-05-10T11:28:08.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.560+0000 7ff48605eb80 1 --2- 172.21.15.100:0/4136751593 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55c10c3db8a0 0x55c10c3ddc90 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:08.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.560+0000 7ff48605eb80 1 --2- 172.21.15.100:0/4136751593 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c10c358eb0 0x55c10c3734f0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:08.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.560+0000 7ff48605eb80 1 --2- 172.21.15.100:0/4136751593 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c10c3531b0 0x55c10c364d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:08.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.560+0000 7ff48605eb80 1 --2- 172.21.15.100:0/4136751593 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c10c358590 0x55c10c365260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:08.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.560+0000 7ff48605eb80 1 -- 172.21.15.100:0/4136751593 >> 172.21.15.100:0/4136751593 conn(0x55c10c27ab70 msgr2=0x55c10c34d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:08.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.560+0000 7ff458ff9700 0 client.17355 ms_handle_reset on v2:172.21.15.178:6838/3760985404 2024-05-10T11:28:08.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.561+0000 7ff48605eb80 1 -- 172.21.15.100:0/4136751593 shutdown_connections 2024-05-10T11:28:08.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.561+0000 7ff48605eb80 1 -- 172.21.15.100:0/4136751593 wait complete. 2024-05-10T11:28:08.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.561+0000 7ff48605eb80 20 client.17355 trim_cache size 0 max 16384 2024-05-10T11:28:08.580 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:28:08.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.561+0000 7ff47b7cb700 1 -- 172.21.15.100:0/456901610 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff45c0b2880 msgr2=0x7ff45c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:08.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.561+0000 7ff47b7cb700 1 --2- 172.21.15.100:0/456901610 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff45c0b2880 0x7ff45c0b4d40 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff4640060b0 tx=0x7ff46400c040 comp rx=0 tx=0).stop 2024-05-10T11:28:08.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.561+0000 7ff47b7cb700 1 -- 172.21.15.100:0/456901610 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff474107760 msgr2=0x7ff4741a5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:08.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.561+0000 7ff47b7cb700 1 --2- 172.21.15.100:0/456901610 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff474107760 0x7ff4741a5840 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7ff46c005910 tx=0x7ff46c0055f0 comp rx=0 tx=0).stop 2024-05-10T11:28:08.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.561+0000 7ff47b7cb700 1 -- 172.21.15.100:0/456901610 shutdown_connections 2024-05-10T11:28:08.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.561+0000 7ff47b7cb700 1 --2- 172.21.15.100:0/456901610 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff45c0b2880 0x7ff45c0b4d40 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:08.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.561+0000 7ff47b7cb700 1 --2- 172.21.15.100:0/456901610 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff474107760 0x7ff4741a5840 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:08.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.561+0000 7ff47b7cb700 1 --2- 172.21.15.100:0/456901610 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff474102e00 0x7ff47419a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:08.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.561+0000 7ff47b7cb700 1 --2- 172.21.15.100:0/456901610 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4741037b0 0x7ff47419ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:08.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.561+0000 7ff47b7cb700 1 -- 172.21.15.100:0/456901610 >> 172.21.15.100:0/456901610 conn(0x7ff474078a10 msgr2=0x7ff4740ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:08.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.562+0000 7ff47b7cb700 1 -- 172.21.15.100:0/456901610 shutdown_connections 2024-05-10T11:28:08.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:08.562+0000 7ff47b7cb700 1 -- 172.21.15.100:0/456901610 wait complete. 2024-05-10T11:28:08.979 INFO:tasks.workunit.client.0.smithi100.stdout: 1 248773 2.74 MB/sec execute 363 sec latency 18.556 ms 2024-05-10T11:28:09.294 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-10T11:28:09.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:28:09 smithi100 ceph-mon[30296]: pgmap v713: 129 pgs: 129 active+clean; 102 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 1.5 MiB/s wr, 171 op/s 2024-05-10T11:28:09.555 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:28:09.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:28:09 smithi178 ceph-mon[33359]: pgmap v713: 129 pgs: 129 active+clean; 102 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 1.5 MiB/s wr, 171 op/s 2024-05-10T11:28:09.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:28:09 smithi188 ceph-mon[34075]: pgmap v713: 129 pgs: 129 active+clean; 102 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 1.5 MiB/s wr, 171 op/s 2024-05-10T11:28:09.978 INFO:tasks.workunit.client.0.smithi100.stdout: 1 249154 2.74 MB/sec execute 364 sec latency 13.315 ms 2024-05-10T11:28:10.978 INFO:tasks.workunit.client.0.smithi100.stdout: 1 249613 2.73 MB/sec execute 365 sec latency 60.256 ms 2024-05-10T11:28:11.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.252+0000 7f7585a70700 1 -- 172.21.15.100:0/1608987443 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7580108410 msgr2=0x7f75801087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:11.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.252+0000 7f7585a70700 1 --2- 172.21.15.100:0/1608987443 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7580108410 0x7f75801087f0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f7570009990 tx=0x7f7570009ca0 comp rx=0 tx=0).stop 2024-05-10T11:28:11.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.252+0000 7f7585a70700 1 -- 172.21.15.100:0/1608987443 shutdown_connections 2024-05-10T11:28:11.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.252+0000 7f7585a70700 1 --2- 172.21.15.100:0/1608987443 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7580102d60 0x7f758010f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:11.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.252+0000 7f7585a70700 1 --2- 172.21.15.100:0/1608987443 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7580108410 0x7f75801087f0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:11.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.252+0000 7f7585a70700 1 --2- 172.21.15.100:0/1608987443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75801023a0 0x7f7580102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:11.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.252+0000 7f7585a70700 1 -- 172.21.15.100:0/1608987443 >> 172.21.15.100:0/1608987443 conn(0x7f7580079350 msgr2=0x7f7580079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:11.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.253+0000 7f7585a70700 1 -- 172.21.15.100:0/1608987443 shutdown_connections 2024-05-10T11:28:11.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.253+0000 7f7585a70700 1 -- 172.21.15.100:0/1608987443 wait complete. 2024-05-10T11:28:11.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.253+0000 7f7585a70700 1 Processor -- start 2024-05-10T11:28:11.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.253+0000 7f7585a70700 1 -- start start 2024-05-10T11:28:11.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.254+0000 7f7585a70700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75801023a0 0x7f75801a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:11.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.254+0000 7f7585a70700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7580102d60 0x7f75801a1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:11.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.254+0000 7f7585a70700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7580108410 0x7f758019ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:11.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.254+0000 7f7585a70700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f75801123e0 con 0x7f7580108410 2024-05-10T11:28:11.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.254+0000 7f7585a70700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7580112260 con 0x7f75801023a0 2024-05-10T11:28:11.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.254+0000 7f7585a70700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7580112560 con 0x7f7580102d60 2024-05-10T11:28:11.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.254+0000 7f757effd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7580102d60 0x7f75801a1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:11.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.254+0000 7f757effd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7580102d60 0x7f75801a1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:43656/0 (socket says 172.21.15.100:43656) 2024-05-10T11:28:11.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.254+0000 7f757effd700 1 -- 172.21.15.100:0/1243905786 learned_addr learned my addr 172.21.15.100:0/1243905786 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:28:11.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.254+0000 7f757ffff700 1 --2- 172.21.15.100:0/1243905786 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7580108410 0x7f758019ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:11.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.254+0000 7f757f7fe700 1 --2- 172.21.15.100:0/1243905786 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75801023a0 0x7f75801a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:11.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.254+0000 7f757effd700 1 -- 172.21.15.100:0/1243905786 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75801023a0 msgr2=0x7f75801a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:11.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.254+0000 7f757effd700 1 --2- 172.21.15.100:0/1243905786 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75801023a0 0x7f75801a18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:11.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.254+0000 7f757effd700 1 -- 172.21.15.100:0/1243905786 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7580108410 msgr2=0x7f758019ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:11.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.254+0000 7f757effd700 1 --2- 172.21.15.100:0/1243905786 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7580108410 0x7f758019ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:11.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.254+0000 7f757effd700 1 -- 172.21.15.100:0/1243905786 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f758019c1f0 con 0x7f7580102d60 2024-05-10T11:28:11.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.254+0000 7f757ffff700 1 --2- 172.21.15.100:0/1243905786 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7580108410 0x7f758019ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:28:11.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.254+0000 7f757effd700 1 --2- 172.21.15.100:0/1243905786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7580102d60 0x7f75801a1e10 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f757400d950 tx=0x7f757400dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:11.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.254+0000 7f757cff9700 1 -- 172.21.15.100:0/1243905786 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7574016070 con 0x7f7580102d60 2024-05-10T11:28:11.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.255+0000 7f7585a70700 1 -- 172.21.15.100:0/1243905786 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f758019c4e0 con 0x7f7580102d60 2024-05-10T11:28:11.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.255+0000 7f757cff9700 1 -- 172.21.15.100:0/1243905786 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f757400fd20 con 0x7f7580102d60 2024-05-10T11:28:11.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.255+0000 7f757cff9700 1 -- 172.21.15.100:0/1243905786 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757400e3b0 con 0x7f7580102d60 2024-05-10T11:28:11.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.255+0000 7f7585a70700 1 -- 172.21.15.100:0/1243905786 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75801a87a0 con 0x7f7580102d60 2024-05-10T11:28:11.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.257+0000 7f757cff9700 1 -- 172.21.15.100:0/1243905786 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f7574019070 con 0x7f7580102d60 2024-05-10T11:28:11.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.258+0000 7f7590303b80 1 Processor -- start 2024-05-10T11:28:11.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.258+0000 7f7590303b80 1 -- start start 2024-05-10T11:28:11.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.258+0000 7f757cff9700 1 --2- 172.21.15.100:0/1243905786 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f75680b2880 0x7f75680b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:11.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.258+0000 7f7590303b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba3c7781b0 0x55ba3c778590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:11.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.258+0000 7f7590303b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ba3c778ad0 0x55ba3c772810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:11.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.258+0000 7f7590303b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba3c772d50 0x55ba3c773190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:11.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.259+0000 7f757cff9700 1 -- 172.21.15.100:0/1243905786 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f75740e2030 con 0x7f7580102d60 2024-05-10T11:28:11.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.259+0000 7f757ffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba3c772d50 0x55ba3c773190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:11.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.259+0000 7f757effd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ba3c778ad0 0x55ba3c772810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:11.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.259+0000 7f757ffff700 1 --2- 172.21.15.100:0/1243905786 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f75680b2880 0x7f75680b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:11.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.259+0000 7f757effd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ba3c778ad0 0x55ba3c772810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:43670/0 (socket says 172.21.15.100:43670) 2024-05-10T11:28:11.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.259+0000 7f757f7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba3c7781b0 0x55ba3c778590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:11.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.259+0000 7f757effd700 1 -- 172.21.15.100:0/2254687596 learned_addr learned my addr 172.21.15.100:0/2254687596 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:28:11.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.259+0000 7f7590303b80 1 -- 172.21.15.100:0/2254687596 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55ba3c6c4940 con 0x55ba3c772d50 2024-05-10T11:28:11.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.259+0000 7f7590303b80 1 -- 172.21.15.100:0/2254687596 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ba3c629da0 con 0x55ba3c7781b0 2024-05-10T11:28:11.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.259+0000 7f7590303b80 1 -- 172.21.15.100:0/2254687596 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55ba3c6c0900 con 0x55ba3c778ad0 2024-05-10T11:28:11.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.259+0000 7f757f7fe700 1 -- 172.21.15.100:0/2254687596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ba3c778ad0 msgr2=0x55ba3c772810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:11.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.259+0000 7f757f7fe700 1 --2- 172.21.15.100:0/2254687596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ba3c778ad0 0x55ba3c772810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:11.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.259+0000 7f757f7fe700 1 -- 172.21.15.100:0/2254687596 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba3c772d50 msgr2=0x55ba3c773190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:11.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.259+0000 7f757f7fe700 1 --2- 172.21.15.100:0/2254687596 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba3c772d50 0x55ba3c773190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:11.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.259+0000 7f757f7fe700 1 -- 172.21.15.100:0/2254687596 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba3c6118d0 con 0x55ba3c7781b0 2024-05-10T11:28:11.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.259+0000 7f757f7fe700 1 --2- 172.21.15.100:0/2254687596 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba3c7781b0 0x55ba3c778590 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f758019cbd0 tx=0x7f7570005fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=122f824191637fbd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:11.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.260+0000 7f757ffff700 1 --2- 172.21.15.100:0/1243905786 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f75680b2880 0x7f75680b4d40 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f7578009550 tx=0x7f7578009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:11.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.260+0000 7f7565ffb700 1 -- 172.21.15.100:0/2254687596 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7570004620 con 0x55ba3c7781b0 2024-05-10T11:28:11.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.260+0000 7f7565ffb700 1 -- 172.21.15.100:0/2254687596 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7570027070 con 0x55ba3c7781b0 2024-05-10T11:28:11.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.260+0000 7f7565ffb700 1 -- 172.21.15.100:0/2254687596 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757000ba30 con 0x55ba3c7781b0 2024-05-10T11:28:11.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.261+0000 7f7590303b80 1 -- 172.21.15.100:0/2254687596 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba3c7781b0 msgr2=0x55ba3c778590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:11.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.261+0000 7f7590303b80 1 --2- 172.21.15.100:0/2254687596 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba3c7781b0 0x55ba3c778590 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f758019cbd0 tx=0x7f7570005fb0 comp rx=0 tx=0).stop 2024-05-10T11:28:11.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.261+0000 7f7590303b80 1 -- 172.21.15.100:0/2254687596 shutdown_connections 2024-05-10T11:28:11.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.261+0000 7f7590303b80 1 --2- 172.21.15.100:0/2254687596 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba3c772d50 0x55ba3c773190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:11.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.261+0000 7f7590303b80 1 --2- 172.21.15.100:0/2254687596 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba3c7781b0 0x55ba3c778590 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:11.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.261+0000 7f7590303b80 1 --2- 172.21.15.100:0/2254687596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ba3c778ad0 0x55ba3c772810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:11.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.261+0000 7f7590303b80 1 -- 172.21.15.100:0/2254687596 >> 172.21.15.100:0/2254687596 conn(0x55ba3c69eb70 msgr2=0x55ba3c76d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:11.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.261+0000 7f7590303b80 1 -- 172.21.15.100:0/2254687596 shutdown_connections 2024-05-10T11:28:11.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.261+0000 7f7590303b80 1 -- 172.21.15.100:0/2254687596 wait complete. 2024-05-10T11:28:11.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.261+0000 7f7590303b80 1 -- start start 2024-05-10T11:28:11.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.261+0000 7f7590303b80 1 Processor -- start 2024-05-10T11:28:11.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.262+0000 7f7590303b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba3c772d50 0x55ba3c77e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:11.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.262+0000 7f7590303b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ba3c7781b0 0x55ba3c77ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:11.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.262+0000 7f7590303b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba3c778ad0 0x55ba3c77bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:11.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.262+0000 7f7590303b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55ba3c77f3f0 con 0x55ba3c772d50 2024-05-10T11:28:11.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.262+0000 7f7590303b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ba3c67ded0 con 0x55ba3c778ad0 2024-05-10T11:28:11.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.262+0000 7f7590303b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55ba3c77f570 con 0x55ba3c7781b0 2024-05-10T11:28:11.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.262+0000 7f7590303b80 10 client.0 _ll_register_callbacks cb 0x55ba3c6585b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:28:11.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.262+0000 7f757f7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba3c772d50 0x55ba3c77e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:11.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.262+0000 7f757f7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba3c772d50 0x55ba3c77e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44844/0 (socket says 172.21.15.100:44844) 2024-05-10T11:28:11.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.262+0000 7f757f7fe700 1 -- 172.21.15.100:0/194658823 learned_addr learned my addr 172.21.15.100:0/194658823 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:28:11.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.262+0000 7f757effd700 1 --2- 172.21.15.100:0/194658823 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ba3c7781b0 0x55ba3c77ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:11.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.262+0000 7f757ffff700 1 --2- 172.21.15.100:0/194658823 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba3c778ad0 0x55ba3c77bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:11.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.262+0000 7f757f7fe700 1 -- 172.21.15.100:0/194658823 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ba3c7781b0 msgr2=0x55ba3c77ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:11.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.262+0000 7f757f7fe700 1 --2- 172.21.15.100:0/194658823 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ba3c7781b0 0x55ba3c77ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:11.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.262+0000 7f757f7fe700 1 -- 172.21.15.100:0/194658823 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba3c778ad0 msgr2=0x55ba3c77bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:11.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.262+0000 7f757f7fe700 1 --2- 172.21.15.100:0/194658823 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba3c778ad0 0x55ba3c77bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:11.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.262+0000 7f757f7fe700 1 -- 172.21.15.100:0/194658823 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba3c67b760 con 0x55ba3c772d50 2024-05-10T11:28:11.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.262+0000 7f757ffff700 1 --2- 172.21.15.100:0/194658823 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba3c778ad0 0x55ba3c77bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:28:11.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.262+0000 7f757effd700 1 --2- 172.21.15.100:0/194658823 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ba3c7781b0 0x55ba3c77ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:28:11.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.262+0000 7f757f7fe700 1 --2- 172.21.15.100:0/194658823 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba3c772d50 0x55ba3c77e6e0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f7570004010 tx=0x7f7570003930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f391555646b1ba7b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:11.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.262+0000 7f755effd700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:28:11.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.262+0000 7f755effd700 1 -- 172.21.15.100:0/194658823 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757000b8b0 con 0x55ba3c772d50 2024-05-10T11:28:11.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.262+0000 7f755effd700 1 -- 172.21.15.100:0/194658823 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f757000ba50 con 0x55ba3c772d50 2024-05-10T11:28:11.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.262+0000 7f7590303b80 1 -- 172.21.15.100:0/194658823 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7570027070 con 0x55ba3c772d50 2024-05-10T11:28:11.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.263+0000 7f755effd700 1 -- 172.21.15.100:0/194658823 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f757003b050 con 0x55ba3c772d50 2024-05-10T11:28:11.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.263+0000 7f7590303b80 10 client.17367 fetch_fsmap learned FSMap version 31 2024-05-10T11:28:11.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.263+0000 7f7590303b80 1 -- 172.21.15.100:0/194658823 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55ba3c789c50 con 0x55ba3c772d50 2024-05-10T11:28:11.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.263+0000 7f755effd700 1 -- 172.21.15.100:0/194658823 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7570027070 con 0x55ba3c772d50 2024-05-10T11:28:11.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.263+0000 7f755effd700 1 -- 172.21.15.100:0/194658823 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f757000f8f0 con 0x55ba3c772d50 2024-05-10T11:28:11.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.263+0000 7f7590303b80 10 client.17367 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:28:11.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.263+0000 7f7590303b80 10 client.17367 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:28:11.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.263+0000 7f7590303b80 20 client.17367 populate_metadata read hostname 'smithi100' 2024-05-10T11:28:11.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.263+0000 7f7590303b80 1 --2- 172.21.15.100:0/194658823 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ba3c78c120 0x55ba3c78e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:11.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.263+0000 7f7590303b80 4 client.17367 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:28:11.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.263+0000 7f7590303b80 1 -- 172.21.15.100:0/194658823 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba3c6d2320 con 0x55ba3c78c120 2024-05-10T11:28:11.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.263+0000 7f757effd700 1 --2- 172.21.15.100:0/194658823 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ba3c78c120 0x55ba3c78e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:11.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.264+0000 7f757effd700 1 --2- 172.21.15.100:0/194658823 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ba3c78c120 0x55ba3c78e510 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adea234695be0576 server_cookie=23741f192b7b6fe4 in_seq=0 out_seq=0 2024-05-10T11:28:11.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.264+0000 7f755effd700 10 client.17367 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:28:11.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.265+0000 7f755effd700 1 -- 172.21.15.100:0/194658823 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ba3c6d2320 con 0x55ba3c78c120 2024-05-10T11:28:11.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.265+0000 7f755effd700 10 client.17367 handle_command_reply: tid=0 2024-05-10T11:28:11.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.265+0000 7f7590303b80 1 client.17367 shutdown 2024-05-10T11:28:11.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.265+0000 7f7590303b80 1 -- 172.21.15.100:0/194658823 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba3c772d50 msgr2=0x55ba3c77e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:11.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.265+0000 7f7590303b80 1 --2- 172.21.15.100:0/194658823 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba3c772d50 0x55ba3c77e6e0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f7570004010 tx=0x7f7570003930 comp rx=0 tx=0).stop 2024-05-10T11:28:11.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.265+0000 7f7590303b80 1 -- 172.21.15.100:0/194658823 shutdown_connections 2024-05-10T11:28:11.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.265+0000 7f7590303b80 1 --2- 172.21.15.100:0/194658823 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ba3c78c120 0x55ba3c78e510 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:11.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.265+0000 7f7590303b80 1 --2- 172.21.15.100:0/194658823 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba3c778ad0 0x55ba3c77bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:11.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.265+0000 7f755effd700 0 client.17367 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:28:11.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.265+0000 7f7590303b80 1 --2- 172.21.15.100:0/194658823 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba3c772d50 0x55ba3c77e6e0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:11.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.265+0000 7f7590303b80 1 --2- 172.21.15.100:0/194658823 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ba3c7781b0 0x55ba3c77ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:11.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.265+0000 7f7590303b80 1 -- 172.21.15.100:0/194658823 >> 172.21.15.100:0/194658823 conn(0x55ba3c69eb70 msgr2=0x55ba3c767b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:11.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.265+0000 7f7590303b80 1 -- 172.21.15.100:0/194658823 shutdown_connections 2024-05-10T11:28:11.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.265+0000 7f7590303b80 1 -- 172.21.15.100:0/194658823 wait complete. 2024-05-10T11:28:11.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.265+0000 7f7590303b80 20 client.17367 trim_cache size 0 max 16384 2024-05-10T11:28:11.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.282+0000 7f7590303b80 1 Processor -- start 2024-05-10T11:28:11.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.282+0000 7f7590303b80 1 -- start start 2024-05-10T11:28:11.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.283+0000 7f7590303b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ba3c772d50 0x55ba3c772810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:11.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.283+0000 7f7590303b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba3c7781b0 0x55ba3c767c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:11.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.283+0000 7f7590303b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba3c778ad0 0x55ba3c77aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:11.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.283+0000 7f7590303b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f75801b3a70 con 0x55ba3c7781b0 2024-05-10T11:28:11.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.283+0000 7f7590303b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f75801b1300 con 0x55ba3c778ad0 2024-05-10T11:28:11.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.283+0000 7f7590303b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f75801af150 con 0x55ba3c772d50 2024-05-10T11:28:11.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.283+0000 7f757ffff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba3c778ad0 0x55ba3c77aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:11.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.283+0000 7f757effd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba3c7781b0 0x55ba3c767c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:11.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.283+0000 7f757effd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba3c7781b0 0x55ba3c767c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44858/0 (socket says 172.21.15.100:44858) 2024-05-10T11:28:11.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.283+0000 7f757ffff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba3c778ad0 0x55ba3c77aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:42410/0 (socket says 172.21.15.100:42410) 2024-05-10T11:28:11.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.283+0000 7f757effd700 1 -- 172.21.15.100:0/2490673655 learned_addr learned my addr 172.21.15.100:0/2490673655 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:28:11.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.283+0000 7f757f7fe700 1 --2- 172.21.15.100:0/2490673655 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ba3c772d50 0x55ba3c772810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:11.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.283+0000 7f757ffff700 1 -- 172.21.15.100:0/2490673655 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ba3c772d50 msgr2=0x55ba3c772810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:11.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.283+0000 7f757ffff700 1 --2- 172.21.15.100:0/2490673655 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ba3c772d50 0x55ba3c772810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:11.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.283+0000 7f757ffff700 1 -- 172.21.15.100:0/2490673655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba3c7781b0 msgr2=0x55ba3c767c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:11.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.283+0000 7f757ffff700 1 --2- 172.21.15.100:0/2490673655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba3c7781b0 0x55ba3c767c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:11.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.283+0000 7f757ffff700 1 -- 172.21.15.100:0/2490673655 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba3c77f010 con 0x55ba3c778ad0 2024-05-10T11:28:11.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.283+0000 7f757ffff700 1 --2- 172.21.15.100:0/2490673655 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba3c778ad0 0x55ba3c77aad0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7578009ec0 tx=0x7f757801d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=563f8fbef4efa947 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:11.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.284+0000 7f75657fa700 1 -- 172.21.15.100:0/2490673655 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7578008570 con 0x55ba3c778ad0 2024-05-10T11:28:11.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.284+0000 7f75657fa700 1 -- 172.21.15.100:0/2490673655 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f757801bd40 con 0x55ba3c778ad0 2024-05-10T11:28:11.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.284+0000 7f75657fa700 1 -- 172.21.15.100:0/2490673655 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7578028870 con 0x55ba3c778ad0 2024-05-10T11:28:11.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.284+0000 7f7590303b80 1 -- 172.21.15.100:0/2490673655 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba3c778ad0 msgr2=0x55ba3c77aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:11.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.284+0000 7f7590303b80 1 --2- 172.21.15.100:0/2490673655 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba3c778ad0 0x55ba3c77aad0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7578009ec0 tx=0x7f757801d000 comp rx=0 tx=0).stop 2024-05-10T11:28:11.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.284+0000 7f7590303b80 1 -- 172.21.15.100:0/2490673655 shutdown_connections 2024-05-10T11:28:11.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.284+0000 7f7590303b80 1 --2- 172.21.15.100:0/2490673655 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba3c778ad0 0x55ba3c77aad0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:11.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.284+0000 7f7590303b80 1 --2- 172.21.15.100:0/2490673655 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ba3c772d50 0x55ba3c772810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:11.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.284+0000 7f7590303b80 1 --2- 172.21.15.100:0/2490673655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba3c7781b0 0x55ba3c767c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:11.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.284+0000 7f7590303b80 1 -- 172.21.15.100:0/2490673655 >> 172.21.15.100:0/2490673655 conn(0x55ba3c69eb70 msgr2=0x55ba3c779520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:11.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.284+0000 7f7590303b80 1 -- 172.21.15.100:0/2490673655 shutdown_connections 2024-05-10T11:28:11.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.284+0000 7f7590303b80 1 -- 172.21.15.100:0/2490673655 wait complete. 2024-05-10T11:28:11.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.285+0000 7f7590303b80 1 -- start start 2024-05-10T11:28:11.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.285+0000 7f7590303b80 1 Processor -- start 2024-05-10T11:28:11.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.285+0000 7f7590303b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ba3c772d50 0x55ba3c78e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:11.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.285+0000 7f7590303b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba3c7781b0 0x55ba3c78e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:11.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.285+0000 7f7590303b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba3c778ad0 0x55ba3c78eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:11.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.285+0000 7f7590303b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f75801af7c0 con 0x55ba3c7781b0 2024-05-10T11:28:11.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.285+0000 7f7590303b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f75801b0c30 con 0x55ba3c778ad0 2024-05-10T11:28:11.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.285+0000 7f7590303b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f75801afe90 con 0x55ba3c772d50 2024-05-10T11:28:11.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.285+0000 7f7590303b80 10 client.0 _ll_register_callbacks cb 0x55ba3c6585b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:28:11.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.285+0000 7f757ffff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba3c778ad0 0x55ba3c78eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:11.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.285+0000 7f757effd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba3c7781b0 0x55ba3c78e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:11.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.285+0000 7f757ffff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba3c778ad0 0x55ba3c78eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:42418/0 (socket says 172.21.15.100:42418) 2024-05-10T11:28:11.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.285+0000 7f757f7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ba3c772d50 0x55ba3c78e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:11.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.285+0000 7f757ffff700 1 -- 172.21.15.100:0/2271490935 learned_addr learned my addr 172.21.15.100:0/2271490935 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:28:11.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.286+0000 7f757ffff700 1 -- 172.21.15.100:0/2271490935 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ba3c772d50 msgr2=0x55ba3c78e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:11.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.286+0000 7f757ffff700 1 --2- 172.21.15.100:0/2271490935 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ba3c772d50 0x55ba3c78e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:11.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.286+0000 7f757ffff700 1 -- 172.21.15.100:0/2271490935 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba3c7781b0 msgr2=0x55ba3c78e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:11.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.286+0000 7f757ffff700 1 --2- 172.21.15.100:0/2271490935 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba3c7781b0 0x55ba3c78e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:11.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.286+0000 7f757ffff700 1 -- 172.21.15.100:0/2271490935 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba3c7f4930 con 0x55ba3c778ad0 2024-05-10T11:28:11.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.286+0000 7f757ffff700 1 --2- 172.21.15.100:0/2271490935 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba3c778ad0 0x55ba3c78eea0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f757802a040 tx=0x7f757801bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf7b2e9982776a2f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:11.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.286+0000 7f755effd700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:28:11.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.286+0000 7f755effd700 1 -- 172.21.15.100:0/2271490935 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7578008570 con 0x55ba3c778ad0 2024-05-10T11:28:11.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.286+0000 7f7590303b80 1 -- 172.21.15.100:0/2271490935 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f757801bd40 con 0x55ba3c778ad0 2024-05-10T11:28:11.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.286+0000 7f755effd700 1 -- 172.21.15.100:0/2271490935 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f7578004e00 con 0x55ba3c778ad0 2024-05-10T11:28:11.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.286+0000 7f755effd700 1 -- 172.21.15.100:0/2271490935 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f757803f050 con 0x55ba3c778ad0 2024-05-10T11:28:11.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.286+0000 7f7590303b80 10 client.26768 fetch_fsmap learned FSMap version 31 2024-05-10T11:28:11.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.286+0000 7f7590303b80 1 -- 172.21.15.100:0/2271490935 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55ba3c7f5c30 con 0x55ba3c778ad0 2024-05-10T11:28:11.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.286+0000 7f755effd700 1 -- 172.21.15.100:0/2271490935 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f757801bd40 con 0x55ba3c778ad0 2024-05-10T11:28:11.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.287+0000 7f755effd700 1 -- 172.21.15.100:0/2271490935 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f757803a030 con 0x55ba3c778ad0 2024-05-10T11:28:11.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.287+0000 7f7590303b80 10 client.26768 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:28:11.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.287+0000 7f7590303b80 10 client.26768 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:28:11.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.287+0000 7f7590303b80 20 client.26768 populate_metadata read hostname 'smithi100' 2024-05-10T11:28:11.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.287+0000 7f7590303b80 1 --2- 172.21.15.100:0/2271490935 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ba3c78c120 0x55ba3c7f8330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:11.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.287+0000 7f7590303b80 4 client.26768 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:28:11.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.287+0000 7f7590303b80 1 -- 172.21.15.100:0/2271490935 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ba3c6cc3c0 con 0x55ba3c78c120 2024-05-10T11:28:11.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.287+0000 7f757f7fe700 1 --2- 172.21.15.100:0/2271490935 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ba3c78c120 0x55ba3c7f8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:11.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.287+0000 7f755effd700 10 client.26768 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:28:11.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.287+0000 7f757f7fe700 1 --2- 172.21.15.100:0/2271490935 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ba3c78c120 0x55ba3c7f8330 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ea5546cf5fe8f98 server_cookie=68b3cc3f457efada in_seq=0 out_seq=0 2024-05-10T11:28:11.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.287+0000 7f755effd700 1 -- 172.21.15.100:0/2271490935 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ba3c6cc3c0 con 0x55ba3c78c120 2024-05-10T11:28:11.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.287+0000 7f755effd700 10 client.26768 handle_command_reply: tid=0 2024-05-10T11:28:11.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.287+0000 7f7590303b80 1 client.26768 shutdown 2024-05-10T11:28:11.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.288+0000 7f7590303b80 1 -- 172.21.15.100:0/2271490935 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba3c778ad0 msgr2=0x55ba3c78eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:11.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.288+0000 7f7590303b80 1 --2- 172.21.15.100:0/2271490935 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba3c778ad0 0x55ba3c78eea0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f757802a040 tx=0x7f757801bf60 comp rx=0 tx=0).stop 2024-05-10T11:28:11.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.288+0000 7f7590303b80 1 -- 172.21.15.100:0/2271490935 shutdown_connections 2024-05-10T11:28:11.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.288+0000 7f7590303b80 1 --2- 172.21.15.100:0/2271490935 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ba3c78c120 0x55ba3c7f8330 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:11.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.288+0000 7f7590303b80 1 --2- 172.21.15.100:0/2271490935 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba3c778ad0 0x55ba3c78eea0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:11.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.288+0000 7f755effd700 0 client.26768 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:28:11.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.288+0000 7f7590303b80 1 --2- 172.21.15.100:0/2271490935 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ba3c772d50 0x55ba3c78e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:11.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.288+0000 7f7590303b80 1 --2- 172.21.15.100:0/2271490935 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba3c7781b0 0x55ba3c78e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:11.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.288+0000 7f7590303b80 1 -- 172.21.15.100:0/2271490935 >> 172.21.15.100:0/2271490935 conn(0x55ba3c69eb70 msgr2=0x55ba3c777490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:11.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.288+0000 7f7590303b80 1 -- 172.21.15.100:0/2271490935 shutdown_connections 2024-05-10T11:28:11.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.288+0000 7f7590303b80 1 -- 172.21.15.100:0/2271490935 wait complete. 2024-05-10T11:28:11.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.288+0000 7f7590303b80 20 client.26768 trim_cache size 0 max 16384 2024-05-10T11:28:11.293 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:28:11.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.288+0000 7f7585a70700 1 -- 172.21.15.100:0/1243905786 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f75680b2880 msgr2=0x7f75680b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:11.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.288+0000 7f7585a70700 1 --2- 172.21.15.100:0/1243905786 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f75680b2880 0x7f75680b4d40 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f7578009550 tx=0x7f7578009f10 comp rx=0 tx=0).stop 2024-05-10T11:28:11.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.288+0000 7f7585a70700 1 -- 172.21.15.100:0/1243905786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7580102d60 msgr2=0x7f75801a1e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:11.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.289+0000 7f7585a70700 1 --2- 172.21.15.100:0/1243905786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7580102d60 0x7f75801a1e10 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f757400d950 tx=0x7f757400dc60 comp rx=0 tx=0).stop 2024-05-10T11:28:11.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.289+0000 7f7585a70700 1 -- 172.21.15.100:0/1243905786 shutdown_connections 2024-05-10T11:28:11.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.289+0000 7f7585a70700 1 --2- 172.21.15.100:0/1243905786 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f75680b2880 0x7f75680b4d40 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:11.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.289+0000 7f7585a70700 1 --2- 172.21.15.100:0/1243905786 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7580108410 0x7f758019ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:11.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.289+0000 7f7585a70700 1 --2- 172.21.15.100:0/1243905786 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75801023a0 0x7f75801a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:11.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.289+0000 7f7585a70700 1 --2- 172.21.15.100:0/1243905786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7580102d60 0x7f75801a1e10 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:11.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.289+0000 7f7585a70700 1 -- 172.21.15.100:0/1243905786 >> 172.21.15.100:0/1243905786 conn(0x7f7580079350 msgr2=0x7f758006b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:11.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.289+0000 7f7585a70700 1 -- 172.21.15.100:0/1243905786 shutdown_connections 2024-05-10T11:28:11.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:11.289+0000 7f7585a70700 1 -- 172.21.15.100:0/1243905786 wait complete. 2024-05-10T11:28:11.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:28:11 smithi100 ceph-mon[30296]: pgmap v714: 129 pgs: 129 active+clean; 102 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 1.2 MiB/s wr, 127 op/s 2024-05-10T11:28:11.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:28:11 smithi178 ceph-mon[33359]: pgmap v714: 129 pgs: 129 active+clean; 102 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 1.2 MiB/s wr, 127 op/s 2024-05-10T11:28:11.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:28:11 smithi188 ceph-mon[34075]: pgmap v714: 129 pgs: 129 active+clean; 102 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 1.2 MiB/s wr, 127 op/s 2024-05-10T11:28:11.979 INFO:tasks.workunit.client.0.smithi100.stdout: 1 250247 2.73 MB/sec execute 366 sec latency 15.262 ms 2024-05-10T11:28:12.979 INFO:tasks.workunit.client.0.smithi100.stdout: 1 250824 2.73 MB/sec execute 367 sec latency 15.069 ms 2024-05-10T11:28:12.998 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T11:28:12.999 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T11:28:13.259 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:28:13.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:28:13 smithi178 ceph-mon[33359]: pgmap v715: 129 pgs: 129 active+clean; 104 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-05-10T11:28:13.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:28:13 smithi188 ceph-mon[34075]: pgmap v715: 129 pgs: 129 active+clean; 104 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-05-10T11:28:13.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:28:13 smithi100 ceph-mon[30296]: pgmap v715: 129 pgs: 129 active+clean; 104 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-05-10T11:28:13.979 INFO:tasks.workunit.client.0.smithi100.stdout: 1 251515 2.74 MB/sec execute 368 sec latency 66.410 ms 2024-05-10T11:28:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.206+0000 7f514d1da640 1 -- 172.21.15.100:0/1516988217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5148105e50 msgr2=0x7f51481062d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.206+0000 7f514d1da640 1 --2- 172.21.15.100:0/1516988217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5148105e50 0x7f51481062d0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f513c009900 tx=0x7f513c02a410 comp rx=0 tx=0).stop 2024-05-10T11:28:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.206+0000 7f514d1da640 1 -- 172.21.15.100:0/1516988217 shutdown_connections 2024-05-10T11:28:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.206+0000 7f514d1da640 1 --2- 172.21.15.100:0/1516988217 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5148106810 0x7f514810d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.206+0000 7f514d1da640 1 --2- 172.21.15.100:0/1516988217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5148105e50 0x7f51481062d0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.206+0000 7f514d1da640 1 --2- 172.21.15.100:0/1516988217 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5148104c60 0x7f5148105080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.206+0000 7f514d1da640 1 -- 172.21.15.100:0/1516988217 >> 172.21.15.100:0/1516988217 conn(0x7f51481001d0 msgr2=0x7f5148102610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.206+0000 7f514d1da640 1 -- 172.21.15.100:0/1516988217 shutdown_connections 2024-05-10T11:28:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.206+0000 7f514d1da640 1 -- 172.21.15.100:0/1516988217 wait complete. 2024-05-10T11:28:14.209 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.207+0000 7f514d1da640 1 Processor -- start 2024-05-10T11:28:14.209 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.207+0000 7f514d1da640 1 -- start start 2024-05-10T11:28:14.209 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.207+0000 7f514d1da640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5148104c60 0x7f514819ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:14.209 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.207+0000 7f514d1da640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5148105e50 0x7f514819b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:14.209 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.207+0000 7f514d1da640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5148106810 0x7f51481a22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:14.209 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.207+0000 7f514d1da640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5148110620 con 0x7f5148104c60 2024-05-10T11:28:14.209 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.207+0000 7f514d1da640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f51481104a0 con 0x7f5148105e50 2024-05-10T11:28:14.209 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.207+0000 7f514d1da640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f51481107a0 con 0x7f5148106810 2024-05-10T11:28:14.210 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.208+0000 7f5146d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5148104c60 0x7f514819ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:14.210 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.208+0000 7f5146d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5148104c60 0x7f514819ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44878/0 (socket says 172.21.15.100:44878) 2024-05-10T11:28:14.210 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.208+0000 7f5146d9d640 1 -- 172.21.15.100:0/3542241622 learned_addr learned my addr 172.21.15.100:0/3542241622 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:28:14.210 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.208+0000 7f514659c640 1 --2- 172.21.15.100:0/3542241622 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5148105e50 0x7f514819b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:14.210 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.208+0000 7f514759e640 1 --2- 172.21.15.100:0/3542241622 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5148106810 0x7f51481a22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:14.210 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.208+0000 7f5146d9d640 1 -- 172.21.15.100:0/3542241622 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5148106810 msgr2=0x7f51481a22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:14.210 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.208+0000 7f5146d9d640 1 --2- 172.21.15.100:0/3542241622 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5148106810 0x7f51481a22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:14.210 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.208+0000 7f5146d9d640 1 -- 172.21.15.100:0/3542241622 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5148105e50 msgr2=0x7f514819b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:14.210 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.208+0000 7f5146d9d640 1 --2- 172.21.15.100:0/3542241622 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5148105e50 0x7f514819b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:14.210 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.208+0000 7f5146d9d640 1 -- 172.21.15.100:0/3542241622 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51481a2950 con 0x7f5148104c60 2024-05-10T11:28:14.210 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.208+0000 7f5146d9d640 1 --2- 172.21.15.100:0/3542241622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5148104c60 0x7f514819ad40 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f513800e9c0 tx=0x7f513800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:14.210 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.208+0000 7f512ffff640 1 -- 172.21.15.100:0/3542241622 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513800cd80 con 0x7f5148104c60 2024-05-10T11:28:14.210 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.208+0000 7f514d1da640 1 -- 172.21.15.100:0/3542241622 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51481a2c40 con 0x7f5148104c60 2024-05-10T11:28:14.210 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.208+0000 7f512ffff640 1 -- 172.21.15.100:0/3542241622 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f51380044b0 con 0x7f5148104c60 2024-05-10T11:28:14.211 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.208+0000 7f512ffff640 1 -- 172.21.15.100:0/3542241622 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5138010430 con 0x7f5148104c60 2024-05-10T11:28:14.211 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.208+0000 7f514d1da640 1 -- 172.21.15.100:0/3542241622 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51481a3190 con 0x7f5148104c60 2024-05-10T11:28:14.213 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.211+0000 7f512ffff640 1 -- 172.21.15.100:0/3542241622 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f5138003f00 con 0x7f5148104c60 2024-05-10T11:28:14.213 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.211+0000 7f514ffd0740 1 Processor -- start 2024-05-10T11:28:14.213 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.211+0000 7f514ffd0740 1 -- start start 2024-05-10T11:28:14.213 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.211+0000 7f514ffd0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560b1c1afdb0 0x560b1c1b01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.211+0000 7f514ffd0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b1c1b07a0 0x560b1c1b52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.211+0000 7f514ffd0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b1c1b5910 0x560b1c1b7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.211+0000 7f514ffd0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x560b1c1145d0 con 0x560b1c1b07a0 2024-05-10T11:28:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.211+0000 7f514ffd0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560b1c0cd890 con 0x560b1c1b5910 2024-05-10T11:28:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.211+0000 7f514ffd0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x560b1c110590 con 0x560b1c1afdb0 2024-05-10T11:28:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.211+0000 7f514759e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560b1c1afdb0 0x560b1c1b01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.211+0000 7f5146d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b1c1b07a0 0x560b1c1b52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.211+0000 7f514759e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560b1c1afdb0 0x560b1c1b01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:43746/0 (socket says 172.21.15.100:43746) 2024-05-10T11:28:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.211+0000 7f514759e640 1 -- 172.21.15.100:0/3462695171 learned_addr learned my addr 172.21.15.100:0/3462695171 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:28:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.211+0000 7f514659c640 1 --2- 172.21.15.100:0/3462695171 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b1c1b5910 0x560b1c1b7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.212+0000 7f514759e640 1 -- 172.21.15.100:0/3462695171 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b1c1b5910 msgr2=0x560b1c1b7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.212+0000 7f514759e640 1 --2- 172.21.15.100:0/3462695171 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b1c1b5910 0x560b1c1b7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.212+0000 7f512ffff640 1 --2- 172.21.15.100:0/3542241622 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f51300b28a0 0x7f51300b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.212+0000 7f514759e640 1 -- 172.21.15.100:0/3462695171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b1c1b07a0 msgr2=0x560b1c1b52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.212+0000 7f514759e640 1 --2- 172.21.15.100:0/3462695171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b1c1b07a0 0x560b1c1b52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.212+0000 7f514759e640 1 -- 172.21.15.100:0/3462695171 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b1be26210 con 0x560b1c1afdb0 2024-05-10T11:28:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.212+0000 7f5146d9d640 1 --2- 172.21.15.100:0/3462695171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b1c1b07a0 0x560b1c1b52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:28:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.212+0000 7f512ffff640 1 -- 172.21.15.100:0/3542241622 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f51380e29c0 con 0x7f5148104c60 2024-05-10T11:28:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.212+0000 7f514759e640 1 --2- 172.21.15.100:0/3542241622 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f51300b28a0 0x7f51300b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.212+0000 7f514759e640 1 --2- 172.21.15.100:0/3462695171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560b1c1afdb0 0x560b1c1b01d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f51400039f0 tx=0x7f5140003eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23d243ae736a83fc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.212+0000 7f512d7fa640 1 -- 172.21.15.100:0/3462695171 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5140022070 con 0x560b1c1afdb0 2024-05-10T11:28:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.212+0000 7f512d7fa640 1 -- 172.21.15.100:0/3462695171 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5140015070 con 0x560b1c1afdb0 2024-05-10T11:28:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.212+0000 7f512d7fa640 1 -- 172.21.15.100:0/3462695171 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5140010860 con 0x560b1c1afdb0 2024-05-10T11:28:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.213+0000 7f514ffd0740 1 -- 172.21.15.100:0/3462695171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560b1c1afdb0 msgr2=0x560b1c1b01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.213+0000 7f514ffd0740 1 --2- 172.21.15.100:0/3462695171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560b1c1afdb0 0x560b1c1b01d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f51400039f0 tx=0x7f5140003eb0 comp rx=0 tx=0).stop 2024-05-10T11:28:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.213+0000 7f514759e640 1 --2- 172.21.15.100:0/3542241622 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f51300b28a0 0x7f51300b4d60 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f514001f000 tx=0x7f5140020000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.213+0000 7f514ffd0740 1 -- 172.21.15.100:0/3462695171 shutdown_connections 2024-05-10T11:28:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.213+0000 7f514ffd0740 1 --2- 172.21.15.100:0/3462695171 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b1c1b5910 0x560b1c1b7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.213+0000 7f514ffd0740 1 --2- 172.21.15.100:0/3462695171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b1c1b07a0 0x560b1c1b52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.213+0000 7f514ffd0740 1 --2- 172.21.15.100:0/3462695171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560b1c1afdb0 0x560b1c1b01d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:14.216 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.213+0000 7f514ffd0740 1 -- 172.21.15.100:0/3462695171 >> 172.21.15.100:0/3462695171 conn(0x560b1c1ab420 msgr2=0x560b1c1ab820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:14.216 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.213+0000 7f514ffd0740 1 -- 172.21.15.100:0/3462695171 shutdown_connections 2024-05-10T11:28:14.216 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.213+0000 7f514ffd0740 1 -- 172.21.15.100:0/3462695171 wait complete. 2024-05-10T11:28:14.216 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.213+0000 7f514ffd0740 1 -- start start 2024-05-10T11:28:14.216 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.214+0000 7f514ffd0740 1 Processor -- start 2024-05-10T11:28:14.219 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.214+0000 7f514ffd0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b1c1afdb0 0x560b1c1b9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:14.219 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.214+0000 7f514ffd0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b1c1b07a0 0x560b1c1ba500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:14.220 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.214+0000 7f514ffd0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560b1c1b5910 0x560b1c1c1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:14.220 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.214+0000 7f514ffd0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x560b1c1b88c0 con 0x560b1c1afdb0 2024-05-10T11:28:14.220 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.214+0000 7f514ffd0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560b1c0d1370 con 0x560b1c1b07a0 2024-05-10T11:28:14.220 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.214+0000 7f514ffd0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x560b1c1b8a40 con 0x560b1c1b5910 2024-05-10T11:28:14.220 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.214+0000 7f514ffd0740 10 client.0 _ll_register_callbacks cb 0x560b1be25a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:28:14.220 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.214+0000 7f5146d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b1c1afdb0 0x560b1c1b9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:14.220 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.214+0000 7f5146d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b1c1afdb0 0x560b1c1b9fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44900/0 (socket says 172.21.15.100:44900) 2024-05-10T11:28:14.220 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.214+0000 7f5146d9d640 1 -- 172.21.15.100:0/2699291440 learned_addr learned my addr 172.21.15.100:0/2699291440 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:28:14.220 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.214+0000 7f514759e640 1 --2- 172.21.15.100:0/2699291440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560b1c1b5910 0x560b1c1c1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:14.220 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.214+0000 7f514659c640 1 --2- 172.21.15.100:0/2699291440 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b1c1b07a0 0x560b1c1ba500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:14.220 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.214+0000 7f5146d9d640 1 -- 172.21.15.100:0/2699291440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560b1c1b5910 msgr2=0x560b1c1c1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:14.220 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.214+0000 7f5146d9d640 1 --2- 172.21.15.100:0/2699291440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560b1c1b5910 0x560b1c1c1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:14.220 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.214+0000 7f5146d9d640 1 -- 172.21.15.100:0/2699291440 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b1c1b07a0 msgr2=0x560b1c1ba500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:14.220 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.214+0000 7f5146d9d640 1 --2- 172.21.15.100:0/2699291440 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b1c1b07a0 0x560b1c1ba500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:14.220 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.214+0000 7f5146d9d640 1 -- 172.21.15.100:0/2699291440 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b1bef2620 con 0x560b1c1afdb0 2024-05-10T11:28:14.221 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.214+0000 7f514659c640 1 --2- 172.21.15.100:0/2699291440 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b1c1b07a0 0x560b1c1ba500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:28:14.221 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.214+0000 7f5146d9d640 1 --2- 172.21.15.100:0/2699291440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b1c1afdb0 0x560b1c1b9fc0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f51380e0160 tx=0x7f5138009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=177977f0f17d5865 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:14.221 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.214+0000 7f512aff5640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:28:14.221 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.215+0000 7f512aff5640 1 -- 172.21.15.100:0/2699291440 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51380a95d0 con 0x560b1c1afdb0 2024-05-10T11:28:14.221 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.215+0000 7f512aff5640 1 -- 172.21.15.100:0/2699291440 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f51380041f0 con 0x560b1c1afdb0 2024-05-10T11:28:14.221 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.215+0000 7f514ffd0740 1 -- 172.21.15.100:0/2699291440 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5140015070 con 0x560b1c1afdb0 2024-05-10T11:28:14.221 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.215+0000 7f512aff5640 1 -- 172.21.15.100:0/2699291440 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f51380a93f0 con 0x560b1c1afdb0 2024-05-10T11:28:14.221 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.215+0000 7f514ffd0740 10 client.17385 fetch_fsmap learned FSMap version 31 2024-05-10T11:28:14.221 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.215+0000 7f514ffd0740 1 -- 172.21.15.100:0/2699291440 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560b1c1c45e0 con 0x560b1c1afdb0 2024-05-10T11:28:14.221 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.215+0000 7f512aff5640 1 -- 172.21.15.100:0/2699291440 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f51380c2070 con 0x560b1c1afdb0 2024-05-10T11:28:14.221 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.215+0000 7f512aff5640 1 -- 172.21.15.100:0/2699291440 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f51380c4140 con 0x560b1c1afdb0 2024-05-10T11:28:14.221 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.215+0000 7f514ffd0740 10 client.17385 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:28:14.221 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.215+0000 7f514ffd0740 10 client.17385 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:28:14.221 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.215+0000 7f514ffd0740 20 client.17385 populate_metadata read hostname 'smithi100' 2024-05-10T11:28:14.221 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.215+0000 7f514ffd0740 1 --2- 172.21.15.100:0/2699291440 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x560b1c1c5c10 0x560b1c1c8060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:14.222 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.215+0000 7f514ffd0740 4 client.17385 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:28:14.222 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.215+0000 7f514ffd0740 1 -- 172.21.15.100:0/2699291440 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b1c1221e0 con 0x560b1c1c5c10 2024-05-10T11:28:14.222 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.215+0000 7f514659c640 1 --2- 172.21.15.100:0/2699291440 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x560b1c1c5c10 0x560b1c1c8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:14.222 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.216+0000 7f514659c640 1 --2- 172.21.15.100:0/2699291440 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x560b1c1c5c10 0x560b1c1c8060 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1eb68b72ca208966 server_cookie=ef93bd20da80566c in_seq=0 out_seq=0 2024-05-10T11:28:14.222 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.216+0000 7f512aff5640 10 client.17385 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:28:14.222 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.216+0000 7f512aff5640 1 -- 172.21.15.100:0/2699291440 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560b1c1221e0 con 0x560b1c1c5c10 2024-05-10T11:28:14.222 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.216+0000 7f512aff5640 10 client.17385 handle_command_reply: tid=0 2024-05-10T11:28:14.222 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.217+0000 7f514ffd0740 1 client.17385 shutdown 2024-05-10T11:28:14.222 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.217+0000 7f514ffd0740 1 -- 172.21.15.100:0/2699291440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b1c1afdb0 msgr2=0x560b1c1b9fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:14.222 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.217+0000 7f514ffd0740 1 --2- 172.21.15.100:0/2699291440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b1c1afdb0 0x560b1c1b9fc0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f51380e0160 tx=0x7f5138009700 comp rx=0 tx=0).stop 2024-05-10T11:28:14.222 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.217+0000 7f514ffd0740 1 -- 172.21.15.100:0/2699291440 shutdown_connections 2024-05-10T11:28:14.222 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.217+0000 7f514ffd0740 1 --2- 172.21.15.100:0/2699291440 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x560b1c1c5c10 0x560b1c1c8060 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:14.222 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.217+0000 7f514ffd0740 1 --2- 172.21.15.100:0/2699291440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560b1c1b5910 0x560b1c1c1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:14.222 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.217+0000 7f514ffd0740 1 --2- 172.21.15.100:0/2699291440 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b1c1b07a0 0x560b1c1ba500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:14.223 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.217+0000 7f512aff5640 0 client.17385 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:28:14.223 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.217+0000 7f514ffd0740 1 --2- 172.21.15.100:0/2699291440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b1c1afdb0 0x560b1c1b9fc0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:14.223 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.217+0000 7f514ffd0740 1 -- 172.21.15.100:0/2699291440 >> 172.21.15.100:0/2699291440 conn(0x560b1c1ab420 msgr2=0x560b1c1aa580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:14.223 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.217+0000 7f514ffd0740 1 -- 172.21.15.100:0/2699291440 shutdown_connections 2024-05-10T11:28:14.223 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.217+0000 7f514ffd0740 1 -- 172.21.15.100:0/2699291440 wait complete. 2024-05-10T11:28:14.223 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.217+0000 7f514ffd0740 20 client.17385 trim_cache size 0 max 16384 2024-05-10T11:28:14.234 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.233+0000 7f514ffd0740 1 Processor -- start 2024-05-10T11:28:14.234 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.233+0000 7f514ffd0740 1 -- start start 2024-05-10T11:28:14.234 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.233+0000 7f514ffd0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b1c1afdb0 0x560b1c1b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:14.234 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.233+0000 7f514ffd0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b1c1b07a0 0x560b1c1b9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:14.234 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.233+0000 7f514ffd0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560b1c1b5910 0x560b1c1ba110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.233+0000 7f514ffd0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f51481afe10 con 0x560b1c1afdb0 2024-05-10T11:28:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.233+0000 7f514ffd0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f51481ad6e0 con 0x560b1c1b07a0 2024-05-10T11:28:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.233+0000 7f514ffd0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f51481ab530 con 0x560b1c1b5910 2024-05-10T11:28:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.233+0000 7f5146d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b1c1afdb0 0x560b1c1b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.233+0000 7f5146d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b1c1afdb0 0x560b1c1b4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44904/0 (socket says 172.21.15.100:44904) 2024-05-10T11:28:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.233+0000 7f5146d9d640 1 -- 172.21.15.100:0/1376581455 learned_addr learned my addr 172.21.15.100:0/1376581455 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:28:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.233+0000 7f514759e640 1 --2- 172.21.15.100:0/1376581455 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560b1c1b5910 0x560b1c1ba110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.233+0000 7f514659c640 1 --2- 172.21.15.100:0/1376581455 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b1c1b07a0 0x560b1c1b9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.234+0000 7f5146d9d640 1 -- 172.21.15.100:0/1376581455 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560b1c1b5910 msgr2=0x560b1c1ba110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.234+0000 7f5146d9d640 1 --2- 172.21.15.100:0/1376581455 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560b1c1b5910 0x560b1c1ba110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.234+0000 7f5146d9d640 1 -- 172.21.15.100:0/1376581455 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b1c1b07a0 msgr2=0x560b1c1b9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.234+0000 7f5146d9d640 1 --2- 172.21.15.100:0/1376581455 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b1c1b07a0 0x560b1c1b9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.234+0000 7f5146d9d640 1 -- 172.21.15.100:0/1376581455 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b1c1ab250 con 0x560b1c1afdb0 2024-05-10T11:28:14.236 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.234+0000 7f5146d9d640 1 --2- 172.21.15.100:0/1376581455 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b1c1afdb0 0x560b1c1b4fb0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f51380a5c70 tx=0x7f5138005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbb39cd2cb2e6dc8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:14.236 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.234+0000 7f512cff9640 1 -- 172.21.15.100:0/1376581455 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51380a95d0 con 0x560b1c1afdb0 2024-05-10T11:28:14.236 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.234+0000 7f512cff9640 1 -- 172.21.15.100:0/1376581455 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f51380c2070 con 0x560b1c1afdb0 2024-05-10T11:28:14.236 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.234+0000 7f512cff9640 1 -- 172.21.15.100:0/1376581455 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51380a5ab0 con 0x560b1c1afdb0 2024-05-10T11:28:14.239 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.235+0000 7f514ffd0740 1 -- 172.21.15.100:0/1376581455 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b1c1afdb0 msgr2=0x560b1c1b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:14.239 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.235+0000 7f514ffd0740 1 --2- 172.21.15.100:0/1376581455 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b1c1afdb0 0x560b1c1b4fb0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f51380a5c70 tx=0x7f5138005880 comp rx=0 tx=0).stop 2024-05-10T11:28:14.239 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.235+0000 7f514ffd0740 1 -- 172.21.15.100:0/1376581455 shutdown_connections 2024-05-10T11:28:14.239 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.235+0000 7f514ffd0740 1 --2- 172.21.15.100:0/1376581455 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560b1c1b5910 0x560b1c1ba110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:14.239 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.235+0000 7f514ffd0740 1 --2- 172.21.15.100:0/1376581455 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b1c1b07a0 0x560b1c1b9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.235+0000 7f514ffd0740 1 --2- 172.21.15.100:0/1376581455 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b1c1afdb0 0x560b1c1b4fb0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.235+0000 7f514ffd0740 1 -- 172.21.15.100:0/1376581455 >> 172.21.15.100:0/1376581455 conn(0x560b1c1ab420 msgr2=0x560b1c1a6420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.235+0000 7f514ffd0740 1 -- 172.21.15.100:0/1376581455 shutdown_connections 2024-05-10T11:28:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.235+0000 7f514ffd0740 1 -- 172.21.15.100:0/1376581455 wait complete. 2024-05-10T11:28:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.235+0000 7f514ffd0740 1 -- start start 2024-05-10T11:28:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.235+0000 7f514ffd0740 1 Processor -- start 2024-05-10T11:28:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.235+0000 7f514ffd0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b1c1afdb0 0x560b1c1be010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.235+0000 7f514ffd0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b1c1b07a0 0x560b1c1be550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.236+0000 7f514ffd0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560b1c1b5910 0x560b1c1bf560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.236+0000 7f514ffd0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f51481abba0 con 0x560b1c1b07a0 2024-05-10T11:28:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.236+0000 7f514ffd0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f51481ad010 con 0x560b1c1afdb0 2024-05-10T11:28:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.236+0000 7f514ffd0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f51481ac270 con 0x560b1c1b5910 2024-05-10T11:28:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.236+0000 7f514ffd0740 10 client.0 _ll_register_callbacks cb 0x7f51380046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:28:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.236+0000 7f514659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b1c1b07a0 0x560b1c1be550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.236+0000 7f514659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b1c1b07a0 0x560b1c1be550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44914/0 (socket says 172.21.15.100:44914) 2024-05-10T11:28:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.236+0000 7f514759e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560b1c1b5910 0x560b1c1bf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.236+0000 7f514659c640 1 -- 172.21.15.100:0/642850736 learned_addr learned my addr 172.21.15.100:0/642850736 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:28:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.236+0000 7f5146d9d640 1 --2- 172.21.15.100:0/642850736 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b1c1afdb0 0x560b1c1be010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.236+0000 7f514759e640 1 -- 172.21.15.100:0/642850736 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b1c1afdb0 msgr2=0x560b1c1be010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.236+0000 7f514759e640 1 --2- 172.21.15.100:0/642850736 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b1c1afdb0 0x560b1c1be010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.236+0000 7f514759e640 1 -- 172.21.15.100:0/642850736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b1c1b07a0 msgr2=0x560b1c1be550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.236+0000 7f514759e640 1 --2- 172.21.15.100:0/642850736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b1c1b07a0 0x560b1c1be550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.236+0000 7f514759e640 1 -- 172.21.15.100:0/642850736 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b1c1bfb50 con 0x560b1c1b5910 2024-05-10T11:28:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.236+0000 7f514659c640 1 --2- 172.21.15.100:0/642850736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b1c1b07a0 0x560b1c1be550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:28:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.236+0000 7f5146d9d640 1 --2- 172.21.15.100:0/642850736 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b1c1afdb0 0x560b1c1be010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:28:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.236+0000 7f514759e640 1 --2- 172.21.15.100:0/642850736 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560b1c1b5910 0x560b1c1bf560 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5140004780 tx=0x7f5140005c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=474acabde15ee300 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.236+0000 7f512aff5640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:28:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.236+0000 7f512aff5640 1 -- 172.21.15.100:0/642850736 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5140022040 con 0x560b1c1b5910 2024-05-10T11:28:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.236+0000 7f512aff5640 1 -- 172.21.15.100:0/642850736 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f5140010d00 con 0x560b1c1b5910 2024-05-10T11:28:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.236+0000 7f514ffd0740 1 -- 172.21.15.100:0/642850736 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51380c2070 con 0x560b1c1b5910 2024-05-10T11:28:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.237+0000 7f512aff5640 1 -- 172.21.15.100:0/642850736 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f514003e050 con 0x560b1c1b5910 2024-05-10T11:28:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.237+0000 7f514ffd0740 10 client.26845 fetch_fsmap learned FSMap version 31 2024-05-10T11:28:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.237+0000 7f514ffd0740 1 -- 172.21.15.100:0/642850736 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560b1c1c0e60 con 0x560b1c1b5910 2024-05-10T11:28:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.237+0000 7f512aff5640 1 -- 172.21.15.100:0/642850736 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f51380c2070 con 0x560b1c1b5910 2024-05-10T11:28:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.237+0000 7f512aff5640 1 -- 172.21.15.100:0/642850736 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f514002f070 con 0x560b1c1b5910 2024-05-10T11:28:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.237+0000 7f514ffd0740 10 client.26845 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:28:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.237+0000 7f514ffd0740 10 client.26845 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:28:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.237+0000 7f514ffd0740 20 client.26845 populate_metadata read hostname 'smithi100' 2024-05-10T11:28:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.237+0000 7f514ffd0740 1 --2- 172.21.15.100:0/642850736 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x560b1c2309b0 0x560b1c232dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.237+0000 7f514ffd0740 4 client.26845 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:28:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.237+0000 7f514ffd0740 1 -- 172.21.15.100:0/642850736 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560b1c11c050 con 0x560b1c2309b0 2024-05-10T11:28:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.237+0000 7f5146d9d640 1 --2- 172.21.15.100:0/642850736 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x560b1c2309b0 0x560b1c232dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.238+0000 7f512aff5640 10 client.26845 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:28:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.238+0000 7f5146d9d640 1 --2- 172.21.15.100:0/642850736 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x560b1c2309b0 0x560b1c232dd0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3893e42c0884c00 server_cookie=f2dd4a3670f52282 in_seq=0 out_seq=0 2024-05-10T11:28:14.261 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.260+0000 7f512aff5640 1 -- 172.21.15.100:0/642850736 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560b1c11c050 con 0x560b1c2309b0 2024-05-10T11:28:14.261 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.260+0000 7f512aff5640 10 client.26845 handle_command_reply: tid=0 2024-05-10T11:28:14.261 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.260+0000 7f514ffd0740 1 client.26845 shutdown 2024-05-10T11:28:14.261 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.260+0000 7f514ffd0740 1 -- 172.21.15.100:0/642850736 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560b1c1b5910 msgr2=0x560b1c1bf560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:14.261 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.260+0000 7f514ffd0740 1 --2- 172.21.15.100:0/642850736 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560b1c1b5910 0x560b1c1bf560 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5140004780 tx=0x7f5140005c10 comp rx=0 tx=0).stop 2024-05-10T11:28:14.261 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.260+0000 7f514ffd0740 1 -- 172.21.15.100:0/642850736 shutdown_connections 2024-05-10T11:28:14.261 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.260+0000 7f514ffd0740 1 --2- 172.21.15.100:0/642850736 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x560b1c2309b0 0x560b1c232dd0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:14.262 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.260+0000 7f514ffd0740 1 --2- 172.21.15.100:0/642850736 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560b1c1b5910 0x560b1c1bf560 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:14.262 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.261+0000 7f512aff5640 0 client.26845 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:28:14.262 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.260+0000 7f514ffd0740 1 --2- 172.21.15.100:0/642850736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b1c1b07a0 0x560b1c1be550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:14.262 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.261+0000 7f514ffd0740 1 --2- 172.21.15.100:0/642850736 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b1c1afdb0 0x560b1c1be010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:14.262 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.261+0000 7f514ffd0740 1 -- 172.21.15.100:0/642850736 >> 172.21.15.100:0/642850736 conn(0x560b1c1ab420 msgr2=0x560b1c1b4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:14.262 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.261+0000 7f514ffd0740 1 -- 172.21.15.100:0/642850736 shutdown_connections 2024-05-10T11:28:14.262 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.261+0000 7f514ffd0740 1 -- 172.21.15.100:0/642850736 wait complete. 2024-05-10T11:28:14.262 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.261+0000 7f514ffd0740 20 client.26845 trim_cache size 0 max 16384 2024-05-10T11:28:14.262 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:28:14.262 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:28:14.263 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:28:14.263 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:28:14.263 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.261+0000 7f514d1da640 1 -- 172.21.15.100:0/3542241622 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f51300b28a0 msgr2=0x7f51300b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:14.263 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.261+0000 7f514d1da640 1 --2- 172.21.15.100:0/3542241622 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f51300b28a0 0x7f51300b4d60 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f514001f000 tx=0x7f5140020000 comp rx=0 tx=0).stop 2024-05-10T11:28:14.263 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.261+0000 7f514d1da640 1 -- 172.21.15.100:0/3542241622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5148104c60 msgr2=0x7f514819ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:14.264 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.261+0000 7f514d1da640 1 --2- 172.21.15.100:0/3542241622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5148104c60 0x7f514819ad40 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f513800e9c0 tx=0x7f513800ee80 comp rx=0 tx=0).stop 2024-05-10T11:28:14.264 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.261+0000 7f514d1da640 1 -- 172.21.15.100:0/3542241622 shutdown_connections 2024-05-10T11:28:14.264 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.261+0000 7f514d1da640 1 --2- 172.21.15.100:0/3542241622 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f51300b28a0 0x7f51300b4d60 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:14.264 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.261+0000 7f514d1da640 1 --2- 172.21.15.100:0/3542241622 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5148106810 0x7f51481a22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:14.264 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.261+0000 7f514d1da640 1 --2- 172.21.15.100:0/3542241622 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5148105e50 0x7f514819b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:14.264 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.261+0000 7f514d1da640 1 --2- 172.21.15.100:0/3542241622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5148104c60 0x7f514819ad40 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:14.264 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.261+0000 7f514d1da640 1 -- 172.21.15.100:0/3542241622 >> 172.21.15.100:0/3542241622 conn(0x7f51481001d0 msgr2=0x7f5148109290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:14.264 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.262+0000 7f514d1da640 1 -- 172.21.15.100:0/3542241622 shutdown_connections 2024-05-10T11:28:14.264 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:14.262+0000 7f514d1da640 1 -- 172.21.15.100:0/3542241622 wait complete. 2024-05-10T11:28:14.979 INFO:tasks.workunit.client.0.smithi100.stdout: 1 251863 2.74 MB/sec execute 369 sec latency 92.044 ms 2024-05-10T11:28:15.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:28:15 smithi178 ceph-mon[33359]: pgmap v716: 129 pgs: 129 active+clean; 103 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 1.3 MiB/s wr, 146 op/s 2024-05-10T11:28:15.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:28:15 smithi188 ceph-mon[34075]: pgmap v716: 129 pgs: 129 active+clean; 103 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 1.3 MiB/s wr, 146 op/s 2024-05-10T11:28:15.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:28:15 smithi100 ceph-mon[30296]: pgmap v716: 129 pgs: 129 active+clean; 103 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 1.3 MiB/s wr, 146 op/s 2024-05-10T11:28:15.979 INFO:tasks.workunit.client.0.smithi100.stdout: 1 252204 2.74 MB/sec execute 370 sec latency 29.565 ms 2024-05-10T11:28:16.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.582+0000 7f6c33016700 1 -- 172.21.15.100:0/3313047995 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c2c102d90 msgr2=0x7f6c2c10f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:16.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.582+0000 7f6c33016700 1 --2- 172.21.15.100:0/3313047995 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c2c102d90 0x7f6c2c10f280 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f6c2400b0a0 tx=0x7f6c2400b3b0 comp rx=0 tx=0).stop 2024-05-10T11:28:16.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.582+0000 7f6c33016700 1 -- 172.21.15.100:0/3313047995 shutdown_connections 2024-05-10T11:28:16.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.582+0000 7f6c33016700 1 --2- 172.21.15.100:0/3313047995 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c2c102d90 0x7f6c2c10f280 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:16.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.582+0000 7f6c33016700 1 --2- 172.21.15.100:0/3313047995 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c2c108440 0x7f6c2c108820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:16.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.582+0000 7f6c33016700 1 --2- 172.21.15.100:0/3313047995 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c2c1023d0 0x7f6c2c102850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:16.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.582+0000 7f6c33016700 1 -- 172.21.15.100:0/3313047995 >> 172.21.15.100:0/3313047995 conn(0x7f6c2c0793e0 msgr2=0x7f6c2c0797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:16.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.583+0000 7f6c33016700 1 -- 172.21.15.100:0/3313047995 shutdown_connections 2024-05-10T11:28:16.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.583+0000 7f6c33016700 1 -- 172.21.15.100:0/3313047995 wait complete. 2024-05-10T11:28:16.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.583+0000 7f6c33016700 1 Processor -- start 2024-05-10T11:28:16.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.583+0000 7f6c33016700 1 -- start start 2024-05-10T11:28:16.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.583+0000 7f6c33016700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c2c1023d0 0x7f6c2c19e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:16.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.583+0000 7f6c33016700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c2c102d90 0x7f6c2c19ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:16.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.583+0000 7f6c33016700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c2c108440 0x7f6c2c1988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:16.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.584+0000 7f6c33016700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2c112440 con 0x7f6c2c102d90 2024-05-10T11:28:16.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.584+0000 7f6c33016700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2c1122c0 con 0x7f6c2c1023d0 2024-05-10T11:28:16.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.584+0000 7f6c33016700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2c1125c0 con 0x7f6c2c108440 2024-05-10T11:28:16.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.584+0000 7f6c2bfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c2c102d90 0x7f6c2c19ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:16.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.584+0000 7f6c30db2700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c2c1023d0 0x7f6c2c19e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:16.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.584+0000 7f6c30db2700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c2c1023d0 0x7f6c2c19e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:44210/0 (socket says 172.21.15.100:44210) 2024-05-10T11:28:16.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.584+0000 7f6c2bfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c2c102d90 0x7f6c2c19ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38306/0 (socket says 172.21.15.100:38306) 2024-05-10T11:28:16.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.584+0000 7f6c30db2700 1 -- 172.21.15.100:0/3526855277 learned_addr learned my addr 172.21.15.100:0/3526855277 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:28:16.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.584+0000 7f6c315b3700 1 --2- 172.21.15.100:0/3526855277 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c2c108440 0x7f6c2c1988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:16.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.584+0000 7f6c30db2700 1 -- 172.21.15.100:0/3526855277 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c2c108440 msgr2=0x7f6c2c1988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:16.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.584+0000 7f6c30db2700 1 --2- 172.21.15.100:0/3526855277 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c2c108440 0x7f6c2c1988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:16.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.584+0000 7f6c30db2700 1 -- 172.21.15.100:0/3526855277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c2c102d90 msgr2=0x7f6c2c19ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:16.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.584+0000 7f6c30db2700 1 --2- 172.21.15.100:0/3526855277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c2c102d90 0x7f6c2c19ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:16.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.584+0000 7f6c30db2700 1 -- 172.21.15.100:0/3526855277 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c2c199000 con 0x7f6c2c1023d0 2024-05-10T11:28:16.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.584+0000 7f6c30db2700 1 --2- 172.21.15.100:0/3526855277 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c2c1023d0 0x7f6c2c19e6e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6c2000e9a0 tx=0x7f6c2000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:16.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.584+0000 7f6c29ffb700 1 -- 172.21.15.100:0/3526855277 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c2000ca10 con 0x7f6c2c1023d0 2024-05-10T11:28:16.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.584+0000 7f6c29ffb700 1 -- 172.21.15.100:0/3526855277 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6c2000cbb0 con 0x7f6c2c1023d0 2024-05-10T11:28:16.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.584+0000 7f6c33016700 1 -- 172.21.15.100:0/3526855277 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c2c199290 con 0x7f6c2c1023d0 2024-05-10T11:28:16.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.584+0000 7f6c29ffb700 1 -- 172.21.15.100:0/3526855277 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c20018680 con 0x7f6c2c1023d0 2024-05-10T11:28:16.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.585+0000 7f6c33016700 1 -- 172.21.15.100:0/3526855277 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c2c1a55b0 con 0x7f6c2c1023d0 2024-05-10T11:28:16.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.587+0000 7f6c29ffb700 1 -- 172.21.15.100:0/3526855277 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f6c20010430 con 0x7f6c2c1023d0 2024-05-10T11:28:16.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.588+0000 7f6c29ffb700 1 --2- 172.21.15.100:0/3526855277 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f6c1c0b2880 0x7f6c1c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:16.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.588+0000 7f6c2bfff700 1 --2- 172.21.15.100:0/3526855277 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f6c1c0b2880 0x7f6c1c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:16.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.588+0000 7f6c29ffb700 1 -- 172.21.15.100:0/3526855277 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f6c200e2710 con 0x7f6c2c1023d0 2024-05-10T11:28:16.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.588+0000 7f6c3d8a9b80 1 Processor -- start 2024-05-10T11:28:16.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.588+0000 7f6c3d8a9b80 1 -- start start 2024-05-10T11:28:16.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.588+0000 7f6c2bfff700 1 --2- 172.21.15.100:0/3526855277 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f6c1c0b2880 0x7f6c1c0b4d40 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c079b10 tx=0x7f6c18009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:16.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.588+0000 7f6c3d8a9b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564811590020 0x564811590400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:16.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.588+0000 7f6c3d8a9b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56481158a780 0x56481158ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:16.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.589+0000 7f6c3d8a9b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56481158b140 0x564811596d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:16.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.589+0000 7f6c3d8a9b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5648114d88e0 con 0x564811590020 2024-05-10T11:28:16.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.589+0000 7f6c3d8a9b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x564811477840 con 0x56481158b140 2024-05-10T11:28:16.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.589+0000 7f6c3d8a9b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5648114d48a0 con 0x56481158a780 2024-05-10T11:28:16.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.589+0000 7f6c30db2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564811590020 0x564811590400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:16.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.589+0000 7f6c30db2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564811590020 0x564811590400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38316/0 (socket says 172.21.15.100:38316) 2024-05-10T11:28:16.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.589+0000 7f6c30db2700 1 -- 172.21.15.100:0/4082881183 learned_addr learned my addr 172.21.15.100:0/4082881183 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:28:16.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.589+0000 7f6c315b3700 1 --2- 172.21.15.100:0/4082881183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56481158b140 0x564811596d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:16.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.589+0000 7f6c2bfff700 1 --2- 172.21.15.100:0/4082881183 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56481158a780 0x56481158ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:16.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.589+0000 7f6c30db2700 1 -- 172.21.15.100:0/4082881183 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56481158a780 msgr2=0x56481158ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:16.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.589+0000 7f6c30db2700 1 --2- 172.21.15.100:0/4082881183 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56481158a780 0x56481158ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:16.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.589+0000 7f6c30db2700 1 -- 172.21.15.100:0/4082881183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56481158b140 msgr2=0x564811596d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:16.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.589+0000 7f6c30db2700 1 --2- 172.21.15.100:0/4082881183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56481158b140 0x564811596d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:16.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.589+0000 7f6c30db2700 1 -- 172.21.15.100:0/4082881183 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564811425950 con 0x564811590020 2024-05-10T11:28:16.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.589+0000 7f6c315b3700 1 --2- 172.21.15.100:0/4082881183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56481158b140 0x564811596d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:28:16.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.589+0000 7f6c30db2700 1 --2- 172.21.15.100:0/4082881183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564811590020 0x564811590400 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f6c200a59f0 tx=0x7f6c200a51d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e4ef9d32caddfa server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:16.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.589+0000 7f6c16ffd700 1 -- 172.21.15.100:0/4082881183 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c200b2120 con 0x564811590020 2024-05-10T11:28:16.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.589+0000 7f6c16ffd700 1 -- 172.21.15.100:0/4082881183 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6c200aa070 con 0x564811590020 2024-05-10T11:28:16.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.589+0000 7f6c16ffd700 1 -- 172.21.15.100:0/4082881183 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c200cc0b0 con 0x564811590020 2024-05-10T11:28:16.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.590+0000 7f6c3d8a9b80 1 -- 172.21.15.100:0/4082881183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564811590020 msgr2=0x564811590400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:16.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.590+0000 7f6c3d8a9b80 1 --2- 172.21.15.100:0/4082881183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564811590020 0x564811590400 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f6c200a59f0 tx=0x7f6c200a51d0 comp rx=0 tx=0).stop 2024-05-10T11:28:16.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.590+0000 7f6c3d8a9b80 1 -- 172.21.15.100:0/4082881183 shutdown_connections 2024-05-10T11:28:16.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.590+0000 7f6c3d8a9b80 1 --2- 172.21.15.100:0/4082881183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56481158b140 0x564811596d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:16.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.590+0000 7f6c3d8a9b80 1 --2- 172.21.15.100:0/4082881183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564811590020 0x564811590400 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:16.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.590+0000 7f6c3d8a9b80 1 --2- 172.21.15.100:0/4082881183 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56481158a780 0x56481158ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:16.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.590+0000 7f6c3d8a9b80 1 -- 172.21.15.100:0/4082881183 >> 172.21.15.100:0/4082881183 conn(0x5648114b2180 msgr2=0x564811580b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:16.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.590+0000 7f6c3d8a9b80 1 -- 172.21.15.100:0/4082881183 shutdown_connections 2024-05-10T11:28:16.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.590+0000 7f6c3d8a9b80 1 -- 172.21.15.100:0/4082881183 wait complete. 2024-05-10T11:28:16.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.591+0000 7f6c3d8a9b80 1 -- start start 2024-05-10T11:28:16.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.591+0000 7f6c3d8a9b80 1 Processor -- start 2024-05-10T11:28:16.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.591+0000 7f6c3d8a9b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56481158a780 0x564811596900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:16.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.591+0000 7f6c3d8a9b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56481158b140 0x564811593ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:16.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.591+0000 7f6c3d8a9b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564811590020 0x5648115941e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:16.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.591+0000 7f6c3d8a9b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5648115974b0 con 0x564811590020 2024-05-10T11:28:16.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.591+0000 7f6c3d8a9b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56481143ecb0 con 0x56481158a780 2024-05-10T11:28:16.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.591+0000 7f6c3d8a9b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x564811597630 con 0x56481158b140 2024-05-10T11:28:16.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.591+0000 7f6c3d8a9b80 10 client.0 _ll_register_callbacks cb 0x5648114709a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:28:16.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.591+0000 7f6c315b3700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564811590020 0x5648115941e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:16.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.591+0000 7f6c2bfff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56481158b140 0x564811593ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:16.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.591+0000 7f6c315b3700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564811590020 0x5648115941e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38324/0 (socket says 172.21.15.100:38324) 2024-05-10T11:28:16.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.591+0000 7f6c2bfff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56481158b140 0x564811593ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:44668/0 (socket says 172.21.15.100:44668) 2024-05-10T11:28:16.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.591+0000 7f6c315b3700 1 -- 172.21.15.100:0/1069312547 learned_addr learned my addr 172.21.15.100:0/1069312547 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:28:16.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.591+0000 7f6c30db2700 1 --2- 172.21.15.100:0/1069312547 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56481158a780 0x564811596900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:16.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.592+0000 7f6c2bfff700 1 -- 172.21.15.100:0/1069312547 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56481158a780 msgr2=0x564811596900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:16.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.592+0000 7f6c2bfff700 1 --2- 172.21.15.100:0/1069312547 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56481158a780 0x564811596900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:16.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.592+0000 7f6c2bfff700 1 -- 172.21.15.100:0/1069312547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564811590020 msgr2=0x5648115941e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:16.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.592+0000 7f6c2bfff700 1 --2- 172.21.15.100:0/1069312547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564811590020 0x5648115941e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:16.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.592+0000 7f6c2bfff700 1 -- 172.21.15.100:0/1069312547 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564811474710 con 0x56481158b140 2024-05-10T11:28:16.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.592+0000 7f6c315b3700 1 --2- 172.21.15.100:0/1069312547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564811590020 0x5648115941e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:28:16.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.592+0000 7f6c2bfff700 1 --2- 172.21.15.100:0/1069312547 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56481158b140 0x564811593ca0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x564811582f10 tx=0x7f6c180047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7fae52331c76ef4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:16.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.592+0000 7f6c0bfff700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:28:16.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.592+0000 7f6c0bfff700 1 -- 172.21.15.100:0/1069312547 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c18004830 con 0x56481158b140 2024-05-10T11:28:16.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.592+0000 7f6c0bfff700 1 -- 172.21.15.100:0/1069312547 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f6c180049d0 con 0x56481158b140 2024-05-10T11:28:16.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.592+0000 7f6c3d8a9b80 1 -- 172.21.15.100:0/1069312547 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c200aa070 con 0x56481158b140 2024-05-10T11:28:16.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.592+0000 7f6c0bfff700 1 -- 172.21.15.100:0/1069312547 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6c18030050 con 0x56481158b140 2024-05-10T11:28:16.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.592+0000 7f6c3d8a9b80 10 client.26857 fetch_fsmap learned FSMap version 31 2024-05-10T11:28:16.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.592+0000 7f6c3d8a9b80 1 -- 172.21.15.100:0/1069312547 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5648115a1d10 con 0x56481158b140 2024-05-10T11:28:16.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.592+0000 7f6c0bfff700 1 -- 172.21.15.100:0/1069312547 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6c200aa070 con 0x56481158b140 2024-05-10T11:28:16.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.593+0000 7f6c0bfff700 1 -- 172.21.15.100:0/1069312547 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f6c18026030 con 0x56481158b140 2024-05-10T11:28:16.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.593+0000 7f6c3d8a9b80 10 client.26857 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:28:16.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.593+0000 7f6c3d8a9b80 10 client.26857 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:28:16.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.593+0000 7f6c3d8a9b80 20 client.26857 populate_metadata read hostname 'smithi100' 2024-05-10T11:28:16.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.593+0000 7f6c3d8a9b80 1 --2- 172.21.15.100:0/1069312547 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5648115a4350 0x5648115a6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:16.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.593+0000 7f6c3d8a9b80 4 client.26857 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:28:16.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.593+0000 7f6c3d8a9b80 1 -- 172.21.15.100:0/1069312547 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648115526d0 con 0x5648115a4350 2024-05-10T11:28:16.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.593+0000 7f6c30db2700 1 --2- 172.21.15.100:0/1069312547 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5648115a4350 0x5648115a6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:16.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.593+0000 7f6c30db2700 1 --2- 172.21.15.100:0/1069312547 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5648115a4350 0x5648115a6740 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4702806c4e71b2a7 server_cookie=d07cd5c0a57ab17d in_seq=0 out_seq=0 2024-05-10T11:28:16.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.593+0000 7f6c0bfff700 10 client.26857 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:28:16.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.594+0000 7f6c0bfff700 1 -- 172.21.15.100:0/1069312547 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5648115526d0 con 0x5648115a4350 2024-05-10T11:28:16.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.594+0000 7f6c0bfff700 10 client.26857 handle_command_reply: tid=0 2024-05-10T11:28:16.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.594+0000 7f6c3d8a9b80 1 client.26857 shutdown 2024-05-10T11:28:16.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.594+0000 7f6c3d8a9b80 1 -- 172.21.15.100:0/1069312547 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56481158b140 msgr2=0x564811593ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:16.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.594+0000 7f6c3d8a9b80 1 --2- 172.21.15.100:0/1069312547 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56481158b140 0x564811593ca0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x564811582f10 tx=0x7f6c180047c0 comp rx=0 tx=0).stop 2024-05-10T11:28:16.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.594+0000 7f6c3d8a9b80 1 -- 172.21.15.100:0/1069312547 shutdown_connections 2024-05-10T11:28:16.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.594+0000 7f6c3d8a9b80 1 --2- 172.21.15.100:0/1069312547 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5648115a4350 0x5648115a6740 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:16.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.594+0000 7f6c3d8a9b80 1 --2- 172.21.15.100:0/1069312547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564811590020 0x5648115941e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:16.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.594+0000 7f6c3d8a9b80 1 --2- 172.21.15.100:0/1069312547 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56481158a780 0x564811596900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:16.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.594+0000 7f6c0bfff700 0 client.26857 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:28:16.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.594+0000 7f6c3d8a9b80 1 --2- 172.21.15.100:0/1069312547 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56481158b140 0x564811593ca0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:16.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.594+0000 7f6c3d8a9b80 1 -- 172.21.15.100:0/1069312547 >> 172.21.15.100:0/1069312547 conn(0x5648114b2180 msgr2=0x56481157b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:16.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.595+0000 7f6c3d8a9b80 1 -- 172.21.15.100:0/1069312547 shutdown_connections 2024-05-10T11:28:16.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.595+0000 7f6c3d8a9b80 1 -- 172.21.15.100:0/1069312547 wait complete. 2024-05-10T11:28:16.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.595+0000 7f6c3d8a9b80 20 client.26857 trim_cache size 0 max 16384 2024-05-10T11:28:16.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.615+0000 7f6c3d8a9b80 1 Processor -- start 2024-05-10T11:28:16.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.615+0000 7f6c3d8a9b80 1 -- start start 2024-05-10T11:28:16.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.616+0000 7f6c3d8a9b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648114b2180 0x5648115a0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:16.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.616+0000 7f6c3d8a9b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56481158a780 0x56481157b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:16.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.616+0000 7f6c3d8a9b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56481158b140 0x5648115ac010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:16.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.616+0000 7f6c3d8a9b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2c1af730 con 0x56481158a780 2024-05-10T11:28:16.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.616+0000 7f6c3d8a9b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2c1acfc0 con 0x5648114b2180 2024-05-10T11:28:16.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.616+0000 7f6c3d8a9b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2c1aae10 con 0x56481158b140 2024-05-10T11:28:16.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.616+0000 7f6c2bfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56481158a780 0x56481157b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:16.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.616+0000 7f6c2bfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56481158a780 0x56481157b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38328/0 (socket says 172.21.15.100:38328) 2024-05-10T11:28:16.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.616+0000 7f6c2bfff700 1 -- 172.21.15.100:0/101089347 learned_addr learned my addr 172.21.15.100:0/101089347 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:28:16.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.616+0000 7f6c315b3700 1 --2- 172.21.15.100:0/101089347 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56481158b140 0x5648115ac010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:16.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.616+0000 7f6c30db2700 1 --2- 172.21.15.100:0/101089347 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648114b2180 0x5648115a0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:16.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.616+0000 7f6c2bfff700 1 -- 172.21.15.100:0/101089347 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56481158b140 msgr2=0x5648115ac010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:16.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.616+0000 7f6c2bfff700 1 --2- 172.21.15.100:0/101089347 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56481158b140 0x5648115ac010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:16.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.616+0000 7f6c2bfff700 1 -- 172.21.15.100:0/101089347 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648114b2180 msgr2=0x5648115a0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:16.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.616+0000 7f6c2bfff700 1 --2- 172.21.15.100:0/101089347 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648114b2180 0x5648115a0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:16.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.616+0000 7f6c2bfff700 1 -- 172.21.15.100:0/101089347 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648115a6dc0 con 0x56481158a780 2024-05-10T11:28:16.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.616+0000 7f6c2bfff700 1 --2- 172.21.15.100:0/101089347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56481158a780 0x56481157b620 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f6c1801e590 tx=0x7f6c18003c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4bef469d647592c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:16.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.616+0000 7f6c167fc700 1 -- 172.21.15.100:0/101089347 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1801e610 con 0x56481158a780 2024-05-10T11:28:16.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.617+0000 7f6c167fc700 1 -- 172.21.15.100:0/101089347 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6c1801ec40 con 0x56481158a780 2024-05-10T11:28:16.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.617+0000 7f6c167fc700 1 -- 172.21.15.100:0/101089347 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c18027aa0 con 0x56481158a780 2024-05-10T11:28:16.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.617+0000 7f6c3d8a9b80 1 -- 172.21.15.100:0/101089347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56481158a780 msgr2=0x56481157b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:16.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.617+0000 7f6c3d8a9b80 1 --2- 172.21.15.100:0/101089347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56481158a780 0x56481157b620 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f6c1801e590 tx=0x7f6c18003c40 comp rx=0 tx=0).stop 2024-05-10T11:28:16.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.617+0000 7f6c3d8a9b80 1 -- 172.21.15.100:0/101089347 shutdown_connections 2024-05-10T11:28:16.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.617+0000 7f6c3d8a9b80 1 --2- 172.21.15.100:0/101089347 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56481158b140 0x5648115ac010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:16.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.617+0000 7f6c3d8a9b80 1 --2- 172.21.15.100:0/101089347 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648114b2180 0x5648115a0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:16.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.617+0000 7f6c3d8a9b80 1 --2- 172.21.15.100:0/101089347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56481158a780 0x56481157b620 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:16.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.617+0000 7f6c3d8a9b80 1 -- 172.21.15.100:0/101089347 >> 172.21.15.100:0/101089347 conn(0x56481158e540 msgr2=0x56481157af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:16.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.618+0000 7f6c3d8a9b80 1 -- 172.21.15.100:0/101089347 shutdown_connections 2024-05-10T11:28:16.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.618+0000 7f6c3d8a9b80 1 -- 172.21.15.100:0/101089347 wait complete. 2024-05-10T11:28:16.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.618+0000 7f6c3d8a9b80 1 -- start start 2024-05-10T11:28:16.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.618+0000 7f6c3d8a9b80 1 Processor -- start 2024-05-10T11:28:16.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.618+0000 7f6c3d8a9b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5648114b2180 0x564811599880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:16.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.618+0000 7f6c3d8a9b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56481158a780 0x564811599dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:16.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.618+0000 7f6c3d8a9b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56481158b140 0x56481159a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:16.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.618+0000 7f6c3d8a9b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2c1ab480 con 0x56481158b140 2024-05-10T11:28:16.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.618+0000 7f6c3d8a9b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2c1ac8f0 con 0x56481158a780 2024-05-10T11:28:16.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.618+0000 7f6c3d8a9b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2c1abb50 con 0x5648114b2180 2024-05-10T11:28:16.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.619+0000 7f6c3d8a9b80 10 client.0 _ll_register_callbacks cb 0x5648114709a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:28:16.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.619+0000 7f6c30db2700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5648114b2180 0x564811599880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:16.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.619+0000 7f6c2bfff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56481158a780 0x564811599dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:16.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.619+0000 7f6c315b3700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56481158b140 0x56481159a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:16.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.619+0000 7f6c2bfff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56481158a780 0x564811599dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:44244/0 (socket says 172.21.15.100:44244) 2024-05-10T11:28:16.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.619+0000 7f6c2bfff700 1 -- 172.21.15.100:0/68603525 learned_addr learned my addr 172.21.15.100:0/68603525 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:28:16.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.619+0000 7f6c2bfff700 1 -- 172.21.15.100:0/68603525 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5648114b2180 msgr2=0x564811599880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:16.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.619+0000 7f6c2bfff700 1 --2- 172.21.15.100:0/68603525 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5648114b2180 0x564811599880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:16.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.619+0000 7f6c2bfff700 1 -- 172.21.15.100:0/68603525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56481158b140 msgr2=0x56481159a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:16.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.619+0000 7f6c2bfff700 1 --2- 172.21.15.100:0/68603525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56481158b140 0x56481159a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:16.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.619+0000 7f6c2bfff700 1 -- 172.21.15.100:0/68603525 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56481160ed90 con 0x56481158a780 2024-05-10T11:28:16.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.619+0000 7f6c2bfff700 1 --2- 172.21.15.100:0/68603525 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56481158a780 0x564811599dc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6c18016040 tx=0x7f6c180056f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9f7cc88ef4df623 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:16.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.619+0000 7f6c0bfff700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:28:16.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.619+0000 7f6c0bfff700 1 -- 172.21.15.100:0/68603525 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c18043070 con 0x56481158a780 2024-05-10T11:28:16.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.619+0000 7f6c0bfff700 1 -- 172.21.15.100:0/68603525 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f6c180343f0 con 0x56481158a780 2024-05-10T11:28:16.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.619+0000 7f6c3d8a9b80 1 -- 172.21.15.100:0/68603525 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c1801ec40 con 0x56481158a780 2024-05-10T11:28:16.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.620+0000 7f6c0bfff700 1 -- 172.21.15.100:0/68603525 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6c1804f050 con 0x56481158a780 2024-05-10T11:28:16.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.620+0000 7f6c3d8a9b80 10 client.26804 fetch_fsmap learned FSMap version 31 2024-05-10T11:28:16.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.620+0000 7f6c3d8a9b80 1 -- 172.21.15.100:0/68603525 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5648116100b0 con 0x56481158a780 2024-05-10T11:28:16.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.620+0000 7f6c0bfff700 1 -- 172.21.15.100:0/68603525 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6c1801ec40 con 0x56481158a780 2024-05-10T11:28:16.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.620+0000 7f6c0bfff700 1 -- 172.21.15.100:0/68603525 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f6c18026050 con 0x56481158a780 2024-05-10T11:28:16.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.620+0000 7f6c3d8a9b80 10 client.26804 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:28:16.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.620+0000 7f6c3d8a9b80 10 client.26804 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:28:16.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.620+0000 7f6c3d8a9b80 20 client.26804 populate_metadata read hostname 'smithi100' 2024-05-10T11:28:16.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.620+0000 7f6c3d8a9b80 1 --2- 172.21.15.100:0/68603525 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x564811590020 0x5648116125c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:16.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.620+0000 7f6c3d8a9b80 4 client.26804 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T11:28:16.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.620+0000 7f6c3d8a9b80 1 -- 172.21.15.100:0/68603525 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5648114e62c0 con 0x564811590020 2024-05-10T11:28:16.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.620+0000 7f6c30db2700 1 --2- 172.21.15.100:0/68603525 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x564811590020 0x5648116125c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:16.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.621+0000 7f6c0bfff700 10 client.26804 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:28:16.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.621+0000 7f6c30db2700 1 --2- 172.21.15.100:0/68603525 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x564811590020 0x5648116125c0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35725b83f7e3ba47 server_cookie=6c158909c35b305b in_seq=0 out_seq=0 2024-05-10T11:28:16.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.622+0000 7f6c0bfff700 1 -- 172.21.15.100:0/68603525 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5648114e62c0 con 0x564811590020 2024-05-10T11:28:16.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.622+0000 7f6c0bfff700 10 client.26804 handle_command_reply: tid=0 2024-05-10T11:28:16.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.622+0000 7f6c3d8a9b80 1 client.26804 shutdown 2024-05-10T11:28:16.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.622+0000 7f6c3d8a9b80 1 -- 172.21.15.100:0/68603525 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56481158a780 msgr2=0x564811599dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:16.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.622+0000 7f6c3d8a9b80 1 --2- 172.21.15.100:0/68603525 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56481158a780 0x564811599dc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6c18016040 tx=0x7f6c180056f0 comp rx=0 tx=0).stop 2024-05-10T11:28:16.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.622+0000 7f6c3d8a9b80 1 -- 172.21.15.100:0/68603525 shutdown_connections 2024-05-10T11:28:16.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.622+0000 7f6c3d8a9b80 1 --2- 172.21.15.100:0/68603525 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x564811590020 0x5648116125c0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:16.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.622+0000 7f6c3d8a9b80 1 --2- 172.21.15.100:0/68603525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56481158b140 0x56481159a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:16.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.622+0000 7f6c3d8a9b80 1 --2- 172.21.15.100:0/68603525 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5648114b2180 0x564811599880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:16.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.622+0000 7f6c3d8a9b80 1 --2- 172.21.15.100:0/68603525 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56481158a780 0x564811599dc0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:16.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.622+0000 7f6c3d8a9b80 1 -- 172.21.15.100:0/68603525 >> 172.21.15.100:0/68603525 conn(0x56481158e540 msgr2=0x564811589770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:16.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.622+0000 7f6c0bfff700 0 client.26804 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:28:16.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.622+0000 7f6c3d8a9b80 1 -- 172.21.15.100:0/68603525 shutdown_connections 2024-05-10T11:28:16.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.622+0000 7f6c3d8a9b80 1 -- 172.21.15.100:0/68603525 wait complete. 2024-05-10T11:28:16.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.623+0000 7f6c3d8a9b80 20 client.26804 trim_cache size 0 max 16384 2024-05-10T11:28:16.627 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:28:16.627 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-05-10T11:28:16.627 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "1e4472de-47b4-48d9-8316-c967fb60110b", 2024-05-10T11:28:16.627 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-05-10T11:28:16.627 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:28:16.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.623+0000 7f6c33016700 1 -- 172.21.15.100:0/3526855277 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f6c1c0b2880 msgr2=0x7f6c1c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:16.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.623+0000 7f6c33016700 1 --2- 172.21.15.100:0/3526855277 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f6c1c0b2880 0x7f6c1c0b4d40 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c079b10 tx=0x7f6c18009260 comp rx=0 tx=0).stop 2024-05-10T11:28:16.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.623+0000 7f6c33016700 1 -- 172.21.15.100:0/3526855277 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c2c1023d0 msgr2=0x7f6c2c19e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:16.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.623+0000 7f6c33016700 1 --2- 172.21.15.100:0/3526855277 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c2c1023d0 0x7f6c2c19e6e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6c2000e9a0 tx=0x7f6c2000ecb0 comp rx=0 tx=0).stop 2024-05-10T11:28:16.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.623+0000 7f6c33016700 1 -- 172.21.15.100:0/3526855277 shutdown_connections 2024-05-10T11:28:16.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.623+0000 7f6c33016700 1 --2- 172.21.15.100:0/3526855277 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f6c1c0b2880 0x7f6c1c0b4d40 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:16.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.623+0000 7f6c33016700 1 --2- 172.21.15.100:0/3526855277 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c2c108440 0x7f6c2c1988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:16.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.623+0000 7f6c33016700 1 --2- 172.21.15.100:0/3526855277 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c2c1023d0 0x7f6c2c19e6e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:16.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.623+0000 7f6c33016700 1 --2- 172.21.15.100:0/3526855277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c2c102d90 0x7f6c2c19ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:16.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.623+0000 7f6c33016700 1 -- 172.21.15.100:0/3526855277 >> 172.21.15.100:0/3526855277 conn(0x7f6c2c0793e0 msgr2=0x7f6c2c101240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:16.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.624+0000 7f6c33016700 1 -- 172.21.15.100:0/3526855277 shutdown_connections 2024-05-10T11:28:16.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:16.624+0000 7f6c33016700 1 -- 172.21.15.100:0/3526855277 wait complete. 2024-05-10T11:28:16.979 INFO:tasks.workunit.client.0.smithi100.stdout: 1 252476 2.73 MB/sec execute 371 sec latency 54.981 ms 2024-05-10T11:28:17.409 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T11:28:17.674 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:28:17.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:28:17 smithi178 ceph-mon[33359]: pgmap v717: 129 pgs: 129 active+clean; 103 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 1.2 MiB/s wr, 139 op/s 2024-05-10T11:28:17.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:28:17 smithi178 ceph-mon[33359]: scrub queued for path: / 2024-05-10T11:28:17.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:28:17 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:28:17.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:28:17 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:28:17.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:28:17 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:28:17.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:28:17 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:28:17.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:28:17 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:28:17.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:28:17 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:28:17.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:28:17 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:28:17.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:28:17 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:28:17.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:28:17 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:28:17.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:28:17 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:28:17.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:28:17 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:28:17.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:28:17 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:28:17.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:28:17 smithi188 ceph-mon[34075]: pgmap v717: 129 pgs: 129 active+clean; 103 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 1.2 MiB/s wr, 139 op/s 2024-05-10T11:28:17.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:28:17 smithi188 ceph-mon[34075]: scrub queued for path: / 2024-05-10T11:28:17.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:28:17 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:28:17.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:28:17 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:28:17.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:28:17 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:28:17.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:28:17 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:28:17.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:28:17 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:28:17.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:28:17 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:28:17.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:28:17 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:28:17.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:28:17 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:28:17.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:28:17 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:28:17.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:28:17 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:28:17.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:28:17 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:28:17.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:28:17 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:28:17.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:28:17 smithi100 ceph-mon[30296]: pgmap v717: 129 pgs: 129 active+clean; 103 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 1.2 MiB/s wr, 139 op/s 2024-05-10T11:28:17.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:28:17 smithi100 ceph-mon[30296]: scrub queued for path: / 2024-05-10T11:28:17.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:28:17 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:28:17.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:28:17 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:28:17.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:28:17 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:28:17.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:28:17 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:28:17.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:28:17 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:28:17.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:28:17 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:28:17.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:28:17 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:28:17.789 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:28:17 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:28:17.789 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:28:17 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:28:17.789 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:28:17 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:28:17.789 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:28:17 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:28:17.789 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:28:17 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:28:17.979 INFO:tasks.workunit.client.0.smithi100.stdout: 1 252948 2.73 MB/sec execute 372 sec latency 15.695 ms 2024-05-10T11:28:18.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:28:18 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:28:18.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:28:18 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:28:18.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:28:18 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:28:18.979 INFO:tasks.workunit.client.0.smithi100.stdout: 1 253496 2.73 MB/sec execute 373 sec latency 53.067 ms 2024-05-10T11:28:19.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:28:19 smithi100 ceph-mon[30296]: pgmap v718: 129 pgs: 129 active+clean; 105 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 1.4 MiB/s wr, 163 op/s 2024-05-10T11:28:19.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.657+0000 7f2ee508b700 1 -- 172.21.15.100:0/698457089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ee0108880 msgr2=0x7f2ee0110400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:19.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.657+0000 7f2ee508b700 1 --2- 172.21.15.100:0/698457089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ee0108880 0x7f2ee0110400 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f2ed800b0d0 tx=0x7f2ed800b3e0 comp rx=0 tx=0).stop 2024-05-10T11:28:19.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.657+0000 7f2ee508b700 1 -- 172.21.15.100:0/698457089 shutdown_connections 2024-05-10T11:28:19.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.657+0000 7f2ee508b700 1 --2- 172.21.15.100:0/698457089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ee0108880 0x7f2ee0110400 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:19.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.657+0000 7f2ee508b700 1 --2- 172.21.15.100:0/698457089 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ee01037b0 0x7f2ee0103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:19.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.657+0000 7f2ee508b700 1 --2- 172.21.15.100:0/698457089 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ee0104160 0x7f2ee0108150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:19.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.657+0000 7f2ee508b700 1 -- 172.21.15.100:0/698457089 >> 172.21.15.100:0/698457089 conn(0x7f2ee007a530 msgr2=0x7f2ee007a940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:19.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.657+0000 7f2ee508b700 1 -- 172.21.15.100:0/698457089 shutdown_connections 2024-05-10T11:28:19.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.657+0000 7f2ee508b700 1 -- 172.21.15.100:0/698457089 wait complete. 2024-05-10T11:28:19.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.658+0000 7f2ee508b700 1 Processor -- start 2024-05-10T11:28:19.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.658+0000 7f2ee508b700 1 -- start start 2024-05-10T11:28:19.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.658+0000 7f2ee508b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ee01037b0 0x7f2ee019f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:19.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.658+0000 7f2ee508b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ee0104160 0x7f2ee019fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:19.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.658+0000 7f2ee508b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ee0108880 0x7f2ee0199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:19.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.658+0000 7f2ee508b700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee0113500 con 0x7f2ee0108880 2024-05-10T11:28:19.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.658+0000 7f2ee508b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee0113380 con 0x7f2ee01037b0 2024-05-10T11:28:19.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.658+0000 7f2ee508b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee0113680 con 0x7f2ee0104160 2024-05-10T11:28:19.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.658+0000 7f2edf59e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ee0108880 0x7f2ee0199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:19.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.658+0000 7f2eded9d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ee01037b0 0x7f2ee019f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:19.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.658+0000 7f2edf59e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ee0108880 0x7f2ee0199780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38356/0 (socket says 172.21.15.100:38356) 2024-05-10T11:28:19.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.658+0000 7f2edf59e700 1 -- 172.21.15.100:0/2710208877 learned_addr learned my addr 172.21.15.100:0/2710208877 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:28:19.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.658+0000 7f2ede59c700 1 --2- 172.21.15.100:0/2710208877 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ee0104160 0x7f2ee019fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:19.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.659+0000 7f2edf59e700 1 -- 172.21.15.100:0/2710208877 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ee0104160 msgr2=0x7f2ee019fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:19.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.659+0000 7f2edf59e700 1 --2- 172.21.15.100:0/2710208877 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ee0104160 0x7f2ee019fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:19.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.659+0000 7f2edf59e700 1 -- 172.21.15.100:0/2710208877 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ee01037b0 msgr2=0x7f2ee019f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:19.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.659+0000 7f2edf59e700 1 --2- 172.21.15.100:0/2710208877 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ee01037b0 0x7f2ee019f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:19.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.659+0000 7f2edf59e700 1 -- 172.21.15.100:0/2710208877 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ee0199f60 con 0x7f2ee0108880 2024-05-10T11:28:19.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.659+0000 7f2eded9d700 1 --2- 172.21.15.100:0/2710208877 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ee01037b0 0x7f2ee019f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:28:19.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.659+0000 7f2edf59e700 1 --2- 172.21.15.100:0/2710208877 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ee0108880 0x7f2ee0199780 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8009850 tx=0x7f2ed8007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:19.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.659+0000 7f2ec7fff700 1 -- 172.21.15.100:0/2710208877 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed801f070 con 0x7f2ee0108880 2024-05-10T11:28:19.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.659+0000 7f2ee508b700 1 -- 172.21.15.100:0/2710208877 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ee019a1f0 con 0x7f2ee0108880 2024-05-10T11:28:19.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.659+0000 7f2ec7fff700 1 -- 172.21.15.100:0/2710208877 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2ed8004c50 con 0x7f2ee0108880 2024-05-10T11:28:19.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.659+0000 7f2ec7fff700 1 -- 172.21.15.100:0/2710208877 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed801b690 con 0x7f2ee0108880 2024-05-10T11:28:19.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.659+0000 7f2ee508b700 1 -- 172.21.15.100:0/2710208877 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ee01a6460 con 0x7f2ee0108880 2024-05-10T11:28:19.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.662+0000 7f2ec7fff700 1 -- 172.21.15.100:0/2710208877 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f2ed8022070 con 0x7f2ee0108880 2024-05-10T11:28:19.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.663+0000 7f2ec7fff700 1 --2- 172.21.15.100:0/2710208877 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2ec80b2880 0x7f2ec80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:19.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.663+0000 7f2eded9d700 1 --2- 172.21.15.100:0/2710208877 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2ec80b2880 0x7f2ec80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:19.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.663+0000 7f2ec7fff700 1 -- 172.21.15.100:0/2710208877 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f2ed80e3480 con 0x7f2ee0108880 2024-05-10T11:28:19.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.663+0000 7f2eef91eb80 1 Processor -- start 2024-05-10T11:28:19.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.663+0000 7f2eef91eb80 1 -- start start 2024-05-10T11:28:19.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.663+0000 7f2eded9d700 1 --2- 172.21.15.100:0/2710208877 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2ec80b2880 0x7f2ec80b4d40 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0010420 tx=0x7f2ed00073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:19.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.663+0000 7f2eef91eb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55800270c1b0 0x55800270c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:19.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.663+0000 7f2eef91eb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55800270cad0 0x558002706810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:19.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.663+0000 7f2eef91eb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558002706d50 0x558002707190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:19.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.663+0000 7f2eef91eb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x558002658940 con 0x558002706d50 2024-05-10T11:28:19.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.663+0000 7f2eef91eb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5580025bdda0 con 0x55800270c1b0 2024-05-10T11:28:19.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.663+0000 7f2eef91eb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558002654900 con 0x55800270cad0 2024-05-10T11:28:19.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.664+0000 7f2edf59e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558002706d50 0x558002707190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:19.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.664+0000 7f2edf59e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558002706d50 0x558002707190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38358/0 (socket says 172.21.15.100:38358) 2024-05-10T11:28:19.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.664+0000 7f2edf59e700 1 -- 172.21.15.100:0/357209137 learned_addr learned my addr 172.21.15.100:0/357209137 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:28:19.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.664+0000 7f2ede59c700 1 --2- 172.21.15.100:0/357209137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55800270cad0 0x558002706810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:19.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.664+0000 7f2eded9d700 1 --2- 172.21.15.100:0/357209137 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55800270c1b0 0x55800270c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:19.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.664+0000 7f2edf59e700 1 -- 172.21.15.100:0/357209137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55800270cad0 msgr2=0x558002706810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:19.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.664+0000 7f2edf59e700 1 --2- 172.21.15.100:0/357209137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55800270cad0 0x558002706810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:19.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.664+0000 7f2edf59e700 1 -- 172.21.15.100:0/357209137 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55800270c1b0 msgr2=0x55800270c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:19.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.664+0000 7f2edf59e700 1 --2- 172.21.15.100:0/357209137 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55800270c1b0 0x55800270c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:19.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.664+0000 7f2edf59e700 1 -- 172.21.15.100:0/357209137 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580025a58d0 con 0x558002706d50 2024-05-10T11:28:19.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.664+0000 7f2edf59e700 1 --2- 172.21.15.100:0/357209137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558002706d50 0x558002707190 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f2ed800bfd0 tx=0x7f2ed80aa1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97dfa3ab57687967 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:19.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.664+0000 7f2ec57fa700 1 -- 172.21.15.100:0/357209137 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed8029030 con 0x558002706d50 2024-05-10T11:28:19.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.664+0000 7f2ec57fa700 1 -- 172.21.15.100:0/357209137 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2ed80aa2e0 con 0x558002706d50 2024-05-10T11:28:19.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.664+0000 7f2ec57fa700 1 -- 172.21.15.100:0/357209137 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed80aeb80 con 0x558002706d50 2024-05-10T11:28:19.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.665+0000 7f2eef91eb80 1 -- 172.21.15.100:0/357209137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558002706d50 msgr2=0x558002707190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:19.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.665+0000 7f2eef91eb80 1 --2- 172.21.15.100:0/357209137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558002706d50 0x558002707190 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f2ed800bfd0 tx=0x7f2ed80aa1a0 comp rx=0 tx=0).stop 2024-05-10T11:28:19.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.665+0000 7f2eef91eb80 1 -- 172.21.15.100:0/357209137 shutdown_connections 2024-05-10T11:28:19.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.665+0000 7f2eef91eb80 1 --2- 172.21.15.100:0/357209137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558002706d50 0x558002707190 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:19.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.665+0000 7f2eef91eb80 1 --2- 172.21.15.100:0/357209137 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55800270c1b0 0x55800270c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:19.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.665+0000 7f2eef91eb80 1 --2- 172.21.15.100:0/357209137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55800270cad0 0x558002706810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:19.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.665+0000 7f2eef91eb80 1 -- 172.21.15.100:0/357209137 >> 172.21.15.100:0/357209137 conn(0x558002632b70 msgr2=0x5580027012a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:19.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.665+0000 7f2eef91eb80 1 -- 172.21.15.100:0/357209137 shutdown_connections 2024-05-10T11:28:19.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.665+0000 7f2eef91eb80 1 -- 172.21.15.100:0/357209137 wait complete. 2024-05-10T11:28:19.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.666+0000 7f2eef91eb80 1 -- start start 2024-05-10T11:28:19.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.666+0000 7f2eef91eb80 1 Processor -- start 2024-05-10T11:28:19.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.666+0000 7f2eef91eb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558002706d50 0x5580027127b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:19.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.666+0000 7f2eef91eb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55800270c1b0 0x55800270fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:19.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.666+0000 7f2eef91eb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55800270cad0 0x5580027100e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:19.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.666+0000 7f2eef91eb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x558002713210 con 0x55800270cad0 2024-05-10T11:28:19.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.666+0000 7f2eef91eb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558002611ed0 con 0x55800270c1b0 2024-05-10T11:28:19.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.666+0000 7f2eef91eb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558002713390 con 0x558002706d50 2024-05-10T11:28:19.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.666+0000 7f2eef91eb80 10 client.0 _ll_register_callbacks cb 0x5580025ec5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:28:19.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.666+0000 7f2eded9d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558002706d50 0x5580027127b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:19.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.666+0000 7f2edf59e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55800270cad0 0x5580027100e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:19.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.666+0000 7f2eded9d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558002706d50 0x5580027127b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:44724/0 (socket says 172.21.15.100:44724) 2024-05-10T11:28:19.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.666+0000 7f2eded9d700 1 -- 172.21.15.100:0/592422356 learned_addr learned my addr 172.21.15.100:0/592422356 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:28:19.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.666+0000 7f2ede59c700 1 --2- 172.21.15.100:0/592422356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55800270c1b0 0x55800270fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:19.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.666+0000 7f2edf59e700 1 -- 172.21.15.100:0/592422356 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558002706d50 msgr2=0x5580027127b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:19.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.666+0000 7f2edf59e700 1 --2- 172.21.15.100:0/592422356 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558002706d50 0x5580027127b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:19.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.666+0000 7f2edf59e700 1 -- 172.21.15.100:0/592422356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55800270c1b0 msgr2=0x55800270fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:19.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.666+0000 7f2edf59e700 1 --2- 172.21.15.100:0/592422356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55800270c1b0 0x55800270fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:19.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.666+0000 7f2edf59e700 1 -- 172.21.15.100:0/592422356 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55800260f760 con 0x55800270cad0 2024-05-10T11:28:19.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.667+0000 7f2edf59e700 1 --2- 172.21.15.100:0/592422356 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55800270cad0 0x5580027100e0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f2ed800bfd0 tx=0x7f2ed80b1f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=294a9291188f3e91 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:19.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.667+0000 7f2ec2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:28:19.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.667+0000 7f2ec2ff5700 1 -- 172.21.15.100:0/592422356 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed8029050 con 0x55800270cad0 2024-05-10T11:28:19.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.667+0000 7f2ec2ff5700 1 -- 172.21.15.100:0/592422356 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f2ed80aaa30 con 0x55800270cad0 2024-05-10T11:28:19.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.667+0000 7f2eef91eb80 1 -- 172.21.15.100:0/592422356 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ed80aa2e0 con 0x55800270cad0 2024-05-10T11:28:19.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.667+0000 7f2ec2ff5700 1 -- 172.21.15.100:0/592422356 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2ed8009cb0 con 0x55800270cad0 2024-05-10T11:28:19.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.667+0000 7f2eef91eb80 10 client.17421 fetch_fsmap learned FSMap version 31 2024-05-10T11:28:19.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.667+0000 7f2eef91eb80 1 -- 172.21.15.100:0/592422356 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558002713210 con 0x55800270cad0 2024-05-10T11:28:19.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.667+0000 7f2ec2ff5700 1 -- 172.21.15.100:0/592422356 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2ed80aa2e0 con 0x55800270cad0 2024-05-10T11:28:19.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.667+0000 7f2ec2ff5700 1 -- 172.21.15.100:0/592422356 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f2ed8011070 con 0x55800270cad0 2024-05-10T11:28:19.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.667+0000 7f2eef91eb80 10 client.17421 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:28:19.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.667+0000 7f2eef91eb80 10 client.17421 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:28:19.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.667+0000 7f2eef91eb80 20 client.17421 populate_metadata read hostname 'smithi100' 2024-05-10T11:28:19.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.667+0000 7f2eef91eb80 1 --2- 172.21.15.100:0/592422356 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55800271fe00 0x5580027221f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:19.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.667+0000 7f2eef91eb80 4 client.17421 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:28:19.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.667+0000 7f2eef91eb80 1 -- 172.21.15.100:0/592422356 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558002666320 con 0x55800271fe00 2024-05-10T11:28:19.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.668+0000 7f2eded9d700 1 --2- 172.21.15.100:0/592422356 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55800271fe00 0x5580027221f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:19.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.668+0000 7f2eded9d700 1 --2- 172.21.15.100:0/592422356 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55800271fe00 0x5580027221f0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77d014e51cd8fc56 server_cookie=6cc8981697d131d5 in_seq=0 out_seq=0 2024-05-10T11:28:19.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.668+0000 7f2ec2ff5700 10 client.17421 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:28:19.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.669+0000 7f2ec2ff5700 1 -- 172.21.15.100:0/592422356 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558002666320 con 0x55800271fe00 2024-05-10T11:28:19.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.669+0000 7f2ec2ff5700 10 client.17421 handle_command_reply: tid=0 2024-05-10T11:28:19.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.669+0000 7f2eef91eb80 1 client.17421 shutdown 2024-05-10T11:28:19.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.669+0000 7f2eef91eb80 1 -- 172.21.15.100:0/592422356 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55800270cad0 msgr2=0x5580027100e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:19.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.669+0000 7f2eef91eb80 1 --2- 172.21.15.100:0/592422356 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55800270cad0 0x5580027100e0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f2ed800bfd0 tx=0x7f2ed80b1f60 comp rx=0 tx=0).stop 2024-05-10T11:28:19.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.670+0000 7f2eef91eb80 1 -- 172.21.15.100:0/592422356 shutdown_connections 2024-05-10T11:28:19.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.670+0000 7f2eef91eb80 1 --2- 172.21.15.100:0/592422356 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55800271fe00 0x5580027221f0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:19.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.670+0000 7f2eef91eb80 1 --2- 172.21.15.100:0/592422356 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55800270cad0 0x5580027100e0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:19.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.670+0000 7f2eef91eb80 1 --2- 172.21.15.100:0/592422356 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558002706d50 0x5580027127b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:19.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.670+0000 7f2ec2ff5700 0 client.17421 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:28:19.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.670+0000 7f2eef91eb80 1 --2- 172.21.15.100:0/592422356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55800270c1b0 0x55800270fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:19.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.670+0000 7f2eef91eb80 1 -- 172.21.15.100:0/592422356 >> 172.21.15.100:0/592422356 conn(0x558002632b70 msgr2=0x5580026fbb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:19.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.670+0000 7f2eef91eb80 1 -- 172.21.15.100:0/592422356 shutdown_connections 2024-05-10T11:28:19.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.670+0000 7f2eef91eb80 1 -- 172.21.15.100:0/592422356 wait complete. 2024-05-10T11:28:19.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.670+0000 7f2eef91eb80 20 client.17421 trim_cache size 0 max 16384 2024-05-10T11:28:19.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.693+0000 7f2eef91eb80 1 Processor -- start 2024-05-10T11:28:19.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.693+0000 7f2eef91eb80 1 -- start start 2024-05-10T11:28:19.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.693+0000 7f2eef91eb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558002706d50 0x558002706810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:19.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.693+0000 7f2eef91eb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55800270c1b0 0x55800271c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:19.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.693+0000 7f2eef91eb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55800270cad0 0x5580026fbc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:19.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.693+0000 7f2eef91eb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee01b17e0 con 0x55800270cad0 2024-05-10T11:28:19.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.693+0000 7f2eef91eb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee01af070 con 0x558002706d50 2024-05-10T11:28:19.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.693+0000 7f2eef91eb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee01acec0 con 0x55800270c1b0 2024-05-10T11:28:19.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.693+0000 7f2edf59e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55800270cad0 0x5580026fbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:19.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.693+0000 7f2edf59e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55800270cad0 0x5580026fbc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38372/0 (socket says 172.21.15.100:38372) 2024-05-10T11:28:19.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.693+0000 7f2ede59c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55800270c1b0 0x55800271c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:19.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.693+0000 7f2edf59e700 1 -- 172.21.15.100:0/1914875575 learned_addr learned my addr 172.21.15.100:0/1914875575 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:28:19.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.693+0000 7f2ede59c700 1 -- 172.21.15.100:0/1914875575 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558002706d50 msgr2=0x558002706810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:19.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.693+0000 7f2ede59c700 1 --2- 172.21.15.100:0/1914875575 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558002706d50 0x558002706810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:19.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.693+0000 7f2ede59c700 1 -- 172.21.15.100:0/1914875575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55800270cad0 msgr2=0x5580026fbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:19.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.693+0000 7f2ede59c700 1 --2- 172.21.15.100:0/1914875575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55800270cad0 0x5580026fbc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:19.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.693+0000 7f2ede59c700 1 -- 172.21.15.100:0/1914875575 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558002702d00 con 0x55800270c1b0 2024-05-10T11:28:19.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.694+0000 7f2ede59c700 1 --2- 172.21.15.100:0/1914875575 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55800270c1b0 0x55800271c260 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x558002703450 tx=0x7f2ed4004320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40cbc2c6ff936b96 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:19.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.694+0000 7f2ec4ff9700 1 -- 172.21.15.100:0/1914875575 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed4004a30 con 0x55800270c1b0 2024-05-10T11:28:19.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.694+0000 7f2ec4ff9700 1 -- 172.21.15.100:0/1914875575 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2ed40054b0 con 0x55800270c1b0 2024-05-10T11:28:19.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.694+0000 7f2ec4ff9700 1 -- 172.21.15.100:0/1914875575 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed4003b80 con 0x55800270c1b0 2024-05-10T11:28:19.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.695+0000 7f2eef91eb80 1 -- 172.21.15.100:0/1914875575 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55800270c1b0 msgr2=0x55800271c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:19.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.695+0000 7f2eef91eb80 1 --2- 172.21.15.100:0/1914875575 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55800270c1b0 0x55800271c260 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x558002703450 tx=0x7f2ed4004320 comp rx=0 tx=0).stop 2024-05-10T11:28:19.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.695+0000 7f2eef91eb80 1 -- 172.21.15.100:0/1914875575 shutdown_connections 2024-05-10T11:28:19.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.695+0000 7f2eef91eb80 1 --2- 172.21.15.100:0/1914875575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55800270cad0 0x5580026fbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:19.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.695+0000 7f2eef91eb80 1 --2- 172.21.15.100:0/1914875575 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558002706d50 0x558002706810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:19.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.695+0000 7f2eef91eb80 1 --2- 172.21.15.100:0/1914875575 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55800270c1b0 0x55800271c260 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:19.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.695+0000 7f2eef91eb80 1 -- 172.21.15.100:0/1914875575 >> 172.21.15.100:0/1914875575 conn(0x558002632b70 msgr2=0x55800270b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:19.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.695+0000 7f2eef91eb80 1 -- 172.21.15.100:0/1914875575 shutdown_connections 2024-05-10T11:28:19.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.695+0000 7f2eef91eb80 1 -- 172.21.15.100:0/1914875575 wait complete. 2024-05-10T11:28:19.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.695+0000 7f2eef91eb80 1 -- start start 2024-05-10T11:28:19.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.695+0000 7f2eef91eb80 1 Processor -- start 2024-05-10T11:28:19.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.695+0000 7f2eef91eb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558002706d50 0x5580027275f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:19.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.695+0000 7f2eef91eb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55800270c1b0 0x558002727b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:19.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.695+0000 7f2eef91eb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55800270cad0 0x55800278d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:19.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.696+0000 7f2eef91eb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee01ad530 con 0x55800270c1b0 2024-05-10T11:28:19.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.696+0000 7f2eef91eb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee01ae9a0 con 0x55800270cad0 2024-05-10T11:28:19.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.696+0000 7f2eef91eb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee01adc00 con 0x558002706d50 2024-05-10T11:28:19.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.696+0000 7f2eef91eb80 10 client.0 _ll_register_callbacks cb 0x5580025ec5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:28:19.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.696+0000 7f2ede59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55800270c1b0 0x558002727b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:19.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.696+0000 7f2eded9d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558002706d50 0x5580027275f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:19.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.696+0000 7f2eded9d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558002706d50 0x5580027275f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:44728/0 (socket says 172.21.15.100:44728) 2024-05-10T11:28:19.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.696+0000 7f2ede59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55800270c1b0 0x558002727b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38384/0 (socket says 172.21.15.100:38384) 2024-05-10T11:28:19.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.696+0000 7f2eded9d700 1 -- 172.21.15.100:0/3220400785 learned_addr learned my addr 172.21.15.100:0/3220400785 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:28:19.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.696+0000 7f2edf59e700 1 --2- 172.21.15.100:0/3220400785 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55800270cad0 0x55800278d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:19.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.696+0000 7f2ede59c700 1 -- 172.21.15.100:0/3220400785 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558002706d50 msgr2=0x5580027275f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:19.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.696+0000 7f2ede59c700 1 --2- 172.21.15.100:0/3220400785 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558002706d50 0x5580027275f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:19.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.696+0000 7f2ede59c700 1 -- 172.21.15.100:0/3220400785 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55800270cad0 msgr2=0x55800278d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:19.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.696+0000 7f2ede59c700 1 --2- 172.21.15.100:0/3220400785 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55800270cad0 0x55800278d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:19.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.696+0000 7f2ede59c700 1 -- 172.21.15.100:0/3220400785 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558002703150 con 0x55800270c1b0 2024-05-10T11:28:19.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.696+0000 7f2eded9d700 1 --2- 172.21.15.100:0/3220400785 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558002706d50 0x5580027275f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:28:19.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.696+0000 7f2ede59c700 1 --2- 172.21.15.100:0/3220400785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55800270c1b0 0x558002727b30 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x5580026c7da0 tx=0x7f2ed4009c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdf561ebfa0ce242 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:19.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.696+0000 7f2ec2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:28:19.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.696+0000 7f2ec2ff5700 1 -- 172.21.15.100:0/3220400785 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed4004d20 con 0x55800270c1b0 2024-05-10T11:28:19.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.696+0000 7f2ec2ff5700 1 -- 172.21.15.100:0/3220400785 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f2ed4003740 con 0x55800270c1b0 2024-05-10T11:28:19.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.696+0000 7f2eef91eb80 1 -- 172.21.15.100:0/3220400785 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ed40054b0 con 0x55800270c1b0 2024-05-10T11:28:19.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.697+0000 7f2ec2ff5700 1 -- 172.21.15.100:0/3220400785 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2ed4034050 con 0x55800270c1b0 2024-05-10T11:28:19.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.697+0000 7f2eef91eb80 10 client.17427 fetch_fsmap learned FSMap version 31 2024-05-10T11:28:19.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.697+0000 7f2eef91eb80 1 -- 172.21.15.100:0/3220400785 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55800278eea0 con 0x55800270c1b0 2024-05-10T11:28:19.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.697+0000 7f2ec2ff5700 1 -- 172.21.15.100:0/3220400785 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2ed40054b0 con 0x55800270c1b0 2024-05-10T11:28:19.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.697+0000 7f2ec2ff5700 1 -- 172.21.15.100:0/3220400785 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f2ed401b030 con 0x55800270c1b0 2024-05-10T11:28:19.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.697+0000 7f2eef91eb80 10 client.17427 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:28:19.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.697+0000 7f2eef91eb80 10 client.17427 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:28:19.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.697+0000 7f2eef91eb80 20 client.17427 populate_metadata read hostname 'smithi100' 2024-05-10T11:28:19.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.697+0000 7f2eef91eb80 1 --2- 172.21.15.100:0/3220400785 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55800278f580 0x5580027919a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:19.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.697+0000 7f2eef91eb80 4 client.17427 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T11:28:19.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.697+0000 7f2eef91eb80 1 -- 172.21.15.100:0/3220400785 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5580026603c0 con 0x55800278f580 2024-05-10T11:28:19.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.697+0000 7f2eded9d700 1 --2- 172.21.15.100:0/3220400785 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55800278f580 0x5580027919a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:19.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.698+0000 7f2eded9d700 1 --2- 172.21.15.100:0/3220400785 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55800278f580 0x5580027919a0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=760666f5e48b666b server_cookie=a84b44eeaa0ab242 in_seq=0 out_seq=0 2024-05-10T11:28:19.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.698+0000 7f2ec2ff5700 10 client.17427 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:28:19.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.698+0000 7f2ec2ff5700 1 -- 172.21.15.100:0/3220400785 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5580026603c0 con 0x55800278f580 2024-05-10T11:28:19.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.698+0000 7f2ec2ff5700 10 client.17427 handle_command_reply: tid=0 2024-05-10T11:28:19.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.698+0000 7f2eef91eb80 1 client.17427 shutdown 2024-05-10T11:28:19.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.698+0000 7f2eef91eb80 1 -- 172.21.15.100:0/3220400785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55800270c1b0 msgr2=0x558002727b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:19.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.698+0000 7f2eef91eb80 1 --2- 172.21.15.100:0/3220400785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55800270c1b0 0x558002727b30 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x5580026c7da0 tx=0x7f2ed4009c60 comp rx=0 tx=0).stop 2024-05-10T11:28:19.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.698+0000 7f2eef91eb80 1 -- 172.21.15.100:0/3220400785 shutdown_connections 2024-05-10T11:28:19.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.698+0000 7f2eef91eb80 1 --2- 172.21.15.100:0/3220400785 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55800278f580 0x5580027919a0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:19.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.698+0000 7f2eef91eb80 1 --2- 172.21.15.100:0/3220400785 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55800270cad0 0x55800278d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:19.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.698+0000 7f2ec2ff5700 0 client.17427 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:28:19.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.698+0000 7f2eef91eb80 1 --2- 172.21.15.100:0/3220400785 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558002706d50 0x5580027275f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:19.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.698+0000 7f2eef91eb80 1 --2- 172.21.15.100:0/3220400785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55800270c1b0 0x558002727b30 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:19.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.698+0000 7f2eef91eb80 1 -- 172.21.15.100:0/3220400785 >> 172.21.15.100:0/3220400785 conn(0x558002632b70 msgr2=0x55800270f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:19.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.698+0000 7f2eef91eb80 1 -- 172.21.15.100:0/3220400785 shutdown_connections 2024-05-10T11:28:19.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.698+0000 7f2eef91eb80 1 -- 172.21.15.100:0/3220400785 wait complete. 2024-05-10T11:28:19.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.698+0000 7f2eef91eb80 20 client.17427 trim_cache size 0 max 16384 2024-05-10T11:28:19.704 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:28:19.704 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T11:28:19.704 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-05-10T11:28:19.705 INFO:teuthology.orchestra.run.smithi100.stdout: "1e4472de-47b4-48d9-8316-c967fb60110b": { 2024-05-10T11:28:19.705 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-05-10T11:28:19.705 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "1e4472de-47b4-48d9-8316-c967fb60110b", 2024-05-10T11:28:19.705 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-05-10T11:28:19.705 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:28:19.705 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:28:19.705 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:28:19.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.699+0000 7f2ee508b700 1 -- 172.21.15.100:0/2710208877 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2ec80b2880 msgr2=0x7f2ec80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:19.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.699+0000 7f2ee508b700 1 --2- 172.21.15.100:0/2710208877 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2ec80b2880 0x7f2ec80b4d40 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0010420 tx=0x7f2ed00073d0 comp rx=0 tx=0).stop 2024-05-10T11:28:19.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.699+0000 7f2ee508b700 1 -- 172.21.15.100:0/2710208877 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ee0108880 msgr2=0x7f2ee0199780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:19.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.699+0000 7f2ee508b700 1 --2- 172.21.15.100:0/2710208877 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ee0108880 0x7f2ee0199780 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8009850 tx=0x7f2ed8007a20 comp rx=0 tx=0).stop 2024-05-10T11:28:19.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.699+0000 7f2ee508b700 1 -- 172.21.15.100:0/2710208877 shutdown_connections 2024-05-10T11:28:19.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.699+0000 7f2ee508b700 1 --2- 172.21.15.100:0/2710208877 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2ec80b2880 0x7f2ec80b4d40 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:19.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.699+0000 7f2ee508b700 1 --2- 172.21.15.100:0/2710208877 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ee0108880 0x7f2ee0199780 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:19.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.699+0000 7f2ee508b700 1 --2- 172.21.15.100:0/2710208877 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ee01037b0 0x7f2ee019f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:19.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.699+0000 7f2ee508b700 1 --2- 172.21.15.100:0/2710208877 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ee0104160 0x7f2ee019fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:19.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.699+0000 7f2ee508b700 1 -- 172.21.15.100:0/2710208877 >> 172.21.15.100:0/2710208877 conn(0x7f2ee007a530 msgr2=0x7f2ee01017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:19.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.699+0000 7f2ee508b700 1 -- 172.21.15.100:0/2710208877 shutdown_connections 2024-05-10T11:28:19.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:19.699+0000 7f2ee508b700 1 -- 172.21.15.100:0/2710208877 wait complete. 2024-05-10T11:28:19.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:28:19 smithi178 ceph-mon[33359]: pgmap v718: 129 pgs: 129 active+clean; 105 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 1.4 MiB/s wr, 163 op/s 2024-05-10T11:28:19.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:28:19 smithi188 ceph-mon[34075]: pgmap v718: 129 pgs: 129 active+clean; 105 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 1.4 MiB/s wr, 163 op/s 2024-05-10T11:28:19.979 INFO:tasks.workunit.client.0.smithi100.stdout: 1 254130 2.73 MB/sec execute 374 sec latency 42.296 ms 2024-05-10T11:28:20.452 INFO:tasks.cephfs.filesystem:scrub status for tag:1e4472de-47b4-48d9-8316-c967fb60110b - {'path': '/', 'tag': '1e4472de-47b4-48d9-8316-c967fb60110b', 'options': 'recursive,force'} 2024-05-10T11:28:20.979 INFO:tasks.workunit.client.0.smithi100.stdout: 1 254652 2.73 MB/sec execute 375 sec latency 15.446 ms 2024-05-10T11:28:21.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:28:21 smithi178 ceph-mon[33359]: pgmap v719: 129 pgs: 129 active+clean; 105 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 1.0 MiB/s wr, 122 op/s 2024-05-10T11:28:21.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:28:21 smithi188 ceph-mon[34075]: pgmap v719: 129 pgs: 129 active+clean; 105 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 1.0 MiB/s wr, 122 op/s 2024-05-10T11:28:21.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:28:21 smithi100 ceph-mon[30296]: pgmap v719: 129 pgs: 129 active+clean; 105 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 1.0 MiB/s wr, 122 op/s 2024-05-10T11:28:21.979 INFO:tasks.workunit.client.0.smithi100.stdout: 1 255227 2.74 MB/sec execute 376 sec latency 11.913 ms 2024-05-10T11:28:22.979 INFO:tasks.workunit.client.0.smithi100.stdout: 1 255708 2.74 MB/sec execute 377 sec latency 13.016 ms 2024-05-10T11:28:23.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:28:23 smithi178 ceph-mon[33359]: pgmap v720: 129 pgs: 129 active+clean; 105 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 1.2 MiB/s wr, 153 op/s 2024-05-10T11:28:23.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:28:23 smithi188 ceph-mon[34075]: pgmap v720: 129 pgs: 129 active+clean; 105 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 1.2 MiB/s wr, 153 op/s 2024-05-10T11:28:23.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:28:23 smithi100 ceph-mon[30296]: pgmap v720: 129 pgs: 129 active+clean; 105 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 1.2 MiB/s wr, 153 op/s 2024-05-10T11:28:23.979 INFO:tasks.workunit.client.0.smithi100.stdout: 1 255982 2.73 MB/sec execute 378 sec latency 15.013 ms 2024-05-10T11:28:24.979 INFO:tasks.workunit.client.0.smithi100.stdout: 1 256461 2.73 MB/sec execute 379 sec latency 15.094 ms 2024-05-10T11:28:25.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:28:25 smithi178 ceph-mon[33359]: pgmap v721: 129 pgs: 129 active+clean; 105 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 1.2 MiB/s wr, 156 op/s 2024-05-10T11:28:25.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:28:25 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:28:25.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:28:25 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:28:25.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:28:25 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:28:25.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:28:25 smithi188 ceph-mon[34075]: pgmap v721: 129 pgs: 129 active+clean; 105 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 1.2 MiB/s wr, 156 op/s 2024-05-10T11:28:25.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:28:25 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:28:25.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:28:25 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:28:25.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:28:25 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:28:25.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:28:25 smithi100 ceph-mon[30296]: pgmap v721: 129 pgs: 129 active+clean; 105 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 1.2 MiB/s wr, 156 op/s 2024-05-10T11:28:25.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:28:25 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:28:25.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:28:25 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:28:25.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:28:25 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:28:25.979 INFO:tasks.workunit.client.0.smithi100.stdout: 1 257018 2.73 MB/sec execute 380 sec latency 15.101 ms 2024-05-10T11:28:26.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:28:26 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:28:26.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:28:26 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:28:26.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:28:26 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:28:26.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:28:26 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:28:26.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:28:26 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:28:26.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:28:26 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:28:26.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:28:26 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:28:26.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:28:26 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:28:26.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:28:26 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:28:26.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:28:26 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:28:26.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:28:26 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:28:26.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:28:26 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:28:26.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:28:26 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:28:26.780 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:28:26 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:28:26.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:28:26 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:28:26.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:28:26 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:28:26.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:28:26 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:28:26.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:28:26 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:28:26.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:28:26 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:28:26.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:28:26 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:28:26.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:28:26 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:28:26.979 INFO:tasks.workunit.client.0.smithi100.stdout: 1 257670 2.73 MB/sec execute 381 sec latency 15.046 ms 2024-05-10T11:28:27.361 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.359+0000 7fa999304640 1 -- 172.21.15.100:0/1170127783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9941045e0 msgr2=0x7fa99410ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:27.361 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.359+0000 7fa999304640 1 --2- 172.21.15.100:0/1170127783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9941045e0 0x7fa99410ae70 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fa98c009ca0 tx=0x7fa98c02a8c0 comp rx=0 tx=0).stop 2024-05-10T11:28:27.361 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.359+0000 7fa999304640 1 -- 172.21.15.100:0/1170127783 shutdown_connections 2024-05-10T11:28:27.361 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.359+0000 7fa999304640 1 --2- 172.21.15.100:0/1170127783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9941045e0 0x7fa99410ae70 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:27.361 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.359+0000 7fa999304640 1 --2- 172.21.15.100:0/1170127783 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa994103c20 0x7fa9941040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:27.361 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.359+0000 7fa999304640 1 --2- 172.21.15.100:0/1170127783 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa994102a30 0x7fa994102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:27.361 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.359+0000 7fa999304640 1 -- 172.21.15.100:0/1170127783 >> 172.21.15.100:0/1170127783 conn(0x7fa9940fdfa0 msgr2=0x7fa9941003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:27.361 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.359+0000 7fa999304640 1 -- 172.21.15.100:0/1170127783 shutdown_connections 2024-05-10T11:28:27.361 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.359+0000 7fa999304640 1 -- 172.21.15.100:0/1170127783 wait complete. 2024-05-10T11:28:27.361 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.359+0000 7fa999304640 1 Processor -- start 2024-05-10T11:28:27.362 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.359+0000 7fa999304640 1 -- start start 2024-05-10T11:28:27.362 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.360+0000 7fa999304640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa994102a30 0x7fa994198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:27.362 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.360+0000 7fa999304640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa994103c20 0x7fa994198fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:27.362 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.360+0000 7fa999304640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9941045e0 0x7fa9941a0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:27.362 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.360+0000 7fa999304640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa99410e460 con 0x7fa994102a30 2024-05-10T11:28:27.362 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.360+0000 7fa999304640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa99410e2e0 con 0x7fa994103c20 2024-05-10T11:28:27.362 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.360+0000 7fa999304640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa99410e5e0 con 0x7fa9941045e0 2024-05-10T11:28:27.362 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.360+0000 7fa992ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa994102a30 0x7fa994198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:27.362 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.360+0000 7fa992ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa994102a30 0x7fa994198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56982/0 (socket says 172.21.15.100:56982) 2024-05-10T11:28:27.362 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.360+0000 7fa992ffd640 1 -- 172.21.15.100:0/4138247603 learned_addr learned my addr 172.21.15.100:0/4138247603 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:28:27.362 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.360+0000 7fa9927fc640 1 --2- 172.21.15.100:0/4138247603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa994103c20 0x7fa994198fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:27.362 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.360+0000 7fa9937fe640 1 --2- 172.21.15.100:0/4138247603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9941045e0 0x7fa9941a0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:27.363 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.360+0000 7fa992ffd640 1 -- 172.21.15.100:0/4138247603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9941045e0 msgr2=0x7fa9941a0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:27.363 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.360+0000 7fa992ffd640 1 --2- 172.21.15.100:0/4138247603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9941045e0 0x7fa9941a0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:27.363 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.360+0000 7fa992ffd640 1 -- 172.21.15.100:0/4138247603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa994103c20 msgr2=0x7fa994198fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:27.363 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.360+0000 7fa992ffd640 1 --2- 172.21.15.100:0/4138247603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa994103c20 0x7fa994198fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:27.363 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.360+0000 7fa992ffd640 1 -- 172.21.15.100:0/4138247603 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9941a06c0 con 0x7fa994102a30 2024-05-10T11:28:27.363 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.360+0000 7fa992ffd640 1 --2- 172.21.15.100:0/4138247603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa994102a30 0x7fa994198aa0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fa98400d770 tx=0x7fa98400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:27.363 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.360+0000 7fa97bfff640 1 -- 172.21.15.100:0/4138247603 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa984017070 con 0x7fa994102a30 2024-05-10T11:28:27.363 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.360+0000 7fa97bfff640 1 -- 172.21.15.100:0/4138247603 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa984009c30 con 0x7fa994102a30 2024-05-10T11:28:27.363 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.360+0000 7fa999304640 1 -- 172.21.15.100:0/4138247603 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9941a09b0 con 0x7fa994102a30 2024-05-10T11:28:27.363 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.360+0000 7fa97bfff640 1 -- 172.21.15.100:0/4138247603 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa984003c50 con 0x7fa994102a30 2024-05-10T11:28:27.363 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.360+0000 7fa999304640 1 -- 172.21.15.100:0/4138247603 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9941a0f00 con 0x7fa994102a30 2024-05-10T11:28:27.366 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.363+0000 7fa99c01b740 1 Processor -- start 2024-05-10T11:28:27.366 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.364+0000 7fa99c01b740 1 -- start start 2024-05-10T11:28:27.366 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.364+0000 7fa97bfff640 1 -- 172.21.15.100:0/4138247603 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fa984003df0 con 0x7fa994102a30 2024-05-10T11:28:27.366 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.364+0000 7fa99c01b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55810c3fd1c0 0x55810c3fd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:27.366 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.364+0000 7fa99c01b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55810c3fdbb0 0x55810c4026b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:27.366 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.364+0000 7fa99c01b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55810c402d20 0x55810c4051b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:27.366 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.364+0000 7fa99c01b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55810c35d5d0 con 0x55810c3fdbb0 2024-05-10T11:28:27.366 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.365+0000 7fa99c01b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55810c316890 con 0x55810c3fd1c0 2024-05-10T11:28:27.366 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.365+0000 7fa99c01b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55810c359590 con 0x55810c402d20 2024-05-10T11:28:27.366 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.365+0000 7fa992ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55810c3fdbb0 0x55810c4026b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:27.366 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.365+0000 7fa9927fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55810c402d20 0x55810c4051b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:27.366 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.365+0000 7fa9927fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55810c402d20 0x55810c4051b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:34596/0 (socket says 172.21.15.100:34596) 2024-05-10T11:28:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.365+0000 7fa9927fc640 1 -- 172.21.15.100:0/2353712179 learned_addr learned my addr 172.21.15.100:0/2353712179 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:28:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.365+0000 7fa9937fe640 1 --2- 172.21.15.100:0/2353712179 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55810c3fd1c0 0x55810c3fd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.365+0000 7fa97bfff640 1 --2- 172.21.15.100:0/4138247603 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa97c0b28a0 0x7fa97c0b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.365+0000 7fa9927fc640 1 -- 172.21.15.100:0/2353712179 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55810c3fd1c0 msgr2=0x55810c3fd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.365+0000 7fa9927fc640 1 --2- 172.21.15.100:0/2353712179 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55810c3fd1c0 0x55810c3fd5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.365+0000 7fa9927fc640 1 -- 172.21.15.100:0/2353712179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55810c3fdbb0 msgr2=0x55810c4026b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.365+0000 7fa9927fc640 1 --2- 172.21.15.100:0/2353712179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55810c3fdbb0 0x55810c4026b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.365+0000 7fa9927fc640 1 -- 172.21.15.100:0/2353712179 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55810c06f210 con 0x55810c402d20 2024-05-10T11:28:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.365+0000 7fa9937fe640 1 --2- 172.21.15.100:0/4138247603 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa97c0b28a0 0x7fa97c0b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.365+0000 7fa97bfff640 1 -- 172.21.15.100:0/4138247603 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fa9840e1ec0 con 0x7fa994102a30 2024-05-10T11:28:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.365+0000 7fa9927fc640 1 --2- 172.21.15.100:0/2353712179 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55810c402d20 0x55810c4051b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa988009ca0 tx=0x7fa988011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f875dcba22090e68 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.366+0000 7fa9797fa640 1 -- 172.21.15.100:0/2353712179 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa988005220 con 0x55810c402d20 2024-05-10T11:28:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.366+0000 7fa9797fa640 1 -- 172.21.15.100:0/2353712179 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa988004c30 con 0x55810c402d20 2024-05-10T11:28:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.366+0000 7fa9797fa640 1 -- 172.21.15.100:0/2353712179 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa988019650 con 0x55810c402d20 2024-05-10T11:28:27.372 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.366+0000 7fa99c01b740 1 -- 172.21.15.100:0/2353712179 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55810c402d20 msgr2=0x55810c4051b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:27.372 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.366+0000 7fa99c01b740 1 --2- 172.21.15.100:0/2353712179 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55810c402d20 0x55810c4051b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa988009ca0 tx=0x7fa988011810 comp rx=0 tx=0).stop 2024-05-10T11:28:27.372 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.366+0000 7fa9937fe640 1 --2- 172.21.15.100:0/4138247603 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa97c0b28a0 0x7fa97c0b4d60 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa98c006010 tx=0x7fa98c0095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:27.372 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.367+0000 7fa99c01b740 1 -- 172.21.15.100:0/2353712179 shutdown_connections 2024-05-10T11:28:27.372 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.367+0000 7fa99c01b740 1 --2- 172.21.15.100:0/2353712179 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55810c402d20 0x55810c4051b0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:27.372 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.367+0000 7fa99c01b740 1 --2- 172.21.15.100:0/2353712179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55810c3fdbb0 0x55810c4026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:27.372 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.367+0000 7fa99c01b740 1 --2- 172.21.15.100:0/2353712179 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55810c3fd1c0 0x55810c3fd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:27.372 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.367+0000 7fa99c01b740 1 -- 172.21.15.100:0/2353712179 >> 172.21.15.100:0/2353712179 conn(0x55810c3f4420 msgr2=0x55810c3f4820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:27.372 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.367+0000 7fa99c01b740 1 -- 172.21.15.100:0/2353712179 shutdown_connections 2024-05-10T11:28:27.372 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.367+0000 7fa99c01b740 1 -- 172.21.15.100:0/2353712179 wait complete. 2024-05-10T11:28:27.372 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.367+0000 7fa99c01b740 1 -- start start 2024-05-10T11:28:27.373 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.367+0000 7fa99c01b740 1 Processor -- start 2024-05-10T11:28:27.373 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.367+0000 7fa99c01b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55810c3fd1c0 0x55810c4072e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:27.373 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.367+0000 7fa99c01b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55810c3fdbb0 0x55810c407820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:27.373 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.367+0000 7fa99c01b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55810c402d20 0x55810c40e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:27.373 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.367+0000 7fa99c01b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55810c405cd0 con 0x55810c3fdbb0 2024-05-10T11:28:27.373 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.367+0000 7fa99c01b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55810c31a370 con 0x55810c402d20 2024-05-10T11:28:27.373 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.367+0000 7fa99c01b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55810c405e50 con 0x55810c3fd1c0 2024-05-10T11:28:27.373 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.367+0000 7fa9927fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55810c3fdbb0 0x55810c407820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:27.373 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.367+0000 7fa99c01b740 10 client.0 _ll_register_callbacks cb 0x55810c06ea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:28:27.373 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.368+0000 7fa9927fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55810c3fdbb0 0x55810c407820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56994/0 (socket says 172.21.15.100:56994) 2024-05-10T11:28:27.373 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.368+0000 7fa9927fc640 1 -- 172.21.15.100:0/1694781673 learned_addr learned my addr 172.21.15.100:0/1694781673 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:28:27.373 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.367+0000 7fa992ffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55810c3fd1c0 0x55810c4072e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:27.373 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.368+0000 7fa9927fc640 1 -- 172.21.15.100:0/1694781673 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55810c3fd1c0 msgr2=0x55810c4072e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:27.374 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.368+0000 7fa9927fc640 1 --2- 172.21.15.100:0/1694781673 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55810c3fd1c0 0x55810c4072e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:27.374 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.368+0000 7fa9927fc640 1 -- 172.21.15.100:0/1694781673 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55810c402d20 msgr2=0x55810c40e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:27.374 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.368+0000 7fa9927fc640 1 --2- 172.21.15.100:0/1694781673 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55810c402d20 0x55810c40e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:27.374 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.368+0000 7fa9927fc640 1 -- 172.21.15.100:0/1694781673 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55810c13b620 con 0x55810c3fdbb0 2024-05-10T11:28:27.374 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.368+0000 7fa992ffd640 1 --2- 172.21.15.100:0/1694781673 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55810c3fd1c0 0x55810c4072e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:28:27.374 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.368+0000 7fa9927fc640 1 --2- 172.21.15.100:0/1694781673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55810c3fdbb0 0x55810c407820 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fa988009380 tx=0x7fa988005090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcf70c76cacdf28b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:27.374 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.368+0000 7fa976ff5640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:28:27.374 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.368+0000 7fa976ff5640 1 -- 172.21.15.100:0/1694781673 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa988029070 con 0x55810c3fdbb0 2024-05-10T11:28:27.374 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.368+0000 7fa976ff5640 1 -- 172.21.15.100:0/1694781673 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fa988004220 con 0x55810c3fdbb0 2024-05-10T11:28:27.374 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.368+0000 7fa99c01b740 1 -- 172.21.15.100:0/1694781673 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa988004c30 con 0x55810c3fdbb0 2024-05-10T11:28:27.374 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.369+0000 7fa976ff5640 1 -- 172.21.15.100:0/1694781673 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa988034050 con 0x55810c3fdbb0 2024-05-10T11:28:27.374 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.369+0000 7fa99c01b740 10 client.17439 fetch_fsmap learned FSMap version 31 2024-05-10T11:28:27.374 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.369+0000 7fa99c01b740 1 -- 172.21.15.100:0/1694781673 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55810c411840 con 0x55810c3fdbb0 2024-05-10T11:28:27.375 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.369+0000 7fa976ff5640 1 -- 172.21.15.100:0/1694781673 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa988004c30 con 0x55810c3fdbb0 2024-05-10T11:28:27.375 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.369+0000 7fa976ff5640 1 -- 172.21.15.100:0/1694781673 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fa988023430 con 0x55810c3fdbb0 2024-05-10T11:28:27.375 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.369+0000 7fa99c01b740 10 client.17439 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:28:27.375 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.369+0000 7fa99c01b740 10 client.17439 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:28:27.375 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.369+0000 7fa99c01b740 20 client.17439 populate_metadata read hostname 'smithi100' 2024-05-10T11:28:27.375 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.369+0000 7fa99c01b740 1 --2- 172.21.15.100:0/1694781673 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55810c412d10 0x55810c4151b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:27.375 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.369+0000 7fa99c01b740 4 client.17439 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:28:27.375 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.369+0000 7fa99c01b740 1 -- 172.21.15.100:0/1694781673 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55810c36b1e0 con 0x55810c412d10 2024-05-10T11:28:27.375 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.370+0000 7fa992ffd640 1 --2- 172.21.15.100:0/1694781673 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55810c412d10 0x55810c4151b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:27.375 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.370+0000 7fa976ff5640 10 client.17439 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:28:27.375 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.370+0000 7fa992ffd640 1 --2- 172.21.15.100:0/1694781673 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55810c412d10 0x55810c4151b0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93cef697a2053173 server_cookie=b2a3d21ef7e35e64 in_seq=0 out_seq=0 2024-05-10T11:28:27.375 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.371+0000 7fa976ff5640 1 -- 172.21.15.100:0/1694781673 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55810c36b1e0 con 0x55810c412d10 2024-05-10T11:28:27.375 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.371+0000 7fa976ff5640 10 client.17439 handle_command_reply: tid=0 2024-05-10T11:28:27.375 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.371+0000 7fa99c01b740 1 client.17439 shutdown 2024-05-10T11:28:27.376 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.371+0000 7fa99c01b740 1 -- 172.21.15.100:0/1694781673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55810c3fdbb0 msgr2=0x55810c407820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:27.376 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.371+0000 7fa99c01b740 1 --2- 172.21.15.100:0/1694781673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55810c3fdbb0 0x55810c407820 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fa988009380 tx=0x7fa988005090 comp rx=0 tx=0).stop 2024-05-10T11:28:27.376 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.371+0000 7fa99c01b740 1 -- 172.21.15.100:0/1694781673 shutdown_connections 2024-05-10T11:28:27.376 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.371+0000 7fa99c01b740 1 --2- 172.21.15.100:0/1694781673 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55810c412d10 0x55810c4151b0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:27.376 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.371+0000 7fa99c01b740 1 --2- 172.21.15.100:0/1694781673 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55810c402d20 0x55810c40e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:27.376 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.371+0000 7fa99c01b740 1 --2- 172.21.15.100:0/1694781673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55810c3fdbb0 0x55810c407820 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:27.376 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.371+0000 7fa99c01b740 1 --2- 172.21.15.100:0/1694781673 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55810c3fd1c0 0x55810c4072e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:27.376 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.371+0000 7fa976ff5640 0 client.17439 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:28:27.376 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.371+0000 7fa99c01b740 1 -- 172.21.15.100:0/1694781673 >> 172.21.15.100:0/1694781673 conn(0x55810c3f4420 msgr2=0x55810c3f3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:27.376 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.371+0000 7fa99c01b740 1 -- 172.21.15.100:0/1694781673 shutdown_connections 2024-05-10T11:28:27.376 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.371+0000 7fa99c01b740 1 -- 172.21.15.100:0/1694781673 wait complete. 2024-05-10T11:28:27.376 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.371+0000 7fa99c01b740 20 client.17439 trim_cache size 0 max 16384 2024-05-10T11:28:27.392 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.387+0000 7fa99c01b740 1 Processor -- start 2024-05-10T11:28:27.392 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.387+0000 7fa99c01b740 1 -- start start 2024-05-10T11:28:27.392 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.388+0000 7fa99c01b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55810c3fd1c0 0x55810c3ef560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:27.392 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.388+0000 7fa99c01b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55810c3fdbb0 0x55810c4072e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:27.392 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.388+0000 7fa99c01b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55810c402d20 0x55810c407820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:27.392 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.388+0000 7fa99c01b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa9941abaf0 con 0x55810c402d20 2024-05-10T11:28:27.392 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.388+0000 7fa99c01b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa9941a93c0 con 0x55810c3fd1c0 2024-05-10T11:28:27.392 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.388+0000 7fa99c01b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa9941a7210 con 0x55810c3fdbb0 2024-05-10T11:28:27.392 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.388+0000 7fa9937fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55810c402d20 0x55810c407820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:27.392 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.388+0000 7fa992ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55810c3fd1c0 0x55810c3ef560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.388+0000 7fa9927fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55810c3fdbb0 0x55810c4072e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.388+0000 7fa992ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55810c3fd1c0 0x55810c3ef560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:54830/0 (socket says 172.21.15.100:54830) 2024-05-10T11:28:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.388+0000 7fa992ffd640 1 -- 172.21.15.100:0/3218923242 learned_addr learned my addr 172.21.15.100:0/3218923242 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:28:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.388+0000 7fa9937fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55810c402d20 0x55810c407820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57008/0 (socket says 172.21.15.100:57008) 2024-05-10T11:28:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.388+0000 7fa9927fc640 1 -- 172.21.15.100:0/3218923242 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55810c3fd1c0 msgr2=0x55810c3ef560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.388+0000 7fa9927fc640 1 --2- 172.21.15.100:0/3218923242 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55810c3fd1c0 0x55810c3ef560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.388+0000 7fa9927fc640 1 -- 172.21.15.100:0/3218923242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55810c402d20 msgr2=0x55810c407820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.388+0000 7fa9927fc640 1 --2- 172.21.15.100:0/3218923242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55810c402d20 0x55810c407820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.388+0000 7fa9927fc640 1 -- 172.21.15.100:0/3218923242 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55810c405bb0 con 0x55810c3fdbb0 2024-05-10T11:28:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.388+0000 7fa992ffd640 1 --2- 172.21.15.100:0/3218923242 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55810c3fd1c0 0x55810c3ef560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:28:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.388+0000 7fa9937fe640 1 --2- 172.21.15.100:0/3218923242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55810c402d20 0x55810c407820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:28:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.388+0000 7fa9927fc640 1 --2- 172.21.15.100:0/3218923242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55810c3fdbb0 0x55810c4072e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa988023880 tx=0x7fa988009b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ced38dba0ac1774 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.388+0000 7fa978ff9640 1 -- 172.21.15.100:0/3218923242 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa988029070 con 0x55810c3fdbb0 2024-05-10T11:28:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.388+0000 7fa978ff9640 1 -- 172.21.15.100:0/3218923242 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa988004500 con 0x55810c3fdbb0 2024-05-10T11:28:27.394 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.388+0000 7fa978ff9640 1 -- 172.21.15.100:0/3218923242 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa988023a70 con 0x55810c3fdbb0 2024-05-10T11:28:27.394 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.389+0000 7fa99c01b740 1 -- 172.21.15.100:0/3218923242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55810c3fdbb0 msgr2=0x55810c4072e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:27.394 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.389+0000 7fa99c01b740 1 --2- 172.21.15.100:0/3218923242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55810c3fdbb0 0x55810c4072e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa988023880 tx=0x7fa988009b70 comp rx=0 tx=0).stop 2024-05-10T11:28:27.394 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.389+0000 7fa99c01b740 1 -- 172.21.15.100:0/3218923242 shutdown_connections 2024-05-10T11:28:27.394 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.389+0000 7fa99c01b740 1 --2- 172.21.15.100:0/3218923242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55810c402d20 0x55810c407820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:27.394 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.389+0000 7fa99c01b740 1 --2- 172.21.15.100:0/3218923242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55810c3fdbb0 0x55810c4072e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:27.394 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.389+0000 7fa99c01b740 1 --2- 172.21.15.100:0/3218923242 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55810c3fd1c0 0x55810c3ef560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:27.394 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.389+0000 7fa99c01b740 1 -- 172.21.15.100:0/3218923242 >> 172.21.15.100:0/3218923242 conn(0x55810c3f4420 msgr2=0x55810c3f4800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:27.394 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.389+0000 7fa99c01b740 1 -- 172.21.15.100:0/3218923242 shutdown_connections 2024-05-10T11:28:27.394 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.389+0000 7fa99c01b740 1 -- 172.21.15.100:0/3218923242 wait complete. 2024-05-10T11:28:27.394 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.389+0000 7fa99c01b740 1 -- start start 2024-05-10T11:28:27.394 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.390+0000 7fa99c01b740 1 Processor -- start 2024-05-10T11:28:27.394 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.390+0000 7fa99c01b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55810c3fd1c0 0x55810c40adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:27.394 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.390+0000 7fa99c01b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55810c3fdbb0 0x55810c40b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:27.395 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.390+0000 7fa99c01b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55810c402d20 0x55810c47b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:27.395 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.390+0000 7fa99c01b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa9941a7880 con 0x55810c402d20 2024-05-10T11:28:27.395 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.390+0000 7fa99c01b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa9941a8cf0 con 0x55810c3fd1c0 2024-05-10T11:28:27.395 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.390+0000 7fa99c01b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa9941a7f50 con 0x55810c3fdbb0 2024-05-10T11:28:27.395 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.390+0000 7fa99c01b740 10 client.0 _ll_register_callbacks cb 0x7fa9840a5900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:28:27.395 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.390+0000 7fa992ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55810c3fd1c0 0x55810c40adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:27.395 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.390+0000 7fa9937fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55810c402d20 0x55810c47b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:27.395 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.390+0000 7fa992ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55810c3fd1c0 0x55810c40adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:54836/0 (socket says 172.21.15.100:54836) 2024-05-10T11:28:27.395 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.390+0000 7fa992ffd640 1 -- 172.21.15.100:0/2272920495 learned_addr learned my addr 172.21.15.100:0/2272920495 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:28:27.395 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.390+0000 7fa9927fc640 1 --2- 172.21.15.100:0/2272920495 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55810c3fdbb0 0x55810c40b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:27.395 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.390+0000 7fa992ffd640 1 -- 172.21.15.100:0/2272920495 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55810c3fdbb0 msgr2=0x55810c40b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:27.395 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.390+0000 7fa992ffd640 1 --2- 172.21.15.100:0/2272920495 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55810c3fdbb0 0x55810c40b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:27.395 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.390+0000 7fa992ffd640 1 -- 172.21.15.100:0/2272920495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55810c402d20 msgr2=0x55810c47b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:27.396 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.390+0000 7fa992ffd640 1 --2- 172.21.15.100:0/2272920495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55810c402d20 0x55810c47b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:27.396 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.390+0000 7fa992ffd640 1 -- 172.21.15.100:0/2272920495 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55810c47bb60 con 0x55810c3fd1c0 2024-05-10T11:28:27.396 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.390+0000 7fa9937fe640 1 --2- 172.21.15.100:0/2272920495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55810c402d20 0x55810c47b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:28:27.396 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.390+0000 7fa992ffd640 1 --2- 172.21.15.100:0/2272920495 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55810c3fd1c0 0x55810c40adc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa9840df6b0 tx=0x7fa984005bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a4fc635ad7dacb9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:27.396 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.390+0000 7fa976ff5640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:28:27.396 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.391+0000 7fa976ff5640 1 -- 172.21.15.100:0/2272920495 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa984020040 con 0x55810c3fd1c0 2024-05-10T11:28:27.396 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.391+0000 7fa976ff5640 1 -- 172.21.15.100:0/2272920495 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fa9840a8690 con 0x55810c3fd1c0 2024-05-10T11:28:27.396 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.391+0000 7fa99c01b740 1 -- 172.21.15.100:0/2272920495 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55810c365050 con 0x55810c3fd1c0 2024-05-10T11:28:27.396 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.391+0000 7fa976ff5640 1 -- 172.21.15.100:0/2272920495 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa9840a9230 con 0x55810c3fd1c0 2024-05-10T11:28:27.396 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.391+0000 7fa99c01b740 10 client.26834 fetch_fsmap learned FSMap version 31 2024-05-10T11:28:27.396 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.391+0000 7fa99c01b740 1 -- 172.21.15.100:0/2272920495 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55810c47ce10 con 0x55810c3fd1c0 2024-05-10T11:28:27.396 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.391+0000 7fa976ff5640 1 -- 172.21.15.100:0/2272920495 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55810c365050 con 0x55810c3fd1c0 2024-05-10T11:28:27.396 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.391+0000 7fa976ff5640 1 -- 172.21.15.100:0/2272920495 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fa9840a4ba0 con 0x55810c3fd1c0 2024-05-10T11:28:27.396 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.391+0000 7fa99c01b740 10 client.26834 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:28:27.397 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.391+0000 7fa99c01b740 10 client.26834 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:28:27.397 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.391+0000 7fa99c01b740 20 client.26834 populate_metadata read hostname 'smithi100' 2024-05-10T11:28:27.397 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.391+0000 7fa99c01b740 1 --2- 172.21.15.100:0/2272920495 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55810c47d740 0x55810c47fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:27.397 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.391+0000 7fa99c01b740 4 client.26834 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:28:27.397 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.391+0000 7fa99c01b740 1 -- 172.21.15.100:0/2272920495 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55810c2e05f0 con 0x55810c47d740 2024-05-10T11:28:27.397 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.391+0000 7fa9927fc640 1 --2- 172.21.15.100:0/2272920495 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55810c47d740 0x55810c47fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:27.397 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.392+0000 7fa976ff5640 10 client.26834 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:28:27.397 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.392+0000 7fa9927fc640 1 --2- 172.21.15.100:0/2272920495 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55810c47d740 0x55810c47fb60 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bca0f15ae215efd server_cookie=76472538189e2bc1 in_seq=0 out_seq=0 2024-05-10T11:28:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.421+0000 7fa976ff5640 1 -- 172.21.15.100:0/2272920495 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55810c2e05f0 con 0x55810c47d740 2024-05-10T11:28:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.421+0000 7fa976ff5640 10 client.26834 handle_command_reply: tid=0 2024-05-10T11:28:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.422+0000 7fa99c01b740 1 client.26834 shutdown 2024-05-10T11:28:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.422+0000 7fa99c01b740 1 -- 172.21.15.100:0/2272920495 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55810c3fd1c0 msgr2=0x55810c40adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.422+0000 7fa99c01b740 1 --2- 172.21.15.100:0/2272920495 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55810c3fd1c0 0x55810c40adc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa9840df6b0 tx=0x7fa984005bf0 comp rx=0 tx=0).stop 2024-05-10T11:28:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.422+0000 7fa99c01b740 1 -- 172.21.15.100:0/2272920495 shutdown_connections 2024-05-10T11:28:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.422+0000 7fa99c01b740 1 --2- 172.21.15.100:0/2272920495 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55810c47d740 0x55810c47fb60 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.422+0000 7fa99c01b740 1 --2- 172.21.15.100:0/2272920495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55810c402d20 0x55810c47b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.422+0000 7fa976ff5640 0 client.26834 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:28:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.422+0000 7fa99c01b740 1 --2- 172.21.15.100:0/2272920495 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55810c3fdbb0 0x55810c40b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.422+0000 7fa99c01b740 1 --2- 172.21.15.100:0/2272920495 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55810c3fd1c0 0x55810c40adc0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.422+0000 7fa99c01b740 1 -- 172.21.15.100:0/2272920495 >> 172.21.15.100:0/2272920495 conn(0x55810c3f4420 msgr2=0x55810c3ef520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.423+0000 7fa99c01b740 1 -- 172.21.15.100:0/2272920495 shutdown_connections 2024-05-10T11:28:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.423+0000 7fa99c01b740 1 -- 172.21.15.100:0/2272920495 wait complete. 2024-05-10T11:28:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.423+0000 7fa99c01b740 20 client.26834 trim_cache size 0 max 16384 2024-05-10T11:28:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:28:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:28:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:28:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:28:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.424+0000 7fa999304640 1 -- 172.21.15.100:0/4138247603 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa97c0b28a0 msgr2=0x7fa97c0b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:27.425 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.424+0000 7fa999304640 1 --2- 172.21.15.100:0/4138247603 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa97c0b28a0 0x7fa97c0b4d60 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa98c006010 tx=0x7fa98c0095c0 comp rx=0 tx=0).stop 2024-05-10T11:28:27.425 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.424+0000 7fa999304640 1 -- 172.21.15.100:0/4138247603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa994102a30 msgr2=0x7fa994198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:27.425 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.424+0000 7fa999304640 1 --2- 172.21.15.100:0/4138247603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa994102a30 0x7fa994198aa0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fa98400d770 tx=0x7fa98400dc30 comp rx=0 tx=0).stop 2024-05-10T11:28:27.425 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.425+0000 7fa999304640 1 -- 172.21.15.100:0/4138247603 shutdown_connections 2024-05-10T11:28:27.425 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.425+0000 7fa999304640 1 --2- 172.21.15.100:0/4138247603 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa97c0b28a0 0x7fa97c0b4d60 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:27.425 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.425+0000 7fa999304640 1 --2- 172.21.15.100:0/4138247603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9941045e0 0x7fa9941a0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:27.425 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.425+0000 7fa999304640 1 --2- 172.21.15.100:0/4138247603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa994103c20 0x7fa994198fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:27.425 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.425+0000 7fa999304640 1 --2- 172.21.15.100:0/4138247603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa994102a30 0x7fa994198aa0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:27.425 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.425+0000 7fa999304640 1 -- 172.21.15.100:0/4138247603 >> 172.21.15.100:0/4138247603 conn(0x7fa9940fdfa0 msgr2=0x7fa994106fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.425+0000 7fa999304640 1 -- 172.21.15.100:0/4138247603 shutdown_connections 2024-05-10T11:28:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:27.426+0000 7fa999304640 1 -- 172.21.15.100:0/4138247603 wait complete. 2024-05-10T11:28:27.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:28:27 smithi178 ceph-mon[33359]: pgmap v722: 129 pgs: 129 active+clean; 107 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.3 MiB/s wr, 158 op/s 2024-05-10T11:28:27.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:28:27 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:28:27.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:28:27 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:28:27.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:28:27 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:28:27.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:28:27 smithi188 ceph-mon[34075]: pgmap v722: 129 pgs: 129 active+clean; 107 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.3 MiB/s wr, 158 op/s 2024-05-10T11:28:27.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:28:27 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:28:27.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:28:27 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:28:27.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:28:27 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:28:27.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:28:27 smithi100 ceph-mon[30296]: pgmap v722: 129 pgs: 129 active+clean; 107 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.3 MiB/s wr, 158 op/s 2024-05-10T11:28:27.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:28:27 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:28:27.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:28:27 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:28:27.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:28:27 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:28:27.979 INFO:tasks.workunit.client.0.smithi100.stdout: 1 258196 2.73 MB/sec execute 382 sec latency 20.487 ms 2024-05-10T11:28:28.979 INFO:tasks.workunit.client.0.smithi100.stdout: 1 258764 2.74 MB/sec execute 383 sec latency 15.000 ms 2024-05-10T11:28:29.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:28:29 smithi178 ceph-mon[33359]: pgmap v723: 129 pgs: 129 active+clean; 104 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 1.4 MiB/s wr, 162 op/s 2024-05-10T11:28:29.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:28:29 smithi188 ceph-mon[34075]: pgmap v723: 129 pgs: 129 active+clean; 104 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 1.4 MiB/s wr, 162 op/s 2024-05-10T11:28:29.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:28:29 smithi100 ceph-mon[30296]: pgmap v723: 129 pgs: 129 active+clean; 104 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 1.4 MiB/s wr, 162 op/s 2024-05-10T11:28:29.979 INFO:tasks.workunit.client.0.smithi100.stdout: 1 259249 2.74 MB/sec execute 384 sec latency 12.728 ms 2024-05-10T11:28:30.979 INFO:tasks.workunit.client.0.smithi100.stdout: 1 259509 2.73 MB/sec execute 385 sec latency 16.594 ms 2024-05-10T11:28:31.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:28:31 smithi178 ceph-mon[33359]: pgmap v724: 129 pgs: 129 active+clean; 102 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 1.1 MiB/s wr, 135 op/s 2024-05-10T11:28:31.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:28:31 smithi188 ceph-mon[34075]: pgmap v724: 129 pgs: 129 active+clean; 102 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 1.1 MiB/s wr, 135 op/s 2024-05-10T11:28:31.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:28:31 smithi100 ceph-mon[30296]: pgmap v724: 129 pgs: 129 active+clean; 102 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 1.1 MiB/s wr, 135 op/s 2024-05-10T11:28:31.980 INFO:tasks.workunit.client.0.smithi100.stdout: 1 259949 2.73 MB/sec execute 386 sec latency 16.881 ms 2024-05-10T11:28:32.979 INFO:tasks.workunit.client.0.smithi100.stdout: 1 260498 2.73 MB/sec execute 387 sec latency 33.175 ms 2024-05-10T11:28:33.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:28:33 smithi178 ceph-mon[33359]: pgmap v725: 129 pgs: 129 active+clean; 104 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-05-10T11:28:33.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:28:33 smithi188 ceph-mon[34075]: pgmap v725: 129 pgs: 129 active+clean; 104 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-05-10T11:28:33.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:28:33 smithi100 ceph-mon[30296]: pgmap v725: 129 pgs: 129 active+clean; 104 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-05-10T11:28:33.979 INFO:tasks.workunit.client.0.smithi100.stdout: 1 261152 2.73 MB/sec execute 388 sec latency 15.386 ms 2024-05-10T11:28:34.979 INFO:tasks.workunit.client.0.smithi100.stdout: 1 261713 2.73 MB/sec execute 389 sec latency 15.992 ms 2024-05-10T11:28:35.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:28:35 smithi178 ceph-mon[33359]: pgmap v726: 129 pgs: 129 active+clean; 104 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 1.4 MiB/s wr, 147 op/s 2024-05-10T11:28:35.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:28:35 smithi188 ceph-mon[34075]: pgmap v726: 129 pgs: 129 active+clean; 104 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 1.4 MiB/s wr, 147 op/s 2024-05-10T11:28:35.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:28:35 smithi100 ceph-mon[30296]: pgmap v726: 129 pgs: 129 active+clean; 104 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 1.4 MiB/s wr, 147 op/s 2024-05-10T11:28:35.979 INFO:tasks.workunit.client.0.smithi100.stdout: 1 262299 2.74 MB/sec execute 390 sec latency 15.801 ms 2024-05-10T11:28:36.979 INFO:tasks.workunit.client.0.smithi100.stdout: 1 262769 2.74 MB/sec execute 391 sec latency 15.040 ms 2024-05-10T11:28:37.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:28:37 smithi178 ceph-mon[33359]: pgmap v727: 129 pgs: 129 active+clean; 105 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 1.4 MiB/s wr, 149 op/s 2024-05-10T11:28:37.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:28:37 smithi188 ceph-mon[34075]: pgmap v727: 129 pgs: 129 active+clean; 105 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 1.4 MiB/s wr, 149 op/s 2024-05-10T11:28:37.790 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:28:37 smithi100 ceph-mon[30296]: pgmap v727: 129 pgs: 129 active+clean; 105 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 1.4 MiB/s wr, 149 op/s 2024-05-10T11:28:37.979 INFO:tasks.workunit.client.0.smithi100.stdout: 1 263018 2.73 MB/sec execute 392 sec latency 13.624 ms 2024-05-10T11:28:38.979 INFO:tasks.workunit.client.0.smithi100.stdout: 1 263456 2.73 MB/sec execute 393 sec latency 14.154 ms 2024-05-10T11:28:39.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:28:39 smithi178 ceph-mon[33359]: pgmap v728: 129 pgs: 129 active+clean; 105 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 1.3 MiB/s wr, 153 op/s 2024-05-10T11:28:39.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:28:39 smithi188 ceph-mon[34075]: pgmap v728: 129 pgs: 129 active+clean; 105 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 1.3 MiB/s wr, 153 op/s 2024-05-10T11:28:39.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:28:39 smithi100 ceph-mon[30296]: pgmap v728: 129 pgs: 129 active+clean; 105 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 1.3 MiB/s wr, 153 op/s 2024-05-10T11:28:39.980 INFO:tasks.workunit.client.0.smithi100.stdout: 1 264021 2.73 MB/sec execute 394 sec latency 13.554 ms 2024-05-10T11:28:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.524+0000 7f3d95fb6640 1 -- 172.21.15.100:0/1004640785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d90101910 msgr2=0x7f3d90101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.524+0000 7f3d95fb6640 1 --2- 172.21.15.100:0/1004640785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d90101910 0x7f3d90101d30 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f3d80009900 tx=0x7f3d8002a410 comp rx=0 tx=0).stop 2024-05-10T11:28:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.524+0000 7f3d95fb6640 1 -- 172.21.15.100:0/1004640785 shutdown_connections 2024-05-10T11:28:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.524+0000 7f3d95fb6640 1 --2- 172.21.15.100:0/1004640785 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d901034c0 0x7f3d90109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.524+0000 7f3d95fb6640 1 --2- 172.21.15.100:0/1004640785 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d90102b00 0x7f3d90102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.524+0000 7f3d95fb6640 1 --2- 172.21.15.100:0/1004640785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d90101910 0x7f3d90101d30 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.524+0000 7f3d95fb6640 1 -- 172.21.15.100:0/1004640785 >> 172.21.15.100:0/1004640785 conn(0x7f3d900fce80 msgr2=0x7f3d900ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.524+0000 7f3d95fb6640 1 -- 172.21.15.100:0/1004640785 shutdown_connections 2024-05-10T11:28:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.524+0000 7f3d95fb6640 1 -- 172.21.15.100:0/1004640785 wait complete. 2024-05-10T11:28:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.525+0000 7f3d95fb6640 1 Processor -- start 2024-05-10T11:28:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.525+0000 7f3d95fb6640 1 -- start start 2024-05-10T11:28:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.525+0000 7f3d95fb6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d90101910 0x7f3d90198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.525+0000 7f3d95fb6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d90102b00 0x7f3d901990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.525+0000 7f3d95fb6640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d901034c0 0x7f3d901a00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.525+0000 7f3d95fb6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9006ad10 con 0x7f3d90101910 2024-05-10T11:28:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.525+0000 7f3d95fb6640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9006ab90 con 0x7f3d901034c0 2024-05-10T11:28:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.525+0000 7f3d95fb6640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9006ae90 con 0x7f3d90102b00 2024-05-10T11:28:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.526+0000 7f3d8f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d90101910 0x7f3d90198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.526+0000 7f3d8f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d90101910 0x7f3d90198b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55476/0 (socket says 172.21.15.100:55476) 2024-05-10T11:28:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.526+0000 7f3d8f7fe640 1 -- 172.21.15.100:0/1763380252 learned_addr learned my addr 172.21.15.100:0/1763380252 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:28:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.526+0000 7f3d8effd640 1 --2- 172.21.15.100:0/1763380252 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d90102b00 0x7f3d901990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.526+0000 7f3d8ffff640 1 --2- 172.21.15.100:0/1763380252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d901034c0 0x7f3d901a00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.526+0000 7f3d8f7fe640 1 -- 172.21.15.100:0/1763380252 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d90102b00 msgr2=0x7f3d901990c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.526+0000 7f3d8f7fe640 1 --2- 172.21.15.100:0/1763380252 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d90102b00 0x7f3d901990c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.526+0000 7f3d8f7fe640 1 -- 172.21.15.100:0/1763380252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d901034c0 msgr2=0x7f3d901a00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.526+0000 7f3d8f7fe640 1 --2- 172.21.15.100:0/1763380252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d901034c0 0x7f3d901a00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.526+0000 7f3d8f7fe640 1 -- 172.21.15.100:0/1763380252 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d901a0750 con 0x7f3d90101910 2024-05-10T11:28:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.526+0000 7f3d8ffff640 1 --2- 172.21.15.100:0/1763380252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d901034c0 0x7f3d901a00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:28:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.526+0000 7f3d8f7fe640 1 --2- 172.21.15.100:0/1763380252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d90101910 0x7f3d90198b80 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f3d80005ab0 tx=0x7f3d800042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.526+0000 7f3d8cff9640 1 -- 172.21.15.100:0/1763380252 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d8003f030 con 0x7f3d90101910 2024-05-10T11:28:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.526+0000 7f3d95fb6640 1 -- 172.21.15.100:0/1763380252 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d901a09e0 con 0x7f3d90101910 2024-05-10T11:28:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.526+0000 7f3d8cff9640 1 -- 172.21.15.100:0/1763380252 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3d80032e10 con 0x7f3d90101910 2024-05-10T11:28:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.526+0000 7f3d8cff9640 1 -- 172.21.15.100:0/1763380252 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d80035070 con 0x7f3d90101910 2024-05-10T11:28:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.526+0000 7f3d95fb6640 1 -- 172.21.15.100:0/1763380252 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d9010c630 con 0x7f3d90101910 2024-05-10T11:28:40.535 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.529+0000 7f3d98daa740 1 Processor -- start 2024-05-10T11:28:40.536 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.529+0000 7f3d98daa740 1 -- start start 2024-05-10T11:28:40.536 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.530+0000 7f3d8cff9640 1 -- 172.21.15.100:0/1763380252 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f3d80042030 con 0x7f3d90101910 2024-05-10T11:28:40.536 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.530+0000 7f3d98daa740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6cab80db0 0x55a6cab811d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:40.536 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.530+0000 7f3d98daa740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a6cab817a0 0x55a6cab862a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:40.536 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.530+0000 7f3d98daa740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6cab86910 0x55a6cab88da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:40.536 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.530+0000 7f3d98daa740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55a6caae55d0 con 0x55a6cab86910 2024-05-10T11:28:40.536 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.530+0000 7f3d98daa740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a6caa9e890 con 0x55a6cab80db0 2024-05-10T11:28:40.536 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.530+0000 7f3d98daa740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55a6caae1590 con 0x55a6cab817a0 2024-05-10T11:28:40.536 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.530+0000 7f3d8effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6cab86910 0x55a6cab88da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:40.536 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.530+0000 7f3d8ffff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6cab80db0 0x55a6cab811d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:40.536 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.530+0000 7f3d8effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6cab86910 0x55a6cab88da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55478/0 (socket says 172.21.15.100:55478) 2024-05-10T11:28:40.536 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.530+0000 7f3d8effd640 1 -- 172.21.15.100:0/2055172732 learned_addr learned my addr 172.21.15.100:0/2055172732 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:28:40.536 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.530+0000 7f3d8ffff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6cab80db0 0x55a6cab811d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:53774/0 (socket says 172.21.15.100:53774) 2024-05-10T11:28:40.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.530+0000 7f3d8f7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a6cab817a0 0x55a6cab862a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:40.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.530+0000 7f3d8ffff640 1 -- 172.21.15.100:0/2055172732 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a6cab817a0 msgr2=0x55a6cab862a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:40.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.530+0000 7f3d8ffff640 1 --2- 172.21.15.100:0/2055172732 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a6cab817a0 0x55a6cab862a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:40.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.530+0000 7f3d8ffff640 1 -- 172.21.15.100:0/2055172732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6cab86910 msgr2=0x55a6cab88da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:40.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.530+0000 7f3d8ffff640 1 --2- 172.21.15.100:0/2055172732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6cab86910 0x55a6cab88da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:40.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.530+0000 7f3d8ffff640 1 -- 172.21.15.100:0/2055172732 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6ca7f7210 con 0x55a6cab80db0 2024-05-10T11:28:40.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.530+0000 7f3d8ffff640 1 --2- 172.21.15.100:0/2055172732 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6cab80db0 0x55a6cab811d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3d88005fd0 tx=0x7f3d88011bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=216de128e3fb74cb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:40.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.531+0000 7f3d75ffb640 1 -- 172.21.15.100:0/2055172732 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d88004d20 con 0x55a6cab80db0 2024-05-10T11:28:40.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.531+0000 7f3d75ffb640 1 -- 172.21.15.100:0/2055172732 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3d8800f9a0 con 0x55a6cab80db0 2024-05-10T11:28:40.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.531+0000 7f3d8cff9640 1 --2- 172.21.15.100:0/1763380252 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f3d780b6cb0 0x7f3d780b9170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:40.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.531+0000 7f3d75ffb640 1 -- 172.21.15.100:0/2055172732 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d88004370 con 0x55a6cab80db0 2024-05-10T11:28:40.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.531+0000 7f3d8f7fe640 1 --2- 172.21.15.100:0/1763380252 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f3d780b6cb0 0x7f3d780b9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:40.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.531+0000 7f3d8cff9640 1 -- 172.21.15.100:0/1763380252 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f3d80102970 con 0x7f3d90101910 2024-05-10T11:28:40.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.531+0000 7f3d98daa740 1 -- 172.21.15.100:0/2055172732 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6cab80db0 msgr2=0x55a6cab811d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:40.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.531+0000 7f3d8f7fe640 1 --2- 172.21.15.100:0/1763380252 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f3d780b6cb0 0x7f3d780b9170 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3d80100480 tx=0x7f3d80108550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:40.538 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.531+0000 7f3d98daa740 1 --2- 172.21.15.100:0/2055172732 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6cab80db0 0x55a6cab811d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3d88005fd0 tx=0x7f3d88011bf0 comp rx=0 tx=0).stop 2024-05-10T11:28:40.538 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.531+0000 7f3d98daa740 1 -- 172.21.15.100:0/2055172732 shutdown_connections 2024-05-10T11:28:40.538 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.531+0000 7f3d98daa740 1 --2- 172.21.15.100:0/2055172732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6cab86910 0x55a6cab88da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:40.538 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.531+0000 7f3d98daa740 1 --2- 172.21.15.100:0/2055172732 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a6cab817a0 0x55a6cab862a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:40.538 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.531+0000 7f3d98daa740 1 --2- 172.21.15.100:0/2055172732 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6cab80db0 0x55a6cab811d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:40.538 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.531+0000 7f3d98daa740 1 -- 172.21.15.100:0/2055172732 >> 172.21.15.100:0/2055172732 conn(0x55a6cab7c420 msgr2=0x55a6cab7c820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:40.538 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.531+0000 7f3d98daa740 1 -- 172.21.15.100:0/2055172732 shutdown_connections 2024-05-10T11:28:40.538 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.531+0000 7f3d98daa740 1 -- 172.21.15.100:0/2055172732 wait complete. 2024-05-10T11:28:40.538 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.532+0000 7f3d98daa740 1 -- start start 2024-05-10T11:28:40.538 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.532+0000 7f3d98daa740 1 Processor -- start 2024-05-10T11:28:40.538 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.532+0000 7f3d98daa740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6cab80db0 0x55a6cab8afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:40.538 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.532+0000 7f3d98daa740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6cab817a0 0x55a6cab8b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:40.538 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.532+0000 7f3d98daa740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a6cab86910 0x55a6cab92520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:40.538 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.532+0000 7f3d98daa740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55a6cab898c0 con 0x55a6cab817a0 2024-05-10T11:28:40.538 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.532+0000 7f3d98daa740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a6caaa2370 con 0x55a6cab80db0 2024-05-10T11:28:40.539 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.532+0000 7f3d98daa740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55a6cab89a40 con 0x55a6cab86910 2024-05-10T11:28:40.539 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.532+0000 7f3d98daa740 10 client.0 _ll_register_callbacks cb 0x55a6ca7f6a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:28:40.539 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.532+0000 7f3d8effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6cab817a0 0x55a6cab8b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:40.539 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.532+0000 7f3d8effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6cab817a0 0x55a6cab8b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55484/0 (socket says 172.21.15.100:55484) 2024-05-10T11:28:40.539 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.532+0000 7f3d8ffff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a6cab86910 0x55a6cab92520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:40.539 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.532+0000 7f3d8effd640 1 -- 172.21.15.100:0/4211152701 learned_addr learned my addr 172.21.15.100:0/4211152701 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:28:40.539 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.532+0000 7f3d8ffff640 1 --2- 172.21.15.100:0/4211152701 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6cab80db0 0x55a6cab8afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:40.539 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.533+0000 7f3d8effd640 1 -- 172.21.15.100:0/4211152701 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a6cab86910 msgr2=0x55a6cab92520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:40.539 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.533+0000 7f3d8effd640 1 --2- 172.21.15.100:0/4211152701 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a6cab86910 0x55a6cab92520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:40.539 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.533+0000 7f3d8effd640 1 -- 172.21.15.100:0/4211152701 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6cab80db0 msgr2=0x55a6cab8afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:40.539 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.533+0000 7f3d8effd640 1 --2- 172.21.15.100:0/4211152701 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6cab80db0 0x55a6cab8afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:40.539 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.533+0000 7f3d8effd640 1 -- 172.21.15.100:0/4211152701 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6ca8c3620 con 0x55a6cab817a0 2024-05-10T11:28:40.539 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.533+0000 7f3d8effd640 1 --2- 172.21.15.100:0/4211152701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6cab817a0 0x55a6cab8b4e0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f3d840126e0 tx=0x7f3d84012ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11182c324f9af149 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:40.539 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.533+0000 7f3d6effd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:28:40.539 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.533+0000 7f3d6effd640 1 -- 172.21.15.100:0/4211152701 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d84012110 con 0x55a6cab817a0 2024-05-10T11:28:40.540 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.533+0000 7f3d98daa740 1 -- 172.21.15.100:0/4211152701 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d8800f9a0 con 0x55a6cab817a0 2024-05-10T11:28:40.540 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.533+0000 7f3d6effd640 1 -- 172.21.15.100:0/4211152701 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f3d84004050 con 0x55a6cab817a0 2024-05-10T11:28:40.540 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.533+0000 7f3d6effd640 1 -- 172.21.15.100:0/4211152701 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f3d84025050 con 0x55a6cab817a0 2024-05-10T11:28:40.540 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.533+0000 7f3d98daa740 10 client.17457 fetch_fsmap learned FSMap version 31 2024-05-10T11:28:40.540 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.533+0000 7f3d98daa740 1 -- 172.21.15.100:0/4211152701 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a6cab955d0 con 0x55a6cab817a0 2024-05-10T11:28:40.540 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.533+0000 7f3d6effd640 1 -- 172.21.15.100:0/4211152701 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3d8800f9a0 con 0x55a6cab817a0 2024-05-10T11:28:40.540 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.534+0000 7f3d6effd640 1 -- 172.21.15.100:0/4211152701 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f3d84003710 con 0x55a6cab817a0 2024-05-10T11:28:40.540 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.534+0000 7f3d98daa740 10 client.17457 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:28:40.540 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.534+0000 7f3d98daa740 10 client.17457 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:28:40.540 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.534+0000 7f3d98daa740 20 client.17457 populate_metadata read hostname 'smithi100' 2024-05-10T11:28:40.540 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.534+0000 7f3d98daa740 1 --2- 172.21.15.100:0/4211152701 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55a6cab96cc0 0x55a6cab99110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:40.540 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.534+0000 7f3d98daa740 4 client.17457 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:28:40.540 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.534+0000 7f3d98daa740 1 -- 172.21.15.100:0/4211152701 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6caaf31e0 con 0x55a6cab96cc0 2024-05-10T11:28:40.540 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.534+0000 7f3d8ffff640 1 --2- 172.21.15.100:0/4211152701 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55a6cab96cc0 0x55a6cab99110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:40.540 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.534+0000 7f3d6effd640 10 client.17457 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:28:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.534+0000 7f3d8ffff640 1 --2- 172.21.15.100:0/4211152701 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55a6cab96cc0 0x55a6cab99110 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eed14b8ba22bffed server_cookie=758271eeb803fcde in_seq=0 out_seq=0 2024-05-10T11:28:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.535+0000 7f3d6effd640 1 -- 172.21.15.100:0/4211152701 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a6caaf31e0 con 0x55a6cab96cc0 2024-05-10T11:28:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.535+0000 7f3d6effd640 10 client.17457 handle_command_reply: tid=0 2024-05-10T11:28:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.535+0000 7f3d98daa740 1 client.17457 shutdown 2024-05-10T11:28:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.535+0000 7f3d98daa740 1 -- 172.21.15.100:0/4211152701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6cab817a0 msgr2=0x55a6cab8b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.535+0000 7f3d98daa740 1 --2- 172.21.15.100:0/4211152701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6cab817a0 0x55a6cab8b4e0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f3d840126e0 tx=0x7f3d84012ba0 comp rx=0 tx=0).stop 2024-05-10T11:28:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.535+0000 7f3d98daa740 1 -- 172.21.15.100:0/4211152701 shutdown_connections 2024-05-10T11:28:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.535+0000 7f3d98daa740 1 --2- 172.21.15.100:0/4211152701 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55a6cab96cc0 0x55a6cab99110 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.535+0000 7f3d98daa740 1 --2- 172.21.15.100:0/4211152701 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a6cab86910 0x55a6cab92520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.535+0000 7f3d6effd640 0 client.17457 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:28:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.535+0000 7f3d98daa740 1 --2- 172.21.15.100:0/4211152701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6cab817a0 0x55a6cab8b4e0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.535+0000 7f3d98daa740 1 --2- 172.21.15.100:0/4211152701 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6cab80db0 0x55a6cab8afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.535+0000 7f3d98daa740 1 -- 172.21.15.100:0/4211152701 >> 172.21.15.100:0/4211152701 conn(0x55a6cab7c420 msgr2=0x55a6cab7b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.535+0000 7f3d98daa740 1 -- 172.21.15.100:0/4211152701 shutdown_connections 2024-05-10T11:28:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.535+0000 7f3d98daa740 1 -- 172.21.15.100:0/4211152701 wait complete. 2024-05-10T11:28:40.542 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.536+0000 7f3d98daa740 20 client.17457 trim_cache size 0 max 16384 2024-05-10T11:28:40.553 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.552+0000 7f3d98daa740 1 Processor -- start 2024-05-10T11:28:40.554 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.552+0000 7f3d98daa740 1 -- start start 2024-05-10T11:28:40.554 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.553+0000 7f3d98daa740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a6cab80db0 0x55a6cab82980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:40.554 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.553+0000 7f3d98daa740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6cab817a0 0x55a6cab82ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:40.554 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.553+0000 7f3d98daa740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6cab86910 0x55a6cab924e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:40.554 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.553+0000 7f3d98daa740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3d901ab9f0 con 0x55a6cab86910 2024-05-10T11:28:40.554 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.553+0000 7f3d98daa740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3d901a92c0 con 0x55a6cab817a0 2024-05-10T11:28:40.554 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.553+0000 7f3d98daa740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3d901a7110 con 0x55a6cab80db0 2024-05-10T11:28:40.554 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.553+0000 7f3d8effd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6cab817a0 0x55a6cab82ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:40.554 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.553+0000 7f3d8ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6cab86910 0x55a6cab924e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:40.554 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.553+0000 7f3d8effd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6cab817a0 0x55a6cab82ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:53790/0 (socket says 172.21.15.100:53790) 2024-05-10T11:28:40.554 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.553+0000 7f3d8effd640 1 -- 172.21.15.100:0/2544281631 learned_addr learned my addr 172.21.15.100:0/2544281631 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:28:40.554 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.553+0000 7f3d8ffff640 1 --2- 172.21.15.100:0/2544281631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a6cab80db0 0x55a6cab82980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:40.554 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.553+0000 7f3d8effd640 1 -- 172.21.15.100:0/2544281631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a6cab80db0 msgr2=0x55a6cab82980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:40.554 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.553+0000 7f3d8effd640 1 --2- 172.21.15.100:0/2544281631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a6cab80db0 0x55a6cab82980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:40.555 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.553+0000 7f3d8effd640 1 -- 172.21.15.100:0/2544281631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6cab86910 msgr2=0x55a6cab924e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:40.555 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.553+0000 7f3d8effd640 1 --2- 172.21.15.100:0/2544281631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6cab86910 0x55a6cab924e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:40.555 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.553+0000 7f3d8effd640 1 -- 172.21.15.100:0/2544281631 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6cab7c250 con 0x55a6cab817a0 2024-05-10T11:28:40.555 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.553+0000 7f3d8ffff640 1 --2- 172.21.15.100:0/2544281631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6cab86910 0x55a6cab924e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:28:40.555 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.553+0000 7f3d8effd640 1 --2- 172.21.15.100:0/2544281631 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6cab817a0 0x55a6cab82ec0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3d840104f0 tx=0x7f3d84012340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86c379dc7bdf6504 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:40.555 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.553+0000 7f3d757fa640 1 -- 172.21.15.100:0/2544281631 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d8402e070 con 0x55a6cab817a0 2024-05-10T11:28:40.555 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.553+0000 7f3d757fa640 1 -- 172.21.15.100:0/2544281631 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3d84005070 con 0x55a6cab817a0 2024-05-10T11:28:40.555 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.554+0000 7f3d757fa640 1 -- 172.21.15.100:0/2544281631 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d8400f920 con 0x55a6cab817a0 2024-05-10T11:28:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.554+0000 7f3d98daa740 1 -- 172.21.15.100:0/2544281631 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6cab817a0 msgr2=0x55a6cab82ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.554+0000 7f3d98daa740 1 --2- 172.21.15.100:0/2544281631 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6cab817a0 0x55a6cab82ec0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3d840104f0 tx=0x7f3d84012340 comp rx=0 tx=0).stop 2024-05-10T11:28:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.554+0000 7f3d98daa740 1 -- 172.21.15.100:0/2544281631 shutdown_connections 2024-05-10T11:28:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.554+0000 7f3d98daa740 1 --2- 172.21.15.100:0/2544281631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6cab86910 0x55a6cab924e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.554+0000 7f3d98daa740 1 --2- 172.21.15.100:0/2544281631 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6cab817a0 0x55a6cab82ec0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.554+0000 7f3d98daa740 1 --2- 172.21.15.100:0/2544281631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a6cab80db0 0x55a6cab82980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.554+0000 7f3d98daa740 1 -- 172.21.15.100:0/2544281631 >> 172.21.15.100:0/2544281631 conn(0x55a6cab7c420 msgr2=0x55a6cab85fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.554+0000 7f3d98daa740 1 -- 172.21.15.100:0/2544281631 shutdown_connections 2024-05-10T11:28:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.554+0000 7f3d98daa740 1 -- 172.21.15.100:0/2544281631 wait complete. 2024-05-10T11:28:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.555+0000 7f3d98daa740 1 -- start start 2024-05-10T11:28:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.555+0000 7f3d98daa740 1 Processor -- start 2024-05-10T11:28:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.555+0000 7f3d98daa740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a6cab80db0 0x55a6cab8fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.555+0000 7f3d98daa740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6cab817a0 0x55a6cab90180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.555+0000 7f3d98daa740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6cab86910 0x55a6cab91190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.555+0000 7f3d98daa740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3d901a7780 con 0x55a6cab86910 2024-05-10T11:28:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.555+0000 7f3d98daa740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3d901a8bf0 con 0x55a6cab817a0 2024-05-10T11:28:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.555+0000 7f3d98daa740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3d901a7e50 con 0x55a6cab80db0 2024-05-10T11:28:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.555+0000 7f3d98daa740 10 client.0 _ll_register_callbacks cb 0x7f3d84009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:28:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.555+0000 7f3d8ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6cab86910 0x55a6cab91190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.555+0000 7f3d8ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6cab86910 0x55a6cab91190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55494/0 (socket says 172.21.15.100:55494) 2024-05-10T11:28:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.555+0000 7f3d8ffff640 1 -- 172.21.15.100:0/2700294017 learned_addr learned my addr 172.21.15.100:0/2700294017 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:28:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.555+0000 7f3d8effd640 1 --2- 172.21.15.100:0/2700294017 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6cab817a0 0x55a6cab90180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.555+0000 7f3d8ffff640 1 --2- 172.21.15.100:0/2700294017 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a6cab80db0 0x55a6cab8fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.555+0000 7f3d8ffff640 1 -- 172.21.15.100:0/2700294017 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a6cab80db0 msgr2=0x55a6cab8fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.555+0000 7f3d8ffff640 1 --2- 172.21.15.100:0/2700294017 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a6cab80db0 0x55a6cab8fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.555+0000 7f3d8ffff640 1 -- 172.21.15.100:0/2700294017 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6cab817a0 msgr2=0x55a6cab90180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.555+0000 7f3d8ffff640 1 --2- 172.21.15.100:0/2700294017 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6cab817a0 0x55a6cab90180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.555+0000 7f3d8ffff640 1 -- 172.21.15.100:0/2700294017 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6cab91780 con 0x55a6cab86910 2024-05-10T11:28:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.556+0000 7f3d8ffff640 1 --2- 172.21.15.100:0/2700294017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6cab86910 0x55a6cab91190 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x55a6cab82000 tx=0x7f3d88009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d17bc967e5dbd76e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.556+0000 7f3d6effd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:28:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.556+0000 7f3d6effd640 1 -- 172.21.15.100:0/2700294017 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d8801e040 con 0x55a6cab86910 2024-05-10T11:28:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.556+0000 7f3d6effd640 1 -- 172.21.15.100:0/2700294017 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f3d8800fdf0 con 0x55a6cab86910 2024-05-10T11:28:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.556+0000 7f3d98daa740 1 -- 172.21.15.100:0/2700294017 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d84005070 con 0x55a6cab86910 2024-05-10T11:28:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.556+0000 7f3d6effd640 1 -- 172.21.15.100:0/2700294017 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f3d8803e050 con 0x55a6cab86910 2024-05-10T11:28:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.556+0000 7f3d98daa740 10 client.17463 fetch_fsmap learned FSMap version 31 2024-05-10T11:28:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.556+0000 7f3d98daa740 1 -- 172.21.15.100:0/2700294017 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a6cac015d0 con 0x55a6cab86910 2024-05-10T11:28:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.556+0000 7f3d6effd640 1 -- 172.21.15.100:0/2700294017 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3d84005070 con 0x55a6cab86910 2024-05-10T11:28:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.556+0000 7f3d6effd640 1 -- 172.21.15.100:0/2700294017 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f3d88024040 con 0x55a6cab86910 2024-05-10T11:28:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.556+0000 7f3d98daa740 10 client.17463 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:28:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.556+0000 7f3d98daa740 10 client.17463 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:28:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.556+0000 7f3d98daa740 20 client.17463 populate_metadata read hostname 'smithi100' 2024-05-10T11:28:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.556+0000 7f3d98daa740 1 --2- 172.21.15.100:0/2700294017 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55a6cac01e70 0x55a6cac04290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.556+0000 7f3d98daa740 4 client.17463 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:28:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.556+0000 7f3d98daa740 1 -- 172.21.15.100:0/2700294017 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a6caaed050 con 0x55a6cac01e70 2024-05-10T11:28:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.557+0000 7f3d8effd640 1 --2- 172.21.15.100:0/2700294017 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55a6cac01e70 0x55a6cac04290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.557+0000 7f3d6effd640 10 client.17463 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:28:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.557+0000 7f3d8effd640 1 --2- 172.21.15.100:0/2700294017 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55a6cac01e70 0x55a6cac04290 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3741f128b7786cd5 server_cookie=5c8dcc55d08e7865 in_seq=0 out_seq=0 2024-05-10T11:28:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.583+0000 7f3d6effd640 1 -- 172.21.15.100:0/2700294017 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a6caaed050 con 0x55a6cac01e70 2024-05-10T11:28:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.583+0000 7f3d6effd640 10 client.17463 handle_command_reply: tid=0 2024-05-10T11:28:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.583+0000 7f3d98daa740 1 client.17463 shutdown 2024-05-10T11:28:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.583+0000 7f3d98daa740 1 -- 172.21.15.100:0/2700294017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6cab86910 msgr2=0x55a6cab91190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.583+0000 7f3d98daa740 1 --2- 172.21.15.100:0/2700294017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6cab86910 0x55a6cab91190 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x55a6cab82000 tx=0x7f3d88009500 comp rx=0 tx=0).stop 2024-05-10T11:28:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.583+0000 7f3d98daa740 1 -- 172.21.15.100:0/2700294017 shutdown_connections 2024-05-10T11:28:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.583+0000 7f3d98daa740 1 --2- 172.21.15.100:0/2700294017 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55a6cac01e70 0x55a6cac04290 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.583+0000 7f3d98daa740 1 --2- 172.21.15.100:0/2700294017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6cab86910 0x55a6cab91190 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.583+0000 7f3d98daa740 1 --2- 172.21.15.100:0/2700294017 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6cab817a0 0x55a6cab90180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.583+0000 7f3d98daa740 1 --2- 172.21.15.100:0/2700294017 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a6cab80db0 0x55a6cab8fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.583+0000 7f3d6effd640 0 client.17463 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:28:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.583+0000 7f3d98daa740 1 -- 172.21.15.100:0/2700294017 >> 172.21.15.100:0/2700294017 conn(0x55a6cab7c420 msgr2=0x55a6cab8af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.584+0000 7f3d98daa740 1 -- 172.21.15.100:0/2700294017 shutdown_connections 2024-05-10T11:28:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.584+0000 7f3d98daa740 1 -- 172.21.15.100:0/2700294017 wait complete. 2024-05-10T11:28:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.584+0000 7f3d98daa740 20 client.17463 trim_cache size 0 max 16384 2024-05-10T11:28:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:28:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:28:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:28:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:28:40.586 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.584+0000 7f3d95fb6640 1 -- 172.21.15.100:0/1763380252 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f3d780b6cb0 msgr2=0x7f3d780b9170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:40.586 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.584+0000 7f3d95fb6640 1 --2- 172.21.15.100:0/1763380252 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f3d780b6cb0 0x7f3d780b9170 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3d80100480 tx=0x7f3d80108550 comp rx=0 tx=0).stop 2024-05-10T11:28:40.586 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.584+0000 7f3d95fb6640 1 -- 172.21.15.100:0/1763380252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d90101910 msgr2=0x7f3d90198b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:40.586 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.584+0000 7f3d95fb6640 1 --2- 172.21.15.100:0/1763380252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d90101910 0x7f3d90198b80 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f3d80005ab0 tx=0x7f3d800042f0 comp rx=0 tx=0).stop 2024-05-10T11:28:40.586 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.584+0000 7f3d95fb6640 1 -- 172.21.15.100:0/1763380252 shutdown_connections 2024-05-10T11:28:40.586 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.584+0000 7f3d95fb6640 1 --2- 172.21.15.100:0/1763380252 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f3d780b6cb0 0x7f3d780b9170 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:40.587 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.584+0000 7f3d95fb6640 1 --2- 172.21.15.100:0/1763380252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d901034c0 0x7f3d901a00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:40.587 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.584+0000 7f3d95fb6640 1 --2- 172.21.15.100:0/1763380252 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d90102b00 0x7f3d901990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:40.587 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.584+0000 7f3d95fb6640 1 --2- 172.21.15.100:0/1763380252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d90101910 0x7f3d90198b80 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:40.587 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.584+0000 7f3d95fb6640 1 -- 172.21.15.100:0/1763380252 >> 172.21.15.100:0/1763380252 conn(0x7f3d900fce80 msgr2=0x7f3d90105f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:40.587 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.585+0000 7f3d95fb6640 1 -- 172.21.15.100:0/1763380252 shutdown_connections 2024-05-10T11:28:40.587 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:40.585+0000 7f3d95fb6640 1 -- 172.21.15.100:0/1763380252 wait complete. 2024-05-10T11:28:40.980 INFO:tasks.workunit.client.0.smithi100.stdout: 1 264654 2.73 MB/sec execute 395 sec latency 21.672 ms 2024-05-10T11:28:41.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:28:41 smithi178 ceph-mon[33359]: pgmap v729: 129 pgs: 129 active+clean; 105 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.2 MiB/s wr, 128 op/s 2024-05-10T11:28:41.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:28:41 smithi188 ceph-mon[34075]: pgmap v729: 129 pgs: 129 active+clean; 105 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.2 MiB/s wr, 128 op/s 2024-05-10T11:28:41.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:28:41 smithi100 ceph-mon[30296]: pgmap v729: 129 pgs: 129 active+clean; 105 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.2 MiB/s wr, 128 op/s 2024-05-10T11:28:41.980 INFO:tasks.workunit.client.0.smithi100.stdout: 1 265119 2.73 MB/sec execute 396 sec latency 14.309 ms 2024-05-10T11:28:42.980 INFO:tasks.workunit.client.0.smithi100.stdout: 1 265821 2.74 MB/sec execute 397 sec latency 46.632 ms 2024-05-10T11:28:43.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:28:43 smithi178 ceph-mon[33359]: pgmap v730: 129 pgs: 129 active+clean; 105 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 1.3 MiB/s wr, 151 op/s 2024-05-10T11:28:43.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:28:43 smithi188 ceph-mon[34075]: pgmap v730: 129 pgs: 129 active+clean; 105 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 1.3 MiB/s wr, 151 op/s 2024-05-10T11:28:43.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:28:43 smithi100 ceph-mon[30296]: pgmap v730: 129 pgs: 129 active+clean; 105 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 1.3 MiB/s wr, 151 op/s 2024-05-10T11:28:43.980 INFO:tasks.workunit.client.0.smithi100.stdout: 1 266285 2.74 MB/sec execute 398 sec latency 13.895 ms 2024-05-10T11:28:44.980 INFO:tasks.workunit.client.0.smithi100.stdout: 1 266522 2.73 MB/sec execute 399 sec latency 13.853 ms 2024-05-10T11:28:45.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:28:45 smithi178 ceph-mon[33359]: pgmap v731: 129 pgs: 129 active+clean; 106 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 1.3 MiB/s wr, 160 op/s 2024-05-10T11:28:45.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:28:45 smithi188 ceph-mon[34075]: pgmap v731: 129 pgs: 129 active+clean; 106 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 1.3 MiB/s wr, 160 op/s 2024-05-10T11:28:45.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:28:45 smithi100 ceph-mon[30296]: pgmap v731: 129 pgs: 129 active+clean; 106 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 1.3 MiB/s wr, 160 op/s 2024-05-10T11:28:45.980 INFO:tasks.workunit.client.0.smithi100.stdout: 1 266924 2.73 MB/sec execute 400 sec latency 17.440 ms 2024-05-10T11:28:46.980 INFO:tasks.workunit.client.0.smithi100.stdout: 1 267451 2.73 MB/sec execute 401 sec latency 15.259 ms 2024-05-10T11:28:47.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:28:47 smithi178 ceph-mon[33359]: pgmap v732: 129 pgs: 129 active+clean; 107 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 1.4 MiB/s wr, 141 op/s 2024-05-10T11:28:47.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:28:47 smithi188 ceph-mon[34075]: pgmap v732: 129 pgs: 129 active+clean; 107 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 1.4 MiB/s wr, 141 op/s 2024-05-10T11:28:47.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:28:47 smithi100 ceph-mon[30296]: pgmap v732: 129 pgs: 129 active+clean; 107 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 1.4 MiB/s wr, 141 op/s 2024-05-10T11:28:47.980 INFO:tasks.workunit.client.0.smithi100.stdout: 1 268063 2.73 MB/sec execute 402 sec latency 12.134 ms 2024-05-10T11:28:48.980 INFO:tasks.workunit.client.0.smithi100.stdout: 1 268611 2.73 MB/sec execute 403 sec latency 17.260 ms 2024-05-10T11:28:49.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:28:49 smithi178 ceph-mon[33359]: pgmap v733: 129 pgs: 129 active+clean; 109 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 1.4 MiB/s wr, 163 op/s 2024-05-10T11:28:49.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:28:49 smithi188 ceph-mon[34075]: pgmap v733: 129 pgs: 129 active+clean; 109 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 1.4 MiB/s wr, 163 op/s 2024-05-10T11:28:49.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:28:49 smithi100 ceph-mon[30296]: pgmap v733: 129 pgs: 129 active+clean; 109 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 1.4 MiB/s wr, 163 op/s 2024-05-10T11:28:49.980 INFO:tasks.workunit.client.0.smithi100.stdout: 1 269348 2.74 MB/sec execute 404 sec latency 19.530 ms 2024-05-10T11:28:50.454 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T11:28:50.720 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:28:50.980 INFO:tasks.workunit.client.0.smithi100.stdout: 1 269743 2.74 MB/sec execute 405 sec latency 16.232 ms 2024-05-10T11:28:51.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:28:51 smithi178 ceph-mon[33359]: pgmap v734: 129 pgs: 129 active+clean; 108 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 1.1 MiB/s wr, 137 op/s 2024-05-10T11:28:51.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:28:51 smithi188 ceph-mon[34075]: pgmap v734: 129 pgs: 129 active+clean; 108 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 1.1 MiB/s wr, 137 op/s 2024-05-10T11:28:51.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:28:51 smithi100 ceph-mon[30296]: pgmap v734: 129 pgs: 129 active+clean; 108 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 1.1 MiB/s wr, 137 op/s 2024-05-10T11:28:51.980 INFO:tasks.workunit.client.0.smithi100.stdout: 1 270012 2.73 MB/sec execute 406 sec latency 15.860 ms 2024-05-10T11:28:52.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.743+0000 7f72c370d700 1 -- 172.21.15.100:0/3938075705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72bc1029e0 msgr2=0x7f72bc102dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:52.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.743+0000 7f72c370d700 1 --2- 172.21.15.100:0/3938075705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72bc1029e0 0x7f72bc102dc0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f72b80099c0 tx=0x7f72b8009cd0 comp rx=0 tx=0).stop 2024-05-10T11:28:52.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.743+0000 7f72c370d700 1 -- 172.21.15.100:0/3938075705 shutdown_connections 2024-05-10T11:28:52.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.743+0000 7f72c370d700 1 --2- 172.21.15.100:0/3938075705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72bc107730 0x7f72bc10f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:52.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.743+0000 7f72c370d700 1 --2- 172.21.15.100:0/3938075705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72bc1029e0 0x7f72bc102dc0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:52.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.743+0000 7f72c370d700 1 --2- 172.21.15.100:0/3938075705 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72bc103390 0x7f72bc107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:52.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.743+0000 7f72c370d700 1 -- 172.21.15.100:0/3938075705 >> 172.21.15.100:0/3938075705 conn(0x7f72bc079350 msgr2=0x7f72bc079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:52.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.743+0000 7f72c370d700 1 -- 172.21.15.100:0/3938075705 shutdown_connections 2024-05-10T11:28:52.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.743+0000 7f72c370d700 1 -- 172.21.15.100:0/3938075705 wait complete. 2024-05-10T11:28:52.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.744+0000 7f72c370d700 1 Processor -- start 2024-05-10T11:28:52.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.744+0000 7f72c370d700 1 -- start start 2024-05-10T11:28:52.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.745+0000 7f72c370d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72bc1029e0 0x7f72bc19f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:52.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.745+0000 7f72c370d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72bc103390 0x7f72bc19fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:52.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.745+0000 7f72c370d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72bc107730 0x7f72bc199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:52.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.745+0000 7f72c370d700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f72bc112460 con 0x7f72bc103390 2024-05-10T11:28:52.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.745+0000 7f72c14a9700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72bc1029e0 0x7f72bc19f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:52.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.745+0000 7f72c370d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f72bc1122e0 con 0x7f72bc107730 2024-05-10T11:28:52.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.745+0000 7f72c0ca8700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72bc103390 0x7f72bc19fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:52.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.745+0000 7f72c370d700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f72bc1125e0 con 0x7f72bc1029e0 2024-05-10T11:28:52.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.745+0000 7f72c14a9700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72bc1029e0 0x7f72bc19f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:59860/0 (socket says 172.21.15.100:59860) 2024-05-10T11:28:52.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.745+0000 7f72c14a9700 1 -- 172.21.15.100:0/2098388781 learned_addr learned my addr 172.21.15.100:0/2098388781 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:28:52.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.746+0000 7f72c1caa700 1 --2- 172.21.15.100:0/2098388781 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72bc107730 0x7f72bc199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:52.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.746+0000 7f72c0ca8700 1 -- 172.21.15.100:0/2098388781 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72bc1029e0 msgr2=0x7f72bc19f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:52.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.746+0000 7f72c0ca8700 1 --2- 172.21.15.100:0/2098388781 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72bc1029e0 0x7f72bc19f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:52.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.746+0000 7f72c0ca8700 1 -- 172.21.15.100:0/2098388781 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72bc107730 msgr2=0x7f72bc199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:52.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.746+0000 7f72c0ca8700 1 --2- 172.21.15.100:0/2098388781 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72bc107730 0x7f72bc199870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:52.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.746+0000 7f72c0ca8700 1 -- 172.21.15.100:0/2098388781 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72bc199fd0 con 0x7f72bc103390 2024-05-10T11:28:52.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.746+0000 7f72c0ca8700 1 --2- 172.21.15.100:0/2098388781 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72bc103390 0x7f72bc19fbf0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f72bc077590 tx=0x7f72ac00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:52.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.746+0000 7f72b27fc700 1 -- 172.21.15.100:0/2098388781 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ac00cbd0 con 0x7f72bc103390 2024-05-10T11:28:52.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.746+0000 7f72b27fc700 1 -- 172.21.15.100:0/2098388781 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f72ac00ee80 con 0x7f72bc103390 2024-05-10T11:28:52.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.746+0000 7f72b27fc700 1 -- 172.21.15.100:0/2098388781 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ac005020 con 0x7f72bc103390 2024-05-10T11:28:52.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.746+0000 7f72c370d700 1 -- 172.21.15.100:0/2098388781 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72bc19a2c0 con 0x7f72bc103390 2024-05-10T11:28:52.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.746+0000 7f72c370d700 1 -- 172.21.15.100:0/2098388781 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72bc1a6580 con 0x7f72bc103390 2024-05-10T11:28:52.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.749+0000 7f72b27fc700 1 -- 172.21.15.100:0/2098388781 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f72ac0051c0 con 0x7f72bc103390 2024-05-10T11:28:52.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.749+0000 7f72b27fc700 1 --2- 172.21.15.100:0/2098388781 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f72a80b2900 0x7f72a80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:52.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.750+0000 7f72c14a9700 1 --2- 172.21.15.100:0/2098388781 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f72a80b2900 0x7f72a80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:52.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.750+0000 7f72b27fc700 1 -- 172.21.15.100:0/2098388781 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f72ac0e2460 con 0x7f72bc103390 2024-05-10T11:28:52.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.750+0000 7f72cdfa0b80 1 Processor -- start 2024-05-10T11:28:52.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.750+0000 7f72c14a9700 1 --2- 172.21.15.100:0/2098388781 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f72a80b2900 0x7f72a80b4dc0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f72b8009990 tx=0x7f72b8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:52.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.750+0000 7f72cdfa0b80 1 -- start start 2024-05-10T11:28:52.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.750+0000 7f72cdfa0b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abd654b4d0 0x55abd654b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:52.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.750+0000 7f72cdfa0b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55abd654bdf0 0x55abd654f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:52.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.750+0000 7f72cdfa0b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55abd654fc80 0x55abd6550100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:52.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.750+0000 7f72cdfa0b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55abd649d860 con 0x55abd654b4d0 2024-05-10T11:28:52.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.750+0000 7f72cdfa0b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55abd643c7c0 con 0x55abd654fc80 2024-05-10T11:28:52.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.750+0000 7f72cdfa0b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55abd6499820 con 0x55abd654bdf0 2024-05-10T11:28:52.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.750+0000 7f72c14a9700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abd654b4d0 0x55abd654b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:52.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.751+0000 7f72c0ca8700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55abd654bdf0 0x55abd654f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:52.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.751+0000 7f72c14a9700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abd654b4d0 0x55abd654b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58006/0 (socket says 172.21.15.100:58006) 2024-05-10T11:28:52.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.751+0000 7f72c14a9700 1 -- 172.21.15.100:0/1051209502 learned_addr learned my addr 172.21.15.100:0/1051209502 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:28:52.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.751+0000 7f72c1caa700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55abd654fc80 0x55abd6550100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:52.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.751+0000 7f72c0ca8700 1 -- 172.21.15.100:0/1051209502 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55abd654fc80 msgr2=0x55abd6550100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:52.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.751+0000 7f72c0ca8700 1 --2- 172.21.15.100:0/1051209502 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55abd654fc80 0x55abd6550100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:52.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.751+0000 7f72c0ca8700 1 -- 172.21.15.100:0/1051209502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abd654b4d0 msgr2=0x55abd654b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:52.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.751+0000 7f72c0ca8700 1 --2- 172.21.15.100:0/1051209502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abd654b4d0 0x55abd654b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:52.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.751+0000 7f72c0ca8700 1 -- 172.21.15.100:0/1051209502 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abd63ea8d0 con 0x55abd654bdf0 2024-05-10T11:28:52.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.751+0000 7f72c14a9700 1 --2- 172.21.15.100:0/1051209502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abd654b4d0 0x55abd654b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:28:52.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.751+0000 7f72c0ca8700 1 --2- 172.21.15.100:0/1051209502 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55abd654bdf0 0x55abd654f6f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f72ac0a9420 tx=0x7f72ac0a8f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5130aeef61d3332 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:52.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.751+0000 7f72a77fe700 1 -- 172.21.15.100:0/1051209502 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ac01c070 con 0x55abd654bdf0 2024-05-10T11:28:52.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.751+0000 7f72a77fe700 1 -- 172.21.15.100:0/1051209502 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f72ac0b0e50 con 0x55abd654bdf0 2024-05-10T11:28:52.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.751+0000 7f72a77fe700 1 -- 172.21.15.100:0/1051209502 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ac0e21c0 con 0x55abd654bdf0 2024-05-10T11:28:52.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.752+0000 7f72cdfa0b80 1 -- 172.21.15.100:0/1051209502 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55abd654bdf0 msgr2=0x55abd654f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:52.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.752+0000 7f72cdfa0b80 1 --2- 172.21.15.100:0/1051209502 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55abd654bdf0 0x55abd654f6f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f72ac0a9420 tx=0x7f72ac0a8f80 comp rx=0 tx=0).stop 2024-05-10T11:28:52.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.752+0000 7f72cdfa0b80 1 -- 172.21.15.100:0/1051209502 shutdown_connections 2024-05-10T11:28:52.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.752+0000 7f72cdfa0b80 1 --2- 172.21.15.100:0/1051209502 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55abd654fc80 0x55abd6550100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:52.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.752+0000 7f72cdfa0b80 1 --2- 172.21.15.100:0/1051209502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abd654b4d0 0x55abd654b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:52.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.752+0000 7f72cdfa0b80 1 --2- 172.21.15.100:0/1051209502 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55abd654bdf0 0x55abd654f6f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:52.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.752+0000 7f72cdfa0b80 1 -- 172.21.15.100:0/1051209502 >> 172.21.15.100:0/1051209502 conn(0x55abd6477100 msgr2=0x55abd6545c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:52.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.752+0000 7f72cdfa0b80 1 -- 172.21.15.100:0/1051209502 shutdown_connections 2024-05-10T11:28:52.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.752+0000 7f72cdfa0b80 1 -- 172.21.15.100:0/1051209502 wait complete. 2024-05-10T11:28:52.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.752+0000 7f72cdfa0b80 1 -- start start 2024-05-10T11:28:52.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.753+0000 7f72cdfa0b80 1 Processor -- start 2024-05-10T11:28:52.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.753+0000 7f72cdfa0b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abd654b4d0 0x55abd65570f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:52.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.753+0000 7f72cdfa0b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55abd654bdf0 0x55abd6554490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:52.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.753+0000 7f72cdfa0b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55abd654fc80 0x55abd65549d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:52.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.753+0000 7f72cdfa0b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55abd6557e30 con 0x55abd654b4d0 2024-05-10T11:28:52.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.753+0000 7f72cdfa0b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55abd6403c30 con 0x55abd654fc80 2024-05-10T11:28:52.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.753+0000 7f72cdfa0b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55abd6557fb0 con 0x55abd654bdf0 2024-05-10T11:28:52.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.753+0000 7f72cdfa0b80 10 client.0 _ll_register_callbacks cb 0x55abd6435920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:28:52.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.753+0000 7f72c14a9700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abd654b4d0 0x55abd65570f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:52.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.753+0000 7f72c0ca8700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55abd654bdf0 0x55abd6554490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:52.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.753+0000 7f72c0ca8700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55abd654bdf0 0x55abd6554490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:59874/0 (socket says 172.21.15.100:59874) 2024-05-10T11:28:52.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.753+0000 7f72c0ca8700 1 -- 172.21.15.100:0/2499570413 learned_addr learned my addr 172.21.15.100:0/2499570413 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:28:52.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.753+0000 7f72c1caa700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55abd654fc80 0x55abd65549d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:52.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.754+0000 7f72c1caa700 1 -- 172.21.15.100:0/2499570413 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55abd654bdf0 msgr2=0x55abd6554490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:52.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.754+0000 7f72c1caa700 1 --2- 172.21.15.100:0/2499570413 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55abd654bdf0 0x55abd6554490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:52.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.754+0000 7f72c1caa700 1 -- 172.21.15.100:0/2499570413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abd654b4d0 msgr2=0x55abd65570f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:52.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.754+0000 7f72c1caa700 1 --2- 172.21.15.100:0/2499570413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abd654b4d0 0x55abd65570f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:52.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.754+0000 7f72c1caa700 1 -- 172.21.15.100:0/2499570413 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abd6439690 con 0x55abd654fc80 2024-05-10T11:28:52.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.754+0000 7f72c1caa700 1 --2- 172.21.15.100:0/2499570413 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55abd654fc80 0x55abd65549d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f72b4011980 tx=0x7f72b4011c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=791870ee82dc74e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:52.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.754+0000 7f72a4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:28:52.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.754+0000 7f72a4ff9700 1 -- 172.21.15.100:0/2499570413 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b4019030 con 0x55abd654fc80 2024-05-10T11:28:52.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.754+0000 7f72a4ff9700 1 -- 172.21.15.100:0/2499570413 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f72b4010040 con 0x55abd654fc80 2024-05-10T11:28:52.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.754+0000 7f72cdfa0b80 1 -- 172.21.15.100:0/2499570413 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72ac0b0e50 con 0x55abd654fc80 2024-05-10T11:28:52.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.754+0000 7f72a4ff9700 1 -- 172.21.15.100:0/2499570413 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f72b4023050 con 0x55abd654fc80 2024-05-10T11:28:52.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.754+0000 7f72cdfa0b80 10 client.26861 fetch_fsmap learned FSMap version 31 2024-05-10T11:28:52.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.754+0000 7f72cdfa0b80 1 -- 172.21.15.100:0/2499570413 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55abd6562710 con 0x55abd654fc80 2024-05-10T11:28:52.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.755+0000 7f72a4ff9700 1 -- 172.21.15.100:0/2499570413 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f72ac0b0e50 con 0x55abd654fc80 2024-05-10T11:28:52.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.755+0000 7f72a4ff9700 1 -- 172.21.15.100:0/2499570413 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f72b400f500 con 0x55abd654fc80 2024-05-10T11:28:52.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.755+0000 7f72cdfa0b80 10 client.26861 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:28:52.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.755+0000 7f72cdfa0b80 10 client.26861 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:28:52.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.755+0000 7f72cdfa0b80 20 client.26861 populate_metadata read hostname 'smithi100' 2024-05-10T11:28:52.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.755+0000 7f72cdfa0b80 1 --2- 172.21.15.100:0/2499570413 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55abd6564d50 0x55abd6567170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:52.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.755+0000 7f72cdfa0b80 4 client.26861 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:28:52.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.755+0000 7f72cdfa0b80 1 -- 172.21.15.100:0/2499570413 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55abd6517650 con 0x55abd6564d50 2024-05-10T11:28:52.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.755+0000 7f72c14a9700 1 --2- 172.21.15.100:0/2499570413 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55abd6564d50 0x55abd6567170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:52.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.755+0000 7f72c14a9700 1 --2- 172.21.15.100:0/2499570413 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55abd6564d50 0x55abd6567170 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0b3fb2d6d924a38 server_cookie=92547f8b89b3a277 in_seq=0 out_seq=0 2024-05-10T11:28:52.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.755+0000 7f72a4ff9700 10 client.26861 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:28:52.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.756+0000 7f72a4ff9700 1 -- 172.21.15.100:0/2499570413 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55abd6517650 con 0x55abd6564d50 2024-05-10T11:28:52.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.756+0000 7f72a4ff9700 10 client.26861 handle_command_reply: tid=0 2024-05-10T11:28:52.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.756+0000 7f72cdfa0b80 1 client.26861 shutdown 2024-05-10T11:28:52.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.756+0000 7f72cdfa0b80 1 -- 172.21.15.100:0/2499570413 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55abd654fc80 msgr2=0x55abd65549d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:52.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.756+0000 7f72cdfa0b80 1 --2- 172.21.15.100:0/2499570413 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55abd654fc80 0x55abd65549d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f72b4011980 tx=0x7f72b4011c90 comp rx=0 tx=0).stop 2024-05-10T11:28:52.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.756+0000 7f72cdfa0b80 1 -- 172.21.15.100:0/2499570413 shutdown_connections 2024-05-10T11:28:52.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.756+0000 7f72cdfa0b80 1 --2- 172.21.15.100:0/2499570413 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55abd6564d50 0x55abd6567170 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:52.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.756+0000 7f72cdfa0b80 1 --2- 172.21.15.100:0/2499570413 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55abd654fc80 0x55abd65549d0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:52.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.756+0000 7f72cdfa0b80 1 --2- 172.21.15.100:0/2499570413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abd654b4d0 0x55abd65570f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:52.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.756+0000 7f72cdfa0b80 1 --2- 172.21.15.100:0/2499570413 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55abd654bdf0 0x55abd6554490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:52.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.756+0000 7f72cdfa0b80 1 -- 172.21.15.100:0/2499570413 >> 172.21.15.100:0/2499570413 conn(0x55abd6477100 msgr2=0x55abd6540720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:52.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.757+0000 7f72a4ff9700 0 client.26861 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:28:52.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.757+0000 7f72cdfa0b80 1 -- 172.21.15.100:0/2499570413 shutdown_connections 2024-05-10T11:28:52.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.757+0000 7f72cdfa0b80 1 -- 172.21.15.100:0/2499570413 wait complete. 2024-05-10T11:28:52.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.757+0000 7f72cdfa0b80 20 client.26861 trim_cache size 0 max 16384 2024-05-10T11:28:52.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.779+0000 7f72cdfa0b80 1 Processor -- start 2024-05-10T11:28:52.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.779+0000 7f72cdfa0b80 1 -- start start 2024-05-10T11:28:52.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.779+0000 7f72cdfa0b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abd654b4d0 0x55abd6554ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:52.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.780+0000 7f72cdfa0b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55abd654bdf0 0x55abd6560ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:52.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.780+0000 7f72cdfa0b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55abd654fc80 0x55abd6556d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:52.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.780+0000 7f72cdfa0b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f72bc1b19a0 con 0x55abd654b4d0 2024-05-10T11:28:52.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.780+0000 7f72cdfa0b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f72bc1af230 con 0x55abd654fc80 2024-05-10T11:28:52.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.780+0000 7f72cdfa0b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f72bc1ad080 con 0x55abd654bdf0 2024-05-10T11:28:52.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.780+0000 7f72c14a9700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abd654b4d0 0x55abd6554ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:52.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.780+0000 7f72c14a9700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abd654b4d0 0x55abd6554ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58030/0 (socket says 172.21.15.100:58030) 2024-05-10T11:28:52.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.780+0000 7f72c14a9700 1 -- 172.21.15.100:0/1472234820 learned_addr learned my addr 172.21.15.100:0/1472234820 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:28:52.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.780+0000 7f72c0ca8700 1 --2- 172.21.15.100:0/1472234820 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55abd654bdf0 0x55abd6560ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:52.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.780+0000 7f72c1caa700 1 --2- 172.21.15.100:0/1472234820 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55abd654fc80 0x55abd6556d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:52.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.780+0000 7f72c14a9700 1 -- 172.21.15.100:0/1472234820 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55abd654bdf0 msgr2=0x55abd6560ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:52.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.780+0000 7f72c14a9700 1 --2- 172.21.15.100:0/1472234820 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55abd654bdf0 0x55abd6560ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:52.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.780+0000 7f72c14a9700 1 -- 172.21.15.100:0/1472234820 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55abd654fc80 msgr2=0x55abd6556d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:52.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.780+0000 7f72c14a9700 1 --2- 172.21.15.100:0/1472234820 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55abd654fc80 0x55abd6556d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:52.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.780+0000 7f72c14a9700 1 -- 172.21.15.100:0/1472234820 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abd653fa80 con 0x55abd654b4d0 2024-05-10T11:28:52.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.780+0000 7f72c0ca8700 1 --2- 172.21.15.100:0/1472234820 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55abd654bdf0 0x55abd6560ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:28:52.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.780+0000 7f72c14a9700 1 --2- 172.21.15.100:0/1472234820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abd654b4d0 0x55abd6554ac0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f72b80059d0 tx=0x7f72b8020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6766cc8c7e64b86c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:52.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.780+0000 7f72a6ffd700 1 -- 172.21.15.100:0/1472234820 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b803c030 con 0x55abd654b4d0 2024-05-10T11:28:52.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.780+0000 7f72a6ffd700 1 -- 172.21.15.100:0/1472234820 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f72b8037070 con 0x55abd654b4d0 2024-05-10T11:28:52.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.780+0000 7f72a6ffd700 1 -- 172.21.15.100:0/1472234820 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b800b790 con 0x55abd654b4d0 2024-05-10T11:28:52.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.781+0000 7f72cdfa0b80 1 -- 172.21.15.100:0/1472234820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abd654b4d0 msgr2=0x55abd6554ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:52.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.781+0000 7f72cdfa0b80 1 --2- 172.21.15.100:0/1472234820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abd654b4d0 0x55abd6554ac0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f72b80059d0 tx=0x7f72b8020000 comp rx=0 tx=0).stop 2024-05-10T11:28:52.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.781+0000 7f72cdfa0b80 1 -- 172.21.15.100:0/1472234820 shutdown_connections 2024-05-10T11:28:52.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.781+0000 7f72cdfa0b80 1 --2- 172.21.15.100:0/1472234820 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55abd654fc80 0x55abd6556d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:52.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.781+0000 7f72cdfa0b80 1 --2- 172.21.15.100:0/1472234820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abd654b4d0 0x55abd6554ac0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:52.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.781+0000 7f72cdfa0b80 1 --2- 172.21.15.100:0/1472234820 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55abd654bdf0 0x55abd6560ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:52.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.781+0000 7f72cdfa0b80 1 -- 172.21.15.100:0/1472234820 >> 172.21.15.100:0/1472234820 conn(0x55abd6477100 msgr2=0x55abd653fe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:52.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.781+0000 7f72cdfa0b80 1 -- 172.21.15.100:0/1472234820 shutdown_connections 2024-05-10T11:28:52.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.781+0000 7f72cdfa0b80 1 -- 172.21.15.100:0/1472234820 wait complete. 2024-05-10T11:28:52.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.782+0000 7f72cdfa0b80 1 -- start start 2024-05-10T11:28:52.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.782+0000 7f72cdfa0b80 1 Processor -- start 2024-05-10T11:28:52.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.782+0000 7f72cdfa0b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abd654b4d0 0x55abd655d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:52.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.782+0000 7f72cdfa0b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55abd654bdf0 0x55abd6558b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:52.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.782+0000 7f72cdfa0b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55abd654fc80 0x55abd6559080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:52.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.782+0000 7f72cdfa0b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f72bc1ad6f0 con 0x55abd654b4d0 2024-05-10T11:28:52.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.782+0000 7f72cdfa0b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f72bc1aeb60 con 0x55abd654bdf0 2024-05-10T11:28:52.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.782+0000 7f72cdfa0b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f72bc1addc0 con 0x55abd654fc80 2024-05-10T11:28:52.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.782+0000 7f72cdfa0b80 10 client.0 _ll_register_callbacks cb 0x55abd6435920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:28:52.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.782+0000 7f72c14a9700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abd654b4d0 0x55abd655d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:52.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.782+0000 7f72c14a9700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abd654b4d0 0x55abd655d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58032/0 (socket says 172.21.15.100:58032) 2024-05-10T11:28:52.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.782+0000 7f72c14a9700 1 -- 172.21.15.100:0/3725497890 learned_addr learned my addr 172.21.15.100:0/3725497890 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:28:52.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.782+0000 7f72c1caa700 1 --2- 172.21.15.100:0/3725497890 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55abd654fc80 0x55abd6559080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:52.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.782+0000 7f72c0ca8700 1 --2- 172.21.15.100:0/3725497890 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55abd654bdf0 0x55abd6558b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:52.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.783+0000 7f72c1caa700 1 -- 172.21.15.100:0/3725497890 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55abd654bdf0 msgr2=0x55abd6558b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:52.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.783+0000 7f72c1caa700 1 --2- 172.21.15.100:0/3725497890 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55abd654bdf0 0x55abd6558b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:52.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.783+0000 7f72c1caa700 1 -- 172.21.15.100:0/3725497890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abd654b4d0 msgr2=0x55abd655d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:52.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.783+0000 7f72c1caa700 1 --2- 172.21.15.100:0/3725497890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abd654b4d0 0x55abd655d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:52.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.783+0000 7f72c1caa700 1 -- 172.21.15.100:0/3725497890 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abd65598c0 con 0x55abd654fc80 2024-05-10T11:28:52.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.783+0000 7f72c14a9700 1 --2- 172.21.15.100:0/3725497890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abd654b4d0 0x55abd655d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:28:52.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.783+0000 7f72c1caa700 1 --2- 172.21.15.100:0/3725497890 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55abd654fc80 0x55abd6559080 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f72b40074e0 tx=0x7f72b400f1d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=876f4b18999b950c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:52.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.783+0000 7f72a4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:28:52.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.783+0000 7f72a4ff9700 1 -- 172.21.15.100:0/3725497890 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b402c060 con 0x55abd654fc80 2024-05-10T11:28:52.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.783+0000 7f72cdfa0b80 1 -- 172.21.15.100:0/3725497890 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72b8037070 con 0x55abd654fc80 2024-05-10T11:28:52.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.783+0000 7f72a4ff9700 1 -- 172.21.15.100:0/3725497890 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f72b400fb90 con 0x55abd654fc80 2024-05-10T11:28:52.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.783+0000 7f72a4ff9700 1 -- 172.21.15.100:0/3725497890 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f72b4019040 con 0x55abd654fc80 2024-05-10T11:28:52.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.783+0000 7f72cdfa0b80 10 client.26935 fetch_fsmap learned FSMap version 31 2024-05-10T11:28:52.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.783+0000 7f72cdfa0b80 1 -- 172.21.15.100:0/3725497890 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55abd65d3a60 con 0x55abd654fc80 2024-05-10T11:28:52.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.783+0000 7f72a4ff9700 1 -- 172.21.15.100:0/3725497890 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f72b8037070 con 0x55abd654fc80 2024-05-10T11:28:52.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.783+0000 7f72a4ff9700 1 -- 172.21.15.100:0/3725497890 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f72b4015c00 con 0x55abd654fc80 2024-05-10T11:28:52.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.784+0000 7f72cdfa0b80 10 client.26935 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:28:52.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.784+0000 7f72cdfa0b80 10 client.26935 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:28:52.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.784+0000 7f72cdfa0b80 20 client.26935 populate_metadata read hostname 'smithi100' 2024-05-10T11:28:52.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.784+0000 7f72cdfa0b80 1 --2- 172.21.15.100:0/3725497890 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55abd65d40c0 0x55abd65d64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:52.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.784+0000 7f72cdfa0b80 4 client.26935 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T11:28:52.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.784+0000 7f72cdfa0b80 1 -- 172.21.15.100:0/3725497890 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55abd64ab240 con 0x55abd65d40c0 2024-05-10T11:28:52.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.784+0000 7f72c14a9700 1 --2- 172.21.15.100:0/3725497890 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55abd65d40c0 0x55abd65d64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:52.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.784+0000 7f72a4ff9700 10 client.26935 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:28:52.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.784+0000 7f72c14a9700 1 --2- 172.21.15.100:0/3725497890 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55abd65d40c0 0x55abd65d64e0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=587f9c02e1ba033e server_cookie=d1e8e4a6da59025 in_seq=0 out_seq=0 2024-05-10T11:28:52.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.785+0000 7f72a4ff9700 1 -- 172.21.15.100:0/3725497890 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55abd64ab240 con 0x55abd65d40c0 2024-05-10T11:28:52.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.785+0000 7f72a4ff9700 10 client.26935 handle_command_reply: tid=0 2024-05-10T11:28:52.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.785+0000 7f72cdfa0b80 1 client.26935 shutdown 2024-05-10T11:28:52.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.785+0000 7f72cdfa0b80 1 -- 172.21.15.100:0/3725497890 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55abd654fc80 msgr2=0x55abd6559080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:52.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.785+0000 7f72cdfa0b80 1 --2- 172.21.15.100:0/3725497890 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55abd654fc80 0x55abd6559080 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f72b40074e0 tx=0x7f72b400f1d0 comp rx=0 tx=0).stop 2024-05-10T11:28:52.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.785+0000 7f72cdfa0b80 1 -- 172.21.15.100:0/3725497890 shutdown_connections 2024-05-10T11:28:52.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.785+0000 7f72cdfa0b80 1 --2- 172.21.15.100:0/3725497890 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55abd65d40c0 0x55abd65d64e0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:52.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.785+0000 7f72cdfa0b80 1 --2- 172.21.15.100:0/3725497890 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55abd654fc80 0x55abd6559080 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:52.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.785+0000 7f72cdfa0b80 1 --2- 172.21.15.100:0/3725497890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abd654b4d0 0x55abd655d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:52.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.785+0000 7f72cdfa0b80 1 --2- 172.21.15.100:0/3725497890 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55abd654bdf0 0x55abd6558b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:52.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.785+0000 7f72cdfa0b80 1 -- 172.21.15.100:0/3725497890 >> 172.21.15.100:0/3725497890 conn(0x55abd6477100 msgr2=0x55abd6552990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:52.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.785+0000 7f72a4ff9700 0 client.26935 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:28:52.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.785+0000 7f72cdfa0b80 1 -- 172.21.15.100:0/3725497890 shutdown_connections 2024-05-10T11:28:52.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.786+0000 7f72cdfa0b80 1 -- 172.21.15.100:0/3725497890 wait complete. 2024-05-10T11:28:52.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.786+0000 7f72cdfa0b80 20 client.26935 trim_cache size 0 max 16384 2024-05-10T11:28:52.793 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:28:52.793 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-05-10T11:28:52.794 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-05-10T11:28:52.794 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:28:52.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.786+0000 7f72c370d700 1 -- 172.21.15.100:0/2098388781 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f72a80b2900 msgr2=0x7f72a80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:52.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.786+0000 7f72c370d700 1 --2- 172.21.15.100:0/2098388781 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f72a80b2900 0x7f72a80b4dc0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f72b8009990 tx=0x7f72b8005960 comp rx=0 tx=0).stop 2024-05-10T11:28:52.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.786+0000 7f72c370d700 1 -- 172.21.15.100:0/2098388781 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72bc103390 msgr2=0x7f72bc19fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:52.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.786+0000 7f72c370d700 1 --2- 172.21.15.100:0/2098388781 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72bc103390 0x7f72bc19fbf0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f72bc077590 tx=0x7f72ac00eca0 comp rx=0 tx=0).stop 2024-05-10T11:28:52.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.786+0000 7f72c370d700 1 -- 172.21.15.100:0/2098388781 shutdown_connections 2024-05-10T11:28:52.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.786+0000 7f72c370d700 1 --2- 172.21.15.100:0/2098388781 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f72a80b2900 0x7f72a80b4dc0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:52.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.786+0000 7f72c370d700 1 --2- 172.21.15.100:0/2098388781 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72bc107730 0x7f72bc199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:52.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.786+0000 7f72c370d700 1 --2- 172.21.15.100:0/2098388781 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72bc1029e0 0x7f72bc19f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:52.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.786+0000 7f72c370d700 1 --2- 172.21.15.100:0/2098388781 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72bc103390 0x7f72bc19fbf0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:52.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.786+0000 7f72c370d700 1 -- 172.21.15.100:0/2098388781 >> 172.21.15.100:0/2098388781 conn(0x7f72bc079350 msgr2=0x7f72bc105fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:52.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.786+0000 7f72c370d700 1 -- 172.21.15.100:0/2098388781 shutdown_connections 2024-05-10T11:28:52.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:52.786+0000 7f72c370d700 1 -- 172.21.15.100:0/2098388781 wait complete. 2024-05-10T11:28:52.980 INFO:tasks.workunit.client.0.smithi100.stdout: 1 270315 2.73 MB/sec execute 407 sec latency 15.822 ms 2024-05-10T11:28:53.510 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T11:28:53.510 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T11:28:53.685 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.684+0000 7fdfef8c8640 1 -- 172.21.15.100:0/2746010387 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfe8108710 msgr2=0x7fdfe810ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:53.685 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.684+0000 7fdfef8c8640 1 --2- 172.21.15.100:0/2746010387 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfe8108710 0x7fdfe810ab60 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0009ca0 tx=0x7fdfe002a8c0 comp rx=0 tx=0).stop 2024-05-10T11:28:53.685 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.684+0000 7fdfef8c8640 1 -- 172.21.15.100:0/2746010387 shutdown_connections 2024-05-10T11:28:53.685 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.684+0000 7fdfef8c8640 1 --2- 172.21.15.100:0/2746010387 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfe8108710 0x7fdfe810ab60 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:53.685 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.684+0000 7fdfef8c8640 1 --2- 172.21.15.100:0/2746010387 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdfe8105de0 0x7fdfe81081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:53.686 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.684+0000 7fdfef8c8640 1 --2- 172.21.15.100:0/2746010387 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfe81034b0 0x7fdfe81058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:53.686 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.685+0000 7fdfef8c8640 1 -- 172.21.15.100:0/2746010387 >> 172.21.15.100:0/2746010387 conn(0x7fdfe80fcec0 msgr2=0x7fdfe80ff2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:53.686 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.685+0000 7fdfef8c8640 1 -- 172.21.15.100:0/2746010387 shutdown_connections 2024-05-10T11:28:53.686 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.685+0000 7fdfef8c8640 1 -- 172.21.15.100:0/2746010387 wait complete. 2024-05-10T11:28:53.688 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.685+0000 7fdfef8c8640 1 Processor -- start 2024-05-10T11:28:53.688 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.685+0000 7fdfef8c8640 1 -- start start 2024-05-10T11:28:53.688 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.686+0000 7fdfef8c8640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdfe81034b0 0x7fdfe8198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:53.688 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.686+0000 7fdfef8c8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfe8105de0 0x7fdfe8198fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:53.688 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.686+0000 7fdfef8c8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfe8108710 0x7fdfe81a0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:53.688 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.686+0000 7fdfef8c8640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe810e460 con 0x7fdfe8108710 2024-05-10T11:28:53.688 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.686+0000 7fdfef8c8640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe810e2e0 con 0x7fdfe81034b0 2024-05-10T11:28:53.689 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.686+0000 7fdfef8c8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe810e5e0 con 0x7fdfe8105de0 2024-05-10T11:28:53.689 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.686+0000 7fdfece63640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfe8105de0 0x7fdfe8198fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:53.689 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.686+0000 7fdfece63640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfe8105de0 0x7fdfe8198fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:59920/0 (socket says 172.21.15.100:59920) 2024-05-10T11:28:53.689 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.686+0000 7fdfece63640 1 -- 172.21.15.100:0/1522205645 learned_addr learned my addr 172.21.15.100:0/1522205645 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:28:53.689 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.686+0000 7fdfede65640 1 --2- 172.21.15.100:0/1522205645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfe8108710 0x7fdfe81a0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:53.689 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.686+0000 7fdfed664640 1 --2- 172.21.15.100:0/1522205645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdfe81034b0 0x7fdfe8198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:53.689 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.686+0000 7fdfece63640 1 -- 172.21.15.100:0/1522205645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdfe81034b0 msgr2=0x7fdfe8198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.686+0000 7fdfece63640 1 --2- 172.21.15.100:0/1522205645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdfe81034b0 0x7fdfe8198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.686+0000 7fdfece63640 1 -- 172.21.15.100:0/1522205645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfe8108710 msgr2=0x7fdfe81a0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.686+0000 7fdfece63640 1 --2- 172.21.15.100:0/1522205645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfe8108710 0x7fdfe81a0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.686+0000 7fdfece63640 1 -- 172.21.15.100:0/1522205645 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfe81a0690 con 0x7fdfe8105de0 2024-05-10T11:28:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.686+0000 7fdfede65640 1 --2- 172.21.15.100:0/1522205645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfe8108710 0x7fdfe81a0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:28:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.687+0000 7fdfece63640 1 --2- 172.21.15.100:0/1522205645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfe8105de0 0x7fdfe8198fb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdfd800e9c0 tx=0x7fdfd800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.687+0000 7fdfde7fc640 1 -- 172.21.15.100:0/1522205645 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd800cd80 con 0x7fdfe8105de0 2024-05-10T11:28:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.687+0000 7fdfde7fc640 1 -- 172.21.15.100:0/1522205645 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdfd80054b0 con 0x7fdfe8105de0 2024-05-10T11:28:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.687+0000 7fdfef8c8640 1 -- 172.21.15.100:0/1522205645 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfe81a0980 con 0x7fdfe8105de0 2024-05-10T11:28:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.687+0000 7fdfde7fc640 1 -- 172.21.15.100:0/1522205645 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd8010430 con 0x7fdfe8105de0 2024-05-10T11:28:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.687+0000 7fdfef8c8640 1 -- 172.21.15.100:0/1522205645 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfe81a0ed0 con 0x7fdfe8105de0 2024-05-10T11:28:53.698 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.690+0000 7fdfde7fc640 1 -- 172.21.15.100:0/1522205645 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fdfd8004f20 con 0x7fdfe8105de0 2024-05-10T11:28:53.698 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.690+0000 7fdff25de740 1 Processor -- start 2024-05-10T11:28:53.698 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.690+0000 7fdff25de740 1 -- start start 2024-05-10T11:28:53.698 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.690+0000 7fdff25de740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5555e60751c0 0x5555e60755e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:53.698 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.690+0000 7fdff25de740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5555e6075bb0 0x5555e607a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.691+0000 7fdff25de740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5555e607ad20 0x5555e607d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.691+0000 7fdff25de740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5555e5fd55d0 con 0x5555e6075bb0 2024-05-10T11:28:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.691+0000 7fdff25de740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5555e5f8e890 con 0x5555e60751c0 2024-05-10T11:28:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.691+0000 7fdff25de740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5555e5fd1590 con 0x5555e607ad20 2024-05-10T11:28:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.691+0000 7fdfed664640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5555e6075bb0 0x5555e607a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.691+0000 7fdfed664640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5555e6075bb0 0x5555e607a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58048/0 (socket says 172.21.15.100:58048) 2024-05-10T11:28:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.691+0000 7fdfed664640 1 -- 172.21.15.100:0/3695089652 learned_addr learned my addr 172.21.15.100:0/3695089652 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:28:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.691+0000 7fdfede65640 1 --2- 172.21.15.100:0/3695089652 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5555e60751c0 0x5555e60755e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.691+0000 7fdfece63640 1 --2- 172.21.15.100:0/3695089652 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5555e607ad20 0x5555e607d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.691+0000 7fdfde7fc640 1 --2- 172.21.15.100:0/1522205645 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fdfd40b28a0 0x7fdfd40b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.691+0000 7fdfed664640 1 -- 172.21.15.100:0/3695089652 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5555e607ad20 msgr2=0x5555e607d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.691+0000 7fdfed664640 1 --2- 172.21.15.100:0/3695089652 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5555e607ad20 0x5555e607d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.691+0000 7fdfed664640 1 -- 172.21.15.100:0/3695089652 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5555e60751c0 msgr2=0x5555e60755e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.691+0000 7fdfed664640 1 --2- 172.21.15.100:0/3695089652 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5555e60751c0 0x5555e60755e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.691+0000 7fdfed664640 1 -- 172.21.15.100:0/3695089652 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555e5ce7210 con 0x5555e6075bb0 2024-05-10T11:28:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.691+0000 7fdfece63640 1 --2- 172.21.15.100:0/3695089652 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5555e607ad20 0x5555e607d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:28:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.691+0000 7fdfede65640 1 --2- 172.21.15.100:0/1522205645 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fdfd40b28a0 0x7fdfd40b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.691+0000 7fdfde7fc640 1 -- 172.21.15.100:0/1522205645 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fdfd80e2bb0 con 0x7fdfe8105de0 2024-05-10T11:28:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.691+0000 7fdfed664640 1 --2- 172.21.15.100:0/3695089652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5555e6075bb0 0x5555e607a6b0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4009cf0 tx=0x7fdfe4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96c7c2db4e770c28 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.692+0000 7fdfd37fe640 1 -- 172.21.15.100:0/3695089652 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe401e070 con 0x5555e6075bb0 2024-05-10T11:28:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.692+0000 7fdfd37fe640 1 -- 172.21.15.100:0/3695089652 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdfe4019ac0 con 0x5555e6075bb0 2024-05-10T11:28:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.692+0000 7fdfd37fe640 1 -- 172.21.15.100:0/3695089652 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe400e950 con 0x5555e6075bb0 2024-05-10T11:28:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.693+0000 7fdff25de740 1 -- 172.21.15.100:0/3695089652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5555e6075bb0 msgr2=0x5555e607a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.693+0000 7fdff25de740 1 --2- 172.21.15.100:0/3695089652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5555e6075bb0 0x5555e607a6b0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4009cf0 tx=0x7fdfe4011810 comp rx=0 tx=0).stop 2024-05-10T11:28:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.693+0000 7fdfede65640 1 --2- 172.21.15.100:0/1522205645 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fdfd40b28a0 0x7fdfd40b4d60 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0004af0 tx=0x7fdfe00095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.693+0000 7fdff25de740 1 -- 172.21.15.100:0/3695089652 shutdown_connections 2024-05-10T11:28:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.693+0000 7fdff25de740 1 --2- 172.21.15.100:0/3695089652 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5555e607ad20 0x5555e607d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.693+0000 7fdff25de740 1 --2- 172.21.15.100:0/3695089652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5555e6075bb0 0x5555e607a6b0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.693+0000 7fdff25de740 1 --2- 172.21.15.100:0/3695089652 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5555e60751c0 0x5555e60755e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.693+0000 7fdff25de740 1 -- 172.21.15.100:0/3695089652 >> 172.21.15.100:0/3695089652 conn(0x5555e606c420 msgr2=0x5555e606c820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.693+0000 7fdff25de740 1 -- 172.21.15.100:0/3695089652 shutdown_connections 2024-05-10T11:28:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.693+0000 7fdff25de740 1 -- 172.21.15.100:0/3695089652 wait complete. 2024-05-10T11:28:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.693+0000 7fdff25de740 1 -- start start 2024-05-10T11:28:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.693+0000 7fdff25de740 1 Processor -- start 2024-05-10T11:28:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.693+0000 7fdff25de740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5555e60751c0 0x5555e607f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.694+0000 7fdff25de740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5555e6075bb0 0x5555e607f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.694+0000 7fdff25de740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5555e607ad20 0x5555e6086940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.694+0000 7fdff25de740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5555e607dcd0 con 0x5555e607ad20 2024-05-10T11:28:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.694+0000 7fdff25de740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5555e5f92370 con 0x5555e60751c0 2024-05-10T11:28:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.694+0000 7fdff25de740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5555e607de50 con 0x5555e6075bb0 2024-05-10T11:28:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.694+0000 7fdff25de740 10 client.0 _ll_register_callbacks cb 0x5555e5ce6a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:28:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.694+0000 7fdfece63640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5555e6075bb0 0x5555e607f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.694+0000 7fdfede65640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5555e607ad20 0x5555e6086940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.694+0000 7fdfed664640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5555e60751c0 0x5555e607f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.694+0000 7fdfece63640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5555e6075bb0 0x5555e607f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:59944/0 (socket says 172.21.15.100:59944) 2024-05-10T11:28:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.694+0000 7fdfed664640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5555e60751c0 0x5555e607f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:54972/0 (socket says 172.21.15.100:54972) 2024-05-10T11:28:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.694+0000 7fdfece63640 1 -- 172.21.15.100:0/4282405866 learned_addr learned my addr 172.21.15.100:0/4282405866 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:28:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.694+0000 7fdfece63640 1 -- 172.21.15.100:0/4282405866 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5555e60751c0 msgr2=0x5555e607f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.694+0000 7fdfece63640 1 --2- 172.21.15.100:0/4282405866 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5555e60751c0 0x5555e607f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.694+0000 7fdfece63640 1 -- 172.21.15.100:0/4282405866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5555e607ad20 msgr2=0x5555e6086940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.694+0000 7fdfece63640 1 --2- 172.21.15.100:0/4282405866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5555e607ad20 0x5555e6086940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.694+0000 7fdfece63640 1 -- 172.21.15.100:0/4282405866 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555e5db3620 con 0x5555e6075bb0 2024-05-10T11:28:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.694+0000 7fdfed664640 1 --2- 172.21.15.100:0/4282405866 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5555e60751c0 0x5555e607f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:28:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.694+0000 7fdfede65640 1 --2- 172.21.15.100:0/4282405866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5555e607ad20 0x5555e6086940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:28:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.694+0000 7fdfece63640 1 --2- 172.21.15.100:0/4282405866 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5555e6075bb0 0x5555e607f970 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdfd800cfd0 tx=0x7fdfd80a96a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d75150fc59f60bd7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.694+0000 7fdfd0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:28:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.695+0000 7fdfd0ff9640 1 -- 172.21.15.100:0/4282405866 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd80ae220 con 0x5555e6075bb0 2024-05-10T11:28:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.695+0000 7fdff25de740 1 -- 172.21.15.100:0/4282405866 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfe4019ac0 con 0x5555e6075bb0 2024-05-10T11:28:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.695+0000 7fdfd0ff9640 1 -- 172.21.15.100:0/4282405866 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fdfd80a90d0 con 0x5555e6075bb0 2024-05-10T11:28:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.695+0000 7fdfd0ff9640 1 -- 172.21.15.100:0/4282405866 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fdfd80b2cd0 con 0x5555e6075bb0 2024-05-10T11:28:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.695+0000 7fdff25de740 10 client.26944 fetch_fsmap learned FSMap version 31 2024-05-10T11:28:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.695+0000 7fdff25de740 1 -- 172.21.15.100:0/4282405866 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5555e60899b0 con 0x5555e6075bb0 2024-05-10T11:28:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.695+0000 7fdfd0ff9640 1 -- 172.21.15.100:0/4282405866 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdfe4019ac0 con 0x5555e6075bb0 2024-05-10T11:28:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.695+0000 7fdfd0ff9640 1 -- 172.21.15.100:0/4282405866 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fdfd80a8cb0 con 0x5555e6075bb0 2024-05-10T11:28:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.695+0000 7fdff25de740 10 client.26944 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:28:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.695+0000 7fdff25de740 10 client.26944 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:28:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.695+0000 7fdff25de740 20 client.26944 populate_metadata read hostname 'smithi100' 2024-05-10T11:28:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.695+0000 7fdff25de740 1 --2- 172.21.15.100:0/4282405866 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5555e608b0a0 0x5555e608d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.695+0000 7fdff25de740 4 client.26944 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:28:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.695+0000 7fdff25de740 1 -- 172.21.15.100:0/4282405866 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555e5fe31e0 con 0x5555e608b0a0 2024-05-10T11:28:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.695+0000 7fdfed664640 1 --2- 172.21.15.100:0/4282405866 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5555e608b0a0 0x5555e608d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.696+0000 7fdfed664640 1 --2- 172.21.15.100:0/4282405866 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5555e608b0a0 0x5555e608d4d0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41e4345652c06d0b server_cookie=c807bfabc95587be in_seq=0 out_seq=0 2024-05-10T11:28:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.696+0000 7fdfd0ff9640 10 client.26944 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:28:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.697+0000 7fdfd0ff9640 1 -- 172.21.15.100:0/4282405866 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5555e5fe31e0 con 0x5555e608b0a0 2024-05-10T11:28:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.697+0000 7fdfd0ff9640 10 client.26944 handle_command_reply: tid=0 2024-05-10T11:28:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.697+0000 7fdff25de740 1 client.26944 shutdown 2024-05-10T11:28:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.697+0000 7fdff25de740 1 -- 172.21.15.100:0/4282405866 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5555e6075bb0 msgr2=0x5555e607f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.697+0000 7fdff25de740 1 --2- 172.21.15.100:0/4282405866 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5555e6075bb0 0x5555e607f970 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdfd800cfd0 tx=0x7fdfd80a96a0 comp rx=0 tx=0).stop 2024-05-10T11:28:53.706 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.697+0000 7fdff25de740 1 -- 172.21.15.100:0/4282405866 shutdown_connections 2024-05-10T11:28:53.706 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.697+0000 7fdff25de740 1 --2- 172.21.15.100:0/4282405866 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5555e608b0a0 0x5555e608d4d0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:53.706 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.697+0000 7fdff25de740 1 --2- 172.21.15.100:0/4282405866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5555e607ad20 0x5555e6086940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:53.706 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.697+0000 7fdff25de740 1 --2- 172.21.15.100:0/4282405866 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5555e6075bb0 0x5555e607f970 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:53.706 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.697+0000 7fdff25de740 1 --2- 172.21.15.100:0/4282405866 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5555e60751c0 0x5555e607f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:53.706 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.697+0000 7fdff25de740 1 -- 172.21.15.100:0/4282405866 >> 172.21.15.100:0/4282405866 conn(0x5555e606c420 msgr2=0x5555e606b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:53.706 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.697+0000 7fdfd0ff9640 0 client.26944 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:28:53.706 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.697+0000 7fdff25de740 1 -- 172.21.15.100:0/4282405866 shutdown_connections 2024-05-10T11:28:53.706 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.697+0000 7fdff25de740 1 -- 172.21.15.100:0/4282405866 wait complete. 2024-05-10T11:28:53.706 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.697+0000 7fdff25de740 20 client.26944 trim_cache size 0 max 16384 2024-05-10T11:28:53.716 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.714+0000 7fdff25de740 1 Processor -- start 2024-05-10T11:28:53.716 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.714+0000 7fdff25de740 1 -- start start 2024-05-10T11:28:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.714+0000 7fdff25de740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5555e60751c0 0x5555e607ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.714+0000 7fdff25de740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5555e6075bb0 0x5555e607f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.714+0000 7fdff25de740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5555e607ad20 0x5555e607f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.714+0000 7fdff25de740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe81ab940 con 0x5555e607ad20 2024-05-10T11:28:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.714+0000 7fdff25de740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe81a9210 con 0x5555e6075bb0 2024-05-10T11:28:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.714+0000 7fdff25de740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe81a7060 con 0x5555e60751c0 2024-05-10T11:28:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.715+0000 7fdfed664640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5555e60751c0 0x5555e607ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.715+0000 7fdfed664640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5555e60751c0 0x5555e607ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:59952/0 (socket says 172.21.15.100:59952) 2024-05-10T11:28:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.715+0000 7fdfede65640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5555e607ad20 0x5555e607f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.715+0000 7fdfed664640 1 -- 172.21.15.100:0/1845961991 learned_addr learned my addr 172.21.15.100:0/1845961991 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:28:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.715+0000 7fdfece63640 1 --2- 172.21.15.100:0/1845961991 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5555e6075bb0 0x5555e607f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.715+0000 7fdfed664640 1 -- 172.21.15.100:0/1845961991 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5555e6075bb0 msgr2=0x5555e607f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.715+0000 7fdfed664640 1 --2- 172.21.15.100:0/1845961991 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5555e6075bb0 0x5555e607f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.715+0000 7fdfed664640 1 -- 172.21.15.100:0/1845961991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5555e607ad20 msgr2=0x5555e607f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.715+0000 7fdfed664640 1 --2- 172.21.15.100:0/1845961991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5555e607ad20 0x5555e607f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.715+0000 7fdfed664640 1 -- 172.21.15.100:0/1845961991 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555e60897d0 con 0x5555e60751c0 2024-05-10T11:28:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.715+0000 7fdfed664640 1 --2- 172.21.15.100:0/1845961991 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5555e60751c0 0x5555e607ea50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4009c40 tx=0x7fdfe4009510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b759477faf67f443 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.715+0000 7fdfd2ffd640 1 -- 172.21.15.100:0/1845961991 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe401e040 con 0x5555e60751c0 2024-05-10T11:28:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.715+0000 7fdfd2ffd640 1 -- 172.21.15.100:0/1845961991 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdfe401f040 con 0x5555e60751c0 2024-05-10T11:28:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.715+0000 7fdfd2ffd640 1 -- 172.21.15.100:0/1845961991 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe400ea50 con 0x5555e60751c0 2024-05-10T11:28:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.716+0000 7fdff25de740 1 -- 172.21.15.100:0/1845961991 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5555e60751c0 msgr2=0x5555e607ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.716+0000 7fdff25de740 1 --2- 172.21.15.100:0/1845961991 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5555e60751c0 0x5555e607ea50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4009c40 tx=0x7fdfe4009510 comp rx=0 tx=0).stop 2024-05-10T11:28:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.716+0000 7fdff25de740 1 -- 172.21.15.100:0/1845961991 shutdown_connections 2024-05-10T11:28:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.716+0000 7fdff25de740 1 --2- 172.21.15.100:0/1845961991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5555e607ad20 0x5555e607f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.716+0000 7fdff25de740 1 --2- 172.21.15.100:0/1845961991 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5555e6075bb0 0x5555e607f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.716+0000 7fdff25de740 1 --2- 172.21.15.100:0/1845961991 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5555e60751c0 0x5555e607ea50 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.716+0000 7fdff25de740 1 -- 172.21.15.100:0/1845961991 >> 172.21.15.100:0/1845961991 conn(0x5555e606c420 msgr2=0x5555e6067400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.716+0000 7fdff25de740 1 -- 172.21.15.100:0/1845961991 shutdown_connections 2024-05-10T11:28:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.716+0000 7fdff25de740 1 -- 172.21.15.100:0/1845961991 wait complete. 2024-05-10T11:28:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.717+0000 7fdff25de740 1 -- start start 2024-05-10T11:28:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.717+0000 7fdff25de740 1 Processor -- start 2024-05-10T11:28:53.722 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.717+0000 7fdff25de740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5555e60751c0 0x5555e6082b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:53.723 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.717+0000 7fdff25de740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5555e6075bb0 0x5555e60830a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:53.723 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.717+0000 7fdff25de740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5555e607ad20 0x5555e60f3980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:53.723 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.717+0000 7fdff25de740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe81a76d0 con 0x5555e60751c0 2024-05-10T11:28:53.723 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.717+0000 7fdff25de740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe81a8b40 con 0x5555e6075bb0 2024-05-10T11:28:53.723 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.717+0000 7fdff25de740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe81a7da0 con 0x5555e607ad20 2024-05-10T11:28:53.723 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.717+0000 7fdff25de740 10 client.0 _ll_register_callbacks cb 0x7fdfd8009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:28:53.723 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.717+0000 7fdfed664640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5555e60751c0 0x5555e6082b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:53.723 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.717+0000 7fdfede65640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5555e607ad20 0x5555e60f3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:53.723 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.717+0000 7fdfed664640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5555e60751c0 0x5555e6082b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58086/0 (socket says 172.21.15.100:58086) 2024-05-10T11:28:53.723 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.717+0000 7fdfed664640 1 -- 172.21.15.100:0/2115840924 learned_addr learned my addr 172.21.15.100:0/2115840924 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:28:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.717+0000 7fdfede65640 1 --2- 172.21.15.100:0/2115840924 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5555e607ad20 0x5555e60f3980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:59956/0 (socket says 172.21.15.100:59956) 2024-05-10T11:28:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.718+0000 7fdfece63640 1 --2- 172.21.15.100:0/2115840924 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5555e6075bb0 0x5555e60830a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.718+0000 7fdfede65640 1 -- 172.21.15.100:0/2115840924 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5555e6075bb0 msgr2=0x5555e60830a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.718+0000 7fdfede65640 1 --2- 172.21.15.100:0/2115840924 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5555e6075bb0 0x5555e60830a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.718+0000 7fdfede65640 1 -- 172.21.15.100:0/2115840924 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5555e60751c0 msgr2=0x5555e6082b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.718+0000 7fdfede65640 1 --2- 172.21.15.100:0/2115840924 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5555e60751c0 0x5555e6082b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.718+0000 7fdfede65640 1 -- 172.21.15.100:0/2115840924 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555e6086120 con 0x5555e607ad20 2024-05-10T11:28:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.718+0000 7fdfece63640 1 --2- 172.21.15.100:0/2115840924 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5555e6075bb0 0x5555e60830a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:28:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.718+0000 7fdfed664640 1 --2- 172.21.15.100:0/2115840924 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5555e60751c0 0x5555e6082b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:28:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.718+0000 7fdfede65640 1 --2- 172.21.15.100:0/2115840924 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5555e607ad20 0x5555e60f3980 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0006010 tx=0x7fdfe002a8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bada3b03af6bbb1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.718+0000 7fdfd0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:28:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.718+0000 7fdfd0ff9640 1 -- 172.21.15.100:0/2115840924 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe0059060 con 0x5555e607ad20 2024-05-10T11:28:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.718+0000 7fdfd0ff9640 1 -- 172.21.15.100:0/2115840924 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fdfe00305c0 con 0x5555e607ad20 2024-05-10T11:28:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.718+0000 7fdff25de740 1 -- 172.21.15.100:0/2115840924 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfe401f040 con 0x5555e607ad20 2024-05-10T11:28:53.725 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.718+0000 7fdfd0ff9640 1 -- 172.21.15.100:0/2115840924 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fdfe00307c0 con 0x5555e607ad20 2024-05-10T11:28:53.725 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.718+0000 7fdff25de740 10 client.26950 fetch_fsmap learned FSMap version 31 2024-05-10T11:28:53.725 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.718+0000 7fdff25de740 1 -- 172.21.15.100:0/2115840924 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5555e60f4df0 con 0x5555e607ad20 2024-05-10T11:28:53.725 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.719+0000 7fdfd0ff9640 1 -- 172.21.15.100:0/2115840924 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdfe401f040 con 0x5555e607ad20 2024-05-10T11:28:53.725 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.719+0000 7fdfd0ff9640 1 -- 172.21.15.100:0/2115840924 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fdfe0053030 con 0x5555e607ad20 2024-05-10T11:28:53.725 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.719+0000 7fdff25de740 10 client.26950 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:28:53.725 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.719+0000 7fdff25de740 10 client.26950 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:28:53.725 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.719+0000 7fdff25de740 20 client.26950 populate_metadata read hostname 'smithi100' 2024-05-10T11:28:53.725 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.719+0000 7fdff25de740 1 --2- 172.21.15.100:0/2115840924 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5555e60f57f0 0x5555e60f7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:53.725 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.719+0000 7fdff25de740 4 client.26950 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:28:53.725 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.719+0000 7fdff25de740 1 -- 172.21.15.100:0/2115840924 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5555e5fdd050 con 0x5555e60f57f0 2024-05-10T11:28:53.725 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.719+0000 7fdfed664640 1 --2- 172.21.15.100:0/2115840924 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5555e60f57f0 0x5555e60f7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:53.725 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.720+0000 7fdfd0ff9640 10 client.26950 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:28:53.725 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.720+0000 7fdfed664640 1 --2- 172.21.15.100:0/2115840924 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5555e60f57f0 0x5555e60f7c10 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62e883bb92a0b466 server_cookie=d1ac6b8cd6e3a552 in_seq=0 out_seq=0 2024-05-10T11:28:53.751 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.750+0000 7fdfd0ff9640 1 -- 172.21.15.100:0/2115840924 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5555e5fdd050 con 0x5555e60f57f0 2024-05-10T11:28:53.751 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.750+0000 7fdfd0ff9640 10 client.26950 handle_command_reply: tid=0 2024-05-10T11:28:53.751 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.750+0000 7fdff25de740 1 client.26950 shutdown 2024-05-10T11:28:53.751 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.750+0000 7fdff25de740 1 -- 172.21.15.100:0/2115840924 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5555e607ad20 msgr2=0x5555e60f3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:53.752 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.750+0000 7fdff25de740 1 --2- 172.21.15.100:0/2115840924 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5555e607ad20 0x5555e60f3980 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0006010 tx=0x7fdfe002a8c0 comp rx=0 tx=0).stop 2024-05-10T11:28:53.752 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.750+0000 7fdff25de740 1 -- 172.21.15.100:0/2115840924 shutdown_connections 2024-05-10T11:28:53.752 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.750+0000 7fdff25de740 1 --2- 172.21.15.100:0/2115840924 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5555e60f57f0 0x5555e60f7c10 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:53.752 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.750+0000 7fdff25de740 1 --2- 172.21.15.100:0/2115840924 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5555e607ad20 0x5555e60f3980 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:53.752 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.750+0000 7fdff25de740 1 --2- 172.21.15.100:0/2115840924 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5555e6075bb0 0x5555e60830a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:53.752 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.750+0000 7fdff25de740 1 --2- 172.21.15.100:0/2115840924 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5555e60751c0 0x5555e6082b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:53.752 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.750+0000 7fdfd0ff9640 0 client.26950 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:28:53.752 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.750+0000 7fdff25de740 1 -- 172.21.15.100:0/2115840924 >> 172.21.15.100:0/2115840924 conn(0x5555e606c420 msgr2=0x5555e607e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:53.752 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.750+0000 7fdff25de740 1 -- 172.21.15.100:0/2115840924 shutdown_connections 2024-05-10T11:28:53.753 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.750+0000 7fdff25de740 1 -- 172.21.15.100:0/2115840924 wait complete. 2024-05-10T11:28:53.753 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.750+0000 7fdff25de740 20 client.26950 trim_cache size 0 max 16384 2024-05-10T11:28:53.753 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:28:53.753 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:28:53.753 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:28:53.753 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:28:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.751+0000 7fdfef8c8640 1 -- 172.21.15.100:0/1522205645 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fdfd40b28a0 msgr2=0x7fdfd40b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.751+0000 7fdfef8c8640 1 --2- 172.21.15.100:0/1522205645 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fdfd40b28a0 0x7fdfd40b4d60 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0004af0 tx=0x7fdfe00095c0 comp rx=0 tx=0).stop 2024-05-10T11:28:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.751+0000 7fdfef8c8640 1 -- 172.21.15.100:0/1522205645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfe8105de0 msgr2=0x7fdfe8198fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.751+0000 7fdfef8c8640 1 --2- 172.21.15.100:0/1522205645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfe8105de0 0x7fdfe8198fb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdfd800e9c0 tx=0x7fdfd800ee80 comp rx=0 tx=0).stop 2024-05-10T11:28:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.751+0000 7fdfef8c8640 1 -- 172.21.15.100:0/1522205645 shutdown_connections 2024-05-10T11:28:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.751+0000 7fdfef8c8640 1 --2- 172.21.15.100:0/1522205645 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fdfd40b28a0 0x7fdfd40b4d60 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.751+0000 7fdfef8c8640 1 --2- 172.21.15.100:0/1522205645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfe8108710 0x7fdfe81a0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.751+0000 7fdfef8c8640 1 --2- 172.21.15.100:0/1522205645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfe8105de0 0x7fdfe8198fb0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.751+0000 7fdfef8c8640 1 --2- 172.21.15.100:0/1522205645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdfe81034b0 0x7fdfe8198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.751+0000 7fdfef8c8640 1 -- 172.21.15.100:0/1522205645 >> 172.21.15.100:0/1522205645 conn(0x7fdfe80fcec0 msgr2=0x7fdfe8103db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.751+0000 7fdfef8c8640 1 -- 172.21.15.100:0/1522205645 shutdown_connections 2024-05-10T11:28:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:28:53.751+0000 7fdfef8c8640 1 -- 172.21.15.100:0/1522205645 wait complete. 2024-05-10T11:28:53.755 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:28:53 smithi178 ceph-mon[33359]: pgmap v735: 129 pgs: 129 active+clean; 107 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 1.3 MiB/s wr, 143 op/s 2024-05-10T11:28:53.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:28:53 smithi188 ceph-mon[34075]: pgmap v735: 129 pgs: 129 active+clean; 107 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 1.3 MiB/s wr, 143 op/s 2024-05-10T11:28:53.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:28:53 smithi100 ceph-mon[30296]: pgmap v735: 129 pgs: 129 active+clean; 107 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 1.3 MiB/s wr, 143 op/s 2024-05-10T11:28:53.789 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:28:53.980 INFO:tasks.workunit.client.0.smithi100.stdout: 1 270758 2.73 MB/sec execute 408 sec latency 16.763 ms 2024-05-10T11:28:54.980 INFO:tasks.workunit.client.0.smithi100.stdout: 1 271297 2.73 MB/sec execute 409 sec latency 19.441 ms 2024-05-10T11:28:55.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:55.531+0000 7fe5664f3700 1 -- 172.21.15.100:0/2317368915 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe560102450 msgr2=0x7fe560110340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:55.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:55.531+0000 7fe5664f3700 1 --2- 172.21.15.100:0/2317368915 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe560102450 0x7fe560110340 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fe55800b0a0 tx=0x7fe55800b3b0 comp rx=0 tx=0).stop 2024-05-10T11:28:55.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:55.531+0000 7fe5664f3700 1 -- 172.21.15.100:0/2317368915 shutdown_connections 2024-05-10T11:28:55.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:55.531+0000 7fe5664f3700 1 --2- 172.21.15.100:0/2317368915 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe560102450 0x7fe560110340 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:55.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:55.531+0000 7fe5664f3700 1 --2- 172.21.15.100:0/2317368915 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe560109520 0x7fe560109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:55.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:55.531+0000 7fe5664f3700 1 --2- 172.21.15.100:0/2317368915 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe560101a90 0x7fe560101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:55.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:55.531+0000 7fe5664f3700 1 -- 172.21.15.100:0/2317368915 >> 172.21.15.100:0/2317368915 conn(0x7fe560078a10 msgr2=0x7fe5600ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:55.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:55.531+0000 7fe5664f3700 1 -- 172.21.15.100:0/2317368915 shutdown_connections 2024-05-10T11:28:55.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:55.531+0000 7fe5664f3700 1 -- 172.21.15.100:0/2317368915 wait complete. 2024-05-10T11:28:55.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:55.531+0000 7fe5664f3700 1 Processor -- start 2024-05-10T11:28:55.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:55.531+0000 7fe5664f3700 1 -- start start 2024-05-10T11:28:55.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:55.532+0000 7fe5664f3700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe560101a90 0x7fe560102c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:55.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:55.532+0000 7fe5664f3700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe560102450 0x7fe5601031b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:55.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:55.532+0000 7fe5664f3700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe560109520 0x7fe5601036f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:55.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:55.532+0000 7fe5664f3700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe560113500 con 0x7fe560101a90 2024-05-10T11:28:55.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:55.532+0000 7fe5664f3700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe560113380 con 0x7fe560109520 2024-05-10T11:28:55.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:55.532+0000 7fe5664f3700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe560113680 con 0x7fe560102450 2024-05-10T11:28:55.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:55.532+0000 7fe55ffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe560101a90 0x7fe560102c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:55.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:55.532+0000 7fe55ffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe560101a90 0x7fe560102c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58096/0 (socket says 172.21.15.100:58096) 2024-05-10T11:28:55.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:55.532+0000 7fe55ffff700 1 -- 172.21.15.100:0/326672907 learned_addr learned my addr 172.21.15.100:0/326672907 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:28:55.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:55.532+0000 7fe55f7fe700 1 --2- 172.21.15.100:0/326672907 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe560102450 0x7fe5601031b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:55.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:55.532+0000 7fe564a90700 1 --2- 172.21.15.100:0/326672907 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe560109520 0x7fe5601036f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:55.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:55.532+0000 7fe55f7fe700 1 -- 172.21.15.100:0/326672907 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe560109520 msgr2=0x7fe5601036f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:55.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:55.532+0000 7fe55f7fe700 1 --2- 172.21.15.100:0/326672907 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe560109520 0x7fe5601036f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:55.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:55.532+0000 7fe55f7fe700 1 -- 172.21.15.100:0/326672907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe560101a90 msgr2=0x7fe560102c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:55.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:55.532+0000 7fe55f7fe700 1 --2- 172.21.15.100:0/326672907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe560101a90 0x7fe560102c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:55.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:55.532+0000 7fe55f7fe700 1 -- 172.21.15.100:0/326672907 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5601a7f40 con 0x7fe560102450 2024-05-10T11:28:55.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:55.532+0000 7fe55ffff700 1 --2- 172.21.15.100:0/326672907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe560101a90 0x7fe560102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:28:55.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:55.533+0000 7fe55f7fe700 1 --2- 172.21.15.100:0/326672907 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe560102450 0x7fe5601031b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe55400e9f0 tx=0x7fe55400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:55.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:55.533+0000 7fe55d7fa700 1 -- 172.21.15.100:0/326672907 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe554004220 con 0x7fe560102450 2024-05-10T11:28:55.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:55.533+0000 7fe55d7fa700 1 -- 172.21.15.100:0/326672907 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe5540043c0 con 0x7fe560102450 2024-05-10T11:28:55.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:55.533+0000 7fe5664f3700 1 -- 172.21.15.100:0/326672907 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5601a81d0 con 0x7fe560102450 2024-05-10T11:28:55.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:55.533+0000 7fe55d7fa700 1 -- 172.21.15.100:0/326672907 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe55400ce30 con 0x7fe560102450 2024-05-10T11:28:55.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:55.533+0000 7fe5664f3700 1 -- 172.21.15.100:0/326672907 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5601a86c0 con 0x7fe560102450 2024-05-10T11:28:55.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:55.534+0000 7fe5664f3700 1 -- 172.21.15.100:0/326672907 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe56006a190 con 0x7fe560102450 2024-05-10T11:28:55.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:55.535+0000 7fe55d7fa700 1 -- 172.21.15.100:0/326672907 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fe554004750 con 0x7fe560102450 2024-05-10T11:28:55.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:55.536+0000 7fe55d7fa700 1 --2- 172.21.15.100:0/326672907 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe5480b2880 0x7fe5480b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:55.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:55.536+0000 7fe55ffff700 1 --2- 172.21.15.100:0/326672907 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe5480b2880 0x7fe5480b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:55.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:55.536+0000 7fe55d7fa700 1 -- 172.21.15.100:0/326672907 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fe5540e2670 con 0x7fe560102450 2024-05-10T11:28:55.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:55.536+0000 7fe55ffff700 1 --2- 172.21.15.100:0/326672907 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe5480b2880 0x7fe5480b4d40 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe550011550 tx=0x7fe550009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:55.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:55.539+0000 7fe55d7fa700 1 -- 172.21.15.100:0/326672907 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fe5540aa060 con 0x7fe560102450 2024-05-10T11:28:55.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:28:55 smithi178 ceph-mon[33359]: pgmap v736: 129 pgs: 129 active+clean; 110 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 1.4 MiB/s wr, 160 op/s 2024-05-10T11:28:55.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:28:55 smithi188 ceph-mon[34075]: pgmap v736: 129 pgs: 129 active+clean; 110 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 1.4 MiB/s wr, 160 op/s 2024-05-10T11:28:55.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:28:55 smithi100 ceph-mon[30296]: pgmap v736: 129 pgs: 129 active+clean; 110 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 1.4 MiB/s wr, 160 op/s 2024-05-10T11:28:55.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:55.843+0000 7fe5664f3700 1 -- 172.21.15.100:0/326672907 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe56002e510 con 0x7fe560102450 2024-05-10T11:28:55.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:55.843+0000 7fe55d7fa700 1 -- 172.21.15.100:0/326672907 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11894 (secure 0 0 0) 0x7fe5540a9960 con 0x7fe560102450 2024-05-10T11:28:55.845 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:28:55.845 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2072225012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2072225012},{"type":"v1","addr":"172.21.15.100:6835","nonce":2072225012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/401939540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":401939540},{"type":"v1","addr":"172.21.15.100:6837","nonce":401939540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14571,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/3946311025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3946311025},{"type":"v1","addr":"172.21.15.100:6839","nonce":3946311025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3736371660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3736371660},{"type":"v1","addr":"172.21.15.100:6841","nonce":3736371660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/2741810791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2741810791},{"type":"v1","addr":"172.21.15.178:6833","nonce":2741810791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1704891041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1704891041},{"type":"v1","addr":"172.21.15.188:6833","nonce":1704891041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T11:14:27.242452+0000","modified":"2024-05-10T11:23:19.557113+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24439,"mds_1":24437,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/2124076312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2124076312},{"type":"v1","addr":"172.21.15.178:6835","nonce":2124076312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/3224358425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3224358425},{"type":"v1","addr":"172.21.15.178:6837","nonce":3224358425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/515547690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":515547690},{"type":"v1","addr":"172.21.15.188:6835","nonce":515547690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/1943342620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1943342620},{"type":"v1","addr":"172.21.15.188:6837","nonce":1943342620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.178:6839/3760985404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3760985404},{"type":"v1","addr":"172.21.15.178:6839","nonce":3760985404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.188:6839/3194917070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":3194917070},{"type":"v1","addr":"172.21.15.188:6839","nonce":3194917070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24439,"qdb_cluster":[24439,24427,24437]},"id":1}]} 2024-05-10T11:28:55.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:55.846+0000 7fe5664f3700 1 -- 172.21.15.100:0/326672907 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe5480b2880 msgr2=0x7fe5480b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:55.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:55.846+0000 7fe5664f3700 1 --2- 172.21.15.100:0/326672907 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe5480b2880 0x7fe5480b4d40 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe550011550 tx=0x7fe550009260 comp rx=0 tx=0).stop 2024-05-10T11:28:55.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:55.846+0000 7fe5664f3700 1 -- 172.21.15.100:0/326672907 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe560102450 msgr2=0x7fe5601031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:55.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:55.846+0000 7fe5664f3700 1 --2- 172.21.15.100:0/326672907 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe560102450 0x7fe5601031b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe55400e9f0 tx=0x7fe55400ed00 comp rx=0 tx=0).stop 2024-05-10T11:28:55.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:55.846+0000 7fe5664f3700 1 -- 172.21.15.100:0/326672907 shutdown_connections 2024-05-10T11:28:55.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:55.846+0000 7fe5664f3700 1 --2- 172.21.15.100:0/326672907 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe5480b2880 0x7fe5480b4d40 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:55.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:55.846+0000 7fe5664f3700 1 --2- 172.21.15.100:0/326672907 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe560109520 0x7fe5601036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:55.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:55.846+0000 7fe5664f3700 1 --2- 172.21.15.100:0/326672907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe560101a90 0x7fe560102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:55.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:55.846+0000 7fe5664f3700 1 --2- 172.21.15.100:0/326672907 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe560102450 0x7fe5601031b0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:55.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:55.846+0000 7fe5664f3700 1 -- 172.21.15.100:0/326672907 >> 172.21.15.100:0/326672907 conn(0x7fe560078a10 msgr2=0x7fe560100240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:55.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:55.846+0000 7fe5664f3700 1 -- 172.21.15.100:0/326672907 shutdown_connections 2024-05-10T11:28:55.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:55.846+0000 7fe5664f3700 1 -- 172.21.15.100:0/326672907 wait complete. 2024-05-10T11:28:55.848 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 31 2024-05-10T11:28:55.980 INFO:tasks.workunit.client.0.smithi100.stdout: 1 271939 2.72 MB/sec execute 410 sec latency 16.367 ms 2024-05-10T11:28:56.572 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-10T11:28:56.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:28:56 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/326672907' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:28:56.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:28:56 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/326672907' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:28:56.838 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:28:56.980 INFO:tasks.workunit.client.0.smithi100.stdout: 1 272497 2.73 MB/sec execute 411 sec latency 15.481 ms 2024-05-10T11:28:57.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:28:56 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/326672907' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:28:57.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:28:57 smithi188 ceph-mon[34075]: pgmap v737: 129 pgs: 129 active+clean; 110 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.3 MiB/s wr, 135 op/s 2024-05-10T11:28:57.789 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:28:57 smithi100 ceph-mon[30296]: pgmap v737: 129 pgs: 129 active+clean; 110 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.3 MiB/s wr, 135 op/s 2024-05-10T11:28:57.980 INFO:tasks.workunit.client.0.smithi100.stdout: 1 272959 2.73 MB/sec execute 412 sec latency 15.873 ms 2024-05-10T11:28:58.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:28:57 smithi178 ceph-mon[33359]: pgmap v737: 129 pgs: 129 active+clean; 110 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.3 MiB/s wr, 135 op/s 2024-05-10T11:28:58.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.561+0000 7f95eef40700 1 -- 172.21.15.100:0/4094938300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95e8102d60 msgr2=0x7f95e810f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:58.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.561+0000 7f95eef40700 1 --2- 172.21.15.100:0/4094938300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95e8102d60 0x7f95e810f250 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f95e000b0a0 tx=0x7f95e000b3b0 comp rx=0 tx=0).stop 2024-05-10T11:28:58.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.561+0000 7f95eef40700 1 -- 172.21.15.100:0/4094938300 shutdown_connections 2024-05-10T11:28:58.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.561+0000 7f95eef40700 1 --2- 172.21.15.100:0/4094938300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95e8102d60 0x7f95e810f250 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:58.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.561+0000 7f95eef40700 1 --2- 172.21.15.100:0/4094938300 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95e8108410 0x7f95e81087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:58.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.561+0000 7f95eef40700 1 --2- 172.21.15.100:0/4094938300 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95e81023a0 0x7f95e8102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:58.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.561+0000 7f95eef40700 1 -- 172.21.15.100:0/4094938300 >> 172.21.15.100:0/4094938300 conn(0x7f95e8079350 msgr2=0x7f95e8079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:58.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.561+0000 7f95eef40700 1 -- 172.21.15.100:0/4094938300 shutdown_connections 2024-05-10T11:28:58.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.561+0000 7f95eef40700 1 -- 172.21.15.100:0/4094938300 wait complete. 2024-05-10T11:28:58.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.562+0000 7f95eef40700 1 Processor -- start 2024-05-10T11:28:58.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.562+0000 7f95eef40700 1 -- start start 2024-05-10T11:28:58.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.562+0000 7f95eef40700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95e81023a0 0x7f95e819a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:58.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.562+0000 7f95eef40700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95e8102d60 0x7f95e819aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:58.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.563+0000 7f95eef40700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95e8108410 0x7f95e81a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:58.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.563+0000 7f95eef40700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f95e8112430 con 0x7f95e8102d60 2024-05-10T11:28:58.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.563+0000 7f95eef40700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f95e81122b0 con 0x7f95e81023a0 2024-05-10T11:28:58.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.563+0000 7f95eef40700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f95e81125b0 con 0x7f95e8108410 2024-05-10T11:28:58.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.563+0000 7f95e7fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95e8102d60 0x7f95e819aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:58.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.563+0000 7f95e7fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95e8102d60 0x7f95e819aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39896/0 (socket says 172.21.15.100:39896) 2024-05-10T11:28:58.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.563+0000 7f95e7fff700 1 -- 172.21.15.100:0/967648567 learned_addr learned my addr 172.21.15.100:0/967648567 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:28:58.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.563+0000 7f95ed4dd700 1 --2- 172.21.15.100:0/967648567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95e8108410 0x7f95e81a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:58.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.563+0000 7f95eccdc700 1 --2- 172.21.15.100:0/967648567 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95e81023a0 0x7f95e819a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:58.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.563+0000 7f95ed4dd700 1 -- 172.21.15.100:0/967648567 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95e81023a0 msgr2=0x7f95e819a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:58.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.563+0000 7f95ed4dd700 1 --2- 172.21.15.100:0/967648567 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95e81023a0 0x7f95e819a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:58.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.563+0000 7f95ed4dd700 1 -- 172.21.15.100:0/967648567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95e8102d60 msgr2=0x7f95e819aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:58.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.563+0000 7f95ed4dd700 1 --2- 172.21.15.100:0/967648567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95e8102d60 0x7f95e819aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:58.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.563+0000 7f95ed4dd700 1 -- 172.21.15.100:0/967648567 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95e81a5f50 con 0x7f95e8108410 2024-05-10T11:28:58.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.563+0000 7f95e7fff700 1 --2- 172.21.15.100:0/967648567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95e8102d60 0x7f95e819aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:28:58.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.563+0000 7f95ed4dd700 1 --2- 172.21.15.100:0/967648567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95e8108410 0x7f95e81a5a10 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f95e000b070 tx=0x7f95e0004c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:58.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.563+0000 7f95e5ffb700 1 -- 172.21.15.100:0/967648567 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e0021070 con 0x7f95e8108410 2024-05-10T11:28:58.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.563+0000 7f95e5ffb700 1 -- 172.21.15.100:0/967648567 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f95e0007b80 con 0x7f95e8108410 2024-05-10T11:28:58.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.563+0000 7f95eef40700 1 -- 172.21.15.100:0/967648567 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95e81a6120 con 0x7f95e8108410 2024-05-10T11:28:58.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.563+0000 7f95e5ffb700 1 -- 172.21.15.100:0/967648567 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e0005040 con 0x7f95e8108410 2024-05-10T11:28:58.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.564+0000 7f95eef40700 1 -- 172.21.15.100:0/967648567 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95e81a6570 con 0x7f95e8108410 2024-05-10T11:28:58.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.566+0000 7f95e5ffb700 1 -- 172.21.15.100:0/967648567 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f95e0024070 con 0x7f95e8108410 2024-05-10T11:28:58.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.567+0000 7f95e5ffb700 1 --2- 172.21.15.100:0/967648567 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f95d80b2880 0x7f95d80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:58.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.567+0000 7f95eccdc700 1 --2- 172.21.15.100:0/967648567 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f95d80b2880 0x7f95d80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:58.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.567+0000 7f95e5ffb700 1 -- 172.21.15.100:0/967648567 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f95e000f020 con 0x7f95e8108410 2024-05-10T11:28:58.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.568+0000 7f95f97d3b80 1 Processor -- start 2024-05-10T11:28:58.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.568+0000 7f95f97d3b80 1 -- start start 2024-05-10T11:28:58.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.568+0000 7f95eccdc700 1 --2- 172.21.15.100:0/967648567 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f95d80b2880 0x7f95d80b4d40 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f95dc009be0 tx=0x7f95dc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:58.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.568+0000 7f95f97d3b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562707959f80 0x56270795a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:58.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.568+0000 7f95f97d3b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56270795a8a0 0x562707954700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:58.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.568+0000 7f95f97d3b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562707954c90 0x562707955110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:58.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.568+0000 7f95f97d3b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5627078a2860 con 0x56270795a8a0 2024-05-10T11:28:58.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.568+0000 7f95f97d3b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5627078417c0 con 0x562707954c90 2024-05-10T11:28:58.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.568+0000 7f95f97d3b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56270789e820 con 0x562707959f80 2024-05-10T11:28:58.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.568+0000 7f95e7fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56270795a8a0 0x562707954700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:58.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.568+0000 7f95ed4dd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562707954c90 0x562707955110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:58.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.568+0000 7f95e7fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56270795a8a0 0x562707954700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39906/0 (socket says 172.21.15.100:39906) 2024-05-10T11:28:58.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.568+0000 7f95eccdc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562707959f80 0x56270795a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:58.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.568+0000 7f95e7fff700 1 -- 172.21.15.100:0/3187085149 learned_addr learned my addr 172.21.15.100:0/3187085149 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:28:58.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.568+0000 7f95ed4dd700 1 -- 172.21.15.100:0/3187085149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562707959f80 msgr2=0x56270795a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:58.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.569+0000 7f95ed4dd700 1 --2- 172.21.15.100:0/3187085149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562707959f80 0x56270795a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:58.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.569+0000 7f95ed4dd700 1 -- 172.21.15.100:0/3187085149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56270795a8a0 msgr2=0x562707954700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:58.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.569+0000 7f95ed4dd700 1 --2- 172.21.15.100:0/3187085149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56270795a8a0 0x562707954700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:58.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.569+0000 7f95ed4dd700 1 -- 172.21.15.100:0/3187085149 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627077ef8d0 con 0x562707954c90 2024-05-10T11:28:58.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.569+0000 7f95eccdc700 1 --2- 172.21.15.100:0/3187085149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562707959f80 0x56270795a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:28:58.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.569+0000 7f95ed4dd700 1 --2- 172.21.15.100:0/3187085149 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562707954c90 0x562707955110 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f95e00e1080 tx=0x7f95e00095a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=141f8024efae4ffd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:58.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.569+0000 7f95d2ffd700 1 -- 172.21.15.100:0/3187085149 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e00b1040 con 0x562707954c90 2024-05-10T11:28:58.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.569+0000 7f95d2ffd700 1 -- 172.21.15.100:0/3187085149 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f95e00cbe50 con 0x562707954c90 2024-05-10T11:28:58.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.569+0000 7f95d2ffd700 1 -- 172.21.15.100:0/3187085149 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e00074e0 con 0x562707954c90 2024-05-10T11:28:58.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.570+0000 7f95f97d3b80 1 -- 172.21.15.100:0/3187085149 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562707954c90 msgr2=0x562707955110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:58.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.570+0000 7f95f97d3b80 1 --2- 172.21.15.100:0/3187085149 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562707954c90 0x562707955110 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f95e00e1080 tx=0x7f95e00095a0 comp rx=0 tx=0).stop 2024-05-10T11:28:58.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.570+0000 7f95f97d3b80 1 -- 172.21.15.100:0/3187085149 shutdown_connections 2024-05-10T11:28:58.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.570+0000 7f95f97d3b80 1 --2- 172.21.15.100:0/3187085149 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562707954c90 0x562707955110 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:58.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.570+0000 7f95f97d3b80 1 --2- 172.21.15.100:0/3187085149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562707959f80 0x56270795a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:58.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.570+0000 7f95f97d3b80 1 --2- 172.21.15.100:0/3187085149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56270795a8a0 0x562707954700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:58.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.570+0000 7f95f97d3b80 1 -- 172.21.15.100:0/3187085149 >> 172.21.15.100:0/3187085149 conn(0x56270787c100 msgr2=0x56270794ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:58.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.570+0000 7f95f97d3b80 1 -- 172.21.15.100:0/3187085149 shutdown_connections 2024-05-10T11:28:58.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.570+0000 7f95f97d3b80 1 -- 172.21.15.100:0/3187085149 wait complete. 2024-05-10T11:28:58.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.570+0000 7f95f97d3b80 1 -- start start 2024-05-10T11:28:58.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.570+0000 7f95f97d3b80 1 Processor -- start 2024-05-10T11:28:58.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.571+0000 7f95f97d3b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562707954c90 0x5627079604d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:58.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.571+0000 7f95f97d3b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562707959f80 0x56270795d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:58.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.571+0000 7f95f97d3b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56270795a8a0 0x56270795de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:58.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.571+0000 7f95f97d3b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x562707961080 con 0x562707954c90 2024-05-10T11:28:58.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.571+0000 7f95f97d3b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562707808c30 con 0x56270795a8a0 2024-05-10T11:28:58.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.571+0000 7f95f97d3b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x562707961200 con 0x562707959f80 2024-05-10T11:28:58.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.571+0000 7f95eccdc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562707954c90 0x5627079604d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:58.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.571+0000 7f95eccdc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562707954c90 0x5627079604d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39920/0 (socket says 172.21.15.100:39920) 2024-05-10T11:28:58.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.571+0000 7f95eccdc700 1 -- 172.21.15.100:0/1730628329 learned_addr learned my addr 172.21.15.100:0/1730628329 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:28:58.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.571+0000 7f95f97d3b80 10 client.0 _ll_register_callbacks cb 0x56270783a920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:28:58.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.571+0000 7f95ed4dd700 1 --2- 172.21.15.100:0/1730628329 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56270795a8a0 0x56270795de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:58.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.571+0000 7f95e7fff700 1 --2- 172.21.15.100:0/1730628329 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562707959f80 0x56270795d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:58.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.571+0000 7f95eccdc700 1 -- 172.21.15.100:0/1730628329 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562707959f80 msgr2=0x56270795d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:58.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.571+0000 7f95eccdc700 1 --2- 172.21.15.100:0/1730628329 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562707959f80 0x56270795d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:58.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.571+0000 7f95eccdc700 1 -- 172.21.15.100:0/1730628329 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56270795a8a0 msgr2=0x56270795de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:58.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.571+0000 7f95eccdc700 1 --2- 172.21.15.100:0/1730628329 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56270795a8a0 0x56270795de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:58.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.571+0000 7f95eccdc700 1 -- 172.21.15.100:0/1730628329 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56270783e690 con 0x562707954c90 2024-05-10T11:28:58.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.571+0000 7f95ed4dd700 1 --2- 172.21.15.100:0/1730628329 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56270795a8a0 0x56270795de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:28:58.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.572+0000 7f95eccdc700 1 --2- 172.21.15.100:0/1730628329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562707954c90 0x5627079604d0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f95dc01fbf0 tx=0x7f95dc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=933c09b348540af5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:58.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.572+0000 7f95c7fff700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:28:58.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.572+0000 7f95c7fff700 1 -- 172.21.15.100:0/1730628329 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95dc004030 con 0x562707954c90 2024-05-10T11:28:58.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.572+0000 7f95c7fff700 1 -- 172.21.15.100:0/1730628329 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f95dc009040 con 0x562707954c90 2024-05-10T11:28:58.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.572+0000 7f95f97d3b80 1 -- 172.21.15.100:0/1730628329 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95e00cbe50 con 0x562707954c90 2024-05-10T11:28:58.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.572+0000 7f95c7fff700 1 -- 172.21.15.100:0/1730628329 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f95dc0041d0 con 0x562707954c90 2024-05-10T11:28:58.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.572+0000 7f95f97d3b80 10 client.17517 fetch_fsmap learned FSMap version 31 2024-05-10T11:28:58.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.572+0000 7f95f97d3b80 1 -- 172.21.15.100:0/1730628329 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56270796b900 con 0x562707954c90 2024-05-10T11:28:58.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.572+0000 7f95c7fff700 1 -- 172.21.15.100:0/1730628329 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f95e00cbe50 con 0x562707954c90 2024-05-10T11:28:58.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.572+0000 7f95c7fff700 1 -- 172.21.15.100:0/1730628329 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f95dc003810 con 0x562707954c90 2024-05-10T11:28:58.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.573+0000 7f95f97d3b80 10 client.17517 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:28:58.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.573+0000 7f95f97d3b80 10 client.17517 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-10T11:28:58.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.573+0000 7f95f97d3b80 20 client.17517 populate_metadata read hostname 'smithi100' 2024-05-10T11:28:58.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.573+0000 7f95f97d3b80 1 --2- 172.21.15.100:0/1730628329 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x56270796df90 0x562707970380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:58.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.573+0000 7f95f97d3b80 4 client.17517 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:28:58.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.573+0000 7f95f97d3b80 1 -- 172.21.15.100:0/1730628329 --> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56270791c650 con 0x56270796df90 2024-05-10T11:28:58.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.573+0000 7f95e7fff700 1 --2- 172.21.15.100:0/1730628329 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x56270796df90 0x562707970380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:58.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.573+0000 7f95e7fff700 1 --2- 172.21.15.100:0/1730628329 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x56270796df90 0x562707970380 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d8d2d1f98e66cd84 server_cookie=6a1004b2db0aa9c8 in_seq=0 out_seq=0 2024-05-10T11:28:58.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.573+0000 7f95c7fff700 10 client.17517 ms_handle_connect on v2:172.21.15.188:6836/1943342620 2024-05-10T11:28:58.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.574+0000 7f95c7fff700 1 -- 172.21.15.100:0/1730628329 <== mds.2 v2:172.21.15.188:6836/1943342620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56270791c650 con 0x56270796df90 2024-05-10T11:28:58.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.574+0000 7f95c7fff700 10 client.17517 handle_command_reply: tid=0 2024-05-10T11:28:58.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.574+0000 7f95f97d3b80 1 client.17517 shutdown 2024-05-10T11:28:58.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.574+0000 7f95f97d3b80 1 -- 172.21.15.100:0/1730628329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562707954c90 msgr2=0x5627079604d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:58.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.574+0000 7f95f97d3b80 1 --2- 172.21.15.100:0/1730628329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562707954c90 0x5627079604d0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f95dc01fbf0 tx=0x7f95dc0047c0 comp rx=0 tx=0).stop 2024-05-10T11:28:58.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.574+0000 7f95f97d3b80 1 -- 172.21.15.100:0/1730628329 shutdown_connections 2024-05-10T11:28:58.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.574+0000 7f95f97d3b80 1 --2- 172.21.15.100:0/1730628329 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x56270796df90 0x562707970380 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:58.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.574+0000 7f95f97d3b80 1 --2- 172.21.15.100:0/1730628329 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56270795a8a0 0x56270795de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:58.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.574+0000 7f95c7fff700 0 client.17517 ms_handle_reset on v2:172.21.15.188:6836/1943342620 2024-05-10T11:28:58.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.574+0000 7f95f97d3b80 1 --2- 172.21.15.100:0/1730628329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562707954c90 0x5627079604d0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:58.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.574+0000 7f95f97d3b80 1 --2- 172.21.15.100:0/1730628329 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562707959f80 0x56270795d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:58.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.574+0000 7f95f97d3b80 1 -- 172.21.15.100:0/1730628329 >> 172.21.15.100:0/1730628329 conn(0x56270787c100 msgr2=0x562707945720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:58.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.574+0000 7f95f97d3b80 1 -- 172.21.15.100:0/1730628329 shutdown_connections 2024-05-10T11:28:58.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.574+0000 7f95f97d3b80 1 -- 172.21.15.100:0/1730628329 wait complete. 2024-05-10T11:28:58.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.574+0000 7f95f97d3b80 20 client.17517 trim_cache size 0 max 16384 2024-05-10T11:28:58.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.593+0000 7f95f97d3b80 1 Processor -- start 2024-05-10T11:28:58.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.593+0000 7f95f97d3b80 1 -- start start 2024-05-10T11:28:58.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.593+0000 7f95f97d3b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562707954c90 0x56270795a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:58.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.593+0000 7f95f97d3b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562707959f80 0x562707958050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:58.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.593+0000 7f95f97d3b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56270795a8a0 0x56270796a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:58.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.593+0000 7f95f97d3b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f95e81af600 con 0x562707954c90 2024-05-10T11:28:58.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.593+0000 7f95f97d3b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f95e81ace90 con 0x562707959f80 2024-05-10T11:28:58.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.593+0000 7f95f97d3b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f95e81aace0 con 0x56270795a8a0 2024-05-10T11:28:58.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.593+0000 7f95eccdc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562707954c90 0x56270795a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:58.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.593+0000 7f95eccdc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562707954c90 0x56270795a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39930/0 (socket says 172.21.15.100:39930) 2024-05-10T11:28:58.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.593+0000 7f95ed4dd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56270795a8a0 0x56270796a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:58.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.593+0000 7f95eccdc700 1 -- 172.21.15.100:0/764225778 learned_addr learned my addr 172.21.15.100:0/764225778 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:28:58.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.593+0000 7f95e7fff700 1 --2- 172.21.15.100:0/764225778 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562707959f80 0x562707958050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:58.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.593+0000 7f95eccdc700 1 -- 172.21.15.100:0/764225778 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56270795a8a0 msgr2=0x56270796a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:58.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.593+0000 7f95eccdc700 1 --2- 172.21.15.100:0/764225778 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56270795a8a0 0x56270796a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:58.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.593+0000 7f95eccdc700 1 -- 172.21.15.100:0/764225778 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562707959f80 msgr2=0x562707958050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:58.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.593+0000 7f95eccdc700 1 --2- 172.21.15.100:0/764225778 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562707959f80 0x562707958050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:58.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.593+0000 7f95eccdc700 1 -- 172.21.15.100:0/764225778 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562707947810 con 0x562707954c90 2024-05-10T11:28:58.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.593+0000 7f95e7fff700 1 --2- 172.21.15.100:0/764225778 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562707959f80 0x562707958050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:28:58.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.594+0000 7f95eccdc700 1 --2- 172.21.15.100:0/764225778 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562707954c90 0x56270795a360 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f95dc016040 tx=0x7f95dc003ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fde014830f184 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:58.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.594+0000 7f95d27fc700 1 -- 172.21.15.100:0/764225778 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95dc0090c0 con 0x562707954c90 2024-05-10T11:28:58.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.594+0000 7f95d27fc700 1 -- 172.21.15.100:0/764225778 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f95dc03a070 con 0x562707954c90 2024-05-10T11:28:58.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.594+0000 7f95d27fc700 1 -- 172.21.15.100:0/764225778 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95dc01dd40 con 0x562707954c90 2024-05-10T11:28:58.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.594+0000 7f95f97d3b80 1 -- 172.21.15.100:0/764225778 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562707954c90 msgr2=0x56270795a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:58.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.594+0000 7f95f97d3b80 1 --2- 172.21.15.100:0/764225778 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562707954c90 0x56270795a360 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f95dc016040 tx=0x7f95dc003ee0 comp rx=0 tx=0).stop 2024-05-10T11:28:58.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.595+0000 7f95f97d3b80 1 -- 172.21.15.100:0/764225778 shutdown_connections 2024-05-10T11:28:58.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.595+0000 7f95f97d3b80 1 --2- 172.21.15.100:0/764225778 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56270795a8a0 0x56270796a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:58.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.595+0000 7f95f97d3b80 1 --2- 172.21.15.100:0/764225778 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562707954c90 0x56270795a360 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:58.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.595+0000 7f95f97d3b80 1 --2- 172.21.15.100:0/764225778 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562707959f80 0x562707958050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:58.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.595+0000 7f95f97d3b80 1 -- 172.21.15.100:0/764225778 >> 172.21.15.100:0/764225778 conn(0x56270787c100 msgr2=0x56270795acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:58.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.595+0000 7f95f97d3b80 1 -- 172.21.15.100:0/764225778 shutdown_connections 2024-05-10T11:28:58.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.595+0000 7f95f97d3b80 1 -- 172.21.15.100:0/764225778 wait complete. 2024-05-10T11:28:58.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.595+0000 7f95f97d3b80 1 -- start start 2024-05-10T11:28:58.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.595+0000 7f95f97d3b80 1 Processor -- start 2024-05-10T11:28:58.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.595+0000 7f95f97d3b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562707954c90 0x562707966630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:58.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.595+0000 7f95f97d3b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562707959f80 0x562707966b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:58.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.596+0000 7f95f97d3b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56270795a8a0 0x562707974e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:58.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.596+0000 7f95f97d3b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f95e81ab350 con 0x562707954c90 2024-05-10T11:28:58.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.596+0000 7f95f97d3b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f95e81ac7c0 con 0x56270795a8a0 2024-05-10T11:28:58.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.596+0000 7f95f97d3b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f95e81aba20 con 0x562707959f80 2024-05-10T11:28:58.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.596+0000 7f95f97d3b80 10 client.0 _ll_register_callbacks cb 0x56270783a920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:28:58.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.596+0000 7f95eccdc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562707954c90 0x562707966630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:58.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.596+0000 7f95eccdc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562707954c90 0x562707966630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39932/0 (socket says 172.21.15.100:39932) 2024-05-10T11:28:58.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.596+0000 7f95eccdc700 1 -- 172.21.15.100:0/3563505835 learned_addr learned my addr 172.21.15.100:0/3563505835 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:28:58.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.596+0000 7f95ed4dd700 1 --2- 172.21.15.100:0/3563505835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56270795a8a0 0x562707974e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:58.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.596+0000 7f95e7fff700 1 --2- 172.21.15.100:0/3563505835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562707959f80 0x562707966b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:58.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.596+0000 7f95eccdc700 1 -- 172.21.15.100:0/3563505835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562707959f80 msgr2=0x562707966b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:58.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.596+0000 7f95eccdc700 1 --2- 172.21.15.100:0/3563505835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562707959f80 0x562707966b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:58.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.596+0000 7f95eccdc700 1 -- 172.21.15.100:0/3563505835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56270795a8a0 msgr2=0x562707974e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:28:58.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.596+0000 7f95eccdc700 1 --2- 172.21.15.100:0/3563505835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56270795a8a0 0x562707974e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:58.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.596+0000 7f95eccdc700 1 -- 172.21.15.100:0/3563505835 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562707954950 con 0x562707954c90 2024-05-10T11:28:58.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.596+0000 7f95eccdc700 1 --2- 172.21.15.100:0/3563505835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562707954c90 0x562707966630 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f95dc011bf0 tx=0x7f95dc0038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=119b387004bff416 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:28:58.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.596+0000 7f95c7fff700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:28:58.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.596+0000 7f95c7fff700 1 -- 172.21.15.100:0/3563505835 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95dc0090c0 con 0x562707954c90 2024-05-10T11:28:58.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.596+0000 7f95c7fff700 1 -- 172.21.15.100:0/3563505835 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f95dc01dbd0 con 0x562707954c90 2024-05-10T11:28:58.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.596+0000 7f95f97d3b80 1 -- 172.21.15.100:0/3563505835 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95dc03a070 con 0x562707954c90 2024-05-10T11:28:58.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.597+0000 7f95c7fff700 1 -- 172.21.15.100:0/3563505835 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f95dc04b050 con 0x562707954c90 2024-05-10T11:28:58.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.597+0000 7f95f97d3b80 10 client.17523 fetch_fsmap learned FSMap version 31 2024-05-10T11:28:58.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.597+0000 7f95f97d3b80 1 -- 172.21.15.100:0/3563505835 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5627079d81d0 con 0x562707954c90 2024-05-10T11:28:58.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.597+0000 7f95c7fff700 1 -- 172.21.15.100:0/3563505835 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f95dc03a070 con 0x562707954c90 2024-05-10T11:28:58.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.597+0000 7f95c7fff700 1 -- 172.21.15.100:0/3563505835 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f95dc035ba0 con 0x562707954c90 2024-05-10T11:28:58.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.597+0000 7f95f97d3b80 10 client.17523 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:28:58.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.597+0000 7f95f97d3b80 10 client.17523 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-10T11:28:58.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.597+0000 7f95f97d3b80 20 client.17523 populate_metadata read hostname 'smithi100' 2024-05-10T11:28:58.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.597+0000 7f95f97d3b80 1 --2- 172.21.15.100:0/3563505835 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x5627079d87b0 0x5627079dabd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:28:58.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.597+0000 7f95f97d3b80 4 client.17523 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:28:58.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.597+0000 7f95f97d3b80 1 -- 172.21.15.100:0/3563505835 --> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5627078b0240 con 0x5627079d87b0 2024-05-10T11:28:58.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.597+0000 7f95e7fff700 1 --2- 172.21.15.100:0/3563505835 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x5627079d87b0 0x5627079dabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:28:58.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.598+0000 7f95c7fff700 10 client.17523 ms_handle_connect on v2:172.21.15.188:6836/1943342620 2024-05-10T11:28:58.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.598+0000 7f95e7fff700 1 --2- 172.21.15.100:0/3563505835 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x5627079d87b0 0x5627079dabd0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5cf7823d07324297 server_cookie=68a0a57ef3218747 in_seq=0 out_seq=0 2024-05-10T11:28:58.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.598+0000 7f95c7fff700 1 -- 172.21.15.100:0/3563505835 <== mds.2 v2:172.21.15.188:6836/1943342620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5627078b0240 con 0x5627079d87b0 2024-05-10T11:28:58.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.598+0000 7f95c7fff700 10 client.17523 handle_command_reply: tid=0 2024-05-10T11:28:58.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.598+0000 7f95f97d3b80 1 client.17523 shutdown 2024-05-10T11:28:58.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.598+0000 7f95f97d3b80 1 -- 172.21.15.100:0/3563505835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562707954c90 msgr2=0x562707966630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:58.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.598+0000 7f95f97d3b80 1 --2- 172.21.15.100:0/3563505835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562707954c90 0x562707966630 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f95dc011bf0 tx=0x7f95dc0038b0 comp rx=0 tx=0).stop 2024-05-10T11:28:58.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.598+0000 7f95f97d3b80 1 -- 172.21.15.100:0/3563505835 shutdown_connections 2024-05-10T11:28:58.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.598+0000 7f95f97d3b80 1 --2- 172.21.15.100:0/3563505835 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x5627079d87b0 0x5627079dabd0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:58.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.598+0000 7f95f97d3b80 1 --2- 172.21.15.100:0/3563505835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56270795a8a0 0x562707974e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:58.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.598+0000 7f95c7fff700 0 client.17523 ms_handle_reset on v2:172.21.15.188:6836/1943342620 2024-05-10T11:28:58.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.598+0000 7f95f97d3b80 1 --2- 172.21.15.100:0/3563505835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562707954c90 0x562707966630 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:58.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.598+0000 7f95f97d3b80 1 --2- 172.21.15.100:0/3563505835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562707959f80 0x562707966b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:58.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.598+0000 7f95f97d3b80 1 -- 172.21.15.100:0/3563505835 >> 172.21.15.100:0/3563505835 conn(0x56270787c100 msgr2=0x562707944e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:58.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.599+0000 7f95f97d3b80 1 -- 172.21.15.100:0/3563505835 shutdown_connections 2024-05-10T11:28:58.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.599+0000 7f95f97d3b80 1 -- 172.21.15.100:0/3563505835 wait complete. 2024-05-10T11:28:58.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.599+0000 7f95f97d3b80 20 client.17523 trim_cache size 0 max 16384 2024-05-10T11:28:58.605 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:28:58.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.599+0000 7f95eef40700 1 -- 172.21.15.100:0/967648567 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f95d80b2880 msgr2=0x7f95d80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:58.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.599+0000 7f95eef40700 1 --2- 172.21.15.100:0/967648567 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f95d80b2880 0x7f95d80b4d40 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f95dc009be0 tx=0x7f95dc009260 comp rx=0 tx=0).stop 2024-05-10T11:28:58.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.599+0000 7f95eef40700 1 -- 172.21.15.100:0/967648567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95e8108410 msgr2=0x7f95e81a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:28:58.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.599+0000 7f95eef40700 1 --2- 172.21.15.100:0/967648567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95e8108410 0x7f95e81a5a10 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f95e000b070 tx=0x7f95e0004c80 comp rx=0 tx=0).stop 2024-05-10T11:28:58.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.599+0000 7f95eef40700 1 -- 172.21.15.100:0/967648567 shutdown_connections 2024-05-10T11:28:58.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.599+0000 7f95eef40700 1 --2- 172.21.15.100:0/967648567 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f95d80b2880 0x7f95d80b4d40 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:58.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.599+0000 7f95eef40700 1 --2- 172.21.15.100:0/967648567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95e8108410 0x7f95e81a5a10 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:58.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.599+0000 7f95eef40700 1 --2- 172.21.15.100:0/967648567 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95e81023a0 0x7f95e819a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:58.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.599+0000 7f95eef40700 1 --2- 172.21.15.100:0/967648567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95e8102d60 0x7f95e819aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:28:58.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.599+0000 7f95eef40700 1 -- 172.21.15.100:0/967648567 >> 172.21.15.100:0/967648567 conn(0x7f95e8079350 msgr2=0x7f95e8101060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:28:58.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.600+0000 7f95eef40700 1 -- 172.21.15.100:0/967648567 shutdown_connections 2024-05-10T11:28:58.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:28:58.600+0000 7f95eef40700 1 -- 172.21.15.100:0/967648567 wait complete. 2024-05-10T11:28:58.980 INFO:tasks.workunit.client.0.smithi100.stdout: 1 273421 2.73 MB/sec execute 413 sec latency 15.909 ms 2024-05-10T11:28:59.471 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-10T11:28:59.728 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:28:59.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:28:59 smithi188 ceph-mon[34075]: pgmap v738: 129 pgs: 129 active+clean; 110 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 1.3 MiB/s wr, 169 op/s 2024-05-10T11:28:59.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:28:59 smithi100 ceph-mon[30296]: pgmap v738: 129 pgs: 129 active+clean; 110 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 1.3 MiB/s wr, 169 op/s 2024-05-10T11:28:59.980 INFO:tasks.workunit.client.0.smithi100.stdout: 1 273676 2.73 MB/sec execute 414 sec latency 13.927 ms 2024-05-10T11:29:00.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:28:59 smithi178 ceph-mon[33359]: pgmap v738: 129 pgs: 129 active+clean; 110 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 1.3 MiB/s wr, 169 op/s 2024-05-10T11:29:00.980 INFO:tasks.workunit.client.0.smithi100.stdout: 1 274081 2.72 MB/sec execute 415 sec latency 15.421 ms 2024-05-10T11:29:01.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.438+0000 7fea05d7b700 1 -- 172.21.15.100:0/2756132659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea00109660 msgr2=0x7fea00109a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:01.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.438+0000 7fea05d7b700 1 --2- 172.21.15.100:0/2756132659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea00109660 0x7fea00109a40 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0009990 tx=0x7fe9f0009ca0 comp rx=0 tx=0).stop 2024-05-10T11:29:01.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.438+0000 7fea05d7b700 1 -- 172.21.15.100:0/2756132659 shutdown_connections 2024-05-10T11:29:01.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.438+0000 7fea05d7b700 1 --2- 172.21.15.100:0/2756132659 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea00104180 0x7fea001103f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:01.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.438+0000 7fea05d7b700 1 --2- 172.21.15.100:0/2756132659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea00109660 0x7fea00109a40 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:01.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.438+0000 7fea05d7b700 1 --2- 172.21.15.100:0/2756132659 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea001037c0 0x7fea00103c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:01.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.438+0000 7fea05d7b700 1 -- 172.21.15.100:0/2756132659 >> 172.21.15.100:0/2756132659 conn(0x7fea0007b430 msgr2=0x7fea0007b840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:01.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.438+0000 7fea05d7b700 1 -- 172.21.15.100:0/2756132659 shutdown_connections 2024-05-10T11:29:01.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.438+0000 7fea05d7b700 1 -- 172.21.15.100:0/2756132659 wait complete. 2024-05-10T11:29:01.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.439+0000 7fea05d7b700 1 Processor -- start 2024-05-10T11:29:01.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.439+0000 7fea05d7b700 1 -- start start 2024-05-10T11:29:01.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.439+0000 7fea05d7b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea001037c0 0x7fea0019f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:01.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.439+0000 7fea05d7b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea00104180 0x7fea0019fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:01.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.439+0000 7fea05d7b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea00109660 0x7fea001998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:01.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.439+0000 7fea05d7b700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fea001135e0 con 0x7fea00104180 2024-05-10T11:29:01.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.439+0000 7fea05d7b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fea00113460 con 0x7fea001037c0 2024-05-10T11:29:01.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.439+0000 7fea05d7b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fea00113760 con 0x7fea00109660 2024-05-10T11:29:01.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.440+0000 7fe9feffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea00104180 0x7fea0019fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:01.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.440+0000 7fe9fffff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea00109660 0x7fea001998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:01.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.440+0000 7fe9feffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea00104180 0x7fea0019fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39958/0 (socket says 172.21.15.100:39958) 2024-05-10T11:29:01.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.440+0000 7fe9feffd700 1 -- 172.21.15.100:0/4182257845 learned_addr learned my addr 172.21.15.100:0/4182257845 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:29:01.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.440+0000 7fe9ff7fe700 1 --2- 172.21.15.100:0/4182257845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea001037c0 0x7fea0019f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:01.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.440+0000 7fe9fffff700 1 -- 172.21.15.100:0/4182257845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea001037c0 msgr2=0x7fea0019f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:01.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.440+0000 7fe9fffff700 1 --2- 172.21.15.100:0/4182257845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea001037c0 0x7fea0019f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:01.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.440+0000 7fe9fffff700 1 -- 172.21.15.100:0/4182257845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea00104180 msgr2=0x7fea0019fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:01.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.440+0000 7fe9fffff700 1 --2- 172.21.15.100:0/4182257845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea00104180 0x7fea0019fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:01.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.440+0000 7fe9fffff700 1 -- 172.21.15.100:0/4182257845 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea0019a040 con 0x7fea00109660 2024-05-10T11:29:01.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.440+0000 7fe9feffd700 1 --2- 172.21.15.100:0/4182257845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea00104180 0x7fea0019fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:29:01.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.440+0000 7fe9ff7fe700 1 --2- 172.21.15.100:0/4182257845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea001037c0 0x7fea0019f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:29:01.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.440+0000 7fe9fffff700 1 --2- 172.21.15.100:0/4182257845 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea00109660 0x7fea001998e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe9f800efc0 tx=0x7fe9f800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:01.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.440+0000 7fe9fcff9700 1 -- 172.21.15.100:0/4182257845 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f80092e0 con 0x7fea00109660 2024-05-10T11:29:01.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.440+0000 7fea05d7b700 1 -- 172.21.15.100:0/4182257845 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea0019a2d0 con 0x7fea00109660 2024-05-10T11:29:01.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.440+0000 7fe9fcff9700 1 -- 172.21.15.100:0/4182257845 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe9f8004bb0 con 0x7fea00109660 2024-05-10T11:29:01.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.440+0000 7fe9fcff9700 1 -- 172.21.15.100:0/4182257845 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f8004020 con 0x7fea00109660 2024-05-10T11:29:01.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.440+0000 7fea05d7b700 1 -- 172.21.15.100:0/4182257845 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea000756d0 con 0x7fea00109660 2024-05-10T11:29:01.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.443+0000 7fe9fcff9700 1 -- 172.21.15.100:0/4182257845 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fe9f801b030 con 0x7fea00109660 2024-05-10T11:29:01.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.444+0000 7fe9fcff9700 1 --2- 172.21.15.100:0/4182257845 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe9e80b2900 0x7fe9e80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:01.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.444+0000 7fe9ff7fe700 1 --2- 172.21.15.100:0/4182257845 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe9e80b2900 0x7fe9e80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:01.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.444+0000 7fe9fcff9700 1 -- 172.21.15.100:0/4182257845 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fe9f80e1f10 con 0x7fea00109660 2024-05-10T11:29:01.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.444+0000 7fea1060eb80 1 Processor -- start 2024-05-10T11:29:01.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.444+0000 7fea1060eb80 1 -- start start 2024-05-10T11:29:01.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.444+0000 7fe9ff7fe700 1 --2- 172.21.15.100:0/4182257845 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe9e80b2900 0x7fe9e80b4dc0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe9f000b2e0 tx=0x7fe9f001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:01.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.445+0000 7fea1060eb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2cd9cc4d0 0x55d2cd9cc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:01.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.445+0000 7fea1060eb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d2cd9ccdf0 0x55d2cd9d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:01.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.445+0000 7fea1060eb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2cd9d0c80 0x55d2cd9d1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:01.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.445+0000 7fea1060eb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55d2cd91e860 con 0x55d2cd9cc4d0 2024-05-10T11:29:01.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.445+0000 7fea1060eb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d2cd8bd7c0 con 0x55d2cd9d0c80 2024-05-10T11:29:01.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.445+0000 7fea1060eb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d2cd91a820 con 0x55d2cd9ccdf0 2024-05-10T11:29:01.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.445+0000 7fe9ff7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2cd9cc4d0 0x55d2cd9cc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:01.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.445+0000 7fe9ff7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2cd9cc4d0 0x55d2cd9cc8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39970/0 (socket says 172.21.15.100:39970) 2024-05-10T11:29:01.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.445+0000 7fe9ff7fe700 1 -- 172.21.15.100:0/3576907722 learned_addr learned my addr 172.21.15.100:0/3576907722 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:29:01.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.445+0000 7fe9feffd700 1 --2- 172.21.15.100:0/3576907722 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d2cd9ccdf0 0x55d2cd9d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:01.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.445+0000 7fe9ff7fe700 1 -- 172.21.15.100:0/3576907722 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d2cd9ccdf0 msgr2=0x55d2cd9d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:01.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.445+0000 7fe9ff7fe700 1 --2- 172.21.15.100:0/3576907722 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d2cd9ccdf0 0x55d2cd9d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:01.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.445+0000 7fe9fffff700 1 --2- 172.21.15.100:0/3576907722 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2cd9d0c80 0x55d2cd9d1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:01.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.445+0000 7fe9ff7fe700 1 -- 172.21.15.100:0/3576907722 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2cd9d0c80 msgr2=0x55d2cd9d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:01.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.445+0000 7fe9ff7fe700 1 --2- 172.21.15.100:0/3576907722 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2cd9d0c80 0x55d2cd9d1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:01.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.445+0000 7fe9ff7fe700 1 -- 172.21.15.100:0/3576907722 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2cd86b8d0 con 0x55d2cd9cc4d0 2024-05-10T11:29:01.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.445+0000 7fe9feffd700 1 --2- 172.21.15.100:0/3576907722 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d2cd9ccdf0 0x55d2cd9d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:29:01.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.445+0000 7fe9fffff700 1 --2- 172.21.15.100:0/3576907722 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2cd9d0c80 0x55d2cd9d1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:29:01.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.445+0000 7fe9ff7fe700 1 --2- 172.21.15.100:0/3576907722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2cd9cc4d0 0x55d2cd9cc8b0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0015000 tx=0x7fe9f001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dc420ed778c73ee server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:01.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.445+0000 7fe9e5ffb700 1 -- 172.21.15.100:0/3576907722 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f0003680 con 0x55d2cd9cc4d0 2024-05-10T11:29:01.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.446+0000 7fe9e5ffb700 1 -- 172.21.15.100:0/3576907722 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe9f0003cb0 con 0x55d2cd9cc4d0 2024-05-10T11:29:01.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.446+0000 7fe9e5ffb700 1 -- 172.21.15.100:0/3576907722 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f000fd10 con 0x55d2cd9cc4d0 2024-05-10T11:29:01.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.446+0000 7fea1060eb80 1 -- 172.21.15.100:0/3576907722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2cd9cc4d0 msgr2=0x55d2cd9cc8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:01.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.446+0000 7fea1060eb80 1 --2- 172.21.15.100:0/3576907722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2cd9cc4d0 0x55d2cd9cc8b0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0015000 tx=0x7fe9f001f000 comp rx=0 tx=0).stop 2024-05-10T11:29:01.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.446+0000 7fea1060eb80 1 -- 172.21.15.100:0/3576907722 shutdown_connections 2024-05-10T11:29:01.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.446+0000 7fea1060eb80 1 --2- 172.21.15.100:0/3576907722 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2cd9d0c80 0x55d2cd9d1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:01.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.446+0000 7fea1060eb80 1 --2- 172.21.15.100:0/3576907722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2cd9cc4d0 0x55d2cd9cc8b0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:01.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.446+0000 7fea1060eb80 1 --2- 172.21.15.100:0/3576907722 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d2cd9ccdf0 0x55d2cd9d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:01.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.446+0000 7fea1060eb80 1 -- 172.21.15.100:0/3576907722 >> 172.21.15.100:0/3576907722 conn(0x55d2cd8f8100 msgr2=0x55d2cd9c6c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:01.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.447+0000 7fea1060eb80 1 -- 172.21.15.100:0/3576907722 shutdown_connections 2024-05-10T11:29:01.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.447+0000 7fea1060eb80 1 -- 172.21.15.100:0/3576907722 wait complete. 2024-05-10T11:29:01.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.447+0000 7fea1060eb80 1 -- start start 2024-05-10T11:29:01.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.447+0000 7fea1060eb80 1 Processor -- start 2024-05-10T11:29:01.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.448+0000 7fea1060eb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d2cd9cc4d0 0x55d2cd9d8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:01.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.448+0000 7fea1060eb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2cd9ccdf0 0x55d2cd9d54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:01.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.448+0000 7fea1060eb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2cd9d0c80 0x55d2cd9d5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:01.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.448+0000 7fea1060eb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55d2cd9d8e30 con 0x55d2cd9ccdf0 2024-05-10T11:29:01.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.448+0000 7fea1060eb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d2cd884c30 con 0x55d2cd9d0c80 2024-05-10T11:29:01.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.448+0000 7fea1060eb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d2cd9d8fb0 con 0x55d2cd9cc4d0 2024-05-10T11:29:01.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.448+0000 7fea1060eb80 10 client.0 _ll_register_callbacks cb 0x55d2cd8b6920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:29:01.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.448+0000 7fe9feffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2cd9ccdf0 0x55d2cd9d54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:01.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.448+0000 7fe9ff7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d2cd9cc4d0 0x55d2cd9d8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:01.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.448+0000 7fe9feffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2cd9ccdf0 0x55d2cd9d54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39972/0 (socket says 172.21.15.100:39972) 2024-05-10T11:29:01.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.448+0000 7fe9feffd700 1 -- 172.21.15.100:0/1777994852 learned_addr learned my addr 172.21.15.100:0/1777994852 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:29:01.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.449+0000 7fe9fffff700 1 --2- 172.21.15.100:0/1777994852 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2cd9d0c80 0x55d2cd9d5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:01.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.449+0000 7fe9ff7fe700 1 -- 172.21.15.100:0/1777994852 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2cd9d0c80 msgr2=0x55d2cd9d5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:01.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.449+0000 7fe9ff7fe700 1 --2- 172.21.15.100:0/1777994852 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2cd9d0c80 0x55d2cd9d5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:01.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.449+0000 7fe9ff7fe700 1 -- 172.21.15.100:0/1777994852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2cd9ccdf0 msgr2=0x55d2cd9d54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:01.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.449+0000 7fe9ff7fe700 1 --2- 172.21.15.100:0/1777994852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2cd9ccdf0 0x55d2cd9d54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:01.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.449+0000 7fe9ff7fe700 1 -- 172.21.15.100:0/1777994852 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2cd8ba690 con 0x55d2cd9cc4d0 2024-05-10T11:29:01.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.449+0000 7fe9feffd700 1 --2- 172.21.15.100:0/1777994852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2cd9ccdf0 0x55d2cd9d54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:29:01.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.449+0000 7fe9fffff700 1 --2- 172.21.15.100:0/1777994852 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2cd9d0c80 0x55d2cd9d5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:29:01.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.449+0000 7fe9ff7fe700 1 --2- 172.21.15.100:0/1777994852 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d2cd9cc4d0 0x55d2cd9d8140 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe9f00098e0 tx=0x7fe9f0003e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d16a9ac0160462cc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:01.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.449+0000 7fe9deffd700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:29:01.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.449+0000 7fe9deffd700 1 -- 172.21.15.100:0/1777994852 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f000f9b0 con 0x55d2cd9cc4d0 2024-05-10T11:29:01.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.449+0000 7fe9deffd700 1 -- 172.21.15.100:0/1777994852 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fe9f000fb50 con 0x55d2cd9cc4d0 2024-05-10T11:29:01.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.449+0000 7fea1060eb80 1 -- 172.21.15.100:0/1777994852 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9f0003cb0 con 0x55d2cd9cc4d0 2024-05-10T11:29:01.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.449+0000 7fe9deffd700 1 -- 172.21.15.100:0/1777994852 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe9f0049050 con 0x55d2cd9cc4d0 2024-05-10T11:29:01.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.449+0000 7fea1060eb80 10 client.26986 fetch_fsmap learned FSMap version 31 2024-05-10T11:29:01.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.449+0000 7fea1060eb80 1 -- 172.21.15.100:0/1777994852 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d2cd9d8e30 con 0x55d2cd9cc4d0 2024-05-10T11:29:01.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.450+0000 7fe9deffd700 1 -- 172.21.15.100:0/1777994852 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe9f0003cb0 con 0x55d2cd9cc4d0 2024-05-10T11:29:01.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.450+0000 7fe9deffd700 1 -- 172.21.15.100:0/1777994852 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fe9f0024440 con 0x55d2cd9cc4d0 2024-05-10T11:29:01.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.450+0000 7fea1060eb80 10 client.26986 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:29:01.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.450+0000 7fea1060eb80 10 client.26986 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-10T11:29:01.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.450+0000 7fea1060eb80 20 client.26986 populate_metadata read hostname 'smithi100' 2024-05-10T11:29:01.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.450+0000 7fea1060eb80 1 --2- 172.21.15.100:0/1777994852 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55d2cd9e5be0 0x55d2cd9e8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:01.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.450+0000 7fea1060eb80 4 client.26986 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:29:01.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.450+0000 7fea1060eb80 1 -- 172.21.15.100:0/1777994852 --> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2cd998650 con 0x55d2cd9e5be0 2024-05-10T11:29:01.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.450+0000 7fe9feffd700 1 --2- 172.21.15.100:0/1777994852 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55d2cd9e5be0 0x55d2cd9e8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:01.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.451+0000 7fe9deffd700 10 client.26986 ms_handle_connect on v2:172.21.15.178:6838/3760985404 2024-05-10T11:29:01.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.451+0000 7fe9feffd700 1 --2- 172.21.15.100:0/1777994852 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55d2cd9e5be0 0x55d2cd9e8000 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3d470ed61322cfa9 server_cookie=5700c7210c20d261 in_seq=0 out_seq=0 2024-05-10T11:29:01.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.453+0000 7fe9deffd700 1 -- 172.21.15.100:0/1777994852 <== mds.1 v2:172.21.15.178:6838/3760985404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d2cd998650 con 0x55d2cd9e5be0 2024-05-10T11:29:01.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.453+0000 7fe9deffd700 10 client.26986 handle_command_reply: tid=0 2024-05-10T11:29:01.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.453+0000 7fea1060eb80 1 client.26986 shutdown 2024-05-10T11:29:01.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.453+0000 7fea1060eb80 1 -- 172.21.15.100:0/1777994852 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d2cd9cc4d0 msgr2=0x55d2cd9d8140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:01.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.453+0000 7fea1060eb80 1 --2- 172.21.15.100:0/1777994852 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d2cd9cc4d0 0x55d2cd9d8140 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe9f00098e0 tx=0x7fe9f0003e60 comp rx=0 tx=0).stop 2024-05-10T11:29:01.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.453+0000 7fea1060eb80 1 -- 172.21.15.100:0/1777994852 shutdown_connections 2024-05-10T11:29:01.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.453+0000 7fea1060eb80 1 --2- 172.21.15.100:0/1777994852 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55d2cd9e5be0 0x55d2cd9e8000 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:01.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.453+0000 7fea1060eb80 1 --2- 172.21.15.100:0/1777994852 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2cd9d0c80 0x55d2cd9d5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:01.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.453+0000 7fea1060eb80 1 --2- 172.21.15.100:0/1777994852 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d2cd9cc4d0 0x55d2cd9d8140 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:01.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.453+0000 7fea1060eb80 1 --2- 172.21.15.100:0/1777994852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2cd9ccdf0 0x55d2cd9d54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:01.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.453+0000 7fea1060eb80 1 -- 172.21.15.100:0/1777994852 >> 172.21.15.100:0/1777994852 conn(0x55d2cd8f8100 msgr2=0x55d2cd9c1720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:01.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.453+0000 7fe9deffd700 0 client.26986 ms_handle_reset on v2:172.21.15.178:6838/3760985404 2024-05-10T11:29:01.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.455+0000 7fea1060eb80 1 -- 172.21.15.100:0/1777994852 shutdown_connections 2024-05-10T11:29:01.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.455+0000 7fea1060eb80 1 -- 172.21.15.100:0/1777994852 wait complete. 2024-05-10T11:29:01.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.455+0000 7fea1060eb80 20 client.26986 trim_cache size 0 max 16384 2024-05-10T11:29:01.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.480+0000 7fea1060eb80 1 Processor -- start 2024-05-10T11:29:01.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.480+0000 7fea1060eb80 1 -- start start 2024-05-10T11:29:01.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.480+0000 7fea1060eb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2cd9cc4d0 0x55d2cd9e1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:01.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.480+0000 7fea1060eb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d2cd9ccdf0 0x55d2cd9c15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:01.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.480+0000 7fea1060eb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2cd9d0c80 0x55d2cd9ed8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:01.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.480+0000 7fea1060eb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fea001af6d0 con 0x55d2cd9cc4d0 2024-05-10T11:29:01.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.480+0000 7fea1060eb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fea001acf60 con 0x55d2cd9d0c80 2024-05-10T11:29:01.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.480+0000 7fea1060eb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fea001aadb0 con 0x55d2cd9ccdf0 2024-05-10T11:29:01.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.480+0000 7fe9ff7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2cd9cc4d0 0x55d2cd9e1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:01.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.480+0000 7fe9feffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d2cd9ccdf0 0x55d2cd9c15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:01.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.480+0000 7fe9ff7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2cd9cc4d0 0x55d2cd9e1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39978/0 (socket says 172.21.15.100:39978) 2024-05-10T11:29:01.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.480+0000 7fe9ff7fe700 1 -- 172.21.15.100:0/3114961972 learned_addr learned my addr 172.21.15.100:0/3114961972 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:29:01.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.480+0000 7fe9fffff700 1 --2- 172.21.15.100:0/3114961972 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2cd9d0c80 0x55d2cd9ed8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:01.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.481+0000 7fe9ff7fe700 1 -- 172.21.15.100:0/3114961972 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d2cd9ccdf0 msgr2=0x55d2cd9c15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:01.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.481+0000 7fe9ff7fe700 1 --2- 172.21.15.100:0/3114961972 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d2cd9ccdf0 0x55d2cd9c15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:01.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.481+0000 7fe9ff7fe700 1 -- 172.21.15.100:0/3114961972 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2cd9d0c80 msgr2=0x55d2cd9ed8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:01.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.481+0000 7fe9ff7fe700 1 --2- 172.21.15.100:0/3114961972 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2cd9d0c80 0x55d2cd9ed8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:01.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.481+0000 7fe9ff7fe700 1 -- 172.21.15.100:0/3114961972 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2cd9d3ab0 con 0x55d2cd9cc4d0 2024-05-10T11:29:01.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.481+0000 7fe9feffd700 1 --2- 172.21.15.100:0/3114961972 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d2cd9ccdf0 0x55d2cd9c15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:29:01.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.481+0000 7fe9ff7fe700 1 --2- 172.21.15.100:0/3114961972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2cd9cc4d0 0x55d2cd9e1ec0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fe9f003b040 tx=0x7fe9f0009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd8d34ab8cc913f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:01.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.481+0000 7fe9e57fa700 1 -- 172.21.15.100:0/3114961972 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f0024440 con 0x55d2cd9cc4d0 2024-05-10T11:29:01.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.481+0000 7fe9e57fa700 1 -- 172.21.15.100:0/3114961972 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe9f002a030 con 0x55d2cd9cc4d0 2024-05-10T11:29:01.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.481+0000 7fe9e57fa700 1 -- 172.21.15.100:0/3114961972 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f0003cb0 con 0x55d2cd9cc4d0 2024-05-10T11:29:01.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.482+0000 7fea1060eb80 1 -- 172.21.15.100:0/3114961972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2cd9cc4d0 msgr2=0x55d2cd9e1ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:01.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.482+0000 7fea1060eb80 1 --2- 172.21.15.100:0/3114961972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2cd9cc4d0 0x55d2cd9e1ec0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fe9f003b040 tx=0x7fe9f0009ca0 comp rx=0 tx=0).stop 2024-05-10T11:29:01.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.482+0000 7fea1060eb80 1 -- 172.21.15.100:0/3114961972 shutdown_connections 2024-05-10T11:29:01.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.482+0000 7fea1060eb80 1 --2- 172.21.15.100:0/3114961972 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2cd9d0c80 0x55d2cd9ed8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:01.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.482+0000 7fea1060eb80 1 --2- 172.21.15.100:0/3114961972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2cd9cc4d0 0x55d2cd9e1ec0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:01.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.482+0000 7fea1060eb80 1 --2- 172.21.15.100:0/3114961972 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d2cd9ccdf0 0x55d2cd9c15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:01.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.482+0000 7fea1060eb80 1 -- 172.21.15.100:0/3114961972 >> 172.21.15.100:0/3114961972 conn(0x55d2cd8f8100 msgr2=0x55d2cd9c0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:01.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.482+0000 7fea1060eb80 1 -- 172.21.15.100:0/3114961972 shutdown_connections 2024-05-10T11:29:01.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.482+0000 7fea1060eb80 1 -- 172.21.15.100:0/3114961972 wait complete. 2024-05-10T11:29:01.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.482+0000 7fea1060eb80 1 -- start start 2024-05-10T11:29:01.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.483+0000 7fea1060eb80 1 Processor -- start 2024-05-10T11:29:01.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.483+0000 7fea1060eb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d2cd9cc4d0 0x55d2cd9db210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:01.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.483+0000 7fea1060eb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2cd9ccdf0 0x55d2cd9db750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:01.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.483+0000 7fea1060eb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2cd9d0c80 0x55d2cd9dbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:01.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.483+0000 7fea1060eb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fea001ab420 con 0x55d2cd9ccdf0 2024-05-10T11:29:01.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.483+0000 7fea1060eb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fea001ac890 con 0x55d2cd9d0c80 2024-05-10T11:29:01.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.483+0000 7fea1060eb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fea001abaf0 con 0x55d2cd9cc4d0 2024-05-10T11:29:01.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.483+0000 7fea1060eb80 10 client.0 _ll_register_callbacks cb 0x55d2cd8b6920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:29:01.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.483+0000 7fe9ff7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d2cd9cc4d0 0x55d2cd9db210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:01.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.483+0000 7fe9feffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2cd9ccdf0 0x55d2cd9db750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:01.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.483+0000 7fe9ff7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d2cd9cc4d0 0x55d2cd9db210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:48924/0 (socket says 172.21.15.100:48924) 2024-05-10T11:29:01.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.483+0000 7fe9ff7fe700 1 -- 172.21.15.100:0/1932437985 learned_addr learned my addr 172.21.15.100:0/1932437985 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:29:01.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.483+0000 7fe9fffff700 1 --2- 172.21.15.100:0/1932437985 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2cd9d0c80 0x55d2cd9dbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:01.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.483+0000 7fe9ff7fe700 1 -- 172.21.15.100:0/1932437985 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2cd9d0c80 msgr2=0x55d2cd9dbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:01.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.483+0000 7fe9ff7fe700 1 --2- 172.21.15.100:0/1932437985 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2cd9d0c80 0x55d2cd9dbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:01.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.483+0000 7fe9ff7fe700 1 -- 172.21.15.100:0/1932437985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2cd9ccdf0 msgr2=0x55d2cd9db750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:01.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.483+0000 7fe9ff7fe700 1 --2- 172.21.15.100:0/1932437985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2cd9ccdf0 0x55d2cd9db750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:01.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.483+0000 7fe9ff7fe700 1 -- 172.21.15.100:0/1932437985 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2cda4e780 con 0x55d2cd9cc4d0 2024-05-10T11:29:01.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.483+0000 7fe9fffff700 1 --2- 172.21.15.100:0/1932437985 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2cd9d0c80 0x55d2cd9dbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:29:01.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.483+0000 7fe9feffd700 1 --2- 172.21.15.100:0/1932437985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2cd9ccdf0 0x55d2cd9db750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:29:01.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.484+0000 7fe9ff7fe700 1 --2- 172.21.15.100:0/1932437985 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d2cd9cc4d0 0x55d2cd9db210 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0000c00 tx=0x7fe9f00056f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a97fcbcd643dccf5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:01.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.484+0000 7fe9deffd700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:29:01.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.484+0000 7fe9deffd700 1 -- 172.21.15.100:0/1932437985 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f005b060 con 0x55d2cd9cc4d0 2024-05-10T11:29:01.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.484+0000 7fea1060eb80 1 -- 172.21.15.100:0/1932437985 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9f002a030 con 0x55d2cd9cc4d0 2024-05-10T11:29:01.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.484+0000 7fe9deffd700 1 -- 172.21.15.100:0/1932437985 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fe9f0003c60 con 0x55d2cd9cc4d0 2024-05-10T11:29:01.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.484+0000 7fe9deffd700 1 -- 172.21.15.100:0/1932437985 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe9f0062050 con 0x55d2cd9cc4d0 2024-05-10T11:29:01.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.484+0000 7fea1060eb80 10 client.26992 fetch_fsmap learned FSMap version 31 2024-05-10T11:29:01.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.484+0000 7fea1060eb80 1 -- 172.21.15.100:0/1932437985 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d2cda4fa80 con 0x55d2cd9cc4d0 2024-05-10T11:29:01.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.484+0000 7fe9deffd700 1 -- 172.21.15.100:0/1932437985 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe9f002a030 con 0x55d2cd9cc4d0 2024-05-10T11:29:01.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.484+0000 7fe9deffd700 1 -- 172.21.15.100:0/1932437985 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fe9f004e440 con 0x55d2cd9cc4d0 2024-05-10T11:29:01.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.485+0000 7fea1060eb80 10 client.26992 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:29:01.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.485+0000 7fea1060eb80 10 client.26992 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-10T11:29:01.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.485+0000 7fea1060eb80 20 client.26992 populate_metadata read hostname 'smithi100' 2024-05-10T11:29:01.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.485+0000 7fea1060eb80 1 --2- 172.21.15.100:0/1932437985 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55d2cda50020 0x55d2cda52440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:01.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.485+0000 7fea1060eb80 4 client.26992 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:29:01.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.485+0000 7fea1060eb80 1 -- 172.21.15.100:0/1932437985 --> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d2cd92c240 con 0x55d2cda50020 2024-05-10T11:29:01.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.485+0000 7fe9feffd700 1 --2- 172.21.15.100:0/1932437985 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55d2cda50020 0x55d2cda52440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:01.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.485+0000 7fe9feffd700 1 --2- 172.21.15.100:0/1932437985 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55d2cda50020 0x55d2cda52440 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ba811c607ada61e4 server_cookie=136be9edd5e7836 in_seq=0 out_seq=0 2024-05-10T11:29:01.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.485+0000 7fe9deffd700 10 client.26992 ms_handle_connect on v2:172.21.15.178:6838/3760985404 2024-05-10T11:29:01.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.486+0000 7fe9deffd700 1 -- 172.21.15.100:0/1932437985 <== mds.1 v2:172.21.15.178:6838/3760985404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d2cd92c240 con 0x55d2cda50020 2024-05-10T11:29:01.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.486+0000 7fe9deffd700 10 client.26992 handle_command_reply: tid=0 2024-05-10T11:29:01.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.486+0000 7fea1060eb80 1 client.26992 shutdown 2024-05-10T11:29:01.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.487+0000 7fea1060eb80 1 -- 172.21.15.100:0/1932437985 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d2cd9cc4d0 msgr2=0x55d2cd9db210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:01.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.487+0000 7fea1060eb80 1 --2- 172.21.15.100:0/1932437985 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d2cd9cc4d0 0x55d2cd9db210 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0000c00 tx=0x7fe9f00056f0 comp rx=0 tx=0).stop 2024-05-10T11:29:01.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.487+0000 7fea1060eb80 1 -- 172.21.15.100:0/1932437985 shutdown_connections 2024-05-10T11:29:01.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.487+0000 7fea1060eb80 1 --2- 172.21.15.100:0/1932437985 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55d2cda50020 0x55d2cda52440 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:01.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.487+0000 7fea1060eb80 1 --2- 172.21.15.100:0/1932437985 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2cd9d0c80 0x55d2cd9dbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:01.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.487+0000 7fea1060eb80 1 --2- 172.21.15.100:0/1932437985 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d2cd9cc4d0 0x55d2cd9db210 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:01.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.487+0000 7fea1060eb80 1 --2- 172.21.15.100:0/1932437985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2cd9ccdf0 0x55d2cd9db750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:01.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.487+0000 7fea1060eb80 1 -- 172.21.15.100:0/1932437985 >> 172.21.15.100:0/1932437985 conn(0x55d2cd8f8100 msgr2=0x55d2cd9cb2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:01.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.487+0000 7fe9deffd700 0 client.26992 ms_handle_reset on v2:172.21.15.178:6838/3760985404 2024-05-10T11:29:01.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.487+0000 7fea1060eb80 1 -- 172.21.15.100:0/1932437985 shutdown_connections 2024-05-10T11:29:01.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.487+0000 7fea1060eb80 1 -- 172.21.15.100:0/1932437985 wait complete. 2024-05-10T11:29:01.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.487+0000 7fea1060eb80 20 client.26992 trim_cache size 0 max 16384 2024-05-10T11:29:01.491 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:29:01.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.488+0000 7fea05d7b700 1 -- 172.21.15.100:0/4182257845 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe9e80b2900 msgr2=0x7fe9e80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:01.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.488+0000 7fea05d7b700 1 --2- 172.21.15.100:0/4182257845 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe9e80b2900 0x7fe9e80b4dc0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe9f000b2e0 tx=0x7fe9f001a040 comp rx=0 tx=0).stop 2024-05-10T11:29:01.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.488+0000 7fea05d7b700 1 -- 172.21.15.100:0/4182257845 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea00109660 msgr2=0x7fea001998e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:01.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.488+0000 7fea05d7b700 1 --2- 172.21.15.100:0/4182257845 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea00109660 0x7fea001998e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe9f800efc0 tx=0x7fe9f800c3a0 comp rx=0 tx=0).stop 2024-05-10T11:29:01.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.488+0000 7fea05d7b700 1 -- 172.21.15.100:0/4182257845 shutdown_connections 2024-05-10T11:29:01.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.488+0000 7fea05d7b700 1 --2- 172.21.15.100:0/4182257845 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe9e80b2900 0x7fe9e80b4dc0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:01.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.488+0000 7fea05d7b700 1 --2- 172.21.15.100:0/4182257845 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea00109660 0x7fea001998e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:01.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.488+0000 7fea05d7b700 1 --2- 172.21.15.100:0/4182257845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea001037c0 0x7fea0019f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:01.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.488+0000 7fea05d7b700 1 --2- 172.21.15.100:0/4182257845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea00104180 0x7fea0019fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:01.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.488+0000 7fea05d7b700 1 -- 172.21.15.100:0/4182257845 >> 172.21.15.100:0/4182257845 conn(0x7fea0007b430 msgr2=0x7fea00102390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:01.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.488+0000 7fea05d7b700 1 -- 172.21.15.100:0/4182257845 shutdown_connections 2024-05-10T11:29:01.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:01.488+0000 7fea05d7b700 1 -- 172.21.15.100:0/4182257845 wait complete. 2024-05-10T11:29:01.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:29:01 smithi188 ceph-mon[34075]: pgmap v739: 129 pgs: 129 active+clean; 110 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 1.1 MiB/s wr, 122 op/s 2024-05-10T11:29:01.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:29:01 smithi100 ceph-mon[30296]: pgmap v739: 129 pgs: 129 active+clean; 110 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 1.1 MiB/s wr, 122 op/s 2024-05-10T11:29:01.980 INFO:tasks.workunit.client.0.smithi100.stdout: 1 274619 2.73 MB/sec execute 416 sec latency 17.881 ms 2024-05-10T11:29:02.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:29:01 smithi178 ceph-mon[33359]: pgmap v739: 129 pgs: 129 active+clean; 110 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 1.1 MiB/s wr, 122 op/s 2024-05-10T11:29:02.215 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-10T11:29:02.491 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:29:02.980 INFO:tasks.workunit.client.0.smithi100.stdout: 1 275234 2.72 MB/sec execute 417 sec latency 17.789 ms 2024-05-10T11:29:03.655 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:29:03 smithi100 ceph-mon[30296]: pgmap v740: 129 pgs: 129 active+clean; 111 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 1.3 MiB/s wr, 124 op/s 2024-05-10T11:29:03.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:29:03 smithi188 ceph-mon[34075]: pgmap v740: 129 pgs: 129 active+clean; 111 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 1.3 MiB/s wr, 124 op/s 2024-05-10T11:29:03.980 INFO:tasks.workunit.client.0.smithi100.stdout: 1 275734 2.72 MB/sec execute 418 sec latency 15.094 ms 2024-05-10T11:29:04.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:29:03 smithi178 ceph-mon[33359]: pgmap v740: 129 pgs: 129 active+clean; 111 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 1.3 MiB/s wr, 124 op/s 2024-05-10T11:29:04.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.237+0000 7f2979584700 1 -- 172.21.15.100:0/206603583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2974102d60 msgr2=0x7f297410f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:04.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.237+0000 7f2979584700 1 --2- 172.21.15.100:0/206603583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2974102d60 0x7f297410f250 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f296c00b0a0 tx=0x7f296c00b3b0 comp rx=0 tx=0).stop 2024-05-10T11:29:04.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.237+0000 7f2979584700 1 -- 172.21.15.100:0/206603583 shutdown_connections 2024-05-10T11:29:04.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.237+0000 7f2979584700 1 --2- 172.21.15.100:0/206603583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2974102d60 0x7f297410f250 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:04.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.237+0000 7f2979584700 1 --2- 172.21.15.100:0/206603583 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2974108410 0x7f29741087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:04.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.237+0000 7f2979584700 1 --2- 172.21.15.100:0/206603583 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29741023a0 0x7f2974102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:04.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.237+0000 7f2979584700 1 -- 172.21.15.100:0/206603583 >> 172.21.15.100:0/206603583 conn(0x7f2974079350 msgr2=0x7f2974079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:04.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.237+0000 7f2979584700 1 -- 172.21.15.100:0/206603583 shutdown_connections 2024-05-10T11:29:04.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.238+0000 7f2979584700 1 -- 172.21.15.100:0/206603583 wait complete. 2024-05-10T11:29:04.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.238+0000 7f2979584700 1 Processor -- start 2024-05-10T11:29:04.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.238+0000 7f2979584700 1 -- start start 2024-05-10T11:29:04.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.239+0000 7f2979584700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29741023a0 0x7f2974104310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:04.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.239+0000 7f2979584700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2974102d60 0x7f2974104850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:04.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.239+0000 7f2979584700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2974108410 0x7f2974104d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:04.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.239+0000 7f2979584700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2974112410 con 0x7f2974108410 2024-05-10T11:29:04.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.239+0000 7f2979584700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2974112290 con 0x7f29741023a0 2024-05-10T11:29:04.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.239+0000 7f2979584700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2974112590 con 0x7f2974102d60 2024-05-10T11:29:04.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.239+0000 7f29737fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2974108410 0x7f2974104d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:04.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.239+0000 7f29727fc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2974102d60 0x7f2974104850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:04.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.239+0000 7f2972ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29741023a0 0x7f2974104310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:04.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.239+0000 7f2972ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29741023a0 0x7f2974104310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:57482/0 (socket says 172.21.15.100:57482) 2024-05-10T11:29:04.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.239+0000 7f2972ffd700 1 -- 172.21.15.100:0/2359902836 learned_addr learned my addr 172.21.15.100:0/2359902836 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:29:04.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.239+0000 7f29727fc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2974102d60 0x7f2974104850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:48952/0 (socket says 172.21.15.100:48952) 2024-05-10T11:29:04.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.239+0000 7f29737fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2974108410 0x7f2974104d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40006/0 (socket says 172.21.15.100:40006) 2024-05-10T11:29:04.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.239+0000 7f29727fc700 1 -- 172.21.15.100:0/2359902836 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29741023a0 msgr2=0x7f2974104310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:04.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.239+0000 7f29727fc700 1 --2- 172.21.15.100:0/2359902836 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29741023a0 0x7f2974104310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:04.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.239+0000 7f29727fc700 1 -- 172.21.15.100:0/2359902836 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2974108410 msgr2=0x7f2974104d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:04.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.239+0000 7f29727fc700 1 --2- 172.21.15.100:0/2359902836 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2974108410 0x7f2974104d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:04.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.239+0000 7f29727fc700 1 -- 172.21.15.100:0/2359902836 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29741a7f50 con 0x7f2974102d60 2024-05-10T11:29:04.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.239+0000 7f2972ffd700 1 --2- 172.21.15.100:0/2359902836 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29741023a0 0x7f2974104310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:29:04.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.239+0000 7f29737fe700 1 --2- 172.21.15.100:0/2359902836 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2974108410 0x7f2974104d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:29:04.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.239+0000 7f29727fc700 1 --2- 172.21.15.100:0/2359902836 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2974102d60 0x7f2974104850 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f296800e9f0 tx=0x7f296800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:04.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.240+0000 7f295bfff700 1 -- 172.21.15.100:0/2359902836 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2968004220 con 0x7f2974102d60 2024-05-10T11:29:04.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.240+0000 7f2979584700 1 -- 172.21.15.100:0/2359902836 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29741a81e0 con 0x7f2974102d60 2024-05-10T11:29:04.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.240+0000 7f295bfff700 1 -- 172.21.15.100:0/2359902836 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f29680043c0 con 0x7f2974102d60 2024-05-10T11:29:04.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.240+0000 7f295bfff700 1 -- 172.21.15.100:0/2359902836 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296800ce30 con 0x7f2974102d60 2024-05-10T11:29:04.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.240+0000 7f2979584700 1 -- 172.21.15.100:0/2359902836 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29741a8720 con 0x7f2974102d60 2024-05-10T11:29:04.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.242+0000 7f295bfff700 1 -- 172.21.15.100:0/2359902836 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f2968004750 con 0x7f2974102d60 2024-05-10T11:29:04.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.243+0000 7f295bfff700 1 --2- 172.21.15.100:0/2359902836 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f295c0b2880 0x7f295c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:04.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.243+0000 7f2972ffd700 1 --2- 172.21.15.100:0/2359902836 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f295c0b2880 0x7f295c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:04.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.243+0000 7f295bfff700 1 -- 172.21.15.100:0/2359902836 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f29680e2520 con 0x7f2974102d60 2024-05-10T11:29:04.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.244+0000 7f2972ffd700 1 --2- 172.21.15.100:0/2359902836 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f295c0b2880 0x7f295c0b4d40 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f296400f4d0 tx=0x7f2964005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:04.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.244+0000 7f2983e17b80 1 Processor -- start 2024-05-10T11:29:04.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.244+0000 7f2983e17b80 1 -- start start 2024-05-10T11:29:04.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.244+0000 7f2983e17b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fe863e4d0 0x563fe863e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:04.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.244+0000 7f2983e17b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563fe863edf0 0x563fe86426f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:04.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.244+0000 7f2983e17b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563fe8642c80 0x563fe8643100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:04.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.244+0000 7f2983e17b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x563fe8590860 con 0x563fe8642c80 2024-05-10T11:29:04.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.244+0000 7f2983e17b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563fe852f7c0 con 0x563fe863e4d0 2024-05-10T11:29:04.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.244+0000 7f2983e17b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563fe858c820 con 0x563fe863edf0 2024-05-10T11:29:04.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.245+0000 7f2972ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fe863e4d0 0x563fe863e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:04.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.245+0000 7f2972ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fe863e4d0 0x563fe863e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:57494/0 (socket says 172.21.15.100:57494) 2024-05-10T11:29:04.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.245+0000 7f2972ffd700 1 -- 172.21.15.100:0/555649012 learned_addr learned my addr 172.21.15.100:0/555649012 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:29:04.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.245+0000 7f29727fc700 1 --2- 172.21.15.100:0/555649012 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563fe863edf0 0x563fe86426f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:04.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.245+0000 7f29737fe700 1 --2- 172.21.15.100:0/555649012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563fe8642c80 0x563fe8643100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:04.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.245+0000 7f2972ffd700 1 -- 172.21.15.100:0/555649012 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563fe863edf0 msgr2=0x563fe86426f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:04.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.245+0000 7f2972ffd700 1 --2- 172.21.15.100:0/555649012 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563fe863edf0 0x563fe86426f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:04.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.245+0000 7f2972ffd700 1 -- 172.21.15.100:0/555649012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563fe8642c80 msgr2=0x563fe8643100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:04.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.245+0000 7f2972ffd700 1 --2- 172.21.15.100:0/555649012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563fe8642c80 0x563fe8643100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:04.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.245+0000 7f2972ffd700 1 -- 172.21.15.100:0/555649012 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fe84dd8d0 con 0x563fe863e4d0 2024-05-10T11:29:04.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.245+0000 7f2972ffd700 1 --2- 172.21.15.100:0/555649012 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fe863e4d0 0x563fe863e8b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2964010750 tx=0x7f2964012000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8388ae4876b3ccec server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:04.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.245+0000 7f29597fa700 1 -- 172.21.15.100:0/555649012 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2964016070 con 0x563fe863e4d0 2024-05-10T11:29:04.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.245+0000 7f29597fa700 1 -- 172.21.15.100:0/555649012 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f296401b8b0 con 0x563fe863e4d0 2024-05-10T11:29:04.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.245+0000 7f29597fa700 1 -- 172.21.15.100:0/555649012 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296401c9b0 con 0x563fe863e4d0 2024-05-10T11:29:04.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.246+0000 7f2983e17b80 1 -- 172.21.15.100:0/555649012 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fe863e4d0 msgr2=0x563fe863e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:04.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.246+0000 7f2983e17b80 1 --2- 172.21.15.100:0/555649012 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fe863e4d0 0x563fe863e8b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2964010750 tx=0x7f2964012000 comp rx=0 tx=0).stop 2024-05-10T11:29:04.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.246+0000 7f2983e17b80 1 -- 172.21.15.100:0/555649012 shutdown_connections 2024-05-10T11:29:04.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.246+0000 7f2983e17b80 1 --2- 172.21.15.100:0/555649012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563fe8642c80 0x563fe8643100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:04.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.246+0000 7f2983e17b80 1 --2- 172.21.15.100:0/555649012 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fe863e4d0 0x563fe863e8b0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:04.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.246+0000 7f2983e17b80 1 --2- 172.21.15.100:0/555649012 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563fe863edf0 0x563fe86426f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:04.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.246+0000 7f2983e17b80 1 -- 172.21.15.100:0/555649012 >> 172.21.15.100:0/555649012 conn(0x563fe856a100 msgr2=0x563fe8638c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:04.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.246+0000 7f2983e17b80 1 -- 172.21.15.100:0/555649012 shutdown_connections 2024-05-10T11:29:04.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.246+0000 7f2983e17b80 1 -- 172.21.15.100:0/555649012 wait complete. 2024-05-10T11:29:04.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.246+0000 7f2983e17b80 1 -- start start 2024-05-10T11:29:04.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.247+0000 7f2983e17b80 1 Processor -- start 2024-05-10T11:29:04.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.247+0000 7f2983e17b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fe863e4d0 0x563fe864a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:04.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.247+0000 7f2983e17b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563fe863edf0 0x563fe8647480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:04.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.247+0000 7f2983e17b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563fe8642c80 0x563fe86479c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:04.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.247+0000 7f2983e17b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x563fe864ae90 con 0x563fe863edf0 2024-05-10T11:29:04.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.247+0000 7f2983e17b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563fe84f6c30 con 0x563fe863e4d0 2024-05-10T11:29:04.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.247+0000 7f2983e17b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563fe864b010 con 0x563fe8642c80 2024-05-10T11:29:04.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.247+0000 7f2983e17b80 10 client.0 _ll_register_callbacks cb 0x563fe8528920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:29:04.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.247+0000 7f29727fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563fe863edf0 0x563fe8647480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:04.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.247+0000 7f29727fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563fe863edf0 0x563fe8647480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40034/0 (socket says 172.21.15.100:40034) 2024-05-10T11:29:04.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.247+0000 7f2972ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fe863e4d0 0x563fe864a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:04.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.247+0000 7f29727fc700 1 -- 172.21.15.100:0/2637971586 learned_addr learned my addr 172.21.15.100:0/2637971586 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:29:04.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.247+0000 7f29737fe700 1 --2- 172.21.15.100:0/2637971586 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563fe8642c80 0x563fe86479c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:04.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.247+0000 7f29727fc700 1 -- 172.21.15.100:0/2637971586 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563fe8642c80 msgr2=0x563fe86479c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:04.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.247+0000 7f29727fc700 1 --2- 172.21.15.100:0/2637971586 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563fe8642c80 0x563fe86479c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:04.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.247+0000 7f29727fc700 1 -- 172.21.15.100:0/2637971586 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fe863e4d0 msgr2=0x563fe864a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:04.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.247+0000 7f29727fc700 1 --2- 172.21.15.100:0/2637971586 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fe863e4d0 0x563fe864a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:04.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.247+0000 7f29727fc700 1 -- 172.21.15.100:0/2637971586 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fe852c690 con 0x563fe863edf0 2024-05-10T11:29:04.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.247+0000 7f29737fe700 1 --2- 172.21.15.100:0/2637971586 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563fe8642c80 0x563fe86479c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:29:04.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.247+0000 7f2972ffd700 1 --2- 172.21.15.100:0/2637971586 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fe863e4d0 0x563fe864a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:29:04.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.247+0000 7f29727fc700 1 --2- 172.21.15.100:0/2637971586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563fe863edf0 0x563fe8647480 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f29680e0030 tx=0x7f29680a9340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fbaedad33a99363 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:04.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.247+0000 7f2956ff5700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:29:04.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.248+0000 7f2956ff5700 1 -- 172.21.15.100:0/2637971586 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296801c040 con 0x563fe863edf0 2024-05-10T11:29:04.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.248+0000 7f2956ff5700 1 -- 172.21.15.100:0/2637971586 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f29680e22c0 con 0x563fe863edf0 2024-05-10T11:29:04.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.248+0000 7f2983e17b80 1 -- 172.21.15.100:0/2637971586 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f296401b8b0 con 0x563fe863edf0 2024-05-10T11:29:04.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.248+0000 7f2956ff5700 1 -- 172.21.15.100:0/2637971586 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f29680a97e0 con 0x563fe863edf0 2024-05-10T11:29:04.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.248+0000 7f2983e17b80 10 client.17553 fetch_fsmap learned FSMap version 31 2024-05-10T11:29:04.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.248+0000 7f2983e17b80 1 -- 172.21.15.100:0/2637971586 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563fe8655700 con 0x563fe863edf0 2024-05-10T11:29:04.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.248+0000 7f2956ff5700 1 -- 172.21.15.100:0/2637971586 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f296401b8b0 con 0x563fe863edf0 2024-05-10T11:29:04.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.248+0000 7f2956ff5700 1 -- 172.21.15.100:0/2637971586 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f296806b020 con 0x563fe863edf0 2024-05-10T11:29:04.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.248+0000 7f2983e17b80 10 client.17553 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:29:04.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.248+0000 7f2983e17b80 10 client.17553 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:29:04.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.248+0000 7f2983e17b80 20 client.17553 populate_metadata read hostname 'smithi100' 2024-05-10T11:29:04.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.248+0000 7f2983e17b80 1 --2- 172.21.15.100:0/2637971586 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x563fe8657d40 0x563fe865a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:04.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.248+0000 7f2983e17b80 4 client.17553 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:29:04.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.248+0000 7f2983e17b80 1 -- 172.21.15.100:0/2637971586 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563fe860a650 con 0x563fe8657d40 2024-05-10T11:29:04.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.248+0000 7f2972ffd700 1 --2- 172.21.15.100:0/2637971586 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x563fe8657d40 0x563fe865a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:04.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.249+0000 7f2956ff5700 10 client.17553 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:29:04.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.249+0000 7f2972ffd700 1 --2- 172.21.15.100:0/2637971586 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x563fe8657d40 0x563fe865a130 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69c9d4b7dc3359f2 server_cookie=973977111e8c8493 in_seq=0 out_seq=0 2024-05-10T11:29:04.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.250+0000 7f2956ff5700 1 -- 172.21.15.100:0/2637971586 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563fe860a650 con 0x563fe8657d40 2024-05-10T11:29:04.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.250+0000 7f2956ff5700 10 client.17553 handle_command_reply: tid=0 2024-05-10T11:29:04.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.250+0000 7f2983e17b80 1 client.17553 shutdown 2024-05-10T11:29:04.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.250+0000 7f2983e17b80 1 -- 172.21.15.100:0/2637971586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563fe863edf0 msgr2=0x563fe8647480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:04.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.250+0000 7f2983e17b80 1 --2- 172.21.15.100:0/2637971586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563fe863edf0 0x563fe8647480 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f29680e0030 tx=0x7f29680a9340 comp rx=0 tx=0).stop 2024-05-10T11:29:04.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.250+0000 7f2983e17b80 1 -- 172.21.15.100:0/2637971586 shutdown_connections 2024-05-10T11:29:04.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.250+0000 7f2983e17b80 1 --2- 172.21.15.100:0/2637971586 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x563fe8657d40 0x563fe865a130 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:04.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.250+0000 7f2983e17b80 1 --2- 172.21.15.100:0/2637971586 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563fe8642c80 0x563fe86479c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:04.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.250+0000 7f2983e17b80 1 --2- 172.21.15.100:0/2637971586 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fe863e4d0 0x563fe864a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:04.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.250+0000 7f2983e17b80 1 --2- 172.21.15.100:0/2637971586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563fe863edf0 0x563fe8647480 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:04.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.250+0000 7f2956ff5700 0 client.17553 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:29:04.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.250+0000 7f2983e17b80 1 -- 172.21.15.100:0/2637971586 >> 172.21.15.100:0/2637971586 conn(0x563fe856a100 msgr2=0x563fe8633720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:04.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.250+0000 7f2983e17b80 1 -- 172.21.15.100:0/2637971586 shutdown_connections 2024-05-10T11:29:04.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.250+0000 7f2983e17b80 1 -- 172.21.15.100:0/2637971586 wait complete. 2024-05-10T11:29:04.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.250+0000 7f2983e17b80 20 client.17553 trim_cache size 0 max 16384 2024-05-10T11:29:04.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.266+0000 7f2983e17b80 1 Processor -- start 2024-05-10T11:29:04.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.266+0000 7f2983e17b80 1 -- start start 2024-05-10T11:29:04.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.267+0000 7f2983e17b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563fe856a100 0x563fe8653f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:04.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.267+0000 7f2983e17b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563fe863e4d0 0x563fe86335a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:04.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.267+0000 7f2983e17b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fe863edf0 0x563fe865f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:04.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.267+0000 7f2983e17b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f29741b3a70 con 0x563fe863e4d0 2024-05-10T11:29:04.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.267+0000 7f2983e17b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f29741b1300 con 0x563fe863edf0 2024-05-10T11:29:04.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.267+0000 7f2983e17b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f29741af150 con 0x563fe856a100 2024-05-10T11:29:04.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.267+0000 7f29727fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563fe863e4d0 0x563fe86335a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:04.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.267+0000 7f29727fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563fe863e4d0 0x563fe86335a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40048/0 (socket says 172.21.15.100:40048) 2024-05-10T11:29:04.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.267+0000 7f29727fc700 1 -- 172.21.15.100:0/2391456997 learned_addr learned my addr 172.21.15.100:0/2391456997 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:29:04.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.267+0000 7f2972ffd700 1 --2- 172.21.15.100:0/2391456997 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563fe856a100 0x563fe8653f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:04.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.267+0000 7f29737fe700 1 --2- 172.21.15.100:0/2391456997 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fe863edf0 0x563fe865f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:04.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.267+0000 7f29727fc700 1 -- 172.21.15.100:0/2391456997 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563fe856a100 msgr2=0x563fe8653f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:04.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.267+0000 7f29727fc700 1 --2- 172.21.15.100:0/2391456997 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563fe856a100 0x563fe8653f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:04.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.267+0000 7f29727fc700 1 -- 172.21.15.100:0/2391456997 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fe863edf0 msgr2=0x563fe865f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:04.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.267+0000 7f29727fc700 1 --2- 172.21.15.100:0/2391456997 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fe863edf0 0x563fe865f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:04.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.267+0000 7f29727fc700 1 -- 172.21.15.100:0/2391456997 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fe863eb00 con 0x563fe863e4d0 2024-05-10T11:29:04.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.267+0000 7f29727fc700 1 --2- 172.21.15.100:0/2391456997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563fe863e4d0 0x563fe86335a0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f29680a9490 tx=0x7f29680a9080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fffdfbeef762f28 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:04.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.267+0000 7f2958ff9700 1 -- 172.21.15.100:0/2391456997 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29680c6030 con 0x563fe863e4d0 2024-05-10T11:29:04.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.267+0000 7f2958ff9700 1 -- 172.21.15.100:0/2391456997 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f29680e1070 con 0x563fe863e4d0 2024-05-10T11:29:04.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.267+0000 7f2958ff9700 1 -- 172.21.15.100:0/2391456997 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296801c070 con 0x563fe863e4d0 2024-05-10T11:29:04.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.268+0000 7f2983e17b80 1 -- 172.21.15.100:0/2391456997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563fe863e4d0 msgr2=0x563fe86335a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:04.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.268+0000 7f2983e17b80 1 --2- 172.21.15.100:0/2391456997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563fe863e4d0 0x563fe86335a0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f29680a9490 tx=0x7f29680a9080 comp rx=0 tx=0).stop 2024-05-10T11:29:04.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.268+0000 7f2983e17b80 1 -- 172.21.15.100:0/2391456997 shutdown_connections 2024-05-10T11:29:04.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.268+0000 7f2983e17b80 1 --2- 172.21.15.100:0/2391456997 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fe863edf0 0x563fe865f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:04.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.268+0000 7f2983e17b80 1 --2- 172.21.15.100:0/2391456997 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563fe856a100 0x563fe8653f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:04.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.268+0000 7f2983e17b80 1 --2- 172.21.15.100:0/2391456997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563fe863e4d0 0x563fe86335a0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:04.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.268+0000 7f2983e17b80 1 -- 172.21.15.100:0/2391456997 >> 172.21.15.100:0/2391456997 conn(0x563fe8647040 msgr2=0x563fe8632e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:04.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.268+0000 7f2983e17b80 1 -- 172.21.15.100:0/2391456997 shutdown_connections 2024-05-10T11:29:04.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.268+0000 7f2983e17b80 1 -- 172.21.15.100:0/2391456997 wait complete. 2024-05-10T11:29:04.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.268+0000 7f2983e17b80 1 -- start start 2024-05-10T11:29:04.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.269+0000 7f2983e17b80 1 Processor -- start 2024-05-10T11:29:04.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.269+0000 7f2983e17b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fe856a100 0x563fe8650730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:04.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.269+0000 7f2983e17b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563fe863e4d0 0x563fe864dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:04.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.269+0000 7f2983e17b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563fe863edf0 0x563fe864e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:04.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.269+0000 7f2983e17b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f29741af7c0 con 0x563fe863edf0 2024-05-10T11:29:04.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.269+0000 7f2983e17b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f29741b0c30 con 0x563fe856a100 2024-05-10T11:29:04.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.269+0000 7f2983e17b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f29741afe90 con 0x563fe863e4d0 2024-05-10T11:29:04.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.269+0000 7f2983e17b80 10 client.0 _ll_register_callbacks cb 0x563fe8528920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:29:04.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.269+0000 7f2972ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fe856a100 0x563fe8650730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:04.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.269+0000 7f29737fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563fe863edf0 0x563fe864e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:04.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.269+0000 7f29727fc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563fe863e4d0 0x563fe864dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:04.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.269+0000 7f2972ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fe856a100 0x563fe8650730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:57528/0 (socket says 172.21.15.100:57528) 2024-05-10T11:29:04.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.269+0000 7f2972ffd700 1 -- 172.21.15.100:0/858288608 learned_addr learned my addr 172.21.15.100:0/858288608 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:29:04.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.269+0000 7f2972ffd700 1 -- 172.21.15.100:0/858288608 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563fe863e4d0 msgr2=0x563fe864dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:04.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.269+0000 7f2972ffd700 1 --2- 172.21.15.100:0/858288608 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563fe863e4d0 0x563fe864dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:04.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.269+0000 7f2972ffd700 1 -- 172.21.15.100:0/858288608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563fe863edf0 msgr2=0x563fe864e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:04.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.269+0000 7f2972ffd700 1 --2- 172.21.15.100:0/858288608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563fe863edf0 0x563fe864e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:04.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.269+0000 7f2972ffd700 1 -- 172.21.15.100:0/858288608 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fe864e850 con 0x563fe856a100 2024-05-10T11:29:04.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.269+0000 7f29737fe700 1 --2- 172.21.15.100:0/858288608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563fe863edf0 0x563fe864e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:29:04.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.270+0000 7f2972ffd700 1 --2- 172.21.15.100:0/858288608 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fe856a100 0x563fe8650730 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2964010750 tx=0x7f2964005d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=493dfaf6e3584220 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:04.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.270+0000 7f2956ff5700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:29:04.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.270+0000 7f2956ff5700 1 -- 172.21.15.100:0/858288608 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2964016040 con 0x563fe856a100 2024-05-10T11:29:04.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.270+0000 7f2956ff5700 1 -- 172.21.15.100:0/858288608 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f296401bb40 con 0x563fe856a100 2024-05-10T11:29:04.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.270+0000 7f2983e17b80 1 -- 172.21.15.100:0/858288608 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29680e1070 con 0x563fe856a100 2024-05-10T11:29:04.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.270+0000 7f2956ff5700 1 -- 172.21.15.100:0/858288608 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2964043050 con 0x563fe856a100 2024-05-10T11:29:04.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.270+0000 7f2983e17b80 10 client.26930 fetch_fsmap learned FSMap version 31 2024-05-10T11:29:04.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.270+0000 7f2983e17b80 1 -- 172.21.15.100:0/858288608 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563fe86c24f0 con 0x563fe856a100 2024-05-10T11:29:04.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.270+0000 7f2956ff5700 1 -- 172.21.15.100:0/858288608 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f29680e1070 con 0x563fe856a100 2024-05-10T11:29:04.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.270+0000 7f2956ff5700 1 -- 172.21.15.100:0/858288608 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f2964025470 con 0x563fe856a100 2024-05-10T11:29:04.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.270+0000 7f2983e17b80 10 client.26930 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:29:04.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.270+0000 7f2983e17b80 10 client.26930 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:29:04.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.270+0000 7f2983e17b80 20 client.26930 populate_metadata read hostname 'smithi100' 2024-05-10T11:29:04.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.270+0000 7f2983e17b80 1 --2- 172.21.15.100:0/858288608 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x563fe8642c80 0x563fe86c4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:04.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.270+0000 7f2983e17b80 4 client.26930 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:29:04.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.270+0000 7f2983e17b80 1 -- 172.21.15.100:0/858288608 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563fe859e240 con 0x563fe8642c80 2024-05-10T11:29:04.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.271+0000 7f29727fc700 1 --2- 172.21.15.100:0/858288608 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x563fe8642c80 0x563fe86c4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:04.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.271+0000 7f29727fc700 1 --2- 172.21.15.100:0/858288608 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x563fe8642c80 0x563fe86c4990 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0f3b44edbb94e89 server_cookie=da3500e032c9db0 in_seq=0 out_seq=0 2024-05-10T11:29:04.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.271+0000 7f2956ff5700 10 client.26930 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:29:04.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.271+0000 7f2956ff5700 1 -- 172.21.15.100:0/858288608 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563fe859e240 con 0x563fe8642c80 2024-05-10T11:29:04.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.271+0000 7f2956ff5700 10 client.26930 handle_command_reply: tid=0 2024-05-10T11:29:04.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.271+0000 7f2983e17b80 1 client.26930 shutdown 2024-05-10T11:29:04.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.271+0000 7f2983e17b80 1 -- 172.21.15.100:0/858288608 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fe856a100 msgr2=0x563fe8650730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:04.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.271+0000 7f2983e17b80 1 --2- 172.21.15.100:0/858288608 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fe856a100 0x563fe8650730 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2964010750 tx=0x7f2964005d10 comp rx=0 tx=0).stop 2024-05-10T11:29:04.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.271+0000 7f2983e17b80 1 -- 172.21.15.100:0/858288608 shutdown_connections 2024-05-10T11:29:04.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.271+0000 7f2983e17b80 1 --2- 172.21.15.100:0/858288608 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x563fe8642c80 0x563fe86c4990 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:04.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.271+0000 7f2983e17b80 1 --2- 172.21.15.100:0/858288608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563fe863edf0 0x563fe864e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:04.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.271+0000 7f2983e17b80 1 --2- 172.21.15.100:0/858288608 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fe856a100 0x563fe8650730 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:04.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.271+0000 7f2983e17b80 1 --2- 172.21.15.100:0/858288608 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563fe863e4d0 0x563fe864dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:04.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.271+0000 7f2956ff5700 0 client.26930 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:29:04.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.271+0000 7f2983e17b80 1 -- 172.21.15.100:0/858288608 >> 172.21.15.100:0/858288608 conn(0x563fe8647040 msgr2=0x563fe8633720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:04.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.271+0000 7f2983e17b80 1 -- 172.21.15.100:0/858288608 shutdown_connections 2024-05-10T11:29:04.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.271+0000 7f2983e17b80 1 -- 172.21.15.100:0/858288608 wait complete. 2024-05-10T11:29:04.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.271+0000 7f2983e17b80 20 client.26930 trim_cache size 0 max 16384 2024-05-10T11:29:04.277 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:29:04.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.272+0000 7f2979584700 1 -- 172.21.15.100:0/2359902836 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f295c0b2880 msgr2=0x7f295c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:04.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.272+0000 7f2979584700 1 --2- 172.21.15.100:0/2359902836 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f295c0b2880 0x7f295c0b4d40 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f296400f4d0 tx=0x7f2964005f90 comp rx=0 tx=0).stop 2024-05-10T11:29:04.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.272+0000 7f2979584700 1 -- 172.21.15.100:0/2359902836 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2974102d60 msgr2=0x7f2974104850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:04.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.272+0000 7f2979584700 1 --2- 172.21.15.100:0/2359902836 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2974102d60 0x7f2974104850 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f296800e9f0 tx=0x7f296800ed00 comp rx=0 tx=0).stop 2024-05-10T11:29:04.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.272+0000 7f2979584700 1 -- 172.21.15.100:0/2359902836 shutdown_connections 2024-05-10T11:29:04.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.272+0000 7f2979584700 1 --2- 172.21.15.100:0/2359902836 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f295c0b2880 0x7f295c0b4d40 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:04.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.272+0000 7f2979584700 1 --2- 172.21.15.100:0/2359902836 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2974108410 0x7f2974104d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:04.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.272+0000 7f2979584700 1 --2- 172.21.15.100:0/2359902836 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29741023a0 0x7f2974104310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:04.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.272+0000 7f2979584700 1 --2- 172.21.15.100:0/2359902836 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2974102d60 0x7f2974104850 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:04.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.272+0000 7f2979584700 1 -- 172.21.15.100:0/2359902836 >> 172.21.15.100:0/2359902836 conn(0x7f2974079350 msgr2=0x7f2974106590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:04.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.272+0000 7f2979584700 1 -- 172.21.15.100:0/2359902836 shutdown_connections 2024-05-10T11:29:04.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:04.272+0000 7f2979584700 1 -- 172.21.15.100:0/2359902836 wait complete. 2024-05-10T11:29:04.981 INFO:tasks.workunit.client.0.smithi100.stdout: 1 276451 2.73 MB/sec execute 419 sec latency 17.882 ms 2024-05-10T11:29:05.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:29:05 smithi188 ceph-mon[34075]: pgmap v741: 129 pgs: 129 active+clean; 112 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 1.3 MiB/s wr, 161 op/s 2024-05-10T11:29:05.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:29:05 smithi100 ceph-mon[30296]: pgmap v741: 129 pgs: 129 active+clean; 112 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 1.3 MiB/s wr, 161 op/s 2024-05-10T11:29:05.980 INFO:tasks.workunit.client.0.smithi100.stdout: 1 276905 2.73 MB/sec execute 420 sec latency 19.867 ms 2024-05-10T11:29:06.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:29:05 smithi178 ceph-mon[33359]: pgmap v741: 129 pgs: 129 active+clean; 112 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 1.3 MiB/s wr, 161 op/s 2024-05-10T11:29:06.024 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T11:29:06.024 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T11:29:06.292 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:29:06.855 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.852+0000 7fb154072640 1 -- 172.21.15.100:0/2596883687 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb14c101910 msgr2=0x7fb14c101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:06.855 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.852+0000 7fb154072640 1 --2- 172.21.15.100:0/2596883687 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb14c101910 0x7fb14c101d30 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fb148009900 tx=0x7fb14802a410 comp rx=0 tx=0).stop 2024-05-10T11:29:06.855 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.853+0000 7fb154072640 1 -- 172.21.15.100:0/2596883687 shutdown_connections 2024-05-10T11:29:06.855 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.853+0000 7fb154072640 1 --2- 172.21.15.100:0/2596883687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb14c1034c0 0x7fb14c109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:06.855 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.853+0000 7fb154072640 1 --2- 172.21.15.100:0/2596883687 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb14c102b00 0x7fb14c102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:06.855 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.853+0000 7fb154072640 1 --2- 172.21.15.100:0/2596883687 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb14c101910 0x7fb14c101d30 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:06.855 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.853+0000 7fb154072640 1 -- 172.21.15.100:0/2596883687 >> 172.21.15.100:0/2596883687 conn(0x7fb14c0fce80 msgr2=0x7fb14c0ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:06.855 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.853+0000 7fb154072640 1 -- 172.21.15.100:0/2596883687 shutdown_connections 2024-05-10T11:29:06.855 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.853+0000 7fb154072640 1 -- 172.21.15.100:0/2596883687 wait complete. 2024-05-10T11:29:06.855 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.853+0000 7fb154072640 1 Processor -- start 2024-05-10T11:29:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.853+0000 7fb154072640 1 -- start start 2024-05-10T11:29:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.853+0000 7fb154072640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb14c101910 0x7fb14c198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.854+0000 7fb154072640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb14c102b00 0x7fb14c1990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.854+0000 7fb154072640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb14c1034c0 0x7fb14c1a00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.854+0000 7fb154072640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb14c10d2f0 con 0x7fb14c101910 2024-05-10T11:29:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.854+0000 7fb154072640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb14c10d170 con 0x7fb14c1034c0 2024-05-10T11:29:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.854+0000 7fb154072640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb14c10d470 con 0x7fb14c102b00 2024-05-10T11:29:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.854+0000 7fb151e0e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb14c101910 0x7fb14c198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.854+0000 7fb151e0e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb14c101910 0x7fb14c198b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41542/0 (socket says 172.21.15.100:41542) 2024-05-10T11:29:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.854+0000 7fb151e0e640 1 -- 172.21.15.100:0/428970399 learned_addr learned my addr 172.21.15.100:0/428970399 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:29:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.854+0000 7fb15160d640 1 --2- 172.21.15.100:0/428970399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb14c102b00 0x7fb14c1990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.854+0000 7fb15260f640 1 --2- 172.21.15.100:0/428970399 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb14c1034c0 0x7fb14c1a00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.854+0000 7fb15260f640 1 -- 172.21.15.100:0/428970399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb14c102b00 msgr2=0x7fb14c1990c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.854+0000 7fb15260f640 1 --2- 172.21.15.100:0/428970399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb14c102b00 0x7fb14c1990c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:06.857 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.854+0000 7fb15260f640 1 -- 172.21.15.100:0/428970399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb14c101910 msgr2=0x7fb14c198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:06.857 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.854+0000 7fb15260f640 1 --2- 172.21.15.100:0/428970399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb14c101910 0x7fb14c198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:06.857 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.854+0000 7fb15260f640 1 -- 172.21.15.100:0/428970399 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb14c1a0750 con 0x7fb14c1034c0 2024-05-10T11:29:06.857 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.854+0000 7fb151e0e640 1 --2- 172.21.15.100:0/428970399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb14c101910 0x7fb14c198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:29:06.857 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.854+0000 7fb15260f640 1 --2- 172.21.15.100:0/428970399 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb14c1034c0 0x7fb14c1a00e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb144009fd0 tx=0x7fb14400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:06.857 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.854+0000 7fb142ffd640 1 -- 172.21.15.100:0/428970399 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb14400ce20 con 0x7fb14c1034c0 2024-05-10T11:29:06.857 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.855+0000 7fb154072640 1 -- 172.21.15.100:0/428970399 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb14c1a0a40 con 0x7fb14c1034c0 2024-05-10T11:29:06.857 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.855+0000 7fb142ffd640 1 -- 172.21.15.100:0/428970399 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb144005020 con 0x7fb14c1034c0 2024-05-10T11:29:06.857 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.855+0000 7fb142ffd640 1 -- 172.21.15.100:0/428970399 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb144010690 con 0x7fb14c1034c0 2024-05-10T11:29:06.857 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.855+0000 7fb154072640 1 -- 172.21.15.100:0/428970399 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb14c1a0f90 con 0x7fb14c1034c0 2024-05-10T11:29:06.859 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.857+0000 7fb142ffd640 1 -- 172.21.15.100:0/428970399 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fb144004a40 con 0x7fb14c1034c0 2024-05-10T11:29:06.859 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.857+0000 7fb156d8c740 1 Processor -- start 2024-05-10T11:29:06.859 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.857+0000 7fb156d8c740 1 -- start start 2024-05-10T11:29:06.859 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.858+0000 7fb156d8c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564b0d03ddb0 0x564b0d03e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:06.859 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.858+0000 7fb156d8c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564b0d03e7a0 0x564b0d0432a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:06.859 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.858+0000 7fb156d8c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564b0d043910 0x564b0d045da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:06.859 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.858+0000 7fb156d8c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x564b0cfa25d0 con 0x564b0d03ddb0 2024-05-10T11:29:06.859 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.858+0000 7fb156d8c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x564b0cf5b890 con 0x564b0d03e7a0 2024-05-10T11:29:06.859 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.858+0000 7fb156d8c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x564b0cf9e590 con 0x564b0d043910 2024-05-10T11:29:06.859 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.858+0000 7fb151e0e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564b0d03e7a0 0x564b0d0432a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:06.859 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.858+0000 7fb15160d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564b0d03ddb0 0x564b0d03e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:06.859 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.858+0000 7fb151e0e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564b0d03e7a0 0x564b0d0432a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:52112/0 (socket says 172.21.15.100:52112) 2024-05-10T11:29:06.860 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.858+0000 7fb151e0e640 1 -- 172.21.15.100:0/1964612521 learned_addr learned my addr 172.21.15.100:0/1964612521 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:29:06.860 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.858+0000 7fb15160d640 1 --2- 172.21.15.100:0/1964612521 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564b0d043910 0x564b0d045da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:06.860 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.858+0000 7fb151e0e640 1 -- 172.21.15.100:0/1964612521 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564b0d043910 msgr2=0x564b0d045da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:06.860 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.858+0000 7fb151e0e640 1 --2- 172.21.15.100:0/1964612521 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564b0d043910 0x564b0d045da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:06.860 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.858+0000 7fb151e0e640 1 -- 172.21.15.100:0/1964612521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564b0d03ddb0 msgr2=0x564b0d03e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:06.860 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.858+0000 7fb151e0e640 1 --2- 172.21.15.100:0/1964612521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564b0d03ddb0 0x564b0d03e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:06.860 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.858+0000 7fb151e0e640 1 -- 172.21.15.100:0/1964612521 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b0ccb4210 con 0x564b0d03e7a0 2024-05-10T11:29:06.860 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.858+0000 7fb151e0e640 1 --2- 172.21.15.100:0/1964612521 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564b0d03e7a0 0x564b0d0432a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb148005df0 tx=0x7fb148005e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9b48fa793b70898 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:06.860 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.858+0000 7fb142ffd640 1 --2- 172.21.15.100:0/428970399 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fb1380b2920 0x7fb1380b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:06.860 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.858+0000 7fb137fff640 1 -- 172.21.15.100:0/1964612521 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb148035030 con 0x564b0d03e7a0 2024-05-10T11:29:06.860 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.858+0000 7fb137fff640 1 -- 172.21.15.100:0/1964612521 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb148040070 con 0x564b0d03e7a0 2024-05-10T11:29:06.861 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.859+0000 7fb137fff640 1 -- 172.21.15.100:0/1964612521 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb148003ca0 con 0x564b0d03e7a0 2024-05-10T11:29:06.861 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.859+0000 7fb15160d640 1 --2- 172.21.15.100:0/428970399 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fb1380b2920 0x7fb1380b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:06.861 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.859+0000 7fb142ffd640 1 -- 172.21.15.100:0/428970399 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fb1440e2bf0 con 0x7fb14c1034c0 2024-05-10T11:29:06.864 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.859+0000 7fb156d8c740 1 -- 172.21.15.100:0/1964612521 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564b0d03e7a0 msgr2=0x564b0d0432a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:06.864 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.859+0000 7fb156d8c740 1 --2- 172.21.15.100:0/1964612521 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564b0d03e7a0 0x564b0d0432a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb148005df0 tx=0x7fb148005e20 comp rx=0 tx=0).stop 2024-05-10T11:29:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.859+0000 7fb15160d640 1 --2- 172.21.15.100:0/428970399 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fb1380b2920 0x7fb1380b4de0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x564b0d03f000 tx=0x7fb13c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.859+0000 7fb156d8c740 1 -- 172.21.15.100:0/1964612521 shutdown_connections 2024-05-10T11:29:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.859+0000 7fb156d8c740 1 --2- 172.21.15.100:0/1964612521 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564b0d043910 0x564b0d045da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.859+0000 7fb156d8c740 1 --2- 172.21.15.100:0/1964612521 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564b0d03e7a0 0x564b0d0432a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.859+0000 7fb156d8c740 1 --2- 172.21.15.100:0/1964612521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564b0d03ddb0 0x564b0d03e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.859+0000 7fb156d8c740 1 -- 172.21.15.100:0/1964612521 >> 172.21.15.100:0/1964612521 conn(0x564b0d039420 msgr2=0x564b0d039820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.860+0000 7fb156d8c740 1 -- 172.21.15.100:0/1964612521 shutdown_connections 2024-05-10T11:29:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.860+0000 7fb156d8c740 1 -- 172.21.15.100:0/1964612521 wait complete. 2024-05-10T11:29:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.860+0000 7fb156d8c740 1 -- start start 2024-05-10T11:29:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.860+0000 7fb156d8c740 1 Processor -- start 2024-05-10T11:29:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.860+0000 7fb156d8c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564b0d03ddb0 0x564b0d04c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:06.866 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.861+0000 7fb156d8c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564b0d03e7a0 0x564b0d04c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:06.866 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.861+0000 7fb156d8c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564b0d043910 0x564b0d053960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:06.866 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.861+0000 7fb156d8c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x564b0d04ad10 con 0x564b0d03ddb0 2024-05-10T11:29:06.866 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.861+0000 7fb156d8c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x564b0cf5f370 con 0x564b0d03e7a0 2024-05-10T11:29:06.866 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.861+0000 7fb156d8c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x564b0d04ae90 con 0x564b0d043910 2024-05-10T11:29:06.866 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.861+0000 7fb156d8c740 10 client.0 _ll_register_callbacks cb 0x564b0ccb3a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:29:06.866 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.861+0000 7fb151e0e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564b0d03ddb0 0x564b0d04c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:06.866 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.861+0000 7fb15260f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564b0d043910 0x564b0d053960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:06.866 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.861+0000 7fb151e0e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564b0d03ddb0 0x564b0d04c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41568/0 (socket says 172.21.15.100:41568) 2024-05-10T11:29:06.866 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.861+0000 7fb151e0e640 1 -- 172.21.15.100:0/3499033043 learned_addr learned my addr 172.21.15.100:0/3499033043 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:29:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.861+0000 7fb15160d640 1 --2- 172.21.15.100:0/3499033043 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564b0d03e7a0 0x564b0d04c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.861+0000 7fb15260f640 1 -- 172.21.15.100:0/3499033043 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564b0d03e7a0 msgr2=0x564b0d04c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.861+0000 7fb15260f640 1 --2- 172.21.15.100:0/3499033043 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564b0d03e7a0 0x564b0d04c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.861+0000 7fb15260f640 1 -- 172.21.15.100:0/3499033043 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564b0d03ddb0 msgr2=0x564b0d04c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.861+0000 7fb15260f640 1 --2- 172.21.15.100:0/3499033043 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564b0d03ddb0 0x564b0d04c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.861+0000 7fb15260f640 1 -- 172.21.15.100:0/3499033043 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b0cd80620 con 0x564b0d043910 2024-05-10T11:29:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.861+0000 7fb151e0e640 1 --2- 172.21.15.100:0/3499033043 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564b0d03ddb0 0x564b0d04c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:29:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.861+0000 7fb15260f640 1 --2- 172.21.15.100:0/3499033043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564b0d043910 0x564b0d053960 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb1440a7120 tx=0x7fb1440aa670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89d7b90e3c1000fb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.861+0000 7fb1357fa640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:29:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.861+0000 7fb1357fa640 1 -- 172.21.15.100:0/3499033043 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1440b2d10 con 0x564b0d043910 2024-05-10T11:29:06.868 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.861+0000 7fb156d8c740 1 -- 172.21.15.100:0/3499033043 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb148040070 con 0x564b0d043910 2024-05-10T11:29:06.868 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.861+0000 7fb1357fa640 1 -- 172.21.15.100:0/3499033043 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fb1440054d0 con 0x564b0d043910 2024-05-10T11:29:06.868 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.862+0000 7fb1357fa640 1 -- 172.21.15.100:0/3499033043 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb1440051d0 con 0x564b0d043910 2024-05-10T11:29:06.868 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.862+0000 7fb156d8c740 10 client.27022 fetch_fsmap learned FSMap version 31 2024-05-10T11:29:06.868 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.862+0000 7fb156d8c740 1 -- 172.21.15.100:0/3499033043 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564b0d056a10 con 0x564b0d043910 2024-05-10T11:29:06.868 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.862+0000 7fb1357fa640 1 -- 172.21.15.100:0/3499033043 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb148040070 con 0x564b0d043910 2024-05-10T11:29:06.868 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.862+0000 7fb1357fa640 1 -- 172.21.15.100:0/3499033043 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fb1440a5820 con 0x564b0d043910 2024-05-10T11:29:06.868 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.862+0000 7fb156d8c740 10 client.27022 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:29:06.868 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.862+0000 7fb156d8c740 10 client.27022 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:29:06.868 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.862+0000 7fb156d8c740 20 client.27022 populate_metadata read hostname 'smithi100' 2024-05-10T11:29:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.862+0000 7fb156d8c740 1 --2- 172.21.15.100:0/3499033043 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x564b0d0581a0 0x564b0d05a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.862+0000 7fb156d8c740 4 client.27022 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:29:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.862+0000 7fb156d8c740 1 -- 172.21.15.100:0/3499033043 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b0cfb01e0 con 0x564b0d0581a0 2024-05-10T11:29:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.862+0000 7fb151e0e640 1 --2- 172.21.15.100:0/3499033043 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x564b0d0581a0 0x564b0d05a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.863+0000 7fb1357fa640 10 client.27022 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:29:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.863+0000 7fb151e0e640 1 --2- 172.21.15.100:0/3499033043 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x564b0d0581a0 0x564b0d05a5d0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6016ba4767e5d854 server_cookie=fde42834a48a94e3 in_seq=0 out_seq=0 2024-05-10T11:29:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.863+0000 7fb1357fa640 1 -- 172.21.15.100:0/3499033043 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564b0cfb01e0 con 0x564b0d0581a0 2024-05-10T11:29:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.863+0000 7fb1357fa640 10 client.27022 handle_command_reply: tid=0 2024-05-10T11:29:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.864+0000 7fb156d8c740 1 client.27022 shutdown 2024-05-10T11:29:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.864+0000 7fb156d8c740 1 -- 172.21.15.100:0/3499033043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564b0d043910 msgr2=0x564b0d053960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.864+0000 7fb156d8c740 1 --2- 172.21.15.100:0/3499033043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564b0d043910 0x564b0d053960 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb1440a7120 tx=0x7fb1440aa670 comp rx=0 tx=0).stop 2024-05-10T11:29:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.864+0000 7fb156d8c740 1 -- 172.21.15.100:0/3499033043 shutdown_connections 2024-05-10T11:29:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.864+0000 7fb156d8c740 1 --2- 172.21.15.100:0/3499033043 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x564b0d0581a0 0x564b0d05a5d0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.864+0000 7fb156d8c740 1 --2- 172.21.15.100:0/3499033043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564b0d043910 0x564b0d053960 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.864+0000 7fb156d8c740 1 --2- 172.21.15.100:0/3499033043 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564b0d03e7a0 0x564b0d04c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.864+0000 7fb1357fa640 0 client.27022 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:29:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.864+0000 7fb156d8c740 1 --2- 172.21.15.100:0/3499033043 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564b0d03ddb0 0x564b0d04c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.864+0000 7fb156d8c740 1 -- 172.21.15.100:0/3499033043 >> 172.21.15.100:0/3499033043 conn(0x564b0d039420 msgr2=0x564b0d038580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.864+0000 7fb156d8c740 1 -- 172.21.15.100:0/3499033043 shutdown_connections 2024-05-10T11:29:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.864+0000 7fb156d8c740 1 -- 172.21.15.100:0/3499033043 wait complete. 2024-05-10T11:29:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.864+0000 7fb156d8c740 20 client.27022 trim_cache size 0 max 16384 2024-05-10T11:29:06.883 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.882+0000 7fb156d8c740 1 Processor -- start 2024-05-10T11:29:06.883 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.882+0000 7fb156d8c740 1 -- start start 2024-05-10T11:29:06.883 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.882+0000 7fb156d8c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564b0d03ddb0 0x564b0d042f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:06.883 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.882+0000 7fb156d8c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564b0d03e7a0 0x564b0d03f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:06.884 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.882+0000 7fb156d8c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564b0d043910 0x564b0d03fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:06.884 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.882+0000 7fb156d8c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb14c1ab9c0 con 0x564b0d03ddb0 2024-05-10T11:29:06.884 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.882+0000 7fb156d8c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb14c1a9290 con 0x564b0d03e7a0 2024-05-10T11:29:06.884 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.882+0000 7fb156d8c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb14c1a70e0 con 0x564b0d043910 2024-05-10T11:29:06.884 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.882+0000 7fb151e0e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564b0d03ddb0 0x564b0d042f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:06.884 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.882+0000 7fb15160d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564b0d03e7a0 0x564b0d03f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:06.884 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.882+0000 7fb151e0e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564b0d03ddb0 0x564b0d042f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41574/0 (socket says 172.21.15.100:41574) 2024-05-10T11:29:06.884 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.882+0000 7fb15260f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564b0d043910 0x564b0d03fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:06.884 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.882+0000 7fb151e0e640 1 -- 172.21.15.100:0/1106864635 learned_addr learned my addr 172.21.15.100:0/1106864635 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:29:06.884 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.883+0000 7fb15260f640 1 -- 172.21.15.100:0/1106864635 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564b0d03e7a0 msgr2=0x564b0d03f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:06.885 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.883+0000 7fb15260f640 1 --2- 172.21.15.100:0/1106864635 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564b0d03e7a0 0x564b0d03f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:06.885 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.883+0000 7fb15260f640 1 -- 172.21.15.100:0/1106864635 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564b0d03ddb0 msgr2=0x564b0d042f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:06.885 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.883+0000 7fb15260f640 1 --2- 172.21.15.100:0/1106864635 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564b0d03ddb0 0x564b0d042f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:06.885 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.883+0000 7fb15260f640 1 -- 172.21.15.100:0/1106864635 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b0d05ac50 con 0x564b0d043910 2024-05-10T11:29:06.885 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.883+0000 7fb15260f640 1 --2- 172.21.15.100:0/1106864635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564b0d043910 0x564b0d03fec0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb1440e0700 tx=0x7fb1440053b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42b90edcfc1aadec server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:06.885 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.883+0000 7fb1377fe640 1 -- 172.21.15.100:0/1106864635 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1440a9c80 con 0x564b0d043910 2024-05-10T11:29:06.885 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.883+0000 7fb1377fe640 1 -- 172.21.15.100:0/1106864635 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb1440b2b00 con 0x564b0d043910 2024-05-10T11:29:06.885 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.883+0000 7fb1377fe640 1 -- 172.21.15.100:0/1106864635 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb144004d30 con 0x564b0d043910 2024-05-10T11:29:06.888 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.884+0000 7fb156d8c740 1 -- 172.21.15.100:0/1106864635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564b0d043910 msgr2=0x564b0d03fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:06.889 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.884+0000 7fb156d8c740 1 --2- 172.21.15.100:0/1106864635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564b0d043910 0x564b0d03fec0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb1440e0700 tx=0x7fb1440053b0 comp rx=0 tx=0).stop 2024-05-10T11:29:06.889 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.884+0000 7fb156d8c740 1 -- 172.21.15.100:0/1106864635 shutdown_connections 2024-05-10T11:29:06.889 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.884+0000 7fb156d8c740 1 --2- 172.21.15.100:0/1106864635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564b0d043910 0x564b0d03fec0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:06.889 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.884+0000 7fb156d8c740 1 --2- 172.21.15.100:0/1106864635 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564b0d03e7a0 0x564b0d03f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:06.889 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.884+0000 7fb156d8c740 1 --2- 172.21.15.100:0/1106864635 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564b0d03ddb0 0x564b0d042f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:06.889 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.884+0000 7fb156d8c740 1 -- 172.21.15.100:0/1106864635 >> 172.21.15.100:0/1106864635 conn(0x564b0d039420 msgr2=0x564b0d0343e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:06.889 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.884+0000 7fb156d8c740 1 -- 172.21.15.100:0/1106864635 shutdown_connections 2024-05-10T11:29:06.889 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.884+0000 7fb156d8c740 1 -- 172.21.15.100:0/1106864635 wait complete. 2024-05-10T11:29:06.889 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.884+0000 7fb156d8c740 1 -- start start 2024-05-10T11:29:06.889 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.884+0000 7fb156d8c740 1 Processor -- start 2024-05-10T11:29:06.889 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.885+0000 7fb156d8c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564b0d03ddb0 0x564b0d050680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:06.889 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.885+0000 7fb156d8c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564b0d03e7a0 0x564b0d050bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:06.889 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.885+0000 7fb156d8c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564b0d043910 0x564b0d051bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:06.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.885+0000 7fb156d8c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb14c1a7750 con 0x564b0d03ddb0 2024-05-10T11:29:06.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.885+0000 7fb156d8c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb14c1a8bc0 con 0x564b0d03e7a0 2024-05-10T11:29:06.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.885+0000 7fb156d8c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb14c1a7e20 con 0x564b0d043910 2024-05-10T11:29:06.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.885+0000 7fb151e0e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564b0d03ddb0 0x564b0d050680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:06.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.885+0000 7fb15160d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564b0d03e7a0 0x564b0d050bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:06.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.885+0000 7fb156d8c740 10 client.0 _ll_register_callbacks cb 0x7fb144005b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:29:06.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.885+0000 7fb15160d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564b0d03e7a0 0x564b0d050bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:52142/0 (socket says 172.21.15.100:52142) 2024-05-10T11:29:06.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.885+0000 7fb151e0e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564b0d03ddb0 0x564b0d050680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41586/0 (socket says 172.21.15.100:41586) 2024-05-10T11:29:06.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.885+0000 7fb15160d640 1 -- 172.21.15.100:0/3482855388 learned_addr learned my addr 172.21.15.100:0/3482855388 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:29:06.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.885+0000 7fb15260f640 1 --2- 172.21.15.100:0/3482855388 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564b0d043910 0x564b0d051bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:06.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.885+0000 7fb15160d640 1 -- 172.21.15.100:0/3482855388 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564b0d043910 msgr2=0x564b0d051bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:06.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.885+0000 7fb15160d640 1 --2- 172.21.15.100:0/3482855388 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564b0d043910 0x564b0d051bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:06.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.885+0000 7fb15160d640 1 -- 172.21.15.100:0/3482855388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564b0d03ddb0 msgr2=0x564b0d050680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:06.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.885+0000 7fb15160d640 1 --2- 172.21.15.100:0/3482855388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564b0d03ddb0 0x564b0d050680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:06.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.885+0000 7fb15160d640 1 -- 172.21.15.100:0/3482855388 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b0d039250 con 0x564b0d03e7a0 2024-05-10T11:29:06.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.885+0000 7fb15260f640 1 --2- 172.21.15.100:0/3482855388 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564b0d043910 0x564b0d051bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:29:06.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.885+0000 7fb151e0e640 1 --2- 172.21.15.100:0/3482855388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564b0d03ddb0 0x564b0d050680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:29:06.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.886+0000 7fb15160d640 1 --2- 172.21.15.100:0/3482855388 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564b0d03e7a0 0x564b0d050bc0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb13c0058c0 tx=0x7fb13c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b3e900de3cd2ba5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:06.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.886+0000 7fb1357fa640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:29:06.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.886+0000 7fb1357fa640 1 -- 172.21.15.100:0/3482855388 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb13c030060 con 0x564b0d03e7a0 2024-05-10T11:29:06.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.886+0000 7fb156d8c740 1 -- 172.21.15.100:0/3482855388 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1440b2b00 con 0x564b0d03e7a0 2024-05-10T11:29:06.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.886+0000 7fb1357fa640 1 -- 172.21.15.100:0/3482855388 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fb13c01a070 con 0x564b0d03e7a0 2024-05-10T11:29:06.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.886+0000 7fb1357fa640 1 -- 172.21.15.100:0/3482855388 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb13c037050 con 0x564b0d03e7a0 2024-05-10T11:29:06.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.886+0000 7fb156d8c740 10 client.26948 fetch_fsmap learned FSMap version 31 2024-05-10T11:29:06.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.886+0000 7fb156d8c740 1 -- 172.21.15.100:0/3482855388 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564b0d0533a0 con 0x564b0d03e7a0 2024-05-10T11:29:06.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.886+0000 7fb1357fa640 1 -- 172.21.15.100:0/3482855388 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb1440b2b00 con 0x564b0d03e7a0 2024-05-10T11:29:06.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.886+0000 7fb1357fa640 1 -- 172.21.15.100:0/3482855388 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fb13c01d070 con 0x564b0d03e7a0 2024-05-10T11:29:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.886+0000 7fb156d8c740 10 client.26948 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T11:29:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.886+0000 7fb156d8c740 10 client.26948 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:29:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.886+0000 7fb156d8c740 20 client.26948 populate_metadata read hostname 'smithi100' 2024-05-10T11:29:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.886+0000 7fb156d8c740 1 --2- 172.21.15.100:0/3482855388 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x564b0d0c32c0 0x564b0d0c56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.886+0000 7fb156d8c740 4 client.26948 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:29:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.886+0000 7fb156d8c740 1 -- 172.21.15.100:0/3482855388 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564b0cfaa050 con 0x564b0d0c32c0 2024-05-10T11:29:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.887+0000 7fb151e0e640 1 --2- 172.21.15.100:0/3482855388 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x564b0d0c32c0 0x564b0d0c56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.887+0000 7fb1357fa640 10 client.26948 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:29:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.887+0000 7fb151e0e640 1 --2- 172.21.15.100:0/3482855388 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x564b0d0c32c0 0x564b0d0c56e0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84c4e965245c79ed server_cookie=b31658c8b3e749fe in_seq=0 out_seq=0 2024-05-10T11:29:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.908+0000 7fb1357fa640 1 -- 172.21.15.100:0/3482855388 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564b0cfaa050 con 0x564b0d0c32c0 2024-05-10T11:29:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.908+0000 7fb1357fa640 10 client.26948 handle_command_reply: tid=0 2024-05-10T11:29:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.908+0000 7fb156d8c740 1 client.26948 shutdown 2024-05-10T11:29:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.908+0000 7fb156d8c740 1 -- 172.21.15.100:0/3482855388 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564b0d03e7a0 msgr2=0x564b0d050bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.908+0000 7fb156d8c740 1 --2- 172.21.15.100:0/3482855388 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564b0d03e7a0 0x564b0d050bc0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb13c0058c0 tx=0x7fb13c004730 comp rx=0 tx=0).stop 2024-05-10T11:29:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.908+0000 7fb156d8c740 1 -- 172.21.15.100:0/3482855388 shutdown_connections 2024-05-10T11:29:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.908+0000 7fb156d8c740 1 --2- 172.21.15.100:0/3482855388 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x564b0d0c32c0 0x564b0d0c56e0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.908+0000 7fb156d8c740 1 --2- 172.21.15.100:0/3482855388 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564b0d043910 0x564b0d051bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.908+0000 7fb156d8c740 1 --2- 172.21.15.100:0/3482855388 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564b0d03e7a0 0x564b0d050bc0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.908+0000 7fb1357fa640 0 client.26948 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:29:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.908+0000 7fb156d8c740 1 --2- 172.21.15.100:0/3482855388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564b0d03ddb0 0x564b0d050680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.908+0000 7fb156d8c740 1 -- 172.21.15.100:0/3482855388 >> 172.21.15.100:0/3482855388 conn(0x564b0d039420 msgr2=0x564b0d03f980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.908+0000 7fb156d8c740 1 -- 172.21.15.100:0/3482855388 shutdown_connections 2024-05-10T11:29:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.908+0000 7fb156d8c740 1 -- 172.21.15.100:0/3482855388 wait complete. 2024-05-10T11:29:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.908+0000 7fb156d8c740 20 client.26948 trim_cache size 0 max 16384 2024-05-10T11:29:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:29:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:29:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:29:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:29:06.912 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.909+0000 7fb154072640 1 -- 172.21.15.100:0/428970399 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fb1380b2920 msgr2=0x7fb1380b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:06.912 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.909+0000 7fb154072640 1 --2- 172.21.15.100:0/428970399 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fb1380b2920 0x7fb1380b4de0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x564b0d03f000 tx=0x7fb13c005850 comp rx=0 tx=0).stop 2024-05-10T11:29:06.912 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.909+0000 7fb154072640 1 -- 172.21.15.100:0/428970399 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb14c1034c0 msgr2=0x7fb14c1a00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:06.912 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.909+0000 7fb154072640 1 --2- 172.21.15.100:0/428970399 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb14c1034c0 0x7fb14c1a00e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb144009fd0 tx=0x7fb14400ef10 comp rx=0 tx=0).stop 2024-05-10T11:29:06.912 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.909+0000 7fb154072640 1 -- 172.21.15.100:0/428970399 shutdown_connections 2024-05-10T11:29:06.912 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.909+0000 7fb154072640 1 --2- 172.21.15.100:0/428970399 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fb1380b2920 0x7fb1380b4de0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:06.912 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.909+0000 7fb154072640 1 --2- 172.21.15.100:0/428970399 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb14c1034c0 0x7fb14c1a00e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:06.912 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.909+0000 7fb154072640 1 --2- 172.21.15.100:0/428970399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb14c102b00 0x7fb14c1990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:06.912 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.909+0000 7fb154072640 1 --2- 172.21.15.100:0/428970399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb14c101910 0x7fb14c198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:06.913 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.909+0000 7fb154072640 1 -- 172.21.15.100:0/428970399 >> 172.21.15.100:0/428970399 conn(0x7fb14c0fce80 msgr2=0x7fb14c06a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:06.913 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.909+0000 7fb154072640 1 -- 172.21.15.100:0/428970399 shutdown_connections 2024-05-10T11:29:06.913 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:06.909+0000 7fb154072640 1 -- 172.21.15.100:0/428970399 wait complete. 2024-05-10T11:29:06.980 INFO:tasks.workunit.client.0.smithi100.stdout: 1 277124 2.73 MB/sec execute 421 sec latency 17.484 ms 2024-05-10T11:29:07.520 INFO:journalctl@ceph.mds.f.smithi188.stdout:May 10 11:29:07 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-f[63940]: ignoring --setuser ceph since I am not root 2024-05-10T11:29:07.520 INFO:journalctl@ceph.mds.f.smithi188.stdout:May 10 11:29:07 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-f[63940]: ignoring --setgroup ceph since I am not root 2024-05-10T11:29:07.520 INFO:journalctl@ceph.mds.f.smithi188.stdout:May 10 11:29:07 smithi188 ceph-mds[63944]: -- 172.21.15.188:0/903803932 <== mon.1 v2:172.21.15.188:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fa2d71da40 con 0x55fa2d6f1000 2024-05-10T11:29:07.520 INFO:journalctl@ceph.mds.f.smithi188.stdout:May 10 11:29:07 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-f[63940]: starting mds.f at 2024-05-10T11:29:07.679 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:29:07 smithi100 ceph-mon[30296]: pgmap v742: 129 pgs: 129 active+clean; 112 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 1.2 MiB/s wr, 124 op/s 2024-05-10T11:29:07.680 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:29:07 smithi100 ceph-mon[30296]: daemon mds.f restarted 2024-05-10T11:29:07.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:29:07 smithi188 ceph-mon[34075]: pgmap v742: 129 pgs: 129 active+clean; 112 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 1.2 MiB/s wr, 124 op/s 2024-05-10T11:29:07.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:29:07 smithi188 ceph-mon[34075]: daemon mds.f restarted 2024-05-10T11:29:07.981 INFO:tasks.workunit.client.0.smithi100.stdout: 1 277416 2.72 MB/sec execute 422 sec latency 22.106 ms 2024-05-10T11:29:08.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:29:07 smithi178 ceph-mon[33359]: pgmap v742: 129 pgs: 129 active+clean; 112 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 1.2 MiB/s wr, 124 op/s 2024-05-10T11:29:08.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:29:07 smithi178 ceph-mon[33359]: daemon mds.f restarted 2024-05-10T11:29:08.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.105+0000 7f187dc24700 1 -- 172.21.15.100:0/672777595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1878101a90 msgr2=0x7f1878101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:08.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.105+0000 7f187dc24700 1 --2- 172.21.15.100:0/672777595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1878101a90 0x7f1878101f10 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f186c009990 tx=0x7f186c009ca0 comp rx=0 tx=0).stop 2024-05-10T11:29:08.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.105+0000 7f187dc24700 1 -- 172.21.15.100:0/672777595 shutdown_connections 2024-05-10T11:29:08.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.105+0000 7f187dc24700 1 --2- 172.21.15.100:0/672777595 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1878102450 0x7f1878110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:08.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.105+0000 7f187dc24700 1 --2- 172.21.15.100:0/672777595 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1878109520 0x7f1878109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:08.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.105+0000 7f187dc24700 1 --2- 172.21.15.100:0/672777595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1878101a90 0x7f1878101f10 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:08.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.105+0000 7f187dc24700 1 -- 172.21.15.100:0/672777595 >> 172.21.15.100:0/672777595 conn(0x7f1878078a10 msgr2=0x7f18780ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:08.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.106+0000 7f187dc24700 1 -- 172.21.15.100:0/672777595 shutdown_connections 2024-05-10T11:29:08.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.106+0000 7f187dc24700 1 -- 172.21.15.100:0/672777595 wait complete. 2024-05-10T11:29:08.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.106+0000 7f187dc24700 1 Processor -- start 2024-05-10T11:29:08.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.107+0000 7f187dc24700 1 -- start start 2024-05-10T11:29:08.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.107+0000 7f187dc24700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1878101a90 0x7f187819f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:08.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.107+0000 7f187dc24700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1878102450 0x7f187819fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:08.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.107+0000 7f187dc24700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1878109520 0x7f1878199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:08.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.107+0000 7f187dc24700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1878113550 con 0x7f1878109520 2024-05-10T11:29:08.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.107+0000 7f187dc24700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f18781133d0 con 0x7f1878102450 2024-05-10T11:29:08.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.107+0000 7f187dc24700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f18781136d0 con 0x7f1878101a90 2024-05-10T11:29:08.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.107+0000 7f18777fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1878101a90 0x7f187819f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:08.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.107+0000 7f1877fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1878109520 0x7f1878199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:08.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.107+0000 7f18777fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1878101a90 0x7f187819f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:42320/0 (socket says 172.21.15.100:42320) 2024-05-10T11:29:08.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.107+0000 7f18777fe700 1 -- 172.21.15.100:0/1936907085 learned_addr learned my addr 172.21.15.100:0/1936907085 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:29:08.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.107+0000 7f1876ffd700 1 --2- 172.21.15.100:0/1936907085 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1878102450 0x7f187819fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:08.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.107+0000 7f18777fe700 1 -- 172.21.15.100:0/1936907085 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1878102450 msgr2=0x7f187819fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:08.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.107+0000 7f18777fe700 1 --2- 172.21.15.100:0/1936907085 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1878102450 0x7f187819fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:08.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.107+0000 7f18777fe700 1 -- 172.21.15.100:0/1936907085 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1878109520 msgr2=0x7f1878199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:08.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.107+0000 7f18777fe700 1 --2- 172.21.15.100:0/1936907085 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1878109520 0x7f1878199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:08.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.107+0000 7f18777fe700 1 -- 172.21.15.100:0/1936907085 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f187819a0b0 con 0x7f1878101a90 2024-05-10T11:29:08.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.107+0000 7f1876ffd700 1 --2- 172.21.15.100:0/1936907085 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1878102450 0x7f187819fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:29:08.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.107+0000 7f1877fff700 1 --2- 172.21.15.100:0/1936907085 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1878109520 0x7f1878199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:29:08.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.108+0000 7f18777fe700 1 --2- 172.21.15.100:0/1936907085 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1878101a90 0x7f187819f790 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f186800e9c0 tx=0x7f186800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:08.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.108+0000 7f1874ff9700 1 -- 172.21.15.100:0/1936907085 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f186800cbd0 con 0x7f1878101a90 2024-05-10T11:29:08.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.108+0000 7f1874ff9700 1 -- 172.21.15.100:0/1936907085 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f186800ee60 con 0x7f1878101a90 2024-05-10T11:29:08.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.108+0000 7f187dc24700 1 -- 172.21.15.100:0/1936907085 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f187819a3a0 con 0x7f1878101a90 2024-05-10T11:29:08.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.108+0000 7f1874ff9700 1 -- 172.21.15.100:0/1936907085 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18680054b0 con 0x7f1878101a90 2024-05-10T11:29:08.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.108+0000 7f187dc24700 1 -- 172.21.15.100:0/1936907085 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18781a6660 con 0x7f1878101a90 2024-05-10T11:29:08.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.110+0000 7f1874ff9700 1 -- 172.21.15.100:0/1936907085 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f1868004750 con 0x7f1878101a90 2024-05-10T11:29:08.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.111+0000 7f1874ff9700 1 --2- 172.21.15.100:0/1936907085 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f18600b2900 0x7f18600b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:08.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.111+0000 7f1876ffd700 1 --2- 172.21.15.100:0/1936907085 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f18600b2900 0x7f18600b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:08.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.111+0000 7f1874ff9700 1 -- 172.21.15.100:0/1936907085 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f18680e2740 con 0x7f1878101a90 2024-05-10T11:29:08.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.112+0000 7f1876ffd700 1 --2- 172.21.15.100:0/1936907085 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f18600b2900 0x7f18600b4dc0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f18780ffaf0 tx=0x7f186c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:08.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.112+0000 7f18884b7b80 1 Processor -- start 2024-05-10T11:29:08.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.112+0000 7f18884b7b80 1 -- start start 2024-05-10T11:29:08.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.112+0000 7f18884b7b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652c7788960 0x5652c7788d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:08.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.112+0000 7f18884b7b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652c7789280 0x5652c778cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:08.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.112+0000 7f18884b7b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5652c778d0c0 0x5652c778d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:08.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.112+0000 7f18884b7b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5652c76d68e0 con 0x5652c7788960 2024-05-10T11:29:08.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.113+0000 7f18884b7b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5652c7675840 con 0x5652c7789280 2024-05-10T11:29:08.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.113+0000 7f18884b7b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5652c76d28a0 con 0x5652c778d0c0 2024-05-10T11:29:08.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.113+0000 7f1877fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5652c778d0c0 0x5652c778d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:08.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.113+0000 7f18777fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652c7788960 0x5652c7788d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:08.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.113+0000 7f1877fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5652c778d0c0 0x5652c778d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:42336/0 (socket says 172.21.15.100:42336) 2024-05-10T11:29:08.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.113+0000 7f1877fff700 1 -- 172.21.15.100:0/3374890021 learned_addr learned my addr 172.21.15.100:0/3374890021 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:29:08.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.113+0000 7f1876ffd700 1 --2- 172.21.15.100:0/3374890021 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652c7789280 0x5652c778cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:08.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.113+0000 7f1877fff700 1 -- 172.21.15.100:0/3374890021 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652c7789280 msgr2=0x5652c778cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:08.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.113+0000 7f1877fff700 1 --2- 172.21.15.100:0/3374890021 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652c7789280 0x5652c778cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:08.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.113+0000 7f1877fff700 1 -- 172.21.15.100:0/3374890021 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652c7788960 msgr2=0x5652c7788d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:08.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.113+0000 7f1877fff700 1 --2- 172.21.15.100:0/3374890021 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652c7788960 0x5652c7788d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:08.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.113+0000 7f1877fff700 1 -- 172.21.15.100:0/3374890021 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652c7623950 con 0x5652c778d0c0 2024-05-10T11:29:08.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.113+0000 7f18777fe700 1 --2- 172.21.15.100:0/3374890021 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652c7788960 0x5652c7788d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:29:08.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.113+0000 7f1877fff700 1 --2- 172.21.15.100:0/3374890021 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5652c778d0c0 0x5652c778d500 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1870003eb0 tx=0x7f1870009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4f56b0404b755ad server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:08.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.113+0000 7f185dffb700 1 -- 172.21.15.100:0/3374890021 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1870019020 con 0x5652c778d0c0 2024-05-10T11:29:08.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.113+0000 7f185dffb700 1 -- 172.21.15.100:0/3374890021 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f18700092d0 con 0x5652c778d0c0 2024-05-10T11:29:08.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.113+0000 7f185dffb700 1 -- 172.21.15.100:0/3374890021 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187000e6f0 con 0x5652c778d0c0 2024-05-10T11:29:08.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.114+0000 7f18884b7b80 1 -- 172.21.15.100:0/3374890021 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5652c778d0c0 msgr2=0x5652c778d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:08.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.114+0000 7f18884b7b80 1 --2- 172.21.15.100:0/3374890021 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5652c778d0c0 0x5652c778d500 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1870003eb0 tx=0x7f1870009920 comp rx=0 tx=0).stop 2024-05-10T11:29:08.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.114+0000 7f18884b7b80 1 -- 172.21.15.100:0/3374890021 shutdown_connections 2024-05-10T11:29:08.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.114+0000 7f18884b7b80 1 --2- 172.21.15.100:0/3374890021 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5652c778d0c0 0x5652c778d500 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:08.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.114+0000 7f18884b7b80 1 --2- 172.21.15.100:0/3374890021 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652c7788960 0x5652c7788d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:08.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.114+0000 7f18884b7b80 1 --2- 172.21.15.100:0/3374890021 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652c7789280 0x5652c778cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:08.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.114+0000 7f18884b7b80 1 -- 172.21.15.100:0/3374890021 >> 172.21.15.100:0/3374890021 conn(0x5652c76b0180 msgr2=0x5652c777ecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:08.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.114+0000 7f18884b7b80 1 -- 172.21.15.100:0/3374890021 shutdown_connections 2024-05-10T11:29:08.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.114+0000 7f18884b7b80 1 -- 172.21.15.100:0/3374890021 wait complete. 2024-05-10T11:29:08.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.115+0000 7f18884b7b80 1 -- start start 2024-05-10T11:29:08.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.115+0000 7f18884b7b80 1 Processor -- start 2024-05-10T11:29:08.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.115+0000 7f18884b7b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5652c7788960 0x5652c7794510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:08.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.115+0000 7f18884b7b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652c7789280 0x5652c77918b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:08.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.115+0000 7f18884b7b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652c778d0c0 0x5652c7791df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:08.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.115+0000 7f18884b7b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5652c77952f0 con 0x5652c7789280 2024-05-10T11:29:08.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.115+0000 7f18884b7b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5652c763ccb0 con 0x5652c778d0c0 2024-05-10T11:29:08.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.115+0000 7f18884b7b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5652c7795470 con 0x5652c7788960 2024-05-10T11:29:08.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.115+0000 7f18884b7b80 10 client.0 _ll_register_callbacks cb 0x5652c766e9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:29:08.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.115+0000 7f1876ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652c7789280 0x5652c77918b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:08.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.115+0000 7f18777fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5652c7788960 0x5652c7794510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:08.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.115+0000 7f1876ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652c7789280 0x5652c77918b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41606/0 (socket says 172.21.15.100:41606) 2024-05-10T11:29:08.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.115+0000 7f1876ffd700 1 -- 172.21.15.100:0/3437739715 learned_addr learned my addr 172.21.15.100:0/3437739715 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:29:08.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.115+0000 7f1877fff700 1 --2- 172.21.15.100:0/3437739715 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652c778d0c0 0x5652c7791df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:08.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.116+0000 7f18777fe700 1 -- 172.21.15.100:0/3437739715 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652c778d0c0 msgr2=0x5652c7791df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:08.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.116+0000 7f18777fe700 1 --2- 172.21.15.100:0/3437739715 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652c778d0c0 0x5652c7791df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:08.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.116+0000 7f18777fe700 1 -- 172.21.15.100:0/3437739715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652c7789280 msgr2=0x5652c77918b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:08.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.116+0000 7f18777fe700 1 --2- 172.21.15.100:0/3437739715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652c7789280 0x5652c77918b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:08.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.116+0000 7f18777fe700 1 -- 172.21.15.100:0/3437739715 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652c7672710 con 0x5652c7788960 2024-05-10T11:29:08.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.116+0000 7f1876ffd700 1 --2- 172.21.15.100:0/3437739715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652c7789280 0x5652c77918b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:29:08.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.116+0000 7f18777fe700 1 --2- 172.21.15.100:0/3437739715 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5652c7788960 0x5652c7794510 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f18680dfee0 tx=0x7f18680a5260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f43bc9153131ed60 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:08.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.116+0000 7f1856ffd700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:29:08.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.116+0000 7f1856ffd700 1 -- 172.21.15.100:0/3437739715 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18680aa100 con 0x5652c7788960 2024-05-10T11:29:08.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.116+0000 7f1856ffd700 1 -- 172.21.15.100:0/3437739715 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f18680b25e0 con 0x5652c7788960 2024-05-10T11:29:08.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.116+0000 7f18884b7b80 1 -- 172.21.15.100:0/3437739715 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18700092d0 con 0x5652c7788960 2024-05-10T11:29:08.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.116+0000 7f1856ffd700 1 -- 172.21.15.100:0/3437739715 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f18680a9a00 con 0x5652c7788960 2024-05-10T11:29:08.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.116+0000 7f18884b7b80 10 client.27046 fetch_fsmap learned FSMap version 33 2024-05-10T11:29:08.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.116+0000 7f18884b7b80 1 -- 172.21.15.100:0/3437739715 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5652c779fb70 con 0x5652c7788960 2024-05-10T11:29:08.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.116+0000 7f1856ffd700 1 -- 172.21.15.100:0/3437739715 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f18700092d0 con 0x5652c7788960 2024-05-10T11:29:08.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.117+0000 7f1856ffd700 1 -- 172.21.15.100:0/3437739715 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f18680c5020 con 0x5652c7788960 2024-05-10T11:29:08.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.117+0000 7f18884b7b80 10 client.27046 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T11:29:08.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.117+0000 7f18884b7b80 10 client.27046 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:29:08.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.117+0000 7f18884b7b80 20 client.27046 populate_metadata read hostname 'smithi100' 2024-05-10T11:29:08.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.117+0000 7f18884b7b80 1 --2- 172.21.15.100:0/3437739715 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5652c77a2100 0x5652c77a4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:08.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.117+0000 7f18884b7b80 4 client.27046 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:29:08.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.117+0000 7f18884b7b80 1 -- 172.21.15.100:0/3437739715 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5652c77506d0 con 0x5652c77a2100 2024-05-10T11:29:08.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.117+0000 7f1876ffd700 1 --2- 172.21.15.100:0/3437739715 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5652c77a2100 0x5652c77a4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:08.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.117+0000 7f1856ffd700 10 client.27046 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:29:08.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.117+0000 7f1876ffd700 1 --2- 172.21.15.100:0/3437739715 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5652c77a2100 0x5652c77a4530 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fbf8ab17158b717 server_cookie=3377c02727f2385d in_seq=0 out_seq=0 2024-05-10T11:29:08.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.118+0000 7f1856ffd700 1 -- 172.21.15.100:0/3437739715 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5652c77506d0 con 0x5652c77a2100 2024-05-10T11:29:08.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.118+0000 7f1856ffd700 10 client.27046 handle_command_reply: tid=0 2024-05-10T11:29:08.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.118+0000 7f18884b7b80 1 client.27046 shutdown 2024-05-10T11:29:08.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.118+0000 7f18884b7b80 1 -- 172.21.15.100:0/3437739715 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5652c7788960 msgr2=0x5652c7794510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:08.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.118+0000 7f18884b7b80 1 --2- 172.21.15.100:0/3437739715 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5652c7788960 0x5652c7794510 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f18680dfee0 tx=0x7f18680a5260 comp rx=0 tx=0).stop 2024-05-10T11:29:08.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.118+0000 7f18884b7b80 1 -- 172.21.15.100:0/3437739715 shutdown_connections 2024-05-10T11:29:08.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.118+0000 7f18884b7b80 1 --2- 172.21.15.100:0/3437739715 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5652c77a2100 0x5652c77a4530 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:08.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.118+0000 7f18884b7b80 1 --2- 172.21.15.100:0/3437739715 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652c778d0c0 0x5652c7791df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:08.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.118+0000 7f1856ffd700 0 client.27046 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:29:08.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.118+0000 7f18884b7b80 1 --2- 172.21.15.100:0/3437739715 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5652c7788960 0x5652c7794510 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:08.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.118+0000 7f18884b7b80 1 --2- 172.21.15.100:0/3437739715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652c7789280 0x5652c77918b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:08.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.118+0000 7f18884b7b80 1 -- 172.21.15.100:0/3437739715 >> 172.21.15.100:0/3437739715 conn(0x5652c76b0180 msgr2=0x5652c77797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:08.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.119+0000 7f18884b7b80 1 -- 172.21.15.100:0/3437739715 shutdown_connections 2024-05-10T11:29:08.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.119+0000 7f18884b7b80 1 -- 172.21.15.100:0/3437739715 wait complete. 2024-05-10T11:29:08.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.119+0000 7f18884b7b80 20 client.27046 trim_cache size 0 max 16384 2024-05-10T11:29:08.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.144+0000 7f18884b7b80 1 Processor -- start 2024-05-10T11:29:08.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.144+0000 7f18884b7b80 1 -- start start 2024-05-10T11:29:08.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.144+0000 7f18884b7b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652c76b0180 0x5652c779e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:08.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.145+0000 7f18884b7b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5652c7788960 0x5652c7792f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:08.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.145+0000 7f18884b7b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652c7789280 0x5652c77934a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:08.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.145+0000 7f18884b7b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f18781af7e0 con 0x5652c7789280 2024-05-10T11:29:08.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.145+0000 7f18884b7b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f18781ad070 con 0x5652c76b0180 2024-05-10T11:29:08.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.145+0000 7f18884b7b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f18781aaee0 con 0x5652c7788960 2024-05-10T11:29:08.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.145+0000 7f1877fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652c7789280 0x5652c77934a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:08.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.145+0000 7f1876ffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5652c7788960 0x5652c7792f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:08.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.145+0000 7f18777fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652c76b0180 0x5652c779e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:08.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.145+0000 7f1876ffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5652c7788960 0x5652c7792f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:42348/0 (socket says 172.21.15.100:42348) 2024-05-10T11:29:08.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.145+0000 7f1877fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652c7789280 0x5652c77934a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41612/0 (socket says 172.21.15.100:41612) 2024-05-10T11:29:08.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.145+0000 7f1876ffd700 1 -- 172.21.15.100:0/948339022 learned_addr learned my addr 172.21.15.100:0/948339022 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:29:08.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.145+0000 7f1877fff700 1 -- 172.21.15.100:0/948339022 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5652c7788960 msgr2=0x5652c7792f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:08.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.145+0000 7f1877fff700 1 --2- 172.21.15.100:0/948339022 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5652c7788960 0x5652c7792f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:08.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.145+0000 7f1877fff700 1 -- 172.21.15.100:0/948339022 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652c76b0180 msgr2=0x5652c779e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:08.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.145+0000 7f1877fff700 1 --2- 172.21.15.100:0/948339022 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652c76b0180 0x5652c779e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:08.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.145+0000 7f1877fff700 1 -- 172.21.15.100:0/948339022 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652c778fe40 con 0x5652c7789280 2024-05-10T11:29:08.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.145+0000 7f1876ffd700 1 --2- 172.21.15.100:0/948339022 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5652c7788960 0x5652c7792f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:29:08.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.145+0000 7f1877fff700 1 --2- 172.21.15.100:0/948339022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652c7789280 0x5652c77934a0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f1870000f80 tx=0x7f1870006000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20939d4cf42a4253 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:08.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.145+0000 7f185d7fa700 1 -- 172.21.15.100:0/948339022 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1870019040 con 0x5652c7789280 2024-05-10T11:29:08.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.146+0000 7f185d7fa700 1 -- 172.21.15.100:0/948339022 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1870023070 con 0x5652c7789280 2024-05-10T11:29:08.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.146+0000 7f185d7fa700 1 -- 172.21.15.100:0/948339022 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1870007b30 con 0x5652c7789280 2024-05-10T11:29:08.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.146+0000 7f18884b7b80 1 -- 172.21.15.100:0/948339022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652c7789280 msgr2=0x5652c77934a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:08.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.146+0000 7f18884b7b80 1 --2- 172.21.15.100:0/948339022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652c7789280 0x5652c77934a0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f1870000f80 tx=0x7f1870006000 comp rx=0 tx=0).stop 2024-05-10T11:29:08.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.146+0000 7f18884b7b80 1 -- 172.21.15.100:0/948339022 shutdown_connections 2024-05-10T11:29:08.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.146+0000 7f18884b7b80 1 --2- 172.21.15.100:0/948339022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652c7789280 0x5652c77934a0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:08.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.146+0000 7f18884b7b80 1 --2- 172.21.15.100:0/948339022 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652c76b0180 0x5652c779e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:08.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.146+0000 7f18884b7b80 1 --2- 172.21.15.100:0/948339022 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5652c7788960 0x5652c7792f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:08.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.146+0000 7f18884b7b80 1 -- 172.21.15.100:0/948339022 >> 172.21.15.100:0/948339022 conn(0x5652c7779b30 msgr2=0x5652c7783bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:08.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.146+0000 7f18884b7b80 1 -- 172.21.15.100:0/948339022 shutdown_connections 2024-05-10T11:29:08.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.146+0000 7f18884b7b80 1 -- 172.21.15.100:0/948339022 wait complete. 2024-05-10T11:29:08.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.147+0000 7f18884b7b80 1 -- start start 2024-05-10T11:29:08.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.147+0000 7f18884b7b80 1 Processor -- start 2024-05-10T11:29:08.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.147+0000 7f18884b7b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5652c76b0180 0x5652c779a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:08.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.147+0000 7f18884b7b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652c7788960 0x5652c779ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:08.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.147+0000 7f18884b7b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652c7789280 0x5652c77978d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:08.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.147+0000 7f18884b7b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f18781ab530 con 0x5652c7788960 2024-05-10T11:29:08.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.147+0000 7f18884b7b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f18781ac9a0 con 0x5652c7789280 2024-05-10T11:29:08.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.147+0000 7f18884b7b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f18781abc00 con 0x5652c76b0180 2024-05-10T11:29:08.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.147+0000 7f18884b7b80 10 client.0 _ll_register_callbacks cb 0x5652c766e9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:29:08.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.147+0000 7f18777fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5652c76b0180 0x5652c779a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:08.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.147+0000 7f18777fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5652c76b0180 0x5652c779a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:42358/0 (socket says 172.21.15.100:42358) 2024-05-10T11:29:08.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.148+0000 7f18777fe700 1 -- 172.21.15.100:0/1053139078 learned_addr learned my addr 172.21.15.100:0/1053139078 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:29:08.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.148+0000 7f1876ffd700 1 --2- 172.21.15.100:0/1053139078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652c7788960 0x5652c779ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:08.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.148+0000 7f1877fff700 1 --2- 172.21.15.100:0/1053139078 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652c7789280 0x5652c77978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:08.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.148+0000 7f1876ffd700 1 -- 172.21.15.100:0/1053139078 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5652c76b0180 msgr2=0x5652c779a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:08.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.148+0000 7f1876ffd700 1 --2- 172.21.15.100:0/1053139078 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5652c76b0180 0x5652c779a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:08.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.148+0000 7f1876ffd700 1 -- 172.21.15.100:0/1053139078 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652c7789280 msgr2=0x5652c77978d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:08.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.148+0000 7f1876ffd700 1 --2- 172.21.15.100:0/1053139078 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652c7789280 0x5652c77978d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:08.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.148+0000 7f1876ffd700 1 -- 172.21.15.100:0/1053139078 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652c7798110 con 0x5652c7788960 2024-05-10T11:29:08.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.148+0000 7f1877fff700 1 --2- 172.21.15.100:0/1053139078 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652c7789280 0x5652c77978d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:29:08.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.148+0000 7f1876ffd700 1 --2- 172.21.15.100:0/1053139078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652c7788960 0x5652c779ae20 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f186c00b2e0 tx=0x7f186c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=913bbf158e0f9d3b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:08.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.148+0000 7f1856ffd700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:29:08.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.148+0000 7f1856ffd700 1 -- 172.21.15.100:0/1053139078 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f186c038070 con 0x5652c7788960 2024-05-10T11:29:08.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.148+0000 7f18884b7b80 1 -- 172.21.15.100:0/1053139078 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1870023070 con 0x5652c7788960 2024-05-10T11:29:08.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.149+0000 7f1856ffd700 1 -- 172.21.15.100:0/1053139078 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f186c034400 con 0x5652c7788960 2024-05-10T11:29:08.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.149+0000 7f1856ffd700 1 -- 172.21.15.100:0/1053139078 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f186c044050 con 0x5652c7788960 2024-05-10T11:29:08.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.149+0000 7f18884b7b80 10 client.17601 fetch_fsmap learned FSMap version 33 2024-05-10T11:29:08.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.149+0000 7f18884b7b80 1 -- 172.21.15.100:0/1053139078 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5652c780d690 con 0x5652c7788960 2024-05-10T11:29:08.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.149+0000 7f1856ffd700 1 -- 172.21.15.100:0/1053139078 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1870023070 con 0x5652c7788960 2024-05-10T11:29:08.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.149+0000 7f1856ffd700 1 -- 172.21.15.100:0/1053139078 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f186c0268f0 con 0x5652c7788960 2024-05-10T11:29:08.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.149+0000 7f18884b7b80 10 client.17601 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T11:29:08.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.149+0000 7f18884b7b80 10 client.17601 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:29:08.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.150+0000 7f18884b7b80 20 client.17601 populate_metadata read hostname 'smithi100' 2024-05-10T11:29:08.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.150+0000 7f18884b7b80 1 --2- 172.21.15.100:0/1053139078 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5652c778d0c0 0x5652c780fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:08.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.150+0000 7f18884b7b80 4 client.17601 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T11:29:08.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.150+0000 7f18884b7b80 1 -- 172.21.15.100:0/1053139078 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5652c76e42c0 con 0x5652c778d0c0 2024-05-10T11:29:08.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.150+0000 7f18777fe700 1 --2- 172.21.15.100:0/1053139078 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5652c778d0c0 0x5652c780fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:08.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.150+0000 7f1856ffd700 10 client.17601 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:29:08.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.150+0000 7f18777fe700 1 --2- 172.21.15.100:0/1053139078 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5652c778d0c0 0x5652c780fdc0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21ac5ddcefdce56 server_cookie=403fc80435cb25a8 in_seq=0 out_seq=0 2024-05-10T11:29:08.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.151+0000 7f1856ffd700 1 -- 172.21.15.100:0/1053139078 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5652c76e42c0 con 0x5652c778d0c0 2024-05-10T11:29:08.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.151+0000 7f1856ffd700 10 client.17601 handle_command_reply: tid=0 2024-05-10T11:29:08.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.151+0000 7f18884b7b80 1 client.17601 shutdown 2024-05-10T11:29:08.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.152+0000 7f18884b7b80 1 -- 172.21.15.100:0/1053139078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652c7788960 msgr2=0x5652c779ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:08.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.152+0000 7f18884b7b80 1 --2- 172.21.15.100:0/1053139078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652c7788960 0x5652c779ae20 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f186c00b2e0 tx=0x7f186c009310 comp rx=0 tx=0).stop 2024-05-10T11:29:08.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.152+0000 7f18884b7b80 1 -- 172.21.15.100:0/1053139078 shutdown_connections 2024-05-10T11:29:08.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.152+0000 7f18884b7b80 1 --2- 172.21.15.100:0/1053139078 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5652c778d0c0 0x5652c780fdc0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:08.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.152+0000 7f18884b7b80 1 --2- 172.21.15.100:0/1053139078 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652c7789280 0x5652c77978d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:08.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.152+0000 7f18884b7b80 1 --2- 172.21.15.100:0/1053139078 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5652c76b0180 0x5652c779a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:08.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.152+0000 7f1856ffd700 0 client.17601 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:29:08.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.152+0000 7f18884b7b80 1 --2- 172.21.15.100:0/1053139078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652c7788960 0x5652c779ae20 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:08.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.152+0000 7f18884b7b80 1 -- 172.21.15.100:0/1053139078 >> 172.21.15.100:0/1053139078 conn(0x5652c7779b30 msgr2=0x5652c778d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:08.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.152+0000 7f18884b7b80 1 -- 172.21.15.100:0/1053139078 shutdown_connections 2024-05-10T11:29:08.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.152+0000 7f18884b7b80 1 -- 172.21.15.100:0/1053139078 wait complete. 2024-05-10T11:29:08.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.152+0000 7f18884b7b80 20 client.17601 trim_cache size 0 max 16384 2024-05-10T11:29:08.157 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:29:08.157 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-05-10T11:29:08.157 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "1f2d16ea-6e3b-4fcd-92d7-fc4e1bf3c1f2", 2024-05-10T11:29:08.157 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-05-10T11:29:08.157 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:29:08.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.152+0000 7f187dc24700 1 -- 172.21.15.100:0/1936907085 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f18600b2900 msgr2=0x7f18600b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:08.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.152+0000 7f187dc24700 1 --2- 172.21.15.100:0/1936907085 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f18600b2900 0x7f18600b4dc0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f18780ffaf0 tx=0x7f186c01a040 comp rx=0 tx=0).stop 2024-05-10T11:29:08.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.152+0000 7f187dc24700 1 -- 172.21.15.100:0/1936907085 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1878101a90 msgr2=0x7f187819f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:08.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.152+0000 7f187dc24700 1 --2- 172.21.15.100:0/1936907085 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1878101a90 0x7f187819f790 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f186800e9c0 tx=0x7f186800ecd0 comp rx=0 tx=0).stop 2024-05-10T11:29:08.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.153+0000 7f187dc24700 1 -- 172.21.15.100:0/1936907085 shutdown_connections 2024-05-10T11:29:08.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.153+0000 7f187dc24700 1 --2- 172.21.15.100:0/1936907085 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f18600b2900 0x7f18600b4dc0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:08.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.153+0000 7f187dc24700 1 --2- 172.21.15.100:0/1936907085 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1878109520 0x7f1878199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:08.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.153+0000 7f187dc24700 1 --2- 172.21.15.100:0/1936907085 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1878101a90 0x7f187819f790 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:08.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.153+0000 7f187dc24700 1 --2- 172.21.15.100:0/1936907085 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1878102450 0x7f187819fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:08.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.153+0000 7f187dc24700 1 -- 172.21.15.100:0/1936907085 >> 172.21.15.100:0/1936907085 conn(0x7f1878078a10 msgr2=0x7f18781002b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:08.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.153+0000 7f187dc24700 1 -- 172.21.15.100:0/1936907085 shutdown_connections 2024-05-10T11:29:08.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:08.153+0000 7f187dc24700 1 -- 172.21.15.100:0/1936907085 wait complete. 2024-05-10T11:29:08.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:29:08 smithi188 ceph-mon[34075]: osdmap e90: 12 total, 12 up, 12 in 2024-05-10T11:29:08.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:29:08 smithi188 ceph-mon[34075]: mds.? [v2:172.21.15.188:6834/3139559450,v1:172.21.15.188:6835/3139559450] up:boot 2024-05-10T11:29:08.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:29:08 smithi188 ceph-mon[34075]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-05-10T11:29:08.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:29:08 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-10T11:29:08.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:29:08 smithi188 ceph-mon[34075]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T11:29:08.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:29:08 smithi100 ceph-mon[30296]: osdmap e90: 12 total, 12 up, 12 in 2024-05-10T11:29:08.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:29:08 smithi100 ceph-mon[30296]: mds.? [v2:172.21.15.188:6834/3139559450,v1:172.21.15.188:6835/3139559450] up:boot 2024-05-10T11:29:08.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:29:08 smithi100 ceph-mon[30296]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-05-10T11:29:08.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:29:08 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-10T11:29:08.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:29:08 smithi100 ceph-mon[30296]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T11:29:08.862 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T11:29:08.980 INFO:tasks.workunit.client.0.smithi100.stdout: 1 277902 2.72 MB/sec execute 423 sec latency 28.362 ms 2024-05-10T11:29:09.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:29:08 smithi178 ceph-mon[33359]: osdmap e90: 12 total, 12 up, 12 in 2024-05-10T11:29:09.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:29:08 smithi178 ceph-mon[33359]: mds.? [v2:172.21.15.188:6834/3139559450,v1:172.21.15.188:6835/3139559450] up:boot 2024-05-10T11:29:09.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:29:08 smithi178 ceph-mon[33359]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-05-10T11:29:09.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:29:08 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-10T11:29:09.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:29:08 smithi178 ceph-mon[33359]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T11:29:09.172 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:29:09.570 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:29:09 smithi100 ceph-mon[30296]: scrub queued for path: / 2024-05-10T11:29:09.570 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:29:09 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:29:09.570 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:29:09 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:29:09.570 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:29:09 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:29:09.570 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:29:09 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:29:09.570 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:29:09 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:29:09.570 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:29:09 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:29:09.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:29:09 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:29:09.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:29:09 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:29:09.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:29:09 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:29:09.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:29:09 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:29:09.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:29:09 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:29:09.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:29:09 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:29:09.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:29:09 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:29:09.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:29:09 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:29:09.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:29:09 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:29:09.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:29:09 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:29:09.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:29:09 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:29:09.572 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:29:09 smithi100 ceph-mon[30296]: pgmap v744: 129 pgs: 129 active+clean; 108 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 1.2 MiB/s wr, 135 op/s 2024-05-10T11:29:09.980 INFO:tasks.workunit.client.0.smithi100.stdout: 1 278521 2.72 MB/sec execute 424 sec latency 16.323 ms 2024-05-10T11:29:10.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:29:09 smithi178 ceph-mon[33359]: scrub queued for path: / 2024-05-10T11:29:10.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:29:09 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:29:10.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:29:09 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:29:10.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:29:09 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:29:10.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:29:09 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:29:10.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:29:09 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:29:10.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:29:09 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:29:10.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:29:09 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:29:10.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:29:09 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:29:10.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:29:09 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:29:10.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:29:09 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:29:10.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:29:09 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:29:10.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:29:09 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:29:10.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:29:09 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:29:10.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:29:09 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:29:10.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:29:09 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:29:10.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:29:09 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:29:10.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:29:09 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:29:10.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:29:09 smithi178 ceph-mon[33359]: pgmap v744: 129 pgs: 129 active+clean; 108 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 1.2 MiB/s wr, 135 op/s 2024-05-10T11:29:10.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:29:09 smithi188 ceph-mon[34075]: scrub queued for path: / 2024-05-10T11:29:10.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:29:09 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:29:10.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:29:09 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:29:10.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:29:09 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:29:10.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:29:09 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:29:10.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:29:09 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:29:10.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:29:09 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:29:10.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:29:09 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:29:10.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:29:09 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:29:10.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:29:09 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:29:10.030 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:29:09 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:29:10.030 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:29:09 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:29:10.030 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:29:09 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:29:10.030 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:29:09 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:29:10.030 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:29:09 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:29:10.030 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:29:09 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:29:10.030 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:29:09 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:29:10.030 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:29:09 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:29:10.030 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:29:09 smithi188 ceph-mon[34075]: pgmap v744: 129 pgs: 129 active+clean; 108 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 1.2 MiB/s wr, 135 op/s 2024-05-10T11:29:10.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.911+0000 7f0864725700 1 -- 172.21.15.100:0/1606645306 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f085c06bc30 msgr2=0x7f085c06c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:10.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.911+0000 7f0864725700 1 --2- 172.21.15.100:0/1606645306 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f085c06bc30 0x7f085c06c0b0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f084c009990 tx=0x7f084c009ca0 comp rx=0 tx=0).stop 2024-05-10T11:29:10.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.912+0000 7f0864725700 1 -- 172.21.15.100:0/1606645306 shutdown_connections 2024-05-10T11:29:10.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.912+0000 7f0864725700 1 --2- 172.21.15.100:0/1606645306 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f085c06c5f0 0x7f085c110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:10.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.912+0000 7f0864725700 1 --2- 172.21.15.100:0/1606645306 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f085c109510 0x7f085c1098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:10.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.912+0000 7f0864725700 1 --2- 172.21.15.100:0/1606645306 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f085c06bc30 0x7f085c06c0b0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:10.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.912+0000 7f0864725700 1 -- 172.21.15.100:0/1606645306 >> 172.21.15.100:0/1606645306 conn(0x7f085c079350 msgr2=0x7f085c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:10.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.912+0000 7f0864725700 1 -- 172.21.15.100:0/1606645306 shutdown_connections 2024-05-10T11:29:10.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.912+0000 7f0864725700 1 -- 172.21.15.100:0/1606645306 wait complete. 2024-05-10T11:29:10.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.912+0000 7f0864725700 1 Processor -- start 2024-05-10T11:29:10.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.912+0000 7f0864725700 1 -- start start 2024-05-10T11:29:10.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.913+0000 7f0864725700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f085c06bc30 0x7f085c19f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:10.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.913+0000 7f0864725700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f085c06c5f0 0x7f085c19fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:10.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.913+0000 7f0864725700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f085c109510 0x7f085c199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:10.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.913+0000 7f0864725700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f085c1134d0 con 0x7f085c06c5f0 2024-05-10T11:29:10.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.913+0000 7f0864725700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f085c113350 con 0x7f085c06bc30 2024-05-10T11:29:10.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.913+0000 7f0864725700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f085c113650 con 0x7f085c109510 2024-05-10T11:29:10.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.913+0000 7f0861cc0700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f085c06c5f0 0x7f085c19fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:10.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.913+0000 7f08624c1700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f085c06bc30 0x7f085c19f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:10.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.913+0000 7f0861cc0700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f085c06c5f0 0x7f085c19fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41640/0 (socket says 172.21.15.100:41640) 2024-05-10T11:29:10.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.913+0000 7f0861cc0700 1 -- 172.21.15.100:0/3922275343 learned_addr learned my addr 172.21.15.100:0/3922275343 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:29:10.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.913+0000 7f0862cc2700 1 --2- 172.21.15.100:0/3922275343 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f085c109510 0x7f085c199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:10.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.913+0000 7f0861cc0700 1 -- 172.21.15.100:0/3922275343 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f085c109510 msgr2=0x7f085c199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:10.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.913+0000 7f0861cc0700 1 --2- 172.21.15.100:0/3922275343 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f085c109510 0x7f085c199870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:10.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.913+0000 7f0861cc0700 1 -- 172.21.15.100:0/3922275343 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f085c06bc30 msgr2=0x7f085c19f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:10.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.913+0000 7f0861cc0700 1 --2- 172.21.15.100:0/3922275343 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f085c06bc30 0x7f085c19f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:10.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.913+0000 7f0861cc0700 1 -- 172.21.15.100:0/3922275343 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f085c199fd0 con 0x7f085c06c5f0 2024-05-10T11:29:10.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.913+0000 7f0861cc0700 1 --2- 172.21.15.100:0/3922275343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f085c06c5f0 0x7f085c19fbf0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f084c006010 tx=0x7f084c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:10.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.914+0000 7f08537fe700 1 -- 172.21.15.100:0/3922275343 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084c027070 con 0x7f085c06c5f0 2024-05-10T11:29:10.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.914+0000 7f08537fe700 1 -- 172.21.15.100:0/3922275343 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f084c022470 con 0x7f085c06c5f0 2024-05-10T11:29:10.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.914+0000 7f0864725700 1 -- 172.21.15.100:0/3922275343 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f085c19a260 con 0x7f085c06c5f0 2024-05-10T11:29:10.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.914+0000 7f08537fe700 1 -- 172.21.15.100:0/3922275343 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084c017650 con 0x7f085c06c5f0 2024-05-10T11:29:10.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.914+0000 7f0864725700 1 -- 172.21.15.100:0/3922275343 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f085c1a6580 con 0x7f085c06c5f0 2024-05-10T11:29:10.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.917+0000 7f08537fe700 1 -- 172.21.15.100:0/3922275343 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f084c009d10 con 0x7f085c06c5f0 2024-05-10T11:29:10.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.917+0000 7f08537fe700 1 --2- 172.21.15.100:0/3922275343 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f08480b2900 0x7f08480b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:10.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.917+0000 7f08537fe700 1 -- 172.21.15.100:0/3922275343 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f084c0e2fc0 con 0x7f085c06c5f0 2024-05-10T11:29:10.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.917+0000 7f08624c1700 1 --2- 172.21.15.100:0/3922275343 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f08480b2900 0x7f08480b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:10.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.918+0000 7f086efb8b80 1 Processor -- start 2024-05-10T11:29:10.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.918+0000 7f08624c1700 1 --2- 172.21.15.100:0/3922275343 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f08480b2900 0x7f08480b4dc0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f085c19a9b0 tx=0x7f0858009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:10.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.918+0000 7f086efb8b80 1 -- start start 2024-05-10T11:29:10.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.918+0000 7f086efb8b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565863b01b0 0x5565863b0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:10.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.918+0000 7f086efb8b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565863b0ad0 0x5565863aa810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:10.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.918+0000 7f086efb8b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5565863aad50 0x5565863ab190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:10.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.918+0000 7f086efb8b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5565862fc940 con 0x5565863aad50 2024-05-10T11:29:10.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.918+0000 7f086efb8b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556586261da0 con 0x5565863b0ad0 2024-05-10T11:29:10.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.918+0000 7f086efb8b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5565862f8900 con 0x5565863b01b0 2024-05-10T11:29:10.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.918+0000 7f0861cc0700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565863b0ad0 0x5565863aa810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:10.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.918+0000 7f0861cc0700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565863b0ad0 0x5565863aa810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:52252/0 (socket says 172.21.15.100:52252) 2024-05-10T11:29:10.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.918+0000 7f08624c1700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565863b01b0 0x5565863b0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:10.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.918+0000 7f0862cc2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5565863aad50 0x5565863ab190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:10.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.918+0000 7f0861cc0700 1 -- 172.21.15.100:0/4228898233 learned_addr learned my addr 172.21.15.100:0/4228898233 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:29:10.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.919+0000 7f0861cc0700 1 -- 172.21.15.100:0/4228898233 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565863b01b0 msgr2=0x5565863b0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:10.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.919+0000 7f0861cc0700 1 --2- 172.21.15.100:0/4228898233 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565863b01b0 0x5565863b0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:10.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.919+0000 7f0861cc0700 1 -- 172.21.15.100:0/4228898233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5565863aad50 msgr2=0x5565863ab190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:10.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.919+0000 7f0861cc0700 1 --2- 172.21.15.100:0/4228898233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5565863aad50 0x5565863ab190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:10.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.919+0000 7f0861cc0700 1 -- 172.21.15.100:0/4228898233 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565862498d0 con 0x5565863b0ad0 2024-05-10T11:29:10.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.919+0000 7f0861cc0700 1 --2- 172.21.15.100:0/4228898233 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565863b0ad0 0x5565863aa810 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f084c0059d0 tx=0x7f084c0a9af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56b80199ba03a411 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:10.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.919+0000 7f0850ff9700 1 -- 172.21.15.100:0/4228898233 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084c0aa2b0 con 0x5565863b0ad0 2024-05-10T11:29:10.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.919+0000 7f0850ff9700 1 -- 172.21.15.100:0/4228898233 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f084c0ae660 con 0x5565863b0ad0 2024-05-10T11:29:10.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.919+0000 7f0850ff9700 1 -- 172.21.15.100:0/4228898233 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084c0b2bf0 con 0x5565863b0ad0 2024-05-10T11:29:10.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.920+0000 7f086efb8b80 1 -- 172.21.15.100:0/4228898233 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565863b0ad0 msgr2=0x5565863aa810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:10.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.920+0000 7f086efb8b80 1 --2- 172.21.15.100:0/4228898233 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565863b0ad0 0x5565863aa810 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f084c0059d0 tx=0x7f084c0a9af0 comp rx=0 tx=0).stop 2024-05-10T11:29:10.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.920+0000 7f086efb8b80 1 -- 172.21.15.100:0/4228898233 shutdown_connections 2024-05-10T11:29:10.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.920+0000 7f086efb8b80 1 --2- 172.21.15.100:0/4228898233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5565863aad50 0x5565863ab190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:10.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.920+0000 7f086efb8b80 1 --2- 172.21.15.100:0/4228898233 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565863b01b0 0x5565863b0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:10.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.920+0000 7f086efb8b80 1 --2- 172.21.15.100:0/4228898233 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565863b0ad0 0x5565863aa810 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:10.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.920+0000 7f086efb8b80 1 -- 172.21.15.100:0/4228898233 >> 172.21.15.100:0/4228898233 conn(0x5565862d6b70 msgr2=0x5565863a52a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:10.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.920+0000 7f086efb8b80 1 -- 172.21.15.100:0/4228898233 shutdown_connections 2024-05-10T11:29:10.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.920+0000 7f086efb8b80 1 -- 172.21.15.100:0/4228898233 wait complete. 2024-05-10T11:29:10.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.920+0000 7f086efb8b80 1 -- start start 2024-05-10T11:29:10.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.921+0000 7f086efb8b80 1 Processor -- start 2024-05-10T11:29:10.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.921+0000 7f086efb8b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5565863aad50 0x5565863b6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:10.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.921+0000 7f086efb8b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565863b01b0 0x5565863b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:10.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.921+0000 7f086efb8b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565863b0ad0 0x5565863b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:10.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.921+0000 7f086efb8b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5565863b7210 con 0x5565863aad50 2024-05-10T11:29:10.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.921+0000 7f086efb8b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5565862b5ed0 con 0x5565863b01b0 2024-05-10T11:29:10.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.921+0000 7f086efb8b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5565863b7390 con 0x5565863b0ad0 2024-05-10T11:29:10.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.921+0000 7f086efb8b80 10 client.0 _ll_register_callbacks cb 0x5565862905b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:29:10.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.921+0000 7f08624c1700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5565863aad50 0x5565863b6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:10.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.921+0000 7f08624c1700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5565863aad50 0x5565863b6700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41652/0 (socket says 172.21.15.100:41652) 2024-05-10T11:29:10.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.921+0000 7f08624c1700 1 -- 172.21.15.100:0/1784836196 learned_addr learned my addr 172.21.15.100:0/1784836196 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:29:10.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.921+0000 7f0861cc0700 1 --2- 172.21.15.100:0/1784836196 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565863b01b0 0x5565863b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:10.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.921+0000 7f0862cc2700 1 --2- 172.21.15.100:0/1784836196 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565863b0ad0 0x5565863b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:10.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.921+0000 7f08624c1700 1 -- 172.21.15.100:0/1784836196 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565863b0ad0 msgr2=0x5565863b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:10.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.921+0000 7f08624c1700 1 --2- 172.21.15.100:0/1784836196 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565863b0ad0 0x5565863b3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:10.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.921+0000 7f08624c1700 1 -- 172.21.15.100:0/1784836196 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565863b01b0 msgr2=0x5565863b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:10.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.921+0000 7f08624c1700 1 --2- 172.21.15.100:0/1784836196 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565863b01b0 0x5565863b3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:10.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.921+0000 7f08624c1700 1 -- 172.21.15.100:0/1784836196 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565862b3760 con 0x5565863aad50 2024-05-10T11:29:10.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.922+0000 7f08624c1700 1 --2- 172.21.15.100:0/1784836196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5565863aad50 0x5565863b6700 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f085801f850 tx=0x7f08580047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f61bca835cb3476 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:10.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.922+0000 7f0845ffb700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:29:10.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.922+0000 7f0845ffb700 1 -- 172.21.15.100:0/1784836196 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0858004320 con 0x5565863aad50 2024-05-10T11:29:10.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.922+0000 7f0845ffb700 1 -- 172.21.15.100:0/1784836196 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f08580044c0 con 0x5565863aad50 2024-05-10T11:29:10.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.922+0000 7f086efb8b80 1 -- 172.21.15.100:0/1784836196 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f084c0ae660 con 0x5565863aad50 2024-05-10T11:29:10.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.922+0000 7f0845ffb700 1 -- 172.21.15.100:0/1784836196 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f0858004830 con 0x5565863aad50 2024-05-10T11:29:10.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.922+0000 7f086efb8b80 10 client.17613 fetch_fsmap learned FSMap version 33 2024-05-10T11:29:10.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.922+0000 7f086efb8b80 1 -- 172.21.15.100:0/1784836196 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5565863c1a70 con 0x5565863aad50 2024-05-10T11:29:10.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.922+0000 7f0845ffb700 1 -- 172.21.15.100:0/1784836196 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f084c0ae660 con 0x5565863aad50 2024-05-10T11:29:10.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.923+0000 7f0845ffb700 1 -- 172.21.15.100:0/1784836196 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f0858026070 con 0x5565863aad50 2024-05-10T11:29:10.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.923+0000 7f086efb8b80 10 client.17613 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T11:29:10.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.923+0000 7f086efb8b80 10 client.17613 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:29:10.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.923+0000 7f086efb8b80 20 client.17613 populate_metadata read hostname 'smithi100' 2024-05-10T11:29:10.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.923+0000 7f086efb8b80 1 --2- 172.21.15.100:0/1784836196 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5565863c40b0 0x5565863c64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:10.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.923+0000 7f086efb8b80 4 client.17613 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:29:10.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.923+0000 7f086efb8b80 1 -- 172.21.15.100:0/1784836196 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55658630a320 con 0x5565863c40b0 2024-05-10T11:29:10.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.923+0000 7f0861cc0700 1 --2- 172.21.15.100:0/1784836196 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5565863c40b0 0x5565863c64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:10.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.924+0000 7f0845ffb700 10 client.17613 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:29:10.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.924+0000 7f0861cc0700 1 --2- 172.21.15.100:0/1784836196 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5565863c40b0 0x5565863c64f0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=925d1c12f44a382b server_cookie=7d4a9d09e5ef4ba7 in_seq=0 out_seq=0 2024-05-10T11:29:10.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.924+0000 7f0845ffb700 1 -- 172.21.15.100:0/1784836196 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55658630a320 con 0x5565863c40b0 2024-05-10T11:29:10.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.924+0000 7f0845ffb700 10 client.17613 handle_command_reply: tid=0 2024-05-10T11:29:10.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.924+0000 7f086efb8b80 1 client.17613 shutdown 2024-05-10T11:29:10.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.924+0000 7f086efb8b80 1 -- 172.21.15.100:0/1784836196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5565863aad50 msgr2=0x5565863b6700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:10.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.924+0000 7f086efb8b80 1 --2- 172.21.15.100:0/1784836196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5565863aad50 0x5565863b6700 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f085801f850 tx=0x7f08580047c0 comp rx=0 tx=0).stop 2024-05-10T11:29:10.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.924+0000 7f086efb8b80 1 -- 172.21.15.100:0/1784836196 shutdown_connections 2024-05-10T11:29:10.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.924+0000 7f086efb8b80 1 --2- 172.21.15.100:0/1784836196 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5565863c40b0 0x5565863c64f0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:10.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.924+0000 7f086efb8b80 1 --2- 172.21.15.100:0/1784836196 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565863b0ad0 0x5565863b3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:10.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.924+0000 7f086efb8b80 1 --2- 172.21.15.100:0/1784836196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5565863aad50 0x5565863b6700 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:10.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.924+0000 7f0845ffb700 0 client.17613 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:29:10.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.924+0000 7f086efb8b80 1 --2- 172.21.15.100:0/1784836196 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565863b01b0 0x5565863b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:10.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.924+0000 7f086efb8b80 1 -- 172.21.15.100:0/1784836196 >> 172.21.15.100:0/1784836196 conn(0x5565862d6b70 msgr2=0x55658639fb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:10.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.925+0000 7f086efb8b80 1 -- 172.21.15.100:0/1784836196 shutdown_connections 2024-05-10T11:29:10.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.925+0000 7f086efb8b80 1 -- 172.21.15.100:0/1784836196 wait complete. 2024-05-10T11:29:10.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.925+0000 7f086efb8b80 20 client.17613 trim_cache size 0 max 16384 2024-05-10T11:29:10.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.947+0000 7f086efb8b80 1 Processor -- start 2024-05-10T11:29:10.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.947+0000 7f086efb8b80 1 -- start start 2024-05-10T11:29:10.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.947+0000 7f086efb8b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565863b01b0 0x5565863aa810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:10.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.947+0000 7f086efb8b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565863b0ad0 0x5565863c0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:10.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.947+0000 7f086efb8b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5565863b1770 0x55658639fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:10.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.947+0000 7f086efb8b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f085c1b3a40 con 0x5565863b1770 2024-05-10T11:29:10.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.947+0000 7f086efb8b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f085c1b12d0 con 0x5565863b0ad0 2024-05-10T11:29:10.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.947+0000 7f086efb8b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f085c1af120 con 0x5565863b01b0 2024-05-10T11:29:10.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.947+0000 7f0862cc2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5565863b1770 0x55658639fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:10.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.947+0000 7f0862cc2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5565863b1770 0x55658639fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41664/0 (socket says 172.21.15.100:41664) 2024-05-10T11:29:10.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.947+0000 7f0862cc2700 1 -- 172.21.15.100:0/385605544 learned_addr learned my addr 172.21.15.100:0/385605544 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:29:10.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.947+0000 7f0861cc0700 1 --2- 172.21.15.100:0/385605544 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565863b0ad0 0x5565863c0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:10.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.947+0000 7f08624c1700 1 --2- 172.21.15.100:0/385605544 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565863b01b0 0x5565863aa810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:10.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.947+0000 7f0862cc2700 1 -- 172.21.15.100:0/385605544 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565863b01b0 msgr2=0x5565863aa810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:10.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.947+0000 7f0862cc2700 1 --2- 172.21.15.100:0/385605544 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565863b01b0 0x5565863aa810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:10.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.947+0000 7f0862cc2700 1 -- 172.21.15.100:0/385605544 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565863b0ad0 msgr2=0x5565863c0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:10.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.947+0000 7f0862cc2700 1 --2- 172.21.15.100:0/385605544 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565863b0ad0 0x5565863c0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:10.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.947+0000 7f0862cc2700 1 -- 172.21.15.100:0/385605544 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565863b2010 con 0x5565863b1770 2024-05-10T11:29:10.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.947+0000 7f08624c1700 1 --2- 172.21.15.100:0/385605544 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565863b01b0 0x5565863aa810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:29:10.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.947+0000 7f0862cc2700 1 --2- 172.21.15.100:0/385605544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5565863b1770 0x55658639fc40 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f08540119d0 tx=0x7f0854009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=816efc0ab5108ad8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:10.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.948+0000 7f0847fff700 1 -- 172.21.15.100:0/385605544 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0854017630 con 0x5565863b1770 2024-05-10T11:29:10.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.948+0000 7f0847fff700 1 -- 172.21.15.100:0/385605544 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0854017c60 con 0x5565863b1770 2024-05-10T11:29:10.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.948+0000 7f0847fff700 1 -- 172.21.15.100:0/385605544 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0854007d70 con 0x5565863b1770 2024-05-10T11:29:10.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.948+0000 7f086efb8b80 1 -- 172.21.15.100:0/385605544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5565863b1770 msgr2=0x55658639fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:10.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.948+0000 7f086efb8b80 1 --2- 172.21.15.100:0/385605544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5565863b1770 0x55658639fc40 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f08540119d0 tx=0x7f0854009920 comp rx=0 tx=0).stop 2024-05-10T11:29:10.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.948+0000 7f086efb8b80 1 -- 172.21.15.100:0/385605544 shutdown_connections 2024-05-10T11:29:10.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.948+0000 7f086efb8b80 1 --2- 172.21.15.100:0/385605544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5565863b1770 0x55658639fc40 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:10.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.948+0000 7f086efb8b80 1 --2- 172.21.15.100:0/385605544 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565863b01b0 0x5565863aa810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:10.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.948+0000 7f086efb8b80 1 --2- 172.21.15.100:0/385605544 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565863b0ad0 0x5565863c0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:10.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.948+0000 7f086efb8b80 1 -- 172.21.15.100:0/385605544 >> 172.21.15.100:0/385605544 conn(0x5565862d6b70 msgr2=0x5565863aebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:10.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.949+0000 7f086efb8b80 1 -- 172.21.15.100:0/385605544 shutdown_connections 2024-05-10T11:29:10.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.949+0000 7f086efb8b80 1 -- 172.21.15.100:0/385605544 wait complete. 2024-05-10T11:29:10.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.949+0000 7f086efb8b80 1 -- start start 2024-05-10T11:29:10.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.949+0000 7f086efb8b80 1 Processor -- start 2024-05-10T11:29:10.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.949+0000 7f086efb8b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5565863b01b0 0x5565863bc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:10.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.949+0000 7f086efb8b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565863b0ad0 0x5565863bcb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:10.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.949+0000 7f086efb8b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565863b1770 0x5565863b95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:10.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.949+0000 7f086efb8b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f085c1af790 con 0x5565863b01b0 2024-05-10T11:29:10.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.949+0000 7f086efb8b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f085c1b0c00 con 0x5565863b0ad0 2024-05-10T11:29:10.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.949+0000 7f086efb8b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f085c1afe60 con 0x5565863b1770 2024-05-10T11:29:10.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.949+0000 7f086efb8b80 10 client.0 _ll_register_callbacks cb 0x5565862905b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:29:10.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.949+0000 7f08624c1700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5565863b01b0 0x5565863bc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:10.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.949+0000 7f08624c1700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5565863b01b0 0x5565863bc5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41666/0 (socket says 172.21.15.100:41666) 2024-05-10T11:29:10.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.949+0000 7f0861cc0700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565863b0ad0 0x5565863bcb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:10.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.949+0000 7f08624c1700 1 -- 172.21.15.100:0/1676495672 learned_addr learned my addr 172.21.15.100:0/1676495672 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:29:10.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.949+0000 7f0862cc2700 1 --2- 172.21.15.100:0/1676495672 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565863b1770 0x5565863b95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:10.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.950+0000 7f08624c1700 1 -- 172.21.15.100:0/1676495672 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565863b1770 msgr2=0x5565863b95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:10.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.950+0000 7f08624c1700 1 --2- 172.21.15.100:0/1676495672 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565863b1770 0x5565863b95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:10.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.950+0000 7f08624c1700 1 -- 172.21.15.100:0/1676495672 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565863b0ad0 msgr2=0x5565863bcb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:10.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.950+0000 7f08624c1700 1 --2- 172.21.15.100:0/1676495672 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565863b0ad0 0x5565863bcb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:10.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.950+0000 7f08624c1700 1 -- 172.21.15.100:0/1676495672 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565863b9af0 con 0x5565863b01b0 2024-05-10T11:29:10.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.950+0000 7f08624c1700 1 --2- 172.21.15.100:0/1676495672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5565863b01b0 0x5565863bc5e0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f0858016040 tx=0x7f0858003af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=315938dcebe43ccd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:10.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.950+0000 7f0845ffb700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:29:10.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.950+0000 7f0845ffb700 1 -- 172.21.15.100:0/1676495672 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085803a060 con 0x5565863b01b0 2024-05-10T11:29:10.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.950+0000 7f0845ffb700 1 -- 172.21.15.100:0/1676495672 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f0858035440 con 0x5565863b01b0 2024-05-10T11:29:10.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.950+0000 7f086efb8b80 1 -- 172.21.15.100:0/1676495672 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0854017c60 con 0x5565863b01b0 2024-05-10T11:29:10.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.950+0000 7f0845ffb700 1 -- 172.21.15.100:0/1676495672 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f0858041050 con 0x5565863b01b0 2024-05-10T11:29:10.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.950+0000 7f086efb8b80 10 client.17619 fetch_fsmap learned FSMap version 33 2024-05-10T11:29:10.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.950+0000 7f086efb8b80 1 -- 172.21.15.100:0/1676495672 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55658642f150 con 0x5565863b01b0 2024-05-10T11:29:10.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.951+0000 7f0845ffb700 1 -- 172.21.15.100:0/1676495672 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0854017c60 con 0x5565863b01b0 2024-05-10T11:29:10.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.951+0000 7f0845ffb700 1 -- 172.21.15.100:0/1676495672 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f0858026040 con 0x5565863b01b0 2024-05-10T11:29:10.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.951+0000 7f086efb8b80 10 client.17619 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T11:29:10.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.951+0000 7f086efb8b80 10 client.17619 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:29:10.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.951+0000 7f086efb8b80 20 client.17619 populate_metadata read hostname 'smithi100' 2024-05-10T11:29:10.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.951+0000 7f086efb8b80 1 --2- 172.21.15.100:0/1676495672 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55658642f830 0x556586431c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:10.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.951+0000 7f086efb8b80 4 client.17619 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T11:29:10.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.951+0000 7f086efb8b80 1 -- 172.21.15.100:0/1676495672 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5565863043c0 con 0x55658642f830 2024-05-10T11:29:10.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.951+0000 7f0861cc0700 1 --2- 172.21.15.100:0/1676495672 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55658642f830 0x556586431c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:10.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.952+0000 7f0861cc0700 1 --2- 172.21.15.100:0/1676495672 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55658642f830 0x556586431c50 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=424e9d0f65195df2 server_cookie=9a4c7d733e50c7b4 in_seq=0 out_seq=0 2024-05-10T11:29:10.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.952+0000 7f0845ffb700 10 client.17619 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:29:10.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.952+0000 7f0845ffb700 1 -- 172.21.15.100:0/1676495672 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5565863043c0 con 0x55658642f830 2024-05-10T11:29:10.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.952+0000 7f0845ffb700 10 client.17619 handle_command_reply: tid=0 2024-05-10T11:29:10.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.952+0000 7f086efb8b80 1 client.17619 shutdown 2024-05-10T11:29:10.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.952+0000 7f086efb8b80 1 -- 172.21.15.100:0/1676495672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5565863b01b0 msgr2=0x5565863bc5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:10.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.952+0000 7f086efb8b80 1 --2- 172.21.15.100:0/1676495672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5565863b01b0 0x5565863bc5e0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f0858016040 tx=0x7f0858003af0 comp rx=0 tx=0).stop 2024-05-10T11:29:10.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.952+0000 7f086efb8b80 1 -- 172.21.15.100:0/1676495672 shutdown_connections 2024-05-10T11:29:10.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.952+0000 7f086efb8b80 1 --2- 172.21.15.100:0/1676495672 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55658642f830 0x556586431c50 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:10.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.952+0000 7f086efb8b80 1 --2- 172.21.15.100:0/1676495672 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565863b1770 0x5565863b95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:10.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.952+0000 7f086efb8b80 1 --2- 172.21.15.100:0/1676495672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5565863b01b0 0x5565863bc5e0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:10.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.952+0000 7f086efb8b80 1 --2- 172.21.15.100:0/1676495672 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565863b0ad0 0x5565863bcb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:10.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.952+0000 7f086efb8b80 1 -- 172.21.15.100:0/1676495672 >> 172.21.15.100:0/1676495672 conn(0x5565862d6b70 msgr2=0x5565863b3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:10.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.952+0000 7f0845ffb700 0 client.17619 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:29:10.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.952+0000 7f086efb8b80 1 -- 172.21.15.100:0/1676495672 shutdown_connections 2024-05-10T11:29:10.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.952+0000 7f086efb8b80 1 -- 172.21.15.100:0/1676495672 wait complete. 2024-05-10T11:29:10.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.952+0000 7f086efb8b80 20 client.17619 trim_cache size 0 max 16384 2024-05-10T11:29:10.957 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:29:10.957 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T11:29:10.958 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-05-10T11:29:10.958 INFO:teuthology.orchestra.run.smithi100.stdout: "1f2d16ea-6e3b-4fcd-92d7-fc4e1bf3c1f2": { 2024-05-10T11:29:10.958 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-05-10T11:29:10.958 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "1f2d16ea-6e3b-4fcd-92d7-fc4e1bf3c1f2", 2024-05-10T11:29:10.958 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-05-10T11:29:10.958 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:29:10.958 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:29:10.958 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:29:10.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.953+0000 7f0864725700 1 -- 172.21.15.100:0/3922275343 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f08480b2900 msgr2=0x7f08480b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:10.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.953+0000 7f0864725700 1 --2- 172.21.15.100:0/3922275343 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f08480b2900 0x7f08480b4dc0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f085c19a9b0 tx=0x7f0858009260 comp rx=0 tx=0).stop 2024-05-10T11:29:10.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.953+0000 7f0864725700 1 -- 172.21.15.100:0/3922275343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f085c06c5f0 msgr2=0x7f085c19fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:10.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.953+0000 7f0864725700 1 --2- 172.21.15.100:0/3922275343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f085c06c5f0 0x7f085c19fbf0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f084c006010 tx=0x7f084c0047b0 comp rx=0 tx=0).stop 2024-05-10T11:29:10.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.953+0000 7f0864725700 1 -- 172.21.15.100:0/3922275343 shutdown_connections 2024-05-10T11:29:10.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.953+0000 7f0864725700 1 --2- 172.21.15.100:0/3922275343 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f08480b2900 0x7f08480b4dc0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:10.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.953+0000 7f0864725700 1 --2- 172.21.15.100:0/3922275343 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f085c109510 0x7f085c199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:10.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.953+0000 7f0864725700 1 --2- 172.21.15.100:0/3922275343 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f085c06bc30 0x7f085c19f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:10.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.953+0000 7f0864725700 1 --2- 172.21.15.100:0/3922275343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f085c06c5f0 0x7f085c19fbf0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:10.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.953+0000 7f0864725700 1 -- 172.21.15.100:0/3922275343 >> 172.21.15.100:0/3922275343 conn(0x7f085c079350 msgr2=0x7f085c101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:10.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.953+0000 7f0864725700 1 -- 172.21.15.100:0/3922275343 shutdown_connections 2024-05-10T11:29:10.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:10.953+0000 7f0864725700 1 -- 172.21.15.100:0/3922275343 wait complete. 2024-05-10T11:29:10.980 INFO:tasks.workunit.client.0.smithi100.stdout: 1 279093 2.72 MB/sec execute 425 sec latency 46.390 ms 2024-05-10T11:29:11.640 INFO:tasks.cephfs.filesystem:scrub status for tag:1f2d16ea-6e3b-4fcd-92d7-fc4e1bf3c1f2 - {'path': '/', 'tag': '1f2d16ea-6e3b-4fcd-92d7-fc4e1bf3c1f2', 'options': 'recursive,force'} 2024-05-10T11:29:11.980 INFO:tasks.workunit.client.0.smithi100.stdout: 1 279706 2.72 MB/sec execute 426 sec latency 15.103 ms 2024-05-10T11:29:12.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:29:11 smithi178 ceph-mon[33359]: pgmap v745: 129 pgs: 129 active+clean; 109 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 1.3 MiB/s wr, 160 op/s 2024-05-10T11:29:12.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:29:11 smithi188 ceph-mon[34075]: pgmap v745: 129 pgs: 129 active+clean; 109 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 1.3 MiB/s wr, 160 op/s 2024-05-10T11:29:12.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:29:11 smithi100 ceph-mon[30296]: pgmap v745: 129 pgs: 129 active+clean; 109 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 1.3 MiB/s wr, 160 op/s 2024-05-10T11:29:12.981 INFO:tasks.workunit.client.0.smithi100.stdout: 1 280241 2.73 MB/sec execute 427 sec latency 56.246 ms 2024-05-10T11:29:13.980 INFO:tasks.workunit.client.0.smithi100.stdout: 1 280599 2.73 MB/sec execute 428 sec latency 13.129 ms 2024-05-10T11:29:14.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:29:13 smithi178 ceph-mon[33359]: pgmap v746: 129 pgs: 129 active+clean; 109 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 1.2 MiB/s wr, 168 op/s 2024-05-10T11:29:14.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:29:13 smithi188 ceph-mon[34075]: pgmap v746: 129 pgs: 129 active+clean; 109 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 1.2 MiB/s wr, 168 op/s 2024-05-10T11:29:14.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:29:13 smithi100 ceph-mon[30296]: pgmap v746: 129 pgs: 129 active+clean; 109 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 1.2 MiB/s wr, 168 op/s 2024-05-10T11:29:14.980 INFO:tasks.workunit.client.0.smithi100.stdout: 1 280859 2.72 MB/sec execute 429 sec latency 16.025 ms 2024-05-10T11:29:15.981 INFO:tasks.workunit.client.0.smithi100.stdout: 1 281364 2.72 MB/sec execute 430 sec latency 15.921 ms 2024-05-10T11:29:16.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:29:15 smithi178 ceph-mon[33359]: pgmap v747: 129 pgs: 129 active+clean; 109 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 1.2 MiB/s wr, 144 op/s 2024-05-10T11:29:16.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:29:15 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:29:16.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:29:15 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:29:16.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:29:15 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:29:16.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:29:15 smithi188 ceph-mon[34075]: pgmap v747: 129 pgs: 129 active+clean; 109 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 1.2 MiB/s wr, 144 op/s 2024-05-10T11:29:16.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:29:15 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:29:16.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:29:15 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:29:16.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:29:15 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:29:16.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:29:15 smithi100 ceph-mon[30296]: pgmap v747: 129 pgs: 129 active+clean; 109 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 1.2 MiB/s wr, 144 op/s 2024-05-10T11:29:16.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:29:15 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:29:16.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:29:15 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:29:16.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:29:15 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:29:16.981 INFO:tasks.workunit.client.0.smithi100.stdout: 1 281892 2.72 MB/sec execute 431 sec latency 15.453 ms 2024-05-10T11:29:17.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:29:17 smithi100 ceph-mon[30296]: pgmap v748: 129 pgs: 129 active+clean; 111 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 1.3 MiB/s wr, 161 op/s 2024-05-10T11:29:17.981 INFO:tasks.workunit.client.0.smithi100.stdout: 1 282567 2.72 MB/sec execute 432 sec latency 15.555 ms 2024-05-10T11:29:18.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:29:17 smithi178 ceph-mon[33359]: pgmap v748: 129 pgs: 129 active+clean; 111 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 1.3 MiB/s wr, 161 op/s 2024-05-10T11:29:18.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:29:17 smithi188 ceph-mon[34075]: pgmap v748: 129 pgs: 129 active+clean; 111 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 1.3 MiB/s wr, 161 op/s 2024-05-10T11:29:18.980 INFO:tasks.workunit.client.0.smithi100.stdout: 1 283120 2.72 MB/sec execute 433 sec latency 15.308 ms 2024-05-10T11:29:19.981 INFO:tasks.workunit.client.0.smithi100.stdout: 1 283629 2.72 MB/sec execute 434 sec latency 12.988 ms 2024-05-10T11:29:20.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:29:19 smithi178 ceph-mon[33359]: pgmap v749: 129 pgs: 129 active+clean; 111 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 1.5 MiB/s wr, 177 op/s 2024-05-10T11:29:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.007+0000 7f55410b4640 1 -- 172.21.15.100:0/473221391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f553c102b20 msgr2=0x7f553c102fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.007+0000 7f55410b4640 1 --2- 172.21.15.100:0/473221391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f553c102b20 0x7f553c102fa0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f5524009900 tx=0x7f552402a410 comp rx=0 tx=0).stop 2024-05-10T11:29:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.007+0000 7f55410b4640 1 -- 172.21.15.100:0/473221391 shutdown_connections 2024-05-10T11:29:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.007+0000 7f55410b4640 1 --2- 172.21.15.100:0/473221391 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f553c1034e0 0x7f553c109d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.007+0000 7f55410b4640 1 --2- 172.21.15.100:0/473221391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f553c102b20 0x7f553c102fa0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.007+0000 7f55410b4640 1 --2- 172.21.15.100:0/473221391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f553c101930 0x7f553c101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.007+0000 7f55410b4640 1 -- 172.21.15.100:0/473221391 >> 172.21.15.100:0/473221391 conn(0x7f553c0fcea0 msgr2=0x7f553c0ff2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.007+0000 7f55410b4640 1 -- 172.21.15.100:0/473221391 shutdown_connections 2024-05-10T11:29:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.007+0000 7f55410b4640 1 -- 172.21.15.100:0/473221391 wait complete. 2024-05-10T11:29:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.008+0000 7f55410b4640 1 Processor -- start 2024-05-10T11:29:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.008+0000 7f55410b4640 1 -- start start 2024-05-10T11:29:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.008+0000 7f55410b4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f553c101930 0x7f553c06bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.008+0000 7f55410b4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f553c102b20 0x7f553c19d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.008+0000 7f55410b4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f553c1034e0 0x7f553c19fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.008+0000 7f55410b4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f553c10d310 con 0x7f553c1034e0 2024-05-10T11:29:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.008+0000 7f55410b4640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f553c10d190 con 0x7f553c101930 2024-05-10T11:29:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.008+0000 7f55410b4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f553c10d490 con 0x7f553c102b20 2024-05-10T11:29:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.008+0000 7f5533fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f553c102b20 0x7f553c19d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.008+0000 7f553b59e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f553c1034e0 0x7f553c19fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.008+0000 7f5533fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f553c102b20 0x7f553c19d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:53092/0 (socket says 172.21.15.100:53092) 2024-05-10T11:29:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.008+0000 7f5533fff640 1 -- 172.21.15.100:0/3947311570 learned_addr learned my addr 172.21.15.100:0/3947311570 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:29:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.009+0000 7f553ad9d640 1 --2- 172.21.15.100:0/3947311570 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f553c101930 0x7f553c06bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.009+0000 7f5533fff640 1 -- 172.21.15.100:0/3947311570 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f553c101930 msgr2=0x7f553c06bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.009+0000 7f5533fff640 1 --2- 172.21.15.100:0/3947311570 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f553c101930 0x7f553c06bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.009+0000 7f5533fff640 1 -- 172.21.15.100:0/3947311570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f553c1034e0 msgr2=0x7f553c19fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.009+0000 7f5533fff640 1 --2- 172.21.15.100:0/3947311570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f553c1034e0 0x7f553c19fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.009+0000 7f5533fff640 1 -- 172.21.15.100:0/3947311570 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f553c1a0700 con 0x7f553c102b20 2024-05-10T11:29:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.009+0000 7f553b59e640 1 --2- 172.21.15.100:0/3947311570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f553c1034e0 0x7f553c19fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:29:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.009+0000 7f5533fff640 1 --2- 172.21.15.100:0/3947311570 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f553c102b20 0x7f553c19d7d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5524009a20 tx=0x7f55240040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.009+0000 7f5538d99640 1 -- 172.21.15.100:0/3947311570 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552403f030 con 0x7f553c102b20 2024-05-10T11:29:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.009+0000 7f55410b4640 1 -- 172.21.15.100:0/3947311570 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f553c1a0990 con 0x7f553c102b20 2024-05-10T11:29:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.009+0000 7f5538d99640 1 -- 172.21.15.100:0/3947311570 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5524003a80 con 0x7f553c102b20 2024-05-10T11:29:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.009+0000 7f5538d99640 1 -- 172.21.15.100:0/3947311570 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5524035070 con 0x7f553c102b20 2024-05-10T11:29:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.009+0000 7f55410b4640 1 -- 172.21.15.100:0/3947311570 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f553c1a0e80 con 0x7f553c102b20 2024-05-10T11:29:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.012+0000 7f5538d99640 1 -- 172.21.15.100:0/3947311570 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f5524042030 con 0x7f553c102b20 2024-05-10T11:29:20.014 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.012+0000 7f5543dd0740 1 Processor -- start 2024-05-10T11:29:20.014 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.012+0000 7f5543dd0740 1 -- start start 2024-05-10T11:29:20.014 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.012+0000 7f5543dd0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bfa5a11c0 0x555bfa5a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:20.014 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.012+0000 7f5543dd0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555bfa5a1bb0 0x555bfa5a66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:20.014 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.012+0000 7f5543dd0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555bfa5a6d20 0x555bfa5a91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:20.014 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.012+0000 7f5543dd0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x555bfa5015d0 con 0x555bfa5a11c0 2024-05-10T11:29:20.014 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.012+0000 7f5543dd0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x555bfa4ba890 con 0x555bfa5a1bb0 2024-05-10T11:29:20.014 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.012+0000 7f5543dd0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x555bfa4fd590 con 0x555bfa5a6d20 2024-05-10T11:29:20.014 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.013+0000 7f553b59e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bfa5a11c0 0x555bfa5a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:20.014 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.013+0000 7f553b59e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bfa5a11c0 0x555bfa5a15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43050/0 (socket says 172.21.15.100:43050) 2024-05-10T11:29:20.014 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.013+0000 7f553b59e640 1 -- 172.21.15.100:0/1694400870 learned_addr learned my addr 172.21.15.100:0/1694400870 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:29:20.014 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.013+0000 7f5533fff640 1 --2- 172.21.15.100:0/1694400870 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555bfa5a6d20 0x555bfa5a91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:20.015 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.013+0000 7f5538d99640 1 --2- 172.21.15.100:0/3947311570 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f55280b28a0 0x7f55280b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:20.015 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.013+0000 7f553ad9d640 1 --2- 172.21.15.100:0/1694400870 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555bfa5a1bb0 0x555bfa5a66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:20.015 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.013+0000 7f553b59e640 1 -- 172.21.15.100:0/1694400870 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555bfa5a6d20 msgr2=0x555bfa5a91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:20.015 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.013+0000 7f553b59e640 1 --2- 172.21.15.100:0/1694400870 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555bfa5a6d20 0x555bfa5a91b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:20.015 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.013+0000 7f553b59e640 1 -- 172.21.15.100:0/1694400870 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555bfa5a1bb0 msgr2=0x555bfa5a66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:20.015 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.013+0000 7f553b59e640 1 --2- 172.21.15.100:0/1694400870 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555bfa5a1bb0 0x555bfa5a66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:20.015 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.013+0000 7f553b59e640 1 -- 172.21.15.100:0/1694400870 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bfa213210 con 0x555bfa5a11c0 2024-05-10T11:29:20.015 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.013+0000 7f5538d99640 1 -- 172.21.15.100:0/3947311570 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f5524102ae0 con 0x7f553c102b20 2024-05-10T11:29:20.015 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.013+0000 7f553b59e640 1 --2- 172.21.15.100:0/3947311570 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f55280b28a0 0x7f55280b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:20.015 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.013+0000 7f553b59e640 1 --2- 172.21.15.100:0/1694400870 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bfa5a11c0 0x555bfa5a15e0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f553400f890 tx=0x7f553400fd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bf7ea26327c2371 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:20.015 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.013+0000 7f553b59e640 1 --2- 172.21.15.100:0/3947311570 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f55280b28a0 0x7f55280b4d60 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5534021000 tx=0x7f553401e000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:20.015 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.014+0000 7f55317fa640 1 -- 172.21.15.100:0/1694400870 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55340120d0 con 0x555bfa5a11c0 2024-05-10T11:29:20.015 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.014+0000 7f55317fa640 1 -- 172.21.15.100:0/1694400870 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5534012270 con 0x555bfa5a11c0 2024-05-10T11:29:20.015 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.014+0000 7f55317fa640 1 -- 172.21.15.100:0/1694400870 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5534005900 con 0x555bfa5a11c0 2024-05-10T11:29:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.014+0000 7f5543dd0740 1 -- 172.21.15.100:0/1694400870 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bfa5a11c0 msgr2=0x555bfa5a15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.014+0000 7f5543dd0740 1 --2- 172.21.15.100:0/1694400870 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bfa5a11c0 0x555bfa5a15e0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f553400f890 tx=0x7f553400fd50 comp rx=0 tx=0).stop 2024-05-10T11:29:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.014+0000 7f5543dd0740 1 -- 172.21.15.100:0/1694400870 shutdown_connections 2024-05-10T11:29:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.014+0000 7f5543dd0740 1 --2- 172.21.15.100:0/1694400870 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555bfa5a6d20 0x555bfa5a91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.014+0000 7f5543dd0740 1 --2- 172.21.15.100:0/1694400870 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555bfa5a1bb0 0x555bfa5a66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.014+0000 7f5543dd0740 1 --2- 172.21.15.100:0/1694400870 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bfa5a11c0 0x555bfa5a15e0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.014+0000 7f5543dd0740 1 -- 172.21.15.100:0/1694400870 >> 172.21.15.100:0/1694400870 conn(0x555bfa598420 msgr2=0x555bfa598820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.014+0000 7f5543dd0740 1 -- 172.21.15.100:0/1694400870 shutdown_connections 2024-05-10T11:29:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.014+0000 7f5543dd0740 1 -- 172.21.15.100:0/1694400870 wait complete. 2024-05-10T11:29:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.015+0000 7f5543dd0740 1 -- start start 2024-05-10T11:29:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.015+0000 7f5543dd0740 1 Processor -- start 2024-05-10T11:29:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.015+0000 7f5543dd0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555bfa5a11c0 0x555bfa5ab460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.015+0000 7f5543dd0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555bfa5a1bb0 0x555bfa5ab9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.015+0000 7f5543dd0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bfa5a6d20 0x555bfa5b29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.015+0000 7f5543dd0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x555bfa5a9cd0 con 0x555bfa5a6d20 2024-05-10T11:29:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.015+0000 7f5543dd0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x555bfa4be370 con 0x555bfa5a1bb0 2024-05-10T11:29:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.015+0000 7f5543dd0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x555bfa5a9e50 con 0x555bfa5a11c0 2024-05-10T11:29:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.015+0000 7f5543dd0740 10 client.0 _ll_register_callbacks cb 0x555bfa212a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:29:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.015+0000 7f553ad9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555bfa5a11c0 0x555bfa5ab460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.015+0000 7f553ad9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555bfa5a11c0 0x555bfa5ab460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:53116/0 (socket says 172.21.15.100:53116) 2024-05-10T11:29:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.015+0000 7f553b59e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bfa5a6d20 0x555bfa5b29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.015+0000 7f553ad9d640 1 -- 172.21.15.100:0/31476434 learned_addr learned my addr 172.21.15.100:0/31476434 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:29:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.015+0000 7f5533fff640 1 --2- 172.21.15.100:0/31476434 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555bfa5a1bb0 0x555bfa5ab9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.016+0000 7f553ad9d640 1 -- 172.21.15.100:0/31476434 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555bfa5a1bb0 msgr2=0x555bfa5ab9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.016+0000 7f553ad9d640 1 --2- 172.21.15.100:0/31476434 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555bfa5a1bb0 0x555bfa5ab9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.016+0000 7f553ad9d640 1 -- 172.21.15.100:0/31476434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bfa5a6d20 msgr2=0x555bfa5b29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.016+0000 7f553ad9d640 1 --2- 172.21.15.100:0/31476434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bfa5a6d20 0x555bfa5b29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.016+0000 7f553ad9d640 1 -- 172.21.15.100:0/31476434 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bfa2df620 con 0x555bfa5a11c0 2024-05-10T11:29:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.016+0000 7f553b59e640 1 --2- 172.21.15.100:0/31476434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bfa5a6d20 0x555bfa5b29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:29:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.016+0000 7f5533fff640 1 --2- 172.21.15.100:0/31476434 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555bfa5a1bb0 0x555bfa5ab9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:29:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.016+0000 7f553ad9d640 1 --2- 172.21.15.100:0/31476434 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555bfa5a11c0 0x555bfa5ab460 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f552c012700 tx=0x7f552c012b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf40d83a9767d57d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.016+0000 7f55227fc640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:29:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.016+0000 7f55227fc640 1 -- 172.21.15.100:0/31476434 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552c005020 con 0x555bfa5a11c0 2024-05-10T11:29:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.016+0000 7f5543dd0740 1 -- 172.21.15.100:0/31476434 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5534012270 con 0x555bfa5a11c0 2024-05-10T11:29:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.016+0000 7f55227fc640 1 -- 172.21.15.100:0/31476434 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f552c0051c0 con 0x555bfa5a11c0 2024-05-10T11:29:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.016+0000 7f55227fc640 1 -- 172.21.15.100:0/31476434 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f552c025050 con 0x555bfa5a11c0 2024-05-10T11:29:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.016+0000 7f5543dd0740 10 client.27082 fetch_fsmap learned FSMap version 33 2024-05-10T11:29:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.016+0000 7f5543dd0740 1 -- 172.21.15.100:0/31476434 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x555bfa5b5ae0 con 0x555bfa5a11c0 2024-05-10T11:29:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.016+0000 7f55227fc640 1 -- 172.21.15.100:0/31476434 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5534012270 con 0x555bfa5a11c0 2024-05-10T11:29:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.016+0000 7f55227fc640 1 -- 172.21.15.100:0/31476434 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f552c00f480 con 0x555bfa5a11c0 2024-05-10T11:29:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.016+0000 7f5543dd0740 10 client.27082 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T11:29:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.017+0000 7f5543dd0740 10 client.27082 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:29:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.017+0000 7f5543dd0740 20 client.27082 populate_metadata read hostname 'smithi100' 2024-05-10T11:29:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.017+0000 7f5543dd0740 1 --2- 172.21.15.100:0/31476434 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x555bfa5b7160 0x555bfa5b95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.017+0000 7f5543dd0740 4 client.27082 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:29:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.017+0000 7f5543dd0740 1 -- 172.21.15.100:0/31476434 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555bfa50f1e0 con 0x555bfa5b7160 2024-05-10T11:29:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.017+0000 7f5533fff640 1 --2- 172.21.15.100:0/31476434 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x555bfa5b7160 0x555bfa5b95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.017+0000 7f55227fc640 10 client.27082 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:29:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.017+0000 7f5533fff640 1 --2- 172.21.15.100:0/31476434 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x555bfa5b7160 0x555bfa5b95b0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=858e57ce721661bb server_cookie=7a835890204b512f in_seq=0 out_seq=0 2024-05-10T11:29:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.018+0000 7f55227fc640 1 -- 172.21.15.100:0/31476434 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555bfa50f1e0 con 0x555bfa5b7160 2024-05-10T11:29:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.018+0000 7f55227fc640 10 client.27082 handle_command_reply: tid=0 2024-05-10T11:29:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.018+0000 7f5543dd0740 1 client.27082 shutdown 2024-05-10T11:29:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.018+0000 7f5543dd0740 1 -- 172.21.15.100:0/31476434 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555bfa5a11c0 msgr2=0x555bfa5ab460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.018+0000 7f5543dd0740 1 --2- 172.21.15.100:0/31476434 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555bfa5a11c0 0x555bfa5ab460 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f552c012700 tx=0x7f552c012b00 comp rx=0 tx=0).stop 2024-05-10T11:29:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.018+0000 7f5543dd0740 1 -- 172.21.15.100:0/31476434 shutdown_connections 2024-05-10T11:29:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.018+0000 7f5543dd0740 1 --2- 172.21.15.100:0/31476434 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x555bfa5b7160 0x555bfa5b95b0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.018+0000 7f5543dd0740 1 --2- 172.21.15.100:0/31476434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bfa5a6d20 0x555bfa5b29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.018+0000 7f5543dd0740 1 --2- 172.21.15.100:0/31476434 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555bfa5a1bb0 0x555bfa5ab9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.018+0000 7f55227fc640 0 client.27082 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:29:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.018+0000 7f5543dd0740 1 --2- 172.21.15.100:0/31476434 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555bfa5a11c0 0x555bfa5ab460 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:20.025 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.018+0000 7f5543dd0740 1 -- 172.21.15.100:0/31476434 >> 172.21.15.100:0/31476434 conn(0x555bfa598420 msgr2=0x555bfa597580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:20.025 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.018+0000 7f5543dd0740 1 -- 172.21.15.100:0/31476434 shutdown_connections 2024-05-10T11:29:20.025 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.018+0000 7f5543dd0740 1 -- 172.21.15.100:0/31476434 wait complete. 2024-05-10T11:29:20.025 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.018+0000 7f5543dd0740 20 client.27082 trim_cache size 0 max 16384 2024-05-10T11:29:20.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:29:19 smithi188 ceph-mon[34075]: pgmap v749: 129 pgs: 129 active+clean; 111 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 1.5 MiB/s wr, 177 op/s 2024-05-10T11:29:20.035 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.034+0000 7f5543dd0740 1 Processor -- start 2024-05-10T11:29:20.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.035+0000 7f5543dd0740 1 -- start start 2024-05-10T11:29:20.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.035+0000 7f5543dd0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bfa5a11c0 0x555bfa59ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:20.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.035+0000 7f5543dd0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555bfa5a1bb0 0x555bfa5aaa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:20.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.035+0000 7f5543dd0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555bfa5a6d20 0x555bfa5aafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:20.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.035+0000 7f5543dd0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f553c1ab970 con 0x555bfa5a11c0 2024-05-10T11:29:20.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.035+0000 7f5543dd0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f553c1a9240 con 0x555bfa5a1bb0 2024-05-10T11:29:20.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.035+0000 7f5543dd0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f553c1a7090 con 0x555bfa5a6d20 2024-05-10T11:29:20.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.035+0000 7f553ad9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bfa5a11c0 0x555bfa59ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:20.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.035+0000 7f553ad9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bfa5a11c0 0x555bfa59ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43074/0 (socket says 172.21.15.100:43074) 2024-05-10T11:29:20.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.035+0000 7f553ad9d640 1 -- 172.21.15.100:0/2740391192 learned_addr learned my addr 172.21.15.100:0/2740391192 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:29:20.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.035+0000 7f553b59e640 1 --2- 172.21.15.100:0/2740391192 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555bfa5a6d20 0x555bfa5aafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:20.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.035+0000 7f5533fff640 1 --2- 172.21.15.100:0/2740391192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555bfa5a1bb0 0x555bfa5aaa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:20.037 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.035+0000 7f553ad9d640 1 -- 172.21.15.100:0/2740391192 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555bfa5a6d20 msgr2=0x555bfa5aafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:20.037 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.035+0000 7f553ad9d640 1 --2- 172.21.15.100:0/2740391192 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555bfa5a6d20 0x555bfa5aafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:20.037 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.035+0000 7f553ad9d640 1 -- 172.21.15.100:0/2740391192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555bfa5a1bb0 msgr2=0x555bfa5aaa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:20.037 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.035+0000 7f553ad9d640 1 --2- 172.21.15.100:0/2740391192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555bfa5a1bb0 0x555bfa5aaa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:20.037 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.035+0000 7f553ad9d640 1 -- 172.21.15.100:0/2740391192 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bfa5b5900 con 0x555bfa5a11c0 2024-05-10T11:29:20.037 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.035+0000 7f553ad9d640 1 --2- 172.21.15.100:0/2740391192 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bfa5a11c0 0x555bfa59ba60 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f552c0058c0 tx=0x7f552c00fee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13ad4d31de993c4a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:20.037 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.036+0000 7f5530ff9640 1 -- 172.21.15.100:0/2740391192 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552c02e070 con 0x555bfa5a11c0 2024-05-10T11:29:20.037 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.036+0000 7f5530ff9640 1 -- 172.21.15.100:0/2740391192 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f552c0054d0 con 0x555bfa5a11c0 2024-05-10T11:29:20.037 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.036+0000 7f5530ff9640 1 -- 172.21.15.100:0/2740391192 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552c022040 con 0x555bfa5a11c0 2024-05-10T11:29:20.039 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.036+0000 7f5543dd0740 1 -- 172.21.15.100:0/2740391192 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bfa5a11c0 msgr2=0x555bfa59ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:20.040 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.036+0000 7f5543dd0740 1 --2- 172.21.15.100:0/2740391192 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bfa5a11c0 0x555bfa59ba60 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f552c0058c0 tx=0x7f552c00fee0 comp rx=0 tx=0).stop 2024-05-10T11:29:20.040 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.036+0000 7f5543dd0740 1 -- 172.21.15.100:0/2740391192 shutdown_connections 2024-05-10T11:29:20.040 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.036+0000 7f5543dd0740 1 --2- 172.21.15.100:0/2740391192 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555bfa5a6d20 0x555bfa5aafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:20.040 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.036+0000 7f5543dd0740 1 --2- 172.21.15.100:0/2740391192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555bfa5a1bb0 0x555bfa5aaa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:20.040 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.036+0000 7f5543dd0740 1 --2- 172.21.15.100:0/2740391192 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bfa5a11c0 0x555bfa59ba60 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:20.040 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.036+0000 7f5543dd0740 1 -- 172.21.15.100:0/2740391192 >> 172.21.15.100:0/2740391192 conn(0x555bfa598420 msgr2=0x555bfa59b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:20.040 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.037+0000 7f5543dd0740 1 -- 172.21.15.100:0/2740391192 shutdown_connections 2024-05-10T11:29:20.040 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.037+0000 7f5543dd0740 1 -- 172.21.15.100:0/2740391192 wait complete. 2024-05-10T11:29:20.040 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.037+0000 7f5543dd0740 1 -- start start 2024-05-10T11:29:20.040 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.037+0000 7f5543dd0740 1 Processor -- start 2024-05-10T11:29:20.040 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.037+0000 7f5543dd0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555bfa5a11c0 0x555bfa5af660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:20.040 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.037+0000 7f5543dd0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555bfa5a1bb0 0x555bfa5afba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:20.040 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.037+0000 7f5543dd0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bfa5a6d20 0x555bfa5b0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:20.041 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.037+0000 7f5543dd0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f553c1a7700 con 0x555bfa5a6d20 2024-05-10T11:29:20.041 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.037+0000 7f5543dd0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f553c1a8b70 con 0x555bfa5a1bb0 2024-05-10T11:29:20.041 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.037+0000 7f5543dd0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f553c1a7dd0 con 0x555bfa5a11c0 2024-05-10T11:29:20.041 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.037+0000 7f5543dd0740 10 client.0 _ll_register_callbacks cb 0x7f552c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:29:20.041 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.037+0000 7f553b59e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bfa5a6d20 0x555bfa5b0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:20.041 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.037+0000 7f553b59e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bfa5a6d20 0x555bfa5b0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43088/0 (socket says 172.21.15.100:43088) 2024-05-10T11:29:20.041 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.037+0000 7f553b59e640 1 -- 172.21.15.100:0/2753834059 learned_addr learned my addr 172.21.15.100:0/2753834059 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:29:20.041 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.037+0000 7f5533fff640 1 --2- 172.21.15.100:0/2753834059 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555bfa5a1bb0 0x555bfa5afba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:20.041 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.037+0000 7f553ad9d640 1 --2- 172.21.15.100:0/2753834059 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555bfa5a11c0 0x555bfa5af660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:20.041 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.037+0000 7f553b59e640 1 -- 172.21.15.100:0/2753834059 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555bfa5a11c0 msgr2=0x555bfa5af660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:20.041 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.037+0000 7f553b59e640 1 --2- 172.21.15.100:0/2753834059 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555bfa5a11c0 0x555bfa5af660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:20.041 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.037+0000 7f553b59e640 1 -- 172.21.15.100:0/2753834059 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555bfa5a1bb0 msgr2=0x555bfa5afba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:20.041 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.037+0000 7f553b59e640 1 --2- 172.21.15.100:0/2753834059 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555bfa5a1bb0 0x555bfa5afba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:20.041 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.038+0000 7f553b59e640 1 -- 172.21.15.100:0/2753834059 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bfa5b11a0 con 0x555bfa5a6d20 2024-05-10T11:29:20.042 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:29:19 smithi100 ceph-mon[30296]: pgmap v749: 129 pgs: 129 active+clean; 111 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 1.5 MiB/s wr, 177 op/s 2024-05-10T11:29:20.043 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.038+0000 7f553b59e640 1 --2- 172.21.15.100:0/2753834059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bfa5a6d20 0x555bfa5b0bb0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f553400f890 tx=0x7f553400acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d7a0a45388a518b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:20.043 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.038+0000 7f55227fc640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:29:20.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.038+0000 7f55227fc640 1 -- 172.21.15.100:0/2753834059 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5534010640 con 0x555bfa5a6d20 2024-05-10T11:29:20.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.038+0000 7f55227fc640 1 -- 172.21.15.100:0/2753834059 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f55340107e0 con 0x555bfa5a6d20 2024-05-10T11:29:20.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.038+0000 7f5543dd0740 1 -- 172.21.15.100:0/2753834059 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f552c0054d0 con 0x555bfa5a6d20 2024-05-10T11:29:20.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.038+0000 7f55227fc640 1 -- 172.21.15.100:0/2753834059 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f553403a050 con 0x555bfa5a6d20 2024-05-10T11:29:20.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.038+0000 7f5543dd0740 10 client.17637 fetch_fsmap learned FSMap version 33 2024-05-10T11:29:20.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.038+0000 7f5543dd0740 1 -- 172.21.15.100:0/2753834059 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x555bfa621a70 con 0x555bfa5a6d20 2024-05-10T11:29:20.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.038+0000 7f55227fc640 1 -- 172.21.15.100:0/2753834059 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f552c0054d0 con 0x555bfa5a6d20 2024-05-10T11:29:20.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.038+0000 7f55227fc640 1 -- 172.21.15.100:0/2753834059 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f55340268b0 con 0x555bfa5a6d20 2024-05-10T11:29:20.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.038+0000 7f5543dd0740 10 client.17637 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T11:29:20.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.039+0000 7f5543dd0740 10 client.17637 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:29:20.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.039+0000 7f5543dd0740 20 client.17637 populate_metadata read hostname 'smithi100' 2024-05-10T11:29:20.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.039+0000 7f5543dd0740 1 --2- 172.21.15.100:0/2753834059 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x555bfa6221c0 0x555bfa6245e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:20.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.039+0000 7f5543dd0740 4 client.17637 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:29:20.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.039+0000 7f5543dd0740 1 -- 172.21.15.100:0/2753834059 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555bfa509050 con 0x555bfa6221c0 2024-05-10T11:29:20.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.039+0000 7f553ad9d640 1 --2- 172.21.15.100:0/2753834059 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x555bfa6221c0 0x555bfa6245e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:20.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.039+0000 7f553ad9d640 1 --2- 172.21.15.100:0/2753834059 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x555bfa6221c0 0x555bfa6245e0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0318406c4e3878a server_cookie=a680283db5a5d241 in_seq=0 out_seq=0 2024-05-10T11:29:20.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.039+0000 7f55227fc640 10 client.17637 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:29:20.059 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.058+0000 7f55227fc640 1 -- 172.21.15.100:0/2753834059 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555bfa509050 con 0x555bfa6221c0 2024-05-10T11:29:20.059 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.058+0000 7f55227fc640 10 client.17637 handle_command_reply: tid=0 2024-05-10T11:29:20.059 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.058+0000 7f5543dd0740 1 client.17637 shutdown 2024-05-10T11:29:20.059 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.059+0000 7f5543dd0740 1 -- 172.21.15.100:0/2753834059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bfa5a6d20 msgr2=0x555bfa5b0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:20.059 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.059+0000 7f5543dd0740 1 --2- 172.21.15.100:0/2753834059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bfa5a6d20 0x555bfa5b0bb0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f553400f890 tx=0x7f553400acb0 comp rx=0 tx=0).stop 2024-05-10T11:29:20.059 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.059+0000 7f5543dd0740 1 -- 172.21.15.100:0/2753834059 shutdown_connections 2024-05-10T11:29:20.059 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.059+0000 7f5543dd0740 1 --2- 172.21.15.100:0/2753834059 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x555bfa6221c0 0x555bfa6245e0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:20.059 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.059+0000 7f5543dd0740 1 --2- 172.21.15.100:0/2753834059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bfa5a6d20 0x555bfa5b0bb0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:20.059 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.059+0000 7f5543dd0740 1 --2- 172.21.15.100:0/2753834059 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555bfa5a1bb0 0x555bfa5afba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:20.060 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.059+0000 7f55227fc640 0 client.17637 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:29:20.060 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.059+0000 7f5543dd0740 1 --2- 172.21.15.100:0/2753834059 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555bfa5a11c0 0x555bfa5af660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:20.060 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.059+0000 7f5543dd0740 1 -- 172.21.15.100:0/2753834059 >> 172.21.15.100:0/2753834059 conn(0x555bfa598420 msgr2=0x555bfa59b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:20.060 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.059+0000 7f5543dd0740 1 -- 172.21.15.100:0/2753834059 shutdown_connections 2024-05-10T11:29:20.060 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.059+0000 7f5543dd0740 1 -- 172.21.15.100:0/2753834059 wait complete. 2024-05-10T11:29:20.060 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.059+0000 7f5543dd0740 20 client.17637 trim_cache size 0 max 16384 2024-05-10T11:29:20.060 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:29:20.060 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:29:20.060 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:29:20.060 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:29:20.061 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.059+0000 7f55410b4640 1 -- 172.21.15.100:0/3947311570 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f55280b28a0 msgr2=0x7f55280b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:20.061 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.059+0000 7f55410b4640 1 --2- 172.21.15.100:0/3947311570 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f55280b28a0 0x7f55280b4d60 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5534021000 tx=0x7f553401e000 comp rx=0 tx=0).stop 2024-05-10T11:29:20.061 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.059+0000 7f55410b4640 1 -- 172.21.15.100:0/3947311570 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f553c102b20 msgr2=0x7f553c19d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:20.061 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.059+0000 7f55410b4640 1 --2- 172.21.15.100:0/3947311570 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f553c102b20 0x7f553c19d7d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5524009a20 tx=0x7f55240040c0 comp rx=0 tx=0).stop 2024-05-10T11:29:20.061 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.059+0000 7f55410b4640 1 -- 172.21.15.100:0/3947311570 shutdown_connections 2024-05-10T11:29:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.059+0000 7f55410b4640 1 --2- 172.21.15.100:0/3947311570 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f55280b28a0 0x7f55280b4d60 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.059+0000 7f55410b4640 1 --2- 172.21.15.100:0/3947311570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f553c1034e0 0x7f553c19fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.059+0000 7f55410b4640 1 --2- 172.21.15.100:0/3947311570 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f553c102b20 0x7f553c19d7d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.059+0000 7f55410b4640 1 --2- 172.21.15.100:0/3947311570 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f553c101930 0x7f553c06bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.059+0000 7f55410b4640 1 -- 172.21.15.100:0/3947311570 >> 172.21.15.100:0/3947311570 conn(0x7f553c0fcea0 msgr2=0x7f553c105f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.060+0000 7f55410b4640 1 -- 172.21.15.100:0/3947311570 shutdown_connections 2024-05-10T11:29:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:20.060+0000 7f55410b4640 1 -- 172.21.15.100:0/3947311570 wait complete. 2024-05-10T11:29:20.981 INFO:tasks.workunit.client.0.smithi100.stdout: 1 284083 2.73 MB/sec execute 435 sec latency 13.940 ms 2024-05-10T11:29:21.830 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:29:21 smithi188 ceph-mon[34075]: pgmap v750: 129 pgs: 129 active+clean; 111 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 1.2 MiB/s wr, 144 op/s 2024-05-10T11:29:21.981 INFO:tasks.workunit.client.0.smithi100.stdout: 1 284329 2.72 MB/sec execute 436 sec latency 15.248 ms 2024-05-10T11:29:22.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:29:21 smithi178 ceph-mon[33359]: pgmap v750: 129 pgs: 129 active+clean; 111 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 1.2 MiB/s wr, 144 op/s 2024-05-10T11:29:22.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:29:21 smithi100 ceph-mon[30296]: pgmap v750: 129 pgs: 129 active+clean; 111 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 1.2 MiB/s wr, 144 op/s 2024-05-10T11:29:22.981 INFO:tasks.workunit.client.0.smithi100.stdout: 1 284755 2.72 MB/sec execute 437 sec latency 16.026 ms 2024-05-10T11:29:23.804 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:29:23 smithi178 ceph-mon[33359]: pgmap v751: 129 pgs: 129 active+clean; 111 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 1.3 MiB/s wr, 131 op/s 2024-05-10T11:29:23.981 INFO:tasks.workunit.client.0.smithi100.stdout: 1 285313 2.72 MB/sec execute 438 sec latency 18.211 ms 2024-05-10T11:29:24.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:29:23 smithi188 ceph-mon[34075]: pgmap v751: 129 pgs: 129 active+clean; 111 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 1.3 MiB/s wr, 131 op/s 2024-05-10T11:29:24.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:29:23 smithi100 ceph-mon[30296]: pgmap v751: 129 pgs: 129 active+clean; 111 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 1.3 MiB/s wr, 131 op/s 2024-05-10T11:29:24.981 INFO:tasks.workunit.client.0.smithi100.stdout: 1 285971 2.72 MB/sec execute 439 sec latency 14.425 ms 2024-05-10T11:29:25.981 INFO:tasks.workunit.client.0.smithi100.stdout: 1 286502 2.72 MB/sec execute 440 sec latency 15.964 ms 2024-05-10T11:29:26.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:29:25 smithi178 ceph-mon[33359]: pgmap v752: 129 pgs: 129 active+clean; 112 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.4 MiB/s wr, 147 op/s 2024-05-10T11:29:26.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:29:25 smithi188 ceph-mon[34075]: pgmap v752: 129 pgs: 129 active+clean; 112 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.4 MiB/s wr, 147 op/s 2024-05-10T11:29:26.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:29:25 smithi100 ceph-mon[30296]: pgmap v752: 129 pgs: 129 active+clean; 112 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.4 MiB/s wr, 147 op/s 2024-05-10T11:29:26.981 INFO:tasks.workunit.client.0.smithi100.stdout: 1 287115 2.72 MB/sec execute 441 sec latency 15.590 ms 2024-05-10T11:29:27.981 INFO:tasks.workunit.client.0.smithi100.stdout: 1 287570 2.73 MB/sec execute 442 sec latency 18.795 ms 2024-05-10T11:29:28.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:29:27 smithi178 ceph-mon[33359]: pgmap v753: 129 pgs: 129 active+clean; 112 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 1.4 MiB/s wr, 147 op/s 2024-05-10T11:29:28.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:29:27 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:29:28.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:29:27 smithi188 ceph-mon[34075]: pgmap v753: 129 pgs: 129 active+clean; 112 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 1.4 MiB/s wr, 147 op/s 2024-05-10T11:29:28.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:29:27 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:29:28.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:29:27 smithi100 ceph-mon[30296]: pgmap v753: 129 pgs: 129 active+clean; 112 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 1.4 MiB/s wr, 147 op/s 2024-05-10T11:29:28.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:29:27 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:29:28.981 INFO:tasks.workunit.client.0.smithi100.stdout: 1 287796 2.72 MB/sec execute 443 sec latency 15.976 ms 2024-05-10T11:29:29.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:29:28 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:29:29.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:29:28 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:29:29.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:29:28 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:29:29.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:29:28 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:29:29.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:29:28 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:29:29.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:29:28 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:29:29.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:29:28 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:29:29.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:29:28 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:29:29.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:29:28 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:29:29.981 INFO:tasks.workunit.client.0.smithi100.stdout: 1 288204 2.72 MB/sec execute 444 sec latency 13.985 ms 2024-05-10T11:29:30.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:29:29 smithi178 ceph-mon[33359]: pgmap v754: 129 pgs: 129 active+clean; 114 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 1.4 MiB/s wr, 152 op/s 2024-05-10T11:29:30.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:29:29 smithi188 ceph-mon[34075]: pgmap v754: 129 pgs: 129 active+clean; 114 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 1.4 MiB/s wr, 152 op/s 2024-05-10T11:29:30.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:29:29 smithi100 ceph-mon[30296]: pgmap v754: 129 pgs: 129 active+clean; 114 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 1.4 MiB/s wr, 152 op/s 2024-05-10T11:29:30.981 INFO:tasks.workunit.client.0.smithi100.stdout: 1 288736 2.72 MB/sec execute 445 sec latency 14.989 ms 2024-05-10T11:29:31.981 INFO:tasks.workunit.client.0.smithi100.stdout: 1 289335 2.72 MB/sec execute 446 sec latency 15.044 ms 2024-05-10T11:29:32.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:29:31 smithi178 ceph-mon[33359]: pgmap v755: 129 pgs: 129 active+clean; 114 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 1.2 MiB/s wr, 146 op/s 2024-05-10T11:29:32.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:29:31 smithi188 ceph-mon[34075]: pgmap v755: 129 pgs: 129 active+clean; 114 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 1.2 MiB/s wr, 146 op/s 2024-05-10T11:29:32.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:29:31 smithi100 ceph-mon[30296]: pgmap v755: 129 pgs: 129 active+clean; 114 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 1.2 MiB/s wr, 146 op/s 2024-05-10T11:29:32.981 INFO:tasks.workunit.client.0.smithi100.stdout: 1 289830 2.71 MB/sec execute 447 sec latency 15.363 ms 2024-05-10T11:29:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.154+0000 7f532c6cf640 1 -- 172.21.15.100:0/1556531787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5324102b00 msgr2=0x7f5324102f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.154+0000 7f532c6cf640 1 --2- 172.21.15.100:0/1556531787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5324102b00 0x7f5324102f80 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f5314009900 tx=0x7f531402a410 comp rx=0 tx=0).stop 2024-05-10T11:29:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.154+0000 7f532c6cf640 1 -- 172.21.15.100:0/1556531787 shutdown_connections 2024-05-10T11:29:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.154+0000 7f532c6cf640 1 --2- 172.21.15.100:0/1556531787 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53241034c0 0x7f5324109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.154+0000 7f532c6cf640 1 --2- 172.21.15.100:0/1556531787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5324102b00 0x7f5324102f80 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.154+0000 7f532c6cf640 1 --2- 172.21.15.100:0/1556531787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5324101910 0x7f5324101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.154+0000 7f532c6cf640 1 -- 172.21.15.100:0/1556531787 >> 172.21.15.100:0/1556531787 conn(0x7f53240fcea0 msgr2=0x7f53240ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.154+0000 7f532c6cf640 1 -- 172.21.15.100:0/1556531787 shutdown_connections 2024-05-10T11:29:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.154+0000 7f532c6cf640 1 -- 172.21.15.100:0/1556531787 wait complete. 2024-05-10T11:29:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.155+0000 7f532c6cf640 1 Processor -- start 2024-05-10T11:29:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.155+0000 7f532c6cf640 1 -- start start 2024-05-10T11:29:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.155+0000 7f532c6cf640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5324101910 0x7f5324105a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.155+0000 7f532c6cf640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5324102b00 0x7f5324104050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.155+0000 7f532c6cf640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53241034c0 0x7f5324104590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.155+0000 7f532c6cf640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f532410d2f0 con 0x7f5324101910 2024-05-10T11:29:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.155+0000 7f532c6cf640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f532410d170 con 0x7f5324102b00 2024-05-10T11:29:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.155+0000 7f532c6cf640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f532410d470 con 0x7f53241034c0 2024-05-10T11:29:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.155+0000 7f532a46b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5324101910 0x7f5324105a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.155+0000 7f5329c6a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5324102b00 0x7f5324104050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.155+0000 7f532a46b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5324101910 0x7f5324105a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47820/0 (socket says 172.21.15.100:47820) 2024-05-10T11:29:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.155+0000 7f5329c6a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5324102b00 0x7f5324104050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:59474/0 (socket says 172.21.15.100:59474) 2024-05-10T11:29:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.155+0000 7f532a46b640 1 -- 172.21.15.100:0/2482769994 learned_addr learned my addr 172.21.15.100:0/2482769994 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:29:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.155+0000 7f532ac6c640 1 --2- 172.21.15.100:0/2482769994 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53241034c0 0x7f5324104590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.156+0000 7f532a46b640 1 -- 172.21.15.100:0/2482769994 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53241034c0 msgr2=0x7f5324104590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.156+0000 7f532a46b640 1 --2- 172.21.15.100:0/2482769994 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53241034c0 0x7f5324104590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.156+0000 7f532a46b640 1 -- 172.21.15.100:0/2482769994 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5324102b00 msgr2=0x7f5324104050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.156+0000 7f532a46b640 1 --2- 172.21.15.100:0/2482769994 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5324102b00 0x7f5324104050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.156+0000 7f532a46b640 1 -- 172.21.15.100:0/2482769994 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5324104e20 con 0x7f5324101910 2024-05-10T11:29:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.156+0000 7f5329c6a640 1 --2- 172.21.15.100:0/2482769994 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5324102b00 0x7f5324104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:29:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.156+0000 7f532a46b640 1 --2- 172.21.15.100:0/2482769994 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5324101910 0x7f5324105a10 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f532000e9c0 tx=0x7f532000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.156+0000 7f531b7fe640 1 -- 172.21.15.100:0/2482769994 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532000cd80 con 0x7f5324101910 2024-05-10T11:29:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.156+0000 7f532c6cf640 1 -- 172.21.15.100:0/2482769994 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53241a2be0 con 0x7f5324101910 2024-05-10T11:29:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.156+0000 7f531b7fe640 1 -- 172.21.15.100:0/2482769994 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f53200044b0 con 0x7f5324101910 2024-05-10T11:29:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.156+0000 7f531b7fe640 1 -- 172.21.15.100:0/2482769994 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5320010430 con 0x7f5324101910 2024-05-10T11:29:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.156+0000 7f532c6cf640 1 -- 172.21.15.100:0/2482769994 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53241a3100 con 0x7f5324101910 2024-05-10T11:29:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.159+0000 7f532f3e2740 1 Processor -- start 2024-05-10T11:29:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.159+0000 7f531b7fe640 1 -- 172.21.15.100:0/2482769994 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f5320003f00 con 0x7f5324101910 2024-05-10T11:29:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.159+0000 7f532f3e2740 1 -- start start 2024-05-10T11:29:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.159+0000 7f532f3e2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558117401db0 0x5581174021d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.159+0000 7f532f3e2740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5581174027a0 0x5581174072a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.159+0000 7f532f3e2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558117407910 0x558117409da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.159+0000 7f532f3e2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5581173665d0 con 0x558117401db0 2024-05-10T11:29:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.159+0000 7f532f3e2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55811731f890 con 0x558117407910 2024-05-10T11:29:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.159+0000 7f532f3e2740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558117362590 con 0x5581174027a0 2024-05-10T11:29:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.159+0000 7f532ac6c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558117401db0 0x5581174021d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.159+0000 7f532ac6c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558117401db0 0x5581174021d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47836/0 (socket says 172.21.15.100:47836) 2024-05-10T11:29:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.159+0000 7f532ac6c640 1 -- 172.21.15.100:0/3245145063 learned_addr learned my addr 172.21.15.100:0/3245145063 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:29:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.160+0000 7f532a46b640 1 --2- 172.21.15.100:0/3245145063 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5581174027a0 0x5581174072a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.160+0000 7f5329c6a640 1 --2- 172.21.15.100:0/3245145063 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558117407910 0x558117409da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.160+0000 7f532ac6c640 1 -- 172.21.15.100:0/3245145063 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5581174027a0 msgr2=0x5581174072a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.160+0000 7f532ac6c640 1 --2- 172.21.15.100:0/3245145063 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5581174027a0 0x5581174072a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.160+0000 7f532ac6c640 1 -- 172.21.15.100:0/3245145063 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558117407910 msgr2=0x558117409da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.160+0000 7f532ac6c640 1 --2- 172.21.15.100:0/3245145063 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558117407910 0x558117409da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.160+0000 7f532ac6c640 1 -- 172.21.15.100:0/3245145063 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558117078210 con 0x558117401db0 2024-05-10T11:29:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.160+0000 7f532ac6c640 1 --2- 172.21.15.100:0/3245145063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558117401db0 0x5581174021d0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f531c009fd0 tx=0x7f531c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba4b1928c7980d22 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.160+0000 7f531b7fe640 1 --2- 172.21.15.100:0/2482769994 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f53100b28a0 0x7f53100b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.160+0000 7f5318ff9640 1 -- 172.21.15.100:0/3245145063 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531c004d20 con 0x558117401db0 2024-05-10T11:29:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.160+0000 7f532a46b640 1 --2- 172.21.15.100:0/2482769994 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f53100b28a0 0x7f53100b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.160+0000 7f5318ff9640 1 -- 172.21.15.100:0/3245145063 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f531c004050 con 0x558117401db0 2024-05-10T11:29:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.160+0000 7f5318ff9640 1 -- 172.21.15.100:0/3245145063 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531c005260 con 0x558117401db0 2024-05-10T11:29:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.160+0000 7f531b7fe640 1 -- 172.21.15.100:0/2482769994 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f53200e29a0 con 0x7f5324101910 2024-05-10T11:29:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.161+0000 7f532f3e2740 1 -- 172.21.15.100:0/3245145063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558117401db0 msgr2=0x5581174021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.161+0000 7f532a46b640 1 --2- 172.21.15.100:0/2482769994 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f53100b28a0 0x7f53100b4d60 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f53200e04b0 tx=0x7f53200e5f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.161+0000 7f532f3e2740 1 --2- 172.21.15.100:0/3245145063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558117401db0 0x5581174021d0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f531c009fd0 tx=0x7f531c0118d0 comp rx=0 tx=0).stop 2024-05-10T11:29:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.161+0000 7f532f3e2740 1 -- 172.21.15.100:0/3245145063 shutdown_connections 2024-05-10T11:29:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.161+0000 7f532f3e2740 1 --2- 172.21.15.100:0/3245145063 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558117407910 0x558117409da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.161+0000 7f532f3e2740 1 --2- 172.21.15.100:0/3245145063 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5581174027a0 0x5581174072a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.161+0000 7f532f3e2740 1 --2- 172.21.15.100:0/3245145063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558117401db0 0x5581174021d0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.161+0000 7f532f3e2740 1 -- 172.21.15.100:0/3245145063 >> 172.21.15.100:0/3245145063 conn(0x5581173fd420 msgr2=0x5581173fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.161+0000 7f532f3e2740 1 -- 172.21.15.100:0/3245145063 shutdown_connections 2024-05-10T11:29:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.161+0000 7f532f3e2740 1 -- 172.21.15.100:0/3245145063 wait complete. 2024-05-10T11:29:33.170 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.161+0000 7f532f3e2740 1 -- start start 2024-05-10T11:29:33.170 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.162+0000 7f532f3e2740 1 Processor -- start 2024-05-10T11:29:33.170 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.162+0000 7f532f3e2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558117401db0 0x55811740bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:33.170 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.162+0000 7f532f3e2740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5581174027a0 0x55811740c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:33.170 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.162+0000 7f532f3e2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558117407910 0x5581174134b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:33.170 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.162+0000 7f532f3e2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55811740a8c0 con 0x558117407910 2024-05-10T11:29:33.170 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.162+0000 7f532f3e2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558117323370 con 0x558117401db0 2024-05-10T11:29:33.170 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.162+0000 7f532f3e2740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55811740aa40 con 0x5581174027a0 2024-05-10T11:29:33.170 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.162+0000 7f532f3e2740 10 client.0 _ll_register_callbacks cb 0x558117077a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:29:33.170 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.162+0000 7f532ac6c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558117407910 0x5581174134b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:33.170 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.162+0000 7f532ac6c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558117407910 0x5581174134b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47838/0 (socket says 172.21.15.100:47838) 2024-05-10T11:29:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.162+0000 7f532ac6c640 1 -- 172.21.15.100:0/830057745 learned_addr learned my addr 172.21.15.100:0/830057745 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:29:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.162+0000 7f532ac6c640 1 --2- 172.21.15.100:0/830057745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558117401db0 0x55811740bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.162+0000 7f5329c6a640 1 --2- 172.21.15.100:0/830057745 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5581174027a0 0x55811740c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.162+0000 7f532ac6c640 1 -- 172.21.15.100:0/830057745 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5581174027a0 msgr2=0x55811740c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.162+0000 7f532ac6c640 1 --2- 172.21.15.100:0/830057745 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5581174027a0 0x55811740c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.162+0000 7f532ac6c640 1 -- 172.21.15.100:0/830057745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558117401db0 msgr2=0x55811740bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.162+0000 7f532ac6c640 1 --2- 172.21.15.100:0/830057745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558117401db0 0x55811740bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.162+0000 7f532ac6c640 1 -- 172.21.15.100:0/830057745 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558117144620 con 0x558117407910 2024-05-10T11:29:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.162+0000 7f5329c6a640 1 --2- 172.21.15.100:0/830057745 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5581174027a0 0x55811740c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:29:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.163+0000 7f532ac6c640 1 --2- 172.21.15.100:0/830057745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558117407910 0x5581174134b0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f531c0048a0 tx=0x7f531c0047f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d8a3e08a7b6a4af server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.163+0000 7f530dffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:29:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.163+0000 7f530dffb640 1 -- 172.21.15.100:0/830057745 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531c02d060 con 0x558117407910 2024-05-10T11:29:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.163+0000 7f532f3e2740 1 -- 172.21.15.100:0/830057745 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f531c004050 con 0x558117407910 2024-05-10T11:29:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.163+0000 7f530dffb640 1 -- 172.21.15.100:0/830057745 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f531c00f4e0 con 0x558117407910 2024-05-10T11:29:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.163+0000 7f530dffb640 1 -- 172.21.15.100:0/830057745 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f531c00f6e0 con 0x558117407910 2024-05-10T11:29:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.163+0000 7f532f3e2740 10 client.17649 fetch_fsmap learned FSMap version 33 2024-05-10T11:29:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.163+0000 7f532f3e2740 1 -- 172.21.15.100:0/830057745 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x558117416490 con 0x558117407910 2024-05-10T11:29:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.163+0000 7f530dffb640 1 -- 172.21.15.100:0/830057745 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f531c004050 con 0x558117407910 2024-05-10T11:29:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.163+0000 7f530dffb640 1 -- 172.21.15.100:0/830057745 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f531c025040 con 0x558117407910 2024-05-10T11:29:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.163+0000 7f532f3e2740 10 client.17649 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T11:29:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.163+0000 7f532f3e2740 10 client.17649 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:29:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.163+0000 7f532f3e2740 20 client.17649 populate_metadata read hostname 'smithi100' 2024-05-10T11:29:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.163+0000 7f532f3e2740 1 --2- 172.21.15.100:0/830057745 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558117417960 0x558117419d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.163+0000 7f532f3e2740 4 client.17649 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:29:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.163+0000 7f532f3e2740 1 -- 172.21.15.100:0/830057745 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581173741e0 con 0x558117417960 2024-05-10T11:29:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.164+0000 7f5329c6a640 1 --2- 172.21.15.100:0/830057745 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558117417960 0x558117419d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.164+0000 7f530dffb640 10 client.17649 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:29:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.164+0000 7f5329c6a640 1 --2- 172.21.15.100:0/830057745 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558117417960 0x558117419d50 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5b5377ac1c9c8ba server_cookie=924fabc30feed0fd in_seq=0 out_seq=0 2024-05-10T11:29:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.165+0000 7f530dffb640 1 -- 172.21.15.100:0/830057745 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5581173741e0 con 0x558117417960 2024-05-10T11:29:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.165+0000 7f530dffb640 10 client.17649 handle_command_reply: tid=0 2024-05-10T11:29:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.165+0000 7f532f3e2740 1 client.17649 shutdown 2024-05-10T11:29:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.165+0000 7f532f3e2740 1 -- 172.21.15.100:0/830057745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558117407910 msgr2=0x5581174134b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.165+0000 7f532f3e2740 1 --2- 172.21.15.100:0/830057745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558117407910 0x5581174134b0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f531c0048a0 tx=0x7f531c0047f0 comp rx=0 tx=0).stop 2024-05-10T11:29:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.165+0000 7f532f3e2740 1 -- 172.21.15.100:0/830057745 shutdown_connections 2024-05-10T11:29:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.165+0000 7f532f3e2740 1 --2- 172.21.15.100:0/830057745 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558117417960 0x558117419d50 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.165+0000 7f532f3e2740 1 --2- 172.21.15.100:0/830057745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558117407910 0x5581174134b0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.165+0000 7f532f3e2740 1 --2- 172.21.15.100:0/830057745 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5581174027a0 0x55811740c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.165+0000 7f532f3e2740 1 --2- 172.21.15.100:0/830057745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558117401db0 0x55811740bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.165+0000 7f530dffb640 0 client.17649 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:29:33.174 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.165+0000 7f532f3e2740 1 -- 172.21.15.100:0/830057745 >> 172.21.15.100:0/830057745 conn(0x5581173fd420 msgr2=0x5581173fc580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:33.174 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.165+0000 7f532f3e2740 1 -- 172.21.15.100:0/830057745 shutdown_connections 2024-05-10T11:29:33.174 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.165+0000 7f532f3e2740 1 -- 172.21.15.100:0/830057745 wait complete. 2024-05-10T11:29:33.174 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.165+0000 7f532f3e2740 20 client.17649 trim_cache size 0 max 16384 2024-05-10T11:29:33.185 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.181+0000 7f532f3e2740 1 Processor -- start 2024-05-10T11:29:33.185 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.181+0000 7f532f3e2740 1 -- start start 2024-05-10T11:29:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.181+0000 7f532f3e2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558117401db0 0x5581174038c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.181+0000 7f532f3e2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581174027a0 0x558117403e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.181+0000 7f532f3e2740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558117407910 0x558117413470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.181+0000 7f532f3e2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f53241afe60 con 0x558117401db0 2024-05-10T11:29:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.181+0000 7f532f3e2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f53241ad730 con 0x5581174027a0 2024-05-10T11:29:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.181+0000 7f532f3e2740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f53241ab580 con 0x558117407910 2024-05-10T11:29:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.181+0000 7f532ac6c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558117401db0 0x5581174038c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.181+0000 7f532ac6c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558117401db0 0x5581174038c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47844/0 (socket says 172.21.15.100:47844) 2024-05-10T11:29:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.181+0000 7f532ac6c640 1 -- 172.21.15.100:0/2304638916 learned_addr learned my addr 172.21.15.100:0/2304638916 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:29:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.181+0000 7f532ac6c640 1 --2- 172.21.15.100:0/2304638916 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558117407910 0x558117413470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.181+0000 7f5329c6a640 1 --2- 172.21.15.100:0/2304638916 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581174027a0 0x558117403e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.181+0000 7f532ac6c640 1 -- 172.21.15.100:0/2304638916 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558117407910 msgr2=0x558117413470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.181+0000 7f532ac6c640 1 --2- 172.21.15.100:0/2304638916 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558117407910 0x558117413470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.181+0000 7f532ac6c640 1 -- 172.21.15.100:0/2304638916 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581174027a0 msgr2=0x558117403e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.181+0000 7f532ac6c640 1 --2- 172.21.15.100:0/2304638916 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581174027a0 0x558117403e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.181+0000 7f532ac6c640 1 -- 172.21.15.100:0/2304638916 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581173fd250 con 0x558117401db0 2024-05-10T11:29:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.182+0000 7f532ac6c640 1 --2- 172.21.15.100:0/2304638916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558117401db0 0x5581174038c0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f531c004630 tx=0x7f531c005cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=701c8356ff753764 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.182+0000 7f530ffff640 1 -- 172.21.15.100:0/2304638916 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531c02d040 con 0x558117401db0 2024-05-10T11:29:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.182+0000 7f530ffff640 1 -- 172.21.15.100:0/2304638916 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f531c004990 con 0x558117401db0 2024-05-10T11:29:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.182+0000 7f530ffff640 1 -- 172.21.15.100:0/2304638916 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531c004030 con 0x558117401db0 2024-05-10T11:29:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.182+0000 7f532f3e2740 1 -- 172.21.15.100:0/2304638916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558117401db0 msgr2=0x5581174038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.182+0000 7f532f3e2740 1 --2- 172.21.15.100:0/2304638916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558117401db0 0x5581174038c0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f531c004630 tx=0x7f531c005cf0 comp rx=0 tx=0).stop 2024-05-10T11:29:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.182+0000 7f532f3e2740 1 -- 172.21.15.100:0/2304638916 shutdown_connections 2024-05-10T11:29:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.182+0000 7f532f3e2740 1 --2- 172.21.15.100:0/2304638916 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558117407910 0x558117413470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.182+0000 7f532f3e2740 1 --2- 172.21.15.100:0/2304638916 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581174027a0 0x558117403e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.182+0000 7f532f3e2740 1 --2- 172.21.15.100:0/2304638916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558117401db0 0x5581174038c0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.182+0000 7f532f3e2740 1 -- 172.21.15.100:0/2304638916 >> 172.21.15.100:0/2304638916 conn(0x5581173fd420 msgr2=0x5581173f83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.183+0000 7f532f3e2740 1 -- 172.21.15.100:0/2304638916 shutdown_connections 2024-05-10T11:29:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.183+0000 7f532f3e2740 1 -- 172.21.15.100:0/2304638916 wait complete. 2024-05-10T11:29:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.183+0000 7f532f3e2740 1 -- start start 2024-05-10T11:29:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.183+0000 7f532f3e2740 1 Processor -- start 2024-05-10T11:29:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.183+0000 7f532f3e2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558117401db0 0x558117410210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.183+0000 7f532f3e2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581174027a0 0x558117410750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.183+0000 7f532f3e2740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558117407910 0x5581174801b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.183+0000 7f532f3e2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f53241abbf0 con 0x5581174027a0 2024-05-10T11:29:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.183+0000 7f532f3e2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f53241ad060 con 0x558117401db0 2024-05-10T11:29:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.183+0000 7f532f3e2740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f53241ac2c0 con 0x558117407910 2024-05-10T11:29:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.183+0000 7f532f3e2740 10 client.0 _ll_register_callbacks cb 0x7f531c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:29:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.183+0000 7f5329c6a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581174027a0 0x558117410750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.183+0000 7f5329c6a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581174027a0 0x558117410750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47852/0 (socket says 172.21.15.100:47852) 2024-05-10T11:29:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.183+0000 7f5329c6a640 1 -- 172.21.15.100:0/1394697662 learned_addr learned my addr 172.21.15.100:0/1394697662 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:29:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.183+0000 7f532ac6c640 1 --2- 172.21.15.100:0/1394697662 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558117401db0 0x558117410210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.183+0000 7f532ac6c640 1 --2- 172.21.15.100:0/1394697662 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558117407910 0x5581174801b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.184+0000 7f5329c6a640 1 -- 172.21.15.100:0/1394697662 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558117407910 msgr2=0x5581174801b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.184+0000 7f5329c6a640 1 --2- 172.21.15.100:0/1394697662 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558117407910 0x5581174801b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.184+0000 7f5329c6a640 1 -- 172.21.15.100:0/1394697662 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558117401db0 msgr2=0x558117410210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.184+0000 7f5329c6a640 1 --2- 172.21.15.100:0/1394697662 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558117401db0 0x558117410210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.184+0000 7f5329c6a640 1 -- 172.21.15.100:0/1394697662 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558117410d90 con 0x5581174027a0 2024-05-10T11:29:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.184+0000 7f5329c6a640 1 --2- 172.21.15.100:0/1394697662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581174027a0 0x558117410750 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f5314009340 tx=0x7f5314031d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57a695fd14e932a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.184+0000 7f530dffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:29:33.190 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.184+0000 7f530dffb640 1 -- 172.21.15.100:0/1394697662 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5314035040 con 0x5581174027a0 2024-05-10T11:29:33.190 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.184+0000 7f530dffb640 1 -- 172.21.15.100:0/1394697662 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f5314004450 con 0x5581174027a0 2024-05-10T11:29:33.190 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.184+0000 7f532f3e2740 1 -- 172.21.15.100:0/1394697662 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f531c004990 con 0x5581174027a0 2024-05-10T11:29:33.190 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.184+0000 7f530dffb640 1 -- 172.21.15.100:0/1394697662 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5314055050 con 0x5581174027a0 2024-05-10T11:29:33.190 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.184+0000 7f532f3e2740 10 client.17655 fetch_fsmap learned FSMap version 33 2024-05-10T11:29:33.190 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.184+0000 7f532f3e2740 1 -- 172.21.15.100:0/1394697662 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5581174818d0 con 0x5581174027a0 2024-05-10T11:29:33.190 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.184+0000 7f530dffb640 1 -- 172.21.15.100:0/1394697662 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f531c004990 con 0x5581174027a0 2024-05-10T11:29:33.190 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.184+0000 7f530dffb640 1 -- 172.21.15.100:0/1394697662 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f53140038f0 con 0x5581174027a0 2024-05-10T11:29:33.190 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.184+0000 7f532f3e2740 10 client.17655 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T11:29:33.190 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.184+0000 7f532f3e2740 10 client.17655 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:29:33.190 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.185+0000 7f532f3e2740 20 client.17655 populate_metadata read hostname 'smithi100' 2024-05-10T11:29:33.190 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.185+0000 7f532f3e2740 1 --2- 172.21.15.100:0/1394697662 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5581174822b0 0x5581174846d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:33.190 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.185+0000 7f532f3e2740 4 client.17655 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:29:33.190 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.185+0000 7f532f3e2740 1 -- 172.21.15.100:0/1394697662 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55811736e050 con 0x5581174822b0 2024-05-10T11:29:33.190 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.185+0000 7f532ac6c640 1 --2- 172.21.15.100:0/1394697662 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5581174822b0 0x5581174846d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:33.191 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.185+0000 7f530dffb640 10 client.17655 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:29:33.191 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.185+0000 7f532ac6c640 1 --2- 172.21.15.100:0/1394697662 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5581174822b0 0x5581174846d0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=551ad5f75540ddf0 server_cookie=b6652d8d7f915baf in_seq=0 out_seq=0 2024-05-10T11:29:33.208 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.208+0000 7f530dffb640 1 -- 172.21.15.100:0/1394697662 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55811736e050 con 0x5581174822b0 2024-05-10T11:29:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.208+0000 7f530dffb640 10 client.17655 handle_command_reply: tid=0 2024-05-10T11:29:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.208+0000 7f532f3e2740 1 client.17655 shutdown 2024-05-10T11:29:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.208+0000 7f532f3e2740 1 -- 172.21.15.100:0/1394697662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581174027a0 msgr2=0x558117410750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.208+0000 7f532f3e2740 1 --2- 172.21.15.100:0/1394697662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581174027a0 0x558117410750 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f5314009340 tx=0x7f5314031d00 comp rx=0 tx=0).stop 2024-05-10T11:29:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.208+0000 7f532f3e2740 1 -- 172.21.15.100:0/1394697662 shutdown_connections 2024-05-10T11:29:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.208+0000 7f532f3e2740 1 --2- 172.21.15.100:0/1394697662 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5581174822b0 0x5581174846d0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.208+0000 7f532f3e2740 1 --2- 172.21.15.100:0/1394697662 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558117407910 0x5581174801b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.208+0000 7f530dffb640 0 client.17655 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:29:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.208+0000 7f532f3e2740 1 --2- 172.21.15.100:0/1394697662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581174027a0 0x558117410750 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.208+0000 7f532f3e2740 1 --2- 172.21.15.100:0/1394697662 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558117401db0 0x558117410210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.208+0000 7f532f3e2740 1 -- 172.21.15.100:0/1394697662 >> 172.21.15.100:0/1394697662 conn(0x5581173fd420 msgr2=0x5581174038c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.208+0000 7f532f3e2740 1 -- 172.21.15.100:0/1394697662 shutdown_connections 2024-05-10T11:29:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.208+0000 7f532f3e2740 1 -- 172.21.15.100:0/1394697662 wait complete. 2024-05-10T11:29:33.210 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.208+0000 7f532f3e2740 20 client.17655 trim_cache size 0 max 16384 2024-05-10T11:29:33.210 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:29:33.210 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:29:33.210 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:29:33.210 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:29:33.211 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.209+0000 7f532c6cf640 1 -- 172.21.15.100:0/2482769994 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f53100b28a0 msgr2=0x7f53100b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:33.211 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.209+0000 7f532c6cf640 1 --2- 172.21.15.100:0/2482769994 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f53100b28a0 0x7f53100b4d60 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f53200e04b0 tx=0x7f53200e5f90 comp rx=0 tx=0).stop 2024-05-10T11:29:33.211 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.209+0000 7f532c6cf640 1 -- 172.21.15.100:0/2482769994 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5324101910 msgr2=0x7f5324105a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:33.211 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.209+0000 7f532c6cf640 1 --2- 172.21.15.100:0/2482769994 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5324101910 0x7f5324105a10 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f532000e9c0 tx=0x7f532000ee80 comp rx=0 tx=0).stop 2024-05-10T11:29:33.211 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.209+0000 7f532c6cf640 1 -- 172.21.15.100:0/2482769994 shutdown_connections 2024-05-10T11:29:33.211 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.209+0000 7f532c6cf640 1 --2- 172.21.15.100:0/2482769994 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f53100b28a0 0x7f53100b4d60 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:33.211 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.209+0000 7f532c6cf640 1 --2- 172.21.15.100:0/2482769994 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53241034c0 0x7f5324104590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:33.211 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.209+0000 7f532c6cf640 1 --2- 172.21.15.100:0/2482769994 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5324102b00 0x7f5324104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:33.211 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.209+0000 7f532c6cf640 1 --2- 172.21.15.100:0/2482769994 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5324101910 0x7f5324105a10 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:33.212 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.209+0000 7f532c6cf640 1 -- 172.21.15.100:0/2482769994 >> 172.21.15.100:0/2482769994 conn(0x7f53240fcea0 msgr2=0x7f532406b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:33.212 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.209+0000 7f532c6cf640 1 -- 172.21.15.100:0/2482769994 shutdown_connections 2024-05-10T11:29:33.212 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:33.209+0000 7f532c6cf640 1 -- 172.21.15.100:0/2482769994 wait complete. 2024-05-10T11:29:33.981 INFO:tasks.workunit.client.0.smithi100.stdout: 1 290616 2.72 MB/sec execute 448 sec latency 16.799 ms 2024-05-10T11:29:34.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:29:33 smithi178 ceph-mon[33359]: pgmap v756: 129 pgs: 129 active+clean; 114 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 1.3 MiB/s wr, 132 op/s 2024-05-10T11:29:34.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:29:33 smithi188 ceph-mon[34075]: pgmap v756: 129 pgs: 129 active+clean; 114 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 1.3 MiB/s wr, 132 op/s 2024-05-10T11:29:34.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:29:33 smithi100 ceph-mon[30296]: pgmap v756: 129 pgs: 129 active+clean; 114 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 1.3 MiB/s wr, 132 op/s 2024-05-10T11:29:34.981 INFO:tasks.workunit.client.0.smithi100.stdout: 1 291022 2.72 MB/sec execute 449 sec latency 15.323 ms 2024-05-10T11:29:35.981 INFO:tasks.workunit.client.0.smithi100.stdout: 1 291292 2.72 MB/sec execute 450 sec latency 14.211 ms 2024-05-10T11:29:36.090 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:29:35 smithi100 ceph-mon[30296]: pgmap v757: 129 pgs: 129 active+clean; 115 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 1.3 MiB/s wr, 150 op/s 2024-05-10T11:29:36.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:29:35 smithi178 ceph-mon[33359]: pgmap v757: 129 pgs: 129 active+clean; 115 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 1.3 MiB/s wr, 150 op/s 2024-05-10T11:29:36.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:29:35 smithi188 ceph-mon[34075]: pgmap v757: 129 pgs: 129 active+clean; 115 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 1.3 MiB/s wr, 150 op/s 2024-05-10T11:29:36.981 INFO:tasks.workunit.client.0.smithi100.stdout: 1 291617 2.72 MB/sec execute 451 sec latency 14.874 ms 2024-05-10T11:29:37.981 INFO:tasks.workunit.client.0.smithi100.stdout: 1 292059 2.71 MB/sec execute 452 sec latency 15.899 ms 2024-05-10T11:29:38.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:29:37 smithi178 ceph-mon[33359]: pgmap v758: 129 pgs: 129 active+clean; 116 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 1.3 MiB/s wr, 146 op/s 2024-05-10T11:29:38.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:29:37 smithi188 ceph-mon[34075]: pgmap v758: 129 pgs: 129 active+clean; 116 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 1.3 MiB/s wr, 146 op/s 2024-05-10T11:29:38.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:29:37 smithi100 ceph-mon[30296]: pgmap v758: 129 pgs: 129 active+clean; 116 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 1.3 MiB/s wr, 146 op/s 2024-05-10T11:29:38.981 INFO:tasks.workunit.client.0.smithi100.stdout: 1 292660 2.72 MB/sec execute 453 sec latency 21.693 ms 2024-05-10T11:29:39.981 INFO:tasks.workunit.client.0.smithi100.stdout: 1 293267 2.71 MB/sec execute 454 sec latency 15.781 ms 2024-05-10T11:29:40.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:29:39 smithi178 ceph-mon[33359]: pgmap v759: 129 pgs: 129 active+clean; 117 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 1.4 MiB/s wr, 150 op/s 2024-05-10T11:29:40.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:29:39 smithi188 ceph-mon[34075]: pgmap v759: 129 pgs: 129 active+clean; 117 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 1.4 MiB/s wr, 150 op/s 2024-05-10T11:29:40.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:29:39 smithi100 ceph-mon[30296]: pgmap v759: 129 pgs: 129 active+clean; 117 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 1.4 MiB/s wr, 150 op/s 2024-05-10T11:29:40.981 INFO:tasks.workunit.client.0.smithi100.stdout: 1 293880 2.72 MB/sec execute 455 sec latency 15.057 ms 2024-05-10T11:29:41.641 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T11:29:41.901 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:29:41.981 INFO:tasks.workunit.client.0.smithi100.stdout: 1 294403 2.72 MB/sec execute 456 sec latency 15.721 ms 2024-05-10T11:29:42.024 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:29:41 smithi100 ceph-mon[30296]: pgmap v760: 129 pgs: 129 active+clean; 116 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 1.2 MiB/s wr, 143 op/s 2024-05-10T11:29:42.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:29:41 smithi178 ceph-mon[33359]: pgmap v760: 129 pgs: 129 active+clean; 116 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 1.2 MiB/s wr, 143 op/s 2024-05-10T11:29:42.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:29:41 smithi188 ceph-mon[34075]: pgmap v760: 129 pgs: 129 active+clean; 116 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 1.2 MiB/s wr, 143 op/s 2024-05-10T11:29:42.981 INFO:tasks.workunit.client.0.smithi100.stdout: 1 294760 2.72 MB/sec execute 457 sec latency 16.002 ms 2024-05-10T11:29:43.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.813+0000 7f135ceaf700 1 -- 172.21.15.100:0/1291257688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1358108410 msgr2=0x7f13581087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:43.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.813+0000 7f135ceaf700 1 --2- 172.21.15.100:0/1291257688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1358108410 0x7f13581087f0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f1348009990 tx=0x7f1348009ca0 comp rx=0 tx=0).stop 2024-05-10T11:29:43.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.813+0000 7f135ceaf700 1 -- 172.21.15.100:0/1291257688 shutdown_connections 2024-05-10T11:29:43.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.813+0000 7f135ceaf700 1 --2- 172.21.15.100:0/1291257688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1358102d60 0x7f135810f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:43.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.813+0000 7f135ceaf700 1 --2- 172.21.15.100:0/1291257688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1358108410 0x7f13581087f0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:43.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.813+0000 7f135ceaf700 1 --2- 172.21.15.100:0/1291257688 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13581023a0 0x7f1358102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:43.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.813+0000 7f135ceaf700 1 -- 172.21.15.100:0/1291257688 >> 172.21.15.100:0/1291257688 conn(0x7f1358079350 msgr2=0x7f1358079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:43.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.813+0000 7f135ceaf700 1 -- 172.21.15.100:0/1291257688 shutdown_connections 2024-05-10T11:29:43.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.813+0000 7f135ceaf700 1 -- 172.21.15.100:0/1291257688 wait complete. 2024-05-10T11:29:43.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.814+0000 7f135ceaf700 1 Processor -- start 2024-05-10T11:29:43.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.814+0000 7f135ceaf700 1 -- start start 2024-05-10T11:29:43.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.814+0000 7f135ceaf700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13581023a0 0x7f13581986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:43.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.814+0000 7f135ceaf700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1358102d60 0x7f1358198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:43.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.814+0000 7f135ceaf700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1358108410 0x7f135819d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:43.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.814+0000 7f135ceaf700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1358112460 con 0x7f13581023a0 2024-05-10T11:29:43.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.814+0000 7f135ceaf700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f13581122e0 con 0x7f1358102d60 2024-05-10T11:29:43.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.814+0000 7f135ceaf700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f13581125e0 con 0x7f1358108410 2024-05-10T11:29:43.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.814+0000 7f135659c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13581023a0 0x7f13581986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:43.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.814+0000 7f1355d9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1358102d60 0x7f1358198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:43.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.814+0000 7f135659c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13581023a0 0x7f13581986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43120/0 (socket says 172.21.15.100:43120) 2024-05-10T11:29:43.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.814+0000 7f1355d9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1358102d60 0x7f1358198c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:43832/0 (socket says 172.21.15.100:43832) 2024-05-10T11:29:43.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.814+0000 7f135659c700 1 -- 172.21.15.100:0/3864566867 learned_addr learned my addr 172.21.15.100:0/3864566867 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:29:43.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.815+0000 7f1356d9d700 1 --2- 172.21.15.100:0/3864566867 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1358108410 0x7f135819d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:43.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.815+0000 7f1355d9b700 1 -- 172.21.15.100:0/3864566867 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1358108410 msgr2=0x7f135819d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:43.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.815+0000 7f1355d9b700 1 --2- 172.21.15.100:0/3864566867 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1358108410 0x7f135819d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:43.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.815+0000 7f1355d9b700 1 -- 172.21.15.100:0/3864566867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13581023a0 msgr2=0x7f13581986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:43.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.815+0000 7f1355d9b700 1 --2- 172.21.15.100:0/3864566867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13581023a0 0x7f13581986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:43.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.815+0000 7f1355d9b700 1 -- 172.21.15.100:0/3864566867 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f135819dc60 con 0x7f1358102d60 2024-05-10T11:29:43.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.815+0000 7f1355d9b700 1 --2- 172.21.15.100:0/3864566867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1358102d60 0x7f1358198c10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f134c00d750 tx=0x7f134c00da60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:43.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.815+0000 7f13477fe700 1 -- 172.21.15.100:0/3864566867 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134c014030 con 0x7f1358102d60 2024-05-10T11:29:43.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.815+0000 7f13477fe700 1 -- 172.21.15.100:0/3864566867 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f134c0044b0 con 0x7f1358102d60 2024-05-10T11:29:43.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.815+0000 7f135ceaf700 1 -- 172.21.15.100:0/3864566867 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f135819df50 con 0x7f1358102d60 2024-05-10T11:29:43.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.815+0000 7f13477fe700 1 -- 172.21.15.100:0/3864566867 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134c005260 con 0x7f1358102d60 2024-05-10T11:29:43.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.815+0000 7f135ceaf700 1 -- 172.21.15.100:0/3864566867 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f135819e490 con 0x7f1358102d60 2024-05-10T11:29:43.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.818+0000 7f13477fe700 1 -- 172.21.15.100:0/3864566867 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f134c0049e0 con 0x7f1358102d60 2024-05-10T11:29:43.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.819+0000 7f13477fe700 1 --2- 172.21.15.100:0/3864566867 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f13400b2880 0x7f13400b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:43.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.819+0000 7f135659c700 1 --2- 172.21.15.100:0/3864566867 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f13400b2880 0x7f13400b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:43.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.819+0000 7f13477fe700 1 -- 172.21.15.100:0/3864566867 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f134c0e21a0 con 0x7f1358102d60 2024-05-10T11:29:43.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.819+0000 7f135659c700 1 --2- 172.21.15.100:0/3864566867 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f13400b2880 0x7f13400b4d40 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f1348006010 tx=0x7f134800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:43.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.819+0000 7f1367742b80 1 Processor -- start 2024-05-10T11:29:43.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.819+0000 7f1367742b80 1 -- start start 2024-05-10T11:29:43.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.820+0000 7f1367742b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ea270f11b0 0x55ea270f1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:43.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.820+0000 7f1367742b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ea270f1ad0 0x55ea270eb810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:43.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.820+0000 7f1367742b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ea270ebd50 0x55ea270ec190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:43.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.820+0000 7f1367742b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55ea2703d940 con 0x55ea270ebd50 2024-05-10T11:29:43.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.820+0000 7f1367742b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ea26fa2da0 con 0x55ea270f1ad0 2024-05-10T11:29:43.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.820+0000 7f1367742b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55ea27039900 con 0x55ea270f11b0 2024-05-10T11:29:43.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.820+0000 7f1356d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ea270ebd50 0x55ea270ec190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:43.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.820+0000 7f1356d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ea270ebd50 0x55ea270ec190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43134/0 (socket says 172.21.15.100:43134) 2024-05-10T11:29:43.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.820+0000 7f1356d9d700 1 -- 172.21.15.100:0/48720959 learned_addr learned my addr 172.21.15.100:0/48720959 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:29:43.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.820+0000 7f1355d9b700 1 --2- 172.21.15.100:0/48720959 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ea270f1ad0 0x55ea270eb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:43.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.820+0000 7f135659c700 1 --2- 172.21.15.100:0/48720959 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ea270f11b0 0x55ea270f1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:43.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.820+0000 7f1356d9d700 1 -- 172.21.15.100:0/48720959 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ea270f11b0 msgr2=0x55ea270f1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:43.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.820+0000 7f1356d9d700 1 --2- 172.21.15.100:0/48720959 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ea270f11b0 0x55ea270f1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:43.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.820+0000 7f1356d9d700 1 -- 172.21.15.100:0/48720959 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ea270f1ad0 msgr2=0x55ea270eb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:43.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.820+0000 7f1356d9d700 1 --2- 172.21.15.100:0/48720959 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ea270f1ad0 0x55ea270eb810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:43.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.820+0000 7f1356d9d700 1 -- 172.21.15.100:0/48720959 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea26f8a8d0 con 0x55ea270ebd50 2024-05-10T11:29:43.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.820+0000 7f1356d9d700 1 --2- 172.21.15.100:0/48720959 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ea270ebd50 0x55ea270ec190 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f1350011680 tx=0x7f1350009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=977e22e2d652a58e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:43.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.820+0000 7f1344ff9700 1 -- 172.21.15.100:0/48720959 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1350019030 con 0x55ea270ebd50 2024-05-10T11:29:43.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.820+0000 7f1344ff9700 1 -- 172.21.15.100:0/48720959 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f13500092e0 con 0x55ea270ebd50 2024-05-10T11:29:43.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.820+0000 7f1344ff9700 1 -- 172.21.15.100:0/48720959 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1350007500 con 0x55ea270ebd50 2024-05-10T11:29:43.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.821+0000 7f1367742b80 1 -- 172.21.15.100:0/48720959 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ea270ebd50 msgr2=0x55ea270ec190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:43.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.821+0000 7f1367742b80 1 --2- 172.21.15.100:0/48720959 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ea270ebd50 0x55ea270ec190 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f1350011680 tx=0x7f1350009920 comp rx=0 tx=0).stop 2024-05-10T11:29:43.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.821+0000 7f1367742b80 1 -- 172.21.15.100:0/48720959 shutdown_connections 2024-05-10T11:29:43.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.821+0000 7f1367742b80 1 --2- 172.21.15.100:0/48720959 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ea270ebd50 0x55ea270ec190 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:43.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.821+0000 7f1367742b80 1 --2- 172.21.15.100:0/48720959 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ea270f11b0 0x55ea270f1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:43.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.821+0000 7f1367742b80 1 --2- 172.21.15.100:0/48720959 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ea270f1ad0 0x55ea270eb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:43.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.821+0000 7f1367742b80 1 -- 172.21.15.100:0/48720959 >> 172.21.15.100:0/48720959 conn(0x55ea27017b70 msgr2=0x55ea270e62a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:43.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.821+0000 7f1367742b80 1 -- 172.21.15.100:0/48720959 shutdown_connections 2024-05-10T11:29:43.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.821+0000 7f1367742b80 1 -- 172.21.15.100:0/48720959 wait complete. 2024-05-10T11:29:43.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.822+0000 7f1367742b80 1 -- start start 2024-05-10T11:29:43.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.822+0000 7f1367742b80 1 Processor -- start 2024-05-10T11:29:43.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.822+0000 7f1367742b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ea270ebd50 0x55ea270f7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:43.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.822+0000 7f1367742b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ea270f11b0 0x55ea270f4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:43.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.822+0000 7f1367742b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ea270f1ad0 0x55ea270f4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:43.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.822+0000 7f1367742b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55ea270f8450 con 0x55ea270f11b0 2024-05-10T11:29:43.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.822+0000 7f1367742b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ea26ff6ed0 con 0x55ea270f1ad0 2024-05-10T11:29:43.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.822+0000 7f1367742b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55ea270f85d0 con 0x55ea270ebd50 2024-05-10T11:29:43.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.822+0000 7f1367742b80 10 client.0 _ll_register_callbacks cb 0x55ea26fd15b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:29:43.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.822+0000 7f1355d9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ea270f11b0 0x55ea270f4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:43.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.822+0000 7f1355d9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ea270f11b0 0x55ea270f4ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43140/0 (socket says 172.21.15.100:43140) 2024-05-10T11:29:43.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.822+0000 7f1355d9b700 1 -- 172.21.15.100:0/1428005957 learned_addr learned my addr 172.21.15.100:0/1428005957 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:29:43.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.822+0000 7f1356d9d700 1 --2- 172.21.15.100:0/1428005957 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ea270f1ad0 0x55ea270f4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:43.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.822+0000 7f135659c700 1 --2- 172.21.15.100:0/1428005957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ea270ebd50 0x55ea270f7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:43.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.822+0000 7f1356d9d700 1 -- 172.21.15.100:0/1428005957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ea270ebd50 msgr2=0x55ea270f7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:43.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.822+0000 7f1356d9d700 1 --2- 172.21.15.100:0/1428005957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ea270ebd50 0x55ea270f7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:43.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.822+0000 7f1356d9d700 1 -- 172.21.15.100:0/1428005957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ea270f11b0 msgr2=0x55ea270f4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:43.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.822+0000 7f1356d9d700 1 --2- 172.21.15.100:0/1428005957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ea270f11b0 0x55ea270f4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:43.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.822+0000 7f1356d9d700 1 -- 172.21.15.100:0/1428005957 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea26ff4760 con 0x55ea270f1ad0 2024-05-10T11:29:43.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.823+0000 7f1355d9b700 1 --2- 172.21.15.100:0/1428005957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ea270f11b0 0x55ea270f4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:29:43.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.823+0000 7f1356d9d700 1 --2- 172.21.15.100:0/1428005957 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ea270f1ad0 0x55ea270f4ff0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1350011650 tx=0x7f1350004a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6cd6418c997b8fff server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:43.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.823+0000 7f133dffb700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:29:43.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.823+0000 7f133dffb700 1 -- 172.21.15.100:0/1428005957 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1350023070 con 0x55ea270f1ad0 2024-05-10T11:29:43.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.823+0000 7f133dffb700 1 -- 172.21.15.100:0/1428005957 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f1350003e10 con 0x55ea270f1ad0 2024-05-10T11:29:43.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.823+0000 7f1367742b80 1 -- 172.21.15.100:0/1428005957 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13500092e0 con 0x55ea270f1ad0 2024-05-10T11:29:43.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.823+0000 7f133dffb700 1 -- 172.21.15.100:0/1428005957 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1350004290 con 0x55ea270f1ad0 2024-05-10T11:29:43.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.823+0000 7f1367742b80 10 client.27035 fetch_fsmap learned FSMap version 33 2024-05-10T11:29:43.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.823+0000 7f1367742b80 1 -- 172.21.15.100:0/1428005957 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ea27102cd0 con 0x55ea270f1ad0 2024-05-10T11:29:43.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.823+0000 7f133dffb700 1 -- 172.21.15.100:0/1428005957 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f13500092e0 con 0x55ea270f1ad0 2024-05-10T11:29:43.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.823+0000 7f133dffb700 1 -- 172.21.15.100:0/1428005957 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f1350028460 con 0x55ea270f1ad0 2024-05-10T11:29:43.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.824+0000 7f1367742b80 10 client.27035 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T11:29:43.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.824+0000 7f1367742b80 10 client.27035 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:29:43.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.824+0000 7f1367742b80 20 client.27035 populate_metadata read hostname 'smithi100' 2024-05-10T11:29:43.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.824+0000 7f1367742b80 1 --2- 172.21.15.100:0/1428005957 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ea27105250 0x55ea27107640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:43.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.824+0000 7f1367742b80 4 client.27035 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:29:43.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.824+0000 7f1367742b80 1 -- 172.21.15.100:0/1428005957 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea2704b320 con 0x55ea27105250 2024-05-10T11:29:43.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.824+0000 7f135659c700 1 --2- 172.21.15.100:0/1428005957 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ea27105250 0x55ea27107640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:43.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.824+0000 7f135659c700 1 --2- 172.21.15.100:0/1428005957 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ea27105250 0x55ea27107640 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c60aad2035b0c6e1 server_cookie=c3357f71cdb14408 in_seq=0 out_seq=0 2024-05-10T11:29:43.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.824+0000 7f133dffb700 10 client.27035 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:29:43.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.825+0000 7f133dffb700 1 -- 172.21.15.100:0/1428005957 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ea2704b320 con 0x55ea27105250 2024-05-10T11:29:43.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.825+0000 7f133dffb700 10 client.27035 handle_command_reply: tid=0 2024-05-10T11:29:43.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.825+0000 7f1367742b80 1 client.27035 shutdown 2024-05-10T11:29:43.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.825+0000 7f1367742b80 1 -- 172.21.15.100:0/1428005957 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ea270f1ad0 msgr2=0x55ea270f4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:43.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.825+0000 7f1367742b80 1 --2- 172.21.15.100:0/1428005957 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ea270f1ad0 0x55ea270f4ff0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1350011650 tx=0x7f1350004a20 comp rx=0 tx=0).stop 2024-05-10T11:29:43.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.825+0000 7f1367742b80 1 -- 172.21.15.100:0/1428005957 shutdown_connections 2024-05-10T11:29:43.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.825+0000 7f1367742b80 1 --2- 172.21.15.100:0/1428005957 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ea27105250 0x55ea27107640 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:43.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.825+0000 7f1367742b80 1 --2- 172.21.15.100:0/1428005957 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ea270f1ad0 0x55ea270f4ff0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:43.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.825+0000 7f133dffb700 0 client.27035 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:29:43.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.825+0000 7f1367742b80 1 --2- 172.21.15.100:0/1428005957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ea270ebd50 0x55ea270f7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:43.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.825+0000 7f1367742b80 1 --2- 172.21.15.100:0/1428005957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ea270f11b0 0x55ea270f4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:43.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.825+0000 7f1367742b80 1 -- 172.21.15.100:0/1428005957 >> 172.21.15.100:0/1428005957 conn(0x55ea27017b70 msgr2=0x55ea270e0b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:43.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.825+0000 7f1367742b80 1 -- 172.21.15.100:0/1428005957 shutdown_connections 2024-05-10T11:29:43.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.825+0000 7f1367742b80 1 -- 172.21.15.100:0/1428005957 wait complete. 2024-05-10T11:29:43.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.825+0000 7f1367742b80 20 client.27035 trim_cache size 0 max 16384 2024-05-10T11:29:43.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.849+0000 7f1367742b80 1 Processor -- start 2024-05-10T11:29:43.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.849+0000 7f1367742b80 1 -- start start 2024-05-10T11:29:43.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.849+0000 7f1367742b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ea270ebd50 0x55ea270eb810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:43.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.849+0000 7f1367742b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ea270f11b0 0x55ea270e0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:43.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.849+0000 7f1367742b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ea270f1ad0 0x55ea270f0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:43.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.849+0000 7f1367742b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f13581b80a0 con 0x55ea270f1ad0 2024-05-10T11:29:43.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.849+0000 7f1367742b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f13581b5930 con 0x55ea270f11b0 2024-05-10T11:29:43.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.849+0000 7f1367742b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f13581b3780 con 0x55ea270ebd50 2024-05-10T11:29:43.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.849+0000 7f1356d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ea270f1ad0 0x55ea270f0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:43.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.849+0000 7f1356d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ea270f1ad0 0x55ea270f0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43156/0 (socket says 172.21.15.100:43156) 2024-05-10T11:29:43.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.849+0000 7f1356d9d700 1 -- 172.21.15.100:0/2987876768 learned_addr learned my addr 172.21.15.100:0/2987876768 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:29:43.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.849+0000 7f135659c700 1 --2- 172.21.15.100:0/2987876768 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ea270ebd50 0x55ea270eb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:43.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.850+0000 7f1355d9b700 1 --2- 172.21.15.100:0/2987876768 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ea270f11b0 0x55ea270e0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:43.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.850+0000 7f1356d9d700 1 -- 172.21.15.100:0/2987876768 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ea270ebd50 msgr2=0x55ea270eb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:43.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.850+0000 7f1356d9d700 1 --2- 172.21.15.100:0/2987876768 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ea270ebd50 0x55ea270eb810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:43.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.850+0000 7f1356d9d700 1 -- 172.21.15.100:0/2987876768 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ea270f11b0 msgr2=0x55ea270e0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:43.939 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:29:43.939 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-05-10T11:29:43.939 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-05-10T11:29:43.939 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:29:43.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.850+0000 7f1356d9d700 1 --2- 172.21.15.100:0/2987876768 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ea270f11b0 0x55ea270e0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:43.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.850+0000 7f1356d9d700 1 -- 172.21.15.100:0/2987876768 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea270e6de0 con 0x55ea270f1ad0 2024-05-10T11:29:43.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.850+0000 7f1356d9d700 1 --2- 172.21.15.100:0/2987876768 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ea270f1ad0 0x55ea270f0310 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f1350028800 tx=0x7f1350004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c065b01b26860ea5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:43.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.850+0000 7f133ffff700 1 -- 172.21.15.100:0/2987876768 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1350019070 con 0x55ea270f1ad0 2024-05-10T11:29:43.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.850+0000 7f133ffff700 1 -- 172.21.15.100:0/2987876768 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1350028460 con 0x55ea270f1ad0 2024-05-10T11:29:43.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.850+0000 7f133ffff700 1 -- 172.21.15.100:0/2987876768 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1350023030 con 0x55ea270f1ad0 2024-05-10T11:29:43.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.851+0000 7f1367742b80 1 -- 172.21.15.100:0/2987876768 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ea270f1ad0 msgr2=0x55ea270f0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:43.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.851+0000 7f1367742b80 1 --2- 172.21.15.100:0/2987876768 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ea270f1ad0 0x55ea270f0310 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f1350028800 tx=0x7f1350004430 comp rx=0 tx=0).stop 2024-05-10T11:29:43.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.851+0000 7f1367742b80 1 -- 172.21.15.100:0/2987876768 shutdown_connections 2024-05-10T11:29:43.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.851+0000 7f1367742b80 1 --2- 172.21.15.100:0/2987876768 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ea270f1ad0 0x55ea270f0310 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:43.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.851+0000 7f1367742b80 1 --2- 172.21.15.100:0/2987876768 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ea270ebd50 0x55ea270eb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:43.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.851+0000 7f1367742b80 1 --2- 172.21.15.100:0/2987876768 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ea270f11b0 0x55ea270e0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:43.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.851+0000 7f1367742b80 1 -- 172.21.15.100:0/2987876768 >> 172.21.15.100:0/2987876768 conn(0x55ea27017b70 msgr2=0x55ea270efc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:43.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.851+0000 7f1367742b80 1 -- 172.21.15.100:0/2987876768 shutdown_connections 2024-05-10T11:29:43.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.851+0000 7f1367742b80 1 -- 172.21.15.100:0/2987876768 wait complete. 2024-05-10T11:29:43.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.851+0000 7f1367742b80 1 -- start start 2024-05-10T11:29:43.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.851+0000 7f1367742b80 1 Processor -- start 2024-05-10T11:29:43.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.851+0000 7f1367742b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ea270ebd50 0x55ea2710a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:43.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.851+0000 7f1367742b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ea270f11b0 0x55ea2710acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:43.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.851+0000 7f1367742b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ea270f1ad0 0x55ea27107c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:43.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.851+0000 7f1367742b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f13581b3df0 con 0x55ea270f11b0 2024-05-10T11:29:43.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.851+0000 7f1367742b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f13581b5260 con 0x55ea270f1ad0 2024-05-10T11:29:43.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.851+0000 7f1367742b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f13581b44c0 con 0x55ea270ebd50 2024-05-10T11:29:43.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.851+0000 7f1367742b80 10 client.0 _ll_register_callbacks cb 0x55ea26fd15b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:29:43.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.851+0000 7f1355d9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ea270f11b0 0x55ea2710acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:43.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.851+0000 7f135659c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ea270ebd50 0x55ea2710a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:43.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.851+0000 7f1355d9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ea270f11b0 0x55ea2710acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43172/0 (socket says 172.21.15.100:43172) 2024-05-10T11:29:43.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.851+0000 7f1355d9b700 1 -- 172.21.15.100:0/2195059020 learned_addr learned my addr 172.21.15.100:0/2195059020 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:29:43.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.852+0000 7f1356d9d700 1 --2- 172.21.15.100:0/2195059020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ea270f1ad0 0x55ea27107c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:43.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.852+0000 7f1355d9b700 1 -- 172.21.15.100:0/2195059020 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ea270ebd50 msgr2=0x55ea2710a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:43.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.852+0000 7f1355d9b700 1 --2- 172.21.15.100:0/2195059020 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ea270ebd50 0x55ea2710a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:43.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.852+0000 7f1355d9b700 1 -- 172.21.15.100:0/2195059020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ea270f1ad0 msgr2=0x55ea27107c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:43.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.852+0000 7f1355d9b700 1 --2- 172.21.15.100:0/2195059020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ea270f1ad0 0x55ea27107c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:43.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.852+0000 7f1355d9b700 1 -- 172.21.15.100:0/2195059020 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea27108490 con 0x55ea270f11b0 2024-05-10T11:29:43.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.852+0000 7f1356d9d700 1 --2- 172.21.15.100:0/2195059020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ea270f1ad0 0x55ea27107c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:29:43.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.852+0000 7f135659c700 1 --2- 172.21.15.100:0/2195059020 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ea270ebd50 0x55ea2710a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:29:43.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.852+0000 7f1355d9b700 1 --2- 172.21.15.100:0/2195059020 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ea270f11b0 0x55ea2710acd0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f134c0a9390 tx=0x7f134c004660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7557451293942f74 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:43.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.852+0000 7f133dffb700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:29:43.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.852+0000 7f133dffb700 1 -- 172.21.15.100:0/2195059020 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134c01d040 con 0x55ea270f11b0 2024-05-10T11:29:43.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.852+0000 7f133dffb700 1 -- 172.21.15.100:0/2195059020 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f134c0b1fe0 con 0x55ea270f11b0 2024-05-10T11:29:43.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.852+0000 7f1367742b80 1 -- 172.21.15.100:0/2195059020 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1350028460 con 0x55ea270f11b0 2024-05-10T11:29:43.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.852+0000 7f133dffb700 1 -- 172.21.15.100:0/2195059020 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f134c0a9400 con 0x55ea270f11b0 2024-05-10T11:29:43.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.852+0000 7f1367742b80 10 client.17673 fetch_fsmap learned FSMap version 33 2024-05-10T11:29:43.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.852+0000 7f1367742b80 1 -- 172.21.15.100:0/2195059020 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ea2716f410 con 0x55ea270f11b0 2024-05-10T11:29:43.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.852+0000 7f133dffb700 1 -- 172.21.15.100:0/2195059020 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1350028460 con 0x55ea270f11b0 2024-05-10T11:29:43.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.853+0000 7f133dffb700 1 -- 172.21.15.100:0/2195059020 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f134c0ca300 con 0x55ea270f11b0 2024-05-10T11:29:43.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.853+0000 7f1367742b80 10 client.17673 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T11:29:43.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.853+0000 7f1367742b80 10 client.17673 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:29:43.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.853+0000 7f1367742b80 20 client.17673 populate_metadata read hostname 'smithi100' 2024-05-10T11:29:43.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.853+0000 7f1367742b80 1 --2- 172.21.15.100:0/2195059020 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ea2716fbf0 0x55ea27172010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:43.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.853+0000 7f1367742b80 4 client.17673 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T11:29:43.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.853+0000 7f1367742b80 1 -- 172.21.15.100:0/2195059020 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ea270453c0 con 0x55ea2716fbf0 2024-05-10T11:29:43.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.853+0000 7f135659c700 1 --2- 172.21.15.100:0/2195059020 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ea2716fbf0 0x55ea27172010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:43.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.853+0000 7f133dffb700 10 client.17673 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:29:43.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.853+0000 7f135659c700 1 --2- 172.21.15.100:0/2195059020 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ea2716fbf0 0x55ea27172010 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f45d22ae4bc7bd0 server_cookie=cb32671d0759064d in_seq=0 out_seq=0 2024-05-10T11:29:43.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.853+0000 7f133dffb700 1 -- 172.21.15.100:0/2195059020 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ea270453c0 con 0x55ea2716fbf0 2024-05-10T11:29:43.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.853+0000 7f133dffb700 10 client.17673 handle_command_reply: tid=0 2024-05-10T11:29:43.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.853+0000 7f1367742b80 1 client.17673 shutdown 2024-05-10T11:29:43.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.853+0000 7f1367742b80 1 -- 172.21.15.100:0/2195059020 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ea270f11b0 msgr2=0x55ea2710acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:43.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.853+0000 7f1367742b80 1 --2- 172.21.15.100:0/2195059020 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ea270f11b0 0x55ea2710acd0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f134c0a9390 tx=0x7f134c004660 comp rx=0 tx=0).stop 2024-05-10T11:29:43.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.854+0000 7f1367742b80 1 -- 172.21.15.100:0/2195059020 shutdown_connections 2024-05-10T11:29:43.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.854+0000 7f1367742b80 1 --2- 172.21.15.100:0/2195059020 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ea2716fbf0 0x55ea27172010 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:43.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.854+0000 7f1367742b80 1 --2- 172.21.15.100:0/2195059020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ea270f1ad0 0x55ea27107c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:43.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.854+0000 7f133dffb700 0 client.17673 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:29:43.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.854+0000 7f1367742b80 1 --2- 172.21.15.100:0/2195059020 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ea270ebd50 0x55ea2710a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:43.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.854+0000 7f1367742b80 1 --2- 172.21.15.100:0/2195059020 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ea270f11b0 0x55ea2710acd0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:43.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.854+0000 7f1367742b80 1 -- 172.21.15.100:0/2195059020 >> 172.21.15.100:0/2195059020 conn(0x55ea27017b70 msgr2=0x55ea270f25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:43.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.854+0000 7f1367742b80 1 -- 172.21.15.100:0/2195059020 shutdown_connections 2024-05-10T11:29:43.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.854+0000 7f1367742b80 1 -- 172.21.15.100:0/2195059020 wait complete. 2024-05-10T11:29:43.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.854+0000 7f1367742b80 20 client.17673 trim_cache size 0 max 16384 2024-05-10T11:29:43.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.854+0000 7f135ceaf700 1 -- 172.21.15.100:0/3864566867 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f13400b2880 msgr2=0x7f13400b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:43.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.854+0000 7f135ceaf700 1 --2- 172.21.15.100:0/3864566867 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f13400b2880 0x7f13400b4d40 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f1348006010 tx=0x7f134800b260 comp rx=0 tx=0).stop 2024-05-10T11:29:43.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.854+0000 7f135ceaf700 1 -- 172.21.15.100:0/3864566867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1358102d60 msgr2=0x7f1358198c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:43.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.854+0000 7f135ceaf700 1 --2- 172.21.15.100:0/3864566867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1358102d60 0x7f1358198c10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f134c00d750 tx=0x7f134c00da60 comp rx=0 tx=0).stop 2024-05-10T11:29:43.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.854+0000 7f135ceaf700 1 -- 172.21.15.100:0/3864566867 shutdown_connections 2024-05-10T11:29:43.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.855+0000 7f135ceaf700 1 --2- 172.21.15.100:0/3864566867 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f13400b2880 0x7f13400b4d40 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:43.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.855+0000 7f135ceaf700 1 --2- 172.21.15.100:0/3864566867 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1358108410 0x7f135819d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:43.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.855+0000 7f135ceaf700 1 --2- 172.21.15.100:0/3864566867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13581023a0 0x7f13581986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:43.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.855+0000 7f135ceaf700 1 --2- 172.21.15.100:0/3864566867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1358102d60 0x7f1358198c10 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:43.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.855+0000 7f135ceaf700 1 -- 172.21.15.100:0/3864566867 >> 172.21.15.100:0/3864566867 conn(0x7f1358079350 msgr2=0x7f1358101280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:43.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.855+0000 7f135ceaf700 1 -- 172.21.15.100:0/3864566867 shutdown_connections 2024-05-10T11:29:43.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:43.855+0000 7f135ceaf700 1 -- 172.21.15.100:0/3864566867 wait complete. 2024-05-10T11:29:43.982 INFO:tasks.workunit.client.0.smithi100.stdout: 1 295017 2.72 MB/sec execute 458 sec latency 18.461 ms 2024-05-10T11:29:44.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:29:43 smithi100 ceph-mon[30296]: pgmap v761: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 1.2 MiB/s wr, 128 op/s 2024-05-10T11:29:44.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:29:43 smithi178 ceph-mon[33359]: pgmap v761: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 1.2 MiB/s wr, 128 op/s 2024-05-10T11:29:44.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:29:43 smithi188 ceph-mon[34075]: pgmap v761: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 1.2 MiB/s wr, 128 op/s 2024-05-10T11:29:44.857 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T11:29:44.858 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T11:29:44.981 INFO:tasks.workunit.client.0.smithi100.stdout: 1 295510 2.71 MB/sec execute 459 sec latency 19.997 ms 2024-05-10T11:29:45.120 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:29:45.981 INFO:tasks.workunit.client.0.smithi100.stdout: 1 296077 2.72 MB/sec execute 460 sec latency 14.314 ms 2024-05-10T11:29:46.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:29:45 smithi178 ceph-mon[33359]: pgmap v762: 129 pgs: 129 active+clean; 116 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 1.3 MiB/s wr, 139 op/s 2024-05-10T11:29:46.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:29:45 smithi188 ceph-mon[34075]: pgmap v762: 129 pgs: 129 active+clean; 116 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 1.3 MiB/s wr, 139 op/s 2024-05-10T11:29:46.280 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:29:45 smithi100 ceph-mon[30296]: pgmap v762: 129 pgs: 129 active+clean; 116 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 1.3 MiB/s wr, 139 op/s 2024-05-10T11:29:46.309 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.309+0000 7f5cc9f90640 1 -- 172.21.15.100:0/1745420301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cc4101910 msgr2=0x7f5cc4101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:46.309 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.309+0000 7f5cc9f90640 1 --2- 172.21.15.100:0/1745420301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cc4101910 0x7f5cc4101d30 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4009900 tx=0x7f5cb402a410 comp rx=0 tx=0).stop 2024-05-10T11:29:46.309 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.309+0000 7f5cc9f90640 1 -- 172.21.15.100:0/1745420301 shutdown_connections 2024-05-10T11:29:46.309 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.309+0000 7f5cc9f90640 1 --2- 172.21.15.100:0/1745420301 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5cc41034c0 0x7f5cc4109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:46.309 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.309+0000 7f5cc9f90640 1 --2- 172.21.15.100:0/1745420301 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5cc4102b00 0x7f5cc4102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:46.309 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.309+0000 7f5cc9f90640 1 --2- 172.21.15.100:0/1745420301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cc4101910 0x7f5cc4101d30 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:46.309 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.309+0000 7f5cc9f90640 1 -- 172.21.15.100:0/1745420301 >> 172.21.15.100:0/1745420301 conn(0x7f5cc40fce80 msgr2=0x7f5cc40ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:46.309 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.309+0000 7f5cc9f90640 1 -- 172.21.15.100:0/1745420301 shutdown_connections 2024-05-10T11:29:46.310 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.309+0000 7f5cc9f90640 1 -- 172.21.15.100:0/1745420301 wait complete. 2024-05-10T11:29:46.310 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.309+0000 7f5cc9f90640 1 Processor -- start 2024-05-10T11:29:46.312 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.310+0000 7f5cc9f90640 1 -- start start 2024-05-10T11:29:46.312 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.310+0000 7f5cc9f90640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cc4101910 0x7f5cc41979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:46.312 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.310+0000 7f5cc9f90640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5cc4102b00 0x7f5cc4197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:46.312 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.310+0000 7f5cc9f90640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5cc41034c0 0x7f5cc419ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:46.312 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.310+0000 7f5cc9f90640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc410d2c0 con 0x7f5cc4101910 2024-05-10T11:29:46.312 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.310+0000 7f5cc9f90640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc410d140 con 0x7f5cc41034c0 2024-05-10T11:29:46.312 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.310+0000 7f5cc9f90640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc410d440 con 0x7f5cc4102b00 2024-05-10T11:29:46.312 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.310+0000 7f5cc37fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cc4101910 0x7f5cc41979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:46.313 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.310+0000 7f5cc37fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cc4101910 0x7f5cc41979d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43190/0 (socket says 172.21.15.100:43190) 2024-05-10T11:29:46.313 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.310+0000 7f5cc37fe640 1 -- 172.21.15.100:0/2463703957 learned_addr learned my addr 172.21.15.100:0/2463703957 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:29:46.313 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.310+0000 7f5cc2ffd640 1 --2- 172.21.15.100:0/2463703957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5cc4102b00 0x7f5cc4197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:46.313 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.310+0000 7f5cc3fff640 1 --2- 172.21.15.100:0/2463703957 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5cc41034c0 0x7f5cc419ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:46.313 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.310+0000 7f5cc37fe640 1 -- 172.21.15.100:0/2463703957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5cc4102b00 msgr2=0x7f5cc4197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:46.313 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.310+0000 7f5cc37fe640 1 --2- 172.21.15.100:0/2463703957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5cc4102b00 0x7f5cc4197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:46.313 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.310+0000 7f5cc37fe640 1 -- 172.21.15.100:0/2463703957 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5cc41034c0 msgr2=0x7f5cc419ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:46.313 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.310+0000 7f5cc37fe640 1 --2- 172.21.15.100:0/2463703957 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5cc41034c0 0x7f5cc419ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:46.313 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.310+0000 7f5cc37fe640 1 -- 172.21.15.100:0/2463703957 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cc419f5f0 con 0x7f5cc4101910 2024-05-10T11:29:46.313 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.310+0000 7f5cc37fe640 1 --2- 172.21.15.100:0/2463703957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cc4101910 0x7f5cc41979d0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4009230 tx=0x7f5cb40040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:46.313 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.311+0000 7f5cc0ff9640 1 -- 172.21.15.100:0/2463703957 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb403f030 con 0x7f5cc4101910 2024-05-10T11:29:46.313 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.311+0000 7f5cc9f90640 1 -- 172.21.15.100:0/2463703957 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cc419f880 con 0x7f5cc4101910 2024-05-10T11:29:46.313 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.311+0000 7f5cc0ff9640 1 -- 172.21.15.100:0/2463703957 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5cb40044c0 con 0x7f5cc4101910 2024-05-10T11:29:46.313 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.311+0000 7f5cc0ff9640 1 -- 172.21.15.100:0/2463703957 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb4035070 con 0x7f5cc4101910 2024-05-10T11:29:46.313 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.311+0000 7f5cc9f90640 1 -- 172.21.15.100:0/2463703957 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cc419fdc0 con 0x7f5cc4101910 2024-05-10T11:29:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.313+0000 7f5cc0ff9640 1 -- 172.21.15.100:0/2463703957 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f5cb4042030 con 0x7f5cc4101910 2024-05-10T11:29:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.313+0000 7f5cccd83740 1 Processor -- start 2024-05-10T11:29:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.313+0000 7f5cccd83740 1 -- start start 2024-05-10T11:29:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.313+0000 7f5cccd83740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612d9bcefc0 0x5612d9bcf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.313+0000 7f5cccd83740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612d9bcf9b0 0x5612d9bd44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.314+0000 7f5cccd83740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612d9bd4b20 0x5612d9bd6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.314+0000 7f5cccd83740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5612d9b315d0 con 0x5612d9bd4b20 2024-05-10T11:29:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.314+0000 7f5cccd83740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5612d9aea890 con 0x5612d9bcf9b0 2024-05-10T11:29:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.314+0000 7f5cccd83740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5612d9b2d590 con 0x5612d9bcefc0 2024-05-10T11:29:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.314+0000 7f5cc2ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612d9bd4b20 0x5612d9bd6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.314+0000 7f5cc3fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612d9bcefc0 0x5612d9bcf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.314+0000 7f5cc2ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612d9bd4b20 0x5612d9bd6fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43192/0 (socket says 172.21.15.100:43192) 2024-05-10T11:29:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.314+0000 7f5cc2ffd640 1 -- 172.21.15.100:0/1990610239 learned_addr learned my addr 172.21.15.100:0/1990610239 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:29:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.314+0000 7f5cc3fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612d9bcefc0 0x5612d9bcf3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:49908/0 (socket says 172.21.15.100:49908) 2024-05-10T11:29:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.314+0000 7f5cc37fe640 1 --2- 172.21.15.100:0/1990610239 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612d9bcf9b0 0x5612d9bd44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.314+0000 7f5cc3fff640 1 -- 172.21.15.100:0/1990610239 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612d9bcf9b0 msgr2=0x5612d9bd44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.314+0000 7f5cc3fff640 1 --2- 172.21.15.100:0/1990610239 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612d9bcf9b0 0x5612d9bd44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.314+0000 7f5cc3fff640 1 -- 172.21.15.100:0/1990610239 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612d9bd4b20 msgr2=0x5612d9bd6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.314+0000 7f5cc3fff640 1 --2- 172.21.15.100:0/1990610239 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612d9bd4b20 0x5612d9bd6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.314+0000 7f5cc3fff640 1 -- 172.21.15.100:0/1990610239 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612d9843210 con 0x5612d9bcefc0 2024-05-10T11:29:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.314+0000 7f5cc2ffd640 1 --2- 172.21.15.100:0/1990610239 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612d9bd4b20 0x5612d9bd6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:29:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.314+0000 7f5cc0ff9640 1 --2- 172.21.15.100:0/2463703957 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f5cac0b28a0 0x7f5cac0b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.314+0000 7f5cc3fff640 1 --2- 172.21.15.100:0/1990610239 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612d9bcefc0 0x5612d9bcf3e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc009f50 tx=0x7f5cbc003e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9922fee22c8dfd99 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.314+0000 7f5cc37fe640 1 --2- 172.21.15.100:0/2463703957 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f5cac0b28a0 0x7f5cac0b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.314+0000 7f5cc0ff9640 1 -- 172.21.15.100:0/2463703957 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f5cb4102a40 con 0x7f5cc4101910 2024-05-10T11:29:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.314+0000 7f5ca9ffb640 1 -- 172.21.15.100:0/1990610239 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc011440 con 0x5612d9bcefc0 2024-05-10T11:29:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.314+0000 7f5ca9ffb640 1 -- 172.21.15.100:0/1990610239 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5cbc015030 con 0x5612d9bcefc0 2024-05-10T11:29:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.314+0000 7f5ca9ffb640 1 -- 172.21.15.100:0/1990610239 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc0107c0 con 0x5612d9bcefc0 2024-05-10T11:29:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.315+0000 7f5cccd83740 1 -- 172.21.15.100:0/1990610239 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612d9bcefc0 msgr2=0x5612d9bcf3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.315+0000 7f5cc37fe640 1 --2- 172.21.15.100:0/2463703957 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f5cac0b28a0 0x7f5cac0b4d60 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4100550 tx=0x7f5cb41064c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.315+0000 7f5cccd83740 1 --2- 172.21.15.100:0/1990610239 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612d9bcefc0 0x5612d9bcf3e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc009f50 tx=0x7f5cbc003e00 comp rx=0 tx=0).stop 2024-05-10T11:29:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.315+0000 7f5cccd83740 1 -- 172.21.15.100:0/1990610239 shutdown_connections 2024-05-10T11:29:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.315+0000 7f5cccd83740 1 --2- 172.21.15.100:0/1990610239 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612d9bd4b20 0x5612d9bd6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.315+0000 7f5cccd83740 1 --2- 172.21.15.100:0/1990610239 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612d9bcf9b0 0x5612d9bd44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.315+0000 7f5cccd83740 1 --2- 172.21.15.100:0/1990610239 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612d9bcefc0 0x5612d9bcf3e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.315+0000 7f5cccd83740 1 -- 172.21.15.100:0/1990610239 >> 172.21.15.100:0/1990610239 conn(0x5612d9bc8420 msgr2=0x5612d9bc8820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.315+0000 7f5cccd83740 1 -- 172.21.15.100:0/1990610239 shutdown_connections 2024-05-10T11:29:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.315+0000 7f5cccd83740 1 -- 172.21.15.100:0/1990610239 wait complete. 2024-05-10T11:29:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.316+0000 7f5cccd83740 1 -- start start 2024-05-10T11:29:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.316+0000 7f5cccd83740 1 Processor -- start 2024-05-10T11:29:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.316+0000 7f5cccd83740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612d9bcefc0 0x5612d9bd9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.316+0000 7f5cccd83740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612d9bcf9b0 0x5612d9bd97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.316+0000 7f5cccd83740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612d9bd4b20 0x5612d9be07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.316+0000 7f5cccd83740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5612d9bd7ad0 con 0x5612d9bcf9b0 2024-05-10T11:29:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.316+0000 7f5cccd83740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5612d9aee370 con 0x5612d9bcefc0 2024-05-10T11:29:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.316+0000 7f5cccd83740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5612d9bd7c50 con 0x5612d9bd4b20 2024-05-10T11:29:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.316+0000 7f5cccd83740 10 client.0 _ll_register_callbacks cb 0x5612d9842a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:29:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.316+0000 7f5cc2ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612d9bcf9b0 0x5612d9bd97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.316+0000 7f5cc2ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612d9bcf9b0 0x5612d9bd97b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43194/0 (socket says 172.21.15.100:43194) 2024-05-10T11:29:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.316+0000 7f5cc2ffd640 1 -- 172.21.15.100:0/1535643748 learned_addr learned my addr 172.21.15.100:0/1535643748 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:29:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.316+0000 7f5cc3fff640 1 --2- 172.21.15.100:0/1535643748 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612d9bd4b20 0x5612d9be07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.316+0000 7f5cc3fff640 1 --2- 172.21.15.100:0/1535643748 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612d9bcefc0 0x5612d9bd9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.317+0000 7f5cc2ffd640 1 -- 172.21.15.100:0/1535643748 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612d9bd4b20 msgr2=0x5612d9be07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.317+0000 7f5cc2ffd640 1 --2- 172.21.15.100:0/1535643748 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612d9bd4b20 0x5612d9be07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.317+0000 7f5cc2ffd640 1 -- 172.21.15.100:0/1535643748 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612d9bcefc0 msgr2=0x5612d9bd9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.317+0000 7f5cc2ffd640 1 --2- 172.21.15.100:0/1535643748 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612d9bcefc0 0x5612d9bd9270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:46.324 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.317+0000 7f5cc2ffd640 1 -- 172.21.15.100:0/1535643748 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612d990f620 con 0x5612d9bcf9b0 2024-05-10T11:29:46.324 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.317+0000 7f5cc3fff640 1 --2- 172.21.15.100:0/1535643748 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612d9bcefc0 0x5612d9bd9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:29:46.324 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.317+0000 7f5cc2ffd640 1 --2- 172.21.15.100:0/1535643748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612d9bcf9b0 0x5612d9bd97b0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f5cb80126e0 tx=0x7f5cb8012ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de881aa678fe5c61 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:46.324 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.317+0000 7f5ca2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:29:46.324 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.317+0000 7f5ca2ffd640 1 -- 172.21.15.100:0/1535643748 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb80121d0 con 0x5612d9bcf9b0 2024-05-10T11:29:46.324 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.317+0000 7f5ca2ffd640 1 -- 172.21.15.100:0/1535643748 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f5cb8004050 con 0x5612d9bcf9b0 2024-05-10T11:29:46.324 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.317+0000 7f5cccd83740 1 -- 172.21.15.100:0/1535643748 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cbc015030 con 0x5612d9bcf9b0 2024-05-10T11:29:46.324 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.317+0000 7f5ca2ffd640 1 -- 172.21.15.100:0/1535643748 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5cb8025050 con 0x5612d9bcf9b0 2024-05-10T11:29:46.324 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.317+0000 7f5cccd83740 10 client.17685 fetch_fsmap learned FSMap version 33 2024-05-10T11:29:46.324 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.317+0000 7f5cccd83740 1 -- 172.21.15.100:0/1535643748 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5612d9be38a0 con 0x5612d9bcf9b0 2024-05-10T11:29:46.324 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.317+0000 7f5ca2ffd640 1 -- 172.21.15.100:0/1535643748 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5cbc015030 con 0x5612d9bcf9b0 2024-05-10T11:29:46.324 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.317+0000 7f5ca2ffd640 1 -- 172.21.15.100:0/1535643748 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f5cb8003820 con 0x5612d9bcf9b0 2024-05-10T11:29:46.324 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.318+0000 7f5cccd83740 10 client.17685 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T11:29:46.324 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.318+0000 7f5cccd83740 10 client.17685 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:29:46.324 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.318+0000 7f5cccd83740 20 client.17685 populate_metadata read hostname 'smithi100' 2024-05-10T11:29:46.325 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.318+0000 7f5cccd83740 1 --2- 172.21.15.100:0/1535643748 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5612d9be53b0 0x5612d9be7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:46.325 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.318+0000 7f5cccd83740 4 client.17685 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:29:46.325 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.318+0000 7f5cccd83740 1 -- 172.21.15.100:0/1535643748 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612d9b3f1e0 con 0x5612d9be53b0 2024-05-10T11:29:46.325 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.318+0000 7f5cc3fff640 1 --2- 172.21.15.100:0/1535643748 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5612d9be53b0 0x5612d9be7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:46.325 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.318+0000 7f5ca2ffd640 10 client.17685 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:29:46.325 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.318+0000 7f5cc3fff640 1 --2- 172.21.15.100:0/1535643748 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5612d9be53b0 0x5612d9be7810 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b628606d510b0c4 server_cookie=eddba599273eef58 in_seq=0 out_seq=0 2024-05-10T11:29:46.325 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.319+0000 7f5ca2ffd640 1 -- 172.21.15.100:0/1535643748 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5612d9b3f1e0 con 0x5612d9be53b0 2024-05-10T11:29:46.325 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.319+0000 7f5ca2ffd640 10 client.17685 handle_command_reply: tid=0 2024-05-10T11:29:46.325 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.319+0000 7f5cccd83740 1 client.17685 shutdown 2024-05-10T11:29:46.325 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.319+0000 7f5cccd83740 1 -- 172.21.15.100:0/1535643748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612d9bcf9b0 msgr2=0x5612d9bd97b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:46.325 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.319+0000 7f5cccd83740 1 --2- 172.21.15.100:0/1535643748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612d9bcf9b0 0x5612d9bd97b0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f5cb80126e0 tx=0x7f5cb8012ba0 comp rx=0 tx=0).stop 2024-05-10T11:29:46.325 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.319+0000 7f5cccd83740 1 -- 172.21.15.100:0/1535643748 shutdown_connections 2024-05-10T11:29:46.325 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.319+0000 7f5cccd83740 1 --2- 172.21.15.100:0/1535643748 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5612d9be53b0 0x5612d9be7810 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:46.325 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.319+0000 7f5cccd83740 1 --2- 172.21.15.100:0/1535643748 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612d9bd4b20 0x5612d9be07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:46.325 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.319+0000 7f5ca2ffd640 0 client.17685 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:29:46.325 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.319+0000 7f5cccd83740 1 --2- 172.21.15.100:0/1535643748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612d9bcf9b0 0x5612d9bd97b0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:46.326 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.319+0000 7f5cccd83740 1 --2- 172.21.15.100:0/1535643748 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612d9bcefc0 0x5612d9bd9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:46.326 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.319+0000 7f5cccd83740 1 -- 172.21.15.100:0/1535643748 >> 172.21.15.100:0/1535643748 conn(0x5612d9bc8420 msgr2=0x5612d9bc7580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:46.326 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.319+0000 7f5cccd83740 1 -- 172.21.15.100:0/1535643748 shutdown_connections 2024-05-10T11:29:46.326 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.319+0000 7f5cccd83740 1 -- 172.21.15.100:0/1535643748 wait complete. 2024-05-10T11:29:46.326 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.319+0000 7f5cccd83740 20 client.17685 trim_cache size 0 max 16384 2024-05-10T11:29:46.341 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.337+0000 7f5cccd83740 1 Processor -- start 2024-05-10T11:29:46.341 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.337+0000 7f5cccd83740 1 -- start start 2024-05-10T11:29:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.337+0000 7f5cccd83740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612d9bcefc0 0x5612d9bc33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.337+0000 7f5cccd83740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612d9bcf9b0 0x5612d9bd0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.337+0000 7f5cccd83740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612d9bd4b20 0x5612d9bd1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.337+0000 7f5cccd83740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc41ab970 con 0x5612d9bcefc0 2024-05-10T11:29:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.337+0000 7f5cccd83740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc41a9240 con 0x5612d9bcf9b0 2024-05-10T11:29:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.337+0000 7f5cccd83740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc41a7090 con 0x5612d9bd4b20 2024-05-10T11:29:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.337+0000 7f5cc3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612d9bcefc0 0x5612d9bc33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.337+0000 7f5cc3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612d9bcefc0 0x5612d9bc33e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43202/0 (socket says 172.21.15.100:43202) 2024-05-10T11:29:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.337+0000 7f5cc3fff640 1 -- 172.21.15.100:0/3640180554 learned_addr learned my addr 172.21.15.100:0/3640180554 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:29:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.337+0000 7f5cc3fff640 1 --2- 172.21.15.100:0/3640180554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612d9bd4b20 0x5612d9bd1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.337+0000 7f5cc2ffd640 1 --2- 172.21.15.100:0/3640180554 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612d9bcf9b0 0x5612d9bd0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.337+0000 7f5cc3fff640 1 -- 172.21.15.100:0/3640180554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612d9bd4b20 msgr2=0x5612d9bd1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.337+0000 7f5cc3fff640 1 --2- 172.21.15.100:0/3640180554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612d9bd4b20 0x5612d9bd1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.337+0000 7f5cc3fff640 1 -- 172.21.15.100:0/3640180554 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612d9bcf9b0 msgr2=0x5612d9bd0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:46.343 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.337+0000 7f5cc3fff640 1 --2- 172.21.15.100:0/3640180554 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612d9bcf9b0 0x5612d9bd0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:46.343 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.337+0000 7f5cc3fff640 1 -- 172.21.15.100:0/3640180554 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612d9bd8320 con 0x5612d9bcefc0 2024-05-10T11:29:46.343 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.337+0000 7f5cc2ffd640 1 --2- 172.21.15.100:0/3640180554 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612d9bcf9b0 0x5612d9bd0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:29:46.343 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.337+0000 7f5cc3fff640 1 --2- 172.21.15.100:0/3640180554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612d9bcefc0 0x5612d9bc33e0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc004410 tx=0x7f5cbc000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=921d2da62acd8567 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:46.343 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.338+0000 7f5ca97fa640 1 -- 172.21.15.100:0/3640180554 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc024040 con 0x5612d9bcefc0 2024-05-10T11:29:46.343 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.338+0000 7f5ca97fa640 1 -- 172.21.15.100:0/3640180554 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5cbc015070 con 0x5612d9bcefc0 2024-05-10T11:29:46.343 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.338+0000 7f5ca97fa640 1 -- 172.21.15.100:0/3640180554 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc011790 con 0x5612d9bcefc0 2024-05-10T11:29:46.343 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.338+0000 7f5cccd83740 1 -- 172.21.15.100:0/3640180554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612d9bcefc0 msgr2=0x5612d9bc33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:46.343 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.338+0000 7f5cccd83740 1 --2- 172.21.15.100:0/3640180554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612d9bcefc0 0x5612d9bc33e0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc004410 tx=0x7f5cbc000f40 comp rx=0 tx=0).stop 2024-05-10T11:29:46.343 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.338+0000 7f5cccd83740 1 -- 172.21.15.100:0/3640180554 shutdown_connections 2024-05-10T11:29:46.343 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.338+0000 7f5cccd83740 1 --2- 172.21.15.100:0/3640180554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612d9bd4b20 0x5612d9bd1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:46.343 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.338+0000 7f5cccd83740 1 --2- 172.21.15.100:0/3640180554 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612d9bcf9b0 0x5612d9bd0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:46.343 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.338+0000 7f5cccd83740 1 --2- 172.21.15.100:0/3640180554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612d9bcefc0 0x5612d9bc33e0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:46.343 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.338+0000 7f5cccd83740 1 -- 172.21.15.100:0/3640180554 >> 172.21.15.100:0/3640180554 conn(0x5612d9bc8420 msgr2=0x5612d9bc8800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:46.343 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.338+0000 7f5cccd83740 1 -- 172.21.15.100:0/3640180554 shutdown_connections 2024-05-10T11:29:46.344 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.338+0000 7f5cccd83740 1 -- 172.21.15.100:0/3640180554 wait complete. 2024-05-10T11:29:46.344 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.339+0000 7f5cccd83740 1 -- start start 2024-05-10T11:29:46.344 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.339+0000 7f5cccd83740 1 Processor -- start 2024-05-10T11:29:46.344 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.339+0000 7f5cccd83740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612d9bcefc0 0x5612d9bdd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:46.344 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.339+0000 7f5cccd83740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612d9bcf9b0 0x5612d9bdd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:46.344 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.339+0000 7f5cccd83740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612d9bd4b20 0x5612d9bde700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:46.344 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.339+0000 7f5cccd83740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc41a7700 con 0x5612d9bcefc0 2024-05-10T11:29:46.344 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.339+0000 7f5cccd83740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc41a8b70 con 0x5612d9bd4b20 2024-05-10T11:29:46.344 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.339+0000 7f5cccd83740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc41a7dd0 con 0x5612d9bcf9b0 2024-05-10T11:29:46.344 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.339+0000 7f5cccd83740 10 client.0 _ll_register_callbacks cb 0x7f5cb8009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:29:46.344 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.339+0000 7f5cc3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612d9bcefc0 0x5612d9bdd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:46.344 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.339+0000 7f5cc2ffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612d9bcf9b0 0x5612d9bdd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:46.344 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.339+0000 7f5cc3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612d9bcefc0 0x5612d9bdd1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43204/0 (socket says 172.21.15.100:43204) 2024-05-10T11:29:46.344 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.339+0000 7f5cc3fff640 1 -- 172.21.15.100:0/954709209 learned_addr learned my addr 172.21.15.100:0/954709209 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:29:46.344 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.339+0000 7f5cc3fff640 1 --2- 172.21.15.100:0/954709209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612d9bd4b20 0x5612d9bde700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.340+0000 7f5cc2ffd640 1 -- 172.21.15.100:0/954709209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612d9bd4b20 msgr2=0x5612d9bde700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.340+0000 7f5cc2ffd640 1 --2- 172.21.15.100:0/954709209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612d9bd4b20 0x5612d9bde700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.340+0000 7f5cc2ffd640 1 -- 172.21.15.100:0/954709209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612d9bcefc0 msgr2=0x5612d9bdd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.340+0000 7f5cc2ffd640 1 --2- 172.21.15.100:0/954709209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612d9bcefc0 0x5612d9bdd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.340+0000 7f5cc2ffd640 1 -- 172.21.15.100:0/954709209 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612d9bdecf0 con 0x5612d9bcf9b0 2024-05-10T11:29:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.340+0000 7f5cc3fff640 1 --2- 172.21.15.100:0/954709209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612d9bcefc0 0x5612d9bdd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:29:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.340+0000 7f5cc2ffd640 1 --2- 172.21.15.100:0/954709209 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612d9bcf9b0 0x5612d9bdd6f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5cb80126b0 tx=0x7f5cb8009f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f19376b64a87650 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.340+0000 7f5ca2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:29:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.340+0000 7f5ca2ffd640 1 -- 172.21.15.100:0/954709209 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb8003980 con 0x5612d9bcf9b0 2024-05-10T11:29:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.340+0000 7f5cccd83740 1 -- 172.21.15.100:0/954709209 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cbc015070 con 0x5612d9bcf9b0 2024-05-10T11:29:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.340+0000 7f5ca2ffd640 1 -- 172.21.15.100:0/954709209 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f5cb8018040 con 0x5612d9bcf9b0 2024-05-10T11:29:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.340+0000 7f5ca2ffd640 1 -- 172.21.15.100:0/954709209 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5cb8038050 con 0x5612d9bcf9b0 2024-05-10T11:29:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.340+0000 7f5cccd83740 10 client.27142 fetch_fsmap learned FSMap version 33 2024-05-10T11:29:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.340+0000 7f5cccd83740 1 -- 172.21.15.100:0/954709209 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5612d9be0000 con 0x5612d9bcf9b0 2024-05-10T11:29:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.340+0000 7f5ca2ffd640 1 -- 172.21.15.100:0/954709209 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5cbc015070 con 0x5612d9bcf9b0 2024-05-10T11:29:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.341+0000 7f5ca2ffd640 1 -- 172.21.15.100:0/954709209 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f5cb800fc70 con 0x5612d9bcf9b0 2024-05-10T11:29:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.341+0000 7f5cccd83740 10 client.27142 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T11:29:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.341+0000 7f5cccd83740 10 client.27142 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:29:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.341+0000 7f5cccd83740 20 client.27142 populate_metadata read hostname 'smithi100' 2024-05-10T11:29:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.341+0000 7f5cccd83740 1 --2- 172.21.15.100:0/954709209 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5612d9c50370 0x5612d9c52790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.341+0000 7f5cccd83740 4 client.27142 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:29:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.341+0000 7f5cccd83740 1 -- 172.21.15.100:0/954709209 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5612d9b39050 con 0x5612d9c50370 2024-05-10T11:29:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.341+0000 7f5cc3fff640 1 --2- 172.21.15.100:0/954709209 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5612d9c50370 0x5612d9c52790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.341+0000 7f5ca2ffd640 10 client.27142 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:29:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.341+0000 7f5cc3fff640 1 --2- 172.21.15.100:0/954709209 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5612d9c50370 0x5612d9c52790 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bba5387fc9aea2ab server_cookie=fa7d6ec417f46561 in_seq=0 out_seq=0 2024-05-10T11:29:46.367 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.367+0000 7f5ca2ffd640 1 -- 172.21.15.100:0/954709209 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5612d9b39050 con 0x5612d9c50370 2024-05-10T11:29:46.367 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.367+0000 7f5ca2ffd640 10 client.27142 handle_command_reply: tid=0 2024-05-10T11:29:46.367 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.367+0000 7f5cccd83740 1 client.27142 shutdown 2024-05-10T11:29:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.367+0000 7f5cccd83740 1 -- 172.21.15.100:0/954709209 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612d9bcf9b0 msgr2=0x5612d9bdd6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.367+0000 7f5cccd83740 1 --2- 172.21.15.100:0/954709209 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612d9bcf9b0 0x5612d9bdd6f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5cb80126b0 tx=0x7f5cb8009f40 comp rx=0 tx=0).stop 2024-05-10T11:29:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.367+0000 7f5cccd83740 1 -- 172.21.15.100:0/954709209 shutdown_connections 2024-05-10T11:29:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.367+0000 7f5cccd83740 1 --2- 172.21.15.100:0/954709209 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5612d9c50370 0x5612d9c52790 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.367+0000 7f5cccd83740 1 --2- 172.21.15.100:0/954709209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612d9bd4b20 0x5612d9bde700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.367+0000 7f5cccd83740 1 --2- 172.21.15.100:0/954709209 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612d9bcf9b0 0x5612d9bdd6f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.367+0000 7f5cccd83740 1 --2- 172.21.15.100:0/954709209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612d9bcefc0 0x5612d9bdd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.367+0000 7f5ca2ffd640 0 client.27142 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:29:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.367+0000 7f5cccd83740 1 -- 172.21.15.100:0/954709209 >> 172.21.15.100:0/954709209 conn(0x5612d9bc8420 msgr2=0x5612d9bc33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.367+0000 7f5cccd83740 1 -- 172.21.15.100:0/954709209 shutdown_connections 2024-05-10T11:29:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.367+0000 7f5cccd83740 1 -- 172.21.15.100:0/954709209 wait complete. 2024-05-10T11:29:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.367+0000 7f5cccd83740 20 client.27142 trim_cache size 0 max 16384 2024-05-10T11:29:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:29:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:29:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:29:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:29:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.368+0000 7f5cc9f90640 1 -- 172.21.15.100:0/2463703957 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f5cac0b28a0 msgr2=0x7f5cac0b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.368+0000 7f5cc9f90640 1 --2- 172.21.15.100:0/2463703957 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f5cac0b28a0 0x7f5cac0b4d60 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4100550 tx=0x7f5cb41064c0 comp rx=0 tx=0).stop 2024-05-10T11:29:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.368+0000 7f5cc9f90640 1 -- 172.21.15.100:0/2463703957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cc4101910 msgr2=0x7f5cc41979d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.368+0000 7f5cc9f90640 1 --2- 172.21.15.100:0/2463703957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cc4101910 0x7f5cc41979d0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4009230 tx=0x7f5cb40040c0 comp rx=0 tx=0).stop 2024-05-10T11:29:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.368+0000 7f5cc9f90640 1 -- 172.21.15.100:0/2463703957 shutdown_connections 2024-05-10T11:29:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.368+0000 7f5cc9f90640 1 --2- 172.21.15.100:0/2463703957 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f5cac0b28a0 0x7f5cac0b4d60 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.368+0000 7f5cc9f90640 1 --2- 172.21.15.100:0/2463703957 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5cc41034c0 0x7f5cc419ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.368+0000 7f5cc9f90640 1 --2- 172.21.15.100:0/2463703957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5cc4102b00 0x7f5cc4197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.368+0000 7f5cc9f90640 1 --2- 172.21.15.100:0/2463703957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cc4101910 0x7f5cc41979d0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.368+0000 7f5cc9f90640 1 -- 172.21.15.100:0/2463703957 >> 172.21.15.100:0/2463703957 conn(0x7f5cc40fce80 msgr2=0x7f5cc4105ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.368+0000 7f5cc9f90640 1 -- 172.21.15.100:0/2463703957 shutdown_connections 2024-05-10T11:29:46.371 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:46.368+0000 7f5cc9f90640 1 -- 172.21.15.100:0/2463703957 wait complete. 2024-05-10T11:29:46.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:46.976+0000 7f4456f33700 1 -- 172.21.15.100:0/3565773568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4450102d60 msgr2=0x7f445010f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:46.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:46.976+0000 7f4456f33700 1 --2- 172.21.15.100:0/3565773568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4450102d60 0x7f445010f250 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f444800b0a0 tx=0x7f444800b3b0 comp rx=0 tx=0).stop 2024-05-10T11:29:46.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:46.976+0000 7f4456f33700 1 -- 172.21.15.100:0/3565773568 shutdown_connections 2024-05-10T11:29:46.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:46.976+0000 7f4456f33700 1 --2- 172.21.15.100:0/3565773568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4450102d60 0x7f445010f250 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:46.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:46.976+0000 7f4456f33700 1 --2- 172.21.15.100:0/3565773568 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4450108410 0x7f44501087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:46.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:46.976+0000 7f4456f33700 1 --2- 172.21.15.100:0/3565773568 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44501023a0 0x7f4450102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:46.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:46.976+0000 7f4456f33700 1 -- 172.21.15.100:0/3565773568 >> 172.21.15.100:0/3565773568 conn(0x7f4450079350 msgr2=0x7f4450079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:46.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:46.976+0000 7f4456f33700 1 -- 172.21.15.100:0/3565773568 shutdown_connections 2024-05-10T11:29:46.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:46.976+0000 7f4456f33700 1 -- 172.21.15.100:0/3565773568 wait complete. 2024-05-10T11:29:46.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:46.977+0000 7f4456f33700 1 Processor -- start 2024-05-10T11:29:46.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:46.977+0000 7f4456f33700 1 -- start start 2024-05-10T11:29:46.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:46.977+0000 7f4456f33700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44501023a0 0x7f4450104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:46.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:46.977+0000 7f4456f33700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4450102d60 0x7f44501045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:46.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:46.977+0000 7f4456f33700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4450108410 0x7f4450104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:46.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:46.977+0000 7f4456f33700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4450112400 con 0x7f44501023a0 2024-05-10T11:29:46.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:46.977+0000 7f4456f33700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4450112280 con 0x7f4450102d60 2024-05-10T11:29:46.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:46.977+0000 7f4456f33700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4450112580 con 0x7f4450108410 2024-05-10T11:29:46.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:46.977+0000 7f4454ccf700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44501023a0 0x7f4450104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:46.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:46.977+0000 7f444ffff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4450102d60 0x7f44501045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:46.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:46.977+0000 7f4454ccf700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44501023a0 0x7f4450104080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44258/0 (socket says 172.21.15.100:44258) 2024-05-10T11:29:46.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:46.977+0000 7f4454ccf700 1 -- 172.21.15.100:0/1408554631 learned_addr learned my addr 172.21.15.100:0/1408554631 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:29:46.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:46.977+0000 7f44554d0700 1 --2- 172.21.15.100:0/1408554631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4450108410 0x7f4450104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:46.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:46.977+0000 7f4454ccf700 1 -- 172.21.15.100:0/1408554631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4450108410 msgr2=0x7f4450104b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:46.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:46.977+0000 7f4454ccf700 1 --2- 172.21.15.100:0/1408554631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4450108410 0x7f4450104b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:46.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:46.977+0000 7f4454ccf700 1 -- 172.21.15.100:0/1408554631 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4450102d60 msgr2=0x7f44501045c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:46.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:46.977+0000 7f4454ccf700 1 --2- 172.21.15.100:0/1408554631 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4450102d60 0x7f44501045c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:46.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:46.977+0000 7f4454ccf700 1 -- 172.21.15.100:0/1408554631 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44501a7f60 con 0x7f44501023a0 2024-05-10T11:29:46.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:46.977+0000 7f444ffff700 1 --2- 172.21.15.100:0/1408554631 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4450102d60 0x7f44501045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:29:46.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:46.978+0000 7f4454ccf700 1 --2- 172.21.15.100:0/1408554631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44501023a0 0x7f4450104080 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f444400e9c0 tx=0x7f444400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:46.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:46.978+0000 7f444dffb700 1 -- 172.21.15.100:0/1408554631 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f444400cbd0 con 0x7f44501023a0 2024-05-10T11:29:46.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:46.978+0000 7f444dffb700 1 -- 172.21.15.100:0/1408554631 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f444400ee60 con 0x7f44501023a0 2024-05-10T11:29:46.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:46.978+0000 7f4456f33700 1 -- 172.21.15.100:0/1408554631 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44501a81f0 con 0x7f44501023a0 2024-05-10T11:29:46.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:46.978+0000 7f444dffb700 1 -- 172.21.15.100:0/1408554631 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44440054b0 con 0x7f44501023a0 2024-05-10T11:29:46.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:46.978+0000 7f4456f33700 1 -- 172.21.15.100:0/1408554631 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44501a86e0 con 0x7f44501023a0 2024-05-10T11:29:46.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:46.979+0000 7f4456f33700 1 -- 172.21.15.100:0/1408554631 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f445006a190 con 0x7f44501023a0 2024-05-10T11:29:46.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:46.980+0000 7f444dffb700 1 -- 172.21.15.100:0/1408554631 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f4444004750 con 0x7f44501023a0 2024-05-10T11:29:46.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:46.981+0000 7f444dffb700 1 --2- 172.21.15.100:0/1408554631 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f44400b2880 0x7f44400b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:46.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:46.981+0000 7f444ffff700 1 --2- 172.21.15.100:0/1408554631 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f44400b2880 0x7f44400b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:46.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:46.982+0000 7f444dffb700 1 -- 172.21.15.100:0/1408554631 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f44440e27b0 con 0x7f44501023a0 2024-05-10T11:29:46.983 INFO:tasks.workunit.client.0.smithi100.stdout: 1 296763 2.71 MB/sec execute 461 sec latency 16.240 ms 2024-05-10T11:29:46.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:46.982+0000 7f444ffff700 1 --2- 172.21.15.100:0/1408554631 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f44400b2880 0x7f44400b4d40 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f4450079a00 tx=0x7f443c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:46.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:46.987+0000 7f444dffb700 1 -- 172.21.15.100:0/1408554631 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f44440aa170 con 0x7f44501023a0 2024-05-10T11:29:47.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:47.290+0000 7f4456f33700 1 -- 172.21.15.100:0/1408554631 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f44501a89a0 con 0x7f44501023a0 2024-05-10T11:29:47.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:47.291+0000 7f444dffb700 1 -- 172.21.15.100:0/1408554631 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11897 (secure 0 0 0) 0x7f44440a9a70 con 0x7f44501023a0 2024-05-10T11:29:47.291 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:29:47.291 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2072225012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2072225012},{"type":"v1","addr":"172.21.15.100:6835","nonce":2072225012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/401939540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":401939540},{"type":"v1","addr":"172.21.15.100:6837","nonce":401939540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14571,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/3946311025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3946311025},{"type":"v1","addr":"172.21.15.100:6839","nonce":3946311025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3736371660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3736371660},{"type":"v1","addr":"172.21.15.100:6841","nonce":3736371660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/2741810791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2741810791},{"type":"v1","addr":"172.21.15.178:6833","nonce":2741810791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1704891041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1704891041},{"type":"v1","addr":"172.21.15.188:6833","nonce":1704891041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T11:14:27.242452+0000","modified":"2024-05-10T11:29:07.529595+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24439,"mds_1":24437,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/2124076312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2124076312},{"type":"v1","addr":"172.21.15.178:6835","nonce":2124076312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/3224358425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3224358425},{"type":"v1","addr":"172.21.15.178:6837","nonce":3224358425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/1943342620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1943342620},{"type":"v1","addr":"172.21.15.188:6837","nonce":1943342620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.178:6839/3760985404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3760985404},{"type":"v1","addr":"172.21.15.178:6839","nonce":3760985404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.188:6839/3194917070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":3194917070},{"type":"v1","addr":"172.21.15.188:6839","nonce":3194917070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27034":{"gid":27034,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/3139559450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3139559450},{"type":"v1","addr":"172.21.15.188:6835","nonce":3139559450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24439,"qdb_cluster":[24427,24437,24439]},"id":1}]} 2024-05-10T11:29:47.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:47.294+0000 7f4456f33700 1 -- 172.21.15.100:0/1408554631 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f44400b2880 msgr2=0x7f44400b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:47.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:47.294+0000 7f4456f33700 1 --2- 172.21.15.100:0/1408554631 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f44400b2880 0x7f44400b4d40 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f4450079a00 tx=0x7f443c009260 comp rx=0 tx=0).stop 2024-05-10T11:29:47.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:47.294+0000 7f4456f33700 1 -- 172.21.15.100:0/1408554631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44501023a0 msgr2=0x7f4450104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:47.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:47.294+0000 7f4456f33700 1 --2- 172.21.15.100:0/1408554631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44501023a0 0x7f4450104080 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f444400e9c0 tx=0x7f444400ecd0 comp rx=0 tx=0).stop 2024-05-10T11:29:47.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:47.294+0000 7f4456f33700 1 -- 172.21.15.100:0/1408554631 shutdown_connections 2024-05-10T11:29:47.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:47.294+0000 7f4456f33700 1 --2- 172.21.15.100:0/1408554631 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f44400b2880 0x7f44400b4d40 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:47.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:47.294+0000 7f4456f33700 1 --2- 172.21.15.100:0/1408554631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4450108410 0x7f4450104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:47.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:47.294+0000 7f4456f33700 1 --2- 172.21.15.100:0/1408554631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44501023a0 0x7f4450104080 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:47.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:47.294+0000 7f4456f33700 1 --2- 172.21.15.100:0/1408554631 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4450102d60 0x7f44501045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:47.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:47.294+0000 7f4456f33700 1 -- 172.21.15.100:0/1408554631 >> 172.21.15.100:0/1408554631 conn(0x7f4450079350 msgr2=0x7f4450101690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:47.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:47.294+0000 7f4456f33700 1 -- 172.21.15.100:0/1408554631 shutdown_connections 2024-05-10T11:29:47.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:47.294+0000 7f4456f33700 1 -- 172.21.15.100:0/1408554631 wait complete. 2024-05-10T11:29:47.296 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 33 2024-05-10T11:29:47.981 INFO:tasks.workunit.client.0.smithi100.stdout: 1 297329 2.72 MB/sec execute 462 sec latency 52.830 ms 2024-05-10T11:29:48.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:29:47 smithi178 ceph-mon[33359]: pgmap v763: 129 pgs: 129 active+clean; 118 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 1.3 MiB/s wr, 155 op/s 2024-05-10T11:29:48.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:29:47 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/1408554631' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:29:48.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:29:47 smithi188 ceph-mon[34075]: pgmap v763: 129 pgs: 129 active+clean; 118 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 1.3 MiB/s wr, 155 op/s 2024-05-10T11:29:48.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:29:47 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/1408554631' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:29:48.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:29:47 smithi100 ceph-mon[30296]: pgmap v763: 129 pgs: 129 active+clean; 118 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 1.3 MiB/s wr, 155 op/s 2024-05-10T11:29:48.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:29:47 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/1408554631' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:29:48.439 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-10T11:29:48.703 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:29:48.982 INFO:tasks.workunit.client.0.smithi100.stdout: 1 297800 2.72 MB/sec execute 463 sec latency 15.876 ms 2024-05-10T11:29:49.982 INFO:tasks.workunit.client.0.smithi100.stdout: 1 298273 2.72 MB/sec execute 464 sec latency 15.284 ms 2024-05-10T11:29:50.210 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:29:49 smithi100 ceph-mon[30296]: pgmap v764: 129 pgs: 129 active+clean; 118 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 1.2 MiB/s wr, 138 op/s 2024-05-10T11:29:50.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:29:49 smithi178 ceph-mon[33359]: pgmap v764: 129 pgs: 129 active+clean; 118 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 1.2 MiB/s wr, 138 op/s 2024-05-10T11:29:50.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:29:49 smithi188 ceph-mon[34075]: pgmap v764: 129 pgs: 129 active+clean; 118 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 1.2 MiB/s wr, 138 op/s 2024-05-10T11:29:50.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.467+0000 7f00008f4700 1 -- 172.21.15.100:0/1818470187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efffc1037b0 msgr2=0x7efffc103b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:50.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.467+0000 7f00008f4700 1 --2- 172.21.15.100:0/1818470187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efffc1037b0 0x7efffc103b90 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7effec0099c0 tx=0x7effec009cd0 comp rx=0 tx=0).stop 2024-05-10T11:29:50.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.467+0000 7f00008f4700 1 -- 172.21.15.100:0/1818470187 shutdown_connections 2024-05-10T11:29:50.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.467+0000 7f00008f4700 1 --2- 172.21.15.100:0/1818470187 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efffc108880 0x7efffc110400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:50.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.467+0000 7f00008f4700 1 --2- 172.21.15.100:0/1818470187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efffc1037b0 0x7efffc103b90 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:50.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.467+0000 7f00008f4700 1 --2- 172.21.15.100:0/1818470187 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efffc104160 0x7efffc108150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:50.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.467+0000 7f00008f4700 1 -- 172.21.15.100:0/1818470187 >> 172.21.15.100:0/1818470187 conn(0x7efffc07a530 msgr2=0x7efffc07a940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:50.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.467+0000 7f00008f4700 1 -- 172.21.15.100:0/1818470187 shutdown_connections 2024-05-10T11:29:50.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.467+0000 7f00008f4700 1 -- 172.21.15.100:0/1818470187 wait complete. 2024-05-10T11:29:50.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.468+0000 7f00008f4700 1 Processor -- start 2024-05-10T11:29:50.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.468+0000 7f00008f4700 1 -- start start 2024-05-10T11:29:50.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.468+0000 7f00008f4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efffc1037b0 0x7efffc19f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:50.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.468+0000 7f00008f4700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efffc104160 0x7efffc19fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:50.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.468+0000 7f00008f4700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efffc108880 0x7efffc199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:50.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.468+0000 7f00008f4700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7efffc113520 con 0x7efffc1037b0 2024-05-10T11:29:50.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.468+0000 7f00008f4700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efffc1133a0 con 0x7efffc104160 2024-05-10T11:29:50.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.468+0000 7f00008f4700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7efffc1136a0 con 0x7efffc108880 2024-05-10T11:29:50.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.468+0000 7efffa59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efffc1037b0 0x7efffc19f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:50.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.468+0000 7efffa59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efffc1037b0 0x7efffc19f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44282/0 (socket says 172.21.15.100:44282) 2024-05-10T11:29:50.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.469+0000 7efffa59c700 1 -- 172.21.15.100:0/4282391859 learned_addr learned my addr 172.21.15.100:0/4282391859 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:29:50.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.469+0000 7efffad9d700 1 --2- 172.21.15.100:0/4282391859 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efffc108880 0x7efffc199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:50.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.469+0000 7efff9d9b700 1 --2- 172.21.15.100:0/4282391859 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efffc104160 0x7efffc19fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:50.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.469+0000 7efffa59c700 1 -- 172.21.15.100:0/4282391859 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efffc108880 msgr2=0x7efffc199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:50.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.469+0000 7efffa59c700 1 --2- 172.21.15.100:0/4282391859 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efffc108880 0x7efffc199840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:50.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.469+0000 7efffa59c700 1 -- 172.21.15.100:0/4282391859 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efffc104160 msgr2=0x7efffc19fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:50.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.469+0000 7efffa59c700 1 --2- 172.21.15.100:0/4282391859 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efffc104160 0x7efffc19fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:50.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.469+0000 7efffa59c700 1 -- 172.21.15.100:0/4282391859 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efffc19a020 con 0x7efffc1037b0 2024-05-10T11:29:50.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.469+0000 7efff9d9b700 1 --2- 172.21.15.100:0/4282391859 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efffc104160 0x7efffc19fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:29:50.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.469+0000 7efffa59c700 1 --2- 172.21.15.100:0/4282391859 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efffc1037b0 0x7efffc19f6c0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7effec006010 tx=0x7effec0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:50.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.469+0000 7effeb7fe700 1 -- 172.21.15.100:0/4282391859 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effec026070 con 0x7efffc1037b0 2024-05-10T11:29:50.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.469+0000 7effeb7fe700 1 -- 172.21.15.100:0/4282391859 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7effec00f460 con 0x7efffc1037b0 2024-05-10T11:29:50.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.469+0000 7f00008f4700 1 -- 172.21.15.100:0/4282391859 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efffc19a2b0 con 0x7efffc1037b0 2024-05-10T11:29:50.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.469+0000 7effeb7fe700 1 -- 172.21.15.100:0/4282391859 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effec009e60 con 0x7efffc1037b0 2024-05-10T11:29:50.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.470+0000 7f00008f4700 1 -- 172.21.15.100:0/4282391859 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efffc1a6520 con 0x7efffc1037b0 2024-05-10T11:29:50.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.472+0000 7effeb7fe700 1 -- 172.21.15.100:0/4282391859 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7effec029070 con 0x7efffc1037b0 2024-05-10T11:29:50.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.473+0000 7effeb7fe700 1 --2- 172.21.15.100:0/4282391859 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7effe40b2880 0x7effe40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:50.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.473+0000 7efff9d9b700 1 --2- 172.21.15.100:0/4282391859 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7effe40b2880 0x7effe40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:50.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.473+0000 7effeb7fe700 1 -- 172.21.15.100:0/4282391859 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7effec0e2dd0 con 0x7efffc1037b0 2024-05-10T11:29:50.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.474+0000 7efff9d9b700 1 --2- 172.21.15.100:0/4282391859 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7effe40b2880 0x7effe40b4d40 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7efff0005fd0 tx=0x7efff0005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:50.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.474+0000 7f000b187b80 1 Processor -- start 2024-05-10T11:29:50.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.474+0000 7f000b187b80 1 -- start start 2024-05-10T11:29:50.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.474+0000 7f000b187b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56116d1d0f80 0x56116d1d1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:50.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.474+0000 7f000b187b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56116d1d18a0 0x56116d1cb700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:50.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.474+0000 7f000b187b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56116d1cbc90 0x56116d1cc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:50.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.474+0000 7f000b187b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56116d119860 con 0x56116d1d0f80 2024-05-10T11:29:50.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.474+0000 7f000b187b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56116d0b87c0 con 0x56116d1d18a0 2024-05-10T11:29:50.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.474+0000 7f000b187b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56116d115820 con 0x56116d1cbc90 2024-05-10T11:29:50.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.474+0000 7efffa59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56116d1d0f80 0x56116d1d1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:50.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.474+0000 7efffa59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56116d1d0f80 0x56116d1d1360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44292/0 (socket says 172.21.15.100:44292) 2024-05-10T11:29:50.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.474+0000 7efffa59c700 1 -- 172.21.15.100:0/1633994520 learned_addr learned my addr 172.21.15.100:0/1633994520 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:29:50.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.474+0000 7efff9d9b700 1 --2- 172.21.15.100:0/1633994520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56116d1d18a0 0x56116d1cb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:50.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.474+0000 7efffad9d700 1 --2- 172.21.15.100:0/1633994520 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56116d1cbc90 0x56116d1cc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:50.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.475+0000 7efffa59c700 1 -- 172.21.15.100:0/1633994520 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56116d1cbc90 msgr2=0x56116d1cc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:50.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.475+0000 7efffa59c700 1 --2- 172.21.15.100:0/1633994520 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56116d1cbc90 0x56116d1cc110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:50.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.475+0000 7efffa59c700 1 -- 172.21.15.100:0/1633994520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56116d1d18a0 msgr2=0x56116d1cb700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:50.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.475+0000 7efffa59c700 1 --2- 172.21.15.100:0/1633994520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56116d1d18a0 0x56116d1cb700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:50.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.475+0000 7efffa59c700 1 -- 172.21.15.100:0/1633994520 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56116d0668d0 con 0x56116d1d0f80 2024-05-10T11:29:50.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.475+0000 7efff9d9b700 1 --2- 172.21.15.100:0/1633994520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56116d1d18a0 0x56116d1cb700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:29:50.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.475+0000 7efffa59c700 1 --2- 172.21.15.100:0/1633994520 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56116d1d0f80 0x56116d1d1360 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7effec0a9e20 tx=0x7effec0a9890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=579434b2a7d0352d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:50.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.475+0000 7effe8ff9700 1 -- 172.21.15.100:0/1633994520 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effec0aa820 con 0x56116d1d0f80 2024-05-10T11:29:50.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.475+0000 7effe8ff9700 1 -- 172.21.15.100:0/1633994520 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7effec0aa120 con 0x56116d1d0f80 2024-05-10T11:29:50.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.475+0000 7effe8ff9700 1 -- 172.21.15.100:0/1633994520 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effec0aa5e0 con 0x56116d1d0f80 2024-05-10T11:29:50.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.476+0000 7f000b187b80 1 -- 172.21.15.100:0/1633994520 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56116d1d0f80 msgr2=0x56116d1d1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:50.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.476+0000 7f000b187b80 1 --2- 172.21.15.100:0/1633994520 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56116d1d0f80 0x56116d1d1360 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7effec0a9e20 tx=0x7effec0a9890 comp rx=0 tx=0).stop 2024-05-10T11:29:50.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.476+0000 7f000b187b80 1 -- 172.21.15.100:0/1633994520 shutdown_connections 2024-05-10T11:29:50.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.476+0000 7f000b187b80 1 --2- 172.21.15.100:0/1633994520 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56116d1cbc90 0x56116d1cc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:50.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.476+0000 7f000b187b80 1 --2- 172.21.15.100:0/1633994520 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56116d1d0f80 0x56116d1d1360 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:50.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.476+0000 7f000b187b80 1 --2- 172.21.15.100:0/1633994520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56116d1d18a0 0x56116d1cb700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:50.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.476+0000 7f000b187b80 1 -- 172.21.15.100:0/1633994520 >> 172.21.15.100:0/1633994520 conn(0x56116d0f3100 msgr2=0x56116d1c1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:50.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.476+0000 7f000b187b80 1 -- 172.21.15.100:0/1633994520 shutdown_connections 2024-05-10T11:29:50.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.476+0000 7f000b187b80 1 -- 172.21.15.100:0/1633994520 wait complete. 2024-05-10T11:29:50.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.476+0000 7f000b187b80 1 -- start start 2024-05-10T11:29:50.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.476+0000 7f000b187b80 1 Processor -- start 2024-05-10T11:29:50.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.477+0000 7f000b187b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56116d1cbc90 0x56116d1d7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:50.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.477+0000 7f000b187b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56116d1d0f80 0x56116d1d4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:50.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.477+0000 7f000b187b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56116d1d18a0 0x56116d1d4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:50.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.477+0000 7f000b187b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56116d1d8080 con 0x56116d1cbc90 2024-05-10T11:29:50.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.477+0000 7f000b187b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56116d07fc30 con 0x56116d1d0f80 2024-05-10T11:29:50.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.477+0000 7f000b187b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56116d1d8200 con 0x56116d1d18a0 2024-05-10T11:29:50.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.477+0000 7f000b187b80 10 client.0 _ll_register_callbacks cb 0x56116d0b1920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:29:50.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.477+0000 7efffa59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56116d1cbc90 0x56116d1d7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:50.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.477+0000 7efffa59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56116d1cbc90 0x56116d1d7520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44308/0 (socket says 172.21.15.100:44308) 2024-05-10T11:29:50.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.477+0000 7efffa59c700 1 -- 172.21.15.100:0/1087880601 learned_addr learned my addr 172.21.15.100:0/1087880601 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:29:50.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.477+0000 7efff9d9b700 1 --2- 172.21.15.100:0/1087880601 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56116d1d0f80 0x56116d1d4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:50.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.477+0000 7efffad9d700 1 --2- 172.21.15.100:0/1087880601 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56116d1d18a0 0x56116d1d4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:50.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.477+0000 7efffa59c700 1 -- 172.21.15.100:0/1087880601 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56116d1d18a0 msgr2=0x56116d1d4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:50.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.477+0000 7efffa59c700 1 --2- 172.21.15.100:0/1087880601 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56116d1d18a0 0x56116d1d4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:50.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.477+0000 7efffa59c700 1 -- 172.21.15.100:0/1087880601 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56116d1d0f80 msgr2=0x56116d1d4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:50.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.477+0000 7efffa59c700 1 --2- 172.21.15.100:0/1087880601 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56116d1d0f80 0x56116d1d4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:50.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.477+0000 7efffa59c700 1 -- 172.21.15.100:0/1087880601 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56116d0b5690 con 0x56116d1cbc90 2024-05-10T11:29:50.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.477+0000 7efff9d9b700 1 --2- 172.21.15.100:0/1087880601 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56116d1d0f80 0x56116d1d4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:29:50.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.477+0000 7efffa59c700 1 --2- 172.21.15.100:0/1087880601 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56116d1cbc90 0x56116d1d7520 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7effec0e08e0 tx=0x7effec0ae560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c166b91d7878f23 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:50.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.477+0000 7effe1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:29:50.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.477+0000 7effe1ffb700 1 -- 172.21.15.100:0/1087880601 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effec0c3020 con 0x56116d1cbc90 2024-05-10T11:29:50.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.477+0000 7effe1ffb700 1 -- 172.21.15.100:0/1087880601 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7effec0aa380 con 0x56116d1cbc90 2024-05-10T11:29:50.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.477+0000 7f000b187b80 1 -- 172.21.15.100:0/1087880601 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effec0aa120 con 0x56116d1cbc90 2024-05-10T11:29:50.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.478+0000 7effe1ffb700 1 -- 172.21.15.100:0/1087880601 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7effec0aa820 con 0x56116d1cbc90 2024-05-10T11:29:50.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.478+0000 7f000b187b80 10 client.17709 fetch_fsmap learned FSMap version 33 2024-05-10T11:29:50.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.478+0000 7f000b187b80 1 -- 172.21.15.100:0/1087880601 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56116d1d8080 con 0x56116d1cbc90 2024-05-10T11:29:50.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.478+0000 7effe1ffb700 1 -- 172.21.15.100:0/1087880601 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7effec0aa120 con 0x56116d1cbc90 2024-05-10T11:29:50.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.478+0000 7effe1ffb700 1 -- 172.21.15.100:0/1087880601 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7effec069c90 con 0x56116d1cbc90 2024-05-10T11:29:50.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.478+0000 7f000b187b80 10 client.17709 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T11:29:50.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.478+0000 7f000b187b80 10 client.17709 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-10T11:29:50.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.478+0000 7f000b187b80 20 client.17709 populate_metadata read hostname 'smithi100' 2024-05-10T11:29:50.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.478+0000 7f000b187b80 1 --2- 172.21.15.100:0/1087880601 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x56116d1e4c10 0x56116d1e7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:50.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.478+0000 7f000b187b80 4 client.17709 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:29:50.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.478+0000 7f000b187b80 1 -- 172.21.15.100:0/1087880601 --> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56116d193650 con 0x56116d1e4c10 2024-05-10T11:29:50.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.478+0000 7efff9d9b700 1 --2- 172.21.15.100:0/1087880601 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x56116d1e4c10 0x56116d1e7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:50.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.479+0000 7effe1ffb700 10 client.17709 ms_handle_connect on v2:172.21.15.188:6836/1943342620 2024-05-10T11:29:50.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.479+0000 7efff9d9b700 1 --2- 172.21.15.100:0/1087880601 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x56116d1e4c10 0x56116d1e7040 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=10d3dce547479dc1 server_cookie=8e006bd8c3515348 in_seq=0 out_seq=0 2024-05-10T11:29:50.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.479+0000 7effe1ffb700 1 -- 172.21.15.100:0/1087880601 <== mds.2 v2:172.21.15.188:6836/1943342620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56116d193650 con 0x56116d1e4c10 2024-05-10T11:29:50.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.479+0000 7effe1ffb700 10 client.17709 handle_command_reply: tid=0 2024-05-10T11:29:50.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.479+0000 7f000b187b80 1 client.17709 shutdown 2024-05-10T11:29:50.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.479+0000 7f000b187b80 1 -- 172.21.15.100:0/1087880601 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56116d1cbc90 msgr2=0x56116d1d7520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:50.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.479+0000 7f000b187b80 1 --2- 172.21.15.100:0/1087880601 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56116d1cbc90 0x56116d1d7520 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7effec0e08e0 tx=0x7effec0ae560 comp rx=0 tx=0).stop 2024-05-10T11:29:50.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.480+0000 7f000b187b80 1 -- 172.21.15.100:0/1087880601 shutdown_connections 2024-05-10T11:29:50.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.480+0000 7f000b187b80 1 --2- 172.21.15.100:0/1087880601 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x56116d1e4c10 0x56116d1e7040 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:50.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.480+0000 7f000b187b80 1 --2- 172.21.15.100:0/1087880601 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56116d1d18a0 0x56116d1d4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:50.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.480+0000 7f000b187b80 1 --2- 172.21.15.100:0/1087880601 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56116d1cbc90 0x56116d1d7520 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:50.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.480+0000 7effe1ffb700 0 client.17709 ms_handle_reset on v2:172.21.15.188:6836/1943342620 2024-05-10T11:29:50.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.480+0000 7f000b187b80 1 --2- 172.21.15.100:0/1087880601 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56116d1d0f80 0x56116d1d4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:50.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.480+0000 7f000b187b80 1 -- 172.21.15.100:0/1087880601 >> 172.21.15.100:0/1087880601 conn(0x56116d0f3100 msgr2=0x56116d1bc720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:50.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.480+0000 7f000b187b80 1 -- 172.21.15.100:0/1087880601 shutdown_connections 2024-05-10T11:29:50.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.480+0000 7f000b187b80 1 -- 172.21.15.100:0/1087880601 wait complete. 2024-05-10T11:29:50.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.480+0000 7f000b187b80 20 client.17709 trim_cache size 0 max 16384 2024-05-10T11:29:50.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.498+0000 7f000b187b80 1 Processor -- start 2024-05-10T11:29:50.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.498+0000 7f000b187b80 1 -- start start 2024-05-10T11:29:50.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.499+0000 7f000b187b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56116d1cbc90 0x56116d1e10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:50.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.499+0000 7f000b187b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56116d1d0f80 0x56116d1d4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:50.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.499+0000 7f000b187b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56116d1d18a0 0x56116d1ec8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:50.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.499+0000 7f000b187b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7efffc1af690 con 0x56116d1d0f80 2024-05-10T11:29:50.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.499+0000 7f000b187b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efffc1acf20 con 0x56116d1d18a0 2024-05-10T11:29:50.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.499+0000 7f000b187b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7efffc1aad90 con 0x56116d1cbc90 2024-05-10T11:29:50.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.499+0000 7efff9d9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56116d1d0f80 0x56116d1d4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:50.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.499+0000 7efff9d9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56116d1d0f80 0x56116d1d4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44316/0 (socket says 172.21.15.100:44316) 2024-05-10T11:29:50.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.499+0000 7efff9d9b700 1 -- 172.21.15.100:0/3686526860 learned_addr learned my addr 172.21.15.100:0/3686526860 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:29:50.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.499+0000 7efffa59c700 1 --2- 172.21.15.100:0/3686526860 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56116d1cbc90 0x56116d1e10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:50.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.499+0000 7efffad9d700 1 --2- 172.21.15.100:0/3686526860 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56116d1d18a0 0x56116d1ec8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:50.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.499+0000 7efff9d9b700 1 -- 172.21.15.100:0/3686526860 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56116d1cbc90 msgr2=0x56116d1e10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:50.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.499+0000 7efff9d9b700 1 --2- 172.21.15.100:0/3686526860 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56116d1cbc90 0x56116d1e10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:50.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.499+0000 7efff9d9b700 1 -- 172.21.15.100:0/3686526860 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56116d1d18a0 msgr2=0x56116d1ec8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:50.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.499+0000 7efff9d9b700 1 --2- 172.21.15.100:0/3686526860 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56116d1d18a0 0x56116d1ec8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:50.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.499+0000 7efff9d9b700 1 -- 172.21.15.100:0/3686526860 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56116d1be400 con 0x56116d1d0f80 2024-05-10T11:29:50.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.499+0000 7efff9d9b700 1 --2- 172.21.15.100:0/3686526860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56116d1d0f80 0x56116d1d4ec0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7efff0009410 tx=0x7efff0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8b3aeacb6568431 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:50.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.500+0000 7effe3fff700 1 -- 172.21.15.100:0/3686526860 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff0004020 con 0x56116d1d0f80 2024-05-10T11:29:50.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.500+0000 7effe3fff700 1 -- 172.21.15.100:0/3686526860 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efff0017050 con 0x56116d1d0f80 2024-05-10T11:29:50.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.500+0000 7effe3fff700 1 -- 172.21.15.100:0/3686526860 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff0004e60 con 0x56116d1d0f80 2024-05-10T11:29:50.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.500+0000 7f000b187b80 1 -- 172.21.15.100:0/3686526860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56116d1d0f80 msgr2=0x56116d1d4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:50.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.500+0000 7f000b187b80 1 --2- 172.21.15.100:0/3686526860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56116d1d0f80 0x56116d1d4ec0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7efff0009410 tx=0x7efff0014000 comp rx=0 tx=0).stop 2024-05-10T11:29:50.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.500+0000 7f000b187b80 1 -- 172.21.15.100:0/3686526860 shutdown_connections 2024-05-10T11:29:50.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.500+0000 7f000b187b80 1 --2- 172.21.15.100:0/3686526860 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56116d1d18a0 0x56116d1ec8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:50.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.500+0000 7f000b187b80 1 --2- 172.21.15.100:0/3686526860 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56116d1cbc90 0x56116d1e10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:50.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.500+0000 7f000b187b80 1 --2- 172.21.15.100:0/3686526860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56116d1d0f80 0x56116d1d4ec0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:50.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.500+0000 7f000b187b80 1 -- 172.21.15.100:0/3686526860 >> 172.21.15.100:0/3686526860 conn(0x56116d0f3100 msgr2=0x56116d1d3240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:50.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.501+0000 7f000b187b80 1 -- 172.21.15.100:0/3686526860 shutdown_connections 2024-05-10T11:29:50.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.501+0000 7f000b187b80 1 -- 172.21.15.100:0/3686526860 wait complete. 2024-05-10T11:29:50.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.501+0000 7f000b187b80 1 -- start start 2024-05-10T11:29:50.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.501+0000 7f000b187b80 1 Processor -- start 2024-05-10T11:29:50.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.501+0000 7f000b187b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56116d1cbc90 0x56116d1dd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:50.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.501+0000 7f000b187b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56116d1d0f80 0x56116d1dd990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:50.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.501+0000 7f000b187b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56116d1d18a0 0x56116d1d5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:50.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.501+0000 7f000b187b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7efffc1ab3e0 con 0x56116d1d18a0 2024-05-10T11:29:50.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.501+0000 7f000b187b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efffc1ac850 con 0x56116d1d0f80 2024-05-10T11:29:50.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.501+0000 7f000b187b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7efffc1abab0 con 0x56116d1cbc90 2024-05-10T11:29:50.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.502+0000 7f000b187b80 10 client.0 _ll_register_callbacks cb 0x56116d0b1920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:29:50.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.502+0000 7efffad9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56116d1d18a0 0x56116d1d5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:50.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.502+0000 7efffad9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56116d1d18a0 0x56116d1d5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44324/0 (socket says 172.21.15.100:44324) 2024-05-10T11:29:50.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.502+0000 7efffad9d700 1 -- 172.21.15.100:0/1736289626 learned_addr learned my addr 172.21.15.100:0/1736289626 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:29:50.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.502+0000 7efffa59c700 1 --2- 172.21.15.100:0/1736289626 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56116d1cbc90 0x56116d1dd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:50.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.502+0000 7efff9d9b700 1 --2- 172.21.15.100:0/1736289626 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56116d1d0f80 0x56116d1dd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:50.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.502+0000 7efffad9d700 1 -- 172.21.15.100:0/1736289626 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56116d1cbc90 msgr2=0x56116d1dd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:50.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.502+0000 7efffad9d700 1 --2- 172.21.15.100:0/1736289626 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56116d1cbc90 0x56116d1dd450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:50.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.502+0000 7efffad9d700 1 -- 172.21.15.100:0/1736289626 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56116d1d0f80 msgr2=0x56116d1dd990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:50.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.502+0000 7efffad9d700 1 --2- 172.21.15.100:0/1736289626 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56116d1d0f80 0x56116d1dd990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:50.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.502+0000 7efffad9d700 1 -- 172.21.15.100:0/1736289626 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56116d1d6490 con 0x56116d1d18a0 2024-05-10T11:29:50.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.502+0000 7efffa59c700 1 --2- 172.21.15.100:0/1736289626 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56116d1cbc90 0x56116d1dd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:29:50.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.502+0000 7efffad9d700 1 --2- 172.21.15.100:0/1736289626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56116d1d18a0 0x56116d1d5d90 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7efff4007f70 tx=0x7efff40124e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68ea7f9a53166460 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:50.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.502+0000 7effe1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:29:50.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.502+0000 7effe1ffb700 1 -- 172.21.15.100:0/1736289626 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff4012cc0 con 0x56116d1d18a0 2024-05-10T11:29:50.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.502+0000 7effe1ffb700 1 -- 172.21.15.100:0/1736289626 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7efff4004bb0 con 0x56116d1d18a0 2024-05-10T11:29:50.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.502+0000 7f000b187b80 1 -- 172.21.15.100:0/1736289626 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efff0017050 con 0x56116d1d18a0 2024-05-10T11:29:50.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.503+0000 7effe1ffb700 1 -- 172.21.15.100:0/1736289626 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7efff4012e60 con 0x56116d1d18a0 2024-05-10T11:29:50.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.503+0000 7f000b187b80 10 client.17715 fetch_fsmap learned FSMap version 33 2024-05-10T11:29:50.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.503+0000 7f000b187b80 1 -- 172.21.15.100:0/1736289626 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56116d24ee90 con 0x56116d1d18a0 2024-05-10T11:29:50.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.503+0000 7effe1ffb700 1 -- 172.21.15.100:0/1736289626 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efff0017050 con 0x56116d1d18a0 2024-05-10T11:29:50.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.503+0000 7effe1ffb700 1 -- 172.21.15.100:0/1736289626 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7efff4016520 con 0x56116d1d18a0 2024-05-10T11:29:50.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.503+0000 7f000b187b80 10 client.17715 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T11:29:50.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.503+0000 7f000b187b80 10 client.17715 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-10T11:29:50.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.503+0000 7f000b187b80 20 client.17715 populate_metadata read hostname 'smithi100' 2024-05-10T11:29:50.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.503+0000 7f000b187b80 1 --2- 172.21.15.100:0/1736289626 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x56116d24f620 0x56116d251a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:50.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.503+0000 7f000b187b80 4 client.17715 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:29:50.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.503+0000 7f000b187b80 1 -- 172.21.15.100:0/1736289626 --> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56116d127240 con 0x56116d24f620 2024-05-10T11:29:50.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.503+0000 7efffa59c700 1 --2- 172.21.15.100:0/1736289626 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x56116d24f620 0x56116d251a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:50.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.504+0000 7effe1ffb700 10 client.17715 ms_handle_connect on v2:172.21.15.188:6836/1943342620 2024-05-10T11:29:50.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.504+0000 7efffa59c700 1 --2- 172.21.15.100:0/1736289626 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x56116d24f620 0x56116d251a40 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6fb9581e23ac8d84 server_cookie=2c6352bc280ccf2c in_seq=0 out_seq=0 2024-05-10T11:29:50.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.504+0000 7effe1ffb700 1 -- 172.21.15.100:0/1736289626 <== mds.2 v2:172.21.15.188:6836/1943342620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56116d127240 con 0x56116d24f620 2024-05-10T11:29:50.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.504+0000 7effe1ffb700 10 client.17715 handle_command_reply: tid=0 2024-05-10T11:29:50.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.504+0000 7f000b187b80 1 client.17715 shutdown 2024-05-10T11:29:50.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.504+0000 7f000b187b80 1 -- 172.21.15.100:0/1736289626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56116d1d18a0 msgr2=0x56116d1d5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:50.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.504+0000 7f000b187b80 1 --2- 172.21.15.100:0/1736289626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56116d1d18a0 0x56116d1d5d90 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7efff4007f70 tx=0x7efff40124e0 comp rx=0 tx=0).stop 2024-05-10T11:29:50.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.504+0000 7f000b187b80 1 -- 172.21.15.100:0/1736289626 shutdown_connections 2024-05-10T11:29:50.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.504+0000 7f000b187b80 1 --2- 172.21.15.100:0/1736289626 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x56116d24f620 0x56116d251a40 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:50.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.504+0000 7f000b187b80 1 --2- 172.21.15.100:0/1736289626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56116d1d18a0 0x56116d1d5d90 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:50.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.504+0000 7f000b187b80 1 --2- 172.21.15.100:0/1736289626 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56116d1cbc90 0x56116d1dd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:50.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.504+0000 7effe1ffb700 0 client.17715 ms_handle_reset on v2:172.21.15.188:6836/1943342620 2024-05-10T11:29:50.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.504+0000 7f000b187b80 1 --2- 172.21.15.100:0/1736289626 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56116d1d0f80 0x56116d1dd990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:50.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.504+0000 7f000b187b80 1 -- 172.21.15.100:0/1736289626 >> 172.21.15.100:0/1736289626 conn(0x56116d0f3100 msgr2=0x56116d1bbe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:50.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.504+0000 7f000b187b80 1 -- 172.21.15.100:0/1736289626 shutdown_connections 2024-05-10T11:29:50.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.504+0000 7f000b187b80 1 -- 172.21.15.100:0/1736289626 wait complete. 2024-05-10T11:29:50.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.504+0000 7f000b187b80 20 client.17715 trim_cache size 0 max 16384 2024-05-10T11:29:50.509 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:29:50.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.505+0000 7f00008f4700 1 -- 172.21.15.100:0/4282391859 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7effe40b2880 msgr2=0x7effe40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:50.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.505+0000 7f00008f4700 1 --2- 172.21.15.100:0/4282391859 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7effe40b2880 0x7effe40b4d40 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7efff0005fd0 tx=0x7efff0005ee0 comp rx=0 tx=0).stop 2024-05-10T11:29:50.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.505+0000 7f00008f4700 1 -- 172.21.15.100:0/4282391859 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efffc1037b0 msgr2=0x7efffc19f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:50.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.505+0000 7f00008f4700 1 --2- 172.21.15.100:0/4282391859 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efffc1037b0 0x7efffc19f6c0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7effec006010 tx=0x7effec0045e0 comp rx=0 tx=0).stop 2024-05-10T11:29:50.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.505+0000 7f00008f4700 1 -- 172.21.15.100:0/4282391859 shutdown_connections 2024-05-10T11:29:50.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.505+0000 7f00008f4700 1 --2- 172.21.15.100:0/4282391859 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7effe40b2880 0x7effe40b4d40 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:50.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.505+0000 7f00008f4700 1 --2- 172.21.15.100:0/4282391859 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efffc108880 0x7efffc199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:50.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.505+0000 7f00008f4700 1 --2- 172.21.15.100:0/4282391859 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efffc1037b0 0x7efffc19f6c0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:50.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.505+0000 7f00008f4700 1 --2- 172.21.15.100:0/4282391859 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efffc104160 0x7efffc19fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:50.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.505+0000 7f00008f4700 1 -- 172.21.15.100:0/4282391859 >> 172.21.15.100:0/4282391859 conn(0x7efffc07a530 msgr2=0x7efffc101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:50.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.505+0000 7f00008f4700 1 -- 172.21.15.100:0/4282391859 shutdown_connections 2024-05-10T11:29:50.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:50.505+0000 7f00008f4700 1 -- 172.21.15.100:0/4282391859 wait complete. 2024-05-10T11:29:50.982 INFO:tasks.workunit.client.0.smithi100.stdout: 1 298529 2.72 MB/sec execute 465 sec latency 22.434 ms 2024-05-10T11:29:51.236 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-10T11:29:51.507 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:29:51.959 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:29:51 smithi100 ceph-mon[30296]: pgmap v765: 129 pgs: 129 active+clean; 119 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 1.1 MiB/s wr, 148 op/s 2024-05-10T11:29:51.981 INFO:tasks.workunit.client.0.smithi100.stdout: 1 298977 2.71 MB/sec execute 466 sec latency 15.512 ms 2024-05-10T11:29:52.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:29:51 smithi178 ceph-mon[33359]: pgmap v765: 129 pgs: 129 active+clean; 119 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 1.1 MiB/s wr, 148 op/s 2024-05-10T11:29:52.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:29:51 smithi188 ceph-mon[34075]: pgmap v765: 129 pgs: 129 active+clean; 119 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 1.1 MiB/s wr, 148 op/s 2024-05-10T11:29:52.982 INFO:tasks.workunit.client.0.smithi100.stdout: 1 299506 2.72 MB/sec execute 467 sec latency 72.397 ms 2024-05-10T11:29:53.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.545+0000 7fa2612a2700 1 -- 172.21.15.100:0/4028925283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa25c108410 msgr2=0x7fa25c1087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:53.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.545+0000 7fa2612a2700 1 --2- 172.21.15.100:0/4028925283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa25c108410 0x7fa25c1087f0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fa24c009990 tx=0x7fa24c009ca0 comp rx=0 tx=0).stop 2024-05-10T11:29:53.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.545+0000 7fa2612a2700 1 -- 172.21.15.100:0/4028925283 shutdown_connections 2024-05-10T11:29:53.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.545+0000 7fa2612a2700 1 --2- 172.21.15.100:0/4028925283 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa25c102d60 0x7fa25c10f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:53.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.545+0000 7fa2612a2700 1 --2- 172.21.15.100:0/4028925283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa25c108410 0x7fa25c1087f0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:53.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.545+0000 7fa2612a2700 1 --2- 172.21.15.100:0/4028925283 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa25c1023a0 0x7fa25c102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:53.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.545+0000 7fa2612a2700 1 -- 172.21.15.100:0/4028925283 >> 172.21.15.100:0/4028925283 conn(0x7fa25c079350 msgr2=0x7fa25c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:53.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.545+0000 7fa2612a2700 1 -- 172.21.15.100:0/4028925283 shutdown_connections 2024-05-10T11:29:53.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.545+0000 7fa2612a2700 1 -- 172.21.15.100:0/4028925283 wait complete. 2024-05-10T11:29:53.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.546+0000 7fa2612a2700 1 Processor -- start 2024-05-10T11:29:53.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.546+0000 7fa2612a2700 1 -- start start 2024-05-10T11:29:53.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.546+0000 7fa2612a2700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa25c1023a0 0x7fa25c19e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:53.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.546+0000 7fa2612a2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa25c102d60 0x7fa25c19eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:53.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.546+0000 7fa2612a2700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa25c108410 0x7fa25c198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:53.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.546+0000 7fa2612a2700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa25c112410 con 0x7fa25c102d60 2024-05-10T11:29:53.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.546+0000 7fa2612a2700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa25c112290 con 0x7fa25c1023a0 2024-05-10T11:29:53.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.546+0000 7fa2612a2700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa25c112590 con 0x7fa25c108410 2024-05-10T11:29:53.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.546+0000 7fa25a7fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa25c102d60 0x7fa25c19eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:53.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.546+0000 7fa25a7fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa25c102d60 0x7fa25c19eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44340/0 (socket says 172.21.15.100:44340) 2024-05-10T11:29:53.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.546+0000 7fa25a7fc700 1 -- 172.21.15.100:0/1060140870 learned_addr learned my addr 172.21.15.100:0/1060140870 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:29:53.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.546+0000 7fa25b7fe700 1 --2- 172.21.15.100:0/1060140870 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa25c108410 0x7fa25c198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:53.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.547+0000 7fa25affd700 1 --2- 172.21.15.100:0/1060140870 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa25c1023a0 0x7fa25c19e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:53.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.547+0000 7fa25a7fc700 1 -- 172.21.15.100:0/1060140870 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa25c108410 msgr2=0x7fa25c198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:53.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.547+0000 7fa25a7fc700 1 --2- 172.21.15.100:0/1060140870 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa25c108410 0x7fa25c198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:53.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.547+0000 7fa25a7fc700 1 -- 172.21.15.100:0/1060140870 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa25c1023a0 msgr2=0x7fa25c19e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:53.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.547+0000 7fa25a7fc700 1 --2- 172.21.15.100:0/1060140870 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa25c1023a0 0x7fa25c19e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:53.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.547+0000 7fa25a7fc700 1 -- 172.21.15.100:0/1060140870 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa25c199050 con 0x7fa25c102d60 2024-05-10T11:29:53.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.547+0000 7fa25affd700 1 --2- 172.21.15.100:0/1060140870 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa25c1023a0 0x7fa25c19e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:29:53.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.547+0000 7fa25b7fe700 1 --2- 172.21.15.100:0/1060140870 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa25c108410 0x7fa25c198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:29:53.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.547+0000 7fa25a7fc700 1 --2- 172.21.15.100:0/1060140870 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa25c102d60 0x7fa25c19eb90 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fa25000c9b0 tx=0x7fa25000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:53.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.547+0000 7fa243fff700 1 -- 172.21.15.100:0/1060140870 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa250007b00 con 0x7fa25c102d60 2024-05-10T11:29:53.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.547+0000 7fa2612a2700 1 -- 172.21.15.100:0/1060140870 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa25c199340 con 0x7fa25c102d60 2024-05-10T11:29:53.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.547+0000 7fa2612a2700 1 -- 172.21.15.100:0/1060140870 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa25c1a55d0 con 0x7fa25c102d60 2024-05-10T11:29:53.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.547+0000 7fa243fff700 1 -- 172.21.15.100:0/1060140870 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa250007ca0 con 0x7fa25c102d60 2024-05-10T11:29:53.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.547+0000 7fa243fff700 1 -- 172.21.15.100:0/1060140870 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa250018660 con 0x7fa25c102d60 2024-05-10T11:29:53.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.550+0000 7fa243fff700 1 -- 172.21.15.100:0/1060140870 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fa25000f450 con 0x7fa25c102d60 2024-05-10T11:29:53.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.551+0000 7fa243fff700 1 --2- 172.21.15.100:0/1060140870 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa2440b2900 0x7fa2440b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:53.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.551+0000 7fa25affd700 1 --2- 172.21.15.100:0/1060140870 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa2440b2900 0x7fa2440b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:53.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.551+0000 7fa243fff700 1 -- 172.21.15.100:0/1060140870 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fa2500e2340 con 0x7fa25c102d60 2024-05-10T11:29:53.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.551+0000 7fa26bb35b80 1 Processor -- start 2024-05-10T11:29:53.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.551+0000 7fa26bb35b80 1 -- start start 2024-05-10T11:29:53.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.551+0000 7fa25affd700 1 --2- 172.21.15.100:0/1060140870 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa2440b2900 0x7fa2440b4dc0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fa24c00b2e0 tx=0x7fa24c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:53.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.551+0000 7fa26bb35b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652f2906bb0 0x5652f2906f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:53.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.551+0000 7fa26bb35b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5652f29052f0 0x5652f2905770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:53.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.551+0000 7fa26bb35b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652f2905cb0 0x5652f2915980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:53.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.551+0000 7fa26bb35b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5652f2857860 con 0x5652f2906bb0 2024-05-10T11:29:53.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.551+0000 7fa26bb35b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5652f27f67c0 con 0x5652f2905cb0 2024-05-10T11:29:53.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.551+0000 7fa26bb35b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5652f2853820 con 0x5652f29052f0 2024-05-10T11:29:53.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.552+0000 7fa25affd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652f2906bb0 0x5652f2906f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:53.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.552+0000 7fa25affd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652f2906bb0 0x5652f2906f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44356/0 (socket says 172.21.15.100:44356) 2024-05-10T11:29:53.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.552+0000 7fa25affd700 1 -- 172.21.15.100:0/2664286152 learned_addr learned my addr 172.21.15.100:0/2664286152 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:29:53.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.552+0000 7fa25a7fc700 1 --2- 172.21.15.100:0/2664286152 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5652f29052f0 0x5652f2905770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:53.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.552+0000 7fa25b7fe700 1 --2- 172.21.15.100:0/2664286152 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652f2905cb0 0x5652f2915980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:53.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.552+0000 7fa25a7fc700 1 -- 172.21.15.100:0/2664286152 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652f2905cb0 msgr2=0x5652f2915980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:53.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.552+0000 7fa25a7fc700 1 --2- 172.21.15.100:0/2664286152 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652f2905cb0 0x5652f2915980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:53.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.552+0000 7fa25a7fc700 1 -- 172.21.15.100:0/2664286152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652f2906bb0 msgr2=0x5652f2906f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:53.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.552+0000 7fa25a7fc700 1 --2- 172.21.15.100:0/2664286152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652f2906bb0 0x5652f2906f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:53.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.552+0000 7fa25a7fc700 1 -- 172.21.15.100:0/2664286152 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652f27a48d0 con 0x5652f29052f0 2024-05-10T11:29:53.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.552+0000 7fa25affd700 1 --2- 172.21.15.100:0/2664286152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652f2906bb0 0x5652f2906f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:29:53.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.552+0000 7fa25a7fc700 1 --2- 172.21.15.100:0/2664286152 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5652f29052f0 0x5652f2905770 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa2500dfbd0 tx=0x7fa2500a9070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2d29965d561fc0b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:53.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.552+0000 7fa2417fa700 1 -- 172.21.15.100:0/2664286152 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2500a9da0 con 0x5652f29052f0 2024-05-10T11:29:53.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.552+0000 7fa2417fa700 1 -- 172.21.15.100:0/2664286152 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa2500a96a0 con 0x5652f29052f0 2024-05-10T11:29:53.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.552+0000 7fa2417fa700 1 -- 172.21.15.100:0/2664286152 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2500a9b60 con 0x5652f29052f0 2024-05-10T11:29:53.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.553+0000 7fa26bb35b80 1 -- 172.21.15.100:0/2664286152 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5652f29052f0 msgr2=0x5652f2905770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:53.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.553+0000 7fa26bb35b80 1 --2- 172.21.15.100:0/2664286152 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5652f29052f0 0x5652f2905770 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa2500dfbd0 tx=0x7fa2500a9070 comp rx=0 tx=0).stop 2024-05-10T11:29:53.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.553+0000 7fa26bb35b80 1 -- 172.21.15.100:0/2664286152 shutdown_connections 2024-05-10T11:29:53.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.553+0000 7fa26bb35b80 1 --2- 172.21.15.100:0/2664286152 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652f2905cb0 0x5652f2915980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:53.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.553+0000 7fa26bb35b80 1 --2- 172.21.15.100:0/2664286152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652f2906bb0 0x5652f2906f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:53.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.553+0000 7fa26bb35b80 1 --2- 172.21.15.100:0/2664286152 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5652f29052f0 0x5652f2905770 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:53.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.553+0000 7fa26bb35b80 1 -- 172.21.15.100:0/2664286152 >> 172.21.15.100:0/2664286152 conn(0x5652f2831100 msgr2=0x5652f28ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:53.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.553+0000 7fa26bb35b80 1 -- 172.21.15.100:0/2664286152 shutdown_connections 2024-05-10T11:29:53.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.553+0000 7fa26bb35b80 1 -- 172.21.15.100:0/2664286152 wait complete. 2024-05-10T11:29:53.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.554+0000 7fa26bb35b80 1 -- start start 2024-05-10T11:29:53.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.554+0000 7fa26bb35b80 1 Processor -- start 2024-05-10T11:29:53.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.554+0000 7fa26bb35b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5652f29052f0 0x5652f2915590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:53.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.554+0000 7fa26bb35b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652f2905cb0 0x5652f2912930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:53.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.554+0000 7fa26bb35b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652f2906bb0 0x5652f2912e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:53.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.554+0000 7fa26bb35b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5652f29162c0 con 0x5652f2905cb0 2024-05-10T11:29:53.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.554+0000 7fa26bb35b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5652f27bdc30 con 0x5652f2906bb0 2024-05-10T11:29:53.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.554+0000 7fa26bb35b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5652f2916440 con 0x5652f29052f0 2024-05-10T11:29:53.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.554+0000 7fa26bb35b80 10 client.0 _ll_register_callbacks cb 0x5652f27ef920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:29:53.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.554+0000 7fa25a7fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652f2905cb0 0x5652f2912930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:53.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.554+0000 7fa25a7fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652f2905cb0 0x5652f2912930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44360/0 (socket says 172.21.15.100:44360) 2024-05-10T11:29:53.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.554+0000 7fa25a7fc700 1 -- 172.21.15.100:0/2078002768 learned_addr learned my addr 172.21.15.100:0/2078002768 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:29:53.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.554+0000 7fa25affd700 1 --2- 172.21.15.100:0/2078002768 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5652f29052f0 0x5652f2915590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:53.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.554+0000 7fa25b7fe700 1 --2- 172.21.15.100:0/2078002768 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652f2906bb0 0x5652f2912e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:53.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.554+0000 7fa25a7fc700 1 -- 172.21.15.100:0/2078002768 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5652f29052f0 msgr2=0x5652f2915590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:53.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.554+0000 7fa25a7fc700 1 --2- 172.21.15.100:0/2078002768 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5652f29052f0 0x5652f2915590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:53.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.554+0000 7fa25a7fc700 1 -- 172.21.15.100:0/2078002768 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652f2906bb0 msgr2=0x5652f2912e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:53.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.554+0000 7fa25a7fc700 1 --2- 172.21.15.100:0/2078002768 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652f2906bb0 0x5652f2912e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:53.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.554+0000 7fa25a7fc700 1 -- 172.21.15.100:0/2078002768 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652f27f3690 con 0x5652f2905cb0 2024-05-10T11:29:53.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.554+0000 7fa25b7fe700 1 --2- 172.21.15.100:0/2078002768 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652f2906bb0 0x5652f2912e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:29:53.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.555+0000 7fa25a7fc700 1 --2- 172.21.15.100:0/2078002768 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652f2905cb0 0x5652f2912930 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fa25000afd0 tx=0x7fa2500b22a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf4144b64f0aa556 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:53.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.555+0000 7fa23eff5700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:29:53.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.555+0000 7fa23eff5700 1 -- 172.21.15.100:0/2078002768 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2500912c0 con 0x5652f2905cb0 2024-05-10T11:29:53.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.555+0000 7fa23eff5700 1 -- 172.21.15.100:0/2078002768 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fa2500a5c90 con 0x5652f2905cb0 2024-05-10T11:29:53.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.555+0000 7fa26bb35b80 1 -- 172.21.15.100:0/2078002768 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2500a96a0 con 0x5652f2905cb0 2024-05-10T11:29:53.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.555+0000 7fa23eff5700 1 -- 172.21.15.100:0/2078002768 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa2500ada50 con 0x5652f2905cb0 2024-05-10T11:29:53.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.555+0000 7fa26bb35b80 10 client.17727 fetch_fsmap learned FSMap version 33 2024-05-10T11:29:53.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.555+0000 7fa26bb35b80 1 -- 172.21.15.100:0/2078002768 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5652f2920af0 con 0x5652f2905cb0 2024-05-10T11:29:53.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.555+0000 7fa23eff5700 1 -- 172.21.15.100:0/2078002768 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa2500a96a0 con 0x5652f2905cb0 2024-05-10T11:29:53.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.555+0000 7fa23eff5700 1 -- 172.21.15.100:0/2078002768 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fa2500beaf0 con 0x5652f2905cb0 2024-05-10T11:29:53.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.555+0000 7fa26bb35b80 10 client.17727 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T11:29:53.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.555+0000 7fa26bb35b80 10 client.17727 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-10T11:29:53.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.555+0000 7fa26bb35b80 20 client.17727 populate_metadata read hostname 'smithi100' 2024-05-10T11:29:53.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.555+0000 7fa26bb35b80 1 --2- 172.21.15.100:0/2078002768 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x5652f2922fc0 0x5652f29253b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:53.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.555+0000 7fa26bb35b80 4 client.17727 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:29:53.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.555+0000 7fa26bb35b80 1 -- 172.21.15.100:0/2078002768 --> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5652f28d1650 con 0x5652f2922fc0 2024-05-10T11:29:53.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.556+0000 7fa25affd700 1 --2- 172.21.15.100:0/2078002768 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x5652f2922fc0 0x5652f29253b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:53.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.556+0000 7fa25affd700 1 --2- 172.21.15.100:0/2078002768 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x5652f2922fc0 0x5652f29253b0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=274b942e08ad0ae7 server_cookie=740e35a2d18fa968 in_seq=0 out_seq=0 2024-05-10T11:29:53.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.556+0000 7fa23eff5700 10 client.17727 ms_handle_connect on v2:172.21.15.178:6838/3760985404 2024-05-10T11:29:53.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.557+0000 7fa23eff5700 1 -- 172.21.15.100:0/2078002768 <== mds.1 v2:172.21.15.178:6838/3760985404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5652f28d1650 con 0x5652f2922fc0 2024-05-10T11:29:53.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.557+0000 7fa23eff5700 10 client.17727 handle_command_reply: tid=0 2024-05-10T11:29:53.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.557+0000 7fa26bb35b80 1 client.17727 shutdown 2024-05-10T11:29:53.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.557+0000 7fa26bb35b80 1 -- 172.21.15.100:0/2078002768 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652f2905cb0 msgr2=0x5652f2912930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:53.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.557+0000 7fa26bb35b80 1 --2- 172.21.15.100:0/2078002768 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652f2905cb0 0x5652f2912930 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fa25000afd0 tx=0x7fa2500b22a0 comp rx=0 tx=0).stop 2024-05-10T11:29:53.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.557+0000 7fa26bb35b80 1 -- 172.21.15.100:0/2078002768 shutdown_connections 2024-05-10T11:29:53.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.557+0000 7fa26bb35b80 1 --2- 172.21.15.100:0/2078002768 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x5652f2922fc0 0x5652f29253b0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:53.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.557+0000 7fa23eff5700 0 client.17727 ms_handle_reset on v2:172.21.15.178:6838/3760985404 2024-05-10T11:29:53.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.557+0000 7fa26bb35b80 1 --2- 172.21.15.100:0/2078002768 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652f2906bb0 0x5652f2912e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:53.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.557+0000 7fa26bb35b80 1 --2- 172.21.15.100:0/2078002768 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5652f29052f0 0x5652f2915590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:53.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.557+0000 7fa26bb35b80 1 --2- 172.21.15.100:0/2078002768 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652f2905cb0 0x5652f2912930 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:53.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.557+0000 7fa26bb35b80 1 -- 172.21.15.100:0/2078002768 >> 172.21.15.100:0/2078002768 conn(0x5652f2831100 msgr2=0x5652f28fa720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:53.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.557+0000 7fa26bb35b80 1 -- 172.21.15.100:0/2078002768 shutdown_connections 2024-05-10T11:29:53.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.557+0000 7fa26bb35b80 1 -- 172.21.15.100:0/2078002768 wait complete. 2024-05-10T11:29:53.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.557+0000 7fa26bb35b80 20 client.17727 trim_cache size 0 max 16384 2024-05-10T11:29:53.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.573+0000 7fa26bb35b80 1 Processor -- start 2024-05-10T11:29:53.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.573+0000 7fa26bb35b80 1 -- start start 2024-05-10T11:29:53.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.574+0000 7fa26bb35b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5652f2831100 0x5652f2912ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:53.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.574+0000 7fa26bb35b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652f29052f0 0x5652f29118d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:53.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.574+0000 7fa26bb35b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652f2905cb0 0x5652f290e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:53.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.574+0000 7fa26bb35b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa248011280 con 0x5652f29052f0 2024-05-10T11:29:53.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.574+0000 7fa26bb35b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa24800eb10 con 0x5652f2905cb0 2024-05-10T11:29:53.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.574+0000 7fa26bb35b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa24800c960 con 0x5652f2831100 2024-05-10T11:29:53.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.574+0000 7fa25a7fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652f29052f0 0x5652f29118d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:53.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.574+0000 7fa25a7fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652f29052f0 0x5652f29118d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44374/0 (socket says 172.21.15.100:44374) 2024-05-10T11:29:53.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.574+0000 7fa25a7fc700 1 -- 172.21.15.100:0/3656587093 learned_addr learned my addr 172.21.15.100:0/3656587093 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:29:53.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.574+0000 7fa25b7fe700 1 --2- 172.21.15.100:0/3656587093 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652f2905cb0 0x5652f290e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:53.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.574+0000 7fa25affd700 1 --2- 172.21.15.100:0/3656587093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5652f2831100 0x5652f2912ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:53.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.574+0000 7fa25a7fc700 1 -- 172.21.15.100:0/3656587093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5652f2831100 msgr2=0x5652f2912ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:53.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.574+0000 7fa25a7fc700 1 --2- 172.21.15.100:0/3656587093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5652f2831100 0x5652f2912ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:53.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.574+0000 7fa25a7fc700 1 -- 172.21.15.100:0/3656587093 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652f2905cb0 msgr2=0x5652f290e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:53.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.574+0000 7fa25a7fc700 1 --2- 172.21.15.100:0/3656587093 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652f2905cb0 0x5652f290e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:53.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.574+0000 7fa25a7fc700 1 -- 172.21.15.100:0/3656587093 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652f2900d40 con 0x5652f29052f0 2024-05-10T11:29:53.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.574+0000 7fa25a7fc700 1 --2- 172.21.15.100:0/3656587093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652f29052f0 0x5652f29118d0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fa25000fcb0 tx=0x7fa2500a9310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4bbdd30608253b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:53.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.574+0000 7fa240ff9700 1 -- 172.21.15.100:0/3656587093 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa250088030 con 0x5652f29052f0 2024-05-10T11:29:53.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.574+0000 7fa240ff9700 1 -- 172.21.15.100:0/3656587093 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa2500a9da0 con 0x5652f29052f0 2024-05-10T11:29:53.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.574+0000 7fa240ff9700 1 -- 172.21.15.100:0/3656587093 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2500a52e0 con 0x5652f29052f0 2024-05-10T11:29:53.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.575+0000 7fa26bb35b80 1 -- 172.21.15.100:0/3656587093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652f29052f0 msgr2=0x5652f29118d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:53.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.575+0000 7fa26bb35b80 1 --2- 172.21.15.100:0/3656587093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652f29052f0 0x5652f29118d0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fa25000fcb0 tx=0x7fa2500a9310 comp rx=0 tx=0).stop 2024-05-10T11:29:53.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.575+0000 7fa26bb35b80 1 -- 172.21.15.100:0/3656587093 shutdown_connections 2024-05-10T11:29:53.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.575+0000 7fa26bb35b80 1 --2- 172.21.15.100:0/3656587093 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652f2905cb0 0x5652f290e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:53.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.575+0000 7fa26bb35b80 1 --2- 172.21.15.100:0/3656587093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5652f2831100 0x5652f2912ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:53.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.575+0000 7fa26bb35b80 1 --2- 172.21.15.100:0/3656587093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652f29052f0 0x5652f29118d0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:53.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.575+0000 7fa26bb35b80 1 -- 172.21.15.100:0/3656587093 >> 172.21.15.100:0/3656587093 conn(0x5652f28f9a80 msgr2=0x5652f28f9e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:53.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.575+0000 7fa26bb35b80 1 -- 172.21.15.100:0/3656587093 shutdown_connections 2024-05-10T11:29:53.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.575+0000 7fa26bb35b80 1 -- 172.21.15.100:0/3656587093 wait complete. 2024-05-10T11:29:53.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.576+0000 7fa26bb35b80 1 -- start start 2024-05-10T11:29:53.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.576+0000 7fa26bb35b80 1 Processor -- start 2024-05-10T11:29:53.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.576+0000 7fa26bb35b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652f2831100 0x5652f2925970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:53.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.576+0000 7fa26bb35b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652f29052f0 0x5652f2925eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:53.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.576+0000 7fa26bb35b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5652f2905cb0 0x5652f298b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:53.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.576+0000 7fa26bb35b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa24800cfd0 con 0x5652f29052f0 2024-05-10T11:29:53.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.576+0000 7fa26bb35b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa24800e440 con 0x5652f2831100 2024-05-10T11:29:53.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.576+0000 7fa26bb35b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa24800d6a0 con 0x5652f2905cb0 2024-05-10T11:29:53.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.576+0000 7fa25a7fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652f29052f0 0x5652f2925eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:53.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.576+0000 7fa26bb35b80 10 client.0 _ll_register_callbacks cb 0x5652f27ef920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:29:53.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.576+0000 7fa25a7fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652f29052f0 0x5652f2925eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44376/0 (socket says 172.21.15.100:44376) 2024-05-10T11:29:53.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.576+0000 7fa25a7fc700 1 -- 172.21.15.100:0/4261491697 learned_addr learned my addr 172.21.15.100:0/4261491697 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:29:53.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.576+0000 7fa25affd700 1 --2- 172.21.15.100:0/4261491697 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652f2831100 0x5652f2925970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:53.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.576+0000 7fa25b7fe700 1 --2- 172.21.15.100:0/4261491697 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5652f2905cb0 0x5652f298b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:53.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.576+0000 7fa25a7fc700 1 -- 172.21.15.100:0/4261491697 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5652f2905cb0 msgr2=0x5652f298b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:53.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.576+0000 7fa25a7fc700 1 --2- 172.21.15.100:0/4261491697 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5652f2905cb0 0x5652f298b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:53.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.576+0000 7fa25a7fc700 1 -- 172.21.15.100:0/4261491697 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652f2831100 msgr2=0x5652f2925970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:53.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.576+0000 7fa25a7fc700 1 --2- 172.21.15.100:0/4261491697 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652f2831100 0x5652f2925970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:53.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.576+0000 7fa25a7fc700 1 -- 172.21.15.100:0/4261491697 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652f298bb40 con 0x5652f29052f0 2024-05-10T11:29:53.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.577+0000 7fa25a7fc700 1 --2- 172.21.15.100:0/4261491697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652f29052f0 0x5652f2925eb0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fa2500a6910 tx=0x7fa2500a9070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b4c4a804e2f636d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:53.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.577+0000 7fa23eff5700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:29:53.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.577+0000 7fa23eff5700 1 -- 172.21.15.100:0/4261491697 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa250088040 con 0x5652f29052f0 2024-05-10T11:29:53.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.577+0000 7fa23eff5700 1 -- 172.21.15.100:0/4261491697 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fa2500a5c90 con 0x5652f29052f0 2024-05-10T11:29:53.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.577+0000 7fa26bb35b80 1 -- 172.21.15.100:0/4261491697 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2500a9da0 con 0x5652f29052f0 2024-05-10T11:29:53.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.577+0000 7fa23eff5700 1 -- 172.21.15.100:0/4261491697 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa2500a96a0 con 0x5652f29052f0 2024-05-10T11:29:53.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.577+0000 7fa26bb35b80 10 client.17733 fetch_fsmap learned FSMap version 33 2024-05-10T11:29:53.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.577+0000 7fa26bb35b80 1 -- 172.21.15.100:0/4261491697 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5652f298bb40 con 0x5652f29052f0 2024-05-10T11:29:53.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.577+0000 7fa23eff5700 1 -- 172.21.15.100:0/4261491697 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa2500a9da0 con 0x5652f29052f0 2024-05-10T11:29:53.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.577+0000 7fa23eff5700 1 -- 172.21.15.100:0/4261491697 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fa25008d020 con 0x5652f29052f0 2024-05-10T11:29:53.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.577+0000 7fa26bb35b80 10 client.17733 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T11:29:53.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.577+0000 7fa26bb35b80 10 client.17733 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-10T11:29:53.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.577+0000 7fa26bb35b80 20 client.17733 populate_metadata read hostname 'smithi100' 2024-05-10T11:29:53.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.577+0000 7fa26bb35b80 1 --2- 172.21.15.100:0/4261491697 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x5652f2906bb0 0x5652f298f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:53.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.577+0000 7fa26bb35b80 4 client.17733 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:29:53.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.577+0000 7fa26bb35b80 1 -- 172.21.15.100:0/4261491697 --> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5652f2865240 con 0x5652f2906bb0 2024-05-10T11:29:53.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.578+0000 7fa25affd700 1 --2- 172.21.15.100:0/4261491697 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x5652f2906bb0 0x5652f298f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:53.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.578+0000 7fa23eff5700 10 client.17733 ms_handle_connect on v2:172.21.15.178:6838/3760985404 2024-05-10T11:29:53.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.578+0000 7fa25affd700 1 --2- 172.21.15.100:0/4261491697 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x5652f2906bb0 0x5652f298f070 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=764ad8d6fe9a74bb server_cookie=5012acdbb3dba2ae in_seq=0 out_seq=0 2024-05-10T11:29:53.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.578+0000 7fa23eff5700 1 -- 172.21.15.100:0/4261491697 <== mds.1 v2:172.21.15.178:6838/3760985404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5652f2865240 con 0x5652f2906bb0 2024-05-10T11:29:53.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.578+0000 7fa23eff5700 10 client.17733 handle_command_reply: tid=0 2024-05-10T11:29:53.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.578+0000 7fa26bb35b80 1 client.17733 shutdown 2024-05-10T11:29:53.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.578+0000 7fa26bb35b80 1 -- 172.21.15.100:0/4261491697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652f29052f0 msgr2=0x5652f2925eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:53.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.578+0000 7fa26bb35b80 1 --2- 172.21.15.100:0/4261491697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652f29052f0 0x5652f2925eb0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fa2500a6910 tx=0x7fa2500a9070 comp rx=0 tx=0).stop 2024-05-10T11:29:53.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.578+0000 7fa26bb35b80 1 -- 172.21.15.100:0/4261491697 shutdown_connections 2024-05-10T11:29:53.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.578+0000 7fa26bb35b80 1 --2- 172.21.15.100:0/4261491697 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x5652f2906bb0 0x5652f298f070 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:53.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.578+0000 7fa26bb35b80 1 --2- 172.21.15.100:0/4261491697 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5652f2905cb0 0x5652f298b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:53.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.579+0000 7fa26bb35b80 1 --2- 172.21.15.100:0/4261491697 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652f2831100 0x5652f2925970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:53.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.579+0000 7fa26bb35b80 1 --2- 172.21.15.100:0/4261491697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652f29052f0 0x5652f2925eb0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:53.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.579+0000 7fa23eff5700 0 client.17733 ms_handle_reset on v2:172.21.15.178:6838/3760985404 2024-05-10T11:29:53.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.579+0000 7fa26bb35b80 1 -- 172.21.15.100:0/4261491697 >> 172.21.15.100:0/4261491697 conn(0x5652f28f9a80 msgr2=0x5652f290e030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:53.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.579+0000 7fa26bb35b80 1 -- 172.21.15.100:0/4261491697 shutdown_connections 2024-05-10T11:29:53.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.579+0000 7fa26bb35b80 1 -- 172.21.15.100:0/4261491697 wait complete. 2024-05-10T11:29:53.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.579+0000 7fa26bb35b80 20 client.17733 trim_cache size 0 max 16384 2024-05-10T11:29:53.584 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:29:53.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.580+0000 7fa2612a2700 1 -- 172.21.15.100:0/1060140870 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa2440b2900 msgr2=0x7fa2440b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:53.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.580+0000 7fa2612a2700 1 --2- 172.21.15.100:0/1060140870 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa2440b2900 0x7fa2440b4dc0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fa24c00b2e0 tx=0x7fa24c01a040 comp rx=0 tx=0).stop 2024-05-10T11:29:53.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.580+0000 7fa2612a2700 1 -- 172.21.15.100:0/1060140870 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa25c102d60 msgr2=0x7fa25c19eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:53.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.580+0000 7fa2612a2700 1 --2- 172.21.15.100:0/1060140870 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa25c102d60 0x7fa25c19eb90 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fa25000c9b0 tx=0x7fa25000ccc0 comp rx=0 tx=0).stop 2024-05-10T11:29:53.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.580+0000 7fa2612a2700 1 -- 172.21.15.100:0/1060140870 shutdown_connections 2024-05-10T11:29:53.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.580+0000 7fa2612a2700 1 --2- 172.21.15.100:0/1060140870 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa2440b2900 0x7fa2440b4dc0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:53.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.580+0000 7fa2612a2700 1 --2- 172.21.15.100:0/1060140870 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa25c108410 0x7fa25c198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:53.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.580+0000 7fa2612a2700 1 --2- 172.21.15.100:0/1060140870 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa25c1023a0 0x7fa25c19e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:53.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.580+0000 7fa2612a2700 1 --2- 172.21.15.100:0/1060140870 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa25c102d60 0x7fa25c19eb90 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:53.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.580+0000 7fa2612a2700 1 -- 172.21.15.100:0/1060140870 >> 172.21.15.100:0/1060140870 conn(0x7fa25c079350 msgr2=0x7fa25c1010d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:53.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.580+0000 7fa2612a2700 1 -- 172.21.15.100:0/1060140870 shutdown_connections 2024-05-10T11:29:53.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:53.580+0000 7fa2612a2700 1 -- 172.21.15.100:0/1060140870 wait complete. 2024-05-10T11:29:53.982 INFO:tasks.workunit.client.0.smithi100.stdout: 1 300073 2.71 MB/sec execute 468 sec latency 61.810 ms 2024-05-10T11:29:54.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:29:53 smithi178 ceph-mon[33359]: pgmap v766: 129 pgs: 129 active+clean; 119 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 1.2 MiB/s wr, 135 op/s 2024-05-10T11:29:54.254 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:29:53 smithi100 ceph-mon[30296]: pgmap v766: 129 pgs: 129 active+clean; 119 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 1.2 MiB/s wr, 135 op/s 2024-05-10T11:29:54.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:29:53 smithi188 ceph-mon[34075]: pgmap v766: 129 pgs: 129 active+clean; 119 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 1.2 MiB/s wr, 135 op/s 2024-05-10T11:29:54.982 INFO:tasks.workunit.client.0.smithi100.stdout: 1 300540 2.71 MB/sec execute 469 sec latency 97.511 ms 2024-05-10T11:29:55.982 INFO:tasks.workunit.client.0.smithi100.stdout: 1 301271 2.72 MB/sec execute 470 sec latency 30.130 ms 2024-05-10T11:29:56.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:29:55 smithi178 ceph-mon[33359]: pgmap v767: 129 pgs: 129 active+clean; 118 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1.3 MiB/s wr, 135 op/s 2024-05-10T11:29:56.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:29:55 smithi188 ceph-mon[34075]: pgmap v767: 129 pgs: 129 active+clean; 118 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1.3 MiB/s wr, 135 op/s 2024-05-10T11:29:56.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:29:55 smithi100 ceph-mon[30296]: pgmap v767: 129 pgs: 129 active+clean; 118 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1.3 MiB/s wr, 135 op/s 2024-05-10T11:29:56.864 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-10T11:29:56.982 INFO:tasks.workunit.client.0.smithi100.stdout: 1 301668 2.72 MB/sec execute 471 sec latency 115.380 ms 2024-05-10T11:29:57.132 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:29:57.982 INFO:tasks.workunit.client.0.smithi100.stdout: 1 301952 2.72 MB/sec execute 472 sec latency 56.412 ms 2024-05-10T11:29:58.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:29:57 smithi178 ceph-mon[33359]: pgmap v768: 129 pgs: 129 active+clean; 119 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 1.4 MiB/s wr, 165 op/s 2024-05-10T11:29:58.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:29:57 smithi188 ceph-mon[34075]: pgmap v768: 129 pgs: 129 active+clean; 119 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 1.4 MiB/s wr, 165 op/s 2024-05-10T11:29:58.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:29:57 smithi100 ceph-mon[30296]: pgmap v768: 129 pgs: 129 active+clean; 119 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 1.4 MiB/s wr, 165 op/s 2024-05-10T11:29:58.982 INFO:tasks.workunit.client.0.smithi100.stdout: 1 302347 2.71 MB/sec execute 473 sec latency 15.147 ms 2024-05-10T11:29:59.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.027+0000 7f0ed1b5e700 1 -- 172.21.15.100:0/2090687873 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ecc1095e0 msgr2=0x7f0ecc1099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:59.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.027+0000 7f0ed1b5e700 1 --2- 172.21.15.100:0/2090687873 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ecc1095e0 0x7f0ecc1099c0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc0099c0 tx=0x7f0ebc009cd0 comp rx=0 tx=0).stop 2024-05-10T11:29:59.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.027+0000 7f0ed1b5e700 1 -- 172.21.15.100:0/2090687873 shutdown_connections 2024-05-10T11:29:59.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.027+0000 7f0ed1b5e700 1 --2- 172.21.15.100:0/2090687873 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ecc103f00 0x7f0ecc1103b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:59.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.027+0000 7f0ed1b5e700 1 --2- 172.21.15.100:0/2090687873 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ecc1095e0 0x7f0ecc1099c0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:59.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.027+0000 7f0ed1b5e700 1 --2- 172.21.15.100:0/2090687873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ecc103540 0x7f0ecc1039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:59.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.027+0000 7f0ed1b5e700 1 -- 172.21.15.100:0/2090687873 >> 172.21.15.100:0/2090687873 conn(0x7f0ecc07a650 msgr2=0x7f0ecc07aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:59.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.027+0000 7f0ed1b5e700 1 -- 172.21.15.100:0/2090687873 shutdown_connections 2024-05-10T11:29:59.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.027+0000 7f0ed1b5e700 1 -- 172.21.15.100:0/2090687873 wait complete. 2024-05-10T11:29:59.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.028+0000 7f0ed1b5e700 1 Processor -- start 2024-05-10T11:29:59.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.028+0000 7f0ed1b5e700 1 -- start start 2024-05-10T11:29:59.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.029+0000 7f0ed1b5e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ecc103540 0x7f0ecc0757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:59.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.029+0000 7f0ed1b5e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ecc103f00 0x7f0ecc075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:59.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.029+0000 7f0ed1b5e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ecc1095e0 0x7f0ecc076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:59.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.029+0000 7f0ed1b5e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0ecc1135b0 con 0x7f0ecc103f00 2024-05-10T11:29:59.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.029+0000 7f0ed1b5e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0ecc113430 con 0x7f0ecc1095e0 2024-05-10T11:29:59.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.029+0000 7f0ed1b5e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0ecc113730 con 0x7f0ecc103540 2024-05-10T11:29:59.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.029+0000 7f0ecb7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ecc103540 0x7f0ecc0757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:59.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.029+0000 7f0ecaffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ecc103f00 0x7f0ecc075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:59.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.029+0000 7f0ecb7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ecc103540 0x7f0ecc0757a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:53900/0 (socket says 172.21.15.100:53900) 2024-05-10T11:29:59.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.029+0000 7f0ecb7fe700 1 -- 172.21.15.100:0/2561369619 learned_addr learned my addr 172.21.15.100:0/2561369619 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:29:59.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.029+0000 7f0ecbfff700 1 --2- 172.21.15.100:0/2561369619 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ecc1095e0 0x7f0ecc076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:59.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.029+0000 7f0ecb7fe700 1 -- 172.21.15.100:0/2561369619 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ecc1095e0 msgr2=0x7f0ecc076220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:59.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.029+0000 7f0ecb7fe700 1 --2- 172.21.15.100:0/2561369619 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ecc1095e0 0x7f0ecc076220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:59.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.029+0000 7f0ecb7fe700 1 -- 172.21.15.100:0/2561369619 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ecc103f00 msgr2=0x7f0ecc075ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:59.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.029+0000 7f0ecb7fe700 1 --2- 172.21.15.100:0/2561369619 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ecc103f00 0x7f0ecc075ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:59.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.029+0000 7f0ecb7fe700 1 -- 172.21.15.100:0/2561369619 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ecc1a7f90 con 0x7f0ecc103540 2024-05-10T11:29:59.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.029+0000 7f0ecbfff700 1 --2- 172.21.15.100:0/2561369619 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ecc1095e0 0x7f0ecc076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:29:59.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.029+0000 7f0ecb7fe700 1 --2- 172.21.15.100:0/2561369619 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ecc103540 0x7f0ecc0757a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc006010 tx=0x7f0ebc004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:59.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.030+0000 7f0ec8ff9700 1 -- 172.21.15.100:0/2561369619 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc026070 con 0x7f0ecc103540 2024-05-10T11:29:59.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.030+0000 7f0ec8ff9700 1 -- 172.21.15.100:0/2561369619 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0ebc004630 con 0x7f0ecc103540 2024-05-10T11:29:59.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.030+0000 7f0ec8ff9700 1 -- 172.21.15.100:0/2561369619 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc00faa0 con 0x7f0ecc103540 2024-05-10T11:29:59.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.030+0000 7f0ed1b5e700 1 -- 172.21.15.100:0/2561369619 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ecc1a8220 con 0x7f0ecc103540 2024-05-10T11:29:59.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.030+0000 7f0ed1b5e700 1 -- 172.21.15.100:0/2561369619 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ecc1a8620 con 0x7f0ecc103540 2024-05-10T11:29:59.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.033+0000 7f0ec8ff9700 1 -- 172.21.15.100:0/2561369619 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f0ebc029070 con 0x7f0ecc103540 2024-05-10T11:29:59.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.033+0000 7f0ec8ff9700 1 --2- 172.21.15.100:0/2561369619 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0eb40b2900 0x7f0eb40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:59.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.034+0000 7f0ecaffd700 1 --2- 172.21.15.100:0/2561369619 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0eb40b2900 0x7f0eb40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:59.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.034+0000 7f0ec8ff9700 1 -- 172.21.15.100:0/2561369619 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f0ebc0e2c40 con 0x7f0ecc103540 2024-05-10T11:29:59.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.034+0000 7f0edc3f1b80 1 Processor -- start 2024-05-10T11:29:59.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.034+0000 7f0edc3f1b80 1 -- start start 2024-05-10T11:29:59.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.034+0000 7f0edc3f1b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b2bc0251b0 0x55b2bc025590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:59.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.034+0000 7f0edc3f1b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b2bc025ad0 0x55b2bc01f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:59.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.034+0000 7f0ecaffd700 1 --2- 172.21.15.100:0/2561369619 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0eb40b2900 0x7f0eb40b4dc0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc07ad30 tx=0x7f0ec0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:59.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.034+0000 7f0edc3f1b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2bc01fd50 0x55b2bc020190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:59.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.034+0000 7f0edc3f1b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55b2bbf71940 con 0x55b2bc025ad0 2024-05-10T11:29:59.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.034+0000 7f0edc3f1b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b2bbed6da0 con 0x55b2bc0251b0 2024-05-10T11:29:59.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.034+0000 7f0edc3f1b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b2bbf6d900 con 0x55b2bc01fd50 2024-05-10T11:29:59.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.034+0000 7f0ecaffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b2bc025ad0 0x55b2bc01f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:59.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.034+0000 7f0ecaffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b2bc025ad0 0x55b2bc01f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35988/0 (socket says 172.21.15.100:35988) 2024-05-10T11:29:59.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.034+0000 7f0ecaffd700 1 -- 172.21.15.100:0/2516536695 learned_addr learned my addr 172.21.15.100:0/2516536695 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:29:59.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.034+0000 7f0ecbfff700 1 --2- 172.21.15.100:0/2516536695 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2bc01fd50 0x55b2bc020190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:59.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.034+0000 7f0ecb7fe700 1 --2- 172.21.15.100:0/2516536695 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b2bc0251b0 0x55b2bc025590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:59.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.034+0000 7f0ecaffd700 1 -- 172.21.15.100:0/2516536695 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2bc01fd50 msgr2=0x55b2bc020190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:59.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.034+0000 7f0ecaffd700 1 --2- 172.21.15.100:0/2516536695 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2bc01fd50 0x55b2bc020190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:59.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.035+0000 7f0ecaffd700 1 -- 172.21.15.100:0/2516536695 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b2bc0251b0 msgr2=0x55b2bc025590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:59.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.035+0000 7f0ecaffd700 1 --2- 172.21.15.100:0/2516536695 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b2bc0251b0 0x55b2bc025590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:59.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.035+0000 7f0ecaffd700 1 -- 172.21.15.100:0/2516536695 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2bbebe8d0 con 0x55b2bc025ad0 2024-05-10T11:29:59.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.035+0000 7f0ecb7fe700 1 --2- 172.21.15.100:0/2516536695 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b2bc0251b0 0x55b2bc025590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:29:59.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.035+0000 7f0ecaffd700 1 --2- 172.21.15.100:0/2516536695 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b2bc025ad0 0x55b2bc01f810 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f0ec0011d20 tx=0x7f0ec0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82e6b27064013630 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:59.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.035+0000 7f0eb1ffb700 1 -- 172.21.15.100:0/2516536695 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec000eb20 con 0x55b2bc025ad0 2024-05-10T11:29:59.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.035+0000 7f0eb1ffb700 1 -- 172.21.15.100:0/2516536695 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0ec000ecc0 con 0x55b2bc025ad0 2024-05-10T11:29:59.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.035+0000 7f0eb1ffb700 1 -- 172.21.15.100:0/2516536695 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec001d610 con 0x55b2bc025ad0 2024-05-10T11:29:59.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.036+0000 7f0edc3f1b80 1 -- 172.21.15.100:0/2516536695 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b2bc025ad0 msgr2=0x55b2bc01f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:59.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.036+0000 7f0edc3f1b80 1 --2- 172.21.15.100:0/2516536695 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b2bc025ad0 0x55b2bc01f810 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f0ec0011d20 tx=0x7f0ec0014000 comp rx=0 tx=0).stop 2024-05-10T11:29:59.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.036+0000 7f0edc3f1b80 1 -- 172.21.15.100:0/2516536695 shutdown_connections 2024-05-10T11:29:59.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.036+0000 7f0edc3f1b80 1 --2- 172.21.15.100:0/2516536695 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2bc01fd50 0x55b2bc020190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:59.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.036+0000 7f0edc3f1b80 1 --2- 172.21.15.100:0/2516536695 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b2bc0251b0 0x55b2bc025590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:59.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.036+0000 7f0edc3f1b80 1 --2- 172.21.15.100:0/2516536695 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b2bc025ad0 0x55b2bc01f810 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:59.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.036+0000 7f0edc3f1b80 1 -- 172.21.15.100:0/2516536695 >> 172.21.15.100:0/2516536695 conn(0x55b2bbf4bb70 msgr2=0x55b2bc01a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:59.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.036+0000 7f0edc3f1b80 1 -- 172.21.15.100:0/2516536695 shutdown_connections 2024-05-10T11:29:59.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.036+0000 7f0edc3f1b80 1 -- 172.21.15.100:0/2516536695 wait complete. 2024-05-10T11:29:59.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.036+0000 7f0edc3f1b80 1 -- start start 2024-05-10T11:29:59.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.037+0000 7f0edc3f1b80 1 Processor -- start 2024-05-10T11:29:59.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.037+0000 7f0edc3f1b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b2bc01fd50 0x55b2bc02b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:59.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.037+0000 7f0edc3f1b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b2bc0251b0 0x55b2bc028b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:59.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.037+0000 7f0edc3f1b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2bc025ad0 0x55b2bc029050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:59.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.037+0000 7f0edc3f1b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55b2bc02c450 con 0x55b2bc01fd50 2024-05-10T11:29:59.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.037+0000 7f0edc3f1b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b2bbf2aed0 con 0x55b2bc0251b0 2024-05-10T11:29:59.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.037+0000 7f0edc3f1b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b2bc02c5d0 con 0x55b2bc025ad0 2024-05-10T11:29:59.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.037+0000 7f0edc3f1b80 10 client.0 _ll_register_callbacks cb 0x55b2bbf055b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:29:59.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.037+0000 7f0ecb7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b2bc01fd50 0x55b2bc02b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:59.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.037+0000 7f0ecbfff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2bc025ad0 0x55b2bc029050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:59.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.037+0000 7f0ecb7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b2bc01fd50 0x55b2bc02b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36002/0 (socket says 172.21.15.100:36002) 2024-05-10T11:29:59.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.037+0000 7f0ecb7fe700 1 -- 172.21.15.100:0/4239740672 learned_addr learned my addr 172.21.15.100:0/4239740672 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:29:59.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.037+0000 7f0ecaffd700 1 --2- 172.21.15.100:0/4239740672 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b2bc0251b0 0x55b2bc028b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:59.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.037+0000 7f0ecb7fe700 1 -- 172.21.15.100:0/4239740672 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2bc025ad0 msgr2=0x55b2bc029050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:59.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.037+0000 7f0ecb7fe700 1 --2- 172.21.15.100:0/4239740672 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2bc025ad0 0x55b2bc029050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:59.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.037+0000 7f0ecb7fe700 1 -- 172.21.15.100:0/4239740672 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b2bc0251b0 msgr2=0x55b2bc028b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:59.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.037+0000 7f0ecb7fe700 1 --2- 172.21.15.100:0/4239740672 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b2bc0251b0 0x55b2bc028b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:59.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.037+0000 7f0ecb7fe700 1 -- 172.21.15.100:0/4239740672 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2bbf28760 con 0x55b2bc01fd50 2024-05-10T11:29:59.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.037+0000 7f0ecbfff700 1 --2- 172.21.15.100:0/4239740672 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2bc025ad0 0x55b2bc029050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:29:59.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.037+0000 7f0ecb7fe700 1 --2- 172.21.15.100:0/4239740672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b2bc01fd50 0x55b2bc02b770 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc0e0750 tx=0x7f0ebc010f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e31b724cb16feab4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:59.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.038+0000 7f0eaaffd700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:29:59.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.038+0000 7f0eaaffd700 1 -- 172.21.15.100:0/4239740672 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc0a9aa0 con 0x55b2bc01fd50 2024-05-10T11:29:59.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.038+0000 7f0eaaffd700 1 -- 172.21.15.100:0/4239740672 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f0ebc0b2ae0 con 0x55b2bc01fd50 2024-05-10T11:29:59.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.038+0000 7f0edc3f1b80 1 -- 172.21.15.100:0/4239740672 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ec000ecc0 con 0x55b2bc01fd50 2024-05-10T11:29:59.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.038+0000 7f0eaaffd700 1 -- 172.21.15.100:0/4239740672 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f0ebc0aa690 con 0x55b2bc01fd50 2024-05-10T11:29:59.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.038+0000 7f0edc3f1b80 10 client.17745 fetch_fsmap learned FSMap version 33 2024-05-10T11:29:59.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.038+0000 7f0edc3f1b80 1 -- 172.21.15.100:0/4239740672 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b2bc036d30 con 0x55b2bc01fd50 2024-05-10T11:29:59.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.038+0000 7f0eaaffd700 1 -- 172.21.15.100:0/4239740672 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0ec000ecc0 con 0x55b2bc01fd50 2024-05-10T11:29:59.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.038+0000 7f0eaaffd700 1 -- 172.21.15.100:0/4239740672 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f0ebc0a5a90 con 0x55b2bc01fd50 2024-05-10T11:29:59.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.038+0000 7f0edc3f1b80 10 client.17745 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T11:29:59.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.038+0000 7f0edc3f1b80 10 client.17745 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:29:59.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.038+0000 7f0edc3f1b80 20 client.17745 populate_metadata read hostname 'smithi100' 2024-05-10T11:29:59.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.038+0000 7f0edc3f1b80 1 --2- 172.21.15.100:0/4239740672 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55b2bc0392a0 0x55b2bc03b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:59.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.038+0000 7f0edc3f1b80 4 client.17745 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:29:59.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.038+0000 7f0edc3f1b80 1 -- 172.21.15.100:0/4239740672 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b2bbf7f320 con 0x55b2bc0392a0 2024-05-10T11:29:59.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.039+0000 7f0ecaffd700 1 --2- 172.21.15.100:0/4239740672 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55b2bc0392a0 0x55b2bc03b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:59.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.039+0000 7f0ecaffd700 1 --2- 172.21.15.100:0/4239740672 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55b2bc0392a0 0x55b2bc03b690 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8459d8ee3bfcd219 server_cookie=4a6e25e5e13cf96e in_seq=0 out_seq=0 2024-05-10T11:29:59.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.039+0000 7f0eaaffd700 10 client.17745 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:29:59.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.040+0000 7f0eaaffd700 1 -- 172.21.15.100:0/4239740672 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b2bbf7f320 con 0x55b2bc0392a0 2024-05-10T11:29:59.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.040+0000 7f0eaaffd700 10 client.17745 handle_command_reply: tid=0 2024-05-10T11:29:59.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.040+0000 7f0edc3f1b80 1 client.17745 shutdown 2024-05-10T11:29:59.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.040+0000 7f0edc3f1b80 1 -- 172.21.15.100:0/4239740672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b2bc01fd50 msgr2=0x55b2bc02b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:59.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.040+0000 7f0edc3f1b80 1 --2- 172.21.15.100:0/4239740672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b2bc01fd50 0x55b2bc02b770 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc0e0750 tx=0x7f0ebc010f30 comp rx=0 tx=0).stop 2024-05-10T11:29:59.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.040+0000 7f0edc3f1b80 1 -- 172.21.15.100:0/4239740672 shutdown_connections 2024-05-10T11:29:59.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.040+0000 7f0edc3f1b80 1 --2- 172.21.15.100:0/4239740672 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55b2bc0392a0 0x55b2bc03b690 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:59.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.040+0000 7f0edc3f1b80 1 --2- 172.21.15.100:0/4239740672 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2bc025ad0 0x55b2bc029050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:59.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.040+0000 7f0edc3f1b80 1 --2- 172.21.15.100:0/4239740672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b2bc01fd50 0x55b2bc02b770 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:59.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.040+0000 7f0eaaffd700 0 client.17745 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:29:59.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.040+0000 7f0edc3f1b80 1 --2- 172.21.15.100:0/4239740672 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b2bc0251b0 0x55b2bc028b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:59.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.040+0000 7f0edc3f1b80 1 -- 172.21.15.100:0/4239740672 >> 172.21.15.100:0/4239740672 conn(0x55b2bbf4bb70 msgr2=0x55b2bc014b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:59.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.040+0000 7f0edc3f1b80 1 -- 172.21.15.100:0/4239740672 shutdown_connections 2024-05-10T11:29:59.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.040+0000 7f0edc3f1b80 1 -- 172.21.15.100:0/4239740672 wait complete. 2024-05-10T11:29:59.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.040+0000 7f0edc3f1b80 20 client.17745 trim_cache size 0 max 16384 2024-05-10T11:29:59.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.055+0000 7f0edc3f1b80 1 Processor -- start 2024-05-10T11:29:59.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.056+0000 7f0edc3f1b80 1 -- start start 2024-05-10T11:29:59.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.056+0000 7f0edc3f1b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2bc01fd50 0x55b2bc01f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:59.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.056+0000 7f0edc3f1b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b2bc0251b0 0x55b2bc026770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:59.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.056+0000 7f0edc3f1b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b2bc025ad0 0x55b2bc0354e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:59.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.056+0000 7f0edc3f1b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0ecc1b3a90 con 0x55b2bc0251b0 2024-05-10T11:29:59.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.056+0000 7f0edc3f1b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0ecc1b1320 con 0x55b2bc025ad0 2024-05-10T11:29:59.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.056+0000 7f0edc3f1b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0ecc1af170 con 0x55b2bc01fd50 2024-05-10T11:29:59.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.056+0000 7f0ecaffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b2bc0251b0 0x55b2bc026770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:59.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.056+0000 7f0ecaffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b2bc0251b0 0x55b2bc026770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36008/0 (socket says 172.21.15.100:36008) 2024-05-10T11:29:59.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.056+0000 7f0ecaffd700 1 -- 172.21.15.100:0/1939856291 learned_addr learned my addr 172.21.15.100:0/1939856291 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:29:59.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.056+0000 7f0ecbfff700 1 --2- 172.21.15.100:0/1939856291 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b2bc025ad0 0x55b2bc0354e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:59.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.056+0000 7f0ecb7fe700 1 --2- 172.21.15.100:0/1939856291 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2bc01fd50 0x55b2bc01f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:59.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.056+0000 7f0ecaffd700 1 -- 172.21.15.100:0/1939856291 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2bc01fd50 msgr2=0x55b2bc01f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:59.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.056+0000 7f0ecaffd700 1 --2- 172.21.15.100:0/1939856291 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2bc01fd50 0x55b2bc01f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:59.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.056+0000 7f0ecaffd700 1 -- 172.21.15.100:0/1939856291 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b2bc025ad0 msgr2=0x55b2bc0354e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:59.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.056+0000 7f0ecaffd700 1 --2- 172.21.15.100:0/1939856291 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b2bc025ad0 0x55b2bc0354e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:59.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.056+0000 7f0ecaffd700 1 -- 172.21.15.100:0/1939856291 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2bc02c330 con 0x55b2bc0251b0 2024-05-10T11:29:59.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.056+0000 7f0ecaffd700 1 --2- 172.21.15.100:0/1939856291 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b2bc0251b0 0x55b2bc026770 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x55b2bc01b040 tx=0x7f0ec001cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4a66dec825f2611 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:59.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.056+0000 7f0eb17fa700 1 -- 172.21.15.100:0/1939856291 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec001ce10 con 0x55b2bc0251b0 2024-05-10T11:29:59.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.057+0000 7f0eb17fa700 1 -- 172.21.15.100:0/1939856291 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0ec000eba0 con 0x55b2bc0251b0 2024-05-10T11:29:59.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.057+0000 7f0eb17fa700 1 -- 172.21.15.100:0/1939856291 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec0003a20 con 0x55b2bc0251b0 2024-05-10T11:29:59.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.057+0000 7f0edc3f1b80 1 -- 172.21.15.100:0/1939856291 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b2bc0251b0 msgr2=0x55b2bc026770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:59.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.057+0000 7f0edc3f1b80 1 --2- 172.21.15.100:0/1939856291 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b2bc0251b0 0x55b2bc026770 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x55b2bc01b040 tx=0x7f0ec001cda0 comp rx=0 tx=0).stop 2024-05-10T11:29:59.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.057+0000 7f0edc3f1b80 1 -- 172.21.15.100:0/1939856291 shutdown_connections 2024-05-10T11:29:59.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.057+0000 7f0edc3f1b80 1 --2- 172.21.15.100:0/1939856291 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b2bc025ad0 0x55b2bc0354e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:59.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.057+0000 7f0edc3f1b80 1 --2- 172.21.15.100:0/1939856291 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2bc01fd50 0x55b2bc01f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:59.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.057+0000 7f0edc3f1b80 1 --2- 172.21.15.100:0/1939856291 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b2bc0251b0 0x55b2bc026770 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:59.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.057+0000 7f0edc3f1b80 1 -- 172.21.15.100:0/1939856291 >> 172.21.15.100:0/1939856291 conn(0x55b2bbf4bb70 msgr2=0x55b2bc0283c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:59.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.057+0000 7f0edc3f1b80 1 -- 172.21.15.100:0/1939856291 shutdown_connections 2024-05-10T11:29:59.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.057+0000 7f0edc3f1b80 1 -- 172.21.15.100:0/1939856291 wait complete. 2024-05-10T11:29:59.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.058+0000 7f0edc3f1b80 1 -- start start 2024-05-10T11:29:59.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.058+0000 7f0edc3f1b80 1 Processor -- start 2024-05-10T11:29:59.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.058+0000 7f0edc3f1b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2bc01fd50 0x55b2bc031a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:59.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.058+0000 7f0edc3f1b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b2bc0251b0 0x55b2bc031f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:59.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.058+0000 7f0edc3f1b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b2bc025ad0 0x55b2bc040350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:59.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.058+0000 7f0edc3f1b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0ecc1af7e0 con 0x55b2bc0251b0 2024-05-10T11:29:59.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.058+0000 7f0edc3f1b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0ecc1b0c50 con 0x55b2bc025ad0 2024-05-10T11:29:59.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.058+0000 7f0edc3f1b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0ecc1afeb0 con 0x55b2bc01fd50 2024-05-10T11:29:59.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.058+0000 7f0edc3f1b80 10 client.0 _ll_register_callbacks cb 0x55b2bbf055b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:29:59.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.058+0000 7f0ecaffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b2bc0251b0 0x55b2bc031f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:59.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.058+0000 7f0ecb7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2bc01fd50 0x55b2bc031a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:59.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.058+0000 7f0ecb7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2bc01fd50 0x55b2bc031a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:53950/0 (socket says 172.21.15.100:53950) 2024-05-10T11:29:59.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.058+0000 7f0ecb7fe700 1 -- 172.21.15.100:0/3138458439 learned_addr learned my addr 172.21.15.100:0/3138458439 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:29:59.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.058+0000 7f0ecbfff700 1 --2- 172.21.15.100:0/3138458439 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b2bc025ad0 0x55b2bc040350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:59.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.059+0000 7f0ecaffd700 1 -- 172.21.15.100:0/3138458439 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2bc01fd50 msgr2=0x55b2bc031a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:59.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.059+0000 7f0ecaffd700 1 --2- 172.21.15.100:0/3138458439 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2bc01fd50 0x55b2bc031a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:59.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.059+0000 7f0ecaffd700 1 -- 172.21.15.100:0/3138458439 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b2bc025ad0 msgr2=0x55b2bc040350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:59.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.059+0000 7f0ecaffd700 1 --2- 172.21.15.100:0/3138458439 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b2bc025ad0 0x55b2bc040350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:59.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.059+0000 7f0ecaffd700 1 -- 172.21.15.100:0/3138458439 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2bc040b60 con 0x55b2bc0251b0 2024-05-10T11:29:59.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.059+0000 7f0ecbfff700 1 --2- 172.21.15.100:0/3138458439 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b2bc025ad0 0x55b2bc040350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:29:59.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.059+0000 7f0ecaffd700 1 --2- 172.21.15.100:0/3138458439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b2bc0251b0 0x55b2bc031f80 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f0ec0035040 tx=0x7f0ec0009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c45d7b9e994b313a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:59.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.059+0000 7f0eaaffd700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:29:59.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.059+0000 7f0eaaffd700 1 -- 172.21.15.100:0/3138458439 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec001ce10 con 0x55b2bc0251b0 2024-05-10T11:29:59.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.059+0000 7f0eaaffd700 1 -- 172.21.15.100:0/3138458439 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f0ec0009040 con 0x55b2bc0251b0 2024-05-10T11:29:59.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.059+0000 7f0edc3f1b80 1 -- 172.21.15.100:0/3138458439 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ec000eba0 con 0x55b2bc0251b0 2024-05-10T11:29:59.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.059+0000 7f0eaaffd700 1 -- 172.21.15.100:0/3138458439 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f0ec00050a0 con 0x55b2bc0251b0 2024-05-10T11:29:59.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.059+0000 7f0edc3f1b80 10 client.17751 fetch_fsmap learned FSMap version 33 2024-05-10T11:29:59.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.059+0000 7f0edc3f1b80 1 -- 172.21.15.100:0/3138458439 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b2bc0a37b0 con 0x55b2bc0251b0 2024-05-10T11:29:59.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.059+0000 7f0eaaffd700 1 -- 172.21.15.100:0/3138458439 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0ec000eba0 con 0x55b2bc0251b0 2024-05-10T11:29:59.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.060+0000 7f0eaaffd700 1 -- 172.21.15.100:0/3138458439 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f0ec001c530 con 0x55b2bc0251b0 2024-05-10T11:29:59.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.060+0000 7f0edc3f1b80 10 client.17751 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T11:29:59.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.060+0000 7f0edc3f1b80 10 client.17751 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:29:59.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.060+0000 7f0edc3f1b80 20 client.17751 populate_metadata read hostname 'smithi100' 2024-05-10T11:29:59.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.060+0000 7f0edc3f1b80 1 --2- 172.21.15.100:0/3138458439 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55b2bc0a3ba0 0x55b2bc0a5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:59.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.060+0000 7f0edc3f1b80 4 client.17751 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:29:59.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.060+0000 7f0edc3f1b80 1 -- 172.21.15.100:0/3138458439 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b2bbf793c0 con 0x55b2bc0a3ba0 2024-05-10T11:29:59.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.060+0000 7f0ecb7fe700 1 --2- 172.21.15.100:0/3138458439 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55b2bc0a3ba0 0x55b2bc0a5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:59.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.060+0000 7f0eaaffd700 10 client.17751 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:29:59.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.061+0000 7f0ecb7fe700 1 --2- 172.21.15.100:0/3138458439 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55b2bc0a3ba0 0x55b2bc0a5fc0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c05129e39fe8dc05 server_cookie=29eb6c11965adfdb in_seq=0 out_seq=0 2024-05-10T11:29:59.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.061+0000 7f0eaaffd700 1 -- 172.21.15.100:0/3138458439 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b2bbf793c0 con 0x55b2bc0a3ba0 2024-05-10T11:29:59.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.061+0000 7f0eaaffd700 10 client.17751 handle_command_reply: tid=0 2024-05-10T11:29:59.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.061+0000 7f0edc3f1b80 1 client.17751 shutdown 2024-05-10T11:29:59.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.061+0000 7f0edc3f1b80 1 -- 172.21.15.100:0/3138458439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b2bc0251b0 msgr2=0x55b2bc031f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:59.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.061+0000 7f0edc3f1b80 1 --2- 172.21.15.100:0/3138458439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b2bc0251b0 0x55b2bc031f80 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f0ec0035040 tx=0x7f0ec0009360 comp rx=0 tx=0).stop 2024-05-10T11:29:59.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.061+0000 7f0edc3f1b80 1 -- 172.21.15.100:0/3138458439 shutdown_connections 2024-05-10T11:29:59.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.061+0000 7f0edc3f1b80 1 --2- 172.21.15.100:0/3138458439 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55b2bc0a3ba0 0x55b2bc0a5fc0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:59.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.061+0000 7f0edc3f1b80 1 --2- 172.21.15.100:0/3138458439 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b2bc025ad0 0x55b2bc040350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:59.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.061+0000 7f0edc3f1b80 1 --2- 172.21.15.100:0/3138458439 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2bc01fd50 0x55b2bc031a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:59.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.061+0000 7f0edc3f1b80 1 --2- 172.21.15.100:0/3138458439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b2bc0251b0 0x55b2bc031f80 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:59.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.061+0000 7f0edc3f1b80 1 -- 172.21.15.100:0/3138458439 >> 172.21.15.100:0/3138458439 conn(0x55b2bbf4bb70 msgr2=0x55b2bc028bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:59.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.061+0000 7f0eaaffd700 0 client.17751 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:29:59.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.061+0000 7f0edc3f1b80 1 -- 172.21.15.100:0/3138458439 shutdown_connections 2024-05-10T11:29:59.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.061+0000 7f0edc3f1b80 1 -- 172.21.15.100:0/3138458439 wait complete. 2024-05-10T11:29:59.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.062+0000 7f0edc3f1b80 20 client.17751 trim_cache size 0 max 16384 2024-05-10T11:29:59.067 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:29:59.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.062+0000 7f0ed1b5e700 1 -- 172.21.15.100:0/2561369619 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0eb40b2900 msgr2=0x7f0eb40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:59.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.062+0000 7f0ed1b5e700 1 --2- 172.21.15.100:0/2561369619 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0eb40b2900 0x7f0eb40b4dc0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc07ad30 tx=0x7f0ec0009260 comp rx=0 tx=0).stop 2024-05-10T11:29:59.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.062+0000 7f0ed1b5e700 1 -- 172.21.15.100:0/2561369619 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ecc103540 msgr2=0x7f0ecc0757a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:59.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.062+0000 7f0ed1b5e700 1 --2- 172.21.15.100:0/2561369619 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ecc103540 0x7f0ecc0757a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc006010 tx=0x7f0ebc004270 comp rx=0 tx=0).stop 2024-05-10T11:29:59.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.062+0000 7f0ed1b5e700 1 -- 172.21.15.100:0/2561369619 shutdown_connections 2024-05-10T11:29:59.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.062+0000 7f0ed1b5e700 1 --2- 172.21.15.100:0/2561369619 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0eb40b2900 0x7f0eb40b4dc0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:59.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.062+0000 7f0ed1b5e700 1 --2- 172.21.15.100:0/2561369619 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ecc1095e0 0x7f0ecc076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:59.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.062+0000 7f0ed1b5e700 1 --2- 172.21.15.100:0/2561369619 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ecc103540 0x7f0ecc0757a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:59.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.062+0000 7f0ed1b5e700 1 --2- 172.21.15.100:0/2561369619 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ecc103f00 0x7f0ecc075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:59.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.062+0000 7f0ed1b5e700 1 -- 172.21.15.100:0/2561369619 >> 172.21.15.100:0/2561369619 conn(0x7f0ecc07a650 msgr2=0x7f0ecc1022b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:59.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.062+0000 7f0ed1b5e700 1 -- 172.21.15.100:0/2561369619 shutdown_connections 2024-05-10T11:29:59.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:29:59.062+0000 7f0ed1b5e700 1 -- 172.21.15.100:0/2561369619 wait complete. 2024-05-10T11:29:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.467+0000 7f42f6f2b640 1 -- 172.21.15.100:0/3557836649 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42f0101e60 msgr2=0x7f42f0102280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.467+0000 7f42f6f2b640 1 --2- 172.21.15.100:0/3557836649 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42f0101e60 0x7f42f0102280 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f42e4009900 tx=0x7f42e402a410 comp rx=0 tx=0).stop 2024-05-10T11:29:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.467+0000 7f42f6f2b640 1 -- 172.21.15.100:0/3557836649 shutdown_connections 2024-05-10T11:29:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.467+0000 7f42f6f2b640 1 --2- 172.21.15.100:0/3557836649 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f42f0103530 0x7f42f0109dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.467+0000 7f42f6f2b640 1 --2- 172.21.15.100:0/3557836649 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42f0102b70 0x7f42f0102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.467+0000 7f42f6f2b640 1 --2- 172.21.15.100:0/3557836649 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42f0101e60 0x7f42f0102280 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.467+0000 7f42f6f2b640 1 -- 172.21.15.100:0/3557836649 >> 172.21.15.100:0/3557836649 conn(0x7f42f00fc650 msgr2=0x7f42f00fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.467+0000 7f42f6f2b640 1 -- 172.21.15.100:0/3557836649 shutdown_connections 2024-05-10T11:29:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.467+0000 7f42f6f2b640 1 -- 172.21.15.100:0/3557836649 wait complete. 2024-05-10T11:29:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.468+0000 7f42f6f2b640 1 Processor -- start 2024-05-10T11:29:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.468+0000 7f42f6f2b640 1 -- start start 2024-05-10T11:29:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.468+0000 7f42f6f2b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42f0101e60 0x7f42f0197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.468+0000 7f42f6f2b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42f0102b70 0x7f42f0197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.468+0000 7f42f6f2b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f42f0103530 0x7f42f019efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.468+0000 7f42f6f2b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f42f010d330 con 0x7f42f0101e60 2024-05-10T11:29:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.468+0000 7f42f6f2b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f42f010d1b0 con 0x7f42f0103530 2024-05-10T11:29:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.468+0000 7f42f6f2b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f42f010d4b0 con 0x7f42f0102b70 2024-05-10T11:29:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.468+0000 7f42f4cc7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42f0101e60 0x7f42f0197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.468+0000 7f42f4cc7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42f0101e60 0x7f42f0197a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36024/0 (socket says 172.21.15.100:36024) 2024-05-10T11:29:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.468+0000 7f42f4cc7640 1 -- 172.21.15.100:0/323974780 learned_addr learned my addr 172.21.15.100:0/323974780 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:29:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.468+0000 7f42effff640 1 --2- 172.21.15.100:0/323974780 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42f0102b70 0x7f42f0197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.468+0000 7f42f54c8640 1 --2- 172.21.15.100:0/323974780 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f42f0103530 0x7f42f019efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.469+0000 7f42f4cc7640 1 -- 172.21.15.100:0/323974780 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42f0102b70 msgr2=0x7f42f0197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.469+0000 7f42f4cc7640 1 --2- 172.21.15.100:0/323974780 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42f0102b70 0x7f42f0197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.469+0000 7f42f4cc7640 1 -- 172.21.15.100:0/323974780 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f42f0103530 msgr2=0x7f42f019efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.469+0000 7f42f4cc7640 1 --2- 172.21.15.100:0/323974780 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f42f0103530 0x7f42f019efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.469+0000 7f42f4cc7640 1 -- 172.21.15.100:0/323974780 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42f019f650 con 0x7f42f0101e60 2024-05-10T11:29:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.469+0000 7f42f4cc7640 1 --2- 172.21.15.100:0/323974780 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42f0101e60 0x7f42f0197a30 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f42e4005ab0 tx=0x7f42e40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.469+0000 7f42edffb640 1 -- 172.21.15.100:0/323974780 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e403f030 con 0x7f42f0101e60 2024-05-10T11:29:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.469+0000 7f42f6f2b640 1 -- 172.21.15.100:0/323974780 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42f019f8e0 con 0x7f42f0101e60 2024-05-10T11:29:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.469+0000 7f42edffb640 1 -- 172.21.15.100:0/323974780 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f42e4032e10 con 0x7f42f0101e60 2024-05-10T11:29:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.469+0000 7f42edffb640 1 -- 172.21.15.100:0/323974780 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e4035070 con 0x7f42f0101e60 2024-05-10T11:29:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.469+0000 7f42f6f2b640 1 -- 172.21.15.100:0/323974780 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42f019fdd0 con 0x7f42f0101e60 2024-05-10T11:29:59.479 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.472+0000 7f42edffb640 1 -- 172.21.15.100:0/323974780 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f42e4042030 con 0x7f42f0101e60 2024-05-10T11:29:59.479 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.472+0000 7f42f9c46740 1 Processor -- start 2024-05-10T11:29:59.479 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.473+0000 7f42f9c46740 1 -- start start 2024-05-10T11:29:59.479 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.473+0000 7f42f9c46740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596c7fb8fc0 0x5596c7fb93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:59.479 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.473+0000 7f42f9c46740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5596c7fb99b0 0x5596c7fbe4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:59.479 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.473+0000 7f42f9c46740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596c7fbeb20 0x5596c7fc0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:59.479 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.473+0000 7f42f9c46740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5596c7f1b5d0 con 0x5596c7fb8fc0 2024-05-10T11:29:59.479 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.473+0000 7f42f9c46740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5596c7ed4890 con 0x5596c7fbeb20 2024-05-10T11:29:59.479 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.473+0000 7f42f9c46740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5596c7f17590 con 0x5596c7fb99b0 2024-05-10T11:29:59.479 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.473+0000 7f42f54c8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596c7fb8fc0 0x5596c7fb93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:59.479 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.473+0000 7f42f54c8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596c7fb8fc0 0x5596c7fb93e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36038/0 (socket says 172.21.15.100:36038) 2024-05-10T11:29:59.480 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.473+0000 7f42f54c8640 1 -- 172.21.15.100:0/1043049410 learned_addr learned my addr 172.21.15.100:0/1043049410 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:29:59.480 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.473+0000 7f42effff640 1 --2- 172.21.15.100:0/1043049410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596c7fbeb20 0x5596c7fc0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:59.480 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.473+0000 7f42f4cc7640 1 --2- 172.21.15.100:0/1043049410 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5596c7fb99b0 0x5596c7fbe4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:59.480 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.473+0000 7f42f54c8640 1 -- 172.21.15.100:0/1043049410 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5596c7fb99b0 msgr2=0x5596c7fbe4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:59.480 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.473+0000 7f42f54c8640 1 --2- 172.21.15.100:0/1043049410 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5596c7fb99b0 0x5596c7fbe4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:59.480 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.473+0000 7f42f54c8640 1 -- 172.21.15.100:0/1043049410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596c7fbeb20 msgr2=0x5596c7fc0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:59.480 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.473+0000 7f42f54c8640 1 --2- 172.21.15.100:0/1043049410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596c7fbeb20 0x5596c7fc0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:59.480 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.473+0000 7f42f54c8640 1 -- 172.21.15.100:0/1043049410 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596c7c2d210 con 0x5596c7fb8fc0 2024-05-10T11:29:59.480 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.473+0000 7f42effff640 1 --2- 172.21.15.100:0/1043049410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596c7fbeb20 0x5596c7fc0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:29:59.480 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.473+0000 7f42edffb640 1 --2- 172.21.15.100:0/323974780 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f42e00b28a0 0x7f42e00b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:59.480 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.473+0000 7f42f54c8640 1 --2- 172.21.15.100:0/1043049410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596c7fb8fc0 0x5596c7fb93e0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f42e8009fd0 tx=0x7f42e80118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f698a50764a84d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:59.480 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.474+0000 7f42f4cc7640 1 --2- 172.21.15.100:0/323974780 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f42e00b28a0 0x7f42e00b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:59.480 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.474+0000 7f42daffd640 1 -- 172.21.15.100:0/1043049410 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e8004d20 con 0x5596c7fb8fc0 2024-05-10T11:29:59.481 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.474+0000 7f42daffd640 1 -- 172.21.15.100:0/1043049410 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f42e8004050 con 0x5596c7fb8fc0 2024-05-10T11:29:59.481 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.474+0000 7f42daffd640 1 -- 172.21.15.100:0/1043049410 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e8005260 con 0x5596c7fb8fc0 2024-05-10T11:29:59.481 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.474+0000 7f42edffb640 1 -- 172.21.15.100:0/323974780 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f42e4102970 con 0x7f42f0101e60 2024-05-10T11:29:59.481 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.474+0000 7f42f9c46740 1 -- 172.21.15.100:0/1043049410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596c7fb8fc0 msgr2=0x5596c7fb93e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:59.481 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.474+0000 7f42f4cc7640 1 --2- 172.21.15.100:0/323974780 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f42e00b28a0 0x7f42e00b4d60 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f42e4100480 tx=0x7f42e4108550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:59.481 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.474+0000 7f42f9c46740 1 --2- 172.21.15.100:0/1043049410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596c7fb8fc0 0x5596c7fb93e0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f42e8009fd0 tx=0x7f42e80118d0 comp rx=0 tx=0).stop 2024-05-10T11:29:59.481 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.475+0000 7f42f9c46740 1 -- 172.21.15.100:0/1043049410 shutdown_connections 2024-05-10T11:29:59.481 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.475+0000 7f42f9c46740 1 --2- 172.21.15.100:0/1043049410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596c7fbeb20 0x5596c7fc0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:59.481 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.475+0000 7f42f9c46740 1 --2- 172.21.15.100:0/1043049410 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5596c7fb99b0 0x5596c7fbe4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:59.481 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.475+0000 7f42f9c46740 1 --2- 172.21.15.100:0/1043049410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596c7fb8fc0 0x5596c7fb93e0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:59.481 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.475+0000 7f42f9c46740 1 -- 172.21.15.100:0/1043049410 >> 172.21.15.100:0/1043049410 conn(0x5596c7fb2420 msgr2=0x5596c7fb2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:59.481 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.475+0000 7f42f9c46740 1 -- 172.21.15.100:0/1043049410 shutdown_connections 2024-05-10T11:29:59.482 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.475+0000 7f42f9c46740 1 -- 172.21.15.100:0/1043049410 wait complete. 2024-05-10T11:29:59.482 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.475+0000 7f42f9c46740 1 -- start start 2024-05-10T11:29:59.482 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.475+0000 7f42f9c46740 1 Processor -- start 2024-05-10T11:29:59.482 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.475+0000 7f42f9c46740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5596c7fb8fc0 0x5596c7fc38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:59.482 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.475+0000 7f42f9c46740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596c7fb99b0 0x5596c7fc5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:59.482 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.475+0000 7f42f9c46740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596c7fbeb20 0x5596c7fc8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:59.482 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.475+0000 7f42f9c46740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5596c7fb6c50 con 0x5596c7fb99b0 2024-05-10T11:29:59.482 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.475+0000 7f42f9c46740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5596c7ed8370 con 0x5596c7fbeb20 2024-05-10T11:29:59.482 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.475+0000 7f42f9c46740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5596c7fb6dd0 con 0x5596c7fb8fc0 2024-05-10T11:29:59.482 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.475+0000 7f42f9c46740 10 client.0 _ll_register_callbacks cb 0x5596c7c2ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:29:59.482 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.476+0000 7f42effff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596c7fb99b0 0x5596c7fc5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:59.482 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.476+0000 7f42effff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596c7fb99b0 0x5596c7fc5df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36044/0 (socket says 172.21.15.100:36044) 2024-05-10T11:29:59.482 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.476+0000 7f42f54c8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5596c7fb8fc0 0x5596c7fc38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:59.483 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.476+0000 7f42effff640 1 -- 172.21.15.100:0/4192609844 learned_addr learned my addr 172.21.15.100:0/4192609844 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:29:59.483 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.476+0000 7f42f54c8640 1 --2- 172.21.15.100:0/4192609844 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596c7fbeb20 0x5596c7fc8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:59.483 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.476+0000 7f42effff640 1 -- 172.21.15.100:0/4192609844 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5596c7fb8fc0 msgr2=0x5596c7fc38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:59.483 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.476+0000 7f42effff640 1 --2- 172.21.15.100:0/4192609844 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5596c7fb8fc0 0x5596c7fc38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:59.483 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.476+0000 7f42effff640 1 -- 172.21.15.100:0/4192609844 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596c7fbeb20 msgr2=0x5596c7fc8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:59.483 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.476+0000 7f42effff640 1 --2- 172.21.15.100:0/4192609844 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596c7fbeb20 0x5596c7fc8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:59.483 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.476+0000 7f42effff640 1 -- 172.21.15.100:0/4192609844 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596c7cf9620 con 0x5596c7fb99b0 2024-05-10T11:29:59.483 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.476+0000 7f42effff640 1 --2- 172.21.15.100:0/4192609844 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596c7fb99b0 0x5596c7fc5df0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f42dc009a30 tx=0x7f42dc009ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be3311a51ccd9aa1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:59.483 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.476+0000 7f42cffff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:29:59.483 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.476+0000 7f42cffff640 1 -- 172.21.15.100:0/4192609844 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42dc004050 con 0x5596c7fb99b0 2024-05-10T11:29:59.483 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.476+0000 7f42cffff640 1 -- 172.21.15.100:0/4192609844 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f42dc0041f0 con 0x5596c7fb99b0 2024-05-10T11:29:59.483 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.476+0000 7f42f9c46740 1 -- 172.21.15.100:0/4192609844 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42e8004050 con 0x5596c7fb99b0 2024-05-10T11:29:59.483 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.476+0000 7f42cffff640 1 -- 172.21.15.100:0/4192609844 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f42dc02b050 con 0x5596c7fb99b0 2024-05-10T11:29:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.476+0000 7f42f9c46740 10 client.17763 fetch_fsmap learned FSMap version 33 2024-05-10T11:29:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.476+0000 7f42f9c46740 1 -- 172.21.15.100:0/4192609844 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5596c7fcb630 con 0x5596c7fb99b0 2024-05-10T11:29:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.477+0000 7f42cffff640 1 -- 172.21.15.100:0/4192609844 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f42e8004050 con 0x5596c7fb99b0 2024-05-10T11:29:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.477+0000 7f42cffff640 1 -- 172.21.15.100:0/4192609844 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f42dc01d400 con 0x5596c7fb99b0 2024-05-10T11:29:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.477+0000 7f42f9c46740 10 client.17763 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T11:29:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.477+0000 7f42f9c46740 10 client.17763 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:29:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.477+0000 7f42f9c46740 20 client.17763 populate_metadata read hostname 'smithi100' 2024-05-10T11:29:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.477+0000 7f42f9c46740 1 --2- 172.21.15.100:0/4192609844 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5596c7fccd70 0x5596c7fcf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.477+0000 7f42f9c46740 4 client.17763 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:29:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.477+0000 7f42f9c46740 1 -- 172.21.15.100:0/4192609844 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596c7f291e0 con 0x5596c7fccd70 2024-05-10T11:29:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.477+0000 7f42f54c8640 1 --2- 172.21.15.100:0/4192609844 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5596c7fccd70 0x5596c7fcf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.477+0000 7f42cffff640 10 client.17763 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:29:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.477+0000 7f42f54c8640 1 --2- 172.21.15.100:0/4192609844 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5596c7fccd70 0x5596c7fcf160 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cd688be1dfd61cd server_cookie=461141ea15b91a1c in_seq=0 out_seq=0 2024-05-10T11:29:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.478+0000 7f42cffff640 1 -- 172.21.15.100:0/4192609844 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5596c7f291e0 con 0x5596c7fccd70 2024-05-10T11:29:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.478+0000 7f42cffff640 10 client.17763 handle_command_reply: tid=0 2024-05-10T11:29:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.478+0000 7f42f9c46740 1 client.17763 shutdown 2024-05-10T11:29:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.478+0000 7f42f9c46740 1 -- 172.21.15.100:0/4192609844 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596c7fb99b0 msgr2=0x5596c7fc5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.478+0000 7f42f9c46740 1 --2- 172.21.15.100:0/4192609844 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596c7fb99b0 0x5596c7fc5df0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f42dc009a30 tx=0x7f42dc009ef0 comp rx=0 tx=0).stop 2024-05-10T11:29:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.478+0000 7f42f9c46740 1 -- 172.21.15.100:0/4192609844 shutdown_connections 2024-05-10T11:29:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.478+0000 7f42f9c46740 1 --2- 172.21.15.100:0/4192609844 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5596c7fccd70 0x5596c7fcf160 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.478+0000 7f42f9c46740 1 --2- 172.21.15.100:0/4192609844 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596c7fbeb20 0x5596c7fc8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.478+0000 7f42f9c46740 1 --2- 172.21.15.100:0/4192609844 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596c7fb99b0 0x5596c7fc5df0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.478+0000 7f42f9c46740 1 --2- 172.21.15.100:0/4192609844 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5596c7fb8fc0 0x5596c7fc38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.478+0000 7f42f9c46740 1 -- 172.21.15.100:0/4192609844 >> 172.21.15.100:0/4192609844 conn(0x5596c7fb2420 msgr2=0x5596c7fb1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.478+0000 7f42cffff640 0 client.17763 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:29:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.478+0000 7f42f9c46740 1 -- 172.21.15.100:0/4192609844 shutdown_connections 2024-05-10T11:29:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.478+0000 7f42f9c46740 1 -- 172.21.15.100:0/4192609844 wait complete. 2024-05-10T11:29:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.479+0000 7f42f9c46740 20 client.17763 trim_cache size 0 max 16384 2024-05-10T11:29:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.495+0000 7f42f9c46740 1 Processor -- start 2024-05-10T11:29:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.495+0000 7f42f9c46740 1 -- start start 2024-05-10T11:29:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.495+0000 7f42f9c46740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5596c7fb8fc0 0x5596c7fb1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.495+0000 7f42f9c46740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596c7fb99b0 0x5596c7fb1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.495+0000 7f42f9c46740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596c7fbeb20 0x5596c7fb4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.495+0000 7f42f9c46740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f42f01ab9e0 con 0x5596c7fbeb20 2024-05-10T11:29:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.495+0000 7f42f9c46740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f42f01a92b0 con 0x5596c7fb99b0 2024-05-10T11:29:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.495+0000 7f42f9c46740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f42f01a7100 con 0x5596c7fb8fc0 2024-05-10T11:29:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.495+0000 7f42f54c8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596c7fbeb20 0x5596c7fb4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.496+0000 7f42f54c8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596c7fbeb20 0x5596c7fb4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36052/0 (socket says 172.21.15.100:36052) 2024-05-10T11:29:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.496+0000 7f42f54c8640 1 -- 172.21.15.100:0/4131764641 learned_addr learned my addr 172.21.15.100:0/4131764641 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:29:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.496+0000 7f42f54c8640 1 --2- 172.21.15.100:0/4131764641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5596c7fb8fc0 0x5596c7fb1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.496+0000 7f42effff640 1 --2- 172.21.15.100:0/4131764641 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596c7fb99b0 0x5596c7fb1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.496+0000 7f42effff640 1 -- 172.21.15.100:0/4131764641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5596c7fb8fc0 msgr2=0x5596c7fb1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.496+0000 7f42effff640 1 --2- 172.21.15.100:0/4131764641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5596c7fb8fc0 0x5596c7fb1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:59.498 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.496+0000 7f42effff640 1 -- 172.21.15.100:0/4131764641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596c7fbeb20 msgr2=0x5596c7fb4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:59.498 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.496+0000 7f42effff640 1 --2- 172.21.15.100:0/4131764641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596c7fbeb20 0x5596c7fb4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:59.498 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.496+0000 7f42effff640 1 -- 172.21.15.100:0/4131764641 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596c7fb2250 con 0x5596c7fb99b0 2024-05-10T11:29:59.498 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.496+0000 7f42f54c8640 1 --2- 172.21.15.100:0/4131764641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596c7fbeb20 0x5596c7fb4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:29:59.498 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.496+0000 7f42effff640 1 --2- 172.21.15.100:0/4131764641 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596c7fb99b0 0x5596c7fb1940 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f42dc01d990 tx=0x7f42dc005db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b28e7ec108102b22 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:59.498 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.496+0000 7f42da7fc640 1 -- 172.21.15.100:0/4131764641 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42dc01d400 con 0x5596c7fb99b0 2024-05-10T11:29:59.498 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.496+0000 7f42da7fc640 1 -- 172.21.15.100:0/4131764641 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f42dc034070 con 0x5596c7fb99b0 2024-05-10T11:29:59.498 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.496+0000 7f42da7fc640 1 -- 172.21.15.100:0/4131764641 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42dc003850 con 0x5596c7fb99b0 2024-05-10T11:29:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.497+0000 7f42f9c46740 1 -- 172.21.15.100:0/4131764641 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596c7fb99b0 msgr2=0x5596c7fb1940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.497+0000 7f42f9c46740 1 --2- 172.21.15.100:0/4131764641 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596c7fb99b0 0x5596c7fb1940 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f42dc01d990 tx=0x7f42dc005db0 comp rx=0 tx=0).stop 2024-05-10T11:29:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.497+0000 7f42f9c46740 1 -- 172.21.15.100:0/4131764641 shutdown_connections 2024-05-10T11:29:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.497+0000 7f42f9c46740 1 --2- 172.21.15.100:0/4131764641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596c7fbeb20 0x5596c7fb4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:59.502 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.497+0000 7f42f9c46740 1 --2- 172.21.15.100:0/4131764641 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596c7fb99b0 0x5596c7fb1940 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:59.502 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.497+0000 7f42f9c46740 1 --2- 172.21.15.100:0/4131764641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5596c7fb8fc0 0x5596c7fb1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:59.502 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.497+0000 7f42f9c46740 1 -- 172.21.15.100:0/4131764641 >> 172.21.15.100:0/4131764641 conn(0x5596c7fb2420 msgr2=0x5596c7fb0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:59.502 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.497+0000 7f42f9c46740 1 -- 172.21.15.100:0/4131764641 shutdown_connections 2024-05-10T11:29:59.502 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.497+0000 7f42f9c46740 1 -- 172.21.15.100:0/4131764641 wait complete. 2024-05-10T11:29:59.502 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.498+0000 7f42f9c46740 1 -- start start 2024-05-10T11:29:59.502 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.498+0000 7f42f9c46740 1 Processor -- start 2024-05-10T11:29:59.502 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.498+0000 7f42f9c46740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596c7fb8fc0 0x5596c7fc3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:59.502 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.498+0000 7f42f9c46740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5596c7fb99b0 0x5596c7fc4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:59.502 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.498+0000 7f42f9c46740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596c7fbeb20 0x5596c7fc6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:59.502 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.498+0000 7f42f9c46740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f42f01a7770 con 0x5596c7fbeb20 2024-05-10T11:29:59.502 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.498+0000 7f42f9c46740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f42f01a8be0 con 0x5596c7fb8fc0 2024-05-10T11:29:59.502 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.498+0000 7f42f9c46740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f42f01a7e40 con 0x5596c7fb99b0 2024-05-10T11:29:59.503 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.498+0000 7f42f9c46740 10 client.0 _ll_register_callbacks cb 0x7f42dc013960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:29:59.503 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.498+0000 7f42f54c8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596c7fbeb20 0x5596c7fc6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:59.503 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.498+0000 7f42f54c8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596c7fbeb20 0x5596c7fc6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36058/0 (socket says 172.21.15.100:36058) 2024-05-10T11:29:59.503 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.498+0000 7f42f54c8640 1 -- 172.21.15.100:0/1138241134 learned_addr learned my addr 172.21.15.100:0/1138241134 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:29:59.503 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.498+0000 7f42effff640 1 --2- 172.21.15.100:0/1138241134 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5596c7fb99b0 0x5596c7fc4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:59.503 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.498+0000 7f42f54c8640 1 --2- 172.21.15.100:0/1138241134 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596c7fb8fc0 0x5596c7fc3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:59.503 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.499+0000 7f42effff640 1 -- 172.21.15.100:0/1138241134 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596c7fb8fc0 msgr2=0x5596c7fc3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:59.503 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.499+0000 7f42effff640 1 --2- 172.21.15.100:0/1138241134 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596c7fb8fc0 0x5596c7fc3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:59.503 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.499+0000 7f42effff640 1 -- 172.21.15.100:0/1138241134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596c7fbeb20 msgr2=0x5596c7fc6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:29:59.503 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.499+0000 7f42effff640 1 --2- 172.21.15.100:0/1138241134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596c7fbeb20 0x5596c7fc6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:59.503 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.499+0000 7f42effff640 1 -- 172.21.15.100:0/1138241134 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596c7fc73c0 con 0x5596c7fb99b0 2024-05-10T11:29:59.503 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.499+0000 7f42f54c8640 1 --2- 172.21.15.100:0/1138241134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596c7fbeb20 0x5596c7fc6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:29:59.503 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.499+0000 7f42effff640 1 --2- 172.21.15.100:0/1138241134 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5596c7fb99b0 0x5596c7fc4070 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f42dc01d7a0 tx=0x7f42dc00ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7946f27edc9ada5b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:29:59.504 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.499+0000 7f42cffff640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:29:59.504 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.499+0000 7f42cffff640 1 -- 172.21.15.100:0/1138241134 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42dc03d060 con 0x5596c7fb99b0 2024-05-10T11:29:59.504 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.499+0000 7f42f9c46740 1 -- 172.21.15.100:0/1138241134 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42dc034070 con 0x5596c7fb99b0 2024-05-10T11:29:59.504 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.499+0000 7f42cffff640 1 -- 172.21.15.100:0/1138241134 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f42dc005310 con 0x5596c7fb99b0 2024-05-10T11:29:59.504 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.499+0000 7f42cffff640 1 -- 172.21.15.100:0/1138241134 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f42dc044050 con 0x5596c7fb99b0 2024-05-10T11:29:59.504 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.499+0000 7f42f9c46740 10 client.27220 fetch_fsmap learned FSMap version 33 2024-05-10T11:29:59.504 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.499+0000 7f42f9c46740 1 -- 172.21.15.100:0/1138241134 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5596c8037620 con 0x5596c7fb99b0 2024-05-10T11:29:59.504 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.500+0000 7f42cffff640 1 -- 172.21.15.100:0/1138241134 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f42dc034070 con 0x5596c7fb99b0 2024-05-10T11:29:59.504 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.500+0000 7f42cffff640 1 -- 172.21.15.100:0/1138241134 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f42dc003ca0 con 0x5596c7fb99b0 2024-05-10T11:29:59.504 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.500+0000 7f42f9c46740 10 client.27220 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T11:29:59.504 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.500+0000 7f42f9c46740 10 client.27220 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:29:59.504 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.500+0000 7f42f9c46740 20 client.27220 populate_metadata read hostname 'smithi100' 2024-05-10T11:29:59.504 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.500+0000 7f42f9c46740 1 --2- 172.21.15.100:0/1138241134 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5596c8037db0 0x5596c803a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:29:59.505 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.500+0000 7f42f9c46740 4 client.27220 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:29:59.505 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.500+0000 7f42f9c46740 1 -- 172.21.15.100:0/1138241134 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5596c7f23050 con 0x5596c8037db0 2024-05-10T11:29:59.505 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.500+0000 7f42f54c8640 1 --2- 172.21.15.100:0/1138241134 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5596c8037db0 0x5596c803a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:29:59.505 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.501+0000 7f42f54c8640 1 --2- 172.21.15.100:0/1138241134 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5596c8037db0 0x5596c803a1a0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adaf1770cd9e5903 server_cookie=1190edc6bd91e432 in_seq=0 out_seq=0 2024-05-10T11:29:59.505 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.501+0000 7f42cffff640 10 client.27220 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:29:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.530+0000 7f42cffff640 1 -- 172.21.15.100:0/1138241134 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5596c7f23050 con 0x5596c8037db0 2024-05-10T11:29:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.530+0000 7f42cffff640 10 client.27220 handle_command_reply: tid=0 2024-05-10T11:29:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.530+0000 7f42f9c46740 1 client.27220 shutdown 2024-05-10T11:29:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.530+0000 7f42f9c46740 1 -- 172.21.15.100:0/1138241134 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5596c7fb99b0 msgr2=0x5596c7fc4070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.530+0000 7f42f9c46740 1 --2- 172.21.15.100:0/1138241134 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5596c7fb99b0 0x5596c7fc4070 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f42dc01d7a0 tx=0x7f42dc00ef90 comp rx=0 tx=0).stop 2024-05-10T11:29:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.530+0000 7f42f9c46740 1 -- 172.21.15.100:0/1138241134 shutdown_connections 2024-05-10T11:29:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.530+0000 7f42f9c46740 1 --2- 172.21.15.100:0/1138241134 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5596c8037db0 0x5596c803a1a0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.530+0000 7f42f9c46740 1 --2- 172.21.15.100:0/1138241134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596c7fbeb20 0x5596c7fc6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.530+0000 7f42cffff640 0 client.27220 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:29:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.530+0000 7f42f9c46740 1 --2- 172.21.15.100:0/1138241134 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5596c7fb99b0 0x5596c7fc4070 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.530+0000 7f42f9c46740 1 --2- 172.21.15.100:0/1138241134 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596c7fb8fc0 0x5596c7fc3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.530+0000 7f42f9c46740 1 -- 172.21.15.100:0/1138241134 >> 172.21.15.100:0/1138241134 conn(0x5596c7fb2420 msgr2=0x5596c7fb1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:59.532 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.530+0000 7f42f9c46740 1 -- 172.21.15.100:0/1138241134 shutdown_connections 2024-05-10T11:29:59.532 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.530+0000 7f42f9c46740 1 -- 172.21.15.100:0/1138241134 wait complete. 2024-05-10T11:29:59.532 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.530+0000 7f42f9c46740 20 client.27220 trim_cache size 0 max 16384 2024-05-10T11:29:59.532 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:29:59.532 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:29:59.532 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:29:59.532 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:29:59.533 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.531+0000 7f42f6f2b640 1 -- 172.21.15.100:0/323974780 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f42e00b28a0 msgr2=0x7f42e00b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:59.533 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.531+0000 7f42f6f2b640 1 --2- 172.21.15.100:0/323974780 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f42e00b28a0 0x7f42e00b4d60 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f42e4100480 tx=0x7f42e4108550 comp rx=0 tx=0).stop 2024-05-10T11:29:59.533 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.531+0000 7f42f6f2b640 1 -- 172.21.15.100:0/323974780 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42f0101e60 msgr2=0x7f42f0197a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:29:59.533 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.531+0000 7f42f6f2b640 1 --2- 172.21.15.100:0/323974780 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42f0101e60 0x7f42f0197a30 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f42e4005ab0 tx=0x7f42e40042f0 comp rx=0 tx=0).stop 2024-05-10T11:29:59.533 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.531+0000 7f42f6f2b640 1 -- 172.21.15.100:0/323974780 shutdown_connections 2024-05-10T11:29:59.534 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.531+0000 7f42f6f2b640 1 --2- 172.21.15.100:0/323974780 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f42e00b28a0 0x7f42e00b4d60 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:59.534 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.531+0000 7f42f6f2b640 1 --2- 172.21.15.100:0/323974780 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f42f0103530 0x7f42f019efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:59.534 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.531+0000 7f42f6f2b640 1 --2- 172.21.15.100:0/323974780 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42f0102b70 0x7f42f0197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:59.534 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.531+0000 7f42f6f2b640 1 --2- 172.21.15.100:0/323974780 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42f0101e60 0x7f42f0197a30 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:29:59.534 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.531+0000 7f42f6f2b640 1 -- 172.21.15.100:0/323974780 >> 172.21.15.100:0/323974780 conn(0x7f42f00fc650 msgr2=0x7f42f0105ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:29:59.534 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.532+0000 7f42f6f2b640 1 -- 172.21.15.100:0/323974780 shutdown_connections 2024-05-10T11:29:59.534 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:29:59.532+0000 7f42f6f2b640 1 -- 172.21.15.100:0/323974780 wait complete. 2024-05-10T11:29:59.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:29:59 smithi100 ceph-mon[30296]: pgmap v769: 129 pgs: 129 active+clean; 120 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 1.2 MiB/s wr, 130 op/s 2024-05-10T11:29:59.982 INFO:tasks.workunit.client.0.smithi100.stdout: 1 302907 2.72 MB/sec execute 474 sec latency 25.782 ms 2024-05-10T11:30:00.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:29:59 smithi178 ceph-mon[33359]: pgmap v769: 129 pgs: 129 active+clean; 120 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 1.2 MiB/s wr, 130 op/s 2024-05-10T11:30:00.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:29:59 smithi188 ceph-mon[34075]: pgmap v769: 129 pgs: 129 active+clean; 120 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 1.2 MiB/s wr, 130 op/s 2024-05-10T11:30:00.970 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T11:30:00.970 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T11:30:00.982 INFO:tasks.workunit.client.0.smithi100.stdout: 1 303501 2.71 MB/sec execute 475 sec latency 22.067 ms 2024-05-10T11:30:01.250 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:30:01.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:30:00 smithi178 ceph-mon[33359]: overall HEALTH_OK 2024-05-10T11:30:01.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:30:00 smithi188 ceph-mon[34075]: overall HEALTH_OK 2024-05-10T11:30:01.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:30:00 smithi100 ceph-mon[30296]: overall HEALTH_OK 2024-05-10T11:30:01.982 INFO:tasks.workunit.client.0.smithi100.stdout: 1 304027 2.71 MB/sec execute 476 sec latency 16.947 ms 2024-05-10T11:30:02.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:30:01 smithi100 ceph-mon[30296]: pgmap v770: 129 pgs: 129 active+clean; 118 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 1.3 MiB/s wr, 166 op/s 2024-05-10T11:30:02.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:30:01 smithi178 ceph-mon[33359]: pgmap v770: 129 pgs: 129 active+clean; 118 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 1.3 MiB/s wr, 166 op/s 2024-05-10T11:30:02.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:30:01 smithi188 ceph-mon[34075]: pgmap v770: 129 pgs: 129 active+clean; 118 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 1.3 MiB/s wr, 166 op/s 2024-05-10T11:30:02.982 INFO:tasks.workunit.client.0.smithi100.stdout: 1 304811 2.72 MB/sec execute 477 sec latency 23.065 ms 2024-05-10T11:30:03.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.055+0000 7f441255f700 1 -- 172.21.15.100:0/3183852477 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f440c108410 msgr2=0x7f440c1087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:03.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.055+0000 7f441255f700 1 --2- 172.21.15.100:0/3183852477 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f440c108410 0x7f440c1087f0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f43fc009990 tx=0x7f43fc009ca0 comp rx=0 tx=0).stop 2024-05-10T11:30:03.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.055+0000 7f441255f700 1 -- 172.21.15.100:0/3183852477 shutdown_connections 2024-05-10T11:30:03.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.055+0000 7f441255f700 1 --2- 172.21.15.100:0/3183852477 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f440c102d60 0x7f440c10f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:03.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.055+0000 7f441255f700 1 --2- 172.21.15.100:0/3183852477 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f440c108410 0x7f440c1087f0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:03.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.055+0000 7f441255f700 1 --2- 172.21.15.100:0/3183852477 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f440c1023a0 0x7f440c102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:03.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.055+0000 7f441255f700 1 -- 172.21.15.100:0/3183852477 >> 172.21.15.100:0/3183852477 conn(0x7f440c079350 msgr2=0x7f440c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:03.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.055+0000 7f441255f700 1 -- 172.21.15.100:0/3183852477 shutdown_connections 2024-05-10T11:30:03.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.055+0000 7f441255f700 1 -- 172.21.15.100:0/3183852477 wait complete. 2024-05-10T11:30:03.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.056+0000 7f441255f700 1 Processor -- start 2024-05-10T11:30:03.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.056+0000 7f441255f700 1 -- start start 2024-05-10T11:30:03.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.057+0000 7f441255f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f440c1023a0 0x7f440c19e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:03.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.057+0000 7f441255f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f440c102d60 0x7f440c19eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:03.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.057+0000 7f441255f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f440c108410 0x7f440c198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:03.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.057+0000 7f441255f700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f440c1123e0 con 0x7f440c102d60 2024-05-10T11:30:03.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.057+0000 7f441255f700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f440c112260 con 0x7f440c108410 2024-05-10T11:30:03.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.057+0000 7f441255f700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f440c112560 con 0x7f440c1023a0 2024-05-10T11:30:03.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.057+0000 7f440b7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f440c102d60 0x7f440c19eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:03.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.057+0000 7f440bfff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f440c1023a0 0x7f440c19e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:03.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.057+0000 7f440bfff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f440c1023a0 0x7f440c19e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:54034/0 (socket says 172.21.15.100:54034) 2024-05-10T11:30:03.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.057+0000 7f440b7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f440c102d60 0x7f440c19eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36076/0 (socket says 172.21.15.100:36076) 2024-05-10T11:30:03.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.057+0000 7f440bfff700 1 -- 172.21.15.100:0/3088561786 learned_addr learned my addr 172.21.15.100:0/3088561786 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:30:03.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.057+0000 7f4410afc700 1 --2- 172.21.15.100:0/3088561786 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f440c108410 0x7f440c198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:03.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.057+0000 7f440bfff700 1 -- 172.21.15.100:0/3088561786 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f440c108410 msgr2=0x7f440c198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:03.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.057+0000 7f440bfff700 1 --2- 172.21.15.100:0/3088561786 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f440c108410 0x7f440c198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:03.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.057+0000 7f440bfff700 1 -- 172.21.15.100:0/3088561786 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f440c102d60 msgr2=0x7f440c19eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:03.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.057+0000 7f440bfff700 1 --2- 172.21.15.100:0/3088561786 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f440c102d60 0x7f440c19eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:03.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.057+0000 7f440bfff700 1 -- 172.21.15.100:0/3088561786 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f440c198f60 con 0x7f440c1023a0 2024-05-10T11:30:03.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.057+0000 7f440b7fe700 1 --2- 172.21.15.100:0/3088561786 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f440c102d60 0x7f440c19eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:30:03.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.057+0000 7f4410afc700 1 --2- 172.21.15.100:0/3088561786 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f440c108410 0x7f440c198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:30:03.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.057+0000 7f440bfff700 1 --2- 172.21.15.100:0/3088561786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f440c1023a0 0x7f440c19e600 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f43fc006010 tx=0x7f43fc00bfc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:03.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.057+0000 7f44097fa700 1 -- 172.21.15.100:0/3088561786 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43fc027070 con 0x7f440c1023a0 2024-05-10T11:30:03.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.057+0000 7f44097fa700 1 -- 172.21.15.100:0/3088561786 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f43fc022470 con 0x7f440c1023a0 2024-05-10T11:30:03.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.058+0000 7f441255f700 1 -- 172.21.15.100:0/3088561786 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f440c1991f0 con 0x7f440c1023a0 2024-05-10T11:30:03.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.058+0000 7f44097fa700 1 -- 172.21.15.100:0/3088561786 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43fc017650 con 0x7f440c1023a0 2024-05-10T11:30:03.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.058+0000 7f441255f700 1 -- 172.21.15.100:0/3088561786 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f440c1a5460 con 0x7f440c1023a0 2024-05-10T11:30:03.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.060+0000 7f44097fa700 1 -- 172.21.15.100:0/3088561786 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f43fc009d10 con 0x7f440c1023a0 2024-05-10T11:30:03.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.061+0000 7f44097fa700 1 --2- 172.21.15.100:0/3088561786 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f43f40b2880 0x7f43f40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:03.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.061+0000 7f440b7fe700 1 --2- 172.21.15.100:0/3088561786 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f43f40b2880 0x7f43f40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:03.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.061+0000 7f44097fa700 1 -- 172.21.15.100:0/3088561786 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f43fc0e3130 con 0x7f440c1023a0 2024-05-10T11:30:03.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.061+0000 7f440b7fe700 1 --2- 172.21.15.100:0/3088561786 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f43f40b2880 0x7f43f40b4d40 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f440c079a90 tx=0x7f4400009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:03.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.062+0000 7f441cdf2b80 1 Processor -- start 2024-05-10T11:30:03.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.062+0000 7f441cdf2b80 1 -- start start 2024-05-10T11:30:03.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.062+0000 7f441cdf2b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db3f2ccc30 0x55db3f2cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:03.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.062+0000 7f441cdf2b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db3f2cb370 0x55db3f2cb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:03.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.062+0000 7f441cdf2b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db3f2cbd30 0x55db3f2dba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:03.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.062+0000 7f441cdf2b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55db3f21d8e0 con 0x55db3f2cbd30 2024-05-10T11:30:03.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.062+0000 7f441cdf2b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55db3f1bc840 con 0x55db3f2cb370 2024-05-10T11:30:03.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.062+0000 7f441cdf2b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55db3f2198a0 con 0x55db3f2ccc30 2024-05-10T11:30:03.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.062+0000 7f4410afc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db3f2cbd30 0x55db3f2dba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:03.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.062+0000 7f440bfff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db3f2ccc30 0x55db3f2cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:03.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.062+0000 7f440bfff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db3f2ccc30 0x55db3f2cd010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:54046/0 (socket says 172.21.15.100:54046) 2024-05-10T11:30:03.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.062+0000 7f4410afc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db3f2cbd30 0x55db3f2dba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36080/0 (socket says 172.21.15.100:36080) 2024-05-10T11:30:03.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.062+0000 7f440bfff700 1 -- 172.21.15.100:0/1457724538 learned_addr learned my addr 172.21.15.100:0/1457724538 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:30:03.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.062+0000 7f440b7fe700 1 --2- 172.21.15.100:0/1457724538 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db3f2cb370 0x55db3f2cb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:03.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.062+0000 7f440bfff700 1 -- 172.21.15.100:0/1457724538 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db3f2cb370 msgr2=0x55db3f2cb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:03.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.062+0000 7f440bfff700 1 --2- 172.21.15.100:0/1457724538 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db3f2cb370 0x55db3f2cb7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:03.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.062+0000 7f440bfff700 1 -- 172.21.15.100:0/1457724538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db3f2cbd30 msgr2=0x55db3f2dba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:03.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.062+0000 7f440bfff700 1 --2- 172.21.15.100:0/1457724538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db3f2cbd30 0x55db3f2dba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:03.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.062+0000 7f440bfff700 1 -- 172.21.15.100:0/1457724538 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db3f16a950 con 0x55db3f2ccc30 2024-05-10T11:30:03.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.062+0000 7f4410afc700 1 --2- 172.21.15.100:0/1457724538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db3f2cbd30 0x55db3f2dba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:30:03.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.062+0000 7f440bfff700 1 --2- 172.21.15.100:0/1457724538 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db3f2ccc30 0x55db3f2cd010 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f43fc0aa410 tx=0x7f43fc0aa060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=118aa85106366e0f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:03.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.063+0000 7f43f27fc700 1 -- 172.21.15.100:0/1457724538 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43fc0ae830 con 0x55db3f2ccc30 2024-05-10T11:30:03.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.063+0000 7f43f27fc700 1 -- 172.21.15.100:0/1457724538 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f43fc0b3250 con 0x55db3f2ccc30 2024-05-10T11:30:03.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.063+0000 7f43f27fc700 1 -- 172.21.15.100:0/1457724538 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43fc0b3020 con 0x55db3f2ccc30 2024-05-10T11:30:03.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.063+0000 7f441cdf2b80 1 -- 172.21.15.100:0/1457724538 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db3f2ccc30 msgr2=0x55db3f2cd010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:03.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.063+0000 7f441cdf2b80 1 --2- 172.21.15.100:0/1457724538 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db3f2ccc30 0x55db3f2cd010 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f43fc0aa410 tx=0x7f43fc0aa060 comp rx=0 tx=0).stop 2024-05-10T11:30:03.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.063+0000 7f441cdf2b80 1 -- 172.21.15.100:0/1457724538 shutdown_connections 2024-05-10T11:30:03.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.063+0000 7f441cdf2b80 1 --2- 172.21.15.100:0/1457724538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db3f2cbd30 0x55db3f2dba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:03.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.063+0000 7f441cdf2b80 1 --2- 172.21.15.100:0/1457724538 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db3f2ccc30 0x55db3f2cd010 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:03.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.063+0000 7f441cdf2b80 1 --2- 172.21.15.100:0/1457724538 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db3f2cb370 0x55db3f2cb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:03.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.063+0000 7f441cdf2b80 1 -- 172.21.15.100:0/1457724538 >> 172.21.15.100:0/1457724538 conn(0x55db3f1f7180 msgr2=0x55db3f2c5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:03.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.064+0000 7f441cdf2b80 1 -- 172.21.15.100:0/1457724538 shutdown_connections 2024-05-10T11:30:03.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.064+0000 7f441cdf2b80 1 -- 172.21.15.100:0/1457724538 wait complete. 2024-05-10T11:30:03.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.064+0000 7f441cdf2b80 1 -- start start 2024-05-10T11:30:03.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.064+0000 7f441cdf2b80 1 Processor -- start 2024-05-10T11:30:03.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.064+0000 7f441cdf2b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db3f2cb370 0x55db3f2db6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:03.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.064+0000 7f441cdf2b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db3f2cbd30 0x55db3f2d8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:03.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.064+0000 7f441cdf2b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db3f2ccc30 0x55db3f2d8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:03.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.064+0000 7f441cdf2b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55db3f2dc100 con 0x55db3f2cbd30 2024-05-10T11:30:03.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.064+0000 7f441cdf2b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55db3f183cb0 con 0x55db3f2cb370 2024-05-10T11:30:03.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.064+0000 7f441cdf2b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55db3f2dc280 con 0x55db3f2ccc30 2024-05-10T11:30:03.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.065+0000 7f441cdf2b80 10 client.0 _ll_register_callbacks cb 0x55db3f1b59a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:30:03.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.065+0000 7f440b7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db3f2cbd30 0x55db3f2d8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:03.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.065+0000 7f440b7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db3f2cbd30 0x55db3f2d8a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36090/0 (socket says 172.21.15.100:36090) 2024-05-10T11:30:03.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.065+0000 7f440b7fe700 1 -- 172.21.15.100:0/1446252989 learned_addr learned my addr 172.21.15.100:0/1446252989 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:30:03.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.065+0000 7f4410afc700 1 --2- 172.21.15.100:0/1446252989 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db3f2ccc30 0x55db3f2d8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:03.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.065+0000 7f440bfff700 1 --2- 172.21.15.100:0/1446252989 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db3f2cb370 0x55db3f2db6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:03.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.065+0000 7f440b7fe700 1 -- 172.21.15.100:0/1446252989 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db3f2ccc30 msgr2=0x55db3f2d8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:03.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.065+0000 7f440b7fe700 1 --2- 172.21.15.100:0/1446252989 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db3f2ccc30 0x55db3f2d8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:03.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.065+0000 7f440b7fe700 1 -- 172.21.15.100:0/1446252989 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db3f2cb370 msgr2=0x55db3f2db6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:03.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.065+0000 7f440b7fe700 1 --2- 172.21.15.100:0/1446252989 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db3f2cb370 0x55db3f2db6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:03.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.065+0000 7f440b7fe700 1 -- 172.21.15.100:0/1446252989 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db3f1b9710 con 0x55db3f2cbd30 2024-05-10T11:30:03.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.065+0000 7f440bfff700 1 --2- 172.21.15.100:0/1446252989 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db3f2cb370 0x55db3f2db6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:30:03.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.065+0000 7f4410afc700 1 --2- 172.21.15.100:0/1446252989 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db3f2ccc30 0x55db3f2d8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:30:03.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.065+0000 7f440b7fe700 1 --2- 172.21.15.100:0/1446252989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db3f2cbd30 0x55db3f2d8a50 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f4400003900 tx=0x7f44000047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6328aa4978e683b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:03.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.065+0000 7f43eb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:30:03.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.065+0000 7f43eb7fe700 1 -- 172.21.15.100:0/1446252989 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440000e940 con 0x55db3f2cbd30 2024-05-10T11:30:03.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.065+0000 7f43eb7fe700 1 -- 172.21.15.100:0/1446252989 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f440000eae0 con 0x55db3f2cbd30 2024-05-10T11:30:03.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.065+0000 7f441cdf2b80 1 -- 172.21.15.100:0/1446252989 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43fc0b3250 con 0x55db3f2cbd30 2024-05-10T11:30:03.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.066+0000 7f43eb7fe700 1 -- 172.21.15.100:0/1446252989 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f4400031050 con 0x55db3f2cbd30 2024-05-10T11:30:03.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.066+0000 7f441cdf2b80 10 client.17781 fetch_fsmap learned FSMap version 33 2024-05-10T11:30:03.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.066+0000 7f441cdf2b80 1 -- 172.21.15.100:0/1446252989 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55db3f2e6950 con 0x55db3f2cbd30 2024-05-10T11:30:03.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.066+0000 7f43eb7fe700 1 -- 172.21.15.100:0/1446252989 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f43fc0b3250 con 0x55db3f2cbd30 2024-05-10T11:30:03.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.066+0000 7f43eb7fe700 1 -- 172.21.15.100:0/1446252989 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f440001f400 con 0x55db3f2cbd30 2024-05-10T11:30:03.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.066+0000 7f441cdf2b80 10 client.17781 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T11:30:03.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.066+0000 7f441cdf2b80 10 client.17781 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:30:03.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.066+0000 7f441cdf2b80 20 client.17781 populate_metadata read hostname 'smithi100' 2024-05-10T11:30:03.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.066+0000 7f441cdf2b80 1 --2- 172.21.15.100:0/1446252989 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55db3f2e8fe0 0x55db3f2eb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:03.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.066+0000 7f441cdf2b80 4 client.17781 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:30:03.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.066+0000 7f441cdf2b80 1 -- 172.21.15.100:0/1446252989 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db3f2976d0 con 0x55db3f2e8fe0 2024-05-10T11:30:03.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.066+0000 7f440bfff700 1 --2- 172.21.15.100:0/1446252989 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55db3f2e8fe0 0x55db3f2eb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:03.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.067+0000 7f440bfff700 1 --2- 172.21.15.100:0/1446252989 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55db3f2e8fe0 0x55db3f2eb3f0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adadb583981b81da server_cookie=9a7dff58a218c4be in_seq=0 out_seq=0 2024-05-10T11:30:03.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.067+0000 7f43eb7fe700 10 client.17781 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:30:03.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.067+0000 7f43eb7fe700 1 -- 172.21.15.100:0/1446252989 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55db3f2976d0 con 0x55db3f2e8fe0 2024-05-10T11:30:03.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.067+0000 7f43eb7fe700 10 client.17781 handle_command_reply: tid=0 2024-05-10T11:30:03.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.068+0000 7f441cdf2b80 1 client.17781 shutdown 2024-05-10T11:30:03.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.068+0000 7f441cdf2b80 1 -- 172.21.15.100:0/1446252989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db3f2cbd30 msgr2=0x55db3f2d8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:03.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.068+0000 7f441cdf2b80 1 --2- 172.21.15.100:0/1446252989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db3f2cbd30 0x55db3f2d8a50 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f4400003900 tx=0x7f44000047c0 comp rx=0 tx=0).stop 2024-05-10T11:30:03.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.068+0000 7f441cdf2b80 1 -- 172.21.15.100:0/1446252989 shutdown_connections 2024-05-10T11:30:03.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.068+0000 7f441cdf2b80 1 --2- 172.21.15.100:0/1446252989 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55db3f2e8fe0 0x55db3f2eb3f0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:03.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.068+0000 7f441cdf2b80 1 --2- 172.21.15.100:0/1446252989 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db3f2ccc30 0x55db3f2d8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:03.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.068+0000 7f441cdf2b80 1 --2- 172.21.15.100:0/1446252989 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db3f2cb370 0x55db3f2db6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:03.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.068+0000 7f441cdf2b80 1 --2- 172.21.15.100:0/1446252989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db3f2cbd30 0x55db3f2d8a50 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:03.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.068+0000 7f441cdf2b80 1 -- 172.21.15.100:0/1446252989 >> 172.21.15.100:0/1446252989 conn(0x55db3f1f7180 msgr2=0x55db3f2c07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:03.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.068+0000 7f43eb7fe700 0 client.17781 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:30:03.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.068+0000 7f441cdf2b80 1 -- 172.21.15.100:0/1446252989 shutdown_connections 2024-05-10T11:30:03.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.068+0000 7f441cdf2b80 1 -- 172.21.15.100:0/1446252989 wait complete. 2024-05-10T11:30:03.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.068+0000 7f441cdf2b80 20 client.17781 trim_cache size 0 max 16384 2024-05-10T11:30:03.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.090+0000 7f441cdf2b80 1 Processor -- start 2024-05-10T11:30:03.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.090+0000 7f441cdf2b80 1 -- start start 2024-05-10T11:30:03.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.091+0000 7f441cdf2b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db3f2cb370 0x55db3f2d79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:03.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.091+0000 7f441cdf2b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db3f2cbd30 0x55db3f2e5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:03.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.091+0000 7f441cdf2b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db3f2ccc30 0x55db3f2d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:03.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.091+0000 7f441cdf2b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f440c1af680 con 0x55db3f2cbd30 2024-05-10T11:30:03.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.091+0000 7f441cdf2b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f440c1acf10 con 0x55db3f2cb370 2024-05-10T11:30:03.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.091+0000 7f441cdf2b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f440c1aad60 con 0x55db3f2ccc30 2024-05-10T11:30:03.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.091+0000 7f440b7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db3f2cbd30 0x55db3f2e5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:03.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.091+0000 7f440b7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db3f2cbd30 0x55db3f2e5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36102/0 (socket says 172.21.15.100:36102) 2024-05-10T11:30:03.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.091+0000 7f440b7fe700 1 -- 172.21.15.100:0/3043448844 learned_addr learned my addr 172.21.15.100:0/3043448844 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:30:03.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.091+0000 7f4410afc700 1 --2- 172.21.15.100:0/3043448844 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db3f2ccc30 0x55db3f2d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:03.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.091+0000 7f440bfff700 1 --2- 172.21.15.100:0/3043448844 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db3f2cb370 0x55db3f2d79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:03.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.091+0000 7f440b7fe700 1 -- 172.21.15.100:0/3043448844 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db3f2ccc30 msgr2=0x55db3f2d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:03.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.091+0000 7f440b7fe700 1 --2- 172.21.15.100:0/3043448844 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db3f2ccc30 0x55db3f2d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:03.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.091+0000 7f440b7fe700 1 -- 172.21.15.100:0/3043448844 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db3f2cb370 msgr2=0x55db3f2d79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:03.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.091+0000 7f440b7fe700 1 --2- 172.21.15.100:0/3043448844 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db3f2cb370 0x55db3f2d79f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:03.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.091+0000 7f440b7fe700 1 -- 172.21.15.100:0/3043448844 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db3f2c77c0 con 0x55db3f2cbd30 2024-05-10T11:30:03.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.091+0000 7f440b7fe700 1 --2- 172.21.15.100:0/3043448844 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db3f2cbd30 0x55db3f2e5150 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f4400005a80 tx=0x7f4400011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcbb912809823c58 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:03.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.091+0000 7f43f1ffb700 1 -- 172.21.15.100:0/3043448844 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440001fe10 con 0x55db3f2cbd30 2024-05-10T11:30:03.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.092+0000 7f43f1ffb700 1 -- 172.21.15.100:0/3043448844 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4400023070 con 0x55db3f2cbd30 2024-05-10T11:30:03.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.092+0000 7f43f1ffb700 1 -- 172.21.15.100:0/3043448844 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4400028470 con 0x55db3f2cbd30 2024-05-10T11:30:03.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.092+0000 7f441cdf2b80 1 -- 172.21.15.100:0/3043448844 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db3f2cbd30 msgr2=0x55db3f2e5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:03.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.092+0000 7f441cdf2b80 1 --2- 172.21.15.100:0/3043448844 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db3f2cbd30 0x55db3f2e5150 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f4400005a80 tx=0x7f4400011bf0 comp rx=0 tx=0).stop 2024-05-10T11:30:03.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.092+0000 7f441cdf2b80 1 -- 172.21.15.100:0/3043448844 shutdown_connections 2024-05-10T11:30:03.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.092+0000 7f441cdf2b80 1 --2- 172.21.15.100:0/3043448844 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db3f2ccc30 0x55db3f2d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:03.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.092+0000 7f441cdf2b80 1 --2- 172.21.15.100:0/3043448844 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db3f2cb370 0x55db3f2d79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:03.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.092+0000 7f441cdf2b80 1 --2- 172.21.15.100:0/3043448844 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db3f2cbd30 0x55db3f2e5150 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:03.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.092+0000 7f441cdf2b80 1 -- 172.21.15.100:0/3043448844 >> 172.21.15.100:0/3043448844 conn(0x55db3f1f7180 msgr2=0x55db3f2bff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:03.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.092+0000 7f441cdf2b80 1 -- 172.21.15.100:0/3043448844 shutdown_connections 2024-05-10T11:30:03.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.092+0000 7f441cdf2b80 1 -- 172.21.15.100:0/3043448844 wait complete. 2024-05-10T11:30:03.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.093+0000 7f441cdf2b80 1 -- start start 2024-05-10T11:30:03.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.093+0000 7f441cdf2b80 1 Processor -- start 2024-05-10T11:30:03.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.093+0000 7f441cdf2b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db3f2cb370 0x55db3f2e19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:03.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.093+0000 7f441cdf2b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db3f2cbd30 0x55db3f2d5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:03.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.093+0000 7f441cdf2b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db3f2ccc30 0x55db3f2d5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:03.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.093+0000 7f441cdf2b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f440c1ab3d0 con 0x55db3f2cbd30 2024-05-10T11:30:03.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.093+0000 7f441cdf2b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f440c1ac840 con 0x55db3f2ccc30 2024-05-10T11:30:03.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.093+0000 7f441cdf2b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f440c1abaa0 con 0x55db3f2cb370 2024-05-10T11:30:03.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.093+0000 7f441cdf2b80 10 client.0 _ll_register_callbacks cb 0x55db3f1b59a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:30:03.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.093+0000 7f440b7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db3f2cbd30 0x55db3f2d5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:03.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.093+0000 7f440bfff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db3f2cb370 0x55db3f2e19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:03.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.093+0000 7f440b7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db3f2cbd30 0x55db3f2d5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36116/0 (socket says 172.21.15.100:36116) 2024-05-10T11:30:03.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.093+0000 7f440bfff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db3f2cb370 0x55db3f2e19a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:54070/0 (socket says 172.21.15.100:54070) 2024-05-10T11:30:03.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.093+0000 7f440b7fe700 1 -- 172.21.15.100:0/923274340 learned_addr learned my addr 172.21.15.100:0/923274340 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:30:03.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.093+0000 7f4410afc700 1 --2- 172.21.15.100:0/923274340 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db3f2ccc30 0x55db3f2d5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:03.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.094+0000 7f440bfff700 1 -- 172.21.15.100:0/923274340 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db3f2ccc30 msgr2=0x55db3f2d5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:03.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.094+0000 7f440bfff700 1 --2- 172.21.15.100:0/923274340 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db3f2ccc30 0x55db3f2d5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:03.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.094+0000 7f440bfff700 1 -- 172.21.15.100:0/923274340 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db3f2cbd30 msgr2=0x55db3f2d5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:03.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.094+0000 7f440bfff700 1 --2- 172.21.15.100:0/923274340 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db3f2cbd30 0x55db3f2d5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:03.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.094+0000 7f440bfff700 1 -- 172.21.15.100:0/923274340 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db3f2d6300 con 0x55db3f2cb370 2024-05-10T11:30:03.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.094+0000 7f440b7fe700 1 --2- 172.21.15.100:0/923274340 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db3f2cbd30 0x55db3f2d5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:30:03.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.094+0000 7f440bfff700 1 --2- 172.21.15.100:0/923274340 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db3f2cb370 0x55db3f2e19a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f43fc0aa410 tx=0x7f43fc0e3330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8eac64d57570c41b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:03.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.094+0000 7f43eb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:30:03.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.094+0000 7f43eb7fe700 1 -- 172.21.15.100:0/923274340 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43fc0aa8e0 con 0x55db3f2cb370 2024-05-10T11:30:03.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.094+0000 7f441cdf2b80 1 -- 172.21.15.100:0/923274340 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4400023070 con 0x55db3f2cb370 2024-05-10T11:30:03.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.094+0000 7f43eb7fe700 1 -- 172.21.15.100:0/923274340 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f43fc0b2dc0 con 0x55db3f2cb370 2024-05-10T11:30:03.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.094+0000 7f43eb7fe700 1 -- 172.21.15.100:0/923274340 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f43fc0b3250 con 0x55db3f2cb370 2024-05-10T11:30:03.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.094+0000 7f441cdf2b80 10 client.27238 fetch_fsmap learned FSMap version 33 2024-05-10T11:30:03.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.094+0000 7f441cdf2b80 1 -- 172.21.15.100:0/923274340 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55db3f353800 con 0x55db3f2cb370 2024-05-10T11:30:03.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.095+0000 7f43eb7fe700 1 -- 172.21.15.100:0/923274340 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4400023070 con 0x55db3f2cb370 2024-05-10T11:30:03.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.095+0000 7f43eb7fe700 1 -- 172.21.15.100:0/923274340 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f43fc0b1b20 con 0x55db3f2cb370 2024-05-10T11:30:03.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.095+0000 7f441cdf2b80 10 client.27238 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T11:30:03.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.095+0000 7f441cdf2b80 10 client.27238 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:30:03.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.095+0000 7f441cdf2b80 20 client.27238 populate_metadata read hostname 'smithi100' 2024-05-10T11:30:03.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.095+0000 7f441cdf2b80 1 --2- 172.21.15.100:0/923274340 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55db3f353d50 0x55db3f356170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:03.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.095+0000 7f441cdf2b80 4 client.27238 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T11:30:03.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.095+0000 7f441cdf2b80 1 -- 172.21.15.100:0/923274340 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55db3f22b2c0 con 0x55db3f353d50 2024-05-10T11:30:03.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.095+0000 7f440b7fe700 1 --2- 172.21.15.100:0/923274340 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55db3f353d50 0x55db3f356170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:03.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.095+0000 7f43eb7fe700 10 client.27238 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:30:03.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.095+0000 7f440b7fe700 1 --2- 172.21.15.100:0/923274340 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55db3f353d50 0x55db3f356170 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c840582acfbadb6 server_cookie=4c3b4bae60df80bc in_seq=0 out_seq=0 2024-05-10T11:30:03.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.096+0000 7f43eb7fe700 1 -- 172.21.15.100:0/923274340 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55db3f22b2c0 con 0x55db3f353d50 2024-05-10T11:30:03.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.096+0000 7f43eb7fe700 10 client.27238 handle_command_reply: tid=0 2024-05-10T11:30:03.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.096+0000 7f441cdf2b80 1 client.27238 shutdown 2024-05-10T11:30:03.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.097+0000 7f441cdf2b80 1 -- 172.21.15.100:0/923274340 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db3f2cb370 msgr2=0x55db3f2e19a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:03.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.097+0000 7f441cdf2b80 1 --2- 172.21.15.100:0/923274340 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db3f2cb370 0x55db3f2e19a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f43fc0aa410 tx=0x7f43fc0e3330 comp rx=0 tx=0).stop 2024-05-10T11:30:03.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.097+0000 7f441cdf2b80 1 -- 172.21.15.100:0/923274340 shutdown_connections 2024-05-10T11:30:03.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.097+0000 7f441cdf2b80 1 --2- 172.21.15.100:0/923274340 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55db3f353d50 0x55db3f356170 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:03.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.097+0000 7f441cdf2b80 1 --2- 172.21.15.100:0/923274340 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db3f2ccc30 0x55db3f2d5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:03.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.097+0000 7f43eb7fe700 0 client.27238 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:30:03.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.097+0000 7f441cdf2b80 1 --2- 172.21.15.100:0/923274340 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db3f2cb370 0x55db3f2e19a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:03.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.097+0000 7f441cdf2b80 1 --2- 172.21.15.100:0/923274340 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db3f2cbd30 0x55db3f2d5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:03.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.097+0000 7f441cdf2b80 1 -- 172.21.15.100:0/923274340 >> 172.21.15.100:0/923274340 conn(0x55db3f1f7180 msgr2=0x55db3f2c07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:03.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.097+0000 7f441cdf2b80 1 -- 172.21.15.100:0/923274340 shutdown_connections 2024-05-10T11:30:03.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.097+0000 7f441cdf2b80 1 -- 172.21.15.100:0/923274340 wait complete. 2024-05-10T11:30:03.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.097+0000 7f441cdf2b80 20 client.27238 trim_cache size 0 max 16384 2024-05-10T11:30:03.102 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:30:03.102 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-05-10T11:30:03.102 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "d5d5f250-b400-4a57-af05-66002115c4a2", 2024-05-10T11:30:03.102 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-05-10T11:30:03.102 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:30:03.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.098+0000 7f441255f700 1 -- 172.21.15.100:0/3088561786 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f43f40b2880 msgr2=0x7f43f40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:03.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.098+0000 7f441255f700 1 --2- 172.21.15.100:0/3088561786 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f43f40b2880 0x7f43f40b4d40 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f440c079a90 tx=0x7f4400009260 comp rx=0 tx=0).stop 2024-05-10T11:30:03.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.098+0000 7f441255f700 1 -- 172.21.15.100:0/3088561786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f440c1023a0 msgr2=0x7f440c19e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:03.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.098+0000 7f441255f700 1 --2- 172.21.15.100:0/3088561786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f440c1023a0 0x7f440c19e600 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f43fc006010 tx=0x7f43fc00bfc0 comp rx=0 tx=0).stop 2024-05-10T11:30:03.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.098+0000 7f441255f700 1 -- 172.21.15.100:0/3088561786 shutdown_connections 2024-05-10T11:30:03.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.098+0000 7f441255f700 1 --2- 172.21.15.100:0/3088561786 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f43f40b2880 0x7f43f40b4d40 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:03.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.098+0000 7f441255f700 1 --2- 172.21.15.100:0/3088561786 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f440c108410 0x7f440c198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:03.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.098+0000 7f441255f700 1 --2- 172.21.15.100:0/3088561786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f440c1023a0 0x7f440c19e600 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:03.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.098+0000 7f441255f700 1 --2- 172.21.15.100:0/3088561786 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f440c102d60 0x7f440c19eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:03.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.098+0000 7f441255f700 1 -- 172.21.15.100:0/3088561786 >> 172.21.15.100:0/3088561786 conn(0x7f440c079350 msgr2=0x7f440c1012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:03.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.098+0000 7f441255f700 1 -- 172.21.15.100:0/3088561786 shutdown_connections 2024-05-10T11:30:03.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:03.098+0000 7f441255f700 1 -- 172.21.15.100:0/3088561786 wait complete. 2024-05-10T11:30:03.875 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T11:30:03.982 INFO:tasks.workunit.client.0.smithi100.stdout: 1 305217 2.72 MB/sec execute 478 sec latency 24.087 ms 2024-05-10T11:30:04.179 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:30:04.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:30:03 smithi178 ceph-mon[33359]: pgmap v771: 129 pgs: 129 active+clean; 118 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 1.3 MiB/s wr, 135 op/s 2024-05-10T11:30:04.254 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:30:03 smithi178 ceph-mon[33359]: scrub queued for path: / 2024-05-10T11:30:04.254 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:30:03 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:30:04.254 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:30:03 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:30:04.254 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:30:03 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:30:04.254 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:30:03 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:30:04.254 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:30:03 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:30:04.254 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:30:03 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:30:04.254 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:30:03 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:30:04.254 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:30:03 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:30:04.254 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:30:03 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:30:04.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:30:03 smithi188 ceph-mon[34075]: pgmap v771: 129 pgs: 129 active+clean; 118 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 1.3 MiB/s wr, 135 op/s 2024-05-10T11:30:04.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:30:03 smithi188 ceph-mon[34075]: scrub queued for path: / 2024-05-10T11:30:04.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:30:03 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:30:04.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:30:03 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:30:04.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:30:03 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:30:04.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:30:03 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:30:04.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:30:03 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:30:04.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:30:03 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:30:04.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:30:03 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:30:04.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:30:03 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:30:04.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:30:03 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:30:04.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:30:03 smithi100 ceph-mon[30296]: pgmap v771: 129 pgs: 129 active+clean; 118 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 1.3 MiB/s wr, 135 op/s 2024-05-10T11:30:04.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:30:03 smithi100 ceph-mon[30296]: scrub queued for path: / 2024-05-10T11:30:04.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:30:03 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:30:04.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:30:03 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:30:04.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:30:03 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:30:04.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:30:03 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:30:04.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:30:03 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:30:04.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:30:03 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:30:04.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:30:03 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:30:04.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:30:03 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:30:04.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:30:03 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:30:04.982 INFO:tasks.workunit.client.0.smithi100.stdout: 1 305492 2.72 MB/sec execute 479 sec latency 20.319 ms 2024-05-10T11:30:05.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:30:04 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:30:05.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:30:04 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:30:05.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:30:04 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:30:05.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:30:04 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:30:05.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:30:04 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:30:05.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:30:04 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:30:05.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:30:04 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:30:05.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:30:04 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:30:05.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:30:04 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:30:05.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:30:04 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:30:05.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:30:04 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:30:05.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:30:04 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:30:05.937 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:30:05 smithi100 ceph-mon[30296]: pgmap v772: 129 pgs: 129 active+clean; 118 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 1.3 MiB/s wr, 138 op/s 2024-05-10T11:30:05.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.943+0000 7fc4b96e3700 1 -- 172.21.15.100:0/889686320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4b4102d60 msgr2=0x7fc4b410f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:05.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.943+0000 7fc4b96e3700 1 --2- 172.21.15.100:0/889686320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4b4102d60 0x7fc4b410f250 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac00b0a0 tx=0x7fc4ac00b3b0 comp rx=0 tx=0).stop 2024-05-10T11:30:05.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.943+0000 7fc4b96e3700 1 -- 172.21.15.100:0/889686320 shutdown_connections 2024-05-10T11:30:05.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.943+0000 7fc4b96e3700 1 --2- 172.21.15.100:0/889686320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4b4102d60 0x7fc4b410f250 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:05.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.943+0000 7fc4b96e3700 1 --2- 172.21.15.100:0/889686320 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4b4108410 0x7fc4b41087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:05.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.943+0000 7fc4b96e3700 1 --2- 172.21.15.100:0/889686320 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc4b41023a0 0x7fc4b4102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:05.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.943+0000 7fc4b96e3700 1 -- 172.21.15.100:0/889686320 >> 172.21.15.100:0/889686320 conn(0x7fc4b4079350 msgr2=0x7fc4b4079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:05.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.943+0000 7fc4b96e3700 1 -- 172.21.15.100:0/889686320 shutdown_connections 2024-05-10T11:30:05.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.943+0000 7fc4b96e3700 1 -- 172.21.15.100:0/889686320 wait complete. 2024-05-10T11:30:05.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.944+0000 7fc4b96e3700 1 Processor -- start 2024-05-10T11:30:05.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.944+0000 7fc4b96e3700 1 -- start start 2024-05-10T11:30:05.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.944+0000 7fc4b96e3700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4b41023a0 0x7fc4b4104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:05.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.944+0000 7fc4b96e3700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc4b4102d60 0x7fc4b41045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:05.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.944+0000 7fc4b96e3700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4b4108410 0x7fc4b4104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:05.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.945+0000 7fc4b96e3700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b4112400 con 0x7fc4b4108410 2024-05-10T11:30:05.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.945+0000 7fc4b96e3700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b4112280 con 0x7fc4b4102d60 2024-05-10T11:30:05.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.945+0000 7fc4b96e3700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b4112580 con 0x7fc4b41023a0 2024-05-10T11:30:05.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.945+0000 7fc4b37fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4b4108410 0x7fc4b4104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:05.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.945+0000 7fc4b2ffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4b41023a0 0x7fc4b4104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:05.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.945+0000 7fc4b27fc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc4b4102d60 0x7fc4b41045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:05.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.945+0000 7fc4b37fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4b4108410 0x7fc4b4104b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36130/0 (socket says 172.21.15.100:36130) 2024-05-10T11:30:05.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.945+0000 7fc4b37fe700 1 -- 172.21.15.100:0/3694763051 learned_addr learned my addr 172.21.15.100:0/3694763051 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:30:05.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.945+0000 7fc4b37fe700 1 -- 172.21.15.100:0/3694763051 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4b41023a0 msgr2=0x7fc4b4104080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:05.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.945+0000 7fc4b37fe700 1 --2- 172.21.15.100:0/3694763051 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4b41023a0 0x7fc4b4104080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:05.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.945+0000 7fc4b37fe700 1 -- 172.21.15.100:0/3694763051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc4b4102d60 msgr2=0x7fc4b41045c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:05.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.945+0000 7fc4b37fe700 1 --2- 172.21.15.100:0/3694763051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc4b4102d60 0x7fc4b41045c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:05.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.945+0000 7fc4b37fe700 1 -- 172.21.15.100:0/3694763051 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4b41a7f10 con 0x7fc4b4108410 2024-05-10T11:30:05.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.945+0000 7fc4b27fc700 1 --2- 172.21.15.100:0/3694763051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc4b4102d60 0x7fc4b41045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:30:05.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.945+0000 7fc4b37fe700 1 --2- 172.21.15.100:0/3694763051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4b4108410 0x7fc4b4104b00 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac00b780 tx=0x7fc4ac004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:05.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.946+0000 7fc49bfff700 1 -- 172.21.15.100:0/3694763051 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac021070 con 0x7fc4b4108410 2024-05-10T11:30:05.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.946+0000 7fc4b96e3700 1 -- 172.21.15.100:0/3694763051 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4b41a81a0 con 0x7fc4b4108410 2024-05-10T11:30:05.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.946+0000 7fc49bfff700 1 -- 172.21.15.100:0/3694763051 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc4ac007b80 con 0x7fc4b4108410 2024-05-10T11:30:05.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.946+0000 7fc49bfff700 1 -- 172.21.15.100:0/3694763051 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac005040 con 0x7fc4b4108410 2024-05-10T11:30:05.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.946+0000 7fc4b96e3700 1 -- 172.21.15.100:0/3694763051 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4b41a85a0 con 0x7fc4b4108410 2024-05-10T11:30:05.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.948+0000 7fc49bfff700 1 -- 172.21.15.100:0/3694763051 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fc4ac024070 con 0x7fc4b4108410 2024-05-10T11:30:05.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.949+0000 7fc49bfff700 1 --2- 172.21.15.100:0/3694763051 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc49c0b2880 0x7fc49c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:05.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.949+0000 7fc4b2ffd700 1 --2- 172.21.15.100:0/3694763051 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc49c0b2880 0x7fc49c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:05.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.949+0000 7fc49bfff700 1 -- 172.21.15.100:0/3694763051 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fc4ac0e3540 con 0x7fc4b4108410 2024-05-10T11:30:05.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.949+0000 7fc4b2ffd700 1 --2- 172.21.15.100:0/3694763051 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc49c0b2880 0x7fc49c0b4d40 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fc4a4009be0 tx=0x7fc4a4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:05.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.950+0000 7fc4c3f76b80 1 Processor -- start 2024-05-10T11:30:05.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.950+0000 7fc4c3f76b80 1 -- start start 2024-05-10T11:30:05.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.950+0000 7fc4c3f76b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d62cce74d0 0x55d62cce78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:05.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.950+0000 7fc4c3f76b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d62cce7df0 0x55d62cceb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:05.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.950+0000 7fc4c3f76b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d62ccebc80 0x55d62ccec100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:05.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.950+0000 7fc4c3f76b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55d62cc39860 con 0x55d62ccebc80 2024-05-10T11:30:05.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.950+0000 7fc4c3f76b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d62cbd87c0 con 0x55d62cce7df0 2024-05-10T11:30:05.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.950+0000 7fc4c3f76b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d62cc35820 con 0x55d62cce74d0 2024-05-10T11:30:05.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.950+0000 7fc4b27fc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d62cce7df0 0x55d62cceb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:05.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.950+0000 7fc4b37fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d62ccebc80 0x55d62ccec100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:05.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.950+0000 7fc4b2ffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d62cce74d0 0x55d62cce78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:05.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.950+0000 7fc4b27fc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d62cce7df0 0x55d62cceb6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:43372/0 (socket says 172.21.15.100:43372) 2024-05-10T11:30:05.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.950+0000 7fc4b27fc700 1 -- 172.21.15.100:0/137527244 learned_addr learned my addr 172.21.15.100:0/137527244 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:30:05.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.950+0000 7fc4b37fe700 1 -- 172.21.15.100:0/137527244 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d62cce74d0 msgr2=0x55d62cce78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:05.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.950+0000 7fc4b37fe700 1 --2- 172.21.15.100:0/137527244 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d62cce74d0 0x55d62cce78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:05.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.950+0000 7fc4b37fe700 1 -- 172.21.15.100:0/137527244 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d62cce7df0 msgr2=0x55d62cceb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:05.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.950+0000 7fc4b37fe700 1 --2- 172.21.15.100:0/137527244 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d62cce7df0 0x55d62cceb6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:05.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.951+0000 7fc4b37fe700 1 -- 172.21.15.100:0/137527244 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d62cb868d0 con 0x55d62ccebc80 2024-05-10T11:30:05.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.951+0000 7fc4b27fc700 1 --2- 172.21.15.100:0/137527244 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d62cce7df0 0x55d62cceb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:30:05.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.951+0000 7fc4b37fe700 1 --2- 172.21.15.100:0/137527244 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d62ccebc80 0x55d62ccec100 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac0e0ce0 tx=0x7fc4ac00b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed37279f4b3bd811 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:05.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.951+0000 7fc4997fa700 1 -- 172.21.15.100:0/137527244 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac02b070 con 0x55d62ccebc80 2024-05-10T11:30:05.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.951+0000 7fc4997fa700 1 -- 172.21.15.100:0/137527244 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc4ac0aa890 con 0x55d62ccebc80 2024-05-10T11:30:05.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.951+0000 7fc4997fa700 1 -- 172.21.15.100:0/137527244 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac0b10d0 con 0x55d62ccebc80 2024-05-10T11:30:05.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.952+0000 7fc4c3f76b80 1 -- 172.21.15.100:0/137527244 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d62ccebc80 msgr2=0x55d62ccec100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:05.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.952+0000 7fc4c3f76b80 1 --2- 172.21.15.100:0/137527244 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d62ccebc80 0x55d62ccec100 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac0e0ce0 tx=0x7fc4ac00b3b0 comp rx=0 tx=0).stop 2024-05-10T11:30:05.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.952+0000 7fc4c3f76b80 1 -- 172.21.15.100:0/137527244 shutdown_connections 2024-05-10T11:30:05.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.952+0000 7fc4c3f76b80 1 --2- 172.21.15.100:0/137527244 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d62ccebc80 0x55d62ccec100 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:05.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.952+0000 7fc4c3f76b80 1 --2- 172.21.15.100:0/137527244 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d62cce74d0 0x55d62cce78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:05.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.952+0000 7fc4c3f76b80 1 --2- 172.21.15.100:0/137527244 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d62cce7df0 0x55d62cceb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:05.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.952+0000 7fc4c3f76b80 1 -- 172.21.15.100:0/137527244 >> 172.21.15.100:0/137527244 conn(0x55d62cc13100 msgr2=0x55d62cce1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:05.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.952+0000 7fc4c3f76b80 1 -- 172.21.15.100:0/137527244 shutdown_connections 2024-05-10T11:30:05.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.952+0000 7fc4c3f76b80 1 -- 172.21.15.100:0/137527244 wait complete. 2024-05-10T11:30:05.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.952+0000 7fc4c3f76b80 1 -- start start 2024-05-10T11:30:05.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.953+0000 7fc4c3f76b80 1 Processor -- start 2024-05-10T11:30:05.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.953+0000 7fc4c3f76b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d62cce74d0 0x55d62ccf3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:05.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.953+0000 7fc4c3f76b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d62cce7df0 0x55d62ccf0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:05.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.953+0000 7fc4c3f76b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d62ccebc80 0x55d62ccf0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:05.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.953+0000 7fc4c3f76b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55d62ccf3c50 con 0x55d62cce74d0 2024-05-10T11:30:05.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.953+0000 7fc4c3f76b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d62cb9fc30 con 0x55d62cce7df0 2024-05-10T11:30:05.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.953+0000 7fc4c3f76b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d62ccf3dd0 con 0x55d62ccebc80 2024-05-10T11:30:05.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.953+0000 7fc4c3f76b80 10 client.0 _ll_register_callbacks cb 0x55d62cbd1920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:30:05.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.953+0000 7fc4b2ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d62cce74d0 0x55d62ccf3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:05.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.953+0000 7fc4b37fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d62ccebc80 0x55d62ccf0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:05.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.953+0000 7fc4b27fc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d62cce7df0 0x55d62ccf0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:05.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.953+0000 7fc4b2ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d62cce74d0 0x55d62ccf3150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36152/0 (socket says 172.21.15.100:36152) 2024-05-10T11:30:05.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.953+0000 7fc4b37fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d62ccebc80 0x55d62ccf0a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:54102/0 (socket says 172.21.15.100:54102) 2024-05-10T11:30:05.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.953+0000 7fc4b2ffd700 1 -- 172.21.15.100:0/2007339811 learned_addr learned my addr 172.21.15.100:0/2007339811 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:30:05.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.953+0000 7fc4b2ffd700 1 -- 172.21.15.100:0/2007339811 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d62ccebc80 msgr2=0x55d62ccf0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:05.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.953+0000 7fc4b2ffd700 1 --2- 172.21.15.100:0/2007339811 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d62ccebc80 0x55d62ccf0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:05.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.953+0000 7fc4b2ffd700 1 -- 172.21.15.100:0/2007339811 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d62cce7df0 msgr2=0x55d62ccf0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:05.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.953+0000 7fc4b2ffd700 1 --2- 172.21.15.100:0/2007339811 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d62cce7df0 0x55d62ccf0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:05.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.953+0000 7fc4b2ffd700 1 -- 172.21.15.100:0/2007339811 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d62cbd5690 con 0x55d62cce74d0 2024-05-10T11:30:05.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.953+0000 7fc4b27fc700 1 --2- 172.21.15.100:0/2007339811 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d62cce7df0 0x55d62ccf0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:30:05.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.954+0000 7fc4b2ffd700 1 --2- 172.21.15.100:0/2007339811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d62cce74d0 0x55d62ccf3150 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fc4a401f8e0 tx=0x7fc4a40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4722b831d44be25f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:05.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.954+0000 7fc496ff5700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:30:05.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.954+0000 7fc496ff5700 1 -- 172.21.15.100:0/2007339811 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a4004300 con 0x55d62cce74d0 2024-05-10T11:30:05.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.954+0000 7fc496ff5700 1 -- 172.21.15.100:0/2007339811 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fc4a4009040 con 0x55d62cce74d0 2024-05-10T11:30:05.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.954+0000 7fc4c3f76b80 1 -- 172.21.15.100:0/2007339811 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4ac0aa890 con 0x55d62cce74d0 2024-05-10T11:30:05.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.954+0000 7fc496ff5700 1 -- 172.21.15.100:0/2007339811 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc4a40044a0 con 0x55d62cce74d0 2024-05-10T11:30:05.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.954+0000 7fc4c3f76b80 10 client.17799 fetch_fsmap learned FSMap version 33 2024-05-10T11:30:05.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.954+0000 7fc4c3f76b80 1 -- 172.21.15.100:0/2007339811 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d62ccfe520 con 0x55d62cce74d0 2024-05-10T11:30:05.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.954+0000 7fc496ff5700 1 -- 172.21.15.100:0/2007339811 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc4ac0aa890 con 0x55d62cce74d0 2024-05-10T11:30:05.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.955+0000 7fc496ff5700 1 -- 172.21.15.100:0/2007339811 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fc4a4028750 con 0x55d62cce74d0 2024-05-10T11:30:05.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.955+0000 7fc4c3f76b80 10 client.17799 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T11:30:05.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.955+0000 7fc4c3f76b80 10 client.17799 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:30:05.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.955+0000 7fc4c3f76b80 20 client.17799 populate_metadata read hostname 'smithi100' 2024-05-10T11:30:05.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.955+0000 7fc4c3f76b80 1 --2- 172.21.15.100:0/2007339811 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d62cd00c00 0x55d62cd03010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:05.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.955+0000 7fc4c3f76b80 4 client.17799 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:30:05.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.955+0000 7fc4c3f76b80 1 -- 172.21.15.100:0/2007339811 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d62ccb3650 con 0x55d62cd00c00 2024-05-10T11:30:05.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.955+0000 7fc4b27fc700 1 --2- 172.21.15.100:0/2007339811 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d62cd00c00 0x55d62cd03010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:05.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.955+0000 7fc4b27fc700 1 --2- 172.21.15.100:0/2007339811 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d62cd00c00 0x55d62cd03010 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=302fd0af659cd5f5 server_cookie=39e31b16f2835853 in_seq=0 out_seq=0 2024-05-10T11:30:05.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.955+0000 7fc496ff5700 10 client.17799 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:30:05.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.956+0000 7fc496ff5700 1 -- 172.21.15.100:0/2007339811 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d62ccb3650 con 0x55d62cd00c00 2024-05-10T11:30:05.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.956+0000 7fc496ff5700 10 client.17799 handle_command_reply: tid=0 2024-05-10T11:30:05.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.956+0000 7fc4c3f76b80 1 client.17799 shutdown 2024-05-10T11:30:05.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.956+0000 7fc4c3f76b80 1 -- 172.21.15.100:0/2007339811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d62cce74d0 msgr2=0x55d62ccf3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:05.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.956+0000 7fc4c3f76b80 1 --2- 172.21.15.100:0/2007339811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d62cce74d0 0x55d62ccf3150 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fc4a401f8e0 tx=0x7fc4a40047c0 comp rx=0 tx=0).stop 2024-05-10T11:30:05.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.956+0000 7fc4c3f76b80 1 -- 172.21.15.100:0/2007339811 shutdown_connections 2024-05-10T11:30:05.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.956+0000 7fc4c3f76b80 1 --2- 172.21.15.100:0/2007339811 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d62cd00c00 0x55d62cd03010 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:05.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.956+0000 7fc4c3f76b80 1 --2- 172.21.15.100:0/2007339811 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d62ccebc80 0x55d62ccf0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:05.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.956+0000 7fc4c3f76b80 1 --2- 172.21.15.100:0/2007339811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d62cce74d0 0x55d62ccf3150 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:05.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.956+0000 7fc4c3f76b80 1 --2- 172.21.15.100:0/2007339811 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d62cce7df0 0x55d62ccf0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:05.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.956+0000 7fc4c3f76b80 1 -- 172.21.15.100:0/2007339811 >> 172.21.15.100:0/2007339811 conn(0x55d62cc13100 msgr2=0x55d62ccdc720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:05.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.956+0000 7fc496ff5700 0 client.17799 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:30:05.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.956+0000 7fc4c3f76b80 1 -- 172.21.15.100:0/2007339811 shutdown_connections 2024-05-10T11:30:05.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.956+0000 7fc4c3f76b80 1 -- 172.21.15.100:0/2007339811 wait complete. 2024-05-10T11:30:05.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.956+0000 7fc4c3f76b80 20 client.17799 trim_cache size 0 max 16384 2024-05-10T11:30:05.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.978+0000 7fc4c3f76b80 1 Processor -- start 2024-05-10T11:30:05.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.978+0000 7fc4c3f76b80 1 -- start start 2024-05-10T11:30:05.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.978+0000 7fc4c3f76b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d62cc13100 0x55d62ccead00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:05.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.978+0000 7fc4c3f76b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d62cce74d0 0x55d62cd08920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:05.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.978+0000 7fc4c3f76b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d62cce7df0 0x55d62cd08e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:05.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.978+0000 7fc4c3f76b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b41b38d0 con 0x55d62cce7df0 2024-05-10T11:30:05.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.978+0000 7fc4c3f76b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b41b1160 con 0x55d62cc13100 2024-05-10T11:30:05.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.978+0000 7fc4c3f76b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b41aefb0 con 0x55d62cce74d0 2024-05-10T11:30:05.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.979+0000 7fc4b37fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d62cce7df0 0x55d62cd08e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:05.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.979+0000 7fc4b37fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d62cce7df0 0x55d62cd08e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36166/0 (socket says 172.21.15.100:36166) 2024-05-10T11:30:05.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.979+0000 7fc4b37fe700 1 -- 172.21.15.100:0/813372625 learned_addr learned my addr 172.21.15.100:0/813372625 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:30:05.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.979+0000 7fc4b27fc700 1 --2- 172.21.15.100:0/813372625 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d62cce74d0 0x55d62cd08920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:05.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.979+0000 7fc4b2ffd700 1 --2- 172.21.15.100:0/813372625 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d62cc13100 0x55d62ccead00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:05.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.979+0000 7fc4b37fe700 1 -- 172.21.15.100:0/813372625 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d62cce74d0 msgr2=0x55d62cd08920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:05.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.979+0000 7fc4b37fe700 1 --2- 172.21.15.100:0/813372625 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d62cce74d0 0x55d62cd08920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:05.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.979+0000 7fc4b37fe700 1 -- 172.21.15.100:0/813372625 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d62cc13100 msgr2=0x55d62ccead00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:05.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.979+0000 7fc4b37fe700 1 --2- 172.21.15.100:0/813372625 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d62cc13100 0x55d62ccead00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:05.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.979+0000 7fc4b37fe700 1 -- 172.21.15.100:0/813372625 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d62cce7b00 con 0x55d62cce7df0 2024-05-10T11:30:05.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.979+0000 7fc4b2ffd700 1 --2- 172.21.15.100:0/813372625 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d62cc13100 0x55d62ccead00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:30:05.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.979+0000 7fc4b37fe700 1 --2- 172.21.15.100:0/813372625 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d62cce7df0 0x55d62cd08e60 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac0e0d30 tx=0x7fc4ac0a6040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6679681bbf5fdbce server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:05.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.979+0000 7fc498ff9700 1 -- 172.21.15.100:0/813372625 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac02b040 con 0x55d62cce7df0 2024-05-10T11:30:05.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.979+0000 7fc498ff9700 1 -- 172.21.15.100:0/813372625 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc4ac0aec40 con 0x55d62cce7df0 2024-05-10T11:30:05.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.979+0000 7fc498ff9700 1 -- 172.21.15.100:0/813372625 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac0b10d0 con 0x55d62cce7df0 2024-05-10T11:30:05.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.980+0000 7fc4c3f76b80 1 -- 172.21.15.100:0/813372625 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d62cce7df0 msgr2=0x55d62cd08e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:05.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.980+0000 7fc4c3f76b80 1 --2- 172.21.15.100:0/813372625 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d62cce7df0 0x55d62cd08e60 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac0e0d30 tx=0x7fc4ac0a6040 comp rx=0 tx=0).stop 2024-05-10T11:30:05.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.980+0000 7fc4c3f76b80 1 -- 172.21.15.100:0/813372625 shutdown_connections 2024-05-10T11:30:05.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.980+0000 7fc4c3f76b80 1 --2- 172.21.15.100:0/813372625 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d62cce7df0 0x55d62cd08e60 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:05.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.980+0000 7fc4c3f76b80 1 --2- 172.21.15.100:0/813372625 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d62cc13100 0x55d62ccead00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:05.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.980+0000 7fc4c3f76b80 1 --2- 172.21.15.100:0/813372625 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d62cce74d0 0x55d62cd08920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:05.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.980+0000 7fc4c3f76b80 1 -- 172.21.15.100:0/813372625 >> 172.21.15.100:0/813372625 conn(0x55d62ccdcab0 msgr2=0x55d62ccea5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:05.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.980+0000 7fc4c3f76b80 1 -- 172.21.15.100:0/813372625 shutdown_connections 2024-05-10T11:30:05.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.980+0000 7fc4c3f76b80 1 -- 172.21.15.100:0/813372625 wait complete. 2024-05-10T11:30:05.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.981+0000 7fc4c3f76b80 1 -- start start 2024-05-10T11:30:05.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.981+0000 7fc4c3f76b80 1 Processor -- start 2024-05-10T11:30:05.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.981+0000 7fc4c3f76b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d62cc13100 0x55d62ccf6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:05.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.981+0000 7fc4c3f76b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d62cce74d0 0x55d62ccf70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:05.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.981+0000 7fc4c3f76b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d62cce7df0 0x55d62ccf75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:05.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.981+0000 7fc4c3f76b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b41af620 con 0x55d62cce74d0 2024-05-10T11:30:05.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.981+0000 7fc4c3f76b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b41b0a90 con 0x55d62cce7df0 2024-05-10T11:30:05.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.981+0000 7fc4c3f76b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b41afcf0 con 0x55d62cc13100 2024-05-10T11:30:05.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.981+0000 7fc4c3f76b80 10 client.0 _ll_register_callbacks cb 0x55d62cbd1920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:30:05.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.981+0000 7fc4b27fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d62cce74d0 0x55d62ccf70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:05.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.981+0000 7fc4b2ffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d62cc13100 0x55d62ccf6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:05.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.981+0000 7fc4b27fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d62cce74d0 0x55d62ccf70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36172/0 (socket says 172.21.15.100:36172) 2024-05-10T11:30:05.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.981+0000 7fc4b2ffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d62cc13100 0x55d62ccf6b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:54122/0 (socket says 172.21.15.100:54122) 2024-05-10T11:30:05.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.981+0000 7fc4b27fc700 1 -- 172.21.15.100:0/3041751092 learned_addr learned my addr 172.21.15.100:0/3041751092 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:30:05.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.981+0000 7fc4b37fe700 1 --2- 172.21.15.100:0/3041751092 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d62cce7df0 0x55d62ccf75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:05.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.981+0000 7fc4b27fc700 1 -- 172.21.15.100:0/3041751092 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d62cc13100 msgr2=0x55d62ccf6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:05.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.981+0000 7fc4b27fc700 1 --2- 172.21.15.100:0/3041751092 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d62cc13100 0x55d62ccf6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:05.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.981+0000 7fc4b27fc700 1 -- 172.21.15.100:0/3041751092 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d62cce7df0 msgr2=0x55d62ccf75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:05.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.981+0000 7fc4b27fc700 1 --2- 172.21.15.100:0/3041751092 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d62cce7df0 0x55d62ccf75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:05.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.981+0000 7fc4b27fc700 1 -- 172.21.15.100:0/3041751092 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d62cd6a4e0 con 0x55d62cce74d0 2024-05-10T11:30:05.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.981+0000 7fc4b2ffd700 1 --2- 172.21.15.100:0/3041751092 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d62cc13100 0x55d62ccf6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:30:05.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.981+0000 7fc4b37fe700 1 --2- 172.21.15.100:0/3041751092 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d62cce7df0 0x55d62ccf75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:30:05.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.982+0000 7fc496ff5700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:30:05.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.982+0000 7fc4b27fc700 1 --2- 172.21.15.100:0/3041751092 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d62cce74d0 0x55d62ccf70a0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x55d62ccf1ac0 tx=0x7fc4a80206e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca7284531ed41924 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:05.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.982+0000 7fc496ff5700 1 -- 172.21.15.100:0/3041751092 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a8004a30 con 0x55d62cce74d0 2024-05-10T11:30:05.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.982+0000 7fc496ff5700 1 -- 172.21.15.100:0/3041751092 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fc4a8004bd0 con 0x55d62cce74d0 2024-05-10T11:30:05.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.982+0000 7fc4c3f76b80 1 -- 172.21.15.100:0/3041751092 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d62cc47240 con 0x55d62cce74d0 2024-05-10T11:30:05.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.982+0000 7fc496ff5700 1 -- 172.21.15.100:0/3041751092 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc4a8031050 con 0x55d62cce74d0 2024-05-10T11:30:05.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.982+0000 7fc4c3f76b80 10 client.17805 fetch_fsmap learned FSMap version 33 2024-05-10T11:30:05.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.982+0000 7fc4c3f76b80 1 -- 172.21.15.100:0/3041751092 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d62cd6b820 con 0x55d62cce74d0 2024-05-10T11:30:05.986 INFO:tasks.workunit.client.0.smithi100.stdout: 1 305890 2.71 MB/sec execute 480 sec latency 14.630 ms 2024-05-10T11:30:05.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.982+0000 7fc496ff5700 1 -- 172.21.15.100:0/3041751092 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55d62cc47240 con 0x55d62cce74d0 2024-05-10T11:30:05.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.982+0000 7fc496ff5700 1 -- 172.21.15.100:0/3041751092 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fc4a8022370 con 0x55d62cce74d0 2024-05-10T11:30:05.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.982+0000 7fc4c3f76b80 10 client.17805 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T11:30:05.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.982+0000 7fc4c3f76b80 10 client.17805 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:30:05.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.982+0000 7fc4c3f76b80 20 client.17805 populate_metadata read hostname 'smithi100' 2024-05-10T11:30:05.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.982+0000 7fc4c3f76b80 1 --2- 172.21.15.100:0/3041751092 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d62ccebc80 0x55d62cd6dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:05.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.982+0000 7fc4c3f76b80 4 client.17805 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T11:30:05.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.982+0000 7fc4c3f76b80 1 -- 172.21.15.100:0/3041751092 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d62cc412e0 con 0x55d62ccebc80 2024-05-10T11:30:05.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.983+0000 7fc4b2ffd700 1 --2- 172.21.15.100:0/3041751092 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d62ccebc80 0x55d62cd6dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:05.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.983+0000 7fc496ff5700 10 client.17805 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:30:05.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.983+0000 7fc4b2ffd700 1 --2- 172.21.15.100:0/3041751092 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d62ccebc80 0x55d62cd6dfa0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=541ca3e87828b795 server_cookie=b19b7c80e7aac3a9 in_seq=0 out_seq=0 2024-05-10T11:30:05.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.983+0000 7fc496ff5700 1 -- 172.21.15.100:0/3041751092 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d62cc412e0 con 0x55d62ccebc80 2024-05-10T11:30:05.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.983+0000 7fc496ff5700 10 client.17805 handle_command_reply: tid=0 2024-05-10T11:30:05.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.983+0000 7fc4c3f76b80 1 client.17805 shutdown 2024-05-10T11:30:05.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.983+0000 7fc4c3f76b80 1 -- 172.21.15.100:0/3041751092 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d62cce74d0 msgr2=0x55d62ccf70a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:05.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.983+0000 7fc4c3f76b80 1 --2- 172.21.15.100:0/3041751092 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d62cce74d0 0x55d62ccf70a0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x55d62ccf1ac0 tx=0x7fc4a80206e0 comp rx=0 tx=0).stop 2024-05-10T11:30:05.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.983+0000 7fc4c3f76b80 1 -- 172.21.15.100:0/3041751092 shutdown_connections 2024-05-10T11:30:05.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.983+0000 7fc4c3f76b80 1 --2- 172.21.15.100:0/3041751092 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d62ccebc80 0x55d62cd6dfa0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:05.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.983+0000 7fc4c3f76b80 1 --2- 172.21.15.100:0/3041751092 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d62cce7df0 0x55d62ccf75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:05.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.983+0000 7fc4c3f76b80 1 --2- 172.21.15.100:0/3041751092 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d62cc13100 0x55d62ccf6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:05.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.983+0000 7fc4c3f76b80 1 --2- 172.21.15.100:0/3041751092 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d62cce74d0 0x55d62ccf70a0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:05.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.983+0000 7fc4c3f76b80 1 -- 172.21.15.100:0/3041751092 >> 172.21.15.100:0/3041751092 conn(0x55d62ccdcab0 msgr2=0x55d62cce6b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:05.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.983+0000 7fc496ff5700 0 client.17805 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:30:05.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.984+0000 7fc4c3f76b80 1 -- 172.21.15.100:0/3041751092 shutdown_connections 2024-05-10T11:30:05.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.984+0000 7fc4c3f76b80 1 -- 172.21.15.100:0/3041751092 wait complete. 2024-05-10T11:30:05.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.984+0000 7fc4c3f76b80 20 client.17805 trim_cache size 0 max 16384 2024-05-10T11:30:05.990 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:30:05.990 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T11:30:05.990 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-05-10T11:30:05.990 INFO:teuthology.orchestra.run.smithi100.stdout: "d5d5f250-b400-4a57-af05-66002115c4a2": { 2024-05-10T11:30:05.990 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-05-10T11:30:05.990 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "d5d5f250-b400-4a57-af05-66002115c4a2", 2024-05-10T11:30:05.990 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-05-10T11:30:05.990 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:30:05.990 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:30:05.990 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:30:05.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.984+0000 7fc4b96e3700 1 -- 172.21.15.100:0/3694763051 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc49c0b2880 msgr2=0x7fc49c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:05.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.984+0000 7fc4b96e3700 1 --2- 172.21.15.100:0/3694763051 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc49c0b2880 0x7fc49c0b4d40 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fc4a4009be0 tx=0x7fc4a4009260 comp rx=0 tx=0).stop 2024-05-10T11:30:05.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.984+0000 7fc4b96e3700 1 -- 172.21.15.100:0/3694763051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4b4108410 msgr2=0x7fc4b4104b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:05.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.984+0000 7fc4b96e3700 1 --2- 172.21.15.100:0/3694763051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4b4108410 0x7fc4b4104b00 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac00b780 tx=0x7fc4ac004d00 comp rx=0 tx=0).stop 2024-05-10T11:30:05.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.984+0000 7fc4b96e3700 1 -- 172.21.15.100:0/3694763051 shutdown_connections 2024-05-10T11:30:05.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.984+0000 7fc4b96e3700 1 --2- 172.21.15.100:0/3694763051 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc49c0b2880 0x7fc49c0b4d40 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:05.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.984+0000 7fc4b96e3700 1 --2- 172.21.15.100:0/3694763051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4b4108410 0x7fc4b4104b00 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:05.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.984+0000 7fc4b96e3700 1 --2- 172.21.15.100:0/3694763051 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4b41023a0 0x7fc4b4104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:05.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.984+0000 7fc4b96e3700 1 --2- 172.21.15.100:0/3694763051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc4b4102d60 0x7fc4b41045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:05.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.984+0000 7fc4b96e3700 1 -- 172.21.15.100:0/3694763051 >> 172.21.15.100:0/3694763051 conn(0x7fc4b4079350 msgr2=0x7fc4b4101600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:05.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.984+0000 7fc4b96e3700 1 -- 172.21.15.100:0/3694763051 shutdown_connections 2024-05-10T11:30:05.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:05.984+0000 7fc4b96e3700 1 -- 172.21.15.100:0/3694763051 wait complete. 2024-05-10T11:30:06.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:30:05 smithi178 ceph-mon[33359]: pgmap v772: 129 pgs: 129 active+clean; 118 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 1.3 MiB/s wr, 138 op/s 2024-05-10T11:30:06.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:30:05 smithi188 ceph-mon[34075]: pgmap v772: 129 pgs: 129 active+clean; 118 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 1.3 MiB/s wr, 138 op/s 2024-05-10T11:30:06.729 INFO:tasks.cephfs.filesystem:scrub status for tag:d5d5f250-b400-4a57-af05-66002115c4a2 - {'path': '/', 'tag': 'd5d5f250-b400-4a57-af05-66002115c4a2', 'options': 'recursive,force'} 2024-05-10T11:30:06.982 INFO:tasks.workunit.client.0.smithi100.stdout: 1 306450 2.72 MB/sec execute 481 sec latency 15.975 ms 2024-05-10T11:30:07.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:30:07 smithi178 ceph-mon[33359]: pgmap v773: 129 pgs: 129 active+clean; 118 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 1.3 MiB/s wr, 176 op/s 2024-05-10T11:30:07.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:30:07 smithi188 ceph-mon[34075]: pgmap v773: 129 pgs: 129 active+clean; 118 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 1.3 MiB/s wr, 176 op/s 2024-05-10T11:30:07.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:30:07 smithi100 ceph-mon[30296]: pgmap v773: 129 pgs: 129 active+clean; 118 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 1.3 MiB/s wr, 176 op/s 2024-05-10T11:30:07.982 INFO:tasks.workunit.client.0.smithi100.stdout: 1 307104 2.71 MB/sec execute 482 sec latency 13.629 ms 2024-05-10T11:30:08.982 INFO:tasks.workunit.client.0.smithi100.stdout: 1 307636 2.71 MB/sec execute 483 sec latency 15.176 ms 2024-05-10T11:30:09.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:30:09 smithi178 ceph-mon[33359]: pgmap v774: 129 pgs: 129 active+clean; 119 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 1.3 MiB/s wr, 137 op/s 2024-05-10T11:30:09.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:30:09 smithi188 ceph-mon[34075]: pgmap v774: 129 pgs: 129 active+clean; 119 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 1.3 MiB/s wr, 137 op/s 2024-05-10T11:30:09.790 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:30:09 smithi100 ceph-mon[30296]: pgmap v774: 129 pgs: 129 active+clean; 119 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 1.3 MiB/s wr, 137 op/s 2024-05-10T11:30:09.982 INFO:tasks.workunit.client.0.smithi100.stdout: 1 308370 2.72 MB/sec execute 484 sec latency 12.862 ms 2024-05-10T11:30:10.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:30:10 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:30:10.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:30:10 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:30:10.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:30:10 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:30:10.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:30:10 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:30:10.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:30:10 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:30:10.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:30:10 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:30:10.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:30:10 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:30:10.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:30:10 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:30:10.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:30:10 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:30:10.982 INFO:tasks.workunit.client.0.smithi100.stdout: 1 308827 2.72 MB/sec execute 485 sec latency 15.737 ms 2024-05-10T11:30:11.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:30:11 smithi178 ceph-mon[33359]: pgmap v775: 129 pgs: 129 active+clean; 120 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 1.3 MiB/s wr, 169 op/s 2024-05-10T11:30:11.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:30:11 smithi188 ceph-mon[34075]: pgmap v775: 129 pgs: 129 active+clean; 120 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 1.3 MiB/s wr, 169 op/s 2024-05-10T11:30:11.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:30:11 smithi100 ceph-mon[30296]: pgmap v775: 129 pgs: 129 active+clean; 120 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 1.3 MiB/s wr, 169 op/s 2024-05-10T11:30:11.982 INFO:tasks.workunit.client.0.smithi100.stdout: 1 309085 2.72 MB/sec execute 486 sec latency 14.505 ms 2024-05-10T11:30:12.638 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.635+0000 7fa85a5a4640 1 -- 172.21.15.100:0/611185422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa854101300 msgr2=0x7fa854101720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:12.638 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.635+0000 7fa85a5a4640 1 --2- 172.21.15.100:0/611185422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa854101300 0x7fa854101720 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fa844009900 tx=0x7fa84402a410 comp rx=0 tx=0).stop 2024-05-10T11:30:12.638 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.635+0000 7fa85a5a4640 1 -- 172.21.15.100:0/611185422 shutdown_connections 2024-05-10T11:30:12.638 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.635+0000 7fa85a5a4640 1 --2- 172.21.15.100:0/611185422 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa854102600 0x7fa854102a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:12.638 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.635+0000 7fa85a5a4640 1 --2- 172.21.15.100:0/611185422 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa854101c60 0x7fa8541020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:12.639 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.635+0000 7fa85a5a4640 1 --2- 172.21.15.100:0/611185422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa854101300 0x7fa854101720 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:12.639 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.635+0000 7fa85a5a4640 1 -- 172.21.15.100:0/611185422 >> 172.21.15.100:0/611185422 conn(0x7fa8540fcea0 msgr2=0x7fa8540ff2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:12.639 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.635+0000 7fa85a5a4640 1 -- 172.21.15.100:0/611185422 shutdown_connections 2024-05-10T11:30:12.639 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.635+0000 7fa85a5a4640 1 -- 172.21.15.100:0/611185422 wait complete. 2024-05-10T11:30:12.639 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.636+0000 7fa85a5a4640 1 Processor -- start 2024-05-10T11:30:12.639 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.636+0000 7fa85a5a4640 1 -- start start 2024-05-10T11:30:12.639 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.636+0000 7fa85a5a4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa854101300 0x7fa85419aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:12.639 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.636+0000 7fa85a5a4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa854101c60 0x7fa85419b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:12.639 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.636+0000 7fa85a5a4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa854102600 0x7fa8541a2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:12.639 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.636+0000 7fa85a5a4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa8541105f0 con 0x7fa854101c60 2024-05-10T11:30:12.639 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.636+0000 7fa85a5a4640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa854110470 con 0x7fa854102600 2024-05-10T11:30:12.639 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.636+0000 7fa85a5a4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa854110770 con 0x7fa854101300 2024-05-10T11:30:12.639 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.636+0000 7fa8537fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa854101c60 0x7fa85419b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:12.639 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.637+0000 7fa8537fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa854101c60 0x7fa85419b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39980/0 (socket says 172.21.15.100:39980) 2024-05-10T11:30:12.640 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.637+0000 7fa8537fe640 1 -- 172.21.15.100:0/2957380312 learned_addr learned my addr 172.21.15.100:0/2957380312 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:30:12.640 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.637+0000 7fa853fff640 1 --2- 172.21.15.100:0/2957380312 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa854101300 0x7fa85419aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:12.640 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.637+0000 7fa858b41640 1 --2- 172.21.15.100:0/2957380312 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa854102600 0x7fa8541a2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:12.640 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.637+0000 7fa8537fe640 1 -- 172.21.15.100:0/2957380312 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa854101300 msgr2=0x7fa85419aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:12.640 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.637+0000 7fa8537fe640 1 --2- 172.21.15.100:0/2957380312 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa854101300 0x7fa85419aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:12.640 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.637+0000 7fa8537fe640 1 -- 172.21.15.100:0/2957380312 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa854102600 msgr2=0x7fa8541a2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:12.640 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.637+0000 7fa8537fe640 1 --2- 172.21.15.100:0/2957380312 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa854102600 0x7fa8541a2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:12.640 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.637+0000 7fa8537fe640 1 -- 172.21.15.100:0/2957380312 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8541a28e0 con 0x7fa854101c60 2024-05-10T11:30:12.640 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.637+0000 7fa8537fe640 1 --2- 172.21.15.100:0/2957380312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa854101c60 0x7fa85419b1e0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fa84800e9f0 tx=0x7fa84800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:12.640 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.637+0000 7fa8517fa640 1 -- 172.21.15.100:0/2957380312 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84800cd80 con 0x7fa854101c60 2024-05-10T11:30:12.640 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.637+0000 7fa85a5a4640 1 -- 172.21.15.100:0/2957380312 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8541a2bd0 con 0x7fa854101c60 2024-05-10T11:30:12.640 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.637+0000 7fa8517fa640 1 -- 172.21.15.100:0/2957380312 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa8480108c0 con 0x7fa854101c60 2024-05-10T11:30:12.640 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.637+0000 7fa8517fa640 1 -- 172.21.15.100:0/2957380312 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8480187c0 con 0x7fa854101c60 2024-05-10T11:30:12.640 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.637+0000 7fa85a5a4640 1 -- 172.21.15.100:0/2957380312 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8541a3170 con 0x7fa854101c60 2024-05-10T11:30:12.647 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.640+0000 7fa8517fa640 1 -- 172.21.15.100:0/2957380312 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fa848010430 con 0x7fa854101c60 2024-05-10T11:30:12.647 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.640+0000 7fa85d396740 1 Processor -- start 2024-05-10T11:30:12.647 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.640+0000 7fa85d396740 1 -- start start 2024-05-10T11:30:12.647 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.640+0000 7fa85d396740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567300c8db0 0x5567300c91d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:12.647 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.640+0000 7fa85d396740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5567300c97a0 0x5567300ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:12.647 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.640+0000 7fa85d396740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5567300ce910 0x5567300d0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.640+0000 7fa85d396740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55673002d5d0 con 0x5567300ce910 2024-05-10T11:30:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.640+0000 7fa85d396740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55672ffe6890 con 0x5567300c97a0 2024-05-10T11:30:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.640+0000 7fa85d396740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556730029590 con 0x5567300c8db0 2024-05-10T11:30:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.640+0000 7fa8537fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5567300ce910 0x5567300d0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.640+0000 7fa853fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5567300c97a0 0x5567300ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.640+0000 7fa8537fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5567300ce910 0x5567300d0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39994/0 (socket says 172.21.15.100:39994) 2024-05-10T11:30:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.640+0000 7fa8537fe640 1 -- 172.21.15.100:0/2615379933 learned_addr learned my addr 172.21.15.100:0/2615379933 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:30:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.640+0000 7fa853fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5567300c97a0 0x5567300ce2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:46450/0 (socket says 172.21.15.100:46450) 2024-05-10T11:30:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.640+0000 7fa858b41640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567300c8db0 0x5567300c91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.641+0000 7fa8517fa640 1 --2- 172.21.15.100:0/2957380312 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa83c0b28a0 0x7fa83c0b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.641+0000 7fa8537fe640 1 -- 172.21.15.100:0/2615379933 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567300c8db0 msgr2=0x5567300c91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.641+0000 7fa8537fe640 1 --2- 172.21.15.100:0/2615379933 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567300c8db0 0x5567300c91d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.641+0000 7fa8537fe640 1 -- 172.21.15.100:0/2615379933 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5567300c97a0 msgr2=0x5567300ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.641+0000 7fa8537fe640 1 --2- 172.21.15.100:0/2615379933 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5567300c97a0 0x5567300ce2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.641+0000 7fa8537fe640 1 -- 172.21.15.100:0/2615379933 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55672fd3f210 con 0x5567300ce910 2024-05-10T11:30:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.641+0000 7fa853fff640 1 --2- 172.21.15.100:0/2615379933 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5567300c97a0 0x5567300ce2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:30:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.641+0000 7fa858b41640 1 --2- 172.21.15.100:0/2957380312 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa83c0b28a0 0x7fa83c0b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.641+0000 7fa8517fa640 1 -- 172.21.15.100:0/2957380312 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fa8480e2900 con 0x7fa854101c60 2024-05-10T11:30:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.641+0000 7fa8537fe640 1 --2- 172.21.15.100:0/2615379933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5567300ce910 0x5567300d0da0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fa8480a9970 tx=0x7fa848024000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=163274823c375e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.641+0000 7fa83a7fc640 1 -- 172.21.15.100:0/2615379933 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8480a5400 con 0x5567300ce910 2024-05-10T11:30:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.641+0000 7fa858b41640 1 --2- 172.21.15.100:0/2957380312 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa83c0b28a0 0x7fa83c0b4d60 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fa84c009fd0 tx=0x7fa84c004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.641+0000 7fa83a7fc640 1 -- 172.21.15.100:0/2615379933 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa8480b2360 con 0x5567300ce910 2024-05-10T11:30:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.641+0000 7fa83a7fc640 1 -- 172.21.15.100:0/2615379933 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8480adf70 con 0x5567300ce910 2024-05-10T11:30:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.642+0000 7fa85d396740 1 -- 172.21.15.100:0/2615379933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5567300ce910 msgr2=0x5567300d0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.642+0000 7fa85d396740 1 --2- 172.21.15.100:0/2615379933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5567300ce910 0x5567300d0da0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fa8480a9970 tx=0x7fa848024000 comp rx=0 tx=0).stop 2024-05-10T11:30:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.642+0000 7fa85d396740 1 -- 172.21.15.100:0/2615379933 shutdown_connections 2024-05-10T11:30:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.642+0000 7fa85d396740 1 --2- 172.21.15.100:0/2615379933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5567300ce910 0x5567300d0da0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.642+0000 7fa85d396740 1 --2- 172.21.15.100:0/2615379933 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5567300c97a0 0x5567300ce2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.642+0000 7fa85d396740 1 --2- 172.21.15.100:0/2615379933 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567300c8db0 0x5567300c91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.642+0000 7fa85d396740 1 -- 172.21.15.100:0/2615379933 >> 172.21.15.100:0/2615379933 conn(0x5567300c4420 msgr2=0x5567300c4820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.642+0000 7fa85d396740 1 -- 172.21.15.100:0/2615379933 shutdown_connections 2024-05-10T11:30:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.642+0000 7fa85d396740 1 -- 172.21.15.100:0/2615379933 wait complete. 2024-05-10T11:30:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.642+0000 7fa85d396740 1 -- start start 2024-05-10T11:30:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.643+0000 7fa85d396740 1 Processor -- start 2024-05-10T11:30:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.643+0000 7fa85d396740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567300c8db0 0x5567300d2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.643+0000 7fa85d396740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5567300c97a0 0x5567300d3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.643+0000 7fa85d396740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5567300ce910 0x5567300da460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.643+0000 7fa85d396740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5567300d1860 con 0x5567300ce910 2024-05-10T11:30:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.643+0000 7fa85d396740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55672ffea370 con 0x5567300c97a0 2024-05-10T11:30:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.643+0000 7fa85d396740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5567300d19e0 con 0x5567300c8db0 2024-05-10T11:30:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.643+0000 7fa85d396740 10 client.0 _ll_register_callbacks cb 0x55672fd3ea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:30:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.643+0000 7fa858b41640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5567300ce910 0x5567300da460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.643+0000 7fa8537fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5567300c97a0 0x5567300d3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.643+0000 7fa858b41640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5567300ce910 0x5567300da460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39996/0 (socket says 172.21.15.100:39996) 2024-05-10T11:30:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.643+0000 7fa8537fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5567300c97a0 0x5567300d3420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:46454/0 (socket says 172.21.15.100:46454) 2024-05-10T11:30:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.643+0000 7fa858b41640 1 -- 172.21.15.100:0/865988853 learned_addr learned my addr 172.21.15.100:0/865988853 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:30:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.643+0000 7fa853fff640 1 --2- 172.21.15.100:0/865988853 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567300c8db0 0x5567300d2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.643+0000 7fa8537fe640 1 -- 172.21.15.100:0/865988853 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567300c8db0 msgr2=0x5567300d2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.643+0000 7fa8537fe640 1 --2- 172.21.15.100:0/865988853 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567300c8db0 0x5567300d2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.643+0000 7fa8537fe640 1 -- 172.21.15.100:0/865988853 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5567300ce910 msgr2=0x5567300da460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.643+0000 7fa8537fe640 1 --2- 172.21.15.100:0/865988853 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5567300ce910 0x5567300da460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.643+0000 7fa8537fe640 1 -- 172.21.15.100:0/865988853 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55672fe0b620 con 0x5567300c97a0 2024-05-10T11:30:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.643+0000 7fa858b41640 1 --2- 172.21.15.100:0/865988853 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5567300ce910 0x5567300da460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:30:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.643+0000 7fa8537fe640 1 --2- 172.21.15.100:0/865988853 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5567300c97a0 0x5567300d3420 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa8480a88f0 tx=0x7fa8480e2c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9eeb313bca305de6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.643+0000 7fa8337fe640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:30:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.644+0000 7fa8337fe640 1 -- 172.21.15.100:0/865988853 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8480a5400 con 0x5567300c97a0 2024-05-10T11:30:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.644+0000 7fa8337fe640 1 -- 172.21.15.100:0/865988853 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fa8480e80c0 con 0x5567300c97a0 2024-05-10T11:30:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.644+0000 7fa85d396740 1 -- 172.21.15.100:0/865988853 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55673003b1e0 con 0x5567300c97a0 2024-05-10T11:30:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.644+0000 7fa8337fe640 1 -- 172.21.15.100:0/865988853 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa8480adf70 con 0x5567300c97a0 2024-05-10T11:30:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.644+0000 7fa85d396740 10 client.27170 fetch_fsmap learned FSMap version 33 2024-05-10T11:30:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.644+0000 7fa85d396740 1 -- 172.21.15.100:0/865988853 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5567300dd440 con 0x5567300c97a0 2024-05-10T11:30:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.644+0000 7fa8337fe640 1 -- 172.21.15.100:0/865988853 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55673003b1e0 con 0x5567300c97a0 2024-05-10T11:30:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.644+0000 7fa8337fe640 1 -- 172.21.15.100:0/865988853 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fa8480b2040 con 0x5567300c97a0 2024-05-10T11:30:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.644+0000 7fa85d396740 10 client.27170 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T11:30:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.644+0000 7fa85d396740 10 client.27170 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:30:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.644+0000 7fa85d396740 20 client.27170 populate_metadata read hostname 'smithi100' 2024-05-10T11:30:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.644+0000 7fa85d396740 1 --2- 172.21.15.100:0/865988853 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5567300de790 0x5567300e0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.644+0000 7fa85d396740 4 client.27170 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:30:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.644+0000 7fa85d396740 1 -- 172.21.15.100:0/865988853 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556730035050 con 0x5567300de790 2024-05-10T11:30:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.645+0000 7fa853fff640 1 --2- 172.21.15.100:0/865988853 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5567300de790 0x5567300e0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.645+0000 7fa853fff640 1 --2- 172.21.15.100:0/865988853 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5567300de790 0x5567300e0b80 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a44c53ac8ce42b15 server_cookie=edbc623c9ebd20fb in_seq=0 out_seq=0 2024-05-10T11:30:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.645+0000 7fa8337fe640 10 client.27170 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:30:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.646+0000 7fa8337fe640 1 -- 172.21.15.100:0/865988853 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556730035050 con 0x5567300de790 2024-05-10T11:30:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.646+0000 7fa8337fe640 10 client.27170 handle_command_reply: tid=0 2024-05-10T11:30:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.646+0000 7fa85d396740 1 client.27170 shutdown 2024-05-10T11:30:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.646+0000 7fa85d396740 1 -- 172.21.15.100:0/865988853 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5567300c97a0 msgr2=0x5567300d3420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.646+0000 7fa85d396740 1 --2- 172.21.15.100:0/865988853 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5567300c97a0 0x5567300d3420 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa8480a88f0 tx=0x7fa8480e2c00 comp rx=0 tx=0).stop 2024-05-10T11:30:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.646+0000 7fa85d396740 1 -- 172.21.15.100:0/865988853 shutdown_connections 2024-05-10T11:30:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.646+0000 7fa85d396740 1 --2- 172.21.15.100:0/865988853 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5567300de790 0x5567300e0b80 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.646+0000 7fa85d396740 1 --2- 172.21.15.100:0/865988853 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5567300ce910 0x5567300da460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:12.654 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.646+0000 7fa8337fe640 0 client.27170 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:30:12.654 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.646+0000 7fa85d396740 1 --2- 172.21.15.100:0/865988853 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5567300c97a0 0x5567300d3420 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:12.654 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.646+0000 7fa85d396740 1 --2- 172.21.15.100:0/865988853 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567300c8db0 0x5567300d2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:12.654 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.646+0000 7fa85d396740 1 -- 172.21.15.100:0/865988853 >> 172.21.15.100:0/865988853 conn(0x5567300c4420 msgr2=0x5567300c3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:12.654 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.646+0000 7fa85d396740 1 -- 172.21.15.100:0/865988853 shutdown_connections 2024-05-10T11:30:12.654 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.646+0000 7fa85d396740 1 -- 172.21.15.100:0/865988853 wait complete. 2024-05-10T11:30:12.654 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.646+0000 7fa85d396740 20 client.27170 trim_cache size 0 max 16384 2024-05-10T11:30:12.665 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.664+0000 7fa85d396740 1 Processor -- start 2024-05-10T11:30:12.665 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.664+0000 7fa85d396740 1 -- start start 2024-05-10T11:30:12.665 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.664+0000 7fa85d396740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567300c8db0 0x5567300c7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:12.665 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.664+0000 7fa85d396740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5567300c97a0 0x5567300c7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:12.665 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.664+0000 7fa85d396740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5567300ce910 0x5567300da420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:12.665 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.664+0000 7fa85d396740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa8541afe10 con 0x5567300ce910 2024-05-10T11:30:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.664+0000 7fa85d396740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa8541ad6e0 con 0x5567300c97a0 2024-05-10T11:30:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.664+0000 7fa85d396740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa8541ab530 con 0x5567300c8db0 2024-05-10T11:30:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.664+0000 7fa858b41640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5567300ce910 0x5567300da420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.664+0000 7fa8537fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5567300c97a0 0x5567300c7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.664+0000 7fa858b41640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5567300ce910 0x5567300da420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39998/0 (socket says 172.21.15.100:39998) 2024-05-10T11:30:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.664+0000 7fa858b41640 1 -- 172.21.15.100:0/3768276182 learned_addr learned my addr 172.21.15.100:0/3768276182 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:30:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.664+0000 7fa853fff640 1 --2- 172.21.15.100:0/3768276182 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567300c8db0 0x5567300c7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.665+0000 7fa8537fe640 1 -- 172.21.15.100:0/3768276182 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567300c8db0 msgr2=0x5567300c7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.665+0000 7fa8537fe640 1 --2- 172.21.15.100:0/3768276182 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567300c8db0 0x5567300c7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.665+0000 7fa8537fe640 1 -- 172.21.15.100:0/3768276182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5567300ce910 msgr2=0x5567300da420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.665+0000 7fa8537fe640 1 --2- 172.21.15.100:0/3768276182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5567300ce910 0x5567300da420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.665+0000 7fa8537fe640 1 -- 172.21.15.100:0/3768276182 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567300e11b0 con 0x5567300c97a0 2024-05-10T11:30:12.667 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.665+0000 7fa8537fe640 1 --2- 172.21.15.100:0/3768276182 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5567300c97a0 0x5567300c7d50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa8480a6ee0 tx=0x7fa8480e2c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5fac6088e11ae server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:12.667 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.665+0000 7fa839ffb640 1 -- 172.21.15.100:0/3768276182 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8480a5400 con 0x5567300c97a0 2024-05-10T11:30:12.667 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.665+0000 7fa839ffb640 1 -- 172.21.15.100:0/3768276182 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa8480a55e0 con 0x5567300c97a0 2024-05-10T11:30:12.667 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.665+0000 7fa839ffb640 1 -- 172.21.15.100:0/3768276182 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8480a8ba0 con 0x5567300c97a0 2024-05-10T11:30:12.670 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.666+0000 7fa85d396740 1 -- 172.21.15.100:0/3768276182 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5567300c97a0 msgr2=0x5567300c7d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:12.670 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.666+0000 7fa85d396740 1 --2- 172.21.15.100:0/3768276182 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5567300c97a0 0x5567300c7d50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa8480a6ee0 tx=0x7fa8480e2c70 comp rx=0 tx=0).stop 2024-05-10T11:30:12.670 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.666+0000 7fa85d396740 1 -- 172.21.15.100:0/3768276182 shutdown_connections 2024-05-10T11:30:12.670 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.666+0000 7fa85d396740 1 --2- 172.21.15.100:0/3768276182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5567300ce910 0x5567300da420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:12.670 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.666+0000 7fa85d396740 1 --2- 172.21.15.100:0/3768276182 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5567300c97a0 0x5567300c7d50 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:12.671 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.666+0000 7fa85d396740 1 --2- 172.21.15.100:0/3768276182 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567300c8db0 0x5567300c7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:12.671 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.666+0000 7fa85d396740 1 -- 172.21.15.100:0/3768276182 >> 172.21.15.100:0/3768276182 conn(0x5567300c4420 msgr2=0x5567300c7100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:12.671 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.666+0000 7fa85d396740 1 -- 172.21.15.100:0/3768276182 shutdown_connections 2024-05-10T11:30:12.671 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.666+0000 7fa85d396740 1 -- 172.21.15.100:0/3768276182 wait complete. 2024-05-10T11:30:12.671 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.666+0000 7fa85d396740 1 -- start start 2024-05-10T11:30:12.671 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.667+0000 7fa85d396740 1 Processor -- start 2024-05-10T11:30:12.671 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.667+0000 7fa85d396740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567300c8db0 0x5567300d7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:12.671 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.667+0000 7fa85d396740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5567300c97a0 0x5567300d80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:12.671 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.667+0000 7fa85d396740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5567300ce910 0x5567300d90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:12.671 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.667+0000 7fa85d396740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa8541abba0 con 0x5567300c97a0 2024-05-10T11:30:12.671 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.667+0000 7fa85d396740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa8541ad010 con 0x5567300ce910 2024-05-10T11:30:12.671 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.667+0000 7fa85d396740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa8541ac270 con 0x5567300c8db0 2024-05-10T11:30:12.671 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.667+0000 7fa85d396740 10 client.0 _ll_register_callbacks cb 0x7fa84c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:30:12.671 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.667+0000 7fa8537fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5567300c97a0 0x5567300d80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:12.672 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.667+0000 7fa8537fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5567300c97a0 0x5567300d80d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40014/0 (socket says 172.21.15.100:40014) 2024-05-10T11:30:12.672 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.667+0000 7fa8537fe640 1 -- 172.21.15.100:0/662114583 learned_addr learned my addr 172.21.15.100:0/662114583 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:30:12.672 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.667+0000 7fa853fff640 1 --2- 172.21.15.100:0/662114583 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567300c8db0 0x5567300d7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:12.672 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.667+0000 7fa858b41640 1 --2- 172.21.15.100:0/662114583 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5567300ce910 0x5567300d90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:12.672 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.667+0000 7fa8537fe640 1 -- 172.21.15.100:0/662114583 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567300c8db0 msgr2=0x5567300d7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:12.672 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.667+0000 7fa8537fe640 1 --2- 172.21.15.100:0/662114583 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567300c8db0 0x5567300d7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:12.672 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.667+0000 7fa8537fe640 1 -- 172.21.15.100:0/662114583 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5567300ce910 msgr2=0x5567300d90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:12.672 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.667+0000 7fa8537fe640 1 --2- 172.21.15.100:0/662114583 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5567300ce910 0x5567300d90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:12.672 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.667+0000 7fa8537fe640 1 -- 172.21.15.100:0/662114583 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567300d96d0 con 0x5567300c97a0 2024-05-10T11:30:12.672 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.667+0000 7fa853fff640 1 --2- 172.21.15.100:0/662114583 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567300c8db0 0x5567300d7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:30:12.672 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.667+0000 7fa858b41640 1 --2- 172.21.15.100:0/662114583 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5567300ce910 0x5567300d90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:30:12.672 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.668+0000 7fa8537fe640 1 --2- 172.21.15.100:0/662114583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5567300c97a0 0x5567300d80d0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fa8480e2d80 tx=0x7fa8480a8dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=890b4d84e5f1cf90 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:12.672 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.668+0000 7fa8337fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:30:12.672 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.668+0000 7fa8337fe640 1 -- 172.21.15.100:0/662114583 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8480a5400 con 0x5567300c97a0 2024-05-10T11:30:12.672 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.668+0000 7fa85d396740 1 -- 172.21.15.100:0/662114583 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8480a55e0 con 0x5567300c97a0 2024-05-10T11:30:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.668+0000 7fa8337fe640 1 -- 172.21.15.100:0/662114583 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fa8480af310 con 0x5567300c97a0 2024-05-10T11:30:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.668+0000 7fa8337fe640 1 -- 172.21.15.100:0/662114583 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa8480a9200 con 0x5567300c97a0 2024-05-10T11:30:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.668+0000 7fa85d396740 10 client.17823 fetch_fsmap learned FSMap version 33 2024-05-10T11:30:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.668+0000 7fa85d396740 1 -- 172.21.15.100:0/662114583 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5567300d96d0 con 0x5567300c97a0 2024-05-10T11:30:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.668+0000 7fa8337fe640 1 -- 172.21.15.100:0/662114583 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa8480a55e0 con 0x5567300c97a0 2024-05-10T11:30:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.668+0000 7fa8337fe640 1 -- 172.21.15.100:0/662114583 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fa848005020 con 0x5567300c97a0 2024-05-10T11:30:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.668+0000 7fa85d396740 10 client.17823 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T11:30:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.668+0000 7fa85d396740 10 client.17823 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:30:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.668+0000 7fa85d396740 20 client.17823 populate_metadata read hostname 'smithi100' 2024-05-10T11:30:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.668+0000 7fa85d396740 1 --2- 172.21.15.100:0/662114583 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556730149630 0x55673014ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.668+0000 7fa85d396740 4 client.17823 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:30:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.668+0000 7fa85d396740 1 -- 172.21.15.100:0/662114583 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55672ffb05f0 con 0x556730149630 2024-05-10T11:30:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.669+0000 7fa853fff640 1 --2- 172.21.15.100:0/662114583 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556730149630 0x55673014ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.669+0000 7fa8337fe640 10 client.17823 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:30:12.674 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.669+0000 7fa853fff640 1 --2- 172.21.15.100:0/662114583 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556730149630 0x55673014ba50 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97afb5cd42d51d0a server_cookie=87840ffd405d6cae in_seq=0 out_seq=0 2024-05-10T11:30:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.691+0000 7fa8337fe640 1 -- 172.21.15.100:0/662114583 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55672ffb05f0 con 0x556730149630 2024-05-10T11:30:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.691+0000 7fa8337fe640 10 client.17823 handle_command_reply: tid=0 2024-05-10T11:30:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.691+0000 7fa85d396740 1 client.17823 shutdown 2024-05-10T11:30:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.691+0000 7fa85d396740 1 -- 172.21.15.100:0/662114583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5567300c97a0 msgr2=0x5567300d80d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.691+0000 7fa85d396740 1 --2- 172.21.15.100:0/662114583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5567300c97a0 0x5567300d80d0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fa8480e2d80 tx=0x7fa8480a8dc0 comp rx=0 tx=0).stop 2024-05-10T11:30:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.691+0000 7fa85d396740 1 -- 172.21.15.100:0/662114583 shutdown_connections 2024-05-10T11:30:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.691+0000 7fa85d396740 1 --2- 172.21.15.100:0/662114583 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556730149630 0x55673014ba50 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.691+0000 7fa85d396740 1 --2- 172.21.15.100:0/662114583 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5567300ce910 0x5567300d90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.691+0000 7fa8337fe640 0 client.17823 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:30:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.691+0000 7fa85d396740 1 --2- 172.21.15.100:0/662114583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5567300c97a0 0x5567300d80d0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.691+0000 7fa85d396740 1 --2- 172.21.15.100:0/662114583 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567300c8db0 0x5567300d7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.691+0000 7fa85d396740 1 -- 172.21.15.100:0/662114583 >> 172.21.15.100:0/662114583 conn(0x5567300c4420 msgr2=0x5567300c7990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.691+0000 7fa85d396740 1 -- 172.21.15.100:0/662114583 shutdown_connections 2024-05-10T11:30:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.691+0000 7fa85d396740 1 -- 172.21.15.100:0/662114583 wait complete. 2024-05-10T11:30:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.692+0000 7fa85d396740 20 client.17823 trim_cache size 0 max 16384 2024-05-10T11:30:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:30:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:30:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:30:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:30:12.694 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.692+0000 7fa85a5a4640 1 -- 172.21.15.100:0/2957380312 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa83c0b28a0 msgr2=0x7fa83c0b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:12.694 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.692+0000 7fa85a5a4640 1 --2- 172.21.15.100:0/2957380312 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa83c0b28a0 0x7fa83c0b4d60 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fa84c009fd0 tx=0x7fa84c004a40 comp rx=0 tx=0).stop 2024-05-10T11:30:12.694 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.692+0000 7fa85a5a4640 1 -- 172.21.15.100:0/2957380312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa854101c60 msgr2=0x7fa85419b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:12.694 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.692+0000 7fa85a5a4640 1 --2- 172.21.15.100:0/2957380312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa854101c60 0x7fa85419b1e0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fa84800e9f0 tx=0x7fa84800eeb0 comp rx=0 tx=0).stop 2024-05-10T11:30:12.694 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.692+0000 7fa85a5a4640 1 -- 172.21.15.100:0/2957380312 shutdown_connections 2024-05-10T11:30:12.694 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.692+0000 7fa85a5a4640 1 --2- 172.21.15.100:0/2957380312 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa83c0b28a0 0x7fa83c0b4d60 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:12.694 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.692+0000 7fa85a5a4640 1 --2- 172.21.15.100:0/2957380312 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa854102600 0x7fa8541a2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:12.694 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.692+0000 7fa85a5a4640 1 --2- 172.21.15.100:0/2957380312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa854101c60 0x7fa85419b1e0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:12.695 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.692+0000 7fa85a5a4640 1 --2- 172.21.15.100:0/2957380312 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa854101300 0x7fa85419aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:12.695 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.692+0000 7fa85a5a4640 1 -- 172.21.15.100:0/2957380312 >> 172.21.15.100:0/2957380312 conn(0x7fa8540fcea0 msgr2=0x7fa8540fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:12.695 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.692+0000 7fa85a5a4640 1 -- 172.21.15.100:0/2957380312 shutdown_connections 2024-05-10T11:30:12.695 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:12.692+0000 7fa85a5a4640 1 -- 172.21.15.100:0/2957380312 wait complete. 2024-05-10T11:30:12.982 INFO:tasks.workunit.client.0.smithi100.stdout: 1 309520 2.71 MB/sec execute 487 sec latency 17.694 ms 2024-05-10T11:30:13.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:30:13 smithi178 ceph-mon[33359]: pgmap v776: 129 pgs: 129 active+clean; 121 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 1.4 MiB/s wr, 141 op/s 2024-05-10T11:30:13.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:30:13 smithi188 ceph-mon[34075]: pgmap v776: 129 pgs: 129 active+clean; 121 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 1.4 MiB/s wr, 141 op/s 2024-05-10T11:30:13.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:30:13 smithi100 ceph-mon[30296]: pgmap v776: 129 pgs: 129 active+clean; 121 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 1.4 MiB/s wr, 141 op/s 2024-05-10T11:30:13.982 INFO:tasks.workunit.client.0.smithi100.stdout: 1 310077 2.72 MB/sec execute 488 sec latency 16.162 ms 2024-05-10T11:30:14.982 INFO:tasks.workunit.client.0.smithi100.stdout: 1 310754 2.71 MB/sec execute 489 sec latency 16.024 ms 2024-05-10T11:30:15.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:30:15 smithi178 ceph-mon[33359]: pgmap v777: 129 pgs: 129 active+clean; 121 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 1.4 MiB/s wr, 141 op/s 2024-05-10T11:30:15.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:30:15 smithi188 ceph-mon[34075]: pgmap v777: 129 pgs: 129 active+clean; 121 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 1.4 MiB/s wr, 141 op/s 2024-05-10T11:30:15.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:30:15 smithi100 ceph-mon[30296]: pgmap v777: 129 pgs: 129 active+clean; 121 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 1.4 MiB/s wr, 141 op/s 2024-05-10T11:30:15.982 INFO:tasks.workunit.client.0.smithi100.stdout: 1 311221 2.71 MB/sec execute 490 sec latency 15.170 ms 2024-05-10T11:30:16.982 INFO:tasks.workunit.client.0.smithi100.stdout: 1 311929 2.72 MB/sec execute 491 sec latency 15.172 ms 2024-05-10T11:30:17.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:30:17 smithi178 ceph-mon[33359]: pgmap v778: 129 pgs: 129 active+clean; 120 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 1.3 MiB/s wr, 164 op/s 2024-05-10T11:30:17.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:30:17 smithi188 ceph-mon[34075]: pgmap v778: 129 pgs: 129 active+clean; 120 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 1.3 MiB/s wr, 164 op/s 2024-05-10T11:30:17.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:30:17 smithi100 ceph-mon[30296]: pgmap v778: 129 pgs: 129 active+clean; 120 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 1.3 MiB/s wr, 164 op/s 2024-05-10T11:30:17.982 INFO:tasks.workunit.client.0.smithi100.stdout: 1 312384 2.72 MB/sec execute 492 sec latency 18.475 ms 2024-05-10T11:30:18.982 INFO:tasks.workunit.client.0.smithi100.stdout: 1 312629 2.72 MB/sec execute 493 sec latency 15.340 ms 2024-05-10T11:30:19.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:30:19 smithi178 ceph-mon[33359]: pgmap v779: 129 pgs: 129 active+clean; 120 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-05-10T11:30:19.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:30:19 smithi188 ceph-mon[34075]: pgmap v779: 129 pgs: 129 active+clean; 120 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-05-10T11:30:19.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:30:19 smithi100 ceph-mon[30296]: pgmap v779: 129 pgs: 129 active+clean; 120 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-05-10T11:30:19.982 INFO:tasks.workunit.client.0.smithi100.stdout: 1 313058 2.71 MB/sec execute 494 sec latency 15.886 ms 2024-05-10T11:30:20.982 INFO:tasks.workunit.client.0.smithi100.stdout: 1 313609 2.72 MB/sec execute 495 sec latency 15.827 ms 2024-05-10T11:30:21.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:30:21 smithi178 ceph-mon[33359]: pgmap v780: 129 pgs: 129 active+clean; 121 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 1.3 MiB/s wr, 162 op/s 2024-05-10T11:30:21.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:30:21 smithi188 ceph-mon[34075]: pgmap v780: 129 pgs: 129 active+clean; 121 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 1.3 MiB/s wr, 162 op/s 2024-05-10T11:30:21.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:30:21 smithi100 ceph-mon[30296]: pgmap v780: 129 pgs: 129 active+clean; 121 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 1.3 MiB/s wr, 162 op/s 2024-05-10T11:30:21.982 INFO:tasks.workunit.client.0.smithi100.stdout: 1 314254 2.71 MB/sec execute 496 sec latency 10.861 ms 2024-05-10T11:30:22.983 INFO:tasks.workunit.client.0.smithi100.stdout: 1 314770 2.71 MB/sec execute 497 sec latency 15.421 ms 2024-05-10T11:30:23.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:30:23 smithi178 ceph-mon[33359]: pgmap v781: 129 pgs: 129 active+clean; 123 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 1.4 MiB/s wr, 151 op/s 2024-05-10T11:30:23.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:30:23 smithi188 ceph-mon[34075]: pgmap v781: 129 pgs: 129 active+clean; 123 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 1.4 MiB/s wr, 151 op/s 2024-05-10T11:30:23.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:30:23 smithi100 ceph-mon[30296]: pgmap v781: 129 pgs: 129 active+clean; 123 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 1.4 MiB/s wr, 151 op/s 2024-05-10T11:30:23.983 INFO:tasks.workunit.client.0.smithi100.stdout: 1 315472 2.72 MB/sec execute 498 sec latency 18.404 ms 2024-05-10T11:30:24.983 INFO:tasks.workunit.client.0.smithi100.stdout: 1 315941 2.72 MB/sec execute 499 sec latency 15.716 ms 2024-05-10T11:30:25.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:30:25 smithi178 ceph-mon[33359]: pgmap v782: 129 pgs: 129 active+clean; 124 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 1.3 MiB/s wr, 141 op/s 2024-05-10T11:30:25.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:30:25 smithi188 ceph-mon[34075]: pgmap v782: 129 pgs: 129 active+clean; 124 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 1.3 MiB/s wr, 141 op/s 2024-05-10T11:30:25.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:30:25 smithi100 ceph-mon[30296]: pgmap v782: 129 pgs: 129 active+clean; 124 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 1.3 MiB/s wr, 141 op/s 2024-05-10T11:30:25.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.788+0000 7f1f6a209640 1 -- 172.21.15.100:0/4189338821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f641086f0 msgr2=0x7f1f6410ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:25.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.788+0000 7f1f6a209640 1 --2- 172.21.15.100:0/4189338821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f641086f0 0x7f1f6410ab40 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c009ca0 tx=0x7f1f5c02a8c0 comp rx=0 tx=0).stop 2024-05-10T11:30:25.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.788+0000 7f1f6a209640 1 -- 172.21.15.100:0/4189338821 shutdown_connections 2024-05-10T11:30:25.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.788+0000 7f1f6a209640 1 --2- 172.21.15.100:0/4189338821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f641086f0 0x7f1f6410ab40 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:25.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.788+0000 7f1f6a209640 1 --2- 172.21.15.100:0/4189338821 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f64105dc0 0x7f1f641081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:25.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.788+0000 7f1f6a209640 1 --2- 172.21.15.100:0/4189338821 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f64103490 0x7f1f64105880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:25.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.788+0000 7f1f6a209640 1 -- 172.21.15.100:0/4189338821 >> 172.21.15.100:0/4189338821 conn(0x7f1f640fce80 msgr2=0x7f1f640ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:25.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.788+0000 7f1f6a209640 1 -- 172.21.15.100:0/4189338821 shutdown_connections 2024-05-10T11:30:25.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.788+0000 7f1f6a209640 1 -- 172.21.15.100:0/4189338821 wait complete. 2024-05-10T11:30:25.792 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.788+0000 7f1f6a209640 1 Processor -- start 2024-05-10T11:30:25.792 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.789+0000 7f1f6a209640 1 -- start start 2024-05-10T11:30:25.792 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.789+0000 7f1f6a209640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f64103490 0x7f1f64198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:25.792 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.789+0000 7f1f6a209640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f64105dc0 0x7f1f64198fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:25.792 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.789+0000 7f1f6a209640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f641086f0 0x7f1f641a0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:25.792 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.789+0000 7f1f6a209640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6410e460 con 0x7f1f64103490 2024-05-10T11:30:25.792 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.789+0000 7f1f6a209640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6410e2e0 con 0x7f1f641086f0 2024-05-10T11:30:25.792 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.789+0000 7f1f6a209640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6410e5e0 con 0x7f1f64105dc0 2024-05-10T11:30:25.792 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.789+0000 7f1f637fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f64103490 0x7f1f64198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:25.792 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.789+0000 7f1f637fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f64103490 0x7f1f64198a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52052/0 (socket says 172.21.15.100:52052) 2024-05-10T11:30:25.792 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.789+0000 7f1f637fe640 1 -- 172.21.15.100:0/2116530245 learned_addr learned my addr 172.21.15.100:0/2116530245 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:30:25.793 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.789+0000 7f1f62ffd640 1 --2- 172.21.15.100:0/2116530245 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f64105dc0 0x7f1f64198fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:25.793 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.789+0000 7f1f63fff640 1 --2- 172.21.15.100:0/2116530245 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f641086f0 0x7f1f641a0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:25.793 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.789+0000 7f1f637fe640 1 -- 172.21.15.100:0/2116530245 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f64105dc0 msgr2=0x7f1f64198fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:25.793 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.789+0000 7f1f637fe640 1 --2- 172.21.15.100:0/2116530245 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f64105dc0 0x7f1f64198fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:25.793 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.789+0000 7f1f637fe640 1 -- 172.21.15.100:0/2116530245 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f641086f0 msgr2=0x7f1f641a0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:25.793 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.789+0000 7f1f637fe640 1 --2- 172.21.15.100:0/2116530245 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f641086f0 0x7f1f641a0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:25.793 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.789+0000 7f1f637fe640 1 -- 172.21.15.100:0/2116530245 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f641a06a0 con 0x7f1f64103490 2024-05-10T11:30:25.793 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.789+0000 7f1f637fe640 1 --2- 172.21.15.100:0/2116530245 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f64103490 0x7f1f64198a80 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f1f5400e9f0 tx=0x7f1f5400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:25.793 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.789+0000 7f1f60ff9640 1 -- 172.21.15.100:0/2116530245 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5400cd80 con 0x7f1f64103490 2024-05-10T11:30:25.793 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.789+0000 7f1f6a209640 1 -- 172.21.15.100:0/2116530245 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f641a0990 con 0x7f1f64103490 2024-05-10T11:30:25.793 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.789+0000 7f1f60ff9640 1 -- 172.21.15.100:0/2116530245 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1f540054b0 con 0x7f1f64103490 2024-05-10T11:30:25.793 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.790+0000 7f1f60ff9640 1 -- 172.21.15.100:0/2116530245 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f54010430 con 0x7f1f64103490 2024-05-10T11:30:25.793 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.790+0000 7f1f6a209640 1 -- 172.21.15.100:0/2116530245 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f641a0e90 con 0x7f1f64103490 2024-05-10T11:30:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.792+0000 7f1f60ff9640 1 -- 172.21.15.100:0/2116530245 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f1f540046c0 con 0x7f1f64103490 2024-05-10T11:30:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.792+0000 7f1f6d004740 1 Processor -- start 2024-05-10T11:30:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.792+0000 7f1f6d004740 1 -- start start 2024-05-10T11:30:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.793+0000 7f1f6d004740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac5aa3edb0 0x55ac5aa3f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.793+0000 7f1f6d004740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ac5aa3f7a0 0x55ac5aa442a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.793+0000 7f1f6d004740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac5aa44910 0x55ac5aa46da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.793+0000 7f1f6d004740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55ac5a9a35d0 con 0x55ac5aa3edb0 2024-05-10T11:30:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.793+0000 7f1f6d004740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ac5a95c890 con 0x55ac5aa44910 2024-05-10T11:30:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.793+0000 7f1f6d004740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55ac5a99f590 con 0x55ac5aa3f7a0 2024-05-10T11:30:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.793+0000 7f1f63fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac5aa3edb0 0x55ac5aa3f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.793+0000 7f1f637fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ac5aa3f7a0 0x55ac5aa442a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.793+0000 7f1f63fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac5aa3edb0 0x55ac5aa3f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52056/0 (socket says 172.21.15.100:52056) 2024-05-10T11:30:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.793+0000 7f1f637fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ac5aa3f7a0 0x55ac5aa442a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:43014/0 (socket says 172.21.15.100:43014) 2024-05-10T11:30:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.793+0000 7f1f63fff640 1 -- 172.21.15.100:0/1094089894 learned_addr learned my addr 172.21.15.100:0/1094089894 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:30:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.793+0000 7f1f62ffd640 1 --2- 172.21.15.100:0/1094089894 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac5aa44910 0x55ac5aa46da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.793+0000 7f1f63fff640 1 -- 172.21.15.100:0/1094089894 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ac5aa3f7a0 msgr2=0x55ac5aa442a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.793+0000 7f1f63fff640 1 --2- 172.21.15.100:0/1094089894 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ac5aa3f7a0 0x55ac5aa442a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:25.803 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.793+0000 7f1f63fff640 1 -- 172.21.15.100:0/1094089894 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac5aa44910 msgr2=0x55ac5aa46da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:25.803 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.793+0000 7f1f63fff640 1 --2- 172.21.15.100:0/1094089894 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac5aa44910 0x55ac5aa46da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:25.803 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.793+0000 7f1f63fff640 1 -- 172.21.15.100:0/1094089894 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac5a6b5210 con 0x55ac5aa3edb0 2024-05-10T11:30:25.803 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.793+0000 7f1f60ff9640 1 --2- 172.21.15.100:0/2116530245 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f1f4c0b28a0 0x7f1f4c0b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:25.803 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.793+0000 7f1f637fe640 1 --2- 172.21.15.100:0/2116530245 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f1f4c0b28a0 0x7f1f4c0b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:25.803 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.793+0000 7f1f63fff640 1 --2- 172.21.15.100:0/1094089894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac5aa3edb0 0x55ac5aa3f1d0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c02ade0 tx=0x7f1f5c005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f127c7ab10cb725 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:25.803 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.793+0000 7f1f60ff9640 1 -- 172.21.15.100:0/2116530245 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f1f540e28b0 con 0x7f1f64103490 2024-05-10T11:30:25.803 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.794+0000 7f1f49ffb640 1 -- 172.21.15.100:0/1094089894 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c0316d0 con 0x55ac5aa3edb0 2024-05-10T11:30:25.803 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.794+0000 7f1f49ffb640 1 -- 172.21.15.100:0/1094089894 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1f5c036030 con 0x55ac5aa3edb0 2024-05-10T11:30:25.803 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.794+0000 7f1f49ffb640 1 -- 172.21.15.100:0/1094089894 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c0043a0 con 0x55ac5aa3edb0 2024-05-10T11:30:25.803 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.794+0000 7f1f6d004740 1 -- 172.21.15.100:0/1094089894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac5aa3edb0 msgr2=0x55ac5aa3f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:25.803 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.794+0000 7f1f637fe640 1 --2- 172.21.15.100:0/2116530245 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f1f4c0b28a0 0x7f1f4c0b4d60 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1f540e03c0 tx=0x7f1f540e8550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:25.804 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.794+0000 7f1f6d004740 1 --2- 172.21.15.100:0/1094089894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac5aa3edb0 0x55ac5aa3f1d0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c02ade0 tx=0x7f1f5c005ef0 comp rx=0 tx=0).stop 2024-05-10T11:30:25.804 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.794+0000 7f1f6d004740 1 -- 172.21.15.100:0/1094089894 shutdown_connections 2024-05-10T11:30:25.804 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.794+0000 7f1f6d004740 1 --2- 172.21.15.100:0/1094089894 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac5aa44910 0x55ac5aa46da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:25.804 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.794+0000 7f1f6d004740 1 --2- 172.21.15.100:0/1094089894 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ac5aa3f7a0 0x55ac5aa442a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:25.804 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.794+0000 7f1f6d004740 1 --2- 172.21.15.100:0/1094089894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac5aa3edb0 0x55ac5aa3f1d0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:25.804 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.794+0000 7f1f6d004740 1 -- 172.21.15.100:0/1094089894 >> 172.21.15.100:0/1094089894 conn(0x55ac5aa3a420 msgr2=0x55ac5aa3a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:25.804 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.794+0000 7f1f6d004740 1 -- 172.21.15.100:0/1094089894 shutdown_connections 2024-05-10T11:30:25.804 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.794+0000 7f1f6d004740 1 -- 172.21.15.100:0/1094089894 wait complete. 2024-05-10T11:30:25.804 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.795+0000 7f1f6d004740 1 -- start start 2024-05-10T11:30:25.804 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.795+0000 7f1f6d004740 1 Processor -- start 2024-05-10T11:30:25.804 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.795+0000 7f1f6d004740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac5aa3edb0 0x55ac5aa48df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:25.804 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.795+0000 7f1f6d004740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ac5aa3f7a0 0x55ac5aa49330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:25.804 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.795+0000 7f1f6d004740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac5aa44910 0x55ac5aa50370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:25.804 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.795+0000 7f1f6d004740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55ac5aa476e0 con 0x55ac5aa44910 2024-05-10T11:30:25.805 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.795+0000 7f1f6d004740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ac5a960370 con 0x55ac5aa3edb0 2024-05-10T11:30:25.805 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.795+0000 7f1f6d004740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55ac5aa47860 con 0x55ac5aa3f7a0 2024-05-10T11:30:25.805 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.795+0000 7f1f6d004740 10 client.0 _ll_register_callbacks cb 0x55ac5a6b4a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:30:25.805 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.795+0000 7f1f63fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac5aa44910 0x55ac5aa50370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:25.805 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.795+0000 7f1f62ffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ac5aa3f7a0 0x55ac5aa49330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:25.805 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.795+0000 7f1f63fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac5aa44910 0x55ac5aa50370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52070/0 (socket says 172.21.15.100:52070) 2024-05-10T11:30:25.805 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.795+0000 7f1f63fff640 1 -- 172.21.15.100:0/714183956 learned_addr learned my addr 172.21.15.100:0/714183956 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:30:25.805 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.795+0000 7f1f63fff640 1 --2- 172.21.15.100:0/714183956 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac5aa3edb0 0x55ac5aa48df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:25.805 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.796+0000 7f1f62ffd640 1 -- 172.21.15.100:0/714183956 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac5aa3edb0 msgr2=0x55ac5aa48df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:25.805 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.796+0000 7f1f62ffd640 1 --2- 172.21.15.100:0/714183956 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac5aa3edb0 0x55ac5aa48df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:25.805 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.796+0000 7f1f62ffd640 1 -- 172.21.15.100:0/714183956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac5aa44910 msgr2=0x55ac5aa50370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:25.805 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.796+0000 7f1f62ffd640 1 --2- 172.21.15.100:0/714183956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac5aa44910 0x55ac5aa50370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:25.805 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.796+0000 7f1f62ffd640 1 -- 172.21.15.100:0/714183956 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac5a781620 con 0x55ac5aa3f7a0 2024-05-10T11:30:25.805 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.796+0000 7f1f62ffd640 1 --2- 172.21.15.100:0/714183956 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ac5aa3f7a0 0x55ac5aa49330 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1f64199f90 tx=0x7f1f58012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8dab16c8801f8c51 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:25.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.796+0000 7f1f42ffd640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:30:25.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.796+0000 7f1f42ffd640 1 -- 172.21.15.100:0/714183956 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5800f890 con 0x55ac5aa3f7a0 2024-05-10T11:30:25.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.796+0000 7f1f6d004740 1 -- 172.21.15.100:0/714183956 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f5c036030 con 0x55ac5aa3f7a0 2024-05-10T11:30:25.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.796+0000 7f1f42ffd640 1 -- 172.21.15.100:0/714183956 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f1f5800fa30 con 0x55ac5aa3f7a0 2024-05-10T11:30:25.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.796+0000 7f1f42ffd640 1 -- 172.21.15.100:0/714183956 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1f5800fc30 con 0x55ac5aa3f7a0 2024-05-10T11:30:25.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.796+0000 7f1f6d004740 10 client.27286 fetch_fsmap learned FSMap version 33 2024-05-10T11:30:25.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.796+0000 7f1f6d004740 1 -- 172.21.15.100:0/714183956 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ac5aa53470 con 0x55ac5aa3f7a0 2024-05-10T11:30:25.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.796+0000 7f1f42ffd640 1 -- 172.21.15.100:0/714183956 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1f5c036030 con 0x55ac5aa3f7a0 2024-05-10T11:30:25.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.797+0000 7f1f42ffd640 1 -- 172.21.15.100:0/714183956 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f1f5801c420 con 0x55ac5aa3f7a0 2024-05-10T11:30:25.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.797+0000 7f1f6d004740 10 client.27286 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T11:30:25.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.797+0000 7f1f6d004740 10 client.27286 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:30:25.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.797+0000 7f1f6d004740 20 client.27286 populate_metadata read hostname 'smithi100' 2024-05-10T11:30:25.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.797+0000 7f1f6d004740 1 --2- 172.21.15.100:0/714183956 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ac5aa54c70 0x55ac5aa57080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:25.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.797+0000 7f1f6d004740 4 client.27286 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:30:25.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.797+0000 7f1f6d004740 1 -- 172.21.15.100:0/714183956 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac5a9b11e0 con 0x55ac5aa54c70 2024-05-10T11:30:25.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.797+0000 7f1f63fff640 1 --2- 172.21.15.100:0/714183956 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ac5aa54c70 0x55ac5aa57080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:25.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.797+0000 7f1f63fff640 1 --2- 172.21.15.100:0/714183956 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ac5aa54c70 0x55ac5aa57080 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5123f6fe5bcafae5 server_cookie=72cbc714f5c90911 in_seq=0 out_seq=0 2024-05-10T11:30:25.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.797+0000 7f1f42ffd640 10 client.27286 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:30:25.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.798+0000 7f1f42ffd640 1 -- 172.21.15.100:0/714183956 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ac5a9b11e0 con 0x55ac5aa54c70 2024-05-10T11:30:25.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.798+0000 7f1f42ffd640 10 client.27286 handle_command_reply: tid=0 2024-05-10T11:30:25.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.798+0000 7f1f6d004740 1 client.27286 shutdown 2024-05-10T11:30:25.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.798+0000 7f1f6d004740 1 -- 172.21.15.100:0/714183956 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ac5aa3f7a0 msgr2=0x55ac5aa49330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:25.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.798+0000 7f1f6d004740 1 --2- 172.21.15.100:0/714183956 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ac5aa3f7a0 0x55ac5aa49330 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1f64199f90 tx=0x7f1f58012b60 comp rx=0 tx=0).stop 2024-05-10T11:30:25.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.798+0000 7f1f6d004740 1 -- 172.21.15.100:0/714183956 shutdown_connections 2024-05-10T11:30:25.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.798+0000 7f1f6d004740 1 --2- 172.21.15.100:0/714183956 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ac5aa54c70 0x55ac5aa57080 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:25.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.798+0000 7f1f6d004740 1 --2- 172.21.15.100:0/714183956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac5aa44910 0x55ac5aa50370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:25.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.798+0000 7f1f6d004740 1 --2- 172.21.15.100:0/714183956 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ac5aa3f7a0 0x55ac5aa49330 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:25.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.798+0000 7f1f42ffd640 0 client.27286 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:30:25.808 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.798+0000 7f1f6d004740 1 --2- 172.21.15.100:0/714183956 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac5aa3edb0 0x55ac5aa48df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:25.808 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.798+0000 7f1f6d004740 1 -- 172.21.15.100:0/714183956 >> 172.21.15.100:0/714183956 conn(0x55ac5aa3a420 msgr2=0x55ac5aa39580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:25.808 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.798+0000 7f1f6d004740 1 -- 172.21.15.100:0/714183956 shutdown_connections 2024-05-10T11:30:25.808 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.798+0000 7f1f6d004740 1 -- 172.21.15.100:0/714183956 wait complete. 2024-05-10T11:30:25.808 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.798+0000 7f1f6d004740 20 client.27286 trim_cache size 0 max 16384 2024-05-10T11:30:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.814+0000 7f1f6d004740 1 Processor -- start 2024-05-10T11:30:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.814+0000 7f1f6d004740 1 -- start start 2024-05-10T11:30:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.815+0000 7f1f6d004740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac5aa3edb0 0x55ac5aa40980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.815+0000 7f1f6d004740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ac5aa3f7a0 0x55ac5aa40ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.815+0000 7f1f6d004740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac5aa44910 0x55ac5aa50330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.815+0000 7f1f6d004740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1f641ada70 con 0x55ac5aa44910 2024-05-10T11:30:25.820 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.815+0000 7f1f6d004740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1f641ab340 con 0x55ac5aa3edb0 2024-05-10T11:30:25.820 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.815+0000 7f1f6d004740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1f641a9190 con 0x55ac5aa3f7a0 2024-05-10T11:30:25.820 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.815+0000 7f1f63fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac5aa44910 0x55ac5aa50330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:25.820 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.815+0000 7f1f63fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac5aa3edb0 0x55ac5aa40980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:25.820 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.815+0000 7f1f62ffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ac5aa3f7a0 0x55ac5aa40ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:25.820 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.815+0000 7f1f63fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac5aa3edb0 0x55ac5aa40980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:36360/0 (socket says 172.21.15.100:36360) 2024-05-10T11:30:25.820 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.815+0000 7f1f63fff640 1 -- 172.21.15.100:0/3535341129 learned_addr learned my addr 172.21.15.100:0/3535341129 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:30:25.820 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.815+0000 7f1f63fff640 1 -- 172.21.15.100:0/3535341129 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ac5aa3f7a0 msgr2=0x55ac5aa40ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:25.820 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.815+0000 7f1f63fff640 1 --2- 172.21.15.100:0/3535341129 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ac5aa3f7a0 0x55ac5aa40ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:25.820 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.815+0000 7f1f63fff640 1 -- 172.21.15.100:0/3535341129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac5aa44910 msgr2=0x55ac5aa50330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:25.820 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.815+0000 7f1f63fff640 1 --2- 172.21.15.100:0/3535341129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac5aa44910 0x55ac5aa50330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:25.820 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.815+0000 7f1f63fff640 1 -- 172.21.15.100:0/3535341129 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac5aa3a250 con 0x55ac5aa3edb0 2024-05-10T11:30:25.820 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.815+0000 7f1f63fff640 1 --2- 172.21.15.100:0/3535341129 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac5aa3edb0 0x55ac5aa40980 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c000f80 tx=0x7f1f5c02b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5aa465b35391a3f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.815+0000 7f1f497fa640 1 -- 172.21.15.100:0/3535341129 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c031d40 con 0x55ac5aa3edb0 2024-05-10T11:30:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.815+0000 7f1f497fa640 1 -- 172.21.15.100:0/3535341129 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1f5c036070 con 0x55ac5aa3edb0 2024-05-10T11:30:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.815+0000 7f1f497fa640 1 -- 172.21.15.100:0/3535341129 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c004840 con 0x55ac5aa3edb0 2024-05-10T11:30:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.816+0000 7f1f6d004740 1 -- 172.21.15.100:0/3535341129 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac5aa3edb0 msgr2=0x55ac5aa40980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.816+0000 7f1f6d004740 1 --2- 172.21.15.100:0/3535341129 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac5aa3edb0 0x55ac5aa40980 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c000f80 tx=0x7f1f5c02b040 comp rx=0 tx=0).stop 2024-05-10T11:30:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.816+0000 7f1f6d004740 1 -- 172.21.15.100:0/3535341129 shutdown_connections 2024-05-10T11:30:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.816+0000 7f1f6d004740 1 --2- 172.21.15.100:0/3535341129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac5aa44910 0x55ac5aa50330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.816+0000 7f1f6d004740 1 --2- 172.21.15.100:0/3535341129 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ac5aa3f7a0 0x55ac5aa40ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.816+0000 7f1f6d004740 1 --2- 172.21.15.100:0/3535341129 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac5aa3edb0 0x55ac5aa40980 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.816+0000 7f1f6d004740 1 -- 172.21.15.100:0/3535341129 >> 172.21.15.100:0/3535341129 conn(0x55ac5aa3a420 msgr2=0x55ac5aa43fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.816+0000 7f1f6d004740 1 -- 172.21.15.100:0/3535341129 shutdown_connections 2024-05-10T11:30:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.816+0000 7f1f6d004740 1 -- 172.21.15.100:0/3535341129 wait complete. 2024-05-10T11:30:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.816+0000 7f1f6d004740 1 -- start start 2024-05-10T11:30:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.817+0000 7f1f6d004740 1 Processor -- start 2024-05-10T11:30:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.817+0000 7f1f6d004740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ac5aa3edb0 0x55ac5aa4da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.817+0000 7f1f6d004740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac5aa3f7a0 0x55ac5aa4dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.817+0000 7f1f6d004740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac5aa44910 0x55ac5aa4ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.817+0000 7f1f6d004740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1f641a9800 con 0x55ac5aa3f7a0 2024-05-10T11:30:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.817+0000 7f1f6d004740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1f641aac70 con 0x55ac5aa44910 2024-05-10T11:30:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.817+0000 7f1f6d004740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1f641a9ed0 con 0x55ac5aa3edb0 2024-05-10T11:30:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.817+0000 7f1f6d004740 10 client.0 _ll_register_callbacks cb 0x7f1f58009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:30:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.817+0000 7f1f62ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac5aa3f7a0 0x55ac5aa4dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.817+0000 7f1f62ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac5aa3f7a0 0x55ac5aa4dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52096/0 (socket says 172.21.15.100:52096) 2024-05-10T11:30:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.817+0000 7f1f63fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ac5aa3edb0 0x55ac5aa4da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.817+0000 7f1f62ffd640 1 -- 172.21.15.100:0/18570715 learned_addr learned my addr 172.21.15.100:0/18570715 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:30:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.817+0000 7f1f63fff640 1 --2- 172.21.15.100:0/18570715 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac5aa44910 0x55ac5aa4ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.817+0000 7f1f62ffd640 1 -- 172.21.15.100:0/18570715 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ac5aa3edb0 msgr2=0x55ac5aa4da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.817+0000 7f1f62ffd640 1 --2- 172.21.15.100:0/18570715 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ac5aa3edb0 0x55ac5aa4da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.817+0000 7f1f62ffd640 1 -- 172.21.15.100:0/18570715 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac5aa44910 msgr2=0x55ac5aa4ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.817+0000 7f1f62ffd640 1 --2- 172.21.15.100:0/18570715 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac5aa44910 0x55ac5aa4ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.817+0000 7f1f62ffd640 1 -- 172.21.15.100:0/18570715 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac5aa40470 con 0x55ac5aa3f7a0 2024-05-10T11:30:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.817+0000 7f1f63fff640 1 --2- 172.21.15.100:0/18570715 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ac5aa3edb0 0x55ac5aa4da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:30:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.817+0000 7f1f62ffd640 1 --2- 172.21.15.100:0/18570715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac5aa3f7a0 0x55ac5aa4dfd0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x55ac5aa4a2d0 tx=0x7f1f58009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88d685cdd47d2dce server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.818+0000 7f1f42ffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:30:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.818+0000 7f1f42ffd640 1 -- 172.21.15.100:0/18570715 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5800fa20 con 0x55ac5aa3f7a0 2024-05-10T11:30:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.818+0000 7f1f42ffd640 1 -- 172.21.15.100:0/18570715 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f1f5801cb20 con 0x55ac5aa3f7a0 2024-05-10T11:30:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.818+0000 7f1f6d004740 1 -- 172.21.15.100:0/18570715 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f5c036070 con 0x55ac5aa3f7a0 2024-05-10T11:30:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.818+0000 7f1f42ffd640 1 -- 172.21.15.100:0/18570715 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1f5801cd20 con 0x55ac5aa3f7a0 2024-05-10T11:30:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.818+0000 7f1f6d004740 10 client.17841 fetch_fsmap learned FSMap version 33 2024-05-10T11:30:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.818+0000 7f1f6d004740 1 -- 172.21.15.100:0/18570715 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ac5aabf540 con 0x55ac5aa3f7a0 2024-05-10T11:30:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.818+0000 7f1f42ffd640 1 -- 172.21.15.100:0/18570715 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1f5c036070 con 0x55ac5aa3f7a0 2024-05-10T11:30:25.824 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.818+0000 7f1f42ffd640 1 -- 172.21.15.100:0/18570715 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f1f58003880 con 0x55ac5aa3f7a0 2024-05-10T11:30:25.824 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.818+0000 7f1f6d004740 10 client.17841 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T11:30:25.824 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.818+0000 7f1f6d004740 10 client.17841 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:30:25.824 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.818+0000 7f1f6d004740 20 client.17841 populate_metadata read hostname 'smithi100' 2024-05-10T11:30:25.824 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.818+0000 7f1f6d004740 1 --2- 172.21.15.100:0/18570715 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ac5aabfe60 0x55ac5aac2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:25.824 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.818+0000 7f1f6d004740 4 client.17841 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:30:25.824 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.818+0000 7f1f6d004740 1 -- 172.21.15.100:0/18570715 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ac5a9ab050 con 0x55ac5aabfe60 2024-05-10T11:30:25.824 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.818+0000 7f1f63fff640 1 --2- 172.21.15.100:0/18570715 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ac5aabfe60 0x55ac5aac2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:25.824 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.819+0000 7f1f42ffd640 10 client.17841 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:30:25.824 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.819+0000 7f1f63fff640 1 --2- 172.21.15.100:0/18570715 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ac5aabfe60 0x55ac5aac2280 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbdc16fd8d65bf60 server_cookie=f1099c9197e97ff0 in_seq=0 out_seq=0 2024-05-10T11:30:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.839+0000 7f1f42ffd640 1 -- 172.21.15.100:0/18570715 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ac5a9ab050 con 0x55ac5aabfe60 2024-05-10T11:30:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.839+0000 7f1f42ffd640 10 client.17841 handle_command_reply: tid=0 2024-05-10T11:30:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.839+0000 7f1f6d004740 1 client.17841 shutdown 2024-05-10T11:30:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.840+0000 7f1f6d004740 1 -- 172.21.15.100:0/18570715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac5aa3f7a0 msgr2=0x55ac5aa4dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.840+0000 7f1f6d004740 1 --2- 172.21.15.100:0/18570715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac5aa3f7a0 0x55ac5aa4dfd0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x55ac5aa4a2d0 tx=0x7f1f58009e30 comp rx=0 tx=0).stop 2024-05-10T11:30:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.840+0000 7f1f6d004740 1 -- 172.21.15.100:0/18570715 shutdown_connections 2024-05-10T11:30:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.840+0000 7f1f6d004740 1 --2- 172.21.15.100:0/18570715 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ac5aabfe60 0x55ac5aac2280 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.840+0000 7f1f6d004740 1 --2- 172.21.15.100:0/18570715 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac5aa44910 0x55ac5aa4ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.840+0000 7f1f6d004740 1 --2- 172.21.15.100:0/18570715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ac5aa3f7a0 0x55ac5aa4dfd0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.840+0000 7f1f6d004740 1 --2- 172.21.15.100:0/18570715 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ac5aa3edb0 0x55ac5aa4da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.840+0000 7f1f6d004740 1 -- 172.21.15.100:0/18570715 >> 172.21.15.100:0/18570715 conn(0x55ac5aa3a420 msgr2=0x55ac5aa43b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.840+0000 7f1f42ffd640 0 client.17841 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:30:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.840+0000 7f1f6d004740 1 -- 172.21.15.100:0/18570715 shutdown_connections 2024-05-10T11:30:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.840+0000 7f1f6d004740 1 -- 172.21.15.100:0/18570715 wait complete. 2024-05-10T11:30:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.840+0000 7f1f6d004740 20 client.17841 trim_cache size 0 max 16384 2024-05-10T11:30:25.842 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:30:25.842 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:30:25.842 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:30:25.842 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:30:25.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.840+0000 7f1f6a209640 1 -- 172.21.15.100:0/2116530245 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f1f4c0b28a0 msgr2=0x7f1f4c0b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:25.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.840+0000 7f1f6a209640 1 --2- 172.21.15.100:0/2116530245 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f1f4c0b28a0 0x7f1f4c0b4d60 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1f540e03c0 tx=0x7f1f540e8550 comp rx=0 tx=0).stop 2024-05-10T11:30:25.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.840+0000 7f1f6a209640 1 -- 172.21.15.100:0/2116530245 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f64103490 msgr2=0x7f1f64198a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:25.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.840+0000 7f1f6a209640 1 --2- 172.21.15.100:0/2116530245 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f64103490 0x7f1f64198a80 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f1f5400e9f0 tx=0x7f1f5400edf0 comp rx=0 tx=0).stop 2024-05-10T11:30:25.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.840+0000 7f1f6a209640 1 -- 172.21.15.100:0/2116530245 shutdown_connections 2024-05-10T11:30:25.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.840+0000 7f1f6a209640 1 --2- 172.21.15.100:0/2116530245 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f1f4c0b28a0 0x7f1f4c0b4d60 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:25.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.840+0000 7f1f6a209640 1 --2- 172.21.15.100:0/2116530245 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f641086f0 0x7f1f641a0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:25.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.840+0000 7f1f6a209640 1 --2- 172.21.15.100:0/2116530245 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f64105dc0 0x7f1f64198fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:25.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.840+0000 7f1f6a209640 1 --2- 172.21.15.100:0/2116530245 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f64103490 0x7f1f64198a80 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:25.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.840+0000 7f1f6a209640 1 -- 172.21.15.100:0/2116530245 >> 172.21.15.100:0/2116530245 conn(0x7f1f640fce80 msgr2=0x7f1f64103dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:25.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.841+0000 7f1f6a209640 1 -- 172.21.15.100:0/2116530245 shutdown_connections 2024-05-10T11:30:25.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:25.841+0000 7f1f6a209640 1 -- 172.21.15.100:0/2116530245 wait complete. 2024-05-10T11:30:25.983 INFO:tasks.workunit.client.0.smithi100.stdout: 1 316191 2.72 MB/sec execute 500 sec latency 19.541 ms 2024-05-10T11:30:26.983 INFO:tasks.workunit.client.0.smithi100.stdout: 1 316642 2.71 MB/sec execute 501 sec latency 18.532 ms 2024-05-10T11:30:27.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:30:27 smithi188 ceph-mon[34075]: pgmap v783: 129 pgs: 129 active+clean; 123 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 1.4 MiB/s wr, 156 op/s 2024-05-10T11:30:27.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:30:27 smithi100 ceph-mon[30296]: pgmap v783: 129 pgs: 129 active+clean; 123 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 1.4 MiB/s wr, 156 op/s 2024-05-10T11:30:27.983 INFO:tasks.workunit.client.0.smithi100.stdout: 1 317201 2.72 MB/sec execute 502 sec latency 14.578 ms 2024-05-10T11:30:28.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:30:27 smithi178 ceph-mon[33359]: pgmap v783: 129 pgs: 129 active+clean; 123 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 1.4 MiB/s wr, 156 op/s 2024-05-10T11:30:28.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:30:28 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:30:28.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:30:28 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:30:28.983 INFO:tasks.workunit.client.0.smithi100.stdout: 1 317804 2.71 MB/sec execute 503 sec latency 17.815 ms 2024-05-10T11:30:29.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:30:28 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:30:29.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:30:29 smithi188 ceph-mon[34075]: pgmap v784: 129 pgs: 129 active+clean; 125 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.5 MiB/s wr, 153 op/s 2024-05-10T11:30:29.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:30:29 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:30:29.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:30:29 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:30:29.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:30:29 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:30:29.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:30:29 smithi100 ceph-mon[30296]: pgmap v784: 129 pgs: 129 active+clean; 125 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.5 MiB/s wr, 153 op/s 2024-05-10T11:30:29.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:30:29 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:30:29.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:30:29 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:30:29.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:30:29 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:30:29.983 INFO:tasks.workunit.client.0.smithi100.stdout: 1 318319 2.71 MB/sec execute 504 sec latency 15.555 ms 2024-05-10T11:30:30.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:30:29 smithi178 ceph-mon[33359]: pgmap v784: 129 pgs: 129 active+clean; 125 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.5 MiB/s wr, 153 op/s 2024-05-10T11:30:30.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:30:29 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:30:30.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:30:29 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:30:30.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:30:29 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:30:30.983 INFO:tasks.workunit.client.0.smithi100.stdout: 1 319023 2.72 MB/sec execute 505 sec latency 23.888 ms 2024-05-10T11:30:31.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:30:31 smithi188 ceph-mon[34075]: pgmap v785: 129 pgs: 129 active+clean; 124 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 1.2 MiB/s wr, 148 op/s 2024-05-10T11:30:31.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:30:31 smithi100 ceph-mon[30296]: pgmap v785: 129 pgs: 129 active+clean; 124 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 1.2 MiB/s wr, 148 op/s 2024-05-10T11:30:31.983 INFO:tasks.workunit.client.0.smithi100.stdout: 1 319489 2.72 MB/sec execute 506 sec latency 15.860 ms 2024-05-10T11:30:32.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:30:31 smithi178 ceph-mon[33359]: pgmap v785: 129 pgs: 129 active+clean; 124 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 1.2 MiB/s wr, 148 op/s 2024-05-10T11:30:32.983 INFO:tasks.workunit.client.0.smithi100.stdout: 1 319751 2.72 MB/sec execute 507 sec latency 13.134 ms 2024-05-10T11:30:33.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:30:33 smithi188 ceph-mon[34075]: pgmap v786: 129 pgs: 129 active+clean; 124 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 1.3 MiB/s wr, 151 op/s 2024-05-10T11:30:33.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:30:33 smithi100 ceph-mon[30296]: pgmap v786: 129 pgs: 129 active+clean; 124 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 1.3 MiB/s wr, 151 op/s 2024-05-10T11:30:33.983 INFO:tasks.workunit.client.0.smithi100.stdout: 1 320183 2.71 MB/sec execute 508 sec latency 16.886 ms 2024-05-10T11:30:34.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:30:33 smithi178 ceph-mon[33359]: pgmap v786: 129 pgs: 129 active+clean; 124 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 1.3 MiB/s wr, 151 op/s 2024-05-10T11:30:34.983 INFO:tasks.workunit.client.0.smithi100.stdout: 1 320739 2.72 MB/sec execute 509 sec latency 23.694 ms 2024-05-10T11:30:35.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:30:35 smithi188 ceph-mon[34075]: pgmap v787: 129 pgs: 129 active+clean; 125 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 1.2 MiB/s wr, 128 op/s 2024-05-10T11:30:35.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:30:35 smithi100 ceph-mon[30296]: pgmap v787: 129 pgs: 129 active+clean; 125 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 1.2 MiB/s wr, 128 op/s 2024-05-10T11:30:35.983 INFO:tasks.workunit.client.0.smithi100.stdout: 1 321387 2.71 MB/sec execute 510 sec latency 15.611 ms 2024-05-10T11:30:36.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:30:35 smithi178 ceph-mon[33359]: pgmap v787: 129 pgs: 129 active+clean; 125 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 1.2 MiB/s wr, 128 op/s 2024-05-10T11:30:36.730 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T11:30:36.983 INFO:tasks.workunit.client.0.smithi100.stdout: 1 321854 2.71 MB/sec execute 511 sec latency 16.808 ms 2024-05-10T11:30:36.993 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:30:37.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:30:37 smithi188 ceph-mon[34075]: pgmap v788: 129 pgs: 129 active+clean; 126 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 1.3 MiB/s wr, 152 op/s 2024-05-10T11:30:37.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:30:37 smithi100 ceph-mon[30296]: pgmap v788: 129 pgs: 129 active+clean; 126 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 1.3 MiB/s wr, 152 op/s 2024-05-10T11:30:37.983 INFO:tasks.workunit.client.0.smithi100.stdout: 1 322560 2.72 MB/sec execute 512 sec latency 15.183 ms 2024-05-10T11:30:38.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:30:37 smithi178 ceph-mon[33359]: pgmap v788: 129 pgs: 129 active+clean; 126 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 1.3 MiB/s wr, 152 op/s 2024-05-10T11:30:38.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.759+0000 7f3515d30700 1 -- 172.21.15.100:0/1281039179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3510107760 msgr2=0x7f351010f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:38.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.759+0000 7f3515d30700 1 --2- 172.21.15.100:0/1281039179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3510107760 0x7f351010f2e0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f350800b1d0 tx=0x7f350800b4e0 comp rx=0 tx=0).stop 2024-05-10T11:30:38.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.759+0000 7f3515d30700 1 -- 172.21.15.100:0/1281039179 shutdown_connections 2024-05-10T11:30:38.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.759+0000 7f3515d30700 1 --2- 172.21.15.100:0/1281039179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3510107760 0x7f351010f2e0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:38.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.760+0000 7f3515d30700 1 --2- 172.21.15.100:0/1281039179 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3510102e00 0x7f35101031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:38.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.760+0000 7f3515d30700 1 --2- 172.21.15.100:0/1281039179 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35101037b0 0x7f3510107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:38.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.760+0000 7f3515d30700 1 -- 172.21.15.100:0/1281039179 >> 172.21.15.100:0/1281039179 conn(0x7f3510078a10 msgr2=0x7f35100ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:38.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.760+0000 7f3515d30700 1 -- 172.21.15.100:0/1281039179 shutdown_connections 2024-05-10T11:30:38.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.760+0000 7f3515d30700 1 -- 172.21.15.100:0/1281039179 wait complete. 2024-05-10T11:30:38.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.760+0000 7f3515d30700 1 Processor -- start 2024-05-10T11:30:38.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.761+0000 7f3515d30700 1 -- start start 2024-05-10T11:30:38.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.761+0000 7f3515d30700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3510102e00 0x7f35101a1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:38.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.761+0000 7f3515d30700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35101037b0 0x7f35101a1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:38.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.761+0000 7f3515d30700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3510107760 0x7f351019baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:38.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.761+0000 7f3515d30700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f35101157a0 con 0x7f35101037b0 2024-05-10T11:30:38.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.761+0000 7f3515d30700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3510115620 con 0x7f3510107760 2024-05-10T11:30:38.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.761+0000 7f3515d30700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3510115920 con 0x7f3510102e00 2024-05-10T11:30:38.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.761+0000 7f350f7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3510102e00 0x7f35101a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:38.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.761+0000 7f350f7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3510102e00 0x7f35101a1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:42258/0 (socket says 172.21.15.100:42258) 2024-05-10T11:30:38.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.761+0000 7f350f7fe700 1 -- 172.21.15.100:0/1976104549 learned_addr learned my addr 172.21.15.100:0/1976104549 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:30:38.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.761+0000 7f350effd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35101037b0 0x7f35101a1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:38.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.761+0000 7f350ffff700 1 --2- 172.21.15.100:0/1976104549 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3510107760 0x7f351019baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:38.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.761+0000 7f350effd700 1 -- 172.21.15.100:0/1976104549 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3510102e00 msgr2=0x7f35101a1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:38.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.761+0000 7f350effd700 1 --2- 172.21.15.100:0/1976104549 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3510102e00 0x7f35101a1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:38.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.761+0000 7f350effd700 1 -- 172.21.15.100:0/1976104549 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3510107760 msgr2=0x7f351019baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:38.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.761+0000 7f350effd700 1 --2- 172.21.15.100:0/1976104549 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3510107760 0x7f351019baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:38.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.761+0000 7f350effd700 1 -- 172.21.15.100:0/1976104549 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f351019c200 con 0x7f35101037b0 2024-05-10T11:30:38.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.761+0000 7f350f7fe700 1 --2- 172.21.15.100:0/1976104549 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3510102e00 0x7f35101a1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:30:38.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.761+0000 7f350ffff700 1 --2- 172.21.15.100:0/1976104549 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3510107760 0x7f351019baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:30:38.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.762+0000 7f350effd700 1 --2- 172.21.15.100:0/1976104549 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35101037b0 0x7f35101a1dd0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f350400d750 tx=0x7f350400da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:38.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.762+0000 7f350cff9700 1 -- 172.21.15.100:0/1976104549 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3504014030 con 0x7f35101037b0 2024-05-10T11:30:38.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.762+0000 7f350cff9700 1 -- 172.21.15.100:0/1976104549 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f35040044b0 con 0x7f35101037b0 2024-05-10T11:30:38.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.762+0000 7f3515d30700 1 -- 172.21.15.100:0/1976104549 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f351019c4f0 con 0x7f35101037b0 2024-05-10T11:30:38.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.762+0000 7f3515d30700 1 -- 172.21.15.100:0/1976104549 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35101a8760 con 0x7f35101037b0 2024-05-10T11:30:38.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.762+0000 7f350cff9700 1 -- 172.21.15.100:0/1976104549 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35040052f0 con 0x7f35101037b0 2024-05-10T11:30:38.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.765+0000 7f350cff9700 1 -- 172.21.15.100:0/1976104549 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f3504005490 con 0x7f35101037b0 2024-05-10T11:30:38.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.765+0000 7f350cff9700 1 --2- 172.21.15.100:0/1976104549 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f34f80b2880 0x7f34f80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:38.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.765+0000 7f350f7fe700 1 --2- 172.21.15.100:0/1976104549 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f34f80b2880 0x7f34f80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:38.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.765+0000 7f350cff9700 1 -- 172.21.15.100:0/1976104549 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f35040e1df0 con 0x7f35101037b0 2024-05-10T11:30:38.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.766+0000 7f350f7fe700 1 --2- 172.21.15.100:0/1976104549 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f34f80b2880 0x7f34f80b4d40 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3510108e90 tx=0x7f350000b480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:38.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.766+0000 7f35205c3b80 1 Processor -- start 2024-05-10T11:30:38.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.766+0000 7f35205c3b80 1 -- start start 2024-05-10T11:30:38.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.766+0000 7f35205c3b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb217b24d0 0x55bb217b28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:38.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.766+0000 7f35205c3b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb217b2df0 0x55bb217b66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:38.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.766+0000 7f35205c3b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb217b6c80 0x55bb217b7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:38.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.766+0000 7f35205c3b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55bb21704860 con 0x55bb217b2df0 2024-05-10T11:30:38.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.766+0000 7f35205c3b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55bb216a37c0 con 0x55bb217b24d0 2024-05-10T11:30:38.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.766+0000 7f35205c3b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55bb21700820 con 0x55bb217b6c80 2024-05-10T11:30:38.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.766+0000 7f350ffff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb217b6c80 0x55bb217b7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:38.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.766+0000 7f350ffff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb217b6c80 0x55bb217b7100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:42266/0 (socket says 172.21.15.100:42266) 2024-05-10T11:30:38.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.766+0000 7f350ffff700 1 -- 172.21.15.100:0/137288504 learned_addr learned my addr 172.21.15.100:0/137288504 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:30:38.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.766+0000 7f350f7fe700 1 --2- 172.21.15.100:0/137288504 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb217b24d0 0x55bb217b28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:38.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.767+0000 7f350effd700 1 --2- 172.21.15.100:0/137288504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb217b2df0 0x55bb217b66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:38.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.767+0000 7f350ffff700 1 -- 172.21.15.100:0/137288504 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb217b24d0 msgr2=0x55bb217b28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:38.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.767+0000 7f350ffff700 1 --2- 172.21.15.100:0/137288504 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb217b24d0 0x55bb217b28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:38.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.767+0000 7f350ffff700 1 -- 172.21.15.100:0/137288504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb217b2df0 msgr2=0x55bb217b66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:38.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.767+0000 7f350ffff700 1 --2- 172.21.15.100:0/137288504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb217b2df0 0x55bb217b66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:38.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.767+0000 7f350ffff700 1 -- 172.21.15.100:0/137288504 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb216518d0 con 0x55bb217b6c80 2024-05-10T11:30:38.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.767+0000 7f350f7fe700 1 --2- 172.21.15.100:0/137288504 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb217b24d0 0x55bb217b28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:30:38.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.767+0000 7f350effd700 1 --2- 172.21.15.100:0/137288504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb217b2df0 0x55bb217b66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:30:38.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.767+0000 7f350ffff700 1 --2- 172.21.15.100:0/137288504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb217b6c80 0x55bb217b7100 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3508009750 tx=0x7f3508009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3369e5880d11a959 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:38.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.767+0000 7f34f5ffb700 1 -- 172.21.15.100:0/137288504 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3508003a00 con 0x55bb217b6c80 2024-05-10T11:30:38.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.767+0000 7f34f5ffb700 1 -- 172.21.15.100:0/137288504 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3508007e50 con 0x55bb217b6c80 2024-05-10T11:30:38.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.767+0000 7f34f5ffb700 1 -- 172.21.15.100:0/137288504 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35080238f0 con 0x55bb217b6c80 2024-05-10T11:30:38.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.768+0000 7f35205c3b80 1 -- 172.21.15.100:0/137288504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb217b6c80 msgr2=0x55bb217b7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:38.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.768+0000 7f35205c3b80 1 --2- 172.21.15.100:0/137288504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb217b6c80 0x55bb217b7100 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3508009750 tx=0x7f3508009780 comp rx=0 tx=0).stop 2024-05-10T11:30:38.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.768+0000 7f35205c3b80 1 -- 172.21.15.100:0/137288504 shutdown_connections 2024-05-10T11:30:38.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.768+0000 7f35205c3b80 1 --2- 172.21.15.100:0/137288504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb217b6c80 0x55bb217b7100 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:38.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.768+0000 7f35205c3b80 1 --2- 172.21.15.100:0/137288504 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb217b24d0 0x55bb217b28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:38.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.768+0000 7f35205c3b80 1 --2- 172.21.15.100:0/137288504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb217b2df0 0x55bb217b66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:38.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.768+0000 7f35205c3b80 1 -- 172.21.15.100:0/137288504 >> 172.21.15.100:0/137288504 conn(0x55bb216de100 msgr2=0x55bb217acc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:38.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.768+0000 7f35205c3b80 1 -- 172.21.15.100:0/137288504 shutdown_connections 2024-05-10T11:30:38.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.768+0000 7f35205c3b80 1 -- 172.21.15.100:0/137288504 wait complete. 2024-05-10T11:30:38.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.768+0000 7f35205c3b80 1 -- start start 2024-05-10T11:30:38.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.769+0000 7f35205c3b80 1 Processor -- start 2024-05-10T11:30:38.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.769+0000 7f35205c3b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb217b24d0 0x55bb217bdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:38.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.769+0000 7f35205c3b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb217b2df0 0x55bb217bb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:38.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.769+0000 7f35205c3b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb217b6c80 0x55bb217bb930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:38.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.769+0000 7f35205c3b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55bb217bee30 con 0x55bb217b2df0 2024-05-10T11:30:38.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.769+0000 7f35205c3b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55bb2166ac30 con 0x55bb217b6c80 2024-05-10T11:30:38.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.769+0000 7f35205c3b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55bb217befb0 con 0x55bb217b24d0 2024-05-10T11:30:38.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.769+0000 7f35205c3b80 10 client.0 _ll_register_callbacks cb 0x55bb2169c920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:30:38.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.769+0000 7f350f7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb217b24d0 0x55bb217bdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:38.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.769+0000 7f350f7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb217b24d0 0x55bb217bdfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:42274/0 (socket says 172.21.15.100:42274) 2024-05-10T11:30:38.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.769+0000 7f350ffff700 1 --2- 172.21.15.100:0/3882491778 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb217b6c80 0x55bb217bb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:38.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.769+0000 7f350f7fe700 1 -- 172.21.15.100:0/3882491778 learned_addr learned my addr 172.21.15.100:0/3882491778 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:30:38.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.769+0000 7f350effd700 1 --2- 172.21.15.100:0/3882491778 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb217b2df0 0x55bb217bb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:38.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.769+0000 7f350f7fe700 1 -- 172.21.15.100:0/3882491778 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb217b6c80 msgr2=0x55bb217bb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:38.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.769+0000 7f350f7fe700 1 --2- 172.21.15.100:0/3882491778 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb217b6c80 0x55bb217bb930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:38.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.769+0000 7f350f7fe700 1 -- 172.21.15.100:0/3882491778 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb217b2df0 msgr2=0x55bb217bb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:38.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.769+0000 7f350f7fe700 1 --2- 172.21.15.100:0/3882491778 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb217b2df0 0x55bb217bb3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:38.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.769+0000 7f350f7fe700 1 -- 172.21.15.100:0/3882491778 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb216a0690 con 0x55bb217b24d0 2024-05-10T11:30:38.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.769+0000 7f350ffff700 1 --2- 172.21.15.100:0/3882491778 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb217b6c80 0x55bb217bb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:30:38.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.770+0000 7f350f7fe700 1 --2- 172.21.15.100:0/3882491778 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb217b24d0 0x55bb217bdfe0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3500011920 tx=0x7f35000047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ff58eb7a5042e18 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:38.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.770+0000 7f34eeffd700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:30:38.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.770+0000 7f34eeffd700 1 -- 172.21.15.100:0/3882491778 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f350001e780 con 0x55bb217b24d0 2024-05-10T11:30:38.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.770+0000 7f34eeffd700 1 -- 172.21.15.100:0/3882491778 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f350001e920 con 0x55bb217b24d0 2024-05-10T11:30:38.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.770+0000 7f35205c3b80 1 -- 172.21.15.100:0/3882491778 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3508007e50 con 0x55bb217b24d0 2024-05-10T11:30:38.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.770+0000 7f34eeffd700 1 -- 172.21.15.100:0/3882491778 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f350001eb20 con 0x55bb217b24d0 2024-05-10T11:30:38.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.770+0000 7f35205c3b80 10 client.27304 fetch_fsmap learned FSMap version 33 2024-05-10T11:30:38.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.770+0000 7f35205c3b80 1 -- 172.21.15.100:0/3882491778 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55bb217c96c0 con 0x55bb217b24d0 2024-05-10T11:30:38.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.770+0000 7f34eeffd700 1 -- 172.21.15.100:0/3882491778 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3508007e50 con 0x55bb217b24d0 2024-05-10T11:30:38.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.770+0000 7f34eeffd700 1 -- 172.21.15.100:0/3882491778 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f350001f430 con 0x55bb217b24d0 2024-05-10T11:30:38.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.771+0000 7f35205c3b80 10 client.27304 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T11:30:38.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.771+0000 7f35205c3b80 10 client.27304 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:30:38.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.771+0000 7f35205c3b80 20 client.27304 populate_metadata read hostname 'smithi100' 2024-05-10T11:30:38.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.771+0000 7f35205c3b80 1 --2- 172.21.15.100:0/3882491778 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55bb217cbd00 0x55bb217ce0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:38.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.771+0000 7f35205c3b80 4 client.27304 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:30:38.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.771+0000 7f35205c3b80 1 -- 172.21.15.100:0/3882491778 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb2177e650 con 0x55bb217cbd00 2024-05-10T11:30:38.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.771+0000 7f350effd700 1 --2- 172.21.15.100:0/3882491778 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55bb217cbd00 0x55bb217ce0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:38.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.771+0000 7f350effd700 1 --2- 172.21.15.100:0/3882491778 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55bb217cbd00 0x55bb217ce0f0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56bc337ce0c138b3 server_cookie=4c6d46aa82c101c7 in_seq=0 out_seq=0 2024-05-10T11:30:38.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.771+0000 7f34eeffd700 10 client.27304 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:30:38.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.772+0000 7f34eeffd700 1 -- 172.21.15.100:0/3882491778 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bb2177e650 con 0x55bb217cbd00 2024-05-10T11:30:38.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.772+0000 7f34eeffd700 10 client.27304 handle_command_reply: tid=0 2024-05-10T11:30:38.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.772+0000 7f35205c3b80 1 client.27304 shutdown 2024-05-10T11:30:38.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.772+0000 7f35205c3b80 1 -- 172.21.15.100:0/3882491778 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb217b24d0 msgr2=0x55bb217bdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:38.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.772+0000 7f35205c3b80 1 --2- 172.21.15.100:0/3882491778 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb217b24d0 0x55bb217bdfe0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3500011920 tx=0x7f35000047c0 comp rx=0 tx=0).stop 2024-05-10T11:30:38.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.772+0000 7f35205c3b80 1 -- 172.21.15.100:0/3882491778 shutdown_connections 2024-05-10T11:30:38.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.772+0000 7f35205c3b80 1 --2- 172.21.15.100:0/3882491778 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55bb217cbd00 0x55bb217ce0f0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:38.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.772+0000 7f35205c3b80 1 --2- 172.21.15.100:0/3882491778 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb217b6c80 0x55bb217bb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:38.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.772+0000 7f35205c3b80 1 --2- 172.21.15.100:0/3882491778 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb217b24d0 0x55bb217bdfe0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:38.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.772+0000 7f35205c3b80 1 --2- 172.21.15.100:0/3882491778 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb217b2df0 0x55bb217bb3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:38.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.772+0000 7f35205c3b80 1 -- 172.21.15.100:0/3882491778 >> 172.21.15.100:0/3882491778 conn(0x55bb216de100 msgr2=0x55bb217b7080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:38.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.772+0000 7f34eeffd700 0 client.27304 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:30:38.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.772+0000 7f35205c3b80 1 -- 172.21.15.100:0/3882491778 shutdown_connections 2024-05-10T11:30:38.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.772+0000 7f35205c3b80 1 -- 172.21.15.100:0/3882491778 wait complete. 2024-05-10T11:30:38.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.772+0000 7f35205c3b80 20 client.27304 trim_cache size 0 max 16384 2024-05-10T11:30:38.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.797+0000 7f35205c3b80 1 Processor -- start 2024-05-10T11:30:38.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.797+0000 7f35205c3b80 1 -- start start 2024-05-10T11:30:38.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.797+0000 7f35205c3b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb217b24d0 0x55bb217c7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:38.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.797+0000 7f35205c3b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb217b2df0 0x55bb217bb960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:38.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.797+0000 7f35205c3b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb217b6c80 0x55bb217a75a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:38.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.797+0000 7f35205c3b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f35101b3ae0 con 0x55bb217b2df0 2024-05-10T11:30:38.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.797+0000 7f35205c3b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f35101b1370 con 0x55bb217b24d0 2024-05-10T11:30:38.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.797+0000 7f35205c3b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f35101af1c0 con 0x55bb217b6c80 2024-05-10T11:30:38.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.797+0000 7f350effd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb217b2df0 0x55bb217bb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:38.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.797+0000 7f350effd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb217b2df0 0x55bb217bb960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34262/0 (socket says 172.21.15.100:34262) 2024-05-10T11:30:38.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.797+0000 7f350f7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb217b24d0 0x55bb217c7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:38.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.797+0000 7f350effd700 1 -- 172.21.15.100:0/2843368803 learned_addr learned my addr 172.21.15.100:0/2843368803 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:30:38.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.797+0000 7f350ffff700 1 --2- 172.21.15.100:0/2843368803 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb217b6c80 0x55bb217a75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:38.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.797+0000 7f350f7fe700 1 -- 172.21.15.100:0/2843368803 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb217b6c80 msgr2=0x55bb217a75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:38.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.797+0000 7f350f7fe700 1 --2- 172.21.15.100:0/2843368803 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb217b6c80 0x55bb217a75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:38.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.797+0000 7f350f7fe700 1 -- 172.21.15.100:0/2843368803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb217b2df0 msgr2=0x55bb217bb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:38.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.797+0000 7f350f7fe700 1 --2- 172.21.15.100:0/2843368803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb217b2df0 0x55bb217bb960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:38.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.797+0000 7f350f7fe700 1 -- 172.21.15.100:0/2843368803 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb217a9810 con 0x55bb217b24d0 2024-05-10T11:30:38.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.797+0000 7f350effd700 1 --2- 172.21.15.100:0/2843368803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb217b2df0 0x55bb217bb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:30:38.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.798+0000 7f350f7fe700 1 --2- 172.21.15.100:0/2843368803 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb217b24d0 0x55bb217c7ef0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3500010ce0 tx=0x7f350001ef50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54908abf68c97732 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:38.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.798+0000 7f34f57fa700 1 -- 172.21.15.100:0/2843368803 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f350001f630 con 0x55bb217b24d0 2024-05-10T11:30:38.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.798+0000 7f34f57fa700 1 -- 172.21.15.100:0/2843368803 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f350001fcf0 con 0x55bb217b24d0 2024-05-10T11:30:38.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.798+0000 7f34f57fa700 1 -- 172.21.15.100:0/2843368803 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35000287f0 con 0x55bb217b24d0 2024-05-10T11:30:38.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.799+0000 7f35205c3b80 1 -- 172.21.15.100:0/2843368803 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb217b24d0 msgr2=0x55bb217c7ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:38.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.799+0000 7f35205c3b80 1 --2- 172.21.15.100:0/2843368803 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb217b24d0 0x55bb217c7ef0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3500010ce0 tx=0x7f350001ef50 comp rx=0 tx=0).stop 2024-05-10T11:30:38.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.799+0000 7f35205c3b80 1 -- 172.21.15.100:0/2843368803 shutdown_connections 2024-05-10T11:30:38.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.799+0000 7f35205c3b80 1 --2- 172.21.15.100:0/2843368803 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb217b6c80 0x55bb217a75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:38.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.799+0000 7f35205c3b80 1 --2- 172.21.15.100:0/2843368803 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb217b24d0 0x55bb217c7ef0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:38.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.799+0000 7f35205c3b80 1 --2- 172.21.15.100:0/2843368803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb217b2df0 0x55bb217bb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:38.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.799+0000 7f35205c3b80 1 -- 172.21.15.100:0/2843368803 >> 172.21.15.100:0/2843368803 conn(0x55bb216de100 msgr2=0x55bb217a6e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:38.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.799+0000 7f35205c3b80 1 -- 172.21.15.100:0/2843368803 shutdown_connections 2024-05-10T11:30:38.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.799+0000 7f35205c3b80 1 -- 172.21.15.100:0/2843368803 wait complete. 2024-05-10T11:30:38.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.799+0000 7f35205c3b80 1 -- start start 2024-05-10T11:30:38.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.800+0000 7f35205c3b80 1 Processor -- start 2024-05-10T11:30:38.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.800+0000 7f35205c3b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb217b24d0 0x55bb2182e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:38.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.800+0000 7f35205c3b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb217b2df0 0x55bb2182ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:38.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.800+0000 7f35205c3b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb217b6c80 0x55bb217bf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:38.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.800+0000 7f35205c3b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f35101af830 con 0x55bb217b6c80 2024-05-10T11:30:38.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.800+0000 7f35205c3b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f35101b0ca0 con 0x55bb217b24d0 2024-05-10T11:30:38.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.800+0000 7f35205c3b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f35101aff00 con 0x55bb217b2df0 2024-05-10T11:30:38.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.800+0000 7f35205c3b80 10 client.0 _ll_register_callbacks cb 0x55bb2169c920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:30:38.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.800+0000 7f350f7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb217b24d0 0x55bb2182e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:38.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.800+0000 7f350effd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb217b2df0 0x55bb2182ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:38.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.800+0000 7f350f7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb217b24d0 0x55bb2182e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:59998/0 (socket says 172.21.15.100:59998) 2024-05-10T11:30:38.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.800+0000 7f350effd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb217b2df0 0x55bb2182ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:42294/0 (socket says 172.21.15.100:42294) 2024-05-10T11:30:38.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.800+0000 7f350f7fe700 1 -- 172.21.15.100:0/426194802 learned_addr learned my addr 172.21.15.100:0/426194802 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:30:38.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.800+0000 7f350ffff700 1 --2- 172.21.15.100:0/426194802 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb217b6c80 0x55bb217bf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:38.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.801+0000 7f350effd700 1 -- 172.21.15.100:0/426194802 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb217b24d0 msgr2=0x55bb2182e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:38.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.801+0000 7f350effd700 1 --2- 172.21.15.100:0/426194802 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb217b24d0 0x55bb2182e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:38.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.801+0000 7f350effd700 1 -- 172.21.15.100:0/426194802 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb217b6c80 msgr2=0x55bb217bf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:38.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.801+0000 7f350effd700 1 --2- 172.21.15.100:0/426194802 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb217b6c80 0x55bb217bf630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:38.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.801+0000 7f350effd700 1 -- 172.21.15.100:0/426194802 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb217bfd00 con 0x55bb217b2df0 2024-05-10T11:30:38.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.801+0000 7f350effd700 1 --2- 172.21.15.100:0/426194802 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb217b2df0 0x55bb2182ede0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f35040df5e0 tx=0x7f35040df460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=872e0722a0f70dad server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:38.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.801+0000 7f34eeffd700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:30:38.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.801+0000 7f34eeffd700 1 -- 172.21.15.100:0/426194802 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35040c7070 con 0x55bb217b2df0 2024-05-10T11:30:38.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.801+0000 7f34eeffd700 1 -- 172.21.15.100:0/426194802 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f3504004a70 con 0x55bb217b2df0 2024-05-10T11:30:38.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.801+0000 7f35205c3b80 1 -- 172.21.15.100:0/426194802 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bb21712240 con 0x55bb217b2df0 2024-05-10T11:30:38.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.801+0000 7f34eeffd700 1 -- 172.21.15.100:0/426194802 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f350408a050 con 0x55bb217b2df0 2024-05-10T11:30:38.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.801+0000 7f35205c3b80 10 client.27310 fetch_fsmap learned FSMap version 33 2024-05-10T11:30:38.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.801+0000 7f35205c3b80 1 -- 172.21.15.100:0/426194802 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55bb218360e0 con 0x55bb217b2df0 2024-05-10T11:30:38.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.802+0000 7f34eeffd700 1 -- 172.21.15.100:0/426194802 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55bb21712240 con 0x55bb217b2df0 2024-05-10T11:30:38.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.802+0000 7f34eeffd700 1 -- 172.21.15.100:0/426194802 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f35040b1a80 con 0x55bb217b2df0 2024-05-10T11:30:38.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.802+0000 7f35205c3b80 10 client.27310 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T11:30:38.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.802+0000 7f35205c3b80 10 client.27310 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:30:38.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.802+0000 7f35205c3b80 20 client.27310 populate_metadata read hostname 'smithi100' 2024-05-10T11:30:38.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.802+0000 7f35205c3b80 1 --2- 172.21.15.100:0/426194802 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55bb21836780 0x55bb21838ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:38.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.802+0000 7f35205c3b80 4 client.27310 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T11:30:38.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.802+0000 7f35205c3b80 1 -- 172.21.15.100:0/426194802 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bb2170c2e0 con 0x55bb21836780 2024-05-10T11:30:38.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.802+0000 7f350f7fe700 1 --2- 172.21.15.100:0/426194802 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55bb21836780 0x55bb21838ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:38.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.802+0000 7f34eeffd700 10 client.27310 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:30:38.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.802+0000 7f350f7fe700 1 --2- 172.21.15.100:0/426194802 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55bb21836780 0x55bb21838ba0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97257f3860f3166a server_cookie=a32beea4a5d3dd9d in_seq=0 out_seq=0 2024-05-10T11:30:38.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.803+0000 7f34eeffd700 1 -- 172.21.15.100:0/426194802 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55bb2170c2e0 con 0x55bb21836780 2024-05-10T11:30:38.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.803+0000 7f34eeffd700 10 client.27310 handle_command_reply: tid=0 2024-05-10T11:30:38.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.803+0000 7f35205c3b80 1 client.27310 shutdown 2024-05-10T11:30:38.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.803+0000 7f35205c3b80 1 -- 172.21.15.100:0/426194802 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb217b2df0 msgr2=0x55bb2182ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:38.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.803+0000 7f35205c3b80 1 --2- 172.21.15.100:0/426194802 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb217b2df0 0x55bb2182ede0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f35040df5e0 tx=0x7f35040df460 comp rx=0 tx=0).stop 2024-05-10T11:30:38.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.803+0000 7f35205c3b80 1 -- 172.21.15.100:0/426194802 shutdown_connections 2024-05-10T11:30:38.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.803+0000 7f35205c3b80 1 --2- 172.21.15.100:0/426194802 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55bb21836780 0x55bb21838ba0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:38.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.803+0000 7f35205c3b80 1 --2- 172.21.15.100:0/426194802 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb217b6c80 0x55bb217bf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:38.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.803+0000 7f35205c3b80 1 --2- 172.21.15.100:0/426194802 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb217b24d0 0x55bb2182e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:38.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.803+0000 7f34eeffd700 0 client.27310 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:30:38.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.803+0000 7f35205c3b80 1 --2- 172.21.15.100:0/426194802 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb217b2df0 0x55bb2182ede0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:38.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.803+0000 7f35205c3b80 1 -- 172.21.15.100:0/426194802 >> 172.21.15.100:0/426194802 conn(0x55bb216de100 msgr2=0x55bb217a7720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:38.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.803+0000 7f35205c3b80 1 -- 172.21.15.100:0/426194802 shutdown_connections 2024-05-10T11:30:38.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.803+0000 7f35205c3b80 1 -- 172.21.15.100:0/426194802 wait complete. 2024-05-10T11:30:38.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.803+0000 7f35205c3b80 20 client.27310 trim_cache size 0 max 16384 2024-05-10T11:30:38.808 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:30:38.808 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-05-10T11:30:38.808 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-05-10T11:30:38.808 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:30:38.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.804+0000 7f3515d30700 1 -- 172.21.15.100:0/1976104549 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f34f80b2880 msgr2=0x7f34f80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:38.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.804+0000 7f3515d30700 1 --2- 172.21.15.100:0/1976104549 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f34f80b2880 0x7f34f80b4d40 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3510108e90 tx=0x7f350000b480 comp rx=0 tx=0).stop 2024-05-10T11:30:38.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.804+0000 7f3515d30700 1 -- 172.21.15.100:0/1976104549 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35101037b0 msgr2=0x7f35101a1dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:38.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.804+0000 7f3515d30700 1 --2- 172.21.15.100:0/1976104549 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35101037b0 0x7f35101a1dd0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f350400d750 tx=0x7f350400da60 comp rx=0 tx=0).stop 2024-05-10T11:30:38.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.804+0000 7f3515d30700 1 -- 172.21.15.100:0/1976104549 shutdown_connections 2024-05-10T11:30:38.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.804+0000 7f3515d30700 1 --2- 172.21.15.100:0/1976104549 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f34f80b2880 0x7f34f80b4d40 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:38.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.804+0000 7f3515d30700 1 --2- 172.21.15.100:0/1976104549 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3510107760 0x7f351019baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:38.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.804+0000 7f3515d30700 1 --2- 172.21.15.100:0/1976104549 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3510102e00 0x7f35101a1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:38.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.804+0000 7f3515d30700 1 --2- 172.21.15.100:0/1976104549 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35101037b0 0x7f35101a1dd0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:38.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.804+0000 7f3515d30700 1 -- 172.21.15.100:0/1976104549 >> 172.21.15.100:0/1976104549 conn(0x7f3510078a10 msgr2=0x7f35100ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:38.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.804+0000 7f3515d30700 1 -- 172.21.15.100:0/1976104549 shutdown_connections 2024-05-10T11:30:38.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:38.804+0000 7f3515d30700 1 -- 172.21.15.100:0/1976104549 wait complete. 2024-05-10T11:30:38.938 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.938+0000 7f5bcccd9640 1 -- 172.21.15.100:0/1392155917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5bc8104610 msgr2=0x7f5bc810aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:38.938 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.938+0000 7f5bcccd9640 1 --2- 172.21.15.100:0/1392155917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5bc8104610 0x7f5bc810aea0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0009ca0 tx=0x7f5bc002a8c0 comp rx=0 tx=0).stop 2024-05-10T11:30:38.938 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.938+0000 7f5bcccd9640 1 -- 172.21.15.100:0/1392155917 shutdown_connections 2024-05-10T11:30:38.938 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.938+0000 7f5bcccd9640 1 --2- 172.21.15.100:0/1392155917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5bc8104610 0x7f5bc810aea0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:38.938 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.938+0000 7f5bcccd9640 1 --2- 172.21.15.100:0/1392155917 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5bc8103c50 0x7f5bc81040d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:38.938 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.938+0000 7f5bcccd9640 1 --2- 172.21.15.100:0/1392155917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5bc8102a60 0x7f5bc8102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:38.938 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.938+0000 7f5bcccd9640 1 -- 172.21.15.100:0/1392155917 >> 172.21.15.100:0/1392155917 conn(0x7f5bc80fdfd0 msgr2=0x7f5bc8100410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:38.938 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.938+0000 7f5bcccd9640 1 -- 172.21.15.100:0/1392155917 shutdown_connections 2024-05-10T11:30:38.939 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.938+0000 7f5bcccd9640 1 -- 172.21.15.100:0/1392155917 wait complete. 2024-05-10T11:30:38.941 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.939+0000 7f5bcccd9640 1 Processor -- start 2024-05-10T11:30:38.941 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.939+0000 7f5bcccd9640 1 -- start start 2024-05-10T11:30:38.941 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.939+0000 7f5bcccd9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5bc8102a60 0x7f5bc8198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:38.941 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.939+0000 7f5bcccd9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5bc8103c50 0x7f5bc8198fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:38.941 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.939+0000 7f5bcccd9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5bc8104610 0x7f5bc81a0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:38.941 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.939+0000 7f5bcccd9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc810e490 con 0x7f5bc8103c50 2024-05-10T11:30:38.941 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.939+0000 7f5bcccd9640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc810e310 con 0x7f5bc8102a60 2024-05-10T11:30:38.941 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.939+0000 7f5bcccd9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc810e610 con 0x7f5bc8104610 2024-05-10T11:30:38.941 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.939+0000 7f5bc5d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5bc8103c50 0x7f5bc8198fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:38.941 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.939+0000 7f5bc5d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5bc8103c50 0x7f5bc8198fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34282/0 (socket says 172.21.15.100:34282) 2024-05-10T11:30:38.941 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.939+0000 7f5bc5d9b640 1 -- 172.21.15.100:0/1405428650 learned_addr learned my addr 172.21.15.100:0/1405428650 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:30:38.941 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.939+0000 7f5bc659c640 1 --2- 172.21.15.100:0/1405428650 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5bc8102a60 0x7f5bc8198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:38.941 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.939+0000 7f5bc6d9d640 1 --2- 172.21.15.100:0/1405428650 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5bc8104610 0x7f5bc81a0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:38.941 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.939+0000 7f5bc659c640 1 -- 172.21.15.100:0/1405428650 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5bc8104610 msgr2=0x7f5bc81a0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:38.941 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.939+0000 7f5bc659c640 1 --2- 172.21.15.100:0/1405428650 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5bc8104610 0x7f5bc81a0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:38.942 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.939+0000 7f5bc659c640 1 -- 172.21.15.100:0/1405428650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5bc8103c50 msgr2=0x7f5bc8198fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:38.942 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.939+0000 7f5bc659c640 1 --2- 172.21.15.100:0/1405428650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5bc8103c50 0x7f5bc8198fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:38.942 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.939+0000 7f5bc659c640 1 -- 172.21.15.100:0/1405428650 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bc81a06c0 con 0x7f5bc8102a60 2024-05-10T11:30:38.942 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.939+0000 7f5bc5d9b640 1 --2- 172.21.15.100:0/1405428650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5bc8103c50 0x7f5bc8198fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:30:38.942 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.940+0000 7f5bc659c640 1 --2- 172.21.15.100:0/1405428650 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5bc8102a60 0x7f5bc8198aa0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5bb800e9c0 tx=0x7f5bb800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:38.942 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.940+0000 7f5bb77fe640 1 -- 172.21.15.100:0/1405428650 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb800ccc0 con 0x7f5bc8102a60 2024-05-10T11:30:38.942 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.940+0000 7f5bb77fe640 1 -- 172.21.15.100:0/1405428650 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5bb80035f0 con 0x7f5bc8102a60 2024-05-10T11:30:38.942 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.940+0000 7f5bb77fe640 1 -- 172.21.15.100:0/1405428650 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb800ce60 con 0x7f5bc8102a60 2024-05-10T11:30:38.942 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.940+0000 7f5bcccd9640 1 -- 172.21.15.100:0/1405428650 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bc81a09b0 con 0x7f5bc8102a60 2024-05-10T11:30:38.942 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.940+0000 7f5bcccd9640 1 -- 172.21.15.100:0/1405428650 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bc81a0eb0 con 0x7f5bc8102a60 2024-05-10T11:30:38.948 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.942+0000 7f5bcf9f1740 1 Processor -- start 2024-05-10T11:30:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.942+0000 7f5bcf9f1740 1 -- start start 2024-05-10T11:30:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.943+0000 7f5bcf9f1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558a6782db0 0x5558a67831d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.943+0000 7f5bcf9f1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558a67837a0 0x5558a67882a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.943+0000 7f5bcf9f1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5558a6788910 0x5558a678ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.943+0000 7f5bcf9f1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5558a66e75d0 con 0x5558a6782db0 2024-05-10T11:30:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.943+0000 7f5bcf9f1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5558a66a0890 con 0x5558a67837a0 2024-05-10T11:30:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.943+0000 7f5bcf9f1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5558a66e3590 con 0x5558a6788910 2024-05-10T11:30:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.943+0000 7f5bb77fe640 1 -- 172.21.15.100:0/1405428650 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f5bb8004000 con 0x7f5bc8102a60 2024-05-10T11:30:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.943+0000 7f5bc6d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558a6782db0 0x5558a67831d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.943+0000 7f5bc6d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558a6782db0 0x5558a67831d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34286/0 (socket says 172.21.15.100:34286) 2024-05-10T11:30:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.943+0000 7f5bc6d9d640 1 -- 172.21.15.100:0/4007205378 learned_addr learned my addr 172.21.15.100:0/4007205378 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:30:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.943+0000 7f5bc5d9b640 1 --2- 172.21.15.100:0/4007205378 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5558a6788910 0x5558a678ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.943+0000 7f5bc659c640 1 --2- 172.21.15.100:0/4007205378 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558a67837a0 0x5558a67882a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.943+0000 7f5bc6d9d640 1 -- 172.21.15.100:0/4007205378 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5558a6788910 msgr2=0x5558a678ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:38.950 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.943+0000 7f5bc6d9d640 1 --2- 172.21.15.100:0/4007205378 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5558a6788910 0x5558a678ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:38.950 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.943+0000 7f5bc6d9d640 1 -- 172.21.15.100:0/4007205378 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558a67837a0 msgr2=0x5558a67882a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:38.950 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.943+0000 7f5bc6d9d640 1 --2- 172.21.15.100:0/4007205378 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558a67837a0 0x5558a67882a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:38.950 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.943+0000 7f5bc6d9d640 1 -- 172.21.15.100:0/4007205378 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558a63f9210 con 0x5558a6782db0 2024-05-10T11:30:38.950 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.943+0000 7f5bc5d9b640 1 --2- 172.21.15.100:0/4007205378 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5558a6788910 0x5558a678ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:30:38.950 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.943+0000 7f5bc659c640 1 --2- 172.21.15.100:0/4007205378 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558a67837a0 0x5558a67882a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:30:38.950 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.943+0000 7f5bc6d9d640 1 --2- 172.21.15.100:0/4007205378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558a6782db0 0x5558a67831d0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f5bc002ade0 tx=0x7f5bc0005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81ef6bfa13787517 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:38.950 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.944+0000 7f5bb4ff9640 1 -- 172.21.15.100:0/4007205378 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc00316d0 con 0x5558a6782db0 2024-05-10T11:30:38.950 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.944+0000 7f5bb4ff9640 1 -- 172.21.15.100:0/4007205378 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5bc0036030 con 0x5558a6782db0 2024-05-10T11:30:38.950 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.944+0000 7f5bb4ff9640 1 -- 172.21.15.100:0/4007205378 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc00043c0 con 0x5558a6782db0 2024-05-10T11:30:38.950 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.944+0000 7f5bb77fe640 1 --2- 172.21.15.100:0/1405428650 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f5bb00b28a0 0x7f5bb00b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:38.950 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.944+0000 7f5bc659c640 1 --2- 172.21.15.100:0/1405428650 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f5bb00b28a0 0x7f5bb00b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:38.950 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.944+0000 7f5bb77fe640 1 -- 172.21.15.100:0/1405428650 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f5bb80e2900 con 0x7f5bc8102a60 2024-05-10T11:30:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.944+0000 7f5bcf9f1740 1 -- 172.21.15.100:0/4007205378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558a6782db0 msgr2=0x5558a67831d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.944+0000 7f5bc659c640 1 --2- 172.21.15.100:0/1405428650 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f5bb00b28a0 0x7f5bb00b4d60 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f5bb80e0410 tx=0x7f5bb8023040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.944+0000 7f5bcf9f1740 1 --2- 172.21.15.100:0/4007205378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558a6782db0 0x5558a67831d0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f5bc002ade0 tx=0x7f5bc0005ef0 comp rx=0 tx=0).stop 2024-05-10T11:30:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.944+0000 7f5bcf9f1740 1 -- 172.21.15.100:0/4007205378 shutdown_connections 2024-05-10T11:30:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.944+0000 7f5bcf9f1740 1 --2- 172.21.15.100:0/4007205378 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5558a6788910 0x5558a678ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.944+0000 7f5bcf9f1740 1 --2- 172.21.15.100:0/4007205378 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558a67837a0 0x5558a67882a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.944+0000 7f5bcf9f1740 1 --2- 172.21.15.100:0/4007205378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558a6782db0 0x5558a67831d0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.944+0000 7f5bcf9f1740 1 -- 172.21.15.100:0/4007205378 >> 172.21.15.100:0/4007205378 conn(0x5558a677e420 msgr2=0x5558a677e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.945+0000 7f5bcf9f1740 1 -- 172.21.15.100:0/4007205378 shutdown_connections 2024-05-10T11:30:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.945+0000 7f5bcf9f1740 1 -- 172.21.15.100:0/4007205378 wait complete. 2024-05-10T11:30:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.945+0000 7f5bcf9f1740 1 -- start start 2024-05-10T11:30:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.945+0000 7f5bcf9f1740 1 Processor -- start 2024-05-10T11:30:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.945+0000 7f5bcf9f1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558a6782db0 0x5558a678ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.945+0000 7f5bcf9f1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558a67837a0 0x5558a678d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.945+0000 7f5bcf9f1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5558a6788910 0x5558a6794380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:38.952 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.945+0000 7f5bcf9f1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5558a678b6e0 con 0x5558a6782db0 2024-05-10T11:30:38.952 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.945+0000 7f5bcf9f1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5558a66a4370 con 0x5558a67837a0 2024-05-10T11:30:38.952 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.945+0000 7f5bcf9f1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5558a678b860 con 0x5558a6788910 2024-05-10T11:30:38.952 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.945+0000 7f5bcf9f1740 10 client.0 _ll_register_callbacks cb 0x5558a63f8a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:30:38.952 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.945+0000 7f5bc6d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558a6782db0 0x5558a678ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:38.952 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.945+0000 7f5bc6d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558a6782db0 0x5558a678ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34294/0 (socket says 172.21.15.100:34294) 2024-05-10T11:30:38.952 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.945+0000 7f5bc6d9d640 1 -- 172.21.15.100:0/1552498880 learned_addr learned my addr 172.21.15.100:0/1552498880 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:30:38.952 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.945+0000 7f5bc5d9b640 1 --2- 172.21.15.100:0/1552498880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558a67837a0 0x5558a678d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:38.952 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.945+0000 7f5bc6d9d640 1 --2- 172.21.15.100:0/1552498880 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5558a6788910 0x5558a6794380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:38.952 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.946+0000 7f5bc6d9d640 1 -- 172.21.15.100:0/1552498880 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5558a6788910 msgr2=0x5558a6794380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:38.952 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.946+0000 7f5bc6d9d640 1 --2- 172.21.15.100:0/1552498880 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5558a6788910 0x5558a6794380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:38.952 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.946+0000 7f5bc6d9d640 1 -- 172.21.15.100:0/1552498880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558a67837a0 msgr2=0x5558a678d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:38.952 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.946+0000 7f5bc6d9d640 1 --2- 172.21.15.100:0/1552498880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558a67837a0 0x5558a678d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:38.952 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.946+0000 7f5bc6d9d640 1 -- 172.21.15.100:0/1552498880 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558a64c5620 con 0x5558a6782db0 2024-05-10T11:30:38.953 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.946+0000 7f5bc5d9b640 1 --2- 172.21.15.100:0/1552498880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558a67837a0 0x5558a678d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:30:38.953 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.946+0000 7f5bc6d9d640 1 --2- 172.21.15.100:0/1552498880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558a6782db0 0x5558a678ce70 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0009bf0 tx=0x7f5bc0031980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d6439e6289ac477 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:38.953 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.946+0000 7f5badffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:30:38.953 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.946+0000 7f5badffb640 1 -- 172.21.15.100:0/1552498880 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc0004550 con 0x5558a6782db0 2024-05-10T11:30:38.953 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.946+0000 7f5badffb640 1 -- 172.21.15.100:0/1552498880 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f5bc00046f0 con 0x5558a6782db0 2024-05-10T11:30:38.953 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.946+0000 7f5bcf9f1740 1 -- 172.21.15.100:0/1552498880 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5bc0036030 con 0x5558a6782db0 2024-05-10T11:30:38.953 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.946+0000 7f5badffb640 1 -- 172.21.15.100:0/1552498880 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5bc005f050 con 0x5558a6782db0 2024-05-10T11:30:38.953 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.946+0000 7f5bcf9f1740 10 client.17871 fetch_fsmap learned FSMap version 33 2024-05-10T11:30:38.953 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.946+0000 7f5bcf9f1740 1 -- 172.21.15.100:0/1552498880 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5558a678b6e0 con 0x5558a6782db0 2024-05-10T11:30:38.953 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.946+0000 7f5badffb640 1 -- 172.21.15.100:0/1552498880 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5bc0036030 con 0x5558a6782db0 2024-05-10T11:30:38.953 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.946+0000 7f5badffb640 1 -- 172.21.15.100:0/1552498880 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f5bc0043070 con 0x5558a6782db0 2024-05-10T11:30:38.953 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.947+0000 7f5bcf9f1740 10 client.17871 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T11:30:38.953 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.947+0000 7f5bcf9f1740 10 client.17871 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:30:38.953 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.947+0000 7f5bcf9f1740 20 client.17871 populate_metadata read hostname 'smithi100' 2024-05-10T11:30:38.953 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.947+0000 7f5bcf9f1740 1 --2- 172.21.15.100:0/1552498880 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5558a6798760 0x5558a679ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:38.954 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.947+0000 7f5bcf9f1740 4 client.17871 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:30:38.954 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.947+0000 7f5bcf9f1740 1 -- 172.21.15.100:0/1552498880 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5558a66f51e0 con 0x5558a6798760 2024-05-10T11:30:38.954 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.947+0000 7f5bc5d9b640 1 --2- 172.21.15.100:0/1552498880 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5558a6798760 0x5558a679ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:38.954 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.947+0000 7f5bc5d9b640 1 --2- 172.21.15.100:0/1552498880 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5558a6798760 0x5558a679ab50 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=608774f4b802976f server_cookie=da551af3a96816fa in_seq=0 out_seq=0 2024-05-10T11:30:38.954 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.947+0000 7f5badffb640 10 client.17871 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:30:38.954 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.948+0000 7f5badffb640 1 -- 172.21.15.100:0/1552498880 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5558a66f51e0 con 0x5558a6798760 2024-05-10T11:30:38.954 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.948+0000 7f5badffb640 10 client.17871 handle_command_reply: tid=0 2024-05-10T11:30:38.954 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.948+0000 7f5bcf9f1740 1 client.17871 shutdown 2024-05-10T11:30:38.954 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.948+0000 7f5bcf9f1740 1 -- 172.21.15.100:0/1552498880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558a6782db0 msgr2=0x5558a678ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:38.954 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.948+0000 7f5bcf9f1740 1 --2- 172.21.15.100:0/1552498880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558a6782db0 0x5558a678ce70 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0009bf0 tx=0x7f5bc0031980 comp rx=0 tx=0).stop 2024-05-10T11:30:38.954 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.948+0000 7f5bcf9f1740 1 -- 172.21.15.100:0/1552498880 shutdown_connections 2024-05-10T11:30:38.954 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.948+0000 7f5bcf9f1740 1 --2- 172.21.15.100:0/1552498880 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5558a6798760 0x5558a679ab50 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:38.954 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.948+0000 7f5bcf9f1740 1 --2- 172.21.15.100:0/1552498880 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5558a6788910 0x5558a6794380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:38.954 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.948+0000 7f5badffb640 0 client.17871 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:30:38.955 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.948+0000 7f5bcf9f1740 1 --2- 172.21.15.100:0/1552498880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558a67837a0 0x5558a678d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:38.955 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.948+0000 7f5bcf9f1740 1 --2- 172.21.15.100:0/1552498880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558a6782db0 0x5558a678ce70 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:38.955 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.948+0000 7f5bcf9f1740 1 -- 172.21.15.100:0/1552498880 >> 172.21.15.100:0/1552498880 conn(0x5558a677e420 msgr2=0x5558a677d580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:38.955 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.948+0000 7f5bcf9f1740 1 -- 172.21.15.100:0/1552498880 shutdown_connections 2024-05-10T11:30:38.955 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.948+0000 7f5bcf9f1740 1 -- 172.21.15.100:0/1552498880 wait complete. 2024-05-10T11:30:38.955 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.948+0000 7f5bcf9f1740 20 client.17871 trim_cache size 0 max 16384 2024-05-10T11:30:38.966 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.965+0000 7f5bcf9f1740 1 Processor -- start 2024-05-10T11:30:38.966 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.965+0000 7f5bcf9f1740 1 -- start start 2024-05-10T11:30:38.966 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.965+0000 7f5bcf9f1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558a677e420 0x5558a678ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.965+0000 7f5bcf9f1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558a6782db0 0x5558a678cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.965+0000 7f5bcf9f1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5558a67837a0 0x5558a6781330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.965+0000 7f5bcf9f1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc81adaa0 con 0x5558a6782db0 2024-05-10T11:30:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.965+0000 7f5bcf9f1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc81ab370 con 0x5558a677e420 2024-05-10T11:30:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.965+0000 7f5bcf9f1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc81a91c0 con 0x5558a67837a0 2024-05-10T11:30:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.965+0000 7f5bc5d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558a6782db0 0x5558a678cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.966+0000 7f5bc5d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558a6782db0 0x5558a678cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34296/0 (socket says 172.21.15.100:34296) 2024-05-10T11:30:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.966+0000 7f5bc5d9b640 1 -- 172.21.15.100:0/2449400473 learned_addr learned my addr 172.21.15.100:0/2449400473 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:30:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.966+0000 7f5bc6d9d640 1 --2- 172.21.15.100:0/2449400473 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5558a67837a0 0x5558a6781330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.966+0000 7f5bc6d9d640 1 --2- 172.21.15.100:0/2449400473 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558a677e420 0x5558a678ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.966+0000 7f5bc5d9b640 1 -- 172.21.15.100:0/2449400473 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5558a67837a0 msgr2=0x5558a6781330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.966+0000 7f5bc5d9b640 1 --2- 172.21.15.100:0/2449400473 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5558a67837a0 0x5558a6781330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.966+0000 7f5bc5d9b640 1 -- 172.21.15.100:0/2449400473 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558a677e420 msgr2=0x5558a678ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.966+0000 7f5bc5d9b640 1 --2- 172.21.15.100:0/2449400473 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558a677e420 0x5558a678ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.966+0000 7f5bc5d9b640 1 -- 172.21.15.100:0/2449400473 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558a67829f0 con 0x5558a6782db0 2024-05-10T11:30:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.966+0000 7f5bc5d9b640 1 --2- 172.21.15.100:0/2449400473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558a6782db0 0x5558a678cfc0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x5558a677f2f0 tx=0x7f5bbc005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a5a83e46fa86a3c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:38.968 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.966+0000 7f5baffff640 1 -- 172.21.15.100:0/2449400473 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bbc00fc20 con 0x5558a6782db0 2024-05-10T11:30:38.968 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.966+0000 7f5baffff640 1 -- 172.21.15.100:0/2449400473 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5bbc00fdc0 con 0x5558a6782db0 2024-05-10T11:30:38.968 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.966+0000 7f5baffff640 1 -- 172.21.15.100:0/2449400473 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bbc004c70 con 0x5558a6782db0 2024-05-10T11:30:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.967+0000 7f5bcf9f1740 1 -- 172.21.15.100:0/2449400473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558a6782db0 msgr2=0x5558a678cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.967+0000 7f5bcf9f1740 1 --2- 172.21.15.100:0/2449400473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558a6782db0 0x5558a678cfc0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x5558a677f2f0 tx=0x7f5bbc005f90 comp rx=0 tx=0).stop 2024-05-10T11:30:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.967+0000 7f5bcf9f1740 1 -- 172.21.15.100:0/2449400473 shutdown_connections 2024-05-10T11:30:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.967+0000 7f5bcf9f1740 1 --2- 172.21.15.100:0/2449400473 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5558a67837a0 0x5558a6781330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.967+0000 7f5bcf9f1740 1 --2- 172.21.15.100:0/2449400473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558a6782db0 0x5558a678cfc0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.967+0000 7f5bcf9f1740 1 --2- 172.21.15.100:0/2449400473 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558a677e420 0x5558a678ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.967+0000 7f5bcf9f1740 1 -- 172.21.15.100:0/2449400473 >> 172.21.15.100:0/2449400473 conn(0x5558a67883e0 msgr2=0x5558a67793e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.967+0000 7f5bcf9f1740 1 -- 172.21.15.100:0/2449400473 shutdown_connections 2024-05-10T11:30:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.967+0000 7f5bcf9f1740 1 -- 172.21.15.100:0/2449400473 wait complete. 2024-05-10T11:30:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.967+0000 7f5bcf9f1740 1 -- start start 2024-05-10T11:30:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.967+0000 7f5bcf9f1740 1 Processor -- start 2024-05-10T11:30:38.972 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.968+0000 7f5bcf9f1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558a677e420 0x5558a6791640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:38.972 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.968+0000 7f5bcf9f1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5558a6782db0 0x5558a6791b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:38.972 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.968+0000 7f5bcf9f1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558a67837a0 0x5558a6792a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:38.972 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.968+0000 7f5bcf9f1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc81a9830 con 0x5558a67837a0 2024-05-10T11:30:38.972 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.968+0000 7f5bcf9f1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc81aaca0 con 0x5558a677e420 2024-05-10T11:30:38.972 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.968+0000 7f5bcf9f1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc81a9f00 con 0x5558a6782db0 2024-05-10T11:30:38.972 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.968+0000 7f5bcf9f1740 10 client.0 _ll_register_callbacks cb 0x7f5bbc009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:30:38.972 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.968+0000 7f5bc6d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558a67837a0 0x5558a6792a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:38.972 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.968+0000 7f5bc6d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558a67837a0 0x5558a6792a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34310/0 (socket says 172.21.15.100:34310) 2024-05-10T11:30:38.972 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.968+0000 7f5bc6d9d640 1 -- 172.21.15.100:0/3989013727 learned_addr learned my addr 172.21.15.100:0/3989013727 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:30:38.972 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.968+0000 7f5bc5d9b640 1 --2- 172.21.15.100:0/3989013727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5558a6782db0 0x5558a6791b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:38.972 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.968+0000 7f5bc6d9d640 1 --2- 172.21.15.100:0/3989013727 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558a677e420 0x5558a6791640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:38.972 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.968+0000 7f5bc6d9d640 1 -- 172.21.15.100:0/3989013727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5558a6782db0 msgr2=0x5558a6791b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:38.972 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.968+0000 7f5bc6d9d640 1 --2- 172.21.15.100:0/3989013727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5558a6782db0 0x5558a6791b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:38.972 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.968+0000 7f5bc6d9d640 1 -- 172.21.15.100:0/3989013727 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558a677e420 msgr2=0x5558a6791640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:38.973 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.968+0000 7f5bc6d9d640 1 --2- 172.21.15.100:0/3989013727 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558a677e420 0x5558a6791640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:38.973 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.968+0000 7f5bc6d9d640 1 -- 172.21.15.100:0/3989013727 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558a677e250 con 0x5558a67837a0 2024-05-10T11:30:38.973 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.968+0000 7f5bc6d9d640 1 --2- 172.21.15.100:0/3989013727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558a67837a0 0x5558a6792a80 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0009620 tx=0x7f5bc0031980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff037190fa1fdea1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:38.973 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.968+0000 7f5badffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:30:38.973 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.968+0000 7f5badffb640 1 -- 172.21.15.100:0/3989013727 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc0004d40 con 0x5558a67837a0 2024-05-10T11:30:38.973 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.968+0000 7f5bcf9f1740 1 -- 172.21.15.100:0/3989013727 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5bbc00fdc0 con 0x5558a67837a0 2024-05-10T11:30:38.973 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.968+0000 7f5badffb640 1 -- 172.21.15.100:0/3989013727 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f5bc0045aa0 con 0x5558a67837a0 2024-05-10T11:30:38.973 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.968+0000 7f5badffb640 1 -- 172.21.15.100:0/3989013727 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5bc0045810 con 0x5558a67837a0 2024-05-10T11:30:38.973 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.968+0000 7f5bcf9f1740 10 client.17877 fetch_fsmap learned FSMap version 33 2024-05-10T11:30:38.973 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.968+0000 7f5bcf9f1740 1 -- 172.21.15.100:0/3989013727 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5558a6803210 con 0x5558a67837a0 2024-05-10T11:30:38.973 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.969+0000 7f5badffb640 1 -- 172.21.15.100:0/3989013727 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5bbc00fdc0 con 0x5558a67837a0 2024-05-10T11:30:38.973 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.969+0000 7f5badffb640 1 -- 172.21.15.100:0/3989013727 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f5bc0043040 con 0x5558a67837a0 2024-05-10T11:30:38.973 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.969+0000 7f5bcf9f1740 10 client.17877 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T11:30:38.973 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.969+0000 7f5bcf9f1740 10 client.17877 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:30:38.973 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.969+0000 7f5bcf9f1740 20 client.17877 populate_metadata read hostname 'smithi100' 2024-05-10T11:30:38.974 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.969+0000 7f5bcf9f1740 1 --2- 172.21.15.100:0/3989013727 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5558a6788910 0x5558a6805bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:38.974 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.969+0000 7f5bcf9f1740 4 client.17877 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:30:38.974 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.969+0000 7f5bcf9f1740 1 -- 172.21.15.100:0/3989013727 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5558a66ef050 con 0x5558a6788910 2024-05-10T11:30:38.974 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.969+0000 7f5bc5d9b640 1 --2- 172.21.15.100:0/3989013727 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5558a6788910 0x5558a6805bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:38.974 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.969+0000 7f5badffb640 10 client.17877 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:30:38.974 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.969+0000 7f5bc5d9b640 1 --2- 172.21.15.100:0/3989013727 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5558a6788910 0x5558a6805bc0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=902ee2ce2b9f58fc server_cookie=d3b49cda9e7cd20 in_seq=0 out_seq=0 2024-05-10T11:30:38.983 INFO:tasks.workunit.client.0.smithi100.stdout: 1 323018 2.72 MB/sec execute 513 sec latency 19.031 ms 2024-05-10T11:30:38.992 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.992+0000 7f5badffb640 1 -- 172.21.15.100:0/3989013727 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5558a66ef050 con 0x5558a6788910 2024-05-10T11:30:38.992 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.992+0000 7f5badffb640 10 client.17877 handle_command_reply: tid=0 2024-05-10T11:30:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.992+0000 7f5bcf9f1740 1 client.17877 shutdown 2024-05-10T11:30:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.992+0000 7f5bcf9f1740 1 -- 172.21.15.100:0/3989013727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558a67837a0 msgr2=0x5558a6792a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.992+0000 7f5bcf9f1740 1 --2- 172.21.15.100:0/3989013727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558a67837a0 0x5558a6792a80 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0009620 tx=0x7f5bc0031980 comp rx=0 tx=0).stop 2024-05-10T11:30:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.992+0000 7f5bcf9f1740 1 -- 172.21.15.100:0/3989013727 shutdown_connections 2024-05-10T11:30:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.992+0000 7f5bcf9f1740 1 --2- 172.21.15.100:0/3989013727 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5558a6788910 0x5558a6805bc0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.992+0000 7f5bcf9f1740 1 --2- 172.21.15.100:0/3989013727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558a67837a0 0x5558a6792a80 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.992+0000 7f5bcf9f1740 1 --2- 172.21.15.100:0/3989013727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5558a6782db0 0x5558a6791b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.992+0000 7f5badffb640 0 client.17877 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:30:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.992+0000 7f5bcf9f1740 1 --2- 172.21.15.100:0/3989013727 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558a677e420 0x5558a6791640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.992+0000 7f5bcf9f1740 1 -- 172.21.15.100:0/3989013727 >> 172.21.15.100:0/3989013727 conn(0x5558a67883e0 msgr2=0x5558a678ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.992+0000 7f5bcf9f1740 1 -- 172.21.15.100:0/3989013727 shutdown_connections 2024-05-10T11:30:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.992+0000 7f5bcf9f1740 1 -- 172.21.15.100:0/3989013727 wait complete. 2024-05-10T11:30:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.992+0000 7f5bcf9f1740 20 client.17877 trim_cache size 0 max 16384 2024-05-10T11:30:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:30:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:30:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:30:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:30:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.993+0000 7f5bcccd9640 1 -- 172.21.15.100:0/1405428650 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f5bb00b28a0 msgr2=0x7f5bb00b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.993+0000 7f5bcccd9640 1 --2- 172.21.15.100:0/1405428650 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f5bb00b28a0 0x7f5bb00b4d60 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f5bb80e0410 tx=0x7f5bb8023040 comp rx=0 tx=0).stop 2024-05-10T11:30:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.993+0000 7f5bcccd9640 1 -- 172.21.15.100:0/1405428650 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5bc8102a60 msgr2=0x7f5bc8198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.993+0000 7f5bcccd9640 1 --2- 172.21.15.100:0/1405428650 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5bc8102a60 0x7f5bc8198aa0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5bb800e9c0 tx=0x7f5bb800ee80 comp rx=0 tx=0).stop 2024-05-10T11:30:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.993+0000 7f5bcccd9640 1 -- 172.21.15.100:0/1405428650 shutdown_connections 2024-05-10T11:30:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.993+0000 7f5bcccd9640 1 --2- 172.21.15.100:0/1405428650 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f5bb00b28a0 0x7f5bb00b4d60 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.993+0000 7f5bcccd9640 1 --2- 172.21.15.100:0/1405428650 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5bc8104610 0x7f5bc81a0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.993+0000 7f5bcccd9640 1 --2- 172.21.15.100:0/1405428650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5bc8103c50 0x7f5bc8198fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.993+0000 7f5bcccd9640 1 --2- 172.21.15.100:0/1405428650 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5bc8102a60 0x7f5bc8198aa0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.993+0000 7f5bcccd9640 1 -- 172.21.15.100:0/1405428650 >> 172.21.15.100:0/1405428650 conn(0x7f5bc80fdfd0 msgr2=0x7f5bc8106f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.993+0000 7f5bcccd9640 1 -- 172.21.15.100:0/1405428650 shutdown_connections 2024-05-10T11:30:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:38.993+0000 7f5bcccd9640 1 -- 172.21.15.100:0/1405428650 wait complete. 2024-05-10T11:30:39.668 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T11:30:39.668 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T11:30:39.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:30:39 smithi188 ceph-mon[34075]: pgmap v789: 129 pgs: 129 active+clean; 127 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 1.4 MiB/s wr, 161 op/s 2024-05-10T11:30:39.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:30:39 smithi100 ceph-mon[30296]: pgmap v789: 129 pgs: 129 active+clean; 127 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 1.4 MiB/s wr, 161 op/s 2024-05-10T11:30:39.923 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:30:39.983 INFO:tasks.workunit.client.0.smithi100.stdout: 1 323247 2.72 MB/sec execute 514 sec latency 18.341 ms 2024-05-10T11:30:40.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:30:39 smithi178 ceph-mon[33359]: pgmap v789: 129 pgs: 129 active+clean; 127 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 1.4 MiB/s wr, 161 op/s 2024-05-10T11:30:40.983 INFO:tasks.workunit.client.0.smithi100.stdout: 1 323602 2.71 MB/sec execute 515 sec latency 15.056 ms 2024-05-10T11:30:41.722 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:30:41 smithi100 ceph-mon[30296]: pgmap v790: 129 pgs: 129 active+clean; 126 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 1.2 MiB/s wr, 151 op/s 2024-05-10T11:30:41.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:41.726+0000 7fe871b93700 1 -- 172.21.15.100:0/2372077032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe86c06b1b0 msgr2=0x7fe86c06b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:41.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:41.726+0000 7fe871b93700 1 --2- 172.21.15.100:0/2372077032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe86c06b1b0 0x7fe86c06b590 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fe85c009990 tx=0x7fe85c009ca0 comp rx=0 tx=0).stop 2024-05-10T11:30:41.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:41.727+0000 7fe871b93700 1 -- 172.21.15.100:0/2372077032 shutdown_connections 2024-05-10T11:30:41.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:41.727+0000 7fe871b93700 1 --2- 172.21.15.100:0/2372077032 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe86c10ff50 0x7fe86c112340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:41.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:41.727+0000 7fe871b93700 1 --2- 172.21.15.100:0/2372077032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe86c06b1b0 0x7fe86c06b590 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:41.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:41.727+0000 7fe871b93700 1 --2- 172.21.15.100:0/2372077032 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe86c06bad0 0x7fe86c10fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:41.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:41.727+0000 7fe871b93700 1 -- 172.21.15.100:0/2372077032 >> 172.21.15.100:0/2372077032 conn(0x7fe86c079350 msgr2=0x7fe86c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:41.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:41.727+0000 7fe871b93700 1 -- 172.21.15.100:0/2372077032 shutdown_connections 2024-05-10T11:30:41.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:41.727+0000 7fe871b93700 1 -- 172.21.15.100:0/2372077032 wait complete. 2024-05-10T11:30:41.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:41.728+0000 7fe871b93700 1 Processor -- start 2024-05-10T11:30:41.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:41.728+0000 7fe871b93700 1 -- start start 2024-05-10T11:30:41.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:41.728+0000 7fe871b93700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe86c06b1b0 0x7fe86c1a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:41.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:41.728+0000 7fe871b93700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe86c06bad0 0x7fe86c1a1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:41.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:41.728+0000 7fe871b93700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe86c10ff50 0x7fe86c19bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:41.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:41.728+0000 7fe871b93700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe86c115710 con 0x7fe86c06b1b0 2024-05-10T11:30:41.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:41.728+0000 7fe871b93700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe86c115590 con 0x7fe86c10ff50 2024-05-10T11:30:41.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:41.728+0000 7fe871b93700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe86c115890 con 0x7fe86c06bad0 2024-05-10T11:30:41.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:41.728+0000 7fe86bfff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe86c10ff50 0x7fe86c19bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:41.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:41.728+0000 7fe86bfff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe86c10ff50 0x7fe86c19bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:60070/0 (socket says 172.21.15.100:60070) 2024-05-10T11:30:41.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:41.728+0000 7fe86bfff700 1 -- 172.21.15.100:0/2581196439 learned_addr learned my addr 172.21.15.100:0/2581196439 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:30:41.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:41.729+0000 7fe86affd700 1 --2- 172.21.15.100:0/2581196439 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe86c06bad0 0x7fe86c1a1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:41.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:41.729+0000 7fe86b7fe700 1 --2- 172.21.15.100:0/2581196439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe86c06b1b0 0x7fe86c1a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:41.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:41.729+0000 7fe86bfff700 1 -- 172.21.15.100:0/2581196439 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe86c06bad0 msgr2=0x7fe86c1a1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:41.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:41.729+0000 7fe86bfff700 1 --2- 172.21.15.100:0/2581196439 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe86c06bad0 0x7fe86c1a1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:41.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:41.729+0000 7fe86bfff700 1 -- 172.21.15.100:0/2581196439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe86c06b1b0 msgr2=0x7fe86c1a1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:41.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:41.729+0000 7fe86bfff700 1 --2- 172.21.15.100:0/2581196439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe86c06b1b0 0x7fe86c1a1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:41.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:41.729+0000 7fe86bfff700 1 -- 172.21.15.100:0/2581196439 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe86c19c230 con 0x7fe86c10ff50 2024-05-10T11:30:41.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:41.729+0000 7fe86affd700 1 --2- 172.21.15.100:0/2581196439 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe86c06bad0 0x7fe86c1a1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:30:41.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:41.729+0000 7fe86bfff700 1 --2- 172.21.15.100:0/2581196439 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe86c10ff50 0x7fe86c19bad0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe86400ef90 tx=0x7fe86400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:41.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:41.729+0000 7fe868ff9700 1 -- 172.21.15.100:0/2581196439 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8640092e0 con 0x7fe86c10ff50 2024-05-10T11:30:41.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:41.729+0000 7fe868ff9700 1 -- 172.21.15.100:0/2581196439 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe864004bb0 con 0x7fe86c10ff50 2024-05-10T11:30:41.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:41.729+0000 7fe871b93700 1 -- 172.21.15.100:0/2581196439 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe86c19c4c0 con 0x7fe86c10ff50 2024-05-10T11:30:41.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:41.729+0000 7fe871b93700 1 -- 172.21.15.100:0/2581196439 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe86c1a87e0 con 0x7fe86c10ff50 2024-05-10T11:30:41.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:41.730+0000 7fe868ff9700 1 -- 172.21.15.100:0/2581196439 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe864004020 con 0x7fe86c10ff50 2024-05-10T11:30:41.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:41.731+0000 7fe871b93700 1 -- 172.21.15.100:0/2581196439 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe86c06a190 con 0x7fe86c10ff50 2024-05-10T11:30:41.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:41.732+0000 7fe868ff9700 1 -- 172.21.15.100:0/2581196439 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fe86401b030 con 0x7fe86c10ff50 2024-05-10T11:30:41.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:41.733+0000 7fe868ff9700 1 --2- 172.21.15.100:0/2581196439 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe8540b2880 0x7fe8540b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:41.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:41.733+0000 7fe86b7fe700 1 --2- 172.21.15.100:0/2581196439 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe8540b2880 0x7fe8540b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:41.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:41.733+0000 7fe868ff9700 1 -- 172.21.15.100:0/2581196439 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fe8640e2220 con 0x7fe86c10ff50 2024-05-10T11:30:41.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:41.733+0000 7fe86b7fe700 1 --2- 172.21.15.100:0/2581196439 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe8540b2880 0x7fe8540b4d40 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fe85c00b2e0 tx=0x7fe85c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:41.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:41.736+0000 7fe868ff9700 1 -- 172.21.15.100:0/2581196439 <== mon.2 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fe864022020 con 0x7fe86c10ff50 2024-05-10T11:30:41.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:30:41 smithi188 ceph-mon[34075]: pgmap v790: 129 pgs: 129 active+clean; 126 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 1.2 MiB/s wr, 151 op/s 2024-05-10T11:30:41.983 INFO:tasks.workunit.client.0.smithi100.stdout: 1 324046 2.71 MB/sec execute 516 sec latency 20.237 ms 2024-05-10T11:30:42.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:30:41 smithi178 ceph-mon[33359]: pgmap v790: 129 pgs: 129 active+clean; 126 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 1.2 MiB/s wr, 151 op/s 2024-05-10T11:30:42.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:42.023+0000 7fe871b93700 1 -- 172.21.15.100:0/2581196439 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe86c02e510 con 0x7fe86c10ff50 2024-05-10T11:30:42.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:42.024+0000 7fe868ff9700 1 -- 172.21.15.100:0/2581196439 <== mon.2 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11897 (secure 0 0 0) 0x7fe864010040 con 0x7fe86c10ff50 2024-05-10T11:30:42.024 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:30:42.024 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2072225012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2072225012},{"type":"v1","addr":"172.21.15.100:6835","nonce":2072225012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/401939540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":401939540},{"type":"v1","addr":"172.21.15.100:6837","nonce":401939540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14571,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/3946311025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3946311025},{"type":"v1","addr":"172.21.15.100:6839","nonce":3946311025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3736371660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3736371660},{"type":"v1","addr":"172.21.15.100:6841","nonce":3736371660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/2741810791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2741810791},{"type":"v1","addr":"172.21.15.178:6833","nonce":2741810791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1704891041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1704891041},{"type":"v1","addr":"172.21.15.188:6833","nonce":1704891041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T11:14:27.242452+0000","modified":"2024-05-10T11:29:07.529595+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24439,"mds_1":24437,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/2124076312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2124076312},{"type":"v1","addr":"172.21.15.178:6835","nonce":2124076312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/3224358425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3224358425},{"type":"v1","addr":"172.21.15.178:6837","nonce":3224358425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/1943342620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1943342620},{"type":"v1","addr":"172.21.15.188:6837","nonce":1943342620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.178:6839/3760985404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3760985404},{"type":"v1","addr":"172.21.15.178:6839","nonce":3760985404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.188:6839/3194917070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":3194917070},{"type":"v1","addr":"172.21.15.188:6839","nonce":3194917070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27034":{"gid":27034,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/3139559450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3139559450},{"type":"v1","addr":"172.21.15.188:6835","nonce":3139559450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24439,"qdb_cluster":[24427,24437,24439]},"id":1}]} 2024-05-10T11:30:42.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:42.026+0000 7fe871b93700 1 -- 172.21.15.100:0/2581196439 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe8540b2880 msgr2=0x7fe8540b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:42.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:42.026+0000 7fe871b93700 1 --2- 172.21.15.100:0/2581196439 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe8540b2880 0x7fe8540b4d40 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fe85c00b2e0 tx=0x7fe85c005fb0 comp rx=0 tx=0).stop 2024-05-10T11:30:42.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:42.026+0000 7fe871b93700 1 -- 172.21.15.100:0/2581196439 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe86c10ff50 msgr2=0x7fe86c19bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:42.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:42.026+0000 7fe871b93700 1 --2- 172.21.15.100:0/2581196439 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe86c10ff50 0x7fe86c19bad0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe86400ef90 tx=0x7fe86400c3a0 comp rx=0 tx=0).stop 2024-05-10T11:30:42.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:42.026+0000 7fe871b93700 1 -- 172.21.15.100:0/2581196439 shutdown_connections 2024-05-10T11:30:42.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:42.026+0000 7fe871b93700 1 --2- 172.21.15.100:0/2581196439 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe8540b2880 0x7fe8540b4d40 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:42.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:42.026+0000 7fe871b93700 1 --2- 172.21.15.100:0/2581196439 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe86c10ff50 0x7fe86c19bad0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:42.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:42.026+0000 7fe871b93700 1 --2- 172.21.15.100:0/2581196439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe86c06b1b0 0x7fe86c1a1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:42.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:42.026+0000 7fe871b93700 1 --2- 172.21.15.100:0/2581196439 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe86c06bad0 0x7fe86c1a1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:42.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:42.026+0000 7fe871b93700 1 -- 172.21.15.100:0/2581196439 >> 172.21.15.100:0/2581196439 conn(0x7fe86c079350 msgr2=0x7fe86c111520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:42.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:42.027+0000 7fe871b93700 1 -- 172.21.15.100:0/2581196439 shutdown_connections 2024-05-10T11:30:42.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:42.027+0000 7fe871b93700 1 -- 172.21.15.100:0/2581196439 wait complete. 2024-05-10T11:30:42.028 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 33 2024-05-10T11:30:42.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:30:42 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/2581196439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:30:42.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:30:42 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/2581196439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:30:42.824 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-10T11:30:42.983 INFO:tasks.workunit.client.0.smithi100.stdout: 1 324663 2.71 MB/sec execute 517 sec latency 21.275 ms 2024-05-10T11:30:43.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:30:42 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/2581196439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:30:43.083 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:30:43.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:30:43 smithi188 ceph-mon[34075]: pgmap v791: 129 pgs: 129 active+clean; 127 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 1.3 MiB/s wr, 143 op/s 2024-05-10T11:30:43.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:30:43 smithi100 ceph-mon[30296]: pgmap v791: 129 pgs: 129 active+clean; 127 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 1.3 MiB/s wr, 143 op/s 2024-05-10T11:30:43.983 INFO:tasks.workunit.client.0.smithi100.stdout: 1 325208 2.71 MB/sec execute 518 sec latency 16.626 ms 2024-05-10T11:30:44.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:30:43 smithi178 ceph-mon[33359]: pgmap v791: 129 pgs: 129 active+clean; 127 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 1.3 MiB/s wr, 143 op/s 2024-05-10T11:30:44.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.841+0000 7f1c0c3fd700 1 -- 172.21.15.100:0/2602908056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c04102d50 msgr2=0x7f1c0410f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:44.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.841+0000 7f1c0c3fd700 1 --2- 172.21.15.100:0/2602908056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c04102d50 0x7f1c0410f240 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc00b0a0 tx=0x7f1bfc00b3b0 comp rx=0 tx=0).stop 2024-05-10T11:30:44.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.841+0000 7f1c0c3fd700 1 -- 172.21.15.100:0/2602908056 shutdown_connections 2024-05-10T11:30:44.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.841+0000 7f1c0c3fd700 1 --2- 172.21.15.100:0/2602908056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c04102d50 0x7f1c0410f240 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:44.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.841+0000 7f1c0c3fd700 1 --2- 172.21.15.100:0/2602908056 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c04108400 0x7f1c041087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:44.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.841+0000 7f1c0c3fd700 1 --2- 172.21.15.100:0/2602908056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c04102390 0x7f1c04102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:44.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.841+0000 7f1c0c3fd700 1 -- 172.21.15.100:0/2602908056 >> 172.21.15.100:0/2602908056 conn(0x7f1c04079310 msgr2=0x7f1c04079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:44.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.841+0000 7f1c0c3fd700 1 -- 172.21.15.100:0/2602908056 shutdown_connections 2024-05-10T11:30:44.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.841+0000 7f1c0c3fd700 1 -- 172.21.15.100:0/2602908056 wait complete. 2024-05-10T11:30:44.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.841+0000 7f1c0c3fd700 1 Processor -- start 2024-05-10T11:30:44.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.842+0000 7f1c0c3fd700 1 -- start start 2024-05-10T11:30:44.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.842+0000 7f1c0c3fd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c04102390 0x7f1c0419f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:44.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.842+0000 7f1c0c3fd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c04102d50 0x7f1c0419fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:44.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.842+0000 7f1c0c3fd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c04108400 0x7f1c041998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:44.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.842+0000 7f1c0c3fd700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1c04112400 con 0x7f1c04108400 2024-05-10T11:30:44.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.842+0000 7f1c0c3fd700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1c04112280 con 0x7f1c04102390 2024-05-10T11:30:44.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.842+0000 7f1c0c3fd700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1c04112580 con 0x7f1c04102d50 2024-05-10T11:30:44.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.842+0000 7f1c0a99a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c04108400 0x7f1c041998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:44.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.842+0000 7f1c09998700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c04102d50 0x7f1c0419fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:44.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.842+0000 7f1c0a99a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c04108400 0x7f1c041998c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34342/0 (socket says 172.21.15.100:34342) 2024-05-10T11:30:44.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.842+0000 7f1c0a99a700 1 -- 172.21.15.100:0/4091659618 learned_addr learned my addr 172.21.15.100:0/4091659618 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:30:44.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.842+0000 7f1c0a199700 1 --2- 172.21.15.100:0/4091659618 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c04102390 0x7f1c0419f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:44.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.842+0000 7f1c0a99a700 1 -- 172.21.15.100:0/4091659618 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c04102d50 msgr2=0x7f1c0419fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:44.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.842+0000 7f1c0a99a700 1 --2- 172.21.15.100:0/4091659618 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c04102d50 0x7f1c0419fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:44.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.842+0000 7f1c0a99a700 1 -- 172.21.15.100:0/4091659618 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c04102390 msgr2=0x7f1c0419f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:44.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.842+0000 7f1c0a99a700 1 --2- 172.21.15.100:0/4091659618 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c04102390 0x7f1c0419f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:44.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.842+0000 7f1c0a99a700 1 -- 172.21.15.100:0/4091659618 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c0419a020 con 0x7f1c04108400 2024-05-10T11:30:44.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.842+0000 7f1c0a199700 1 --2- 172.21.15.100:0/4091659618 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c04102390 0x7f1c0419f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:30:44.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.842+0000 7f1c0a99a700 1 --2- 172.21.15.100:0/4091659618 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c04108400 0x7f1c041998c0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc00b780 tx=0x7f1bfc004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:44.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.843+0000 7f1bfb7fe700 1 -- 172.21.15.100:0/4091659618 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc021070 con 0x7f1c04108400 2024-05-10T11:30:44.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.843+0000 7f1c0c3fd700 1 -- 172.21.15.100:0/4091659618 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c0419a2b0 con 0x7f1c04108400 2024-05-10T11:30:44.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.843+0000 7f1bfb7fe700 1 -- 172.21.15.100:0/4091659618 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1bfc007b80 con 0x7f1c04108400 2024-05-10T11:30:44.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.843+0000 7f1bfb7fe700 1 -- 172.21.15.100:0/4091659618 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc0052a0 con 0x7f1c04108400 2024-05-10T11:30:44.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.843+0000 7f1c0c3fd700 1 -- 172.21.15.100:0/4091659618 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c04104310 con 0x7f1c04108400 2024-05-10T11:30:44.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.846+0000 7f1bfb7fe700 1 -- 172.21.15.100:0/4091659618 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f1bfc024070 con 0x7f1c04108400 2024-05-10T11:30:44.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.846+0000 7f1bfb7fe700 1 --2- 172.21.15.100:0/4091659618 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f1bf00b2880 0x7f1bf00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:44.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.846+0000 7f1c0a199700 1 --2- 172.21.15.100:0/4091659618 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f1bf00b2880 0x7f1bf00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:44.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.846+0000 7f1bfb7fe700 1 -- 172.21.15.100:0/4091659618 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f1bfc00f020 con 0x7f1c04108400 2024-05-10T11:30:44.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.847+0000 7f1c16c90b80 1 Processor -- start 2024-05-10T11:30:44.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.847+0000 7f1c0a199700 1 --2- 172.21.15.100:0/4091659618 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f1bf00b2880 0x7f1bf00b4d40 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f1c0419aa00 tx=0x7f1c000073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:44.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.847+0000 7f1c16c90b80 1 -- start start 2024-05-10T11:30:44.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.847+0000 7f1c16c90b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5594875ed4d0 0x5594875ed8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:44.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.847+0000 7f1c16c90b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594875eddf0 0x5594875f16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:44.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.847+0000 7f1c16c90b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594875f1c80 0x5594875f2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:44.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.847+0000 7f1c16c90b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55948753f860 con 0x5594875f1c80 2024-05-10T11:30:44.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.847+0000 7f1c16c90b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5594874de7c0 con 0x5594875ed4d0 2024-05-10T11:30:44.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.847+0000 7f1c16c90b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55948753b820 con 0x5594875eddf0 2024-05-10T11:30:44.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.847+0000 7f1c0a99a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594875f1c80 0x5594875f2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:44.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.847+0000 7f1c09998700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594875eddf0 0x5594875f16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:44.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.847+0000 7f1c0a99a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594875f1c80 0x5594875f2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34344/0 (socket says 172.21.15.100:34344) 2024-05-10T11:30:44.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.847+0000 7f1c0a99a700 1 -- 172.21.15.100:0/3640021189 learned_addr learned my addr 172.21.15.100:0/3640021189 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:30:44.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.848+0000 7f1c0a199700 1 --2- 172.21.15.100:0/3640021189 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5594875ed4d0 0x5594875ed8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:44.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.848+0000 7f1c0a99a700 1 -- 172.21.15.100:0/3640021189 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594875eddf0 msgr2=0x5594875f16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:44.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.848+0000 7f1c0a99a700 1 --2- 172.21.15.100:0/3640021189 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594875eddf0 0x5594875f16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:44.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.848+0000 7f1c0a99a700 1 -- 172.21.15.100:0/3640021189 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5594875ed4d0 msgr2=0x5594875ed8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:44.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.848+0000 7f1c0a99a700 1 --2- 172.21.15.100:0/3640021189 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5594875ed4d0 0x5594875ed8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:44.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.848+0000 7f1c0a99a700 1 -- 172.21.15.100:0/3640021189 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55948748c8d0 con 0x5594875f1c80 2024-05-10T11:30:44.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.848+0000 7f1c09998700 1 --2- 172.21.15.100:0/3640021189 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594875eddf0 0x5594875f16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:30:44.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.848+0000 7f1c0a199700 1 --2- 172.21.15.100:0/3640021189 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5594875ed4d0 0x5594875ed8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:30:44.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.848+0000 7f1c0a99a700 1 --2- 172.21.15.100:0/3640021189 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594875f1c80 0x5594875f2100 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc0a8000 tx=0x7f1bfc009c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18edf222f7b04257 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:44.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.848+0000 7f1bf8d70700 1 -- 172.21.15.100:0/3640021189 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc0af140 con 0x5594875f1c80 2024-05-10T11:30:44.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.848+0000 7f1bf8d70700 1 -- 172.21.15.100:0/3640021189 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1bfc0b3b60 con 0x5594875f1c80 2024-05-10T11:30:44.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.848+0000 7f1bf8d70700 1 -- 172.21.15.100:0/3640021189 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc0b36d0 con 0x5594875f1c80 2024-05-10T11:30:44.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.849+0000 7f1c16c90b80 1 -- 172.21.15.100:0/3640021189 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594875f1c80 msgr2=0x5594875f2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:44.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.849+0000 7f1c16c90b80 1 --2- 172.21.15.100:0/3640021189 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594875f1c80 0x5594875f2100 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc0a8000 tx=0x7f1bfc009c50 comp rx=0 tx=0).stop 2024-05-10T11:30:44.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.849+0000 7f1c16c90b80 1 -- 172.21.15.100:0/3640021189 shutdown_connections 2024-05-10T11:30:44.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.849+0000 7f1c16c90b80 1 --2- 172.21.15.100:0/3640021189 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594875f1c80 0x5594875f2100 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:44.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.849+0000 7f1c16c90b80 1 --2- 172.21.15.100:0/3640021189 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5594875ed4d0 0x5594875ed8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:44.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.849+0000 7f1c16c90b80 1 --2- 172.21.15.100:0/3640021189 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594875eddf0 0x5594875f16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:44.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.849+0000 7f1c16c90b80 1 -- 172.21.15.100:0/3640021189 >> 172.21.15.100:0/3640021189 conn(0x559487519100 msgr2=0x5594875e7c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:44.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.849+0000 7f1c16c90b80 1 -- 172.21.15.100:0/3640021189 shutdown_connections 2024-05-10T11:30:44.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.849+0000 7f1c16c90b80 1 -- 172.21.15.100:0/3640021189 wait complete. 2024-05-10T11:30:44.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.850+0000 7f1c16c90b80 1 -- start start 2024-05-10T11:30:44.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.850+0000 7f1c16c90b80 1 Processor -- start 2024-05-10T11:30:44.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.850+0000 7f1c16c90b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594875ed4d0 0x5594875f9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:44.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.850+0000 7f1c16c90b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594875eddf0 0x5594875f6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:44.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.850+0000 7f1c16c90b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5594875f1c80 0x5594875f6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:44.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.850+0000 7f1c16c90b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5594875f9c50 con 0x5594875ed4d0 2024-05-10T11:30:44.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.850+0000 7f1c0a199700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594875ed4d0 0x5594875f9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:44.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.850+0000 7f1c0a199700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594875ed4d0 0x5594875f9150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34354/0 (socket says 172.21.15.100:34354) 2024-05-10T11:30:44.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.850+0000 7f1c16c90b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5594874a5c30 con 0x5594875f1c80 2024-05-10T11:30:44.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.850+0000 7f1c0a199700 1 -- 172.21.15.100:0/2622953246 learned_addr learned my addr 172.21.15.100:0/2622953246 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:30:44.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.850+0000 7f1c16c90b80 1 -- 172.21.15.100:0/2622953246 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5594875f9dd0 con 0x5594875eddf0 2024-05-10T11:30:44.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.850+0000 7f1c0a99a700 1 --2- 172.21.15.100:0/2622953246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5594875f1c80 0x5594875f6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:44.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.850+0000 7f1c16c90b80 10 client.0 _ll_register_callbacks cb 0x5594874d7920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:30:44.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.850+0000 7f1c09998700 1 --2- 172.21.15.100:0/2622953246 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594875eddf0 0x5594875f6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:44.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.851+0000 7f1c0a199700 1 -- 172.21.15.100:0/2622953246 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594875eddf0 msgr2=0x5594875f6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:44.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.851+0000 7f1c0a199700 1 --2- 172.21.15.100:0/2622953246 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594875eddf0 0x5594875f6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:44.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.851+0000 7f1c0a199700 1 -- 172.21.15.100:0/2622953246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5594875f1c80 msgr2=0x5594875f6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:44.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.851+0000 7f1c0a199700 1 --2- 172.21.15.100:0/2622953246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5594875f1c80 0x5594875f6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:44.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.851+0000 7f1c0a199700 1 -- 172.21.15.100:0/2622953246 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594874db690 con 0x5594875ed4d0 2024-05-10T11:30:44.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.851+0000 7f1c0a199700 1 --2- 172.21.15.100:0/2622953246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594875ed4d0 0x5594875f9150 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f1c00011840 tx=0x7f1c000047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ab7694d93e98325 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:44.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.851+0000 7f1bedffb700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:30:44.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.851+0000 7f1bedffb700 1 -- 172.21.15.100:0/2622953246 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0001f840 con 0x5594875ed4d0 2024-05-10T11:30:44.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.851+0000 7f1bedffb700 1 -- 172.21.15.100:0/2622953246 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f1c0001f9e0 con 0x5594875ed4d0 2024-05-10T11:30:44.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.851+0000 7f1c16c90b80 1 -- 172.21.15.100:0/2622953246 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bfc0b3b60 con 0x5594875ed4d0 2024-05-10T11:30:44.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.851+0000 7f1bedffb700 1 -- 172.21.15.100:0/2622953246 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1c0001fbe0 con 0x5594875ed4d0 2024-05-10T11:30:44.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.851+0000 7f1c16c90b80 10 client.17895 fetch_fsmap learned FSMap version 33 2024-05-10T11:30:44.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.851+0000 7f1c16c90b80 1 -- 172.21.15.100:0/2622953246 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5594876044d0 con 0x5594875ed4d0 2024-05-10T11:30:44.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.851+0000 7f1bedffb700 1 -- 172.21.15.100:0/2622953246 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1bfc0b3b60 con 0x5594875ed4d0 2024-05-10T11:30:44.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.852+0000 7f1bedffb700 1 -- 172.21.15.100:0/2622953246 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f1c00005320 con 0x5594875ed4d0 2024-05-10T11:30:44.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.852+0000 7f1c16c90b80 10 client.17895 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T11:30:44.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.852+0000 7f1c16c90b80 10 client.17895 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-10T11:30:44.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.852+0000 7f1c16c90b80 20 client.17895 populate_metadata read hostname 'smithi100' 2024-05-10T11:30:44.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.852+0000 7f1c16c90b80 1 --2- 172.21.15.100:0/2622953246 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x559487606b10 0x559487608f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:44.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.852+0000 7f1c16c90b80 4 client.17895 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:30:44.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.852+0000 7f1c16c90b80 1 -- 172.21.15.100:0/2622953246 --> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594875b9650 con 0x559487606b10 2024-05-10T11:30:44.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.852+0000 7f1c09998700 1 --2- 172.21.15.100:0/2622953246 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x559487606b10 0x559487608f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:44.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.852+0000 7f1bedffb700 10 client.17895 ms_handle_connect on v2:172.21.15.188:6836/1943342620 2024-05-10T11:30:44.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.852+0000 7f1c09998700 1 --2- 172.21.15.100:0/2622953246 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x559487606b10 0x559487608f00 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7aa6e2d85a542223 server_cookie=26d4e33a29d3987 in_seq=0 out_seq=0 2024-05-10T11:30:44.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.853+0000 7f1bedffb700 1 -- 172.21.15.100:0/2622953246 <== mds.2 v2:172.21.15.188:6836/1943342620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5594875b9650 con 0x559487606b10 2024-05-10T11:30:44.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.853+0000 7f1bedffb700 10 client.17895 handle_command_reply: tid=0 2024-05-10T11:30:44.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.853+0000 7f1c16c90b80 1 client.17895 shutdown 2024-05-10T11:30:44.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.853+0000 7f1c16c90b80 1 -- 172.21.15.100:0/2622953246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594875ed4d0 msgr2=0x5594875f9150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:44.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.853+0000 7f1c16c90b80 1 --2- 172.21.15.100:0/2622953246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594875ed4d0 0x5594875f9150 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f1c00011840 tx=0x7f1c000047c0 comp rx=0 tx=0).stop 2024-05-10T11:30:44.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.853+0000 7f1c16c90b80 1 -- 172.21.15.100:0/2622953246 shutdown_connections 2024-05-10T11:30:44.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.853+0000 7f1c16c90b80 1 --2- 172.21.15.100:0/2622953246 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x559487606b10 0x559487608f00 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:44.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.853+0000 7f1c16c90b80 1 --2- 172.21.15.100:0/2622953246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5594875f1c80 0x5594875f6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:44.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.853+0000 7f1c16c90b80 1 --2- 172.21.15.100:0/2622953246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594875ed4d0 0x5594875f9150 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:44.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.853+0000 7f1c16c90b80 1 --2- 172.21.15.100:0/2622953246 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594875eddf0 0x5594875f6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:44.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.853+0000 7f1c16c90b80 1 -- 172.21.15.100:0/2622953246 >> 172.21.15.100:0/2622953246 conn(0x559487519100 msgr2=0x5594875e7c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:44.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.853+0000 7f1bedffb700 0 client.17895 ms_handle_reset on v2:172.21.15.188:6836/1943342620 2024-05-10T11:30:44.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.853+0000 7f1c16c90b80 1 -- 172.21.15.100:0/2622953246 shutdown_connections 2024-05-10T11:30:44.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.853+0000 7f1c16c90b80 1 -- 172.21.15.100:0/2622953246 wait complete. 2024-05-10T11:30:44.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.853+0000 7f1c16c90b80 20 client.17895 trim_cache size 0 max 16384 2024-05-10T11:30:44.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.871+0000 7f1c16c90b80 1 Processor -- start 2024-05-10T11:30:44.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.871+0000 7f1c16c90b80 1 -- start start 2024-05-10T11:30:44.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.871+0000 7f1c16c90b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594875ed4d0 0x559487602ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:44.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.871+0000 7f1c16c90b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5594875eddf0 0x5594875e25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:44.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.871+0000 7f1c16c90b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594875f1c80 0x55948760e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:44.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.871+0000 7f1c16c90b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1c041b3ac0 con 0x5594875ed4d0 2024-05-10T11:30:44.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.872+0000 7f1c16c90b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1c041b1350 con 0x5594875eddf0 2024-05-10T11:30:44.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.872+0000 7f1c16c90b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1c041af1a0 con 0x5594875f1c80 2024-05-10T11:30:44.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.872+0000 7f1c0a199700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594875ed4d0 0x559487602ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:44.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.872+0000 7f1c0a199700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594875ed4d0 0x559487602ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34356/0 (socket says 172.21.15.100:34356) 2024-05-10T11:30:44.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.872+0000 7f1c0a199700 1 -- 172.21.15.100:0/2699603833 learned_addr learned my addr 172.21.15.100:0/2699603833 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:30:44.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.872+0000 7f1c0a99a700 1 --2- 172.21.15.100:0/2699603833 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594875f1c80 0x55948760e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:44.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.872+0000 7f1c09998700 1 --2- 172.21.15.100:0/2699603833 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5594875eddf0 0x5594875e25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:44.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.872+0000 7f1c0a199700 1 -- 172.21.15.100:0/2699603833 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594875f1c80 msgr2=0x55948760e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:44.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.872+0000 7f1c0a199700 1 --2- 172.21.15.100:0/2699603833 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594875f1c80 0x55948760e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:44.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.872+0000 7f1c0a199700 1 -- 172.21.15.100:0/2699603833 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5594875eddf0 msgr2=0x5594875e25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:44.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.872+0000 7f1c0a199700 1 --2- 172.21.15.100:0/2699603833 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5594875eddf0 0x5594875e25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:44.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.872+0000 7f1c0a199700 1 -- 172.21.15.100:0/2699603833 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594875e02d0 con 0x5594875ed4d0 2024-05-10T11:30:44.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.872+0000 7f1c0a199700 1 --2- 172.21.15.100:0/2699603833 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594875ed4d0 0x559487602ca0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f1c000075e0 tx=0x7f1c00010a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4bfa956731190cf server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:44.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.872+0000 7f1beffff700 1 -- 172.21.15.100:0/2699603833 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c00039030 con 0x5594875ed4d0 2024-05-10T11:30:44.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.872+0000 7f1beffff700 1 -- 172.21.15.100:0/2699603833 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1c00023070 con 0x5594875ed4d0 2024-05-10T11:30:44.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.872+0000 7f1beffff700 1 -- 172.21.15.100:0/2699603833 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c00004c50 con 0x5594875ed4d0 2024-05-10T11:30:44.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.873+0000 7f1c16c90b80 1 -- 172.21.15.100:0/2699603833 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594875ed4d0 msgr2=0x559487602ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:44.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.873+0000 7f1c16c90b80 1 --2- 172.21.15.100:0/2699603833 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594875ed4d0 0x559487602ca0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f1c000075e0 tx=0x7f1c00010a40 comp rx=0 tx=0).stop 2024-05-10T11:30:44.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.873+0000 7f1c16c90b80 1 -- 172.21.15.100:0/2699603833 shutdown_connections 2024-05-10T11:30:44.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.873+0000 7f1c16c90b80 1 --2- 172.21.15.100:0/2699603833 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594875f1c80 0x55948760e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:44.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.873+0000 7f1c16c90b80 1 --2- 172.21.15.100:0/2699603833 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594875ed4d0 0x559487602ca0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:44.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.873+0000 7f1c16c90b80 1 --2- 172.21.15.100:0/2699603833 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5594875eddf0 0x5594875e25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:44.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.873+0000 7f1c16c90b80 1 -- 172.21.15.100:0/2699603833 >> 172.21.15.100:0/2699603833 conn(0x559487519100 msgr2=0x5594875e1e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:44.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.873+0000 7f1c16c90b80 1 -- 172.21.15.100:0/2699603833 shutdown_connections 2024-05-10T11:30:44.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.873+0000 7f1c16c90b80 1 -- 172.21.15.100:0/2699603833 wait complete. 2024-05-10T11:30:44.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.873+0000 7f1c16c90b80 1 -- start start 2024-05-10T11:30:44.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.873+0000 7f1c16c90b80 1 Processor -- start 2024-05-10T11:30:44.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.874+0000 7f1c16c90b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5594875ed4d0 0x5594875fc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:44.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.874+0000 7f1c16c90b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594875eddf0 0x5594875fccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:44.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.874+0000 7f1c16c90b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594875f1c80 0x5594876683b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:44.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.874+0000 7f1c16c90b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1c041af810 con 0x5594875eddf0 2024-05-10T11:30:44.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.874+0000 7f1c16c90b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1c041b0c80 con 0x5594875ed4d0 2024-05-10T11:30:44.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.874+0000 7f1c16c90b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1c041afee0 con 0x5594875f1c80 2024-05-10T11:30:44.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.874+0000 7f1c16c90b80 10 client.0 _ll_register_callbacks cb 0x5594874d7920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:30:44.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.874+0000 7f1c0a99a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594875f1c80 0x5594876683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:44.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.874+0000 7f1c09998700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594875eddf0 0x5594875fccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:44.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.874+0000 7f1c0a99a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594875f1c80 0x5594876683b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:42456/0 (socket says 172.21.15.100:42456) 2024-05-10T11:30:44.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.874+0000 7f1c0a99a700 1 -- 172.21.15.100:0/4269345876 learned_addr learned my addr 172.21.15.100:0/4269345876 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:30:44.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.874+0000 7f1c0a199700 1 --2- 172.21.15.100:0/4269345876 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5594875ed4d0 0x5594875fc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:44.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.874+0000 7f1c0a99a700 1 -- 172.21.15.100:0/4269345876 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5594875ed4d0 msgr2=0x5594875fc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:44.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.874+0000 7f1c0a99a700 1 --2- 172.21.15.100:0/4269345876 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5594875ed4d0 0x5594875fc7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:44.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.874+0000 7f1c0a99a700 1 -- 172.21.15.100:0/4269345876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594875eddf0 msgr2=0x5594875fccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:44.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.874+0000 7f1c0a99a700 1 --2- 172.21.15.100:0/4269345876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594875eddf0 0x5594875fccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:44.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.874+0000 7f1c0a99a700 1 -- 172.21.15.100:0/4269345876 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594875e4810 con 0x5594875f1c80 2024-05-10T11:30:44.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.875+0000 7f1c0a99a700 1 --2- 172.21.15.100:0/4269345876 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594875f1c80 0x5594876683b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc0a6e10 tx=0x7f1bfc0099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=860ac4fb41ebb697 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:44.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.875+0000 7f1bedffb700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:30:44.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.875+0000 7f1bedffb700 1 -- 172.21.15.100:0/4269345876 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc06e070 con 0x5594875f1c80 2024-05-10T11:30:44.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.875+0000 7f1bedffb700 1 -- 172.21.15.100:0/4269345876 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f1bfc0a7380 con 0x5594875f1c80 2024-05-10T11:30:44.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.875+0000 7f1c16c90b80 1 -- 172.21.15.100:0/4269345876 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c00023070 con 0x5594875f1c80 2024-05-10T11:30:44.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.875+0000 7f1bedffb700 1 -- 172.21.15.100:0/4269345876 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1bfc0af140 con 0x5594875f1c80 2024-05-10T11:30:44.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.875+0000 7f1c16c90b80 10 client.27352 fetch_fsmap learned FSMap version 33 2024-05-10T11:30:44.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.875+0000 7f1c16c90b80 1 -- 172.21.15.100:0/4269345876 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559487670de0 con 0x5594875f1c80 2024-05-10T11:30:44.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.875+0000 7f1bedffb700 1 -- 172.21.15.100:0/4269345876 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1c00023070 con 0x5594875f1c80 2024-05-10T11:30:44.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.875+0000 7f1bedffb700 1 -- 172.21.15.100:0/4269345876 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f1bfc011070 con 0x5594875f1c80 2024-05-10T11:30:44.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.875+0000 7f1c16c90b80 10 client.27352 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T11:30:44.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.875+0000 7f1c16c90b80 10 client.27352 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-10T11:30:44.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.875+0000 7f1c16c90b80 20 client.27352 populate_metadata read hostname 'smithi100' 2024-05-10T11:30:44.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.875+0000 7f1c16c90b80 1 --2- 172.21.15.100:0/4269345876 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x559487671350 0x559487673770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:44.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.876+0000 7f1c16c90b80 4 client.27352 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:30:44.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.876+0000 7f1c16c90b80 1 -- 172.21.15.100:0/4269345876 --> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55948754d240 con 0x559487671350 2024-05-10T11:30:44.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.876+0000 7f1c0a199700 1 --2- 172.21.15.100:0/4269345876 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x559487671350 0x559487673770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:44.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.876+0000 7f1bedffb700 10 client.27352 ms_handle_connect on v2:172.21.15.188:6836/1943342620 2024-05-10T11:30:44.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.876+0000 7f1c0a199700 1 --2- 172.21.15.100:0/4269345876 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x559487671350 0x559487673770 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=112db0cd58ae972d server_cookie=e833b0409225b476 in_seq=0 out_seq=0 2024-05-10T11:30:44.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.876+0000 7f1bedffb700 1 -- 172.21.15.100:0/4269345876 <== mds.2 v2:172.21.15.188:6836/1943342620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55948754d240 con 0x559487671350 2024-05-10T11:30:44.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.876+0000 7f1bedffb700 10 client.27352 handle_command_reply: tid=0 2024-05-10T11:30:44.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.876+0000 7f1c16c90b80 1 client.27352 shutdown 2024-05-10T11:30:44.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.876+0000 7f1c16c90b80 1 -- 172.21.15.100:0/4269345876 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594875f1c80 msgr2=0x5594876683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:44.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.876+0000 7f1c16c90b80 1 --2- 172.21.15.100:0/4269345876 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594875f1c80 0x5594876683b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc0a6e10 tx=0x7f1bfc0099e0 comp rx=0 tx=0).stop 2024-05-10T11:30:44.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.876+0000 7f1c16c90b80 1 -- 172.21.15.100:0/4269345876 shutdown_connections 2024-05-10T11:30:44.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.876+0000 7f1c16c90b80 1 --2- 172.21.15.100:0/4269345876 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x559487671350 0x559487673770 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:44.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.876+0000 7f1c16c90b80 1 --2- 172.21.15.100:0/4269345876 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594875f1c80 0x5594876683b0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:44.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.876+0000 7f1c16c90b80 1 --2- 172.21.15.100:0/4269345876 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5594875ed4d0 0x5594875fc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:44.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.876+0000 7f1c16c90b80 1 --2- 172.21.15.100:0/4269345876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594875eddf0 0x5594875fccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:44.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.876+0000 7f1bedffb700 0 client.27352 ms_handle_reset on v2:172.21.15.188:6836/1943342620 2024-05-10T11:30:44.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.876+0000 7f1c16c90b80 1 -- 172.21.15.100:0/4269345876 >> 172.21.15.100:0/4269345876 conn(0x559487519100 msgr2=0x5594875e2720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:44.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.877+0000 7f1c16c90b80 1 -- 172.21.15.100:0/4269345876 shutdown_connections 2024-05-10T11:30:44.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.877+0000 7f1c16c90b80 1 -- 172.21.15.100:0/4269345876 wait complete. 2024-05-10T11:30:44.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.877+0000 7f1c16c90b80 20 client.27352 trim_cache size 0 max 16384 2024-05-10T11:30:44.881 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:30:44.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.877+0000 7f1c0c3fd700 1 -- 172.21.15.100:0/4091659618 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f1bf00b2880 msgr2=0x7f1bf00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:44.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.877+0000 7f1c0c3fd700 1 --2- 172.21.15.100:0/4091659618 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f1bf00b2880 0x7f1bf00b4d40 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f1c0419aa00 tx=0x7f1c000073d0 comp rx=0 tx=0).stop 2024-05-10T11:30:44.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.877+0000 7f1c0c3fd700 1 -- 172.21.15.100:0/4091659618 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c04108400 msgr2=0x7f1c041998c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:44.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.877+0000 7f1c0c3fd700 1 --2- 172.21.15.100:0/4091659618 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c04108400 0x7f1c041998c0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc00b780 tx=0x7f1bfc004d00 comp rx=0 tx=0).stop 2024-05-10T11:30:44.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.877+0000 7f1c0c3fd700 1 -- 172.21.15.100:0/4091659618 shutdown_connections 2024-05-10T11:30:44.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.877+0000 7f1c0c3fd700 1 --2- 172.21.15.100:0/4091659618 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f1bf00b2880 0x7f1bf00b4d40 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:44.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.877+0000 7f1c0c3fd700 1 --2- 172.21.15.100:0/4091659618 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c04108400 0x7f1c041998c0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:44.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.877+0000 7f1c0c3fd700 1 --2- 172.21.15.100:0/4091659618 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c04102390 0x7f1c0419f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:44.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.877+0000 7f1c0c3fd700 1 --2- 172.21.15.100:0/4091659618 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c04102d50 0x7f1c0419fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:44.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.877+0000 7f1c0c3fd700 1 -- 172.21.15.100:0/4091659618 >> 172.21.15.100:0/4091659618 conn(0x7f1c04079310 msgr2=0x7f1c04106760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:44.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.877+0000 7f1c0c3fd700 1 -- 172.21.15.100:0/4091659618 shutdown_connections 2024-05-10T11:30:44.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:44.877+0000 7f1c0c3fd700 1 -- 172.21.15.100:0/4091659618 wait complete. 2024-05-10T11:30:44.983 INFO:tasks.workunit.client.0.smithi100.stdout: 1 325800 2.72 MB/sec execute 519 sec latency 17.595 ms 2024-05-10T11:30:45.616 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:30:45 smithi100 ceph-mon[30296]: pgmap v792: 129 pgs: 129 active+clean; 128 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 1.3 MiB/s wr, 139 op/s 2024-05-10T11:30:45.643 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-10T11:30:45.907 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:30:45.983 INFO:tasks.workunit.client.0.smithi100.stdout: 1 326344 2.72 MB/sec execute 520 sec latency 15.535 ms 2024-05-10T11:30:46.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:30:45 smithi178 ceph-mon[33359]: pgmap v792: 129 pgs: 129 active+clean; 128 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 1.3 MiB/s wr, 139 op/s 2024-05-10T11:30:46.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:30:45 smithi188 ceph-mon[34075]: pgmap v792: 129 pgs: 129 active+clean; 128 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 1.3 MiB/s wr, 139 op/s 2024-05-10T11:30:46.983 INFO:tasks.workunit.client.0.smithi100.stdout: 1 326677 2.72 MB/sec execute 521 sec latency 15.607 ms 2024-05-10T11:30:47.714 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:30:47 smithi100 ceph-mon[30296]: pgmap v793: 129 pgs: 129 active+clean; 129 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 1.4 MiB/s wr, 154 op/s 2024-05-10T11:30:47.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.735+0000 7f577dd31700 1 -- 172.21.15.100:0/1107540903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5778102d60 msgr2=0x7f577810f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:47.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.735+0000 7f577dd31700 1 --2- 172.21.15.100:0/1107540903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5778102d60 0x7f577810f250 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f577000b0a0 tx=0x7f577000b3b0 comp rx=0 tx=0).stop 2024-05-10T11:30:47.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.735+0000 7f577dd31700 1 -- 172.21.15.100:0/1107540903 shutdown_connections 2024-05-10T11:30:47.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.735+0000 7f577dd31700 1 --2- 172.21.15.100:0/1107540903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5778102d60 0x7f577810f250 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:47.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.735+0000 7f577dd31700 1 --2- 172.21.15.100:0/1107540903 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5778108410 0x7f57781087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:47.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.735+0000 7f577dd31700 1 --2- 172.21.15.100:0/1107540903 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57781023a0 0x7f5778102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:47.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.735+0000 7f577dd31700 1 -- 172.21.15.100:0/1107540903 >> 172.21.15.100:0/1107540903 conn(0x7f5778079350 msgr2=0x7f5778079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:47.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.735+0000 7f577dd31700 1 -- 172.21.15.100:0/1107540903 shutdown_connections 2024-05-10T11:30:47.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.735+0000 7f577dd31700 1 -- 172.21.15.100:0/1107540903 wait complete. 2024-05-10T11:30:47.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.736+0000 7f577dd31700 1 Processor -- start 2024-05-10T11:30:47.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.736+0000 7f577dd31700 1 -- start start 2024-05-10T11:30:47.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.736+0000 7f577dd31700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57781023a0 0x7f577819a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:47.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.736+0000 7f577dd31700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5778102d60 0x7f577819ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:47.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.736+0000 7f577dd31700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5778108410 0x7f57781a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:47.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.736+0000 7f577dd31700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5778112410 con 0x7f5778102d60 2024-05-10T11:30:47.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.736+0000 7f577dd31700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5778112290 con 0x7f5778108410 2024-05-10T11:30:47.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.736+0000 7f577dd31700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5778112590 con 0x7f57781023a0 2024-05-10T11:30:47.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.736+0000 7f5776ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5778102d60 0x7f577819ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:47.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.736+0000 7f57777fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57781023a0 0x7f577819a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:47.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.736+0000 7f5776ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5778102d60 0x7f577819ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43486/0 (socket says 172.21.15.100:43486) 2024-05-10T11:30:47.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.736+0000 7f5776ffd700 1 -- 172.21.15.100:0/3606746129 learned_addr learned my addr 172.21.15.100:0/3606746129 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:30:47.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.736+0000 7f57777fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57781023a0 0x7f577819a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:36102/0 (socket says 172.21.15.100:36102) 2024-05-10T11:30:47.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.736+0000 7f5777fff700 1 --2- 172.21.15.100:0/3606746129 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5778108410 0x7f57781a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:47.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.737+0000 7f57777fe700 1 -- 172.21.15.100:0/3606746129 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5778108410 msgr2=0x7f57781a5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:47.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.737+0000 7f57777fe700 1 --2- 172.21.15.100:0/3606746129 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5778108410 0x7f57781a5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:47.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.737+0000 7f57777fe700 1 -- 172.21.15.100:0/3606746129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5778102d60 msgr2=0x7f577819ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:47.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.737+0000 7f57777fe700 1 --2- 172.21.15.100:0/3606746129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5778102d60 0x7f577819ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:47.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.737+0000 7f57777fe700 1 -- 172.21.15.100:0/3606746129 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57781a5ed0 con 0x7f57781023a0 2024-05-10T11:30:47.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.737+0000 7f5777fff700 1 --2- 172.21.15.100:0/3606746129 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5778108410 0x7f57781a5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:30:47.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.737+0000 7f57777fe700 1 --2- 172.21.15.100:0/3606746129 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57781023a0 0x7f577819a910 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f576800e9f0 tx=0x7f576800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:47.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.737+0000 7f5774ff9700 1 -- 172.21.15.100:0/3606746129 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576800c9c0 con 0x7f57781023a0 2024-05-10T11:30:47.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.737+0000 7f5774ff9700 1 -- 172.21.15.100:0/3606746129 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5768010430 con 0x7f57781023a0 2024-05-10T11:30:47.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.737+0000 7f577dd31700 1 -- 172.21.15.100:0/3606746129 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57781a60a0 con 0x7f57781023a0 2024-05-10T11:30:47.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.737+0000 7f5774ff9700 1 -- 172.21.15.100:0/3606746129 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5768018680 con 0x7f57781023a0 2024-05-10T11:30:47.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.737+0000 7f577dd31700 1 -- 172.21.15.100:0/3606746129 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57781a65a0 con 0x7f57781023a0 2024-05-10T11:30:47.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.740+0000 7f5774ff9700 1 -- 172.21.15.100:0/3606746129 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f576800cb60 con 0x7f57781023a0 2024-05-10T11:30:47.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.741+0000 7f57885c4b80 1 Processor -- start 2024-05-10T11:30:47.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.741+0000 7f57885c4b80 1 -- start start 2024-05-10T11:30:47.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.741+0000 7f5774ff9700 1 --2- 172.21.15.100:0/3606746129 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f57600b2900 0x7f57600b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:47.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.741+0000 7f57885c4b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56499d419f80 0x56499d41a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:47.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.741+0000 7f5777fff700 1 --2- 172.21.15.100:0/3606746129 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f57600b2900 0x7f57600b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:47.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.741+0000 7f57777fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56499d419f80 0x56499d41a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:47.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.741+0000 7f57885c4b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56499d41a8a0 0x56499d414700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:47.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.741+0000 7f57777fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56499d419f80 0x56499d41a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43502/0 (socket says 172.21.15.100:43502) 2024-05-10T11:30:47.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.741+0000 7f57777fe700 1 -- 172.21.15.100:0/1015105424 learned_addr learned my addr 172.21.15.100:0/1015105424 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:30:47.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.741+0000 7f5774ff9700 1 -- 172.21.15.100:0/3606746129 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f57680e25a0 con 0x7f57781023a0 2024-05-10T11:30:47.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.741+0000 7f57885c4b80 1 --2- 172.21.15.100:0/1015105424 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56499d414c90 0x56499d415110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:47.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.741+0000 7f57885c4b80 1 -- 172.21.15.100:0/1015105424 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56499d362860 con 0x56499d419f80 2024-05-10T11:30:47.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.741+0000 7f57885c4b80 1 -- 172.21.15.100:0/1015105424 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56499d3017c0 con 0x56499d41a8a0 2024-05-10T11:30:47.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.741+0000 7f57885c4b80 1 -- 172.21.15.100:0/1015105424 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56499d35e820 con 0x56499d414c90 2024-05-10T11:30:47.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.742+0000 7f5776ffd700 1 --2- 172.21.15.100:0/1015105424 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56499d41a8a0 0x56499d414700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:47.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.742+0000 7f57777fe700 1 -- 172.21.15.100:0/1015105424 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56499d414c90 msgr2=0x56499d415110 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T11:30:47.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.742+0000 7f57777fe700 1 --2- 172.21.15.100:0/1015105424 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56499d414c90 0x56499d415110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:47.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.742+0000 7f57777fe700 1 -- 172.21.15.100:0/1015105424 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56499d41a8a0 msgr2=0x56499d414700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:47.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.742+0000 7f57777fe700 1 --2- 172.21.15.100:0/1015105424 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56499d41a8a0 0x56499d414700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:47.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.742+0000 7f57777fe700 1 -- 172.21.15.100:0/1015105424 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56499d2af8d0 con 0x56499d419f80 2024-05-10T11:30:47.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.742+0000 7f5777fff700 1 --2- 172.21.15.100:0/3606746129 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f57600b2900 0x7f57600b4dc0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f5770009fd0 tx=0x7f577000b720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:47.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.742+0000 7f57777fe700 1 --2- 172.21.15.100:0/1015105424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56499d419f80 0x56499d41a360 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f5768009230 tx=0x7f57680a96b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9a2126a78499412 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:47.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.742+0000 7f575dffb700 1 -- 172.21.15.100:0/1015105424 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57680a9370 con 0x56499d419f80 2024-05-10T11:30:47.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.742+0000 7f575dffb700 1 -- 172.21.15.100:0/1015105424 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f57680a9f60 con 0x56499d419f80 2024-05-10T11:30:47.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.742+0000 7f575dffb700 1 -- 172.21.15.100:0/1015105424 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57680b26c0 con 0x56499d419f80 2024-05-10T11:30:47.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.743+0000 7f57885c4b80 1 -- 172.21.15.100:0/1015105424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56499d419f80 msgr2=0x56499d41a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:47.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.743+0000 7f57885c4b80 1 --2- 172.21.15.100:0/1015105424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56499d419f80 0x56499d41a360 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f5768009230 tx=0x7f57680a96b0 comp rx=0 tx=0).stop 2024-05-10T11:30:47.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.743+0000 7f57885c4b80 1 -- 172.21.15.100:0/1015105424 shutdown_connections 2024-05-10T11:30:47.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.743+0000 7f57885c4b80 1 --2- 172.21.15.100:0/1015105424 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56499d414c90 0x56499d415110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:47.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.743+0000 7f57885c4b80 1 --2- 172.21.15.100:0/1015105424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56499d419f80 0x56499d41a360 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:47.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.743+0000 7f57885c4b80 1 --2- 172.21.15.100:0/1015105424 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56499d41a8a0 0x56499d414700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:47.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.743+0000 7f57885c4b80 1 -- 172.21.15.100:0/1015105424 >> 172.21.15.100:0/1015105424 conn(0x56499d33c100 msgr2=0x56499d40ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:47.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.743+0000 7f57885c4b80 1 -- 172.21.15.100:0/1015105424 shutdown_connections 2024-05-10T11:30:47.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.743+0000 7f57885c4b80 1 -- 172.21.15.100:0/1015105424 wait complete. 2024-05-10T11:30:47.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.744+0000 7f57885c4b80 1 -- start start 2024-05-10T11:30:47.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.744+0000 7f57885c4b80 1 Processor -- start 2024-05-10T11:30:47.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.744+0000 7f57885c4b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56499d414c90 0x56499d420590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:47.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.744+0000 7f57885c4b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56499d419f80 0x56499d41d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:47.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.744+0000 7f57885c4b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56499d41a8a0 0x56499d41de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:47.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.744+0000 7f57885c4b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56499d4212c0 con 0x56499d419f80 2024-05-10T11:30:47.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.744+0000 7f57885c4b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56499d2c8c30 con 0x56499d414c90 2024-05-10T11:30:47.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.744+0000 7f57885c4b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56499d421440 con 0x56499d41a8a0 2024-05-10T11:30:47.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.744+0000 7f57885c4b80 10 client.0 _ll_register_callbacks cb 0x56499d2fa920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:30:47.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.744+0000 7f57777fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56499d414c90 0x56499d420590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:47.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.744+0000 7f5776ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56499d419f80 0x56499d41d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:47.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.744+0000 7f57777fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56499d414c90 0x56499d420590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:59120/0 (socket says 172.21.15.100:59120) 2024-05-10T11:30:47.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.744+0000 7f57777fe700 1 -- 172.21.15.100:0/923018292 learned_addr learned my addr 172.21.15.100:0/923018292 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:30:47.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.744+0000 7f5777fff700 1 --2- 172.21.15.100:0/923018292 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56499d41a8a0 0x56499d41de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:47.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.745+0000 7f57777fe700 1 -- 172.21.15.100:0/923018292 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56499d41a8a0 msgr2=0x56499d41de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:47.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.745+0000 7f57777fe700 1 --2- 172.21.15.100:0/923018292 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56499d41a8a0 0x56499d41de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:47.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.745+0000 7f57777fe700 1 -- 172.21.15.100:0/923018292 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56499d419f80 msgr2=0x56499d41d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:47.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.745+0000 7f57777fe700 1 --2- 172.21.15.100:0/923018292 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56499d419f80 0x56499d41d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:47.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.745+0000 7f57777fe700 1 -- 172.21.15.100:0/923018292 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56499d2fe690 con 0x56499d414c90 2024-05-10T11:30:47.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.745+0000 7f57777fe700 1 --2- 172.21.15.100:0/923018292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56499d414c90 0x56499d420590 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f57680a9560 tx=0x7f576806f630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c19be756be7df00 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:47.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.745+0000 7f5756ffd700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:30:47.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.745+0000 7f5756ffd700 1 -- 172.21.15.100:0/923018292 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57680a9370 con 0x56499d414c90 2024-05-10T11:30:47.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.745+0000 7f5756ffd700 1 -- 172.21.15.100:0/923018292 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f57680e2300 con 0x56499d414c90 2024-05-10T11:30:47.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.745+0000 7f57885c4b80 1 -- 172.21.15.100:0/923018292 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57680a9f60 con 0x56499d414c90 2024-05-10T11:30:47.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.745+0000 7f5756ffd700 1 -- 172.21.15.100:0/923018292 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f57680b26c0 con 0x56499d414c90 2024-05-10T11:30:47.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.745+0000 7f57885c4b80 10 client.27266 fetch_fsmap learned FSMap version 33 2024-05-10T11:30:47.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.745+0000 7f57885c4b80 1 -- 172.21.15.100:0/923018292 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56499d42bbc0 con 0x56499d414c90 2024-05-10T11:30:47.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.745+0000 7f5756ffd700 1 -- 172.21.15.100:0/923018292 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f57680a9f60 con 0x56499d414c90 2024-05-10T11:30:47.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.746+0000 7f5756ffd700 1 -- 172.21.15.100:0/923018292 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f57680b2230 con 0x56499d414c90 2024-05-10T11:30:47.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.746+0000 7f57885c4b80 10 client.27266 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T11:30:47.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.746+0000 7f57885c4b80 10 client.27266 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-10T11:30:47.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.746+0000 7f57885c4b80 20 client.27266 populate_metadata read hostname 'smithi100' 2024-05-10T11:30:47.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.746+0000 7f57885c4b80 1 --2- 172.21.15.100:0/923018292 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x56499d42e090 0x56499d430480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:47.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.746+0000 7f57885c4b80 4 client.27266 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:30:47.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.746+0000 7f57885c4b80 1 -- 172.21.15.100:0/923018292 --> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56499d3dc650 con 0x56499d42e090 2024-05-10T11:30:47.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.746+0000 7f5776ffd700 1 --2- 172.21.15.100:0/923018292 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x56499d42e090 0x56499d430480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:47.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.746+0000 7f5756ffd700 10 client.27266 ms_handle_connect on v2:172.21.15.178:6838/3760985404 2024-05-10T11:30:47.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.746+0000 7f5776ffd700 1 --2- 172.21.15.100:0/923018292 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x56499d42e090 0x56499d430480 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=112efcb302e4f805 server_cookie=ad0c23b73bce472 in_seq=0 out_seq=0 2024-05-10T11:30:47.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.747+0000 7f5756ffd700 1 -- 172.21.15.100:0/923018292 <== mds.1 v2:172.21.15.178:6838/3760985404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56499d3dc650 con 0x56499d42e090 2024-05-10T11:30:47.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.747+0000 7f5756ffd700 10 client.27266 handle_command_reply: tid=0 2024-05-10T11:30:47.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.747+0000 7f57885c4b80 1 client.27266 shutdown 2024-05-10T11:30:47.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.747+0000 7f57885c4b80 1 -- 172.21.15.100:0/923018292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56499d414c90 msgr2=0x56499d420590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:47.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.747+0000 7f57885c4b80 1 --2- 172.21.15.100:0/923018292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56499d414c90 0x56499d420590 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f57680a9560 tx=0x7f576806f630 comp rx=0 tx=0).stop 2024-05-10T11:30:47.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.747+0000 7f57885c4b80 1 -- 172.21.15.100:0/923018292 shutdown_connections 2024-05-10T11:30:47.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.747+0000 7f57885c4b80 1 --2- 172.21.15.100:0/923018292 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x56499d42e090 0x56499d430480 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:47.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.747+0000 7f5756ffd700 0 client.27266 ms_handle_reset on v2:172.21.15.178:6838/3760985404 2024-05-10T11:30:47.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.747+0000 7f57885c4b80 1 --2- 172.21.15.100:0/923018292 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56499d41a8a0 0x56499d41de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:47.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.747+0000 7f57885c4b80 1 --2- 172.21.15.100:0/923018292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56499d414c90 0x56499d420590 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:47.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.747+0000 7f57885c4b80 1 --2- 172.21.15.100:0/923018292 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56499d419f80 0x56499d41d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:47.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.747+0000 7f57885c4b80 1 -- 172.21.15.100:0/923018292 >> 172.21.15.100:0/923018292 conn(0x56499d33c100 msgr2=0x56499d405720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:47.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.747+0000 7f57885c4b80 1 -- 172.21.15.100:0/923018292 shutdown_connections 2024-05-10T11:30:47.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.747+0000 7f57885c4b80 1 -- 172.21.15.100:0/923018292 wait complete. 2024-05-10T11:30:47.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.747+0000 7f57885c4b80 20 client.27266 trim_cache size 0 max 16384 2024-05-10T11:30:47.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.766+0000 7f57885c4b80 1 Processor -- start 2024-05-10T11:30:47.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.766+0000 7f57885c4b80 1 -- start start 2024-05-10T11:30:47.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.766+0000 7f57885c4b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56499d414c90 0x56499d41b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:47.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.766+0000 7f57885c4b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56499d419f80 0x56499d4334a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:47.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.766+0000 7f57885c4b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56499d41a8a0 0x56499d4339e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:47.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.766+0000 7f57885c4b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f57781af750 con 0x56499d41a8a0 2024-05-10T11:30:47.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.766+0000 7f57885c4b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f57781acfe0 con 0x56499d414c90 2024-05-10T11:30:47.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.766+0000 7f57885c4b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f57781aae50 con 0x56499d419f80 2024-05-10T11:30:47.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.766+0000 7f5777fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56499d41a8a0 0x56499d4339e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:47.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.766+0000 7f5776ffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56499d419f80 0x56499d4334a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:47.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.766+0000 7f5777fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56499d41a8a0 0x56499d4339e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43518/0 (socket says 172.21.15.100:43518) 2024-05-10T11:30:47.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.766+0000 7f5777fff700 1 -- 172.21.15.100:0/2925980921 learned_addr learned my addr 172.21.15.100:0/2925980921 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:30:47.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.766+0000 7f57777fe700 1 --2- 172.21.15.100:0/2925980921 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56499d414c90 0x56499d41b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:47.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.766+0000 7f5776ffd700 1 -- 172.21.15.100:0/2925980921 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56499d414c90 msgr2=0x56499d41b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:47.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.766+0000 7f5776ffd700 1 --2- 172.21.15.100:0/2925980921 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56499d414c90 0x56499d41b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:47.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.766+0000 7f5776ffd700 1 -- 172.21.15.100:0/2925980921 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56499d41a8a0 msgr2=0x56499d4339e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:47.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.766+0000 7f5776ffd700 1 --2- 172.21.15.100:0/2925980921 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56499d41a8a0 0x56499d4339e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:47.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.766+0000 7f5776ffd700 1 -- 172.21.15.100:0/2925980921 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56499d433f20 con 0x56499d419f80 2024-05-10T11:30:47.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.766+0000 7f57777fe700 1 --2- 172.21.15.100:0/2925980921 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56499d414c90 0x56499d41b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:30:47.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.767+0000 7f5776ffd700 1 --2- 172.21.15.100:0/2925980921 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56499d419f80 0x56499d4334a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x56499d41acc0 tx=0x7f576c009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81de7a11e2c88e50 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:47.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.767+0000 7f575d7fa700 1 -- 172.21.15.100:0/2925980921 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576c004830 con 0x56499d419f80 2024-05-10T11:30:47.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.767+0000 7f575d7fa700 1 -- 172.21.15.100:0/2925980921 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f576c005020 con 0x56499d419f80 2024-05-10T11:30:47.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.767+0000 7f575d7fa700 1 -- 172.21.15.100:0/2925980921 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576c004e60 con 0x56499d419f80 2024-05-10T11:30:47.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.768+0000 7f57885c4b80 1 -- 172.21.15.100:0/2925980921 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56499d419f80 msgr2=0x56499d4334a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:47.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.768+0000 7f57885c4b80 1 --2- 172.21.15.100:0/2925980921 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56499d419f80 0x56499d4334a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x56499d41acc0 tx=0x7f576c009260 comp rx=0 tx=0).stop 2024-05-10T11:30:47.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.768+0000 7f57885c4b80 1 -- 172.21.15.100:0/2925980921 shutdown_connections 2024-05-10T11:30:47.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.768+0000 7f57885c4b80 1 --2- 172.21.15.100:0/2925980921 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56499d41a8a0 0x56499d4339e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:47.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.768+0000 7f57885c4b80 1 --2- 172.21.15.100:0/2925980921 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56499d414c90 0x56499d41b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:47.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.768+0000 7f57885c4b80 1 --2- 172.21.15.100:0/2925980921 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56499d419f80 0x56499d4334a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:47.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.768+0000 7f57885c4b80 1 -- 172.21.15.100:0/2925980921 >> 172.21.15.100:0/2925980921 conn(0x56499d33c100 msgr2=0x56499d41acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:47.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.768+0000 7f57885c4b80 1 -- 172.21.15.100:0/2925980921 shutdown_connections 2024-05-10T11:30:47.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.768+0000 7f57885c4b80 1 -- 172.21.15.100:0/2925980921 wait complete. 2024-05-10T11:30:47.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.768+0000 7f57885c4b80 1 -- start start 2024-05-10T11:30:47.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.768+0000 7f57885c4b80 1 Processor -- start 2024-05-10T11:30:47.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.769+0000 7f57885c4b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56499d414c90 0x56499d4907c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:47.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.769+0000 7f57885c4b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56499d419f80 0x56499d490d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:47.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.769+0000 7f57885c4b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56499d41a8a0 0x56499d432e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:47.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.769+0000 7f57885c4b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f57781ab4a0 con 0x56499d41a8a0 2024-05-10T11:30:47.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.769+0000 7f57885c4b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f57781ac910 con 0x56499d414c90 2024-05-10T11:30:47.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.769+0000 7f57885c4b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f57781abb70 con 0x56499d419f80 2024-05-10T11:30:47.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.769+0000 7f57885c4b80 10 client.0 _ll_register_callbacks cb 0x56499d2fa920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:30:47.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.769+0000 7f57777fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56499d414c90 0x56499d4907c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:47.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.769+0000 7f5777fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56499d41a8a0 0x56499d432e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:47.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.769+0000 7f57777fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56499d414c90 0x56499d4907c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:59134/0 (socket says 172.21.15.100:59134) 2024-05-10T11:30:47.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.769+0000 7f57777fe700 1 -- 172.21.15.100:0/4279230209 learned_addr learned my addr 172.21.15.100:0/4279230209 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:30:47.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.769+0000 7f5776ffd700 1 --2- 172.21.15.100:0/4279230209 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56499d419f80 0x56499d490d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:47.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.769+0000 7f57777fe700 1 -- 172.21.15.100:0/4279230209 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56499d419f80 msgr2=0x56499d490d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:47.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.769+0000 7f57777fe700 1 --2- 172.21.15.100:0/4279230209 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56499d419f80 0x56499d490d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:47.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.769+0000 7f57777fe700 1 -- 172.21.15.100:0/4279230209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56499d41a8a0 msgr2=0x56499d432e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:47.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.769+0000 7f57777fe700 1 --2- 172.21.15.100:0/4279230209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56499d41a8a0 0x56499d432e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:47.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.769+0000 7f57777fe700 1 -- 172.21.15.100:0/4279230209 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56499d433500 con 0x56499d414c90 2024-05-10T11:30:47.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.769+0000 7f5776ffd700 1 --2- 172.21.15.100:0/4279230209 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56499d419f80 0x56499d490d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:30:47.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.769+0000 7f5777fff700 1 --2- 172.21.15.100:0/4279230209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56499d41a8a0 0x56499d432e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:30:47.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.770+0000 7f57777fe700 1 --2- 172.21.15.100:0/4279230209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56499d414c90 0x56499d4907c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5768009230 tx=0x7f5768024000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e69b60482b326b7e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:47.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.770+0000 7f5756ffd700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:30:47.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.770+0000 7f5756ffd700 1 -- 172.21.15.100:0/4279230209 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57680a9370 con 0x56499d414c90 2024-05-10T11:30:47.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.770+0000 7f5756ffd700 1 -- 172.21.15.100:0/4279230209 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f57680a9cd0 con 0x56499d414c90 2024-05-10T11:30:47.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.770+0000 7f57885c4b80 1 -- 172.21.15.100:0/4279230209 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f576c005020 con 0x56499d414c90 2024-05-10T11:30:47.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.770+0000 7f5756ffd700 1 -- 172.21.15.100:0/4279230209 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f57680a9f60 con 0x56499d414c90 2024-05-10T11:30:47.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.770+0000 7f57885c4b80 10 client.27272 fetch_fsmap learned FSMap version 33 2024-05-10T11:30:47.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.770+0000 7f57885c4b80 1 -- 172.21.15.100:0/4279230209 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56499d498110 con 0x56499d414c90 2024-05-10T11:30:47.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.770+0000 7f5756ffd700 1 -- 172.21.15.100:0/4279230209 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f576c005020 con 0x56499d414c90 2024-05-10T11:30:47.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.770+0000 7f5756ffd700 1 -- 172.21.15.100:0/4279230209 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f576806fce0 con 0x56499d414c90 2024-05-10T11:30:47.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.770+0000 7f57885c4b80 10 client.27272 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T11:30:47.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.770+0000 7f57885c4b80 10 client.27272 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-10T11:30:47.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.770+0000 7f57885c4b80 20 client.27272 populate_metadata read hostname 'smithi100' 2024-05-10T11:30:47.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.770+0000 7f57885c4b80 1 --2- 172.21.15.100:0/4279230209 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x56499d42e090 0x56499d49a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:47.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.770+0000 7f57885c4b80 4 client.27272 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:30:47.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.770+0000 7f57885c4b80 1 -- 172.21.15.100:0/4279230209 --> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56499d370240 con 0x56499d42e090 2024-05-10T11:30:47.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.771+0000 7f5776ffd700 1 --2- 172.21.15.100:0/4279230209 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x56499d42e090 0x56499d49a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:47.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.771+0000 7f5756ffd700 10 client.27272 ms_handle_connect on v2:172.21.15.178:6838/3760985404 2024-05-10T11:30:47.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.771+0000 7f5776ffd700 1 --2- 172.21.15.100:0/4279230209 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x56499d42e090 0x56499d49a610 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=56bfcecc3a45cc32 server_cookie=8e850b94c764f5ce in_seq=0 out_seq=0 2024-05-10T11:30:47.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.771+0000 7f5756ffd700 1 -- 172.21.15.100:0/4279230209 <== mds.1 v2:172.21.15.178:6838/3760985404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56499d370240 con 0x56499d42e090 2024-05-10T11:30:47.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.771+0000 7f5756ffd700 10 client.27272 handle_command_reply: tid=0 2024-05-10T11:30:47.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.771+0000 7f57885c4b80 1 client.27272 shutdown 2024-05-10T11:30:47.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.772+0000 7f57885c4b80 1 -- 172.21.15.100:0/4279230209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56499d414c90 msgr2=0x56499d4907c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:47.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.772+0000 7f57885c4b80 1 --2- 172.21.15.100:0/4279230209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56499d414c90 0x56499d4907c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5768009230 tx=0x7f5768024000 comp rx=0 tx=0).stop 2024-05-10T11:30:47.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.772+0000 7f57885c4b80 1 -- 172.21.15.100:0/4279230209 shutdown_connections 2024-05-10T11:30:47.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.772+0000 7f57885c4b80 1 --2- 172.21.15.100:0/4279230209 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x56499d42e090 0x56499d49a610 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:47.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.772+0000 7f57885c4b80 1 --2- 172.21.15.100:0/4279230209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56499d41a8a0 0x56499d432e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:47.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.772+0000 7f57885c4b80 1 --2- 172.21.15.100:0/4279230209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56499d414c90 0x56499d4907c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:47.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.772+0000 7f57885c4b80 1 --2- 172.21.15.100:0/4279230209 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56499d419f80 0x56499d490d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:47.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.772+0000 7f57885c4b80 1 -- 172.21.15.100:0/4279230209 >> 172.21.15.100:0/4279230209 conn(0x56499d33c100 msgr2=0x56499d41b400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:47.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.772+0000 7f5756ffd700 0 client.27272 ms_handle_reset on v2:172.21.15.178:6838/3760985404 2024-05-10T11:30:47.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.772+0000 7f57885c4b80 1 -- 172.21.15.100:0/4279230209 shutdown_connections 2024-05-10T11:30:47.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.772+0000 7f57885c4b80 1 -- 172.21.15.100:0/4279230209 wait complete. 2024-05-10T11:30:47.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.772+0000 7f57885c4b80 20 client.27272 trim_cache size 0 max 16384 2024-05-10T11:30:47.778 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:30:47.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.772+0000 7f577dd31700 1 -- 172.21.15.100:0/3606746129 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f57600b2900 msgr2=0x7f57600b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:47.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.773+0000 7f577dd31700 1 --2- 172.21.15.100:0/3606746129 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f57600b2900 0x7f57600b4dc0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f5770009fd0 tx=0x7f577000b720 comp rx=0 tx=0).stop 2024-05-10T11:30:47.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.773+0000 7f577dd31700 1 -- 172.21.15.100:0/3606746129 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57781023a0 msgr2=0x7f577819a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:47.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.773+0000 7f577dd31700 1 --2- 172.21.15.100:0/3606746129 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57781023a0 0x7f577819a910 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f576800e9f0 tx=0x7f576800ed00 comp rx=0 tx=0).stop 2024-05-10T11:30:47.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.773+0000 7f577dd31700 1 -- 172.21.15.100:0/3606746129 shutdown_connections 2024-05-10T11:30:47.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.773+0000 7f577dd31700 1 --2- 172.21.15.100:0/3606746129 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f57600b2900 0x7f57600b4dc0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:47.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.773+0000 7f577dd31700 1 --2- 172.21.15.100:0/3606746129 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5778108410 0x7f57781a5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:47.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.773+0000 7f577dd31700 1 --2- 172.21.15.100:0/3606746129 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57781023a0 0x7f577819a910 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:47.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.773+0000 7f577dd31700 1 --2- 172.21.15.100:0/3606746129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5778102d60 0x7f577819ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:47.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.773+0000 7f577dd31700 1 -- 172.21.15.100:0/3606746129 >> 172.21.15.100:0/3606746129 conn(0x7f5778079350 msgr2=0x7f5778100fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:47.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.773+0000 7f577dd31700 1 -- 172.21.15.100:0/3606746129 shutdown_connections 2024-05-10T11:30:47.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:47.773+0000 7f577dd31700 1 -- 172.21.15.100:0/3606746129 wait complete. 2024-05-10T11:30:47.983 INFO:tasks.workunit.client.0.smithi100.stdout: 1 326905 2.71 MB/sec execute 522 sec latency 38.307 ms 2024-05-10T11:30:48.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:30:47 smithi178 ceph-mon[33359]: pgmap v793: 129 pgs: 129 active+clean; 129 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 1.4 MiB/s wr, 154 op/s 2024-05-10T11:30:48.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:30:47 smithi188 ceph-mon[34075]: pgmap v793: 129 pgs: 129 active+clean; 129 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 1.4 MiB/s wr, 154 op/s 2024-05-10T11:30:48.523 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-10T11:30:48.796 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:30:48.983 INFO:tasks.workunit.client.0.smithi100.stdout: 1 327324 2.71 MB/sec execute 523 sec latency 13.769 ms 2024-05-10T11:30:49.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:30:49 smithi100 ceph-mon[30296]: pgmap v794: 129 pgs: 129 active+clean; 129 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 1.4 MiB/s wr, 159 op/s 2024-05-10T11:30:49.983 INFO:tasks.workunit.client.0.smithi100.stdout: 1 327848 2.71 MB/sec execute 524 sec latency 21.014 ms 2024-05-10T11:30:50.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:30:49 smithi178 ceph-mon[33359]: pgmap v794: 129 pgs: 129 active+clean; 129 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 1.4 MiB/s wr, 159 op/s 2024-05-10T11:30:50.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:30:49 smithi188 ceph-mon[34075]: pgmap v794: 129 pgs: 129 active+clean; 129 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 1.4 MiB/s wr, 159 op/s 2024-05-10T11:30:50.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.646+0000 7f91293d4700 1 -- 172.21.15.100:0/1784816458 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f912406bbf0 msgr2=0x7f912406c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:50.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.646+0000 7f91293d4700 1 --2- 172.21.15.100:0/1784816458 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f912406bbf0 0x7f912406c070 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f91180099c0 tx=0x7f9118009cd0 comp rx=0 tx=0).stop 2024-05-10T11:30:50.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.646+0000 7f91293d4700 1 -- 172.21.15.100:0/1784816458 shutdown_connections 2024-05-10T11:30:50.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.646+0000 7f91293d4700 1 --2- 172.21.15.100:0/1784816458 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f912406c5b0 0x7f9124110350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:50.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.646+0000 7f91293d4700 1 --2- 172.21.15.100:0/1784816458 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91241095c0 0x7f91241099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:50.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.646+0000 7f91293d4700 1 --2- 172.21.15.100:0/1784816458 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f912406bbf0 0x7f912406c070 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:50.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.646+0000 7f91293d4700 1 -- 172.21.15.100:0/1784816458 >> 172.21.15.100:0/1784816458 conn(0x7f9124079260 msgr2=0x7f9124079670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:50.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.646+0000 7f91293d4700 1 -- 172.21.15.100:0/1784816458 shutdown_connections 2024-05-10T11:30:50.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.646+0000 7f91293d4700 1 -- 172.21.15.100:0/1784816458 wait complete. 2024-05-10T11:30:50.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.647+0000 7f91293d4700 1 Processor -- start 2024-05-10T11:30:50.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.647+0000 7f91293d4700 1 -- start start 2024-05-10T11:30:50.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.647+0000 7f91293d4700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f912406bbf0 0x7f912419f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:50.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.648+0000 7f91293d4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f912406c5b0 0x7f912419fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:50.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.648+0000 7f91293d4700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91241095c0 0x7f9124199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:50.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.648+0000 7f91293d4700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9124113530 con 0x7f912406c5b0 2024-05-10T11:30:50.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.648+0000 7f91293d4700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f91241133b0 con 0x7f91241095c0 2024-05-10T11:30:50.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.648+0000 7f91293d4700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f91241136b0 con 0x7f912406bbf0 2024-05-10T11:30:50.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.648+0000 7f91227fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f912406c5b0 0x7f912419fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:50.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.648+0000 7f91227fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f912406c5b0 0x7f912419fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43542/0 (socket says 172.21.15.100:43542) 2024-05-10T11:30:50.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.648+0000 7f9122ffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f912406bbf0 0x7f912419f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:50.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.648+0000 7f91227fc700 1 -- 172.21.15.100:0/3305561504 learned_addr learned my addr 172.21.15.100:0/3305561504 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:30:50.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.648+0000 7f91237fe700 1 --2- 172.21.15.100:0/3305561504 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91241095c0 0x7f9124199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:50.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.648+0000 7f91227fc700 1 -- 172.21.15.100:0/3305561504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f912406bbf0 msgr2=0x7f912419f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:50.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.648+0000 7f91227fc700 1 --2- 172.21.15.100:0/3305561504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f912406bbf0 0x7f912419f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:50.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.648+0000 7f91227fc700 1 -- 172.21.15.100:0/3305561504 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91241095c0 msgr2=0x7f9124199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:50.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.648+0000 7f91227fc700 1 --2- 172.21.15.100:0/3305561504 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91241095c0 0x7f9124199890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:50.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.648+0000 7f91227fc700 1 -- 172.21.15.100:0/3305561504 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9124199ff0 con 0x7f912406c5b0 2024-05-10T11:30:50.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.648+0000 7f9122ffd700 1 --2- 172.21.15.100:0/3305561504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f912406bbf0 0x7f912419f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:30:50.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.648+0000 7f91227fc700 1 --2- 172.21.15.100:0/3305561504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f912406c5b0 0x7f912419fc10 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f9118006010 tx=0x7f91180045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:50.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.648+0000 7f910bfff700 1 -- 172.21.15.100:0/3305561504 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9118026070 con 0x7f912406c5b0 2024-05-10T11:30:50.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.648+0000 7f910bfff700 1 -- 172.21.15.100:0/3305561504 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f911800f460 con 0x7f912406c5b0 2024-05-10T11:30:50.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.648+0000 7f91293d4700 1 -- 172.21.15.100:0/3305561504 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f912419a280 con 0x7f912406c5b0 2024-05-10T11:30:50.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.648+0000 7f910bfff700 1 -- 172.21.15.100:0/3305561504 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9118009e60 con 0x7f912406c5b0 2024-05-10T11:30:50.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.648+0000 7f91293d4700 1 -- 172.21.15.100:0/3305561504 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91241a65a0 con 0x7f912406c5b0 2024-05-10T11:30:50.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.651+0000 7f910bfff700 1 -- 172.21.15.100:0/3305561504 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f9118029070 con 0x7f912406c5b0 2024-05-10T11:30:50.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.652+0000 7f910bfff700 1 --2- 172.21.15.100:0/3305561504 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f910c0b2880 0x7f910c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:50.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.652+0000 7f9122ffd700 1 --2- 172.21.15.100:0/3305561504 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f910c0b2880 0x7f910c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:50.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.652+0000 7f910bfff700 1 -- 172.21.15.100:0/3305561504 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f91180e2dd0 con 0x7f912406c5b0 2024-05-10T11:30:50.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.652+0000 7f9133c67b80 1 Processor -- start 2024-05-10T11:30:50.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.652+0000 7f9122ffd700 1 --2- 172.21.15.100:0/3305561504 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f910c0b2880 0x7f910c0b4d40 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f912419a9d0 tx=0x7f9114009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:50.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.652+0000 7f9133c67b80 1 -- start start 2024-05-10T11:30:50.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.653+0000 7f9133c67b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562501a144d0 0x562501a148b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:50.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.653+0000 7f9133c67b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562501a14df0 0x562501a186f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:50.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.653+0000 7f9133c67b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562501a18c80 0x562501a19100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:50.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.653+0000 7f9133c67b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x562501966860 con 0x562501a144d0 2024-05-10T11:30:50.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.653+0000 7f9122ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562501a144d0 0x562501a148b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:50.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.653+0000 7f9122ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562501a144d0 0x562501a148b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43546/0 (socket says 172.21.15.100:43546) 2024-05-10T11:30:50.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.653+0000 7f9122ffd700 1 -- 172.21.15.100:0/2158088777 learned_addr learned my addr 172.21.15.100:0/2158088777 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:30:50.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.653+0000 7f9133c67b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5625019057c0 con 0x562501a18c80 2024-05-10T11:30:50.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.653+0000 7f9133c67b80 1 -- 172.21.15.100:0/2158088777 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x562501962820 con 0x562501a14df0 2024-05-10T11:30:50.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.653+0000 7f91237fe700 1 --2- 172.21.15.100:0/2158088777 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562501a18c80 0x562501a19100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:50.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.653+0000 7f91227fc700 1 --2- 172.21.15.100:0/2158088777 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562501a14df0 0x562501a186f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:50.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.653+0000 7f9122ffd700 1 -- 172.21.15.100:0/2158088777 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562501a14df0 msgr2=0x562501a186f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:50.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.653+0000 7f9122ffd700 1 --2- 172.21.15.100:0/2158088777 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562501a14df0 0x562501a186f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:50.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.653+0000 7f9122ffd700 1 -- 172.21.15.100:0/2158088777 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562501a18c80 msgr2=0x562501a19100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:50.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.653+0000 7f9122ffd700 1 --2- 172.21.15.100:0/2158088777 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562501a18c80 0x562501a19100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:50.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.653+0000 7f9122ffd700 1 -- 172.21.15.100:0/2158088777 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625018b38d0 con 0x562501a144d0 2024-05-10T11:30:50.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.653+0000 7f9122ffd700 1 --2- 172.21.15.100:0/2158088777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562501a144d0 0x562501a148b0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f91140043f0 tx=0x7f9114014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77727c7e9cce7ad5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:50.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.653+0000 7f91097fa700 1 -- 172.21.15.100:0/2158088777 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9114003d10 con 0x562501a144d0 2024-05-10T11:30:50.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.653+0000 7f91097fa700 1 -- 172.21.15.100:0/2158088777 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f911401e8b0 con 0x562501a144d0 2024-05-10T11:30:50.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.653+0000 7f91097fa700 1 -- 172.21.15.100:0/2158088777 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9114026730 con 0x562501a144d0 2024-05-10T11:30:50.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.654+0000 7f9133c67b80 1 -- 172.21.15.100:0/2158088777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562501a144d0 msgr2=0x562501a148b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:50.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.654+0000 7f9133c67b80 1 --2- 172.21.15.100:0/2158088777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562501a144d0 0x562501a148b0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f91140043f0 tx=0x7f9114014000 comp rx=0 tx=0).stop 2024-05-10T11:30:50.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.654+0000 7f9133c67b80 1 -- 172.21.15.100:0/2158088777 shutdown_connections 2024-05-10T11:30:50.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.654+0000 7f9133c67b80 1 --2- 172.21.15.100:0/2158088777 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562501a18c80 0x562501a19100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:50.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.654+0000 7f9133c67b80 1 --2- 172.21.15.100:0/2158088777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562501a144d0 0x562501a148b0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:50.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.654+0000 7f9133c67b80 1 --2- 172.21.15.100:0/2158088777 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562501a14df0 0x562501a186f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:50.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.654+0000 7f9133c67b80 1 -- 172.21.15.100:0/2158088777 >> 172.21.15.100:0/2158088777 conn(0x562501940100 msgr2=0x562501a0ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:50.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.654+0000 7f9133c67b80 1 -- 172.21.15.100:0/2158088777 shutdown_connections 2024-05-10T11:30:50.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.654+0000 7f9133c67b80 1 -- 172.21.15.100:0/2158088777 wait complete. 2024-05-10T11:30:50.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.655+0000 7f9133c67b80 1 -- start start 2024-05-10T11:30:50.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.655+0000 7f9133c67b80 1 Processor -- start 2024-05-10T11:30:50.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.655+0000 7f9133c67b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562501a144d0 0x562501a20010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:50.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.655+0000 7f9133c67b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562501a14df0 0x562501a1d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:50.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.655+0000 7f9133c67b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562501a18c80 0x562501a1d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:50.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.655+0000 7f9133c67b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x562501a20e90 con 0x562501a18c80 2024-05-10T11:30:50.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.655+0000 7f9133c67b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5625018ccc30 con 0x562501a14df0 2024-05-10T11:30:50.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.655+0000 7f9133c67b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x562501a21010 con 0x562501a144d0 2024-05-10T11:30:50.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.655+0000 7f9133c67b80 10 client.0 _ll_register_callbacks cb 0x5625018fe920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:30:50.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.655+0000 7f9122ffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562501a144d0 0x562501a20010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:50.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.655+0000 7f91237fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562501a18c80 0x562501a1d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:50.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.655+0000 7f9122ffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562501a144d0 0x562501a20010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:36162/0 (socket says 172.21.15.100:36162) 2024-05-10T11:30:50.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.655+0000 7f91237fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562501a18c80 0x562501a1d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43560/0 (socket says 172.21.15.100:43560) 2024-05-10T11:30:50.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.655+0000 7f9122ffd700 1 -- 172.21.15.100:0/3910542734 learned_addr learned my addr 172.21.15.100:0/3910542734 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:30:50.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.655+0000 7f91227fc700 1 --2- 172.21.15.100:0/3910542734 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562501a14df0 0x562501a1d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:50.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.655+0000 7f9122ffd700 1 -- 172.21.15.100:0/3910542734 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562501a14df0 msgr2=0x562501a1d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:50.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.655+0000 7f9122ffd700 1 --2- 172.21.15.100:0/3910542734 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562501a14df0 0x562501a1d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:50.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.655+0000 7f9122ffd700 1 -- 172.21.15.100:0/3910542734 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562501a18c80 msgr2=0x562501a1d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:50.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.655+0000 7f9122ffd700 1 --2- 172.21.15.100:0/3910542734 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562501a18c80 0x562501a1d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:50.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.655+0000 7f9122ffd700 1 -- 172.21.15.100:0/3910542734 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562501902690 con 0x562501a144d0 2024-05-10T11:30:50.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.655+0000 7f91227fc700 1 --2- 172.21.15.100:0/3910542734 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562501a14df0 0x562501a1d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:30:50.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.656+0000 7f9122ffd700 1 --2- 172.21.15.100:0/3910542734 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562501a144d0 0x562501a20010 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9114003db0 tx=0x7f9114003de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70a48f88450612bf server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:50.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.656+0000 7f9106ff5700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:30:50.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.656+0000 7f9106ff5700 1 -- 172.21.15.100:0/3910542734 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f911401ec70 con 0x562501a144d0 2024-05-10T11:30:50.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.656+0000 7f9106ff5700 1 -- 172.21.15.100:0/3910542734 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f9114005040 con 0x562501a144d0 2024-05-10T11:30:50.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.656+0000 7f9133c67b80 1 -- 172.21.15.100:0/3910542734 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f911401e8b0 con 0x562501a144d0 2024-05-10T11:30:50.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.656+0000 7f9106ff5700 1 -- 172.21.15.100:0/3910542734 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f911403a050 con 0x562501a144d0 2024-05-10T11:30:50.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.656+0000 7f9133c67b80 10 client.27376 fetch_fsmap learned FSMap version 33 2024-05-10T11:30:50.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.656+0000 7f9133c67b80 1 -- 172.21.15.100:0/3910542734 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x562501a2b660 con 0x562501a144d0 2024-05-10T11:30:50.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.656+0000 7f9106ff5700 1 -- 172.21.15.100:0/3910542734 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f911401e8b0 con 0x562501a144d0 2024-05-10T11:30:50.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.656+0000 7f9106ff5700 1 -- 172.21.15.100:0/3910542734 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f9114004c70 con 0x562501a144d0 2024-05-10T11:30:50.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.656+0000 7f9133c67b80 10 client.27376 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T11:30:50.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.656+0000 7f9133c67b80 10 client.27376 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:30:50.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.656+0000 7f9133c67b80 20 client.27376 populate_metadata read hostname 'smithi100' 2024-05-10T11:30:50.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.657+0000 7f9133c67b80 1 --2- 172.21.15.100:0/3910542734 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562501a2dbe0 0x562501a2ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:50.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.657+0000 7f9133c67b80 4 client.27376 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:30:50.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.657+0000 7f9133c67b80 1 -- 172.21.15.100:0/3910542734 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5625019e0650 con 0x562501a2dbe0 2024-05-10T11:30:50.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.657+0000 7f91227fc700 1 --2- 172.21.15.100:0/3910542734 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562501a2dbe0 0x562501a2ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:50.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.657+0000 7f91227fc700 1 --2- 172.21.15.100:0/3910542734 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562501a2dbe0 0x562501a2ffd0 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ac8a342098aa643 server_cookie=79585f784c68070c in_seq=0 out_seq=0 2024-05-10T11:30:50.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.657+0000 7f9106ff5700 10 client.27376 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:30:50.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.658+0000 7f9106ff5700 1 -- 172.21.15.100:0/3910542734 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5625019e0650 con 0x562501a2dbe0 2024-05-10T11:30:50.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.658+0000 7f9106ff5700 10 client.27376 handle_command_reply: tid=0 2024-05-10T11:30:50.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.658+0000 7f9133c67b80 1 client.27376 shutdown 2024-05-10T11:30:50.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.659+0000 7f9133c67b80 1 -- 172.21.15.100:0/3910542734 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562501a144d0 msgr2=0x562501a20010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:50.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.659+0000 7f9133c67b80 1 --2- 172.21.15.100:0/3910542734 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562501a144d0 0x562501a20010 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9114003db0 tx=0x7f9114003de0 comp rx=0 tx=0).stop 2024-05-10T11:30:50.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.659+0000 7f9133c67b80 1 -- 172.21.15.100:0/3910542734 shutdown_connections 2024-05-10T11:30:50.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.659+0000 7f9133c67b80 1 --2- 172.21.15.100:0/3910542734 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562501a2dbe0 0x562501a2ffd0 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:50.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.659+0000 7f9133c67b80 1 --2- 172.21.15.100:0/3910542734 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562501a18c80 0x562501a1d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:50.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.659+0000 7f9133c67b80 1 --2- 172.21.15.100:0/3910542734 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562501a144d0 0x562501a20010 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:50.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.659+0000 7f9133c67b80 1 --2- 172.21.15.100:0/3910542734 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562501a14df0 0x562501a1d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:50.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.659+0000 7f9106ff5700 0 client.27376 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:30:50.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.659+0000 7f9133c67b80 1 -- 172.21.15.100:0/3910542734 >> 172.21.15.100:0/3910542734 conn(0x562501940100 msgr2=0x562501a09720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:50.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.659+0000 7f9133c67b80 1 -- 172.21.15.100:0/3910542734 shutdown_connections 2024-05-10T11:30:50.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.659+0000 7f9133c67b80 1 -- 172.21.15.100:0/3910542734 wait complete. 2024-05-10T11:30:50.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.659+0000 7f9133c67b80 20 client.27376 trim_cache size 0 max 16384 2024-05-10T11:30:50.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.674+0000 7f9133c67b80 1 Processor -- start 2024-05-10T11:30:50.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.674+0000 7f9133c67b80 1 -- start start 2024-05-10T11:30:50.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.674+0000 7f9133c67b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562501940100 0x562501a1c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:50.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.674+0000 7f9133c67b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562501a144d0 0x562501a16520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:50.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.674+0000 7f9133c67b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562501a14df0 0x562501a16a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:50.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.674+0000 7f9133c67b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f91241af6b0 con 0x562501a14df0 2024-05-10T11:30:50.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.674+0000 7f9133c67b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f91241acf40 con 0x562501a144d0 2024-05-10T11:30:50.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.674+0000 7f9133c67b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f91241aad90 con 0x562501940100 2024-05-10T11:30:50.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.674+0000 7f91237fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562501a14df0 0x562501a16a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:50.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.674+0000 7f91237fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562501a14df0 0x562501a16a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43576/0 (socket says 172.21.15.100:43576) 2024-05-10T11:30:50.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.674+0000 7f91237fe700 1 -- 172.21.15.100:0/841450594 learned_addr learned my addr 172.21.15.100:0/841450594 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:30:50.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.674+0000 7f9122ffd700 1 --2- 172.21.15.100:0/841450594 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562501940100 0x562501a1c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:50.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.675+0000 7f91227fc700 1 --2- 172.21.15.100:0/841450594 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562501a144d0 0x562501a16520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:50.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.675+0000 7f91237fe700 1 -- 172.21.15.100:0/841450594 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562501940100 msgr2=0x562501a1c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:50.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.675+0000 7f91237fe700 1 --2- 172.21.15.100:0/841450594 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562501940100 0x562501a1c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:50.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.675+0000 7f91237fe700 1 -- 172.21.15.100:0/841450594 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562501a144d0 msgr2=0x562501a16520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:50.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.675+0000 7f91237fe700 1 --2- 172.21.15.100:0/841450594 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562501a144d0 0x562501a16520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:50.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.675+0000 7f91237fe700 1 -- 172.21.15.100:0/841450594 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562501a20ee0 con 0x562501a14df0 2024-05-10T11:30:50.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.675+0000 7f91237fe700 1 --2- 172.21.15.100:0/841450594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562501a14df0 0x562501a16a60 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f911c009e40 tx=0x7f911c009e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d1a87163ccd5893 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:50.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.675+0000 7f9108ff9700 1 -- 172.21.15.100:0/841450594 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f911c01b070 con 0x562501a14df0 2024-05-10T11:30:50.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.675+0000 7f9108ff9700 1 -- 172.21.15.100:0/841450594 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f911c004db0 con 0x562501a14df0 2024-05-10T11:30:50.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.675+0000 7f9108ff9700 1 -- 172.21.15.100:0/841450594 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f911c005a50 con 0x562501a14df0 2024-05-10T11:30:50.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.676+0000 7f9133c67b80 1 -- 172.21.15.100:0/841450594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562501a14df0 msgr2=0x562501a16a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:50.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.676+0000 7f9133c67b80 1 --2- 172.21.15.100:0/841450594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562501a14df0 0x562501a16a60 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f911c009e40 tx=0x7f911c009e70 comp rx=0 tx=0).stop 2024-05-10T11:30:50.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.676+0000 7f9133c67b80 1 -- 172.21.15.100:0/841450594 shutdown_connections 2024-05-10T11:30:50.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.676+0000 7f9133c67b80 1 --2- 172.21.15.100:0/841450594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562501a14df0 0x562501a16a60 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:50.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.676+0000 7f9133c67b80 1 --2- 172.21.15.100:0/841450594 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562501940100 0x562501a1c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:50.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.676+0000 7f9133c67b80 1 --2- 172.21.15.100:0/841450594 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562501a144d0 0x562501a16520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:50.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.676+0000 7f9133c67b80 1 -- 172.21.15.100:0/841450594 >> 172.21.15.100:0/841450594 conn(0x562501a08a80 msgr2=0x562501a09720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:50.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.676+0000 7f9133c67b80 1 -- 172.21.15.100:0/841450594 shutdown_connections 2024-05-10T11:30:50.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.676+0000 7f9133c67b80 1 -- 172.21.15.100:0/841450594 wait complete. 2024-05-10T11:30:50.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.677+0000 7f9133c67b80 1 -- start start 2024-05-10T11:30:50.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.677+0000 7f9133c67b80 1 Processor -- start 2024-05-10T11:30:50.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.677+0000 7f9133c67b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562501940100 0x562501a30b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:50.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.677+0000 7f9133c67b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562501a144d0 0x562501a31080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:50.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.677+0000 7f9133c67b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562501a14df0 0x562501a315c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:50.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.677+0000 7f9133c67b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f91241ab400 con 0x562501a14df0 2024-05-10T11:30:50.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.677+0000 7f9133c67b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f91241ac870 con 0x562501a144d0 2024-05-10T11:30:50.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.677+0000 7f9133c67b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f91241abad0 con 0x562501940100 2024-05-10T11:30:50.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.677+0000 7f9133c67b80 10 client.0 _ll_register_callbacks cb 0x5625018fe920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:30:50.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.677+0000 7f91237fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562501a14df0 0x562501a315c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:50.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.677+0000 7f91237fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562501a14df0 0x562501a315c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43586/0 (socket says 172.21.15.100:43586) 2024-05-10T11:30:50.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.677+0000 7f91237fe700 1 -- 172.21.15.100:0/2268894736 learned_addr learned my addr 172.21.15.100:0/2268894736 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:30:50.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.678+0000 7f91227fc700 1 --2- 172.21.15.100:0/2268894736 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562501a144d0 0x562501a31080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:50.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.678+0000 7f91237fe700 1 -- 172.21.15.100:0/2268894736 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562501940100 msgr2=0x562501a30b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T11:30:50.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.678+0000 7f91237fe700 1 --2- 172.21.15.100:0/2268894736 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562501940100 0x562501a30b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:50.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.678+0000 7f91237fe700 1 -- 172.21.15.100:0/2268894736 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562501a144d0 msgr2=0x562501a31080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:50.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.678+0000 7f91237fe700 1 --2- 172.21.15.100:0/2268894736 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562501a144d0 0x562501a31080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:50.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.678+0000 7f91237fe700 1 -- 172.21.15.100:0/2268894736 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562501a0b810 con 0x562501a14df0 2024-05-10T11:30:50.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.678+0000 7f91237fe700 1 --2- 172.21.15.100:0/2268894736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562501a14df0 0x562501a315c0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f911c009e10 tx=0x7f911c0052d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15bf5627b265b786 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:50.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.678+0000 7f9106ff5700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:30:50.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.678+0000 7f9106ff5700 1 -- 172.21.15.100:0/2268894736 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f911c01b070 con 0x562501a14df0 2024-05-10T11:30:50.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.678+0000 7f9106ff5700 1 -- 172.21.15.100:0/2268894736 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f911c005770 con 0x562501a14df0 2024-05-10T11:30:50.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.678+0000 7f9133c67b80 1 -- 172.21.15.100:0/2268894736 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f911c004db0 con 0x562501a14df0 2024-05-10T11:30:50.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.678+0000 7f9106ff5700 1 -- 172.21.15.100:0/2268894736 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f911c005970 con 0x562501a14df0 2024-05-10T11:30:50.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.678+0000 7f9133c67b80 10 client.17937 fetch_fsmap learned FSMap version 33 2024-05-10T11:30:50.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.678+0000 7f9133c67b80 1 -- 172.21.15.100:0/2268894736 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x562501a981d0 con 0x562501a14df0 2024-05-10T11:30:50.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.679+0000 7f9106ff5700 1 -- 172.21.15.100:0/2268894736 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f911c004db0 con 0x562501a14df0 2024-05-10T11:30:50.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.679+0000 7f9106ff5700 1 -- 172.21.15.100:0/2268894736 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f911c0294a0 con 0x562501a14df0 2024-05-10T11:30:50.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.679+0000 7f9133c67b80 10 client.17937 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T11:30:50.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.679+0000 7f9133c67b80 10 client.17937 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:30:50.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.679+0000 7f9133c67b80 20 client.17937 populate_metadata read hostname 'smithi100' 2024-05-10T11:30:50.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.679+0000 7f9133c67b80 1 --2- 172.21.15.100:0/2268894736 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562501a18c80 0x562501a9a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:50.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.679+0000 7f9133c67b80 4 client.17937 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:30:50.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.679+0000 7f9133c67b80 1 -- 172.21.15.100:0/2268894736 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562501974240 con 0x562501a18c80 2024-05-10T11:30:50.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.679+0000 7f9122ffd700 1 --2- 172.21.15.100:0/2268894736 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562501a18c80 0x562501a9a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:50.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.680+0000 7f9106ff5700 10 client.17937 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:30:50.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.680+0000 7f9122ffd700 1 --2- 172.21.15.100:0/2268894736 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562501a18c80 0x562501a9a830 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2216e88993d9775 server_cookie=78acfb11667c4b96 in_seq=0 out_seq=0 2024-05-10T11:30:50.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.680+0000 7f9106ff5700 1 -- 172.21.15.100:0/2268894736 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562501974240 con 0x562501a18c80 2024-05-10T11:30:50.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.680+0000 7f9106ff5700 10 client.17937 handle_command_reply: tid=0 2024-05-10T11:30:50.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.680+0000 7f9133c67b80 1 client.17937 shutdown 2024-05-10T11:30:50.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.680+0000 7f9133c67b80 1 -- 172.21.15.100:0/2268894736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562501a14df0 msgr2=0x562501a315c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:50.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.680+0000 7f9133c67b80 1 --2- 172.21.15.100:0/2268894736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562501a14df0 0x562501a315c0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f911c009e10 tx=0x7f911c0052d0 comp rx=0 tx=0).stop 2024-05-10T11:30:50.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.680+0000 7f9133c67b80 1 -- 172.21.15.100:0/2268894736 shutdown_connections 2024-05-10T11:30:50.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.680+0000 7f9133c67b80 1 --2- 172.21.15.100:0/2268894736 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562501a18c80 0x562501a9a830 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:50.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.680+0000 7f9133c67b80 1 --2- 172.21.15.100:0/2268894736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562501a14df0 0x562501a315c0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:50.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.680+0000 7f9133c67b80 1 --2- 172.21.15.100:0/2268894736 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562501940100 0x562501a30b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:50.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.680+0000 7f9133c67b80 1 --2- 172.21.15.100:0/2268894736 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562501a144d0 0x562501a31080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:50.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.680+0000 7f9133c67b80 1 -- 172.21.15.100:0/2268894736 >> 172.21.15.100:0/2268894736 conn(0x562501a08a80 msgr2=0x562501a132e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:50.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.680+0000 7f9106ff5700 0 client.17937 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:30:50.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.680+0000 7f9133c67b80 1 -- 172.21.15.100:0/2268894736 shutdown_connections 2024-05-10T11:30:50.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.680+0000 7f9133c67b80 1 -- 172.21.15.100:0/2268894736 wait complete. 2024-05-10T11:30:50.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.680+0000 7f9133c67b80 20 client.17937 trim_cache size 0 max 16384 2024-05-10T11:30:50.685 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:30:50.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.681+0000 7f91293d4700 1 -- 172.21.15.100:0/3305561504 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f910c0b2880 msgr2=0x7f910c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:50.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.681+0000 7f91293d4700 1 --2- 172.21.15.100:0/3305561504 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f910c0b2880 0x7f910c0b4d40 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f912419a9d0 tx=0x7f9114009260 comp rx=0 tx=0).stop 2024-05-10T11:30:50.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.681+0000 7f91293d4700 1 -- 172.21.15.100:0/3305561504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f912406c5b0 msgr2=0x7f912419fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:50.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.681+0000 7f91293d4700 1 --2- 172.21.15.100:0/3305561504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f912406c5b0 0x7f912419fc10 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f9118006010 tx=0x7f91180045e0 comp rx=0 tx=0).stop 2024-05-10T11:30:50.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.681+0000 7f91293d4700 1 -- 172.21.15.100:0/3305561504 shutdown_connections 2024-05-10T11:30:50.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.681+0000 7f91293d4700 1 --2- 172.21.15.100:0/3305561504 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f910c0b2880 0x7f910c0b4d40 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:50.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.681+0000 7f91293d4700 1 --2- 172.21.15.100:0/3305561504 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91241095c0 0x7f9124199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:50.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.681+0000 7f91293d4700 1 --2- 172.21.15.100:0/3305561504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f912406bbf0 0x7f912419f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:50.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.681+0000 7f91293d4700 1 --2- 172.21.15.100:0/3305561504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f912406c5b0 0x7f912419fc10 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:50.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.681+0000 7f91293d4700 1 -- 172.21.15.100:0/3305561504 >> 172.21.15.100:0/3305561504 conn(0x7f9124079260 msgr2=0x7f91241013b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:50.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.681+0000 7f91293d4700 1 -- 172.21.15.100:0/3305561504 shutdown_connections 2024-05-10T11:30:50.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:50.681+0000 7f91293d4700 1 -- 172.21.15.100:0/3305561504 wait complete. 2024-05-10T11:30:50.984 INFO:tasks.workunit.client.0.smithi100.stdout: 1 328434 2.71 MB/sec execute 525 sec latency 20.296 ms 2024-05-10T11:30:51.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:30:51 smithi100 ceph-mon[30296]: pgmap v795: 129 pgs: 129 active+clean; 130 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 1.1 MiB/s wr, 128 op/s 2024-05-10T11:30:51.983 INFO:tasks.workunit.client.0.smithi100.stdout: 1 328921 2.71 MB/sec execute 526 sec latency 15.435 ms 2024-05-10T11:30:52.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:30:51 smithi178 ceph-mon[33359]: pgmap v795: 129 pgs: 129 active+clean; 130 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 1.1 MiB/s wr, 128 op/s 2024-05-10T11:30:52.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:30:51 smithi188 ceph-mon[34075]: pgmap v795: 129 pgs: 129 active+clean; 130 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 1.1 MiB/s wr, 128 op/s 2024-05-10T11:30:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.085+0000 7fb722e2c640 1 -- 172.21.15.100:0/2429590681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb71c102b00 msgr2=0x7fb71c102f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.085+0000 7fb722e2c640 1 --2- 172.21.15.100:0/2429590681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb71c102b00 0x7fb71c102f80 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fb708009900 tx=0x7fb70802a410 comp rx=0 tx=0).stop 2024-05-10T11:30:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.085+0000 7fb722e2c640 1 -- 172.21.15.100:0/2429590681 shutdown_connections 2024-05-10T11:30:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.085+0000 7fb722e2c640 1 --2- 172.21.15.100:0/2429590681 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb71c1034c0 0x7fb71c109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.085+0000 7fb722e2c640 1 --2- 172.21.15.100:0/2429590681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb71c102b00 0x7fb71c102f80 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.085+0000 7fb722e2c640 1 --2- 172.21.15.100:0/2429590681 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb71c101910 0x7fb71c101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.086+0000 7fb722e2c640 1 -- 172.21.15.100:0/2429590681 >> 172.21.15.100:0/2429590681 conn(0x7fb71c0fce80 msgr2=0x7fb71c0ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.086+0000 7fb722e2c640 1 -- 172.21.15.100:0/2429590681 shutdown_connections 2024-05-10T11:30:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.086+0000 7fb722e2c640 1 -- 172.21.15.100:0/2429590681 wait complete. 2024-05-10T11:30:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.086+0000 7fb722e2c640 1 Processor -- start 2024-05-10T11:30:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.086+0000 7fb722e2c640 1 -- start start 2024-05-10T11:30:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.086+0000 7fb722e2c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb71c101910 0x7fb71c19aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:52.089 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.086+0000 7fb722e2c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb71c102b00 0x7fb71c19b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:52.089 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.086+0000 7fb722e2c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb71c1034c0 0x7fb71c1a2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:52.089 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.086+0000 7fb722e2c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb71c06abc0 con 0x7fb71c102b00 2024-05-10T11:30:52.089 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.086+0000 7fb722e2c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb71c06aa40 con 0x7fb71c101910 2024-05-10T11:30:52.089 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.086+0000 7fb722e2c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb71c06ad40 con 0x7fb71c1034c0 2024-05-10T11:30:52.089 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.087+0000 7fb71bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb71c102b00 0x7fb71c19b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:52.089 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.087+0000 7fb71bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb71c102b00 0x7fb71c19b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43604/0 (socket says 172.21.15.100:43604) 2024-05-10T11:30:52.089 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.087+0000 7fb71bfff640 1 -- 172.21.15.100:0/1590384912 learned_addr learned my addr 172.21.15.100:0/1590384912 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:30:52.089 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.087+0000 7fb7213c9640 1 --2- 172.21.15.100:0/1590384912 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb71c1034c0 0x7fb71c1a2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:52.089 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.087+0000 7fb720bc8640 1 --2- 172.21.15.100:0/1590384912 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb71c101910 0x7fb71c19aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:52.089 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.087+0000 7fb71bfff640 1 -- 172.21.15.100:0/1590384912 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb71c1034c0 msgr2=0x7fb71c1a2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:52.089 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.087+0000 7fb71bfff640 1 --2- 172.21.15.100:0/1590384912 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb71c1034c0 0x7fb71c1a2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:52.089 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.087+0000 7fb71bfff640 1 -- 172.21.15.100:0/1590384912 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb71c101910 msgr2=0x7fb71c19aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:52.090 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.087+0000 7fb71bfff640 1 --2- 172.21.15.100:0/1590384912 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb71c101910 0x7fb71c19aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:52.090 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.087+0000 7fb71bfff640 1 -- 172.21.15.100:0/1590384912 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb71c1a28e0 con 0x7fb71c102b00 2024-05-10T11:30:52.090 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.087+0000 7fb7213c9640 1 --2- 172.21.15.100:0/1590384912 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb71c1034c0 0x7fb71c1a2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:30:52.090 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.087+0000 7fb71bfff640 1 --2- 172.21.15.100:0/1590384912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb71c102b00 0x7fb71c19b1e0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fb708031460 tx=0x7fb708003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:52.090 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.087+0000 7fb719ffb640 1 -- 172.21.15.100:0/1590384912 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb70803f070 con 0x7fb71c102b00 2024-05-10T11:30:52.090 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.087+0000 7fb719ffb640 1 -- 172.21.15.100:0/1590384912 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb708031ba0 con 0x7fb71c102b00 2024-05-10T11:30:52.090 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.087+0000 7fb719ffb640 1 -- 172.21.15.100:0/1590384912 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb708035030 con 0x7fb71c102b00 2024-05-10T11:30:52.090 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.087+0000 7fb722e2c640 1 -- 172.21.15.100:0/1590384912 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb71c1a2b70 con 0x7fb71c102b00 2024-05-10T11:30:52.090 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.088+0000 7fb722e2c640 1 -- 172.21.15.100:0/1590384912 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb71c1a3060 con 0x7fb71c102b00 2024-05-10T11:30:52.097 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.090+0000 7fb725c1f740 1 Processor -- start 2024-05-10T11:30:52.097 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.090+0000 7fb725c1f740 1 -- start start 2024-05-10T11:30:52.097 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.090+0000 7fb719ffb640 1 -- 172.21.15.100:0/1590384912 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fb708042070 con 0x7fb71c102b00 2024-05-10T11:30:52.097 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.090+0000 7fb725c1f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d556e07db0 0x55d556e081d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:52.097 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.090+0000 7fb725c1f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d556e087a0 0x55d556e0d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:52.097 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.090+0000 7fb725c1f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d556e0d910 0x55d556e0fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:52.098 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.090+0000 7fb725c1f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55d556d6c5d0 con 0x55d556e087a0 2024-05-10T11:30:52.098 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.090+0000 7fb725c1f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d556d25890 con 0x55d556e07db0 2024-05-10T11:30:52.098 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.090+0000 7fb725c1f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d556d68590 con 0x55d556e0d910 2024-05-10T11:30:52.098 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.091+0000 7fb720bc8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d556e087a0 0x55d556e0d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:52.098 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.091+0000 7fb720bc8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d556e087a0 0x55d556e0d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43610/0 (socket says 172.21.15.100:43610) 2024-05-10T11:30:52.098 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.091+0000 7fb720bc8640 1 -- 172.21.15.100:0/2468174158 learned_addr learned my addr 172.21.15.100:0/2468174158 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:30:52.098 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.091+0000 7fb71bfff640 1 --2- 172.21.15.100:0/2468174158 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d556e0d910 0x55d556e0fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:52.098 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.091+0000 7fb7213c9640 1 --2- 172.21.15.100:0/2468174158 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d556e07db0 0x55d556e081d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:52.098 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.091+0000 7fb720bc8640 1 -- 172.21.15.100:0/2468174158 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d556e0d910 msgr2=0x55d556e0fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:52.098 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.091+0000 7fb720bc8640 1 --2- 172.21.15.100:0/2468174158 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d556e0d910 0x55d556e0fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:52.098 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.091+0000 7fb720bc8640 1 -- 172.21.15.100:0/2468174158 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d556e07db0 msgr2=0x55d556e081d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:52.098 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.091+0000 7fb720bc8640 1 --2- 172.21.15.100:0/2468174158 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d556e07db0 0x55d556e081d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:52.099 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.091+0000 7fb720bc8640 1 -- 172.21.15.100:0/2468174158 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d556a7e210 con 0x55d556e087a0 2024-05-10T11:30:52.099 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.091+0000 7fb7213c9640 1 --2- 172.21.15.100:0/2468174158 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d556e07db0 0x55d556e081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:30:52.099 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.091+0000 7fb71bfff640 1 --2- 172.21.15.100:0/2468174158 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d556e0d910 0x55d556e0fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:30:52.099 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.091+0000 7fb720bc8640 1 --2- 172.21.15.100:0/2468174158 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d556e087a0 0x55d556e0d2a0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fb710009c50 tx=0x7fb710011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=304de852d46855d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:52.099 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.091+0000 7fb719ffb640 1 --2- 172.21.15.100:0/1590384912 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fb70c0b28a0 0x7fb70c0b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:52.099 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.091+0000 7fb706ffd640 1 -- 172.21.15.100:0/2468174158 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb710004230 con 0x55d556e087a0 2024-05-10T11:30:52.099 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.091+0000 7fb706ffd640 1 -- 172.21.15.100:0/2468174158 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb7100043d0 con 0x55d556e087a0 2024-05-10T11:30:52.099 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.091+0000 7fb706ffd640 1 -- 172.21.15.100:0/2468174158 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb710019640 con 0x55d556e087a0 2024-05-10T11:30:52.099 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.091+0000 7fb7213c9640 1 --2- 172.21.15.100:0/1590384912 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fb70c0b28a0 0x7fb70c0b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:52.099 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.091+0000 7fb719ffb640 1 -- 172.21.15.100:0/1590384912 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fb708102d30 con 0x7fb71c102b00 2024-05-10T11:30:52.099 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.092+0000 7fb725c1f740 1 -- 172.21.15.100:0/2468174158 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d556e087a0 msgr2=0x55d556e0d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:52.100 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.092+0000 7fb7213c9640 1 --2- 172.21.15.100:0/1590384912 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fb70c0b28a0 0x7fb70c0b4d60 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x55d556e09000 tx=0x7fb714009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:52.100 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.092+0000 7fb725c1f740 1 --2- 172.21.15.100:0/2468174158 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d556e087a0 0x55d556e0d2a0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fb710009c50 tx=0x7fb710011810 comp rx=0 tx=0).stop 2024-05-10T11:30:52.100 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.092+0000 7fb725c1f740 1 -- 172.21.15.100:0/2468174158 shutdown_connections 2024-05-10T11:30:52.100 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.092+0000 7fb725c1f740 1 --2- 172.21.15.100:0/2468174158 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d556e0d910 0x55d556e0fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:52.100 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.092+0000 7fb725c1f740 1 --2- 172.21.15.100:0/2468174158 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d556e087a0 0x55d556e0d2a0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:52.100 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.092+0000 7fb725c1f740 1 --2- 172.21.15.100:0/2468174158 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d556e07db0 0x55d556e081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:52.100 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.092+0000 7fb725c1f740 1 -- 172.21.15.100:0/2468174158 >> 172.21.15.100:0/2468174158 conn(0x55d556e03420 msgr2=0x55d556e03820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:52.100 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.092+0000 7fb725c1f740 1 -- 172.21.15.100:0/2468174158 shutdown_connections 2024-05-10T11:30:52.100 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.092+0000 7fb725c1f740 1 -- 172.21.15.100:0/2468174158 wait complete. 2024-05-10T11:30:52.100 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.093+0000 7fb725c1f740 1 -- start start 2024-05-10T11:30:52.100 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.093+0000 7fb725c1f740 1 Processor -- start 2024-05-10T11:30:52.100 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.093+0000 7fb725c1f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d556e07db0 0x55d556e11f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:52.100 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.093+0000 7fb725c1f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d556e087a0 0x55d556e124d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:52.100 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.093+0000 7fb725c1f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d556e0d910 0x55d556e19510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:52.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.093+0000 7fb725c1f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55d556e108c0 con 0x55d556e07db0 2024-05-10T11:30:52.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.093+0000 7fb725c1f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d556d29370 con 0x55d556e087a0 2024-05-10T11:30:52.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.093+0000 7fb725c1f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d556e10a40 con 0x55d556e0d910 2024-05-10T11:30:52.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.093+0000 7fb725c1f740 10 client.0 _ll_register_callbacks cb 0x55d556a7da00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:30:52.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.093+0000 7fb720bc8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d556e07db0 0x55d556e11f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:52.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.093+0000 7fb720bc8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d556e07db0 0x55d556e11f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43624/0 (socket says 172.21.15.100:43624) 2024-05-10T11:30:52.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.093+0000 7fb720bc8640 1 -- 172.21.15.100:0/2894011172 learned_addr learned my addr 172.21.15.100:0/2894011172 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:30:52.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.093+0000 7fb7213c9640 1 --2- 172.21.15.100:0/2894011172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d556e0d910 0x55d556e19510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:52.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.093+0000 7fb71bfff640 1 --2- 172.21.15.100:0/2894011172 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d556e087a0 0x55d556e124d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:52.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.093+0000 7fb720bc8640 1 -- 172.21.15.100:0/2894011172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d556e0d910 msgr2=0x55d556e19510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:52.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.093+0000 7fb720bc8640 1 --2- 172.21.15.100:0/2894011172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d556e0d910 0x55d556e19510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:52.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.093+0000 7fb720bc8640 1 -- 172.21.15.100:0/2894011172 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d556e087a0 msgr2=0x55d556e124d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:52.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.093+0000 7fb720bc8640 1 --2- 172.21.15.100:0/2894011172 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d556e087a0 0x55d556e124d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:52.102 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.093+0000 7fb720bc8640 1 -- 172.21.15.100:0/2894011172 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d556b4a620 con 0x55d556e07db0 2024-05-10T11:30:52.102 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.093+0000 7fb71bfff640 1 --2- 172.21.15.100:0/2894011172 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d556e087a0 0x55d556e124d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:30:52.102 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.093+0000 7fb7213c9640 1 --2- 172.21.15.100:0/2894011172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d556e0d910 0x55d556e19510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:30:52.102 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.094+0000 7fb720bc8640 1 --2- 172.21.15.100:0/2894011172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d556e07db0 0x55d556e11f90 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fb710000c00 tx=0x7fb7100038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8140aaa5740612d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:52.102 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.094+0000 7fb6fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:30:52.102 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.094+0000 7fb6fbfff640 1 -- 172.21.15.100:0/2894011172 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb710028020 con 0x55d556e07db0 2024-05-10T11:30:52.102 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.094+0000 7fb6fbfff640 1 -- 172.21.15.100:0/2894011172 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fb710005240 con 0x55d556e07db0 2024-05-10T11:30:52.102 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.094+0000 7fb725c1f740 1 -- 172.21.15.100:0/2894011172 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7100043d0 con 0x55d556e07db0 2024-05-10T11:30:52.102 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.094+0000 7fb6fbfff640 1 -- 172.21.15.100:0/2894011172 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb710033050 con 0x55d556e07db0 2024-05-10T11:30:52.102 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.094+0000 7fb725c1f740 10 client.17949 fetch_fsmap learned FSMap version 33 2024-05-10T11:30:52.102 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.094+0000 7fb725c1f740 1 -- 172.21.15.100:0/2894011172 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d556e1c4f0 con 0x55d556e07db0 2024-05-10T11:30:52.102 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.094+0000 7fb6fbfff640 1 -- 172.21.15.100:0/2894011172 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb7100043d0 con 0x55d556e07db0 2024-05-10T11:30:52.103 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.094+0000 7fb6fbfff640 1 -- 172.21.15.100:0/2894011172 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fb710018070 con 0x55d556e07db0 2024-05-10T11:30:52.103 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.094+0000 7fb725c1f740 10 client.17949 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T11:30:52.103 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.094+0000 7fb725c1f740 10 client.17949 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:30:52.103 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.094+0000 7fb725c1f740 20 client.17949 populate_metadata read hostname 'smithi100' 2024-05-10T11:30:52.103 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.094+0000 7fb725c1f740 1 --2- 172.21.15.100:0/2894011172 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d556e1da40 0x55d556e1fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:52.103 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.094+0000 7fb725c1f740 4 client.17949 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:30:52.103 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.094+0000 7fb725c1f740 1 -- 172.21.15.100:0/2894011172 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d556d7a1e0 con 0x55d556e1da40 2024-05-10T11:30:52.103 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.095+0000 7fb71bfff640 1 --2- 172.21.15.100:0/2894011172 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d556e1da40 0x55d556e1fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:52.103 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.095+0000 7fb6fbfff640 10 client.17949 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:30:52.103 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.095+0000 7fb71bfff640 1 --2- 172.21.15.100:0/2894011172 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d556e1da40 0x55d556e1fe70 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96032bf57c6bbae2 server_cookie=ce7190be49e8b6d1 in_seq=0 out_seq=0 2024-05-10T11:30:52.103 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.096+0000 7fb6fbfff640 1 -- 172.21.15.100:0/2894011172 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d556d7a1e0 con 0x55d556e1da40 2024-05-10T11:30:52.103 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.096+0000 7fb6fbfff640 10 client.17949 handle_command_reply: tid=0 2024-05-10T11:30:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.096+0000 7fb725c1f740 1 client.17949 shutdown 2024-05-10T11:30:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.096+0000 7fb725c1f740 1 -- 172.21.15.100:0/2894011172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d556e07db0 msgr2=0x55d556e11f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.096+0000 7fb725c1f740 1 --2- 172.21.15.100:0/2894011172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d556e07db0 0x55d556e11f90 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fb710000c00 tx=0x7fb7100038a0 comp rx=0 tx=0).stop 2024-05-10T11:30:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.096+0000 7fb725c1f740 1 -- 172.21.15.100:0/2894011172 shutdown_connections 2024-05-10T11:30:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.096+0000 7fb725c1f740 1 --2- 172.21.15.100:0/2894011172 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d556e1da40 0x55d556e1fe70 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.096+0000 7fb725c1f740 1 --2- 172.21.15.100:0/2894011172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d556e0d910 0x55d556e19510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.096+0000 7fb725c1f740 1 --2- 172.21.15.100:0/2894011172 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d556e087a0 0x55d556e124d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.096+0000 7fb6fbfff640 0 client.17949 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:30:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.096+0000 7fb725c1f740 1 --2- 172.21.15.100:0/2894011172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d556e07db0 0x55d556e11f90 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.096+0000 7fb725c1f740 1 -- 172.21.15.100:0/2894011172 >> 172.21.15.100:0/2894011172 conn(0x55d556e03420 msgr2=0x55d556e02580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.096+0000 7fb725c1f740 1 -- 172.21.15.100:0/2894011172 shutdown_connections 2024-05-10T11:30:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.096+0000 7fb725c1f740 1 -- 172.21.15.100:0/2894011172 wait complete. 2024-05-10T11:30:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.096+0000 7fb725c1f740 20 client.17949 trim_cache size 0 max 16384 2024-05-10T11:30:52.116 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.111+0000 7fb725c1f740 1 Processor -- start 2024-05-10T11:30:52.116 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.112+0000 7fb725c1f740 1 -- start start 2024-05-10T11:30:52.116 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.112+0000 7fb725c1f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d556e07db0 0x55d556e02400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:52.116 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.112+0000 7fb725c1f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d556e087a0 0x55d556e02940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:52.116 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.112+0000 7fb725c1f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d556e0d910 0x55d556e194d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:52.116 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.112+0000 7fb725c1f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb71c1afd80 con 0x55d556e0d910 2024-05-10T11:30:52.116 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.112+0000 7fb725c1f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb71c1ad650 con 0x55d556e087a0 2024-05-10T11:30:52.116 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.112+0000 7fb725c1f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb71c1ab4a0 con 0x55d556e07db0 2024-05-10T11:30:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.112+0000 7fb7213c9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d556e0d910 0x55d556e194d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.112+0000 7fb7213c9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d556e0d910 0x55d556e194d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43636/0 (socket says 172.21.15.100:43636) 2024-05-10T11:30:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.112+0000 7fb720bc8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d556e07db0 0x55d556e02400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.112+0000 7fb71bfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d556e087a0 0x55d556e02940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.112+0000 7fb7213c9640 1 -- 172.21.15.100:0/1119813495 learned_addr learned my addr 172.21.15.100:0/1119813495 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:30:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.112+0000 7fb7213c9640 1 -- 172.21.15.100:0/1119813495 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d556e07db0 msgr2=0x55d556e02400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.112+0000 7fb7213c9640 1 --2- 172.21.15.100:0/1119813495 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d556e07db0 0x55d556e02400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.112+0000 7fb7213c9640 1 -- 172.21.15.100:0/1119813495 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d556e087a0 msgr2=0x55d556e02940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.112+0000 7fb7213c9640 1 --2- 172.21.15.100:0/1119813495 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d556e087a0 0x55d556e02940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.112+0000 7fb7213c9640 1 -- 172.21.15.100:0/1119813495 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d556e0a1b0 con 0x55d556e0d910 2024-05-10T11:30:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.112+0000 7fb7213c9640 1 --2- 172.21.15.100:0/1119813495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d556e0d910 0x55d556e194d0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fb714003aa0 tx=0x7fb71401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2996635bfa53048 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.112+0000 7fb7067fc640 1 -- 172.21.15.100:0/1119813495 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71402e070 con 0x55d556e0d910 2024-05-10T11:30:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.112+0000 7fb7067fc640 1 -- 172.21.15.100:0/1119813495 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb714004670 con 0x55d556e0d910 2024-05-10T11:30:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.112+0000 7fb7067fc640 1 -- 172.21.15.100:0/1119813495 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb714029680 con 0x55d556e0d910 2024-05-10T11:30:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.113+0000 7fb725c1f740 1 -- 172.21.15.100:0/1119813495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d556e0d910 msgr2=0x55d556e194d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.113+0000 7fb725c1f740 1 --2- 172.21.15.100:0/1119813495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d556e0d910 0x55d556e194d0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fb714003aa0 tx=0x7fb71401b000 comp rx=0 tx=0).stop 2024-05-10T11:30:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.113+0000 7fb725c1f740 1 -- 172.21.15.100:0/1119813495 shutdown_connections 2024-05-10T11:30:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.113+0000 7fb725c1f740 1 --2- 172.21.15.100:0/1119813495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d556e0d910 0x55d556e194d0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.113+0000 7fb725c1f740 1 --2- 172.21.15.100:0/1119813495 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d556e087a0 0x55d556e02940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.113+0000 7fb725c1f740 1 --2- 172.21.15.100:0/1119813495 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d556e07db0 0x55d556e02400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.113+0000 7fb725c1f740 1 -- 172.21.15.100:0/1119813495 >> 172.21.15.100:0/1119813495 conn(0x55d556e03420 msgr2=0x55d556e01cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.113+0000 7fb725c1f740 1 -- 172.21.15.100:0/1119813495 shutdown_connections 2024-05-10T11:30:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.113+0000 7fb725c1f740 1 -- 172.21.15.100:0/1119813495 wait complete. 2024-05-10T11:30:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.113+0000 7fb725c1f740 1 -- start start 2024-05-10T11:30:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.114+0000 7fb725c1f740 1 Processor -- start 2024-05-10T11:30:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.114+0000 7fb725c1f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d556e07db0 0x55d556e16310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.114+0000 7fb725c1f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d556e087a0 0x55d556e16850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.114+0000 7fb725c1f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d556e0d910 0x55d556e86370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.114+0000 7fb725c1f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb71c1abb10 con 0x55d556e087a0 2024-05-10T11:30:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.114+0000 7fb725c1f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb71c1acf80 con 0x55d556e07db0 2024-05-10T11:30:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.114+0000 7fb725c1f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb71c1ac1e0 con 0x55d556e0d910 2024-05-10T11:30:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.114+0000 7fb725c1f740 10 client.0 _ll_register_callbacks cb 0x7fb7081066e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:30:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.114+0000 7fb71bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d556e087a0 0x55d556e16850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.114+0000 7fb71bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d556e087a0 0x55d556e16850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43646/0 (socket says 172.21.15.100:43646) 2024-05-10T11:30:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.114+0000 7fb71bfff640 1 -- 172.21.15.100:0/3366977388 learned_addr learned my addr 172.21.15.100:0/3366977388 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:30:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.114+0000 7fb7213c9640 1 --2- 172.21.15.100:0/3366977388 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d556e0d910 0x55d556e86370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.114+0000 7fb720bc8640 1 --2- 172.21.15.100:0/3366977388 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d556e07db0 0x55d556e16310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.114+0000 7fb71bfff640 1 -- 172.21.15.100:0/3366977388 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d556e0d910 msgr2=0x55d556e86370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.114+0000 7fb71bfff640 1 --2- 172.21.15.100:0/3366977388 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d556e0d910 0x55d556e86370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.114+0000 7fb71bfff640 1 -- 172.21.15.100:0/3366977388 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d556e07db0 msgr2=0x55d556e16310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.114+0000 7fb71bfff640 1 --2- 172.21.15.100:0/3366977388 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d556e07db0 0x55d556e16310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.114+0000 7fb71bfff640 1 -- 172.21.15.100:0/3366977388 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d556e868b0 con 0x55d556e087a0 2024-05-10T11:30:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.114+0000 7fb7213c9640 1 --2- 172.21.15.100:0/3366977388 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d556e0d910 0x55d556e86370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:30:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.115+0000 7fb71bfff640 1 --2- 172.21.15.100:0/3366977388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d556e087a0 0x55d556e16850 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fb7080c9da0 tx=0x7fb70802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61295c1d4019234a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.115+0000 7fb6fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:30:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.115+0000 7fb6fbfff640 1 -- 172.21.15.100:0/3366977388 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb708049040 con 0x55d556e087a0 2024-05-10T11:30:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.115+0000 7fb6fbfff640 1 -- 172.21.15.100:0/3366977388 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fb708031d50 con 0x55d556e087a0 2024-05-10T11:30:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.115+0000 7fb725c1f740 1 -- 172.21.15.100:0/3366977388 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb714004670 con 0x55d556e087a0 2024-05-10T11:30:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.115+0000 7fb6fbfff640 1 -- 172.21.15.100:0/3366977388 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb7080c9810 con 0x55d556e087a0 2024-05-10T11:30:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.115+0000 7fb725c1f740 10 client.17955 fetch_fsmap learned FSMap version 33 2024-05-10T11:30:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.115+0000 7fb725c1f740 1 -- 172.21.15.100:0/3366977388 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d556e87ba0 con 0x55d556e087a0 2024-05-10T11:30:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.115+0000 7fb6fbfff640 1 -- 172.21.15.100:0/3366977388 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb714004670 con 0x55d556e087a0 2024-05-10T11:30:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.115+0000 7fb6fbfff640 1 -- 172.21.15.100:0/3366977388 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fb7080c6330 con 0x55d556e087a0 2024-05-10T11:30:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.115+0000 7fb725c1f740 10 client.17955 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T11:30:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.115+0000 7fb725c1f740 10 client.17955 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:30:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.115+0000 7fb725c1f740 20 client.17955 populate_metadata read hostname 'smithi100' 2024-05-10T11:30:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.115+0000 7fb725c1f740 1 --2- 172.21.15.100:0/3366977388 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d556e88550 0x55d556e8a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.115+0000 7fb725c1f740 4 client.17955 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:30:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.115+0000 7fb725c1f740 1 -- 172.21.15.100:0/3366977388 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d556d74050 con 0x55d556e88550 2024-05-10T11:30:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.116+0000 7fb720bc8640 1 --2- 172.21.15.100:0/3366977388 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d556e88550 0x55d556e8a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.116+0000 7fb6fbfff640 10 client.17955 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:30:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.116+0000 7fb720bc8640 1 --2- 172.21.15.100:0/3366977388 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d556e88550 0x55d556e8a970 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4001cbb76bef61d server_cookie=f344f9679d93da16 in_seq=0 out_seq=0 2024-05-10T11:30:52.138 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.137+0000 7fb6fbfff640 1 -- 172.21.15.100:0/3366977388 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d556d74050 con 0x55d556e88550 2024-05-10T11:30:52.138 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.138+0000 7fb6fbfff640 10 client.17955 handle_command_reply: tid=0 2024-05-10T11:30:52.139 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.138+0000 7fb725c1f740 1 client.17955 shutdown 2024-05-10T11:30:52.139 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.138+0000 7fb725c1f740 1 -- 172.21.15.100:0/3366977388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d556e087a0 msgr2=0x55d556e16850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:52.139 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.138+0000 7fb725c1f740 1 --2- 172.21.15.100:0/3366977388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d556e087a0 0x55d556e16850 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fb7080c9da0 tx=0x7fb70802a410 comp rx=0 tx=0).stop 2024-05-10T11:30:52.139 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.138+0000 7fb725c1f740 1 -- 172.21.15.100:0/3366977388 shutdown_connections 2024-05-10T11:30:52.139 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.138+0000 7fb725c1f740 1 --2- 172.21.15.100:0/3366977388 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d556e88550 0x55d556e8a970 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:52.139 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.138+0000 7fb725c1f740 1 --2- 172.21.15.100:0/3366977388 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d556e0d910 0x55d556e86370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:52.139 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.138+0000 7fb725c1f740 1 --2- 172.21.15.100:0/3366977388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d556e087a0 0x55d556e16850 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:52.139 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.138+0000 7fb6fbfff640 0 client.17955 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:30:52.139 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.138+0000 7fb725c1f740 1 --2- 172.21.15.100:0/3366977388 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d556e07db0 0x55d556e16310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:52.139 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.138+0000 7fb725c1f740 1 -- 172.21.15.100:0/3366977388 >> 172.21.15.100:0/3366977388 conn(0x55d556e03420 msgr2=0x55d556e02580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:52.139 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.138+0000 7fb725c1f740 1 -- 172.21.15.100:0/3366977388 shutdown_connections 2024-05-10T11:30:52.139 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.138+0000 7fb725c1f740 1 -- 172.21.15.100:0/3366977388 wait complete. 2024-05-10T11:30:52.139 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.138+0000 7fb725c1f740 20 client.17955 trim_cache size 0 max 16384 2024-05-10T11:30:52.140 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:30:52.140 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:30:52.140 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:30:52.140 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:30:52.141 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.139+0000 7fb722e2c640 1 -- 172.21.15.100:0/1590384912 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fb70c0b28a0 msgr2=0x7fb70c0b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:52.141 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.139+0000 7fb722e2c640 1 --2- 172.21.15.100:0/1590384912 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fb70c0b28a0 0x7fb70c0b4d60 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x55d556e09000 tx=0x7fb714009500 comp rx=0 tx=0).stop 2024-05-10T11:30:52.141 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.139+0000 7fb722e2c640 1 -- 172.21.15.100:0/1590384912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb71c102b00 msgr2=0x7fb71c19b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:52.141 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.139+0000 7fb722e2c640 1 --2- 172.21.15.100:0/1590384912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb71c102b00 0x7fb71c19b1e0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fb708031460 tx=0x7fb708003c10 comp rx=0 tx=0).stop 2024-05-10T11:30:52.141 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.139+0000 7fb722e2c640 1 -- 172.21.15.100:0/1590384912 shutdown_connections 2024-05-10T11:30:52.141 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.139+0000 7fb722e2c640 1 --2- 172.21.15.100:0/1590384912 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fb70c0b28a0 0x7fb70c0b4d60 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:52.141 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.139+0000 7fb722e2c640 1 --2- 172.21.15.100:0/1590384912 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb71c1034c0 0x7fb71c1a2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:52.141 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.139+0000 7fb722e2c640 1 --2- 172.21.15.100:0/1590384912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb71c102b00 0x7fb71c19b1e0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:52.141 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.139+0000 7fb722e2c640 1 --2- 172.21.15.100:0/1590384912 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb71c101910 0x7fb71c19aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:52.141 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.139+0000 7fb722e2c640 1 -- 172.21.15.100:0/1590384912 >> 172.21.15.100:0/1590384912 conn(0x7fb71c0fce80 msgr2=0x7fb71c0fdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:52.142 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.139+0000 7fb722e2c640 1 -- 172.21.15.100:0/1590384912 shutdown_connections 2024-05-10T11:30:52.142 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:30:52.139+0000 7fb722e2c640 1 -- 172.21.15.100:0/1590384912 wait complete. 2024-05-10T11:30:52.613 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T11:30:52.614 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T11:30:52.881 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:30:52.983 INFO:tasks.workunit.client.0.smithi100.stdout: 1 329634 2.71 MB/sec execute 527 sec latency 19.761 ms 2024-05-10T11:30:53.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:30:53 smithi100 ceph-mon[30296]: pgmap v796: 129 pgs: 129 active+clean; 128 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 1.2 MiB/s wr, 132 op/s 2024-05-10T11:30:53.830 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:30:53 smithi188 ceph-mon[34075]: pgmap v796: 129 pgs: 129 active+clean; 128 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 1.2 MiB/s wr, 132 op/s 2024-05-10T11:30:53.984 INFO:tasks.workunit.client.0.smithi100.stdout: 1 330094 2.72 MB/sec execute 528 sec latency 15.141 ms 2024-05-10T11:30:54.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:30:53 smithi178 ceph-mon[33359]: pgmap v796: 129 pgs: 129 active+clean; 128 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 1.2 MiB/s wr, 132 op/s 2024-05-10T11:30:54.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.652+0000 7ff701ce0700 1 -- 172.21.15.100:0/1179022382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6fc103ef0 msgr2=0x7ff6fc1103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:54.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.652+0000 7ff701ce0700 1 --2- 172.21.15.100:0/1179022382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6fc103ef0 0x7ff6fc1103d0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7ff6f400b0a0 tx=0x7ff6f400b3b0 comp rx=0 tx=0).stop 2024-05-10T11:30:54.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.652+0000 7ff701ce0700 1 -- 172.21.15.100:0/1179022382 shutdown_connections 2024-05-10T11:30:54.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.652+0000 7ff701ce0700 1 --2- 172.21.15.100:0/1179022382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6fc103ef0 0x7ff6fc1103d0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:54.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.652+0000 7ff701ce0700 1 --2- 172.21.15.100:0/1179022382 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff6fc108460 0x7ff6fc108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:54.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.652+0000 7ff701ce0700 1 --2- 172.21.15.100:0/1179022382 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6fc103550 0x7ff6fc1039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:54.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.652+0000 7ff701ce0700 1 -- 172.21.15.100:0/1179022382 >> 172.21.15.100:0/1179022382 conn(0x7ff6fc07a600 msgr2=0x7ff6fc07aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:54.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.652+0000 7ff701ce0700 1 -- 172.21.15.100:0/1179022382 shutdown_connections 2024-05-10T11:30:54.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.652+0000 7ff701ce0700 1 -- 172.21.15.100:0/1179022382 wait complete. 2024-05-10T11:30:54.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.653+0000 7ff701ce0700 1 Processor -- start 2024-05-10T11:30:54.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.653+0000 7ff701ce0700 1 -- start start 2024-05-10T11:30:54.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.653+0000 7ff701ce0700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6fc103550 0x7ff6fc19f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:54.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.653+0000 7ff701ce0700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6fc103ef0 0x7ff6fc19fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:54.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.653+0000 7ff701ce0700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff6fc108460 0x7ff6fc199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:54.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.653+0000 7ff701ce0700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff6fc113570 con 0x7ff6fc103ef0 2024-05-10T11:30:54.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.653+0000 7ff701ce0700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff6fc1133f0 con 0x7ff6fc108460 2024-05-10T11:30:54.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.653+0000 7ff701ce0700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff6fc1136f0 con 0x7ff6fc103550 2024-05-10T11:30:54.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.653+0000 7ff6faffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6fc103ef0 0x7ff6fc19fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:54.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.653+0000 7ff6fb7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6fc103550 0x7ff6fc19f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:54.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.653+0000 7ff6faffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6fc103ef0 0x7ff6fc19fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43666/0 (socket says 172.21.15.100:43666) 2024-05-10T11:30:54.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.653+0000 7ff6faffd700 1 -- 172.21.15.100:0/2679610850 learned_addr learned my addr 172.21.15.100:0/2679610850 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:30:54.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.653+0000 7ff6fbfff700 1 --2- 172.21.15.100:0/2679610850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff6fc108460 0x7ff6fc199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:54.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.654+0000 7ff6faffd700 1 -- 172.21.15.100:0/2679610850 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6fc103550 msgr2=0x7ff6fc19f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:54.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.654+0000 7ff6faffd700 1 --2- 172.21.15.100:0/2679610850 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6fc103550 0x7ff6fc19f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:54.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.654+0000 7ff6faffd700 1 -- 172.21.15.100:0/2679610850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff6fc108460 msgr2=0x7ff6fc199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:54.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.654+0000 7ff6faffd700 1 --2- 172.21.15.100:0/2679610850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff6fc108460 0x7ff6fc199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:54.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.654+0000 7ff6faffd700 1 -- 172.21.15.100:0/2679610850 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6fc19a010 con 0x7ff6fc103ef0 2024-05-10T11:30:54.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.654+0000 7ff6fbfff700 1 --2- 172.21.15.100:0/2679610850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff6fc108460 0x7ff6fc199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:30:54.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.654+0000 7ff6faffd700 1 --2- 172.21.15.100:0/2679610850 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6fc103ef0 0x7ff6fc19fbf0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7ff6f000e9c0 tx=0x7ff6f000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:54.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.654+0000 7ff6f8ff9700 1 -- 172.21.15.100:0/2679610850 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f000cbd0 con 0x7ff6fc103ef0 2024-05-10T11:30:54.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.654+0000 7ff701ce0700 1 -- 172.21.15.100:0/2679610850 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6fc19a2a0 con 0x7ff6fc103ef0 2024-05-10T11:30:54.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.654+0000 7ff6f8ff9700 1 -- 172.21.15.100:0/2679610850 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff6f000ee60 con 0x7ff6fc103ef0 2024-05-10T11:30:54.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.654+0000 7ff6f8ff9700 1 -- 172.21.15.100:0/2679610850 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f00054b0 con 0x7ff6fc103ef0 2024-05-10T11:30:54.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.654+0000 7ff701ce0700 1 -- 172.21.15.100:0/2679610850 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6fc1a6510 con 0x7ff6fc103ef0 2024-05-10T11:30:54.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.657+0000 7ff6f8ff9700 1 -- 172.21.15.100:0/2679610850 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7ff6f0004750 con 0x7ff6fc103ef0 2024-05-10T11:30:54.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.657+0000 7ff6f8ff9700 1 --2- 172.21.15.100:0/2679610850 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff6e40b2880 0x7ff6e40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:54.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.657+0000 7ff6fb7fe700 1 --2- 172.21.15.100:0/2679610850 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff6e40b2880 0x7ff6e40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:54.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.657+0000 7ff6f8ff9700 1 -- 172.21.15.100:0/2679610850 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7ff6f00e2840 con 0x7ff6fc103ef0 2024-05-10T11:30:54.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.658+0000 7ff70c573b80 1 Processor -- start 2024-05-10T11:30:54.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.658+0000 7ff6fb7fe700 1 --2- 172.21.15.100:0/2679610850 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff6e40b2880 0x7ff6e40b4d40 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc104730 tx=0x7ff6ec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:54.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.658+0000 7ff70c573b80 1 -- start start 2024-05-10T11:30:54.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.658+0000 7ff70c573b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb30148960 0x55eb30148d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:54.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.658+0000 7ff70c573b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb30149280 0x55eb3014cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:54.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.658+0000 7ff70c573b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eb3014d0c0 0x55eb3014d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:54.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.658+0000 7ff70c573b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55eb300968e0 con 0x55eb30149280 2024-05-10T11:30:54.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.658+0000 7ff70c573b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55eb30035840 con 0x55eb3014d0c0 2024-05-10T11:30:54.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.658+0000 7ff70c573b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55eb300928a0 con 0x55eb30148960 2024-05-10T11:30:54.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.658+0000 7ff6faffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb30149280 0x55eb3014cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:54.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.659+0000 7ff6faffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb30149280 0x55eb3014cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43678/0 (socket says 172.21.15.100:43678) 2024-05-10T11:30:54.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.659+0000 7ff6faffd700 1 -- 172.21.15.100:0/1023713043 learned_addr learned my addr 172.21.15.100:0/1023713043 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:30:54.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.659+0000 7ff6fb7fe700 1 --2- 172.21.15.100:0/1023713043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb30148960 0x55eb30148d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:54.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.659+0000 7ff6fbfff700 1 --2- 172.21.15.100:0/1023713043 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eb3014d0c0 0x55eb3014d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:54.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.659+0000 7ff6fb7fe700 1 -- 172.21.15.100:0/1023713043 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eb3014d0c0 msgr2=0x55eb3014d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:54.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.659+0000 7ff6fb7fe700 1 --2- 172.21.15.100:0/1023713043 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eb3014d0c0 0x55eb3014d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:54.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.659+0000 7ff6fb7fe700 1 -- 172.21.15.100:0/1023713043 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb30149280 msgr2=0x55eb3014cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:54.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.659+0000 7ff6fb7fe700 1 --2- 172.21.15.100:0/1023713043 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb30149280 0x55eb3014cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:54.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.659+0000 7ff6fb7fe700 1 -- 172.21.15.100:0/1023713043 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb2ffe3950 con 0x55eb30148960 2024-05-10T11:30:54.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.659+0000 7ff6fb7fe700 1 --2- 172.21.15.100:0/1023713043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb30148960 0x55eb30148d40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec009fd0 tx=0x7ff6ec014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26a991a7728d7be0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:54.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.659+0000 7ff6e1ffb700 1 -- 172.21.15.100:0/1023713043 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ec004020 con 0x55eb30148960 2024-05-10T11:30:54.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.659+0000 7ff6e1ffb700 1 -- 172.21.15.100:0/1023713043 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff6ec00e810 con 0x55eb30148960 2024-05-10T11:30:54.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.659+0000 7ff6e1ffb700 1 -- 172.21.15.100:0/1023713043 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ec003990 con 0x55eb30148960 2024-05-10T11:30:54.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.660+0000 7ff70c573b80 1 -- 172.21.15.100:0/1023713043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb30148960 msgr2=0x55eb30148d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:54.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.660+0000 7ff70c573b80 1 --2- 172.21.15.100:0/1023713043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb30148960 0x55eb30148d40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec009fd0 tx=0x7ff6ec014000 comp rx=0 tx=0).stop 2024-05-10T11:30:54.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.660+0000 7ff70c573b80 1 -- 172.21.15.100:0/1023713043 shutdown_connections 2024-05-10T11:30:54.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.660+0000 7ff70c573b80 1 --2- 172.21.15.100:0/1023713043 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eb3014d0c0 0x55eb3014d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:54.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.660+0000 7ff70c573b80 1 --2- 172.21.15.100:0/1023713043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb30148960 0x55eb30148d40 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:54.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.660+0000 7ff70c573b80 1 --2- 172.21.15.100:0/1023713043 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb30149280 0x55eb3014cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:54.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.660+0000 7ff70c573b80 1 -- 172.21.15.100:0/1023713043 >> 172.21.15.100:0/1023713043 conn(0x55eb30070180 msgr2=0x55eb3013ecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:54.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.660+0000 7ff70c573b80 1 -- 172.21.15.100:0/1023713043 shutdown_connections 2024-05-10T11:30:54.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.660+0000 7ff70c573b80 1 -- 172.21.15.100:0/1023713043 wait complete. 2024-05-10T11:30:54.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.660+0000 7ff70c573b80 1 -- start start 2024-05-10T11:30:54.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.661+0000 7ff70c573b80 1 Processor -- start 2024-05-10T11:30:54.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.661+0000 7ff70c573b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb30148960 0x55eb301544b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:54.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.661+0000 7ff70c573b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb30149280 0x55eb301518a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:54.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.661+0000 7ff70c573b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eb3014d0c0 0x55eb30151de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:54.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.661+0000 7ff70c573b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55eb301552f0 con 0x55eb30149280 2024-05-10T11:30:54.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.661+0000 7ff70c573b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55eb2fffccb0 con 0x55eb3014d0c0 2024-05-10T11:30:54.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.661+0000 7ff70c573b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55eb30155470 con 0x55eb30148960 2024-05-10T11:30:54.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.661+0000 7ff70c573b80 10 client.0 _ll_register_callbacks cb 0x55eb3002e9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:30:54.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.661+0000 7ff6fb7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb30148960 0x55eb301544b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:54.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.661+0000 7ff6fb7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb30148960 0x55eb301544b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:36272/0 (socket says 172.21.15.100:36272) 2024-05-10T11:30:54.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.661+0000 7ff6fb7fe700 1 -- 172.21.15.100:0/2437865715 learned_addr learned my addr 172.21.15.100:0/2437865715 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:30:54.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.661+0000 7ff6faffd700 1 --2- 172.21.15.100:0/2437865715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb30149280 0x55eb301518a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:54.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.661+0000 7ff6fbfff700 1 --2- 172.21.15.100:0/2437865715 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eb3014d0c0 0x55eb30151de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:54.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.662+0000 7ff6fb7fe700 1 -- 172.21.15.100:0/2437865715 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eb3014d0c0 msgr2=0x55eb30151de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:54.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.662+0000 7ff6fb7fe700 1 --2- 172.21.15.100:0/2437865715 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eb3014d0c0 0x55eb30151de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:54.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.662+0000 7ff6fb7fe700 1 -- 172.21.15.100:0/2437865715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb30149280 msgr2=0x55eb301518a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:54.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.662+0000 7ff6fb7fe700 1 --2- 172.21.15.100:0/2437865715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb30149280 0x55eb301518a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:54.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.662+0000 7ff6fb7fe700 1 -- 172.21.15.100:0/2437865715 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb30032710 con 0x55eb30148960 2024-05-10T11:30:54.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.662+0000 7ff6fb7fe700 1 --2- 172.21.15.100:0/2437865715 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb30148960 0x55eb301544b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec00eb90 tx=0x7ff6ec00ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7aed72858a3f316d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:54.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.662+0000 7ff6daffd700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:30:54.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.662+0000 7ff6daffd700 1 -- 172.21.15.100:0/2437865715 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ec026890 con 0x55eb30148960 2024-05-10T11:30:54.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.662+0000 7ff6daffd700 1 -- 172.21.15.100:0/2437865715 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7ff6ec026a30 con 0x55eb30148960 2024-05-10T11:30:54.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.662+0000 7ff70c573b80 1 -- 172.21.15.100:0/2437865715 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6ec00e810 con 0x55eb30148960 2024-05-10T11:30:54.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.662+0000 7ff6daffd700 1 -- 172.21.15.100:0/2437865715 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff6ec039050 con 0x55eb30148960 2024-05-10T11:30:54.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.662+0000 7ff70c573b80 10 client.27409 fetch_fsmap learned FSMap version 33 2024-05-10T11:30:54.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.662+0000 7ff70c573b80 1 -- 172.21.15.100:0/2437865715 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55eb3015fab0 con 0x55eb30148960 2024-05-10T11:30:54.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.662+0000 7ff6daffd700 1 -- 172.21.15.100:0/2437865715 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff6ec00e810 con 0x55eb30148960 2024-05-10T11:30:54.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.663+0000 7ff6daffd700 1 -- 172.21.15.100:0/2437865715 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7ff6ec02e020 con 0x55eb30148960 2024-05-10T11:30:54.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.663+0000 7ff70c573b80 10 client.27409 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T11:30:54.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.663+0000 7ff70c573b80 10 client.27409 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:30:54.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.663+0000 7ff70c573b80 20 client.27409 populate_metadata read hostname 'smithi100' 2024-05-10T11:30:54.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.663+0000 7ff70c573b80 1 --2- 172.21.15.100:0/2437865715 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55eb30161f80 0x55eb30164370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:54.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.663+0000 7ff70c573b80 4 client.27409 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:30:54.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.663+0000 7ff70c573b80 1 -- 172.21.15.100:0/2437865715 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb301106d0 con 0x55eb30161f80 2024-05-10T11:30:54.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.663+0000 7ff6faffd700 1 --2- 172.21.15.100:0/2437865715 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55eb30161f80 0x55eb30164370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:54.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.663+0000 7ff6daffd700 10 client.27409 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:30:54.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.663+0000 7ff6faffd700 1 --2- 172.21.15.100:0/2437865715 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55eb30161f80 0x55eb30164370 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a6f07bcd50f04ef server_cookie=71c879927a2ad8ce in_seq=0 out_seq=0 2024-05-10T11:30:54.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.664+0000 7ff6daffd700 1 -- 172.21.15.100:0/2437865715 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55eb301106d0 con 0x55eb30161f80 2024-05-10T11:30:54.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.664+0000 7ff6daffd700 10 client.27409 handle_command_reply: tid=0 2024-05-10T11:30:54.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.664+0000 7ff70c573b80 1 client.27409 shutdown 2024-05-10T11:30:54.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.664+0000 7ff70c573b80 1 -- 172.21.15.100:0/2437865715 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb30148960 msgr2=0x55eb301544b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:54.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.664+0000 7ff70c573b80 1 --2- 172.21.15.100:0/2437865715 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb30148960 0x55eb301544b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec00eb90 tx=0x7ff6ec00ec70 comp rx=0 tx=0).stop 2024-05-10T11:30:54.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.664+0000 7ff70c573b80 1 -- 172.21.15.100:0/2437865715 shutdown_connections 2024-05-10T11:30:54.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.664+0000 7ff70c573b80 1 --2- 172.21.15.100:0/2437865715 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55eb30161f80 0x55eb30164370 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:54.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.664+0000 7ff70c573b80 1 --2- 172.21.15.100:0/2437865715 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eb3014d0c0 0x55eb30151de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:54.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.664+0000 7ff70c573b80 1 --2- 172.21.15.100:0/2437865715 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb30148960 0x55eb301544b0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:54.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.664+0000 7ff70c573b80 1 --2- 172.21.15.100:0/2437865715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb30149280 0x55eb301518a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:54.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.664+0000 7ff70c573b80 1 -- 172.21.15.100:0/2437865715 >> 172.21.15.100:0/2437865715 conn(0x55eb30070180 msgr2=0x55eb301397a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:54.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.664+0000 7ff6daffd700 0 client.27409 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:30:54.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.665+0000 7ff70c573b80 1 -- 172.21.15.100:0/2437865715 shutdown_connections 2024-05-10T11:30:54.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.665+0000 7ff70c573b80 1 -- 172.21.15.100:0/2437865715 wait complete. 2024-05-10T11:30:54.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.665+0000 7ff70c573b80 20 client.27409 trim_cache size 0 max 16384 2024-05-10T11:30:54.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.688+0000 7ff70c573b80 1 Processor -- start 2024-05-10T11:30:54.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.688+0000 7ff70c573b80 1 -- start start 2024-05-10T11:30:54.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.688+0000 7ff70c573b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb30148960 0x55eb3015e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:54.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.688+0000 7ff70c573b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eb30149280 0x55eb3014f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:54.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.688+0000 7ff70c573b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb3014d0c0 0x55eb30143360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:54.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.688+0000 7ff70c573b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff6fc1af780 con 0x55eb30148960 2024-05-10T11:30:54.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.688+0000 7ff70c573b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff6fc1ad010 con 0x55eb30149280 2024-05-10T11:30:54.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.688+0000 7ff70c573b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff6fc1aae60 con 0x55eb3014d0c0 2024-05-10T11:30:54.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.688+0000 7ff6fb7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb30148960 0x55eb3015e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:54.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.688+0000 7ff6fb7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb30148960 0x55eb3015e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43684/0 (socket says 172.21.15.100:43684) 2024-05-10T11:30:54.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.688+0000 7ff6fb7fe700 1 -- 172.21.15.100:0/3734777950 learned_addr learned my addr 172.21.15.100:0/3734777950 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:30:54.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.688+0000 7ff6faffd700 1 --2- 172.21.15.100:0/3734777950 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eb30149280 0x55eb3014f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:54.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.688+0000 7ff6fbfff700 1 --2- 172.21.15.100:0/3734777950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb3014d0c0 0x55eb30143360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:54.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.688+0000 7ff6faffd700 1 -- 172.21.15.100:0/3734777950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb3014d0c0 msgr2=0x55eb30143360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:54.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.688+0000 7ff6faffd700 1 --2- 172.21.15.100:0/3734777950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb3014d0c0 0x55eb30143360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:54.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.688+0000 7ff6faffd700 1 -- 172.21.15.100:0/3734777950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb30148960 msgr2=0x55eb3015e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:54.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.688+0000 7ff6faffd700 1 --2- 172.21.15.100:0/3734777950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb30148960 0x55eb3015e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:54.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.688+0000 7ff6faffd700 1 -- 172.21.15.100:0/3734777950 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb3014fe10 con 0x55eb30149280 2024-05-10T11:30:54.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.688+0000 7ff6fb7fe700 1 --2- 172.21.15.100:0/3734777950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb30148960 0x55eb3015e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:30:54.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.689+0000 7ff6faffd700 1 --2- 172.21.15.100:0/3734777950 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eb30149280 0x55eb3014f370 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff6f00a6d70 tx=0x7ff6f00dfeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8198becea001740 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:54.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.689+0000 7ff6e17fa700 1 -- 172.21.15.100:0/3734777950 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f00a9610 con 0x55eb30149280 2024-05-10T11:30:54.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.689+0000 7ff6e17fa700 1 -- 172.21.15.100:0/3734777950 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff6f008f070 con 0x55eb30149280 2024-05-10T11:30:54.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.689+0000 7ff6e17fa700 1 -- 172.21.15.100:0/3734777950 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f0005690 con 0x55eb30149280 2024-05-10T11:30:54.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.689+0000 7ff70c573b80 1 -- 172.21.15.100:0/3734777950 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eb30149280 msgr2=0x55eb3014f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:54.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.689+0000 7ff70c573b80 1 --2- 172.21.15.100:0/3734777950 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eb30149280 0x55eb3014f370 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff6f00a6d70 tx=0x7ff6f00dfeb0 comp rx=0 tx=0).stop 2024-05-10T11:30:54.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.690+0000 7ff70c573b80 1 -- 172.21.15.100:0/3734777950 shutdown_connections 2024-05-10T11:30:54.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.690+0000 7ff70c573b80 1 --2- 172.21.15.100:0/3734777950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb3014d0c0 0x55eb30143360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:54.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.690+0000 7ff70c573b80 1 --2- 172.21.15.100:0/3734777950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb30148960 0x55eb3015e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:54.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.690+0000 7ff70c573b80 1 --2- 172.21.15.100:0/3734777950 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eb30149280 0x55eb3014f370 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:54.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.690+0000 7ff70c573b80 1 -- 172.21.15.100:0/3734777950 >> 172.21.15.100:0/3734777950 conn(0x55eb30070180 msgr2=0x55eb301397a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:54.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.690+0000 7ff70c573b80 1 -- 172.21.15.100:0/3734777950 shutdown_connections 2024-05-10T11:30:54.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.690+0000 7ff70c573b80 1 -- 172.21.15.100:0/3734777950 wait complete. 2024-05-10T11:30:54.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.690+0000 7ff70c573b80 1 -- start start 2024-05-10T11:30:54.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.690+0000 7ff70c573b80 1 Processor -- start 2024-05-10T11:30:54.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.690+0000 7ff70c573b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eb30148960 0x55eb3015aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:54.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.690+0000 7ff70c573b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb30149280 0x55eb301576d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:54.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.690+0000 7ff70c573b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb3014d0c0 0x55eb30157c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:54.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.690+0000 7ff70c573b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff6fc1ab4d0 con 0x55eb3014d0c0 2024-05-10T11:30:54.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.690+0000 7ff70c573b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff6fc1ac940 con 0x55eb30148960 2024-05-10T11:30:54.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.690+0000 7ff70c573b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff6fc1abba0 con 0x55eb30149280 2024-05-10T11:30:54.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.691+0000 7ff70c573b80 10 client.0 _ll_register_callbacks cb 0x55eb3002e9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:30:54.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.691+0000 7ff6fbfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb3014d0c0 0x55eb30157c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:54.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.691+0000 7ff6fb7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eb30148960 0x55eb3015aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:54.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.691+0000 7ff6fbfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb3014d0c0 0x55eb30157c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43696/0 (socket says 172.21.15.100:43696) 2024-05-10T11:30:54.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.691+0000 7ff6fbfff700 1 -- 172.21.15.100:0/4133899916 learned_addr learned my addr 172.21.15.100:0/4133899916 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:30:54.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.691+0000 7ff6faffd700 1 --2- 172.21.15.100:0/4133899916 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb30149280 0x55eb301576d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:54.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.691+0000 7ff6fbfff700 1 -- 172.21.15.100:0/4133899916 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb30149280 msgr2=0x55eb301576d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:54.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.691+0000 7ff6fbfff700 1 --2- 172.21.15.100:0/4133899916 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb30149280 0x55eb301576d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:54.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.691+0000 7ff6fbfff700 1 -- 172.21.15.100:0/4133899916 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eb30148960 msgr2=0x55eb3015aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:54.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.691+0000 7ff6fbfff700 1 --2- 172.21.15.100:0/4133899916 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eb30148960 0x55eb3015aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:54.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.691+0000 7ff6fbfff700 1 -- 172.21.15.100:0/4133899916 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb30158290 con 0x55eb3014d0c0 2024-05-10T11:30:54.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.691+0000 7ff6fb7fe700 1 --2- 172.21.15.100:0/4133899916 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eb30148960 0x55eb3015aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:30:54.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.691+0000 7ff6fbfff700 1 --2- 172.21.15.100:0/4133899916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb3014d0c0 0x55eb30157c10 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7ff6f4015000 tx=0x7ff6f4003d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6c3f2cb59e5be1c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:54.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.691+0000 7ff6daffd700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:30:54.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.691+0000 7ff6daffd700 1 -- 172.21.15.100:0/4133899916 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f40078f0 con 0x55eb3014d0c0 2024-05-10T11:30:54.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.691+0000 7ff6daffd700 1 -- 172.21.15.100:0/4133899916 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7ff6f4007a90 con 0x55eb3014d0c0 2024-05-10T11:30:54.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.691+0000 7ff70c573b80 1 -- 172.21.15.100:0/4133899916 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6f008f070 con 0x55eb3014d0c0 2024-05-10T11:30:54.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.691+0000 7ff6daffd700 1 -- 172.21.15.100:0/4133899916 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff6f4032050 con 0x55eb3014d0c0 2024-05-10T11:30:54.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.691+0000 7ff70c573b80 10 client.17973 fetch_fsmap learned FSMap version 33 2024-05-10T11:30:54.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.692+0000 7ff70c573b80 1 -- 172.21.15.100:0/4133899916 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55eb301cd440 con 0x55eb3014d0c0 2024-05-10T11:30:54.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.692+0000 7ff6daffd700 1 -- 172.21.15.100:0/4133899916 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff6f008f070 con 0x55eb3014d0c0 2024-05-10T11:30:54.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.692+0000 7ff6daffd700 1 -- 172.21.15.100:0/4133899916 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7ff6f402d030 con 0x55eb3014d0c0 2024-05-10T11:30:54.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.692+0000 7ff70c573b80 10 client.17973 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T11:30:54.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.692+0000 7ff70c573b80 10 client.17973 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:30:54.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.692+0000 7ff70c573b80 20 client.17973 populate_metadata read hostname 'smithi100' 2024-05-10T11:30:54.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.692+0000 7ff70c573b80 1 --2- 172.21.15.100:0/4133899916 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55eb301cdbd0 0x55eb301cfff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:54.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.692+0000 7ff70c573b80 4 client.17973 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T11:30:54.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.692+0000 7ff70c573b80 1 -- 172.21.15.100:0/4133899916 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55eb300a42c0 con 0x55eb301cdbd0 2024-05-10T11:30:54.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.692+0000 7ff6fb7fe700 1 --2- 172.21.15.100:0/4133899916 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55eb301cdbd0 0x55eb301cfff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:54.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.692+0000 7ff6fb7fe700 1 --2- 172.21.15.100:0/4133899916 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55eb301cdbd0 0x55eb301cfff0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d76378cc299f81a server_cookie=41bccc169d038a7 in_seq=0 out_seq=0 2024-05-10T11:30:54.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.692+0000 7ff6daffd700 10 client.17973 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:30:54.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.694+0000 7ff6daffd700 1 -- 172.21.15.100:0/4133899916 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55eb300a42c0 con 0x55eb301cdbd0 2024-05-10T11:30:54.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.694+0000 7ff6daffd700 10 client.17973 handle_command_reply: tid=0 2024-05-10T11:30:54.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.694+0000 7ff70c573b80 1 client.17973 shutdown 2024-05-10T11:30:54.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.694+0000 7ff70c573b80 1 -- 172.21.15.100:0/4133899916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb3014d0c0 msgr2=0x55eb30157c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:54.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.694+0000 7ff70c573b80 1 --2- 172.21.15.100:0/4133899916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb3014d0c0 0x55eb30157c10 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7ff6f4015000 tx=0x7ff6f4003d70 comp rx=0 tx=0).stop 2024-05-10T11:30:54.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.694+0000 7ff70c573b80 1 -- 172.21.15.100:0/4133899916 shutdown_connections 2024-05-10T11:30:54.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.694+0000 7ff70c573b80 1 --2- 172.21.15.100:0/4133899916 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55eb301cdbd0 0x55eb301cfff0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:54.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.695+0000 7ff70c573b80 1 --2- 172.21.15.100:0/4133899916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb3014d0c0 0x55eb30157c10 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:54.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.695+0000 7ff6daffd700 0 client.17973 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:30:54.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.695+0000 7ff70c573b80 1 --2- 172.21.15.100:0/4133899916 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eb30148960 0x55eb3015aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:54.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.695+0000 7ff70c573b80 1 --2- 172.21.15.100:0/4133899916 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb30149280 0x55eb301576d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:54.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.695+0000 7ff70c573b80 1 -- 172.21.15.100:0/4133899916 >> 172.21.15.100:0/4133899916 conn(0x55eb30070180 msgr2=0x55eb301397a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:54.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.695+0000 7ff70c573b80 1 -- 172.21.15.100:0/4133899916 shutdown_connections 2024-05-10T11:30:54.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.695+0000 7ff70c573b80 1 -- 172.21.15.100:0/4133899916 wait complete. 2024-05-10T11:30:54.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.695+0000 7ff70c573b80 20 client.17973 trim_cache size 0 max 16384 2024-05-10T11:30:54.700 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:30:54.700 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-05-10T11:30:54.700 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "c02b1dc4-c812-4960-adcb-77fc8319f22b", 2024-05-10T11:30:54.701 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-05-10T11:30:54.701 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:30:54.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.696+0000 7ff701ce0700 1 -- 172.21.15.100:0/2679610850 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff6e40b2880 msgr2=0x7ff6e40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:54.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.696+0000 7ff701ce0700 1 --2- 172.21.15.100:0/2679610850 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff6e40b2880 0x7ff6e40b4d40 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc104730 tx=0x7ff6ec009260 comp rx=0 tx=0).stop 2024-05-10T11:30:54.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.696+0000 7ff701ce0700 1 -- 172.21.15.100:0/2679610850 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6fc103ef0 msgr2=0x7ff6fc19fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:54.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.696+0000 7ff701ce0700 1 --2- 172.21.15.100:0/2679610850 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6fc103ef0 0x7ff6fc19fbf0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7ff6f000e9c0 tx=0x7ff6f000ecd0 comp rx=0 tx=0).stop 2024-05-10T11:30:54.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.696+0000 7ff701ce0700 1 -- 172.21.15.100:0/2679610850 shutdown_connections 2024-05-10T11:30:54.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.696+0000 7ff701ce0700 1 --2- 172.21.15.100:0/2679610850 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff6e40b2880 0x7ff6e40b4d40 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:54.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.696+0000 7ff701ce0700 1 --2- 172.21.15.100:0/2679610850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff6fc108460 0x7ff6fc199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:54.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.696+0000 7ff701ce0700 1 --2- 172.21.15.100:0/2679610850 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6fc103550 0x7ff6fc19f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:54.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.696+0000 7ff701ce0700 1 --2- 172.21.15.100:0/2679610850 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6fc103ef0 0x7ff6fc19fbf0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:54.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.696+0000 7ff701ce0700 1 -- 172.21.15.100:0/2679610850 >> 172.21.15.100:0/2679610850 conn(0x7ff6fc07a600 msgr2=0x7ff6fc1016a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:54.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.696+0000 7ff701ce0700 1 -- 172.21.15.100:0/2679610850 shutdown_connections 2024-05-10T11:30:54.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:54.696+0000 7ff701ce0700 1 -- 172.21.15.100:0/2679610850 wait complete. 2024-05-10T11:30:54.983 INFO:tasks.workunit.client.0.smithi100.stdout: 1 330293 2.71 MB/sec execute 529 sec latency 30.004 ms 2024-05-10T11:30:55.424 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T11:30:55.682 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:30:55.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:30:55 smithi100 ceph-mon[30296]: pgmap v797: 129 pgs: 129 active+clean; 128 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 1.2 MiB/s wr, 143 op/s 2024-05-10T11:30:55.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:30:55 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:30:55.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:30:55 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:30:55.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:30:55 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:30:55.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:30:55 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:30:55.804 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:30:55 smithi178 ceph-mon[33359]: pgmap v797: 129 pgs: 129 active+clean; 128 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 1.2 MiB/s wr, 143 op/s 2024-05-10T11:30:55.804 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:30:55 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:30:55.804 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:30:55 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:30:55.804 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:30:55 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:30:55.804 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:30:55 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:30:55.984 INFO:tasks.workunit.client.0.smithi100.stdout: 1 330596 2.71 MB/sec execute 530 sec latency 15.563 ms 2024-05-10T11:30:56.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:30:55 smithi188 ceph-mon[34075]: pgmap v797: 129 pgs: 129 active+clean; 128 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 1.2 MiB/s wr, 143 op/s 2024-05-10T11:30:56.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:30:55 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:30:56.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:30:55 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:30:56.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:30:55 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:30:56.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:30:55 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:30:56.916 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:30:56 smithi100 ceph-mon[30296]: scrub queued for path: / 2024-05-10T11:30:56.916 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:30:56 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:30:56.916 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:30:56 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:30:56.916 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:30:56 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:30:56.916 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:30:56 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:30:56.916 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:30:56 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:30:56.916 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:30:56 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:30:56.916 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:30:56 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:30:56.917 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:30:56 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:30:56.917 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:30:56 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:30:56.984 INFO:tasks.workunit.client.0.smithi100.stdout: 1 331061 2.71 MB/sec execute 531 sec latency 15.565 ms 2024-05-10T11:30:57.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:30:56 smithi178 ceph-mon[33359]: scrub queued for path: / 2024-05-10T11:30:57.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:30:56 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:30:57.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:30:56 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:30:57.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:30:56 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:30:57.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:30:56 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:30:57.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:30:56 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:30:57.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:30:56 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:30:57.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:30:56 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:30:57.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:30:56 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:30:57.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:30:56 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:30:57.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:30:56 smithi188 ceph-mon[34075]: scrub queued for path: / 2024-05-10T11:30:57.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:30:56 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:30:57.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:30:56 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:30:57.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:30:56 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:30:57.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:30:56 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:30:57.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:30:56 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:30:57.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:30:56 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:30:57.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:30:56 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:30:57.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:30:56 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:30:57.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:30:56 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:30:57.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:30:57 smithi100 ceph-mon[30296]: pgmap v798: 129 pgs: 129 active+clean; 127 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 1.3 MiB/s wr, 133 op/s 2024-05-10T11:30:57.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.492+0000 7f7602e26700 1 -- 172.21.15.100:0/2829962388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75fc1034f0 msgr2=0x7f75fc103970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:57.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.492+0000 7f7602e26700 1 --2- 172.21.15.100:0/2829962388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75fc1034f0 0x7f75fc103970 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f75e8009990 tx=0x7f75e8009ca0 comp rx=0 tx=0).stop 2024-05-10T11:30:57.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.492+0000 7f7602e26700 1 -- 172.21.15.100:0/2829962388 shutdown_connections 2024-05-10T11:30:57.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.492+0000 7f7602e26700 1 --2- 172.21.15.100:0/2829962388 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75fc103eb0 0x7f75fc110390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:57.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.492+0000 7f7602e26700 1 --2- 172.21.15.100:0/2829962388 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75fc109600 0x7f75fc1099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:57.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.492+0000 7f7602e26700 1 --2- 172.21.15.100:0/2829962388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75fc1034f0 0x7f75fc103970 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:57.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.492+0000 7f7602e26700 1 -- 172.21.15.100:0/2829962388 >> 172.21.15.100:0/2829962388 conn(0x7f75fc07a650 msgr2=0x7f75fc07aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:57.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.492+0000 7f7602e26700 1 -- 172.21.15.100:0/2829962388 shutdown_connections 2024-05-10T11:30:57.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.492+0000 7f7602e26700 1 -- 172.21.15.100:0/2829962388 wait complete. 2024-05-10T11:30:57.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.493+0000 7f7602e26700 1 Processor -- start 2024-05-10T11:30:57.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.493+0000 7f7602e26700 1 -- start start 2024-05-10T11:30:57.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.493+0000 7f7602e26700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75fc1034f0 0x7f75fc0757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:57.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.493+0000 7f7602e26700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75fc103eb0 0x7f75fc075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:57.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.493+0000 7f7602e26700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75fc109600 0x7f75fc076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:57.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.493+0000 7f7602e26700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f75fc113520 con 0x7f75fc103eb0 2024-05-10T11:30:57.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.493+0000 7f7602e26700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f75fc1133a0 con 0x7f75fc109600 2024-05-10T11:30:57.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.493+0000 7f7602e26700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f75fc1136a0 con 0x7f75fc1034f0 2024-05-10T11:30:57.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.493+0000 7f75fbfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75fc103eb0 0x7f75fc075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:57.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.493+0000 7f75fbfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75fc103eb0 0x7f75fc075ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49812/0 (socket says 172.21.15.100:49812) 2024-05-10T11:30:57.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.493+0000 7f75fbfff700 1 -- 172.21.15.100:0/4117374207 learned_addr learned my addr 172.21.15.100:0/4117374207 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:30:57.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.493+0000 7f76013c3700 1 --2- 172.21.15.100:0/4117374207 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75fc109600 0x7f75fc076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:57.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.493+0000 7f7600bc2700 1 --2- 172.21.15.100:0/4117374207 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75fc1034f0 0x7f75fc0757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:57.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.494+0000 7f75fbfff700 1 -- 172.21.15.100:0/4117374207 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75fc1034f0 msgr2=0x7f75fc0757a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:57.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.494+0000 7f75fbfff700 1 --2- 172.21.15.100:0/4117374207 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75fc1034f0 0x7f75fc0757a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:57.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.494+0000 7f75fbfff700 1 -- 172.21.15.100:0/4117374207 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75fc109600 msgr2=0x7f75fc076220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:57.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.494+0000 7f75fbfff700 1 --2- 172.21.15.100:0/4117374207 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75fc109600 0x7f75fc076220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:57.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.494+0000 7f75fbfff700 1 -- 172.21.15.100:0/4117374207 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75fc1a7f70 con 0x7f75fc103eb0 2024-05-10T11:30:57.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.494+0000 7f75fbfff700 1 --2- 172.21.15.100:0/4117374207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75fc103eb0 0x7f75fc075ce0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f75e8006010 tx=0x7f75e80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:57.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.494+0000 7f75f9ffb700 1 -- 172.21.15.100:0/4117374207 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e8027070 con 0x7f75fc103eb0 2024-05-10T11:30:57.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.494+0000 7f75f9ffb700 1 -- 172.21.15.100:0/4117374207 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f75e8022470 con 0x7f75fc103eb0 2024-05-10T11:30:57.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.494+0000 7f7602e26700 1 -- 172.21.15.100:0/4117374207 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75fc1a8200 con 0x7f75fc103eb0 2024-05-10T11:30:57.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.494+0000 7f75f9ffb700 1 -- 172.21.15.100:0/4117374207 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e8017650 con 0x7f75fc103eb0 2024-05-10T11:30:57.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.494+0000 7f7602e26700 1 -- 172.21.15.100:0/4117374207 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75fc1a8650 con 0x7f75fc103eb0 2024-05-10T11:30:57.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.497+0000 7f75f9ffb700 1 -- 172.21.15.100:0/4117374207 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f75e8009d10 con 0x7f75fc103eb0 2024-05-10T11:30:57.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.498+0000 7f75f9ffb700 1 --2- 172.21.15.100:0/4117374207 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f75ec0b2880 0x7f75ec0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:57.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.498+0000 7f7600bc2700 1 --2- 172.21.15.100:0/4117374207 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f75ec0b2880 0x7f75ec0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:57.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.498+0000 7f75f9ffb700 1 -- 172.21.15.100:0/4117374207 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f75e80e2fe0 con 0x7f75fc103eb0 2024-05-10T11:30:57.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.498+0000 7f7600bc2700 1 --2- 172.21.15.100:0/4117374207 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f75ec0b2880 0x7f75ec0b4d40 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f75f0009be0 tx=0x7f75f0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:57.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.499+0000 7f760d6b9b80 1 Processor -- start 2024-05-10T11:30:57.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.499+0000 7f760d6b9b80 1 -- start start 2024-05-10T11:30:57.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.499+0000 7f760d6b9b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5595930af4d0 0x5595930af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:57.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.499+0000 7f760d6b9b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595930afdf0 0x5595930b36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:57.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.499+0000 7f760d6b9b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595930b3c80 0x5595930b4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:57.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.499+0000 7f760d6b9b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x559593001860 con 0x5595930b3c80 2024-05-10T11:30:57.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.499+0000 7f760d6b9b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559592fa07c0 con 0x5595930af4d0 2024-05-10T11:30:57.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.499+0000 7f760d6b9b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x559592ffd820 con 0x5595930afdf0 2024-05-10T11:30:57.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.499+0000 7f76013c3700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595930b3c80 0x5595930b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:57.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.499+0000 7f75fbfff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595930afdf0 0x5595930b36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:57.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.499+0000 7f75fbfff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595930afdf0 0x5595930b36f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:55514/0 (socket says 172.21.15.100:55514) 2024-05-10T11:30:57.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.499+0000 7f7600bc2700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5595930af4d0 0x5595930af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:57.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.499+0000 7f76013c3700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595930b3c80 0x5595930b4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49820/0 (socket says 172.21.15.100:49820) 2024-05-10T11:30:57.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.499+0000 7f75fbfff700 1 -- 172.21.15.100:0/2010544510 learned_addr learned my addr 172.21.15.100:0/2010544510 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:30:57.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.499+0000 7f75fbfff700 1 -- 172.21.15.100:0/2010544510 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5595930af4d0 msgr2=0x5595930af8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:57.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.499+0000 7f75fbfff700 1 --2- 172.21.15.100:0/2010544510 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5595930af4d0 0x5595930af8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:57.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.499+0000 7f75fbfff700 1 -- 172.21.15.100:0/2010544510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595930b3c80 msgr2=0x5595930b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:57.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.499+0000 7f75fbfff700 1 --2- 172.21.15.100:0/2010544510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595930b3c80 0x5595930b4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:57.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.499+0000 7f75fbfff700 1 -- 172.21.15.100:0/2010544510 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559592f4e8d0 con 0x5595930afdf0 2024-05-10T11:30:57.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.499+0000 7f76013c3700 1 --2- 172.21.15.100:0/2010544510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595930b3c80 0x5595930b4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:30:57.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.500+0000 7f75fbfff700 1 --2- 172.21.15.100:0/2010544510 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595930afdf0 0x5595930b36f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f75e80a63b0 tx=0x7f75e80a9ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3459b97e8bdc1310 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:57.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.500+0000 7f75e6ffd700 1 -- 172.21.15.100:0/2010544510 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e80aa330 con 0x5595930afdf0 2024-05-10T11:30:57.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.500+0000 7f75e6ffd700 1 -- 172.21.15.100:0/2010544510 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f75e80ae6e0 con 0x5595930afdf0 2024-05-10T11:30:57.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.500+0000 7f75e6ffd700 1 -- 172.21.15.100:0/2010544510 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e80b2c70 con 0x5595930afdf0 2024-05-10T11:30:57.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.500+0000 7f760d6b9b80 1 -- 172.21.15.100:0/2010544510 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595930afdf0 msgr2=0x5595930b36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:57.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.500+0000 7f760d6b9b80 1 --2- 172.21.15.100:0/2010544510 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595930afdf0 0x5595930b36f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f75e80a63b0 tx=0x7f75e80a9ba0 comp rx=0 tx=0).stop 2024-05-10T11:30:57.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.501+0000 7f760d6b9b80 1 -- 172.21.15.100:0/2010544510 shutdown_connections 2024-05-10T11:30:57.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.501+0000 7f760d6b9b80 1 --2- 172.21.15.100:0/2010544510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595930b3c80 0x5595930b4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:57.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.501+0000 7f760d6b9b80 1 --2- 172.21.15.100:0/2010544510 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5595930af4d0 0x5595930af8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:57.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.501+0000 7f760d6b9b80 1 --2- 172.21.15.100:0/2010544510 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595930afdf0 0x5595930b36f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:57.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.501+0000 7f760d6b9b80 1 -- 172.21.15.100:0/2010544510 >> 172.21.15.100:0/2010544510 conn(0x559592fdb100 msgr2=0x5595930a9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:57.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.501+0000 7f760d6b9b80 1 -- 172.21.15.100:0/2010544510 shutdown_connections 2024-05-10T11:30:57.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.501+0000 7f760d6b9b80 1 -- 172.21.15.100:0/2010544510 wait complete. 2024-05-10T11:30:57.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.501+0000 7f760d6b9b80 1 -- start start 2024-05-10T11:30:57.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.501+0000 7f760d6b9b80 1 Processor -- start 2024-05-10T11:30:57.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.501+0000 7f760d6b9b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595930af4d0 0x5595930bb110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:57.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.501+0000 7f760d6b9b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595930afdf0 0x5595930b84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:57.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.501+0000 7f760d6b9b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5595930b3c80 0x5595930b89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:57.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.501+0000 7f760d6b9b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5595930bbc50 con 0x5595930af4d0 2024-05-10T11:30:57.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.501+0000 7f760d6b9b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559592f67c30 con 0x5595930b3c80 2024-05-10T11:30:57.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.501+0000 7f760d6b9b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5595930bbdd0 con 0x5595930afdf0 2024-05-10T11:30:57.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.501+0000 7f760d6b9b80 10 client.0 _ll_register_callbacks cb 0x559592f99920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:30:57.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.502+0000 7f7600bc2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595930af4d0 0x5595930bb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:57.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.502+0000 7f75fbfff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595930afdf0 0x5595930b84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:57.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.502+0000 7f75fbfff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595930afdf0 0x5595930b84b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:55518/0 (socket says 172.21.15.100:55518) 2024-05-10T11:30:57.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.502+0000 7f75fbfff700 1 -- 172.21.15.100:0/634355664 learned_addr learned my addr 172.21.15.100:0/634355664 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:30:57.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.502+0000 7f76013c3700 1 --2- 172.21.15.100:0/634355664 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5595930b3c80 0x5595930b89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:57.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.502+0000 7f75fbfff700 1 -- 172.21.15.100:0/634355664 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5595930b3c80 msgr2=0x5595930b89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:57.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.502+0000 7f75fbfff700 1 --2- 172.21.15.100:0/634355664 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5595930b3c80 0x5595930b89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:57.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.502+0000 7f75fbfff700 1 -- 172.21.15.100:0/634355664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595930af4d0 msgr2=0x5595930bb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:57.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.502+0000 7f75fbfff700 1 --2- 172.21.15.100:0/634355664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595930af4d0 0x5595930bb110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:57.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.502+0000 7f75fbfff700 1 -- 172.21.15.100:0/634355664 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559592f9d690 con 0x5595930afdf0 2024-05-10T11:30:57.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.502+0000 7f76013c3700 1 --2- 172.21.15.100:0/634355664 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5595930b3c80 0x5595930b89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:30:57.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.502+0000 7f7600bc2700 1 --2- 172.21.15.100:0/634355664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595930af4d0 0x5595930bb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:30:57.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.502+0000 7f75fbfff700 1 --2- 172.21.15.100:0/634355664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595930afdf0 0x5595930b84b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f75e80a9c10 tx=0x7f75e80aa7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4022fd3cf1b4715f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:57.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.502+0000 7f75dbfff700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:30:57.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.502+0000 7f75dbfff700 1 -- 172.21.15.100:0/634355664 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e80c4060 con 0x5595930afdf0 2024-05-10T11:30:57.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.502+0000 7f760d6b9b80 1 -- 172.21.15.100:0/634355664 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75e80ae6e0 con 0x5595930afdf0 2024-05-10T11:30:57.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.502+0000 7f75dbfff700 1 -- 172.21.15.100:0/634355664 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f75e80b2e80 con 0x5595930afdf0 2024-05-10T11:30:57.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.502+0000 7f75dbfff700 1 -- 172.21.15.100:0/634355664 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f75e80aa330 con 0x5595930afdf0 2024-05-10T11:30:57.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.502+0000 7f760d6b9b80 10 client.27427 fetch_fsmap learned FSMap version 33 2024-05-10T11:30:57.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.502+0000 7f760d6b9b80 1 -- 172.21.15.100:0/634355664 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5595930bbc50 con 0x5595930afdf0 2024-05-10T11:30:57.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.503+0000 7f75dbfff700 1 -- 172.21.15.100:0/634355664 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f75e80ae6e0 con 0x5595930afdf0 2024-05-10T11:30:57.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.503+0000 7f75dbfff700 1 -- 172.21.15.100:0/634355664 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f75e80a5ca0 con 0x5595930afdf0 2024-05-10T11:30:57.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.503+0000 7f760d6b9b80 10 client.27427 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T11:30:57.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.503+0000 7f760d6b9b80 10 client.27427 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:30:57.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.503+0000 7f760d6b9b80 20 client.27427 populate_metadata read hostname 'smithi100' 2024-05-10T11:30:57.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.503+0000 7f760d6b9b80 1 --2- 172.21.15.100:0/634355664 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5595930c8830 0x5595930cac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:57.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.503+0000 7f760d6b9b80 4 client.27427 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:30:57.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.503+0000 7f760d6b9b80 1 -- 172.21.15.100:0/634355664 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55959307b650 con 0x5595930c8830 2024-05-10T11:30:57.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.503+0000 7f7600bc2700 1 --2- 172.21.15.100:0/634355664 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5595930c8830 0x5595930cac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:57.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.503+0000 7f75dbfff700 10 client.27427 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:30:57.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.503+0000 7f7600bc2700 1 --2- 172.21.15.100:0/634355664 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5595930c8830 0x5595930cac20 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=251927ab1892a1c8 server_cookie=da8fd0dac575e0fc in_seq=0 out_seq=0 2024-05-10T11:30:57.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.504+0000 7f75dbfff700 1 -- 172.21.15.100:0/634355664 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55959307b650 con 0x5595930c8830 2024-05-10T11:30:57.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.504+0000 7f75dbfff700 10 client.27427 handle_command_reply: tid=0 2024-05-10T11:30:57.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.504+0000 7f760d6b9b80 1 client.27427 shutdown 2024-05-10T11:30:57.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.504+0000 7f760d6b9b80 1 -- 172.21.15.100:0/634355664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595930afdf0 msgr2=0x5595930b84b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:57.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.504+0000 7f760d6b9b80 1 --2- 172.21.15.100:0/634355664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595930afdf0 0x5595930b84b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f75e80a9c10 tx=0x7f75e80aa7c0 comp rx=0 tx=0).stop 2024-05-10T11:30:57.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.505+0000 7f760d6b9b80 1 -- 172.21.15.100:0/634355664 shutdown_connections 2024-05-10T11:30:57.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.505+0000 7f760d6b9b80 1 --2- 172.21.15.100:0/634355664 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5595930c8830 0x5595930cac20 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:57.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.505+0000 7f760d6b9b80 1 --2- 172.21.15.100:0/634355664 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5595930b3c80 0x5595930b89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:57.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.505+0000 7f760d6b9b80 1 --2- 172.21.15.100:0/634355664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595930af4d0 0x5595930bb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:57.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.505+0000 7f760d6b9b80 1 --2- 172.21.15.100:0/634355664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595930afdf0 0x5595930b84b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:57.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.505+0000 7f75dbfff700 0 client.27427 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:30:57.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.505+0000 7f760d6b9b80 1 -- 172.21.15.100:0/634355664 >> 172.21.15.100:0/634355664 conn(0x559592fdb100 msgr2=0x5595930a4720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:57.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.505+0000 7f760d6b9b80 1 -- 172.21.15.100:0/634355664 shutdown_connections 2024-05-10T11:30:57.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.505+0000 7f760d6b9b80 1 -- 172.21.15.100:0/634355664 wait complete. 2024-05-10T11:30:57.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.505+0000 7f760d6b9b80 20 client.27427 trim_cache size 0 max 16384 2024-05-10T11:30:57.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.527+0000 7f760d6b9b80 1 Processor -- start 2024-05-10T11:30:57.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.527+0000 7f760d6b9b80 1 -- start start 2024-05-10T11:30:57.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.527+0000 7f760d6b9b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595930af4d0 0x5595930b7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:57.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.527+0000 7f760d6b9b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595930afdf0 0x5595930c4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:57.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.527+0000 7f760d6b9b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5595930b3c80 0x5595930a45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:57.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.527+0000 7f760d6b9b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f75fc1b3a90 con 0x5595930af4d0 2024-05-10T11:30:57.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.527+0000 7f760d6b9b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f75fc1b1320 con 0x5595930b3c80 2024-05-10T11:30:57.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.527+0000 7f760d6b9b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f75fc1af170 con 0x5595930afdf0 2024-05-10T11:30:57.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.527+0000 7f7600bc2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595930af4d0 0x5595930b7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:57.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.527+0000 7f75fbfff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595930afdf0 0x5595930c4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:57.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.527+0000 7f7600bc2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595930af4d0 0x5595930b7410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49846/0 (socket says 172.21.15.100:49846) 2024-05-10T11:30:57.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.527+0000 7f7600bc2700 1 -- 172.21.15.100:0/2897669079 learned_addr learned my addr 172.21.15.100:0/2897669079 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:30:57.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.528+0000 7f76013c3700 1 --2- 172.21.15.100:0/2897669079 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5595930b3c80 0x5595930a45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:57.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.528+0000 7f7600bc2700 1 -- 172.21.15.100:0/2897669079 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595930afdf0 msgr2=0x5595930c4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:57.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.528+0000 7f7600bc2700 1 --2- 172.21.15.100:0/2897669079 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595930afdf0 0x5595930c4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:57.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.528+0000 7f7600bc2700 1 -- 172.21.15.100:0/2897669079 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5595930b3c80 msgr2=0x5595930a45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:57.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.528+0000 7f7600bc2700 1 --2- 172.21.15.100:0/2897669079 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5595930b3c80 0x5595930a45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:57.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.528+0000 7f7600bc2700 1 -- 172.21.15.100:0/2897669079 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595930cb2a0 con 0x5595930af4d0 2024-05-10T11:30:57.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.528+0000 7f75fbfff700 1 --2- 172.21.15.100:0/2897669079 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595930afdf0 0x5595930c4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:30:57.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.528+0000 7f76013c3700 1 --2- 172.21.15.100:0/2897669079 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5595930b3c80 0x5595930a45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:30:57.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.528+0000 7f7600bc2700 1 --2- 172.21.15.100:0/2897669079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595930af4d0 0x5595930b7410 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x5595930cb4d0 tx=0x7f75f0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed96f39cec4f883a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:57.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.528+0000 7f75e67fc700 1 -- 172.21.15.100:0/2897669079 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f0004460 con 0x5595930af4d0 2024-05-10T11:30:57.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.528+0000 7f75e67fc700 1 -- 172.21.15.100:0/2897669079 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f75f002e070 con 0x5595930af4d0 2024-05-10T11:30:57.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.528+0000 7f75e67fc700 1 -- 172.21.15.100:0/2897669079 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f001fb00 con 0x5595930af4d0 2024-05-10T11:30:57.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.529+0000 7f760d6b9b80 1 -- 172.21.15.100:0/2897669079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595930af4d0 msgr2=0x5595930b7410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:57.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.529+0000 7f760d6b9b80 1 --2- 172.21.15.100:0/2897669079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595930af4d0 0x5595930b7410 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x5595930cb4d0 tx=0x7f75f0014000 comp rx=0 tx=0).stop 2024-05-10T11:30:57.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.529+0000 7f760d6b9b80 1 -- 172.21.15.100:0/2897669079 shutdown_connections 2024-05-10T11:30:57.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.529+0000 7f760d6b9b80 1 --2- 172.21.15.100:0/2897669079 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5595930b3c80 0x5595930a45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:57.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.529+0000 7f760d6b9b80 1 --2- 172.21.15.100:0/2897669079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595930af4d0 0x5595930b7410 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:57.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.529+0000 7f760d6b9b80 1 --2- 172.21.15.100:0/2897669079 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595930afdf0 0x5595930c4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:57.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.529+0000 7f760d6b9b80 1 -- 172.21.15.100:0/2897669079 >> 172.21.15.100:0/2897669079 conn(0x559592fdb100 msgr2=0x5595930a3e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:57.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.529+0000 7f760d6b9b80 1 -- 172.21.15.100:0/2897669079 shutdown_connections 2024-05-10T11:30:57.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.529+0000 7f760d6b9b80 1 -- 172.21.15.100:0/2897669079 wait complete. 2024-05-10T11:30:57.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.529+0000 7f760d6b9b80 1 -- start start 2024-05-10T11:30:57.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.529+0000 7f760d6b9b80 1 Processor -- start 2024-05-10T11:30:57.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.530+0000 7f760d6b9b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5595930af4d0 0x5595930c15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:57.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.530+0000 7f760d6b9b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595930afdf0 0x5595930bc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:57.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.530+0000 7f760d6b9b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595930b3c80 0x5595930bcf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:57.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.530+0000 7f760d6b9b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f75fc1af7e0 con 0x5595930b3c80 2024-05-10T11:30:57.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.530+0000 7f760d6b9b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f75fc1b0c50 con 0x5595930af4d0 2024-05-10T11:30:57.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.530+0000 7f760d6b9b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f75fc1afeb0 con 0x5595930afdf0 2024-05-10T11:30:57.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.530+0000 7f760d6b9b80 10 client.0 _ll_register_callbacks cb 0x559592f99920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:30:57.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.530+0000 7f76013c3700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595930b3c80 0x5595930bcf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:57.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.530+0000 7f7600bc2700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5595930af4d0 0x5595930c15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:57.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.530+0000 7f76013c3700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595930b3c80 0x5595930bcf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49852/0 (socket says 172.21.15.100:49852) 2024-05-10T11:30:57.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.530+0000 7f76013c3700 1 -- 172.21.15.100:0/1542538280 learned_addr learned my addr 172.21.15.100:0/1542538280 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:30:57.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.530+0000 7f75fbfff700 1 --2- 172.21.15.100:0/1542538280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595930afdf0 0x5595930bc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:57.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.530+0000 7f7600bc2700 1 -- 172.21.15.100:0/1542538280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595930afdf0 msgr2=0x5595930bc9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:57.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.530+0000 7f7600bc2700 1 --2- 172.21.15.100:0/1542538280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595930afdf0 0x5595930bc9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:57.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.530+0000 7f7600bc2700 1 -- 172.21.15.100:0/1542538280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595930b3c80 msgr2=0x5595930bcf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:30:57.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.530+0000 7f7600bc2700 1 --2- 172.21.15.100:0/1542538280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595930b3c80 0x5595930bcf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:57.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.530+0000 7f7600bc2700 1 -- 172.21.15.100:0/1542538280 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595930ab520 con 0x5595930af4d0 2024-05-10T11:30:57.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.530+0000 7f76013c3700 1 --2- 172.21.15.100:0/1542538280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595930b3c80 0x5595930bcf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:30:57.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.531+0000 7f7600bc2700 1 --2- 172.21.15.100:0/1542538280 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5595930af4d0 0x5595930c15f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x5595930cb4d0 tx=0x7f75f0004570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=593b7668b8c9577b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:30:57.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.531+0000 7f75dbfff700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:30:57.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.531+0000 7f75dbfff700 1 -- 172.21.15.100:0/1542538280 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f0005610 con 0x5595930af4d0 2024-05-10T11:30:57.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.531+0000 7f760d6b9b80 1 -- 172.21.15.100:0/1542538280 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75f002e070 con 0x5595930af4d0 2024-05-10T11:30:57.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.531+0000 7f75dbfff700 1 -- 172.21.15.100:0/1542538280 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f75f00293b0 con 0x5595930af4d0 2024-05-10T11:30:57.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.531+0000 7f75dbfff700 1 -- 172.21.15.100:0/1542538280 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f75f0044050 con 0x5595930af4d0 2024-05-10T11:30:57.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.531+0000 7f760d6b9b80 10 client.27344 fetch_fsmap learned FSMap version 33 2024-05-10T11:30:57.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.531+0000 7f760d6b9b80 1 -- 172.21.15.100:0/1542538280 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559593132fd0 con 0x5595930af4d0 2024-05-10T11:30:57.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.531+0000 7f75dbfff700 1 -- 172.21.15.100:0/1542538280 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f75f002e070 con 0x5595930af4d0 2024-05-10T11:30:57.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.531+0000 7f75dbfff700 1 -- 172.21.15.100:0/1542538280 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f75f001fc50 con 0x5595930af4d0 2024-05-10T11:30:57.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.531+0000 7f760d6b9b80 10 client.27344 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T11:30:57.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.531+0000 7f760d6b9b80 10 client.27344 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:30:57.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.531+0000 7f760d6b9b80 20 client.27344 populate_metadata read hostname 'smithi100' 2024-05-10T11:30:57.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.531+0000 7f760d6b9b80 1 --2- 172.21.15.100:0/1542538280 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x559593133690 0x559593135ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:30:57.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.531+0000 7f760d6b9b80 4 client.27344 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T11:30:57.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.531+0000 7f760d6b9b80 1 -- 172.21.15.100:0/1542538280 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55959300f240 con 0x559593133690 2024-05-10T11:30:57.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.532+0000 7f75fbfff700 1 --2- 172.21.15.100:0/1542538280 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x559593133690 0x559593135ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:30:57.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.532+0000 7f75fbfff700 1 --2- 172.21.15.100:0/1542538280 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x559593133690 0x559593135ab0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=428b9d1aadc14404 server_cookie=59840b46f1f1f9f1 in_seq=0 out_seq=0 2024-05-10T11:30:57.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.532+0000 7f75dbfff700 10 client.27344 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:30:57.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.532+0000 7f75dbfff700 1 -- 172.21.15.100:0/1542538280 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55959300f240 con 0x559593133690 2024-05-10T11:30:57.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.532+0000 7f75dbfff700 10 client.27344 handle_command_reply: tid=0 2024-05-10T11:30:57.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.532+0000 7f760d6b9b80 1 client.27344 shutdown 2024-05-10T11:30:57.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.532+0000 7f760d6b9b80 1 -- 172.21.15.100:0/1542538280 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5595930af4d0 msgr2=0x5595930c15f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:57.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.532+0000 7f760d6b9b80 1 --2- 172.21.15.100:0/1542538280 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5595930af4d0 0x5595930c15f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x5595930cb4d0 tx=0x7f75f0004570 comp rx=0 tx=0).stop 2024-05-10T11:30:57.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.532+0000 7f760d6b9b80 1 -- 172.21.15.100:0/1542538280 shutdown_connections 2024-05-10T11:30:57.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.532+0000 7f760d6b9b80 1 --2- 172.21.15.100:0/1542538280 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x559593133690 0x559593135ab0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:57.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.532+0000 7f760d6b9b80 1 --2- 172.21.15.100:0/1542538280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595930b3c80 0x5595930bcf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:57.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.532+0000 7f760d6b9b80 1 --2- 172.21.15.100:0/1542538280 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5595930af4d0 0x5595930c15f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:57.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.532+0000 7f760d6b9b80 1 --2- 172.21.15.100:0/1542538280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595930afdf0 0x5595930bc9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:57.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.532+0000 7f760d6b9b80 1 -- 172.21.15.100:0/1542538280 >> 172.21.15.100:0/1542538280 conn(0x559592fdb100 msgr2=0x5595930a4720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:57.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.532+0000 7f75dbfff700 0 client.27344 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:30:57.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.533+0000 7f760d6b9b80 1 -- 172.21.15.100:0/1542538280 shutdown_connections 2024-05-10T11:30:57.822 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:30:57.822 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T11:30:57.822 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-05-10T11:30:57.822 INFO:teuthology.orchestra.run.smithi100.stdout: "c02b1dc4-c812-4960-adcb-77fc8319f22b": { 2024-05-10T11:30:57.822 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-05-10T11:30:57.822 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "c02b1dc4-c812-4960-adcb-77fc8319f22b", 2024-05-10T11:30:57.822 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-05-10T11:30:57.822 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:30:57.822 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:30:57.822 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:30:57.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.533+0000 7f760d6b9b80 1 -- 172.21.15.100:0/1542538280 wait complete. 2024-05-10T11:30:57.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.533+0000 7f760d6b9b80 20 client.27344 trim_cache size 0 max 16384 2024-05-10T11:30:57.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.533+0000 7f7602e26700 1 -- 172.21.15.100:0/4117374207 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f75ec0b2880 msgr2=0x7f75ec0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:57.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.533+0000 7f7602e26700 1 --2- 172.21.15.100:0/4117374207 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f75ec0b2880 0x7f75ec0b4d40 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f75f0009be0 tx=0x7f75f0009260 comp rx=0 tx=0).stop 2024-05-10T11:30:57.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.533+0000 7f7602e26700 1 -- 172.21.15.100:0/4117374207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75fc103eb0 msgr2=0x7f75fc075ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:30:57.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.533+0000 7f7602e26700 1 --2- 172.21.15.100:0/4117374207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75fc103eb0 0x7f75fc075ce0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f75e8006010 tx=0x7f75e80047b0 comp rx=0 tx=0).stop 2024-05-10T11:30:57.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.534+0000 7f7602e26700 1 -- 172.21.15.100:0/4117374207 shutdown_connections 2024-05-10T11:30:57.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.534+0000 7f7602e26700 1 --2- 172.21.15.100:0/4117374207 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f75ec0b2880 0x7f75ec0b4d40 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:57.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.534+0000 7f7602e26700 1 --2- 172.21.15.100:0/4117374207 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75fc109600 0x7f75fc076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:57.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.534+0000 7f7602e26700 1 --2- 172.21.15.100:0/4117374207 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75fc1034f0 0x7f75fc0757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:57.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.534+0000 7f7602e26700 1 --2- 172.21.15.100:0/4117374207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75fc103eb0 0x7f75fc075ce0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:30:57.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.534+0000 7f7602e26700 1 -- 172.21.15.100:0/4117374207 >> 172.21.15.100:0/4117374207 conn(0x7f75fc07a650 msgr2=0x7f75fc1021e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:30:57.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.534+0000 7f7602e26700 1 -- 172.21.15.100:0/4117374207 shutdown_connections 2024-05-10T11:30:57.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:30:57.534+0000 7f7602e26700 1 -- 172.21.15.100:0/4117374207 wait complete. 2024-05-10T11:30:57.984 INFO:tasks.workunit.client.0.smithi100.stdout: 1 331577 2.71 MB/sec execute 532 sec latency 43.843 ms 2024-05-10T11:30:58.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:30:57 smithi178 ceph-mon[33359]: pgmap v798: 129 pgs: 129 active+clean; 127 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 1.3 MiB/s wr, 133 op/s 2024-05-10T11:30:58.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:30:57 smithi188 ceph-mon[34075]: pgmap v798: 129 pgs: 129 active+clean; 127 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 1.3 MiB/s wr, 133 op/s 2024-05-10T11:30:58.984 INFO:tasks.workunit.client.0.smithi100.stdout: 1 332185 2.70 MB/sec execute 533 sec latency 62.280 ms 2024-05-10T11:30:59.984 INFO:tasks.workunit.client.0.smithi100.stdout: 1 332503 2.70 MB/sec execute 534 sec latency 84.844 ms 2024-05-10T11:31:00.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:30:59 smithi178 ceph-mon[33359]: pgmap v799: 129 pgs: 129 active+clean; 128 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 1.2 MiB/s wr, 160 op/s 2024-05-10T11:31:00.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:30:59 smithi188 ceph-mon[34075]: pgmap v799: 129 pgs: 129 active+clean; 128 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 1.2 MiB/s wr, 160 op/s 2024-05-10T11:31:00.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:30:59 smithi100 ceph-mon[30296]: pgmap v799: 129 pgs: 129 active+clean; 128 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 1.2 MiB/s wr, 160 op/s 2024-05-10T11:31:00.858 INFO:tasks.cephfs.filesystem:scrub status for tag:c02b1dc4-c812-4960-adcb-77fc8319f22b - {'path': '/', 'tag': 'c02b1dc4-c812-4960-adcb-77fc8319f22b', 'options': 'recursive,force'} 2024-05-10T11:31:00.984 INFO:tasks.workunit.client.0.smithi100.stdout: 1 333201 2.71 MB/sec execute 535 sec latency 42.422 ms 2024-05-10T11:31:01.984 INFO:tasks.workunit.client.0.smithi100.stdout: 1 333662 2.71 MB/sec execute 536 sec latency 14.821 ms 2024-05-10T11:31:02.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:31:01 smithi178 ceph-mon[33359]: pgmap v800: 129 pgs: 129 active+clean; 127 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 1.0 MiB/s wr, 131 op/s 2024-05-10T11:31:02.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:31:01 smithi188 ceph-mon[34075]: pgmap v800: 129 pgs: 129 active+clean; 127 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 1.0 MiB/s wr, 131 op/s 2024-05-10T11:31:02.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:31:01 smithi100 ceph-mon[30296]: pgmap v800: 129 pgs: 129 active+clean; 127 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 1.0 MiB/s wr, 131 op/s 2024-05-10T11:31:02.984 INFO:tasks.workunit.client.0.smithi100.stdout: 1 333884 2.71 MB/sec execute 537 sec latency 13.787 ms 2024-05-10T11:31:03.984 INFO:tasks.workunit.client.0.smithi100.stdout: 1 334219 2.70 MB/sec execute 538 sec latency 16.821 ms 2024-05-10T11:31:04.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:31:03 smithi178 ceph-mon[33359]: pgmap v801: 129 pgs: 129 active+clean; 127 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 1.2 MiB/s wr, 151 op/s 2024-05-10T11:31:04.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:31:03 smithi188 ceph-mon[34075]: pgmap v801: 129 pgs: 129 active+clean; 127 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 1.2 MiB/s wr, 151 op/s 2024-05-10T11:31:04.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:31:03 smithi100 ceph-mon[30296]: pgmap v801: 129 pgs: 129 active+clean; 127 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 1.2 MiB/s wr, 151 op/s 2024-05-10T11:31:04.984 INFO:tasks.workunit.client.0.smithi100.stdout: 1 334664 2.70 MB/sec execute 539 sec latency 19.304 ms 2024-05-10T11:31:05.236 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.233+0000 7ff5973c9640 1 -- 172.21.15.100:0/615651965 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5901045c0 msgr2=0x7ff59010ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:05.236 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.233+0000 7ff5973c9640 1 --2- 172.21.15.100:0/615651965 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5901045c0 0x7ff59010ae50 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7ff588009ca0 tx=0x7ff58802a8c0 comp rx=0 tx=0).stop 2024-05-10T11:31:05.236 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.233+0000 7ff5973c9640 1 -- 172.21.15.100:0/615651965 shutdown_connections 2024-05-10T11:31:05.236 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.233+0000 7ff5973c9640 1 --2- 172.21.15.100:0/615651965 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5901045c0 0x7ff59010ae50 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:05.236 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.233+0000 7ff5973c9640 1 --2- 172.21.15.100:0/615651965 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff590103c00 0x7ff590104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:05.236 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.233+0000 7ff5973c9640 1 --2- 172.21.15.100:0/615651965 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff590102a10 0x7ff590102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:05.236 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.233+0000 7ff5973c9640 1 -- 172.21.15.100:0/615651965 >> 172.21.15.100:0/615651965 conn(0x7ff5900fdfa0 msgr2=0x7ff5901003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:31:05.236 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.233+0000 7ff5973c9640 1 -- 172.21.15.100:0/615651965 shutdown_connections 2024-05-10T11:31:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.233+0000 7ff5973c9640 1 -- 172.21.15.100:0/615651965 wait complete. 2024-05-10T11:31:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.234+0000 7ff5973c9640 1 Processor -- start 2024-05-10T11:31:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.234+0000 7ff5973c9640 1 -- start start 2024-05-10T11:31:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.234+0000 7ff5973c9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff590102a10 0x7ff590198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.234+0000 7ff5973c9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff590103c00 0x7ff590198fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.234+0000 7ff5973c9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5901045c0 0x7ff5901a0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.234+0000 7ff5973c9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff59010e440 con 0x7ff5901045c0 2024-05-10T11:31:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.234+0000 7ff5973c9640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff59010e2c0 con 0x7ff590102a10 2024-05-10T11:31:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.234+0000 7ff5973c9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff59010e5c0 con 0x7ff590103c00 2024-05-10T11:31:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.234+0000 7ff595966640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5901045c0 0x7ff5901a0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.234+0000 7ff595165640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff590102a10 0x7ff590198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.234+0000 7ff594964640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff590103c00 0x7ff590198fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.234+0000 7ff595165640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff590102a10 0x7ff590198a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:54220/0 (socket says 172.21.15.100:54220) 2024-05-10T11:31:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.234+0000 7ff595165640 1 -- 172.21.15.100:0/1804312297 learned_addr learned my addr 172.21.15.100:0/1804312297 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:31:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.235+0000 7ff595165640 1 -- 172.21.15.100:0/1804312297 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff590103c00 msgr2=0x7ff590198fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.235+0000 7ff595165640 1 --2- 172.21.15.100:0/1804312297 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff590103c00 0x7ff590198fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.235+0000 7ff595165640 1 -- 172.21.15.100:0/1804312297 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5901045c0 msgr2=0x7ff5901a0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.235+0000 7ff595165640 1 --2- 172.21.15.100:0/1804312297 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5901045c0 0x7ff5901a0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.235+0000 7ff595165640 1 -- 172.21.15.100:0/1804312297 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5901a06a0 con 0x7ff590102a10 2024-05-10T11:31:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.235+0000 7ff594964640 1 --2- 172.21.15.100:0/1804312297 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff590103c00 0x7ff590198fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:31:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.235+0000 7ff595165640 1 --2- 172.21.15.100:0/1804312297 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff590102a10 0x7ff590198a80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff58c00e9c0 tx=0x7ff58c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:31:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.235+0000 7ff5867fc640 1 -- 172.21.15.100:0/1804312297 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff58c00cd80 con 0x7ff590102a10 2024-05-10T11:31:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.235+0000 7ff5973c9640 1 -- 172.21.15.100:0/1804312297 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5901a0990 con 0x7ff590102a10 2024-05-10T11:31:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.235+0000 7ff5867fc640 1 -- 172.21.15.100:0/1804312297 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff58c0044b0 con 0x7ff590102a10 2024-05-10T11:31:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.235+0000 7ff5867fc640 1 -- 172.21.15.100:0/1804312297 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff58c010430 con 0x7ff590102a10 2024-05-10T11:31:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.235+0000 7ff5973c9640 1 -- 172.21.15.100:0/1804312297 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5901a0ee0 con 0x7ff590102a10 2024-05-10T11:31:05.241 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.238+0000 7ff59a1b6740 1 Processor -- start 2024-05-10T11:31:05.241 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.238+0000 7ff59a1b6740 1 -- start start 2024-05-10T11:31:05.241 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.238+0000 7ff5867fc640 1 -- 172.21.15.100:0/1804312297 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7ff58c003f00 con 0x7ff590102a10 2024-05-10T11:31:05.241 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.239+0000 7ff59a1b6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562221d8edb0 0x562221d8f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:05.241 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.239+0000 7ff59a1b6740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562221d8f7a0 0x562221d942a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:05.241 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.239+0000 7ff59a1b6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562221d94910 0x562221d96da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:05.242 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.239+0000 7ff59a1b6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x562221cf35d0 con 0x562221d94910 2024-05-10T11:31:05.242 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.239+0000 7ff59a1b6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562221cac890 con 0x562221d8edb0 2024-05-10T11:31:05.242 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.239+0000 7ff59a1b6740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x562221cef590 con 0x562221d8f7a0 2024-05-10T11:31:05.242 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.239+0000 7ff595165640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562221d8f7a0 0x562221d942a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:05.242 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.239+0000 7ff594964640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562221d94910 0x562221d96da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:05.242 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.239+0000 7ff595966640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562221d8edb0 0x562221d8f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:05.242 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.239+0000 7ff595165640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562221d8f7a0 0x562221d942a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:55560/0 (socket says 172.21.15.100:55560) 2024-05-10T11:31:05.242 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.239+0000 7ff595165640 1 -- 172.21.15.100:0/2859681686 learned_addr learned my addr 172.21.15.100:0/2859681686 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:31:05.242 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.239+0000 7ff594964640 1 -- 172.21.15.100:0/2859681686 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562221d8f7a0 msgr2=0x562221d942a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:05.242 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.239+0000 7ff594964640 1 --2- 172.21.15.100:0/2859681686 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562221d8f7a0 0x562221d942a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:05.242 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.239+0000 7ff594964640 1 -- 172.21.15.100:0/2859681686 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562221d8edb0 msgr2=0x562221d8f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:05.242 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.239+0000 7ff594964640 1 --2- 172.21.15.100:0/2859681686 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562221d8edb0 0x562221d8f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:05.243 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.239+0000 7ff594964640 1 -- 172.21.15.100:0/2859681686 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562221a05210 con 0x562221d94910 2024-05-10T11:31:05.243 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.240+0000 7ff594964640 1 --2- 172.21.15.100:0/2859681686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562221d94910 0x562221d96da0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7ff580009fa0 tx=0x7ff5800118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcf0ae3046ba07c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:31:05.243 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.240+0000 7ff5867fc640 1 --2- 172.21.15.100:0/1804312297 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff57c0b28a0 0x7ff57c0b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:05.243 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.240+0000 7ff57b7fe640 1 -- 172.21.15.100:0/2859681686 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff580004310 con 0x562221d94910 2024-05-10T11:31:05.243 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.240+0000 7ff57b7fe640 1 -- 172.21.15.100:0/2859681686 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff5800044b0 con 0x562221d94910 2024-05-10T11:31:05.243 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.240+0000 7ff57b7fe640 1 -- 172.21.15.100:0/2859681686 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5800035f0 con 0x562221d94910 2024-05-10T11:31:05.243 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.240+0000 7ff595966640 1 --2- 172.21.15.100:0/1804312297 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff57c0b28a0 0x7ff57c0b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:05.243 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.240+0000 7ff5867fc640 1 -- 172.21.15.100:0/1804312297 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7ff58c0e29c0 con 0x7ff590102a10 2024-05-10T11:31:05.243 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.241+0000 7ff59a1b6740 1 -- 172.21.15.100:0/2859681686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562221d94910 msgr2=0x562221d96da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:05.243 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.241+0000 7ff595966640 1 --2- 172.21.15.100:0/1804312297 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff57c0b28a0 0x7ff57c0b4d60 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff588006010 tx=0x7ff5880319a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:31:05.243 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.241+0000 7ff59a1b6740 1 --2- 172.21.15.100:0/2859681686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562221d94910 0x562221d96da0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7ff580009fa0 tx=0x7ff5800118d0 comp rx=0 tx=0).stop 2024-05-10T11:31:05.243 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.241+0000 7ff59a1b6740 1 -- 172.21.15.100:0/2859681686 shutdown_connections 2024-05-10T11:31:05.243 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.241+0000 7ff59a1b6740 1 --2- 172.21.15.100:0/2859681686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562221d94910 0x562221d96da0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:05.243 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.241+0000 7ff59a1b6740 1 --2- 172.21.15.100:0/2859681686 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562221d8f7a0 0x562221d942a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:05.244 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.241+0000 7ff59a1b6740 1 --2- 172.21.15.100:0/2859681686 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562221d8edb0 0x562221d8f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:05.244 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.241+0000 7ff59a1b6740 1 -- 172.21.15.100:0/2859681686 >> 172.21.15.100:0/2859681686 conn(0x562221d8a420 msgr2=0x562221d8a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:31:05.244 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.241+0000 7ff59a1b6740 1 -- 172.21.15.100:0/2859681686 shutdown_connections 2024-05-10T11:31:05.244 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.241+0000 7ff59a1b6740 1 -- 172.21.15.100:0/2859681686 wait complete. 2024-05-10T11:31:05.244 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.241+0000 7ff59a1b6740 1 -- start start 2024-05-10T11:31:05.244 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.241+0000 7ff59a1b6740 1 Processor -- start 2024-05-10T11:31:05.247 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.242+0000 7ff59a1b6740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562221d8edb0 0x562221d99030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:05.248 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.242+0000 7ff59a1b6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562221d8f7a0 0x562221d99570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:05.248 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.242+0000 7ff59a1b6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562221d94910 0x562221da05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:05.248 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.242+0000 7ff59a1b6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x562221d978c0 con 0x562221d8f7a0 2024-05-10T11:31:05.248 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.242+0000 7ff59a1b6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562221cb0370 con 0x562221d94910 2024-05-10T11:31:05.248 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.242+0000 7ff59a1b6740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x562221d97a40 con 0x562221d8edb0 2024-05-10T11:31:05.248 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.242+0000 7ff59a1b6740 10 client.0 _ll_register_callbacks cb 0x562221a04a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:31:05.248 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.242+0000 7ff594964640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562221d8f7a0 0x562221d99570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:05.248 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.242+0000 7ff594964640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562221d8f7a0 0x562221d99570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49900/0 (socket says 172.21.15.100:49900) 2024-05-10T11:31:05.248 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.242+0000 7ff595966640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562221d94910 0x562221da05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:05.248 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.242+0000 7ff595165640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562221d8edb0 0x562221d99030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:05.248 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.242+0000 7ff594964640 1 -- 172.21.15.100:0/2462715641 learned_addr learned my addr 172.21.15.100:0/2462715641 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:31:05.248 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.242+0000 7ff594964640 1 -- 172.21.15.100:0/2462715641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562221d8edb0 msgr2=0x562221d99030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:05.249 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.242+0000 7ff594964640 1 --2- 172.21.15.100:0/2462715641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562221d8edb0 0x562221d99030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:05.249 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.242+0000 7ff594964640 1 -- 172.21.15.100:0/2462715641 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562221d94910 msgr2=0x562221da05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:05.249 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.242+0000 7ff594964640 1 --2- 172.21.15.100:0/2462715641 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562221d94910 0x562221da05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:05.249 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.242+0000 7ff594964640 1 -- 172.21.15.100:0/2462715641 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562221ad1620 con 0x562221d8f7a0 2024-05-10T11:31:05.249 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.242+0000 7ff595165640 1 --2- 172.21.15.100:0/2462715641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562221d8edb0 0x562221d99030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:31:05.249 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.242+0000 7ff594964640 1 --2- 172.21.15.100:0/2462715641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562221d8f7a0 0x562221d99570 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7ff580009d60 tx=0x7ff5800035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fd15fd536910b0a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:31:05.249 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.242+0000 7ff578ff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:31:05.249 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.243+0000 7ff578ff9640 1 -- 172.21.15.100:0/2462715641 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff580027030 con 0x562221d8f7a0 2024-05-10T11:31:05.249 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.243+0000 7ff578ff9640 1 -- 172.21.15.100:0/2462715641 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7ff5800036e0 con 0x562221d8f7a0 2024-05-10T11:31:05.249 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.243+0000 7ff59a1b6740 1 -- 172.21.15.100:0/2462715641 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5800044b0 con 0x562221d8f7a0 2024-05-10T11:31:05.249 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.243+0000 7ff578ff9640 1 -- 172.21.15.100:0/2462715641 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff580034050 con 0x562221d8f7a0 2024-05-10T11:31:05.249 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.243+0000 7ff59a1b6740 10 client.18003 fetch_fsmap learned FSMap version 33 2024-05-10T11:31:05.249 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.243+0000 7ff59a1b6740 1 -- 172.21.15.100:0/2462715641 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x562221da3590 con 0x562221d8f7a0 2024-05-10T11:31:05.250 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.243+0000 7ff578ff9640 1 -- 172.21.15.100:0/2462715641 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff5800044b0 con 0x562221d8f7a0 2024-05-10T11:31:05.250 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.243+0000 7ff578ff9640 1 -- 172.21.15.100:0/2462715641 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7ff58000fb00 con 0x562221d8f7a0 2024-05-10T11:31:05.250 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.243+0000 7ff59a1b6740 10 client.18003 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T11:31:05.250 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.243+0000 7ff59a1b6740 10 client.18003 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:31:05.250 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.243+0000 7ff59a1b6740 20 client.18003 populate_metadata read hostname 'smithi100' 2024-05-10T11:31:05.250 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.243+0000 7ff59a1b6740 1 --2- 172.21.15.100:0/2462715641 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562221da4a40 0x562221da6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:05.250 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.243+0000 7ff59a1b6740 4 client.18003 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:31:05.250 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.243+0000 7ff59a1b6740 1 -- 172.21.15.100:0/2462715641 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562221d011e0 con 0x562221da4a40 2024-05-10T11:31:05.250 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.243+0000 7ff595165640 1 --2- 172.21.15.100:0/2462715641 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562221da4a40 0x562221da6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:05.250 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.244+0000 7ff595165640 1 --2- 172.21.15.100:0/2462715641 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562221da4a40 0x562221da6e70 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a5362f707c0c61e server_cookie=55c8d4f09105fd9c in_seq=0 out_seq=0 2024-05-10T11:31:05.250 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.244+0000 7ff578ff9640 10 client.18003 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:31:05.250 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.244+0000 7ff578ff9640 1 -- 172.21.15.100:0/2462715641 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562221d011e0 con 0x562221da4a40 2024-05-10T11:31:05.250 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.244+0000 7ff578ff9640 10 client.18003 handle_command_reply: tid=0 2024-05-10T11:31:05.251 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.244+0000 7ff59a1b6740 1 client.18003 shutdown 2024-05-10T11:31:05.251 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.245+0000 7ff59a1b6740 1 -- 172.21.15.100:0/2462715641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562221d8f7a0 msgr2=0x562221d99570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:05.251 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.245+0000 7ff59a1b6740 1 --2- 172.21.15.100:0/2462715641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562221d8f7a0 0x562221d99570 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7ff580009d60 tx=0x7ff5800035f0 comp rx=0 tx=0).stop 2024-05-10T11:31:05.251 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.245+0000 7ff59a1b6740 1 -- 172.21.15.100:0/2462715641 shutdown_connections 2024-05-10T11:31:05.251 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.245+0000 7ff59a1b6740 1 --2- 172.21.15.100:0/2462715641 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562221da4a40 0x562221da6e70 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:05.251 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.245+0000 7ff59a1b6740 1 --2- 172.21.15.100:0/2462715641 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562221d94910 0x562221da05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:05.251 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.245+0000 7ff59a1b6740 1 --2- 172.21.15.100:0/2462715641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562221d8f7a0 0x562221d99570 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:05.251 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.245+0000 7ff59a1b6740 1 --2- 172.21.15.100:0/2462715641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562221d8edb0 0x562221d99030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:05.251 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.245+0000 7ff59a1b6740 1 -- 172.21.15.100:0/2462715641 >> 172.21.15.100:0/2462715641 conn(0x562221d8a420 msgr2=0x562221d89580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:31:05.251 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.245+0000 7ff578ff9640 0 client.18003 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:31:05.251 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.245+0000 7ff59a1b6740 1 -- 172.21.15.100:0/2462715641 shutdown_connections 2024-05-10T11:31:05.251 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.245+0000 7ff59a1b6740 1 -- 172.21.15.100:0/2462715641 wait complete. 2024-05-10T11:31:05.251 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.245+0000 7ff59a1b6740 20 client.18003 trim_cache size 0 max 16384 2024-05-10T11:31:05.264 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.262+0000 7ff59a1b6740 1 Processor -- start 2024-05-10T11:31:05.264 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.262+0000 7ff59a1b6740 1 -- start start 2024-05-10T11:31:05.264 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.262+0000 7ff59a1b6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562221d8edb0 0x562221d96d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:05.264 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.262+0000 7ff59a1b6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562221d8f7a0 0x562221d99030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:05.264 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.262+0000 7ff59a1b6740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562221d94910 0x562221d99570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:05.264 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.262+0000 7ff59a1b6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff5901b4210 con 0x562221d8f7a0 2024-05-10T11:31:05.264 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.262+0000 7ff59a1b6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff5901b1ae0 con 0x562221d8edb0 2024-05-10T11:31:05.264 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.262+0000 7ff59a1b6740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff5901af930 con 0x562221d94910 2024-05-10T11:31:05.264 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.262+0000 7ff594964640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562221d8f7a0 0x562221d99030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:05.265 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.262+0000 7ff595165640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562221d8edb0 0x562221d96d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:05.265 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.262+0000 7ff594964640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562221d8f7a0 0x562221d99030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49906/0 (socket says 172.21.15.100:49906) 2024-05-10T11:31:05.265 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.262+0000 7ff594964640 1 -- 172.21.15.100:0/125569705 learned_addr learned my addr 172.21.15.100:0/125569705 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:31:05.265 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.262+0000 7ff595966640 1 --2- 172.21.15.100:0/125569705 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562221d94910 0x562221d99570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:05.265 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.262+0000 7ff595165640 1 -- 172.21.15.100:0/125569705 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562221d94910 msgr2=0x562221d99570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:05.265 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.262+0000 7ff595165640 1 --2- 172.21.15.100:0/125569705 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562221d94910 0x562221d99570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:05.265 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.262+0000 7ff595165640 1 -- 172.21.15.100:0/125569705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562221d8f7a0 msgr2=0x562221d99030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:05.265 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.263+0000 7ff595165640 1 --2- 172.21.15.100:0/125569705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562221d8f7a0 0x562221d99030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:05.265 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.263+0000 7ff595165640 1 -- 172.21.15.100:0/125569705 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562221d8a250 con 0x562221d8edb0 2024-05-10T11:31:05.265 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.263+0000 7ff594964640 1 --2- 172.21.15.100:0/125569705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562221d8f7a0 0x562221d99030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:31:05.265 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.263+0000 7ff595165640 1 --2- 172.21.15.100:0/125569705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562221d8edb0 0x562221d96d00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff58c0a9920 tx=0x7ff58c0e4e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca0f9489e186eda5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:31:05.265 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.263+0000 7ff57affd640 1 -- 172.21.15.100:0/125569705 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff58c0a99f0 con 0x562221d8edb0 2024-05-10T11:31:05.265 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.263+0000 7ff57affd640 1 -- 172.21.15.100:0/125569705 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff58c0b23c0 con 0x562221d8edb0 2024-05-10T11:31:05.265 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.263+0000 7ff57affd640 1 -- 172.21.15.100:0/125569705 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff58c0b2a80 con 0x562221d8edb0 2024-05-10T11:31:05.265 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.264+0000 7ff59a1b6740 1 -- 172.21.15.100:0/125569705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562221d8edb0 msgr2=0x562221d96d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:05.266 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.264+0000 7ff59a1b6740 1 --2- 172.21.15.100:0/125569705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562221d8edb0 0x562221d96d00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff58c0a9920 tx=0x7ff58c0e4e10 comp rx=0 tx=0).stop 2024-05-10T11:31:05.266 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.264+0000 7ff59a1b6740 1 -- 172.21.15.100:0/125569705 shutdown_connections 2024-05-10T11:31:05.266 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.264+0000 7ff59a1b6740 1 --2- 172.21.15.100:0/125569705 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562221d94910 0x562221d99570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:05.266 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.264+0000 7ff59a1b6740 1 --2- 172.21.15.100:0/125569705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562221d8f7a0 0x562221d99030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:05.266 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.264+0000 7ff59a1b6740 1 --2- 172.21.15.100:0/125569705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562221d8edb0 0x562221d96d00 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:05.266 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.264+0000 7ff59a1b6740 1 -- 172.21.15.100:0/125569705 >> 172.21.15.100:0/125569705 conn(0x562221d8a420 msgr2=0x562221d853e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:31:05.266 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.264+0000 7ff59a1b6740 1 -- 172.21.15.100:0/125569705 shutdown_connections 2024-05-10T11:31:05.266 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.264+0000 7ff59a1b6740 1 -- 172.21.15.100:0/125569705 wait complete. 2024-05-10T11:31:05.266 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.264+0000 7ff59a1b6740 1 -- start start 2024-05-10T11:31:05.266 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.264+0000 7ff59a1b6740 1 Processor -- start 2024-05-10T11:31:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.264+0000 7ff59a1b6740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562221d8edb0 0x562221d9ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.264+0000 7ff59a1b6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562221d8f7a0 0x562221d9d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.264+0000 7ff59a1b6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562221d94910 0x562221d9e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:05.270 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.264+0000 7ff59a1b6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff5901affa0 con 0x562221d8f7a0 2024-05-10T11:31:05.270 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.264+0000 7ff59a1b6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff5901b1410 con 0x562221d94910 2024-05-10T11:31:05.270 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.264+0000 7ff59a1b6740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff5901b0670 con 0x562221d8edb0 2024-05-10T11:31:05.270 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.264+0000 7ff59a1b6740 10 client.0 _ll_register_callbacks cb 0x7ff580005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:31:05.270 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.264+0000 7ff594964640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562221d8f7a0 0x562221d9d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:05.270 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.264+0000 7ff595165640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562221d8edb0 0x562221d9ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:05.270 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.264+0000 7ff595165640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562221d8edb0 0x562221d9ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:55596/0 (socket says 172.21.15.100:55596) 2024-05-10T11:31:05.270 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.264+0000 7ff595165640 1 -- 172.21.15.100:0/1345256051 learned_addr learned my addr 172.21.15.100:0/1345256051 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:31:05.270 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.265+0000 7ff595966640 1 --2- 172.21.15.100:0/1345256051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562221d94910 0x562221d9e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:05.270 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.265+0000 7ff595165640 1 -- 172.21.15.100:0/1345256051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562221d94910 msgr2=0x562221d9e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:05.270 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.265+0000 7ff595165640 1 --2- 172.21.15.100:0/1345256051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562221d94910 0x562221d9e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:05.270 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.265+0000 7ff595165640 1 -- 172.21.15.100:0/1345256051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562221d8f7a0 msgr2=0x562221d9d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:05.270 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.265+0000 7ff595165640 1 --2- 172.21.15.100:0/1345256051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562221d8f7a0 0x562221d9d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:05.270 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.265+0000 7ff595165640 1 -- 172.21.15.100:0/1345256051 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562221d9e940 con 0x562221d8edb0 2024-05-10T11:31:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.265+0000 7ff594964640 1 --2- 172.21.15.100:0/1345256051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562221d8f7a0 0x562221d9d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:31:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.265+0000 7ff595165640 1 --2- 172.21.15.100:0/1345256051 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562221d8edb0 0x562221d9ce00 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff58c0a6e90 tx=0x7ff58c000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fec574512f9d2b87 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:31:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.265+0000 7ff578ff9640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:31:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.265+0000 7ff578ff9640 1 -- 172.21.15.100:0/1345256051 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff58c0a99f0 con 0x562221d8edb0 2024-05-10T11:31:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.265+0000 7ff59a1b6740 1 -- 172.21.15.100:0/1345256051 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562221cfb050 con 0x562221d8edb0 2024-05-10T11:31:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.265+0000 7ff578ff9640 1 -- 172.21.15.100:0/1345256051 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7ff58c0a6210 con 0x562221d8edb0 2024-05-10T11:31:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.265+0000 7ff578ff9640 1 -- 172.21.15.100:0/1345256051 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff58c0a9c20 con 0x562221d8edb0 2024-05-10T11:31:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.265+0000 7ff59a1b6740 10 client.27451 fetch_fsmap learned FSMap version 33 2024-05-10T11:31:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.265+0000 7ff59a1b6740 1 -- 172.21.15.100:0/1345256051 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x562221d9fca0 con 0x562221d8edb0 2024-05-10T11:31:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.265+0000 7ff578ff9640 1 -- 172.21.15.100:0/1345256051 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x562221cfb050 con 0x562221d8edb0 2024-05-10T11:31:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.266+0000 7ff578ff9640 1 -- 172.21.15.100:0/1345256051 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7ff58c0c4110 con 0x562221d8edb0 2024-05-10T11:31:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.266+0000 7ff59a1b6740 10 client.27451 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T11:31:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.266+0000 7ff59a1b6740 10 client.27451 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:31:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.266+0000 7ff59a1b6740 20 client.27451 populate_metadata read hostname 'smithi100' 2024-05-10T11:31:05.272 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.266+0000 7ff59a1b6740 1 --2- 172.21.15.100:0/1345256051 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562221e0f770 0x562221e11b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:05.272 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.266+0000 7ff59a1b6740 4 client.27451 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:31:05.272 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.266+0000 7ff59a1b6740 1 -- 172.21.15.100:0/1345256051 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562221c765f0 con 0x562221e0f770 2024-05-10T11:31:05.272 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.266+0000 7ff594964640 1 --2- 172.21.15.100:0/1345256051 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562221e0f770 0x562221e11b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:05.272 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.266+0000 7ff578ff9640 10 client.27451 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:31:05.272 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.266+0000 7ff594964640 1 --2- 172.21.15.100:0/1345256051 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562221e0f770 0x562221e11b90 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f74ecd74cfee12a2 server_cookie=ba53ff9e5818cba2 in_seq=0 out_seq=0 2024-05-10T11:31:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.287+0000 7ff578ff9640 1 -- 172.21.15.100:0/1345256051 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562221c765f0 con 0x562221e0f770 2024-05-10T11:31:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.287+0000 7ff578ff9640 10 client.27451 handle_command_reply: tid=0 2024-05-10T11:31:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.287+0000 7ff59a1b6740 1 client.27451 shutdown 2024-05-10T11:31:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.287+0000 7ff59a1b6740 1 -- 172.21.15.100:0/1345256051 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562221d8edb0 msgr2=0x562221d9ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.287+0000 7ff59a1b6740 1 --2- 172.21.15.100:0/1345256051 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562221d8edb0 0x562221d9ce00 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff58c0a6e90 tx=0x7ff58c000bc0 comp rx=0 tx=0).stop 2024-05-10T11:31:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.287+0000 7ff59a1b6740 1 -- 172.21.15.100:0/1345256051 shutdown_connections 2024-05-10T11:31:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.287+0000 7ff59a1b6740 1 --2- 172.21.15.100:0/1345256051 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562221e0f770 0x562221e11b90 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.287+0000 7ff59a1b6740 1 --2- 172.21.15.100:0/1345256051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562221d94910 0x562221d9e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.287+0000 7ff578ff9640 0 client.27451 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:31:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.287+0000 7ff59a1b6740 1 --2- 172.21.15.100:0/1345256051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562221d8f7a0 0x562221d9d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.287+0000 7ff59a1b6740 1 --2- 172.21.15.100:0/1345256051 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562221d8edb0 0x562221d9ce00 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.287+0000 7ff59a1b6740 1 -- 172.21.15.100:0/1345256051 >> 172.21.15.100:0/1345256051 conn(0x562221d8a420 msgr2=0x562221d90a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:31:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.287+0000 7ff59a1b6740 1 -- 172.21.15.100:0/1345256051 shutdown_connections 2024-05-10T11:31:05.289 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.287+0000 7ff59a1b6740 1 -- 172.21.15.100:0/1345256051 wait complete. 2024-05-10T11:31:05.289 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.287+0000 7ff59a1b6740 20 client.27451 trim_cache size 0 max 16384 2024-05-10T11:31:05.289 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:31:05.289 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:31:05.289 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:31:05.289 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:31:05.290 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.288+0000 7ff5973c9640 1 -- 172.21.15.100:0/1804312297 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff57c0b28a0 msgr2=0x7ff57c0b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:05.290 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.288+0000 7ff5973c9640 1 --2- 172.21.15.100:0/1804312297 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff57c0b28a0 0x7ff57c0b4d60 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff588006010 tx=0x7ff5880319a0 comp rx=0 tx=0).stop 2024-05-10T11:31:05.290 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.288+0000 7ff5973c9640 1 -- 172.21.15.100:0/1804312297 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff590102a10 msgr2=0x7ff590198a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:05.290 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.288+0000 7ff5973c9640 1 --2- 172.21.15.100:0/1804312297 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff590102a10 0x7ff590198a80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff58c00e9c0 tx=0x7ff58c00ee80 comp rx=0 tx=0).stop 2024-05-10T11:31:05.290 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.288+0000 7ff5973c9640 1 -- 172.21.15.100:0/1804312297 shutdown_connections 2024-05-10T11:31:05.290 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.288+0000 7ff5973c9640 1 --2- 172.21.15.100:0/1804312297 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff57c0b28a0 0x7ff57c0b4d60 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:05.290 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.288+0000 7ff5973c9640 1 --2- 172.21.15.100:0/1804312297 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5901045c0 0x7ff5901a0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.288+0000 7ff5973c9640 1 --2- 172.21.15.100:0/1804312297 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff590103c00 0x7ff590198fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.288+0000 7ff5973c9640 1 --2- 172.21.15.100:0/1804312297 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff590102a10 0x7ff590198a80 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.288+0000 7ff5973c9640 1 -- 172.21.15.100:0/1804312297 >> 172.21.15.100:0/1804312297 conn(0x7ff5900fdfa0 msgr2=0x7ff590106f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:31:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.288+0000 7ff5973c9640 1 -- 172.21.15.100:0/1804312297 shutdown_connections 2024-05-10T11:31:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:05.288+0000 7ff5973c9640 1 -- 172.21.15.100:0/1804312297 wait complete. 2024-05-10T11:31:05.984 INFO:tasks.workunit.client.0.smithi100.stdout: 1 335271 2.70 MB/sec execute 540 sec latency 16.119 ms 2024-05-10T11:31:06.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:31:05 smithi178 ceph-mon[33359]: pgmap v802: 129 pgs: 129 active+clean; 128 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 1.3 MiB/s wr, 149 op/s 2024-05-10T11:31:06.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:31:05 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:31:06.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:31:05 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:31:06.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:31:05 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:31:06.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:31:05 smithi188 ceph-mon[34075]: pgmap v802: 129 pgs: 129 active+clean; 128 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 1.3 MiB/s wr, 149 op/s 2024-05-10T11:31:06.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:31:05 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:31:06.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:31:05 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:31:06.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:31:05 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:31:06.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:31:05 smithi100 ceph-mon[30296]: pgmap v802: 129 pgs: 129 active+clean; 128 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 1.3 MiB/s wr, 149 op/s 2024-05-10T11:31:06.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:31:05 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:31:06.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:31:05 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:31:06.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:31:05 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:31:06.984 INFO:tasks.workunit.client.0.smithi100.stdout: 1 335835 2.70 MB/sec execute 541 sec latency 15.836 ms 2024-05-10T11:31:07.839 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:31:07 smithi100 ceph-mon[30296]: pgmap v803: 129 pgs: 129 active+clean; 129 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 1.3 MiB/s wr, 131 op/s 2024-05-10T11:31:07.984 INFO:tasks.workunit.client.0.smithi100.stdout: 1 336446 2.71 MB/sec execute 542 sec latency 14.745 ms 2024-05-10T11:31:08.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:31:07 smithi178 ceph-mon[33359]: pgmap v803: 129 pgs: 129 active+clean; 129 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 1.3 MiB/s wr, 131 op/s 2024-05-10T11:31:08.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:31:07 smithi188 ceph-mon[34075]: pgmap v803: 129 pgs: 129 active+clean; 129 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 1.3 MiB/s wr, 131 op/s 2024-05-10T11:31:08.984 INFO:tasks.workunit.client.0.smithi100.stdout: 1 336991 2.71 MB/sec execute 543 sec latency 15.125 ms 2024-05-10T11:31:09.984 INFO:tasks.workunit.client.0.smithi100.stdout: 1 337313 2.71 MB/sec execute 544 sec latency 16.472 ms 2024-05-10T11:31:10.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:31:09 smithi178 ceph-mon[33359]: pgmap v804: 129 pgs: 129 active+clean; 129 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 1.2 MiB/s wr, 161 op/s 2024-05-10T11:31:10.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:31:09 smithi188 ceph-mon[34075]: pgmap v804: 129 pgs: 129 active+clean; 129 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 1.2 MiB/s wr, 161 op/s 2024-05-10T11:31:10.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:31:09 smithi100 ceph-mon[30296]: pgmap v804: 129 pgs: 129 active+clean; 129 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 1.2 MiB/s wr, 161 op/s 2024-05-10T11:31:10.984 INFO:tasks.workunit.client.0.smithi100.stdout: 1 337560 2.70 MB/sec execute 545 sec latency 14.870 ms 2024-05-10T11:31:11.984 INFO:tasks.workunit.client.0.smithi100.stdout: 1 338035 2.70 MB/sec execute 546 sec latency 18.621 ms 2024-05-10T11:31:12.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:31:11 smithi178 ceph-mon[33359]: pgmap v805: 129 pgs: 129 active+clean; 129 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 1.0 MiB/s wr, 115 op/s 2024-05-10T11:31:12.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:31:11 smithi188 ceph-mon[34075]: pgmap v805: 129 pgs: 129 active+clean; 129 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 1.0 MiB/s wr, 115 op/s 2024-05-10T11:31:12.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:31:11 smithi100 ceph-mon[30296]: pgmap v805: 129 pgs: 129 active+clean; 129 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 1.0 MiB/s wr, 115 op/s 2024-05-10T11:31:12.984 INFO:tasks.workunit.client.0.smithi100.stdout: 1 338599 2.70 MB/sec execute 547 sec latency 15.211 ms 2024-05-10T11:31:13.984 INFO:tasks.workunit.client.0.smithi100.stdout: 1 339285 2.70 MB/sec execute 548 sec latency 50.570 ms 2024-05-10T11:31:14.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:31:13 smithi178 ceph-mon[33359]: pgmap v806: 129 pgs: 129 active+clean; 131 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 1.3 MiB/s wr, 140 op/s 2024-05-10T11:31:14.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:31:13 smithi188 ceph-mon[34075]: pgmap v806: 129 pgs: 129 active+clean; 131 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 1.3 MiB/s wr, 140 op/s 2024-05-10T11:31:14.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:31:13 smithi100 ceph-mon[30296]: pgmap v806: 129 pgs: 129 active+clean; 131 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 1.3 MiB/s wr, 140 op/s 2024-05-10T11:31:14.984 INFO:tasks.workunit.client.0.smithi100.stdout: 1 339755 2.70 MB/sec execute 549 sec latency 16.846 ms 2024-05-10T11:31:15.985 INFO:tasks.workunit.client.0.smithi100.stdout: 1 340325 2.70 MB/sec execute 550 sec latency 15.674 ms 2024-05-10T11:31:16.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:31:15 smithi178 ceph-mon[33359]: pgmap v807: 129 pgs: 129 active+clean; 131 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 1.3 MiB/s wr, 142 op/s 2024-05-10T11:31:16.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:31:15 smithi188 ceph-mon[34075]: pgmap v807: 129 pgs: 129 active+clean; 131 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 1.3 MiB/s wr, 142 op/s 2024-05-10T11:31:16.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:31:15 smithi100 ceph-mon[30296]: pgmap v807: 129 pgs: 129 active+clean; 131 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 1.3 MiB/s wr, 142 op/s 2024-05-10T11:31:16.984 INFO:tasks.workunit.client.0.smithi100.stdout: 1 340800 2.71 MB/sec execute 551 sec latency 13.276 ms 2024-05-10T11:31:17.984 INFO:tasks.workunit.client.0.smithi100.stdout: 1 341054 2.70 MB/sec execute 552 sec latency 16.085 ms 2024-05-10T11:31:18.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:31:17 smithi178 ceph-mon[33359]: pgmap v808: 129 pgs: 129 active+clean; 130 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 1.2 MiB/s wr, 133 op/s 2024-05-10T11:31:18.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:31:17 smithi188 ceph-mon[34075]: pgmap v808: 129 pgs: 129 active+clean; 130 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 1.2 MiB/s wr, 133 op/s 2024-05-10T11:31:18.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:31:17 smithi100 ceph-mon[30296]: pgmap v808: 129 pgs: 129 active+clean; 130 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 1.2 MiB/s wr, 133 op/s 2024-05-10T11:31:18.384 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.382+0000 7f18e42d1640 1 -- 172.21.15.100:0/3968786426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18dc105dc0 msgr2=0x7f18dc1081b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:18.384 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.382+0000 7f18e42d1640 1 --2- 172.21.15.100:0/3968786426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18dc105dc0 0x7f18dc1081b0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f18cc009900 tx=0x7f18cc02a410 comp rx=0 tx=0).stop 2024-05-10T11:31:18.384 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.382+0000 7f18e42d1640 1 -- 172.21.15.100:0/3968786426 shutdown_connections 2024-05-10T11:31:18.384 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.382+0000 7f18e42d1640 1 --2- 172.21.15.100:0/3968786426 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f18dc1086f0 0x7f18dc10ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:18.384 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.382+0000 7f18e42d1640 1 --2- 172.21.15.100:0/3968786426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18dc105dc0 0x7f18dc1081b0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:18.384 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.382+0000 7f18e42d1640 1 --2- 172.21.15.100:0/3968786426 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f18dc103490 0x7f18dc105880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:18.384 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.382+0000 7f18e42d1640 1 -- 172.21.15.100:0/3968786426 >> 172.21.15.100:0/3968786426 conn(0x7f18dc0fce80 msgr2=0x7f18dc0ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:31:18.384 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.382+0000 7f18e42d1640 1 -- 172.21.15.100:0/3968786426 shutdown_connections 2024-05-10T11:31:18.384 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.382+0000 7f18e42d1640 1 -- 172.21.15.100:0/3968786426 wait complete. 2024-05-10T11:31:18.384 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.382+0000 7f18e42d1640 1 Processor -- start 2024-05-10T11:31:18.384 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.382+0000 7f18e42d1640 1 -- start start 2024-05-10T11:31:18.384 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.383+0000 7f18e42d1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f18dc103490 0x7f18dc198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:18.385 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.383+0000 7f18e42d1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f18dc105dc0 0x7f18dc199020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:18.385 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.383+0000 7f18e42d1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18dc1086f0 0x7f18dc1a0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:18.385 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.383+0000 7f18e42d1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f18dc10e3e0 con 0x7f18dc1086f0 2024-05-10T11:31:18.385 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.383+0000 7f18e42d1640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f18dc10e260 con 0x7f18dc105dc0 2024-05-10T11:31:18.385 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.383+0000 7f18e42d1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f18dc10e560 con 0x7f18dc103490 2024-05-10T11:31:18.385 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.383+0000 7f18e286e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18dc1086f0 0x7f18dc1a0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:18.385 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.383+0000 7f18e206d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f18dc103490 0x7f18dc198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:18.385 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.383+0000 7f18e286e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18dc1086f0 0x7f18dc1a0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44448/0 (socket says 172.21.15.100:44448) 2024-05-10T11:31:18.385 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.383+0000 7f18e286e640 1 -- 172.21.15.100:0/3880340955 learned_addr learned my addr 172.21.15.100:0/3880340955 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:31:18.385 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.383+0000 7f18e206d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f18dc103490 0x7f18dc198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:44056/0 (socket says 172.21.15.100:44056) 2024-05-10T11:31:18.385 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.383+0000 7f18e186c640 1 --2- 172.21.15.100:0/3880340955 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f18dc105dc0 0x7f18dc199020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:18.385 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.383+0000 7f18e286e640 1 -- 172.21.15.100:0/3880340955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f18dc103490 msgr2=0x7f18dc198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:18.385 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.383+0000 7f18e286e640 1 --2- 172.21.15.100:0/3880340955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f18dc103490 0x7f18dc198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:18.385 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.383+0000 7f18e286e640 1 -- 172.21.15.100:0/3880340955 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f18dc105dc0 msgr2=0x7f18dc199020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:18.385 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.383+0000 7f18e286e640 1 --2- 172.21.15.100:0/3880340955 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f18dc105dc0 0x7f18dc199020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:18.386 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.383+0000 7f18e286e640 1 -- 172.21.15.100:0/3880340955 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18dc1a0700 con 0x7f18dc1086f0 2024-05-10T11:31:18.386 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.383+0000 7f18e206d640 1 --2- 172.21.15.100:0/3880340955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f18dc103490 0x7f18dc198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:31:18.386 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.383+0000 7f18e286e640 1 --2- 172.21.15.100:0/3880340955 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18dc1086f0 0x7f18dc1a0090 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f18d400eb10 tx=0x7f18d400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:31:18.386 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.383+0000 7f18d37fe640 1 -- 172.21.15.100:0/3880340955 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d4004b20 con 0x7f18dc1086f0 2024-05-10T11:31:18.386 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.383+0000 7f18d37fe640 1 -- 172.21.15.100:0/3880340955 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f18d4004cc0 con 0x7f18dc1086f0 2024-05-10T11:31:18.386 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.383+0000 7f18e42d1640 1 -- 172.21.15.100:0/3880340955 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18dc1a09f0 con 0x7f18dc1086f0 2024-05-10T11:31:18.386 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.383+0000 7f18d37fe640 1 -- 172.21.15.100:0/3880340955 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d4010690 con 0x7f18dc1086f0 2024-05-10T11:31:18.386 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.383+0000 7f18e42d1640 1 -- 172.21.15.100:0/3880340955 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18dc1a0ec0 con 0x7f18dc1086f0 2024-05-10T11:31:18.392 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.386+0000 7f18e6fee740 1 Processor -- start 2024-05-10T11:31:18.392 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.386+0000 7f18d37fe640 1 -- 172.21.15.100:0/3880340955 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f18d4005020 con 0x7f18dc1086f0 2024-05-10T11:31:18.392 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.386+0000 7f18e6fee740 1 -- start start 2024-05-10T11:31:18.392 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.386+0000 7f18e6fee740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558bb7d791c0 0x558bb7d795e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:18.392 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.386+0000 7f18e6fee740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558bb7d79bb0 0x558bb7d7e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:18.392 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.386+0000 7f18e6fee740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bb7d7ed20 0x558bb7d811b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.386+0000 7f18e6fee740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x558bb7cd95d0 con 0x558bb7d7ed20 2024-05-10T11:31:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.386+0000 7f18e6fee740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558bb7c92890 con 0x558bb7d79bb0 2024-05-10T11:31:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.386+0000 7f18e6fee740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558bb7cd5590 con 0x558bb7d791c0 2024-05-10T11:31:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.387+0000 7f18e186c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558bb7d791c0 0x558bb7d795e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.387+0000 7f18e206d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558bb7d79bb0 0x558bb7d7e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.387+0000 7f18e186c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558bb7d791c0 0x558bb7d795e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:44058/0 (socket says 172.21.15.100:44058) 2024-05-10T11:31:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.387+0000 7f18e186c640 1 -- 172.21.15.100:0/3296468588 learned_addr learned my addr 172.21.15.100:0/3296468588 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:31:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.387+0000 7f18e186c640 1 --2- 172.21.15.100:0/3296468588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bb7d7ed20 0x558bb7d811b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.387+0000 7f18e206d640 1 -- 172.21.15.100:0/3296468588 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558bb7d791c0 msgr2=0x558bb7d795e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.387+0000 7f18e206d640 1 --2- 172.21.15.100:0/3296468588 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558bb7d791c0 0x558bb7d795e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.387+0000 7f18e206d640 1 -- 172.21.15.100:0/3296468588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bb7d7ed20 msgr2=0x558bb7d811b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.387+0000 7f18e206d640 1 --2- 172.21.15.100:0/3296468588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bb7d7ed20 0x558bb7d811b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.387+0000 7f18e206d640 1 -- 172.21.15.100:0/3296468588 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bb79eb210 con 0x558bb7d79bb0 2024-05-10T11:31:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.387+0000 7f18d37fe640 1 --2- 172.21.15.100:0/3880340955 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f18c80b28a0 0x7f18c80b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:18.394 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.387+0000 7f18e186c640 1 --2- 172.21.15.100:0/3880340955 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f18c80b28a0 0x7f18c80b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:18.394 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.387+0000 7f18e206d640 1 --2- 172.21.15.100:0/3296468588 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558bb7d79bb0 0x558bb7d7e6b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f18d800af30 tx=0x7f18d800f850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69250b84599b9440 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:31:18.394 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.387+0000 7f18d37fe640 1 -- 172.21.15.100:0/3880340955 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f18d40e2af0 con 0x7f18dc1086f0 2024-05-10T11:31:18.394 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.387+0000 7f18d0bfe640 1 -- 172.21.15.100:0/3296468588 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d801d070 con 0x558bb7d79bb0 2024-05-10T11:31:18.394 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.387+0000 7f18d0bfe640 1 -- 172.21.15.100:0/3296468588 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f18d8010cd0 con 0x558bb7d79bb0 2024-05-10T11:31:18.394 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.387+0000 7f18d0bfe640 1 -- 172.21.15.100:0/3296468588 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d8018bd0 con 0x558bb7d79bb0 2024-05-10T11:31:18.394 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.388+0000 7f18e186c640 1 --2- 172.21.15.100:0/3880340955 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f18c80b28a0 0x7f18c80b4d60 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f18cc0319f0 tx=0x7f18cc031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:31:18.394 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.388+0000 7f18e6fee740 1 -- 172.21.15.100:0/3296468588 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558bb7d79bb0 msgr2=0x558bb7d7e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:18.394 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.388+0000 7f18e6fee740 1 --2- 172.21.15.100:0/3296468588 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558bb7d79bb0 0x558bb7d7e6b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f18d800af30 tx=0x7f18d800f850 comp rx=0 tx=0).stop 2024-05-10T11:31:18.394 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.388+0000 7f18e6fee740 1 -- 172.21.15.100:0/3296468588 shutdown_connections 2024-05-10T11:31:18.394 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.388+0000 7f18e6fee740 1 --2- 172.21.15.100:0/3296468588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bb7d7ed20 0x558bb7d811b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:18.394 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.388+0000 7f18e6fee740 1 --2- 172.21.15.100:0/3296468588 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558bb7d79bb0 0x558bb7d7e6b0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:18.394 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.388+0000 7f18e6fee740 1 --2- 172.21.15.100:0/3296468588 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558bb7d791c0 0x558bb7d795e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:18.394 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.388+0000 7f18e6fee740 1 -- 172.21.15.100:0/3296468588 >> 172.21.15.100:0/3296468588 conn(0x558bb7d70420 msgr2=0x558bb7d70820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:31:18.395 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.388+0000 7f18e6fee740 1 -- 172.21.15.100:0/3296468588 shutdown_connections 2024-05-10T11:31:18.395 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.388+0000 7f18e6fee740 1 -- 172.21.15.100:0/3296468588 wait complete. 2024-05-10T11:31:18.395 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.389+0000 7f18e6fee740 1 -- start start 2024-05-10T11:31:18.395 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.389+0000 7f18e6fee740 1 Processor -- start 2024-05-10T11:31:18.395 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.389+0000 7f18e6fee740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558bb7d791c0 0x558bb7d83400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:18.395 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.389+0000 7f18e6fee740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bb7d79bb0 0x558bb7d83940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:18.395 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.389+0000 7f18e6fee740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558bb7d7ed20 0x558bb7d8a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:18.395 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.389+0000 7f18e6fee740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x558bb7d81cd0 con 0x558bb7d79bb0 2024-05-10T11:31:18.395 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.389+0000 7f18e6fee740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558bb7c96370 con 0x558bb7d791c0 2024-05-10T11:31:18.395 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.389+0000 7f18e6fee740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558bb7d81e50 con 0x558bb7d7ed20 2024-05-10T11:31:18.395 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.389+0000 7f18e6fee740 10 client.0 _ll_register_callbacks cb 0x558bb79eaa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:31:18.396 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.389+0000 7f18e186c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bb7d79bb0 0x558bb7d83940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:18.396 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.389+0000 7f18e186c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bb7d79bb0 0x558bb7d83940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44470/0 (socket says 172.21.15.100:44470) 2024-05-10T11:31:18.396 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.389+0000 7f18e186c640 1 -- 172.21.15.100:0/2737462426 learned_addr learned my addr 172.21.15.100:0/2737462426 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:31:18.396 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.389+0000 7f18e206d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558bb7d791c0 0x558bb7d83400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:18.396 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.389+0000 7f18e286e640 1 --2- 172.21.15.100:0/2737462426 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558bb7d7ed20 0x558bb7d8a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:18.396 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.389+0000 7f18e186c640 1 -- 172.21.15.100:0/2737462426 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558bb7d7ed20 msgr2=0x558bb7d8a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:18.396 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.389+0000 7f18e186c640 1 --2- 172.21.15.100:0/2737462426 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558bb7d7ed20 0x558bb7d8a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:18.396 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.389+0000 7f18e186c640 1 -- 172.21.15.100:0/2737462426 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558bb7d791c0 msgr2=0x558bb7d83400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:18.396 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.389+0000 7f18e186c640 1 --2- 172.21.15.100:0/2737462426 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558bb7d791c0 0x558bb7d83400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:18.396 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.389+0000 7f18e186c640 1 -- 172.21.15.100:0/2737462426 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558bb7ab7620 con 0x558bb7d79bb0 2024-05-10T11:31:18.396 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.389+0000 7f18e206d640 1 --2- 172.21.15.100:0/2737462426 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558bb7d791c0 0x558bb7d83400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:31:18.396 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.390+0000 7f18e186c640 1 --2- 172.21.15.100:0/2737462426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bb7d79bb0 0x558bb7d83940 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f18cc02a930 tx=0x7f18cc02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3915288c540bbfc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:31:18.396 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.390+0000 7f18c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:31:18.396 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.390+0000 7f18c5ffb640 1 -- 172.21.15.100:0/2737462426 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18cc050070 con 0x558bb7d79bb0 2024-05-10T11:31:18.397 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.390+0000 7f18e6fee740 1 -- 172.21.15.100:0/2737462426 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18d8010cd0 con 0x558bb7d79bb0 2024-05-10T11:31:18.397 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.390+0000 7f18c5ffb640 1 -- 172.21.15.100:0/2737462426 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f18cc032040 con 0x558bb7d79bb0 2024-05-10T11:31:18.397 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.390+0000 7f18c5ffb640 1 -- 172.21.15.100:0/2737462426 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f18cc05c050 con 0x558bb7d79bb0 2024-05-10T11:31:18.397 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.390+0000 7f18e6fee740 10 client.18021 fetch_fsmap learned FSMap version 33 2024-05-10T11:31:18.397 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.390+0000 7f18e6fee740 1 -- 172.21.15.100:0/2737462426 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x558bb7d8da20 con 0x558bb7d79bb0 2024-05-10T11:31:18.397 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.390+0000 7f18c5ffb640 1 -- 172.21.15.100:0/2737462426 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f18d8010cd0 con 0x558bb7d79bb0 2024-05-10T11:31:18.397 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.390+0000 7f18c5ffb640 1 -- 172.21.15.100:0/2737462426 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f18cc049070 con 0x558bb7d79bb0 2024-05-10T11:31:18.397 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.390+0000 7f18e6fee740 10 client.18021 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T11:31:18.397 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.390+0000 7f18e6fee740 10 client.18021 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:31:18.397 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.390+0000 7f18e6fee740 20 client.18021 populate_metadata read hostname 'smithi100' 2024-05-10T11:31:18.397 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.390+0000 7f18e6fee740 1 --2- 172.21.15.100:0/2737462426 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558bb7d8f110 0x558bb7d91560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:18.397 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.390+0000 7f18e6fee740 4 client.18021 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:31:18.397 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.390+0000 7f18e6fee740 1 -- 172.21.15.100:0/2737462426 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558bb7ce71e0 con 0x558bb7d8f110 2024-05-10T11:31:18.397 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.391+0000 7f18e206d640 1 --2- 172.21.15.100:0/2737462426 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558bb7d8f110 0x558bb7d91560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:18.397 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.391+0000 7f18c5ffb640 10 client.18021 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:31:18.398 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.391+0000 7f18e206d640 1 --2- 172.21.15.100:0/2737462426 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558bb7d8f110 0x558bb7d91560 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5dab226da64f089 server_cookie=9ae8be28e0985d4b in_seq=0 out_seq=0 2024-05-10T11:31:18.398 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.392+0000 7f18c5ffb640 1 -- 172.21.15.100:0/2737462426 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558bb7ce71e0 con 0x558bb7d8f110 2024-05-10T11:31:18.398 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.392+0000 7f18c5ffb640 10 client.18021 handle_command_reply: tid=0 2024-05-10T11:31:18.398 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.392+0000 7f18e6fee740 1 client.18021 shutdown 2024-05-10T11:31:18.398 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.392+0000 7f18e6fee740 1 -- 172.21.15.100:0/2737462426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bb7d79bb0 msgr2=0x558bb7d83940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:18.398 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.392+0000 7f18e6fee740 1 --2- 172.21.15.100:0/2737462426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bb7d79bb0 0x558bb7d83940 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f18cc02a930 tx=0x7f18cc02a410 comp rx=0 tx=0).stop 2024-05-10T11:31:18.398 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.392+0000 7f18e6fee740 1 -- 172.21.15.100:0/2737462426 shutdown_connections 2024-05-10T11:31:18.398 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.392+0000 7f18e6fee740 1 --2- 172.21.15.100:0/2737462426 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558bb7d8f110 0x558bb7d91560 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:18.398 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.392+0000 7f18e6fee740 1 --2- 172.21.15.100:0/2737462426 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558bb7d7ed20 0x558bb7d8a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:18.398 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.392+0000 7f18e6fee740 1 --2- 172.21.15.100:0/2737462426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bb7d79bb0 0x558bb7d83940 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:18.398 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.392+0000 7f18c5ffb640 0 client.18021 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:31:18.398 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.392+0000 7f18e6fee740 1 --2- 172.21.15.100:0/2737462426 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558bb7d791c0 0x558bb7d83400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:18.398 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.392+0000 7f18e6fee740 1 -- 172.21.15.100:0/2737462426 >> 172.21.15.100:0/2737462426 conn(0x558bb7d70420 msgr2=0x558bb7d6f580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:31:18.399 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.392+0000 7f18e6fee740 1 -- 172.21.15.100:0/2737462426 shutdown_connections 2024-05-10T11:31:18.399 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.392+0000 7f18e6fee740 1 -- 172.21.15.100:0/2737462426 wait complete. 2024-05-10T11:31:18.399 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.392+0000 7f18e6fee740 20 client.18021 trim_cache size 0 max 16384 2024-05-10T11:31:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.409+0000 7f18e6fee740 1 Processor -- start 2024-05-10T11:31:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.409+0000 7f18e6fee740 1 -- start start 2024-05-10T11:31:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.409+0000 7f18e6fee740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558bb7d791c0 0x558bb7d73330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.409+0000 7f18e6fee740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bb7d79bb0 0x558bb7d73870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.409+0000 7f18e6fee740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558bb7d7ed20 0x558bb7d95e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.409+0000 7f18e6fee740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f18dc1ab9d0 con 0x558bb7d79bb0 2024-05-10T11:31:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.409+0000 7f18e6fee740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f18dc1a92a0 con 0x558bb7d7ed20 2024-05-10T11:31:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.409+0000 7f18e6fee740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f18dc1a70f0 con 0x558bb7d791c0 2024-05-10T11:31:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.409+0000 7f18e186c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bb7d79bb0 0x558bb7d73870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.409+0000 7f18e186c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bb7d79bb0 0x558bb7d73870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44484/0 (socket says 172.21.15.100:44484) 2024-05-10T11:31:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.409+0000 7f18e186c640 1 -- 172.21.15.100:0/2421741664 learned_addr learned my addr 172.21.15.100:0/2421741664 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:31:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.409+0000 7f18e206d640 1 --2- 172.21.15.100:0/2421741664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558bb7d791c0 0x558bb7d73330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.409+0000 7f18e286e640 1 --2- 172.21.15.100:0/2421741664 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558bb7d7ed20 0x558bb7d95e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.409+0000 7f18e186c640 1 -- 172.21.15.100:0/2421741664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558bb7d791c0 msgr2=0x558bb7d73330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.409+0000 7f18e186c640 1 --2- 172.21.15.100:0/2421741664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558bb7d791c0 0x558bb7d73330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.409+0000 7f18e186c640 1 -- 172.21.15.100:0/2421741664 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558bb7d7ed20 msgr2=0x558bb7d95e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.409+0000 7f18e186c640 1 --2- 172.21.15.100:0/2421741664 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558bb7d7ed20 0x558bb7d95e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.409+0000 7f18e186c640 1 -- 172.21.15.100:0/2421741664 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bb7d7e0a0 con 0x558bb7d79bb0 2024-05-10T11:31:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.409+0000 7f18e286e640 1 --2- 172.21.15.100:0/2421741664 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558bb7d7ed20 0x558bb7d95e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:31:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.409+0000 7f18e186c640 1 --2- 172.21.15.100:0/2421741664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bb7d79bb0 0x558bb7d73870 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f18cc0057c0 tx=0x7f18cc0318c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc73f7ec71671212 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:31:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.410+0000 7f18c7fff640 1 -- 172.21.15.100:0/2421741664 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18cc050040 con 0x558bb7d79bb0 2024-05-10T11:31:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.410+0000 7f18c7fff640 1 -- 172.21.15.100:0/2421741664 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f18cc04c640 con 0x558bb7d79bb0 2024-05-10T11:31:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.410+0000 7f18c7fff640 1 -- 172.21.15.100:0/2421741664 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18cc035040 con 0x558bb7d79bb0 2024-05-10T11:31:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.410+0000 7f18e6fee740 1 -- 172.21.15.100:0/2421741664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bb7d79bb0 msgr2=0x558bb7d73870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.410+0000 7f18e6fee740 1 --2- 172.21.15.100:0/2421741664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bb7d79bb0 0x558bb7d73870 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f18cc0057c0 tx=0x7f18cc0318c0 comp rx=0 tx=0).stop 2024-05-10T11:31:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.410+0000 7f18e6fee740 1 -- 172.21.15.100:0/2421741664 shutdown_connections 2024-05-10T11:31:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.410+0000 7f18e6fee740 1 --2- 172.21.15.100:0/2421741664 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558bb7d7ed20 0x558bb7d95e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.410+0000 7f18e6fee740 1 --2- 172.21.15.100:0/2421741664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bb7d79bb0 0x558bb7d73870 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.410+0000 7f18e6fee740 1 --2- 172.21.15.100:0/2421741664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558bb7d791c0 0x558bb7d73330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.410+0000 7f18e6fee740 1 -- 172.21.15.100:0/2421741664 >> 172.21.15.100:0/2421741664 conn(0x558bb7d70420 msgr2=0x558bb7d7add0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:31:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.410+0000 7f18e6fee740 1 -- 172.21.15.100:0/2421741664 shutdown_connections 2024-05-10T11:31:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.410+0000 7f18e6fee740 1 -- 172.21.15.100:0/2421741664 wait complete. 2024-05-10T11:31:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.411+0000 7f18e6fee740 1 -- start start 2024-05-10T11:31:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.411+0000 7f18e6fee740 1 Processor -- start 2024-05-10T11:31:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.411+0000 7f18e6fee740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558bb7d791c0 0x558bb7d876a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.411+0000 7f18e6fee740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558bb7d79bb0 0x558bb7d87be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.411+0000 7f18e6fee740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bb7d7ed20 0x558bb7d88bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.411+0000 7f18e6fee740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f18dc1a7760 con 0x558bb7d7ed20 2024-05-10T11:31:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.411+0000 7f18e6fee740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f18dc1a8bd0 con 0x558bb7d79bb0 2024-05-10T11:31:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.411+0000 7f18e6fee740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f18dc1a7e30 con 0x558bb7d791c0 2024-05-10T11:31:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.411+0000 7f18e6fee740 10 client.0 _ll_register_callbacks cb 0x7f18cc032da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:31:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.411+0000 7f18e286e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bb7d7ed20 0x558bb7d88bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.411+0000 7f18e286e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bb7d7ed20 0x558bb7d88bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44498/0 (socket says 172.21.15.100:44498) 2024-05-10T11:31:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.411+0000 7f18e206d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558bb7d791c0 0x558bb7d876a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.411+0000 7f18e186c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558bb7d79bb0 0x558bb7d87be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.411+0000 7f18e286e640 1 -- 172.21.15.100:0/3709901213 learned_addr learned my addr 172.21.15.100:0/3709901213 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:31:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.411+0000 7f18e286e640 1 -- 172.21.15.100:0/3709901213 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558bb7d791c0 msgr2=0x558bb7d876a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.411+0000 7f18e286e640 1 --2- 172.21.15.100:0/3709901213 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558bb7d791c0 0x558bb7d876a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.411+0000 7f18e286e640 1 -- 172.21.15.100:0/3709901213 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558bb7d79bb0 msgr2=0x558bb7d87be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.411+0000 7f18e286e640 1 --2- 172.21.15.100:0/3709901213 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558bb7d79bb0 0x558bb7d87be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.411+0000 7f18e286e640 1 -- 172.21.15.100:0/3709901213 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558bb7d891e0 con 0x558bb7d7ed20 2024-05-10T11:31:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.412+0000 7f18e286e640 1 --2- 172.21.15.100:0/3709901213 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bb7d7ed20 0x558bb7d88bf0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f18d40e0290 tx=0x7f18d4005650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c559e88d8e00fc53 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:31:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.412+0000 7f18c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:31:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.412+0000 7f18c5ffb640 1 -- 172.21.15.100:0/3709901213 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d40a9db0 con 0x558bb7d7ed20 2024-05-10T11:31:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.412+0000 7f18c5ffb640 1 -- 172.21.15.100:0/3709901213 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f18d40b2990 con 0x558bb7d7ed20 2024-05-10T11:31:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.412+0000 7f18e6fee740 1 -- 172.21.15.100:0/3709901213 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18cc04c640 con 0x558bb7d7ed20 2024-05-10T11:31:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.412+0000 7f18c5ffb640 1 -- 172.21.15.100:0/3709901213 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f18d40ae160 con 0x558bb7d7ed20 2024-05-10T11:31:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.412+0000 7f18e6fee740 10 client.18027 fetch_fsmap learned FSMap version 33 2024-05-10T11:31:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.412+0000 7f18e6fee740 1 -- 172.21.15.100:0/3709901213 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x558bb7df9a20 con 0x558bb7d7ed20 2024-05-10T11:31:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.412+0000 7f18c5ffb640 1 -- 172.21.15.100:0/3709901213 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f18cc04c640 con 0x558bb7d7ed20 2024-05-10T11:31:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.412+0000 7f18c5ffb640 1 -- 172.21.15.100:0/3709901213 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f18d40a8b10 con 0x558bb7d7ed20 2024-05-10T11:31:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.412+0000 7f18e6fee740 10 client.18027 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T11:31:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.412+0000 7f18e6fee740 10 client.18027 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:31:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.412+0000 7f18e6fee740 20 client.18027 populate_metadata read hostname 'smithi100' 2024-05-10T11:31:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.412+0000 7f18e6fee740 1 --2- 172.21.15.100:0/3709901213 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558bb7dfa3a0 0x558bb7dfc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.412+0000 7f18e6fee740 4 client.18027 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:31:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.412+0000 7f18e6fee740 1 -- 172.21.15.100:0/3709901213 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558bb7ce1050 con 0x558bb7dfa3a0 2024-05-10T11:31:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.412+0000 7f18e206d640 1 --2- 172.21.15.100:0/3709901213 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558bb7dfa3a0 0x558bb7dfc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.413+0000 7f18c5ffb640 10 client.18027 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:31:18.418 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.413+0000 7f18e206d640 1 --2- 172.21.15.100:0/3709901213 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558bb7dfa3a0 0x558bb7dfc7c0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c51cc5cb338b685b server_cookie=85eb7245b18aed80 in_seq=0 out_seq=0 2024-05-10T11:31:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.435+0000 7f18c5ffb640 1 -- 172.21.15.100:0/3709901213 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558bb7ce1050 con 0x558bb7dfa3a0 2024-05-10T11:31:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.435+0000 7f18c5ffb640 10 client.18027 handle_command_reply: tid=0 2024-05-10T11:31:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.435+0000 7f18e6fee740 1 client.18027 shutdown 2024-05-10T11:31:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.436+0000 7f18e6fee740 1 -- 172.21.15.100:0/3709901213 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bb7d7ed20 msgr2=0x558bb7d88bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.436+0000 7f18e6fee740 1 --2- 172.21.15.100:0/3709901213 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bb7d7ed20 0x558bb7d88bf0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f18d40e0290 tx=0x7f18d4005650 comp rx=0 tx=0).stop 2024-05-10T11:31:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.436+0000 7f18e6fee740 1 -- 172.21.15.100:0/3709901213 shutdown_connections 2024-05-10T11:31:18.437 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.436+0000 7f18e6fee740 1 --2- 172.21.15.100:0/3709901213 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558bb7dfa3a0 0x558bb7dfc7c0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:18.437 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.436+0000 7f18e6fee740 1 --2- 172.21.15.100:0/3709901213 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bb7d7ed20 0x558bb7d88bf0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:18.437 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.436+0000 7f18e6fee740 1 --2- 172.21.15.100:0/3709901213 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558bb7d79bb0 0x558bb7d87be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:18.437 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.436+0000 7f18c5ffb640 0 client.18027 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:31:18.437 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.436+0000 7f18e6fee740 1 --2- 172.21.15.100:0/3709901213 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558bb7d791c0 0x558bb7d876a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:18.437 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.436+0000 7f18e6fee740 1 -- 172.21.15.100:0/3709901213 >> 172.21.15.100:0/3709901213 conn(0x558bb7d70420 msgr2=0x558bb7d73330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:31:18.437 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.436+0000 7f18e6fee740 1 -- 172.21.15.100:0/3709901213 shutdown_connections 2024-05-10T11:31:18.437 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.436+0000 7f18e6fee740 1 -- 172.21.15.100:0/3709901213 wait complete. 2024-05-10T11:31:18.437 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.436+0000 7f18e6fee740 20 client.18027 trim_cache size 0 max 16384 2024-05-10T11:31:18.437 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:31:18.437 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:31:18.437 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:31:18.438 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:31:18.438 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.436+0000 7f18e42d1640 1 -- 172.21.15.100:0/3880340955 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f18c80b28a0 msgr2=0x7f18c80b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:18.439 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.436+0000 7f18e42d1640 1 --2- 172.21.15.100:0/3880340955 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f18c80b28a0 0x7f18c80b4d60 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f18cc0319f0 tx=0x7f18cc031400 comp rx=0 tx=0).stop 2024-05-10T11:31:18.439 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.436+0000 7f18e42d1640 1 -- 172.21.15.100:0/3880340955 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18dc1086f0 msgr2=0x7f18dc1a0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:18.439 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.436+0000 7f18e42d1640 1 --2- 172.21.15.100:0/3880340955 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18dc1086f0 0x7f18dc1a0090 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f18d400eb10 tx=0x7f18d400c2d0 comp rx=0 tx=0).stop 2024-05-10T11:31:18.439 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.436+0000 7f18e42d1640 1 -- 172.21.15.100:0/3880340955 shutdown_connections 2024-05-10T11:31:18.439 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.436+0000 7f18e42d1640 1 --2- 172.21.15.100:0/3880340955 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f18c80b28a0 0x7f18c80b4d60 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:18.439 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.436+0000 7f18e42d1640 1 --2- 172.21.15.100:0/3880340955 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18dc1086f0 0x7f18dc1a0090 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:18.439 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.436+0000 7f18e42d1640 1 --2- 172.21.15.100:0/3880340955 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f18dc105dc0 0x7f18dc199020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:18.439 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.436+0000 7f18e42d1640 1 --2- 172.21.15.100:0/3880340955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f18dc103490 0x7f18dc198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:18.439 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.436+0000 7f18e42d1640 1 -- 172.21.15.100:0/3880340955 >> 172.21.15.100:0/3880340955 conn(0x7f18dc0fce80 msgr2=0x7f18dc1052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:31:18.439 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.437+0000 7f18e42d1640 1 -- 172.21.15.100:0/3880340955 shutdown_connections 2024-05-10T11:31:18.439 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:18.437+0000 7f18e42d1640 1 -- 172.21.15.100:0/3880340955 wait complete. 2024-05-10T11:31:18.985 INFO:tasks.workunit.client.0.smithi100.stdout: 1 341457 2.70 MB/sec execute 553 sec latency 15.887 ms 2024-05-10T11:31:19.985 INFO:tasks.workunit.client.0.smithi100.stdout: 1 342006 2.70 MB/sec execute 554 sec latency 21.103 ms 2024-05-10T11:31:20.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:31:19 smithi178 ceph-mon[33359]: pgmap v809: 129 pgs: 129 active+clean; 131 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 1.3 MiB/s wr, 164 op/s 2024-05-10T11:31:20.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:31:19 smithi188 ceph-mon[34075]: pgmap v809: 129 pgs: 129 active+clean; 131 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 1.3 MiB/s wr, 164 op/s 2024-05-10T11:31:20.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:31:19 smithi100 ceph-mon[30296]: pgmap v809: 129 pgs: 129 active+clean; 131 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 1.3 MiB/s wr, 164 op/s 2024-05-10T11:31:20.985 INFO:tasks.workunit.client.0.smithi100.stdout: 1 342602 2.70 MB/sec execute 555 sec latency 16.420 ms 2024-05-10T11:31:21.985 INFO:tasks.workunit.client.0.smithi100.stdout: 1 343109 2.70 MB/sec execute 556 sec latency 15.412 ms 2024-05-10T11:31:22.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:31:21 smithi178 ceph-mon[33359]: pgmap v810: 129 pgs: 129 active+clean; 132 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 1.2 MiB/s wr, 126 op/s 2024-05-10T11:31:22.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:31:21 smithi188 ceph-mon[34075]: pgmap v810: 129 pgs: 129 active+clean; 132 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 1.2 MiB/s wr, 126 op/s 2024-05-10T11:31:22.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:31:21 smithi100 ceph-mon[30296]: pgmap v810: 129 pgs: 129 active+clean; 132 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 1.2 MiB/s wr, 126 op/s 2024-05-10T11:31:22.985 INFO:tasks.workunit.client.0.smithi100.stdout: 1 343830 2.71 MB/sec execute 557 sec latency 19.395 ms 2024-05-10T11:31:23.830 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:31:23 smithi188 ceph-mon[34075]: pgmap v811: 129 pgs: 129 active+clean; 133 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 1.3 MiB/s wr, 151 op/s 2024-05-10T11:31:23.985 INFO:tasks.workunit.client.0.smithi100.stdout: 1 344293 2.71 MB/sec execute 558 sec latency 14.196 ms 2024-05-10T11:31:24.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:31:23 smithi178 ceph-mon[33359]: pgmap v811: 129 pgs: 129 active+clean; 133 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 1.3 MiB/s wr, 151 op/s 2024-05-10T11:31:24.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:31:23 smithi100 ceph-mon[30296]: pgmap v811: 129 pgs: 129 active+clean; 133 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 1.3 MiB/s wr, 151 op/s 2024-05-10T11:31:24.985 INFO:tasks.workunit.client.0.smithi100.stdout: 1 344522 2.70 MB/sec execute 559 sec latency 16.981 ms 2024-05-10T11:31:25.985 INFO:tasks.workunit.client.0.smithi100.stdout: 1 344895 2.70 MB/sec execute 560 sec latency 19.914 ms 2024-05-10T11:31:26.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:31:25 smithi178 ceph-mon[33359]: pgmap v812: 129 pgs: 129 active+clean; 132 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 1.3 MiB/s wr, 142 op/s 2024-05-10T11:31:26.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:31:25 smithi188 ceph-mon[34075]: pgmap v812: 129 pgs: 129 active+clean; 132 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 1.3 MiB/s wr, 142 op/s 2024-05-10T11:31:26.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:31:25 smithi100 ceph-mon[30296]: pgmap v812: 129 pgs: 129 active+clean; 132 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 1.3 MiB/s wr, 142 op/s 2024-05-10T11:31:26.985 INFO:tasks.workunit.client.0.smithi100.stdout: 1 345451 2.70 MB/sec execute 561 sec latency 16.106 ms 2024-05-10T11:31:27.985 INFO:tasks.workunit.client.0.smithi100.stdout: 1 345986 2.70 MB/sec execute 562 sec latency 18.167 ms 2024-05-10T11:31:28.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:31:27 smithi178 ceph-mon[33359]: pgmap v813: 129 pgs: 129 active+clean; 134 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 1.3 MiB/s wr, 129 op/s 2024-05-10T11:31:28.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:31:27 smithi188 ceph-mon[34075]: pgmap v813: 129 pgs: 129 active+clean; 134 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 1.3 MiB/s wr, 129 op/s 2024-05-10T11:31:28.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:31:27 smithi100 ceph-mon[30296]: pgmap v813: 129 pgs: 129 active+clean; 134 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 1.3 MiB/s wr, 129 op/s 2024-05-10T11:31:28.985 INFO:tasks.workunit.client.0.smithi100.stdout: 1 346546 2.70 MB/sec execute 563 sec latency 18.176 ms 2024-05-10T11:31:29.985 INFO:tasks.workunit.client.0.smithi100.stdout: 1 347268 2.70 MB/sec execute 564 sec latency 15.731 ms 2024-05-10T11:31:30.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:31:29 smithi178 ceph-mon[33359]: pgmap v814: 129 pgs: 129 active+clean; 135 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 1.4 MiB/s wr, 162 op/s 2024-05-10T11:31:30.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:31:29 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:31:30.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:31:29 smithi188 ceph-mon[34075]: pgmap v814: 129 pgs: 129 active+clean; 135 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 1.4 MiB/s wr, 162 op/s 2024-05-10T11:31:30.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:31:29 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:31:30.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:31:29 smithi100 ceph-mon[30296]: pgmap v814: 129 pgs: 129 active+clean; 135 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 1.4 MiB/s wr, 162 op/s 2024-05-10T11:31:30.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:31:29 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:31:30.859 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T11:31:30.882 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:31:30 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:31:30.883 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:31:30 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:31:30.883 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:31:30 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:31:30.985 INFO:tasks.workunit.client.0.smithi100.stdout: 1 347705 2.71 MB/sec execute 565 sec latency 14.697 ms 2024-05-10T11:31:31.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:31:30 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:31:31.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:31:30 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:31:31.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:31:30 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:31:31.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:31:30 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:31:31.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:31:30 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:31:31.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:31:30 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:31:31.118 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:31:31.535 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.532+0000 7fae1e459640 1 -- 172.21.15.100:0/1675617092 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae18102b00 msgr2=0x7fae18102f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:31.536 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.532+0000 7fae1e459640 1 --2- 172.21.15.100:0/1675617092 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae18102b00 0x7fae18102f80 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fae0c009900 tx=0x7fae0c02a410 comp rx=0 tx=0).stop 2024-05-10T11:31:31.536 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.532+0000 7fae1e459640 1 -- 172.21.15.100:0/1675617092 shutdown_connections 2024-05-10T11:31:31.536 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.532+0000 7fae1e459640 1 --2- 172.21.15.100:0/1675617092 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae181034c0 0x7fae18109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:31.536 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.532+0000 7fae1e459640 1 --2- 172.21.15.100:0/1675617092 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae18102b00 0x7fae18102f80 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:31.536 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.532+0000 7fae1e459640 1 --2- 172.21.15.100:0/1675617092 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae18101910 0x7fae18101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:31.536 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.532+0000 7fae1e459640 1 -- 172.21.15.100:0/1675617092 >> 172.21.15.100:0/1675617092 conn(0x7fae180fce80 msgr2=0x7fae180ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:31:31.536 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.532+0000 7fae1e459640 1 -- 172.21.15.100:0/1675617092 shutdown_connections 2024-05-10T11:31:31.536 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.532+0000 7fae1e459640 1 -- 172.21.15.100:0/1675617092 wait complete. 2024-05-10T11:31:31.536 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.532+0000 7fae1e459640 1 Processor -- start 2024-05-10T11:31:31.536 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.532+0000 7fae1e459640 1 -- start start 2024-05-10T11:31:31.536 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.533+0000 7fae1e459640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae18101910 0x7fae18198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:31.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.533+0000 7fae1e459640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae18102b00 0x7fae18199030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:31.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.533+0000 7fae1e459640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae181034c0 0x7fae181a00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:31.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.533+0000 7fae1e459640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fae1810d2d0 con 0x7fae18101910 2024-05-10T11:31:31.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.533+0000 7fae1e459640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fae1810d150 con 0x7fae18102b00 2024-05-10T11:31:31.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.533+0000 7fae1e459640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fae1810d450 con 0x7fae181034c0 2024-05-10T11:31:31.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.533+0000 7fae17fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae18101910 0x7fae18198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:31.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.533+0000 7fae17fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae18101910 0x7fae18198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35908/0 (socket says 172.21.15.100:35908) 2024-05-10T11:31:31.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.533+0000 7fae17fff640 1 -- 172.21.15.100:0/2882105171 learned_addr learned my addr 172.21.15.100:0/2882105171 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:31:31.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.533+0000 7fae1c9f6640 1 --2- 172.21.15.100:0/2882105171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae181034c0 0x7fae181a00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:31.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.533+0000 7fae177fe640 1 --2- 172.21.15.100:0/2882105171 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae18102b00 0x7fae18199030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:31.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.533+0000 7fae17fff640 1 -- 172.21.15.100:0/2882105171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae181034c0 msgr2=0x7fae181a00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:31.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.533+0000 7fae17fff640 1 --2- 172.21.15.100:0/2882105171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae181034c0 0x7fae181a00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:31.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.533+0000 7fae17fff640 1 -- 172.21.15.100:0/2882105171 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae18102b00 msgr2=0x7fae18199030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:31.538 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.533+0000 7fae17fff640 1 --2- 172.21.15.100:0/2882105171 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae18102b00 0x7fae18199030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:31.538 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.533+0000 7fae17fff640 1 -- 172.21.15.100:0/2882105171 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae181a0710 con 0x7fae18101910 2024-05-10T11:31:31.538 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.533+0000 7fae17fff640 1 --2- 172.21.15.100:0/2882105171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae18101910 0x7fae18198af0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fae0800e9c0 tx=0x7fae0800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:31:31.538 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.534+0000 7fae157fa640 1 -- 172.21.15.100:0/2882105171 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae0800cd80 con 0x7fae18101910 2024-05-10T11:31:31.538 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.534+0000 7fae157fa640 1 -- 172.21.15.100:0/2882105171 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fae080044b0 con 0x7fae18101910 2024-05-10T11:31:31.538 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.534+0000 7fae1e459640 1 -- 172.21.15.100:0/2882105171 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae181a0a00 con 0x7fae18101910 2024-05-10T11:31:31.538 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.534+0000 7fae157fa640 1 -- 172.21.15.100:0/2882105171 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae08010430 con 0x7fae18101910 2024-05-10T11:31:31.538 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.534+0000 7fae1e459640 1 -- 172.21.15.100:0/2882105171 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae181a0f70 con 0x7fae18101910 2024-05-10T11:31:31.545 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.537+0000 7fae2117b740 1 Processor -- start 2024-05-10T11:31:31.545 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.537+0000 7fae2117b740 1 -- start start 2024-05-10T11:31:31.545 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.537+0000 7fae2117b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55917da541c0 0x55917da545e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:31.545 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.537+0000 7fae2117b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55917da54bb0 0x55917da596b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:31.545 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.537+0000 7fae2117b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55917da59d20 0x55917da5c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:31.545 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.537+0000 7fae2117b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55917d9b45d0 con 0x55917da59d20 2024-05-10T11:31:31.545 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.537+0000 7fae2117b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55917d96d890 con 0x55917da54bb0 2024-05-10T11:31:31.545 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.537+0000 7fae2117b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55917d9b0590 con 0x55917da541c0 2024-05-10T11:31:31.546 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.537+0000 7fae157fa640 1 -- 172.21.15.100:0/2882105171 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fae08003f00 con 0x7fae18101910 2024-05-10T11:31:31.546 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.537+0000 7fae177fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55917da59d20 0x55917da5c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:31.546 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.537+0000 7fae177fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55917da59d20 0x55917da5c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35914/0 (socket says 172.21.15.100:35914) 2024-05-10T11:31:31.546 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.537+0000 7fae177fe640 1 -- 172.21.15.100:0/3334501867 learned_addr learned my addr 172.21.15.100:0/3334501867 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:31:31.546 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.537+0000 7fae1c9f6640 1 --2- 172.21.15.100:0/3334501867 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55917da541c0 0x55917da545e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:31.546 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.538+0000 7fae1c9f6640 1 -- 172.21.15.100:0/3334501867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55917da54bb0 msgr2=0x55917da596b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:31.546 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.538+0000 7fae1c9f6640 1 --2- 172.21.15.100:0/3334501867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55917da54bb0 0x55917da596b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:31.546 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.538+0000 7fae1c9f6640 1 -- 172.21.15.100:0/3334501867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55917da59d20 msgr2=0x55917da5c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:31.546 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.538+0000 7fae1c9f6640 1 --2- 172.21.15.100:0/3334501867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55917da59d20 0x55917da5c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:31.546 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.538+0000 7fae1c9f6640 1 -- 172.21.15.100:0/3334501867 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55917d6c6210 con 0x55917da541c0 2024-05-10T11:31:31.546 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.538+0000 7fae177fe640 1 --2- 172.21.15.100:0/3334501867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55917da59d20 0x55917da5c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:31:31.546 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.538+0000 7fae1c9f6640 1 --2- 172.21.15.100:0/3334501867 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55917da541c0 0x55917da545e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fae10009f00 tx=0x7fae10011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a977fbad6284c0b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:31:31.547 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.538+0000 7fadfe7fc640 1 -- 172.21.15.100:0/3334501867 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae10004d20 con 0x55917da541c0 2024-05-10T11:31:31.547 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.538+0000 7fadfe7fc640 1 -- 172.21.15.100:0/3334501867 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fae10004030 con 0x55917da541c0 2024-05-10T11:31:31.547 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.538+0000 7fae157fa640 1 --2- 172.21.15.100:0/2882105171 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fae000b2920 0x7fae000b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:31.547 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.538+0000 7fadfe7fc640 1 -- 172.21.15.100:0/3334501867 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae10005240 con 0x55917da541c0 2024-05-10T11:31:31.547 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.538+0000 7fae17fff640 1 --2- 172.21.15.100:0/2882105171 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fae000b2920 0x7fae000b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:31.547 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.538+0000 7fae157fa640 1 -- 172.21.15.100:0/2882105171 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fae080e28e0 con 0x7fae18101910 2024-05-10T11:31:31.547 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.539+0000 7fae2117b740 1 -- 172.21.15.100:0/3334501867 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55917da541c0 msgr2=0x55917da545e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:31.547 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.539+0000 7fae17fff640 1 --2- 172.21.15.100:0/2882105171 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fae000b2920 0x7fae000b4de0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fae080a98a0 tx=0x7fae080cc040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:31:31.547 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.539+0000 7fae2117b740 1 --2- 172.21.15.100:0/3334501867 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55917da541c0 0x55917da545e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fae10009f00 tx=0x7fae10011810 comp rx=0 tx=0).stop 2024-05-10T11:31:31.547 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.539+0000 7fae2117b740 1 -- 172.21.15.100:0/3334501867 shutdown_connections 2024-05-10T11:31:31.548 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.539+0000 7fae2117b740 1 --2- 172.21.15.100:0/3334501867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55917da59d20 0x55917da5c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:31.548 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.539+0000 7fae2117b740 1 --2- 172.21.15.100:0/3334501867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55917da54bb0 0x55917da596b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:31.548 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.539+0000 7fae2117b740 1 --2- 172.21.15.100:0/3334501867 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55917da541c0 0x55917da545e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:31.548 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.539+0000 7fae2117b740 1 -- 172.21.15.100:0/3334501867 >> 172.21.15.100:0/3334501867 conn(0x55917da4f830 msgr2=0x55917da4fc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:31:31.548 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.539+0000 7fae2117b740 1 -- 172.21.15.100:0/3334501867 shutdown_connections 2024-05-10T11:31:31.548 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.539+0000 7fae2117b740 1 -- 172.21.15.100:0/3334501867 wait complete. 2024-05-10T11:31:31.548 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.539+0000 7fae2117b740 1 -- start start 2024-05-10T11:31:31.548 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.539+0000 7fae2117b740 1 Processor -- start 2024-05-10T11:31:31.548 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.540+0000 7fae2117b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55917da541c0 0x55917da5e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:31.548 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.540+0000 7fae2117b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55917da54bb0 0x55917da5e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:31.548 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.540+0000 7fae2117b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55917da59d20 0x55917da65960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:31.549 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.540+0000 7fae2117b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55917da5ccd0 con 0x55917da59d20 2024-05-10T11:31:31.549 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.540+0000 7fae2117b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55917d971370 con 0x55917da54bb0 2024-05-10T11:31:31.549 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.540+0000 7fae2117b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55917da5ce50 con 0x55917da541c0 2024-05-10T11:31:31.549 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.540+0000 7fae2117b740 10 client.0 _ll_register_callbacks cb 0x55917d6c5a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:31:31.549 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.540+0000 7fae1c9f6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55917da541c0 0x55917da5e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:31.549 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.540+0000 7fae1c9f6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55917da541c0 0x55917da5e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:43816/0 (socket says 172.21.15.100:43816) 2024-05-10T11:31:31.549 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.540+0000 7fae1c9f6640 1 -- 172.21.15.100:0/2357807881 learned_addr learned my addr 172.21.15.100:0/2357807881 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:31:31.549 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.540+0000 7fae177fe640 1 --2- 172.21.15.100:0/2357807881 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55917da54bb0 0x55917da5e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:31.549 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.540+0000 7fae1c9f6640 1 --2- 172.21.15.100:0/2357807881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55917da59d20 0x55917da65960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:31.549 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.540+0000 7fae177fe640 1 -- 172.21.15.100:0/2357807881 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55917da541c0 msgr2=0x55917da5e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:31.549 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.540+0000 7fae177fe640 1 --2- 172.21.15.100:0/2357807881 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55917da541c0 0x55917da5e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:31.549 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.540+0000 7fae177fe640 1 -- 172.21.15.100:0/2357807881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55917da59d20 msgr2=0x55917da65960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:31.549 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.540+0000 7fae177fe640 1 --2- 172.21.15.100:0/2357807881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55917da59d20 0x55917da65960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:31.550 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.540+0000 7fae177fe640 1 -- 172.21.15.100:0/2357807881 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55917d792620 con 0x55917da54bb0 2024-05-10T11:31:31.550 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.540+0000 7fae177fe640 1 --2- 172.21.15.100:0/2357807881 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55917da54bb0 0x55917da5e920 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x55917da50700 tx=0x7fae0c0317a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d925fabfb44e2abb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:31:31.550 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.540+0000 7fadf77fe640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:31:31.550 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.541+0000 7fadf77fe640 1 -- 172.21.15.100:0/2357807881 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae0c035040 con 0x55917da54bb0 2024-05-10T11:31:31.550 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.541+0000 7fadf77fe640 1 -- 172.21.15.100:0/2357807881 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fae0c004240 con 0x55917da54bb0 2024-05-10T11:31:31.550 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.541+0000 7fae2117b740 1 -- 172.21.15.100:0/2357807881 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae10004030 con 0x55917da54bb0 2024-05-10T11:31:31.550 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.541+0000 7fadf77fe640 1 -- 172.21.15.100:0/2357807881 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fae0c04e050 con 0x55917da54bb0 2024-05-10T11:31:31.550 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.541+0000 7fae2117b740 10 client.27386 fetch_fsmap learned FSMap version 33 2024-05-10T11:31:31.550 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.541+0000 7fae2117b740 1 -- 172.21.15.100:0/2357807881 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55917da689b0 con 0x55917da54bb0 2024-05-10T11:31:31.550 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.541+0000 7fadf77fe640 1 -- 172.21.15.100:0/2357807881 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fae10004030 con 0x55917da54bb0 2024-05-10T11:31:31.551 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.541+0000 7fadf77fe640 1 -- 172.21.15.100:0/2357807881 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fae0c03c440 con 0x55917da54bb0 2024-05-10T11:31:31.551 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.541+0000 7fae2117b740 10 client.27386 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T11:31:31.551 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.541+0000 7fae2117b740 10 client.27386 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:31:31.551 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.541+0000 7fae2117b740 20 client.27386 populate_metadata read hostname 'smithi100' 2024-05-10T11:31:31.551 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.541+0000 7fae2117b740 1 --2- 172.21.15.100:0/2357807881 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55917da6a0e0 0x55917da6c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:31.551 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.541+0000 7fae2117b740 4 client.27386 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:31:31.551 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.541+0000 7fae2117b740 1 -- 172.21.15.100:0/2357807881 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55917d9c21e0 con 0x55917da6a0e0 2024-05-10T11:31:31.551 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.541+0000 7fae1c9f6640 1 --2- 172.21.15.100:0/2357807881 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55917da6a0e0 0x55917da6c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:31.551 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.542+0000 7fadf77fe640 10 client.27386 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:31:31.551 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.542+0000 7fae1c9f6640 1 --2- 172.21.15.100:0/2357807881 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55917da6a0e0 0x55917da6c4d0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4448fb5415b5eeec server_cookie=6cbcad09fb17daac in_seq=0 out_seq=0 2024-05-10T11:31:31.551 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.542+0000 7fadf77fe640 1 -- 172.21.15.100:0/2357807881 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55917d9c21e0 con 0x55917da6a0e0 2024-05-10T11:31:31.552 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.542+0000 7fadf77fe640 10 client.27386 handle_command_reply: tid=0 2024-05-10T11:31:31.552 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.542+0000 7fae2117b740 1 client.27386 shutdown 2024-05-10T11:31:31.552 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.543+0000 7fae2117b740 1 -- 172.21.15.100:0/2357807881 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55917da54bb0 msgr2=0x55917da5e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:31.552 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.543+0000 7fae2117b740 1 --2- 172.21.15.100:0/2357807881 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55917da54bb0 0x55917da5e920 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x55917da50700 tx=0x7fae0c0317a0 comp rx=0 tx=0).stop 2024-05-10T11:31:31.552 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.543+0000 7fae2117b740 1 -- 172.21.15.100:0/2357807881 shutdown_connections 2024-05-10T11:31:31.552 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.543+0000 7fae2117b740 1 --2- 172.21.15.100:0/2357807881 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55917da6a0e0 0x55917da6c4d0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:31.552 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.543+0000 7fae2117b740 1 --2- 172.21.15.100:0/2357807881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55917da59d20 0x55917da65960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:31.552 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.543+0000 7fae2117b740 1 --2- 172.21.15.100:0/2357807881 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55917da54bb0 0x55917da5e920 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:31.552 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.543+0000 7fadf77fe640 0 client.27386 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:31:31.552 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.543+0000 7fae2117b740 1 --2- 172.21.15.100:0/2357807881 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55917da541c0 0x55917da5e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:31.552 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.543+0000 7fae2117b740 1 -- 172.21.15.100:0/2357807881 >> 172.21.15.100:0/2357807881 conn(0x55917da4f830 msgr2=0x55917da4a580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:31:31.552 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.543+0000 7fae2117b740 1 -- 172.21.15.100:0/2357807881 shutdown_connections 2024-05-10T11:31:31.552 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.543+0000 7fae2117b740 1 -- 172.21.15.100:0/2357807881 wait complete. 2024-05-10T11:31:31.553 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.543+0000 7fae2117b740 20 client.27386 trim_cache size 0 max 16384 2024-05-10T11:31:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.558+0000 7fae2117b740 1 Processor -- start 2024-05-10T11:31:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.558+0000 7fae2117b740 1 -- start start 2024-05-10T11:31:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.559+0000 7fae2117b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55917da541c0 0x55917da5e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.559+0000 7fae2117b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55917da54bb0 0x55917da5e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.559+0000 7fae2117b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55917da59d20 0x55917da55d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.559+0000 7fae2117b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fae181ab9b0 con 0x55917da541c0 2024-05-10T11:31:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.559+0000 7fae2117b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fae181a9280 con 0x55917da59d20 2024-05-10T11:31:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.559+0000 7fae2117b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fae181a70d0 con 0x55917da54bb0 2024-05-10T11:31:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.559+0000 7fae1c9f6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55917da541c0 0x55917da5e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.559+0000 7fae1c9f6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55917da541c0 0x55917da5e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35930/0 (socket says 172.21.15.100:35930) 2024-05-10T11:31:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.559+0000 7fae1c9f6640 1 -- 172.21.15.100:0/184217415 learned_addr learned my addr 172.21.15.100:0/184217415 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:31:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.559+0000 7fae177fe640 1 --2- 172.21.15.100:0/184217415 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55917da54bb0 0x55917da5e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.559+0000 7fae1c9f6640 1 --2- 172.21.15.100:0/184217415 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55917da59d20 0x55917da55d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.559+0000 7fae1c9f6640 1 -- 172.21.15.100:0/184217415 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55917da54bb0 msgr2=0x55917da5e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.559+0000 7fae1c9f6640 1 --2- 172.21.15.100:0/184217415 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55917da54bb0 0x55917da5e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.559+0000 7fae1c9f6640 1 -- 172.21.15.100:0/184217415 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55917da59d20 msgr2=0x55917da55d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.559+0000 7fae1c9f6640 1 --2- 172.21.15.100:0/184217415 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55917da59d20 0x55917da55d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.559+0000 7fae1c9f6640 1 -- 172.21.15.100:0/184217415 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55917da6cb00 con 0x55917da541c0 2024-05-10T11:31:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.559+0000 7fae1c9f6640 1 --2- 172.21.15.100:0/184217415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55917da541c0 0x55917da5e3e0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x55917da6cd00 tx=0x7fae10009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca2449337ff5e339 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:31:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.559+0000 7fadfdffb640 1 -- 172.21.15.100:0/184217415 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae1001e040 con 0x55917da541c0 2024-05-10T11:31:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.559+0000 7fadfdffb640 1 -- 172.21.15.100:0/184217415 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fae100046a0 con 0x55917da541c0 2024-05-10T11:31:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.559+0000 7fadfdffb640 1 -- 172.21.15.100:0/184217415 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae10005020 con 0x55917da541c0 2024-05-10T11:31:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.560+0000 7fae2117b740 1 -- 172.21.15.100:0/184217415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55917da541c0 msgr2=0x55917da5e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.560+0000 7fae2117b740 1 --2- 172.21.15.100:0/184217415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55917da541c0 0x55917da5e3e0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x55917da6cd00 tx=0x7fae10009500 comp rx=0 tx=0).stop 2024-05-10T11:31:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.560+0000 7fae2117b740 1 -- 172.21.15.100:0/184217415 shutdown_connections 2024-05-10T11:31:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.560+0000 7fae2117b740 1 --2- 172.21.15.100:0/184217415 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55917da59d20 0x55917da55d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.560+0000 7fae2117b740 1 --2- 172.21.15.100:0/184217415 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55917da54bb0 0x55917da5e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.560+0000 7fae2117b740 1 --2- 172.21.15.100:0/184217415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55917da541c0 0x55917da5e3e0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.560+0000 7fae2117b740 1 -- 172.21.15.100:0/184217415 >> 172.21.15.100:0/184217415 conn(0x55917da4f830 msgr2=0x55917da593c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:31:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.560+0000 7fae2117b740 1 -- 172.21.15.100:0/184217415 shutdown_connections 2024-05-10T11:31:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.560+0000 7fae2117b740 1 -- 172.21.15.100:0/184217415 wait complete. 2024-05-10T11:31:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.560+0000 7fae2117b740 1 -- start start 2024-05-10T11:31:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.561+0000 7fae2117b740 1 Processor -- start 2024-05-10T11:31:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.561+0000 7fae2117b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55917da541c0 0x55917da623b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.561+0000 7fae2117b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55917da54bb0 0x55917da628f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.561+0000 7fae2117b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55917da59d20 0x55917da63900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.561+0000 7fae2117b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fae181a7740 con 0x55917da54bb0 2024-05-10T11:31:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.561+0000 7fae2117b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fae181a8bb0 con 0x55917da59d20 2024-05-10T11:31:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.561+0000 7fae2117b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fae181a7e10 con 0x55917da541c0 2024-05-10T11:31:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.561+0000 7fae2117b740 10 client.0 _ll_register_callbacks cb 0x7fae0c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:31:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.561+0000 7fae177fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55917da54bb0 0x55917da628f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.561+0000 7fae177fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55917da54bb0 0x55917da628f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35938/0 (socket says 172.21.15.100:35938) 2024-05-10T11:31:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.561+0000 7fae177fe640 1 -- 172.21.15.100:0/1511400079 learned_addr learned my addr 172.21.15.100:0/1511400079 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:31:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.561+0000 7fae1c9f6640 1 --2- 172.21.15.100:0/1511400079 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55917da541c0 0x55917da623b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.561+0000 7fae1c9f6640 1 --2- 172.21.15.100:0/1511400079 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55917da59d20 0x55917da63900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.561+0000 7fae177fe640 1 -- 172.21.15.100:0/1511400079 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55917da541c0 msgr2=0x55917da623b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.561+0000 7fae177fe640 1 --2- 172.21.15.100:0/1511400079 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55917da541c0 0x55917da623b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.561+0000 7fae177fe640 1 -- 172.21.15.100:0/1511400079 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55917da59d20 msgr2=0x55917da63900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.561+0000 7fae177fe640 1 --2- 172.21.15.100:0/1511400079 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55917da59d20 0x55917da63900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.561+0000 7fae177fe640 1 -- 172.21.15.100:0/1511400079 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55917da63ef0 con 0x55917da54bb0 2024-05-10T11:31:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.561+0000 7fae177fe640 1 --2- 172.21.15.100:0/1511400079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55917da54bb0 0x55917da628f0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fae0c009850 tx=0x7fae0c02ab10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=987bdf4f7716a36f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:31:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.561+0000 7fadf77fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:31:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.561+0000 7fadf77fe640 1 -- 172.21.15.100:0/1511400079 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae0c057060 con 0x55917da54bb0 2024-05-10T11:31:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.561+0000 7fadf77fe640 1 -- 172.21.15.100:0/1511400079 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fae0c03cc70 con 0x55917da54bb0 2024-05-10T11:31:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.561+0000 7fae2117b740 1 -- 172.21.15.100:0/1511400079 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae100046a0 con 0x55917da54bb0 2024-05-10T11:31:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.562+0000 7fadf77fe640 1 -- 172.21.15.100:0/1511400079 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fae0c05e050 con 0x55917da54bb0 2024-05-10T11:31:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.562+0000 7fae2117b740 10 client.18045 fetch_fsmap learned FSMap version 33 2024-05-10T11:31:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.562+0000 7fae2117b740 1 -- 172.21.15.100:0/1511400079 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55917da65270 con 0x55917da54bb0 2024-05-10T11:31:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.562+0000 7fadf77fe640 1 -- 172.21.15.100:0/1511400079 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fae100046a0 con 0x55917da54bb0 2024-05-10T11:31:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.562+0000 7fadf77fe640 1 -- 172.21.15.100:0/1511400079 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fae0c003840 con 0x55917da54bb0 2024-05-10T11:31:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.562+0000 7fae2117b740 10 client.18045 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T11:31:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.562+0000 7fae2117b740 10 client.18045 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:31:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.562+0000 7fae2117b740 20 client.18045 populate_metadata read hostname 'smithi100' 2024-05-10T11:31:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.562+0000 7fae2117b740 1 --2- 172.21.15.100:0/1511400079 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55917dad5000 0x55917dad7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:31.569 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.562+0000 7fae2117b740 4 client.18045 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:31:31.569 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.562+0000 7fae2117b740 1 -- 172.21.15.100:0/1511400079 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55917d9bc050 con 0x55917dad5000 2024-05-10T11:31:31.569 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.562+0000 7fae1c9f6640 1 --2- 172.21.15.100:0/1511400079 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55917dad5000 0x55917dad7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:31.569 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.563+0000 7fadf77fe640 10 client.18045 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:31:31.569 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.563+0000 7fae1c9f6640 1 --2- 172.21.15.100:0/1511400079 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55917dad5000 0x55917dad7420 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95e275fdb5d97ab6 server_cookie=a64c1ff2865d0b5e in_seq=0 out_seq=0 2024-05-10T11:31:31.588 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.586+0000 7fadf77fe640 1 -- 172.21.15.100:0/1511400079 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55917d9bc050 con 0x55917dad5000 2024-05-10T11:31:31.588 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.586+0000 7fadf77fe640 10 client.18045 handle_command_reply: tid=0 2024-05-10T11:31:31.588 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.586+0000 7fae2117b740 1 client.18045 shutdown 2024-05-10T11:31:31.588 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.586+0000 7fae2117b740 1 -- 172.21.15.100:0/1511400079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55917da54bb0 msgr2=0x55917da628f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:31.589 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.586+0000 7fae2117b740 1 --2- 172.21.15.100:0/1511400079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55917da54bb0 0x55917da628f0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fae0c009850 tx=0x7fae0c02ab10 comp rx=0 tx=0).stop 2024-05-10T11:31:31.589 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.587+0000 7fae2117b740 1 -- 172.21.15.100:0/1511400079 shutdown_connections 2024-05-10T11:31:31.589 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.587+0000 7fae2117b740 1 --2- 172.21.15.100:0/1511400079 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55917dad5000 0x55917dad7420 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:31.589 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.587+0000 7fae2117b740 1 --2- 172.21.15.100:0/1511400079 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55917da59d20 0x55917da63900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:31.589 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.587+0000 7fae2117b740 1 --2- 172.21.15.100:0/1511400079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55917da54bb0 0x55917da628f0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:31.589 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.587+0000 7fadf77fe640 0 client.18045 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:31:31.589 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.587+0000 7fae2117b740 1 --2- 172.21.15.100:0/1511400079 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55917da541c0 0x55917da623b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:31.589 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.587+0000 7fae2117b740 1 -- 172.21.15.100:0/1511400079 >> 172.21.15.100:0/1511400079 conn(0x55917da4f830 msgr2=0x55917da49cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:31:31.589 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.587+0000 7fae2117b740 1 -- 172.21.15.100:0/1511400079 shutdown_connections 2024-05-10T11:31:31.589 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.587+0000 7fae2117b740 1 -- 172.21.15.100:0/1511400079 wait complete. 2024-05-10T11:31:31.589 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.587+0000 7fae2117b740 20 client.18045 trim_cache size 0 max 16384 2024-05-10T11:31:31.590 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:31:31.590 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:31:31.590 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:31:31.590 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:31:31.591 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.587+0000 7fae1e459640 1 -- 172.21.15.100:0/2882105171 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fae000b2920 msgr2=0x7fae000b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:31.591 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.587+0000 7fae1e459640 1 --2- 172.21.15.100:0/2882105171 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fae000b2920 0x7fae000b4de0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fae080a98a0 tx=0x7fae080cc040 comp rx=0 tx=0).stop 2024-05-10T11:31:31.591 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.587+0000 7fae1e459640 1 -- 172.21.15.100:0/2882105171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae18101910 msgr2=0x7fae18198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:31.591 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.587+0000 7fae1e459640 1 --2- 172.21.15.100:0/2882105171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae18101910 0x7fae18198af0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fae0800e9c0 tx=0x7fae0800ee80 comp rx=0 tx=0).stop 2024-05-10T11:31:31.591 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.588+0000 7fae1e459640 1 -- 172.21.15.100:0/2882105171 shutdown_connections 2024-05-10T11:31:31.591 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.588+0000 7fae1e459640 1 --2- 172.21.15.100:0/2882105171 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fae000b2920 0x7fae000b4de0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:31.591 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.588+0000 7fae1e459640 1 --2- 172.21.15.100:0/2882105171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae181034c0 0x7fae181a00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:31.591 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.588+0000 7fae1e459640 1 --2- 172.21.15.100:0/2882105171 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae18102b00 0x7fae18199030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:31.591 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.588+0000 7fae1e459640 1 --2- 172.21.15.100:0/2882105171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae18101910 0x7fae18198af0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:31.591 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.588+0000 7fae1e459640 1 -- 172.21.15.100:0/2882105171 >> 172.21.15.100:0/2882105171 conn(0x7fae180fce80 msgr2=0x7fae1806a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:31:31.591 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.588+0000 7fae1e459640 1 -- 172.21.15.100:0/2882105171 shutdown_connections 2024-05-10T11:31:31.591 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:31.588+0000 7fae1e459640 1 -- 172.21.15.100:0/2882105171 wait complete. 2024-05-10T11:31:31.985 INFO:tasks.workunit.client.0.smithi100.stdout: 1 347978 2.70 MB/sec execute 566 sec latency 98.723 ms 2024-05-10T11:31:32.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:31:31 smithi178 ceph-mon[33359]: pgmap v815: 129 pgs: 129 active+clean; 134 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 1.1 MiB/s wr, 128 op/s 2024-05-10T11:31:32.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:31:31 smithi188 ceph-mon[34075]: pgmap v815: 129 pgs: 129 active+clean; 134 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 1.1 MiB/s wr, 128 op/s 2024-05-10T11:31:32.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:31:31 smithi100 ceph-mon[30296]: pgmap v815: 129 pgs: 129 active+clean; 134 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 1.1 MiB/s wr, 128 op/s 2024-05-10T11:31:32.985 INFO:tasks.workunit.client.0.smithi100.stdout: 1 348163 2.70 MB/sec execute 567 sec latency 114.557 ms 2024-05-10T11:31:33.985 INFO:tasks.workunit.client.0.smithi100.stdout: 1 348485 2.70 MB/sec execute 568 sec latency 90.372 ms 2024-05-10T11:31:34.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:31:33 smithi178 ceph-mon[33359]: pgmap v816: 129 pgs: 129 active+clean; 134 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 1.2 MiB/s wr, 154 op/s 2024-05-10T11:31:34.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:31:33 smithi188 ceph-mon[34075]: pgmap v816: 129 pgs: 129 active+clean; 134 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 1.2 MiB/s wr, 154 op/s 2024-05-10T11:31:34.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:31:33 smithi100 ceph-mon[30296]: pgmap v816: 129 pgs: 129 active+clean; 134 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 1.2 MiB/s wr, 154 op/s 2024-05-10T11:31:34.985 INFO:tasks.workunit.client.0.smithi100.stdout: 1 348995 2.70 MB/sec execute 569 sec latency 92.099 ms 2024-05-10T11:31:35.878 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:31:35 smithi100 ceph-mon[30296]: pgmap v817: 129 pgs: 129 active+clean; 136 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 1.4 MiB/s wr, 143 op/s 2024-05-10T11:31:35.985 INFO:tasks.workunit.client.0.smithi100.stdout: 1 349578 2.69 MB/sec execute 570 sec latency 15.290 ms 2024-05-10T11:31:36.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:31:35 smithi178 ceph-mon[33359]: pgmap v817: 129 pgs: 129 active+clean; 136 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 1.4 MiB/s wr, 143 op/s 2024-05-10T11:31:36.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:31:35 smithi188 ceph-mon[34075]: pgmap v817: 129 pgs: 129 active+clean; 136 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 1.4 MiB/s wr, 143 op/s 2024-05-10T11:31:36.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.142+0000 7f37301e4700 1 -- 172.21.15.100:0/4212794730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3728103ee0 msgr2=0x7f3728110370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:36.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.142+0000 7f37301e4700 1 --2- 172.21.15.100:0/4212794730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3728103ee0 0x7f3728110370 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f372000b0a0 tx=0x7f372000b3b0 comp rx=0 tx=0).stop 2024-05-10T11:31:36.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.142+0000 7f37301e4700 1 -- 172.21.15.100:0/4212794730 shutdown_connections 2024-05-10T11:31:36.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.142+0000 7f37301e4700 1 --2- 172.21.15.100:0/4212794730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3728103ee0 0x7f3728110370 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:36.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.142+0000 7f37301e4700 1 --2- 172.21.15.100:0/4212794730 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3728109530 0x7f3728109910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:36.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.142+0000 7f37301e4700 1 --2- 172.21.15.100:0/4212794730 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3728103520 0x7f37281039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:36.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.142+0000 7f37301e4700 1 -- 172.21.15.100:0/4212794730 >> 172.21.15.100:0/4212794730 conn(0x7f372807a650 msgr2=0x7f372807aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:31:36.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.143+0000 7f37301e4700 1 -- 172.21.15.100:0/4212794730 shutdown_connections 2024-05-10T11:31:36.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.143+0000 7f37301e4700 1 -- 172.21.15.100:0/4212794730 wait complete. 2024-05-10T11:31:36.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.143+0000 7f37301e4700 1 Processor -- start 2024-05-10T11:31:36.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.143+0000 7f37301e4700 1 -- start start 2024-05-10T11:31:36.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.144+0000 7f37301e4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3728103520 0x7f372819f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:36.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.144+0000 7f37301e4700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3728103ee0 0x7f372819fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:36.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.144+0000 7f37301e4700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3728109530 0x7f3728199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:36.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.144+0000 7f37301e4700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3728113580 con 0x7f3728103520 2024-05-10T11:31:36.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.144+0000 7f37301e4700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3728113400 con 0x7f3728109530 2024-05-10T11:31:36.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.144+0000 7f37301e4700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3728113700 con 0x7f3728103ee0 2024-05-10T11:31:36.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.144+0000 7f372df80700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3728103520 0x7f372819f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:36.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.144+0000 7f372d77f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3728103ee0 0x7f372819fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:36.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.144+0000 7f372df80700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3728103520 0x7f372819f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35952/0 (socket says 172.21.15.100:35952) 2024-05-10T11:31:36.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.144+0000 7f372d77f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3728103ee0 0x7f372819fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:43842/0 (socket says 172.21.15.100:43842) 2024-05-10T11:31:36.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.144+0000 7f372df80700 1 -- 172.21.15.100:0/2605977551 learned_addr learned my addr 172.21.15.100:0/2605977551 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:31:36.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.144+0000 7f372e781700 1 --2- 172.21.15.100:0/2605977551 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3728109530 0x7f3728199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:36.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.144+0000 7f372e781700 1 -- 172.21.15.100:0/2605977551 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3728103ee0 msgr2=0x7f372819fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:36.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.144+0000 7f372e781700 1 --2- 172.21.15.100:0/2605977551 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3728103ee0 0x7f372819fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:36.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.144+0000 7f372e781700 1 -- 172.21.15.100:0/2605977551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3728103520 msgr2=0x7f372819f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:36.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.144+0000 7f372e781700 1 --2- 172.21.15.100:0/2605977551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3728103520 0x7f372819f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:36.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.144+0000 7f372e781700 1 -- 172.21.15.100:0/2605977551 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3728199fc0 con 0x7f3728109530 2024-05-10T11:31:36.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.144+0000 7f372df80700 1 --2- 172.21.15.100:0/2605977551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3728103520 0x7f372819f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:31:36.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.144+0000 7f372d77f700 1 --2- 172.21.15.100:0/2605977551 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3728103ee0 0x7f372819fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:31:36.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.144+0000 7f372e781700 1 --2- 172.21.15.100:0/2605977551 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3728109530 0x7f3728199860 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f372000b070 tx=0x7f3720004c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:31:36.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.145+0000 7f371effd700 1 -- 172.21.15.100:0/2605977551 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3720021070 con 0x7f3728109530 2024-05-10T11:31:36.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.145+0000 7f37301e4700 1 -- 172.21.15.100:0/2605977551 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f372819a250 con 0x7f3728109530 2024-05-10T11:31:36.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.145+0000 7f37301e4700 1 -- 172.21.15.100:0/2605977551 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37280756d0 con 0x7f3728109530 2024-05-10T11:31:36.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.145+0000 7f371effd700 1 -- 172.21.15.100:0/2605977551 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3720007b80 con 0x7f3728109530 2024-05-10T11:31:36.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.145+0000 7f371effd700 1 -- 172.21.15.100:0/2605977551 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372001de10 con 0x7f3728109530 2024-05-10T11:31:36.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.147+0000 7f371effd700 1 -- 172.21.15.100:0/2605977551 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f3720024070 con 0x7f3728109530 2024-05-10T11:31:36.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.148+0000 7f371effd700 1 --2- 172.21.15.100:0/2605977551 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f37140b2900 0x7f37140b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:36.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.148+0000 7f372df80700 1 --2- 172.21.15.100:0/2605977551 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f37140b2900 0x7f37140b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:36.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.148+0000 7f371effd700 1 -- 172.21.15.100:0/2605977551 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f372000f020 con 0x7f3728109530 2024-05-10T11:31:36.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.148+0000 7f372df80700 1 --2- 172.21.15.100:0/2605977551 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f37140b2900 0x7f37140b4dc0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f372819a9a0 tx=0x7f3724009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:31:36.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.149+0000 7f373aa77b80 1 Processor -- start 2024-05-10T11:31:36.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.149+0000 7f373aa77b80 1 -- start start 2024-05-10T11:31:36.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.149+0000 7f373aa77b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574c78414d0 0x5574c78418b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:36.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.149+0000 7f373aa77b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5574c7841df0 0x5574c78456f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:36.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.149+0000 7f373aa77b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574c7845c80 0x5574c7846100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:36.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.149+0000 7f373aa77b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5574c7793860 con 0x5574c78414d0 2024-05-10T11:31:36.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.149+0000 7f373aa77b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5574c77327c0 con 0x5574c7841df0 2024-05-10T11:31:36.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.149+0000 7f373aa77b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5574c778f820 con 0x5574c7845c80 2024-05-10T11:31:36.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.149+0000 7f372df80700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574c78414d0 0x5574c78418b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:36.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.149+0000 7f372d77f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5574c7841df0 0x5574c78456f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:36.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.149+0000 7f372e781700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574c7845c80 0x5574c7846100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:36.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.149+0000 7f372d77f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5574c7841df0 0x5574c78456f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:38542/0 (socket says 172.21.15.100:38542) 2024-05-10T11:31:36.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.149+0000 7f372d77f700 1 -- 172.21.15.100:0/2287561566 learned_addr learned my addr 172.21.15.100:0/2287561566 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:31:36.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.149+0000 7f372e781700 1 -- 172.21.15.100:0/2287561566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5574c7841df0 msgr2=0x5574c78456f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:36.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.149+0000 7f372e781700 1 --2- 172.21.15.100:0/2287561566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5574c7841df0 0x5574c78456f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:36.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.149+0000 7f372e781700 1 -- 172.21.15.100:0/2287561566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574c78414d0 msgr2=0x5574c78418b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:36.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.149+0000 7f372e781700 1 --2- 172.21.15.100:0/2287561566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574c78414d0 0x5574c78418b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:36.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.149+0000 7f372e781700 1 -- 172.21.15.100:0/2287561566 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574c76e08d0 con 0x5574c7845c80 2024-05-10T11:31:36.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.150+0000 7f372e781700 1 --2- 172.21.15.100:0/2287561566 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574c7845c80 0x5574c7846100 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f372000b780 tx=0x7f372000cf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae0593a8282e6641 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:31:36.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.150+0000 7f3713fff700 1 -- 172.21.15.100:0/2287561566 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37200aab20 con 0x5574c7845c80 2024-05-10T11:31:36.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.150+0000 7f3713fff700 1 -- 172.21.15.100:0/2287561566 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f37200aeed0 con 0x5574c7845c80 2024-05-10T11:31:36.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.150+0000 7f3713fff700 1 -- 172.21.15.100:0/2287561566 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37200a6ef0 con 0x5574c7845c80 2024-05-10T11:31:36.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.151+0000 7f373aa77b80 1 -- 172.21.15.100:0/2287561566 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574c7845c80 msgr2=0x5574c7846100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:36.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.151+0000 7f373aa77b80 1 --2- 172.21.15.100:0/2287561566 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574c7845c80 0x5574c7846100 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f372000b780 tx=0x7f372000cf90 comp rx=0 tx=0).stop 2024-05-10T11:31:36.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.151+0000 7f373aa77b80 1 -- 172.21.15.100:0/2287561566 shutdown_connections 2024-05-10T11:31:36.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.151+0000 7f373aa77b80 1 --2- 172.21.15.100:0/2287561566 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574c7845c80 0x5574c7846100 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:36.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.151+0000 7f373aa77b80 1 --2- 172.21.15.100:0/2287561566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574c78414d0 0x5574c78418b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:36.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.151+0000 7f373aa77b80 1 --2- 172.21.15.100:0/2287561566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5574c7841df0 0x5574c78456f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:36.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.151+0000 7f373aa77b80 1 -- 172.21.15.100:0/2287561566 >> 172.21.15.100:0/2287561566 conn(0x5574c776d100 msgr2=0x5574c783bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:31:36.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.151+0000 7f373aa77b80 1 -- 172.21.15.100:0/2287561566 shutdown_connections 2024-05-10T11:31:36.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.151+0000 7f373aa77b80 1 -- 172.21.15.100:0/2287561566 wait complete. 2024-05-10T11:31:36.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.151+0000 7f373aa77b80 1 -- start start 2024-05-10T11:31:36.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.151+0000 7f373aa77b80 1 Processor -- start 2024-05-10T11:31:36.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.152+0000 7f373aa77b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574c78414d0 0x5574c784d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:36.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.152+0000 7f373aa77b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5574c7841df0 0x5574c784a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:36.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.152+0000 7f373aa77b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574c7845c80 0x5574c784a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:36.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.152+0000 7f373aa77b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5574c784dc50 con 0x5574c78414d0 2024-05-10T11:31:36.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.152+0000 7f373aa77b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5574c76f9c30 con 0x5574c7841df0 2024-05-10T11:31:36.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.152+0000 7f373aa77b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5574c784ddd0 con 0x5574c7845c80 2024-05-10T11:31:36.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.152+0000 7f372df80700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574c78414d0 0x5574c784d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:36.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.152+0000 7f372df80700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574c78414d0 0x5574c784d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35972/0 (socket says 172.21.15.100:35972) 2024-05-10T11:31:36.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.152+0000 7f372df80700 1 -- 172.21.15.100:0/1255790103 learned_addr learned my addr 172.21.15.100:0/1255790103 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:31:36.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.152+0000 7f372d77f700 1 --2- 172.21.15.100:0/1255790103 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5574c7841df0 0x5574c784a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:36.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.152+0000 7f372e781700 1 --2- 172.21.15.100:0/1255790103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574c7845c80 0x5574c784a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:36.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.152+0000 7f373aa77b80 10 client.0 _ll_register_callbacks cb 0x5574c772b920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:31:36.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.152+0000 7f372df80700 1 -- 172.21.15.100:0/1255790103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574c7845c80 msgr2=0x5574c784a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:36.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.152+0000 7f372df80700 1 --2- 172.21.15.100:0/1255790103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574c7845c80 0x5574c784a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:36.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.152+0000 7f372df80700 1 -- 172.21.15.100:0/1255790103 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5574c7841df0 msgr2=0x5574c784a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:36.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.152+0000 7f372df80700 1 --2- 172.21.15.100:0/1255790103 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5574c7841df0 0x5574c784a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:36.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.152+0000 7f372df80700 1 -- 172.21.15.100:0/1255790103 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574c772f690 con 0x5574c78414d0 2024-05-10T11:31:36.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.152+0000 7f372d77f700 1 --2- 172.21.15.100:0/1255790103 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5574c7841df0 0x5574c784a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:31:36.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.153+0000 7f372df80700 1 --2- 172.21.15.100:0/1255790103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574c78414d0 0x5574c784d050 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f372401cd80 tx=0x7f37240047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e31520a473d3581e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:31:36.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.153+0000 7f37117fa700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:31:36.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.153+0000 7f37117fa700 1 -- 172.21.15.100:0/1255790103 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372402b060 con 0x5574c78414d0 2024-05-10T11:31:36.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.153+0000 7f37117fa700 1 -- 172.21.15.100:0/1255790103 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f372401dd10 con 0x5574c78414d0 2024-05-10T11:31:36.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.153+0000 7f373aa77b80 1 -- 172.21.15.100:0/1255790103 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37200aeed0 con 0x5574c78414d0 2024-05-10T11:31:36.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.153+0000 7f37117fa700 1 -- 172.21.15.100:0/1255790103 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f3724032050 con 0x5574c78414d0 2024-05-10T11:31:36.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.153+0000 7f373aa77b80 10 client.18057 fetch_fsmap learned FSMap version 33 2024-05-10T11:31:36.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.153+0000 7f373aa77b80 1 -- 172.21.15.100:0/1255790103 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5574c78583d0 con 0x5574c78414d0 2024-05-10T11:31:36.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.153+0000 7f37117fa700 1 -- 172.21.15.100:0/1255790103 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f37200aeed0 con 0x5574c78414d0 2024-05-10T11:31:36.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.153+0000 7f37117fa700 1 -- 172.21.15.100:0/1255790103 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f372401a390 con 0x5574c78414d0 2024-05-10T11:31:36.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.153+0000 7f373aa77b80 10 client.18057 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T11:31:36.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.153+0000 7f373aa77b80 10 client.18057 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:31:36.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.154+0000 7f373aa77b80 20 client.18057 populate_metadata read hostname 'smithi100' 2024-05-10T11:31:36.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.154+0000 7f373aa77b80 1 --2- 172.21.15.100:0/1255790103 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5574c785aa60 0x5574c785ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:36.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.154+0000 7f373aa77b80 4 client.18057 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:31:36.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.154+0000 7f373aa77b80 1 -- 172.21.15.100:0/1255790103 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574c780d650 con 0x5574c785aa60 2024-05-10T11:31:36.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.154+0000 7f372d77f700 1 --2- 172.21.15.100:0/1255790103 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5574c785aa60 0x5574c785ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:36.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.154+0000 7f37117fa700 10 client.18057 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:31:36.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.154+0000 7f372d77f700 1 --2- 172.21.15.100:0/1255790103 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5574c785aa60 0x5574c785ced0 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8962d6aaca6da540 server_cookie=22ddef51259e477a in_seq=0 out_seq=0 2024-05-10T11:31:36.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.155+0000 7f37117fa700 1 -- 172.21.15.100:0/1255790103 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5574c780d650 con 0x5574c785aa60 2024-05-10T11:31:36.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.155+0000 7f37117fa700 10 client.18057 handle_command_reply: tid=0 2024-05-10T11:31:36.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.155+0000 7f373aa77b80 1 client.18057 shutdown 2024-05-10T11:31:36.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.155+0000 7f373aa77b80 1 -- 172.21.15.100:0/1255790103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574c78414d0 msgr2=0x5574c784d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:36.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.155+0000 7f373aa77b80 1 --2- 172.21.15.100:0/1255790103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574c78414d0 0x5574c784d050 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f372401cd80 tx=0x7f37240047c0 comp rx=0 tx=0).stop 2024-05-10T11:31:36.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.155+0000 7f373aa77b80 1 -- 172.21.15.100:0/1255790103 shutdown_connections 2024-05-10T11:31:36.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.155+0000 7f373aa77b80 1 --2- 172.21.15.100:0/1255790103 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5574c785aa60 0x5574c785ced0 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:36.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.155+0000 7f373aa77b80 1 --2- 172.21.15.100:0/1255790103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574c7845c80 0x5574c784a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:36.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.155+0000 7f373aa77b80 1 --2- 172.21.15.100:0/1255790103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574c78414d0 0x5574c784d050 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:36.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.155+0000 7f37117fa700 0 client.18057 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:31:36.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.155+0000 7f373aa77b80 1 --2- 172.21.15.100:0/1255790103 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5574c7841df0 0x5574c784a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:36.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.155+0000 7f373aa77b80 1 -- 172.21.15.100:0/1255790103 >> 172.21.15.100:0/1255790103 conn(0x5574c776d100 msgr2=0x5574c7836720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:31:36.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.155+0000 7f373aa77b80 1 -- 172.21.15.100:0/1255790103 shutdown_connections 2024-05-10T11:31:36.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.155+0000 7f373aa77b80 1 -- 172.21.15.100:0/1255790103 wait complete. 2024-05-10T11:31:36.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.155+0000 7f373aa77b80 20 client.18057 trim_cache size 0 max 16384 2024-05-10T11:31:36.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.180+0000 7f373aa77b80 1 Processor -- start 2024-05-10T11:31:36.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.180+0000 7f373aa77b80 1 -- start start 2024-05-10T11:31:36.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.180+0000 7f373aa77b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5574c78414d0 0x5574c7856ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:36.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.180+0000 7f373aa77b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574c7841df0 0x5574c784cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:36.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.180+0000 7f373aa77b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574c7845c80 0x5574c78409f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:36.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.180+0000 7f373aa77b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f37281b3a90 con 0x5574c7841df0 2024-05-10T11:31:36.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.180+0000 7f373aa77b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f37281b1320 con 0x5574c78414d0 2024-05-10T11:31:36.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.180+0000 7f373aa77b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f37281af170 con 0x5574c7845c80 2024-05-10T11:31:36.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.180+0000 7f372e781700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574c7845c80 0x5574c78409f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:36.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.180+0000 7f372e781700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574c7845c80 0x5574c78409f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:43856/0 (socket says 172.21.15.100:43856) 2024-05-10T11:31:36.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.180+0000 7f372d77f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574c7841df0 0x5574c784cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:36.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.180+0000 7f372e781700 1 -- 172.21.15.100:0/3979323775 learned_addr learned my addr 172.21.15.100:0/3979323775 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:31:36.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.180+0000 7f372df80700 1 --2- 172.21.15.100:0/3979323775 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5574c78414d0 0x5574c7856ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:36.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.181+0000 7f372e781700 1 -- 172.21.15.100:0/3979323775 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5574c78414d0 msgr2=0x5574c7856ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:36.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.181+0000 7f372e781700 1 --2- 172.21.15.100:0/3979323775 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5574c78414d0 0x5574c7856ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:36.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.181+0000 7f372e781700 1 -- 172.21.15.100:0/3979323775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574c7841df0 msgr2=0x5574c784cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:36.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.181+0000 7f372e781700 1 --2- 172.21.15.100:0/3979323775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574c7841df0 0x5574c784cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:36.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.181+0000 7f372e781700 1 -- 172.21.15.100:0/3979323775 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574c7845a30 con 0x5574c7845c80 2024-05-10T11:31:36.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.181+0000 7f372d77f700 1 --2- 172.21.15.100:0/3979323775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574c7841df0 0x5574c784cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:31:36.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.181+0000 7f372e781700 1 --2- 172.21.15.100:0/3979323775 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574c7845c80 0x5574c78409f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3720009e60 tx=0x7f37200cd040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc0c980c4d762dca server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:31:36.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.181+0000 7f37137fe700 1 -- 172.21.15.100:0/3979323775 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37200c5170 con 0x5574c7845c80 2024-05-10T11:31:36.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.181+0000 7f37137fe700 1 -- 172.21.15.100:0/3979323775 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f37200abe50 con 0x5574c7845c80 2024-05-10T11:31:36.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.181+0000 7f37137fe700 1 -- 172.21.15.100:0/3979323775 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37200a6db0 con 0x5574c7845c80 2024-05-10T11:31:36.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.182+0000 7f373aa77b80 1 -- 172.21.15.100:0/3979323775 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574c7845c80 msgr2=0x5574c78409f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:36.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.182+0000 7f373aa77b80 1 --2- 172.21.15.100:0/3979323775 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574c7845c80 0x5574c78409f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3720009e60 tx=0x7f37200cd040 comp rx=0 tx=0).stop 2024-05-10T11:31:36.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.182+0000 7f373aa77b80 1 -- 172.21.15.100:0/3979323775 shutdown_connections 2024-05-10T11:31:36.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.182+0000 7f373aa77b80 1 --2- 172.21.15.100:0/3979323775 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574c7845c80 0x5574c78409f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:36.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.182+0000 7f373aa77b80 1 --2- 172.21.15.100:0/3979323775 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5574c78414d0 0x5574c7856ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:36.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.182+0000 7f373aa77b80 1 --2- 172.21.15.100:0/3979323775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574c7841df0 0x5574c784cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:36.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.182+0000 7f373aa77b80 1 -- 172.21.15.100:0/3979323775 >> 172.21.15.100:0/3979323775 conn(0x5574c776d100 msgr2=0x5574c78402e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:31:36.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.182+0000 7f373aa77b80 1 -- 172.21.15.100:0/3979323775 shutdown_connections 2024-05-10T11:31:36.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.182+0000 7f373aa77b80 1 -- 172.21.15.100:0/3979323775 wait complete. 2024-05-10T11:31:36.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.183+0000 7f373aa77b80 1 -- start start 2024-05-10T11:31:36.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.183+0000 7f373aa77b80 1 Processor -- start 2024-05-10T11:31:36.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.183+0000 7f373aa77b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574c78414d0 0x5574c784f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:36.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.183+0000 7f373aa77b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5574c7841df0 0x5574c784f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:36.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.183+0000 7f373aa77b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574c7845c80 0x5574c78c4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:36.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.183+0000 7f373aa77b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f37281af7e0 con 0x5574c7845c80 2024-05-10T11:31:36.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.183+0000 7f373aa77b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f37281b0c50 con 0x5574c7841df0 2024-05-10T11:31:36.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.183+0000 7f373aa77b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f37281afeb0 con 0x5574c78414d0 2024-05-10T11:31:36.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.183+0000 7f372df80700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574c78414d0 0x5574c784f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:36.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.183+0000 7f372e781700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574c7845c80 0x5574c78c4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:36.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.183+0000 7f372df80700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574c78414d0 0x5574c784f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:43866/0 (socket says 172.21.15.100:43866) 2024-05-10T11:31:36.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.183+0000 7f372df80700 1 -- 172.21.15.100:0/3947061315 learned_addr learned my addr 172.21.15.100:0/3947061315 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:31:36.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.183+0000 7f372d77f700 1 --2- 172.21.15.100:0/3947061315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5574c7841df0 0x5574c784f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:36.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.183+0000 7f373aa77b80 10 client.0 _ll_register_callbacks cb 0x5574c772b920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:31:36.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.184+0000 7f372e781700 1 -- 172.21.15.100:0/3947061315 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574c78414d0 msgr2=0x5574c784f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:36.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.184+0000 7f372e781700 1 --2- 172.21.15.100:0/3947061315 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574c78414d0 0x5574c784f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:36.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.184+0000 7f372e781700 1 -- 172.21.15.100:0/3947061315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5574c7841df0 msgr2=0x5574c784f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:36.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.184+0000 7f372e781700 1 --2- 172.21.15.100:0/3947061315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5574c7841df0 0x5574c784f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:36.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.184+0000 7f372e781700 1 -- 172.21.15.100:0/3947061315 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574c78c48d0 con 0x5574c7845c80 2024-05-10T11:31:36.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.184+0000 7f372df80700 1 --2- 172.21.15.100:0/3947061315 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574c78414d0 0x5574c784f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:31:36.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.184+0000 7f372d77f700 1 --2- 172.21.15.100:0/3947061315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5574c7841df0 0x5574c784f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:31:36.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.184+0000 7f372e781700 1 --2- 172.21.15.100:0/3947061315 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574c7845c80 0x5574c78c4390 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f37200aaab0 tx=0x7f37200095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2f262b3a2961d50 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:31:36.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.184+0000 7f37117fa700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:31:36.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.184+0000 7f37117fa700 1 -- 172.21.15.100:0/3947061315 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37200c5170 con 0x5574c7845c80 2024-05-10T11:31:36.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.184+0000 7f37117fa700 1 -- 172.21.15.100:0/3947061315 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f3720007d30 con 0x5574c7845c80 2024-05-10T11:31:36.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.184+0000 7f373aa77b80 1 -- 172.21.15.100:0/3947061315 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37200abe50 con 0x5574c7845c80 2024-05-10T11:31:36.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.184+0000 7f37117fa700 1 -- 172.21.15.100:0/3947061315 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f37200aab20 con 0x5574c7845c80 2024-05-10T11:31:36.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.184+0000 7f373aa77b80 10 client.18063 fetch_fsmap learned FSMap version 33 2024-05-10T11:31:36.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.185+0000 7f373aa77b80 1 -- 172.21.15.100:0/3947061315 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5574c78c5ad0 con 0x5574c7845c80 2024-05-10T11:31:36.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.185+0000 7f37117fa700 1 -- 172.21.15.100:0/3947061315 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f37200abe50 con 0x5574c7845c80 2024-05-10T11:31:36.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.185+0000 7f37117fa700 1 -- 172.21.15.100:0/3947061315 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f3720011070 con 0x5574c7845c80 2024-05-10T11:31:36.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.185+0000 7f373aa77b80 10 client.18063 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T11:31:36.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.185+0000 7f373aa77b80 10 client.18063 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:31:36.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.185+0000 7f373aa77b80 20 client.18063 populate_metadata read hostname 'smithi100' 2024-05-10T11:31:36.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.185+0000 7f373aa77b80 1 --2- 172.21.15.100:0/3947061315 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5574c78c5da0 0x5574c78c8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:36.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.185+0000 7f373aa77b80 4 client.18063 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T11:31:36.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.185+0000 7f373aa77b80 1 -- 172.21.15.100:0/3947061315 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5574c77a1240 con 0x5574c78c5da0 2024-05-10T11:31:36.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.185+0000 7f372df80700 1 --2- 172.21.15.100:0/3947061315 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5574c78c5da0 0x5574c78c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:36.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.186+0000 7f37117fa700 10 client.18063 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:31:36.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.186+0000 7f372df80700 1 --2- 172.21.15.100:0/3947061315 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5574c78c5da0 0x5574c78c8190 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfa281c3ad63f27b server_cookie=58e1705e8dac3895 in_seq=0 out_seq=0 2024-05-10T11:31:36.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.186+0000 7f37117fa700 1 -- 172.21.15.100:0/3947061315 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5574c77a1240 con 0x5574c78c5da0 2024-05-10T11:31:36.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.186+0000 7f37117fa700 10 client.18063 handle_command_reply: tid=0 2024-05-10T11:31:36.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.186+0000 7f373aa77b80 1 client.18063 shutdown 2024-05-10T11:31:36.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.186+0000 7f373aa77b80 1 -- 172.21.15.100:0/3947061315 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574c7845c80 msgr2=0x5574c78c4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:36.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.186+0000 7f373aa77b80 1 --2- 172.21.15.100:0/3947061315 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574c7845c80 0x5574c78c4390 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f37200aaab0 tx=0x7f37200095a0 comp rx=0 tx=0).stop 2024-05-10T11:31:36.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.186+0000 7f373aa77b80 1 -- 172.21.15.100:0/3947061315 shutdown_connections 2024-05-10T11:31:36.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.186+0000 7f373aa77b80 1 --2- 172.21.15.100:0/3947061315 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5574c78c5da0 0x5574c78c8190 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:36.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.186+0000 7f373aa77b80 1 --2- 172.21.15.100:0/3947061315 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574c7845c80 0x5574c78c4390 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:36.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.186+0000 7f373aa77b80 1 --2- 172.21.15.100:0/3947061315 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574c78414d0 0x5574c784f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:36.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.186+0000 7f373aa77b80 1 --2- 172.21.15.100:0/3947061315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5574c7841df0 0x5574c784f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:36.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.186+0000 7f37117fa700 0 client.18063 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:31:36.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.186+0000 7f373aa77b80 1 -- 172.21.15.100:0/3947061315 >> 172.21.15.100:0/3947061315 conn(0x5574c776d100 msgr2=0x5574c7840b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:31:36.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.186+0000 7f373aa77b80 1 -- 172.21.15.100:0/3947061315 shutdown_connections 2024-05-10T11:31:36.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.186+0000 7f373aa77b80 1 -- 172.21.15.100:0/3947061315 wait complete. 2024-05-10T11:31:36.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.187+0000 7f373aa77b80 20 client.18063 trim_cache size 0 max 16384 2024-05-10T11:31:36.196 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:31:36.197 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-05-10T11:31:36.197 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-05-10T11:31:36.197 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:31:36.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.187+0000 7f37301e4700 1 -- 172.21.15.100:0/2605977551 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f37140b2900 msgr2=0x7f37140b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:36.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.187+0000 7f37301e4700 1 --2- 172.21.15.100:0/2605977551 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f37140b2900 0x7f37140b4dc0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f372819a9a0 tx=0x7f3724009260 comp rx=0 tx=0).stop 2024-05-10T11:31:36.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.187+0000 7f37301e4700 1 -- 172.21.15.100:0/2605977551 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3728109530 msgr2=0x7f3728199860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:36.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.187+0000 7f37301e4700 1 --2- 172.21.15.100:0/2605977551 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3728109530 0x7f3728199860 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f372000b070 tx=0x7f3720004c80 comp rx=0 tx=0).stop 2024-05-10T11:31:36.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.187+0000 7f37301e4700 1 -- 172.21.15.100:0/2605977551 shutdown_connections 2024-05-10T11:31:36.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.187+0000 7f37301e4700 1 --2- 172.21.15.100:0/2605977551 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f37140b2900 0x7f37140b4dc0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:36.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.187+0000 7f37301e4700 1 --2- 172.21.15.100:0/2605977551 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3728109530 0x7f3728199860 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:36.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.187+0000 7f37301e4700 1 --2- 172.21.15.100:0/2605977551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3728103520 0x7f372819f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:36.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.187+0000 7f37301e4700 1 --2- 172.21.15.100:0/2605977551 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3728103ee0 0x7f372819fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:36.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.187+0000 7f37301e4700 1 -- 172.21.15.100:0/2605977551 >> 172.21.15.100:0/2605977551 conn(0x7f372807a650 msgr2=0x7f37281023a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:31:36.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.187+0000 7f37301e4700 1 -- 172.21.15.100:0/2605977551 shutdown_connections 2024-05-10T11:31:36.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:36.187+0000 7f37301e4700 1 -- 172.21.15.100:0/2605977551 wait complete. 2024-05-10T11:31:36.985 INFO:tasks.workunit.client.0.smithi100.stdout: 1 350095 2.69 MB/sec execute 571 sec latency 17.293 ms 2024-05-10T11:31:37.008 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T11:31:37.008 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T11:31:37.332 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:31:37.985 INFO:tasks.workunit.client.0.smithi100.stdout: 1 350890 2.70 MB/sec execute 572 sec latency 12.491 ms 2024-05-10T11:31:38.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:31:37 smithi178 ceph-mon[33359]: pgmap v818: 129 pgs: 129 active+clean; 135 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 1.2 MiB/s wr, 132 op/s 2024-05-10T11:31:38.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:31:37 smithi188 ceph-mon[34075]: pgmap v818: 129 pgs: 129 active+clean; 135 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 1.2 MiB/s wr, 132 op/s 2024-05-10T11:31:38.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:31:37 smithi100 ceph-mon[30296]: pgmap v818: 129 pgs: 129 active+clean; 135 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 1.2 MiB/s wr, 132 op/s 2024-05-10T11:31:38.985 INFO:tasks.workunit.client.0.smithi100.stdout: 1 351295 2.70 MB/sec execute 573 sec latency 18.116 ms 2024-05-10T11:31:39.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:39.034+0000 7f0aca899700 1 -- 172.21.15.100:0/3291467737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ac4107760 msgr2=0x7f0ac410f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:39.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:39.034+0000 7f0aca899700 1 --2- 172.21.15.100:0/3291467737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ac4107760 0x7f0ac410f2e0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f0abc00b0d0 tx=0x7f0abc00b3e0 comp rx=0 tx=0).stop 2024-05-10T11:31:39.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:39.035+0000 7f0aca899700 1 -- 172.21.15.100:0/3291467737 shutdown_connections 2024-05-10T11:31:39.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:39.035+0000 7f0aca899700 1 --2- 172.21.15.100:0/3291467737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ac4107760 0x7f0ac410f2e0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:39.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:39.035+0000 7f0aca899700 1 --2- 172.21.15.100:0/3291467737 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ac4102e00 0x7f0ac41031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:39.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:39.035+0000 7f0aca899700 1 --2- 172.21.15.100:0/3291467737 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ac41037b0 0x7f0ac4107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:39.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:39.035+0000 7f0aca899700 1 -- 172.21.15.100:0/3291467737 >> 172.21.15.100:0/3291467737 conn(0x7f0ac4078a10 msgr2=0x7f0ac40ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:31:39.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:39.035+0000 7f0aca899700 1 -- 172.21.15.100:0/3291467737 shutdown_connections 2024-05-10T11:31:39.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:39.035+0000 7f0aca899700 1 -- 172.21.15.100:0/3291467737 wait complete. 2024-05-10T11:31:39.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:39.035+0000 7f0aca899700 1 Processor -- start 2024-05-10T11:31:39.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:39.035+0000 7f0aca899700 1 -- start start 2024-05-10T11:31:39.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:39.036+0000 7f0aca899700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ac4102e00 0x7f0ac419a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:39.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:39.036+0000 7f0aca899700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ac41037b0 0x7f0ac419ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:39.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:39.036+0000 7f0aca899700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ac4107760 0x7f0ac41a5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:39.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:39.036+0000 7f0aca899700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac4112400 con 0x7f0ac4102e00 2024-05-10T11:31:39.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:39.036+0000 7f0aca899700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac4112280 con 0x7f0ac4107760 2024-05-10T11:31:39.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:39.036+0000 7f0aca899700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac4112580 con 0x7f0ac41037b0 2024-05-10T11:31:39.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:39.036+0000 7f0ac3fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ac4102e00 0x7f0ac419a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:39.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:39.036+0000 7f0ac37fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ac41037b0 0x7f0ac419ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:39.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:39.036+0000 7f0ac3fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ac4102e00 0x7f0ac419a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50102/0 (socket says 172.21.15.100:50102) 2024-05-10T11:31:39.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:39.036+0000 7f0ac3fff700 1 -- 172.21.15.100:0/3385211749 learned_addr learned my addr 172.21.15.100:0/3385211749 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:31:39.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:39.036+0000 7f0ac8e36700 1 --2- 172.21.15.100:0/3385211749 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ac4107760 0x7f0ac41a5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:39.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:39.036+0000 7f0ac3fff700 1 -- 172.21.15.100:0/3385211749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ac41037b0 msgr2=0x7f0ac419ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:39.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:39.036+0000 7f0ac3fff700 1 --2- 172.21.15.100:0/3385211749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ac41037b0 0x7f0ac419ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:39.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:39.036+0000 7f0ac3fff700 1 -- 172.21.15.100:0/3385211749 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ac4107760 msgr2=0x7f0ac41a5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:39.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:39.036+0000 7f0ac3fff700 1 --2- 172.21.15.100:0/3385211749 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ac4107760 0x7f0ac41a5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:39.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:39.036+0000 7f0ac3fff700 1 -- 172.21.15.100:0/3385211749 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ac41a5d60 con 0x7f0ac4102e00 2024-05-10T11:31:39.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:39.036+0000 7f0ac37fe700 1 --2- 172.21.15.100:0/3385211749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ac41037b0 0x7f0ac419ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:31:39.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:39.036+0000 7f0ac8e36700 1 --2- 172.21.15.100:0/3385211749 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ac4107760 0x7f0ac41a5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:31:39.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:39.036+0000 7f0ac3fff700 1 --2- 172.21.15.100:0/3385211749 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ac4102e00 0x7f0ac419a7a0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f0ab400e9a0 tx=0x7f0ab400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:31:39.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:39.036+0000 7f0ac17fa700 1 -- 172.21.15.100:0/3385211749 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab400ca10 con 0x7f0ac4102e00 2024-05-10T11:31:39.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:39.036+0000 7f0ac17fa700 1 -- 172.21.15.100:0/3385211749 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0ab400cbb0 con 0x7f0ac4102e00 2024-05-10T11:31:39.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:39.037+0000 7f0aca899700 1 -- 172.21.15.100:0/3385211749 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ac41a5ff0 con 0x7f0ac4102e00 2024-05-10T11:31:39.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:39.037+0000 7f0ac17fa700 1 -- 172.21.15.100:0/3385211749 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab4018680 con 0x7f0ac4102e00 2024-05-10T11:31:39.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:39.037+0000 7f0aca899700 1 -- 172.21.15.100:0/3385211749 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ac41a64a0 con 0x7f0ac4102e00 2024-05-10T11:31:39.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:39.037+0000 7f0aca899700 1 -- 172.21.15.100:0/3385211749 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ac41020e0 con 0x7f0ac4102e00 2024-05-10T11:31:39.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:39.039+0000 7f0ac17fa700 1 -- 172.21.15.100:0/3385211749 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f0ab4010430 con 0x7f0ac4102e00 2024-05-10T11:31:39.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:39.040+0000 7f0ac17fa700 1 --2- 172.21.15.100:0/3385211749 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0aac0b2880 0x7f0aac0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:39.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:39.040+0000 7f0ac37fe700 1 --2- 172.21.15.100:0/3385211749 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0aac0b2880 0x7f0aac0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:39.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:39.040+0000 7f0ac17fa700 1 -- 172.21.15.100:0/3385211749 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f0ab40e2ae0 con 0x7f0ac4102e00 2024-05-10T11:31:39.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:39.040+0000 7f0ac37fe700 1 --2- 172.21.15.100:0/3385211749 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0aac0b2880 0x7f0aac0b4d40 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f0ac4104f40 tx=0x7f0ab8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:31:39.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:39.043+0000 7f0ac17fa700 1 -- 172.21.15.100:0/3385211749 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f0ab40aa4a0 con 0x7f0ac4102e00 2024-05-10T11:31:39.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:39.325+0000 7f0aca899700 1 -- 172.21.15.100:0/3385211749 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0ac402e510 con 0x7f0ac4102e00 2024-05-10T11:31:39.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:39.325+0000 7f0ac17fa700 1 -- 172.21.15.100:0/3385211749 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11897 (secure 0 0 0) 0x7f0ab40a9da0 con 0x7f0ac4102e00 2024-05-10T11:31:39.326 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:31:39.326 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2072225012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2072225012},{"type":"v1","addr":"172.21.15.100:6835","nonce":2072225012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/401939540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":401939540},{"type":"v1","addr":"172.21.15.100:6837","nonce":401939540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14571,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/3946311025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3946311025},{"type":"v1","addr":"172.21.15.100:6839","nonce":3946311025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3736371660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3736371660},{"type":"v1","addr":"172.21.15.100:6841","nonce":3736371660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/2741810791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2741810791},{"type":"v1","addr":"172.21.15.178:6833","nonce":2741810791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1704891041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1704891041},{"type":"v1","addr":"172.21.15.188:6833","nonce":1704891041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T11:14:27.242452+0000","modified":"2024-05-10T11:29:07.529595+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24439,"mds_1":24437,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/2124076312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2124076312},{"type":"v1","addr":"172.21.15.178:6835","nonce":2124076312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/3224358425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3224358425},{"type":"v1","addr":"172.21.15.178:6837","nonce":3224358425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/1943342620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1943342620},{"type":"v1","addr":"172.21.15.188:6837","nonce":1943342620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.178:6839/3760985404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3760985404},{"type":"v1","addr":"172.21.15.178:6839","nonce":3760985404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.188:6839/3194917070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":3194917070},{"type":"v1","addr":"172.21.15.188:6839","nonce":3194917070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27034":{"gid":27034,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/3139559450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3139559450},{"type":"v1","addr":"172.21.15.188:6835","nonce":3139559450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24439,"qdb_cluster":[24427,24437,24439]},"id":1}]} 2024-05-10T11:31:39.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:39.328+0000 7f0aca899700 1 -- 172.21.15.100:0/3385211749 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0aac0b2880 msgr2=0x7f0aac0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:39.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:39.328+0000 7f0aca899700 1 --2- 172.21.15.100:0/3385211749 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0aac0b2880 0x7f0aac0b4d40 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f0ac4104f40 tx=0x7f0ab8009260 comp rx=0 tx=0).stop 2024-05-10T11:31:39.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:39.328+0000 7f0aca899700 1 -- 172.21.15.100:0/3385211749 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ac4102e00 msgr2=0x7f0ac419a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:39.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:39.328+0000 7f0aca899700 1 --2- 172.21.15.100:0/3385211749 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ac4102e00 0x7f0ac419a7a0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f0ab400e9a0 tx=0x7f0ab400ecb0 comp rx=0 tx=0).stop 2024-05-10T11:31:39.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:39.328+0000 7f0aca899700 1 -- 172.21.15.100:0/3385211749 shutdown_connections 2024-05-10T11:31:39.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:39.328+0000 7f0aca899700 1 --2- 172.21.15.100:0/3385211749 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0aac0b2880 0x7f0aac0b4d40 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:39.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:39.328+0000 7f0aca899700 1 --2- 172.21.15.100:0/3385211749 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ac4107760 0x7f0ac41a5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:39.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:39.328+0000 7f0aca899700 1 --2- 172.21.15.100:0/3385211749 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ac4102e00 0x7f0ac419a7a0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:39.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:39.328+0000 7f0aca899700 1 --2- 172.21.15.100:0/3385211749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ac41037b0 0x7f0ac419ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:39.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:39.328+0000 7f0aca899700 1 -- 172.21.15.100:0/3385211749 >> 172.21.15.100:0/3385211749 conn(0x7f0ac4078a10 msgr2=0x7f0ac40ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:31:39.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:39.328+0000 7f0aca899700 1 -- 172.21.15.100:0/3385211749 shutdown_connections 2024-05-10T11:31:39.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:39.328+0000 7f0aca899700 1 -- 172.21.15.100:0/3385211749 wait complete. 2024-05-10T11:31:39.329 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 33 2024-05-10T11:31:39.963 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:31:39 smithi100 ceph-mon[30296]: pgmap v819: 129 pgs: 129 active+clean; 135 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 1.2 MiB/s wr, 155 op/s 2024-05-10T11:31:39.963 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:31:39 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/3385211749' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:31:39.985 INFO:tasks.workunit.client.0.smithi100.stdout: 1 351589 2.70 MB/sec execute 574 sec latency 12.491 ms 2024-05-10T11:31:39.991 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-10T11:31:40.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:31:39 smithi178 ceph-mon[33359]: pgmap v819: 129 pgs: 129 active+clean; 135 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 1.2 MiB/s wr, 155 op/s 2024-05-10T11:31:40.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:31:39 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/3385211749' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:31:40.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:31:39 smithi188 ceph-mon[34075]: pgmap v819: 129 pgs: 129 active+clean; 135 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 1.2 MiB/s wr, 155 op/s 2024-05-10T11:31:40.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:31:39 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/3385211749' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:31:40.252 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:31:40.986 INFO:tasks.workunit.client.0.smithi100.stdout: 1 351943 2.70 MB/sec execute 575 sec latency 47.347 ms 2024-05-10T11:31:41.767 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:31:41 smithi100 ceph-mon[30296]: pgmap v820: 129 pgs: 129 active+clean; 136 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 1.1 MiB/s wr, 127 op/s 2024-05-10T11:31:41.985 INFO:tasks.workunit.client.0.smithi100.stdout: 1 352443 2.69 MB/sec execute 576 sec latency 25.407 ms 2024-05-10T11:31:42.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:31:41 smithi178 ceph-mon[33359]: pgmap v820: 129 pgs: 129 active+clean; 136 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 1.1 MiB/s wr, 127 op/s 2024-05-10T11:31:42.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.025+0000 7ff25e91e700 1 -- 172.21.15.100:0/174814655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff258109500 msgr2=0x7ff2581098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:42.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.025+0000 7ff25e91e700 1 --2- 172.21.15.100:0/174814655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff258109500 0x7ff2581098e0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7ff248009990 tx=0x7ff248009ca0 comp rx=0 tx=0).stop 2024-05-10T11:31:42.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.025+0000 7ff25e91e700 1 -- 172.21.15.100:0/174814655 shutdown_connections 2024-05-10T11:31:42.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.025+0000 7ff25e91e700 1 --2- 172.21.15.100:0/174814655 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff25806c3d0 0x7ff258110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:42.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.025+0000 7ff25e91e700 1 --2- 172.21.15.100:0/174814655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff258109500 0x7ff2581098e0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:42.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.025+0000 7ff25e91e700 1 --2- 172.21.15.100:0/174814655 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff25806ba10 0x7ff25806be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:42.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.025+0000 7ff25e91e700 1 -- 172.21.15.100:0/174814655 >> 172.21.15.100:0/174814655 conn(0x7ff258079310 msgr2=0x7ff258079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:31:42.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.025+0000 7ff25e91e700 1 -- 172.21.15.100:0/174814655 shutdown_connections 2024-05-10T11:31:42.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.025+0000 7ff25e91e700 1 -- 172.21.15.100:0/174814655 wait complete. 2024-05-10T11:31:42.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.026+0000 7ff25e91e700 1 Processor -- start 2024-05-10T11:31:42.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.026+0000 7ff25e91e700 1 -- start start 2024-05-10T11:31:42.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.026+0000 7ff25e91e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff25806ba10 0x7ff258102460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:42.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.026+0000 7ff25e91e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff25806c3d0 0x7ff2581029a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:42.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.026+0000 7ff25e91e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff258109500 0x7ff258102ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:42.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.026+0000 7ff25e91e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff258113530 con 0x7ff258109500 2024-05-10T11:31:42.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.026+0000 7ff25e91e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff2581133b0 con 0x7ff25806c3d0 2024-05-10T11:31:42.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.026+0000 7ff25e91e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff2581136b0 con 0x7ff25806ba10 2024-05-10T11:31:42.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.026+0000 7ff25cebb700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff258109500 0x7ff258102ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:42.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.026+0000 7ff25cebb700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff258109500 0x7ff258102ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50110/0 (socket says 172.21.15.100:50110) 2024-05-10T11:31:42.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.026+0000 7ff25cebb700 1 -- 172.21.15.100:0/2013613752 learned_addr learned my addr 172.21.15.100:0/2013613752 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:31:42.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.026+0000 7ff257fff700 1 --2- 172.21.15.100:0/2013613752 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff25806ba10 0x7ff258102460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:42.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.026+0000 7ff2577fe700 1 --2- 172.21.15.100:0/2013613752 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff25806c3d0 0x7ff2581029a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:42.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.027+0000 7ff257fff700 1 -- 172.21.15.100:0/2013613752 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff25806c3d0 msgr2=0x7ff2581029a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:42.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.027+0000 7ff257fff700 1 --2- 172.21.15.100:0/2013613752 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff25806c3d0 0x7ff2581029a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:42.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.027+0000 7ff257fff700 1 -- 172.21.15.100:0/2013613752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff258109500 msgr2=0x7ff258102ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:42.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.027+0000 7ff257fff700 1 --2- 172.21.15.100:0/2013613752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff258109500 0x7ff258102ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:42.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.027+0000 7ff257fff700 1 -- 172.21.15.100:0/2013613752 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2581a7ff0 con 0x7ff25806ba10 2024-05-10T11:31:42.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.027+0000 7ff257fff700 1 --2- 172.21.15.100:0/2013613752 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff25806ba10 0x7ff258102460 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff248000c00 tx=0x7ff2480057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:31:42.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.027+0000 7ff2557fa700 1 -- 172.21.15.100:0/2013613752 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff248026070 con 0x7ff25806ba10 2024-05-10T11:31:42.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.027+0000 7ff25e91e700 1 -- 172.21.15.100:0/2013613752 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2581a8280 con 0x7ff25806ba10 2024-05-10T11:31:42.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.027+0000 7ff2557fa700 1 -- 172.21.15.100:0/2013613752 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff248021410 con 0x7ff25806ba10 2024-05-10T11:31:42.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.027+0000 7ff25e91e700 1 -- 172.21.15.100:0/2013613752 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2581a8730 con 0x7ff25806ba10 2024-05-10T11:31:42.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.027+0000 7ff2557fa700 1 -- 172.21.15.100:0/2013613752 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff248017650 con 0x7ff25806ba10 2024-05-10T11:31:42.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:31:41 smithi188 ceph-mon[34075]: pgmap v820: 129 pgs: 129 active+clean; 136 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 1.1 MiB/s wr, 127 op/s 2024-05-10T11:31:42.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.030+0000 7ff2557fa700 1 -- 172.21.15.100:0/2013613752 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7ff248029070 con 0x7ff25806ba10 2024-05-10T11:31:42.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.031+0000 7ff2557fa700 1 --2- 172.21.15.100:0/2013613752 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff2400b2900 0x7ff2400b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:42.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.031+0000 7ff2577fe700 1 --2- 172.21.15.100:0/2013613752 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff2400b2900 0x7ff2400b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:42.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.031+0000 7ff2557fa700 1 -- 172.21.15.100:0/2013613752 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7ff2480e32c0 con 0x7ff25806ba10 2024-05-10T11:31:42.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.031+0000 7ff2577fe700 1 --2- 172.21.15.100:0/2013613752 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff2400b2900 0x7ff2400b4dc0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff258079a50 tx=0x7ff24c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:31:42.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.032+0000 7ff2691b1b80 1 Processor -- start 2024-05-10T11:31:42.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.032+0000 7ff2691b1b80 1 -- start start 2024-05-10T11:31:42.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.032+0000 7ff2691b1b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6bcd0c1b0 0x55c6bcd0c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:42.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.032+0000 7ff2691b1b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6bcd0cad0 0x55c6bcd06810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:42.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.032+0000 7ff2691b1b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6bcd06d50 0x55c6bcd07190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:42.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.032+0000 7ff2691b1b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55c6bcc58940 con 0x55c6bcd06d50 2024-05-10T11:31:42.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.032+0000 7ff2691b1b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c6bcbbdda0 con 0x55c6bcd0cad0 2024-05-10T11:31:42.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.032+0000 7ff2691b1b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c6bcc54900 con 0x55c6bcd0c1b0 2024-05-10T11:31:42.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.032+0000 7ff25cebb700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6bcd06d50 0x55c6bcd07190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:42.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.032+0000 7ff257fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6bcd0c1b0 0x55c6bcd0c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:42.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.032+0000 7ff257fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6bcd0c1b0 0x55c6bcd0c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:50444/0 (socket says 172.21.15.100:50444) 2024-05-10T11:31:42.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.032+0000 7ff25cebb700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6bcd06d50 0x55c6bcd07190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50122/0 (socket says 172.21.15.100:50122) 2024-05-10T11:31:42.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.032+0000 7ff257fff700 1 -- 172.21.15.100:0/389028487 learned_addr learned my addr 172.21.15.100:0/389028487 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:31:42.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.032+0000 7ff2577fe700 1 --2- 172.21.15.100:0/389028487 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6bcd0cad0 0x55c6bcd06810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:42.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.032+0000 7ff257fff700 1 -- 172.21.15.100:0/389028487 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6bcd0cad0 msgr2=0x55c6bcd06810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:42.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.032+0000 7ff257fff700 1 --2- 172.21.15.100:0/389028487 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6bcd0cad0 0x55c6bcd06810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:42.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.032+0000 7ff257fff700 1 -- 172.21.15.100:0/389028487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6bcd06d50 msgr2=0x55c6bcd07190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:42.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.032+0000 7ff257fff700 1 --2- 172.21.15.100:0/389028487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6bcd06d50 0x55c6bcd07190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:42.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.032+0000 7ff257fff700 1 -- 172.21.15.100:0/389028487 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6bcba58d0 con 0x55c6bcd0c1b0 2024-05-10T11:31:42.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.033+0000 7ff257fff700 1 --2- 172.21.15.100:0/389028487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6bcd0c1b0 0x55c6bcd0c590 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff2480a6630 tx=0x7ff2480aa4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a470ebc8514e35ad server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:31:42.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.033+0000 7ff23e7fc700 1 -- 172.21.15.100:0/389028487 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2480ae960 con 0x55c6bcd0c1b0 2024-05-10T11:31:42.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.033+0000 7ff23e7fc700 1 -- 172.21.15.100:0/389028487 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff2480b3380 con 0x55c6bcd0c1b0 2024-05-10T11:31:42.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.033+0000 7ff23e7fc700 1 -- 172.21.15.100:0/389028487 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2480aa810 con 0x55c6bcd0c1b0 2024-05-10T11:31:42.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.033+0000 7ff2691b1b80 1 -- 172.21.15.100:0/389028487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6bcd0c1b0 msgr2=0x55c6bcd0c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:42.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.033+0000 7ff2691b1b80 1 --2- 172.21.15.100:0/389028487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6bcd0c1b0 0x55c6bcd0c590 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff2480a6630 tx=0x7ff2480aa4b0 comp rx=0 tx=0).stop 2024-05-10T11:31:42.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.034+0000 7ff2691b1b80 1 -- 172.21.15.100:0/389028487 shutdown_connections 2024-05-10T11:31:42.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.034+0000 7ff2691b1b80 1 --2- 172.21.15.100:0/389028487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6bcd06d50 0x55c6bcd07190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:42.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.034+0000 7ff2691b1b80 1 --2- 172.21.15.100:0/389028487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6bcd0c1b0 0x55c6bcd0c590 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:42.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.034+0000 7ff2691b1b80 1 --2- 172.21.15.100:0/389028487 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6bcd0cad0 0x55c6bcd06810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:42.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.034+0000 7ff2691b1b80 1 -- 172.21.15.100:0/389028487 >> 172.21.15.100:0/389028487 conn(0x55c6bcc32b70 msgr2=0x55c6bcd012a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:31:42.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.034+0000 7ff2691b1b80 1 -- 172.21.15.100:0/389028487 shutdown_connections 2024-05-10T11:31:42.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.034+0000 7ff2691b1b80 1 -- 172.21.15.100:0/389028487 wait complete. 2024-05-10T11:31:42.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.034+0000 7ff2691b1b80 1 -- start start 2024-05-10T11:31:42.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.034+0000 7ff2691b1b80 1 Processor -- start 2024-05-10T11:31:42.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.034+0000 7ff2691b1b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6bcd06d50 0x55c6bcd127b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:42.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.034+0000 7ff2691b1b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6bcd0c1b0 0x55c6bcd0fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:42.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.034+0000 7ff2691b1b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6bcd0cad0 0x55c6bcd100e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:42.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.035+0000 7ff2691b1b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55c6bcd13210 con 0x55c6bcd0cad0 2024-05-10T11:31:42.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.035+0000 7ff2691b1b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c6bcc11ed0 con 0x55c6bcd06d50 2024-05-10T11:31:42.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.035+0000 7ff2691b1b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c6bcd13390 con 0x55c6bcd0c1b0 2024-05-10T11:31:42.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.035+0000 7ff2577fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6bcd0c1b0 0x55c6bcd0fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:42.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.035+0000 7ff2691b1b80 10 client.0 _ll_register_callbacks cb 0x55c6bcbec5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:31:42.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.035+0000 7ff2577fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6bcd0c1b0 0x55c6bcd0fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:50448/0 (socket says 172.21.15.100:50448) 2024-05-10T11:31:42.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.035+0000 7ff2577fe700 1 -- 172.21.15.100:0/1088937375 learned_addr learned my addr 172.21.15.100:0/1088937375 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:31:42.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.035+0000 7ff257fff700 1 --2- 172.21.15.100:0/1088937375 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6bcd06d50 0x55c6bcd127b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:42.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.035+0000 7ff25cebb700 1 --2- 172.21.15.100:0/1088937375 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6bcd0cad0 0x55c6bcd100e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:42.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.035+0000 7ff2577fe700 1 -- 172.21.15.100:0/1088937375 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6bcd06d50 msgr2=0x55c6bcd127b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:42.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.035+0000 7ff2577fe700 1 --2- 172.21.15.100:0/1088937375 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6bcd06d50 0x55c6bcd127b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:42.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.035+0000 7ff2577fe700 1 -- 172.21.15.100:0/1088937375 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6bcd0cad0 msgr2=0x55c6bcd100e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:42.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.035+0000 7ff2577fe700 1 --2- 172.21.15.100:0/1088937375 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6bcd0cad0 0x55c6bcd100e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:42.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.035+0000 7ff2577fe700 1 -- 172.21.15.100:0/1088937375 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6bcc0f760 con 0x55c6bcd0c1b0 2024-05-10T11:31:42.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.035+0000 7ff2577fe700 1 --2- 172.21.15.100:0/1088937375 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6bcd0c1b0 0x55c6bcd0fba0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff24c011fd0 tx=0x7ff24c0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fbcba409fde3088 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:31:42.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.035+0000 7ff2377fe700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:31:42.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.036+0000 7ff2377fe700 1 -- 172.21.15.100:0/1088937375 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff24c01c820 con 0x55c6bcd0c1b0 2024-05-10T11:31:42.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.036+0000 7ff2691b1b80 1 -- 172.21.15.100:0/1088937375 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2480b3380 con 0x55c6bcd0c1b0 2024-05-10T11:31:42.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.036+0000 7ff2377fe700 1 -- 172.21.15.100:0/1088937375 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7ff24c01c9c0 con 0x55c6bcd0c1b0 2024-05-10T11:31:42.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.036+0000 7ff2377fe700 1 -- 172.21.15.100:0/1088937375 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff24c032050 con 0x55c6bcd0c1b0 2024-05-10T11:31:42.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.036+0000 7ff2691b1b80 10 client.27523 fetch_fsmap learned FSMap version 33 2024-05-10T11:31:42.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.036+0000 7ff2691b1b80 1 -- 172.21.15.100:0/1088937375 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c6bcd1db70 con 0x55c6bcd0c1b0 2024-05-10T11:31:42.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.036+0000 7ff2377fe700 1 -- 172.21.15.100:0/1088937375 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff2480b3380 con 0x55c6bcd0c1b0 2024-05-10T11:31:42.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.036+0000 7ff2377fe700 1 -- 172.21.15.100:0/1088937375 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7ff24c00eb90 con 0x55c6bcd0c1b0 2024-05-10T11:31:42.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.036+0000 7ff2691b1b80 10 client.27523 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T11:31:42.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.036+0000 7ff2691b1b80 10 client.27523 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-10T11:31:42.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.036+0000 7ff2691b1b80 20 client.27523 populate_metadata read hostname 'smithi100' 2024-05-10T11:31:42.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.036+0000 7ff2691b1b80 1 --2- 172.21.15.100:0/1088937375 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55c6bcd201b0 0x55c6bcd225a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:42.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.036+0000 7ff2691b1b80 4 client.27523 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:31:42.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.036+0000 7ff2691b1b80 1 -- 172.21.15.100:0/1088937375 --> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6bcc66320 con 0x55c6bcd201b0 2024-05-10T11:31:42.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.037+0000 7ff257fff700 1 --2- 172.21.15.100:0/1088937375 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55c6bcd201b0 0x55c6bcd225a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:42.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.037+0000 7ff2377fe700 10 client.27523 ms_handle_connect on v2:172.21.15.188:6836/1943342620 2024-05-10T11:31:42.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.037+0000 7ff257fff700 1 --2- 172.21.15.100:0/1088937375 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55c6bcd201b0 0x55c6bcd225a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9bcd714c2a5f50f9 server_cookie=ffbd6af95a827115 in_seq=0 out_seq=0 2024-05-10T11:31:42.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.038+0000 7ff2377fe700 1 -- 172.21.15.100:0/1088937375 <== mds.2 v2:172.21.15.188:6836/1943342620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c6bcc66320 con 0x55c6bcd201b0 2024-05-10T11:31:42.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.038+0000 7ff2377fe700 10 client.27523 handle_command_reply: tid=0 2024-05-10T11:31:42.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.038+0000 7ff2691b1b80 1 client.27523 shutdown 2024-05-10T11:31:42.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.038+0000 7ff2691b1b80 1 -- 172.21.15.100:0/1088937375 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6bcd0c1b0 msgr2=0x55c6bcd0fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:42.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.038+0000 7ff2691b1b80 1 --2- 172.21.15.100:0/1088937375 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6bcd0c1b0 0x55c6bcd0fba0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff24c011fd0 tx=0x7ff24c0047c0 comp rx=0 tx=0).stop 2024-05-10T11:31:42.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.038+0000 7ff2691b1b80 1 -- 172.21.15.100:0/1088937375 shutdown_connections 2024-05-10T11:31:42.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.038+0000 7ff2691b1b80 1 --2- 172.21.15.100:0/1088937375 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55c6bcd201b0 0x55c6bcd225a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:42.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.038+0000 7ff2377fe700 0 client.27523 ms_handle_reset on v2:172.21.15.188:6836/1943342620 2024-05-10T11:31:42.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.038+0000 7ff2691b1b80 1 --2- 172.21.15.100:0/1088937375 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6bcd0cad0 0x55c6bcd100e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:42.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.038+0000 7ff2691b1b80 1 --2- 172.21.15.100:0/1088937375 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6bcd06d50 0x55c6bcd127b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:42.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.038+0000 7ff2691b1b80 1 --2- 172.21.15.100:0/1088937375 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6bcd0c1b0 0x55c6bcd0fba0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:42.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.038+0000 7ff2691b1b80 1 -- 172.21.15.100:0/1088937375 >> 172.21.15.100:0/1088937375 conn(0x55c6bcc32b70 msgr2=0x55c6bccfbb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:31:42.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.038+0000 7ff2691b1b80 1 -- 172.21.15.100:0/1088937375 shutdown_connections 2024-05-10T11:31:42.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.038+0000 7ff2691b1b80 1 -- 172.21.15.100:0/1088937375 wait complete. 2024-05-10T11:31:42.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.038+0000 7ff2691b1b80 20 client.27523 trim_cache size 0 max 16384 2024-05-10T11:31:42.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.053+0000 7ff2691b1b80 1 Processor -- start 2024-05-10T11:31:42.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.053+0000 7ff2691b1b80 1 -- start start 2024-05-10T11:31:42.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.054+0000 7ff2691b1b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6bcd06d50 0x55c6bcd06810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:42.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.054+0000 7ff2691b1b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6bcd0c1b0 0x55c6bcd1c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:42.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.054+0000 7ff2691b1b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6bcd0cad0 0x55c6bccfbc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:42.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.054+0000 7ff2691b1b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff2581b3aa0 con 0x55c6bcd06d50 2024-05-10T11:31:42.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.054+0000 7ff2691b1b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff2581b1330 con 0x55c6bcd0cad0 2024-05-10T11:31:42.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.054+0000 7ff2691b1b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff2581af180 con 0x55c6bcd0c1b0 2024-05-10T11:31:42.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.054+0000 7ff257fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6bcd06d50 0x55c6bcd06810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:42.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.054+0000 7ff257fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6bcd06d50 0x55c6bcd06810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50140/0 (socket says 172.21.15.100:50140) 2024-05-10T11:31:42.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.054+0000 7ff257fff700 1 -- 172.21.15.100:0/1617074275 learned_addr learned my addr 172.21.15.100:0/1617074275 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:31:42.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.054+0000 7ff25cebb700 1 --2- 172.21.15.100:0/1617074275 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6bcd0cad0 0x55c6bccfbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:42.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.054+0000 7ff2577fe700 1 --2- 172.21.15.100:0/1617074275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6bcd0c1b0 0x55c6bcd1c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:42.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.054+0000 7ff25cebb700 1 -- 172.21.15.100:0/1617074275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6bcd0c1b0 msgr2=0x55c6bcd1c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:42.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.054+0000 7ff25cebb700 1 --2- 172.21.15.100:0/1617074275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6bcd0c1b0 0x55c6bcd1c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:42.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.054+0000 7ff25cebb700 1 -- 172.21.15.100:0/1617074275 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6bcd06d50 msgr2=0x55c6bcd06810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:42.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.054+0000 7ff25cebb700 1 --2- 172.21.15.100:0/1617074275 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6bcd06d50 0x55c6bcd06810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:42.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.054+0000 7ff25cebb700 1 -- 172.21.15.100:0/1617074275 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6bcd0c7e0 con 0x55c6bcd0cad0 2024-05-10T11:31:42.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.054+0000 7ff257fff700 1 --2- 172.21.15.100:0/1617074275 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6bcd06d50 0x55c6bcd06810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:31:42.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.054+0000 7ff25cebb700 1 --2- 172.21.15.100:0/1617074275 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6bcd0cad0 0x55c6bccfbc40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff250009550 tx=0x7ff2500091b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b337010ef38de88 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:31:42.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.055+0000 7ff23dffb700 1 -- 172.21.15.100:0/1617074275 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff250007e40 con 0x55c6bcd0cad0 2024-05-10T11:31:42.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.055+0000 7ff23dffb700 1 -- 172.21.15.100:0/1617074275 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff25001b070 con 0x55c6bcd0cad0 2024-05-10T11:31:42.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.055+0000 7ff23dffb700 1 -- 172.21.15.100:0/1617074275 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2500177c0 con 0x55c6bcd0cad0 2024-05-10T11:31:42.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.055+0000 7ff2691b1b80 1 -- 172.21.15.100:0/1617074275 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6bcd0cad0 msgr2=0x55c6bccfbc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:42.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.055+0000 7ff2691b1b80 1 --2- 172.21.15.100:0/1617074275 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6bcd0cad0 0x55c6bccfbc40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff250009550 tx=0x7ff2500091b0 comp rx=0 tx=0).stop 2024-05-10T11:31:42.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.055+0000 7ff2691b1b80 1 -- 172.21.15.100:0/1617074275 shutdown_connections 2024-05-10T11:31:42.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.055+0000 7ff2691b1b80 1 --2- 172.21.15.100:0/1617074275 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6bcd0cad0 0x55c6bccfbc40 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:42.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.055+0000 7ff2691b1b80 1 --2- 172.21.15.100:0/1617074275 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6bcd06d50 0x55c6bcd06810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:42.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.055+0000 7ff2691b1b80 1 --2- 172.21.15.100:0/1617074275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6bcd0c1b0 0x55c6bcd1c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:42.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.055+0000 7ff2691b1b80 1 -- 172.21.15.100:0/1617074275 >> 172.21.15.100:0/1617074275 conn(0x55c6bcc32b70 msgr2=0x55c6bcd0e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:31:42.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.055+0000 7ff2691b1b80 1 -- 172.21.15.100:0/1617074275 shutdown_connections 2024-05-10T11:31:42.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.055+0000 7ff2691b1b80 1 -- 172.21.15.100:0/1617074275 wait complete. 2024-05-10T11:31:42.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.056+0000 7ff2691b1b80 1 -- start start 2024-05-10T11:31:42.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.056+0000 7ff2691b1b80 1 Processor -- start 2024-05-10T11:31:42.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.056+0000 7ff2691b1b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6bcd06d50 0x55c6bcd18a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:42.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.056+0000 7ff2691b1b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6bcd0c1b0 0x55c6bcd27390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:42.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.056+0000 7ff2691b1b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6bcd0cad0 0x55c6bcd278d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:42.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.056+0000 7ff2691b1b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff2581af7f0 con 0x55c6bcd0cad0 2024-05-10T11:31:42.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.056+0000 7ff2691b1b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff2581b0c60 con 0x55c6bcd06d50 2024-05-10T11:31:42.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.056+0000 7ff2691b1b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff2581afec0 con 0x55c6bcd0c1b0 2024-05-10T11:31:42.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.056+0000 7ff2691b1b80 10 client.0 _ll_register_callbacks cb 0x55c6bcbec5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:31:42.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.056+0000 7ff25cebb700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6bcd0cad0 0x55c6bcd278d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:42.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.057+0000 7ff257fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6bcd06d50 0x55c6bcd18a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:42.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.057+0000 7ff25cebb700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6bcd0cad0 0x55c6bcd278d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50154/0 (socket says 172.21.15.100:50154) 2024-05-10T11:31:42.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.057+0000 7ff25cebb700 1 -- 172.21.15.100:0/2808605388 learned_addr learned my addr 172.21.15.100:0/2808605388 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:31:42.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.057+0000 7ff2577fe700 1 --2- 172.21.15.100:0/2808605388 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6bcd0c1b0 0x55c6bcd27390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:42.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.057+0000 7ff257fff700 1 -- 172.21.15.100:0/2808605388 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6bcd0c1b0 msgr2=0x55c6bcd27390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:42.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.057+0000 7ff257fff700 1 --2- 172.21.15.100:0/2808605388 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6bcd0c1b0 0x55c6bcd27390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:42.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.057+0000 7ff257fff700 1 -- 172.21.15.100:0/2808605388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6bcd0cad0 msgr2=0x55c6bcd278d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:42.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.057+0000 7ff257fff700 1 --2- 172.21.15.100:0/2808605388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6bcd0cad0 0x55c6bcd278d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:42.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.057+0000 7ff257fff700 1 -- 172.21.15.100:0/2808605388 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6bcd27fe0 con 0x55c6bcd06d50 2024-05-10T11:31:42.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.057+0000 7ff257fff700 1 --2- 172.21.15.100:0/2808605388 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6bcd06d50 0x55c6bcd18a30 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff2480098e0 tx=0x7ff2480e3170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5e2ede75359553e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:31:42.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.057+0000 7ff2377fe700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:31:42.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.057+0000 7ff2377fe700 1 -- 172.21.15.100:0/2808605388 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2480c9060 con 0x55c6bcd06d50 2024-05-10T11:31:42.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.057+0000 7ff2377fe700 1 -- 172.21.15.100:0/2808605388 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7ff2480094b0 con 0x55c6bcd06d50 2024-05-10T11:31:42.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.057+0000 7ff2691b1b80 1 -- 172.21.15.100:0/2808605388 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff25001b070 con 0x55c6bcd06d50 2024-05-10T11:31:42.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.057+0000 7ff2377fe700 1 -- 172.21.15.100:0/2808605388 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff248089050 con 0x55c6bcd06d50 2024-05-10T11:31:42.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.057+0000 7ff2691b1b80 10 client.27434 fetch_fsmap learned FSMap version 33 2024-05-10T11:31:42.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.057+0000 7ff2691b1b80 1 -- 172.21.15.100:0/2808605388 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c6bcd8a950 con 0x55c6bcd06d50 2024-05-10T11:31:42.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.058+0000 7ff2377fe700 1 -- 172.21.15.100:0/2808605388 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff25001b070 con 0x55c6bcd06d50 2024-05-10T11:31:42.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.058+0000 7ff2377fe700 1 -- 172.21.15.100:0/2808605388 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7ff2480aa8a0 con 0x55c6bcd06d50 2024-05-10T11:31:42.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.058+0000 7ff2691b1b80 10 client.27434 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T11:31:42.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.058+0000 7ff2691b1b80 10 client.27434 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-10T11:31:42.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.058+0000 7ff2691b1b80 20 client.27434 populate_metadata read hostname 'smithi100' 2024-05-10T11:31:42.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.058+0000 7ff2691b1b80 1 --2- 172.21.15.100:0/2808605388 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55c6bcd201b0 0x55c6bcd8cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:42.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.058+0000 7ff2691b1b80 4 client.27434 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:31:42.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.058+0000 7ff2691b1b80 1 -- 172.21.15.100:0/2808605388 --> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c6bcc603c0 con 0x55c6bcd201b0 2024-05-10T11:31:42.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.059+0000 7ff2577fe700 1 --2- 172.21.15.100:0/2808605388 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55c6bcd201b0 0x55c6bcd8cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:42.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.059+0000 7ff2577fe700 1 --2- 172.21.15.100:0/2808605388 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55c6bcd201b0 0x55c6bcd8cee0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=85b587c78af6aadd server_cookie=e8f91087cb77dec6 in_seq=0 out_seq=0 2024-05-10T11:31:42.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.059+0000 7ff2377fe700 10 client.27434 ms_handle_connect on v2:172.21.15.188:6836/1943342620 2024-05-10T11:31:42.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.059+0000 7ff2377fe700 1 -- 172.21.15.100:0/2808605388 <== mds.2 v2:172.21.15.188:6836/1943342620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c6bcc603c0 con 0x55c6bcd201b0 2024-05-10T11:31:42.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.059+0000 7ff2377fe700 10 client.27434 handle_command_reply: tid=0 2024-05-10T11:31:42.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.059+0000 7ff2691b1b80 1 client.27434 shutdown 2024-05-10T11:31:42.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.059+0000 7ff2691b1b80 1 -- 172.21.15.100:0/2808605388 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6bcd06d50 msgr2=0x55c6bcd18a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:42.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.059+0000 7ff2691b1b80 1 --2- 172.21.15.100:0/2808605388 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6bcd06d50 0x55c6bcd18a30 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff2480098e0 tx=0x7ff2480e3170 comp rx=0 tx=0).stop 2024-05-10T11:31:42.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.059+0000 7ff2691b1b80 1 -- 172.21.15.100:0/2808605388 shutdown_connections 2024-05-10T11:31:42.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.059+0000 7ff2691b1b80 1 --2- 172.21.15.100:0/2808605388 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55c6bcd201b0 0x55c6bcd8cee0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:42.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.059+0000 7ff2691b1b80 1 --2- 172.21.15.100:0/2808605388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6bcd0cad0 0x55c6bcd278d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:42.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.059+0000 7ff2691b1b80 1 --2- 172.21.15.100:0/2808605388 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6bcd06d50 0x55c6bcd18a30 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:42.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.059+0000 7ff2691b1b80 1 --2- 172.21.15.100:0/2808605388 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6bcd0c1b0 0x55c6bcd27390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:42.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.059+0000 7ff2691b1b80 1 -- 172.21.15.100:0/2808605388 >> 172.21.15.100:0/2808605388 conn(0x55c6bcc32b70 msgr2=0x55c6bcd0ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:31:42.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.059+0000 7ff2377fe700 0 client.27434 ms_handle_reset on v2:172.21.15.188:6836/1943342620 2024-05-10T11:31:42.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.060+0000 7ff2691b1b80 1 -- 172.21.15.100:0/2808605388 shutdown_connections 2024-05-10T11:31:42.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.060+0000 7ff2691b1b80 1 -- 172.21.15.100:0/2808605388 wait complete. 2024-05-10T11:31:42.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.060+0000 7ff2691b1b80 20 client.27434 trim_cache size 0 max 16384 2024-05-10T11:31:42.064 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:31:42.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.060+0000 7ff25e91e700 1 -- 172.21.15.100:0/2013613752 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff2400b2900 msgr2=0x7ff2400b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:42.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.060+0000 7ff25e91e700 1 --2- 172.21.15.100:0/2013613752 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff2400b2900 0x7ff2400b4dc0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff258079a50 tx=0x7ff24c009260 comp rx=0 tx=0).stop 2024-05-10T11:31:42.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.060+0000 7ff25e91e700 1 -- 172.21.15.100:0/2013613752 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff25806ba10 msgr2=0x7ff258102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:42.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.060+0000 7ff25e91e700 1 --2- 172.21.15.100:0/2013613752 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff25806ba10 0x7ff258102460 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff248000c00 tx=0x7ff2480057c0 comp rx=0 tx=0).stop 2024-05-10T11:31:42.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.060+0000 7ff25e91e700 1 -- 172.21.15.100:0/2013613752 shutdown_connections 2024-05-10T11:31:42.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.060+0000 7ff25e91e700 1 --2- 172.21.15.100:0/2013613752 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff2400b2900 0x7ff2400b4dc0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:42.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.060+0000 7ff25e91e700 1 --2- 172.21.15.100:0/2013613752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff258109500 0x7ff258102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:42.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.060+0000 7ff25e91e700 1 --2- 172.21.15.100:0/2013613752 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff25806ba10 0x7ff258102460 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:42.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.060+0000 7ff25e91e700 1 --2- 172.21.15.100:0/2013613752 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff25806c3d0 0x7ff2581029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:42.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.060+0000 7ff25e91e700 1 -- 172.21.15.100:0/2013613752 >> 172.21.15.100:0/2013613752 conn(0x7ff258079310 msgr2=0x7ff258101230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:31:42.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.060+0000 7ff25e91e700 1 -- 172.21.15.100:0/2013613752 shutdown_connections 2024-05-10T11:31:42.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:42.061+0000 7ff25e91e700 1 -- 172.21.15.100:0/2013613752 wait complete. 2024-05-10T11:31:42.810 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-10T11:31:42.985 INFO:tasks.workunit.client.0.smithi100.stdout: 1 353125 2.70 MB/sec execute 577 sec latency 18.754 ms 2024-05-10T11:31:43.084 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:31:43.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:31:43 smithi100 ceph-mon[30296]: pgmap v821: 129 pgs: 129 active+clean; 137 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 1.3 MiB/s wr, 136 op/s 2024-05-10T11:31:43.985 INFO:tasks.workunit.client.0.smithi100.stdout: 1 353707 2.69 MB/sec execute 578 sec latency 18.612 ms 2024-05-10T11:31:44.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:31:43 smithi178 ceph-mon[33359]: pgmap v821: 129 pgs: 129 active+clean; 137 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 1.3 MiB/s wr, 136 op/s 2024-05-10T11:31:44.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:31:43 smithi188 ceph-mon[34075]: pgmap v821: 129 pgs: 129 active+clean; 137 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 1.3 MiB/s wr, 136 op/s 2024-05-10T11:31:44.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.687+0000 7ff6d21f0640 1 -- 172.21.15.100:0/540600190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6cc1012e0 msgr2=0x7ff6cc101700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:44.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.687+0000 7ff6d21f0640 1 --2- 172.21.15.100:0/540600190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6cc1012e0 0x7ff6cc101700 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc009900 tx=0x7ff6bc02a410 comp rx=0 tx=0).stop 2024-05-10T11:31:44.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.687+0000 7ff6d21f0640 1 -- 172.21.15.100:0/540600190 shutdown_connections 2024-05-10T11:31:44.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.687+0000 7ff6d21f0640 1 --2- 172.21.15.100:0/540600190 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6cc1025e0 0x7ff6cc102a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:44.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.687+0000 7ff6d21f0640 1 --2- 172.21.15.100:0/540600190 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff6cc101c40 0x7ff6cc1020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:44.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.687+0000 7ff6d21f0640 1 --2- 172.21.15.100:0/540600190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6cc1012e0 0x7ff6cc101700 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:44.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.687+0000 7ff6d21f0640 1 -- 172.21.15.100:0/540600190 >> 172.21.15.100:0/540600190 conn(0x7ff6cc0fce80 msgr2=0x7ff6cc0ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:31:44.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.687+0000 7ff6d21f0640 1 -- 172.21.15.100:0/540600190 shutdown_connections 2024-05-10T11:31:44.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.687+0000 7ff6d21f0640 1 -- 172.21.15.100:0/540600190 wait complete. 2024-05-10T11:31:44.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.688+0000 7ff6d21f0640 1 Processor -- start 2024-05-10T11:31:44.691 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.688+0000 7ff6d21f0640 1 -- start start 2024-05-10T11:31:44.691 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.688+0000 7ff6d21f0640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff6cc1012e0 0x7ff6cc19ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:44.691 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.688+0000 7ff6d21f0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6cc101c40 0x7ff6cc19b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:44.691 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.688+0000 7ff6d21f0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6cc1025e0 0x7ff6cc1a2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:44.691 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.688+0000 7ff6d21f0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff6cc1105b0 con 0x7ff6cc1025e0 2024-05-10T11:31:44.691 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.688+0000 7ff6d21f0640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff6cc110430 con 0x7ff6cc1012e0 2024-05-10T11:31:44.691 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.688+0000 7ff6d21f0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff6cc110730 con 0x7ff6cc101c40 2024-05-10T11:31:44.691 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.688+0000 7ff6cbfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6cc1025e0 0x7ff6cc1a2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:44.691 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.688+0000 7ff6cbfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6cc1025e0 0x7ff6cc1a2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50166/0 (socket says 172.21.15.100:50166) 2024-05-10T11:31:44.691 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.688+0000 7ff6cbfff640 1 -- 172.21.15.100:0/879537034 learned_addr learned my addr 172.21.15.100:0/879537034 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:31:44.691 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.688+0000 7ff6c3fff640 1 --2- 172.21.15.100:0/879537034 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6cc101c40 0x7ff6cc19b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:44.691 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.688+0000 7ff6cb7fe640 1 --2- 172.21.15.100:0/879537034 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff6cc1012e0 0x7ff6cc19ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:44.691 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.689+0000 7ff6cbfff640 1 -- 172.21.15.100:0/879537034 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6cc101c40 msgr2=0x7ff6cc19b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:44.692 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.689+0000 7ff6cbfff640 1 --2- 172.21.15.100:0/879537034 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6cc101c40 0x7ff6cc19b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:44.692 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.689+0000 7ff6cbfff640 1 -- 172.21.15.100:0/879537034 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff6cc1012e0 msgr2=0x7ff6cc19ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:44.692 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.689+0000 7ff6cbfff640 1 --2- 172.21.15.100:0/879537034 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff6cc1012e0 0x7ff6cc19ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:44.692 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.689+0000 7ff6cbfff640 1 -- 172.21.15.100:0/879537034 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6cc1a2910 con 0x7ff6cc1025e0 2024-05-10T11:31:44.692 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.689+0000 7ff6cbfff640 1 --2- 172.21.15.100:0/879537034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6cc1025e0 0x7ff6cc1a2270 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7ff6c400eb10 tx=0x7ff6c400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:31:44.692 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.689+0000 7ff6c97fa640 1 -- 172.21.15.100:0/879537034 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c4004b20 con 0x7ff6cc1025e0 2024-05-10T11:31:44.692 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.689+0000 7ff6d21f0640 1 -- 172.21.15.100:0/879537034 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6cc1a2c00 con 0x7ff6cc1025e0 2024-05-10T11:31:44.692 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.689+0000 7ff6c97fa640 1 -- 172.21.15.100:0/879537034 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff6c4004cc0 con 0x7ff6cc1025e0 2024-05-10T11:31:44.692 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.689+0000 7ff6c97fa640 1 -- 172.21.15.100:0/879537034 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c4010690 con 0x7ff6cc1025e0 2024-05-10T11:31:44.692 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.689+0000 7ff6d21f0640 1 -- 172.21.15.100:0/879537034 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6cc1a3150 con 0x7ff6cc1025e0 2024-05-10T11:31:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.691+0000 7ff6c97fa640 1 -- 172.21.15.100:0/879537034 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7ff6c4005020 con 0x7ff6cc1025e0 2024-05-10T11:31:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.692+0000 7ff6d4fe7740 1 Processor -- start 2024-05-10T11:31:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.692+0000 7ff6d4fe7740 1 -- start start 2024-05-10T11:31:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.692+0000 7ff6d4fe7740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584fc97edb0 0x5584fc97f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.692+0000 7ff6d4fe7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5584fc97f7a0 0x5584fc9842a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.692+0000 7ff6d4fe7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584fc984910 0x5584fc986da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.692+0000 7ff6d4fe7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5584fc8e35d0 con 0x5584fc984910 2024-05-10T11:31:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.692+0000 7ff6d4fe7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5584fc89c890 con 0x5584fc97f7a0 2024-05-10T11:31:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.692+0000 7ff6d4fe7740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5584fc8df590 con 0x5584fc97edb0 2024-05-10T11:31:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.692+0000 7ff6c3fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584fc97edb0 0x5584fc97f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.692+0000 7ff6c3fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584fc97edb0 0x5584fc97f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:50486/0 (socket says 172.21.15.100:50486) 2024-05-10T11:31:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.692+0000 7ff6c3fff640 1 -- 172.21.15.100:0/2070527278 learned_addr learned my addr 172.21.15.100:0/2070527278 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:31:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.692+0000 7ff6c3fff640 1 --2- 172.21.15.100:0/2070527278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584fc984910 0x5584fc986da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.692+0000 7ff6c97fa640 1 --2- 172.21.15.100:0/879537034 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff6b80b28a0 0x7ff6b80b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.692+0000 7ff6cb7fe640 1 --2- 172.21.15.100:0/2070527278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5584fc97f7a0 0x5584fc9842a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.692+0000 7ff6cbfff640 1 --2- 172.21.15.100:0/879537034 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff6b80b28a0 0x7ff6b80b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.692+0000 7ff6c3fff640 1 -- 172.21.15.100:0/2070527278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5584fc97f7a0 msgr2=0x5584fc9842a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.692+0000 7ff6c3fff640 1 --2- 172.21.15.100:0/2070527278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5584fc97f7a0 0x5584fc9842a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.692+0000 7ff6c3fff640 1 -- 172.21.15.100:0/2070527278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584fc984910 msgr2=0x5584fc986da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.692+0000 7ff6c3fff640 1 --2- 172.21.15.100:0/2070527278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584fc984910 0x5584fc986da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.692+0000 7ff6c3fff640 1 -- 172.21.15.100:0/2070527278 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584fc5f5210 con 0x5584fc97edb0 2024-05-10T11:31:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.693+0000 7ff6c3fff640 1 --2- 172.21.15.100:0/2070527278 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584fc97edb0 0x5584fc97f1d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff6b4012850 tx=0x7ff6b400b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf876cf149e877ac server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:31:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.693+0000 7ff6c97fa640 1 -- 172.21.15.100:0/879537034 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7ff6c40e2b10 con 0x7ff6cc1025e0 2024-05-10T11:31:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.693+0000 7ff6cbfff640 1 --2- 172.21.15.100:0/879537034 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff6b80b28a0 0x7ff6b80b4d60 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff6c40a9d60 tx=0x7ff6c40a4310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:31:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.693+0000 7ff6c1ffb640 1 -- 172.21.15.100:0/2070527278 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b4012dd0 con 0x5584fc97edb0 2024-05-10T11:31:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.693+0000 7ff6c1ffb640 1 -- 172.21.15.100:0/2070527278 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff6b4010770 con 0x5584fc97edb0 2024-05-10T11:31:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.693+0000 7ff6c1ffb640 1 -- 172.21.15.100:0/2070527278 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b40037f0 con 0x5584fc97edb0 2024-05-10T11:31:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.694+0000 7ff6d4fe7740 1 -- 172.21.15.100:0/2070527278 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584fc97edb0 msgr2=0x5584fc97f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.694+0000 7ff6d4fe7740 1 --2- 172.21.15.100:0/2070527278 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584fc97edb0 0x5584fc97f1d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff6b4012850 tx=0x7ff6b400b860 comp rx=0 tx=0).stop 2024-05-10T11:31:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.694+0000 7ff6d4fe7740 1 -- 172.21.15.100:0/2070527278 shutdown_connections 2024-05-10T11:31:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.694+0000 7ff6d4fe7740 1 --2- 172.21.15.100:0/2070527278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584fc984910 0x5584fc986da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.694+0000 7ff6d4fe7740 1 --2- 172.21.15.100:0/2070527278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5584fc97f7a0 0x5584fc9842a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.694+0000 7ff6d4fe7740 1 --2- 172.21.15.100:0/2070527278 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584fc97edb0 0x5584fc97f1d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.694+0000 7ff6d4fe7740 1 -- 172.21.15.100:0/2070527278 >> 172.21.15.100:0/2070527278 conn(0x5584fc97a420 msgr2=0x5584fc97a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:31:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.694+0000 7ff6d4fe7740 1 -- 172.21.15.100:0/2070527278 shutdown_connections 2024-05-10T11:31:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.694+0000 7ff6d4fe7740 1 -- 172.21.15.100:0/2070527278 wait complete. 2024-05-10T11:31:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.694+0000 7ff6d4fe7740 1 -- start start 2024-05-10T11:31:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.694+0000 7ff6d4fe7740 1 Processor -- start 2024-05-10T11:31:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.695+0000 7ff6d4fe7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5584fc97edb0 0x5584fc988fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.695+0000 7ff6d4fe7740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584fc97f7a0 0x5584fc989500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.695+0000 7ff6d4fe7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584fc984910 0x5584fc990540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.695+0000 7ff6d4fe7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5584fc9878c0 con 0x5584fc984910 2024-05-10T11:31:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.695+0000 7ff6d4fe7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5584fc8a0370 con 0x5584fc97edb0 2024-05-10T11:31:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.695+0000 7ff6c3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584fc984910 0x5584fc990540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.695+0000 7ff6d4fe7740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5584fc987a40 con 0x5584fc97f7a0 2024-05-10T11:31:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.695+0000 7ff6cb7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584fc97f7a0 0x5584fc989500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.695+0000 7ff6c3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584fc984910 0x5584fc990540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50192/0 (socket says 172.21.15.100:50192) 2024-05-10T11:31:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.695+0000 7ff6c3fff640 1 -- 172.21.15.100:0/3163593887 learned_addr learned my addr 172.21.15.100:0/3163593887 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:31:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.695+0000 7ff6cb7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584fc97f7a0 0x5584fc989500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:50502/0 (socket says 172.21.15.100:50502) 2024-05-10T11:31:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.695+0000 7ff6d4fe7740 10 client.0 _ll_register_callbacks cb 0x5584fc5f4a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:31:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.695+0000 7ff6c3fff640 1 --2- 172.21.15.100:0/3163593887 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5584fc97edb0 0x5584fc988fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.695+0000 7ff6cb7fe640 1 -- 172.21.15.100:0/3163593887 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5584fc97edb0 msgr2=0x5584fc988fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.695+0000 7ff6cb7fe640 1 --2- 172.21.15.100:0/3163593887 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5584fc97edb0 0x5584fc988fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.695+0000 7ff6cb7fe640 1 -- 172.21.15.100:0/3163593887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584fc984910 msgr2=0x5584fc990540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.695+0000 7ff6cb7fe640 1 --2- 172.21.15.100:0/3163593887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584fc984910 0x5584fc990540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.695+0000 7ff6cb7fe640 1 -- 172.21.15.100:0/3163593887 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584fc6c1620 con 0x5584fc97f7a0 2024-05-10T11:31:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.695+0000 7ff6cb7fe640 1 --2- 172.21.15.100:0/3163593887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584fc97f7a0 0x5584fc989500 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc005ab0 tx=0x7ff6bc0317a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea1c2ed61820ce22 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:31:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.695+0000 7ff6aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:31:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.695+0000 7ff6aaffd640 1 -- 172.21.15.100:0/3163593887 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc035070 con 0x5584fc97f7a0 2024-05-10T11:31:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.696+0000 7ff6aaffd640 1 -- 172.21.15.100:0/3163593887 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7ff6bc004240 con 0x5584fc97f7a0 2024-05-10T11:31:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.696+0000 7ff6d4fe7740 1 -- 172.21.15.100:0/3163593887 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6b4010770 con 0x5584fc97f7a0 2024-05-10T11:31:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.696+0000 7ff6aaffd640 1 -- 172.21.15.100:0/3163593887 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff6bc04d050 con 0x5584fc97f7a0 2024-05-10T11:31:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.696+0000 7ff6d4fe7740 10 client.27541 fetch_fsmap learned FSMap version 33 2024-05-10T11:31:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.696+0000 7ff6d4fe7740 1 -- 172.21.15.100:0/3163593887 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5584fc9935a0 con 0x5584fc97f7a0 2024-05-10T11:31:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.696+0000 7ff6aaffd640 1 -- 172.21.15.100:0/3163593887 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff6b4010770 con 0x5584fc97f7a0 2024-05-10T11:31:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.696+0000 7ff6aaffd640 1 -- 172.21.15.100:0/3163593887 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7ff6bc03b430 con 0x5584fc97f7a0 2024-05-10T11:31:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.696+0000 7ff6d4fe7740 10 client.27541 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T11:31:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.696+0000 7ff6d4fe7740 10 client.27541 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:31:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.696+0000 7ff6d4fe7740 20 client.27541 populate_metadata read hostname 'smithi100' 2024-05-10T11:31:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.696+0000 7ff6d4fe7740 1 --2- 172.21.15.100:0/3163593887 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5584fc994d30 0x5584fc997120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.696+0000 7ff6d4fe7740 4 client.27541 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:31:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.696+0000 7ff6d4fe7740 1 -- 172.21.15.100:0/3163593887 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5584fc8f11e0 con 0x5584fc994d30 2024-05-10T11:31:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.696+0000 7ff6c3fff640 1 --2- 172.21.15.100:0/3163593887 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5584fc994d30 0x5584fc997120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.697+0000 7ff6c3fff640 1 --2- 172.21.15.100:0/3163593887 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5584fc994d30 0x5584fc997120 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff959cbe7078a2de server_cookie=16bd3668e589ce54 in_seq=0 out_seq=0 2024-05-10T11:31:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.697+0000 7ff6aaffd640 10 client.27541 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:31:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.698+0000 7ff6aaffd640 1 -- 172.21.15.100:0/3163593887 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5584fc8f11e0 con 0x5584fc994d30 2024-05-10T11:31:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.698+0000 7ff6aaffd640 10 client.27541 handle_command_reply: tid=0 2024-05-10T11:31:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.698+0000 7ff6d4fe7740 1 client.27541 shutdown 2024-05-10T11:31:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.698+0000 7ff6d4fe7740 1 -- 172.21.15.100:0/3163593887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584fc97f7a0 msgr2=0x5584fc989500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.698+0000 7ff6d4fe7740 1 --2- 172.21.15.100:0/3163593887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584fc97f7a0 0x5584fc989500 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc005ab0 tx=0x7ff6bc0317a0 comp rx=0 tx=0).stop 2024-05-10T11:31:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.698+0000 7ff6d4fe7740 1 -- 172.21.15.100:0/3163593887 shutdown_connections 2024-05-10T11:31:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.698+0000 7ff6d4fe7740 1 --2- 172.21.15.100:0/3163593887 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5584fc994d30 0x5584fc997120 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.698+0000 7ff6d4fe7740 1 --2- 172.21.15.100:0/3163593887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584fc984910 0x5584fc990540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.698+0000 7ff6d4fe7740 1 --2- 172.21.15.100:0/3163593887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584fc97f7a0 0x5584fc989500 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.698+0000 7ff6d4fe7740 1 --2- 172.21.15.100:0/3163593887 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5584fc97edb0 0x5584fc988fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.698+0000 7ff6aaffd640 0 client.27541 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:31:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.698+0000 7ff6d4fe7740 1 -- 172.21.15.100:0/3163593887 >> 172.21.15.100:0/3163593887 conn(0x5584fc97a420 msgr2=0x5584fc979580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:31:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.698+0000 7ff6d4fe7740 1 -- 172.21.15.100:0/3163593887 shutdown_connections 2024-05-10T11:31:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.698+0000 7ff6d4fe7740 1 -- 172.21.15.100:0/3163593887 wait complete. 2024-05-10T11:31:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.698+0000 7ff6d4fe7740 20 client.27541 trim_cache size 0 max 16384 2024-05-10T11:31:44.716 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.715+0000 7ff6d4fe7740 1 Processor -- start 2024-05-10T11:31:44.716 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.715+0000 7ff6d4fe7740 1 -- start start 2024-05-10T11:31:44.716 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.715+0000 7ff6d4fe7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584fc97edb0 0x5584fc978cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:44.716 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.715+0000 7ff6d4fe7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5584fc97f7a0 0x5584fc979230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:44.716 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.715+0000 7ff6d4fe7740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584fc984910 0x5584fc979770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:44.716 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.715+0000 7ff6d4fe7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff6cc1afd70 con 0x5584fc97edb0 2024-05-10T11:31:44.716 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.715+0000 7ff6d4fe7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff6cc1ad640 con 0x5584fc97f7a0 2024-05-10T11:31:44.716 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.715+0000 7ff6d4fe7740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff6cc1ab490 con 0x5584fc984910 2024-05-10T11:31:44.716 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.715+0000 7ff6c3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584fc97edb0 0x5584fc978cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:44.716 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.715+0000 7ff6c3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584fc97edb0 0x5584fc978cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50204/0 (socket says 172.21.15.100:50204) 2024-05-10T11:31:44.716 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.715+0000 7ff6c3fff640 1 -- 172.21.15.100:0/1604686996 learned_addr learned my addr 172.21.15.100:0/1604686996 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:31:44.716 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.715+0000 7ff6cb7fe640 1 --2- 172.21.15.100:0/1604686996 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5584fc97f7a0 0x5584fc979230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:44.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.715+0000 7ff6c3fff640 1 --2- 172.21.15.100:0/1604686996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584fc984910 0x5584fc979770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:44.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.715+0000 7ff6cb7fe640 1 -- 172.21.15.100:0/1604686996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584fc984910 msgr2=0x5584fc979770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:44.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.715+0000 7ff6cb7fe640 1 --2- 172.21.15.100:0/1604686996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584fc984910 0x5584fc979770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:44.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.715+0000 7ff6cb7fe640 1 -- 172.21.15.100:0/1604686996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584fc97edb0 msgr2=0x5584fc978cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:44.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.715+0000 7ff6cb7fe640 1 --2- 172.21.15.100:0/1604686996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584fc97edb0 0x5584fc978cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:44.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.715+0000 7ff6cb7fe640 1 -- 172.21.15.100:0/1604686996 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584fc9977a0 con 0x5584fc97f7a0 2024-05-10T11:31:44.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.715+0000 7ff6c3fff640 1 --2- 172.21.15.100:0/1604686996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584fc97edb0 0x5584fc978cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:31:44.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.715+0000 7ff6cb7fe640 1 --2- 172.21.15.100:0/1604686996 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5584fc97f7a0 0x5584fc979230 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc0096c0 tx=0x7ff6bc031a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34533a32b6d7a4b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:31:44.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.716+0000 7ff6c17fa640 1 -- 172.21.15.100:0/1604686996 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc056070 con 0x5584fc97f7a0 2024-05-10T11:31:44.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.716+0000 7ff6c17fa640 1 -- 172.21.15.100:0/1604686996 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff6bc005850 con 0x5584fc97f7a0 2024-05-10T11:31:44.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.716+0000 7ff6c17fa640 1 -- 172.21.15.100:0/1604686996 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc035040 con 0x5584fc97f7a0 2024-05-10T11:31:44.721 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.716+0000 7ff6d4fe7740 1 -- 172.21.15.100:0/1604686996 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5584fc97f7a0 msgr2=0x5584fc979230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:44.721 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.716+0000 7ff6d4fe7740 1 --2- 172.21.15.100:0/1604686996 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5584fc97f7a0 0x5584fc979230 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc0096c0 tx=0x7ff6bc031a00 comp rx=0 tx=0).stop 2024-05-10T11:31:44.721 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.716+0000 7ff6d4fe7740 1 -- 172.21.15.100:0/1604686996 shutdown_connections 2024-05-10T11:31:44.721 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.716+0000 7ff6d4fe7740 1 --2- 172.21.15.100:0/1604686996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584fc984910 0x5584fc979770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:44.721 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.716+0000 7ff6d4fe7740 1 --2- 172.21.15.100:0/1604686996 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5584fc97f7a0 0x5584fc979230 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:44.721 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.716+0000 7ff6d4fe7740 1 --2- 172.21.15.100:0/1604686996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584fc97edb0 0x5584fc978cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:44.721 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.716+0000 7ff6d4fe7740 1 -- 172.21.15.100:0/1604686996 >> 172.21.15.100:0/1604686996 conn(0x5584fc97a420 msgr2=0x5584fc980980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:31:44.721 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.717+0000 7ff6d4fe7740 1 -- 172.21.15.100:0/1604686996 shutdown_connections 2024-05-10T11:31:44.722 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.717+0000 7ff6d4fe7740 1 -- 172.21.15.100:0/1604686996 wait complete. 2024-05-10T11:31:44.722 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.717+0000 7ff6d4fe7740 1 -- start start 2024-05-10T11:31:44.722 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.717+0000 7ff6d4fe7740 1 Processor -- start 2024-05-10T11:31:44.722 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.717+0000 7ff6d4fe7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5584fc97edb0 0x5584fc98d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:44.722 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.717+0000 7ff6d4fe7740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584fc97f7a0 0x5584fc98db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:44.722 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.717+0000 7ff6d4fe7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584fc984910 0x5584fc98eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:44.722 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.717+0000 7ff6d4fe7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff6cc1abb00 con 0x5584fc984910 2024-05-10T11:31:44.722 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.717+0000 7ff6d4fe7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff6cc1acf70 con 0x5584fc97edb0 2024-05-10T11:31:44.722 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.717+0000 7ff6d4fe7740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff6cc1ac1d0 con 0x5584fc97f7a0 2024-05-10T11:31:44.722 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.717+0000 7ff6d4fe7740 10 client.0 _ll_register_callbacks cb 0x7ff6bc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:31:44.722 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.717+0000 7ff6c3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584fc984910 0x5584fc98eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:44.722 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.717+0000 7ff6cb7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584fc97f7a0 0x5584fc98db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:44.722 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.717+0000 7ff6c3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584fc984910 0x5584fc98eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50220/0 (socket says 172.21.15.100:50220) 2024-05-10T11:31:44.722 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.717+0000 7ff6c3fff640 1 -- 172.21.15.100:0/120872167 learned_addr learned my addr 172.21.15.100:0/120872167 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:31:44.723 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.717+0000 7ff6c3fff640 1 --2- 172.21.15.100:0/120872167 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5584fc97edb0 0x5584fc98d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:44.723 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.718+0000 7ff6c3fff640 1 -- 172.21.15.100:0/120872167 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584fc97f7a0 msgr2=0x5584fc98db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:44.723 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.718+0000 7ff6c3fff640 1 --2- 172.21.15.100:0/120872167 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584fc97f7a0 0x5584fc98db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:44.723 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.718+0000 7ff6c3fff640 1 -- 172.21.15.100:0/120872167 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5584fc97edb0 msgr2=0x5584fc98d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:44.723 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.718+0000 7ff6c3fff640 1 --2- 172.21.15.100:0/120872167 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5584fc97edb0 0x5584fc98d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:44.723 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.718+0000 7ff6c3fff640 1 -- 172.21.15.100:0/120872167 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584fc975a70 con 0x5584fc984910 2024-05-10T11:31:44.723 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.718+0000 7ff6c3fff640 1 --2- 172.21.15.100:0/120872167 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584fc984910 0x5584fc98eb80 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7ff6b400b830 tx=0x7ff6b4014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fc91d24da0d206c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:31:44.723 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.718+0000 7ff6aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:31:44.723 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.718+0000 7ff6aaffd640 1 -- 172.21.15.100:0/120872167 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b4004020 con 0x5584fc984910 2024-05-10T11:31:44.723 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.718+0000 7ff6aaffd640 1 -- 172.21.15.100:0/120872167 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7ff6b402dc70 con 0x5584fc984910 2024-05-10T11:31:44.723 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.718+0000 7ff6d4fe7740 1 -- 172.21.15.100:0/120872167 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6bc005850 con 0x5584fc984910 2024-05-10T11:31:44.723 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.718+0000 7ff6aaffd640 1 -- 172.21.15.100:0/120872167 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff6b403e050 con 0x5584fc984910 2024-05-10T11:31:44.723 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.718+0000 7ff6d4fe7740 10 client.18105 fetch_fsmap learned FSMap version 33 2024-05-10T11:31:44.723 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.718+0000 7ff6d4fe7740 1 -- 172.21.15.100:0/120872167 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5584fc9ff720 con 0x5584fc984910 2024-05-10T11:31:44.724 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.718+0000 7ff6aaffd640 1 -- 172.21.15.100:0/120872167 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff6bc005850 con 0x5584fc984910 2024-05-10T11:31:44.724 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.719+0000 7ff6aaffd640 1 -- 172.21.15.100:0/120872167 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7ff6b40035f0 con 0x5584fc984910 2024-05-10T11:31:44.724 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.719+0000 7ff6d4fe7740 10 client.18105 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T11:31:44.724 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.719+0000 7ff6d4fe7740 10 client.18105 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:31:44.724 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.719+0000 7ff6d4fe7740 20 client.18105 populate_metadata read hostname 'smithi100' 2024-05-10T11:31:44.724 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.719+0000 7ff6d4fe7740 1 --2- 172.21.15.100:0/120872167 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5584fc9fffc0 0x5584fca023e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:44.724 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.719+0000 7ff6d4fe7740 4 client.18105 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:31:44.724 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.719+0000 7ff6d4fe7740 1 -- 172.21.15.100:0/120872167 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5584fc8eb050 con 0x5584fc9fffc0 2024-05-10T11:31:44.724 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.719+0000 7ff6cb7fe640 1 --2- 172.21.15.100:0/120872167 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5584fc9fffc0 0x5584fca023e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:44.724 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.719+0000 7ff6aaffd640 10 client.18105 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:31:44.724 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.719+0000 7ff6cb7fe640 1 --2- 172.21.15.100:0/120872167 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5584fc9fffc0 0x5584fca023e0 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9942c2acf6ed35c2 server_cookie=ad8b0b6fd10e5d94 in_seq=0 out_seq=0 2024-05-10T11:31:44.739 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.738+0000 7ff6aaffd640 1 -- 172.21.15.100:0/120872167 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5584fc8eb050 con 0x5584fc9fffc0 2024-05-10T11:31:44.739 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.738+0000 7ff6aaffd640 10 client.18105 handle_command_reply: tid=0 2024-05-10T11:31:44.739 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.738+0000 7ff6d4fe7740 1 client.18105 shutdown 2024-05-10T11:31:44.739 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.738+0000 7ff6d4fe7740 1 -- 172.21.15.100:0/120872167 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584fc984910 msgr2=0x5584fc98eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:44.739 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.738+0000 7ff6d4fe7740 1 --2- 172.21.15.100:0/120872167 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584fc984910 0x5584fc98eb80 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7ff6b400b830 tx=0x7ff6b4014040 comp rx=0 tx=0).stop 2024-05-10T11:31:44.739 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.738+0000 7ff6d4fe7740 1 -- 172.21.15.100:0/120872167 shutdown_connections 2024-05-10T11:31:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.738+0000 7ff6d4fe7740 1 --2- 172.21.15.100:0/120872167 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5584fc9fffc0 0x5584fca023e0 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.738+0000 7ff6d4fe7740 1 --2- 172.21.15.100:0/120872167 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584fc984910 0x5584fc98eb80 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.738+0000 7ff6d4fe7740 1 --2- 172.21.15.100:0/120872167 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584fc97f7a0 0x5584fc98db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.738+0000 7ff6d4fe7740 1 --2- 172.21.15.100:0/120872167 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5584fc97edb0 0x5584fc98d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.738+0000 7ff6d4fe7740 1 -- 172.21.15.100:0/120872167 >> 172.21.15.100:0/120872167 conn(0x5584fc97a420 msgr2=0x5584fc988c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:31:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.738+0000 7ff6aaffd640 0 client.18105 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:31:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.738+0000 7ff6d4fe7740 1 -- 172.21.15.100:0/120872167 shutdown_connections 2024-05-10T11:31:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.738+0000 7ff6d4fe7740 1 -- 172.21.15.100:0/120872167 wait complete. 2024-05-10T11:31:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.739+0000 7ff6d4fe7740 20 client.18105 trim_cache size 0 max 16384 2024-05-10T11:31:44.741 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:31:44.741 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:31:44.741 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:31:44.741 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:31:44.742 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.739+0000 7ff6d21f0640 1 -- 172.21.15.100:0/879537034 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff6b80b28a0 msgr2=0x7ff6b80b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:44.742 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.739+0000 7ff6d21f0640 1 --2- 172.21.15.100:0/879537034 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff6b80b28a0 0x7ff6b80b4d60 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff6c40a9d60 tx=0x7ff6c40a4310 comp rx=0 tx=0).stop 2024-05-10T11:31:44.742 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.739+0000 7ff6d21f0640 1 -- 172.21.15.100:0/879537034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6cc1025e0 msgr2=0x7ff6cc1a2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:44.742 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.739+0000 7ff6d21f0640 1 --2- 172.21.15.100:0/879537034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6cc1025e0 0x7ff6cc1a2270 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7ff6c400eb10 tx=0x7ff6c400c2d0 comp rx=0 tx=0).stop 2024-05-10T11:31:44.742 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.739+0000 7ff6d21f0640 1 -- 172.21.15.100:0/879537034 shutdown_connections 2024-05-10T11:31:44.742 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.739+0000 7ff6d21f0640 1 --2- 172.21.15.100:0/879537034 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff6b80b28a0 0x7ff6b80b4d60 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:44.742 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.739+0000 7ff6d21f0640 1 --2- 172.21.15.100:0/879537034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6cc1025e0 0x7ff6cc1a2270 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:44.742 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.739+0000 7ff6d21f0640 1 --2- 172.21.15.100:0/879537034 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6cc101c40 0x7ff6cc19b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:44.743 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.739+0000 7ff6d21f0640 1 --2- 172.21.15.100:0/879537034 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff6cc1012e0 0x7ff6cc19ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:44.743 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.739+0000 7ff6d21f0640 1 -- 172.21.15.100:0/879537034 >> 172.21.15.100:0/879537034 conn(0x7ff6cc0fce80 msgr2=0x7ff6cc1001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:31:44.743 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.739+0000 7ff6d21f0640 1 -- 172.21.15.100:0/879537034 shutdown_connections 2024-05-10T11:31:44.743 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:44.739+0000 7ff6d21f0640 1 -- 172.21.15.100:0/879537034 wait complete. 2024-05-10T11:31:44.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.848+0000 7ff63a2e8700 1 -- 172.21.15.100:0/3355780737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff634108410 msgr2=0x7ff6341087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:44.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.848+0000 7ff63a2e8700 1 --2- 172.21.15.100:0/3355780737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff634108410 0x7ff6341087f0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7ff624009990 tx=0x7ff624009ca0 comp rx=0 tx=0).stop 2024-05-10T11:31:44.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.848+0000 7ff63a2e8700 1 -- 172.21.15.100:0/3355780737 shutdown_connections 2024-05-10T11:31:44.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.848+0000 7ff63a2e8700 1 --2- 172.21.15.100:0/3355780737 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff634102d60 0x7ff63410f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:44.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.848+0000 7ff63a2e8700 1 --2- 172.21.15.100:0/3355780737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff634108410 0x7ff6341087f0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:44.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.848+0000 7ff63a2e8700 1 --2- 172.21.15.100:0/3355780737 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff6341023a0 0x7ff634102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:44.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.848+0000 7ff63a2e8700 1 -- 172.21.15.100:0/3355780737 >> 172.21.15.100:0/3355780737 conn(0x7ff634079350 msgr2=0x7ff634079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:31:44.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.848+0000 7ff63a2e8700 1 -- 172.21.15.100:0/3355780737 shutdown_connections 2024-05-10T11:31:44.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.848+0000 7ff63a2e8700 1 -- 172.21.15.100:0/3355780737 wait complete. 2024-05-10T11:31:44.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.849+0000 7ff63a2e8700 1 Processor -- start 2024-05-10T11:31:44.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.849+0000 7ff63a2e8700 1 -- start start 2024-05-10T11:31:44.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.849+0000 7ff63a2e8700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff6341023a0 0x7ff63419a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:44.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.849+0000 7ff63a2e8700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff634102d60 0x7ff63419ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:44.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.849+0000 7ff63a2e8700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff634108410 0x7ff6341a5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:44.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.849+0000 7ff63a2e8700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff6341123e0 con 0x7ff634102d60 2024-05-10T11:31:44.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.849+0000 7ff63a2e8700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff634112260 con 0x7ff6341023a0 2024-05-10T11:31:44.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.849+0000 7ff63a2e8700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff634112560 con 0x7ff634108410 2024-05-10T11:31:44.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.849+0000 7ff6337fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff634102d60 0x7ff63419ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:44.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.850+0000 7ff6337fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff634102d60 0x7ff63419ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50226/0 (socket says 172.21.15.100:50226) 2024-05-10T11:31:44.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.850+0000 7ff6337fe700 1 -- 172.21.15.100:0/3715050285 learned_addr learned my addr 172.21.15.100:0/3715050285 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:31:44.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.850+0000 7ff638885700 1 --2- 172.21.15.100:0/3715050285 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff634108410 0x7ff6341a5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:44.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.850+0000 7ff633fff700 1 --2- 172.21.15.100:0/3715050285 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff6341023a0 0x7ff63419a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:44.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.850+0000 7ff6337fe700 1 -- 172.21.15.100:0/3715050285 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff634108410 msgr2=0x7ff6341a5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:44.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.850+0000 7ff6337fe700 1 --2- 172.21.15.100:0/3715050285 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff634108410 0x7ff6341a5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:44.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.850+0000 7ff6337fe700 1 -- 172.21.15.100:0/3715050285 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff6341023a0 msgr2=0x7ff63419a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:44.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.850+0000 7ff6337fe700 1 --2- 172.21.15.100:0/3715050285 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff6341023a0 0x7ff63419a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:44.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.850+0000 7ff6337fe700 1 -- 172.21.15.100:0/3715050285 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6341a5ea0 con 0x7ff634102d60 2024-05-10T11:31:44.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.850+0000 7ff6337fe700 1 --2- 172.21.15.100:0/3715050285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff634102d60 0x7ff63419ae20 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7ff62800e9a0 tx=0x7ff62800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:31:44.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.850+0000 7ff6317fa700 1 -- 172.21.15.100:0/3715050285 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff628020070 con 0x7ff634102d60 2024-05-10T11:31:44.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.850+0000 7ff63a2e8700 1 -- 172.21.15.100:0/3715050285 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff634198730 con 0x7ff634102d60 2024-05-10T11:31:44.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.850+0000 7ff6317fa700 1 -- 172.21.15.100:0/3715050285 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff628013d00 con 0x7ff634102d60 2024-05-10T11:31:44.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.850+0000 7ff6317fa700 1 -- 172.21.15.100:0/3715050285 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62801b730 con 0x7ff634102d60 2024-05-10T11:31:44.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.850+0000 7ff63a2e8700 1 -- 172.21.15.100:0/3715050285 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff634198cc0 con 0x7ff634102d60 2024-05-10T11:31:44.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.853+0000 7ff6317fa700 1 -- 172.21.15.100:0/3715050285 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7ff628023070 con 0x7ff634102d60 2024-05-10T11:31:44.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.854+0000 7ff6317fa700 1 --2- 172.21.15.100:0/3715050285 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff61c0b2900 0x7ff61c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:44.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.854+0000 7ff633fff700 1 --2- 172.21.15.100:0/3715050285 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff61c0b2900 0x7ff61c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:44.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.854+0000 7ff6317fa700 1 -- 172.21.15.100:0/3715050285 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7ff6280e6350 con 0x7ff634102d60 2024-05-10T11:31:44.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.854+0000 7ff644b7bb80 1 Processor -- start 2024-05-10T11:31:44.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.855+0000 7ff644b7bb80 1 -- start start 2024-05-10T11:31:44.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.855+0000 7ff633fff700 1 --2- 172.21.15.100:0/3715050285 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff61c0b2900 0x7ff61c0b4dc0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7ff62400b2e0 tx=0x7ff624005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:31:44.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.855+0000 7ff644b7bb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4c29851b0 0x55f4c2985590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:44.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.855+0000 7ff644b7bb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f4c2985ad0 0x55f4c297f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:44.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.855+0000 7ff644b7bb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4c297fd50 0x55f4c2980190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:44.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.855+0000 7ff644b7bb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55f4c28d1940 con 0x55f4c297fd50 2024-05-10T11:31:44.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.855+0000 7ff644b7bb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f4c2836da0 con 0x55f4c29851b0 2024-05-10T11:31:44.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.855+0000 7ff644b7bb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55f4c28cd900 con 0x55f4c2985ad0 2024-05-10T11:31:44.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.855+0000 7ff638885700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4c297fd50 0x55f4c2980190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:44.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.855+0000 7ff638885700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4c297fd50 0x55f4c2980190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50234/0 (socket says 172.21.15.100:50234) 2024-05-10T11:31:44.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.855+0000 7ff638885700 1 -- 172.21.15.100:0/4159177323 learned_addr learned my addr 172.21.15.100:0/4159177323 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:31:44.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.855+0000 7ff6337fe700 1 --2- 172.21.15.100:0/4159177323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f4c2985ad0 0x55f4c297f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:44.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.855+0000 7ff633fff700 1 --2- 172.21.15.100:0/4159177323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4c29851b0 0x55f4c2985590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:44.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.855+0000 7ff6337fe700 1 -- 172.21.15.100:0/4159177323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4c29851b0 msgr2=0x55f4c2985590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:44.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.855+0000 7ff6337fe700 1 --2- 172.21.15.100:0/4159177323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4c29851b0 0x55f4c2985590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:44.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.855+0000 7ff6337fe700 1 -- 172.21.15.100:0/4159177323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4c297fd50 msgr2=0x55f4c2980190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:44.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.855+0000 7ff6337fe700 1 --2- 172.21.15.100:0/4159177323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4c297fd50 0x55f4c2980190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:44.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.855+0000 7ff6337fe700 1 -- 172.21.15.100:0/4159177323 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4c281e8d0 con 0x55f4c2985ad0 2024-05-10T11:31:44.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.856+0000 7ff6337fe700 1 --2- 172.21.15.100:0/4159177323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f4c2985ad0 0x55f4c297f810 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff6280e3be0 tx=0x7ff6280acef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=edd70495c78399d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:31:44.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.856+0000 7ff61a7fc700 1 -- 172.21.15.100:0/4159177323 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6280addb0 con 0x55f4c2985ad0 2024-05-10T11:31:44.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.856+0000 7ff61a7fc700 1 -- 172.21.15.100:0/4159177323 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff6280ad6b0 con 0x55f4c2985ad0 2024-05-10T11:31:44.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.856+0000 7ff61a7fc700 1 -- 172.21.15.100:0/4159177323 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6280b6080 con 0x55f4c2985ad0 2024-05-10T11:31:44.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.857+0000 7ff644b7bb80 1 -- 172.21.15.100:0/4159177323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f4c2985ad0 msgr2=0x55f4c297f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:44.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.857+0000 7ff644b7bb80 1 --2- 172.21.15.100:0/4159177323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f4c2985ad0 0x55f4c297f810 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff6280e3be0 tx=0x7ff6280acef0 comp rx=0 tx=0).stop 2024-05-10T11:31:44.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.857+0000 7ff644b7bb80 1 -- 172.21.15.100:0/4159177323 shutdown_connections 2024-05-10T11:31:44.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.857+0000 7ff644b7bb80 1 --2- 172.21.15.100:0/4159177323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4c297fd50 0x55f4c2980190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:44.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.857+0000 7ff644b7bb80 1 --2- 172.21.15.100:0/4159177323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4c29851b0 0x55f4c2985590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:44.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.857+0000 7ff644b7bb80 1 --2- 172.21.15.100:0/4159177323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f4c2985ad0 0x55f4c297f810 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:44.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.857+0000 7ff644b7bb80 1 -- 172.21.15.100:0/4159177323 >> 172.21.15.100:0/4159177323 conn(0x55f4c28abb70 msgr2=0x55f4c297a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:31:44.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.857+0000 7ff644b7bb80 1 -- 172.21.15.100:0/4159177323 shutdown_connections 2024-05-10T11:31:44.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.857+0000 7ff644b7bb80 1 -- 172.21.15.100:0/4159177323 wait complete. 2024-05-10T11:31:44.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.857+0000 7ff644b7bb80 1 -- start start 2024-05-10T11:31:44.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.858+0000 7ff644b7bb80 1 Processor -- start 2024-05-10T11:31:44.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.858+0000 7ff644b7bb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4c297fd50 0x55f4c298b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:44.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.858+0000 7ff644b7bb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f4c29851b0 0x55f4c2988b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:44.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.858+0000 7ff644b7bb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4c2985ad0 0x55f4c2989050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:44.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.858+0000 7ff644b7bb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55f4c298c430 con 0x55f4c2985ad0 2024-05-10T11:31:44.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.858+0000 7ff644b7bb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f4c288aed0 con 0x55f4c297fd50 2024-05-10T11:31:44.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.858+0000 7ff644b7bb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55f4c298c5b0 con 0x55f4c29851b0 2024-05-10T11:31:44.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.858+0000 7ff644b7bb80 10 client.0 _ll_register_callbacks cb 0x55f4c28655b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:31:44.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.858+0000 7ff638885700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4c2985ad0 0x55f4c2989050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:44.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.858+0000 7ff638885700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4c2985ad0 0x55f4c2989050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50246/0 (socket says 172.21.15.100:50246) 2024-05-10T11:31:44.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.858+0000 7ff638885700 1 -- 172.21.15.100:0/406453154 learned_addr learned my addr 172.21.15.100:0/406453154 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:31:44.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.858+0000 7ff633fff700 1 --2- 172.21.15.100:0/406453154 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4c297fd50 0x55f4c298b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:44.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.858+0000 7ff6337fe700 1 --2- 172.21.15.100:0/406453154 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f4c29851b0 0x55f4c2988b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:44.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.858+0000 7ff638885700 1 -- 172.21.15.100:0/406453154 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f4c29851b0 msgr2=0x55f4c2988b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:44.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.858+0000 7ff638885700 1 --2- 172.21.15.100:0/406453154 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f4c29851b0 0x55f4c2988b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:44.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.858+0000 7ff638885700 1 -- 172.21.15.100:0/406453154 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4c297fd50 msgr2=0x55f4c298b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:44.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.858+0000 7ff638885700 1 --2- 172.21.15.100:0/406453154 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4c297fd50 0x55f4c298b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:44.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.858+0000 7ff638885700 1 -- 172.21.15.100:0/406453154 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4c2888760 con 0x55f4c2985ad0 2024-05-10T11:31:44.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.858+0000 7ff633fff700 1 --2- 172.21.15.100:0/406453154 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4c297fd50 0x55f4c298b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:31:44.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.859+0000 7ff638885700 1 --2- 172.21.15.100:0/406453154 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4c2985ad0 0x55f4c2989050 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7ff62c0047a0 tx=0x7ff62c014600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b918bc2529841d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:31:44.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.859+0000 7ff6137fe700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:31:44.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.859+0000 7ff6137fe700 1 -- 172.21.15.100:0/406453154 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62c0092d0 con 0x55f4c2985ad0 2024-05-10T11:31:44.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.859+0000 7ff6137fe700 1 -- 172.21.15.100:0/406453154 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7ff62c014b90 con 0x55f4c2985ad0 2024-05-10T11:31:44.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.859+0000 7ff644b7bb80 1 -- 172.21.15.100:0/406453154 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6280ad6b0 con 0x55f4c2985ad0 2024-05-10T11:31:44.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.859+0000 7ff6137fe700 1 -- 172.21.15.100:0/406453154 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff62c014d90 con 0x55f4c2985ad0 2024-05-10T11:31:44.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.859+0000 7ff644b7bb80 10 client.18117 fetch_fsmap learned FSMap version 33 2024-05-10T11:31:44.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.859+0000 7ff644b7bb80 1 -- 172.21.15.100:0/406453154 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f4c2996d70 con 0x55f4c2985ad0 2024-05-10T11:31:44.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.859+0000 7ff6137fe700 1 -- 172.21.15.100:0/406453154 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff6280ad6b0 con 0x55f4c2985ad0 2024-05-10T11:31:44.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.859+0000 7ff6137fe700 1 -- 172.21.15.100:0/406453154 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7ff62c01e420 con 0x55f4c2985ad0 2024-05-10T11:31:44.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.859+0000 7ff644b7bb80 10 client.18117 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T11:31:44.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.859+0000 7ff644b7bb80 10 client.18117 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-10T11:31:44.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.859+0000 7ff644b7bb80 20 client.18117 populate_metadata read hostname 'smithi100' 2024-05-10T11:31:44.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.860+0000 7ff644b7bb80 1 --2- 172.21.15.100:0/406453154 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55f4c29993b0 0x55f4c299b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:44.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.860+0000 7ff644b7bb80 4 client.18117 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:31:44.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.860+0000 7ff644b7bb80 1 -- 172.21.15.100:0/406453154 --> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4c28df320 con 0x55f4c29993b0 2024-05-10T11:31:44.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.860+0000 7ff633fff700 1 --2- 172.21.15.100:0/406453154 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55f4c29993b0 0x55f4c299b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:44.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.860+0000 7ff6137fe700 10 client.18117 ms_handle_connect on v2:172.21.15.178:6838/3760985404 2024-05-10T11:31:44.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.860+0000 7ff633fff700 1 --2- 172.21.15.100:0/406453154 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55f4c29993b0 0x55f4c299b7a0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f1407484384503c1 server_cookie=9845159dde0a8168 in_seq=0 out_seq=0 2024-05-10T11:31:44.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.861+0000 7ff6137fe700 1 -- 172.21.15.100:0/406453154 <== mds.1 v2:172.21.15.178:6838/3760985404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f4c28df320 con 0x55f4c29993b0 2024-05-10T11:31:44.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.861+0000 7ff6137fe700 10 client.18117 handle_command_reply: tid=0 2024-05-10T11:31:44.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.861+0000 7ff644b7bb80 1 client.18117 shutdown 2024-05-10T11:31:44.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.861+0000 7ff644b7bb80 1 -- 172.21.15.100:0/406453154 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4c2985ad0 msgr2=0x55f4c2989050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:44.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.861+0000 7ff644b7bb80 1 --2- 172.21.15.100:0/406453154 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4c2985ad0 0x55f4c2989050 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7ff62c0047a0 tx=0x7ff62c014600 comp rx=0 tx=0).stop 2024-05-10T11:31:44.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.861+0000 7ff644b7bb80 1 -- 172.21.15.100:0/406453154 shutdown_connections 2024-05-10T11:31:44.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.861+0000 7ff644b7bb80 1 --2- 172.21.15.100:0/406453154 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55f4c29993b0 0x55f4c299b7a0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:44.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.861+0000 7ff644b7bb80 1 --2- 172.21.15.100:0/406453154 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4c2985ad0 0x55f4c2989050 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:44.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.861+0000 7ff644b7bb80 1 --2- 172.21.15.100:0/406453154 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4c297fd50 0x55f4c298b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:44.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.861+0000 7ff644b7bb80 1 --2- 172.21.15.100:0/406453154 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f4c29851b0 0x55f4c2988b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:44.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.861+0000 7ff644b7bb80 1 -- 172.21.15.100:0/406453154 >> 172.21.15.100:0/406453154 conn(0x55f4c28abb70 msgr2=0x55f4c2974b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:31:44.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.861+0000 7ff6137fe700 0 client.18117 ms_handle_reset on v2:172.21.15.178:6838/3760985404 2024-05-10T11:31:44.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.861+0000 7ff644b7bb80 1 -- 172.21.15.100:0/406453154 shutdown_connections 2024-05-10T11:31:44.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.861+0000 7ff644b7bb80 1 -- 172.21.15.100:0/406453154 wait complete. 2024-05-10T11:31:44.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.862+0000 7ff644b7bb80 20 client.18117 trim_cache size 0 max 16384 2024-05-10T11:31:44.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.879+0000 7ff644b7bb80 1 Processor -- start 2024-05-10T11:31:44.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.879+0000 7ff644b7bb80 1 -- start start 2024-05-10T11:31:44.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.879+0000 7ff644b7bb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4c297fd50 0x55f4c29891e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:44.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.879+0000 7ff644b7bb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4c29851b0 0x55f4c298afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:44.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.879+0000 7ff644b7bb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f4c2985ad0 0x55f4c29954c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:44.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.879+0000 7ff644b7bb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff6341af7d0 con 0x55f4c297fd50 2024-05-10T11:31:44.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.879+0000 7ff644b7bb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff6341ad060 con 0x55f4c29851b0 2024-05-10T11:31:44.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.879+0000 7ff644b7bb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff6341aaeb0 con 0x55f4c2985ad0 2024-05-10T11:31:44.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.879+0000 7ff633fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4c297fd50 0x55f4c29891e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:44.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.879+0000 7ff633fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4c297fd50 0x55f4c29891e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50252/0 (socket says 172.21.15.100:50252) 2024-05-10T11:31:44.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.879+0000 7ff633fff700 1 -- 172.21.15.100:0/3355418023 learned_addr learned my addr 172.21.15.100:0/3355418023 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:31:44.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.879+0000 7ff638885700 1 --2- 172.21.15.100:0/3355418023 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f4c2985ad0 0x55f4c29954c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:44.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.879+0000 7ff6337fe700 1 --2- 172.21.15.100:0/3355418023 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4c29851b0 0x55f4c298afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:44.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.879+0000 7ff633fff700 1 -- 172.21.15.100:0/3355418023 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f4c2985ad0 msgr2=0x55f4c29954c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:44.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.879+0000 7ff633fff700 1 --2- 172.21.15.100:0/3355418023 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f4c2985ad0 0x55f4c29954c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:44.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.879+0000 7ff633fff700 1 -- 172.21.15.100:0/3355418023 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4c29851b0 msgr2=0x55f4c298afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:44.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.879+0000 7ff633fff700 1 --2- 172.21.15.100:0/3355418023 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4c29851b0 0x55f4c298afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:44.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.880+0000 7ff633fff700 1 -- 172.21.15.100:0/3355418023 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4c29834e0 con 0x55f4c297fd50 2024-05-10T11:31:44.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.880+0000 7ff633fff700 1 --2- 172.21.15.100:0/3355418023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4c297fd50 0x55f4c29891e0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7ff624009990 tx=0x7ff624022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7a42743d58e9357 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:31:44.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.880+0000 7ff619ffb700 1 -- 172.21.15.100:0/3355418023 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62400f660 con 0x55f4c297fd50 2024-05-10T11:31:44.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.881+0000 7ff619ffb700 1 -- 172.21.15.100:0/3355418023 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff62400fc90 con 0x55f4c297fd50 2024-05-10T11:31:44.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.881+0000 7ff619ffb700 1 -- 172.21.15.100:0/3355418023 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff624033a90 con 0x55f4c297fd50 2024-05-10T11:31:44.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.881+0000 7ff644b7bb80 1 -- 172.21.15.100:0/3355418023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4c297fd50 msgr2=0x55f4c29891e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:44.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.881+0000 7ff644b7bb80 1 --2- 172.21.15.100:0/3355418023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4c297fd50 0x55f4c29891e0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7ff624009990 tx=0x7ff624022000 comp rx=0 tx=0).stop 2024-05-10T11:31:44.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.882+0000 7ff644b7bb80 1 -- 172.21.15.100:0/3355418023 shutdown_connections 2024-05-10T11:31:44.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.882+0000 7ff644b7bb80 1 --2- 172.21.15.100:0/3355418023 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f4c2985ad0 0x55f4c29954c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:44.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.882+0000 7ff644b7bb80 1 --2- 172.21.15.100:0/3355418023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4c297fd50 0x55f4c29891e0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:44.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.882+0000 7ff644b7bb80 1 --2- 172.21.15.100:0/3355418023 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4c29851b0 0x55f4c298afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:44.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.882+0000 7ff644b7bb80 1 -- 172.21.15.100:0/3355418023 >> 172.21.15.100:0/3355418023 conn(0x55f4c28abb70 msgr2=0x55f4c2988ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:31:44.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.882+0000 7ff644b7bb80 1 -- 172.21.15.100:0/3355418023 shutdown_connections 2024-05-10T11:31:44.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.882+0000 7ff644b7bb80 1 -- 172.21.15.100:0/3355418023 wait complete. 2024-05-10T11:31:44.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.882+0000 7ff644b7bb80 1 -- start start 2024-05-10T11:31:44.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.882+0000 7ff644b7bb80 1 Processor -- start 2024-05-10T11:31:44.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.883+0000 7ff644b7bb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4c297fd50 0x55f4c2991c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:44.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.883+0000 7ff644b7bb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4c29851b0 0x55f4c29a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:44.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.883+0000 7ff644b7bb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f4c2985ad0 0x55f4c29a0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:44.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.883+0000 7ff644b7bb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff6341ab520 con 0x55f4c297fd50 2024-05-10T11:31:44.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.883+0000 7ff644b7bb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff6341ac990 con 0x55f4c29851b0 2024-05-10T11:31:44.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.883+0000 7ff644b7bb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff6341abbf0 con 0x55f4c2985ad0 2024-05-10T11:31:44.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.883+0000 7ff644b7bb80 10 client.0 _ll_register_callbacks cb 0x55f4c28655b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:31:44.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.883+0000 7ff633fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4c297fd50 0x55f4c2991c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:44.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.883+0000 7ff633fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4c297fd50 0x55f4c2991c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50264/0 (socket says 172.21.15.100:50264) 2024-05-10T11:31:44.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.883+0000 7ff633fff700 1 -- 172.21.15.100:0/2220190110 learned_addr learned my addr 172.21.15.100:0/2220190110 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:31:44.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.883+0000 7ff638885700 1 --2- 172.21.15.100:0/2220190110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f4c2985ad0 0x55f4c29a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:44.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.883+0000 7ff638885700 1 -- 172.21.15.100:0/2220190110 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4c29851b0 msgr2=0x55f4c29a03f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T11:31:44.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.883+0000 7ff638885700 1 --2- 172.21.15.100:0/2220190110 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4c29851b0 0x55f4c29a03f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:44.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.883+0000 7ff638885700 1 -- 172.21.15.100:0/2220190110 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4c297fd50 msgr2=0x55f4c2991c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:44.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.883+0000 7ff638885700 1 --2- 172.21.15.100:0/2220190110 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4c297fd50 0x55f4c2991c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:44.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.883+0000 7ff638885700 1 -- 172.21.15.100:0/2220190110 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4c29a0f70 con 0x55f4c2985ad0 2024-05-10T11:31:44.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.883+0000 7ff633fff700 1 --2- 172.21.15.100:0/2220190110 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4c297fd50 0x55f4c2991c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:31:44.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.884+0000 7ff638885700 1 --2- 172.21.15.100:0/2220190110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f4c2985ad0 0x55f4c29a0930 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff62c01ea30 tx=0x7ff62c009300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=619b84094b8671f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:31:44.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.884+0000 7ff6137fe700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:31:44.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.884+0000 7ff6137fe700 1 -- 172.21.15.100:0/2220190110 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62c01eab0 con 0x55f4c2985ad0 2024-05-10T11:31:44.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.884+0000 7ff6137fe700 1 -- 172.21.15.100:0/2220190110 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7ff62c01ec50 con 0x55f4c2985ad0 2024-05-10T11:31:44.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.884+0000 7ff644b7bb80 1 -- 172.21.15.100:0/2220190110 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff62400fc90 con 0x55f4c2985ad0 2024-05-10T11:31:44.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.884+0000 7ff6137fe700 1 -- 172.21.15.100:0/2220190110 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff62c01ee50 con 0x55f4c2985ad0 2024-05-10T11:31:44.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.885+0000 7ff644b7bb80 10 client.27565 fetch_fsmap learned FSMap version 33 2024-05-10T11:31:44.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.885+0000 7ff644b7bb80 1 -- 172.21.15.100:0/2220190110 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f4c2a039e0 con 0x55f4c2985ad0 2024-05-10T11:31:44.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.885+0000 7ff6137fe700 1 -- 172.21.15.100:0/2220190110 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff62c007500 con 0x55f4c2985ad0 2024-05-10T11:31:44.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.885+0000 7ff6137fe700 1 -- 172.21.15.100:0/2220190110 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7ff62c016400 con 0x55f4c2985ad0 2024-05-10T11:31:44.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.885+0000 7ff644b7bb80 10 client.27565 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T11:31:44.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.885+0000 7ff644b7bb80 10 client.27565 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-10T11:31:44.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.885+0000 7ff644b7bb80 20 client.27565 populate_metadata read hostname 'smithi100' 2024-05-10T11:31:44.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.885+0000 7ff644b7bb80 1 --2- 172.21.15.100:0/2220190110 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55f4c2a04090 0x55f4c2a064b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:44.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.885+0000 7ff644b7bb80 4 client.27565 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:31:44.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.885+0000 7ff644b7bb80 1 -- 172.21.15.100:0/2220190110 --> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7ff62400fc90 con 0x55f4c2a04090 2024-05-10T11:31:44.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.886+0000 7ff633fff700 1 --2- 172.21.15.100:0/2220190110 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55f4c2a04090 0x55f4c2a064b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:44.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.886+0000 7ff633fff700 1 --2- 172.21.15.100:0/2220190110 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55f4c2a04090 0x55f4c2a064b0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=36f1b25690208c29 server_cookie=3698a881e65bf6e1 in_seq=0 out_seq=0 2024-05-10T11:31:44.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.886+0000 7ff6137fe700 10 client.27565 ms_handle_connect on v2:172.21.15.178:6838/3760985404 2024-05-10T11:31:44.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.886+0000 7ff6137fe700 1 -- 172.21.15.100:0/2220190110 <== mds.1 v2:172.21.15.178:6838/3760985404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7ff62400fc90 con 0x55f4c2a04090 2024-05-10T11:31:44.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.886+0000 7ff6137fe700 10 client.27565 handle_command_reply: tid=0 2024-05-10T11:31:44.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.887+0000 7ff644b7bb80 1 client.27565 shutdown 2024-05-10T11:31:44.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.887+0000 7ff644b7bb80 1 -- 172.21.15.100:0/2220190110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f4c2985ad0 msgr2=0x55f4c29a0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:44.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.887+0000 7ff644b7bb80 1 --2- 172.21.15.100:0/2220190110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f4c2985ad0 0x55f4c29a0930 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff62c01ea30 tx=0x7ff62c009300 comp rx=0 tx=0).stop 2024-05-10T11:31:44.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.887+0000 7ff644b7bb80 1 -- 172.21.15.100:0/2220190110 shutdown_connections 2024-05-10T11:31:44.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.887+0000 7ff644b7bb80 1 --2- 172.21.15.100:0/2220190110 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55f4c2a04090 0x55f4c2a064b0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:44.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.887+0000 7ff644b7bb80 1 --2- 172.21.15.100:0/2220190110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f4c2985ad0 0x55f4c29a0930 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:44.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.887+0000 7ff644b7bb80 1 --2- 172.21.15.100:0/2220190110 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4c297fd50 0x55f4c2991c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:44.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.887+0000 7ff644b7bb80 1 --2- 172.21.15.100:0/2220190110 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4c29851b0 0x55f4c29a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:44.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.887+0000 7ff6137fe700 0 client.27565 ms_handle_reset on v2:172.21.15.178:6838/3760985404 2024-05-10T11:31:44.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.887+0000 7ff644b7bb80 1 -- 172.21.15.100:0/2220190110 >> 172.21.15.100:0/2220190110 conn(0x55f4c28abb70 msgr2=0x55f4c2985eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:31:44.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.887+0000 7ff644b7bb80 1 -- 172.21.15.100:0/2220190110 shutdown_connections 2024-05-10T11:31:44.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.887+0000 7ff644b7bb80 1 -- 172.21.15.100:0/2220190110 wait complete. 2024-05-10T11:31:44.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.887+0000 7ff644b7bb80 20 client.27565 trim_cache size 0 max 16384 2024-05-10T11:31:44.892 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:31:44.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.888+0000 7ff63a2e8700 1 -- 172.21.15.100:0/3715050285 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff61c0b2900 msgr2=0x7ff61c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:44.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.888+0000 7ff63a2e8700 1 --2- 172.21.15.100:0/3715050285 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff61c0b2900 0x7ff61c0b4dc0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7ff62400b2e0 tx=0x7ff624005fb0 comp rx=0 tx=0).stop 2024-05-10T11:31:44.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.888+0000 7ff63a2e8700 1 -- 172.21.15.100:0/3715050285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff634102d60 msgr2=0x7ff63419ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:44.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.888+0000 7ff63a2e8700 1 --2- 172.21.15.100:0/3715050285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff634102d60 0x7ff63419ae20 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7ff62800e9a0 tx=0x7ff62800ecb0 comp rx=0 tx=0).stop 2024-05-10T11:31:44.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.888+0000 7ff63a2e8700 1 -- 172.21.15.100:0/3715050285 shutdown_connections 2024-05-10T11:31:44.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.888+0000 7ff63a2e8700 1 --2- 172.21.15.100:0/3715050285 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff61c0b2900 0x7ff61c0b4dc0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:44.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.888+0000 7ff63a2e8700 1 --2- 172.21.15.100:0/3715050285 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff634108410 0x7ff6341a5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:44.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.888+0000 7ff63a2e8700 1 --2- 172.21.15.100:0/3715050285 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff6341023a0 0x7ff63419a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:44.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.888+0000 7ff63a2e8700 1 --2- 172.21.15.100:0/3715050285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff634102d60 0x7ff63419ae20 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:44.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.888+0000 7ff63a2e8700 1 -- 172.21.15.100:0/3715050285 >> 172.21.15.100:0/3715050285 conn(0x7ff634079350 msgr2=0x7ff634101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:31:44.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.888+0000 7ff63a2e8700 1 -- 172.21.15.100:0/3715050285 shutdown_connections 2024-05-10T11:31:44.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:44.888+0000 7ff63a2e8700 1 -- 172.21.15.100:0/3715050285 wait complete. 2024-05-10T11:31:44.985 INFO:tasks.workunit.client.0.smithi100.stdout: 1 354445 2.70 MB/sec execute 579 sec latency 42.453 ms 2024-05-10T11:31:45.582 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-10T11:31:45.852 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:31:45.975 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:31:45 smithi100 ceph-mon[30296]: pgmap v822: 129 pgs: 129 active+clean; 137 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 1.2 MiB/s wr, 123 op/s 2024-05-10T11:31:45.985 INFO:tasks.workunit.client.0.smithi100.stdout: 1 354838 2.70 MB/sec execute 580 sec latency 28.429 ms 2024-05-10T11:31:46.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:31:45 smithi178 ceph-mon[33359]: pgmap v822: 129 pgs: 129 active+clean; 137 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 1.2 MiB/s wr, 123 op/s 2024-05-10T11:31:46.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:31:45 smithi188 ceph-mon[34075]: pgmap v822: 129 pgs: 129 active+clean; 137 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 1.2 MiB/s wr, 123 op/s 2024-05-10T11:31:46.985 INFO:tasks.workunit.client.0.smithi100.stdout: 1 355141 2.70 MB/sec execute 581 sec latency 12.542 ms 2024-05-10T11:31:47.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.547+0000 7f2b224a8700 1 -- 172.21.15.100:0/3990209137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b1c101b80 msgr2=0x7f2b1c102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:47.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.547+0000 7f2b224a8700 1 --2- 172.21.15.100:0/3990209137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b1c101b80 0x7f2b1c102000 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f2b100099c0 tx=0x7f2b10009cd0 comp rx=0 tx=0).stop 2024-05-10T11:31:47.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.547+0000 7f2b224a8700 1 -- 172.21.15.100:0/3990209137 shutdown_connections 2024-05-10T11:31:47.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.547+0000 7f2b224a8700 1 --2- 172.21.15.100:0/3990209137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b1c102540 0x7f2b1c10e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:47.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.547+0000 7f2b224a8700 1 --2- 172.21.15.100:0/3990209137 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b1c107be0 0x7f2b1c107fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:47.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.547+0000 7f2b224a8700 1 --2- 172.21.15.100:0/3990209137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b1c101b80 0x7f2b1c102000 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:47.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.547+0000 7f2b224a8700 1 -- 172.21.15.100:0/3990209137 >> 172.21.15.100:0/3990209137 conn(0x7f2b1c078a10 msgr2=0x7f2b1c0ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:31:47.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.547+0000 7f2b224a8700 1 -- 172.21.15.100:0/3990209137 shutdown_connections 2024-05-10T11:31:47.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.547+0000 7f2b224a8700 1 -- 172.21.15.100:0/3990209137 wait complete. 2024-05-10T11:31:47.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.548+0000 7f2b224a8700 1 Processor -- start 2024-05-10T11:31:47.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.548+0000 7f2b224a8700 1 -- start start 2024-05-10T11:31:47.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.548+0000 7f2b224a8700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b1c101b80 0x7f2b1c198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:47.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.548+0000 7f2b224a8700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b1c102540 0x7f2b1c198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:47.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.548+0000 7f2b224a8700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b1c107be0 0x7f2b1c199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:47.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.548+0000 7f2b224a8700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1c112430 con 0x7f2b1c107be0 2024-05-10T11:31:47.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.548+0000 7f2b224a8700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1c1122b0 con 0x7f2b1c102540 2024-05-10T11:31:47.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.548+0000 7f2b224a8700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1c1125b0 con 0x7f2b1c101b80 2024-05-10T11:31:47.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.549+0000 7f2b20a45700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b1c107be0 0x7f2b1c199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:47.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.549+0000 7f2b20a45700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b1c107be0 0x7f2b1c199080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43432/0 (socket says 172.21.15.100:43432) 2024-05-10T11:31:47.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.549+0000 7f2b20a45700 1 -- 172.21.15.100:0/546256809 learned_addr learned my addr 172.21.15.100:0/546256809 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:31:47.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.549+0000 7f2b1bfff700 1 --2- 172.21.15.100:0/546256809 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b1c101b80 0x7f2b1c198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:47.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.549+0000 7f2b1b7fe700 1 --2- 172.21.15.100:0/546256809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b1c102540 0x7f2b1c198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:47.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.549+0000 7f2b20a45700 1 -- 172.21.15.100:0/546256809 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b1c101b80 msgr2=0x7f2b1c198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:47.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.549+0000 7f2b20a45700 1 --2- 172.21.15.100:0/546256809 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b1c101b80 0x7f2b1c198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:47.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.549+0000 7f2b20a45700 1 -- 172.21.15.100:0/546256809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b1c102540 msgr2=0x7f2b1c198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:47.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.549+0000 7f2b20a45700 1 --2- 172.21.15.100:0/546256809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b1c102540 0x7f2b1c198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:47.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.549+0000 7f2b20a45700 1 -- 172.21.15.100:0/546256809 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b1c1a7e70 con 0x7f2b1c107be0 2024-05-10T11:31:47.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.549+0000 7f2b20a45700 1 --2- 172.21.15.100:0/546256809 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b1c107be0 0x7f2b1c199080 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f2b1400ef30 tx=0x7f2b1400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:31:47.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.549+0000 7f2b197fa700 1 -- 172.21.15.100:0/546256809 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b140092e0 con 0x7f2b1c107be0 2024-05-10T11:31:47.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.549+0000 7f2b197fa700 1 -- 172.21.15.100:0/546256809 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2b14004bb0 con 0x7f2b1c107be0 2024-05-10T11:31:47.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.549+0000 7f2b224a8700 1 -- 172.21.15.100:0/546256809 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b1c1a8100 con 0x7f2b1c107be0 2024-05-10T11:31:47.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.549+0000 7f2b197fa700 1 -- 172.21.15.100:0/546256809 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b14004020 con 0x7f2b1c107be0 2024-05-10T11:31:47.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.549+0000 7f2b224a8700 1 -- 172.21.15.100:0/546256809 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b1c1a8640 con 0x7f2b1c107be0 2024-05-10T11:31:47.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.552+0000 7f2b197fa700 1 -- 172.21.15.100:0/546256809 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f2b1401b030 con 0x7f2b1c107be0 2024-05-10T11:31:47.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.552+0000 7f2b197fa700 1 --2- 172.21.15.100:0/546256809 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2b040b2880 0x7f2b040b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:47.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.553+0000 7f2b1bfff700 1 --2- 172.21.15.100:0/546256809 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2b040b2880 0x7f2b040b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:47.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.553+0000 7f2b197fa700 1 -- 172.21.15.100:0/546256809 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f2b140e1ff0 con 0x7f2b1c107be0 2024-05-10T11:31:47.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.553+0000 7f2b1bfff700 1 --2- 172.21.15.100:0/546256809 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2b040b2880 0x7f2b040b4d40 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c005950 tx=0x7f2b0c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:31:47.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.553+0000 7f2b2cd3bb80 1 Processor -- start 2024-05-10T11:31:47.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.553+0000 7f2b2cd3bb80 1 -- start start 2024-05-10T11:31:47.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.554+0000 7f2b2cd3bb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bf84c9d4d0 0x55bf84c9d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:47.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.554+0000 7f2b2cd3bb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bf84c9ddf0 0x55bf84ca16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:47.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.554+0000 7f2b2cd3bb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf84ca1c80 0x55bf84ca2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:47.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.554+0000 7f2b2cd3bb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55bf84bef860 con 0x55bf84ca1c80 2024-05-10T11:31:47.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.554+0000 7f2b2cd3bb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55bf84b8e7c0 con 0x55bf84c9d4d0 2024-05-10T11:31:47.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.554+0000 7f2b2cd3bb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55bf84beb820 con 0x55bf84c9ddf0 2024-05-10T11:31:47.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.554+0000 7f2b20a45700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf84ca1c80 0x55bf84ca2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:47.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.554+0000 7f2b1bfff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bf84c9d4d0 0x55bf84c9d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:47.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.554+0000 7f2b20a45700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf84ca1c80 0x55bf84ca2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43438/0 (socket says 172.21.15.100:43438) 2024-05-10T11:31:47.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.554+0000 7f2b20a45700 1 -- 172.21.15.100:0/2770546279 learned_addr learned my addr 172.21.15.100:0/2770546279 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:31:47.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.554+0000 7f2b1b7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bf84c9ddf0 0x55bf84ca16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:47.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.554+0000 7f2b20a45700 1 -- 172.21.15.100:0/2770546279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bf84c9ddf0 msgr2=0x55bf84ca16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:47.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.554+0000 7f2b20a45700 1 --2- 172.21.15.100:0/2770546279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bf84c9ddf0 0x55bf84ca16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:47.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.554+0000 7f2b20a45700 1 -- 172.21.15.100:0/2770546279 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bf84c9d4d0 msgr2=0x55bf84c9d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:47.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.554+0000 7f2b20a45700 1 --2- 172.21.15.100:0/2770546279 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bf84c9d4d0 0x55bf84c9d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:47.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.554+0000 7f2b20a45700 1 -- 172.21.15.100:0/2770546279 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf84b3c8d0 con 0x55bf84ca1c80 2024-05-10T11:31:47.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.554+0000 7f2b1bfff700 1 --2- 172.21.15.100:0/2770546279 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bf84c9d4d0 0x55bf84c9d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:31:47.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.554+0000 7f2b20a45700 1 --2- 172.21.15.100:0/2770546279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf84ca1c80 0x55bf84ca2100 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f2b140a53c0 tx=0x7f2b140a9220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d58a549d8385aa server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:31:47.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.555+0000 7f2b027fc700 1 -- 172.21.15.100:0/2770546279 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b140a9340 con 0x55bf84ca1c80 2024-05-10T11:31:47.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.555+0000 7f2b027fc700 1 -- 172.21.15.100:0/2770546279 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2b140ad6f0 con 0x55bf84ca1c80 2024-05-10T11:31:47.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.555+0000 7f2b027fc700 1 -- 172.21.15.100:0/2770546279 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b140b1f10 con 0x55bf84ca1c80 2024-05-10T11:31:47.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.555+0000 7f2b2cd3bb80 1 -- 172.21.15.100:0/2770546279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf84ca1c80 msgr2=0x55bf84ca2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:47.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.555+0000 7f2b2cd3bb80 1 --2- 172.21.15.100:0/2770546279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf84ca1c80 0x55bf84ca2100 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f2b140a53c0 tx=0x7f2b140a9220 comp rx=0 tx=0).stop 2024-05-10T11:31:47.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.556+0000 7f2b2cd3bb80 1 -- 172.21.15.100:0/2770546279 shutdown_connections 2024-05-10T11:31:47.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.556+0000 7f2b2cd3bb80 1 --2- 172.21.15.100:0/2770546279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf84ca1c80 0x55bf84ca2100 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:47.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.556+0000 7f2b2cd3bb80 1 --2- 172.21.15.100:0/2770546279 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bf84c9d4d0 0x55bf84c9d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:47.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.556+0000 7f2b2cd3bb80 1 --2- 172.21.15.100:0/2770546279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bf84c9ddf0 0x55bf84ca16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:47.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.556+0000 7f2b2cd3bb80 1 -- 172.21.15.100:0/2770546279 >> 172.21.15.100:0/2770546279 conn(0x55bf84bc9100 msgr2=0x55bf84c97c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:31:47.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.556+0000 7f2b2cd3bb80 1 -- 172.21.15.100:0/2770546279 shutdown_connections 2024-05-10T11:31:47.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.556+0000 7f2b2cd3bb80 1 -- 172.21.15.100:0/2770546279 wait complete. 2024-05-10T11:31:47.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.556+0000 7f2b2cd3bb80 1 -- start start 2024-05-10T11:31:47.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.556+0000 7f2b2cd3bb80 1 Processor -- start 2024-05-10T11:31:47.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.557+0000 7f2b2cd3bb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf84c9d4d0 0x55bf84ca90c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:47.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.557+0000 7f2b2cd3bb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bf84c9ddf0 0x55bf84ca64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:47.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.557+0000 7f2b2cd3bb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bf84ca1c80 0x55bf84ca69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:47.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.557+0000 7f2b2cd3bb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55bf84ca9e90 con 0x55bf84c9d4d0 2024-05-10T11:31:47.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.557+0000 7f2b2cd3bb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55bf84b55c30 con 0x55bf84c9ddf0 2024-05-10T11:31:47.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.557+0000 7f2b2cd3bb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55bf84caa010 con 0x55bf84ca1c80 2024-05-10T11:31:47.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.557+0000 7f2b2cd3bb80 10 client.0 _ll_register_callbacks cb 0x55bf84b87920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:31:47.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.557+0000 7f2b1bfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf84c9d4d0 0x55bf84ca90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:47.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.557+0000 7f2b1bfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf84c9d4d0 0x55bf84ca90c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43454/0 (socket says 172.21.15.100:43454) 2024-05-10T11:31:47.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.557+0000 7f2b1bfff700 1 -- 172.21.15.100:0/3196286488 learned_addr learned my addr 172.21.15.100:0/3196286488 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:31:47.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.557+0000 7f2b20a45700 1 --2- 172.21.15.100:0/3196286488 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bf84ca1c80 0x55bf84ca69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:47.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.557+0000 7f2b1b7fe700 1 --2- 172.21.15.100:0/3196286488 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bf84c9ddf0 0x55bf84ca64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:47.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.557+0000 7f2b1bfff700 1 -- 172.21.15.100:0/3196286488 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bf84ca1c80 msgr2=0x55bf84ca69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:47.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.557+0000 7f2b1bfff700 1 --2- 172.21.15.100:0/3196286488 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bf84ca1c80 0x55bf84ca69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:47.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.557+0000 7f2b1bfff700 1 -- 172.21.15.100:0/3196286488 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bf84c9ddf0 msgr2=0x55bf84ca64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:47.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.557+0000 7f2b1bfff700 1 --2- 172.21.15.100:0/3196286488 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bf84c9ddf0 0x55bf84ca64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:47.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.557+0000 7f2b1bfff700 1 -- 172.21.15.100:0/3196286488 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf84b8b690 con 0x55bf84c9d4d0 2024-05-10T11:31:47.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.557+0000 7f2b1bfff700 1 --2- 172.21.15.100:0/3196286488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf84c9d4d0 0x55bf84ca90c0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c011860 tx=0x7f2b0c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7841abdbddef0531 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:31:47.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.557+0000 7f2afb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:31:47.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.558+0000 7f2afb7fe700 1 -- 172.21.15.100:0/3196286488 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c01f840 con 0x55bf84c9d4d0 2024-05-10T11:31:47.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.558+0000 7f2afb7fe700 1 -- 172.21.15.100:0/3196286488 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f2b0c01f9e0 con 0x55bf84c9d4d0 2024-05-10T11:31:47.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.558+0000 7f2b2cd3bb80 1 -- 172.21.15.100:0/3196286488 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b140ad6f0 con 0x55bf84c9d4d0 2024-05-10T11:31:47.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.558+0000 7f2afb7fe700 1 -- 172.21.15.100:0/3196286488 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f2b0c01fbe0 con 0x55bf84c9d4d0 2024-05-10T11:31:47.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.558+0000 7f2b2cd3bb80 10 client.18135 fetch_fsmap learned FSMap version 33 2024-05-10T11:31:47.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.558+0000 7f2b2cd3bb80 1 -- 172.21.15.100:0/3196286488 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55bf84cb4750 con 0x55bf84c9d4d0 2024-05-10T11:31:47.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.559+0000 7f2afb7fe700 1 -- 172.21.15.100:0/3196286488 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2b0c023070 con 0x55bf84c9d4d0 2024-05-10T11:31:47.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.559+0000 7f2afb7fe700 1 -- 172.21.15.100:0/3196286488 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f2b0c005130 con 0x55bf84c9d4d0 2024-05-10T11:31:47.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.559+0000 7f2b2cd3bb80 10 client.18135 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T11:31:47.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.559+0000 7f2b2cd3bb80 10 client.18135 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:31:47.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.559+0000 7f2b2cd3bb80 20 client.18135 populate_metadata read hostname 'smithi100' 2024-05-10T11:31:47.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.559+0000 7f2b2cd3bb80 1 --2- 172.21.15.100:0/3196286488 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55bf84cb6d90 0x55bf84cb91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:47.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.559+0000 7f2b2cd3bb80 4 client.18135 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:31:47.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.559+0000 7f2b2cd3bb80 1 -- 172.21.15.100:0/3196286488 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2b140ad6f0 con 0x55bf84cb6d90 2024-05-10T11:31:47.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.559+0000 7f2b1b7fe700 1 --2- 172.21.15.100:0/3196286488 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55bf84cb6d90 0x55bf84cb91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:47.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.560+0000 7f2afb7fe700 10 client.18135 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:31:47.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.560+0000 7f2b1b7fe700 1 --2- 172.21.15.100:0/3196286488 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55bf84cb6d90 0x55bf84cb91a0 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73952a31bdb78742 server_cookie=399bf50df57bbe70 in_seq=0 out_seq=0 2024-05-10T11:31:47.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.560+0000 7f2afb7fe700 1 -- 172.21.15.100:0/3196286488 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f2b140ad6f0 con 0x55bf84cb6d90 2024-05-10T11:31:47.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.560+0000 7f2afb7fe700 10 client.18135 handle_command_reply: tid=0 2024-05-10T11:31:47.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.561+0000 7f2b2cd3bb80 1 client.18135 shutdown 2024-05-10T11:31:47.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.561+0000 7f2b2cd3bb80 1 -- 172.21.15.100:0/3196286488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf84c9d4d0 msgr2=0x55bf84ca90c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:47.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.561+0000 7f2b2cd3bb80 1 --2- 172.21.15.100:0/3196286488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf84c9d4d0 0x55bf84ca90c0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c011860 tx=0x7f2b0c0047c0 comp rx=0 tx=0).stop 2024-05-10T11:31:47.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.561+0000 7f2b2cd3bb80 1 -- 172.21.15.100:0/3196286488 shutdown_connections 2024-05-10T11:31:47.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.561+0000 7f2b2cd3bb80 1 --2- 172.21.15.100:0/3196286488 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55bf84cb6d90 0x55bf84cb91a0 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:47.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.561+0000 7f2b2cd3bb80 1 --2- 172.21.15.100:0/3196286488 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bf84ca1c80 0x55bf84ca69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:47.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.561+0000 7f2b2cd3bb80 1 --2- 172.21.15.100:0/3196286488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf84c9d4d0 0x55bf84ca90c0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:47.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.561+0000 7f2b2cd3bb80 1 --2- 172.21.15.100:0/3196286488 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bf84c9ddf0 0x55bf84ca64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:47.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.561+0000 7f2b2cd3bb80 1 -- 172.21.15.100:0/3196286488 >> 172.21.15.100:0/3196286488 conn(0x55bf84bc9100 msgr2=0x55bf84c97c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:31:47.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.561+0000 7f2afb7fe700 0 client.18135 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:31:47.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.561+0000 7f2b2cd3bb80 1 -- 172.21.15.100:0/3196286488 shutdown_connections 2024-05-10T11:31:47.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.561+0000 7f2b2cd3bb80 1 -- 172.21.15.100:0/3196286488 wait complete. 2024-05-10T11:31:47.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.562+0000 7f2b2cd3bb80 20 client.18135 trim_cache size 0 max 16384 2024-05-10T11:31:47.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.578+0000 7f2b2cd3bb80 1 Processor -- start 2024-05-10T11:31:47.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.578+0000 7f2b2cd3bb80 1 -- start start 2024-05-10T11:31:47.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.578+0000 7f2b2cd3bb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf84c9ddf0 0x55bf84cb2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:47.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.578+0000 7f2b2cd3bb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bf84ca1c80 0x55bf84c925a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:47.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.578+0000 7f2b2cd3bb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bf84cbea70 0x55bf84cbeef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:47.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.578+0000 7f2b2cd3bb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1c1b3b90 con 0x55bf84c9ddf0 2024-05-10T11:31:47.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.578+0000 7f2b2cd3bb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1c1b1420 con 0x55bf84cbea70 2024-05-10T11:31:47.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.578+0000 7f2b2cd3bb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1c1af270 con 0x55bf84ca1c80 2024-05-10T11:31:47.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.578+0000 7f2b1bfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf84c9ddf0 0x55bf84cb2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:47.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.578+0000 7f2b1b7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bf84ca1c80 0x55bf84c925a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:47.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.578+0000 7f2b1b7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bf84ca1c80 0x55bf84c925a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:57490/0 (socket says 172.21.15.100:57490) 2024-05-10T11:31:47.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.578+0000 7f2b1b7fe700 1 -- 172.21.15.100:0/3857734724 learned_addr learned my addr 172.21.15.100:0/3857734724 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:31:47.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.578+0000 7f2b20a45700 1 --2- 172.21.15.100:0/3857734724 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bf84cbea70 0x55bf84cbeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:47.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.579+0000 7f2b1b7fe700 1 -- 172.21.15.100:0/3857734724 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bf84cbea70 msgr2=0x55bf84cbeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:47.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.579+0000 7f2b1b7fe700 1 --2- 172.21.15.100:0/3857734724 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bf84cbea70 0x55bf84cbeef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:47.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.579+0000 7f2b1b7fe700 1 -- 172.21.15.100:0/3857734724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf84c9ddf0 msgr2=0x55bf84cb2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:47.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.579+0000 7f2b1b7fe700 1 --2- 172.21.15.100:0/3857734724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf84c9ddf0 0x55bf84cb2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:47.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.579+0000 7f2b1b7fe700 1 -- 172.21.15.100:0/3857734724 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf84c99520 con 0x55bf84ca1c80 2024-05-10T11:31:47.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.579+0000 7f2b20a45700 1 --2- 172.21.15.100:0/3857734724 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bf84cbea70 0x55bf84cbeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:31:47.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.579+0000 7f2b1bfff700 1 --2- 172.21.15.100:0/3857734724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf84c9ddf0 0x55bf84cb2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:31:47.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.579+0000 7f2b1b7fe700 1 --2- 172.21.15.100:0/3857734724 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bf84ca1c80 0x55bf84c925a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2b10003760 tx=0x7f2b1001a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ee942d34ca0f397 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:31:47.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.579+0000 7f2b01ffb700 1 -- 172.21.15.100:0/3857734724 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b10005670 con 0x55bf84ca1c80 2024-05-10T11:31:47.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.579+0000 7f2b01ffb700 1 -- 172.21.15.100:0/3857734724 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2b1002a070 con 0x55bf84ca1c80 2024-05-10T11:31:47.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.579+0000 7f2b01ffb700 1 -- 172.21.15.100:0/3857734724 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1000bb60 con 0x55bf84ca1c80 2024-05-10T11:31:47.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.580+0000 7f2b2cd3bb80 1 -- 172.21.15.100:0/3857734724 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bf84ca1c80 msgr2=0x55bf84c925a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:47.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.580+0000 7f2b2cd3bb80 1 --2- 172.21.15.100:0/3857734724 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bf84ca1c80 0x55bf84c925a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2b10003760 tx=0x7f2b1001a040 comp rx=0 tx=0).stop 2024-05-10T11:31:47.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.580+0000 7f2b2cd3bb80 1 -- 172.21.15.100:0/3857734724 shutdown_connections 2024-05-10T11:31:47.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.580+0000 7f2b2cd3bb80 1 --2- 172.21.15.100:0/3857734724 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bf84cbea70 0x55bf84cbeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:47.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.580+0000 7f2b2cd3bb80 1 --2- 172.21.15.100:0/3857734724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf84c9ddf0 0x55bf84cb2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:47.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.580+0000 7f2b2cd3bb80 1 --2- 172.21.15.100:0/3857734724 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bf84ca1c80 0x55bf84c925a0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:47.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.580+0000 7f2b2cd3bb80 1 -- 172.21.15.100:0/3857734724 >> 172.21.15.100:0/3857734724 conn(0x55bf84bc9100 msgr2=0x55bf84c91e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:31:47.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.580+0000 7f2b2cd3bb80 1 -- 172.21.15.100:0/3857734724 shutdown_connections 2024-05-10T11:31:47.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.580+0000 7f2b2cd3bb80 1 -- 172.21.15.100:0/3857734724 wait complete. 2024-05-10T11:31:47.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.580+0000 7f2b2cd3bb80 1 -- start start 2024-05-10T11:31:47.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.581+0000 7f2b2cd3bb80 1 Processor -- start 2024-05-10T11:31:47.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.581+0000 7f2b2cd3bb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bf84c9ddf0 0x55bf84caf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:47.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.581+0000 7f2b2cd3bb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf84ca1c80 0x55bf84caabe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:47.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.581+0000 7f2b2cd3bb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bf84cbea70 0x55bf84cab120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:47.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.581+0000 7f2b2cd3bb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1c1af8e0 con 0x55bf84ca1c80 2024-05-10T11:31:47.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.581+0000 7f2b2cd3bb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1c1b0d50 con 0x55bf84c9ddf0 2024-05-10T11:31:47.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.581+0000 7f2b2cd3bb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1c1affb0 con 0x55bf84cbea70 2024-05-10T11:31:47.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.581+0000 7f2b1b7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf84ca1c80 0x55bf84caabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:47.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.581+0000 7f2b1b7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf84ca1c80 0x55bf84caabe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43472/0 (socket says 172.21.15.100:43472) 2024-05-10T11:31:47.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.581+0000 7f2b1b7fe700 1 -- 172.21.15.100:0/1202111832 learned_addr learned my addr 172.21.15.100:0/1202111832 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:31:47.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.581+0000 7f2b1bfff700 1 --2- 172.21.15.100:0/1202111832 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bf84c9ddf0 0x55bf84caf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:47.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.581+0000 7f2b20a45700 1 --2- 172.21.15.100:0/1202111832 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bf84cbea70 0x55bf84cab120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:47.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.581+0000 7f2b2cd3bb80 10 client.0 _ll_register_callbacks cb 0x55bf84b87920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:31:47.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.582+0000 7f2b1b7fe700 1 -- 172.21.15.100:0/1202111832 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bf84cbea70 msgr2=0x55bf84cab120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:47.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.582+0000 7f2b1b7fe700 1 --2- 172.21.15.100:0/1202111832 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bf84cbea70 0x55bf84cab120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:47.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.582+0000 7f2b1b7fe700 1 -- 172.21.15.100:0/1202111832 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bf84c9ddf0 msgr2=0x55bf84caf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:47.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.582+0000 7f2b1b7fe700 1 --2- 172.21.15.100:0/1202111832 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bf84c9ddf0 0x55bf84caf810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:47.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.582+0000 7f2b1b7fe700 1 -- 172.21.15.100:0/1202111832 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf84cab960 con 0x55bf84ca1c80 2024-05-10T11:31:47.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.582+0000 7f2b20a45700 1 --2- 172.21.15.100:0/1202111832 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bf84cbea70 0x55bf84cab120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:31:47.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.582+0000 7f2b1b7fe700 1 --2- 172.21.15.100:0/1202111832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf84ca1c80 0x55bf84caabe0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f2b1000b580 tx=0x7f2b100095d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2133def62416fa77 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:31:47.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.582+0000 7f2afb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:31:47.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.582+0000 7f2afb7fe700 1 -- 172.21.15.100:0/1202111832 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b100039c0 con 0x55bf84ca1c80 2024-05-10T11:31:47.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.582+0000 7f2b2cd3bb80 1 -- 172.21.15.100:0/1202111832 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b1002a070 con 0x55bf84ca1c80 2024-05-10T11:31:47.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.582+0000 7f2afb7fe700 1 -- 172.21.15.100:0/1202111832 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f2b10003b60 con 0x55bf84ca1c80 2024-05-10T11:31:47.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.582+0000 7f2afb7fe700 1 -- 172.21.15.100:0/1202111832 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f2b10003d60 con 0x55bf84ca1c80 2024-05-10T11:31:47.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.582+0000 7f2b2cd3bb80 10 client.18141 fetch_fsmap learned FSMap version 33 2024-05-10T11:31:47.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.582+0000 7f2b2cd3bb80 1 -- 172.21.15.100:0/1202111832 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55bf84d21670 con 0x55bf84ca1c80 2024-05-10T11:31:47.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.582+0000 7f2afb7fe700 1 -- 172.21.15.100:0/1202111832 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2b1002a070 con 0x55bf84ca1c80 2024-05-10T11:31:47.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.583+0000 7f2afb7fe700 1 -- 172.21.15.100:0/1202111832 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f2b10032020 con 0x55bf84ca1c80 2024-05-10T11:31:47.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.583+0000 7f2b2cd3bb80 10 client.18141 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T11:31:47.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.583+0000 7f2b2cd3bb80 10 client.18141 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:31:47.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.583+0000 7f2b2cd3bb80 20 client.18141 populate_metadata read hostname 'smithi100' 2024-05-10T11:31:47.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.583+0000 7f2b2cd3bb80 1 --2- 172.21.15.100:0/1202111832 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55bf84d21c10 0x55bf84d24030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:47.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.583+0000 7f2b2cd3bb80 4 client.18141 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:31:47.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.583+0000 7f2b2cd3bb80 1 -- 172.21.15.100:0/1202111832 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bf84c69650 con 0x55bf84d21c10 2024-05-10T11:31:47.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.583+0000 7f2b1bfff700 1 --2- 172.21.15.100:0/1202111832 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55bf84d21c10 0x55bf84d24030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:47.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.583+0000 7f2afb7fe700 10 client.18141 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:31:47.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.583+0000 7f2b1bfff700 1 --2- 172.21.15.100:0/1202111832 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55bf84d21c10 0x55bf84d24030 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d8ac8c2eedb2781 server_cookie=9719704131ce5df8 in_seq=0 out_seq=0 2024-05-10T11:31:47.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.584+0000 7f2afb7fe700 1 -- 172.21.15.100:0/1202111832 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bf84c69650 con 0x55bf84d21c10 2024-05-10T11:31:47.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.584+0000 7f2afb7fe700 10 client.18141 handle_command_reply: tid=0 2024-05-10T11:31:47.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.584+0000 7f2b2cd3bb80 1 client.18141 shutdown 2024-05-10T11:31:47.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.584+0000 7f2b2cd3bb80 1 -- 172.21.15.100:0/1202111832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf84ca1c80 msgr2=0x55bf84caabe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:47.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.584+0000 7f2b2cd3bb80 1 --2- 172.21.15.100:0/1202111832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf84ca1c80 0x55bf84caabe0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f2b1000b580 tx=0x7f2b100095d0 comp rx=0 tx=0).stop 2024-05-10T11:31:47.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.584+0000 7f2b2cd3bb80 1 -- 172.21.15.100:0/1202111832 shutdown_connections 2024-05-10T11:31:47.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.584+0000 7f2b2cd3bb80 1 --2- 172.21.15.100:0/1202111832 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55bf84d21c10 0x55bf84d24030 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:47.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.584+0000 7f2b2cd3bb80 1 --2- 172.21.15.100:0/1202111832 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bf84cbea70 0x55bf84cab120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:47.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.584+0000 7f2b2cd3bb80 1 --2- 172.21.15.100:0/1202111832 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bf84c9ddf0 0x55bf84caf810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:47.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.584+0000 7f2b2cd3bb80 1 --2- 172.21.15.100:0/1202111832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bf84ca1c80 0x55bf84caabe0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:47.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.584+0000 7f2afb7fe700 0 client.18141 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:31:47.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.584+0000 7f2b2cd3bb80 1 -- 172.21.15.100:0/1202111832 >> 172.21.15.100:0/1202111832 conn(0x55bf84bc9100 msgr2=0x55bf84c92720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:31:47.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.584+0000 7f2b2cd3bb80 1 -- 172.21.15.100:0/1202111832 shutdown_connections 2024-05-10T11:31:47.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.584+0000 7f2b2cd3bb80 1 -- 172.21.15.100:0/1202111832 wait complete. 2024-05-10T11:31:47.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.584+0000 7f2b2cd3bb80 20 client.18141 trim_cache size 0 max 16384 2024-05-10T11:31:47.592 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:31:47.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.584+0000 7f2b224a8700 1 -- 172.21.15.100:0/546256809 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2b040b2880 msgr2=0x7f2b040b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:47.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.584+0000 7f2b224a8700 1 --2- 172.21.15.100:0/546256809 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2b040b2880 0x7f2b040b4d40 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c005950 tx=0x7f2b0c00b410 comp rx=0 tx=0).stop 2024-05-10T11:31:47.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.584+0000 7f2b224a8700 1 -- 172.21.15.100:0/546256809 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b1c107be0 msgr2=0x7f2b1c199080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:47.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.584+0000 7f2b224a8700 1 --2- 172.21.15.100:0/546256809 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b1c107be0 0x7f2b1c199080 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f2b1400ef30 tx=0x7f2b1400c3a0 comp rx=0 tx=0).stop 2024-05-10T11:31:47.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.585+0000 7f2b224a8700 1 -- 172.21.15.100:0/546256809 shutdown_connections 2024-05-10T11:31:47.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.585+0000 7f2b224a8700 1 --2- 172.21.15.100:0/546256809 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2b040b2880 0x7f2b040b4d40 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:47.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.585+0000 7f2b224a8700 1 --2- 172.21.15.100:0/546256809 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b1c107be0 0x7f2b1c199080 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:47.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.585+0000 7f2b224a8700 1 --2- 172.21.15.100:0/546256809 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b1c101b80 0x7f2b1c198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:47.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.585+0000 7f2b224a8700 1 --2- 172.21.15.100:0/546256809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b1c102540 0x7f2b1c198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:47.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.585+0000 7f2b224a8700 1 -- 172.21.15.100:0/546256809 >> 172.21.15.100:0/546256809 conn(0x7f2b1c078a10 msgr2=0x7f2b1c1000f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:31:47.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.585+0000 7f2b224a8700 1 -- 172.21.15.100:0/546256809 shutdown_connections 2024-05-10T11:31:47.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:47.585+0000 7f2b224a8700 1 -- 172.21.15.100:0/546256809 wait complete. 2024-05-10T11:31:47.985 INFO:tasks.workunit.client.0.smithi100.stdout: 1 355522 2.70 MB/sec execute 582 sec latency 45.392 ms 2024-05-10T11:31:48.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:31:47 smithi178 ceph-mon[33359]: pgmap v823: 129 pgs: 129 active+clean; 139 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 1.3 MiB/s wr, 152 op/s 2024-05-10T11:31:48.007 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:31:47 smithi100 ceph-mon[30296]: pgmap v823: 129 pgs: 129 active+clean; 139 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 1.3 MiB/s wr, 152 op/s 2024-05-10T11:31:48.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:31:47 smithi188 ceph-mon[34075]: pgmap v823: 129 pgs: 129 active+clean; 139 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 1.3 MiB/s wr, 152 op/s 2024-05-10T11:31:48.986 INFO:tasks.workunit.client.0.smithi100.stdout: 1 355992 2.69 MB/sec execute 583 sec latency 15.918 ms 2024-05-10T11:31:49.466 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T11:31:49.466 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T11:31:49.740 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:31:49.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:31:49 smithi100 ceph-mon[30296]: pgmap v824: 129 pgs: 129 active+clean; 135 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-05-10T11:31:49.986 INFO:tasks.workunit.client.0.smithi100.stdout: 1 356682 2.70 MB/sec execute 584 sec latency 20.759 ms 2024-05-10T11:31:50.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:31:49 smithi178 ceph-mon[33359]: pgmap v824: 129 pgs: 129 active+clean; 135 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-05-10T11:31:50.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:31:49 smithi188 ceph-mon[34075]: pgmap v824: 129 pgs: 129 active+clean; 135 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-05-10T11:31:50.986 INFO:tasks.workunit.client.0.smithi100.stdout: 1 357256 2.70 MB/sec execute 585 sec latency 16.279 ms 2024-05-10T11:31:51.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.440+0000 7f096d488700 1 -- 172.21.15.100:0/1703880915 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0968107730 msgr2=0x7f096810f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:51.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.440+0000 7f096d488700 1 --2- 172.21.15.100:0/1703880915 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0968107730 0x7f096810f2d0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f096000b0d0 tx=0x7f096000b3e0 comp rx=0 tx=0).stop 2024-05-10T11:31:51.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.440+0000 7f096d488700 1 -- 172.21.15.100:0/1703880915 shutdown_connections 2024-05-10T11:31:51.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.440+0000 7f096d488700 1 --2- 172.21.15.100:0/1703880915 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0968107730 0x7f096810f2d0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:51.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.440+0000 7f096d488700 1 --2- 172.21.15.100:0/1703880915 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09681029e0 0x7f0968102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:51.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.440+0000 7f096d488700 1 --2- 172.21.15.100:0/1703880915 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0968103390 0x7f0968107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:51.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.440+0000 7f096d488700 1 -- 172.21.15.100:0/1703880915 >> 172.21.15.100:0/1703880915 conn(0x7f0968079350 msgr2=0x7f0968079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:31:51.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.440+0000 7f096d488700 1 -- 172.21.15.100:0/1703880915 shutdown_connections 2024-05-10T11:31:51.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.440+0000 7f096d488700 1 -- 172.21.15.100:0/1703880915 wait complete. 2024-05-10T11:31:51.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.441+0000 7f096d488700 1 Processor -- start 2024-05-10T11:31:51.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.441+0000 7f096d488700 1 -- start start 2024-05-10T11:31:51.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.441+0000 7f096d488700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09681029e0 0x7f096819f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:51.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.441+0000 7f096d488700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0968103390 0x7f096819fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:51.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.441+0000 7f096d488700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0968107730 0x7f09681998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:51.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.441+0000 7f096d488700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0968112470 con 0x7f0968103390 2024-05-10T11:31:51.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.441+0000 7f096d488700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f09681122f0 con 0x7f0968107730 2024-05-10T11:31:51.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.441+0000 7f096d488700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f09681125f0 con 0x7f09681029e0 2024-05-10T11:31:51.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.441+0000 7f09667fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0968103390 0x7f096819fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:51.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.441+0000 7f09667fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0968103390 0x7f096819fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43490/0 (socket says 172.21.15.100:43490) 2024-05-10T11:31:51.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.441+0000 7f09667fc700 1 -- 172.21.15.100:0/3073267913 learned_addr learned my addr 172.21.15.100:0/3073267913 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:31:51.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.441+0000 7f0966ffd700 1 --2- 172.21.15.100:0/3073267913 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09681029e0 0x7f096819f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:51.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.442+0000 7f09677fe700 1 --2- 172.21.15.100:0/3073267913 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0968107730 0x7f09681998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:51.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.442+0000 7f09667fc700 1 -- 172.21.15.100:0/3073267913 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09681029e0 msgr2=0x7f096819f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:51.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.442+0000 7f09667fc700 1 --2- 172.21.15.100:0/3073267913 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09681029e0 0x7f096819f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:51.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.442+0000 7f09667fc700 1 -- 172.21.15.100:0/3073267913 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0968107730 msgr2=0x7f09681998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:51.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.442+0000 7f09667fc700 1 --2- 172.21.15.100:0/3073267913 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0968107730 0x7f09681998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:51.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.442+0000 7f09667fc700 1 -- 172.21.15.100:0/3073267913 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f096819a010 con 0x7f0968103390 2024-05-10T11:31:51.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.442+0000 7f09677fe700 1 --2- 172.21.15.100:0/3073267913 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0968107730 0x7f09681998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:31:51.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.442+0000 7f0966ffd700 1 --2- 172.21.15.100:0/3073267913 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09681029e0 0x7f096819f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:31:51.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.442+0000 7f09667fc700 1 --2- 172.21.15.100:0/3073267913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0968103390 0x7f096819fbe0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f095c00e940 tx=0x7f095c00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:31:51.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.442+0000 7f094ffff700 1 -- 172.21.15.100:0/3073267913 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095c00cbd0 con 0x7f0968103390 2024-05-10T11:31:51.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.442+0000 7f096d488700 1 -- 172.21.15.100:0/3073267913 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f096819a300 con 0x7f0968103390 2024-05-10T11:31:51.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.442+0000 7f094ffff700 1 -- 172.21.15.100:0/3073267913 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f095c00ee90 con 0x7f0968103390 2024-05-10T11:31:51.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.442+0000 7f094ffff700 1 -- 172.21.15.100:0/3073267913 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095c018680 con 0x7f0968103390 2024-05-10T11:31:51.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.442+0000 7f096d488700 1 -- 172.21.15.100:0/3073267913 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0968103fd0 con 0x7f0968103390 2024-05-10T11:31:51.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.445+0000 7f094ffff700 1 -- 172.21.15.100:0/3073267913 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f095c010430 con 0x7f0968103390 2024-05-10T11:31:51.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.446+0000 7f094ffff700 1 --2- 172.21.15.100:0/3073267913 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f09500b2880 0x7f09500b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:51.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.446+0000 7f0966ffd700 1 --2- 172.21.15.100:0/3073267913 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f09500b2880 0x7f09500b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:51.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.446+0000 7f094ffff700 1 -- 172.21.15.100:0/3073267913 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f095c0e2680 con 0x7f0968103390 2024-05-10T11:31:51.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.446+0000 7f0966ffd700 1 --2- 172.21.15.100:0/3073267913 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f09500b2880 0x7f09500b4d40 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f0968106c60 tx=0x7f0958012040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:31:51.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.447+0000 7f0977d1bb80 1 Processor -- start 2024-05-10T11:31:51.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.447+0000 7f0977d1bb80 1 -- start start 2024-05-10T11:31:51.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.447+0000 7f0977d1bb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fa05917230 0x55fa05917610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:51.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.447+0000 7f0977d1bb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa05917b50 0x55fa05911890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:51.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.447+0000 7f0977d1bb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa05911dd0 0x55fa05912210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:51.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.447+0000 7f0977d1bb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55fa058639c0 con 0x55fa05917b50 2024-05-10T11:31:51.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.447+0000 7f0977d1bb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55fa057c8e20 con 0x55fa05911dd0 2024-05-10T11:31:51.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.447+0000 7f0977d1bb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55fa0585f980 con 0x55fa05917230 2024-05-10T11:31:51.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.447+0000 7f09667fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa05917b50 0x55fa05911890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:51.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.447+0000 7f09667fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa05917b50 0x55fa05911890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43504/0 (socket says 172.21.15.100:43504) 2024-05-10T11:31:51.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.447+0000 7f0966ffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fa05917230 0x55fa05917610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:51.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.447+0000 7f09667fc700 1 -- 172.21.15.100:0/749781903 learned_addr learned my addr 172.21.15.100:0/749781903 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:31:51.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.447+0000 7f09677fe700 1 --2- 172.21.15.100:0/749781903 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa05911dd0 0x55fa05912210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:51.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.448+0000 7f09677fe700 1 -- 172.21.15.100:0/749781903 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fa05917230 msgr2=0x55fa05917610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:51.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.448+0000 7f09677fe700 1 --2- 172.21.15.100:0/749781903 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fa05917230 0x55fa05917610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:51.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.448+0000 7f09677fe700 1 -- 172.21.15.100:0/749781903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa05917b50 msgr2=0x55fa05911890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:51.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.448+0000 7f09677fe700 1 --2- 172.21.15.100:0/749781903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa05917b50 0x55fa05911890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:51.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.448+0000 7f09677fe700 1 -- 172.21.15.100:0/749781903 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa057b0950 con 0x55fa05911dd0 2024-05-10T11:31:51.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.448+0000 7f0966ffd700 1 --2- 172.21.15.100:0/749781903 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fa05917230 0x55fa05917610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:31:51.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.448+0000 7f09677fe700 1 --2- 172.21.15.100:0/749781903 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa05911dd0 0x55fa05912210 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f096000b770 tx=0x7f0960009d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=996c7408d8bc8e85 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:31:51.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.448+0000 7f094d7fa700 1 -- 172.21.15.100:0/749781903 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0960020030 con 0x55fa05911dd0 2024-05-10T11:31:51.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.448+0000 7f094d7fa700 1 -- 172.21.15.100:0/749781903 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0960009360 con 0x55fa05911dd0 2024-05-10T11:31:51.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.448+0000 7f094d7fa700 1 -- 172.21.15.100:0/749781903 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0960007940 con 0x55fa05911dd0 2024-05-10T11:31:51.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.449+0000 7f0977d1bb80 1 -- 172.21.15.100:0/749781903 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa05911dd0 msgr2=0x55fa05912210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:51.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.449+0000 7f0977d1bb80 1 --2- 172.21.15.100:0/749781903 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa05911dd0 0x55fa05912210 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f096000b770 tx=0x7f0960009d90 comp rx=0 tx=0).stop 2024-05-10T11:31:51.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.449+0000 7f0977d1bb80 1 -- 172.21.15.100:0/749781903 shutdown_connections 2024-05-10T11:31:51.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.449+0000 7f0977d1bb80 1 --2- 172.21.15.100:0/749781903 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa05911dd0 0x55fa05912210 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:51.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.449+0000 7f0977d1bb80 1 --2- 172.21.15.100:0/749781903 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fa05917230 0x55fa05917610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:51.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.449+0000 7f0977d1bb80 1 --2- 172.21.15.100:0/749781903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa05917b50 0x55fa05911890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:51.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.449+0000 7f0977d1bb80 1 -- 172.21.15.100:0/749781903 >> 172.21.15.100:0/749781903 conn(0x55fa0583dbf0 msgr2=0x55fa0590c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:31:51.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.449+0000 7f0977d1bb80 1 -- 172.21.15.100:0/749781903 shutdown_connections 2024-05-10T11:31:51.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.449+0000 7f0977d1bb80 1 -- 172.21.15.100:0/749781903 wait complete. 2024-05-10T11:31:51.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.449+0000 7f0977d1bb80 1 -- start start 2024-05-10T11:31:51.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.450+0000 7f0977d1bb80 1 Processor -- start 2024-05-10T11:31:51.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.450+0000 7f0977d1bb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa05911dd0 0x55fa0591d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:51.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.450+0000 7f0977d1bb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa05917230 0x55fa0591aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:51.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.450+0000 7f0977d1bb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fa05917b50 0x55fa0591b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:51.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.450+0000 7f0977d1bb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55fa0591e470 con 0x55fa05917230 2024-05-10T11:31:51.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.450+0000 7f0977d1bb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55fa0581cf50 con 0x55fa05911dd0 2024-05-10T11:31:51.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.450+0000 7f0977d1bb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55fa0591e5f0 con 0x55fa05917b50 2024-05-10T11:31:51.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.450+0000 7f0977d1bb80 10 client.0 _ll_register_callbacks cb 0x55fa057f7630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:31:51.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.450+0000 7f0966ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa05911dd0 0x55fa0591d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:51.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.450+0000 7f0966ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa05911dd0 0x55fa0591d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:56940/0 (socket says 172.21.15.100:56940) 2024-05-10T11:31:51.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.450+0000 7f0966ffd700 1 -- 172.21.15.100:0/3370370291 learned_addr learned my addr 172.21.15.100:0/3370370291 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:31:51.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.450+0000 7f09677fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fa05917b50 0x55fa0591b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:51.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.450+0000 7f09667fc700 1 --2- 172.21.15.100:0/3370370291 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa05917230 0x55fa0591aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:51.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.451+0000 7f0966ffd700 1 -- 172.21.15.100:0/3370370291 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fa05917b50 msgr2=0x55fa0591b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:51.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.451+0000 7f0966ffd700 1 --2- 172.21.15.100:0/3370370291 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fa05917b50 0x55fa0591b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:51.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.451+0000 7f0966ffd700 1 -- 172.21.15.100:0/3370370291 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa05917230 msgr2=0x55fa0591aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:51.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.451+0000 7f0966ffd700 1 --2- 172.21.15.100:0/3370370291 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa05917230 0x55fa0591aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:51.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.451+0000 7f0966ffd700 1 -- 172.21.15.100:0/3370370291 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa0581a7e0 con 0x55fa05911dd0 2024-05-10T11:31:51.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.451+0000 7f09667fc700 1 --2- 172.21.15.100:0/3370370291 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa05917230 0x55fa0591aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:31:51.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.451+0000 7f0966ffd700 1 --2- 172.21.15.100:0/3370370291 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa05911dd0 0x55fa0591d6b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0958004060 tx=0x7f09580047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6303860aec9dcc57 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:31:51.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.451+0000 7f094aff5700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:31:51.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.451+0000 7f094aff5700 1 -- 172.21.15.100:0/3370370291 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095801eb00 con 0x55fa05911dd0 2024-05-10T11:31:51.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.451+0000 7f094aff5700 1 -- 172.21.15.100:0/3370370291 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f095801eca0 con 0x55fa05911dd0 2024-05-10T11:31:51.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.452+0000 7f0977d1bb80 1 -- 172.21.15.100:0/3370370291 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0960009360 con 0x55fa05911dd0 2024-05-10T11:31:51.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.452+0000 7f094aff5700 1 -- 172.21.15.100:0/3370370291 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f0958028400 con 0x55fa05911dd0 2024-05-10T11:31:51.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.452+0000 7f0977d1bb80 10 client.27497 fetch_fsmap learned FSMap version 33 2024-05-10T11:31:51.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.452+0000 7f0977d1bb80 1 -- 172.21.15.100:0/3370370291 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55fa05928c90 con 0x55fa05911dd0 2024-05-10T11:31:51.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.452+0000 7f094aff5700 1 -- 172.21.15.100:0/3370370291 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0960009360 con 0x55fa05911dd0 2024-05-10T11:31:51.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.452+0000 7f094aff5700 1 -- 172.21.15.100:0/3370370291 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f09580313f0 con 0x55fa05911dd0 2024-05-10T11:31:51.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.452+0000 7f0977d1bb80 10 client.27497 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T11:31:51.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.452+0000 7f0977d1bb80 10 client.27497 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:31:51.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.452+0000 7f0977d1bb80 20 client.27497 populate_metadata read hostname 'smithi100' 2024-05-10T11:31:51.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.452+0000 7f0977d1bb80 1 --2- 172.21.15.100:0/3370370291 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55fa0592b780 0x55fa0592dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:51.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.452+0000 7f0977d1bb80 4 client.27497 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:31:51.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.452+0000 7f0977d1bb80 1 -- 172.21.15.100:0/3370370291 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa058713a0 con 0x55fa0592b780 2024-05-10T11:31:51.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.453+0000 7f09667fc700 1 --2- 172.21.15.100:0/3370370291 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55fa0592b780 0x55fa0592dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:51.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.453+0000 7f094aff5700 10 client.27497 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:31:51.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.453+0000 7f09667fc700 1 --2- 172.21.15.100:0/3370370291 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55fa0592b780 0x55fa0592dc00 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b50a33f30d8b2422 server_cookie=be18a0b58d617aa in_seq=0 out_seq=0 2024-05-10T11:31:51.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.454+0000 7f094aff5700 1 -- 172.21.15.100:0/3370370291 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fa058713a0 con 0x55fa0592b780 2024-05-10T11:31:51.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.454+0000 7f094aff5700 10 client.27497 handle_command_reply: tid=0 2024-05-10T11:31:51.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.454+0000 7f0977d1bb80 1 client.27497 shutdown 2024-05-10T11:31:51.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.454+0000 7f0977d1bb80 1 -- 172.21.15.100:0/3370370291 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa05911dd0 msgr2=0x55fa0591d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:51.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.454+0000 7f0977d1bb80 1 --2- 172.21.15.100:0/3370370291 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa05911dd0 0x55fa0591d6b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0958004060 tx=0x7f09580047c0 comp rx=0 tx=0).stop 2024-05-10T11:31:51.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.454+0000 7f0977d1bb80 1 -- 172.21.15.100:0/3370370291 shutdown_connections 2024-05-10T11:31:51.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.454+0000 7f0977d1bb80 1 --2- 172.21.15.100:0/3370370291 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55fa0592b780 0x55fa0592dc00 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:51.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.454+0000 7f0977d1bb80 1 --2- 172.21.15.100:0/3370370291 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fa05917b50 0x55fa0591b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:51.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.454+0000 7f0977d1bb80 1 --2- 172.21.15.100:0/3370370291 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa05911dd0 0x55fa0591d6b0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:51.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.454+0000 7f0977d1bb80 1 --2- 172.21.15.100:0/3370370291 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa05917230 0x55fa0591aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:51.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.454+0000 7f094aff5700 0 client.27497 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:31:51.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.454+0000 7f0977d1bb80 1 -- 172.21.15.100:0/3370370291 >> 172.21.15.100:0/3370370291 conn(0x55fa0583dbf0 msgr2=0x55fa05906bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:31:51.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.454+0000 7f0977d1bb80 1 -- 172.21.15.100:0/3370370291 shutdown_connections 2024-05-10T11:31:51.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.455+0000 7f0977d1bb80 1 -- 172.21.15.100:0/3370370291 wait complete. 2024-05-10T11:31:51.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.455+0000 7f0977d1bb80 20 client.27497 trim_cache size 0 max 16384 2024-05-10T11:31:51.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.478+0000 7f0977d1bb80 1 Processor -- start 2024-05-10T11:31:51.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.478+0000 7f0977d1bb80 1 -- start start 2024-05-10T11:31:51.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.478+0000 7f0977d1bb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa05911dd0 0x55fa05911890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:51.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.478+0000 7f0977d1bb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa05917230 0x55fa0591cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:51.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.478+0000 7f0977d1bb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fa05917b50 0x55fa05927500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:51.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.478+0000 7f0977d1bb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f09540112b0 con 0x55fa05917230 2024-05-10T11:31:51.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.478+0000 7f0977d1bb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f095400eb40 con 0x55fa05911dd0 2024-05-10T11:31:51.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.478+0000 7f0977d1bb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f095400c990 con 0x55fa05917b50 2024-05-10T11:31:51.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.478+0000 7f09667fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa05917230 0x55fa0591cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:51.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.478+0000 7f09667fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa05917230 0x55fa0591cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43518/0 (socket says 172.21.15.100:43518) 2024-05-10T11:31:51.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.478+0000 7f09667fc700 1 -- 172.21.15.100:0/3345645838 learned_addr learned my addr 172.21.15.100:0/3345645838 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:31:51.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.478+0000 7f09677fe700 1 --2- 172.21.15.100:0/3345645838 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fa05917b50 0x55fa05927500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:51.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.479+0000 7f0966ffd700 1 --2- 172.21.15.100:0/3345645838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa05911dd0 0x55fa05911890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:51.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.479+0000 7f09677fe700 1 -- 172.21.15.100:0/3345645838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa05911dd0 msgr2=0x55fa05911890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:51.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.479+0000 7f09677fe700 1 --2- 172.21.15.100:0/3345645838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa05911dd0 0x55fa05911890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:51.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.479+0000 7f09677fe700 1 -- 172.21.15.100:0/3345645838 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa05917230 msgr2=0x55fa0591cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:51.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.479+0000 7f09677fe700 1 --2- 172.21.15.100:0/3345645838 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa05917230 0x55fa0591cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:51.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.479+0000 7f09677fe700 1 -- 172.21.15.100:0/3345645838 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa05917950 con 0x55fa05917b50 2024-05-10T11:31:51.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.479+0000 7f09667fc700 1 --2- 172.21.15.100:0/3345645838 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa05917230 0x55fa0591cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:31:51.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.479+0000 7f09677fe700 1 --2- 172.21.15.100:0/3345645838 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fa05917b50 0x55fa05927500 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f096000b420 tx=0x7f09600061a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55b5ed615f10e133 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:31:51.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.479+0000 7f094cff9700 1 -- 172.21.15.100:0/3345645838 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0960020050 con 0x55fa05917b50 2024-05-10T11:31:51.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.479+0000 7f094cff9700 1 -- 172.21.15.100:0/3345645838 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0960021040 con 0x55fa05917b50 2024-05-10T11:31:51.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.479+0000 7f094cff9700 1 -- 172.21.15.100:0/3345645838 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0960007a20 con 0x55fa05917b50 2024-05-10T11:31:51.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.480+0000 7f0977d1bb80 1 -- 172.21.15.100:0/3345645838 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fa05917b50 msgr2=0x55fa05927500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:51.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.480+0000 7f0977d1bb80 1 --2- 172.21.15.100:0/3345645838 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fa05917b50 0x55fa05927500 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f096000b420 tx=0x7f09600061a0 comp rx=0 tx=0).stop 2024-05-10T11:31:51.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.480+0000 7f0977d1bb80 1 -- 172.21.15.100:0/3345645838 shutdown_connections 2024-05-10T11:31:51.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.480+0000 7f0977d1bb80 1 --2- 172.21.15.100:0/3345645838 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fa05917b50 0x55fa05927500 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:51.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.480+0000 7f0977d1bb80 1 --2- 172.21.15.100:0/3345645838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa05911dd0 0x55fa05911890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:51.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.480+0000 7f0977d1bb80 1 --2- 172.21.15.100:0/3345645838 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa05917230 0x55fa0591cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:51.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.480+0000 7f0977d1bb80 1 -- 172.21.15.100:0/3345645838 >> 172.21.15.100:0/3345645838 conn(0x55fa0583dbf0 msgr2=0x55fa05916410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:31:51.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.481+0000 7f0977d1bb80 1 -- 172.21.15.100:0/3345645838 shutdown_connections 2024-05-10T11:31:51.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.481+0000 7f0977d1bb80 1 -- 172.21.15.100:0/3345645838 wait complete. 2024-05-10T11:31:51.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.481+0000 7f0977d1bb80 1 -- start start 2024-05-10T11:31:51.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.481+0000 7f0977d1bb80 1 Processor -- start 2024-05-10T11:31:51.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.481+0000 7f0977d1bb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa05911dd0 0x55fa05923880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:51.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.481+0000 7f0977d1bb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fa05917230 0x55fa05923dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:51.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.481+0000 7f0977d1bb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa05917b50 0x55fa05920850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:51.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.481+0000 7f0977d1bb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f095400d000 con 0x55fa05911dd0 2024-05-10T11:31:51.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.481+0000 7f0977d1bb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f095400e470 con 0x55fa05917b50 2024-05-10T11:31:51.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.481+0000 7f0977d1bb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f095400d6d0 con 0x55fa05917230 2024-05-10T11:31:51.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.481+0000 7f0977d1bb80 10 client.0 _ll_register_callbacks cb 0x55fa057f7630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:31:51.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.481+0000 7f0966ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa05911dd0 0x55fa05923880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:51.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.481+0000 7f09667fc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fa05917230 0x55fa05923dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:51.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.481+0000 7f09667fc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fa05917230 0x55fa05923dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:57532/0 (socket says 172.21.15.100:57532) 2024-05-10T11:31:51.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.481+0000 7f09667fc700 1 -- 172.21.15.100:0/2229908604 learned_addr learned my addr 172.21.15.100:0/2229908604 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:31:51.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.481+0000 7f0966ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa05911dd0 0x55fa05923880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43532/0 (socket says 172.21.15.100:43532) 2024-05-10T11:31:51.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.482+0000 7f09677fe700 1 --2- 172.21.15.100:0/2229908604 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa05917b50 0x55fa05920850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:51.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.482+0000 7f0966ffd700 1 -- 172.21.15.100:0/2229908604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fa05917230 msgr2=0x55fa05923dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:51.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.482+0000 7f0966ffd700 1 --2- 172.21.15.100:0/2229908604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fa05917230 0x55fa05923dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:51.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.482+0000 7f0966ffd700 1 -- 172.21.15.100:0/2229908604 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa05917b50 msgr2=0x55fa05920850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:51.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.482+0000 7f0966ffd700 1 --2- 172.21.15.100:0/2229908604 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa05917b50 0x55fa05920850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:51.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.482+0000 7f0966ffd700 1 -- 172.21.15.100:0/2229908604 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa05920d90 con 0x55fa05911dd0 2024-05-10T11:31:51.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.482+0000 7f09667fc700 1 --2- 172.21.15.100:0/2229908604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fa05917230 0x55fa05923dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:31:51.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.482+0000 7f09677fe700 1 --2- 172.21.15.100:0/2229908604 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa05917b50 0x55fa05920850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:31:51.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.482+0000 7f0966ffd700 1 --2- 172.21.15.100:0/2229908604 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa05911dd0 0x55fa05923880 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f0958031860 tx=0x7f095801e300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef21f3b07f8789f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:31:51.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.482+0000 7f094aff5700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:31:51.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.482+0000 7f094aff5700 1 -- 172.21.15.100:0/2229908604 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0958004390 con 0x55fa05911dd0 2024-05-10T11:31:51.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.482+0000 7f094aff5700 1 -- 172.21.15.100:0/2229908604 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f0958031c60 con 0x55fa05911dd0 2024-05-10T11:31:51.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.482+0000 7f0977d1bb80 1 -- 172.21.15.100:0/2229908604 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0960021040 con 0x55fa05911dd0 2024-05-10T11:31:51.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.482+0000 7f094aff5700 1 -- 172.21.15.100:0/2229908604 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f0958049050 con 0x55fa05911dd0 2024-05-10T11:31:51.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.482+0000 7f0977d1bb80 10 client.18159 fetch_fsmap learned FSMap version 33 2024-05-10T11:31:51.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.482+0000 7f0977d1bb80 1 -- 172.21.15.100:0/2229908604 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55fa059968b0 con 0x55fa05911dd0 2024-05-10T11:31:51.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.483+0000 7f094aff5700 1 -- 172.21.15.100:0/2229908604 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0960021040 con 0x55fa05911dd0 2024-05-10T11:31:51.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.483+0000 7f094aff5700 1 -- 172.21.15.100:0/2229908604 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f09580287a0 con 0x55fa05911dd0 2024-05-10T11:31:51.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.483+0000 7f0977d1bb80 10 client.18159 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T11:31:51.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.483+0000 7f0977d1bb80 10 client.18159 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:31:51.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.483+0000 7f0977d1bb80 20 client.18159 populate_metadata read hostname 'smithi100' 2024-05-10T11:31:51.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.483+0000 7f0977d1bb80 1 --2- 172.21.15.100:0/2229908604 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55fa05996fb0 0x55fa059993d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:51.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.483+0000 7f0977d1bb80 4 client.18159 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T11:31:51.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.483+0000 7f0977d1bb80 1 -- 172.21.15.100:0/2229908604 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55fa0586b440 con 0x55fa05996fb0 2024-05-10T11:31:51.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.483+0000 7f09667fc700 1 --2- 172.21.15.100:0/2229908604 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55fa05996fb0 0x55fa059993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:51.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.484+0000 7f094aff5700 10 client.18159 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:31:51.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.484+0000 7f09667fc700 1 --2- 172.21.15.100:0/2229908604 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55fa05996fb0 0x55fa059993d0 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8149a0198c37a408 server_cookie=d27c482cf0df4c67 in_seq=0 out_seq=0 2024-05-10T11:31:51.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.485+0000 7f094aff5700 1 -- 172.21.15.100:0/2229908604 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55fa0586b440 con 0x55fa05996fb0 2024-05-10T11:31:51.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.485+0000 7f094aff5700 10 client.18159 handle_command_reply: tid=0 2024-05-10T11:31:51.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.485+0000 7f0977d1bb80 1 client.18159 shutdown 2024-05-10T11:31:51.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.485+0000 7f0977d1bb80 1 -- 172.21.15.100:0/2229908604 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa05911dd0 msgr2=0x55fa05923880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:51.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.485+0000 7f0977d1bb80 1 --2- 172.21.15.100:0/2229908604 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa05911dd0 0x55fa05923880 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f0958031860 tx=0x7f095801e300 comp rx=0 tx=0).stop 2024-05-10T11:31:51.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.485+0000 7f0977d1bb80 1 -- 172.21.15.100:0/2229908604 shutdown_connections 2024-05-10T11:31:51.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.485+0000 7f0977d1bb80 1 --2- 172.21.15.100:0/2229908604 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55fa05996fb0 0x55fa059993d0 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:51.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.485+0000 7f0977d1bb80 1 --2- 172.21.15.100:0/2229908604 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa05917b50 0x55fa05920850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:51.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.485+0000 7f0977d1bb80 1 --2- 172.21.15.100:0/2229908604 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa05911dd0 0x55fa05923880 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:51.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.485+0000 7f094aff5700 0 client.18159 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:31:51.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.485+0000 7f0977d1bb80 1 --2- 172.21.15.100:0/2229908604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fa05917230 0x55fa05923dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:51.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.485+0000 7f0977d1bb80 1 -- 172.21.15.100:0/2229908604 >> 172.21.15.100:0/2229908604 conn(0x55fa0583dbf0 msgr2=0x55fa05910880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:31:51.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.485+0000 7f0977d1bb80 1 -- 172.21.15.100:0/2229908604 shutdown_connections 2024-05-10T11:31:51.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.485+0000 7f0977d1bb80 1 -- 172.21.15.100:0/2229908604 wait complete. 2024-05-10T11:31:51.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.485+0000 7f0977d1bb80 20 client.18159 trim_cache size 0 max 16384 2024-05-10T11:31:51.489 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:31:51.489 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-05-10T11:31:51.489 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "cd1721a1-cb81-4583-9312-f59658a36774", 2024-05-10T11:31:51.489 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-05-10T11:31:51.489 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:31:51.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.486+0000 7f096d488700 1 -- 172.21.15.100:0/3073267913 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f09500b2880 msgr2=0x7f09500b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:51.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.486+0000 7f096d488700 1 --2- 172.21.15.100:0/3073267913 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f09500b2880 0x7f09500b4d40 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f0968106c60 tx=0x7f0958012040 comp rx=0 tx=0).stop 2024-05-10T11:31:51.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.486+0000 7f096d488700 1 -- 172.21.15.100:0/3073267913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0968103390 msgr2=0x7f096819fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:51.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.486+0000 7f096d488700 1 --2- 172.21.15.100:0/3073267913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0968103390 0x7f096819fbe0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f095c00e940 tx=0x7f095c00ec50 comp rx=0 tx=0).stop 2024-05-10T11:31:51.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.486+0000 7f096d488700 1 -- 172.21.15.100:0/3073267913 shutdown_connections 2024-05-10T11:31:51.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.486+0000 7f096d488700 1 --2- 172.21.15.100:0/3073267913 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f09500b2880 0x7f09500b4d40 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:51.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.486+0000 7f096d488700 1 --2- 172.21.15.100:0/3073267913 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0968107730 0x7f09681998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:51.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.486+0000 7f096d488700 1 --2- 172.21.15.100:0/3073267913 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09681029e0 0x7f096819f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:51.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.486+0000 7f096d488700 1 --2- 172.21.15.100:0/3073267913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0968103390 0x7f096819fbe0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:51.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.486+0000 7f096d488700 1 -- 172.21.15.100:0/3073267913 >> 172.21.15.100:0/3073267913 conn(0x7f0968079350 msgr2=0x7f096806bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:31:51.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.486+0000 7f096d488700 1 -- 172.21.15.100:0/3073267913 shutdown_connections 2024-05-10T11:31:51.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:51.486+0000 7f096d488700 1 -- 172.21.15.100:0/3073267913 wait complete. 2024-05-10T11:31:51.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:31:51 smithi100 ceph-mon[30296]: pgmap v825: 129 pgs: 129 active+clean; 135 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 1.2 MiB/s wr, 135 op/s 2024-05-10T11:31:51.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:31:51 smithi100 ceph-mon[30296]: scrub queued for path: / 2024-05-10T11:31:51.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:31:51 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:31:51.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:31:51 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:31:51.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:31:51 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:31:51.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:31:51 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:31:51.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:31:51 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:31:51.789 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:31:51 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:31:51.789 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:31:51 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:31:51.789 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:31:51 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:31:51.789 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:31:51 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:31:51.789 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:31:51 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:31:51.789 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:31:51 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:31:51.985 INFO:tasks.workunit.client.0.smithi100.stdout: 1 357985 2.70 MB/sec execute 586 sec latency 49.308 ms 2024-05-10T11:31:52.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:31:51 smithi178 ceph-mon[33359]: pgmap v825: 129 pgs: 129 active+clean; 135 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 1.2 MiB/s wr, 135 op/s 2024-05-10T11:31:52.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:31:51 smithi178 ceph-mon[33359]: scrub queued for path: / 2024-05-10T11:31:52.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:31:51 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:31:52.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:31:51 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:31:52.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:31:51 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:31:52.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:31:51 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:31:52.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:31:51 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:31:52.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:31:51 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:31:52.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:31:51 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:31:52.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:31:51 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:31:52.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:31:51 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:31:52.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:31:51 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:31:52.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:31:51 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:31:52.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:31:51 smithi188 ceph-mon[34075]: pgmap v825: 129 pgs: 129 active+clean; 135 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 1.2 MiB/s wr, 135 op/s 2024-05-10T11:31:52.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:31:51 smithi188 ceph-mon[34075]: scrub queued for path: / 2024-05-10T11:31:52.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:31:51 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:31:52.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:31:51 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:31:52.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:31:51 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:31:52.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:31:51 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:31:52.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:31:51 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:31:52.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:31:51 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:31:52.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:31:51 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:31:52.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:31:51 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:31:52.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:31:51 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:31:52.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:31:51 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:31:52.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:31:51 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:31:52.189 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T11:31:52.446 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:31:52.832 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:31:52 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:31:52.832 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:31:52 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:31:52.833 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:31:52 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:31:52.833 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:31:52 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:31:52.986 INFO:tasks.workunit.client.0.smithi100.stdout: 1 358451 2.70 MB/sec execute 587 sec latency 10.216 ms 2024-05-10T11:31:53.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:31:52 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:31:53.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:31:52 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:31:53.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:31:52 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:31:53.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:31:52 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:31:53.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:31:52 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:31:53.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:31:52 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:31:53.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:31:52 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:31:53.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:31:52 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:31:53.986 INFO:tasks.workunit.client.0.smithi100.stdout: 1 358711 2.70 MB/sec execute 588 sec latency 13.085 ms 2024-05-10T11:31:54.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:31:53 smithi178 ceph-mon[33359]: pgmap v826: 129 pgs: 129 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 1.3 MiB/s wr, 152 op/s 2024-05-10T11:31:54.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:31:53 smithi188 ceph-mon[34075]: pgmap v826: 129 pgs: 129 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 1.3 MiB/s wr, 152 op/s 2024-05-10T11:31:54.031 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:31:53 smithi100 ceph-mon[30296]: pgmap v826: 129 pgs: 129 active+clean; 135 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 1.3 MiB/s wr, 152 op/s 2024-05-10T11:31:54.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.390+0000 7f8dd0911700 1 -- 172.21.15.100:0/516459678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8dcc1023a0 msgr2=0x7f8dcc102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:54.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.390+0000 7f8dd0911700 1 --2- 172.21.15.100:0/516459678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8dcc1023a0 0x7f8dcc102820 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f8dc0009990 tx=0x7f8dc0009ca0 comp rx=0 tx=0).stop 2024-05-10T11:31:54.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.390+0000 7f8dd0911700 1 -- 172.21.15.100:0/516459678 shutdown_connections 2024-05-10T11:31:54.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.390+0000 7f8dd0911700 1 --2- 172.21.15.100:0/516459678 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8dcc102d60 0x7f8dcc10f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:54.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.390+0000 7f8dd0911700 1 --2- 172.21.15.100:0/516459678 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8dcc108410 0x7f8dcc1087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:54.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.390+0000 7f8dd0911700 1 --2- 172.21.15.100:0/516459678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8dcc1023a0 0x7f8dcc102820 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:54.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.390+0000 7f8dd0911700 1 -- 172.21.15.100:0/516459678 >> 172.21.15.100:0/516459678 conn(0x7f8dcc079350 msgr2=0x7f8dcc079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:31:54.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.390+0000 7f8dd0911700 1 -- 172.21.15.100:0/516459678 shutdown_connections 2024-05-10T11:31:54.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.390+0000 7f8dd0911700 1 -- 172.21.15.100:0/516459678 wait complete. 2024-05-10T11:31:54.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.391+0000 7f8dd0911700 1 Processor -- start 2024-05-10T11:31:54.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.391+0000 7f8dd0911700 1 -- start start 2024-05-10T11:31:54.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.391+0000 7f8dd0911700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8dcc1023a0 0x7f8dcc198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:54.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.391+0000 7f8dd0911700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8dcc102d60 0x7f8dcc198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:54.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.391+0000 7f8dd0911700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8dcc108410 0x7f8dcc1991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:54.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.391+0000 7f8dd0911700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8dcc1123e0 con 0x7f8dcc102d60 2024-05-10T11:31:54.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.391+0000 7f8dd0911700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8dcc112260 con 0x7f8dcc1023a0 2024-05-10T11:31:54.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.391+0000 7f8dd0911700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8dcc112560 con 0x7f8dcc108410 2024-05-10T11:31:54.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.391+0000 7f8dc9d9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8dcc102d60 0x7f8dcc198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:54.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.391+0000 7f8dc9d9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8dcc102d60 0x7f8dcc198cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43556/0 (socket says 172.21.15.100:43556) 2024-05-10T11:31:54.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.392+0000 7f8dc9d9b700 1 -- 172.21.15.100:0/2275257622 learned_addr learned my addr 172.21.15.100:0/2275257622 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:31:54.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.392+0000 7f8dca59c700 1 --2- 172.21.15.100:0/2275257622 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8dcc1023a0 0x7f8dcc198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:54.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.392+0000 7f8dcad9d700 1 --2- 172.21.15.100:0/2275257622 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8dcc108410 0x7f8dcc1991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:54.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.392+0000 7f8dcad9d700 1 -- 172.21.15.100:0/2275257622 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8dcc1023a0 msgr2=0x7f8dcc198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:54.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.392+0000 7f8dcad9d700 1 --2- 172.21.15.100:0/2275257622 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8dcc1023a0 0x7f8dcc198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:54.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.392+0000 7f8dcad9d700 1 -- 172.21.15.100:0/2275257622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8dcc102d60 msgr2=0x7f8dcc198cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:54.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.392+0000 7f8dcad9d700 1 --2- 172.21.15.100:0/2275257622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8dcc102d60 0x7f8dcc198cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:54.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.392+0000 7f8dcad9d700 1 -- 172.21.15.100:0/2275257622 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dcc1a7fe0 con 0x7f8dcc108410 2024-05-10T11:31:54.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.392+0000 7f8dc9d9b700 1 --2- 172.21.15.100:0/2275257622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8dcc102d60 0x7f8dcc198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:31:54.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.392+0000 7f8dca59c700 1 --2- 172.21.15.100:0/2275257622 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8dcc1023a0 0x7f8dcc198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:31:54.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.392+0000 7f8dcad9d700 1 --2- 172.21.15.100:0/2275257622 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8dcc108410 0x7f8dcc1991f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8dc400efc0 tx=0x7f8dc400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:31:54.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.392+0000 7f8dbb7fe700 1 -- 172.21.15.100:0/2275257622 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc40092e0 con 0x7f8dcc108410 2024-05-10T11:31:54.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.392+0000 7f8dd0911700 1 -- 172.21.15.100:0/2275257622 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dcc1a8270 con 0x7f8dcc108410 2024-05-10T11:31:54.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.392+0000 7f8dbb7fe700 1 -- 172.21.15.100:0/2275257622 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8dc4004bb0 con 0x7f8dcc108410 2024-05-10T11:31:54.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.392+0000 7f8dbb7fe700 1 -- 172.21.15.100:0/2275257622 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc4004020 con 0x7f8dcc108410 2024-05-10T11:31:54.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.392+0000 7f8dd0911700 1 -- 172.21.15.100:0/2275257622 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dcc1a87b0 con 0x7f8dcc108410 2024-05-10T11:31:54.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.395+0000 7f8dbb7fe700 1 -- 172.21.15.100:0/2275257622 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f8dc401b030 con 0x7f8dcc108410 2024-05-10T11:31:54.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.396+0000 7f8dbb7fe700 1 --2- 172.21.15.100:0/2275257622 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8db40b2880 0x7f8db40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:54.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.396+0000 7f8dca59c700 1 --2- 172.21.15.100:0/2275257622 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8db40b2880 0x7f8db40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:54.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.396+0000 7f8dbb7fe700 1 -- 172.21.15.100:0/2275257622 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f8dc40e2010 con 0x7f8dcc108410 2024-05-10T11:31:54.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.396+0000 7f8dca59c700 1 --2- 172.21.15.100:0/2275257622 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8db40b2880 0x7f8db40b4d40 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc009fd0 tx=0x7f8dbc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:31:54.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.396+0000 7f8ddb1a4b80 1 Processor -- start 2024-05-10T11:31:54.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.397+0000 7f8ddb1a4b80 1 -- start start 2024-05-10T11:31:54.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.397+0000 7f8ddb1a4b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a12e69a1b0 0x55a12e69a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:54.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.397+0000 7f8ddb1a4b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a12e69aad0 0x55a12e694810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:54.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.397+0000 7f8ddb1a4b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a12e694d50 0x55a12e695190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:54.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.397+0000 7f8ddb1a4b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55a12e5e6940 con 0x55a12e69aad0 2024-05-10T11:31:54.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.397+0000 7f8ddb1a4b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a12e54bda0 con 0x55a12e69a1b0 2024-05-10T11:31:54.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.397+0000 7f8ddb1a4b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55a12e5e2900 con 0x55a12e694d50 2024-05-10T11:31:54.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.397+0000 7f8dca59c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a12e69a1b0 0x55a12e69a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:54.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.397+0000 7f8dc9d9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a12e69aad0 0x55a12e694810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:54.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.397+0000 7f8dca59c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a12e69a1b0 0x55a12e69a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:56984/0 (socket says 172.21.15.100:56984) 2024-05-10T11:31:54.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.397+0000 7f8dca59c700 1 -- 172.21.15.100:0/1768366993 learned_addr learned my addr 172.21.15.100:0/1768366993 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:31:54.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.397+0000 7f8dcad9d700 1 --2- 172.21.15.100:0/1768366993 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a12e694d50 0x55a12e695190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:54.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.397+0000 7f8dca59c700 1 -- 172.21.15.100:0/1768366993 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a12e694d50 msgr2=0x55a12e695190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:54.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.397+0000 7f8dca59c700 1 --2- 172.21.15.100:0/1768366993 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a12e694d50 0x55a12e695190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:54.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.397+0000 7f8dca59c700 1 -- 172.21.15.100:0/1768366993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a12e69aad0 msgr2=0x55a12e694810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:54.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.397+0000 7f8dca59c700 1 --2- 172.21.15.100:0/1768366993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a12e69aad0 0x55a12e694810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:54.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.397+0000 7f8dca59c700 1 -- 172.21.15.100:0/1768366993 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a12e5338d0 con 0x55a12e69a1b0 2024-05-10T11:31:54.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.397+0000 7f8dca59c700 1 --2- 172.21.15.100:0/1768366993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a12e69a1b0 0x55a12e69a590 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc00e720 tx=0x7f8dbc014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bb164eaa7ed861e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:31:54.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.398+0000 7f8db8ff9700 1 -- 172.21.15.100:0/1768366993 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dbc026600 con 0x55a12e69a1b0 2024-05-10T11:31:54.549 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:31:54.549 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T11:31:54.549 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-05-10T11:31:54.549 INFO:teuthology.orchestra.run.smithi100.stdout: "cd1721a1-cb81-4583-9312-f59658a36774": { 2024-05-10T11:31:54.549 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-05-10T11:31:54.549 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "cd1721a1-cb81-4583-9312-f59658a36774", 2024-05-10T11:31:54.550 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-05-10T11:31:54.550 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:31:54.550 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:31:54.550 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:31:54.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.398+0000 7f8db8ff9700 1 -- 172.21.15.100:0/1768366993 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8dbc026c30 con 0x55a12e69a1b0 2024-05-10T11:31:54.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.398+0000 7f8db8ff9700 1 -- 172.21.15.100:0/1768366993 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dbc01ebe0 con 0x55a12e69a1b0 2024-05-10T11:31:54.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.398+0000 7f8ddb1a4b80 1 -- 172.21.15.100:0/1768366993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a12e69a1b0 msgr2=0x55a12e69a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:54.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.398+0000 7f8ddb1a4b80 1 --2- 172.21.15.100:0/1768366993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a12e69a1b0 0x55a12e69a590 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc00e720 tx=0x7f8dbc014000 comp rx=0 tx=0).stop 2024-05-10T11:31:54.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.398+0000 7f8ddb1a4b80 1 -- 172.21.15.100:0/1768366993 shutdown_connections 2024-05-10T11:31:54.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.398+0000 7f8ddb1a4b80 1 --2- 172.21.15.100:0/1768366993 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a12e694d50 0x55a12e695190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:54.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.398+0000 7f8ddb1a4b80 1 --2- 172.21.15.100:0/1768366993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a12e69a1b0 0x55a12e69a590 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:54.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.398+0000 7f8ddb1a4b80 1 --2- 172.21.15.100:0/1768366993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a12e69aad0 0x55a12e694810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:54.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.398+0000 7f8ddb1a4b80 1 -- 172.21.15.100:0/1768366993 >> 172.21.15.100:0/1768366993 conn(0x55a12e5c0b70 msgr2=0x55a12e68f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:31:54.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.399+0000 7f8ddb1a4b80 1 -- 172.21.15.100:0/1768366993 shutdown_connections 2024-05-10T11:31:54.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.399+0000 7f8ddb1a4b80 1 -- 172.21.15.100:0/1768366993 wait complete. 2024-05-10T11:31:54.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.399+0000 7f8ddb1a4b80 1 -- start start 2024-05-10T11:31:54.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.399+0000 7f8ddb1a4b80 1 Processor -- start 2024-05-10T11:31:54.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.399+0000 7f8ddb1a4b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a12e694d50 0x55a12e6a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:54.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.399+0000 7f8ddb1a4b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a12e69a1b0 0x55a12e69db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:54.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.399+0000 7f8ddb1a4b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a12e69aad0 0x55a12e69e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:54.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.399+0000 7f8ddb1a4b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55a12e6a1450 con 0x55a12e69a1b0 2024-05-10T11:31:54.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.399+0000 7f8ddb1a4b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a12e59fed0 con 0x55a12e69aad0 2024-05-10T11:31:54.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.399+0000 7f8ddb1a4b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55a12e6a15d0 con 0x55a12e694d50 2024-05-10T11:31:54.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.399+0000 7f8ddb1a4b80 10 client.0 _ll_register_callbacks cb 0x55a12e57a5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:31:54.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.399+0000 7f8dc9d9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a12e69a1b0 0x55a12e69db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:54.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.399+0000 7f8dc9d9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a12e69a1b0 0x55a12e69db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43586/0 (socket says 172.21.15.100:43586) 2024-05-10T11:31:54.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.400+0000 7f8dc9d9b700 1 -- 172.21.15.100:0/4184959969 learned_addr learned my addr 172.21.15.100:0/4184959969 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:31:54.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.400+0000 7f8dca59c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a12e694d50 0x55a12e6a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:54.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.400+0000 7f8dcad9d700 1 --2- 172.21.15.100:0/4184959969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a12e69aad0 0x55a12e69e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:54.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.400+0000 7f8dc9d9b700 1 -- 172.21.15.100:0/4184959969 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a12e694d50 msgr2=0x55a12e6a07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:54.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.400+0000 7f8dc9d9b700 1 --2- 172.21.15.100:0/4184959969 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a12e694d50 0x55a12e6a07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:54.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.400+0000 7f8dc9d9b700 1 -- 172.21.15.100:0/4184959969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a12e69aad0 msgr2=0x55a12e69e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:54.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.400+0000 7f8dc9d9b700 1 --2- 172.21.15.100:0/4184959969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a12e69aad0 0x55a12e69e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:54.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.400+0000 7f8dc9d9b700 1 -- 172.21.15.100:0/4184959969 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a12e59d760 con 0x55a12e69a1b0 2024-05-10T11:31:54.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.400+0000 7f8dca59c700 1 --2- 172.21.15.100:0/4184959969 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a12e694d50 0x55a12e6a07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:31:54.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.400+0000 7f8dc9d9b700 1 --2- 172.21.15.100:0/4184959969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a12e69a1b0 0x55a12e69db40 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f8dc000b650 tx=0x7f8dc000bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c7cd1677b3eda11 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:31:54.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.400+0000 7f8db1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:31:54.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.400+0000 7f8db1ffb700 1 -- 172.21.15.100:0/4184959969 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc0003880 con 0x55a12e69a1b0 2024-05-10T11:31:54.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.400+0000 7f8db1ffb700 1 -- 172.21.15.100:0/4184959969 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f8dc0003a20 con 0x55a12e69a1b0 2024-05-10T11:31:54.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.400+0000 7f8ddb1a4b80 1 -- 172.21.15.100:0/4184959969 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8dbc026c30 con 0x55a12e69a1b0 2024-05-10T11:31:54.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.400+0000 7f8db1ffb700 1 -- 172.21.15.100:0/4184959969 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f8dc0031050 con 0x55a12e69a1b0 2024-05-10T11:31:54.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.400+0000 7f8ddb1a4b80 10 client.18171 fetch_fsmap learned FSMap version 33 2024-05-10T11:31:54.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.400+0000 7f8ddb1a4b80 1 -- 172.21.15.100:0/4184959969 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a12e6abd60 con 0x55a12e69a1b0 2024-05-10T11:31:54.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.401+0000 7f8db1ffb700 1 -- 172.21.15.100:0/4184959969 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8dbc026c30 con 0x55a12e69a1b0 2024-05-10T11:31:54.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.401+0000 7f8db1ffb700 1 -- 172.21.15.100:0/4184959969 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f8dc002c030 con 0x55a12e69a1b0 2024-05-10T11:31:54.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.401+0000 7f8ddb1a4b80 10 client.18171 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T11:31:54.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.401+0000 7f8ddb1a4b80 10 client.18171 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:31:54.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.401+0000 7f8ddb1a4b80 20 client.18171 populate_metadata read hostname 'smithi100' 2024-05-10T11:31:54.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.401+0000 7f8ddb1a4b80 1 --2- 172.21.15.100:0/4184959969 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55a12e6ae440 0x55a12e6b0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:54.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.401+0000 7f8ddb1a4b80 4 client.18171 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:31:54.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.401+0000 7f8ddb1a4b80 1 -- 172.21.15.100:0/4184959969 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a12e5f4320 con 0x55a12e6ae440 2024-05-10T11:31:54.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.401+0000 7f8dca59c700 1 --2- 172.21.15.100:0/4184959969 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55a12e6ae440 0x55a12e6b0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:54.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.402+0000 7f8db1ffb700 10 client.18171 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:31:54.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.402+0000 7f8dca59c700 1 --2- 172.21.15.100:0/4184959969 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55a12e6ae440 0x55a12e6b0830 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76b3cf15d2f3e05d server_cookie=a6a40a6e9cc44790 in_seq=0 out_seq=0 2024-05-10T11:31:54.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.402+0000 7f8db1ffb700 1 -- 172.21.15.100:0/4184959969 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a12e5f4320 con 0x55a12e6ae440 2024-05-10T11:31:54.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.402+0000 7f8db1ffb700 10 client.18171 handle_command_reply: tid=0 2024-05-10T11:31:54.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.402+0000 7f8ddb1a4b80 1 client.18171 shutdown 2024-05-10T11:31:54.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.402+0000 7f8ddb1a4b80 1 -- 172.21.15.100:0/4184959969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a12e69a1b0 msgr2=0x55a12e69db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:54.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.402+0000 7f8ddb1a4b80 1 --2- 172.21.15.100:0/4184959969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a12e69a1b0 0x55a12e69db40 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f8dc000b650 tx=0x7f8dc000bf30 comp rx=0 tx=0).stop 2024-05-10T11:31:54.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.402+0000 7f8ddb1a4b80 1 -- 172.21.15.100:0/4184959969 shutdown_connections 2024-05-10T11:31:54.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.402+0000 7f8ddb1a4b80 1 --2- 172.21.15.100:0/4184959969 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55a12e6ae440 0x55a12e6b0830 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:54.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.403+0000 7f8ddb1a4b80 1 --2- 172.21.15.100:0/4184959969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a12e69aad0 0x55a12e69e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:54.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.403+0000 7f8ddb1a4b80 1 --2- 172.21.15.100:0/4184959969 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a12e694d50 0x55a12e6a07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:54.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.403+0000 7f8ddb1a4b80 1 --2- 172.21.15.100:0/4184959969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a12e69a1b0 0x55a12e69db40 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:54.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.403+0000 7f8ddb1a4b80 1 -- 172.21.15.100:0/4184959969 >> 172.21.15.100:0/4184959969 conn(0x55a12e5c0b70 msgr2=0x55a12e689b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:31:54.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.403+0000 7f8db1ffb700 0 client.18171 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:31:54.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.403+0000 7f8ddb1a4b80 1 -- 172.21.15.100:0/4184959969 shutdown_connections 2024-05-10T11:31:54.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.403+0000 7f8ddb1a4b80 1 -- 172.21.15.100:0/4184959969 wait complete. 2024-05-10T11:31:54.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.403+0000 7f8ddb1a4b80 20 client.18171 trim_cache size 0 max 16384 2024-05-10T11:31:54.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.424+0000 7f8ddb1a4b80 1 Processor -- start 2024-05-10T11:31:54.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.424+0000 7f8ddb1a4b80 1 -- start start 2024-05-10T11:31:54.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.424+0000 7f8ddb1a4b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a12e694d50 0x55a12e69e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:54.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.424+0000 7f8ddb1a4b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a12e69a1b0 0x55a12e6aa4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:54.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.424+0000 7f8ddb1a4b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a12e69aad0 0x55a12e689c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:54.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.424+0000 7f8ddb1a4b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8dcc1b3ba0 con 0x55a12e69a1b0 2024-05-10T11:31:54.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.424+0000 7f8ddb1a4b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8dcc1b1430 con 0x55a12e69aad0 2024-05-10T11:31:54.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.424+0000 7f8ddb1a4b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8dcc1af280 con 0x55a12e694d50 2024-05-10T11:31:54.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.424+0000 7f8dc9d9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a12e69a1b0 0x55a12e6aa4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:54.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.424+0000 7f8dc9d9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a12e69a1b0 0x55a12e6aa4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43592/0 (socket says 172.21.15.100:43592) 2024-05-10T11:31:54.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.424+0000 7f8dc9d9b700 1 -- 172.21.15.100:0/3709004866 learned_addr learned my addr 172.21.15.100:0/3709004866 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:31:54.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.424+0000 7f8dca59c700 1 --2- 172.21.15.100:0/3709004866 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a12e694d50 0x55a12e69e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:54.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.424+0000 7f8dcad9d700 1 --2- 172.21.15.100:0/3709004866 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a12e69aad0 0x55a12e689c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:54.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.424+0000 7f8dc9d9b700 1 -- 172.21.15.100:0/3709004866 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a12e694d50 msgr2=0x55a12e69e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:54.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.424+0000 7f8dc9d9b700 1 --2- 172.21.15.100:0/3709004866 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a12e694d50 0x55a12e69e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:54.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.424+0000 7f8dc9d9b700 1 -- 172.21.15.100:0/3709004866 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a12e69aad0 msgr2=0x55a12e689c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:54.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.424+0000 7f8dc9d9b700 1 --2- 172.21.15.100:0/3709004866 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a12e69aad0 0x55a12e689c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:54.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.424+0000 7f8dc9d9b700 1 -- 172.21.15.100:0/3709004866 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a12e69c0b0 con 0x55a12e69a1b0 2024-05-10T11:31:54.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.424+0000 7f8dca59c700 1 --2- 172.21.15.100:0/3709004866 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a12e694d50 0x55a12e69e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:31:54.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.425+0000 7f8dc9d9b700 1 --2- 172.21.15.100:0/3709004866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a12e69a1b0 0x55a12e6aa4e0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f8dc00098e0 tx=0x7f8dc0009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=871d8d5d2de3ca05 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:31:54.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.425+0000 7f8db3fff700 1 -- 172.21.15.100:0/3709004866 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc0039020 con 0x55a12e69a1b0 2024-05-10T11:31:54.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.425+0000 7f8db3fff700 1 -- 172.21.15.100:0/3709004866 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8dc0024910 con 0x55a12e69a1b0 2024-05-10T11:31:54.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.425+0000 7f8db3fff700 1 -- 172.21.15.100:0/3709004866 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc000fdc0 con 0x55a12e69a1b0 2024-05-10T11:31:54.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.426+0000 7f8ddb1a4b80 1 -- 172.21.15.100:0/3709004866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a12e69a1b0 msgr2=0x55a12e6aa4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:54.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.426+0000 7f8ddb1a4b80 1 --2- 172.21.15.100:0/3709004866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a12e69a1b0 0x55a12e6aa4e0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f8dc00098e0 tx=0x7f8dc0009310 comp rx=0 tx=0).stop 2024-05-10T11:31:54.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.426+0000 7f8ddb1a4b80 1 -- 172.21.15.100:0/3709004866 shutdown_connections 2024-05-10T11:31:54.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.426+0000 7f8ddb1a4b80 1 --2- 172.21.15.100:0/3709004866 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a12e69aad0 0x55a12e689c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:54.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.426+0000 7f8ddb1a4b80 1 --2- 172.21.15.100:0/3709004866 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a12e694d50 0x55a12e69e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:54.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.426+0000 7f8ddb1a4b80 1 --2- 172.21.15.100:0/3709004866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a12e69a1b0 0x55a12e6aa4e0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:54.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.426+0000 7f8ddb1a4b80 1 -- 172.21.15.100:0/3709004866 >> 172.21.15.100:0/3709004866 conn(0x55a12e5c0b70 msgr2=0x55a12e69db00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:31:54.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.426+0000 7f8ddb1a4b80 1 -- 172.21.15.100:0/3709004866 shutdown_connections 2024-05-10T11:31:54.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.426+0000 7f8ddb1a4b80 1 -- 172.21.15.100:0/3709004866 wait complete. 2024-05-10T11:31:54.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.426+0000 7f8ddb1a4b80 1 -- start start 2024-05-10T11:31:54.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.426+0000 7f8ddb1a4b80 1 Processor -- start 2024-05-10T11:31:54.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.426+0000 7f8ddb1a4b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a12e694d50 0x55a12e6a6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:54.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.426+0000 7f8ddb1a4b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a12e69a1b0 0x55a12e6a6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:54.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.426+0000 7f8ddb1a4b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a12e69aad0 0x55a12e6b50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:54.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.426+0000 7f8ddb1a4b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8dcc1af8f0 con 0x55a12e69a1b0 2024-05-10T11:31:54.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.426+0000 7f8ddb1a4b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8dcc1b0d60 con 0x55a12e694d50 2024-05-10T11:31:54.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.426+0000 7f8ddb1a4b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8dcc1affc0 con 0x55a12e69aad0 2024-05-10T11:31:54.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.426+0000 7f8ddb1a4b80 10 client.0 _ll_register_callbacks cb 0x55a12e57a5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:31:54.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.426+0000 7f8dca59c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a12e694d50 0x55a12e6a6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:54.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.426+0000 7f8dc9d9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a12e69a1b0 0x55a12e6a6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:54.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.427+0000 7f8dc9d9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a12e69a1b0 0x55a12e6a6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43596/0 (socket says 172.21.15.100:43596) 2024-05-10T11:31:54.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.427+0000 7f8dc9d9b700 1 -- 172.21.15.100:0/1614979574 learned_addr learned my addr 172.21.15.100:0/1614979574 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:31:54.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.427+0000 7f8dcad9d700 1 --2- 172.21.15.100:0/1614979574 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a12e69aad0 0x55a12e6b50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:54.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.427+0000 7f8dc9d9b700 1 -- 172.21.15.100:0/1614979574 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a12e69aad0 msgr2=0x55a12e6b50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:54.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.427+0000 7f8dc9d9b700 1 --2- 172.21.15.100:0/1614979574 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a12e69aad0 0x55a12e6b50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:54.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.427+0000 7f8dc9d9b700 1 -- 172.21.15.100:0/1614979574 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a12e694d50 msgr2=0x55a12e6a6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:54.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.427+0000 7f8dc9d9b700 1 --2- 172.21.15.100:0/1614979574 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a12e694d50 0x55a12e6a6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:54.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.427+0000 7f8dc9d9b700 1 -- 172.21.15.100:0/1614979574 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a12e6b57a0 con 0x55a12e69a1b0 2024-05-10T11:31:54.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.427+0000 7f8dca59c700 1 --2- 172.21.15.100:0/1614979574 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a12e694d50 0x55a12e6a6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:31:54.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.427+0000 7f8dc9d9b700 1 --2- 172.21.15.100:0/1614979574 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a12e69a1b0 0x55a12e6a6da0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f8dc000b650 tx=0x7f8dc0025f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d756caa0a8ca62ea server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:31:54.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.427+0000 7f8db1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:31:54.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.427+0000 7f8db1ffb700 1 -- 172.21.15.100:0/1614979574 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc0039030 con 0x55a12e69a1b0 2024-05-10T11:31:54.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.427+0000 7f8db1ffb700 1 -- 172.21.15.100:0/1614979574 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f8dc0024680 con 0x55a12e69a1b0 2024-05-10T11:31:54.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.427+0000 7f8ddb1a4b80 1 -- 172.21.15.100:0/1614979574 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8dc0024910 con 0x55a12e69a1b0 2024-05-10T11:31:54.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.427+0000 7f8db1ffb700 1 -- 172.21.15.100:0/1614979574 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f8dc004d050 con 0x55a12e69a1b0 2024-05-10T11:31:54.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.427+0000 7f8ddb1a4b80 10 client.18177 fetch_fsmap learned FSMap version 33 2024-05-10T11:31:54.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.427+0000 7f8ddb1a4b80 1 -- 172.21.15.100:0/1614979574 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a12e718670 con 0x55a12e69a1b0 2024-05-10T11:31:54.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.428+0000 7f8db1ffb700 1 -- 172.21.15.100:0/1614979574 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8dc0024910 con 0x55a12e69a1b0 2024-05-10T11:31:54.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.428+0000 7f8db1ffb700 1 -- 172.21.15.100:0/1614979574 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f8dc002c040 con 0x55a12e69a1b0 2024-05-10T11:31:54.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.428+0000 7f8ddb1a4b80 10 client.18177 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T11:31:54.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.428+0000 7f8ddb1a4b80 10 client.18177 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:31:54.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.428+0000 7f8ddb1a4b80 20 client.18177 populate_metadata read hostname 'smithi100' 2024-05-10T11:31:54.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.428+0000 7f8ddb1a4b80 1 --2- 172.21.15.100:0/1614979574 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55a12e718cc0 0x55a12e71b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:54.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.428+0000 7f8ddb1a4b80 4 client.18177 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T11:31:54.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.428+0000 7f8ddb1a4b80 1 -- 172.21.15.100:0/1614979574 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a12e5ee3c0 con 0x55a12e718cc0 2024-05-10T11:31:54.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.428+0000 7f8dca59c700 1 --2- 172.21.15.100:0/1614979574 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55a12e718cc0 0x55a12e71b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:54.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.428+0000 7f8db1ffb700 10 client.18177 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:31:54.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.428+0000 7f8dca59c700 1 --2- 172.21.15.100:0/1614979574 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55a12e718cc0 0x55a12e71b0e0 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27da4c3cba1dab2a server_cookie=1dd57072583d78da in_seq=0 out_seq=0 2024-05-10T11:31:54.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.429+0000 7f8db1ffb700 1 -- 172.21.15.100:0/1614979574 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55a12e5ee3c0 con 0x55a12e718cc0 2024-05-10T11:31:54.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.429+0000 7f8db1ffb700 10 client.18177 handle_command_reply: tid=0 2024-05-10T11:31:54.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.429+0000 7f8ddb1a4b80 1 client.18177 shutdown 2024-05-10T11:31:54.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.429+0000 7f8ddb1a4b80 1 -- 172.21.15.100:0/1614979574 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a12e69a1b0 msgr2=0x55a12e6a6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:54.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.429+0000 7f8ddb1a4b80 1 --2- 172.21.15.100:0/1614979574 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a12e69a1b0 0x55a12e6a6da0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f8dc000b650 tx=0x7f8dc0025f10 comp rx=0 tx=0).stop 2024-05-10T11:31:54.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.429+0000 7f8ddb1a4b80 1 -- 172.21.15.100:0/1614979574 shutdown_connections 2024-05-10T11:31:54.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.429+0000 7f8ddb1a4b80 1 --2- 172.21.15.100:0/1614979574 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55a12e718cc0 0x55a12e71b0e0 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:54.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.429+0000 7f8ddb1a4b80 1 --2- 172.21.15.100:0/1614979574 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a12e69aad0 0x55a12e6b50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:54.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.429+0000 7f8ddb1a4b80 1 --2- 172.21.15.100:0/1614979574 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a12e694d50 0x55a12e6a6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:54.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.429+0000 7f8db1ffb700 0 client.18177 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:31:54.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.429+0000 7f8ddb1a4b80 1 --2- 172.21.15.100:0/1614979574 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a12e69a1b0 0x55a12e6a6da0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:54.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.429+0000 7f8ddb1a4b80 1 -- 172.21.15.100:0/1614979574 >> 172.21.15.100:0/1614979574 conn(0x55a12e5c0b70 msgr2=0x55a12e69aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:31:54.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.429+0000 7f8ddb1a4b80 1 -- 172.21.15.100:0/1614979574 shutdown_connections 2024-05-10T11:31:54.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.429+0000 7f8ddb1a4b80 1 -- 172.21.15.100:0/1614979574 wait complete. 2024-05-10T11:31:54.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.430+0000 7f8ddb1a4b80 20 client.18177 trim_cache size 0 max 16384 2024-05-10T11:31:54.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.430+0000 7f8dd0911700 1 -- 172.21.15.100:0/2275257622 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8db40b2880 msgr2=0x7f8db40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:54.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.430+0000 7f8dd0911700 1 --2- 172.21.15.100:0/2275257622 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8db40b2880 0x7f8db40b4d40 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc009fd0 tx=0x7f8dbc009260 comp rx=0 tx=0).stop 2024-05-10T11:31:54.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.430+0000 7f8dd0911700 1 -- 172.21.15.100:0/2275257622 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8dcc108410 msgr2=0x7f8dcc1991f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:54.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.430+0000 7f8dd0911700 1 --2- 172.21.15.100:0/2275257622 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8dcc108410 0x7f8dcc1991f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8dc400efc0 tx=0x7f8dc400c3a0 comp rx=0 tx=0).stop 2024-05-10T11:31:54.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.430+0000 7f8dd0911700 1 -- 172.21.15.100:0/2275257622 shutdown_connections 2024-05-10T11:31:54.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.430+0000 7f8dd0911700 1 --2- 172.21.15.100:0/2275257622 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8db40b2880 0x7f8db40b4d40 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:54.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.430+0000 7f8dd0911700 1 --2- 172.21.15.100:0/2275257622 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8dcc108410 0x7f8dcc1991f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:54.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.430+0000 7f8dd0911700 1 --2- 172.21.15.100:0/2275257622 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8dcc1023a0 0x7f8dcc198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:54.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.430+0000 7f8dd0911700 1 --2- 172.21.15.100:0/2275257622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8dcc102d60 0x7f8dcc198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:54.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.430+0000 7f8dd0911700 1 -- 172.21.15.100:0/2275257622 >> 172.21.15.100:0/2275257622 conn(0x7f8dcc079350 msgr2=0x7f8dcc101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:31:54.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.430+0000 7f8dd0911700 1 -- 172.21.15.100:0/2275257622 shutdown_connections 2024-05-10T11:31:54.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:31:54.430+0000 7f8dd0911700 1 -- 172.21.15.100:0/2275257622 wait complete. 2024-05-10T11:31:54.986 INFO:tasks.workunit.client.0.smithi100.stdout: 1 359140 2.70 MB/sec execute 589 sec latency 13.397 ms 2024-05-10T11:31:55.314 INFO:tasks.cephfs.filesystem:scrub status for tag:cd1721a1-cb81-4583-9312-f59658a36774 - {'path': '/', 'tag': 'cd1721a1-cb81-4583-9312-f59658a36774', 'options': 'recursive,force'} 2024-05-10T11:31:55.986 INFO:tasks.workunit.client.0.smithi100.stdout: 1 359702 2.70 MB/sec execute 590 sec latency 15.743 ms 2024-05-10T11:31:56.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:31:55 smithi178 ceph-mon[33359]: pgmap v827: 129 pgs: 129 active+clean; 134 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 1.3 MiB/s wr, 146 op/s 2024-05-10T11:31:56.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:31:55 smithi188 ceph-mon[34075]: pgmap v827: 129 pgs: 129 active+clean; 134 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 1.3 MiB/s wr, 146 op/s 2024-05-10T11:31:56.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:31:55 smithi100 ceph-mon[30296]: pgmap v827: 129 pgs: 129 active+clean; 134 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 1.3 MiB/s wr, 146 op/s 2024-05-10T11:31:56.986 INFO:tasks.workunit.client.0.smithi100.stdout: 1 360346 2.70 MB/sec execute 591 sec latency 15.080 ms 2024-05-10T11:31:57.841 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.839+0000 7f3d1b21d640 1 -- 172.21.15.100:0/2458222740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d140769f0 msgr2=0x7f3d14074e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:57.841 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.839+0000 7f3d1b21d640 1 --2- 172.21.15.100:0/2458222740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d140769f0 0x7f3d14074e40 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f3d08009900 tx=0x7f3d0802a410 comp rx=0 tx=0).stop 2024-05-10T11:31:57.841 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.839+0000 7f3d1b21d640 1 -- 172.21.15.100:0/2458222740 shutdown_connections 2024-05-10T11:31:57.842 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.839+0000 7f3d1b21d640 1 --2- 172.21.15.100:0/2458222740 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d1410a8e0 0x7f3d1410cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:57.842 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.839+0000 7f3d1b21d640 1 --2- 172.21.15.100:0/2458222740 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d14075410 0x7f3d14075890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:57.842 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.839+0000 7f3d1b21d640 1 --2- 172.21.15.100:0/2458222740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d140769f0 0x7f3d14074e40 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:57.842 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.839+0000 7f3d1b21d640 1 -- 172.21.15.100:0/2458222740 >> 172.21.15.100:0/2458222740 conn(0x7f3d140fdf90 msgr2=0x7f3d141003d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:31:57.842 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.839+0000 7f3d1b21d640 1 -- 172.21.15.100:0/2458222740 shutdown_connections 2024-05-10T11:31:57.842 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.839+0000 7f3d1b21d640 1 -- 172.21.15.100:0/2458222740 wait complete. 2024-05-10T11:31:57.842 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.840+0000 7f3d1b21d640 1 Processor -- start 2024-05-10T11:31:57.842 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.840+0000 7f3d1b21d640 1 -- start start 2024-05-10T11:31:57.842 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.840+0000 7f3d1b21d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d14075410 0x7f3d1419acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:57.842 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.840+0000 7f3d1b21d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d140769f0 0x7f3d1419b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:57.842 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.840+0000 7f3d1b21d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d1410a8e0 0x7f3d141a2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:57.842 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.840+0000 7f3d1b21d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3d141105c0 con 0x7f3d140769f0 2024-05-10T11:31:57.842 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.840+0000 7f3d1b21d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3d14110440 con 0x7f3d14075410 2024-05-10T11:31:57.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.840+0000 7f3d1b21d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3d14110740 con 0x7f3d1410a8e0 2024-05-10T11:31:57.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.840+0000 7f3d18fb9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d14075410 0x7f3d1419acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:57.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.840+0000 7f3d13fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d140769f0 0x7f3d1419b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:57.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.840+0000 7f3d18fb9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d14075410 0x7f3d1419acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:59572/0 (socket says 172.21.15.100:59572) 2024-05-10T11:31:57.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.840+0000 7f3d13fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d140769f0 0x7f3d1419b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46978/0 (socket says 172.21.15.100:46978) 2024-05-10T11:31:57.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.840+0000 7f3d18fb9640 1 -- 172.21.15.100:0/2471276001 learned_addr learned my addr 172.21.15.100:0/2471276001 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:31:57.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.840+0000 7f3d197ba640 1 --2- 172.21.15.100:0/2471276001 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d1410a8e0 0x7f3d141a2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:57.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.840+0000 7f3d18fb9640 1 -- 172.21.15.100:0/2471276001 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d1410a8e0 msgr2=0x7f3d141a2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:57.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.840+0000 7f3d18fb9640 1 --2- 172.21.15.100:0/2471276001 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d1410a8e0 0x7f3d141a2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:57.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.840+0000 7f3d18fb9640 1 -- 172.21.15.100:0/2471276001 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d140769f0 msgr2=0x7f3d1419b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:57.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.840+0000 7f3d18fb9640 1 --2- 172.21.15.100:0/2471276001 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d140769f0 0x7f3d1419b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:57.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.840+0000 7f3d18fb9640 1 -- 172.21.15.100:0/2471276001 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d141a28e0 con 0x7f3d14075410 2024-05-10T11:31:57.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.840+0000 7f3d197ba640 1 --2- 172.21.15.100:0/2471276001 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d1410a8e0 0x7f3d141a2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:31:57.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.840+0000 7f3d13fff640 1 --2- 172.21.15.100:0/2471276001 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d140769f0 0x7f3d1419b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:31:57.844 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.841+0000 7f3d18fb9640 1 --2- 172.21.15.100:0/2471276001 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d14075410 0x7f3d1419acf0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3d08009230 tx=0x7f3d080042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:31:57.844 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.841+0000 7f3d11ffb640 1 -- 172.21.15.100:0/2471276001 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d0803f030 con 0x7f3d14075410 2024-05-10T11:31:57.844 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.841+0000 7f3d11ffb640 1 -- 172.21.15.100:0/2471276001 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3d08032e10 con 0x7f3d14075410 2024-05-10T11:31:57.844 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.841+0000 7f3d1b21d640 1 -- 172.21.15.100:0/2471276001 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d141a2b70 con 0x7f3d14075410 2024-05-10T11:31:57.844 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.841+0000 7f3d11ffb640 1 -- 172.21.15.100:0/2471276001 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d08035070 con 0x7f3d14075410 2024-05-10T11:31:57.844 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.841+0000 7f3d1b21d640 1 -- 172.21.15.100:0/2471276001 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d141a30b0 con 0x7f3d14075410 2024-05-10T11:31:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.844+0000 7f3d1e00b740 1 Processor -- start 2024-05-10T11:31:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.844+0000 7f3d1e00b740 1 -- start start 2024-05-10T11:31:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.844+0000 7f3d11ffb640 1 -- 172.21.15.100:0/2471276001 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f3d08042030 con 0x7f3d14075410 2024-05-10T11:31:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.844+0000 7f3d1e00b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5619da866db0 0x5619da8671d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.844+0000 7f3d1e00b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5619da8677a0 0x5619da86c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.844+0000 7f3d1e00b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619da86c910 0x5619da86eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.844+0000 7f3d1e00b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5619da7cb5d0 con 0x5619da86c910 2024-05-10T11:31:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.844+0000 7f3d1e00b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5619da784890 con 0x5619da8677a0 2024-05-10T11:31:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.844+0000 7f3d1e00b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5619da7c7590 con 0x5619da866db0 2024-05-10T11:31:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.844+0000 7f3d13fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619da86c910 0x5619da86eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.844+0000 7f3d13fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619da86c910 0x5619da86eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46990/0 (socket says 172.21.15.100:46990) 2024-05-10T11:31:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.844+0000 7f3d13fff640 1 -- 172.21.15.100:0/4251538524 learned_addr learned my addr 172.21.15.100:0/4251538524 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:31:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.844+0000 7f3d18fb9640 1 --2- 172.21.15.100:0/4251538524 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5619da8677a0 0x5619da86c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.844+0000 7f3d197ba640 1 --2- 172.21.15.100:0/4251538524 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5619da866db0 0x5619da8671d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.844+0000 7f3d13fff640 1 -- 172.21.15.100:0/4251538524 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5619da866db0 msgr2=0x5619da8671d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.844+0000 7f3d13fff640 1 --2- 172.21.15.100:0/4251538524 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5619da866db0 0x5619da8671d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.844+0000 7f3d13fff640 1 -- 172.21.15.100:0/4251538524 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5619da8677a0 msgr2=0x5619da86c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.844+0000 7f3d13fff640 1 --2- 172.21.15.100:0/4251538524 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5619da8677a0 0x5619da86c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.844+0000 7f3d13fff640 1 -- 172.21.15.100:0/4251538524 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619da4dd210 con 0x5619da86c910 2024-05-10T11:31:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.844+0000 7f3d197ba640 1 --2- 172.21.15.100:0/4251538524 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5619da866db0 0x5619da8671d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:31:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.844+0000 7f3d13fff640 1 --2- 172.21.15.100:0/4251538524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619da86c910 0x5619da86eda0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f3d000114f0 tx=0x7f3d000119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0f7b04526cfa9c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:31:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.845+0000 7f3d11ffb640 1 --2- 172.21.15.100:0/2471276001 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f3d040b2920 0x7f3d040b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.845+0000 7f3d197ba640 1 --2- 172.21.15.100:0/2471276001 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f3d040b2920 0x7f3d040b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.845+0000 7f3cfeffd640 1 -- 172.21.15.100:0/4251538524 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d0001e070 con 0x5619da86c910 2024-05-10T11:31:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.845+0000 7f3d11ffb640 1 -- 172.21.15.100:0/2471276001 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f3d08102b40 con 0x7f3d14075410 2024-05-10T11:31:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.845+0000 7f3cfeffd640 1 -- 172.21.15.100:0/4251538524 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3d000044c0 con 0x5619da86c910 2024-05-10T11:31:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.845+0000 7f3cfeffd640 1 -- 172.21.15.100:0/4251538524 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d00003e60 con 0x5619da86c910 2024-05-10T11:31:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.845+0000 7f3d1e00b740 1 -- 172.21.15.100:0/4251538524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619da86c910 msgr2=0x5619da86eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.845+0000 7f3d1e00b740 1 --2- 172.21.15.100:0/4251538524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619da86c910 0x5619da86eda0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f3d000114f0 tx=0x7f3d000119b0 comp rx=0 tx=0).stop 2024-05-10T11:31:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.845+0000 7f3d197ba640 1 --2- 172.21.15.100:0/2471276001 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f3d040b2920 0x7f3d040b4de0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x5619da868000 tx=0x7f3d0c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:31:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.846+0000 7f3d1e00b740 1 -- 172.21.15.100:0/4251538524 shutdown_connections 2024-05-10T11:31:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.846+0000 7f3d1e00b740 1 --2- 172.21.15.100:0/4251538524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619da86c910 0x5619da86eda0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.846+0000 7f3d1e00b740 1 --2- 172.21.15.100:0/4251538524 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5619da8677a0 0x5619da86c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.846+0000 7f3d1e00b740 1 --2- 172.21.15.100:0/4251538524 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5619da866db0 0x5619da8671d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.846+0000 7f3d1e00b740 1 -- 172.21.15.100:0/4251538524 >> 172.21.15.100:0/4251538524 conn(0x5619da862420 msgr2=0x5619da862820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:31:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.846+0000 7f3d1e00b740 1 -- 172.21.15.100:0/4251538524 shutdown_connections 2024-05-10T11:31:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.846+0000 7f3d1e00b740 1 -- 172.21.15.100:0/4251538524 wait complete. 2024-05-10T11:31:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.846+0000 7f3d1e00b740 1 -- start start 2024-05-10T11:31:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.846+0000 7f3d1e00b740 1 Processor -- start 2024-05-10T11:31:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.846+0000 7f3d1e00b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5619da866db0 0x5619da871030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.846+0000 7f3d1e00b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619da8677a0 0x5619da871570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.846+0000 7f3d1e00b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5619da86c910 0x5619da8785b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.846+0000 7f3d1e00b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5619da86f8c0 con 0x5619da8677a0 2024-05-10T11:31:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.846+0000 7f3d1e00b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5619da788370 con 0x5619da86c910 2024-05-10T11:31:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.846+0000 7f3d1e00b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5619da86fa40 con 0x5619da866db0 2024-05-10T11:31:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.846+0000 7f3d1e00b740 10 client.0 _ll_register_callbacks cb 0x5619da4dca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:31:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.846+0000 7f3d13fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619da8677a0 0x5619da871570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.846+0000 7f3d13fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619da8677a0 0x5619da871570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47000/0 (socket says 172.21.15.100:47000) 2024-05-10T11:31:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.846+0000 7f3d197ba640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5619da86c910 0x5619da8785b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.846+0000 7f3d13fff640 1 -- 172.21.15.100:0/2018543290 learned_addr learned my addr 172.21.15.100:0/2018543290 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:31:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.847+0000 7f3d18fb9640 1 --2- 172.21.15.100:0/2018543290 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5619da866db0 0x5619da871030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.847+0000 7f3d13fff640 1 -- 172.21.15.100:0/2018543290 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5619da866db0 msgr2=0x5619da871030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.847+0000 7f3d13fff640 1 --2- 172.21.15.100:0/2018543290 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5619da866db0 0x5619da871030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.847+0000 7f3d13fff640 1 -- 172.21.15.100:0/2018543290 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5619da86c910 msgr2=0x5619da8785b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.847+0000 7f3d13fff640 1 --2- 172.21.15.100:0/2018543290 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5619da86c910 0x5619da8785b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.847+0000 7f3d13fff640 1 -- 172.21.15.100:0/2018543290 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619da5a9620 con 0x5619da8677a0 2024-05-10T11:31:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.847+0000 7f3d197ba640 1 --2- 172.21.15.100:0/2018543290 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5619da86c910 0x5619da8785b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:31:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.847+0000 7f3d13fff640 1 --2- 172.21.15.100:0/2018543290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619da8677a0 0x5619da871570 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f3d000114c0 tx=0x7f3d00003ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=151e1d2883a07e85 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:31:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.847+0000 7f3cf3fff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:31:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.847+0000 7f3cf3fff640 1 -- 172.21.15.100:0/2018543290 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d0001e070 con 0x5619da8677a0 2024-05-10T11:31:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.847+0000 7f3cf3fff640 1 -- 172.21.15.100:0/2018543290 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f3d00005040 con 0x5619da8677a0 2024-05-10T11:31:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.847+0000 7f3d1e00b740 1 -- 172.21.15.100:0/2018543290 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5619da7d91e0 con 0x5619da8677a0 2024-05-10T11:31:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.847+0000 7f3cf3fff640 1 -- 172.21.15.100:0/2018543290 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f3d00031050 con 0x5619da8677a0 2024-05-10T11:31:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.847+0000 7f3d1e00b740 10 client.18189 fetch_fsmap learned FSMap version 33 2024-05-10T11:31:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.847+0000 7f3d1e00b740 1 -- 172.21.15.100:0/2018543290 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5619da87b590 con 0x5619da8677a0 2024-05-10T11:31:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.847+0000 7f3cf3fff640 1 -- 172.21.15.100:0/2018543290 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x5619da7d91e0 con 0x5619da8677a0 2024-05-10T11:31:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.848+0000 7f3cf3fff640 1 -- 172.21.15.100:0/2018543290 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f3d00004bf0 con 0x5619da8677a0 2024-05-10T11:31:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.848+0000 7f3d1e00b740 10 client.18189 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T11:31:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.848+0000 7f3d1e00b740 10 client.18189 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:31:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.848+0000 7f3d1e00b740 20 client.18189 populate_metadata read hostname 'smithi100' 2024-05-10T11:31:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.848+0000 7f3d1e00b740 1 --2- 172.21.15.100:0/2018543290 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5619da87ca00 0x5619da87ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.848+0000 7f3d1e00b740 4 client.18189 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:31:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.848+0000 7f3d1e00b740 1 -- 172.21.15.100:0/2018543290 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5619da7d3050 con 0x5619da87ca00 2024-05-10T11:31:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.848+0000 7f3d18fb9640 1 --2- 172.21.15.100:0/2018543290 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5619da87ca00 0x5619da87ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.848+0000 7f3cf3fff640 10 client.18189 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:31:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.848+0000 7f3d18fb9640 1 --2- 172.21.15.100:0/2018543290 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5619da87ca00 0x5619da87ee60 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0c378045eee656e server_cookie=8a39793cdda99bf6 in_seq=0 out_seq=0 2024-05-10T11:31:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.849+0000 7f3cf3fff640 1 -- 172.21.15.100:0/2018543290 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5619da7d3050 con 0x5619da87ca00 2024-05-10T11:31:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.849+0000 7f3cf3fff640 10 client.18189 handle_command_reply: tid=0 2024-05-10T11:31:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.849+0000 7f3d1e00b740 1 client.18189 shutdown 2024-05-10T11:31:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.849+0000 7f3d1e00b740 1 -- 172.21.15.100:0/2018543290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619da8677a0 msgr2=0x5619da871570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.849+0000 7f3d1e00b740 1 --2- 172.21.15.100:0/2018543290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619da8677a0 0x5619da871570 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f3d000114c0 tx=0x7f3d00003ff0 comp rx=0 tx=0).stop 2024-05-10T11:31:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.849+0000 7f3d1e00b740 1 -- 172.21.15.100:0/2018543290 shutdown_connections 2024-05-10T11:31:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.849+0000 7f3d1e00b740 1 --2- 172.21.15.100:0/2018543290 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5619da87ca00 0x5619da87ee60 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.849+0000 7f3d1e00b740 1 --2- 172.21.15.100:0/2018543290 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5619da86c910 0x5619da8785b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.849+0000 7f3d1e00b740 1 --2- 172.21.15.100:0/2018543290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619da8677a0 0x5619da871570 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.849+0000 7f3d1e00b740 1 --2- 172.21.15.100:0/2018543290 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5619da866db0 0x5619da871030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.849+0000 7f3d1e00b740 1 -- 172.21.15.100:0/2018543290 >> 172.21.15.100:0/2018543290 conn(0x5619da862420 msgr2=0x5619da861580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:31:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.849+0000 7f3cf3fff640 0 client.18189 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:31:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.849+0000 7f3d1e00b740 1 -- 172.21.15.100:0/2018543290 shutdown_connections 2024-05-10T11:31:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.849+0000 7f3d1e00b740 1 -- 172.21.15.100:0/2018543290 wait complete. 2024-05-10T11:31:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.850+0000 7f3d1e00b740 20 client.18189 trim_cache size 0 max 16384 2024-05-10T11:31:57.870 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.866+0000 7f3d1e00b740 1 Processor -- start 2024-05-10T11:31:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.866+0000 7f3d1e00b740 1 -- start start 2024-05-10T11:31:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.866+0000 7f3d1e00b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5619da866db0 0x5619da871030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.866+0000 7f3d1e00b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5619da8677a0 0x5619da871570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.866+0000 7f3d1e00b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619da86c910 0x5619da878570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.866+0000 7f3d1e00b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3d141afde0 con 0x5619da86c910 2024-05-10T11:31:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.866+0000 7f3d1e00b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3d141ad6b0 con 0x5619da8677a0 2024-05-10T11:31:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.866+0000 7f3d1e00b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3d141ab500 con 0x5619da866db0 2024-05-10T11:31:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.866+0000 7f3d197ba640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619da86c910 0x5619da878570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.867+0000 7f3d18fb9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5619da866db0 0x5619da871030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.867+0000 7f3d197ba640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619da86c910 0x5619da878570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47012/0 (socket says 172.21.15.100:47012) 2024-05-10T11:31:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.867+0000 7f3d18fb9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5619da866db0 0x5619da871030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:54386/0 (socket says 172.21.15.100:54386) 2024-05-10T11:31:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.867+0000 7f3d197ba640 1 -- 172.21.15.100:0/964261331 learned_addr learned my addr 172.21.15.100:0/964261331 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:31:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.867+0000 7f3d13fff640 1 --2- 172.21.15.100:0/964261331 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5619da8677a0 0x5619da871570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.867+0000 7f3d197ba640 1 -- 172.21.15.100:0/964261331 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5619da866db0 msgr2=0x5619da871030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.867+0000 7f3d197ba640 1 --2- 172.21.15.100:0/964261331 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5619da866db0 0x5619da871030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.867+0000 7f3d197ba640 1 -- 172.21.15.100:0/964261331 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5619da8677a0 msgr2=0x5619da871570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.867+0000 7f3d197ba640 1 --2- 172.21.15.100:0/964261331 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5619da8677a0 0x5619da871570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.867+0000 7f3d197ba640 1 -- 172.21.15.100:0/964261331 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619da862250 con 0x5619da86c910 2024-05-10T11:31:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.867+0000 7f3d18fb9640 1 --2- 172.21.15.100:0/964261331 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5619da866db0 0x5619da871030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:31:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.867+0000 7f3d13fff640 1 --2- 172.21.15.100:0/964261331 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5619da8677a0 0x5619da871570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:31:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.867+0000 7f3d197ba640 1 --2- 172.21.15.100:0/964261331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619da86c910 0x5619da878570 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c003a70 tx=0x7f3d0c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2bb1aace8c00820 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:31:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.867+0000 7f3cfe7fc640 1 -- 172.21.15.100:0/964261331 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c010650 con 0x5619da86c910 2024-05-10T11:31:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.867+0000 7f3cfe7fc640 1 -- 172.21.15.100:0/964261331 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3d0c010c80 con 0x5619da86c910 2024-05-10T11:31:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.867+0000 7f3cfe7fc640 1 -- 172.21.15.100:0/964261331 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c029cb0 con 0x5619da86c910 2024-05-10T11:31:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.868+0000 7f3d1e00b740 1 -- 172.21.15.100:0/964261331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619da86c910 msgr2=0x5619da878570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.868+0000 7f3d1e00b740 1 --2- 172.21.15.100:0/964261331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619da86c910 0x5619da878570 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c003a70 tx=0x7f3d0c015000 comp rx=0 tx=0).stop 2024-05-10T11:31:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.868+0000 7f3d1e00b740 1 -- 172.21.15.100:0/964261331 shutdown_connections 2024-05-10T11:31:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.868+0000 7f3d1e00b740 1 --2- 172.21.15.100:0/964261331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619da86c910 0x5619da878570 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.868+0000 7f3d1e00b740 1 --2- 172.21.15.100:0/964261331 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5619da8677a0 0x5619da871570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.868+0000 7f3d1e00b740 1 --2- 172.21.15.100:0/964261331 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5619da866db0 0x5619da871030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.868+0000 7f3d1e00b740 1 -- 172.21.15.100:0/964261331 >> 172.21.15.100:0/964261331 conn(0x5619da862420 msgr2=0x5619da85d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:31:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.868+0000 7f3d1e00b740 1 -- 172.21.15.100:0/964261331 shutdown_connections 2024-05-10T11:31:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.868+0000 7f3d1e00b740 1 -- 172.21.15.100:0/964261331 wait complete. 2024-05-10T11:31:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.868+0000 7f3d1e00b740 1 -- start start 2024-05-10T11:31:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.868+0000 7f3d1e00b740 1 Processor -- start 2024-05-10T11:31:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.869+0000 7f3d1e00b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619da866db0 0x5619da878630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.869+0000 7f3d1e00b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5619da8677a0 0x5619da8750d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.869+0000 7f3d1e00b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5619da86c910 0x5619da8756e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.869+0000 7f3d1e00b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3d141abb70 con 0x5619da866db0 2024-05-10T11:31:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.869+0000 7f3d1e00b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3d141acfe0 con 0x5619da86c910 2024-05-10T11:31:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.869+0000 7f3d1e00b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3d141ac240 con 0x5619da8677a0 2024-05-10T11:31:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.869+0000 7f3d1e00b740 10 client.0 _ll_register_callbacks cb 0x7f3d0001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:31:57.874 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.869+0000 7f3d18fb9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619da866db0 0x5619da878630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:57.874 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.869+0000 7f3d18fb9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619da866db0 0x5619da878630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47018/0 (socket says 172.21.15.100:47018) 2024-05-10T11:31:57.874 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.869+0000 7f3d18fb9640 1 -- 172.21.15.100:0/2484283503 learned_addr learned my addr 172.21.15.100:0/2484283503 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:31:57.874 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.869+0000 7f3d13fff640 1 --2- 172.21.15.100:0/2484283503 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5619da8677a0 0x5619da8750d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:57.874 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.869+0000 7f3d197ba640 1 --2- 172.21.15.100:0/2484283503 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5619da86c910 0x5619da8756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:57.874 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.869+0000 7f3d18fb9640 1 -- 172.21.15.100:0/2484283503 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5619da8677a0 msgr2=0x5619da8750d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:57.874 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.869+0000 7f3d18fb9640 1 --2- 172.21.15.100:0/2484283503 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5619da8677a0 0x5619da8750d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:57.874 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.869+0000 7f3d18fb9640 1 -- 172.21.15.100:0/2484283503 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5619da86c910 msgr2=0x5619da8756e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:31:57.874 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.869+0000 7f3d18fb9640 1 --2- 172.21.15.100:0/2484283503 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5619da86c910 0x5619da8756e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:57.874 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.869+0000 7f3d18fb9640 1 -- 172.21.15.100:0/2484283503 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619da8e5350 con 0x5619da866db0 2024-05-10T11:31:57.874 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.869+0000 7f3d18fb9640 1 --2- 172.21.15.100:0/2484283503 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619da866db0 0x5619da878630 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f3d080c9d90 tx=0x7f3d08102f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6069dc980aa80a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:31:57.874 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.869+0000 7f3cf3fff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:31:57.874 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.869+0000 7f3cf3fff640 1 -- 172.21.15.100:0/2484283503 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d08049040 con 0x5619da866db0 2024-05-10T11:31:57.874 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.869+0000 7f3cf3fff640 1 -- 172.21.15.100:0/2484283503 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f3d080c6090 con 0x5619da866db0 2024-05-10T11:31:57.875 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.869+0000 7f3d1e00b740 1 -- 172.21.15.100:0/2484283503 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d0c010c80 con 0x5619da866db0 2024-05-10T11:31:57.875 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.870+0000 7f3cf3fff640 1 -- 172.21.15.100:0/2484283503 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f3d080c9570 con 0x5619da866db0 2024-05-10T11:31:57.875 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.870+0000 7f3d1e00b740 10 client.18195 fetch_fsmap learned FSMap version 33 2024-05-10T11:31:57.875 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.870+0000 7f3d1e00b740 1 -- 172.21.15.100:0/2484283503 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5619da8e6730 con 0x5619da866db0 2024-05-10T11:31:57.875 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.870+0000 7f3cf3fff640 1 -- 172.21.15.100:0/2484283503 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3d0c010c80 con 0x5619da866db0 2024-05-10T11:31:57.875 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.870+0000 7f3cf3fff640 1 -- 172.21.15.100:0/2484283503 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f3d080ca060 con 0x5619da866db0 2024-05-10T11:31:57.875 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.870+0000 7f3d1e00b740 10 client.18195 fetch_fsmap finished waiting for FSMap version 33 2024-05-10T11:31:57.875 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.870+0000 7f3d1e00b740 10 client.18195 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:31:57.875 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.870+0000 7f3d1e00b740 20 client.18195 populate_metadata read hostname 'smithi100' 2024-05-10T11:31:57.875 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.870+0000 7f3d1e00b740 1 --2- 172.21.15.100:0/2484283503 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5619da8e7060 0x5619da8e9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:31:57.875 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.870+0000 7f3d1e00b740 4 client.18195 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:31:57.875 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.870+0000 7f3d1e00b740 1 -- 172.21.15.100:0/2484283503 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5619da74e5f0 con 0x5619da8e7060 2024-05-10T11:31:57.875 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.870+0000 7f3d13fff640 1 --2- 172.21.15.100:0/2484283503 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5619da8e7060 0x5619da8e9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:31:57.875 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.870+0000 7f3cf3fff640 10 client.18195 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:31:57.875 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.870+0000 7f3d13fff640 1 --2- 172.21.15.100:0/2484283503 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5619da8e7060 0x5619da8e9480 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13f169e9ccfbb1c0 server_cookie=8dbc116c3549aa25 in_seq=0 out_seq=0 2024-05-10T11:31:57.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.890+0000 7f3cf3fff640 1 -- 172.21.15.100:0/2484283503 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5619da74e5f0 con 0x5619da8e7060 2024-05-10T11:31:57.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.890+0000 7f3cf3fff640 10 client.18195 handle_command_reply: tid=0 2024-05-10T11:31:57.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.890+0000 7f3d1e00b740 1 client.18195 shutdown 2024-05-10T11:31:57.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.890+0000 7f3d1e00b740 1 -- 172.21.15.100:0/2484283503 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619da866db0 msgr2=0x5619da878630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:57.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.890+0000 7f3d1e00b740 1 --2- 172.21.15.100:0/2484283503 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619da866db0 0x5619da878630 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f3d080c9d90 tx=0x7f3d08102f00 comp rx=0 tx=0).stop 2024-05-10T11:31:57.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.890+0000 7f3d1e00b740 1 -- 172.21.15.100:0/2484283503 shutdown_connections 2024-05-10T11:31:57.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.890+0000 7f3d1e00b740 1 --2- 172.21.15.100:0/2484283503 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5619da8e7060 0x5619da8e9480 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:57.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.890+0000 7f3d1e00b740 1 --2- 172.21.15.100:0/2484283503 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5619da86c910 0x5619da8756e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:57.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.890+0000 7f3d1e00b740 1 --2- 172.21.15.100:0/2484283503 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5619da8677a0 0x5619da8750d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:57.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.890+0000 7f3d1e00b740 1 --2- 172.21.15.100:0/2484283503 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619da866db0 0x5619da878630 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:57.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.890+0000 7f3cf3fff640 0 client.18195 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:31:57.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.890+0000 7f3d1e00b740 1 -- 172.21.15.100:0/2484283503 >> 172.21.15.100:0/2484283503 conn(0x5619da862420 msgr2=0x5619da870fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:31:57.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.890+0000 7f3d1e00b740 1 -- 172.21.15.100:0/2484283503 shutdown_connections 2024-05-10T11:31:57.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.890+0000 7f3d1e00b740 1 -- 172.21.15.100:0/2484283503 wait complete. 2024-05-10T11:31:57.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.890+0000 7f3d1e00b740 20 client.18195 trim_cache size 0 max 16384 2024-05-10T11:31:57.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:31:57.892 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:31:57.892 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:31:57.892 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:31:57.893 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.891+0000 7f3d1b21d640 1 -- 172.21.15.100:0/2471276001 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f3d040b2920 msgr2=0x7f3d040b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:57.893 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.891+0000 7f3d1b21d640 1 --2- 172.21.15.100:0/2471276001 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f3d040b2920 0x7f3d040b4de0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x5619da868000 tx=0x7f3d0c009500 comp rx=0 tx=0).stop 2024-05-10T11:31:57.893 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.891+0000 7f3d1b21d640 1 -- 172.21.15.100:0/2471276001 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d14075410 msgr2=0x7f3d1419acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:31:57.893 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.891+0000 7f3d1b21d640 1 --2- 172.21.15.100:0/2471276001 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d14075410 0x7f3d1419acf0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3d08009230 tx=0x7f3d080042f0 comp rx=0 tx=0).stop 2024-05-10T11:31:57.893 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.891+0000 7f3d1b21d640 1 -- 172.21.15.100:0/2471276001 shutdown_connections 2024-05-10T11:31:57.893 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.891+0000 7f3d1b21d640 1 --2- 172.21.15.100:0/2471276001 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f3d040b2920 0x7f3d040b4de0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:57.893 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.891+0000 7f3d1b21d640 1 --2- 172.21.15.100:0/2471276001 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d1410a8e0 0x7f3d141a2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:57.893 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.891+0000 7f3d1b21d640 1 --2- 172.21.15.100:0/2471276001 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d140769f0 0x7f3d1419b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:57.893 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.891+0000 7f3d1b21d640 1 --2- 172.21.15.100:0/2471276001 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d14075410 0x7f3d1419acf0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:31:57.893 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.891+0000 7f3d1b21d640 1 -- 172.21.15.100:0/2471276001 >> 172.21.15.100:0/2471276001 conn(0x7f3d140fdf90 msgr2=0x7f3d141067e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:31:57.893 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.891+0000 7f3d1b21d640 1 -- 172.21.15.100:0/2471276001 shutdown_connections 2024-05-10T11:31:57.893 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:31:57.891+0000 7f3d1b21d640 1 -- 172.21.15.100:0/2471276001 wait complete. 2024-05-10T11:31:57.986 INFO:tasks.workunit.client.0.smithi100.stdout: 1 360864 2.70 MB/sec execute 592 sec latency 19.534 ms 2024-05-10T11:31:58.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:31:57 smithi178 ceph-mon[33359]: pgmap v828: 129 pgs: 129 active+clean; 138 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 1.6 MiB/s wr, 186 op/s 2024-05-10T11:31:58.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:31:57 smithi188 ceph-mon[34075]: pgmap v828: 129 pgs: 129 active+clean; 138 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 1.6 MiB/s wr, 186 op/s 2024-05-10T11:31:58.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:31:57 smithi100 ceph-mon[30296]: pgmap v828: 129 pgs: 129 active+clean; 138 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 1.6 MiB/s wr, 186 op/s 2024-05-10T11:31:58.986 INFO:tasks.workunit.client.0.smithi100.stdout: 1 361565 2.70 MB/sec execute 593 sec latency 15.347 ms 2024-05-10T11:31:59.986 INFO:tasks.workunit.client.0.smithi100.stdout: 1 362042 2.70 MB/sec execute 594 sec latency 15.070 ms 2024-05-10T11:32:00.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:31:59 smithi178 ceph-mon[33359]: pgmap v829: 129 pgs: 129 active+clean; 138 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 1.3 MiB/s wr, 144 op/s 2024-05-10T11:32:00.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:31:59 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:32:00.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:31:59 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:32:00.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:31:59 smithi188 ceph-mon[34075]: pgmap v829: 129 pgs: 129 active+clean; 138 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 1.3 MiB/s wr, 144 op/s 2024-05-10T11:32:00.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:31:59 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:32:00.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:31:59 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:32:00.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:31:59 smithi100 ceph-mon[30296]: pgmap v829: 129 pgs: 129 active+clean; 138 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 1.3 MiB/s wr, 144 op/s 2024-05-10T11:32:00.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:31:59 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:32:00.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:31:59 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:32:00.986 INFO:tasks.workunit.client.0.smithi100.stdout: 1 362327 2.70 MB/sec execute 595 sec latency 14.684 ms 2024-05-10T11:32:01.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:32:00 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:32:01.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:32:00 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:32:01.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:32:00 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:32:01.986 INFO:tasks.workunit.client.0.smithi100.stdout: 1 362738 2.70 MB/sec execute 596 sec latency 15.305 ms 2024-05-10T11:32:02.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:32:01 smithi178 ceph-mon[33359]: pgmap v830: 129 pgs: 129 active+clean; 138 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 1.3 MiB/s wr, 161 op/s 2024-05-10T11:32:02.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:32:01 smithi188 ceph-mon[34075]: pgmap v830: 129 pgs: 129 active+clean; 138 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 1.3 MiB/s wr, 161 op/s 2024-05-10T11:32:02.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:32:01 smithi100 ceph-mon[30296]: pgmap v830: 129 pgs: 129 active+clean; 138 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 1.3 MiB/s wr, 161 op/s 2024-05-10T11:32:02.986 INFO:tasks.workunit.client.0.smithi100.stdout: 1 363320 2.70 MB/sec execute 597 sec latency 20.268 ms 2024-05-10T11:32:03.986 INFO:tasks.workunit.client.0.smithi100.stdout: 1 363971 2.70 MB/sec execute 598 sec latency 15.614 ms 2024-05-10T11:32:04.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:32:03 smithi178 ceph-mon[33359]: pgmap v831: 129 pgs: 129 active+clean; 139 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 1.5 MiB/s wr, 170 op/s 2024-05-10T11:32:04.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:32:03 smithi188 ceph-mon[34075]: pgmap v831: 129 pgs: 129 active+clean; 139 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 1.5 MiB/s wr, 170 op/s 2024-05-10T11:32:04.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:32:03 smithi100 ceph-mon[30296]: pgmap v831: 129 pgs: 129 active+clean; 139 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 1.5 MiB/s wr, 170 op/s 2024-05-10T11:32:04.986 INFO:tasks.workunit.client.0.smithi100.stdout: 1 364551 2.70 MB/sec execute 599 sec latency 16.372 ms 2024-05-10T11:32:05.986 INFO:tasks.workunit.client.0.smithi100.stdout: 1 cleanup 600 sec 2024-05-10T11:32:06.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:32:05 smithi178 ceph-mon[33359]: pgmap v832: 129 pgs: 129 active+clean; 139 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 1.4 MiB/s wr, 137 op/s 2024-05-10T11:32:06.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:32:05 smithi188 ceph-mon[34075]: pgmap v832: 129 pgs: 129 active+clean; 139 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 1.4 MiB/s wr, 137 op/s 2024-05-10T11:32:06.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:32:05 smithi100 ceph-mon[30296]: pgmap v832: 129 pgs: 129 active+clean; 139 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 1.4 MiB/s wr, 137 op/s 2024-05-10T11:32:06.804 INFO:tasks.workunit.client.0.smithi100.stdout: 0 cleanup 601 sec 2024-05-10T11:32:06.804 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-05-10T11:32:06.804 INFO:tasks.workunit.client.0.smithi100.stdout: Operation Count AvgLat MaxLat 2024-05-10T11:32:06.804 INFO:tasks.workunit.client.0.smithi100.stdout: ---------------------------------------- 2024-05-10T11:32:06.804 INFO:tasks.workunit.client.0.smithi100.stdout: NTCreateX 52073 3.599 125.209 2024-05-10T11:32:06.804 INFO:tasks.workunit.client.0.smithi100.stdout: Close 38227 0.046 2.398 2024-05-10T11:32:06.804 INFO:tasks.workunit.client.0.smithi100.stdout: Rename 2206 7.044 65.422 2024-05-10T11:32:06.804 INFO:tasks.workunit.client.0.smithi100.stdout: Unlink 10529 3.979 69.461 2024-05-10T11:32:06.804 INFO:tasks.workunit.client.0.smithi100.stdout: Qpathinfo 47241 3.039 114.550 2024-05-10T11:32:06.805 INFO:tasks.workunit.client.0.smithi100.stdout: Qfileinfo 8238 0.071 1.161 2024-05-10T11:32:06.805 INFO:tasks.workunit.client.0.smithi100.stdout: Qfsinfo 8651 0.301 3.924 2024-05-10T11:32:06.805 INFO:tasks.workunit.client.0.smithi100.stdout: Sfileinfo 4250 1.362 12.490 2024-05-10T11:32:06.805 INFO:tasks.workunit.client.0.smithi100.stdout: Find 18260 10.056 209.066 2024-05-10T11:32:06.805 INFO:tasks.workunit.client.0.smithi100.stdout: WriteX 25733 0.156 43.766 2024-05-10T11:32:06.805 INFO:tasks.workunit.client.0.smithi100.stdout: ReadX 81659 0.004 0.540 2024-05-10T11:32:06.805 INFO:tasks.workunit.client.0.smithi100.stdout: LockX 170 0.639 1.129 2024-05-10T11:32:06.805 INFO:tasks.workunit.client.0.smithi100.stdout: UnlockX 170 0.793 46.629 2024-05-10T11:32:06.805 INFO:tasks.workunit.client.0.smithi100.stdout: Flush 3654 3.264 96.981 2024-05-10T11:32:06.805 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-05-10T11:32:06.805 INFO:tasks.workunit.client.0.smithi100.stdout:Throughput 2.70115 MB/sec 1 clients 1 procs max_latency=209.072 ms 2024-05-10T11:32:06.806 INFO:tasks.workunit.client.0.smithi100.stderr:+ dbench 10 2024-05-10T11:32:06.806 INFO:tasks.workunit.client.0.smithi100.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-05-10T11:32:06.806 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-05-10T11:32:06.806 INFO:tasks.workunit.client.0.smithi100.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-05-10T11:32:06.806 INFO:tasks.workunit.client.0.smithi100.stdout:8 of 10 processes prepared for launch 0 sec 2024-05-10T11:32:06.906 INFO:tasks.workunit.client.0.smithi100.stdout:10 of 10 processes prepared for launch 0 sec 2024-05-10T11:32:06.906 INFO:tasks.workunit.client.0.smithi100.stdout:releasing clients 2024-05-10T11:32:07.906 INFO:tasks.workunit.client.0.smithi100.stdout: 10 266 100.26 MB/sec warmup 1 sec latency 60.417 ms 2024-05-10T11:32:08.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:32:07 smithi178 ceph-mon[33359]: pgmap v833: 129 pgs: 129 active+clean; 139 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 1.5 MiB/s wr, 183 op/s 2024-05-10T11:32:08.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:32:07 smithi188 ceph-mon[34075]: pgmap v833: 129 pgs: 129 active+clean; 139 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 1.5 MiB/s wr, 183 op/s 2024-05-10T11:32:08.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:32:07 smithi100 ceph-mon[30296]: pgmap v833: 129 pgs: 129 active+clean; 139 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 1.5 MiB/s wr, 183 op/s 2024-05-10T11:32:08.906 INFO:tasks.workunit.client.0.smithi100.stdout: 10 502 87.34 MB/sec warmup 2 sec latency 61.427 ms 2024-05-10T11:32:09.906 INFO:tasks.workunit.client.0.smithi100.stdout: 10 697 70.69 MB/sec warmup 3 sec latency 103.129 ms 2024-05-10T11:32:10.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:32:09 smithi178 ceph-mon[33359]: pgmap v834: 129 pgs: 129 active+clean; 153 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 2.4 MiB/s wr, 134 op/s 2024-05-10T11:32:10.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:32:09 smithi188 ceph-mon[34075]: pgmap v834: 129 pgs: 129 active+clean; 153 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 2.4 MiB/s wr, 134 op/s 2024-05-10T11:32:10.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:32:09 smithi100 ceph-mon[30296]: pgmap v834: 129 pgs: 129 active+clean; 153 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 2.4 MiB/s wr, 134 op/s 2024-05-10T11:32:10.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:32:10 smithi100 ceph-mon[30296]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T11:32:10.906 INFO:tasks.workunit.client.0.smithi100.stdout: 10 846 53.90 MB/sec warmup 4 sec latency 113.439 ms 2024-05-10T11:32:11.003 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.000+0000 7fc627b19640 1 -- 172.21.15.100:0/2244487131 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc620102b00 msgr2=0x7fc620102f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:11.003 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.000+0000 7fc627b19640 1 --2- 172.21.15.100:0/2244487131 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc620102b00 0x7fc620102f80 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc610009900 tx=0x7fc61002a410 comp rx=0 tx=0).stop 2024-05-10T11:32:11.003 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.000+0000 7fc627b19640 1 -- 172.21.15.100:0/2244487131 shutdown_connections 2024-05-10T11:32:11.003 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.000+0000 7fc627b19640 1 --2- 172.21.15.100:0/2244487131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6201034c0 0x7fc620109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:11.004 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.000+0000 7fc627b19640 1 --2- 172.21.15.100:0/2244487131 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc620102b00 0x7fc620102f80 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:11.004 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.000+0000 7fc627b19640 1 --2- 172.21.15.100:0/2244487131 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc620101910 0x7fc620101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:11.004 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.000+0000 7fc627b19640 1 -- 172.21.15.100:0/2244487131 >> 172.21.15.100:0/2244487131 conn(0x7fc6200fce80 msgr2=0x7fc6200ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:32:11.004 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.000+0000 7fc627b19640 1 -- 172.21.15.100:0/2244487131 shutdown_connections 2024-05-10T11:32:11.004 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.000+0000 7fc627b19640 1 -- 172.21.15.100:0/2244487131 wait complete. 2024-05-10T11:32:11.004 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.001+0000 7fc627b19640 1 Processor -- start 2024-05-10T11:32:11.004 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.001+0000 7fc627b19640 1 -- start start 2024-05-10T11:32:11.004 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.001+0000 7fc627b19640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc620101910 0x7fc620198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:11.004 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.001+0000 7fc627b19640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc620102b00 0x7fc620199060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:11.004 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.001+0000 7fc627b19640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6201034c0 0x7fc6201a00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:11.005 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.001+0000 7fc627b19640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc62010d2f0 con 0x7fc6201034c0 2024-05-10T11:32:11.005 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.001+0000 7fc627b19640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc62010d170 con 0x7fc620102b00 2024-05-10T11:32:11.005 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.001+0000 7fc627b19640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc62010d470 con 0x7fc620101910 2024-05-10T11:32:11.005 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.002+0000 7fc6250b4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc620102b00 0x7fc620199060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:11.005 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.002+0000 7fc6260b6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6201034c0 0x7fc6201a00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:11.005 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.002+0000 7fc6258b5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc620101910 0x7fc620198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:11.005 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.002+0000 7fc6250b4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc620102b00 0x7fc620199060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:39380/0 (socket says 172.21.15.100:39380) 2024-05-10T11:32:11.005 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.002+0000 7fc6250b4640 1 -- 172.21.15.100:0/3109085859 learned_addr learned my addr 172.21.15.100:0/3109085859 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:32:11.005 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.002+0000 7fc6258b5640 1 -- 172.21.15.100:0/3109085859 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc620102b00 msgr2=0x7fc620199060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:11.005 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.002+0000 7fc6258b5640 1 --2- 172.21.15.100:0/3109085859 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc620102b00 0x7fc620199060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:11.005 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.002+0000 7fc6258b5640 1 -- 172.21.15.100:0/3109085859 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6201034c0 msgr2=0x7fc6201a00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:11.005 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.002+0000 7fc6258b5640 1 --2- 172.21.15.100:0/3109085859 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6201034c0 0x7fc6201a00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:11.005 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.002+0000 7fc6258b5640 1 -- 172.21.15.100:0/3109085859 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6201a0740 con 0x7fc620101910 2024-05-10T11:32:11.005 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.002+0000 7fc6258b5640 1 --2- 172.21.15.100:0/3109085859 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc620101910 0x7fc620198b20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc61c00e9c0 tx=0x7fc61c00edc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:32:11.006 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.002+0000 7fc616ffd640 1 -- 172.21.15.100:0/3109085859 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61c00ccc0 con 0x7fc620101910 2024-05-10T11:32:11.006 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.002+0000 7fc616ffd640 1 -- 172.21.15.100:0/3109085859 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc61c0035f0 con 0x7fc620101910 2024-05-10T11:32:11.006 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.002+0000 7fc627b19640 1 -- 172.21.15.100:0/3109085859 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6201a0a30 con 0x7fc620101910 2024-05-10T11:32:11.006 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.002+0000 7fc616ffd640 1 -- 172.21.15.100:0/3109085859 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61c00ce60 con 0x7fc620101910 2024-05-10T11:32:11.006 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.002+0000 7fc627b19640 1 -- 172.21.15.100:0/3109085859 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6201a0f30 con 0x7fc620101910 2024-05-10T11:32:11.006 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:32:10 smithi178 ceph-mon[33359]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T11:32:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.005+0000 7fc616ffd640 1 -- 172.21.15.100:0/3109085859 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fc61c0037a0 con 0x7fc620101910 2024-05-10T11:32:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.005+0000 7fc62a836740 1 Processor -- start 2024-05-10T11:32:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.006+0000 7fc62a836740 1 -- start start 2024-05-10T11:32:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.006+0000 7fc62a836740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f9a833cdb0 0x55f9a833d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.006+0000 7fc62a836740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9a833d7a0 0x55f9a83422a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.006+0000 7fc62a836740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f9a8342910 0x55f9a8344da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.006+0000 7fc62a836740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55f9a82a15d0 con 0x55f9a833d7a0 2024-05-10T11:32:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.006+0000 7fc62a836740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f9a825a890 con 0x55f9a8342910 2024-05-10T11:32:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.006+0000 7fc62a836740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55f9a829d590 con 0x55f9a833cdb0 2024-05-10T11:32:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.006+0000 7fc6260b6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f9a833cdb0 0x55f9a833d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.006+0000 7fc6258b5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9a833d7a0 0x55f9a83422a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.006+0000 7fc6260b6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f9a833cdb0 0x55f9a833d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:55776/0 (socket says 172.21.15.100:55776) 2024-05-10T11:32:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.006+0000 7fc6250b4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f9a8342910 0x55f9a8344da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.006+0000 7fc6260b6640 1 -- 172.21.15.100:0/1623947416 learned_addr learned my addr 172.21.15.100:0/1623947416 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:32:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.006+0000 7fc6250b4640 1 -- 172.21.15.100:0/1623947416 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f9a833cdb0 msgr2=0x55f9a833d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.006+0000 7fc6250b4640 1 --2- 172.21.15.100:0/1623947416 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f9a833cdb0 0x55f9a833d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.006+0000 7fc6250b4640 1 -- 172.21.15.100:0/1623947416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9a833d7a0 msgr2=0x55f9a83422a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.006+0000 7fc6250b4640 1 --2- 172.21.15.100:0/1623947416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9a833d7a0 0x55f9a83422a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.006+0000 7fc6250b4640 1 -- 172.21.15.100:0/1623947416 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9a7fb3210 con 0x55f9a8342910 2024-05-10T11:32:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.007+0000 7fc616ffd640 1 --2- 172.21.15.100:0/3109085859 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc60c0b28a0 0x7fc60c0b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.007+0000 7fc6260b6640 1 --2- 172.21.15.100:0/3109085859 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc60c0b28a0 0x7fc60c0b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.007+0000 7fc6250b4640 1 --2- 172.21.15.100:0/1623947416 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f9a8342910 0x55f9a8344da0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc610000c00 tx=0x7fc610005e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29143460b6a532d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:32:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.007+0000 7fc616ffd640 1 -- 172.21.15.100:0/3109085859 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fc61c0e2820 con 0x7fc620101910 2024-05-10T11:32:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.007+0000 7fc60bfff640 1 -- 172.21.15.100:0/1623947416 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc610035030 con 0x55f9a8342910 2024-05-10T11:32:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.007+0000 7fc60bfff640 1 -- 172.21.15.100:0/1623947416 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc610040070 con 0x55f9a8342910 2024-05-10T11:32:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.007+0000 7fc60bfff640 1 -- 172.21.15.100:0/1623947416 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc610003850 con 0x55f9a8342910 2024-05-10T11:32:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.008+0000 7fc62a836740 1 -- 172.21.15.100:0/1623947416 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f9a8342910 msgr2=0x55f9a8344da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.008+0000 7fc6260b6640 1 --2- 172.21.15.100:0/3109085859 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc60c0b28a0 0x7fc60c0b4d60 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x55f9a833e000 tx=0x7fc618009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:32:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.008+0000 7fc62a836740 1 --2- 172.21.15.100:0/1623947416 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f9a8342910 0x55f9a8344da0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc610000c00 tx=0x7fc610005e40 comp rx=0 tx=0).stop 2024-05-10T11:32:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.008+0000 7fc62a836740 1 -- 172.21.15.100:0/1623947416 shutdown_connections 2024-05-10T11:32:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.008+0000 7fc62a836740 1 --2- 172.21.15.100:0/1623947416 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f9a8342910 0x55f9a8344da0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.008+0000 7fc62a836740 1 --2- 172.21.15.100:0/1623947416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9a833d7a0 0x55f9a83422a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.008+0000 7fc62a836740 1 --2- 172.21.15.100:0/1623947416 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f9a833cdb0 0x55f9a833d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.008+0000 7fc62a836740 1 -- 172.21.15.100:0/1623947416 >> 172.21.15.100:0/1623947416 conn(0x55f9a8338420 msgr2=0x55f9a8338820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:32:11.015 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.009+0000 7fc62a836740 1 -- 172.21.15.100:0/1623947416 shutdown_connections 2024-05-10T11:32:11.015 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.009+0000 7fc62a836740 1 -- 172.21.15.100:0/1623947416 wait complete. 2024-05-10T11:32:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.009+0000 7fc62a836740 1 -- start start 2024-05-10T11:32:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.009+0000 7fc62a836740 1 Processor -- start 2024-05-10T11:32:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.010+0000 7fc62a836740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9a833cdb0 0x55f9a8346f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.010+0000 7fc62a836740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f9a833d7a0 0x55f9a8347440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.010+0000 7fc62a836740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f9a8342910 0x55f9a834e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.010+0000 7fc62a836740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55f9a8345860 con 0x55f9a833cdb0 2024-05-10T11:32:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.010+0000 7fc62a836740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f9a825e370 con 0x55f9a8342910 2024-05-10T11:32:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.010+0000 7fc62a836740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55f9a83459e0 con 0x55f9a833d7a0 2024-05-10T11:32:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.010+0000 7fc62a836740 10 client.0 _ll_register_callbacks cb 0x55f9a7fb2a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:32:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.010+0000 7fc6258b5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9a833cdb0 0x55f9a8346f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.010+0000 7fc6258b5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9a833cdb0 0x55f9a8346f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36348/0 (socket says 172.21.15.100:36348) 2024-05-10T11:32:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.010+0000 7fc6258b5640 1 -- 172.21.15.100:0/3293042002 learned_addr learned my addr 172.21.15.100:0/3293042002 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:32:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.010+0000 7fc6250b4640 1 --2- 172.21.15.100:0/3293042002 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f9a833d7a0 0x55f9a8347440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:11.017 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.010+0000 7fc6260b6640 1 --2- 172.21.15.100:0/3293042002 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f9a8342910 0x55f9a834e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:11.017 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.010+0000 7fc6258b5640 1 -- 172.21.15.100:0/3293042002 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f9a833d7a0 msgr2=0x55f9a8347440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:11.017 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.010+0000 7fc6258b5640 1 --2- 172.21.15.100:0/3293042002 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f9a833d7a0 0x55f9a8347440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:11.017 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.010+0000 7fc6258b5640 1 -- 172.21.15.100:0/3293042002 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f9a8342910 msgr2=0x55f9a834e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:11.017 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.010+0000 7fc6258b5640 1 --2- 172.21.15.100:0/3293042002 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f9a8342910 0x55f9a834e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:11.017 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.010+0000 7fc6258b5640 1 -- 172.21.15.100:0/3293042002 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9a807f620 con 0x55f9a833cdb0 2024-05-10T11:32:11.017 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.010+0000 7fc6260b6640 1 --2- 172.21.15.100:0/3293042002 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f9a8342910 0x55f9a834e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:32:11.017 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.011+0000 7fc6258b5640 1 --2- 172.21.15.100:0/3293042002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9a833cdb0 0x55f9a8346f00 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fc61c0a97e0 tx=0x7fc61c0a9970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d25d9d5ab69875a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:32:11.017 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.011+0000 7fc6097fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:32:11.017 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.011+0000 7fc6097fa640 1 -- 172.21.15.100:0/3293042002 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61c0a95f0 con 0x55f9a833cdb0 2024-05-10T11:32:11.017 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.011+0000 7fc6097fa640 1 -- 172.21.15.100:0/3293042002 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fc61c0b24b0 con 0x55f9a833cdb0 2024-05-10T11:32:11.017 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.011+0000 7fc62a836740 1 -- 172.21.15.100:0/3293042002 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc610040070 con 0x55f9a833cdb0 2024-05-10T11:32:11.017 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.011+0000 7fc6097fa640 1 -- 172.21.15.100:0/3293042002 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc61c0aa1e0 con 0x55f9a833cdb0 2024-05-10T11:32:11.017 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.012+0000 7fc62a836740 10 client.18207 fetch_fsmap learned FSMap version 35 2024-05-10T11:32:11.018 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.012+0000 7fc62a836740 1 -- 172.21.15.100:0/3293042002 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f9a8351530 con 0x55f9a833cdb0 2024-05-10T11:32:11.018 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.012+0000 7fc6097fa640 1 -- 172.21.15.100:0/3293042002 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc610040070 con 0x55f9a833cdb0 2024-05-10T11:32:11.018 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.012+0000 7fc6097fa640 1 -- 172.21.15.100:0/3293042002 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7fc61c06f290 con 0x55f9a833cdb0 2024-05-10T11:32:11.018 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.012+0000 7fc62a836740 10 client.18207 fetch_fsmap finished waiting for FSMap version 35 2024-05-10T11:32:11.018 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.012+0000 7fc62a836740 10 client.18207 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:32:11.018 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.012+0000 7fc62a836740 20 client.18207 populate_metadata read hostname 'smithi100' 2024-05-10T11:32:11.018 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.012+0000 7fc62a836740 1 --2- 172.21.15.100:0/3293042002 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55f9a8352c50 0x55f9a83550a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:11.018 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.012+0000 7fc62a836740 4 client.18207 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:32:11.018 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.012+0000 7fc62a836740 1 -- 172.21.15.100:0/3293042002 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9a82af1e0 con 0x55f9a8352c50 2024-05-10T11:32:11.018 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.013+0000 7fc6250b4640 1 --2- 172.21.15.100:0/3293042002 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55f9a8352c50 0x55f9a83550a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:11.018 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.013+0000 7fc6250b4640 1 --2- 172.21.15.100:0/3293042002 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55f9a8352c50 0x55f9a83550a0 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c60e82442cddcf4d server_cookie=f1eea0d68b4c0bcd in_seq=0 out_seq=0 2024-05-10T11:32:11.018 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.013+0000 7fc6097fa640 10 client.18207 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:32:11.018 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.015+0000 7fc6097fa640 1 -- 172.21.15.100:0/3293042002 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f9a82af1e0 con 0x55f9a8352c50 2024-05-10T11:32:11.019 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.015+0000 7fc6097fa640 10 client.18207 handle_command_reply: tid=0 2024-05-10T11:32:11.019 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.015+0000 7fc62a836740 1 client.18207 shutdown 2024-05-10T11:32:11.020 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.015+0000 7fc62a836740 1 -- 172.21.15.100:0/3293042002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9a833cdb0 msgr2=0x55f9a8346f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:11.020 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.015+0000 7fc62a836740 1 --2- 172.21.15.100:0/3293042002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9a833cdb0 0x55f9a8346f00 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fc61c0a97e0 tx=0x7fc61c0a9970 comp rx=0 tx=0).stop 2024-05-10T11:32:11.020 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.015+0000 7fc62a836740 1 -- 172.21.15.100:0/3293042002 shutdown_connections 2024-05-10T11:32:11.020 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.015+0000 7fc62a836740 1 --2- 172.21.15.100:0/3293042002 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55f9a8352c50 0x55f9a83550a0 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:11.020 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.015+0000 7fc62a836740 1 --2- 172.21.15.100:0/3293042002 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f9a8342910 0x55f9a834e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:11.021 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.016+0000 7fc6097fa640 0 client.18207 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:32:11.021 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.016+0000 7fc62a836740 1 --2- 172.21.15.100:0/3293042002 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f9a833d7a0 0x55f9a8347440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:11.021 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.016+0000 7fc62a836740 1 --2- 172.21.15.100:0/3293042002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9a833cdb0 0x55f9a8346f00 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:11.021 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.016+0000 7fc62a836740 1 -- 172.21.15.100:0/3293042002 >> 172.21.15.100:0/3293042002 conn(0x55f9a8338420 msgr2=0x55f9a8337580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:32:11.021 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.016+0000 7fc62a836740 1 -- 172.21.15.100:0/3293042002 shutdown_connections 2024-05-10T11:32:11.021 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.016+0000 7fc62a836740 1 -- 172.21.15.100:0/3293042002 wait complete. 2024-05-10T11:32:11.021 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.016+0000 7fc62a836740 20 client.18207 trim_cache size 0 max 16384 2024-05-10T11:32:11.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:32:10 smithi188 ceph-mon[34075]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T11:32:11.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.038+0000 7fc62a836740 1 Processor -- start 2024-05-10T11:32:11.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.038+0000 7fc62a836740 1 -- start start 2024-05-10T11:32:11.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.038+0000 7fc62a836740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f9a833cdb0 0x55f9a835a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:11.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.038+0000 7fc62a836740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f9a833d7a0 0x55f9a834e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:11.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.038+0000 7fc62a836740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9a8342910 0x55f9a835aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:11.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.038+0000 7fc62a836740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc6201afdb0 con 0x55f9a8342910 2024-05-10T11:32:11.046 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.038+0000 7fc62a836740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc6201ad650 con 0x55f9a833d7a0 2024-05-10T11:32:11.046 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.038+0000 7fc62a836740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc6201ab4a0 con 0x55f9a833cdb0 2024-05-10T11:32:11.046 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.038+0000 7fc6260b6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9a8342910 0x55f9a835aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:11.046 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.038+0000 7fc6260b6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9a8342910 0x55f9a835aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36354/0 (socket says 172.21.15.100:36354) 2024-05-10T11:32:11.046 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.038+0000 7fc6260b6640 1 -- 172.21.15.100:0/3412152716 learned_addr learned my addr 172.21.15.100:0/3412152716 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:32:11.047 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.038+0000 7fc6258b5640 1 --2- 172.21.15.100:0/3412152716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f9a833cdb0 0x55f9a835a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:11.047 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.039+0000 7fc6250b4640 1 --2- 172.21.15.100:0/3412152716 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f9a833d7a0 0x55f9a834e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:11.047 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.039+0000 7fc6260b6640 1 -- 172.21.15.100:0/3412152716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f9a833cdb0 msgr2=0x55f9a835a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:11.047 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.039+0000 7fc6260b6640 1 --2- 172.21.15.100:0/3412152716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f9a833cdb0 0x55f9a835a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:11.047 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.039+0000 7fc6260b6640 1 -- 172.21.15.100:0/3412152716 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f9a833d7a0 msgr2=0x55f9a834e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:11.048 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.039+0000 7fc6260b6640 1 --2- 172.21.15.100:0/3412152716 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f9a833d7a0 0x55f9a834e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:11.048 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.039+0000 7fc6260b6640 1 -- 172.21.15.100:0/3412152716 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9a833c910 con 0x55f9a8342910 2024-05-10T11:32:11.048 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.039+0000 7fc6260b6640 1 --2- 172.21.15.100:0/3412152716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9a8342910 0x55f9a835aef0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fc61800f980 tx=0x7fc618017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b72a5ee320d05fb0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:32:11.048 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.039+0000 7fc60b7fe640 1 -- 172.21.15.100:0/3412152716 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc618012020 con 0x55f9a8342910 2024-05-10T11:32:11.048 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.039+0000 7fc60b7fe640 1 -- 172.21.15.100:0/3412152716 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc618003e00 con 0x55f9a8342910 2024-05-10T11:32:11.048 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.039+0000 7fc60b7fe640 1 -- 172.21.15.100:0/3412152716 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61801d4d0 con 0x55f9a8342910 2024-05-10T11:32:11.048 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.040+0000 7fc62a836740 1 -- 172.21.15.100:0/3412152716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9a8342910 msgr2=0x55f9a835aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:11.049 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.040+0000 7fc62a836740 1 --2- 172.21.15.100:0/3412152716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9a8342910 0x55f9a835aef0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fc61800f980 tx=0x7fc618017000 comp rx=0 tx=0).stop 2024-05-10T11:32:11.049 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.040+0000 7fc62a836740 1 -- 172.21.15.100:0/3412152716 shutdown_connections 2024-05-10T11:32:11.049 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.040+0000 7fc62a836740 1 --2- 172.21.15.100:0/3412152716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9a8342910 0x55f9a835aef0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:11.049 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.040+0000 7fc62a836740 1 --2- 172.21.15.100:0/3412152716 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f9a833d7a0 0x55f9a834e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:11.049 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.040+0000 7fc62a836740 1 --2- 172.21.15.100:0/3412152716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f9a833cdb0 0x55f9a835a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:11.049 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.040+0000 7fc62a836740 1 -- 172.21.15.100:0/3412152716 >> 172.21.15.100:0/3412152716 conn(0x55f9a8338420 msgr2=0x55f9a8338800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:32:11.049 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.040+0000 7fc62a836740 1 -- 172.21.15.100:0/3412152716 shutdown_connections 2024-05-10T11:32:11.049 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.040+0000 7fc62a836740 1 -- 172.21.15.100:0/3412152716 wait complete. 2024-05-10T11:32:11.049 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.040+0000 7fc62a836740 1 -- start start 2024-05-10T11:32:11.049 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.041+0000 7fc62a836740 1 Processor -- start 2024-05-10T11:32:11.050 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.041+0000 7fc62a836740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9a833cdb0 0x55f9a834be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:11.050 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.041+0000 7fc62a836740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f9a833d7a0 0x55f9a834c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:11.050 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.041+0000 7fc62a836740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f9a8342910 0x55f9a834d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:11.050 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.041+0000 7fc62a836740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc6201abb10 con 0x55f9a833cdb0 2024-05-10T11:32:11.050 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.041+0000 7fc62a836740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc6201acf80 con 0x55f9a833d7a0 2024-05-10T11:32:11.051 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.041+0000 7fc62a836740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc6201ac1e0 con 0x55f9a8342910 2024-05-10T11:32:11.051 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.041+0000 7fc62a836740 10 client.0 _ll_register_callbacks cb 0x7fc61c0e8240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:32:11.051 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.041+0000 7fc6258b5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9a833cdb0 0x55f9a834be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:11.051 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.041+0000 7fc6260b6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f9a8342910 0x55f9a834d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:11.051 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.041+0000 7fc6258b5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9a833cdb0 0x55f9a834be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36360/0 (socket says 172.21.15.100:36360) 2024-05-10T11:32:11.051 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.041+0000 7fc6258b5640 1 -- 172.21.15.100:0/2185618408 learned_addr learned my addr 172.21.15.100:0/2185618408 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:32:11.051 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.041+0000 7fc6250b4640 1 --2- 172.21.15.100:0/2185618408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f9a833d7a0 0x55f9a834c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:11.051 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.041+0000 7fc6258b5640 1 -- 172.21.15.100:0/2185618408 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f9a8342910 msgr2=0x55f9a834d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:11.051 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.042+0000 7fc6258b5640 1 --2- 172.21.15.100:0/2185618408 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f9a8342910 0x55f9a834d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:11.051 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.042+0000 7fc6258b5640 1 -- 172.21.15.100:0/2185618408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f9a833d7a0 msgr2=0x55f9a834c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:11.051 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.042+0000 7fc6258b5640 1 --2- 172.21.15.100:0/2185618408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f9a833d7a0 0x55f9a834c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:11.051 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.042+0000 7fc6258b5640 1 -- 172.21.15.100:0/2185618408 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9a834d950 con 0x55f9a833cdb0 2024-05-10T11:32:11.052 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.042+0000 7fc6260b6640 1 --2- 172.21.15.100:0/2185618408 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f9a8342910 0x55f9a834d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:32:11.052 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.042+0000 7fc6258b5640 1 --2- 172.21.15.100:0/2185618408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9a833cdb0 0x55f9a834be10 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fc61c06f600 tx=0x7fc61c0b2a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6f6b60d627cf564 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:32:11.052 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.042+0000 7fc6097fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:32:11.052 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.042+0000 7fc6097fa640 1 -- 172.21.15.100:0/2185618408 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61c0a9250 con 0x55f9a833cdb0 2024-05-10T11:32:11.052 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.042+0000 7fc6097fa640 1 -- 172.21.15.100:0/2185618408 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fc61c0a60d0 con 0x55f9a833cdb0 2024-05-10T11:32:11.052 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.042+0000 7fc62a836740 1 -- 172.21.15.100:0/2185618408 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc618003e00 con 0x55f9a833cdb0 2024-05-10T11:32:11.052 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.042+0000 7fc6097fa640 1 -- 172.21.15.100:0/2185618408 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc61c0a95f0 con 0x55f9a833cdb0 2024-05-10T11:32:11.052 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.042+0000 7fc62a836740 10 client.18213 fetch_fsmap learned FSMap version 35 2024-05-10T11:32:11.052 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.042+0000 7fc62a836740 1 -- 172.21.15.100:0/2185618408 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f9a83c6070 con 0x55f9a833cdb0 2024-05-10T11:32:11.052 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.043+0000 7fc6097fa640 1 -- 172.21.15.100:0/2185618408 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc618003e00 con 0x55f9a833cdb0 2024-05-10T11:32:11.052 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.043+0000 7fc6097fa640 1 -- 172.21.15.100:0/2185618408 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7fc61c0a87b0 con 0x55f9a833cdb0 2024-05-10T11:32:11.052 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.043+0000 7fc62a836740 10 client.18213 fetch_fsmap finished waiting for FSMap version 35 2024-05-10T11:32:11.052 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.043+0000 7fc62a836740 10 client.18213 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:32:11.052 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.043+0000 7fc62a836740 20 client.18213 populate_metadata read hostname 'smithi100' 2024-05-10T11:32:11.052 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.043+0000 7fc62a836740 1 --2- 172.21.15.100:0/2185618408 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55f9a83c6a40 0x55f9a83c8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:11.053 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.043+0000 7fc62a836740 4 client.18213 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:32:11.053 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.043+0000 7fc62a836740 1 -- 172.21.15.100:0/2185618408 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f9a82a9050 con 0x55f9a83c6a40 2024-05-10T11:32:11.053 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.043+0000 7fc6250b4640 1 --2- 172.21.15.100:0/2185618408 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55f9a83c6a40 0x55f9a83c8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:11.053 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.043+0000 7fc6097fa640 10 client.18213 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:32:11.053 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.044+0000 7fc6250b4640 1 --2- 172.21.15.100:0/2185618408 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55f9a83c6a40 0x55f9a83c8e60 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b78c884bf861136f server_cookie=db6c4cf858b4dae5 in_seq=0 out_seq=0 2024-05-10T11:32:11.463 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.461+0000 7fc6097fa640 1 -- 172.21.15.100:0/2185618408 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f9a82a9050 con 0x55f9a83c6a40 2024-05-10T11:32:11.463 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.461+0000 7fc6097fa640 10 client.18213 handle_command_reply: tid=0 2024-05-10T11:32:11.463 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.461+0000 7fc62a836740 1 client.18213 shutdown 2024-05-10T11:32:11.463 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.461+0000 7fc62a836740 1 -- 172.21.15.100:0/2185618408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9a833cdb0 msgr2=0x55f9a834be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:11.463 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.461+0000 7fc62a836740 1 --2- 172.21.15.100:0/2185618408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9a833cdb0 0x55f9a834be10 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fc61c06f600 tx=0x7fc61c0b2a90 comp rx=0 tx=0).stop 2024-05-10T11:32:11.463 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.461+0000 7fc62a836740 1 -- 172.21.15.100:0/2185618408 shutdown_connections 2024-05-10T11:32:11.463 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.461+0000 7fc62a836740 1 --2- 172.21.15.100:0/2185618408 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55f9a83c6a40 0x55f9a83c8e60 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:11.463 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.461+0000 7fc62a836740 1 --2- 172.21.15.100:0/2185618408 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f9a8342910 0x55f9a834d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:11.463 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.461+0000 7fc62a836740 1 --2- 172.21.15.100:0/2185618408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f9a833d7a0 0x55f9a834c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:11.463 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.461+0000 7fc62a836740 1 --2- 172.21.15.100:0/2185618408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9a833cdb0 0x55f9a834be10 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:11.463 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.461+0000 7fc62a836740 1 -- 172.21.15.100:0/2185618408 >> 172.21.15.100:0/2185618408 conn(0x55f9a8338420 msgr2=0x55f9a833b330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:32:11.463 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.461+0000 7fc6097fa640 0 client.18213 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:32:11.463 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.461+0000 7fc62a836740 1 -- 172.21.15.100:0/2185618408 shutdown_connections 2024-05-10T11:32:11.464 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.461+0000 7fc62a836740 1 -- 172.21.15.100:0/2185618408 wait complete. 2024-05-10T11:32:11.464 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.461+0000 7fc62a836740 20 client.18213 trim_cache size 0 max 16384 2024-05-10T11:32:11.464 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:32:11.464 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:32:11.464 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:32:11.464 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:32:11.465 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.462+0000 7fc627b19640 1 -- 172.21.15.100:0/3109085859 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc60c0b28a0 msgr2=0x7fc60c0b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:11.465 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.462+0000 7fc627b19640 1 --2- 172.21.15.100:0/3109085859 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc60c0b28a0 0x7fc60c0b4d60 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x55f9a833e000 tx=0x7fc618009500 comp rx=0 tx=0).stop 2024-05-10T11:32:11.465 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.462+0000 7fc627b19640 1 -- 172.21.15.100:0/3109085859 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc620101910 msgr2=0x7fc620198b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:11.465 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.462+0000 7fc627b19640 1 --2- 172.21.15.100:0/3109085859 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc620101910 0x7fc620198b20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc61c00e9c0 tx=0x7fc61c00edc0 comp rx=0 tx=0).stop 2024-05-10T11:32:11.465 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.462+0000 7fc627b19640 1 -- 172.21.15.100:0/3109085859 shutdown_connections 2024-05-10T11:32:11.465 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.462+0000 7fc627b19640 1 --2- 172.21.15.100:0/3109085859 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc60c0b28a0 0x7fc60c0b4d60 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:11.465 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.462+0000 7fc627b19640 1 --2- 172.21.15.100:0/3109085859 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6201034c0 0x7fc6201a00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:11.465 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.462+0000 7fc627b19640 1 --2- 172.21.15.100:0/3109085859 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc620102b00 0x7fc620199060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:11.466 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.462+0000 7fc627b19640 1 --2- 172.21.15.100:0/3109085859 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc620101910 0x7fc620198b20 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:11.466 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.462+0000 7fc627b19640 1 -- 172.21.15.100:0/3109085859 >> 172.21.15.100:0/3109085859 conn(0x7fc6200fce80 msgr2=0x7fc62006a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:32:11.466 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.462+0000 7fc627b19640 1 -- 172.21.15.100:0/3109085859 shutdown_connections 2024-05-10T11:32:11.466 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:11.462+0000 7fc627b19640 1 -- 172.21.15.100:0/3109085859 wait complete. 2024-05-10T11:32:11.906 INFO:tasks.workunit.client.0.smithi100.stdout: 10 922 43.51 MB/sec warmup 5 sec latency 373.568 ms 2024-05-10T11:32:12.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:32:11 smithi178 ceph-mon[33359]: pgmap v835: 129 pgs: 129 active+clean; 217 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 8.1 MiB/s wr, 174 op/s 2024-05-10T11:32:12.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:32:11 smithi178 ceph-mon[33359]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T11:32:12.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:32:11 smithi188 ceph-mon[34075]: pgmap v835: 129 pgs: 129 active+clean; 217 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 8.1 MiB/s wr, 174 op/s 2024-05-10T11:32:12.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:32:11 smithi188 ceph-mon[34075]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T11:32:12.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:32:11 smithi100 ceph-mon[30296]: pgmap v835: 129 pgs: 129 active+clean; 217 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 8.1 MiB/s wr, 174 op/s 2024-05-10T11:32:12.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:32:11 smithi100 ceph-mon[30296]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T11:32:12.906 INFO:tasks.workunit.client.0.smithi100.stdout: 10 1039 37.97 MB/sec warmup 6 sec latency 156.149 ms 2024-05-10T11:32:13.906 INFO:tasks.workunit.client.0.smithi100.stdout: 10 1150 32.86 MB/sec warmup 7 sec latency 220.907 ms 2024-05-10T11:32:14.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:32:13 smithi178 ceph-mon[33359]: pgmap v836: 129 pgs: 129 active+clean; 304 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 16 MiB/s wr, 232 op/s 2024-05-10T11:32:14.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:32:13 smithi188 ceph-mon[34075]: pgmap v836: 129 pgs: 129 active+clean; 304 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 16 MiB/s wr, 232 op/s 2024-05-10T11:32:14.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:32:13 smithi100 ceph-mon[30296]: pgmap v836: 129 pgs: 129 active+clean; 304 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 16 MiB/s wr, 232 op/s 2024-05-10T11:32:14.906 INFO:tasks.workunit.client.0.smithi100.stdout: 10 1256 28.92 MB/sec warmup 8 sec latency 161.019 ms 2024-05-10T11:32:15.906 INFO:tasks.workunit.client.0.smithi100.stdout: 10 1481 28.64 MB/sec warmup 9 sec latency 78.311 ms 2024-05-10T11:32:16.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:32:15 smithi178 ceph-mon[33359]: pgmap v837: 129 pgs: 129 active+clean; 327 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 18 MiB/s wr, 323 op/s 2024-05-10T11:32:16.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:32:15 smithi178 ceph-mon[33359]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T11:32:16.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:32:15 smithi188 ceph-mon[34075]: pgmap v837: 129 pgs: 129 active+clean; 327 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 18 MiB/s wr, 323 op/s 2024-05-10T11:32:16.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:32:15 smithi188 ceph-mon[34075]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T11:32:16.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:32:15 smithi100 ceph-mon[30296]: pgmap v837: 129 pgs: 129 active+clean; 327 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 18 MiB/s wr, 323 op/s 2024-05-10T11:32:16.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:32:15 smithi100 ceph-mon[30296]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T11:32:16.906 INFO:tasks.workunit.client.0.smithi100.stdout: 10 1663 26.20 MB/sec warmup 10 sec latency 80.126 ms 2024-05-10T11:32:17.906 INFO:tasks.workunit.client.0.smithi100.stdout: 10 1864 24.00 MB/sec warmup 11 sec latency 86.713 ms 2024-05-10T11:32:18.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:32:17 smithi178 ceph-mon[33359]: pgmap v838: 129 pgs: 129 active+clean; 322 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 19 MiB/s wr, 442 op/s 2024-05-10T11:32:18.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:32:17 smithi188 ceph-mon[34075]: pgmap v838: 129 pgs: 129 active+clean; 322 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 19 MiB/s wr, 442 op/s 2024-05-10T11:32:18.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:32:17 smithi100 ceph-mon[30296]: pgmap v838: 129 pgs: 129 active+clean; 322 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 19 MiB/s wr, 442 op/s 2024-05-10T11:32:18.906 INFO:tasks.workunit.client.0.smithi100.stdout: 10 2055 22.38 MB/sec warmup 12 sec latency 67.025 ms 2024-05-10T11:32:19.907 INFO:tasks.workunit.client.0.smithi100.stdout: 10 2221 21.01 MB/sec warmup 13 sec latency 119.464 ms 2024-05-10T11:32:20.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:32:19 smithi178 ceph-mon[33359]: pgmap v839: 129 pgs: 129 active+clean; 322 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 19 MiB/s wr, 417 op/s 2024-05-10T11:32:20.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:32:19 smithi188 ceph-mon[34075]: pgmap v839: 129 pgs: 129 active+clean; 322 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 19 MiB/s wr, 417 op/s 2024-05-10T11:32:20.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:32:19 smithi100 ceph-mon[30296]: pgmap v839: 129 pgs: 129 active+clean; 322 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 19 MiB/s wr, 417 op/s 2024-05-10T11:32:20.906 INFO:tasks.workunit.client.0.smithi100.stdout: 10 2467 20.72 MB/sec warmup 14 sec latency 66.722 ms 2024-05-10T11:32:21.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:32:20 smithi178 ceph-mon[33359]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T11:32:21.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:32:20 smithi188 ceph-mon[34075]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T11:32:21.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:32:20 smithi100 ceph-mon[30296]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T11:32:21.906 INFO:tasks.workunit.client.0.smithi100.stdout: 10 2683 20.19 MB/sec warmup 15 sec latency 101.239 ms 2024-05-10T11:32:22.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:32:21 smithi188 ceph-mon[34075]: pgmap v840: 129 pgs: 129 active+clean; 325 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 18 MiB/s wr, 461 op/s 2024-05-10T11:32:22.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:32:21 smithi100 ceph-mon[30296]: pgmap v840: 129 pgs: 129 active+clean; 325 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 18 MiB/s wr, 461 op/s 2024-05-10T11:32:22.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:32:21 smithi178 ceph-mon[33359]: pgmap v840: 129 pgs: 129 active+clean; 325 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 18 MiB/s wr, 461 op/s 2024-05-10T11:32:22.906 INFO:tasks.workunit.client.0.smithi100.stdout: 10 2861 19.75 MB/sec warmup 16 sec latency 96.027 ms 2024-05-10T11:32:23.906 INFO:tasks.workunit.client.0.smithi100.stdout: 10 3151 20.52 MB/sec warmup 17 sec latency 55.304 ms 2024-05-10T11:32:24.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:32:23 smithi188 ceph-mon[34075]: pgmap v841: 129 pgs: 129 active+clean; 327 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 12 MiB/s wr, 444 op/s 2024-05-10T11:32:24.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:32:23 smithi100 ceph-mon[30296]: pgmap v841: 129 pgs: 129 active+clean; 327 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 12 MiB/s wr, 444 op/s 2024-05-10T11:32:24.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:32:23 smithi178 ceph-mon[33359]: pgmap v841: 129 pgs: 129 active+clean; 327 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 12 MiB/s wr, 444 op/s 2024-05-10T11:32:24.588 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.587+0000 7f637a893640 1 -- 172.21.15.100:0/2508993191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f637410d3f0 msgr2=0x7f637410f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:24.588 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.587+0000 7f637a893640 1 --2- 172.21.15.100:0/2508993191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f637410d3f0 0x7f637410f7e0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f6368009900 tx=0x7f636802a3f0 comp rx=0 tx=0).stop 2024-05-10T11:32:24.588 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.587+0000 7f637a893640 1 -- 172.21.15.100:0/2508993191 shutdown_connections 2024-05-10T11:32:24.588 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.587+0000 7f637a893640 1 --2- 172.21.15.100:0/2508993191 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f637410fd70 0x7f6374112230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:24.588 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.587+0000 7f637a893640 1 --2- 172.21.15.100:0/2508993191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f637410d3f0 0x7f637410f7e0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:24.588 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.587+0000 7f637a893640 1 --2- 172.21.15.100:0/2508993191 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f637410aac0 0x7f637410ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:24.588 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.587+0000 7f637a893640 1 -- 172.21.15.100:0/2508993191 >> 172.21.15.100:0/2508993191 conn(0x7f637406fbe0 msgr2=0x7f6374072020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:32:24.588 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.587+0000 7f637a893640 1 -- 172.21.15.100:0/2508993191 shutdown_connections 2024-05-10T11:32:24.588 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.588+0000 7f637a893640 1 -- 172.21.15.100:0/2508993191 wait complete. 2024-05-10T11:32:24.588 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.588+0000 7f637a893640 1 Processor -- start 2024-05-10T11:32:24.590 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.588+0000 7f637a893640 1 -- start start 2024-05-10T11:32:24.590 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.589+0000 7f637a893640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f637410aac0 0x7f63741a12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:24.590 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.589+0000 7f637a893640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f637410d3f0 0x7f63741a1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:24.590 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.589+0000 7f637a893640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f637410fd70 0x7f63741a8890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:24.590 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.589+0000 7f637a893640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6374115af0 con 0x7f637410fd70 2024-05-10T11:32:24.590 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.589+0000 7f637a893640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6374115970 con 0x7f637410d3f0 2024-05-10T11:32:24.590 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.589+0000 7f637a893640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6374115c70 con 0x7f637410aac0 2024-05-10T11:32:24.590 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.589+0000 7f6373fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f637410aac0 0x7f63741a12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:24.590 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.589+0000 7f6378e30640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f637410fd70 0x7f63741a8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:24.590 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.589+0000 7f6373fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f637410aac0 0x7f63741a12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:40874/0 (socket says 172.21.15.100:40874) 2024-05-10T11:32:24.591 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.589+0000 7f63737fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f637410d3f0 0x7f63741a1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:24.591 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.589+0000 7f6373fff640 1 -- 172.21.15.100:0/3062004519 learned_addr learned my addr 172.21.15.100:0/3062004519 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:32:24.591 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.589+0000 7f63737fe640 1 -- 172.21.15.100:0/3062004519 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f637410aac0 msgr2=0x7f63741a12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:24.591 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.589+0000 7f63737fe640 1 --2- 172.21.15.100:0/3062004519 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f637410aac0 0x7f63741a12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:24.591 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.589+0000 7f63737fe640 1 -- 172.21.15.100:0/3062004519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f637410fd70 msgr2=0x7f63741a8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:24.591 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.589+0000 7f63737fe640 1 --2- 172.21.15.100:0/3062004519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f637410fd70 0x7f63741a8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:24.591 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.589+0000 7f63737fe640 1 -- 172.21.15.100:0/3062004519 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63741a8f00 con 0x7f637410d3f0 2024-05-10T11:32:24.591 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.589+0000 7f6378e30640 1 --2- 172.21.15.100:0/3062004519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f637410fd70 0x7f63741a8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:32:24.591 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.590+0000 7f63737fe640 1 --2- 172.21.15.100:0/3062004519 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f637410d3f0 0x7f63741a1820 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6368005ab0 tx=0x7f63680042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:32:24.591 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.590+0000 7f63717fa640 1 -- 172.21.15.100:0/3062004519 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636803f070 con 0x7f637410d3f0 2024-05-10T11:32:24.591 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.590+0000 7f637a893640 1 -- 172.21.15.100:0/3062004519 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f637410aea0 con 0x7f637410d3f0 2024-05-10T11:32:24.591 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.590+0000 7f637a893640 1 -- 172.21.15.100:0/3062004519 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f637410b3e0 con 0x7f637410d3f0 2024-05-10T11:32:24.591 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.590+0000 7f63717fa640 1 -- 172.21.15.100:0/3062004519 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6368032e10 con 0x7f637410d3f0 2024-05-10T11:32:24.591 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.590+0000 7f63717fa640 1 -- 172.21.15.100:0/3062004519 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6368035070 con 0x7f637410d3f0 2024-05-10T11:32:24.594 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.593+0000 7f637d5b2740 1 Processor -- start 2024-05-10T11:32:24.595 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.593+0000 7f637d5b2740 1 -- start start 2024-05-10T11:32:24.595 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.593+0000 7f63717fa640 1 -- 172.21.15.100:0/3062004519 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f6368003a00 con 0x7f637410d3f0 2024-05-10T11:32:24.595 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.593+0000 7f637d5b2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56242bf28db0 0x56242bf291d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:24.595 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.593+0000 7f637d5b2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56242bf297a0 0x56242bf2e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:24.595 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.593+0000 7f637d5b2740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56242bf2e910 0x56242bf30da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:24.595 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.593+0000 7f637d5b2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56242be8d5d0 con 0x56242bf297a0 2024-05-10T11:32:24.595 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.593+0000 7f637d5b2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56242be46890 con 0x56242bf28db0 2024-05-10T11:32:24.595 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.593+0000 7f637d5b2740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56242be89590 con 0x56242bf2e910 2024-05-10T11:32:24.595 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.594+0000 7f6378e30640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56242bf28db0 0x56242bf291d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:24.595 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.594+0000 7f6373fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56242bf297a0 0x56242bf2e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:24.595 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.594+0000 7f6378e30640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56242bf28db0 0x56242bf291d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:46794/0 (socket says 172.21.15.100:46794) 2024-05-10T11:32:24.596 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.594+0000 7f63737fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56242bf2e910 0x56242bf30da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:24.596 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.594+0000 7f6378e30640 1 -- 172.21.15.100:0/2104603425 learned_addr learned my addr 172.21.15.100:0/2104603425 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:32:24.596 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.594+0000 7f6373fff640 1 -- 172.21.15.100:0/2104603425 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56242bf2e910 msgr2=0x56242bf30da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:24.596 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.594+0000 7f6373fff640 1 --2- 172.21.15.100:0/2104603425 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56242bf2e910 0x56242bf30da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:24.596 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.594+0000 7f6373fff640 1 -- 172.21.15.100:0/2104603425 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56242bf28db0 msgr2=0x56242bf291d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:24.596 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.594+0000 7f6373fff640 1 --2- 172.21.15.100:0/2104603425 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56242bf28db0 0x56242bf291d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:24.597 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.594+0000 7f6373fff640 1 -- 172.21.15.100:0/2104603425 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56242bb9f210 con 0x56242bf297a0 2024-05-10T11:32:24.597 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.594+0000 7f6378e30640 1 --2- 172.21.15.100:0/2104603425 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56242bf28db0 0x56242bf291d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:32:24.597 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.594+0000 7f6373fff640 1 --2- 172.21.15.100:0/2104603425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56242bf297a0 0x56242bf2e2a0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f6364009c60 tx=0x7f6364011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bd8a2ed68010c8e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:32:24.597 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.594+0000 7f63717fa640 1 --2- 172.21.15.100:0/3062004519 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f635c0b28a0 0x7f635c0b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:24.597 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.595+0000 7f635a7fc640 1 -- 172.21.15.100:0/2104603425 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6364005220 con 0x56242bf297a0 2024-05-10T11:32:24.597 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.595+0000 7f6378e30640 1 --2- 172.21.15.100:0/3062004519 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f635c0b28a0 0x7f635c0b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:24.597 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.595+0000 7f635a7fc640 1 -- 172.21.15.100:0/2104603425 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6364004c30 con 0x56242bf297a0 2024-05-10T11:32:24.597 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.595+0000 7f635a7fc640 1 -- 172.21.15.100:0/2104603425 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6364019690 con 0x56242bf297a0 2024-05-10T11:32:24.597 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.595+0000 7f63717fa640 1 -- 172.21.15.100:0/3062004519 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f63681027c0 con 0x7f637410d3f0 2024-05-10T11:32:24.601 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.596+0000 7f637d5b2740 1 -- 172.21.15.100:0/2104603425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56242bf297a0 msgr2=0x56242bf2e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:24.601 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.596+0000 7f6378e30640 1 --2- 172.21.15.100:0/3062004519 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f635c0b28a0 0x7f635c0b4d60 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x56242bf2a000 tx=0x7f636c012040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:32:24.601 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.596+0000 7f637d5b2740 1 --2- 172.21.15.100:0/2104603425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56242bf297a0 0x56242bf2e2a0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f6364009c60 tx=0x7f6364011810 comp rx=0 tx=0).stop 2024-05-10T11:32:24.601 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.596+0000 7f637d5b2740 1 -- 172.21.15.100:0/2104603425 shutdown_connections 2024-05-10T11:32:24.601 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.596+0000 7f637d5b2740 1 --2- 172.21.15.100:0/2104603425 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56242bf2e910 0x56242bf30da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:24.602 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.596+0000 7f637d5b2740 1 --2- 172.21.15.100:0/2104603425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56242bf297a0 0x56242bf2e2a0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:24.602 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.596+0000 7f637d5b2740 1 --2- 172.21.15.100:0/2104603425 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56242bf28db0 0x56242bf291d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:24.602 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.596+0000 7f637d5b2740 1 -- 172.21.15.100:0/2104603425 >> 172.21.15.100:0/2104603425 conn(0x56242bf24420 msgr2=0x56242bf24820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:32:24.602 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.596+0000 7f637d5b2740 1 -- 172.21.15.100:0/2104603425 shutdown_connections 2024-05-10T11:32:24.602 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.596+0000 7f637d5b2740 1 -- 172.21.15.100:0/2104603425 wait complete. 2024-05-10T11:32:24.602 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.596+0000 7f637d5b2740 1 -- start start 2024-05-10T11:32:24.602 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.596+0000 7f637d5b2740 1 Processor -- start 2024-05-10T11:32:24.602 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.597+0000 7f637d5b2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56242bf28db0 0x56242bf33020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:24.602 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.597+0000 7f637d5b2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56242bf297a0 0x56242bf33560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:24.602 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.597+0000 7f637d5b2740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56242bf2e910 0x56242bf3a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:24.602 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.597+0000 7f637d5b2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56242bf318c0 con 0x56242bf297a0 2024-05-10T11:32:24.602 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.597+0000 7f637d5b2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56242be4a370 con 0x56242bf28db0 2024-05-10T11:32:24.602 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.597+0000 7f637d5b2740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56242bf31a40 con 0x56242bf2e910 2024-05-10T11:32:24.602 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.597+0000 7f637d5b2740 10 client.0 _ll_register_callbacks cb 0x56242bb9ea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:32:24.603 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.597+0000 7f63737fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56242bf297a0 0x56242bf33560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:24.603 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.597+0000 7f6373fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56242bf28db0 0x56242bf33020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:24.603 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.597+0000 7f6378e30640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56242bf2e910 0x56242bf3a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:24.603 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.597+0000 7f6378e30640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56242bf2e910 0x56242bf3a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:40898/0 (socket says 172.21.15.100:40898) 2024-05-10T11:32:24.603 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.597+0000 7f6378e30640 1 -- 172.21.15.100:0/3282825632 learned_addr learned my addr 172.21.15.100:0/3282825632 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:32:24.603 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.597+0000 7f6373fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56242bf28db0 0x56242bf33020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:46806/0 (socket says 172.21.15.100:46806) 2024-05-10T11:32:24.603 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.597+0000 7f6373fff640 1 -- 172.21.15.100:0/3282825632 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56242bf2e910 msgr2=0x56242bf3a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:24.603 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.597+0000 7f6373fff640 1 --2- 172.21.15.100:0/3282825632 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56242bf2e910 0x56242bf3a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:24.603 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.597+0000 7f6373fff640 1 -- 172.21.15.100:0/3282825632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56242bf297a0 msgr2=0x56242bf33560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:24.603 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.597+0000 7f6373fff640 1 --2- 172.21.15.100:0/3282825632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56242bf297a0 0x56242bf33560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:24.603 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.597+0000 7f6373fff640 1 -- 172.21.15.100:0/3282825632 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56242bc6b620 con 0x56242bf28db0 2024-05-10T11:32:24.603 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.597+0000 7f6373fff640 1 --2- 172.21.15.100:0/3282825632 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56242bf28db0 0x56242bf33020 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x56242bf252f0 tx=0x7f63640048f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc642745b5120e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:32:24.603 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.597+0000 7f63537fe640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:32:24.603 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.598+0000 7f63537fe640 1 -- 172.21.15.100:0/3282825632 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6364029070 con 0x56242bf28db0 2024-05-10T11:32:24.603 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.598+0000 7f63537fe640 1 -- 172.21.15.100:0/3282825632 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f6364023630 con 0x56242bf28db0 2024-05-10T11:32:24.604 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.598+0000 7f637d5b2740 1 -- 172.21.15.100:0/3282825632 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6364004c30 con 0x56242bf28db0 2024-05-10T11:32:24.604 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.598+0000 7f63537fe640 1 -- 172.21.15.100:0/3282825632 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f6364035050 con 0x56242bf28db0 2024-05-10T11:32:24.604 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.598+0000 7f637d5b2740 10 client.27569 fetch_fsmap learned FSMap version 37 2024-05-10T11:32:24.604 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.598+0000 7f637d5b2740 1 -- 172.21.15.100:0/3282825632 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56242bf3d580 con 0x56242bf28db0 2024-05-10T11:32:24.604 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.598+0000 7f63537fe640 1 -- 172.21.15.100:0/3282825632 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6364004c30 con 0x56242bf28db0 2024-05-10T11:32:24.604 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.598+0000 7f63537fe640 1 -- 172.21.15.100:0/3282825632 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f6364019430 con 0x56242bf28db0 2024-05-10T11:32:24.604 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.598+0000 7f637d5b2740 10 client.27569 fetch_fsmap finished waiting for FSMap version 37 2024-05-10T11:32:24.604 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.598+0000 7f637d5b2740 10 client.27569 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:32:24.604 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.598+0000 7f637d5b2740 20 client.27569 populate_metadata read hostname 'smithi100' 2024-05-10T11:32:24.604 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.598+0000 7f637d5b2740 1 --2- 172.21.15.100:0/3282825632 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56242bf3e9f0 0x56242bf40e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:24.604 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.598+0000 7f637d5b2740 4 client.27569 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:32:24.604 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.598+0000 7f637d5b2740 1 -- 172.21.15.100:0/3282825632 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56242be9b1e0 con 0x56242bf3e9f0 2024-05-10T11:32:24.604 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.598+0000 7f63737fe640 1 --2- 172.21.15.100:0/3282825632 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56242bf3e9f0 0x56242bf40e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:24.604 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.599+0000 7f63537fe640 10 client.27569 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:32:24.604 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.599+0000 7f63737fe640 1 --2- 172.21.15.100:0/3282825632 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56242bf3e9f0 0x56242bf40e20 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ee35c2da5e64bc3 server_cookie=c18a2bbe265635ef in_seq=0 out_seq=0 2024-05-10T11:32:24.605 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.599+0000 7f63537fe640 1 -- 172.21.15.100:0/3282825632 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56242be9b1e0 con 0x56242bf3e9f0 2024-05-10T11:32:24.605 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.599+0000 7f63537fe640 10 client.27569 handle_command_reply: tid=0 2024-05-10T11:32:24.605 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.600+0000 7f637d5b2740 1 client.27569 shutdown 2024-05-10T11:32:24.605 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.600+0000 7f637d5b2740 1 -- 172.21.15.100:0/3282825632 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56242bf28db0 msgr2=0x56242bf33020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:24.605 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.600+0000 7f637d5b2740 1 --2- 172.21.15.100:0/3282825632 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56242bf28db0 0x56242bf33020 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x56242bf252f0 tx=0x7f63640048f0 comp rx=0 tx=0).stop 2024-05-10T11:32:24.605 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.600+0000 7f637d5b2740 1 -- 172.21.15.100:0/3282825632 shutdown_connections 2024-05-10T11:32:24.605 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.600+0000 7f637d5b2740 1 --2- 172.21.15.100:0/3282825632 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56242bf3e9f0 0x56242bf40e20 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:24.605 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.600+0000 7f637d5b2740 1 --2- 172.21.15.100:0/3282825632 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56242bf2e910 0x56242bf3a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:24.605 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.600+0000 7f637d5b2740 1 --2- 172.21.15.100:0/3282825632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56242bf297a0 0x56242bf33560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:24.605 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.600+0000 7f63537fe640 0 client.27569 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:32:24.605 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.600+0000 7f637d5b2740 1 --2- 172.21.15.100:0/3282825632 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56242bf28db0 0x56242bf33020 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:24.605 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.600+0000 7f637d5b2740 1 -- 172.21.15.100:0/3282825632 >> 172.21.15.100:0/3282825632 conn(0x56242bf24420 msgr2=0x56242bf23580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:32:24.605 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.600+0000 7f637d5b2740 1 -- 172.21.15.100:0/3282825632 shutdown_connections 2024-05-10T11:32:24.605 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.600+0000 7f637d5b2740 1 -- 172.21.15.100:0/3282825632 wait complete. 2024-05-10T11:32:24.605 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.600+0000 7f637d5b2740 20 client.27569 trim_cache size 0 max 16384 2024-05-10T11:32:24.627 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.625+0000 7f637d5b2740 1 Processor -- start 2024-05-10T11:32:24.627 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.625+0000 7f637d5b2740 1 -- start start 2024-05-10T11:32:24.627 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.625+0000 7f637d5b2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56242bf28db0 0x56242bf27330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:24.627 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.625+0000 7f637d5b2740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56242bf297a0 0x56242bf27870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:24.627 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.625+0000 7f637d5b2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56242bf2e910 0x56242bf32640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:24.627 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.625+0000 7f637d5b2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f63741b4220 con 0x56242bf2e910 2024-05-10T11:32:24.627 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.625+0000 7f637d5b2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f63741b1a60 con 0x56242bf28db0 2024-05-10T11:32:24.628 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.625+0000 7f637d5b2740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f63741af8b0 con 0x56242bf297a0 2024-05-10T11:32:24.628 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.625+0000 7f63737fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56242bf297a0 0x56242bf27870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:24.628 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.625+0000 7f6373fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56242bf28db0 0x56242bf27330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:24.628 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.625+0000 7f63737fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56242bf297a0 0x56242bf27870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:40904/0 (socket says 172.21.15.100:40904) 2024-05-10T11:32:24.628 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.625+0000 7f6378e30640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56242bf2e910 0x56242bf32640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:24.628 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.625+0000 7f63737fe640 1 -- 172.21.15.100:0/93187379 learned_addr learned my addr 172.21.15.100:0/93187379 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:32:24.628 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.625+0000 7f63737fe640 1 -- 172.21.15.100:0/93187379 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56242bf28db0 msgr2=0x56242bf27330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:24.628 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.625+0000 7f63737fe640 1 --2- 172.21.15.100:0/93187379 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56242bf28db0 0x56242bf27330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:24.628 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.625+0000 7f63737fe640 1 -- 172.21.15.100:0/93187379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56242bf2e910 msgr2=0x56242bf32640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:24.628 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.625+0000 7f63737fe640 1 --2- 172.21.15.100:0/93187379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56242bf2e910 0x56242bf32640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:24.628 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.625+0000 7f63737fe640 1 -- 172.21.15.100:0/93187379 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56242bf24250 con 0x56242bf297a0 2024-05-10T11:32:24.628 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.625+0000 7f6378e30640 1 --2- 172.21.15.100:0/93187379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56242bf2e910 0x56242bf32640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:32:24.628 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.626+0000 7f63737fe640 1 --2- 172.21.15.100:0/93187379 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56242bf297a0 0x56242bf27870 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f636803b580 tx=0x7f6368108550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2a8a173af602e75 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:32:24.628 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.626+0000 7f6359ffb640 1 -- 172.21.15.100:0/93187379 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6368048040 con 0x56242bf297a0 2024-05-10T11:32:24.629 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.626+0000 7f6359ffb640 1 -- 172.21.15.100:0/93187379 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f63680d2220 con 0x56242bf297a0 2024-05-10T11:32:24.629 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.626+0000 7f6359ffb640 1 -- 172.21.15.100:0/93187379 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63680ca180 con 0x56242bf297a0 2024-05-10T11:32:24.629 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.627+0000 7f637d5b2740 1 -- 172.21.15.100:0/93187379 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56242bf297a0 msgr2=0x56242bf27870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:24.629 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.627+0000 7f637d5b2740 1 --2- 172.21.15.100:0/93187379 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56242bf297a0 0x56242bf27870 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f636803b580 tx=0x7f6368108550 comp rx=0 tx=0).stop 2024-05-10T11:32:24.629 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.627+0000 7f637d5b2740 1 -- 172.21.15.100:0/93187379 shutdown_connections 2024-05-10T11:32:24.629 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.627+0000 7f637d5b2740 1 --2- 172.21.15.100:0/93187379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56242bf2e910 0x56242bf32640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:24.629 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.627+0000 7f637d5b2740 1 --2- 172.21.15.100:0/93187379 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56242bf297a0 0x56242bf27870 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:24.629 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.627+0000 7f637d5b2740 1 --2- 172.21.15.100:0/93187379 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56242bf28db0 0x56242bf27330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:24.629 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.627+0000 7f637d5b2740 1 -- 172.21.15.100:0/93187379 >> 172.21.15.100:0/93187379 conn(0x56242bf24420 msgr2=0x56242bf24800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:32:24.629 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.627+0000 7f637d5b2740 1 -- 172.21.15.100:0/93187379 shutdown_connections 2024-05-10T11:32:24.629 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.627+0000 7f637d5b2740 1 -- 172.21.15.100:0/93187379 wait complete. 2024-05-10T11:32:24.629 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.627+0000 7f637d5b2740 1 -- start start 2024-05-10T11:32:24.632 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.628+0000 7f637d5b2740 1 Processor -- start 2024-05-10T11:32:24.632 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.628+0000 7f637d5b2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56242bf28db0 0x56242bf36f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:24.632 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.628+0000 7f637d5b2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56242bf297a0 0x56242bf37480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.628+0000 7f637d5b2740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56242bf2e910 0x56242bf38460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.628+0000 7f637d5b2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f63741aff20 con 0x56242bf28db0 2024-05-10T11:32:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.628+0000 7f637d5b2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f63741b1390 con 0x56242bf297a0 2024-05-10T11:32:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.628+0000 7f637d5b2740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f63741b05f0 con 0x56242bf2e910 2024-05-10T11:32:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.628+0000 7f637d5b2740 10 client.0 _ll_register_callbacks cb 0x7f636c009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:32:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.628+0000 7f63737fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56242bf297a0 0x56242bf37480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.628+0000 7f63737fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56242bf297a0 0x56242bf37480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:46820/0 (socket says 172.21.15.100:46820) 2024-05-10T11:32:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.628+0000 7f63737fe640 1 -- 172.21.15.100:0/4199041700 learned_addr learned my addr 172.21.15.100:0/4199041700 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:32:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.628+0000 7f6373fff640 1 --2- 172.21.15.100:0/4199041700 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56242bf28db0 0x56242bf36f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.628+0000 7f6378e30640 1 --2- 172.21.15.100:0/4199041700 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56242bf2e910 0x56242bf38460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.629+0000 7f63737fe640 1 -- 172.21.15.100:0/4199041700 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56242bf2e910 msgr2=0x56242bf38460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.629+0000 7f63737fe640 1 --2- 172.21.15.100:0/4199041700 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56242bf2e910 0x56242bf38460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.629+0000 7f63737fe640 1 -- 172.21.15.100:0/4199041700 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56242bf28db0 msgr2=0x56242bf36f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:24.634 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.629+0000 7f63737fe640 1 --2- 172.21.15.100:0/4199041700 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56242bf28db0 0x56242bf36f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:24.634 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.629+0000 7f63737fe640 1 -- 172.21.15.100:0/4199041700 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56242bf38a50 con 0x56242bf297a0 2024-05-10T11:32:24.634 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.629+0000 7f6378e30640 1 --2- 172.21.15.100:0/4199041700 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56242bf2e910 0x56242bf38460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:32:24.634 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.629+0000 7f63737fe640 1 --2- 172.21.15.100:0/4199041700 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56242bf297a0 0x56242bf37480 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f63681002d0 tx=0x7f63680c9910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9231ee69cf718a25 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:32:24.634 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.629+0000 7f63537fe640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:32:24.634 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.629+0000 7f63537fe640 1 -- 172.21.15.100:0/4199041700 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6368048070 con 0x56242bf297a0 2024-05-10T11:32:24.634 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.629+0000 7f63537fe640 1 -- 172.21.15.100:0/4199041700 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f63680c5d10 con 0x56242bf297a0 2024-05-10T11:32:24.634 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.629+0000 7f637d5b2740 1 -- 172.21.15.100:0/4199041700 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56242be95050 con 0x56242bf297a0 2024-05-10T11:32:24.634 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.629+0000 7f63537fe640 1 -- 172.21.15.100:0/4199041700 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f63680c91f0 con 0x56242bf297a0 2024-05-10T11:32:24.634 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.629+0000 7f637d5b2740 10 client.27575 fetch_fsmap learned FSMap version 37 2024-05-10T11:32:24.634 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.629+0000 7f637d5b2740 1 -- 172.21.15.100:0/4199041700 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56242bf38a50 con 0x56242bf297a0 2024-05-10T11:32:24.634 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.629+0000 7f63537fe640 1 -- 172.21.15.100:0/4199041700 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x56242be95050 con 0x56242bf297a0 2024-05-10T11:32:24.634 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.630+0000 7f63537fe640 1 -- 172.21.15.100:0/4199041700 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f636804b060 con 0x56242bf297a0 2024-05-10T11:32:24.634 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.630+0000 7f637d5b2740 10 client.27575 fetch_fsmap finished waiting for FSMap version 37 2024-05-10T11:32:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.630+0000 7f637d5b2740 10 client.27575 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:32:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.630+0000 7f637d5b2740 20 client.27575 populate_metadata read hostname 'smithi100' 2024-05-10T11:32:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.630+0000 7f637d5b2740 1 --2- 172.21.15.100:0/4199041700 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56242bfb1d50 0x56242bfb4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.630+0000 7f637d5b2740 4 client.27575 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:32:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.630+0000 7f637d5b2740 1 -- 172.21.15.100:0/4199041700 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56242be105f0 con 0x56242bfb1d50 2024-05-10T11:32:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.630+0000 7f6373fff640 1 --2- 172.21.15.100:0/4199041700 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56242bfb1d50 0x56242bfb4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.630+0000 7f63537fe640 10 client.27575 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:32:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.631+0000 7f6373fff640 1 --2- 172.21.15.100:0/4199041700 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56242bfb1d50 0x56242bfb4170 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24cd8d5bd2c157ff server_cookie=9e2875fdec158332 in_seq=0 out_seq=0 2024-05-10T11:32:24.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.775+0000 7f63537fe640 1 -- 172.21.15.100:0/4199041700 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56242be105f0 con 0x56242bfb1d50 2024-05-10T11:32:24.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.775+0000 7f63537fe640 10 client.27575 handle_command_reply: tid=0 2024-05-10T11:32:24.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.775+0000 7f637d5b2740 1 client.27575 shutdown 2024-05-10T11:32:24.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.775+0000 7f637d5b2740 1 -- 172.21.15.100:0/4199041700 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56242bf297a0 msgr2=0x56242bf37480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:24.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.775+0000 7f637d5b2740 1 --2- 172.21.15.100:0/4199041700 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56242bf297a0 0x56242bf37480 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f63681002d0 tx=0x7f63680c9910 comp rx=0 tx=0).stop 2024-05-10T11:32:24.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.775+0000 7f637d5b2740 1 -- 172.21.15.100:0/4199041700 shutdown_connections 2024-05-10T11:32:24.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.775+0000 7f637d5b2740 1 --2- 172.21.15.100:0/4199041700 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56242bfb1d50 0x56242bfb4170 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:24.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.775+0000 7f637d5b2740 1 --2- 172.21.15.100:0/4199041700 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56242bf2e910 0x56242bf38460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:24.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.775+0000 7f637d5b2740 1 --2- 172.21.15.100:0/4199041700 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56242bf297a0 0x56242bf37480 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:24.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.775+0000 7f63537fe640 0 client.27575 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:32:24.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.775+0000 7f637d5b2740 1 --2- 172.21.15.100:0/4199041700 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56242bf28db0 0x56242bf36f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:24.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.775+0000 7f637d5b2740 1 -- 172.21.15.100:0/4199041700 >> 172.21.15.100:0/4199041700 conn(0x56242bf24420 msgr2=0x56242bf27330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:32:24.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.776+0000 7f637d5b2740 1 -- 172.21.15.100:0/4199041700 shutdown_connections 2024-05-10T11:32:24.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.776+0000 7f637d5b2740 1 -- 172.21.15.100:0/4199041700 wait complete. 2024-05-10T11:32:24.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.776+0000 7f637d5b2740 20 client.27575 trim_cache size 0 max 16384 2024-05-10T11:32:24.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:32:24.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:32:24.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:32:24.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:32:24.778 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.776+0000 7f637a893640 1 -- 172.21.15.100:0/3062004519 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f635c0b28a0 msgr2=0x7f635c0b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:24.778 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.776+0000 7f637a893640 1 --2- 172.21.15.100:0/3062004519 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f635c0b28a0 0x7f635c0b4d60 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x56242bf2a000 tx=0x7f636c012040 comp rx=0 tx=0).stop 2024-05-10T11:32:24.778 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.776+0000 7f637a893640 1 -- 172.21.15.100:0/3062004519 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f637410d3f0 msgr2=0x7f63741a1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:24.778 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.776+0000 7f637a893640 1 --2- 172.21.15.100:0/3062004519 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f637410d3f0 0x7f63741a1820 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6368005ab0 tx=0x7f63680042f0 comp rx=0 tx=0).stop 2024-05-10T11:32:24.778 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.777+0000 7f637a893640 1 -- 172.21.15.100:0/3062004519 shutdown_connections 2024-05-10T11:32:24.779 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.777+0000 7f637a893640 1 --2- 172.21.15.100:0/3062004519 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f635c0b28a0 0x7f635c0b4d60 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:24.779 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.777+0000 7f637a893640 1 --2- 172.21.15.100:0/3062004519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f637410fd70 0x7f63741a8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:24.779 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.777+0000 7f637a893640 1 --2- 172.21.15.100:0/3062004519 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f637410d3f0 0x7f63741a1820 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:24.779 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.777+0000 7f637a893640 1 --2- 172.21.15.100:0/3062004519 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f637410aac0 0x7f63741a12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:24.779 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.777+0000 7f637a893640 1 -- 172.21.15.100:0/3062004519 >> 172.21.15.100:0/3062004519 conn(0x7f637406fbe0 msgr2=0x7f6374109ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:32:24.779 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.777+0000 7f637a893640 1 -- 172.21.15.100:0/3062004519 shutdown_connections 2024-05-10T11:32:24.779 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:24.777+0000 7f637a893640 1 -- 172.21.15.100:0/3062004519 wait complete. 2024-05-10T11:32:24.906 INFO:tasks.workunit.client.0.smithi100.stdout: 10 3387 20.10 MB/sec warmup 18 sec latency 160.308 ms 2024-05-10T11:32:25.315 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T11:32:25.671 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:32:25.906 INFO:tasks.workunit.client.0.smithi100.stdout: 10 3521 19.52 MB/sec warmup 19 sec latency 80.471 ms 2024-05-10T11:32:26.014 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:32:25 smithi100 ceph-mon[30296]: pgmap v842: 129 pgs: 129 active+clean; 335 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 5.4 MiB/s wr, 379 op/s 2024-05-10T11:32:26.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:32:25 smithi188 ceph-mon[34075]: pgmap v842: 129 pgs: 129 active+clean; 335 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 5.4 MiB/s wr, 379 op/s 2024-05-10T11:32:26.054 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:32:25 smithi178 ceph-mon[33359]: pgmap v842: 129 pgs: 129 active+clean; 335 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 5.4 MiB/s wr, 379 op/s 2024-05-10T11:32:26.907 INFO:tasks.workunit.client.0.smithi100.stdout: 10 3631 18.78 MB/sec warmup 20 sec latency 102.404 ms 2024-05-10T11:32:27.909 INFO:tasks.workunit.client.0.smithi100.stdout: 10 3816 18.68 MB/sec warmup 21 sec latency 110.203 ms 2024-05-10T11:32:28.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:32:27 smithi188 ceph-mon[34075]: pgmap v843: 129 pgs: 129 active+clean; 347 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 4.3 MiB/s wr, 343 op/s 2024-05-10T11:32:28.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:32:27 smithi100 ceph-mon[30296]: pgmap v843: 129 pgs: 129 active+clean; 347 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 4.3 MiB/s wr, 343 op/s 2024-05-10T11:32:28.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.104+0000 7f161cfd6700 1 -- 172.21.15.100:0/2926815647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1618102d50 msgr2=0x7f161810f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:28.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.104+0000 7f161cfd6700 1 --2- 172.21.15.100:0/2926815647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1618102d50 0x7f161810f240 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f161000b0a0 tx=0x7f161000b3b0 comp rx=0 tx=0).stop 2024-05-10T11:32:28.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.104+0000 7f161cfd6700 1 -- 172.21.15.100:0/2926815647 shutdown_connections 2024-05-10T11:32:28.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.104+0000 7f161cfd6700 1 --2- 172.21.15.100:0/2926815647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1618102d50 0x7f161810f240 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:28.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.104+0000 7f161cfd6700 1 --2- 172.21.15.100:0/2926815647 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1618108400 0x7f16181087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:28.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.104+0000 7f161cfd6700 1 --2- 172.21.15.100:0/2926815647 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1618102390 0x7f1618102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:28.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.104+0000 7f161cfd6700 1 -- 172.21.15.100:0/2926815647 >> 172.21.15.100:0/2926815647 conn(0x7f1618079310 msgr2=0x7f1618079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:32:28.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.104+0000 7f161cfd6700 1 -- 172.21.15.100:0/2926815647 shutdown_connections 2024-05-10T11:32:28.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.104+0000 7f161cfd6700 1 -- 172.21.15.100:0/2926815647 wait complete. 2024-05-10T11:32:28.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.105+0000 7f161cfd6700 1 Processor -- start 2024-05-10T11:32:28.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.105+0000 7f161cfd6700 1 -- start start 2024-05-10T11:32:28.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.105+0000 7f161cfd6700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1618102390 0x7f161819d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:28.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.105+0000 7f161cfd6700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1618102d50 0x7f161819db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:28.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.105+0000 7f161cfd6700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1618108400 0x7f16181976b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:28.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.105+0000 7f161cfd6700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1618111330 con 0x7f1618108400 2024-05-10T11:32:28.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.105+0000 7f161cfd6700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f16181111b0 con 0x7f1618102390 2024-05-10T11:32:28.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.105+0000 7f161cfd6700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f16181114b0 con 0x7f1618102d50 2024-05-10T11:32:28.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.105+0000 7f1616d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1618108400 0x7f16181976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:28.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.105+0000 7f161659c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1618102390 0x7f161819d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:28.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.105+0000 7f1615d9b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1618102d50 0x7f161819db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:28.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.105+0000 7f161659c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1618102390 0x7f161819d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:41436/0 (socket says 172.21.15.100:41436) 2024-05-10T11:32:28.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.105+0000 7f1616d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1618108400 0x7f16181976b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:32854/0 (socket says 172.21.15.100:32854) 2024-05-10T11:32:28.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.105+0000 7f161659c700 1 -- 172.21.15.100:0/2294798775 learned_addr learned my addr 172.21.15.100:0/2294798775 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:32:28.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.106+0000 7f1616d9d700 1 -- 172.21.15.100:0/2294798775 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1618102d50 msgr2=0x7f161819db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:28.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.106+0000 7f1616d9d700 1 --2- 172.21.15.100:0/2294798775 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1618102d50 0x7f161819db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:28.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.106+0000 7f1616d9d700 1 -- 172.21.15.100:0/2294798775 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1618102390 msgr2=0x7f161819d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:28.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.106+0000 7f1616d9d700 1 --2- 172.21.15.100:0/2294798775 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1618102390 0x7f161819d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:28.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.106+0000 7f1616d9d700 1 -- 172.21.15.100:0/2294798775 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1618197f90 con 0x7f1618108400 2024-05-10T11:32:28.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.106+0000 7f161659c700 1 --2- 172.21.15.100:0/2294798775 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1618102390 0x7f161819d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:32:28.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.106+0000 7f1616d9d700 1 --2- 172.21.15.100:0/2294798775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1618108400 0x7f16181976b0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f161000b070 tx=0x7f1610004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:32:28.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.106+0000 7f16077fe700 1 -- 172.21.15.100:0/2294798775 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1610021070 con 0x7f1618108400 2024-05-10T11:32:28.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.107+0000 7f16077fe700 1 -- 172.21.15.100:0/2294798775 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1610007b80 con 0x7f1618108400 2024-05-10T11:32:28.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.107+0000 7f16077fe700 1 -- 172.21.15.100:0/2294798775 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1610005040 con 0x7f1618108400 2024-05-10T11:32:28.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.107+0000 7f161cfd6700 1 -- 172.21.15.100:0/2294798775 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1618198220 con 0x7f1618108400 2024-05-10T11:32:28.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.107+0000 7f161cfd6700 1 -- 172.21.15.100:0/2294798775 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16181a87a0 con 0x7f1618108400 2024-05-10T11:32:28.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.110+0000 7f16077fe700 1 -- 172.21.15.100:0/2294798775 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f1610024070 con 0x7f1618108400 2024-05-10T11:32:28.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.111+0000 7f16077fe700 1 --2- 172.21.15.100:0/2294798775 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f16000b29d0 0x7f16000b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:28.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.112+0000 7f1627869b80 1 Processor -- start 2024-05-10T11:32:28.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.112+0000 7f16077fe700 1 -- 172.21.15.100:0/2294798775 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f16100a6cd0 con 0x7f1618108400 2024-05-10T11:32:28.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.112+0000 7f161659c700 1 --2- 172.21.15.100:0/2294798775 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f16000b29d0 0x7f16000b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:28.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.112+0000 7f1627869b80 1 -- start start 2024-05-10T11:32:28.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.112+0000 7f1627869b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55598bb2f1b0 0x55598bb2f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:28.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.112+0000 7f1627869b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55598bb2fad0 0x55598bb29810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:28.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.112+0000 7f1627869b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55598bb29d50 0x55598bb2a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:28.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.112+0000 7f1627869b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55598ba7b940 con 0x55598bb2fad0 2024-05-10T11:32:28.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.112+0000 7f1627869b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55598b9e0da0 con 0x55598bb2f1b0 2024-05-10T11:32:28.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.112+0000 7f1627869b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55598ba77900 con 0x55598bb29d50 2024-05-10T11:32:28.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.112+0000 7f161659c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55598bb2f1b0 0x55598bb2f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:28.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.112+0000 7f161659c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55598bb2f1b0 0x55598bb2f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:41452/0 (socket says 172.21.15.100:41452) 2024-05-10T11:32:28.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.112+0000 7f161659c700 1 -- 172.21.15.100:0/3910904982 learned_addr learned my addr 172.21.15.100:0/3910904982 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:32:28.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.112+0000 7f1615d9b700 1 --2- 172.21.15.100:0/3910904982 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55598bb2fad0 0x55598bb29810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:28.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.113+0000 7f161659c700 1 --2- 172.21.15.100:0/2294798775 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f16000b29d0 0x7f16000b4e90 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f160800dfd0 tx=0x7f160800f040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:32:28.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.113+0000 7f161659c700 1 -- 172.21.15.100:0/3910904982 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55598bb29d50 msgr2=0x55598bb2a190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T11:32:28.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.113+0000 7f161659c700 1 --2- 172.21.15.100:0/3910904982 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55598bb29d50 0x55598bb2a190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:28.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.113+0000 7f161659c700 1 -- 172.21.15.100:0/3910904982 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55598bb2fad0 msgr2=0x55598bb29810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:28.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.113+0000 7f161659c700 1 --2- 172.21.15.100:0/3910904982 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55598bb2fad0 0x55598bb29810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:28.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.113+0000 7f161659c700 1 -- 172.21.15.100:0/3910904982 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55598b9c88d0 con 0x55598bb2f1b0 2024-05-10T11:32:28.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.113+0000 7f1615d9b700 1 --2- 172.21.15.100:0/3910904982 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55598bb2fad0 0x55598bb29810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:32:28.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.113+0000 7f161659c700 1 --2- 172.21.15.100:0/3910904982 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55598bb2f1b0 0x55598bb2f590 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f160801b5f0 tx=0x7f1608014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e435916660e6f5b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:32:28.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.113+0000 7f1604ff9700 1 -- 172.21.15.100:0/3910904982 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f160801b0d0 con 0x55598bb2f1b0 2024-05-10T11:32:28.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.113+0000 7f1604ff9700 1 -- 172.21.15.100:0/3910904982 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f160801b270 con 0x55598bb2f1b0 2024-05-10T11:32:28.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.113+0000 7f1604ff9700 1 -- 172.21.15.100:0/3910904982 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1608019df0 con 0x55598bb2f1b0 2024-05-10T11:32:28.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.114+0000 7f1627869b80 1 -- 172.21.15.100:0/3910904982 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55598bb2f1b0 msgr2=0x55598bb2f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:28.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.114+0000 7f1627869b80 1 --2- 172.21.15.100:0/3910904982 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55598bb2f1b0 0x55598bb2f590 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f160801b5f0 tx=0x7f1608014000 comp rx=0 tx=0).stop 2024-05-10T11:32:28.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.114+0000 7f1627869b80 1 -- 172.21.15.100:0/3910904982 shutdown_connections 2024-05-10T11:32:28.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.114+0000 7f1627869b80 1 --2- 172.21.15.100:0/3910904982 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55598bb29d50 0x55598bb2a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:28.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.114+0000 7f1627869b80 1 --2- 172.21.15.100:0/3910904982 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55598bb2f1b0 0x55598bb2f590 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:28.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.114+0000 7f1627869b80 1 --2- 172.21.15.100:0/3910904982 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55598bb2fad0 0x55598bb29810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:28.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.114+0000 7f1627869b80 1 -- 172.21.15.100:0/3910904982 >> 172.21.15.100:0/3910904982 conn(0x55598ba55b70 msgr2=0x55598bb242a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:32:28.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.115+0000 7f1627869b80 1 -- 172.21.15.100:0/3910904982 shutdown_connections 2024-05-10T11:32:28.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.115+0000 7f1627869b80 1 -- 172.21.15.100:0/3910904982 wait complete. 2024-05-10T11:32:28.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.115+0000 7f1627869b80 1 -- start start 2024-05-10T11:32:28.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.116+0000 7f1627869b80 1 Processor -- start 2024-05-10T11:32:28.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.116+0000 7f1627869b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55598bb29d50 0x55598bb357e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:28.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.117+0000 7f1627869b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55598bb2f1b0 0x55598bb32b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:28.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.117+0000 7f1627869b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55598bb2fad0 0x55598bb330c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:28.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.117+0000 7f1627869b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55598bb36450 con 0x55598bb2fad0 2024-05-10T11:32:28.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.117+0000 7f1627869b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55598ba34ed0 con 0x55598bb29d50 2024-05-10T11:32:28.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.117+0000 7f1627869b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55598bb365d0 con 0x55598bb2f1b0 2024-05-10T11:32:28.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.117+0000 7f1627869b80 10 client.0 _ll_register_callbacks cb 0x55598ba0f5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:32:28.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.117+0000 7f1616d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55598bb2fad0 0x55598bb330c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:28.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.117+0000 7f1616d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55598bb2fad0 0x55598bb330c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:32864/0 (socket says 172.21.15.100:32864) 2024-05-10T11:32:28.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.117+0000 7f1616d9d700 1 -- 172.21.15.100:0/1951172450 learned_addr learned my addr 172.21.15.100:0/1951172450 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:32:28.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.117+0000 7f161659c700 1 --2- 172.21.15.100:0/1951172450 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55598bb29d50 0x55598bb357e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:28.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.117+0000 7f1615d9b700 1 --2- 172.21.15.100:0/1951172450 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55598bb2f1b0 0x55598bb32b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:28.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.117+0000 7f1616d9d700 1 -- 172.21.15.100:0/1951172450 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55598bb2f1b0 msgr2=0x55598bb32b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:28.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.117+0000 7f1616d9d700 1 --2- 172.21.15.100:0/1951172450 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55598bb2f1b0 0x55598bb32b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:28.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.117+0000 7f1616d9d700 1 -- 172.21.15.100:0/1951172450 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55598bb29d50 msgr2=0x55598bb357e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:28.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.117+0000 7f1616d9d700 1 --2- 172.21.15.100:0/1951172450 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55598bb29d50 0x55598bb357e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:28.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.117+0000 7f1616d9d700 1 -- 172.21.15.100:0/1951172450 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55598ba32760 con 0x55598bb2fad0 2024-05-10T11:32:28.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.117+0000 7f161659c700 1 --2- 172.21.15.100:0/1951172450 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55598bb29d50 0x55598bb357e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:32:28.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.117+0000 7f1616d9d700 1 --2- 172.21.15.100:0/1951172450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55598bb2fad0 0x55598bb330c0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f161000b3f0 tx=0x7f16100aade0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e8a05f8f4f58c8c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:32:28.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.117+0000 7f15fdffb700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:32:28.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.118+0000 7f15fdffb700 1 -- 172.21.15.100:0/1951172450 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161002b030 con 0x55598bb2fad0 2024-05-10T11:32:28.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.118+0000 7f15fdffb700 1 -- 172.21.15.100:0/1951172450 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f1610007d30 con 0x55598bb2fad0 2024-05-10T11:32:28.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.118+0000 7f1627869b80 1 -- 172.21.15.100:0/1951172450 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55598ba89320 con 0x55598bb2fad0 2024-05-10T11:32:28.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.118+0000 7f15fdffb700 1 -- 172.21.15.100:0/1951172450 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f16100b34b0 con 0x55598bb2fad0 2024-05-10T11:32:28.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.118+0000 7f1627869b80 10 client.18243 fetch_fsmap learned FSMap version 37 2024-05-10T11:32:28.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.118+0000 7f1627869b80 1 -- 172.21.15.100:0/1951172450 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55598bb40da0 con 0x55598bb2fad0 2024-05-10T11:32:28.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.118+0000 7f15fdffb700 1 -- 172.21.15.100:0/1951172450 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55598ba89320 con 0x55598bb2fad0 2024-05-10T11:32:28.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.118+0000 7f15fdffb700 1 -- 172.21.15.100:0/1951172450 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f1610011070 con 0x55598bb2fad0 2024-05-10T11:32:28.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.118+0000 7f1627869b80 10 client.18243 fetch_fsmap finished waiting for FSMap version 37 2024-05-10T11:32:28.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.118+0000 7f1627869b80 10 client.18243 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:32:28.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.118+0000 7f1627869b80 20 client.18243 populate_metadata read hostname 'smithi100' 2024-05-10T11:32:28.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.119+0000 7f1627869b80 1 --2- 172.21.15.100:0/1951172450 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55598bb43360 0x55598bb45780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:28.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.119+0000 7f1627869b80 4 client.18243 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:32:28.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.119+0000 7f1627869b80 1 -- 172.21.15.100:0/1951172450 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55598ba833c0 con 0x55598bb43360 2024-05-10T11:32:28.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.119+0000 7f161659c700 1 --2- 172.21.15.100:0/1951172450 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55598bb43360 0x55598bb45780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:28.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.119+0000 7f15fdffb700 10 client.18243 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:32:28.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.119+0000 7f161659c700 1 --2- 172.21.15.100:0/1951172450 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55598bb43360 0x55598bb45780 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42c175e44b4211a server_cookie=a228d52dc4e69b2d in_seq=0 out_seq=0 2024-05-10T11:32:28.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.120+0000 7f15fdffb700 1 -- 172.21.15.100:0/1951172450 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55598ba833c0 con 0x55598bb43360 2024-05-10T11:32:28.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.120+0000 7f15fdffb700 10 client.18243 handle_command_reply: tid=0 2024-05-10T11:32:28.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.120+0000 7f1627869b80 1 client.18243 shutdown 2024-05-10T11:32:28.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.121+0000 7f1627869b80 1 -- 172.21.15.100:0/1951172450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55598bb2fad0 msgr2=0x55598bb330c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:28.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.121+0000 7f1627869b80 1 --2- 172.21.15.100:0/1951172450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55598bb2fad0 0x55598bb330c0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f161000b3f0 tx=0x7f16100aade0 comp rx=0 tx=0).stop 2024-05-10T11:32:28.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.121+0000 7f1627869b80 1 -- 172.21.15.100:0/1951172450 shutdown_connections 2024-05-10T11:32:28.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.121+0000 7f1627869b80 1 --2- 172.21.15.100:0/1951172450 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55598bb43360 0x55598bb45780 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:28.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.121+0000 7f15fdffb700 0 client.18243 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:32:28.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.121+0000 7f1627869b80 1 --2- 172.21.15.100:0/1951172450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55598bb2fad0 0x55598bb330c0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:28.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.121+0000 7f1627869b80 1 --2- 172.21.15.100:0/1951172450 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55598bb29d50 0x55598bb357e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:28.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.121+0000 7f1627869b80 1 --2- 172.21.15.100:0/1951172450 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55598bb2f1b0 0x55598bb32b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:28.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.121+0000 7f1627869b80 1 -- 172.21.15.100:0/1951172450 >> 172.21.15.100:0/1951172450 conn(0x55598ba55b70 msgr2=0x55598bb1eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:32:28.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.122+0000 7f1627869b80 1 -- 172.21.15.100:0/1951172450 shutdown_connections 2024-05-10T11:32:28.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.122+0000 7f1627869b80 1 -- 172.21.15.100:0/1951172450 wait complete. 2024-05-10T11:32:28.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.122+0000 7f1627869b80 20 client.18243 trim_cache size 0 max 16384 2024-05-10T11:32:28.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.151+0000 7f1627869b80 1 Processor -- start 2024-05-10T11:32:28.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.151+0000 7f1627869b80 1 -- start start 2024-05-10T11:32:28.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.152+0000 7f1627869b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55598bb29d50 0x55598bb3f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:28.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.152+0000 7f1627869b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55598bb2f1b0 0x55598bb1ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:28.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.152+0000 7f1627869b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55598bb2fad0 0x55598bb2e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:28.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.152+0000 7f1627869b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f16181b3a50 con 0x55598bb29d50 2024-05-10T11:32:28.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.152+0000 7f1627869b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f16181b1250 con 0x55598bb2fad0 2024-05-10T11:32:28.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.152+0000 7f1627869b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f16181af0a0 con 0x55598bb2f1b0 2024-05-10T11:32:28.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.152+0000 7f161659c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55598bb29d50 0x55598bb3f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:28.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.152+0000 7f161659c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55598bb29d50 0x55598bb3f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:32878/0 (socket says 172.21.15.100:32878) 2024-05-10T11:32:28.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.152+0000 7f161659c700 1 -- 172.21.15.100:0/2495049271 learned_addr learned my addr 172.21.15.100:0/2495049271 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:32:28.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.152+0000 7f1615d9b700 1 --2- 172.21.15.100:0/2495049271 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55598bb2f1b0 0x55598bb1ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:28.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.152+0000 7f1616d9d700 1 --2- 172.21.15.100:0/2495049271 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55598bb2fad0 0x55598bb2e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:28.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.152+0000 7f161659c700 1 -- 172.21.15.100:0/2495049271 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55598bb2f1b0 msgr2=0x55598bb1ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:28.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.152+0000 7f161659c700 1 --2- 172.21.15.100:0/2495049271 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55598bb2f1b0 0x55598bb1ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:28.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.152+0000 7f161659c700 1 -- 172.21.15.100:0/2495049271 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55598bb2fad0 msgr2=0x55598bb2e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:28.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.152+0000 7f161659c700 1 --2- 172.21.15.100:0/2495049271 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55598bb2fad0 0x55598bb2e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:28.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.152+0000 7f161659c700 1 -- 172.21.15.100:0/2495049271 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55598bb45de0 con 0x55598bb29d50 2024-05-10T11:32:28.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.152+0000 7f1615d9b700 1 --2- 172.21.15.100:0/2495049271 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55598bb2f1b0 0x55598bb1ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:32:28.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.153+0000 7f161659c700 1 --2- 172.21.15.100:0/2495049271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55598bb29d50 0x55598bb3f4e0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f160800f010 tx=0x7f1608009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99ef1926d8fea3b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:32:28.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.153+0000 7f15fffff700 1 -- 172.21.15.100:0/2495049271 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f160802a040 con 0x55598bb29d50 2024-05-10T11:32:28.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.153+0000 7f15fffff700 1 -- 172.21.15.100:0/2495049271 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1608005c80 con 0x55598bb29d50 2024-05-10T11:32:28.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.153+0000 7f15fffff700 1 -- 172.21.15.100:0/2495049271 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1608005040 con 0x55598bb29d50 2024-05-10T11:32:28.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.154+0000 7f1627869b80 1 -- 172.21.15.100:0/2495049271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55598bb29d50 msgr2=0x55598bb3f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:28.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.154+0000 7f1627869b80 1 --2- 172.21.15.100:0/2495049271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55598bb29d50 0x55598bb3f4e0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f160800f010 tx=0x7f1608009fd0 comp rx=0 tx=0).stop 2024-05-10T11:32:28.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.154+0000 7f1627869b80 1 -- 172.21.15.100:0/2495049271 shutdown_connections 2024-05-10T11:32:28.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.154+0000 7f1627869b80 1 --2- 172.21.15.100:0/2495049271 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55598bb2fad0 0x55598bb2e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:28.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.154+0000 7f1627869b80 1 --2- 172.21.15.100:0/2495049271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55598bb29d50 0x55598bb3f4e0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:28.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.154+0000 7f1627869b80 1 --2- 172.21.15.100:0/2495049271 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55598bb2f1b0 0x55598bb1ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:28.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.154+0000 7f1627869b80 1 -- 172.21.15.100:0/2495049271 >> 172.21.15.100:0/2495049271 conn(0x55598ba55b70 msgr2=0x55598bb2d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:32:28.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.154+0000 7f1627869b80 1 -- 172.21.15.100:0/2495049271 shutdown_connections 2024-05-10T11:32:28.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.154+0000 7f1627869b80 1 -- 172.21.15.100:0/2495049271 wait complete. 2024-05-10T11:32:28.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.154+0000 7f1627869b80 1 -- start start 2024-05-10T11:32:28.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.155+0000 7f1627869b80 1 Processor -- start 2024-05-10T11:32:28.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.155+0000 7f1627869b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55598bb29d50 0x55598bb3bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:28.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.155+0000 7f1627869b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55598bb2f1b0 0x55598bb4a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:28.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.155+0000 7f161659c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55598bb29d50 0x55598bb3bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:28.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.155+0000 7f161659c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55598bb29d50 0x55598bb3bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:41478/0 (socket says 172.21.15.100:41478) 2024-05-10T11:32:28.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.155+0000 7f1615d9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55598bb2f1b0 0x55598bb4a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:28.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.155+0000 7f1615d9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55598bb2f1b0 0x55598bb4a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:32886/0 (socket says 172.21.15.100:32886) 2024-05-10T11:32:28.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.155+0000 7f1627869b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55598bb2fad0 0x55598bb4a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:28.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.155+0000 7f1627869b80 1 -- 172.21.15.100:0/1470509077 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f16181af710 con 0x55598bb2f1b0 2024-05-10T11:32:28.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.155+0000 7f161659c700 1 -- 172.21.15.100:0/1470509077 learned_addr learned my addr 172.21.15.100:0/1470509077 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:32:28.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.155+0000 7f1627869b80 1 -- 172.21.15.100:0/1470509077 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f16181b0b80 con 0x55598bb29d50 2024-05-10T11:32:28.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.155+0000 7f1627869b80 1 -- 172.21.15.100:0/1470509077 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f16181afde0 con 0x55598bb2fad0 2024-05-10T11:32:28.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.156+0000 7f1616d9d700 1 --2- 172.21.15.100:0/1470509077 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55598bb2fad0 0x55598bb4a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:28.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.156+0000 7f1627869b80 10 client.0 _ll_register_callbacks cb 0x55598ba0f5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:32:28.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.156+0000 7f161659c700 1 -- 172.21.15.100:0/1470509077 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55598bb2fad0 msgr2=0x55598bb4a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:28.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.156+0000 7f161659c700 1 --2- 172.21.15.100:0/1470509077 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55598bb2fad0 0x55598bb4a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:28.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.156+0000 7f161659c700 1 -- 172.21.15.100:0/1470509077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55598bb2f1b0 msgr2=0x55598bb4a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:28.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.156+0000 7f161659c700 1 --2- 172.21.15.100:0/1470509077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55598bb2f1b0 0x55598bb4a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:28.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.156+0000 7f161659c700 1 -- 172.21.15.100:0/1470509077 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55598bb4af30 con 0x55598bb29d50 2024-05-10T11:32:28.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.156+0000 7f161659c700 1 --2- 172.21.15.100:0/1470509077 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55598bb29d50 0x55598bb3bb60 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1608035040 tx=0x7f160801bbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ced660497d4ce3e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:32:28.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.156+0000 7f15fdffb700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:32:28.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.156+0000 7f15fdffb700 1 -- 172.21.15.100:0/1470509077 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1608040060 con 0x55598bb29d50 2024-05-10T11:32:28.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.156+0000 7f15fdffb700 1 -- 172.21.15.100:0/1470509077 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f1608005250 con 0x55598bb29d50 2024-05-10T11:32:28.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.156+0000 7f1627869b80 1 -- 172.21.15.100:0/1470509077 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1608005c80 con 0x55598bb29d50 2024-05-10T11:32:28.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.157+0000 7f15fdffb700 1 -- 172.21.15.100:0/1470509077 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f1608005450 con 0x55598bb29d50 2024-05-10T11:32:28.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.157+0000 7f1627869b80 10 client.27593 fetch_fsmap learned FSMap version 37 2024-05-10T11:32:28.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.157+0000 7f1627869b80 1 -- 172.21.15.100:0/1470509077 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55598bbad970 con 0x55598bb29d50 2024-05-10T11:32:28.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.157+0000 7f15fdffb700 1 -- 172.21.15.100:0/1470509077 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1608005c80 con 0x55598bb29d50 2024-05-10T11:32:28.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.157+0000 7f15fdffb700 1 -- 172.21.15.100:0/1470509077 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f1608004820 con 0x55598bb29d50 2024-05-10T11:32:28.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.157+0000 7f1627869b80 10 client.27593 fetch_fsmap finished waiting for FSMap version 37 2024-05-10T11:32:28.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.157+0000 7f1627869b80 10 client.27593 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:32:28.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.157+0000 7f1627869b80 20 client.27593 populate_metadata read hostname 'smithi100' 2024-05-10T11:32:28.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.157+0000 7f1627869b80 1 --2- 172.21.15.100:0/1470509077 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55598bbade80 0x55598bbb02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:28.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.157+0000 7f1627869b80 4 client.27593 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T11:32:28.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.157+0000 7f1627869b80 1 -- 172.21.15.100:0/1470509077 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55598b8fb250 con 0x55598bbade80 2024-05-10T11:32:28.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.157+0000 7f1615d9b700 1 --2- 172.21.15.100:0/1470509077 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55598bbade80 0x55598bbb02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:28.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.158+0000 7f15fdffb700 10 client.27593 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:32:28.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.158+0000 7f1615d9b700 1 --2- 172.21.15.100:0/1470509077 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55598bbade80 0x55598bbb02a0 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2a6f7dbf51babfa server_cookie=54c3ca3331122aef in_seq=0 out_seq=0 2024-05-10T11:32:28.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.158+0000 7f15fdffb700 1 -- 172.21.15.100:0/1470509077 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55598b8fb250 con 0x55598bbade80 2024-05-10T11:32:28.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.158+0000 7f15fdffb700 10 client.27593 handle_command_reply: tid=0 2024-05-10T11:32:28.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.158+0000 7f1627869b80 1 client.27593 shutdown 2024-05-10T11:32:28.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.158+0000 7f1627869b80 1 -- 172.21.15.100:0/1470509077 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55598bb29d50 msgr2=0x55598bb3bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:28.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.158+0000 7f1627869b80 1 --2- 172.21.15.100:0/1470509077 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55598bb29d50 0x55598bb3bb60 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1608035040 tx=0x7f160801bbb0 comp rx=0 tx=0).stop 2024-05-10T11:32:28.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.158+0000 7f1627869b80 1 -- 172.21.15.100:0/1470509077 shutdown_connections 2024-05-10T11:32:28.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.158+0000 7f1627869b80 1 --2- 172.21.15.100:0/1470509077 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55598bbade80 0x55598bbb02a0 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:28.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.158+0000 7f1627869b80 1 --2- 172.21.15.100:0/1470509077 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55598bb2fad0 0x55598bb4a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:28.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.158+0000 7f1627869b80 1 --2- 172.21.15.100:0/1470509077 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55598bb29d50 0x55598bb3bb60 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:28.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.158+0000 7f1627869b80 1 --2- 172.21.15.100:0/1470509077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55598bb2f1b0 0x55598bb4a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:28.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.158+0000 7f1627869b80 1 -- 172.21.15.100:0/1470509077 >> 172.21.15.100:0/1470509077 conn(0x55598ba55b70 msgr2=0x55598bb2e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:32:28.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.158+0000 7f15fdffb700 0 client.27593 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:32:28.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.158+0000 7f1627869b80 1 -- 172.21.15.100:0/1470509077 shutdown_connections 2024-05-10T11:32:28.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.159+0000 7f1627869b80 1 -- 172.21.15.100:0/1470509077 wait complete. 2024-05-10T11:32:28.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.159+0000 7f1627869b80 20 client.27593 trim_cache size 0 max 16384 2024-05-10T11:32:28.164 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:32:28.165 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-05-10T11:32:28.165 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-05-10T11:32:28.165 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:32:28.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.159+0000 7f161cfd6700 1 -- 172.21.15.100:0/2294798775 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f16000b29d0 msgr2=0x7f16000b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:28.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.159+0000 7f161cfd6700 1 --2- 172.21.15.100:0/2294798775 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f16000b29d0 0x7f16000b4e90 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f160800dfd0 tx=0x7f160800f040 comp rx=0 tx=0).stop 2024-05-10T11:32:28.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.160+0000 7f161cfd6700 1 -- 172.21.15.100:0/2294798775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1618108400 msgr2=0x7f16181976b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:28.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.160+0000 7f161cfd6700 1 --2- 172.21.15.100:0/2294798775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1618108400 0x7f16181976b0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f161000b070 tx=0x7f1610004c80 comp rx=0 tx=0).stop 2024-05-10T11:32:28.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.160+0000 7f161cfd6700 1 -- 172.21.15.100:0/2294798775 shutdown_connections 2024-05-10T11:32:28.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.160+0000 7f161cfd6700 1 --2- 172.21.15.100:0/2294798775 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f16000b29d0 0x7f16000b4e90 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:28.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.160+0000 7f161cfd6700 1 --2- 172.21.15.100:0/2294798775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1618108400 0x7f16181976b0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:28.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.160+0000 7f161cfd6700 1 --2- 172.21.15.100:0/2294798775 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1618102390 0x7f161819d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:28.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.160+0000 7f161cfd6700 1 --2- 172.21.15.100:0/2294798775 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1618102d50 0x7f161819db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:28.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.160+0000 7f161cfd6700 1 -- 172.21.15.100:0/2294798775 >> 172.21.15.100:0/2294798775 conn(0x7f1618079310 msgr2=0x7f1618101130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:32:28.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.160+0000 7f161cfd6700 1 -- 172.21.15.100:0/2294798775 shutdown_connections 2024-05-10T11:32:28.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:28.160+0000 7f161cfd6700 1 -- 172.21.15.100:0/2294798775 wait complete. 2024-05-10T11:32:28.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:32:27 smithi178 ceph-mon[33359]: pgmap v843: 129 pgs: 129 active+clean; 347 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 4.3 MiB/s wr, 343 op/s 2024-05-10T11:32:28.907 INFO:tasks.workunit.client.0.smithi100.stdout: 10 3985 18.05 MB/sec warmup 22 sec latency 87.617 ms 2024-05-10T11:32:28.950 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T11:32:28.951 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T11:32:29.295 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:32:29.907 INFO:tasks.workunit.client.0.smithi100.stdout: 10 4078 17.46 MB/sec warmup 23 sec latency 172.247 ms 2024-05-10T11:32:30.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:32:29 smithi188 ceph-mon[34075]: pgmap v844: 129 pgs: 129 active+clean; 346 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 4.4 MiB/s wr, 253 op/s 2024-05-10T11:32:30.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:32:29 smithi100 ceph-mon[30296]: pgmap v844: 129 pgs: 129 active+clean; 346 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 4.4 MiB/s wr, 253 op/s 2024-05-10T11:32:30.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:32:29 smithi178 ceph-mon[33359]: pgmap v844: 129 pgs: 129 active+clean; 346 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 4.4 MiB/s wr, 253 op/s 2024-05-10T11:32:30.907 INFO:tasks.workunit.client.0.smithi100.stdout: 10 4214 16.96 MB/sec warmup 24 sec latency 94.172 ms 2024-05-10T11:32:31.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:32:30 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:32:31.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:32:30 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:32:31.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:32:30 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:32:31.907 INFO:tasks.workunit.client.0.smithi100.stdout: 10 4373 16.68 MB/sec warmup 25 sec latency 101.832 ms 2024-05-10T11:32:32.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:32:31 smithi188 ceph-mon[34075]: pgmap v845: 129 pgs: 129 active+clean; 350 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 5.0 MiB/s wr, 276 op/s 2024-05-10T11:32:32.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:32:31 smithi100 ceph-mon[30296]: pgmap v845: 129 pgs: 129 active+clean; 350 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 5.0 MiB/s wr, 276 op/s 2024-05-10T11:32:32.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:32:31 smithi178 ceph-mon[33359]: pgmap v845: 129 pgs: 129 active+clean; 350 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 5.0 MiB/s wr, 276 op/s 2024-05-10T11:32:32.907 INFO:tasks.workunit.client.0.smithi100.stdout: 10 4492 16.12 MB/sec warmup 26 sec latency 194.685 ms 2024-05-10T11:32:33.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:32:32 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:32:33.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:32:32 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:32:33.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:32:32 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:32:33.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:32:32 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:32:33.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:32:32 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:32:33.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:32:32 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:32:33.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:32:32 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:32:33.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:32:32 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:32:33.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:32:32 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:32:33.909 INFO:tasks.workunit.client.0.smithi100.stdout: 10 4652 15.62 MB/sec warmup 27 sec latency 94.388 ms 2024-05-10T11:32:34.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:32:33 smithi100 ceph-mon[30296]: pgmap v846: 129 pgs: 129 active+clean; 351 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 5.4 MiB/s wr, 264 op/s 2024-05-10T11:32:34.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:32:33 smithi178 ceph-mon[33359]: pgmap v846: 129 pgs: 129 active+clean; 351 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 5.4 MiB/s wr, 264 op/s 2024-05-10T11:32:34.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:32:33 smithi188 ceph-mon[34075]: pgmap v846: 129 pgs: 129 active+clean; 351 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 5.4 MiB/s wr, 264 op/s 2024-05-10T11:32:34.908 INFO:tasks.workunit.client.0.smithi100.stdout: 10 4802 15.24 MB/sec warmup 28 sec latency 117.227 ms 2024-05-10T11:32:35.907 INFO:tasks.workunit.client.0.smithi100.stdout: 10 4968 14.94 MB/sec warmup 29 sec latency 85.832 ms 2024-05-10T11:32:36.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:32:35 smithi178 ceph-mon[33359]: pgmap v847: 129 pgs: 129 active+clean; 346 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 5.6 MiB/s wr, 266 op/s 2024-05-10T11:32:36.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:32:35 smithi188 ceph-mon[34075]: pgmap v847: 129 pgs: 129 active+clean; 346 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 5.6 MiB/s wr, 266 op/s 2024-05-10T11:32:36.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:32:35 smithi100 ceph-mon[30296]: pgmap v847: 129 pgs: 129 active+clean; 346 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 5.6 MiB/s wr, 266 op/s 2024-05-10T11:32:36.907 INFO:tasks.workunit.client.0.smithi100.stdout: 10 5105 14.59 MB/sec warmup 30 sec latency 140.576 ms 2024-05-10T11:32:37.910 INFO:tasks.workunit.client.0.smithi100.stdout: 10 5259 14.30 MB/sec warmup 31 sec latency 145.401 ms 2024-05-10T11:32:37.921 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.908+0000 7fc7180f8640 1 -- 172.21.15.100:0/3060186327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7101045e0 msgr2=0x7fc71010ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:37.921 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.908+0000 7fc7180f8640 1 --2- 172.21.15.100:0/3060186327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7101045e0 0x7fc71010ae70 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fc708009ca0 tx=0x7fc70802a8c0 comp rx=0 tx=0).stop 2024-05-10T11:32:37.921 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.909+0000 7fc7180f8640 1 -- 172.21.15.100:0/3060186327 shutdown_connections 2024-05-10T11:32:37.921 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.909+0000 7fc7180f8640 1 --2- 172.21.15.100:0/3060186327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7101045e0 0x7fc71010ae70 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:37.921 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.909+0000 7fc7180f8640 1 --2- 172.21.15.100:0/3060186327 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc710103c20 0x7fc7101040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:37.921 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.909+0000 7fc7180f8640 1 --2- 172.21.15.100:0/3060186327 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc710102a30 0x7fc710102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:37.921 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.909+0000 7fc7180f8640 1 -- 172.21.15.100:0/3060186327 >> 172.21.15.100:0/3060186327 conn(0x7fc7100fdfa0 msgr2=0x7fc7101003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:32:37.921 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.909+0000 7fc7180f8640 1 -- 172.21.15.100:0/3060186327 shutdown_connections 2024-05-10T11:32:37.921 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.909+0000 7fc7180f8640 1 -- 172.21.15.100:0/3060186327 wait complete. 2024-05-10T11:32:37.921 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.909+0000 7fc7180f8640 1 Processor -- start 2024-05-10T11:32:37.921 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.909+0000 7fc7180f8640 1 -- start start 2024-05-10T11:32:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.909+0000 7fc7180f8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc710102a30 0x7fc710076760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.909+0000 7fc7180f8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc710103c20 0x7fc710074da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.909+0000 7fc7180f8640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7101045e0 0x7fc7100752e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.909+0000 7fc7180f8640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc71010e460 con 0x7fc710103c20 2024-05-10T11:32:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.909+0000 7fc7180f8640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc71010e2e0 con 0x7fc7101045e0 2024-05-10T11:32:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.909+0000 7fc7180f8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc71010e5e0 con 0x7fc710102a30 2024-05-10T11:32:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.910+0000 7fc715e94640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc710102a30 0x7fc710076760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.910+0000 7fc715693640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc710103c20 0x7fc710074da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.910+0000 7fc715e94640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc710102a30 0x7fc710076760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:59066/0 (socket says 172.21.15.100:59066) 2024-05-10T11:32:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.910+0000 7fc715693640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc710103c20 0x7fc710074da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56900/0 (socket says 172.21.15.100:56900) 2024-05-10T11:32:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.910+0000 7fc715e94640 1 -- 172.21.15.100:0/3365641652 learned_addr learned my addr 172.21.15.100:0/3365641652 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:32:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.910+0000 7fc716695640 1 --2- 172.21.15.100:0/3365641652 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7101045e0 0x7fc7100752e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.910+0000 7fc715693640 1 -- 172.21.15.100:0/3365641652 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc710102a30 msgr2=0x7fc710076760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.910+0000 7fc715693640 1 --2- 172.21.15.100:0/3365641652 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc710102a30 0x7fc710076760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.910+0000 7fc715693640 1 -- 172.21.15.100:0/3365641652 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7101045e0 msgr2=0x7fc7100752e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.910+0000 7fc715693640 1 --2- 172.21.15.100:0/3365641652 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7101045e0 0x7fc7100752e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.910+0000 7fc715693640 1 -- 172.21.15.100:0/3365641652 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc710075b70 con 0x7fc710103c20 2024-05-10T11:32:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.910+0000 7fc715e94640 1 --2- 172.21.15.100:0/3365641652 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc710102a30 0x7fc710076760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:32:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.910+0000 7fc715693640 1 --2- 172.21.15.100:0/3365641652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc710103c20 0x7fc710074da0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fc70000e9f0 tx=0x7fc70000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:32:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.910+0000 7fc706ffd640 1 -- 172.21.15.100:0/3365641652 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70000ccc0 con 0x7fc710103c20 2024-05-10T11:32:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.910+0000 7fc706ffd640 1 -- 172.21.15.100:0/3365641652 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc7000108c0 con 0x7fc710103c20 2024-05-10T11:32:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.910+0000 7fc7180f8640 1 -- 172.21.15.100:0/3365641652 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7101a2bd0 con 0x7fc710103c20 2024-05-10T11:32:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.910+0000 7fc706ffd640 1 -- 172.21.15.100:0/3365641652 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7000187e0 con 0x7fc710103c20 2024-05-10T11:32:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.910+0000 7fc7180f8640 1 -- 172.21.15.100:0/3365641652 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7101a3120 con 0x7fc710103c20 2024-05-10T11:32:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.913+0000 7fc706ffd640 1 -- 172.21.15.100:0/3365641652 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fc700010430 con 0x7fc710103c20 2024-05-10T11:32:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.913+0000 7fc71ae0e740 1 Processor -- start 2024-05-10T11:32:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.913+0000 7fc71ae0e740 1 -- start start 2024-05-10T11:32:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.913+0000 7fc71ae0e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563935e45db0 0x563935e461d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.913+0000 7fc71ae0e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563935e467a0 0x563935e4b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.913+0000 7fc71ae0e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563935e4b910 0x563935e4dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.913+0000 7fc71ae0e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x563935daa5d0 con 0x563935e467a0 2024-05-10T11:32:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.913+0000 7fc71ae0e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563935d63890 con 0x563935e45db0 2024-05-10T11:32:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.913+0000 7fc71ae0e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563935da6590 con 0x563935e4b910 2024-05-10T11:32:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.913+0000 7fc715e94640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563935e467a0 0x563935e4b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.913+0000 7fc715e94640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563935e467a0 0x563935e4b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56910/0 (socket says 172.21.15.100:56910) 2024-05-10T11:32:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.913+0000 7fc715e94640 1 -- 172.21.15.100:0/879923380 learned_addr learned my addr 172.21.15.100:0/879923380 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:32:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.913+0000 7fc715693640 1 --2- 172.21.15.100:0/879923380 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563935e4b910 0x563935e4dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.913+0000 7fc716695640 1 --2- 172.21.15.100:0/879923380 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563935e45db0 0x563935e461d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.914+0000 7fc715e94640 1 -- 172.21.15.100:0/879923380 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563935e4b910 msgr2=0x563935e4dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.914+0000 7fc715e94640 1 --2- 172.21.15.100:0/879923380 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563935e4b910 0x563935e4dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.914+0000 7fc715e94640 1 -- 172.21.15.100:0/879923380 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563935e45db0 msgr2=0x563935e461d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.914+0000 7fc715e94640 1 --2- 172.21.15.100:0/879923380 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563935e45db0 0x563935e461d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.914+0000 7fc715e94640 1 -- 172.21.15.100:0/879923380 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563935abc210 con 0x563935e467a0 2024-05-10T11:32:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.914+0000 7fc706ffd640 1 --2- 172.21.15.100:0/3365641652 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc6fc0b28e0 0x7fc6fc0b4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.914+0000 7fc716695640 1 --2- 172.21.15.100:0/879923380 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563935e45db0 0x563935e461d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:32:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.914+0000 7fc716695640 1 --2- 172.21.15.100:0/3365641652 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc6fc0b28e0 0x7fc6fc0b4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.914+0000 7fc706ffd640 1 -- 172.21.15.100:0/3365641652 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fc7000e2840 con 0x7fc710103c20 2024-05-10T11:32:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.914+0000 7fc715e94640 1 --2- 172.21.15.100:0/879923380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563935e467a0 0x563935e4b2a0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fc70c0115f0 tx=0x7fc70c011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b61b51ad9eb42ca3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:32:37.926 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.914+0000 7fc6fbfff640 1 -- 172.21.15.100:0/879923380 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70c004020 con 0x563935e467a0 2024-05-10T11:32:37.926 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.914+0000 7fc6fbfff640 1 -- 172.21.15.100:0/879923380 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc70c0041c0 con 0x563935e467a0 2024-05-10T11:32:37.926 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.914+0000 7fc6fbfff640 1 -- 172.21.15.100:0/879923380 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70c004020 con 0x563935e467a0 2024-05-10T11:32:37.926 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.915+0000 7fc71ae0e740 1 -- 172.21.15.100:0/879923380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563935e467a0 msgr2=0x563935e4b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:37.926 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.915+0000 7fc716695640 1 --2- 172.21.15.100:0/3365641652 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc6fc0b28e0 0x7fc6fc0b4da0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x563935e47000 tx=0x7fc70802adc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:32:37.926 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.915+0000 7fc71ae0e740 1 --2- 172.21.15.100:0/879923380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563935e467a0 0x563935e4b2a0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fc70c0115f0 tx=0x7fc70c011ab0 comp rx=0 tx=0).stop 2024-05-10T11:32:37.926 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.915+0000 7fc71ae0e740 1 -- 172.21.15.100:0/879923380 shutdown_connections 2024-05-10T11:32:37.926 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.915+0000 7fc71ae0e740 1 --2- 172.21.15.100:0/879923380 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563935e4b910 0x563935e4dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:37.926 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.915+0000 7fc71ae0e740 1 --2- 172.21.15.100:0/879923380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563935e467a0 0x563935e4b2a0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:37.926 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.915+0000 7fc71ae0e740 1 --2- 172.21.15.100:0/879923380 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563935e45db0 0x563935e461d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:37.926 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.915+0000 7fc71ae0e740 1 -- 172.21.15.100:0/879923380 >> 172.21.15.100:0/879923380 conn(0x563935e41420 msgr2=0x563935e41820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:32:37.927 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.915+0000 7fc71ae0e740 1 -- 172.21.15.100:0/879923380 shutdown_connections 2024-05-10T11:32:37.927 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.915+0000 7fc71ae0e740 1 -- 172.21.15.100:0/879923380 wait complete. 2024-05-10T11:32:37.927 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.915+0000 7fc71ae0e740 1 -- start start 2024-05-10T11:32:37.927 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.915+0000 7fc71ae0e740 1 Processor -- start 2024-05-10T11:32:37.927 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.916+0000 7fc71ae0e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563935e45db0 0x563935e500b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:37.927 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.916+0000 7fc71ae0e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563935e467a0 0x563935e505f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:37.927 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.916+0000 7fc71ae0e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563935e4b910 0x563935e575c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:37.927 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.916+0000 7fc71ae0e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x563935e4e8c0 con 0x563935e4b910 2024-05-10T11:32:37.927 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.916+0000 7fc71ae0e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563935d67370 con 0x563935e45db0 2024-05-10T11:32:37.927 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.916+0000 7fc71ae0e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563935e4ea40 con 0x563935e467a0 2024-05-10T11:32:37.928 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.916+0000 7fc71ae0e740 10 client.0 _ll_register_callbacks cb 0x563935abba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:32:37.928 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.916+0000 7fc716695640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563935e4b910 0x563935e575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:37.928 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.916+0000 7fc716695640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563935e4b910 0x563935e575c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56914/0 (socket says 172.21.15.100:56914) 2024-05-10T11:32:37.928 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.916+0000 7fc716695640 1 -- 172.21.15.100:0/588015493 learned_addr learned my addr 172.21.15.100:0/588015493 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:32:37.928 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.916+0000 7fc715693640 1 --2- 172.21.15.100:0/588015493 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563935e467a0 0x563935e505f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:37.928 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.916+0000 7fc715e94640 1 --2- 172.21.15.100:0/588015493 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563935e45db0 0x563935e500b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:37.928 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.916+0000 7fc716695640 1 -- 172.21.15.100:0/588015493 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563935e467a0 msgr2=0x563935e505f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:37.928 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.916+0000 7fc716695640 1 --2- 172.21.15.100:0/588015493 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563935e467a0 0x563935e505f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:37.928 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.916+0000 7fc716695640 1 -- 172.21.15.100:0/588015493 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563935e45db0 msgr2=0x563935e500b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:37.928 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.916+0000 7fc716695640 1 --2- 172.21.15.100:0/588015493 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563935e45db0 0x563935e500b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:37.928 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.916+0000 7fc716695640 1 -- 172.21.15.100:0/588015493 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563935b88620 con 0x563935e4b910 2024-05-10T11:32:37.928 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.916+0000 7fc715693640 1 --2- 172.21.15.100:0/588015493 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563935e467a0 0x563935e505f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:32:37.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.916+0000 7fc716695640 1 --2- 172.21.15.100:0/588015493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563935e4b910 0x563935e575c0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fc708009a60 tx=0x7fc70802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2795f2c3ac92e2ea server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:32:37.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.916+0000 7fc6f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:32:37.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.916+0000 7fc6f97fa640 1 -- 172.21.15.100:0/588015493 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc708003e00 con 0x563935e4b910 2024-05-10T11:32:37.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.916+0000 7fc6f97fa640 1 -- 172.21.15.100:0/588015493 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fc708004040 con 0x563935e4b910 2024-05-10T11:32:37.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.916+0000 7fc71ae0e740 1 -- 172.21.15.100:0/588015493 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc70c0041c0 con 0x563935e4b910 2024-05-10T11:32:37.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.917+0000 7fc6f97fa640 1 -- 172.21.15.100:0/588015493 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fc708004280 con 0x563935e4b910 2024-05-10T11:32:37.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.917+0000 7fc71ae0e740 10 client.18261 fetch_fsmap learned FSMap version 37 2024-05-10T11:32:37.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.917+0000 7fc71ae0e740 1 -- 172.21.15.100:0/588015493 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x563935e5a5a0 con 0x563935e4b910 2024-05-10T11:32:37.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.917+0000 7fc6f97fa640 1 -- 172.21.15.100:0/588015493 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc70c0041c0 con 0x563935e4b910 2024-05-10T11:32:37.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.917+0000 7fc6f97fa640 1 -- 172.21.15.100:0/588015493 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7fc708051070 con 0x563935e4b910 2024-05-10T11:32:37.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.917+0000 7fc71ae0e740 10 client.18261 fetch_fsmap finished waiting for FSMap version 37 2024-05-10T11:32:37.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.917+0000 7fc71ae0e740 10 client.18261 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:32:37.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.917+0000 7fc71ae0e740 20 client.18261 populate_metadata read hostname 'smithi100' 2024-05-10T11:32:37.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.917+0000 7fc71ae0e740 1 --2- 172.21.15.100:0/588015493 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x563935e5be10 0x563935e5e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:37.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.917+0000 7fc71ae0e740 4 client.18261 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:32:37.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.917+0000 7fc71ae0e740 1 -- 172.21.15.100:0/588015493 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563935db81e0 con 0x563935e5be10 2024-05-10T11:32:37.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.917+0000 7fc715e94640 1 --2- 172.21.15.100:0/588015493 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x563935e5be10 0x563935e5e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:37.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.918+0000 7fc6f97fa640 10 client.18261 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:32:37.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.918+0000 7fc715e94640 1 --2- 172.21.15.100:0/588015493 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x563935e5be10 0x563935e5e250 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b787cfae8c8b1d63 server_cookie=421d53619e264919 in_seq=0 out_seq=0 2024-05-10T11:32:37.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.918+0000 7fc6f97fa640 1 -- 172.21.15.100:0/588015493 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563935db81e0 con 0x563935e5be10 2024-05-10T11:32:37.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.918+0000 7fc6f97fa640 10 client.18261 handle_command_reply: tid=0 2024-05-10T11:32:37.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.918+0000 7fc71ae0e740 1 client.18261 shutdown 2024-05-10T11:32:37.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.918+0000 7fc71ae0e740 1 -- 172.21.15.100:0/588015493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563935e4b910 msgr2=0x563935e575c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:37.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.918+0000 7fc71ae0e740 1 --2- 172.21.15.100:0/588015493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563935e4b910 0x563935e575c0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fc708009a60 tx=0x7fc70802a8c0 comp rx=0 tx=0).stop 2024-05-10T11:32:37.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.918+0000 7fc71ae0e740 1 -- 172.21.15.100:0/588015493 shutdown_connections 2024-05-10T11:32:37.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.918+0000 7fc71ae0e740 1 --2- 172.21.15.100:0/588015493 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x563935e5be10 0x563935e5e250 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:37.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.918+0000 7fc71ae0e740 1 --2- 172.21.15.100:0/588015493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563935e4b910 0x563935e575c0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:37.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.918+0000 7fc6f97fa640 0 client.18261 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:32:37.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.918+0000 7fc71ae0e740 1 --2- 172.21.15.100:0/588015493 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563935e467a0 0x563935e505f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:37.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.919+0000 7fc71ae0e740 1 --2- 172.21.15.100:0/588015493 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563935e45db0 0x563935e500b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:37.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.919+0000 7fc71ae0e740 1 -- 172.21.15.100:0/588015493 >> 172.21.15.100:0/588015493 conn(0x563935e41420 msgr2=0x563935e40580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:32:37.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.919+0000 7fc71ae0e740 1 -- 172.21.15.100:0/588015493 shutdown_connections 2024-05-10T11:32:37.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.919+0000 7fc71ae0e740 1 -- 172.21.15.100:0/588015493 wait complete. 2024-05-10T11:32:37.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.919+0000 7fc71ae0e740 20 client.18261 trim_cache size 0 max 16384 2024-05-10T11:32:37.939 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.933+0000 7fc71ae0e740 1 Processor -- start 2024-05-10T11:32:37.939 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.933+0000 7fc71ae0e740 1 -- start start 2024-05-10T11:32:37.939 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.934+0000 7fc71ae0e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563935e41420 0x563935e44a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:37.939 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.934+0000 7fc71ae0e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563935e45db0 0x563935e57580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:37.939 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.934+0000 7fc71ae0e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563935e467a0 0x563935e62b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:37.939 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.934+0000 7fc71ae0e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc7101afee0 con 0x563935e41420 2024-05-10T11:32:37.940 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.934+0000 7fc71ae0e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc7101ad720 con 0x563935e467a0 2024-05-10T11:32:37.940 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.934+0000 7fc71ae0e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc7101ab570 con 0x563935e45db0 2024-05-10T11:32:37.940 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.934+0000 7fc715e94640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563935e41420 0x563935e44a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:37.940 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.934+0000 7fc715e94640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563935e41420 0x563935e44a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56930/0 (socket says 172.21.15.100:56930) 2024-05-10T11:32:37.940 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.934+0000 7fc715e94640 1 -- 172.21.15.100:0/2622221681 learned_addr learned my addr 172.21.15.100:0/2622221681 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:32:37.940 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.934+0000 7fc715693640 1 --2- 172.21.15.100:0/2622221681 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563935e45db0 0x563935e57580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:37.940 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.934+0000 7fc716695640 1 --2- 172.21.15.100:0/2622221681 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563935e467a0 0x563935e62b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:37.940 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.934+0000 7fc715e94640 1 -- 172.21.15.100:0/2622221681 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563935e45db0 msgr2=0x563935e57580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:37.940 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.934+0000 7fc715e94640 1 --2- 172.21.15.100:0/2622221681 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563935e45db0 0x563935e57580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:37.940 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.934+0000 7fc715e94640 1 -- 172.21.15.100:0/2622221681 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563935e467a0 msgr2=0x563935e62b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:37.940 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.934+0000 7fc715e94640 1 --2- 172.21.15.100:0/2622221681 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563935e467a0 0x563935e62b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:37.940 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.934+0000 7fc715e94640 1 -- 172.21.15.100:0/2622221681 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563935e41250 con 0x563935e41420 2024-05-10T11:32:37.940 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.934+0000 7fc716695640 1 --2- 172.21.15.100:0/2622221681 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563935e467a0 0x563935e62b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:32:37.940 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.934+0000 7fc715e94640 1 --2- 172.21.15.100:0/2622221681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563935e41420 0x563935e44a40 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fc70c00ffc0 tx=0x7fc70c011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5556cdf0967af727 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:32:37.940 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.934+0000 7fc6fb7fe640 1 -- 172.21.15.100:0/2622221681 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70c004020 con 0x563935e41420 2024-05-10T11:32:37.941 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.934+0000 7fc6fb7fe640 1 -- 172.21.15.100:0/2622221681 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc70c0041c0 con 0x563935e41420 2024-05-10T11:32:37.941 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.934+0000 7fc6fb7fe640 1 -- 172.21.15.100:0/2622221681 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70c00fba0 con 0x563935e41420 2024-05-10T11:32:37.941 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.935+0000 7fc71ae0e740 1 -- 172.21.15.100:0/2622221681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563935e41420 msgr2=0x563935e44a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:37.941 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.935+0000 7fc71ae0e740 1 --2- 172.21.15.100:0/2622221681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563935e41420 0x563935e44a40 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fc70c00ffc0 tx=0x7fc70c011ab0 comp rx=0 tx=0).stop 2024-05-10T11:32:37.941 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.935+0000 7fc71ae0e740 1 -- 172.21.15.100:0/2622221681 shutdown_connections 2024-05-10T11:32:37.941 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.935+0000 7fc71ae0e740 1 --2- 172.21.15.100:0/2622221681 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563935e467a0 0x563935e62b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:37.941 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.935+0000 7fc71ae0e740 1 --2- 172.21.15.100:0/2622221681 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563935e45db0 0x563935e57580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:37.941 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.935+0000 7fc71ae0e740 1 --2- 172.21.15.100:0/2622221681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563935e41420 0x563935e44a40 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:37.941 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.935+0000 7fc71ae0e740 1 -- 172.21.15.100:0/2622221681 >> 172.21.15.100:0/2622221681 conn(0x563935e4b3e0 msgr2=0x563935e44330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:32:37.941 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.935+0000 7fc71ae0e740 1 -- 172.21.15.100:0/2622221681 shutdown_connections 2024-05-10T11:32:37.941 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.935+0000 7fc71ae0e740 1 -- 172.21.15.100:0/2622221681 wait complete. 2024-05-10T11:32:37.942 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.935+0000 7fc71ae0e740 1 -- start start 2024-05-10T11:32:37.942 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.935+0000 7fc71ae0e740 1 Processor -- start 2024-05-10T11:32:37.942 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.936+0000 7fc71ae0e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563935e41420 0x563935e54e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:37.942 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.936+0000 7fc71ae0e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563935e45db0 0x563935e55340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:37.942 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.936+0000 7fc71ae0e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563935e467a0 0x563935e56300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:37.942 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.936+0000 7fc71ae0e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc7101abbe0 con 0x563935e41420 2024-05-10T11:32:37.942 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.936+0000 7fc71ae0e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc7101ad050 con 0x563935e467a0 2024-05-10T11:32:37.942 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.936+0000 7fc71ae0e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc7101ac2b0 con 0x563935e45db0 2024-05-10T11:32:37.942 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.936+0000 7fc71ae0e740 10 client.0 _ll_register_callbacks cb 0x7fc70c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:32:37.942 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.936+0000 7fc715e94640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563935e41420 0x563935e54e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:37.942 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.936+0000 7fc715e94640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563935e41420 0x563935e54e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56932/0 (socket says 172.21.15.100:56932) 2024-05-10T11:32:37.942 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.936+0000 7fc715e94640 1 -- 172.21.15.100:0/2252971244 learned_addr learned my addr 172.21.15.100:0/2252971244 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:32:37.943 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.936+0000 7fc715693640 1 --2- 172.21.15.100:0/2252971244 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563935e45db0 0x563935e55340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:37.943 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.936+0000 7fc716695640 1 --2- 172.21.15.100:0/2252971244 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563935e467a0 0x563935e56300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:37.943 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.936+0000 7fc715e94640 1 -- 172.21.15.100:0/2252971244 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563935e45db0 msgr2=0x563935e55340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:37.943 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.936+0000 7fc715e94640 1 --2- 172.21.15.100:0/2252971244 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563935e45db0 0x563935e55340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:37.943 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.936+0000 7fc715e94640 1 -- 172.21.15.100:0/2252971244 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563935e467a0 msgr2=0x563935e56300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:37.943 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.936+0000 7fc715e94640 1 --2- 172.21.15.100:0/2252971244 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563935e467a0 0x563935e56300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:37.943 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.936+0000 7fc715e94640 1 -- 172.21.15.100:0/2252971244 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563935e568f0 con 0x563935e41420 2024-05-10T11:32:37.943 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.936+0000 7fc715e94640 1 --2- 172.21.15.100:0/2252971244 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563935e41420 0x563935e54e00 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fc70c009de0 tx=0x7fc70c009e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=838ff56bb2354611 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:32:37.943 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.936+0000 7fc6f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:32:37.943 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.936+0000 7fc6f97fa640 1 -- 172.21.15.100:0/2252971244 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70c00faf0 con 0x563935e41420 2024-05-10T11:32:37.943 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.936+0000 7fc71ae0e740 1 -- 172.21.15.100:0/2252971244 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc70c0041c0 con 0x563935e41420 2024-05-10T11:32:37.943 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.936+0000 7fc6f97fa640 1 -- 172.21.15.100:0/2252971244 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fc70c00fc90 con 0x563935e41420 2024-05-10T11:32:37.943 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.937+0000 7fc6f97fa640 1 -- 172.21.15.100:0/2252971244 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fc70c047050 con 0x563935e41420 2024-05-10T11:32:37.943 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.937+0000 7fc71ae0e740 10 client.18267 fetch_fsmap learned FSMap version 37 2024-05-10T11:32:37.944 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.937+0000 7fc71ae0e740 1 -- 172.21.15.100:0/2252971244 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x563935ec6790 con 0x563935e41420 2024-05-10T11:32:37.944 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.937+0000 7fc6f97fa640 1 -- 172.21.15.100:0/2252971244 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc70c0041c0 con 0x563935e41420 2024-05-10T11:32:37.944 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.937+0000 7fc6f97fa640 1 -- 172.21.15.100:0/2252971244 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7fc70c02bce0 con 0x563935e41420 2024-05-10T11:32:37.944 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.937+0000 7fc71ae0e740 10 client.18267 fetch_fsmap finished waiting for FSMap version 37 2024-05-10T11:32:37.944 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.937+0000 7fc71ae0e740 10 client.18267 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:32:37.944 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.937+0000 7fc71ae0e740 20 client.18267 populate_metadata read hostname 'smithi100' 2024-05-10T11:32:37.944 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.937+0000 7fc71ae0e740 1 --2- 172.21.15.100:0/2252971244 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x563935e4b910 0x563935ec9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:37.944 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.937+0000 7fc71ae0e740 4 client.18267 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:32:37.944 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.937+0000 7fc71ae0e740 1 -- 172.21.15.100:0/2252971244 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563935db2050 con 0x563935e4b910 2024-05-10T11:32:37.944 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.937+0000 7fc715693640 1 --2- 172.21.15.100:0/2252971244 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x563935e4b910 0x563935ec9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:37.944 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.937+0000 7fc6f97fa640 10 client.18267 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:32:37.944 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:37.937+0000 7fc715693640 1 --2- 172.21.15.100:0/2252971244 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x563935e4b910 0x563935ec9000 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abc2a1523ed25513 server_cookie=5cf7e7143ae9af63 in_seq=0 out_seq=0 2024-05-10T11:32:38.037 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:38.037+0000 7fc6f97fa640 1 -- 172.21.15.100:0/2252971244 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563935db2050 con 0x563935e4b910 2024-05-10T11:32:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:38.037+0000 7fc6f97fa640 10 client.18267 handle_command_reply: tid=0 2024-05-10T11:32:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:38.037+0000 7fc71ae0e740 1 client.18267 shutdown 2024-05-10T11:32:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:38.037+0000 7fc71ae0e740 1 -- 172.21.15.100:0/2252971244 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563935e41420 msgr2=0x563935e54e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:38.037+0000 7fc71ae0e740 1 --2- 172.21.15.100:0/2252971244 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563935e41420 0x563935e54e00 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fc70c009de0 tx=0x7fc70c009e10 comp rx=0 tx=0).stop 2024-05-10T11:32:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:38.037+0000 7fc71ae0e740 1 -- 172.21.15.100:0/2252971244 shutdown_connections 2024-05-10T11:32:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:38.037+0000 7fc71ae0e740 1 --2- 172.21.15.100:0/2252971244 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x563935e4b910 0x563935ec9000 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:38.037+0000 7fc71ae0e740 1 --2- 172.21.15.100:0/2252971244 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563935e467a0 0x563935e56300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:38.037+0000 7fc71ae0e740 1 --2- 172.21.15.100:0/2252971244 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563935e45db0 0x563935e55340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:38.037+0000 7fc71ae0e740 1 --2- 172.21.15.100:0/2252971244 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563935e41420 0x563935e54e00 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:38.037+0000 7fc6f97fa640 0 client.18267 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:32:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:38.037+0000 7fc71ae0e740 1 -- 172.21.15.100:0/2252971244 >> 172.21.15.100:0/2252971244 conn(0x563935e4b3e0 msgr2=0x563935e44330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:32:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:38.037+0000 7fc71ae0e740 1 -- 172.21.15.100:0/2252971244 shutdown_connections 2024-05-10T11:32:38.039 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:38.037+0000 7fc71ae0e740 1 -- 172.21.15.100:0/2252971244 wait complete. 2024-05-10T11:32:38.039 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:38.037+0000 7fc71ae0e740 20 client.18267 trim_cache size 0 max 16384 2024-05-10T11:32:38.039 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:32:38.039 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:32:38.039 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:32:38.039 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:32:38.040 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:38.038+0000 7fc7180f8640 1 -- 172.21.15.100:0/3365641652 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc6fc0b28e0 msgr2=0x7fc6fc0b4da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:38.040 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:38.038+0000 7fc7180f8640 1 --2- 172.21.15.100:0/3365641652 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc6fc0b28e0 0x7fc6fc0b4da0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x563935e47000 tx=0x7fc70802adc0 comp rx=0 tx=0).stop 2024-05-10T11:32:38.040 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:38.038+0000 7fc7180f8640 1 -- 172.21.15.100:0/3365641652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc710103c20 msgr2=0x7fc710074da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:38.040 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:38.038+0000 7fc7180f8640 1 --2- 172.21.15.100:0/3365641652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc710103c20 0x7fc710074da0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fc70000e9f0 tx=0x7fc70000eeb0 comp rx=0 tx=0).stop 2024-05-10T11:32:38.040 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:38.038+0000 7fc7180f8640 1 -- 172.21.15.100:0/3365641652 shutdown_connections 2024-05-10T11:32:38.040 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:38.038+0000 7fc7180f8640 1 --2- 172.21.15.100:0/3365641652 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc6fc0b28e0 0x7fc6fc0b4da0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:38.041 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:38.038+0000 7fc7180f8640 1 --2- 172.21.15.100:0/3365641652 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7101045e0 0x7fc7100752e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:38.041 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:38.038+0000 7fc7180f8640 1 --2- 172.21.15.100:0/3365641652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc710103c20 0x7fc710074da0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:38.041 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:38.038+0000 7fc7180f8640 1 --2- 172.21.15.100:0/3365641652 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc710102a30 0x7fc710076760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:38.041 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:38.038+0000 7fc7180f8640 1 -- 172.21.15.100:0/3365641652 >> 172.21.15.100:0/3365641652 conn(0x7fc7100fdfa0 msgr2=0x7fc710106f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:32:38.041 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:38.039+0000 7fc7180f8640 1 -- 172.21.15.100:0/3365641652 shutdown_connections 2024-05-10T11:32:38.041 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:38.039+0000 7fc7180f8640 1 -- 172.21.15.100:0/3365641652 wait complete. 2024-05-10T11:32:38.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:32:37 smithi178 ceph-mon[33359]: pgmap v848: 129 pgs: 129 active+clean; 339 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 4.9 MiB/s wr, 284 op/s 2024-05-10T11:32:38.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:32:37 smithi188 ceph-mon[34075]: pgmap v848: 129 pgs: 129 active+clean; 339 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 4.9 MiB/s wr, 284 op/s 2024-05-10T11:32:38.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:32:37 smithi100 ceph-mon[30296]: pgmap v848: 129 pgs: 129 active+clean; 339 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 4.9 MiB/s wr, 284 op/s 2024-05-10T11:32:38.907 INFO:tasks.workunit.client.0.smithi100.stdout: 10 5418 14.32 MB/sec warmup 32 sec latency 160.850 ms 2024-05-10T11:32:39.907 INFO:tasks.workunit.client.0.smithi100.stdout: 10 5584 14.27 MB/sec warmup 33 sec latency 162.697 ms 2024-05-10T11:32:40.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:32:39 smithi178 ceph-mon[33359]: pgmap v849: 129 pgs: 129 active+clean; 338 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 3.9 MiB/s wr, 274 op/s 2024-05-10T11:32:40.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:32:39 smithi188 ceph-mon[34075]: pgmap v849: 129 pgs: 129 active+clean; 338 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 3.9 MiB/s wr, 274 op/s 2024-05-10T11:32:40.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:32:39 smithi100 ceph-mon[30296]: pgmap v849: 129 pgs: 129 active+clean; 338 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 3.9 MiB/s wr, 274 op/s 2024-05-10T11:32:40.907 INFO:tasks.workunit.client.0.smithi100.stdout: 10 5769 14.05 MB/sec warmup 34 sec latency 118.090 ms 2024-05-10T11:32:41.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:41.120+0000 7f742a0bb700 1 -- 172.21.15.100:0/3292359812 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7424103390 msgr2=0x7f7424107000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:41.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:41.120+0000 7f742a0bb700 1 --2- 172.21.15.100:0/3292359812 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7424103390 0x7f7424107000 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f74180099c0 tx=0x7f7418009cd0 comp rx=0 tx=0).stop 2024-05-10T11:32:41.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:41.121+0000 7f742a0bb700 1 -- 172.21.15.100:0/3292359812 shutdown_connections 2024-05-10T11:32:41.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:41.121+0000 7f742a0bb700 1 --2- 172.21.15.100:0/3292359812 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7424107730 0x7f742410f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:41.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:41.121+0000 7f742a0bb700 1 --2- 172.21.15.100:0/3292359812 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74241029e0 0x7f7424102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:41.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:41.121+0000 7f742a0bb700 1 --2- 172.21.15.100:0/3292359812 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7424103390 0x7f7424107000 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:41.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:41.121+0000 7f742a0bb700 1 -- 172.21.15.100:0/3292359812 >> 172.21.15.100:0/3292359812 conn(0x7f7424079350 msgr2=0x7f7424079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:32:41.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:41.121+0000 7f742a0bb700 1 -- 172.21.15.100:0/3292359812 shutdown_connections 2024-05-10T11:32:41.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:41.121+0000 7f742a0bb700 1 -- 172.21.15.100:0/3292359812 wait complete. 2024-05-10T11:32:41.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:41.121+0000 7f742a0bb700 1 Processor -- start 2024-05-10T11:32:41.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:41.122+0000 7f742a0bb700 1 -- start start 2024-05-10T11:32:41.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:41.122+0000 7f742a0bb700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74241029e0 0x7f7424074d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:41.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:41.122+0000 7f742a0bb700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7424103390 0x7f742406de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:41.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:41.122+0000 7f742a0bb700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7424107730 0x7f742406e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:41.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:41.122+0000 7f742a0bb700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7424111350 con 0x7f7424107730 2024-05-10T11:32:41.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:41.122+0000 7f742a0bb700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f74241111d0 con 0x7f7424103390 2024-05-10T11:32:41.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:41.122+0000 7f742a0bb700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f74241114d0 con 0x7f74241029e0 2024-05-10T11:32:41.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:41.122+0000 7f74237fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74241029e0 0x7f7424074d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:41.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:41.122+0000 7f7422ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7424103390 0x7f742406de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:41.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:41.122+0000 7f7423fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7424107730 0x7f742406e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:41.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:41.122+0000 7f7423fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7424107730 0x7f742406e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56948/0 (socket says 172.21.15.100:56948) 2024-05-10T11:32:41.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:41.122+0000 7f7423fff700 1 -- 172.21.15.100:0/416953691 learned_addr learned my addr 172.21.15.100:0/416953691 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:32:41.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:41.123+0000 7f7423fff700 1 -- 172.21.15.100:0/416953691 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74241029e0 msgr2=0x7f7424074d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:41.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:41.123+0000 7f7423fff700 1 --2- 172.21.15.100:0/416953691 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74241029e0 0x7f7424074d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:41.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:41.123+0000 7f7423fff700 1 -- 172.21.15.100:0/416953691 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7424103390 msgr2=0x7f742406de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:41.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:41.123+0000 7f7423fff700 1 --2- 172.21.15.100:0/416953691 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7424103390 0x7f742406de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:41.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:41.123+0000 7f7423fff700 1 -- 172.21.15.100:0/416953691 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f742406eab0 con 0x7f7424107730 2024-05-10T11:32:41.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:41.123+0000 7f7423fff700 1 --2- 172.21.15.100:0/416953691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7424107730 0x7f742406e380 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f741c00ef30 tx=0x7f741c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:32:41.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:41.123+0000 7f7420ff9700 1 -- 172.21.15.100:0/416953691 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741c0092e0 con 0x7f7424107730 2024-05-10T11:32:41.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:41.123+0000 7f7420ff9700 1 -- 172.21.15.100:0/416953691 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f741c004bb0 con 0x7f7424107730 2024-05-10T11:32:41.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:41.123+0000 7f7420ff9700 1 -- 172.21.15.100:0/416953691 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741c004020 con 0x7f7424107730 2024-05-10T11:32:41.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:41.123+0000 7f742a0bb700 1 -- 172.21.15.100:0/416953691 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7424072890 con 0x7f7424107730 2024-05-10T11:32:41.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:41.124+0000 7f742a0bb700 1 -- 172.21.15.100:0/416953691 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7424072df0 con 0x7f7424107730 2024-05-10T11:32:41.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:41.125+0000 7f742a0bb700 1 -- 172.21.15.100:0/416953691 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f742406ec80 con 0x7f7424107730 2024-05-10T11:32:41.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:41.127+0000 7f7420ff9700 1 -- 172.21.15.100:0/416953691 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f741c01b030 con 0x7f7424107730 2024-05-10T11:32:41.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:41.128+0000 7f7420ff9700 1 --2- 172.21.15.100:0/416953691 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f740c0b2900 0x7f740c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:41.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:41.128+0000 7f7420ff9700 1 -- 172.21.15.100:0/416953691 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f741c0e1ee0 con 0x7f7424107730 2024-05-10T11:32:41.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:41.128+0000 7f74237fe700 1 --2- 172.21.15.100:0/416953691 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f740c0b2900 0x7f740c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:41.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:41.129+0000 7f74237fe700 1 --2- 172.21.15.100:0/416953691 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f740c0b2900 0x7f740c0b4dc0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f7414009c60 tx=0x7f7414009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:32:41.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:41.133+0000 7f7420ff9700 1 -- 172.21.15.100:0/416953691 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f741c022020 con 0x7f7424107730 2024-05-10T11:32:41.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:41.564+0000 7f742a0bb700 1 -- 172.21.15.100:0/416953691 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f742406f320 con 0x7f7424107730 2024-05-10T11:32:41.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:41.565+0000 7f7420ff9700 1 -- 172.21.15.100:0/416953691 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11902 (secure 0 0 0) 0x7f741c010040 con 0x7f7424107730 2024-05-10T11:32:41.565 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:32:41.565 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2072225012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2072225012},{"type":"v1","addr":"172.21.15.100:6835","nonce":2072225012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/401939540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":401939540},{"type":"v1","addr":"172.21.15.100:6837","nonce":401939540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14571,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/3946311025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3946311025},{"type":"v1","addr":"172.21.15.100:6839","nonce":3946311025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3736371660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3736371660},{"type":"v1","addr":"172.21.15.100:6841","nonce":3736371660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/2741810791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2741810791},{"type":"v1","addr":"172.21.15.178:6833","nonce":2741810791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1704891041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1704891041},{"type":"v1","addr":"172.21.15.188:6833","nonce":1704891041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T11:14:27.242452+0000","modified":"2024-05-10T11:32:19.578743+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24439,"mds_1":24437,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/2124076312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2124076312},{"type":"v1","addr":"172.21.15.178:6835","nonce":2124076312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/3224358425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3224358425},{"type":"v1","addr":"172.21.15.178:6837","nonce":3224358425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/1943342620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1943342620},{"type":"v1","addr":"172.21.15.188:6837","nonce":1943342620}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.178:6839/3760985404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3760985404},{"type":"v1","addr":"172.21.15.178:6839","nonce":3760985404}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.188:6839/3194917070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":3194917070},{"type":"v1","addr":"172.21.15.188:6839","nonce":3194917070}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27034":{"gid":27034,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/3139559450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3139559450},{"type":"v1","addr":"172.21.15.188:6835","nonce":3139559450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24439,"qdb_cluster":[24439,24427,24437]},"id":1}]} 2024-05-10T11:32:41.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:41.568+0000 7f742a0bb700 1 -- 172.21.15.100:0/416953691 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f740c0b2900 msgr2=0x7f740c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:41.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:41.568+0000 7f742a0bb700 1 --2- 172.21.15.100:0/416953691 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f740c0b2900 0x7f740c0b4dc0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f7414009c60 tx=0x7f7414009260 comp rx=0 tx=0).stop 2024-05-10T11:32:41.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:41.568+0000 7f742a0bb700 1 -- 172.21.15.100:0/416953691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7424107730 msgr2=0x7f742406e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:41.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:41.568+0000 7f742a0bb700 1 --2- 172.21.15.100:0/416953691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7424107730 0x7f742406e380 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f741c00ef30 tx=0x7f741c00c3a0 comp rx=0 tx=0).stop 2024-05-10T11:32:41.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:41.569+0000 7f742a0bb700 1 -- 172.21.15.100:0/416953691 shutdown_connections 2024-05-10T11:32:41.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:41.569+0000 7f742a0bb700 1 --2- 172.21.15.100:0/416953691 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f740c0b2900 0x7f740c0b4dc0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:41.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:41.569+0000 7f742a0bb700 1 --2- 172.21.15.100:0/416953691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7424107730 0x7f742406e380 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:41.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:41.569+0000 7f742a0bb700 1 --2- 172.21.15.100:0/416953691 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74241029e0 0x7f7424074d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:41.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:41.569+0000 7f742a0bb700 1 --2- 172.21.15.100:0/416953691 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7424103390 0x7f742406de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:41.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:41.569+0000 7f742a0bb700 1 -- 172.21.15.100:0/416953691 >> 172.21.15.100:0/416953691 conn(0x7f7424079350 msgr2=0x7f7424100ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:32:41.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:41.569+0000 7f742a0bb700 1 -- 172.21.15.100:0/416953691 shutdown_connections 2024-05-10T11:32:41.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:41.569+0000 7f742a0bb700 1 -- 172.21.15.100:0/416953691 wait complete. 2024-05-10T11:32:41.570 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 37 2024-05-10T11:32:41.907 INFO:tasks.workunit.client.0.smithi100.stdout: 10 5981 13.89 MB/sec warmup 35 sec latency 83.844 ms 2024-05-10T11:32:42.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:32:41 smithi178 ceph-mon[33359]: pgmap v850: 129 pgs: 129 active+clean; 331 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 903 KiB/s rd, 3.6 MiB/s wr, 268 op/s 2024-05-10T11:32:42.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:32:41 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/416953691' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:32:42.277 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:32:41 smithi100 ceph-mon[30296]: pgmap v850: 129 pgs: 129 active+clean; 331 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 903 KiB/s rd, 3.6 MiB/s wr, 268 op/s 2024-05-10T11:32:42.277 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:32:41 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/416953691' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:32:42.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:32:41 smithi188 ceph-mon[34075]: pgmap v850: 129 pgs: 129 active+clean; 331 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 903 KiB/s rd, 3.6 MiB/s wr, 268 op/s 2024-05-10T11:32:42.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:32:41 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/416953691' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:32:42.907 INFO:tasks.workunit.client.0.smithi100.stdout: 10 6142 13.69 MB/sec warmup 36 sec latency 99.565 ms 2024-05-10T11:32:43.015 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-10T11:32:43.379 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:32:43.907 INFO:tasks.workunit.client.0.smithi100.stdout: 10 6377 13.85 MB/sec warmup 37 sec latency 66.780 ms 2024-05-10T11:32:44.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:32:43 smithi178 ceph-mon[33359]: pgmap v851: 129 pgs: 129 active+clean; 336 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 3.4 MiB/s wr, 267 op/s 2024-05-10T11:32:44.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:32:43 smithi188 ceph-mon[34075]: pgmap v851: 129 pgs: 129 active+clean; 336 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 3.4 MiB/s wr, 267 op/s 2024-05-10T11:32:44.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:32:43 smithi100 ceph-mon[30296]: pgmap v851: 129 pgs: 129 active+clean; 336 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 3.4 MiB/s wr, 267 op/s 2024-05-10T11:32:44.907 INFO:tasks.workunit.client.0.smithi100.stdout: 10 6537 13.68 MB/sec warmup 38 sec latency 141.735 ms 2024-05-10T11:32:45.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.209+0000 7f95f404a700 1 -- 172.21.15.100:0/4082359863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95ec102e00 msgr2=0x7f95ec1031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:45.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.209+0000 7f95f404a700 1 --2- 172.21.15.100:0/4082359863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95ec102e00 0x7f95ec1031e0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f95e800b0d0 tx=0x7f95e800b3e0 comp rx=0 tx=0).stop 2024-05-10T11:32:45.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.209+0000 7f95f404a700 1 -- 172.21.15.100:0/4082359863 shutdown_connections 2024-05-10T11:32:45.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.209+0000 7f95f404a700 1 --2- 172.21.15.100:0/4082359863 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95ec107760 0x7f95ec10f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:45.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.209+0000 7f95f404a700 1 --2- 172.21.15.100:0/4082359863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95ec102e00 0x7f95ec1031e0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:45.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.209+0000 7f95f404a700 1 --2- 172.21.15.100:0/4082359863 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95ec1037b0 0x7f95ec107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:45.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.209+0000 7f95f404a700 1 -- 172.21.15.100:0/4082359863 >> 172.21.15.100:0/4082359863 conn(0x7f95ec078a10 msgr2=0x7f95ec0ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:32:45.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.209+0000 7f95f404a700 1 -- 172.21.15.100:0/4082359863 shutdown_connections 2024-05-10T11:32:45.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.209+0000 7f95f404a700 1 -- 172.21.15.100:0/4082359863 wait complete. 2024-05-10T11:32:45.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.210+0000 7f95f404a700 1 Processor -- start 2024-05-10T11:32:45.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.210+0000 7f95f404a700 1 -- start start 2024-05-10T11:32:45.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.210+0000 7f95f404a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95ec102e00 0x7f95ec19e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:45.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.210+0000 7f95f404a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95ec1037b0 0x7f95ec19ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:45.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.210+0000 7f95f404a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95ec107760 0x7f95ec198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:45.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.210+0000 7f95f404a700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f95ec112430 con 0x7f95ec102e00 2024-05-10T11:32:45.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.210+0000 7f95f404a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f95ec1122b0 con 0x7f95ec107760 2024-05-10T11:32:45.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.210+0000 7f95f404a700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f95ec1125b0 con 0x7f95ec1037b0 2024-05-10T11:32:45.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.210+0000 7f95f1de6700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95ec102e00 0x7f95ec19e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:45.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.210+0000 7f95f25e7700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95ec107760 0x7f95ec198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:45.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.210+0000 7f95f1de6700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95ec102e00 0x7f95ec19e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56966/0 (socket says 172.21.15.100:56966) 2024-05-10T11:32:45.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.210+0000 7f95f1de6700 1 -- 172.21.15.100:0/663463900 learned_addr learned my addr 172.21.15.100:0/663463900 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:32:45.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.210+0000 7f95f25e7700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95ec107760 0x7f95ec198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:53072/0 (socket says 172.21.15.100:53072) 2024-05-10T11:32:45.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.210+0000 7f95f15e5700 1 --2- 172.21.15.100:0/663463900 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95ec1037b0 0x7f95ec19ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:45.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.211+0000 7f95f25e7700 1 -- 172.21.15.100:0/663463900 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95ec1037b0 msgr2=0x7f95ec19ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:45.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.211+0000 7f95f25e7700 1 --2- 172.21.15.100:0/663463900 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95ec1037b0 0x7f95ec19ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:45.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.211+0000 7f95f25e7700 1 -- 172.21.15.100:0/663463900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95ec102e00 msgr2=0x7f95ec19e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:45.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.211+0000 7f95f25e7700 1 --2- 172.21.15.100:0/663463900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95ec102e00 0x7f95ec19e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:45.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.211+0000 7f95f25e7700 1 -- 172.21.15.100:0/663463900 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95ec198f80 con 0x7f95ec107760 2024-05-10T11:32:45.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.211+0000 7f95f15e5700 1 --2- 172.21.15.100:0/663463900 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95ec1037b0 0x7f95ec19ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:32:45.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.211+0000 7f95f25e7700 1 --2- 172.21.15.100:0/663463900 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95ec107760 0x7f95ec198750 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f95e400e970 tx=0x7f95e400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:32:45.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.211+0000 7f95e2ffd700 1 -- 172.21.15.100:0/663463900 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e4004220 con 0x7f95ec107760 2024-05-10T11:32:45.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.211+0000 7f95f404a700 1 -- 172.21.15.100:0/663463900 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95ec199270 con 0x7f95ec107760 2024-05-10T11:32:45.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.211+0000 7f95e2ffd700 1 -- 172.21.15.100:0/663463900 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f95e40043c0 con 0x7f95ec107760 2024-05-10T11:32:45.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.211+0000 7f95e2ffd700 1 -- 172.21.15.100:0/663463900 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e4005620 con 0x7f95ec107760 2024-05-10T11:32:45.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.211+0000 7f95f404a700 1 -- 172.21.15.100:0/663463900 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95ec101a90 con 0x7f95ec107760 2024-05-10T11:32:45.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.214+0000 7f95e2ffd700 1 -- 172.21.15.100:0/663463900 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f95e4004750 con 0x7f95ec107760 2024-05-10T11:32:45.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.215+0000 7f95e2ffd700 1 --2- 172.21.15.100:0/663463900 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f95d80b2900 0x7f95d80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:45.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.215+0000 7f95f1de6700 1 --2- 172.21.15.100:0/663463900 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f95d80b2900 0x7f95d80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:45.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.215+0000 7f95e2ffd700 1 -- 172.21.15.100:0/663463900 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f95e40e2380 con 0x7f95ec107760 2024-05-10T11:32:45.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.215+0000 7f95f1de6700 1 --2- 172.21.15.100:0/663463900 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f95d80b2900 0x7f95d80b4dc0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f95e8009fd0 tx=0x7f95e800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:32:45.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.216+0000 7f95fe8ddb80 1 Processor -- start 2024-05-10T11:32:45.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.216+0000 7f95fe8ddb80 1 -- start start 2024-05-10T11:32:45.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.217+0000 7f95fe8ddb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a6f176c4d0 0x55a6f176c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:45.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.217+0000 7f95fe8ddb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6f176cdf0 0x55a6f17706f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:45.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.217+0000 7f95fe8ddb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6f1770c80 0x55a6f1771100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:45.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.217+0000 7f95fe8ddb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55a6f16be860 con 0x55a6f1770c80 2024-05-10T11:32:45.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.217+0000 7f95fe8ddb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a6f165d7c0 con 0x55a6f176cdf0 2024-05-10T11:32:45.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.217+0000 7f95fe8ddb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55a6f16ba820 con 0x55a6f176c4d0 2024-05-10T11:32:45.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.217+0000 7f95f1de6700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a6f176c4d0 0x55a6f176c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:45.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.217+0000 7f95f1de6700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a6f176c4d0 0x55a6f176c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:59168/0 (socket says 172.21.15.100:59168) 2024-05-10T11:32:45.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.217+0000 7f95f1de6700 1 -- 172.21.15.100:0/4252794673 learned_addr learned my addr 172.21.15.100:0/4252794673 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:32:45.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.217+0000 7f95f25e7700 1 --2- 172.21.15.100:0/4252794673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6f1770c80 0x55a6f1771100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:45.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.217+0000 7f95f15e5700 1 --2- 172.21.15.100:0/4252794673 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6f176cdf0 0x55a6f17706f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:45.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.217+0000 7f95f1de6700 1 -- 172.21.15.100:0/4252794673 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6f176cdf0 msgr2=0x55a6f17706f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:45.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.217+0000 7f95f1de6700 1 --2- 172.21.15.100:0/4252794673 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6f176cdf0 0x55a6f17706f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:45.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.217+0000 7f95f1de6700 1 -- 172.21.15.100:0/4252794673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6f1770c80 msgr2=0x55a6f1771100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:45.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.217+0000 7f95f1de6700 1 --2- 172.21.15.100:0/4252794673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6f1770c80 0x55a6f1771100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:45.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.217+0000 7f95f1de6700 1 -- 172.21.15.100:0/4252794673 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6f160b8d0 con 0x55a6f176c4d0 2024-05-10T11:32:45.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.218+0000 7f95f1de6700 1 --2- 172.21.15.100:0/4252794673 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a6f176c4d0 0x55a6f176c8b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f95e8009610 tx=0x7f95e801e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2b4829687048e18 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:32:45.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.218+0000 7f95d7fff700 1 -- 172.21.15.100:0/4252794673 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e802e070 con 0x55a6f176c4d0 2024-05-10T11:32:45.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.218+0000 7f95d7fff700 1 -- 172.21.15.100:0/4252794673 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f95e8020070 con 0x55a6f176c4d0 2024-05-10T11:32:45.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.218+0000 7f95d7fff700 1 -- 172.21.15.100:0/4252794673 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e8007840 con 0x55a6f176c4d0 2024-05-10T11:32:45.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.219+0000 7f95fe8ddb80 1 -- 172.21.15.100:0/4252794673 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a6f176c4d0 msgr2=0x55a6f176c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:45.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.219+0000 7f95fe8ddb80 1 --2- 172.21.15.100:0/4252794673 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a6f176c4d0 0x55a6f176c8b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f95e8009610 tx=0x7f95e801e000 comp rx=0 tx=0).stop 2024-05-10T11:32:45.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.219+0000 7f95fe8ddb80 1 -- 172.21.15.100:0/4252794673 shutdown_connections 2024-05-10T11:32:45.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.219+0000 7f95fe8ddb80 1 --2- 172.21.15.100:0/4252794673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6f1770c80 0x55a6f1771100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:45.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.219+0000 7f95fe8ddb80 1 --2- 172.21.15.100:0/4252794673 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a6f176c4d0 0x55a6f176c8b0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:45.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.219+0000 7f95fe8ddb80 1 --2- 172.21.15.100:0/4252794673 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6f176cdf0 0x55a6f17706f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:45.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.219+0000 7f95fe8ddb80 1 -- 172.21.15.100:0/4252794673 >> 172.21.15.100:0/4252794673 conn(0x55a6f1698100 msgr2=0x55a6f1766c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:32:45.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.219+0000 7f95fe8ddb80 1 -- 172.21.15.100:0/4252794673 shutdown_connections 2024-05-10T11:32:45.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.219+0000 7f95fe8ddb80 1 -- 172.21.15.100:0/4252794673 wait complete. 2024-05-10T11:32:45.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.220+0000 7f95fe8ddb80 1 -- start start 2024-05-10T11:32:45.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.220+0000 7f95fe8ddb80 1 Processor -- start 2024-05-10T11:32:45.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.220+0000 7f95fe8ddb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6f176c4d0 0x55a6f17780d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:45.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.221+0000 7f95f1de6700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6f176c4d0 0x55a6f17780d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:45.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.221+0000 7f95f1de6700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6f176c4d0 0x55a6f17780d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56988/0 (socket says 172.21.15.100:56988) 2024-05-10T11:32:45.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.221+0000 7f95f1de6700 1 -- 172.21.15.100:0/68291752 learned_addr learned my addr 172.21.15.100:0/68291752 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:32:45.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.221+0000 7f95fe8ddb80 1 --2- 172.21.15.100:0/68291752 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a6f176cdf0 0x55a6f1775470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:45.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.221+0000 7f95fe8ddb80 1 --2- 172.21.15.100:0/68291752 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6f1770c80 0x55a6f17759b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:45.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.221+0000 7f95f15e5700 1 --2- 172.21.15.100:0/68291752 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a6f176cdf0 0x55a6f1775470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:45.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.221+0000 7f95fe8ddb80 1 -- 172.21.15.100:0/68291752 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55a6f1778e30 con 0x55a6f176c4d0 2024-05-10T11:32:45.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.221+0000 7f95fe8ddb80 1 -- 172.21.15.100:0/68291752 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a6f1624c30 con 0x55a6f1770c80 2024-05-10T11:32:45.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.221+0000 7f95fe8ddb80 1 -- 172.21.15.100:0/68291752 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55a6f1778fb0 con 0x55a6f176cdf0 2024-05-10T11:32:45.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.221+0000 7f95f25e7700 1 --2- 172.21.15.100:0/68291752 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6f1770c80 0x55a6f17759b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:45.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.222+0000 7f95f25e7700 1 -- 172.21.15.100:0/68291752 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a6f176cdf0 msgr2=0x55a6f1775470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:45.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.222+0000 7f95f25e7700 1 --2- 172.21.15.100:0/68291752 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a6f176cdf0 0x55a6f1775470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:45.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.222+0000 7f95f25e7700 1 -- 172.21.15.100:0/68291752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6f176c4d0 msgr2=0x55a6f17780d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:45.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.222+0000 7f95f25e7700 1 --2- 172.21.15.100:0/68291752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6f176c4d0 0x55a6f17780d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:45.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.222+0000 7f95f25e7700 1 -- 172.21.15.100:0/68291752 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6f165a690 con 0x55a6f1770c80 2024-05-10T11:32:45.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.222+0000 7f95f15e5700 1 --2- 172.21.15.100:0/68291752 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a6f176cdf0 0x55a6f1775470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:32:45.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.222+0000 7f95f1de6700 1 --2- 172.21.15.100:0/68291752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6f176c4d0 0x55a6f17780d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:32:45.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.222+0000 7f95f25e7700 1 --2- 172.21.15.100:0/68291752 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6f1770c80 0x55a6f17759b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f95e40dfb20 tx=0x7f95e40a9dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46bac35d03f65f08 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:32:45.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.222+0000 7f95fe8ddb80 10 client.0 _ll_register_callbacks cb 0x55a6f1656920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:32:45.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.222+0000 7f95fe8ddb80 1 -- 172.21.15.100:0/68291752 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95e8020070 con 0x55a6f1770c80 2024-05-10T11:32:45.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.222+0000 7f95d57fa700 10 client.27626 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:32:45.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.222+0000 7f95d57fa700 1 -- 172.21.15.100:0/68291752 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e401c040 con 0x55a6f1770c80 2024-05-10T11:32:45.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.222+0000 7f95d57fa700 1 -- 172.21.15.100:0/68291752 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f95e40a98a0 con 0x55a6f1770c80 2024-05-10T11:32:45.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.222+0000 7f95d57fa700 1 -- 172.21.15.100:0/68291752 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f95e40a9aa0 con 0x55a6f1770c80 2024-05-10T11:32:45.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.223+0000 7f95fe8ddb80 10 client.27626 fetch_fsmap learned FSMap version 37 2024-05-10T11:32:45.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.223+0000 7f95fe8ddb80 1 -- 172.21.15.100:0/68291752 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({fsmap=37}) -- 0x55a6f17832f0 con 0x55a6f1770c80 2024-05-10T11:32:45.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.223+0000 7f95d57fa700 1 -- 172.21.15.100:0/68291752 <== mon.2 v2:172.21.15.178:3300/0 4 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f95e40b2010 con 0x55a6f1770c80 2024-05-10T11:32:45.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.223+0000 7f95fe8ddb80 10 client.27626 fetch_fsmap finished waiting for FSMap version 37 2024-05-10T11:32:45.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.223+0000 7f95fe8ddb80 10 client.27626 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-10T11:32:45.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.223+0000 7f95fe8ddb80 20 client.27626 populate_metadata read hostname 'smithi100' 2024-05-10T11:32:45.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.223+0000 7f95fe8ddb80 1 --2- 172.21.15.100:0/68291752 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55a6f1785940 0x55a6f1787e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:45.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.223+0000 7f95fe8ddb80 4 client.27626 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:32:45.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.223+0000 7f95fe8ddb80 1 -- 172.21.15.100:0/68291752 --> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6f1738650 con 0x55a6f1785940 2024-05-10T11:32:45.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.223+0000 7f95f1de6700 1 --2- 172.21.15.100:0/68291752 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55a6f1785940 0x55a6f1787e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:45.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.224+0000 7f95d57fa700 10 client.27626 ms_handle_connect on v2:172.21.15.188:6836/1943342620 2024-05-10T11:32:45.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.224+0000 7f95f1de6700 1 --2- 172.21.15.100:0/68291752 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55a6f1785940 0x55a6f1787e00 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ca00a467b853dc1c server_cookie=3711de9e1015ef75 in_seq=0 out_seq=0 2024-05-10T11:32:45.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.225+0000 7f95d57fa700 1 -- 172.21.15.100:0/68291752 <== mds.2 v2:172.21.15.188:6836/1943342620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a6f1738650 con 0x55a6f1785940 2024-05-10T11:32:45.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.225+0000 7f95d57fa700 10 client.27626 handle_command_reply: tid=0 2024-05-10T11:32:45.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.225+0000 7f95fe8ddb80 1 client.27626 shutdown 2024-05-10T11:32:45.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.226+0000 7f95fe8ddb80 1 -- 172.21.15.100:0/68291752 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6f1770c80 msgr2=0x55a6f17759b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:45.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.226+0000 7f95fe8ddb80 1 --2- 172.21.15.100:0/68291752 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6f1770c80 0x55a6f17759b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f95e40dfb20 tx=0x7f95e40a9dc0 comp rx=0 tx=0).stop 2024-05-10T11:32:45.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.226+0000 7f95fe8ddb80 1 -- 172.21.15.100:0/68291752 shutdown_connections 2024-05-10T11:32:45.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.226+0000 7f95fe8ddb80 1 --2- 172.21.15.100:0/68291752 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55a6f1785940 0x55a6f1787e00 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:45.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.226+0000 7f95fe8ddb80 1 --2- 172.21.15.100:0/68291752 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6f1770c80 0x55a6f17759b0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:45.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.226+0000 7f95fe8ddb80 1 --2- 172.21.15.100:0/68291752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6f176c4d0 0x55a6f17780d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:45.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.226+0000 7f95fe8ddb80 1 --2- 172.21.15.100:0/68291752 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a6f176cdf0 0x55a6f1775470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:45.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.226+0000 7f95d57fa700 0 client.27626 ms_handle_reset on v2:172.21.15.188:6836/1943342620 2024-05-10T11:32:45.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.226+0000 7f95fe8ddb80 1 -- 172.21.15.100:0/68291752 >> 172.21.15.100:0/68291752 conn(0x55a6f1698100 msgr2=0x55a6f1761720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:32:45.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.226+0000 7f95fe8ddb80 1 -- 172.21.15.100:0/68291752 shutdown_connections 2024-05-10T11:32:45.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.226+0000 7f95fe8ddb80 1 -- 172.21.15.100:0/68291752 wait complete. 2024-05-10T11:32:45.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.226+0000 7f95fe8ddb80 20 client.27626 trim_cache size 0 max 16384 2024-05-10T11:32:45.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.250+0000 7f95fe8ddb80 1 Processor -- start 2024-05-10T11:32:45.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.250+0000 7f95fe8ddb80 1 -- start start 2024-05-10T11:32:45.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.250+0000 7f95fe8ddb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6f176c4d0 0x55a6f1781ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:45.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.250+0000 7f95fe8ddb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a6f176cdf0 0x55a6f1774470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:45.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.250+0000 7f95fe8ddb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6f1770c80 0x55a6f176e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:45.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.250+0000 7f95fe8ddb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f95ec1b3b80 con 0x55a6f1770c80 2024-05-10T11:32:45.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.250+0000 7f95fe8ddb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f95ec1b1380 con 0x55a6f176c4d0 2024-05-10T11:32:45.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.250+0000 7f95fe8ddb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f95ec1af1d0 con 0x55a6f176cdf0 2024-05-10T11:32:45.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.250+0000 7f95f25e7700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6f1770c80 0x55a6f176e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:45.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.250+0000 7f95f1de6700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6f176c4d0 0x55a6f1781ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:45.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.250+0000 7f95f15e5700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a6f176cdf0 0x55a6f1774470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:45.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.250+0000 7f95f15e5700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a6f176cdf0 0x55a6f1774470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:59182/0 (socket says 172.21.15.100:59182) 2024-05-10T11:32:45.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.250+0000 7f95f1de6700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6f176c4d0 0x55a6f1781ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:53094/0 (socket says 172.21.15.100:53094) 2024-05-10T11:32:45.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.250+0000 7f95f15e5700 1 -- 172.21.15.100:0/3687594679 learned_addr learned my addr 172.21.15.100:0/3687594679 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:32:45.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.251+0000 7f95f15e5700 1 -- 172.21.15.100:0/3687594679 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6f176c4d0 msgr2=0x55a6f1781ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:45.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.251+0000 7f95f15e5700 1 --2- 172.21.15.100:0/3687594679 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6f176c4d0 0x55a6f1781ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:45.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.251+0000 7f95f15e5700 1 -- 172.21.15.100:0/3687594679 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6f1770c80 msgr2=0x55a6f176e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:45.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.251+0000 7f95f15e5700 1 --2- 172.21.15.100:0/3687594679 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6f1770c80 0x55a6f176e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:45.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.251+0000 7f95f15e5700 1 -- 172.21.15.100:0/3687594679 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6f1775cc0 con 0x55a6f176cdf0 2024-05-10T11:32:45.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.251+0000 7f95f25e7700 1 --2- 172.21.15.100:0/3687594679 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6f1770c80 0x55a6f176e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:32:45.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.251+0000 7f95f15e5700 1 --2- 172.21.15.100:0/3687594679 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a6f176cdf0 0x55a6f1774470 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f95dc012550 tx=0x7f95dc012860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27fa7fb03dc0a52e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:32:45.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.251+0000 7f95d77fe700 1 -- 172.21.15.100:0/3687594679 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95dc01f070 con 0x55a6f176cdf0 2024-05-10T11:32:45.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.251+0000 7f95d77fe700 1 -- 172.21.15.100:0/3687594679 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f95dc0044b0 con 0x55a6f176cdf0 2024-05-10T11:32:45.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.251+0000 7f95d77fe700 1 -- 172.21.15.100:0/3687594679 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95dc005080 con 0x55a6f176cdf0 2024-05-10T11:32:45.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.252+0000 7f95fe8ddb80 1 -- 172.21.15.100:0/3687594679 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a6f176cdf0 msgr2=0x55a6f1774470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:45.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.252+0000 7f95fe8ddb80 1 --2- 172.21.15.100:0/3687594679 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a6f176cdf0 0x55a6f1774470 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f95dc012550 tx=0x7f95dc012860 comp rx=0 tx=0).stop 2024-05-10T11:32:45.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.252+0000 7f95fe8ddb80 1 -- 172.21.15.100:0/3687594679 shutdown_connections 2024-05-10T11:32:45.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.252+0000 7f95fe8ddb80 1 --2- 172.21.15.100:0/3687594679 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6f1770c80 0x55a6f176e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:45.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.252+0000 7f95fe8ddb80 1 --2- 172.21.15.100:0/3687594679 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6f176c4d0 0x55a6f1781ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:45.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.252+0000 7f95fe8ddb80 1 --2- 172.21.15.100:0/3687594679 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a6f176cdf0 0x55a6f1774470 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:45.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.252+0000 7f95fe8ddb80 1 -- 172.21.15.100:0/3687594679 >> 172.21.15.100:0/3687594679 conn(0x55a6f1698100 msgr2=0x55a6f1773d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:32:45.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.252+0000 7f95fe8ddb80 1 -- 172.21.15.100:0/3687594679 shutdown_connections 2024-05-10T11:32:45.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.252+0000 7f95fe8ddb80 1 -- 172.21.15.100:0/3687594679 wait complete. 2024-05-10T11:32:45.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.253+0000 7f95fe8ddb80 1 -- start start 2024-05-10T11:32:45.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.253+0000 7f95fe8ddb80 1 Processor -- start 2024-05-10T11:32:45.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.253+0000 7f95fe8ddb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a6f176c4d0 0x55a6f1779200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:45.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.253+0000 7f95fe8ddb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6f176cdf0 0x55a6f1779740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:45.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.253+0000 7f95fe8ddb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6f1770c80 0x55a6f17ee2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:45.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.253+0000 7f95fe8ddb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f95ec1af840 con 0x55a6f176cdf0 2024-05-10T11:32:45.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.253+0000 7f95fe8ddb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f95ec1b0cb0 con 0x55a6f1770c80 2024-05-10T11:32:45.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.253+0000 7f95fe8ddb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f95ec1aff10 con 0x55a6f176c4d0 2024-05-10T11:32:45.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.253+0000 7f95fe8ddb80 10 client.0 _ll_register_callbacks cb 0x55a6f1656920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:32:45.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.254+0000 7f95f1de6700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a6f176c4d0 0x55a6f1779200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:45.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.254+0000 7f95f1de6700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a6f176c4d0 0x55a6f1779200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:59188/0 (socket says 172.21.15.100:59188) 2024-05-10T11:32:45.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.254+0000 7f95f1de6700 1 -- 172.21.15.100:0/2708509197 learned_addr learned my addr 172.21.15.100:0/2708509197 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:32:45.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.254+0000 7f95f15e5700 1 --2- 172.21.15.100:0/2708509197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6f176cdf0 0x55a6f1779740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:45.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.254+0000 7f95f25e7700 1 --2- 172.21.15.100:0/2708509197 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6f1770c80 0x55a6f17ee2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:45.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.254+0000 7f95f1de6700 1 -- 172.21.15.100:0/2708509197 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6f1770c80 msgr2=0x55a6f17ee2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:45.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.254+0000 7f95f1de6700 1 --2- 172.21.15.100:0/2708509197 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6f1770c80 0x55a6f17ee2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:45.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.254+0000 7f95f1de6700 1 -- 172.21.15.100:0/2708509197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6f176cdf0 msgr2=0x55a6f1779740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:45.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.254+0000 7f95f1de6700 1 --2- 172.21.15.100:0/2708509197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6f176cdf0 0x55a6f1779740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:45.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.254+0000 7f95f1de6700 1 -- 172.21.15.100:0/2708509197 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6f1779fe0 con 0x55a6f176c4d0 2024-05-10T11:32:45.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.254+0000 7f95f25e7700 1 --2- 172.21.15.100:0/2708509197 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6f1770c80 0x55a6f17ee2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:32:45.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.254+0000 7f95f1de6700 1 --2- 172.21.15.100:0/2708509197 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a6f176c4d0 0x55a6f1779200 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f95e800b010 tx=0x7f95e8005970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b90aa2cdee68586 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:32:45.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.254+0000 7f95d57fa700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:32:45.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.254+0000 7f95d57fa700 1 -- 172.21.15.100:0/2708509197 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e802e040 con 0x55a6f176c4d0 2024-05-10T11:32:45.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.254+0000 7f95fe8ddb80 1 -- 172.21.15.100:0/2708509197 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95dc0044b0 con 0x55a6f176c4d0 2024-05-10T11:32:45.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.255+0000 7f95d57fa700 1 -- 172.21.15.100:0/2708509197 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f95e803c1c0 con 0x55a6f176c4d0 2024-05-10T11:32:45.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.255+0000 7f95d57fa700 1 -- 172.21.15.100:0/2708509197 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f95e803c3c0 con 0x55a6f176c4d0 2024-05-10T11:32:45.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.255+0000 7f95fe8ddb80 10 client.27724 fetch_fsmap learned FSMap version 37 2024-05-10T11:32:45.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.255+0000 7f95fe8ddb80 1 -- 172.21.15.100:0/2708509197 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55a6f17ef8c0 con 0x55a6f176c4d0 2024-05-10T11:32:45.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.255+0000 7f95d57fa700 1 -- 172.21.15.100:0/2708509197 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f95dc0044b0 con 0x55a6f176c4d0 2024-05-10T11:32:45.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.255+0000 7f95d57fa700 1 -- 172.21.15.100:0/2708509197 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f95e8034040 con 0x55a6f176c4d0 2024-05-10T11:32:45.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.255+0000 7f95fe8ddb80 10 client.27724 fetch_fsmap finished waiting for FSMap version 37 2024-05-10T11:32:45.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.255+0000 7f95fe8ddb80 10 client.27724 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-10T11:32:45.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.255+0000 7f95fe8ddb80 20 client.27724 populate_metadata read hostname 'smithi100' 2024-05-10T11:32:45.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.255+0000 7f95fe8ddb80 1 --2- 172.21.15.100:0/2708509197 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55a6f17efb50 0x55a6f17f1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:45.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.255+0000 7f95fe8ddb80 4 client.27724 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:32:45.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.255+0000 7f95fe8ddb80 1 -- 172.21.15.100:0/2708509197 --> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a6f16cc240 con 0x55a6f17efb50 2024-05-10T11:32:45.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.256+0000 7f95f15e5700 1 --2- 172.21.15.100:0/2708509197 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55a6f17efb50 0x55a6f17f1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:45.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.256+0000 7f95f15e5700 1 --2- 172.21.15.100:0/2708509197 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55a6f17efb50 0x55a6f17f1f40 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4380c1ae66555222 server_cookie=b2fc97c7feb1fc99 in_seq=0 out_seq=0 2024-05-10T11:32:45.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.256+0000 7f95d57fa700 10 client.27724 ms_handle_connect on v2:172.21.15.188:6836/1943342620 2024-05-10T11:32:45.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.259+0000 7f95d57fa700 1 -- 172.21.15.100:0/2708509197 <== mds.2 v2:172.21.15.188:6836/1943342620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a6f16cc240 con 0x55a6f17efb50 2024-05-10T11:32:45.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.259+0000 7f95d57fa700 10 client.27724 handle_command_reply: tid=0 2024-05-10T11:32:45.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.259+0000 7f95fe8ddb80 1 client.27724 shutdown 2024-05-10T11:32:45.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.260+0000 7f95fe8ddb80 1 -- 172.21.15.100:0/2708509197 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a6f176c4d0 msgr2=0x55a6f1779200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:45.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.260+0000 7f95fe8ddb80 1 --2- 172.21.15.100:0/2708509197 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a6f176c4d0 0x55a6f1779200 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f95e800b010 tx=0x7f95e8005970 comp rx=0 tx=0).stop 2024-05-10T11:32:45.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.260+0000 7f95fe8ddb80 1 -- 172.21.15.100:0/2708509197 shutdown_connections 2024-05-10T11:32:45.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.260+0000 7f95fe8ddb80 1 --2- 172.21.15.100:0/2708509197 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55a6f17efb50 0x55a6f17f1f40 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:45.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.261+0000 7f95fe8ddb80 1 --2- 172.21.15.100:0/2708509197 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6f1770c80 0x55a6f17ee2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:45.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.261+0000 7f95fe8ddb80 1 --2- 172.21.15.100:0/2708509197 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a6f176c4d0 0x55a6f1779200 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:45.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.261+0000 7f95fe8ddb80 1 --2- 172.21.15.100:0/2708509197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6f176cdf0 0x55a6f1779740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:45.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.261+0000 7f95fe8ddb80 1 -- 172.21.15.100:0/2708509197 >> 172.21.15.100:0/2708509197 conn(0x55a6f1698100 msgr2=0x55a6f17745f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:32:45.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.261+0000 7f95d57fa700 0 client.27724 ms_handle_reset on v2:172.21.15.188:6836/1943342620 2024-05-10T11:32:45.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.261+0000 7f95fe8ddb80 1 -- 172.21.15.100:0/2708509197 shutdown_connections 2024-05-10T11:32:45.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.261+0000 7f95fe8ddb80 1 -- 172.21.15.100:0/2708509197 wait complete. 2024-05-10T11:32:45.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.261+0000 7f95fe8ddb80 20 client.27724 trim_cache size 0 max 16384 2024-05-10T11:32:45.264 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:32:45.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.262+0000 7f95f404a700 1 -- 172.21.15.100:0/663463900 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f95d80b2900 msgr2=0x7f95d80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:45.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.262+0000 7f95f404a700 1 --2- 172.21.15.100:0/663463900 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f95d80b2900 0x7f95d80b4dc0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f95e8009fd0 tx=0x7f95e800b040 comp rx=0 tx=0).stop 2024-05-10T11:32:45.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.262+0000 7f95f404a700 1 -- 172.21.15.100:0/663463900 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95ec107760 msgr2=0x7f95ec198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:45.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.262+0000 7f95f404a700 1 --2- 172.21.15.100:0/663463900 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95ec107760 0x7f95ec198750 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f95e400e970 tx=0x7f95e400ec80 comp rx=0 tx=0).stop 2024-05-10T11:32:45.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.262+0000 7f95f404a700 1 -- 172.21.15.100:0/663463900 shutdown_connections 2024-05-10T11:32:45.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.262+0000 7f95f404a700 1 --2- 172.21.15.100:0/663463900 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f95d80b2900 0x7f95d80b4dc0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:45.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.262+0000 7f95f404a700 1 --2- 172.21.15.100:0/663463900 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95ec107760 0x7f95ec198750 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:45.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.262+0000 7f95f404a700 1 --2- 172.21.15.100:0/663463900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95ec102e00 0x7f95ec19e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:45.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.262+0000 7f95f404a700 1 --2- 172.21.15.100:0/663463900 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95ec1037b0 0x7f95ec19ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:45.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.262+0000 7f95f404a700 1 -- 172.21.15.100:0/663463900 >> 172.21.15.100:0/663463900 conn(0x7f95ec078a10 msgr2=0x7f95ec0ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:32:45.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.263+0000 7f95f404a700 1 -- 172.21.15.100:0/663463900 shutdown_connections 2024-05-10T11:32:45.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:45.263+0000 7f95f404a700 1 -- 172.21.15.100:0/663463900 wait complete. 2024-05-10T11:32:45.907 INFO:tasks.workunit.client.0.smithi100.stdout: 10 6684 13.56 MB/sec warmup 39 sec latency 62.651 ms 2024-05-10T11:32:46.030 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-10T11:32:46.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:32:45 smithi178 ceph-mon[33359]: pgmap v852: 129 pgs: 129 active+clean; 341 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 3.2 MiB/s wr, 264 op/s 2024-05-10T11:32:46.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:32:45 smithi188 ceph-mon[34075]: pgmap v852: 129 pgs: 129 active+clean; 341 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 3.2 MiB/s wr, 264 op/s 2024-05-10T11:32:46.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:32:45 smithi100 ceph-mon[30296]: pgmap v852: 129 pgs: 129 active+clean; 341 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 3.2 MiB/s wr, 264 op/s 2024-05-10T11:32:46.378 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:32:46.907 INFO:tasks.workunit.client.0.smithi100.stdout: 10 6897 13.69 MB/sec warmup 40 sec latency 64.416 ms 2024-05-10T11:32:47.907 INFO:tasks.workunit.client.0.smithi100.stdout: 10 7141 13.68 MB/sec warmup 41 sec latency 77.252 ms 2024-05-10T11:32:48.091 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:32:47 smithi100 ceph-mon[30296]: pgmap v853: 129 pgs: 129 active+clean; 345 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 560 KiB/s rd, 3.4 MiB/s wr, 258 op/s 2024-05-10T11:32:48.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:32:47 smithi178 ceph-mon[33359]: pgmap v853: 129 pgs: 129 active+clean; 345 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 560 KiB/s rd, 3.4 MiB/s wr, 258 op/s 2024-05-10T11:32:48.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.252+0000 7ff3be49d700 1 -- 172.21.15.100:0/3826509112 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3b8108400 msgr2=0x7ff3b81087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:48.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.252+0000 7ff3be49d700 1 --2- 172.21.15.100:0/3826509112 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3b8108400 0x7ff3b81087e0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7ff3b000b0a0 tx=0x7ff3b000b3b0 comp rx=0 tx=0).stop 2024-05-10T11:32:48.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.252+0000 7ff3be49d700 1 -- 172.21.15.100:0/3826509112 shutdown_connections 2024-05-10T11:32:48.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.252+0000 7ff3be49d700 1 --2- 172.21.15.100:0/3826509112 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3b8102d50 0x7ff3b810f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:48.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.252+0000 7ff3be49d700 1 --2- 172.21.15.100:0/3826509112 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3b8108400 0x7ff3b81087e0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:48.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.252+0000 7ff3be49d700 1 --2- 172.21.15.100:0/3826509112 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff3b8102390 0x7ff3b8102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:48.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.252+0000 7ff3be49d700 1 -- 172.21.15.100:0/3826509112 >> 172.21.15.100:0/3826509112 conn(0x7ff3b8079310 msgr2=0x7ff3b8079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:32:48.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.252+0000 7ff3be49d700 1 -- 172.21.15.100:0/3826509112 shutdown_connections 2024-05-10T11:32:48.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.252+0000 7ff3be49d700 1 -- 172.21.15.100:0/3826509112 wait complete. 2024-05-10T11:32:48.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.253+0000 7ff3be49d700 1 Processor -- start 2024-05-10T11:32:48.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.253+0000 7ff3be49d700 1 -- start start 2024-05-10T11:32:48.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.254+0000 7ff3be49d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff3b8102390 0x7ff3b819cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:48.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.254+0000 7ff3be49d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3b8102d50 0x7ff3b819d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:48.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.254+0000 7ff3be49d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3b8108400 0x7ff3b81a79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:48.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.254+0000 7ff3be49d700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b806c820 con 0x7ff3b8108400 2024-05-10T11:32:48.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.254+0000 7ff3be49d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b806c6a0 con 0x7ff3b8102390 2024-05-10T11:32:48.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.254+0000 7ff3be49d700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b806c9a0 con 0x7ff3b8102d50 2024-05-10T11:32:48.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.254+0000 7ff3b7fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff3b8102390 0x7ff3b819cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:48.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.254+0000 7ff3bca3a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3b8108400 0x7ff3b81a79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:48.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.254+0000 7ff3b77fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3b8102d50 0x7ff3b819d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:48.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.254+0000 7ff3b7fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff3b8102390 0x7ff3b819cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:40794/0 (socket says 172.21.15.100:40794) 2024-05-10T11:32:48.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.254+0000 7ff3b7fff700 1 -- 172.21.15.100:0/928935277 learned_addr learned my addr 172.21.15.100:0/928935277 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:32:48.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.255+0000 7ff3bca3a700 1 -- 172.21.15.100:0/928935277 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3b8102d50 msgr2=0x7ff3b819d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:48.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.255+0000 7ff3bca3a700 1 --2- 172.21.15.100:0/928935277 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3b8102d50 0x7ff3b819d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:48.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.255+0000 7ff3bca3a700 1 -- 172.21.15.100:0/928935277 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff3b8102390 msgr2=0x7ff3b819cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:48.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.255+0000 7ff3bca3a700 1 --2- 172.21.15.100:0/928935277 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff3b8102390 0x7ff3b819cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:48.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.255+0000 7ff3bca3a700 1 -- 172.21.15.100:0/928935277 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3b81a7f80 con 0x7ff3b8108400 2024-05-10T11:32:48.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.255+0000 7ff3b7fff700 1 --2- 172.21.15.100:0/928935277 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff3b8102390 0x7ff3b819cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:32:48.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.255+0000 7ff3bca3a700 1 --2- 172.21.15.100:0/928935277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3b8108400 0x7ff3b81a79c0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7ff3a800d720 tx=0x7ff3a800da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:32:48.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.255+0000 7ff3b57fa700 1 -- 172.21.15.100:0/928935277 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a8016070 con 0x7ff3b8108400 2024-05-10T11:32:48.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.255+0000 7ff3be49d700 1 -- 172.21.15.100:0/928935277 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3b81a8210 con 0x7ff3b8108400 2024-05-10T11:32:48.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.255+0000 7ff3b57fa700 1 -- 172.21.15.100:0/928935277 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff3a800f690 con 0x7ff3b8108400 2024-05-10T11:32:48.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.255+0000 7ff3b57fa700 1 -- 172.21.15.100:0/928935277 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a80038c0 con 0x7ff3b8108400 2024-05-10T11:32:48.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.255+0000 7ff3be49d700 1 -- 172.21.15.100:0/928935277 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3b81a8750 con 0x7ff3b8108400 2024-05-10T11:32:48.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.258+0000 7ff3b57fa700 1 -- 172.21.15.100:0/928935277 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7ff3a8004220 con 0x7ff3b8108400 2024-05-10T11:32:48.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.259+0000 7ff3b57fa700 1 --2- 172.21.15.100:0/928935277 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff3a00b2900 0x7ff3a00b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:48.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.259+0000 7ff3b7fff700 1 --2- 172.21.15.100:0/928935277 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff3a00b2900 0x7ff3a00b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:48.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.259+0000 7ff3b57fa700 1 -- 172.21.15.100:0/928935277 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7ff3a80e1c20 con 0x7ff3b8108400 2024-05-10T11:32:48.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.260+0000 7ff3b7fff700 1 --2- 172.21.15.100:0/928935277 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff3a00b2900 0x7ff3a00b4dc0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ff3b000b780 tx=0x7ff3b0009c40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:32:48.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.260+0000 7ff3c8d30b80 1 Processor -- start 2024-05-10T11:32:48.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.260+0000 7ff3c8d30b80 1 -- start start 2024-05-10T11:32:48.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.260+0000 7ff3c8d30b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557b396814d0 0x557b396818b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:48.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.260+0000 7ff3c8d30b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b39681df0 0x557b396856f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:48.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.261+0000 7ff3c8d30b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557b39685c80 0x557b39686100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:48.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.261+0000 7ff3c8d30b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x557b395d3860 con 0x557b39681df0 2024-05-10T11:32:48.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.261+0000 7ff3c8d30b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x557b395727c0 con 0x557b396814d0 2024-05-10T11:32:48.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.261+0000 7ff3c8d30b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x557b395cf820 con 0x557b39685c80 2024-05-10T11:32:48.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.261+0000 7ff3b7fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557b396814d0 0x557b396818b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:48.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.261+0000 7ff3b77fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b39681df0 0x557b396856f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:48.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.261+0000 7ff3b77fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b39681df0 0x557b396856f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48510/0 (socket says 172.21.15.100:48510) 2024-05-10T11:32:48.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.261+0000 7ff3b77fe700 1 -- 172.21.15.100:0/2748917405 learned_addr learned my addr 172.21.15.100:0/2748917405 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:32:48.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.261+0000 7ff3b7fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557b396814d0 0x557b396818b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:40808/0 (socket says 172.21.15.100:40808) 2024-05-10T11:32:48.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.261+0000 7ff3bca3a700 1 --2- 172.21.15.100:0/2748917405 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557b39685c80 0x557b39686100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:48.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.261+0000 7ff3b77fe700 1 -- 172.21.15.100:0/2748917405 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557b39685c80 msgr2=0x557b39686100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:48.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.261+0000 7ff3b77fe700 1 --2- 172.21.15.100:0/2748917405 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557b39685c80 0x557b39686100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:48.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.261+0000 7ff3b77fe700 1 -- 172.21.15.100:0/2748917405 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557b396814d0 msgr2=0x557b396818b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:48.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.261+0000 7ff3b77fe700 1 --2- 172.21.15.100:0/2748917405 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557b396814d0 0x557b396818b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:48.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.261+0000 7ff3b77fe700 1 -- 172.21.15.100:0/2748917405 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b395208d0 con 0x557b39681df0 2024-05-10T11:32:48.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.261+0000 7ff3b77fe700 1 --2- 172.21.15.100:0/2748917405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b39681df0 0x557b396856f0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac009cf0 tx=0x7ff3ac011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76d131d2e4f9221d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:32:48.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.261+0000 7ff39e7fc700 1 -- 172.21.15.100:0/2748917405 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3ac011cc0 con 0x557b39681df0 2024-05-10T11:32:48.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.262+0000 7ff39e7fc700 1 -- 172.21.15.100:0/2748917405 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff3ac0044e0 con 0x557b39681df0 2024-05-10T11:32:48.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.262+0000 7ff39e7fc700 1 -- 172.21.15.100:0/2748917405 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3ac0193e0 con 0x557b39681df0 2024-05-10T11:32:48.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.262+0000 7ff3c8d30b80 1 -- 172.21.15.100:0/2748917405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b39681df0 msgr2=0x557b396856f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:48.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.262+0000 7ff3c8d30b80 1 --2- 172.21.15.100:0/2748917405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b39681df0 0x557b396856f0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac009cf0 tx=0x7ff3ac011440 comp rx=0 tx=0).stop 2024-05-10T11:32:48.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.263+0000 7ff3c8d30b80 1 -- 172.21.15.100:0/2748917405 shutdown_connections 2024-05-10T11:32:48.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.263+0000 7ff3c8d30b80 1 --2- 172.21.15.100:0/2748917405 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557b39685c80 0x557b39686100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:48.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.263+0000 7ff3c8d30b80 1 --2- 172.21.15.100:0/2748917405 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557b396814d0 0x557b396818b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:48.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.263+0000 7ff3c8d30b80 1 --2- 172.21.15.100:0/2748917405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b39681df0 0x557b396856f0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:48.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.263+0000 7ff3c8d30b80 1 -- 172.21.15.100:0/2748917405 >> 172.21.15.100:0/2748917405 conn(0x557b395ad100 msgr2=0x557b3967bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:32:48.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.263+0000 7ff3c8d30b80 1 -- 172.21.15.100:0/2748917405 shutdown_connections 2024-05-10T11:32:48.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.263+0000 7ff3c8d30b80 1 -- 172.21.15.100:0/2748917405 wait complete. 2024-05-10T11:32:48.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.263+0000 7ff3c8d30b80 1 -- start start 2024-05-10T11:32:48.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.263+0000 7ff3c8d30b80 1 Processor -- start 2024-05-10T11:32:48.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.264+0000 7ff3c8d30b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557b396814d0 0x557b3968d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:48.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.264+0000 7ff3c8d30b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b39681df0 0x557b3968a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:48.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.264+0000 7ff3c8d30b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557b39685c80 0x557b3968a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:48.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.264+0000 7ff3c8d30b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x557b3968de90 con 0x557b39681df0 2024-05-10T11:32:48.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.264+0000 7ff3c8d30b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x557b39539c30 con 0x557b396814d0 2024-05-10T11:32:48.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.264+0000 7ff3b77fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b39681df0 0x557b3968a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:48.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.264+0000 7ff3c8d30b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x557b3968e010 con 0x557b39685c80 2024-05-10T11:32:48.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.264+0000 7ff3b77fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b39681df0 0x557b3968a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48524/0 (socket says 172.21.15.100:48524) 2024-05-10T11:32:48.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.264+0000 7ff3b77fe700 1 -- 172.21.15.100:0/865166380 learned_addr learned my addr 172.21.15.100:0/865166380 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:32:48.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.264+0000 7ff3b7fff700 1 --2- 172.21.15.100:0/865166380 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557b396814d0 0x557b3968d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:48.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.264+0000 7ff3c8d30b80 10 client.0 _ll_register_callbacks cb 0x557b3956b920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:32:48.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.264+0000 7ff3bca3a700 1 --2- 172.21.15.100:0/865166380 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557b39685c80 0x557b3968a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:48.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.265+0000 7ff3b77fe700 1 -- 172.21.15.100:0/865166380 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557b39685c80 msgr2=0x557b3968a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:48.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.265+0000 7ff3b77fe700 1 --2- 172.21.15.100:0/865166380 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557b39685c80 0x557b3968a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:48.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.265+0000 7ff3b77fe700 1 -- 172.21.15.100:0/865166380 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557b396814d0 msgr2=0x557b3968d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:48.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.265+0000 7ff3b77fe700 1 --2- 172.21.15.100:0/865166380 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557b396814d0 0x557b3968d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:48.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.265+0000 7ff3b77fe700 1 -- 172.21.15.100:0/865166380 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b3956f690 con 0x557b39681df0 2024-05-10T11:32:48.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.265+0000 7ff3b77fe700 1 --2- 172.21.15.100:0/865166380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b39681df0 0x557b3968a300 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac000c00 tx=0x7ff3ac003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96e3dd9238426a2b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:32:48.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.265+0000 7ff3977fe700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:32:48.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.265+0000 7ff3977fe700 1 -- 172.21.15.100:0/865166380 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3ac027030 con 0x557b39681df0 2024-05-10T11:32:48.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.265+0000 7ff3977fe700 1 -- 172.21.15.100:0/865166380 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7ff3ac01f050 con 0x557b39681df0 2024-05-10T11:32:48.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.265+0000 7ff3c8d30b80 1 -- 172.21.15.100:0/865166380 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3ac0044e0 con 0x557b39681df0 2024-05-10T11:32:48.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.266+0000 7ff3977fe700 1 -- 172.21.15.100:0/865166380 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7ff3ac031050 con 0x557b39681df0 2024-05-10T11:32:48.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.266+0000 7ff3c8d30b80 10 client.18303 fetch_fsmap learned FSMap version 37 2024-05-10T11:32:48.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.266+0000 7ff3c8d30b80 1 -- 172.21.15.100:0/865166380 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x557b39698680 con 0x557b39681df0 2024-05-10T11:32:48.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.266+0000 7ff3977fe700 1 -- 172.21.15.100:0/865166380 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff3ac0044e0 con 0x557b39681df0 2024-05-10T11:32:48.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.266+0000 7ff3977fe700 1 -- 172.21.15.100:0/865166380 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7ff3ac018070 con 0x557b39681df0 2024-05-10T11:32:48.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.266+0000 7ff3c8d30b80 10 client.18303 fetch_fsmap finished waiting for FSMap version 37 2024-05-10T11:32:48.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.266+0000 7ff3c8d30b80 10 client.18303 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-10T11:32:48.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.266+0000 7ff3c8d30b80 20 client.18303 populate_metadata read hostname 'smithi100' 2024-05-10T11:32:48.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.266+0000 7ff3c8d30b80 1 --2- 172.21.15.100:0/865166380 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x557b3969ac60 0x557b3969d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:48.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.266+0000 7ff3c8d30b80 4 client.18303 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:32:48.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.266+0000 7ff3c8d30b80 1 -- 172.21.15.100:0/865166380 --> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b3964d650 con 0x557b3969ac60 2024-05-10T11:32:48.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.267+0000 7ff3b7fff700 1 --2- 172.21.15.100:0/865166380 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x557b3969ac60 0x557b3969d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:48.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.267+0000 7ff3977fe700 10 client.18303 ms_handle_connect on v2:172.21.15.178:6838/3760985404 2024-05-10T11:32:48.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.267+0000 7ff3b7fff700 1 --2- 172.21.15.100:0/865166380 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x557b3969ac60 0x557b3969d050 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e234fc1780549f76 server_cookie=5784b16533239836 in_seq=0 out_seq=0 2024-05-10T11:32:48.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.268+0000 7ff3977fe700 1 -- 172.21.15.100:0/865166380 <== mds.1 v2:172.21.15.178:6838/3760985404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557b3964d650 con 0x557b3969ac60 2024-05-10T11:32:48.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.268+0000 7ff3977fe700 10 client.18303 handle_command_reply: tid=0 2024-05-10T11:32:48.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.268+0000 7ff3c8d30b80 1 client.18303 shutdown 2024-05-10T11:32:48.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.268+0000 7ff3c8d30b80 1 -- 172.21.15.100:0/865166380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b39681df0 msgr2=0x557b3968a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:48.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.268+0000 7ff3c8d30b80 1 --2- 172.21.15.100:0/865166380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b39681df0 0x557b3968a300 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac000c00 tx=0x7ff3ac003680 comp rx=0 tx=0).stop 2024-05-10T11:32:48.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.268+0000 7ff3c8d30b80 1 -- 172.21.15.100:0/865166380 shutdown_connections 2024-05-10T11:32:48.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.268+0000 7ff3c8d30b80 1 --2- 172.21.15.100:0/865166380 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x557b3969ac60 0x557b3969d050 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:48.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.268+0000 7ff3977fe700 0 client.18303 ms_handle_reset on v2:172.21.15.178:6838/3760985404 2024-05-10T11:32:48.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.268+0000 7ff3c8d30b80 1 --2- 172.21.15.100:0/865166380 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557b39685c80 0x557b3968a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:48.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.268+0000 7ff3c8d30b80 1 --2- 172.21.15.100:0/865166380 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557b396814d0 0x557b3968d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:48.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.268+0000 7ff3c8d30b80 1 --2- 172.21.15.100:0/865166380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b39681df0 0x557b3968a300 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:48.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.268+0000 7ff3c8d30b80 1 -- 172.21.15.100:0/865166380 >> 172.21.15.100:0/865166380 conn(0x557b395ad100 msgr2=0x557b39676720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:32:48.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.269+0000 7ff3c8d30b80 1 -- 172.21.15.100:0/865166380 shutdown_connections 2024-05-10T11:32:48.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.269+0000 7ff3c8d30b80 1 -- 172.21.15.100:0/865166380 wait complete. 2024-05-10T11:32:48.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.269+0000 7ff3c8d30b80 20 client.18303 trim_cache size 0 max 16384 2024-05-10T11:32:48.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:32:47 smithi188 ceph-mon[34075]: pgmap v853: 129 pgs: 129 active+clean; 345 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 560 KiB/s rd, 3.4 MiB/s wr, 258 op/s 2024-05-10T11:32:48.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.291+0000 7ff3c8d30b80 1 Processor -- start 2024-05-10T11:32:48.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.291+0000 7ff3c8d30b80 1 -- start start 2024-05-10T11:32:48.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.291+0000 7ff3c8d30b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557b396814d0 0x557b39696f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:48.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.291+0000 7ff3c8d30b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557b39681df0 0x557b396765a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:48.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.291+0000 7ff3c8d30b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b39685c80 0x557b396835f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:48.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.291+0000 7ff3c8d30b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b81af870 con 0x557b39685c80 2024-05-10T11:32:48.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.291+0000 7ff3c8d30b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b81ad070 con 0x557b39681df0 2024-05-10T11:32:48.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.291+0000 7ff3c8d30b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b81aaee0 con 0x557b396814d0 2024-05-10T11:32:48.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.291+0000 7ff3b7fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557b396814d0 0x557b39696f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:48.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.291+0000 7ff3b7fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557b396814d0 0x557b39696f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:49712/0 (socket says 172.21.15.100:49712) 2024-05-10T11:32:48.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.291+0000 7ff3b7fff700 1 -- 172.21.15.100:0/3819468500 learned_addr learned my addr 172.21.15.100:0/3819468500 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:32:48.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.292+0000 7ff3b77fe700 1 --2- 172.21.15.100:0/3819468500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557b39681df0 0x557b396765a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:48.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.292+0000 7ff3bca3a700 1 --2- 172.21.15.100:0/3819468500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b39685c80 0x557b396835f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:48.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.292+0000 7ff3b7fff700 1 -- 172.21.15.100:0/3819468500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557b39681df0 msgr2=0x557b396765a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:48.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.292+0000 7ff3b7fff700 1 --2- 172.21.15.100:0/3819468500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557b39681df0 0x557b396765a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:48.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.292+0000 7ff3b7fff700 1 -- 172.21.15.100:0/3819468500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b39685c80 msgr2=0x557b396835f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:48.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.292+0000 7ff3b7fff700 1 --2- 172.21.15.100:0/3819468500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b39685c80 0x557b396835f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:48.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.292+0000 7ff3b7fff700 1 -- 172.21.15.100:0/3819468500 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b39678810 con 0x557b396814d0 2024-05-10T11:32:48.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.292+0000 7ff3bca3a700 1 --2- 172.21.15.100:0/3819468500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b39685c80 0x557b396835f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:32:48.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.292+0000 7ff3b7fff700 1 --2- 172.21.15.100:0/3819468500 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557b396814d0 0x557b39696f20 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff3b0006070 tx=0x7ff3b001d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de4b5ce75256c5ed server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:32:48.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.292+0000 7ff39dffb700 1 -- 172.21.15.100:0/3819468500 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b0021070 con 0x557b396814d0 2024-05-10T11:32:48.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.292+0000 7ff39dffb700 1 -- 172.21.15.100:0/3819468500 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff3b003de50 con 0x557b396814d0 2024-05-10T11:32:48.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.292+0000 7ff39dffb700 1 -- 172.21.15.100:0/3819468500 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b003cb40 con 0x557b396814d0 2024-05-10T11:32:48.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.293+0000 7ff3c8d30b80 1 -- 172.21.15.100:0/3819468500 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557b396814d0 msgr2=0x557b39696f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:48.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.293+0000 7ff3c8d30b80 1 --2- 172.21.15.100:0/3819468500 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557b396814d0 0x557b39696f20 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff3b0006070 tx=0x7ff3b001d000 comp rx=0 tx=0).stop 2024-05-10T11:32:48.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.293+0000 7ff3c8d30b80 1 -- 172.21.15.100:0/3819468500 shutdown_connections 2024-05-10T11:32:48.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.293+0000 7ff3c8d30b80 1 --2- 172.21.15.100:0/3819468500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b39685c80 0x557b396835f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:48.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.293+0000 7ff3c8d30b80 1 --2- 172.21.15.100:0/3819468500 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557b396814d0 0x557b39696f20 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:48.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.293+0000 7ff3c8d30b80 1 --2- 172.21.15.100:0/3819468500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557b39681df0 0x557b396765a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:48.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.293+0000 7ff3c8d30b80 1 -- 172.21.15.100:0/3819468500 >> 172.21.15.100:0/3819468500 conn(0x557b395ad100 msgr2=0x557b39675e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:32:48.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.293+0000 7ff3c8d30b80 1 -- 172.21.15.100:0/3819468500 shutdown_connections 2024-05-10T11:32:48.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.294+0000 7ff3c8d30b80 1 -- 172.21.15.100:0/3819468500 wait complete. 2024-05-10T11:32:48.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.294+0000 7ff3c8d30b80 1 -- start start 2024-05-10T11:32:48.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.294+0000 7ff3c8d30b80 1 Processor -- start 2024-05-10T11:32:48.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.295+0000 7ff3c8d30b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b396814d0 0x557b3968e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:48.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.295+0000 7ff3c8d30b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557b39681df0 0x557b3968e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:48.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.295+0000 7ff3c8d30b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557b39685c80 0x557b397033d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:48.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.295+0000 7ff3c8d30b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b81ab530 con 0x557b396814d0 2024-05-10T11:32:48.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.295+0000 7ff3c8d30b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b81ac9a0 con 0x557b39685c80 2024-05-10T11:32:48.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.295+0000 7ff3c8d30b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b81abc00 con 0x557b39681df0 2024-05-10T11:32:48.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.295+0000 7ff3c8d30b80 10 client.0 _ll_register_callbacks cb 0x7ff3b00092e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:32:48.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.295+0000 7ff3b7fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b396814d0 0x557b3968e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:48.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.295+0000 7ff3bca3a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557b39685c80 0x557b397033d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:48.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.295+0000 7ff3bca3a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557b39685c80 0x557b397033d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:40826/0 (socket says 172.21.15.100:40826) 2024-05-10T11:32:48.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.295+0000 7ff3b7fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b396814d0 0x557b3968e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48550/0 (socket says 172.21.15.100:48550) 2024-05-10T11:32:48.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.295+0000 7ff3bca3a700 1 -- 172.21.15.100:0/621619867 learned_addr learned my addr 172.21.15.100:0/621619867 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:32:48.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.295+0000 7ff3b77fe700 1 --2- 172.21.15.100:0/621619867 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557b39681df0 0x557b3968e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:48.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.295+0000 7ff3bca3a700 1 -- 172.21.15.100:0/621619867 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557b39681df0 msgr2=0x557b3968e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:48.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.295+0000 7ff3bca3a700 1 --2- 172.21.15.100:0/621619867 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557b39681df0 0x557b3968e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:48.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.295+0000 7ff3bca3a700 1 -- 172.21.15.100:0/621619867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b396814d0 msgr2=0x557b3968e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:48.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.295+0000 7ff3bca3a700 1 --2- 172.21.15.100:0/621619867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b396814d0 0x557b3968e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:48.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.295+0000 7ff3bca3a700 1 -- 172.21.15.100:0/621619867 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b39703910 con 0x557b39685c80 2024-05-10T11:32:48.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.295+0000 7ff3b7fff700 1 --2- 172.21.15.100:0/621619867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b396814d0 0x557b3968e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:32:48.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.295+0000 7ff3bca3a700 1 --2- 172.21.15.100:0/621619867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557b39685c80 0x557b397033d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff3a801a000 tx=0x7ff3a8004660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17a920bce2a4878e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:32:48.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.295+0000 7ff3977fe700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:32:48.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.296+0000 7ff3977fe700 1 -- 172.21.15.100:0/621619867 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a80a8e80 con 0x557b39685c80 2024-05-10T11:32:48.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.296+0000 7ff3977fe700 1 -- 172.21.15.100:0/621619867 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7ff3a80b1850 con 0x557b39685c80 2024-05-10T11:32:48.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.296+0000 7ff3c8d30b80 1 -- 172.21.15.100:0/621619867 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3b003de50 con 0x557b39685c80 2024-05-10T11:32:48.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.296+0000 7ff3977fe700 1 -- 172.21.15.100:0/621619867 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7ff3a80ad230 con 0x557b39685c80 2024-05-10T11:32:48.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.296+0000 7ff3c8d30b80 10 client.27650 fetch_fsmap learned FSMap version 37 2024-05-10T11:32:48.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.296+0000 7ff3c8d30b80 1 -- 172.21.15.100:0/621619867 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x557b39704a80 con 0x557b39685c80 2024-05-10T11:32:48.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.296+0000 7ff3977fe700 1 -- 172.21.15.100:0/621619867 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff3b003de50 con 0x557b39685c80 2024-05-10T11:32:48.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.296+0000 7ff3977fe700 1 -- 172.21.15.100:0/621619867 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7ff3a80a7b50 con 0x557b39685c80 2024-05-10T11:32:48.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.296+0000 7ff3c8d30b80 10 client.27650 fetch_fsmap finished waiting for FSMap version 37 2024-05-10T11:32:48.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.296+0000 7ff3c8d30b80 10 client.27650 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-10T11:32:48.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.296+0000 7ff3c8d30b80 20 client.27650 populate_metadata read hostname 'smithi100' 2024-05-10T11:32:48.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.297+0000 7ff3c8d30b80 1 --2- 172.21.15.100:0/621619867 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x557b3969ac60 0x557b397071d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:48.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.297+0000 7ff3c8d30b80 4 client.27650 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:32:48.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.297+0000 7ff3c8d30b80 1 -- 172.21.15.100:0/621619867 --> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557b395e1240 con 0x557b3969ac60 2024-05-10T11:32:48.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.297+0000 7ff3b7fff700 1 --2- 172.21.15.100:0/621619867 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x557b3969ac60 0x557b397071d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:48.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.297+0000 7ff3977fe700 10 client.27650 ms_handle_connect on v2:172.21.15.178:6838/3760985404 2024-05-10T11:32:48.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.297+0000 7ff3b7fff700 1 --2- 172.21.15.100:0/621619867 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x557b3969ac60 0x557b397071d0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=543961913585058d server_cookie=39c985ac53a8a7f0 in_seq=0 out_seq=0 2024-05-10T11:32:48.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.297+0000 7ff3977fe700 1 -- 172.21.15.100:0/621619867 <== mds.1 v2:172.21.15.178:6838/3760985404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557b395e1240 con 0x557b3969ac60 2024-05-10T11:32:48.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.297+0000 7ff3977fe700 10 client.27650 handle_command_reply: tid=0 2024-05-10T11:32:48.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.297+0000 7ff3c8d30b80 1 client.27650 shutdown 2024-05-10T11:32:48.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.298+0000 7ff3c8d30b80 1 -- 172.21.15.100:0/621619867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557b39685c80 msgr2=0x557b397033d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:48.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.298+0000 7ff3c8d30b80 1 --2- 172.21.15.100:0/621619867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557b39685c80 0x557b397033d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff3a801a000 tx=0x7ff3a8004660 comp rx=0 tx=0).stop 2024-05-10T11:32:48.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.298+0000 7ff3c8d30b80 1 -- 172.21.15.100:0/621619867 shutdown_connections 2024-05-10T11:32:48.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.298+0000 7ff3c8d30b80 1 --2- 172.21.15.100:0/621619867 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x557b3969ac60 0x557b397071d0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:48.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.298+0000 7ff3c8d30b80 1 --2- 172.21.15.100:0/621619867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557b39685c80 0x557b397033d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:48.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.298+0000 7ff3c8d30b80 1 --2- 172.21.15.100:0/621619867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b396814d0 0x557b3968e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:48.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.298+0000 7ff3c8d30b80 1 --2- 172.21.15.100:0/621619867 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557b39681df0 0x557b3968e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:48.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.298+0000 7ff3977fe700 0 client.27650 ms_handle_reset on v2:172.21.15.178:6838/3760985404 2024-05-10T11:32:48.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.298+0000 7ff3c8d30b80 1 -- 172.21.15.100:0/621619867 >> 172.21.15.100:0/621619867 conn(0x557b395ad100 msgr2=0x557b39676720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:32:48.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.298+0000 7ff3c8d30b80 1 -- 172.21.15.100:0/621619867 shutdown_connections 2024-05-10T11:32:48.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.298+0000 7ff3c8d30b80 1 -- 172.21.15.100:0/621619867 wait complete. 2024-05-10T11:32:48.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.298+0000 7ff3c8d30b80 20 client.27650 trim_cache size 0 max 16384 2024-05-10T11:32:48.305 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:32:48.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.299+0000 7ff3be49d700 1 -- 172.21.15.100:0/928935277 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff3a00b2900 msgr2=0x7ff3a00b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:48.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.299+0000 7ff3be49d700 1 --2- 172.21.15.100:0/928935277 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff3a00b2900 0x7ff3a00b4dc0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ff3b000b780 tx=0x7ff3b0009c40 comp rx=0 tx=0).stop 2024-05-10T11:32:48.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.299+0000 7ff3be49d700 1 -- 172.21.15.100:0/928935277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3b8108400 msgr2=0x7ff3b81a79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:48.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.299+0000 7ff3be49d700 1 --2- 172.21.15.100:0/928935277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3b8108400 0x7ff3b81a79c0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7ff3a800d720 tx=0x7ff3a800da30 comp rx=0 tx=0).stop 2024-05-10T11:32:48.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.299+0000 7ff3be49d700 1 -- 172.21.15.100:0/928935277 shutdown_connections 2024-05-10T11:32:48.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.299+0000 7ff3be49d700 1 --2- 172.21.15.100:0/928935277 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff3a00b2900 0x7ff3a00b4dc0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:48.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.299+0000 7ff3be49d700 1 --2- 172.21.15.100:0/928935277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3b8108400 0x7ff3b81a79c0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:48.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.299+0000 7ff3be49d700 1 --2- 172.21.15.100:0/928935277 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff3b8102390 0x7ff3b819cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:48.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.299+0000 7ff3be49d700 1 --2- 172.21.15.100:0/928935277 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3b8102d50 0x7ff3b819d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:48.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.299+0000 7ff3be49d700 1 -- 172.21.15.100:0/928935277 >> 172.21.15.100:0/928935277 conn(0x7ff3b8079310 msgr2=0x7ff3b8101b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:32:48.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.299+0000 7ff3be49d700 1 -- 172.21.15.100:0/928935277 shutdown_connections 2024-05-10T11:32:48.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:48.299+0000 7ff3be49d700 1 -- 172.21.15.100:0/928935277 wait complete. 2024-05-10T11:32:48.907 INFO:tasks.workunit.client.0.smithi100.stdout: 10 7276 13.50 MB/sec warmup 42 sec latency 83.951 ms 2024-05-10T11:32:49.069 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-10T11:32:49.435 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:32:49.907 INFO:tasks.workunit.client.0.smithi100.stdout: 10 7400 13.38 MB/sec warmup 43 sec latency 151.263 ms 2024-05-10T11:32:50.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:32:49 smithi178 ceph-mon[33359]: pgmap v854: 129 pgs: 129 active+clean; 353 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 4.2 MiB/s wr, 287 op/s 2024-05-10T11:32:50.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:32:49 smithi188 ceph-mon[34075]: pgmap v854: 129 pgs: 129 active+clean; 353 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 4.2 MiB/s wr, 287 op/s 2024-05-10T11:32:50.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:32:49 smithi100 ceph-mon[30296]: pgmap v854: 129 pgs: 129 active+clean; 353 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 4.2 MiB/s wr, 287 op/s 2024-05-10T11:32:50.907 INFO:tasks.workunit.client.0.smithi100.stdout: 10 7598 13.42 MB/sec warmup 44 sec latency 146.965 ms 2024-05-10T11:32:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.171+0000 7f95bdeb0640 1 -- 172.21.15.100:0/2314658603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95b8103c90 msgr2=0x7f95b8104110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.171+0000 7f95bdeb0640 1 --2- 172.21.15.100:0/2314658603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95b8103c90 0x7f95b8104110 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f95ac009930 tx=0x7f95ac02a420 comp rx=0 tx=0).stop 2024-05-10T11:32:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.171+0000 7f95bdeb0640 1 -- 172.21.15.100:0/2314658603 shutdown_connections 2024-05-10T11:32:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.171+0000 7f95bdeb0640 1 --2- 172.21.15.100:0/2314658603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95b8104650 0x7f95b810aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.171+0000 7f95bdeb0640 1 --2- 172.21.15.100:0/2314658603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95b8103c90 0x7f95b8104110 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.171+0000 7f95bdeb0640 1 --2- 172.21.15.100:0/2314658603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95b8100cc0 0x7f95b81010e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.171+0000 7f95bdeb0640 1 -- 172.21.15.100:0/2314658603 >> 172.21.15.100:0/2314658603 conn(0x7f95b80fc610 msgr2=0x7f95b80fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:32:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.172+0000 7f95bdeb0640 1 -- 172.21.15.100:0/2314658603 shutdown_connections 2024-05-10T11:32:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.172+0000 7f95bdeb0640 1 -- 172.21.15.100:0/2314658603 wait complete. 2024-05-10T11:32:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.172+0000 7f95bdeb0640 1 Processor -- start 2024-05-10T11:32:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.172+0000 7f95bdeb0640 1 -- start start 2024-05-10T11:32:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.173+0000 7f95bdeb0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95b8100cc0 0x7f95b819ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.173+0000 7f95bdeb0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95b8103c90 0x7f95b819b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:51.176 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.173+0000 7f95bdeb0640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95b8104650 0x7f95b81a22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:51.176 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.173+0000 7f95bdeb0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f95b810e400 con 0x7f95b8103c90 2024-05-10T11:32:51.176 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.173+0000 7f95bdeb0640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f95b810e280 con 0x7f95b8104650 2024-05-10T11:32:51.176 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.173+0000 7f95bdeb0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f95b810e580 con 0x7f95b8100cc0 2024-05-10T11:32:51.176 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.173+0000 7f95b6ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95b8103c90 0x7f95b819b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:51.176 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.173+0000 7f95b7fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95b8104650 0x7f95b81a22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:51.176 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.173+0000 7f95b77fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95b8100cc0 0x7f95b819ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:51.176 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.173+0000 7f95b6ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95b8103c90 0x7f95b819b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48570/0 (socket says 172.21.15.100:48570) 2024-05-10T11:32:51.176 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.173+0000 7f95b6ffd640 1 -- 172.21.15.100:0/2533304769 learned_addr learned my addr 172.21.15.100:0/2533304769 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:32:51.176 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.173+0000 7f95b6ffd640 1 -- 172.21.15.100:0/2533304769 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95b8100cc0 msgr2=0x7f95b819ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:51.176 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.173+0000 7f95b6ffd640 1 --2- 172.21.15.100:0/2533304769 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95b8100cc0 0x7f95b819ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:51.176 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.173+0000 7f95b6ffd640 1 -- 172.21.15.100:0/2533304769 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95b8104650 msgr2=0x7f95b81a22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:51.176 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.173+0000 7f95b6ffd640 1 --2- 172.21.15.100:0/2533304769 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95b8104650 0x7f95b81a22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:51.176 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.173+0000 7f95b6ffd640 1 -- 172.21.15.100:0/2533304769 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95b81a2960 con 0x7f95b8103c90 2024-05-10T11:32:51.176 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.173+0000 7f95b7fff640 1 --2- 172.21.15.100:0/2533304769 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95b8104650 0x7f95b81a22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:32:51.177 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.173+0000 7f95b6ffd640 1 --2- 172.21.15.100:0/2533304769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95b8103c90 0x7f95b819b2e0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f95ac009900 tx=0x7f95ac0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:32:51.177 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.174+0000 7f95b4ff9640 1 -- 172.21.15.100:0/2533304769 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ac03f030 con 0x7f95b8103c90 2024-05-10T11:32:51.177 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.174+0000 7f95b4ff9640 1 -- 172.21.15.100:0/2533304769 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f95ac003c90 con 0x7f95b8103c90 2024-05-10T11:32:51.177 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.174+0000 7f95bdeb0640 1 -- 172.21.15.100:0/2533304769 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95b81a2bf0 con 0x7f95b8103c90 2024-05-10T11:32:51.177 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.174+0000 7f95b4ff9640 1 -- 172.21.15.100:0/2533304769 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ac035070 con 0x7f95b8103c90 2024-05-10T11:32:51.177 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.174+0000 7f95bdeb0640 1 -- 172.21.15.100:0/2533304769 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95b81a3130 con 0x7f95b8103c90 2024-05-10T11:32:51.179 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.177+0000 7f95c0bcb740 1 Processor -- start 2024-05-10T11:32:51.179 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.177+0000 7f95c0bcb740 1 -- start start 2024-05-10T11:32:51.179 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.177+0000 7f95c0bcb740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651a7b76db0 0x5651a7b771d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:51.179 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.177+0000 7f95c0bcb740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5651a7b777a0 0x5651a7b7c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:51.179 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.177+0000 7f95c0bcb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651a7b7c910 0x5651a7b7eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:51.180 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.177+0000 7f95c0bcb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5651a7adb5d0 con 0x5651a7b7c910 2024-05-10T11:32:51.180 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.177+0000 7f95c0bcb740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5651a7a94890 con 0x5651a7b777a0 2024-05-10T11:32:51.180 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.177+0000 7f95c0bcb740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5651a7ad7590 con 0x5651a7b76db0 2024-05-10T11:32:51.180 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.178+0000 7f95b77fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5651a7b777a0 0x5651a7b7c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:51.180 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.178+0000 7f95b7fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651a7b76db0 0x5651a7b771d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:51.180 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.178+0000 7f95b77fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5651a7b777a0 0x5651a7b7c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:40852/0 (socket says 172.21.15.100:40852) 2024-05-10T11:32:51.180 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.178+0000 7f95b77fe640 1 -- 172.21.15.100:0/3482672261 learned_addr learned my addr 172.21.15.100:0/3482672261 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:32:51.180 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.178+0000 7f95b4ff9640 1 -- 172.21.15.100:0/2533304769 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f95ac042030 con 0x7f95b8103c90 2024-05-10T11:32:51.180 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.178+0000 7f95b77fe640 1 -- 172.21.15.100:0/3482672261 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651a7b76db0 msgr2=0x5651a7b771d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:51.180 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.178+0000 7f95b77fe640 1 --2- 172.21.15.100:0/3482672261 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651a7b76db0 0x5651a7b771d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:51.180 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.178+0000 7f95b6ffd640 1 --2- 172.21.15.100:0/3482672261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651a7b7c910 0x5651a7b7eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:51.180 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.178+0000 7f95b77fe640 1 -- 172.21.15.100:0/3482672261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651a7b7c910 msgr2=0x5651a7b7eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:51.180 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.178+0000 7f95b77fe640 1 --2- 172.21.15.100:0/3482672261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651a7b7c910 0x5651a7b7eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.178+0000 7f95b77fe640 1 -- 172.21.15.100:0/3482672261 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651a77ed210 con 0x5651a7b777a0 2024-05-10T11:32:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.178+0000 7f95b6ffd640 1 --2- 172.21.15.100:0/3482672261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651a7b7c910 0x5651a7b7eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:32:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.178+0000 7f95b77fe640 1 --2- 172.21.15.100:0/3482672261 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5651a7b777a0 0x5651a7b7c2a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f95a800bc70 tx=0x7f95a80107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e8945541229af7f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:32:51.184 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.179+0000 7f959dffb640 1 -- 172.21.15.100:0/3482672261 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a8004020 con 0x5651a7b777a0 2024-05-10T11:32:51.184 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.179+0000 7f959dffb640 1 -- 172.21.15.100:0/3482672261 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f95a80041c0 con 0x5651a7b777a0 2024-05-10T11:32:51.184 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.179+0000 7f959dffb640 1 -- 172.21.15.100:0/3482672261 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a80116a0 con 0x5651a7b777a0 2024-05-10T11:32:51.184 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.179+0000 7f95b4ff9640 1 --2- 172.21.15.100:0/2533304769 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f95a00b2920 0x7f95a00b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:51.184 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.179+0000 7f95b7fff640 1 --2- 172.21.15.100:0/2533304769 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f95a00b2920 0x7f95a00b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:51.184 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.179+0000 7f95b4ff9640 1 -- 172.21.15.100:0/2533304769 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f95ac102f90 con 0x7f95b8103c90 2024-05-10T11:32:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.180+0000 7f95c0bcb740 1 -- 172.21.15.100:0/3482672261 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5651a7b777a0 msgr2=0x5651a7b7c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.180+0000 7f95c0bcb740 1 --2- 172.21.15.100:0/3482672261 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5651a7b777a0 0x5651a7b7c2a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f95a800bc70 tx=0x7f95a80107a0 comp rx=0 tx=0).stop 2024-05-10T11:32:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.180+0000 7f95b7fff640 1 --2- 172.21.15.100:0/2533304769 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f95a00b2920 0x7f95a00b4de0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x5651a7b78000 tx=0x7f95b0009800 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:32:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.180+0000 7f95c0bcb740 1 -- 172.21.15.100:0/3482672261 shutdown_connections 2024-05-10T11:32:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.180+0000 7f95c0bcb740 1 --2- 172.21.15.100:0/3482672261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651a7b7c910 0x5651a7b7eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.180+0000 7f95c0bcb740 1 --2- 172.21.15.100:0/3482672261 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5651a7b777a0 0x5651a7b7c2a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.180+0000 7f95c0bcb740 1 --2- 172.21.15.100:0/3482672261 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651a7b76db0 0x5651a7b771d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.180+0000 7f95c0bcb740 1 -- 172.21.15.100:0/3482672261 >> 172.21.15.100:0/3482672261 conn(0x5651a7b72420 msgr2=0x5651a7b72820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:32:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.180+0000 7f95c0bcb740 1 -- 172.21.15.100:0/3482672261 shutdown_connections 2024-05-10T11:32:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.180+0000 7f95c0bcb740 1 -- 172.21.15.100:0/3482672261 wait complete. 2024-05-10T11:32:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.180+0000 7f95c0bcb740 1 -- start start 2024-05-10T11:32:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.181+0000 7f95c0bcb740 1 Processor -- start 2024-05-10T11:32:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.181+0000 7f95c0bcb740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651a7b76db0 0x5651a7b81000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:51.186 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.181+0000 7f95c0bcb740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5651a7b777a0 0x5651a7b81540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:51.186 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.181+0000 7f95c0bcb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651a7b7c910 0x5651a7b88580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:51.186 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.181+0000 7f95c0bcb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5651a7b7f8c0 con 0x5651a7b7c910 2024-05-10T11:32:51.186 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.181+0000 7f95c0bcb740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5651a7a98370 con 0x5651a7b777a0 2024-05-10T11:32:51.186 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.181+0000 7f95c0bcb740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5651a7b7fa40 con 0x5651a7b76db0 2024-05-10T11:32:51.186 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.181+0000 7f95b7fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651a7b7c910 0x5651a7b88580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:51.186 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.181+0000 7f95b7fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651a7b7c910 0x5651a7b88580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48584/0 (socket says 172.21.15.100:48584) 2024-05-10T11:32:51.186 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.181+0000 7f95b7fff640 1 -- 172.21.15.100:0/1156484419 learned_addr learned my addr 172.21.15.100:0/1156484419 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:32:51.186 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.181+0000 7f95b77fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651a7b76db0 0x5651a7b81000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:51.186 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.181+0000 7f95c0bcb740 10 client.0 _ll_register_callbacks cb 0x5651a77eca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:32:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.181+0000 7f95b6ffd640 1 --2- 172.21.15.100:0/1156484419 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5651a7b777a0 0x5651a7b81540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.181+0000 7f95b7fff640 1 -- 172.21.15.100:0/1156484419 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651a7b76db0 msgr2=0x5651a7b81000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.181+0000 7f95b7fff640 1 --2- 172.21.15.100:0/1156484419 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651a7b76db0 0x5651a7b81000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.181+0000 7f95b7fff640 1 -- 172.21.15.100:0/1156484419 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5651a7b777a0 msgr2=0x5651a7b81540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.181+0000 7f95b7fff640 1 --2- 172.21.15.100:0/1156484419 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5651a7b777a0 0x5651a7b81540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.181+0000 7f95b7fff640 1 -- 172.21.15.100:0/1156484419 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651a78b9620 con 0x5651a7b7c910 2024-05-10T11:32:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.181+0000 7f95b6ffd640 1 --2- 172.21.15.100:0/1156484419 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5651a7b777a0 0x5651a7b81540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:32:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.182+0000 7f95b7fff640 1 --2- 172.21.15.100:0/1156484419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651a7b7c910 0x5651a7b88580 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f95b0011c70 tx=0x7f95b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe25f8d36551fe77 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:32:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.182+0000 7f9596ffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:32:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.182+0000 7f9596ffd640 1 -- 172.21.15.100:0/1156484419 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b0020620 con 0x5651a7b7c910 2024-05-10T11:32:51.188 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.182+0000 7f9596ffd640 1 -- 172.21.15.100:0/1156484419 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f95b00207c0 con 0x5651a7b7c910 2024-05-10T11:32:51.188 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.182+0000 7f95c0bcb740 1 -- 172.21.15.100:0/1156484419 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95a80041c0 con 0x5651a7b7c910 2024-05-10T11:32:51.188 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.182+0000 7f9596ffd640 1 -- 172.21.15.100:0/1156484419 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f95b0033050 con 0x5651a7b7c910 2024-05-10T11:32:51.188 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.182+0000 7f95c0bcb740 10 client.18318 fetch_fsmap learned FSMap version 37 2024-05-10T11:32:51.188 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.182+0000 7f95c0bcb740 1 -- 172.21.15.100:0/1156484419 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5651a7b8b5a0 con 0x5651a7b7c910 2024-05-10T11:32:51.188 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.183+0000 7f9596ffd640 1 -- 172.21.15.100:0/1156484419 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f95a80041c0 con 0x5651a7b7c910 2024-05-10T11:32:51.188 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.183+0000 7f9596ffd640 1 -- 172.21.15.100:0/1156484419 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f95b000f040 con 0x5651a7b7c910 2024-05-10T11:32:51.188 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.183+0000 7f95c0bcb740 10 client.18318 fetch_fsmap finished waiting for FSMap version 37 2024-05-10T11:32:51.188 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.183+0000 7f95c0bcb740 10 client.18318 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:32:51.188 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.183+0000 7f95c0bcb740 20 client.18318 populate_metadata read hostname 'smithi100' 2024-05-10T11:32:51.188 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.183+0000 7f95c0bcb740 1 --2- 172.21.15.100:0/1156484419 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5651a7b8cda0 0x5651a7b8f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:51.188 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.183+0000 7f95c0bcb740 4 client.18318 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:32:51.188 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.183+0000 7f95c0bcb740 1 -- 172.21.15.100:0/1156484419 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651a7ae91e0 con 0x5651a7b8cda0 2024-05-10T11:32:51.190 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.184+0000 7f95b77fe640 1 --2- 172.21.15.100:0/1156484419 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5651a7b8cda0 0x5651a7b8f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:51.190 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.184+0000 7f95b77fe640 1 --2- 172.21.15.100:0/1156484419 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5651a7b8cda0 0x5651a7b8f1b0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b03ebc070c75d29 server_cookie=44bad558792f422e in_seq=0 out_seq=0 2024-05-10T11:32:51.190 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.184+0000 7f9596ffd640 10 client.18318 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:32:51.190 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.185+0000 7f9596ffd640 1 -- 172.21.15.100:0/1156484419 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5651a7ae91e0 con 0x5651a7b8cda0 2024-05-10T11:32:51.190 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.185+0000 7f9596ffd640 10 client.18318 handle_command_reply: tid=0 2024-05-10T11:32:51.190 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.185+0000 7f95c0bcb740 1 client.18318 shutdown 2024-05-10T11:32:51.190 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.185+0000 7f95c0bcb740 1 -- 172.21.15.100:0/1156484419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651a7b7c910 msgr2=0x5651a7b88580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:51.190 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.185+0000 7f95c0bcb740 1 --2- 172.21.15.100:0/1156484419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651a7b7c910 0x5651a7b88580 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f95b0011c70 tx=0x7f95b0004a40 comp rx=0 tx=0).stop 2024-05-10T11:32:51.191 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.185+0000 7f95c0bcb740 1 -- 172.21.15.100:0/1156484419 shutdown_connections 2024-05-10T11:32:51.191 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.185+0000 7f95c0bcb740 1 --2- 172.21.15.100:0/1156484419 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5651a7b8cda0 0x5651a7b8f1b0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:51.191 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.186+0000 7f95c0bcb740 1 --2- 172.21.15.100:0/1156484419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651a7b7c910 0x5651a7b88580 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:51.191 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.186+0000 7f95c0bcb740 1 --2- 172.21.15.100:0/1156484419 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5651a7b777a0 0x5651a7b81540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:51.191 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.186+0000 7f9596ffd640 0 client.18318 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:32:51.191 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.186+0000 7f95c0bcb740 1 --2- 172.21.15.100:0/1156484419 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651a7b76db0 0x5651a7b81000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:51.191 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.186+0000 7f95c0bcb740 1 -- 172.21.15.100:0/1156484419 >> 172.21.15.100:0/1156484419 conn(0x5651a7b72420 msgr2=0x5651a7b71580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:32:51.191 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.186+0000 7f95c0bcb740 1 -- 172.21.15.100:0/1156484419 shutdown_connections 2024-05-10T11:32:51.191 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.186+0000 7f95c0bcb740 1 -- 172.21.15.100:0/1156484419 wait complete. 2024-05-10T11:32:51.191 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.186+0000 7f95c0bcb740 20 client.18318 trim_cache size 0 max 16384 2024-05-10T11:32:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.203+0000 7f95c0bcb740 1 Processor -- start 2024-05-10T11:32:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.203+0000 7f95c0bcb740 1 -- start start 2024-05-10T11:32:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.204+0000 7f95c0bcb740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5651a7b76db0 0x5651a7b78980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.204+0000 7f95c0bcb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651a7b777a0 0x5651a7b78ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.204+0000 7f95c0bcb740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651a7b7c910 0x5651a7b88540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.204+0000 7f95c0bcb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f95b81afec0 con 0x5651a7b777a0 2024-05-10T11:32:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.204+0000 7f95c0bcb740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f95b81ad700 con 0x5651a7b76db0 2024-05-10T11:32:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.204+0000 7f95c0bcb740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f95b81ab550 con 0x5651a7b7c910 2024-05-10T11:32:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.204+0000 7f95b6ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651a7b777a0 0x5651a7b78ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.204+0000 7f95b7fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651a7b7c910 0x5651a7b88540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.204+0000 7f95b6ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651a7b777a0 0x5651a7b78ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48594/0 (socket says 172.21.15.100:48594) 2024-05-10T11:32:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.204+0000 7f95b6ffd640 1 -- 172.21.15.100:0/2632691369 learned_addr learned my addr 172.21.15.100:0/2632691369 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:32:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.204+0000 7f95b77fe640 1 --2- 172.21.15.100:0/2632691369 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5651a7b76db0 0x5651a7b78980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.204+0000 7f95b6ffd640 1 -- 172.21.15.100:0/2632691369 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651a7b7c910 msgr2=0x5651a7b88540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.204+0000 7f95b6ffd640 1 --2- 172.21.15.100:0/2632691369 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651a7b7c910 0x5651a7b88540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.204+0000 7f95b6ffd640 1 -- 172.21.15.100:0/2632691369 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5651a7b76db0 msgr2=0x5651a7b78980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.204+0000 7f95b6ffd640 1 --2- 172.21.15.100:0/2632691369 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5651a7b76db0 0x5651a7b78980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.204+0000 7f95b6ffd640 1 -- 172.21.15.100:0/2632691369 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651a7b76340 con 0x5651a7b777a0 2024-05-10T11:32:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.204+0000 7f95b6ffd640 1 --2- 172.21.15.100:0/2632691369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651a7b777a0 0x5651a7b78ec0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f95ac0c6240 tx=0x7f95ac0c9ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d065eaf5e4b5335d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:32:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.205+0000 7f959d7fa640 1 -- 172.21.15.100:0/2632691369 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ac049070 con 0x5651a7b777a0 2024-05-10T11:32:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.205+0000 7f959d7fa640 1 -- 172.21.15.100:0/2632691369 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f95ac003e40 con 0x5651a7b777a0 2024-05-10T11:32:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.205+0000 7f959d7fa640 1 -- 172.21.15.100:0/2632691369 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ac0d30b0 con 0x5651a7b777a0 2024-05-10T11:32:51.210 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.206+0000 7f95c0bcb740 1 -- 172.21.15.100:0/2632691369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651a7b777a0 msgr2=0x5651a7b78ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:51.210 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.206+0000 7f95c0bcb740 1 --2- 172.21.15.100:0/2632691369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651a7b777a0 0x5651a7b78ec0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f95ac0c6240 tx=0x7f95ac0c9ba0 comp rx=0 tx=0).stop 2024-05-10T11:32:51.211 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.206+0000 7f95c0bcb740 1 -- 172.21.15.100:0/2632691369 shutdown_connections 2024-05-10T11:32:51.211 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.206+0000 7f95c0bcb740 1 --2- 172.21.15.100:0/2632691369 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651a7b7c910 0x5651a7b88540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:51.211 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.206+0000 7f95c0bcb740 1 --2- 172.21.15.100:0/2632691369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651a7b777a0 0x5651a7b78ec0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:51.211 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.206+0000 7f95c0bcb740 1 --2- 172.21.15.100:0/2632691369 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5651a7b76db0 0x5651a7b78980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:51.211 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.206+0000 7f95c0bcb740 1 -- 172.21.15.100:0/2632691369 >> 172.21.15.100:0/2632691369 conn(0x5651a7b72420 msgr2=0x5651a7b7bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:32:51.211 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.206+0000 7f95c0bcb740 1 -- 172.21.15.100:0/2632691369 shutdown_connections 2024-05-10T11:32:51.211 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.206+0000 7f95c0bcb740 1 -- 172.21.15.100:0/2632691369 wait complete. 2024-05-10T11:32:51.211 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.206+0000 7f95c0bcb740 1 -- start start 2024-05-10T11:32:51.211 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.207+0000 7f95c0bcb740 1 Processor -- start 2024-05-10T11:32:51.211 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.207+0000 7f95c0bcb740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5651a7b76db0 0x5651a7b858a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:51.211 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.207+0000 7f95c0bcb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651a7b777a0 0x5651a7b85de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:51.211 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.207+0000 7f95c0bcb740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651a7b7c910 0x5651a7b86df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.207+0000 7f95c0bcb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f95b81abbc0 con 0x5651a7b777a0 2024-05-10T11:32:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.207+0000 7f95c0bcb740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f95b81ad030 con 0x5651a7b76db0 2024-05-10T11:32:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.207+0000 7f95c0bcb740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f95b81ac290 con 0x5651a7b7c910 2024-05-10T11:32:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.207+0000 7f95c0bcb740 10 client.0 _ll_register_callbacks cb 0x7f95b00118f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:32:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.207+0000 7f95b6ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651a7b777a0 0x5651a7b85de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.207+0000 7f95b6ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651a7b777a0 0x5651a7b85de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48606/0 (socket says 172.21.15.100:48606) 2024-05-10T11:32:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.207+0000 7f95b6ffd640 1 -- 172.21.15.100:0/1196738720 learned_addr learned my addr 172.21.15.100:0/1196738720 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:32:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.207+0000 7f95b77fe640 1 --2- 172.21.15.100:0/1196738720 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5651a7b76db0 0x5651a7b858a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.207+0000 7f95b7fff640 1 --2- 172.21.15.100:0/1196738720 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651a7b7c910 0x5651a7b86df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.208+0000 7f95b6ffd640 1 -- 172.21.15.100:0/1196738720 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651a7b7c910 msgr2=0x5651a7b86df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.208+0000 7f95b6ffd640 1 --2- 172.21.15.100:0/1196738720 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651a7b7c910 0x5651a7b86df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.208+0000 7f95b6ffd640 1 -- 172.21.15.100:0/1196738720 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5651a7b76db0 msgr2=0x5651a7b858a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.208+0000 7f95b6ffd640 1 --2- 172.21.15.100:0/1196738720 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5651a7b76db0 0x5651a7b858a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:51.213 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.208+0000 7f95b6ffd640 1 -- 172.21.15.100:0/1196738720 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651a7b873e0 con 0x5651a7b777a0 2024-05-10T11:32:51.213 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.208+0000 7f95b6ffd640 1 --2- 172.21.15.100:0/1196738720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651a7b777a0 0x5651a7b85de0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f95ac0ca410 tx=0x7f95ac03b510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d3cc677cd2188cd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:32:51.213 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.208+0000 7f9596ffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:32:51.213 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.208+0000 7f9596ffd640 1 -- 172.21.15.100:0/1196738720 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ac049070 con 0x5651a7b777a0 2024-05-10T11:32:51.213 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.208+0000 7f9596ffd640 1 -- 172.21.15.100:0/1196738720 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f95ac0ca4b0 con 0x5651a7b777a0 2024-05-10T11:32:51.213 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.208+0000 7f95c0bcb740 1 -- 172.21.15.100:0/1196738720 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5651a7ae3050 con 0x5651a7b777a0 2024-05-10T11:32:51.213 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.209+0000 7f9596ffd640 1 -- 172.21.15.100:0/1196738720 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f95ac0c9d60 con 0x5651a7b777a0 2024-05-10T11:32:51.213 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.209+0000 7f95c0bcb740 10 client.18324 fetch_fsmap learned FSMap version 37 2024-05-10T11:32:51.213 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.209+0000 7f95c0bcb740 1 -- 172.21.15.100:0/1196738720 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5651a7b886f0 con 0x5651a7b777a0 2024-05-10T11:32:51.213 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.209+0000 7f9596ffd640 1 -- 172.21.15.100:0/1196738720 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x5651a7ae3050 con 0x5651a7b777a0 2024-05-10T11:32:51.213 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.209+0000 7f9596ffd640 1 -- 172.21.15.100:0/1196738720 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f95ac0e6c00 con 0x5651a7b777a0 2024-05-10T11:32:51.214 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.209+0000 7f95c0bcb740 10 client.18324 fetch_fsmap finished waiting for FSMap version 37 2024-05-10T11:32:51.214 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.209+0000 7f95c0bcb740 10 client.18324 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:32:51.214 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.209+0000 7f95c0bcb740 20 client.18324 populate_metadata read hostname 'smithi100' 2024-05-10T11:32:51.214 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.209+0000 7f95c0bcb740 1 --2- 172.21.15.100:0/1196738720 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5651a7bf7a50 0x5651a7bf9e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:51.214 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.209+0000 7f95c0bcb740 4 client.18324 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:32:51.214 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.209+0000 7f95c0bcb740 1 -- 172.21.15.100:0/1196738720 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5651a7a5e5f0 con 0x5651a7bf7a50 2024-05-10T11:32:51.214 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.209+0000 7f95b77fe640 1 --2- 172.21.15.100:0/1196738720 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5651a7bf7a50 0x5651a7bf9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:51.214 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.210+0000 7f9596ffd640 10 client.18324 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:32:51.214 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.210+0000 7f95b77fe640 1 --2- 172.21.15.100:0/1196738720 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5651a7bf7a50 0x5651a7bf9e70 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c142e63999294fd9 server_cookie=7157e7a3bef39dea in_seq=0 out_seq=0 2024-05-10T11:32:51.296 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.294+0000 7f9596ffd640 1 -- 172.21.15.100:0/1196738720 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5651a7a5e5f0 con 0x5651a7bf7a50 2024-05-10T11:32:51.296 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.294+0000 7f9596ffd640 10 client.18324 handle_command_reply: tid=0 2024-05-10T11:32:51.296 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.294+0000 7f95c0bcb740 1 client.18324 shutdown 2024-05-10T11:32:51.296 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.294+0000 7f95c0bcb740 1 -- 172.21.15.100:0/1196738720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651a7b777a0 msgr2=0x5651a7b85de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:51.296 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.294+0000 7f95c0bcb740 1 --2- 172.21.15.100:0/1196738720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651a7b777a0 0x5651a7b85de0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f95ac0ca410 tx=0x7f95ac03b510 comp rx=0 tx=0).stop 2024-05-10T11:32:51.296 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.294+0000 7f95c0bcb740 1 -- 172.21.15.100:0/1196738720 shutdown_connections 2024-05-10T11:32:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.294+0000 7f95c0bcb740 1 --2- 172.21.15.100:0/1196738720 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5651a7bf7a50 0x5651a7bf9e70 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.294+0000 7f95c0bcb740 1 --2- 172.21.15.100:0/1196738720 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651a7b7c910 0x5651a7b86df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.295+0000 7f95c0bcb740 1 --2- 172.21.15.100:0/1196738720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651a7b777a0 0x5651a7b85de0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.295+0000 7f9596ffd640 0 client.18324 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:32:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.295+0000 7f95c0bcb740 1 --2- 172.21.15.100:0/1196738720 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5651a7b76db0 0x5651a7b858a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.295+0000 7f95c0bcb740 1 -- 172.21.15.100:0/1196738720 >> 172.21.15.100:0/1196738720 conn(0x5651a7b72420 msgr2=0x5651a7b81000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:32:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.295+0000 7f95c0bcb740 1 -- 172.21.15.100:0/1196738720 shutdown_connections 2024-05-10T11:32:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.295+0000 7f95c0bcb740 1 -- 172.21.15.100:0/1196738720 wait complete. 2024-05-10T11:32:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.295+0000 7f95c0bcb740 20 client.18324 trim_cache size 0 max 16384 2024-05-10T11:32:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:32:51.298 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:32:51.298 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:32:51.298 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:32:51.299 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.298+0000 7f95bdeb0640 1 -- 172.21.15.100:0/2533304769 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f95a00b2920 msgr2=0x7f95a00b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:51.299 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.298+0000 7f95bdeb0640 1 --2- 172.21.15.100:0/2533304769 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f95a00b2920 0x7f95a00b4de0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x5651a7b78000 tx=0x7f95b0009800 comp rx=0 tx=0).stop 2024-05-10T11:32:51.299 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.298+0000 7f95bdeb0640 1 -- 172.21.15.100:0/2533304769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95b8103c90 msgr2=0x7f95b819b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:51.299 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.298+0000 7f95bdeb0640 1 --2- 172.21.15.100:0/2533304769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95b8103c90 0x7f95b819b2e0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f95ac009900 tx=0x7f95ac0042f0 comp rx=0 tx=0).stop 2024-05-10T11:32:51.299 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.298+0000 7f95bdeb0640 1 -- 172.21.15.100:0/2533304769 shutdown_connections 2024-05-10T11:32:51.299 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.298+0000 7f95bdeb0640 1 --2- 172.21.15.100:0/2533304769 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f95a00b2920 0x7f95a00b4de0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:51.299 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.298+0000 7f95bdeb0640 1 --2- 172.21.15.100:0/2533304769 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95b8104650 0x7f95b81a22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:51.299 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.298+0000 7f95bdeb0640 1 --2- 172.21.15.100:0/2533304769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95b8103c90 0x7f95b819b2e0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:51.300 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.298+0000 7f95bdeb0640 1 --2- 172.21.15.100:0/2533304769 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95b8100cc0 0x7f95b819ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:51.300 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.298+0000 7f95bdeb0640 1 -- 172.21.15.100:0/2533304769 >> 172.21.15.100:0/2533304769 conn(0x7f95b80fc610 msgr2=0x7f95b8101a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:32:51.300 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.298+0000 7f95bdeb0640 1 -- 172.21.15.100:0/2533304769 shutdown_connections 2024-05-10T11:32:51.300 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:32:51.298+0000 7f95bdeb0640 1 -- 172.21.15.100:0/2533304769 wait complete. 2024-05-10T11:32:51.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.302+0000 7f4c0b6a8700 1 -- 172.21.15.100:0/878214098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c040756d0 msgr2=0x7f4c04075ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:51.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.302+0000 7f4c0b6a8700 1 --2- 172.21.15.100:0/878214098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c040756d0 0x7f4c04075ab0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f4c00009990 tx=0x7f4c00009ca0 comp rx=0 tx=0).stop 2024-05-10T11:32:51.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.302+0000 7f4c0b6a8700 1 -- 172.21.15.100:0/878214098 shutdown_connections 2024-05-10T11:32:51.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.302+0000 7f4c0b6a8700 1 --2- 172.21.15.100:0/878214098 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c04114670 0x7f4c04116a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:51.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.302+0000 7f4c0b6a8700 1 --2- 172.21.15.100:0/878214098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c040756d0 0x7f4c04075ab0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:51.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.302+0000 7f4c0b6a8700 1 --2- 172.21.15.100:0/878214098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c04075ff0 0x7f4c04114130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:51.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.302+0000 7f4c0b6a8700 1 -- 172.21.15.100:0/878214098 >> 172.21.15.100:0/878214098 conn(0x7f4c0407a600 msgr2=0x7f4c0407aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:32:51.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.302+0000 7f4c0b6a8700 1 -- 172.21.15.100:0/878214098 shutdown_connections 2024-05-10T11:32:51.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.303+0000 7f4c0b6a8700 1 -- 172.21.15.100:0/878214098 wait complete. 2024-05-10T11:32:51.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.303+0000 7f4c0b6a8700 1 Processor -- start 2024-05-10T11:32:51.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.303+0000 7f4c0b6a8700 1 -- start start 2024-05-10T11:32:51.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.304+0000 7f4c0b6a8700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c040756d0 0x7f4c041a0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:51.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.304+0000 7f4c0b6a8700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c04075ff0 0x7f4c041a0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:51.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.304+0000 7f4c0b6a8700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c04114670 0x7f4c041a4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:51.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.304+0000 7f4c0b6a8700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4c04119b10 con 0x7f4c04114670 2024-05-10T11:32:51.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.304+0000 7f4c0b6a8700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4c04119990 con 0x7f4c040756d0 2024-05-10T11:32:51.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.304+0000 7f4c0b6a8700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4c04119c90 con 0x7f4c04075ff0 2024-05-10T11:32:51.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.304+0000 7f4c09c45700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c04114670 0x7f4c041a4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:51.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.304+0000 7f4c09444700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c040756d0 0x7f4c041a0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:51.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.304+0000 7f4c09444700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c040756d0 0x7f4c041a0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:40890/0 (socket says 172.21.15.100:40890) 2024-05-10T11:32:51.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.304+0000 7f4c09c45700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c04114670 0x7f4c041a4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48622/0 (socket says 172.21.15.100:48622) 2024-05-10T11:32:51.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.304+0000 7f4c09444700 1 -- 172.21.15.100:0/4073853745 learned_addr learned my addr 172.21.15.100:0/4073853745 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:32:51.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.304+0000 7f4c08c43700 1 --2- 172.21.15.100:0/4073853745 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c04075ff0 0x7f4c041a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:51.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.304+0000 7f4c09c45700 1 -- 172.21.15.100:0/4073853745 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c04075ff0 msgr2=0x7f4c041a0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:51.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.304+0000 7f4c09c45700 1 --2- 172.21.15.100:0/4073853745 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c04075ff0 0x7f4c041a0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:51.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.304+0000 7f4c09c45700 1 -- 172.21.15.100:0/4073853745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c040756d0 msgr2=0x7f4c041a0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:51.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.304+0000 7f4c09c45700 1 --2- 172.21.15.100:0/4073853745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c040756d0 0x7f4c041a0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:51.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.304+0000 7f4c09c45700 1 -- 172.21.15.100:0/4073853745 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c041a5050 con 0x7f4c04114670 2024-05-10T11:32:51.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.304+0000 7f4c09444700 1 --2- 172.21.15.100:0/4073853745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c040756d0 0x7f4c041a0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:32:51.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.305+0000 7f4c09c45700 1 --2- 172.21.15.100:0/4073853745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c04114670 0x7f4c041a4910 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc00ef90 tx=0x7f4bfc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:32:51.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.305+0000 7f4bfa7fc700 1 -- 172.21.15.100:0/4073853745 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bfc0092e0 con 0x7f4c04114670 2024-05-10T11:32:51.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.305+0000 7f4bfa7fc700 1 -- 172.21.15.100:0/4073853745 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4bfc004bb0 con 0x7f4c04114670 2024-05-10T11:32:51.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.305+0000 7f4bfa7fc700 1 -- 172.21.15.100:0/4073853745 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bfc004020 con 0x7f4c04114670 2024-05-10T11:32:51.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.305+0000 7f4c0b6a8700 1 -- 172.21.15.100:0/4073853745 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c041a52e0 con 0x7f4c04114670 2024-05-10T11:32:51.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.305+0000 7f4c0b6a8700 1 -- 172.21.15.100:0/4073853745 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c041a5710 con 0x7f4c04114670 2024-05-10T11:32:51.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.308+0000 7f4bfa7fc700 1 -- 172.21.15.100:0/4073853745 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f4bfc01b030 con 0x7f4c04114670 2024-05-10T11:32:51.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.309+0000 7f4bfa7fc700 1 --2- 172.21.15.100:0/4073853745 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4bf00b29d0 0x7f4bf00b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:51.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.310+0000 7f4c09444700 1 --2- 172.21.15.100:0/4073853745 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4bf00b29d0 0x7f4bf00b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:51.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.310+0000 7f4bfa7fc700 1 -- 172.21.15.100:0/4073853745 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f4bfc0e1f70 con 0x7f4c04114670 2024-05-10T11:32:51.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.310+0000 7f4c09444700 1 --2- 172.21.15.100:0/4073853745 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4bf00b29d0 0x7f4bf00b4e90 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4c00006010 tx=0x7f4c0000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:32:51.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.311+0000 7f4c15f3bb80 1 Processor -- start 2024-05-10T11:32:51.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.311+0000 7f4c15f3bb80 1 -- start start 2024-05-10T11:32:51.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.311+0000 7f4c15f3bb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eed63fa1b0 0x55eed63fa590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:51.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.311+0000 7f4c15f3bb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eed63faad0 0x55eed63f4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:51.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.311+0000 7f4c15f3bb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eed63f4d50 0x55eed63f5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:51.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.311+0000 7f4c15f3bb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55eed6346940 con 0x55eed63faad0 2024-05-10T11:32:51.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.311+0000 7f4c15f3bb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55eed62abda0 con 0x55eed63f4d50 2024-05-10T11:32:51.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.311+0000 7f4c15f3bb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55eed6342900 con 0x55eed63fa1b0 2024-05-10T11:32:51.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.311+0000 7f4c09444700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eed63fa1b0 0x55eed63fa590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:51.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.311+0000 7f4c09444700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eed63fa1b0 0x55eed63fa590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:49796/0 (socket says 172.21.15.100:49796) 2024-05-10T11:32:51.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.311+0000 7f4c09c45700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eed63f4d50 0x55eed63f5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:51.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.311+0000 7f4c09444700 1 -- 172.21.15.100:0/233196095 learned_addr learned my addr 172.21.15.100:0/233196095 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:32:51.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.312+0000 7f4c08c43700 1 --2- 172.21.15.100:0/233196095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eed63faad0 0x55eed63f4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:51.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.312+0000 7f4c09444700 1 -- 172.21.15.100:0/233196095 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eed63f4d50 msgr2=0x55eed63f5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:51.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.312+0000 7f4c09444700 1 --2- 172.21.15.100:0/233196095 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eed63f4d50 0x55eed63f5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:51.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.312+0000 7f4c09444700 1 -- 172.21.15.100:0/233196095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eed63faad0 msgr2=0x55eed63f4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:51.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.312+0000 7f4c09444700 1 --2- 172.21.15.100:0/233196095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eed63faad0 0x55eed63f4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:51.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.312+0000 7f4c09444700 1 -- 172.21.15.100:0/233196095 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eed62938d0 con 0x55eed63fa1b0 2024-05-10T11:32:51.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.312+0000 7f4c09444700 1 --2- 172.21.15.100:0/233196095 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eed63fa1b0 0x55eed63fa590 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4c00005cd0 tx=0x7f4c0001f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25bca387703645ad server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:32:51.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.312+0000 7f4bef7fe700 1 -- 172.21.15.100:0/233196095 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c0000bab0 con 0x55eed63fa1b0 2024-05-10T11:32:51.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.312+0000 7f4bef7fe700 1 -- 172.21.15.100:0/233196095 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4c00005630 con 0x55eed63fa1b0 2024-05-10T11:32:51.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.313+0000 7f4bef7fe700 1 -- 172.21.15.100:0/233196095 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c00026430 con 0x55eed63fa1b0 2024-05-10T11:32:51.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.313+0000 7f4c15f3bb80 1 -- 172.21.15.100:0/233196095 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eed63fa1b0 msgr2=0x55eed63fa590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:51.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.313+0000 7f4c15f3bb80 1 --2- 172.21.15.100:0/233196095 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eed63fa1b0 0x55eed63fa590 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4c00005cd0 tx=0x7f4c0001f000 comp rx=0 tx=0).stop 2024-05-10T11:32:51.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.313+0000 7f4c15f3bb80 1 -- 172.21.15.100:0/233196095 shutdown_connections 2024-05-10T11:32:51.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.314+0000 7f4c15f3bb80 1 --2- 172.21.15.100:0/233196095 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eed63f4d50 0x55eed63f5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:51.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.314+0000 7f4c15f3bb80 1 --2- 172.21.15.100:0/233196095 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eed63fa1b0 0x55eed63fa590 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:51.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.314+0000 7f4c15f3bb80 1 --2- 172.21.15.100:0/233196095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eed63faad0 0x55eed63f4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:51.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.314+0000 7f4c15f3bb80 1 -- 172.21.15.100:0/233196095 >> 172.21.15.100:0/233196095 conn(0x55eed6320b70 msgr2=0x55eed63ef2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:32:51.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.314+0000 7f4c15f3bb80 1 -- 172.21.15.100:0/233196095 shutdown_connections 2024-05-10T11:32:51.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.314+0000 7f4c15f3bb80 1 -- 172.21.15.100:0/233196095 wait complete. 2024-05-10T11:32:51.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.314+0000 7f4c15f3bb80 1 -- start start 2024-05-10T11:32:51.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.314+0000 7f4c15f3bb80 1 Processor -- start 2024-05-10T11:32:51.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.315+0000 7f4c15f3bb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eed63f4d50 0x55eed64005c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:51.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.315+0000 7f4c15f3bb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eed63fa1b0 0x55eed63fd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:51.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.315+0000 7f4c15f3bb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eed63faad0 0x55eed63fdea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:51.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.315+0000 7f4c15f3bb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55eed6401490 con 0x55eed63faad0 2024-05-10T11:32:51.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.315+0000 7f4c15f3bb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55eed62ffed0 con 0x55eed63fa1b0 2024-05-10T11:32:51.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.315+0000 7f4c15f3bb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55eed6401610 con 0x55eed63f4d50 2024-05-10T11:32:51.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.315+0000 7f4c09444700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eed63f4d50 0x55eed64005c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:51.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.315+0000 7f4c09444700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eed63f4d50 0x55eed64005c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:49808/0 (socket says 172.21.15.100:49808) 2024-05-10T11:32:51.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.315+0000 7f4c09444700 1 -- 172.21.15.100:0/2069028876 learned_addr learned my addr 172.21.15.100:0/2069028876 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:32:51.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.315+0000 7f4c15f3bb80 10 client.0 _ll_register_callbacks cb 0x55eed62da5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:32:51.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.315+0000 7f4c08c43700 1 --2- 172.21.15.100:0/2069028876 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eed63fa1b0 0x55eed63fd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:51.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.315+0000 7f4c09c45700 1 --2- 172.21.15.100:0/2069028876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eed63faad0 0x55eed63fdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:51.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.315+0000 7f4c08c43700 1 -- 172.21.15.100:0/2069028876 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eed63f4d50 msgr2=0x55eed64005c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:51.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.315+0000 7f4c08c43700 1 --2- 172.21.15.100:0/2069028876 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eed63f4d50 0x55eed64005c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:51.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.315+0000 7f4c08c43700 1 -- 172.21.15.100:0/2069028876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eed63faad0 msgr2=0x55eed63fdea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:51.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.315+0000 7f4c08c43700 1 --2- 172.21.15.100:0/2069028876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eed63faad0 0x55eed63fdea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:51.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.315+0000 7f4c08c43700 1 -- 172.21.15.100:0/2069028876 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eed62fd760 con 0x55eed63fa1b0 2024-05-10T11:32:51.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.315+0000 7f4c09c45700 1 --2- 172.21.15.100:0/2069028876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eed63faad0 0x55eed63fdea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:32:51.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.315+0000 7f4c08c43700 1 --2- 172.21.15.100:0/2069028876 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eed63fa1b0 0x55eed63fd960 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x55eed63f0ed0 tx=0x7f4bf4012960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54fc858949555405 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:32:51.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.316+0000 7f4becff9700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:32:51.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.316+0000 7f4becff9700 1 -- 172.21.15.100:0/2069028876 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf4010430 con 0x55eed63fa1b0 2024-05-10T11:32:51.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.316+0000 7f4c15f3bb80 1 -- 172.21.15.100:0/2069028876 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c00005630 con 0x55eed63fa1b0 2024-05-10T11:32:51.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.316+0000 7f4becff9700 1 -- 172.21.15.100:0/2069028876 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f4bf40105d0 con 0x55eed63fa1b0 2024-05-10T11:32:51.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.316+0000 7f4becff9700 1 -- 172.21.15.100:0/2069028876 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f4bf40107d0 con 0x55eed63fa1b0 2024-05-10T11:32:51.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.316+0000 7f4c15f3bb80 10 client.27680 fetch_fsmap learned FSMap version 37 2024-05-10T11:32:51.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.316+0000 7f4c15f3bb80 1 -- 172.21.15.100:0/2069028876 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55eed640bcb0 con 0x55eed63fa1b0 2024-05-10T11:32:51.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.316+0000 7f4becff9700 1 -- 172.21.15.100:0/2069028876 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4c00005630 con 0x55eed63fa1b0 2024-05-10T11:32:51.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.317+0000 7f4becff9700 1 -- 172.21.15.100:0/2069028876 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f4bf401c890 con 0x55eed63fa1b0 2024-05-10T11:32:51.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.317+0000 7f4c15f3bb80 10 client.27680 fetch_fsmap finished waiting for FSMap version 37 2024-05-10T11:32:51.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.317+0000 7f4c15f3bb80 10 client.27680 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:32:51.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.317+0000 7f4c15f3bb80 20 client.27680 populate_metadata read hostname 'smithi100' 2024-05-10T11:32:51.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.317+0000 7f4c15f3bb80 1 --2- 172.21.15.100:0/2069028876 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55eed640e6c0 0x55eed6410ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:51.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.317+0000 7f4c15f3bb80 4 client.27680 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:32:51.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.317+0000 7f4c15f3bb80 1 -- 172.21.15.100:0/2069028876 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eed6354320 con 0x55eed640e6c0 2024-05-10T11:32:51.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.317+0000 7f4c09444700 1 --2- 172.21.15.100:0/2069028876 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55eed640e6c0 0x55eed6410ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:51.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.317+0000 7f4c09444700 1 --2- 172.21.15.100:0/2069028876 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55eed640e6c0 0x55eed6410ae0 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa1848bc984037a7 server_cookie=f27ce13bff5e9a36 in_seq=0 out_seq=0 2024-05-10T11:32:51.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.318+0000 7f4becff9700 10 client.27680 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:32:51.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.319+0000 7f4becff9700 1 -- 172.21.15.100:0/2069028876 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55eed6354320 con 0x55eed640e6c0 2024-05-10T11:32:51.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.319+0000 7f4becff9700 10 client.27680 handle_command_reply: tid=0 2024-05-10T11:32:51.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.319+0000 7f4c15f3bb80 1 client.27680 shutdown 2024-05-10T11:32:51.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.319+0000 7f4c15f3bb80 1 -- 172.21.15.100:0/2069028876 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eed63fa1b0 msgr2=0x55eed63fd960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:51.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.319+0000 7f4c15f3bb80 1 --2- 172.21.15.100:0/2069028876 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eed63fa1b0 0x55eed63fd960 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x55eed63f0ed0 tx=0x7f4bf4012960 comp rx=0 tx=0).stop 2024-05-10T11:32:51.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.320+0000 7f4c15f3bb80 1 -- 172.21.15.100:0/2069028876 shutdown_connections 2024-05-10T11:32:51.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.320+0000 7f4c15f3bb80 1 --2- 172.21.15.100:0/2069028876 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55eed640e6c0 0x55eed6410ae0 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:51.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.320+0000 7f4c15f3bb80 1 --2- 172.21.15.100:0/2069028876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eed63faad0 0x55eed63fdea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:51.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.320+0000 7f4c15f3bb80 1 --2- 172.21.15.100:0/2069028876 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eed63f4d50 0x55eed64005c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:51.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.320+0000 7f4becff9700 0 client.27680 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:32:51.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.320+0000 7f4c15f3bb80 1 --2- 172.21.15.100:0/2069028876 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eed63fa1b0 0x55eed63fd960 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:51.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.320+0000 7f4c15f3bb80 1 -- 172.21.15.100:0/2069028876 >> 172.21.15.100:0/2069028876 conn(0x55eed6320b70 msgr2=0x55eed63e9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:32:51.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.320+0000 7f4c15f3bb80 1 -- 172.21.15.100:0/2069028876 shutdown_connections 2024-05-10T11:32:51.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.320+0000 7f4c15f3bb80 1 -- 172.21.15.100:0/2069028876 wait complete. 2024-05-10T11:32:51.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.320+0000 7f4c15f3bb80 20 client.27680 trim_cache size 0 max 16384 2024-05-10T11:32:51.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.347+0000 7f4c15f3bb80 1 Processor -- start 2024-05-10T11:32:51.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.348+0000 7f4c15f3bb80 1 -- start start 2024-05-10T11:32:51.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.348+0000 7f4c15f3bb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eed63f4d50 0x55eed63fb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:51.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.348+0000 7f4c15f3bb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eed63fa1b0 0x55eed63fe480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:51.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.348+0000 7f4c15f3bb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eed63faad0 0x55eed63e9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:51.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.348+0000 7f4c15f3bb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4c041b7f00 con 0x55eed63fa1b0 2024-05-10T11:32:51.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.348+0000 7f4c15f3bb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4c041b5700 con 0x55eed63f4d50 2024-05-10T11:32:51.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.348+0000 7f4c15f3bb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4c041b3550 con 0x55eed63faad0 2024-05-10T11:32:51.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.348+0000 7f4c09c45700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eed63faad0 0x55eed63e9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:51.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.348+0000 7f4c09444700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eed63f4d50 0x55eed63fb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:51.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.348+0000 7f4c09c45700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eed63faad0 0x55eed63e9c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:49814/0 (socket says 172.21.15.100:49814) 2024-05-10T11:32:51.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.348+0000 7f4c09c45700 1 -- 172.21.15.100:0/4085228888 learned_addr learned my addr 172.21.15.100:0/4085228888 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:32:51.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.348+0000 7f4c08c43700 1 --2- 172.21.15.100:0/4085228888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eed63fa1b0 0x55eed63fe480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:51.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.348+0000 7f4c09c45700 1 -- 172.21.15.100:0/4085228888 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eed63f4d50 msgr2=0x55eed63fb5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:51.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.348+0000 7f4c09c45700 1 --2- 172.21.15.100:0/4085228888 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eed63f4d50 0x55eed63fb5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:51.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.348+0000 7f4c09c45700 1 -- 172.21.15.100:0/4085228888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eed63fa1b0 msgr2=0x55eed63fe480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:51.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.348+0000 7f4c09c45700 1 --2- 172.21.15.100:0/4085228888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eed63fa1b0 0x55eed63fe480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:51.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.348+0000 7f4c09c45700 1 -- 172.21.15.100:0/4085228888 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eed6411140 con 0x55eed63faad0 2024-05-10T11:32:51.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.349+0000 7f4c09c45700 1 --2- 172.21.15.100:0/4085228888 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eed63faad0 0x55eed63e9c40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc00c9f0 tx=0x7f4bfc0a8dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6276f1bdd4178685 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:32:51.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.349+0000 7f4beeffd700 1 -- 172.21.15.100:0/4085228888 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bfc0c2020 con 0x55eed63faad0 2024-05-10T11:32:51.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.349+0000 7f4beeffd700 1 -- 172.21.15.100:0/4085228888 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4bfc0a92c0 con 0x55eed63faad0 2024-05-10T11:32:51.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.349+0000 7f4beeffd700 1 -- 172.21.15.100:0/4085228888 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bfc0b1c20 con 0x55eed63faad0 2024-05-10T11:32:51.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.350+0000 7f4c15f3bb80 1 -- 172.21.15.100:0/4085228888 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eed63faad0 msgr2=0x55eed63e9c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:51.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.350+0000 7f4c15f3bb80 1 --2- 172.21.15.100:0/4085228888 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eed63faad0 0x55eed63e9c40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc00c9f0 tx=0x7f4bfc0a8dd0 comp rx=0 tx=0).stop 2024-05-10T11:32:51.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.350+0000 7f4c15f3bb80 1 -- 172.21.15.100:0/4085228888 shutdown_connections 2024-05-10T11:32:51.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.350+0000 7f4c15f3bb80 1 --2- 172.21.15.100:0/4085228888 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eed63faad0 0x55eed63e9c40 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:51.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.350+0000 7f4c15f3bb80 1 --2- 172.21.15.100:0/4085228888 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eed63f4d50 0x55eed63fb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:51.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.350+0000 7f4c15f3bb80 1 --2- 172.21.15.100:0/4085228888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eed63fa1b0 0x55eed63fe480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:51.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.350+0000 7f4c15f3bb80 1 -- 172.21.15.100:0/4085228888 >> 172.21.15.100:0/4085228888 conn(0x55eed6320b70 msgr2=0x55eed63faeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:32:51.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.350+0000 7f4c15f3bb80 1 -- 172.21.15.100:0/4085228888 shutdown_connections 2024-05-10T11:32:51.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.350+0000 7f4c15f3bb80 1 -- 172.21.15.100:0/4085228888 wait complete. 2024-05-10T11:32:51.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.350+0000 7f4c15f3bb80 1 -- start start 2024-05-10T11:32:51.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.350+0000 7f4c15f3bb80 1 Processor -- start 2024-05-10T11:32:51.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.351+0000 7f4c15f3bb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eed63f4d50 0x55eed64068f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:51.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.351+0000 7f4c15f3bb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eed63fa1b0 0x55eed6406e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:51.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.351+0000 7f4c15f3bb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eed63faad0 0x55eed64153c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:51.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.351+0000 7f4c15f3bb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4c041b3bc0 con 0x55eed63f4d50 2024-05-10T11:32:51.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.351+0000 7f4c15f3bb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4c041b5030 con 0x55eed63faad0 2024-05-10T11:32:51.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.351+0000 7f4c15f3bb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4c041b4290 con 0x55eed63fa1b0 2024-05-10T11:32:51.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.351+0000 7f4c15f3bb80 10 client.0 _ll_register_callbacks cb 0x55eed62da5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:32:51.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.351+0000 7f4c09444700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eed63f4d50 0x55eed64068f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:51.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.351+0000 7f4c09444700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eed63f4d50 0x55eed64068f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48666/0 (socket says 172.21.15.100:48666) 2024-05-10T11:32:51.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.351+0000 7f4c09444700 1 -- 172.21.15.100:0/2389837353 learned_addr learned my addr 172.21.15.100:0/2389837353 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:32:51.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.351+0000 7f4c09c45700 1 --2- 172.21.15.100:0/2389837353 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eed63faad0 0x55eed64153c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:51.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.352+0000 7f4c08c43700 1 --2- 172.21.15.100:0/2389837353 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eed63fa1b0 0x55eed6406e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:51.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.352+0000 7f4c09444700 1 -- 172.21.15.100:0/2389837353 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eed63fa1b0 msgr2=0x55eed6406e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:51.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.352+0000 7f4c09444700 1 --2- 172.21.15.100:0/2389837353 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eed63fa1b0 0x55eed6406e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:51.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.352+0000 7f4c09444700 1 -- 172.21.15.100:0/2389837353 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eed63faad0 msgr2=0x55eed64153c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:51.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.352+0000 7f4c09444700 1 --2- 172.21.15.100:0/2389837353 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eed63faad0 0x55eed64153c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:51.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.352+0000 7f4c09444700 1 -- 172.21.15.100:0/2389837353 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eed640bae0 con 0x55eed63f4d50 2024-05-10T11:32:51.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.352+0000 7f4c09444700 1 --2- 172.21.15.100:0/2389837353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eed63f4d50 0x55eed64068f0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f4c0000b650 tx=0x7f4c0000ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66fb5ba864916a16 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:32:51.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.352+0000 7f4becff9700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:32:51.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.352+0000 7f4becff9700 1 -- 172.21.15.100:0/2389837353 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c0003f070 con 0x55eed63f4d50 2024-05-10T11:32:51.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.352+0000 7f4becff9700 1 -- 172.21.15.100:0/2389837353 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f4c00003d10 con 0x55eed63f4d50 2024-05-10T11:32:51.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.352+0000 7f4c15f3bb80 1 -- 172.21.15.100:0/2389837353 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4bfc0a92c0 con 0x55eed63f4d50 2024-05-10T11:32:51.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.353+0000 7f4becff9700 1 -- 172.21.15.100:0/2389837353 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f4c00048050 con 0x55eed63f4d50 2024-05-10T11:32:51.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.353+0000 7f4c15f3bb80 10 client.18339 fetch_fsmap learned FSMap version 37 2024-05-10T11:32:51.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.353+0000 7f4c15f3bb80 1 -- 172.21.15.100:0/2389837353 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55eed64784b0 con 0x55eed63f4d50 2024-05-10T11:32:51.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.353+0000 7f4becff9700 1 -- 172.21.15.100:0/2389837353 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4bfc0a92c0 con 0x55eed63f4d50 2024-05-10T11:32:51.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.353+0000 7f4becff9700 1 -- 172.21.15.100:0/2389837353 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f4c00042020 con 0x55eed63f4d50 2024-05-10T11:32:51.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.353+0000 7f4c15f3bb80 10 client.18339 fetch_fsmap finished waiting for FSMap version 37 2024-05-10T11:32:51.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.353+0000 7f4c15f3bb80 10 client.18339 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:32:51.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.353+0000 7f4c15f3bb80 20 client.18339 populate_metadata read hostname 'smithi100' 2024-05-10T11:32:51.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.353+0000 7f4c15f3bb80 1 --2- 172.21.15.100:0/2389837353 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55eed6478740 0x55eed647ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:51.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.355+0000 7f4c15f3bb80 4 client.18339 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:32:51.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.355+0000 7f4c15f3bb80 1 -- 172.21.15.100:0/2389837353 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55eed634e3c0 con 0x55eed6478740 2024-05-10T11:32:51.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.355+0000 7f4c08c43700 1 --2- 172.21.15.100:0/2389837353 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55eed6478740 0x55eed647ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:51.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.355+0000 7f4becff9700 10 client.18339 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:32:51.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.355+0000 7f4c08c43700 1 --2- 172.21.15.100:0/2389837353 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55eed6478740 0x55eed647ab30 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffb9a404ea9bd0e4 server_cookie=403f1172d829eb18 in_seq=0 out_seq=0 2024-05-10T11:32:51.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.355+0000 7f4becff9700 1 -- 172.21.15.100:0/2389837353 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55eed634e3c0 con 0x55eed6478740 2024-05-10T11:32:51.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.355+0000 7f4becff9700 10 client.18339 handle_command_reply: tid=0 2024-05-10T11:32:51.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.355+0000 7f4c15f3bb80 1 client.18339 shutdown 2024-05-10T11:32:51.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.355+0000 7f4c15f3bb80 1 -- 172.21.15.100:0/2389837353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eed63f4d50 msgr2=0x55eed64068f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:51.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.355+0000 7f4c15f3bb80 1 --2- 172.21.15.100:0/2389837353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eed63f4d50 0x55eed64068f0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f4c0000b650 tx=0x7f4c0000ba90 comp rx=0 tx=0).stop 2024-05-10T11:32:51.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.355+0000 7f4c15f3bb80 1 -- 172.21.15.100:0/2389837353 shutdown_connections 2024-05-10T11:32:51.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.355+0000 7f4c15f3bb80 1 --2- 172.21.15.100:0/2389837353 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55eed6478740 0x55eed647ab30 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:51.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.355+0000 7f4c15f3bb80 1 --2- 172.21.15.100:0/2389837353 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eed63faad0 0x55eed64153c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:51.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.355+0000 7f4c15f3bb80 1 --2- 172.21.15.100:0/2389837353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eed63f4d50 0x55eed64068f0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:51.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.355+0000 7f4c15f3bb80 1 --2- 172.21.15.100:0/2389837353 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eed63fa1b0 0x55eed6406e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:51.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.355+0000 7f4c15f3bb80 1 -- 172.21.15.100:0/2389837353 >> 172.21.15.100:0/2389837353 conn(0x55eed6320b70 msgr2=0x55eed63fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:32:51.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.355+0000 7f4becff9700 0 client.18339 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:32:51.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.356+0000 7f4c15f3bb80 1 -- 172.21.15.100:0/2389837353 shutdown_connections 2024-05-10T11:32:51.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.356+0000 7f4c15f3bb80 1 -- 172.21.15.100:0/2389837353 wait complete. 2024-05-10T11:32:51.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.356+0000 7f4c15f3bb80 20 client.18339 trim_cache size 0 max 16384 2024-05-10T11:32:51.360 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:32:51.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.357+0000 7f4c0b6a8700 1 -- 172.21.15.100:0/4073853745 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4bf00b29d0 msgr2=0x7f4bf00b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:51.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.357+0000 7f4c0b6a8700 1 --2- 172.21.15.100:0/4073853745 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4bf00b29d0 0x7f4bf00b4e90 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4c00006010 tx=0x7f4c0000b260 comp rx=0 tx=0).stop 2024-05-10T11:32:51.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.357+0000 7f4c0b6a8700 1 -- 172.21.15.100:0/4073853745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c04114670 msgr2=0x7f4c041a4910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:51.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.357+0000 7f4c0b6a8700 1 --2- 172.21.15.100:0/4073853745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c04114670 0x7f4c041a4910 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc00ef90 tx=0x7f4bfc00c3a0 comp rx=0 tx=0).stop 2024-05-10T11:32:51.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.357+0000 7f4c0b6a8700 1 -- 172.21.15.100:0/4073853745 shutdown_connections 2024-05-10T11:32:51.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.357+0000 7f4c0b6a8700 1 --2- 172.21.15.100:0/4073853745 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4bf00b29d0 0x7f4bf00b4e90 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:51.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.357+0000 7f4c0b6a8700 1 --2- 172.21.15.100:0/4073853745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c04114670 0x7f4c041a4910 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:51.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.357+0000 7f4c0b6a8700 1 --2- 172.21.15.100:0/4073853745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c040756d0 0x7f4c041a0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:51.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.357+0000 7f4c0b6a8700 1 --2- 172.21.15.100:0/4073853745 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c04075ff0 0x7f4c041a0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:51.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.357+0000 7f4c0b6a8700 1 -- 172.21.15.100:0/4073853745 >> 172.21.15.100:0/4073853745 conn(0x7f4c0407a600 msgr2=0x7f4c04113510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:32:51.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.357+0000 7f4c0b6a8700 1 -- 172.21.15.100:0/4073853745 shutdown_connections 2024-05-10T11:32:51.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:51.357+0000 7f4c0b6a8700 1 -- 172.21.15.100:0/4073853745 wait complete. 2024-05-10T11:32:51.907 INFO:tasks.workunit.client.0.smithi100.stdout: 10 7759 13.33 MB/sec warmup 45 sec latency 146.701 ms 2024-05-10T11:32:52.013 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:32:51 smithi100 ceph-mon[30296]: pgmap v855: 129 pgs: 129 active+clean; 354 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 4.2 MiB/s wr, 221 op/s 2024-05-10T11:32:52.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:32:51 smithi178 ceph-mon[33359]: pgmap v855: 129 pgs: 129 active+clean; 354 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 4.2 MiB/s wr, 221 op/s 2024-05-10T11:32:52.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:32:51 smithi188 ceph-mon[34075]: pgmap v855: 129 pgs: 129 active+clean; 354 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 4.2 MiB/s wr, 221 op/s 2024-05-10T11:32:52.908 INFO:tasks.workunit.client.0.smithi100.stdout: 10 7903 13.15 MB/sec warmup 46 sec latency 110.845 ms 2024-05-10T11:32:53.048 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T11:32:53.048 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T11:32:53.403 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:32:53.908 INFO:tasks.workunit.client.0.smithi100.stdout: 10 8070 12.97 MB/sec warmup 47 sec latency 75.060 ms 2024-05-10T11:32:54.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:32:53 smithi178 ceph-mon[33359]: pgmap v856: 129 pgs: 129 active+clean; 354 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 821 KiB/s rd, 4.9 MiB/s wr, 256 op/s 2024-05-10T11:32:54.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:32:53 smithi188 ceph-mon[34075]: pgmap v856: 129 pgs: 129 active+clean; 354 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 821 KiB/s rd, 4.9 MiB/s wr, 256 op/s 2024-05-10T11:32:54.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:32:53 smithi100 ceph-mon[30296]: pgmap v856: 129 pgs: 129 active+clean; 354 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 821 KiB/s rd, 4.9 MiB/s wr, 256 op/s 2024-05-10T11:32:54.908 INFO:tasks.workunit.client.0.smithi100.stdout: 10 8238 12.84 MB/sec warmup 48 sec latency 63.332 ms 2024-05-10T11:32:55.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.292+0000 7f1efc97f700 1 -- 172.21.15.100:0/2413708488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ef8102080 msgr2=0x7f1ef810e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:55.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.292+0000 7f1efc97f700 1 --2- 172.21.15.100:0/2413708488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ef8102080 0x7f1ef810e590 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f1ef000b0a0 tx=0x7f1ef000b3b0 comp rx=0 tx=0).stop 2024-05-10T11:32:55.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.292+0000 7f1efc97f700 1 -- 172.21.15.100:0/2413708488 shutdown_connections 2024-05-10T11:32:55.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.292+0000 7f1efc97f700 1 --2- 172.21.15.100:0/2413708488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ef8102080 0x7f1ef810e590 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:55.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.292+0000 7f1efc97f700 1 --2- 172.21.15.100:0/2413708488 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ef81076c0 0x7f1ef8107aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:55.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.292+0000 7f1efc97f700 1 --2- 172.21.15.100:0/2413708488 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ef81016c0 0x7f1ef8101b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:55.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.292+0000 7f1efc97f700 1 -- 172.21.15.100:0/2413708488 >> 172.21.15.100:0/2413708488 conn(0x7f1ef80785a0 msgr2=0x7f1ef80ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:32:55.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.292+0000 7f1efc97f700 1 -- 172.21.15.100:0/2413708488 shutdown_connections 2024-05-10T11:32:55.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.292+0000 7f1efc97f700 1 -- 172.21.15.100:0/2413708488 wait complete. 2024-05-10T11:32:55.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.293+0000 7f1efc97f700 1 Processor -- start 2024-05-10T11:32:55.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.293+0000 7f1efc97f700 1 -- start start 2024-05-10T11:32:55.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.294+0000 7f1efc97f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ef81016c0 0x7f1ef80fda10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:55.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.294+0000 7f1efc97f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ef8102080 0x7f1ef80fdf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:55.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.294+0000 7f1efc97f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ef81076c0 0x7f1ef80fe490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:55.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.294+0000 7f1efc97f700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef81113b0 con 0x7f1ef81076c0 2024-05-10T11:32:55.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.294+0000 7f1efc97f700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef8111230 con 0x7f1ef8102080 2024-05-10T11:32:55.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.294+0000 7f1efc97f700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef8111530 con 0x7f1ef81016c0 2024-05-10T11:32:55.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.294+0000 7f1ef6d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ef81076c0 0x7f1ef80fe490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:55.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.294+0000 7f1ef659c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ef81016c0 0x7f1ef80fda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:55.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.294+0000 7f1ef6d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ef81076c0 0x7f1ef80fe490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48690/0 (socket says 172.21.15.100:48690) 2024-05-10T11:32:55.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.294+0000 7f1ef5d9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ef8102080 0x7f1ef80fdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:55.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.294+0000 7f1ef6d9d700 1 -- 172.21.15.100:0/9727099 learned_addr learned my addr 172.21.15.100:0/9727099 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:32:55.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.294+0000 7f1ef5d9b700 1 -- 172.21.15.100:0/9727099 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ef81016c0 msgr2=0x7f1ef80fda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:55.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.294+0000 7f1ef5d9b700 1 --2- 172.21.15.100:0/9727099 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ef81016c0 0x7f1ef80fda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:55.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.294+0000 7f1ef5d9b700 1 -- 172.21.15.100:0/9727099 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ef81076c0 msgr2=0x7f1ef80fe490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:55.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.294+0000 7f1ef5d9b700 1 --2- 172.21.15.100:0/9727099 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ef81076c0 0x7f1ef80fe490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:55.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.294+0000 7f1ef5d9b700 1 -- 172.21.15.100:0/9727099 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ef81a7fc0 con 0x7f1ef8102080 2024-05-10T11:32:55.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.294+0000 7f1ef659c700 1 --2- 172.21.15.100:0/9727099 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ef81016c0 0x7f1ef80fda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:32:55.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.294+0000 7f1ef6d9d700 1 --2- 172.21.15.100:0/9727099 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ef81076c0 0x7f1ef80fe490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:32:55.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.295+0000 7f1ef5d9b700 1 --2- 172.21.15.100:0/9727099 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ef8102080 0x7f1ef80fdf50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1eec009800 tx=0x7f1eec009b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:32:55.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.295+0000 7f1ee77fe700 1 -- 172.21.15.100:0/9727099 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eec018020 con 0x7f1ef8102080 2024-05-10T11:32:55.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.295+0000 7f1ee77fe700 1 -- 172.21.15.100:0/9727099 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1eec014cb0 con 0x7f1ef8102080 2024-05-10T11:32:55.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.295+0000 7f1efc97f700 1 -- 172.21.15.100:0/9727099 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ef81a8250 con 0x7f1ef8102080 2024-05-10T11:32:55.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.295+0000 7f1ee77fe700 1 -- 172.21.15.100:0/9727099 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eec013d20 con 0x7f1ef8102080 2024-05-10T11:32:55.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.295+0000 7f1efc97f700 1 -- 172.21.15.100:0/9727099 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ef81a87e0 con 0x7f1ef8102080 2024-05-10T11:32:55.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.298+0000 7f1ee77fe700 1 -- 172.21.15.100:0/9727099 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f1eec01b020 con 0x7f1ef8102080 2024-05-10T11:32:55.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.299+0000 7f1ee77fe700 1 --2- 172.21.15.100:0/9727099 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f1ee00b2900 0x7f1ee00b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:55.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.300+0000 7f1ee77fe700 1 -- 172.21.15.100:0/9727099 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f1eec0e1e30 con 0x7f1ef8102080 2024-05-10T11:32:55.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.300+0000 7f1ef659c700 1 --2- 172.21.15.100:0/9727099 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f1ee00b2900 0x7f1ee00b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:55.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.301+0000 7f1ef659c700 1 --2- 172.21.15.100:0/9727099 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f1ee00b2900 0x7f1ee00b4dc0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f1ee8009c90 tx=0x7f1ee8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:32:55.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.302+0000 7f1f07212b80 1 Processor -- start 2024-05-10T11:32:55.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.302+0000 7f1f07212b80 1 -- start start 2024-05-10T11:32:55.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.303+0000 7f1f07212b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564d4d80230 0x5564d4d80610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:55.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.303+0000 7f1f07212b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564d4d80b50 0x5564d4d7a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:55.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.303+0000 7f1f07212b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564d4d7add0 0x5564d4d7b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:55.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.303+0000 7f1f07212b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5564d4ccc9c0 con 0x5564d4d7add0 2024-05-10T11:32:55.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.303+0000 7f1f07212b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5564d4c31e20 con 0x5564d4d80230 2024-05-10T11:32:55.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.303+0000 7f1f07212b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5564d4cc8980 con 0x5564d4d80b50 2024-05-10T11:32:55.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.303+0000 7f1ef659c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564d4d80230 0x5564d4d80610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:55.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.303+0000 7f1ef5d9b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564d4d80b50 0x5564d4d7a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:55.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.303+0000 7f1ef5d9b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564d4d80b50 0x5564d4d7a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:49854/0 (socket says 172.21.15.100:49854) 2024-05-10T11:32:55.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.303+0000 7f1ef5d9b700 1 -- 172.21.15.100:0/1178727175 learned_addr learned my addr 172.21.15.100:0/1178727175 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:32:55.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.303+0000 7f1ef6d9d700 1 --2- 172.21.15.100:0/1178727175 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564d4d7add0 0x5564d4d7b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:55.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.303+0000 7f1ef659c700 1 -- 172.21.15.100:0/1178727175 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564d4d80b50 msgr2=0x5564d4d7a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:55.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.303+0000 7f1ef659c700 1 --2- 172.21.15.100:0/1178727175 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564d4d80b50 0x5564d4d7a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:55.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.303+0000 7f1ef659c700 1 -- 172.21.15.100:0/1178727175 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564d4d7add0 msgr2=0x5564d4d7b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:55.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.303+0000 7f1ef659c700 1 --2- 172.21.15.100:0/1178727175 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564d4d7add0 0x5564d4d7b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:55.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.303+0000 7f1ef659c700 1 -- 172.21.15.100:0/1178727175 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564d4c19950 con 0x5564d4d80230 2024-05-10T11:32:55.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.304+0000 7f1ef5d9b700 1 --2- 172.21.15.100:0/1178727175 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564d4d80b50 0x5564d4d7a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:32:55.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.304+0000 7f1ef659c700 1 --2- 172.21.15.100:0/1178727175 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564d4d80230 0x5564d4d80610 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1ee8010a40 tx=0x7f1ee8013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c1792c88eef62b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:32:55.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.304+0000 7f1ee4ff9700 1 -- 172.21.15.100:0/1178727175 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee8011bb0 con 0x5564d4d80230 2024-05-10T11:32:55.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.304+0000 7f1ee4ff9700 1 -- 172.21.15.100:0/1178727175 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1ee8011d50 con 0x5564d4d80230 2024-05-10T11:32:55.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.304+0000 7f1ee4ff9700 1 -- 172.21.15.100:0/1178727175 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee801c5f0 con 0x5564d4d80230 2024-05-10T11:32:55.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.305+0000 7f1f07212b80 1 -- 172.21.15.100:0/1178727175 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564d4d80230 msgr2=0x5564d4d80610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:55.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.305+0000 7f1f07212b80 1 --2- 172.21.15.100:0/1178727175 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564d4d80230 0x5564d4d80610 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1ee8010a40 tx=0x7f1ee8013000 comp rx=0 tx=0).stop 2024-05-10T11:32:55.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.305+0000 7f1f07212b80 1 -- 172.21.15.100:0/1178727175 shutdown_connections 2024-05-10T11:32:55.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.305+0000 7f1f07212b80 1 --2- 172.21.15.100:0/1178727175 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564d4d7add0 0x5564d4d7b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:55.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.305+0000 7f1f07212b80 1 --2- 172.21.15.100:0/1178727175 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564d4d80230 0x5564d4d80610 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:55.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.305+0000 7f1f07212b80 1 --2- 172.21.15.100:0/1178727175 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564d4d80b50 0x5564d4d7a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:55.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.305+0000 7f1f07212b80 1 -- 172.21.15.100:0/1178727175 >> 172.21.15.100:0/1178727175 conn(0x5564d4ca6bf0 msgr2=0x5564d4d75320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:32:55.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.305+0000 7f1f07212b80 1 -- 172.21.15.100:0/1178727175 shutdown_connections 2024-05-10T11:32:55.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.305+0000 7f1f07212b80 1 -- 172.21.15.100:0/1178727175 wait complete. 2024-05-10T11:32:55.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.306+0000 7f1f07212b80 1 -- start start 2024-05-10T11:32:55.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.306+0000 7f1f07212b80 1 Processor -- start 2024-05-10T11:32:55.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.307+0000 7f1f07212b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564d4d7add0 0x5564d4d86760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:55.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.307+0000 7f1f07212b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564d4d80230 0x5564d4d83b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:55.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.307+0000 7f1f07212b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564d4d80b50 0x5564d4d84040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:55.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.307+0000 7f1f07212b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5564d4d87570 con 0x5564d4d80230 2024-05-10T11:32:55.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.307+0000 7f1f07212b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5564d4c85f50 con 0x5564d4d7add0 2024-05-10T11:32:55.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.307+0000 7f1f07212b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5564d4d876f0 con 0x5564d4d80b50 2024-05-10T11:32:55.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.307+0000 7f1ef5d9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564d4d80230 0x5564d4d83b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:55.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.307+0000 7f1ef659c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564d4d7add0 0x5564d4d86760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:55.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.307+0000 7f1f07212b80 10 client.0 _ll_register_callbacks cb 0x5564d4c60630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:32:55.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.307+0000 7f1ef659c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564d4d7add0 0x5564d4d86760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:40956/0 (socket says 172.21.15.100:40956) 2024-05-10T11:32:55.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.307+0000 7f1ef659c700 1 -- 172.21.15.100:0/718774956 learned_addr learned my addr 172.21.15.100:0/718774956 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:32:55.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.307+0000 7f1ef6d9d700 1 --2- 172.21.15.100:0/718774956 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564d4d80b50 0x5564d4d84040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:55.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.308+0000 7f1ef6d9d700 1 -- 172.21.15.100:0/718774956 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564d4d7add0 msgr2=0x5564d4d86760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:55.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.308+0000 7f1ef6d9d700 1 --2- 172.21.15.100:0/718774956 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564d4d7add0 0x5564d4d86760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:55.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.308+0000 7f1ef6d9d700 1 -- 172.21.15.100:0/718774956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564d4d80230 msgr2=0x5564d4d83b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:55.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.308+0000 7f1ef6d9d700 1 --2- 172.21.15.100:0/718774956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564d4d80230 0x5564d4d83b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:55.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.308+0000 7f1ef6d9d700 1 -- 172.21.15.100:0/718774956 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564d4c837e0 con 0x5564d4d80b50 2024-05-10T11:32:55.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.308+0000 7f1ef5d9b700 1 --2- 172.21.15.100:0/718774956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564d4d80230 0x5564d4d83b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:32:55.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.308+0000 7f1ef6d9d700 1 --2- 172.21.15.100:0/718774956 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564d4d80b50 0x5564d4d84040 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1ef000b780 tx=0x7f1ef0003c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60e8a61b10418074 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:32:55.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.308+0000 7f1eddffb700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:32:55.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.308+0000 7f1eddffb700 1 -- 172.21.15.100:0/718774956 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef00076e0 con 0x5564d4d80b50 2024-05-10T11:32:55.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.308+0000 7f1f07212b80 1 -- 172.21.15.100:0/718774956 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ee8011d50 con 0x5564d4d80b50 2024-05-10T11:32:55.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.308+0000 7f1eddffb700 1 -- 172.21.15.100:0/718774956 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f1ef0007880 con 0x5564d4d80b50 2024-05-10T11:32:55.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.308+0000 7f1eddffb700 1 -- 172.21.15.100:0/718774956 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f1ef0007a80 con 0x5564d4d80b50 2024-05-10T11:32:55.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.308+0000 7f1f07212b80 10 client.27784 fetch_fsmap learned FSMap version 37 2024-05-10T11:32:55.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.308+0000 7f1f07212b80 1 -- 172.21.15.100:0/718774956 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5564d4d91c10 con 0x5564d4d80b50 2024-05-10T11:32:55.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.309+0000 7f1eddffb700 1 -- 172.21.15.100:0/718774956 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1ee8011d50 con 0x5564d4d80b50 2024-05-10T11:32:55.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.309+0000 7f1eddffb700 1 -- 172.21.15.100:0/718774956 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f1ef0023070 con 0x5564d4d80b50 2024-05-10T11:32:55.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.309+0000 7f1f07212b80 10 client.27784 fetch_fsmap finished waiting for FSMap version 37 2024-05-10T11:32:55.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.309+0000 7f1f07212b80 10 client.27784 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:32:55.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.309+0000 7f1f07212b80 20 client.27784 populate_metadata read hostname 'smithi100' 2024-05-10T11:32:55.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.309+0000 7f1f07212b80 1 --2- 172.21.15.100:0/718774956 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5564d4d941a0 0x5564d4d96590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:55.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.309+0000 7f1f07212b80 4 client.27784 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:32:55.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.309+0000 7f1f07212b80 1 -- 172.21.15.100:0/718774956 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5564d4cda3a0 con 0x5564d4d941a0 2024-05-10T11:32:55.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.309+0000 7f1ef659c700 1 --2- 172.21.15.100:0/718774956 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5564d4d941a0 0x5564d4d96590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:55.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.310+0000 7f1eddffb700 10 client.27784 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:32:55.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.310+0000 7f1ef659c700 1 --2- 172.21.15.100:0/718774956 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5564d4d941a0 0x5564d4d96590 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64e11007d4442863 server_cookie=f0030fbd71f44d9b in_seq=0 out_seq=0 2024-05-10T11:32:55.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.310+0000 7f1eddffb700 1 -- 172.21.15.100:0/718774956 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5564d4cda3a0 con 0x5564d4d941a0 2024-05-10T11:32:55.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.310+0000 7f1eddffb700 10 client.27784 handle_command_reply: tid=0 2024-05-10T11:32:55.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.311+0000 7f1f07212b80 1 client.27784 shutdown 2024-05-10T11:32:55.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.311+0000 7f1f07212b80 1 -- 172.21.15.100:0/718774956 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564d4d80b50 msgr2=0x5564d4d84040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:55.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.311+0000 7f1f07212b80 1 --2- 172.21.15.100:0/718774956 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564d4d80b50 0x5564d4d84040 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1ef000b780 tx=0x7f1ef0003c80 comp rx=0 tx=0).stop 2024-05-10T11:32:55.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.311+0000 7f1f07212b80 1 -- 172.21.15.100:0/718774956 shutdown_connections 2024-05-10T11:32:55.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.311+0000 7f1f07212b80 1 --2- 172.21.15.100:0/718774956 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5564d4d941a0 0x5564d4d96590 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:55.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.311+0000 7f1f07212b80 1 --2- 172.21.15.100:0/718774956 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564d4d80b50 0x5564d4d84040 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:55.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.311+0000 7f1f07212b80 1 --2- 172.21.15.100:0/718774956 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564d4d7add0 0x5564d4d86760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:55.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.311+0000 7f1f07212b80 1 --2- 172.21.15.100:0/718774956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564d4d80230 0x5564d4d83b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:55.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.311+0000 7f1eddffb700 0 client.27784 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:32:55.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.311+0000 7f1f07212b80 1 -- 172.21.15.100:0/718774956 >> 172.21.15.100:0/718774956 conn(0x5564d4ca6bf0 msgr2=0x5564d4d6fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:32:55.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.311+0000 7f1f07212b80 1 -- 172.21.15.100:0/718774956 shutdown_connections 2024-05-10T11:32:55.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.311+0000 7f1f07212b80 1 -- 172.21.15.100:0/718774956 wait complete. 2024-05-10T11:32:55.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.311+0000 7f1f07212b80 20 client.27784 trim_cache size 0 max 16384 2024-05-10T11:32:55.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.351+0000 7f1f07212b80 1 Processor -- start 2024-05-10T11:32:55.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.351+0000 7f1f07212b80 1 -- start start 2024-05-10T11:32:55.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.351+0000 7f1f07212b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564d4d7add0 0x5564d4d7a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:55.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.351+0000 7f1f07212b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564d4d80230 0x5564d4d90600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:55.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.351+0000 7f1f07212b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564d4d80b50 0x5564d4d81640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:55.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.351+0000 7f1f07212b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef81b3b60 con 0x5564d4d80b50 2024-05-10T11:32:55.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.351+0000 7f1f07212b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef81b1360 con 0x5564d4d7add0 2024-05-10T11:32:55.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.351+0000 7f1f07212b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef81af1b0 con 0x5564d4d80230 2024-05-10T11:32:55.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.352+0000 7f1ef659c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564d4d7add0 0x5564d4d7a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:55.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.352+0000 7f1ef6d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564d4d80b50 0x5564d4d81640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:55.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.352+0000 7f1ef6d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564d4d80b50 0x5564d4d81640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48722/0 (socket says 172.21.15.100:48722) 2024-05-10T11:32:55.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.352+0000 7f1ef6d9d700 1 -- 172.21.15.100:0/1210405565 learned_addr learned my addr 172.21.15.100:0/1210405565 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:32:55.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.352+0000 7f1ef5d9b700 1 --2- 172.21.15.100:0/1210405565 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564d4d80230 0x5564d4d90600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:55.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.353+0000 7f1ef6d9d700 1 -- 172.21.15.100:0/1210405565 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564d4d80230 msgr2=0x5564d4d90600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:55.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.353+0000 7f1ef6d9d700 1 --2- 172.21.15.100:0/1210405565 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564d4d80230 0x5564d4d90600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:55.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.353+0000 7f1ef6d9d700 1 -- 172.21.15.100:0/1210405565 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564d4d7add0 msgr2=0x5564d4d7a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:55.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.353+0000 7f1ef6d9d700 1 --2- 172.21.15.100:0/1210405565 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564d4d7add0 0x5564d4d7a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:55.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.353+0000 7f1ef6d9d700 1 -- 172.21.15.100:0/1210405565 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564d4d7fc50 con 0x5564d4d80b50 2024-05-10T11:32:55.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.353+0000 7f1ef6d9d700 1 --2- 172.21.15.100:0/1210405565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564d4d80b50 0x5564d4d81640 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f1ef000b070 tx=0x7f1ef0003d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=871ffae96ca65a25 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:32:55.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.353+0000 7f1edffff700 1 -- 172.21.15.100:0/1210405565 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef0007da0 con 0x5564d4d80b50 2024-05-10T11:32:55.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.354+0000 7f1edffff700 1 -- 172.21.15.100:0/1210405565 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1ef0005c60 con 0x5564d4d80b50 2024-05-10T11:32:55.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.354+0000 7f1edffff700 1 -- 172.21.15.100:0/1210405565 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef0025930 con 0x5564d4d80b50 2024-05-10T11:32:55.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.355+0000 7f1f07212b80 1 -- 172.21.15.100:0/1210405565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564d4d80b50 msgr2=0x5564d4d81640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:55.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.355+0000 7f1f07212b80 1 --2- 172.21.15.100:0/1210405565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564d4d80b50 0x5564d4d81640 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f1ef000b070 tx=0x7f1ef0003d10 comp rx=0 tx=0).stop 2024-05-10T11:32:55.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.355+0000 7f1f07212b80 1 -- 172.21.15.100:0/1210405565 shutdown_connections 2024-05-10T11:32:55.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.355+0000 7f1f07212b80 1 --2- 172.21.15.100:0/1210405565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564d4d80b50 0x5564d4d81640 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:55.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.355+0000 7f1f07212b80 1 --2- 172.21.15.100:0/1210405565 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564d4d7add0 0x5564d4d7a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:55.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.355+0000 7f1f07212b80 1 --2- 172.21.15.100:0/1210405565 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564d4d80230 0x5564d4d90600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:55.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.355+0000 7f1f07212b80 1 -- 172.21.15.100:0/1210405565 >> 172.21.15.100:0/1210405565 conn(0x5564d4ca6bf0 msgr2=0x5564d4d80f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:32:55.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.356+0000 7f1f07212b80 1 -- 172.21.15.100:0/1210405565 shutdown_connections 2024-05-10T11:32:55.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.357+0000 7f1f07212b80 1 -- 172.21.15.100:0/1210405565 wait complete. 2024-05-10T11:32:55.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.357+0000 7f1f07212b80 1 -- start start 2024-05-10T11:32:55.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.357+0000 7f1f07212b80 1 Processor -- start 2024-05-10T11:32:55.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.358+0000 7f1f07212b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564d4d7add0 0x5564d4d9b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:55.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.359+0000 7f1f07212b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564d4d80230 0x5564d4d9bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:55.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.359+0000 7f1f07212b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564d4d80b50 0x5564d4d9c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:55.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.359+0000 7f1f07212b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef81af820 con 0x5564d4d7add0 2024-05-10T11:32:55.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.359+0000 7f1f07212b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef81b0c90 con 0x5564d4d80230 2024-05-10T11:32:55.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.359+0000 7f1f07212b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef81afef0 con 0x5564d4d80b50 2024-05-10T11:32:55.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.359+0000 7f1f07212b80 10 client.0 _ll_register_callbacks cb 0x5564d4c60630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:32:55.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.359+0000 7f1ef659c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564d4d7add0 0x5564d4d9b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:55.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.359+0000 7f1ef659c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564d4d7add0 0x5564d4d9b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48724/0 (socket says 172.21.15.100:48724) 2024-05-10T11:32:55.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.359+0000 7f1ef659c700 1 -- 172.21.15.100:0/1372776729 learned_addr learned my addr 172.21.15.100:0/1372776729 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:32:55.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.360+0000 7f1ef5d9b700 1 --2- 172.21.15.100:0/1372776729 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564d4d80230 0x5564d4d9bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:55.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.360+0000 7f1ef6d9d700 1 --2- 172.21.15.100:0/1372776729 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564d4d80b50 0x5564d4d9c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:55.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.360+0000 7f1ef659c700 1 -- 172.21.15.100:0/1372776729 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564d4d80b50 msgr2=0x5564d4d9c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:55.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.360+0000 7f1ef659c700 1 --2- 172.21.15.100:0/1372776729 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564d4d80b50 0x5564d4d9c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:55.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.360+0000 7f1ef659c700 1 -- 172.21.15.100:0/1372776729 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564d4d80230 msgr2=0x5564d4d9bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:32:55.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.360+0000 7f1ef659c700 1 --2- 172.21.15.100:0/1372776729 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564d4d80230 0x5564d4d9bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:55.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.360+0000 7f1ef659c700 1 -- 172.21.15.100:0/1372776729 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564d4dfdaf0 con 0x5564d4d7add0 2024-05-10T11:32:55.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.360+0000 7f1ef659c700 1 --2- 172.21.15.100:0/1372776729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564d4d7add0 0x5564d4d9b740 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f1ee80095c0 tx=0x7f1ee8010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8b36cfe34a78e4f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:32:55.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.361+0000 7f1eddffb700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:32:55.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.361+0000 7f1eddffb700 1 -- 172.21.15.100:0/1372776729 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee8005450 con 0x5564d4d7add0 2024-05-10T11:32:55.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.361+0000 7f1eddffb700 1 -- 172.21.15.100:0/1372776729 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f1ee80055f0 con 0x5564d4d7add0 2024-05-10T11:32:55.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.361+0000 7f1f07212b80 1 -- 172.21.15.100:0/1372776729 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ef0005c60 con 0x5564d4d7add0 2024-05-10T11:32:55.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.361+0000 7f1eddffb700 1 -- 172.21.15.100:0/1372776729 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f1ee8042050 con 0x5564d4d7add0 2024-05-10T11:32:55.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.361+0000 7f1f07212b80 10 client.18357 fetch_fsmap learned FSMap version 37 2024-05-10T11:32:55.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.361+0000 7f1f07212b80 1 -- 172.21.15.100:0/1372776729 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5564d4dfedd0 con 0x5564d4d7add0 2024-05-10T11:32:55.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.362+0000 7f1eddffb700 1 -- 172.21.15.100:0/1372776729 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1ef0005c60 con 0x5564d4d7add0 2024-05-10T11:32:55.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.362+0000 7f1eddffb700 1 -- 172.21.15.100:0/1372776729 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f1ee8034390 con 0x5564d4d7add0 2024-05-10T11:32:55.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.362+0000 7f1f07212b80 10 client.18357 fetch_fsmap finished waiting for FSMap version 37 2024-05-10T11:32:55.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.362+0000 7f1f07212b80 10 client.18357 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:32:55.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.362+0000 7f1f07212b80 20 client.18357 populate_metadata read hostname 'smithi100' 2024-05-10T11:32:55.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.362+0000 7f1f07212b80 1 --2- 172.21.15.100:0/1372776729 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5564d4e10270 0x5564d4e12660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:32:55.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.362+0000 7f1f07212b80 4 client.18357 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T11:32:55.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.362+0000 7f1f07212b80 1 -- 172.21.15.100:0/1372776729 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5564d4cd4440 con 0x5564d4e10270 2024-05-10T11:32:55.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.364+0000 7f1ef5d9b700 1 --2- 172.21.15.100:0/1372776729 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5564d4e10270 0x5564d4e12660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:32:55.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.366+0000 7f1eddffb700 10 client.18357 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:32:55.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.366+0000 7f1ef5d9b700 1 --2- 172.21.15.100:0/1372776729 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5564d4e10270 0x5564d4e12660 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b156c6bcaaec44 server_cookie=9aef149520a6de2f in_seq=0 out_seq=0 2024-05-10T11:32:55.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.368+0000 7f1eddffb700 1 -- 172.21.15.100:0/1372776729 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5564d4cd4440 con 0x5564d4e10270 2024-05-10T11:32:55.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.368+0000 7f1eddffb700 10 client.18357 handle_command_reply: tid=0 2024-05-10T11:32:55.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.368+0000 7f1f07212b80 1 client.18357 shutdown 2024-05-10T11:32:55.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.369+0000 7f1f07212b80 1 -- 172.21.15.100:0/1372776729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564d4d7add0 msgr2=0x5564d4d9b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:55.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.369+0000 7f1f07212b80 1 --2- 172.21.15.100:0/1372776729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564d4d7add0 0x5564d4d9b740 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f1ee80095c0 tx=0x7f1ee8010d20 comp rx=0 tx=0).stop 2024-05-10T11:32:55.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.369+0000 7f1f07212b80 1 -- 172.21.15.100:0/1372776729 shutdown_connections 2024-05-10T11:32:55.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.369+0000 7f1f07212b80 1 --2- 172.21.15.100:0/1372776729 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5564d4e10270 0x5564d4e12660 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:55.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.369+0000 7f1f07212b80 1 --2- 172.21.15.100:0/1372776729 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564d4d80b50 0x5564d4d9c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:55.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.369+0000 7f1f07212b80 1 --2- 172.21.15.100:0/1372776729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564d4d7add0 0x5564d4d9b740 secure :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f1ee80095c0 tx=0x7f1ee8010d20 comp rx=0 tx=0).stop 2024-05-10T11:32:55.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.369+0000 7f1f07212b80 1 --2- 172.21.15.100:0/1372776729 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564d4d80230 0x5564d4d9bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:55.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.369+0000 7f1f07212b80 1 -- 172.21.15.100:0/1372776729 >> 172.21.15.100:0/1372776729 conn(0x5564d4ca6bf0 msgr2=0x5564d4d7f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:32:55.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.369+0000 7f1eddffb700 0 client.18357 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:32:55.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.369+0000 7f1f07212b80 1 -- 172.21.15.100:0/1372776729 shutdown_connections 2024-05-10T11:32:55.371 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:32:55.371 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-05-10T11:32:55.371 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "1f15c68f-47bd-4996-892d-0ec1d929d6cf", 2024-05-10T11:32:55.371 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-05-10T11:32:55.371 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:32:55.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.370+0000 7f1f07212b80 1 -- 172.21.15.100:0/1372776729 wait complete. 2024-05-10T11:32:55.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.370+0000 7f1f07212b80 20 client.18357 trim_cache size 0 max 16384 2024-05-10T11:32:55.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.372+0000 7f1efc97f700 1 -- 172.21.15.100:0/9727099 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f1ee00b2900 msgr2=0x7f1ee00b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:55.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.372+0000 7f1efc97f700 1 --2- 172.21.15.100:0/9727099 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f1ee00b2900 0x7f1ee00b4dc0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f1ee8009c90 tx=0x7f1ee8009260 comp rx=0 tx=0).stop 2024-05-10T11:32:55.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.372+0000 7f1efc97f700 1 -- 172.21.15.100:0/9727099 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ef8102080 msgr2=0x7f1ef80fdf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:32:55.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.372+0000 7f1efc97f700 1 --2- 172.21.15.100:0/9727099 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ef8102080 0x7f1ef80fdf50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1eec009800 tx=0x7f1eec009b10 comp rx=0 tx=0).stop 2024-05-10T11:32:55.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.372+0000 7f1efc97f700 1 -- 172.21.15.100:0/9727099 shutdown_connections 2024-05-10T11:32:55.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.372+0000 7f1efc97f700 1 --2- 172.21.15.100:0/9727099 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f1ee00b2900 0x7f1ee00b4dc0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:55.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.372+0000 7f1efc97f700 1 --2- 172.21.15.100:0/9727099 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ef81076c0 0x7f1ef80fe490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:55.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.372+0000 7f1efc97f700 1 --2- 172.21.15.100:0/9727099 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ef81016c0 0x7f1ef80fda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:55.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.372+0000 7f1efc97f700 1 --2- 172.21.15.100:0/9727099 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ef8102080 0x7f1ef80fdf50 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:32:55.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.372+0000 7f1efc97f700 1 -- 172.21.15.100:0/9727099 >> 172.21.15.100:0/9727099 conn(0x7f1ef80785a0 msgr2=0x7f1ef80fffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:32:55.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.372+0000 7f1efc97f700 1 -- 172.21.15.100:0/9727099 shutdown_connections 2024-05-10T11:32:55.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:32:55.372+0000 7f1efc97f700 1 -- 172.21.15.100:0/9727099 wait complete. 2024-05-10T11:32:55.778 INFO:journalctl@ceph.mds.i.smithi188.stdout:May 10 11:32:55 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-i[64554]: 2024-05-10T11:32:55.493+0000 7fd152433700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000004a2 [...2,head] /volumes/qa/sv_0/96765652-39ef-4cf3-b20a-a70f7e9d774d/client.0/tmp/clients/client7/~dmtmp/PWRPNT/ auth{2=1} v1551 ap=2 f(v18 m2024-05-10T11:32:36.827789+0000 7=7+0) n(v1 rc2024-05-10T11:32:36.827789+0000 b1462582 8=7+1) (inest sync->mix g=2) caps={15207=pAsLsXsFs/-@1044} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x5608017a8100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000004a2:[<0x10000000420/PWRPNT v220>,<0x10000000412/~dmtmp v34>,<0x10000000007/client7 v1940>,<0x10000000006/clients v1743>,<0x10000000005/tmp v1642>,<0x10000000003/client.0 v1616>,<0x10000000002/96765652-39ef-4cf3-b20a-a70f7e9d774d v1609>,<0x10000000001/sv_0 v1620>,<0x10000000000/qa v1598>,<0x1/volumes v1566>]//[]","memoryvalue":"(2)0x100000004a2:[<0x10000000420/PWRPNT v1551>,<0x10000000412/~dmtmp v486>,<0x10000000007/client7 v2564>,<0x10000000006/clients v1934>,<0x10000000005/tmp v1733>,<0x10000000003/client.0 v1703>,<0x10000000002/96765652-39ef-4cf3-b20a-a70f7e9d774d v1696>,<0x10000000001/sv_0 v1708>,<0x10000000000/qa v1683>,<0x1/volumes v1652>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-10T11:32:36.733033+0000 8=8+0)","ondisk_value.rstat":"n(v0 rc2024-05-10T11:32:36.827789+0000 b1462582 8=7+1)","memory_value.dirstat":"f(v18 m2024-05-10T11:32:36.827789+0000 7=7+0)","memory_value.rstat":"n(v1 rc2024-05-10T11:32:36.827789+0000 b1462582 8=7+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-05-10T11:32:55.908 INFO:tasks.workunit.client.0.smithi100.stdout: 10 8388 12.72 MB/sec warmup 49 sec latency 81.359 ms 2024-05-10T11:32:56.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:32:55 smithi178 ceph-mon[33359]: pgmap v857: 129 pgs: 129 active+clean; 352 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 4.9 MiB/s wr, 253 op/s 2024-05-10T11:32:56.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:32:55 smithi178 ceph-mon[33359]: scrub queued for path: / 2024-05-10T11:32:56.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:32:55 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:32:56.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:32:55 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:32:56.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:32:55 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:32:56.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:32:55 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:32:56.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:32:55 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:32:56.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:32:55 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:32:56.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:32:55 smithi178 ceph-mon[33359]: Scrub error on inode 0x100000004a2 (/volumes/qa/sv_0/96765652-39ef-4cf3-b20a-a70f7e9d774d/client.0/tmp/clients/client7/~dmtmp/PWRPNT) see mds.i log and `damage ls` output for details 2024-05-10T11:32:56.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:32:55 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:32:56.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:32:55 smithi188 ceph-mon[34075]: pgmap v857: 129 pgs: 129 active+clean; 352 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 4.9 MiB/s wr, 253 op/s 2024-05-10T11:32:56.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:32:55 smithi188 ceph-mon[34075]: scrub queued for path: / 2024-05-10T11:32:56.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:32:55 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:32:56.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:32:55 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:32:56.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:32:55 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:32:56.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:32:55 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:32:56.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:32:55 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:32:56.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:32:55 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:32:56.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:32:55 smithi188 ceph-mon[34075]: Scrub error on inode 0x100000004a2 (/volumes/qa/sv_0/96765652-39ef-4cf3-b20a-a70f7e9d774d/client.0/tmp/clients/client7/~dmtmp/PWRPNT) see mds.i log and `damage ls` output for details 2024-05-10T11:32:56.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:32:55 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:32:56.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:32:55 smithi100 ceph-mon[30296]: pgmap v857: 129 pgs: 129 active+clean; 352 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 4.9 MiB/s wr, 253 op/s 2024-05-10T11:32:56.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:32:55 smithi100 ceph-mon[30296]: scrub queued for path: / 2024-05-10T11:32:56.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:32:55 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:32:56.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:32:55 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:32:56.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:32:55 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:32:56.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:32:55 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:32:56.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:32:55 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:32:56.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:32:55 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:32:56.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:32:55 smithi100 ceph-mon[30296]: Scrub error on inode 0x100000004a2 (/volumes/qa/sv_0/96765652-39ef-4cf3-b20a-a70f7e9d774d/client.0/tmp/clients/client7/~dmtmp/PWRPNT) see mds.i log and `damage ls` output for details 2024-05-10T11:32:56.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:32:55 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:32:56.908 INFO:tasks.workunit.client.0.smithi100.stdout: 10 8536 12.55 MB/sec warmup 50 sec latency 115.528 ms 2024-05-10T11:32:56.958 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T11:32:57.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:32:56 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:32:57.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:32:56 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:32:57.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:32:56 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:32:57.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:32:56 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:32:57.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:32:56 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:32:57.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:32:56 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:32:57.306 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:32:57.908 INFO:tasks.workunit.client.0.smithi100.stdout: 10 8740 12.59 MB/sec warmup 51 sec latency 93.471 ms 2024-05-10T11:32:58.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:32:57 smithi100 ceph-mon[30296]: pgmap v858: 129 pgs: 129 active+clean; 353 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 916 KiB/s rd, 5.2 MiB/s wr, 306 op/s 2024-05-10T11:32:58.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:32:57 smithi178 ceph-mon[33359]: pgmap v858: 129 pgs: 129 active+clean; 353 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 916 KiB/s rd, 5.2 MiB/s wr, 306 op/s 2024-05-10T11:32:58.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:32:57 smithi188 ceph-mon[34075]: pgmap v858: 129 pgs: 129 active+clean; 353 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 916 KiB/s rd, 5.2 MiB/s wr, 306 op/s 2024-05-10T11:32:58.908 INFO:tasks.workunit.client.0.smithi100.stdout: 10 8923 12.49 MB/sec warmup 52 sec latency 112.778 ms 2024-05-10T11:32:59.908 INFO:tasks.workunit.client.0.smithi100.stdout: 10 9080 12.41 MB/sec warmup 53 sec latency 67.368 ms 2024-05-10T11:33:00.121 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:32:59 smithi100 ceph-mon[30296]: pgmap v859: 129 pgs: 129 active+clean; 355 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.1 MiB/s wr, 360 op/s 2024-05-10T11:33:00.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.167+0000 7f7958328700 1 -- 172.21.15.100:0/1275856367 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79501056b0 msgr2=0x7f7950105b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:00.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.167+0000 7f7958328700 1 --2- 172.21.15.100:0/1275856367 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79501056b0 0x7f7950105b30 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f7940009990 tx=0x7f7940009ca0 comp rx=0 tx=0).stop 2024-05-10T11:33:00.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.167+0000 7f7958328700 1 -- 172.21.15.100:0/1275856367 shutdown_connections 2024-05-10T11:33:00.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.167+0000 7f7958328700 1 --2- 172.21.15.100:0/1275856367 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7950106070 0x7f7950112580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:00.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.167+0000 7f7958328700 1 --2- 172.21.15.100:0/1275856367 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f795010b6b0 0x7f795010ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:00.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.167+0000 7f7958328700 1 --2- 172.21.15.100:0/1275856367 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79501056b0 0x7f7950105b30 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:00.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.167+0000 7f7958328700 1 -- 172.21.15.100:0/1275856367 >> 172.21.15.100:0/1275856367 conn(0x7f7950101180 msgr2=0x7f79501035a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:33:00.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.167+0000 7f7958328700 1 -- 172.21.15.100:0/1275856367 shutdown_connections 2024-05-10T11:33:00.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.167+0000 7f7958328700 1 -- 172.21.15.100:0/1275856367 wait complete. 2024-05-10T11:33:00.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.168+0000 7f7958328700 1 Processor -- start 2024-05-10T11:33:00.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.168+0000 7f7958328700 1 -- start start 2024-05-10T11:33:00.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.168+0000 7f7958328700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f79501056b0 0x7f79500779b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:00.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.168+0000 7f7958328700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7950106070 0x7f7950077ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:00.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.168+0000 7f7958328700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f795010b6b0 0x7f7950078430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:00.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.168+0000 7f79558c3700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7950106070 0x7f7950077ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:00.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.169+0000 7f79558c3700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7950106070 0x7f7950077ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:51382/0 (socket says 172.21.15.100:51382) 2024-05-10T11:33:00.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.169+0000 7f79558c3700 1 -- 172.21.15.100:0/3758974522 learned_addr learned my addr 172.21.15.100:0/3758974522 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:33:00.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.169+0000 7f79568c5700 1 --2- 172.21.15.100:0/3758974522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f795010b6b0 0x7f7950078430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:00.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.169+0000 7f7958328700 1 -- 172.21.15.100:0/3758974522 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f79501157b0 con 0x7f795010b6b0 2024-05-10T11:33:00.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.169+0000 7f7958328700 1 -- 172.21.15.100:0/3758974522 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7950115630 con 0x7f79501056b0 2024-05-10T11:33:00.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.169+0000 7f7958328700 1 -- 172.21.15.100:0/3758974522 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7950115930 con 0x7f7950106070 2024-05-10T11:33:00.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.169+0000 7f79560c4700 1 --2- 172.21.15.100:0/3758974522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f79501056b0 0x7f79500779b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:00.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.169+0000 7f79568c5700 1 -- 172.21.15.100:0/3758974522 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7950106070 msgr2=0x7f7950077ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:00.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.169+0000 7f79568c5700 1 --2- 172.21.15.100:0/3758974522 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7950106070 0x7f7950077ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:00.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.169+0000 7f79568c5700 1 -- 172.21.15.100:0/3758974522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f79501056b0 msgr2=0x7f79500779b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:00.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.169+0000 7f79568c5700 1 --2- 172.21.15.100:0/3758974522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f79501056b0 0x7f79500779b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:00.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.169+0000 7f79568c5700 1 -- 172.21.15.100:0/3758974522 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79501ac3f0 con 0x7f795010b6b0 2024-05-10T11:33:00.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.169+0000 7f79558c3700 1 --2- 172.21.15.100:0/3758974522 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7950106070 0x7f7950077ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:33:00.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.170+0000 7f79568c5700 1 --2- 172.21.15.100:0/3758974522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f795010b6b0 0x7f7950078430 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f794800efb0 tx=0x7f794800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:33:00.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.170+0000 7f79477fe700 1 -- 172.21.15.100:0/3758974522 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f794800ca50 con 0x7f795010b6b0 2024-05-10T11:33:00.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.170+0000 7f79477fe700 1 -- 172.21.15.100:0/3758974522 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f794800cbf0 con 0x7f795010b6b0 2024-05-10T11:33:00.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.170+0000 7f79477fe700 1 -- 172.21.15.100:0/3758974522 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79480046f0 con 0x7f795010b6b0 2024-05-10T11:33:00.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.170+0000 7f7958328700 1 -- 172.21.15.100:0/3758974522 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79501ac6e0 con 0x7f795010b6b0 2024-05-10T11:33:00.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.170+0000 7f7958328700 1 -- 172.21.15.100:0/3758974522 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79501acbd0 con 0x7f795010b6b0 2024-05-10T11:33:00.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.174+0000 7f79477fe700 1 -- 172.21.15.100:0/3758974522 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f7948012050 con 0x7f795010b6b0 2024-05-10T11:33:00.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.175+0000 7f79477fe700 1 --2- 172.21.15.100:0/3758974522 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f793c0b2900 0x7f793c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:00.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.175+0000 7f7962bbbb80 1 Processor -- start 2024-05-10T11:33:00.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.175+0000 7f79477fe700 1 -- 172.21.15.100:0/3758974522 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f79480e1e10 con 0x7f795010b6b0 2024-05-10T11:33:00.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.176+0000 7f79560c4700 1 --2- 172.21.15.100:0/3758974522 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f793c0b2900 0x7f793c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:00.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.176+0000 7f7962bbbb80 1 -- start start 2024-05-10T11:33:00.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.176+0000 7f7962bbbb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f7f15201b0 0x55f7f1520590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:00.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.176+0000 7f79560c4700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f7f15201b0 0x55f7f1520590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:00.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.176+0000 7f7962bbbb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7f1520ad0 0x55f7f151a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:00.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.176+0000 7f7962bbbb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f7f151ad50 0x55f7f151b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:00.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.176+0000 7f79560c4700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f7f15201b0 0x55f7f1520590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:51398/0 (socket says 172.21.15.100:51398) 2024-05-10T11:33:00.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.176+0000 7f79560c4700 1 -- 172.21.15.100:0/342801314 learned_addr learned my addr 172.21.15.100:0/342801314 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:33:00.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.176+0000 7f7962bbbb80 1 -- 172.21.15.100:0/342801314 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55f7f146c940 con 0x55f7f1520ad0 2024-05-10T11:33:00.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.176+0000 7f7962bbbb80 1 -- 172.21.15.100:0/342801314 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f7f13d1da0 con 0x55f7f151ad50 2024-05-10T11:33:00.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.176+0000 7f7962bbbb80 1 -- 172.21.15.100:0/342801314 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55f7f1468900 con 0x55f7f15201b0 2024-05-10T11:33:00.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.176+0000 7f79560c4700 1 --2- 172.21.15.100:0/3758974522 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f793c0b2900 0x7f793c0b4dc0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f794c009b60 tx=0x7f794c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:33:00.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.177+0000 7f79558c3700 1 --2- 172.21.15.100:0/342801314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7f1520ad0 0x55f7f151a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:00.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.177+0000 7f79568c5700 1 --2- 172.21.15.100:0/342801314 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f7f151ad50 0x55f7f151b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:00.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.177+0000 7f79560c4700 1 -- 172.21.15.100:0/342801314 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f7f151ad50 msgr2=0x55f7f151b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:00.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.177+0000 7f79560c4700 1 --2- 172.21.15.100:0/342801314 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f7f151ad50 0x55f7f151b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:00.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.177+0000 7f79560c4700 1 -- 172.21.15.100:0/342801314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7f1520ad0 msgr2=0x55f7f151a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:00.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.177+0000 7f79560c4700 1 --2- 172.21.15.100:0/342801314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7f1520ad0 0x55f7f151a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:00.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.177+0000 7f79560c4700 1 -- 172.21.15.100:0/342801314 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7f13b98d0 con 0x55f7f15201b0 2024-05-10T11:33:00.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.177+0000 7f79560c4700 1 --2- 172.21.15.100:0/342801314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f7f15201b0 0x55f7f1520590 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f794c00fc10 tx=0x7f794c012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e863d67643a06f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:33:00.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.177+0000 7f7944d70700 1 -- 172.21.15.100:0/342801314 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f794c022070 con 0x55f7f15201b0 2024-05-10T11:33:00.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.177+0000 7f7944d70700 1 -- 172.21.15.100:0/342801314 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f794c004830 con 0x55f7f15201b0 2024-05-10T11:33:00.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.177+0000 7f7944d70700 1 -- 172.21.15.100:0/342801314 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f794c005310 con 0x55f7f15201b0 2024-05-10T11:33:00.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.178+0000 7f7962bbbb80 1 -- 172.21.15.100:0/342801314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f7f15201b0 msgr2=0x55f7f1520590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:00.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.178+0000 7f7962bbbb80 1 --2- 172.21.15.100:0/342801314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f7f15201b0 0x55f7f1520590 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f794c00fc10 tx=0x7f794c012000 comp rx=0 tx=0).stop 2024-05-10T11:33:00.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.178+0000 7f7962bbbb80 1 -- 172.21.15.100:0/342801314 shutdown_connections 2024-05-10T11:33:00.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.178+0000 7f7962bbbb80 1 --2- 172.21.15.100:0/342801314 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f7f151ad50 0x55f7f151b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:00.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.178+0000 7f7962bbbb80 1 --2- 172.21.15.100:0/342801314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f7f15201b0 0x55f7f1520590 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:00.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.178+0000 7f7962bbbb80 1 --2- 172.21.15.100:0/342801314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7f1520ad0 0x55f7f151a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:00.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.178+0000 7f7962bbbb80 1 -- 172.21.15.100:0/342801314 >> 172.21.15.100:0/342801314 conn(0x55f7f1446b70 msgr2=0x55f7f15152a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:33:00.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.179+0000 7f7962bbbb80 1 -- 172.21.15.100:0/342801314 shutdown_connections 2024-05-10T11:33:00.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.179+0000 7f7962bbbb80 1 -- 172.21.15.100:0/342801314 wait complete. 2024-05-10T11:33:00.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.179+0000 7f7962bbbb80 1 -- start start 2024-05-10T11:33:00.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.179+0000 7f7962bbbb80 1 Processor -- start 2024-05-10T11:33:00.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.179+0000 7f7962bbbb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7f151ad50 0x55f7f1526770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:00.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.180+0000 7f7962bbbb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f7f15201b0 0x55f7f1523b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:00.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.180+0000 7f7962bbbb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f7f1520ad0 0x55f7f1524050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:00.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.180+0000 7f7962bbbb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55f7f1527450 con 0x55f7f151ad50 2024-05-10T11:33:00.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.180+0000 7f7962bbbb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f7f1425ed0 con 0x55f7f15201b0 2024-05-10T11:33:00.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.180+0000 7f7962bbbb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55f7f15275d0 con 0x55f7f1520ad0 2024-05-10T11:33:00.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.180+0000 7f7962bbbb80 10 client.0 _ll_register_callbacks cb 0x55f7f14005b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:33:00.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.180+0000 7f79560c4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7f151ad50 0x55f7f1526770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:00.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.180+0000 7f79560c4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7f151ad50 0x55f7f1526770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56518/0 (socket says 172.21.15.100:56518) 2024-05-10T11:33:00.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.180+0000 7f79560c4700 1 -- 172.21.15.100:0/52857089 learned_addr learned my addr 172.21.15.100:0/52857089 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:33:00.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.180+0000 7f79558c3700 1 --2- 172.21.15.100:0/52857089 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f7f15201b0 0x55f7f1523b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:00.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.180+0000 7f79568c5700 1 --2- 172.21.15.100:0/52857089 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f7f1520ad0 0x55f7f1524050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:00.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.180+0000 7f79560c4700 1 -- 172.21.15.100:0/52857089 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f7f1520ad0 msgr2=0x55f7f1524050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:00.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.180+0000 7f79560c4700 1 --2- 172.21.15.100:0/52857089 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f7f1520ad0 0x55f7f1524050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:00.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.180+0000 7f79560c4700 1 -- 172.21.15.100:0/52857089 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f7f15201b0 msgr2=0x55f7f1523b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:00.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.180+0000 7f79560c4700 1 --2- 172.21.15.100:0/52857089 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f7f15201b0 0x55f7f1523b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:00.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.180+0000 7f79560c4700 1 -- 172.21.15.100:0/52857089 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7f1423760 con 0x55f7f151ad50 2024-05-10T11:33:00.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.180+0000 7f79560c4700 1 --2- 172.21.15.100:0/52857089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7f151ad50 0x55f7f1526770 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f794c009fd0 tx=0x7f794c011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=893dc3b9c704d870 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:33:00.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.180+0000 7f7939ffb700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:33:00.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.181+0000 7f7939ffb700 1 -- 172.21.15.100:0/52857089 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f794c022040 con 0x55f7f151ad50 2024-05-10T11:33:00.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.181+0000 7f7962bbbb80 1 -- 172.21.15.100:0/52857089 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f794c004830 con 0x55f7f151ad50 2024-05-10T11:33:00.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.181+0000 7f7939ffb700 1 -- 172.21.15.100:0/52857089 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f794c01e450 con 0x55f7f151ad50 2024-05-10T11:33:00.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.181+0000 7f7939ffb700 1 -- 172.21.15.100:0/52857089 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f794c03b050 con 0x55f7f151ad50 2024-05-10T11:33:00.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.181+0000 7f7962bbbb80 10 client.18369 fetch_fsmap learned FSMap version 37 2024-05-10T11:33:00.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.181+0000 7f7962bbbb80 1 -- 172.21.15.100:0/52857089 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55f7f1531d30 con 0x55f7f151ad50 2024-05-10T11:33:00.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.182+0000 7f7939ffb700 1 -- 172.21.15.100:0/52857089 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f794c004830 con 0x55f7f151ad50 2024-05-10T11:33:00.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.182+0000 7f7939ffb700 1 -- 172.21.15.100:0/52857089 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f794c032620 con 0x55f7f151ad50 2024-05-10T11:33:00.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.182+0000 7f7962bbbb80 10 client.18369 fetch_fsmap finished waiting for FSMap version 37 2024-05-10T11:33:00.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.182+0000 7f7962bbbb80 10 client.18369 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:33:00.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.182+0000 7f7962bbbb80 20 client.18369 populate_metadata read hostname 'smithi100' 2024-05-10T11:33:00.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.182+0000 7f7962bbbb80 1 --2- 172.21.15.100:0/52857089 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55f7f1534260 0x55f7f1536650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:00.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.182+0000 7f7962bbbb80 4 client.18369 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:33:00.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.182+0000 7f7962bbbb80 1 -- 172.21.15.100:0/52857089 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f7f147a320 con 0x55f7f1534260 2024-05-10T11:33:00.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.182+0000 7f79558c3700 1 --2- 172.21.15.100:0/52857089 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55f7f1534260 0x55f7f1536650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:00.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.183+0000 7f79558c3700 1 --2- 172.21.15.100:0/52857089 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55f7f1534260 0x55f7f1536650 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66ae11afc15f9ccc server_cookie=84efe427af1a7dd2 in_seq=0 out_seq=0 2024-05-10T11:33:00.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.183+0000 7f7939ffb700 10 client.18369 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:33:00.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.183+0000 7f7939ffb700 1 -- 172.21.15.100:0/52857089 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f7f147a320 con 0x55f7f1534260 2024-05-10T11:33:00.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.183+0000 7f7939ffb700 10 client.18369 handle_command_reply: tid=0 2024-05-10T11:33:00.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.183+0000 7f7962bbbb80 1 client.18369 shutdown 2024-05-10T11:33:00.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.183+0000 7f7962bbbb80 1 -- 172.21.15.100:0/52857089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7f151ad50 msgr2=0x55f7f1526770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:00.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.183+0000 7f7962bbbb80 1 --2- 172.21.15.100:0/52857089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7f151ad50 0x55f7f1526770 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f794c009fd0 tx=0x7f794c011b00 comp rx=0 tx=0).stop 2024-05-10T11:33:00.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.184+0000 7f7962bbbb80 1 -- 172.21.15.100:0/52857089 shutdown_connections 2024-05-10T11:33:00.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.184+0000 7f7962bbbb80 1 --2- 172.21.15.100:0/52857089 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55f7f1534260 0x55f7f1536650 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:00.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.184+0000 7f7962bbbb80 1 --2- 172.21.15.100:0/52857089 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f7f1520ad0 0x55f7f1524050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:00.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.184+0000 7f7962bbbb80 1 --2- 172.21.15.100:0/52857089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7f151ad50 0x55f7f1526770 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:00.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.184+0000 7f7962bbbb80 1 --2- 172.21.15.100:0/52857089 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f7f15201b0 0x55f7f1523b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:00.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.184+0000 7f7962bbbb80 1 -- 172.21.15.100:0/52857089 >> 172.21.15.100:0/52857089 conn(0x55f7f1446b70 msgr2=0x55f7f150fb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:33:00.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.184+0000 7f7939ffb700 0 client.18369 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:33:00.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.184+0000 7f7962bbbb80 1 -- 172.21.15.100:0/52857089 shutdown_connections 2024-05-10T11:33:00.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.184+0000 7f7962bbbb80 1 -- 172.21.15.100:0/52857089 wait complete. 2024-05-10T11:33:00.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.184+0000 7f7962bbbb80 20 client.18369 trim_cache size 0 max 16384 2024-05-10T11:33:00.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.216+0000 7f7962bbbb80 1 Processor -- start 2024-05-10T11:33:00.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.216+0000 7f7962bbbb80 1 -- start start 2024-05-10T11:33:00.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.216+0000 7f7962bbbb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f7f151ad50 0x55f7f151e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:00.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.216+0000 7f7962bbbb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7f15201b0 0x55f7f15240a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:00.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.216+0000 7f7962bbbb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f7f1520ad0 0x55f7f15304e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:00.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.216+0000 7f7962bbbb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7934011310 con 0x55f7f15201b0 2024-05-10T11:33:00.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.216+0000 7f7962bbbb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f793400eb10 con 0x55f7f1520ad0 2024-05-10T11:33:00.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.216+0000 7f7962bbbb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f793400c960 con 0x55f7f151ad50 2024-05-10T11:33:00.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.216+0000 7f79558c3700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7f15201b0 0x55f7f15240a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:00.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.216+0000 7f79558c3700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7f15201b0 0x55f7f15240a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56522/0 (socket says 172.21.15.100:56522) 2024-05-10T11:33:00.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.216+0000 7f79558c3700 1 -- 172.21.15.100:0/543154246 learned_addr learned my addr 172.21.15.100:0/543154246 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:33:00.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.216+0000 7f79560c4700 1 --2- 172.21.15.100:0/543154246 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f7f151ad50 0x55f7f151e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:00.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.217+0000 7f79568c5700 1 --2- 172.21.15.100:0/543154246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f7f1520ad0 0x55f7f15304e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:00.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.217+0000 7f79558c3700 1 -- 172.21.15.100:0/543154246 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f7f151ad50 msgr2=0x55f7f151e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:00.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.217+0000 7f79558c3700 1 --2- 172.21.15.100:0/543154246 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f7f151ad50 0x55f7f151e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:00.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.217+0000 7f79558c3700 1 -- 172.21.15.100:0/543154246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f7f1520ad0 msgr2=0x55f7f15304e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:00.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.217+0000 7f79558c3700 1 --2- 172.21.15.100:0/543154246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f7f1520ad0 0x55f7f15304e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:00.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.217+0000 7f79558c3700 1 -- 172.21.15.100:0/543154246 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7f1520900 con 0x55f7f15201b0 2024-05-10T11:33:00.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.217+0000 7f79558c3700 1 --2- 172.21.15.100:0/543154246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7f15201b0 0x55f7f15240a0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f794000b650 tx=0x7f794000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cea381716683e3c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:33:00.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.217+0000 7f793bfff700 1 -- 172.21.15.100:0/543154246 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f794002e030 con 0x55f7f15201b0 2024-05-10T11:33:00.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.217+0000 7f793bfff700 1 -- 172.21.15.100:0/543154246 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f79400055c0 con 0x55f7f15201b0 2024-05-10T11:33:00.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.217+0000 7f793bfff700 1 -- 172.21.15.100:0/543154246 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79400291d0 con 0x55f7f15201b0 2024-05-10T11:33:00.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.218+0000 7f7962bbbb80 1 -- 172.21.15.100:0/543154246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7f15201b0 msgr2=0x55f7f15240a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:00.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.218+0000 7f7962bbbb80 1 --2- 172.21.15.100:0/543154246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7f15201b0 0x55f7f15240a0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f794000b650 tx=0x7f794000b260 comp rx=0 tx=0).stop 2024-05-10T11:33:00.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.218+0000 7f7962bbbb80 1 -- 172.21.15.100:0/543154246 shutdown_connections 2024-05-10T11:33:00.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.218+0000 7f7962bbbb80 1 --2- 172.21.15.100:0/543154246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f7f1520ad0 0x55f7f15304e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:00.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.218+0000 7f7962bbbb80 1 --2- 172.21.15.100:0/543154246 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f7f151ad50 0x55f7f151e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:00.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.218+0000 7f7962bbbb80 1 --2- 172.21.15.100:0/543154246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7f15201b0 0x55f7f15240a0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:00.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.218+0000 7f7962bbbb80 1 -- 172.21.15.100:0/543154246 >> 172.21.15.100:0/543154246 conn(0x55f7f1446b70 msgr2=0x55f7f1520eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:33:00.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.218+0000 7f7962bbbb80 1 -- 172.21.15.100:0/543154246 shutdown_connections 2024-05-10T11:33:00.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.218+0000 7f7962bbbb80 1 -- 172.21.15.100:0/543154246 wait complete. 2024-05-10T11:33:00.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.219+0000 7f7962bbbb80 1 -- start start 2024-05-10T11:33:00.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.219+0000 7f7962bbbb80 1 Processor -- start 2024-05-10T11:33:00.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.219+0000 7f7962bbbb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7f151ad50 0x55f7f152c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:00.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.219+0000 7f7962bbbb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f7f15201b0 0x55f7f152cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:00.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.219+0000 7f7962bbbb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f7f1520ad0 0x55f7f153acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:00.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.219+0000 7f7962bbbb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f793400cfd0 con 0x55f7f151ad50 2024-05-10T11:33:00.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.219+0000 7f7962bbbb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f793400e440 con 0x55f7f1520ad0 2024-05-10T11:33:00.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.219+0000 7f7962bbbb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f793400d6a0 con 0x55f7f15201b0 2024-05-10T11:33:00.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.219+0000 7f7962bbbb80 10 client.0 _ll_register_callbacks cb 0x55f7f14005b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:33:00.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.219+0000 7f79560c4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7f151ad50 0x55f7f152c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:00.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.219+0000 7f79560c4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7f151ad50 0x55f7f152c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56534/0 (socket says 172.21.15.100:56534) 2024-05-10T11:33:00.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.219+0000 7f79560c4700 1 -- 172.21.15.100:0/3008064319 learned_addr learned my addr 172.21.15.100:0/3008064319 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:33:00.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.219+0000 7f79558c3700 1 --2- 172.21.15.100:0/3008064319 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f7f15201b0 0x55f7f152cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:00.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.220+0000 7f79568c5700 1 --2- 172.21.15.100:0/3008064319 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f7f1520ad0 0x55f7f153acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:00.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.220+0000 7f79558c3700 1 -- 172.21.15.100:0/3008064319 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f7f1520ad0 msgr2=0x55f7f153acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:00.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.220+0000 7f79558c3700 1 --2- 172.21.15.100:0/3008064319 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f7f1520ad0 0x55f7f153acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:00.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.220+0000 7f79558c3700 1 -- 172.21.15.100:0/3008064319 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7f151ad50 msgr2=0x55f7f152c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:00.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.220+0000 7f79558c3700 1 --2- 172.21.15.100:0/3008064319 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7f151ad50 0x55f7f152c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:00.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.220+0000 7f79558c3700 1 -- 172.21.15.100:0/3008064319 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7f153b300 con 0x55f7f15201b0 2024-05-10T11:33:00.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.220+0000 7f79568c5700 1 --2- 172.21.15.100:0/3008064319 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f7f1520ad0 0x55f7f153acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:33:00.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.220+0000 7f79558c3700 1 --2- 172.21.15.100:0/3008064319 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f7f15201b0 0x55f7f152cda0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f79400098e0 tx=0x7f7940003c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1801e2c0e794a911 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:33:00.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.220+0000 7f7939ffb700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:33:00.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.220+0000 7f7939ffb700 1 -- 172.21.15.100:0/3008064319 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7940003680 con 0x55f7f15201b0 2024-05-10T11:33:00.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.220+0000 7f7939ffb700 1 -- 172.21.15.100:0/3008064319 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f7940003820 con 0x55f7f15201b0 2024-05-10T11:33:00.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.220+0000 7f7962bbbb80 1 -- 172.21.15.100:0/3008064319 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79400055c0 con 0x55f7f15201b0 2024-05-10T11:33:00.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.220+0000 7f7939ffb700 1 -- 172.21.15.100:0/3008064319 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f7940040050 con 0x55f7f15201b0 2024-05-10T11:33:00.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.220+0000 7f7962bbbb80 10 client.27808 fetch_fsmap learned FSMap version 37 2024-05-10T11:33:00.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.220+0000 7f7962bbbb80 1 -- 172.21.15.100:0/3008064319 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55f7f159e1a0 con 0x55f7f15201b0 2024-05-10T11:33:00.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.221+0000 7f7939ffb700 1 -- 172.21.15.100:0/3008064319 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7940029660 con 0x55f7f15201b0 2024-05-10T11:33:00.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.221+0000 7f7939ffb700 1 -- 172.21.15.100:0/3008064319 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f794000f580 con 0x55f7f15201b0 2024-05-10T11:33:00.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.221+0000 7f7962bbbb80 10 client.27808 fetch_fsmap finished waiting for FSMap version 37 2024-05-10T11:33:00.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.221+0000 7f7962bbbb80 10 client.27808 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:33:00.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.221+0000 7f7962bbbb80 20 client.27808 populate_metadata read hostname 'smithi100' 2024-05-10T11:33:00.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.221+0000 7f7962bbbb80 1 --2- 172.21.15.100:0/3008064319 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55f7f159e7d0 0x55f7f15a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:00.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.221+0000 7f7962bbbb80 4 client.27808 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T11:33:00.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.221+0000 7f7962bbbb80 1 -- 172.21.15.100:0/3008064319 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f79400055c0 con 0x55f7f159e7d0 2024-05-10T11:33:00.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.221+0000 7f79560c4700 1 --2- 172.21.15.100:0/3008064319 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55f7f159e7d0 0x55f7f15a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:00.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.222+0000 7f7939ffb700 10 client.27808 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:33:00.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.222+0000 7f79560c4700 1 --2- 172.21.15.100:0/3008064319 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55f7f159e7d0 0x55f7f15a0bf0 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b53e4632c1ef2ad8 server_cookie=9cfaae73a7be3619 in_seq=0 out_seq=0 2024-05-10T11:33:00.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.222+0000 7f7939ffb700 1 -- 172.21.15.100:0/3008064319 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f79400055c0 con 0x55f7f159e7d0 2024-05-10T11:33:00.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.222+0000 7f7939ffb700 10 client.27808 handle_command_reply: tid=0 2024-05-10T11:33:00.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.222+0000 7f7962bbbb80 1 client.27808 shutdown 2024-05-10T11:33:00.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.222+0000 7f7962bbbb80 1 -- 172.21.15.100:0/3008064319 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f7f15201b0 msgr2=0x55f7f152cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:00.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.222+0000 7f7962bbbb80 1 --2- 172.21.15.100:0/3008064319 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f7f15201b0 0x55f7f152cda0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f79400098e0 tx=0x7f7940003c80 comp rx=0 tx=0).stop 2024-05-10T11:33:00.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.222+0000 7f7962bbbb80 1 -- 172.21.15.100:0/3008064319 shutdown_connections 2024-05-10T11:33:00.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.222+0000 7f7962bbbb80 1 --2- 172.21.15.100:0/3008064319 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55f7f159e7d0 0x55f7f15a0bf0 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:00.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.222+0000 7f7939ffb700 0 client.27808 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:33:00.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.222+0000 7f7962bbbb80 1 --2- 172.21.15.100:0/3008064319 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f7f1520ad0 0x55f7f153acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:00.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.222+0000 7f7962bbbb80 1 --2- 172.21.15.100:0/3008064319 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7f151ad50 0x55f7f152c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:00.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.222+0000 7f7962bbbb80 1 --2- 172.21.15.100:0/3008064319 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f7f15201b0 0x55f7f152cda0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:00.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.222+0000 7f7962bbbb80 1 -- 172.21.15.100:0/3008064319 >> 172.21.15.100:0/3008064319 conn(0x55f7f1446b70 msgr2=0x55f7f151f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:33:00.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.223+0000 7f7962bbbb80 1 -- 172.21.15.100:0/3008064319 shutdown_connections 2024-05-10T11:33:00.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.223+0000 7f7962bbbb80 1 -- 172.21.15.100:0/3008064319 wait complete. 2024-05-10T11:33:00.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.223+0000 7f7962bbbb80 20 client.27808 trim_cache size 0 max 16384 2024-05-10T11:33:00.229 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:33:00.229 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T11:33:00.229 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-05-10T11:33:00.229 INFO:teuthology.orchestra.run.smithi100.stdout: "1f15c68f-47bd-4996-892d-0ec1d929d6cf": { 2024-05-10T11:33:00.229 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-05-10T11:33:00.229 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "1f15c68f-47bd-4996-892d-0ec1d929d6cf", 2024-05-10T11:33:00.229 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-05-10T11:33:00.229 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:33:00.229 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:33:00.229 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:33:00.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.223+0000 7f7958328700 1 -- 172.21.15.100:0/3758974522 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f793c0b2900 msgr2=0x7f793c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:00.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.223+0000 7f7958328700 1 --2- 172.21.15.100:0/3758974522 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f793c0b2900 0x7f793c0b4dc0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f794c009b60 tx=0x7f794c009260 comp rx=0 tx=0).stop 2024-05-10T11:33:00.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.223+0000 7f7958328700 1 -- 172.21.15.100:0/3758974522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f795010b6b0 msgr2=0x7f7950078430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:00.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.223+0000 7f7958328700 1 --2- 172.21.15.100:0/3758974522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f795010b6b0 0x7f7950078430 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f794800efb0 tx=0x7f794800c3a0 comp rx=0 tx=0).stop 2024-05-10T11:33:00.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.224+0000 7f7958328700 1 -- 172.21.15.100:0/3758974522 shutdown_connections 2024-05-10T11:33:00.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.224+0000 7f7958328700 1 --2- 172.21.15.100:0/3758974522 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f793c0b2900 0x7f793c0b4dc0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:00.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.224+0000 7f7958328700 1 --2- 172.21.15.100:0/3758974522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f795010b6b0 0x7f7950078430 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:00.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.224+0000 7f7958328700 1 --2- 172.21.15.100:0/3758974522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f79501056b0 0x7f79500779b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:00.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.224+0000 7f7958328700 1 --2- 172.21.15.100:0/3758974522 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7950106070 0x7f7950077ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:00.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.224+0000 7f7958328700 1 -- 172.21.15.100:0/3758974522 >> 172.21.15.100:0/3758974522 conn(0x7f7950101180 msgr2=0x7f7950101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:33:00.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.224+0000 7f7958328700 1 -- 172.21.15.100:0/3758974522 shutdown_connections 2024-05-10T11:33:00.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:00.224+0000 7f7958328700 1 -- 172.21.15.100:0/3758974522 wait complete. 2024-05-10T11:33:00.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:32:59 smithi178 ceph-mon[33359]: pgmap v859: 129 pgs: 129 active+clean; 355 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.1 MiB/s wr, 360 op/s 2024-05-10T11:33:00.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:32:59 smithi188 ceph-mon[34075]: pgmap v859: 129 pgs: 129 active+clean; 355 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.1 MiB/s wr, 360 op/s 2024-05-10T11:33:00.908 INFO:tasks.workunit.client.0.smithi100.stdout: 10 9222 12.36 MB/sec warmup 54 sec latency 80.238 ms 2024-05-10T11:33:01.084 INFO:tasks.cephfs.filesystem:scrub status for tag:1f15c68f-47bd-4996-892d-0ec1d929d6cf - {'path': '/', 'tag': '1f15c68f-47bd-4996-892d-0ec1d929d6cf', 'options': 'recursive,force'} 2024-05-10T11:33:01.908 INFO:tasks.workunit.client.0.smithi100.stdout: 10 9410 12.38 MB/sec warmup 55 sec latency 70.560 ms 2024-05-10T11:33:02.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:33:01 smithi178 ceph-mon[33359]: pgmap v860: 129 pgs: 129 active+clean; 354 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 819 KiB/s rd, 4.2 MiB/s wr, 338 op/s 2024-05-10T11:33:02.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:33:01 smithi188 ceph-mon[34075]: pgmap v860: 129 pgs: 129 active+clean; 354 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 819 KiB/s rd, 4.2 MiB/s wr, 338 op/s 2024-05-10T11:33:02.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:33:01 smithi100 ceph-mon[30296]: pgmap v860: 129 pgs: 129 active+clean; 354 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 819 KiB/s rd, 4.2 MiB/s wr, 338 op/s 2024-05-10T11:33:02.908 INFO:tasks.workunit.client.0.smithi100.stdout: 10 9614 12.35 MB/sec warmup 56 sec latency 63.906 ms 2024-05-10T11:33:03.908 INFO:tasks.workunit.client.0.smithi100.stdout: 10 9788 12.27 MB/sec warmup 57 sec latency 69.657 ms 2024-05-10T11:33:04.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:33:03 smithi178 ceph-mon[33359]: pgmap v861: 129 pgs: 129 active+clean; 354 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.5 MiB/s wr, 400 op/s 2024-05-10T11:33:04.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:33:03 smithi188 ceph-mon[34075]: pgmap v861: 129 pgs: 129 active+clean; 354 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.5 MiB/s wr, 400 op/s 2024-05-10T11:33:04.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:33:03 smithi100 ceph-mon[30296]: pgmap v861: 129 pgs: 129 active+clean; 354 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.5 MiB/s wr, 400 op/s 2024-05-10T11:33:04.435 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.435+0000 7f0ad2be7640 1 -- 172.21.15.100:0/3279338563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0acc102b00 msgr2=0x7f0acc102f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:04.435 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.435+0000 7f0ad2be7640 1 --2- 172.21.15.100:0/3279338563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0acc102b00 0x7f0acc102f80 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8009ca0 tx=0x7f0ab802a8c0 comp rx=0 tx=0).stop 2024-05-10T11:33:04.435 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.435+0000 7f0ad2be7640 1 -- 172.21.15.100:0/3279338563 shutdown_connections 2024-05-10T11:33:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.435+0000 7f0ad2be7640 1 --2- 172.21.15.100:0/3279338563 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0acc1034c0 0x7f0acc109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.435+0000 7f0ad2be7640 1 --2- 172.21.15.100:0/3279338563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0acc102b00 0x7f0acc102f80 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.435+0000 7f0ad2be7640 1 --2- 172.21.15.100:0/3279338563 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0acc101910 0x7f0acc101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.435+0000 7f0ad2be7640 1 -- 172.21.15.100:0/3279338563 >> 172.21.15.100:0/3279338563 conn(0x7f0acc0fce80 msgr2=0x7f0acc0ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:33:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.435+0000 7f0ad2be7640 1 -- 172.21.15.100:0/3279338563 shutdown_connections 2024-05-10T11:33:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.435+0000 7f0ad2be7640 1 -- 172.21.15.100:0/3279338563 wait complete. 2024-05-10T11:33:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.436+0000 7f0ad2be7640 1 Processor -- start 2024-05-10T11:33:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.436+0000 7f0ad2be7640 1 -- start start 2024-05-10T11:33:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.436+0000 7f0ad2be7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0acc101910 0x7f0acc1058a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.436+0000 7f0ad2be7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0acc102b00 0x7f0acc103ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.436+0000 7f0ad2be7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0acc1034c0 0x7f0acc104420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.436+0000 7f0ad2be7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0acc10d340 con 0x7f0acc101910 2024-05-10T11:33:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.436+0000 7f0ad2be7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0acc10d1c0 con 0x7f0acc1034c0 2024-05-10T11:33:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.436+0000 7f0ad2be7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0acc10d4c0 con 0x7f0acc102b00 2024-05-10T11:33:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.436+0000 7f0ad0983640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0acc101910 0x7f0acc1058a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.436+0000 7f0ad0983640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0acc101910 0x7f0acc1058a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56554/0 (socket says 172.21.15.100:56554) 2024-05-10T11:33:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.436+0000 7f0ad0983640 1 -- 172.21.15.100:0/2990429217 learned_addr learned my addr 172.21.15.100:0/2990429217 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:33:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.436+0000 7f0ad1184640 1 --2- 172.21.15.100:0/2990429217 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0acc1034c0 0x7f0acc104420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.436+0000 7f0ac3fff640 1 --2- 172.21.15.100:0/2990429217 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0acc102b00 0x7f0acc103ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.437+0000 7f0ad0983640 1 -- 172.21.15.100:0/2990429217 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0acc102b00 msgr2=0x7f0acc103ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.437+0000 7f0ad0983640 1 --2- 172.21.15.100:0/2990429217 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0acc102b00 0x7f0acc103ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.437+0000 7f0ad0983640 1 -- 172.21.15.100:0/2990429217 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0acc1034c0 msgr2=0x7f0acc104420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.437+0000 7f0ad0983640 1 --2- 172.21.15.100:0/2990429217 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0acc1034c0 0x7f0acc104420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.437+0000 7f0ad0983640 1 -- 172.21.15.100:0/2990429217 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0acc104cb0 con 0x7f0acc101910 2024-05-10T11:33:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.437+0000 7f0ac3fff640 1 --2- 172.21.15.100:0/2990429217 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0acc102b00 0x7f0acc103ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:33:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.437+0000 7f0ad1184640 1 --2- 172.21.15.100:0/2990429217 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0acc1034c0 0x7f0acc104420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:33:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.437+0000 7f0ad0983640 1 --2- 172.21.15.100:0/2990429217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0acc101910 0x7f0acc1058a0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f0abc00e9c0 tx=0x7f0abc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:33:04.440 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.437+0000 7f0aca7fc640 1 -- 172.21.15.100:0/2990429217 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0abc00cd80 con 0x7f0acc101910 2024-05-10T11:33:04.440 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.437+0000 7f0aca7fc640 1 -- 172.21.15.100:0/2990429217 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0abc0044b0 con 0x7f0acc101910 2024-05-10T11:33:04.440 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.437+0000 7f0ad2be7640 1 -- 172.21.15.100:0/2990429217 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0acc1a2c00 con 0x7f0acc101910 2024-05-10T11:33:04.440 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.437+0000 7f0aca7fc640 1 -- 172.21.15.100:0/2990429217 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0abc010430 con 0x7f0acc101910 2024-05-10T11:33:04.440 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.437+0000 7f0ad2be7640 1 -- 172.21.15.100:0/2990429217 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0acc1a3150 con 0x7f0acc101910 2024-05-10T11:33:04.441 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.440+0000 7f0ad59dd740 1 Processor -- start 2024-05-10T11:33:04.442 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.440+0000 7f0ad59dd740 1 -- start start 2024-05-10T11:33:04.442 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.441+0000 7f0aca7fc640 1 -- 172.21.15.100:0/2990429217 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f0abc003f00 con 0x7f0acc101910 2024-05-10T11:33:04.442 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.441+0000 7f0ad59dd740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f95ca96db0 0x55f95ca971d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:04.442 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.441+0000 7f0ad59dd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f95ca977a0 0x55f95ca9c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:04.442 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.441+0000 7f0ad59dd740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f95ca9c910 0x55f95ca9eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:04.442 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.441+0000 7f0ad59dd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55f95c9fb5d0 con 0x55f95ca977a0 2024-05-10T11:33:04.442 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.441+0000 7f0ad59dd740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f95c9b4890 con 0x55f95ca96db0 2024-05-10T11:33:04.442 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.441+0000 7f0ad59dd740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55f95c9f7590 con 0x55f95ca9c910 2024-05-10T11:33:04.442 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.441+0000 7f0ac3fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f95ca9c910 0x55f95ca9eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:04.442 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.441+0000 7f0ad0983640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f95ca977a0 0x55f95ca9c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:04.442 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.441+0000 7f0ac3fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f95ca9c910 0x55f95ca9eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:51462/0 (socket says 172.21.15.100:51462) 2024-05-10T11:33:04.442 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.441+0000 7f0ac3fff640 1 -- 172.21.15.100:0/3718538159 learned_addr learned my addr 172.21.15.100:0/3718538159 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:33:04.442 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.441+0000 7f0ad1184640 1 --2- 172.21.15.100:0/3718538159 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f95ca96db0 0x55f95ca971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.441+0000 7f0ad1184640 1 -- 172.21.15.100:0/3718538159 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f95ca9c910 msgr2=0x55f95ca9eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.441+0000 7f0ad1184640 1 --2- 172.21.15.100:0/3718538159 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f95ca9c910 0x55f95ca9eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.441+0000 7f0ad1184640 1 -- 172.21.15.100:0/3718538159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f95ca977a0 msgr2=0x55f95ca9c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.441+0000 7f0ad1184640 1 --2- 172.21.15.100:0/3718538159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f95ca977a0 0x55f95ca9c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.441+0000 7f0ad1184640 1 -- 172.21.15.100:0/3718538159 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f95c70d210 con 0x55f95ca96db0 2024-05-10T11:33:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.441+0000 7f0ac3fff640 1 --2- 172.21.15.100:0/3718538159 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f95ca9c910 0x55f95ca9eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:33:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.441+0000 7f0ad1184640 1 --2- 172.21.15.100:0/3718538159 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f95ca96db0 0x55f95ca971d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0ac4009f50 tx=0x7f0ac40118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=138311b735f45b36 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:33:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.442+0000 7f0aca7fc640 1 --2- 172.21.15.100:0/2990429217 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0ab00b28f0 0x7f0ab00b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.442+0000 7f0ac2ffd640 1 -- 172.21.15.100:0/3718538159 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac4004310 con 0x55f95ca96db0 2024-05-10T11:33:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.442+0000 7f0ac2ffd640 1 -- 172.21.15.100:0/3718538159 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0ac40044b0 con 0x55f95ca96db0 2024-05-10T11:33:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.442+0000 7f0ac2ffd640 1 -- 172.21.15.100:0/3718538159 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac40035f0 con 0x55f95ca96db0 2024-05-10T11:33:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.442+0000 7f0ad0983640 1 --2- 172.21.15.100:0/2990429217 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0ab00b28f0 0x7f0ab00b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.442+0000 7f0aca7fc640 1 -- 172.21.15.100:0/2990429217 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f0abc0e2a50 con 0x7f0acc101910 2024-05-10T11:33:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.442+0000 7f0ad59dd740 1 -- 172.21.15.100:0/3718538159 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f95ca96db0 msgr2=0x55f95ca971d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.443+0000 7f0ad59dd740 1 --2- 172.21.15.100:0/3718538159 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f95ca96db0 0x55f95ca971d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0ac4009f50 tx=0x7f0ac40118d0 comp rx=0 tx=0).stop 2024-05-10T11:33:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.443+0000 7f0ad0983640 1 --2- 172.21.15.100:0/2990429217 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0ab00b28f0 0x7f0ab00b4db0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f0abc0e0560 tx=0x7f0abc0e7040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:33:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.443+0000 7f0ad59dd740 1 -- 172.21.15.100:0/3718538159 shutdown_connections 2024-05-10T11:33:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.443+0000 7f0ad59dd740 1 --2- 172.21.15.100:0/3718538159 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f95ca9c910 0x55f95ca9eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.443+0000 7f0ad59dd740 1 --2- 172.21.15.100:0/3718538159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f95ca977a0 0x55f95ca9c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.443+0000 7f0ad59dd740 1 --2- 172.21.15.100:0/3718538159 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f95ca96db0 0x55f95ca971d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.443+0000 7f0ad59dd740 1 -- 172.21.15.100:0/3718538159 >> 172.21.15.100:0/3718538159 conn(0x55f95ca92420 msgr2=0x55f95ca92820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:33:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.443+0000 7f0ad59dd740 1 -- 172.21.15.100:0/3718538159 shutdown_connections 2024-05-10T11:33:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.443+0000 7f0ad59dd740 1 -- 172.21.15.100:0/3718538159 wait complete. 2024-05-10T11:33:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.443+0000 7f0ad59dd740 1 -- start start 2024-05-10T11:33:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.443+0000 7f0ad59dd740 1 Processor -- start 2024-05-10T11:33:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.444+0000 7f0ad59dd740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f95ca96db0 0x55f95caa0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:04.446 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.444+0000 7f0ad59dd740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f95ca977a0 0x55f95caa1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:04.446 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.444+0000 7f0ad59dd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f95ca9c910 0x55f95caa8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:04.449 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.444+0000 7f0ad59dd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55f95ca9f8c0 con 0x55f95ca9c910 2024-05-10T11:33:04.449 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.444+0000 7f0ad59dd740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f95c9b8370 con 0x55f95ca96db0 2024-05-10T11:33:04.449 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.444+0000 7f0ad59dd740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55f95ca9fa40 con 0x55f95ca977a0 2024-05-10T11:33:04.449 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.444+0000 7f0ad59dd740 10 client.0 _ll_register_callbacks cb 0x55f95c70ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:33:04.449 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.444+0000 7f0ad1184640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f95ca96db0 0x55f95caa0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:04.449 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.444+0000 7f0ac3fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f95ca977a0 0x55f95caa1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:04.449 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.444+0000 7f0ad1184640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f95ca96db0 0x55f95caa0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:52418/0 (socket says 172.21.15.100:52418) 2024-05-10T11:33:04.449 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.444+0000 7f0ad1184640 1 -- 172.21.15.100:0/3609658285 learned_addr learned my addr 172.21.15.100:0/3609658285 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:33:04.450 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.444+0000 7f0ac3fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f95ca977a0 0x55f95caa1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:51478/0 (socket says 172.21.15.100:51478) 2024-05-10T11:33:04.450 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.444+0000 7f0ad1184640 1 --2- 172.21.15.100:0/3609658285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f95ca9c910 0x55f95caa8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:04.450 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.444+0000 7f0ac3fff640 1 -- 172.21.15.100:0/3609658285 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f95ca96db0 msgr2=0x55f95caa0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:04.450 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.444+0000 7f0ac3fff640 1 --2- 172.21.15.100:0/3609658285 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f95ca96db0 0x55f95caa0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:04.450 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.444+0000 7f0ac3fff640 1 -- 172.21.15.100:0/3609658285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f95ca9c910 msgr2=0x55f95caa8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:04.450 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.444+0000 7f0ac3fff640 1 --2- 172.21.15.100:0/3609658285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f95ca9c910 0x55f95caa8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:04.450 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.444+0000 7f0ac3fff640 1 -- 172.21.15.100:0/3609658285 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f95c7d9620 con 0x55f95ca977a0 2024-05-10T11:33:04.450 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.444+0000 7f0ac3fff640 1 --2- 172.21.15.100:0/3609658285 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f95ca977a0 0x55f95caa1500 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x55f95ca932f0 tx=0x7f0ab8031a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94d38786c381a63a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:33:04.450 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.444+0000 7f0aaffff640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:33:04.450 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.445+0000 7f0aaffff640 1 -- 172.21.15.100:0/3609658285 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab8003b70 con 0x55f95ca977a0 2024-05-10T11:33:04.450 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.445+0000 7f0ad59dd740 1 -- 172.21.15.100:0/3609658285 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ac40044b0 con 0x55f95ca977a0 2024-05-10T11:33:04.450 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.445+0000 7f0aaffff640 1 -- 172.21.15.100:0/3609658285 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f0ab8003d10 con 0x55f95ca977a0 2024-05-10T11:33:04.450 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.445+0000 7f0aaffff640 1 -- 172.21.15.100:0/3609658285 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f0ab8045850 con 0x55f95ca977a0 2024-05-10T11:33:04.450 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.445+0000 7f0ad59dd740 10 client.27820 fetch_fsmap learned FSMap version 37 2024-05-10T11:33:04.451 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.445+0000 7f0ad59dd740 1 -- 172.21.15.100:0/3609658285 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55f95caab500 con 0x55f95ca977a0 2024-05-10T11:33:04.451 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.445+0000 7f0aaffff640 1 -- 172.21.15.100:0/3609658285 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0ac40044b0 con 0x55f95ca977a0 2024-05-10T11:33:04.451 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.445+0000 7f0aaffff640 1 -- 172.21.15.100:0/3609658285 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f0ab80044d0 con 0x55f95ca977a0 2024-05-10T11:33:04.451 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.445+0000 7f0ad59dd740 10 client.27820 fetch_fsmap finished waiting for FSMap version 37 2024-05-10T11:33:04.451 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.445+0000 7f0ad59dd740 10 client.27820 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:33:04.451 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.445+0000 7f0ad59dd740 20 client.27820 populate_metadata read hostname 'smithi100' 2024-05-10T11:33:04.451 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.445+0000 7f0ad59dd740 1 --2- 172.21.15.100:0/3609658285 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55f95caacce0 0x55f95caaf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:04.451 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.445+0000 7f0ad59dd740 4 client.27820 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:33:04.451 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.445+0000 7f0ad59dd740 1 -- 172.21.15.100:0/3609658285 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f95ca091e0 con 0x55f95caacce0 2024-05-10T11:33:04.451 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.446+0000 7f0ad1184640 1 --2- 172.21.15.100:0/3609658285 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55f95caacce0 0x55f95caaf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:04.451 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.446+0000 7f0aaffff640 10 client.27820 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:33:04.451 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.446+0000 7f0ad1184640 1 --2- 172.21.15.100:0/3609658285 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55f95caacce0 0x55f95caaf120 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da778d10d3c1ea7f server_cookie=fd018ef3ee9e0430 in_seq=0 out_seq=0 2024-05-10T11:33:04.451 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.447+0000 7f0aaffff640 1 -- 172.21.15.100:0/3609658285 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f95ca091e0 con 0x55f95caacce0 2024-05-10T11:33:04.451 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.447+0000 7f0aaffff640 10 client.27820 handle_command_reply: tid=0 2024-05-10T11:33:04.452 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.447+0000 7f0ad59dd740 1 client.27820 shutdown 2024-05-10T11:33:04.452 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.447+0000 7f0ad59dd740 1 -- 172.21.15.100:0/3609658285 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f95ca977a0 msgr2=0x55f95caa1500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:04.452 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.447+0000 7f0ad59dd740 1 --2- 172.21.15.100:0/3609658285 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f95ca977a0 0x55f95caa1500 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x55f95ca932f0 tx=0x7f0ab8031a80 comp rx=0 tx=0).stop 2024-05-10T11:33:04.452 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.447+0000 7f0ad59dd740 1 -- 172.21.15.100:0/3609658285 shutdown_connections 2024-05-10T11:33:04.452 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.447+0000 7f0ad59dd740 1 --2- 172.21.15.100:0/3609658285 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55f95caacce0 0x55f95caaf120 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:04.452 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.447+0000 7f0ad59dd740 1 --2- 172.21.15.100:0/3609658285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f95ca9c910 0x55f95caa8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:04.452 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.447+0000 7f0ad59dd740 1 --2- 172.21.15.100:0/3609658285 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f95ca977a0 0x55f95caa1500 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:04.452 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.447+0000 7f0aaffff640 0 client.27820 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:33:04.452 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.447+0000 7f0ad59dd740 1 --2- 172.21.15.100:0/3609658285 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f95ca96db0 0x55f95caa0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:04.452 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.447+0000 7f0ad59dd740 1 -- 172.21.15.100:0/3609658285 >> 172.21.15.100:0/3609658285 conn(0x55f95ca92420 msgr2=0x55f95ca91580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:33:04.452 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.447+0000 7f0ad59dd740 1 -- 172.21.15.100:0/3609658285 shutdown_connections 2024-05-10T11:33:04.452 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.447+0000 7f0ad59dd740 1 -- 172.21.15.100:0/3609658285 wait complete. 2024-05-10T11:33:04.452 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.447+0000 7f0ad59dd740 20 client.27820 trim_cache size 0 max 16384 2024-05-10T11:33:04.472 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.470+0000 7f0ad59dd740 1 Processor -- start 2024-05-10T11:33:04.472 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.471+0000 7f0ad59dd740 1 -- start start 2024-05-10T11:33:04.472 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.471+0000 7f0ad59dd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f95ca96db0 0x55f95ca95330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:04.472 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.471+0000 7f0ad59dd740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f95ca977a0 0x55f95caa8500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:04.473 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.471+0000 7f0ad59dd740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f95ca9c910 0x55f95ca95870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:04.473 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.471+0000 7f0ad59dd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0acc1aba70 con 0x55f95ca96db0 2024-05-10T11:33:04.473 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.471+0000 7f0ad59dd740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0acc1a92b0 con 0x55f95ca9c910 2024-05-10T11:33:04.473 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.471+0000 7f0ad59dd740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0acc1a7120 con 0x55f95ca977a0 2024-05-10T11:33:04.473 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.471+0000 7f0ad1184640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f95ca96db0 0x55f95ca95330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:04.473 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.471+0000 7f0ad1184640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f95ca96db0 0x55f95ca95330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56578/0 (socket says 172.21.15.100:56578) 2024-05-10T11:33:04.473 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.471+0000 7f0ad1184640 1 -- 172.21.15.100:0/1874584786 learned_addr learned my addr 172.21.15.100:0/1874584786 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:33:04.473 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.471+0000 7f0ac3fff640 1 --2- 172.21.15.100:0/1874584786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f95ca977a0 0x55f95caa8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:04.473 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.471+0000 7f0ad1184640 1 --2- 172.21.15.100:0/1874584786 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f95ca9c910 0x55f95ca95870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:04.473 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.471+0000 7f0ad1184640 1 -- 172.21.15.100:0/1874584786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f95ca977a0 msgr2=0x55f95caa8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:04.473 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.471+0000 7f0ad1184640 1 --2- 172.21.15.100:0/1874584786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f95ca977a0 0x55f95caa8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:04.473 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.471+0000 7f0ad1184640 1 -- 172.21.15.100:0/1874584786 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f95ca9c910 msgr2=0x55f95ca95870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:04.473 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.471+0000 7f0ad1184640 1 --2- 172.21.15.100:0/1874584786 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f95ca9c910 0x55f95ca95870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:04.473 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.471+0000 7f0ad1184640 1 -- 172.21.15.100:0/1874584786 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f95caaf7a0 con 0x55f95ca96db0 2024-05-10T11:33:04.473 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.472+0000 7f0ad1184640 1 --2- 172.21.15.100:0/1874584786 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f95ca96db0 0x55f95ca95330 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f0ac4009710 tx=0x7f0ac40096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c74dd4c4706deed server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:33:04.474 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.472+0000 7f0ac27fc640 1 -- 172.21.15.100:0/1874584786 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac40043a0 con 0x55f95ca96db0 2024-05-10T11:33:04.474 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.472+0000 7f0ac27fc640 1 -- 172.21.15.100:0/1874584786 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0ac4003b80 con 0x55f95ca96db0 2024-05-10T11:33:04.474 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.472+0000 7f0ac27fc640 1 -- 172.21.15.100:0/1874584786 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac402e5d0 con 0x55f95ca96db0 2024-05-10T11:33:04.476 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.473+0000 7f0ad59dd740 1 -- 172.21.15.100:0/1874584786 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f95ca96db0 msgr2=0x55f95ca95330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:04.476 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.473+0000 7f0ad59dd740 1 --2- 172.21.15.100:0/1874584786 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f95ca96db0 0x55f95ca95330 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f0ac4009710 tx=0x7f0ac40096a0 comp rx=0 tx=0).stop 2024-05-10T11:33:04.476 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.473+0000 7f0ad59dd740 1 -- 172.21.15.100:0/1874584786 shutdown_connections 2024-05-10T11:33:04.476 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.473+0000 7f0ad59dd740 1 --2- 172.21.15.100:0/1874584786 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f95ca9c910 0x55f95ca95870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:04.476 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.473+0000 7f0ad59dd740 1 --2- 172.21.15.100:0/1874584786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f95ca977a0 0x55f95caa8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:04.476 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.473+0000 7f0ad59dd740 1 --2- 172.21.15.100:0/1874584786 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f95ca96db0 0x55f95ca95330 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:04.476 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.473+0000 7f0ad59dd740 1 -- 172.21.15.100:0/1874584786 >> 172.21.15.100:0/1874584786 conn(0x55f95ca92420 msgr2=0x55f95ca98940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:33:04.477 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.473+0000 7f0ad59dd740 1 -- 172.21.15.100:0/1874584786 shutdown_connections 2024-05-10T11:33:04.477 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.473+0000 7f0ad59dd740 1 -- 172.21.15.100:0/1874584786 wait complete. 2024-05-10T11:33:04.477 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.473+0000 7f0ad59dd740 1 -- start start 2024-05-10T11:33:04.477 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.474+0000 7f0ad59dd740 1 Processor -- start 2024-05-10T11:33:04.477 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.474+0000 7f0ad59dd740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f95ca96db0 0x55f95caa5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:04.477 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.474+0000 7f0ad59dd740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f95ca977a0 0x55f95caa6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:04.477 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.474+0000 7f0ad59dd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f95ca9c910 0x55f95caa7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:04.477 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.474+0000 7f0ad59dd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0acc1a7770 con 0x55f95ca9c910 2024-05-10T11:33:04.477 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.474+0000 7f0ad59dd740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0acc1a8be0 con 0x55f95ca96db0 2024-05-10T11:33:04.477 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.474+0000 7f0ad59dd740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0acc1a7e40 con 0x55f95ca977a0 2024-05-10T11:33:04.477 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.474+0000 7f0ad59dd740 10 client.0 _ll_register_callbacks cb 0x7f0ac4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:33:04.477 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.474+0000 7f0ac3fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f95ca977a0 0x55f95caa6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:04.477 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.474+0000 7f0ad1184640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f95ca9c910 0x55f95caa7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:04.477 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.474+0000 7f0ac3fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f95ca977a0 0x55f95caa6320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:51488/0 (socket says 172.21.15.100:51488) 2024-05-10T11:33:04.478 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.474+0000 7f0ac3fff640 1 -- 172.21.15.100:0/2434818517 learned_addr learned my addr 172.21.15.100:0/2434818517 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:33:04.478 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.474+0000 7f0ac3fff640 1 -- 172.21.15.100:0/2434818517 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f95ca96db0 msgr2=0x55f95caa5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:04.478 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.474+0000 7f0ac3fff640 1 --2- 172.21.15.100:0/2434818517 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f95ca96db0 0x55f95caa5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:04.478 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.475+0000 7f0ac3fff640 1 -- 172.21.15.100:0/2434818517 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f95ca9c910 msgr2=0x55f95caa7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:04.478 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.475+0000 7f0ac3fff640 1 --2- 172.21.15.100:0/2434818517 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f95ca9c910 0x55f95caa7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:04.478 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.475+0000 7f0ac3fff640 1 -- 172.21.15.100:0/2434818517 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f95caa7920 con 0x55f95ca977a0 2024-05-10T11:33:04.478 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.475+0000 7f0ad1184640 1 --2- 172.21.15.100:0/2434818517 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f95ca9c910 0x55f95caa7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:33:04.478 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.475+0000 7f0ac3fff640 1 --2- 172.21.15.100:0/2434818517 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f95ca977a0 0x55f95caa6320 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8005d50 tx=0x7f0ab8030e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1dcf46572ee97c06 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:33:04.478 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.475+0000 7f0aaffff640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:33:04.478 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.475+0000 7f0aaffff640 1 -- 172.21.15.100:0/2434818517 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab80459f0 con 0x55f95ca977a0 2024-05-10T11:33:04.478 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.475+0000 7f0ad59dd740 1 -- 172.21.15.100:0/2434818517 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ac4003b80 con 0x55f95ca977a0 2024-05-10T11:33:04.478 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.475+0000 7f0aaffff640 1 -- 172.21.15.100:0/2434818517 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f0ab8045b90 con 0x55f95ca977a0 2024-05-10T11:33:04.478 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.475+0000 7f0aaffff640 1 -- 172.21.15.100:0/2434818517 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f0ab8045d90 con 0x55f95ca977a0 2024-05-10T11:33:04.478 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.475+0000 7f0ad59dd740 10 client.27826 fetch_fsmap learned FSMap version 37 2024-05-10T11:33:04.478 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.475+0000 7f0ad59dd740 1 -- 172.21.15.100:0/2434818517 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55f95cb17890 con 0x55f95ca977a0 2024-05-10T11:33:04.479 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.476+0000 7f0aaffff640 1 -- 172.21.15.100:0/2434818517 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0ac4003b80 con 0x55f95ca977a0 2024-05-10T11:33:04.479 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.476+0000 7f0aaffff640 1 -- 172.21.15.100:0/2434818517 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f0ab8064030 con 0x55f95ca977a0 2024-05-10T11:33:04.479 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.476+0000 7f0ad59dd740 10 client.27826 fetch_fsmap finished waiting for FSMap version 37 2024-05-10T11:33:04.479 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.476+0000 7f0ad59dd740 10 client.27826 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:33:04.479 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.476+0000 7f0ad59dd740 20 client.27826 populate_metadata read hostname 'smithi100' 2024-05-10T11:33:04.479 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.476+0000 7f0ad59dd740 1 --2- 172.21.15.100:0/2434818517 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55f95cb18310 0x55f95cb1a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:04.479 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.476+0000 7f0ad59dd740 4 client.27826 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:33:04.479 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.476+0000 7f0ad59dd740 1 -- 172.21.15.100:0/2434818517 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f95ca03050 con 0x55f95cb18310 2024-05-10T11:33:04.479 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.476+0000 7f0ad1184640 1 --2- 172.21.15.100:0/2434818517 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55f95cb18310 0x55f95cb1a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:04.480 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.476+0000 7f0aaffff640 10 client.27826 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:33:04.480 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.477+0000 7f0ad1184640 1 --2- 172.21.15.100:0/2434818517 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55f95cb18310 0x55f95cb1a730 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76ceb0c19c33de8b server_cookie=5e88c9b352c3d84 in_seq=0 out_seq=0 2024-05-10T11:33:04.551 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.550+0000 7f0aaffff640 1 -- 172.21.15.100:0/2434818517 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f95ca03050 con 0x55f95cb18310 2024-05-10T11:33:04.551 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.550+0000 7f0aaffff640 10 client.27826 handle_command_reply: tid=0 2024-05-10T11:33:04.552 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.550+0000 7f0ad59dd740 1 client.27826 shutdown 2024-05-10T11:33:04.552 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.550+0000 7f0ad59dd740 1 -- 172.21.15.100:0/2434818517 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f95ca977a0 msgr2=0x55f95caa6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:04.552 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.550+0000 7f0ad59dd740 1 --2- 172.21.15.100:0/2434818517 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f95ca977a0 0x55f95caa6320 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8005d50 tx=0x7f0ab8030e50 comp rx=0 tx=0).stop 2024-05-10T11:33:04.552 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.550+0000 7f0ad59dd740 1 -- 172.21.15.100:0/2434818517 shutdown_connections 2024-05-10T11:33:04.552 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.550+0000 7f0ad59dd740 1 --2- 172.21.15.100:0/2434818517 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55f95cb18310 0x55f95cb1a730 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:04.552 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.550+0000 7f0ad59dd740 1 --2- 172.21.15.100:0/2434818517 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f95ca9c910 0x55f95caa7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:04.552 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.550+0000 7f0ad59dd740 1 --2- 172.21.15.100:0/2434818517 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f95ca977a0 0x55f95caa6320 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:04.552 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.550+0000 7f0ad59dd740 1 --2- 172.21.15.100:0/2434818517 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f95ca96db0 0x55f95caa5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:04.553 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.550+0000 7f0ad59dd740 1 -- 172.21.15.100:0/2434818517 >> 172.21.15.100:0/2434818517 conn(0x55f95ca92420 msgr2=0x55f95ca95330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:33:04.553 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.550+0000 7f0aaffff640 0 client.27826 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:33:04.553 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.550+0000 7f0ad59dd740 1 -- 172.21.15.100:0/2434818517 shutdown_connections 2024-05-10T11:33:04.553 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.550+0000 7f0ad59dd740 1 -- 172.21.15.100:0/2434818517 wait complete. 2024-05-10T11:33:04.553 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.550+0000 7f0ad59dd740 20 client.27826 trim_cache size 0 max 16384 2024-05-10T11:33:04.553 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:33:04.553 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:33:04.553 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:33:04.553 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:33:04.554 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.551+0000 7f0ad2be7640 1 -- 172.21.15.100:0/2990429217 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0ab00b28f0 msgr2=0x7f0ab00b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:04.554 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.551+0000 7f0ad2be7640 1 --2- 172.21.15.100:0/2990429217 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0ab00b28f0 0x7f0ab00b4db0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f0abc0e0560 tx=0x7f0abc0e7040 comp rx=0 tx=0).stop 2024-05-10T11:33:04.554 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.551+0000 7f0ad2be7640 1 -- 172.21.15.100:0/2990429217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0acc101910 msgr2=0x7f0acc1058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:04.555 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.551+0000 7f0ad2be7640 1 --2- 172.21.15.100:0/2990429217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0acc101910 0x7f0acc1058a0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f0abc00e9c0 tx=0x7f0abc00ee80 comp rx=0 tx=0).stop 2024-05-10T11:33:04.555 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.551+0000 7f0ad2be7640 1 -- 172.21.15.100:0/2990429217 shutdown_connections 2024-05-10T11:33:04.555 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.551+0000 7f0ad2be7640 1 --2- 172.21.15.100:0/2990429217 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0ab00b28f0 0x7f0ab00b4db0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:04.555 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.551+0000 7f0ad2be7640 1 --2- 172.21.15.100:0/2990429217 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0acc1034c0 0x7f0acc104420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:04.555 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.551+0000 7f0ad2be7640 1 --2- 172.21.15.100:0/2990429217 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0acc102b00 0x7f0acc103ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:04.555 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.551+0000 7f0ad2be7640 1 --2- 172.21.15.100:0/2990429217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0acc101910 0x7f0acc1058a0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:04.555 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.551+0000 7f0ad2be7640 1 -- 172.21.15.100:0/2990429217 >> 172.21.15.100:0/2990429217 conn(0x7f0acc0fce80 msgr2=0x7f0acc0fd570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:33:04.555 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.551+0000 7f0ad2be7640 1 -- 172.21.15.100:0/2990429217 shutdown_connections 2024-05-10T11:33:04.555 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:04.552+0000 7f0ad2be7640 1 -- 172.21.15.100:0/2990429217 wait complete. 2024-05-10T11:33:04.908 INFO:tasks.workunit.client.0.smithi100.stdout: 10 9940 12.12 MB/sec warmup 58 sec latency 97.978 ms 2024-05-10T11:33:05.908 INFO:tasks.workunit.client.0.smithi100.stdout: 10 10112 12.10 MB/sec warmup 59 sec latency 72.825 ms 2024-05-10T11:33:06.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:33:05 smithi178 ceph-mon[33359]: pgmap v862: 129 pgs: 129 active+clean; 356 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 4.0 MiB/s wr, 373 op/s 2024-05-10T11:33:06.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:33:05 smithi188 ceph-mon[34075]: pgmap v862: 129 pgs: 129 active+clean; 356 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 4.0 MiB/s wr, 373 op/s 2024-05-10T11:33:06.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:33:05 smithi100 ceph-mon[30296]: pgmap v862: 129 pgs: 129 active+clean; 356 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 4.0 MiB/s wr, 373 op/s 2024-05-10T11:33:06.908 INFO:tasks.workunit.client.0.smithi100.stdout: 10 10270 11.98 MB/sec warmup 60 sec latency 102.252 ms 2024-05-10T11:33:07.908 INFO:tasks.workunit.client.0.smithi100.stdout: 10 10482 12.06 MB/sec warmup 61 sec latency 60.617 ms 2024-05-10T11:33:08.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:33:07 smithi178 ceph-mon[33359]: pgmap v863: 129 pgs: 129 active+clean; 357 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 892 KiB/s rd, 4.2 MiB/s wr, 370 op/s 2024-05-10T11:33:08.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:33:07 smithi188 ceph-mon[34075]: pgmap v863: 129 pgs: 129 active+clean; 357 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 892 KiB/s rd, 4.2 MiB/s wr, 370 op/s 2024-05-10T11:33:08.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:33:07 smithi100 ceph-mon[30296]: pgmap v863: 129 pgs: 129 active+clean; 357 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 892 KiB/s rd, 4.2 MiB/s wr, 370 op/s 2024-05-10T11:33:08.908 INFO:tasks.workunit.client.0.smithi100.stdout: 10 10693 12.03 MB/sec warmup 62 sec latency 56.524 ms 2024-05-10T11:33:09.908 INFO:tasks.workunit.client.0.smithi100.stdout: 10 10884 12.06 MB/sec warmup 63 sec latency 89.243 ms 2024-05-10T11:33:10.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:33:09 smithi178 ceph-mon[33359]: pgmap v864: 129 pgs: 129 active+clean; 359 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 4.0 MiB/s wr, 342 op/s 2024-05-10T11:33:10.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:33:09 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:33:10.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:33:09 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:33:10.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:33:09 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:33:10.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:33:09 smithi188 ceph-mon[34075]: pgmap v864: 129 pgs: 129 active+clean; 359 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 4.0 MiB/s wr, 342 op/s 2024-05-10T11:33:10.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:33:09 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:33:10.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:33:09 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:33:10.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:33:09 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:33:10.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:33:09 smithi100 ceph-mon[30296]: pgmap v864: 129 pgs: 129 active+clean; 359 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 4.0 MiB/s wr, 342 op/s 2024-05-10T11:33:10.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:33:09 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:33:10.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:33:09 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:33:10.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:33:09 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:33:10.908 INFO:tasks.workunit.client.0.smithi100.stdout: 10 11032 11.91 MB/sec warmup 64 sec latency 74.572 ms 2024-05-10T11:33:11.908 INFO:tasks.workunit.client.0.smithi100.stdout: 10 11208 11.87 MB/sec warmup 65 sec latency 64.326 ms 2024-05-10T11:33:12.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:33:11 smithi178 ceph-mon[33359]: pgmap v865: 129 pgs: 129 active+clean; 360 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 3.9 MiB/s wr, 281 op/s 2024-05-10T11:33:12.266 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:33:11 smithi100 ceph-mon[30296]: pgmap v865: 129 pgs: 129 active+clean; 360 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 3.9 MiB/s wr, 281 op/s 2024-05-10T11:33:12.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:33:11 smithi188 ceph-mon[34075]: pgmap v865: 129 pgs: 129 active+clean; 360 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 3.9 MiB/s wr, 281 op/s 2024-05-10T11:33:12.908 INFO:tasks.workunit.client.0.smithi100.stdout: 10 11427 11.88 MB/sec warmup 66 sec latency 89.457 ms 2024-05-10T11:33:13.908 INFO:tasks.workunit.client.0.smithi100.stdout: 10 11606 11.91 MB/sec warmup 67 sec latency 52.298 ms 2024-05-10T11:33:14.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:33:13 smithi178 ceph-mon[33359]: pgmap v866: 129 pgs: 129 active+clean; 363 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 990 KiB/s rd, 4.7 MiB/s wr, 308 op/s 2024-05-10T11:33:14.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:33:13 smithi188 ceph-mon[34075]: pgmap v866: 129 pgs: 129 active+clean; 363 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 990 KiB/s rd, 4.7 MiB/s wr, 308 op/s 2024-05-10T11:33:14.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:33:13 smithi100 ceph-mon[30296]: pgmap v866: 129 pgs: 129 active+clean; 363 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 990 KiB/s rd, 4.7 MiB/s wr, 308 op/s 2024-05-10T11:33:14.908 INFO:tasks.workunit.client.0.smithi100.stdout: 10 11797 11.94 MB/sec warmup 68 sec latency 59.719 ms 2024-05-10T11:33:15.908 INFO:tasks.workunit.client.0.smithi100.stdout: 10 12068 12.01 MB/sec warmup 69 sec latency 70.415 ms 2024-05-10T11:33:16.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:33:15 smithi178 ceph-mon[33359]: pgmap v867: 129 pgs: 129 active+clean; 365 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 885 KiB/s rd, 4.6 MiB/s wr, 276 op/s 2024-05-10T11:33:16.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:33:15 smithi188 ceph-mon[34075]: pgmap v867: 129 pgs: 129 active+clean; 365 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 885 KiB/s rd, 4.6 MiB/s wr, 276 op/s 2024-05-10T11:33:16.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:33:15 smithi100 ceph-mon[30296]: pgmap v867: 129 pgs: 129 active+clean; 365 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 885 KiB/s rd, 4.6 MiB/s wr, 276 op/s 2024-05-10T11:33:16.908 INFO:tasks.workunit.client.0.smithi100.stdout: 10 12194 11.90 MB/sec warmup 70 sec latency 57.027 ms 2024-05-10T11:33:17.684 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.682+0000 7f6f543af640 1 -- 172.21.15.100:0/697366802 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f4c103c10 msgr2=0x7f6f4c104090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:17.684 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.682+0000 7f6f543af640 1 --2- 172.21.15.100:0/697366802 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f4c103c10 0x7f6f4c104090 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c009900 tx=0x7f6f3c02a410 comp rx=0 tx=0).stop 2024-05-10T11:33:17.684 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.682+0000 7f6f543af640 1 -- 172.21.15.100:0/697366802 shutdown_connections 2024-05-10T11:33:17.684 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.682+0000 7f6f543af640 1 --2- 172.21.15.100:0/697366802 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f4c1045d0 0x7f6f4c10ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:17.684 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.682+0000 7f6f543af640 1 --2- 172.21.15.100:0/697366802 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f4c103c10 0x7f6f4c104090 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:17.684 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.683+0000 7f6f543af640 1 --2- 172.21.15.100:0/697366802 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f4c102a20 0x7f6f4c102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:17.684 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.683+0000 7f6f543af640 1 -- 172.21.15.100:0/697366802 >> 172.21.15.100:0/697366802 conn(0x7f6f4c0fdf70 msgr2=0x7f6f4c1003d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:33:17.684 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.683+0000 7f6f543af640 1 -- 172.21.15.100:0/697366802 shutdown_connections 2024-05-10T11:33:17.685 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.683+0000 7f6f543af640 1 -- 172.21.15.100:0/697366802 wait complete. 2024-05-10T11:33:17.685 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.683+0000 7f6f543af640 1 Processor -- start 2024-05-10T11:33:17.685 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.683+0000 7f6f543af640 1 -- start start 2024-05-10T11:33:17.685 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.683+0000 7f6f543af640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f4c102a20 0x7f6f4c198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:17.685 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.683+0000 7f6f543af640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f4c103c10 0x7f6f4c199000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:17.685 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.684+0000 7f6f543af640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f4c1045d0 0x7f6f4c1a0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:17.685 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.684+0000 7f6f543af640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4c10e450 con 0x7f6f4c103c10 2024-05-10T11:33:17.685 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.684+0000 7f6f543af640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4c10e2d0 con 0x7f6f4c102a20 2024-05-10T11:33:17.685 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.684+0000 7f6f543af640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4c10e5d0 con 0x7f6f4c1045d0 2024-05-10T11:33:17.685 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.684+0000 7f6f5214b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f4c102a20 0x7f6f4c198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:17.685 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.684+0000 7f6f5194a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f4c103c10 0x7f6f4c199000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:17.685 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.684+0000 7f6f5214b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f4c102a20 0x7f6f4c198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:39662/0 (socket says 172.21.15.100:39662) 2024-05-10T11:33:17.685 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.684+0000 7f6f5214b640 1 -- 172.21.15.100:0/3440525003 learned_addr learned my addr 172.21.15.100:0/3440525003 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:33:17.685 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.684+0000 7f6f5294c640 1 --2- 172.21.15.100:0/3440525003 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f4c1045d0 0x7f6f4c1a0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:17.686 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.684+0000 7f6f5214b640 1 -- 172.21.15.100:0/3440525003 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f4c1045d0 msgr2=0x7f6f4c1a0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:17.686 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.684+0000 7f6f5214b640 1 --2- 172.21.15.100:0/3440525003 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f4c1045d0 0x7f6f4c1a0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:17.686 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.684+0000 7f6f5214b640 1 -- 172.21.15.100:0/3440525003 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f4c103c10 msgr2=0x7f6f4c199000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:17.686 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.684+0000 7f6f5214b640 1 --2- 172.21.15.100:0/3440525003 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f4c103c10 0x7f6f4c199000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:17.686 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.684+0000 7f6f5214b640 1 -- 172.21.15.100:0/3440525003 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f4c1a06e0 con 0x7f6f4c102a20 2024-05-10T11:33:17.686 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.684+0000 7f6f5194a640 1 --2- 172.21.15.100:0/3440525003 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f4c103c10 0x7f6f4c199000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:33:17.687 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.684+0000 7f6f5214b640 1 --2- 172.21.15.100:0/3440525003 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f4c102a20 0x7f6f4c198ac0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6f4800e9c0 tx=0x7f6f4800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:33:17.687 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.685+0000 7f6f437fe640 1 -- 172.21.15.100:0/3440525003 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f4800cde0 con 0x7f6f4c102a20 2024-05-10T11:33:17.687 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.685+0000 7f6f543af640 1 -- 172.21.15.100:0/3440525003 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f4c074c40 con 0x7f6f4c102a20 2024-05-10T11:33:17.687 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.685+0000 7f6f437fe640 1 -- 172.21.15.100:0/3440525003 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6f480108c0 con 0x7f6f4c102a20 2024-05-10T11:33:17.687 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.685+0000 7f6f437fe640 1 -- 172.21.15.100:0/3440525003 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f48018820 con 0x7f6f4c102a20 2024-05-10T11:33:17.687 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.685+0000 7f6f543af640 1 -- 172.21.15.100:0/3440525003 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f4c0751b0 con 0x7f6f4c102a20 2024-05-10T11:33:17.689 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.688+0000 7f6f571a5740 1 Processor -- start 2024-05-10T11:33:17.689 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.688+0000 7f6f571a5740 1 -- start start 2024-05-10T11:33:17.689 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.688+0000 7f6f437fe640 1 -- 172.21.15.100:0/3440525003 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f6f4801f020 con 0x7f6f4c102a20 2024-05-10T11:33:17.689 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.688+0000 7f6f571a5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ec3492db0 0x555ec34931d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:17.689 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.688+0000 7f6f571a5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ec34937a0 0x555ec34982a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:17.689 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.688+0000 7f6f571a5740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ec3498910 0x555ec349ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:17.689 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.688+0000 7f6f571a5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x555ec33f75d0 con 0x555ec34937a0 2024-05-10T11:33:17.689 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.688+0000 7f6f571a5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x555ec33b0890 con 0x555ec3492db0 2024-05-10T11:33:17.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.688+0000 7f6f571a5740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x555ec33f3590 con 0x555ec3498910 2024-05-10T11:33:17.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.688+0000 7f6f5294c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ec3492db0 0x555ec34931d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:17.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.688+0000 7f6f5194a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ec3498910 0x555ec349ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:17.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.688+0000 7f6f5294c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ec3492db0 0x555ec34931d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:39678/0 (socket says 172.21.15.100:39678) 2024-05-10T11:33:17.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.688+0000 7f6f5294c640 1 -- 172.21.15.100:0/1099500153 learned_addr learned my addr 172.21.15.100:0/1099500153 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:33:17.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.688+0000 7f6f5214b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ec34937a0 0x555ec34982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:17.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.689+0000 7f6f5294c640 1 -- 172.21.15.100:0/1099500153 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ec3498910 msgr2=0x555ec349ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:17.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.689+0000 7f6f5294c640 1 --2- 172.21.15.100:0/1099500153 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ec3498910 0x555ec349ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:17.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.689+0000 7f6f5294c640 1 -- 172.21.15.100:0/1099500153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ec34937a0 msgr2=0x555ec34982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:17.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.689+0000 7f6f5294c640 1 --2- 172.21.15.100:0/1099500153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ec34937a0 0x555ec34982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:17.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.689+0000 7f6f5294c640 1 -- 172.21.15.100:0/1099500153 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ec3109210 con 0x555ec3492db0 2024-05-10T11:33:17.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.689+0000 7f6f5214b640 1 --2- 172.21.15.100:0/1099500153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ec34937a0 0x555ec34982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:33:17.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.689+0000 7f6f5294c640 1 --2- 172.21.15.100:0/1099500153 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ec3492db0 0x555ec34931d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6f44009fd0 tx=0x7f6f440118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7aa06d990a76bf41 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:33:17.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.689+0000 7f6f40bfe640 1 -- 172.21.15.100:0/1099500153 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f44004d20 con 0x555ec3492db0 2024-05-10T11:33:17.691 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.689+0000 7f6f40bfe640 1 -- 172.21.15.100:0/1099500153 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6f44004050 con 0x555ec3492db0 2024-05-10T11:33:17.691 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.689+0000 7f6f40bfe640 1 -- 172.21.15.100:0/1099500153 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f44005260 con 0x555ec3492db0 2024-05-10T11:33:17.691 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.689+0000 7f6f437fe640 1 --2- 172.21.15.100:0/3440525003 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f6f380b28a0 0x7f6f380b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:17.691 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.689+0000 7f6f5214b640 1 --2- 172.21.15.100:0/3440525003 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f6f380b28a0 0x7f6f380b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:17.695 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.690+0000 7f6f437fe640 1 -- 172.21.15.100:0/3440525003 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f6f480e26f0 con 0x7f6f4c102a20 2024-05-10T11:33:17.695 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.690+0000 7f6f571a5740 1 -- 172.21.15.100:0/1099500153 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ec3492db0 msgr2=0x555ec34931d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:17.695 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.690+0000 7f6f571a5740 1 --2- 172.21.15.100:0/1099500153 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ec3492db0 0x555ec34931d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6f44009fd0 tx=0x7f6f440118d0 comp rx=0 tx=0).stop 2024-05-10T11:33:17.695 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.690+0000 7f6f5214b640 1 --2- 172.21.15.100:0/3440525003 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f6f380b28a0 0x7f6f380b4d60 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f6f480e0200 tx=0x7f6f48023650 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:33:17.695 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.690+0000 7f6f571a5740 1 -- 172.21.15.100:0/1099500153 shutdown_connections 2024-05-10T11:33:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.690+0000 7f6f571a5740 1 --2- 172.21.15.100:0/1099500153 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ec3498910 0x555ec349ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.690+0000 7f6f571a5740 1 --2- 172.21.15.100:0/1099500153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ec34937a0 0x555ec34982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.690+0000 7f6f571a5740 1 --2- 172.21.15.100:0/1099500153 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ec3492db0 0x555ec34931d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.690+0000 7f6f571a5740 1 -- 172.21.15.100:0/1099500153 >> 172.21.15.100:0/1099500153 conn(0x555ec348e420 msgr2=0x555ec348e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:33:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.690+0000 7f6f571a5740 1 -- 172.21.15.100:0/1099500153 shutdown_connections 2024-05-10T11:33:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.690+0000 7f6f571a5740 1 -- 172.21.15.100:0/1099500153 wait complete. 2024-05-10T11:33:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.691+0000 7f6f571a5740 1 -- start start 2024-05-10T11:33:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.691+0000 7f6f571a5740 1 Processor -- start 2024-05-10T11:33:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.691+0000 7f6f571a5740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ec3492db0 0x555ec349cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.691+0000 7f6f571a5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ec34937a0 0x555ec349d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.691+0000 7f6f571a5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ec3498910 0x555ec34a4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.691+0000 7f6f571a5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x555ec349b8c0 con 0x555ec34937a0 2024-05-10T11:33:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.691+0000 7f6f571a5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x555ec33b4370 con 0x555ec3498910 2024-05-10T11:33:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.691+0000 7f6f571a5740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x555ec349ba40 con 0x555ec3492db0 2024-05-10T11:33:17.697 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.691+0000 7f6f571a5740 10 client.0 _ll_register_callbacks cb 0x555ec3108a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:33:17.697 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.691+0000 7f6f5194a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ec34937a0 0x555ec349d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:17.697 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.691+0000 7f6f5294c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ec3492db0 0x555ec349cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:17.697 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.691+0000 7f6f5194a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ec34937a0 0x555ec349d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59448/0 (socket says 172.21.15.100:59448) 2024-05-10T11:33:17.698 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.691+0000 7f6f5194a640 1 -- 172.21.15.100:0/3828058878 learned_addr learned my addr 172.21.15.100:0/3828058878 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:33:17.698 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.691+0000 7f6f5294c640 1 --2- 172.21.15.100:0/3828058878 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ec3498910 0x555ec34a4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:17.698 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.692+0000 7f6f5194a640 1 -- 172.21.15.100:0/3828058878 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ec3492db0 msgr2=0x555ec349cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:17.698 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.692+0000 7f6f5194a640 1 --2- 172.21.15.100:0/3828058878 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ec3492db0 0x555ec349cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:17.698 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.692+0000 7f6f5194a640 1 -- 172.21.15.100:0/3828058878 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ec3498910 msgr2=0x555ec34a4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:17.698 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.692+0000 7f6f5194a640 1 --2- 172.21.15.100:0/3828058878 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ec3498910 0x555ec34a4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:17.698 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.692+0000 7f6f5194a640 1 -- 172.21.15.100:0/3828058878 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ec31d5620 con 0x555ec34937a0 2024-05-10T11:33:17.698 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.692+0000 7f6f5194a640 1 --2- 172.21.15.100:0/3828058878 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ec34937a0 0x555ec349d450 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c031460 tx=0x7f6f3c03bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f85566fefc681393 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:33:17.698 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.692+0000 7f6f35ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:33:17.698 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.692+0000 7f6f35ffb640 1 -- 172.21.15.100:0/3828058878 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c035040 con 0x555ec34937a0 2024-05-10T11:33:17.698 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.692+0000 7f6f571a5740 1 -- 172.21.15.100:0/3828058878 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f44004050 con 0x555ec34937a0 2024-05-10T11:33:17.698 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.692+0000 7f6f35ffb640 1 -- 172.21.15.100:0/3828058878 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f6f3c0037f0 con 0x555ec34937a0 2024-05-10T11:33:17.698 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.692+0000 7f6f35ffb640 1 -- 172.21.15.100:0/3828058878 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f6f3c053050 con 0x555ec34937a0 2024-05-10T11:33:17.698 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.693+0000 7f6f571a5740 10 client.18405 fetch_fsmap learned FSMap version 37 2024-05-10T11:33:17.699 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.693+0000 7f6f571a5740 1 -- 172.21.15.100:0/3828058878 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x555ec34a7530 con 0x555ec34937a0 2024-05-10T11:33:17.699 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.693+0000 7f6f35ffb640 1 -- 172.21.15.100:0/3828058878 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6f44004050 con 0x555ec34937a0 2024-05-10T11:33:17.699 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.693+0000 7f6f35ffb640 1 -- 172.21.15.100:0/3828058878 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f6f3c04e030 con 0x555ec34937a0 2024-05-10T11:33:17.699 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.693+0000 7f6f571a5740 10 client.18405 fetch_fsmap finished waiting for FSMap version 37 2024-05-10T11:33:17.699 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.693+0000 7f6f571a5740 10 client.18405 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:33:17.699 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.693+0000 7f6f571a5740 20 client.18405 populate_metadata read hostname 'smithi100' 2024-05-10T11:33:17.699 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.693+0000 7f6f571a5740 1 --2- 172.21.15.100:0/3828058878 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x555ec34a9070 0x555ec34ab460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:17.699 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.693+0000 7f6f571a5740 4 client.18405 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:33:17.699 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.693+0000 7f6f571a5740 1 -- 172.21.15.100:0/3828058878 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ec34051e0 con 0x555ec34a9070 2024-05-10T11:33:17.699 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.694+0000 7f6f5294c640 1 --2- 172.21.15.100:0/3828058878 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x555ec34a9070 0x555ec34ab460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:17.699 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.694+0000 7f6f5294c640 1 --2- 172.21.15.100:0/3828058878 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x555ec34a9070 0x555ec34ab460 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e397809c50b488d server_cookie=18370db6abeef664 in_seq=0 out_seq=0 2024-05-10T11:33:17.699 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.694+0000 7f6f35ffb640 10 client.18405 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:33:17.699 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.695+0000 7f6f35ffb640 1 -- 172.21.15.100:0/3828058878 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555ec34051e0 con 0x555ec34a9070 2024-05-10T11:33:17.699 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.695+0000 7f6f35ffb640 10 client.18405 handle_command_reply: tid=0 2024-05-10T11:33:17.699 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.695+0000 7f6f571a5740 1 client.18405 shutdown 2024-05-10T11:33:17.700 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.695+0000 7f6f571a5740 1 -- 172.21.15.100:0/3828058878 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ec34937a0 msgr2=0x555ec349d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:17.700 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.695+0000 7f6f571a5740 1 --2- 172.21.15.100:0/3828058878 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ec34937a0 0x555ec349d450 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c031460 tx=0x7f6f3c03bef0 comp rx=0 tx=0).stop 2024-05-10T11:33:17.700 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.695+0000 7f6f571a5740 1 -- 172.21.15.100:0/3828058878 shutdown_connections 2024-05-10T11:33:17.700 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.695+0000 7f6f571a5740 1 --2- 172.21.15.100:0/3828058878 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x555ec34a9070 0x555ec34ab460 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:17.700 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.695+0000 7f6f571a5740 1 --2- 172.21.15.100:0/3828058878 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ec3498910 0x555ec34a4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:17.700 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.695+0000 7f6f571a5740 1 --2- 172.21.15.100:0/3828058878 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ec34937a0 0x555ec349d450 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:17.700 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.695+0000 7f6f571a5740 1 --2- 172.21.15.100:0/3828058878 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ec3492db0 0x555ec349cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:17.700 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.695+0000 7f6f571a5740 1 -- 172.21.15.100:0/3828058878 >> 172.21.15.100:0/3828058878 conn(0x555ec348e420 msgr2=0x555ec348d580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:33:17.700 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.695+0000 7f6f35ffb640 0 client.18405 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:33:17.700 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.695+0000 7f6f571a5740 1 -- 172.21.15.100:0/3828058878 shutdown_connections 2024-05-10T11:33:17.700 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.695+0000 7f6f571a5740 1 -- 172.21.15.100:0/3828058878 wait complete. 2024-05-10T11:33:17.700 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.695+0000 7f6f571a5740 20 client.18405 trim_cache size 0 max 16384 2024-05-10T11:33:17.730 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.724+0000 7f6f571a5740 1 Processor -- start 2024-05-10T11:33:17.730 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.724+0000 7f6f571a5740 1 -- start start 2024-05-10T11:33:17.730 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.724+0000 7f6f571a5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ec3492db0 0x555ec349c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:17.731 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.724+0000 7f6f571a5740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ec34937a0 0x555ec34a4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:17.731 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.724+0000 7f6f571a5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ec3498910 0x555ec3491230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:17.731 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.724+0000 7f6f571a5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4c1aba00 con 0x555ec3492db0 2024-05-10T11:33:17.731 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.724+0000 7f6f571a5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4c1a9240 con 0x555ec3498910 2024-05-10T11:33:17.731 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.724+0000 7f6f571a5740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4c1a7090 con 0x555ec34937a0 2024-05-10T11:33:17.731 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.724+0000 7f6f5294c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ec3492db0 0x555ec349c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:17.731 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.724+0000 7f6f5294c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ec3492db0 0x555ec349c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59454/0 (socket says 172.21.15.100:59454) 2024-05-10T11:33:17.731 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.724+0000 7f6f5294c640 1 -- 172.21.15.100:0/19869618 learned_addr learned my addr 172.21.15.100:0/19869618 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:33:17.732 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.724+0000 7f6f5194a640 1 --2- 172.21.15.100:0/19869618 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ec34937a0 0x555ec34a4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:17.732 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.724+0000 7f6f5294c640 1 --2- 172.21.15.100:0/19869618 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ec3498910 0x555ec3491230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:17.732 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.725+0000 7f6f5294c640 1 -- 172.21.15.100:0/19869618 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ec34937a0 msgr2=0x555ec34a4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:17.732 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.725+0000 7f6f5294c640 1 --2- 172.21.15.100:0/19869618 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ec34937a0 0x555ec34a4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:17.732 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.725+0000 7f6f5294c640 1 -- 172.21.15.100:0/19869618 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ec3498910 msgr2=0x555ec3491230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:17.732 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.725+0000 7f6f5294c640 1 --2- 172.21.15.100:0/19869618 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ec3498910 0x555ec3491230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:17.732 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.725+0000 7f6f5294c640 1 -- 172.21.15.100:0/19869618 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ec348e250 con 0x555ec3492db0 2024-05-10T11:33:17.733 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.725+0000 7f6f5194a640 1 --2- 172.21.15.100:0/19869618 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ec34937a0 0x555ec34a4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:33:17.733 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.725+0000 7f6f5294c640 1 --2- 172.21.15.100:0/19869618 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ec3492db0 0x555ec349c530 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f6f44014040 tx=0x7f6f440118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c58a5962e5d56bd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:33:17.733 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.725+0000 7f6f37fff640 1 -- 172.21.15.100:0/19869618 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f44003c30 con 0x555ec3492db0 2024-05-10T11:33:17.733 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.725+0000 7f6f37fff640 1 -- 172.21.15.100:0/19869618 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6f440046c0 con 0x555ec3492db0 2024-05-10T11:33:17.733 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.725+0000 7f6f37fff640 1 -- 172.21.15.100:0/19869618 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f4402e870 con 0x555ec3492db0 2024-05-10T11:33:17.733 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.726+0000 7f6f571a5740 1 -- 172.21.15.100:0/19869618 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ec3492db0 msgr2=0x555ec349c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:17.733 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.726+0000 7f6f571a5740 1 --2- 172.21.15.100:0/19869618 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ec3492db0 0x555ec349c530 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f6f44014040 tx=0x7f6f440118d0 comp rx=0 tx=0).stop 2024-05-10T11:33:17.733 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.726+0000 7f6f571a5740 1 -- 172.21.15.100:0/19869618 shutdown_connections 2024-05-10T11:33:17.734 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.726+0000 7f6f571a5740 1 --2- 172.21.15.100:0/19869618 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ec3498910 0x555ec3491230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:17.734 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.726+0000 7f6f571a5740 1 --2- 172.21.15.100:0/19869618 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ec34937a0 0x555ec34a4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:17.734 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.726+0000 7f6f571a5740 1 --2- 172.21.15.100:0/19869618 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ec3492db0 0x555ec349c530 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:17.734 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.726+0000 7f6f571a5740 1 -- 172.21.15.100:0/19869618 >> 172.21.15.100:0/19869618 conn(0x555ec348e420 msgr2=0x555ec34aae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:33:17.734 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.726+0000 7f6f571a5740 1 -- 172.21.15.100:0/19869618 shutdown_connections 2024-05-10T11:33:17.734 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.726+0000 7f6f571a5740 1 -- 172.21.15.100:0/19869618 wait complete. 2024-05-10T11:33:17.734 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.726+0000 7f6f571a5740 1 -- start start 2024-05-10T11:33:17.735 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.727+0000 7f6f571a5740 1 Processor -- start 2024-05-10T11:33:17.735 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.727+0000 7f6f571a5740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ec3492db0 0x555ec34a1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:17.735 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.727+0000 7f6f571a5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ec34937a0 0x555ec34a22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:17.736 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.727+0000 7f6f571a5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ec3498910 0x555ec34a3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:17.736 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.727+0000 7f6f571a5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4c1a7700 con 0x555ec3498910 2024-05-10T11:33:17.736 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.727+0000 7f6f571a5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4c1a8b70 con 0x555ec34937a0 2024-05-10T11:33:17.736 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.727+0000 7f6f571a5740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4c1a7dd0 con 0x555ec3492db0 2024-05-10T11:33:17.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.727+0000 7f6f571a5740 10 client.0 _ll_register_callbacks cb 0x7f6f3c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:33:17.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.727+0000 7f6f5294c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ec3498910 0x555ec34a3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:17.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.727+0000 7f6f5194a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ec34937a0 0x555ec34a22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:17.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.727+0000 7f6f5294c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ec3498910 0x555ec34a3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59466/0 (socket says 172.21.15.100:59466) 2024-05-10T11:33:17.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.727+0000 7f6f5294c640 1 -- 172.21.15.100:0/3338755135 learned_addr learned my addr 172.21.15.100:0/3338755135 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:33:17.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.727+0000 7f6f5294c640 1 --2- 172.21.15.100:0/3338755135 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ec3492db0 0x555ec34a1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:17.738 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.727+0000 7f6f5194a640 1 -- 172.21.15.100:0/3338755135 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ec3492db0 msgr2=0x555ec34a1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:17.738 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.727+0000 7f6f5194a640 1 --2- 172.21.15.100:0/3338755135 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ec3492db0 0x555ec34a1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:17.738 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.727+0000 7f6f5194a640 1 -- 172.21.15.100:0/3338755135 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ec3498910 msgr2=0x555ec34a3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:17.738 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.727+0000 7f6f5194a640 1 --2- 172.21.15.100:0/3338755135 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ec3498910 0x555ec34a3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:17.738 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.727+0000 7f6f5194a640 1 -- 172.21.15.100:0/3338755135 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ec3489a70 con 0x555ec34937a0 2024-05-10T11:33:17.738 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.727+0000 7f6f5194a640 1 --2- 172.21.15.100:0/3338755135 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ec34937a0 0x555ec34a22f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c006010 tx=0x7f6f3c03bc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3dff5ed499592b77 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:33:17.738 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.727+0000 7f6f35ffb640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:33:17.738 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.728+0000 7f6f35ffb640 1 -- 172.21.15.100:0/3338755135 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c035030 con 0x555ec34937a0 2024-05-10T11:33:17.739 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.728+0000 7f6f571a5740 1 -- 172.21.15.100:0/3338755135 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f440046c0 con 0x555ec34937a0 2024-05-10T11:33:17.739 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.728+0000 7f6f35ffb640 1 -- 172.21.15.100:0/3338755135 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f6f3c0465f0 con 0x555ec34937a0 2024-05-10T11:33:17.739 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.728+0000 7f6f35ffb640 1 -- 172.21.15.100:0/3338755135 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f6f3c066050 con 0x555ec34937a0 2024-05-10T11:33:17.739 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.728+0000 7f6f571a5740 10 client.27746 fetch_fsmap learned FSMap version 37 2024-05-10T11:33:17.739 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.728+0000 7f6f571a5740 1 -- 172.21.15.100:0/3338755135 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x555ec3513680 con 0x555ec34937a0 2024-05-10T11:33:17.739 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.728+0000 7f6f35ffb640 1 -- 172.21.15.100:0/3338755135 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6f440046c0 con 0x555ec34937a0 2024-05-10T11:33:17.739 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.728+0000 7f6f35ffb640 1 -- 172.21.15.100:0/3338755135 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f6f3c04e040 con 0x555ec34937a0 2024-05-10T11:33:17.739 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.728+0000 7f6f571a5740 10 client.27746 fetch_fsmap finished waiting for FSMap version 37 2024-05-10T11:33:17.739 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.728+0000 7f6f571a5740 10 client.27746 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:33:17.740 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.728+0000 7f6f571a5740 20 client.27746 populate_metadata read hostname 'smithi100' 2024-05-10T11:33:17.740 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.728+0000 7f6f571a5740 1 --2- 172.21.15.100:0/3338755135 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x555ec3514030 0x555ec3516450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:17.740 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.728+0000 7f6f571a5740 4 client.27746 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:33:17.740 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.728+0000 7f6f571a5740 1 -- 172.21.15.100:0/3338755135 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555ec349cd00 con 0x555ec3514030 2024-05-10T11:33:17.740 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.728+0000 7f6f5294c640 1 --2- 172.21.15.100:0/3338755135 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x555ec3514030 0x555ec3516450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:17.740 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.729+0000 7f6f5294c640 1 --2- 172.21.15.100:0/3338755135 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x555ec3514030 0x555ec3516450 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15f53df75799e838 server_cookie=91c94abea57177a6 in_seq=0 out_seq=0 2024-05-10T11:33:17.740 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.729+0000 7f6f35ffb640 10 client.27746 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:33:17.887 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.886+0000 7f6f35ffb640 1 -- 172.21.15.100:0/3338755135 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555ec349cd00 con 0x555ec3514030 2024-05-10T11:33:17.887 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.886+0000 7f6f35ffb640 10 client.27746 handle_command_reply: tid=0 2024-05-10T11:33:17.887 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.886+0000 7f6f571a5740 1 client.27746 shutdown 2024-05-10T11:33:17.887 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.886+0000 7f6f571a5740 1 -- 172.21.15.100:0/3338755135 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ec34937a0 msgr2=0x555ec34a22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:17.887 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.886+0000 7f6f571a5740 1 --2- 172.21.15.100:0/3338755135 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ec34937a0 0x555ec34a22f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c006010 tx=0x7f6f3c03bc50 comp rx=0 tx=0).stop 2024-05-10T11:33:17.887 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.886+0000 7f6f571a5740 1 -- 172.21.15.100:0/3338755135 shutdown_connections 2024-05-10T11:33:17.887 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.886+0000 7f6f571a5740 1 --2- 172.21.15.100:0/3338755135 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x555ec3514030 0x555ec3516450 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:17.887 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.886+0000 7f6f571a5740 1 --2- 172.21.15.100:0/3338755135 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ec3498910 0x555ec34a3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:17.888 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.886+0000 7f6f571a5740 1 --2- 172.21.15.100:0/3338755135 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ec34937a0 0x555ec34a22f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:17.888 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.886+0000 7f6f571a5740 1 --2- 172.21.15.100:0/3338755135 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ec3492db0 0x555ec34a1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:17.888 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.886+0000 7f6f35ffb640 0 client.27746 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:33:17.888 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.886+0000 7f6f571a5740 1 -- 172.21.15.100:0/3338755135 >> 172.21.15.100:0/3338755135 conn(0x555ec348e420 msgr2=0x555ec34aae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:33:17.888 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.887+0000 7f6f571a5740 1 -- 172.21.15.100:0/3338755135 shutdown_connections 2024-05-10T11:33:17.888 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.887+0000 7f6f571a5740 1 -- 172.21.15.100:0/3338755135 wait complete. 2024-05-10T11:33:17.888 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.887+0000 7f6f571a5740 20 client.27746 trim_cache size 0 max 16384 2024-05-10T11:33:17.888 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:33:17.888 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:33:17.888 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:33:17.888 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:33:17.889 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.887+0000 7f6f543af640 1 -- 172.21.15.100:0/3440525003 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f6f380b28a0 msgr2=0x7f6f380b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:17.889 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.887+0000 7f6f543af640 1 --2- 172.21.15.100:0/3440525003 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f6f380b28a0 0x7f6f380b4d60 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f6f480e0200 tx=0x7f6f48023650 comp rx=0 tx=0).stop 2024-05-10T11:33:17.889 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.887+0000 7f6f543af640 1 -- 172.21.15.100:0/3440525003 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f4c102a20 msgr2=0x7f6f4c198ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:17.889 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.887+0000 7f6f543af640 1 --2- 172.21.15.100:0/3440525003 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f4c102a20 0x7f6f4c198ac0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6f4800e9c0 tx=0x7f6f4800ee80 comp rx=0 tx=0).stop 2024-05-10T11:33:17.889 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.887+0000 7f6f543af640 1 -- 172.21.15.100:0/3440525003 shutdown_connections 2024-05-10T11:33:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.887+0000 7f6f543af640 1 --2- 172.21.15.100:0/3440525003 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f6f380b28a0 0x7f6f380b4d60 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.887+0000 7f6f543af640 1 --2- 172.21.15.100:0/3440525003 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f4c1045d0 0x7f6f4c1a0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.887+0000 7f6f543af640 1 --2- 172.21.15.100:0/3440525003 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f4c103c10 0x7f6f4c199000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.887+0000 7f6f543af640 1 --2- 172.21.15.100:0/3440525003 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f4c102a20 0x7f6f4c198ac0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.887+0000 7f6f543af640 1 -- 172.21.15.100:0/3440525003 >> 172.21.15.100:0/3440525003 conn(0x7f6f4c0fdf70 msgr2=0x7f6f4c106f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:33:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.888+0000 7f6f543af640 1 -- 172.21.15.100:0/3440525003 shutdown_connections 2024-05-10T11:33:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:17.888+0000 7f6f543af640 1 -- 172.21.15.100:0/3440525003 wait complete. 2024-05-10T11:33:17.908 INFO:tasks.workunit.client.0.smithi100.stdout: 10 12348 11.84 MB/sec warmup 71 sec latency 139.520 ms 2024-05-10T11:33:18.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:33:17 smithi178 ceph-mon[33359]: pgmap v868: 129 pgs: 129 active+clean; 368 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 4.8 MiB/s wr, 280 op/s 2024-05-10T11:33:18.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:33:17 smithi188 ceph-mon[34075]: pgmap v868: 129 pgs: 129 active+clean; 368 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 4.8 MiB/s wr, 280 op/s 2024-05-10T11:33:18.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:33:17 smithi100 ceph-mon[30296]: pgmap v868: 129 pgs: 129 active+clean; 368 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 4.8 MiB/s wr, 280 op/s 2024-05-10T11:33:18.910 INFO:tasks.workunit.client.0.smithi100.stdout: 10 12554 11.86 MB/sec warmup 72 sec latency 75.887 ms 2024-05-10T11:33:19.910 INFO:tasks.workunit.client.0.smithi100.stdout: 10 12715 11.81 MB/sec warmup 73 sec latency 94.087 ms 2024-05-10T11:33:20.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:33:19 smithi178 ceph-mon[33359]: pgmap v869: 129 pgs: 129 active+clean; 369 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.9 MiB/s wr, 319 op/s 2024-05-10T11:33:20.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:33:19 smithi188 ceph-mon[34075]: pgmap v869: 129 pgs: 129 active+clean; 369 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.9 MiB/s wr, 319 op/s 2024-05-10T11:33:20.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:33:19 smithi100 ceph-mon[30296]: pgmap v869: 129 pgs: 129 active+clean; 369 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.9 MiB/s wr, 319 op/s 2024-05-10T11:33:20.910 INFO:tasks.workunit.client.0.smithi100.stdout: 10 12889 11.75 MB/sec warmup 74 sec latency 60.336 ms 2024-05-10T11:33:21.910 INFO:tasks.workunit.client.0.smithi100.stdout: 10 13088 11.72 MB/sec warmup 75 sec latency 74.314 ms 2024-05-10T11:33:22.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:33:21 smithi178 ceph-mon[33359]: pgmap v870: 129 pgs: 129 active+clean; 371 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 4.9 MiB/s wr, 281 op/s 2024-05-10T11:33:22.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:33:21 smithi188 ceph-mon[34075]: pgmap v870: 129 pgs: 129 active+clean; 371 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 4.9 MiB/s wr, 281 op/s 2024-05-10T11:33:22.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:33:21 smithi100 ceph-mon[30296]: pgmap v870: 129 pgs: 129 active+clean; 371 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 4.9 MiB/s wr, 281 op/s 2024-05-10T11:33:22.910 INFO:tasks.workunit.client.0.smithi100.stdout: 10 13274 11.65 MB/sec warmup 76 sec latency 78.982 ms 2024-05-10T11:33:23.910 INFO:tasks.workunit.client.0.smithi100.stdout: 10 13469 11.62 MB/sec warmup 77 sec latency 87.327 ms 2024-05-10T11:33:24.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:33:23 smithi178 ceph-mon[33359]: pgmap v871: 129 pgs: 129 active+clean; 369 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 908 KiB/s rd, 4.9 MiB/s wr, 306 op/s 2024-05-10T11:33:24.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:33:23 smithi188 ceph-mon[34075]: pgmap v871: 129 pgs: 129 active+clean; 369 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 908 KiB/s rd, 4.9 MiB/s wr, 306 op/s 2024-05-10T11:33:24.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:33:23 smithi100 ceph-mon[30296]: pgmap v871: 129 pgs: 129 active+clean; 369 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 908 KiB/s rd, 4.9 MiB/s wr, 306 op/s 2024-05-10T11:33:24.910 INFO:tasks.workunit.client.0.smithi100.stdout: 10 13669 11.66 MB/sec warmup 78 sec latency 87.789 ms 2024-05-10T11:33:25.910 INFO:tasks.workunit.client.0.smithi100.stdout: 10 13832 11.56 MB/sec warmup 79 sec latency 100.494 ms 2024-05-10T11:33:26.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:33:25 smithi178 ceph-mon[33359]: pgmap v872: 129 pgs: 129 active+clean; 373 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 5.0 MiB/s wr, 329 op/s 2024-05-10T11:33:26.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:33:25 smithi188 ceph-mon[34075]: pgmap v872: 129 pgs: 129 active+clean; 373 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 5.0 MiB/s wr, 329 op/s 2024-05-10T11:33:26.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:33:25 smithi100 ceph-mon[30296]: pgmap v872: 129 pgs: 129 active+clean; 373 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 5.0 MiB/s wr, 329 op/s 2024-05-10T11:33:26.910 INFO:tasks.workunit.client.0.smithi100.stdout: 10 14024 11.56 MB/sec warmup 80 sec latency 72.907 ms 2024-05-10T11:33:27.910 INFO:tasks.workunit.client.0.smithi100.stdout: 10 14215 11.50 MB/sec warmup 81 sec latency 61.001 ms 2024-05-10T11:33:28.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:33:27 smithi178 ceph-mon[33359]: pgmap v873: 129 pgs: 129 active+clean; 369 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 4.8 MiB/s wr, 327 op/s 2024-05-10T11:33:28.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:33:27 smithi188 ceph-mon[34075]: pgmap v873: 129 pgs: 129 active+clean; 369 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 4.8 MiB/s wr, 327 op/s 2024-05-10T11:33:28.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:33:27 smithi100 ceph-mon[30296]: pgmap v873: 129 pgs: 129 active+clean; 369 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 4.8 MiB/s wr, 327 op/s 2024-05-10T11:33:28.910 INFO:tasks.workunit.client.0.smithi100.stdout: 10 14455 11.54 MB/sec warmup 82 sec latency 63.276 ms 2024-05-10T11:33:29.911 INFO:tasks.workunit.client.0.smithi100.stdout: 10 14645 11.52 MB/sec warmup 83 sec latency 53.780 ms 2024-05-10T11:33:30.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:33:29 smithi178 ceph-mon[33359]: pgmap v874: 129 pgs: 129 active+clean; 369 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1016 KiB/s rd, 4.5 MiB/s wr, 342 op/s 2024-05-10T11:33:30.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:33:29 smithi188 ceph-mon[34075]: pgmap v874: 129 pgs: 129 active+clean; 369 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1016 KiB/s rd, 4.5 MiB/s wr, 342 op/s 2024-05-10T11:33:30.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:33:29 smithi100 ceph-mon[30296]: pgmap v874: 129 pgs: 129 active+clean; 369 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1016 KiB/s rd, 4.5 MiB/s wr, 342 op/s 2024-05-10T11:33:30.910 INFO:tasks.workunit.client.0.smithi100.stdout: 10 14835 11.58 MB/sec warmup 84 sec latency 75.015 ms 2024-05-10T11:33:31.021 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.020+0000 7f46c1c1c640 1 -- 172.21.15.100:0/2482209619 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46bc102b00 msgr2=0x7f46bc102f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:31.022 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.020+0000 7f46c1c1c640 1 --2- 172.21.15.100:0/2482209619 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46bc102b00 0x7f46bc102f80 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f46b0009900 tx=0x7f46b002a410 comp rx=0 tx=0).stop 2024-05-10T11:33:31.022 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.020+0000 7f46c1c1c640 1 -- 172.21.15.100:0/2482209619 shutdown_connections 2024-05-10T11:33:31.022 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.020+0000 7f46c1c1c640 1 --2- 172.21.15.100:0/2482209619 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46bc1034c0 0x7f46bc109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:31.022 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.020+0000 7f46c1c1c640 1 --2- 172.21.15.100:0/2482209619 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46bc102b00 0x7f46bc102f80 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:31.022 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.020+0000 7f46c1c1c640 1 --2- 172.21.15.100:0/2482209619 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46bc101910 0x7f46bc101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:31.022 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.020+0000 7f46c1c1c640 1 -- 172.21.15.100:0/2482209619 >> 172.21.15.100:0/2482209619 conn(0x7f46bc0fce80 msgr2=0x7f46bc0ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:33:31.022 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.020+0000 7f46c1c1c640 1 -- 172.21.15.100:0/2482209619 shutdown_connections 2024-05-10T11:33:31.022 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.020+0000 7f46c1c1c640 1 -- 172.21.15.100:0/2482209619 wait complete. 2024-05-10T11:33:31.022 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.021+0000 7f46c1c1c640 1 Processor -- start 2024-05-10T11:33:31.024 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.021+0000 7f46c1c1c640 1 -- start start 2024-05-10T11:33:31.025 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.021+0000 7f46c1c1c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46bc101910 0x7f46bc197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:31.025 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.021+0000 7f46c1c1c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46bc102b00 0x7f46bc197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:31.025 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.021+0000 7f46c1c1c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46bc1034c0 0x7f46bc19efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:31.025 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.021+0000 7f46c1c1c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f46bc10d2f0 con 0x7f46bc101910 2024-05-10T11:33:31.025 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.021+0000 7f46c1c1c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f46bc10d170 con 0x7f46bc102b00 2024-05-10T11:33:31.025 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.021+0000 7f46c1c1c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f46bc10d470 con 0x7f46bc1034c0 2024-05-10T11:33:31.025 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.021+0000 7f46bb7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46bc101910 0x7f46bc197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:31.025 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.021+0000 7f46bb7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46bc101910 0x7f46bc197a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52970/0 (socket says 172.21.15.100:52970) 2024-05-10T11:33:31.025 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.021+0000 7f46bb7fe640 1 -- 172.21.15.100:0/781796691 learned_addr learned my addr 172.21.15.100:0/781796691 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:33:31.025 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.021+0000 7f46baffd640 1 --2- 172.21.15.100:0/781796691 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46bc102b00 0x7f46bc197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:31.025 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.022+0000 7f46bbfff640 1 --2- 172.21.15.100:0/781796691 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46bc1034c0 0x7f46bc19efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:31.025 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.022+0000 7f46bb7fe640 1 -- 172.21.15.100:0/781796691 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46bc1034c0 msgr2=0x7f46bc19efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:31.025 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.022+0000 7f46bb7fe640 1 --2- 172.21.15.100:0/781796691 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46bc1034c0 0x7f46bc19efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:31.025 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.022+0000 7f46bb7fe640 1 -- 172.21.15.100:0/781796691 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46bc102b00 msgr2=0x7f46bc197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:31.026 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.022+0000 7f46bb7fe640 1 --2- 172.21.15.100:0/781796691 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46bc102b00 0x7f46bc197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:31.026 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.022+0000 7f46bb7fe640 1 -- 172.21.15.100:0/781796691 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46bc06a160 con 0x7f46bc101910 2024-05-10T11:33:31.026 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.022+0000 7f46baffd640 1 --2- 172.21.15.100:0/781796691 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46bc102b00 0x7f46bc197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:33:31.026 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.022+0000 7f46bb7fe640 1 --2- 172.21.15.100:0/781796691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46bc101910 0x7f46bc197a10 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f46ac00e9c0 tx=0x7f46ac00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:33:31.026 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.022+0000 7f46b8ff9640 1 -- 172.21.15.100:0/781796691 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ac00cde0 con 0x7f46bc101910 2024-05-10T11:33:31.026 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.022+0000 7f46b8ff9640 1 -- 172.21.15.100:0/781796691 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f46ac0108c0 con 0x7f46bc101910 2024-05-10T11:33:31.026 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.022+0000 7f46c1c1c640 1 -- 172.21.15.100:0/781796691 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46bc06a450 con 0x7f46bc101910 2024-05-10T11:33:31.026 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.022+0000 7f46b8ff9640 1 -- 172.21.15.100:0/781796691 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ac018870 con 0x7f46bc101910 2024-05-10T11:33:31.027 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.022+0000 7f46c1c1c640 1 -- 172.21.15.100:0/781796691 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46bc06a9c0 con 0x7f46bc101910 2024-05-10T11:33:31.029 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.025+0000 7f46c4a19740 1 Processor -- start 2024-05-10T11:33:31.029 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.025+0000 7f46b8ff9640 1 -- 172.21.15.100:0/781796691 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f46ac01f020 con 0x7f46bc101910 2024-05-10T11:33:31.029 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.026+0000 7f46c4a19740 1 -- start start 2024-05-10T11:33:31.029 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.026+0000 7f46c4a19740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c7037695c0 0x55c7037699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:31.029 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.026+0000 7f46c4a19740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c703769fb0 0x55c70376eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.026+0000 7f46c4a19740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c70376f120 0x55c7037715b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.026+0000 7f46c4a19740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55c7036c55d0 con 0x55c703769fb0 2024-05-10T11:33:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.026+0000 7f46c4a19740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c70367e890 con 0x55c7037695c0 2024-05-10T11:33:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.026+0000 7f46c4a19740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c7036c1590 con 0x55c70376f120 2024-05-10T11:33:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.026+0000 7f46bbfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c7037695c0 0x55c7037699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.026+0000 7f46bbfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c7037695c0 0x55c7037699e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:42688/0 (socket says 172.21.15.100:42688) 2024-05-10T11:33:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.026+0000 7f46bbfff640 1 -- 172.21.15.100:0/4258385074 learned_addr learned my addr 172.21.15.100:0/4258385074 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:33:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.026+0000 7f46baffd640 1 --2- 172.21.15.100:0/4258385074 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c70376f120 0x55c7037715b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.026+0000 7f46bb7fe640 1 --2- 172.21.15.100:0/4258385074 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c703769fb0 0x55c70376eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.026+0000 7f46bbfff640 1 -- 172.21.15.100:0/4258385074 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c70376f120 msgr2=0x55c7037715b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.026+0000 7f46bbfff640 1 --2- 172.21.15.100:0/4258385074 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c70376f120 0x55c7037715b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.026+0000 7f46bbfff640 1 -- 172.21.15.100:0/4258385074 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c703769fb0 msgr2=0x55c70376eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:31.031 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.026+0000 7f46bbfff640 1 --2- 172.21.15.100:0/4258385074 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c703769fb0 0x55c70376eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:31.031 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.026+0000 7f46bbfff640 1 -- 172.21.15.100:0/4258385074 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7033d7210 con 0x55c7037695c0 2024-05-10T11:33:31.031 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.026+0000 7f46baffd640 1 --2- 172.21.15.100:0/4258385074 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c70376f120 0x55c7037715b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:33:31.031 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.027+0000 7f46bbfff640 1 --2- 172.21.15.100:0/4258385074 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c7037695c0 0x55c7037699e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f46b4009fd0 tx=0x7f46b40118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a584c442c8d23eb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:33:31.031 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.027+0000 7f46a1ffb640 1 -- 172.21.15.100:0/4258385074 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b4004d20 con 0x55c7037695c0 2024-05-10T11:33:31.031 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.027+0000 7f46a1ffb640 1 -- 172.21.15.100:0/4258385074 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f46b4004050 con 0x55c7037695c0 2024-05-10T11:33:31.031 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.027+0000 7f46a1ffb640 1 -- 172.21.15.100:0/4258385074 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b4005260 con 0x55c7037695c0 2024-05-10T11:33:31.031 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.027+0000 7f46b8ff9640 1 --2- 172.21.15.100:0/781796691 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f46a40b2920 0x7f46a40b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:31.031 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.027+0000 7f46bb7fe640 1 --2- 172.21.15.100:0/781796691 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f46a40b2920 0x7f46a40b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:31.031 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.028+0000 7f46b8ff9640 1 -- 172.21.15.100:0/781796691 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f46ac0e26d0 con 0x7f46bc101910 2024-05-10T11:33:31.031 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.028+0000 7f46c4a19740 1 -- 172.21.15.100:0/4258385074 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c7037695c0 msgr2=0x55c7037699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:31.031 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.028+0000 7f46c4a19740 1 --2- 172.21.15.100:0/4258385074 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c7037695c0 0x55c7037699e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f46b4009fd0 tx=0x7f46b40118d0 comp rx=0 tx=0).stop 2024-05-10T11:33:31.031 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.028+0000 7f46c4a19740 1 -- 172.21.15.100:0/4258385074 shutdown_connections 2024-05-10T11:33:31.031 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.028+0000 7f46bb7fe640 1 --2- 172.21.15.100:0/781796691 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f46a40b2920 0x7f46a40b4de0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f46ac080d40 tx=0x7f46ac0e8530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:33:31.031 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.028+0000 7f46c4a19740 1 --2- 172.21.15.100:0/4258385074 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c70376f120 0x55c7037715b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:31.032 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.028+0000 7f46c4a19740 1 --2- 172.21.15.100:0/4258385074 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c703769fb0 0x55c70376eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:31.032 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.028+0000 7f46c4a19740 1 --2- 172.21.15.100:0/4258385074 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c7037695c0 0x55c7037699e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:31.032 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.028+0000 7f46c4a19740 1 -- 172.21.15.100:0/4258385074 >> 172.21.15.100:0/4258385074 conn(0x55c70375c420 msgr2=0x55c70375c820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:33:31.032 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.028+0000 7f46c4a19740 1 -- 172.21.15.100:0/4258385074 shutdown_connections 2024-05-10T11:33:31.032 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.028+0000 7f46c4a19740 1 -- 172.21.15.100:0/4258385074 wait complete. 2024-05-10T11:33:31.032 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.028+0000 7f46c4a19740 1 -- start start 2024-05-10T11:33:31.034 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.029+0000 7f46c4a19740 1 Processor -- start 2024-05-10T11:33:31.034 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.029+0000 7f46c4a19740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c7037695c0 0x55c703773770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:31.035 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.029+0000 7f46c4a19740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c703769fb0 0x55c703773cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:31.035 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.029+0000 7f46c4a19740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c70376f120 0x55c70377acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:31.035 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.029+0000 7f46c4a19740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55c7037720d0 con 0x55c70376f120 2024-05-10T11:33:31.035 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.029+0000 7f46c4a19740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c703682370 con 0x55c7037695c0 2024-05-10T11:33:31.035 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.029+0000 7f46c4a19740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c703772250 con 0x55c703769fb0 2024-05-10T11:33:31.035 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.029+0000 7f46bbfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c70376f120 0x55c70377acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:31.035 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.029+0000 7f46c4a19740 10 client.0 _ll_register_callbacks cb 0x55c7033d6a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:33:31.035 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.029+0000 7f46bbfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c7037695c0 0x55c703773770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:31.035 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.029+0000 7f46bbfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c70376f120 0x55c70377acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52982/0 (socket says 172.21.15.100:52982) 2024-05-10T11:33:31.035 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.029+0000 7f46bbfff640 1 -- 172.21.15.100:0/3298987103 learned_addr learned my addr 172.21.15.100:0/3298987103 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:33:31.035 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.029+0000 7f46baffd640 1 --2- 172.21.15.100:0/3298987103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c703769fb0 0x55c703773cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:31.035 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.030+0000 7f46baffd640 1 -- 172.21.15.100:0/3298987103 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c7037695c0 msgr2=0x55c703773770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:31.035 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.030+0000 7f46baffd640 1 --2- 172.21.15.100:0/3298987103 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c7037695c0 0x55c703773770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:31.035 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.030+0000 7f46baffd640 1 -- 172.21.15.100:0/3298987103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c70376f120 msgr2=0x55c70377acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:31.035 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.030+0000 7f46baffd640 1 --2- 172.21.15.100:0/3298987103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c70376f120 0x55c70377acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.030+0000 7f46baffd640 1 -- 172.21.15.100:0/3298987103 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7034a3620 con 0x55c703769fb0 2024-05-10T11:33:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.030+0000 7f46bbfff640 1 --2- 172.21.15.100:0/3298987103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c70376f120 0x55c70377acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:33:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.030+0000 7f46baffd640 1 --2- 172.21.15.100:0/3298987103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c703769fb0 0x55c703773cb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f46b0005ab0 tx=0x7f46b0031ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=efffccfac5d979df server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:33:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.030+0000 7f469affd640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:33:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.030+0000 7f469affd640 1 -- 172.21.15.100:0/3298987103 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b0035040 con 0x55c703769fb0 2024-05-10T11:33:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.030+0000 7f469affd640 1 -- 172.21.15.100:0/3298987103 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f46b0004450 con 0x55c703769fb0 2024-05-10T11:33:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.030+0000 7f46c4a19740 1 -- 172.21.15.100:0/3298987103 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46b4004050 con 0x55c703769fb0 2024-05-10T11:33:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.031+0000 7f469affd640 1 -- 172.21.15.100:0/3298987103 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f46b00035f0 con 0x55c703769fb0 2024-05-10T11:33:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.031+0000 7f46c4a19740 10 client.27856 fetch_fsmap learned FSMap version 37 2024-05-10T11:33:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.031+0000 7f46c4a19740 1 -- 172.21.15.100:0/3298987103 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55c70377dd40 con 0x55c703769fb0 2024-05-10T11:33:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.031+0000 7f469affd640 1 -- 172.21.15.100:0/3298987103 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f46b4004050 con 0x55c703769fb0 2024-05-10T11:33:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.031+0000 7f469affd640 1 -- 172.21.15.100:0/3298987103 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f46b003c790 con 0x55c703769fb0 2024-05-10T11:33:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.031+0000 7f46c4a19740 10 client.27856 fetch_fsmap finished waiting for FSMap version 37 2024-05-10T11:33:31.037 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.032+0000 7f46c4a19740 10 client.27856 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:33:31.037 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.032+0000 7f46c4a19740 20 client.27856 populate_metadata read hostname 'smithi100' 2024-05-10T11:33:31.037 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.032+0000 7f46c4a19740 1 --2- 172.21.15.100:0/3298987103 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c70377f4d0 0x55c703781910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:31.037 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.032+0000 7f46c4a19740 4 client.27856 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:33:31.037 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.032+0000 7f46c4a19740 1 -- 172.21.15.100:0/3298987103 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7036d31e0 con 0x55c70377f4d0 2024-05-10T11:33:31.037 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.032+0000 7f46bbfff640 1 --2- 172.21.15.100:0/3298987103 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c70377f4d0 0x55c703781910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:31.037 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.033+0000 7f469affd640 10 client.27856 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:33:31.037 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.033+0000 7f46bbfff640 1 --2- 172.21.15.100:0/3298987103 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c70377f4d0 0x55c703781910 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a2904198cf6a58b server_cookie=30b0e129965a07bd in_seq=0 out_seq=0 2024-05-10T11:33:31.039 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.034+0000 7f469affd640 1 -- 172.21.15.100:0/3298987103 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c7036d31e0 con 0x55c70377f4d0 2024-05-10T11:33:31.039 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.034+0000 7f469affd640 10 client.27856 handle_command_reply: tid=0 2024-05-10T11:33:31.039 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.034+0000 7f46c4a19740 1 client.27856 shutdown 2024-05-10T11:33:31.039 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.034+0000 7f46c4a19740 1 -- 172.21.15.100:0/3298987103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c703769fb0 msgr2=0x55c703773cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:31.039 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.034+0000 7f46c4a19740 1 --2- 172.21.15.100:0/3298987103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c703769fb0 0x55c703773cb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f46b0005ab0 tx=0x7f46b0031ca0 comp rx=0 tx=0).stop 2024-05-10T11:33:31.039 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.034+0000 7f46c4a19740 1 -- 172.21.15.100:0/3298987103 shutdown_connections 2024-05-10T11:33:31.039 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.034+0000 7f46c4a19740 1 --2- 172.21.15.100:0/3298987103 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c70377f4d0 0x55c703781910 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:31.039 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.034+0000 7f469affd640 0 client.27856 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:33:31.039 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.035+0000 7f46c4a19740 1 --2- 172.21.15.100:0/3298987103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c70376f120 0x55c70377acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:31.039 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.035+0000 7f46c4a19740 1 --2- 172.21.15.100:0/3298987103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c703769fb0 0x55c703773cb0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:31.039 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.035+0000 7f46c4a19740 1 --2- 172.21.15.100:0/3298987103 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c7037695c0 0x55c703773770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:31.039 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.035+0000 7f46c4a19740 1 -- 172.21.15.100:0/3298987103 >> 172.21.15.100:0/3298987103 conn(0x55c70375c420 msgr2=0x55c70375b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:33:31.039 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.035+0000 7f46c4a19740 1 -- 172.21.15.100:0/3298987103 shutdown_connections 2024-05-10T11:33:31.039 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.035+0000 7f46c4a19740 1 -- 172.21.15.100:0/3298987103 wait complete. 2024-05-10T11:33:31.040 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.035+0000 7f46c4a19740 20 client.27856 trim_cache size 0 max 16384 2024-05-10T11:33:31.062 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.060+0000 7f46c4a19740 1 Processor -- start 2024-05-10T11:33:31.062 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.060+0000 7f46c4a19740 1 -- start start 2024-05-10T11:33:31.062 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.060+0000 7f46c4a19740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c7037695c0 0x55c70375acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:31.062 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.060+0000 7f46c4a19740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c703769fb0 0x55c70377acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.060+0000 7f46c4a19740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c70376f120 0x55c70375b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.060+0000 7f46c4a19740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f46bc1abad0 con 0x55c70376f120 2024-05-10T11:33:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.060+0000 7f46c4a19740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f46bc1a9310 con 0x55c7037695c0 2024-05-10T11:33:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.060+0000 7f46c4a19740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f46bc1a7160 con 0x55c703769fb0 2024-05-10T11:33:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.060+0000 7f46bbfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c70376f120 0x55c70375b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.060+0000 7f46bbfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c7037695c0 0x55c70375acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.060+0000 7f46baffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c703769fb0 0x55c70377acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.060+0000 7f46bbfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c7037695c0 0x55c70375acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:42714/0 (socket says 172.21.15.100:42714) 2024-05-10T11:33:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.060+0000 7f46baffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c703769fb0 0x55c70377acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:35326/0 (socket says 172.21.15.100:35326) 2024-05-10T11:33:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.060+0000 7f46bbfff640 1 -- 172.21.15.100:0/1267739092 learned_addr learned my addr 172.21.15.100:0/1267739092 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:33:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.060+0000 7f46bbfff640 1 -- 172.21.15.100:0/1267739092 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c703769fb0 msgr2=0x55c70377acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.060+0000 7f46bbfff640 1 --2- 172.21.15.100:0/1267739092 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c703769fb0 0x55c70377acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.060+0000 7f46bbfff640 1 -- 172.21.15.100:0/1267739092 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c70376f120 msgr2=0x55c70375b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:31.064 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.060+0000 7f46bbfff640 1 --2- 172.21.15.100:0/1267739092 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c70376f120 0x55c70375b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:31.064 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.060+0000 7f46bbfff640 1 -- 172.21.15.100:0/1267739092 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c703774800 con 0x55c7037695c0 2024-05-10T11:33:31.064 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.061+0000 7f46baffd640 1 --2- 172.21.15.100:0/1267739092 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c703769fb0 0x55c70377acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:33:31.064 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.061+0000 7f46bbfff640 1 --2- 172.21.15.100:0/1267739092 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c7037695c0 0x55c70375acf0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f46b401b040 tx=0x7f46b4005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c568939993c591da server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:33:31.064 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.061+0000 7f46a17fa640 1 -- 172.21.15.100:0/1267739092 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b402e050 con 0x55c7037695c0 2024-05-10T11:33:31.064 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.061+0000 7f46a17fa640 1 -- 172.21.15.100:0/1267739092 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f46b402e1f0 con 0x55c7037695c0 2024-05-10T11:33:31.064 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.061+0000 7f46a17fa640 1 -- 172.21.15.100:0/1267739092 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b400f740 con 0x55c7037695c0 2024-05-10T11:33:31.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.062+0000 7f46c4a19740 1 -- 172.21.15.100:0/1267739092 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c7037695c0 msgr2=0x55c70375acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:31.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.062+0000 7f46c4a19740 1 --2- 172.21.15.100:0/1267739092 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c7037695c0 0x55c70375acf0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f46b401b040 tx=0x7f46b4005fd0 comp rx=0 tx=0).stop 2024-05-10T11:33:31.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.062+0000 7f46c4a19740 1 -- 172.21.15.100:0/1267739092 shutdown_connections 2024-05-10T11:33:31.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.062+0000 7f46c4a19740 1 --2- 172.21.15.100:0/1267739092 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c70376f120 0x55c70375b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:31.068 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.062+0000 7f46c4a19740 1 --2- 172.21.15.100:0/1267739092 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c703769fb0 0x55c70377acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:31.068 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.062+0000 7f46c4a19740 1 --2- 172.21.15.100:0/1267739092 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c7037695c0 0x55c70375acf0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:31.068 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.062+0000 7f46c4a19740 1 -- 172.21.15.100:0/1267739092 >> 172.21.15.100:0/1267739092 conn(0x55c70375c420 msgr2=0x55c70376b130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:33:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.062+0000 7f46c4a19740 1 -- 172.21.15.100:0/1267739092 shutdown_connections 2024-05-10T11:33:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.062+0000 7f46c4a19740 1 -- 172.21.15.100:0/1267739092 wait complete. 2024-05-10T11:33:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.062+0000 7f46c4a19740 1 -- start start 2024-05-10T11:33:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.063+0000 7f46c4a19740 1 Processor -- start 2024-05-10T11:33:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.063+0000 7f46c4a19740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c7037695c0 0x55c703778090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.063+0000 7f46c4a19740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c703769fb0 0x55c7037785d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.063+0000 7f46c4a19740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c70376f120 0x55c703779540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.063+0000 7f46c4a19740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f46bc1a77d0 con 0x55c703769fb0 2024-05-10T11:33:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.063+0000 7f46c4a19740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f46bc1a8c40 con 0x55c7037695c0 2024-05-10T11:33:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.063+0000 7f46c4a19740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f46bc1a7ea0 con 0x55c70376f120 2024-05-10T11:33:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.063+0000 7f46c4a19740 10 client.0 _ll_register_callbacks cb 0x7f46b0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:33:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.063+0000 7f46baffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c703769fb0 0x55c7037785d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.063+0000 7f46baffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c703769fb0 0x55c7037785d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52986/0 (socket says 172.21.15.100:52986) 2024-05-10T11:33:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.063+0000 7f46baffd640 1 -- 172.21.15.100:0/2677975865 learned_addr learned my addr 172.21.15.100:0/2677975865 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:33:31.070 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.063+0000 7f46bbfff640 1 --2- 172.21.15.100:0/2677975865 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c7037695c0 0x55c703778090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:31.070 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.063+0000 7f46bbfff640 1 --2- 172.21.15.100:0/2677975865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c70376f120 0x55c703779540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:31.070 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.063+0000 7f46baffd640 1 -- 172.21.15.100:0/2677975865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c70376f120 msgr2=0x55c703779540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:31.070 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.064+0000 7f46baffd640 1 --2- 172.21.15.100:0/2677975865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c70376f120 0x55c703779540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:31.070 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.064+0000 7f46baffd640 1 -- 172.21.15.100:0/2677975865 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c7037695c0 msgr2=0x55c703778090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:31.070 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.064+0000 7f46baffd640 1 --2- 172.21.15.100:0/2677975865 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c7037695c0 0x55c703778090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:31.070 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.064+0000 7f46baffd640 1 -- 172.21.15.100:0/2677975865 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c703779b30 con 0x55c703769fb0 2024-05-10T11:33:31.070 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.064+0000 7f46baffd640 1 --2- 172.21.15.100:0/2677975865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c703769fb0 0x55c7037785d0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f46b0009310 tx=0x7f46b0004480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c1714988e37b4b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:33:31.070 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.064+0000 7f469affd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:33:31.070 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.064+0000 7f469affd640 1 -- 172.21.15.100:0/2677975865 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b0035070 con 0x55c703769fb0 2024-05-10T11:33:31.071 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.064+0000 7f469affd640 1 -- 172.21.15.100:0/2677975865 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f46b003c650 con 0x55c703769fb0 2024-05-10T11:33:31.071 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.064+0000 7f46c4a19740 1 -- 172.21.15.100:0/2677975865 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46b402e1f0 con 0x55c703769fb0 2024-05-10T11:33:31.071 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.064+0000 7f469affd640 1 -- 172.21.15.100:0/2677975865 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f46b004e030 con 0x55c703769fb0 2024-05-10T11:33:31.071 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.064+0000 7f46c4a19740 10 client.18429 fetch_fsmap learned FSMap version 37 2024-05-10T11:33:31.071 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.064+0000 7f46c4a19740 1 -- 172.21.15.100:0/2677975865 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55c70377aee0 con 0x55c703769fb0 2024-05-10T11:33:31.071 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.065+0000 7f469affd640 1 -- 172.21.15.100:0/2677975865 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f46b402e1f0 con 0x55c703769fb0 2024-05-10T11:33:31.071 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.065+0000 7f469affd640 1 -- 172.21.15.100:0/2677975865 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f46b0003c20 con 0x55c703769fb0 2024-05-10T11:33:31.071 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.065+0000 7f46c4a19740 10 client.18429 fetch_fsmap finished waiting for FSMap version 37 2024-05-10T11:33:31.071 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.065+0000 7f46c4a19740 10 client.18429 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:33:31.071 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.065+0000 7f46c4a19740 20 client.18429 populate_metadata read hostname 'smithi100' 2024-05-10T11:33:31.071 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.065+0000 7f46c4a19740 1 --2- 172.21.15.100:0/2677975865 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c70377f4d0 0x55c7037ec530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:31.071 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.065+0000 7f46c4a19740 4 client.18429 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:33:31.071 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.065+0000 7f46c4a19740 1 -- 172.21.15.100:0/2677975865 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c703769d70 con 0x55c70377f4d0 2024-05-10T11:33:31.071 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.065+0000 7f46bbfff640 1 --2- 172.21.15.100:0/2677975865 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c70377f4d0 0x55c7037ec530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:31.071 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.065+0000 7f46bbfff640 1 --2- 172.21.15.100:0/2677975865 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c70377f4d0 0x55c7037ec530 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82058c92ec8beac2 server_cookie=67637285c65b1d62 in_seq=0 out_seq=0 2024-05-10T11:33:31.072 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.065+0000 7f469affd640 10 client.18429 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:33:31.086 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T11:33:31.151 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.150+0000 7f469affd640 1 -- 172.21.15.100:0/2677975865 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c703769d70 con 0x55c70377f4d0 2024-05-10T11:33:31.151 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.150+0000 7f469affd640 10 client.18429 handle_command_reply: tid=0 2024-05-10T11:33:31.151 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.150+0000 7f46c4a19740 1 client.18429 shutdown 2024-05-10T11:33:31.152 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.150+0000 7f46c4a19740 1 -- 172.21.15.100:0/2677975865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c703769fb0 msgr2=0x55c7037785d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:31.152 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.150+0000 7f46c4a19740 1 --2- 172.21.15.100:0/2677975865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c703769fb0 0x55c7037785d0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f46b0009310 tx=0x7f46b0004480 comp rx=0 tx=0).stop 2024-05-10T11:33:31.152 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.151+0000 7f46c4a19740 1 -- 172.21.15.100:0/2677975865 shutdown_connections 2024-05-10T11:33:31.152 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.151+0000 7f46c4a19740 1 --2- 172.21.15.100:0/2677975865 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c70377f4d0 0x55c7037ec530 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:31.152 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.151+0000 7f46c4a19740 1 --2- 172.21.15.100:0/2677975865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c70376f120 0x55c703779540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:31.153 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.151+0000 7f46c4a19740 1 --2- 172.21.15.100:0/2677975865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c703769fb0 0x55c7037785d0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:31.153 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.151+0000 7f469affd640 0 client.18429 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:33:31.153 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.151+0000 7f46c4a19740 1 --2- 172.21.15.100:0/2677975865 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c7037695c0 0x55c703778090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:31.153 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.151+0000 7f46c4a19740 1 -- 172.21.15.100:0/2677975865 >> 172.21.15.100:0/2677975865 conn(0x55c70375c420 msgr2=0x55c70375f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:33:31.153 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.151+0000 7f46c4a19740 1 -- 172.21.15.100:0/2677975865 shutdown_connections 2024-05-10T11:33:31.153 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.151+0000 7f46c4a19740 1 -- 172.21.15.100:0/2677975865 wait complete. 2024-05-10T11:33:31.153 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.151+0000 7f46c4a19740 20 client.18429 trim_cache size 0 max 16384 2024-05-10T11:33:31.153 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:33:31.153 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:33:31.153 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:33:31.153 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:33:31.154 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.151+0000 7f46c1c1c640 1 -- 172.21.15.100:0/781796691 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f46a40b2920 msgr2=0x7f46a40b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:31.154 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.151+0000 7f46c1c1c640 1 --2- 172.21.15.100:0/781796691 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f46a40b2920 0x7f46a40b4de0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f46ac080d40 tx=0x7f46ac0e8530 comp rx=0 tx=0).stop 2024-05-10T11:33:31.154 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.151+0000 7f46c1c1c640 1 -- 172.21.15.100:0/781796691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46bc101910 msgr2=0x7f46bc197a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:31.154 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.151+0000 7f46c1c1c640 1 --2- 172.21.15.100:0/781796691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46bc101910 0x7f46bc197a10 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f46ac00e9c0 tx=0x7f46ac00ee80 comp rx=0 tx=0).stop 2024-05-10T11:33:31.154 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.151+0000 7f46c1c1c640 1 -- 172.21.15.100:0/781796691 shutdown_connections 2024-05-10T11:33:31.154 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.151+0000 7f46c1c1c640 1 --2- 172.21.15.100:0/781796691 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f46a40b2920 0x7f46a40b4de0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:31.155 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.151+0000 7f46c1c1c640 1 --2- 172.21.15.100:0/781796691 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46bc1034c0 0x7f46bc19efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:31.155 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.151+0000 7f46c1c1c640 1 --2- 172.21.15.100:0/781796691 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46bc102b00 0x7f46bc197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:31.155 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.151+0000 7f46c1c1c640 1 --2- 172.21.15.100:0/781796691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46bc101910 0x7f46bc197a10 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:31.155 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.151+0000 7f46c1c1c640 1 -- 172.21.15.100:0/781796691 >> 172.21.15.100:0/781796691 conn(0x7f46bc0fce80 msgr2=0x7f46bc105f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:33:31.155 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.152+0000 7f46c1c1c640 1 -- 172.21.15.100:0/781796691 shutdown_connections 2024-05-10T11:33:31.155 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:31.152+0000 7f46c1c1c640 1 -- 172.21.15.100:0/781796691 wait complete. 2024-05-10T11:33:31.472 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:33:31.910 INFO:tasks.workunit.client.0.smithi100.stdout: 10 15034 11.57 MB/sec warmup 85 sec latency 115.725 ms 2024-05-10T11:33:32.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:33:31 smithi178 ceph-mon[33359]: pgmap v875: 129 pgs: 129 active+clean; 372 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 4.4 MiB/s wr, 303 op/s 2024-05-10T11:33:32.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:33:31 smithi188 ceph-mon[34075]: pgmap v875: 129 pgs: 129 active+clean; 372 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 4.4 MiB/s wr, 303 op/s 2024-05-10T11:33:32.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:33:31 smithi100 ceph-mon[30296]: pgmap v875: 129 pgs: 129 active+clean; 372 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 4.4 MiB/s wr, 303 op/s 2024-05-10T11:33:32.910 INFO:tasks.workunit.client.0.smithi100.stdout: 10 15194 11.56 MB/sec warmup 86 sec latency 161.444 ms 2024-05-10T11:33:33.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:33:32 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:33:33.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:33:32 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:33:33.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:33:32 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:33:33.910 INFO:tasks.workunit.client.0.smithi100.stdout: 10 15319 11.49 MB/sec warmup 87 sec latency 79.536 ms 2024-05-10T11:33:34.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:33:33 smithi178 ceph-mon[33359]: pgmap v876: 129 pgs: 129 active+clean; 375 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 4.4 MiB/s wr, 311 op/s 2024-05-10T11:33:34.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:33:33 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:33:34.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:33:33 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:33:34.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:33:33 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:33:34.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:33:33 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:33:34.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:33:33 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:33:34.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:33:33 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:33:34.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:33:33 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:33:34.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:33:33 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:33:34.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:33:33 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:33:34.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:33:33 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:33:34.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:33:33 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:33:34.273 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:33:33 smithi100 ceph-mon[30296]: pgmap v876: 129 pgs: 129 active+clean; 375 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 4.4 MiB/s wr, 311 op/s 2024-05-10T11:33:34.273 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:33:33 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:33:34.274 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:33:33 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:33:34.274 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:33:33 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:33:34.274 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:33:33 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:33:34.274 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:33:33 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:33:34.274 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:33:33 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:33:34.274 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:33:33 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:33:34.274 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:33:33 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:33:34.274 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:33:33 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:33:34.274 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:33:33 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:33:34.274 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:33:33 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:33:34.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:33:33 smithi188 ceph-mon[34075]: pgmap v876: 129 pgs: 129 active+clean; 375 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 4.4 MiB/s wr, 311 op/s 2024-05-10T11:33:34.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:33:33 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:33:34.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:33:33 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:33:34.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:33:33 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:33:34.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:33:33 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:33:34.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:33:33 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:33:34.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:33:33 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:33:34.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:33:33 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:33:34.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:33:33 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:33:34.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:33:33 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:33:34.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:33:33 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:33:34.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:33:33 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:33:34.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.320+0000 7f2479192700 1 -- 172.21.15.100:0/1508495725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2474108410 msgr2=0x7f24741087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:34.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.320+0000 7f2479192700 1 --2- 172.21.15.100:0/1508495725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2474108410 0x7f24741087f0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f2464009990 tx=0x7f2464009ca0 comp rx=0 tx=0).stop 2024-05-10T11:33:34.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.320+0000 7f2479192700 1 -- 172.21.15.100:0/1508495725 shutdown_connections 2024-05-10T11:33:34.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.320+0000 7f2479192700 1 --2- 172.21.15.100:0/1508495725 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2474102d60 0x7f247410f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:34.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.320+0000 7f2479192700 1 --2- 172.21.15.100:0/1508495725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2474108410 0x7f24741087f0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:34.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.320+0000 7f2479192700 1 --2- 172.21.15.100:0/1508495725 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24741023a0 0x7f2474102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:34.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.320+0000 7f2479192700 1 -- 172.21.15.100:0/1508495725 >> 172.21.15.100:0/1508495725 conn(0x7f2474079350 msgr2=0x7f2474079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:33:34.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.320+0000 7f2479192700 1 -- 172.21.15.100:0/1508495725 shutdown_connections 2024-05-10T11:33:34.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.320+0000 7f2479192700 1 -- 172.21.15.100:0/1508495725 wait complete. 2024-05-10T11:33:34.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.321+0000 7f2479192700 1 Processor -- start 2024-05-10T11:33:34.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.321+0000 7f2479192700 1 -- start start 2024-05-10T11:33:34.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.322+0000 7f2479192700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24741023a0 0x7f2474198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:34.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.322+0000 7f2479192700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2474102d60 0x7f2474198cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:34.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.322+0000 7f2479192700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2474108410 0x7f2474199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:34.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.322+0000 7f247259c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2474102d60 0x7f2474198cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:34.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.322+0000 7f2479192700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2474112500 con 0x7f2474108410 2024-05-10T11:33:34.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.322+0000 7f2479192700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2474112380 con 0x7f2474102d60 2024-05-10T11:33:34.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.322+0000 7f2479192700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2474112680 con 0x7f24741023a0 2024-05-10T11:33:34.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.322+0000 7f2472d9d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24741023a0 0x7f2474198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:34.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.322+0000 7f2472d9d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24741023a0 0x7f2474198790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:35352/0 (socket says 172.21.15.100:35352) 2024-05-10T11:33:34.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.322+0000 7f2472d9d700 1 -- 172.21.15.100:0/4261413738 learned_addr learned my addr 172.21.15.100:0/4261413738 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:33:34.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.322+0000 7f247259c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2474102d60 0x7f2474198cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:42732/0 (socket says 172.21.15.100:42732) 2024-05-10T11:33:34.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.322+0000 7f247359e700 1 --2- 172.21.15.100:0/4261413738 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2474108410 0x7f2474199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:34.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.323+0000 7f2472d9d700 1 -- 172.21.15.100:0/4261413738 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2474102d60 msgr2=0x7f2474198cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:34.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.323+0000 7f2472d9d700 1 --2- 172.21.15.100:0/4261413738 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2474102d60 0x7f2474198cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:34.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.323+0000 7f2472d9d700 1 -- 172.21.15.100:0/4261413738 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2474108410 msgr2=0x7f2474199210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:34.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.323+0000 7f2472d9d700 1 --2- 172.21.15.100:0/4261413738 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2474108410 0x7f2474199210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:34.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.323+0000 7f2472d9d700 1 -- 172.21.15.100:0/4261413738 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24741a8000 con 0x7f24741023a0 2024-05-10T11:33:34.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.323+0000 7f2472d9d700 1 --2- 172.21.15.100:0/4261413738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24741023a0 0x7f2474198790 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2464000c00 tx=0x7f24640047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:33:34.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.323+0000 7f245bfff700 1 -- 172.21.15.100:0/4261413738 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246400fa90 con 0x7f24741023a0 2024-05-10T11:33:34.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.323+0000 7f245bfff700 1 -- 172.21.15.100:0/4261413738 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f246400fc30 con 0x7f24741023a0 2024-05-10T11:33:34.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.323+0000 7f245bfff700 1 -- 172.21.15.100:0/4261413738 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2464017430 con 0x7f24741023a0 2024-05-10T11:33:34.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.323+0000 7f2479192700 1 -- 172.21.15.100:0/4261413738 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24741a8290 con 0x7f24741023a0 2024-05-10T11:33:34.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.324+0000 7f2479192700 1 -- 172.21.15.100:0/4261413738 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24741a86e0 con 0x7f24741023a0 2024-05-10T11:33:34.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.327+0000 7f245bfff700 1 -- 172.21.15.100:0/4261413738 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f2464009d10 con 0x7f24741023a0 2024-05-10T11:33:34.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.328+0000 7f245bfff700 1 --2- 172.21.15.100:0/4261413738 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f245c0b2940 0x7f245c0b4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:34.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.328+0000 7f247259c700 1 --2- 172.21.15.100:0/4261413738 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f245c0b2940 0x7f245c0b4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:34.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.328+0000 7f245bfff700 1 -- 172.21.15.100:0/4261413738 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f24640e3210 con 0x7f24741023a0 2024-05-10T11:33:34.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.329+0000 7f2483a25b80 1 Processor -- start 2024-05-10T11:33:34.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.329+0000 7f247259c700 1 --2- 172.21.15.100:0/4261413738 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f245c0b2940 0x7f245c0b4e00 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f247419a8f0 tx=0x7f2468009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:33:34.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.329+0000 7f2483a25b80 1 -- start start 2024-05-10T11:33:34.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.329+0000 7f2483a25b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5594d07381b0 0x5594d0738590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:34.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.329+0000 7f2483a25b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594d0738ad0 0x5594d0732810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:34.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.329+0000 7f2483a25b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594d0732d50 0x5594d0733190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:34.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.329+0000 7f2483a25b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5594d0684940 con 0x5594d0738ad0 2024-05-10T11:33:34.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.330+0000 7f2483a25b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5594d05e9da0 con 0x5594d07381b0 2024-05-10T11:33:34.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.330+0000 7f2483a25b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5594d0680900 con 0x5594d0732d50 2024-05-10T11:33:34.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.330+0000 7f247259c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594d0738ad0 0x5594d0732810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:34.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.330+0000 7f247259c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594d0738ad0 0x5594d0732810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53012/0 (socket says 172.21.15.100:53012) 2024-05-10T11:33:34.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.330+0000 7f247259c700 1 -- 172.21.15.100:0/1103013539 learned_addr learned my addr 172.21.15.100:0/1103013539 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:33:34.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.330+0000 7f2472d9d700 1 --2- 172.21.15.100:0/1103013539 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5594d07381b0 0x5594d0738590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:34.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.330+0000 7f247359e700 1 --2- 172.21.15.100:0/1103013539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594d0732d50 0x5594d0733190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:34.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.330+0000 7f247259c700 1 -- 172.21.15.100:0/1103013539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594d0732d50 msgr2=0x5594d0733190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:34.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.330+0000 7f247259c700 1 --2- 172.21.15.100:0/1103013539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594d0732d50 0x5594d0733190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:34.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.330+0000 7f247259c700 1 -- 172.21.15.100:0/1103013539 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5594d07381b0 msgr2=0x5594d0738590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:34.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.330+0000 7f247259c700 1 --2- 172.21.15.100:0/1103013539 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5594d07381b0 0x5594d0738590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:34.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.330+0000 7f247259c700 1 -- 172.21.15.100:0/1103013539 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594d05d18d0 con 0x5594d0738ad0 2024-05-10T11:33:34.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.330+0000 7f247259c700 1 --2- 172.21.15.100:0/1103013539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594d0738ad0 0x5594d0732810 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f2468009fd0 tx=0x7f2468011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19637509bba44e96 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:33:34.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.331+0000 7f24597fa700 1 -- 172.21.15.100:0/1103013539 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246801b040 con 0x5594d0738ad0 2024-05-10T11:33:34.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.331+0000 7f24597fa700 1 -- 172.21.15.100:0/1103013539 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2468024070 con 0x5594d0738ad0 2024-05-10T11:33:34.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.331+0000 7f24597fa700 1 -- 172.21.15.100:0/1103013539 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24680136b0 con 0x5594d0738ad0 2024-05-10T11:33:34.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.332+0000 7f2483a25b80 1 -- 172.21.15.100:0/1103013539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594d0738ad0 msgr2=0x5594d0732810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:34.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.332+0000 7f2483a25b80 1 --2- 172.21.15.100:0/1103013539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594d0738ad0 0x5594d0732810 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f2468009fd0 tx=0x7f2468011000 comp rx=0 tx=0).stop 2024-05-10T11:33:34.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.332+0000 7f2483a25b80 1 -- 172.21.15.100:0/1103013539 shutdown_connections 2024-05-10T11:33:34.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.332+0000 7f2483a25b80 1 --2- 172.21.15.100:0/1103013539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594d0732d50 0x5594d0733190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:34.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.332+0000 7f2483a25b80 1 --2- 172.21.15.100:0/1103013539 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5594d07381b0 0x5594d0738590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:34.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.332+0000 7f2483a25b80 1 --2- 172.21.15.100:0/1103013539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594d0738ad0 0x5594d0732810 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:34.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.332+0000 7f2483a25b80 1 -- 172.21.15.100:0/1103013539 >> 172.21.15.100:0/1103013539 conn(0x5594d065eb70 msgr2=0x5594d072d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:33:34.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.332+0000 7f2483a25b80 1 -- 172.21.15.100:0/1103013539 shutdown_connections 2024-05-10T11:33:34.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.332+0000 7f2483a25b80 1 -- 172.21.15.100:0/1103013539 wait complete. 2024-05-10T11:33:34.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.333+0000 7f2483a25b80 1 -- start start 2024-05-10T11:33:34.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.333+0000 7f2483a25b80 1 Processor -- start 2024-05-10T11:33:34.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.333+0000 7f2483a25b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594d0732d50 0x5594d073e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:34.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.334+0000 7f2483a25b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594d07381b0 0x5594d073baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:34.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.334+0000 7f2483a25b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5594d0738ad0 0x5594d073c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:34.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.334+0000 7f2472d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594d0732d50 0x5594d073e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:34.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.334+0000 7f2472d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594d0732d50 0x5594d073e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53022/0 (socket says 172.21.15.100:53022) 2024-05-10T11:33:34.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.334+0000 7f2472d9d700 1 -- 172.21.15.100:0/22265558 learned_addr learned my addr 172.21.15.100:0/22265558 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:33:34.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.334+0000 7f2483a25b80 1 -- 172.21.15.100:0/22265558 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5594d073f4a0 con 0x5594d0732d50 2024-05-10T11:33:34.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.334+0000 7f2483a25b80 1 -- 172.21.15.100:0/22265558 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5594d063ded0 con 0x5594d0738ad0 2024-05-10T11:33:34.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.334+0000 7f2483a25b80 1 -- 172.21.15.100:0/22265558 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5594d073f620 con 0x5594d07381b0 2024-05-10T11:33:34.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.334+0000 7f247359e700 1 --2- 172.21.15.100:0/22265558 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5594d0738ad0 0x5594d073c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:34.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.334+0000 7f247259c700 1 --2- 172.21.15.100:0/22265558 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594d07381b0 0x5594d073baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:34.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.334+0000 7f2483a25b80 10 client.0 _ll_register_callbacks cb 0x5594d06185b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:33:34.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.334+0000 7f247359e700 1 -- 172.21.15.100:0/22265558 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594d07381b0 msgr2=0x5594d073baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:34.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.334+0000 7f247359e700 1 --2- 172.21.15.100:0/22265558 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594d07381b0 0x5594d073baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:34.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.334+0000 7f247359e700 1 -- 172.21.15.100:0/22265558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594d0732d50 msgr2=0x5594d073e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:34.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.334+0000 7f247359e700 1 --2- 172.21.15.100:0/22265558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594d0732d50 0x5594d073e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:34.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.334+0000 7f247359e700 1 -- 172.21.15.100:0/22265558 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594d063b760 con 0x5594d0738ad0 2024-05-10T11:33:34.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.334+0000 7f247359e700 1 --2- 172.21.15.100:0/22265558 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5594d0738ad0 0x5594d073c030 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f246c007b60 tx=0x7f246c007c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8aa3127285964659 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:33:34.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.335+0000 7f2456ff5700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:33:34.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.335+0000 7f2456ff5700 1 -- 172.21.15.100:0/22265558 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246c01a070 con 0x5594d0738ad0 2024-05-10T11:33:34.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.335+0000 7f2456ff5700 1 -- 172.21.15.100:0/22265558 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f246c003c00 con 0x5594d0738ad0 2024-05-10T11:33:34.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.335+0000 7f2483a25b80 1 -- 172.21.15.100:0/22265558 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2468024070 con 0x5594d0738ad0 2024-05-10T11:33:34.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.335+0000 7f2456ff5700 1 -- 172.21.15.100:0/22265558 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f246c025050 con 0x5594d0738ad0 2024-05-10T11:33:34.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.335+0000 7f2483a25b80 10 client.27776 fetch_fsmap learned FSMap version 37 2024-05-10T11:33:34.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.335+0000 7f2483a25b80 1 -- 172.21.15.100:0/22265558 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5594d0749de0 con 0x5594d0738ad0 2024-05-10T11:33:34.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.336+0000 7f2456ff5700 1 -- 172.21.15.100:0/22265558 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2468024070 con 0x5594d0738ad0 2024-05-10T11:33:34.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.336+0000 7f2456ff5700 1 -- 172.21.15.100:0/22265558 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f246c016a10 con 0x5594d0738ad0 2024-05-10T11:33:34.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.336+0000 7f2483a25b80 10 client.27776 fetch_fsmap finished waiting for FSMap version 37 2024-05-10T11:33:34.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.337+0000 7f2483a25b80 10 client.27776 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:33:34.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.337+0000 7f2483a25b80 20 client.27776 populate_metadata read hostname 'smithi100' 2024-05-10T11:33:34.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.337+0000 7f2483a25b80 1 --2- 172.21.15.100:0/22265558 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5594d074c820 0x5594d074ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:34.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.337+0000 7f2483a25b80 4 client.27776 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:33:34.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.337+0000 7f2483a25b80 1 -- 172.21.15.100:0/22265558 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594d0692320 con 0x5594d074c820 2024-05-10T11:33:34.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.337+0000 7f2472d9d700 1 --2- 172.21.15.100:0/22265558 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5594d074c820 0x5594d074ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:34.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.337+0000 7f2472d9d700 1 --2- 172.21.15.100:0/22265558 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5594d074c820 0x5594d074ecc0 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b74cc7554635286d server_cookie=e19666f22c2b517c in_seq=0 out_seq=0 2024-05-10T11:33:34.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.337+0000 7f2456ff5700 10 client.27776 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:33:34.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.338+0000 7f2456ff5700 1 -- 172.21.15.100:0/22265558 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5594d0692320 con 0x5594d074c820 2024-05-10T11:33:34.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.338+0000 7f2456ff5700 10 client.27776 handle_command_reply: tid=0 2024-05-10T11:33:34.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.338+0000 7f2483a25b80 1 client.27776 shutdown 2024-05-10T11:33:34.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.338+0000 7f2483a25b80 1 -- 172.21.15.100:0/22265558 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5594d0738ad0 msgr2=0x5594d073c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:34.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.338+0000 7f2483a25b80 1 --2- 172.21.15.100:0/22265558 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5594d0738ad0 0x5594d073c030 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f246c007b60 tx=0x7f246c007c40 comp rx=0 tx=0).stop 2024-05-10T11:33:34.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.338+0000 7f2483a25b80 1 -- 172.21.15.100:0/22265558 shutdown_connections 2024-05-10T11:33:34.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.338+0000 7f2483a25b80 1 --2- 172.21.15.100:0/22265558 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5594d074c820 0x5594d074ecc0 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:34.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.339+0000 7f2483a25b80 1 --2- 172.21.15.100:0/22265558 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5594d0738ad0 0x5594d073c030 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:34.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.339+0000 7f2483a25b80 1 --2- 172.21.15.100:0/22265558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594d0732d50 0x5594d073e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:34.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.339+0000 7f2483a25b80 1 --2- 172.21.15.100:0/22265558 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594d07381b0 0x5594d073baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:34.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.339+0000 7f2483a25b80 1 -- 172.21.15.100:0/22265558 >> 172.21.15.100:0/22265558 conn(0x5594d065eb70 msgr2=0x5594d0727b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:33:34.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.339+0000 7f2456ff5700 0 client.27776 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:33:34.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.339+0000 7f2483a25b80 1 -- 172.21.15.100:0/22265558 shutdown_connections 2024-05-10T11:33:34.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.339+0000 7f2483a25b80 1 -- 172.21.15.100:0/22265558 wait complete. 2024-05-10T11:33:34.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.339+0000 7f2483a25b80 20 client.27776 trim_cache size 0 max 16384 2024-05-10T11:33:34.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.368+0000 7f2483a25b80 1 Processor -- start 2024-05-10T11:33:34.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.368+0000 7f2483a25b80 1 -- start start 2024-05-10T11:33:34.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.368+0000 7f2483a25b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594d0732d50 0x5594d0748530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:34.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.368+0000 7f2483a25b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5594d07381b0 0x5594d073de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:34.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.368+0000 7f2483a25b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594d0738ad0 0x5594d0727c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:34.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.368+0000 7f2483a25b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f24741b3b50 con 0x5594d0732d50 2024-05-10T11:33:34.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.368+0000 7f2483a25b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f24741b1350 con 0x5594d07381b0 2024-05-10T11:33:34.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.368+0000 7f2483a25b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f24741af1a0 con 0x5594d0738ad0 2024-05-10T11:33:34.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.368+0000 7f2472d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594d0732d50 0x5594d0748530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:34.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.368+0000 7f247259c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5594d07381b0 0x5594d073de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:34.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.368+0000 7f2472d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594d0732d50 0x5594d0748530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53036/0 (socket says 172.21.15.100:53036) 2024-05-10T11:33:34.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.368+0000 7f2472d9d700 1 -- 172.21.15.100:0/1224901016 learned_addr learned my addr 172.21.15.100:0/1224901016 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:33:34.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.368+0000 7f247359e700 1 --2- 172.21.15.100:0/1224901016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594d0738ad0 0x5594d0727c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:34.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.368+0000 7f2472d9d700 1 -- 172.21.15.100:0/1224901016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594d0738ad0 msgr2=0x5594d0727c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:34.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.368+0000 7f2472d9d700 1 --2- 172.21.15.100:0/1224901016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594d0738ad0 0x5594d0727c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:34.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.368+0000 7f2472d9d700 1 -- 172.21.15.100:0/1224901016 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5594d07381b0 msgr2=0x5594d073de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:34.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.368+0000 7f2472d9d700 1 --2- 172.21.15.100:0/1224901016 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5594d07381b0 0x5594d073de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:34.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.368+0000 7f2472d9d700 1 -- 172.21.15.100:0/1224901016 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594d0749c10 con 0x5594d0732d50 2024-05-10T11:33:34.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.368+0000 7f247259c700 1 --2- 172.21.15.100:0/1224901016 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5594d07381b0 0x5594d073de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:33:34.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.369+0000 7f2472d9d700 1 --2- 172.21.15.100:0/1224901016 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594d0732d50 0x5594d0748530 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f24640aa490 tx=0x7f24640e2040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33f9916cbb8f891f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:33:34.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.369+0000 7f2458ff9700 1 -- 172.21.15.100:0/1224901016 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2464092110 con 0x5594d0732d50 2024-05-10T11:33:34.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.369+0000 7f2458ff9700 1 -- 172.21.15.100:0/1224901016 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f24640b32d0 con 0x5594d0732d50 2024-05-10T11:33:34.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.369+0000 7f2458ff9700 1 -- 172.21.15.100:0/1224901016 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24640b1ba0 con 0x5594d0732d50 2024-05-10T11:33:34.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.370+0000 7f2483a25b80 1 -- 172.21.15.100:0/1224901016 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594d0732d50 msgr2=0x5594d0748530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:34.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.370+0000 7f2483a25b80 1 --2- 172.21.15.100:0/1224901016 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594d0732d50 0x5594d0748530 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f24640aa490 tx=0x7f24640e2040 comp rx=0 tx=0).stop 2024-05-10T11:33:34.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.370+0000 7f2483a25b80 1 -- 172.21.15.100:0/1224901016 shutdown_connections 2024-05-10T11:33:34.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.370+0000 7f2483a25b80 1 --2- 172.21.15.100:0/1224901016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594d0738ad0 0x5594d0727c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:34.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.370+0000 7f2483a25b80 1 --2- 172.21.15.100:0/1224901016 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594d0732d50 0x5594d0748530 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:34.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.370+0000 7f2483a25b80 1 --2- 172.21.15.100:0/1224901016 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5594d07381b0 0x5594d073de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:34.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.370+0000 7f2483a25b80 1 -- 172.21.15.100:0/1224901016 >> 172.21.15.100:0/1224901016 conn(0x5594d065eb70 msgr2=0x5594d073bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:33:34.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.370+0000 7f2483a25b80 1 -- 172.21.15.100:0/1224901016 shutdown_connections 2024-05-10T11:33:34.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.370+0000 7f2483a25b80 1 -- 172.21.15.100:0/1224901016 wait complete. 2024-05-10T11:33:34.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.370+0000 7f2483a25b80 1 -- start start 2024-05-10T11:33:34.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.370+0000 7f2483a25b80 1 Processor -- start 2024-05-10T11:33:34.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.371+0000 7f2483a25b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594d0732d50 0x5594d0744c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:34.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.371+0000 7f2483a25b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5594d07381b0 0x5594d07539a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:34.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.371+0000 7f2483a25b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594d0738ad0 0x5594d0753ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:34.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.371+0000 7f2483a25b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f24741af810 con 0x5594d0732d50 2024-05-10T11:33:34.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.371+0000 7f2483a25b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f24741b0c80 con 0x5594d07381b0 2024-05-10T11:33:34.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.371+0000 7f2483a25b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f24741afee0 con 0x5594d0738ad0 2024-05-10T11:33:34.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.371+0000 7f2483a25b80 10 client.0 _ll_register_callbacks cb 0x5594d06185b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:33:34.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.371+0000 7f2472d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594d0732d50 0x5594d0744c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:34.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.371+0000 7f2472d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594d0732d50 0x5594d0744c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53048/0 (socket says 172.21.15.100:53048) 2024-05-10T11:33:34.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.371+0000 7f2472d9d700 1 -- 172.21.15.100:0/2734238552 learned_addr learned my addr 172.21.15.100:0/2734238552 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:33:34.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.371+0000 7f247359e700 1 --2- 172.21.15.100:0/2734238552 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594d0738ad0 0x5594d0753ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:34.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.371+0000 7f247259c700 1 --2- 172.21.15.100:0/2734238552 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5594d07381b0 0x5594d07539a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:34.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.371+0000 7f2472d9d700 1 -- 172.21.15.100:0/2734238552 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594d0738ad0 msgr2=0x5594d0753ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:34.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.371+0000 7f2472d9d700 1 --2- 172.21.15.100:0/2734238552 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594d0738ad0 0x5594d0753ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:34.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.371+0000 7f2472d9d700 1 -- 172.21.15.100:0/2734238552 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5594d07381b0 msgr2=0x5594d07539a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:34.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.371+0000 7f2472d9d700 1 --2- 172.21.15.100:0/2734238552 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5594d07381b0 0x5594d07539a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:34.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.371+0000 7f2472d9d700 1 -- 172.21.15.100:0/2734238552 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594d0754520 con 0x5594d0732d50 2024-05-10T11:33:34.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.371+0000 7f247359e700 1 --2- 172.21.15.100:0/2734238552 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594d0738ad0 0x5594d0753ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:33:34.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.371+0000 7f2456ff5700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:33:34.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.371+0000 7f2472d9d700 1 --2- 172.21.15.100:0/2734238552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594d0732d50 0x5594d0744c10 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f24640a7770 tx=0x7f2464009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=456314c9c2486c84 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:33:34.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.372+0000 7f2456ff5700 1 -- 172.21.15.100:0/2734238552 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24640c3070 con 0x5594d0732d50 2024-05-10T11:33:34.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.372+0000 7f2456ff5700 1 -- 172.21.15.100:0/2734238552 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f24640a6af0 con 0x5594d0732d50 2024-05-10T11:33:34.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.372+0000 7f2483a25b80 1 -- 172.21.15.100:0/2734238552 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24640b32d0 con 0x5594d0732d50 2024-05-10T11:33:34.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.372+0000 7f2456ff5700 1 -- 172.21.15.100:0/2734238552 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f2464092110 con 0x5594d0732d50 2024-05-10T11:33:34.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.372+0000 7f2483a25b80 10 client.18447 fetch_fsmap learned FSMap version 37 2024-05-10T11:33:34.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.372+0000 7f2483a25b80 1 -- 172.21.15.100:0/2734238552 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5594d07b6fc0 con 0x5594d0732d50 2024-05-10T11:33:34.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.372+0000 7f2456ff5700 1 -- 172.21.15.100:0/2734238552 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f24640b32d0 con 0x5594d0732d50 2024-05-10T11:33:34.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.372+0000 7f2456ff5700 1 -- 172.21.15.100:0/2734238552 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f24640cb1a0 con 0x5594d0732d50 2024-05-10T11:33:34.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.372+0000 7f2483a25b80 10 client.18447 fetch_fsmap finished waiting for FSMap version 37 2024-05-10T11:33:34.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.372+0000 7f2483a25b80 10 client.18447 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:33:34.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.372+0000 7f2483a25b80 20 client.18447 populate_metadata read hostname 'smithi100' 2024-05-10T11:33:34.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.372+0000 7f2483a25b80 1 --2- 172.21.15.100:0/2734238552 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5594d07b7620 0x5594d07b9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:34.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.372+0000 7f2483a25b80 4 client.18447 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T11:33:34.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.372+0000 7f2483a25b80 1 -- 172.21.15.100:0/2734238552 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5594d068c3c0 con 0x5594d07b7620 2024-05-10T11:33:34.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.373+0000 7f247259c700 1 --2- 172.21.15.100:0/2734238552 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5594d07b7620 0x5594d07b9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:34.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.373+0000 7f2456ff5700 10 client.18447 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:33:34.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.373+0000 7f247259c700 1 --2- 172.21.15.100:0/2734238552 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5594d07b7620 0x5594d07b9a40 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbaf4860f2b75cbf server_cookie=9b0c94a941b82015 in_seq=0 out_seq=0 2024-05-10T11:33:34.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.374+0000 7f2456ff5700 1 -- 172.21.15.100:0/2734238552 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5594d068c3c0 con 0x5594d07b7620 2024-05-10T11:33:34.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.374+0000 7f2456ff5700 10 client.18447 handle_command_reply: tid=0 2024-05-10T11:33:34.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.374+0000 7f2483a25b80 1 client.18447 shutdown 2024-05-10T11:33:34.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.374+0000 7f2483a25b80 1 -- 172.21.15.100:0/2734238552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594d0732d50 msgr2=0x5594d0744c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:34.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.374+0000 7f2483a25b80 1 --2- 172.21.15.100:0/2734238552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594d0732d50 0x5594d0744c10 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f24640a7770 tx=0x7f2464009310 comp rx=0 tx=0).stop 2024-05-10T11:33:34.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.374+0000 7f2483a25b80 1 -- 172.21.15.100:0/2734238552 shutdown_connections 2024-05-10T11:33:34.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.374+0000 7f2483a25b80 1 --2- 172.21.15.100:0/2734238552 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5594d07b7620 0x5594d07b9a40 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:34.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.374+0000 7f2456ff5700 0 client.18447 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:33:34.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.374+0000 7f2483a25b80 1 --2- 172.21.15.100:0/2734238552 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594d0738ad0 0x5594d0753ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:34.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.374+0000 7f2483a25b80 1 --2- 172.21.15.100:0/2734238552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594d0732d50 0x5594d0744c10 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:34.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.374+0000 7f2483a25b80 1 --2- 172.21.15.100:0/2734238552 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5594d07381b0 0x5594d07539a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:34.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.374+0000 7f2483a25b80 1 -- 172.21.15.100:0/2734238552 >> 172.21.15.100:0/2734238552 conn(0x5594d065eb70 msgr2=0x5594d0738ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:33:34.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.374+0000 7f2483a25b80 1 -- 172.21.15.100:0/2734238552 shutdown_connections 2024-05-10T11:33:34.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.374+0000 7f2483a25b80 1 -- 172.21.15.100:0/2734238552 wait complete. 2024-05-10T11:33:34.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.374+0000 7f2483a25b80 20 client.18447 trim_cache size 0 max 16384 2024-05-10T11:33:34.378 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:33:34.379 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-05-10T11:33:34.379 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-05-10T11:33:34.379 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:33:34.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.375+0000 7f2479192700 1 -- 172.21.15.100:0/4261413738 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f245c0b2940 msgr2=0x7f245c0b4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:34.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.375+0000 7f2479192700 1 --2- 172.21.15.100:0/4261413738 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f245c0b2940 0x7f245c0b4e00 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f247419a8f0 tx=0x7f2468009190 comp rx=0 tx=0).stop 2024-05-10T11:33:34.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.375+0000 7f2479192700 1 -- 172.21.15.100:0/4261413738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24741023a0 msgr2=0x7f2474198790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:34.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.375+0000 7f2479192700 1 --2- 172.21.15.100:0/4261413738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24741023a0 0x7f2474198790 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2464000c00 tx=0x7f24640047b0 comp rx=0 tx=0).stop 2024-05-10T11:33:34.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.375+0000 7f2479192700 1 -- 172.21.15.100:0/4261413738 shutdown_connections 2024-05-10T11:33:34.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.375+0000 7f2479192700 1 --2- 172.21.15.100:0/4261413738 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f245c0b2940 0x7f245c0b4e00 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:34.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.375+0000 7f2479192700 1 --2- 172.21.15.100:0/4261413738 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2474108410 0x7f2474199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:34.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.375+0000 7f2479192700 1 --2- 172.21.15.100:0/4261413738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24741023a0 0x7f2474198790 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:34.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.375+0000 7f2479192700 1 --2- 172.21.15.100:0/4261413738 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2474102d60 0x7f2474198cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:34.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.375+0000 7f2479192700 1 -- 172.21.15.100:0/4261413738 >> 172.21.15.100:0/4261413738 conn(0x7f2474079350 msgr2=0x7f24741010d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:33:34.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.375+0000 7f2479192700 1 -- 172.21.15.100:0/4261413738 shutdown_connections 2024-05-10T11:33:34.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:34.376+0000 7f2479192700 1 -- 172.21.15.100:0/4261413738 wait complete. 2024-05-10T11:33:34.910 INFO:tasks.workunit.client.0.smithi100.stdout: 10 15485 11.51 MB/sec warmup 88 sec latency 119.304 ms 2024-05-10T11:33:35.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:33:34 smithi100 ceph-mon[30296]: Adjusting osd_memory_target on smithi178 to 176.1M 2024-05-10T11:33:35.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:33:34 smithi100 ceph-mon[30296]: Unable to set osd_memory_target on smithi178 to 184736563: error parsing value: Value '184736563' is below minimum 939524096 2024-05-10T11:33:35.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:33:34 smithi100 ceph-mon[30296]: Adjusting osd_memory_target on smithi100 to 176.1M 2024-05-10T11:33:35.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:33:34 smithi100 ceph-mon[30296]: Unable to set osd_memory_target on smithi100 to 184709324: error parsing value: Value '184709324' is below minimum 939524096 2024-05-10T11:33:35.119 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T11:33:35.119 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T11:33:35.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:33:34 smithi178 ceph-mon[33359]: Adjusting osd_memory_target on smithi178 to 176.1M 2024-05-10T11:33:35.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:33:34 smithi178 ceph-mon[33359]: Unable to set osd_memory_target on smithi178 to 184736563: error parsing value: Value '184736563' is below minimum 939524096 2024-05-10T11:33:35.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:33:34 smithi178 ceph-mon[33359]: Adjusting osd_memory_target on smithi100 to 176.1M 2024-05-10T11:33:35.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:33:34 smithi178 ceph-mon[33359]: Unable to set osd_memory_target on smithi100 to 184709324: error parsing value: Value '184709324' is below minimum 939524096 2024-05-10T11:33:35.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:33:34 smithi188 ceph-mon[34075]: Adjusting osd_memory_target on smithi178 to 176.1M 2024-05-10T11:33:35.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:33:35 smithi188 ceph-mon[34075]: Unable to set osd_memory_target on smithi178 to 184736563: error parsing value: Value '184736563' is below minimum 939524096 2024-05-10T11:33:35.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:33:35 smithi188 ceph-mon[34075]: Adjusting osd_memory_target on smithi100 to 176.1M 2024-05-10T11:33:35.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:33:35 smithi188 ceph-mon[34075]: Unable to set osd_memory_target on smithi100 to 184709324: error parsing value: Value '184709324' is below minimum 939524096 2024-05-10T11:33:35.506 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:33:35.911 INFO:tasks.workunit.client.0.smithi100.stdout: 10 15646 11.49 MB/sec warmup 89 sec latency 116.091 ms 2024-05-10T11:33:36.197 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:33:36 smithi100 ceph-mon[30296]: pgmap v877: 129 pgs: 129 active+clean; 373 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.8 MiB/s wr, 320 op/s 2024-05-10T11:33:36.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:33:36 smithi178 ceph-mon[33359]: pgmap v877: 129 pgs: 129 active+clean; 373 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.8 MiB/s wr, 320 op/s 2024-05-10T11:33:36.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:33:36 smithi188 ceph-mon[34075]: pgmap v877: 129 pgs: 129 active+clean; 373 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.8 MiB/s wr, 320 op/s 2024-05-10T11:33:36.911 INFO:tasks.workunit.client.0.smithi100.stdout: 10 15801 11.42 MB/sec warmup 90 sec latency 52.605 ms 2024-05-10T11:33:37.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:37.892+0000 7f9274429700 1 -- 172.21.15.100:0/1736425888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f926c107730 msgr2=0x7f926c10f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:37.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:37.892+0000 7f9274429700 1 --2- 172.21.15.100:0/1736425888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f926c107730 0x7f926c10f2d0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f92640099c0 tx=0x7f9264009cd0 comp rx=0 tx=0).stop 2024-05-10T11:33:37.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:37.892+0000 7f9274429700 1 -- 172.21.15.100:0/1736425888 shutdown_connections 2024-05-10T11:33:37.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:37.892+0000 7f9274429700 1 --2- 172.21.15.100:0/1736425888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f926c107730 0x7f926c10f2d0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:37.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:37.892+0000 7f9274429700 1 --2- 172.21.15.100:0/1736425888 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f926c1029e0 0x7f926c102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:37.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:37.892+0000 7f9274429700 1 --2- 172.21.15.100:0/1736425888 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f926c103390 0x7f926c107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:37.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:37.892+0000 7f9274429700 1 -- 172.21.15.100:0/1736425888 >> 172.21.15.100:0/1736425888 conn(0x7f926c079350 msgr2=0x7f926c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:33:37.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:37.892+0000 7f9274429700 1 -- 172.21.15.100:0/1736425888 shutdown_connections 2024-05-10T11:33:37.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:37.892+0000 7f9274429700 1 -- 172.21.15.100:0/1736425888 wait complete. 2024-05-10T11:33:37.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:37.893+0000 7f9274429700 1 Processor -- start 2024-05-10T11:33:37.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:37.893+0000 7f9274429700 1 -- start start 2024-05-10T11:33:37.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:37.893+0000 7f9274429700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f926c1029e0 0x7f926c19f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:37.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:37.893+0000 7f9274429700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f926c103390 0x7f926c19f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:37.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:37.893+0000 7f9274429700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f926c107730 0x7f926c197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:37.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:37.893+0000 7f9274429700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f926c112450 con 0x7f926c103390 2024-05-10T11:33:37.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:37.893+0000 7f92719c4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f926c103390 0x7f926c19f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:37.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:37.893+0000 7f92721c5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f926c1029e0 0x7f926c19f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:37.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:37.893+0000 7f9274429700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f926c1122d0 con 0x7f926c1029e0 2024-05-10T11:33:37.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:37.893+0000 7f9274429700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f926c1125d0 con 0x7f926c107730 2024-05-10T11:33:37.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:37.893+0000 7f92721c5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f926c1029e0 0x7f926c19f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:47498/0 (socket says 172.21.15.100:47498) 2024-05-10T11:33:37.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:37.893+0000 7f92719c4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f926c103390 0x7f926c19f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45814/0 (socket says 172.21.15.100:45814) 2024-05-10T11:33:37.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:37.893+0000 7f92721c5700 1 -- 172.21.15.100:0/2225662249 learned_addr learned my addr 172.21.15.100:0/2225662249 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:33:37.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:37.893+0000 7f92729c6700 1 --2- 172.21.15.100:0/2225662249 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f926c107730 0x7f926c197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:37.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:37.894+0000 7f92721c5700 1 -- 172.21.15.100:0/2225662249 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f926c107730 msgr2=0x7f926c197920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:37.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:37.894+0000 7f92721c5700 1 --2- 172.21.15.100:0/2225662249 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f926c107730 0x7f926c197920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:37.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:37.894+0000 7f92721c5700 1 -- 172.21.15.100:0/2225662249 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f926c103390 msgr2=0x7f926c19f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:37.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:37.894+0000 7f92721c5700 1 --2- 172.21.15.100:0/2225662249 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f926c103390 0x7f926c19f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:37.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:37.894+0000 7f92721c5700 1 -- 172.21.15.100:0/2225662249 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f926c198130 con 0x7f926c1029e0 2024-05-10T11:33:37.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:37.894+0000 7f92721c5700 1 --2- 172.21.15.100:0/2225662249 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f926c1029e0 0x7f926c19f470 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f926800ef10 tx=0x7f926800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:33:37.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:37.894+0000 7f92637fe700 1 -- 172.21.15.100:0/2225662249 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926800c9c0 con 0x7f926c1029e0 2024-05-10T11:33:37.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:37.894+0000 7f9274429700 1 -- 172.21.15.100:0/2225662249 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f926c1983c0 con 0x7f926c1029e0 2024-05-10T11:33:37.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:37.894+0000 7f92637fe700 1 -- 172.21.15.100:0/2225662249 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9268019070 con 0x7f926c1029e0 2024-05-10T11:33:37.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:37.894+0000 7f92637fe700 1 -- 172.21.15.100:0/2225662249 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9268014480 con 0x7f926c1029e0 2024-05-10T11:33:37.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:37.894+0000 7f9274429700 1 -- 172.21.15.100:0/2225662249 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f926c1b5330 con 0x7f926c1029e0 2024-05-10T11:33:37.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:37.895+0000 7f9274429700 1 -- 172.21.15.100:0/2225662249 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f926c06b420 con 0x7f926c1029e0 2024-05-10T11:33:37.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:37.897+0000 7f92637fe700 1 -- 172.21.15.100:0/2225662249 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f9268012050 con 0x7f926c1029e0 2024-05-10T11:33:37.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:37.897+0000 7f92637fe700 1 --2- 172.21.15.100:0/2225662249 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f92580b2880 0x7f92580b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:37.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:37.897+0000 7f92637fe700 1 -- 172.21.15.100:0/2225662249 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f92680e2500 con 0x7f926c1029e0 2024-05-10T11:33:37.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:37.897+0000 7f92719c4700 1 --2- 172.21.15.100:0/2225662249 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f92580b2880 0x7f92580b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:37.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:37.898+0000 7f92719c4700 1 --2- 172.21.15.100:0/2225662249 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f92580b2880 0x7f92580b4d40 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f926c198c00 tx=0x7f925c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:33:37.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:37.902+0000 7f92637fe700 1 -- 172.21.15.100:0/2225662249 <== mon.2 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f9268010040 con 0x7f926c1029e0 2024-05-10T11:33:37.911 INFO:tasks.workunit.client.0.smithi100.stdout: 10 15920 11.32 MB/sec warmup 91 sec latency 83.777 ms 2024-05-10T11:33:38.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:33:38 smithi100 ceph-mon[30296]: pgmap v878: 129 pgs: 129 active+clean; 378 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 807 KiB/s rd, 4.6 MiB/s wr, 284 op/s 2024-05-10T11:33:38.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:38.334+0000 7f9274429700 1 -- 172.21.15.100:0/2225662249 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f926c1b5500 con 0x7f926c1029e0 2024-05-10T11:33:38.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:38.335+0000 7f92637fe700 1 -- 172.21.15.100:0/2225662249 <== mon.2 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11902 (secure 0 0 0) 0x7f92680a9e60 con 0x7f926c1029e0 2024-05-10T11:33:38.336 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:33:38.336 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2072225012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2072225012},{"type":"v1","addr":"172.21.15.100:6835","nonce":2072225012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/401939540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":401939540},{"type":"v1","addr":"172.21.15.100:6837","nonce":401939540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14571,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/3946311025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3946311025},{"type":"v1","addr":"172.21.15.100:6839","nonce":3946311025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3736371660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3736371660},{"type":"v1","addr":"172.21.15.100:6841","nonce":3736371660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/2741810791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2741810791},{"type":"v1","addr":"172.21.15.178:6833","nonce":2741810791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1704891041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1704891041},{"type":"v1","addr":"172.21.15.188:6833","nonce":1704891041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T11:14:27.242452+0000","modified":"2024-05-10T11:32:19.578743+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24439,"mds_1":24437,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/2124076312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2124076312},{"type":"v1","addr":"172.21.15.178:6835","nonce":2124076312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/3224358425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3224358425},{"type":"v1","addr":"172.21.15.178:6837","nonce":3224358425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/1943342620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1943342620},{"type":"v1","addr":"172.21.15.188:6837","nonce":1943342620}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.178:6839/3760985404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3760985404},{"type":"v1","addr":"172.21.15.178:6839","nonce":3760985404}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.188:6839/3194917070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":3194917070},{"type":"v1","addr":"172.21.15.188:6839","nonce":3194917070}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27034":{"gid":27034,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/3139559450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3139559450},{"type":"v1","addr":"172.21.15.188:6835","nonce":3139559450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24439,"qdb_cluster":[24439,24427,24437]},"id":1}]} 2024-05-10T11:33:38.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:38.339+0000 7f9274429700 1 -- 172.21.15.100:0/2225662249 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f92580b2880 msgr2=0x7f92580b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:38.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:38.339+0000 7f9274429700 1 --2- 172.21.15.100:0/2225662249 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f92580b2880 0x7f92580b4d40 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f926c198c00 tx=0x7f925c009260 comp rx=0 tx=0).stop 2024-05-10T11:33:38.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:38.339+0000 7f9274429700 1 -- 172.21.15.100:0/2225662249 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f926c1029e0 msgr2=0x7f926c19f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:38.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:38.339+0000 7f9274429700 1 --2- 172.21.15.100:0/2225662249 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f926c1029e0 0x7f926c19f470 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f926800ef10 tx=0x7f926800c3a0 comp rx=0 tx=0).stop 2024-05-10T11:33:38.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:38.339+0000 7f9274429700 1 -- 172.21.15.100:0/2225662249 shutdown_connections 2024-05-10T11:33:38.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:38.339+0000 7f9274429700 1 --2- 172.21.15.100:0/2225662249 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f92580b2880 0x7f92580b4d40 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:38.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:38.339+0000 7f9274429700 1 --2- 172.21.15.100:0/2225662249 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f926c107730 0x7f926c197920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:38.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:38.339+0000 7f9274429700 1 --2- 172.21.15.100:0/2225662249 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f926c1029e0 0x7f926c19f470 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:38.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:38.339+0000 7f9274429700 1 --2- 172.21.15.100:0/2225662249 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f926c103390 0x7f926c19f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:38.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:38.339+0000 7f9274429700 1 -- 172.21.15.100:0/2225662249 >> 172.21.15.100:0/2225662249 conn(0x7f926c079350 msgr2=0x7f926c100fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:33:38.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:38.339+0000 7f9274429700 1 -- 172.21.15.100:0/2225662249 shutdown_connections 2024-05-10T11:33:38.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:38.339+0000 7f9274429700 1 -- 172.21.15.100:0/2225662249 wait complete. 2024-05-10T11:33:38.340 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 37 2024-05-10T11:33:38.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:33:38 smithi178 ceph-mon[33359]: pgmap v878: 129 pgs: 129 active+clean; 378 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 807 KiB/s rd, 4.6 MiB/s wr, 284 op/s 2024-05-10T11:33:38.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:33:38 smithi188 ceph-mon[34075]: pgmap v878: 129 pgs: 129 active+clean; 378 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 807 KiB/s rd, 4.6 MiB/s wr, 284 op/s 2024-05-10T11:33:38.912 INFO:tasks.workunit.client.0.smithi100.stdout: 10 16093 11.29 MB/sec warmup 92 sec latency 88.410 ms 2024-05-10T11:33:39.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:33:39 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/2225662249' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:33:39.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:33:39 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/2225662249' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:33:39.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:33:39 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/2225662249' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:33:39.913 INFO:tasks.workunit.client.0.smithi100.stdout: 10 16258 11.25 MB/sec warmup 93 sec latency 145.220 ms 2024-05-10T11:33:40.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:33:40 smithi178 ceph-mon[33359]: pgmap v879: 129 pgs: 129 active+clean; 380 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 4.6 MiB/s wr, 286 op/s 2024-05-10T11:33:40.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:33:40 smithi188 ceph-mon[34075]: pgmap v879: 129 pgs: 129 active+clean; 380 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 4.6 MiB/s wr, 286 op/s 2024-05-10T11:33:40.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:33:40 smithi100 ceph-mon[30296]: pgmap v879: 129 pgs: 129 active+clean; 380 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 4.6 MiB/s wr, 286 op/s 2024-05-10T11:33:40.913 INFO:tasks.workunit.client.0.smithi100.stdout: 10 16402 11.19 MB/sec warmup 94 sec latency 181.852 ms 2024-05-10T11:33:41.913 INFO:tasks.workunit.client.0.smithi100.stdout: 10 16528 11.19 MB/sec warmup 95 sec latency 200.813 ms 2024-05-10T11:33:42.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:33:42 smithi178 ceph-mon[33359]: pgmap v880: 129 pgs: 129 active+clean; 383 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 4.7 MiB/s wr, 271 op/s 2024-05-10T11:33:42.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:33:42 smithi188 ceph-mon[34075]: pgmap v880: 129 pgs: 129 active+clean; 383 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 4.7 MiB/s wr, 271 op/s 2024-05-10T11:33:42.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:33:42 smithi100 ceph-mon[30296]: pgmap v880: 129 pgs: 129 active+clean; 383 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 4.7 MiB/s wr, 271 op/s 2024-05-10T11:33:42.913 INFO:tasks.workunit.client.0.smithi100.stdout: 10 16631 11.09 MB/sec warmup 96 sec latency 239.489 ms 2024-05-10T11:33:43.913 INFO:tasks.workunit.client.0.smithi100.stdout: 10 16782 11.16 MB/sec warmup 97 sec latency 224.349 ms 2024-05-10T11:33:44.285 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.284+0000 7f71c4c47640 1 -- 172.21.15.100:0/91200402 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71c0103490 msgr2=0x7f71c0105880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:44.285 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.284+0000 7f71c4c47640 1 --2- 172.21.15.100:0/91200402 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71c0103490 0x7f71c0105880 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f71b0009900 tx=0x7f71b002a400 comp rx=0 tx=0).stop 2024-05-10T11:33:44.285 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.285+0000 7f71c4c47640 1 -- 172.21.15.100:0/91200402 shutdown_connections 2024-05-10T11:33:44.285 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.285+0000 7f71c4c47640 1 --2- 172.21.15.100:0/91200402 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71c01086f0 0x7f71c010ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:44.285 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.285+0000 7f71c4c47640 1 --2- 172.21.15.100:0/91200402 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71c0105dc0 0x7f71c01081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:44.285 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.285+0000 7f71c4c47640 1 --2- 172.21.15.100:0/91200402 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71c0103490 0x7f71c0105880 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:44.285 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.285+0000 7f71c4c47640 1 -- 172.21.15.100:0/91200402 >> 172.21.15.100:0/91200402 conn(0x7f71c00fce80 msgr2=0x7f71c00ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:33:44.285 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.285+0000 7f71c4c47640 1 -- 172.21.15.100:0/91200402 shutdown_connections 2024-05-10T11:33:44.285 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.285+0000 7f71c4c47640 1 -- 172.21.15.100:0/91200402 wait complete. 2024-05-10T11:33:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.285+0000 7f71c4c47640 1 Processor -- start 2024-05-10T11:33:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.286+0000 7f71c4c47640 1 -- start start 2024-05-10T11:33:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.286+0000 7f71c4c47640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71c0103490 0x7f71c0196900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.286+0000 7f71c4c47640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71c0105dc0 0x7f71c0196e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.286+0000 7f71c4c47640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71c01086f0 0x7f71c019deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.286+0000 7f71c4c47640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f71c010c1f0 con 0x7f71c0103490 2024-05-10T11:33:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.286+0000 7f71c4c47640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f71c010c070 con 0x7f71c01086f0 2024-05-10T11:33:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.286+0000 7f71c4c47640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f71c010c370 con 0x7f71c0105dc0 2024-05-10T11:33:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.286+0000 7f71be59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71c0103490 0x7f71c0196900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.286+0000 7f71bdd9b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71c0105dc0 0x7f71c0196e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.286+0000 7f71bdd9b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71c0105dc0 0x7f71c0196e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:49034/0 (socket says 172.21.15.100:49034) 2024-05-10T11:33:44.288 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.286+0000 7f71bdd9b640 1 -- 172.21.15.100:0/772020256 learned_addr learned my addr 172.21.15.100:0/772020256 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:33:44.288 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.286+0000 7f71bed9d640 1 --2- 172.21.15.100:0/772020256 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71c01086f0 0x7f71c019deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:44.288 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.287+0000 7f71be59c640 1 -- 172.21.15.100:0/772020256 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71c0105dc0 msgr2=0x7f71c0196e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:44.288 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.287+0000 7f71be59c640 1 --2- 172.21.15.100:0/772020256 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71c0105dc0 0x7f71c0196e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:44.288 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.287+0000 7f71be59c640 1 -- 172.21.15.100:0/772020256 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71c01086f0 msgr2=0x7f71c019deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:44.288 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.287+0000 7f71be59c640 1 --2- 172.21.15.100:0/772020256 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71c01086f0 0x7f71c019deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:44.288 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.287+0000 7f71be59c640 1 -- 172.21.15.100:0/772020256 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71c019e520 con 0x7f71c0103490 2024-05-10T11:33:44.288 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.287+0000 7f71be59c640 1 --2- 172.21.15.100:0/772020256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71c0103490 0x7f71c0196900 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f71b0005ab0 tx=0x7f71b00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:33:44.288 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.287+0000 7f71bed9d640 1 --2- 172.21.15.100:0/772020256 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71c01086f0 0x7f71c019deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:33:44.289 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.287+0000 7f71af7fe640 1 -- 172.21.15.100:0/772020256 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b003f070 con 0x7f71c0103490 2024-05-10T11:33:44.289 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.288+0000 7f71af7fe640 1 -- 172.21.15.100:0/772020256 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f71b0032e10 con 0x7f71c0103490 2024-05-10T11:33:44.289 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.288+0000 7f71c4c47640 1 -- 172.21.15.100:0/772020256 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71c019e7b0 con 0x7f71c0103490 2024-05-10T11:33:44.289 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.288+0000 7f71c4c47640 1 -- 172.21.15.100:0/772020256 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71c019ecf0 con 0x7f71c0103490 2024-05-10T11:33:44.289 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.289+0000 7f71af7fe640 1 -- 172.21.15.100:0/772020256 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b0035070 con 0x7f71c0103490 2024-05-10T11:33:44.292 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.292+0000 7f71c7963740 1 Processor -- start 2024-05-10T11:33:44.294 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.292+0000 7f71c7963740 1 -- start start 2024-05-10T11:33:44.294 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.293+0000 7f71c7963740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b31ec8d5c0 0x55b31ec8d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:44.294 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.293+0000 7f71c7963740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b31ec8dfb0 0x55b31ec92ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:44.294 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.293+0000 7f71c7963740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b31ec93120 0x55b31ec955b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:44.294 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.293+0000 7f71c7963740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55b31ebe95d0 con 0x55b31ec8dfb0 2024-05-10T11:33:44.294 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.293+0000 7f71c7963740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b31eba2890 con 0x55b31ec8d5c0 2024-05-10T11:33:44.294 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.293+0000 7f71c7963740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b31ebe5590 con 0x55b31ec93120 2024-05-10T11:33:44.294 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.293+0000 7f71af7fe640 1 -- 172.21.15.100:0/772020256 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f71b0043400 con 0x7f71c0103490 2024-05-10T11:33:44.294 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.293+0000 7f71bdd9b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b31ec93120 0x55b31ec955b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:44.295 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.293+0000 7f71bdd9b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b31ec93120 0x55b31ec955b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:49042/0 (socket says 172.21.15.100:49042) 2024-05-10T11:33:44.295 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.293+0000 7f71bdd9b640 1 -- 172.21.15.100:0/1301782979 learned_addr learned my addr 172.21.15.100:0/1301782979 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:33:44.295 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.293+0000 7f71bed9d640 1 --2- 172.21.15.100:0/1301782979 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b31ec8d5c0 0x55b31ec8d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:44.295 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.293+0000 7f71be59c640 1 --2- 172.21.15.100:0/1301782979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b31ec8dfb0 0x55b31ec92ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:44.295 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.294+0000 7f71bdd9b640 1 -- 172.21.15.100:0/1301782979 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b31ec8d5c0 msgr2=0x55b31ec8d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:44.295 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.294+0000 7f71bdd9b640 1 --2- 172.21.15.100:0/1301782979 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b31ec8d5c0 0x55b31ec8d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:44.295 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.294+0000 7f71bdd9b640 1 -- 172.21.15.100:0/1301782979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b31ec8dfb0 msgr2=0x55b31ec92ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:44.295 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.294+0000 7f71bdd9b640 1 --2- 172.21.15.100:0/1301782979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b31ec8dfb0 0x55b31ec92ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:44.295 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.294+0000 7f71bdd9b640 1 -- 172.21.15.100:0/1301782979 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b31e8fb210 con 0x55b31ec93120 2024-05-10T11:33:44.295 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.294+0000 7f71bdd9b640 1 --2- 172.21.15.100:0/1301782979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b31ec93120 0x55b31ec955b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f71b4009d40 tx=0x7f71b4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1888d1b61add1b6d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:33:44.300 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.294+0000 7f71acff9640 1 -- 172.21.15.100:0/1301782979 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b401e070 con 0x55b31ec93120 2024-05-10T11:33:44.300 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.294+0000 7f71acff9640 1 -- 172.21.15.100:0/1301782979 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f71b4019ac0 con 0x55b31ec93120 2024-05-10T11:33:44.300 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.295+0000 7f71af7fe640 1 --2- 172.21.15.100:0/772020256 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f71a80b29a0 0x7f71a80b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:44.301 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.295+0000 7f71acff9640 1 -- 172.21.15.100:0/1301782979 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b400ea10 con 0x55b31ec93120 2024-05-10T11:33:44.301 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.295+0000 7f71af7fe640 1 -- 172.21.15.100:0/772020256 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f71b0102af0 con 0x7f71c0103490 2024-05-10T11:33:44.301 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.295+0000 7f71bed9d640 1 --2- 172.21.15.100:0/772020256 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f71a80b29a0 0x7f71a80b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:44.301 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.295+0000 7f71c7963740 1 -- 172.21.15.100:0/1301782979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b31ec93120 msgr2=0x55b31ec955b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:44.301 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.295+0000 7f71c7963740 1 --2- 172.21.15.100:0/1301782979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b31ec93120 0x55b31ec955b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f71b4009d40 tx=0x7f71b4011810 comp rx=0 tx=0).stop 2024-05-10T11:33:44.301 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.295+0000 7f71c7963740 1 -- 172.21.15.100:0/1301782979 shutdown_connections 2024-05-10T11:33:44.301 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.295+0000 7f71c7963740 1 --2- 172.21.15.100:0/1301782979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b31ec93120 0x55b31ec955b0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:44.301 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.295+0000 7f71c7963740 1 --2- 172.21.15.100:0/1301782979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b31ec8dfb0 0x55b31ec92ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:44.301 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.295+0000 7f71c7963740 1 --2- 172.21.15.100:0/1301782979 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b31ec8d5c0 0x55b31ec8d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:44.301 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.295+0000 7f71c7963740 1 -- 172.21.15.100:0/1301782979 >> 172.21.15.100:0/1301782979 conn(0x55b31ec88c30 msgr2=0x55b31ec89030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:33:44.301 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.295+0000 7f71bed9d640 1 --2- 172.21.15.100:0/772020256 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f71a80b29a0 0x7f71a80b4e60 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x55b31ec8e810 tx=0x7f71b800f040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:33:44.301 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.295+0000 7f71c7963740 1 -- 172.21.15.100:0/1301782979 shutdown_connections 2024-05-10T11:33:44.301 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.296+0000 7f71c7963740 1 -- 172.21.15.100:0/1301782979 wait complete. 2024-05-10T11:33:44.301 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.296+0000 7f71c7963740 1 -- start start 2024-05-10T11:33:44.301 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.296+0000 7f71c7963740 1 Processor -- start 2024-05-10T11:33:44.302 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.296+0000 7f71c7963740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b31ec8d5c0 0x55b31ec97830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:44.302 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.296+0000 7f71c7963740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b31ec8dfb0 0x55b31ec97d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:44.302 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.296+0000 7f71c7963740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b31ec93120 0x55b31ec9edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:44.302 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.296+0000 7f71c7963740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55b31ec96170 con 0x55b31ec8d5c0 2024-05-10T11:33:44.302 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.296+0000 7f71c7963740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b31eba6370 con 0x55b31ec93120 2024-05-10T11:33:44.302 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.296+0000 7f71c7963740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b31ec962f0 con 0x55b31ec8dfb0 2024-05-10T11:33:44.302 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.296+0000 7f71be59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b31ec8d5c0 0x55b31ec97830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:44.302 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.296+0000 7f71bed9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b31ec93120 0x55b31ec9edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:44.302 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.296+0000 7f71bdd9b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b31ec8dfb0 0x55b31ec97d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:44.302 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.296+0000 7f71bed9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b31ec93120 0x55b31ec9edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:47524/0 (socket says 172.21.15.100:47524) 2024-05-10T11:33:44.302 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.296+0000 7f71bdd9b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b31ec8dfb0 0x55b31ec97d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:49054/0 (socket says 172.21.15.100:49054) 2024-05-10T11:33:44.302 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.296+0000 7f71bed9d640 1 -- 172.21.15.100:0/787008572 learned_addr learned my addr 172.21.15.100:0/787008572 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:33:44.302 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.296+0000 7f71be59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b31ec8d5c0 0x55b31ec97830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45854/0 (socket says 172.21.15.100:45854) 2024-05-10T11:33:44.302 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.297+0000 7f71c7963740 10 client.0 _ll_register_callbacks cb 0x55b31e8faa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:33:44.303 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.297+0000 7f71bed9d640 1 -- 172.21.15.100:0/787008572 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b31ec8dfb0 msgr2=0x55b31ec97d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:44.303 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.297+0000 7f71bed9d640 1 --2- 172.21.15.100:0/787008572 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b31ec8dfb0 0x55b31ec97d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:44.303 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.297+0000 7f71bed9d640 1 -- 172.21.15.100:0/787008572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b31ec8d5c0 msgr2=0x55b31ec97830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:44.303 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.297+0000 7f71bed9d640 1 --2- 172.21.15.100:0/787008572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b31ec8d5c0 0x55b31ec97830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:44.303 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.297+0000 7f71bed9d640 1 -- 172.21.15.100:0/787008572 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b31e9c7620 con 0x55b31ec93120 2024-05-10T11:33:44.303 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.297+0000 7f71be59c640 1 --2- 172.21.15.100:0/787008572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b31ec8d5c0 0x55b31ec97830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:33:44.303 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.297+0000 7f71bed9d640 1 --2- 172.21.15.100:0/787008572 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b31ec93120 0x55b31ec9edb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f71b801adc0 tx=0x7f71b8004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f62105428b405b53 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:33:44.303 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.297+0000 7f71a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:33:44.303 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.297+0000 7f71a5ffb640 1 -- 172.21.15.100:0/787008572 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b802a060 con 0x55b31ec93120 2024-05-10T11:33:44.303 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.297+0000 7f71c7963740 1 -- 172.21.15.100:0/787008572 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71b4019ac0 con 0x55b31ec93120 2024-05-10T11:33:44.303 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.297+0000 7f71a5ffb640 1 -- 172.21.15.100:0/787008572 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f71b8025600 con 0x55b31ec93120 2024-05-10T11:33:44.303 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.298+0000 7f71a5ffb640 1 -- 172.21.15.100:0/787008572 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f71b8031050 con 0x55b31ec93120 2024-05-10T11:33:44.303 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.298+0000 7f71c7963740 10 client.27800 fetch_fsmap learned FSMap version 37 2024-05-10T11:33:44.304 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.298+0000 7f71c7963740 1 -- 172.21.15.100:0/787008572 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55b31eca1bd0 con 0x55b31ec93120 2024-05-10T11:33:44.304 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.298+0000 7f71a5ffb640 1 -- 172.21.15.100:0/787008572 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f71b4019ac0 con 0x55b31ec93120 2024-05-10T11:33:44.304 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.298+0000 7f71a5ffb640 1 -- 172.21.15.100:0/787008572 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f71b8019830 con 0x55b31ec93120 2024-05-10T11:33:44.304 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.298+0000 7f71c7963740 10 client.27800 fetch_fsmap finished waiting for FSMap version 37 2024-05-10T11:33:44.304 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.298+0000 7f71c7963740 10 client.27800 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:33:44.304 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.298+0000 7f71c7963740 20 client.27800 populate_metadata read hostname 'smithi100' 2024-05-10T11:33:44.304 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.298+0000 7f71c7963740 1 --2- 172.21.15.100:0/787008572 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55b31eca33d0 0x55b31eca5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:44.304 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.298+0000 7f71c7963740 4 client.27800 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:33:44.304 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.298+0000 7f71c7963740 1 -- 172.21.15.100:0/787008572 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b31ebf71e0 con 0x55b31eca33d0 2024-05-10T11:33:44.305 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.298+0000 7f71be59c640 1 --2- 172.21.15.100:0/787008572 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55b31eca33d0 0x55b31eca5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:44.305 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.299+0000 7f71be59c640 1 --2- 172.21.15.100:0/787008572 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55b31eca33d0 0x55b31eca5890 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27108fdbc4679351 server_cookie=b66312eb6b54c2d9 in_seq=0 out_seq=0 2024-05-10T11:33:44.305 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.299+0000 7f71a5ffb640 10 client.27800 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:33:44.305 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.299+0000 7f71a5ffb640 1 -- 172.21.15.100:0/787008572 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b31ebf71e0 con 0x55b31eca33d0 2024-05-10T11:33:44.305 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.299+0000 7f71a5ffb640 10 client.27800 handle_command_reply: tid=0 2024-05-10T11:33:44.305 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.299+0000 7f71c7963740 1 client.27800 shutdown 2024-05-10T11:33:44.305 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.300+0000 7f71c7963740 1 -- 172.21.15.100:0/787008572 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b31ec93120 msgr2=0x55b31ec9edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:44.305 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.300+0000 7f71c7963740 1 --2- 172.21.15.100:0/787008572 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b31ec93120 0x55b31ec9edb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f71b801adc0 tx=0x7f71b8004ab0 comp rx=0 tx=0).stop 2024-05-10T11:33:44.305 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.300+0000 7f71c7963740 1 -- 172.21.15.100:0/787008572 shutdown_connections 2024-05-10T11:33:44.305 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.300+0000 7f71c7963740 1 --2- 172.21.15.100:0/787008572 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55b31eca33d0 0x55b31eca5890 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:44.305 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.300+0000 7f71c7963740 1 --2- 172.21.15.100:0/787008572 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b31ec93120 0x55b31ec9edb0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:44.305 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.300+0000 7f71a5ffb640 0 client.27800 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:33:44.305 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.300+0000 7f71c7963740 1 --2- 172.21.15.100:0/787008572 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b31ec8dfb0 0x55b31ec97d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:44.305 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.300+0000 7f71c7963740 1 --2- 172.21.15.100:0/787008572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b31ec8d5c0 0x55b31ec97830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:44.306 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.300+0000 7f71c7963740 1 -- 172.21.15.100:0/787008572 >> 172.21.15.100:0/787008572 conn(0x55b31ec88c30 msgr2=0x55b31ec7f580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:33:44.306 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.300+0000 7f71c7963740 1 -- 172.21.15.100:0/787008572 shutdown_connections 2024-05-10T11:33:44.306 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.300+0000 7f71c7963740 1 -- 172.21.15.100:0/787008572 wait complete. 2024-05-10T11:33:44.306 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.300+0000 7f71c7963740 20 client.27800 trim_cache size 0 max 16384 2024-05-10T11:33:44.327 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.326+0000 7f71c7963740 1 Processor -- start 2024-05-10T11:33:44.328 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.326+0000 7f71c7963740 1 -- start start 2024-05-10T11:33:44.328 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.326+0000 7f71c7963740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b31ec8d5c0 0x55b31ec8f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:44.328 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.326+0000 7f71c7963740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b31ec8dfb0 0x55b31ec8f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:44.328 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.326+0000 7f71c7963740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b31ec93120 0x55b31ec9ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:44.328 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.326+0000 7f71c7963740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f71a0012410 con 0x55b31ec8dfb0 2024-05-10T11:33:44.328 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.326+0000 7f71c7963740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f71a000fc50 con 0x55b31ec93120 2024-05-10T11:33:44.328 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.326+0000 7f71c7963740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f71a000daa0 con 0x55b31ec8d5c0 2024-05-10T11:33:44.328 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.326+0000 7f71bdd9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b31ec8dfb0 0x55b31ec8f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:44.328 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.326+0000 7f71be59c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b31ec8d5c0 0x55b31ec8f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:44.329 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.326+0000 7f71be59c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b31ec8d5c0 0x55b31ec8f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:49058/0 (socket says 172.21.15.100:49058) 2024-05-10T11:33:44.329 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.326+0000 7f71be59c640 1 -- 172.21.15.100:0/225705607 learned_addr learned my addr 172.21.15.100:0/225705607 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:33:44.329 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.326+0000 7f71bed9d640 1 --2- 172.21.15.100:0/225705607 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b31ec93120 0x55b31ec9ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:44.329 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.327+0000 7f71bed9d640 1 -- 172.21.15.100:0/225705607 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b31ec8d5c0 msgr2=0x55b31ec8f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:44.329 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.327+0000 7f71bed9d640 1 --2- 172.21.15.100:0/225705607 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b31ec8d5c0 0x55b31ec8f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:44.329 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.327+0000 7f71bed9d640 1 -- 172.21.15.100:0/225705607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b31ec8dfb0 msgr2=0x55b31ec8f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:44.329 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.327+0000 7f71bed9d640 1 --2- 172.21.15.100:0/225705607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b31ec8dfb0 0x55b31ec8f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:44.329 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.327+0000 7f71bed9d640 1 -- 172.21.15.100:0/225705607 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b31ec9fea0 con 0x55b31ec93120 2024-05-10T11:33:44.330 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.327+0000 7f71be59c640 1 --2- 172.21.15.100:0/225705607 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b31ec8d5c0 0x55b31ec8f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:33:44.330 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.327+0000 7f71bed9d640 1 --2- 172.21.15.100:0/225705607 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b31ec93120 0x55b31ec9ecd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f71b800ce50 tx=0x7f71b800d890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e715f44708750d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:33:44.330 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.327+0000 7f71a7fff640 1 -- 172.21.15.100:0/225705607 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b8039070 con 0x55b31ec93120 2024-05-10T11:33:44.330 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.327+0000 7f71a7fff640 1 -- 172.21.15.100:0/225705607 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f71b8019e90 con 0x55b31ec93120 2024-05-10T11:33:44.330 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.327+0000 7f71a7fff640 1 -- 172.21.15.100:0/225705607 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b8024ca0 con 0x55b31ec93120 2024-05-10T11:33:44.335 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.328+0000 7f71c7963740 1 -- 172.21.15.100:0/225705607 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b31ec93120 msgr2=0x55b31ec9ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:44.335 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.328+0000 7f71c7963740 1 --2- 172.21.15.100:0/225705607 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b31ec93120 0x55b31ec9ecd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f71b800ce50 tx=0x7f71b800d890 comp rx=0 tx=0).stop 2024-05-10T11:33:44.336 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.328+0000 7f71c7963740 1 -- 172.21.15.100:0/225705607 shutdown_connections 2024-05-10T11:33:44.336 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.328+0000 7f71c7963740 1 --2- 172.21.15.100:0/225705607 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b31ec93120 0x55b31ec9ecd0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:44.336 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.328+0000 7f71c7963740 1 --2- 172.21.15.100:0/225705607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b31ec8dfb0 0x55b31ec8f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:44.336 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.328+0000 7f71c7963740 1 --2- 172.21.15.100:0/225705607 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b31ec8d5c0 0x55b31ec8f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:44.336 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.328+0000 7f71c7963740 1 -- 172.21.15.100:0/225705607 >> 172.21.15.100:0/225705607 conn(0x55b31ec88c30 msgr2=0x55b31ec7b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:33:44.336 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.328+0000 7f71c7963740 1 -- 172.21.15.100:0/225705607 shutdown_connections 2024-05-10T11:33:44.336 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.328+0000 7f71c7963740 1 -- 172.21.15.100:0/225705607 wait complete. 2024-05-10T11:33:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.329+0000 7f71c7963740 1 -- start start 2024-05-10T11:33:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.329+0000 7f71c7963740 1 Processor -- start 2024-05-10T11:33:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.329+0000 7f71c7963740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b31ec8d5c0 0x55b31ec9c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.329+0000 7f71c7963740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b31ec8dfb0 0x55b31ec9c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.329+0000 7f71c7963740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b31ec93120 0x55b31ec9d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.329+0000 7f71c7963740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f71a000e110 con 0x55b31ec8dfb0 2024-05-10T11:33:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.329+0000 7f71c7963740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f71a000f580 con 0x55b31ec8d5c0 2024-05-10T11:33:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.329+0000 7f71c7963740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f71a000e7e0 con 0x55b31ec93120 2024-05-10T11:33:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.329+0000 7f71bdd9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b31ec8dfb0 0x55b31ec9c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.329+0000 7f71c7963740 10 client.0 _ll_register_callbacks cb 0x7f71b800d340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:33:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.329+0000 7f71be59c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b31ec8d5c0 0x55b31ec9c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.329+0000 7f71bdd9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b31ec8dfb0 0x55b31ec9c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45878/0 (socket says 172.21.15.100:45878) 2024-05-10T11:33:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.329+0000 7f71be59c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b31ec8d5c0 0x55b31ec9c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:47544/0 (socket says 172.21.15.100:47544) 2024-05-10T11:33:44.338 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.329+0000 7f71bdd9b640 1 -- 172.21.15.100:0/909565738 learned_addr learned my addr 172.21.15.100:0/909565738 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:33:44.338 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.329+0000 7f71bed9d640 1 --2- 172.21.15.100:0/909565738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b31ec93120 0x55b31ec9d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:44.338 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.330+0000 7f71bdd9b640 1 -- 172.21.15.100:0/909565738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b31ec93120 msgr2=0x55b31ec9d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:44.338 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.330+0000 7f71bdd9b640 1 --2- 172.21.15.100:0/909565738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b31ec93120 0x55b31ec9d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:44.338 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.330+0000 7f71bdd9b640 1 -- 172.21.15.100:0/909565738 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b31ec8d5c0 msgr2=0x55b31ec9c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:44.338 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.330+0000 7f71bdd9b640 1 --2- 172.21.15.100:0/909565738 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b31ec8d5c0 0x55b31ec9c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:44.338 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.330+0000 7f71bdd9b640 1 -- 172.21.15.100:0/909565738 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b31ec9df40 con 0x55b31ec8dfb0 2024-05-10T11:33:44.338 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.330+0000 7f71be59c640 1 --2- 172.21.15.100:0/909565738 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b31ec8d5c0 0x55b31ec9c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:33:44.338 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.330+0000 7f71bdd9b640 1 --2- 172.21.15.100:0/909565738 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b31ec8dfb0 0x55b31ec9c940 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f71b401c040 tx=0x7f71b40048b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24b8c3d5299f4389 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:33:44.338 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.330+0000 7f71a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:33:44.338 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.330+0000 7f71a5ffb640 1 -- 172.21.15.100:0/909565738 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b401e040 con 0x55b31ec8dfb0 2024-05-10T11:33:44.339 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.330+0000 7f71a5ffb640 1 -- 172.21.15.100:0/909565738 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f71b4004c30 con 0x55b31ec8dfb0 2024-05-10T11:33:44.339 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.330+0000 7f71c7963740 1 -- 172.21.15.100:0/909565738 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71b8019e90 con 0x55b31ec8dfb0 2024-05-10T11:33:44.339 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.330+0000 7f71a5ffb640 1 -- 172.21.15.100:0/909565738 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f71b4035050 con 0x55b31ec8dfb0 2024-05-10T11:33:44.339 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.330+0000 7f71c7963740 10 client.18471 fetch_fsmap learned FSMap version 37 2024-05-10T11:33:44.339 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.330+0000 7f71c7963740 1 -- 172.21.15.100:0/909565738 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55b31ed0dd50 con 0x55b31ec8dfb0 2024-05-10T11:33:44.339 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.330+0000 7f71a5ffb640 1 -- 172.21.15.100:0/909565738 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f71b8019e90 con 0x55b31ec8dfb0 2024-05-10T11:33:44.339 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.331+0000 7f71a5ffb640 1 -- 172.21.15.100:0/909565738 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f71b402ca50 con 0x55b31ec8dfb0 2024-05-10T11:33:44.339 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.331+0000 7f71c7963740 10 client.18471 fetch_fsmap finished waiting for FSMap version 37 2024-05-10T11:33:44.339 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.331+0000 7f71c7963740 10 client.18471 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:33:44.339 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.331+0000 7f71c7963740 20 client.18471 populate_metadata read hostname 'smithi100' 2024-05-10T11:33:44.339 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.331+0000 7f71c7963740 1 --2- 172.21.15.100:0/909565738 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55b31ed0e450 0x55b31ed10870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:44.339 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.331+0000 7f71c7963740 4 client.18471 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:33:44.339 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.331+0000 7f71c7963740 1 -- 172.21.15.100:0/909565738 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b31ebf1050 con 0x55b31ed0e450 2024-05-10T11:33:44.339 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.331+0000 7f71be59c640 1 --2- 172.21.15.100:0/909565738 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55b31ed0e450 0x55b31ed10870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:44.340 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.331+0000 7f71a5ffb640 10 client.18471 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:33:44.340 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.331+0000 7f71be59c640 1 --2- 172.21.15.100:0/909565738 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55b31ed0e450 0x55b31ed10870 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67ae93664de58014 server_cookie=db87d26df5f7200 in_seq=0 out_seq=0 2024-05-10T11:33:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.482+0000 7f71a5ffb640 1 -- 172.21.15.100:0/909565738 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b31ebf1050 con 0x55b31ed0e450 2024-05-10T11:33:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.482+0000 7f71a5ffb640 10 client.18471 handle_command_reply: tid=0 2024-05-10T11:33:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.482+0000 7f71c7963740 1 client.18471 shutdown 2024-05-10T11:33:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.483+0000 7f71c7963740 1 -- 172.21.15.100:0/909565738 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b31ec8dfb0 msgr2=0x55b31ec9c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.483+0000 7f71c7963740 1 --2- 172.21.15.100:0/909565738 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b31ec8dfb0 0x55b31ec9c940 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f71b401c040 tx=0x7f71b40048b0 comp rx=0 tx=0).stop 2024-05-10T11:33:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.484+0000 7f71c7963740 1 -- 172.21.15.100:0/909565738 shutdown_connections 2024-05-10T11:33:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.484+0000 7f71c7963740 1 --2- 172.21.15.100:0/909565738 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55b31ed0e450 0x55b31ed10870 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.484+0000 7f71c7963740 1 --2- 172.21.15.100:0/909565738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b31ec93120 0x55b31ec9d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.484+0000 7f71a5ffb640 0 client.18471 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:33:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.484+0000 7f71c7963740 1 --2- 172.21.15.100:0/909565738 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b31ec8dfb0 0x55b31ec9c940 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.484+0000 7f71c7963740 1 --2- 172.21.15.100:0/909565738 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b31ec8d5c0 0x55b31ec9c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.484+0000 7f71c7963740 1 -- 172.21.15.100:0/909565738 >> 172.21.15.100:0/909565738 conn(0x55b31ec88c30 msgr2=0x55b31ec97780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:33:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.485+0000 7f71c7963740 1 -- 172.21.15.100:0/909565738 shutdown_connections 2024-05-10T11:33:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.485+0000 7f71c7963740 1 -- 172.21.15.100:0/909565738 wait complete. 2024-05-10T11:33:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.485+0000 7f71c7963740 20 client.18471 trim_cache size 0 max 16384 2024-05-10T11:33:44.486 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:33:44.486 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:33:44.486 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:33:44.486 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:33:44.487 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.486+0000 7f71c4c47640 1 -- 172.21.15.100:0/772020256 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f71a80b29a0 msgr2=0x7f71a80b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:44.487 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.486+0000 7f71c4c47640 1 --2- 172.21.15.100:0/772020256 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f71a80b29a0 0x7f71a80b4e60 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x55b31ec8e810 tx=0x7f71b800f040 comp rx=0 tx=0).stop 2024-05-10T11:33:44.487 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.486+0000 7f71c4c47640 1 -- 172.21.15.100:0/772020256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71c0103490 msgr2=0x7f71c0196900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:44.487 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.486+0000 7f71c4c47640 1 --2- 172.21.15.100:0/772020256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71c0103490 0x7f71c0196900 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f71b0005ab0 tx=0x7f71b00042f0 comp rx=0 tx=0).stop 2024-05-10T11:33:44.487 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.486+0000 7f71c4c47640 1 -- 172.21.15.100:0/772020256 shutdown_connections 2024-05-10T11:33:44.487 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.486+0000 7f71c4c47640 1 --2- 172.21.15.100:0/772020256 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f71a80b29a0 0x7f71a80b4e60 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:44.487 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.486+0000 7f71c4c47640 1 --2- 172.21.15.100:0/772020256 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71c01086f0 0x7f71c019deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:44.487 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.486+0000 7f71c4c47640 1 --2- 172.21.15.100:0/772020256 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71c0105dc0 0x7f71c0196e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:44.487 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.486+0000 7f71c4c47640 1 --2- 172.21.15.100:0/772020256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71c0103490 0x7f71c0196900 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:44.487 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.486+0000 7f71c4c47640 1 -- 172.21.15.100:0/772020256 >> 172.21.15.100:0/772020256 conn(0x7f71c00fce80 msgr2=0x7f71c0103d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:33:44.487 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.486+0000 7f71c4c47640 1 -- 172.21.15.100:0/772020256 shutdown_connections 2024-05-10T11:33:44.487 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:44.486+0000 7f71c4c47640 1 -- 172.21.15.100:0/772020256 wait complete. 2024-05-10T11:33:44.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:33:44 smithi178 ceph-mon[33359]: pgmap v881: 129 pgs: 129 active+clean; 381 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 4.5 MiB/s wr, 271 op/s 2024-05-10T11:33:44.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:33:44 smithi188 ceph-mon[34075]: pgmap v881: 129 pgs: 129 active+clean; 381 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 4.5 MiB/s wr, 271 op/s 2024-05-10T11:33:44.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:33:44 smithi100 ceph-mon[30296]: pgmap v881: 129 pgs: 129 active+clean; 381 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 4.5 MiB/s wr, 271 op/s 2024-05-10T11:33:44.913 INFO:tasks.workunit.client.0.smithi100.stdout: 10 16942 11.14 MB/sec warmup 98 sec latency 187.123 ms 2024-05-10T11:33:45.105 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-10T11:33:45.451 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:33:45.913 INFO:tasks.workunit.client.0.smithi100.stdout: 10 17105 11.07 MB/sec warmup 99 sec latency 144.724 ms 2024-05-10T11:33:46.309 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:33:46 smithi100 ceph-mon[30296]: pgmap v882: 129 pgs: 129 active+clean; 380 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 976 KiB/s rd, 4.5 MiB/s wr, 287 op/s 2024-05-10T11:33:46.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:33:46 smithi178 ceph-mon[33359]: pgmap v882: 129 pgs: 129 active+clean; 380 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 976 KiB/s rd, 4.5 MiB/s wr, 287 op/s 2024-05-10T11:33:46.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:33:46 smithi188 ceph-mon[34075]: pgmap v882: 129 pgs: 129 active+clean; 380 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 976 KiB/s rd, 4.5 MiB/s wr, 287 op/s 2024-05-10T11:33:46.913 INFO:tasks.workunit.client.0.smithi100.stdout: 10 17237 10.98 MB/sec warmup 100 sec latency 163.664 ms 2024-05-10T11:33:47.913 INFO:tasks.workunit.client.0.smithi100.stdout: 10 17360 10.89 MB/sec warmup 101 sec latency 153.163 ms 2024-05-10T11:33:48.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:33:48 smithi178 ceph-mon[33359]: pgmap v883: 129 pgs: 129 active+clean; 377 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 4.0 MiB/s wr, 269 op/s 2024-05-10T11:33:48.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:33:48 smithi188 ceph-mon[34075]: pgmap v883: 129 pgs: 129 active+clean; 377 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 4.0 MiB/s wr, 269 op/s 2024-05-10T11:33:48.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:33:48 smithi100 ceph-mon[30296]: pgmap v883: 129 pgs: 129 active+clean; 377 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 4.0 MiB/s wr, 269 op/s 2024-05-10T11:33:48.913 INFO:tasks.workunit.client.0.smithi100.stdout: 10 17515 10.86 MB/sec warmup 102 sec latency 136.845 ms 2024-05-10T11:33:49.914 INFO:tasks.workunit.client.0.smithi100.stdout: 10 17652 10.81 MB/sec warmup 103 sec latency 145.640 ms 2024-05-10T11:33:50.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:33:50 smithi100 ceph-mon[30296]: pgmap v884: 129 pgs: 129 active+clean; 380 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 756 KiB/s rd, 3.6 MiB/s wr, 277 op/s 2024-05-10T11:33:50.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:33:50 smithi178 ceph-mon[33359]: pgmap v884: 129 pgs: 129 active+clean; 380 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 756 KiB/s rd, 3.6 MiB/s wr, 277 op/s 2024-05-10T11:33:50.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:33:50 smithi188 ceph-mon[34075]: pgmap v884: 129 pgs: 129 active+clean; 380 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 756 KiB/s rd, 3.6 MiB/s wr, 277 op/s 2024-05-10T11:33:50.913 INFO:tasks.workunit.client.0.smithi100.stdout: 10 17789 10.81 MB/sec warmup 104 sec latency 176.618 ms 2024-05-10T11:33:51.913 INFO:tasks.workunit.client.0.smithi100.stdout: 10 17960 10.83 MB/sec warmup 105 sec latency 137.073 ms 2024-05-10T11:33:52.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:33:52 smithi178 ceph-mon[33359]: pgmap v885: 129 pgs: 129 active+clean; 381 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 3.4 MiB/s wr, 254 op/s 2024-05-10T11:33:52.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:33:52 smithi188 ceph-mon[34075]: pgmap v885: 129 pgs: 129 active+clean; 381 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 3.4 MiB/s wr, 254 op/s 2024-05-10T11:33:52.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:33:52 smithi100 ceph-mon[30296]: pgmap v885: 129 pgs: 129 active+clean; 381 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 3.4 MiB/s wr, 254 op/s 2024-05-10T11:33:52.913 INFO:tasks.workunit.client.0.smithi100.stdout: 10 18158 10.87 MB/sec warmup 106 sec latency 145.314 ms 2024-05-10T11:33:53.914 INFO:tasks.workunit.client.0.smithi100.stdout: 10 18306 10.83 MB/sec warmup 107 sec latency 179.779 ms 2024-05-10T11:33:54.330 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:33:54 smithi188 ceph-mon[34075]: pgmap v886: 129 pgs: 129 active+clean; 383 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 3.3 MiB/s wr, 266 op/s 2024-05-10T11:33:54.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:33:54 smithi178 ceph-mon[33359]: pgmap v886: 129 pgs: 129 active+clean; 383 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 3.3 MiB/s wr, 266 op/s 2024-05-10T11:33:54.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:33:54 smithi100 ceph-mon[30296]: pgmap v886: 129 pgs: 129 active+clean; 383 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 3.3 MiB/s wr, 266 op/s 2024-05-10T11:33:54.914 INFO:tasks.workunit.client.0.smithi100.stdout: 10 18424 10.81 MB/sec warmup 108 sec latency 140.145 ms 2024-05-10T11:33:55.913 INFO:tasks.workunit.client.0.smithi100.stdout: 10 18575 10.83 MB/sec warmup 109 sec latency 139.242 ms 2024-05-10T11:33:56.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.078+0000 7fdb042bf700 1 -- 172.21.15.100:0/2306867337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdafc103ee0 msgr2=0x7fdafc110370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:56.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.078+0000 7fdb042bf700 1 --2- 172.21.15.100:0/2306867337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdafc103ee0 0x7fdafc110370 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fdaf400b0a0 tx=0x7fdaf400b3b0 comp rx=0 tx=0).stop 2024-05-10T11:33:56.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.078+0000 7fdb042bf700 1 -- 172.21.15.100:0/2306867337 shutdown_connections 2024-05-10T11:33:56.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.078+0000 7fdb042bf700 1 --2- 172.21.15.100:0/2306867337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdafc103ee0 0x7fdafc110370 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:56.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.078+0000 7fdb042bf700 1 --2- 172.21.15.100:0/2306867337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdafc109530 0x7fdafc109910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:56.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.078+0000 7fdb042bf700 1 --2- 172.21.15.100:0/2306867337 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdafc103520 0x7fdafc1039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:56.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.078+0000 7fdb042bf700 1 -- 172.21.15.100:0/2306867337 >> 172.21.15.100:0/2306867337 conn(0x7fdafc07a570 msgr2=0x7fdafc07a980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:33:56.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.078+0000 7fdb042bf700 1 -- 172.21.15.100:0/2306867337 shutdown_connections 2024-05-10T11:33:56.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.079+0000 7fdb042bf700 1 -- 172.21.15.100:0/2306867337 wait complete. 2024-05-10T11:33:56.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.079+0000 7fdb042bf700 1 Processor -- start 2024-05-10T11:33:56.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.079+0000 7fdb042bf700 1 -- start start 2024-05-10T11:33:56.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.080+0000 7fdb042bf700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdafc103520 0x7fdafc10f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:56.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.080+0000 7fdb042bf700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdafc103ee0 0x7fdafc10f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:56.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.080+0000 7fdb042bf700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdafc109530 0x7fdafc1a4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:56.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.080+0000 7fdb0205b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdafc103520 0x7fdafc10f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:56.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.080+0000 7fdb042bf700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdafc113580 con 0x7fdafc103520 2024-05-10T11:33:56.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.080+0000 7fdb042bf700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdafc113400 con 0x7fdafc103ee0 2024-05-10T11:33:56.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.080+0000 7fdb042bf700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fdafc113700 con 0x7fdafc109530 2024-05-10T11:33:56.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.080+0000 7fdb0205b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdafc103520 0x7fdafc10f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51608/0 (socket says 172.21.15.100:51608) 2024-05-10T11:33:56.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.080+0000 7fdb0205b700 1 -- 172.21.15.100:0/3987037015 learned_addr learned my addr 172.21.15.100:0/3987037015 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:33:56.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.080+0000 7fdb0185a700 1 --2- 172.21.15.100:0/3987037015 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdafc103ee0 0x7fdafc10f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:56.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.080+0000 7fdb0285c700 1 --2- 172.21.15.100:0/3987037015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdafc109530 0x7fdafc1a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:56.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.081+0000 7fdb0205b700 1 -- 172.21.15.100:0/3987037015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdafc109530 msgr2=0x7fdafc1a4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:56.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.081+0000 7fdb0205b700 1 --2- 172.21.15.100:0/3987037015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdafc109530 0x7fdafc1a4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:56.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.081+0000 7fdb0205b700 1 -- 172.21.15.100:0/3987037015 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdafc103ee0 msgr2=0x7fdafc10f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:56.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.081+0000 7fdb0205b700 1 --2- 172.21.15.100:0/3987037015 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdafc103ee0 0x7fdafc10f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:56.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.081+0000 7fdb0205b700 1 -- 172.21.15.100:0/3987037015 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdafc1a4e00 con 0x7fdafc103520 2024-05-10T11:33:56.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.081+0000 7fdb0185a700 1 --2- 172.21.15.100:0/3987037015 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdafc103ee0 0x7fdafc10f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:33:56.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.081+0000 7fdb0205b700 1 --2- 172.21.15.100:0/3987037015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdafc103520 0x7fdafc10f420 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fdaf800e9c0 tx=0x7fdaf800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:33:56.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.081+0000 7fdaf37fe700 1 -- 172.21.15.100:0/3987037015 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf800cbd0 con 0x7fdafc103520 2024-05-10T11:33:56.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.081+0000 7fdaf37fe700 1 -- 172.21.15.100:0/3987037015 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdaf800ee60 con 0x7fdafc103520 2024-05-10T11:33:56.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.081+0000 7fdaf37fe700 1 -- 172.21.15.100:0/3987037015 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf80054b0 con 0x7fdafc103520 2024-05-10T11:33:56.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.081+0000 7fdb042bf700 1 -- 172.21.15.100:0/3987037015 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdafc1a5090 con 0x7fdafc103520 2024-05-10T11:33:56.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.081+0000 7fdb042bf700 1 -- 172.21.15.100:0/3987037015 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdafc1ac990 con 0x7fdafc103520 2024-05-10T11:33:56.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.086+0000 7fdaf37fe700 1 -- 172.21.15.100:0/3987037015 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fdaf8004750 con 0x7fdafc103520 2024-05-10T11:33:56.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.087+0000 7fdb0eb52b80 1 Processor -- start 2024-05-10T11:33:56.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.087+0000 7fdb0eb52b80 1 -- start start 2024-05-10T11:33:56.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.087+0000 7fdaf37fe700 1 --2- 172.21.15.100:0/3987037015 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fdae80b2900 0x7fdae80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:56.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.088+0000 7fdaf37fe700 1 -- 172.21.15.100:0/3987037015 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fdaf80e2800 con 0x7fdafc103520 2024-05-10T11:33:56.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.088+0000 7fdb0eb52b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eca0a911b0 0x55eca0a91590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:56.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.088+0000 7fdb0eb52b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eca0a91ad0 0x55eca0a8b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:56.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.088+0000 7fdb0eb52b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eca0a8bd50 0x55eca0a8c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:56.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.088+0000 7fdb0eb52b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55eca09dd940 con 0x55eca0a911b0 2024-05-10T11:33:56.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.088+0000 7fdb0eb52b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55eca0942da0 con 0x55eca0a8bd50 2024-05-10T11:33:56.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.088+0000 7fdb0eb52b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55eca09d9900 con 0x55eca0a91ad0 2024-05-10T11:33:56.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.088+0000 7fdb0285c700 1 --2- 172.21.15.100:0/3987037015 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fdae80b2900 0x7fdae80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:56.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.088+0000 7fdb0185a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eca0a91ad0 0x55eca0a8b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:56.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.088+0000 7fdb0285c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eca0a8bd50 0x55eca0a8c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:56.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.088+0000 7fdb0185a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eca0a91ad0 0x55eca0a8b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:54512/0 (socket says 172.21.15.100:54512) 2024-05-10T11:33:56.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.088+0000 7fdb0185a700 1 -- 172.21.15.100:0/3348030902 learned_addr learned my addr 172.21.15.100:0/3348030902 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:33:56.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.088+0000 7fdb0205b700 1 --2- 172.21.15.100:0/3348030902 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eca0a911b0 0x55eca0a91590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:56.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.088+0000 7fdb0285c700 1 --2- 172.21.15.100:0/3987037015 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fdae80b2900 0x7fdae80b4dc0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fdaf4009e60 tx=0x7fdaf40095d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:33:56.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.089+0000 7fdb0285c700 1 -- 172.21.15.100:0/3348030902 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eca0a91ad0 msgr2=0x55eca0a8b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:56.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.089+0000 7fdb0285c700 1 --2- 172.21.15.100:0/3348030902 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eca0a91ad0 0x55eca0a8b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:56.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.089+0000 7fdb0285c700 1 -- 172.21.15.100:0/3348030902 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eca0a911b0 msgr2=0x55eca0a91590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:56.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.089+0000 7fdb0285c700 1 --2- 172.21.15.100:0/3348030902 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eca0a911b0 0x55eca0a91590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:56.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.089+0000 7fdb0285c700 1 -- 172.21.15.100:0/3348030902 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eca092a8d0 con 0x55eca0a8bd50 2024-05-10T11:33:56.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.089+0000 7fdb0185a700 1 --2- 172.21.15.100:0/3348030902 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eca0a91ad0 0x55eca0a8b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:33:56.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.089+0000 7fdb0285c700 1 --2- 172.21.15.100:0/3348030902 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eca0a8bd50 0x55eca0a8c190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fdaf40062a0 tx=0x7fdaf4028000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48df272c418e2810 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:33:56.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.089+0000 7fdaf0d70700 1 -- 172.21.15.100:0/3348030902 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf40076e0 con 0x55eca0a8bd50 2024-05-10T11:33:56.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.089+0000 7fdaf0d70700 1 -- 172.21.15.100:0/3348030902 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdaf4007d10 con 0x55eca0a8bd50 2024-05-10T11:33:56.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.089+0000 7fdaf0d70700 1 -- 172.21.15.100:0/3348030902 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf4005680 con 0x55eca0a8bd50 2024-05-10T11:33:56.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.090+0000 7fdb0eb52b80 1 -- 172.21.15.100:0/3348030902 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eca0a8bd50 msgr2=0x55eca0a8c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:56.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.090+0000 7fdb0eb52b80 1 --2- 172.21.15.100:0/3348030902 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eca0a8bd50 0x55eca0a8c190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fdaf40062a0 tx=0x7fdaf4028000 comp rx=0 tx=0).stop 2024-05-10T11:33:56.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.090+0000 7fdb0eb52b80 1 -- 172.21.15.100:0/3348030902 shutdown_connections 2024-05-10T11:33:56.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.090+0000 7fdb0eb52b80 1 --2- 172.21.15.100:0/3348030902 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eca0a8bd50 0x55eca0a8c190 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:56.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.090+0000 7fdb0eb52b80 1 --2- 172.21.15.100:0/3348030902 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eca0a911b0 0x55eca0a91590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:56.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.090+0000 7fdb0eb52b80 1 --2- 172.21.15.100:0/3348030902 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eca0a91ad0 0x55eca0a8b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:56.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.090+0000 7fdb0eb52b80 1 -- 172.21.15.100:0/3348030902 >> 172.21.15.100:0/3348030902 conn(0x55eca09b7b70 msgr2=0x55eca0a862a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:33:56.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.090+0000 7fdb0eb52b80 1 -- 172.21.15.100:0/3348030902 shutdown_connections 2024-05-10T11:33:56.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.090+0000 7fdb0eb52b80 1 -- 172.21.15.100:0/3348030902 wait complete. 2024-05-10T11:33:56.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.091+0000 7fdb0eb52b80 1 -- start start 2024-05-10T11:33:56.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.091+0000 7fdb0eb52b80 1 Processor -- start 2024-05-10T11:33:56.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.091+0000 7fdb0eb52b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eca0a8bd50 0x55eca0a977a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:56.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.091+0000 7fdb0eb52b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eca0a911b0 0x55eca0a94b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:56.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.091+0000 7fdb0eb52b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eca0a91ad0 0x55eca0a95080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:56.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.091+0000 7fdb0eb52b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55eca0a98390 con 0x55eca0a8bd50 2024-05-10T11:33:56.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.091+0000 7fdb0eb52b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55eca0996ed0 con 0x55eca0a91ad0 2024-05-10T11:33:56.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.091+0000 7fdb0eb52b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55eca0a98510 con 0x55eca0a911b0 2024-05-10T11:33:56.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.091+0000 7fdb0eb52b80 10 client.0 _ll_register_callbacks cb 0x55eca09715b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:33:56.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.091+0000 7fdb0185a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eca0a911b0 0x55eca0a94b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:56.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.091+0000 7fdb0185a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eca0a911b0 0x55eca0a94b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:54526/0 (socket says 172.21.15.100:54526) 2024-05-10T11:33:56.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.091+0000 7fdb0285c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eca0a91ad0 0x55eca0a95080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:56.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.091+0000 7fdb0185a700 1 -- 172.21.15.100:0/2486403274 learned_addr learned my addr 172.21.15.100:0/2486403274 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:33:56.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.092+0000 7fdb0205b700 1 --2- 172.21.15.100:0/2486403274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eca0a8bd50 0x55eca0a977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:56.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.092+0000 7fdb0185a700 1 -- 172.21.15.100:0/2486403274 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eca0a91ad0 msgr2=0x55eca0a95080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:56.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.092+0000 7fdb0185a700 1 --2- 172.21.15.100:0/2486403274 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eca0a91ad0 0x55eca0a95080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:56.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.092+0000 7fdb0185a700 1 -- 172.21.15.100:0/2486403274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eca0a8bd50 msgr2=0x55eca0a977a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:56.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.092+0000 7fdb0185a700 1 --2- 172.21.15.100:0/2486403274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eca0a8bd50 0x55eca0a977a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:56.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.092+0000 7fdb0185a700 1 -- 172.21.15.100:0/2486403274 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eca0994760 con 0x55eca0a911b0 2024-05-10T11:33:56.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.092+0000 7fdb0285c700 1 --2- 172.21.15.100:0/2486403274 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eca0a91ad0 0x55eca0a95080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:33:56.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.092+0000 7fdb0205b700 1 --2- 172.21.15.100:0/2486403274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eca0a8bd50 0x55eca0a977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:33:56.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.092+0000 7fdae5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:33:56.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.092+0000 7fdb0185a700 1 --2- 172.21.15.100:0/2486403274 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eca0a911b0 0x55eca0a94b40 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fdaec012d00 tx=0x7fdaec00f6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d933a75037976de server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:33:56.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.092+0000 7fdae5ffb700 1 -- 172.21.15.100:0/2486403274 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec00fc30 con 0x55eca0a911b0 2024-05-10T11:33:56.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.092+0000 7fdae5ffb700 1 -- 172.21.15.100:0/2486403274 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fdaec00fdd0 con 0x55eca0a911b0 2024-05-10T11:33:56.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.092+0000 7fdb0eb52b80 1 -- 172.21.15.100:0/2486403274 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdaf4007d10 con 0x55eca0a911b0 2024-05-10T11:33:56.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.093+0000 7fdae5ffb700 1 -- 172.21.15.100:0/2486403274 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fdaec004830 con 0x55eca0a911b0 2024-05-10T11:33:56.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.093+0000 7fdb0eb52b80 10 client.27913 fetch_fsmap learned FSMap version 37 2024-05-10T11:33:56.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.093+0000 7fdb0eb52b80 1 -- 172.21.15.100:0/2486403274 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55eca0aa2d30 con 0x55eca0a911b0 2024-05-10T11:33:56.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.093+0000 7fdae5ffb700 1 -- 172.21.15.100:0/2486403274 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdaf4007d10 con 0x55eca0a911b0 2024-05-10T11:33:56.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.093+0000 7fdae5ffb700 1 -- 172.21.15.100:0/2486403274 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7fdaec005510 con 0x55eca0a911b0 2024-05-10T11:33:56.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.093+0000 7fdb0eb52b80 10 client.27913 fetch_fsmap finished waiting for FSMap version 37 2024-05-10T11:33:56.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.093+0000 7fdb0eb52b80 10 client.27913 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-10T11:33:56.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.093+0000 7fdb0eb52b80 20 client.27913 populate_metadata read hostname 'smithi100' 2024-05-10T11:33:56.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.093+0000 7fdb0eb52b80 1 --2- 172.21.15.100:0/2486403274 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55eca0aa5370 0x55eca0aa7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:56.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.093+0000 7fdb0eb52b80 4 client.27913 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:33:56.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.093+0000 7fdb0eb52b80 1 -- 172.21.15.100:0/2486403274 --> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eca09eb320 con 0x55eca0aa5370 2024-05-10T11:33:56.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.094+0000 7fdb0205b700 1 --2- 172.21.15.100:0/2486403274 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55eca0aa5370 0x55eca0aa7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:56.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.094+0000 7fdae5ffb700 10 client.27913 ms_handle_connect on v2:172.21.15.188:6836/1943342620 2024-05-10T11:33:56.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.094+0000 7fdb0205b700 1 --2- 172.21.15.100:0/2486403274 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55eca0aa5370 0x55eca0aa7760 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d7a0ff45372dc843 server_cookie=522f9f87a067ce91 in_seq=0 out_seq=0 2024-05-10T11:33:56.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.096+0000 7fdae5ffb700 1 -- 172.21.15.100:0/2486403274 <== mds.2 v2:172.21.15.188:6836/1943342620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55eca09eb320 con 0x55eca0aa5370 2024-05-10T11:33:56.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.096+0000 7fdae5ffb700 10 client.27913 handle_command_reply: tid=0 2024-05-10T11:33:56.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.096+0000 7fdb0eb52b80 1 client.27913 shutdown 2024-05-10T11:33:56.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.096+0000 7fdb0eb52b80 1 -- 172.21.15.100:0/2486403274 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eca0a911b0 msgr2=0x55eca0a94b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:56.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.096+0000 7fdb0eb52b80 1 --2- 172.21.15.100:0/2486403274 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eca0a911b0 0x55eca0a94b40 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fdaec012d00 tx=0x7fdaec00f6a0 comp rx=0 tx=0).stop 2024-05-10T11:33:56.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.096+0000 7fdb0eb52b80 1 -- 172.21.15.100:0/2486403274 shutdown_connections 2024-05-10T11:33:56.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.096+0000 7fdb0eb52b80 1 --2- 172.21.15.100:0/2486403274 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55eca0aa5370 0x55eca0aa7760 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:56.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.096+0000 7fdae5ffb700 0 client.27913 ms_handle_reset on v2:172.21.15.188:6836/1943342620 2024-05-10T11:33:56.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.096+0000 7fdb0eb52b80 1 --2- 172.21.15.100:0/2486403274 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eca0a91ad0 0x55eca0a95080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:56.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.096+0000 7fdb0eb52b80 1 --2- 172.21.15.100:0/2486403274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eca0a8bd50 0x55eca0a977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:56.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.096+0000 7fdb0eb52b80 1 --2- 172.21.15.100:0/2486403274 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eca0a911b0 0x55eca0a94b40 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:56.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.096+0000 7fdb0eb52b80 1 -- 172.21.15.100:0/2486403274 >> 172.21.15.100:0/2486403274 conn(0x55eca09b7b70 msgr2=0x55eca0a80b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:33:56.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.096+0000 7fdb0eb52b80 1 -- 172.21.15.100:0/2486403274 shutdown_connections 2024-05-10T11:33:56.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.096+0000 7fdb0eb52b80 1 -- 172.21.15.100:0/2486403274 wait complete. 2024-05-10T11:33:56.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.096+0000 7fdb0eb52b80 20 client.27913 trim_cache size 0 max 16384 2024-05-10T11:33:56.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.121+0000 7fdb0eb52b80 1 Processor -- start 2024-05-10T11:33:56.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.121+0000 7fdb0eb52b80 1 -- start start 2024-05-10T11:33:56.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.121+0000 7fdb0eb52b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eca0a8bd50 0x55eca0a8b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:56.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.121+0000 7fdb0eb52b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eca0a911b0 0x55eca0a8f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:56.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.121+0000 7fdb0eb52b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eca0a91ad0 0x55eca0aa1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:56.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.121+0000 7fdb0eb52b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdafc1b7d30 con 0x55eca0a91ad0 2024-05-10T11:33:56.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.121+0000 7fdb0eb52b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdafc1b5530 con 0x55eca0a8bd50 2024-05-10T11:33:56.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.121+0000 7fdb0eb52b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fdafc1b3380 con 0x55eca0a911b0 2024-05-10T11:33:56.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.121+0000 7fdb0285c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eca0a91ad0 0x55eca0aa1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:56.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.121+0000 7fdb0285c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eca0a91ad0 0x55eca0aa1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51638/0 (socket says 172.21.15.100:51638) 2024-05-10T11:33:56.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.121+0000 7fdb0285c700 1 -- 172.21.15.100:0/3106001895 learned_addr learned my addr 172.21.15.100:0/3106001895 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:33:56.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.121+0000 7fdb0185a700 1 --2- 172.21.15.100:0/3106001895 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eca0a911b0 0x55eca0a8f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:56.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.121+0000 7fdb0205b700 1 --2- 172.21.15.100:0/3106001895 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eca0a8bd50 0x55eca0a8b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:56.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.122+0000 7fdb0285c700 1 -- 172.21.15.100:0/3106001895 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eca0a911b0 msgr2=0x55eca0a8f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:56.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.122+0000 7fdb0285c700 1 --2- 172.21.15.100:0/3106001895 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eca0a911b0 0x55eca0a8f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:56.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.122+0000 7fdb0285c700 1 -- 172.21.15.100:0/3106001895 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eca0a8bd50 msgr2=0x55eca0a8b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:56.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.122+0000 7fdb0285c700 1 --2- 172.21.15.100:0/3106001895 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eca0a8bd50 0x55eca0a8b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:56.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.122+0000 7fdb0285c700 1 -- 172.21.15.100:0/3106001895 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eca0aa7cf0 con 0x55eca0a91ad0 2024-05-10T11:33:56.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.122+0000 7fdb0285c700 1 --2- 172.21.15.100:0/3106001895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eca0a91ad0 0x55eca0aa1450 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fdaf4009d10 tx=0x7fdaf400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60c313e6cb945fa0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:33:56.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.122+0000 7fdae7fff700 1 -- 172.21.15.100:0/3106001895 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf40053c0 con 0x55eca0a91ad0 2024-05-10T11:33:56.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.122+0000 7fdae7fff700 1 -- 172.21.15.100:0/3106001895 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdaf40059f0 con 0x55eca0a91ad0 2024-05-10T11:33:56.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.122+0000 7fdae7fff700 1 -- 172.21.15.100:0/3106001895 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf402b450 con 0x55eca0a91ad0 2024-05-10T11:33:56.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.123+0000 7fdb0eb52b80 1 -- 172.21.15.100:0/3106001895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eca0a91ad0 msgr2=0x55eca0aa1450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:56.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.123+0000 7fdb0eb52b80 1 --2- 172.21.15.100:0/3106001895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eca0a91ad0 0x55eca0aa1450 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fdaf4009d10 tx=0x7fdaf400b8b0 comp rx=0 tx=0).stop 2024-05-10T11:33:56.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.123+0000 7fdb0eb52b80 1 -- 172.21.15.100:0/3106001895 shutdown_connections 2024-05-10T11:33:56.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.123+0000 7fdb0eb52b80 1 --2- 172.21.15.100:0/3106001895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eca0a91ad0 0x55eca0aa1450 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:56.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.123+0000 7fdb0eb52b80 1 --2- 172.21.15.100:0/3106001895 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eca0a8bd50 0x55eca0a8b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:56.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.123+0000 7fdb0eb52b80 1 --2- 172.21.15.100:0/3106001895 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eca0a911b0 0x55eca0a8f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:56.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.123+0000 7fdb0eb52b80 1 -- 172.21.15.100:0/3106001895 >> 172.21.15.100:0/3106001895 conn(0x55eca09b7b70 msgr2=0x55eca0a91eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:33:56.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.123+0000 7fdb0eb52b80 1 -- 172.21.15.100:0/3106001895 shutdown_connections 2024-05-10T11:33:56.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.123+0000 7fdb0eb52b80 1 -- 172.21.15.100:0/3106001895 wait complete. 2024-05-10T11:33:56.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.124+0000 7fdb0eb52b80 1 -- start start 2024-05-10T11:33:56.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.124+0000 7fdb0eb52b80 1 Processor -- start 2024-05-10T11:33:56.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.124+0000 7fdb0eb52b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eca0a8bd50 0x55eca0aac7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:56.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.124+0000 7fdb0eb52b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eca0a911b0 0x55eca0aacd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:56.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.124+0000 7fdb0eb52b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eca0a91ad0 0x55eca0aad250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:56.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.124+0000 7fdb0eb52b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdafc1b39f0 con 0x55eca0a91ad0 2024-05-10T11:33:56.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.124+0000 7fdb0eb52b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdafc1b4e60 con 0x55eca0a911b0 2024-05-10T11:33:56.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.124+0000 7fdb0285c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eca0a91ad0 0x55eca0aad250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:56.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.124+0000 7fdb0285c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eca0a91ad0 0x55eca0aad250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51656/0 (socket says 172.21.15.100:51656) 2024-05-10T11:33:56.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.124+0000 7fdb0285c700 1 -- 172.21.15.100:0/688529451 learned_addr learned my addr 172.21.15.100:0/688529451 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:33:56.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.124+0000 7fdb0205b700 1 --2- 172.21.15.100:0/688529451 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eca0a8bd50 0x55eca0aac7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:56.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.124+0000 7fdb0185a700 1 --2- 172.21.15.100:0/688529451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eca0a911b0 0x55eca0aacd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:56.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.124+0000 7fdb0eb52b80 1 -- 172.21.15.100:0/688529451 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fdafc1b40c0 con 0x55eca0a8bd50 2024-05-10T11:33:56.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.125+0000 7fdb0eb52b80 10 client.0 _ll_register_callbacks cb 0x55eca09715b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:33:56.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.125+0000 7fdb0285c700 1 -- 172.21.15.100:0/688529451 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eca0a8bd50 msgr2=0x55eca0aac7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:56.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.125+0000 7fdb0285c700 1 --2- 172.21.15.100:0/688529451 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eca0a8bd50 0x55eca0aac7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:56.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.125+0000 7fdb0285c700 1 -- 172.21.15.100:0/688529451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eca0a911b0 msgr2=0x55eca0aacd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:56.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.125+0000 7fdb0285c700 1 --2- 172.21.15.100:0/688529451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eca0a911b0 0x55eca0aacd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:56.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.125+0000 7fdb0285c700 1 -- 172.21.15.100:0/688529451 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eca0b0eb80 con 0x55eca0a91ad0 2024-05-10T11:33:56.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.125+0000 7fdb0185a700 1 --2- 172.21.15.100:0/688529451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eca0a911b0 0x55eca0aacd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:33:56.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.125+0000 7fdb0285c700 1 --2- 172.21.15.100:0/688529451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eca0a91ad0 0x55eca0aad250 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fdaf4007740 tx=0x7fdaf4007770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51b1fac7b0c7ee90 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:33:56.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.125+0000 7fdae5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:33:56.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.125+0000 7fdae5ffb700 1 -- 172.21.15.100:0/688529451 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf4038030 con 0x55eca0a91ad0 2024-05-10T11:33:56.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.125+0000 7fdae5ffb700 1 -- 172.21.15.100:0/688529451 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fdaf4031050 con 0x55eca0a91ad0 2024-05-10T11:33:56.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.125+0000 7fdb0eb52b80 1 -- 172.21.15.100:0/688529451 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdaf40059f0 con 0x55eca0a91ad0 2024-05-10T11:33:56.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.126+0000 7fdae5ffb700 1 -- 172.21.15.100:0/688529451 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fdaf40055d0 con 0x55eca0a91ad0 2024-05-10T11:33:56.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.126+0000 7fdb0eb52b80 10 client.18489 fetch_fsmap learned FSMap version 37 2024-05-10T11:33:56.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.126+0000 7fdb0eb52b80 1 -- 172.21.15.100:0/688529451 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55eca0b0fe40 con 0x55eca0a91ad0 2024-05-10T11:33:56.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.126+0000 7fdae5ffb700 1 -- 172.21.15.100:0/688529451 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdaf4005770 con 0x55eca0a91ad0 2024-05-10T11:33:56.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.127+0000 7fdae5ffb700 1 -- 172.21.15.100:0/688529451 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7fdaf4011070 con 0x55eca0a91ad0 2024-05-10T11:33:56.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.127+0000 7fdb0eb52b80 10 client.18489 fetch_fsmap finished waiting for FSMap version 37 2024-05-10T11:33:56.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.127+0000 7fdb0eb52b80 10 client.18489 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-10T11:33:56.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.127+0000 7fdb0eb52b80 20 client.18489 populate_metadata read hostname 'smithi100' 2024-05-10T11:33:56.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.127+0000 7fdb0eb52b80 1 --2- 172.21.15.100:0/688529451 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55eca0b100c0 0x55eca0b124b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:56.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.127+0000 7fdb0eb52b80 4 client.18489 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:33:56.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.127+0000 7fdb0eb52b80 1 -- 172.21.15.100:0/688529451 --> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fdaf40059f0 con 0x55eca0b100c0 2024-05-10T11:33:56.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.127+0000 7fdb0205b700 1 --2- 172.21.15.100:0/688529451 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55eca0b100c0 0x55eca0b124b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:56.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.128+0000 7fdb0205b700 1 --2- 172.21.15.100:0/688529451 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55eca0b100c0 0x55eca0b124b0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6b32d4c5ff948970 server_cookie=c2f2c440149cd2f5 in_seq=0 out_seq=0 2024-05-10T11:33:56.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.128+0000 7fdae5ffb700 10 client.18489 ms_handle_connect on v2:172.21.15.188:6836/1943342620 2024-05-10T11:33:56.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.128+0000 7fdae5ffb700 1 -- 172.21.15.100:0/688529451 <== mds.2 v2:172.21.15.188:6836/1943342620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fdaf40059f0 con 0x55eca0b100c0 2024-05-10T11:33:56.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.128+0000 7fdae5ffb700 10 client.18489 handle_command_reply: tid=0 2024-05-10T11:33:56.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.128+0000 7fdb0eb52b80 1 client.18489 shutdown 2024-05-10T11:33:56.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.128+0000 7fdb0eb52b80 1 -- 172.21.15.100:0/688529451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eca0a91ad0 msgr2=0x55eca0aad250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:56.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.128+0000 7fdb0eb52b80 1 --2- 172.21.15.100:0/688529451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eca0a91ad0 0x55eca0aad250 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fdaf4007740 tx=0x7fdaf4007770 comp rx=0 tx=0).stop 2024-05-10T11:33:56.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.128+0000 7fdb0eb52b80 1 -- 172.21.15.100:0/688529451 shutdown_connections 2024-05-10T11:33:56.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.128+0000 7fdb0eb52b80 1 --2- 172.21.15.100:0/688529451 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55eca0b100c0 0x55eca0b124b0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:56.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.128+0000 7fdb0eb52b80 1 --2- 172.21.15.100:0/688529451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eca0a91ad0 0x55eca0aad250 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:56.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.128+0000 7fdb0eb52b80 1 --2- 172.21.15.100:0/688529451 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eca0a8bd50 0x55eca0aac7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:56.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.128+0000 7fdb0eb52b80 1 --2- 172.21.15.100:0/688529451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eca0a911b0 0x55eca0aacd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:56.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.128+0000 7fdae5ffb700 0 client.18489 ms_handle_reset on v2:172.21.15.188:6836/1943342620 2024-05-10T11:33:56.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.128+0000 7fdb0eb52b80 1 -- 172.21.15.100:0/688529451 >> 172.21.15.100:0/688529451 conn(0x55eca09b7b70 msgr2=0x55eca0a8fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:33:56.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.128+0000 7fdb0eb52b80 1 -- 172.21.15.100:0/688529451 shutdown_connections 2024-05-10T11:33:56.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.128+0000 7fdb0eb52b80 1 -- 172.21.15.100:0/688529451 wait complete. 2024-05-10T11:33:56.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.128+0000 7fdb0eb52b80 20 client.18489 trim_cache size 0 max 16384 2024-05-10T11:33:56.132 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:33:56.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.129+0000 7fdb042bf700 1 -- 172.21.15.100:0/3987037015 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fdae80b2900 msgr2=0x7fdae80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:56.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.129+0000 7fdb042bf700 1 --2- 172.21.15.100:0/3987037015 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fdae80b2900 0x7fdae80b4dc0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fdaf4009e60 tx=0x7fdaf40095d0 comp rx=0 tx=0).stop 2024-05-10T11:33:56.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.129+0000 7fdb042bf700 1 -- 172.21.15.100:0/3987037015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdafc103520 msgr2=0x7fdafc10f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:56.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.129+0000 7fdb042bf700 1 --2- 172.21.15.100:0/3987037015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdafc103520 0x7fdafc10f420 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fdaf800e9c0 tx=0x7fdaf800ecd0 comp rx=0 tx=0).stop 2024-05-10T11:33:56.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.129+0000 7fdb042bf700 1 -- 172.21.15.100:0/3987037015 shutdown_connections 2024-05-10T11:33:56.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.129+0000 7fdb042bf700 1 --2- 172.21.15.100:0/3987037015 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fdae80b2900 0x7fdae80b4dc0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:56.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.129+0000 7fdb042bf700 1 --2- 172.21.15.100:0/3987037015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdafc109530 0x7fdafc1a4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:56.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.129+0000 7fdb042bf700 1 --2- 172.21.15.100:0/3987037015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdafc103520 0x7fdafc10f420 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:56.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.129+0000 7fdb042bf700 1 --2- 172.21.15.100:0/3987037015 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdafc103ee0 0x7fdafc10f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:56.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.129+0000 7fdb042bf700 1 -- 172.21.15.100:0/3987037015 >> 172.21.15.100:0/3987037015 conn(0x7fdafc07a570 msgr2=0x7fdafc101d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:33:56.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.130+0000 7fdb042bf700 1 -- 172.21.15.100:0/3987037015 shutdown_connections 2024-05-10T11:33:56.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:56.130+0000 7fdb042bf700 1 -- 172.21.15.100:0/3987037015 wait complete. 2024-05-10T11:33:56.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:33:56 smithi100 ceph-mon[30296]: pgmap v887: 129 pgs: 129 active+clean; 388 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 3.6 MiB/s wr, 253 op/s 2024-05-10T11:33:56.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:33:56 smithi178 ceph-mon[33359]: pgmap v887: 129 pgs: 129 active+clean; 388 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 3.6 MiB/s wr, 253 op/s 2024-05-10T11:33:56.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:33:56 smithi188 ceph-mon[34075]: pgmap v887: 129 pgs: 129 active+clean; 388 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 3.6 MiB/s wr, 253 op/s 2024-05-10T11:33:56.914 INFO:tasks.workunit.client.0.smithi100.stdout: 10 18738 10.80 MB/sec warmup 110 sec latency 67.955 ms 2024-05-10T11:33:56.930 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-10T11:33:57.284 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:33:57.614 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.614+0000 7f3a83512640 1 -- 172.21.15.100:0/1526393921 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a7c101950 msgr2=0x7f3a7c101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:57.614 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.614+0000 7f3a83512640 1 --2- 172.21.15.100:0/1526393921 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a7c101950 0x7f3a7c101d70 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f3a78009900 tx=0x7f3a7802a410 comp rx=0 tx=0).stop 2024-05-10T11:33:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.614+0000 7f3a83512640 1 -- 172.21.15.100:0/1526393921 shutdown_connections 2024-05-10T11:33:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.614+0000 7f3a83512640 1 --2- 172.21.15.100:0/1526393921 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a7c103500 0x7f3a7c109d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.614+0000 7f3a83512640 1 --2- 172.21.15.100:0/1526393921 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a7c102b40 0x7f3a7c102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.614+0000 7f3a83512640 1 --2- 172.21.15.100:0/1526393921 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a7c101950 0x7f3a7c101d70 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.614+0000 7f3a83512640 1 -- 172.21.15.100:0/1526393921 >> 172.21.15.100:0/1526393921 conn(0x7f3a7c0fcec0 msgr2=0x7f3a7c0ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:33:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.614+0000 7f3a83512640 1 -- 172.21.15.100:0/1526393921 shutdown_connections 2024-05-10T11:33:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.614+0000 7f3a83512640 1 -- 172.21.15.100:0/1526393921 wait complete. 2024-05-10T11:33:57.616 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.615+0000 7f3a83512640 1 Processor -- start 2024-05-10T11:33:57.616 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.615+0000 7f3a83512640 1 -- start start 2024-05-10T11:33:57.616 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.615+0000 7f3a83512640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a7c101950 0x7f3a7c10f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:57.616 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.615+0000 7f3a83512640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a7c102b40 0x7f3a7c10de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:57.617 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.615+0000 7f3a83512640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a7c103500 0x7f3a7c10e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:57.617 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.615+0000 7f3a83512640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7c10d300 con 0x7f3a7c103500 2024-05-10T11:33:57.617 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.615+0000 7f3a83512640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7c10d180 con 0x7f3a7c101950 2024-05-10T11:33:57.617 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.615+0000 7f3a83512640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7c10d480 con 0x7f3a7c102b40 2024-05-10T11:33:57.617 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.616+0000 7f3a812ae640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a7c101950 0x7f3a7c10f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:57.617 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.616+0000 7f3a81aaf640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a7c103500 0x7f3a7c10e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:57.617 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.616+0000 7f3a81aaf640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a7c103500 0x7f3a7c10e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39652/0 (socket says 172.21.15.100:39652) 2024-05-10T11:33:57.617 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.616+0000 7f3a81aaf640 1 -- 172.21.15.100:0/3689677066 learned_addr learned my addr 172.21.15.100:0/3689677066 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:33:57.617 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.616+0000 7f3a80aad640 1 --2- 172.21.15.100:0/3689677066 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a7c102b40 0x7f3a7c10de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:57.618 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.616+0000 7f3a81aaf640 1 -- 172.21.15.100:0/3689677066 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a7c102b40 msgr2=0x7f3a7c10de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:57.618 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.616+0000 7f3a81aaf640 1 --2- 172.21.15.100:0/3689677066 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a7c102b40 0x7f3a7c10de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:57.618 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.616+0000 7f3a81aaf640 1 -- 172.21.15.100:0/3689677066 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a7c101950 msgr2=0x7f3a7c10f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:57.618 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.616+0000 7f3a81aaf640 1 --2- 172.21.15.100:0/3689677066 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a7c101950 0x7f3a7c10f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:57.618 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.616+0000 7f3a81aaf640 1 -- 172.21.15.100:0/3689677066 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a7c19ca90 con 0x7f3a7c103500 2024-05-10T11:33:57.618 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.616+0000 7f3a81aaf640 1 --2- 172.21.15.100:0/3689677066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a7c103500 0x7f3a7c10e3d0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f3a7400eb10 tx=0x7f3a7400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:33:57.618 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.616+0000 7f3a727fc640 1 -- 172.21.15.100:0/3689677066 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a74004b20 con 0x7f3a7c103500 2024-05-10T11:33:57.618 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.617+0000 7f3a727fc640 1 -- 172.21.15.100:0/3689677066 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3a74004cc0 con 0x7f3a7c103500 2024-05-10T11:33:57.618 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.617+0000 7f3a83512640 1 -- 172.21.15.100:0/3689677066 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a7c19cd80 con 0x7f3a7c103500 2024-05-10T11:33:57.618 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.617+0000 7f3a727fc640 1 -- 172.21.15.100:0/3689677066 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a74010690 con 0x7f3a7c103500 2024-05-10T11:33:57.619 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.617+0000 7f3a83512640 1 -- 172.21.15.100:0/3689677066 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a7c19d280 con 0x7f3a7c103500 2024-05-10T11:33:57.620 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.620+0000 7f3a727fc640 1 -- 172.21.15.100:0/3689677066 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f3a74005020 con 0x7f3a7c103500 2024-05-10T11:33:57.622 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.621+0000 7f3a8622a740 1 Processor -- start 2024-05-10T11:33:57.622 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.621+0000 7f3a727fc640 1 --2- 172.21.15.100:0/3689677066 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f3a680b28a0 0x7f3a680b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:57.622 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.621+0000 7f3a8622a740 1 -- start start 2024-05-10T11:33:57.622 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.621+0000 7f3a8622a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b222ca7db0 0x55b222ca81d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:57.622 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.621+0000 7f3a8622a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b222ca87a0 0x55b222cad2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:57.622 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.621+0000 7f3a8622a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b222cad910 0x55b222cafda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:57.622 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.621+0000 7f3a8622a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55b222c0c5d0 con 0x55b222ca7db0 2024-05-10T11:33:57.622 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.621+0000 7f3a8622a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b222bc5890 con 0x55b222cad910 2024-05-10T11:33:57.622 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.621+0000 7f3a8622a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b222c08590 con 0x55b222ca87a0 2024-05-10T11:33:57.622 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.622+0000 7f3a80aad640 1 --2- 172.21.15.100:0/3689677066 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f3a680b28a0 0x7f3a680b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:57.622 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.622+0000 7f3a727fc640 1 -- 172.21.15.100:0/3689677066 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f3a740e2b20 con 0x7f3a7c103500 2024-05-10T11:33:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.622+0000 7f3a812ae640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b222ca7db0 0x55b222ca81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.622+0000 7f3a812ae640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b222ca7db0 0x55b222ca81d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39662/0 (socket says 172.21.15.100:39662) 2024-05-10T11:33:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.622+0000 7f3a812ae640 1 -- 172.21.15.100:0/4226145946 learned_addr learned my addr 172.21.15.100:0/4226145946 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:33:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.622+0000 7f3a81aaf640 1 --2- 172.21.15.100:0/4226145946 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b222cad910 0x55b222cafda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.622+0000 7f3a812ae640 1 -- 172.21.15.100:0/4226145946 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b222ca87a0 msgr2=0x55b222cad2a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T11:33:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.622+0000 7f3a812ae640 1 --2- 172.21.15.100:0/4226145946 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b222ca87a0 0x55b222cad2a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.622+0000 7f3a812ae640 1 -- 172.21.15.100:0/4226145946 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b222cad910 msgr2=0x55b222cafda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.622+0000 7f3a812ae640 1 --2- 172.21.15.100:0/4226145946 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b222cad910 0x55b222cafda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.622+0000 7f3a812ae640 1 -- 172.21.15.100:0/4226145946 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b22291e210 con 0x55b222ca7db0 2024-05-10T11:33:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.622+0000 7f3a812ae640 1 --2- 172.21.15.100:0/4226145946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b222ca7db0 0x55b222ca81d0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f3a78009900 tx=0x7f3a78005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b72656a58f82446 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:33:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.623+0000 7f3a80aad640 1 --2- 172.21.15.100:0/3689677066 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f3a680b28a0 0x7f3a680b4d60 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c10f5c0 tx=0x7f3a6c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:33:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.623+0000 7f3a677fe640 1 -- 172.21.15.100:0/4226145946 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a78035030 con 0x55b222ca7db0 2024-05-10T11:33:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.623+0000 7f3a677fe640 1 -- 172.21.15.100:0/4226145946 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3a78040070 con 0x55b222ca7db0 2024-05-10T11:33:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.623+0000 7f3a677fe640 1 -- 172.21.15.100:0/4226145946 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a78003850 con 0x55b222ca7db0 2024-05-10T11:33:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.624+0000 7f3a8622a740 1 -- 172.21.15.100:0/4226145946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b222ca7db0 msgr2=0x55b222ca81d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.624+0000 7f3a8622a740 1 --2- 172.21.15.100:0/4226145946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b222ca7db0 0x55b222ca81d0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f3a78009900 tx=0x7f3a78005dd0 comp rx=0 tx=0).stop 2024-05-10T11:33:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.625+0000 7f3a8622a740 1 -- 172.21.15.100:0/4226145946 shutdown_connections 2024-05-10T11:33:57.626 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.625+0000 7f3a8622a740 1 --2- 172.21.15.100:0/4226145946 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b222cad910 0x55b222cafda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:57.626 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.625+0000 7f3a8622a740 1 --2- 172.21.15.100:0/4226145946 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b222ca87a0 0x55b222cad2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:57.627 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.625+0000 7f3a8622a740 1 --2- 172.21.15.100:0/4226145946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b222ca7db0 0x55b222ca81d0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:57.627 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.625+0000 7f3a8622a740 1 -- 172.21.15.100:0/4226145946 >> 172.21.15.100:0/4226145946 conn(0x55b222ca3420 msgr2=0x55b222ca3820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:33:57.627 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.625+0000 7f3a8622a740 1 -- 172.21.15.100:0/4226145946 shutdown_connections 2024-05-10T11:33:57.627 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.625+0000 7f3a8622a740 1 -- 172.21.15.100:0/4226145946 wait complete. 2024-05-10T11:33:57.627 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.625+0000 7f3a8622a740 1 -- start start 2024-05-10T11:33:57.627 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.626+0000 7f3a8622a740 1 Processor -- start 2024-05-10T11:33:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.626+0000 7f3a8622a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b222ca7db0 0x55b222cb2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.626+0000 7f3a8622a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b222ca87a0 0x55b222cb2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.626+0000 7f3a8622a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b222cad910 0x55b222cb95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.626+0000 7f3a8622a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55b222cb0860 con 0x55b222ca7db0 2024-05-10T11:33:57.633 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.626+0000 7f3a8622a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b222bc9370 con 0x55b222ca87a0 2024-05-10T11:33:57.633 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.626+0000 7f3a8622a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b222cb09e0 con 0x55b222cad910 2024-05-10T11:33:57.633 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.626+0000 7f3a812ae640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b222ca7db0 0x55b222cb2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:57.633 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.626+0000 7f3a812ae640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b222ca7db0 0x55b222cb2030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39678/0 (socket says 172.21.15.100:39678) 2024-05-10T11:33:57.633 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.626+0000 7f3a812ae640 1 -- 172.21.15.100:0/1844484341 learned_addr learned my addr 172.21.15.100:0/1844484341 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:33:57.633 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.626+0000 7f3a8622a740 10 client.0 _ll_register_callbacks cb 0x55b22291da00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:33:57.633 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.626+0000 7f3a80aad640 1 --2- 172.21.15.100:0/1844484341 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b222ca87a0 0x55b222cb2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:57.633 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.626+0000 7f3a81aaf640 1 --2- 172.21.15.100:0/1844484341 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b222cad910 0x55b222cb95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:57.633 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.627+0000 7f3a812ae640 1 -- 172.21.15.100:0/1844484341 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b222cad910 msgr2=0x55b222cb95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:57.634 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.627+0000 7f3a812ae640 1 --2- 172.21.15.100:0/1844484341 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b222cad910 0x55b222cb95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:57.634 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.627+0000 7f3a812ae640 1 -- 172.21.15.100:0/1844484341 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b222ca87a0 msgr2=0x55b222cb2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:57.634 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.627+0000 7f3a812ae640 1 --2- 172.21.15.100:0/1844484341 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b222ca87a0 0x55b222cb2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:57.634 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.627+0000 7f3a812ae640 1 -- 172.21.15.100:0/1844484341 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2229ea620 con 0x55b222ca7db0 2024-05-10T11:33:57.634 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.627+0000 7f3a81aaf640 1 --2- 172.21.15.100:0/1844484341 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b222cad910 0x55b222cb95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:33:57.634 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.627+0000 7f3a812ae640 1 --2- 172.21.15.100:0/1844484341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b222ca7db0 0x55b222cb2030 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f3a78005ab0 tx=0x7f3a780318f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c157ced3523b18cd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:33:57.634 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.627+0000 7f3a64ff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:33:57.634 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.627+0000 7f3a64ff9640 1 -- 172.21.15.100:0/1844484341 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a78052060 con 0x55b222ca7db0 2024-05-10T11:33:57.634 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.627+0000 7f3a8622a740 1 -- 172.21.15.100:0/1844484341 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a78040070 con 0x55b222ca7db0 2024-05-10T11:33:57.634 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.627+0000 7f3a64ff9640 1 -- 172.21.15.100:0/1844484341 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f3a7803c440 con 0x55b222ca7db0 2024-05-10T11:33:57.634 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.627+0000 7f3a64ff9640 1 -- 172.21.15.100:0/1844484341 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f3a78035070 con 0x55b222ca7db0 2024-05-10T11:33:57.634 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.627+0000 7f3a8622a740 10 client.18501 fetch_fsmap learned FSMap version 37 2024-05-10T11:33:57.634 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.627+0000 7f3a8622a740 1 -- 172.21.15.100:0/1844484341 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55b222cbc570 con 0x55b222ca7db0 2024-05-10T11:33:57.634 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.628+0000 7f3a64ff9640 1 -- 172.21.15.100:0/1844484341 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3a78040070 con 0x55b222ca7db0 2024-05-10T11:33:57.635 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.628+0000 7f3a64ff9640 1 -- 172.21.15.100:0/1844484341 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f3a7804c030 con 0x55b222ca7db0 2024-05-10T11:33:57.635 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.628+0000 7f3a8622a740 10 client.18501 fetch_fsmap finished waiting for FSMap version 37 2024-05-10T11:33:57.635 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.628+0000 7f3a8622a740 10 client.18501 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:33:57.635 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.628+0000 7f3a8622a740 20 client.18501 populate_metadata read hostname 'smithi100' 2024-05-10T11:33:57.635 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.628+0000 7f3a8622a740 1 --2- 172.21.15.100:0/1844484341 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55b222cbd8c0 0x55b222cbfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:57.635 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.628+0000 7f3a8622a740 4 client.18501 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:33:57.635 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.628+0000 7f3a8622a740 1 -- 172.21.15.100:0/1844484341 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b222c1a1e0 con 0x55b222cbd8c0 2024-05-10T11:33:57.635 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.628+0000 7f3a80aad640 1 --2- 172.21.15.100:0/1844484341 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55b222cbd8c0 0x55b222cbfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:57.635 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.629+0000 7f3a64ff9640 10 client.18501 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:33:57.635 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.629+0000 7f3a80aad640 1 --2- 172.21.15.100:0/1844484341 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55b222cbd8c0 0x55b222cbfcb0 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff3db2eeda69de09 server_cookie=67132ea01c690c39 in_seq=0 out_seq=0 2024-05-10T11:33:57.635 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.630+0000 7f3a64ff9640 1 -- 172.21.15.100:0/1844484341 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b222c1a1e0 con 0x55b222cbd8c0 2024-05-10T11:33:57.635 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.631+0000 7f3a64ff9640 10 client.18501 handle_command_reply: tid=0 2024-05-10T11:33:57.635 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.631+0000 7f3a8622a740 1 client.18501 shutdown 2024-05-10T11:33:57.635 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.631+0000 7f3a8622a740 1 -- 172.21.15.100:0/1844484341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b222ca7db0 msgr2=0x55b222cb2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:57.635 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.631+0000 7f3a8622a740 1 --2- 172.21.15.100:0/1844484341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b222ca7db0 0x55b222cb2030 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f3a78005ab0 tx=0x7f3a780318f0 comp rx=0 tx=0).stop 2024-05-10T11:33:57.636 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.631+0000 7f3a8622a740 1 -- 172.21.15.100:0/1844484341 shutdown_connections 2024-05-10T11:33:57.636 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.631+0000 7f3a8622a740 1 --2- 172.21.15.100:0/1844484341 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55b222cbd8c0 0x55b222cbfcb0 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:57.636 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.631+0000 7f3a8622a740 1 --2- 172.21.15.100:0/1844484341 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b222cad910 0x55b222cb95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:57.636 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.631+0000 7f3a8622a740 1 --2- 172.21.15.100:0/1844484341 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b222ca87a0 0x55b222cb2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:57.636 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.631+0000 7f3a8622a740 1 --2- 172.21.15.100:0/1844484341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b222ca7db0 0x55b222cb2030 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:57.636 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.631+0000 7f3a8622a740 1 -- 172.21.15.100:0/1844484341 >> 172.21.15.100:0/1844484341 conn(0x55b222ca3420 msgr2=0x55b222ca2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:33:57.636 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.631+0000 7f3a64ff9640 0 client.18501 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:33:57.636 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.631+0000 7f3a8622a740 1 -- 172.21.15.100:0/1844484341 shutdown_connections 2024-05-10T11:33:57.636 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.631+0000 7f3a8622a740 1 -- 172.21.15.100:0/1844484341 wait complete. 2024-05-10T11:33:57.636 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.631+0000 7f3a8622a740 20 client.18501 trim_cache size 0 max 16384 2024-05-10T11:33:57.656 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.656+0000 7f3a8622a740 1 Processor -- start 2024-05-10T11:33:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.656+0000 7f3a8622a740 1 -- start start 2024-05-10T11:33:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.656+0000 7f3a8622a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b222ca7db0 0x55b222ca2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.656+0000 7f3a8622a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b222ca87a0 0x55b222ca2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.656+0000 7f3a8622a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b222cad910 0x55b222cb9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.656+0000 7f3a8622a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7c1bca50 con 0x55b222cad910 2024-05-10T11:33:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.656+0000 7f3a8622a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7c1ba290 con 0x55b222ca87a0 2024-05-10T11:33:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.656+0000 7f3a8622a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7c050740 con 0x55b222ca7db0 2024-05-10T11:33:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.656+0000 7f3a81aaf640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b222cad910 0x55b222cb9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.656+0000 7f3a81aaf640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b222cad910 0x55b222cb9540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39680/0 (socket says 172.21.15.100:39680) 2024-05-10T11:33:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.656+0000 7f3a81aaf640 1 -- 172.21.15.100:0/2644339653 learned_addr learned my addr 172.21.15.100:0/2644339653 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:33:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.656+0000 7f3a812ae640 1 --2- 172.21.15.100:0/2644339653 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b222ca7db0 0x55b222ca2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.656+0000 7f3a81aaf640 1 -- 172.21.15.100:0/2644339653 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b222ca7db0 msgr2=0x55b222ca2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.656+0000 7f3a81aaf640 1 --2- 172.21.15.100:0/2644339653 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b222ca7db0 0x55b222ca2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.656+0000 7f3a81aaf640 1 -- 172.21.15.100:0/2644339653 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b222ca87a0 msgr2=0x55b222ca2940 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T11:33:57.658 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.656+0000 7f3a81aaf640 1 --2- 172.21.15.100:0/2644339653 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b222ca87a0 0x55b222ca2940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:57.658 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.656+0000 7f3a81aaf640 1 -- 172.21.15.100:0/2644339653 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b222bccb80 con 0x55b222cad910 2024-05-10T11:33:57.658 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.657+0000 7f3a81aaf640 1 --2- 172.21.15.100:0/2644339653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b222cad910 0x55b222cb9540 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f3a740049d0 tx=0x7f3a7400cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14a8da566089b759 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:33:57.662 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.657+0000 7f3a66ffd640 1 -- 172.21.15.100:0/2644339653 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a740c9070 con 0x55b222cad910 2024-05-10T11:33:57.662 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.657+0000 7f3a66ffd640 1 -- 172.21.15.100:0/2644339653 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3a740aa4e0 con 0x55b222cad910 2024-05-10T11:33:57.662 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.657+0000 7f3a66ffd640 1 -- 172.21.15.100:0/2644339653 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a740aa040 con 0x55b222cad910 2024-05-10T11:33:57.662 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.658+0000 7f3a8622a740 1 -- 172.21.15.100:0/2644339653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b222cad910 msgr2=0x55b222cb9540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:57.662 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.658+0000 7f3a8622a740 1 --2- 172.21.15.100:0/2644339653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b222cad910 0x55b222cb9540 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f3a740049d0 tx=0x7f3a7400cee0 comp rx=0 tx=0).stop 2024-05-10T11:33:57.662 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.658+0000 7f3a8622a740 1 -- 172.21.15.100:0/2644339653 shutdown_connections 2024-05-10T11:33:57.662 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.658+0000 7f3a8622a740 1 --2- 172.21.15.100:0/2644339653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b222cad910 0x55b222cb9540 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:57.662 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.658+0000 7f3a8622a740 1 --2- 172.21.15.100:0/2644339653 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b222ca87a0 0x55b222ca2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:57.663 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.658+0000 7f3a8622a740 1 --2- 172.21.15.100:0/2644339653 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b222ca7db0 0x55b222ca2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:57.663 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.658+0000 7f3a8622a740 1 -- 172.21.15.100:0/2644339653 >> 172.21.15.100:0/2644339653 conn(0x55b222ca3420 msgr2=0x55b222ca1cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:33:57.663 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.658+0000 7f3a8622a740 1 -- 172.21.15.100:0/2644339653 shutdown_connections 2024-05-10T11:33:57.663 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.658+0000 7f3a8622a740 1 -- 172.21.15.100:0/2644339653 wait complete. 2024-05-10T11:33:57.663 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.658+0000 7f3a8622a740 1 -- start start 2024-05-10T11:33:57.663 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.659+0000 7f3a8622a740 1 Processor -- start 2024-05-10T11:33:57.663 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.659+0000 7f3a8622a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b222ca7db0 0x55b222cb7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:57.663 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.659+0000 7f3a8622a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b222ca87a0 0x55b222cb7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:57.663 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.659+0000 7f3a8622a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b222cad910 0x55b222cb8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:57.663 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.659+0000 7f3a8622a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7c1b8750 con 0x55b222cad910 2024-05-10T11:33:57.663 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.659+0000 7f3a8622a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7c1b9bc0 con 0x55b222ca87a0 2024-05-10T11:33:57.663 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.659+0000 7f3a8622a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7c1b8e20 con 0x55b222ca7db0 2024-05-10T11:33:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.659+0000 7f3a812ae640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b222ca7db0 0x55b222cb7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.659+0000 7f3a8622a740 10 client.0 _ll_register_callbacks cb 0x7f3a74004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:33:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.659+0000 7f3a812ae640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b222ca7db0 0x55b222cb7030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:33248/0 (socket says 172.21.15.100:33248) 2024-05-10T11:33:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.659+0000 7f3a812ae640 1 -- 172.21.15.100:0/400524640 learned_addr learned my addr 172.21.15.100:0/400524640 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:33:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.659+0000 7f3a81aaf640 1 --2- 172.21.15.100:0/400524640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b222cad910 0x55b222cb8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.659+0000 7f3a80aad640 1 --2- 172.21.15.100:0/400524640 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b222ca87a0 0x55b222cb7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.660+0000 7f3a812ae640 1 -- 172.21.15.100:0/400524640 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b222ca87a0 msgr2=0x55b222cb7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.660+0000 7f3a812ae640 1 --2- 172.21.15.100:0/400524640 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b222ca87a0 0x55b222cb7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.660+0000 7f3a812ae640 1 -- 172.21.15.100:0/400524640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b222cad910 msgr2=0x55b222cb8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.660+0000 7f3a812ae640 1 --2- 172.21.15.100:0/400524640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b222cad910 0x55b222cb8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.660+0000 7f3a812ae640 1 -- 172.21.15.100:0/400524640 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b222cb8b70 con 0x55b222ca7db0 2024-05-10T11:33:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.660+0000 7f3a81aaf640 1 --2- 172.21.15.100:0/400524640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b222cad910 0x55b222cb8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:33:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.660+0000 7f3a812ae640 1 --2- 172.21.15.100:0/400524640 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b222ca7db0 0x55b222cb7030 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3a78031a40 tx=0x7f3a78044a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23531d973e2b7a2e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:33:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.660+0000 7f3a64ff9640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:33:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.660+0000 7f3a64ff9640 1 -- 172.21.15.100:0/400524640 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a78062060 con 0x55b222ca7db0 2024-05-10T11:33:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.660+0000 7f3a64ff9640 1 -- 172.21.15.100:0/400524640 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f3a78004450 con 0x55b222ca7db0 2024-05-10T11:33:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.660+0000 7f3a8622a740 1 -- 172.21.15.100:0/400524640 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a740aa4e0 con 0x55b222ca7db0 2024-05-10T11:33:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.660+0000 7f3a64ff9640 1 -- 172.21.15.100:0/400524640 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f3a78052040 con 0x55b222ca7db0 2024-05-10T11:33:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.660+0000 7f3a8622a740 10 client.27934 fetch_fsmap learned FSMap version 37 2024-05-10T11:33:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.660+0000 7f3a8622a740 1 -- 172.21.15.100:0/400524640 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55b222d28400 con 0x55b222ca7db0 2024-05-10T11:33:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.661+0000 7f3a64ff9640 1 -- 172.21.15.100:0/400524640 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3a740aa4e0 con 0x55b222ca7db0 2024-05-10T11:33:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.661+0000 7f3a64ff9640 1 -- 172.21.15.100:0/400524640 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f3a7804c040 con 0x55b222ca7db0 2024-05-10T11:33:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.661+0000 7f3a8622a740 10 client.27934 fetch_fsmap finished waiting for FSMap version 37 2024-05-10T11:33:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.661+0000 7f3a8622a740 10 client.27934 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:33:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.661+0000 7f3a8622a740 20 client.27934 populate_metadata read hostname 'smithi100' 2024-05-10T11:33:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.661+0000 7f3a8622a740 1 --2- 172.21.15.100:0/400524640 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55b222d28e90 0x55b222d2b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.661+0000 7f3a8622a740 4 client.27934 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:33:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.661+0000 7f3a8622a740 1 -- 172.21.15.100:0/400524640 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b222c14050 con 0x55b222d28e90 2024-05-10T11:33:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.661+0000 7f3a80aad640 1 --2- 172.21.15.100:0/400524640 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55b222d28e90 0x55b222d2b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.661+0000 7f3a64ff9640 10 client.27934 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:33:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.661+0000 7f3a80aad640 1 --2- 172.21.15.100:0/400524640 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55b222d28e90 0x55b222d2b2b0 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da362e4b4d5d74a5 server_cookie=56a3815336727c1b in_seq=0 out_seq=0 2024-05-10T11:33:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.760+0000 7f3a64ff9640 1 -- 172.21.15.100:0/400524640 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b222c14050 con 0x55b222d28e90 2024-05-10T11:33:57.761 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.760+0000 7f3a64ff9640 10 client.27934 handle_command_reply: tid=0 2024-05-10T11:33:57.761 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.760+0000 7f3a8622a740 1 client.27934 shutdown 2024-05-10T11:33:57.761 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.760+0000 7f3a8622a740 1 -- 172.21.15.100:0/400524640 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b222ca7db0 msgr2=0x55b222cb7030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:57.761 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.760+0000 7f3a8622a740 1 --2- 172.21.15.100:0/400524640 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b222ca7db0 0x55b222cb7030 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3a78031a40 tx=0x7f3a78044a40 comp rx=0 tx=0).stop 2024-05-10T11:33:57.762 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.761+0000 7f3a8622a740 1 -- 172.21.15.100:0/400524640 shutdown_connections 2024-05-10T11:33:57.762 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.761+0000 7f3a8622a740 1 --2- 172.21.15.100:0/400524640 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55b222d28e90 0x55b222d2b2b0 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:57.762 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.761+0000 7f3a8622a740 1 --2- 172.21.15.100:0/400524640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b222cad910 0x55b222cb8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:57.762 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.761+0000 7f3a8622a740 1 --2- 172.21.15.100:0/400524640 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b222ca87a0 0x55b222cb7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:57.762 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.761+0000 7f3a8622a740 1 --2- 172.21.15.100:0/400524640 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b222ca7db0 0x55b222cb7030 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:57.762 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.761+0000 7f3a8622a740 1 -- 172.21.15.100:0/400524640 >> 172.21.15.100:0/400524640 conn(0x55b222ca3420 msgr2=0x55b222ca2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:33:57.762 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.761+0000 7f3a64ff9640 0 client.27934 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:33:57.762 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.761+0000 7f3a8622a740 1 -- 172.21.15.100:0/400524640 shutdown_connections 2024-05-10T11:33:57.762 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.762+0000 7f3a8622a740 1 -- 172.21.15.100:0/400524640 wait complete. 2024-05-10T11:33:57.762 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.762+0000 7f3a8622a740 20 client.27934 trim_cache size 0 max 16384 2024-05-10T11:33:57.763 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:33:57.763 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:33:57.763 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:33:57.763 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:33:57.764 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.763+0000 7f3a83512640 1 -- 172.21.15.100:0/3689677066 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f3a680b28a0 msgr2=0x7f3a680b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:57.764 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.763+0000 7f3a83512640 1 --2- 172.21.15.100:0/3689677066 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f3a680b28a0 0x7f3a680b4d60 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c10f5c0 tx=0x7f3a6c005f90 comp rx=0 tx=0).stop 2024-05-10T11:33:57.764 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.763+0000 7f3a83512640 1 -- 172.21.15.100:0/3689677066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a7c103500 msgr2=0x7f3a7c10e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:57.764 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.763+0000 7f3a83512640 1 --2- 172.21.15.100:0/3689677066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a7c103500 0x7f3a7c10e3d0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f3a7400eb10 tx=0x7f3a7400c2d0 comp rx=0 tx=0).stop 2024-05-10T11:33:57.764 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.763+0000 7f3a83512640 1 -- 172.21.15.100:0/3689677066 shutdown_connections 2024-05-10T11:33:57.764 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.763+0000 7f3a83512640 1 --2- 172.21.15.100:0/3689677066 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f3a680b28a0 0x7f3a680b4d60 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:57.764 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.763+0000 7f3a83512640 1 --2- 172.21.15.100:0/3689677066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a7c103500 0x7f3a7c10e3d0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:57.764 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.763+0000 7f3a83512640 1 --2- 172.21.15.100:0/3689677066 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a7c102b40 0x7f3a7c10de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:57.764 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.763+0000 7f3a83512640 1 --2- 172.21.15.100:0/3689677066 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a7c101950 0x7f3a7c10f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:57.764 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.763+0000 7f3a83512640 1 -- 172.21.15.100:0/3689677066 >> 172.21.15.100:0/3689677066 conn(0x7f3a7c0fcec0 msgr2=0x7f3a7c0fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:33:57.764 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.763+0000 7f3a83512640 1 -- 172.21.15.100:0/3689677066 shutdown_connections 2024-05-10T11:33:57.764 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:33:57.763+0000 7f3a83512640 1 -- 172.21.15.100:0/3689677066 wait complete. 2024-05-10T11:33:57.914 INFO:tasks.workunit.client.0.smithi100.stdout: 10 18896 10.78 MB/sec warmup 111 sec latency 112.662 ms 2024-05-10T11:33:58.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:33:58 smithi178 ceph-mon[33359]: pgmap v888: 129 pgs: 129 active+clean; 393 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 3.8 MiB/s wr, 240 op/s 2024-05-10T11:33:58.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:33:58 smithi188 ceph-mon[34075]: pgmap v888: 129 pgs: 129 active+clean; 393 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 3.8 MiB/s wr, 240 op/s 2024-05-10T11:33:58.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:33:58 smithi100 ceph-mon[30296]: pgmap v888: 129 pgs: 129 active+clean; 393 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 3.8 MiB/s wr, 240 op/s 2024-05-10T11:33:58.914 INFO:tasks.workunit.client.0.smithi100.stdout: 10 19060 10.77 MB/sec warmup 112 sec latency 68.029 ms 2024-05-10T11:33:59.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.210+0000 7f53fdffb700 1 -- 172.21.15.100:0/1571255934 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f8009480 con 0x7f5400102d50 2024-05-10T11:33:59.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.211+0000 7f54058f1700 1 -- 172.21.15.100:0/1571255934 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5400102d50 msgr2=0x7f540010f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:59.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.211+0000 7f54058f1700 1 --2- 172.21.15.100:0/1571255934 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5400102d50 0x7f540010f240 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f53f800b0a0 tx=0x7f53f800b3b0 comp rx=0 tx=0).stop 2024-05-10T11:33:59.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.211+0000 7f54058f1700 1 -- 172.21.15.100:0/1571255934 shutdown_connections 2024-05-10T11:33:59.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.211+0000 7f54058f1700 1 --2- 172.21.15.100:0/1571255934 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5400102d50 0x7f540010f240 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:59.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.211+0000 7f54058f1700 1 --2- 172.21.15.100:0/1571255934 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5400108400 0x7f54001087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:59.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.211+0000 7f54058f1700 1 --2- 172.21.15.100:0/1571255934 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5400102390 0x7f5400102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:59.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.211+0000 7f54058f1700 1 -- 172.21.15.100:0/1571255934 >> 172.21.15.100:0/1571255934 conn(0x7f5400079310 msgr2=0x7f5400079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:33:59.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.211+0000 7f54058f1700 1 -- 172.21.15.100:0/1571255934 shutdown_connections 2024-05-10T11:33:59.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.211+0000 7f54058f1700 1 -- 172.21.15.100:0/1571255934 wait complete. 2024-05-10T11:33:59.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.212+0000 7f54058f1700 1 Processor -- start 2024-05-10T11:33:59.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.212+0000 7f54058f1700 1 -- start start 2024-05-10T11:33:59.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.212+0000 7f54058f1700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5400102390 0x7f54001042a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:59.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.212+0000 7f54058f1700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5400102d50 0x7f54001047e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:59.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.213+0000 7f54058f1700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5400108400 0x7f5400104d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:59.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.213+0000 7f54058f1700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5400112490 con 0x7f5400102390 2024-05-10T11:33:59.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.213+0000 7f54058f1700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5400112310 con 0x7f5400102d50 2024-05-10T11:33:59.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.213+0000 7f54058f1700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5400112610 con 0x7f5400108400 2024-05-10T11:33:59.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.213+0000 7f53fe7fc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5400102d50 0x7f54001047e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:59.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.213+0000 7f53feffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5400102390 0x7f54001042a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:59.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.213+0000 7f53fe7fc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5400102d50 0x7f54001047e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:46560/0 (socket says 172.21.15.100:46560) 2024-05-10T11:33:59.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.213+0000 7f53fe7fc700 1 -- 172.21.15.100:0/3709339780 learned_addr learned my addr 172.21.15.100:0/3709339780 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:33:59.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.213+0000 7f53ff7fe700 1 --2- 172.21.15.100:0/3709339780 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5400108400 0x7f5400104d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:59.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.213+0000 7f53fe7fc700 1 -- 172.21.15.100:0/3709339780 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5400108400 msgr2=0x7f5400104d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:59.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.213+0000 7f53fe7fc700 1 --2- 172.21.15.100:0/3709339780 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5400108400 0x7f5400104d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:59.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.213+0000 7f53fe7fc700 1 -- 172.21.15.100:0/3709339780 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5400102390 msgr2=0x7f54001042a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:59.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.213+0000 7f53fe7fc700 1 --2- 172.21.15.100:0/3709339780 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5400102390 0x7f54001042a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:59.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.213+0000 7f53fe7fc700 1 -- 172.21.15.100:0/3709339780 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54001a7f90 con 0x7f5400102d50 2024-05-10T11:33:59.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.213+0000 7f53fe7fc700 1 --2- 172.21.15.100:0/3709339780 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5400102d50 0x7f54001047e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f53f400e9a0 tx=0x7f53f400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:33:59.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.214+0000 7f54048ef700 1 -- 172.21.15.100:0/3709339780 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f400ca10 con 0x7f5400102d50 2024-05-10T11:33:59.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.214+0000 7f54048ef700 1 -- 172.21.15.100:0/3709339780 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f53f400cbb0 con 0x7f5400102d50 2024-05-10T11:33:59.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.214+0000 7f54048ef700 1 -- 172.21.15.100:0/3709339780 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f4018680 con 0x7f5400102d50 2024-05-10T11:33:59.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.214+0000 7f54058f1700 1 -- 172.21.15.100:0/3709339780 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54001a8220 con 0x7f5400102d50 2024-05-10T11:33:59.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.214+0000 7f54058f1700 1 -- 172.21.15.100:0/3709339780 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54001a87b0 con 0x7f5400102d50 2024-05-10T11:33:59.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.217+0000 7f54048ef700 1 -- 172.21.15.100:0/3709339780 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f53f4010430 con 0x7f5400102d50 2024-05-10T11:33:59.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.218+0000 7f54048ef700 1 --2- 172.21.15.100:0/3709339780 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f53e80b2900 0x7f53e80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:59.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.218+0000 7f53feffd700 1 --2- 172.21.15.100:0/3709339780 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f53e80b2900 0x7f53e80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:59.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.218+0000 7f54048ef700 1 -- 172.21.15.100:0/3709339780 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f53f40e2660 con 0x7f5400102d50 2024-05-10T11:33:59.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.218+0000 7f53feffd700 1 --2- 172.21.15.100:0/3709339780 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f53e80b2900 0x7f53e80b4dc0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f53f0009be0 tx=0x7f53f0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:33:59.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.218+0000 7f5410184b80 1 Processor -- start 2024-05-10T11:33:59.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.219+0000 7f5410184b80 1 -- start start 2024-05-10T11:33:59.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.219+0000 7f5410184b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ce3ec99c0 0x555ce3ec9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:59.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.219+0000 7f5410184b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ce3eca2e0 0x555ce3ec4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:59.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.219+0000 7f5410184b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ce3ec4560 0x555ce3ec49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:59.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.219+0000 7f5410184b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x555ce3e0d940 con 0x555ce3eca2e0 2024-05-10T11:33:59.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.219+0000 7f5410184b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x555ce3d72da0 con 0x555ce3ec4560 2024-05-10T11:33:59.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.219+0000 7f5410184b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x555ce3e09900 con 0x555ce3ec99c0 2024-05-10T11:33:59.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.219+0000 7f53feffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ce3ec99c0 0x555ce3ec9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:59.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.219+0000 7f53feffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ce3ec99c0 0x555ce3ec9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:33276/0 (socket says 172.21.15.100:33276) 2024-05-10T11:33:59.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.219+0000 7f53feffd700 1 -- 172.21.15.100:0/279984239 learned_addr learned my addr 172.21.15.100:0/279984239 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:33:59.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.219+0000 7f53fe7fc700 1 --2- 172.21.15.100:0/279984239 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ce3eca2e0 0x555ce3ec4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:59.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.219+0000 7f53ff7fe700 1 --2- 172.21.15.100:0/279984239 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ce3ec4560 0x555ce3ec49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:59.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.219+0000 7f53fe7fc700 1 -- 172.21.15.100:0/279984239 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ce3ec99c0 msgr2=0x555ce3ec9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:59.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.219+0000 7f53fe7fc700 1 --2- 172.21.15.100:0/279984239 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ce3ec99c0 0x555ce3ec9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:59.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.220+0000 7f53fe7fc700 1 -- 172.21.15.100:0/279984239 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ce3ec4560 msgr2=0x555ce3ec49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:59.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.220+0000 7f53fe7fc700 1 --2- 172.21.15.100:0/279984239 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ce3ec4560 0x555ce3ec49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:59.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.220+0000 7f53fe7fc700 1 -- 172.21.15.100:0/279984239 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ce3d5a8d0 con 0x555ce3eca2e0 2024-05-10T11:33:59.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.220+0000 7f53ff7fe700 1 --2- 172.21.15.100:0/279984239 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ce3ec4560 0x555ce3ec49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:33:59.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.220+0000 7f53fe7fc700 1 --2- 172.21.15.100:0/279984239 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ce3eca2e0 0x555ce3ec4020 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f53f40dfe00 tx=0x7f53f40b2140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb133e520c775a7f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:33:59.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.220+0000 7f53e5ffb700 1 -- 172.21.15.100:0/279984239 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f40aa020 con 0x555ce3eca2e0 2024-05-10T11:33:59.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.220+0000 7f53e5ffb700 1 -- 172.21.15.100:0/279984239 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f53f40a9920 con 0x555ce3eca2e0 2024-05-10T11:33:59.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.220+0000 7f53e5ffb700 1 -- 172.21.15.100:0/279984239 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f400cdf0 con 0x555ce3eca2e0 2024-05-10T11:33:59.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.221+0000 7f5410184b80 1 -- 172.21.15.100:0/279984239 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ce3eca2e0 msgr2=0x555ce3ec4020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:59.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.221+0000 7f5410184b80 1 --2- 172.21.15.100:0/279984239 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ce3eca2e0 0x555ce3ec4020 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f53f40dfe00 tx=0x7f53f40b2140 comp rx=0 tx=0).stop 2024-05-10T11:33:59.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.221+0000 7f5410184b80 1 -- 172.21.15.100:0/279984239 shutdown_connections 2024-05-10T11:33:59.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.221+0000 7f5410184b80 1 --2- 172.21.15.100:0/279984239 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ce3ec4560 0x555ce3ec49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:59.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.221+0000 7f5410184b80 1 --2- 172.21.15.100:0/279984239 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ce3ec99c0 0x555ce3ec9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:59.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.221+0000 7f5410184b80 1 --2- 172.21.15.100:0/279984239 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ce3eca2e0 0x555ce3ec4020 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:59.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.221+0000 7f5410184b80 1 -- 172.21.15.100:0/279984239 >> 172.21.15.100:0/279984239 conn(0x555ce3de7b70 msgr2=0x555ce3eb62a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:33:59.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.222+0000 7f5410184b80 1 -- 172.21.15.100:0/279984239 shutdown_connections 2024-05-10T11:33:59.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.222+0000 7f5410184b80 1 -- 172.21.15.100:0/279984239 wait complete. 2024-05-10T11:33:59.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.222+0000 7f5410184b80 1 -- start start 2024-05-10T11:33:59.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.222+0000 7f5410184b80 1 Processor -- start 2024-05-10T11:33:59.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.223+0000 7f5410184b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ce3ec4560 0x555ce3ecffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:59.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.223+0000 7f5410184b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ce3ec99c0 0x555ce3ecd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:59.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.223+0000 7f5410184b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ce3eca2e0 0x555ce3ecd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:59.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.223+0000 7f5410184b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x555ce3ed0ba0 con 0x555ce3ec4560 2024-05-10T11:33:59.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.223+0000 7f5410184b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x555ce3dc6ed0 con 0x555ce3ec99c0 2024-05-10T11:33:59.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.223+0000 7f5410184b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x555ce3ed0d20 con 0x555ce3eca2e0 2024-05-10T11:33:59.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.223+0000 7f5410184b80 10 client.0 _ll_register_callbacks cb 0x555ce3da15b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:33:59.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.223+0000 7f53feffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ce3ec4560 0x555ce3ecffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:59.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.223+0000 7f53feffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ce3ec4560 0x555ce3ecffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39718/0 (socket says 172.21.15.100:39718) 2024-05-10T11:33:59.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.223+0000 7f53feffd700 1 -- 172.21.15.100:0/636207074 learned_addr learned my addr 172.21.15.100:0/636207074 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:33:59.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.223+0000 7f53fe7fc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ce3ec99c0 0x555ce3ecd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:59.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.223+0000 7f53ff7fe700 1 --2- 172.21.15.100:0/636207074 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ce3eca2e0 0x555ce3ecd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:59.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.223+0000 7f53fe7fc700 1 -- 172.21.15.100:0/636207074 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ce3eca2e0 msgr2=0x555ce3ecd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:59.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.223+0000 7f53fe7fc700 1 --2- 172.21.15.100:0/636207074 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ce3eca2e0 0x555ce3ecd890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:59.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.224+0000 7f53fe7fc700 1 -- 172.21.15.100:0/636207074 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ce3ec4560 msgr2=0x555ce3ecffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:59.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.224+0000 7f53fe7fc700 1 --2- 172.21.15.100:0/636207074 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ce3ec4560 0x555ce3ecffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:59.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.224+0000 7f53fe7fc700 1 -- 172.21.15.100:0/636207074 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ce3dc4760 con 0x555ce3ec99c0 2024-05-10T11:33:59.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.224+0000 7f53feffd700 1 --2- 172.21.15.100:0/636207074 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ce3ec4560 0x555ce3ecffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:33:59.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.224+0000 7f53fe7fc700 1 --2- 172.21.15.100:0/636207074 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ce3ec99c0 0x555ce3ecd350 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f53f400e970 tx=0x7f53f40adda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=709fdf98dda3a8ff server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:33:59.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.224+0000 7f53deffd700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:33:59.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.224+0000 7f53deffd700 1 -- 172.21.15.100:0/636207074 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f40aa020 con 0x555ce3ec99c0 2024-05-10T11:33:59.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.224+0000 7f53deffd700 1 -- 172.21.15.100:0/636207074 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f53f40a5050 con 0x555ce3ec99c0 2024-05-10T11:33:59.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.224+0000 7f5410184b80 1 -- 172.21.15.100:0/636207074 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53f40a9920 con 0x555ce3ec99c0 2024-05-10T11:33:59.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.225+0000 7f53deffd700 1 -- 172.21.15.100:0/636207074 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f53f40cabd0 con 0x555ce3ec99c0 2024-05-10T11:33:59.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.225+0000 7f5410184b80 10 client.27851 fetch_fsmap learned FSMap version 37 2024-05-10T11:33:59.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.225+0000 7f5410184b80 1 -- 172.21.15.100:0/636207074 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x555ce3edb4b0 con 0x555ce3ec99c0 2024-05-10T11:33:59.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.225+0000 7f53deffd700 1 -- 172.21.15.100:0/636207074 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f53f40a9920 con 0x555ce3ec99c0 2024-05-10T11:33:59.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.225+0000 7f53deffd700 1 -- 172.21.15.100:0/636207074 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f53f40a8c10 con 0x555ce3ec99c0 2024-05-10T11:33:59.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.225+0000 7f5410184b80 10 client.27851 fetch_fsmap finished waiting for FSMap version 37 2024-05-10T11:33:59.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.225+0000 7f5410184b80 10 client.27851 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-10T11:33:59.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.225+0000 7f5410184b80 20 client.27851 populate_metadata read hostname 'smithi100' 2024-05-10T11:33:59.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.225+0000 7f5410184b80 1 --2- 172.21.15.100:0/636207074 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x555ce3edd980 0x555ce3edfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:59.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.225+0000 7f5410184b80 4 client.27851 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:33:59.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.225+0000 7f5410184b80 1 -- 172.21.15.100:0/636207074 --> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ce3e1b320 con 0x555ce3edd980 2024-05-10T11:33:59.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.225+0000 7f53feffd700 1 --2- 172.21.15.100:0/636207074 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x555ce3edd980 0x555ce3edfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:59.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.226+0000 7f53deffd700 10 client.27851 ms_handle_connect on v2:172.21.15.178:6838/3760985404 2024-05-10T11:33:59.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.226+0000 7f53feffd700 1 --2- 172.21.15.100:0/636207074 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x555ce3edd980 0x555ce3edfd70 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3741c9dea3523371 server_cookie=ec53727283b37125 in_seq=0 out_seq=0 2024-05-10T11:33:59.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.227+0000 7f53deffd700 1 -- 172.21.15.100:0/636207074 <== mds.1 v2:172.21.15.178:6838/3760985404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555ce3e1b320 con 0x555ce3edd980 2024-05-10T11:33:59.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.227+0000 7f53deffd700 10 client.27851 handle_command_reply: tid=0 2024-05-10T11:33:59.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.227+0000 7f5410184b80 1 client.27851 shutdown 2024-05-10T11:33:59.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.227+0000 7f5410184b80 1 -- 172.21.15.100:0/636207074 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ce3ec99c0 msgr2=0x555ce3ecd350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:59.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.227+0000 7f5410184b80 1 --2- 172.21.15.100:0/636207074 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ce3ec99c0 0x555ce3ecd350 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f53f400e970 tx=0x7f53f40adda0 comp rx=0 tx=0).stop 2024-05-10T11:33:59.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.227+0000 7f5410184b80 1 -- 172.21.15.100:0/636207074 shutdown_connections 2024-05-10T11:33:59.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.227+0000 7f5410184b80 1 --2- 172.21.15.100:0/636207074 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x555ce3edd980 0x555ce3edfd70 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:59.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.227+0000 7f5410184b80 1 --2- 172.21.15.100:0/636207074 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ce3eca2e0 0x555ce3ecd890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:59.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.227+0000 7f5410184b80 1 --2- 172.21.15.100:0/636207074 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ce3ec4560 0x555ce3ecffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:59.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.227+0000 7f5410184b80 1 --2- 172.21.15.100:0/636207074 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ce3ec99c0 0x555ce3ecd350 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:59.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.227+0000 7f53deffd700 0 client.27851 ms_handle_reset on v2:172.21.15.178:6838/3760985404 2024-05-10T11:33:59.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.227+0000 7f5410184b80 1 -- 172.21.15.100:0/636207074 >> 172.21.15.100:0/636207074 conn(0x555ce3de7b70 msgr2=0x555ce3eb0b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:33:59.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.227+0000 7f5410184b80 1 -- 172.21.15.100:0/636207074 shutdown_connections 2024-05-10T11:33:59.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.227+0000 7f5410184b80 1 -- 172.21.15.100:0/636207074 wait complete. 2024-05-10T11:33:59.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.228+0000 7f5410184b80 20 client.27851 trim_cache size 0 max 16384 2024-05-10T11:33:59.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.247+0000 7f5410184b80 1 Processor -- start 2024-05-10T11:33:59.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.247+0000 7f5410184b80 1 -- start start 2024-05-10T11:33:59.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.247+0000 7f5410184b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ce3ec4560 0x555ce3ec7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:59.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.248+0000 7f5410184b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ce3ec99c0 0x555ce3eb0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:59.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.248+0000 7f5410184b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ce3eca2e0 0x555ce3ee29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:59.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.248+0000 7f5410184b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f54001af800 con 0x555ce3ec99c0 2024-05-10T11:33:59.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.248+0000 7f5410184b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f54001ad000 con 0x555ce3eca2e0 2024-05-10T11:33:59.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.248+0000 7f5410184b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f54001aae80 con 0x555ce3ec4560 2024-05-10T11:33:59.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.248+0000 7f53fe7fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ce3ec99c0 0x555ce3eb0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:59.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.248+0000 7f53fe7fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ce3ec99c0 0x555ce3eb0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39728/0 (socket says 172.21.15.100:39728) 2024-05-10T11:33:59.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.248+0000 7f53fe7fc700 1 -- 172.21.15.100:0/37991494 learned_addr learned my addr 172.21.15.100:0/37991494 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:33:59.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.248+0000 7f53feffd700 1 --2- 172.21.15.100:0/37991494 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ce3ec4560 0x555ce3ec7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:59.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.248+0000 7f53ff7fe700 1 --2- 172.21.15.100:0/37991494 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ce3eca2e0 0x555ce3ee29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:59.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.248+0000 7f53fe7fc700 1 -- 172.21.15.100:0/37991494 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ce3ec4560 msgr2=0x555ce3ec7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:59.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.248+0000 7f53fe7fc700 1 --2- 172.21.15.100:0/37991494 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ce3ec4560 0x555ce3ec7960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:59.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.248+0000 7f53fe7fc700 1 -- 172.21.15.100:0/37991494 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ce3eca2e0 msgr2=0x555ce3ee29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:59.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.248+0000 7f53fe7fc700 1 --2- 172.21.15.100:0/37991494 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ce3eca2e0 0x555ce3ee29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:59.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.248+0000 7f53fe7fc700 1 -- 172.21.15.100:0/37991494 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ce3ec7ea0 con 0x555ce3ec99c0 2024-05-10T11:33:59.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.248+0000 7f53ff7fe700 1 --2- 172.21.15.100:0/37991494 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ce3eca2e0 0x555ce3ee29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:33:59.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.248+0000 7f53fe7fc700 1 --2- 172.21.15.100:0/37991494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ce3ec99c0 0x555ce3eb0c40 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f53f40b2940 tx=0x7f53f40b2140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=799f1d61940927ae server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:33:59.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.249+0000 7f53e57fa700 1 -- 172.21.15.100:0/37991494 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f40a50b0 con 0x555ce3ec99c0 2024-05-10T11:33:59.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.249+0000 7f53e57fa700 1 -- 172.21.15.100:0/37991494 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f53f4010e50 con 0x555ce3ec99c0 2024-05-10T11:33:59.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.249+0000 7f53e57fa700 1 -- 172.21.15.100:0/37991494 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f40a8de0 con 0x555ce3ec99c0 2024-05-10T11:33:59.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.250+0000 7f5410184b80 1 -- 172.21.15.100:0/37991494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ce3ec99c0 msgr2=0x555ce3eb0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:59.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.250+0000 7f5410184b80 1 --2- 172.21.15.100:0/37991494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ce3ec99c0 0x555ce3eb0c40 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f53f40b2940 tx=0x7f53f40b2140 comp rx=0 tx=0).stop 2024-05-10T11:33:59.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.250+0000 7f5410184b80 1 -- 172.21.15.100:0/37991494 shutdown_connections 2024-05-10T11:33:59.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.250+0000 7f5410184b80 1 --2- 172.21.15.100:0/37991494 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ce3eca2e0 0x555ce3ee29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:59.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.250+0000 7f5410184b80 1 --2- 172.21.15.100:0/37991494 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ce3ec4560 0x555ce3ec7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:59.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.250+0000 7f5410184b80 1 --2- 172.21.15.100:0/37991494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ce3ec99c0 0x555ce3eb0c40 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:59.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.250+0000 7f5410184b80 1 -- 172.21.15.100:0/37991494 >> 172.21.15.100:0/37991494 conn(0x555ce3de7b70 msgr2=0x555ce3ec7250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:33:59.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.250+0000 7f5410184b80 1 -- 172.21.15.100:0/37991494 shutdown_connections 2024-05-10T11:33:59.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.250+0000 7f5410184b80 1 -- 172.21.15.100:0/37991494 wait complete. 2024-05-10T11:33:59.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.250+0000 7f5410184b80 1 -- start start 2024-05-10T11:33:59.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.251+0000 7f5410184b80 1 Processor -- start 2024-05-10T11:33:59.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.251+0000 7f5410184b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ce3ec4560 0x555ce3edf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:59.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.251+0000 7f5410184b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ce3ec99c0 0x555ce3edfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:59.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.251+0000 7f5410184b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ce3eca2e0 0x555ce3ee0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:59.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.251+0000 7f5410184b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f54001ab4c0 con 0x555ce3eca2e0 2024-05-10T11:33:59.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.251+0000 7f5410184b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f54001ac930 con 0x555ce3ec99c0 2024-05-10T11:33:59.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.251+0000 7f5410184b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f54001abb90 con 0x555ce3ec4560 2024-05-10T11:33:59.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.252+0000 7f5410184b80 10 client.0 _ll_register_callbacks cb 0x555ce3da15b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:33:59.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.251+0000 7f53ff7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ce3eca2e0 0x555ce3ee0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:59.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.251+0000 7f53feffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ce3ec4560 0x555ce3edf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:59.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.252+0000 7f53feffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ce3ec4560 0x555ce3edf990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:33308/0 (socket says 172.21.15.100:33308) 2024-05-10T11:33:59.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.252+0000 7f53feffd700 1 -- 172.21.15.100:0/567134923 learned_addr learned my addr 172.21.15.100:0/567134923 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:33:59.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.252+0000 7f53ff7fe700 1 -- 172.21.15.100:0/567134923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ce3ec4560 msgr2=0x555ce3edf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:59.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.252+0000 7f53ff7fe700 1 --2- 172.21.15.100:0/567134923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ce3ec4560 0x555ce3edf990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:59.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.252+0000 7f53ff7fe700 1 -- 172.21.15.100:0/567134923 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ce3ec99c0 msgr2=0x555ce3edfed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:33:59.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.252+0000 7f53ff7fe700 1 --2- 172.21.15.100:0/567134923 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ce3ec99c0 0x555ce3edfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:59.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.252+0000 7f53ff7fe700 1 -- 172.21.15.100:0/567134923 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ce3f45f10 con 0x555ce3eca2e0 2024-05-10T11:33:59.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.252+0000 7f53feffd700 1 --2- 172.21.15.100:0/567134923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ce3ec4560 0x555ce3edf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:33:59.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.252+0000 7f53ff7fe700 1 --2- 172.21.15.100:0/567134923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ce3eca2e0 0x555ce3ee0410 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f53f800b0a0 tx=0x7f53f800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1725045ea1eff835 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:33:59.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.252+0000 7f53deffd700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:33:59.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.252+0000 7f53deffd700 1 -- 172.21.15.100:0/567134923 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f8004ca0 con 0x555ce3eca2e0 2024-05-10T11:33:59.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.252+0000 7f53deffd700 1 -- 172.21.15.100:0/567134923 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f53f80074e0 con 0x555ce3eca2e0 2024-05-10T11:33:59.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.252+0000 7f5410184b80 1 -- 172.21.15.100:0/567134923 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53f4010e50 con 0x555ce3eca2e0 2024-05-10T11:33:59.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.253+0000 7f53deffd700 1 -- 172.21.15.100:0/567134923 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f53f8004e40 con 0x555ce3eca2e0 2024-05-10T11:33:59.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.253+0000 7f5410184b80 10 client.18525 fetch_fsmap learned FSMap version 37 2024-05-10T11:33:59.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.253+0000 7f5410184b80 1 -- 172.21.15.100:0/567134923 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x555ce3f47160 con 0x555ce3eca2e0 2024-05-10T11:33:59.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.253+0000 7f53deffd700 1 -- 172.21.15.100:0/567134923 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f53f8007b70 con 0x555ce3eca2e0 2024-05-10T11:33:59.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.253+0000 7f53deffd700 1 -- 172.21.15.100:0/567134923 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f53f8011070 con 0x555ce3eca2e0 2024-05-10T11:33:59.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.253+0000 7f5410184b80 10 client.18525 fetch_fsmap finished waiting for FSMap version 37 2024-05-10T11:33:59.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.253+0000 7f5410184b80 10 client.18525 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-10T11:33:59.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.253+0000 7f5410184b80 20 client.18525 populate_metadata read hostname 'smithi100' 2024-05-10T11:33:59.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.253+0000 7f5410184b80 1 --2- 172.21.15.100:0/567134923 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x555ce3f478d0 0x555ce3f49cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:33:59.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.253+0000 7f5410184b80 4 client.18525 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:33:59.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.253+0000 7f5410184b80 1 -- 172.21.15.100:0/567134923 --> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f53f4010e50 con 0x555ce3f478d0 2024-05-10T11:33:59.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.254+0000 7f53feffd700 1 --2- 172.21.15.100:0/567134923 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x555ce3f478d0 0x555ce3f49cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:33:59.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.254+0000 7f53deffd700 10 client.18525 ms_handle_connect on v2:172.21.15.178:6838/3760985404 2024-05-10T11:33:59.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.254+0000 7f53feffd700 1 --2- 172.21.15.100:0/567134923 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x555ce3f478d0 0x555ce3f49cf0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e95c1537937d4a0c server_cookie=e9fe5bfacc9b8810 in_seq=0 out_seq=0 2024-05-10T11:33:59.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.254+0000 7f53deffd700 1 -- 172.21.15.100:0/567134923 <== mds.1 v2:172.21.15.178:6838/3760985404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f53f4010e50 con 0x555ce3f478d0 2024-05-10T11:33:59.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.254+0000 7f53deffd700 10 client.18525 handle_command_reply: tid=0 2024-05-10T11:33:59.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.254+0000 7f5410184b80 1 client.18525 shutdown 2024-05-10T11:33:59.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.255+0000 7f5410184b80 1 -- 172.21.15.100:0/567134923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ce3eca2e0 msgr2=0x555ce3ee0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:59.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.255+0000 7f5410184b80 1 --2- 172.21.15.100:0/567134923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ce3eca2e0 0x555ce3ee0410 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f53f800b0a0 tx=0x7f53f800be20 comp rx=0 tx=0).stop 2024-05-10T11:33:59.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.255+0000 7f5410184b80 1 -- 172.21.15.100:0/567134923 shutdown_connections 2024-05-10T11:33:59.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.255+0000 7f5410184b80 1 --2- 172.21.15.100:0/567134923 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x555ce3f478d0 0x555ce3f49cf0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:59.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.255+0000 7f5410184b80 1 --2- 172.21.15.100:0/567134923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ce3eca2e0 0x555ce3ee0410 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:59.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.255+0000 7f5410184b80 1 --2- 172.21.15.100:0/567134923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ce3ec4560 0x555ce3edf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:59.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.255+0000 7f5410184b80 1 --2- 172.21.15.100:0/567134923 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ce3ec99c0 0x555ce3edfed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:59.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.255+0000 7f5410184b80 1 -- 172.21.15.100:0/567134923 >> 172.21.15.100:0/567134923 conn(0x555ce3de7b70 msgr2=0x555ce3eca6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:33:59.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.255+0000 7f53deffd700 0 client.18525 ms_handle_reset on v2:172.21.15.178:6838/3760985404 2024-05-10T11:33:59.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.255+0000 7f5410184b80 1 -- 172.21.15.100:0/567134923 shutdown_connections 2024-05-10T11:33:59.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.255+0000 7f5410184b80 1 -- 172.21.15.100:0/567134923 wait complete. 2024-05-10T11:33:59.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.255+0000 7f5410184b80 20 client.18525 trim_cache size 0 max 16384 2024-05-10T11:33:59.260 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:33:59.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.256+0000 7f54058f1700 1 -- 172.21.15.100:0/3709339780 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f53e80b2900 msgr2=0x7f53e80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:59.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.256+0000 7f54058f1700 1 --2- 172.21.15.100:0/3709339780 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f53e80b2900 0x7f53e80b4dc0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f53f0009be0 tx=0x7f53f0009260 comp rx=0 tx=0).stop 2024-05-10T11:33:59.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.256+0000 7f54058f1700 1 -- 172.21.15.100:0/3709339780 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5400102d50 msgr2=0x7f54001047e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:33:59.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.256+0000 7f54058f1700 1 --2- 172.21.15.100:0/3709339780 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5400102d50 0x7f54001047e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f53f400e9a0 tx=0x7f53f400ecb0 comp rx=0 tx=0).stop 2024-05-10T11:33:59.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.256+0000 7f54058f1700 1 -- 172.21.15.100:0/3709339780 shutdown_connections 2024-05-10T11:33:59.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.256+0000 7f54058f1700 1 --2- 172.21.15.100:0/3709339780 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f53e80b2900 0x7f53e80b4dc0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:59.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.256+0000 7f54058f1700 1 --2- 172.21.15.100:0/3709339780 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5400108400 0x7f5400104d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:59.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.256+0000 7f54058f1700 1 --2- 172.21.15.100:0/3709339780 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5400102390 0x7f54001042a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:59.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.256+0000 7f54058f1700 1 --2- 172.21.15.100:0/3709339780 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5400102d50 0x7f54001047e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:33:59.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.256+0000 7f54058f1700 1 -- 172.21.15.100:0/3709339780 >> 172.21.15.100:0/3709339780 conn(0x7f5400079310 msgr2=0x7f54001064e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:33:59.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.256+0000 7f54058f1700 1 -- 172.21.15.100:0/3709339780 shutdown_connections 2024-05-10T11:33:59.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:33:59.256+0000 7f54058f1700 1 -- 172.21.15.100:0/3709339780 wait complete. 2024-05-10T11:33:59.914 INFO:tasks.workunit.client.0.smithi100.stdout: 10 19191 10.70 MB/sec warmup 113 sec latency 116.119 ms 2024-05-10T11:34:00.406 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:34:00 smithi100 ceph-mon[30296]: pgmap v889: 129 pgs: 129 active+clean; 389 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 3.9 MiB/s wr, 236 op/s 2024-05-10T11:34:00.442 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-10T11:34:00.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:34:00 smithi178 ceph-mon[33359]: pgmap v889: 129 pgs: 129 active+clean; 389 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 3.9 MiB/s wr, 236 op/s 2024-05-10T11:34:00.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:34:00 smithi188 ceph-mon[34075]: pgmap v889: 129 pgs: 129 active+clean; 389 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 3.9 MiB/s wr, 236 op/s 2024-05-10T11:34:00.823 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:34:00.914 INFO:tasks.workunit.client.0.smithi100.stdout: 10 19337 10.63 MB/sec warmup 114 sec latency 85.092 ms 2024-05-10T11:34:01.914 INFO:tasks.workunit.client.0.smithi100.stdout: 10 19494 10.60 MB/sec warmup 115 sec latency 85.475 ms 2024-05-10T11:34:02.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:34:02 smithi100 ceph-mon[30296]: pgmap v890: 129 pgs: 129 active+clean; 390 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 4.2 MiB/s wr, 239 op/s 2024-05-10T11:34:02.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:34:02 smithi178 ceph-mon[33359]: pgmap v890: 129 pgs: 129 active+clean; 390 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 4.2 MiB/s wr, 239 op/s 2024-05-10T11:34:02.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:34:02 smithi188 ceph-mon[34075]: pgmap v890: 129 pgs: 129 active+clean; 390 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 4.2 MiB/s wr, 239 op/s 2024-05-10T11:34:02.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.723+0000 7ffb21449700 1 -- 172.21.15.100:0/4115628322 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffb1c104100 msgr2=0x7ffb1c108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:02.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.723+0000 7ffb21449700 1 --2- 172.21.15.100:0/4115628322 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffb1c104100 0x7ffb1c108120 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ffb100099c0 tx=0x7ffb10009cd0 comp rx=0 tx=0).stop 2024-05-10T11:34:02.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.723+0000 7ffb21449700 1 -- 172.21.15.100:0/4115628322 shutdown_connections 2024-05-10T11:34:02.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.723+0000 7ffb21449700 1 --2- 172.21.15.100:0/4115628322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb1c108850 0x7ffb1c1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:02.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.723+0000 7ffb21449700 1 --2- 172.21.15.100:0/4115628322 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb1c103750 0x7ffb1c103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:02.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.723+0000 7ffb21449700 1 --2- 172.21.15.100:0/4115628322 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffb1c104100 0x7ffb1c108120 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:02.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.723+0000 7ffb21449700 1 -- 172.21.15.100:0/4115628322 >> 172.21.15.100:0/4115628322 conn(0x7ffb1c07a600 msgr2=0x7ffb1c07aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:34:02.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.723+0000 7ffb21449700 1 -- 172.21.15.100:0/4115628322 shutdown_connections 2024-05-10T11:34:02.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.723+0000 7ffb21449700 1 -- 172.21.15.100:0/4115628322 wait complete. 2024-05-10T11:34:02.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.724+0000 7ffb21449700 1 Processor -- start 2024-05-10T11:34:02.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.724+0000 7ffb21449700 1 -- start start 2024-05-10T11:34:02.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.724+0000 7ffb21449700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb1c103750 0x7ffb1c19f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:02.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.724+0000 7ffb21449700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffb1c104100 0x7ffb1c19fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:02.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.724+0000 7ffb21449700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb1c108850 0x7ffb1c199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:02.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.724+0000 7ffb21449700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ffb1c1134c0 con 0x7ffb1c103750 2024-05-10T11:34:02.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.724+0000 7ffb21449700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ffb1c113340 con 0x7ffb1c104100 2024-05-10T11:34:02.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.724+0000 7ffb21449700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ffb1c113640 con 0x7ffb1c108850 2024-05-10T11:34:02.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.724+0000 7ffb1affd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb1c103750 0x7ffb1c19f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:02.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.724+0000 7ffb1b7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb1c108850 0x7ffb1c199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:02.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.724+0000 7ffb1affd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb1c103750 0x7ffb1c19f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39764/0 (socket says 172.21.15.100:39764) 2024-05-10T11:34:02.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.724+0000 7ffb1b7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb1c108850 0x7ffb1c199840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:33314/0 (socket says 172.21.15.100:33314) 2024-05-10T11:34:02.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.724+0000 7ffb1affd700 1 -- 172.21.15.100:0/1102953179 learned_addr learned my addr 172.21.15.100:0/1102953179 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:34:02.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.725+0000 7ffb1a7fc700 1 --2- 172.21.15.100:0/1102953179 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffb1c104100 0x7ffb1c19fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:02.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.725+0000 7ffb1affd700 1 -- 172.21.15.100:0/1102953179 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb1c108850 msgr2=0x7ffb1c199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:02.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.725+0000 7ffb1affd700 1 --2- 172.21.15.100:0/1102953179 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb1c108850 0x7ffb1c199840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:02.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.725+0000 7ffb1affd700 1 -- 172.21.15.100:0/1102953179 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffb1c104100 msgr2=0x7ffb1c19fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:02.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.725+0000 7ffb1affd700 1 --2- 172.21.15.100:0/1102953179 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffb1c104100 0x7ffb1c19fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:02.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.725+0000 7ffb1affd700 1 -- 172.21.15.100:0/1102953179 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb1c19a020 con 0x7ffb1c103750 2024-05-10T11:34:02.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.725+0000 7ffb1b7fe700 1 --2- 172.21.15.100:0/1102953179 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb1c108850 0x7ffb1c199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:34:02.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.725+0000 7ffb1a7fc700 1 --2- 172.21.15.100:0/1102953179 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffb1c104100 0x7ffb1c19fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:34:02.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.725+0000 7ffb1affd700 1 --2- 172.21.15.100:0/1102953179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb1c103750 0x7ffb1c19f6c0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c00d950 tx=0x7ffb0c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:34:02.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.725+0000 7ffb03fff700 1 -- 172.21.15.100:0/1102953179 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c016070 con 0x7ffb1c103750 2024-05-10T11:34:02.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.726+0000 7ffb21449700 1 -- 172.21.15.100:0/1102953179 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb1c19a310 con 0x7ffb1c103750 2024-05-10T11:34:02.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.726+0000 7ffb03fff700 1 -- 172.21.15.100:0/1102953179 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffb0c009dd0 con 0x7ffb1c103750 2024-05-10T11:34:02.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.726+0000 7ffb03fff700 1 -- 172.21.15.100:0/1102953179 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c020060 con 0x7ffb1c103750 2024-05-10T11:34:02.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.726+0000 7ffb21449700 1 -- 172.21.15.100:0/1102953179 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb1c1a6520 con 0x7ffb1c103750 2024-05-10T11:34:02.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.729+0000 7ffb03fff700 1 -- 172.21.15.100:0/1102953179 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7ffb0c019070 con 0x7ffb1c103750 2024-05-10T11:34:02.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.730+0000 7ffb03fff700 1 --2- 172.21.15.100:0/1102953179 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ffb040b2880 0x7ffb040b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:02.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.731+0000 7ffb1a7fc700 1 --2- 172.21.15.100:0/1102953179 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ffb040b2880 0x7ffb040b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:02.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.731+0000 7ffb03fff700 1 -- 172.21.15.100:0/1102953179 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7ffb0c0e2520 con 0x7ffb1c103750 2024-05-10T11:34:02.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.731+0000 7ffb1a7fc700 1 --2- 172.21.15.100:0/1102953179 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ffb040b2880 0x7ffb040b4d40 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c103130 tx=0x7ffb10005ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:34:02.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.731+0000 7ffb2bcdcb80 1 Processor -- start 2024-05-10T11:34:02.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.731+0000 7ffb2bcdcb80 1 -- start start 2024-05-10T11:34:02.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.732+0000 7ffb2bcdcb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5614f49638e0 0x5614f4963cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:02.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.732+0000 7ffb2bcdcb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614f4964200 0x5614f4967b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:02.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.732+0000 7ffb2bcdcb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5614f4968040 0x5614f4968480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:02.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.732+0000 7ffb2bcdcb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5614f48b1860 con 0x5614f4964200 2024-05-10T11:34:02.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.732+0000 7ffb2bcdcb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5614f48507c0 con 0x5614f4968040 2024-05-10T11:34:02.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.732+0000 7ffb2bcdcb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5614f48ad820 con 0x5614f49638e0 2024-05-10T11:34:02.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.732+0000 7ffb1a7fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614f4964200 0x5614f4967b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:02.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.732+0000 7ffb1b7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5614f4968040 0x5614f4968480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:02.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.732+0000 7ffb1a7fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614f4964200 0x5614f4967b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39780/0 (socket says 172.21.15.100:39780) 2024-05-10T11:34:02.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.732+0000 7ffb1a7fc700 1 -- 172.21.15.100:0/2965522015 learned_addr learned my addr 172.21.15.100:0/2965522015 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:34:02.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.732+0000 7ffb1affd700 1 --2- 172.21.15.100:0/2965522015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5614f49638e0 0x5614f4963cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:02.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.732+0000 7ffb1a7fc700 1 -- 172.21.15.100:0/2965522015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5614f49638e0 msgr2=0x5614f4963cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:02.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.732+0000 7ffb1a7fc700 1 --2- 172.21.15.100:0/2965522015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5614f49638e0 0x5614f4963cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:02.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.732+0000 7ffb1a7fc700 1 -- 172.21.15.100:0/2965522015 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5614f4968040 msgr2=0x5614f4968480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:02.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.732+0000 7ffb1a7fc700 1 --2- 172.21.15.100:0/2965522015 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5614f4968040 0x5614f4968480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:02.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.732+0000 7ffb1a7fc700 1 -- 172.21.15.100:0/2965522015 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614f47fe8d0 con 0x5614f4964200 2024-05-10T11:34:02.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.732+0000 7ffb1b7fe700 1 --2- 172.21.15.100:0/2965522015 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5614f4968040 0x5614f4968480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:34:02.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.732+0000 7ffb1a7fc700 1 --2- 172.21.15.100:0/2965522015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614f4964200 0x5614f4967b00 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7ffb10009750 tx=0x7ffb1001a070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80d3b419925ef58 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:34:02.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.733+0000 7ffb017fa700 1 -- 172.21.15.100:0/2965522015 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb100039c0 con 0x5614f4964200 2024-05-10T11:34:02.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.733+0000 7ffb017fa700 1 -- 172.21.15.100:0/2965522015 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffb1000b790 con 0x5614f4964200 2024-05-10T11:34:02.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.733+0000 7ffb017fa700 1 -- 172.21.15.100:0/2965522015 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb100107f0 con 0x5614f4964200 2024-05-10T11:34:02.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.733+0000 7ffb2bcdcb80 1 -- 172.21.15.100:0/2965522015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614f4964200 msgr2=0x5614f4967b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:02.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.733+0000 7ffb2bcdcb80 1 --2- 172.21.15.100:0/2965522015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614f4964200 0x5614f4967b00 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7ffb10009750 tx=0x7ffb1001a070 comp rx=0 tx=0).stop 2024-05-10T11:34:02.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.734+0000 7ffb2bcdcb80 1 -- 172.21.15.100:0/2965522015 shutdown_connections 2024-05-10T11:34:02.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.734+0000 7ffb2bcdcb80 1 --2- 172.21.15.100:0/2965522015 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5614f4968040 0x5614f4968480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:02.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.734+0000 7ffb2bcdcb80 1 --2- 172.21.15.100:0/2965522015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5614f49638e0 0x5614f4963cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:02.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.734+0000 7ffb2bcdcb80 1 --2- 172.21.15.100:0/2965522015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614f4964200 0x5614f4967b00 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:02.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.734+0000 7ffb2bcdcb80 1 -- 172.21.15.100:0/2965522015 >> 172.21.15.100:0/2965522015 conn(0x5614f488b100 msgr2=0x5614f4959c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:34:02.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.734+0000 7ffb2bcdcb80 1 -- 172.21.15.100:0/2965522015 shutdown_connections 2024-05-10T11:34:02.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.734+0000 7ffb2bcdcb80 1 -- 172.21.15.100:0/2965522015 wait complete. 2024-05-10T11:34:02.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.734+0000 7ffb2bcdcb80 1 -- start start 2024-05-10T11:34:02.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.734+0000 7ffb2bcdcb80 1 Processor -- start 2024-05-10T11:34:02.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.734+0000 7ffb2bcdcb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5614f49638e0 0x5614f496f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:02.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.735+0000 7ffb2bcdcb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5614f4964200 0x5614f496c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:02.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.735+0000 7ffb2bcdcb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614f4968040 0x5614f496cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:02.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.735+0000 7ffb2bcdcb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5614f49701b0 con 0x5614f4968040 2024-05-10T11:34:02.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.735+0000 7ffb2bcdcb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5614f4817c30 con 0x5614f4964200 2024-05-10T11:34:02.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.735+0000 7ffb2bcdcb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5614f4970330 con 0x5614f49638e0 2024-05-10T11:34:02.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.735+0000 7ffb2bcdcb80 10 client.0 _ll_register_callbacks cb 0x5614f4849920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:34:02.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.735+0000 7ffb1affd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5614f49638e0 0x5614f496f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:02.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.735+0000 7ffb1affd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5614f49638e0 0x5614f496f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:33330/0 (socket says 172.21.15.100:33330) 2024-05-10T11:34:02.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.735+0000 7ffb1affd700 1 -- 172.21.15.100:0/2762650379 learned_addr learned my addr 172.21.15.100:0/2762650379 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:34:02.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.735+0000 7ffb1a7fc700 1 --2- 172.21.15.100:0/2762650379 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5614f4964200 0x5614f496c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:02.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.735+0000 7ffb1b7fe700 1 --2- 172.21.15.100:0/2762650379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614f4968040 0x5614f496cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:02.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.735+0000 7ffb1affd700 1 -- 172.21.15.100:0/2762650379 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5614f4964200 msgr2=0x5614f496c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:02.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.735+0000 7ffb1affd700 1 --2- 172.21.15.100:0/2762650379 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5614f4964200 0x5614f496c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:02.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.735+0000 7ffb1affd700 1 -- 172.21.15.100:0/2762650379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614f4968040 msgr2=0x5614f496cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:02.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.735+0000 7ffb1affd700 1 --2- 172.21.15.100:0/2762650379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614f4968040 0x5614f496cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:02.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.735+0000 7ffb1affd700 1 -- 172.21.15.100:0/2762650379 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614f484d690 con 0x5614f49638e0 2024-05-10T11:34:02.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.735+0000 7ffb1b7fe700 1 --2- 172.21.15.100:0/2762650379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614f4968040 0x5614f496cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:34:02.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.735+0000 7ffb1affd700 1 --2- 172.21.15.100:0/2762650379 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5614f49638e0 0x5614f496f4d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c000c00 tx=0x7ffb0c0a92f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45d1a26674b4b358 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:34:02.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.735+0000 7ffafeff5700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:34:02.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.736+0000 7ffafeff5700 1 -- 172.21.15.100:0/2762650379 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c0adb90 con 0x5614f49638e0 2024-05-10T11:34:02.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.736+0000 7ffafeff5700 1 -- 172.21.15.100:0/2762650379 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7ffb0c0cc0c0 con 0x5614f49638e0 2024-05-10T11:34:02.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.736+0000 7ffb2bcdcb80 1 -- 172.21.15.100:0/2762650379 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb1000b790 con 0x5614f49638e0 2024-05-10T11:34:02.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.736+0000 7ffafeff5700 1 -- 172.21.15.100:0/2762650379 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7ffb0c0b2640 con 0x5614f49638e0 2024-05-10T11:34:02.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.736+0000 7ffb2bcdcb80 10 client.27961 fetch_fsmap learned FSMap version 37 2024-05-10T11:34:02.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.736+0000 7ffb2bcdcb80 1 -- 172.21.15.100:0/2762650379 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5614f497a860 con 0x5614f49638e0 2024-05-10T11:34:02.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.736+0000 7ffafeff5700 1 -- 172.21.15.100:0/2762650379 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffb1000b790 con 0x5614f49638e0 2024-05-10T11:34:02.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.736+0000 7ffafeff5700 1 -- 172.21.15.100:0/2762650379 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7ffb0c06b020 con 0x5614f49638e0 2024-05-10T11:34:02.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.736+0000 7ffb2bcdcb80 10 client.27961 fetch_fsmap finished waiting for FSMap version 37 2024-05-10T11:34:02.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.736+0000 7ffb2bcdcb80 10 client.27961 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:34:02.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.736+0000 7ffb2bcdcb80 20 client.27961 populate_metadata read hostname 'smithi100' 2024-05-10T11:34:02.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.737+0000 7ffb2bcdcb80 1 --2- 172.21.15.100:0/2762650379 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5614f497cea0 0x5614f497f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:02.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.737+0000 7ffb2bcdcb80 4 client.27961 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:34:02.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.737+0000 7ffb2bcdcb80 1 -- 172.21.15.100:0/2762650379 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614f492b650 con 0x5614f497cea0 2024-05-10T11:34:02.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.737+0000 7ffb1a7fc700 1 --2- 172.21.15.100:0/2762650379 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5614f497cea0 0x5614f497f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:02.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.737+0000 7ffb1a7fc700 1 --2- 172.21.15.100:0/2762650379 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5614f497cea0 0x5614f497f2c0 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67b76996560df9fd server_cookie=91138736edd89b41 in_seq=0 out_seq=0 2024-05-10T11:34:02.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.737+0000 7ffafeff5700 10 client.27961 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:34:02.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.738+0000 7ffafeff5700 1 -- 172.21.15.100:0/2762650379 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5614f492b650 con 0x5614f497cea0 2024-05-10T11:34:02.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.738+0000 7ffafeff5700 10 client.27961 handle_command_reply: tid=0 2024-05-10T11:34:02.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.738+0000 7ffb2bcdcb80 1 client.27961 shutdown 2024-05-10T11:34:02.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.738+0000 7ffb2bcdcb80 1 -- 172.21.15.100:0/2762650379 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5614f49638e0 msgr2=0x5614f496f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:02.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.738+0000 7ffb2bcdcb80 1 --2- 172.21.15.100:0/2762650379 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5614f49638e0 0x5614f496f4d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c000c00 tx=0x7ffb0c0a92f0 comp rx=0 tx=0).stop 2024-05-10T11:34:02.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.738+0000 7ffb2bcdcb80 1 -- 172.21.15.100:0/2762650379 shutdown_connections 2024-05-10T11:34:02.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.738+0000 7ffb2bcdcb80 1 --2- 172.21.15.100:0/2762650379 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5614f497cea0 0x5614f497f2c0 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:02.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.738+0000 7ffafeff5700 0 client.27961 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:34:02.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.738+0000 7ffb2bcdcb80 1 --2- 172.21.15.100:0/2762650379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614f4968040 0x5614f496cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:02.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.738+0000 7ffb2bcdcb80 1 --2- 172.21.15.100:0/2762650379 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5614f49638e0 0x5614f496f4d0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:02.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.738+0000 7ffb2bcdcb80 1 --2- 172.21.15.100:0/2762650379 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5614f4964200 0x5614f496c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:02.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.738+0000 7ffb2bcdcb80 1 -- 172.21.15.100:0/2762650379 >> 172.21.15.100:0/2762650379 conn(0x5614f488b100 msgr2=0x5614f4954720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:34:02.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.739+0000 7ffb2bcdcb80 1 -- 172.21.15.100:0/2762650379 shutdown_connections 2024-05-10T11:34:02.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.739+0000 7ffb2bcdcb80 1 -- 172.21.15.100:0/2762650379 wait complete. 2024-05-10T11:34:02.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.739+0000 7ffb2bcdcb80 20 client.27961 trim_cache size 0 max 16384 2024-05-10T11:34:02.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.766+0000 7ffb2bcdcb80 1 Processor -- start 2024-05-10T11:34:02.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.767+0000 7ffb2bcdcb80 1 -- start start 2024-05-10T11:34:02.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.767+0000 7ffb2bcdcb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614f49638e0 0x5614f49792a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:02.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.767+0000 7ffb1affd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614f49638e0 0x5614f49792a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:02.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.767+0000 7ffb1affd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614f49638e0 0x5614f49792a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39788/0 (socket says 172.21.15.100:39788) 2024-05-10T11:34:02.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.767+0000 7ffb1affd700 1 -- 172.21.15.100:0/2486895905 learned_addr learned my addr 172.21.15.100:0/2486895905 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:34:02.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.767+0000 7ffb2bcdcb80 1 --2- 172.21.15.100:0/2486895905 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5614f4964200 0x5614f4953710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:02.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.767+0000 7ffb2bcdcb80 1 --2- 172.21.15.100:0/2486895905 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5614f4968040 0x5614f496a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:02.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.767+0000 7ffb2bcdcb80 1 -- 172.21.15.100:0/2486895905 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ffb1c1af760 con 0x5614f49638e0 2024-05-10T11:34:02.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.767+0000 7ffb2bcdcb80 1 -- 172.21.15.100:0/2486895905 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ffb1c1acf60 con 0x5614f4964200 2024-05-10T11:34:02.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.767+0000 7ffb2bcdcb80 1 -- 172.21.15.100:0/2486895905 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ffb1c1aadd0 con 0x5614f4968040 2024-05-10T11:34:02.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.768+0000 7ffb1affd700 1 -- 172.21.15.100:0/2486895905 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5614f4968040 msgr2=0x5614f496a370 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T11:34:02.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.768+0000 7ffb1affd700 1 --2- 172.21.15.100:0/2486895905 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5614f4968040 0x5614f496a370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:02.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.768+0000 7ffb1affd700 1 -- 172.21.15.100:0/2486895905 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5614f4964200 msgr2=0x5614f4953710 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T11:34:02.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.768+0000 7ffb1affd700 1 --2- 172.21.15.100:0/2486895905 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5614f4964200 0x5614f4953710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:02.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.768+0000 7ffb1affd700 1 -- 172.21.15.100:0/2486895905 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614f4956810 con 0x5614f49638e0 2024-05-10T11:34:02.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.768+0000 7ffb1affd700 1 --2- 172.21.15.100:0/2486895905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614f49638e0 0x5614f49792a0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c0dfd10 tx=0x7ffb0c0e2280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bd07753b752d877 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:34:02.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.769+0000 7ffb00ff9700 1 -- 172.21.15.100:0/2486895905 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c0adb90 con 0x5614f49638e0 2024-05-10T11:34:02.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.769+0000 7ffb00ff9700 1 -- 172.21.15.100:0/2486895905 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffb0c0e1030 con 0x5614f49638e0 2024-05-10T11:34:02.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.770+0000 7ffb00ff9700 1 -- 172.21.15.100:0/2486895905 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c0106a0 con 0x5614f49638e0 2024-05-10T11:34:02.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.770+0000 7ffb2bcdcb80 1 -- 172.21.15.100:0/2486895905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614f49638e0 msgr2=0x5614f49792a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:02.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.770+0000 7ffb2bcdcb80 1 --2- 172.21.15.100:0/2486895905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614f49638e0 0x5614f49792a0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c0dfd10 tx=0x7ffb0c0e2280 comp rx=0 tx=0).stop 2024-05-10T11:34:02.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.770+0000 7ffb2bcdcb80 1 -- 172.21.15.100:0/2486895905 shutdown_connections 2024-05-10T11:34:02.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.770+0000 7ffb2bcdcb80 1 --2- 172.21.15.100:0/2486895905 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5614f4968040 0x5614f496a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:02.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.770+0000 7ffb2bcdcb80 1 --2- 172.21.15.100:0/2486895905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614f49638e0 0x5614f49792a0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:02.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.770+0000 7ffb2bcdcb80 1 --2- 172.21.15.100:0/2486895905 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5614f4964200 0x5614f4953710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:02.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.770+0000 7ffb2bcdcb80 1 -- 172.21.15.100:0/2486895905 >> 172.21.15.100:0/2486895905 conn(0x5614f488b100 msgr2=0x5614f4969c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:34:02.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.771+0000 7ffb2bcdcb80 1 -- 172.21.15.100:0/2486895905 shutdown_connections 2024-05-10T11:34:02.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.771+0000 7ffb2bcdcb80 1 -- 172.21.15.100:0/2486895905 wait complete. 2024-05-10T11:34:02.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.771+0000 7ffb2bcdcb80 1 -- start start 2024-05-10T11:34:02.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.771+0000 7ffb2bcdcb80 1 Processor -- start 2024-05-10T11:34:02.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.772+0000 7ffb2bcdcb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5614f49638e0 0x5614f4962fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:02.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.772+0000 7ffb2bcdcb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614f4964200 0x5614f495f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:02.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.772+0000 7ffb2bcdcb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5614f4968040 0x5614f495f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:02.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.772+0000 7ffb2bcdcb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ffb1c1ab420 con 0x5614f4964200 2024-05-10T11:34:02.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.772+0000 7ffb2bcdcb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ffb1c1ac890 con 0x5614f49638e0 2024-05-10T11:34:02.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.772+0000 7ffb2bcdcb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ffb1c1abaf0 con 0x5614f4968040 2024-05-10T11:34:02.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.772+0000 7ffb2bcdcb80 10 client.0 _ll_register_callbacks cb 0x5614f4849920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:34:02.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.772+0000 7ffb1a7fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614f4964200 0x5614f495f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:02.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.772+0000 7ffb1a7fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614f4964200 0x5614f495f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39792/0 (socket says 172.21.15.100:39792) 2024-05-10T11:34:02.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.772+0000 7ffb1a7fc700 1 -- 172.21.15.100:0/63139909 learned_addr learned my addr 172.21.15.100:0/63139909 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:34:02.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.772+0000 7ffb1b7fe700 1 --2- 172.21.15.100:0/63139909 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5614f4968040 0x5614f495f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:02.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.773+0000 7ffb1a7fc700 1 -- 172.21.15.100:0/63139909 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5614f4968040 msgr2=0x5614f495f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:02.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.773+0000 7ffb1affd700 1 --2- 172.21.15.100:0/63139909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5614f49638e0 0x5614f4962fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:02.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.773+0000 7ffb1a7fc700 1 --2- 172.21.15.100:0/63139909 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5614f4968040 0x5614f495f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:02.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.773+0000 7ffb1a7fc700 1 -- 172.21.15.100:0/63139909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5614f49638e0 msgr2=0x5614f4962fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:02.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.773+0000 7ffb1a7fc700 1 --2- 172.21.15.100:0/63139909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5614f49638e0 0x5614f4962fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:02.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.773+0000 7ffb1a7fc700 1 -- 172.21.15.100:0/63139909 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614f49600e0 con 0x5614f4964200 2024-05-10T11:34:02.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.773+0000 7ffb1a7fc700 1 --2- 172.21.15.100:0/63139909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614f4964200 0x5614f495f380 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7ffb100059d0 tx=0x7ffb10003870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d18472f6d72107cd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:34:02.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.773+0000 7ffafeff5700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:34:02.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.774+0000 7ffafeff5700 1 -- 172.21.15.100:0/63139909 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb10010470 con 0x5614f4964200 2024-05-10T11:34:02.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.774+0000 7ffb2bcdcb80 1 -- 172.21.15.100:0/63139909 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb0c0e1030 con 0x5614f4964200 2024-05-10T11:34:02.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.774+0000 7ffafeff5700 1 -- 172.21.15.100:0/63139909 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7ffb10010610 con 0x5614f4964200 2024-05-10T11:34:02.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.774+0000 7ffafeff5700 1 -- 172.21.15.100:0/63139909 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7ffb10010810 con 0x5614f4964200 2024-05-10T11:34:02.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.774+0000 7ffb2bcdcb80 10 client.18543 fetch_fsmap learned FSMap version 37 2024-05-10T11:34:02.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.774+0000 7ffb2bcdcb80 1 -- 172.21.15.100:0/63139909 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5614f4975600 con 0x5614f4964200 2024-05-10T11:34:02.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.775+0000 7ffafeff5700 1 -- 172.21.15.100:0/63139909 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffb0c0e1030 con 0x5614f4964200 2024-05-10T11:34:02.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.775+0000 7ffafeff5700 1 -- 172.21.15.100:0/63139909 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7ffb1003f110 con 0x5614f4964200 2024-05-10T11:34:02.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.775+0000 7ffb2bcdcb80 10 client.18543 fetch_fsmap finished waiting for FSMap version 37 2024-05-10T11:34:02.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.775+0000 7ffb2bcdcb80 10 client.18543 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:34:02.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.775+0000 7ffb2bcdcb80 20 client.18543 populate_metadata read hostname 'smithi100' 2024-05-10T11:34:02.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.775+0000 7ffb2bcdcb80 1 --2- 172.21.15.100:0/63139909 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5614f497cea0 0x5614f4975940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:02.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.775+0000 7ffb2bcdcb80 4 client.18543 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:34:02.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.775+0000 7ffb2bcdcb80 1 -- 172.21.15.100:0/63139909 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5614f48bf240 con 0x5614f497cea0 2024-05-10T11:34:02.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.776+0000 7ffb1affd700 1 --2- 172.21.15.100:0/63139909 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5614f497cea0 0x5614f4975940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:02.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.779+0000 7ffb1affd700 1 --2- 172.21.15.100:0/63139909 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5614f497cea0 0x5614f4975940 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd280d762c7a56f9 server_cookie=20d941d5ab7ded35 in_seq=0 out_seq=0 2024-05-10T11:34:02.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.779+0000 7ffafeff5700 10 client.18543 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:34:02.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.780+0000 7ffafeff5700 1 -- 172.21.15.100:0/63139909 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5614f48bf240 con 0x5614f497cea0 2024-05-10T11:34:02.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.780+0000 7ffafeff5700 10 client.18543 handle_command_reply: tid=0 2024-05-10T11:34:02.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.780+0000 7ffb2bcdcb80 1 client.18543 shutdown 2024-05-10T11:34:02.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.784+0000 7ffb2bcdcb80 1 -- 172.21.15.100:0/63139909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614f4964200 msgr2=0x5614f495f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:02.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.784+0000 7ffb2bcdcb80 1 --2- 172.21.15.100:0/63139909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614f4964200 0x5614f495f380 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7ffb100059d0 tx=0x7ffb10003870 comp rx=0 tx=0).stop 2024-05-10T11:34:02.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.785+0000 7ffb2bcdcb80 1 -- 172.21.15.100:0/63139909 shutdown_connections 2024-05-10T11:34:02.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.785+0000 7ffb2bcdcb80 1 --2- 172.21.15.100:0/63139909 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5614f497cea0 0x5614f4975940 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:02.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.785+0000 7ffb2bcdcb80 1 --2- 172.21.15.100:0/63139909 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5614f4968040 0x5614f495f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:02.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.785+0000 7ffb2bcdcb80 1 --2- 172.21.15.100:0/63139909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5614f49638e0 0x5614f4962fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:02.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.785+0000 7ffb2bcdcb80 1 --2- 172.21.15.100:0/63139909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614f4964200 0x5614f495f380 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:02.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.785+0000 7ffb2bcdcb80 1 -- 172.21.15.100:0/63139909 >> 172.21.15.100:0/63139909 conn(0x5614f488b100 msgr2=0x5614f4953e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:34:02.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.787+0000 7ffafeff5700 0 client.18543 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:34:02.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.787+0000 7ffb2bcdcb80 1 -- 172.21.15.100:0/63139909 shutdown_connections 2024-05-10T11:34:02.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.787+0000 7ffb2bcdcb80 1 -- 172.21.15.100:0/63139909 wait complete. 2024-05-10T11:34:02.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.788+0000 7ffb2bcdcb80 20 client.18543 trim_cache size 0 max 16384 2024-05-10T11:34:02.788 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:34:02.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.790+0000 7ffb21449700 1 -- 172.21.15.100:0/1102953179 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ffb040b2880 msgr2=0x7ffb040b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:02.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.790+0000 7ffb21449700 1 --2- 172.21.15.100:0/1102953179 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ffb040b2880 0x7ffb040b4d40 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c103130 tx=0x7ffb10005ab0 comp rx=0 tx=0).stop 2024-05-10T11:34:02.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.791+0000 7ffb21449700 1 -- 172.21.15.100:0/1102953179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb1c103750 msgr2=0x7ffb1c19f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:02.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.791+0000 7ffb21449700 1 --2- 172.21.15.100:0/1102953179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb1c103750 0x7ffb1c19f6c0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c00d950 tx=0x7ffb0c00dc60 comp rx=0 tx=0).stop 2024-05-10T11:34:02.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.791+0000 7ffb21449700 1 -- 172.21.15.100:0/1102953179 shutdown_connections 2024-05-10T11:34:02.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.791+0000 7ffb21449700 1 --2- 172.21.15.100:0/1102953179 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ffb040b2880 0x7ffb040b4d40 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:02.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.791+0000 7ffb21449700 1 --2- 172.21.15.100:0/1102953179 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb1c108850 0x7ffb1c199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:02.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.791+0000 7ffb21449700 1 --2- 172.21.15.100:0/1102953179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb1c103750 0x7ffb1c19f6c0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:02.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.791+0000 7ffb21449700 1 --2- 172.21.15.100:0/1102953179 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffb1c104100 0x7ffb1c19fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:02.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.791+0000 7ffb21449700 1 -- 172.21.15.100:0/1102953179 >> 172.21.15.100:0/1102953179 conn(0x7ffb1c07a600 msgr2=0x7ffb1c101880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:34:02.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.792+0000 7ffb21449700 1 -- 172.21.15.100:0/1102953179 shutdown_connections 2024-05-10T11:34:02.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:02.792+0000 7ffb21449700 1 -- 172.21.15.100:0/1102953179 wait complete. 2024-05-10T11:34:02.914 INFO:tasks.workunit.client.0.smithi100.stdout: 10 19663 10.58 MB/sec warmup 116 sec latency 95.623 ms 2024-05-10T11:34:03.914 INFO:tasks.workunit.client.0.smithi100.stdout: 10 19885 10.60 MB/sec warmup 117 sec latency 58.752 ms 2024-05-10T11:34:04.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:34:04 smithi178 ceph-mon[33359]: pgmap v891: 129 pgs: 129 active+clean; 393 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 4.4 MiB/s wr, 271 op/s 2024-05-10T11:34:04.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:34:04 smithi188 ceph-mon[34075]: pgmap v891: 129 pgs: 129 active+clean; 393 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 4.4 MiB/s wr, 271 op/s 2024-05-10T11:34:04.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:34:04 smithi100 ceph-mon[30296]: pgmap v891: 129 pgs: 129 active+clean; 393 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 4.4 MiB/s wr, 271 op/s 2024-05-10T11:34:04.687 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T11:34:04.688 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T11:34:04.914 INFO:tasks.workunit.client.0.smithi100.stdout: 10 20066 10.59 MB/sec warmup 118 sec latency 77.433 ms 2024-05-10T11:34:05.052 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:34:05.914 INFO:tasks.workunit.client.0.smithi100.stdout: 10 20231 10.57 MB/sec warmup 119 sec latency 84.945 ms 2024-05-10T11:34:06.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:34:06 smithi178 ceph-mon[33359]: pgmap v892: 129 pgs: 129 active+clean; 391 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 4.3 MiB/s wr, 261 op/s 2024-05-10T11:34:06.521 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:34:06 smithi100 ceph-mon[30296]: pgmap v892: 129 pgs: 129 active+clean; 391 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 4.3 MiB/s wr, 261 op/s 2024-05-10T11:34:06.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:34:06 smithi188 ceph-mon[34075]: pgmap v892: 129 pgs: 129 active+clean; 391 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 4.3 MiB/s wr, 261 op/s 2024-05-10T11:34:06.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.974+0000 7f6661a91700 1 -- 172.21.15.100:0/2330228343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f665c109510 msgr2=0x7f665c1098f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:06.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.974+0000 7f6661a91700 1 --2- 172.21.15.100:0/2330228343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f665c109510 0x7f665c1098f0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f664c00b0a0 tx=0x7f664c00b3b0 comp rx=0 tx=0).stop 2024-05-10T11:34:06.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.974+0000 7f6661a91700 1 -- 172.21.15.100:0/2330228343 shutdown_connections 2024-05-10T11:34:06.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.974+0000 7f6661a91700 1 --2- 172.21.15.100:0/2330228343 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f665c06bb40 0x7f665c110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:06.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.974+0000 7f6661a91700 1 --2- 172.21.15.100:0/2330228343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f665c109510 0x7f665c1098f0 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:06.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.974+0000 7f6661a91700 1 --2- 172.21.15.100:0/2330228343 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f665c06b180 0x7f665c06b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:06.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.974+0000 7f6661a91700 1 -- 172.21.15.100:0/2330228343 >> 172.21.15.100:0/2330228343 conn(0x7f665c079350 msgr2=0x7f665c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:34:06.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.974+0000 7f6661a91700 1 -- 172.21.15.100:0/2330228343 shutdown_connections 2024-05-10T11:34:06.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.974+0000 7f6661a91700 1 -- 172.21.15.100:0/2330228343 wait complete. 2024-05-10T11:34:06.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.975+0000 7f6661a91700 1 Processor -- start 2024-05-10T11:34:06.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.975+0000 7f6661a91700 1 -- start start 2024-05-10T11:34:06.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.976+0000 7f6661a91700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f665c06b180 0x7f665c19f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:06.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.976+0000 7f6661a91700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f665c06bb40 0x7f665c19fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:06.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.976+0000 7f6661a91700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f665c109510 0x7f665c1998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:06.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.976+0000 7f6661a91700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f665c113520 con 0x7f665c06bb40 2024-05-10T11:34:06.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.976+0000 7f6661a91700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f665c1133a0 con 0x7f665c109510 2024-05-10T11:34:06.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.976+0000 7f6661a91700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f665c1136a0 con 0x7f665c06b180 2024-05-10T11:34:06.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.976+0000 7f665affd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f665c06bb40 0x7f665c19fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:06.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.976+0000 7f665affd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f665c06bb40 0x7f665c19fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59970/0 (socket says 172.21.15.100:59970) 2024-05-10T11:34:06.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.976+0000 7f665affd700 1 -- 172.21.15.100:0/2935355612 learned_addr learned my addr 172.21.15.100:0/2935355612 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:34:06.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.976+0000 7f665bfff700 1 --2- 172.21.15.100:0/2935355612 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f665c109510 0x7f665c1998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:06.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.976+0000 7f665b7fe700 1 --2- 172.21.15.100:0/2935355612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f665c06b180 0x7f665c19f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:06.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.976+0000 7f665affd700 1 -- 172.21.15.100:0/2935355612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f665c06b180 msgr2=0x7f665c19f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:06.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.976+0000 7f665affd700 1 --2- 172.21.15.100:0/2935355612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f665c06b180 0x7f665c19f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:06.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.976+0000 7f665affd700 1 -- 172.21.15.100:0/2935355612 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f665c109510 msgr2=0x7f665c1998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:06.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.976+0000 7f665affd700 1 --2- 172.21.15.100:0/2935355612 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f665c109510 0x7f665c1998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:06.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.976+0000 7f665affd700 1 -- 172.21.15.100:0/2935355612 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f665c19a040 con 0x7f665c06bb40 2024-05-10T11:34:06.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.977+0000 7f665b7fe700 1 --2- 172.21.15.100:0/2935355612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f665c06b180 0x7f665c19f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:34:06.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.977+0000 7f665bfff700 1 --2- 172.21.15.100:0/2935355612 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f665c109510 0x7f665c1998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:34:06.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.977+0000 7f665affd700 1 --2- 172.21.15.100:0/2935355612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f665c06bb40 0x7f665c19fc60 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f665000eb30 tx=0x7f665000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:34:06.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.977+0000 7f6658ff9700 1 -- 172.21.15.100:0/2935355612 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665000cc90 con 0x7f665c06bb40 2024-05-10T11:34:06.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.977+0000 7f6661a91700 1 -- 172.21.15.100:0/2935355612 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f665c19a330 con 0x7f665c06bb40 2024-05-10T11:34:06.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.977+0000 7f6658ff9700 1 -- 172.21.15.100:0/2935355612 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f665000ce30 con 0x7f665c06bb40 2024-05-10T11:34:06.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.977+0000 7f6661a91700 1 -- 172.21.15.100:0/2935355612 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f665c1a65f0 con 0x7f665c06bb40 2024-05-10T11:34:06.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.977+0000 7f6658ff9700 1 -- 172.21.15.100:0/2935355612 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6650003e10 con 0x7f665c06bb40 2024-05-10T11:34:06.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.980+0000 7f6658ff9700 1 -- 172.21.15.100:0/2935355612 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f6650003fd0 con 0x7f665c06bb40 2024-05-10T11:34:06.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.981+0000 7f6658ff9700 1 --2- 172.21.15.100:0/2935355612 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f66440b2b00 0x7f66440b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:06.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.982+0000 7f665b7fe700 1 --2- 172.21.15.100:0/2935355612 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f66440b2b00 0x7f66440b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:06.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.982+0000 7f6658ff9700 1 -- 172.21.15.100:0/2935355612 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f6650003bb0 con 0x7f665c06bb40 2024-05-10T11:34:06.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.982+0000 7f666c324b80 1 Processor -- start 2024-05-10T11:34:06.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.982+0000 7f666c324b80 1 -- start start 2024-05-10T11:34:06.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.982+0000 7f665b7fe700 1 --2- 172.21.15.100:0/2935355612 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f66440b2b00 0x7f66440b4fc0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f664c009e60 tx=0x7f664c00b740 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:34:06.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.983+0000 7f666c324b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556badb961e0 0x556badb965c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:06.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.983+0000 7f666c324b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556badb92890 0x556badb92d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:06.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.983+0000 7f666c324b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556badb93250 0x556badba7500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:06.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.983+0000 7f666c324b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x556badae49c0 con 0x556badb92890 2024-05-10T11:34:06.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.983+0000 7f666c324b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556bada49e20 con 0x556badb93250 2024-05-10T11:34:06.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.983+0000 7f666c324b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556badae0980 con 0x556badb961e0 2024-05-10T11:34:06.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.983+0000 7f665affd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556badb92890 0x556badb92d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:06.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.983+0000 7f665affd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556badb92890 0x556badb92d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59982/0 (socket says 172.21.15.100:59982) 2024-05-10T11:34:06.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.983+0000 7f665b7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556badb961e0 0x556badb965c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:06.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.983+0000 7f665affd700 1 -- 172.21.15.100:0/2479529982 learned_addr learned my addr 172.21.15.100:0/2479529982 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:34:06.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.983+0000 7f665bfff700 1 --2- 172.21.15.100:0/2479529982 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556badb93250 0x556badba7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:06.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.983+0000 7f665bfff700 1 -- 172.21.15.100:0/2479529982 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556badb961e0 msgr2=0x556badb965c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:06.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.983+0000 7f665bfff700 1 --2- 172.21.15.100:0/2479529982 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556badb961e0 0x556badb965c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:06.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.983+0000 7f665bfff700 1 -- 172.21.15.100:0/2479529982 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556badb92890 msgr2=0x556badb92d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:06.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.983+0000 7f665bfff700 1 --2- 172.21.15.100:0/2479529982 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556badb92890 0x556badb92d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:06.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.983+0000 7f665bfff700 1 -- 172.21.15.100:0/2479529982 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bada31950 con 0x556badb93250 2024-05-10T11:34:06.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.983+0000 7f665affd700 1 --2- 172.21.15.100:0/2479529982 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556badb92890 0x556badb92d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:34:06.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.983+0000 7f665bfff700 1 --2- 172.21.15.100:0/2479529982 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556badb93250 0x556badba7500 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6654009fc0 tx=0x7f6654011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dba01969694bc6c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:34:06.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.984+0000 7f6641ffb700 1 -- 172.21.15.100:0/2479529982 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6654004310 con 0x556badb93250 2024-05-10T11:34:06.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.984+0000 7f6641ffb700 1 -- 172.21.15.100:0/2479529982 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f66540044b0 con 0x556badb93250 2024-05-10T11:34:06.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.984+0000 7f6641ffb700 1 -- 172.21.15.100:0/2479529982 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6654003680 con 0x556badb93250 2024-05-10T11:34:06.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.984+0000 7f666c324b80 1 -- 172.21.15.100:0/2479529982 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556badb93250 msgr2=0x556badba7500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:06.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.984+0000 7f666c324b80 1 --2- 172.21.15.100:0/2479529982 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556badb93250 0x556badba7500 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6654009fc0 tx=0x7f6654011720 comp rx=0 tx=0).stop 2024-05-10T11:34:06.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.984+0000 7f666c324b80 1 -- 172.21.15.100:0/2479529982 shutdown_connections 2024-05-10T11:34:06.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.984+0000 7f666c324b80 1 --2- 172.21.15.100:0/2479529982 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556badb93250 0x556badba7500 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:06.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.984+0000 7f666c324b80 1 --2- 172.21.15.100:0/2479529982 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556badb961e0 0x556badb965c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:06.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.984+0000 7f666c324b80 1 --2- 172.21.15.100:0/2479529982 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556badb92890 0x556badb92d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:06.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.984+0000 7f666c324b80 1 -- 172.21.15.100:0/2479529982 >> 172.21.15.100:0/2479529982 conn(0x556badabebf0 msgr2=0x556badb8d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:34:06.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.985+0000 7f666c324b80 1 -- 172.21.15.100:0/2479529982 shutdown_connections 2024-05-10T11:34:06.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.985+0000 7f666c324b80 1 -- 172.21.15.100:0/2479529982 wait complete. 2024-05-10T11:34:06.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.985+0000 7f666c324b80 1 -- start start 2024-05-10T11:34:06.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.985+0000 7f666c324b80 1 Processor -- start 2024-05-10T11:34:06.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.985+0000 7f666c324b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556badb92890 0x556badba7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:06.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.985+0000 7f666c324b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556badb93250 0x556badba44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:06.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.985+0000 7f666c324b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556badb961e0 0x556badba49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:06.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.986+0000 7f666c324b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x556badba7ce0 con 0x556badb92890 2024-05-10T11:34:06.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.986+0000 7f666c324b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556bada9df50 con 0x556badb93250 2024-05-10T11:34:06.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.986+0000 7f666c324b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556badba7e60 con 0x556badb961e0 2024-05-10T11:34:06.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.986+0000 7f666c324b80 10 client.0 _ll_register_callbacks cb 0x556bada78630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:34:06.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.986+0000 7f665bfff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556badb961e0 0x556badba49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:06.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.986+0000 7f665b7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556badb92890 0x556badba7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:06.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.986+0000 7f665bfff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556badb961e0 0x556badba49e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:40692/0 (socket says 172.21.15.100:40692) 2024-05-10T11:34:06.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.986+0000 7f665bfff700 1 -- 172.21.15.100:0/779279787 learned_addr learned my addr 172.21.15.100:0/779279787 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:34:06.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.986+0000 7f665affd700 1 --2- 172.21.15.100:0/779279787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556badb93250 0x556badba44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:06.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.986+0000 7f665bfff700 1 -- 172.21.15.100:0/779279787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556badb93250 msgr2=0x556badba44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:06.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.986+0000 7f665bfff700 1 --2- 172.21.15.100:0/779279787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556badb93250 0x556badba44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:06.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.986+0000 7f665bfff700 1 -- 172.21.15.100:0/779279787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556badb92890 msgr2=0x556badba7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:06.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.986+0000 7f665bfff700 1 --2- 172.21.15.100:0/779279787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556badb92890 0x556badba7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:06.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.986+0000 7f665bfff700 1 -- 172.21.15.100:0/779279787 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bada9b7e0 con 0x556badb961e0 2024-05-10T11:34:06.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.986+0000 7f665b7fe700 1 --2- 172.21.15.100:0/779279787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556badb92890 0x556badba7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:34:06.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.986+0000 7f665affd700 1 --2- 172.21.15.100:0/779279787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556badb93250 0x556badba44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:34:06.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.986+0000 7f665bfff700 1 --2- 172.21.15.100:0/779279787 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556badb961e0 0x556badba49e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6654009d00 tx=0x7f6654009d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c969bfc355b59674 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:34:06.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.986+0000 7f663affd700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:34:06.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.987+0000 7f663affd700 1 -- 172.21.15.100:0/779279787 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6654026020 con 0x556badb961e0 2024-05-10T11:34:06.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.987+0000 7f663affd700 1 -- 172.21.15.100:0/779279787 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f6654003890 con 0x556badb961e0 2024-05-10T11:34:06.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.987+0000 7f666c324b80 1 -- 172.21.15.100:0/779279787 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66540044b0 con 0x556badb961e0 2024-05-10T11:34:06.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.987+0000 7f663affd700 1 -- 172.21.15.100:0/779279787 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f6654031050 con 0x556badb961e0 2024-05-10T11:34:06.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.987+0000 7f666c324b80 10 client.27976 fetch_fsmap learned FSMap version 37 2024-05-10T11:34:06.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.987+0000 7f666c324b80 1 -- 172.21.15.100:0/779279787 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x556badbb24e0 con 0x556badb961e0 2024-05-10T11:34:06.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.987+0000 7f663affd700 1 -- 172.21.15.100:0/779279787 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f66540044b0 con 0x556badb961e0 2024-05-10T11:34:06.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.987+0000 7f663affd700 1 -- 172.21.15.100:0/779279787 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f6654018070 con 0x556badb961e0 2024-05-10T11:34:06.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.987+0000 7f666c324b80 10 client.27976 fetch_fsmap finished waiting for FSMap version 37 2024-05-10T11:34:06.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.987+0000 7f666c324b80 10 client.27976 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:34:06.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.987+0000 7f666c324b80 20 client.27976 populate_metadata read hostname 'smithi100' 2024-05-10T11:34:06.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.987+0000 7f666c324b80 1 --2- 172.21.15.100:0/779279787 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556badbb4ac0 0x556badbb6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:06.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.987+0000 7f666c324b80 4 client.27976 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:34:06.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.987+0000 7f666c324b80 1 -- 172.21.15.100:0/779279787 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556badaf23a0 con 0x556badbb4ac0 2024-05-10T11:34:06.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.988+0000 7f665b7fe700 1 --2- 172.21.15.100:0/779279787 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556badbb4ac0 0x556badbb6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:06.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.988+0000 7f663affd700 10 client.27976 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:34:06.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.988+0000 7f665b7fe700 1 --2- 172.21.15.100:0/779279787 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556badbb4ac0 0x556badbb6eb0 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4301428e036d3260 server_cookie=a98c8125e8b68217 in_seq=0 out_seq=0 2024-05-10T11:34:06.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.989+0000 7f663affd700 1 -- 172.21.15.100:0/779279787 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556badaf23a0 con 0x556badbb4ac0 2024-05-10T11:34:06.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.989+0000 7f663affd700 10 client.27976 handle_command_reply: tid=0 2024-05-10T11:34:06.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.989+0000 7f666c324b80 1 client.27976 shutdown 2024-05-10T11:34:06.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.989+0000 7f666c324b80 1 -- 172.21.15.100:0/779279787 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556badb961e0 msgr2=0x556badba49e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:06.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.989+0000 7f666c324b80 1 --2- 172.21.15.100:0/779279787 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556badb961e0 0x556badba49e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6654009d00 tx=0x7f6654009d30 comp rx=0 tx=0).stop 2024-05-10T11:34:06.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.989+0000 7f666c324b80 1 -- 172.21.15.100:0/779279787 shutdown_connections 2024-05-10T11:34:06.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.989+0000 7f666c324b80 1 --2- 172.21.15.100:0/779279787 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556badbb4ac0 0x556badbb6eb0 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:06.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.989+0000 7f666c324b80 1 --2- 172.21.15.100:0/779279787 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556badb961e0 0x556badba49e0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:06.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.989+0000 7f666c324b80 1 --2- 172.21.15.100:0/779279787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556badb92890 0x556badba7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:06.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.989+0000 7f666c324b80 1 --2- 172.21.15.100:0/779279787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556badb93250 0x556badba44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:06.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.989+0000 7f666c324b80 1 -- 172.21.15.100:0/779279787 >> 172.21.15.100:0/779279787 conn(0x556badabebf0 msgr2=0x556badb87bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:34:06.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.989+0000 7f663affd700 0 client.27976 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:34:06.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.989+0000 7f666c324b80 1 -- 172.21.15.100:0/779279787 shutdown_connections 2024-05-10T11:34:06.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.989+0000 7f666c324b80 1 -- 172.21.15.100:0/779279787 wait complete. 2024-05-10T11:34:06.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:06.989+0000 7f666c324b80 20 client.27976 trim_cache size 0 max 16384 2024-05-10T11:34:07.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.017+0000 7f666c324b80 1 Processor -- start 2024-05-10T11:34:07.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.017+0000 7f666c324b80 1 -- start start 2024-05-10T11:34:07.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.017+0000 7f666c324b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556badb92890 0x556badb87cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:07.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.017+0000 7f666c324b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556badb93250 0x556badba61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:07.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.017+0000 7f666c324b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556badb961e0 0x556badba66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:07.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.017+0000 7f666c324b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f665c1af930 con 0x556badb92890 2024-05-10T11:34:07.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.017+0000 7f666c324b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f665c1ad130 con 0x556badb961e0 2024-05-10T11:34:07.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.017+0000 7f666c324b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f665c1aaf80 con 0x556badb93250 2024-05-10T11:34:07.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.017+0000 7f665b7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556badb92890 0x556badb87cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:07.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.017+0000 7f665b7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556badb92890 0x556badb87cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59992/0 (socket says 172.21.15.100:59992) 2024-05-10T11:34:07.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.017+0000 7f665b7fe700 1 -- 172.21.15.100:0/1487845287 learned_addr learned my addr 172.21.15.100:0/1487845287 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:34:07.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.017+0000 7f665affd700 1 --2- 172.21.15.100:0/1487845287 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556badb93250 0x556badba61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:07.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.017+0000 7f665bfff700 1 --2- 172.21.15.100:0/1487845287 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556badb961e0 0x556badba66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:07.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.017+0000 7f665b7fe700 1 -- 172.21.15.100:0/1487845287 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556badb93250 msgr2=0x556badba61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:07.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.017+0000 7f665b7fe700 1 --2- 172.21.15.100:0/1487845287 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556badb93250 0x556badba61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:07.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.017+0000 7f665b7fe700 1 -- 172.21.15.100:0/1487845287 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556badb961e0 msgr2=0x556badba66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:07.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.017+0000 7f665b7fe700 1 --2- 172.21.15.100:0/1487845287 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556badb961e0 0x556badba66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:07.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.017+0000 7f665b7fe700 1 -- 172.21.15.100:0/1487845287 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556badbb7380 con 0x556badb92890 2024-05-10T11:34:07.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.017+0000 7f665bfff700 1 --2- 172.21.15.100:0/1487845287 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556badb961e0 0x556badba66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:34:07.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.018+0000 7f665b7fe700 1 --2- 172.21.15.100:0/1487845287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556badb92890 0x556badb87cc0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f664c00b0a0 tx=0x7f664c020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49b67da3310c0c53 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:34:07.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.018+0000 7f66417fa700 1 -- 172.21.15.100:0/1487845287 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664c034330 con 0x556badb92890 2024-05-10T11:34:07.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.018+0000 7f66417fa700 1 -- 172.21.15.100:0/1487845287 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f664c03a070 con 0x556badb92890 2024-05-10T11:34:07.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.018+0000 7f66417fa700 1 -- 172.21.15.100:0/1487845287 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664c01c860 con 0x556badb92890 2024-05-10T11:34:07.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.019+0000 7f666c324b80 1 -- 172.21.15.100:0/1487845287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556badb92890 msgr2=0x556badb87cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:07.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.019+0000 7f666c324b80 1 --2- 172.21.15.100:0/1487845287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556badb92890 0x556badb87cc0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f664c00b0a0 tx=0x7f664c020000 comp rx=0 tx=0).stop 2024-05-10T11:34:07.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.019+0000 7f666c324b80 1 -- 172.21.15.100:0/1487845287 shutdown_connections 2024-05-10T11:34:07.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.019+0000 7f666c324b80 1 --2- 172.21.15.100:0/1487845287 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556badb961e0 0x556badba66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:07.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.019+0000 7f666c324b80 1 --2- 172.21.15.100:0/1487845287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556badb92890 0x556badb87cc0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:07.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.019+0000 7f666c324b80 1 --2- 172.21.15.100:0/1487845287 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556badb93250 0x556badba61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:07.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.019+0000 7f666c324b80 1 -- 172.21.15.100:0/1487845287 >> 172.21.15.100:0/1487845287 conn(0x556badabebf0 msgr2=0x556badba5aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:34:07.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.019+0000 7f666c324b80 1 -- 172.21.15.100:0/1487845287 shutdown_connections 2024-05-10T11:34:07.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.019+0000 7f666c324b80 1 -- 172.21.15.100:0/1487845287 wait complete. 2024-05-10T11:34:07.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.019+0000 7f666c324b80 1 -- start start 2024-05-10T11:34:07.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.020+0000 7f666c324b80 1 Processor -- start 2024-05-10T11:34:07.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.020+0000 7f666c324b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556badb92890 0x556badbad4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:07.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.020+0000 7f666c324b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556badb93250 0x556badbaa0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:07.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.020+0000 7f666c324b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556badb961e0 0x556badbaa600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:07.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.020+0000 7f665b7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556badb92890 0x556badbad4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:07.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.020+0000 7f666c324b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f665c1ab5f0 con 0x556badb92890 2024-05-10T11:34:07.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.020+0000 7f665b7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556badb92890 0x556badbad4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60002/0 (socket says 172.21.15.100:60002) 2024-05-10T11:34:07.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.020+0000 7f665b7fe700 1 -- 172.21.15.100:0/4136836807 learned_addr learned my addr 172.21.15.100:0/4136836807 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:34:07.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.021+0000 7f666c324b80 1 -- 172.21.15.100:0/4136836807 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f665c1aca60 con 0x556badb961e0 2024-05-10T11:34:07.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.021+0000 7f666c324b80 1 -- 172.21.15.100:0/4136836807 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f665c1abcc0 con 0x556badb93250 2024-05-10T11:34:07.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.021+0000 7f665affd700 1 --2- 172.21.15.100:0/4136836807 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556badb93250 0x556badbaa0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:07.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.021+0000 7f666c324b80 10 client.0 _ll_register_callbacks cb 0x556bada78630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:34:07.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.021+0000 7f665bfff700 1 --2- 172.21.15.100:0/4136836807 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556badb961e0 0x556badbaa600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:07.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.021+0000 7f665b7fe700 1 -- 172.21.15.100:0/4136836807 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556badb93250 msgr2=0x556badbaa0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:07.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.021+0000 7f665b7fe700 1 --2- 172.21.15.100:0/4136836807 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556badb93250 0x556badbaa0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:07.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.021+0000 7f665b7fe700 1 -- 172.21.15.100:0/4136836807 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556badb961e0 msgr2=0x556badbaa600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:07.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.021+0000 7f665b7fe700 1 --2- 172.21.15.100:0/4136836807 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556badb961e0 0x556badbaa600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:07.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.021+0000 7f665b7fe700 1 -- 172.21.15.100:0/4136836807 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556badbaac80 con 0x556badb92890 2024-05-10T11:34:07.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.021+0000 7f665b7fe700 1 --2- 172.21.15.100:0/4136836807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556badb92890 0x556badbad4d0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f664c004b20 tx=0x7f664c0094c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb9288e9546e714d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:34:07.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.021+0000 7f663affd700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:34:07.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.021+0000 7f663affd700 1 -- 172.21.15.100:0/4136836807 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664c043060 con 0x556badb92890 2024-05-10T11:34:07.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.021+0000 7f663affd700 1 -- 172.21.15.100:0/4136836807 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f664c034330 con 0x556badb92890 2024-05-10T11:34:07.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.022+0000 7f666c324b80 1 -- 172.21.15.100:0/4136836807 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f664c03a070 con 0x556badb92890 2024-05-10T11:34:07.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.022+0000 7f663affd700 1 -- 172.21.15.100:0/4136836807 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f664c034570 con 0x556badb92890 2024-05-10T11:34:07.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.022+0000 7f666c324b80 10 client.18561 fetch_fsmap learned FSMap version 37 2024-05-10T11:34:07.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.022+0000 7f666c324b80 1 -- 172.21.15.100:0/4136836807 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x556badc20040 con 0x556badb92890 2024-05-10T11:34:07.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.022+0000 7f663affd700 1 -- 172.21.15.100:0/4136836807 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f664c03e450 con 0x556badb92890 2024-05-10T11:34:07.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.022+0000 7f663affd700 1 -- 172.21.15.100:0/4136836807 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f664c011070 con 0x556badb92890 2024-05-10T11:34:07.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.022+0000 7f666c324b80 10 client.18561 fetch_fsmap finished waiting for FSMap version 37 2024-05-10T11:34:07.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.022+0000 7f666c324b80 10 client.18561 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:34:07.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.022+0000 7f666c324b80 20 client.18561 populate_metadata read hostname 'smithi100' 2024-05-10T11:34:07.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.022+0000 7f666c324b80 1 --2- 172.21.15.100:0/4136836807 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556badc205a0 0x556badc229c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:07.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.022+0000 7f666c324b80 4 client.18561 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T11:34:07.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.022+0000 7f666c324b80 1 -- 172.21.15.100:0/4136836807 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f664c03a070 con 0x556badc205a0 2024-05-10T11:34:07.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.023+0000 7f665affd700 1 --2- 172.21.15.100:0/4136836807 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556badc205a0 0x556badc229c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:07.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.023+0000 7f663affd700 10 client.18561 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:34:07.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.023+0000 7f665affd700 1 --2- 172.21.15.100:0/4136836807 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556badc205a0 0x556badc229c0 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffc73e09645aa3bf server_cookie=b4046149c70fc33c in_seq=0 out_seq=0 2024-05-10T11:34:07.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.024+0000 7f663affd700 1 -- 172.21.15.100:0/4136836807 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f664c03a070 con 0x556badc205a0 2024-05-10T11:34:07.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.024+0000 7f663affd700 10 client.18561 handle_command_reply: tid=0 2024-05-10T11:34:07.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.024+0000 7f666c324b80 1 client.18561 shutdown 2024-05-10T11:34:07.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.024+0000 7f666c324b80 1 -- 172.21.15.100:0/4136836807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556badb92890 msgr2=0x556badbad4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:07.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.024+0000 7f666c324b80 1 --2- 172.21.15.100:0/4136836807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556badb92890 0x556badbad4d0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f664c004b20 tx=0x7f664c0094c0 comp rx=0 tx=0).stop 2024-05-10T11:34:07.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.024+0000 7f666c324b80 1 -- 172.21.15.100:0/4136836807 shutdown_connections 2024-05-10T11:34:07.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.024+0000 7f666c324b80 1 --2- 172.21.15.100:0/4136836807 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556badc205a0 0x556badc229c0 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:07.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.024+0000 7f663affd700 0 client.18561 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:34:07.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.024+0000 7f666c324b80 1 --2- 172.21.15.100:0/4136836807 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556badb961e0 0x556badbaa600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:07.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.025+0000 7f666c324b80 1 --2- 172.21.15.100:0/4136836807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556badb92890 0x556badbad4d0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:07.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.025+0000 7f666c324b80 1 --2- 172.21.15.100:0/4136836807 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556badb93250 0x556badbaa0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:07.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.025+0000 7f666c324b80 1 -- 172.21.15.100:0/4136836807 >> 172.21.15.100:0/4136836807 conn(0x556badabebf0 msgr2=0x556badba45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:34:07.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.025+0000 7f666c324b80 1 -- 172.21.15.100:0/4136836807 shutdown_connections 2024-05-10T11:34:07.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.025+0000 7f666c324b80 1 -- 172.21.15.100:0/4136836807 wait complete. 2024-05-10T11:34:07.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.025+0000 7f666c324b80 20 client.18561 trim_cache size 0 max 16384 2024-05-10T11:34:07.027 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:34:07.028 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-05-10T11:34:07.028 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "13915d86-3bd1-433f-acaf-75c8a06d28a0", 2024-05-10T11:34:07.028 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-05-10T11:34:07.028 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:34:07.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.025+0000 7f6661a91700 1 -- 172.21.15.100:0/2935355612 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f66440b2b00 msgr2=0x7f66440b4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:07.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.025+0000 7f6661a91700 1 --2- 172.21.15.100:0/2935355612 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f66440b2b00 0x7f66440b4fc0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f664c009e60 tx=0x7f664c00b740 comp rx=0 tx=0).stop 2024-05-10T11:34:07.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.026+0000 7f6661a91700 1 -- 172.21.15.100:0/2935355612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f665c06bb40 msgr2=0x7f665c19fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:07.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.026+0000 7f6661a91700 1 --2- 172.21.15.100:0/2935355612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f665c06bb40 0x7f665c19fc60 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f665000eb30 tx=0x7f665000ee40 comp rx=0 tx=0).stop 2024-05-10T11:34:07.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.026+0000 7f6661a91700 1 -- 172.21.15.100:0/2935355612 shutdown_connections 2024-05-10T11:34:07.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.026+0000 7f6661a91700 1 --2- 172.21.15.100:0/2935355612 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f66440b2b00 0x7f66440b4fc0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:07.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.026+0000 7f6661a91700 1 --2- 172.21.15.100:0/2935355612 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f665c109510 0x7f665c1998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:07.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.026+0000 7f6661a91700 1 --2- 172.21.15.100:0/2935355612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f665c06b180 0x7f665c19f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:07.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.026+0000 7f6661a91700 1 --2- 172.21.15.100:0/2935355612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f665c06bb40 0x7f665c19fc60 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:07.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.026+0000 7f6661a91700 1 -- 172.21.15.100:0/2935355612 >> 172.21.15.100:0/2935355612 conn(0x7f665c079350 msgr2=0x7f665c1012a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:34:07.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.026+0000 7f6661a91700 1 -- 172.21.15.100:0/2935355612 shutdown_connections 2024-05-10T11:34:07.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:07.026+0000 7f6661a91700 1 -- 172.21.15.100:0/2935355612 wait complete. 2024-05-10T11:34:07.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:34:07 smithi100 ceph-mon[30296]: pgmap v893: 129 pgs: 129 active+clean; 387 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 902 KiB/s rd, 4.2 MiB/s wr, 300 op/s 2024-05-10T11:34:07.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:34:07 smithi188 ceph-mon[34075]: pgmap v893: 129 pgs: 129 active+clean; 387 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 902 KiB/s rd, 4.2 MiB/s wr, 300 op/s 2024-05-10T11:34:07.529 INFO:journalctl@ceph.mds.i.smithi188.stdout:May 10 11:34:07 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-i[64554]: 2024-05-10T11:34:07.101+0000 7fd152433700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000004d2 [...2,head] /volumes/qa/sv_0/96765652-39ef-4cf3-b20a-a70f7e9d774d/client.0/tmp/clients/client8/~dmtmp/COREL/ auth{2=1} v3769 ap=2 f(v78 m2024-05-10T11:33:47.627616+0000 11=11+0) n(v4 rc2024-05-10T11:33:47.627616+0000 b2861056 12=11+1) (inest sync->mix g=2) caps={15207=pAsLsXsFs/-@2523} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x560801803700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000004d2:[<0x1000000041f/COREL v50>,<0x10000000410/~dmtmp v34>,<0x10000000007/client8 v1938>,<0x10000000006/clients v1743>,<0x10000000005/tmp v1642>,<0x10000000003/client.0 v1616>,<0x10000000002/96765652-39ef-4cf3-b20a-a70f7e9d774d v1609>,<0x10000000001/sv_0 v1620>,<0x10000000000/qa v1598>,<0x1/volumes v1566>]//[]","memoryvalue":"(2)0x100000004d2:[<0x1000000041f/COREL v3769>,<0x10000000410/~dmtmp v1627>,<0x10000000007/client8 v3258>,<0x10000000006/clients v2188>,<0x10000000005/tmp v1880>,<0x10000000003/client.0 v1847>,<0x10000000002/96765652-39ef-4cf3-b20a-a70f7e9d774d v1837>,<0x10000000001/sv_0 v1843>,<0x10000000000/qa v1817>,<0x1/volumes v1778>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-10T11:33:47.593677+0000 12=12+0)","ondisk_value.rstat":"n(v0 rc2024-05-10T11:33:47.627616+0000 b2861056 12=11+1)","memory_value.dirstat":"f(v78 m2024-05-10T11:33:47.627616+0000 11=11+0)","memory_value.rstat":"n(v4 rc2024-05-10T11:33:47.627616+0000 b2861056 12=11+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-05-10T11:34:07.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:34:07 smithi178 ceph-mon[33359]: pgmap v893: 129 pgs: 129 active+clean; 387 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 902 KiB/s rd, 4.2 MiB/s wr, 300 op/s 2024-05-10T11:34:07.879 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T11:34:07.914 INFO:tasks.workunit.client.0.smithi100.stdout: 10 20631 17.34 MB/sec execute 1 sec latency 112.205 ms 2024-05-10T11:34:08.255 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:34:08.296 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:34:08 smithi100 ceph-mon[30296]: scrub queued for path: / 2024-05-10T11:34:08.296 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:34:08 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:34:08.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:34:08 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:34:08.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:34:08 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:34:08.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:34:08 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:34:08.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:34:08 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:34:08.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:34:08 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:34:08.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:34:08 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:34:08.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:34:08 smithi100 ceph-mon[30296]: Scrub error on inode 0x100000004d2 (/volumes/qa/sv_0/96765652-39ef-4cf3-b20a-a70f7e9d774d/client.0/tmp/clients/client8/~dmtmp/COREL) see mds.i log and `damage ls` output for details 2024-05-10T11:34:08.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:34:08 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:34:08.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:34:08 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:34:08.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:34:08 smithi178 ceph-mon[33359]: scrub queued for path: / 2024-05-10T11:34:08.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:34:08 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:34:08.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:34:08 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:34:08.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:34:08 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:34:08.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:34:08 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:34:08.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:34:08 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:34:08.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:34:08 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:34:08.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:34:08 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:34:08.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:34:08 smithi178 ceph-mon[33359]: Scrub error on inode 0x100000004d2 (/volumes/qa/sv_0/96765652-39ef-4cf3-b20a-a70f7e9d774d/client.0/tmp/clients/client8/~dmtmp/COREL) see mds.i log and `damage ls` output for details 2024-05-10T11:34:08.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:34:08 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:34:08.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:34:08 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:34:08.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:34:08 smithi188 ceph-mon[34075]: scrub queued for path: / 2024-05-10T11:34:08.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:34:08 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:34:08.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:34:08 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:34:08.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:34:08 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:34:08.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:34:08 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:34:08.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:34:08 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:34:08.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:34:08 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:34:08.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:34:08 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:34:08.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:34:08 smithi188 ceph-mon[34075]: Scrub error on inode 0x100000004d2 (/volumes/qa/sv_0/96765652-39ef-4cf3-b20a-a70f7e9d774d/client.0/tmp/clients/client8/~dmtmp/COREL) see mds.i log and `damage ls` output for details 2024-05-10T11:34:08.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:34:08 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:34:08.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:34:08 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:34:08.914 INFO:tasks.workunit.client.0.smithi100.stdout: 10 20764 10.34 MB/sec execute 2 sec latency 72.171 ms 2024-05-10T11:34:09.567 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:34:09 smithi100 ceph-mon[30296]: pgmap v894: 129 pgs: 129 active+clean; 388 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 3.8 MiB/s wr, 294 op/s 2024-05-10T11:34:09.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:34:09 smithi178 ceph-mon[33359]: pgmap v894: 129 pgs: 129 active+clean; 388 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 3.8 MiB/s wr, 294 op/s 2024-05-10T11:34:09.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:34:09 smithi188 ceph-mon[34075]: pgmap v894: 129 pgs: 129 active+clean; 388 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 3.8 MiB/s wr, 294 op/s 2024-05-10T11:34:09.914 INFO:tasks.workunit.client.0.smithi100.stdout: 10 20932 8.97 MB/sec execute 3 sec latency 88.134 ms 2024-05-10T11:34:10.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.226+0000 7fe878930700 1 -- 172.21.15.100:0/889547837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8741037b0 msgr2=0x7fe874103b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:10.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.226+0000 7fe878930700 1 --2- 172.21.15.100:0/889547837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8741037b0 0x7fe874103b90 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fe8640099c0 tx=0x7fe864009cd0 comp rx=0 tx=0).stop 2024-05-10T11:34:10.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.226+0000 7fe878930700 1 -- 172.21.15.100:0/889547837 shutdown_connections 2024-05-10T11:34:10.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.226+0000 7fe878930700 1 --2- 172.21.15.100:0/889547837 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe874108880 0x7fe874110400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:10.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.226+0000 7fe878930700 1 --2- 172.21.15.100:0/889547837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8741037b0 0x7fe874103b90 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:10.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.226+0000 7fe878930700 1 --2- 172.21.15.100:0/889547837 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe874104160 0x7fe874108150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:10.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.226+0000 7fe878930700 1 -- 172.21.15.100:0/889547837 >> 172.21.15.100:0/889547837 conn(0x7fe87407a580 msgr2=0x7fe87407a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:34:10.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.226+0000 7fe878930700 1 -- 172.21.15.100:0/889547837 shutdown_connections 2024-05-10T11:34:10.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.227+0000 7fe878930700 1 -- 172.21.15.100:0/889547837 wait complete. 2024-05-10T11:34:10.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.228+0000 7fe878930700 1 Processor -- start 2024-05-10T11:34:10.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.228+0000 7fe878930700 1 -- start start 2024-05-10T11:34:10.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.228+0000 7fe878930700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8741037b0 0x7fe87419f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:10.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.228+0000 7fe878930700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe874104160 0x7fe87419fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:10.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.228+0000 7fe878930700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe874108880 0x7fe874199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:10.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.228+0000 7fe878930700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe8741135a0 con 0x7fe8741037b0 2024-05-10T11:34:10.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.228+0000 7fe878930700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe874113420 con 0x7fe874104160 2024-05-10T11:34:10.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.228+0000 7fe878930700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe874113720 con 0x7fe874108880 2024-05-10T11:34:10.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.229+0000 7fe871d9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe874104160 0x7fe87419fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:10.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.229+0000 7fe871d9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe874104160 0x7fe87419fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:48586/0 (socket says 172.21.15.100:48586) 2024-05-10T11:34:10.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.229+0000 7fe871d9b700 1 -- 172.21.15.100:0/2638558576 learned_addr learned my addr 172.21.15.100:0/2638558576 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:34:10.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.229+0000 7fe872d9d700 1 --2- 172.21.15.100:0/2638558576 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe874108880 0x7fe874199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:10.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.229+0000 7fe87259c700 1 --2- 172.21.15.100:0/2638558576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8741037b0 0x7fe87419f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:10.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.229+0000 7fe871d9b700 1 -- 172.21.15.100:0/2638558576 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe874108880 msgr2=0x7fe874199880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:10.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.229+0000 7fe871d9b700 1 --2- 172.21.15.100:0/2638558576 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe874108880 0x7fe874199880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:10.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.229+0000 7fe871d9b700 1 -- 172.21.15.100:0/2638558576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8741037b0 msgr2=0x7fe87419f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:10.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.229+0000 7fe871d9b700 1 --2- 172.21.15.100:0/2638558576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8741037b0 0x7fe87419f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:10.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.229+0000 7fe871d9b700 1 -- 172.21.15.100:0/2638558576 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe874199fe0 con 0x7fe874104160 2024-05-10T11:34:10.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.229+0000 7fe872d9d700 1 --2- 172.21.15.100:0/2638558576 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe874108880 0x7fe874199880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:34:10.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.229+0000 7fe871d9b700 1 --2- 172.21.15.100:0/2638558576 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe874104160 0x7fe87419fc00 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe86800e9a0 tx=0x7fe86800ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:34:10.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.229+0000 7fe8637fe700 1 -- 172.21.15.100:0/2638558576 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86800cbd0 con 0x7fe874104160 2024-05-10T11:34:10.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.229+0000 7fe8637fe700 1 -- 172.21.15.100:0/2638558576 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe86800cd70 con 0x7fe874104160 2024-05-10T11:34:10.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.230+0000 7fe8637fe700 1 -- 172.21.15.100:0/2638558576 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe868018680 con 0x7fe874104160 2024-05-10T11:34:10.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.230+0000 7fe878930700 1 -- 172.21.15.100:0/2638558576 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe87419a2d0 con 0x7fe874104160 2024-05-10T11:34:10.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.230+0000 7fe878930700 1 -- 172.21.15.100:0/2638558576 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8741a6590 con 0x7fe874104160 2024-05-10T11:34:10.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.234+0000 7fe8637fe700 1 -- 172.21.15.100:0/2638558576 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fe868010950 con 0x7fe874104160 2024-05-10T11:34:10.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.235+0000 7fe8637fe700 1 --2- 172.21.15.100:0/2638558576 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe85c0b2940 0x7fe85c0b4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:10.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.236+0000 7fe87259c700 1 --2- 172.21.15.100:0/2638558576 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe85c0b2940 0x7fe85c0b4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:10.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.236+0000 7fe8637fe700 1 -- 172.21.15.100:0/2638558576 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fe8680e2890 con 0x7fe874104160 2024-05-10T11:34:10.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.236+0000 7fe87259c700 1 --2- 172.21.15.100:0/2638558576 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe85c0b2940 0x7fe85c0b4e00 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe864000c00 tx=0x7fe864003700 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:34:10.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.236+0000 7fe8831c3b80 1 Processor -- start 2024-05-10T11:34:10.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.237+0000 7fe8831c3b80 1 -- start start 2024-05-10T11:34:10.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.237+0000 7fe8831c3b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5586957648e0 0x558695764cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:10.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.237+0000 7fe8831c3b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558695765200 0x558695768b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:10.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.237+0000 7fe8831c3b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558695769040 0x5586957694a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:10.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.237+0000 7fe8831c3b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5586956b2860 con 0x558695765200 2024-05-10T11:34:10.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.237+0000 7fe8831c3b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5586956517c0 con 0x558695769040 2024-05-10T11:34:10.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.237+0000 7fe8831c3b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5586956ae820 con 0x5586957648e0 2024-05-10T11:34:10.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.237+0000 7fe871d9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558695765200 0x558695768b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:10.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.237+0000 7fe872d9d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558695769040 0x5586957694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:10.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.237+0000 7fe871d9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558695765200 0x558695768b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60034/0 (socket says 172.21.15.100:60034) 2024-05-10T11:34:10.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.237+0000 7fe871d9b700 1 -- 172.21.15.100:0/3654362247 learned_addr learned my addr 172.21.15.100:0/3654362247 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:34:10.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.237+0000 7fe87259c700 1 --2- 172.21.15.100:0/3654362247 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5586957648e0 0x558695764cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:10.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.238+0000 7fe872d9d700 1 -- 172.21.15.100:0/3654362247 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5586957648e0 msgr2=0x558695764cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:10.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.238+0000 7fe872d9d700 1 --2- 172.21.15.100:0/3654362247 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5586957648e0 0x558695764cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:10.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.238+0000 7fe872d9d700 1 -- 172.21.15.100:0/3654362247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558695765200 msgr2=0x558695768b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:10.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.238+0000 7fe872d9d700 1 --2- 172.21.15.100:0/3654362247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558695765200 0x558695768b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:10.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.238+0000 7fe872d9d700 1 -- 172.21.15.100:0/3654362247 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586955ff8d0 con 0x558695769040 2024-05-10T11:34:10.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.238+0000 7fe872d9d700 1 --2- 172.21.15.100:0/3654362247 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558695769040 0x5586957694a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe86c009420 tx=0x7fe86c009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bed04055b946593 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:34:10.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.238+0000 7fe860ff9700 1 -- 172.21.15.100:0/3654362247 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86c014610 con 0x558695769040 2024-05-10T11:34:10.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.238+0000 7fe860ff9700 1 -- 172.21.15.100:0/3654362247 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe86c019070 con 0x558695769040 2024-05-10T11:34:10.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.238+0000 7fe860ff9700 1 -- 172.21.15.100:0/3654362247 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86c0139b0 con 0x558695769040 2024-05-10T11:34:10.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.239+0000 7fe8831c3b80 1 -- 172.21.15.100:0/3654362247 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558695769040 msgr2=0x5586957694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:10.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.239+0000 7fe8831c3b80 1 --2- 172.21.15.100:0/3654362247 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558695769040 0x5586957694a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe86c009420 tx=0x7fe86c009920 comp rx=0 tx=0).stop 2024-05-10T11:34:10.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.239+0000 7fe8831c3b80 1 -- 172.21.15.100:0/3654362247 shutdown_connections 2024-05-10T11:34:10.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.239+0000 7fe8831c3b80 1 --2- 172.21.15.100:0/3654362247 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558695769040 0x5586957694a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:10.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.239+0000 7fe8831c3b80 1 --2- 172.21.15.100:0/3654362247 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5586957648e0 0x558695764cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:10.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.239+0000 7fe8831c3b80 1 --2- 172.21.15.100:0/3654362247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558695765200 0x558695768b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:10.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.239+0000 7fe8831c3b80 1 -- 172.21.15.100:0/3654362247 >> 172.21.15.100:0/3654362247 conn(0x55869568c100 msgr2=0x55869575ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:34:10.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.239+0000 7fe8831c3b80 1 -- 172.21.15.100:0/3654362247 shutdown_connections 2024-05-10T11:34:10.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.240+0000 7fe8831c3b80 1 -- 172.21.15.100:0/3654362247 wait complete. 2024-05-10T11:34:10.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.240+0000 7fe8831c3b80 1 -- start start 2024-05-10T11:34:10.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.240+0000 7fe8831c3b80 1 Processor -- start 2024-05-10T11:34:10.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.240+0000 7fe8831c3b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5586957648e0 0x558695770450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:10.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.240+0000 7fe8831c3b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558695765200 0x55869576d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:10.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.240+0000 7fe8831c3b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558695769040 0x55869576dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:10.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.241+0000 7fe8831c3b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x558695771250 con 0x558695769040 2024-05-10T11:34:10.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.241+0000 7fe8831c3b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558695618c30 con 0x558695765200 2024-05-10T11:34:10.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.241+0000 7fe8831c3b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5586957713d0 con 0x5586957648e0 2024-05-10T11:34:10.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.241+0000 7fe872d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558695769040 0x55869576dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:10.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.241+0000 7fe872d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558695769040 0x55869576dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60038/0 (socket says 172.21.15.100:60038) 2024-05-10T11:34:10.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.241+0000 7fe872d9d700 1 -- 172.21.15.100:0/1659067156 learned_addr learned my addr 172.21.15.100:0/1659067156 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:34:10.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.241+0000 7fe8831c3b80 10 client.0 _ll_register_callbacks cb 0x55869564a920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:34:10.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.241+0000 7fe872d9d700 1 -- 172.21.15.100:0/1659067156 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5586957648e0 msgr2=0x558695770450 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T11:34:10.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.241+0000 7fe872d9d700 1 --2- 172.21.15.100:0/1659067156 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5586957648e0 0x558695770450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:10.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.241+0000 7fe872d9d700 1 -- 172.21.15.100:0/1659067156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558695765200 msgr2=0x55869576d7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T11:34:10.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.241+0000 7fe872d9d700 1 --2- 172.21.15.100:0/1659067156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558695765200 0x55869576d7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:10.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.241+0000 7fe872d9d700 1 -- 172.21.15.100:0/1659067156 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55869564e690 con 0x558695769040 2024-05-10T11:34:10.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.241+0000 7fe872d9d700 1 --2- 172.21.15.100:0/1659067156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558695769040 0x55869576dd30 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fe86c009fa0 tx=0x7fe86c0135f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cf50386bfa27ff9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:34:10.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.241+0000 7fe859ffb700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:34:10.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.241+0000 7fe859ffb700 1 -- 172.21.15.100:0/1659067156 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86c013ad0 con 0x558695769040 2024-05-10T11:34:10.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.242+0000 7fe859ffb700 1 -- 172.21.15.100:0/1659067156 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fe86c013c70 con 0x558695769040 2024-05-10T11:34:10.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.242+0000 7fe8831c3b80 1 -- 172.21.15.100:0/1659067156 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe86c019070 con 0x558695769040 2024-05-10T11:34:10.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.242+0000 7fe859ffb700 1 -- 172.21.15.100:0/1659067156 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fe86c013e70 con 0x558695769040 2024-05-10T11:34:10.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.242+0000 7fe8831c3b80 10 client.18573 fetch_fsmap learned FSMap version 37 2024-05-10T11:34:10.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.242+0000 7fe8831c3b80 1 -- 172.21.15.100:0/1659067156 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55869577ba60 con 0x558695769040 2024-05-10T11:34:10.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.242+0000 7fe859ffb700 1 -- 172.21.15.100:0/1659067156 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe86c0054d0 con 0x558695769040 2024-05-10T11:34:10.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.242+0000 7fe859ffb700 1 -- 172.21.15.100:0/1659067156 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7fe86c01d720 con 0x558695769040 2024-05-10T11:34:10.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.242+0000 7fe8831c3b80 10 client.18573 fetch_fsmap finished waiting for FSMap version 37 2024-05-10T11:34:10.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.242+0000 7fe8831c3b80 10 client.18573 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:34:10.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.242+0000 7fe8831c3b80 20 client.18573 populate_metadata read hostname 'smithi100' 2024-05-10T11:34:10.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.242+0000 7fe8831c3b80 1 --2- 172.21.15.100:0/1659067156 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55869577e040 0x558695780430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:10.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.242+0000 7fe8831c3b80 4 client.18573 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:34:10.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.242+0000 7fe8831c3b80 1 -- 172.21.15.100:0/1659067156 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe86c019070 con 0x55869577e040 2024-05-10T11:34:10.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.243+0000 7fe87259c700 1 --2- 172.21.15.100:0/1659067156 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55869577e040 0x558695780430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:10.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.243+0000 7fe859ffb700 10 client.18573 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:34:10.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.243+0000 7fe87259c700 1 --2- 172.21.15.100:0/1659067156 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55869577e040 0x558695780430 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=639c3e4dbf0d596 server_cookie=1fde7d7290827e96 in_seq=0 out_seq=0 2024-05-10T11:34:10.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.245+0000 7fe859ffb700 1 -- 172.21.15.100:0/1659067156 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fe86c019070 con 0x55869577e040 2024-05-10T11:34:10.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.245+0000 7fe859ffb700 10 client.18573 handle_command_reply: tid=0 2024-05-10T11:34:10.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.245+0000 7fe8831c3b80 1 client.18573 shutdown 2024-05-10T11:34:10.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.245+0000 7fe8831c3b80 1 -- 172.21.15.100:0/1659067156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558695769040 msgr2=0x55869576dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:10.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.245+0000 7fe8831c3b80 1 --2- 172.21.15.100:0/1659067156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558695769040 0x55869576dd30 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fe86c009fa0 tx=0x7fe86c0135f0 comp rx=0 tx=0).stop 2024-05-10T11:34:10.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.245+0000 7fe8831c3b80 1 -- 172.21.15.100:0/1659067156 shutdown_connections 2024-05-10T11:34:10.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.245+0000 7fe8831c3b80 1 --2- 172.21.15.100:0/1659067156 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55869577e040 0x558695780430 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:10.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.246+0000 7fe8831c3b80 1 --2- 172.21.15.100:0/1659067156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558695769040 0x55869576dd30 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:10.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.246+0000 7fe859ffb700 0 client.18573 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:34:10.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.246+0000 7fe8831c3b80 1 --2- 172.21.15.100:0/1659067156 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5586957648e0 0x558695770450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:10.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.246+0000 7fe8831c3b80 1 --2- 172.21.15.100:0/1659067156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558695765200 0x55869576d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:10.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.246+0000 7fe8831c3b80 1 -- 172.21.15.100:0/1659067156 >> 172.21.15.100:0/1659067156 conn(0x55869568c100 msgr2=0x558695755720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:34:10.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.246+0000 7fe8831c3b80 1 -- 172.21.15.100:0/1659067156 shutdown_connections 2024-05-10T11:34:10.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.246+0000 7fe8831c3b80 1 -- 172.21.15.100:0/1659067156 wait complete. 2024-05-10T11:34:10.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.246+0000 7fe8831c3b80 20 client.18573 trim_cache size 0 max 16384 2024-05-10T11:34:10.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.279+0000 7fe8831c3b80 1 Processor -- start 2024-05-10T11:34:10.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.279+0000 7fe8831c3b80 1 -- start start 2024-05-10T11:34:10.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.280+0000 7fe8831c3b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5586957648e0 0x55869577a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:10.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.280+0000 7fe8831c3b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558695765200 0x55869576b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:10.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.280+0000 7fe8831c3b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558695769040 0x558695785a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:10.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.280+0000 7fe8831c3b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe8741af900 con 0x5586957648e0 2024-05-10T11:34:10.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.280+0000 7fe8831c3b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe8741ad100 con 0x558695765200 2024-05-10T11:34:10.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.280+0000 7fe8831c3b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe8741aaf50 con 0x558695769040 2024-05-10T11:34:10.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.280+0000 7fe87259c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5586957648e0 0x55869577a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:10.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.280+0000 7fe87259c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5586957648e0 0x55869577a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60050/0 (socket says 172.21.15.100:60050) 2024-05-10T11:34:10.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.280+0000 7fe87259c700 1 -- 172.21.15.100:0/2989030585 learned_addr learned my addr 172.21.15.100:0/2989030585 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:34:10.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.280+0000 7fe871d9b700 1 --2- 172.21.15.100:0/2989030585 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558695765200 0x55869576b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:10.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.280+0000 7fe872d9d700 1 --2- 172.21.15.100:0/2989030585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558695769040 0x558695785a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:10.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.280+0000 7fe87259c700 1 -- 172.21.15.100:0/2989030585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558695769040 msgr2=0x558695785a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:10.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.280+0000 7fe87259c700 1 --2- 172.21.15.100:0/2989030585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558695769040 0x558695785a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:10.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.280+0000 7fe87259c700 1 -- 172.21.15.100:0/2989030585 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558695765200 msgr2=0x55869576b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:10.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.280+0000 7fe87259c700 1 --2- 172.21.15.100:0/2989030585 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558695765200 0x55869576b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:10.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.280+0000 7fe87259c700 1 -- 172.21.15.100:0/2989030585 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558695757810 con 0x5586957648e0 2024-05-10T11:34:10.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.280+0000 7fe87259c700 1 --2- 172.21.15.100:0/2989030585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5586957648e0 0x55869577a310 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fe8640098e0 tx=0x7fe86401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32ad051a2fa07ecb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:34:10.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.281+0000 7fe85bfff700 1 -- 172.21.15.100:0/2989030585 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe864005520 con 0x5586957648e0 2024-05-10T11:34:10.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.281+0000 7fe85bfff700 1 -- 172.21.15.100:0/2989030585 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe864038070 con 0x5586957648e0 2024-05-10T11:34:10.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.281+0000 7fe85bfff700 1 -- 172.21.15.100:0/2989030585 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86400bae0 con 0x5586957648e0 2024-05-10T11:34:10.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.282+0000 7fe8831c3b80 1 -- 172.21.15.100:0/2989030585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5586957648e0 msgr2=0x55869577a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:10.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.282+0000 7fe8831c3b80 1 --2- 172.21.15.100:0/2989030585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5586957648e0 0x55869577a310 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fe8640098e0 tx=0x7fe86401f000 comp rx=0 tx=0).stop 2024-05-10T11:34:10.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.282+0000 7fe8831c3b80 1 -- 172.21.15.100:0/2989030585 shutdown_connections 2024-05-10T11:34:10.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.282+0000 7fe8831c3b80 1 --2- 172.21.15.100:0/2989030585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558695769040 0x558695785a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:10.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.282+0000 7fe8831c3b80 1 --2- 172.21.15.100:0/2989030585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5586957648e0 0x55869577a310 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:10.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.282+0000 7fe8831c3b80 1 --2- 172.21.15.100:0/2989030585 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558695765200 0x55869576b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:10.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.282+0000 7fe8831c3b80 1 -- 172.21.15.100:0/2989030585 >> 172.21.15.100:0/2989030585 conn(0x55869568c100 msgr2=0x558695755720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:34:10.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.282+0000 7fe8831c3b80 1 -- 172.21.15.100:0/2989030585 shutdown_connections 2024-05-10T11:34:10.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.282+0000 7fe8831c3b80 1 -- 172.21.15.100:0/2989030585 wait complete. 2024-05-10T11:34:10.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.282+0000 7fe8831c3b80 1 -- start start 2024-05-10T11:34:10.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.282+0000 7fe8831c3b80 1 Processor -- start 2024-05-10T11:34:10.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.283+0000 7fe8831c3b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5586957648e0 0x5586957729a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:10.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.283+0000 7fe8831c3b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558695765200 0x558695772ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:10.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.283+0000 7fe8831c3b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558695769040 0x5586957e75b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:10.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.283+0000 7fe8831c3b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe8741ab5c0 con 0x5586957648e0 2024-05-10T11:34:10.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.283+0000 7fe8831c3b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe8741aca30 con 0x558695765200 2024-05-10T11:34:10.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.283+0000 7fe8831c3b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe8741abc90 con 0x558695769040 2024-05-10T11:34:10.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.283+0000 7fe8831c3b80 10 client.0 _ll_register_callbacks cb 0x55869564a920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:34:10.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.283+0000 7fe87259c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5586957648e0 0x5586957729a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:10.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.283+0000 7fe87259c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5586957648e0 0x5586957729a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60064/0 (socket says 172.21.15.100:60064) 2024-05-10T11:34:10.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.283+0000 7fe87259c700 1 -- 172.21.15.100:0/2019700641 learned_addr learned my addr 172.21.15.100:0/2019700641 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:34:10.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.283+0000 7fe871d9b700 1 --2- 172.21.15.100:0/2019700641 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558695765200 0x558695772ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:10.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.283+0000 7fe872d9d700 1 --2- 172.21.15.100:0/2019700641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558695769040 0x5586957e75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:10.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.283+0000 7fe87259c700 1 -- 172.21.15.100:0/2019700641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558695769040 msgr2=0x5586957e75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:10.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.283+0000 7fe87259c700 1 --2- 172.21.15.100:0/2019700641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558695769040 0x5586957e75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:10.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.283+0000 7fe87259c700 1 -- 172.21.15.100:0/2019700641 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558695765200 msgr2=0x558695772ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:10.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.283+0000 7fe87259c700 1 --2- 172.21.15.100:0/2019700641 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558695765200 0x558695772ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:10.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.283+0000 7fe87259c700 1 -- 172.21.15.100:0/2019700641 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586957e7af0 con 0x5586957648e0 2024-05-10T11:34:10.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.283+0000 7fe872d9d700 1 --2- 172.21.15.100:0/2019700641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558695769040 0x5586957e75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:34:10.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.284+0000 7fe87259c700 1 --2- 172.21.15.100:0/2019700641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5586957648e0 0x5586957729a0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fe864005090 tx=0x7fe864009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb3f8ae9c47704f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:34:10.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.284+0000 7fe859ffb700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:34:10.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.284+0000 7fe859ffb700 1 -- 172.21.15.100:0/2019700641 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe864042d20 con 0x5586957648e0 2024-05-10T11:34:10.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.284+0000 7fe859ffb700 1 -- 172.21.15.100:0/2019700641 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fe8640037c0 con 0x5586957648e0 2024-05-10T11:34:10.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.284+0000 7fe8831c3b80 1 -- 172.21.15.100:0/2019700641 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe864038070 con 0x5586957648e0 2024-05-10T11:34:10.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.284+0000 7fe859ffb700 1 -- 172.21.15.100:0/2019700641 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fe8640039e0 con 0x5586957648e0 2024-05-10T11:34:10.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.284+0000 7fe8831c3b80 10 client.18579 fetch_fsmap learned FSMap version 37 2024-05-10T11:34:10.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.284+0000 7fe8831c3b80 1 -- 172.21.15.100:0/2019700641 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55869575d2d0 con 0x5586957648e0 2024-05-10T11:34:10.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.284+0000 7fe859ffb700 1 -- 172.21.15.100:0/2019700641 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe864038070 con 0x5586957648e0 2024-05-10T11:34:10.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.285+0000 7fe859ffb700 1 -- 172.21.15.100:0/2019700641 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7fe864031560 con 0x5586957648e0 2024-05-10T11:34:10.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.285+0000 7fe8831c3b80 10 client.18579 fetch_fsmap finished waiting for FSMap version 37 2024-05-10T11:34:10.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.285+0000 7fe8831c3b80 10 client.18579 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:34:10.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.285+0000 7fe8831c3b80 20 client.18579 populate_metadata read hostname 'smithi100' 2024-05-10T11:34:10.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.285+0000 7fe8831c3b80 1 --2- 172.21.15.100:0/2019700641 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55869575da90 0x55869575feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:10.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.285+0000 7fe8831c3b80 4 client.18579 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T11:34:10.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.285+0000 7fe8831c3b80 1 -- 172.21.15.100:0/2019700641 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55869572c650 con 0x55869575da90 2024-05-10T11:34:10.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.285+0000 7fe871d9b700 1 --2- 172.21.15.100:0/2019700641 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55869575da90 0x55869575feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:10.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.285+0000 7fe871d9b700 1 --2- 172.21.15.100:0/2019700641 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55869575da90 0x55869575feb0 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f53488e8c343610 server_cookie=15870a1dd3369e68 in_seq=0 out_seq=0 2024-05-10T11:34:10.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.285+0000 7fe859ffb700 10 client.18579 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:34:10.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.285+0000 7fe859ffb700 1 -- 172.21.15.100:0/2019700641 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55869572c650 con 0x55869575da90 2024-05-10T11:34:10.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.285+0000 7fe859ffb700 10 client.18579 handle_command_reply: tid=0 2024-05-10T11:34:10.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.285+0000 7fe8831c3b80 1 client.18579 shutdown 2024-05-10T11:34:10.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.286+0000 7fe8831c3b80 1 -- 172.21.15.100:0/2019700641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5586957648e0 msgr2=0x5586957729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:10.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.286+0000 7fe8831c3b80 1 --2- 172.21.15.100:0/2019700641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5586957648e0 0x5586957729a0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fe864005090 tx=0x7fe864009510 comp rx=0 tx=0).stop 2024-05-10T11:34:10.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.286+0000 7fe8831c3b80 1 -- 172.21.15.100:0/2019700641 shutdown_connections 2024-05-10T11:34:10.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.286+0000 7fe8831c3b80 1 --2- 172.21.15.100:0/2019700641 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55869575da90 0x55869575feb0 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:10.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.286+0000 7fe8831c3b80 1 --2- 172.21.15.100:0/2019700641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558695769040 0x5586957e75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:10.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.286+0000 7fe8831c3b80 1 --2- 172.21.15.100:0/2019700641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5586957648e0 0x5586957729a0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:10.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.286+0000 7fe8831c3b80 1 --2- 172.21.15.100:0/2019700641 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558695765200 0x558695772ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:10.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.286+0000 7fe8831c3b80 1 -- 172.21.15.100:0/2019700641 >> 172.21.15.100:0/2019700641 conn(0x55869568c100 msgr2=0x5586957636f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:34:10.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.286+0000 7fe859ffb700 0 client.18579 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:34:10.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.286+0000 7fe8831c3b80 1 -- 172.21.15.100:0/2019700641 shutdown_connections 2024-05-10T11:34:10.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.286+0000 7fe8831c3b80 1 -- 172.21.15.100:0/2019700641 wait complete. 2024-05-10T11:34:10.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.286+0000 7fe8831c3b80 20 client.18579 trim_cache size 0 max 16384 2024-05-10T11:34:10.292 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:34:10.292 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T11:34:10.292 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-05-10T11:34:10.292 INFO:teuthology.orchestra.run.smithi100.stdout: "13915d86-3bd1-433f-acaf-75c8a06d28a0": { 2024-05-10T11:34:10.292 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-05-10T11:34:10.293 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "13915d86-3bd1-433f-acaf-75c8a06d28a0", 2024-05-10T11:34:10.293 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-05-10T11:34:10.293 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:34:10.293 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:34:10.293 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:34:10.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.287+0000 7fe878930700 1 -- 172.21.15.100:0/2638558576 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe85c0b2940 msgr2=0x7fe85c0b4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:10.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.287+0000 7fe878930700 1 --2- 172.21.15.100:0/2638558576 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe85c0b2940 0x7fe85c0b4e00 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe864000c00 tx=0x7fe864003700 comp rx=0 tx=0).stop 2024-05-10T11:34:10.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.287+0000 7fe878930700 1 -- 172.21.15.100:0/2638558576 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe874104160 msgr2=0x7fe87419fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:10.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.287+0000 7fe878930700 1 --2- 172.21.15.100:0/2638558576 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe874104160 0x7fe87419fc00 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe86800e9a0 tx=0x7fe86800ed60 comp rx=0 tx=0).stop 2024-05-10T11:34:10.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.287+0000 7fe878930700 1 -- 172.21.15.100:0/2638558576 shutdown_connections 2024-05-10T11:34:10.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.287+0000 7fe878930700 1 --2- 172.21.15.100:0/2638558576 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe85c0b2940 0x7fe85c0b4e00 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:10.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.287+0000 7fe878930700 1 --2- 172.21.15.100:0/2638558576 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe874108880 0x7fe874199880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:10.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.287+0000 7fe878930700 1 --2- 172.21.15.100:0/2638558576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8741037b0 0x7fe87419f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:10.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.287+0000 7fe878930700 1 --2- 172.21.15.100:0/2638558576 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe874104160 0x7fe87419fc00 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:10.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.287+0000 7fe878930700 1 -- 172.21.15.100:0/2638558576 >> 172.21.15.100:0/2638558576 conn(0x7fe87407a580 msgr2=0x7fe874101d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:34:10.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.287+0000 7fe878930700 1 -- 172.21.15.100:0/2638558576 shutdown_connections 2024-05-10T11:34:10.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:10.287+0000 7fe878930700 1 -- 172.21.15.100:0/2638558576 wait complete. 2024-05-10T11:34:10.914 INFO:tasks.workunit.client.0.smithi100.stdout: 10 21134 11.09 MB/sec execute 4 sec latency 77.398 ms 2024-05-10T11:34:10.918 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.918+0000 7f6045072640 1 -- 172.21.15.100:0/916887613 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6040103490 msgr2=0x7f6040105880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:10.918 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.918+0000 7f6045072640 1 --2- 172.21.15.100:0/916887613 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6040103490 0x7f6040105880 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6030009860 tx=0x7f603002a340 comp rx=0 tx=0).stop 2024-05-10T11:34:10.918 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.918+0000 7f6045072640 1 -- 172.21.15.100:0/916887613 shutdown_connections 2024-05-10T11:34:10.918 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.918+0000 7f6045072640 1 --2- 172.21.15.100:0/916887613 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60401086f0 0x7f604010ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:10.918 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.918+0000 7f6045072640 1 --2- 172.21.15.100:0/916887613 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6040105dc0 0x7f60401081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:10.918 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.918+0000 7f6045072640 1 --2- 172.21.15.100:0/916887613 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6040103490 0x7f6040105880 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:10.918 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.918+0000 7f6045072640 1 -- 172.21.15.100:0/916887613 >> 172.21.15.100:0/916887613 conn(0x7f60400fcea0 msgr2=0x7f60400ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:34:10.919 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.918+0000 7f6045072640 1 -- 172.21.15.100:0/916887613 shutdown_connections 2024-05-10T11:34:10.919 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.918+0000 7f6045072640 1 -- 172.21.15.100:0/916887613 wait complete. 2024-05-10T11:34:10.920 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.919+0000 7f6045072640 1 Processor -- start 2024-05-10T11:34:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.919+0000 7f6045072640 1 -- start start 2024-05-10T11:34:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.919+0000 7f6045072640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6040103490 0x7f60401968b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.919+0000 7f6045072640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6040105dc0 0x7f6040196df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.919+0000 7f6045072640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60401086f0 0x7f604019de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.919+0000 7f6045072640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f604010c1f0 con 0x7f6040105dc0 2024-05-10T11:34:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.919+0000 7f6045072640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f604010c070 con 0x7f6040103490 2024-05-10T11:34:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.919+0000 7f6045072640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f604010c370 con 0x7f60401086f0 2024-05-10T11:34:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.920+0000 7f603e59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6040105dc0 0x7f6040196df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.920+0000 7f603ed9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6040103490 0x7f60401968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.920+0000 7f603f59e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60401086f0 0x7f604019de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.920+0000 7f603ed9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6040103490 0x7f60401968b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:48638/0 (socket says 172.21.15.100:48638) 2024-05-10T11:34:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.920+0000 7f603ed9d640 1 -- 172.21.15.100:0/2000621423 learned_addr learned my addr 172.21.15.100:0/2000621423 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:34:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.920+0000 7f603ed9d640 1 -- 172.21.15.100:0/2000621423 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60401086f0 msgr2=0x7f604019de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.920+0000 7f603ed9d640 1 --2- 172.21.15.100:0/2000621423 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60401086f0 0x7f604019de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.920+0000 7f603ed9d640 1 -- 172.21.15.100:0/2000621423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6040105dc0 msgr2=0x7f6040196df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.920+0000 7f603ed9d640 1 --2- 172.21.15.100:0/2000621423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6040105dc0 0x7f6040196df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.920+0000 7f603ed9d640 1 -- 172.21.15.100:0/2000621423 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f604019e4d0 con 0x7f6040103490 2024-05-10T11:34:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.920+0000 7f603f59e640 1 --2- 172.21.15.100:0/2000621423 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60401086f0 0x7f604019de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:34:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.920+0000 7f603ed9d640 1 --2- 172.21.15.100:0/2000621423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6040103490 0x7f60401968b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6030005cd0 tx=0x7f603002ae50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:34:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.920+0000 7f6027fff640 1 -- 172.21.15.100:0/2000621423 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6030042070 con 0x7f6040103490 2024-05-10T11:34:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.920+0000 7f6027fff640 1 -- 172.21.15.100:0/2000621423 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6030004050 con 0x7f6040103490 2024-05-10T11:34:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.920+0000 7f6045072640 1 -- 172.21.15.100:0/2000621423 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f604019e760 con 0x7f6040103490 2024-05-10T11:34:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.920+0000 7f6027fff640 1 -- 172.21.15.100:0/2000621423 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6030037040 con 0x7f6040103490 2024-05-10T11:34:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.921+0000 7f6045072640 1 -- 172.21.15.100:0/2000621423 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f604019eca0 con 0x7f6040103490 2024-05-10T11:34:10.923 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.923+0000 7f6027fff640 1 -- 172.21.15.100:0/2000621423 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f6030045070 con 0x7f6040103490 2024-05-10T11:34:10.925 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.924+0000 7f6047d93740 1 Processor -- start 2024-05-10T11:34:10.925 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.924+0000 7f6047d93740 1 -- start start 2024-05-10T11:34:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.924+0000 7f6047d93740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56547dc7edb0 0x56547dc7f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.924+0000 7f6047d93740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56547dc7f7a0 0x56547dc842a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.924+0000 7f6047d93740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56547dc84910 0x56547dc86da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.924+0000 7f6047d93740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56547dbe35d0 con 0x56547dc7f7a0 2024-05-10T11:34:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.924+0000 7f6047d93740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56547db9c890 con 0x56547dc84910 2024-05-10T11:34:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.924+0000 7f6047d93740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56547dbdf590 con 0x56547dc7edb0 2024-05-10T11:34:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.924+0000 7f6027fff640 1 --2- 172.21.15.100:0/2000621423 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f60280b2920 0x7f60280b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.924+0000 7f603f59e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56547dc7edb0 0x56547dc7f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.924+0000 7f603f59e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56547dc7edb0 0x56547dc7f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:40806/0 (socket says 172.21.15.100:40806) 2024-05-10T11:34:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.924+0000 7f603f59e640 1 -- 172.21.15.100:0/285393630 learned_addr learned my addr 172.21.15.100:0/285393630 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:34:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.924+0000 7f603ed9d640 1 --2- 172.21.15.100:0/285393630 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56547dc7f7a0 0x56547dc842a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.924+0000 7f603e59c640 1 --2- 172.21.15.100:0/285393630 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56547dc84910 0x56547dc86da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.924+0000 7f603f59e640 1 --2- 172.21.15.100:0/2000621423 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f60280b2920 0x7f60280b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.925+0000 7f6027fff640 1 -- 172.21.15.100:0/2000621423 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f6030101d90 con 0x7f6040103490 2024-05-10T11:34:10.927 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.925+0000 7f603f59e640 1 -- 172.21.15.100:0/285393630 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56547dc84910 msgr2=0x56547dc86da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:10.927 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.925+0000 7f603f59e640 1 --2- 172.21.15.100:0/285393630 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56547dc84910 0x56547dc86da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:10.927 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.925+0000 7f603f59e640 1 -- 172.21.15.100:0/285393630 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56547dc7f7a0 msgr2=0x56547dc842a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:10.927 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.925+0000 7f603f59e640 1 --2- 172.21.15.100:0/285393630 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56547dc7f7a0 0x56547dc842a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:10.927 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.925+0000 7f603f59e640 1 -- 172.21.15.100:0/285393630 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56547d8f5210 con 0x56547dc7edb0 2024-05-10T11:34:10.927 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.925+0000 7f603f59e640 1 --2- 172.21.15.100:0/285393630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56547dc7edb0 0x56547dc7f1d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f603801c950 tx=0x7f603801ce10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d804083a81798aa server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:34:10.927 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.925+0000 7f603f59e640 1 --2- 172.21.15.100:0/2000621423 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f60280b2920 0x7f60280b4de0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f603801e730 tx=0x7f6038022000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:34:10.927 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.925+0000 7f60257fa640 1 -- 172.21.15.100:0/285393630 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f603802d070 con 0x56547dc7edb0 2024-05-10T11:34:10.927 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.925+0000 7f60257fa640 1 -- 172.21.15.100:0/285393630 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6038003e00 con 0x56547dc7edb0 2024-05-10T11:34:10.927 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.925+0000 7f60257fa640 1 -- 172.21.15.100:0/285393630 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6038004030 con 0x56547dc7edb0 2024-05-10T11:34:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.926+0000 7f6047d93740 1 -- 172.21.15.100:0/285393630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56547dc7edb0 msgr2=0x56547dc7f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.926+0000 7f6047d93740 1 --2- 172.21.15.100:0/285393630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56547dc7edb0 0x56547dc7f1d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f603801c950 tx=0x7f603801ce10 comp rx=0 tx=0).stop 2024-05-10T11:34:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.926+0000 7f6047d93740 1 -- 172.21.15.100:0/285393630 shutdown_connections 2024-05-10T11:34:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.926+0000 7f6047d93740 1 --2- 172.21.15.100:0/285393630 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56547dc84910 0x56547dc86da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.926+0000 7f6047d93740 1 --2- 172.21.15.100:0/285393630 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56547dc7f7a0 0x56547dc842a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.926+0000 7f6047d93740 1 --2- 172.21.15.100:0/285393630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56547dc7edb0 0x56547dc7f1d0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.926+0000 7f6047d93740 1 -- 172.21.15.100:0/285393630 >> 172.21.15.100:0/285393630 conn(0x56547dc7a420 msgr2=0x56547dc7a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:34:10.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.926+0000 7f6047d93740 1 -- 172.21.15.100:0/285393630 shutdown_connections 2024-05-10T11:34:10.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.926+0000 7f6047d93740 1 -- 172.21.15.100:0/285393630 wait complete. 2024-05-10T11:34:10.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.927+0000 7f6047d93740 1 -- start start 2024-05-10T11:34:10.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.927+0000 7f6047d93740 1 Processor -- start 2024-05-10T11:34:10.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.927+0000 7f6047d93740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56547dc7edb0 0x56547dc88f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:10.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.927+0000 7f6047d93740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56547dc7f7a0 0x56547dc894d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:10.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.927+0000 7f6047d93740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56547dc84910 0x56547dc90510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:10.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.927+0000 7f6047d93740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56547dc87860 con 0x56547dc84910 2024-05-10T11:34:10.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.927+0000 7f6047d93740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56547dba0370 con 0x56547dc7edb0 2024-05-10T11:34:10.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.927+0000 7f6047d93740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56547dc879e0 con 0x56547dc7f7a0 2024-05-10T11:34:10.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.927+0000 7f6047d93740 10 client.0 _ll_register_callbacks cb 0x56547d8f4a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:34:10.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.927+0000 7f603ed9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56547dc7edb0 0x56547dc88f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:10.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.927+0000 7f603e59c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56547dc7f7a0 0x56547dc894d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:10.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.927+0000 7f603ed9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56547dc7edb0 0x56547dc88f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:48656/0 (socket says 172.21.15.100:48656) 2024-05-10T11:34:10.932 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.927+0000 7f603ed9d640 1 -- 172.21.15.100:0/2609038374 learned_addr learned my addr 172.21.15.100:0/2609038374 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:34:10.932 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.927+0000 7f603f59e640 1 --2- 172.21.15.100:0/2609038374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56547dc84910 0x56547dc90510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:10.932 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.927+0000 7f603ed9d640 1 -- 172.21.15.100:0/2609038374 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56547dc7f7a0 msgr2=0x56547dc894d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:10.932 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.927+0000 7f603ed9d640 1 --2- 172.21.15.100:0/2609038374 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56547dc7f7a0 0x56547dc894d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:10.932 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.928+0000 7f603ed9d640 1 -- 172.21.15.100:0/2609038374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56547dc84910 msgr2=0x56547dc90510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:10.932 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.928+0000 7f603ed9d640 1 --2- 172.21.15.100:0/2609038374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56547dc84910 0x56547dc90510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:10.932 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.928+0000 7f603ed9d640 1 -- 172.21.15.100:0/2609038374 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56547d9c1620 con 0x56547dc7edb0 2024-05-10T11:34:10.932 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.928+0000 7f603f59e640 1 --2- 172.21.15.100:0/2609038374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56547dc84910 0x56547dc90510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:34:10.932 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.928+0000 7f603ed9d640 1 --2- 172.21.15.100:0/2609038374 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56547dc7edb0 0x56547dc88f90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6030009230 tx=0x7f6030004610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dabd8c8cb9ba389f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:34:10.932 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.928+0000 7f6022ff5640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:34:10.932 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.928+0000 7f6022ff5640 1 -- 172.21.15.100:0/2609038374 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60300c8810 con 0x56547dc7edb0 2024-05-10T11:34:10.932 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.928+0000 7f6022ff5640 1 -- 172.21.15.100:0/2609038374 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f60300054d0 con 0x56547dc7edb0 2024-05-10T11:34:10.932 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.928+0000 7f6047d93740 1 -- 172.21.15.100:0/2609038374 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6038003e00 con 0x56547dc7edb0 2024-05-10T11:34:10.932 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.928+0000 7f6022ff5640 1 -- 172.21.15.100:0/2609038374 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f60300c8bb0 con 0x56547dc7edb0 2024-05-10T11:34:10.932 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.928+0000 7f6047d93740 10 client.27908 fetch_fsmap learned FSMap version 37 2024-05-10T11:34:10.932 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.928+0000 7f6047d93740 1 -- 172.21.15.100:0/2609038374 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56547dc934f0 con 0x56547dc7edb0 2024-05-10T11:34:10.933 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.928+0000 7f6022ff5640 1 -- 172.21.15.100:0/2609038374 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6038003e00 con 0x56547dc7edb0 2024-05-10T11:34:10.933 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.929+0000 7f6022ff5640 1 -- 172.21.15.100:0/2609038374 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f60300d1a50 con 0x56547dc7edb0 2024-05-10T11:34:10.933 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.929+0000 7f6047d93740 10 client.27908 fetch_fsmap finished waiting for FSMap version 37 2024-05-10T11:34:10.933 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.929+0000 7f6047d93740 10 client.27908 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:34:10.933 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.929+0000 7f6047d93740 20 client.27908 populate_metadata read hostname 'smithi100' 2024-05-10T11:34:10.933 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.929+0000 7f6047d93740 1 --2- 172.21.15.100:0/2609038374 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56547dc94cd0 0x56547dc97150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:10.933 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.929+0000 7f6047d93740 4 client.27908 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:34:10.933 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.929+0000 7f6047d93740 1 -- 172.21.15.100:0/2609038374 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56547dbf11e0 con 0x56547dc94cd0 2024-05-10T11:34:10.933 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.929+0000 7f603e59c640 1 --2- 172.21.15.100:0/2609038374 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56547dc94cd0 0x56547dc97150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:10.933 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.929+0000 7f6022ff5640 10 client.27908 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:34:10.933 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.929+0000 7f603e59c640 1 --2- 172.21.15.100:0/2609038374 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56547dc94cd0 0x56547dc97150 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb5b7d61f99e31b3 server_cookie=fc29387a7bd4d7b3 in_seq=0 out_seq=0 2024-05-10T11:34:10.934 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.931+0000 7f6022ff5640 1 -- 172.21.15.100:0/2609038374 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56547dbf11e0 con 0x56547dc94cd0 2024-05-10T11:34:10.934 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.931+0000 7f6022ff5640 10 client.27908 handle_command_reply: tid=0 2024-05-10T11:34:10.934 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.931+0000 7f6047d93740 1 client.27908 shutdown 2024-05-10T11:34:10.935 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.931+0000 7f6047d93740 1 -- 172.21.15.100:0/2609038374 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56547dc7edb0 msgr2=0x56547dc88f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:10.935 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.931+0000 7f6047d93740 1 --2- 172.21.15.100:0/2609038374 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56547dc7edb0 0x56547dc88f90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6030009230 tx=0x7f6030004610 comp rx=0 tx=0).stop 2024-05-10T11:34:10.935 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.931+0000 7f6047d93740 1 -- 172.21.15.100:0/2609038374 shutdown_connections 2024-05-10T11:34:10.935 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.931+0000 7f6047d93740 1 --2- 172.21.15.100:0/2609038374 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56547dc94cd0 0x56547dc97150 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:10.935 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.931+0000 7f6047d93740 1 --2- 172.21.15.100:0/2609038374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56547dc84910 0x56547dc90510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:10.935 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.931+0000 7f6047d93740 1 --2- 172.21.15.100:0/2609038374 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56547dc7f7a0 0x56547dc894d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:10.935 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.931+0000 7f6047d93740 1 --2- 172.21.15.100:0/2609038374 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56547dc7edb0 0x56547dc88f90 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:10.935 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.931+0000 7f6047d93740 1 -- 172.21.15.100:0/2609038374 >> 172.21.15.100:0/2609038374 conn(0x56547dc7a420 msgr2=0x56547dc79580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:34:10.935 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.931+0000 7f6022ff5640 0 client.27908 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:34:10.935 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.932+0000 7f6047d93740 1 -- 172.21.15.100:0/2609038374 shutdown_connections 2024-05-10T11:34:10.935 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.932+0000 7f6047d93740 1 -- 172.21.15.100:0/2609038374 wait complete. 2024-05-10T11:34:10.935 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.932+0000 7f6047d93740 20 client.27908 trim_cache size 0 max 16384 2024-05-10T11:34:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.961+0000 7f6047d93740 1 Processor -- start 2024-05-10T11:34:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.961+0000 7f6047d93740 1 -- start start 2024-05-10T11:34:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.961+0000 7f6047d93740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56547dc7edb0 0x56547dc84010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.961+0000 7f6047d93740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56547dc7f7a0 0x56547dc753e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.961+0000 7f6047d93740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56547dc84910 0x56547dc7d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.961+0000 7f6047d93740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f60401ab970 con 0x56547dc84910 2024-05-10T11:34:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.961+0000 7f6047d93740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f60401a91b0 con 0x56547dc7f7a0 2024-05-10T11:34:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.961+0000 7f6047d93740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f60401a7000 con 0x56547dc7edb0 2024-05-10T11:34:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.962+0000 7f603ed9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56547dc7edb0 0x56547dc84010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.962+0000 7f603f59e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56547dc84910 0x56547dc7d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.962+0000 7f603ed9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56547dc7edb0 0x56547dc84010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:40818/0 (socket says 172.21.15.100:40818) 2024-05-10T11:34:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.962+0000 7f603ed9d640 1 -- 172.21.15.100:0/1490922654 learned_addr learned my addr 172.21.15.100:0/1490922654 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:34:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.962+0000 7f603e59c640 1 --2- 172.21.15.100:0/1490922654 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56547dc7f7a0 0x56547dc753e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.962+0000 7f603f59e640 1 -- 172.21.15.100:0/1490922654 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56547dc7edb0 msgr2=0x56547dc84010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.962+0000 7f603f59e640 1 --2- 172.21.15.100:0/1490922654 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56547dc7edb0 0x56547dc84010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.962+0000 7f603f59e640 1 -- 172.21.15.100:0/1490922654 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56547dc7f7a0 msgr2=0x56547dc753e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:10.964 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.962+0000 7f603f59e640 1 --2- 172.21.15.100:0/1490922654 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56547dc7f7a0 0x56547dc753e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:10.964 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.962+0000 7f603f59e640 1 -- 172.21.15.100:0/1490922654 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56547dc7a250 con 0x56547dc84910 2024-05-10T11:34:10.964 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.962+0000 7f603e59c640 1 --2- 172.21.15.100:0/1490922654 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56547dc7f7a0 0x56547dc753e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:34:10.964 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.962+0000 7f603f59e640 1 --2- 172.21.15.100:0/1490922654 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56547dc84910 0x56547dc7d330 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f6038009600 tx=0x7f6038009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71f4c6c9327e5d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:34:10.964 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.962+0000 7f6024ff9640 1 -- 172.21.15.100:0/1490922654 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f603802d040 con 0x56547dc84910 2024-05-10T11:34:10.964 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.962+0000 7f6024ff9640 1 -- 172.21.15.100:0/1490922654 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6038038070 con 0x56547dc84910 2024-05-10T11:34:10.964 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.962+0000 7f6024ff9640 1 -- 172.21.15.100:0/1490922654 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6038033480 con 0x56547dc84910 2024-05-10T11:34:10.967 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.963+0000 7f6047d93740 1 -- 172.21.15.100:0/1490922654 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56547dc84910 msgr2=0x56547dc7d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:10.968 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.963+0000 7f6047d93740 1 --2- 172.21.15.100:0/1490922654 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56547dc84910 0x56547dc7d330 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f6038009600 tx=0x7f6038009900 comp rx=0 tx=0).stop 2024-05-10T11:34:10.968 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.963+0000 7f6047d93740 1 -- 172.21.15.100:0/1490922654 shutdown_connections 2024-05-10T11:34:10.968 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.963+0000 7f6047d93740 1 --2- 172.21.15.100:0/1490922654 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56547dc84910 0x56547dc7d330 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:10.968 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.963+0000 7f6047d93740 1 --2- 172.21.15.100:0/1490922654 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56547dc7f7a0 0x56547dc753e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:10.968 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.963+0000 7f6047d93740 1 --2- 172.21.15.100:0/1490922654 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56547dc7edb0 0x56547dc84010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:10.968 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.963+0000 7f6047d93740 1 -- 172.21.15.100:0/1490922654 >> 172.21.15.100:0/1490922654 conn(0x56547dc7a420 msgr2=0x56547dc87d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:34:10.968 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.963+0000 7f6047d93740 1 -- 172.21.15.100:0/1490922654 shutdown_connections 2024-05-10T11:34:10.968 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.963+0000 7f6047d93740 1 -- 172.21.15.100:0/1490922654 wait complete. 2024-05-10T11:34:10.968 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.964+0000 7f6047d93740 1 -- start start 2024-05-10T11:34:10.968 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.964+0000 7f6047d93740 1 Processor -- start 2024-05-10T11:34:10.968 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.964+0000 7f6047d93740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56547dc7edb0 0x56547dc8ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:10.968 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.964+0000 7f6047d93740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56547dc7f7a0 0x56547dc8cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:10.968 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.964+0000 7f6047d93740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56547dc84910 0x56547dcfd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:10.968 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.964+0000 7f6047d93740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f60401a7670 con 0x56547dc7f7a0 2024-05-10T11:34:10.969 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.964+0000 7f6047d93740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f60401a8ae0 con 0x56547dc7edb0 2024-05-10T11:34:10.969 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.964+0000 7f6047d93740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f60401a7d40 con 0x56547dc84910 2024-05-10T11:34:10.969 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.964+0000 7f6047d93740 10 client.0 _ll_register_callbacks cb 0x7f60300097b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:34:10.969 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.964+0000 7f603e59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56547dc7f7a0 0x56547dc8cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:10.969 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.964+0000 7f603ed9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56547dc7edb0 0x56547dc8ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:10.969 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.964+0000 7f603e59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56547dc7f7a0 0x56547dc8cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60128/0 (socket says 172.21.15.100:60128) 2024-05-10T11:34:10.969 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.964+0000 7f603ed9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56547dc7edb0 0x56547dc8ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:48664/0 (socket says 172.21.15.100:48664) 2024-05-10T11:34:10.969 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.964+0000 7f603e59c640 1 -- 172.21.15.100:0/1654254867 learned_addr learned my addr 172.21.15.100:0/1654254867 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:34:10.969 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.964+0000 7f603f59e640 1 --2- 172.21.15.100:0/1654254867 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56547dc84910 0x56547dcfd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:10.969 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.965+0000 7f603e59c640 1 -- 172.21.15.100:0/1654254867 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56547dc84910 msgr2=0x56547dcfd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:10.969 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.965+0000 7f603e59c640 1 --2- 172.21.15.100:0/1654254867 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56547dc84910 0x56547dcfd5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:10.969 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.965+0000 7f603e59c640 1 -- 172.21.15.100:0/1654254867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56547dc7edb0 msgr2=0x56547dc8ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:10.969 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.965+0000 7f603e59c640 1 --2- 172.21.15.100:0/1654254867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56547dc7edb0 0x56547dc8ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:10.969 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.965+0000 7f603e59c640 1 -- 172.21.15.100:0/1654254867 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56547dcfdb00 con 0x56547dc7f7a0 2024-05-10T11:34:10.970 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.965+0000 7f603e59c640 1 --2- 172.21.15.100:0/1654254867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56547dc7f7a0 0x56547dc8cf90 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f6034009540 tx=0x7f6034004260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=200ef8d06da7004e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:34:10.970 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.965+0000 7f6022ff5640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:34:10.970 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.965+0000 7f6022ff5640 1 -- 172.21.15.100:0/1654254867 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6034024030 con 0x56547dc7f7a0 2024-05-10T11:34:10.970 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.965+0000 7f6022ff5640 1 -- 172.21.15.100:0/1654254867 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f6034005020 con 0x56547dc7f7a0 2024-05-10T11:34:10.970 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.965+0000 7f6047d93740 1 -- 172.21.15.100:0/1654254867 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6038038070 con 0x56547dc7f7a0 2024-05-10T11:34:10.970 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.965+0000 7f6022ff5640 1 -- 172.21.15.100:0/1654254867 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f603402e050 con 0x56547dc7f7a0 2024-05-10T11:34:10.970 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.965+0000 7f6047d93740 10 client.18597 fetch_fsmap learned FSMap version 37 2024-05-10T11:34:10.970 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.965+0000 7f6047d93740 1 -- 172.21.15.100:0/1654254867 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56547dd07660 con 0x56547dc7f7a0 2024-05-10T11:34:10.970 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.966+0000 7f6022ff5640 1 -- 172.21.15.100:0/1654254867 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6038038070 con 0x56547dc7f7a0 2024-05-10T11:34:10.970 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.966+0000 7f6022ff5640 1 -- 172.21.15.100:0/1654254867 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f6034004c90 con 0x56547dc7f7a0 2024-05-10T11:34:10.970 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.966+0000 7f6047d93740 10 client.18597 fetch_fsmap finished waiting for FSMap version 37 2024-05-10T11:34:10.970 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.966+0000 7f6047d93740 10 client.18597 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:34:10.970 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.966+0000 7f6047d93740 20 client.18597 populate_metadata read hostname 'smithi100' 2024-05-10T11:34:10.970 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.966+0000 7f6047d93740 1 --2- 172.21.15.100:0/1654254867 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56547dd080a0 0x56547dd0a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:10.971 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.966+0000 7f6047d93740 4 client.18597 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:34:10.971 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.966+0000 7f6047d93740 1 -- 172.21.15.100:0/1654254867 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56547dbeb050 con 0x56547dd080a0 2024-05-10T11:34:10.971 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.966+0000 7f603ed9d640 1 --2- 172.21.15.100:0/1654254867 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56547dd080a0 0x56547dd0a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:10.971 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.966+0000 7f603ed9d640 1 --2- 172.21.15.100:0/1654254867 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56547dd080a0 0x56547dd0a4c0 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b141ea857812de1a server_cookie=52efa6509b62a070 in_seq=0 out_seq=0 2024-05-10T11:34:10.971 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:10.966+0000 7f6022ff5640 10 client.18597 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:34:11.070 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:11.069+0000 7f6022ff5640 1 -- 172.21.15.100:0/1654254867 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56547dbeb050 con 0x56547dd080a0 2024-05-10T11:34:11.070 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:11.069+0000 7f6022ff5640 10 client.18597 handle_command_reply: tid=0 2024-05-10T11:34:11.070 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:11.069+0000 7f6047d93740 1 client.18597 shutdown 2024-05-10T11:34:11.070 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:11.070+0000 7f6047d93740 1 -- 172.21.15.100:0/1654254867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56547dc7f7a0 msgr2=0x56547dc8cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:11.070 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:11.070+0000 7f6047d93740 1 --2- 172.21.15.100:0/1654254867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56547dc7f7a0 0x56547dc8cf90 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f6034009540 tx=0x7f6034004260 comp rx=0 tx=0).stop 2024-05-10T11:34:11.071 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:11.070+0000 7f6047d93740 1 -- 172.21.15.100:0/1654254867 shutdown_connections 2024-05-10T11:34:11.071 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:11.070+0000 7f6047d93740 1 --2- 172.21.15.100:0/1654254867 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56547dd080a0 0x56547dd0a4c0 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:11.071 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:11.070+0000 7f6047d93740 1 --2- 172.21.15.100:0/1654254867 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56547dc84910 0x56547dcfd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:11.071 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:11.070+0000 7f6047d93740 1 --2- 172.21.15.100:0/1654254867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56547dc7f7a0 0x56547dc8cf90 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:11.071 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:11.070+0000 7f6022ff5640 0 client.18597 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:34:11.071 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:11.070+0000 7f6047d93740 1 --2- 172.21.15.100:0/1654254867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56547dc7edb0 0x56547dc8ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:11.071 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:11.070+0000 7f6047d93740 1 -- 172.21.15.100:0/1654254867 >> 172.21.15.100:0/1654254867 conn(0x56547dc7a420 msgr2=0x56547dc84010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:34:11.071 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:11.070+0000 7f6047d93740 1 -- 172.21.15.100:0/1654254867 shutdown_connections 2024-05-10T11:34:11.071 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:11.070+0000 7f6047d93740 1 -- 172.21.15.100:0/1654254867 wait complete. 2024-05-10T11:34:11.071 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:11.070+0000 7f6047d93740 20 client.18597 trim_cache size 0 max 16384 2024-05-10T11:34:11.071 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:34:11.072 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:34:11.072 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:34:11.072 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:34:11.072 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:11.071+0000 7f6045072640 1 -- 172.21.15.100:0/2000621423 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f60280b2920 msgr2=0x7f60280b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:11.073 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:11.071+0000 7f6045072640 1 --2- 172.21.15.100:0/2000621423 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f60280b2920 0x7f60280b4de0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f603801e730 tx=0x7f6038022000 comp rx=0 tx=0).stop 2024-05-10T11:34:11.073 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:11.071+0000 7f6045072640 1 -- 172.21.15.100:0/2000621423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6040103490 msgr2=0x7f60401968b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:11.073 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:11.071+0000 7f6045072640 1 --2- 172.21.15.100:0/2000621423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6040103490 0x7f60401968b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6030005cd0 tx=0x7f603002ae50 comp rx=0 tx=0).stop 2024-05-10T11:34:11.073 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:11.071+0000 7f6045072640 1 -- 172.21.15.100:0/2000621423 shutdown_connections 2024-05-10T11:34:11.073 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:11.071+0000 7f6045072640 1 --2- 172.21.15.100:0/2000621423 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f60280b2920 0x7f60280b4de0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:11.073 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:11.071+0000 7f6045072640 1 --2- 172.21.15.100:0/2000621423 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60401086f0 0x7f604019de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:11.073 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:11.071+0000 7f6045072640 1 --2- 172.21.15.100:0/2000621423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6040105dc0 0x7f6040196df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:11.073 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:11.071+0000 7f6045072640 1 --2- 172.21.15.100:0/2000621423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6040103490 0x7f60401968b0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:11.073 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:11.071+0000 7f6045072640 1 -- 172.21.15.100:0/2000621423 >> 172.21.15.100:0/2000621423 conn(0x7f60400fcea0 msgr2=0x7f6040103dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:34:11.073 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:11.072+0000 7f6045072640 1 -- 172.21.15.100:0/2000621423 shutdown_connections 2024-05-10T11:34:11.073 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:11.072+0000 7f6045072640 1 -- 172.21.15.100:0/2000621423 wait complete. 2024-05-10T11:34:11.326 INFO:tasks.cephfs.filesystem:scrub status for tag:13915d86-3bd1-433f-acaf-75c8a06d28a0 - {'path': '/', 'tag': '13915d86-3bd1-433f-acaf-75c8a06d28a0', 'options': 'recursive,force'} 2024-05-10T11:34:11.914 INFO:tasks.workunit.client.0.smithi100.stdout: 10 21351 10.72 MB/sec execute 5 sec latency 107.244 ms 2024-05-10T11:34:12.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:34:11 smithi178 ceph-mon[33359]: pgmap v895: 129 pgs: 129 active+clean; 393 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 987 KiB/s rd, 4.2 MiB/s wr, 353 op/s 2024-05-10T11:34:12.030 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:34:11 smithi188 ceph-mon[34075]: pgmap v895: 129 pgs: 129 active+clean; 393 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 987 KiB/s rd, 4.2 MiB/s wr, 353 op/s 2024-05-10T11:34:12.040 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:34:11 smithi100 ceph-mon[30296]: pgmap v895: 129 pgs: 129 active+clean; 393 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 987 KiB/s rd, 4.2 MiB/s wr, 353 op/s 2024-05-10T11:34:12.914 INFO:tasks.workunit.client.0.smithi100.stdout: 10 21500 10.83 MB/sec execute 6 sec latency 200.129 ms 2024-05-10T11:34:13.914 INFO:tasks.workunit.client.0.smithi100.stdout: 10 21678 11.28 MB/sec execute 7 sec latency 53.287 ms 2024-05-10T11:34:14.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:34:13 smithi178 ceph-mon[33359]: pgmap v896: 129 pgs: 129 active+clean; 395 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 4.0 MiB/s wr, 339 op/s 2024-05-10T11:34:14.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:34:13 smithi188 ceph-mon[34075]: pgmap v896: 129 pgs: 129 active+clean; 395 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 4.0 MiB/s wr, 339 op/s 2024-05-10T11:34:14.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:34:13 smithi100 ceph-mon[30296]: pgmap v896: 129 pgs: 129 active+clean; 395 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 4.0 MiB/s wr, 339 op/s 2024-05-10T11:34:14.914 INFO:tasks.workunit.client.0.smithi100.stdout: 10 21888 11.70 MB/sec execute 8 sec latency 59.376 ms 2024-05-10T11:34:15.914 INFO:tasks.workunit.client.0.smithi100.stdout: 10 22080 11.24 MB/sec execute 9 sec latency 82.135 ms 2024-05-10T11:34:16.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:34:15 smithi178 ceph-mon[33359]: pgmap v897: 129 pgs: 129 active+clean; 399 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 4.2 MiB/s wr, 331 op/s 2024-05-10T11:34:16.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:34:15 smithi188 ceph-mon[34075]: pgmap v897: 129 pgs: 129 active+clean; 399 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 4.2 MiB/s wr, 331 op/s 2024-05-10T11:34:16.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:34:15 smithi100 ceph-mon[30296]: pgmap v897: 129 pgs: 129 active+clean; 399 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 4.2 MiB/s wr, 331 op/s 2024-05-10T11:34:16.914 INFO:tasks.workunit.client.0.smithi100.stdout: 10 22218 10.94 MB/sec execute 10 sec latency 98.500 ms 2024-05-10T11:34:17.914 INFO:tasks.workunit.client.0.smithi100.stdout: 10 22352 10.13 MB/sec execute 11 sec latency 154.731 ms 2024-05-10T11:34:18.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:34:17 smithi178 ceph-mon[33359]: pgmap v898: 129 pgs: 129 active+clean; 406 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 4.8 MiB/s wr, 361 op/s 2024-05-10T11:34:18.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:34:17 smithi188 ceph-mon[34075]: pgmap v898: 129 pgs: 129 active+clean; 406 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 4.8 MiB/s wr, 361 op/s 2024-05-10T11:34:18.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:34:17 smithi100 ceph-mon[30296]: pgmap v898: 129 pgs: 129 active+clean; 406 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 4.8 MiB/s wr, 361 op/s 2024-05-10T11:34:18.914 INFO:tasks.workunit.client.0.smithi100.stdout: 10 22550 10.51 MB/sec execute 12 sec latency 67.510 ms 2024-05-10T11:34:19.914 INFO:tasks.workunit.client.0.smithi100.stdout: 10 22717 10.28 MB/sec execute 13 sec latency 116.152 ms 2024-05-10T11:34:20.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:34:19 smithi178 ceph-mon[33359]: pgmap v899: 129 pgs: 129 active+clean; 405 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 4.5 MiB/s wr, 308 op/s 2024-05-10T11:34:20.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:34:19 smithi188 ceph-mon[34075]: pgmap v899: 129 pgs: 129 active+clean; 405 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 4.5 MiB/s wr, 308 op/s 2024-05-10T11:34:20.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:34:19 smithi100 ceph-mon[30296]: pgmap v899: 129 pgs: 129 active+clean; 405 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 4.5 MiB/s wr, 308 op/s 2024-05-10T11:34:20.914 INFO:tasks.workunit.client.0.smithi100.stdout: 10 22882 9.95 MB/sec execute 14 sec latency 124.751 ms 2024-05-10T11:34:21.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:34:20 smithi178 ceph-mon[33359]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T11:34:21.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:34:20 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:34:21.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:34:20 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:34:21.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:34:20 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:34:21.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:34:20 smithi188 ceph-mon[34075]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T11:34:21.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:34:20 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:34:21.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:34:20 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:34:21.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:34:20 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:34:21.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:34:20 smithi100 ceph-mon[30296]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T11:34:21.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:34:20 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:34:21.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:34:20 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:34:21.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:34:20 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:34:21.915 INFO:tasks.workunit.client.0.smithi100.stdout: 10 23051 10.13 MB/sec execute 15 sec latency 154.506 ms 2024-05-10T11:34:22.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:34:21 smithi178 ceph-mon[33359]: pgmap v900: 129 pgs: 129 active+clean; 406 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 980 KiB/s rd, 4.9 MiB/s wr, 339 op/s 2024-05-10T11:34:22.064 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:34:21 smithi100 ceph-mon[30296]: pgmap v900: 129 pgs: 129 active+clean; 406 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 980 KiB/s rd, 4.9 MiB/s wr, 339 op/s 2024-05-10T11:34:22.064 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:34:21 smithi188 ceph-mon[34075]: pgmap v900: 129 pgs: 129 active+clean; 406 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 980 KiB/s rd, 4.9 MiB/s wr, 339 op/s 2024-05-10T11:34:22.914 INFO:tasks.workunit.client.0.smithi100.stdout: 10 23222 9.70 MB/sec execute 16 sec latency 88.863 ms 2024-05-10T11:34:23.915 INFO:tasks.workunit.client.0.smithi100.stdout: 10 23389 9.50 MB/sec execute 17 sec latency 68.779 ms 2024-05-10T11:34:24.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:34:23 smithi178 ceph-mon[33359]: pgmap v901: 129 pgs: 129 active+clean; 408 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.6 MiB/s wr, 313 op/s 2024-05-10T11:34:24.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:34:23 smithi188 ceph-mon[34075]: pgmap v901: 129 pgs: 129 active+clean; 408 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.6 MiB/s wr, 313 op/s 2024-05-10T11:34:24.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:34:23 smithi100 ceph-mon[30296]: pgmap v901: 129 pgs: 129 active+clean; 408 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.6 MiB/s wr, 313 op/s 2024-05-10T11:34:24.914 INFO:tasks.workunit.client.0.smithi100.stdout: 10 23554 9.19 MB/sec execute 18 sec latency 68.660 ms 2024-05-10T11:34:25.915 INFO:tasks.workunit.client.0.smithi100.stdout: 10 23752 9.28 MB/sec execute 19 sec latency 54.916 ms 2024-05-10T11:34:26.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:34:25 smithi178 ceph-mon[33359]: pgmap v902: 129 pgs: 129 active+clean; 404 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 4.5 MiB/s wr, 278 op/s 2024-05-10T11:34:26.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:34:25 smithi188 ceph-mon[34075]: pgmap v902: 129 pgs: 129 active+clean; 404 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 4.5 MiB/s wr, 278 op/s 2024-05-10T11:34:26.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:34:25 smithi100 ceph-mon[30296]: pgmap v902: 129 pgs: 129 active+clean; 404 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 4.5 MiB/s wr, 278 op/s 2024-05-10T11:34:26.915 INFO:tasks.workunit.client.0.smithi100.stdout: 10 23943 9.55 MB/sec execute 20 sec latency 69.335 ms 2024-05-10T11:34:27.915 INFO:tasks.workunit.client.0.smithi100.stdout: 10 24158 9.69 MB/sec execute 21 sec latency 99.291 ms 2024-05-10T11:34:28.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:34:27 smithi178 ceph-mon[33359]: pgmap v903: 129 pgs: 129 active+clean; 406 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.7 MiB/s wr, 328 op/s 2024-05-10T11:34:28.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:34:27 smithi188 ceph-mon[34075]: pgmap v903: 129 pgs: 129 active+clean; 406 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.7 MiB/s wr, 328 op/s 2024-05-10T11:34:28.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:34:27 smithi100 ceph-mon[30296]: pgmap v903: 129 pgs: 129 active+clean; 406 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.7 MiB/s wr, 328 op/s 2024-05-10T11:34:28.784 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.782+0000 7f45d8699640 1 -- 172.21.15.100:0/3787451322 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45d0105e00 msgr2=0x7f45d01081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:28.784 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.782+0000 7f45d8699640 1 --2- 172.21.15.100:0/3787451322 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45d0105e00 0x7f45d01081f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f45cc009ca0 tx=0x7f45cc02a8c0 comp rx=0 tx=0).stop 2024-05-10T11:34:28.784 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.782+0000 7f45d8699640 1 -- 172.21.15.100:0/3787451322 shutdown_connections 2024-05-10T11:34:28.784 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.782+0000 7f45d8699640 1 --2- 172.21.15.100:0/3787451322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45d0108730 0x7f45d010ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:28.784 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.782+0000 7f45d8699640 1 --2- 172.21.15.100:0/3787451322 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45d0105e00 0x7f45d01081f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:28.784 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.782+0000 7f45d8699640 1 --2- 172.21.15.100:0/3787451322 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45d01034d0 0x7f45d01058c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:28.785 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.782+0000 7f45d8699640 1 -- 172.21.15.100:0/3787451322 >> 172.21.15.100:0/3787451322 conn(0x7f45d00fce80 msgr2=0x7f45d00ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:34:28.785 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.782+0000 7f45d8699640 1 -- 172.21.15.100:0/3787451322 shutdown_connections 2024-05-10T11:34:28.785 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.782+0000 7f45d8699640 1 -- 172.21.15.100:0/3787451322 wait complete. 2024-05-10T11:34:28.785 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.783+0000 7f45d8699640 1 Processor -- start 2024-05-10T11:34:28.785 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.783+0000 7f45d8699640 1 -- start start 2024-05-10T11:34:28.785 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.783+0000 7f45d8699640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45d01034d0 0x7f45d0102e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:28.785 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.783+0000 7f45d8699640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45d0105e00 0x7f45d0101480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:28.785 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.783+0000 7f45d8699640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45d0108730 0x7f45d01019c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:28.785 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.783+0000 7f45d8699640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f45d010e440 con 0x7f45d0108730 2024-05-10T11:34:28.785 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.783+0000 7f45d8699640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f45d010e2c0 con 0x7f45d0105e00 2024-05-10T11:34:28.785 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.783+0000 7f45d8699640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f45d010e5c0 con 0x7f45d01034d0 2024-05-10T11:34:28.785 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.783+0000 7f45d6435640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45d01034d0 0x7f45d0102e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:28.785 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.783+0000 7f45d6c36640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45d0108730 0x7f45d01019c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:28.785 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.783+0000 7f45d5c34640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45d0105e00 0x7f45d0101480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.783+0000 7f45d6435640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45d01034d0 0x7f45d0102e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:54606/0 (socket says 172.21.15.100:54606) 2024-05-10T11:34:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.783+0000 7f45d6435640 1 -- 172.21.15.100:0/934999884 learned_addr learned my addr 172.21.15.100:0/934999884 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:34:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.783+0000 7f45d6c36640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45d0108730 0x7f45d01019c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44316/0 (socket says 172.21.15.100:44316) 2024-05-10T11:34:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.783+0000 7f45d5c34640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45d0105e00 0x7f45d0101480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:57832/0 (socket says 172.21.15.100:57832) 2024-05-10T11:34:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.784+0000 7f45d6435640 1 -- 172.21.15.100:0/934999884 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45d0105e00 msgr2=0x7f45d0101480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.784+0000 7f45d6435640 1 --2- 172.21.15.100:0/934999884 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45d0105e00 0x7f45d0101480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.784+0000 7f45d6435640 1 -- 172.21.15.100:0/934999884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45d0108730 msgr2=0x7f45d01019c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.784+0000 7f45d6435640 1 --2- 172.21.15.100:0/934999884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45d0108730 0x7f45d01019c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.784+0000 7f45d6435640 1 -- 172.21.15.100:0/934999884 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45d0102250 con 0x7f45d01034d0 2024-05-10T11:34:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.784+0000 7f45d5c34640 1 --2- 172.21.15.100:0/934999884 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45d0105e00 0x7f45d0101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:34:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.784+0000 7f45d6c36640 1 --2- 172.21.15.100:0/934999884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45d0108730 0x7f45d01019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:34:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.784+0000 7f45d6435640 1 --2- 172.21.15.100:0/934999884 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45d01034d0 0x7f45d0102e40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f45c400e9f0 tx=0x7f45c400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:34:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.784+0000 7f45c37fe640 1 -- 172.21.15.100:0/934999884 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c400cd80 con 0x7f45d01034d0 2024-05-10T11:34:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.784+0000 7f45d8699640 1 -- 172.21.15.100:0/934999884 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45d01a2c30 con 0x7f45d01034d0 2024-05-10T11:34:28.787 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.784+0000 7f45d8699640 1 -- 172.21.15.100:0/934999884 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45d01a3150 con 0x7f45d01034d0 2024-05-10T11:34:28.787 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.784+0000 7f45c37fe640 1 -- 172.21.15.100:0/934999884 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f45c40054b0 con 0x7f45d01034d0 2024-05-10T11:34:28.787 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.784+0000 7f45c37fe640 1 -- 172.21.15.100:0/934999884 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c4010430 con 0x7f45d01034d0 2024-05-10T11:34:28.787 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.787+0000 7f45c37fe640 1 -- 172.21.15.100:0/934999884 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f45c40046c0 con 0x7f45d01034d0 2024-05-10T11:34:28.788 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.788+0000 7f45c37fe640 1 --2- 172.21.15.100:0/934999884 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f45bc0b2920 0x7f45bc0b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:28.788 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.789+0000 7f45c37fe640 1 -- 172.21.15.100:0/934999884 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f45c40e28c0 con 0x7f45d01034d0 2024-05-10T11:34:28.788 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.789+0000 7f45d5c34640 1 --2- 172.21.15.100:0/934999884 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f45bc0b2920 0x7f45bc0b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.789+0000 7f45d5c34640 1 --2- 172.21.15.100:0/934999884 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f45bc0b2920 0x7f45bc0b4de0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f45cc000f80 tx=0x7f45cc031000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:34:28.915 INFO:tasks.workunit.client.0.smithi100.stdout: 10 24399 10.08 MB/sec execute 22 sec latency 66.000 ms 2024-05-10T11:34:28.923 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.923+0000 7f45db3ba740 1 Processor -- start 2024-05-10T11:34:28.924 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.923+0000 7f45db3ba740 1 -- start start 2024-05-10T11:34:28.925 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.924+0000 7f45db3ba740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5570911addb0 0x5570911ae1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:28.925 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.924+0000 7f45db3ba740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5570911ae7a0 0x5570911b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:28.925 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.924+0000 7f45db3ba740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5570911b3910 0x5570911b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:28.925 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.924+0000 7f45db3ba740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5570911125d0 con 0x5570911ae7a0 2024-05-10T11:34:28.925 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.924+0000 7f45db3ba740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5570910cb890 con 0x5570911addb0 2024-05-10T11:34:28.925 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.924+0000 7f45db3ba740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55709110e590 con 0x5570911b3910 2024-05-10T11:34:28.925 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.924+0000 7f45d5c34640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5570911ae7a0 0x5570911b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:28.925 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.924+0000 7f45d6c36640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5570911b3910 0x5570911b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:28.925 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.924+0000 7f45d5c34640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5570911ae7a0 0x5570911b32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44324/0 (socket says 172.21.15.100:44324) 2024-05-10T11:34:28.925 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.924+0000 7f45d5c34640 1 -- 172.21.15.100:0/2179790726 learned_addr learned my addr 172.21.15.100:0/2179790726 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:34:28.925 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.924+0000 7f45d6435640 1 --2- 172.21.15.100:0/2179790726 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5570911addb0 0x5570911ae1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:28.925 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.924+0000 7f45d5c34640 1 -- 172.21.15.100:0/2179790726 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5570911b3910 msgr2=0x5570911b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:28.925 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.924+0000 7f45d5c34640 1 --2- 172.21.15.100:0/2179790726 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5570911b3910 0x5570911b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:28.926 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.924+0000 7f45d5c34640 1 -- 172.21.15.100:0/2179790726 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5570911addb0 msgr2=0x5570911ae1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:28.926 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.924+0000 7f45d5c34640 1 --2- 172.21.15.100:0/2179790726 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5570911addb0 0x5570911ae1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:28.926 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.924+0000 7f45d5c34640 1 -- 172.21.15.100:0/2179790726 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557090e24210 con 0x5570911ae7a0 2024-05-10T11:34:28.926 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.924+0000 7f45d5c34640 1 --2- 172.21.15.100:0/2179790726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5570911ae7a0 0x5570911b32a0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f45cc035000 tx=0x7f45cc03f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f387e330ee9477a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:34:28.926 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.925+0000 7f45c0ff9640 1 -- 172.21.15.100:0/2179790726 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45cc004040 con 0x5570911ae7a0 2024-05-10T11:34:28.926 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.925+0000 7f45c0ff9640 1 -- 172.21.15.100:0/2179790726 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f45cc04f070 con 0x5570911ae7a0 2024-05-10T11:34:28.926 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.925+0000 7f45c0ff9640 1 -- 172.21.15.100:0/2179790726 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45cc030bf0 con 0x5570911ae7a0 2024-05-10T11:34:28.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.926+0000 7f45db3ba740 1 -- 172.21.15.100:0/2179790726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5570911ae7a0 msgr2=0x5570911b32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:28.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.926+0000 7f45db3ba740 1 --2- 172.21.15.100:0/2179790726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5570911ae7a0 0x5570911b32a0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f45cc035000 tx=0x7f45cc03f000 comp rx=0 tx=0).stop 2024-05-10T11:34:28.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.926+0000 7f45db3ba740 1 -- 172.21.15.100:0/2179790726 shutdown_connections 2024-05-10T11:34:28.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.926+0000 7f45db3ba740 1 --2- 172.21.15.100:0/2179790726 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5570911b3910 0x5570911b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:28.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.926+0000 7f45db3ba740 1 --2- 172.21.15.100:0/2179790726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5570911ae7a0 0x5570911b32a0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:28.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.926+0000 7f45db3ba740 1 --2- 172.21.15.100:0/2179790726 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5570911addb0 0x5570911ae1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:28.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.926+0000 7f45db3ba740 1 -- 172.21.15.100:0/2179790726 >> 172.21.15.100:0/2179790726 conn(0x5570911a9420 msgr2=0x5570911a9820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:34:28.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.926+0000 7f45db3ba740 1 -- 172.21.15.100:0/2179790726 shutdown_connections 2024-05-10T11:34:28.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.926+0000 7f45db3ba740 1 -- 172.21.15.100:0/2179790726 wait complete. 2024-05-10T11:34:28.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.926+0000 7f45db3ba740 1 -- start start 2024-05-10T11:34:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.926+0000 7f45db3ba740 1 Processor -- start 2024-05-10T11:34:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.926+0000 7f45db3ba740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5570911addb0 0x5570911c9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.926+0000 7f45db3ba740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5570911b3910 0x5570911c95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.926+0000 7f45db3ba740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5570911ce570 0x5570911d0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.926+0000 7f45db3ba740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5570911c76f0 con 0x5570911b3910 2024-05-10T11:34:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.926+0000 7f45db3ba740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5570910cf370 con 0x5570911ce570 2024-05-10T11:34:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.926+0000 7f45db3ba740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5570911c7870 con 0x5570911addb0 2024-05-10T11:34:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.926+0000 7f45db3ba740 10 client.0 _ll_register_callbacks cb 0x557090e23a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:34:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.927+0000 7f45d6435640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5570911addb0 0x5570911c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.927+0000 7f45d6435640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5570911addb0 0x5570911c9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:54620/0 (socket says 172.21.15.100:54620) 2024-05-10T11:34:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.927+0000 7f45d6435640 1 -- 172.21.15.100:0/1079959992 learned_addr learned my addr 172.21.15.100:0/1079959992 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:34:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.927+0000 7f45d6c36640 1 --2- 172.21.15.100:0/1079959992 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5570911ce570 0x5570911d0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.927+0000 7f45d5c34640 1 --2- 172.21.15.100:0/1079959992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5570911b3910 0x5570911c95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.927+0000 7f45d5c34640 1 -- 172.21.15.100:0/1079959992 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5570911addb0 msgr2=0x5570911c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.927+0000 7f45d5c34640 1 --2- 172.21.15.100:0/1079959992 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5570911addb0 0x5570911c9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.927+0000 7f45d5c34640 1 -- 172.21.15.100:0/1079959992 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5570911ce570 msgr2=0x5570911d0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.927+0000 7f45d5c34640 1 --2- 172.21.15.100:0/1079959992 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5570911ce570 0x5570911d0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.927+0000 7f45d5c34640 1 -- 172.21.15.100:0/1079959992 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557090ef0620 con 0x5570911b3910 2024-05-10T11:34:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.927+0000 7f45d6435640 1 --2- 172.21.15.100:0/1079959992 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5570911addb0 0x5570911c9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:34:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.927+0000 7f45d6c36640 1 --2- 172.21.15.100:0/1079959992 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5570911ce570 0x5570911d0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:34:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.927+0000 7f45d5c34640 1 --2- 172.21.15.100:0/1079959992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5570911b3910 0x5570911c95b0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x5570911aec60 tx=0x7f45cc030c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef29c3900128ff15 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:34:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.927+0000 7f45b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:34:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.927+0000 7f45b9ffb640 1 -- 172.21.15.100:0/1079959992 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45cc058070 con 0x5570911b3910 2024-05-10T11:34:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.927+0000 7f45b9ffb640 1 -- 172.21.15.100:0/1079959992 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f45cc004450 con 0x5570911b3910 2024-05-10T11:34:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.927+0000 7f45db3ba740 1 -- 172.21.15.100:0/1079959992 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45cc04f070 con 0x5570911b3910 2024-05-10T11:34:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.928+0000 7f45b9ffb640 1 -- 172.21.15.100:0/1079959992 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f45cc030e10 con 0x5570911b3910 2024-05-10T11:34:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.928+0000 7f45db3ba740 10 client.18609 fetch_fsmap learned FSMap version 38 2024-05-10T11:34:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.928+0000 7f45db3ba740 1 -- 172.21.15.100:0/1079959992 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5570911c76f0 con 0x5570911b3910 2024-05-10T11:34:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.928+0000 7f45b9ffb640 1 -- 172.21.15.100:0/1079959992 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f45cc04f070 con 0x5570911b3910 2024-05-10T11:34:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.928+0000 7f45b9ffb640 1 -- 172.21.15.100:0/1079959992 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7f45cc043040 con 0x5570911b3910 2024-05-10T11:34:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.928+0000 7f45db3ba740 10 client.18609 fetch_fsmap finished waiting for FSMap version 38 2024-05-10T11:34:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.928+0000 7f45db3ba740 10 client.18609 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:34:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.928+0000 7f45db3ba740 20 client.18609 populate_metadata read hostname 'smithi100' 2024-05-10T11:34:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.928+0000 7f45db3ba740 1 --2- 172.21.15.100:0/1079959992 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5570911d4e00 0x5570911d7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.928+0000 7f45db3ba740 4 client.18609 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:34:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.928+0000 7f45db3ba740 1 -- 172.21.15.100:0/1079959992 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570911201e0 con 0x5570911d4e00 2024-05-10T11:34:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.928+0000 7f45d6435640 1 --2- 172.21.15.100:0/1079959992 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5570911d4e00 0x5570911d7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.929+0000 7f45d6435640 1 --2- 172.21.15.100:0/1079959992 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5570911d4e00 0x5570911d7220 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71c6a6db8ad04ae6 server_cookie=a487a945fad87631 in_seq=0 out_seq=0 2024-05-10T11:34:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.929+0000 7f45b9ffb640 10 client.18609 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:34:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.929+0000 7f45b9ffb640 1 -- 172.21.15.100:0/1079959992 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5570911201e0 con 0x5570911d4e00 2024-05-10T11:34:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.929+0000 7f45b9ffb640 10 client.18609 handle_command_reply: tid=0 2024-05-10T11:34:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.929+0000 7f45db3ba740 1 client.18609 shutdown 2024-05-10T11:34:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.929+0000 7f45db3ba740 1 -- 172.21.15.100:0/1079959992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5570911b3910 msgr2=0x5570911c95b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.929+0000 7f45db3ba740 1 --2- 172.21.15.100:0/1079959992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5570911b3910 0x5570911c95b0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x5570911aec60 tx=0x7f45cc030c20 comp rx=0 tx=0).stop 2024-05-10T11:34:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.929+0000 7f45db3ba740 1 -- 172.21.15.100:0/1079959992 shutdown_connections 2024-05-10T11:34:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.930+0000 7f45db3ba740 1 --2- 172.21.15.100:0/1079959992 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5570911d4e00 0x5570911d7220 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.930+0000 7f45db3ba740 1 --2- 172.21.15.100:0/1079959992 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5570911ce570 0x5570911d0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:28.935 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.930+0000 7f45db3ba740 1 --2- 172.21.15.100:0/1079959992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5570911b3910 0x5570911c95b0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:28.935 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.930+0000 7f45b9ffb640 0 client.18609 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:34:28.935 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.930+0000 7f45db3ba740 1 --2- 172.21.15.100:0/1079959992 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5570911addb0 0x5570911c9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:28.935 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.930+0000 7f45db3ba740 1 -- 172.21.15.100:0/1079959992 >> 172.21.15.100:0/1079959992 conn(0x5570911a9420 msgr2=0x5570911a8580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:34:28.935 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.930+0000 7f45db3ba740 1 -- 172.21.15.100:0/1079959992 shutdown_connections 2024-05-10T11:34:28.935 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.930+0000 7f45db3ba740 1 -- 172.21.15.100:0/1079959992 wait complete. 2024-05-10T11:34:28.935 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.930+0000 7f45db3ba740 20 client.18609 trim_cache size 0 max 16384 2024-05-10T11:34:28.955 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.955+0000 7f45db3ba740 1 Processor -- start 2024-05-10T11:34:28.955 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.955+0000 7f45db3ba740 1 -- start start 2024-05-10T11:34:28.956 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.955+0000 7f45db3ba740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5570911addb0 0x5570911a9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:28.956 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.955+0000 7f45db3ba740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5570911b3910 0x5570911c8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:28.956 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.955+0000 7f45db3ba740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5570911ce570 0x5570911c88a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:28.957 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.955+0000 7f45db3ba740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f45b4012370 con 0x5570911ce570 2024-05-10T11:34:28.957 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.955+0000 7f45db3ba740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f45b400fbb0 con 0x5570911b3910 2024-05-10T11:34:28.957 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.956+0000 7f45db3ba740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f45b400da00 con 0x5570911addb0 2024-05-10T11:34:28.957 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.956+0000 7f45d5c34640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5570911b3910 0x5570911c8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:28.957 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.956+0000 7f45d6435640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5570911addb0 0x5570911a9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:28.957 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.956+0000 7f45d5c34640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5570911b3910 0x5570911c8360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:57854/0 (socket says 172.21.15.100:57854) 2024-05-10T11:34:28.957 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.956+0000 7f45d5c34640 1 -- 172.21.15.100:0/3105453642 learned_addr learned my addr 172.21.15.100:0/3105453642 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:34:28.957 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.956+0000 7f45d6435640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5570911addb0 0x5570911a9800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:54624/0 (socket says 172.21.15.100:54624) 2024-05-10T11:34:28.957 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.956+0000 7f45d6c36640 1 --2- 172.21.15.100:0/3105453642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5570911ce570 0x5570911c88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:28.957 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.956+0000 7f45d5c34640 1 -- 172.21.15.100:0/3105453642 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5570911addb0 msgr2=0x5570911a9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:28.957 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.956+0000 7f45d5c34640 1 --2- 172.21.15.100:0/3105453642 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5570911addb0 0x5570911a9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:28.957 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.956+0000 7f45d5c34640 1 -- 172.21.15.100:0/3105453642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5570911ce570 msgr2=0x5570911c88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:28.957 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.956+0000 7f45d5c34640 1 --2- 172.21.15.100:0/3105453642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5570911ce570 0x5570911c88a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:28.957 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.956+0000 7f45d5c34640 1 -- 172.21.15.100:0/3105453642 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570911ca100 con 0x5570911b3910 2024-05-10T11:34:28.958 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.956+0000 7f45d6c36640 1 --2- 172.21.15.100:0/3105453642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5570911ce570 0x5570911c88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:34:28.958 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.957+0000 7f45d5c34640 1 --2- 172.21.15.100:0/3105453642 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5570911b3910 0x5570911c8360 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f45cc030fd0 tx=0x7f45cc02a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd0d0d4c7397606f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:34:28.959 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.957+0000 7f45bbfff640 1 -- 172.21.15.100:0/3105453642 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45cc058070 con 0x5570911b3910 2024-05-10T11:34:28.959 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.957+0000 7f45bbfff640 1 -- 172.21.15.100:0/3105453642 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f45cc04f070 con 0x5570911b3910 2024-05-10T11:34:28.959 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.957+0000 7f45bbfff640 1 -- 172.21.15.100:0/3105453642 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45cc0045d0 con 0x5570911b3910 2024-05-10T11:34:28.959 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.958+0000 7f45db3ba740 1 -- 172.21.15.100:0/3105453642 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5570911b3910 msgr2=0x5570911c8360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:28.959 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.958+0000 7f45db3ba740 1 --2- 172.21.15.100:0/3105453642 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5570911b3910 0x5570911c8360 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f45cc030fd0 tx=0x7f45cc02a8c0 comp rx=0 tx=0).stop 2024-05-10T11:34:28.959 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.958+0000 7f45db3ba740 1 -- 172.21.15.100:0/3105453642 shutdown_connections 2024-05-10T11:34:28.959 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.958+0000 7f45db3ba740 1 --2- 172.21.15.100:0/3105453642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5570911ce570 0x5570911c88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:28.959 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.958+0000 7f45db3ba740 1 --2- 172.21.15.100:0/3105453642 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5570911b3910 0x5570911c8360 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:28.959 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.958+0000 7f45db3ba740 1 --2- 172.21.15.100:0/3105453642 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5570911addb0 0x5570911a9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:28.959 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.958+0000 7f45db3ba740 1 -- 172.21.15.100:0/3105453642 >> 172.21.15.100:0/3105453642 conn(0x5570911a9420 msgr2=0x5570911a4530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:34:28.960 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.959+0000 7f45db3ba740 1 -- 172.21.15.100:0/3105453642 shutdown_connections 2024-05-10T11:34:28.960 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.959+0000 7f45db3ba740 1 -- 172.21.15.100:0/3105453642 wait complete. 2024-05-10T11:34:28.961 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.959+0000 7f45db3ba740 1 -- start start 2024-05-10T11:34:28.961 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.959+0000 7f45db3ba740 1 Processor -- start 2024-05-10T11:34:28.961 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.960+0000 7f45db3ba740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5570911addb0 0x55709123be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:28.961 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.960+0000 7f45db3ba740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5570911b3910 0x55709123c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:28.961 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.960+0000 7f45db3ba740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5570911ce570 0x5570911e7460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:28.961 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.960+0000 7f45db3ba740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f45b400e070 con 0x5570911b3910 2024-05-10T11:34:28.961 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.960+0000 7f45db3ba740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f45b400f4e0 con 0x5570911addb0 2024-05-10T11:34:28.962 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.960+0000 7f45db3ba740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f45b400e740 con 0x5570911ce570 2024-05-10T11:34:28.962 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.960+0000 7f45db3ba740 10 client.0 _ll_register_callbacks cb 0x7f45c4004a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:34:28.962 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.960+0000 7f45d6435640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5570911addb0 0x55709123be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:28.962 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.960+0000 7f45d5c34640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5570911b3910 0x55709123c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:28.962 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.960+0000 7f45d6435640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5570911addb0 0x55709123be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:57866/0 (socket says 172.21.15.100:57866) 2024-05-10T11:34:28.962 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.960+0000 7f45d6435640 1 -- 172.21.15.100:0/3131747137 learned_addr learned my addr 172.21.15.100:0/3131747137 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:34:28.962 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.960+0000 7f45d6c36640 1 --2- 172.21.15.100:0/3131747137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5570911ce570 0x5570911e7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:28.962 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.961+0000 7f45d6435640 1 -- 172.21.15.100:0/3131747137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5570911ce570 msgr2=0x5570911e7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:28.962 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.961+0000 7f45d6435640 1 --2- 172.21.15.100:0/3131747137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5570911ce570 0x5570911e7460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:28.962 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.961+0000 7f45d6435640 1 -- 172.21.15.100:0/3131747137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5570911b3910 msgr2=0x55709123c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:28.962 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.961+0000 7f45d6435640 1 --2- 172.21.15.100:0/3131747137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5570911b3910 0x55709123c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:28.962 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.961+0000 7f45d6435640 1 -- 172.21.15.100:0/3131747137 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570911e7aa0 con 0x5570911addb0 2024-05-10T11:34:28.962 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.961+0000 7f45d6435640 1 --2- 172.21.15.100:0/3131747137 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5570911addb0 0x55709123be20 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f45c40a9820 tx=0x7f45c40dfed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a40f5a95480719 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:34:28.963 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.961+0000 7f45b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:34:28.963 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.962+0000 7f45b9ffb640 1 -- 172.21.15.100:0/3131747137 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c40a9630 con 0x5570911addb0 2024-05-10T11:34:28.963 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.962+0000 7f45b9ffb640 1 -- 172.21.15.100:0/3131747137 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f45c40a6110 con 0x5570911addb0 2024-05-10T11:34:28.964 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.962+0000 7f45db3ba740 1 -- 172.21.15.100:0/3131747137 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45cc04f070 con 0x5570911addb0 2024-05-10T11:34:28.964 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.963+0000 7f45b9ffb640 1 -- 172.21.15.100:0/3131747137 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f45c40aa220 con 0x5570911addb0 2024-05-10T11:34:28.964 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.963+0000 7f45db3ba740 10 client.27932 fetch_fsmap learned FSMap version 38 2024-05-10T11:34:28.964 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.963+0000 7f45db3ba740 1 -- 172.21.15.100:0/3131747137 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5570911e8e20 con 0x5570911addb0 2024-05-10T11:34:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.965+0000 7f45b9ffb640 1 -- 172.21.15.100:0/3131747137 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f45cc04f070 con 0x5570911addb0 2024-05-10T11:34:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.965+0000 7f45b9ffb640 1 -- 172.21.15.100:0/3131747137 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7f45c40a8d80 con 0x5570911addb0 2024-05-10T11:34:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.965+0000 7f45db3ba740 10 client.27932 fetch_fsmap finished waiting for FSMap version 38 2024-05-10T11:34:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.965+0000 7f45db3ba740 10 client.27932 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:34:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.965+0000 7f45db3ba740 20 client.27932 populate_metadata read hostname 'smithi100' 2024-05-10T11:34:28.966 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.965+0000 7f45db3ba740 1 --2- 172.21.15.100:0/3131747137 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5570912481d0 0x55709124a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:28.966 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.965+0000 7f45db3ba740 4 client.27932 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:34:28.966 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.965+0000 7f45db3ba740 1 -- 172.21.15.100:0/3131747137 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55709111a050 con 0x5570912481d0 2024-05-10T11:34:28.966 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.966+0000 7f45d5c34640 1 --2- 172.21.15.100:0/3131747137 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5570912481d0 0x55709124a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:28.967 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.968+0000 7f45d5c34640 1 --2- 172.21.15.100:0/3131747137 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5570912481d0 0x55709124a5f0 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7658a4aa11e458c7 server_cookie=5cf40ed0c222a15c in_seq=0 out_seq=0 2024-05-10T11:34:28.968 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:28.968+0000 7f45b9ffb640 10 client.27932 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:34:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:29.080+0000 7f45b9ffb640 1 -- 172.21.15.100:0/3131747137 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55709111a050 con 0x5570912481d0 2024-05-10T11:34:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:29.080+0000 7f45b9ffb640 10 client.27932 handle_command_reply: tid=0 2024-05-10T11:34:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:29.080+0000 7f45db3ba740 1 client.27932 shutdown 2024-05-10T11:34:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:29.080+0000 7f45db3ba740 1 -- 172.21.15.100:0/3131747137 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5570911addb0 msgr2=0x55709123be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:29.080+0000 7f45db3ba740 1 --2- 172.21.15.100:0/3131747137 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5570911addb0 0x55709123be20 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f45c40a9820 tx=0x7f45c40dfed0 comp rx=0 tx=0).stop 2024-05-10T11:34:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:29.080+0000 7f45db3ba740 1 -- 172.21.15.100:0/3131747137 shutdown_connections 2024-05-10T11:34:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:29.080+0000 7f45db3ba740 1 --2- 172.21.15.100:0/3131747137 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5570912481d0 0x55709124a5f0 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:29.080+0000 7f45db3ba740 1 --2- 172.21.15.100:0/3131747137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5570911ce570 0x5570911e7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:29.080+0000 7f45b9ffb640 0 client.27932 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:34:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:29.080+0000 7f45db3ba740 1 --2- 172.21.15.100:0/3131747137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5570911b3910 0x55709123c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:29.080+0000 7f45db3ba740 1 --2- 172.21.15.100:0/3131747137 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5570911addb0 0x55709123be20 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:29.080+0000 7f45db3ba740 1 -- 172.21.15.100:0/3131747137 >> 172.21.15.100:0/3131747137 conn(0x5570911a9420 msgr2=0x5570911a9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:34:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:29.080+0000 7f45db3ba740 1 -- 172.21.15.100:0/3131747137 shutdown_connections 2024-05-10T11:34:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:29.080+0000 7f45db3ba740 1 -- 172.21.15.100:0/3131747137 wait complete. 2024-05-10T11:34:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:29.081+0000 7f45db3ba740 20 client.27932 trim_cache size 0 max 16384 2024-05-10T11:34:29.083 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:34:29.083 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:34:29.083 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:34:29.083 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:34:29.084 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:29.081+0000 7f45d8699640 1 -- 172.21.15.100:0/934999884 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f45bc0b2920 msgr2=0x7f45bc0b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:29.084 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:29.081+0000 7f45d8699640 1 --2- 172.21.15.100:0/934999884 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f45bc0b2920 0x7f45bc0b4de0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f45cc000f80 tx=0x7f45cc031000 comp rx=0 tx=0).stop 2024-05-10T11:34:29.084 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:29.081+0000 7f45d8699640 1 -- 172.21.15.100:0/934999884 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45d01034d0 msgr2=0x7f45d0102e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:29.084 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:29.081+0000 7f45d8699640 1 --2- 172.21.15.100:0/934999884 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45d01034d0 0x7f45d0102e40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f45c400e9f0 tx=0x7f45c400edf0 comp rx=0 tx=0).stop 2024-05-10T11:34:29.084 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:29.081+0000 7f45d8699640 1 -- 172.21.15.100:0/934999884 shutdown_connections 2024-05-10T11:34:29.084 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:29.081+0000 7f45d8699640 1 --2- 172.21.15.100:0/934999884 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f45bc0b2920 0x7f45bc0b4de0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:29.084 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:29.081+0000 7f45d8699640 1 --2- 172.21.15.100:0/934999884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45d0108730 0x7f45d01019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:29.084 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:29.081+0000 7f45d8699640 1 --2- 172.21.15.100:0/934999884 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45d0105e00 0x7f45d0101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:29.084 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:29.081+0000 7f45d8699640 1 --2- 172.21.15.100:0/934999884 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45d01034d0 0x7f45d0102e40 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:29.084 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:29.081+0000 7f45d8699640 1 -- 172.21.15.100:0/934999884 >> 172.21.15.100:0/934999884 conn(0x7f45d00fce80 msgr2=0x7f45d0104050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:34:29.085 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:29.081+0000 7f45d8699640 1 -- 172.21.15.100:0/934999884 shutdown_connections 2024-05-10T11:34:29.085 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:29.081+0000 7f45d8699640 1 -- 172.21.15.100:0/934999884 wait complete. 2024-05-10T11:34:29.915 INFO:tasks.workunit.client.0.smithi100.stdout: 10 24557 9.95 MB/sec execute 23 sec latency 97.821 ms 2024-05-10T11:34:30.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:34:29 smithi178 ceph-mon[33359]: pgmap v904: 129 pgs: 129 active+clean; 407 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 4.1 MiB/s wr, 275 op/s 2024-05-10T11:34:30.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:34:29 smithi188 ceph-mon[34075]: pgmap v904: 129 pgs: 129 active+clean; 407 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 4.1 MiB/s wr, 275 op/s 2024-05-10T11:34:30.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:34:29 smithi100 ceph-mon[30296]: pgmap v904: 129 pgs: 129 active+clean; 407 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 4.1 MiB/s wr, 275 op/s 2024-05-10T11:34:30.915 INFO:tasks.workunit.client.0.smithi100.stdout: 10 24716 9.88 MB/sec execute 24 sec latency 67.144 ms 2024-05-10T11:34:31.915 INFO:tasks.workunit.client.0.smithi100.stdout: 10 24902 9.97 MB/sec execute 25 sec latency 78.139 ms 2024-05-10T11:34:32.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:34:31 smithi178 ceph-mon[33359]: pgmap v905: 129 pgs: 129 active+clean; 406 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.5 MiB/s wr, 307 op/s 2024-05-10T11:34:32.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:34:31 smithi188 ceph-mon[34075]: pgmap v905: 129 pgs: 129 active+clean; 406 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.5 MiB/s wr, 307 op/s 2024-05-10T11:34:32.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:34:31 smithi100 ceph-mon[30296]: pgmap v905: 129 pgs: 129 active+clean; 406 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.5 MiB/s wr, 307 op/s 2024-05-10T11:34:32.915 INFO:tasks.workunit.client.0.smithi100.stdout: 10 25088 10.02 MB/sec execute 26 sec latency 56.388 ms 2024-05-10T11:34:33.920 INFO:tasks.workunit.client.0.smithi100.stdout: 10 25248 9.99 MB/sec execute 27 sec latency 87.880 ms 2024-05-10T11:34:34.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:34:33 smithi178 ceph-mon[33359]: pgmap v906: 129 pgs: 129 active+clean; 409 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 4.4 MiB/s wr, 303 op/s 2024-05-10T11:34:34.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:34:33 smithi188 ceph-mon[34075]: pgmap v906: 129 pgs: 129 active+clean; 409 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 4.4 MiB/s wr, 303 op/s 2024-05-10T11:34:34.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:34:33 smithi100 ceph-mon[30296]: pgmap v906: 129 pgs: 129 active+clean; 409 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 4.4 MiB/s wr, 303 op/s 2024-05-10T11:34:34.915 INFO:tasks.workunit.client.0.smithi100.stdout: 10 25431 9.86 MB/sec execute 28 sec latency 74.584 ms 2024-05-10T11:34:35.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:34:34 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:34:35.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:34:34 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:34:35.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:34:34 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:34:35.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:34:34 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:34:35.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:34:34 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:34:35.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:34:34 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:34:35.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:34:34 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:34:35.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:34:34 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:34:35.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:34:34 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:34:35.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:34:34 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:34:35.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:34:34 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:34:35.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:34:34 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:34:35.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:34:34 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:34:35.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:34:34 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:34:35.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:34:34 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:34:35.915 INFO:tasks.workunit.client.0.smithi100.stdout: 10 25591 9.80 MB/sec execute 29 sec latency 93.464 ms 2024-05-10T11:34:36.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:34:35 smithi178 ceph-mon[33359]: pgmap v907: 129 pgs: 129 active+clean; 411 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 4.4 MiB/s wr, 263 op/s 2024-05-10T11:34:36.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:34:35 smithi178 ceph-mon[33359]: Adjusting osd_memory_target on smithi188 to 176.1M 2024-05-10T11:34:36.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:34:35 smithi178 ceph-mon[33359]: Unable to set osd_memory_target on smithi188 to 184753049: error parsing value: Value '184753049' is below minimum 939524096 2024-05-10T11:34:36.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:34:35 smithi188 ceph-mon[34075]: pgmap v907: 129 pgs: 129 active+clean; 411 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 4.4 MiB/s wr, 263 op/s 2024-05-10T11:34:36.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:34:35 smithi188 ceph-mon[34075]: Adjusting osd_memory_target on smithi188 to 176.1M 2024-05-10T11:34:36.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:34:35 smithi188 ceph-mon[34075]: Unable to set osd_memory_target on smithi188 to 184753049: error parsing value: Value '184753049' is below minimum 939524096 2024-05-10T11:34:36.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:34:35 smithi100 ceph-mon[30296]: pgmap v907: 129 pgs: 129 active+clean; 411 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 4.4 MiB/s wr, 263 op/s 2024-05-10T11:34:36.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:34:35 smithi100 ceph-mon[30296]: Adjusting osd_memory_target on smithi188 to 176.1M 2024-05-10T11:34:36.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:34:35 smithi100 ceph-mon[30296]: Unable to set osd_memory_target on smithi188 to 184753049: error parsing value: Value '184753049' is below minimum 939524096 2024-05-10T11:34:36.915 INFO:tasks.workunit.client.0.smithi100.stdout: 10 25757 9.74 MB/sec execute 30 sec latency 98.884 ms 2024-05-10T11:34:37.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:34:36 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:34:37.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:34:36 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:34:37.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:34:36 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:34:37.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:34:36 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:34:37.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:34:36 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:34:37.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:34:36 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:34:37.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:34:36 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:34:37.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:34:36 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:34:37.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:34:36 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:34:37.915 INFO:tasks.workunit.client.0.smithi100.stdout: 10 25979 9.94 MB/sec execute 31 sec latency 59.308 ms 2024-05-10T11:34:38.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:34:37 smithi178 ceph-mon[33359]: pgmap v908: 129 pgs: 129 active+clean; 417 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 943 KiB/s rd, 5.1 MiB/s wr, 326 op/s 2024-05-10T11:34:38.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:34:37 smithi188 ceph-mon[34075]: pgmap v908: 129 pgs: 129 active+clean; 417 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 943 KiB/s rd, 5.1 MiB/s wr, 326 op/s 2024-05-10T11:34:38.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:34:37 smithi100 ceph-mon[30296]: pgmap v908: 129 pgs: 129 active+clean; 417 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 943 KiB/s rd, 5.1 MiB/s wr, 326 op/s 2024-05-10T11:34:38.915 INFO:tasks.workunit.client.0.smithi100.stdout: 10 26149 10.00 MB/sec execute 32 sec latency 70.226 ms 2024-05-10T11:34:39.915 INFO:tasks.workunit.client.0.smithi100.stdout: 10 26337 10.05 MB/sec execute 33 sec latency 73.797 ms 2024-05-10T11:34:40.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:34:39 smithi178 ceph-mon[33359]: pgmap v909: 129 pgs: 129 active+clean; 418 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 4.7 MiB/s wr, 259 op/s 2024-05-10T11:34:40.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:34:39 smithi188 ceph-mon[34075]: pgmap v909: 129 pgs: 129 active+clean; 418 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 4.7 MiB/s wr, 259 op/s 2024-05-10T11:34:40.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:34:39 smithi100 ceph-mon[30296]: pgmap v909: 129 pgs: 129 active+clean; 418 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 4.7 MiB/s wr, 259 op/s 2024-05-10T11:34:40.915 INFO:tasks.workunit.client.0.smithi100.stdout: 10 26529 9.90 MB/sec execute 34 sec latency 59.410 ms 2024-05-10T11:34:41.328 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T11:34:41.915 INFO:tasks.workunit.client.0.smithi100.stdout: 10 26691 9.75 MB/sec execute 35 sec latency 64.170 ms 2024-05-10T11:34:42.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:34:41 smithi178 ceph-mon[33359]: pgmap v910: 129 pgs: 129 active+clean; 418 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 4.9 MiB/s wr, 291 op/s 2024-05-10T11:34:42.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:34:41 smithi188 ceph-mon[34075]: pgmap v910: 129 pgs: 129 active+clean; 418 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 4.9 MiB/s wr, 291 op/s 2024-05-10T11:34:42.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:34:41 smithi100 ceph-mon[30296]: pgmap v910: 129 pgs: 129 active+clean; 418 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 4.9 MiB/s wr, 291 op/s 2024-05-10T11:34:42.489 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.489+0000 7f268d138640 1 -- 172.21.15.100:0/2858034517 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f26881034f0 msgr2=0x7f26881058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:42.490 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.489+0000 7f268d138640 1 --2- 172.21.15.100:0/2858034517 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f26881034f0 0x7f26881058e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2678009860 tx=0x7f267802a340 comp rx=0 tx=0).stop 2024-05-10T11:34:42.490 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.489+0000 7f268d138640 1 -- 172.21.15.100:0/2858034517 shutdown_connections 2024-05-10T11:34:42.490 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.489+0000 7f268d138640 1 --2- 172.21.15.100:0/2858034517 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2688108750 0x7f268810ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:42.490 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.489+0000 7f268d138640 1 --2- 172.21.15.100:0/2858034517 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2688105e20 0x7f2688108210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:42.490 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.489+0000 7f268d138640 1 --2- 172.21.15.100:0/2858034517 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f26881034f0 0x7f26881058e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:42.490 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.489+0000 7f268d138640 1 -- 172.21.15.100:0/2858034517 >> 172.21.15.100:0/2858034517 conn(0x7f26880fce80 msgr2=0x7f26880ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:34:42.490 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.489+0000 7f268d138640 1 -- 172.21.15.100:0/2858034517 shutdown_connections 2024-05-10T11:34:42.490 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.489+0000 7f268d138640 1 -- 172.21.15.100:0/2858034517 wait complete. 2024-05-10T11:34:42.492 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.490+0000 7f268d138640 1 Processor -- start 2024-05-10T11:34:42.492 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.490+0000 7f268d138640 1 -- start start 2024-05-10T11:34:42.492 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.490+0000 7f268d138640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26881034f0 0x7f268819acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:42.492 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.491+0000 7f268d138640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2688105e20 0x7f268819b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:42.493 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.491+0000 7f268d138640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2688108750 0x7f26881a2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:42.493 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.491+0000 7f268d138640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f268810e3e0 con 0x7f26881034f0 2024-05-10T11:34:42.493 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.491+0000 7f268d138640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f268810e260 con 0x7f2688105e20 2024-05-10T11:34:42.493 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.491+0000 7f268d138640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f268810e560 con 0x7f2688108750 2024-05-10T11:34:42.493 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.491+0000 7f2686d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26881034f0 0x7f268819acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:42.493 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.491+0000 7f2686d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26881034f0 0x7f268819acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56400/0 (socket says 172.21.15.100:56400) 2024-05-10T11:34:42.493 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.491+0000 7f2686d9d640 1 -- 172.21.15.100:0/1834187193 learned_addr learned my addr 172.21.15.100:0/1834187193 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:34:42.493 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.491+0000 7f268659c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2688105e20 0x7f268819b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:42.494 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.491+0000 7f268759e640 1 --2- 172.21.15.100:0/1834187193 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2688108750 0x7f26881a2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:42.494 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.491+0000 7f2686d9d640 1 -- 172.21.15.100:0/1834187193 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2688108750 msgr2=0x7f26881a2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:42.494 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.491+0000 7f2686d9d640 1 --2- 172.21.15.100:0/1834187193 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2688108750 0x7f26881a2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:42.494 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.491+0000 7f2686d9d640 1 -- 172.21.15.100:0/1834187193 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2688105e20 msgr2=0x7f268819b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:42.494 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.491+0000 7f2686d9d640 1 --2- 172.21.15.100:0/1834187193 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2688105e20 0x7f268819b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:42.494 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.491+0000 7f2686d9d640 1 -- 172.21.15.100:0/1834187193 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26881a2900 con 0x7f26881034f0 2024-05-10T11:34:42.494 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.491+0000 7f268659c640 1 --2- 172.21.15.100:0/1834187193 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2688105e20 0x7f268819b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:34:42.494 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.491+0000 7f268759e640 1 --2- 172.21.15.100:0/1834187193 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2688108750 0x7f26881a2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:34:42.494 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.491+0000 7f2686d9d640 1 --2- 172.21.15.100:0/1834187193 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26881034f0 0x7f268819acf0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f267802a860 tx=0x7f267802ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:34:42.494 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.492+0000 7f266ffff640 1 -- 172.21.15.100:0/1834187193 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2678037040 con 0x7f26881034f0 2024-05-10T11:34:42.494 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.492+0000 7f268d138640 1 -- 172.21.15.100:0/1834187193 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26881a2b90 con 0x7f26881034f0 2024-05-10T11:34:42.494 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.492+0000 7f266ffff640 1 -- 172.21.15.100:0/1834187193 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2678041070 con 0x7f26881034f0 2024-05-10T11:34:42.494 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.492+0000 7f266ffff640 1 -- 172.21.15.100:0/1834187193 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26780052b0 con 0x7f26881034f0 2024-05-10T11:34:42.495 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.492+0000 7f268d138640 1 -- 172.21.15.100:0/1834187193 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26881a3070 con 0x7f26881034f0 2024-05-10T11:34:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.495+0000 7f266ffff640 1 -- 172.21.15.100:0/1834187193 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f2678005450 con 0x7f26881034f0 2024-05-10T11:34:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.495+0000 7f268fe52740 1 Processor -- start 2024-05-10T11:34:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.495+0000 7f268fe52740 1 -- start start 2024-05-10T11:34:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.495+0000 7f268fe52740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56533674cdb0 0x56533674d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.495+0000 7f268fe52740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56533674d7a0 0x5653367522a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.495+0000 7f268fe52740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565336752910 0x565336754da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.495+0000 7f268fe52740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5653366b15d0 con 0x565336752910 2024-05-10T11:34:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.495+0000 7f268fe52740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56533666a890 con 0x56533674d7a0 2024-05-10T11:34:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.495+0000 7f268fe52740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5653366ad590 con 0x56533674cdb0 2024-05-10T11:34:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.495+0000 7f268759e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56533674cdb0 0x56533674d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.495+0000 7f268659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565336752910 0x565336754da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:42.502 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.495+0000 7f2686d9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56533674d7a0 0x5653367522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:42.502 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.495+0000 7f268759e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56533674cdb0 0x56533674d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:34796/0 (socket says 172.21.15.100:34796) 2024-05-10T11:34:42.502 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.495+0000 7f268759e640 1 -- 172.21.15.100:0/3079664794 learned_addr learned my addr 172.21.15.100:0/3079664794 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:34:42.502 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.495+0000 7f268659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565336752910 0x565336754da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56412/0 (socket says 172.21.15.100:56412) 2024-05-10T11:34:42.502 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.496+0000 7f268759e640 1 -- 172.21.15.100:0/3079664794 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56533674d7a0 msgr2=0x5653367522a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:42.502 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.496+0000 7f268759e640 1 --2- 172.21.15.100:0/3079664794 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56533674d7a0 0x5653367522a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:42.502 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.496+0000 7f268759e640 1 -- 172.21.15.100:0/3079664794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565336752910 msgr2=0x565336754da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:42.502 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.496+0000 7f268759e640 1 --2- 172.21.15.100:0/3079664794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565336752910 0x565336754da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:42.502 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.496+0000 7f268759e640 1 -- 172.21.15.100:0/3079664794 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653363c3210 con 0x56533674cdb0 2024-05-10T11:34:42.502 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.496+0000 7f268659c640 1 --2- 172.21.15.100:0/3079664794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565336752910 0x565336754da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:34:42.502 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.496+0000 7f266ffff640 1 --2- 172.21.15.100:0/1834187193 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f26700b28a0 0x7f26700b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:42.502 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.496+0000 7f268759e640 1 --2- 172.21.15.100:0/3079664794 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56533674cdb0 0x56533674d1d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f26800117e0 tx=0x7f2680011ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3136908ba5bde09a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:34:42.503 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.496+0000 7f2686d9d640 1 --2- 172.21.15.100:0/1834187193 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f26700b28a0 0x7f26700b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:42.503 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.496+0000 7f266ffff640 1 -- 172.21.15.100:0/1834187193 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f26781024f0 con 0x7f26881034f0 2024-05-10T11:34:42.503 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.496+0000 7f266d7fa640 1 -- 172.21.15.100:0/3079664794 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2680011310 con 0x56533674cdb0 2024-05-10T11:34:42.503 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.496+0000 7f266d7fa640 1 -- 172.21.15.100:0/3079664794 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2680003e00 con 0x56533674cdb0 2024-05-10T11:34:42.503 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.496+0000 7f266d7fa640 1 -- 172.21.15.100:0/3079664794 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2680004430 con 0x56533674cdb0 2024-05-10T11:34:42.503 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.497+0000 7f2686d9d640 1 --2- 172.21.15.100:0/1834187193 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f26700b28a0 0x7f26700b4d60 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x56533674e210 tx=0x7f26780d2040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:34:42.503 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.497+0000 7f268fe52740 1 -- 172.21.15.100:0/3079664794 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56533674cdb0 msgr2=0x56533674d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:42.503 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.497+0000 7f268fe52740 1 --2- 172.21.15.100:0/3079664794 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56533674cdb0 0x56533674d1d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f26800117e0 tx=0x7f2680011ca0 comp rx=0 tx=0).stop 2024-05-10T11:34:42.503 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.497+0000 7f268fe52740 1 -- 172.21.15.100:0/3079664794 shutdown_connections 2024-05-10T11:34:42.503 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.498+0000 7f268fe52740 1 --2- 172.21.15.100:0/3079664794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565336752910 0x565336754da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:42.503 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.498+0000 7f268fe52740 1 --2- 172.21.15.100:0/3079664794 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56533674d7a0 0x5653367522a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:42.503 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.498+0000 7f268fe52740 1 --2- 172.21.15.100:0/3079664794 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56533674cdb0 0x56533674d1d0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:42.503 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.498+0000 7f268fe52740 1 -- 172.21.15.100:0/3079664794 >> 172.21.15.100:0/3079664794 conn(0x565336748420 msgr2=0x565336748820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:34:42.504 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.498+0000 7f268fe52740 1 -- 172.21.15.100:0/3079664794 shutdown_connections 2024-05-10T11:34:42.504 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.498+0000 7f268fe52740 1 -- 172.21.15.100:0/3079664794 wait complete. 2024-05-10T11:34:42.504 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.498+0000 7f268fe52740 1 -- start start 2024-05-10T11:34:42.504 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.498+0000 7f268fe52740 1 Processor -- start 2024-05-10T11:34:42.504 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.498+0000 7f268fe52740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56533674cdb0 0x565336757060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:42.504 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.498+0000 7f268fe52740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56533674d7a0 0x5653367575a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:42.504 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.498+0000 7f268fe52740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565336752910 0x56533675e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:42.504 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.498+0000 7f268fe52740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5653367558c0 con 0x56533674cdb0 2024-05-10T11:34:42.504 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.498+0000 7f268fe52740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56533666e370 con 0x56533674d7a0 2024-05-10T11:34:42.504 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.498+0000 7f268fe52740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x565336755a40 con 0x565336752910 2024-05-10T11:34:42.504 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.498+0000 7f268fe52740 10 client.0 _ll_register_callbacks cb 0x5653363c2a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:34:42.505 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.498+0000 7f268759e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56533674cdb0 0x565336757060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:42.505 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.498+0000 7f268759e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56533674cdb0 0x565336757060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56426/0 (socket says 172.21.15.100:56426) 2024-05-10T11:34:42.505 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.498+0000 7f268759e640 1 -- 172.21.15.100:0/788163891 learned_addr learned my addr 172.21.15.100:0/788163891 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:34:42.505 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.498+0000 7f268659c640 1 --2- 172.21.15.100:0/788163891 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56533674d7a0 0x5653367575a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:42.505 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.498+0000 7f268759e640 1 --2- 172.21.15.100:0/788163891 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565336752910 0x56533675e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:42.505 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.499+0000 7f268759e640 1 -- 172.21.15.100:0/788163891 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565336752910 msgr2=0x56533675e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:42.506 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.499+0000 7f268759e640 1 --2- 172.21.15.100:0/788163891 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565336752910 0x56533675e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:42.506 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.499+0000 7f268759e640 1 -- 172.21.15.100:0/788163891 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56533674d7a0 msgr2=0x5653367575a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:42.506 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.499+0000 7f268759e640 1 --2- 172.21.15.100:0/788163891 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56533674d7a0 0x5653367575a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:42.506 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.499+0000 7f268759e640 1 -- 172.21.15.100:0/788163891 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56533648f620 con 0x56533674cdb0 2024-05-10T11:34:42.506 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.499+0000 7f268659c640 1 --2- 172.21.15.100:0/788163891 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56533674d7a0 0x5653367575a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:34:42.506 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.499+0000 7f268759e640 1 --2- 172.21.15.100:0/788163891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56533674cdb0 0x565336757060 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f268000a010 tx=0x7f2680009e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75ca250bfd659a89 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:34:42.506 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.499+0000 7f266aff5640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:34:42.506 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.499+0000 7f266aff5640 1 -- 172.21.15.100:0/788163891 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2680020050 con 0x56533674cdb0 2024-05-10T11:34:42.506 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.499+0000 7f266aff5640 1 -- 172.21.15.100:0/788163891 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f2680004a40 con 0x56533674cdb0 2024-05-10T11:34:42.507 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.499+0000 7f268fe52740 1 -- 172.21.15.100:0/788163891 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2680003e00 con 0x56533674cdb0 2024-05-10T11:34:42.507 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.499+0000 7f266aff5640 1 -- 172.21.15.100:0/788163891 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f2680032050 con 0x56533674cdb0 2024-05-10T11:34:42.507 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.499+0000 7f268fe52740 10 client.18627 fetch_fsmap learned FSMap version 38 2024-05-10T11:34:42.507 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.499+0000 7f268fe52740 1 -- 172.21.15.100:0/788163891 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x565336761630 con 0x56533674cdb0 2024-05-10T11:34:42.507 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.500+0000 7f266aff5640 1 -- 172.21.15.100:0/788163891 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2680003e00 con 0x56533674cdb0 2024-05-10T11:34:42.507 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.500+0000 7f266aff5640 1 -- 172.21.15.100:0/788163891 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7f2680018020 con 0x56533674cdb0 2024-05-10T11:34:42.507 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.500+0000 7f268fe52740 10 client.18627 fetch_fsmap finished waiting for FSMap version 38 2024-05-10T11:34:42.507 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.500+0000 7f268fe52740 10 client.18627 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:34:42.508 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.500+0000 7f268fe52740 20 client.18627 populate_metadata read hostname 'smithi100' 2024-05-10T11:34:42.508 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.500+0000 7f268fe52740 1 --2- 172.21.15.100:0/788163891 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x565336762af0 0x565336764f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:42.508 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.500+0000 7f268fe52740 4 client.18627 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:34:42.508 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.500+0000 7f268fe52740 1 -- 172.21.15.100:0/788163891 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653366bf1e0 con 0x565336762af0 2024-05-10T11:34:42.508 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.500+0000 7f268659c640 1 --2- 172.21.15.100:0/788163891 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x565336762af0 0x565336764f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:42.508 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.501+0000 7f266aff5640 10 client.18627 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:34:42.508 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.501+0000 7f268659c640 1 --2- 172.21.15.100:0/788163891 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x565336762af0 0x565336764f10 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f23a527a5d145183 server_cookie=c3e7c266397bbcec in_seq=0 out_seq=0 2024-05-10T11:34:42.510 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.502+0000 7f266aff5640 1 -- 172.21.15.100:0/788163891 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5653366bf1e0 con 0x565336762af0 2024-05-10T11:34:42.510 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.502+0000 7f266aff5640 10 client.18627 handle_command_reply: tid=0 2024-05-10T11:34:42.510 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.502+0000 7f268fe52740 1 client.18627 shutdown 2024-05-10T11:34:42.510 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.502+0000 7f268fe52740 1 -- 172.21.15.100:0/788163891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56533674cdb0 msgr2=0x565336757060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:42.510 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.502+0000 7f268fe52740 1 --2- 172.21.15.100:0/788163891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56533674cdb0 0x565336757060 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f268000a010 tx=0x7f2680009e00 comp rx=0 tx=0).stop 2024-05-10T11:34:42.511 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.502+0000 7f268fe52740 1 -- 172.21.15.100:0/788163891 shutdown_connections 2024-05-10T11:34:42.511 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.502+0000 7f268fe52740 1 --2- 172.21.15.100:0/788163891 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x565336762af0 0x565336764f10 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:42.511 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.502+0000 7f266aff5640 0 client.18627 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:34:42.511 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.502+0000 7f268fe52740 1 --2- 172.21.15.100:0/788163891 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565336752910 0x56533675e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:42.511 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.502+0000 7f268fe52740 1 --2- 172.21.15.100:0/788163891 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56533674d7a0 0x5653367575a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:42.511 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.502+0000 7f268fe52740 1 --2- 172.21.15.100:0/788163891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56533674cdb0 0x565336757060 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:42.511 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.502+0000 7f268fe52740 1 -- 172.21.15.100:0/788163891 >> 172.21.15.100:0/788163891 conn(0x565336748420 msgr2=0x565336747580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:34:42.511 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.502+0000 7f268fe52740 1 -- 172.21.15.100:0/788163891 shutdown_connections 2024-05-10T11:34:42.511 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.502+0000 7f268fe52740 1 -- 172.21.15.100:0/788163891 wait complete. 2024-05-10T11:34:42.512 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.502+0000 7f268fe52740 20 client.18627 trim_cache size 0 max 16384 2024-05-10T11:34:42.532 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.530+0000 7f268fe52740 1 Processor -- start 2024-05-10T11:34:42.532 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.531+0000 7f268fe52740 1 -- start start 2024-05-10T11:34:42.532 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.531+0000 7f268fe52740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56533674cdb0 0x565336757500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:42.532 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.531+0000 7f268fe52740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56533674d7a0 0x56533674ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:42.532 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.531+0000 7f268fe52740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565336752910 0x5653367697d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:42.532 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.531+0000 7f268fe52740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f26881aff00 con 0x565336752910 2024-05-10T11:34:42.533 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.531+0000 7f268fe52740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f26881ad740 con 0x56533674cdb0 2024-05-10T11:34:42.533 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.531+0000 7f268fe52740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f26881ab590 con 0x56533674d7a0 2024-05-10T11:34:42.533 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.532+0000 7f268759e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56533674cdb0 0x565336757500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:42.533 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.532+0000 7f268659c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56533674d7a0 0x56533674ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:42.533 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.532+0000 7f268759e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56533674cdb0 0x565336757500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:39952/0 (socket says 172.21.15.100:39952) 2024-05-10T11:34:42.533 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.532+0000 7f268759e640 1 -- 172.21.15.100:0/575341197 learned_addr learned my addr 172.21.15.100:0/575341197 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:34:42.533 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.532+0000 7f268759e640 1 --2- 172.21.15.100:0/575341197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565336752910 0x5653367697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:42.534 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.532+0000 7f268659c640 1 -- 172.21.15.100:0/575341197 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56533674cdb0 msgr2=0x565336757500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:42.534 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.532+0000 7f268659c640 1 --2- 172.21.15.100:0/575341197 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56533674cdb0 0x565336757500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:42.534 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.532+0000 7f268659c640 1 -- 172.21.15.100:0/575341197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565336752910 msgr2=0x5653367697d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:42.534 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.532+0000 7f268659c640 1 --2- 172.21.15.100:0/575341197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565336752910 0x5653367697d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:42.534 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.532+0000 7f268659c640 1 -- 172.21.15.100:0/575341197 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565336748250 con 0x56533674d7a0 2024-05-10T11:34:42.534 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.533+0000 7f268659c640 1 --2- 172.21.15.100:0/575341197 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56533674d7a0 0x56533674ba40 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x565336765780 tx=0x7f267c009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93b4f9aace150f7c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:34:42.534 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.533+0000 7f266cff9640 1 -- 172.21.15.100:0/575341197 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267c00fd30 con 0x56533674d7a0 2024-05-10T11:34:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.533+0000 7f266cff9640 1 -- 172.21.15.100:0/575341197 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f267c005020 con 0x56533674d7a0 2024-05-10T11:34:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.533+0000 7f266cff9640 1 -- 172.21.15.100:0/575341197 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267c004a00 con 0x56533674d7a0 2024-05-10T11:34:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.534+0000 7f268fe52740 1 -- 172.21.15.100:0/575341197 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56533674d7a0 msgr2=0x56533674ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.534+0000 7f268fe52740 1 --2- 172.21.15.100:0/575341197 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56533674d7a0 0x56533674ba40 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x565336765780 tx=0x7f267c009260 comp rx=0 tx=0).stop 2024-05-10T11:34:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.534+0000 7f268fe52740 1 -- 172.21.15.100:0/575341197 shutdown_connections 2024-05-10T11:34:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.534+0000 7f268fe52740 1 --2- 172.21.15.100:0/575341197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565336752910 0x5653367697d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.534+0000 7f268fe52740 1 --2- 172.21.15.100:0/575341197 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56533674d7a0 0x56533674ba40 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.534+0000 7f268fe52740 1 --2- 172.21.15.100:0/575341197 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56533674cdb0 0x565336757500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:42.536 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.534+0000 7f268fe52740 1 -- 172.21.15.100:0/575341197 >> 172.21.15.100:0/575341197 conn(0x565336748420 msgr2=0x56533674b330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:34:42.536 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.536+0000 7f268fe52740 1 -- 172.21.15.100:0/575341197 shutdown_connections 2024-05-10T11:34:42.536 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.536+0000 7f268fe52740 1 -- 172.21.15.100:0/575341197 wait complete. 2024-05-10T11:34:42.536 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.536+0000 7f268fe52740 1 -- start start 2024-05-10T11:34:42.536 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.537+0000 7f268fe52740 1 Processor -- start 2024-05-10T11:34:42.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.537+0000 7f268fe52740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56533674cdb0 0x56533675b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:42.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.537+0000 7f268fe52740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56533674d7a0 0x56533675ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:42.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.537+0000 7f268fe52740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565336752910 0x56533675ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:42.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.537+0000 7f268fe52740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f26881abc00 con 0x565336752910 2024-05-10T11:34:42.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.537+0000 7f268fe52740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f26881ad070 con 0x56533674d7a0 2024-05-10T11:34:42.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.537+0000 7f268fe52740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f26881ac2d0 con 0x56533674cdb0 2024-05-10T11:34:42.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.537+0000 7f268fe52740 10 client.0 _ll_register_callbacks cb 0x7f267c009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:34:42.538 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.537+0000 7f268759e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56533674cdb0 0x56533675b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:42.538 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.537+0000 7f268759e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56533674cdb0 0x56533675b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:34830/0 (socket says 172.21.15.100:34830) 2024-05-10T11:34:42.538 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.537+0000 7f268759e640 1 -- 172.21.15.100:0/2046093589 learned_addr learned my addr 172.21.15.100:0/2046093589 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:34:42.538 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.538+0000 7f268659c640 1 --2- 172.21.15.100:0/2046093589 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56533674d7a0 0x56533675ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:42.538 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.538+0000 7f268759e640 1 --2- 172.21.15.100:0/2046093589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565336752910 0x56533675ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:42.538 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.538+0000 7f268759e640 1 -- 172.21.15.100:0/2046093589 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56533674d7a0 msgr2=0x56533675ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:42.538 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.538+0000 7f268759e640 1 --2- 172.21.15.100:0/2046093589 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56533674d7a0 0x56533675ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:42.538 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.538+0000 7f268759e640 1 -- 172.21.15.100:0/2046093589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565336752910 msgr2=0x56533675ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:42.538 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.538+0000 7f268759e640 1 --2- 172.21.15.100:0/2046093589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565336752910 0x56533675ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:42.539 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.538+0000 7f268759e640 1 -- 172.21.15.100:0/2046093589 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56533675d030 con 0x56533674cdb0 2024-05-10T11:34:42.540 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.540+0000 7f268759e640 1 --2- 172.21.15.100:0/2046093589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56533674cdb0 0x56533675b540 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2680003970 tx=0x7f2680009e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f8182242109107e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:34:42.540 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.540+0000 7f266aff5640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:34:42.540 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.540+0000 7f266aff5640 1 -- 172.21.15.100:0/2046093589 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2680020040 con 0x56533674cdb0 2024-05-10T11:34:42.540 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.540+0000 7f266aff5640 1 -- 172.21.15.100:0/2046093589 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f268000fc30 con 0x56533674cdb0 2024-05-10T11:34:42.540 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.541+0000 7f268fe52740 1 -- 172.21.15.100:0/2046093589 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f267c005020 con 0x56533674cdb0 2024-05-10T11:34:42.542 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.542+0000 7f266aff5640 1 -- 172.21.15.100:0/2046093589 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f2680023040 con 0x56533674cdb0 2024-05-10T11:34:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.544+0000 7f268fe52740 10 client.28048 fetch_fsmap learned FSMap version 38 2024-05-10T11:34:42.544 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.544+0000 7f268fe52740 1 -- 172.21.15.100:0/2046093589 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5653367cd560 con 0x56533674cdb0 2024-05-10T11:34:42.544 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.544+0000 7f266aff5640 1 -- 172.21.15.100:0/2046093589 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f267c005020 con 0x56533674cdb0 2024-05-10T11:34:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.545+0000 7f266aff5640 1 -- 172.21.15.100:0/2046093589 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7f2680018070 con 0x56533674cdb0 2024-05-10T11:34:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.545+0000 7f268fe52740 10 client.28048 fetch_fsmap finished waiting for FSMap version 38 2024-05-10T11:34:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.545+0000 7f268fe52740 10 client.28048 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:34:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.545+0000 7f268fe52740 20 client.28048 populate_metadata read hostname 'smithi100' 2024-05-10T11:34:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.545+0000 7f268fe52740 1 --2- 172.21.15.100:0/2046093589 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x565336762af0 0x5653367cff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.545+0000 7f268fe52740 4 client.28048 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:34:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.545+0000 7f268fe52740 1 -- 172.21.15.100:0/2046093589 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5653366b9050 con 0x565336762af0 2024-05-10T11:34:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.545+0000 7f268659c640 1 --2- 172.21.15.100:0/2046093589 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x565336762af0 0x5653367cff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:42.547 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.546+0000 7f268659c640 1 --2- 172.21.15.100:0/2046093589 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x565336762af0 0x5653367cff80 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2782125f7b00d9ee server_cookie=72856e30e1f6aae1 in_seq=0 out_seq=0 2024-05-10T11:34:42.547 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.546+0000 7f266aff5640 10 client.28048 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:34:42.641 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.640+0000 7f266aff5640 1 -- 172.21.15.100:0/2046093589 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5653366b9050 con 0x565336762af0 2024-05-10T11:34:42.641 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.641+0000 7f266aff5640 10 client.28048 handle_command_reply: tid=0 2024-05-10T11:34:42.642 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.641+0000 7f268fe52740 1 client.28048 shutdown 2024-05-10T11:34:42.642 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.641+0000 7f268fe52740 1 -- 172.21.15.100:0/2046093589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56533674cdb0 msgr2=0x56533675b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:42.642 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.641+0000 7f268fe52740 1 --2- 172.21.15.100:0/2046093589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56533674cdb0 0x56533675b540 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2680003970 tx=0x7f2680009e00 comp rx=0 tx=0).stop 2024-05-10T11:34:42.642 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.641+0000 7f268fe52740 1 -- 172.21.15.100:0/2046093589 shutdown_connections 2024-05-10T11:34:42.642 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.641+0000 7f268fe52740 1 --2- 172.21.15.100:0/2046093589 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x565336762af0 0x5653367cff80 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:42.642 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.641+0000 7f268fe52740 1 --2- 172.21.15.100:0/2046093589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565336752910 0x56533675ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:42.642 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.641+0000 7f268fe52740 1 --2- 172.21.15.100:0/2046093589 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56533674d7a0 0x56533675ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:42.642 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.641+0000 7f268fe52740 1 --2- 172.21.15.100:0/2046093589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56533674cdb0 0x56533675b540 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:42.642 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.641+0000 7f268fe52740 1 -- 172.21.15.100:0/2046093589 >> 172.21.15.100:0/2046093589 conn(0x565336748420 msgr2=0x565336746cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:34:42.642 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.641+0000 7f266aff5640 0 client.28048 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:34:42.642 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.641+0000 7f268fe52740 1 -- 172.21.15.100:0/2046093589 shutdown_connections 2024-05-10T11:34:42.643 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.641+0000 7f268fe52740 1 -- 172.21.15.100:0/2046093589 wait complete. 2024-05-10T11:34:42.643 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.641+0000 7f268fe52740 20 client.28048 trim_cache size 0 max 16384 2024-05-10T11:34:42.643 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:34:42.643 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:34:42.643 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:34:42.643 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:34:42.644 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.642+0000 7f268d138640 1 -- 172.21.15.100:0/1834187193 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f26700b28a0 msgr2=0x7f26700b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:42.644 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.642+0000 7f268d138640 1 --2- 172.21.15.100:0/1834187193 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f26700b28a0 0x7f26700b4d60 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x56533674e210 tx=0x7f26780d2040 comp rx=0 tx=0).stop 2024-05-10T11:34:42.644 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.642+0000 7f268d138640 1 -- 172.21.15.100:0/1834187193 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26881034f0 msgr2=0x7f268819acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:42.645 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.642+0000 7f268d138640 1 --2- 172.21.15.100:0/1834187193 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26881034f0 0x7f268819acf0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f267802a860 tx=0x7f267802ad70 comp rx=0 tx=0).stop 2024-05-10T11:34:42.645 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.642+0000 7f268d138640 1 -- 172.21.15.100:0/1834187193 shutdown_connections 2024-05-10T11:34:42.645 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.642+0000 7f268d138640 1 --2- 172.21.15.100:0/1834187193 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f26700b28a0 0x7f26700b4d60 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:42.645 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.642+0000 7f268d138640 1 --2- 172.21.15.100:0/1834187193 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2688108750 0x7f26881a2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:42.645 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.642+0000 7f268d138640 1 --2- 172.21.15.100:0/1834187193 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2688105e20 0x7f268819b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:42.645 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.642+0000 7f268d138640 1 --2- 172.21.15.100:0/1834187193 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26881034f0 0x7f268819acf0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:42.645 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.642+0000 7f268d138640 1 -- 172.21.15.100:0/1834187193 >> 172.21.15.100:0/1834187193 conn(0x7f26880fce80 msgr2=0x7f2688102a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:34:42.646 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.642+0000 7f268d138640 1 -- 172.21.15.100:0/1834187193 shutdown_connections 2024-05-10T11:34:42.646 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:42.642+0000 7f268d138640 1 -- 172.21.15.100:0/1834187193 wait complete. 2024-05-10T11:34:42.915 INFO:tasks.workunit.client.0.smithi100.stdout: 10 26840 9.72 MB/sec execute 36 sec latency 147.788 ms 2024-05-10T11:34:43.212 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:34:43.915 INFO:tasks.workunit.client.0.smithi100.stdout: 10 26982 9.64 MB/sec execute 37 sec latency 89.703 ms 2024-05-10T11:34:44.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:34:43 smithi178 ceph-mon[33359]: pgmap v911: 129 pgs: 129 active+clean; 416 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 922 KiB/s rd, 4.5 MiB/s wr, 282 op/s 2024-05-10T11:34:44.004 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:34:43 smithi100 ceph-mon[30296]: pgmap v911: 129 pgs: 129 active+clean; 416 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 922 KiB/s rd, 4.5 MiB/s wr, 282 op/s 2024-05-10T11:34:44.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:34:43 smithi188 ceph-mon[34075]: pgmap v911: 129 pgs: 129 active+clean; 416 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 922 KiB/s rd, 4.5 MiB/s wr, 282 op/s 2024-05-10T11:34:44.915 INFO:tasks.workunit.client.0.smithi100.stdout: 10 27188 9.81 MB/sec execute 38 sec latency 63.027 ms 2024-05-10T11:34:45.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.557+0000 7f5805cf4700 1 -- 172.21.15.100:0/92322381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5800108410 msgr2=0x7f58001087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:45.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.557+0000 7f5805cf4700 1 --2- 172.21.15.100:0/92322381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5800108410 0x7f58001087f0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f57f0009990 tx=0x7f57f0009ca0 comp rx=0 tx=0).stop 2024-05-10T11:34:45.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.558+0000 7f5805cf4700 1 -- 172.21.15.100:0/92322381 shutdown_connections 2024-05-10T11:34:45.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.558+0000 7f5805cf4700 1 --2- 172.21.15.100:0/92322381 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5800102d60 0x7f580010f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:45.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.558+0000 7f5805cf4700 1 --2- 172.21.15.100:0/92322381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5800108410 0x7f58001087f0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:45.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.558+0000 7f5805cf4700 1 --2- 172.21.15.100:0/92322381 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58001023a0 0x7f5800102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:45.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.558+0000 7f5805cf4700 1 -- 172.21.15.100:0/92322381 >> 172.21.15.100:0/92322381 conn(0x7f5800079350 msgr2=0x7f5800079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:34:45.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.559+0000 7f5805cf4700 1 -- 172.21.15.100:0/92322381 shutdown_connections 2024-05-10T11:34:45.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.559+0000 7f5805cf4700 1 -- 172.21.15.100:0/92322381 wait complete. 2024-05-10T11:34:45.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.560+0000 7f5805cf4700 1 Processor -- start 2024-05-10T11:34:45.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.560+0000 7f5805cf4700 1 -- start start 2024-05-10T11:34:45.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.560+0000 7f5805cf4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58001023a0 0x7f5800117820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:45.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.560+0000 7f5805cf4700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5800102d60 0x7f58001118a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:45.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.561+0000 7f5805cf4700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5800108410 0x7f5800111de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:45.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.561+0000 7f5805cf4700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5800110120 con 0x7f58001023a0 2024-05-10T11:34:45.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.561+0000 7f5805cf4700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f580010ffa0 con 0x7f5800108410 2024-05-10T11:34:45.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.561+0000 7f5805cf4700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f58001102a0 con 0x7f5800102d60 2024-05-10T11:34:45.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.561+0000 7f57ff7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58001023a0 0x7f5800117820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:45.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.561+0000 7f57ff7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58001023a0 0x7f5800117820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56472/0 (socket says 172.21.15.100:56472) 2024-05-10T11:34:45.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.561+0000 7f57ff7fe700 1 -- 172.21.15.100:0/1424406592 learned_addr learned my addr 172.21.15.100:0/1424406592 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:34:45.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.561+0000 7f57feffd700 1 --2- 172.21.15.100:0/1424406592 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5800102d60 0x7f58001118a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:45.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.561+0000 7f57fffff700 1 --2- 172.21.15.100:0/1424406592 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5800108410 0x7f5800111de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:45.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.561+0000 7f57fffff700 1 -- 172.21.15.100:0/1424406592 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5800102d60 msgr2=0x7f58001118a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:45.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.561+0000 7f57fffff700 1 --2- 172.21.15.100:0/1424406592 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5800102d60 0x7f58001118a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:45.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.561+0000 7f57fffff700 1 -- 172.21.15.100:0/1424406592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58001023a0 msgr2=0x7f5800117820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:45.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.561+0000 7f57fffff700 1 --2- 172.21.15.100:0/1424406592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58001023a0 0x7f5800117820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:45.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.561+0000 7f57fffff700 1 -- 172.21.15.100:0/1424406592 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5800112450 con 0x7f5800108410 2024-05-10T11:34:45.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.562+0000 7f57fffff700 1 --2- 172.21.15.100:0/1424406592 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5800108410 0x7f5800111de0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f57f800bb00 tx=0x7f57f800be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:34:45.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.562+0000 7f57fcff9700 1 -- 172.21.15.100:0/1424406592 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f8019070 con 0x7f5800108410 2024-05-10T11:34:45.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.562+0000 7f57fcff9700 1 -- 172.21.15.100:0/1424406592 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f57f8014440 con 0x7f5800108410 2024-05-10T11:34:45.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.563+0000 7f57fcff9700 1 -- 172.21.15.100:0/1424406592 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f80125a0 con 0x7f5800108410 2024-05-10T11:34:45.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.563+0000 7f5805cf4700 1 -- 172.21.15.100:0/1424406592 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58001b5270 con 0x7f5800108410 2024-05-10T11:34:45.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.563+0000 7f5805cf4700 1 -- 172.21.15.100:0/1424406592 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58001b5610 con 0x7f5800108410 2024-05-10T11:34:45.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.567+0000 7f57fcff9700 1 -- 172.21.15.100:0/1424406592 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f57f8003aa0 con 0x7f5800108410 2024-05-10T11:34:45.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.568+0000 7f57fcff9700 1 --2- 172.21.15.100:0/1424406592 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f57e80b2910 0x7f57e80b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:45.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.569+0000 7f57fcff9700 1 -- 172.21.15.100:0/1424406592 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f57f80e16f0 con 0x7f5800108410 2024-05-10T11:34:45.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.570+0000 7f57ff7fe700 1 --2- 172.21.15.100:0/1424406592 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f57e80b2910 0x7f57e80b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:45.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.572+0000 7f5810587b80 1 Processor -- start 2024-05-10T11:34:45.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.573+0000 7f5810587b80 1 -- start start 2024-05-10T11:34:45.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.573+0000 7f57ff7fe700 1 --2- 172.21.15.100:0/1424406592 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f57e80b2910 0x7f57e80b4dd0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f57f0009750 tx=0x7f57f001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:34:45.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.573+0000 7f5810587b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bdfb017190 0x55bdfb017570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:45.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.573+0000 7f5810587b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bdfb017ab0 0x55bdfb011820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:45.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.573+0000 7f5810587b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bdfb011db0 0x55bdfb012210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:45.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.573+0000 7f5810587b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55bdfaf52940 con 0x55bdfb011db0 2024-05-10T11:34:45.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.573+0000 7f5810587b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55bdfaeb7da0 con 0x55bdfb017190 2024-05-10T11:34:45.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.573+0000 7f5810587b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55bdfaf4e900 con 0x55bdfb017ab0 2024-05-10T11:34:45.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.573+0000 7f57feffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bdfb017ab0 0x55bdfb011820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:45.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.573+0000 7f57feffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bdfb017ab0 0x55bdfb011820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:34858/0 (socket says 172.21.15.100:34858) 2024-05-10T11:34:45.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.573+0000 7f57feffd700 1 -- 172.21.15.100:0/1840433176 learned_addr learned my addr 172.21.15.100:0/1840433176 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:34:45.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.573+0000 7f57fffff700 1 --2- 172.21.15.100:0/1840433176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bdfb011db0 0x55bdfb012210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:45.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.574+0000 7f57ff7fe700 1 --2- 172.21.15.100:0/1840433176 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bdfb017190 0x55bdfb017570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:45.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.574+0000 7f57fffff700 1 -- 172.21.15.100:0/1840433176 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bdfb017ab0 msgr2=0x55bdfb011820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:45.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.574+0000 7f57fffff700 1 --2- 172.21.15.100:0/1840433176 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bdfb017ab0 0x55bdfb011820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:45.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.574+0000 7f57fffff700 1 -- 172.21.15.100:0/1840433176 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bdfb017190 msgr2=0x55bdfb017570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:45.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.574+0000 7f57fffff700 1 --2- 172.21.15.100:0/1840433176 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bdfb017190 0x55bdfb017570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:45.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.574+0000 7f57fffff700 1 -- 172.21.15.100:0/1840433176 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdfae9f8d0 con 0x55bdfb011db0 2024-05-10T11:34:45.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.574+0000 7f57fffff700 1 --2- 172.21.15.100:0/1840433176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bdfb011db0 0x55bdfb012210 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f57f80dee30 tx=0x7f57f80a90d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c1d95dce43f2007 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:34:45.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.575+0000 7f57e5ffb700 1 -- 172.21.15.100:0/1840433176 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f80acd80 con 0x55bdfb011db0 2024-05-10T11:34:45.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.576+0000 7f57e5ffb700 1 -- 172.21.15.100:0/1840433176 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f57f80c2070 con 0x55bdfb011db0 2024-05-10T11:34:45.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.576+0000 7f57e5ffb700 1 -- 172.21.15.100:0/1840433176 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f80a8c30 con 0x55bdfb011db0 2024-05-10T11:34:45.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.577+0000 7f5810587b80 1 -- 172.21.15.100:0/1840433176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bdfb011db0 msgr2=0x55bdfb012210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:45.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.578+0000 7f5810587b80 1 --2- 172.21.15.100:0/1840433176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bdfb011db0 0x55bdfb012210 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f57f80dee30 tx=0x7f57f80a90d0 comp rx=0 tx=0).stop 2024-05-10T11:34:45.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.578+0000 7f5810587b80 1 -- 172.21.15.100:0/1840433176 shutdown_connections 2024-05-10T11:34:45.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.578+0000 7f5810587b80 1 --2- 172.21.15.100:0/1840433176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bdfb011db0 0x55bdfb012210 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:45.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.578+0000 7f5810587b80 1 --2- 172.21.15.100:0/1840433176 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bdfb017190 0x55bdfb017570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:45.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.578+0000 7f5810587b80 1 --2- 172.21.15.100:0/1840433176 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bdfb017ab0 0x55bdfb011820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:45.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.578+0000 7f5810587b80 1 -- 172.21.15.100:0/1840433176 >> 172.21.15.100:0/1840433176 conn(0x55bdfaf2cb70 msgr2=0x55bdfaffb2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:34:45.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.581+0000 7f5810587b80 1 -- 172.21.15.100:0/1840433176 shutdown_connections 2024-05-10T11:34:45.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.582+0000 7f5810587b80 1 -- 172.21.15.100:0/1840433176 wait complete. 2024-05-10T11:34:45.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.582+0000 7f5810587b80 1 -- start start 2024-05-10T11:34:45.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.582+0000 7f5810587b80 1 Processor -- start 2024-05-10T11:34:45.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.583+0000 7f5810587b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bdfb011db0 0x55bdfb01d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:45.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.583+0000 7f5810587b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bdfb017190 0x55bdfb01ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:45.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.583+0000 7f5810587b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bdfb017ab0 0x55bdfb01b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:45.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.584+0000 7f5810587b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55bdfb01e530 con 0x55bdfb017ab0 2024-05-10T11:34:45.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.584+0000 7f5810587b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55bdfaf0bed0 con 0x55bdfb017190 2024-05-10T11:34:45.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.584+0000 7f5810587b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55bdfb01e6b0 con 0x55bdfb011db0 2024-05-10T11:34:45.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.584+0000 7f5810587b80 10 client.0 _ll_register_callbacks cb 0x55bdfaee65b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:34:45.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.584+0000 7f57feffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bdfb017190 0x55bdfb01ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:45.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.584+0000 7f57feffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bdfb017190 0x55bdfb01ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:39996/0 (socket says 172.21.15.100:39996) 2024-05-10T11:34:45.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.584+0000 7f57feffd700 1 -- 172.21.15.100:0/1327681241 learned_addr learned my addr 172.21.15.100:0/1327681241 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:34:45.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.584+0000 7f57ff7fe700 1 --2- 172.21.15.100:0/1327681241 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bdfb011db0 0x55bdfb01d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:45.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.584+0000 7f57fffff700 1 --2- 172.21.15.100:0/1327681241 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bdfb017ab0 0x55bdfb01b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:45.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.584+0000 7f57feffd700 1 -- 172.21.15.100:0/1327681241 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bdfb011db0 msgr2=0x55bdfb01d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:45.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.584+0000 7f57feffd700 1 --2- 172.21.15.100:0/1327681241 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bdfb011db0 0x55bdfb01d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:45.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.584+0000 7f57feffd700 1 -- 172.21.15.100:0/1327681241 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bdfb017ab0 msgr2=0x55bdfb01b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:45.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.584+0000 7f57feffd700 1 --2- 172.21.15.100:0/1327681241 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bdfb017ab0 0x55bdfb01b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:45.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.584+0000 7f57feffd700 1 -- 172.21.15.100:0/1327681241 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdfaf09760 con 0x55bdfb017190 2024-05-10T11:34:45.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.585+0000 7f57feffd700 1 --2- 172.21.15.100:0/1327681241 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bdfb017190 0x55bdfb01ab20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f57f4009390 tx=0x7f57f40118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b71217a92d595c79 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:34:45.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.585+0000 7f57deffd700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:34:45.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.585+0000 7f57deffd700 1 -- 172.21.15.100:0/1327681241 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f4014650 con 0x55bdfb017190 2024-05-10T11:34:45.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.585+0000 7f57deffd700 1 -- 172.21.15.100:0/1327681241 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f57f40147f0 con 0x55bdfb017190 2024-05-10T11:34:45.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.585+0000 7f5810587b80 1 -- 172.21.15.100:0/1327681241 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57f80c2070 con 0x55bdfb017190 2024-05-10T11:34:45.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.587+0000 7f57deffd700 1 -- 172.21.15.100:0/1327681241 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f57f40149f0 con 0x55bdfb017190 2024-05-10T11:34:45.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.587+0000 7f5810587b80 10 client.27953 fetch_fsmap learned FSMap version 38 2024-05-10T11:34:45.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.587+0000 7f5810587b80 1 -- 172.21.15.100:0/1327681241 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55bdfb028c90 con 0x55bdfb017190 2024-05-10T11:34:45.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.589+0000 7f57deffd700 1 -- 172.21.15.100:0/1327681241 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f57f4014b90 con 0x55bdfb017190 2024-05-10T11:34:45.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.589+0000 7f57deffd700 1 -- 172.21.15.100:0/1327681241 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7f57f40133a0 con 0x55bdfb017190 2024-05-10T11:34:45.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.589+0000 7f5810587b80 10 client.27953 fetch_fsmap finished waiting for FSMap version 38 2024-05-10T11:34:45.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.589+0000 7f5810587b80 10 client.27953 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:34:45.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.589+0000 7f5810587b80 20 client.27953 populate_metadata read hostname 'smithi100' 2024-05-10T11:34:45.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.589+0000 7f5810587b80 1 --2- 172.21.15.100:0/1327681241 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55bdfb02b320 0x55bdfb02d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:45.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.589+0000 7f5810587b80 4 client.27953 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:34:45.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.589+0000 7f5810587b80 1 -- 172.21.15.100:0/1327681241 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f57f80c2070 con 0x55bdfb02b320 2024-05-10T11:34:45.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.589+0000 7f57ff7fe700 1 --2- 172.21.15.100:0/1327681241 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55bdfb02b320 0x55bdfb02d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:45.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.590+0000 7f57deffd700 10 client.27953 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:34:45.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.590+0000 7f57ff7fe700 1 --2- 172.21.15.100:0/1327681241 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55bdfb02b320 0x55bdfb02d780 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e51a5d6eb784c15 server_cookie=e67debb090053115 in_seq=0 out_seq=0 2024-05-10T11:34:45.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.591+0000 7f57deffd700 1 -- 172.21.15.100:0/1327681241 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f57f80c2070 con 0x55bdfb02b320 2024-05-10T11:34:45.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.591+0000 7f57deffd700 10 client.27953 handle_command_reply: tid=0 2024-05-10T11:34:45.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.591+0000 7f5810587b80 1 client.27953 shutdown 2024-05-10T11:34:45.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.592+0000 7f5810587b80 1 -- 172.21.15.100:0/1327681241 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bdfb017190 msgr2=0x55bdfb01ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:45.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.592+0000 7f5810587b80 1 --2- 172.21.15.100:0/1327681241 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bdfb017190 0x55bdfb01ab20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f57f4009390 tx=0x7f57f40118d0 comp rx=0 tx=0).stop 2024-05-10T11:34:45.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.592+0000 7f5810587b80 1 -- 172.21.15.100:0/1327681241 shutdown_connections 2024-05-10T11:34:45.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.592+0000 7f5810587b80 1 --2- 172.21.15.100:0/1327681241 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55bdfb02b320 0x55bdfb02d780 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:45.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.592+0000 7f5810587b80 1 --2- 172.21.15.100:0/1327681241 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bdfb017ab0 0x55bdfb01b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:45.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.592+0000 7f57deffd700 0 client.27953 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:34:45.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.592+0000 7f5810587b80 1 --2- 172.21.15.100:0/1327681241 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bdfb011db0 0x55bdfb01d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:45.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.592+0000 7f5810587b80 1 --2- 172.21.15.100:0/1327681241 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bdfb017190 0x55bdfb01ab20 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:45.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.592+0000 7f5810587b80 1 -- 172.21.15.100:0/1327681241 >> 172.21.15.100:0/1327681241 conn(0x55bdfaf2cb70 msgr2=0x55bdfaff5b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:34:45.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.592+0000 7f5810587b80 1 -- 172.21.15.100:0/1327681241 shutdown_connections 2024-05-10T11:34:45.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.592+0000 7f5810587b80 1 -- 172.21.15.100:0/1327681241 wait complete. 2024-05-10T11:34:45.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.593+0000 7f5810587b80 20 client.27953 trim_cache size 0 max 16384 2024-05-10T11:34:45.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.629+0000 7f5810587b80 1 Processor -- start 2024-05-10T11:34:45.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.629+0000 7f5810587b80 1 -- start start 2024-05-10T11:34:45.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.630+0000 7f5810587b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bdfaf2cb70 0x55bdfb01b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:45.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.630+0000 7f5810587b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bdfb011db0 0x55bdfaff5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:45.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.630+0000 7f5810587b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bdfb017190 0x55bdfaffff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:45.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.630+0000 7f5810587b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f58001b7e80 con 0x55bdfaf2cb70 2024-05-10T11:34:45.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.630+0000 7f5810587b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f58000754b0 con 0x55bdfb017190 2024-05-10T11:34:45.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.630+0000 7f57fffff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bdfb017190 0x55bdfaffff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:45.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.630+0000 7f57fffff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bdfb017190 0x55bdfaffff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:40002/0 (socket says 172.21.15.100:40002) 2024-05-10T11:34:45.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.630+0000 7f57fffff700 1 -- 172.21.15.100:0/643454925 learned_addr learned my addr 172.21.15.100:0/643454925 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:34:45.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.630+0000 7f5810587b80 1 -- 172.21.15.100:0/643454925 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5800073300 con 0x55bdfb011db0 2024-05-10T11:34:45.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.631+0000 7f57fffff700 1 -- 172.21.15.100:0/643454925 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bdfb011db0 msgr2=0x55bdfaff5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:45.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.631+0000 7f57fffff700 1 --2- 172.21.15.100:0/643454925 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bdfb011db0 0x55bdfaff5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:45.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.631+0000 7f57fffff700 1 -- 172.21.15.100:0/643454925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bdfaf2cb70 msgr2=0x55bdfb01b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:45.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.631+0000 7f57fffff700 1 --2- 172.21.15.100:0/643454925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bdfaf2cb70 0x55bdfb01b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:45.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.631+0000 7f57fffff700 1 -- 172.21.15.100:0/643454925 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdfb01ba30 con 0x55bdfb017190 2024-05-10T11:34:45.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.631+0000 7f57fffff700 1 --2- 172.21.15.100:0/643454925 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bdfb017190 0x55bdfaffff10 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f57f8009440 tx=0x7f57f80a9240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75a6ac8fcc7d528d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:34:45.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.631+0000 7f57e57fa700 1 -- 172.21.15.100:0/643454925 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f80a8320 con 0x55bdfb017190 2024-05-10T11:34:45.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.631+0000 7f57e57fa700 1 -- 172.21.15.100:0/643454925 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f57f80c2040 con 0x55bdfb017190 2024-05-10T11:34:45.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.631+0000 7f57e57fa700 1 -- 172.21.15.100:0/643454925 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f80a4df0 con 0x55bdfb017190 2024-05-10T11:34:45.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.632+0000 7f5810587b80 1 -- 172.21.15.100:0/643454925 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bdfb017190 msgr2=0x55bdfaffff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:45.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.633+0000 7f5810587b80 1 --2- 172.21.15.100:0/643454925 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bdfb017190 0x55bdfaffff10 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f57f8009440 tx=0x7f57f80a9240 comp rx=0 tx=0).stop 2024-05-10T11:34:45.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.633+0000 7f5810587b80 1 -- 172.21.15.100:0/643454925 shutdown_connections 2024-05-10T11:34:45.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.633+0000 7f5810587b80 1 --2- 172.21.15.100:0/643454925 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bdfb017190 0x55bdfaffff10 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:45.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.633+0000 7f5810587b80 1 --2- 172.21.15.100:0/643454925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bdfaf2cb70 0x55bdfb01b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:45.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.633+0000 7f5810587b80 1 --2- 172.21.15.100:0/643454925 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bdfb011db0 0x55bdfaff5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:45.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.633+0000 7f5810587b80 1 -- 172.21.15.100:0/643454925 >> 172.21.15.100:0/643454925 conn(0x55bdfb015390 msgr2=0x55bdfafff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:34:45.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.633+0000 7f5810587b80 1 -- 172.21.15.100:0/643454925 shutdown_connections 2024-05-10T11:34:45.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.633+0000 7f5810587b80 1 -- 172.21.15.100:0/643454925 wait complete. 2024-05-10T11:34:45.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.633+0000 7f5810587b80 1 -- start start 2024-05-10T11:34:45.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.634+0000 7f5810587b80 1 Processor -- start 2024-05-10T11:34:45.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.634+0000 7f5810587b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bdfaf2cb70 0x55bdfb023970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:45.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.634+0000 7f5810587b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bdfb011db0 0x55bdfb023eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:45.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.634+0000 7f5810587b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bdfb017190 0x55bdfb020940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:45.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.634+0000 7f5810587b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5800073970 con 0x55bdfb011db0 2024-05-10T11:34:45.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.634+0000 7f5810587b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5800074de0 con 0x55bdfaf2cb70 2024-05-10T11:34:45.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.634+0000 7f5810587b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5800074040 con 0x55bdfb017190 2024-05-10T11:34:45.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.634+0000 7f57feffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bdfb011db0 0x55bdfb023eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:45.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.634+0000 7f57feffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bdfb011db0 0x55bdfb023eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56510/0 (socket says 172.21.15.100:56510) 2024-05-10T11:34:45.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.634+0000 7f57feffd700 1 -- 172.21.15.100:0/2506114750 learned_addr learned my addr 172.21.15.100:0/2506114750 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:34:45.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.635+0000 7f57fffff700 1 --2- 172.21.15.100:0/2506114750 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bdfb017190 0x55bdfb020940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:45.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.635+0000 7f57ff7fe700 1 --2- 172.21.15.100:0/2506114750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bdfaf2cb70 0x55bdfb023970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:45.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.635+0000 7f5810587b80 10 client.0 _ll_register_callbacks cb 0x55bdfaee65b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:34:45.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.635+0000 7f57ff7fe700 1 -- 172.21.15.100:0/2506114750 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bdfb017190 msgr2=0x55bdfb020940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:45.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.635+0000 7f57ff7fe700 1 --2- 172.21.15.100:0/2506114750 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bdfb017190 0x55bdfb020940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:45.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.635+0000 7f57ff7fe700 1 -- 172.21.15.100:0/2506114750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bdfb011db0 msgr2=0x55bdfb023eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:45.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.635+0000 7f57ff7fe700 1 --2- 172.21.15.100:0/2506114750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bdfb011db0 0x55bdfb023eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:45.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.635+0000 7f57ff7fe700 1 -- 172.21.15.100:0/2506114750 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdfb020e80 con 0x55bdfaf2cb70 2024-05-10T11:34:45.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.635+0000 7f57ff7fe700 1 --2- 172.21.15.100:0/2506114750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bdfaf2cb70 0x55bdfb023970 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f57f000b5c0 tx=0x7f57f00057e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c5e89b1d3caa4ad server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:34:45.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.635+0000 7f57deffd700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:34:45.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.635+0000 7f57deffd700 1 -- 172.21.15.100:0/2506114750 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f000fc50 con 0x55bdfaf2cb70 2024-05-10T11:34:45.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.635+0000 7f57deffd700 1 -- 172.21.15.100:0/2506114750 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f57f000fdf0 con 0x55bdfaf2cb70 2024-05-10T11:34:45.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.635+0000 7f5810587b80 1 -- 172.21.15.100:0/2506114750 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57f80c2040 con 0x55bdfaf2cb70 2024-05-10T11:34:45.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.636+0000 7f57deffd700 1 -- 172.21.15.100:0/2506114750 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f57f003e050 con 0x55bdfaf2cb70 2024-05-10T11:34:45.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.636+0000 7f5810587b80 10 client.27959 fetch_fsmap learned FSMap version 39 2024-05-10T11:34:45.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.636+0000 7f5810587b80 1 -- 172.21.15.100:0/2506114750 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55bdfb0963c0 con 0x55bdfaf2cb70 2024-05-10T11:34:45.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.636+0000 7f57deffd700 1 -- 172.21.15.100:0/2506114750 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f57f80c2040 con 0x55bdfaf2cb70 2024-05-10T11:34:45.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.636+0000 7f57deffd700 1 -- 172.21.15.100:0/2506114750 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f57f0023910 con 0x55bdfaf2cb70 2024-05-10T11:34:45.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.636+0000 7f5810587b80 10 client.27959 fetch_fsmap finished waiting for FSMap version 39 2024-05-10T11:34:45.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.636+0000 7f5810587b80 10 client.27959 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:34:45.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.636+0000 7f5810587b80 20 client.27959 populate_metadata read hostname 'smithi100' 2024-05-10T11:34:45.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.636+0000 7f5810587b80 1 --2- 172.21.15.100:0/2506114750 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55bdfb017ab0 0x55bdfb098b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:45.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.636+0000 7f5810587b80 4 client.27959 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T11:34:45.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.636+0000 7f5810587b80 1 -- 172.21.15.100:0/2506114750 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bdfaf60320 con 0x55bdfb017ab0 2024-05-10T11:34:45.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.636+0000 7f57feffd700 1 --2- 172.21.15.100:0/2506114750 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55bdfb017ab0 0x55bdfb098b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:45.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.637+0000 7f57deffd700 10 client.27959 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:34:45.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.637+0000 7f57feffd700 1 --2- 172.21.15.100:0/2506114750 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55bdfb017ab0 0x55bdfb098b10 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e7301772701011e server_cookie=285e455568bb77ec in_seq=0 out_seq=0 2024-05-10T11:34:45.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.637+0000 7f57deffd700 1 -- 172.21.15.100:0/2506114750 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55bdfaf60320 con 0x55bdfb017ab0 2024-05-10T11:34:45.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.637+0000 7f57deffd700 10 client.27959 handle_command_reply: tid=0 2024-05-10T11:34:45.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.637+0000 7f5810587b80 1 client.27959 shutdown 2024-05-10T11:34:45.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.637+0000 7f5810587b80 1 -- 172.21.15.100:0/2506114750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bdfaf2cb70 msgr2=0x55bdfb023970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:45.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.637+0000 7f5810587b80 1 --2- 172.21.15.100:0/2506114750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bdfaf2cb70 0x55bdfb023970 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f57f000b5c0 tx=0x7f57f00057e0 comp rx=0 tx=0).stop 2024-05-10T11:34:45.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.637+0000 7f5810587b80 1 -- 172.21.15.100:0/2506114750 shutdown_connections 2024-05-10T11:34:45.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.637+0000 7f5810587b80 1 --2- 172.21.15.100:0/2506114750 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55bdfb017ab0 0x55bdfb098b10 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:45.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.637+0000 7f5810587b80 1 --2- 172.21.15.100:0/2506114750 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bdfb017190 0x55bdfb020940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:45.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.637+0000 7f5810587b80 1 --2- 172.21.15.100:0/2506114750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bdfaf2cb70 0x55bdfb023970 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:45.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.637+0000 7f57deffd700 0 client.27959 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:34:45.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.637+0000 7f5810587b80 1 --2- 172.21.15.100:0/2506114750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bdfb011db0 0x55bdfb023eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:45.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.637+0000 7f5810587b80 1 -- 172.21.15.100:0/2506114750 >> 172.21.15.100:0/2506114750 conn(0x55bdfb015390 msgr2=0x55bdfb000090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:34:45.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.638+0000 7f5810587b80 1 -- 172.21.15.100:0/2506114750 shutdown_connections 2024-05-10T11:34:45.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.638+0000 7f5810587b80 1 -- 172.21.15.100:0/2506114750 wait complete. 2024-05-10T11:34:45.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.638+0000 7f5810587b80 20 client.27959 trim_cache size 0 max 16384 2024-05-10T11:34:45.642 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:34:45.642 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-05-10T11:34:45.642 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-05-10T11:34:45.642 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:34:45.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.638+0000 7f5805cf4700 1 -- 172.21.15.100:0/1424406592 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f57e80b2910 msgr2=0x7f57e80b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:45.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.638+0000 7f5805cf4700 1 --2- 172.21.15.100:0/1424406592 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f57e80b2910 0x7f57e80b4dd0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f57f0009750 tx=0x7f57f001a040 comp rx=0 tx=0).stop 2024-05-10T11:34:45.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.638+0000 7f5805cf4700 1 -- 172.21.15.100:0/1424406592 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5800108410 msgr2=0x7f5800111de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:45.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.638+0000 7f5805cf4700 1 --2- 172.21.15.100:0/1424406592 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5800108410 0x7f5800111de0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f57f800bb00 tx=0x7f57f800be10 comp rx=0 tx=0).stop 2024-05-10T11:34:45.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.639+0000 7f5805cf4700 1 -- 172.21.15.100:0/1424406592 shutdown_connections 2024-05-10T11:34:45.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.639+0000 7f5805cf4700 1 --2- 172.21.15.100:0/1424406592 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f57e80b2910 0x7f57e80b4dd0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:45.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.639+0000 7f5805cf4700 1 --2- 172.21.15.100:0/1424406592 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5800108410 0x7f5800111de0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:45.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.639+0000 7f5805cf4700 1 --2- 172.21.15.100:0/1424406592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58001023a0 0x7f5800117820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:45.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.639+0000 7f5805cf4700 1 --2- 172.21.15.100:0/1424406592 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5800102d60 0x7f58001118a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:45.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.639+0000 7f5805cf4700 1 -- 172.21.15.100:0/1424406592 >> 172.21.15.100:0/1424406592 conn(0x7f5800079350 msgr2=0x7f5800101360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:34:45.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.639+0000 7f5805cf4700 1 -- 172.21.15.100:0/1424406592 shutdown_connections 2024-05-10T11:34:45.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:45.639+0000 7f5805cf4700 1 -- 172.21.15.100:0/1424406592 wait complete. 2024-05-10T11:34:45.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:34:45 smithi100 ceph-mon[30296]: pgmap v912: 129 pgs: 129 active+clean; 417 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 833 KiB/s rd, 4.5 MiB/s wr, 277 op/s 2024-05-10T11:34:45.915 INFO:tasks.workunit.client.0.smithi100.stdout: 10 27381 9.80 MB/sec execute 39 sec latency 105.795 ms 2024-05-10T11:34:46.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:34:45 smithi178 ceph-mon[33359]: pgmap v912: 129 pgs: 129 active+clean; 417 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 833 KiB/s rd, 4.5 MiB/s wr, 277 op/s 2024-05-10T11:34:46.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:34:45 smithi188 ceph-mon[34075]: pgmap v912: 129 pgs: 129 active+clean; 417 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 833 KiB/s rd, 4.5 MiB/s wr, 277 op/s 2024-05-10T11:34:46.706 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T11:34:46.706 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T11:34:46.915 INFO:tasks.workunit.client.0.smithi100.stdout: 10 27563 9.79 MB/sec execute 40 sec latency 65.037 ms 2024-05-10T11:34:47.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:34:46 smithi178 ceph-mon[33359]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T11:34:47.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:34:46 smithi188 ceph-mon[34075]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T11:34:47.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:34:46 smithi100 ceph-mon[30296]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T11:34:47.073 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:34:47.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:34:47 smithi100 ceph-mon[30296]: pgmap v913: 129 pgs: 129 active+clean; 419 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 5.0 MiB/s wr, 324 op/s 2024-05-10T11:34:47.915 INFO:tasks.workunit.client.0.smithi100.stdout: 10 27719 9.83 MB/sec execute 41 sec latency 78.960 ms 2024-05-10T11:34:48.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:34:47 smithi178 ceph-mon[33359]: pgmap v913: 129 pgs: 129 active+clean; 419 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 5.0 MiB/s wr, 324 op/s 2024-05-10T11:34:48.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:34:47 smithi188 ceph-mon[34075]: pgmap v913: 129 pgs: 129 active+clean; 419 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 5.0 MiB/s wr, 324 op/s 2024-05-10T11:34:48.915 INFO:tasks.workunit.client.0.smithi100.stdout: 10 27876 9.74 MB/sec execute 42 sec latency 91.748 ms 2024-05-10T11:34:49.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:48.972+0000 7f6985b16700 1 -- 172.21.15.100:0/751369604 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6980104100 msgr2=0x7f6980108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:49.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:48.972+0000 7f6985b16700 1 --2- 172.21.15.100:0/751369604 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6980104100 0x7f6980108120 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f69680099c0 tx=0x7f6968009cd0 comp rx=0 tx=0).stop 2024-05-10T11:34:49.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:48.972+0000 7f6985b16700 1 -- 172.21.15.100:0/751369604 shutdown_connections 2024-05-10T11:34:49.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:48.972+0000 7f6985b16700 1 --2- 172.21.15.100:0/751369604 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6980108850 0x7f69801103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:49.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:48.972+0000 7f6985b16700 1 --2- 172.21.15.100:0/751369604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6980103750 0x7f6980103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:49.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:48.972+0000 7f6985b16700 1 --2- 172.21.15.100:0/751369604 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6980104100 0x7f6980108120 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:49.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:48.972+0000 7f6985b16700 1 -- 172.21.15.100:0/751369604 >> 172.21.15.100:0/751369604 conn(0x7f698007a600 msgr2=0x7f698007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:34:49.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:48.972+0000 7f6985b16700 1 -- 172.21.15.100:0/751369604 shutdown_connections 2024-05-10T11:34:49.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:48.972+0000 7f6985b16700 1 -- 172.21.15.100:0/751369604 wait complete. 2024-05-10T11:34:49.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:48.973+0000 7f6985b16700 1 Processor -- start 2024-05-10T11:34:49.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:48.973+0000 7f6985b16700 1 -- start start 2024-05-10T11:34:49.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:48.973+0000 7f6985b16700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6980103750 0x7f698019f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:49.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:48.973+0000 7f6985b16700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6980104100 0x7f698019fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:49.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:48.973+0000 7f6985b16700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6980108850 0x7f6980199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:49.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:48.973+0000 7f6985b16700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6980113510 con 0x7f6980103750 2024-05-10T11:34:49.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:48.973+0000 7f6985b16700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6980113390 con 0x7f6980108850 2024-05-10T11:34:49.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:48.973+0000 7f6985b16700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6980113690 con 0x7f6980104100 2024-05-10T11:34:49.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:48.973+0000 7f697f7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6980103750 0x7f698019f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:49.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:48.973+0000 7f697f7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6980103750 0x7f698019f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45716/0 (socket says 172.21.15.100:45716) 2024-05-10T11:34:49.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:48.973+0000 7f6977fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6980104100 0x7f698019fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:49.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:48.973+0000 7f697f7fe700 1 -- 172.21.15.100:0/2591505709 learned_addr learned my addr 172.21.15.100:0/2591505709 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:34:49.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:48.973+0000 7f697ffff700 1 --2- 172.21.15.100:0/2591505709 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6980108850 0x7f6980199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:49.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:48.973+0000 7f697f7fe700 1 -- 172.21.15.100:0/2591505709 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6980104100 msgr2=0x7f698019fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:49.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:48.973+0000 7f697f7fe700 1 --2- 172.21.15.100:0/2591505709 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6980104100 0x7f698019fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:49.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:48.973+0000 7f697f7fe700 1 -- 172.21.15.100:0/2591505709 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6980108850 msgr2=0x7f6980199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:49.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:48.973+0000 7f697f7fe700 1 --2- 172.21.15.100:0/2591505709 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6980108850 0x7f6980199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:49.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:48.973+0000 7f697f7fe700 1 -- 172.21.15.100:0/2591505709 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6980199ff0 con 0x7f6980103750 2024-05-10T11:34:49.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:48.973+0000 7f697f7fe700 1 --2- 172.21.15.100:0/2591505709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6980103750 0x7f698019f690 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f697000c9b0 tx=0x7f697000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:34:49.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:48.974+0000 7f697d7fa700 1 -- 172.21.15.100:0/2591505709 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6970007b00 con 0x7f6980103750 2024-05-10T11:34:49.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:48.974+0000 7f697d7fa700 1 -- 172.21.15.100:0/2591505709 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f697000ce50 con 0x7f6980103750 2024-05-10T11:34:49.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:48.974+0000 7f6985b16700 1 -- 172.21.15.100:0/2591505709 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f698019a2e0 con 0x7f6980103750 2024-05-10T11:34:49.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:48.974+0000 7f697d7fa700 1 -- 172.21.15.100:0/2591505709 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6970018660 con 0x7f6980103750 2024-05-10T11:34:49.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:48.974+0000 7f6985b16700 1 -- 172.21.15.100:0/2591505709 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69800756d0 con 0x7f6980103750 2024-05-10T11:34:49.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:48.975+0000 7f6985b16700 1 -- 172.21.15.100:0/2591505709 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6980104be0 con 0x7f6980103750 2024-05-10T11:34:49.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:48.977+0000 7f697d7fa700 1 -- 172.21.15.100:0/2591505709 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f697000f450 con 0x7f6980103750 2024-05-10T11:34:49.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:48.978+0000 7f697d7fa700 1 --2- 172.21.15.100:0/2591505709 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f696c0b2880 0x7f696c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:49.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:48.978+0000 7f697d7fa700 1 -- 172.21.15.100:0/2591505709 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f69700e24a0 con 0x7f6980103750 2024-05-10T11:34:49.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:48.978+0000 7f6977fff700 1 --2- 172.21.15.100:0/2591505709 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f696c0b2880 0x7f696c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:49.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:48.979+0000 7f6977fff700 1 --2- 172.21.15.100:0/2591505709 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f696c0b2880 0x7f696c0b4d40 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f69801030b0 tx=0x7f6968005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:34:49.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:48.987+0000 7f697d7fa700 1 -- 172.21.15.100:0/2591505709 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f6970014030 con 0x7f6980103750 2024-05-10T11:34:49.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:49.417+0000 7f6985b16700 1 -- 172.21.15.100:0/2591505709 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f698002e510 con 0x7f6980103750 2024-05-10T11:34:49.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:49.418+0000 7f697d7fa700 1 -- 172.21.15.100:0/2591505709 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11900 (secure 0 0 0) 0x7f69700a9e60 con 0x7f6980103750 2024-05-10T11:34:49.420 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:34:49.420 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2072225012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2072225012},{"type":"v1","addr":"172.21.15.100:6835","nonce":2072225012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/401939540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":401939540},{"type":"v1","addr":"172.21.15.100:6837","nonce":401939540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14571,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/3946311025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3946311025},{"type":"v1","addr":"172.21.15.100:6839","nonce":3946311025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3736371660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3736371660},{"type":"v1","addr":"172.21.15.100:6841","nonce":3736371660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/2741810791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2741810791},{"type":"v1","addr":"172.21.15.178:6833","nonce":2741810791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1704891041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1704891041},{"type":"v1","addr":"172.21.15.188:6833","nonce":1704891041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T11:14:27.242452+0000","modified":"2024-05-10T11:34:45.584275+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24439,"mds_1":24437,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/2124076312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2124076312},{"type":"v1","addr":"172.21.15.178:6835","nonce":2124076312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/3224358425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3224358425},{"type":"v1","addr":"172.21.15.178:6837","nonce":3224358425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/1943342620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1943342620},{"type":"v1","addr":"172.21.15.188:6837","nonce":1943342620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.178:6839/3760985404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3760985404},{"type":"v1","addr":"172.21.15.178:6839","nonce":3760985404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.188:6839/3194917070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":3194917070},{"type":"v1","addr":"172.21.15.188:6839","nonce":3194917070}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27034":{"gid":27034,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/3139559450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3139559450},{"type":"v1","addr":"172.21.15.188:6835","nonce":3139559450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24439,"qdb_cluster":[24427,24437,24439]},"id":1}]} 2024-05-10T11:34:49.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:49.422+0000 7f6985b16700 1 -- 172.21.15.100:0/2591505709 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f696c0b2880 msgr2=0x7f696c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:49.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:49.422+0000 7f6985b16700 1 --2- 172.21.15.100:0/2591505709 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f696c0b2880 0x7f696c0b4d40 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f69801030b0 tx=0x7f6968005fb0 comp rx=0 tx=0).stop 2024-05-10T11:34:49.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:49.422+0000 7f6985b16700 1 -- 172.21.15.100:0/2591505709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6980103750 msgr2=0x7f698019f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:49.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:49.422+0000 7f6985b16700 1 --2- 172.21.15.100:0/2591505709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6980103750 0x7f698019f690 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f697000c9b0 tx=0x7f697000ccc0 comp rx=0 tx=0).stop 2024-05-10T11:34:49.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:49.422+0000 7f6985b16700 1 -- 172.21.15.100:0/2591505709 shutdown_connections 2024-05-10T11:34:49.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:49.422+0000 7f6985b16700 1 --2- 172.21.15.100:0/2591505709 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f696c0b2880 0x7f696c0b4d40 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:49.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:49.422+0000 7f6985b16700 1 --2- 172.21.15.100:0/2591505709 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6980108850 0x7f6980199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:49.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:49.422+0000 7f6985b16700 1 --2- 172.21.15.100:0/2591505709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6980103750 0x7f698019f690 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:49.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:49.422+0000 7f6985b16700 1 --2- 172.21.15.100:0/2591505709 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6980104100 0x7f698019fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:49.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:49.422+0000 7f6985b16700 1 -- 172.21.15.100:0/2591505709 >> 172.21.15.100:0/2591505709 conn(0x7f698007a600 msgr2=0x7f69801017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:34:49.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:49.422+0000 7f6985b16700 1 -- 172.21.15.100:0/2591505709 shutdown_connections 2024-05-10T11:34:49.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:49.422+0000 7f6985b16700 1 -- 172.21.15.100:0/2591505709 wait complete. 2024-05-10T11:34:49.424 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 39 2024-05-10T11:34:49.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:34:49 smithi100 ceph-mon[30296]: pgmap v914: 129 pgs: 129 active+clean; 420 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 4.5 MiB/s wr, 309 op/s 2024-05-10T11:34:49.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:34:49 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/2591505709' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:34:49.916 INFO:tasks.workunit.client.0.smithi100.stdout: 10 28041 9.87 MB/sec execute 43 sec latency 104.564 ms 2024-05-10T11:34:50.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:34:49 smithi178 ceph-mon[33359]: pgmap v914: 129 pgs: 129 active+clean; 420 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 4.5 MiB/s wr, 309 op/s 2024-05-10T11:34:50.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:34:49 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/2591505709' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:34:50.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:34:49 smithi188 ceph-mon[34075]: pgmap v914: 129 pgs: 129 active+clean; 420 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 4.5 MiB/s wr, 309 op/s 2024-05-10T11:34:50.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:34:49 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/2591505709' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:34:50.429 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-10T11:34:50.793 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:34:50.915 INFO:tasks.workunit.client.0.smithi100.stdout: 10 28191 9.72 MB/sec execute 44 sec latency 71.553 ms 2024-05-10T11:34:51.915 INFO:tasks.workunit.client.0.smithi100.stdout: 10 28372 9.67 MB/sec execute 45 sec latency 130.971 ms 2024-05-10T11:34:51.967 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:34:51 smithi100 ceph-mon[30296]: pgmap v915: 129 pgs: 129 active+clean; 416 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 852 KiB/s rd, 4.6 MiB/s wr, 308 op/s 2024-05-10T11:34:52.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:34:51 smithi178 ceph-mon[33359]: pgmap v915: 129 pgs: 129 active+clean; 416 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 852 KiB/s rd, 4.6 MiB/s wr, 308 op/s 2024-05-10T11:34:52.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:34:51 smithi188 ceph-mon[34075]: pgmap v915: 129 pgs: 129 active+clean; 416 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 852 KiB/s rd, 4.6 MiB/s wr, 308 op/s 2024-05-10T11:34:52.916 INFO:tasks.workunit.client.0.smithi100.stdout: 10 28530 9.60 MB/sec execute 46 sec latency 111.376 ms 2024-05-10T11:34:53.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.039+0000 7fc9a33cc700 1 -- 172.21.15.100:0/3679064467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc99c108400 msgr2=0x7fc99c1087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:53.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.039+0000 7fc9a33cc700 1 --2- 172.21.15.100:0/3679064467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc99c108400 0x7fc99c1087e0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fc998009990 tx=0x7fc998009ca0 comp rx=0 tx=0).stop 2024-05-10T11:34:53.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.039+0000 7fc9a33cc700 1 -- 172.21.15.100:0/3679064467 shutdown_connections 2024-05-10T11:34:53.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.039+0000 7fc9a33cc700 1 --2- 172.21.15.100:0/3679064467 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc99c102d50 0x7fc99c10f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:53.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.039+0000 7fc9a33cc700 1 --2- 172.21.15.100:0/3679064467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc99c108400 0x7fc99c1087e0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:53.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.039+0000 7fc9a33cc700 1 --2- 172.21.15.100:0/3679064467 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc99c102390 0x7fc99c102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:53.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.039+0000 7fc9a33cc700 1 -- 172.21.15.100:0/3679064467 >> 172.21.15.100:0/3679064467 conn(0x7fc99c079310 msgr2=0x7fc99c079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:34:53.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.039+0000 7fc9a33cc700 1 -- 172.21.15.100:0/3679064467 shutdown_connections 2024-05-10T11:34:53.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.039+0000 7fc9a33cc700 1 -- 172.21.15.100:0/3679064467 wait complete. 2024-05-10T11:34:53.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.040+0000 7fc9a33cc700 1 Processor -- start 2024-05-10T11:34:53.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.040+0000 7fc9a33cc700 1 -- start start 2024-05-10T11:34:53.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.040+0000 7fc9a33cc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc99c102390 0x7fc99c1a02d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:53.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.040+0000 7fc9a33cc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc99c102d50 0x7fc99c1a0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:53.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.040+0000 7fc9a33cc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc99c108400 0x7fc99c1a4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:53.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.040+0000 7fc9a33cc700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc99c112420 con 0x7fc99c108400 2024-05-10T11:34:53.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.040+0000 7fc9a33cc700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc99c1122a0 con 0x7fc99c102390 2024-05-10T11:34:53.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.040+0000 7fc9a33cc700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc99c1125a0 con 0x7fc99c102d50 2024-05-10T11:34:53.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.040+0000 7fc9a1168700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc99c102390 0x7fc99c1a02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:53.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.041+0000 7fc9a1168700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc99c102390 0x7fc99c1a02d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:51204/0 (socket says 172.21.15.100:51204) 2024-05-10T11:34:53.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.041+0000 7fc9a1168700 1 -- 172.21.15.100:0/3596167397 learned_addr learned my addr 172.21.15.100:0/3596167397 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:34:53.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.041+0000 7fc9a0967700 1 --2- 172.21.15.100:0/3596167397 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc99c102d50 0x7fc99c1a0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:53.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.041+0000 7fc9a1969700 1 --2- 172.21.15.100:0/3596167397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc99c108400 0x7fc99c1a4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:53.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.041+0000 7fc9a1168700 1 -- 172.21.15.100:0/3596167397 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc99c102d50 msgr2=0x7fc99c1a0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:53.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.041+0000 7fc9a1168700 1 --2- 172.21.15.100:0/3596167397 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc99c102d50 0x7fc99c1a0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:53.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.041+0000 7fc9a1168700 1 -- 172.21.15.100:0/3596167397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc99c108400 msgr2=0x7fc99c1a4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:53.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.041+0000 7fc9a1168700 1 --2- 172.21.15.100:0/3596167397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc99c108400 0x7fc99c1a4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:53.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.041+0000 7fc9a1168700 1 -- 172.21.15.100:0/3596167397 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc99c1a5150 con 0x7fc99c102390 2024-05-10T11:34:53.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.041+0000 7fc9a1969700 1 --2- 172.21.15.100:0/3596167397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc99c108400 0x7fc99c1a4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:34:53.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.041+0000 7fc9a1168700 1 --2- 172.21.15.100:0/3596167397 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc99c102390 0x7fc99c1a02d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc99800b650 tx=0x7fc99800bfc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:34:53.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.041+0000 7fc9927fc700 1 -- 172.21.15.100:0/3596167397 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc998027070 con 0x7fc99c102390 2024-05-10T11:34:53.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.042+0000 7fc9927fc700 1 -- 172.21.15.100:0/3596167397 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc998022470 con 0x7fc99c102390 2024-05-10T11:34:53.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.042+0000 7fc9a33cc700 1 -- 172.21.15.100:0/3596167397 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc99c1a53e0 con 0x7fc99c102390 2024-05-10T11:34:53.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.042+0000 7fc9a33cc700 1 -- 172.21.15.100:0/3596167397 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc99c1accc0 con 0x7fc99c102390 2024-05-10T11:34:53.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.042+0000 7fc9927fc700 1 -- 172.21.15.100:0/3596167397 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc998027070 con 0x7fc99c102390 2024-05-10T11:34:53.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.045+0000 7fc9927fc700 1 -- 172.21.15.100:0/3596167397 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fc998009d10 con 0x7fc99c102390 2024-05-10T11:34:53.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.047+0000 7fc9927fc700 1 --2- 172.21.15.100:0/3596167397 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc9880b2a80 0x7fc9880b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:53.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.047+0000 7fc9adc5fb80 1 Processor -- start 2024-05-10T11:34:53.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.047+0000 7fc9adc5fb80 1 -- start start 2024-05-10T11:34:53.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.047+0000 7fc9927fc700 1 -- 172.21.15.100:0/3596167397 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fc9980e2c40 con 0x7fc99c102390 2024-05-10T11:34:53.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.047+0000 7fc9a0967700 1 --2- 172.21.15.100:0/3596167397 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc9880b2a80 0x7fc9880b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:53.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.047+0000 7fc9adc5fb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a945e71b0 0x559a945e7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:53.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.047+0000 7fc9adc5fb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a945e7ad0 0x559a945e1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:53.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.047+0000 7fc9adc5fb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a945e1d50 0x559a945e2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:53.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.047+0000 7fc9adc5fb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x559a94533940 con 0x559a945e71b0 2024-05-10T11:34:53.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.047+0000 7fc9adc5fb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559a94498da0 con 0x559a945e1d50 2024-05-10T11:34:53.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.047+0000 7fc9adc5fb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x559a9452f900 con 0x559a945e7ad0 2024-05-10T11:34:53.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.048+0000 7fc9a0967700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a945e7ad0 0x559a945e1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:53.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.048+0000 7fc9a0967700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a945e7ad0 0x559a945e1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:49126/0 (socket says 172.21.15.100:49126) 2024-05-10T11:34:53.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.048+0000 7fc9a0967700 1 -- 172.21.15.100:0/1314478558 learned_addr learned my addr 172.21.15.100:0/1314478558 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:34:53.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.048+0000 7fc9a1168700 1 --2- 172.21.15.100:0/1314478558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a945e71b0 0x559a945e7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:53.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.048+0000 7fc9a0967700 1 --2- 172.21.15.100:0/3596167397 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc9880b2a80 0x7fc9880b4f40 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fc99c079930 tx=0x7fc98c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:34:53.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.048+0000 7fc9a1969700 1 --2- 172.21.15.100:0/1314478558 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a945e1d50 0x559a945e2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:53.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.048+0000 7fc9a0967700 1 -- 172.21.15.100:0/1314478558 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a945e1d50 msgr2=0x559a945e2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:53.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.048+0000 7fc9a0967700 1 --2- 172.21.15.100:0/1314478558 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a945e1d50 0x559a945e2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:53.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.048+0000 7fc9a0967700 1 -- 172.21.15.100:0/1314478558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a945e71b0 msgr2=0x559a945e7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:53.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.048+0000 7fc9a0967700 1 --2- 172.21.15.100:0/1314478558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a945e71b0 0x559a945e7590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:53.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.048+0000 7fc9a0967700 1 -- 172.21.15.100:0/1314478558 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a944808d0 con 0x559a945e7ad0 2024-05-10T11:34:53.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.048+0000 7fc9a1168700 1 --2- 172.21.15.100:0/1314478558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a945e71b0 0x559a945e7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:34:53.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.048+0000 7fc9a0967700 1 --2- 172.21.15.100:0/1314478558 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a945e7ad0 0x559a945e1810 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc98c010f40 tx=0x7fc98c01b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4bfb8bb3af6d3d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:34:53.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.049+0000 7fc9877fe700 1 -- 172.21.15.100:0/1314478558 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c011db0 con 0x559a945e7ad0 2024-05-10T11:34:53.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.049+0000 7fc9877fe700 1 -- 172.21.15.100:0/1314478558 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc98c021070 con 0x559a945e7ad0 2024-05-10T11:34:53.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.049+0000 7fc9877fe700 1 -- 172.21.15.100:0/1314478558 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c01d690 con 0x559a945e7ad0 2024-05-10T11:34:53.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.050+0000 7fc9adc5fb80 1 -- 172.21.15.100:0/1314478558 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a945e7ad0 msgr2=0x559a945e1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:53.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.050+0000 7fc9adc5fb80 1 --2- 172.21.15.100:0/1314478558 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a945e7ad0 0x559a945e1810 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc98c010f40 tx=0x7fc98c01b000 comp rx=0 tx=0).stop 2024-05-10T11:34:53.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.050+0000 7fc9adc5fb80 1 -- 172.21.15.100:0/1314478558 shutdown_connections 2024-05-10T11:34:53.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.050+0000 7fc9adc5fb80 1 --2- 172.21.15.100:0/1314478558 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a945e1d50 0x559a945e2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:53.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.050+0000 7fc9adc5fb80 1 --2- 172.21.15.100:0/1314478558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a945e71b0 0x559a945e7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:53.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.050+0000 7fc9adc5fb80 1 --2- 172.21.15.100:0/1314478558 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a945e7ad0 0x559a945e1810 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:53.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.050+0000 7fc9adc5fb80 1 -- 172.21.15.100:0/1314478558 >> 172.21.15.100:0/1314478558 conn(0x559a9450db70 msgr2=0x559a945dc2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:34:53.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.050+0000 7fc9adc5fb80 1 -- 172.21.15.100:0/1314478558 shutdown_connections 2024-05-10T11:34:53.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.050+0000 7fc9adc5fb80 1 -- 172.21.15.100:0/1314478558 wait complete. 2024-05-10T11:34:53.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.050+0000 7fc9adc5fb80 1 -- start start 2024-05-10T11:34:53.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.050+0000 7fc9adc5fb80 1 Processor -- start 2024-05-10T11:34:53.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.051+0000 7fc9adc5fb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a945e1d50 0x559a945ed6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:53.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.051+0000 7fc9adc5fb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a945e71b0 0x559a945eaa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:53.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.051+0000 7fc9adc5fb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a945e7ad0 0x559a945eafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:53.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.051+0000 7fc9adc5fb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x559a945ee4f0 con 0x559a945e71b0 2024-05-10T11:34:53.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.051+0000 7fc9adc5fb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559a944eced0 con 0x559a945e1d50 2024-05-10T11:34:53.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.051+0000 7fc9adc5fb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x559a945ee670 con 0x559a945e7ad0 2024-05-10T11:34:53.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.051+0000 7fc9adc5fb80 10 client.0 _ll_register_callbacks cb 0x559a944c75b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:34:53.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.051+0000 7fc9a1168700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a945e1d50 0x559a945ed6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:53.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.051+0000 7fc9a1168700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a945e1d50 0x559a945ed6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:51216/0 (socket says 172.21.15.100:51216) 2024-05-10T11:34:53.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.051+0000 7fc9a1168700 1 -- 172.21.15.100:0/1626886716 learned_addr learned my addr 172.21.15.100:0/1626886716 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:34:53.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.051+0000 7fc9a0967700 1 --2- 172.21.15.100:0/1626886716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a945e71b0 0x559a945eaa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:53.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.051+0000 7fc9a1969700 1 --2- 172.21.15.100:0/1626886716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a945e7ad0 0x559a945eafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:53.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.052+0000 7fc9a1168700 1 -- 172.21.15.100:0/1626886716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a945e7ad0 msgr2=0x559a945eafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:53.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.052+0000 7fc9a1168700 1 --2- 172.21.15.100:0/1626886716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a945e7ad0 0x559a945eafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:53.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.052+0000 7fc9a1168700 1 -- 172.21.15.100:0/1626886716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a945e71b0 msgr2=0x559a945eaa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:53.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.052+0000 7fc9a1168700 1 --2- 172.21.15.100:0/1626886716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a945e71b0 0x559a945eaa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:53.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.052+0000 7fc9a1168700 1 -- 172.21.15.100:0/1626886716 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a944ea760 con 0x559a945e1d50 2024-05-10T11:34:53.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.052+0000 7fc9a1969700 1 --2- 172.21.15.100:0/1626886716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a945e7ad0 0x559a945eafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:34:53.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.052+0000 7fc9a0967700 1 --2- 172.21.15.100:0/1626886716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a945e71b0 0x559a945eaa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:34:53.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.052+0000 7fc9a1168700 1 --2- 172.21.15.100:0/1626886716 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a945e1d50 0x559a945ed6e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc9980a7200 tx=0x7fc998022be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb666c12f4ecba17 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:34:53.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.052+0000 7fc984ff9700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:34:53.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.053+0000 7fc984ff9700 1 -- 172.21.15.100:0/1626886716 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9980ae340 con 0x559a945e1d50 2024-05-10T11:34:53.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.053+0000 7fc984ff9700 1 -- 172.21.15.100:0/1626886716 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fc9980b2ae0 con 0x559a945e1d50 2024-05-10T11:34:53.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.053+0000 7fc9adc5fb80 1 -- 172.21.15.100:0/1626886716 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc98c021070 con 0x559a945e1d50 2024-05-10T11:34:53.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.053+0000 7fc984ff9700 1 -- 172.21.15.100:0/1626886716 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc9980b2d60 con 0x559a945e1d50 2024-05-10T11:34:53.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.053+0000 7fc9adc5fb80 10 client.27977 fetch_fsmap learned FSMap version 39 2024-05-10T11:34:53.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.053+0000 7fc9adc5fb80 1 -- 172.21.15.100:0/1626886716 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559a945f8d50 con 0x559a945e1d50 2024-05-10T11:34:53.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.054+0000 7fc984ff9700 1 -- 172.21.15.100:0/1626886716 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc98c021070 con 0x559a945e1d50 2024-05-10T11:34:53.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.054+0000 7fc984ff9700 1 -- 172.21.15.100:0/1626886716 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fc99800fa90 con 0x559a945e1d50 2024-05-10T11:34:53.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.054+0000 7fc9adc5fb80 10 client.27977 fetch_fsmap finished waiting for FSMap version 39 2024-05-10T11:34:53.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.054+0000 7fc9adc5fb80 10 client.27977 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-10T11:34:53.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.054+0000 7fc9adc5fb80 20 client.27977 populate_metadata read hostname 'smithi100' 2024-05-10T11:34:53.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.054+0000 7fc9adc5fb80 1 --2- 172.21.15.100:0/1626886716 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x559a945fb2c0 0x559a945fd720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:53.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.054+0000 7fc9adc5fb80 4 client.27977 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:34:53.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.054+0000 7fc9adc5fb80 1 -- 172.21.15.100:0/1626886716 --> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a94541320 con 0x559a945fb2c0 2024-05-10T11:34:53.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.054+0000 7fc9a0967700 1 --2- 172.21.15.100:0/1626886716 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x559a945fb2c0 0x559a945fd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:53.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.055+0000 7fc9a0967700 1 --2- 172.21.15.100:0/1626886716 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x559a945fb2c0 0x559a945fd720 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cdb573c050b25aa4 server_cookie=1c478c0649562c0a in_seq=0 out_seq=0 2024-05-10T11:34:53.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.055+0000 7fc984ff9700 10 client.27977 ms_handle_connect on v2:172.21.15.188:6836/1943342620 2024-05-10T11:34:53.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.056+0000 7fc984ff9700 1 -- 172.21.15.100:0/1626886716 <== mds.2 v2:172.21.15.188:6836/1943342620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559a94541320 con 0x559a945fb2c0 2024-05-10T11:34:53.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.056+0000 7fc984ff9700 10 client.27977 handle_command_reply: tid=0 2024-05-10T11:34:53.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.056+0000 7fc9adc5fb80 1 client.27977 shutdown 2024-05-10T11:34:53.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.056+0000 7fc9adc5fb80 1 -- 172.21.15.100:0/1626886716 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a945e1d50 msgr2=0x559a945ed6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:53.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.056+0000 7fc9adc5fb80 1 --2- 172.21.15.100:0/1626886716 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a945e1d50 0x559a945ed6e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc9980a7200 tx=0x7fc998022be0 comp rx=0 tx=0).stop 2024-05-10T11:34:53.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.056+0000 7fc9adc5fb80 1 -- 172.21.15.100:0/1626886716 shutdown_connections 2024-05-10T11:34:53.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.056+0000 7fc9adc5fb80 1 --2- 172.21.15.100:0/1626886716 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x559a945fb2c0 0x559a945fd720 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:53.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.056+0000 7fc9adc5fb80 1 --2- 172.21.15.100:0/1626886716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a945e7ad0 0x559a945eafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:53.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.056+0000 7fc9adc5fb80 1 --2- 172.21.15.100:0/1626886716 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a945e1d50 0x559a945ed6e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:53.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.056+0000 7fc984ff9700 0 client.27977 ms_handle_reset on v2:172.21.15.188:6836/1943342620 2024-05-10T11:34:53.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.056+0000 7fc9adc5fb80 1 --2- 172.21.15.100:0/1626886716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a945e71b0 0x559a945eaa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:53.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.056+0000 7fc9adc5fb80 1 -- 172.21.15.100:0/1626886716 >> 172.21.15.100:0/1626886716 conn(0x559a9450db70 msgr2=0x559a945d6b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:34:53.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.057+0000 7fc9adc5fb80 1 -- 172.21.15.100:0/1626886716 shutdown_connections 2024-05-10T11:34:53.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.057+0000 7fc9adc5fb80 1 -- 172.21.15.100:0/1626886716 wait complete. 2024-05-10T11:34:53.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.057+0000 7fc9adc5fb80 20 client.27977 trim_cache size 0 max 16384 2024-05-10T11:34:53.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.080+0000 7fc9adc5fb80 1 Processor -- start 2024-05-10T11:34:53.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.080+0000 7fc9adc5fb80 1 -- start start 2024-05-10T11:34:53.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.080+0000 7fc9adc5fb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a945e1d50 0x559a945e1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:53.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.080+0000 7fc9adc5fb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a945e71b0 0x559a945f7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:53.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.080+0000 7fc9adc5fb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a945e7ad0 0x559a945e85f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:53.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.080+0000 7fc9adc5fb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc99c1b7f90 con 0x559a945e71b0 2024-05-10T11:34:53.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.080+0000 7fc9a1168700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a945e1d50 0x559a945e1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:53.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.080+0000 7fc9adc5fb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc99c1b57c0 con 0x559a945e1d50 2024-05-10T11:34:53.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.080+0000 7fc9adc5fb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc99c1b3610 con 0x559a945e7ad0 2024-05-10T11:34:53.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.080+0000 7fc9a1168700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a945e1d50 0x559a945e1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:51218/0 (socket says 172.21.15.100:51218) 2024-05-10T11:34:53.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.080+0000 7fc9a1168700 1 -- 172.21.15.100:0/3092318275 learned_addr learned my addr 172.21.15.100:0/3092318275 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:34:53.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.080+0000 7fc9a0967700 1 --2- 172.21.15.100:0/3092318275 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a945e71b0 0x559a945f7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:53.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.081+0000 7fc9a1969700 1 --2- 172.21.15.100:0/3092318275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a945e7ad0 0x559a945e85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:53.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.081+0000 7fc9a0967700 1 -- 172.21.15.100:0/3092318275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a945e7ad0 msgr2=0x559a945e85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:53.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.081+0000 7fc9a0967700 1 --2- 172.21.15.100:0/3092318275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a945e7ad0 0x559a945e85f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:53.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.081+0000 7fc9a0967700 1 -- 172.21.15.100:0/3092318275 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a945e1d50 msgr2=0x559a945e1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:53.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.081+0000 7fc9a0967700 1 --2- 172.21.15.100:0/3092318275 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a945e1d50 0x559a945e1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:53.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.081+0000 7fc9a0967700 1 -- 172.21.15.100:0/3092318275 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a945ee660 con 0x559a945e71b0 2024-05-10T11:34:53.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.081+0000 7fc9a0967700 1 --2- 172.21.15.100:0/3092318275 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a945e71b0 0x559a945f7580 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x559a945e7eb0 tx=0x7fc98c028040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fde4f2fc0c89490f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:34:53.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.081+0000 7fc986ffd700 1 -- 172.21.15.100:0/3092318275 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c036070 con 0x559a945e71b0 2024-05-10T11:34:53.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.081+0000 7fc986ffd700 1 -- 172.21.15.100:0/3092318275 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc98c021030 con 0x559a945e71b0 2024-05-10T11:34:53.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.082+0000 7fc986ffd700 1 -- 172.21.15.100:0/3092318275 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c031610 con 0x559a945e71b0 2024-05-10T11:34:53.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.082+0000 7fc9adc5fb80 1 -- 172.21.15.100:0/3092318275 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a945e71b0 msgr2=0x559a945f7580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:53.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.082+0000 7fc9adc5fb80 1 --2- 172.21.15.100:0/3092318275 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a945e71b0 0x559a945f7580 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x559a945e7eb0 tx=0x7fc98c028040 comp rx=0 tx=0).stop 2024-05-10T11:34:53.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.082+0000 7fc9adc5fb80 1 -- 172.21.15.100:0/3092318275 shutdown_connections 2024-05-10T11:34:53.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.082+0000 7fc9adc5fb80 1 --2- 172.21.15.100:0/3092318275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a945e7ad0 0x559a945e85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:53.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.082+0000 7fc9adc5fb80 1 --2- 172.21.15.100:0/3092318275 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a945e1d50 0x559a945e1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:53.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.082+0000 7fc9adc5fb80 1 --2- 172.21.15.100:0/3092318275 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a945e71b0 0x559a945f7580 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:53.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.082+0000 7fc9adc5fb80 1 -- 172.21.15.100:0/3092318275 >> 172.21.15.100:0/3092318275 conn(0x559a9450db70 msgr2=0x559a945e7ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:34:53.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.082+0000 7fc9adc5fb80 1 -- 172.21.15.100:0/3092318275 shutdown_connections 2024-05-10T11:34:53.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.083+0000 7fc9adc5fb80 1 -- 172.21.15.100:0/3092318275 wait complete. 2024-05-10T11:34:53.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.083+0000 7fc9adc5fb80 1 -- start start 2024-05-10T11:34:53.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.083+0000 7fc9adc5fb80 1 Processor -- start 2024-05-10T11:34:53.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.083+0000 7fc9adc5fb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a945e1d50 0x559a945f3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:53.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.083+0000 7fc9adc5fb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a945e71b0 0x559a945f4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:53.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.083+0000 7fc9adc5fb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a945e7ad0 0x559a946023c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:53.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.083+0000 7fc9adc5fb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc99c1b3c80 con 0x559a945e71b0 2024-05-10T11:34:53.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.083+0000 7fc9adc5fb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc99c1b50f0 con 0x559a945e1d50 2024-05-10T11:34:53.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.083+0000 7fc9adc5fb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc99c1b4350 con 0x559a945e7ad0 2024-05-10T11:34:53.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.084+0000 7fc9adc5fb80 10 client.0 _ll_register_callbacks cb 0x559a944c75b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:34:53.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.084+0000 7fc9a0967700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a945e71b0 0x559a945f4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:53.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.084+0000 7fc9a1168700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a945e1d50 0x559a945f3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:53.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.084+0000 7fc9a1168700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a945e1d50 0x559a945f3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:51234/0 (socket says 172.21.15.100:51234) 2024-05-10T11:34:53.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.084+0000 7fc9a0967700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a945e71b0 0x559a945f4020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45738/0 (socket says 172.21.15.100:45738) 2024-05-10T11:34:53.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.084+0000 7fc9a1168700 1 -- 172.21.15.100:0/3196621913 learned_addr learned my addr 172.21.15.100:0/3196621913 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:34:53.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.084+0000 7fc9a1969700 1 --2- 172.21.15.100:0/3196621913 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a945e7ad0 0x559a946023c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:53.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.084+0000 7fc9a0967700 1 -- 172.21.15.100:0/3196621913 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a945e7ad0 msgr2=0x559a946023c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:53.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.084+0000 7fc9a0967700 1 --2- 172.21.15.100:0/3196621913 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a945e7ad0 0x559a946023c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:53.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.084+0000 7fc9a0967700 1 -- 172.21.15.100:0/3196621913 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a945e1d50 msgr2=0x559a945f3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:53.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.084+0000 7fc9a0967700 1 --2- 172.21.15.100:0/3196621913 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a945e1d50 0x559a945f3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:53.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.084+0000 7fc9a0967700 1 -- 172.21.15.100:0/3196621913 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a945ddd00 con 0x559a945e71b0 2024-05-10T11:34:53.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.084+0000 7fc9a0967700 1 --2- 172.21.15.100:0/3196621913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a945e71b0 0x559a945f4020 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fc98c00b000 tx=0x7fc98c0056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3713b635022e11c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:34:53.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.084+0000 7fc984ff9700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:34:53.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.084+0000 7fc984ff9700 1 -- 172.21.15.100:0/3196621913 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c036040 con 0x559a945e71b0 2024-05-10T11:34:53.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.084+0000 7fc984ff9700 1 -- 172.21.15.100:0/3196621913 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fc98c0315b0 con 0x559a945e71b0 2024-05-10T11:34:53.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.084+0000 7fc9adc5fb80 1 -- 172.21.15.100:0/3196621913 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc98c021030 con 0x559a945e71b0 2024-05-10T11:34:53.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.085+0000 7fc984ff9700 1 -- 172.21.15.100:0/3196621913 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc98c0317b0 con 0x559a945e71b0 2024-05-10T11:34:53.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.085+0000 7fc9adc5fb80 10 client.18672 fetch_fsmap learned FSMap version 39 2024-05-10T11:34:53.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.085+0000 7fc9adc5fb80 1 -- 172.21.15.100:0/3196621913 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559a945ddae0 con 0x559a945e71b0 2024-05-10T11:34:53.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.085+0000 7fc984ff9700 1 -- 172.21.15.100:0/3196621913 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc98c021030 con 0x559a945e71b0 2024-05-10T11:34:53.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.085+0000 7fc984ff9700 1 -- 172.21.15.100:0/3196621913 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fc98c026c00 con 0x559a945e71b0 2024-05-10T11:34:53.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.085+0000 7fc9adc5fb80 10 client.18672 fetch_fsmap finished waiting for FSMap version 39 2024-05-10T11:34:53.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.085+0000 7fc9adc5fb80 10 client.18672 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-10T11:34:53.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.085+0000 7fc9adc5fb80 20 client.18672 populate_metadata read hostname 'smithi100' 2024-05-10T11:34:53.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.085+0000 7fc9adc5fb80 1 --2- 172.21.15.100:0/3196621913 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x559a94665710 0x559a94667b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:53.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.085+0000 7fc9adc5fb80 4 client.18672 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:34:53.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.085+0000 7fc9adc5fb80 1 -- 172.21.15.100:0/3196621913 --> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559a9453b3c0 con 0x559a94665710 2024-05-10T11:34:53.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.086+0000 7fc9a1168700 1 --2- 172.21.15.100:0/3196621913 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x559a94665710 0x559a94667b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:53.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.087+0000 7fc984ff9700 10 client.18672 ms_handle_connect on v2:172.21.15.188:6836/1943342620 2024-05-10T11:34:53.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.087+0000 7fc9a1168700 1 --2- 172.21.15.100:0/3196621913 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x559a94665710 0x559a94667b00 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e9e07e227b73df82 server_cookie=5919a36be00ae336 in_seq=0 out_seq=0 2024-05-10T11:34:53.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.087+0000 7fc984ff9700 1 -- 172.21.15.100:0/3196621913 <== mds.2 v2:172.21.15.188:6836/1943342620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559a9453b3c0 con 0x559a94665710 2024-05-10T11:34:53.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.087+0000 7fc984ff9700 10 client.18672 handle_command_reply: tid=0 2024-05-10T11:34:53.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.087+0000 7fc9adc5fb80 1 client.18672 shutdown 2024-05-10T11:34:53.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.087+0000 7fc9adc5fb80 1 -- 172.21.15.100:0/3196621913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a945e71b0 msgr2=0x559a945f4020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:53.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.087+0000 7fc9adc5fb80 1 --2- 172.21.15.100:0/3196621913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a945e71b0 0x559a945f4020 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fc98c00b000 tx=0x7fc98c0056a0 comp rx=0 tx=0).stop 2024-05-10T11:34:53.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.087+0000 7fc9adc5fb80 1 -- 172.21.15.100:0/3196621913 shutdown_connections 2024-05-10T11:34:53.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.087+0000 7fc9adc5fb80 1 --2- 172.21.15.100:0/3196621913 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x559a94665710 0x559a94667b00 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:53.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.087+0000 7fc9adc5fb80 1 --2- 172.21.15.100:0/3196621913 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a945e7ad0 0x559a946023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:53.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.087+0000 7fc9adc5fb80 1 --2- 172.21.15.100:0/3196621913 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a945e1d50 0x559a945f3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:53.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.087+0000 7fc9adc5fb80 1 --2- 172.21.15.100:0/3196621913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a945e71b0 0x559a945f4020 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:53.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.087+0000 7fc9adc5fb80 1 -- 172.21.15.100:0/3196621913 >> 172.21.15.100:0/3196621913 conn(0x559a9450db70 msgr2=0x559a945ec080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:34:53.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.087+0000 7fc984ff9700 0 client.18672 ms_handle_reset on v2:172.21.15.188:6836/1943342620 2024-05-10T11:34:53.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.087+0000 7fc9adc5fb80 1 -- 172.21.15.100:0/3196621913 shutdown_connections 2024-05-10T11:34:53.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.087+0000 7fc9adc5fb80 1 -- 172.21.15.100:0/3196621913 wait complete. 2024-05-10T11:34:53.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.088+0000 7fc9adc5fb80 20 client.18672 trim_cache size 0 max 16384 2024-05-10T11:34:53.096 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:34:53.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.088+0000 7fc9a33cc700 1 -- 172.21.15.100:0/3596167397 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc9880b2a80 msgr2=0x7fc9880b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:53.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.088+0000 7fc9a33cc700 1 --2- 172.21.15.100:0/3596167397 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc9880b2a80 0x7fc9880b4f40 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fc99c079930 tx=0x7fc98c00b410 comp rx=0 tx=0).stop 2024-05-10T11:34:53.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.088+0000 7fc9a33cc700 1 -- 172.21.15.100:0/3596167397 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc99c102390 msgr2=0x7fc99c1a02d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:53.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.088+0000 7fc9a33cc700 1 --2- 172.21.15.100:0/3596167397 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc99c102390 0x7fc99c1a02d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc99800b650 tx=0x7fc99800bfc0 comp rx=0 tx=0).stop 2024-05-10T11:34:53.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.088+0000 7fc9a33cc700 1 -- 172.21.15.100:0/3596167397 shutdown_connections 2024-05-10T11:34:53.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.088+0000 7fc9a33cc700 1 --2- 172.21.15.100:0/3596167397 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc9880b2a80 0x7fc9880b4f40 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:53.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.088+0000 7fc9a33cc700 1 --2- 172.21.15.100:0/3596167397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc99c108400 0x7fc99c1a4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:53.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.088+0000 7fc9a33cc700 1 --2- 172.21.15.100:0/3596167397 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc99c102390 0x7fc99c1a02d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:53.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.088+0000 7fc9a33cc700 1 --2- 172.21.15.100:0/3596167397 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc99c102d50 0x7fc99c1a0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:53.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.088+0000 7fc9a33cc700 1 -- 172.21.15.100:0/3596167397 >> 172.21.15.100:0/3596167397 conn(0x7fc99c079310 msgr2=0x7fc99c06bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:34:53.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.089+0000 7fc9a33cc700 1 -- 172.21.15.100:0/3596167397 shutdown_connections 2024-05-10T11:34:53.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:53.089+0000 7fc9a33cc700 1 -- 172.21.15.100:0/3596167397 wait complete. 2024-05-10T11:34:53.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:34:53 smithi100 ceph-mon[30296]: pgmap v916: 129 pgs: 129 active+clean; 419 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 886 KiB/s rd, 4.3 MiB/s wr, 313 op/s 2024-05-10T11:34:53.915 INFO:tasks.workunit.client.0.smithi100.stdout: 10 28689 9.45 MB/sec execute 47 sec latency 77.256 ms 2024-05-10T11:34:54.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:34:53 smithi178 ceph-mon[33359]: pgmap v916: 129 pgs: 129 active+clean; 419 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 886 KiB/s rd, 4.3 MiB/s wr, 313 op/s 2024-05-10T11:34:54.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:34:53 smithi188 ceph-mon[34075]: pgmap v916: 129 pgs: 129 active+clean; 419 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 886 KiB/s rd, 4.3 MiB/s wr, 313 op/s 2024-05-10T11:34:54.381 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-10T11:34:54.740 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:34:54.915 INFO:tasks.workunit.client.0.smithi100.stdout: 10 28863 9.46 MB/sec execute 48 sec latency 102.734 ms 2024-05-10T11:34:55.773 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.773+0000 7f3772dda640 1 -- 172.21.15.100:0/2390685447 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f376c1045e0 msgr2=0x7f376c10ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:55.774 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.773+0000 7f3772dda640 1 --2- 172.21.15.100:0/2390685447 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f376c1045e0 0x7f376c10ae70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3764009be0 tx=0x7f376402a6c0 comp rx=0 tx=0).stop 2024-05-10T11:34:55.774 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.773+0000 7f3772dda640 1 -- 172.21.15.100:0/2390685447 shutdown_connections 2024-05-10T11:34:55.774 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.773+0000 7f3772dda640 1 --2- 172.21.15.100:0/2390685447 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f376c1045e0 0x7f376c10ae70 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:55.774 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.773+0000 7f3772dda640 1 --2- 172.21.15.100:0/2390685447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f376c103c20 0x7f376c1040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:55.774 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.773+0000 7f3772dda640 1 --2- 172.21.15.100:0/2390685447 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f376c102a30 0x7f376c102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:55.774 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.773+0000 7f3772dda640 1 -- 172.21.15.100:0/2390685447 >> 172.21.15.100:0/2390685447 conn(0x7f376c0fdfa0 msgr2=0x7f376c1003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:34:55.774 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.773+0000 7f3772dda640 1 -- 172.21.15.100:0/2390685447 shutdown_connections 2024-05-10T11:34:55.774 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.773+0000 7f3772dda640 1 -- 172.21.15.100:0/2390685447 wait complete. 2024-05-10T11:34:55.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.774+0000 7f3772dda640 1 Processor -- start 2024-05-10T11:34:55.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.774+0000 7f3772dda640 1 -- start start 2024-05-10T11:34:55.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.774+0000 7f3772dda640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f376c102a30 0x7f376c076760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:55.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.774+0000 7f3772dda640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f376c103c20 0x7f376c074da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:55.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.775+0000 7f3772dda640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f376c1045e0 0x7f376c0752e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:55.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.775+0000 7f3772dda640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f376c10e410 con 0x7f376c1045e0 2024-05-10T11:34:55.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.775+0000 7f3772dda640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f376c10e290 con 0x7f376c103c20 2024-05-10T11:34:55.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.775+0000 7f3772dda640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f376c10e590 con 0x7f376c102a30 2024-05-10T11:34:55.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.775+0000 7f376bfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f376c103c20 0x7f376c074da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:55.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.775+0000 7f376bfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f376c103c20 0x7f376c074da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:51244/0 (socket says 172.21.15.100:51244) 2024-05-10T11:34:55.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.775+0000 7f376bfff640 1 -- 172.21.15.100:0/275217944 learned_addr learned my addr 172.21.15.100:0/275217944 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:34:55.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.775+0000 7f3770b76640 1 --2- 172.21.15.100:0/275217944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f376c102a30 0x7f376c076760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:55.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.775+0000 7f3771377640 1 --2- 172.21.15.100:0/275217944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f376c1045e0 0x7f376c0752e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:55.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.775+0000 7f376bfff640 1 -- 172.21.15.100:0/275217944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f376c102a30 msgr2=0x7f376c076760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:55.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.775+0000 7f376bfff640 1 --2- 172.21.15.100:0/275217944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f376c102a30 0x7f376c076760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:55.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.775+0000 7f376bfff640 1 -- 172.21.15.100:0/275217944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f376c1045e0 msgr2=0x7f376c0752e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:55.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.775+0000 7f376bfff640 1 --2- 172.21.15.100:0/275217944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f376c1045e0 0x7f376c0752e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:55.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.775+0000 7f376bfff640 1 -- 172.21.15.100:0/275217944 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f376c075820 con 0x7f376c103c20 2024-05-10T11:34:55.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.775+0000 7f376bfff640 1 --2- 172.21.15.100:0/275217944 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f376c103c20 0x7f376c074da0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f375800e9c0 tx=0x7f375800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:34:55.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.776+0000 7f3769ffb640 1 -- 172.21.15.100:0/275217944 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375800cde0 con 0x7f376c103c20 2024-05-10T11:34:55.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.776+0000 7f3769ffb640 1 -- 172.21.15.100:0/275217944 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f37580035f0 con 0x7f376c103c20 2024-05-10T11:34:55.778 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.776+0000 7f3769ffb640 1 -- 172.21.15.100:0/275217944 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3758010690 con 0x7f376c103c20 2024-05-10T11:34:55.778 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.776+0000 7f3772dda640 1 -- 172.21.15.100:0/275217944 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f376c075b10 con 0x7f376c103c20 2024-05-10T11:34:55.778 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.776+0000 7f3772dda640 1 -- 172.21.15.100:0/275217944 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f376c1a7330 con 0x7f376c103c20 2024-05-10T11:34:55.779 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.779+0000 7f3769ffb640 1 -- 172.21.15.100:0/275217944 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f3758004000 con 0x7f376c103c20 2024-05-10T11:34:55.779 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.779+0000 7f3775bc8740 1 Processor -- start 2024-05-10T11:34:55.779 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.780+0000 7f3775bc8740 1 -- start start 2024-05-10T11:34:55.782 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.780+0000 7f3775bc8740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef09916db0 0x55ef099171d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:55.782 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.780+0000 7f3775bc8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ef099177a0 0x55ef0991c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:55.782 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.780+0000 7f3775bc8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef0991c910 0x55ef0991eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:55.782 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.780+0000 7f3775bc8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55ef0987b5d0 con 0x55ef099177a0 2024-05-10T11:34:55.782 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.780+0000 7f3775bc8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ef09834890 con 0x55ef0991c910 2024-05-10T11:34:55.782 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.780+0000 7f3775bc8740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55ef09877590 con 0x55ef09916db0 2024-05-10T11:34:55.782 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.780+0000 7f3771377640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef09916db0 0x55ef099171d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:55.782 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.780+0000 7f3770b76640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ef099177a0 0x55ef0991c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:55.782 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.780+0000 7f3771377640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef09916db0 0x55ef099171d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:49164/0 (socket says 172.21.15.100:49164) 2024-05-10T11:34:55.782 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.780+0000 7f3771377640 1 -- 172.21.15.100:0/50030102 learned_addr learned my addr 172.21.15.100:0/50030102 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:34:55.782 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.780+0000 7f376bfff640 1 --2- 172.21.15.100:0/50030102 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef0991c910 0x55ef0991eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:55.782 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.781+0000 7f3771377640 1 -- 172.21.15.100:0/50030102 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef0991c910 msgr2=0x55ef0991eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:55.782 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.781+0000 7f3771377640 1 --2- 172.21.15.100:0/50030102 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef0991c910 0x55ef0991eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:55.783 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.781+0000 7f3771377640 1 -- 172.21.15.100:0/50030102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ef099177a0 msgr2=0x55ef0991c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:55.783 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.781+0000 7f3771377640 1 --2- 172.21.15.100:0/50030102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ef099177a0 0x55ef0991c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:55.783 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.781+0000 7f3771377640 1 -- 172.21.15.100:0/50030102 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef0958d210 con 0x55ef09916db0 2024-05-10T11:34:55.783 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.781+0000 7f3770b76640 1 --2- 172.21.15.100:0/50030102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ef099177a0 0x55ef0991c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:34:55.783 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.781+0000 7f3769ffb640 1 --2- 172.21.15.100:0/275217944 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f375c0b2920 0x7f375c0b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:55.783 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.781+0000 7f3769ffb640 1 -- 172.21.15.100:0/275217944 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f37580e2790 con 0x7f376c103c20 2024-05-10T11:34:55.783 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.781+0000 7f3770b76640 1 --2- 172.21.15.100:0/275217944 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f375c0b2920 0x7f375c0b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:55.783 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.781+0000 7f3771377640 1 --2- 172.21.15.100:0/50030102 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef09916db0 0x55ef099171d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3764034040 tx=0x7f3764030410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74441d0b768f0754 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:34:55.783 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.781+0000 7f3756ffd640 1 -- 172.21.15.100:0/50030102 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3764041030 con 0x55ef09916db0 2024-05-10T11:34:55.783 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.782+0000 7f3756ffd640 1 -- 172.21.15.100:0/50030102 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3764037040 con 0x55ef09916db0 2024-05-10T11:34:55.785 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.782+0000 7f3756ffd640 1 -- 172.21.15.100:0/50030102 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3764036040 con 0x55ef09916db0 2024-05-10T11:34:55.785 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.783+0000 7f3775bc8740 1 -- 172.21.15.100:0/50030102 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef09916db0 msgr2=0x55ef099171d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:55.785 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.783+0000 7f3775bc8740 1 --2- 172.21.15.100:0/50030102 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef09916db0 0x55ef099171d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3764034040 tx=0x7f3764030410 comp rx=0 tx=0).stop 2024-05-10T11:34:55.785 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.783+0000 7f3770b76640 1 --2- 172.21.15.100:0/275217944 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f375c0b2920 0x7f375c0b4de0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x55ef09917ba0 tx=0x7f376000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:34:55.785 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.783+0000 7f3775bc8740 1 -- 172.21.15.100:0/50030102 shutdown_connections 2024-05-10T11:34:55.785 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.783+0000 7f3775bc8740 1 --2- 172.21.15.100:0/50030102 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef0991c910 0x55ef0991eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:55.785 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.783+0000 7f3775bc8740 1 --2- 172.21.15.100:0/50030102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ef099177a0 0x55ef0991c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:55.785 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.783+0000 7f3775bc8740 1 --2- 172.21.15.100:0/50030102 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef09916db0 0x55ef099171d0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:55.785 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.783+0000 7f3775bc8740 1 -- 172.21.15.100:0/50030102 >> 172.21.15.100:0/50030102 conn(0x55ef09912420 msgr2=0x55ef09912820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:34:55.785 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.784+0000 7f3775bc8740 1 -- 172.21.15.100:0/50030102 shutdown_connections 2024-05-10T11:34:55.785 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.784+0000 7f3775bc8740 1 -- 172.21.15.100:0/50030102 wait complete. 2024-05-10T11:34:55.785 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.784+0000 7f3775bc8740 1 -- start start 2024-05-10T11:34:55.785 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.784+0000 7f3775bc8740 1 Processor -- start 2024-05-10T11:34:55.786 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.785+0000 7f3775bc8740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef09916db0 0x55ef09920fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:55.786 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.785+0000 7f3775bc8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ef099177a0 0x55ef09921520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:55.786 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.785+0000 7f3775bc8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef0991c910 0x55ef09928560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:55.786 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.785+0000 7f3775bc8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55ef0991f900 con 0x55ef099177a0 2024-05-10T11:34:55.786 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.785+0000 7f3775bc8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ef09838370 con 0x55ef0991c910 2024-05-10T11:34:55.786 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.785+0000 7f3775bc8740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55ef0991fa80 con 0x55ef09916db0 2024-05-10T11:34:55.786 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.785+0000 7f3771377640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef0991c910 0x55ef09928560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:55.786 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.785+0000 7f3771377640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef0991c910 0x55ef09928560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:51254/0 (socket says 172.21.15.100:51254) 2024-05-10T11:34:55.786 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.785+0000 7f3771377640 1 -- 172.21.15.100:0/1209320566 learned_addr learned my addr 172.21.15.100:0/1209320566 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:34:55.786 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.785+0000 7f3775bc8740 10 client.0 _ll_register_callbacks cb 0x55ef0958ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:34:55.786 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.785+0000 7f3770b76640 1 --2- 172.21.15.100:0/1209320566 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef09916db0 0x55ef09920fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:55.786 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.785+0000 7f376bfff640 1 --2- 172.21.15.100:0/1209320566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ef099177a0 0x55ef09921520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:55.789 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.785+0000 7f3771377640 1 -- 172.21.15.100:0/1209320566 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef09916db0 msgr2=0x55ef09920fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:55.789 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.785+0000 7f3771377640 1 --2- 172.21.15.100:0/1209320566 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef09916db0 0x55ef09920fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:55.789 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.785+0000 7f3771377640 1 -- 172.21.15.100:0/1209320566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ef099177a0 msgr2=0x55ef09921520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:55.789 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.785+0000 7f3771377640 1 --2- 172.21.15.100:0/1209320566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ef099177a0 0x55ef09921520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:55.789 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.785+0000 7f3771377640 1 -- 172.21.15.100:0/1209320566 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef09659620 con 0x55ef0991c910 2024-05-10T11:34:55.789 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.785+0000 7f376bfff640 1 --2- 172.21.15.100:0/1209320566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ef099177a0 0x55ef09921520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:34:55.789 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.786+0000 7f3771377640 1 --2- 172.21.15.100:0/1209320566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef0991c910 0x55ef09928560 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3764031040 tx=0x7f3764005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25bdbf3501b20e55 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:34:55.789 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.786+0000 7f374bfff640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:34:55.790 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.786+0000 7f374bfff640 1 -- 172.21.15.100:0/1209320566 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f376404a030 con 0x55ef0991c910 2024-05-10T11:34:55.790 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.786+0000 7f374bfff640 1 -- 172.21.15.100:0/1209320566 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f37640305c0 con 0x55ef0991c910 2024-05-10T11:34:55.790 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.786+0000 7f3775bc8740 1 -- 172.21.15.100:0/1209320566 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3764037040 con 0x55ef0991c910 2024-05-10T11:34:55.790 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.786+0000 7f374bfff640 1 -- 172.21.15.100:0/1209320566 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f3764055050 con 0x55ef0991c910 2024-05-10T11:34:55.790 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.786+0000 7f3775bc8740 10 client.27995 fetch_fsmap learned FSMap version 40 2024-05-10T11:34:55.790 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.787+0000 7f3775bc8740 1 -- 172.21.15.100:0/1209320566 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55ef0992b520 con 0x55ef0991c910 2024-05-10T11:34:55.790 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.787+0000 7f374bfff640 1 -- 172.21.15.100:0/1209320566 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3764037040 con 0x55ef0991c910 2024-05-10T11:34:55.790 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.787+0000 7f374bfff640 1 -- 172.21.15.100:0/1209320566 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 40) ==== 8350+0+0 (secure 0 0 0) 0x7f376403c040 con 0x55ef0991c910 2024-05-10T11:34:55.790 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.787+0000 7f3775bc8740 10 client.27995 fetch_fsmap finished waiting for FSMap version 40 2024-05-10T11:34:55.790 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.787+0000 7f3775bc8740 10 client.27995 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:34:55.790 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.787+0000 7f3775bc8740 20 client.27995 populate_metadata read hostname 'smithi100' 2024-05-10T11:34:55.790 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.787+0000 7f3775bc8740 1 --2- 172.21.15.100:0/1209320566 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ef0992c870 0x55ef0992ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:55.790 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.787+0000 7f3775bc8740 4 client.27995 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:34:55.790 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.787+0000 7f3775bc8740 1 -- 172.21.15.100:0/1209320566 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef098891e0 con 0x55ef0992c870 2024-05-10T11:34:55.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.787+0000 7f3770b76640 1 --2- 172.21.15.100:0/1209320566 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ef0992c870 0x55ef0992ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:55.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.788+0000 7f374bfff640 10 client.27995 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:34:55.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.788+0000 7f3770b76640 1 --2- 172.21.15.100:0/1209320566 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ef0992c870 0x55ef0992ec60 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53e1f73960d53fc3 server_cookie=2dd5059cc173670c in_seq=0 out_seq=0 2024-05-10T11:34:55.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.788+0000 7f374bfff640 1 -- 172.21.15.100:0/1209320566 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ef098891e0 con 0x55ef0992c870 2024-05-10T11:34:55.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.788+0000 7f374bfff640 10 client.27995 handle_command_reply: tid=0 2024-05-10T11:34:55.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.789+0000 7f3775bc8740 1 client.27995 shutdown 2024-05-10T11:34:55.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.789+0000 7f3775bc8740 1 -- 172.21.15.100:0/1209320566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef0991c910 msgr2=0x55ef09928560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:55.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.789+0000 7f3775bc8740 1 --2- 172.21.15.100:0/1209320566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef0991c910 0x55ef09928560 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3764031040 tx=0x7f3764005fd0 comp rx=0 tx=0).stop 2024-05-10T11:34:55.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.789+0000 7f3775bc8740 1 -- 172.21.15.100:0/1209320566 shutdown_connections 2024-05-10T11:34:55.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.789+0000 7f3775bc8740 1 --2- 172.21.15.100:0/1209320566 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ef0992c870 0x55ef0992ec60 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:55.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.789+0000 7f3775bc8740 1 --2- 172.21.15.100:0/1209320566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef0991c910 0x55ef09928560 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:55.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.789+0000 7f374bfff640 0 client.27995 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:34:55.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.789+0000 7f3775bc8740 1 --2- 172.21.15.100:0/1209320566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ef099177a0 0x55ef09921520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:55.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.789+0000 7f3775bc8740 1 --2- 172.21.15.100:0/1209320566 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef09916db0 0x55ef09920fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:55.792 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.789+0000 7f3775bc8740 1 -- 172.21.15.100:0/1209320566 >> 172.21.15.100:0/1209320566 conn(0x55ef09912420 msgr2=0x55ef09911580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:34:55.792 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.789+0000 7f3775bc8740 1 -- 172.21.15.100:0/1209320566 shutdown_connections 2024-05-10T11:34:55.792 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.789+0000 7f3775bc8740 1 -- 172.21.15.100:0/1209320566 wait complete. 2024-05-10T11:34:55.792 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.789+0000 7f3775bc8740 20 client.27995 trim_cache size 0 max 16384 2024-05-10T11:34:55.820 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.814+0000 7f3775bc8740 1 Processor -- start 2024-05-10T11:34:55.820 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.814+0000 7f3775bc8740 1 -- start start 2024-05-10T11:34:55.820 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.814+0000 7f3775bc8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef09916db0 0x55ef09915a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:55.820 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.814+0000 7f3775bc8740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef099177a0 0x55ef09928520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:55.820 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.814+0000 7f3775bc8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ef0991c910 0x55ef099334b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:55.820 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.814+0000 7f3775bc8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f376c1b4090 con 0x55ef0991c910 2024-05-10T11:34:55.820 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.814+0000 7f3775bc8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f376c1b1930 con 0x55ef09916db0 2024-05-10T11:34:55.820 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.814+0000 7f3775bc8740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f376c1af780 con 0x55ef099177a0 2024-05-10T11:34:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.814+0000 7f3770b76640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef09916db0 0x55ef09915a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.815+0000 7f3770b76640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef09916db0 0x55ef09915a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:51260/0 (socket says 172.21.15.100:51260) 2024-05-10T11:34:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.815+0000 7f3770b76640 1 -- 172.21.15.100:0/2858376839 learned_addr learned my addr 172.21.15.100:0/2858376839 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:34:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.815+0000 7f376bfff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef099177a0 0x55ef09928520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.815+0000 7f3771377640 1 --2- 172.21.15.100:0/2858376839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ef0991c910 0x55ef099334b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.815+0000 7f3770b76640 1 -- 172.21.15.100:0/2858376839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef099177a0 msgr2=0x55ef09928520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.815+0000 7f3770b76640 1 --2- 172.21.15.100:0/2858376839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef099177a0 0x55ef09928520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.815+0000 7f3770b76640 1 -- 172.21.15.100:0/2858376839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ef0991c910 msgr2=0x55ef099334b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.815+0000 7f3770b76640 1 --2- 172.21.15.100:0/2858376839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ef0991c910 0x55ef099334b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.815+0000 7f3770b76640 1 -- 172.21.15.100:0/2858376839 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef099169f0 con 0x55ef09916db0 2024-05-10T11:34:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.815+0000 7f3770b76640 1 --2- 172.21.15.100:0/2858376839 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef09916db0 0x55ef09915a40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3760005fd0 tx=0x7f3760019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4d4bd81e27bb861 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:34:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.815+0000 7f37567fc640 1 -- 172.21.15.100:0/2858376839 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f376001ed50 con 0x55ef09916db0 2024-05-10T11:34:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.815+0000 7f37567fc640 1 -- 172.21.15.100:0/2858376839 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f376002f070 con 0x55ef09916db0 2024-05-10T11:34:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.815+0000 7f37567fc640 1 -- 172.21.15.100:0/2858376839 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f376001d8a0 con 0x55ef09916db0 2024-05-10T11:34:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.816+0000 7f3775bc8740 1 -- 172.21.15.100:0/2858376839 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef09916db0 msgr2=0x55ef09915a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.816+0000 7f3775bc8740 1 --2- 172.21.15.100:0/2858376839 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef09916db0 0x55ef09915a40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3760005fd0 tx=0x7f3760019000 comp rx=0 tx=0).stop 2024-05-10T11:34:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.816+0000 7f3775bc8740 1 -- 172.21.15.100:0/2858376839 shutdown_connections 2024-05-10T11:34:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.816+0000 7f3775bc8740 1 --2- 172.21.15.100:0/2858376839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ef0991c910 0x55ef099334b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.816+0000 7f3775bc8740 1 --2- 172.21.15.100:0/2858376839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef099177a0 0x55ef09928520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.816+0000 7f3775bc8740 1 --2- 172.21.15.100:0/2858376839 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef09916db0 0x55ef09915a40 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.816+0000 7f3775bc8740 1 -- 172.21.15.100:0/2858376839 >> 172.21.15.100:0/2858376839 conn(0x55ef09912420 msgr2=0x55ef09915330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:34:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.816+0000 7f3775bc8740 1 -- 172.21.15.100:0/2858376839 shutdown_connections 2024-05-10T11:34:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.816+0000 7f3775bc8740 1 -- 172.21.15.100:0/2858376839 wait complete. 2024-05-10T11:34:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.817+0000 7f3775bc8740 1 -- start start 2024-05-10T11:34:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.817+0000 7f3775bc8740 1 Processor -- start 2024-05-10T11:34:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.817+0000 7f3775bc8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef099177a0 0x55ef09925ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.817+0000 7f3775bc8740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef0991c910 0x55ef099263e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.817+0000 7f3775bc8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ef099273f0 0x55ef09927870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.817+0000 7f3775bc8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f376c1afdf0 con 0x55ef099273f0 2024-05-10T11:34:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.817+0000 7f3775bc8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f376c1b1260 con 0x55ef099177a0 2024-05-10T11:34:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.817+0000 7f3775bc8740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f376c1b04c0 con 0x55ef0991c910 2024-05-10T11:34:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.817+0000 7f3775bc8740 10 client.0 _ll_register_callbacks cb 0x7f376000b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:34:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.818+0000 7f3771377640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ef099273f0 0x55ef09927870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.818+0000 7f3771377640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ef099273f0 0x55ef09927870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45780/0 (socket says 172.21.15.100:45780) 2024-05-10T11:34:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.818+0000 7f3771377640 1 -- 172.21.15.100:0/162687086 learned_addr learned my addr 172.21.15.100:0/162687086 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:34:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.818+0000 7f376bfff640 1 --2- 172.21.15.100:0/162687086 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef0991c910 0x55ef099263e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.818+0000 7f3770b76640 1 --2- 172.21.15.100:0/162687086 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef099177a0 0x55ef09925ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.818+0000 7f3771377640 1 -- 172.21.15.100:0/162687086 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef0991c910 msgr2=0x55ef099263e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.818+0000 7f3771377640 1 --2- 172.21.15.100:0/162687086 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef0991c910 0x55ef099263e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.818+0000 7f3771377640 1 -- 172.21.15.100:0/162687086 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef099177a0 msgr2=0x55ef09925ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.818+0000 7f3771377640 1 --2- 172.21.15.100:0/162687086 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef099177a0 0x55ef09925ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.818+0000 7f3771377640 1 -- 172.21.15.100:0/162687086 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef0990def0 con 0x55ef099273f0 2024-05-10T11:34:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.818+0000 7f3770b76640 1 --2- 172.21.15.100:0/162687086 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef099177a0 0x55ef09925ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:34:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.818+0000 7f3771377640 1 --2- 172.21.15.100:0/162687086 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ef099273f0 0x55ef09927870 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f3764009300 tx=0x7f3764031650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4a925bb4737a7cc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:34:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.819+0000 7f374bfff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:34:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.819+0000 7f374bfff640 1 -- 172.21.15.100:0/162687086 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f376404a040 con 0x55ef099273f0 2024-05-10T11:34:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.819+0000 7f374bfff640 1 -- 172.21.15.100:0/162687086 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f3764030ca0 con 0x55ef099273f0 2024-05-10T11:34:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.819+0000 7f3775bc8740 1 -- 172.21.15.100:0/162687086 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f376002f070 con 0x55ef099273f0 2024-05-10T11:34:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.819+0000 7f374bfff640 1 -- 172.21.15.100:0/162687086 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f3764041040 con 0x55ef099273f0 2024-05-10T11:34:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.819+0000 7f3775bc8740 10 client.18690 fetch_fsmap learned FSMap version 40 2024-05-10T11:34:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.819+0000 7f3775bc8740 1 -- 172.21.15.100:0/162687086 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55ef099130f0 con 0x55ef099273f0 2024-05-10T11:34:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.819+0000 7f374bfff640 1 -- 172.21.15.100:0/162687086 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f376002f070 con 0x55ef099273f0 2024-05-10T11:34:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.820+0000 7f374bfff640 1 -- 172.21.15.100:0/162687086 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) ==== 8350+0+0 (secure 0 0 0) 0x7f376403c040 con 0x55ef099273f0 2024-05-10T11:34:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.820+0000 7f3775bc8740 10 client.18690 fetch_fsmap finished waiting for FSMap version 40 2024-05-10T11:34:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.820+0000 7f3775bc8740 10 client.18690 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:34:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.820+0000 7f3775bc8740 20 client.18690 populate_metadata read hostname 'smithi100' 2024-05-10T11:34:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.820+0000 7f3775bc8740 1 --2- 172.21.15.100:0/162687086 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ef09997c00 0x55ef0999a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.820+0000 7f3775bc8740 4 client.18690 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:34:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.820+0000 7f3775bc8740 1 -- 172.21.15.100:0/162687086 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ef09920dd0 con 0x55ef09997c00 2024-05-10T11:34:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.820+0000 7f3770b76640 1 --2- 172.21.15.100:0/162687086 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ef09997c00 0x55ef0999a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:55.826 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.821+0000 7f3770b76640 1 --2- 172.21.15.100:0/162687086 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ef09997c00 0x55ef0999a020 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=495d9d871cacefe0 server_cookie=a4159399f8c72e04 in_seq=0 out_seq=0 2024-05-10T11:34:55.826 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.821+0000 7f374bfff640 10 client.18690 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:34:55.916 INFO:tasks.workunit.client.0.smithi100.stdout: 10 29024 9.37 MB/sec execute 49 sec latency 127.755 ms 2024-05-10T11:34:55.938 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.937+0000 7f374bfff640 1 -- 172.21.15.100:0/162687086 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ef09920dd0 con 0x55ef09997c00 2024-05-10T11:34:55.938 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.937+0000 7f374bfff640 10 client.18690 handle_command_reply: tid=0 2024-05-10T11:34:55.938 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.937+0000 7f3775bc8740 1 client.18690 shutdown 2024-05-10T11:34:55.938 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.937+0000 7f3775bc8740 1 -- 172.21.15.100:0/162687086 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ef099273f0 msgr2=0x55ef09927870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:55.938 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.937+0000 7f3775bc8740 1 --2- 172.21.15.100:0/162687086 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ef099273f0 0x55ef09927870 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f3764009300 tx=0x7f3764031650 comp rx=0 tx=0).stop 2024-05-10T11:34:55.938 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.938+0000 7f3775bc8740 1 -- 172.21.15.100:0/162687086 shutdown_connections 2024-05-10T11:34:55.938 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.938+0000 7f3775bc8740 1 --2- 172.21.15.100:0/162687086 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ef09997c00 0x55ef0999a020 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:55.938 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.938+0000 7f3775bc8740 1 --2- 172.21.15.100:0/162687086 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ef099273f0 0x55ef09927870 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:55.939 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.938+0000 7f3775bc8740 1 --2- 172.21.15.100:0/162687086 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef0991c910 0x55ef099263e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:55.939 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.938+0000 7f3775bc8740 1 --2- 172.21.15.100:0/162687086 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef099177a0 0x55ef09925ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:55.939 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.938+0000 7f374bfff640 0 client.18690 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:34:55.939 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.938+0000 7f3775bc8740 1 -- 172.21.15.100:0/162687086 >> 172.21.15.100:0/162687086 conn(0x55ef09912420 msgr2=0x55ef09910cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:34:55.939 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.938+0000 7f3775bc8740 1 -- 172.21.15.100:0/162687086 shutdown_connections 2024-05-10T11:34:55.939 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.938+0000 7f3775bc8740 1 -- 172.21.15.100:0/162687086 wait complete. 2024-05-10T11:34:55.939 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.938+0000 7f3775bc8740 20 client.18690 trim_cache size 0 max 16384 2024-05-10T11:34:55.939 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:34:55.939 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:34:55.939 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:34:55.939 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:34:55.940 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.939+0000 7f3772dda640 1 -- 172.21.15.100:0/275217944 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f375c0b2920 msgr2=0x7f375c0b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:55.940 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.939+0000 7f3772dda640 1 --2- 172.21.15.100:0/275217944 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f375c0b2920 0x7f375c0b4de0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x55ef09917ba0 tx=0x7f376000b390 comp rx=0 tx=0).stop 2024-05-10T11:34:55.940 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.939+0000 7f3772dda640 1 -- 172.21.15.100:0/275217944 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f376c103c20 msgr2=0x7f376c074da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:55.940 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.939+0000 7f3772dda640 1 --2- 172.21.15.100:0/275217944 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f376c103c20 0x7f376c074da0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f375800e9c0 tx=0x7f375800ee80 comp rx=0 tx=0).stop 2024-05-10T11:34:55.940 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.939+0000 7f3772dda640 1 -- 172.21.15.100:0/275217944 shutdown_connections 2024-05-10T11:34:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.939+0000 7f3772dda640 1 --2- 172.21.15.100:0/275217944 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f375c0b2920 0x7f375c0b4de0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.939+0000 7f3772dda640 1 --2- 172.21.15.100:0/275217944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f376c1045e0 0x7f376c0752e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.939+0000 7f3772dda640 1 --2- 172.21.15.100:0/275217944 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f376c103c20 0x7f376c074da0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.939+0000 7f3772dda640 1 --2- 172.21.15.100:0/275217944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f376c102a30 0x7f376c076760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.939+0000 7f3772dda640 1 -- 172.21.15.100:0/275217944 >> 172.21.15.100:0/275217944 conn(0x7f376c0fdfa0 msgr2=0x7f376c106e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:34:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.939+0000 7f3772dda640 1 -- 172.21.15.100:0/275217944 shutdown_connections 2024-05-10T11:34:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:34:55.939+0000 7f3772dda640 1 -- 172.21.15.100:0/275217944 wait complete. 2024-05-10T11:34:55.965 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:34:55 smithi100 ceph-mon[30296]: pgmap v917: 129 pgs: 129 active+clean; 421 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 810 KiB/s rd, 4.3 MiB/s wr, 302 op/s 2024-05-10T11:34:55.965 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:34:55 smithi100 ceph-mon[30296]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T11:34:56.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:34:55 smithi178 ceph-mon[33359]: pgmap v917: 129 pgs: 129 active+clean; 421 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 810 KiB/s rd, 4.3 MiB/s wr, 302 op/s 2024-05-10T11:34:56.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:34:55 smithi178 ceph-mon[33359]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T11:34:56.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:34:55 smithi188 ceph-mon[34075]: pgmap v917: 129 pgs: 129 active+clean; 421 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 810 KiB/s rd, 4.3 MiB/s wr, 302 op/s 2024-05-10T11:34:56.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:34:55 smithi188 ceph-mon[34075]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T11:34:56.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.858+0000 7fce3e73e700 1 -- 172.21.15.100:0/2858652394 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce3806bb90 msgr2=0x7fce3806c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:56.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.858+0000 7fce3e73e700 1 --2- 172.21.15.100:0/2858652394 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce3806bb90 0x7fce3806c010 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fce2c009990 tx=0x7fce2c009ca0 comp rx=0 tx=0).stop 2024-05-10T11:34:56.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.858+0000 7fce3e73e700 1 -- 172.21.15.100:0/2858652394 shutdown_connections 2024-05-10T11:34:56.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.858+0000 7fce3e73e700 1 --2- 172.21.15.100:0/2858652394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce3806c550 0x7fce38110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:56.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.858+0000 7fce3e73e700 1 --2- 172.21.15.100:0/2858652394 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce38109500 0x7fce381098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:56.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.858+0000 7fce3e73e700 1 --2- 172.21.15.100:0/2858652394 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce3806bb90 0x7fce3806c010 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:56.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.858+0000 7fce3e73e700 1 -- 172.21.15.100:0/2858652394 >> 172.21.15.100:0/2858652394 conn(0x7fce38079310 msgr2=0x7fce38079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:34:56.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.858+0000 7fce3e73e700 1 -- 172.21.15.100:0/2858652394 shutdown_connections 2024-05-10T11:34:56.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.858+0000 7fce3e73e700 1 -- 172.21.15.100:0/2858652394 wait complete. 2024-05-10T11:34:56.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.859+0000 7fce3e73e700 1 Processor -- start 2024-05-10T11:34:56.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.859+0000 7fce3e73e700 1 -- start start 2024-05-10T11:34:56.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.859+0000 7fce3e73e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce3806bb90 0x7fce3810a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:56.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.859+0000 7fce3e73e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce3806c550 0x7fce3810a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:56.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.859+0000 7fce3e73e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce38109500 0x7fce3810ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:56.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.859+0000 7fce3e73e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fce38113510 con 0x7fce3806c550 2024-05-10T11:34:56.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.859+0000 7fce3e73e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fce38113390 con 0x7fce38109500 2024-05-10T11:34:56.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.859+0000 7fce3e73e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fce38113690 con 0x7fce3806bb90 2024-05-10T11:34:56.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.859+0000 7fce377fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce3806c550 0x7fce3810a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:56.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.859+0000 7fce37fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce3806bb90 0x7fce3810a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:56.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.859+0000 7fce377fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce3806c550 0x7fce3810a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46868/0 (socket says 172.21.15.100:46868) 2024-05-10T11:34:56.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.859+0000 7fce37fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce3806bb90 0x7fce3810a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:59844/0 (socket says 172.21.15.100:59844) 2024-05-10T11:34:56.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.859+0000 7fce377fe700 1 -- 172.21.15.100:0/1360427810 learned_addr learned my addr 172.21.15.100:0/1360427810 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:34:56.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.859+0000 7fce3ccdb700 1 --2- 172.21.15.100:0/1360427810 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce38109500 0x7fce3810ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:56.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.860+0000 7fce377fe700 1 -- 172.21.15.100:0/1360427810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce3806bb90 msgr2=0x7fce3810a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:56.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.860+0000 7fce377fe700 1 --2- 172.21.15.100:0/1360427810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce3806bb90 0x7fce3810a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:56.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.860+0000 7fce377fe700 1 -- 172.21.15.100:0/1360427810 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce38109500 msgr2=0x7fce3810ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:56.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.860+0000 7fce377fe700 1 --2- 172.21.15.100:0/1360427810 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce38109500 0x7fce3810ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:56.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.860+0000 7fce377fe700 1 -- 172.21.15.100:0/1360427810 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce381ac370 con 0x7fce3806c550 2024-05-10T11:34:56.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.860+0000 7fce377fe700 1 --2- 172.21.15.100:0/1360427810 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce3806c550 0x7fce3810a850 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fce2c00b650 tx=0x7fce2c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:34:56.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.860+0000 7fce357fa700 1 -- 172.21.15.100:0/1360427810 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce2c038070 con 0x7fce3806c550 2024-05-10T11:34:56.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.860+0000 7fce3e73e700 1 -- 172.21.15.100:0/1360427810 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce3819ba00 con 0x7fce3806c550 2024-05-10T11:34:56.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.860+0000 7fce3e73e700 1 -- 172.21.15.100:0/1360427810 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce3819bf30 con 0x7fce3806c550 2024-05-10T11:34:56.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.860+0000 7fce357fa700 1 -- 172.21.15.100:0/1360427810 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fce2c033480 con 0x7fce3806c550 2024-05-10T11:34:56.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.860+0000 7fce357fa700 1 -- 172.21.15.100:0/1360427810 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce2c017650 con 0x7fce3806c550 2024-05-10T11:34:56.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.864+0000 7fce48fd1b80 1 Processor -- start 2024-05-10T11:34:56.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.864+0000 7fce48fd1b80 1 -- start start 2024-05-10T11:34:56.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.864+0000 7fce357fa700 1 -- 172.21.15.100:0/1360427810 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fce2c03b070 con 0x7fce3806c550 2024-05-10T11:34:56.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.864+0000 7fce48fd1b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd546f61b0 0x55dd546f6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:56.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.865+0000 7fce48fd1b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd546f6ad0 0x55dd546f0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:56.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.865+0000 7fce48fd1b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd546f0d50 0x55dd546f1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:56.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.865+0000 7fce48fd1b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55dd54642940 con 0x55dd546f6ad0 2024-05-10T11:34:56.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.865+0000 7fce48fd1b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55dd545a7da0 con 0x55dd546f61b0 2024-05-10T11:34:56.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.865+0000 7fce48fd1b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55dd5463e900 con 0x55dd546f0d50 2024-05-10T11:34:56.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.865+0000 7fce37fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd546f6ad0 0x55dd546f0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:56.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.865+0000 7fce3ccdb700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd546f61b0 0x55dd546f6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:56.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.865+0000 7fce377fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd546f0d50 0x55dd546f1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:56.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.865+0000 7fce37fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd546f6ad0 0x55dd546f0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46882/0 (socket says 172.21.15.100:46882) 2024-05-10T11:34:56.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.865+0000 7fce37fff700 1 -- 172.21.15.100:0/3518818081 learned_addr learned my addr 172.21.15.100:0/3518818081 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:34:56.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.865+0000 7fce37fff700 1 -- 172.21.15.100:0/3518818081 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd546f0d50 msgr2=0x55dd546f1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:56.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.865+0000 7fce37fff700 1 --2- 172.21.15.100:0/3518818081 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd546f0d50 0x55dd546f1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:56.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.865+0000 7fce37fff700 1 -- 172.21.15.100:0/3518818081 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd546f61b0 msgr2=0x55dd546f6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:56.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.865+0000 7fce37fff700 1 --2- 172.21.15.100:0/3518818081 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd546f61b0 0x55dd546f6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:56.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.865+0000 7fce37fff700 1 -- 172.21.15.100:0/3518818081 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd5458f8d0 con 0x55dd546f6ad0 2024-05-10T11:34:56.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.865+0000 7fce3ccdb700 1 --2- 172.21.15.100:0/3518818081 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd546f61b0 0x55dd546f6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:34:56.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.866+0000 7fce357fa700 1 --2- 172.21.15.100:0/1360427810 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fce200b2980 0x7fce200b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:56.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.866+0000 7fce37fff700 1 --2- 172.21.15.100:0/3518818081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd546f6ad0 0x55dd546f0810 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fce28009d00 tx=0x7fce28011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=666685d81021e9b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:34:56.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.866+0000 7fce357fa700 1 -- 172.21.15.100:0/1360427810 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fce2c0f4330 con 0x7fce3806c550 2024-05-10T11:34:56.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.866+0000 7fce3ccdb700 1 --2- 172.21.15.100:0/1360427810 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fce200b2980 0x7fce200b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:56.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.866+0000 7fce1e7fc700 1 -- 172.21.15.100:0/3518818081 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce28004230 con 0x55dd546f6ad0 2024-05-10T11:34:56.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.866+0000 7fce1e7fc700 1 -- 172.21.15.100:0/3518818081 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fce280043d0 con 0x55dd546f6ad0 2024-05-10T11:34:56.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.866+0000 7fce1e7fc700 1 -- 172.21.15.100:0/3518818081 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce280197e0 con 0x55dd546f6ad0 2024-05-10T11:34:56.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.867+0000 7fce48fd1b80 1 -- 172.21.15.100:0/3518818081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd546f6ad0 msgr2=0x55dd546f0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:56.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.867+0000 7fce3ccdb700 1 --2- 172.21.15.100:0/1360427810 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fce200b2980 0x7fce200b4e40 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fce300075c0 tx=0x7fce3000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:34:56.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.867+0000 7fce48fd1b80 1 --2- 172.21.15.100:0/3518818081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd546f6ad0 0x55dd546f0810 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fce28009d00 tx=0x7fce28011720 comp rx=0 tx=0).stop 2024-05-10T11:34:56.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.867+0000 7fce48fd1b80 1 -- 172.21.15.100:0/3518818081 shutdown_connections 2024-05-10T11:34:56.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.867+0000 7fce48fd1b80 1 --2- 172.21.15.100:0/3518818081 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd546f0d50 0x55dd546f1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:56.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.867+0000 7fce48fd1b80 1 --2- 172.21.15.100:0/3518818081 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd546f61b0 0x55dd546f6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:56.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.867+0000 7fce48fd1b80 1 --2- 172.21.15.100:0/3518818081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd546f6ad0 0x55dd546f0810 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:56.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.867+0000 7fce48fd1b80 1 -- 172.21.15.100:0/3518818081 >> 172.21.15.100:0/3518818081 conn(0x55dd5461cb70 msgr2=0x55dd546eb2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:34:56.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.867+0000 7fce48fd1b80 1 -- 172.21.15.100:0/3518818081 shutdown_connections 2024-05-10T11:34:56.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.867+0000 7fce48fd1b80 1 -- 172.21.15.100:0/3518818081 wait complete. 2024-05-10T11:34:56.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.868+0000 7fce48fd1b80 1 -- start start 2024-05-10T11:34:56.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.868+0000 7fce48fd1b80 1 Processor -- start 2024-05-10T11:34:56.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.868+0000 7fce48fd1b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd546f0d50 0x55dd546fc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:56.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.868+0000 7fce48fd1b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd546f61b0 0x55dd546f9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:56.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.868+0000 7fce48fd1b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd546f6ad0 0x55dd546fa050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:56.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.868+0000 7fce48fd1b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55dd546fd450 con 0x55dd546f0d50 2024-05-10T11:34:56.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.868+0000 7fce48fd1b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55dd545fbed0 con 0x55dd546f6ad0 2024-05-10T11:34:56.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.868+0000 7fce48fd1b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55dd546fd5d0 con 0x55dd546f61b0 2024-05-10T11:34:56.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.869+0000 7fce48fd1b80 10 client.0 _ll_register_callbacks cb 0x55dd545d65b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:34:56.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.869+0000 7fce377fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd546f61b0 0x55dd546f9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:56.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.869+0000 7fce377fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd546f61b0 0x55dd546f9b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:59862/0 (socket says 172.21.15.100:59862) 2024-05-10T11:34:56.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.869+0000 7fce377fe700 1 -- 172.21.15.100:0/1610673668 learned_addr learned my addr 172.21.15.100:0/1610673668 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:34:56.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.869+0000 7fce3ccdb700 1 --2- 172.21.15.100:0/1610673668 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd546f6ad0 0x55dd546fa050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:56.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.869+0000 7fce377fe700 1 -- 172.21.15.100:0/1610673668 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd546f6ad0 msgr2=0x55dd546fa050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:56.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.869+0000 7fce377fe700 1 --2- 172.21.15.100:0/1610673668 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd546f6ad0 0x55dd546fa050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:56.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.869+0000 7fce377fe700 1 -- 172.21.15.100:0/1610673668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd546f0d50 msgr2=0x55dd546fc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:56.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.869+0000 7fce37fff700 1 --2- 172.21.15.100:0/1610673668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd546f0d50 0x55dd546fc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:56.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.869+0000 7fce377fe700 1 --2- 172.21.15.100:0/1610673668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd546f0d50 0x55dd546fc770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:56.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.869+0000 7fce377fe700 1 -- 172.21.15.100:0/1610673668 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd545f9760 con 0x55dd546f61b0 2024-05-10T11:34:56.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.869+0000 7fce37fff700 1 --2- 172.21.15.100:0/1610673668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd546f0d50 0x55dd546fc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:34:56.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.869+0000 7fce377fe700 1 --2- 172.21.15.100:0/1610673668 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd546f61b0 0x55dd546f9b10 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fce2c0bb5b0 tx=0x7fce2c00bf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24fba7a2f787875a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:34:56.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.869+0000 7fce177fe700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:34:56.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.869+0000 7fce177fe700 1 -- 172.21.15.100:0/1610673668 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce2c0bb3f0 con 0x55dd546f61b0 2024-05-10T11:34:56.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.870+0000 7fce177fe700 1 -- 172.21.15.100:0/1610673668 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fce2c0094b0 con 0x55dd546f61b0 2024-05-10T11:34:56.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.870+0000 7fce48fd1b80 1 -- 172.21.15.100:0/1610673668 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce280043d0 con 0x55dd546f61b0 2024-05-10T11:34:56.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.870+0000 7fce177fe700 1 -- 172.21.15.100:0/1610673668 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fce2c0bad90 con 0x55dd546f61b0 2024-05-10T11:34:56.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.870+0000 7fce48fd1b80 10 client.28108 fetch_fsmap learned FSMap version 40 2024-05-10T11:34:56.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.870+0000 7fce48fd1b80 1 -- 172.21.15.100:0/1610673668 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55dd546fd500 con 0x55dd546f61b0 2024-05-10T11:34:56.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.870+0000 7fce177fe700 1 -- 172.21.15.100:0/1610673668 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fce280043d0 con 0x55dd546f61b0 2024-05-10T11:34:56.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.870+0000 7fce177fe700 1 -- 172.21.15.100:0/1610673668 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 40) ==== 8350+0+0 (secure 0 0 0) 0x7fce2c0c3f60 con 0x55dd546f61b0 2024-05-10T11:34:56.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.870+0000 7fce48fd1b80 10 client.28108 fetch_fsmap finished waiting for FSMap version 40 2024-05-10T11:34:56.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.870+0000 7fce48fd1b80 10 client.28108 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-10T11:34:56.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.870+0000 7fce48fd1b80 20 client.28108 populate_metadata read hostname 'smithi100' 2024-05-10T11:34:56.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.870+0000 7fce48fd1b80 1 --2- 172.21.15.100:0/1610673668 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55dd5470a040 0x55dd5470c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:56.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.870+0000 7fce48fd1b80 4 client.28108 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:34:56.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.870+0000 7fce48fd1b80 1 -- 172.21.15.100:0/1610673668 --> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd54650320 con 0x55dd5470a040 2024-05-10T11:34:56.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.870+0000 7fce37fff700 1 --2- 172.21.15.100:0/1610673668 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55dd5470a040 0x55dd5470c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:56.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.871+0000 7fce37fff700 1 --2- 172.21.15.100:0/1610673668 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55dd5470a040 0x55dd5470c4c0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e91fe79836bf7b33 server_cookie=f33a869bc1d088b9 in_seq=0 out_seq=0 2024-05-10T11:34:56.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.871+0000 7fce177fe700 10 client.28108 ms_handle_connect on v2:172.21.15.178:6838/3760985404 2024-05-10T11:34:56.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.872+0000 7fce177fe700 1 -- 172.21.15.100:0/1610673668 <== mds.1 v2:172.21.15.178:6838/3760985404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dd54650320 con 0x55dd5470a040 2024-05-10T11:34:56.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.872+0000 7fce177fe700 10 client.28108 handle_command_reply: tid=0 2024-05-10T11:34:56.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.872+0000 7fce48fd1b80 1 client.28108 shutdown 2024-05-10T11:34:56.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.872+0000 7fce48fd1b80 1 -- 172.21.15.100:0/1610673668 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd546f61b0 msgr2=0x55dd546f9b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:56.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.872+0000 7fce48fd1b80 1 --2- 172.21.15.100:0/1610673668 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd546f61b0 0x55dd546f9b10 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fce2c0bb5b0 tx=0x7fce2c00bf90 comp rx=0 tx=0).stop 2024-05-10T11:34:56.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.872+0000 7fce48fd1b80 1 -- 172.21.15.100:0/1610673668 shutdown_connections 2024-05-10T11:34:56.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.872+0000 7fce48fd1b80 1 --2- 172.21.15.100:0/1610673668 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55dd5470a040 0x55dd5470c4c0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:56.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.872+0000 7fce48fd1b80 1 --2- 172.21.15.100:0/1610673668 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd546f6ad0 0x55dd546fa050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:56.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.872+0000 7fce177fe700 0 client.28108 ms_handle_reset on v2:172.21.15.178:6838/3760985404 2024-05-10T11:34:56.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.872+0000 7fce48fd1b80 1 --2- 172.21.15.100:0/1610673668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd546f0d50 0x55dd546fc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:56.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.872+0000 7fce48fd1b80 1 --2- 172.21.15.100:0/1610673668 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd546f61b0 0x55dd546f9b10 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:56.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.872+0000 7fce48fd1b80 1 -- 172.21.15.100:0/1610673668 >> 172.21.15.100:0/1610673668 conn(0x55dd5461cb70 msgr2=0x55dd546e5b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:34:56.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.873+0000 7fce48fd1b80 1 -- 172.21.15.100:0/1610673668 shutdown_connections 2024-05-10T11:34:56.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.873+0000 7fce48fd1b80 1 -- 172.21.15.100:0/1610673668 wait complete. 2024-05-10T11:34:56.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.873+0000 7fce48fd1b80 20 client.28108 trim_cache size 0 max 16384 2024-05-10T11:34:56.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.901+0000 7fce48fd1b80 1 Processor -- start 2024-05-10T11:34:56.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.901+0000 7fce48fd1b80 1 -- start start 2024-05-10T11:34:56.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.901+0000 7fce48fd1b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd546f0d50 0x55dd5470c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:56.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.901+0000 7fce48fd1b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd546f61b0 0x55dd54706510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:56.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.901+0000 7fce48fd1b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd546f6ad0 0x55dd546e5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:56.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.901+0000 7fce48fd1b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fce381af680 con 0x55dd546f61b0 2024-05-10T11:34:56.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.901+0000 7fce48fd1b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fce381acee0 con 0x55dd546f6ad0 2024-05-10T11:34:56.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.901+0000 7fce48fd1b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fce381a29f0 con 0x55dd546f0d50 2024-05-10T11:34:56.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.901+0000 7fce377fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd546f61b0 0x55dd54706510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:56.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.901+0000 7fce377fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd546f61b0 0x55dd54706510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46886/0 (socket says 172.21.15.100:46886) 2024-05-10T11:34:56.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.901+0000 7fce377fe700 1 -- 172.21.15.100:0/2896923950 learned_addr learned my addr 172.21.15.100:0/2896923950 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:34:56.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.901+0000 7fce37fff700 1 --2- 172.21.15.100:0/2896923950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd546f0d50 0x55dd5470c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:56.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.901+0000 7fce3ccdb700 1 --2- 172.21.15.100:0/2896923950 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd546f6ad0 0x55dd546e5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:56.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.902+0000 7fce377fe700 1 -- 172.21.15.100:0/2896923950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd546f0d50 msgr2=0x55dd5470c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:56.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.902+0000 7fce377fe700 1 --2- 172.21.15.100:0/2896923950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd546f0d50 0x55dd5470c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:56.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.902+0000 7fce377fe700 1 -- 172.21.15.100:0/2896923950 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd546f6ad0 msgr2=0x55dd546e5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:56.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.902+0000 7fce377fe700 1 --2- 172.21.15.100:0/2896923950 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd546f6ad0 0x55dd546e5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:56.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.902+0000 7fce377fe700 1 -- 172.21.15.100:0/2896923950 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd546ebde0 con 0x55dd546f61b0 2024-05-10T11:34:56.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.902+0000 7fce3ccdb700 1 --2- 172.21.15.100:0/2896923950 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd546f6ad0 0x55dd546e5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:34:56.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.902+0000 7fce37fff700 1 --2- 172.21.15.100:0/2896923950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd546f0d50 0x55dd5470c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:34:56.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.902+0000 7fce377fe700 1 --2- 172.21.15.100:0/2896923950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd546f61b0 0x55dd54706510 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fce2c006010 tx=0x7fce2c0bfb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1645a70214d3e1e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:34:56.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.902+0000 7fce1dffb700 1 -- 172.21.15.100:0/2896923950 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce2c0baf70 con 0x55dd546f61b0 2024-05-10T11:34:56.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.902+0000 7fce1dffb700 1 -- 172.21.15.100:0/2896923950 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fce2c0bb130 con 0x55dd546f61b0 2024-05-10T11:34:56.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.902+0000 7fce1dffb700 1 -- 172.21.15.100:0/2896923950 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce2c0a30d0 con 0x55dd546f61b0 2024-05-10T11:34:56.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.903+0000 7fce48fd1b80 1 -- 172.21.15.100:0/2896923950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd546f61b0 msgr2=0x55dd54706510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:56.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.903+0000 7fce48fd1b80 1 --2- 172.21.15.100:0/2896923950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd546f61b0 0x55dd54706510 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fce2c006010 tx=0x7fce2c0bfb50 comp rx=0 tx=0).stop 2024-05-10T11:34:56.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.903+0000 7fce48fd1b80 1 -- 172.21.15.100:0/2896923950 shutdown_connections 2024-05-10T11:34:56.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.903+0000 7fce48fd1b80 1 --2- 172.21.15.100:0/2896923950 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd546f6ad0 0x55dd546e5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:56.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.903+0000 7fce48fd1b80 1 --2- 172.21.15.100:0/2896923950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd546f0d50 0x55dd5470c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:56.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.903+0000 7fce48fd1b80 1 --2- 172.21.15.100:0/2896923950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd546f61b0 0x55dd54706510 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:56.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.903+0000 7fce48fd1b80 1 -- 172.21.15.100:0/2896923950 >> 172.21.15.100:0/2896923950 conn(0x55dd5461cb70 msgr2=0x55dd546f8420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:34:56.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.903+0000 7fce48fd1b80 1 -- 172.21.15.100:0/2896923950 shutdown_connections 2024-05-10T11:34:56.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.903+0000 7fce48fd1b80 1 -- 172.21.15.100:0/2896923950 wait complete. 2024-05-10T11:34:56.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.903+0000 7fce48fd1b80 1 -- start start 2024-05-10T11:34:56.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.904+0000 7fce48fd1b80 1 Processor -- start 2024-05-10T11:34:56.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.904+0000 7fce48fd1b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd546f0d50 0x55dd54702c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:56.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.904+0000 7fce48fd1b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd546f61b0 0x55dd547111c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:56.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.904+0000 7fce48fd1b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd546f6ad0 0x55dd54711700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:56.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.904+0000 7fce48fd1b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fce381a3060 con 0x55dd546f6ad0 2024-05-10T11:34:56.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.904+0000 7fce48fd1b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fce381ac810 con 0x55dd546f0d50 2024-05-10T11:34:56.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.904+0000 7fce377fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd546f61b0 0x55dd547111c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:56.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.904+0000 7fce37fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd546f0d50 0x55dd54702c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:56.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.904+0000 7fce377fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd546f61b0 0x55dd547111c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:59892/0 (socket says 172.21.15.100:59892) 2024-05-10T11:34:56.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.904+0000 7fce377fe700 1 -- 172.21.15.100:0/2893610440 learned_addr learned my addr 172.21.15.100:0/2893610440 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:34:56.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.904+0000 7fce48fd1b80 1 -- 172.21.15.100:0/2893610440 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fce381a3730 con 0x55dd546f61b0 2024-05-10T11:34:56.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.904+0000 7fce3ccdb700 1 --2- 172.21.15.100:0/2893610440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd546f6ad0 0x55dd54711700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:56.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.904+0000 7fce48fd1b80 10 client.0 _ll_register_callbacks cb 0x55dd545d65b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:34:56.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.905+0000 7fce377fe700 1 -- 172.21.15.100:0/2893610440 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd546f0d50 msgr2=0x55dd54702c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:56.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.905+0000 7fce377fe700 1 --2- 172.21.15.100:0/2893610440 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd546f0d50 0x55dd54702c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:56.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.905+0000 7fce377fe700 1 -- 172.21.15.100:0/2893610440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd546f6ad0 msgr2=0x55dd54711700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:34:56.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.905+0000 7fce377fe700 1 --2- 172.21.15.100:0/2893610440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd546f6ad0 0x55dd54711700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:56.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.905+0000 7fce377fe700 1 -- 172.21.15.100:0/2893610440 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd546fd330 con 0x55dd546f61b0 2024-05-10T11:34:56.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.905+0000 7fce37fff700 1 --2- 172.21.15.100:0/2893610440 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd546f0d50 0x55dd54702c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:34:56.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.905+0000 7fce377fe700 1 --2- 172.21.15.100:0/2893610440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd546f61b0 0x55dd547111c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fce2c0bb5b0 tx=0x7fce2c0096e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=790a9bdeead5efdd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:34:56.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.905+0000 7fce177fe700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:34:56.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.905+0000 7fce177fe700 1 -- 172.21.15.100:0/2893610440 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce2c0baf70 con 0x55dd546f61b0 2024-05-10T11:34:56.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.905+0000 7fce177fe700 1 -- 172.21.15.100:0/2893610440 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fce2c0f7380 con 0x55dd546f61b0 2024-05-10T11:34:56.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.905+0000 7fce48fd1b80 1 -- 172.21.15.100:0/2893610440 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce2c0bb130 con 0x55dd546f61b0 2024-05-10T11:34:56.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.905+0000 7fce177fe700 1 -- 172.21.15.100:0/2893610440 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fce2c0c4440 con 0x55dd546f61b0 2024-05-10T11:34:56.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.905+0000 7fce48fd1b80 10 client.28114 fetch_fsmap learned FSMap version 40 2024-05-10T11:34:56.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.905+0000 7fce48fd1b80 1 -- 172.21.15.100:0/2893610440 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55dd546fd330 con 0x55dd546f61b0 2024-05-10T11:34:56.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.906+0000 7fce177fe700 1 -- 172.21.15.100:0/2893610440 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fce2c0bb130 con 0x55dd546f61b0 2024-05-10T11:34:56.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.906+0000 7fce177fe700 1 -- 172.21.15.100:0/2893610440 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 40) ==== 8350+0+0 (secure 0 0 0) 0x7fce2c0a1cb0 con 0x55dd546f61b0 2024-05-10T11:34:56.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.906+0000 7fce48fd1b80 10 client.28114 fetch_fsmap finished waiting for FSMap version 40 2024-05-10T11:34:56.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.906+0000 7fce48fd1b80 10 client.28114 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-10T11:34:56.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.906+0000 7fce48fd1b80 20 client.28114 populate_metadata read hostname 'smithi100' 2024-05-10T11:34:56.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.906+0000 7fce48fd1b80 1 --2- 172.21.15.100:0/2893610440 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55dd5470a040 0x55dd54776810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:34:56.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.906+0000 7fce48fd1b80 4 client.28114 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:34:56.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.906+0000 7fce48fd1b80 1 -- 172.21.15.100:0/2893610440 --> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dd5464a3c0 con 0x55dd5470a040 2024-05-10T11:34:56.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.906+0000 7fce37fff700 1 --2- 172.21.15.100:0/2893610440 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55dd5470a040 0x55dd54776810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:34:56.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.907+0000 7fce37fff700 1 --2- 172.21.15.100:0/2893610440 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55dd5470a040 0x55dd54776810 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=80bc60f89c1580ed server_cookie=8674e121acff8da5 in_seq=0 out_seq=0 2024-05-10T11:34:56.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.907+0000 7fce177fe700 10 client.28114 ms_handle_connect on v2:172.21.15.178:6838/3760985404 2024-05-10T11:34:56.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.908+0000 7fce177fe700 1 -- 172.21.15.100:0/2893610440 <== mds.1 v2:172.21.15.178:6838/3760985404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dd5464a3c0 con 0x55dd5470a040 2024-05-10T11:34:56.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.908+0000 7fce177fe700 10 client.28114 handle_command_reply: tid=0 2024-05-10T11:34:56.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.908+0000 7fce48fd1b80 1 client.28114 shutdown 2024-05-10T11:34:56.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.908+0000 7fce48fd1b80 1 -- 172.21.15.100:0/2893610440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd546f61b0 msgr2=0x55dd547111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:56.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.908+0000 7fce48fd1b80 1 --2- 172.21.15.100:0/2893610440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd546f61b0 0x55dd547111c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fce2c0bb5b0 tx=0x7fce2c0096e0 comp rx=0 tx=0).stop 2024-05-10T11:34:56.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.908+0000 7fce48fd1b80 1 -- 172.21.15.100:0/2893610440 shutdown_connections 2024-05-10T11:34:56.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.908+0000 7fce48fd1b80 1 --2- 172.21.15.100:0/2893610440 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55dd5470a040 0x55dd54776810 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:56.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.908+0000 7fce48fd1b80 1 --2- 172.21.15.100:0/2893610440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd546f6ad0 0x55dd54711700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:56.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.908+0000 7fce48fd1b80 1 --2- 172.21.15.100:0/2893610440 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd546f0d50 0x55dd54702c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:56.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.908+0000 7fce48fd1b80 1 --2- 172.21.15.100:0/2893610440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd546f61b0 0x55dd547111c0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:56.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.908+0000 7fce48fd1b80 1 -- 172.21.15.100:0/2893610440 >> 172.21.15.100:0/2893610440 conn(0x55dd5461cb70 msgr2=0x55dd546f6ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:34:56.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.908+0000 7fce177fe700 0 client.28114 ms_handle_reset on v2:172.21.15.178:6838/3760985404 2024-05-10T11:34:56.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.908+0000 7fce48fd1b80 1 -- 172.21.15.100:0/2893610440 shutdown_connections 2024-05-10T11:34:56.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.908+0000 7fce48fd1b80 1 -- 172.21.15.100:0/2893610440 wait complete. 2024-05-10T11:34:56.915 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:34:56.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.908+0000 7fce48fd1b80 20 client.28114 trim_cache size 0 max 16384 2024-05-10T11:34:56.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.909+0000 7fce3e73e700 1 -- 172.21.15.100:0/1360427810 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fce200b2980 msgr2=0x7fce200b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:56.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.909+0000 7fce3e73e700 1 --2- 172.21.15.100:0/1360427810 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fce200b2980 0x7fce200b4e40 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fce300075c0 tx=0x7fce3000b040 comp rx=0 tx=0).stop 2024-05-10T11:34:56.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.909+0000 7fce3e73e700 1 -- 172.21.15.100:0/1360427810 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce3806c550 msgr2=0x7fce3810a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:34:56.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.909+0000 7fce3e73e700 1 --2- 172.21.15.100:0/1360427810 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce3806c550 0x7fce3810a850 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fce2c00b650 tx=0x7fce2c0047b0 comp rx=0 tx=0).stop 2024-05-10T11:34:56.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.909+0000 7fce3e73e700 1 -- 172.21.15.100:0/1360427810 shutdown_connections 2024-05-10T11:34:56.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.910+0000 7fce3e73e700 1 --2- 172.21.15.100:0/1360427810 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fce200b2980 0x7fce200b4e40 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:56.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.910+0000 7fce3e73e700 1 --2- 172.21.15.100:0/1360427810 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce38109500 0x7fce3810ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:56.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.910+0000 7fce3e73e700 1 --2- 172.21.15.100:0/1360427810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce3806bb90 0x7fce3810a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:56.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.910+0000 7fce3e73e700 1 --2- 172.21.15.100:0/1360427810 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce3806c550 0x7fce3810a850 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:34:56.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.910+0000 7fce3e73e700 1 -- 172.21.15.100:0/1360427810 >> 172.21.15.100:0/1360427810 conn(0x7fce38079310 msgr2=0x7fce3810e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:34:56.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.910+0000 7fce3e73e700 1 -- 172.21.15.100:0/1360427810 shutdown_connections 2024-05-10T11:34:56.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:34:56.910+0000 7fce3e73e700 1 -- 172.21.15.100:0/1360427810 wait complete. 2024-05-10T11:34:56.917 INFO:tasks.workunit.client.0.smithi100.stdout: 10 29212 9.48 MB/sec execute 50 sec latency 160.111 ms 2024-05-10T11:34:57.916 INFO:tasks.workunit.client.0.smithi100.stdout: 10 29407 9.54 MB/sec execute 51 sec latency 59.622 ms 2024-05-10T11:34:57.970 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:34:57 smithi100 ceph-mon[30296]: pgmap v918: 129 pgs: 129 active+clean; 420 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 4.2 MiB/s wr, 302 op/s 2024-05-10T11:34:58.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:34:57 smithi178 ceph-mon[33359]: pgmap v918: 129 pgs: 129 active+clean; 420 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 4.2 MiB/s wr, 302 op/s 2024-05-10T11:34:58.018 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-10T11:34:58.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:34:57 smithi188 ceph-mon[34075]: pgmap v918: 129 pgs: 129 active+clean; 420 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 4.2 MiB/s wr, 302 op/s 2024-05-10T11:34:58.368 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:34:58.916 INFO:tasks.workunit.client.0.smithi100.stdout: 10 29611 9.64 MB/sec execute 52 sec latency 111.327 ms 2024-05-10T11:34:59.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:34:59 smithi100 ceph-mon[30296]: pgmap v919: 129 pgs: 129 active+clean; 421 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 3.8 MiB/s wr, 281 op/s 2024-05-10T11:34:59.916 INFO:tasks.workunit.client.0.smithi100.stdout: 10 29808 9.66 MB/sec execute 53 sec latency 62.166 ms 2024-05-10T11:35:00.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:34:59 smithi178 ceph-mon[33359]: pgmap v919: 129 pgs: 129 active+clean; 421 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 3.8 MiB/s wr, 281 op/s 2024-05-10T11:35:00.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:34:59 smithi188 ceph-mon[34075]: pgmap v919: 129 pgs: 129 active+clean; 421 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 3.8 MiB/s wr, 281 op/s 2024-05-10T11:35:00.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.548+0000 7f5451616700 1 -- 172.21.15.100:0/3183879535 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f544c109510 msgr2=0x7f544c1098f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:00.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.548+0000 7f5451616700 1 --2- 172.21.15.100:0/3183879535 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f544c109510 0x7f544c1098f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f543c0098f0 tx=0x7f543c009c00 comp rx=0 tx=0).stop 2024-05-10T11:35:00.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.548+0000 7f5451616700 1 -- 172.21.15.100:0/3183879535 shutdown_connections 2024-05-10T11:35:00.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.548+0000 7f5451616700 1 --2- 172.21.15.100:0/3183879535 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f544c06bb40 0x7f544c110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:00.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.548+0000 7f5451616700 1 --2- 172.21.15.100:0/3183879535 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f544c109510 0x7f544c1098f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:00.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.548+0000 7f5451616700 1 --2- 172.21.15.100:0/3183879535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f544c06b180 0x7f544c06b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:00.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.548+0000 7f5451616700 1 -- 172.21.15.100:0/3183879535 >> 172.21.15.100:0/3183879535 conn(0x7f544c079350 msgr2=0x7f544c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:00.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.548+0000 7f5451616700 1 -- 172.21.15.100:0/3183879535 shutdown_connections 2024-05-10T11:35:00.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.549+0000 7f5451616700 1 -- 172.21.15.100:0/3183879535 wait complete. 2024-05-10T11:35:00.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.549+0000 7f5451616700 1 Processor -- start 2024-05-10T11:35:00.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.550+0000 7f5451616700 1 -- start start 2024-05-10T11:35:00.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.550+0000 7f5451616700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f544c06b180 0x7f544c10e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:00.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.551+0000 7f5451616700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f544c06bb40 0x7f544c10eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:00.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.551+0000 7f5451616700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f544c109510 0x7f544c1a4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:00.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.551+0000 7f5451616700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f544c113520 con 0x7f544c109510 2024-05-10T11:35:00.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.551+0000 7f5451616700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f544c1133a0 con 0x7f544c06bb40 2024-05-10T11:35:00.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.551+0000 7f5451616700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f544c1136a0 con 0x7f544c06b180 2024-05-10T11:35:00.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.551+0000 7f544affd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f544c06b180 0x7f544c10e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:00.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.551+0000 7f544affd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f544c06b180 0x7f544c10e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:59910/0 (socket says 172.21.15.100:59910) 2024-05-10T11:35:00.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.551+0000 7f544affd700 1 -- 172.21.15.100:0/3402924746 learned_addr learned my addr 172.21.15.100:0/3402924746 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:35:00.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.552+0000 7f544affd700 1 -- 172.21.15.100:0/3402924746 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f544c06bb40 msgr2=0x7f544c10eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:00.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.552+0000 7f544affd700 1 --2- 172.21.15.100:0/3402924746 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f544c06bb40 0x7f544c10eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:00.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.552+0000 7f544affd700 1 -- 172.21.15.100:0/3402924746 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f544c109510 msgr2=0x7f544c1a4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:00.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.552+0000 7f544affd700 1 --2- 172.21.15.100:0/3402924746 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f544c109510 0x7f544c1a4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:00.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.552+0000 7f544affd700 1 -- 172.21.15.100:0/3402924746 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f544c1a5150 con 0x7f544c06b180 2024-05-10T11:35:00.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.552+0000 7f544affd700 1 --2- 172.21.15.100:0/3402924746 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f544c06b180 0x7f544c10e600 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f543c005950 tx=0x7f543c00b560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:00.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.554+0000 7f5433fff700 1 -- 172.21.15.100:0/3402924746 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f543c00b8c0 con 0x7f544c06b180 2024-05-10T11:35:00.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.554+0000 7f5433fff700 1 -- 172.21.15.100:0/3402924746 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f543c00ba60 con 0x7f544c06b180 2024-05-10T11:35:00.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.554+0000 7f5433fff700 1 -- 172.21.15.100:0/3402924746 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f543c003ba0 con 0x7f544c06b180 2024-05-10T11:35:00.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.555+0000 7f5451616700 1 -- 172.21.15.100:0/3402924746 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f544c1a53e0 con 0x7f544c06b180 2024-05-10T11:35:00.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.555+0000 7f5451616700 1 -- 172.21.15.100:0/3402924746 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f544c1acca0 con 0x7f544c06b180 2024-05-10T11:35:00.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.558+0000 7f5433fff700 1 -- 172.21.15.100:0/3402924746 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f543c00bc10 con 0x7f544c06b180 2024-05-10T11:35:00.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.562+0000 7f545bea9b80 1 Processor -- start 2024-05-10T11:35:00.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.562+0000 7f545bea9b80 1 -- start start 2024-05-10T11:35:00.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.562+0000 7f5433fff700 1 --2- 172.21.15.100:0/3402924746 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f54340b2a50 0x7f54340b4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:00.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.563+0000 7f544b7fe700 1 --2- 172.21.15.100:0/3402924746 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f54340b2a50 0x7f54340b4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:00.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.563+0000 7f5433fff700 1 -- 172.21.15.100:0/3402924746 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f543c0e3670 con 0x7f544c06b180 2024-05-10T11:35:00.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.563+0000 7f545bea9b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f5b0491b0 0x558f5b049590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:00.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.563+0000 7f545bea9b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f5b049ad0 0x558f5b043810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:00.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.563+0000 7f545bea9b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558f5b043d50 0x558f5b044190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:00.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.563+0000 7f545bea9b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x558f5af95940 con 0x558f5b0491b0 2024-05-10T11:35:00.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.563+0000 7f545bea9b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558f5aefada0 con 0x558f5b049ad0 2024-05-10T11:35:00.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.563+0000 7f545bea9b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558f5af91900 con 0x558f5b043d50 2024-05-10T11:35:00.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.563+0000 7f544affd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f5b0491b0 0x558f5b049590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:00.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.563+0000 7f544affd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f5b0491b0 0x558f5b049590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46912/0 (socket says 172.21.15.100:46912) 2024-05-10T11:35:00.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.563+0000 7f544affd700 1 -- 172.21.15.100:0/1566238960 learned_addr learned my addr 172.21.15.100:0/1566238960 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:35:00.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.564+0000 7f544affd700 1 -- 172.21.15.100:0/1566238960 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558f5b043d50 msgr2=0x558f5b044190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:00.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.564+0000 7f544affd700 1 --2- 172.21.15.100:0/1566238960 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558f5b043d50 0x558f5b044190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:00.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.564+0000 7f544affd700 1 -- 172.21.15.100:0/1566238960 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f5b049ad0 msgr2=0x558f5b043810 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T11:35:00.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.564+0000 7f544affd700 1 --2- 172.21.15.100:0/1566238960 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f5b049ad0 0x558f5b043810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:00.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.564+0000 7f544affd700 1 -- 172.21.15.100:0/1566238960 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f5aee28d0 con 0x558f5b0491b0 2024-05-10T11:35:00.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.564+0000 7f544affd700 1 --2- 172.21.15.100:0/1566238960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f5b0491b0 0x558f5b049590 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f543c009fd0 tx=0x7f543c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c91849ee0edd878f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:00.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.564+0000 7f54317fa700 1 -- 172.21.15.100:0/1566238960 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f543c0b3070 con 0x558f5b0491b0 2024-05-10T11:35:00.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.565+0000 7f54317fa700 1 -- 172.21.15.100:0/1566238960 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f543c0aa470 con 0x558f5b0491b0 2024-05-10T11:35:00.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.565+0000 7f54317fa700 1 -- 172.21.15.100:0/1566238960 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f543c0e3370 con 0x558f5b0491b0 2024-05-10T11:35:00.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.565+0000 7f545bea9b80 1 -- 172.21.15.100:0/1566238960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f5b0491b0 msgr2=0x558f5b049590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:00.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.565+0000 7f545bea9b80 1 --2- 172.21.15.100:0/1566238960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f5b0491b0 0x558f5b049590 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f543c009fd0 tx=0x7f543c009260 comp rx=0 tx=0).stop 2024-05-10T11:35:00.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.566+0000 7f544b7fe700 1 --2- 172.21.15.100:0/3402924746 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f54340b2a50 0x7f54340b4f10 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f5444009710 tx=0x7f5444009620 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:00.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.566+0000 7f545bea9b80 1 -- 172.21.15.100:0/1566238960 shutdown_connections 2024-05-10T11:35:00.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.566+0000 7f545bea9b80 1 --2- 172.21.15.100:0/1566238960 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558f5b043d50 0x558f5b044190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:00.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.566+0000 7f545bea9b80 1 --2- 172.21.15.100:0/1566238960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f5b0491b0 0x558f5b049590 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:00.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.566+0000 7f545bea9b80 1 --2- 172.21.15.100:0/1566238960 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f5b049ad0 0x558f5b043810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:00.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.566+0000 7f545bea9b80 1 -- 172.21.15.100:0/1566238960 >> 172.21.15.100:0/1566238960 conn(0x558f5af6fb70 msgr2=0x558f5b03e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:00.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.566+0000 7f545bea9b80 1 -- 172.21.15.100:0/1566238960 shutdown_connections 2024-05-10T11:35:00.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.566+0000 7f545bea9b80 1 -- 172.21.15.100:0/1566238960 wait complete. 2024-05-10T11:35:00.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.567+0000 7f545bea9b80 1 -- start start 2024-05-10T11:35:00.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.567+0000 7f545bea9b80 1 Processor -- start 2024-05-10T11:35:00.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.568+0000 7f545bea9b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558f5b043d50 0x558f5b04f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:00.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.568+0000 7f545bea9b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f5b0491b0 0x558f5b04cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:00.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.568+0000 7f545bea9b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f5b049ad0 0x558f5b04cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:00.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.568+0000 7f545bea9b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x558f5b050210 con 0x558f5b049ad0 2024-05-10T11:35:00.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.568+0000 7f545bea9b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558f5af4eed0 con 0x558f5b0491b0 2024-05-10T11:35:00.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.568+0000 7f545bea9b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558f5b050390 con 0x558f5b043d50 2024-05-10T11:35:00.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.568+0000 7f545bea9b80 10 client.0 _ll_register_callbacks cb 0x558f5af295b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:35:00.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.568+0000 7f544affd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558f5b043d50 0x558f5b04f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:00.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.568+0000 7f544affd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558f5b043d50 0x558f5b04f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:59926/0 (socket says 172.21.15.100:59926) 2024-05-10T11:35:00.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.568+0000 7f544b7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f5b049ad0 0x558f5b04cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:00.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.568+0000 7f544affd700 1 -- 172.21.15.100:0/964564915 learned_addr learned my addr 172.21.15.100:0/964564915 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:35:00.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.568+0000 7f544a7fc700 1 --2- 172.21.15.100:0/964564915 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f5b0491b0 0x558f5b04cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:00.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.569+0000 7f544a7fc700 1 -- 172.21.15.100:0/964564915 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558f5b043d50 msgr2=0x558f5b04f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:00.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.569+0000 7f544a7fc700 1 --2- 172.21.15.100:0/964564915 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558f5b043d50 0x558f5b04f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:00.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.569+0000 7f544a7fc700 1 -- 172.21.15.100:0/964564915 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f5b049ad0 msgr2=0x558f5b04cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:00.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.569+0000 7f544a7fc700 1 --2- 172.21.15.100:0/964564915 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f5b049ad0 0x558f5b04cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:00.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.569+0000 7f544a7fc700 1 -- 172.21.15.100:0/964564915 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f5af4c760 con 0x558f5b0491b0 2024-05-10T11:35:00.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.569+0000 7f544a7fc700 1 --2- 172.21.15.100:0/964564915 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f5b0491b0 0x558f5b04cab0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x558f5b049eb0 tx=0x7f544000e8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f28ab05d145a5cce server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:00.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.569+0000 7f542eff5700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:35:00.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.569+0000 7f542eff5700 1 -- 172.21.15.100:0/964564915 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5440004220 con 0x558f5b0491b0 2024-05-10T11:35:00.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.569+0000 7f542eff5700 1 -- 172.21.15.100:0/964564915 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f54400043c0 con 0x558f5b0491b0 2024-05-10T11:35:00.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.569+0000 7f545bea9b80 1 -- 172.21.15.100:0/964564915 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f543c0aa470 con 0x558f5b0491b0 2024-05-10T11:35:00.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.569+0000 7f542eff5700 1 -- 172.21.15.100:0/964564915 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f5440004830 con 0x558f5b0491b0 2024-05-10T11:35:00.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.569+0000 7f545bea9b80 10 client.28025 fetch_fsmap learned FSMap version 40 2024-05-10T11:35:00.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.569+0000 7f545bea9b80 1 -- 172.21.15.100:0/964564915 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x558f5b05ab00 con 0x558f5b0491b0 2024-05-10T11:35:00.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.570+0000 7f542eff5700 1 -- 172.21.15.100:0/964564915 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f543c0aa470 con 0x558f5b0491b0 2024-05-10T11:35:00.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.570+0000 7f542eff5700 1 -- 172.21.15.100:0/964564915 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 40) ==== 8350+0+0 (secure 0 0 0) 0x7f54400054d0 con 0x558f5b0491b0 2024-05-10T11:35:00.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.570+0000 7f545bea9b80 10 client.28025 fetch_fsmap finished waiting for FSMap version 40 2024-05-10T11:35:00.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.570+0000 7f545bea9b80 10 client.28025 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:35:00.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.570+0000 7f545bea9b80 20 client.28025 populate_metadata read hostname 'smithi100' 2024-05-10T11:35:00.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.570+0000 7f545bea9b80 1 --2- 172.21.15.100:0/964564915 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558f5b05d190 0x558f5b05f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:00.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.570+0000 7f545bea9b80 4 client.28025 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:35:00.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.570+0000 7f545bea9b80 1 -- 172.21.15.100:0/964564915 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f5afa3320 con 0x558f5b05d190 2024-05-10T11:35:00.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.570+0000 7f544affd700 1 --2- 172.21.15.100:0/964564915 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558f5b05d190 0x558f5b05f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:00.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.571+0000 7f542eff5700 10 client.28025 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:35:00.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.571+0000 7f544affd700 1 --2- 172.21.15.100:0/964564915 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558f5b05d190 0x558f5b05f5b0 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85acc8dc0ebcf841 server_cookie=90241a2423d259cd in_seq=0 out_seq=0 2024-05-10T11:35:00.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.572+0000 7f542eff5700 1 -- 172.21.15.100:0/964564915 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558f5afa3320 con 0x558f5b05d190 2024-05-10T11:35:00.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.572+0000 7f542eff5700 10 client.28025 handle_command_reply: tid=0 2024-05-10T11:35:00.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.572+0000 7f545bea9b80 1 client.28025 shutdown 2024-05-10T11:35:00.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.572+0000 7f545bea9b80 1 -- 172.21.15.100:0/964564915 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f5b0491b0 msgr2=0x558f5b04cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:00.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.572+0000 7f545bea9b80 1 --2- 172.21.15.100:0/964564915 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f5b0491b0 0x558f5b04cab0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x558f5b049eb0 tx=0x7f544000e8b0 comp rx=0 tx=0).stop 2024-05-10T11:35:00.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.572+0000 7f545bea9b80 1 -- 172.21.15.100:0/964564915 shutdown_connections 2024-05-10T11:35:00.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.572+0000 7f545bea9b80 1 --2- 172.21.15.100:0/964564915 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558f5b05d190 0x558f5b05f5b0 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:00.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.572+0000 7f545bea9b80 1 --2- 172.21.15.100:0/964564915 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f5b049ad0 0x558f5b04cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:00.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.572+0000 7f545bea9b80 1 --2- 172.21.15.100:0/964564915 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558f5b043d50 0x558f5b04f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:00.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.572+0000 7f545bea9b80 1 --2- 172.21.15.100:0/964564915 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f5b0491b0 0x558f5b04cab0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:00.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.572+0000 7f545bea9b80 1 -- 172.21.15.100:0/964564915 >> 172.21.15.100:0/964564915 conn(0x558f5af6fb70 msgr2=0x558f5b038b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:00.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.572+0000 7f542eff5700 0 client.28025 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:35:00.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.572+0000 7f545bea9b80 1 -- 172.21.15.100:0/964564915 shutdown_connections 2024-05-10T11:35:00.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.572+0000 7f545bea9b80 1 -- 172.21.15.100:0/964564915 wait complete. 2024-05-10T11:35:00.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.572+0000 7f545bea9b80 20 client.28025 trim_cache size 0 max 16384 2024-05-10T11:35:00.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.593+0000 7f545bea9b80 1 Processor -- start 2024-05-10T11:35:00.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.593+0000 7f545bea9b80 1 -- start start 2024-05-10T11:35:00.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.593+0000 7f545bea9b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f5b043d50 0x558f5b043810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:00.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.593+0000 7f545bea9b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558f5b0491b0 0x558f5b059260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:00.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.593+0000 7f545bea9b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f5b049ad0 0x558f5b038c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:00.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.593+0000 7f545bea9b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f544c1b7ed0 con 0x558f5b043d50 2024-05-10T11:35:00.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.593+0000 7f545bea9b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f544c1b5730 con 0x558f5b049ad0 2024-05-10T11:35:00.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.593+0000 7f545bea9b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f544c1b3580 con 0x558f5b0491b0 2024-05-10T11:35:00.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.593+0000 7f544affd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f5b043d50 0x558f5b043810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:00.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.593+0000 7f544affd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f5b043d50 0x558f5b043810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46926/0 (socket says 172.21.15.100:46926) 2024-05-10T11:35:00.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.593+0000 7f544a7fc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558f5b0491b0 0x558f5b059260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:00.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.593+0000 7f544b7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f5b049ad0 0x558f5b038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:00.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.593+0000 7f544affd700 1 -- 172.21.15.100:0/2335093376 learned_addr learned my addr 172.21.15.100:0/2335093376 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:35:00.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.594+0000 7f544affd700 1 -- 172.21.15.100:0/2335093376 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558f5b0491b0 msgr2=0x558f5b059260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:00.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.594+0000 7f544affd700 1 --2- 172.21.15.100:0/2335093376 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558f5b0491b0 0x558f5b059260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:00.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.594+0000 7f544affd700 1 -- 172.21.15.100:0/2335093376 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f5b049ad0 msgr2=0x558f5b038c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:00.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.594+0000 7f544affd700 1 --2- 172.21.15.100:0/2335093376 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f5b049ad0 0x558f5b038c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:00.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.594+0000 7f544affd700 1 -- 172.21.15.100:0/2335093376 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f5b0597a0 con 0x558f5b043d50 2024-05-10T11:35:00.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.594+0000 7f544a7fc700 1 --2- 172.21.15.100:0/2335093376 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558f5b0491b0 0x558f5b059260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:35:00.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.594+0000 7f544b7fe700 1 --2- 172.21.15.100:0/2335093376 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f5b049ad0 0x558f5b038c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:35:00.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.594+0000 7f544affd700 1 --2- 172.21.15.100:0/2335093376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f5b043d50 0x558f5b043810 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f543c0aa8f0 tx=0x7f543c003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=542bf1c3d16c8b98 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:00.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.594+0000 7f5430ff9700 1 -- 172.21.15.100:0/2335093376 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f543c0aa470 con 0x558f5b043d50 2024-05-10T11:35:00.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.594+0000 7f5430ff9700 1 -- 172.21.15.100:0/2335093376 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f543c0ab060 con 0x558f5b043d50 2024-05-10T11:35:00.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.594+0000 7f5430ff9700 1 -- 172.21.15.100:0/2335093376 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f543c0e3370 con 0x558f5b043d50 2024-05-10T11:35:00.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.595+0000 7f545bea9b80 1 -- 172.21.15.100:0/2335093376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f5b043d50 msgr2=0x558f5b043810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:00.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.595+0000 7f545bea9b80 1 --2- 172.21.15.100:0/2335093376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f5b043d50 0x558f5b043810 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f543c0aa8f0 tx=0x7f543c003680 comp rx=0 tx=0).stop 2024-05-10T11:35:00.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.595+0000 7f545bea9b80 1 -- 172.21.15.100:0/2335093376 shutdown_connections 2024-05-10T11:35:00.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.595+0000 7f545bea9b80 1 --2- 172.21.15.100:0/2335093376 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f5b049ad0 0x558f5b038c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:00.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.595+0000 7f545bea9b80 1 --2- 172.21.15.100:0/2335093376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f5b043d50 0x558f5b043810 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:00.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.595+0000 7f545bea9b80 1 --2- 172.21.15.100:0/2335093376 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558f5b0491b0 0x558f5b059260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:00.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.595+0000 7f545bea9b80 1 -- 172.21.15.100:0/2335093376 >> 172.21.15.100:0/2335093376 conn(0x558f5af6fb70 msgr2=0x558f5b04b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:00.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.595+0000 7f545bea9b80 1 -- 172.21.15.100:0/2335093376 shutdown_connections 2024-05-10T11:35:00.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.595+0000 7f545bea9b80 1 -- 172.21.15.100:0/2335093376 wait complete. 2024-05-10T11:35:00.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.596+0000 7f545bea9b80 1 -- start start 2024-05-10T11:35:00.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.596+0000 7f545bea9b80 1 Processor -- start 2024-05-10T11:35:00.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.596+0000 7f545bea9b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558f5b043d50 0x558f5b0555e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:00.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.596+0000 7f545bea9b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f5b0491b0 0x558f5b055b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:00.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.596+0000 7f545bea9b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f5b049ad0 0x558f5b063e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:00.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.596+0000 7f545bea9b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f544c1b3bf0 con 0x558f5b0491b0 2024-05-10T11:35:00.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.596+0000 7f545bea9b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f544c1b5060 con 0x558f5b049ad0 2024-05-10T11:35:00.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.596+0000 7f545bea9b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f544c1b42c0 con 0x558f5b043d50 2024-05-10T11:35:00.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.596+0000 7f545bea9b80 10 client.0 _ll_register_callbacks cb 0x558f5af295b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:35:00.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.596+0000 7f544a7fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f5b0491b0 0x558f5b055b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:00.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.596+0000 7f544a7fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f5b0491b0 0x558f5b055b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46936/0 (socket says 172.21.15.100:46936) 2024-05-10T11:35:00.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.596+0000 7f544b7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f5b049ad0 0x558f5b063e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:00.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.596+0000 7f544affd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558f5b043d50 0x558f5b0555e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:00.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.596+0000 7f544a7fc700 1 -- 172.21.15.100:0/366223526 learned_addr learned my addr 172.21.15.100:0/366223526 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:35:00.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.597+0000 7f544a7fc700 1 -- 172.21.15.100:0/366223526 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558f5b043d50 msgr2=0x558f5b0555e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:00.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.597+0000 7f544a7fc700 1 --2- 172.21.15.100:0/366223526 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558f5b043d50 0x558f5b0555e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:00.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.597+0000 7f544a7fc700 1 -- 172.21.15.100:0/366223526 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f5b049ad0 msgr2=0x558f5b063e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:00.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.597+0000 7f544a7fc700 1 --2- 172.21.15.100:0/366223526 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f5b049ad0 0x558f5b063e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:00.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.597+0000 7f544a7fc700 1 -- 172.21.15.100:0/366223526 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f5b064370 con 0x558f5b0491b0 2024-05-10T11:35:00.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.597+0000 7f544a7fc700 1 --2- 172.21.15.100:0/366223526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f5b0491b0 0x558f5b055b20 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x558f5b03ee40 tx=0x7f5440017e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=899225c88d96de1d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:00.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.597+0000 7f542eff5700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:35:00.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.597+0000 7f542eff5700 1 -- 172.21.15.100:0/366223526 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5440009310 con 0x558f5b0491b0 2024-05-10T11:35:00.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.597+0000 7f545bea9b80 1 -- 172.21.15.100:0/366223526 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f543c0ab060 con 0x558f5b0491b0 2024-05-10T11:35:00.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.597+0000 7f542eff5700 1 -- 172.21.15.100:0/366223526 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f5440005d40 con 0x558f5b0491b0 2024-05-10T11:35:00.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.598+0000 7f542eff5700 1 -- 172.21.15.100:0/366223526 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f5440010430 con 0x558f5b0491b0 2024-05-10T11:35:00.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.598+0000 7f545bea9b80 10 client.18720 fetch_fsmap learned FSMap version 40 2024-05-10T11:35:00.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.598+0000 7f545bea9b80 1 -- 172.21.15.100:0/366223526 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x558f5b0c7150 con 0x558f5b0491b0 2024-05-10T11:35:00.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.598+0000 7f542eff5700 1 -- 172.21.15.100:0/366223526 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f543c0ab060 con 0x558f5b0491b0 2024-05-10T11:35:00.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.598+0000 7f542eff5700 1 -- 172.21.15.100:0/366223526 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) ==== 8350+0+0 (secure 0 0 0) 0x7f544002f630 con 0x558f5b0491b0 2024-05-10T11:35:00.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.598+0000 7f545bea9b80 10 client.18720 fetch_fsmap finished waiting for FSMap version 40 2024-05-10T11:35:00.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.598+0000 7f545bea9b80 10 client.18720 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:35:00.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.598+0000 7f545bea9b80 20 client.18720 populate_metadata read hostname 'smithi100' 2024-05-10T11:35:00.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.598+0000 7f545bea9b80 1 --2- 172.21.15.100:0/366223526 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558f5b0c7910 0x558f5b0c9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:00.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.598+0000 7f545bea9b80 4 client.18720 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:35:00.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.598+0000 7f545bea9b80 1 -- 172.21.15.100:0/366223526 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558f5af9d3c0 con 0x558f5b0c7910 2024-05-10T11:35:00.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.598+0000 7f544affd700 1 --2- 172.21.15.100:0/366223526 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558f5b0c7910 0x558f5b0c9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:00.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.599+0000 7f544affd700 1 --2- 172.21.15.100:0/366223526 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558f5b0c7910 0x558f5b0c9d30 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=125bfc1560963333 server_cookie=1f531f5b171344bc in_seq=0 out_seq=0 2024-05-10T11:35:00.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.599+0000 7f542eff5700 10 client.18720 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:35:00.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.599+0000 7f542eff5700 1 -- 172.21.15.100:0/366223526 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558f5af9d3c0 con 0x558f5b0c7910 2024-05-10T11:35:00.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.599+0000 7f542eff5700 10 client.18720 handle_command_reply: tid=0 2024-05-10T11:35:00.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.599+0000 7f545bea9b80 1 client.18720 shutdown 2024-05-10T11:35:00.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.599+0000 7f545bea9b80 1 -- 172.21.15.100:0/366223526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f5b0491b0 msgr2=0x558f5b055b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:00.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.599+0000 7f545bea9b80 1 --2- 172.21.15.100:0/366223526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f5b0491b0 0x558f5b055b20 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x558f5b03ee40 tx=0x7f5440017e90 comp rx=0 tx=0).stop 2024-05-10T11:35:00.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.599+0000 7f545bea9b80 1 -- 172.21.15.100:0/366223526 shutdown_connections 2024-05-10T11:35:00.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.600+0000 7f545bea9b80 1 --2- 172.21.15.100:0/366223526 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558f5b0c7910 0x558f5b0c9d30 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:00.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.600+0000 7f545bea9b80 1 --2- 172.21.15.100:0/366223526 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f5b049ad0 0x558f5b063e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:00.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.600+0000 7f545bea9b80 1 --2- 172.21.15.100:0/366223526 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558f5b043d50 0x558f5b0555e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:00.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.600+0000 7f545bea9b80 1 --2- 172.21.15.100:0/366223526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f5b0491b0 0x558f5b055b20 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:00.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.600+0000 7f545bea9b80 1 -- 172.21.15.100:0/366223526 >> 172.21.15.100:0/366223526 conn(0x558f5af6fb70 msgr2=0x558f5b049ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:00.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.600+0000 7f542eff5700 0 client.18720 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:35:00.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.600+0000 7f545bea9b80 1 -- 172.21.15.100:0/366223526 shutdown_connections 2024-05-10T11:35:00.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.600+0000 7f545bea9b80 1 -- 172.21.15.100:0/366223526 wait complete. 2024-05-10T11:35:00.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.600+0000 7f545bea9b80 20 client.18720 trim_cache size 0 max 16384 2024-05-10T11:35:00.604 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:35:00.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.601+0000 7f5451616700 1 -- 172.21.15.100:0/3402924746 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f54340b2a50 msgr2=0x7f54340b4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:00.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.601+0000 7f5451616700 1 --2- 172.21.15.100:0/3402924746 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f54340b2a50 0x7f54340b4f10 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f5444009710 tx=0x7f5444009620 comp rx=0 tx=0).stop 2024-05-10T11:35:00.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.601+0000 7f5451616700 1 -- 172.21.15.100:0/3402924746 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f544c06b180 msgr2=0x7f544c10e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:00.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.601+0000 7f5451616700 1 --2- 172.21.15.100:0/3402924746 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f544c06b180 0x7f544c10e600 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f543c005950 tx=0x7f543c00b560 comp rx=0 tx=0).stop 2024-05-10T11:35:00.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.601+0000 7f5451616700 1 -- 172.21.15.100:0/3402924746 shutdown_connections 2024-05-10T11:35:00.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.601+0000 7f5451616700 1 --2- 172.21.15.100:0/3402924746 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f54340b2a50 0x7f54340b4f10 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:00.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.601+0000 7f5451616700 1 --2- 172.21.15.100:0/3402924746 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f544c109510 0x7f544c1a4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:00.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.601+0000 7f5451616700 1 --2- 172.21.15.100:0/3402924746 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f544c06b180 0x7f544c10e600 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:00.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.601+0000 7f5451616700 1 --2- 172.21.15.100:0/3402924746 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f544c06bb40 0x7f544c10eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:00.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.601+0000 7f5451616700 1 -- 172.21.15.100:0/3402924746 >> 172.21.15.100:0/3402924746 conn(0x7f544c079350 msgr2=0x7f544c102ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:00.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.601+0000 7f5451616700 1 -- 172.21.15.100:0/3402924746 shutdown_connections 2024-05-10T11:35:00.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:00.601+0000 7f5451616700 1 -- 172.21.15.100:0/3402924746 wait complete. 2024-05-10T11:35:00.916 INFO:tasks.workunit.client.0.smithi100.stdout: 10 29982 9.61 MB/sec execute 54 sec latency 66.127 ms 2024-05-10T11:35:01.916 INFO:tasks.workunit.client.0.smithi100.stdout: 10 30138 9.61 MB/sec execute 55 sec latency 65.618 ms 2024-05-10T11:35:02.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:35:01 smithi178 ceph-mon[33359]: pgmap v920: 129 pgs: 129 active+clean; 422 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 795 KiB/s rd, 4.0 MiB/s wr, 274 op/s 2024-05-10T11:35:02.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:35:01 smithi188 ceph-mon[34075]: pgmap v920: 129 pgs: 129 active+clean; 422 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 795 KiB/s rd, 4.0 MiB/s wr, 274 op/s 2024-05-10T11:35:02.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:35:01 smithi100 ceph-mon[30296]: pgmap v920: 129 pgs: 129 active+clean; 422 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 795 KiB/s rd, 4.0 MiB/s wr, 274 op/s 2024-05-10T11:35:02.403 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T11:35:02.404 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T11:35:02.749 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:35:02.916 INFO:tasks.workunit.client.0.smithi100.stdout: 10 30304 9.66 MB/sec execute 56 sec latency 239.410 ms 2024-05-10T11:35:03.916 INFO:tasks.workunit.client.0.smithi100.stdout: 10 30544 9.79 MB/sec execute 57 sec latency 57.982 ms 2024-05-10T11:35:04.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:35:03 smithi178 ceph-mon[33359]: pgmap v921: 129 pgs: 129 active+clean; 429 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 961 KiB/s rd, 4.2 MiB/s wr, 299 op/s 2024-05-10T11:35:04.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:35:03 smithi188 ceph-mon[34075]: pgmap v921: 129 pgs: 129 active+clean; 429 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 961 KiB/s rd, 4.2 MiB/s wr, 299 op/s 2024-05-10T11:35:04.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:35:03 smithi100 ceph-mon[30296]: pgmap v921: 129 pgs: 129 active+clean; 429 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 961 KiB/s rd, 4.2 MiB/s wr, 299 op/s 2024-05-10T11:35:04.916 INFO:tasks.workunit.client.0.smithi100.stdout: 10 30675 9.72 MB/sec execute 58 sec latency 113.531 ms 2024-05-10T11:35:05.916 INFO:tasks.workunit.client.0.smithi100.stdout: 10 30779 9.67 MB/sec execute 59 sec latency 123.138 ms 2024-05-10T11:35:06.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:35:05 smithi178 ceph-mon[33359]: pgmap v922: 129 pgs: 129 active+clean; 432 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 4.3 MiB/s wr, 256 op/s 2024-05-10T11:35:06.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:35:05 smithi188 ceph-mon[34075]: pgmap v922: 129 pgs: 129 active+clean; 432 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 4.3 MiB/s wr, 256 op/s 2024-05-10T11:35:06.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:35:05 smithi100 ceph-mon[30296]: pgmap v922: 129 pgs: 129 active+clean; 432 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 4.3 MiB/s wr, 256 op/s 2024-05-10T11:35:06.916 INFO:tasks.workunit.client.0.smithi100.stdout: 10 30940 9.71 MB/sec execute 60 sec latency 140.875 ms 2024-05-10T11:35:07.916 INFO:tasks.workunit.client.0.smithi100.stdout: 10 31086 9.68 MB/sec execute 61 sec latency 133.424 ms 2024-05-10T11:35:08.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:35:07 smithi178 ceph-mon[33359]: pgmap v923: 129 pgs: 129 active+clean; 437 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 5.0 MiB/s wr, 320 op/s 2024-05-10T11:35:08.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:35:07 smithi188 ceph-mon[34075]: pgmap v923: 129 pgs: 129 active+clean; 437 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 5.0 MiB/s wr, 320 op/s 2024-05-10T11:35:08.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:35:07 smithi100 ceph-mon[30296]: pgmap v923: 129 pgs: 129 active+clean; 437 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 5.0 MiB/s wr, 320 op/s 2024-05-10T11:35:08.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.614+0000 7f7078964700 1 -- 172.21.15.100:0/177639672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7074102d60 msgr2=0x7f707410f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:08.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.614+0000 7f7078964700 1 --2- 172.21.15.100:0/177639672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7074102d60 0x7f707410f250 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f706c00b0a0 tx=0x7f706c00b3b0 comp rx=0 tx=0).stop 2024-05-10T11:35:08.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.614+0000 7f7078964700 1 -- 172.21.15.100:0/177639672 shutdown_connections 2024-05-10T11:35:08.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.614+0000 7f7078964700 1 --2- 172.21.15.100:0/177639672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7074102d60 0x7f707410f250 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:08.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.614+0000 7f7078964700 1 --2- 172.21.15.100:0/177639672 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7074108410 0x7f70741087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:08.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.614+0000 7f7078964700 1 --2- 172.21.15.100:0/177639672 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70741023a0 0x7f7074102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:08.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.614+0000 7f7078964700 1 -- 172.21.15.100:0/177639672 >> 172.21.15.100:0/177639672 conn(0x7f7074079350 msgr2=0x7f7074079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:08.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.614+0000 7f7078964700 1 -- 172.21.15.100:0/177639672 shutdown_connections 2024-05-10T11:35:08.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.615+0000 7f7078964700 1 -- 172.21.15.100:0/177639672 wait complete. 2024-05-10T11:35:08.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.616+0000 7f7078964700 1 Processor -- start 2024-05-10T11:35:08.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.616+0000 7f7078964700 1 -- start start 2024-05-10T11:35:08.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.616+0000 7f7078964700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70741023a0 0x7f707419d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:08.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.616+0000 7f7078964700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7074102d60 0x7f707419dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:08.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.616+0000 7f7078964700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7074108410 0x7f7074197660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:08.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.616+0000 7f7078964700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f70741113a0 con 0x7f7074102d60 2024-05-10T11:35:08.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.616+0000 7f7078964700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7074111220 con 0x7f7074108410 2024-05-10T11:35:08.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.616+0000 7f7078964700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7074111520 con 0x7f70741023a0 2024-05-10T11:35:08.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.616+0000 7f707259c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70741023a0 0x7f707419d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:08.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.616+0000 7f7071d9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7074102d60 0x7f707419dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:08.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.616+0000 7f7071d9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7074102d60 0x7f707419dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48070/0 (socket says 172.21.15.100:48070) 2024-05-10T11:35:08.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.616+0000 7f7071d9b700 1 -- 172.21.15.100:0/2185505947 learned_addr learned my addr 172.21.15.100:0/2185505947 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:35:08.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.616+0000 7f7072d9d700 1 --2- 172.21.15.100:0/2185505947 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7074108410 0x7f7074197660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:08.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.617+0000 7f7071d9b700 1 -- 172.21.15.100:0/2185505947 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70741023a0 msgr2=0x7f707419d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:08.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.617+0000 7f7071d9b700 1 --2- 172.21.15.100:0/2185505947 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70741023a0 0x7f707419d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:08.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.617+0000 7f7071d9b700 1 -- 172.21.15.100:0/2185505947 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7074108410 msgr2=0x7f7074197660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:08.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.617+0000 7f7071d9b700 1 --2- 172.21.15.100:0/2185505947 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7074108410 0x7f7074197660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:08.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.617+0000 7f7071d9b700 1 -- 172.21.15.100:0/2185505947 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7074197f40 con 0x7f7074102d60 2024-05-10T11:35:08.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.617+0000 7f7072d9d700 1 --2- 172.21.15.100:0/2185505947 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7074108410 0x7f7074197660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:35:08.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.617+0000 7f7071d9b700 1 --2- 172.21.15.100:0/2185505947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7074102d60 0x7f707419dad0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f706800eb30 tx=0x7f706800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:08.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.617+0000 7f70637fe700 1 -- 172.21.15.100:0/2185505947 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f706800cc90 con 0x7f7074102d60 2024-05-10T11:35:08.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.617+0000 7f70637fe700 1 -- 172.21.15.100:0/2185505947 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f706800ce30 con 0x7f7074102d60 2024-05-10T11:35:08.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.617+0000 7f7078964700 1 -- 172.21.15.100:0/2185505947 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70741981d0 con 0x7f7074102d60 2024-05-10T11:35:08.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.617+0000 7f70637fe700 1 -- 172.21.15.100:0/2185505947 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7068010430 con 0x7f7074102d60 2024-05-10T11:35:08.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.617+0000 7f7078964700 1 -- 172.21.15.100:0/2185505947 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70741a4510 con 0x7f7074102d60 2024-05-10T11:35:08.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.620+0000 7f70637fe700 1 -- 172.21.15.100:0/2185505947 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f7068004020 con 0x7f7074102d60 2024-05-10T11:35:08.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.621+0000 7f70637fe700 1 --2- 172.21.15.100:0/2185505947 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f705c0b2980 0x7f705c0b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:08.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.621+0000 7f70637fe700 1 -- 172.21.15.100:0/2185505947 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f70680e2790 con 0x7f7074102d60 2024-05-10T11:35:08.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.621+0000 7f707259c700 1 --2- 172.21.15.100:0/2185505947 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f705c0b2980 0x7f705c0b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:08.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.621+0000 7f70831f7b80 1 Processor -- start 2024-05-10T11:35:08.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.622+0000 7f70831f7b80 1 -- start start 2024-05-10T11:35:08.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.622+0000 7f707259c700 1 --2- 172.21.15.100:0/2185505947 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f705c0b2980 0x7f705c0b4e40 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f706400be00 tx=0x7f706400d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:08.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.622+0000 7f70831f7b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4ecbf5420 0x55d4ecbf5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:08.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.622+0000 7f70831f7b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d4ecbe7370 0x55d4ecbe77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:08.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.622+0000 7f70831f7b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d4ecbe7d30 0x55d4ecbfc150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:08.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.622+0000 7f70831f7b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55d4ecb398e0 con 0x55d4ecbe7d30 2024-05-10T11:35:08.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.622+0000 7f70831f7b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d4ecad8840 con 0x55d4ecbf5420 2024-05-10T11:35:08.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.622+0000 7f70831f7b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d4ecb358a0 con 0x55d4ecbe7370 2024-05-10T11:35:08.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.622+0000 7f7072d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d4ecbe7d30 0x55d4ecbfc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:08.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.622+0000 7f7072d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d4ecbe7d30 0x55d4ecbfc150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48082/0 (socket says 172.21.15.100:48082) 2024-05-10T11:35:08.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.622+0000 7f7072d9d700 1 -- 172.21.15.100:0/2625028293 learned_addr learned my addr 172.21.15.100:0/2625028293 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:35:08.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.622+0000 7f7071d9b700 1 --2- 172.21.15.100:0/2625028293 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d4ecbe7370 0x55d4ecbe77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:08.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.622+0000 7f707259c700 1 --2- 172.21.15.100:0/2625028293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4ecbf5420 0x55d4ecbf5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:08.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.622+0000 7f7072d9d700 1 -- 172.21.15.100:0/2625028293 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d4ecbe7370 msgr2=0x55d4ecbe77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:08.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.622+0000 7f7072d9d700 1 --2- 172.21.15.100:0/2625028293 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d4ecbe7370 0x55d4ecbe77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:08.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.622+0000 7f7072d9d700 1 -- 172.21.15.100:0/2625028293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4ecbf5420 msgr2=0x55d4ecbf5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:08.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.622+0000 7f7072d9d700 1 --2- 172.21.15.100:0/2625028293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4ecbf5420 0x55d4ecbf5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:08.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.622+0000 7f7072d9d700 1 -- 172.21.15.100:0/2625028293 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4eca86950 con 0x55d4ecbe7d30 2024-05-10T11:35:08.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.623+0000 7f7072d9d700 1 --2- 172.21.15.100:0/2625028293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d4ecbe7d30 0x55d4ecbfc150 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f706c015000 tx=0x7f706c009d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=309b04586009403d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:08.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.623+0000 7f7060ff9700 1 -- 172.21.15.100:0/2625028293 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f706c021070 con 0x55d4ecbe7d30 2024-05-10T11:35:08.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.623+0000 7f7060ff9700 1 -- 172.21.15.100:0/2625028293 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f706c0074e0 con 0x55d4ecbe7d30 2024-05-10T11:35:08.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.623+0000 7f7060ff9700 1 -- 172.21.15.100:0/2625028293 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f706c0052a0 con 0x55d4ecbe7d30 2024-05-10T11:35:08.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.624+0000 7f70831f7b80 1 -- 172.21.15.100:0/2625028293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d4ecbe7d30 msgr2=0x55d4ecbfc150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:08.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.624+0000 7f70831f7b80 1 --2- 172.21.15.100:0/2625028293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d4ecbe7d30 0x55d4ecbfc150 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f706c015000 tx=0x7f706c009d50 comp rx=0 tx=0).stop 2024-05-10T11:35:08.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.624+0000 7f70831f7b80 1 -- 172.21.15.100:0/2625028293 shutdown_connections 2024-05-10T11:35:08.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.624+0000 7f70831f7b80 1 --2- 172.21.15.100:0/2625028293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d4ecbe7d30 0x55d4ecbfc150 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:08.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.624+0000 7f70831f7b80 1 --2- 172.21.15.100:0/2625028293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4ecbf5420 0x55d4ecbf5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:08.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.624+0000 7f70831f7b80 1 --2- 172.21.15.100:0/2625028293 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d4ecbe7370 0x55d4ecbe77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:08.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.624+0000 7f70831f7b80 1 -- 172.21.15.100:0/2625028293 >> 172.21.15.100:0/2625028293 conn(0x55d4ecb13180 msgr2=0x55d4ecbe1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:08.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.624+0000 7f70831f7b80 1 -- 172.21.15.100:0/2625028293 shutdown_connections 2024-05-10T11:35:08.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.624+0000 7f70831f7b80 1 -- 172.21.15.100:0/2625028293 wait complete. 2024-05-10T11:35:08.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.624+0000 7f70831f7b80 1 -- start start 2024-05-10T11:35:08.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.624+0000 7f70831f7b80 1 Processor -- start 2024-05-10T11:35:08.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.625+0000 7f70831f7b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4ecbe7370 0x55d4ecbfbd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:08.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.625+0000 7f70831f7b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d4ecbe7d30 0x55d4ecbf90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:08.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.625+0000 7f70831f7b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d4ecbf5420 0x55d4ecbf9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:08.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.625+0000 7f70831f7b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55d4ecbfc7e0 con 0x55d4ecbf5420 2024-05-10T11:35:08.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.625+0000 7f70831f7b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d4eca9fcb0 con 0x55d4ecbe7370 2024-05-10T11:35:08.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.625+0000 7f70831f7b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d4ecbfc960 con 0x55d4ecbe7d30 2024-05-10T11:35:08.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.625+0000 7f7071d9b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d4ecbe7d30 0x55d4ecbf90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:08.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.625+0000 7f707259c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4ecbe7370 0x55d4ecbfbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:08.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.625+0000 7f7071d9b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d4ecbe7d30 0x55d4ecbf90e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:56458/0 (socket says 172.21.15.100:56458) 2024-05-10T11:35:08.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.625+0000 7f7071d9b700 1 -- 172.21.15.100:0/1651980258 learned_addr learned my addr 172.21.15.100:0/1651980258 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:35:08.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.625+0000 7f707259c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4ecbe7370 0x55d4ecbfbd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:38540/0 (socket says 172.21.15.100:38540) 2024-05-10T11:35:08.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.625+0000 7f7072d9d700 1 --2- 172.21.15.100:0/1651980258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d4ecbf5420 0x55d4ecbf9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:08.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.625+0000 7f70831f7b80 10 client.0 _ll_register_callbacks cb 0x55d4ecad19a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:35:08.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.625+0000 7f7071d9b700 1 -- 172.21.15.100:0/1651980258 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4ecbe7370 msgr2=0x55d4ecbfbd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:08.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.625+0000 7f7071d9b700 1 --2- 172.21.15.100:0/1651980258 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4ecbe7370 0x55d4ecbfbd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:08.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.625+0000 7f7071d9b700 1 -- 172.21.15.100:0/1651980258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d4ecbf5420 msgr2=0x55d4ecbf9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:08.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.625+0000 7f7071d9b700 1 --2- 172.21.15.100:0/1651980258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d4ecbf5420 0x55d4ecbf9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:08.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.625+0000 7f7071d9b700 1 -- 172.21.15.100:0/1651980258 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4ecad5710 con 0x55d4ecbe7d30 2024-05-10T11:35:08.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.625+0000 7f7071d9b700 1 --2- 172.21.15.100:0/1651980258 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d4ecbe7d30 0x55d4ecbf90e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f70680a6c60 tx=0x7f70680a5270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dda8ccb8f3914c55 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:08.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.625+0000 7f7059ffb700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:35:08.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.626+0000 7f7059ffb700 1 -- 172.21.15.100:0/1651980258 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70680aa0f0 con 0x55d4ecbe7d30 2024-05-10T11:35:08.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.626+0000 7f7059ffb700 1 -- 172.21.15.100:0/1651980258 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f70680a9c50 con 0x55d4ecbe7d30 2024-05-10T11:35:08.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.626+0000 7f70831f7b80 1 -- 172.21.15.100:0/1651980258 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f706c0074e0 con 0x55d4ecbe7d30 2024-05-10T11:35:08.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.626+0000 7f7059ffb700 1 -- 172.21.15.100:0/1651980258 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f70680a99f0 con 0x55d4ecbe7d30 2024-05-10T11:35:08.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.626+0000 7f70831f7b80 10 client.28141 fetch_fsmap learned FSMap version 40 2024-05-10T11:35:08.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.626+0000 7f70831f7b80 1 -- 172.21.15.100:0/1651980258 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55d4ecc06f80 con 0x55d4ecbe7d30 2024-05-10T11:35:08.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.626+0000 7f7059ffb700 1 -- 172.21.15.100:0/1651980258 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f706c0074e0 con 0x55d4ecbe7d30 2024-05-10T11:35:08.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.626+0000 7f7059ffb700 1 -- 172.21.15.100:0/1651980258 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 40) ==== 8350+0+0 (secure 0 0 0) 0x7f70680cc070 con 0x55d4ecbe7d30 2024-05-10T11:35:08.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.626+0000 7f70831f7b80 10 client.28141 fetch_fsmap finished waiting for FSMap version 40 2024-05-10T11:35:08.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.626+0000 7f70831f7b80 10 client.28141 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:35:08.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.627+0000 7f70831f7b80 20 client.28141 populate_metadata read hostname 'smithi100' 2024-05-10T11:35:08.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.627+0000 7f70831f7b80 1 --2- 172.21.15.100:0/1651980258 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d4ecc095c0 0x55d4ecc0b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:08.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.627+0000 7f70831f7b80 4 client.28141 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:35:08.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.627+0000 7f70831f7b80 1 -- 172.21.15.100:0/1651980258 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d4ecbb36d0 con 0x55d4ecc095c0 2024-05-10T11:35:08.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.627+0000 7f707259c700 1 --2- 172.21.15.100:0/1651980258 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d4ecc095c0 0x55d4ecc0b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:08.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.627+0000 7f707259c700 1 --2- 172.21.15.100:0/1651980258 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d4ecc095c0 0x55d4ecc0b9d0 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=967c5da5a3e1283e server_cookie=68f6afa193301d18 in_seq=0 out_seq=0 2024-05-10T11:35:08.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.627+0000 7f7059ffb700 10 client.28141 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:35:08.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.628+0000 7f7059ffb700 1 -- 172.21.15.100:0/1651980258 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d4ecbb36d0 con 0x55d4ecc095c0 2024-05-10T11:35:08.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.628+0000 7f7059ffb700 10 client.28141 handle_command_reply: tid=0 2024-05-10T11:35:08.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.628+0000 7f70831f7b80 1 client.28141 shutdown 2024-05-10T11:35:08.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.628+0000 7f70831f7b80 1 -- 172.21.15.100:0/1651980258 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d4ecbe7d30 msgr2=0x55d4ecbf90e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:08.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.628+0000 7f70831f7b80 1 --2- 172.21.15.100:0/1651980258 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d4ecbe7d30 0x55d4ecbf90e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f70680a6c60 tx=0x7f70680a5270 comp rx=0 tx=0).stop 2024-05-10T11:35:08.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.628+0000 7f70831f7b80 1 -- 172.21.15.100:0/1651980258 shutdown_connections 2024-05-10T11:35:08.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.628+0000 7f70831f7b80 1 --2- 172.21.15.100:0/1651980258 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d4ecc095c0 0x55d4ecc0b9d0 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:08.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.628+0000 7f70831f7b80 1 --2- 172.21.15.100:0/1651980258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d4ecbf5420 0x55d4ecbf9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:08.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.628+0000 7f70831f7b80 1 --2- 172.21.15.100:0/1651980258 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4ecbe7370 0x55d4ecbfbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:08.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.628+0000 7f70831f7b80 1 --2- 172.21.15.100:0/1651980258 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d4ecbe7d30 0x55d4ecbf90e0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:08.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.628+0000 7f7059ffb700 0 client.28141 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:35:08.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.628+0000 7f70831f7b80 1 -- 172.21.15.100:0/1651980258 >> 172.21.15.100:0/1651980258 conn(0x55d4ecb13180 msgr2=0x55d4ecbdc7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:08.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.629+0000 7f70831f7b80 1 -- 172.21.15.100:0/1651980258 shutdown_connections 2024-05-10T11:35:08.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.629+0000 7f70831f7b80 1 -- 172.21.15.100:0/1651980258 wait complete. 2024-05-10T11:35:08.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.629+0000 7f70831f7b80 20 client.28141 trim_cache size 0 max 16384 2024-05-10T11:35:08.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.659+0000 7f70831f7b80 1 Processor -- start 2024-05-10T11:35:08.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.659+0000 7f70831f7b80 1 -- start start 2024-05-10T11:35:08.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.659+0000 7f70831f7b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d4ecbe7370 0x55d4ecbf3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:08.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.659+0000 7f70831f7b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4ecbe7d30 0x55d4ecc05870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:08.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.659+0000 7f70831f7b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d4ecbf5420 0x55d4ecbdc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:08.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.659+0000 7f70831f7b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f70741af7c0 con 0x55d4ecbf5420 2024-05-10T11:35:08.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.659+0000 7f70831f7b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f70741ad020 con 0x55d4ecbe7d30 2024-05-10T11:35:08.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.659+0000 7f70831f7b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f70741aae70 con 0x55d4ecbe7370 2024-05-10T11:35:08.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.659+0000 7f7072d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d4ecbf5420 0x55d4ecbdc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:08.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.659+0000 7f7072d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d4ecbf5420 0x55d4ecbdc620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48100/0 (socket says 172.21.15.100:48100) 2024-05-10T11:35:08.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.659+0000 7f7072d9d700 1 -- 172.21.15.100:0/3949454693 learned_addr learned my addr 172.21.15.100:0/3949454693 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:35:08.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.659+0000 7f7071d9b700 1 --2- 172.21.15.100:0/3949454693 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4ecbe7d30 0x55d4ecc05870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:08.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.660+0000 7f707259c700 1 --2- 172.21.15.100:0/3949454693 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d4ecbe7370 0x55d4ecbf3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:08.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.660+0000 7f7071d9b700 1 -- 172.21.15.100:0/3949454693 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d4ecbe7370 msgr2=0x55d4ecbf3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:08.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.660+0000 7f7071d9b700 1 --2- 172.21.15.100:0/3949454693 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d4ecbe7370 0x55d4ecbf3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:08.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.660+0000 7f7071d9b700 1 -- 172.21.15.100:0/3949454693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d4ecbf5420 msgr2=0x55d4ecbdc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:08.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.660+0000 7f7071d9b700 1 --2- 172.21.15.100:0/3949454693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d4ecbf5420 0x55d4ecbdc620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:08.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.660+0000 7f7071d9b700 1 -- 172.21.15.100:0/3949454693 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4ecbf8040 con 0x55d4ecbe7d30 2024-05-10T11:35:08.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.660+0000 7f7071d9b700 1 --2- 172.21.15.100:0/3949454693 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4ecbe7d30 0x55d4ecc05870 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f70680a96f0 tx=0x7f7068000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f368bfe0f692bbe server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:08.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.662+0000 7f705bfff700 1 -- 172.21.15.100:0/3949454693 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70680aa0f0 con 0x55d4ecbe7d30 2024-05-10T11:35:08.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.662+0000 7f705bfff700 1 -- 172.21.15.100:0/3949454693 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f70680c3070 con 0x55d4ecbe7d30 2024-05-10T11:35:08.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.662+0000 7f705bfff700 1 -- 172.21.15.100:0/3949454693 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70680b2850 con 0x55d4ecbe7d30 2024-05-10T11:35:08.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.663+0000 7f70831f7b80 1 -- 172.21.15.100:0/3949454693 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4ecbe7d30 msgr2=0x55d4ecc05870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:08.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.663+0000 7f70831f7b80 1 --2- 172.21.15.100:0/3949454693 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4ecbe7d30 0x55d4ecc05870 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f70680a96f0 tx=0x7f7068000bc0 comp rx=0 tx=0).stop 2024-05-10T11:35:08.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.663+0000 7f70831f7b80 1 -- 172.21.15.100:0/3949454693 shutdown_connections 2024-05-10T11:35:08.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.663+0000 7f70831f7b80 1 --2- 172.21.15.100:0/3949454693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d4ecbf5420 0x55d4ecbdc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:08.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.663+0000 7f70831f7b80 1 --2- 172.21.15.100:0/3949454693 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d4ecbe7370 0x55d4ecbf3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:08.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.663+0000 7f70831f7b80 1 --2- 172.21.15.100:0/3949454693 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4ecbe7d30 0x55d4ecc05870 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:08.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.663+0000 7f70831f7b80 1 -- 172.21.15.100:0/3949454693 >> 172.21.15.100:0/3949454693 conn(0x55d4ecb13180 msgr2=0x55d4ecbdbf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:08.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.663+0000 7f70831f7b80 1 -- 172.21.15.100:0/3949454693 shutdown_connections 2024-05-10T11:35:08.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.663+0000 7f70831f7b80 1 -- 172.21.15.100:0/3949454693 wait complete. 2024-05-10T11:35:08.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.664+0000 7f70831f7b80 1 -- start start 2024-05-10T11:35:08.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.664+0000 7f70831f7b80 1 Processor -- start 2024-05-10T11:35:08.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.664+0000 7f70831f7b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d4ecbe7370 0x55d4ecbfebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:08.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.664+0000 7f70831f7b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4ecbe7d30 0x55d4ecbff100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:08.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.664+0000 7f707259c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d4ecbe7370 0x55d4ecbfebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:08.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.664+0000 7f707259c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d4ecbe7370 0x55d4ecbfebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48114/0 (socket says 172.21.15.100:48114) 2024-05-10T11:35:08.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.664+0000 7f70831f7b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d4ecbf5420 0x55d4ecc73e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:08.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.664+0000 7f707259c700 1 -- 172.21.15.100:0/3145786977 learned_addr learned my addr 172.21.15.100:0/3145786977 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:35:08.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.664+0000 7f70831f7b80 1 -- 172.21.15.100:0/3145786977 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f70741ab4e0 con 0x55d4ecbe7370 2024-05-10T11:35:08.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.664+0000 7f70831f7b80 1 -- 172.21.15.100:0/3145786977 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f70741ac950 con 0x55d4ecbe7d30 2024-05-10T11:35:08.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.664+0000 7f70831f7b80 1 -- 172.21.15.100:0/3145786977 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f70741abbb0 con 0x55d4ecbf5420 2024-05-10T11:35:08.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.664+0000 7f70831f7b80 10 client.0 _ll_register_callbacks cb 0x55d4ecad19a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:35:08.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.665+0000 7f7071d9b700 1 --2- 172.21.15.100:0/3145786977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4ecbe7d30 0x55d4ecbff100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:08.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.665+0000 7f7072d9d700 1 --2- 172.21.15.100:0/3145786977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d4ecbf5420 0x55d4ecc73e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:08.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.665+0000 7f707259c700 1 -- 172.21.15.100:0/3145786977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d4ecbf5420 msgr2=0x55d4ecc73e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:08.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.665+0000 7f707259c700 1 --2- 172.21.15.100:0/3145786977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d4ecbf5420 0x55d4ecc73e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:08.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.665+0000 7f707259c700 1 -- 172.21.15.100:0/3145786977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4ecbe7d30 msgr2=0x55d4ecbff100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:08.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.665+0000 7f707259c700 1 --2- 172.21.15.100:0/3145786977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4ecbe7d30 0x55d4ecbff100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:08.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.665+0000 7f707259c700 1 -- 172.21.15.100:0/3145786977 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4ecbff9a0 con 0x55d4ecbe7370 2024-05-10T11:35:08.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.665+0000 7f7072d9d700 1 --2- 172.21.15.100:0/3145786977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d4ecbf5420 0x55d4ecc73e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:35:08.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.665+0000 7f707259c700 1 --2- 172.21.15.100:0/3145786977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d4ecbe7370 0x55d4ecbfebc0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f706400d010 tx=0x7f7064004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50b020f6f3c945c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:08.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.665+0000 7f7059ffb700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:35:08.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.665+0000 7f7059ffb700 1 -- 172.21.15.100:0/3145786977 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f706402c460 con 0x55d4ecbe7370 2024-05-10T11:35:08.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.665+0000 7f7059ffb700 1 -- 172.21.15.100:0/3145786977 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f7064021040 con 0x55d4ecbe7370 2024-05-10T11:35:08.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.665+0000 7f70831f7b80 1 -- 172.21.15.100:0/3145786977 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70680c3070 con 0x55d4ecbe7370 2024-05-10T11:35:08.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.666+0000 7f7059ffb700 1 -- 172.21.15.100:0/3145786977 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f706403d050 con 0x55d4ecbe7370 2024-05-10T11:35:08.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.666+0000 7f70831f7b80 10 client.18738 fetch_fsmap learned FSMap version 40 2024-05-10T11:35:08.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.666+0000 7f70831f7b80 1 -- 172.21.15.100:0/3145786977 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55d4ecc754f0 con 0x55d4ecbe7370 2024-05-10T11:35:08.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.666+0000 7f7059ffb700 1 -- 172.21.15.100:0/3145786977 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f70680c3070 con 0x55d4ecbe7370 2024-05-10T11:35:08.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.666+0000 7f7059ffb700 1 -- 172.21.15.100:0/3145786977 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) ==== 8350+0+0 (secure 0 0 0) 0x7f7064035770 con 0x55d4ecbe7370 2024-05-10T11:35:08.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.666+0000 7f70831f7b80 10 client.18738 fetch_fsmap finished waiting for FSMap version 40 2024-05-10T11:35:08.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.666+0000 7f70831f7b80 10 client.18738 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:35:08.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.666+0000 7f70831f7b80 20 client.18738 populate_metadata read hostname 'smithi100' 2024-05-10T11:35:08.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.666+0000 7f70831f7b80 1 --2- 172.21.15.100:0/3145786977 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d4ecc75be0 0x55d4ecc78000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:08.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.666+0000 7f70831f7b80 4 client.18738 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T11:35:08.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.666+0000 7f70831f7b80 1 -- 172.21.15.100:0/3145786977 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d4ecb472c0 con 0x55d4ecc75be0 2024-05-10T11:35:08.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.667+0000 7f7071d9b700 1 --2- 172.21.15.100:0/3145786977 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d4ecc75be0 0x55d4ecc78000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:08.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.667+0000 7f7059ffb700 10 client.18738 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:35:08.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.667+0000 7f7071d9b700 1 --2- 172.21.15.100:0/3145786977 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d4ecc75be0 0x55d4ecc78000 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8432b0a5270dd1d7 server_cookie=54a8b4294cf6adef in_seq=0 out_seq=0 2024-05-10T11:35:08.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.668+0000 7f7059ffb700 1 -- 172.21.15.100:0/3145786977 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d4ecb472c0 con 0x55d4ecc75be0 2024-05-10T11:35:08.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.668+0000 7f7059ffb700 10 client.18738 handle_command_reply: tid=0 2024-05-10T11:35:08.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.668+0000 7f70831f7b80 1 client.18738 shutdown 2024-05-10T11:35:08.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.668+0000 7f70831f7b80 1 -- 172.21.15.100:0/3145786977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d4ecbe7370 msgr2=0x55d4ecbfebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:08.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.668+0000 7f70831f7b80 1 --2- 172.21.15.100:0/3145786977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d4ecbe7370 0x55d4ecbfebc0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f706400d010 tx=0x7f7064004750 comp rx=0 tx=0).stop 2024-05-10T11:35:08.079 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:35:08.079 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-05-10T11:35:08.079 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "0cc4ab3a-9228-40ce-82c9-870aa5cb61f3", 2024-05-10T11:35:08.079 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-05-10T11:35:08.079 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:35:08.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.668+0000 7f70831f7b80 1 -- 172.21.15.100:0/3145786977 shutdown_connections 2024-05-10T11:35:08.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.668+0000 7f70831f7b80 1 --2- 172.21.15.100:0/3145786977 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d4ecc75be0 0x55d4ecc78000 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:08.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.668+0000 7f7059ffb700 0 client.18738 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:35:08.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.668+0000 7f70831f7b80 1 --2- 172.21.15.100:0/3145786977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d4ecbf5420 0x55d4ecc73e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:08.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.668+0000 7f70831f7b80 1 --2- 172.21.15.100:0/3145786977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d4ecbe7370 0x55d4ecbfebc0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:08.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.668+0000 7f70831f7b80 1 --2- 172.21.15.100:0/3145786977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4ecbe7d30 0x55d4ecbff100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:08.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.668+0000 7f70831f7b80 1 -- 172.21.15.100:0/3145786977 >> 172.21.15.100:0/3145786977 conn(0x55d4ecb13180 msgr2=0x55d4ecbf3690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:08.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.669+0000 7f70831f7b80 1 -- 172.21.15.100:0/3145786977 shutdown_connections 2024-05-10T11:35:08.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.669+0000 7f70831f7b80 1 -- 172.21.15.100:0/3145786977 wait complete. 2024-05-10T11:35:08.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.669+0000 7f70831f7b80 20 client.18738 trim_cache size 0 max 16384 2024-05-10T11:35:08.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.669+0000 7f7078964700 1 -- 172.21.15.100:0/2185505947 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f705c0b2980 msgr2=0x7f705c0b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:08.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.669+0000 7f7078964700 1 --2- 172.21.15.100:0/2185505947 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f705c0b2980 0x7f705c0b4e40 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f706400be00 tx=0x7f706400d040 comp rx=0 tx=0).stop 2024-05-10T11:35:08.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.669+0000 7f7078964700 1 -- 172.21.15.100:0/2185505947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7074102d60 msgr2=0x7f707419dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:08.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.669+0000 7f7078964700 1 --2- 172.21.15.100:0/2185505947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7074102d60 0x7f707419dad0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f706800eb30 tx=0x7f706800ee40 comp rx=0 tx=0).stop 2024-05-10T11:35:08.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.670+0000 7f7078964700 1 -- 172.21.15.100:0/2185505947 shutdown_connections 2024-05-10T11:35:08.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.670+0000 7f7078964700 1 --2- 172.21.15.100:0/2185505947 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f705c0b2980 0x7f705c0b4e40 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:08.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.670+0000 7f7078964700 1 --2- 172.21.15.100:0/2185505947 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7074108410 0x7f7074197660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:08.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.670+0000 7f7078964700 1 --2- 172.21.15.100:0/2185505947 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70741023a0 0x7f707419d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:08.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.670+0000 7f7078964700 1 --2- 172.21.15.100:0/2185505947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7074102d60 0x7f707419dad0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:08.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.670+0000 7f7078964700 1 -- 172.21.15.100:0/2185505947 >> 172.21.15.100:0/2185505947 conn(0x7f7074079350 msgr2=0x7f70741011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:08.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.670+0000 7f7078964700 1 -- 172.21.15.100:0/2185505947 shutdown_connections 2024-05-10T11:35:08.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:07.670+0000 7f7078964700 1 -- 172.21.15.100:0/2185505947 wait complete. 2024-05-10T11:35:08.916 INFO:tasks.workunit.client.0.smithi100.stdout: 10 31210 9.61 MB/sec execute 62 sec latency 194.059 ms 2024-05-10T11:35:09.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:35:08 smithi178 ceph-mon[33359]: scrub queued for path: / 2024-05-10T11:35:09.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:35:08 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:35:09.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:35:08 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:35:09.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:35:08 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:35:09.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:35:08 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:35:09.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:35:08 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:35:09.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:35:08 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:35:09.015 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:35:08 smithi100 ceph-mon[30296]: scrub queued for path: / 2024-05-10T11:35:09.015 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:35:08 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:35:09.015 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:35:08 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:35:09.015 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:35:08 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:35:09.015 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:35:08 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:35:09.015 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:35:08 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:35:09.015 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:35:08 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:35:09.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:35:08 smithi188 ceph-mon[34075]: scrub queued for path: / 2024-05-10T11:35:09.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:35:08 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:35:09.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:35:08 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:35:09.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:35:08 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:35:09.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:35:08 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:35:09.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:35:08 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:35:09.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:35:08 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:35:09.099 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.099+0000 7f52bb5e4640 1 -- 172.21.15.100:0/4059509049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52b4103c00 msgr2=0x7f52b4104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:09.100 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.099+0000 7f52bb5e4640 1 --2- 172.21.15.100:0/4059509049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52b4103c00 0x7f52b4104080 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f52a4009900 tx=0x7f52a402a410 comp rx=0 tx=0).stop 2024-05-10T11:35:09.100 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.099+0000 7f52bb5e4640 1 -- 172.21.15.100:0/4059509049 shutdown_connections 2024-05-10T11:35:09.100 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.099+0000 7f52bb5e4640 1 --2- 172.21.15.100:0/4059509049 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52b41045c0 0x7f52b410ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:09.100 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.099+0000 7f52bb5e4640 1 --2- 172.21.15.100:0/4059509049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52b4103c00 0x7f52b4104080 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:09.100 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.099+0000 7f52bb5e4640 1 --2- 172.21.15.100:0/4059509049 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f52b4102a10 0x7f52b4102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:09.100 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.099+0000 7f52bb5e4640 1 -- 172.21.15.100:0/4059509049 >> 172.21.15.100:0/4059509049 conn(0x7f52b40fdfa0 msgr2=0x7f52b41003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:09.100 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.099+0000 7f52bb5e4640 1 -- 172.21.15.100:0/4059509049 shutdown_connections 2024-05-10T11:35:09.100 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.099+0000 7f52bb5e4640 1 -- 172.21.15.100:0/4059509049 wait complete. 2024-05-10T11:35:09.102 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.100+0000 7f52bb5e4640 1 Processor -- start 2024-05-10T11:35:09.102 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.100+0000 7f52bb5e4640 1 -- start start 2024-05-10T11:35:09.102 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.100+0000 7f52bb5e4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f52b4102a10 0x7f52b4198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:09.103 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.100+0000 7f52bb5e4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52b4103c00 0x7f52b4199000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:09.103 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.100+0000 7f52bb5e4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52b41045c0 0x7f52b41a0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:09.103 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.100+0000 7f52bb5e4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f52b410e3f0 con 0x7f52b4103c00 2024-05-10T11:35:09.103 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.100+0000 7f52bb5e4640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f52b410e270 con 0x7f52b41045c0 2024-05-10T11:35:09.103 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.100+0000 7f52bb5e4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f52b410e570 con 0x7f52b4102a10 2024-05-10T11:35:09.103 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.100+0000 7f52b9380640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f52b4102a10 0x7f52b4198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:09.103 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.100+0000 7f52b8b7f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52b4103c00 0x7f52b4199000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:09.103 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.100+0000 7f52b8b7f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52b4103c00 0x7f52b4199000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48142/0 (socket says 172.21.15.100:48142) 2024-05-10T11:35:09.103 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.100+0000 7f52b9380640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f52b4102a10 0x7f52b4198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:56502/0 (socket says 172.21.15.100:56502) 2024-05-10T11:35:09.103 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.100+0000 7f52b8b7f640 1 -- 172.21.15.100:0/3985802804 learned_addr learned my addr 172.21.15.100:0/3985802804 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:35:09.103 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.100+0000 7f52b9b81640 1 --2- 172.21.15.100:0/3985802804 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52b41045c0 0x7f52b41a0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:09.103 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.101+0000 7f52b9b81640 1 -- 172.21.15.100:0/3985802804 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f52b4102a10 msgr2=0x7f52b4198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:09.103 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.101+0000 7f52b9b81640 1 --2- 172.21.15.100:0/3985802804 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f52b4102a10 0x7f52b4198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:09.104 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.101+0000 7f52b9b81640 1 -- 172.21.15.100:0/3985802804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52b4103c00 msgr2=0x7f52b4199000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:09.104 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.101+0000 7f52b9b81640 1 --2- 172.21.15.100:0/3985802804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52b4103c00 0x7f52b4199000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:09.104 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.101+0000 7f52b9b81640 1 -- 172.21.15.100:0/3985802804 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52b41a06e0 con 0x7f52b41045c0 2024-05-10T11:35:09.104 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.101+0000 7f52b8b7f640 1 --2- 172.21.15.100:0/3985802804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52b4103c00 0x7f52b4199000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:35:09.104 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.101+0000 7f52b9380640 1 --2- 172.21.15.100:0/3985802804 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f52b4102a10 0x7f52b4198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:35:09.104 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.101+0000 7f52b9b81640 1 --2- 172.21.15.100:0/3985802804 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52b41045c0 0x7f52b41a0070 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f52ac00ec50 tx=0x7f52ac00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:09.104 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.101+0000 7f52aa7fc640 1 -- 172.21.15.100:0/3985802804 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ac003b70 con 0x7f52b41045c0 2024-05-10T11:35:09.104 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.101+0000 7f52bb5e4640 1 -- 172.21.15.100:0/3985802804 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52b41a09d0 con 0x7f52b41045c0 2024-05-10T11:35:09.104 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.101+0000 7f52bb5e4640 1 -- 172.21.15.100:0/3985802804 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52b41a0f20 con 0x7f52b41045c0 2024-05-10T11:35:09.104 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.101+0000 7f52aa7fc640 1 -- 172.21.15.100:0/3985802804 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f52ac003d10 con 0x7f52b41045c0 2024-05-10T11:35:09.104 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.101+0000 7f52aa7fc640 1 -- 172.21.15.100:0/3985802804 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ac0107a0 con 0x7f52b41045c0 2024-05-10T11:35:09.105 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.104+0000 7f52aa7fc640 1 -- 172.21.15.100:0/3985802804 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f52ac003fc0 con 0x7f52b41045c0 2024-05-10T11:35:09.107 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.105+0000 7f52be3d9740 1 Processor -- start 2024-05-10T11:35:09.107 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.106+0000 7f52aa7fc640 1 --2- 172.21.15.100:0/3985802804 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f52a00b2920 0x7f52a00b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:09.107 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.106+0000 7f52be3d9740 1 -- start start 2024-05-10T11:35:09.107 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.106+0000 7f52be3d9740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fbb1d68db0 0x55fbb1d691d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:09.107 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.106+0000 7f52b8b7f640 1 --2- 172.21.15.100:0/3985802804 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f52a00b2920 0x7f52a00b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:09.107 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.106+0000 7f52be3d9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fbb1d697a0 0x55fbb1d6e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:09.107 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.106+0000 7f52aa7fc640 1 -- 172.21.15.100:0/3985802804 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f52ac0e2a10 con 0x7f52b41045c0 2024-05-10T11:35:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.106+0000 7f52be3d9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbb1d72d20 0x55fbb1d751b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.106+0000 7f52be3d9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55fbb1ccd5d0 con 0x55fbb1d72d20 2024-05-10T11:35:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.106+0000 7f52be3d9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55fbb1c86890 con 0x55fbb1d697a0 2024-05-10T11:35:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.106+0000 7f52be3d9740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55fbb1cc9590 con 0x55fbb1d68db0 2024-05-10T11:35:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.106+0000 7f52b9380640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fbb1d68db0 0x55fbb1d691d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.106+0000 7f52b9380640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fbb1d68db0 0x55fbb1d691d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:56516/0 (socket says 172.21.15.100:56516) 2024-05-10T11:35:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.106+0000 7f52b9380640 1 -- 172.21.15.100:0/2689617229 learned_addr learned my addr 172.21.15.100:0/2689617229 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:35:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.106+0000 7f52b9b81640 1 --2- 172.21.15.100:0/2689617229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbb1d72d20 0x55fbb1d751b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.106+0000 7f52b8b7f640 1 --2- 172.21.15.100:0/2689617229 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fbb1d697a0 0x55fbb1d6e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.106+0000 7f52b9380640 1 -- 172.21.15.100:0/2689617229 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fbb1d697a0 msgr2=0x55fbb1d6e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.106+0000 7f52b9380640 1 --2- 172.21.15.100:0/2689617229 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fbb1d697a0 0x55fbb1d6e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.106+0000 7f52b9380640 1 -- 172.21.15.100:0/2689617229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbb1d72d20 msgr2=0x55fbb1d751b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.106+0000 7f52b9380640 1 --2- 172.21.15.100:0/2689617229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbb1d72d20 0x55fbb1d751b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.106+0000 7f52b9380640 1 -- 172.21.15.100:0/2689617229 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbb19df210 con 0x55fbb1d68db0 2024-05-10T11:35:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.107+0000 7f52b8b7f640 1 --2- 172.21.15.100:0/3985802804 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f52a00b2920 0x7f52a00b4de0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f52b4199fd0 tx=0x7f52a4009560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.107+0000 7f52b9380640 1 --2- 172.21.15.100:0/2689617229 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fbb1d68db0 0x55fbb1d691d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f52b00114f0 tx=0x7f52b00119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac5cca544e5e3645 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.107+0000 7f529f7fe640 1 -- 172.21.15.100:0/2689617229 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b0004020 con 0x55fbb1d68db0 2024-05-10T11:35:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.107+0000 7f529f7fe640 1 -- 172.21.15.100:0/2689617229 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f52b00035f0 con 0x55fbb1d68db0 2024-05-10T11:35:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.107+0000 7f529f7fe640 1 -- 172.21.15.100:0/2689617229 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b000f820 con 0x55fbb1d68db0 2024-05-10T11:35:09.114 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.108+0000 7f52be3d9740 1 -- 172.21.15.100:0/2689617229 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fbb1d68db0 msgr2=0x55fbb1d691d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:09.114 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.108+0000 7f52be3d9740 1 --2- 172.21.15.100:0/2689617229 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fbb1d68db0 0x55fbb1d691d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f52b00114f0 tx=0x7f52b00119b0 comp rx=0 tx=0).stop 2024-05-10T11:35:09.114 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.108+0000 7f52be3d9740 1 -- 172.21.15.100:0/2689617229 shutdown_connections 2024-05-10T11:35:09.114 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.108+0000 7f52be3d9740 1 --2- 172.21.15.100:0/2689617229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbb1d72d20 0x55fbb1d751b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:09.114 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.108+0000 7f52be3d9740 1 --2- 172.21.15.100:0/2689617229 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fbb1d697a0 0x55fbb1d6e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:09.114 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.108+0000 7f52be3d9740 1 --2- 172.21.15.100:0/2689617229 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fbb1d68db0 0x55fbb1d691d0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:09.115 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.108+0000 7f52be3d9740 1 -- 172.21.15.100:0/2689617229 >> 172.21.15.100:0/2689617229 conn(0x55fbb1d64420 msgr2=0x55fbb1d64820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:09.115 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.108+0000 7f52be3d9740 1 -- 172.21.15.100:0/2689617229 shutdown_connections 2024-05-10T11:35:09.115 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.108+0000 7f52be3d9740 1 -- 172.21.15.100:0/2689617229 wait complete. 2024-05-10T11:35:09.115 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.108+0000 7f52be3d9740 1 -- start start 2024-05-10T11:35:09.115 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.108+0000 7f52be3d9740 1 Processor -- start 2024-05-10T11:35:09.115 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.109+0000 7f52be3d9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fbb1d68db0 0x55fbb1d77380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:09.115 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.109+0000 7f52be3d9740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fbb1d697a0 0x55fbb1d778c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:09.115 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.109+0000 7f52be3d9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbb1d72d20 0x55fbb1d7e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:09.115 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.109+0000 7f52be3d9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55fbb1d75cd0 con 0x55fbb1d72d20 2024-05-10T11:35:09.115 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.109+0000 7f52be3d9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55fbb1c8a370 con 0x55fbb1d68db0 2024-05-10T11:35:09.115 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.109+0000 7f52be3d9740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55fbb1d75e50 con 0x55fbb1d697a0 2024-05-10T11:35:09.115 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.109+0000 7f52be3d9740 10 client.0 _ll_register_callbacks cb 0x55fbb19dea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:35:09.116 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.109+0000 7f52b9b81640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbb1d72d20 0x55fbb1d7e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:09.116 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.109+0000 7f52b9380640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fbb1d68db0 0x55fbb1d77380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:09.116 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.109+0000 7f52b9b81640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbb1d72d20 0x55fbb1d7e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48154/0 (socket says 172.21.15.100:48154) 2024-05-10T11:35:09.116 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.109+0000 7f52b8b7f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fbb1d697a0 0x55fbb1d778c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:09.116 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.109+0000 7f52b9b81640 1 -- 172.21.15.100:0/3863312665 learned_addr learned my addr 172.21.15.100:0/3863312665 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:35:09.116 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.109+0000 7f52b9380640 1 -- 172.21.15.100:0/3863312665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fbb1d697a0 msgr2=0x55fbb1d778c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:09.116 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.109+0000 7f52b9380640 1 --2- 172.21.15.100:0/3863312665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fbb1d697a0 0x55fbb1d778c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:09.116 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.109+0000 7f52b9380640 1 -- 172.21.15.100:0/3863312665 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbb1d72d20 msgr2=0x55fbb1d7e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:09.116 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.109+0000 7f52b9380640 1 --2- 172.21.15.100:0/3863312665 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbb1d72d20 0x55fbb1d7e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:09.116 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.109+0000 7f52b9380640 1 -- 172.21.15.100:0/3863312665 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbb1aab620 con 0x55fbb1d68db0 2024-05-10T11:35:09.116 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.110+0000 7f52b9380640 1 --2- 172.21.15.100:0/3863312665 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fbb1d68db0 0x55fbb1d77380 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f52b001b040 tx=0x7f52b0004260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=709a8bf5224c8f91 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:09.116 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.110+0000 7f529cff9640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:35:09.116 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.110+0000 7f529cff9640 1 -- 172.21.15.100:0/3863312665 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b0028020 con 0x55fbb1d68db0 2024-05-10T11:35:09.116 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.110+0000 7f529cff9640 1 -- 172.21.15.100:0/3863312665 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f52b0016040 con 0x55fbb1d68db0 2024-05-10T11:35:09.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.110+0000 7f52be3d9740 1 -- 172.21.15.100:0/3863312665 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52b00035f0 con 0x55fbb1d68db0 2024-05-10T11:35:09.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.110+0000 7f529cff9640 1 -- 172.21.15.100:0/3863312665 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f52b0033050 con 0x55fbb1d68db0 2024-05-10T11:35:09.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.110+0000 7f52be3d9740 10 client.28061 fetch_fsmap learned FSMap version 40 2024-05-10T11:35:09.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.110+0000 7f52be3d9740 1 -- 172.21.15.100:0/3863312665 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55fbb1d818e0 con 0x55fbb1d68db0 2024-05-10T11:35:09.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.110+0000 7f529cff9640 1 -- 172.21.15.100:0/3863312665 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f52b00035f0 con 0x55fbb1d68db0 2024-05-10T11:35:09.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.110+0000 7f529cff9640 1 -- 172.21.15.100:0/3863312665 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 40) ==== 8350+0+0 (secure 0 0 0) 0x7f52b00049c0 con 0x55fbb1d68db0 2024-05-10T11:35:09.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.110+0000 7f52be3d9740 10 client.28061 fetch_fsmap finished waiting for FSMap version 40 2024-05-10T11:35:09.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.110+0000 7f52be3d9740 10 client.28061 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:35:09.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.110+0000 7f52be3d9740 20 client.28061 populate_metadata read hostname 'smithi100' 2024-05-10T11:35:09.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.110+0000 7f52be3d9740 1 --2- 172.21.15.100:0/3863312665 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55fbb1d82e70 0x55fbb1d85260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:09.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.110+0000 7f52be3d9740 4 client.28061 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:35:09.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.110+0000 7f52be3d9740 1 -- 172.21.15.100:0/3863312665 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fbb1cdb1e0 con 0x55fbb1d82e70 2024-05-10T11:35:09.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.111+0000 7f52b8b7f640 1 --2- 172.21.15.100:0/3863312665 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55fbb1d82e70 0x55fbb1d85260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:09.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.111+0000 7f529cff9640 10 client.28061 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:35:09.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.111+0000 7f52b8b7f640 1 --2- 172.21.15.100:0/3863312665 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55fbb1d82e70 0x55fbb1d85260 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce3ee223cbe92e74 server_cookie=371e758304f0bcb4 in_seq=0 out_seq=0 2024-05-10T11:35:09.118 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.112+0000 7f529cff9640 1 -- 172.21.15.100:0/3863312665 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fbb1cdb1e0 con 0x55fbb1d82e70 2024-05-10T11:35:09.118 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.112+0000 7f529cff9640 10 client.28061 handle_command_reply: tid=0 2024-05-10T11:35:09.118 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.112+0000 7f52be3d9740 1 client.28061 shutdown 2024-05-10T11:35:09.118 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.112+0000 7f52be3d9740 1 -- 172.21.15.100:0/3863312665 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fbb1d68db0 msgr2=0x55fbb1d77380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:09.118 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.112+0000 7f52be3d9740 1 --2- 172.21.15.100:0/3863312665 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fbb1d68db0 0x55fbb1d77380 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f52b001b040 tx=0x7f52b0004260 comp rx=0 tx=0).stop 2024-05-10T11:35:09.118 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.112+0000 7f52be3d9740 1 -- 172.21.15.100:0/3863312665 shutdown_connections 2024-05-10T11:35:09.118 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.112+0000 7f52be3d9740 1 --2- 172.21.15.100:0/3863312665 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55fbb1d82e70 0x55fbb1d85260 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:09.118 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.112+0000 7f52be3d9740 1 --2- 172.21.15.100:0/3863312665 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbb1d72d20 0x55fbb1d7e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:09.118 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.112+0000 7f529cff9640 0 client.28061 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:35:09.118 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.112+0000 7f52be3d9740 1 --2- 172.21.15.100:0/3863312665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fbb1d697a0 0x55fbb1d778c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:09.118 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.112+0000 7f52be3d9740 1 --2- 172.21.15.100:0/3863312665 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fbb1d68db0 0x55fbb1d77380 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:09.118 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.112+0000 7f52be3d9740 1 -- 172.21.15.100:0/3863312665 >> 172.21.15.100:0/3863312665 conn(0x55fbb1d64420 msgr2=0x55fbb1d63580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:09.118 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.112+0000 7f52be3d9740 1 -- 172.21.15.100:0/3863312665 shutdown_connections 2024-05-10T11:35:09.119 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.112+0000 7f52be3d9740 1 -- 172.21.15.100:0/3863312665 wait complete. 2024-05-10T11:35:09.119 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.112+0000 7f52be3d9740 20 client.28061 trim_cache size 0 max 16384 2024-05-10T11:35:09.142 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.136+0000 7f52be3d9740 1 Processor -- start 2024-05-10T11:35:09.142 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.136+0000 7f52be3d9740 1 -- start start 2024-05-10T11:35:09.142 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.137+0000 7f52be3d9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbb1d68db0 0x55fbb1d769a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:09.142 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.137+0000 7f52be3d9740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fbb1d697a0 0x55fbb1d7e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:09.142 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.137+0000 7f52be3d9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fbb1d72d20 0x55fbb1d76ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:09.142 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.137+0000 7f52be3d9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f52b41ab8f0 con 0x55fbb1d68db0 2024-05-10T11:35:09.142 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.137+0000 7f52be3d9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f52b41a9190 con 0x55fbb1d72d20 2024-05-10T11:35:09.142 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.137+0000 7f52be3d9740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f52b41a6fe0 con 0x55fbb1d697a0 2024-05-10T11:35:09.143 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.137+0000 7f52b9380640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbb1d68db0 0x55fbb1d769a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:09.143 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.137+0000 7f52b9380640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbb1d68db0 0x55fbb1d769a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48164/0 (socket says 172.21.15.100:48164) 2024-05-10T11:35:09.143 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.137+0000 7f52b9380640 1 -- 172.21.15.100:0/3148784039 learned_addr learned my addr 172.21.15.100:0/3148784039 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:35:09.143 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.137+0000 7f52b8b7f640 1 --2- 172.21.15.100:0/3148784039 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fbb1d697a0 0x55fbb1d7e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:09.143 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.137+0000 7f52b9b81640 1 --2- 172.21.15.100:0/3148784039 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fbb1d72d20 0x55fbb1d76ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:09.143 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.137+0000 7f52b9380640 1 -- 172.21.15.100:0/3148784039 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fbb1d697a0 msgr2=0x55fbb1d7e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:09.143 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.137+0000 7f52b9380640 1 --2- 172.21.15.100:0/3148784039 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fbb1d697a0 0x55fbb1d7e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:09.143 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.137+0000 7f52b9380640 1 -- 172.21.15.100:0/3148784039 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fbb1d72d20 msgr2=0x55fbb1d76ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:09.143 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.137+0000 7f52b9380640 1 --2- 172.21.15.100:0/3148784039 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fbb1d72d20 0x55fbb1d76ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:09.143 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.137+0000 7f52b9380640 1 -- 172.21.15.100:0/3148784039 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbb1d69b80 con 0x55fbb1d68db0 2024-05-10T11:35:09.143 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.137+0000 7f52b9b81640 1 --2- 172.21.15.100:0/3148784039 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fbb1d72d20 0x55fbb1d76ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:35:09.143 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.138+0000 7f52b9380640 1 --2- 172.21.15.100:0/3148784039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbb1d68db0 0x55fbb1d769a0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f52b0004e00 tx=0x7f52b0003cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c662f2a97aca3d7b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:09.143 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.138+0000 7f529effd640 1 -- 172.21.15.100:0/3148784039 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b0028070 con 0x55fbb1d68db0 2024-05-10T11:35:09.143 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.138+0000 7f529effd640 1 -- 172.21.15.100:0/3148784039 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f52b0004af0 con 0x55fbb1d68db0 2024-05-10T11:35:09.143 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.138+0000 7f529effd640 1 -- 172.21.15.100:0/3148784039 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b00053c0 con 0x55fbb1d68db0 2024-05-10T11:35:09.144 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.139+0000 7f52be3d9740 1 -- 172.21.15.100:0/3148784039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbb1d68db0 msgr2=0x55fbb1d769a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:09.144 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.139+0000 7f52be3d9740 1 --2- 172.21.15.100:0/3148784039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbb1d68db0 0x55fbb1d769a0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f52b0004e00 tx=0x7f52b0003cb0 comp rx=0 tx=0).stop 2024-05-10T11:35:09.144 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.139+0000 7f52be3d9740 1 -- 172.21.15.100:0/3148784039 shutdown_connections 2024-05-10T11:35:09.144 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.139+0000 7f52be3d9740 1 --2- 172.21.15.100:0/3148784039 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fbb1d72d20 0x55fbb1d76ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:09.144 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.139+0000 7f52be3d9740 1 --2- 172.21.15.100:0/3148784039 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fbb1d697a0 0x55fbb1d7e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:09.144 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.139+0000 7f52be3d9740 1 --2- 172.21.15.100:0/3148784039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbb1d68db0 0x55fbb1d769a0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:09.144 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.139+0000 7f52be3d9740 1 -- 172.21.15.100:0/3148784039 >> 172.21.15.100:0/3148784039 conn(0x55fbb1d64420 msgr2=0x55fbb1d63580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:09.144 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.139+0000 7f52be3d9740 1 -- 172.21.15.100:0/3148784039 shutdown_connections 2024-05-10T11:35:09.144 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.139+0000 7f52be3d9740 1 -- 172.21.15.100:0/3148784039 wait complete. 2024-05-10T11:35:09.144 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.139+0000 7f52be3d9740 1 -- start start 2024-05-10T11:35:09.144 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.140+0000 7f52be3d9740 1 Processor -- start 2024-05-10T11:35:09.144 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.140+0000 7f52be3d9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fbb1d68db0 0x55fbb1d7aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:09.144 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.140+0000 7f52be3d9740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fbb1d697a0 0x55fbb1d7cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:09.144 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.140+0000 7f52be3d9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbb1d72d20 0x55fbb1d7d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:09.144 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.140+0000 7f52be3d9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f52b41a7650 con 0x55fbb1d72d20 2024-05-10T11:35:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.140+0000 7f52be3d9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f52b41a8ac0 con 0x55fbb1d68db0 2024-05-10T11:35:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.140+0000 7f52be3d9740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f52b41a7d20 con 0x55fbb1d697a0 2024-05-10T11:35:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.140+0000 7f52be3d9740 10 client.0 _ll_register_callbacks cb 0x7f52ac004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:35:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.140+0000 7f52b9b81640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbb1d72d20 0x55fbb1d7d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.140+0000 7f52b9380640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fbb1d68db0 0x55fbb1d7aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.140+0000 7f52b8b7f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fbb1d697a0 0x55fbb1d7cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.140+0000 7f52b9b81640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbb1d72d20 0x55fbb1d7d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48172/0 (socket says 172.21.15.100:48172) 2024-05-10T11:35:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.140+0000 7f52b8b7f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fbb1d697a0 0x55fbb1d7cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:56536/0 (socket says 172.21.15.100:56536) 2024-05-10T11:35:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.140+0000 7f52b9b81640 1 -- 172.21.15.100:0/475641682 learned_addr learned my addr 172.21.15.100:0/475641682 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:35:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.141+0000 7f52b9380640 1 -- 172.21.15.100:0/475641682 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fbb1d697a0 msgr2=0x55fbb1d7cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.141+0000 7f52b9380640 1 --2- 172.21.15.100:0/475641682 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fbb1d697a0 0x55fbb1d7cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.141+0000 7f52b9380640 1 -- 172.21.15.100:0/475641682 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbb1d72d20 msgr2=0x55fbb1d7d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.141+0000 7f52b9380640 1 --2- 172.21.15.100:0/475641682 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbb1d72d20 0x55fbb1d7d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.141+0000 7f52b9380640 1 -- 172.21.15.100:0/475641682 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbb1d7dd10 con 0x55fbb1d68db0 2024-05-10T11:35:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.141+0000 7f52b8b7f640 1 --2- 172.21.15.100:0/475641682 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fbb1d697a0 0x55fbb1d7cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:35:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.141+0000 7f52b9380640 1 --2- 172.21.15.100:0/475641682 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fbb1d68db0 0x55fbb1d7aa50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x55fbb1d64b40 tx=0x7f52b0005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f17d761d01b8c53a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.141+0000 7f529cff9640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:35:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.141+0000 7f529cff9640 1 -- 172.21.15.100:0/475641682 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b0045060 con 0x55fbb1d68db0 2024-05-10T11:35:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.141+0000 7f529cff9640 1 -- 172.21.15.100:0/475641682 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f52b0004870 con 0x55fbb1d68db0 2024-05-10T11:35:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.141+0000 7f52be3d9740 1 -- 172.21.15.100:0/475641682 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52b0004af0 con 0x55fbb1d68db0 2024-05-10T11:35:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.141+0000 7f529cff9640 1 -- 172.21.15.100:0/475641682 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f52b0028040 con 0x55fbb1d68db0 2024-05-10T11:35:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.141+0000 7f52be3d9740 10 client.28067 fetch_fsmap learned FSMap version 40 2024-05-10T11:35:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.142+0000 7f52be3d9740 1 -- 172.21.15.100:0/475641682 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55fbb1df20a0 con 0x55fbb1d68db0 2024-05-10T11:35:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.142+0000 7f529cff9640 1 -- 172.21.15.100:0/475641682 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f52b00035f0 con 0x55fbb1d68db0 2024-05-10T11:35:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.142+0000 7f529cff9640 1 -- 172.21.15.100:0/475641682 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 40) ==== 8350+0+0 (secure 0 0 0) 0x7f52b000f6e0 con 0x55fbb1d68db0 2024-05-10T11:35:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.142+0000 7f52be3d9740 10 client.28067 fetch_fsmap finished waiting for FSMap version 40 2024-05-10T11:35:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.142+0000 7f52be3d9740 10 client.28067 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:35:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.142+0000 7f52be3d9740 20 client.28067 populate_metadata read hostname 'smithi100' 2024-05-10T11:35:09.148 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.142+0000 7f52be3d9740 1 --2- 172.21.15.100:0/475641682 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55fbb1df2b00 0x55fbb1df4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:09.148 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.142+0000 7f52be3d9740 4 client.28067 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:35:09.148 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.142+0000 7f52be3d9740 1 -- 172.21.15.100:0/475641682 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f52b0004af0 con 0x55fbb1df2b00 2024-05-10T11:35:09.148 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.142+0000 7f52b8b7f640 1 --2- 172.21.15.100:0/475641682 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55fbb1df2b00 0x55fbb1df4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:09.148 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.143+0000 7f529cff9640 10 client.28067 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:35:09.148 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.143+0000 7f52b8b7f640 1 --2- 172.21.15.100:0/475641682 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55fbb1df2b00 0x55fbb1df4f20 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4919b49d48dc615 server_cookie=5ac83cfeb519826e in_seq=0 out_seq=0 2024-05-10T11:35:09.281 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.280+0000 7f529cff9640 1 -- 172.21.15.100:0/475641682 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f52b0004af0 con 0x55fbb1df2b00 2024-05-10T11:35:09.281 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.280+0000 7f529cff9640 10 client.28067 handle_command_reply: tid=0 2024-05-10T11:35:09.281 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.281+0000 7f52be3d9740 1 client.28067 shutdown 2024-05-10T11:35:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.281+0000 7f52be3d9740 1 -- 172.21.15.100:0/475641682 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fbb1d68db0 msgr2=0x55fbb1d7aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.281+0000 7f52be3d9740 1 --2- 172.21.15.100:0/475641682 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fbb1d68db0 0x55fbb1d7aa50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x55fbb1d64b40 tx=0x7f52b0005dc0 comp rx=0 tx=0).stop 2024-05-10T11:35:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.281+0000 7f52be3d9740 1 -- 172.21.15.100:0/475641682 shutdown_connections 2024-05-10T11:35:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.281+0000 7f52be3d9740 1 --2- 172.21.15.100:0/475641682 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55fbb1df2b00 0x55fbb1df4f20 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.281+0000 7f52be3d9740 1 --2- 172.21.15.100:0/475641682 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbb1d72d20 0x55fbb1d7d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.281+0000 7f52be3d9740 1 --2- 172.21.15.100:0/475641682 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fbb1d697a0 0x55fbb1d7cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.281+0000 7f529cff9640 0 client.28067 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:35:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.281+0000 7f52be3d9740 1 --2- 172.21.15.100:0/475641682 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fbb1d68db0 0x55fbb1d7aa50 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.281+0000 7f52be3d9740 1 -- 172.21.15.100:0/475641682 >> 172.21.15.100:0/475641682 conn(0x55fbb1d64420 msgr2=0x55fbb1d67100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.281+0000 7f52be3d9740 1 -- 172.21.15.100:0/475641682 shutdown_connections 2024-05-10T11:35:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.281+0000 7f52be3d9740 1 -- 172.21.15.100:0/475641682 wait complete. 2024-05-10T11:35:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.281+0000 7f52be3d9740 20 client.28067 trim_cache size 0 max 16384 2024-05-10T11:35:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:35:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:35:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:35:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:35:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.282+0000 7f52bb5e4640 1 -- 172.21.15.100:0/3985802804 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f52a00b2920 msgr2=0x7f52a00b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.282+0000 7f52bb5e4640 1 --2- 172.21.15.100:0/3985802804 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f52a00b2920 0x7f52a00b4de0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f52b4199fd0 tx=0x7f52a4009560 comp rx=0 tx=0).stop 2024-05-10T11:35:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.282+0000 7f52bb5e4640 1 -- 172.21.15.100:0/3985802804 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52b41045c0 msgr2=0x7f52b41a0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.282+0000 7f52bb5e4640 1 --2- 172.21.15.100:0/3985802804 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52b41045c0 0x7f52b41a0070 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f52ac00ec50 tx=0x7f52ac00c6a0 comp rx=0 tx=0).stop 2024-05-10T11:35:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.282+0000 7f52bb5e4640 1 -- 172.21.15.100:0/3985802804 shutdown_connections 2024-05-10T11:35:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.282+0000 7f52bb5e4640 1 --2- 172.21.15.100:0/3985802804 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f52a00b2920 0x7f52a00b4de0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.282+0000 7f52bb5e4640 1 --2- 172.21.15.100:0/3985802804 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52b41045c0 0x7f52b41a0070 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.282+0000 7f52bb5e4640 1 --2- 172.21.15.100:0/3985802804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52b4103c00 0x7f52b4199000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.282+0000 7f52bb5e4640 1 --2- 172.21.15.100:0/3985802804 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f52b4102a10 0x7f52b4198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.282+0000 7f52bb5e4640 1 -- 172.21.15.100:0/3985802804 >> 172.21.15.100:0/3985802804 conn(0x7f52b40fdfa0 msgr2=0x7f52b4107020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.282+0000 7f52bb5e4640 1 -- 172.21.15.100:0/3985802804 shutdown_connections 2024-05-10T11:35:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:09.282+0000 7f52bb5e4640 1 -- 172.21.15.100:0/3985802804 wait complete. 2024-05-10T11:35:09.917 INFO:tasks.workunit.client.0.smithi100.stdout: 10 31303 9.48 MB/sec execute 63 sec latency 162.719 ms 2024-05-10T11:35:10.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:35:09 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:35:10.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:35:09 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:35:10.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:35:09 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:35:10.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:35:09 smithi178 ceph-mon[33359]: pgmap v924: 129 pgs: 129 active+clean; 437 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 871 KiB/s rd, 4.9 MiB/s wr, 292 op/s 2024-05-10T11:35:10.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:35:09 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:35:10.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:35:09 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:35:10.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:35:09 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:35:10.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:35:09 smithi188 ceph-mon[34075]: pgmap v924: 129 pgs: 129 active+clean; 437 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 871 KiB/s rd, 4.9 MiB/s wr, 292 op/s 2024-05-10T11:35:10.030 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:35:09 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:35:10.031 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:35:09 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:35:10.031 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:35:09 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:35:10.031 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:35:09 smithi100 ceph-mon[30296]: pgmap v924: 129 pgs: 129 active+clean; 437 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 871 KiB/s rd, 4.9 MiB/s wr, 292 op/s 2024-05-10T11:35:10.917 INFO:tasks.workunit.client.0.smithi100.stdout: 10 31428 9.39 MB/sec execute 64 sec latency 153.793 ms 2024-05-10T11:35:11.917 INFO:tasks.workunit.client.0.smithi100.stdout: 10 31561 9.32 MB/sec execute 65 sec latency 158.095 ms 2024-05-10T11:35:12.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:35:11 smithi178 ceph-mon[33359]: pgmap v925: 129 pgs: 129 active+clean; 440 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1019 KiB/s rd, 4.9 MiB/s wr, 335 op/s 2024-05-10T11:35:12.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:35:11 smithi188 ceph-mon[34075]: pgmap v925: 129 pgs: 129 active+clean; 440 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1019 KiB/s rd, 4.9 MiB/s wr, 335 op/s 2024-05-10T11:35:12.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:35:11 smithi100 ceph-mon[30296]: pgmap v925: 129 pgs: 129 active+clean; 440 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1019 KiB/s rd, 4.9 MiB/s wr, 335 op/s 2024-05-10T11:35:12.917 INFO:tasks.workunit.client.0.smithi100.stdout: 10 31745 9.36 MB/sec execute 66 sec latency 94.360 ms 2024-05-10T11:35:13.376 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T11:35:13.729 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:35:13.917 INFO:tasks.workunit.client.0.smithi100.stdout: 10 31866 9.27 MB/sec execute 67 sec latency 165.419 ms 2024-05-10T11:35:14.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:35:13 smithi178 ceph-mon[33359]: pgmap v926: 129 pgs: 129 active+clean; 439 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 952 KiB/s rd, 4.4 MiB/s wr, 340 op/s 2024-05-10T11:35:14.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:35:13 smithi188 ceph-mon[34075]: pgmap v926: 129 pgs: 129 active+clean; 439 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 952 KiB/s rd, 4.4 MiB/s wr, 340 op/s 2024-05-10T11:35:14.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:35:13 smithi100 ceph-mon[30296]: pgmap v926: 129 pgs: 129 active+clean; 439 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 952 KiB/s rd, 4.4 MiB/s wr, 340 op/s 2024-05-10T11:35:14.917 INFO:tasks.workunit.client.0.smithi100.stdout: 10 32017 9.22 MB/sec execute 68 sec latency 98.062 ms 2024-05-10T11:35:15.917 INFO:tasks.workunit.client.0.smithi100.stdout: 10 32170 9.19 MB/sec execute 69 sec latency 122.411 ms 2024-05-10T11:35:16.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:35:15 smithi178 ceph-mon[33359]: pgmap v927: 129 pgs: 129 active+clean; 430 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 3.9 MiB/s wr, 320 op/s 2024-05-10T11:35:16.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:35:15 smithi188 ceph-mon[34075]: pgmap v927: 129 pgs: 129 active+clean; 430 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 3.9 MiB/s wr, 320 op/s 2024-05-10T11:35:16.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:35:15 smithi100 ceph-mon[30296]: pgmap v927: 129 pgs: 129 active+clean; 430 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 3.9 MiB/s wr, 320 op/s 2024-05-10T11:35:16.917 INFO:tasks.workunit.client.0.smithi100.stdout: 10 32325 9.14 MB/sec execute 70 sec latency 110.861 ms 2024-05-10T11:35:17.917 INFO:tasks.workunit.client.0.smithi100.stdout: 10 32491 9.15 MB/sec execute 71 sec latency 111.070 ms 2024-05-10T11:35:18.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:35:17 smithi178 ceph-mon[33359]: pgmap v928: 129 pgs: 129 active+clean; 427 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 4.0 MiB/s wr, 385 op/s 2024-05-10T11:35:18.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:35:17 smithi188 ceph-mon[34075]: pgmap v928: 129 pgs: 129 active+clean; 427 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 4.0 MiB/s wr, 385 op/s 2024-05-10T11:35:18.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:35:17 smithi100 ceph-mon[30296]: pgmap v928: 129 pgs: 129 active+clean; 427 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 4.0 MiB/s wr, 385 op/s 2024-05-10T11:35:18.917 INFO:tasks.workunit.client.0.smithi100.stdout: 10 32654 9.13 MB/sec execute 72 sec latency 114.806 ms 2024-05-10T11:35:19.917 INFO:tasks.workunit.client.0.smithi100.stdout: 10 32811 9.08 MB/sec execute 73 sec latency 117.449 ms 2024-05-10T11:35:19.958 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:35:19 smithi100 ceph-mon[30296]: pgmap v929: 129 pgs: 129 active+clean; 427 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 3.1 MiB/s wr, 310 op/s 2024-05-10T11:35:20.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:35:19 smithi178 ceph-mon[33359]: pgmap v929: 129 pgs: 129 active+clean; 427 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 3.1 MiB/s wr, 310 op/s 2024-05-10T11:35:20.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:35:19 smithi188 ceph-mon[34075]: pgmap v929: 129 pgs: 129 active+clean; 427 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 3.1 MiB/s wr, 310 op/s 2024-05-10T11:35:20.917 INFO:tasks.workunit.client.0.smithi100.stdout: 10 32967 9.04 MB/sec execute 74 sec latency 105.738 ms 2024-05-10T11:35:21.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:35:20 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:35:21.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:35:20 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:35:21.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:35:20 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:35:21.031 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:35:20 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:35:21.031 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:35:20 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:35:21.031 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:35:20 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:35:21.031 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:35:20 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:35:21.032 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:35:20 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:35:21.032 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:35:20 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:35:21.917 INFO:tasks.workunit.client.0.smithi100.stdout: 10 33126 9.12 MB/sec execute 75 sec latency 99.241 ms 2024-05-10T11:35:22.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:35:21 smithi178 ceph-mon[33359]: pgmap v930: 129 pgs: 129 active+clean; 429 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 3.2 MiB/s wr, 339 op/s 2024-05-10T11:35:22.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:35:21 smithi188 ceph-mon[34075]: pgmap v930: 129 pgs: 129 active+clean; 429 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 3.2 MiB/s wr, 339 op/s 2024-05-10T11:35:22.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:35:21 smithi100 ceph-mon[30296]: pgmap v930: 129 pgs: 129 active+clean; 429 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 3.2 MiB/s wr, 339 op/s 2024-05-10T11:35:22.444 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.444+0000 7f62584e8640 1 -- 172.21.15.100:0/251529950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6250103570 msgr2=0x7f6250109e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:22.444 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.444+0000 7f62584e8640 1 --2- 172.21.15.100:0/251529950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6250103570 0x7f6250109e00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6248009be0 tx=0x7f624802a6d0 comp rx=0 tx=0).stop 2024-05-10T11:35:22.444 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.444+0000 7f62584e8640 1 -- 172.21.15.100:0/251529950 shutdown_connections 2024-05-10T11:35:22.444 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.444+0000 7f62584e8640 1 --2- 172.21.15.100:0/251529950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6250103570 0x7f6250109e00 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:22.444 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.444+0000 7f62584e8640 1 --2- 172.21.15.100:0/251529950 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6250102bb0 0x7f6250103030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:22.444 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.444+0000 7f62584e8640 1 --2- 172.21.15.100:0/251529950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62501019c0 0x7f6250101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:22.445 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.444+0000 7f62584e8640 1 -- 172.21.15.100:0/251529950 >> 172.21.15.100:0/251529950 conn(0x7f62500fcf30 msgr2=0x7f62500ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:22.445 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.444+0000 7f62584e8640 1 -- 172.21.15.100:0/251529950 shutdown_connections 2024-05-10T11:35:22.445 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.444+0000 7f62584e8640 1 -- 172.21.15.100:0/251529950 wait complete. 2024-05-10T11:35:22.447 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.445+0000 7f62584e8640 1 Processor -- start 2024-05-10T11:35:22.447 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.445+0000 7f62584e8640 1 -- start start 2024-05-10T11:35:22.447 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.445+0000 7f62584e8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62501019c0 0x7f625019eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:22.447 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.445+0000 7f62584e8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6250102bb0 0x7f625019f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:22.447 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.445+0000 7f62584e8640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6250103570 0x7f62501a64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:22.447 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.445+0000 7f62584e8640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f625010e410 con 0x7f6250102bb0 2024-05-10T11:35:22.448 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.445+0000 7f62584e8640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f625010e290 con 0x7f6250103570 2024-05-10T11:35:22.448 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.445+0000 7f62584e8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f625010e590 con 0x7f62501019c0 2024-05-10T11:35:22.448 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.445+0000 7f6255a83640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6250102bb0 0x7f625019f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:22.448 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.445+0000 7f6255a83640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6250102bb0 0x7f625019f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55082/0 (socket says 172.21.15.100:55082) 2024-05-10T11:35:22.448 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.445+0000 7f6255a83640 1 -- 172.21.15.100:0/2500550350 learned_addr learned my addr 172.21.15.100:0/2500550350 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:35:22.448 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.445+0000 7f6256284640 1 --2- 172.21.15.100:0/2500550350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62501019c0 0x7f625019eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:22.448 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.445+0000 7f6256a85640 1 --2- 172.21.15.100:0/2500550350 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6250103570 0x7f62501a64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:22.448 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.446+0000 7f6255a83640 1 -- 172.21.15.100:0/2500550350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62501019c0 msgr2=0x7f625019eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:22.448 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.446+0000 7f6255a83640 1 --2- 172.21.15.100:0/2500550350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62501019c0 0x7f625019eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:22.448 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.446+0000 7f6255a83640 1 -- 172.21.15.100:0/2500550350 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6250103570 msgr2=0x7f62501a64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:22.448 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.446+0000 7f6255a83640 1 --2- 172.21.15.100:0/2500550350 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6250103570 0x7f62501a64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:22.448 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.446+0000 7f6255a83640 1 -- 172.21.15.100:0/2500550350 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62501a6b10 con 0x7f6250102bb0 2024-05-10T11:35:22.448 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.446+0000 7f6256a85640 1 --2- 172.21.15.100:0/2500550350 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6250103570 0x7f62501a64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:35:22.449 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.446+0000 7f6256284640 1 --2- 172.21.15.100:0/2500550350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62501019c0 0x7f625019eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:35:22.449 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.446+0000 7f6255a83640 1 --2- 172.21.15.100:0/2500550350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6250102bb0 0x7f625019f430 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f624000ea60 tx=0x7f624000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:22.449 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.446+0000 7f62477fe640 1 -- 172.21.15.100:0/2500550350 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f624000cd60 con 0x7f6250102bb0 2024-05-10T11:35:22.449 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.446+0000 7f62477fe640 1 -- 172.21.15.100:0/2500550350 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f62400035f0 con 0x7f6250102bb0 2024-05-10T11:35:22.449 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.446+0000 7f62584e8640 1 -- 172.21.15.100:0/2500550350 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62501a6e00 con 0x7f6250102bb0 2024-05-10T11:35:22.449 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.446+0000 7f62477fe640 1 -- 172.21.15.100:0/2500550350 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6240010690 con 0x7f6250102bb0 2024-05-10T11:35:22.449 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.446+0000 7f62584e8640 1 -- 172.21.15.100:0/2500550350 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62501a72c0 con 0x7f6250102bb0 2024-05-10T11:35:22.452 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.450+0000 7f625b206740 1 Processor -- start 2024-05-10T11:35:22.452 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.450+0000 7f625b206740 1 -- start start 2024-05-10T11:35:22.452 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.451+0000 7f625b206740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb5ea40db0 0x55fb5ea411d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:22.452 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.451+0000 7f625b206740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fb5ea417a0 0x55fb5ea462a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:22.452 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.451+0000 7f625b206740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fb5ea46910 0x55fb5ea48da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:22.452 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.451+0000 7f625b206740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55fb5e9a55d0 con 0x55fb5ea40db0 2024-05-10T11:35:22.452 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.451+0000 7f625b206740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55fb5e95e890 con 0x55fb5ea417a0 2024-05-10T11:35:22.452 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.451+0000 7f625b206740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55fb5e9a1590 con 0x55fb5ea46910 2024-05-10T11:35:22.452 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.451+0000 7f6256a85640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb5ea40db0 0x55fb5ea411d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:22.452 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.451+0000 7f6256a85640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb5ea40db0 0x55fb5ea411d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55084/0 (socket says 172.21.15.100:55084) 2024-05-10T11:35:22.453 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.451+0000 7f6256284640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fb5ea417a0 0x55fb5ea462a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:22.453 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.451+0000 7f6256a85640 1 -- 172.21.15.100:0/47915279 learned_addr learned my addr 172.21.15.100:0/47915279 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:35:22.453 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.451+0000 7f6256284640 1 -- 172.21.15.100:0/47915279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fb5ea46910 msgr2=0x55fb5ea48da0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T11:35:22.453 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.451+0000 7f6256284640 1 --2- 172.21.15.100:0/47915279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fb5ea46910 0x55fb5ea48da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:22.453 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.451+0000 7f6256284640 1 -- 172.21.15.100:0/47915279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb5ea40db0 msgr2=0x55fb5ea411d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:22.453 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.451+0000 7f6256284640 1 --2- 172.21.15.100:0/47915279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb5ea40db0 0x55fb5ea411d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:22.453 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.451+0000 7f6256284640 1 -- 172.21.15.100:0/47915279 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb5e6b7210 con 0x55fb5ea417a0 2024-05-10T11:35:22.453 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.451+0000 7f6256a85640 1 --2- 172.21.15.100:0/47915279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb5ea40db0 0x55fb5ea411d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:35:22.453 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.452+0000 7f6256284640 1 --2- 172.21.15.100:0/47915279 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fb5ea417a0 0x55fb5ea462a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6250102a10 tx=0x7f624c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbd280aa22b390b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:22.453 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.452+0000 7f62477fe640 1 -- 172.21.15.100:0/2500550350 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f6240004000 con 0x7f6250102bb0 2024-05-10T11:35:22.453 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.452+0000 7f6244bfe640 1 -- 172.21.15.100:0/47915279 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f624c00f4e0 con 0x55fb5ea417a0 2024-05-10T11:35:22.453 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.452+0000 7f6244bfe640 1 -- 172.21.15.100:0/47915279 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f624c00fb10 con 0x55fb5ea417a0 2024-05-10T11:35:22.453 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.452+0000 7f6244bfe640 1 -- 172.21.15.100:0/47915279 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f624c019b00 con 0x55fb5ea417a0 2024-05-10T11:35:22.458 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.453+0000 7f625b206740 1 -- 172.21.15.100:0/47915279 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fb5ea417a0 msgr2=0x55fb5ea462a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:22.458 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.453+0000 7f625b206740 1 --2- 172.21.15.100:0/47915279 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fb5ea417a0 0x55fb5ea462a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6250102a10 tx=0x7f624c0118d0 comp rx=0 tx=0).stop 2024-05-10T11:35:22.458 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.453+0000 7f625b206740 1 -- 172.21.15.100:0/47915279 shutdown_connections 2024-05-10T11:35:22.458 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.453+0000 7f625b206740 1 --2- 172.21.15.100:0/47915279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fb5ea46910 0x55fb5ea48da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:22.458 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.453+0000 7f625b206740 1 --2- 172.21.15.100:0/47915279 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fb5ea417a0 0x55fb5ea462a0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:22.458 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.453+0000 7f625b206740 1 --2- 172.21.15.100:0/47915279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb5ea40db0 0x55fb5ea411d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:22.461 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.453+0000 7f625b206740 1 -- 172.21.15.100:0/47915279 >> 172.21.15.100:0/47915279 conn(0x55fb5ea3c420 msgr2=0x55fb5ea3c820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:22.462 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.453+0000 7f62477fe640 1 --2- 172.21.15.100:0/2500550350 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f623c0b28f0 0x7f623c0b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:22.462 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.453+0000 7f625b206740 1 -- 172.21.15.100:0/47915279 shutdown_connections 2024-05-10T11:35:22.462 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.453+0000 7f625b206740 1 -- 172.21.15.100:0/47915279 wait complete. 2024-05-10T11:35:22.462 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.453+0000 7f6256284640 1 --2- 172.21.15.100:0/2500550350 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f623c0b28f0 0x7f623c0b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:22.462 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.453+0000 7f62477fe640 1 -- 172.21.15.100:0/2500550350 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f62400e27b0 con 0x7f6250102bb0 2024-05-10T11:35:22.462 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.453+0000 7f625b206740 1 -- start start 2024-05-10T11:35:22.462 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.453+0000 7f625b206740 1 Processor -- start 2024-05-10T11:35:22.462 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.454+0000 7f6256284640 1 --2- 172.21.15.100:0/2500550350 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f623c0b28f0 0x7f623c0b4db0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f624c009530 tx=0x7f624c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:22.462 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.454+0000 7f625b206740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb5ea40db0 0x55fb5ea4b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:22.462 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.454+0000 7f625b206740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fb5ea417a0 0x55fb5ea4b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:22.462 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.454+0000 7f625b206740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fb5ea46910 0x55fb5ea52580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:22.463 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.454+0000 7f6256284640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb5ea40db0 0x55fb5ea4b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:22.463 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.454+0000 7f625b206740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55fb5ea498c0 con 0x55fb5ea40db0 2024-05-10T11:35:22.463 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.454+0000 7f625b206740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55fb5e962370 con 0x55fb5ea417a0 2024-05-10T11:35:22.463 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.454+0000 7f625b206740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55fb5ea49a40 con 0x55fb5ea46910 2024-05-10T11:35:22.463 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.454+0000 7f6255a83640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fb5ea417a0 0x55fb5ea4b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:22.463 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.454+0000 7f6255a83640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fb5ea417a0 0x55fb5ea4b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:33694/0 (socket says 172.21.15.100:33694) 2024-05-10T11:35:22.463 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.454+0000 7f6255a83640 1 -- 172.21.15.100:0/643783432 learned_addr learned my addr 172.21.15.100:0/643783432 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:35:22.463 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.454+0000 7f6256a85640 1 --2- 172.21.15.100:0/643783432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fb5ea46910 0x55fb5ea52580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:22.463 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.454+0000 7f625b206740 10 client.0 _ll_register_callbacks cb 0x55fb5e6b6a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:35:22.463 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.454+0000 7f6255a83640 1 -- 172.21.15.100:0/643783432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fb5ea46910 msgr2=0x55fb5ea52580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:22.463 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.454+0000 7f6255a83640 1 --2- 172.21.15.100:0/643783432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fb5ea46910 0x55fb5ea52580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:22.463 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.454+0000 7f6255a83640 1 -- 172.21.15.100:0/643783432 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb5ea40db0 msgr2=0x55fb5ea4b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:22.463 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.454+0000 7f6255a83640 1 --2- 172.21.15.100:0/643783432 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb5ea40db0 0x55fb5ea4b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:22.464 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.454+0000 7f6255a83640 1 -- 172.21.15.100:0/643783432 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb5e783620 con 0x55fb5ea417a0 2024-05-10T11:35:22.464 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.454+0000 7f6256a85640 1 --2- 172.21.15.100:0/643783432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fb5ea46910 0x55fb5ea52580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:35:22.464 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.454+0000 7f6256284640 1 --2- 172.21.15.100:0/643783432 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb5ea40db0 0x55fb5ea4b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:35:22.464 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.454+0000 7f6255a83640 1 --2- 172.21.15.100:0/643783432 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fb5ea417a0 0x55fb5ea4b540 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f62400a6ce0 tx=0x7f62400a94b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76aa341ae5dea2c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:22.464 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.454+0000 7f6239ffb640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:35:22.464 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.455+0000 7f6239ffb640 1 -- 172.21.15.100:0/643783432 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62400aa170 con 0x55fb5ea417a0 2024-05-10T11:35:22.464 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.455+0000 7f6239ffb640 1 -- 172.21.15.100:0/643783432 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f62400a5d00 con 0x55fb5ea417a0 2024-05-10T11:35:22.464 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.455+0000 7f625b206740 1 -- 172.21.15.100:0/643783432 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f624c00fb10 con 0x55fb5ea417a0 2024-05-10T11:35:22.464 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.455+0000 7f6239ffb640 1 -- 172.21.15.100:0/643783432 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f62400a9a70 con 0x55fb5ea417a0 2024-05-10T11:35:22.464 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.455+0000 7f625b206740 10 client.28079 fetch_fsmap learned FSMap version 40 2024-05-10T11:35:22.464 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.455+0000 7f625b206740 1 -- 172.21.15.100:0/643783432 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55fb5ea555b0 con 0x55fb5ea417a0 2024-05-10T11:35:22.464 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.455+0000 7f6239ffb640 1 -- 172.21.15.100:0/643783432 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f624c00fb10 con 0x55fb5ea417a0 2024-05-10T11:35:22.464 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.455+0000 7f6239ffb640 1 -- 172.21.15.100:0/643783432 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 40) ==== 8350+0+0 (secure 0 0 0) 0x7f62400a8cd0 con 0x55fb5ea417a0 2024-05-10T11:35:22.464 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.455+0000 7f625b206740 10 client.28079 fetch_fsmap finished waiting for FSMap version 40 2024-05-10T11:35:22.465 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.455+0000 7f625b206740 10 client.28079 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:35:22.465 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.455+0000 7f625b206740 20 client.28079 populate_metadata read hostname 'smithi100' 2024-05-10T11:35:22.465 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.455+0000 7f625b206740 1 --2- 172.21.15.100:0/643783432 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55fb5ea56c20 0x55fb5ea59090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:22.465 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.455+0000 7f625b206740 4 client.28079 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:35:22.465 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.455+0000 7f625b206740 1 -- 172.21.15.100:0/643783432 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb5e9b31e0 con 0x55fb5ea56c20 2024-05-10T11:35:22.465 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.456+0000 7f6256284640 1 --2- 172.21.15.100:0/643783432 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55fb5ea56c20 0x55fb5ea59090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:22.465 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.456+0000 7f6256284640 1 --2- 172.21.15.100:0/643783432 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55fb5ea56c20 0x55fb5ea59090 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb0003046f8feffd server_cookie=fd07d470a5fd1159 in_seq=0 out_seq=0 2024-05-10T11:35:22.465 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.456+0000 7f6239ffb640 10 client.28079 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:35:22.465 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.457+0000 7f6239ffb640 1 -- 172.21.15.100:0/643783432 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fb5e9b31e0 con 0x55fb5ea56c20 2024-05-10T11:35:22.465 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.457+0000 7f6239ffb640 10 client.28079 handle_command_reply: tid=0 2024-05-10T11:35:22.465 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.457+0000 7f625b206740 1 client.28079 shutdown 2024-05-10T11:35:22.465 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.457+0000 7f625b206740 1 -- 172.21.15.100:0/643783432 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fb5ea417a0 msgr2=0x55fb5ea4b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:22.465 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.457+0000 7f625b206740 1 --2- 172.21.15.100:0/643783432 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fb5ea417a0 0x55fb5ea4b540 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f62400a6ce0 tx=0x7f62400a94b0 comp rx=0 tx=0).stop 2024-05-10T11:35:22.465 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.457+0000 7f625b206740 1 -- 172.21.15.100:0/643783432 shutdown_connections 2024-05-10T11:35:22.465 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.457+0000 7f625b206740 1 --2- 172.21.15.100:0/643783432 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55fb5ea56c20 0x55fb5ea59090 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:22.466 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.457+0000 7f625b206740 1 --2- 172.21.15.100:0/643783432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fb5ea46910 0x55fb5ea52580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:22.466 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.457+0000 7f6239ffb640 0 client.28079 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:35:22.466 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.457+0000 7f625b206740 1 --2- 172.21.15.100:0/643783432 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fb5ea417a0 0x55fb5ea4b540 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:22.466 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.457+0000 7f625b206740 1 --2- 172.21.15.100:0/643783432 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb5ea40db0 0x55fb5ea4b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:22.466 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.457+0000 7f625b206740 1 -- 172.21.15.100:0/643783432 >> 172.21.15.100:0/643783432 conn(0x55fb5ea3c420 msgr2=0x55fb5ea3b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:22.466 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.457+0000 7f625b206740 1 -- 172.21.15.100:0/643783432 shutdown_connections 2024-05-10T11:35:22.466 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.457+0000 7f625b206740 1 -- 172.21.15.100:0/643783432 wait complete. 2024-05-10T11:35:22.466 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.457+0000 7f625b206740 20 client.28079 trim_cache size 0 max 16384 2024-05-10T11:35:22.482 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.481+0000 7f625b206740 1 Processor -- start 2024-05-10T11:35:22.482 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.481+0000 7f625b206740 1 -- start start 2024-05-10T11:35:22.482 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.481+0000 7f625b206740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fb5ea40db0 0x55fb5ea52c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:22.482 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.481+0000 7f625b206740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb5ea417a0 0x55fb5ea373e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:22.482 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.481+0000 7f625b206740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fb5ea46910 0x55fb5ea42a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:22.482 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.481+0000 7f625b206740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f62501b40b0 con 0x55fb5ea417a0 2024-05-10T11:35:22.482 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.481+0000 7f625b206740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f62501b1950 con 0x55fb5ea40db0 2024-05-10T11:35:22.482 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.481+0000 7f625b206740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f62501af7a0 con 0x55fb5ea46910 2024-05-10T11:35:22.482 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.482+0000 7f6256284640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fb5ea40db0 0x55fb5ea52c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:22.482 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.482+0000 7f6256a85640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fb5ea46910 0x55fb5ea42a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:22.482 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.482+0000 7f6256a85640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fb5ea46910 0x55fb5ea42a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:46870/0 (socket says 172.21.15.100:46870) 2024-05-10T11:35:22.482 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.482+0000 7f6256a85640 1 -- 172.21.15.100:0/3730855727 learned_addr learned my addr 172.21.15.100:0/3730855727 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:35:22.482 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.482+0000 7f6255a83640 1 --2- 172.21.15.100:0/3730855727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb5ea417a0 0x55fb5ea373e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:22.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.482+0000 7f6256284640 1 -- 172.21.15.100:0/3730855727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fb5ea46910 msgr2=0x55fb5ea42a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:22.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.482+0000 7f6256284640 1 --2- 172.21.15.100:0/3730855727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fb5ea46910 0x55fb5ea42a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:22.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.482+0000 7f6256284640 1 -- 172.21.15.100:0/3730855727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb5ea417a0 msgr2=0x55fb5ea373e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:22.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.482+0000 7f6256284640 1 --2- 172.21.15.100:0/3730855727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb5ea417a0 0x55fb5ea373e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:22.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.482+0000 7f6256284640 1 -- 172.21.15.100:0/3730855727 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb5ea45ca0 con 0x55fb5ea40db0 2024-05-10T11:35:22.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.482+0000 7f6256284640 1 --2- 172.21.15.100:0/3730855727 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fb5ea40db0 0x55fb5ea52c10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x55fb5ea3d2f0 tx=0x7f624c020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78497a225ebeb91a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:22.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.482+0000 7f623bfff640 1 -- 172.21.15.100:0/3730855727 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f624c02a040 con 0x55fb5ea40db0 2024-05-10T11:35:22.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.483+0000 7f623bfff640 1 -- 172.21.15.100:0/3730855727 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f624c00fb50 con 0x55fb5ea40db0 2024-05-10T11:35:22.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.483+0000 7f623bfff640 1 -- 172.21.15.100:0/3730855727 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f624c004bc0 con 0x55fb5ea40db0 2024-05-10T11:35:22.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.483+0000 7f625b206740 1 -- 172.21.15.100:0/3730855727 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fb5ea40db0 msgr2=0x55fb5ea52c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:22.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.484+0000 7f625b206740 1 --2- 172.21.15.100:0/3730855727 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fb5ea40db0 0x55fb5ea52c10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x55fb5ea3d2f0 tx=0x7f624c020000 comp rx=0 tx=0).stop 2024-05-10T11:35:22.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.484+0000 7f625b206740 1 -- 172.21.15.100:0/3730855727 shutdown_connections 2024-05-10T11:35:22.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.484+0000 7f625b206740 1 --2- 172.21.15.100:0/3730855727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fb5ea46910 0x55fb5ea42a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:22.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.484+0000 7f625b206740 1 --2- 172.21.15.100:0/3730855727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb5ea417a0 0x55fb5ea373e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:22.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.484+0000 7f625b206740 1 --2- 172.21.15.100:0/3730855727 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fb5ea40db0 0x55fb5ea52c10 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:22.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.484+0000 7f625b206740 1 -- 172.21.15.100:0/3730855727 >> 172.21.15.100:0/3730855727 conn(0x55fb5ea3c420 msgr2=0x55fb5ea46020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:22.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.484+0000 7f625b206740 1 -- 172.21.15.100:0/3730855727 shutdown_connections 2024-05-10T11:35:22.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.484+0000 7f625b206740 1 -- 172.21.15.100:0/3730855727 wait complete. 2024-05-10T11:35:22.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.484+0000 7f625b206740 1 -- start start 2024-05-10T11:35:22.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.484+0000 7f625b206740 1 Processor -- start 2024-05-10T11:35:22.488 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.485+0000 7f625b206740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fb5ea40db0 0x55fb5ea4f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:22.488 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.485+0000 7f625b206740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb5ea417a0 0x55fb5ea4f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:22.488 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.485+0000 7f625b206740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fb5ea46910 0x55fb5ea50620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:22.488 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.485+0000 7f625b206740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f62501afe10 con 0x55fb5ea417a0 2024-05-10T11:35:22.488 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.485+0000 7f625b206740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f62501b1280 con 0x55fb5ea40db0 2024-05-10T11:35:22.488 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.485+0000 7f625b206740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f62501b04e0 con 0x55fb5ea46910 2024-05-10T11:35:22.488 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.485+0000 7f625b206740 10 client.0 _ll_register_callbacks cb 0x7f62400a5af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:35:22.488 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.485+0000 7f6256a85640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fb5ea46910 0x55fb5ea50620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:22.488 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.485+0000 7f6256284640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fb5ea40db0 0x55fb5ea4f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:22.488 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.485+0000 7f6256a85640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fb5ea46910 0x55fb5ea50620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:46886/0 (socket says 172.21.15.100:46886) 2024-05-10T11:35:22.488 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.485+0000 7f6256a85640 1 -- 172.21.15.100:0/3426860047 learned_addr learned my addr 172.21.15.100:0/3426860047 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:35:22.488 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.485+0000 7f6255a83640 1 --2- 172.21.15.100:0/3426860047 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb5ea417a0 0x55fb5ea4f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:22.488 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.485+0000 7f6256284640 1 -- 172.21.15.100:0/3426860047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fb5ea46910 msgr2=0x55fb5ea50620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:22.488 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.485+0000 7f6256284640 1 --2- 172.21.15.100:0/3426860047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fb5ea46910 0x55fb5ea50620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:22.489 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.485+0000 7f6256284640 1 -- 172.21.15.100:0/3426860047 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb5ea417a0 msgr2=0x55fb5ea4f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:22.489 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.485+0000 7f6256284640 1 --2- 172.21.15.100:0/3426860047 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb5ea417a0 0x55fb5ea4f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:22.489 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.485+0000 7f6256284640 1 -- 172.21.15.100:0/3426860047 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb5ea50c10 con 0x55fb5ea40db0 2024-05-10T11:35:22.489 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.485+0000 7f6256284640 1 --2- 172.21.15.100:0/3426860047 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fb5ea40db0 0x55fb5ea4f190 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f624c011910 tx=0x7f624c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e9f72a851c4b574 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:22.489 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.485+0000 7f6239ffb640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:35:22.489 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.486+0000 7f6239ffb640 1 -- 172.21.15.100:0/3426860047 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f624c042070 con 0x55fb5ea40db0 2024-05-10T11:35:22.489 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.486+0000 7f625b206740 1 -- 172.21.15.100:0/3426860047 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f624c00fb50 con 0x55fb5ea40db0 2024-05-10T11:35:22.489 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.486+0000 7f6239ffb640 1 -- 172.21.15.100:0/3426860047 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f624c00fe00 con 0x55fb5ea40db0 2024-05-10T11:35:22.489 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.486+0000 7f6239ffb640 1 -- 172.21.15.100:0/3426860047 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f624c04e050 con 0x55fb5ea40db0 2024-05-10T11:35:22.489 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.486+0000 7f625b206740 10 client.28085 fetch_fsmap learned FSMap version 40 2024-05-10T11:35:22.489 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.486+0000 7f625b206740 1 -- 172.21.15.100:0/3426860047 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55fb5ea51f20 con 0x55fb5ea40db0 2024-05-10T11:35:22.489 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.486+0000 7f6239ffb640 1 -- 172.21.15.100:0/3426860047 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f624c00fb50 con 0x55fb5ea40db0 2024-05-10T11:35:22.489 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.486+0000 7f6239ffb640 1 -- 172.21.15.100:0/3426860047 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 40) ==== 8350+0+0 (secure 0 0 0) 0x7f624c0195c0 con 0x55fb5ea40db0 2024-05-10T11:35:22.489 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.486+0000 7f625b206740 10 client.28085 fetch_fsmap finished waiting for FSMap version 40 2024-05-10T11:35:22.490 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.486+0000 7f625b206740 10 client.28085 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:35:22.490 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.486+0000 7f625b206740 20 client.28085 populate_metadata read hostname 'smithi100' 2024-05-10T11:35:22.490 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.486+0000 7f625b206740 1 --2- 172.21.15.100:0/3426860047 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55fb5eac1b00 0x55fb5eac3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:22.490 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.486+0000 7f625b206740 4 client.28085 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:35:22.490 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.486+0000 7f625b206740 1 -- 172.21.15.100:0/3426860047 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fb5e9ad050 con 0x55fb5eac1b00 2024-05-10T11:35:22.490 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.487+0000 7f6255a83640 1 --2- 172.21.15.100:0/3426860047 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55fb5eac1b00 0x55fb5eac3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:22.490 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.487+0000 7f6255a83640 1 --2- 172.21.15.100:0/3426860047 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55fb5eac1b00 0x55fb5eac3f20 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=918b902afd200e44 server_cookie=8e492ca9930402fd in_seq=0 out_seq=0 2024-05-10T11:35:22.490 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.487+0000 7f6239ffb640 10 client.28085 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:35:22.598 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.597+0000 7f6239ffb640 1 -- 172.21.15.100:0/3426860047 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fb5e9ad050 con 0x55fb5eac1b00 2024-05-10T11:35:22.598 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.597+0000 7f6239ffb640 10 client.28085 handle_command_reply: tid=0 2024-05-10T11:35:22.598 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.597+0000 7f625b206740 1 client.28085 shutdown 2024-05-10T11:35:22.598 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.597+0000 7f625b206740 1 -- 172.21.15.100:0/3426860047 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fb5ea40db0 msgr2=0x55fb5ea4f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:22.598 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.597+0000 7f625b206740 1 --2- 172.21.15.100:0/3426860047 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fb5ea40db0 0x55fb5ea4f190 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f624c011910 tx=0x7f624c0046c0 comp rx=0 tx=0).stop 2024-05-10T11:35:22.598 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.597+0000 7f625b206740 1 -- 172.21.15.100:0/3426860047 shutdown_connections 2024-05-10T11:35:22.598 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.597+0000 7f625b206740 1 --2- 172.21.15.100:0/3426860047 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55fb5eac1b00 0x55fb5eac3f20 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:22.599 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.597+0000 7f6239ffb640 0 client.28085 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:35:22.599 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.598+0000 7f625b206740 1 --2- 172.21.15.100:0/3426860047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fb5ea46910 0x55fb5ea50620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:22.599 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.598+0000 7f625b206740 1 --2- 172.21.15.100:0/3426860047 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb5ea417a0 0x55fb5ea4f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:22.599 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.598+0000 7f625b206740 1 --2- 172.21.15.100:0/3426860047 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fb5ea40db0 0x55fb5ea4f190 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:22.599 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.598+0000 7f625b206740 1 -- 172.21.15.100:0/3426860047 >> 172.21.15.100:0/3426860047 conn(0x55fb5ea3c420 msgr2=0x55fb5ea42a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:22.599 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.598+0000 7f625b206740 1 -- 172.21.15.100:0/3426860047 shutdown_connections 2024-05-10T11:35:22.599 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.598+0000 7f625b206740 1 -- 172.21.15.100:0/3426860047 wait complete. 2024-05-10T11:35:22.599 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.598+0000 7f625b206740 20 client.28085 trim_cache size 0 max 16384 2024-05-10T11:35:22.600 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:35:22.600 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:35:22.600 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:35:22.600 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:35:22.601 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.600+0000 7f62584e8640 1 -- 172.21.15.100:0/2500550350 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f623c0b28f0 msgr2=0x7f623c0b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:22.601 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.600+0000 7f62584e8640 1 --2- 172.21.15.100:0/2500550350 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f623c0b28f0 0x7f623c0b4db0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f624c009530 tx=0x7f624c000bc0 comp rx=0 tx=0).stop 2024-05-10T11:35:22.601 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.600+0000 7f62584e8640 1 -- 172.21.15.100:0/2500550350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6250102bb0 msgr2=0x7f625019f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:22.601 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.600+0000 7f62584e8640 1 --2- 172.21.15.100:0/2500550350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6250102bb0 0x7f625019f430 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f624000ea60 tx=0x7f624000ef20 comp rx=0 tx=0).stop 2024-05-10T11:35:22.601 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.600+0000 7f62584e8640 1 -- 172.21.15.100:0/2500550350 shutdown_connections 2024-05-10T11:35:22.602 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.600+0000 7f62584e8640 1 --2- 172.21.15.100:0/2500550350 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f623c0b28f0 0x7f623c0b4db0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:22.602 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.600+0000 7f62584e8640 1 --2- 172.21.15.100:0/2500550350 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6250103570 0x7f62501a64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:22.602 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.601+0000 7f62584e8640 1 --2- 172.21.15.100:0/2500550350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6250102bb0 0x7f625019f430 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:22.602 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.601+0000 7f62584e8640 1 --2- 172.21.15.100:0/2500550350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62501019c0 0x7f625019eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:22.602 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.601+0000 7f62584e8640 1 -- 172.21.15.100:0/2500550350 >> 172.21.15.100:0/2500550350 conn(0x7f62500fcf30 msgr2=0x7f6250103dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:22.602 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.601+0000 7f62584e8640 1 -- 172.21.15.100:0/2500550350 shutdown_connections 2024-05-10T11:35:22.602 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:22.601+0000 7f62584e8640 1 -- 172.21.15.100:0/2500550350 wait complete. 2024-05-10T11:35:22.919 INFO:tasks.workunit.client.0.smithi100.stdout: 10 33291 9.15 MB/sec execute 76 sec latency 163.994 ms 2024-05-10T11:35:23.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.108+0000 7f51ad83e700 1 -- 172.21.15.100:0/3945544787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51a8108440 msgr2=0x7f51a8108820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:23.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.108+0000 7f51ad83e700 1 --2- 172.21.15.100:0/3945544787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51a8108440 0x7f51a8108820 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f5198009990 tx=0x7f5198009ca0 comp rx=0 tx=0).stop 2024-05-10T11:35:23.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.109+0000 7f51ad83e700 1 -- 172.21.15.100:0/3945544787 shutdown_connections 2024-05-10T11:35:23.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.109+0000 7f51ad83e700 1 --2- 172.21.15.100:0/3945544787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51a8102d90 0x7f51a810f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:23.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.109+0000 7f51ad83e700 1 --2- 172.21.15.100:0/3945544787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51a8108440 0x7f51a8108820 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:23.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.109+0000 7f51ad83e700 1 --2- 172.21.15.100:0/3945544787 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51a81023d0 0x7f51a8102850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:23.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.109+0000 7f51ad83e700 1 -- 172.21.15.100:0/3945544787 >> 172.21.15.100:0/3945544787 conn(0x7f51a80793e0 msgr2=0x7f51a80797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:23.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.111+0000 7f51ad83e700 1 -- 172.21.15.100:0/3945544787 shutdown_connections 2024-05-10T11:35:23.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.111+0000 7f51ad83e700 1 -- 172.21.15.100:0/3945544787 wait complete. 2024-05-10T11:35:23.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.112+0000 7f51ad83e700 1 Processor -- start 2024-05-10T11:35:23.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.112+0000 7f51ad83e700 1 -- start start 2024-05-10T11:35:23.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.112+0000 7f51ad83e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51a81023d0 0x7f51a819e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:23.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.112+0000 7f51ad83e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51a8102d90 0x7f51a819eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:23.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.112+0000 7f51ad83e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51a8108440 0x7f51a81987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:23.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.112+0000 7f51ad83e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f51a8112440 con 0x7f51a8102d90 2024-05-10T11:35:23.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.112+0000 7f51ad83e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f51a81122c0 con 0x7f51a8108440 2024-05-10T11:35:23.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.112+0000 7f51ad83e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f51a81125c0 con 0x7f51a81023d0 2024-05-10T11:35:23.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.113+0000 7f51a77fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51a8108440 0x7f51a81987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:23.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.113+0000 7f51a77fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51a8108440 0x7f51a81987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:33740/0 (socket says 172.21.15.100:33740) 2024-05-10T11:35:23.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.113+0000 7f51a77fe700 1 -- 172.21.15.100:0/2133841681 learned_addr learned my addr 172.21.15.100:0/2133841681 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:35:23.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.113+0000 7f51a67fc700 1 --2- 172.21.15.100:0/2133841681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51a8102d90 0x7f51a819eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:23.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.113+0000 7f51a6ffd700 1 --2- 172.21.15.100:0/2133841681 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51a81023d0 0x7f51a819e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:23.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.113+0000 7f51a77fe700 1 -- 172.21.15.100:0/2133841681 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51a81023d0 msgr2=0x7f51a819e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:23.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.113+0000 7f51a77fe700 1 --2- 172.21.15.100:0/2133841681 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51a81023d0 0x7f51a819e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:23.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.113+0000 7f51a77fe700 1 -- 172.21.15.100:0/2133841681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51a8102d90 msgr2=0x7f51a819eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:23.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.113+0000 7f51a77fe700 1 --2- 172.21.15.100:0/2133841681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51a8102d90 0x7f51a819eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:23.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.113+0000 7f51a77fe700 1 -- 172.21.15.100:0/2133841681 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51a8198fc0 con 0x7f51a8108440 2024-05-10T11:35:23.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.113+0000 7f51a6ffd700 1 --2- 172.21.15.100:0/2133841681 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51a81023d0 0x7f51a819e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:35:23.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.113+0000 7f51a77fe700 1 --2- 172.21.15.100:0/2133841681 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51a8108440 0x7f51a81987e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f51a000ef30 tx=0x7f51a000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:23.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.113+0000 7f51ac83c700 1 -- 172.21.15.100:0/2133841681 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a00092e0 con 0x7f51a8108440 2024-05-10T11:35:23.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.114+0000 7f51ad83e700 1 -- 172.21.15.100:0/2133841681 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51a8199250 con 0x7f51a8108440 2024-05-10T11:35:23.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.114+0000 7f51ac83c700 1 -- 172.21.15.100:0/2133841681 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f51a0004bb0 con 0x7f51a8108440 2024-05-10T11:35:23.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.114+0000 7f51ad83e700 1 -- 172.21.15.100:0/2133841681 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51a806bb70 con 0x7f51a8108440 2024-05-10T11:35:23.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.114+0000 7f51ac83c700 1 -- 172.21.15.100:0/2133841681 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a0004020 con 0x7f51a8108440 2024-05-10T11:35:23.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.117+0000 7f51ac83c700 1 -- 172.21.15.100:0/2133841681 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f51a001b030 con 0x7f51a8108440 2024-05-10T11:35:23.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.118+0000 7f51ac83c700 1 --2- 172.21.15.100:0/2133841681 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f51900bb110 0x7f51900bd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:23.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.118+0000 7f51ac83c700 1 -- 172.21.15.100:0/2133841681 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f51a00e1f30 con 0x7f51a8108440 2024-05-10T11:35:23.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.118+0000 7f51a6ffd700 1 --2- 172.21.15.100:0/2133841681 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f51900bb110 0x7f51900bd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:23.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.119+0000 7f51a6ffd700 1 --2- 172.21.15.100:0/2133841681 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f51900bb110 0x7f51900bd5d0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f519800b2e0 tx=0x7f519801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:23.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.124+0000 7f51b80d1b80 1 Processor -- start 2024-05-10T11:35:23.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.125+0000 7f51b80d1b80 1 -- start start 2024-05-10T11:35:23.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.125+0000 7f51b80d1b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecebdf4d0 0x555ecebdf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:23.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.125+0000 7f51b80d1b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ecebdfdf0 0x555ecebe36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:23.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.125+0000 7f51b80d1b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ecebe3c80 0x555ecebe4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:23.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.125+0000 7f51b80d1b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x555eceb31860 con 0x555ecebdf4d0 2024-05-10T11:35:23.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.125+0000 7f51b80d1b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x555ecead07c0 con 0x555ecebdfdf0 2024-05-10T11:35:23.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.125+0000 7f51b80d1b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x555eceb2d820 con 0x555ecebe3c80 2024-05-10T11:35:23.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.125+0000 7f51a67fc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ecebdfdf0 0x555ecebe36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:23.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.125+0000 7f51a67fc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ecebdfdf0 0x555ecebe36f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:33742/0 (socket says 172.21.15.100:33742) 2024-05-10T11:35:23.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.125+0000 7f51a67fc700 1 -- 172.21.15.100:0/3745059608 learned_addr learned my addr 172.21.15.100:0/3745059608 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:35:23.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.125+0000 7f51a77fe700 1 --2- 172.21.15.100:0/3745059608 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ecebe3c80 0x555ecebe4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:23.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.125+0000 7f51a6ffd700 1 --2- 172.21.15.100:0/3745059608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecebdf4d0 0x555ecebdf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:23.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.126+0000 7f51a77fe700 1 -- 172.21.15.100:0/3745059608 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ecebdfdf0 msgr2=0x555ecebe36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:23.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.126+0000 7f51a77fe700 1 --2- 172.21.15.100:0/3745059608 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ecebdfdf0 0x555ecebe36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:23.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.126+0000 7f51a77fe700 1 -- 172.21.15.100:0/3745059608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecebdf4d0 msgr2=0x555ecebdf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:23.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.126+0000 7f51a77fe700 1 --2- 172.21.15.100:0/3745059608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecebdf4d0 0x555ecebdf8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:23.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.126+0000 7f51a77fe700 1 -- 172.21.15.100:0/3745059608 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ecea7e8d0 con 0x555ecebe3c80 2024-05-10T11:35:23.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.126+0000 7f51a77fe700 1 --2- 172.21.15.100:0/3745059608 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ecebe3c80 0x555ecebe4100 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f51a0009510 tx=0x7f51a0009ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b9956fc11d7cf75 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:23.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.126+0000 7f518dffb700 1 -- 172.21.15.100:0/3745059608 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a00a9980 con 0x555ecebe3c80 2024-05-10T11:35:23.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.126+0000 7f518dffb700 1 -- 172.21.15.100:0/3745059608 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f51a00a9280 con 0x555ecebe3c80 2024-05-10T11:35:23.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.127+0000 7f518dffb700 1 -- 172.21.15.100:0/3745059608 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a00b1e20 con 0x555ecebe3c80 2024-05-10T11:35:23.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.127+0000 7f51b80d1b80 1 -- 172.21.15.100:0/3745059608 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ecebe3c80 msgr2=0x555ecebe4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:23.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.127+0000 7f51b80d1b80 1 --2- 172.21.15.100:0/3745059608 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ecebe3c80 0x555ecebe4100 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f51a0009510 tx=0x7f51a0009ed0 comp rx=0 tx=0).stop 2024-05-10T11:35:23.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.128+0000 7f51b80d1b80 1 -- 172.21.15.100:0/3745059608 shutdown_connections 2024-05-10T11:35:23.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.128+0000 7f51b80d1b80 1 --2- 172.21.15.100:0/3745059608 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ecebe3c80 0x555ecebe4100 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:23.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.128+0000 7f51b80d1b80 1 --2- 172.21.15.100:0/3745059608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecebdf4d0 0x555ecebdf8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:23.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.128+0000 7f51b80d1b80 1 --2- 172.21.15.100:0/3745059608 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ecebdfdf0 0x555ecebe36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:23.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.128+0000 7f51b80d1b80 1 -- 172.21.15.100:0/3745059608 >> 172.21.15.100:0/3745059608 conn(0x555eceb0b100 msgr2=0x555ecebd9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:23.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.128+0000 7f51b80d1b80 1 -- 172.21.15.100:0/3745059608 shutdown_connections 2024-05-10T11:35:23.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.128+0000 7f51b80d1b80 1 -- 172.21.15.100:0/3745059608 wait complete. 2024-05-10T11:35:23.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.129+0000 7f51b80d1b80 1 -- start start 2024-05-10T11:35:23.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.129+0000 7f51b80d1b80 1 Processor -- start 2024-05-10T11:35:23.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.129+0000 7f51b80d1b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ecebdf4d0 0x555ecebeaf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:23.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.129+0000 7f51b80d1b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ecebdfdf0 0x555ecebe8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:23.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.129+0000 7f51b80d1b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecebe3c80 0x555ecebe8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:23.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.129+0000 7f51b80d1b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x555ecebebf30 con 0x555ecebe3c80 2024-05-10T11:35:23.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.129+0000 7f51b80d1b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x555ecea97c30 con 0x555ecebdfdf0 2024-05-10T11:35:23.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.129+0000 7f51b80d1b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x555ecebec0b0 con 0x555ecebdf4d0 2024-05-10T11:35:23.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.129+0000 7f51b80d1b80 10 client.0 _ll_register_callbacks cb 0x555eceac9920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:35:23.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.129+0000 7f51a6ffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ecebdf4d0 0x555ecebeaf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:23.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.129+0000 7f51a6ffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ecebdf4d0 0x555ecebeaf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:46924/0 (socket says 172.21.15.100:46924) 2024-05-10T11:35:23.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.129+0000 7f51a6ffd700 1 -- 172.21.15.100:0/3645573107 learned_addr learned my addr 172.21.15.100:0/3645573107 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:35:23.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.130+0000 7f51a77fe700 1 --2- 172.21.15.100:0/3645573107 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecebe3c80 0x555ecebe8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:23.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.130+0000 7f51a6ffd700 1 -- 172.21.15.100:0/3645573107 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ecebdfdf0 msgr2=0x555ecebe8310 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T11:35:23.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.130+0000 7f51a6ffd700 1 --2- 172.21.15.100:0/3645573107 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ecebdfdf0 0x555ecebe8310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:23.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.130+0000 7f51a6ffd700 1 -- 172.21.15.100:0/3645573107 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecebe3c80 msgr2=0x555ecebe8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:23.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.130+0000 7f51a6ffd700 1 --2- 172.21.15.100:0/3645573107 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecebe3c80 0x555ecebe8850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:23.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.130+0000 7f51a6ffd700 1 -- 172.21.15.100:0/3645573107 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555eceacd690 con 0x555ecebdf4d0 2024-05-10T11:35:23.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.130+0000 7f51a77fe700 1 --2- 172.21.15.100:0/3645573107 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecebe3c80 0x555ecebe8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:35:23.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.130+0000 7f51a6ffd700 1 --2- 172.21.15.100:0/3645573107 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ecebdf4d0 0x555ecebeaf70 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5198015000 tx=0x7f5198009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=391691738cf4e3be server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:23.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.130+0000 7f5186ffd700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:35:23.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.131+0000 7f5186ffd700 1 -- 172.21.15.100:0/3645573107 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5198023810 con 0x555ecebdf4d0 2024-05-10T11:35:23.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.131+0000 7f5186ffd700 1 -- 172.21.15.100:0/3645573107 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f51980239b0 con 0x555ecebdf4d0 2024-05-10T11:35:23.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.131+0000 7f51b80d1b80 1 -- 172.21.15.100:0/3645573107 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51a00a9280 con 0x555ecebdf4d0 2024-05-10T11:35:23.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.131+0000 7f5186ffd700 1 -- 172.21.15.100:0/3645573107 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f5198023bb0 con 0x555ecebdf4d0 2024-05-10T11:35:23.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.131+0000 7f51b80d1b80 10 client.28189 fetch_fsmap learned FSMap version 40 2024-05-10T11:35:23.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.131+0000 7f51b80d1b80 1 -- 172.21.15.100:0/3645573107 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x555ecebf6690 con 0x555ecebdf4d0 2024-05-10T11:35:23.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.131+0000 7f5186ffd700 1 -- 172.21.15.100:0/3645573107 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f51a00a9280 con 0x555ecebdf4d0 2024-05-10T11:35:23.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.132+0000 7f5186ffd700 1 -- 172.21.15.100:0/3645573107 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 40) ==== 8350+0+0 (secure 0 0 0) 0x7f5198022070 con 0x555ecebdf4d0 2024-05-10T11:35:23.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.132+0000 7f51b80d1b80 10 client.28189 fetch_fsmap finished waiting for FSMap version 40 2024-05-10T11:35:23.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.132+0000 7f51b80d1b80 10 client.28189 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:35:23.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.132+0000 7f51b80d1b80 20 client.28189 populate_metadata read hostname 'smithi100' 2024-05-10T11:35:23.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.132+0000 7f51b80d1b80 1 --2- 172.21.15.100:0/3645573107 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x555ecebf8d20 0x555ecebfb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:23.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.132+0000 7f51b80d1b80 4 client.28189 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:35:23.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.132+0000 7f51b80d1b80 1 -- 172.21.15.100:0/3645573107 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ecebab650 con 0x555ecebf8d20 2024-05-10T11:35:23.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.132+0000 7f51a67fc700 1 --2- 172.21.15.100:0/3645573107 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x555ecebf8d20 0x555ecebfb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:23.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.132+0000 7f5186ffd700 10 client.28189 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:35:23.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.133+0000 7f51a67fc700 1 --2- 172.21.15.100:0/3645573107 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x555ecebf8d20 0x555ecebfb1b0 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4341ca2e9b7b8887 server_cookie=13928e89a750f9d in_seq=0 out_seq=0 2024-05-10T11:35:23.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.133+0000 7f5186ffd700 1 -- 172.21.15.100:0/3645573107 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555ecebab650 con 0x555ecebf8d20 2024-05-10T11:35:23.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.133+0000 7f5186ffd700 10 client.28189 handle_command_reply: tid=0 2024-05-10T11:35:23.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.133+0000 7f51b80d1b80 1 client.28189 shutdown 2024-05-10T11:35:23.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.134+0000 7f51b80d1b80 1 -- 172.21.15.100:0/3645573107 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ecebdf4d0 msgr2=0x555ecebeaf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:23.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.134+0000 7f51b80d1b80 1 --2- 172.21.15.100:0/3645573107 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ecebdf4d0 0x555ecebeaf70 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5198015000 tx=0x7f5198009ca0 comp rx=0 tx=0).stop 2024-05-10T11:35:23.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.135+0000 7f51b80d1b80 1 -- 172.21.15.100:0/3645573107 shutdown_connections 2024-05-10T11:35:23.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.135+0000 7f51b80d1b80 1 --2- 172.21.15.100:0/3645573107 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x555ecebf8d20 0x555ecebfb1b0 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:23.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.135+0000 7f51b80d1b80 1 --2- 172.21.15.100:0/3645573107 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecebe3c80 0x555ecebe8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:23.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.135+0000 7f51b80d1b80 1 --2- 172.21.15.100:0/3645573107 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ecebdf4d0 0x555ecebeaf70 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:23.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.135+0000 7f51b80d1b80 1 --2- 172.21.15.100:0/3645573107 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ecebdfdf0 0x555ecebe8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:23.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.135+0000 7f5186ffd700 0 client.28189 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:35:23.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.135+0000 7f51b80d1b80 1 -- 172.21.15.100:0/3645573107 >> 172.21.15.100:0/3645573107 conn(0x555eceb0b100 msgr2=0x555ecebd9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:23.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.135+0000 7f51b80d1b80 1 -- 172.21.15.100:0/3645573107 shutdown_connections 2024-05-10T11:35:23.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.135+0000 7f51b80d1b80 1 -- 172.21.15.100:0/3645573107 wait complete. 2024-05-10T11:35:23.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.135+0000 7f51b80d1b80 20 client.28189 trim_cache size 0 max 16384 2024-05-10T11:35:23.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.167+0000 7f51b80d1b80 1 Processor -- start 2024-05-10T11:35:23.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.167+0000 7f51b80d1b80 1 -- start start 2024-05-10T11:35:23.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.168+0000 7f51b80d1b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555eceb0b100 0x555ecebde9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:23.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.168+0000 7f51b80d1b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecebdf4d0 0x555ecebe1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:23.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.168+0000 7f51b80d1b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ecebdfdf0 0x555ecebe1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:23.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.168+0000 7f51a67fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecebdf4d0 0x555ecebe1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:23.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.168+0000 7f51b80d1b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f51940112b0 con 0x555ecebdf4d0 2024-05-10T11:35:23.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.168+0000 7f51b80d1b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f519400eb10 con 0x555eceb0b100 2024-05-10T11:35:23.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.168+0000 7f51b80d1b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f519400c960 con 0x555ecebdfdf0 2024-05-10T11:35:23.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.168+0000 7f51a67fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecebdf4d0 0x555ecebe1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55168/0 (socket says 172.21.15.100:55168) 2024-05-10T11:35:23.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.168+0000 7f51a67fc700 1 -- 172.21.15.100:0/1645409516 learned_addr learned my addr 172.21.15.100:0/1645409516 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:35:23.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.168+0000 7f51a6ffd700 1 --2- 172.21.15.100:0/1645409516 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555eceb0b100 0x555ecebde9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:23.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.168+0000 7f51a77fe700 1 --2- 172.21.15.100:0/1645409516 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ecebdfdf0 0x555ecebe1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:23.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.169+0000 7f51a67fc700 1 -- 172.21.15.100:0/1645409516 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ecebdfdf0 msgr2=0x555ecebe1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:23.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.169+0000 7f51a67fc700 1 --2- 172.21.15.100:0/1645409516 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ecebdfdf0 0x555ecebe1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:23.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.169+0000 7f51a67fc700 1 -- 172.21.15.100:0/1645409516 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555eceb0b100 msgr2=0x555ecebde9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:23.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.169+0000 7f51a67fc700 1 --2- 172.21.15.100:0/1645409516 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555eceb0b100 0x555ecebde9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:23.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.169+0000 7f51a67fc700 1 -- 172.21.15.100:0/1645409516 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ecebfb830 con 0x555ecebdf4d0 2024-05-10T11:35:23.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.169+0000 7f51a6ffd700 1 --2- 172.21.15.100:0/1645409516 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555eceb0b100 0x555ecebde9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:35:23.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.169+0000 7f51a67fc700 1 --2- 172.21.15.100:0/1645409516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecebdf4d0 0x555ecebe1520 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x555ecebe98e0 tx=0x7f519c010710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb704121ffcafd77 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:23.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.169+0000 7f518d7fa700 1 -- 172.21.15.100:0/1645409516 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f519c010d90 con 0x555ecebdf4d0 2024-05-10T11:35:23.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.169+0000 7f518d7fa700 1 -- 172.21.15.100:0/1645409516 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f519c021070 con 0x555ecebdf4d0 2024-05-10T11:35:23.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.169+0000 7f518d7fa700 1 -- 172.21.15.100:0/1645409516 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f519c003680 con 0x555ecebdf4d0 2024-05-10T11:35:23.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.170+0000 7f51b80d1b80 1 -- 172.21.15.100:0/1645409516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecebdf4d0 msgr2=0x555ecebe1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:23.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.170+0000 7f51b80d1b80 1 --2- 172.21.15.100:0/1645409516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecebdf4d0 0x555ecebe1520 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x555ecebe98e0 tx=0x7f519c010710 comp rx=0 tx=0).stop 2024-05-10T11:35:23.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.170+0000 7f51b80d1b80 1 -- 172.21.15.100:0/1645409516 shutdown_connections 2024-05-10T11:35:23.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.170+0000 7f51b80d1b80 1 --2- 172.21.15.100:0/1645409516 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ecebdfdf0 0x555ecebe1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:23.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.170+0000 7f51b80d1b80 1 --2- 172.21.15.100:0/1645409516 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555eceb0b100 0x555ecebde9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:23.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.170+0000 7f51b80d1b80 1 --2- 172.21.15.100:0/1645409516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecebdf4d0 0x555ecebe1520 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:23.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.170+0000 7f51b80d1b80 1 -- 172.21.15.100:0/1645409516 >> 172.21.15.100:0/1645409516 conn(0x555ecebd4ab0 msgr2=0x555ecebde2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:23.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.170+0000 7f51b80d1b80 1 -- 172.21.15.100:0/1645409516 shutdown_connections 2024-05-10T11:35:23.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.171+0000 7f51b80d1b80 1 -- 172.21.15.100:0/1645409516 wait complete. 2024-05-10T11:35:23.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.171+0000 7f51b80d1b80 1 -- start start 2024-05-10T11:35:23.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.171+0000 7f51b80d1b80 1 Processor -- start 2024-05-10T11:35:23.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.171+0000 7f51b80d1b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555eceb0b100 0x555ecebf1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:23.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.172+0000 7f51b80d1b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecebdf4d0 0x555ecebf18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:23.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.172+0000 7f51b80d1b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ecebdfdf0 0x555ecebee340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:23.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.172+0000 7f51b80d1b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f519400cfd0 con 0x555ecebdf4d0 2024-05-10T11:35:23.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.172+0000 7f51b80d1b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f519400e440 con 0x555ecebdfdf0 2024-05-10T11:35:23.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.172+0000 7f51b80d1b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f519400d6a0 con 0x555eceb0b100 2024-05-10T11:35:23.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.172+0000 7f51b80d1b80 10 client.0 _ll_register_callbacks cb 0x555eceac9920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:35:23.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.172+0000 7f51a6ffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555eceb0b100 0x555ecebf1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:23.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.172+0000 7f51a67fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecebdf4d0 0x555ecebf18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:23.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.172+0000 7f51a67fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecebdf4d0 0x555ecebf18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55184/0 (socket says 172.21.15.100:55184) 2024-05-10T11:35:23.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.172+0000 7f51a67fc700 1 -- 172.21.15.100:0/4131544788 learned_addr learned my addr 172.21.15.100:0/4131544788 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:35:23.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.172+0000 7f51a77fe700 1 --2- 172.21.15.100:0/4131544788 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ecebdfdf0 0x555ecebee340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:23.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.172+0000 7f51a67fc700 1 -- 172.21.15.100:0/4131544788 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555eceb0b100 msgr2=0x555ecebf1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:23.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.172+0000 7f51a67fc700 1 --2- 172.21.15.100:0/4131544788 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555eceb0b100 0x555ecebf1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:23.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.172+0000 7f51a67fc700 1 -- 172.21.15.100:0/4131544788 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ecebdfdf0 msgr2=0x555ecebee340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:23.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.172+0000 7f51a67fc700 1 --2- 172.21.15.100:0/4131544788 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ecebdfdf0 0x555ecebee340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:23.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.172+0000 7f51a67fc700 1 -- 172.21.15.100:0/4131544788 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ecebeeae0 con 0x555ecebdf4d0 2024-05-10T11:35:23.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.172+0000 7f51a6ffd700 1 --2- 172.21.15.100:0/4131544788 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555eceb0b100 0x555ecebf1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:35:23.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.172+0000 7f51a77fe700 1 --2- 172.21.15.100:0/4131544788 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ecebdfdf0 0x555ecebee340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:35:23.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.172+0000 7f51a67fc700 1 --2- 172.21.15.100:0/4131544788 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecebdf4d0 0x555ecebf18b0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f519c001eb0 tx=0x7f519c01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13a8b3ca82c4edda server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:23.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.173+0000 7f5186ffd700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:35:23.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.173+0000 7f5186ffd700 1 -- 172.21.15.100:0/4131544788 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f519c010e60 con 0x555ecebdf4d0 2024-05-10T11:35:23.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.173+0000 7f5186ffd700 1 -- 172.21.15.100:0/4131544788 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f519c01c5d0 con 0x555ecebdf4d0 2024-05-10T11:35:23.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.173+0000 7f51b80d1b80 1 -- 172.21.15.100:0/4131544788 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f519c021070 con 0x555ecebdf4d0 2024-05-10T11:35:23.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.173+0000 7f5186ffd700 1 -- 172.21.15.100:0/4131544788 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f519c004020 con 0x555ecebdf4d0 2024-05-10T11:35:23.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.173+0000 7f51b80d1b80 10 client.18789 fetch_fsmap learned FSMap version 40 2024-05-10T11:35:23.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.173+0000 7f51b80d1b80 1 -- 172.21.15.100:0/4131544788 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x555ecec64130 con 0x555ecebdf4d0 2024-05-10T11:35:23.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.173+0000 7f5186ffd700 1 -- 172.21.15.100:0/4131544788 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f519c029070 con 0x555ecebdf4d0 2024-05-10T11:35:23.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.174+0000 7f5186ffd700 1 -- 172.21.15.100:0/4131544788 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) ==== 8350+0+0 (secure 0 0 0) 0x7f519c003cb0 con 0x555ecebdf4d0 2024-05-10T11:35:23.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.174+0000 7f51b80d1b80 10 client.18789 fetch_fsmap finished waiting for FSMap version 40 2024-05-10T11:35:23.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.174+0000 7f51b80d1b80 10 client.18789 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:35:23.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.174+0000 7f51b80d1b80 20 client.18789 populate_metadata read hostname 'smithi100' 2024-05-10T11:35:23.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.174+0000 7f51b80d1b80 1 --2- 172.21.15.100:0/4131544788 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x555ecebe3c80 0x555ecec66880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:23.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.174+0000 7f51b80d1b80 4 client.18789 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T11:35:23.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.174+0000 7f51b80d1b80 1 -- 172.21.15.100:0/4131544788 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f519c021070 con 0x555ecebe3c80 2024-05-10T11:35:23.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.174+0000 7f51a6ffd700 1 --2- 172.21.15.100:0/4131544788 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x555ecebe3c80 0x555ecec66880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:23.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.175+0000 7f5186ffd700 10 client.18789 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:35:23.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.175+0000 7f51a6ffd700 1 --2- 172.21.15.100:0/4131544788 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x555ecebe3c80 0x555ecec66880 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70f994a0df3515ab server_cookie=906bd98a6f812138 in_seq=0 out_seq=0 2024-05-10T11:35:23.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.176+0000 7f5186ffd700 1 -- 172.21.15.100:0/4131544788 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f519c021070 con 0x555ecebe3c80 2024-05-10T11:35:23.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.176+0000 7f5186ffd700 10 client.18789 handle_command_reply: tid=0 2024-05-10T11:35:23.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.176+0000 7f51b80d1b80 1 client.18789 shutdown 2024-05-10T11:35:23.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.176+0000 7f51b80d1b80 1 -- 172.21.15.100:0/4131544788 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecebdf4d0 msgr2=0x555ecebf18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:23.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.176+0000 7f51b80d1b80 1 --2- 172.21.15.100:0/4131544788 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecebdf4d0 0x555ecebf18b0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f519c001eb0 tx=0x7f519c01ca70 comp rx=0 tx=0).stop 2024-05-10T11:35:23.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.176+0000 7f51b80d1b80 1 -- 172.21.15.100:0/4131544788 shutdown_connections 2024-05-10T11:35:23.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.176+0000 7f51b80d1b80 1 --2- 172.21.15.100:0/4131544788 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x555ecebe3c80 0x555ecec66880 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:23.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.176+0000 7f51b80d1b80 1 --2- 172.21.15.100:0/4131544788 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ecebdfdf0 0x555ecebee340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:23.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.176+0000 7f51b80d1b80 1 --2- 172.21.15.100:0/4131544788 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555eceb0b100 0x555ecebf1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:23.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.176+0000 7f51b80d1b80 1 --2- 172.21.15.100:0/4131544788 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecebdf4d0 0x555ecebf18b0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:23.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.176+0000 7f51b80d1b80 1 -- 172.21.15.100:0/4131544788 >> 172.21.15.100:0/4131544788 conn(0x555ecebd4ab0 msgr2=0x555ecebdeb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:23.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.176+0000 7f5186ffd700 0 client.18789 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:35:23.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.177+0000 7f51b80d1b80 1 -- 172.21.15.100:0/4131544788 shutdown_connections 2024-05-10T11:35:23.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.177+0000 7f51b80d1b80 1 -- 172.21.15.100:0/4131544788 wait complete. 2024-05-10T11:35:23.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.177+0000 7f51b80d1b80 20 client.18789 trim_cache size 0 max 16384 2024-05-10T11:35:23.179 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:35:23.179 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-05-10T11:35:23.179 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-05-10T11:35:23.179 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:35:23.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.178+0000 7f51ad83e700 1 -- 172.21.15.100:0/2133841681 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f51900bb110 msgr2=0x7f51900bd5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:23.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.178+0000 7f51ad83e700 1 --2- 172.21.15.100:0/2133841681 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f51900bb110 0x7f51900bd5d0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f519800b2e0 tx=0x7f519801a040 comp rx=0 tx=0).stop 2024-05-10T11:35:23.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.178+0000 7f51ad83e700 1 -- 172.21.15.100:0/2133841681 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51a8108440 msgr2=0x7f51a81987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:23.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.178+0000 7f51ad83e700 1 --2- 172.21.15.100:0/2133841681 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51a8108440 0x7f51a81987e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f51a000ef30 tx=0x7f51a000c3a0 comp rx=0 tx=0).stop 2024-05-10T11:35:23.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.178+0000 7f51ad83e700 1 -- 172.21.15.100:0/2133841681 shutdown_connections 2024-05-10T11:35:23.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.178+0000 7f51ad83e700 1 --2- 172.21.15.100:0/2133841681 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f51900bb110 0x7f51900bd5d0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:23.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.178+0000 7f51ad83e700 1 --2- 172.21.15.100:0/2133841681 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51a8108440 0x7f51a81987e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:23.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.178+0000 7f51ad83e700 1 --2- 172.21.15.100:0/2133841681 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51a81023d0 0x7f51a819e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:23.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.178+0000 7f51ad83e700 1 --2- 172.21.15.100:0/2133841681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51a8102d90 0x7f51a819eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:23.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.178+0000 7f51ad83e700 1 -- 172.21.15.100:0/2133841681 >> 172.21.15.100:0/2133841681 conn(0x7f51a80793e0 msgr2=0x7f51a81012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:23.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.178+0000 7f51ad83e700 1 -- 172.21.15.100:0/2133841681 shutdown_connections 2024-05-10T11:35:23.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:23.178+0000 7f51ad83e700 1 -- 172.21.15.100:0/2133841681 wait complete. 2024-05-10T11:35:23.919 INFO:tasks.workunit.client.0.smithi100.stdout: 10 33519 9.30 MB/sec execute 77 sec latency 82.521 ms 2024-05-10T11:35:23.937 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:35:23 smithi100 ceph-mon[30296]: pgmap v931: 129 pgs: 129 active+clean; 431 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 2.9 MiB/s wr, 278 op/s 2024-05-10T11:35:23.976 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T11:35:23.976 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T11:35:24.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:35:23 smithi178 ceph-mon[33359]: pgmap v931: 129 pgs: 129 active+clean; 431 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 2.9 MiB/s wr, 278 op/s 2024-05-10T11:35:24.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:35:23 smithi188 ceph-mon[34075]: pgmap v931: 129 pgs: 129 active+clean; 431 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 2.9 MiB/s wr, 278 op/s 2024-05-10T11:35:24.381 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:35:24.919 INFO:tasks.workunit.client.0.smithi100.stdout: 10 33681 9.27 MB/sec execute 78 sec latency 69.667 ms 2024-05-10T11:35:25.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:35:25 smithi100 ceph-mon[30296]: pgmap v932: 129 pgs: 129 active+clean; 434 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 551 KiB/s rd, 3.2 MiB/s wr, 236 op/s 2024-05-10T11:35:25.919 INFO:tasks.workunit.client.0.smithi100.stdout: 10 33813 9.25 MB/sec execute 79 sec latency 57.413 ms 2024-05-10T11:35:26.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:35:25 smithi178 ceph-mon[33359]: pgmap v932: 129 pgs: 129 active+clean; 434 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 551 KiB/s rd, 3.2 MiB/s wr, 236 op/s 2024-05-10T11:35:26.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:35:25 smithi188 ceph-mon[34075]: pgmap v932: 129 pgs: 129 active+clean; 434 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 551 KiB/s rd, 3.2 MiB/s wr, 236 op/s 2024-05-10T11:35:26.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:26.315+0000 7fa7fe040700 1 -- 172.21.15.100:0/3032036218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7f8108410 msgr2=0x7fa7f81087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:26.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:26.315+0000 7fa7fe040700 1 --2- 172.21.15.100:0/3032036218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7f8108410 0x7fa7f81087f0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fa7e800b0a0 tx=0x7fa7e800b3b0 comp rx=0 tx=0).stop 2024-05-10T11:35:26.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:26.315+0000 7fa7fe040700 1 -- 172.21.15.100:0/3032036218 shutdown_connections 2024-05-10T11:35:26.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:26.315+0000 7fa7fe040700 1 --2- 172.21.15.100:0/3032036218 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7f8102d60 0x7fa7f810f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:26.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:26.315+0000 7fa7fe040700 1 --2- 172.21.15.100:0/3032036218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7f8108410 0x7fa7f81087f0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:26.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:26.315+0000 7fa7fe040700 1 --2- 172.21.15.100:0/3032036218 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7f81023a0 0x7fa7f8102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:26.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:26.315+0000 7fa7fe040700 1 -- 172.21.15.100:0/3032036218 >> 172.21.15.100:0/3032036218 conn(0x7fa7f8079350 msgr2=0x7fa7f8079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:26.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:26.315+0000 7fa7fe040700 1 -- 172.21.15.100:0/3032036218 shutdown_connections 2024-05-10T11:35:26.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:26.315+0000 7fa7fe040700 1 -- 172.21.15.100:0/3032036218 wait complete. 2024-05-10T11:35:26.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:26.316+0000 7fa7fe040700 1 Processor -- start 2024-05-10T11:35:26.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:26.316+0000 7fa7fe040700 1 -- start start 2024-05-10T11:35:26.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:26.317+0000 7fa7fe040700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7f81023a0 0x7fa7f819e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:26.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:26.317+0000 7fa7fe040700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7f8102d60 0x7fa7f819ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:26.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:26.317+0000 7fa7fe040700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7f8108410 0x7fa7f8198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:26.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:26.317+0000 7fa7fe040700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f8112410 con 0x7fa7f8102d60 2024-05-10T11:35:26.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:26.317+0000 7fa7fe040700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f8112290 con 0x7fa7f81023a0 2024-05-10T11:35:26.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:26.317+0000 7fa7eeffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7f8102d60 0x7fa7f819ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:26.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:26.317+0000 7fa7fe040700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f8112590 con 0x7fa7f8108410 2024-05-10T11:35:26.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:26.317+0000 7fa7f77fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7f81023a0 0x7fa7f819e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:26.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:26.317+0000 7fa7eeffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7f8102d60 0x7fa7f819ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55202/0 (socket says 172.21.15.100:55202) 2024-05-10T11:35:26.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:26.317+0000 7fa7eeffd700 1 -- 172.21.15.100:0/1879623884 learned_addr learned my addr 172.21.15.100:0/1879623884 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:35:26.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:26.317+0000 7fa7f7fff700 1 --2- 172.21.15.100:0/1879623884 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7f8108410 0x7fa7f8198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:26.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:26.318+0000 7fa7f77fe700 1 -- 172.21.15.100:0/1879623884 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7f8108410 msgr2=0x7fa7f8198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:26.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:26.318+0000 7fa7f77fe700 1 --2- 172.21.15.100:0/1879623884 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7f8108410 0x7fa7f8198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:26.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:26.318+0000 7fa7f77fe700 1 -- 172.21.15.100:0/1879623884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7f8102d60 msgr2=0x7fa7f819ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:26.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:26.318+0000 7fa7f77fe700 1 --2- 172.21.15.100:0/1879623884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7f8102d60 0x7fa7f819ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:26.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:26.318+0000 7fa7f77fe700 1 -- 172.21.15.100:0/1879623884 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7f8199030 con 0x7fa7f81023a0 2024-05-10T11:35:26.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:26.318+0000 7fa7f77fe700 1 --2- 172.21.15.100:0/1879623884 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7f81023a0 0x7fa7f819e680 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa7e800b070 tx=0x7fa7e8004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:26.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:26.318+0000 7fa7f57fa700 1 -- 172.21.15.100:0/1879623884 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e8021070 con 0x7fa7f81023a0 2024-05-10T11:35:26.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:26.318+0000 7fa7f57fa700 1 -- 172.21.15.100:0/1879623884 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa7e8007b80 con 0x7fa7f81023a0 2024-05-10T11:35:26.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:26.318+0000 7fa7fe040700 1 -- 172.21.15.100:0/1879623884 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7f81992c0 con 0x7fa7f81023a0 2024-05-10T11:35:26.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:26.319+0000 7fa7f57fa700 1 -- 172.21.15.100:0/1879623884 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e801de20 con 0x7fa7f81023a0 2024-05-10T11:35:26.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:26.319+0000 7fa7fe040700 1 -- 172.21.15.100:0/1879623884 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7f81a5600 con 0x7fa7f81023a0 2024-05-10T11:35:26.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:26.320+0000 7fa7fe040700 1 -- 172.21.15.100:0/1879623884 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7f8199620 con 0x7fa7f81023a0 2024-05-10T11:35:26.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:26.321+0000 7fa7f57fa700 1 -- 172.21.15.100:0/1879623884 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fa7e8024070 con 0x7fa7f81023a0 2024-05-10T11:35:26.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:26.322+0000 7fa7f57fa700 1 --2- 172.21.15.100:0/1879623884 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa7e40b2900 0x7fa7e40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:26.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:26.322+0000 7fa7f57fa700 1 -- 172.21.15.100:0/1879623884 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fa7e800f020 con 0x7fa7f81023a0 2024-05-10T11:35:26.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:26.322+0000 7fa7eeffd700 1 --2- 172.21.15.100:0/1879623884 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa7e40b2900 0x7fa7e40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:26.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:26.323+0000 7fa7eeffd700 1 --2- 172.21.15.100:0/1879623884 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa7e40b2900 0x7fa7e40b4dc0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa7f8199f10 tx=0x7fa7e0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:26.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:26.330+0000 7fa7f57fa700 1 -- 172.21.15.100:0/1879623884 <== mon.2 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fa7e802b020 con 0x7fa7f81023a0 2024-05-10T11:35:26.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:26.764+0000 7fa7fe040700 1 -- 172.21.15.100:0/1879623884 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa7f806a190 con 0x7fa7f81023a0 2024-05-10T11:35:26.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:26.765+0000 7fa7f57fa700 1 -- 172.21.15.100:0/1879623884 <== mon.2 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11898 (secure 0 0 0) 0x7fa7e80ab2f0 con 0x7fa7f81023a0 2024-05-10T11:35:26.767 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:35:26.767 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2072225012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2072225012},{"type":"v1","addr":"172.21.15.100:6835","nonce":2072225012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/401939540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":401939540},{"type":"v1","addr":"172.21.15.100:6837","nonce":401939540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14571,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/3946311025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3946311025},{"type":"v1","addr":"172.21.15.100:6839","nonce":3946311025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3736371660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3736371660},{"type":"v1","addr":"172.21.15.100:6841","nonce":3736371660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/2741810791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2741810791},{"type":"v1","addr":"172.21.15.178:6833","nonce":2741810791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1704891041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1704891041},{"type":"v1","addr":"172.21.15.188:6833","nonce":1704891041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T11:14:27.242452+0000","modified":"2024-05-10T11:34:54.585396+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24439,"mds_1":24437,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/2124076312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2124076312},{"type":"v1","addr":"172.21.15.178:6835","nonce":2124076312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/3224358425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3224358425},{"type":"v1","addr":"172.21.15.178:6837","nonce":3224358425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/1943342620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1943342620},{"type":"v1","addr":"172.21.15.188:6837","nonce":1943342620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.178:6839/3760985404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3760985404},{"type":"v1","addr":"172.21.15.178:6839","nonce":3760985404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.188:6839/3194917070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":3194917070},{"type":"v1","addr":"172.21.15.188:6839","nonce":3194917070}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27034":{"gid":27034,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/3139559450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3139559450},{"type":"v1","addr":"172.21.15.188:6835","nonce":3139559450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24439,"qdb_cluster":[24439,24427,24437]},"id":1}]} 2024-05-10T11:35:26.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:26.768+0000 7fa7fe040700 1 -- 172.21.15.100:0/1879623884 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa7e40b2900 msgr2=0x7fa7e40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:26.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:26.768+0000 7fa7fe040700 1 --2- 172.21.15.100:0/1879623884 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa7e40b2900 0x7fa7e40b4dc0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa7f8199f10 tx=0x7fa7e0009260 comp rx=0 tx=0).stop 2024-05-10T11:35:26.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:26.768+0000 7fa7fe040700 1 -- 172.21.15.100:0/1879623884 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7f81023a0 msgr2=0x7fa7f819e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:26.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:26.768+0000 7fa7fe040700 1 --2- 172.21.15.100:0/1879623884 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7f81023a0 0x7fa7f819e680 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa7e800b070 tx=0x7fa7e8004d00 comp rx=0 tx=0).stop 2024-05-10T11:35:26.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:26.769+0000 7fa7fe040700 1 -- 172.21.15.100:0/1879623884 shutdown_connections 2024-05-10T11:35:26.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:26.769+0000 7fa7fe040700 1 --2- 172.21.15.100:0/1879623884 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa7e40b2900 0x7fa7e40b4dc0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:26.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:26.769+0000 7fa7fe040700 1 --2- 172.21.15.100:0/1879623884 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7f8108410 0x7fa7f8198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:26.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:26.769+0000 7fa7fe040700 1 --2- 172.21.15.100:0/1879623884 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7f81023a0 0x7fa7f819e680 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:26.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:26.769+0000 7fa7fe040700 1 --2- 172.21.15.100:0/1879623884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7f8102d60 0x7fa7f819ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:26.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:26.769+0000 7fa7fe040700 1 -- 172.21.15.100:0/1879623884 >> 172.21.15.100:0/1879623884 conn(0x7fa7f8079350 msgr2=0x7fa7f8101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:26.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:26.769+0000 7fa7fe040700 1 -- 172.21.15.100:0/1879623884 shutdown_connections 2024-05-10T11:35:26.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:26.769+0000 7fa7fe040700 1 -- 172.21.15.100:0/1879623884 wait complete. 2024-05-10T11:35:26.771 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 40 2024-05-10T11:35:26.919 INFO:tasks.workunit.client.0.smithi100.stdout: 10 33952 9.25 MB/sec execute 80 sec latency 81.657 ms 2024-05-10T11:35:27.441 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-10T11:35:27.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:35:27 smithi100 ceph-mon[30296]: pgmap v933: 129 pgs: 129 active+clean; 441 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 780 KiB/s rd, 4.2 MiB/s wr, 270 op/s 2024-05-10T11:35:27.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:35:27 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/1879623884' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:35:27.815 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:35:27.919 INFO:tasks.workunit.client.0.smithi100.stdout: 10 34146 9.35 MB/sec execute 81 sec latency 77.623 ms 2024-05-10T11:35:28.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:35:27 smithi178 ceph-mon[33359]: pgmap v933: 129 pgs: 129 active+clean; 441 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 780 KiB/s rd, 4.2 MiB/s wr, 270 op/s 2024-05-10T11:35:28.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:35:27 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/1879623884' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:35:28.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:35:27 smithi188 ceph-mon[34075]: pgmap v933: 129 pgs: 129 active+clean; 441 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 780 KiB/s rd, 4.2 MiB/s wr, 270 op/s 2024-05-10T11:35:28.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:35:27 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/1879623884' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:35:28.919 INFO:tasks.workunit.client.0.smithi100.stdout: 10 34274 9.30 MB/sec execute 82 sec latency 138.887 ms 2024-05-10T11:35:29.919 INFO:tasks.workunit.client.0.smithi100.stdout: 10 34397 9.26 MB/sec execute 83 sec latency 173.404 ms 2024-05-10T11:35:30.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:35:29 smithi178 ceph-mon[33359]: pgmap v934: 129 pgs: 129 active+clean; 444 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 4.0 MiB/s wr, 243 op/s 2024-05-10T11:35:30.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:35:29 smithi188 ceph-mon[34075]: pgmap v934: 129 pgs: 129 active+clean; 444 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 4.0 MiB/s wr, 243 op/s 2024-05-10T11:35:30.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:35:29 smithi100 ceph-mon[30296]: pgmap v934: 129 pgs: 129 active+clean; 444 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 4.0 MiB/s wr, 243 op/s 2024-05-10T11:35:30.919 INFO:tasks.workunit.client.0.smithi100.stdout: 10 34495 9.19 MB/sec execute 84 sec latency 189.859 ms 2024-05-10T11:35:31.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:35:30 smithi178 ceph-mon[33359]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T11:35:31.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:35:30 smithi188 ceph-mon[34075]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T11:35:31.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:35:30 smithi100 ceph-mon[30296]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T11:35:31.919 INFO:tasks.workunit.client.0.smithi100.stdout: 10 34641 9.11 MB/sec execute 85 sec latency 82.610 ms 2024-05-10T11:35:32.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:35:31 smithi178 ceph-mon[33359]: pgmap v935: 129 pgs: 129 active+clean; 448 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 4.8 MiB/s wr, 257 op/s 2024-05-10T11:35:32.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:35:31 smithi188 ceph-mon[34075]: pgmap v935: 129 pgs: 129 active+clean; 448 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 4.8 MiB/s wr, 257 op/s 2024-05-10T11:35:32.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:35:31 smithi100 ceph-mon[30296]: pgmap v935: 129 pgs: 129 active+clean; 448 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 4.8 MiB/s wr, 257 op/s 2024-05-10T11:35:32.919 INFO:tasks.workunit.client.0.smithi100.stdout: 10 34790 9.12 MB/sec execute 86 sec latency 145.567 ms 2024-05-10T11:35:33.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:35:33 smithi100 ceph-mon[30296]: pgmap v936: 129 pgs: 129 active+clean; 449 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 4.5 MiB/s wr, 244 op/s 2024-05-10T11:35:33.922 INFO:tasks.workunit.client.0.smithi100.stdout: 10 34941 9.05 MB/sec execute 87 sec latency 74.750 ms 2024-05-10T11:35:34.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:35:33 smithi178 ceph-mon[33359]: pgmap v936: 129 pgs: 129 active+clean; 449 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 4.5 MiB/s wr, 244 op/s 2024-05-10T11:35:34.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:35:33 smithi188 ceph-mon[34075]: pgmap v936: 129 pgs: 129 active+clean; 449 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 4.5 MiB/s wr, 244 op/s 2024-05-10T11:35:34.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.203+0000 7f8e1c16e700 1 -- 172.21.15.100:0/3606093175 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e1410b6c0 msgr2=0x7f8e1410baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:34.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.203+0000 7f8e1c16e700 1 --2- 172.21.15.100:0/3606093175 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e1410b6c0 0x7f8e1410baa0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8e10009990 tx=0x7f8e10009ca0 comp rx=0 tx=0).stop 2024-05-10T11:35:34.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.204+0000 7f8e1c16e700 1 -- 172.21.15.100:0/3606093175 shutdown_connections 2024-05-10T11:35:34.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.204+0000 7f8e1c16e700 1 --2- 172.21.15.100:0/3606093175 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e14106080 0x7f8e14112590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:34.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.204+0000 7f8e1c16e700 1 --2- 172.21.15.100:0/3606093175 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e1410b6c0 0x7f8e1410baa0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:34.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.204+0000 7f8e1c16e700 1 --2- 172.21.15.100:0/3606093175 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e141056c0 0x7f8e14105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:34.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.204+0000 7f8e1c16e700 1 -- 172.21.15.100:0/3606093175 >> 172.21.15.100:0/3606093175 conn(0x7f8e141011b0 msgr2=0x7f8e141035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:34.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.204+0000 7f8e1c16e700 1 -- 172.21.15.100:0/3606093175 shutdown_connections 2024-05-10T11:35:34.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.204+0000 7f8e1c16e700 1 -- 172.21.15.100:0/3606093175 wait complete. 2024-05-10T11:35:34.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.205+0000 7f8e1c16e700 1 Processor -- start 2024-05-10T11:35:34.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.205+0000 7f8e1c16e700 1 -- start start 2024-05-10T11:35:34.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.205+0000 7f8e1c16e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e141056c0 0x7f8e140779b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:34.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.205+0000 7f8e1c16e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e14106080 0x7f8e14077ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:34.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.205+0000 7f8e1c16e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e1410b6c0 0x7f8e14078430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:34.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.205+0000 7f8e1c16e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8e141157c0 con 0x7f8e1410b6c0 2024-05-10T11:35:34.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.205+0000 7f8e1c16e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8e14115640 con 0x7f8e141056c0 2024-05-10T11:35:34.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.205+0000 7f8e1c16e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8e14115940 con 0x7f8e14106080 2024-05-10T11:35:34.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.205+0000 7f8e19f0a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e141056c0 0x7f8e140779b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:34.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.205+0000 7f8e19709700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e14106080 0x7f8e14077ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:34.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.205+0000 7f8e19f0a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e141056c0 0x7f8e140779b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:35978/0 (socket says 172.21.15.100:35978) 2024-05-10T11:35:34.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.205+0000 7f8e19f0a700 1 -- 172.21.15.100:0/106357992 learned_addr learned my addr 172.21.15.100:0/106357992 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:35:34.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.205+0000 7f8e19709700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e14106080 0x7f8e14077ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:57578/0 (socket says 172.21.15.100:57578) 2024-05-10T11:35:34.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.205+0000 7f8e1a70b700 1 --2- 172.21.15.100:0/106357992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e1410b6c0 0x7f8e14078430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:34.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.206+0000 7f8e19f0a700 1 -- 172.21.15.100:0/106357992 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e14106080 msgr2=0x7f8e14077ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:34.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.206+0000 7f8e19f0a700 1 --2- 172.21.15.100:0/106357992 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e14106080 0x7f8e14077ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:34.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.206+0000 7f8e19f0a700 1 -- 172.21.15.100:0/106357992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e1410b6c0 msgr2=0x7f8e14078430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:34.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.206+0000 7f8e19f0a700 1 --2- 172.21.15.100:0/106357992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e1410b6c0 0x7f8e14078430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:34.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.206+0000 7f8e19f0a700 1 -- 172.21.15.100:0/106357992 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e141ac380 con 0x7f8e141056c0 2024-05-10T11:35:34.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.206+0000 7f8e19f0a700 1 --2- 172.21.15.100:0/106357992 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e141056c0 0x7f8e140779b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8e10000c00 tx=0x7f8e1000bfc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:34.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.206+0000 7f8e0affd700 1 -- 172.21.15.100:0/106357992 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e10027070 con 0x7f8e141056c0 2024-05-10T11:35:34.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.206+0000 7f8e1c16e700 1 -- 172.21.15.100:0/106357992 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e141ac610 con 0x7f8e141056c0 2024-05-10T11:35:34.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.206+0000 7f8e0affd700 1 -- 172.21.15.100:0/106357992 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8e10022470 con 0x7f8e141056c0 2024-05-10T11:35:34.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.206+0000 7f8e1c16e700 1 -- 172.21.15.100:0/106357992 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e141acac0 con 0x7f8e141056c0 2024-05-10T11:35:34.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.206+0000 7f8e0affd700 1 -- 172.21.15.100:0/106357992 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e10017650 con 0x7f8e141056c0 2024-05-10T11:35:34.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.209+0000 7f8e0affd700 1 -- 172.21.15.100:0/106357992 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f8e1002a070 con 0x7f8e141056c0 2024-05-10T11:35:34.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.209+0000 7f8e0affd700 1 --2- 172.21.15.100:0/106357992 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8e000b2980 0x7f8e000b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:34.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.209+0000 7f8e0affd700 1 -- 172.21.15.100:0/106357992 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f8e100e3360 con 0x7f8e141056c0 2024-05-10T11:35:34.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.210+0000 7f8e19709700 1 --2- 172.21.15.100:0/106357992 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8e000b2980 0x7f8e000b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:34.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.210+0000 7f8e19709700 1 --2- 172.21.15.100:0/106357992 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8e000b2980 0x7f8e000b4e40 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f8e14079170 tx=0x7f8e04009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:34.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.219+0000 7f8e26a01b80 1 Processor -- start 2024-05-10T11:35:34.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.219+0000 7f8e26a01b80 1 -- start start 2024-05-10T11:35:34.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.219+0000 7f8e26a01b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561dd7f2a4d0 0x561dd7f2a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:34.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.219+0000 7f8e26a01b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dd7f2adf0 0x561dd7f2e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:34.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.219+0000 7f8e26a01b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561dd7f2ec80 0x561dd7f2f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:34.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.219+0000 7f8e26a01b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x561dd7e7c860 con 0x561dd7f2adf0 2024-05-10T11:35:34.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.219+0000 7f8e26a01b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561dd7e1b7c0 con 0x561dd7f2ec80 2024-05-10T11:35:34.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.219+0000 7f8e26a01b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x561dd7e78820 con 0x561dd7f2a4d0 2024-05-10T11:35:34.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.219+0000 7f8e1a70b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561dd7f2ec80 0x561dd7f2f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:34.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.219+0000 7f8e1a70b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561dd7f2ec80 0x561dd7f2f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:35994/0 (socket says 172.21.15.100:35994) 2024-05-10T11:35:34.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.219+0000 7f8e1a70b700 1 -- 172.21.15.100:0/3363273111 learned_addr learned my addr 172.21.15.100:0/3363273111 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:35:34.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.219+0000 7f8e19709700 1 --2- 172.21.15.100:0/3363273111 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dd7f2adf0 0x561dd7f2e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:34.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.219+0000 7f8e19f0a700 1 --2- 172.21.15.100:0/3363273111 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561dd7f2a4d0 0x561dd7f2a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:34.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.220+0000 7f8e1a70b700 1 -- 172.21.15.100:0/3363273111 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561dd7f2a4d0 msgr2=0x561dd7f2a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:34.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.220+0000 7f8e1a70b700 1 --2- 172.21.15.100:0/3363273111 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561dd7f2a4d0 0x561dd7f2a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:34.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.220+0000 7f8e1a70b700 1 -- 172.21.15.100:0/3363273111 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dd7f2adf0 msgr2=0x561dd7f2e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:34.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.220+0000 7f8e1a70b700 1 --2- 172.21.15.100:0/3363273111 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dd7f2adf0 0x561dd7f2e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:34.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.220+0000 7f8e1a70b700 1 -- 172.21.15.100:0/3363273111 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561dd7dc98d0 con 0x561dd7f2ec80 2024-05-10T11:35:34.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.220+0000 7f8e1a70b700 1 --2- 172.21.15.100:0/3363273111 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561dd7f2ec80 0x561dd7f2f100 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c003f30 tx=0x7f8e0c009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76dcd5c25d3a13e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:34.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.220+0000 7f8dfffff700 1 -- 172.21.15.100:0/3363273111 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c004670 con 0x561dd7f2ec80 2024-05-10T11:35:34.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.220+0000 7f8dfffff700 1 -- 172.21.15.100:0/3363273111 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8e0c01a070 con 0x561dd7f2ec80 2024-05-10T11:35:34.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.220+0000 7f8dfffff700 1 -- 172.21.15.100:0/3363273111 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c0156e0 con 0x561dd7f2ec80 2024-05-10T11:35:34.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.221+0000 7f8e26a01b80 1 -- 172.21.15.100:0/3363273111 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561dd7f2ec80 msgr2=0x561dd7f2f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:34.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.221+0000 7f8e26a01b80 1 --2- 172.21.15.100:0/3363273111 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561dd7f2ec80 0x561dd7f2f100 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c003f30 tx=0x7f8e0c009c20 comp rx=0 tx=0).stop 2024-05-10T11:35:34.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.221+0000 7f8e26a01b80 1 -- 172.21.15.100:0/3363273111 shutdown_connections 2024-05-10T11:35:34.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.221+0000 7f8e26a01b80 1 --2- 172.21.15.100:0/3363273111 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561dd7f2ec80 0x561dd7f2f100 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:34.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.221+0000 7f8e26a01b80 1 --2- 172.21.15.100:0/3363273111 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561dd7f2a4d0 0x561dd7f2a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:34.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.221+0000 7f8e26a01b80 1 --2- 172.21.15.100:0/3363273111 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dd7f2adf0 0x561dd7f2e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:34.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.221+0000 7f8e26a01b80 1 -- 172.21.15.100:0/3363273111 >> 172.21.15.100:0/3363273111 conn(0x561dd7e56100 msgr2=0x561dd7f24c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:34.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.221+0000 7f8e26a01b80 1 -- 172.21.15.100:0/3363273111 shutdown_connections 2024-05-10T11:35:34.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.221+0000 7f8e26a01b80 1 -- 172.21.15.100:0/3363273111 wait complete. 2024-05-10T11:35:34.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.222+0000 7f8e26a01b80 1 -- start start 2024-05-10T11:35:34.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.222+0000 7f8e26a01b80 1 Processor -- start 2024-05-10T11:35:34.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.223+0000 7f8e26a01b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561dd7f2a4d0 0x561dd7f36000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:34.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.223+0000 7f8e26a01b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561dd7f2adf0 0x561dd7f333a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:34.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.223+0000 7f8e26a01b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dd7f2ec80 0x561dd7f338e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:34.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.223+0000 7f8e19f0a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561dd7f2a4d0 0x561dd7f36000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:34.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.223+0000 7f8e19f0a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561dd7f2a4d0 0x561dd7f36000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:35996/0 (socket says 172.21.15.100:35996) 2024-05-10T11:35:34.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.223+0000 7f8e19709700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561dd7f2adf0 0x561dd7f333a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:34.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.223+0000 7f8e19709700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561dd7f2adf0 0x561dd7f333a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:57602/0 (socket says 172.21.15.100:57602) 2024-05-10T11:35:34.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.223+0000 7f8e1a70b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dd7f2ec80 0x561dd7f338e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:34.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.223+0000 7f8e1a70b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dd7f2ec80 0x561dd7f338e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57474/0 (socket says 172.21.15.100:57474) 2024-05-10T11:35:34.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.223+0000 7f8e19f0a700 1 -- 172.21.15.100:0/2958823367 learned_addr learned my addr 172.21.15.100:0/2958823367 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:35:34.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.223+0000 7f8e26a01b80 1 -- 172.21.15.100:0/2958823367 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x561dd7f36e90 con 0x561dd7f2ec80 2024-05-10T11:35:34.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.223+0000 7f8e26a01b80 1 -- 172.21.15.100:0/2958823367 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561dd7de2c30 con 0x561dd7f2a4d0 2024-05-10T11:35:34.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.223+0000 7f8e26a01b80 1 -- 172.21.15.100:0/2958823367 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x561dd7f37010 con 0x561dd7f2adf0 2024-05-10T11:35:34.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.223+0000 7f8e26a01b80 10 client.0 _ll_register_callbacks cb 0x561dd7e14920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:35:34.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.224+0000 7f8e19f0a700 1 -- 172.21.15.100:0/2958823367 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561dd7f2adf0 msgr2=0x561dd7f333a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:34.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.224+0000 7f8e19f0a700 1 --2- 172.21.15.100:0/2958823367 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561dd7f2adf0 0x561dd7f333a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:34.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.224+0000 7f8e19f0a700 1 -- 172.21.15.100:0/2958823367 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dd7f2ec80 msgr2=0x561dd7f338e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:34.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.224+0000 7f8e19f0a700 1 --2- 172.21.15.100:0/2958823367 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dd7f2ec80 0x561dd7f338e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:34.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.224+0000 7f8e19f0a700 1 -- 172.21.15.100:0/2958823367 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561dd7e18690 con 0x561dd7f2a4d0 2024-05-10T11:35:34.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.224+0000 7f8e26a01b80 1 -- 172.21.15.100:0/2958823367 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e0c01a070 con 0x561dd7f2a4d0 2024-05-10T11:35:34.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.224+0000 7f8e19f0a700 1 --2- 172.21.15.100:0/2958823367 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561dd7f2a4d0 0x561dd7f36000 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8e10005f50 tx=0x7f8e10022b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ec10e750e366acb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:34.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.224+0000 7f8dfd7fa700 10 client.28118 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:35:34.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.224+0000 7f8dfd7fa700 1 -- 172.21.15.100:0/2958823367 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e100aa650 con 0x561dd7f2a4d0 2024-05-10T11:35:34.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.224+0000 7f8dfd7fa700 1 -- 172.21.15.100:0/2958823367 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f8e100920b0 con 0x561dd7f2a4d0 2024-05-10T11:35:34.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.224+0000 7f8dfd7fa700 1 -- 172.21.15.100:0/2958823367 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f8e100aea00 con 0x561dd7f2a4d0 2024-05-10T11:35:34.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.224+0000 7f8e26a01b80 10 client.28118 fetch_fsmap learned FSMap version 41 2024-05-10T11:35:34.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.224+0000 7f8e26a01b80 1 -- 172.21.15.100:0/2958823367 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({fsmap=41}) -- 0x561dd7f41550 con 0x561dd7f2a4d0 2024-05-10T11:35:34.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.225+0000 7f8dfd7fa700 1 -- 172.21.15.100:0/2958823367 <== mon.2 v2:172.21.15.178:3300/0 4 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f8e100aa8b0 con 0x561dd7f2a4d0 2024-05-10T11:35:34.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.225+0000 7f8e26a01b80 10 client.28118 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:35:34.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.225+0000 7f8e26a01b80 10 client.28118 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-10T11:35:34.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.225+0000 7f8e26a01b80 20 client.28118 populate_metadata read hostname 'smithi100' 2024-05-10T11:35:34.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.225+0000 7f8e26a01b80 1 --2- 172.21.15.100:0/2958823367 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x561dd7f43a20 0x561dd7f45e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:34.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.225+0000 7f8e26a01b80 4 client.28118 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:35:34.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.225+0000 7f8e26a01b80 1 -- 172.21.15.100:0/2958823367 --> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561dd7ef6650 con 0x561dd7f43a20 2024-05-10T11:35:34.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.225+0000 7f8e19709700 1 --2- 172.21.15.100:0/2958823367 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x561dd7f43a20 0x561dd7f45e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:34.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.226+0000 7f8dfd7fa700 10 client.28118 ms_handle_connect on v2:172.21.15.188:6836/1943342620 2024-05-10T11:35:34.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.226+0000 7f8e19709700 1 --2- 172.21.15.100:0/2958823367 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x561dd7f43a20 0x561dd7f45e10 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2f11094d52b00c7b server_cookie=af9ffae321a36ff4 in_seq=0 out_seq=0 2024-05-10T11:35:34.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.226+0000 7f8dfd7fa700 1 -- 172.21.15.100:0/2958823367 <== mds.2 v2:172.21.15.188:6836/1943342620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561dd7ef6650 con 0x561dd7f43a20 2024-05-10T11:35:34.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.226+0000 7f8dfd7fa700 10 client.28118 handle_command_reply: tid=0 2024-05-10T11:35:34.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.226+0000 7f8e26a01b80 1 client.28118 shutdown 2024-05-10T11:35:34.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.226+0000 7f8e26a01b80 1 -- 172.21.15.100:0/2958823367 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561dd7f2a4d0 msgr2=0x561dd7f36000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:34.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.226+0000 7f8e26a01b80 1 --2- 172.21.15.100:0/2958823367 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561dd7f2a4d0 0x561dd7f36000 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8e10005f50 tx=0x7f8e10022b50 comp rx=0 tx=0).stop 2024-05-10T11:35:34.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.227+0000 7f8e26a01b80 1 -- 172.21.15.100:0/2958823367 shutdown_connections 2024-05-10T11:35:34.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.227+0000 7f8e26a01b80 1 --2- 172.21.15.100:0/2958823367 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x561dd7f43a20 0x561dd7f45e10 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:34.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.227+0000 7f8dfd7fa700 0 client.28118 ms_handle_reset on v2:172.21.15.188:6836/1943342620 2024-05-10T11:35:34.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.227+0000 7f8e26a01b80 1 --2- 172.21.15.100:0/2958823367 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dd7f2ec80 0x561dd7f338e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:34.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.227+0000 7f8e26a01b80 1 --2- 172.21.15.100:0/2958823367 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561dd7f2a4d0 0x561dd7f36000 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:34.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.227+0000 7f8e26a01b80 1 --2- 172.21.15.100:0/2958823367 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561dd7f2adf0 0x561dd7f333a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:34.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.227+0000 7f8e26a01b80 1 -- 172.21.15.100:0/2958823367 >> 172.21.15.100:0/2958823367 conn(0x561dd7e56100 msgr2=0x561dd7f2f080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:34.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.227+0000 7f8e26a01b80 1 -- 172.21.15.100:0/2958823367 shutdown_connections 2024-05-10T11:35:34.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.227+0000 7f8e26a01b80 1 -- 172.21.15.100:0/2958823367 wait complete. 2024-05-10T11:35:34.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.227+0000 7f8e26a01b80 20 client.28118 trim_cache size 0 max 16384 2024-05-10T11:35:34.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.252+0000 7f8e26a01b80 1 Processor -- start 2024-05-10T11:35:34.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.252+0000 7f8e26a01b80 1 -- start start 2024-05-10T11:35:34.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.252+0000 7f8e26a01b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561dd7f2a4d0 0x561dd7f3ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:34.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.252+0000 7f8e26a01b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dd7f2adf0 0x561dd7f33f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:34.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.252+0000 7f8e26a01b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561dd7f2ec80 0x561dd7f2c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:34.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.252+0000 7f8e26a01b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8e141b7df0 con 0x561dd7f2adf0 2024-05-10T11:35:34.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.252+0000 7f8e26a01b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8e141b5680 con 0x561dd7f2a4d0 2024-05-10T11:35:34.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.252+0000 7f8e26a01b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8e141b34d0 con 0x561dd7f2ec80 2024-05-10T11:35:34.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.252+0000 7f8e19709700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dd7f2adf0 0x561dd7f33f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:34.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.253+0000 7f8e19709700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dd7f2adf0 0x561dd7f33f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57486/0 (socket says 172.21.15.100:57486) 2024-05-10T11:35:34.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.253+0000 7f8e19709700 1 -- 172.21.15.100:0/2686482276 learned_addr learned my addr 172.21.15.100:0/2686482276 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:35:34.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.253+0000 7f8e19f0a700 1 --2- 172.21.15.100:0/2686482276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561dd7f2a4d0 0x561dd7f3ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:34.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.253+0000 7f8e1a70b700 1 --2- 172.21.15.100:0/2686482276 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561dd7f2ec80 0x561dd7f2c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:34.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.253+0000 7f8e19709700 1 -- 172.21.15.100:0/2686482276 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561dd7f2ec80 msgr2=0x561dd7f2c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:34.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.253+0000 7f8e19709700 1 --2- 172.21.15.100:0/2686482276 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561dd7f2ec80 0x561dd7f2c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:34.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.253+0000 7f8e19709700 1 -- 172.21.15.100:0/2686482276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561dd7f2a4d0 msgr2=0x561dd7f3ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:34.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.253+0000 7f8e19709700 1 --2- 172.21.15.100:0/2686482276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561dd7f2a4d0 0x561dd7f3ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:34.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.253+0000 7f8e19709700 1 -- 172.21.15.100:0/2686482276 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561dd7f21810 con 0x561dd7f2adf0 2024-05-10T11:35:34.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.253+0000 7f8e19709700 1 --2- 172.21.15.100:0/2686482276 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dd7f2adf0 0x561dd7f33f40 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f8e04011b20 tx=0x7f8e04014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f615d99650c034e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:34.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.253+0000 7f8dff7fe700 1 -- 172.21.15.100:0/2686482276 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e04004a30 con 0x561dd7f2adf0 2024-05-10T11:35:34.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.253+0000 7f8dff7fe700 1 -- 172.21.15.100:0/2686482276 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8e0402a900 con 0x561dd7f2adf0 2024-05-10T11:35:34.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.253+0000 7f8dff7fe700 1 -- 172.21.15.100:0/2686482276 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e040054a0 con 0x561dd7f2adf0 2024-05-10T11:35:34.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.254+0000 7f8e26a01b80 1 -- 172.21.15.100:0/2686482276 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dd7f2adf0 msgr2=0x561dd7f33f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:34.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.254+0000 7f8e26a01b80 1 --2- 172.21.15.100:0/2686482276 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dd7f2adf0 0x561dd7f33f40 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f8e04011b20 tx=0x7f8e04014000 comp rx=0 tx=0).stop 2024-05-10T11:35:34.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.254+0000 7f8e26a01b80 1 -- 172.21.15.100:0/2686482276 shutdown_connections 2024-05-10T11:35:34.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.254+0000 7f8e26a01b80 1 --2- 172.21.15.100:0/2686482276 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561dd7f2ec80 0x561dd7f2c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:34.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.254+0000 7f8e26a01b80 1 --2- 172.21.15.100:0/2686482276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561dd7f2a4d0 0x561dd7f3ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:34.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.254+0000 7f8e26a01b80 1 --2- 172.21.15.100:0/2686482276 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dd7f2adf0 0x561dd7f33f40 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:34.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.254+0000 7f8e26a01b80 1 -- 172.21.15.100:0/2686482276 >> 172.21.15.100:0/2686482276 conn(0x561dd7e56100 msgr2=0x561dd7f1f720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:34.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.255+0000 7f8e26a01b80 1 -- 172.21.15.100:0/2686482276 shutdown_connections 2024-05-10T11:35:34.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.255+0000 7f8e26a01b80 1 -- 172.21.15.100:0/2686482276 wait complete. 2024-05-10T11:35:34.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.255+0000 7f8e26a01b80 1 -- start start 2024-05-10T11:35:34.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.255+0000 7f8e26a01b80 1 Processor -- start 2024-05-10T11:35:34.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.256+0000 7f8e26a01b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dd7f2a4d0 0x561dd7f3c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:34.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.256+0000 7f8e26a01b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561dd7f2adf0 0x561dd7f37c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:34.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.256+0000 7f8e26a01b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561dd7f2ec80 0x561dd7f381c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:34.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.256+0000 7f8e26a01b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8e141b3b40 con 0x561dd7f2a4d0 2024-05-10T11:35:34.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.256+0000 7f8e26a01b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8e141b4fb0 con 0x561dd7f2ec80 2024-05-10T11:35:34.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.256+0000 7f8e19709700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561dd7f2adf0 0x561dd7f37c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:34.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.256+0000 7f8e19f0a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dd7f2a4d0 0x561dd7f3c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:34.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.256+0000 7f8e26a01b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8e141b4210 con 0x561dd7f2adf0 2024-05-10T11:35:34.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.256+0000 7f8e19f0a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dd7f2a4d0 0x561dd7f3c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57492/0 (socket says 172.21.15.100:57492) 2024-05-10T11:35:34.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.256+0000 7f8e19f0a700 1 -- 172.21.15.100:0/2993906488 learned_addr learned my addr 172.21.15.100:0/2993906488 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:35:34.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.256+0000 7f8e1a70b700 1 --2- 172.21.15.100:0/2993906488 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561dd7f2ec80 0x561dd7f381c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:34.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.256+0000 7f8e19709700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561dd7f2adf0 0x561dd7f37c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:57612/0 (socket says 172.21.15.100:57612) 2024-05-10T11:35:34.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.256+0000 7f8e26a01b80 10 client.0 _ll_register_callbacks cb 0x561dd7e14920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:35:34.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.256+0000 7f8e1a70b700 1 -- 172.21.15.100:0/2993906488 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561dd7f2adf0 msgr2=0x561dd7f37c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:34.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.256+0000 7f8e1a70b700 1 --2- 172.21.15.100:0/2993906488 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561dd7f2adf0 0x561dd7f37c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:34.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.256+0000 7f8e1a70b700 1 -- 172.21.15.100:0/2993906488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dd7f2a4d0 msgr2=0x561dd7f3c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:34.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.256+0000 7f8e1a70b700 1 --2- 172.21.15.100:0/2993906488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dd7f2a4d0 0x561dd7f3c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:34.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.256+0000 7f8e1a70b700 1 -- 172.21.15.100:0/2993906488 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561dd7f38a00 con 0x561dd7f2ec80 2024-05-10T11:35:34.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.257+0000 7f8e1a70b700 1 --2- 172.21.15.100:0/2993906488 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561dd7f2ec80 0x561dd7f381c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c003f00 tx=0x7f8e0c008cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a10c3c17504bd608 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:34.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.257+0000 7f8dfd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:35:34.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.257+0000 7f8dfd7fa700 1 -- 172.21.15.100:0/2993906488 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c015480 con 0x561dd7f2ec80 2024-05-10T11:35:34.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.257+0000 7f8dfd7fa700 1 -- 172.21.15.100:0/2993906488 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f8e0c015620 con 0x561dd7f2ec80 2024-05-10T11:35:34.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.257+0000 7f8e26a01b80 1 -- 172.21.15.100:0/2993906488 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e0402a900 con 0x561dd7f2ec80 2024-05-10T11:35:34.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.257+0000 7f8dfd7fa700 1 -- 172.21.15.100:0/2993906488 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f8e0c032050 con 0x561dd7f2ec80 2024-05-10T11:35:34.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.257+0000 7f8e26a01b80 10 client.28124 fetch_fsmap learned FSMap version 41 2024-05-10T11:35:34.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.257+0000 7f8e26a01b80 1 -- 172.21.15.100:0/2993906488 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x561dd7fae340 con 0x561dd7f2ec80 2024-05-10T11:35:34.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.257+0000 7f8dfd7fa700 1 -- 172.21.15.100:0/2993906488 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8e0c015cf0 con 0x561dd7f2ec80 2024-05-10T11:35:34.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.257+0000 7f8dfd7fa700 1 -- 172.21.15.100:0/2993906488 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f8e0c0049e0 con 0x561dd7f2ec80 2024-05-10T11:35:34.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.257+0000 7f8e26a01b80 10 client.28124 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:35:34.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.257+0000 7f8e26a01b80 10 client.28124 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-10T11:35:34.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.257+0000 7f8e26a01b80 20 client.28124 populate_metadata read hostname 'smithi100' 2024-05-10T11:35:34.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.258+0000 7f8e26a01b80 1 --2- 172.21.15.100:0/2993906488 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x561dd7fae720 0x561dd7fb0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:34.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.258+0000 7f8e26a01b80 4 client.28124 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:35:34.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.258+0000 7f8e26a01b80 1 -- 172.21.15.100:0/2993906488 --> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f8e0402a900 con 0x561dd7fae720 2024-05-10T11:35:34.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.258+0000 7f8e19f0a700 1 --2- 172.21.15.100:0/2993906488 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x561dd7fae720 0x561dd7fb0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:34.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.258+0000 7f8dfd7fa700 10 client.28124 ms_handle_connect on v2:172.21.15.188:6836/1943342620 2024-05-10T11:35:34.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.258+0000 7f8e19f0a700 1 --2- 172.21.15.100:0/2993906488 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x561dd7fae720 0x561dd7fb0b10 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=28d23e7262bb51c5 server_cookie=5b03e1dbcbb4418d in_seq=0 out_seq=0 2024-05-10T11:35:34.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.258+0000 7f8dfd7fa700 1 -- 172.21.15.100:0/2993906488 <== mds.2 v2:172.21.15.188:6836/1943342620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f8e0402a900 con 0x561dd7fae720 2024-05-10T11:35:34.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.258+0000 7f8dfd7fa700 10 client.28124 handle_command_reply: tid=0 2024-05-10T11:35:34.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.258+0000 7f8e26a01b80 1 client.28124 shutdown 2024-05-10T11:35:34.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.259+0000 7f8e26a01b80 1 -- 172.21.15.100:0/2993906488 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561dd7f2ec80 msgr2=0x561dd7f381c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:34.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.259+0000 7f8e26a01b80 1 --2- 172.21.15.100:0/2993906488 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561dd7f2ec80 0x561dd7f381c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c003f00 tx=0x7f8e0c008cf0 comp rx=0 tx=0).stop 2024-05-10T11:35:34.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.259+0000 7f8e26a01b80 1 -- 172.21.15.100:0/2993906488 shutdown_connections 2024-05-10T11:35:34.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.259+0000 7f8e26a01b80 1 --2- 172.21.15.100:0/2993906488 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x561dd7fae720 0x561dd7fb0b10 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:34.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.259+0000 7f8dfd7fa700 0 client.28124 ms_handle_reset on v2:172.21.15.188:6836/1943342620 2024-05-10T11:35:34.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.259+0000 7f8e26a01b80 1 --2- 172.21.15.100:0/2993906488 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561dd7f2ec80 0x561dd7f381c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:34.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.259+0000 7f8e26a01b80 1 --2- 172.21.15.100:0/2993906488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dd7f2a4d0 0x561dd7f3c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:34.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.259+0000 7f8e26a01b80 1 --2- 172.21.15.100:0/2993906488 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561dd7f2adf0 0x561dd7f37c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:34.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.259+0000 7f8e26a01b80 1 -- 172.21.15.100:0/2993906488 >> 172.21.15.100:0/2993906488 conn(0x561dd7e56100 msgr2=0x561dd7f2d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:34.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.259+0000 7f8e26a01b80 1 -- 172.21.15.100:0/2993906488 shutdown_connections 2024-05-10T11:35:34.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.259+0000 7f8e26a01b80 1 -- 172.21.15.100:0/2993906488 wait complete. 2024-05-10T11:35:34.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.259+0000 7f8e26a01b80 20 client.28124 trim_cache size 0 max 16384 2024-05-10T11:35:34.263 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:35:34.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.260+0000 7f8e1c16e700 1 -- 172.21.15.100:0/106357992 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8e000b2980 msgr2=0x7f8e000b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:34.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.260+0000 7f8e1c16e700 1 --2- 172.21.15.100:0/106357992 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8e000b2980 0x7f8e000b4e40 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f8e14079170 tx=0x7f8e04009260 comp rx=0 tx=0).stop 2024-05-10T11:35:34.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.260+0000 7f8e1c16e700 1 -- 172.21.15.100:0/106357992 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e141056c0 msgr2=0x7f8e140779b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:34.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.260+0000 7f8e1c16e700 1 --2- 172.21.15.100:0/106357992 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e141056c0 0x7f8e140779b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8e10000c00 tx=0x7f8e1000bfc0 comp rx=0 tx=0).stop 2024-05-10T11:35:34.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.260+0000 7f8e1c16e700 1 -- 172.21.15.100:0/106357992 shutdown_connections 2024-05-10T11:35:34.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.260+0000 7f8e1c16e700 1 --2- 172.21.15.100:0/106357992 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8e000b2980 0x7f8e000b4e40 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:34.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.260+0000 7f8e1c16e700 1 --2- 172.21.15.100:0/106357992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e1410b6c0 0x7f8e14078430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:34.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.260+0000 7f8e1c16e700 1 --2- 172.21.15.100:0/106357992 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e141056c0 0x7f8e140779b0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:34.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.260+0000 7f8e1c16e700 1 --2- 172.21.15.100:0/106357992 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e14106080 0x7f8e14077ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:34.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.260+0000 7f8e1c16e700 1 -- 172.21.15.100:0/106357992 >> 172.21.15.100:0/106357992 conn(0x7f8e141011b0 msgr2=0x7f8e14101a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:34.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.260+0000 7f8e1c16e700 1 -- 172.21.15.100:0/106357992 shutdown_connections 2024-05-10T11:35:34.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:34.260+0000 7f8e1c16e700 1 -- 172.21.15.100:0/106357992 wait complete. 2024-05-10T11:35:34.919 INFO:tasks.workunit.client.0.smithi100.stdout: 10 35116 9.08 MB/sec execute 88 sec latency 132.322 ms 2024-05-10T11:35:35.050 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-10T11:35:35.417 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:35:35.732 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.731+0000 7fbc8b854640 1 -- 172.21.15.100:0/2158281761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc84108790 msgr2=0x7fbc8410ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:35.732 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.731+0000 7fbc8b854640 1 --2- 172.21.15.100:0/2158281761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc84108790 0x7fbc8410ab80 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c00df70 tx=0x7fbc7c02e630 comp rx=0 tx=0).stop 2024-05-10T11:35:35.732 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.732+0000 7fbc8b854640 1 -- 172.21.15.100:0/2158281761 shutdown_connections 2024-05-10T11:35:35.732 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.732+0000 7fbc8b854640 1 --2- 172.21.15.100:0/2158281761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc84108790 0x7fbc8410ab80 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:35.732 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.732+0000 7fbc8b854640 1 --2- 172.21.15.100:0/2158281761 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc84105e60 0x7fbc84108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:35.732 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.732+0000 7fbc8b854640 1 --2- 172.21.15.100:0/2158281761 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc84103530 0x7fbc84105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:35.732 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.732+0000 7fbc8b854640 1 -- 172.21.15.100:0/2158281761 >> 172.21.15.100:0/2158281761 conn(0x7fbc840fce60 msgr2=0x7fbc840ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:35.732 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.732+0000 7fbc8b854640 1 -- 172.21.15.100:0/2158281761 shutdown_connections 2024-05-10T11:35:35.732 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.732+0000 7fbc8b854640 1 -- 172.21.15.100:0/2158281761 wait complete. 2024-05-10T11:35:35.734 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.733+0000 7fbc8b854640 1 Processor -- start 2024-05-10T11:35:35.734 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.733+0000 7fbc8b854640 1 -- start start 2024-05-10T11:35:35.735 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.733+0000 7fbc8b854640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc84103530 0x7fbc841968d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:35.735 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.733+0000 7fbc8b854640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc84105e60 0x7fbc84196e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:35.735 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.733+0000 7fbc8b854640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc84108790 0x7fbc8419de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:35.735 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.733+0000 7fbc8b854640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8410c210 con 0x7fbc84103530 2024-05-10T11:35:35.735 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.733+0000 7fbc8b854640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8410c090 con 0x7fbc84108790 2024-05-10T11:35:35.735 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.733+0000 7fbc8b854640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8410c390 con 0x7fbc84105e60 2024-05-10T11:35:35.735 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.733+0000 7fbc895f0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc84103530 0x7fbc841968d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:35.735 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.733+0000 7fbc895f0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc84103530 0x7fbc841968d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57520/0 (socket says 172.21.15.100:57520) 2024-05-10T11:35:35.735 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.733+0000 7fbc895f0640 1 -- 172.21.15.100:0/1441866037 learned_addr learned my addr 172.21.15.100:0/1441866037 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:35:35.735 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.733+0000 7fbc89df1640 1 --2- 172.21.15.100:0/1441866037 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc84108790 0x7fbc8419de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:35.735 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.734+0000 7fbc895f0640 1 -- 172.21.15.100:0/1441866037 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc84105e60 msgr2=0x7fbc84196e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:35.735 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.734+0000 7fbc895f0640 1 --2- 172.21.15.100:0/1441866037 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc84105e60 0x7fbc84196e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:35.735 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.734+0000 7fbc895f0640 1 -- 172.21.15.100:0/1441866037 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc84108790 msgr2=0x7fbc8419de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:35.735 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.734+0000 7fbc895f0640 1 --2- 172.21.15.100:0/1441866037 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc84108790 0x7fbc8419de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:35.736 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.734+0000 7fbc895f0640 1 -- 172.21.15.100:0/1441866037 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc8419e4f0 con 0x7fbc84103530 2024-05-10T11:35:35.736 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.734+0000 7fbc895f0640 1 --2- 172.21.15.100:0/1441866037 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc84103530 0x7fbc841968d0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fbc8000e9c0 tx=0x7fbc8000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:35.736 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.734+0000 7fbc7a7fc640 1 -- 172.21.15.100:0/1441866037 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc8000ccc0 con 0x7fbc84103530 2024-05-10T11:35:35.736 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.734+0000 7fbc7a7fc640 1 -- 172.21.15.100:0/1441866037 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbc800054b0 con 0x7fbc84103530 2024-05-10T11:35:35.736 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.734+0000 7fbc7a7fc640 1 -- 172.21.15.100:0/1441866037 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc80010430 con 0x7fbc84103530 2024-05-10T11:35:35.736 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.734+0000 7fbc8b854640 1 -- 172.21.15.100:0/1441866037 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc8419e7e0 con 0x7fbc84103530 2024-05-10T11:35:35.736 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.735+0000 7fbc8b854640 1 -- 172.21.15.100:0/1441866037 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc8419ece0 con 0x7fbc84103530 2024-05-10T11:35:35.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.738+0000 7fbc7a7fc640 1 -- 172.21.15.100:0/1441866037 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fbc800046c0 con 0x7fbc84103530 2024-05-10T11:35:35.740 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.738+0000 7fbc8e569740 1 Processor -- start 2024-05-10T11:35:35.740 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.738+0000 7fbc8e569740 1 -- start start 2024-05-10T11:35:35.740 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.738+0000 7fbc8e569740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585458025c0 0x5585458029e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:35.740 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.738+0000 7fbc8e569740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558545802fb0 0x558545807ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:35.740 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.738+0000 7fbc8e569740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558545808120 0x55854580a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:35.740 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.738+0000 7fbc8e569740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55854575e5d0 con 0x5585458025c0 2024-05-10T11:35:35.740 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.739+0000 7fbc8e569740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558545717890 con 0x558545808120 2024-05-10T11:35:35.740 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.739+0000 7fbc8e569740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55854575a590 con 0x558545802fb0 2024-05-10T11:35:35.740 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.739+0000 7fbc89df1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585458025c0 0x5585458029e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:35.740 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.739+0000 7fbc89df1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585458025c0 0x5585458029e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57530/0 (socket says 172.21.15.100:57530) 2024-05-10T11:35:35.740 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.739+0000 7fbc89df1640 1 -- 172.21.15.100:0/1438179701 learned_addr learned my addr 172.21.15.100:0/1438179701 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:35:35.740 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.739+0000 7fbc895f0640 1 --2- 172.21.15.100:0/1438179701 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558545802fb0 0x558545807ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:35.740 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.739+0000 7fbc88def640 1 --2- 172.21.15.100:0/1438179701 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558545808120 0x55854580a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:35.740 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.739+0000 7fbc88def640 1 -- 172.21.15.100:0/1438179701 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558545802fb0 msgr2=0x558545807ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:35.741 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.739+0000 7fbc88def640 1 --2- 172.21.15.100:0/1438179701 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558545802fb0 0x558545807ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:35.741 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.739+0000 7fbc88def640 1 -- 172.21.15.100:0/1438179701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585458025c0 msgr2=0x5585458029e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:35.741 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.739+0000 7fbc88def640 1 --2- 172.21.15.100:0/1438179701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585458025c0 0x5585458029e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:35.741 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.739+0000 7fbc88def640 1 -- 172.21.15.100:0/1438179701 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558545470210 con 0x558545808120 2024-05-10T11:35:35.741 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.739+0000 7fbc89df1640 1 --2- 172.21.15.100:0/1438179701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585458025c0 0x5585458029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:35:35.741 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.739+0000 7fbc88def640 1 --2- 172.21.15.100:0/1438179701 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558545808120 0x55854580a5b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbc8406b710 tx=0x7fbc7400e350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=144b850a82e92b01 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:35.741 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.739+0000 7fbc7a7fc640 1 --2- 172.21.15.100:0/1441866037 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fbc700b2970 0x7fbc700b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:35.741 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.739+0000 7fbc6f7fe640 1 -- 172.21.15.100:0/1438179701 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc7400ed20 con 0x558545808120 2024-05-10T11:35:35.741 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.740+0000 7fbc6f7fe640 1 -- 172.21.15.100:0/1438179701 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbc740044c0 con 0x558545808120 2024-05-10T11:35:35.741 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.740+0000 7fbc89df1640 1 --2- 172.21.15.100:0/1441866037 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fbc700b2970 0x7fbc700b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:35.741 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.740+0000 7fbc6f7fe640 1 -- 172.21.15.100:0/1438179701 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc74004a00 con 0x558545808120 2024-05-10T11:35:35.741 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.740+0000 7fbc7a7fc640 1 -- 172.21.15.100:0/1441866037 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fbc800e29a0 con 0x7fbc84103530 2024-05-10T11:35:35.744 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.740+0000 7fbc8e569740 1 -- 172.21.15.100:0/1438179701 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558545808120 msgr2=0x55854580a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:35.744 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.740+0000 7fbc89df1640 1 --2- 172.21.15.100:0/1441866037 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fbc700b2970 0x7fbc700b4e30 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x558545803810 tx=0x7fbc7c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:35.744 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.740+0000 7fbc8e569740 1 --2- 172.21.15.100:0/1438179701 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558545808120 0x55854580a5b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbc8406b710 tx=0x7fbc7400e350 comp rx=0 tx=0).stop 2024-05-10T11:35:35.744 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.741+0000 7fbc8e569740 1 -- 172.21.15.100:0/1438179701 shutdown_connections 2024-05-10T11:35:35.744 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.741+0000 7fbc8e569740 1 --2- 172.21.15.100:0/1438179701 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558545808120 0x55854580a5b0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:35.744 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.741+0000 7fbc8e569740 1 --2- 172.21.15.100:0/1438179701 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558545802fb0 0x558545807ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:35.744 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.741+0000 7fbc8e569740 1 --2- 172.21.15.100:0/1438179701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585458025c0 0x5585458029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:35.744 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.741+0000 7fbc8e569740 1 -- 172.21.15.100:0/1438179701 >> 172.21.15.100:0/1438179701 conn(0x5585457fdc30 msgr2=0x5585457fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:35.744 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.741+0000 7fbc8e569740 1 -- 172.21.15.100:0/1438179701 shutdown_connections 2024-05-10T11:35:35.745 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.741+0000 7fbc8e569740 1 -- 172.21.15.100:0/1438179701 wait complete. 2024-05-10T11:35:35.745 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.742+0000 7fbc8e569740 1 -- start start 2024-05-10T11:35:35.745 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.742+0000 7fbc8e569740 1 Processor -- start 2024-05-10T11:35:35.745 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.742+0000 7fbc8e569740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585458025c0 0x55854580c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:35.745 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.742+0000 7fbc895f0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585458025c0 0x55854580c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:35.745 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.742+0000 7fbc895f0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585458025c0 0x55854580c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57534/0 (socket says 172.21.15.100:57534) 2024-05-10T11:35:35.745 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.742+0000 7fbc895f0640 1 -- 172.21.15.100:0/2178368164 learned_addr learned my addr 172.21.15.100:0/2178368164 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:35:35.745 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.742+0000 7fbc8e569740 1 --2- 172.21.15.100:0/2178368164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558545802fb0 0x55854580cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:35.745 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.742+0000 7fbc8e569740 1 --2- 172.21.15.100:0/2178368164 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558545808120 0x558545813da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:35.745 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.743+0000 7fbc8e569740 1 -- 172.21.15.100:0/2178368164 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55854580b120 con 0x5585458025c0 2024-05-10T11:35:35.745 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.743+0000 7fbc88def640 1 --2- 172.21.15.100:0/2178368164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558545802fb0 0x55854580cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:35.745 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.743+0000 7fbc89df1640 1 --2- 172.21.15.100:0/2178368164 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558545808120 0x558545813da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:35.745 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.743+0000 7fbc8e569740 1 -- 172.21.15.100:0/2178368164 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55854571b370 con 0x558545802fb0 2024-05-10T11:35:35.745 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.743+0000 7fbc8e569740 1 -- 172.21.15.100:0/2178368164 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55854580b2a0 con 0x558545808120 2024-05-10T11:35:35.745 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.743+0000 7fbc8e569740 10 client.0 _ll_register_callbacks cb 0x55854546fa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:35:35.746 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.743+0000 7fbc88def640 1 -- 172.21.15.100:0/2178368164 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558545808120 msgr2=0x558545813da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:35.746 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.743+0000 7fbc88def640 1 --2- 172.21.15.100:0/2178368164 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558545808120 0x558545813da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:35.746 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.743+0000 7fbc88def640 1 -- 172.21.15.100:0/2178368164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585458025c0 msgr2=0x55854580c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:35.746 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.743+0000 7fbc88def640 1 --2- 172.21.15.100:0/2178368164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585458025c0 0x55854580c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:35.746 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.743+0000 7fbc88def640 1 -- 172.21.15.100:0/2178368164 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55854553c620 con 0x558545802fb0 2024-05-10T11:35:35.746 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.743+0000 7fbc8e569740 1 -- 172.21.15.100:0/2178368164 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55854576c1e0 con 0x558545802fb0 2024-05-10T11:35:35.746 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.743+0000 7fbc895f0640 1 --2- 172.21.15.100:0/2178368164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585458025c0 0x55854580c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:35:35.746 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.744+0000 7fbc88def640 1 --2- 172.21.15.100:0/2178368164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558545802fb0 0x55854580cd60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbc74006010 tx=0x7fbc74009870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d3971fab7f07e50 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:35.746 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.744+0000 7fbc6cff9640 10 client.28136 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:35:35.746 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.744+0000 7fbc6cff9640 1 -- 172.21.15.100:0/2178368164 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc74004bb0 con 0x558545802fb0 2024-05-10T11:35:35.746 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.744+0000 7fbc6cff9640 1 -- 172.21.15.100:0/2178368164 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fbc74004d50 con 0x558545802fb0 2024-05-10T11:35:35.746 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.744+0000 7fbc6cff9640 1 -- 172.21.15.100:0/2178368164 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fbc74018470 con 0x558545802fb0 2024-05-10T11:35:35.746 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.744+0000 7fbc8e569740 10 client.28136 fetch_fsmap learned FSMap version 41 2024-05-10T11:35:35.746 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.744+0000 7fbc8e569740 1 -- 172.21.15.100:0/2178368164 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({fsmap=41}) -- 0x558545816ac0 con 0x558545802fb0 2024-05-10T11:35:35.748 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.745+0000 7fbc6cff9640 1 -- 172.21.15.100:0/2178368164 <== mon.2 v2:172.21.15.178:3300/0 4 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fbc74018610 con 0x558545802fb0 2024-05-10T11:35:35.748 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.745+0000 7fbc8e569740 10 client.28136 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:35:35.749 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.745+0000 7fbc8e569740 10 client.28136 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:35:35.749 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.745+0000 7fbc8e569740 20 client.28136 populate_metadata read hostname 'smithi100' 2024-05-10T11:35:35.749 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.745+0000 7fbc8e569740 1 --2- 172.21.15.100:0/2178368164 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558545817e10 0x55854581a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:35.749 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.745+0000 7fbc8e569740 4 client.28136 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:35:35.749 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.745+0000 7fbc8e569740 1 -- 172.21.15.100:0/2178368164 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558545766050 con 0x558545817e10 2024-05-10T11:35:35.749 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.745+0000 7fbc895f0640 1 --2- 172.21.15.100:0/2178368164 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558545817e10 0x55854581a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:35.749 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.745+0000 7fbc895f0640 1 --2- 172.21.15.100:0/2178368164 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558545817e10 0x55854581a200 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a4680c2ec46c709 server_cookie=c83da75dee7ad90f in_seq=0 out_seq=0 2024-05-10T11:35:35.749 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.745+0000 7fbc6cff9640 10 client.28136 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:35:35.749 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.746+0000 7fbc6cff9640 1 -- 172.21.15.100:0/2178368164 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558545766050 con 0x558545817e10 2024-05-10T11:35:35.749 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.746+0000 7fbc6cff9640 10 client.28136 handle_command_reply: tid=0 2024-05-10T11:35:35.749 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.746+0000 7fbc8e569740 1 client.28136 shutdown 2024-05-10T11:35:35.749 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.746+0000 7fbc8e569740 1 -- 172.21.15.100:0/2178368164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558545802fb0 msgr2=0x55854580cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:35.749 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.746+0000 7fbc8e569740 1 --2- 172.21.15.100:0/2178368164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558545802fb0 0x55854580cd60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbc74006010 tx=0x7fbc74009870 comp rx=0 tx=0).stop 2024-05-10T11:35:35.749 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.746+0000 7fbc8e569740 1 -- 172.21.15.100:0/2178368164 shutdown_connections 2024-05-10T11:35:35.749 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.746+0000 7fbc8e569740 1 --2- 172.21.15.100:0/2178368164 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558545817e10 0x55854581a200 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:35.750 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.746+0000 7fbc8e569740 1 --2- 172.21.15.100:0/2178368164 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558545808120 0x558545813da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:35.750 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.746+0000 7fbc8e569740 1 --2- 172.21.15.100:0/2178368164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558545802fb0 0x55854580cd60 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:35.750 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.746+0000 7fbc6cff9640 0 client.28136 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:35:35.750 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.746+0000 7fbc8e569740 1 --2- 172.21.15.100:0/2178368164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585458025c0 0x55854580c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:35.750 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.746+0000 7fbc8e569740 1 -- 172.21.15.100:0/2178368164 >> 172.21.15.100:0/2178368164 conn(0x5585457fdc30 msgr2=0x5585457f4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:35.750 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.747+0000 7fbc8e569740 1 -- 172.21.15.100:0/2178368164 shutdown_connections 2024-05-10T11:35:35.750 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.747+0000 7fbc8e569740 1 -- 172.21.15.100:0/2178368164 wait complete. 2024-05-10T11:35:35.750 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.747+0000 7fbc8e569740 20 client.28136 trim_cache size 0 max 16384 2024-05-10T11:35:35.769 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.768+0000 7fbc8e569740 1 Processor -- start 2024-05-10T11:35:35.769 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.768+0000 7fbc8e569740 1 -- start start 2024-05-10T11:35:35.769 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.768+0000 7fbc8e569740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5585458025c0 0x55854580baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:35.769 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.768+0000 7fbc8e569740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558545802fb0 0x55854580c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:35.769 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.768+0000 7fbc8e569740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558545808120 0x55854581e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:35.769 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.768+0000 7fbc8e569740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbc841aba80 con 0x558545808120 2024-05-10T11:35:35.770 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.768+0000 7fbc8e569740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbc841a9350 con 0x558545802fb0 2024-05-10T11:35:35.770 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.768+0000 7fbc8e569740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbc841a71a0 con 0x5585458025c0 2024-05-10T11:35:35.770 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.768+0000 7fbc89df1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558545808120 0x55854581e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:35.770 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.769+0000 7fbc88def640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558545802fb0 0x55854580c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:35.770 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.769+0000 7fbc89df1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558545808120 0x55854581e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57544/0 (socket says 172.21.15.100:57544) 2024-05-10T11:35:35.770 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.769+0000 7fbc89df1640 1 -- 172.21.15.100:0/2882262043 learned_addr learned my addr 172.21.15.100:0/2882262043 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:35:35.770 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.769+0000 7fbc895f0640 1 --2- 172.21.15.100:0/2882262043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5585458025c0 0x55854580baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:35.770 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.769+0000 7fbc88def640 1 -- 172.21.15.100:0/2882262043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5585458025c0 msgr2=0x55854580baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:35.770 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.769+0000 7fbc88def640 1 --2- 172.21.15.100:0/2882262043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5585458025c0 0x55854580baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:35.770 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.769+0000 7fbc88def640 1 -- 172.21.15.100:0/2882262043 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558545808120 msgr2=0x55854581e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:35.770 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.769+0000 7fbc88def640 1 --2- 172.21.15.100:0/2882262043 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558545808120 0x55854581e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:35.770 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.769+0000 7fbc88def640 1 -- 172.21.15.100:0/2882262043 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558545803390 con 0x558545802fb0 2024-05-10T11:35:35.770 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.769+0000 7fbc89df1640 1 --2- 172.21.15.100:0/2882262043 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558545808120 0x55854581e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:35:35.770 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.769+0000 7fbc88def640 1 --2- 172.21.15.100:0/2882262043 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558545802fb0 0x55854580c030 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x5585457f1020 tx=0x7fbc7401e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fcc79fb238705686 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:35.771 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.769+0000 7fbc6effd640 1 -- 172.21.15.100:0/2882262043 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc74005a70 con 0x558545802fb0 2024-05-10T11:35:35.771 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.769+0000 7fbc6effd640 1 -- 172.21.15.100:0/2882262043 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55854576c1e0 con 0x558545802fb0 2024-05-10T11:35:35.771 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.769+0000 7fbc6effd640 1 -- 172.21.15.100:0/2882262043 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc74037440 con 0x558545802fb0 2024-05-10T11:35:35.774 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.770+0000 7fbc8e569740 1 -- 172.21.15.100:0/2882262043 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558545802fb0 msgr2=0x55854580c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:35.774 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.770+0000 7fbc8e569740 1 --2- 172.21.15.100:0/2882262043 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558545802fb0 0x55854580c030 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x5585457f1020 tx=0x7fbc7401e000 comp rx=0 tx=0).stop 2024-05-10T11:35:35.774 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.770+0000 7fbc8e569740 1 -- 172.21.15.100:0/2882262043 shutdown_connections 2024-05-10T11:35:35.774 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.770+0000 7fbc8e569740 1 --2- 172.21.15.100:0/2882262043 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558545808120 0x55854581e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:35.774 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.770+0000 7fbc8e569740 1 --2- 172.21.15.100:0/2882262043 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558545802fb0 0x55854580c030 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:35.774 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.770+0000 7fbc8e569740 1 --2- 172.21.15.100:0/2882262043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5585458025c0 0x55854580baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:35.774 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.770+0000 7fbc8e569740 1 -- 172.21.15.100:0/2882262043 >> 172.21.15.100:0/2882262043 conn(0x5585457fdc30 msgr2=0x558545800c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:35.774 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.770+0000 7fbc8e569740 1 -- 172.21.15.100:0/2882262043 shutdown_connections 2024-05-10T11:35:35.774 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.771+0000 7fbc8e569740 1 -- 172.21.15.100:0/2882262043 wait complete. 2024-05-10T11:35:35.775 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.771+0000 7fbc8e569740 1 -- start start 2024-05-10T11:35:35.775 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.771+0000 7fbc8e569740 1 Processor -- start 2024-05-10T11:35:35.775 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.771+0000 7fbc8e569740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5585458025c0 0x558545810f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:35.775 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.771+0000 7fbc8e569740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558545802fb0 0x558545811480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:35.775 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.771+0000 7fbc8e569740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558545808120 0x558545812490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:35.775 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.771+0000 7fbc8e569740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbc841a7810 con 0x558545802fb0 2024-05-10T11:35:35.775 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.771+0000 7fbc8e569740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbc841a8c80 con 0x5585458025c0 2024-05-10T11:35:35.775 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.771+0000 7fbc8e569740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbc841a7ee0 con 0x558545808120 2024-05-10T11:35:35.775 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.772+0000 7fbc8e569740 10 client.0 _ll_register_callbacks cb 0x7fbc80005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:35:35.775 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.772+0000 7fbc895f0640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5585458025c0 0x558545810f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:35.775 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.772+0000 7fbc895f0640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5585458025c0 0x558545810f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:36068/0 (socket says 172.21.15.100:36068) 2024-05-10T11:35:35.775 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.772+0000 7fbc895f0640 1 -- 172.21.15.100:0/2973650144 learned_addr learned my addr 172.21.15.100:0/2973650144 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:35:35.775 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.772+0000 7fbc88def640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558545802fb0 0x558545811480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:35.775 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.772+0000 7fbc89df1640 1 --2- 172.21.15.100:0/2973650144 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558545808120 0x558545812490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:35.775 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.772+0000 7fbc88def640 1 -- 172.21.15.100:0/2973650144 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558545808120 msgr2=0x558545812490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:35.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.772+0000 7fbc88def640 1 --2- 172.21.15.100:0/2973650144 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558545808120 0x558545812490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:35.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.772+0000 7fbc88def640 1 -- 172.21.15.100:0/2973650144 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5585458025c0 msgr2=0x558545810f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:35.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.772+0000 7fbc88def640 1 --2- 172.21.15.100:0/2973650144 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5585458025c0 0x558545810f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:35.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.772+0000 7fbc88def640 1 -- 172.21.15.100:0/2973650144 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558545812a80 con 0x558545802fb0 2024-05-10T11:35:35.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.772+0000 7fbc895f0640 1 --2- 172.21.15.100:0/2973650144 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5585458025c0 0x558545810f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:35:35.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.772+0000 7fbc88def640 1 --2- 172.21.15.100:0/2973650144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558545802fb0 0x558545811480 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fbc7400e390 tx=0x7fbc740186a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa0ac1abc83a0d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:35.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.772+0000 7fbc6cff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:35:35.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.772+0000 7fbc6cff9640 1 -- 172.21.15.100:0/2973650144 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc74004bb0 con 0x558545802fb0 2024-05-10T11:35:35.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.772+0000 7fbc6cff9640 1 -- 172.21.15.100:0/2973650144 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fbc74004d50 con 0x558545802fb0 2024-05-10T11:35:35.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.772+0000 7fbc8e569740 1 -- 172.21.15.100:0/2973650144 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55854576c1e0 con 0x558545802fb0 2024-05-10T11:35:35.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.773+0000 7fbc6cff9640 1 -- 172.21.15.100:0/2973650144 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fbc7404a050 con 0x558545802fb0 2024-05-10T11:35:35.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.773+0000 7fbc8e569740 10 client.18831 fetch_fsmap learned FSMap version 41 2024-05-10T11:35:35.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.773+0000 7fbc8e569740 1 -- 172.21.15.100:0/2973650144 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x558545882950 con 0x558545802fb0 2024-05-10T11:35:35.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.773+0000 7fbc6cff9640 1 -- 172.21.15.100:0/2973650144 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55854576c1e0 con 0x558545802fb0 2024-05-10T11:35:35.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.773+0000 7fbc6cff9640 1 -- 172.21.15.100:0/2973650144 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fbc74035020 con 0x558545802fb0 2024-05-10T11:35:35.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.773+0000 7fbc8e569740 10 client.18831 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:35:35.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.773+0000 7fbc8e569740 10 client.18831 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:35:35.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.773+0000 7fbc8e569740 20 client.18831 populate_metadata read hostname 'smithi100' 2024-05-10T11:35:35.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.773+0000 7fbc8e569740 1 --2- 172.21.15.100:0/2973650144 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5585458832a0 0x5585458856c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:35.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.773+0000 7fbc8e569740 4 client.18831 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:35:35.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.773+0000 7fbc8e569740 1 -- 172.21.15.100:0/2973650144 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5585456e15f0 con 0x5585458832a0 2024-05-10T11:35:35.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.773+0000 7fbc895f0640 1 --2- 172.21.15.100:0/2973650144 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5585458832a0 0x5585458856c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:35.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.774+0000 7fbc6cff9640 10 client.18831 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:35:35.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.774+0000 7fbc895f0640 1 --2- 172.21.15.100:0/2973650144 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5585458832a0 0x5585458856c0 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9a023ad5a8f885f server_cookie=c7cbfafba2689ca1 in_seq=0 out_seq=0 2024-05-10T11:35:35.799 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:35:35 smithi100 ceph-mon[30296]: pgmap v937: 129 pgs: 129 active+clean; 450 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 4.6 MiB/s wr, 249 op/s 2024-05-10T11:35:35.854 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.853+0000 7fbc6cff9640 1 -- 172.21.15.100:0/2973650144 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5585456e15f0 con 0x5585458832a0 2024-05-10T11:35:35.854 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.853+0000 7fbc6cff9640 10 client.18831 handle_command_reply: tid=0 2024-05-10T11:35:35.854 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.853+0000 7fbc8e569740 1 client.18831 shutdown 2024-05-10T11:35:35.855 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.854+0000 7fbc8e569740 1 -- 172.21.15.100:0/2973650144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558545802fb0 msgr2=0x558545811480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:35.855 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.854+0000 7fbc8e569740 1 --2- 172.21.15.100:0/2973650144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558545802fb0 0x558545811480 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fbc7400e390 tx=0x7fbc740186a0 comp rx=0 tx=0).stop 2024-05-10T11:35:35.855 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.854+0000 7fbc8e569740 1 -- 172.21.15.100:0/2973650144 shutdown_connections 2024-05-10T11:35:35.855 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.854+0000 7fbc8e569740 1 --2- 172.21.15.100:0/2973650144 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5585458832a0 0x5585458856c0 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:35.855 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.854+0000 7fbc8e569740 1 --2- 172.21.15.100:0/2973650144 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558545808120 0x558545812490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:35.855 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.854+0000 7fbc8e569740 1 --2- 172.21.15.100:0/2973650144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558545802fb0 0x558545811480 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:35.855 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.854+0000 7fbc8e569740 1 --2- 172.21.15.100:0/2973650144 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5585458025c0 0x558545810f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:35.855 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.854+0000 7fbc8e569740 1 -- 172.21.15.100:0/2973650144 >> 172.21.15.100:0/2973650144 conn(0x5585457fdc30 msgr2=0x558545800c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:35.855 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.854+0000 7fbc6cff9640 0 client.18831 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:35:35.855 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.854+0000 7fbc8e569740 1 -- 172.21.15.100:0/2973650144 shutdown_connections 2024-05-10T11:35:35.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.854+0000 7fbc8e569740 1 -- 172.21.15.100:0/2973650144 wait complete. 2024-05-10T11:35:35.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.854+0000 7fbc8e569740 20 client.18831 trim_cache size 0 max 16384 2024-05-10T11:35:35.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:35:35.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:35:35.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:35:35.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:35:35.857 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.855+0000 7fbc8b854640 1 -- 172.21.15.100:0/1441866037 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fbc700b2970 msgr2=0x7fbc700b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:35.857 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.855+0000 7fbc8b854640 1 --2- 172.21.15.100:0/1441866037 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fbc700b2970 0x7fbc700b4e30 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x558545803810 tx=0x7fbc7c005fd0 comp rx=0 tx=0).stop 2024-05-10T11:35:35.857 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.855+0000 7fbc8b854640 1 -- 172.21.15.100:0/1441866037 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc84103530 msgr2=0x7fbc841968d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:35.857 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.855+0000 7fbc8b854640 1 --2- 172.21.15.100:0/1441866037 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc84103530 0x7fbc841968d0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fbc8000e9c0 tx=0x7fbc8000ee80 comp rx=0 tx=0).stop 2024-05-10T11:35:35.857 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.855+0000 7fbc8b854640 1 -- 172.21.15.100:0/1441866037 shutdown_connections 2024-05-10T11:35:35.857 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.855+0000 7fbc8b854640 1 --2- 172.21.15.100:0/1441866037 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fbc700b2970 0x7fbc700b4e30 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:35.858 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.855+0000 7fbc8b854640 1 --2- 172.21.15.100:0/1441866037 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc84108790 0x7fbc8419de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:35.858 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.855+0000 7fbc8b854640 1 --2- 172.21.15.100:0/1441866037 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc84105e60 0x7fbc84196e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:35.858 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.855+0000 7fbc8b854640 1 --2- 172.21.15.100:0/1441866037 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc84103530 0x7fbc841968d0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:35.858 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.855+0000 7fbc8b854640 1 -- 172.21.15.100:0/1441866037 >> 172.21.15.100:0/1441866037 conn(0x7fbc840fce60 msgr2=0x7fbc841040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:35.858 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.855+0000 7fbc8b854640 1 -- 172.21.15.100:0/1441866037 shutdown_connections 2024-05-10T11:35:35.858 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:35.855+0000 7fbc8b854640 1 -- 172.21.15.100:0/1441866037 wait complete. 2024-05-10T11:35:35.919 INFO:tasks.workunit.client.0.smithi100.stdout: 10 35284 9.12 MB/sec execute 89 sec latency 86.440 ms 2024-05-10T11:35:36.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:35:35 smithi178 ceph-mon[33359]: pgmap v937: 129 pgs: 129 active+clean; 450 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 4.6 MiB/s wr, 249 op/s 2024-05-10T11:35:36.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:35:35 smithi188 ceph-mon[34075]: pgmap v937: 129 pgs: 129 active+clean; 450 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 4.6 MiB/s wr, 249 op/s 2024-05-10T11:35:36.919 INFO:tasks.workunit.client.0.smithi100.stdout: 10 35452 9.08 MB/sec execute 90 sec latency 66.997 ms 2024-05-10T11:35:37.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.567+0000 7f163f0e9700 1 -- 172.21.15.100:0/3580900817 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16381023d0 msgr2=0x7f1638102850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:37.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.567+0000 7f163f0e9700 1 --2- 172.21.15.100:0/3580900817 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16381023d0 0x7f1638102850 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f1624009990 tx=0x7f1624009ca0 comp rx=0 tx=0).stop 2024-05-10T11:35:37.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.567+0000 7f163f0e9700 1 -- 172.21.15.100:0/3580900817 shutdown_connections 2024-05-10T11:35:37.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.567+0000 7f163f0e9700 1 --2- 172.21.15.100:0/3580900817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1638102d90 0x7f163810f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:37.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.567+0000 7f163f0e9700 1 --2- 172.21.15.100:0/3580900817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1638108440 0x7f1638108820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:37.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.567+0000 7f163f0e9700 1 --2- 172.21.15.100:0/3580900817 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16381023d0 0x7f1638102850 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:37.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.567+0000 7f163f0e9700 1 -- 172.21.15.100:0/3580900817 >> 172.21.15.100:0/3580900817 conn(0x7f16380793e0 msgr2=0x7f16380797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:37.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.567+0000 7f163f0e9700 1 -- 172.21.15.100:0/3580900817 shutdown_connections 2024-05-10T11:35:37.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.567+0000 7f163f0e9700 1 -- 172.21.15.100:0/3580900817 wait complete. 2024-05-10T11:35:37.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.568+0000 7f163f0e9700 1 Processor -- start 2024-05-10T11:35:37.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.568+0000 7f163f0e9700 1 -- start start 2024-05-10T11:35:37.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.569+0000 7f163f0e9700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16381023d0 0x7f16381a00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:37.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.569+0000 7f163f0e9700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1638102d90 0x7f16381a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:37.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.569+0000 7f163f0e9700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1638108440 0x7f16381a49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:37.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.569+0000 7f163f0e9700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1638111370 con 0x7f16381023d0 2024-05-10T11:35:37.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.569+0000 7f163f0e9700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f16381111f0 con 0x7f1638108440 2024-05-10T11:35:37.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.569+0000 7f163f0e9700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f16381114f0 con 0x7f1638102d90 2024-05-10T11:35:37.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.569+0000 7f163ce85700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16381023d0 0x7f16381a00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:37.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.569+0000 7f163ce85700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16381023d0 0x7f16381a00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47698/0 (socket says 172.21.15.100:47698) 2024-05-10T11:35:37.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.569+0000 7f163ce85700 1 -- 172.21.15.100:0/3243728641 learned_addr learned my addr 172.21.15.100:0/3243728641 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:35:37.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.569+0000 7f163d686700 1 --2- 172.21.15.100:0/3243728641 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1638108440 0x7f16381a49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:37.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.569+0000 7f1637fff700 1 --2- 172.21.15.100:0/3243728641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1638102d90 0x7f16381a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:37.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.569+0000 7f163ce85700 1 -- 172.21.15.100:0/3243728641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1638102d90 msgr2=0x7f16381a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:37.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.569+0000 7f163ce85700 1 --2- 172.21.15.100:0/3243728641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1638102d90 0x7f16381a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:37.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.569+0000 7f163ce85700 1 -- 172.21.15.100:0/3243728641 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1638108440 msgr2=0x7f16381a49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:37.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.569+0000 7f163ce85700 1 --2- 172.21.15.100:0/3243728641 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1638108440 0x7f16381a49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:37.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.569+0000 7f163ce85700 1 -- 172.21.15.100:0/3243728641 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16381a5100 con 0x7f16381023d0 2024-05-10T11:35:37.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.569+0000 7f163d686700 1 --2- 172.21.15.100:0/3243728641 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1638108440 0x7f16381a49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:35:37.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.570+0000 7f163ce85700 1 --2- 172.21.15.100:0/3243728641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16381023d0 0x7f16381a00f0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f162c00eb30 tx=0x7f162c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:37.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.570+0000 7f1635ffb700 1 -- 172.21.15.100:0/3243728641 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f162c00cc90 con 0x7f16381023d0 2024-05-10T11:35:37.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.570+0000 7f163f0e9700 1 -- 172.21.15.100:0/3243728641 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16381a53f0 con 0x7f16381023d0 2024-05-10T11:35:37.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.570+0000 7f1635ffb700 1 -- 172.21.15.100:0/3243728641 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f162c00ce30 con 0x7f16381023d0 2024-05-10T11:35:37.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.570+0000 7f163f0e9700 1 -- 172.21.15.100:0/3243728641 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16381acc70 con 0x7f16381023d0 2024-05-10T11:35:37.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.570+0000 7f1635ffb700 1 -- 172.21.15.100:0/3243728641 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f162c003e30 con 0x7f16381023d0 2024-05-10T11:35:37.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.574+0000 7f1635ffb700 1 -- 172.21.15.100:0/3243728641 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f162c004000 con 0x7f16381023d0 2024-05-10T11:35:37.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.575+0000 7f164997cb80 1 Processor -- start 2024-05-10T11:35:37.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.575+0000 7f164997cb80 1 -- start start 2024-05-10T11:35:37.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.576+0000 7f1635ffb700 1 --2- 172.21.15.100:0/3243728641 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f16280b28e0 0x7f16280b4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:37.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.576+0000 7f163d686700 1 --2- 172.21.15.100:0/3243728641 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f16280b28e0 0x7f16280b4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:37.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.576+0000 7f164997cb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a656601b0 0x555a65660590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:37.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.576+0000 7f164997cb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555a65660ad0 0x555a6565a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:37.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.576+0000 7f1637fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555a65660ad0 0x555a6565a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:37.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.576+0000 7f1637fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555a65660ad0 0x555a6565a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:48888/0 (socket says 172.21.15.100:48888) 2024-05-10T11:35:37.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.576+0000 7f164997cb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a6565ad50 0x555a6565b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:37.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.576+0000 7f1637fff700 1 -- 172.21.15.100:0/758696752 learned_addr learned my addr 172.21.15.100:0/758696752 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:35:37.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.576+0000 7f164997cb80 1 -- 172.21.15.100:0/758696752 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x555a655ac940 con 0x555a656601b0 2024-05-10T11:35:37.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.576+0000 7f164997cb80 1 -- 172.21.15.100:0/758696752 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x555a65511da0 con 0x555a6565ad50 2024-05-10T11:35:37.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.576+0000 7f164997cb80 1 -- 172.21.15.100:0/758696752 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x555a655a8900 con 0x555a65660ad0 2024-05-10T11:35:37.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.577+0000 7f1635ffb700 1 -- 172.21.15.100:0/3243728641 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f162c0e2770 con 0x7f16381023d0 2024-05-10T11:35:37.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.577+0000 7f1637fff700 1 -- 172.21.15.100:0/758696752 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a6565ad50 msgr2=0x555a6565b190 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T11:35:37.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.577+0000 7f1637fff700 1 --2- 172.21.15.100:0/758696752 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a6565ad50 0x555a6565b190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:37.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.577+0000 7f1637fff700 1 -- 172.21.15.100:0/758696752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a656601b0 msgr2=0x555a65660590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:37.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.577+0000 7f1637fff700 1 --2- 172.21.15.100:0/758696752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a656601b0 0x555a65660590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:37.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.577+0000 7f1637fff700 1 -- 172.21.15.100:0/758696752 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a654f98d0 con 0x555a65660ad0 2024-05-10T11:35:37.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.577+0000 7f1637fff700 1 --2- 172.21.15.100:0/758696752 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555a65660ad0 0x555a6565a810 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1624009990 tx=0x7f16240096a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=252b37ae5bfccbb8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:37.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.577+0000 7f163d686700 1 --2- 172.21.15.100:0/3243728641 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f16280b28e0 0x7f16280b4da0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f16300093a0 tx=0x7f163000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:37.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.578+0000 7f1622ffd700 1 -- 172.21.15.100:0/758696752 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1624027070 con 0x555a65660ad0 2024-05-10T11:35:37.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.578+0000 7f1622ffd700 1 -- 172.21.15.100:0/758696752 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f16240056a0 con 0x555a65660ad0 2024-05-10T11:35:37.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.578+0000 7f1622ffd700 1 -- 172.21.15.100:0/758696752 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f162400bad0 con 0x555a65660ad0 2024-05-10T11:35:37.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.579+0000 7f164997cb80 1 -- 172.21.15.100:0/758696752 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555a65660ad0 msgr2=0x555a6565a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:37.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.579+0000 7f164997cb80 1 --2- 172.21.15.100:0/758696752 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555a65660ad0 0x555a6565a810 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1624009990 tx=0x7f16240096a0 comp rx=0 tx=0).stop 2024-05-10T11:35:37.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.579+0000 7f164997cb80 1 -- 172.21.15.100:0/758696752 shutdown_connections 2024-05-10T11:35:37.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.579+0000 7f164997cb80 1 --2- 172.21.15.100:0/758696752 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a6565ad50 0x555a6565b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:37.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.579+0000 7f164997cb80 1 --2- 172.21.15.100:0/758696752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a656601b0 0x555a65660590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:37.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.579+0000 7f164997cb80 1 --2- 172.21.15.100:0/758696752 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555a65660ad0 0x555a6565a810 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:37.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.579+0000 7f164997cb80 1 -- 172.21.15.100:0/758696752 >> 172.21.15.100:0/758696752 conn(0x555a65586b70 msgr2=0x555a656552a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:37.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.580+0000 7f164997cb80 1 -- 172.21.15.100:0/758696752 shutdown_connections 2024-05-10T11:35:37.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.580+0000 7f164997cb80 1 -- 172.21.15.100:0/758696752 wait complete. 2024-05-10T11:35:37.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.580+0000 7f164997cb80 1 -- start start 2024-05-10T11:35:37.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.581+0000 7f164997cb80 1 Processor -- start 2024-05-10T11:35:37.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.581+0000 7f164997cb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a6565ad50 0x555a65666840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:37.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.581+0000 7f164997cb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a656601b0 0x555a65663be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:37.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.581+0000 7f164997cb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555a65660ad0 0x555a65664120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:37.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.581+0000 7f164997cb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x555a656673f0 con 0x555a656601b0 2024-05-10T11:35:37.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.581+0000 7f164997cb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x555a65565ed0 con 0x555a6565ad50 2024-05-10T11:35:37.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.581+0000 7f164997cb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x555a65667570 con 0x555a65660ad0 2024-05-10T11:35:37.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.581+0000 7f164997cb80 10 client.0 _ll_register_callbacks cb 0x555a655405b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:35:37.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.582+0000 7f163d686700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555a65660ad0 0x555a65664120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:37.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.582+0000 7f1637fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a656601b0 0x555a65663be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:37.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.582+0000 7f163d686700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555a65660ad0 0x555a65664120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:48896/0 (socket says 172.21.15.100:48896) 2024-05-10T11:35:37.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.582+0000 7f163d686700 1 -- 172.21.15.100:0/3122544233 learned_addr learned my addr 172.21.15.100:0/3122544233 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:35:37.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.582+0000 7f163ce85700 1 --2- 172.21.15.100:0/3122544233 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a6565ad50 0x555a65666840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:37.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.582+0000 7f163d686700 1 -- 172.21.15.100:0/3122544233 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a6565ad50 msgr2=0x555a65666840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:37.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.582+0000 7f163d686700 1 --2- 172.21.15.100:0/3122544233 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a6565ad50 0x555a65666840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:37.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.582+0000 7f163d686700 1 -- 172.21.15.100:0/3122544233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a656601b0 msgr2=0x555a65663be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:37.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.582+0000 7f163d686700 1 --2- 172.21.15.100:0/3122544233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a656601b0 0x555a65663be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:37.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.582+0000 7f163d686700 1 -- 172.21.15.100:0/3122544233 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a65563760 con 0x555a65660ad0 2024-05-10T11:35:37.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.582+0000 7f163d686700 1 --2- 172.21.15.100:0/3122544233 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555a65660ad0 0x555a65664120 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1630015f00 tx=0x7f1630004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f4ef2f61327d787 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:37.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.582+0000 7f1617fff700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:35:37.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.583+0000 7f1617fff700 1 -- 172.21.15.100:0/3122544233 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1630026060 con 0x555a65660ad0 2024-05-10T11:35:37.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.583+0000 7f164997cb80 1 -- 172.21.15.100:0/3122544233 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16240056a0 con 0x555a65660ad0 2024-05-10T11:35:37.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.583+0000 7f1617fff700 1 -- 172.21.15.100:0/3122544233 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f1630016cf0 con 0x555a65660ad0 2024-05-10T11:35:37.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.583+0000 7f1617fff700 1 -- 172.21.15.100:0/3122544233 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f163002d050 con 0x555a65660ad0 2024-05-10T11:35:37.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.583+0000 7f164997cb80 10 client.28246 fetch_fsmap learned FSMap version 41 2024-05-10T11:35:37.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.583+0000 7f164997cb80 1 -- 172.21.15.100:0/3122544233 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x555a65671e60 con 0x555a65660ad0 2024-05-10T11:35:37.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.583+0000 7f1617fff700 1 -- 172.21.15.100:0/3122544233 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f16240056a0 con 0x555a65660ad0 2024-05-10T11:35:37.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.583+0000 7f1617fff700 1 -- 172.21.15.100:0/3122544233 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f16300143a0 con 0x555a65660ad0 2024-05-10T11:35:37.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.583+0000 7f164997cb80 10 client.28246 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:35:37.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.583+0000 7f164997cb80 10 client.28246 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-10T11:35:37.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.584+0000 7f164997cb80 20 client.28246 populate_metadata read hostname 'smithi100' 2024-05-10T11:35:37.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.584+0000 7f164997cb80 1 --2- 172.21.15.100:0/3122544233 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x555a656743f0 0x555a656767e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:37.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.584+0000 7f164997cb80 4 client.28246 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:35:37.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.584+0000 7f164997cb80 1 -- 172.21.15.100:0/3122544233 --> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a655ba320 con 0x555a656743f0 2024-05-10T11:35:37.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.584+0000 7f163ce85700 1 --2- 172.21.15.100:0/3122544233 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x555a656743f0 0x555a656767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:37.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.585+0000 7f163ce85700 1 --2- 172.21.15.100:0/3122544233 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x555a656743f0 0x555a656767e0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=95162379ec9013bc server_cookie=b7634dded39068c3 in_seq=0 out_seq=0 2024-05-10T11:35:37.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.585+0000 7f1617fff700 10 client.28246 ms_handle_connect on v2:172.21.15.178:6838/3760985404 2024-05-10T11:35:37.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.586+0000 7f1617fff700 1 -- 172.21.15.100:0/3122544233 <== mds.1 v2:172.21.15.178:6838/3760985404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555a655ba320 con 0x555a656743f0 2024-05-10T11:35:37.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.586+0000 7f1617fff700 10 client.28246 handle_command_reply: tid=0 2024-05-10T11:35:37.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.586+0000 7f164997cb80 1 client.28246 shutdown 2024-05-10T11:35:37.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.586+0000 7f164997cb80 1 -- 172.21.15.100:0/3122544233 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555a65660ad0 msgr2=0x555a65664120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:37.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.586+0000 7f164997cb80 1 --2- 172.21.15.100:0/3122544233 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555a65660ad0 0x555a65664120 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1630015f00 tx=0x7f1630004ad0 comp rx=0 tx=0).stop 2024-05-10T11:35:37.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.586+0000 7f164997cb80 1 -- 172.21.15.100:0/3122544233 shutdown_connections 2024-05-10T11:35:37.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.586+0000 7f164997cb80 1 --2- 172.21.15.100:0/3122544233 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x555a656743f0 0x555a656767e0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:37.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.586+0000 7f164997cb80 1 --2- 172.21.15.100:0/3122544233 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555a65660ad0 0x555a65664120 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:37.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.586+0000 7f164997cb80 1 --2- 172.21.15.100:0/3122544233 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a6565ad50 0x555a65666840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:37.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.586+0000 7f164997cb80 1 --2- 172.21.15.100:0/3122544233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a656601b0 0x555a65663be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:37.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.586+0000 7f164997cb80 1 -- 172.21.15.100:0/3122544233 >> 172.21.15.100:0/3122544233 conn(0x555a65586b70 msgr2=0x555a6564fb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:37.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.586+0000 7f1617fff700 0 client.28246 ms_handle_reset on v2:172.21.15.178:6838/3760985404 2024-05-10T11:35:37.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.586+0000 7f164997cb80 1 -- 172.21.15.100:0/3122544233 shutdown_connections 2024-05-10T11:35:37.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.586+0000 7f164997cb80 1 -- 172.21.15.100:0/3122544233 wait complete. 2024-05-10T11:35:37.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.586+0000 7f164997cb80 20 client.28246 trim_cache size 0 max 16384 2024-05-10T11:35:37.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.606+0000 7f164997cb80 1 Processor -- start 2024-05-10T11:35:37.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.607+0000 7f164997cb80 1 -- start start 2024-05-10T11:35:37.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.607+0000 7f164997cb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555a6565ad50 0x555a6565a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:37.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.607+0000 7f164997cb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a656601b0 0x555a65670480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:37.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.607+0000 7f164997cb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a65660ad0 0x555a6564fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:37.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.607+0000 7f164997cb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f16381b8010 con 0x555a656601b0 2024-05-10T11:35:37.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.607+0000 7f164997cb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f16381b58a0 con 0x555a65660ad0 2024-05-10T11:35:37.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.607+0000 7f164997cb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f16381b36f0 con 0x555a6565ad50 2024-05-10T11:35:37.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.607+0000 7f1637fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a656601b0 0x555a65670480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:37.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.607+0000 7f163d686700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a65660ad0 0x555a6564fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:37.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.607+0000 7f1637fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a656601b0 0x555a65670480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47714/0 (socket says 172.21.15.100:47714) 2024-05-10T11:35:37.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.607+0000 7f163d686700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a65660ad0 0x555a6564fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:54514/0 (socket says 172.21.15.100:54514) 2024-05-10T11:35:37.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.607+0000 7f1637fff700 1 -- 172.21.15.100:0/4198267504 learned_addr learned my addr 172.21.15.100:0/4198267504 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:35:37.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.607+0000 7f163ce85700 1 --2- 172.21.15.100:0/4198267504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555a6565ad50 0x555a6565a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:37.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.607+0000 7f163d686700 1 -- 172.21.15.100:0/4198267504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555a6565ad50 msgr2=0x555a6565a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:37.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.607+0000 7f163d686700 1 --2- 172.21.15.100:0/4198267504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555a6565ad50 0x555a6565a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:37.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.607+0000 7f163d686700 1 -- 172.21.15.100:0/4198267504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a656601b0 msgr2=0x555a65670480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:37.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.607+0000 7f163d686700 1 --2- 172.21.15.100:0/4198267504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a656601b0 0x555a65670480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:37.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.607+0000 7f163d686700 1 -- 172.21.15.100:0/4198267504 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a65656ae0 con 0x555a65660ad0 2024-05-10T11:35:37.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.608+0000 7f1637fff700 1 --2- 172.21.15.100:0/4198267504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a656601b0 0x555a65670480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:35:37.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.608+0000 7f163d686700 1 --2- 172.21.15.100:0/4198267504 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a65660ad0 0x555a6564fc40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1630000f80 tx=0x7f16300095e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15ba863cba9c5c93 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:37.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.608+0000 7f16227fc700 1 -- 172.21.15.100:0/4198267504 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16300143a0 con 0x555a65660ad0 2024-05-10T11:35:37.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.608+0000 7f16227fc700 1 -- 172.21.15.100:0/4198267504 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1630035070 con 0x555a65660ad0 2024-05-10T11:35:37.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.608+0000 7f16227fc700 1 -- 172.21.15.100:0/4198267504 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163003a900 con 0x555a65660ad0 2024-05-10T11:35:37.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.609+0000 7f164997cb80 1 -- 172.21.15.100:0/4198267504 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a65660ad0 msgr2=0x555a6564fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:37.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.609+0000 7f164997cb80 1 --2- 172.21.15.100:0/4198267504 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a65660ad0 0x555a6564fc40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1630000f80 tx=0x7f16300095e0 comp rx=0 tx=0).stop 2024-05-10T11:35:37.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.609+0000 7f164997cb80 1 -- 172.21.15.100:0/4198267504 shutdown_connections 2024-05-10T11:35:37.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.609+0000 7f164997cb80 1 --2- 172.21.15.100:0/4198267504 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a65660ad0 0x555a6564fc40 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:37.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.609+0000 7f164997cb80 1 --2- 172.21.15.100:0/4198267504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555a6565ad50 0x555a6565a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:37.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.609+0000 7f164997cb80 1 --2- 172.21.15.100:0/4198267504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a656601b0 0x555a65670480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:37.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.609+0000 7f164997cb80 1 -- 172.21.15.100:0/4198267504 >> 172.21.15.100:0/4198267504 conn(0x555a65586b70 msgr2=0x555a65660ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:37.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.609+0000 7f164997cb80 1 -- 172.21.15.100:0/4198267504 shutdown_connections 2024-05-10T11:35:37.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.609+0000 7f164997cb80 1 -- 172.21.15.100:0/4198267504 wait complete. 2024-05-10T11:35:37.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.609+0000 7f164997cb80 1 -- start start 2024-05-10T11:35:37.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.610+0000 7f164997cb80 1 Processor -- start 2024-05-10T11:35:37.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.610+0000 7f164997cb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555a6565ad50 0x555a6567c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:37.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.610+0000 7f164997cb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a656601b0 0x555a6566c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:37.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.610+0000 7f164997cb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a65660ad0 0x555a6566cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:37.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.610+0000 7f164997cb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f16381b3d60 con 0x555a65660ad0 2024-05-10T11:35:37.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.610+0000 7f164997cb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f16381b51d0 con 0x555a656601b0 2024-05-10T11:35:37.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.610+0000 7f164997cb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f16381b4430 con 0x555a6565ad50 2024-05-10T11:35:37.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.610+0000 7f164997cb80 10 client.0 _ll_register_callbacks cb 0x555a655405b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:35:37.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.610+0000 7f1637fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a656601b0 0x555a6566c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:37.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.610+0000 7f1637fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a656601b0 0x555a6566c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:54520/0 (socket says 172.21.15.100:54520) 2024-05-10T11:35:37.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.610+0000 7f1637fff700 1 -- 172.21.15.100:0/2144126408 learned_addr learned my addr 172.21.15.100:0/2144126408 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:35:37.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.610+0000 7f163d686700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a65660ad0 0x555a6566cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:37.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.611+0000 7f163ce85700 1 --2- 172.21.15.100:0/2144126408 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555a6565ad50 0x555a6567c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:37.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.611+0000 7f1637fff700 1 -- 172.21.15.100:0/2144126408 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555a6565ad50 msgr2=0x555a6567c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:37.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.611+0000 7f1637fff700 1 --2- 172.21.15.100:0/2144126408 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555a6565ad50 0x555a6567c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:37.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.611+0000 7f1637fff700 1 -- 172.21.15.100:0/2144126408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a65660ad0 msgr2=0x555a6566cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:37.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.611+0000 7f1637fff700 1 --2- 172.21.15.100:0/2144126408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a65660ad0 0x555a6566cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:37.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.611+0000 7f1637fff700 1 -- 172.21.15.100:0/2144126408 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a6567b050 con 0x555a656601b0 2024-05-10T11:35:37.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.611+0000 7f163ce85700 1 --2- 172.21.15.100:0/2144126408 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555a6565ad50 0x555a6567c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:35:37.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.611+0000 7f1637fff700 1 --2- 172.21.15.100:0/2144126408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a656601b0 0x555a6566c800 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1624000c00 tx=0x7f16240056d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3e6d385bef9ce86 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:37.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.611+0000 7f1617fff700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:35:37.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.611+0000 7f1617fff700 1 -- 172.21.15.100:0/2144126408 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1624032030 con 0x555a656601b0 2024-05-10T11:35:37.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.612+0000 7f164997cb80 1 -- 172.21.15.100:0/2144126408 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1630035070 con 0x555a656601b0 2024-05-10T11:35:37.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.612+0000 7f1617fff700 1 -- 172.21.15.100:0/2144126408 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f162400b9f0 con 0x555a656601b0 2024-05-10T11:35:37.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.612+0000 7f1617fff700 1 -- 172.21.15.100:0/2144126408 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f1624027040 con 0x555a656601b0 2024-05-10T11:35:37.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.612+0000 7f164997cb80 10 client.28157 fetch_fsmap learned FSMap version 41 2024-05-10T11:35:37.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.612+0000 7f164997cb80 1 -- 172.21.15.100:0/2144126408 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x555a656ddeb0 con 0x555a656601b0 2024-05-10T11:35:37.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.612+0000 7f1617fff700 1 -- 172.21.15.100:0/2144126408 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1630035070 con 0x555a656601b0 2024-05-10T11:35:37.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.613+0000 7f1617fff700 1 -- 172.21.15.100:0/2144126408 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f1624021070 con 0x555a656601b0 2024-05-10T11:35:37.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.613+0000 7f164997cb80 10 client.28157 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:35:37.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.613+0000 7f164997cb80 10 client.28157 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-10T11:35:37.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.613+0000 7f164997cb80 20 client.28157 populate_metadata read hostname 'smithi100' 2024-05-10T11:35:37.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.613+0000 7f164997cb80 1 --2- 172.21.15.100:0/2144126408 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x555a656de0e0 0x555a656e04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:37.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.613+0000 7f164997cb80 4 client.28157 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:35:37.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.613+0000 7f164997cb80 1 -- 172.21.15.100:0/2144126408 --> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555a655b43c0 con 0x555a656de0e0 2024-05-10T11:35:37.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.614+0000 7f163ce85700 1 --2- 172.21.15.100:0/2144126408 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x555a656de0e0 0x555a656e04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:37.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.617+0000 7f163ce85700 1 --2- 172.21.15.100:0/2144126408 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x555a656de0e0 0x555a656e04d0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4e76411cb181bed1 server_cookie=3582b7395b8eef8 in_seq=0 out_seq=0 2024-05-10T11:35:37.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.617+0000 7f1617fff700 10 client.28157 ms_handle_connect on v2:172.21.15.178:6838/3760985404 2024-05-10T11:35:37.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.617+0000 7f1617fff700 1 -- 172.21.15.100:0/2144126408 <== mds.1 v2:172.21.15.178:6838/3760985404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555a655b43c0 con 0x555a656de0e0 2024-05-10T11:35:37.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.617+0000 7f1617fff700 10 client.28157 handle_command_reply: tid=0 2024-05-10T11:35:37.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.617+0000 7f164997cb80 1 client.28157 shutdown 2024-05-10T11:35:37.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.618+0000 7f164997cb80 1 -- 172.21.15.100:0/2144126408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a656601b0 msgr2=0x555a6566c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:37.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.618+0000 7f164997cb80 1 --2- 172.21.15.100:0/2144126408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a656601b0 0x555a6566c800 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1624000c00 tx=0x7f16240056d0 comp rx=0 tx=0).stop 2024-05-10T11:35:37.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.618+0000 7f164997cb80 1 -- 172.21.15.100:0/2144126408 shutdown_connections 2024-05-10T11:35:37.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.618+0000 7f164997cb80 1 --2- 172.21.15.100:0/2144126408 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x555a656de0e0 0x555a656e04d0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:37.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.618+0000 7f164997cb80 1 --2- 172.21.15.100:0/2144126408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a65660ad0 0x555a6566cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:37.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.618+0000 7f164997cb80 1 --2- 172.21.15.100:0/2144126408 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555a6565ad50 0x555a6567c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:37.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.618+0000 7f164997cb80 1 --2- 172.21.15.100:0/2144126408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a656601b0 0x555a6566c800 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:37.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.618+0000 7f164997cb80 1 -- 172.21.15.100:0/2144126408 >> 172.21.15.100:0/2144126408 conn(0x555a65586b70 msgr2=0x555a65661740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:37.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.618+0000 7f1617fff700 0 client.28157 ms_handle_reset on v2:172.21.15.178:6838/3760985404 2024-05-10T11:35:37.622 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:35:37.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.619+0000 7f164997cb80 1 -- 172.21.15.100:0/2144126408 shutdown_connections 2024-05-10T11:35:37.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.621+0000 7f164997cb80 1 -- 172.21.15.100:0/2144126408 wait complete. 2024-05-10T11:35:37.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.621+0000 7f164997cb80 20 client.28157 trim_cache size 0 max 16384 2024-05-10T11:35:37.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.623+0000 7f163f0e9700 1 -- 172.21.15.100:0/3243728641 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f16280b28e0 msgr2=0x7f16280b4da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:37.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.623+0000 7f163f0e9700 1 --2- 172.21.15.100:0/3243728641 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f16280b28e0 0x7f16280b4da0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f16300093a0 tx=0x7f163000b040 comp rx=0 tx=0).stop 2024-05-10T11:35:37.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.623+0000 7f163f0e9700 1 -- 172.21.15.100:0/3243728641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16381023d0 msgr2=0x7f16381a00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:37.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.623+0000 7f163f0e9700 1 --2- 172.21.15.100:0/3243728641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16381023d0 0x7f16381a00f0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f162c00eb30 tx=0x7f162c00ee40 comp rx=0 tx=0).stop 2024-05-10T11:35:37.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.623+0000 7f163f0e9700 1 -- 172.21.15.100:0/3243728641 shutdown_connections 2024-05-10T11:35:37.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.623+0000 7f163f0e9700 1 --2- 172.21.15.100:0/3243728641 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f16280b28e0 0x7f16280b4da0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:37.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.623+0000 7f163f0e9700 1 --2- 172.21.15.100:0/3243728641 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1638108440 0x7f16381a49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:37.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.623+0000 7f163f0e9700 1 --2- 172.21.15.100:0/3243728641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16381023d0 0x7f16381a00f0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:37.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.623+0000 7f163f0e9700 1 --2- 172.21.15.100:0/3243728641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1638102d90 0x7f16381a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:37.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.623+0000 7f163f0e9700 1 -- 172.21.15.100:0/3243728641 >> 172.21.15.100:0/3243728641 conn(0x7f16380793e0 msgr2=0x7f1638101400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:37.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.626+0000 7f163f0e9700 1 -- 172.21.15.100:0/3243728641 shutdown_connections 2024-05-10T11:35:37.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:37.626+0000 7f163f0e9700 1 -- 172.21.15.100:0/3243728641 wait complete. 2024-05-10T11:35:37.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:35:37 smithi100 ceph-mon[30296]: pgmap v938: 129 pgs: 129 active+clean; 440 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 871 KiB/s rd, 4.8 MiB/s wr, 281 op/s 2024-05-10T11:35:37.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:35:37 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:35:37.920 INFO:tasks.workunit.client.0.smithi100.stdout: 10 35610 9.06 MB/sec execute 91 sec latency 98.257 ms 2024-05-10T11:35:38.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:35:37 smithi178 ceph-mon[33359]: pgmap v938: 129 pgs: 129 active+clean; 440 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 871 KiB/s rd, 4.8 MiB/s wr, 281 op/s 2024-05-10T11:35:38.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:35:37 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:35:38.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:35:37 smithi188 ceph-mon[34075]: pgmap v938: 129 pgs: 129 active+clean; 440 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 871 KiB/s rd, 4.8 MiB/s wr, 281 op/s 2024-05-10T11:35:38.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:35:37 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:35:38.920 INFO:tasks.workunit.client.0.smithi100.stdout: 10 35761 8.97 MB/sec execute 92 sec latency 81.084 ms 2024-05-10T11:35:39.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:35:38 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:35:39.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:35:38 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:35:39.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:35:38 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:35:39.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:35:38 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:35:39.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:35:38 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:35:39.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:35:38 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:35:39.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:35:38 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:35:39.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:35:38 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:35:39.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:35:38 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:35:39.281 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-10T11:35:39.659 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:35:39.920 INFO:tasks.workunit.client.0.smithi100.stdout: 10 35938 8.92 MB/sec execute 93 sec latency 86.366 ms 2024-05-10T11:35:40.403 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:35:40 smithi100 ceph-mon[30296]: pgmap v939: 129 pgs: 129 active+clean; 439 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 3.9 MiB/s wr, 282 op/s 2024-05-10T11:35:40.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:35:40 smithi178 ceph-mon[33359]: pgmap v939: 129 pgs: 129 active+clean; 439 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 3.9 MiB/s wr, 282 op/s 2024-05-10T11:35:40.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:35:40 smithi188 ceph-mon[34075]: pgmap v939: 129 pgs: 129 active+clean; 439 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 3.9 MiB/s wr, 282 op/s 2024-05-10T11:35:40.920 INFO:tasks.workunit.client.0.smithi100.stdout: 10 36116 8.91 MB/sec execute 94 sec latency 90.937 ms 2024-05-10T11:35:41.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.821+0000 7f96fbabf700 1 -- 172.21.15.100:0/4226033211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96f4103f00 msgr2=0x7f96f41103b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:41.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.821+0000 7f96fbabf700 1 --2- 172.21.15.100:0/4226033211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96f4103f00 0x7f96f41103b0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f96ec00b0d0 tx=0x7f96ec00b3e0 comp rx=0 tx=0).stop 2024-05-10T11:35:41.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.821+0000 7f96fbabf700 1 -- 172.21.15.100:0/4226033211 shutdown_connections 2024-05-10T11:35:41.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.821+0000 7f96fbabf700 1 --2- 172.21.15.100:0/4226033211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96f4103f00 0x7f96f41103b0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:41.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.821+0000 7f96fbabf700 1 --2- 172.21.15.100:0/4226033211 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96f41095e0 0x7f96f41099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:41.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.821+0000 7f96fbabf700 1 --2- 172.21.15.100:0/4226033211 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96f4103540 0x7f96f41039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:41.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.821+0000 7f96fbabf700 1 -- 172.21.15.100:0/4226033211 >> 172.21.15.100:0/4226033211 conn(0x7f96f407a650 msgr2=0x7f96f407aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:41.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.821+0000 7f96fbabf700 1 -- 172.21.15.100:0/4226033211 shutdown_connections 2024-05-10T11:35:41.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.821+0000 7f96fbabf700 1 -- 172.21.15.100:0/4226033211 wait complete. 2024-05-10T11:35:41.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.822+0000 7f96fbabf700 1 Processor -- start 2024-05-10T11:35:41.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.822+0000 7f96fbabf700 1 -- start start 2024-05-10T11:35:41.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.823+0000 7f96fbabf700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96f4103540 0x7f96f419f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:41.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.823+0000 7f96fbabf700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96f4103f00 0x7f96f419fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:41.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.823+0000 7f96fbabf700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96f41095e0 0x7f96f4199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:41.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.823+0000 7f96fbabf700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f96f41135f0 con 0x7f96f4103f00 2024-05-10T11:35:41.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.823+0000 7f96fbabf700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f96f4113470 con 0x7f96f4103540 2024-05-10T11:35:41.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.823+0000 7f96fbabf700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f96f4113770 con 0x7f96f41095e0 2024-05-10T11:35:41.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.823+0000 7f96f905a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96f4103f00 0x7f96f419fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:41.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.823+0000 7f96f905a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96f4103f00 0x7f96f419fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47730/0 (socket says 172.21.15.100:47730) 2024-05-10T11:35:41.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.823+0000 7f96f905a700 1 -- 172.21.15.100:0/3731789437 learned_addr learned my addr 172.21.15.100:0/3731789437 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:35:41.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.823+0000 7f96fa05c700 1 --2- 172.21.15.100:0/3731789437 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96f41095e0 0x7f96f4199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:41.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.823+0000 7f96f985b700 1 --2- 172.21.15.100:0/3731789437 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96f4103540 0x7f96f419f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:41.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.823+0000 7f96fa05c700 1 -- 172.21.15.100:0/3731789437 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96f4103540 msgr2=0x7f96f419f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:41.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.823+0000 7f96fa05c700 1 --2- 172.21.15.100:0/3731789437 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96f4103540 0x7f96f419f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:41.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.823+0000 7f96fa05c700 1 -- 172.21.15.100:0/3731789437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96f4103f00 msgr2=0x7f96f419fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:41.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.823+0000 7f96fa05c700 1 --2- 172.21.15.100:0/3731789437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96f4103f00 0x7f96f419fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:41.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.823+0000 7f96fa05c700 1 -- 172.21.15.100:0/3731789437 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96f4199ff0 con 0x7f96f41095e0 2024-05-10T11:35:41.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.823+0000 7f96f905a700 1 --2- 172.21.15.100:0/3731789437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96f4103f00 0x7f96f419fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:35:41.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.824+0000 7f96fa05c700 1 --2- 172.21.15.100:0/3731789437 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96f41095e0 0x7f96f4199890 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f96ec00b0d0 tx=0x7f96ec007a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:41.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.824+0000 7f96eaffd700 1 -- 172.21.15.100:0/3731789437 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ec01f070 con 0x7f96f41095e0 2024-05-10T11:35:41.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.824+0000 7f96fbabf700 1 -- 172.21.15.100:0/3731789437 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96f419a280 con 0x7f96f41095e0 2024-05-10T11:35:41.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.824+0000 7f96eaffd700 1 -- 172.21.15.100:0/3731789437 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f96ec004c50 con 0x7f96f41095e0 2024-05-10T11:35:41.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.824+0000 7f96eaffd700 1 -- 172.21.15.100:0/3731789437 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ec01b430 con 0x7f96f41095e0 2024-05-10T11:35:41.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.824+0000 7f96fbabf700 1 -- 172.21.15.100:0/3731789437 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96f41a65a0 con 0x7f96f41095e0 2024-05-10T11:35:41.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.828+0000 7f96eaffd700 1 -- 172.21.15.100:0/3731789437 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f96ec022070 con 0x7f96f41095e0 2024-05-10T11:35:41.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.829+0000 7f96eaffd700 1 --2- 172.21.15.100:0/3731789437 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f96e00b2900 0x7f96e00b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:41.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.829+0000 7f96eaffd700 1 -- 172.21.15.100:0/3731789437 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f96ec00f020 con 0x7f96f41095e0 2024-05-10T11:35:41.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.829+0000 7f96f985b700 1 --2- 172.21.15.100:0/3731789437 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f96e00b2900 0x7f96e00b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:41.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.829+0000 7f9706352b80 1 Processor -- start 2024-05-10T11:35:41.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.829+0000 7f9706352b80 1 -- start start 2024-05-10T11:35:41.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.830+0000 7f9706352b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55690213b1f0 0x55690213b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:41.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.830+0000 7f9706352b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556902137810 0x556902137c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:41.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.830+0000 7f9706352b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5569021381d0 0x55690214c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:41.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.830+0000 7f9706352b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x556902089940 con 0x556902137810 2024-05-10T11:35:41.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.830+0000 7f9706352b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556901feeda0 con 0x5569021381d0 2024-05-10T11:35:41.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.830+0000 7f9706352b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556902085900 con 0x55690213b1f0 2024-05-10T11:35:41.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.830+0000 7f96fa05c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5569021381d0 0x55690214c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:41.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.830+0000 7f96fa05c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5569021381d0 0x55690214c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:54540/0 (socket says 172.21.15.100:54540) 2024-05-10T11:35:41.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.830+0000 7f96fa05c700 1 -- 172.21.15.100:0/375641665 learned_addr learned my addr 172.21.15.100:0/375641665 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:35:41.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.830+0000 7f96f985b700 1 --2- 172.21.15.100:0/375641665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55690213b1f0 0x55690213b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:41.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.830+0000 7f96f905a700 1 --2- 172.21.15.100:0/375641665 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556902137810 0x556902137c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:41.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.831+0000 7f96fa05c700 1 -- 172.21.15.100:0/375641665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55690213b1f0 msgr2=0x55690213b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:41.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.831+0000 7f96fa05c700 1 --2- 172.21.15.100:0/375641665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55690213b1f0 0x55690213b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:41.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.831+0000 7f96fa05c700 1 -- 172.21.15.100:0/375641665 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556902137810 msgr2=0x556902137c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:41.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.831+0000 7f96fa05c700 1 --2- 172.21.15.100:0/375641665 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556902137810 0x556902137c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:41.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.831+0000 7f96fa05c700 1 -- 172.21.15.100:0/375641665 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556901fd68d0 con 0x5569021381d0 2024-05-10T11:35:41.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.831+0000 7f96f985b700 1 --2- 172.21.15.100:0/3731789437 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f96e00b2900 0x7f96e00b4dc0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f96f419a9d0 tx=0x7f96f0011720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:41.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.831+0000 7f96fa05c700 1 --2- 172.21.15.100:0/375641665 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5569021381d0 0x55690214c480 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f96ec0aa3c0 tx=0x7f96ec006230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94adaee9dab8b1b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:41.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.832+0000 7f96dffff700 1 -- 172.21.15.100:0/375641665 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ec004e00 con 0x5569021381d0 2024-05-10T11:35:41.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.832+0000 7f96dffff700 1 -- 172.21.15.100:0/375641665 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f96ec0aa1d0 con 0x5569021381d0 2024-05-10T11:35:41.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.832+0000 7f96dffff700 1 -- 172.21.15.100:0/375641665 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ec0074e0 con 0x5569021381d0 2024-05-10T11:35:41.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.833+0000 7f9706352b80 1 -- 172.21.15.100:0/375641665 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5569021381d0 msgr2=0x55690214c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:41.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.833+0000 7f9706352b80 1 --2- 172.21.15.100:0/375641665 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5569021381d0 0x55690214c480 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f96ec0aa3c0 tx=0x7f96ec006230 comp rx=0 tx=0).stop 2024-05-10T11:35:41.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.833+0000 7f9706352b80 1 -- 172.21.15.100:0/375641665 shutdown_connections 2024-05-10T11:35:41.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.833+0000 7f9706352b80 1 --2- 172.21.15.100:0/375641665 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5569021381d0 0x55690214c480 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:41.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.833+0000 7f9706352b80 1 --2- 172.21.15.100:0/375641665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55690213b1f0 0x55690213b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:41.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.833+0000 7f9706352b80 1 --2- 172.21.15.100:0/375641665 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556902137810 0x556902137c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:41.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.833+0000 7f9706352b80 1 -- 172.21.15.100:0/375641665 >> 172.21.15.100:0/375641665 conn(0x556902063b70 msgr2=0x5569021322a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:41.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.833+0000 7f9706352b80 1 -- 172.21.15.100:0/375641665 shutdown_connections 2024-05-10T11:35:41.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.833+0000 7f9706352b80 1 -- 172.21.15.100:0/375641665 wait complete. 2024-05-10T11:35:41.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.833+0000 7f9706352b80 1 -- start start 2024-05-10T11:35:41.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.834+0000 7f9706352b80 1 Processor -- start 2024-05-10T11:35:41.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.834+0000 7f9706352b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556902137810 0x55690214c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:41.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.834+0000 7f9706352b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5569021381d0 0x556902149520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:41.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.834+0000 7f9706352b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55690213b1f0 0x556902149a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:41.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.834+0000 7f9706352b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55690214ca20 con 0x556902137810 2024-05-10T11:35:41.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.834+0000 7f9706352b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556902042ed0 con 0x55690213b1f0 2024-05-10T11:35:41.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.834+0000 7f9706352b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55690214cba0 con 0x5569021381d0 2024-05-10T11:35:41.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.834+0000 7f9706352b80 10 client.0 _ll_register_callbacks cb 0x55690201d5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:35:41.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.834+0000 7f96f985b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556902137810 0x55690214c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:41.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.835+0000 7f96f905a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5569021381d0 0x556902149520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:41.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.835+0000 7f96f985b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556902137810 0x55690214c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47750/0 (socket says 172.21.15.100:47750) 2024-05-10T11:35:41.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.835+0000 7f96f985b700 1 -- 172.21.15.100:0/1050837941 learned_addr learned my addr 172.21.15.100:0/1050837941 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:35:41.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.835+0000 7f96fa05c700 1 --2- 172.21.15.100:0/1050837941 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55690213b1f0 0x556902149a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:41.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.835+0000 7f96fa05c700 1 -- 172.21.15.100:0/1050837941 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5569021381d0 msgr2=0x556902149520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:41.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.835+0000 7f96fa05c700 1 --2- 172.21.15.100:0/1050837941 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5569021381d0 0x556902149520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:41.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.835+0000 7f96fa05c700 1 -- 172.21.15.100:0/1050837941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556902137810 msgr2=0x55690214c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:41.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.835+0000 7f96fa05c700 1 --2- 172.21.15.100:0/1050837941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556902137810 0x55690214c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:41.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.835+0000 7f96fa05c700 1 -- 172.21.15.100:0/1050837941 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556902040760 con 0x55690213b1f0 2024-05-10T11:35:41.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.835+0000 7f96f985b700 1 --2- 172.21.15.100:0/1050837941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556902137810 0x55690214c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:35:41.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.835+0000 7f96fa05c700 1 --2- 172.21.15.100:0/1050837941 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55690213b1f0 0x556902149a60 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f96ec0aec30 tx=0x7f96ec00b730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7dc2502ec938595 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:41.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.835+0000 7f96dd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:35:41.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.835+0000 7f96dd7fa700 1 -- 172.21.15.100:0/1050837941 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ec0c3070 con 0x55690213b1f0 2024-05-10T11:35:41.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.835+0000 7f96dd7fa700 1 -- 172.21.15.100:0/1050837941 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f96ec0920e0 con 0x55690213b1f0 2024-05-10T11:35:41.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.836+0000 7f9706352b80 1 -- 172.21.15.100:0/1050837941 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96ec0aa1d0 con 0x55690213b1f0 2024-05-10T11:35:41.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.836+0000 7f96dd7fa700 1 -- 172.21.15.100:0/1050837941 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f96ec0922e0 con 0x55690213b1f0 2024-05-10T11:35:41.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.836+0000 7f9706352b80 10 client.28169 fetch_fsmap learned FSMap version 41 2024-05-10T11:35:41.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.836+0000 7f9706352b80 1 -- 172.21.15.100:0/1050837941 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55690214ca20 con 0x55690213b1f0 2024-05-10T11:35:41.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.836+0000 7f96dd7fa700 1 -- 172.21.15.100:0/1050837941 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f96ec0aa1d0 con 0x55690213b1f0 2024-05-10T11:35:41.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.836+0000 7f96dd7fa700 1 -- 172.21.15.100:0/1050837941 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f96ec011070 con 0x55690213b1f0 2024-05-10T11:35:41.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.836+0000 7f9706352b80 10 client.28169 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:35:41.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.836+0000 7f9706352b80 10 client.28169 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:35:41.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.836+0000 7f9706352b80 20 client.28169 populate_metadata read hostname 'smithi100' 2024-05-10T11:35:41.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.836+0000 7f9706352b80 1 --2- 172.21.15.100:0/1050837941 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556902159650 0x55690215ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:41.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.836+0000 7f9706352b80 4 client.28169 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:35:41.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.836+0000 7f9706352b80 1 -- 172.21.15.100:0/1050837941 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556902097320 con 0x556902159650 2024-05-10T11:35:41.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.837+0000 7f96f985b700 1 --2- 172.21.15.100:0/1050837941 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556902159650 0x55690215ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:41.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.838+0000 7f96dd7fa700 10 client.28169 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:35:41.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.838+0000 7f96f985b700 1 --2- 172.21.15.100:0/1050837941 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556902159650 0x55690215ba40 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=614cc84d6ab2f94e server_cookie=1cdf491a9af45213 in_seq=0 out_seq=0 2024-05-10T11:35:41.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.838+0000 7f96dd7fa700 1 -- 172.21.15.100:0/1050837941 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556902097320 con 0x556902159650 2024-05-10T11:35:41.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.838+0000 7f96dd7fa700 10 client.28169 handle_command_reply: tid=0 2024-05-10T11:35:41.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.839+0000 7f9706352b80 1 client.28169 shutdown 2024-05-10T11:35:41.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.839+0000 7f9706352b80 1 -- 172.21.15.100:0/1050837941 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55690213b1f0 msgr2=0x556902149a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:41.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.839+0000 7f9706352b80 1 --2- 172.21.15.100:0/1050837941 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55690213b1f0 0x556902149a60 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f96ec0aec30 tx=0x7f96ec00b730 comp rx=0 tx=0).stop 2024-05-10T11:35:41.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.839+0000 7f9706352b80 1 -- 172.21.15.100:0/1050837941 shutdown_connections 2024-05-10T11:35:41.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.839+0000 7f9706352b80 1 --2- 172.21.15.100:0/1050837941 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556902159650 0x55690215ba40 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:41.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.839+0000 7f9706352b80 1 --2- 172.21.15.100:0/1050837941 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55690213b1f0 0x556902149a60 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:41.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.839+0000 7f9706352b80 1 --2- 172.21.15.100:0/1050837941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556902137810 0x55690214c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:41.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.839+0000 7f9706352b80 1 --2- 172.21.15.100:0/1050837941 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5569021381d0 0x556902149520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:41.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.839+0000 7f9706352b80 1 -- 172.21.15.100:0/1050837941 >> 172.21.15.100:0/1050837941 conn(0x556902063b70 msgr2=0x55690212cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:41.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.839+0000 7f96dd7fa700 0 client.28169 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:35:41.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.839+0000 7f9706352b80 1 -- 172.21.15.100:0/1050837941 shutdown_connections 2024-05-10T11:35:41.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.839+0000 7f9706352b80 1 -- 172.21.15.100:0/1050837941 wait complete. 2024-05-10T11:35:41.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.839+0000 7f9706352b80 20 client.28169 trim_cache size 0 max 16384 2024-05-10T11:35:41.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.864+0000 7f9706352b80 1 Processor -- start 2024-05-10T11:35:41.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.864+0000 7f9706352b80 1 -- start start 2024-05-10T11:35:41.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.864+0000 7f9706352b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556902137810 0x55690212cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:41.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.864+0000 7f9706352b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5569021381d0 0x556902144b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:41.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.865+0000 7f9706352b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55690213b1f0 0x5569021612f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:41.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.865+0000 7f9706352b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f96f41af630 con 0x556902137810 2024-05-10T11:35:41.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.865+0000 7f9706352b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f96f41acec0 con 0x5569021381d0 2024-05-10T11:35:41.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.865+0000 7f9706352b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f96f41aad10 con 0x55690213b1f0 2024-05-10T11:35:41.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.865+0000 7f96f985b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556902137810 0x55690212cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:41.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.865+0000 7f96f985b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556902137810 0x55690212cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47766/0 (socket says 172.21.15.100:47766) 2024-05-10T11:35:41.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.865+0000 7f96f985b700 1 -- 172.21.15.100:0/1015458428 learned_addr learned my addr 172.21.15.100:0/1015458428 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:35:41.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.865+0000 7f96fa05c700 1 --2- 172.21.15.100:0/1015458428 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55690213b1f0 0x5569021612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:41.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.865+0000 7f96f905a700 1 --2- 172.21.15.100:0/1015458428 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5569021381d0 0x556902144b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:41.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.865+0000 7f96fa05c700 1 -- 172.21.15.100:0/1015458428 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5569021381d0 msgr2=0x556902144b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:41.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.865+0000 7f96fa05c700 1 --2- 172.21.15.100:0/1015458428 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5569021381d0 0x556902144b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:41.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.865+0000 7f96fa05c700 1 -- 172.21.15.100:0/1015458428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556902137810 msgr2=0x55690212cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:41.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.865+0000 7f96fa05c700 1 --2- 172.21.15.100:0/1015458428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556902137810 0x55690212cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:41.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.865+0000 7f96fa05c700 1 -- 172.21.15.100:0/1015458428 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556902133390 con 0x55690213b1f0 2024-05-10T11:35:41.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.865+0000 7f96f985b700 1 --2- 172.21.15.100:0/1015458428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556902137810 0x55690212cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:35:41.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.865+0000 7f96fa05c700 1 --2- 172.21.15.100:0/1015458428 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55690213b1f0 0x5569021612f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f96ec0aa3c0 tx=0x7f96ec006230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d4f7990a430c3d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:41.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.866+0000 7f96df7fe700 1 -- 172.21.15.100:0/1015458428 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ec0c3070 con 0x55690213b1f0 2024-05-10T11:35:41.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.866+0000 7f96df7fe700 1 -- 172.21.15.100:0/1015458428 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f96ec0aa1d0 con 0x55690213b1f0 2024-05-10T11:35:41.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.866+0000 7f96df7fe700 1 -- 172.21.15.100:0/1015458428 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ec0aa6c0 con 0x55690213b1f0 2024-05-10T11:35:41.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.866+0000 7f9706352b80 1 -- 172.21.15.100:0/1015458428 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55690213b1f0 msgr2=0x5569021612f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:41.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.866+0000 7f9706352b80 1 --2- 172.21.15.100:0/1015458428 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55690213b1f0 0x5569021612f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f96ec0aa3c0 tx=0x7f96ec006230 comp rx=0 tx=0).stop 2024-05-10T11:35:41.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.866+0000 7f9706352b80 1 -- 172.21.15.100:0/1015458428 shutdown_connections 2024-05-10T11:35:41.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.866+0000 7f9706352b80 1 --2- 172.21.15.100:0/1015458428 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55690213b1f0 0x5569021612f0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:41.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.867+0000 7f9706352b80 1 --2- 172.21.15.100:0/1015458428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556902137810 0x55690212cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:41.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.867+0000 7f9706352b80 1 --2- 172.21.15.100:0/1015458428 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5569021381d0 0x556902144b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:41.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.867+0000 7f9706352b80 1 -- 172.21.15.100:0/1015458428 >> 172.21.15.100:0/1015458428 conn(0x556902063b70 msgr2=0x556902144430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:41.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.867+0000 7f9706352b80 1 -- 172.21.15.100:0/1015458428 shutdown_connections 2024-05-10T11:35:41.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.867+0000 7f9706352b80 1 -- 172.21.15.100:0/1015458428 wait complete. 2024-05-10T11:35:41.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.867+0000 7f9706352b80 1 -- start start 2024-05-10T11:35:41.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.867+0000 7f9706352b80 1 Processor -- start 2024-05-10T11:35:41.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.868+0000 7f9706352b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556902137810 0x556902151df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:41.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.868+0000 7f9706352b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569021381d0 0x556902152330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:41.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.868+0000 7f9706352b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55690213b1f0 0x5569021457b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:41.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.868+0000 7f9706352b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f96f41ab380 con 0x5569021381d0 2024-05-10T11:35:41.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.868+0000 7f9706352b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f96f41ac7f0 con 0x55690213b1f0 2024-05-10T11:35:41.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.868+0000 7f9706352b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f96f41aba50 con 0x556902137810 2024-05-10T11:35:41.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.868+0000 7f9706352b80 10 client.0 _ll_register_callbacks cb 0x55690201d5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:35:41.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.868+0000 7f96f985b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556902137810 0x556902151df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:41.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.868+0000 7f96f985b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556902137810 0x556902151df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:48958/0 (socket says 172.21.15.100:48958) 2024-05-10T11:35:41.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.868+0000 7f96f985b700 1 -- 172.21.15.100:0/184957278 learned_addr learned my addr 172.21.15.100:0/184957278 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:35:41.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.868+0000 7f96fa05c700 1 --2- 172.21.15.100:0/184957278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55690213b1f0 0x5569021457b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:41.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.868+0000 7f96f985b700 1 -- 172.21.15.100:0/184957278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55690213b1f0 msgr2=0x5569021457b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:41.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.868+0000 7f96f985b700 1 --2- 172.21.15.100:0/184957278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55690213b1f0 0x5569021457b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:41.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.868+0000 7f96f985b700 1 -- 172.21.15.100:0/184957278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569021381d0 msgr2=0x556902152330 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T11:35:41.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.868+0000 7f96f985b700 1 --2- 172.21.15.100:0/184957278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569021381d0 0x556902152330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:41.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.868+0000 7f96f985b700 1 -- 172.21.15.100:0/184957278 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556902145cf0 con 0x556902137810 2024-05-10T11:35:41.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.868+0000 7f96f985b700 1 --2- 172.21.15.100:0/184957278 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556902137810 0x556902151df0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f96f001ed20 tx=0x7f96f0018000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ccef83e3cdc1cc5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:41.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.868+0000 7f96dd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:35:41.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.868+0000 7f96dd7fa700 1 -- 172.21.15.100:0/184957278 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f002f060 con 0x556902137810 2024-05-10T11:35:41.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.868+0000 7f9706352b80 1 -- 172.21.15.100:0/184957278 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96ec0aa1d0 con 0x556902137810 2024-05-10T11:35:41.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.868+0000 7f96dd7fa700 1 -- 172.21.15.100:0/184957278 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f96f001fad0 con 0x556902137810 2024-05-10T11:35:41.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.869+0000 7f96dd7fa700 1 -- 172.21.15.100:0/184957278 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f96f0036050 con 0x556902137810 2024-05-10T11:35:41.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.869+0000 7f9706352b80 10 client.28270 fetch_fsmap learned FSMap version 41 2024-05-10T11:35:41.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.869+0000 7f9706352b80 1 -- 172.21.15.100:0/184957278 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5569021c3600 con 0x556902137810 2024-05-10T11:35:41.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.869+0000 7f96dd7fa700 1 -- 172.21.15.100:0/184957278 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f96ec0aa1d0 con 0x556902137810 2024-05-10T11:35:41.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.869+0000 7f96dd7fa700 1 -- 172.21.15.100:0/184957278 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f96f000f450 con 0x556902137810 2024-05-10T11:35:41.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.869+0000 7f9706352b80 10 client.28270 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:35:41.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.869+0000 7f9706352b80 10 client.28270 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:35:41.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.869+0000 7f9706352b80 20 client.28270 populate_metadata read hostname 'smithi100' 2024-05-10T11:35:41.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.869+0000 7f9706352b80 1 --2- 172.21.15.100:0/184957278 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5569021c3cf0 0x5569021c6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:41.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.869+0000 7f9706352b80 4 client.28270 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:35:41.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.869+0000 7f9706352b80 1 -- 172.21.15.100:0/184957278 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5569020913c0 con 0x5569021c3cf0 2024-05-10T11:35:41.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.869+0000 7f96f905a700 1 --2- 172.21.15.100:0/184957278 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5569021c3cf0 0x5569021c6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:41.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.870+0000 7f96f905a700 1 --2- 172.21.15.100:0/184957278 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5569021c3cf0 0x5569021c6110 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bde3490ab2168e26 server_cookie=a6dcd756be0a3ada in_seq=0 out_seq=0 2024-05-10T11:35:41.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.870+0000 7f96dd7fa700 10 client.28270 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:35:41.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.871+0000 7f96dd7fa700 1 -- 172.21.15.100:0/184957278 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5569020913c0 con 0x5569021c3cf0 2024-05-10T11:35:41.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.871+0000 7f96dd7fa700 10 client.28270 handle_command_reply: tid=0 2024-05-10T11:35:41.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.871+0000 7f9706352b80 1 client.28270 shutdown 2024-05-10T11:35:41.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.871+0000 7f9706352b80 1 -- 172.21.15.100:0/184957278 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556902137810 msgr2=0x556902151df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:41.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.871+0000 7f9706352b80 1 --2- 172.21.15.100:0/184957278 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556902137810 0x556902151df0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f96f001ed20 tx=0x7f96f0018000 comp rx=0 tx=0).stop 2024-05-10T11:35:41.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.871+0000 7f9706352b80 1 -- 172.21.15.100:0/184957278 shutdown_connections 2024-05-10T11:35:41.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.871+0000 7f9706352b80 1 --2- 172.21.15.100:0/184957278 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5569021c3cf0 0x5569021c6110 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:41.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.871+0000 7f96dd7fa700 0 client.28270 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:35:41.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.871+0000 7f9706352b80 1 --2- 172.21.15.100:0/184957278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55690213b1f0 0x5569021457b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:41.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.871+0000 7f9706352b80 1 --2- 172.21.15.100:0/184957278 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556902137810 0x556902151df0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:41.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.871+0000 7f9706352b80 1 --2- 172.21.15.100:0/184957278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569021381d0 0x556902152330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:41.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.871+0000 7f9706352b80 1 -- 172.21.15.100:0/184957278 >> 172.21.15.100:0/184957278 conn(0x556902063b70 msgr2=0x556902147e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:41.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.871+0000 7f9706352b80 1 -- 172.21.15.100:0/184957278 shutdown_connections 2024-05-10T11:35:41.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.871+0000 7f9706352b80 1 -- 172.21.15.100:0/184957278 wait complete. 2024-05-10T11:35:41.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.872+0000 7f9706352b80 20 client.28270 trim_cache size 0 max 16384 2024-05-10T11:35:41.875 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:35:41.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.872+0000 7f96fbabf700 1 -- 172.21.15.100:0/3731789437 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f96e00b2900 msgr2=0x7f96e00b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:41.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.872+0000 7f96fbabf700 1 --2- 172.21.15.100:0/3731789437 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f96e00b2900 0x7f96e00b4dc0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f96f419a9d0 tx=0x7f96f0011720 comp rx=0 tx=0).stop 2024-05-10T11:35:41.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.872+0000 7f96fbabf700 1 -- 172.21.15.100:0/3731789437 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96f41095e0 msgr2=0x7f96f4199890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:41.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.872+0000 7f96fbabf700 1 --2- 172.21.15.100:0/3731789437 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96f41095e0 0x7f96f4199890 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f96ec00b0d0 tx=0x7f96ec007a00 comp rx=0 tx=0).stop 2024-05-10T11:35:41.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.872+0000 7f96fbabf700 1 -- 172.21.15.100:0/3731789437 shutdown_connections 2024-05-10T11:35:41.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.872+0000 7f96fbabf700 1 --2- 172.21.15.100:0/3731789437 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f96e00b2900 0x7f96e00b4dc0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:41.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.872+0000 7f96fbabf700 1 --2- 172.21.15.100:0/3731789437 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96f41095e0 0x7f96f4199890 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:41.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.872+0000 7f96fbabf700 1 --2- 172.21.15.100:0/3731789437 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96f4103540 0x7f96f419f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:41.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.872+0000 7f96fbabf700 1 --2- 172.21.15.100:0/3731789437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96f4103f00 0x7f96f419fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:41.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.872+0000 7f96fbabf700 1 -- 172.21.15.100:0/3731789437 >> 172.21.15.100:0/3731789437 conn(0x7f96f407a650 msgr2=0x7f96f4102210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:41.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.873+0000 7f96fbabf700 1 -- 172.21.15.100:0/3731789437 shutdown_connections 2024-05-10T11:35:41.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:41.873+0000 7f96fbabf700 1 -- 172.21.15.100:0/3731789437 wait complete. 2024-05-10T11:35:41.920 INFO:tasks.workunit.client.0.smithi100.stdout: 10 36271 8.89 MB/sec execute 95 sec latency 99.787 ms 2024-05-10T11:35:42.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:35:42 smithi178 ceph-mon[33359]: pgmap v940: 129 pgs: 129 active+clean; 427 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 3.6 MiB/s wr, 278 op/s 2024-05-10T11:35:42.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:35:42 smithi188 ceph-mon[34075]: pgmap v940: 129 pgs: 129 active+clean; 427 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 3.6 MiB/s wr, 278 op/s 2024-05-10T11:35:42.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:35:42 smithi100 ceph-mon[30296]: pgmap v940: 129 pgs: 129 active+clean; 427 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 3.6 MiB/s wr, 278 op/s 2024-05-10T11:35:42.920 INFO:tasks.workunit.client.0.smithi100.stdout: 10 36504 9.02 MB/sec execute 96 sec latency 64.285 ms 2024-05-10T11:35:43.758 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T11:35:43.758 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T11:35:43.920 INFO:tasks.workunit.client.0.smithi100.stdout: 10 36743 9.08 MB/sec execute 97 sec latency 58.606 ms 2024-05-10T11:35:44.138 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:35:44.146 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:35:44 smithi100 ceph-mon[30296]: pgmap v941: 129 pgs: 129 active+clean; 428 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 2.8 MiB/s wr, 280 op/s 2024-05-10T11:35:44.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:35:44 smithi178 ceph-mon[33359]: pgmap v941: 129 pgs: 129 active+clean; 428 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 2.8 MiB/s wr, 280 op/s 2024-05-10T11:35:44.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:35:44 smithi188 ceph-mon[34075]: pgmap v941: 129 pgs: 129 active+clean; 428 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 2.8 MiB/s wr, 280 op/s 2024-05-10T11:35:44.920 INFO:tasks.workunit.client.0.smithi100.stdout: 10 36931 9.19 MB/sec execute 98 sec latency 72.699 ms 2024-05-10T11:35:45.920 INFO:tasks.workunit.client.0.smithi100.stdout: 10 37120 9.20 MB/sec execute 99 sec latency 90.429 ms 2024-05-10T11:35:46.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.129+0000 7f976f9d4700 1 -- 172.21.15.100:0/3171762778 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9768108410 msgr2=0x7f97681087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:46.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.129+0000 7f976f9d4700 1 --2- 172.21.15.100:0/3171762778 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9768108410 0x7f97681087f0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f9764009990 tx=0x7f9764009ca0 comp rx=0 tx=0).stop 2024-05-10T11:35:46.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.130+0000 7f976f9d4700 1 -- 172.21.15.100:0/3171762778 shutdown_connections 2024-05-10T11:35:46.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.130+0000 7f976f9d4700 1 --2- 172.21.15.100:0/3171762778 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9768102d60 0x7f976810f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:46.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.130+0000 7f976f9d4700 1 --2- 172.21.15.100:0/3171762778 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9768108410 0x7f97681087f0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:46.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.130+0000 7f976f9d4700 1 --2- 172.21.15.100:0/3171762778 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97681023a0 0x7f9768102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:46.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.130+0000 7f976f9d4700 1 -- 172.21.15.100:0/3171762778 >> 172.21.15.100:0/3171762778 conn(0x7f9768079350 msgr2=0x7f9768079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:46.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.130+0000 7f976f9d4700 1 -- 172.21.15.100:0/3171762778 shutdown_connections 2024-05-10T11:35:46.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.130+0000 7f976f9d4700 1 -- 172.21.15.100:0/3171762778 wait complete. 2024-05-10T11:35:46.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.131+0000 7f976f9d4700 1 Processor -- start 2024-05-10T11:35:46.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.131+0000 7f976f9d4700 1 -- start start 2024-05-10T11:35:46.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.132+0000 7f976f9d4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97681023a0 0x7f976819f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:46.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.132+0000 7f976f9d4700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9768102d60 0x7f976819fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:46.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.132+0000 7f976f9d4700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9768108410 0x7f9768199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:46.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.132+0000 7f976d770700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97681023a0 0x7f976819f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:46.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.132+0000 7f976d770700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97681023a0 0x7f976819f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47776/0 (socket says 172.21.15.100:47776) 2024-05-10T11:35:46.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.132+0000 7f976d770700 1 -- 172.21.15.100:0/1663188204 learned_addr learned my addr 172.21.15.100:0/1663188204 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:35:46.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.132+0000 7f976f9d4700 1 -- 172.21.15.100:0/1663188204 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9768113500 con 0x7f97681023a0 2024-05-10T11:35:46.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.132+0000 7f976f9d4700 1 -- 172.21.15.100:0/1663188204 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9768113380 con 0x7f9768102d60 2024-05-10T11:35:46.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.132+0000 7f976f9d4700 1 -- 172.21.15.100:0/1663188204 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9768113680 con 0x7f9768108410 2024-05-10T11:35:46.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.132+0000 7f976df71700 1 --2- 172.21.15.100:0/1663188204 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9768108410 0x7f9768199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:46.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.132+0000 7f976d770700 1 -- 172.21.15.100:0/1663188204 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9768108410 msgr2=0x7f9768199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:46.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.132+0000 7f976d770700 1 --2- 172.21.15.100:0/1663188204 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9768108410 0x7f9768199840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:46.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.132+0000 7f976d770700 1 -- 172.21.15.100:0/1663188204 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9768102d60 msgr2=0x7f976819fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:46.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.132+0000 7f976d770700 1 --2- 172.21.15.100:0/1663188204 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9768102d60 0x7f976819fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:46.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.132+0000 7f976d770700 1 -- 172.21.15.100:0/1663188204 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f976819a020 con 0x7f97681023a0 2024-05-10T11:35:46.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.133+0000 7f976d770700 1 --2- 172.21.15.100:0/1663188204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97681023a0 0x7f976819f6c0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f976400b2e0 tx=0x7f976400bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:46.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.133+0000 7f975e7fc700 1 -- 172.21.15.100:0/1663188204 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9764026070 con 0x7f97681023a0 2024-05-10T11:35:46.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.133+0000 7f975e7fc700 1 -- 172.21.15.100:0/1663188204 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9764021410 con 0x7f97681023a0 2024-05-10T11:35:46.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.133+0000 7f975e7fc700 1 -- 172.21.15.100:0/1663188204 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9764017650 con 0x7f97681023a0 2024-05-10T11:35:46.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.133+0000 7f976f9d4700 1 -- 172.21.15.100:0/1663188204 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f976819a2b0 con 0x7f97681023a0 2024-05-10T11:35:46.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.133+0000 7f976f9d4700 1 -- 172.21.15.100:0/1663188204 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97681a6520 con 0x7f97681023a0 2024-05-10T11:35:46.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.137+0000 7f975e7fc700 1 -- 172.21.15.100:0/1663188204 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f9764029070 con 0x7f97681023a0 2024-05-10T11:35:46.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.138+0000 7f975e7fc700 1 --2- 172.21.15.100:0/1663188204 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f97540b2900 0x7f97540b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:46.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.138+0000 7f977a267b80 1 Processor -- start 2024-05-10T11:35:46.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.139+0000 7f976cf6f700 1 --2- 172.21.15.100:0/1663188204 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f97540b2900 0x7f97540b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:46.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.139+0000 7f975e7fc700 1 -- 172.21.15.100:0/1663188204 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f97640e3380 con 0x7f97681023a0 2024-05-10T11:35:46.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.139+0000 7f977a267b80 1 -- start start 2024-05-10T11:35:46.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.139+0000 7f977a267b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573dac74550 0x5573dac74930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:46.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.139+0000 7f977a267b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5573dac74e70 0x5573dac78770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:46.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.139+0000 7f977a267b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5573dac78d00 0x5573dac79180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:46.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.139+0000 7f977a267b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5573dabc68e0 con 0x5573dac74550 2024-05-10T11:35:46.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.139+0000 7f977a267b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5573dab65840 con 0x5573dac74e70 2024-05-10T11:35:46.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.139+0000 7f977a267b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5573dabc28a0 con 0x5573dac78d00 2024-05-10T11:35:46.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.139+0000 7f976df71700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5573dac78d00 0x5573dac79180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:46.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.139+0000 7f976d770700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573dac74550 0x5573dac74930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:46.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.139+0000 7f976d770700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573dac74550 0x5573dac74930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47778/0 (socket says 172.21.15.100:47778) 2024-05-10T11:35:46.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.139+0000 7f976d770700 1 -- 172.21.15.100:0/993961376 learned_addr learned my addr 172.21.15.100:0/993961376 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:35:46.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.139+0000 7f976df71700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5573dac78d00 0x5573dac79180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:48976/0 (socket says 172.21.15.100:48976) 2024-05-10T11:35:46.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.140+0000 7f976cf6f700 1 --2- 172.21.15.100:0/1663188204 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f97540b2900 0x7f97540b4dc0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f9758005950 tx=0x7f97580058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:46.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.140+0000 7f976cf6f700 1 --2- 172.21.15.100:0/993961376 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5573dac74e70 0x5573dac78770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:46.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.140+0000 7f976df71700 1 -- 172.21.15.100:0/993961376 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5573dac74e70 msgr2=0x5573dac78770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:46.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.140+0000 7f976df71700 1 --2- 172.21.15.100:0/993961376 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5573dac74e70 0x5573dac78770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:46.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.140+0000 7f976df71700 1 -- 172.21.15.100:0/993961376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573dac74550 msgr2=0x5573dac74930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:46.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.140+0000 7f976df71700 1 --2- 172.21.15.100:0/993961376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573dac74550 0x5573dac74930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:46.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.140+0000 7f976df71700 1 -- 172.21.15.100:0/993961376 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573dab13950 con 0x5573dac78d00 2024-05-10T11:35:46.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.140+0000 7f976d770700 1 --2- 172.21.15.100:0/993961376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573dac74550 0x5573dac74930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:35:46.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.140+0000 7f976df71700 1 --2- 172.21.15.100:0/993961376 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5573dac78d00 0x5573dac79180 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9760011580 tx=0x7f9760009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef40ead460f2667a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:46.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.140+0000 7f97537fe700 1 -- 172.21.15.100:0/993961376 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9760011e80 con 0x5573dac78d00 2024-05-10T11:35:46.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.140+0000 7f97537fe700 1 -- 172.21.15.100:0/993961376 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9760004bb0 con 0x5573dac78d00 2024-05-10T11:35:46.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.140+0000 7f97537fe700 1 -- 172.21.15.100:0/993961376 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9760004640 con 0x5573dac78d00 2024-05-10T11:35:46.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.141+0000 7f977a267b80 1 -- 172.21.15.100:0/993961376 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5573dac78d00 msgr2=0x5573dac79180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:46.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.141+0000 7f977a267b80 1 --2- 172.21.15.100:0/993961376 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5573dac78d00 0x5573dac79180 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9760011580 tx=0x7f9760009920 comp rx=0 tx=0).stop 2024-05-10T11:35:46.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.141+0000 7f977a267b80 1 -- 172.21.15.100:0/993961376 shutdown_connections 2024-05-10T11:35:46.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.141+0000 7f977a267b80 1 --2- 172.21.15.100:0/993961376 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5573dac78d00 0x5573dac79180 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:46.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.142+0000 7f977a267b80 1 --2- 172.21.15.100:0/993961376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573dac74550 0x5573dac74930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:46.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.142+0000 7f977a267b80 1 --2- 172.21.15.100:0/993961376 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5573dac74e70 0x5573dac78770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:46.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.142+0000 7f977a267b80 1 -- 172.21.15.100:0/993961376 >> 172.21.15.100:0/993961376 conn(0x5573daba0180 msgr2=0x5573dac6ecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:46.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.142+0000 7f977a267b80 1 -- 172.21.15.100:0/993961376 shutdown_connections 2024-05-10T11:35:46.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.142+0000 7f977a267b80 1 -- 172.21.15.100:0/993961376 wait complete. 2024-05-10T11:35:46.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.142+0000 7f977a267b80 1 -- start start 2024-05-10T11:35:46.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.142+0000 7f977a267b80 1 Processor -- start 2024-05-10T11:35:46.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.143+0000 7f977a267b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5573dac74550 0x5573dac7ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:46.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.143+0000 7f977a267b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5573dac74e70 0x5573dac7d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:46.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.143+0000 7f977a267b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573dac78d00 0x5573dac7d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:46.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.143+0000 7f976d770700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5573dac74550 0x5573dac7ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:46.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.143+0000 7f976d770700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5573dac74550 0x5573dac7ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:48984/0 (socket says 172.21.15.100:48984) 2024-05-10T11:35:46.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.143+0000 7f976d770700 1 -- 172.21.15.100:0/2064699596 learned_addr learned my addr 172.21.15.100:0/2064699596 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:35:46.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.143+0000 7f976cf6f700 1 --2- 172.21.15.100:0/2064699596 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5573dac74e70 0x5573dac7d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:46.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.143+0000 7f977a267b80 1 -- 172.21.15.100:0/2064699596 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5573dac80f60 con 0x5573dac78d00 2024-05-10T11:35:46.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.143+0000 7f977a267b80 1 -- 172.21.15.100:0/2064699596 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5573dab2ccb0 con 0x5573dac74e70 2024-05-10T11:35:46.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.143+0000 7f977a267b80 1 -- 172.21.15.100:0/2064699596 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5573dac810e0 con 0x5573dac74550 2024-05-10T11:35:46.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.143+0000 7f976df71700 1 --2- 172.21.15.100:0/2064699596 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573dac78d00 0x5573dac7d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:46.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.143+0000 7f977a267b80 10 client.0 _ll_register_callbacks cb 0x5573dab5e9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:35:46.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.144+0000 7f976df71700 1 -- 172.21.15.100:0/2064699596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5573dac74550 msgr2=0x5573dac7ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:46.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.144+0000 7f976df71700 1 --2- 172.21.15.100:0/2064699596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5573dac74550 0x5573dac7ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:46.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.144+0000 7f976df71700 1 -- 172.21.15.100:0/2064699596 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5573dac74e70 msgr2=0x5573dac7d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:46.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.144+0000 7f976df71700 1 --2- 172.21.15.100:0/2064699596 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5573dac74e70 0x5573dac7d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:46.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.144+0000 7f976df71700 1 -- 172.21.15.100:0/2064699596 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573dab62710 con 0x5573dac78d00 2024-05-10T11:35:46.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.144+0000 7f976df71700 1 --2- 172.21.15.100:0/2064699596 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573dac78d00 0x5573dac7d890 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f9760004260 tx=0x7f9760004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3421ab3c4c7e4a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:46.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.144+0000 7f9750ff9700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:35:46.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.144+0000 7f9750ff9700 1 -- 172.21.15.100:0/2064699596 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976002b060 con 0x5573dac78d00 2024-05-10T11:35:46.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.144+0000 7f9750ff9700 1 -- 172.21.15.100:0/2064699596 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f9760003e10 con 0x5573dac78d00 2024-05-10T11:35:46.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.145+0000 7f977a267b80 1 -- 172.21.15.100:0/2064699596 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9760004bb0 con 0x5573dac78d00 2024-05-10T11:35:46.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.145+0000 7f9750ff9700 1 -- 172.21.15.100:0/2064699596 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f9760032050 con 0x5573dac78d00 2024-05-10T11:35:46.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.145+0000 7f977a267b80 10 client.18873 fetch_fsmap learned FSMap version 41 2024-05-10T11:35:46.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.145+0000 7f977a267b80 1 -- 172.21.15.100:0/2064699596 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5573dac8fa10 con 0x5573dac78d00 2024-05-10T11:35:46.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.145+0000 7f9750ff9700 1 -- 172.21.15.100:0/2064699596 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9760004bb0 con 0x5573dac78d00 2024-05-10T11:35:46.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.145+0000 7f9750ff9700 1 -- 172.21.15.100:0/2064699596 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f9760025030 con 0x5573dac78d00 2024-05-10T11:35:46.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.145+0000 7f977a267b80 10 client.18873 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:35:46.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.145+0000 7f977a267b80 10 client.18873 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:35:46.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.145+0000 7f977a267b80 20 client.18873 populate_metadata read hostname 'smithi100' 2024-05-10T11:35:46.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.145+0000 7f977a267b80 1 --2- 172.21.15.100:0/2064699596 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5573dac92390 0x5573dac94830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:46.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.145+0000 7f977a267b80 4 client.18873 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:35:46.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.145+0000 7f977a267b80 1 -- 172.21.15.100:0/2064699596 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5573dac406d0 con 0x5573dac92390 2024-05-10T11:35:46.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.146+0000 7f976d770700 1 --2- 172.21.15.100:0/2064699596 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5573dac92390 0x5573dac94830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:46.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.146+0000 7f9750ff9700 10 client.18873 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:35:46.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.146+0000 7f976d770700 1 --2- 172.21.15.100:0/2064699596 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5573dac92390 0x5573dac94830 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e4fb9948f0344d1 server_cookie=b1a9ec454f08a005 in_seq=0 out_seq=0 2024-05-10T11:35:46.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.147+0000 7f9750ff9700 1 -- 172.21.15.100:0/2064699596 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5573dac406d0 con 0x5573dac92390 2024-05-10T11:35:46.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.147+0000 7f9750ff9700 10 client.18873 handle_command_reply: tid=0 2024-05-10T11:35:46.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.147+0000 7f977a267b80 1 client.18873 shutdown 2024-05-10T11:35:46.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.147+0000 7f977a267b80 1 -- 172.21.15.100:0/2064699596 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573dac78d00 msgr2=0x5573dac7d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:46.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.147+0000 7f977a267b80 1 --2- 172.21.15.100:0/2064699596 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573dac78d00 0x5573dac7d890 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f9760004260 tx=0x7f9760004ad0 comp rx=0 tx=0).stop 2024-05-10T11:35:46.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.147+0000 7f977a267b80 1 -- 172.21.15.100:0/2064699596 shutdown_connections 2024-05-10T11:35:46.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.147+0000 7f977a267b80 1 --2- 172.21.15.100:0/2064699596 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5573dac92390 0x5573dac94830 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:46.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.148+0000 7f977a267b80 1 --2- 172.21.15.100:0/2064699596 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573dac78d00 0x5573dac7d890 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:46.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.148+0000 7f9750ff9700 0 client.18873 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:35:46.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.148+0000 7f977a267b80 1 --2- 172.21.15.100:0/2064699596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5573dac74550 0x5573dac7ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:46.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.148+0000 7f977a267b80 1 --2- 172.21.15.100:0/2064699596 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5573dac74e70 0x5573dac7d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:46.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.148+0000 7f977a267b80 1 -- 172.21.15.100:0/2064699596 >> 172.21.15.100:0/2064699596 conn(0x5573daba0180 msgr2=0x5573dac79100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:46.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.148+0000 7f977a267b80 1 -- 172.21.15.100:0/2064699596 shutdown_connections 2024-05-10T11:35:46.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.148+0000 7f977a267b80 1 -- 172.21.15.100:0/2064699596 wait complete. 2024-05-10T11:35:46.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.148+0000 7f977a267b80 20 client.18873 trim_cache size 0 max 16384 2024-05-10T11:35:46.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.177+0000 7f977a267b80 1 Processor -- start 2024-05-10T11:35:46.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.177+0000 7f977a267b80 1 -- start start 2024-05-10T11:35:46.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.177+0000 7f977a267b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5573dac74550 0x5573dac69620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:46.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.177+0000 7f977a267b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5573dac74e70 0x5573dac76670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:46.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.177+0000 7f977a267b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573dac78d00 0x5573dac76bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:46.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.178+0000 7f977a267b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f97681af5d0 con 0x5573dac78d00 2024-05-10T11:35:46.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.178+0000 7f977a267b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f97681ace60 con 0x5573dac74550 2024-05-10T11:35:46.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.178+0000 7f977a267b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f97681aacb0 con 0x5573dac74e70 2024-05-10T11:35:46.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.178+0000 7f976df71700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573dac78d00 0x5573dac76bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:46.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.178+0000 7f976df71700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573dac78d00 0x5573dac76bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47804/0 (socket says 172.21.15.100:47804) 2024-05-10T11:35:46.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.178+0000 7f976df71700 1 -- 172.21.15.100:0/343331943 learned_addr learned my addr 172.21.15.100:0/343331943 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:35:46.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.178+0000 7f976cf6f700 1 --2- 172.21.15.100:0/343331943 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5573dac74e70 0x5573dac76670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:46.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.178+0000 7f976d770700 1 --2- 172.21.15.100:0/343331943 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5573dac74550 0x5573dac69620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:46.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.178+0000 7f976d770700 1 -- 172.21.15.100:0/343331943 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5573dac74e70 msgr2=0x5573dac76670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:46.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.178+0000 7f976d770700 1 --2- 172.21.15.100:0/343331943 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5573dac74e70 0x5573dac76670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:46.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.178+0000 7f976d770700 1 -- 172.21.15.100:0/343331943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573dac78d00 msgr2=0x5573dac76bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:46.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.178+0000 7f976d770700 1 --2- 172.21.15.100:0/343331943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573dac78d00 0x5573dac76bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:46.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.178+0000 7f976d770700 1 -- 172.21.15.100:0/343331943 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573dac7b8c0 con 0x5573dac74550 2024-05-10T11:35:46.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.178+0000 7f976df71700 1 --2- 172.21.15.100:0/343331943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573dac78d00 0x5573dac76bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:35:46.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.178+0000 7f976d770700 1 --2- 172.21.15.100:0/343331943 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5573dac74550 0x5573dac69620 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x5573dac950e0 tx=0x7f97640e09f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc765bea2b42b1db server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:46.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.178+0000 7f9752ffd700 1 -- 172.21.15.100:0/343331943 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9764009590 con 0x5573dac74550 2024-05-10T11:35:46.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.179+0000 7f9752ffd700 1 -- 172.21.15.100:0/343331943 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f97640e0c30 con 0x5573dac74550 2024-05-10T11:35:46.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.179+0000 7f9752ffd700 1 -- 172.21.15.100:0/343331943 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97640aa930 con 0x5573dac74550 2024-05-10T11:35:46.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.179+0000 7f977a267b80 1 -- 172.21.15.100:0/343331943 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5573dac74550 msgr2=0x5573dac69620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:46.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.179+0000 7f977a267b80 1 --2- 172.21.15.100:0/343331943 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5573dac74550 0x5573dac69620 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x5573dac950e0 tx=0x7f97640e09f0 comp rx=0 tx=0).stop 2024-05-10T11:35:46.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.179+0000 7f977a267b80 1 -- 172.21.15.100:0/343331943 shutdown_connections 2024-05-10T11:35:46.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.179+0000 7f977a267b80 1 --2- 172.21.15.100:0/343331943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573dac78d00 0x5573dac76bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:46.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.179+0000 7f977a267b80 1 --2- 172.21.15.100:0/343331943 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5573dac74550 0x5573dac69620 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:46.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.179+0000 7f977a267b80 1 --2- 172.21.15.100:0/343331943 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5573dac74e70 0x5573dac76670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:46.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.179+0000 7f977a267b80 1 -- 172.21.15.100:0/343331943 >> 172.21.15.100:0/343331943 conn(0x5573daba0180 msgr2=0x5573dac68f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:46.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.180+0000 7f977a267b80 1 -- 172.21.15.100:0/343331943 shutdown_connections 2024-05-10T11:35:46.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.180+0000 7f977a267b80 1 -- 172.21.15.100:0/343331943 wait complete. 2024-05-10T11:35:46.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.180+0000 7f977a267b80 1 -- start start 2024-05-10T11:35:46.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.180+0000 7f977a267b80 1 Processor -- start 2024-05-10T11:35:46.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.181+0000 7f977a267b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5573dac74550 0x5573dac99900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:46.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.181+0000 7f977a267b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5573dac74e70 0x5573dac99e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:46.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.181+0000 7f977a267b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573dac78d00 0x5573dac9a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:46.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.181+0000 7f977a267b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f97681ab320 con 0x5573dac78d00 2024-05-10T11:35:46.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.181+0000 7f977a267b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f97681ac790 con 0x5573dac74550 2024-05-10T11:35:46.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.181+0000 7f977a267b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f97681ab9f0 con 0x5573dac74e70 2024-05-10T11:35:46.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.181+0000 7f977a267b80 10 client.0 _ll_register_callbacks cb 0x5573dab5e9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:35:46.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.181+0000 7f976d770700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5573dac74550 0x5573dac99900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:46.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.181+0000 7f976cf6f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5573dac74e70 0x5573dac99e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:46.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.181+0000 7f976cf6f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5573dac74e70 0x5573dac99e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:49004/0 (socket says 172.21.15.100:49004) 2024-05-10T11:35:46.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.181+0000 7f976d770700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5573dac74550 0x5573dac99900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:54614/0 (socket says 172.21.15.100:54614) 2024-05-10T11:35:46.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.181+0000 7f976cf6f700 1 -- 172.21.15.100:0/308262299 learned_addr learned my addr 172.21.15.100:0/308262299 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:35:46.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.181+0000 7f976df71700 1 --2- 172.21.15.100:0/308262299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573dac78d00 0x5573dac9a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:46.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.181+0000 7f976cf6f700 1 -- 172.21.15.100:0/308262299 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5573dac74550 msgr2=0x5573dac99900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:46.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.181+0000 7f976cf6f700 1 --2- 172.21.15.100:0/308262299 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5573dac74550 0x5573dac99900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:46.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.181+0000 7f976cf6f700 1 -- 172.21.15.100:0/308262299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573dac78d00 msgr2=0x5573dac9a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:46.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.181+0000 7f976cf6f700 1 --2- 172.21.15.100:0/308262299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573dac78d00 0x5573dac9a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:46.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.181+0000 7f976cf6f700 1 -- 172.21.15.100:0/308262299 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573dacfbcf0 con 0x5573dac74e70 2024-05-10T11:35:46.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.182+0000 7f976cf6f700 1 --2- 172.21.15.100:0/308262299 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5573dac74e70 0x5573dac99e40 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9768079a30 tx=0x7f9758004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e05f76da356da14d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:46.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.182+0000 7f9750ff9700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:35:46.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.182+0000 7f9750ff9700 1 -- 172.21.15.100:0/308262299 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9758004830 con 0x5573dac74e70 2024-05-10T11:35:46.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.182+0000 7f977a267b80 1 -- 172.21.15.100:0/308262299 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97640e0c30 con 0x5573dac74e70 2024-05-10T11:35:46.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.182+0000 7f9750ff9700 1 -- 172.21.15.100:0/308262299 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f97580049d0 con 0x5573dac74e70 2024-05-10T11:35:46.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.182+0000 7f9750ff9700 1 -- 172.21.15.100:0/308262299 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f9758032050 con 0x5573dac74e70 2024-05-10T11:35:46.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.182+0000 7f977a267b80 10 client.28288 fetch_fsmap learned FSMap version 41 2024-05-10T11:35:46.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.182+0000 7f977a267b80 1 -- 172.21.15.100:0/308262299 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5573dacfd070 con 0x5573dac74e70 2024-05-10T11:35:46.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.182+0000 7f9750ff9700 1 -- 172.21.15.100:0/308262299 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f97580054b0 con 0x5573dac74e70 2024-05-10T11:35:46.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.183+0000 7f9750ff9700 1 -- 172.21.15.100:0/308262299 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f9758026070 con 0x5573dac74e70 2024-05-10T11:35:46.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.183+0000 7f977a267b80 10 client.28288 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:35:46.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.183+0000 7f977a267b80 10 client.28288 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:35:46.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.183+0000 7f977a267b80 20 client.28288 populate_metadata read hostname 'smithi100' 2024-05-10T11:35:46.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.183+0000 7f977a267b80 1 --2- 172.21.15.100:0/308262299 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5573dac92390 0x5573dacff820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:46.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.183+0000 7f977a267b80 4 client.28288 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T11:35:46.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.183+0000 7f977a267b80 1 -- 172.21.15.100:0/308262299 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f97640e0c30 con 0x5573dac92390 2024-05-10T11:35:46.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.183+0000 7f976d770700 1 --2- 172.21.15.100:0/308262299 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5573dac92390 0x5573dacff820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:46.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.183+0000 7f9750ff9700 10 client.28288 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:35:46.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.184+0000 7f976d770700 1 --2- 172.21.15.100:0/308262299 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5573dac92390 0x5573dacff820 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff202fd237ca622e server_cookie=8eeb4f1839808314 in_seq=0 out_seq=0 2024-05-10T11:35:46.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.185+0000 7f9750ff9700 1 -- 172.21.15.100:0/308262299 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f97640e0c30 con 0x5573dac92390 2024-05-10T11:35:46.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.185+0000 7f9750ff9700 10 client.28288 handle_command_reply: tid=0 2024-05-10T11:35:46.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.185+0000 7f977a267b80 1 client.28288 shutdown 2024-05-10T11:35:46.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.185+0000 7f977a267b80 1 -- 172.21.15.100:0/308262299 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5573dac74e70 msgr2=0x5573dac99e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:46.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.185+0000 7f977a267b80 1 --2- 172.21.15.100:0/308262299 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5573dac74e70 0x5573dac99e40 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9768079a30 tx=0x7f9758004750 comp rx=0 tx=0).stop 2024-05-10T11:35:46.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.185+0000 7f977a267b80 1 -- 172.21.15.100:0/308262299 shutdown_connections 2024-05-10T11:35:46.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.185+0000 7f977a267b80 1 --2- 172.21.15.100:0/308262299 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5573dac92390 0x5573dacff820 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:46.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.185+0000 7f977a267b80 1 --2- 172.21.15.100:0/308262299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573dac78d00 0x5573dac9a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:46.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.186+0000 7f977a267b80 1 --2- 172.21.15.100:0/308262299 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5573dac74550 0x5573dac99900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:46.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.186+0000 7f977a267b80 1 --2- 172.21.15.100:0/308262299 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5573dac74e70 0x5573dac99e40 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:46.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.186+0000 7f977a267b80 1 -- 172.21.15.100:0/308262299 >> 172.21.15.100:0/308262299 conn(0x5573daba0180 msgr2=0x5573dac697a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:46.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.186+0000 7f9750ff9700 0 client.28288 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:35:46.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.186+0000 7f977a267b80 1 -- 172.21.15.100:0/308262299 shutdown_connections 2024-05-10T11:35:46.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.186+0000 7f977a267b80 1 -- 172.21.15.100:0/308262299 wait complete. 2024-05-10T11:35:46.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.186+0000 7f977a267b80 20 client.28288 trim_cache size 0 max 16384 2024-05-10T11:35:46.189 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:35:46.189 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-05-10T11:35:46.189 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "8d2a3db4-b712-4de6-b636-e155fc51fdc2", 2024-05-10T11:35:46.189 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-05-10T11:35:46.189 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:35:46.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.187+0000 7f976f9d4700 1 -- 172.21.15.100:0/1663188204 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f97540b2900 msgr2=0x7f97540b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:46.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.187+0000 7f976f9d4700 1 --2- 172.21.15.100:0/1663188204 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f97540b2900 0x7f97540b4dc0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f9758005950 tx=0x7f97580058e0 comp rx=0 tx=0).stop 2024-05-10T11:35:46.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.187+0000 7f976f9d4700 1 -- 172.21.15.100:0/1663188204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97681023a0 msgr2=0x7f976819f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:46.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.187+0000 7f976f9d4700 1 --2- 172.21.15.100:0/1663188204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97681023a0 0x7f976819f6c0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f976400b2e0 tx=0x7f976400bfd0 comp rx=0 tx=0).stop 2024-05-10T11:35:46.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.187+0000 7f976f9d4700 1 -- 172.21.15.100:0/1663188204 shutdown_connections 2024-05-10T11:35:46.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.187+0000 7f976f9d4700 1 --2- 172.21.15.100:0/1663188204 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f97540b2900 0x7f97540b4dc0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:46.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.187+0000 7f976f9d4700 1 --2- 172.21.15.100:0/1663188204 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9768108410 0x7f9768199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:46.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.187+0000 7f976f9d4700 1 --2- 172.21.15.100:0/1663188204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97681023a0 0x7f976819f6c0 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:46.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.187+0000 7f976f9d4700 1 --2- 172.21.15.100:0/1663188204 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9768102d60 0x7f976819fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:46.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.187+0000 7f976f9d4700 1 -- 172.21.15.100:0/1663188204 >> 172.21.15.100:0/1663188204 conn(0x7f9768079350 msgr2=0x7f9768101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:46.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.187+0000 7f976f9d4700 1 -- 172.21.15.100:0/1663188204 shutdown_connections 2024-05-10T11:35:46.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:46.187+0000 7f976f9d4700 1 -- 172.21.15.100:0/1663188204 wait complete. 2024-05-10T11:35:46.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:35:46 smithi100 ceph-mon[30296]: pgmap v942: 129 pgs: 129 active+clean; 434 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 3.6 MiB/s wr, 279 op/s 2024-05-10T11:35:46.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:35:46 smithi178 ceph-mon[33359]: pgmap v942: 129 pgs: 129 active+clean; 434 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 3.6 MiB/s wr, 279 op/s 2024-05-10T11:35:46.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:35:46 smithi188 ceph-mon[34075]: pgmap v942: 129 pgs: 129 active+clean; 434 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 3.6 MiB/s wr, 279 op/s 2024-05-10T11:35:46.920 INFO:tasks.workunit.client.0.smithi100.stdout: 10 37319 9.25 MB/sec execute 100 sec latency 101.516 ms 2024-05-10T11:35:47.133 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T11:35:47.489 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:35:47.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:35:47 smithi178 ceph-mon[33359]: scrub queued for path: / 2024-05-10T11:35:47.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:35:47 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:35:47.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:35:47 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:35:47.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:35:47 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:35:47.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:35:47 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:35:47.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:35:47 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:35:47.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:35:47 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:35:47.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:35:47 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:35:47.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:35:47 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:35:47.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:35:47 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:35:47.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:35:47 smithi188 ceph-mon[34075]: scrub queued for path: / 2024-05-10T11:35:47.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:35:47 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:35:47.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:35:47 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:35:47.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:35:47 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:35:47.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:35:47 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:35:47.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:35:47 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:35:47.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:35:47 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:35:47.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:35:47 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:35:47.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:35:47 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:35:47.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:35:47 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:35:47.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:35:47 smithi100 ceph-mon[30296]: scrub queued for path: / 2024-05-10T11:35:47.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:35:47 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:35:47.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:35:47 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:35:47.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:35:47 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:35:47.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:35:47 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:35:47.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:35:47 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:35:47.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:35:47 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:35:47.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:35:47 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:35:47.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:35:47 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:35:47.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:35:47 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:35:47.920 INFO:tasks.workunit.client.0.smithi100.stdout: 10 37448 9.21 MB/sec execute 101 sec latency 66.409 ms 2024-05-10T11:35:48.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:35:48 smithi178 ceph-mon[33359]: pgmap v943: 129 pgs: 129 active+clean; 440 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 4.5 MiB/s wr, 286 op/s 2024-05-10T11:35:48.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:35:48 smithi188 ceph-mon[34075]: pgmap v943: 129 pgs: 129 active+clean; 440 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 4.5 MiB/s wr, 286 op/s 2024-05-10T11:35:48.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:35:48 smithi100 ceph-mon[30296]: pgmap v943: 129 pgs: 129 active+clean; 440 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 4.5 MiB/s wr, 286 op/s 2024-05-10T11:35:48.920 INFO:tasks.workunit.client.0.smithi100.stdout: 10 37593 9.20 MB/sec execute 102 sec latency 64.369 ms 2024-05-10T11:35:48.970 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.967+0000 7fea6da83640 1 -- 172.21.15.100:0/3627666516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea681034c0 msgr2=0x7fea68109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:48.970 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.967+0000 7fea6da83640 1 --2- 172.21.15.100:0/3627666516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea681034c0 0x7fea68109d50 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fea60009ca0 tx=0x7fea6002a8c0 comp rx=0 tx=0).stop 2024-05-10T11:35:48.970 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.967+0000 7fea6da83640 1 -- 172.21.15.100:0/3627666516 shutdown_connections 2024-05-10T11:35:48.970 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.967+0000 7fea6da83640 1 --2- 172.21.15.100:0/3627666516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea681034c0 0x7fea68109d50 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:48.970 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.967+0000 7fea6da83640 1 --2- 172.21.15.100:0/3627666516 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea68102b00 0x7fea68102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:48.970 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.967+0000 7fea6da83640 1 --2- 172.21.15.100:0/3627666516 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea68101910 0x7fea68101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:48.970 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.967+0000 7fea6da83640 1 -- 172.21.15.100:0/3627666516 >> 172.21.15.100:0/3627666516 conn(0x7fea680fcea0 msgr2=0x7fea680ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:48.970 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.967+0000 7fea6da83640 1 -- 172.21.15.100:0/3627666516 shutdown_connections 2024-05-10T11:35:48.970 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.967+0000 7fea6da83640 1 -- 172.21.15.100:0/3627666516 wait complete. 2024-05-10T11:35:48.970 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.968+0000 7fea6da83640 1 Processor -- start 2024-05-10T11:35:48.971 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.968+0000 7fea6da83640 1 -- start start 2024-05-10T11:35:48.971 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.968+0000 7fea6da83640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea68101910 0x7fea6819aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:48.971 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.968+0000 7fea6da83640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea68102b00 0x7fea6819b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:48.971 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.968+0000 7fea6da83640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea681034c0 0x7fea681a2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:48.971 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.968+0000 7fea6da83640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fea6810d320 con 0x7fea68101910 2024-05-10T11:35:48.971 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.968+0000 7fea6da83640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fea6810d1a0 con 0x7fea681034c0 2024-05-10T11:35:48.971 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.968+0000 7fea6da83640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fea6810d4a0 con 0x7fea68102b00 2024-05-10T11:35:48.971 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.968+0000 7fea677fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea68101910 0x7fea6819aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:48.971 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.968+0000 7fea677fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea68101910 0x7fea6819aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44890/0 (socket says 172.21.15.100:44890) 2024-05-10T11:35:48.971 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.968+0000 7fea677fe640 1 -- 172.21.15.100:0/3470102452 learned_addr learned my addr 172.21.15.100:0/3470102452 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:35:48.971 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.968+0000 7fea5ffff640 1 --2- 172.21.15.100:0/3470102452 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea68102b00 0x7fea6819b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:48.971 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.968+0000 7fea67fff640 1 --2- 172.21.15.100:0/3470102452 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea681034c0 0x7fea681a2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:48.971 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.968+0000 7fea677fe640 1 -- 172.21.15.100:0/3470102452 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea68102b00 msgr2=0x7fea6819b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:48.971 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.968+0000 7fea677fe640 1 --2- 172.21.15.100:0/3470102452 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea68102b00 0x7fea6819b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:48.972 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.968+0000 7fea677fe640 1 -- 172.21.15.100:0/3470102452 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea681034c0 msgr2=0x7fea681a2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:48.972 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.968+0000 7fea677fe640 1 --2- 172.21.15.100:0/3470102452 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea681034c0 0x7fea681a2150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:48.972 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.968+0000 7fea677fe640 1 -- 172.21.15.100:0/3470102452 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea681a27c0 con 0x7fea68101910 2024-05-10T11:35:48.972 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.969+0000 7fea677fe640 1 --2- 172.21.15.100:0/3470102452 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea68101910 0x7fea6819aba0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fea5800c970 tx=0x7fea5800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:48.972 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.969+0000 7fea657fa640 1 -- 172.21.15.100:0/3470102452 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea58007c10 con 0x7fea68101910 2024-05-10T11:35:48.972 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.969+0000 7fea657fa640 1 -- 172.21.15.100:0/3470102452 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fea58007db0 con 0x7fea68101910 2024-05-10T11:35:48.972 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.969+0000 7fea6da83640 1 -- 172.21.15.100:0/3470102452 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea681a2ab0 con 0x7fea68101910 2024-05-10T11:35:48.972 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.969+0000 7fea657fa640 1 -- 172.21.15.100:0/3470102452 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5800f7d0 con 0x7fea68101910 2024-05-10T11:35:48.972 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.969+0000 7fea6da83640 1 -- 172.21.15.100:0/3470102452 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea681a3000 con 0x7fea68101910 2024-05-10T11:35:48.980 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.971+0000 7fea707a5740 1 Processor -- start 2024-05-10T11:35:48.980 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.971+0000 7fea657fa640 1 -- 172.21.15.100:0/3470102452 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fea58018480 con 0x7fea68101910 2024-05-10T11:35:48.981 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.971+0000 7fea707a5740 1 -- start start 2024-05-10T11:35:48.981 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.972+0000 7fea707a5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed1f79edb0 0x55ed1f79f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:48.981 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.972+0000 7fea707a5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed1f79f7a0 0x55ed1f7a42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:48.981 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.972+0000 7fea707a5740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ed1f7a4910 0x55ed1f7a6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:48.981 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.972+0000 7fea707a5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55ed1f7035d0 con 0x55ed1f79f7a0 2024-05-10T11:35:48.981 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.972+0000 7fea707a5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ed1f6bc890 con 0x55ed1f79edb0 2024-05-10T11:35:48.981 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.972+0000 7fea707a5740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55ed1f6ff590 con 0x55ed1f7a4910 2024-05-10T11:35:48.981 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.972+0000 7fea677fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed1f79f7a0 0x55ed1f7a42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:48.981 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.972+0000 7fea67fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed1f79edb0 0x55ed1f79f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:48.981 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.972+0000 7fea677fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed1f79f7a0 0x55ed1f7a42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44900/0 (socket says 172.21.15.100:44900) 2024-05-10T11:35:48.982 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.972+0000 7fea677fe640 1 -- 172.21.15.100:0/325500536 learned_addr learned my addr 172.21.15.100:0/325500536 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:35:48.982 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.972+0000 7fea5ffff640 1 --2- 172.21.15.100:0/325500536 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ed1f7a4910 0x55ed1f7a6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:48.982 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.972+0000 7fea677fe640 1 -- 172.21.15.100:0/325500536 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ed1f7a4910 msgr2=0x55ed1f7a6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:48.982 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.972+0000 7fea677fe640 1 --2- 172.21.15.100:0/325500536 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ed1f7a4910 0x55ed1f7a6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:48.982 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.972+0000 7fea677fe640 1 -- 172.21.15.100:0/325500536 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed1f79edb0 msgr2=0x55ed1f79f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:48.982 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.972+0000 7fea677fe640 1 --2- 172.21.15.100:0/325500536 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed1f79edb0 0x55ed1f79f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:48.982 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.972+0000 7fea677fe640 1 -- 172.21.15.100:0/325500536 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed1f415210 con 0x55ed1f79f7a0 2024-05-10T11:35:48.982 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.972+0000 7fea657fa640 1 --2- 172.21.15.100:0/3470102452 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fea540b28a0 0x7fea540b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:48.982 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.972+0000 7fea67fff640 1 --2- 172.21.15.100:0/3470102452 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fea540b28a0 0x7fea540b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:48.982 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.972+0000 7fea677fe640 1 --2- 172.21.15.100:0/325500536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed1f79f7a0 0x55ed1f7a42a0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fea580e8c10 tx=0x7fea580e8d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1539085cd1c4594e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:48.982 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.972+0000 7fea657fa640 1 -- 172.21.15.100:0/3470102452 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fea580e2940 con 0x7fea68101910 2024-05-10T11:35:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.972+0000 7fea5dffb640 1 -- 172.21.15.100:0/325500536 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea580a5b50 con 0x55ed1f79f7a0 2024-05-10T11:35:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.973+0000 7fea5dffb640 1 -- 172.21.15.100:0/325500536 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fea580b23b0 con 0x55ed1f79f7a0 2024-05-10T11:35:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.973+0000 7fea5dffb640 1 -- 172.21.15.100:0/325500536 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea580a9400 con 0x55ed1f79f7a0 2024-05-10T11:35:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.973+0000 7fea707a5740 1 -- 172.21.15.100:0/325500536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed1f79f7a0 msgr2=0x55ed1f7a42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.973+0000 7fea67fff640 1 --2- 172.21.15.100:0/3470102452 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fea540b28a0 0x7fea540b4d60 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fea60005cc0 tx=0x7fea60005c30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.973+0000 7fea707a5740 1 --2- 172.21.15.100:0/325500536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed1f79f7a0 0x55ed1f7a42a0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fea580e8c10 tx=0x7fea580e8d00 comp rx=0 tx=0).stop 2024-05-10T11:35:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.973+0000 7fea707a5740 1 -- 172.21.15.100:0/325500536 shutdown_connections 2024-05-10T11:35:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.973+0000 7fea707a5740 1 --2- 172.21.15.100:0/325500536 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ed1f7a4910 0x55ed1f7a6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.973+0000 7fea707a5740 1 --2- 172.21.15.100:0/325500536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed1f79f7a0 0x55ed1f7a42a0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.973+0000 7fea707a5740 1 --2- 172.21.15.100:0/325500536 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed1f79edb0 0x55ed1f79f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.973+0000 7fea707a5740 1 -- 172.21.15.100:0/325500536 >> 172.21.15.100:0/325500536 conn(0x55ed1f79a420 msgr2=0x55ed1f79a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.973+0000 7fea707a5740 1 -- 172.21.15.100:0/325500536 shutdown_connections 2024-05-10T11:35:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.973+0000 7fea707a5740 1 -- 172.21.15.100:0/325500536 wait complete. 2024-05-10T11:35:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.974+0000 7fea707a5740 1 -- start start 2024-05-10T11:35:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.974+0000 7fea707a5740 1 Processor -- start 2024-05-10T11:35:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.974+0000 7fea707a5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed1f79edb0 0x55ed1f7a8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.974+0000 7fea707a5740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ed1f79f7a0 0x55ed1f7a94e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.974+0000 7fea707a5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed1f7a4910 0x55ed1f7b0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.974+0000 7fea707a5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55ed1f7a78c0 con 0x55ed1f79edb0 2024-05-10T11:35:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.974+0000 7fea707a5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ed1f6c0370 con 0x55ed1f7a4910 2024-05-10T11:35:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.974+0000 7fea707a5740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55ed1f7a7a40 con 0x55ed1f79f7a0 2024-05-10T11:35:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.974+0000 7fea677fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed1f79edb0 0x55ed1f7a8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.974+0000 7fea707a5740 10 client.0 _ll_register_callbacks cb 0x55ed1f414a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:35:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.974+0000 7fea677fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed1f79edb0 0x55ed1f7a8fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44908/0 (socket says 172.21.15.100:44908) 2024-05-10T11:35:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.974+0000 7fea677fe640 1 -- 172.21.15.100:0/1718977995 learned_addr learned my addr 172.21.15.100:0/1718977995 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:35:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.974+0000 7fea5ffff640 1 --2- 172.21.15.100:0/1718977995 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ed1f79f7a0 0x55ed1f7a94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.974+0000 7fea67fff640 1 --2- 172.21.15.100:0/1718977995 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed1f7a4910 0x55ed1f7b0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.974+0000 7fea677fe640 1 -- 172.21.15.100:0/1718977995 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ed1f79f7a0 msgr2=0x55ed1f7a94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.974+0000 7fea677fe640 1 --2- 172.21.15.100:0/1718977995 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ed1f79f7a0 0x55ed1f7a94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.974+0000 7fea677fe640 1 -- 172.21.15.100:0/1718977995 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed1f7a4910 msgr2=0x55ed1f7b0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.974+0000 7fea677fe640 1 --2- 172.21.15.100:0/1718977995 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed1f7a4910 0x55ed1f7b0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.974+0000 7fea677fe640 1 -- 172.21.15.100:0/1718977995 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed1f4e1620 con 0x55ed1f79edb0 2024-05-10T11:35:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.974+0000 7fea67fff640 1 --2- 172.21.15.100:0/1718977995 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed1f7a4910 0x55ed1f7b0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:35:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.975+0000 7fea677fe640 1 --2- 172.21.15.100:0/1718977995 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed1f79edb0 0x55ed1f7a8fa0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fea580e8c10 tx=0x7fea580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2ed618ed18d7720 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.975+0000 7fea46ffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:35:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.975+0000 7fea46ffd640 1 -- 172.21.15.100:0/1718977995 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea580a95e0 con 0x55ed1f79edb0 2024-05-10T11:35:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.975+0000 7fea46ffd640 1 -- 172.21.15.100:0/1718977995 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fea58067920 con 0x55ed1f79edb0 2024-05-10T11:35:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.975+0000 7fea707a5740 1 -- 172.21.15.100:0/1718977995 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea580b23b0 con 0x55ed1f79edb0 2024-05-10T11:35:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.975+0000 7fea46ffd640 1 -- 172.21.15.100:0/1718977995 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fea580a9a60 con 0x55ed1f79edb0 2024-05-10T11:35:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.975+0000 7fea707a5740 10 client.18888 fetch_fsmap learned FSMap version 41 2024-05-10T11:35:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.975+0000 7fea707a5740 1 -- 172.21.15.100:0/1718977995 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55ed1f7b3500 con 0x55ed1f79edb0 2024-05-10T11:35:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.975+0000 7fea46ffd640 1 -- 172.21.15.100:0/1718977995 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fea580b23b0 con 0x55ed1f79edb0 2024-05-10T11:35:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.975+0000 7fea46ffd640 1 -- 172.21.15.100:0/1718977995 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fea580a5f20 con 0x55ed1f79edb0 2024-05-10T11:35:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.975+0000 7fea707a5740 10 client.18888 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:35:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.975+0000 7fea707a5740 10 client.18888 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:35:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.975+0000 7fea707a5740 20 client.18888 populate_metadata read hostname 'smithi100' 2024-05-10T11:35:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.975+0000 7fea707a5740 1 --2- 172.21.15.100:0/1718977995 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ed1f7b4850 0x55ed1f7b6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:48.987 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.975+0000 7fea707a5740 4 client.18888 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:35:48.987 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.975+0000 7fea707a5740 1 -- 172.21.15.100:0/1718977995 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed1f7111e0 con 0x55ed1f7b4850 2024-05-10T11:35:48.987 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.975+0000 7fea5ffff640 1 --2- 172.21.15.100:0/1718977995 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ed1f7b4850 0x55ed1f7b6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:48.987 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.976+0000 7fea46ffd640 10 client.18888 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:35:48.987 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.976+0000 7fea5ffff640 1 --2- 172.21.15.100:0/1718977995 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ed1f7b4850 0x55ed1f7b6c40 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2198e27672147dde server_cookie=742990e2d92b125 in_seq=0 out_seq=0 2024-05-10T11:35:48.987 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.976+0000 7fea46ffd640 1 -- 172.21.15.100:0/1718977995 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ed1f7111e0 con 0x55ed1f7b4850 2024-05-10T11:35:48.987 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.976+0000 7fea46ffd640 10 client.18888 handle_command_reply: tid=0 2024-05-10T11:35:48.987 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.976+0000 7fea707a5740 1 client.18888 shutdown 2024-05-10T11:35:48.987 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.977+0000 7fea707a5740 1 -- 172.21.15.100:0/1718977995 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed1f79edb0 msgr2=0x55ed1f7a8fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:48.987 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.977+0000 7fea707a5740 1 --2- 172.21.15.100:0/1718977995 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed1f79edb0 0x55ed1f7a8fa0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fea580e8c10 tx=0x7fea580046c0 comp rx=0 tx=0).stop 2024-05-10T11:35:48.987 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.977+0000 7fea707a5740 1 -- 172.21.15.100:0/1718977995 shutdown_connections 2024-05-10T11:35:48.987 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.977+0000 7fea707a5740 1 --2- 172.21.15.100:0/1718977995 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ed1f7b4850 0x55ed1f7b6c40 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:48.987 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.977+0000 7fea707a5740 1 --2- 172.21.15.100:0/1718977995 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed1f7a4910 0x55ed1f7b0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:48.987 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.977+0000 7fea46ffd640 0 client.18888 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:35:48.988 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.977+0000 7fea707a5740 1 --2- 172.21.15.100:0/1718977995 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ed1f79f7a0 0x55ed1f7a94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:48.988 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.977+0000 7fea707a5740 1 --2- 172.21.15.100:0/1718977995 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed1f79edb0 0x55ed1f7a8fa0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:48.988 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.977+0000 7fea707a5740 1 -- 172.21.15.100:0/1718977995 >> 172.21.15.100:0/1718977995 conn(0x55ed1f79a420 msgr2=0x55ed1f799580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:48.988 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.977+0000 7fea707a5740 1 -- 172.21.15.100:0/1718977995 shutdown_connections 2024-05-10T11:35:48.988 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.977+0000 7fea707a5740 1 -- 172.21.15.100:0/1718977995 wait complete. 2024-05-10T11:35:48.988 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.977+0000 7fea707a5740 20 client.18888 trim_cache size 0 max 16384 2024-05-10T11:35:49.006 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.997+0000 7fea707a5740 1 Processor -- start 2024-05-10T11:35:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.997+0000 7fea707a5740 1 -- start start 2024-05-10T11:35:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.998+0000 7fea707a5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed1f79a420 0x55ed1f79da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.998+0000 7fea707a5740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ed1f79edb0 0x55ed1f7b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.998+0000 7fea707a5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed1f79f7a0 0x55ed1f7a85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.998+0000 7fea707a5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fea681afc40 con 0x55ed1f79a420 2024-05-10T11:35:49.008 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.998+0000 7fea707a5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fea681ad510 con 0x55ed1f79f7a0 2024-05-10T11:35:49.008 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.998+0000 7fea707a5740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fea681ab360 con 0x55ed1f79edb0 2024-05-10T11:35:49.008 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.998+0000 7fea677fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed1f79a420 0x55ed1f79da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:49.008 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.998+0000 7fea677fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed1f79a420 0x55ed1f79da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44918/0 (socket says 172.21.15.100:44918) 2024-05-10T11:35:49.008 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.998+0000 7fea677fe640 1 -- 172.21.15.100:0/958827124 learned_addr learned my addr 172.21.15.100:0/958827124 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:35:49.008 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.998+0000 7fea5ffff640 1 --2- 172.21.15.100:0/958827124 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ed1f79edb0 0x55ed1f7b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:49.009 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.998+0000 7fea67fff640 1 --2- 172.21.15.100:0/958827124 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed1f79f7a0 0x55ed1f7a85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:49.009 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.998+0000 7fea677fe640 1 -- 172.21.15.100:0/958827124 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ed1f79edb0 msgr2=0x55ed1f7b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:49.009 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.998+0000 7fea677fe640 1 --2- 172.21.15.100:0/958827124 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ed1f79edb0 0x55ed1f7b04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:49.009 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.998+0000 7fea677fe640 1 -- 172.21.15.100:0/958827124 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed1f79f7a0 msgr2=0x55ed1f7a85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:49.009 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.998+0000 7fea677fe640 1 --2- 172.21.15.100:0/958827124 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed1f79f7a0 0x55ed1f7a85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:49.009 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.998+0000 7fea677fe640 1 -- 172.21.15.100:0/958827124 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed1f79a250 con 0x55ed1f79a420 2024-05-10T11:35:49.010 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.998+0000 7fea677fe640 1 --2- 172.21.15.100:0/958827124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed1f79a420 0x55ed1f79da40 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fea580a5d10 tx=0x7fea580a97a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4af33db8231aa30 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:49.010 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.999+0000 7fea5d7fa640 1 -- 172.21.15.100:0/958827124 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea580c4280 con 0x55ed1f79a420 2024-05-10T11:35:49.010 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.999+0000 7fea5d7fa640 1 -- 172.21.15.100:0/958827124 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fea580c8070 con 0x55ed1f79a420 2024-05-10T11:35:49.010 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:48.999+0000 7fea5d7fa640 1 -- 172.21.15.100:0/958827124 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea580a9400 con 0x55ed1f79a420 2024-05-10T11:35:49.010 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.000+0000 7fea707a5740 1 -- 172.21.15.100:0/958827124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed1f79a420 msgr2=0x55ed1f79da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:49.010 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.000+0000 7fea707a5740 1 --2- 172.21.15.100:0/958827124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed1f79a420 0x55ed1f79da40 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fea580a5d10 tx=0x7fea580a97a0 comp rx=0 tx=0).stop 2024-05-10T11:35:49.011 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.000+0000 7fea707a5740 1 -- 172.21.15.100:0/958827124 shutdown_connections 2024-05-10T11:35:49.011 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.000+0000 7fea707a5740 1 --2- 172.21.15.100:0/958827124 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed1f79f7a0 0x55ed1f7a85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:49.011 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.000+0000 7fea707a5740 1 --2- 172.21.15.100:0/958827124 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ed1f79edb0 0x55ed1f7b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:49.011 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.000+0000 7fea707a5740 1 --2- 172.21.15.100:0/958827124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed1f79a420 0x55ed1f79da40 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:49.011 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.000+0000 7fea707a5740 1 -- 172.21.15.100:0/958827124 >> 172.21.15.100:0/958827124 conn(0x55ed1f7a0260 msgr2=0x55ed1f79d330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:49.011 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.000+0000 7fea707a5740 1 -- 172.21.15.100:0/958827124 shutdown_connections 2024-05-10T11:35:49.011 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.000+0000 7fea707a5740 1 -- 172.21.15.100:0/958827124 wait complete. 2024-05-10T11:35:49.011 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.000+0000 7fea707a5740 1 -- start start 2024-05-10T11:35:49.012 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.000+0000 7fea707a5740 1 Processor -- start 2024-05-10T11:35:49.012 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.001+0000 7fea707a5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed1f79a420 0x55ed1f7adf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:49.012 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.001+0000 7fea707a5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed1f79edb0 0x55ed1f7ae4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:49.012 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.001+0000 7fea707a5740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ed1f79f7a0 0x55ed1f7af440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:49.012 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.001+0000 7fea707a5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fea681ab9d0 con 0x55ed1f79a420 2024-05-10T11:35:49.012 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.001+0000 7fea707a5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fea681ace40 con 0x55ed1f79edb0 2024-05-10T11:35:49.013 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.001+0000 7fea707a5740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fea681ac0a0 con 0x55ed1f79f7a0 2024-05-10T11:35:49.013 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.001+0000 7fea677fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed1f79a420 0x55ed1f7adf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:49.013 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.001+0000 7fea677fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed1f79a420 0x55ed1f7adf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44934/0 (socket says 172.21.15.100:44934) 2024-05-10T11:35:49.013 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.001+0000 7fea5ffff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed1f79edb0 0x55ed1f7ae4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:49.013 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.001+0000 7fea677fe640 1 -- 172.21.15.100:0/3030171821 learned_addr learned my addr 172.21.15.100:0/3030171821 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:35:49.013 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.001+0000 7fea67fff640 1 --2- 172.21.15.100:0/3030171821 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ed1f79f7a0 0x55ed1f7af440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:49.014 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.001+0000 7fea707a5740 10 client.0 _ll_register_callbacks cb 0x7fea50005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:35:49.014 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.001+0000 7fea5ffff640 1 -- 172.21.15.100:0/3030171821 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ed1f79f7a0 msgr2=0x55ed1f7af440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:49.014 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.001+0000 7fea5ffff640 1 --2- 172.21.15.100:0/3030171821 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ed1f79f7a0 0x55ed1f7af440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:49.014 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.001+0000 7fea5ffff640 1 -- 172.21.15.100:0/3030171821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed1f79a420 msgr2=0x55ed1f7adf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:49.014 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.001+0000 7fea5ffff640 1 --2- 172.21.15.100:0/3030171821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed1f79a420 0x55ed1f7adf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:49.014 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.001+0000 7fea5ffff640 1 -- 172.21.15.100:0/3030171821 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed1f7afa30 con 0x55ed1f79edb0 2024-05-10T11:35:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.002+0000 7fea5ffff640 1 --2- 172.21.15.100:0/3030171821 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed1f79edb0 0x55ed1f7ae4a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x55ed1f7b7410 tx=0x7fea50004410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a45eaaed74d7a6f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.002+0000 7fea46ffd640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:35:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.002+0000 7fea46ffd640 1 -- 172.21.15.100:0/3030171821 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea500037f0 con 0x55ed1f79edb0 2024-05-10T11:35:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.002+0000 7fea46ffd640 1 -- 172.21.15.100:0/3030171821 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fea50003990 con 0x55ed1f79edb0 2024-05-10T11:35:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.002+0000 7fea707a5740 1 -- 172.21.15.100:0/3030171821 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea580c8070 con 0x55ed1f79edb0 2024-05-10T11:35:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.002+0000 7fea46ffd640 1 -- 172.21.15.100:0/3030171821 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fea50030050 con 0x55ed1f79edb0 2024-05-10T11:35:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.002+0000 7fea707a5740 10 client.28202 fetch_fsmap learned FSMap version 41 2024-05-10T11:35:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.002+0000 7fea707a5740 1 -- 172.21.15.100:0/3030171821 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55ed1f81f300 con 0x55ed1f79edb0 2024-05-10T11:35:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.002+0000 7fea46ffd640 1 -- 172.21.15.100:0/3030171821 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fea580c8070 con 0x55ed1f79edb0 2024-05-10T11:35:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.003+0000 7fea46ffd640 1 -- 172.21.15.100:0/3030171821 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fea50012a70 con 0x55ed1f79edb0 2024-05-10T11:35:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.003+0000 7fea707a5740 10 client.28202 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:35:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.003+0000 7fea707a5740 10 client.28202 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:35:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.003+0000 7fea707a5740 20 client.28202 populate_metadata read hostname 'smithi100' 2024-05-10T11:35:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.003+0000 7fea707a5740 1 --2- 172.21.15.100:0/3030171821 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ed1f7a4910 0x55ed1f821d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.003+0000 7fea707a5740 4 client.28202 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:35:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.003+0000 7fea707a5740 1 -- 172.21.15.100:0/3030171821 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ed1f70b050 con 0x55ed1f7a4910 2024-05-10T11:35:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.003+0000 7fea677fe640 1 --2- 172.21.15.100:0/3030171821 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ed1f7a4910 0x55ed1f821d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.004+0000 7fea677fe640 1 --2- 172.21.15.100:0/3030171821 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ed1f7a4910 0x55ed1f821d60 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b61901f36f4d9e1 server_cookie=523591992d014cb in_seq=0 out_seq=0 2024-05-10T11:35:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.004+0000 7fea46ffd640 10 client.28202 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:35:49.115 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.114+0000 7fea46ffd640 1 -- 172.21.15.100:0/3030171821 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ed1f70b050 con 0x55ed1f7a4910 2024-05-10T11:35:49.115 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.114+0000 7fea46ffd640 10 client.28202 handle_command_reply: tid=0 2024-05-10T11:35:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.114+0000 7fea707a5740 1 client.28202 shutdown 2024-05-10T11:35:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.115+0000 7fea707a5740 1 -- 172.21.15.100:0/3030171821 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed1f79edb0 msgr2=0x55ed1f7ae4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.115+0000 7fea707a5740 1 --2- 172.21.15.100:0/3030171821 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed1f79edb0 0x55ed1f7ae4a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x55ed1f7b7410 tx=0x7fea50004410 comp rx=0 tx=0).stop 2024-05-10T11:35:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.115+0000 7fea707a5740 1 -- 172.21.15.100:0/3030171821 shutdown_connections 2024-05-10T11:35:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.115+0000 7fea707a5740 1 --2- 172.21.15.100:0/3030171821 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ed1f7a4910 0x55ed1f821d60 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.115+0000 7fea707a5740 1 --2- 172.21.15.100:0/3030171821 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ed1f79f7a0 0x55ed1f7af440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.115+0000 7fea46ffd640 0 client.28202 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:35:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.115+0000 7fea707a5740 1 --2- 172.21.15.100:0/3030171821 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed1f79edb0 0x55ed1f7ae4a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.115+0000 7fea707a5740 1 --2- 172.21.15.100:0/3030171821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed1f79a420 0x55ed1f7adf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.115+0000 7fea707a5740 1 -- 172.21.15.100:0/3030171821 >> 172.21.15.100:0/3030171821 conn(0x55ed1f7a0260 msgr2=0x55ed1f798cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.115+0000 7fea707a5740 1 -- 172.21.15.100:0/3030171821 shutdown_connections 2024-05-10T11:35:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.115+0000 7fea707a5740 1 -- 172.21.15.100:0/3030171821 wait complete. 2024-05-10T11:35:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.115+0000 7fea707a5740 20 client.28202 trim_cache size 0 max 16384 2024-05-10T11:35:49.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:35:49.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:35:49.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:35:49.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:35:49.118 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.116+0000 7fea6da83640 1 -- 172.21.15.100:0/3470102452 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fea540b28a0 msgr2=0x7fea540b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.116+0000 7fea6da83640 1 --2- 172.21.15.100:0/3470102452 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fea540b28a0 0x7fea540b4d60 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fea60005cc0 tx=0x7fea60005c30 comp rx=0 tx=0).stop 2024-05-10T11:35:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.116+0000 7fea6da83640 1 -- 172.21.15.100:0/3470102452 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea68101910 msgr2=0x7fea6819aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.116+0000 7fea6da83640 1 --2- 172.21.15.100:0/3470102452 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea68101910 0x7fea6819aba0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fea5800c970 tx=0x7fea5800ce30 comp rx=0 tx=0).stop 2024-05-10T11:35:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.116+0000 7fea6da83640 1 -- 172.21.15.100:0/3470102452 shutdown_connections 2024-05-10T11:35:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.116+0000 7fea6da83640 1 --2- 172.21.15.100:0/3470102452 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fea540b28a0 0x7fea540b4d60 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.116+0000 7fea6da83640 1 --2- 172.21.15.100:0/3470102452 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea681034c0 0x7fea681a2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.116+0000 7fea6da83640 1 --2- 172.21.15.100:0/3470102452 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea68102b00 0x7fea6819b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.116+0000 7fea6da83640 1 --2- 172.21.15.100:0/3470102452 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea68101910 0x7fea6819aba0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.116+0000 7fea6da83640 1 -- 172.21.15.100:0/3470102452 >> 172.21.15.100:0/3470102452 conn(0x7fea680fcea0 msgr2=0x7fea6806b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.116+0000 7fea6da83640 1 -- 172.21.15.100:0/3470102452 shutdown_connections 2024-05-10T11:35:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:35:49.116+0000 7fea6da83640 1 -- 172.21.15.100:0/3470102452 wait complete. 2024-05-10T11:35:49.920 INFO:tasks.workunit.client.0.smithi100.stdout: 10 37735 9.16 MB/sec execute 103 sec latency 123.516 ms 2024-05-10T11:35:50.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.054+0000 7fc6062f5700 1 -- 172.21.15.100:0/2278966997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc600108410 msgr2=0x7fc6001087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:50.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.054+0000 7fc6062f5700 1 --2- 172.21.15.100:0/2278966997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc600108410 0x7fc6001087f0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fc5f0009990 tx=0x7fc5f0009ca0 comp rx=0 tx=0).stop 2024-05-10T11:35:50.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.054+0000 7fc6062f5700 1 -- 172.21.15.100:0/2278966997 shutdown_connections 2024-05-10T11:35:50.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.054+0000 7fc6062f5700 1 --2- 172.21.15.100:0/2278966997 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc600102d60 0x7fc60010f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:50.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.054+0000 7fc6062f5700 1 --2- 172.21.15.100:0/2278966997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc600108410 0x7fc6001087f0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:50.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.054+0000 7fc6062f5700 1 --2- 172.21.15.100:0/2278966997 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6001023a0 0x7fc600102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:50.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.054+0000 7fc6062f5700 1 -- 172.21.15.100:0/2278966997 >> 172.21.15.100:0/2278966997 conn(0x7fc600079350 msgr2=0x7fc600079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:50.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.054+0000 7fc6062f5700 1 -- 172.21.15.100:0/2278966997 shutdown_connections 2024-05-10T11:35:50.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.054+0000 7fc6062f5700 1 -- 172.21.15.100:0/2278966997 wait complete. 2024-05-10T11:35:50.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.055+0000 7fc6062f5700 1 Processor -- start 2024-05-10T11:35:50.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.055+0000 7fc6062f5700 1 -- start start 2024-05-10T11:35:50.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.055+0000 7fc6062f5700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6001023a0 0x7fc6001a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:50.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.055+0000 7fc6062f5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc600102d60 0x7fc6001a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:50.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.055+0000 7fc6062f5700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc600108410 0x7fc60019bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:50.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.055+0000 7fc6062f5700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc600111350 con 0x7fc600108410 2024-05-10T11:35:50.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.055+0000 7fc6062f5700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc6001111d0 con 0x7fc600102d60 2024-05-10T11:35:50.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.055+0000 7fc6062f5700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc6001114d0 con 0x7fc6001023a0 2024-05-10T11:35:50.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.056+0000 7fc5ff7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc600102d60 0x7fc6001a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:50.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.056+0000 7fc5ff7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc600102d60 0x7fc6001a1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:50134/0 (socket says 172.21.15.100:50134) 2024-05-10T11:35:50.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.056+0000 7fc5ff7fe700 1 -- 172.21.15.100:0/1755621505 learned_addr learned my addr 172.21.15.100:0/1755621505 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:35:50.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.056+0000 7fc604892700 1 --2- 172.21.15.100:0/1755621505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc600108410 0x7fc60019bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:50.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.056+0000 7fc5fffff700 1 --2- 172.21.15.100:0/1755621505 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6001023a0 0x7fc6001a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:50.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.056+0000 7fc5ff7fe700 1 -- 172.21.15.100:0/1755621505 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6001023a0 msgr2=0x7fc6001a19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:50.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.056+0000 7fc5ff7fe700 1 --2- 172.21.15.100:0/1755621505 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6001023a0 0x7fc6001a19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:50.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.056+0000 7fc5ff7fe700 1 -- 172.21.15.100:0/1755621505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc600108410 msgr2=0x7fc60019bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:50.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.056+0000 7fc5ff7fe700 1 --2- 172.21.15.100:0/1755621505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc600108410 0x7fc60019bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:50.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.056+0000 7fc5ff7fe700 1 -- 172.21.15.100:0/1755621505 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc60019c600 con 0x7fc600102d60 2024-05-10T11:35:50.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.056+0000 7fc604892700 1 --2- 172.21.15.100:0/1755621505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc600108410 0x7fc60019bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:35:50.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.056+0000 7fc5ff7fe700 1 --2- 172.21.15.100:0/1755621505 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc600102d60 0x7fc6001a1f10 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc5f400e9c0 tx=0x7fc5f400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:50.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.056+0000 7fc5fd7fa700 1 -- 172.21.15.100:0/1755621505 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f400cbd0 con 0x7fc600102d60 2024-05-10T11:35:50.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.056+0000 7fc6062f5700 1 -- 172.21.15.100:0/1755621505 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc60019c890 con 0x7fc600102d60 2024-05-10T11:35:50.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.056+0000 7fc6062f5700 1 -- 172.21.15.100:0/1755621505 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6001a8990 con 0x7fc600102d60 2024-05-10T11:35:50.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.056+0000 7fc5fd7fa700 1 -- 172.21.15.100:0/1755621505 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc5f400ee60 con 0x7fc600102d60 2024-05-10T11:35:50.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.057+0000 7fc5fd7fa700 1 -- 172.21.15.100:0/1755621505 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f40054b0 con 0x7fc600102d60 2024-05-10T11:35:50.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.059+0000 7fc5fd7fa700 1 -- 172.21.15.100:0/1755621505 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fc5f4004750 con 0x7fc600102d60 2024-05-10T11:35:50.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.060+0000 7fc5fd7fa700 1 --2- 172.21.15.100:0/1755621505 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc5e80b2880 0x7fc5e80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:50.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.060+0000 7fc5fd7fa700 1 -- 172.21.15.100:0/1755621505 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fc5f40e2840 con 0x7fc600102d60 2024-05-10T11:35:50.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.060+0000 7fc5fffff700 1 --2- 172.21.15.100:0/1755621505 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc5e80b2880 0x7fc5e80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:50.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.062+0000 7fc5fffff700 1 --2- 172.21.15.100:0/1755621505 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc5e80b2880 0x7fc5e80b4d40 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc5f000b2e0 tx=0x7fc5f0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:50.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.062+0000 7fc610b88b80 1 Processor -- start 2024-05-10T11:35:50.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.062+0000 7fc610b88b80 1 -- start start 2024-05-10T11:35:50.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.062+0000 7fc610b88b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c60f36f1b0 0x55c60f36f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:50.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.063+0000 7fc610b88b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c60f36fad0 0x55c60f369810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:50.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.063+0000 7fc610b88b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c60f369d50 0x55c60f36a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:50.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.063+0000 7fc610b88b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55c60f2bb940 con 0x55c60f369d50 2024-05-10T11:35:50.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.063+0000 7fc610b88b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c60f220da0 con 0x55c60f36fad0 2024-05-10T11:35:50.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.063+0000 7fc610b88b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c60f2b7900 con 0x55c60f36f1b0 2024-05-10T11:35:50.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.063+0000 7fc5fffff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c60f36f1b0 0x55c60f36f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:50.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.063+0000 7fc5fffff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c60f36f1b0 0x55c60f36f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:34176/0 (socket says 172.21.15.100:34176) 2024-05-10T11:35:50.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.063+0000 7fc5fffff700 1 -- 172.21.15.100:0/931801127 learned_addr learned my addr 172.21.15.100:0/931801127 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:35:50.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.063+0000 7fc604892700 1 --2- 172.21.15.100:0/931801127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c60f369d50 0x55c60f36a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:50.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.063+0000 7fc5ff7fe700 1 --2- 172.21.15.100:0/931801127 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c60f36fad0 0x55c60f369810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:50.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.063+0000 7fc604892700 1 -- 172.21.15.100:0/931801127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c60f36f1b0 msgr2=0x55c60f36f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:50.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.063+0000 7fc604892700 1 --2- 172.21.15.100:0/931801127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c60f36f1b0 0x55c60f36f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:50.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.063+0000 7fc604892700 1 -- 172.21.15.100:0/931801127 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c60f36fad0 msgr2=0x55c60f369810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:50.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.063+0000 7fc604892700 1 --2- 172.21.15.100:0/931801127 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c60f36fad0 0x55c60f369810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:50.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.063+0000 7fc604892700 1 -- 172.21.15.100:0/931801127 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c60f2088d0 con 0x55c60f369d50 2024-05-10T11:35:50.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.064+0000 7fc604892700 1 --2- 172.21.15.100:0/931801127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c60f369d50 0x55c60f36a190 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8003db0 tx=0x7fc5f8009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f52e1a2f76d0a7e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:50.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.064+0000 7fc5e67fc700 1 -- 172.21.15.100:0/931801127 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f8019030 con 0x55c60f369d50 2024-05-10T11:35:50.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.064+0000 7fc5e67fc700 1 -- 172.21.15.100:0/931801127 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc5f8004bb0 con 0x55c60f369d50 2024-05-10T11:35:50.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.064+0000 7fc5e67fc700 1 -- 172.21.15.100:0/931801127 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f8007bf0 con 0x55c60f369d50 2024-05-10T11:35:50.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.066+0000 7fc610b88b80 1 -- 172.21.15.100:0/931801127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c60f369d50 msgr2=0x55c60f36a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:50.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.066+0000 7fc610b88b80 1 --2- 172.21.15.100:0/931801127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c60f369d50 0x55c60f36a190 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8003db0 tx=0x7fc5f8009920 comp rx=0 tx=0).stop 2024-05-10T11:35:50.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.066+0000 7fc610b88b80 1 -- 172.21.15.100:0/931801127 shutdown_connections 2024-05-10T11:35:50.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.066+0000 7fc610b88b80 1 --2- 172.21.15.100:0/931801127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c60f369d50 0x55c60f36a190 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:50.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.066+0000 7fc610b88b80 1 --2- 172.21.15.100:0/931801127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c60f36f1b0 0x55c60f36f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:50.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.066+0000 7fc610b88b80 1 --2- 172.21.15.100:0/931801127 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c60f36fad0 0x55c60f369810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:50.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.066+0000 7fc610b88b80 1 -- 172.21.15.100:0/931801127 >> 172.21.15.100:0/931801127 conn(0x55c60f295b70 msgr2=0x55c60f3642a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:50.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.066+0000 7fc610b88b80 1 -- 172.21.15.100:0/931801127 shutdown_connections 2024-05-10T11:35:50.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.067+0000 7fc610b88b80 1 -- 172.21.15.100:0/931801127 wait complete. 2024-05-10T11:35:50.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.067+0000 7fc610b88b80 1 -- start start 2024-05-10T11:35:50.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.067+0000 7fc610b88b80 1 Processor -- start 2024-05-10T11:35:50.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.068+0000 7fc610b88b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c60f369d50 0x55c60f375760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:50.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.068+0000 7fc610b88b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c60f36f1b0 0x55c60f372b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:50.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.068+0000 7fc610b88b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c60f36fad0 0x55c60f373040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:50.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.068+0000 7fc610b88b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55c60f376450 con 0x55c60f36fad0 2024-05-10T11:35:50.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.068+0000 7fc610b88b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c60f274ed0 con 0x55c60f369d50 2024-05-10T11:35:50.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.068+0000 7fc610b88b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c60f3765d0 con 0x55c60f36f1b0 2024-05-10T11:35:50.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.068+0000 7fc610b88b80 10 client.0 _ll_register_callbacks cb 0x55c60f24f5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:35:50.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.068+0000 7fc5ff7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c60f36f1b0 0x55c60f372b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:50.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.068+0000 7fc5ff7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c60f36f1b0 0x55c60f372b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:34182/0 (socket says 172.21.15.100:34182) 2024-05-10T11:35:50.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.068+0000 7fc5ff7fe700 1 -- 172.21.15.100:0/4269373971 learned_addr learned my addr 172.21.15.100:0/4269373971 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:35:50.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.068+0000 7fc604892700 1 --2- 172.21.15.100:0/4269373971 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c60f36fad0 0x55c60f373040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:50.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.068+0000 7fc5fffff700 1 --2- 172.21.15.100:0/4269373971 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c60f369d50 0x55c60f375760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:50.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.068+0000 7fc5ff7fe700 1 -- 172.21.15.100:0/4269373971 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c60f369d50 msgr2=0x55c60f375760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:50.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.068+0000 7fc5ff7fe700 1 --2- 172.21.15.100:0/4269373971 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c60f369d50 0x55c60f375760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:50.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.068+0000 7fc5ff7fe700 1 -- 172.21.15.100:0/4269373971 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c60f36fad0 msgr2=0x55c60f373040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:50.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.069+0000 7fc5ff7fe700 1 --2- 172.21.15.100:0/4269373971 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c60f36fad0 0x55c60f373040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:50.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.069+0000 7fc5ff7fe700 1 -- 172.21.15.100:0/4269373971 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c60f272760 con 0x55c60f36f1b0 2024-05-10T11:35:50.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.069+0000 7fc5ff7fe700 1 --2- 172.21.15.100:0/4269373971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c60f36f1b0 0x55c60f372b00 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc5f40e0030 tx=0x7fc5f4016630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9aa12689294fea67 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:50.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.069+0000 7fc5df7fe700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:35:50.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.069+0000 7fc5df7fe700 1 -- 172.21.15.100:0/4269373971 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f40b22a0 con 0x55c60f36f1b0 2024-05-10T11:35:50.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.069+0000 7fc610b88b80 1 -- 172.21.15.100:0/4269373971 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5f8004bb0 con 0x55c60f36f1b0 2024-05-10T11:35:50.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.069+0000 7fc5df7fe700 1 -- 172.21.15.100:0/4269373971 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fc5f40b26e0 con 0x55c60f36f1b0 2024-05-10T11:35:50.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.069+0000 7fc5df7fe700 1 -- 172.21.15.100:0/4269373971 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fc5f40a9610 con 0x55c60f36f1b0 2024-05-10T11:35:50.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.069+0000 7fc610b88b80 10 client.28318 fetch_fsmap learned FSMap version 41 2024-05-10T11:35:50.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.069+0000 7fc610b88b80 1 -- 172.21.15.100:0/4269373971 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55c60f380b10 con 0x55c60f36f1b0 2024-05-10T11:35:50.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.069+0000 7fc5df7fe700 1 -- 172.21.15.100:0/4269373971 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc5f8004bb0 con 0x55c60f36f1b0 2024-05-10T11:35:50.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.070+0000 7fc5df7fe700 1 -- 172.21.15.100:0/4269373971 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fc5f40a8d60 con 0x55c60f36f1b0 2024-05-10T11:35:50.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.070+0000 7fc610b88b80 10 client.28318 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:35:50.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.070+0000 7fc610b88b80 10 client.28318 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:35:50.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.070+0000 7fc610b88b80 20 client.28318 populate_metadata read hostname 'smithi100' 2024-05-10T11:35:50.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.070+0000 7fc610b88b80 1 --2- 172.21.15.100:0/4269373971 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c60f3830a0 0x55c60f3854c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:50.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.070+0000 7fc610b88b80 4 client.28318 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:35:50.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.070+0000 7fc610b88b80 1 -- 172.21.15.100:0/4269373971 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c60f2c9320 con 0x55c60f3830a0 2024-05-10T11:35:50.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.070+0000 7fc5fffff700 1 --2- 172.21.15.100:0/4269373971 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c60f3830a0 0x55c60f3854c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:50.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.070+0000 7fc5fffff700 1 --2- 172.21.15.100:0/4269373971 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c60f3830a0 0x55c60f3854c0 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cea85e1c64d72e35 server_cookie=995b9b6ab50516c8 in_seq=0 out_seq=0 2024-05-10T11:35:50.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.071+0000 7fc5df7fe700 10 client.28318 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:35:50.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.071+0000 7fc5df7fe700 1 -- 172.21.15.100:0/4269373971 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c60f2c9320 con 0x55c60f3830a0 2024-05-10T11:35:50.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.071+0000 7fc5df7fe700 10 client.28318 handle_command_reply: tid=0 2024-05-10T11:35:50.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.071+0000 7fc610b88b80 1 client.28318 shutdown 2024-05-10T11:35:50.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.072+0000 7fc610b88b80 1 -- 172.21.15.100:0/4269373971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c60f36f1b0 msgr2=0x55c60f372b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:50.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.072+0000 7fc610b88b80 1 --2- 172.21.15.100:0/4269373971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c60f36f1b0 0x55c60f372b00 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc5f40e0030 tx=0x7fc5f4016630 comp rx=0 tx=0).stop 2024-05-10T11:35:50.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.072+0000 7fc610b88b80 1 -- 172.21.15.100:0/4269373971 shutdown_connections 2024-05-10T11:35:50.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.072+0000 7fc610b88b80 1 --2- 172.21.15.100:0/4269373971 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c60f3830a0 0x55c60f3854c0 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:50.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.072+0000 7fc610b88b80 1 --2- 172.21.15.100:0/4269373971 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c60f36fad0 0x55c60f373040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:50.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.072+0000 7fc5df7fe700 0 client.28318 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:35:50.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.072+0000 7fc610b88b80 1 --2- 172.21.15.100:0/4269373971 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c60f369d50 0x55c60f375760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:50.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.072+0000 7fc610b88b80 1 --2- 172.21.15.100:0/4269373971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c60f36f1b0 0x55c60f372b00 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:50.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.072+0000 7fc610b88b80 1 -- 172.21.15.100:0/4269373971 >> 172.21.15.100:0/4269373971 conn(0x55c60f295b70 msgr2=0x55c60f35eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:50.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.072+0000 7fc610b88b80 1 -- 172.21.15.100:0/4269373971 shutdown_connections 2024-05-10T11:35:50.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.072+0000 7fc610b88b80 1 -- 172.21.15.100:0/4269373971 wait complete. 2024-05-10T11:35:50.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.072+0000 7fc610b88b80 20 client.28318 trim_cache size 0 max 16384 2024-05-10T11:35:50.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.109+0000 7fc610b88b80 1 Processor -- start 2024-05-10T11:35:50.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.109+0000 7fc610b88b80 1 -- start start 2024-05-10T11:35:50.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.109+0000 7fc610b88b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c60f369d50 0x55c60f369810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:50.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.109+0000 7fc610b88b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c60f36f1b0 0x55c60f370740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:50.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.109+0000 7fc610b88b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c60f36fad0 0x55c60f37f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:50.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.109+0000 7fc610b88b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc6001af7f0 con 0x55c60f36f1b0 2024-05-10T11:35:50.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.109+0000 7fc610b88b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc6001ad080 con 0x55c60f369d50 2024-05-10T11:35:50.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.109+0000 7fc610b88b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc6001aaf90 con 0x55c60f36fad0 2024-05-10T11:35:50.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.110+0000 7fc5ff7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c60f36f1b0 0x55c60f370740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:50.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.110+0000 7fc604892700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c60f36fad0 0x55c60f37f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:50.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.110+0000 7fc5ff7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c60f36f1b0 0x55c60f370740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44974/0 (socket says 172.21.15.100:44974) 2024-05-10T11:35:50.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.110+0000 7fc5ff7fe700 1 -- 172.21.15.100:0/3472233901 learned_addr learned my addr 172.21.15.100:0/3472233901 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:35:50.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.110+0000 7fc5fffff700 1 --2- 172.21.15.100:0/3472233901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c60f369d50 0x55c60f369810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:50.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.110+0000 7fc5ff7fe700 1 -- 172.21.15.100:0/3472233901 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c60f36fad0 msgr2=0x55c60f37f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:50.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.110+0000 7fc5ff7fe700 1 --2- 172.21.15.100:0/3472233901 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c60f36fad0 0x55c60f37f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:50.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.110+0000 7fc5ff7fe700 1 -- 172.21.15.100:0/3472233901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c60f369d50 msgr2=0x55c60f369810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:50.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.110+0000 7fc5ff7fe700 1 --2- 172.21.15.100:0/3472233901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c60f369d50 0x55c60f369810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:50.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.110+0000 7fc5ff7fe700 1 -- 172.21.15.100:0/3472233901 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c60f3731a0 con 0x55c60f36f1b0 2024-05-10T11:35:50.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.110+0000 7fc5fffff700 1 --2- 172.21.15.100:0/3472233901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c60f369d50 0x55c60f369810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:35:50.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.111+0000 7fc5ff7fe700 1 --2- 172.21.15.100:0/3472233901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c60f36f1b0 0x55c60f370740 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fc5f40e0350 tx=0x7fc5f40c42b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bde3b86b6b35218b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:50.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.111+0000 7fc5e5ffb700 1 -- 172.21.15.100:0/3472233901 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f406d020 con 0x55c60f36f1b0 2024-05-10T11:35:50.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.111+0000 7fc5e5ffb700 1 -- 172.21.15.100:0/3472233901 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc5f40b22a0 con 0x55c60f36f1b0 2024-05-10T11:35:50.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.111+0000 7fc5e5ffb700 1 -- 172.21.15.100:0/3472233901 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f40adeb0 con 0x55c60f36f1b0 2024-05-10T11:35:50.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.112+0000 7fc610b88b80 1 -- 172.21.15.100:0/3472233901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c60f36f1b0 msgr2=0x55c60f370740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:50.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.112+0000 7fc610b88b80 1 --2- 172.21.15.100:0/3472233901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c60f36f1b0 0x55c60f370740 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fc5f40e0350 tx=0x7fc5f40c42b0 comp rx=0 tx=0).stop 2024-05-10T11:35:50.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.112+0000 7fc610b88b80 1 -- 172.21.15.100:0/3472233901 shutdown_connections 2024-05-10T11:35:50.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.112+0000 7fc610b88b80 1 --2- 172.21.15.100:0/3472233901 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c60f36fad0 0x55c60f37f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:50.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.112+0000 7fc610b88b80 1 --2- 172.21.15.100:0/3472233901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c60f369d50 0x55c60f369810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:50.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.112+0000 7fc610b88b80 1 --2- 172.21.15.100:0/3472233901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c60f36f1b0 0x55c60f370740 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:50.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.112+0000 7fc610b88b80 1 -- 172.21.15.100:0/3472233901 >> 172.21.15.100:0/3472233901 conn(0x55c60f295b70 msgr2=0x55c60f3713f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:50.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.112+0000 7fc610b88b80 1 -- 172.21.15.100:0/3472233901 shutdown_connections 2024-05-10T11:35:50.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.112+0000 7fc610b88b80 1 -- 172.21.15.100:0/3472233901 wait complete. 2024-05-10T11:35:50.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.113+0000 7fc610b88b80 1 -- start start 2024-05-10T11:35:50.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.113+0000 7fc610b88b80 1 Processor -- start 2024-05-10T11:35:50.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.113+0000 7fc610b88b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c60f369d50 0x55c60f38a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:50.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.113+0000 7fc610b88b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c60f36f1b0 0x55c60f38aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:50.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.113+0000 7fc610b88b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c60f36fad0 0x55c60f38b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:50.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.113+0000 7fc610b88b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc6001ab540 con 0x55c60f36fad0 2024-05-10T11:35:50.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.113+0000 7fc610b88b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc6001ac9b0 con 0x55c60f369d50 2024-05-10T11:35:50.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.113+0000 7fc610b88b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc6001abc10 con 0x55c60f36f1b0 2024-05-10T11:35:50.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.113+0000 7fc610b88b80 10 client.0 _ll_register_callbacks cb 0x55c60f24f5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:35:50.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.113+0000 7fc5fffff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c60f369d50 0x55c60f38a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:50.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.114+0000 7fc5ff7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c60f36f1b0 0x55c60f38aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:50.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.114+0000 7fc5fffff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c60f369d50 0x55c60f38a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:50162/0 (socket says 172.21.15.100:50162) 2024-05-10T11:35:50.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.114+0000 7fc5fffff700 1 -- 172.21.15.100:0/1203393322 learned_addr learned my addr 172.21.15.100:0/1203393322 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:35:50.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.114+0000 7fc604892700 1 --2- 172.21.15.100:0/1203393322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c60f36fad0 0x55c60f38b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:50.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.114+0000 7fc5ff7fe700 1 -- 172.21.15.100:0/1203393322 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c60f369d50 msgr2=0x55c60f38a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:50.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.114+0000 7fc5ff7fe700 1 --2- 172.21.15.100:0/1203393322 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c60f369d50 0x55c60f38a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:50.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.114+0000 7fc5ff7fe700 1 -- 172.21.15.100:0/1203393322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c60f36fad0 msgr2=0x55c60f38b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:35:50.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.114+0000 7fc5ff7fe700 1 --2- 172.21.15.100:0/1203393322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c60f36fad0 0x55c60f38b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:50.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.114+0000 7fc5ff7fe700 1 -- 172.21.15.100:0/1203393322 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c60f3728d0 con 0x55c60f36f1b0 2024-05-10T11:35:50.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.114+0000 7fc5fffff700 1 --2- 172.21.15.100:0/1203393322 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c60f369d50 0x55c60f38a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:35:50.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.114+0000 7fc604892700 1 --2- 172.21.15.100:0/1203393322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c60f36fad0 0x55c60f38b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:35:50.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.114+0000 7fc5ff7fe700 1 --2- 172.21.15.100:0/1203393322 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c60f36f1b0 0x55c60f38aca0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x55c60f360da0 tx=0x7fc5f40b29e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=332805b0447bae4b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:35:50.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.114+0000 7fc5df7fe700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:35:50.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.114+0000 7fc5df7fe700 1 -- 172.21.15.100:0/1203393322 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f406d070 con 0x55c60f36f1b0 2024-05-10T11:35:50.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.114+0000 7fc5df7fe700 1 -- 172.21.15.100:0/1203393322 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fc5f406f400 con 0x55c60f36f1b0 2024-05-10T11:35:50.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.114+0000 7fc610b88b80 1 -- 172.21.15.100:0/1203393322 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c60f2c33c0 con 0x55c60f36f1b0 2024-05-10T11:35:50.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.115+0000 7fc5df7fe700 1 -- 172.21.15.100:0/1203393322 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fc5f40a54c0 con 0x55c60f36f1b0 2024-05-10T11:35:50.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.115+0000 7fc610b88b80 10 client.28324 fetch_fsmap learned FSMap version 41 2024-05-10T11:35:50.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.115+0000 7fc610b88b80 1 -- 172.21.15.100:0/1203393322 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55c60f3728d0 con 0x55c60f36f1b0 2024-05-10T11:35:50.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.115+0000 7fc5df7fe700 1 -- 172.21.15.100:0/1203393322 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55c60f2c33c0 con 0x55c60f36f1b0 2024-05-10T11:35:50.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.115+0000 7fc5df7fe700 1 -- 172.21.15.100:0/1203393322 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fc5f40a87d0 con 0x55c60f36f1b0 2024-05-10T11:35:50.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.115+0000 7fc610b88b80 10 client.28324 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:35:50.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.115+0000 7fc610b88b80 10 client.28324 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:35:50.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.115+0000 7fc610b88b80 20 client.28324 populate_metadata read hostname 'smithi100' 2024-05-10T11:35:50.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.115+0000 7fc610b88b80 1 --2- 172.21.15.100:0/1203393322 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c60f3830a0 0x55c60f3efe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:35:50.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.115+0000 7fc610b88b80 4 client.28324 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T11:35:50.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.115+0000 7fc610b88b80 1 -- 172.21.15.100:0/1203393322 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c60f13b250 con 0x55c60f3830a0 2024-05-10T11:35:50.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.115+0000 7fc5fffff700 1 --2- 172.21.15.100:0/1203393322 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c60f3830a0 0x55c60f3efe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:35:50.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.116+0000 7fc5df7fe700 10 client.28324 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:35:50.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.116+0000 7fc5fffff700 1 --2- 172.21.15.100:0/1203393322 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c60f3830a0 0x55c60f3efe60 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fbaf6dcd860e245 server_cookie=af3434d25c22824 in_seq=0 out_seq=0 2024-05-10T11:35:50.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.116+0000 7fc5df7fe700 1 -- 172.21.15.100:0/1203393322 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55c60f13b250 con 0x55c60f3830a0 2024-05-10T11:35:50.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.116+0000 7fc5df7fe700 10 client.28324 handle_command_reply: tid=0 2024-05-10T11:35:50.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.116+0000 7fc610b88b80 1 client.28324 shutdown 2024-05-10T11:35:50.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.116+0000 7fc610b88b80 1 -- 172.21.15.100:0/1203393322 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c60f36f1b0 msgr2=0x55c60f38aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:50.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.116+0000 7fc610b88b80 1 --2- 172.21.15.100:0/1203393322 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c60f36f1b0 0x55c60f38aca0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x55c60f360da0 tx=0x7fc5f40b29e0 comp rx=0 tx=0).stop 2024-05-10T11:35:50.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.116+0000 7fc610b88b80 1 -- 172.21.15.100:0/1203393322 shutdown_connections 2024-05-10T11:35:50.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.116+0000 7fc610b88b80 1 --2- 172.21.15.100:0/1203393322 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c60f3830a0 0x55c60f3efe60 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:50.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.116+0000 7fc610b88b80 1 --2- 172.21.15.100:0/1203393322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c60f36fad0 0x55c60f38b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:50.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.116+0000 7fc5df7fe700 0 client.28324 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:35:50.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.116+0000 7fc610b88b80 1 --2- 172.21.15.100:0/1203393322 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c60f369d50 0x55c60f38a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:50.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.116+0000 7fc610b88b80 1 --2- 172.21.15.100:0/1203393322 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c60f36f1b0 0x55c60f38aca0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:50.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.116+0000 7fc610b88b80 1 -- 172.21.15.100:0/1203393322 >> 172.21.15.100:0/1203393322 conn(0x55c60f295b70 msgr2=0x55c60f371c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:50.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.116+0000 7fc610b88b80 1 -- 172.21.15.100:0/1203393322 shutdown_connections 2024-05-10T11:35:50.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.117+0000 7fc610b88b80 1 -- 172.21.15.100:0/1203393322 wait complete. 2024-05-10T11:35:50.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.117+0000 7fc610b88b80 20 client.28324 trim_cache size 0 max 16384 2024-05-10T11:35:50.120 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:35:50.120 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T11:35:50.120 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-05-10T11:35:50.120 INFO:teuthology.orchestra.run.smithi100.stdout: "8d2a3db4-b712-4de6-b636-e155fc51fdc2": { 2024-05-10T11:35:50.120 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-05-10T11:35:50.120 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "8d2a3db4-b712-4de6-b636-e155fc51fdc2", 2024-05-10T11:35:50.121 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-05-10T11:35:50.121 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:35:50.121 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:35:50.121 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:35:50.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.117+0000 7fc6062f5700 1 -- 172.21.15.100:0/1755621505 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc5e80b2880 msgr2=0x7fc5e80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:50.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.117+0000 7fc6062f5700 1 --2- 172.21.15.100:0/1755621505 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc5e80b2880 0x7fc5e80b4d40 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc5f000b2e0 tx=0x7fc5f0005fb0 comp rx=0 tx=0).stop 2024-05-10T11:35:50.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.117+0000 7fc6062f5700 1 -- 172.21.15.100:0/1755621505 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc600102d60 msgr2=0x7fc6001a1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:35:50.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.117+0000 7fc6062f5700 1 --2- 172.21.15.100:0/1755621505 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc600102d60 0x7fc6001a1f10 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc5f400e9c0 tx=0x7fc5f400ecd0 comp rx=0 tx=0).stop 2024-05-10T11:35:50.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.117+0000 7fc6062f5700 1 -- 172.21.15.100:0/1755621505 shutdown_connections 2024-05-10T11:35:50.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.117+0000 7fc6062f5700 1 --2- 172.21.15.100:0/1755621505 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc5e80b2880 0x7fc5e80b4d40 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:50.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.117+0000 7fc6062f5700 1 --2- 172.21.15.100:0/1755621505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc600108410 0x7fc60019bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:50.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.117+0000 7fc6062f5700 1 --2- 172.21.15.100:0/1755621505 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6001023a0 0x7fc6001a19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:50.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.117+0000 7fc6062f5700 1 --2- 172.21.15.100:0/1755621505 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc600102d60 0x7fc6001a1f10 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:35:50.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.117+0000 7fc6062f5700 1 -- 172.21.15.100:0/1755621505 >> 172.21.15.100:0/1755621505 conn(0x7fc600079350 msgr2=0x7fc600100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:35:50.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.118+0000 7fc6062f5700 1 -- 172.21.15.100:0/1755621505 shutdown_connections 2024-05-10T11:35:50.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:35:50.118+0000 7fc6062f5700 1 -- 172.21.15.100:0/1755621505 wait complete. 2024-05-10T11:35:50.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:35:50 smithi100 ceph-mon[30296]: pgmap v944: 129 pgs: 129 active+clean; 444 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 848 KiB/s rd, 4.4 MiB/s wr, 302 op/s 2024-05-10T11:35:50.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:35:50 smithi178 ceph-mon[33359]: pgmap v944: 129 pgs: 129 active+clean; 444 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 848 KiB/s rd, 4.4 MiB/s wr, 302 op/s 2024-05-10T11:35:50.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:35:50 smithi188 ceph-mon[34075]: pgmap v944: 129 pgs: 129 active+clean; 444 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 848 KiB/s rd, 4.4 MiB/s wr, 302 op/s 2024-05-10T11:35:50.921 INFO:tasks.workunit.client.0.smithi100.stdout: 10 37909 9.16 MB/sec execute 104 sec latency 95.267 ms 2024-05-10T11:35:51.054 INFO:tasks.cephfs.filesystem:scrub status for tag:8d2a3db4-b712-4de6-b636-e155fc51fdc2 - {'path': '/', 'tag': '8d2a3db4-b712-4de6-b636-e155fc51fdc2', 'options': 'recursive,force'} 2024-05-10T11:35:51.921 INFO:tasks.workunit.client.0.smithi100.stdout: 10 38065 9.12 MB/sec execute 105 sec latency 75.619 ms 2024-05-10T11:35:52.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:35:52 smithi178 ceph-mon[33359]: pgmap v945: 129 pgs: 129 active+clean; 449 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 859 KiB/s rd, 4.7 MiB/s wr, 306 op/s 2024-05-10T11:35:52.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:35:52 smithi188 ceph-mon[34075]: pgmap v945: 129 pgs: 129 active+clean; 449 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 859 KiB/s rd, 4.7 MiB/s wr, 306 op/s 2024-05-10T11:35:52.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:35:52 smithi100 ceph-mon[30296]: pgmap v945: 129 pgs: 129 active+clean; 449 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 859 KiB/s rd, 4.7 MiB/s wr, 306 op/s 2024-05-10T11:35:52.921 INFO:tasks.workunit.client.0.smithi100.stdout: 10 38208 9.11 MB/sec execute 106 sec latency 105.178 ms 2024-05-10T11:35:53.921 INFO:tasks.workunit.client.0.smithi100.stdout: 10 38384 9.16 MB/sec execute 107 sec latency 56.889 ms 2024-05-10T11:35:54.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:35:54 smithi178 ceph-mon[33359]: pgmap v946: 129 pgs: 129 active+clean; 445 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 4.6 MiB/s wr, 297 op/s 2024-05-10T11:35:54.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:35:54 smithi188 ceph-mon[34075]: pgmap v946: 129 pgs: 129 active+clean; 445 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 4.6 MiB/s wr, 297 op/s 2024-05-10T11:35:54.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:35:54 smithi100 ceph-mon[30296]: pgmap v946: 129 pgs: 129 active+clean; 445 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 4.6 MiB/s wr, 297 op/s 2024-05-10T11:35:54.921 INFO:tasks.workunit.client.0.smithi100.stdout: 10 38571 9.17 MB/sec execute 108 sec latency 68.744 ms 2024-05-10T11:35:55.921 INFO:tasks.workunit.client.0.smithi100.stdout: 10 38741 9.11 MB/sec execute 109 sec latency 146.988 ms 2024-05-10T11:35:56.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:35:56 smithi178 ceph-mon[33359]: pgmap v947: 129 pgs: 129 active+clean; 444 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1003 KiB/s rd, 5.2 MiB/s wr, 351 op/s 2024-05-10T11:35:56.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:35:56 smithi188 ceph-mon[34075]: pgmap v947: 129 pgs: 129 active+clean; 444 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1003 KiB/s rd, 5.2 MiB/s wr, 351 op/s 2024-05-10T11:35:56.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:35:56 smithi100 ceph-mon[30296]: pgmap v947: 129 pgs: 129 active+clean; 444 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1003 KiB/s rd, 5.2 MiB/s wr, 351 op/s 2024-05-10T11:35:56.921 INFO:tasks.workunit.client.0.smithi100.stdout: 10 38896 9.06 MB/sec execute 110 sec latency 121.581 ms 2024-05-10T11:35:57.921 INFO:tasks.workunit.client.0.smithi100.stdout: 10 39069 9.03 MB/sec execute 111 sec latency 82.805 ms 2024-05-10T11:35:58.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:35:58 smithi178 ceph-mon[33359]: pgmap v948: 129 pgs: 129 active+clean; 441 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 5.0 MiB/s wr, 365 op/s 2024-05-10T11:35:58.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:35:58 smithi188 ceph-mon[34075]: pgmap v948: 129 pgs: 129 active+clean; 441 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 5.0 MiB/s wr, 365 op/s 2024-05-10T11:35:58.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:35:58 smithi100 ceph-mon[30296]: pgmap v948: 129 pgs: 129 active+clean; 441 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 5.0 MiB/s wr, 365 op/s 2024-05-10T11:35:58.921 INFO:tasks.workunit.client.0.smithi100.stdout: 10 39249 9.04 MB/sec execute 112 sec latency 58.082 ms 2024-05-10T11:35:59.921 INFO:tasks.workunit.client.0.smithi100.stdout: 10 39411 9.04 MB/sec execute 113 sec latency 62.249 ms 2024-05-10T11:36:00.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:36:00 smithi178 ceph-mon[33359]: pgmap v949: 129 pgs: 129 active+clean; 441 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 3.9 MiB/s wr, 358 op/s 2024-05-10T11:36:00.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:36:00 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:36:00.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:36:00 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:36:00.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:36:00 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:36:00.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:36:00 smithi188 ceph-mon[34075]: pgmap v949: 129 pgs: 129 active+clean; 441 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 3.9 MiB/s wr, 358 op/s 2024-05-10T11:36:00.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:36:00 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:36:00.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:36:00 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:36:00.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:36:00 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:36:00.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:36:00 smithi100 ceph-mon[30296]: pgmap v949: 129 pgs: 129 active+clean; 441 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 3.9 MiB/s wr, 358 op/s 2024-05-10T11:36:00.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:36:00 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:36:00.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:36:00 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:36:00.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:36:00 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:36:00.921 INFO:tasks.workunit.client.0.smithi100.stdout: 10 39637 9.13 MB/sec execute 114 sec latency 72.577 ms 2024-05-10T11:36:01.921 INFO:tasks.workunit.client.0.smithi100.stdout: 10 39846 9.17 MB/sec execute 115 sec latency 150.434 ms 2024-05-10T11:36:02.245 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.245+0000 7fd1877c1640 1 -- 172.21.15.100:0/3117157628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd180100d00 msgr2=0x7fd180101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:02.246 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.245+0000 7fd1877c1640 1 --2- 172.21.15.100:0/3117157628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd180100d00 0x7fd180101120 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fd170009ca0 tx=0x7fd17002a8b0 comp rx=0 tx=0).stop 2024-05-10T11:36:02.246 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.245+0000 7fd1877c1640 1 -- 172.21.15.100:0/3117157628 shutdown_connections 2024-05-10T11:36:02.246 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.245+0000 7fd1877c1640 1 --2- 172.21.15.100:0/3117157628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd180104730 0x7fd18010afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:02.246 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.245+0000 7fd1877c1640 1 --2- 172.21.15.100:0/3117157628 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd180103d70 0x7fd1801041f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:02.246 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.245+0000 7fd1877c1640 1 --2- 172.21.15.100:0/3117157628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd180100d00 0x7fd180101120 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:02.246 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.245+0000 7fd1877c1640 1 -- 172.21.15.100:0/3117157628 >> 172.21.15.100:0/3117157628 conn(0x7fd1800fc630 msgr2=0x7fd1800fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:36:02.246 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.245+0000 7fd1877c1640 1 -- 172.21.15.100:0/3117157628 shutdown_connections 2024-05-10T11:36:02.246 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.245+0000 7fd1877c1640 1 -- 172.21.15.100:0/3117157628 wait complete. 2024-05-10T11:36:02.248 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.246+0000 7fd1877c1640 1 Processor -- start 2024-05-10T11:36:02.248 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.246+0000 7fd1877c1640 1 -- start start 2024-05-10T11:36:02.248 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.246+0000 7fd1877c1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd180100d00 0x7fd180198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:02.248 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.246+0000 7fd1877c1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd180103d70 0x7fd180199060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:02.248 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.246+0000 7fd1877c1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd180104730 0x7fd1801a00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:02.248 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.246+0000 7fd1877c1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd18010e480 con 0x7fd180104730 2024-05-10T11:36:02.248 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.246+0000 7fd1877c1640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd18010e300 con 0x7fd180103d70 2024-05-10T11:36:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.246+0000 7fd1877c1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd18010e600 con 0x7fd180100d00 2024-05-10T11:36:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.247+0000 7fd18555d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd180100d00 0x7fd180198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.247+0000 7fd185d5e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd180104730 0x7fd1801a00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.247+0000 7fd18555d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd180100d00 0x7fd180198b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:46752/0 (socket says 172.21.15.100:46752) 2024-05-10T11:36:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.247+0000 7fd18555d640 1 -- 172.21.15.100:0/2704049174 learned_addr learned my addr 172.21.15.100:0/2704049174 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:36:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.247+0000 7fd184d5c640 1 --2- 172.21.15.100:0/2704049174 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd180103d70 0x7fd180199060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.247+0000 7fd18555d640 1 -- 172.21.15.100:0/2704049174 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd180103d70 msgr2=0x7fd180199060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.247+0000 7fd18555d640 1 --2- 172.21.15.100:0/2704049174 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd180103d70 0x7fd180199060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.247+0000 7fd18555d640 1 -- 172.21.15.100:0/2704049174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd180104730 msgr2=0x7fd1801a00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.247+0000 7fd18555d640 1 --2- 172.21.15.100:0/2704049174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd180104730 0x7fd1801a00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.247+0000 7fd18555d640 1 -- 172.21.15.100:0/2704049174 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1801a0740 con 0x7fd180100d00 2024-05-10T11:36:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.247+0000 7fd185d5e640 1 --2- 172.21.15.100:0/2704049174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd180104730 0x7fd1801a00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:36:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.247+0000 7fd18555d640 1 --2- 172.21.15.100:0/2704049174 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd180100d00 0x7fd180198b20 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd170000f80 tx=0x7fd170003b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:36:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.247+0000 7fd17e7fc640 1 -- 172.21.15.100:0/2704049174 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd170004240 con 0x7fd180100d00 2024-05-10T11:36:02.250 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.247+0000 7fd1877c1640 1 -- 172.21.15.100:0/2704049174 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1801a09d0 con 0x7fd180100d00 2024-05-10T11:36:02.250 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.247+0000 7fd1877c1640 1 -- 172.21.15.100:0/2704049174 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1801a0f70 con 0x7fd180100d00 2024-05-10T11:36:02.250 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.247+0000 7fd17e7fc640 1 -- 172.21.15.100:0/2704049174 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd170040070 con 0x7fd180100d00 2024-05-10T11:36:02.250 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.247+0000 7fd17e7fc640 1 -- 172.21.15.100:0/2704049174 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd170045ac0 con 0x7fd180100d00 2024-05-10T11:36:02.252 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.251+0000 7fd18a5b2740 1 Processor -- start 2024-05-10T11:36:02.252 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.251+0000 7fd18a5b2740 1 -- start start 2024-05-10T11:36:02.252 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.251+0000 7fd17e7fc640 1 -- 172.21.15.100:0/2704049174 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fd170045c60 con 0x7fd180100d00 2024-05-10T11:36:02.252 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.251+0000 7fd18a5b2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56511e2cedb0 0x56511e2cf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:02.252 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.251+0000 7fd18a5b2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56511e2cf7a0 0x56511e2d42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:02.252 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.251+0000 7fd18a5b2740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56511e2d4910 0x56511e2d6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:02.252 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.251+0000 7fd18a5b2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56511e2335d0 con 0x56511e2cedb0 2024-05-10T11:36:02.252 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.251+0000 7fd18a5b2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56511e1ec890 con 0x56511e2cf7a0 2024-05-10T11:36:02.252 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.251+0000 7fd18a5b2740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56511e22f590 con 0x56511e2d4910 2024-05-10T11:36:02.253 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.251+0000 7fd185d5e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56511e2cedb0 0x56511e2cf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:02.253 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.251+0000 7fd185d5e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56511e2cedb0 0x56511e2cf1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44208/0 (socket says 172.21.15.100:44208) 2024-05-10T11:36:02.253 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.251+0000 7fd185d5e640 1 -- 172.21.15.100:0/2763440337 learned_addr learned my addr 172.21.15.100:0/2763440337 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:36:02.253 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.251+0000 7fd184d5c640 1 --2- 172.21.15.100:0/2763440337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56511e2d4910 0x56511e2d6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:02.253 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.251+0000 7fd18555d640 1 --2- 172.21.15.100:0/2763440337 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56511e2cf7a0 0x56511e2d42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:02.253 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.251+0000 7fd185d5e640 1 -- 172.21.15.100:0/2763440337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56511e2d4910 msgr2=0x56511e2d6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:02.253 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.251+0000 7fd185d5e640 1 --2- 172.21.15.100:0/2763440337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56511e2d4910 0x56511e2d6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:02.253 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.251+0000 7fd185d5e640 1 -- 172.21.15.100:0/2763440337 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56511e2cf7a0 msgr2=0x56511e2d42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:02.253 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.251+0000 7fd185d5e640 1 --2- 172.21.15.100:0/2763440337 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56511e2cf7a0 0x56511e2d42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:02.253 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.251+0000 7fd185d5e640 1 -- 172.21.15.100:0/2763440337 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56511df45210 con 0x56511e2cedb0 2024-05-10T11:36:02.253 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.252+0000 7fd185d5e640 1 --2- 172.21.15.100:0/2763440337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56511e2cedb0 0x56511e2cf1d0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fd178011540 tx=0x7fd178011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd18c455f5a425ba server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:36:02.253 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.252+0000 7fd1677fe640 1 -- 172.21.15.100:0/2763440337 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17800f5c0 con 0x56511e2cedb0 2024-05-10T11:36:02.253 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.252+0000 7fd1677fe640 1 -- 172.21.15.100:0/2763440337 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd17800fbf0 con 0x56511e2cedb0 2024-05-10T11:36:02.253 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.252+0000 7fd1677fe640 1 -- 172.21.15.100:0/2763440337 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd178019b20 con 0x56511e2cedb0 2024-05-10T11:36:02.257 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.252+0000 7fd17e7fc640 1 --2- 172.21.15.100:0/2704049174 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fd1680b28a0 0x7fd1680b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:02.257 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.253+0000 7fd18555d640 1 --2- 172.21.15.100:0/2704049174 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fd1680b28a0 0x7fd1680b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:02.257 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.253+0000 7fd18a5b2740 1 -- 172.21.15.100:0/2763440337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56511e2cedb0 msgr2=0x56511e2cf1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:02.257 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.253+0000 7fd18a5b2740 1 --2- 172.21.15.100:0/2763440337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56511e2cedb0 0x56511e2cf1d0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fd178011540 tx=0x7fd178011a00 comp rx=0 tx=0).stop 2024-05-10T11:36:02.257 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.253+0000 7fd17e7fc640 1 -- 172.21.15.100:0/2704049174 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fd170103840 con 0x7fd180100d00 2024-05-10T11:36:02.257 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.253+0000 7fd18555d640 1 --2- 172.21.15.100:0/2704049174 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fd1680b28a0 0x7fd1680b4d60 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fd170101350 tx=0x7fd170109550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:36:02.257 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.253+0000 7fd18a5b2740 1 -- 172.21.15.100:0/2763440337 shutdown_connections 2024-05-10T11:36:02.257 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.253+0000 7fd18a5b2740 1 --2- 172.21.15.100:0/2763440337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56511e2d4910 0x56511e2d6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:02.258 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.253+0000 7fd18a5b2740 1 --2- 172.21.15.100:0/2763440337 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56511e2cf7a0 0x56511e2d42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:02.258 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.253+0000 7fd18a5b2740 1 --2- 172.21.15.100:0/2763440337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56511e2cedb0 0x56511e2cf1d0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:02.258 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.253+0000 7fd18a5b2740 1 -- 172.21.15.100:0/2763440337 >> 172.21.15.100:0/2763440337 conn(0x56511e2ca420 msgr2=0x56511e2ca820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:36:02.258 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.253+0000 7fd18a5b2740 1 -- 172.21.15.100:0/2763440337 shutdown_connections 2024-05-10T11:36:02.258 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.253+0000 7fd18a5b2740 1 -- 172.21.15.100:0/2763440337 wait complete. 2024-05-10T11:36:02.258 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.254+0000 7fd18a5b2740 1 -- start start 2024-05-10T11:36:02.258 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.254+0000 7fd18a5b2740 1 Processor -- start 2024-05-10T11:36:02.258 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.254+0000 7fd18a5b2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56511e2cedb0 0x56511e2dd460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:02.258 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.254+0000 7fd18a5b2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56511e2cf7a0 0x56511e2dd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:02.258 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.254+0000 7fd18a5b2740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56511e2d4910 0x56511e2e49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:02.258 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.254+0000 7fd18a5b2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56511e2d78c0 con 0x56511e2cedb0 2024-05-10T11:36:02.258 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.254+0000 7fd18a5b2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56511e1f0370 con 0x56511e2cf7a0 2024-05-10T11:36:02.258 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.254+0000 7fd18a5b2740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56511e2d7a40 con 0x56511e2d4910 2024-05-10T11:36:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.254+0000 7fd18a5b2740 10 client.0 _ll_register_callbacks cb 0x56511df44a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:36:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.254+0000 7fd184d5c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56511e2cf7a0 0x56511e2dd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.254+0000 7fd184d5c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56511e2cf7a0 0x56511e2dd9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:56692/0 (socket says 172.21.15.100:56692) 2024-05-10T11:36:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.254+0000 7fd184d5c640 1 -- 172.21.15.100:0/1871653054 learned_addr learned my addr 172.21.15.100:0/1871653054 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:36:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.254+0000 7fd185d5e640 1 --2- 172.21.15.100:0/1871653054 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56511e2d4910 0x56511e2e49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.254+0000 7fd185d5e640 1 --2- 172.21.15.100:0/1871653054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56511e2cedb0 0x56511e2dd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.255+0000 7fd184d5c640 1 -- 172.21.15.100:0/1871653054 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56511e2d4910 msgr2=0x56511e2e49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.255+0000 7fd184d5c640 1 --2- 172.21.15.100:0/1871653054 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56511e2d4910 0x56511e2e49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.255+0000 7fd184d5c640 1 -- 172.21.15.100:0/1871653054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56511e2cedb0 msgr2=0x56511e2dd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.255+0000 7fd184d5c640 1 --2- 172.21.15.100:0/1871653054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56511e2cedb0 0x56511e2dd460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.255+0000 7fd184d5c640 1 -- 172.21.15.100:0/1871653054 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56511e011620 con 0x56511e2cf7a0 2024-05-10T11:36:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.255+0000 7fd184d5c640 1 --2- 172.21.15.100:0/1871653054 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56511e2cf7a0 0x56511e2dd9a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd174012700 tx=0x7fd174012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=293aca188069a22c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:36:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.255+0000 7fd164ff9640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:36:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.255+0000 7fd164ff9640 1 -- 172.21.15.100:0/1871653054 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17400f8c0 con 0x56511e2cf7a0 2024-05-10T11:36:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.255+0000 7fd18a5b2740 1 -- 172.21.15.100:0/1871653054 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd17800fbf0 con 0x56511e2cf7a0 2024-05-10T11:36:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.255+0000 7fd164ff9640 1 -- 172.21.15.100:0/1871653054 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fd17400fa60 con 0x56511e2cf7a0 2024-05-10T11:36:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.255+0000 7fd164ff9640 1 -- 172.21.15.100:0/1871653054 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fd174025050 con 0x56511e2cf7a0 2024-05-10T11:36:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.256+0000 7fd18a5b2740 10 client.28229 fetch_fsmap learned FSMap version 41 2024-05-10T11:36:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.256+0000 7fd18a5b2740 1 -- 172.21.15.100:0/1871653054 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x56511e2e7a30 con 0x56511e2cf7a0 2024-05-10T11:36:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.256+0000 7fd164ff9640 1 -- 172.21.15.100:0/1871653054 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd1740054b0 con 0x56511e2cf7a0 2024-05-10T11:36:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.256+0000 7fd164ff9640 1 -- 172.21.15.100:0/1871653054 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fd174004950 con 0x56511e2cf7a0 2024-05-10T11:36:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.256+0000 7fd18a5b2740 10 client.28229 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:36:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.256+0000 7fd18a5b2740 10 client.28229 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:36:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.256+0000 7fd18a5b2740 20 client.28229 populate_metadata read hostname 'smithi100' 2024-05-10T11:36:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.256+0000 7fd18a5b2740 1 --2- 172.21.15.100:0/1871653054 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56511e2e90a0 0x56511e2eb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.256+0000 7fd18a5b2740 4 client.28229 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:36:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.256+0000 7fd18a5b2740 1 -- 172.21.15.100:0/1871653054 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd17800fbf0 con 0x56511e2e90a0 2024-05-10T11:36:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.256+0000 7fd185d5e640 1 --2- 172.21.15.100:0/1871653054 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56511e2e90a0 0x56511e2eb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.257+0000 7fd185d5e640 1 --2- 172.21.15.100:0/1871653054 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56511e2e90a0 0x56511e2eb4d0 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d994bc36015409d server_cookie=2e18c7be0160a981 in_seq=0 out_seq=0 2024-05-10T11:36:02.261 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.257+0000 7fd164ff9640 10 client.28229 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:36:02.262 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.259+0000 7fd164ff9640 1 -- 172.21.15.100:0/1871653054 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fd17800fbf0 con 0x56511e2e90a0 2024-05-10T11:36:02.262 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.259+0000 7fd164ff9640 10 client.28229 handle_command_reply: tid=0 2024-05-10T11:36:02.262 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.259+0000 7fd18a5b2740 1 client.28229 shutdown 2024-05-10T11:36:02.262 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.260+0000 7fd18a5b2740 1 -- 172.21.15.100:0/1871653054 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56511e2cf7a0 msgr2=0x56511e2dd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:02.262 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.260+0000 7fd18a5b2740 1 --2- 172.21.15.100:0/1871653054 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56511e2cf7a0 0x56511e2dd9a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd174012700 tx=0x7fd174012bc0 comp rx=0 tx=0).stop 2024-05-10T11:36:02.262 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.260+0000 7fd18a5b2740 1 -- 172.21.15.100:0/1871653054 shutdown_connections 2024-05-10T11:36:02.262 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.260+0000 7fd18a5b2740 1 --2- 172.21.15.100:0/1871653054 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56511e2e90a0 0x56511e2eb4d0 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:02.262 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.260+0000 7fd18a5b2740 1 --2- 172.21.15.100:0/1871653054 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56511e2d4910 0x56511e2e49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:02.262 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.260+0000 7fd18a5b2740 1 --2- 172.21.15.100:0/1871653054 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56511e2cf7a0 0x56511e2dd9a0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:02.262 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.260+0000 7fd18a5b2740 1 --2- 172.21.15.100:0/1871653054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56511e2cedb0 0x56511e2dd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:02.263 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.260+0000 7fd18a5b2740 1 -- 172.21.15.100:0/1871653054 >> 172.21.15.100:0/1871653054 conn(0x56511e2ca420 msgr2=0x56511e2c9580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:36:02.263 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.260+0000 7fd164ff9640 0 client.28229 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:36:02.263 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.260+0000 7fd18a5b2740 1 -- 172.21.15.100:0/1871653054 shutdown_connections 2024-05-10T11:36:02.263 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.260+0000 7fd18a5b2740 1 -- 172.21.15.100:0/1871653054 wait complete. 2024-05-10T11:36:02.263 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.260+0000 7fd18a5b2740 20 client.28229 trim_cache size 0 max 16384 2024-05-10T11:36:02.286 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.285+0000 7fd18a5b2740 1 Processor -- start 2024-05-10T11:36:02.286 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.285+0000 7fd18a5b2740 1 -- start start 2024-05-10T11:36:02.286 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.285+0000 7fd18a5b2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56511e2cedb0 0x56511e2d0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:02.287 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.285+0000 7fd18a5b2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56511e2cf7a0 0x56511e2d0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:02.287 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.285+0000 7fd18a5b2740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56511e2d4910 0x56511e2e49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:02.287 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.285+0000 7fd18a5b2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd1801ab9c0 con 0x56511e2cf7a0 2024-05-10T11:36:02.287 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.285+0000 7fd18a5b2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd1801a9290 con 0x56511e2cedb0 2024-05-10T11:36:02.287 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.285+0000 7fd18a5b2740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd1801a70e0 con 0x56511e2d4910 2024-05-10T11:36:02.287 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.285+0000 7fd184d5c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56511e2cf7a0 0x56511e2d0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:02.287 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.285+0000 7fd185d5e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56511e2cedb0 0x56511e2d0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:02.287 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.285+0000 7fd185d5e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56511e2cedb0 0x56511e2d0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:56698/0 (socket says 172.21.15.100:56698) 2024-05-10T11:36:02.287 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.285+0000 7fd184d5c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56511e2cf7a0 0x56511e2d0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44220/0 (socket says 172.21.15.100:44220) 2024-05-10T11:36:02.287 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.285+0000 7fd185d5e640 1 -- 172.21.15.100:0/2944964578 learned_addr learned my addr 172.21.15.100:0/2944964578 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:36:02.287 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.285+0000 7fd185d5e640 1 --2- 172.21.15.100:0/2944964578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56511e2d4910 0x56511e2e49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:02.287 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.286+0000 7fd185d5e640 1 -- 172.21.15.100:0/2944964578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56511e2d4910 msgr2=0x56511e2e49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:02.287 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.286+0000 7fd185d5e640 1 --2- 172.21.15.100:0/2944964578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56511e2d4910 0x56511e2e49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:02.287 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.286+0000 7fd185d5e640 1 -- 172.21.15.100:0/2944964578 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56511e2cf7a0 msgr2=0x56511e2d0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:02.288 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.286+0000 7fd185d5e640 1 --2- 172.21.15.100:0/2944964578 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56511e2cf7a0 0x56511e2d0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:02.288 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.286+0000 7fd185d5e640 1 -- 172.21.15.100:0/2944964578 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56511e2ca250 con 0x56511e2cedb0 2024-05-10T11:36:02.288 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.286+0000 7fd184d5c640 1 --2- 172.21.15.100:0/2944964578 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56511e2cf7a0 0x56511e2d0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:36:02.288 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.286+0000 7fd185d5e640 1 --2- 172.21.15.100:0/2944964578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56511e2cedb0 0x56511e2d0a60 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd178009d90 tx=0x7fd178005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50a53edbdf018a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:36:02.288 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.286+0000 7fd166ffd640 1 -- 172.21.15.100:0/2944964578 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1780052b0 con 0x56511e2cedb0 2024-05-10T11:36:02.288 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.286+0000 7fd166ffd640 1 -- 172.21.15.100:0/2944964578 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd178019cb0 con 0x56511e2cedb0 2024-05-10T11:36:02.288 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.286+0000 7fd166ffd640 1 -- 172.21.15.100:0/2944964578 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17800f7c0 con 0x56511e2cedb0 2024-05-10T11:36:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.287+0000 7fd18a5b2740 1 -- 172.21.15.100:0/2944964578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56511e2cedb0 msgr2=0x56511e2d0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.287+0000 7fd18a5b2740 1 --2- 172.21.15.100:0/2944964578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56511e2cedb0 0x56511e2d0a60 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd178009d90 tx=0x7fd178005f90 comp rx=0 tx=0).stop 2024-05-10T11:36:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.287+0000 7fd18a5b2740 1 -- 172.21.15.100:0/2944964578 shutdown_connections 2024-05-10T11:36:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.287+0000 7fd18a5b2740 1 --2- 172.21.15.100:0/2944964578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56511e2d4910 0x56511e2e49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.287+0000 7fd18a5b2740 1 --2- 172.21.15.100:0/2944964578 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56511e2cf7a0 0x56511e2d0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.287+0000 7fd18a5b2740 1 --2- 172.21.15.100:0/2944964578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56511e2cedb0 0x56511e2d0a60 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.287+0000 7fd18a5b2740 1 -- 172.21.15.100:0/2944964578 >> 172.21.15.100:0/2944964578 conn(0x56511e2ca420 msgr2=0x56511e2ca800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:36:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.287+0000 7fd18a5b2740 1 -- 172.21.15.100:0/2944964578 shutdown_connections 2024-05-10T11:36:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.287+0000 7fd18a5b2740 1 -- 172.21.15.100:0/2944964578 wait complete. 2024-05-10T11:36:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.287+0000 7fd18a5b2740 1 -- start start 2024-05-10T11:36:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.288+0000 7fd18a5b2740 1 Processor -- start 2024-05-10T11:36:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.288+0000 7fd18a5b2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56511e2cedb0 0x56511e2e1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.288+0000 7fd18a5b2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56511e2cf7a0 0x56511e2e23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.288+0000 7fd18a5b2740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56511e2d4910 0x56511e2e33f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.288+0000 7fd18a5b2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd1801a7750 con 0x56511e2cedb0 2024-05-10T11:36:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.288+0000 7fd18a5b2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd1801a8bc0 con 0x56511e2cf7a0 2024-05-10T11:36:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.288+0000 7fd18a5b2740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd1801a7e20 con 0x56511e2d4910 2024-05-10T11:36:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.288+0000 7fd18a5b2740 10 client.0 _ll_register_callbacks cb 0x7fd174009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:36:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.288+0000 7fd185d5e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56511e2cedb0 0x56511e2e1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.288+0000 7fd185d5e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56511e2cedb0 0x56511e2e1ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44234/0 (socket says 172.21.15.100:44234) 2024-05-10T11:36:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.288+0000 7fd185d5e640 1 -- 172.21.15.100:0/3175205399 learned_addr learned my addr 172.21.15.100:0/3175205399 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:36:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.288+0000 7fd184d5c640 1 --2- 172.21.15.100:0/3175205399 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56511e2cf7a0 0x56511e2e23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.288+0000 7fd185d5e640 1 --2- 172.21.15.100:0/3175205399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56511e2d4910 0x56511e2e33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.288+0000 7fd185d5e640 1 -- 172.21.15.100:0/3175205399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56511e2d4910 msgr2=0x56511e2e33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.288+0000 7fd185d5e640 1 --2- 172.21.15.100:0/3175205399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56511e2d4910 0x56511e2e33f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.288+0000 7fd185d5e640 1 -- 172.21.15.100:0/3175205399 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56511e2cf7a0 msgr2=0x56511e2e23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.288+0000 7fd185d5e640 1 --2- 172.21.15.100:0/3175205399 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56511e2cf7a0 0x56511e2e23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.288+0000 7fd185d5e640 1 -- 172.21.15.100:0/3175205399 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56511e2cb0f0 con 0x56511e2cedb0 2024-05-10T11:36:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.289+0000 7fd185d5e640 1 --2- 172.21.15.100:0/3175205399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56511e2cedb0 0x56511e2e1ea0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fd178009350 tx=0x7fd178005670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a87598b55f25bef4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:36:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.289+0000 7fd164ff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:36:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.289+0000 7fd164ff9640 1 -- 172.21.15.100:0/3175205399 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1780198c0 con 0x56511e2cedb0 2024-05-10T11:36:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.289+0000 7fd18a5b2740 1 -- 172.21.15.100:0/3175205399 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd178019cb0 con 0x56511e2cedb0 2024-05-10T11:36:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.289+0000 7fd164ff9640 1 -- 172.21.15.100:0/3175205399 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fd178019a60 con 0x56511e2cedb0 2024-05-10T11:36:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.289+0000 7fd164ff9640 1 -- 172.21.15.100:0/3175205399 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fd17800f740 con 0x56511e2cedb0 2024-05-10T11:36:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.289+0000 7fd18a5b2740 10 client.18930 fetch_fsmap learned FSMap version 41 2024-05-10T11:36:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.289+0000 7fd18a5b2740 1 -- 172.21.15.100:0/3175205399 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x56511e2e4b20 con 0x56511e2cedb0 2024-05-10T11:36:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.289+0000 7fd164ff9640 1 -- 172.21.15.100:0/3175205399 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd178019cb0 con 0x56511e2cedb0 2024-05-10T11:36:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.289+0000 7fd164ff9640 1 -- 172.21.15.100:0/3175205399 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fd178040070 con 0x56511e2cedb0 2024-05-10T11:36:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.289+0000 7fd18a5b2740 10 client.18930 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:36:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.289+0000 7fd18a5b2740 10 client.18930 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:36:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.289+0000 7fd18a5b2740 20 client.18930 populate_metadata read hostname 'smithi100' 2024-05-10T11:36:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.289+0000 7fd18a5b2740 1 --2- 172.21.15.100:0/3175205399 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56511e353c30 0x56511e356020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.290+0000 7fd18a5b2740 4 client.18930 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:36:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.290+0000 7fd18a5b2740 1 -- 172.21.15.100:0/3175205399 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56511e2411e0 con 0x56511e353c30 2024-05-10T11:36:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.290+0000 7fd184d5c640 1 --2- 172.21.15.100:0/3175205399 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56511e353c30 0x56511e356020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.290+0000 7fd184d5c640 1 --2- 172.21.15.100:0/3175205399 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56511e353c30 0x56511e356020 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8b3e7c3b39db96e server_cookie=c593605d06eccd09 in_seq=0 out_seq=0 2024-05-10T11:36:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.290+0000 7fd164ff9640 10 client.18930 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:36:02.378 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.377+0000 7fd164ff9640 1 -- 172.21.15.100:0/3175205399 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56511e2411e0 con 0x56511e353c30 2024-05-10T11:36:02.378 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.377+0000 7fd164ff9640 10 client.18930 handle_command_reply: tid=0 2024-05-10T11:36:02.378 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.377+0000 7fd18a5b2740 1 client.18930 shutdown 2024-05-10T11:36:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.377+0000 7fd18a5b2740 1 -- 172.21.15.100:0/3175205399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56511e2cedb0 msgr2=0x56511e2e1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.377+0000 7fd18a5b2740 1 --2- 172.21.15.100:0/3175205399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56511e2cedb0 0x56511e2e1ea0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fd178009350 tx=0x7fd178005670 comp rx=0 tx=0).stop 2024-05-10T11:36:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.378+0000 7fd18a5b2740 1 -- 172.21.15.100:0/3175205399 shutdown_connections 2024-05-10T11:36:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.378+0000 7fd18a5b2740 1 --2- 172.21.15.100:0/3175205399 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56511e353c30 0x56511e356020 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.378+0000 7fd18a5b2740 1 --2- 172.21.15.100:0/3175205399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56511e2d4910 0x56511e2e33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.378+0000 7fd18a5b2740 1 --2- 172.21.15.100:0/3175205399 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56511e2cf7a0 0x56511e2e23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.378+0000 7fd18a5b2740 1 --2- 172.21.15.100:0/3175205399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56511e2cedb0 0x56511e2e1ea0 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.378+0000 7fd18a5b2740 1 -- 172.21.15.100:0/3175205399 >> 172.21.15.100:0/3175205399 conn(0x56511e2ca420 msgr2=0x56511e2dd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:36:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.378+0000 7fd164ff9640 0 client.18930 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:36:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.378+0000 7fd18a5b2740 1 -- 172.21.15.100:0/3175205399 shutdown_connections 2024-05-10T11:36:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.378+0000 7fd18a5b2740 1 -- 172.21.15.100:0/3175205399 wait complete. 2024-05-10T11:36:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.378+0000 7fd18a5b2740 20 client.18930 trim_cache size 0 max 16384 2024-05-10T11:36:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:36:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:36:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:36:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:36:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.379+0000 7fd1877c1640 1 -- 172.21.15.100:0/2704049174 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fd1680b28a0 msgr2=0x7fd1680b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.379+0000 7fd1877c1640 1 --2- 172.21.15.100:0/2704049174 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fd1680b28a0 0x7fd1680b4d60 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fd170101350 tx=0x7fd170109550 comp rx=0 tx=0).stop 2024-05-10T11:36:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.379+0000 7fd1877c1640 1 -- 172.21.15.100:0/2704049174 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd180100d00 msgr2=0x7fd180198b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.379+0000 7fd1877c1640 1 --2- 172.21.15.100:0/2704049174 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd180100d00 0x7fd180198b20 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd170000f80 tx=0x7fd170003b80 comp rx=0 tx=0).stop 2024-05-10T11:36:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.379+0000 7fd1877c1640 1 -- 172.21.15.100:0/2704049174 shutdown_connections 2024-05-10T11:36:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.379+0000 7fd1877c1640 1 --2- 172.21.15.100:0/2704049174 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fd1680b28a0 0x7fd1680b4d60 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.379+0000 7fd1877c1640 1 --2- 172.21.15.100:0/2704049174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd180104730 0x7fd1801a00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.379+0000 7fd1877c1640 1 --2- 172.21.15.100:0/2704049174 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd180103d70 0x7fd180199060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.379+0000 7fd1877c1640 1 --2- 172.21.15.100:0/2704049174 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd180100d00 0x7fd180198b20 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.379+0000 7fd1877c1640 1 -- 172.21.15.100:0/2704049174 >> 172.21.15.100:0/2704049174 conn(0x7fd1800fc630 msgr2=0x7fd1801070c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:36:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.379+0000 7fd1877c1640 1 -- 172.21.15.100:0/2704049174 shutdown_connections 2024-05-10T11:36:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:02.379+0000 7fd1877c1640 1 -- 172.21.15.100:0/2704049174 wait complete. 2024-05-10T11:36:02.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:36:02 smithi178 ceph-mon[33359]: pgmap v950: 129 pgs: 129 active+clean; 442 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 986 KiB/s rd, 4.0 MiB/s wr, 355 op/s 2024-05-10T11:36:02.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:36:02 smithi188 ceph-mon[34075]: pgmap v950: 129 pgs: 129 active+clean; 442 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 986 KiB/s rd, 4.0 MiB/s wr, 355 op/s 2024-05-10T11:36:02.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:36:02 smithi100 ceph-mon[30296]: pgmap v950: 129 pgs: 129 active+clean; 442 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 986 KiB/s rd, 4.0 MiB/s wr, 355 op/s 2024-05-10T11:36:02.921 INFO:tasks.workunit.client.0.smithi100.stdout: 10 40017 9.14 MB/sec execute 116 sec latency 102.742 ms 2024-05-10T11:36:03.921 INFO:tasks.workunit.client.0.smithi100.stdout: 10 40172 9.15 MB/sec execute 117 sec latency 112.147 ms 2024-05-10T11:36:04.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:36:04 smithi178 ceph-mon[33359]: pgmap v951: 129 pgs: 129 active+clean; 441 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 756 KiB/s rd, 3.6 MiB/s wr, 303 op/s 2024-05-10T11:36:04.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:36:04 smithi188 ceph-mon[34075]: pgmap v951: 129 pgs: 129 active+clean; 441 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 756 KiB/s rd, 3.6 MiB/s wr, 303 op/s 2024-05-10T11:36:04.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:36:04 smithi100 ceph-mon[30296]: pgmap v951: 129 pgs: 129 active+clean; 441 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 756 KiB/s rd, 3.6 MiB/s wr, 303 op/s 2024-05-10T11:36:04.921 INFO:tasks.workunit.client.0.smithi100.stdout: 10 40339 9.16 MB/sec execute 118 sec latency 65.234 ms 2024-05-10T11:36:05.921 INFO:tasks.workunit.client.0.smithi100.stdout: 10 40517 9.17 MB/sec execute 119 sec latency 60.900 ms 2024-05-10T11:36:06.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:36:06 smithi178 ceph-mon[33359]: pgmap v952: 129 pgs: 129 active+clean; 447 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 4.1 MiB/s wr, 326 op/s 2024-05-10T11:36:06.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:36:06 smithi188 ceph-mon[34075]: pgmap v952: 129 pgs: 129 active+clean; 447 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 4.1 MiB/s wr, 326 op/s 2024-05-10T11:36:06.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:36:06 smithi100 ceph-mon[30296]: pgmap v952: 129 pgs: 129 active+clean; 447 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 4.1 MiB/s wr, 326 op/s 2024-05-10T11:36:06.922 INFO:tasks.workunit.client.0.smithi100.stdout: 10 40710 9.21 MB/sec execute 120 sec latency 60.783 ms 2024-05-10T11:36:07.923 INFO:tasks.workunit.client.0.smithi100.stdout: 10 40890 9.18 MB/sec execute 121 sec latency 68.584 ms 2024-05-10T11:36:08.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:36:08 smithi178 ceph-mon[33359]: pgmap v953: 129 pgs: 129 active+clean; 452 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 4.3 MiB/s wr, 298 op/s 2024-05-10T11:36:08.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:36:08 smithi188 ceph-mon[34075]: pgmap v953: 129 pgs: 129 active+clean; 452 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 4.3 MiB/s wr, 298 op/s 2024-05-10T11:36:08.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:36:08 smithi100 ceph-mon[30296]: pgmap v953: 129 pgs: 129 active+clean; 452 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 4.3 MiB/s wr, 298 op/s 2024-05-10T11:36:08.923 INFO:tasks.workunit.client.0.smithi100.stdout: 10 41043 9.16 MB/sec execute 122 sec latency 70.542 ms 2024-05-10T11:36:09.924 INFO:tasks.workunit.client.0.smithi100.stdout: 10 41205 9.14 MB/sec execute 123 sec latency 71.148 ms 2024-05-10T11:36:10.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:36:10 smithi178 ceph-mon[33359]: pgmap v954: 129 pgs: 129 active+clean; 454 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 4.1 MiB/s wr, 270 op/s 2024-05-10T11:36:10.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:36:10 smithi188 ceph-mon[34075]: pgmap v954: 129 pgs: 129 active+clean; 454 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 4.1 MiB/s wr, 270 op/s 2024-05-10T11:36:10.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:36:10 smithi100 ceph-mon[30296]: pgmap v954: 129 pgs: 129 active+clean; 454 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 4.1 MiB/s wr, 270 op/s 2024-05-10T11:36:10.924 INFO:tasks.workunit.client.0.smithi100.stdout: 10 41419 9.20 MB/sec execute 124 sec latency 106.218 ms 2024-05-10T11:36:11.924 INFO:tasks.workunit.client.0.smithi100.stdout: 10 41600 9.23 MB/sec execute 125 sec latency 95.342 ms 2024-05-10T11:36:12.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:36:12 smithi178 ceph-mon[33359]: pgmap v955: 129 pgs: 129 active+clean; 458 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 982 KiB/s rd, 5.0 MiB/s wr, 298 op/s 2024-05-10T11:36:12.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:36:12 smithi188 ceph-mon[34075]: pgmap v955: 129 pgs: 129 active+clean; 458 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 982 KiB/s rd, 5.0 MiB/s wr, 298 op/s 2024-05-10T11:36:12.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:36:12 smithi100 ceph-mon[30296]: pgmap v955: 129 pgs: 129 active+clean; 458 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 982 KiB/s rd, 5.0 MiB/s wr, 298 op/s 2024-05-10T11:36:12.924 INFO:tasks.workunit.client.0.smithi100.stdout: 10 41743 9.19 MB/sec execute 126 sec latency 73.494 ms 2024-05-10T11:36:13.924 INFO:tasks.workunit.client.0.smithi100.stdout: 10 41919 9.18 MB/sec execute 127 sec latency 61.215 ms 2024-05-10T11:36:14.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:36:14 smithi178 ceph-mon[33359]: pgmap v956: 129 pgs: 129 active+clean; 457 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 4.5 MiB/s wr, 252 op/s 2024-05-10T11:36:14.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:36:14 smithi188 ceph-mon[34075]: pgmap v956: 129 pgs: 129 active+clean; 457 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 4.5 MiB/s wr, 252 op/s 2024-05-10T11:36:14.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:36:14 smithi100 ceph-mon[30296]: pgmap v956: 129 pgs: 129 active+clean; 457 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 4.5 MiB/s wr, 252 op/s 2024-05-10T11:36:14.924 INFO:tasks.workunit.client.0.smithi100.stdout: 10 42102 9.18 MB/sec execute 128 sec latency 66.174 ms 2024-05-10T11:36:15.513 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.512+0000 7ff979b9e640 1 -- 172.21.15.100:0/2010992211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9741019f0 msgr2=0x7ff974101e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:15.513 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.512+0000 7ff979b9e640 1 --2- 172.21.15.100:0/2010992211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9741019f0 0x7ff974101e70 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7ff960009900 tx=0x7ff96002a410 comp rx=0 tx=0).stop 2024-05-10T11:36:15.513 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.512+0000 7ff979b9e640 1 -- 172.21.15.100:0/2010992211 shutdown_connections 2024-05-10T11:36:15.513 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.512+0000 7ff979b9e640 1 --2- 172.21.15.100:0/2010992211 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9741023b0 0x7ff974109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:15.513 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.512+0000 7ff979b9e640 1 --2- 172.21.15.100:0/2010992211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9741019f0 0x7ff974101e70 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:15.513 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.512+0000 7ff979b9e640 1 --2- 172.21.15.100:0/2010992211 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff974102cc0 0x7ff9741030e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:15.513 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.512+0000 7ff979b9e640 1 -- 172.21.15.100:0/2010992211 >> 172.21.15.100:0/2010992211 conn(0x7ff9740fc440 msgr2=0x7ff9740fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:36:15.513 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.513+0000 7ff979b9e640 1 -- 172.21.15.100:0/2010992211 shutdown_connections 2024-05-10T11:36:15.513 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.513+0000 7ff979b9e640 1 -- 172.21.15.100:0/2010992211 wait complete. 2024-05-10T11:36:15.515 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.513+0000 7ff979b9e640 1 Processor -- start 2024-05-10T11:36:15.515 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.514+0000 7ff979b9e640 1 -- start start 2024-05-10T11:36:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.514+0000 7ff979b9e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9741019f0 0x7ff97410c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.514+0000 7ff979b9e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9741023b0 0x7ff97410a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.514+0000 7ff979b9e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff974102cc0 0x7ff97410aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.514+0000 7ff979b9e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff97410e270 con 0x7ff9741019f0 2024-05-10T11:36:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.514+0000 7ff979b9e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff97410e0f0 con 0x7ff9741023b0 2024-05-10T11:36:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.514+0000 7ff979b9e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff97410e3f0 con 0x7ff974102cc0 2024-05-10T11:36:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.514+0000 7ff978b9c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9741019f0 0x7ff97410c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.514+0000 7ff978b9c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9741019f0 0x7ff97410c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43234/0 (socket says 172.21.15.100:43234) 2024-05-10T11:36:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.514+0000 7ff978b9c640 1 -- 172.21.15.100:0/4178751353 learned_addr learned my addr 172.21.15.100:0/4178751353 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:36:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.514+0000 7ff973fff640 1 --2- 172.21.15.100:0/4178751353 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9741023b0 0x7ff97410a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.514+0000 7ff97939d640 1 --2- 172.21.15.100:0/4178751353 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff974102cc0 0x7ff97410aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.514+0000 7ff978b9c640 1 -- 172.21.15.100:0/4178751353 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff974102cc0 msgr2=0x7ff97410aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.514+0000 7ff978b9c640 1 --2- 172.21.15.100:0/4178751353 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff974102cc0 0x7ff97410aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:15.517 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.514+0000 7ff978b9c640 1 -- 172.21.15.100:0/4178751353 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9741023b0 msgr2=0x7ff97410a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:15.517 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.514+0000 7ff978b9c640 1 --2- 172.21.15.100:0/4178751353 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9741023b0 0x7ff97410a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:15.517 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.514+0000 7ff978b9c640 1 -- 172.21.15.100:0/4178751353 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff97410b760 con 0x7ff9741019f0 2024-05-10T11:36:15.517 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.515+0000 7ff978b9c640 1 --2- 172.21.15.100:0/4178751353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9741019f0 0x7ff97410c2d0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7ff96800e9c0 tx=0x7ff96800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:36:15.517 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.515+0000 7ff971ffb640 1 -- 172.21.15.100:0/4178751353 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96800cde0 con 0x7ff9741019f0 2024-05-10T11:36:15.517 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.515+0000 7ff979b9e640 1 -- 172.21.15.100:0/4178751353 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9741a2b00 con 0x7ff9741019f0 2024-05-10T11:36:15.517 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.515+0000 7ff971ffb640 1 -- 172.21.15.100:0/4178751353 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff9680108c0 con 0x7ff9741019f0 2024-05-10T11:36:15.517 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.515+0000 7ff971ffb640 1 -- 172.21.15.100:0/4178751353 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff968018820 con 0x7ff9741019f0 2024-05-10T11:36:15.517 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.515+0000 7ff979b9e640 1 -- 172.21.15.100:0/4178751353 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9741a3070 con 0x7ff9741019f0 2024-05-10T11:36:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.518+0000 7ff971ffb640 1 -- 172.21.15.100:0/4178751353 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7ff96801f020 con 0x7ff9741019f0 2024-05-10T11:36:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.518+0000 7ff97dbf8740 1 Processor -- start 2024-05-10T11:36:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.518+0000 7ff97dbf8740 1 -- start start 2024-05-10T11:36:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.518+0000 7ff97dbf8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e6f8ba7f80 0x55e6f8ba83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.518+0000 7ff97dbf8740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e6f8ba8970 0x55e6f8bad470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.519+0000 7ff97dbf8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6f8badae0 0x55e6f8baff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.519+0000 7ff97dbf8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55e6f8b0c5d0 con 0x55e6f8badae0 2024-05-10T11:36:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.519+0000 7ff97dbf8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e6f8ac5890 con 0x55e6f8ba7f80 2024-05-10T11:36:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.519+0000 7ff97dbf8740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55e6f8b08590 con 0x55e6f8ba8970 2024-05-10T11:36:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.519+0000 7ff973fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6f8badae0 0x55e6f8baff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.519+0000 7ff973fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6f8badae0 0x55e6f8baff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43246/0 (socket says 172.21.15.100:43246) 2024-05-10T11:36:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.519+0000 7ff973fff640 1 -- 172.21.15.100:0/208882910 learned_addr learned my addr 172.21.15.100:0/208882910 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:36:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.519+0000 7ff978b9c640 1 --2- 172.21.15.100:0/208882910 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e6f8ba8970 0x55e6f8bad470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:15.521 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.519+0000 7ff97939d640 1 --2- 172.21.15.100:0/208882910 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e6f8ba7f80 0x55e6f8ba83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:15.521 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.519+0000 7ff97939d640 1 -- 172.21.15.100:0/208882910 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e6f8ba8970 msgr2=0x55e6f8bad470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:15.521 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.519+0000 7ff97939d640 1 --2- 172.21.15.100:0/208882910 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e6f8ba8970 0x55e6f8bad470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:15.521 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.519+0000 7ff97939d640 1 -- 172.21.15.100:0/208882910 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6f8badae0 msgr2=0x55e6f8baff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:15.521 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.519+0000 7ff97939d640 1 --2- 172.21.15.100:0/208882910 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6f8badae0 0x55e6f8baff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:15.521 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.519+0000 7ff97939d640 1 -- 172.21.15.100:0/208882910 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6f881e210 con 0x55e6f8ba7f80 2024-05-10T11:36:15.521 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.519+0000 7ff971ffb640 1 --2- 172.21.15.100:0/4178751353 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff9640b27e0 0x7ff9640b4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:15.521 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.519+0000 7ff97939d640 1 --2- 172.21.15.100:0/208882910 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e6f8ba7f80 0x55e6f8ba83a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff96c009f00 tx=0x7ff96c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77cf42a73106be0f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:36:15.521 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.520+0000 7ff978b9c640 1 --2- 172.21.15.100:0/4178751353 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff9640b27e0 0x7ff9640b4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:15.521 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.520+0000 7ff95effd640 1 -- 172.21.15.100:0/208882910 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96c004d20 con 0x55e6f8ba7f80 2024-05-10T11:36:15.521 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.520+0000 7ff95effd640 1 -- 172.21.15.100:0/208882910 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff96c004030 con 0x55e6f8ba7f80 2024-05-10T11:36:15.521 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.520+0000 7ff95effd640 1 -- 172.21.15.100:0/208882910 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96c005280 con 0x55e6f8ba7f80 2024-05-10T11:36:15.521 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.520+0000 7ff971ffb640 1 -- 172.21.15.100:0/4178751353 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7ff9680e26f0 con 0x7ff9741019f0 2024-05-10T11:36:15.524 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.521+0000 7ff978b9c640 1 --2- 172.21.15.100:0/4178751353 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff9640b27e0 0x7ff9640b4ca0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ff9680e0200 tx=0x7ff9680e8550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:36:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.521+0000 7ff97dbf8740 1 -- 172.21.15.100:0/208882910 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e6f8ba7f80 msgr2=0x55e6f8ba83a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.521+0000 7ff97dbf8740 1 --2- 172.21.15.100:0/208882910 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e6f8ba7f80 0x55e6f8ba83a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff96c009f00 tx=0x7ff96c011810 comp rx=0 tx=0).stop 2024-05-10T11:36:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.521+0000 7ff97dbf8740 1 -- 172.21.15.100:0/208882910 shutdown_connections 2024-05-10T11:36:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.521+0000 7ff97dbf8740 1 --2- 172.21.15.100:0/208882910 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6f8badae0 0x55e6f8baff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.521+0000 7ff97dbf8740 1 --2- 172.21.15.100:0/208882910 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e6f8ba8970 0x55e6f8bad470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.521+0000 7ff97dbf8740 1 --2- 172.21.15.100:0/208882910 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e6f8ba7f80 0x55e6f8ba83a0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.521+0000 7ff97dbf8740 1 -- 172.21.15.100:0/208882910 >> 172.21.15.100:0/208882910 conn(0x55e6f8ba35f0 msgr2=0x55e6f8ba39f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:36:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.521+0000 7ff97dbf8740 1 -- 172.21.15.100:0/208882910 shutdown_connections 2024-05-10T11:36:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.521+0000 7ff97dbf8740 1 -- 172.21.15.100:0/208882910 wait complete. 2024-05-10T11:36:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.521+0000 7ff97dbf8740 1 -- start start 2024-05-10T11:36:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.521+0000 7ff97dbf8740 1 Processor -- start 2024-05-10T11:36:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.522+0000 7ff97dbf8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e6f8ba7f80 0x55e6f8bb2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.522+0000 7ff97dbf8740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e6f8ba8970 0x55e6f8bb27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.522+0000 7ff97dbf8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6f8badae0 0x55e6f8bb97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.522+0000 7ff97dbf8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55e6f8bb0a90 con 0x55e6f8badae0 2024-05-10T11:36:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.522+0000 7ff97dbf8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e6f8ac9370 con 0x55e6f8ba7f80 2024-05-10T11:36:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.522+0000 7ff97dbf8740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55e6f8bb0c10 con 0x55e6f8ba8970 2024-05-10T11:36:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.522+0000 7ff97dbf8740 10 client.0 _ll_register_callbacks cb 0x55e6f881da00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:36:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.522+0000 7ff973fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e6f8ba8970 0x55e6f8bb27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.522+0000 7ff973fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e6f8ba8970 0x55e6f8bb27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:39978/0 (socket says 172.21.15.100:39978) 2024-05-10T11:36:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.522+0000 7ff973fff640 1 -- 172.21.15.100:0/3626492028 learned_addr learned my addr 172.21.15.100:0/3626492028 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:36:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.522+0000 7ff97939d640 1 --2- 172.21.15.100:0/3626492028 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e6f8ba7f80 0x55e6f8bb2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.522+0000 7ff97939d640 1 --2- 172.21.15.100:0/3626492028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6f8badae0 0x55e6f8bb97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.522+0000 7ff973fff640 1 -- 172.21.15.100:0/3626492028 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e6f8ba7f80 msgr2=0x55e6f8bb2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.522+0000 7ff973fff640 1 --2- 172.21.15.100:0/3626492028 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e6f8ba7f80 0x55e6f8bb2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.522+0000 7ff973fff640 1 -- 172.21.15.100:0/3626492028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6f8badae0 msgr2=0x55e6f8bb97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.522+0000 7ff973fff640 1 --2- 172.21.15.100:0/3626492028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6f8badae0 0x55e6f8bb97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:15.527 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.522+0000 7ff973fff640 1 -- 172.21.15.100:0/3626492028 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6f88ea620 con 0x55e6f8ba8970 2024-05-10T11:36:15.527 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.523+0000 7ff973fff640 1 --2- 172.21.15.100:0/3626492028 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e6f8ba8970 0x55e6f8bb27b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff96002aa00 tx=0x7ff9600042e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d13f4163dad054 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:36:15.527 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.523+0000 7ff953fff640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:36:15.527 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.523+0000 7ff953fff640 1 -- 172.21.15.100:0/3626492028 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff960035040 con 0x55e6f8ba8970 2024-05-10T11:36:15.527 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.523+0000 7ff953fff640 1 -- 172.21.15.100:0/3626492028 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7ff960003a00 con 0x55e6f8ba8970 2024-05-10T11:36:15.527 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.523+0000 7ff97dbf8740 1 -- 172.21.15.100:0/3626492028 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff96c004030 con 0x55e6f8ba8970 2024-05-10T11:36:15.527 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.523+0000 7ff953fff640 1 -- 172.21.15.100:0/3626492028 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7ff960003c00 con 0x55e6f8ba8970 2024-05-10T11:36:15.527 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.523+0000 7ff97dbf8740 10 client.28354 fetch_fsmap learned FSMap version 41 2024-05-10T11:36:15.527 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.523+0000 7ff97dbf8740 1 -- 172.21.15.100:0/3626492028 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55e6f8bbc840 con 0x55e6f8ba8970 2024-05-10T11:36:15.527 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.524+0000 7ff953fff640 1 -- 172.21.15.100:0/3626492028 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff96c004030 con 0x55e6f8ba8970 2024-05-10T11:36:15.527 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.524+0000 7ff953fff640 1 -- 172.21.15.100:0/3626492028 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7ff96003c440 con 0x55e6f8ba8970 2024-05-10T11:36:15.527 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.524+0000 7ff97dbf8740 10 client.28354 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:36:15.527 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.524+0000 7ff97dbf8740 10 client.28354 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:36:15.527 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.524+0000 7ff97dbf8740 20 client.28354 populate_metadata read hostname 'smithi100' 2024-05-10T11:36:15.527 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.524+0000 7ff97dbf8740 1 --2- 172.21.15.100:0/3626492028 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55e6f8bbdfd0 0x55e6f8bc03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:15.528 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.524+0000 7ff97dbf8740 4 client.28354 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:36:15.528 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.524+0000 7ff97dbf8740 1 -- 172.21.15.100:0/3626492028 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e6f8b1a1e0 con 0x55e6f8bbdfd0 2024-05-10T11:36:15.528 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.524+0000 7ff97939d640 1 --2- 172.21.15.100:0/3626492028 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55e6f8bbdfd0 0x55e6f8bc03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:15.528 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.525+0000 7ff953fff640 10 client.28354 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:36:15.528 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.525+0000 7ff97939d640 1 --2- 172.21.15.100:0/3626492028 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55e6f8bbdfd0 0x55e6f8bc03e0 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e53302e25b3c3890 server_cookie=fa0696d5a51aee76 in_seq=0 out_seq=0 2024-05-10T11:36:15.529 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.525+0000 7ff953fff640 1 -- 172.21.15.100:0/3626492028 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e6f8b1a1e0 con 0x55e6f8bbdfd0 2024-05-10T11:36:15.529 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.525+0000 7ff953fff640 10 client.28354 handle_command_reply: tid=0 2024-05-10T11:36:15.529 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.526+0000 7ff97dbf8740 1 client.28354 shutdown 2024-05-10T11:36:15.529 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.526+0000 7ff97dbf8740 1 -- 172.21.15.100:0/3626492028 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e6f8ba8970 msgr2=0x55e6f8bb27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:15.529 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.526+0000 7ff97dbf8740 1 --2- 172.21.15.100:0/3626492028 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e6f8ba8970 0x55e6f8bb27b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff96002aa00 tx=0x7ff9600042e0 comp rx=0 tx=0).stop 2024-05-10T11:36:15.529 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.526+0000 7ff97dbf8740 1 -- 172.21.15.100:0/3626492028 shutdown_connections 2024-05-10T11:36:15.529 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.526+0000 7ff97dbf8740 1 --2- 172.21.15.100:0/3626492028 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55e6f8bbdfd0 0x55e6f8bc03e0 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:15.529 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.526+0000 7ff953fff640 0 client.28354 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:36:15.529 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.526+0000 7ff97dbf8740 1 --2- 172.21.15.100:0/3626492028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6f8badae0 0x55e6f8bb97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:15.529 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.526+0000 7ff97dbf8740 1 --2- 172.21.15.100:0/3626492028 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e6f8ba8970 0x55e6f8bb27b0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:15.529 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.526+0000 7ff97dbf8740 1 --2- 172.21.15.100:0/3626492028 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e6f8ba7f80 0x55e6f8bb2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:15.529 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.526+0000 7ff97dbf8740 1 -- 172.21.15.100:0/3626492028 >> 172.21.15.100:0/3626492028 conn(0x55e6f8ba35f0 msgr2=0x55e6f8ba2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:36:15.530 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.526+0000 7ff97dbf8740 1 -- 172.21.15.100:0/3626492028 shutdown_connections 2024-05-10T11:36:15.530 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.526+0000 7ff97dbf8740 1 -- 172.21.15.100:0/3626492028 wait complete. 2024-05-10T11:36:15.530 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.526+0000 7ff97dbf8740 20 client.28354 trim_cache size 0 max 16384 2024-05-10T11:36:15.550 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.550+0000 7ff97dbf8740 1 Processor -- start 2024-05-10T11:36:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.550+0000 7ff97dbf8740 1 -- start start 2024-05-10T11:36:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.550+0000 7ff97dbf8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6f8ba7f80 0x55e6f8ba9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.550+0000 7ff97dbf8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e6f8ba8970 0x55e6f8baa1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.550+0000 7ff97dbf8740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e6f8badae0 0x55e6f8bb97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.550+0000 7ff97dbf8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff9741afc90 con 0x55e6f8ba7f80 2024-05-10T11:36:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.550+0000 7ff97dbf8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff9741ad560 con 0x55e6f8ba8970 2024-05-10T11:36:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.550+0000 7ff97dbf8740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff9741ab3b0 con 0x55e6f8badae0 2024-05-10T11:36:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.551+0000 7ff973fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e6f8ba8970 0x55e6f8baa1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.551+0000 7ff97939d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6f8ba7f80 0x55e6f8ba9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.551+0000 7ff973fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e6f8ba8970 0x55e6f8baa1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:40644/0 (socket says 172.21.15.100:40644) 2024-05-10T11:36:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.551+0000 7ff97939d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6f8ba7f80 0x55e6f8ba9c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43258/0 (socket says 172.21.15.100:43258) 2024-05-10T11:36:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.551+0000 7ff973fff640 1 -- 172.21.15.100:0/354279631 learned_addr learned my addr 172.21.15.100:0/354279631 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:36:15.553 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.551+0000 7ff973fff640 1 -- 172.21.15.100:0/354279631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e6f8badae0 msgr2=0x55e6f8bb97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:15.553 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.551+0000 7ff973fff640 1 --2- 172.21.15.100:0/354279631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e6f8badae0 0x55e6f8bb97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:15.553 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.551+0000 7ff973fff640 1 -- 172.21.15.100:0/354279631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6f8ba7f80 msgr2=0x55e6f8ba9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:15.553 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.551+0000 7ff973fff640 1 --2- 172.21.15.100:0/354279631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6f8ba7f80 0x55e6f8ba9c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:15.553 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.551+0000 7ff973fff640 1 -- 172.21.15.100:0/354279631 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6f8ba3420 con 0x55e6f8ba8970 2024-05-10T11:36:15.553 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.551+0000 7ff97939d640 1 --2- 172.21.15.100:0/354279631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6f8ba7f80 0x55e6f8ba9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:36:15.553 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.551+0000 7ff973fff640 1 --2- 172.21.15.100:0/354279631 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e6f8ba8970 0x55e6f8baa1d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff96003c8b0 tx=0x7ff960001e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97e1ea1c821ca92 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:36:15.553 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.552+0000 7ff95e7fc640 1 -- 172.21.15.100:0/354279631 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff960035070 con 0x55e6f8ba8970 2024-05-10T11:36:15.553 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.552+0000 7ff95e7fc640 1 -- 172.21.15.100:0/354279631 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff9600057f0 con 0x55e6f8ba8970 2024-05-10T11:36:15.554 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.552+0000 7ff95e7fc640 1 -- 172.21.15.100:0/354279631 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96003c640 con 0x55e6f8ba8970 2024-05-10T11:36:15.554 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.552+0000 7ff97dbf8740 1 -- 172.21.15.100:0/354279631 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e6f8ba8970 msgr2=0x55e6f8baa1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:15.554 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.552+0000 7ff97dbf8740 1 --2- 172.21.15.100:0/354279631 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e6f8ba8970 0x55e6f8baa1d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff96003c8b0 tx=0x7ff960001e70 comp rx=0 tx=0).stop 2024-05-10T11:36:15.554 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.553+0000 7ff97dbf8740 1 -- 172.21.15.100:0/354279631 shutdown_connections 2024-05-10T11:36:15.554 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.553+0000 7ff97dbf8740 1 --2- 172.21.15.100:0/354279631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e6f8badae0 0x55e6f8bb97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:15.554 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.553+0000 7ff97dbf8740 1 --2- 172.21.15.100:0/354279631 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e6f8ba8970 0x55e6f8baa1d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:15.554 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.553+0000 7ff97dbf8740 1 --2- 172.21.15.100:0/354279631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6f8ba7f80 0x55e6f8ba9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:15.554 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.553+0000 7ff97dbf8740 1 -- 172.21.15.100:0/354279631 >> 172.21.15.100:0/354279631 conn(0x55e6f8ba35f0 msgr2=0x55e6f8bb2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:36:15.554 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.553+0000 7ff97dbf8740 1 -- 172.21.15.100:0/354279631 shutdown_connections 2024-05-10T11:36:15.554 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.553+0000 7ff97dbf8740 1 -- 172.21.15.100:0/354279631 wait complete. 2024-05-10T11:36:15.554 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.553+0000 7ff97dbf8740 1 -- start start 2024-05-10T11:36:15.554 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.553+0000 7ff97dbf8740 1 Processor -- start 2024-05-10T11:36:15.556 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.554+0000 7ff97dbf8740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e6f8ba7f80 0x55e6f8bb6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.554+0000 7ff97dbf8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e6f8ba8970 0x55e6f8bb7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.554+0000 7ff97dbf8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6f8badae0 0x55e6f8bb83e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.554+0000 7ff97dbf8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff9741aba20 con 0x55e6f8badae0 2024-05-10T11:36:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.554+0000 7ff97dbf8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff9741ace90 con 0x55e6f8ba8970 2024-05-10T11:36:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.554+0000 7ff97dbf8740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff9741ac0f0 con 0x55e6f8ba7f80 2024-05-10T11:36:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.554+0000 7ff97dbf8740 10 client.0 _ll_register_callbacks cb 0x7ff960009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:36:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.554+0000 7ff97939d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6f8badae0 0x55e6f8bb83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.554+0000 7ff973fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e6f8ba8970 0x55e6f8bb7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.554+0000 7ff97939d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e6f8ba7f80 0x55e6f8bb6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.554+0000 7ff97939d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e6f8ba7f80 0x55e6f8bb6ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:39984/0 (socket says 172.21.15.100:39984) 2024-05-10T11:36:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.554+0000 7ff973fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e6f8ba8970 0x55e6f8bb7420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:40658/0 (socket says 172.21.15.100:40658) 2024-05-10T11:36:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.554+0000 7ff97939d640 1 -- 172.21.15.100:0/371169499 learned_addr learned my addr 172.21.15.100:0/371169499 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:36:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.554+0000 7ff97939d640 1 -- 172.21.15.100:0/371169499 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e6f8ba7f80 msgr2=0x55e6f8bb6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.554+0000 7ff97939d640 1 --2- 172.21.15.100:0/371169499 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e6f8ba7f80 0x55e6f8bb6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:15.558 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.554+0000 7ff97939d640 1 -- 172.21.15.100:0/371169499 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e6f8ba8970 msgr2=0x55e6f8bb7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:15.558 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.554+0000 7ff97939d640 1 --2- 172.21.15.100:0/371169499 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e6f8ba8970 0x55e6f8bb7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:15.558 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.554+0000 7ff97939d640 1 -- 172.21.15.100:0/371169499 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6f8bb89d0 con 0x55e6f8badae0 2024-05-10T11:36:15.558 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.554+0000 7ff97939d640 1 --2- 172.21.15.100:0/371169499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6f8badae0 0x55e6f8bb83e0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7ff96c009ed0 tx=0x7ff96c004260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cccfbcfb81c9952 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:36:15.558 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.554+0000 7ff953fff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:36:15.558 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.555+0000 7ff953fff640 1 -- 172.21.15.100:0/371169499 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96c01e040 con 0x55e6f8badae0 2024-05-10T11:36:15.558 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.555+0000 7ff97dbf8740 1 -- 172.21.15.100:0/371169499 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9600057f0 con 0x55e6f8badae0 2024-05-10T11:36:15.558 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.555+0000 7ff953fff640 1 -- 172.21.15.100:0/371169499 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7ff96c003bd0 con 0x55e6f8badae0 2024-05-10T11:36:15.558 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.555+0000 7ff953fff640 1 -- 172.21.15.100:0/371169499 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7ff96c03c050 con 0x55e6f8badae0 2024-05-10T11:36:15.558 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.555+0000 7ff97dbf8740 10 client.18948 fetch_fsmap learned FSMap version 41 2024-05-10T11:36:15.558 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.555+0000 7ff97dbf8740 1 -- 172.21.15.100:0/371169499 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55e6f8c288a0 con 0x55e6f8badae0 2024-05-10T11:36:15.558 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.555+0000 7ff953fff640 1 -- 172.21.15.100:0/371169499 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff9600057f0 con 0x55e6f8badae0 2024-05-10T11:36:15.558 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.556+0000 7ff953fff640 1 -- 172.21.15.100:0/371169499 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7ff96c024040 con 0x55e6f8badae0 2024-05-10T11:36:15.558 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.556+0000 7ff97dbf8740 10 client.18948 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:36:15.559 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.556+0000 7ff97dbf8740 10 client.18948 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:36:15.559 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.556+0000 7ff97dbf8740 20 client.18948 populate_metadata read hostname 'smithi100' 2024-05-10T11:36:15.559 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.556+0000 7ff97dbf8740 1 --2- 172.21.15.100:0/371169499 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55e6f8bbdfd0 0x55e6f8c2b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:15.559 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.556+0000 7ff97dbf8740 4 client.18948 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:36:15.559 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.556+0000 7ff97dbf8740 1 -- 172.21.15.100:0/371169499 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e6f8b14050 con 0x55e6f8bbdfd0 2024-05-10T11:36:15.559 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.556+0000 7ff973fff640 1 --2- 172.21.15.100:0/371169499 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55e6f8bbdfd0 0x55e6f8c2b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:15.559 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.556+0000 7ff973fff640 1 --2- 172.21.15.100:0/371169499 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55e6f8bbdfd0 0x55e6f8c2b120 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b391cfac056dd11a server_cookie=40c5923fd2f40de4 in_seq=0 out_seq=0 2024-05-10T11:36:15.559 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.556+0000 7ff953fff640 10 client.18948 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:36:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.650+0000 7ff953fff640 1 -- 172.21.15.100:0/371169499 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e6f8b14050 con 0x55e6f8bbdfd0 2024-05-10T11:36:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.650+0000 7ff953fff640 10 client.18948 handle_command_reply: tid=0 2024-05-10T11:36:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.650+0000 7ff97dbf8740 1 client.18948 shutdown 2024-05-10T11:36:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.651+0000 7ff97dbf8740 1 -- 172.21.15.100:0/371169499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6f8badae0 msgr2=0x55e6f8bb83e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.651+0000 7ff97dbf8740 1 --2- 172.21.15.100:0/371169499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6f8badae0 0x55e6f8bb83e0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7ff96c009ed0 tx=0x7ff96c004260 comp rx=0 tx=0).stop 2024-05-10T11:36:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.651+0000 7ff97dbf8740 1 -- 172.21.15.100:0/371169499 shutdown_connections 2024-05-10T11:36:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.651+0000 7ff97dbf8740 1 --2- 172.21.15.100:0/371169499 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55e6f8bbdfd0 0x55e6f8c2b120 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.651+0000 7ff97dbf8740 1 --2- 172.21.15.100:0/371169499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6f8badae0 0x55e6f8bb83e0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.651+0000 7ff97dbf8740 1 --2- 172.21.15.100:0/371169499 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e6f8ba8970 0x55e6f8bb7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.651+0000 7ff953fff640 0 client.18948 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:36:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.651+0000 7ff97dbf8740 1 --2- 172.21.15.100:0/371169499 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e6f8ba7f80 0x55e6f8bb6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.651+0000 7ff97dbf8740 1 -- 172.21.15.100:0/371169499 >> 172.21.15.100:0/371169499 conn(0x55e6f8ba35f0 msgr2=0x55e6f8ba1ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:36:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.651+0000 7ff97dbf8740 1 -- 172.21.15.100:0/371169499 shutdown_connections 2024-05-10T11:36:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.651+0000 7ff97dbf8740 1 -- 172.21.15.100:0/371169499 wait complete. 2024-05-10T11:36:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.651+0000 7ff97dbf8740 20 client.18948 trim_cache size 0 max 16384 2024-05-10T11:36:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:36:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:36:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:36:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:36:15.654 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.652+0000 7ff979b9e640 1 -- 172.21.15.100:0/4178751353 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff9640b27e0 msgr2=0x7ff9640b4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:15.654 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.652+0000 7ff979b9e640 1 --2- 172.21.15.100:0/4178751353 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff9640b27e0 0x7ff9640b4ca0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ff9680e0200 tx=0x7ff9680e8550 comp rx=0 tx=0).stop 2024-05-10T11:36:15.654 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.652+0000 7ff979b9e640 1 -- 172.21.15.100:0/4178751353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9741019f0 msgr2=0x7ff97410c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:15.654 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.652+0000 7ff979b9e640 1 --2- 172.21.15.100:0/4178751353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9741019f0 0x7ff97410c2d0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7ff96800e9c0 tx=0x7ff96800ee80 comp rx=0 tx=0).stop 2024-05-10T11:36:15.654 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.652+0000 7ff979b9e640 1 -- 172.21.15.100:0/4178751353 shutdown_connections 2024-05-10T11:36:15.655 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.652+0000 7ff979b9e640 1 --2- 172.21.15.100:0/4178751353 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff9640b27e0 0x7ff9640b4ca0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:15.655 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.652+0000 7ff979b9e640 1 --2- 172.21.15.100:0/4178751353 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff974102cc0 0x7ff97410aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:15.655 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.652+0000 7ff979b9e640 1 --2- 172.21.15.100:0/4178751353 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9741023b0 0x7ff97410a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:15.655 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.652+0000 7ff979b9e640 1 --2- 172.21.15.100:0/4178751353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9741019f0 0x7ff97410c2d0 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:15.655 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.652+0000 7ff979b9e640 1 -- 172.21.15.100:0/4178751353 >> 172.21.15.100:0/4178751353 conn(0x7ff9740fc440 msgr2=0x7ff974105e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:36:15.655 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.652+0000 7ff979b9e640 1 -- 172.21.15.100:0/4178751353 shutdown_connections 2024-05-10T11:36:15.655 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:15.652+0000 7ff979b9e640 1 -- 172.21.15.100:0/4178751353 wait complete. 2024-05-10T11:36:15.924 INFO:tasks.workunit.client.0.smithi100.stdout: 10 42262 9.16 MB/sec execute 129 sec latency 124.882 ms 2024-05-10T11:36:16.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:36:16 smithi178 ceph-mon[33359]: pgmap v957: 129 pgs: 129 active+clean; 455 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 990 KiB/s rd, 5.0 MiB/s wr, 294 op/s 2024-05-10T11:36:16.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:36:16 smithi188 ceph-mon[34075]: pgmap v957: 129 pgs: 129 active+clean; 455 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 990 KiB/s rd, 5.0 MiB/s wr, 294 op/s 2024-05-10T11:36:16.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:36:16 smithi100 ceph-mon[30296]: pgmap v957: 129 pgs: 129 active+clean; 455 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 990 KiB/s rd, 5.0 MiB/s wr, 294 op/s 2024-05-10T11:36:16.925 INFO:tasks.workunit.client.0.smithi100.stdout: 10 42413 9.12 MB/sec execute 130 sec latency 104.745 ms 2024-05-10T11:36:17.926 INFO:tasks.workunit.client.0.smithi100.stdout: 10 42626 9.15 MB/sec execute 131 sec latency 58.629 ms 2024-05-10T11:36:18.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:36:18 smithi178 ceph-mon[33359]: pgmap v958: 129 pgs: 129 active+clean; 455 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 KiB/s rd, 4.9 MiB/s wr, 293 op/s 2024-05-10T11:36:18.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:36:18 smithi188 ceph-mon[34075]: pgmap v958: 129 pgs: 129 active+clean; 455 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 KiB/s rd, 4.9 MiB/s wr, 293 op/s 2024-05-10T11:36:18.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:36:18 smithi100 ceph-mon[30296]: pgmap v958: 129 pgs: 129 active+clean; 455 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 KiB/s rd, 4.9 MiB/s wr, 293 op/s 2024-05-10T11:36:18.926 INFO:tasks.workunit.client.0.smithi100.stdout: 10 42842 9.18 MB/sec execute 132 sec latency 87.920 ms 2024-05-10T11:36:19.926 INFO:tasks.workunit.client.0.smithi100.stdout: 10 43002 9.16 MB/sec execute 133 sec latency 60.054 ms 2024-05-10T11:36:20.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:36:20 smithi178 ceph-mon[33359]: pgmap v959: 129 pgs: 129 active+clean; 453 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 4.3 MiB/s wr, 268 op/s 2024-05-10T11:36:20.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:36:20 smithi188 ceph-mon[34075]: pgmap v959: 129 pgs: 129 active+clean; 453 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 4.3 MiB/s wr, 268 op/s 2024-05-10T11:36:20.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:36:20 smithi100 ceph-mon[30296]: pgmap v959: 129 pgs: 129 active+clean; 453 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 4.3 MiB/s wr, 268 op/s 2024-05-10T11:36:20.926 INFO:tasks.workunit.client.0.smithi100.stdout: 10 43171 9.19 MB/sec execute 134 sec latency 81.163 ms 2024-05-10T11:36:21.055 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T11:36:21.430 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:36:21.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:36:21 smithi178 ceph-mon[33359]: pgmap v960: 129 pgs: 129 active+clean; 459 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 4.7 MiB/s wr, 331 op/s 2024-05-10T11:36:21.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:36:21 smithi188 ceph-mon[34075]: pgmap v960: 129 pgs: 129 active+clean; 459 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 4.7 MiB/s wr, 331 op/s 2024-05-10T11:36:21.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:36:21 smithi100 ceph-mon[30296]: pgmap v960: 129 pgs: 129 active+clean; 459 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 4.7 MiB/s wr, 331 op/s 2024-05-10T11:36:21.927 INFO:tasks.workunit.client.0.smithi100.stdout: 10 43346 9.22 MB/sec execute 135 sec latency 149.465 ms 2024-05-10T11:36:22.927 INFO:tasks.workunit.client.0.smithi100.stdout: 10 43513 9.23 MB/sec execute 136 sec latency 198.607 ms 2024-05-10T11:36:23.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:36:23 smithi100 ceph-mon[30296]: pgmap v961: 129 pgs: 129 active+clean; 459 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 680 KiB/s rd, 4.0 MiB/s wr, 290 op/s 2024-05-10T11:36:23.927 INFO:tasks.workunit.client.0.smithi100.stdout: 10 43666 9.21 MB/sec execute 137 sec latency 67.663 ms 2024-05-10T11:36:24.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:36:23 smithi178 ceph-mon[33359]: pgmap v961: 129 pgs: 129 active+clean; 459 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 680 KiB/s rd, 4.0 MiB/s wr, 290 op/s 2024-05-10T11:36:24.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:36:23 smithi188 ceph-mon[34075]: pgmap v961: 129 pgs: 129 active+clean; 459 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 680 KiB/s rd, 4.0 MiB/s wr, 290 op/s 2024-05-10T11:36:24.448 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:36:24.448 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-05-10T11:36:24.449 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-05-10T11:36:24.449 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:36:24.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.207+0000 7f6defb8f700 1 -- 172.21.15.100:0/3091885578 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6de806bad0 msgr2=0x7f6de810fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:24.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.207+0000 7f6defb8f700 1 --2- 172.21.15.100:0/3091885578 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6de806bad0 0x7f6de810fa10 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f6dd8009990 tx=0x7f6dd8009ca0 comp rx=0 tx=0).stop 2024-05-10T11:36:24.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.207+0000 7f6defb8f700 1 -- 172.21.15.100:0/3091885578 shutdown_connections 2024-05-10T11:36:24.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.207+0000 7f6defb8f700 1 --2- 172.21.15.100:0/3091885578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6de810ff50 0x7f6de8112340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:24.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.207+0000 7f6defb8f700 1 --2- 172.21.15.100:0/3091885578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6de806b1b0 0x7f6de806b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:24.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.207+0000 7f6defb8f700 1 --2- 172.21.15.100:0/3091885578 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6de806bad0 0x7f6de810fa10 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:24.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.207+0000 7f6defb8f700 1 -- 172.21.15.100:0/3091885578 >> 172.21.15.100:0/3091885578 conn(0x7f6de8079350 msgr2=0x7f6de8079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:36:24.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.207+0000 7f6defb8f700 1 -- 172.21.15.100:0/3091885578 shutdown_connections 2024-05-10T11:36:24.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.207+0000 7f6defb8f700 1 -- 172.21.15.100:0/3091885578 wait complete. 2024-05-10T11:36:24.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.208+0000 7f6defb8f700 1 Processor -- start 2024-05-10T11:36:24.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.208+0000 7f6defb8f700 1 -- start start 2024-05-10T11:36:24.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.208+0000 7f6defb8f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6de806b1b0 0x7f6de8104680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:24.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.208+0000 7f6defb8f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6de806bad0 0x7f6de8104bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:24.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.209+0000 7f6defb8f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6de810ff50 0x7f6de8105100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:24.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.209+0000 7f6defb8f700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6de8115710 con 0x7f6de810ff50 2024-05-10T11:36:24.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.209+0000 7f6defb8f700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6de8115590 con 0x7f6de806b1b0 2024-05-10T11:36:24.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.209+0000 7f6defb8f700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6de8115890 con 0x7f6de806bad0 2024-05-10T11:36:24.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.209+0000 7f6ded12a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6de806bad0 0x7f6de8104bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:24.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.209+0000 7f6ded92b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6de806b1b0 0x7f6de8104680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:24.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.209+0000 7f6ded12a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6de806bad0 0x7f6de8104bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:47864/0 (socket says 172.21.15.100:47864) 2024-05-10T11:36:24.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.209+0000 7f6ded12a700 1 -- 172.21.15.100:0/2990115877 learned_addr learned my addr 172.21.15.100:0/2990115877 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:36:24.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.209+0000 7f6ded92b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6de806b1b0 0x7f6de8104680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:58418/0 (socket says 172.21.15.100:58418) 2024-05-10T11:36:24.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.209+0000 7f6dee12c700 1 --2- 172.21.15.100:0/2990115877 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6de810ff50 0x7f6de8105100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:24.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.209+0000 7f6ded12a700 1 -- 172.21.15.100:0/2990115877 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6de806b1b0 msgr2=0x7f6de8104680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:24.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.209+0000 7f6ded12a700 1 --2- 172.21.15.100:0/2990115877 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6de806b1b0 0x7f6de8104680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:24.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.209+0000 7f6ded12a700 1 -- 172.21.15.100:0/2990115877 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6de810ff50 msgr2=0x7f6de8105100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:24.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.209+0000 7f6ded12a700 1 --2- 172.21.15.100:0/2990115877 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6de810ff50 0x7f6de8105100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:24.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.209+0000 7f6ded12a700 1 -- 172.21.15.100:0/2990115877 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6de81ac370 con 0x7f6de806bad0 2024-05-10T11:36:24.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.209+0000 7f6ded92b700 1 --2- 172.21.15.100:0/2990115877 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6de806b1b0 0x7f6de8104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:36:24.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.209+0000 7f6ded12a700 1 --2- 172.21.15.100:0/2990115877 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6de806bad0 0x7f6de8104bc0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6dd80095b0 tx=0x7f6dd800bfa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:36:24.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.209+0000 7f6ddeffd700 1 -- 172.21.15.100:0/2990115877 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd8026070 con 0x7f6de806bad0 2024-05-10T11:36:24.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.210+0000 7f6ddeffd700 1 -- 172.21.15.100:0/2990115877 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6dd8021410 con 0x7f6de806bad0 2024-05-10T11:36:24.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.210+0000 7f6defb8f700 1 -- 172.21.15.100:0/2990115877 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6de81ac600 con 0x7f6de806bad0 2024-05-10T11:36:24.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.210+0000 7f6ddeffd700 1 -- 172.21.15.100:0/2990115877 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd8017650 con 0x7f6de806bad0 2024-05-10T11:36:24.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.210+0000 7f6defb8f700 1 -- 172.21.15.100:0/2990115877 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6de81aca90 con 0x7f6de806bad0 2024-05-10T11:36:24.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.213+0000 7f6ddeffd700 1 -- 172.21.15.100:0/2990115877 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f6dd8029070 con 0x7f6de806bad0 2024-05-10T11:36:24.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.214+0000 7f6ddeffd700 1 --2- 172.21.15.100:0/2990115877 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f6dd40b2900 0x7f6dd40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:24.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.214+0000 7f6ddeffd700 1 -- 172.21.15.100:0/2990115877 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f6dd80e3430 con 0x7f6de806bad0 2024-05-10T11:36:24.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.214+0000 7f6ded92b700 1 --2- 172.21.15.100:0/2990115877 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f6dd40b2900 0x7f6dd40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:24.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.215+0000 7f6dfa422b80 1 Processor -- start 2024-05-10T11:36:24.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.215+0000 7f6ded92b700 1 --2- 172.21.15.100:0/2990115877 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f6dd40b2900 0x7f6dd40b4dc0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f6de4007fd0 tx=0x7f6de40073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:36:24.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.215+0000 7f6dfa422b80 1 -- start start 2024-05-10T11:36:24.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.215+0000 7f6dfa422b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a8f5d61b0 0x556a8f5d6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:24.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.215+0000 7f6dfa422b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a8f5d6ad0 0x556a8f5d0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:24.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.215+0000 7f6dfa422b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a8f5d0d50 0x556a8f5d1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:24.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.215+0000 7f6dfa422b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x556a8f522940 con 0x556a8f5d6ad0 2024-05-10T11:36:24.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.215+0000 7f6dfa422b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556a8f487da0 con 0x556a8f5d61b0 2024-05-10T11:36:24.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.215+0000 7f6dfa422b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556a8f51e900 con 0x556a8f5d0d50 2024-05-10T11:36:24.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.215+0000 7f6dee12c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a8f5d0d50 0x556a8f5d1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:24.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.215+0000 7f6ded12a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a8f5d6ad0 0x556a8f5d0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:24.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.215+0000 7f6dee12c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a8f5d0d50 0x556a8f5d1190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:47880/0 (socket says 172.21.15.100:47880) 2024-05-10T11:36:24.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.215+0000 7f6ded12a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a8f5d6ad0 0x556a8f5d0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38306/0 (socket says 172.21.15.100:38306) 2024-05-10T11:36:24.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.215+0000 7f6dee12c700 1 -- 172.21.15.100:0/335449417 learned_addr learned my addr 172.21.15.100:0/335449417 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:36:24.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.215+0000 7f6ded92b700 1 --2- 172.21.15.100:0/335449417 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a8f5d61b0 0x556a8f5d6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:24.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.216+0000 7f6ded12a700 1 -- 172.21.15.100:0/335449417 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a8f5d0d50 msgr2=0x556a8f5d1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:24.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.216+0000 7f6ded12a700 1 --2- 172.21.15.100:0/335449417 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a8f5d0d50 0x556a8f5d1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:24.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.216+0000 7f6ded12a700 1 -- 172.21.15.100:0/335449417 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a8f5d61b0 msgr2=0x556a8f5d6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:24.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.216+0000 7f6ded12a700 1 --2- 172.21.15.100:0/335449417 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a8f5d61b0 0x556a8f5d6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:24.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.216+0000 7f6ded12a700 1 -- 172.21.15.100:0/335449417 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a8f46f8d0 con 0x556a8f5d6ad0 2024-05-10T11:36:24.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.216+0000 7f6ded12a700 1 --2- 172.21.15.100:0/335449417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a8f5d6ad0 0x556a8f5d0810 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f6dd80aa6b0 tx=0x7f6dd80aa0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=987194a62fe8938b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:36:24.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.216+0000 7f6dd3fff700 1 -- 172.21.15.100:0/335449417 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd80aae20 con 0x556a8f5d6ad0 2024-05-10T11:36:24.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.216+0000 7f6dd3fff700 1 -- 172.21.15.100:0/335449417 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6dd80aa720 con 0x556a8f5d6ad0 2024-05-10T11:36:24.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.216+0000 7f6dd3fff700 1 -- 172.21.15.100:0/335449417 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd80920d0 con 0x556a8f5d6ad0 2024-05-10T11:36:24.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.217+0000 7f6dfa422b80 1 -- 172.21.15.100:0/335449417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a8f5d6ad0 msgr2=0x556a8f5d0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:24.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.217+0000 7f6dfa422b80 1 --2- 172.21.15.100:0/335449417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a8f5d6ad0 0x556a8f5d0810 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f6dd80aa6b0 tx=0x7f6dd80aa0f0 comp rx=0 tx=0).stop 2024-05-10T11:36:24.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.217+0000 7f6dfa422b80 1 -- 172.21.15.100:0/335449417 shutdown_connections 2024-05-10T11:36:24.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.217+0000 7f6dfa422b80 1 --2- 172.21.15.100:0/335449417 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a8f5d0d50 0x556a8f5d1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:24.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.217+0000 7f6dfa422b80 1 --2- 172.21.15.100:0/335449417 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a8f5d61b0 0x556a8f5d6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:24.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.217+0000 7f6dfa422b80 1 --2- 172.21.15.100:0/335449417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a8f5d6ad0 0x556a8f5d0810 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:24.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.217+0000 7f6dfa422b80 1 -- 172.21.15.100:0/335449417 >> 172.21.15.100:0/335449417 conn(0x556a8f4fcb70 msgr2=0x556a8f5cb2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:36:24.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.217+0000 7f6dfa422b80 1 -- 172.21.15.100:0/335449417 shutdown_connections 2024-05-10T11:36:24.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.217+0000 7f6dfa422b80 1 -- 172.21.15.100:0/335449417 wait complete. 2024-05-10T11:36:24.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.218+0000 7f6dfa422b80 1 -- start start 2024-05-10T11:36:24.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.218+0000 7f6dfa422b80 1 Processor -- start 2024-05-10T11:36:24.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.218+0000 7f6dfa422b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a8f5d0d50 0x556a8f5dc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:24.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.218+0000 7f6dfa422b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a8f5d61b0 0x556a8f5d9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:24.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.218+0000 7f6dfa422b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a8f5d6ad0 0x556a8f5da010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:24.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.218+0000 7f6dfa422b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x556a8f5dd210 con 0x556a8f5d0d50 2024-05-10T11:36:24.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.218+0000 7f6dfa422b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556a8f4dbed0 con 0x556a8f5d61b0 2024-05-10T11:36:24.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.218+0000 7f6dfa422b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556a8f5dd390 con 0x556a8f5d6ad0 2024-05-10T11:36:24.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.218+0000 7f6dfa422b80 10 client.0 _ll_register_callbacks cb 0x556a8f4b65b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:36:24.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.218+0000 7f6ded92b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a8f5d0d50 0x556a8f5dc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:24.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.218+0000 7f6dee12c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a8f5d6ad0 0x556a8f5da010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:24.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.218+0000 7f6dee12c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a8f5d6ad0 0x556a8f5da010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:47894/0 (socket says 172.21.15.100:47894) 2024-05-10T11:36:24.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.218+0000 7f6dee12c700 1 -- 172.21.15.100:0/502396395 learned_addr learned my addr 172.21.15.100:0/502396395 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:36:24.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.219+0000 7f6ded12a700 1 --2- 172.21.15.100:0/502396395 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a8f5d61b0 0x556a8f5d9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:24.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.219+0000 7f6dee12c700 1 -- 172.21.15.100:0/502396395 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a8f5d61b0 msgr2=0x556a8f5d9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:24.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.219+0000 7f6dee12c700 1 --2- 172.21.15.100:0/502396395 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a8f5d61b0 0x556a8f5d9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:24.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.219+0000 7f6dee12c700 1 -- 172.21.15.100:0/502396395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a8f5d0d50 msgr2=0x556a8f5dc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:24.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.219+0000 7f6dee12c700 1 --2- 172.21.15.100:0/502396395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a8f5d0d50 0x556a8f5dc730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:24.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.219+0000 7f6dee12c700 1 -- 172.21.15.100:0/502396395 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a8f4d9760 con 0x556a8f5d6ad0 2024-05-10T11:36:24.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.219+0000 7f6dee12c700 1 --2- 172.21.15.100:0/502396395 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a8f5d6ad0 0x556a8f5da010 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6de0012e80 tx=0x7f6de0014600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=601b79ef575836e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:36:24.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.219+0000 7f6dd17fa700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:36:24.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.219+0000 7f6dd17fa700 1 -- 172.21.15.100:0/502396395 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de0014b90 con 0x556a8f5d6ad0 2024-05-10T11:36:24.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.219+0000 7f6dd17fa700 1 -- 172.21.15.100:0/502396395 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f6de0014d30 con 0x556a8f5d6ad0 2024-05-10T11:36:24.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.219+0000 7f6dfa422b80 1 -- 172.21.15.100:0/502396395 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6dd80aa720 con 0x556a8f5d6ad0 2024-05-10T11:36:24.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.220+0000 7f6dd17fa700 1 -- 172.21.15.100:0/502396395 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f6de0028050 con 0x556a8f5d6ad0 2024-05-10T11:36:24.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.220+0000 7f6dfa422b80 10 client.28369 fetch_fsmap learned FSMap version 41 2024-05-10T11:36:24.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.220+0000 7f6dfa422b80 1 -- 172.21.15.100:0/502396395 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x556a8f5e7a10 con 0x556a8f5d6ad0 2024-05-10T11:36:24.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.220+0000 7f6dd17fa700 1 -- 172.21.15.100:0/502396395 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6de0007990 con 0x556a8f5d6ad0 2024-05-10T11:36:24.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.220+0000 7f6dd17fa700 1 -- 172.21.15.100:0/502396395 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f6de0016570 con 0x556a8f5d6ad0 2024-05-10T11:36:24.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.220+0000 7f6dfa422b80 10 client.28369 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:36:24.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.220+0000 7f6dfa422b80 10 client.28369 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:36:24.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.220+0000 7f6dfa422b80 20 client.28369 populate_metadata read hostname 'smithi100' 2024-05-10T11:36:24.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.220+0000 7f6dfa422b80 1 --2- 172.21.15.100:0/502396395 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556a8f5ea050 0x556a8f5ec440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:24.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.221+0000 7f6dfa422b80 4 client.28369 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:36:24.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.221+0000 7f6dfa422b80 1 -- 172.21.15.100:0/502396395 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6dd80aa720 con 0x556a8f5ea050 2024-05-10T11:36:24.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.221+0000 7f6ded92b700 1 --2- 172.21.15.100:0/502396395 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556a8f5ea050 0x556a8f5ec440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:24.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.221+0000 7f6dd17fa700 10 client.28369 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:36:24.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.221+0000 7f6ded92b700 1 --2- 172.21.15.100:0/502396395 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556a8f5ea050 0x556a8f5ec440 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89d87bbb9fe70fa3 server_cookie=b3809b1fb290fc91 in_seq=0 out_seq=0 2024-05-10T11:36:24.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.222+0000 7f6dd17fa700 1 -- 172.21.15.100:0/502396395 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f6dd80aa720 con 0x556a8f5ea050 2024-05-10T11:36:24.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.222+0000 7f6dd17fa700 10 client.28369 handle_command_reply: tid=0 2024-05-10T11:36:24.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.222+0000 7f6dfa422b80 1 client.28369 shutdown 2024-05-10T11:36:24.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.222+0000 7f6dfa422b80 1 -- 172.21.15.100:0/502396395 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a8f5d6ad0 msgr2=0x556a8f5da010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:24.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.222+0000 7f6dfa422b80 1 --2- 172.21.15.100:0/502396395 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a8f5d6ad0 0x556a8f5da010 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6de0012e80 tx=0x7f6de0014600 comp rx=0 tx=0).stop 2024-05-10T11:36:24.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.222+0000 7f6dfa422b80 1 -- 172.21.15.100:0/502396395 shutdown_connections 2024-05-10T11:36:24.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.222+0000 7f6dfa422b80 1 --2- 172.21.15.100:0/502396395 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556a8f5ea050 0x556a8f5ec440 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:24.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.222+0000 7f6dfa422b80 1 --2- 172.21.15.100:0/502396395 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a8f5d6ad0 0x556a8f5da010 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:24.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.222+0000 7f6dfa422b80 1 --2- 172.21.15.100:0/502396395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a8f5d0d50 0x556a8f5dc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:24.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.222+0000 7f6dfa422b80 1 --2- 172.21.15.100:0/502396395 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a8f5d61b0 0x556a8f5d9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:24.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.222+0000 7f6dfa422b80 1 -- 172.21.15.100:0/502396395 >> 172.21.15.100:0/502396395 conn(0x556a8f4fcb70 msgr2=0x556a8f5c5b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:36:24.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.222+0000 7f6dd17fa700 0 client.28369 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:36:24.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.223+0000 7f6dfa422b80 1 -- 172.21.15.100:0/502396395 shutdown_connections 2024-05-10T11:36:24.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.223+0000 7f6dfa422b80 1 -- 172.21.15.100:0/502396395 wait complete. 2024-05-10T11:36:24.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.223+0000 7f6dfa422b80 20 client.28369 trim_cache size 0 max 16384 2024-05-10T11:36:24.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.254+0000 7f6dfa422b80 1 Processor -- start 2024-05-10T11:36:24.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.254+0000 7f6dfa422b80 1 -- start start 2024-05-10T11:36:24.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.254+0000 7f6dfa422b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a8f5d0d50 0x556a8f5d0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:24.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.254+0000 7f6dfa422b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a8f5d61b0 0x556a8f5e6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:24.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.254+0000 7f6dfa422b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a8f5d6ad0 0x556a8f5d4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:24.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.254+0000 7f6dfa422b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6de81b8010 con 0x556a8f5d6ad0 2024-05-10T11:36:24.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.254+0000 7f6dfa422b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6de81b58a0 con 0x556a8f5d61b0 2024-05-10T11:36:24.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.254+0000 7f6dfa422b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6de81b36f0 con 0x556a8f5d0d50 2024-05-10T11:36:24.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.254+0000 7f6dee12c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a8f5d6ad0 0x556a8f5d4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:24.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.254+0000 7f6dee12c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a8f5d6ad0 0x556a8f5d4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38326/0 (socket says 172.21.15.100:38326) 2024-05-10T11:36:24.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.254+0000 7f6dee12c700 1 -- 172.21.15.100:0/4276158827 learned_addr learned my addr 172.21.15.100:0/4276158827 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:36:24.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.254+0000 7f6ded12a700 1 --2- 172.21.15.100:0/4276158827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a8f5d61b0 0x556a8f5e6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:24.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.254+0000 7f6ded92b700 1 --2- 172.21.15.100:0/4276158827 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a8f5d0d50 0x556a8f5d0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:24.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.254+0000 7f6dee12c700 1 -- 172.21.15.100:0/4276158827 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a8f5d0d50 msgr2=0x556a8f5d0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:24.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.254+0000 7f6dee12c700 1 --2- 172.21.15.100:0/4276158827 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a8f5d0d50 0x556a8f5d0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:24.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.254+0000 7f6dee12c700 1 -- 172.21.15.100:0/4276158827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a8f5d61b0 msgr2=0x556a8f5e6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:24.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.255+0000 7f6dee12c700 1 --2- 172.21.15.100:0/4276158827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a8f5d61b0 0x556a8f5e6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:24.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.255+0000 7f6dee12c700 1 -- 172.21.15.100:0/4276158827 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a8f5d8080 con 0x556a8f5d6ad0 2024-05-10T11:36:24.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.255+0000 7f6ded92b700 1 --2- 172.21.15.100:0/4276158827 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a8f5d0d50 0x556a8f5d0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:36:24.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.255+0000 7f6ded12a700 1 --2- 172.21.15.100:0/4276158827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a8f5d61b0 0x556a8f5e6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:36:24.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.255+0000 7f6dee12c700 1 --2- 172.21.15.100:0/4276158827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a8f5d6ad0 0x556a8f5d4150 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f6de0016fb0 tx=0x7f6de0009dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=960313e2f0134e09 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:36:24.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.255+0000 7f6dd37fe700 1 -- 172.21.15.100:0/4276158827 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de0014d90 con 0x556a8f5d6ad0 2024-05-10T11:36:24.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.255+0000 7f6dd37fe700 1 -- 172.21.15.100:0/4276158827 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6de0003a00 con 0x556a8f5d6ad0 2024-05-10T11:36:24.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.255+0000 7f6dd37fe700 1 -- 172.21.15.100:0/4276158827 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de0007760 con 0x556a8f5d6ad0 2024-05-10T11:36:24.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.256+0000 7f6dfa422b80 1 -- 172.21.15.100:0/4276158827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a8f5d6ad0 msgr2=0x556a8f5d4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:24.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.256+0000 7f6dfa422b80 1 --2- 172.21.15.100:0/4276158827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a8f5d6ad0 0x556a8f5d4150 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f6de0016fb0 tx=0x7f6de0009dd0 comp rx=0 tx=0).stop 2024-05-10T11:36:24.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.256+0000 7f6dfa422b80 1 -- 172.21.15.100:0/4276158827 shutdown_connections 2024-05-10T11:36:24.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.256+0000 7f6dfa422b80 1 --2- 172.21.15.100:0/4276158827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a8f5d6ad0 0x556a8f5d4150 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:24.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.256+0000 7f6dfa422b80 1 --2- 172.21.15.100:0/4276158827 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a8f5d0d50 0x556a8f5d0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:24.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.256+0000 7f6dfa422b80 1 --2- 172.21.15.100:0/4276158827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a8f5d61b0 0x556a8f5e6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:24.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.256+0000 7f6dfa422b80 1 -- 172.21.15.100:0/4276158827 >> 172.21.15.100:0/4276158827 conn(0x556a8f4fcb70 msgr2=0x556a8f5d3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:36:24.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.256+0000 7f6dfa422b80 1 -- 172.21.15.100:0/4276158827 shutdown_connections 2024-05-10T11:36:24.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.257+0000 7f6dfa422b80 1 -- 172.21.15.100:0/4276158827 wait complete. 2024-05-10T11:36:24.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.257+0000 7f6dfa422b80 1 -- start start 2024-05-10T11:36:24.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.257+0000 7f6dfa422b80 1 Processor -- start 2024-05-10T11:36:24.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.257+0000 7f6dfa422b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a8f5d0d50 0x556a8f5e2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:24.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.258+0000 7f6dfa422b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a8f5d61b0 0x556a8f5f1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:24.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.258+0000 7f6dfa422b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a8f5d6ad0 0x556a8f5f19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:24.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.258+0000 7f6ded92b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a8f5d0d50 0x556a8f5e2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:24.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.258+0000 7f6ded92b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a8f5d0d50 0x556a8f5e2ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:47916/0 (socket says 172.21.15.100:47916) 2024-05-10T11:36:24.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.258+0000 7f6ded92b700 1 -- 172.21.15.100:0/2025577660 learned_addr learned my addr 172.21.15.100:0/2025577660 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:36:24.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.258+0000 7f6dee12c700 1 --2- 172.21.15.100:0/2025577660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a8f5d6ad0 0x556a8f5f19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:24.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.258+0000 7f6dfa422b80 1 -- 172.21.15.100:0/2025577660 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6de81b3d60 con 0x556a8f5d6ad0 2024-05-10T11:36:24.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.258+0000 7f6dfa422b80 1 -- 172.21.15.100:0/2025577660 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6de81b51d0 con 0x556a8f5d61b0 2024-05-10T11:36:24.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.258+0000 7f6dfa422b80 1 -- 172.21.15.100:0/2025577660 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6de81b4430 con 0x556a8f5d0d50 2024-05-10T11:36:24.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.258+0000 7f6dfa422b80 10 client.0 _ll_register_callbacks cb 0x556a8f4b65b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:36:24.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.258+0000 7f6ded12a700 1 --2- 172.21.15.100:0/2025577660 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a8f5d61b0 0x556a8f5f1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:24.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.258+0000 7f6ded92b700 1 -- 172.21.15.100:0/2025577660 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a8f5d61b0 msgr2=0x556a8f5f1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:24.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.258+0000 7f6ded92b700 1 --2- 172.21.15.100:0/2025577660 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a8f5d61b0 0x556a8f5f1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:24.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.258+0000 7f6ded92b700 1 -- 172.21.15.100:0/2025577660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a8f5d6ad0 msgr2=0x556a8f5f19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:24.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.258+0000 7f6ded92b700 1 --2- 172.21.15.100:0/2025577660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a8f5d6ad0 0x556a8f5f19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:24.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.258+0000 7f6ded92b700 1 -- 172.21.15.100:0/2025577660 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a8f653810 con 0x556a8f5d0d50 2024-05-10T11:36:24.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.259+0000 7f6ded92b700 1 --2- 172.21.15.100:0/2025577660 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a8f5d0d50 0x556a8f5e2ba0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6de4010af0 tx=0x7f6de4013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c84a164b4ed6309d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:36:24.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.259+0000 7f6dd17fa700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:36:24.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.259+0000 7f6dd17fa700 1 -- 172.21.15.100:0/2025577660 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de4028130 con 0x556a8f5d0d50 2024-05-10T11:36:24.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.259+0000 7f6dd17fa700 1 -- 172.21.15.100:0/2025577660 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f6de40282d0 con 0x556a8f5d0d50 2024-05-10T11:36:24.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.259+0000 7f6dfa422b80 1 -- 172.21.15.100:0/2025577660 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6de0003a00 con 0x556a8f5d0d50 2024-05-10T11:36:24.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.259+0000 7f6dd17fa700 1 -- 172.21.15.100:0/2025577660 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f6de4039050 con 0x556a8f5d0d50 2024-05-10T11:36:24.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.259+0000 7f6dfa422b80 10 client.28375 fetch_fsmap learned FSMap version 41 2024-05-10T11:36:24.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.259+0000 7f6dfa422b80 1 -- 172.21.15.100:0/2025577660 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x556a8f6549a0 con 0x556a8f5d0d50 2024-05-10T11:36:24.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.260+0000 7f6dd17fa700 1 -- 172.21.15.100:0/2025577660 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6de4015070 con 0x556a8f5d0d50 2024-05-10T11:36:24.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.260+0000 7f6dd17fa700 1 -- 172.21.15.100:0/2025577660 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f6de4028ce0 con 0x556a8f5d0d50 2024-05-10T11:36:24.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.260+0000 7f6dfa422b80 10 client.28375 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:36:24.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.260+0000 7f6dfa422b80 10 client.28375 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:36:24.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.260+0000 7f6dfa422b80 20 client.28375 populate_metadata read hostname 'smithi100' 2024-05-10T11:36:24.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.260+0000 7f6dfa422b80 1 --2- 172.21.15.100:0/2025577660 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556a8f655110 0x556a8f657530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:24.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.260+0000 7f6dfa422b80 4 client.28375 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T11:36:24.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.260+0000 7f6dfa422b80 1 -- 172.21.15.100:0/2025577660 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f6de0003a00 con 0x556a8f655110 2024-05-10T11:36:24.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.260+0000 7f6ded12a700 1 --2- 172.21.15.100:0/2025577660 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556a8f655110 0x556a8f657530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:24.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.261+0000 7f6dd17fa700 10 client.28375 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:36:24.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.261+0000 7f6ded12a700 1 --2- 172.21.15.100:0/2025577660 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556a8f655110 0x556a8f657530 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9529cb47811f4e4 server_cookie=fe9dba1b4fd7b5c2 in_seq=0 out_seq=0 2024-05-10T11:36:24.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.261+0000 7f6dd17fa700 1 -- 172.21.15.100:0/2025577660 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f6de0003a00 con 0x556a8f655110 2024-05-10T11:36:24.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.261+0000 7f6dd17fa700 10 client.28375 handle_command_reply: tid=0 2024-05-10T11:36:24.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.261+0000 7f6dfa422b80 1 client.28375 shutdown 2024-05-10T11:36:24.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.261+0000 7f6dfa422b80 1 -- 172.21.15.100:0/2025577660 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a8f5d0d50 msgr2=0x556a8f5e2ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:24.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.261+0000 7f6dfa422b80 1 --2- 172.21.15.100:0/2025577660 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a8f5d0d50 0x556a8f5e2ba0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6de4010af0 tx=0x7f6de4013000 comp rx=0 tx=0).stop 2024-05-10T11:36:24.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.261+0000 7f6dfa422b80 1 -- 172.21.15.100:0/2025577660 shutdown_connections 2024-05-10T11:36:24.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.261+0000 7f6dfa422b80 1 --2- 172.21.15.100:0/2025577660 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556a8f655110 0x556a8f657530 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:24.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.261+0000 7f6dfa422b80 1 --2- 172.21.15.100:0/2025577660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a8f5d6ad0 0x556a8f5f19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:24.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.261+0000 7f6dd17fa700 0 client.28375 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:36:24.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.262+0000 7f6dfa422b80 1 --2- 172.21.15.100:0/2025577660 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a8f5d0d50 0x556a8f5e2ba0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:24.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.262+0000 7f6dfa422b80 1 --2- 172.21.15.100:0/2025577660 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a8f5d61b0 0x556a8f5f1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:24.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.262+0000 7f6dfa422b80 1 -- 172.21.15.100:0/2025577660 >> 172.21.15.100:0/2025577660 conn(0x556a8f4fcb70 msgr2=0x556a8f5d6eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:36:24.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.262+0000 7f6dfa422b80 1 -- 172.21.15.100:0/2025577660 shutdown_connections 2024-05-10T11:36:24.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.262+0000 7f6dfa422b80 1 -- 172.21.15.100:0/2025577660 wait complete. 2024-05-10T11:36:24.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.262+0000 7f6dfa422b80 20 client.28375 trim_cache size 0 max 16384 2024-05-10T11:36:24.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.263+0000 7f6defb8f700 1 -- 172.21.15.100:0/2990115877 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f6dd40b2900 msgr2=0x7f6dd40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:24.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.263+0000 7f6defb8f700 1 --2- 172.21.15.100:0/2990115877 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f6dd40b2900 0x7f6dd40b4dc0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f6de4007fd0 tx=0x7f6de40073d0 comp rx=0 tx=0).stop 2024-05-10T11:36:24.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.263+0000 7f6defb8f700 1 -- 172.21.15.100:0/2990115877 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6de806bad0 msgr2=0x7f6de8104bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:24.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.263+0000 7f6defb8f700 1 --2- 172.21.15.100:0/2990115877 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6de806bad0 0x7f6de8104bc0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6dd80095b0 tx=0x7f6dd800bfa0 comp rx=0 tx=0).stop 2024-05-10T11:36:24.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.263+0000 7f6defb8f700 1 -- 172.21.15.100:0/2990115877 shutdown_connections 2024-05-10T11:36:24.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.263+0000 7f6defb8f700 1 --2- 172.21.15.100:0/2990115877 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f6dd40b2900 0x7f6dd40b4dc0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:24.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.263+0000 7f6defb8f700 1 --2- 172.21.15.100:0/2990115877 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6de810ff50 0x7f6de8105100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:24.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.263+0000 7f6defb8f700 1 --2- 172.21.15.100:0/2990115877 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6de806b1b0 0x7f6de8104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:24.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.263+0000 7f6defb8f700 1 --2- 172.21.15.100:0/2990115877 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6de806bad0 0x7f6de8104bc0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:24.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.263+0000 7f6defb8f700 1 -- 172.21.15.100:0/2990115877 >> 172.21.15.100:0/2990115877 conn(0x7f6de8079350 msgr2=0x7f6de8111460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:36:24.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.263+0000 7f6defb8f700 1 -- 172.21.15.100:0/2990115877 shutdown_connections 2024-05-10T11:36:24.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:24.263+0000 7f6defb8f700 1 -- 172.21.15.100:0/2990115877 wait complete. 2024-05-10T11:36:24.927 INFO:tasks.workunit.client.0.smithi100.stdout: 10 43802 9.20 MB/sec execute 138 sec latency 79.231 ms 2024-05-10T11:36:25.912 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:36:25 smithi100 ceph-mon[30296]: pgmap v962: 129 pgs: 129 active+clean; 462 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 4.5 MiB/s wr, 307 op/s 2024-05-10T11:36:25.926 INFO:tasks.workunit.client.0.smithi100.stdout: 10 43949 9.18 MB/sec execute 139 sec latency 164.911 ms 2024-05-10T11:36:26.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:36:25 smithi178 ceph-mon[33359]: pgmap v962: 129 pgs: 129 active+clean; 462 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 4.5 MiB/s wr, 307 op/s 2024-05-10T11:36:26.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:36:25 smithi188 ceph-mon[34075]: pgmap v962: 129 pgs: 129 active+clean; 462 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 4.5 MiB/s wr, 307 op/s 2024-05-10T11:36:26.927 INFO:tasks.workunit.client.0.smithi100.stdout: 10 44072 9.16 MB/sec execute 140 sec latency 129.501 ms 2024-05-10T11:36:27.927 INFO:tasks.workunit.client.0.smithi100.stdout: 10 44211 9.12 MB/sec execute 141 sec latency 115.609 ms 2024-05-10T11:36:28.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:36:27 smithi178 ceph-mon[33359]: pgmap v963: 129 pgs: 129 active+clean; 464 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 4.4 MiB/s wr, 303 op/s 2024-05-10T11:36:28.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:36:27 smithi188 ceph-mon[34075]: pgmap v963: 129 pgs: 129 active+clean; 464 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 4.4 MiB/s wr, 303 op/s 2024-05-10T11:36:28.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:36:27 smithi100 ceph-mon[30296]: pgmap v963: 129 pgs: 129 active+clean; 464 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 4.4 MiB/s wr, 303 op/s 2024-05-10T11:36:28.158 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T11:36:28.159 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T11:36:28.528 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:36:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.794+0000 7f4f3e99e640 1 -- 172.21.15.100:0/3166748343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f38101950 msgr2=0x7f4f38101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.794+0000 7f4f3e99e640 1 --2- 172.21.15.100:0/3166748343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f38101950 0x7f4f38101d70 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f4f28009900 tx=0x7f4f2802a410 comp rx=0 tx=0).stop 2024-05-10T11:36:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.794+0000 7f4f3e99e640 1 -- 172.21.15.100:0/3166748343 shutdown_connections 2024-05-10T11:36:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.794+0000 7f4f3e99e640 1 --2- 172.21.15.100:0/3166748343 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f38103500 0x7f4f38109d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.794+0000 7f4f3e99e640 1 --2- 172.21.15.100:0/3166748343 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f38102b40 0x7f4f38102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.794+0000 7f4f3e99e640 1 --2- 172.21.15.100:0/3166748343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f38101950 0x7f4f38101d70 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.794+0000 7f4f3e99e640 1 -- 172.21.15.100:0/3166748343 >> 172.21.15.100:0/3166748343 conn(0x7f4f380fcec0 msgr2=0x7f4f380ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:36:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.794+0000 7f4f3e99e640 1 -- 172.21.15.100:0/3166748343 shutdown_connections 2024-05-10T11:36:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.794+0000 7f4f3e99e640 1 -- 172.21.15.100:0/3166748343 wait complete. 2024-05-10T11:36:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.795+0000 7f4f3e99e640 1 Processor -- start 2024-05-10T11:36:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.795+0000 7f4f3e99e640 1 -- start start 2024-05-10T11:36:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.795+0000 7f4f3e99e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f38101950 0x7f4f381058e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.795+0000 7f4f3e99e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f38102b40 0x7f4f38103f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.795+0000 7f4f3e99e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f38103500 0x7f4f38104460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.795+0000 7f4f3e99e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3810d330 con 0x7f4f38103500 2024-05-10T11:36:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.795+0000 7f4f3e99e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3810d1b0 con 0x7f4f38101950 2024-05-10T11:36:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.795+0000 7f4f3e99e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3810d4b0 con 0x7f4f38102b40 2024-05-10T11:36:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.795+0000 7f4f37fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f38101950 0x7f4f381058e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.795+0000 7f4f2ffff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f38102b40 0x7f4f38103f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.795+0000 7f4f37fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f38101950 0x7f4f381058e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:41006/0 (socket says 172.21.15.100:41006) 2024-05-10T11:36:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.795+0000 7f4f3cf3b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f38103500 0x7f4f38104460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.795+0000 7f4f37fff640 1 -- 172.21.15.100:0/3150464549 learned_addr learned my addr 172.21.15.100:0/3150464549 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:36:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.796+0000 7f4f37fff640 1 -- 172.21.15.100:0/3150464549 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f38102b40 msgr2=0x7f4f38103f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.796+0000 7f4f37fff640 1 --2- 172.21.15.100:0/3150464549 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f38102b40 0x7f4f38103f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.796+0000 7f4f37fff640 1 -- 172.21.15.100:0/3150464549 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f38103500 msgr2=0x7f4f38104460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.796+0000 7f4f37fff640 1 --2- 172.21.15.100:0/3150464549 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f38103500 0x7f4f38104460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.796+0000 7f4f37fff640 1 -- 172.21.15.100:0/3150464549 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f38104cf0 con 0x7f4f38101950 2024-05-10T11:36:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.796+0000 7f4f2ffff640 1 --2- 172.21.15.100:0/3150464549 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f38102b40 0x7f4f38103f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:36:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.796+0000 7f4f3cf3b640 1 --2- 172.21.15.100:0/3150464549 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f38103500 0x7f4f38104460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:36:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.796+0000 7f4f37fff640 1 --2- 172.21.15.100:0/3150464549 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f38101950 0x7f4f381058e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4f28005ab0 tx=0x7f4f280042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:36:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.796+0000 7f4f35ffb640 1 -- 172.21.15.100:0/3150464549 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f2803f030 con 0x7f4f38101950 2024-05-10T11:36:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.796+0000 7f4f35ffb640 1 -- 172.21.15.100:0/3150464549 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4f28032e10 con 0x7f4f38101950 2024-05-10T11:36:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.796+0000 7f4f3e99e640 1 -- 172.21.15.100:0/3150464549 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f381a2b80 con 0x7f4f38101950 2024-05-10T11:36:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.796+0000 7f4f35ffb640 1 -- 172.21.15.100:0/3150464549 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f28035070 con 0x7f4f38101950 2024-05-10T11:36:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.796+0000 7f4f3e99e640 1 -- 172.21.15.100:0/3150464549 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f381a30d0 con 0x7f4f38101950 2024-05-10T11:36:28.802 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.799+0000 7f4f35ffb640 1 -- 172.21.15.100:0/3150464549 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f4f28042030 con 0x7f4f38101950 2024-05-10T11:36:28.802 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.800+0000 7f4f4178b740 1 Processor -- start 2024-05-10T11:36:28.802 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.800+0000 7f4f4178b740 1 -- start start 2024-05-10T11:36:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.800+0000 7f4f4178b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce34bc65c0 0x55ce34bc69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.800+0000 7f4f4178b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce34bc6fb0 0x55ce34bcbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.800+0000 7f4f4178b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce34bcc120 0x55ce34bce5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.800+0000 7f4f4178b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55ce34b225d0 con 0x55ce34bc6fb0 2024-05-10T11:36:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.800+0000 7f4f4178b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ce34adb890 con 0x55ce34bcc120 2024-05-10T11:36:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.800+0000 7f4f4178b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55ce34b1e590 con 0x55ce34bc65c0 2024-05-10T11:36:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.800+0000 7f4f37fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce34bc6fb0 0x55ce34bcbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.800+0000 7f4f3cf3b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce34bc65c0 0x55ce34bc69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.800+0000 7f4f3cf3b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce34bc65c0 0x55ce34bc69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:45672/0 (socket says 172.21.15.100:45672) 2024-05-10T11:36:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.801+0000 7f4f3cf3b640 1 -- 172.21.15.100:0/392743642 learned_addr learned my addr 172.21.15.100:0/392743642 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:36:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.801+0000 7f4f2ffff640 1 --2- 172.21.15.100:0/392743642 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce34bcc120 0x55ce34bce5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.801+0000 7f4f35ffb640 1 --2- 172.21.15.100:0/3150464549 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4f240b28a0 0x7f4f240b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.801+0000 7f4f3cf3b640 1 -- 172.21.15.100:0/392743642 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce34bcc120 msgr2=0x55ce34bce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.801+0000 7f4f3cf3b640 1 --2- 172.21.15.100:0/392743642 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce34bcc120 0x55ce34bce5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.801+0000 7f4f3cf3b640 1 -- 172.21.15.100:0/392743642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce34bc6fb0 msgr2=0x55ce34bcbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.801+0000 7f4f3cf3b640 1 --2- 172.21.15.100:0/392743642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce34bc6fb0 0x55ce34bcbab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.801+0000 7f4f3cf3b640 1 -- 172.21.15.100:0/392743642 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce34834210 con 0x55ce34bc65c0 2024-05-10T11:36:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.801+0000 7f4f35ffb640 1 -- 172.21.15.100:0/3150464549 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f4f28102a00 con 0x7f4f38101950 2024-05-10T11:36:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.801+0000 7f4f3cf3b640 1 --2- 172.21.15.100:0/3150464549 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4f240b28a0 0x7f4f240b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.801+0000 7f4f2ffff640 1 --2- 172.21.15.100:0/392743642 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce34bcc120 0x55ce34bce5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:36:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.801+0000 7f4f37fff640 1 --2- 172.21.15.100:0/392743642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce34bc6fb0 0x55ce34bcbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:36:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.801+0000 7f4f3cf3b640 1 --2- 172.21.15.100:0/392743642 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce34bc65c0 0x55ce34bc69e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4f300117e0 tx=0x7f4f30011ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c30f2bc625cc51ce server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:36:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.802+0000 7f4f3cf3b640 1 --2- 172.21.15.100:0/3150464549 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4f240b28a0 0x7f4f240b4d60 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f4f30022000 tx=0x7f4f30023000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:36:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.802+0000 7f4f2e7fc640 1 -- 172.21.15.100:0/392743642 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f30010770 con 0x55ce34bc65c0 2024-05-10T11:36:28.805 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.802+0000 7f4f2e7fc640 1 -- 172.21.15.100:0/392743642 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4f30010da0 con 0x55ce34bc65c0 2024-05-10T11:36:28.805 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.802+0000 7f4f2e7fc640 1 -- 172.21.15.100:0/392743642 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3001cc60 con 0x55ce34bc65c0 2024-05-10T11:36:28.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.803+0000 7f4f4178b740 1 -- 172.21.15.100:0/392743642 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce34bc65c0 msgr2=0x55ce34bc69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:28.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.803+0000 7f4f4178b740 1 --2- 172.21.15.100:0/392743642 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce34bc65c0 0x55ce34bc69e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4f300117e0 tx=0x7f4f30011ca0 comp rx=0 tx=0).stop 2024-05-10T11:36:28.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.803+0000 7f4f4178b740 1 -- 172.21.15.100:0/392743642 shutdown_connections 2024-05-10T11:36:28.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.803+0000 7f4f4178b740 1 --2- 172.21.15.100:0/392743642 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce34bcc120 0x55ce34bce5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:28.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.803+0000 7f4f4178b740 1 --2- 172.21.15.100:0/392743642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce34bc6fb0 0x55ce34bcbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:28.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.803+0000 7f4f4178b740 1 --2- 172.21.15.100:0/392743642 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce34bc65c0 0x55ce34bc69e0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:28.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.803+0000 7f4f4178b740 1 -- 172.21.15.100:0/392743642 >> 172.21.15.100:0/392743642 conn(0x55ce34bc1c30 msgr2=0x55ce34bc2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:36:28.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.803+0000 7f4f4178b740 1 -- 172.21.15.100:0/392743642 shutdown_connections 2024-05-10T11:36:28.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.803+0000 7f4f4178b740 1 -- 172.21.15.100:0/392743642 wait complete. 2024-05-10T11:36:28.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.804+0000 7f4f4178b740 1 -- start start 2024-05-10T11:36:28.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.804+0000 7f4f4178b740 1 Processor -- start 2024-05-10T11:36:28.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.804+0000 7f4f4178b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce34bc65c0 0x55ce34bd08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:28.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.804+0000 7f4f4178b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce34bc6fb0 0x55ce34bd0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:28.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.804+0000 7f4f4178b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce34bcc120 0x55ce34bd7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:28.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.804+0000 7f4f2ffff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce34bc6fb0 0x55ce34bd0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:28.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.804+0000 7f4f2ffff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce34bc6fb0 0x55ce34bd0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:41034/0 (socket says 172.21.15.100:41034) 2024-05-10T11:36:28.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.804+0000 7f4f2ffff640 1 -- 172.21.15.100:0/1331072328 learned_addr learned my addr 172.21.15.100:0/1331072328 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:36:28.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.804+0000 7f4f37fff640 1 --2- 172.21.15.100:0/1331072328 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce34bc65c0 0x55ce34bd08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:28.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.804+0000 7f4f4178b740 1 -- 172.21.15.100:0/1331072328 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55ce34bcf170 con 0x55ce34bcc120 2024-05-10T11:36:28.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.805+0000 7f4f4178b740 1 -- 172.21.15.100:0/1331072328 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ce34adf370 con 0x55ce34bc6fb0 2024-05-10T11:36:28.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.805+0000 7f4f4178b740 1 -- 172.21.15.100:0/1331072328 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55ce34bcf2f0 con 0x55ce34bc65c0 2024-05-10T11:36:28.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.805+0000 7f4f3cf3b640 1 --2- 172.21.15.100:0/1331072328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce34bcc120 0x55ce34bd7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:28.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.805+0000 7f4f4178b740 10 client.0 _ll_register_callbacks cb 0x55ce34833a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:36:28.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.805+0000 7f4f3cf3b640 1 -- 172.21.15.100:0/1331072328 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce34bc65c0 msgr2=0x55ce34bd08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:28.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.805+0000 7f4f3cf3b640 1 --2- 172.21.15.100:0/1331072328 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce34bc65c0 0x55ce34bd08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:28.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.805+0000 7f4f3cf3b640 1 -- 172.21.15.100:0/1331072328 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce34bc6fb0 msgr2=0x55ce34bd0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:28.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.805+0000 7f4f3cf3b640 1 --2- 172.21.15.100:0/1331072328 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce34bc6fb0 0x55ce34bd0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:28.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.805+0000 7f4f3cf3b640 1 -- 172.21.15.100:0/1331072328 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce34900620 con 0x55ce34bcc120 2024-05-10T11:36:28.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.805+0000 7f4f3cf3b640 1 --2- 172.21.15.100:0/1331072328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce34bcc120 0x55ce34bd7e50 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x55ce34bc7810 tx=0x7f4f30005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b499d0306a01c5f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:36:28.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.805+0000 7f4f177fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:36:28.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.805+0000 7f4f177fe640 1 -- 172.21.15.100:0/1331072328 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f30010770 con 0x55ce34bcc120 2024-05-10T11:36:28.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.805+0000 7f4f4178b740 1 -- 172.21.15.100:0/1331072328 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f30010da0 con 0x55ce34bcc120 2024-05-10T11:36:28.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.805+0000 7f4f177fe640 1 -- 172.21.15.100:0/1331072328 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f4f30010910 con 0x55ce34bcc120 2024-05-10T11:36:28.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.806+0000 7f4f177fe640 1 -- 172.21.15.100:0/1331072328 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f4f3003a050 con 0x55ce34bcc120 2024-05-10T11:36:28.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.806+0000 7f4f4178b740 10 client.18978 fetch_fsmap learned FSMap version 41 2024-05-10T11:36:28.813 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.806+0000 7f4f4178b740 1 -- 172.21.15.100:0/1331072328 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55ce34bdae30 con 0x55ce34bcc120 2024-05-10T11:36:28.813 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.806+0000 7f4f177fe640 1 -- 172.21.15.100:0/1331072328 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4f30010da0 con 0x55ce34bcc120 2024-05-10T11:36:28.813 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.806+0000 7f4f177fe640 1 -- 172.21.15.100:0/1331072328 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f4f30029c80 con 0x55ce34bcc120 2024-05-10T11:36:28.813 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.806+0000 7f4f4178b740 10 client.18978 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:36:28.813 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.806+0000 7f4f4178b740 10 client.18978 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:36:28.813 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.806+0000 7f4f4178b740 20 client.18978 populate_metadata read hostname 'smithi100' 2024-05-10T11:36:28.813 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.806+0000 7f4f4178b740 1 --2- 172.21.15.100:0/1331072328 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ce34bdc7f0 0x55ce34bdec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:28.813 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.806+0000 7f4f4178b740 4 client.18978 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:36:28.813 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.806+0000 7f4f4178b740 1 -- 172.21.15.100:0/1331072328 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ce34b301e0 con 0x55ce34bdc7f0 2024-05-10T11:36:28.813 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.807+0000 7f4f37fff640 1 --2- 172.21.15.100:0/1331072328 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ce34bdc7f0 0x55ce34bdec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:28.813 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.807+0000 7f4f177fe640 10 client.18978 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:36:28.813 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.807+0000 7f4f37fff640 1 --2- 172.21.15.100:0/1331072328 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ce34bdc7f0 0x55ce34bdec10 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecd0f9b2e4328bcb server_cookie=f81dcc0678081e05 in_seq=0 out_seq=0 2024-05-10T11:36:28.813 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.808+0000 7f4f177fe640 1 -- 172.21.15.100:0/1331072328 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ce34b301e0 con 0x55ce34bdc7f0 2024-05-10T11:36:28.813 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.808+0000 7f4f177fe640 10 client.18978 handle_command_reply: tid=0 2024-05-10T11:36:28.813 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.808+0000 7f4f4178b740 1 client.18978 shutdown 2024-05-10T11:36:28.814 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.808+0000 7f4f4178b740 1 -- 172.21.15.100:0/1331072328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce34bcc120 msgr2=0x55ce34bd7e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:28.814 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.808+0000 7f4f4178b740 1 --2- 172.21.15.100:0/1331072328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce34bcc120 0x55ce34bd7e50 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x55ce34bc7810 tx=0x7f4f30005be0 comp rx=0 tx=0).stop 2024-05-10T11:36:28.814 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.808+0000 7f4f4178b740 1 -- 172.21.15.100:0/1331072328 shutdown_connections 2024-05-10T11:36:28.814 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.808+0000 7f4f4178b740 1 --2- 172.21.15.100:0/1331072328 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ce34bdc7f0 0x55ce34bdec10 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:28.814 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.808+0000 7f4f4178b740 1 --2- 172.21.15.100:0/1331072328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce34bcc120 0x55ce34bd7e50 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:28.814 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.808+0000 7f4f177fe640 0 client.18978 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:36:28.814 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.808+0000 7f4f4178b740 1 --2- 172.21.15.100:0/1331072328 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce34bc6fb0 0x55ce34bd0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:28.814 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.808+0000 7f4f4178b740 1 --2- 172.21.15.100:0/1331072328 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce34bc65c0 0x55ce34bd08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:28.814 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.808+0000 7f4f4178b740 1 -- 172.21.15.100:0/1331072328 >> 172.21.15.100:0/1331072328 conn(0x55ce34bc1c30 msgr2=0x55ce34bb8580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:36:28.814 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.808+0000 7f4f4178b740 1 -- 172.21.15.100:0/1331072328 shutdown_connections 2024-05-10T11:36:28.814 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.809+0000 7f4f4178b740 1 -- 172.21.15.100:0/1331072328 wait complete. 2024-05-10T11:36:28.815 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.809+0000 7f4f4178b740 20 client.18978 trim_cache size 0 max 16384 2024-05-10T11:36:28.836 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.836+0000 7f4f4178b740 1 Processor -- start 2024-05-10T11:36:28.836 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.836+0000 7f4f4178b740 1 -- start start 2024-05-10T11:36:28.836 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.836+0000 7f4f4178b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce34bc65c0 0x55ce34bc4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:28.837 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.837+0000 7f4f4178b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce34bc6fb0 0x55ce34bc4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:28.837 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.837+0000 7f4f4178b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce34bcc120 0x55ce34bc5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:28.837 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.837+0000 7f4f37fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce34bc65c0 0x55ce34bc4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:28.838 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.837+0000 7f4f37fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce34bc65c0 0x55ce34bc4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:41048/0 (socket says 172.21.15.100:41048) 2024-05-10T11:36:28.838 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.837+0000 7f4f4178b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4f381ab9a0 con 0x55ce34bcc120 2024-05-10T11:36:28.838 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.837+0000 7f4f4178b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4f381a9270 con 0x55ce34bc65c0 2024-05-10T11:36:28.838 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.837+0000 7f4f4178b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4f381a70c0 con 0x55ce34bc6fb0 2024-05-10T11:36:28.838 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.837+0000 7f4f37fff640 1 -- 172.21.15.100:0/4239338225 learned_addr learned my addr 172.21.15.100:0/4239338225 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:36:28.838 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.837+0000 7f4f3cf3b640 1 --2- 172.21.15.100:0/4239338225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce34bcc120 0x55ce34bc5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:28.838 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.837+0000 7f4f3cf3b640 1 -- 172.21.15.100:0/4239338225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce34bc6fb0 msgr2=0x55ce34bc4e50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T11:36:28.838 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.837+0000 7f4f3cf3b640 1 --2- 172.21.15.100:0/4239338225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce34bc6fb0 0x55ce34bc4e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:28.838 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.837+0000 7f4f3cf3b640 1 -- 172.21.15.100:0/4239338225 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce34bc65c0 msgr2=0x55ce34bc4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:28.838 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.837+0000 7f4f3cf3b640 1 --2- 172.21.15.100:0/4239338225 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce34bc65c0 0x55ce34bc4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:28.838 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.837+0000 7f4f3cf3b640 1 -- 172.21.15.100:0/4239338225 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce34bdef80 con 0x55ce34bcc120 2024-05-10T11:36:28.838 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.837+0000 7f4f3cf3b640 1 --2- 172.21.15.100:0/4239338225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce34bcc120 0x55ce34bc5390 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f4f300346e0 tx=0x7f4f30011760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a3daf72a69df368 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:36:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.838+0000 7f4f2dffb640 1 -- 172.21.15.100:0/4239338225 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3001c860 con 0x55ce34bcc120 2024-05-10T11:36:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.838+0000 7f4f2dffb640 1 -- 172.21.15.100:0/4239338225 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4f3001ce50 con 0x55ce34bcc120 2024-05-10T11:36:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.838+0000 7f4f2dffb640 1 -- 172.21.15.100:0/4239338225 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f300107d0 con 0x55ce34bcc120 2024-05-10T11:36:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.839+0000 7f4f4178b740 1 -- 172.21.15.100:0/4239338225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce34bcc120 msgr2=0x55ce34bc5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.839+0000 7f4f4178b740 1 --2- 172.21.15.100:0/4239338225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce34bcc120 0x55ce34bc5390 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f4f300346e0 tx=0x7f4f30011760 comp rx=0 tx=0).stop 2024-05-10T11:36:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.839+0000 7f4f4178b740 1 -- 172.21.15.100:0/4239338225 shutdown_connections 2024-05-10T11:36:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.839+0000 7f4f4178b740 1 --2- 172.21.15.100:0/4239338225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce34bcc120 0x55ce34bc5390 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.839+0000 7f4f4178b740 1 --2- 172.21.15.100:0/4239338225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce34bc6fb0 0x55ce34bc4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.839+0000 7f4f4178b740 1 --2- 172.21.15.100:0/4239338225 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce34bc65c0 0x55ce34bc4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.839+0000 7f4f4178b740 1 -- 172.21.15.100:0/4239338225 >> 172.21.15.100:0/4239338225 conn(0x55ce34bc1c30 msgr2=0x55ce34bc8230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:36:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.840+0000 7f4f4178b740 1 -- 172.21.15.100:0/4239338225 shutdown_connections 2024-05-10T11:36:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.840+0000 7f4f4178b740 1 -- 172.21.15.100:0/4239338225 wait complete. 2024-05-10T11:36:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.840+0000 7f4f4178b740 1 -- start start 2024-05-10T11:36:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.840+0000 7f4f4178b740 1 Processor -- start 2024-05-10T11:36:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.841+0000 7f4f4178b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce34bc65c0 0x55ce34bd41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.841+0000 7f4f4178b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce34bc6fb0 0x55ce34bd46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.841+0000 7f4f4178b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce34bcc120 0x55ce34c45040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.841+0000 7f4f4178b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4f381a7730 con 0x55ce34bc65c0 2024-05-10T11:36:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.841+0000 7f4f4178b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4f381a8ba0 con 0x55ce34bcc120 2024-05-10T11:36:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.841+0000 7f4f4178b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4f381a7e00 con 0x55ce34bc6fb0 2024-05-10T11:36:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.841+0000 7f4f2ffff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce34bc6fb0 0x55ce34bd46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.841+0000 7f4f2ffff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce34bc6fb0 0x55ce34bd46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:45686/0 (socket says 172.21.15.100:45686) 2024-05-10T11:36:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.841+0000 7f4f2ffff640 1 -- 172.21.15.100:0/751625062 learned_addr learned my addr 172.21.15.100:0/751625062 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:36:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.841+0000 7f4f3cf3b640 1 --2- 172.21.15.100:0/751625062 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce34bcc120 0x55ce34c45040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.841+0000 7f4f37fff640 1 --2- 172.21.15.100:0/751625062 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce34bc65c0 0x55ce34bd41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.841+0000 7f4f4178b740 10 client.0 _ll_register_callbacks cb 0x7f4f28108400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:36:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.841+0000 7f4f2ffff640 1 -- 172.21.15.100:0/751625062 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce34bcc120 msgr2=0x55ce34c45040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.841+0000 7f4f2ffff640 1 --2- 172.21.15.100:0/751625062 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce34bcc120 0x55ce34c45040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.841+0000 7f4f2ffff640 1 -- 172.21.15.100:0/751625062 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce34bc65c0 msgr2=0x55ce34bd41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.841+0000 7f4f2ffff640 1 --2- 172.21.15.100:0/751625062 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce34bc65c0 0x55ce34bd41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.841+0000 7f4f2ffff640 1 -- 172.21.15.100:0/751625062 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce34bd77b0 con 0x55ce34bc6fb0 2024-05-10T11:36:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.842+0000 7f4f2ffff640 1 --2- 172.21.15.100:0/751625062 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce34bc6fb0 0x55ce34bd46e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4f20012c20 tx=0x7f4f2000f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12436935c654250 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:36:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.842+0000 7f4f177fe640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:36:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.842+0000 7f4f177fe640 1 -- 172.21.15.100:0/751625062 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f2000fd40 con 0x55ce34bc6fb0 2024-05-10T11:36:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.842+0000 7f4f177fe640 1 -- 172.21.15.100:0/751625062 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f4f200120d0 con 0x55ce34bc6fb0 2024-05-10T11:36:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.842+0000 7f4f4178b740 1 -- 172.21.15.100:0/751625062 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f3001ce50 con 0x55ce34bc6fb0 2024-05-10T11:36:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.842+0000 7f4f177fe640 1 -- 172.21.15.100:0/751625062 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f4f20029050 con 0x55ce34bc6fb0 2024-05-10T11:36:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.842+0000 7f4f4178b740 10 client.28390 fetch_fsmap learned FSMap version 41 2024-05-10T11:36:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.843+0000 7f4f4178b740 1 -- 172.21.15.100:0/751625062 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55ce34c46620 con 0x55ce34bc6fb0 2024-05-10T11:36:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.843+0000 7f4f177fe640 1 -- 172.21.15.100:0/751625062 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4f3001ce50 con 0x55ce34bc6fb0 2024-05-10T11:36:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.843+0000 7f4f177fe640 1 -- 172.21.15.100:0/751625062 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f4f2001c400 con 0x55ce34bc6fb0 2024-05-10T11:36:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.843+0000 7f4f4178b740 10 client.28390 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:36:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.843+0000 7f4f4178b740 10 client.28390 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:36:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.843+0000 7f4f4178b740 20 client.28390 populate_metadata read hostname 'smithi100' 2024-05-10T11:36:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.843+0000 7f4f4178b740 1 --2- 172.21.15.100:0/751625062 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ce34bdc7f0 0x55ce34c49110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.843+0000 7f4f4178b740 4 client.28390 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:36:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.843+0000 7f4f4178b740 1 -- 172.21.15.100:0/751625062 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ce34b2a050 con 0x55ce34bdc7f0 2024-05-10T11:36:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.843+0000 7f4f37fff640 1 --2- 172.21.15.100:0/751625062 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ce34bdc7f0 0x55ce34c49110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.844+0000 7f4f177fe640 10 client.28390 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:36:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:28.844+0000 7f4f37fff640 1 --2- 172.21.15.100:0/751625062 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ce34bdc7f0 0x55ce34c49110 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e8a0eb9b97339cf server_cookie=2e6868db57d754da in_seq=0 out_seq=0 2024-05-10T11:36:28.931 INFO:tasks.workunit.client.0.smithi100.stdout: 10 44353 9.13 MB/sec execute 142 sec latency 127.863 ms 2024-05-10T11:36:29.015 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:29.014+0000 7f4f177fe640 1 -- 172.21.15.100:0/751625062 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ce34b2a050 con 0x55ce34bdc7f0 2024-05-10T11:36:29.015 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:29.014+0000 7f4f177fe640 10 client.28390 handle_command_reply: tid=0 2024-05-10T11:36:29.015 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:29.014+0000 7f4f4178b740 1 client.28390 shutdown 2024-05-10T11:36:29.015 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:29.014+0000 7f4f4178b740 1 -- 172.21.15.100:0/751625062 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce34bc6fb0 msgr2=0x55ce34bd46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:29.015 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:29.014+0000 7f4f4178b740 1 --2- 172.21.15.100:0/751625062 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce34bc6fb0 0x55ce34bd46e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4f20012c20 tx=0x7f4f2000f790 comp rx=0 tx=0).stop 2024-05-10T11:36:29.015 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:29.014+0000 7f4f4178b740 1 -- 172.21.15.100:0/751625062 shutdown_connections 2024-05-10T11:36:29.015 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:29.014+0000 7f4f4178b740 1 --2- 172.21.15.100:0/751625062 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ce34bdc7f0 0x55ce34c49110 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:29.015 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:29.014+0000 7f4f4178b740 1 --2- 172.21.15.100:0/751625062 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce34bcc120 0x55ce34c45040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:29.015 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:29.014+0000 7f4f4178b740 1 --2- 172.21.15.100:0/751625062 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce34bc6fb0 0x55ce34bd46e0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:29.015 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:29.014+0000 7f4f4178b740 1 --2- 172.21.15.100:0/751625062 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce34bc65c0 0x55ce34bd41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:29.015 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:29.014+0000 7f4f4178b740 1 -- 172.21.15.100:0/751625062 >> 172.21.15.100:0/751625062 conn(0x55ce34bc1c30 msgr2=0x55ce34bc8230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:36:29.015 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:29.014+0000 7f4f177fe640 0 client.28390 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:36:29.016 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:29.014+0000 7f4f4178b740 1 -- 172.21.15.100:0/751625062 shutdown_connections 2024-05-10T11:36:29.016 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:29.014+0000 7f4f4178b740 1 -- 172.21.15.100:0/751625062 wait complete. 2024-05-10T11:36:29.016 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:29.015+0000 7f4f4178b740 20 client.28390 trim_cache size 0 max 16384 2024-05-10T11:36:29.016 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:36:29.016 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:36:29.016 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:36:29.016 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:36:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:29.016+0000 7f4f3e99e640 1 -- 172.21.15.100:0/3150464549 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4f240b28a0 msgr2=0x7f4f240b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:29.016+0000 7f4f3e99e640 1 --2- 172.21.15.100:0/3150464549 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4f240b28a0 0x7f4f240b4d60 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f4f30022000 tx=0x7f4f30023000 comp rx=0 tx=0).stop 2024-05-10T11:36:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:29.016+0000 7f4f3e99e640 1 -- 172.21.15.100:0/3150464549 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f38101950 msgr2=0x7f4f381058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:29.018 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:29.016+0000 7f4f3e99e640 1 --2- 172.21.15.100:0/3150464549 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f38101950 0x7f4f381058e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4f28005ab0 tx=0x7f4f280042f0 comp rx=0 tx=0).stop 2024-05-10T11:36:29.018 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:29.016+0000 7f4f3e99e640 1 -- 172.21.15.100:0/3150464549 shutdown_connections 2024-05-10T11:36:29.018 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:29.016+0000 7f4f3e99e640 1 --2- 172.21.15.100:0/3150464549 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4f240b28a0 0x7f4f240b4d60 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:29.018 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:29.016+0000 7f4f3e99e640 1 --2- 172.21.15.100:0/3150464549 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f38103500 0x7f4f38104460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:29.018 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:29.016+0000 7f4f3e99e640 1 --2- 172.21.15.100:0/3150464549 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f38102b40 0x7f4f38103f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:29.018 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:29.016+0000 7f4f3e99e640 1 --2- 172.21.15.100:0/3150464549 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f38101950 0x7f4f381058e0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:29.018 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:29.016+0000 7f4f3e99e640 1 -- 172.21.15.100:0/3150464549 >> 172.21.15.100:0/3150464549 conn(0x7f4f380fcec0 msgr2=0x7f4f380fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:36:29.018 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:29.017+0000 7f4f3e99e640 1 -- 172.21.15.100:0/3150464549 shutdown_connections 2024-05-10T11:36:29.018 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:29.017+0000 7f4f3e99e640 1 -- 172.21.15.100:0/3150464549 wait complete. 2024-05-10T11:36:29.931 INFO:tasks.workunit.client.0.smithi100.stdout: 10 44502 9.11 MB/sec execute 143 sec latency 122.113 ms 2024-05-10T11:36:30.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:36:29 smithi178 ceph-mon[33359]: pgmap v964: 129 pgs: 129 active+clean; 463 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 4.0 MiB/s wr, 257 op/s 2024-05-10T11:36:30.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:36:29 smithi188 ceph-mon[34075]: pgmap v964: 129 pgs: 129 active+clean; 463 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 4.0 MiB/s wr, 257 op/s 2024-05-10T11:36:30.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:36:29 smithi100 ceph-mon[30296]: pgmap v964: 129 pgs: 129 active+clean; 463 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 4.0 MiB/s wr, 257 op/s 2024-05-10T11:36:30.931 INFO:tasks.workunit.client.0.smithi100.stdout: 10 44661 9.13 MB/sec execute 144 sec latency 138.172 ms 2024-05-10T11:36:31.931 INFO:tasks.workunit.client.0.smithi100.stdout: 10 44775 9.09 MB/sec execute 145 sec latency 115.039 ms 2024-05-10T11:36:32.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:36:31 smithi178 ceph-mon[33359]: pgmap v965: 129 pgs: 129 active+clean; 464 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 973 KiB/s rd, 4.5 MiB/s wr, 299 op/s 2024-05-10T11:36:32.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:36:31 smithi188 ceph-mon[34075]: pgmap v965: 129 pgs: 129 active+clean; 464 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 973 KiB/s rd, 4.5 MiB/s wr, 299 op/s 2024-05-10T11:36:32.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:36:31 smithi100 ceph-mon[30296]: pgmap v965: 129 pgs: 129 active+clean; 464 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 973 KiB/s rd, 4.5 MiB/s wr, 299 op/s 2024-05-10T11:36:32.931 INFO:tasks.workunit.client.0.smithi100.stdout: 10 44903 9.06 MB/sec execute 146 sec latency 158.441 ms 2024-05-10T11:36:33.931 INFO:tasks.workunit.client.0.smithi100.stdout: 10 45052 9.05 MB/sec execute 147 sec latency 212.344 ms 2024-05-10T11:36:34.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:36:33 smithi178 ceph-mon[33359]: pgmap v966: 129 pgs: 129 active+clean; 464 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 812 KiB/s rd, 3.9 MiB/s wr, 249 op/s 2024-05-10T11:36:34.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:36:33 smithi188 ceph-mon[34075]: pgmap v966: 129 pgs: 129 active+clean; 464 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 812 KiB/s rd, 3.9 MiB/s wr, 249 op/s 2024-05-10T11:36:34.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:36:33 smithi100 ceph-mon[30296]: pgmap v966: 129 pgs: 129 active+clean; 464 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 812 KiB/s rd, 3.9 MiB/s wr, 249 op/s 2024-05-10T11:36:34.931 INFO:tasks.workunit.client.0.smithi100.stdout: 10 45205 9.03 MB/sec execute 148 sec latency 185.467 ms 2024-05-10T11:36:35.931 INFO:tasks.workunit.client.0.smithi100.stdout: 10 45366 9.05 MB/sec execute 149 sec latency 111.064 ms 2024-05-10T11:36:36.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:36:35 smithi178 ceph-mon[33359]: pgmap v967: 129 pgs: 129 active+clean; 460 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 3.9 MiB/s wr, 246 op/s 2024-05-10T11:36:36.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:36:35 smithi188 ceph-mon[34075]: pgmap v967: 129 pgs: 129 active+clean; 460 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 3.9 MiB/s wr, 246 op/s 2024-05-10T11:36:36.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:36:35 smithi100 ceph-mon[30296]: pgmap v967: 129 pgs: 129 active+clean; 460 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 3.9 MiB/s wr, 246 op/s 2024-05-10T11:36:36.931 INFO:tasks.workunit.client.0.smithi100.stdout: 10 45498 9.03 MB/sec execute 150 sec latency 125.265 ms 2024-05-10T11:36:37.931 INFO:tasks.workunit.client.0.smithi100.stdout: 10 45626 9.02 MB/sec execute 151 sec latency 168.255 ms 2024-05-10T11:36:38.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:36:37 smithi178 ceph-mon[33359]: pgmap v968: 129 pgs: 129 active+clean; 462 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 3.6 MiB/s wr, 283 op/s 2024-05-10T11:36:38.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:36:37 smithi188 ceph-mon[34075]: pgmap v968: 129 pgs: 129 active+clean; 462 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 3.6 MiB/s wr, 283 op/s 2024-05-10T11:36:38.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:36:37 smithi100 ceph-mon[30296]: pgmap v968: 129 pgs: 129 active+clean; 462 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 3.6 MiB/s wr, 283 op/s 2024-05-10T11:36:38.931 INFO:tasks.workunit.client.0.smithi100.stdout: 10 45809 9.08 MB/sec execute 152 sec latency 115.723 ms 2024-05-10T11:36:39.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:36:38 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:36:39.010 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:36:38 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:36:39.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:36:38 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:36:39.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:39.759+0000 7f9c5bb60700 1 -- 172.21.15.100:0/3248088682 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c5410fb60 msgr2=0x7f9c5410ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:39.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:39.759+0000 7f9c5bb60700 1 --2- 172.21.15.100:0/3248088682 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c5410fb60 0x7f9c5410ff40 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9c440098f0 tx=0x7f9c44009c00 comp rx=0 tx=0).stop 2024-05-10T11:36:39.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:39.759+0000 7f9c5bb60700 1 -- 172.21.15.100:0/3248088682 shutdown_connections 2024-05-10T11:36:39.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:39.759+0000 7f9c5bb60700 1 --2- 172.21.15.100:0/3248088682 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c5410a490 0x7f9c541169a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:39.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:39.759+0000 7f9c5bb60700 1 --2- 172.21.15.100:0/3248088682 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c5410fb60 0x7f9c5410ff40 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:39.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:39.759+0000 7f9c5bb60700 1 --2- 172.21.15.100:0/3248088682 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c54109ad0 0x7f9c54109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:39.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:39.759+0000 7f9c5bb60700 1 -- 172.21.15.100:0/3248088682 >> 172.21.15.100:0/3248088682 conn(0x7f9c54076c70 msgr2=0x7f9c541082c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:36:39.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:39.760+0000 7f9c5bb60700 1 -- 172.21.15.100:0/3248088682 shutdown_connections 2024-05-10T11:36:39.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:39.760+0000 7f9c5bb60700 1 -- 172.21.15.100:0/3248088682 wait complete. 2024-05-10T11:36:39.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:39.761+0000 7f9c5bb60700 1 Processor -- start 2024-05-10T11:36:39.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:39.761+0000 7f9c5bb60700 1 -- start start 2024-05-10T11:36:39.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:39.761+0000 7f9c5bb60700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c54109ad0 0x7f9c54074cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:39.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:39.761+0000 7f9c5bb60700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c5410a490 0x7f9c5406dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:39.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:39.761+0000 7f9c5bb60700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c5410fb60 0x7f9c5406e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:39.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:39.761+0000 7f9c5bb60700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9c54119bd0 con 0x7f9c5410a490 2024-05-10T11:36:39.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:39.761+0000 7f9c5bb60700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9c54119a50 con 0x7f9c5410fb60 2024-05-10T11:36:39.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:39.761+0000 7f9c5bb60700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9c54119d50 con 0x7f9c54109ad0 2024-05-10T11:36:39.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:39.761+0000 7f9c5a0fd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c5410fb60 0x7f9c5406e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:39.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:39.762+0000 7f9c5a0fd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c5410fb60 0x7f9c5406e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:34526/0 (socket says 172.21.15.100:34526) 2024-05-10T11:36:39.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:39.762+0000 7f9c5a0fd700 1 -- 172.21.15.100:0/3847031346 learned_addr learned my addr 172.21.15.100:0/3847031346 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:36:39.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:39.762+0000 7f9c590fb700 1 --2- 172.21.15.100:0/3847031346 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c5410a490 0x7f9c5406dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:39.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:39.762+0000 7f9c5a0fd700 1 -- 172.21.15.100:0/3847031346 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c54109ad0 msgr2=0x7f9c54074cf0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-10T11:36:39.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:39.762+0000 7f9c5a0fd700 1 --2- 172.21.15.100:0/3847031346 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c54109ad0 0x7f9c54074cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:39.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:39.762+0000 7f9c5a0fd700 1 -- 172.21.15.100:0/3847031346 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c5410a490 msgr2=0x7f9c5406dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:39.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:39.762+0000 7f9c5a0fd700 1 --2- 172.21.15.100:0/3847031346 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c5410a490 0x7f9c5406dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:39.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:39.762+0000 7f9c5a0fd700 1 -- 172.21.15.100:0/3847031346 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c5406e930 con 0x7f9c5410fb60 2024-05-10T11:36:39.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:39.762+0000 7f9c5a0fd700 1 --2- 172.21.15.100:0/3847031346 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c5410fb60 0x7f9c5406e2c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c00ef30 tx=0x7f9c4c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:36:39.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:39.762+0000 7f9c4affd700 1 -- 172.21.15.100:0/3847031346 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c0092e0 con 0x7f9c5410fb60 2024-05-10T11:36:39.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:39.763+0000 7f9c4affd700 1 -- 172.21.15.100:0/3847031346 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c4c004bb0 con 0x7f9c5410fb60 2024-05-10T11:36:39.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:39.763+0000 7f9c5bb60700 1 -- 172.21.15.100:0/3847031346 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c5406ebc0 con 0x7f9c5410fb60 2024-05-10T11:36:39.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:39.763+0000 7f9c4affd700 1 -- 172.21.15.100:0/3847031346 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c004020 con 0x7f9c5410fb60 2024-05-10T11:36:39.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:39.763+0000 7f9c5bb60700 1 -- 172.21.15.100:0/3847031346 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c54072b90 con 0x7f9c5410fb60 2024-05-10T11:36:39.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:39.764+0000 7f9c5bb60700 1 -- 172.21.15.100:0/3847031346 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c5410b1d0 con 0x7f9c5410fb60 2024-05-10T11:36:39.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:39.766+0000 7f9c4affd700 1 -- 172.21.15.100:0/3847031346 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f9c4c01b030 con 0x7f9c5410fb60 2024-05-10T11:36:39.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:39.767+0000 7f9c4affd700 1 --2- 172.21.15.100:0/3847031346 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f9c400b2930 0x7f9c400b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:39.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:39.767+0000 7f9c4affd700 1 -- 172.21.15.100:0/3847031346 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f9c4c0e2100 con 0x7f9c5410fb60 2024-05-10T11:36:39.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:39.767+0000 7f9c598fc700 1 --2- 172.21.15.100:0/3847031346 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f9c400b2930 0x7f9c400b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:39.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:39.768+0000 7f9c598fc700 1 --2- 172.21.15.100:0/3847031346 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f9c400b2930 0x7f9c400b4df0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f9c44009230 tx=0x7f9c44009f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:36:39.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:39.771+0000 7f9c4affd700 1 -- 172.21.15.100:0/3847031346 <== mon.2 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f9c4c022020 con 0x7f9c5410fb60 2024-05-10T11:36:39.931 INFO:tasks.workunit.client.0.smithi100.stdout: 10 45975 9.08 MB/sec execute 153 sec latency 223.283 ms 2024-05-10T11:36:40.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:36:39 smithi178 ceph-mon[33359]: pgmap v969: 129 pgs: 129 active+clean; 462 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 3.2 MiB/s wr, 242 op/s 2024-05-10T11:36:40.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:36:39 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:36:40.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:36:39 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:36:40.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:36:39 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:36:40.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:36:39 smithi188 ceph-mon[34075]: pgmap v969: 129 pgs: 129 active+clean; 462 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 3.2 MiB/s wr, 242 op/s 2024-05-10T11:36:40.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:36:39 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:36:40.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:36:39 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:36:40.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:36:39 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:36:40.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:36:39 smithi100 ceph-mon[30296]: pgmap v969: 129 pgs: 129 active+clean; 462 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 3.2 MiB/s wr, 242 op/s 2024-05-10T11:36:40.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:36:39 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:36:40.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:36:39 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:36:40.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:36:39 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:36:40.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:40.189+0000 7f9c5bb60700 1 -- 172.21.15.100:0/3847031346 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9c54073660 con 0x7f9c5410fb60 2024-05-10T11:36:40.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:40.189+0000 7f9c4affd700 1 -- 172.21.15.100:0/3847031346 <== mon.2 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+11897 (secure 0 0 0) 0x7f9c4c010040 con 0x7f9c5410fb60 2024-05-10T11:36:40.190 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:36:40.191 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2072225012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2072225012},{"type":"v1","addr":"172.21.15.100:6835","nonce":2072225012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/401939540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":401939540},{"type":"v1","addr":"172.21.15.100:6837","nonce":401939540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14571,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/3946311025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3946311025},{"type":"v1","addr":"172.21.15.100:6839","nonce":3946311025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3736371660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3736371660},{"type":"v1","addr":"172.21.15.100:6841","nonce":3736371660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/2741810791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2741810791},{"type":"v1","addr":"172.21.15.178:6833","nonce":2741810791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1704891041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1704891041},{"type":"v1","addr":"172.21.15.188:6833","nonce":1704891041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":41,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T11:14:27.242452+0000","modified":"2024-05-10T11:35:29.588072+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24439,"mds_1":24437,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/2124076312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2124076312},{"type":"v1","addr":"172.21.15.178:6835","nonce":2124076312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/3224358425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3224358425},{"type":"v1","addr":"172.21.15.178:6837","nonce":3224358425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/1943342620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1943342620},{"type":"v1","addr":"172.21.15.188:6837","nonce":1943342620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.178:6839/3760985404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3760985404},{"type":"v1","addr":"172.21.15.178:6839","nonce":3760985404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.188:6839/3194917070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":3194917070},{"type":"v1","addr":"172.21.15.188:6839","nonce":3194917070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27034":{"gid":27034,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/3139559450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3139559450},{"type":"v1","addr":"172.21.15.188:6835","nonce":3139559450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24439,"qdb_cluster":[24437,24439,24427]},"id":1}]} 2024-05-10T11:36:40.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:40.193+0000 7f9c5bb60700 1 -- 172.21.15.100:0/3847031346 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f9c400b2930 msgr2=0x7f9c400b4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:40.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:40.193+0000 7f9c5bb60700 1 --2- 172.21.15.100:0/3847031346 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f9c400b2930 0x7f9c400b4df0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f9c44009230 tx=0x7f9c44009f50 comp rx=0 tx=0).stop 2024-05-10T11:36:40.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:40.193+0000 7f9c5bb60700 1 -- 172.21.15.100:0/3847031346 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c5410fb60 msgr2=0x7f9c5406e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:40.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:40.193+0000 7f9c5bb60700 1 --2- 172.21.15.100:0/3847031346 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c5410fb60 0x7f9c5406e2c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c00ef30 tx=0x7f9c4c00c3a0 comp rx=0 tx=0).stop 2024-05-10T11:36:40.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:40.193+0000 7f9c5bb60700 1 -- 172.21.15.100:0/3847031346 shutdown_connections 2024-05-10T11:36:40.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:40.193+0000 7f9c5bb60700 1 --2- 172.21.15.100:0/3847031346 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f9c400b2930 0x7f9c400b4df0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:40.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:40.193+0000 7f9c5bb60700 1 --2- 172.21.15.100:0/3847031346 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c5410fb60 0x7f9c5406e2c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:40.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:40.193+0000 7f9c5bb60700 1 --2- 172.21.15.100:0/3847031346 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c54109ad0 0x7f9c54074cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:40.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:40.193+0000 7f9c5bb60700 1 --2- 172.21.15.100:0/3847031346 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c5410a490 0x7f9c5406dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:40.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:40.193+0000 7f9c5bb60700 1 -- 172.21.15.100:0/3847031346 >> 172.21.15.100:0/3847031346 conn(0x7f9c54076c70 msgr2=0x7f9c54077330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:36:40.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:40.194+0000 7f9c5bb60700 1 -- 172.21.15.100:0/3847031346 shutdown_connections 2024-05-10T11:36:40.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:40.194+0000 7f9c5bb60700 1 -- 172.21.15.100:0/3847031346 wait complete. 2024-05-10T11:36:40.195 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 41 2024-05-10T11:36:40.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:36:40 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/3847031346' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:36:40.931 INFO:tasks.workunit.client.0.smithi100.stdout: 10 46116 9.05 MB/sec execute 154 sec latency 114.468 ms 2024-05-10T11:36:41.000 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-10T11:36:41.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:36:40 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/3847031346' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:36:41.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:36:40 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/3847031346' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:36:41.349 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:36:41.748 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:36:41 smithi100 ceph-mon[30296]: pgmap v970: 129 pgs: 129 active+clean; 466 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 4.0 MiB/s wr, 282 op/s 2024-05-10T11:36:41.931 INFO:tasks.workunit.client.0.smithi100.stdout: 10 46241 9.01 MB/sec execute 155 sec latency 66.821 ms 2024-05-10T11:36:42.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:36:41 smithi178 ceph-mon[33359]: pgmap v970: 129 pgs: 129 active+clean; 466 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 4.0 MiB/s wr, 282 op/s 2024-05-10T11:36:42.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:36:41 smithi188 ceph-mon[34075]: pgmap v970: 129 pgs: 129 active+clean; 466 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 4.0 MiB/s wr, 282 op/s 2024-05-10T11:36:42.145 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.144+0000 7fae14862640 1 -- 172.21.15.100:0/2658265929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae101045c0 msgr2=0x7fae1010ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:42.145 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.144+0000 7fae14862640 1 --2- 172.21.15.100:0/2658265929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae101045c0 0x7fae1010ae50 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fae08009ca0 tx=0x7fae0802a8c0 comp rx=0 tx=0).stop 2024-05-10T11:36:42.145 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.145+0000 7fae14862640 1 -- 172.21.15.100:0/2658265929 shutdown_connections 2024-05-10T11:36:42.145 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.145+0000 7fae14862640 1 --2- 172.21.15.100:0/2658265929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae101045c0 0x7fae1010ae50 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:42.145 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.145+0000 7fae14862640 1 --2- 172.21.15.100:0/2658265929 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae10103c00 0x7fae10104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:42.145 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.145+0000 7fae14862640 1 --2- 172.21.15.100:0/2658265929 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae10102a10 0x7fae10102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:42.146 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.145+0000 7fae14862640 1 -- 172.21.15.100:0/2658265929 >> 172.21.15.100:0/2658265929 conn(0x7fae100fdfa0 msgr2=0x7fae101003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:36:42.146 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.145+0000 7fae14862640 1 -- 172.21.15.100:0/2658265929 shutdown_connections 2024-05-10T11:36:42.146 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.145+0000 7fae14862640 1 -- 172.21.15.100:0/2658265929 wait complete. 2024-05-10T11:36:42.148 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.146+0000 7fae14862640 1 Processor -- start 2024-05-10T11:36:42.148 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.146+0000 7fae14862640 1 -- start start 2024-05-10T11:36:42.148 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.146+0000 7fae14862640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae10102a10 0x7fae1019ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:42.148 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.146+0000 7fae14862640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae10103c00 0x7fae1019b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:42.148 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.146+0000 7fae14862640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae101045c0 0x7fae101a21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:42.148 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.146+0000 7fae14862640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fae10110650 con 0x7fae10102a10 2024-05-10T11:36:42.148 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.146+0000 7fae14862640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fae101104d0 con 0x7fae10103c00 2024-05-10T11:36:42.148 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.146+0000 7fae14862640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fae101107d0 con 0x7fae101045c0 2024-05-10T11:36:42.148 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.146+0000 7fae0e59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae10102a10 0x7fae1019ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:42.148 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.146+0000 7fae0e59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae10102a10 0x7fae1019ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45856/0 (socket says 172.21.15.100:45856) 2024-05-10T11:36:42.148 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.146+0000 7fae0e59c640 1 -- 172.21.15.100:0/2638167320 learned_addr learned my addr 172.21.15.100:0/2638167320 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:36:42.148 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.147+0000 7fae0ed9d640 1 --2- 172.21.15.100:0/2638167320 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae101045c0 0x7fae101a21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:42.149 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.147+0000 7fae07fff640 1 --2- 172.21.15.100:0/2638167320 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae10103c00 0x7fae1019b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:42.149 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.147+0000 7fae0e59c640 1 -- 172.21.15.100:0/2638167320 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae101045c0 msgr2=0x7fae101a21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:42.149 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.147+0000 7fae0e59c640 1 --2- 172.21.15.100:0/2638167320 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae101045c0 0x7fae101a21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:42.149 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.147+0000 7fae0e59c640 1 -- 172.21.15.100:0/2638167320 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae10103c00 msgr2=0x7fae1019b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:42.149 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.147+0000 7fae0e59c640 1 --2- 172.21.15.100:0/2638167320 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae10103c00 0x7fae1019b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:42.149 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.147+0000 7fae0e59c640 1 -- 172.21.15.100:0/2638167320 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae101a2860 con 0x7fae10102a10 2024-05-10T11:36:42.149 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.147+0000 7fae07fff640 1 --2- 172.21.15.100:0/2638167320 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae10103c00 0x7fae1019b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:36:42.149 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.147+0000 7fae0e59c640 1 --2- 172.21.15.100:0/2638167320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae10102a10 0x7fae1019ac50 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fae0000e9c0 tx=0x7fae0000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:36:42.149 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.147+0000 7fae077fe640 1 -- 172.21.15.100:0/2638167320 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae0000cd80 con 0x7fae10102a10 2024-05-10T11:36:42.149 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.147+0000 7fae077fe640 1 -- 172.21.15.100:0/2638167320 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fae000044b0 con 0x7fae10102a10 2024-05-10T11:36:42.149 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.147+0000 7fae14862640 1 -- 172.21.15.100:0/2638167320 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae101a2b50 con 0x7fae10102a10 2024-05-10T11:36:42.149 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.147+0000 7fae077fe640 1 -- 172.21.15.100:0/2638167320 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae00010430 con 0x7fae10102a10 2024-05-10T11:36:42.149 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.148+0000 7fae14862640 1 -- 172.21.15.100:0/2638167320 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae101a30a0 con 0x7fae10102a10 2024-05-10T11:36:42.151 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.151+0000 7fae077fe640 1 -- 172.21.15.100:0/2638167320 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fae00003f00 con 0x7fae10102a10 2024-05-10T11:36:42.151 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.152+0000 7fae17577740 1 Processor -- start 2024-05-10T11:36:42.152 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.152+0000 7fae17577740 1 -- start start 2024-05-10T11:36:42.153 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.152+0000 7fae17577740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577bc17adb0 0x5577bc17b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:42.153 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.152+0000 7fae17577740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5577bc17b7a0 0x5577bc1802a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:42.153 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.152+0000 7fae17577740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5577bc180910 0x5577bc182da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:42.153 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.152+0000 7fae17577740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5577bc0df5d0 con 0x5577bc17adb0 2024-05-10T11:36:42.153 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.152+0000 7fae17577740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5577bc098890 con 0x5577bc180910 2024-05-10T11:36:42.153 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.152+0000 7fae17577740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5577bc0db590 con 0x5577bc17b7a0 2024-05-10T11:36:42.153 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.152+0000 7fae077fe640 1 --2- 172.21.15.100:0/2638167320 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fadfc0b2920 0x7fadfc0b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:42.153 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.152+0000 7fae0ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577bc17adb0 0x5577bc17b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:42.153 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.152+0000 7fae0e59c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5577bc17b7a0 0x5577bc1802a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:42.153 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.152+0000 7fae0ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577bc17adb0 0x5577bc17b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45862/0 (socket says 172.21.15.100:45862) 2024-05-10T11:36:42.153 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.152+0000 7fae0ed9d640 1 -- 172.21.15.100:0/2391583468 learned_addr learned my addr 172.21.15.100:0/2391583468 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:36:42.153 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.152+0000 7fae0ed9d640 1 --2- 172.21.15.100:0/2638167320 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fadfc0b2920 0x7fadfc0b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:42.153 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.152+0000 7fae07fff640 1 --2- 172.21.15.100:0/2391583468 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5577bc180910 0x5577bc182da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:42.154 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.153+0000 7fae077fe640 1 -- 172.21.15.100:0/2638167320 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fae000e2b20 con 0x7fae10102a10 2024-05-10T11:36:42.154 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.153+0000 7fae07fff640 1 -- 172.21.15.100:0/2391583468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5577bc17b7a0 msgr2=0x5577bc1802a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:42.154 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.153+0000 7fae07fff640 1 --2- 172.21.15.100:0/2391583468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5577bc17b7a0 0x5577bc1802a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:42.154 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.153+0000 7fae07fff640 1 -- 172.21.15.100:0/2391583468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577bc17adb0 msgr2=0x5577bc17b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:42.154 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.153+0000 7fae07fff640 1 --2- 172.21.15.100:0/2391583468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577bc17adb0 0x5577bc17b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:42.154 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.153+0000 7fae07fff640 1 -- 172.21.15.100:0/2391583468 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577bbdf1210 con 0x5577bc180910 2024-05-10T11:36:42.157 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.153+0000 7fae07fff640 1 --2- 172.21.15.100:0/2391583468 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5577bc180910 0x5577bc182da0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fadf8009c40 tx=0x7fadf8010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11173f6dd62df2fb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:36:42.157 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.153+0000 7fae0ed9d640 1 --2- 172.21.15.100:0/2638167320 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fadfc0b2920 0x7fadfc0b4de0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fae08009a60 tx=0x7fae080095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:36:42.157 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.153+0000 7fae04ff9640 1 -- 172.21.15.100:0/2391583468 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf8004030 con 0x5577bc180910 2024-05-10T11:36:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.153+0000 7fae04ff9640 1 -- 172.21.15.100:0/2391583468 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fadf80035f0 con 0x5577bc180910 2024-05-10T11:36:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.153+0000 7fae04ff9640 1 -- 172.21.15.100:0/2391583468 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf8012690 con 0x5577bc180910 2024-05-10T11:36:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.154+0000 7fae17577740 1 -- 172.21.15.100:0/2391583468 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5577bc180910 msgr2=0x5577bc182da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.154+0000 7fae17577740 1 --2- 172.21.15.100:0/2391583468 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5577bc180910 0x5577bc182da0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fadf8009c40 tx=0x7fadf8010740 comp rx=0 tx=0).stop 2024-05-10T11:36:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.154+0000 7fae17577740 1 -- 172.21.15.100:0/2391583468 shutdown_connections 2024-05-10T11:36:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.154+0000 7fae17577740 1 --2- 172.21.15.100:0/2391583468 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5577bc180910 0x5577bc182da0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.154+0000 7fae17577740 1 --2- 172.21.15.100:0/2391583468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5577bc17b7a0 0x5577bc1802a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.154+0000 7fae17577740 1 --2- 172.21.15.100:0/2391583468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577bc17adb0 0x5577bc17b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.154+0000 7fae17577740 1 -- 172.21.15.100:0/2391583468 >> 172.21.15.100:0/2391583468 conn(0x5577bc176420 msgr2=0x5577bc176820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:36:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.154+0000 7fae17577740 1 -- 172.21.15.100:0/2391583468 shutdown_connections 2024-05-10T11:36:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.154+0000 7fae17577740 1 -- 172.21.15.100:0/2391583468 wait complete. 2024-05-10T11:36:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.155+0000 7fae17577740 1 -- start start 2024-05-10T11:36:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.155+0000 7fae17577740 1 Processor -- start 2024-05-10T11:36:42.159 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.155+0000 7fae17577740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577bc17adb0 0x5577bc185000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:42.159 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.155+0000 7fae17577740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5577bc17b7a0 0x5577bc185540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:42.159 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.155+0000 7fae17577740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5577bc180910 0x5577bc18c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:42.159 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.155+0000 7fae17577740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5577bc1838c0 con 0x5577bc17adb0 2024-05-10T11:36:42.159 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.155+0000 7fae17577740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5577bc09c370 con 0x5577bc17b7a0 2024-05-10T11:36:42.159 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.155+0000 7fae17577740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5577bc183a40 con 0x5577bc180910 2024-05-10T11:36:42.159 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.155+0000 7fae0e59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577bc17adb0 0x5577bc185000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:42.159 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.155+0000 7fae17577740 10 client.0 _ll_register_callbacks cb 0x5577bbdf0a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:36:42.159 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.155+0000 7fae0e59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577bc17adb0 0x5577bc185000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45870/0 (socket says 172.21.15.100:45870) 2024-05-10T11:36:42.159 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.155+0000 7fae0e59c640 1 -- 172.21.15.100:0/3923862958 learned_addr learned my addr 172.21.15.100:0/3923862958 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:36:42.159 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.155+0000 7fae07fff640 1 --2- 172.21.15.100:0/3923862958 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5577bc17b7a0 0x5577bc185540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:42.159 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.155+0000 7fae0ed9d640 1 --2- 172.21.15.100:0/3923862958 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5577bc180910 0x5577bc18c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:42.159 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.156+0000 7fae0e59c640 1 -- 172.21.15.100:0/3923862958 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5577bc180910 msgr2=0x5577bc18c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:42.159 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.156+0000 7fae0e59c640 1 --2- 172.21.15.100:0/3923862958 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5577bc180910 0x5577bc18c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:42.159 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.156+0000 7fae0e59c640 1 -- 172.21.15.100:0/3923862958 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5577bc17b7a0 msgr2=0x5577bc185540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.156+0000 7fae0e59c640 1 --2- 172.21.15.100:0/3923862958 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5577bc17b7a0 0x5577bc185540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.156+0000 7fae0e59c640 1 -- 172.21.15.100:0/3923862958 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577bbebd620 con 0x5577bc17adb0 2024-05-10T11:36:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.156+0000 7fae0e59c640 1 --2- 172.21.15.100:0/3923862958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577bc17adb0 0x5577bc185000 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fae000a9d10 tx=0x7fae000a9790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21aea8dc9ce64b16 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:36:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.156+0000 7fadf5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:36:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.156+0000 7fadf5ffb640 1 -- 172.21.15.100:0/3923862958 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae000a9d80 con 0x5577bc17adb0 2024-05-10T11:36:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.156+0000 7fadf5ffb640 1 -- 172.21.15.100:0/3923862958 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fae000041f0 con 0x5577bc17adb0 2024-05-10T11:36:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.156+0000 7fae17577740 1 -- 172.21.15.100:0/3923862958 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadf80035f0 con 0x5577bc17adb0 2024-05-10T11:36:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.156+0000 7fadf5ffb640 1 -- 172.21.15.100:0/3923862958 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fae000ae130 con 0x5577bc17adb0 2024-05-10T11:36:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.156+0000 7fae17577740 10 client.19002 fetch_fsmap learned FSMap version 41 2024-05-10T11:36:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.156+0000 7fae17577740 1 -- 172.21.15.100:0/3923862958 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5577bc18f610 con 0x5577bc17adb0 2024-05-10T11:36:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.157+0000 7fadf5ffb640 1 -- 172.21.15.100:0/3923862958 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fadf80035f0 con 0x5577bc17adb0 2024-05-10T11:36:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.157+0000 7fadf5ffb640 1 -- 172.21.15.100:0/3923862958 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fae000b2750 con 0x5577bc17adb0 2024-05-10T11:36:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.157+0000 7fae17577740 10 client.19002 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:36:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.157+0000 7fae17577740 10 client.19002 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:36:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.157+0000 7fae17577740 20 client.19002 populate_metadata read hostname 'smithi100' 2024-05-10T11:36:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.157+0000 7fae17577740 1 --2- 172.21.15.100:0/3923862958 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5577bc190d50 0x5577bc1931e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.157+0000 7fae17577740 4 client.19002 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:36:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.157+0000 7fae17577740 1 -- 172.21.15.100:0/3923862958 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577bc0ed1e0 con 0x5577bc190d50 2024-05-10T11:36:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.157+0000 7fae07fff640 1 --2- 172.21.15.100:0/3923862958 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5577bc190d50 0x5577bc1931e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.158+0000 7fadf5ffb640 10 client.19002 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:36:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.158+0000 7fae07fff640 1 --2- 172.21.15.100:0/3923862958 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5577bc190d50 0x5577bc1931e0 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c82a77dd97bac62 server_cookie=45a56f7cc9e03d86 in_seq=0 out_seq=0 2024-05-10T11:36:42.162 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.159+0000 7fadf5ffb640 1 -- 172.21.15.100:0/3923862958 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5577bc0ed1e0 con 0x5577bc190d50 2024-05-10T11:36:42.162 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.159+0000 7fadf5ffb640 10 client.19002 handle_command_reply: tid=0 2024-05-10T11:36:42.162 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.159+0000 7fae17577740 1 client.19002 shutdown 2024-05-10T11:36:42.162 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.159+0000 7fae17577740 1 -- 172.21.15.100:0/3923862958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577bc17adb0 msgr2=0x5577bc185000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:42.162 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.159+0000 7fae17577740 1 --2- 172.21.15.100:0/3923862958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577bc17adb0 0x5577bc185000 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fae000a9d10 tx=0x7fae000a9790 comp rx=0 tx=0).stop 2024-05-10T11:36:42.162 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.159+0000 7fae17577740 1 -- 172.21.15.100:0/3923862958 shutdown_connections 2024-05-10T11:36:42.163 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.159+0000 7fae17577740 1 --2- 172.21.15.100:0/3923862958 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5577bc190d50 0x5577bc1931e0 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:42.163 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.159+0000 7fae17577740 1 --2- 172.21.15.100:0/3923862958 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5577bc180910 0x5577bc18c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:42.163 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.159+0000 7fae17577740 1 --2- 172.21.15.100:0/3923862958 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5577bc17b7a0 0x5577bc185540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:42.163 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.159+0000 7fadf5ffb640 0 client.19002 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:36:42.163 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.159+0000 7fae17577740 1 --2- 172.21.15.100:0/3923862958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577bc17adb0 0x5577bc185000 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:42.163 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.159+0000 7fae17577740 1 -- 172.21.15.100:0/3923862958 >> 172.21.15.100:0/3923862958 conn(0x5577bc176420 msgr2=0x5577bc175580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:36:42.163 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.159+0000 7fae17577740 1 -- 172.21.15.100:0/3923862958 shutdown_connections 2024-05-10T11:36:42.163 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.159+0000 7fae17577740 1 -- 172.21.15.100:0/3923862958 wait complete. 2024-05-10T11:36:42.163 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.160+0000 7fae17577740 20 client.19002 trim_cache size 0 max 16384 2024-05-10T11:36:42.193 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.193+0000 7fae17577740 1 Processor -- start 2024-05-10T11:36:42.193 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.193+0000 7fae17577740 1 -- start start 2024-05-10T11:36:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.193+0000 7fae17577740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5577bc17adb0 0x5577bc179a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.194+0000 7fae17577740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5577bc17b7a0 0x5577bc184c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.194+0000 7fae0e59c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5577bc17adb0 0x5577bc179a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.194+0000 7fae0e59c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5577bc17adb0 0x5577bc179a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:34560/0 (socket says 172.21.15.100:34560) 2024-05-10T11:36:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.194+0000 7fae17577740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577bc180910 0x5577bc185150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.194+0000 7fae07fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5577bc17b7a0 0x5577bc184c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.194+0000 7fae0e59c640 1 -- 172.21.15.100:0/1205773984 learned_addr learned my addr 172.21.15.100:0/1205773984 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:36:42.195 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.194+0000 7fae17577740 1 -- 172.21.15.100:0/1205773984 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fae101aba40 con 0x5577bc180910 2024-05-10T11:36:42.195 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.194+0000 7fae17577740 1 -- 172.21.15.100:0/1205773984 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fae101a9310 con 0x5577bc17adb0 2024-05-10T11:36:42.195 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.194+0000 7fae17577740 1 -- 172.21.15.100:0/1205773984 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fae101a7160 con 0x5577bc17b7a0 2024-05-10T11:36:42.195 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.194+0000 7fae0ed9d640 1 --2- 172.21.15.100:0/1205773984 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577bc180910 0x5577bc185150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:42.195 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.195+0000 7fae07fff640 1 -- 172.21.15.100:0/1205773984 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5577bc17adb0 msgr2=0x5577bc179a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:42.196 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.195+0000 7fae07fff640 1 --2- 172.21.15.100:0/1205773984 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5577bc17adb0 0x5577bc179a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:42.196 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.195+0000 7fae07fff640 1 -- 172.21.15.100:0/1205773984 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577bc180910 msgr2=0x5577bc185150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:42.196 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.195+0000 7fae07fff640 1 --2- 172.21.15.100:0/1205773984 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577bc180910 0x5577bc185150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:42.196 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.195+0000 7fae07fff640 1 -- 172.21.15.100:0/1205773984 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577bc176250 con 0x5577bc17b7a0 2024-05-10T11:36:42.196 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.195+0000 7fae07fff640 1 --2- 172.21.15.100:0/1205773984 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5577bc17b7a0 0x5577bc184c10 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fadf8009c10 tx=0x7fadf801e040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5efc88f62f1722c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:36:42.196 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.195+0000 7fadf7fff640 1 -- 172.21.15.100:0/1205773984 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf8005070 con 0x5577bc17b7a0 2024-05-10T11:36:42.196 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.195+0000 7fadf7fff640 1 -- 172.21.15.100:0/1205773984 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x5577bc0ed1e0 con 0x5577bc17b7a0 2024-05-10T11:36:42.196 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.195+0000 7fadf7fff640 1 -- 172.21.15.100:0/1205773984 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf8027470 con 0x5577bc17b7a0 2024-05-10T11:36:42.197 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.196+0000 7fae17577740 1 -- 172.21.15.100:0/1205773984 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5577bc17b7a0 msgr2=0x5577bc184c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:42.197 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.196+0000 7fae17577740 1 --2- 172.21.15.100:0/1205773984 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5577bc17b7a0 0x5577bc184c10 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fadf8009c10 tx=0x7fadf801e040 comp rx=0 tx=0).stop 2024-05-10T11:36:42.197 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.197+0000 7fae17577740 1 -- 172.21.15.100:0/1205773984 shutdown_connections 2024-05-10T11:36:42.197 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.197+0000 7fae17577740 1 --2- 172.21.15.100:0/1205773984 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577bc180910 0x5577bc185150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:42.197 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.197+0000 7fae17577740 1 --2- 172.21.15.100:0/1205773984 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5577bc17b7a0 0x5577bc184c10 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:42.197 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.197+0000 7fae17577740 1 --2- 172.21.15.100:0/1205773984 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5577bc17adb0 0x5577bc179a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:42.197 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.197+0000 7fae17577740 1 -- 172.21.15.100:0/1205773984 >> 172.21.15.100:0/1205773984 conn(0x5577bc176420 msgr2=0x5577bc179330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:36:42.198 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.197+0000 7fae17577740 1 -- 172.21.15.100:0/1205773984 shutdown_connections 2024-05-10T11:36:42.198 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.197+0000 7fae17577740 1 -- 172.21.15.100:0/1205773984 wait complete. 2024-05-10T11:36:42.198 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.198+0000 7fae17577740 1 -- start start 2024-05-10T11:36:42.200 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.198+0000 7fae17577740 1 Processor -- start 2024-05-10T11:36:42.200 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.199+0000 7fae17577740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5577bc17adb0 0x5577bc189650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:42.200 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.199+0000 7fae17577740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5577bc17b7a0 0x5577bc189b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:42.200 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.199+0000 7fae17577740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577bc180910 0x5577bc18ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:42.200 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.199+0000 7fae17577740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fae101a77d0 con 0x5577bc180910 2024-05-10T11:36:42.200 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.199+0000 7fae17577740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fae101a8c40 con 0x5577bc17adb0 2024-05-10T11:36:42.200 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.199+0000 7fae17577740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fae101a7ea0 con 0x5577bc17b7a0 2024-05-10T11:36:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.199+0000 7fae17577740 10 client.0 _ll_register_callbacks cb 0x7fae00009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:36:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.199+0000 7fae0ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577bc180910 0x5577bc18ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.199+0000 7fae0ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577bc180910 0x5577bc18ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45882/0 (socket says 172.21.15.100:45882) 2024-05-10T11:36:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.199+0000 7fae07fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5577bc17b7a0 0x5577bc189b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.199+0000 7fae0ed9d640 1 -- 172.21.15.100:0/4121243436 learned_addr learned my addr 172.21.15.100:0/4121243436 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:36:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.199+0000 7fae0e59c640 1 --2- 172.21.15.100:0/4121243436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5577bc17adb0 0x5577bc189650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.199+0000 7fae0ed9d640 1 -- 172.21.15.100:0/4121243436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5577bc17b7a0 msgr2=0x5577bc189b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.199+0000 7fae0ed9d640 1 --2- 172.21.15.100:0/4121243436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5577bc17b7a0 0x5577bc189b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.199+0000 7fae0ed9d640 1 -- 172.21.15.100:0/4121243436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5577bc17adb0 msgr2=0x5577bc189650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.199+0000 7fae0ed9d640 1 --2- 172.21.15.100:0/4121243436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5577bc17adb0 0x5577bc189650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.199+0000 7fae0ed9d640 1 -- 172.21.15.100:0/4121243436 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577bc18b140 con 0x5577bc180910 2024-05-10T11:36:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.199+0000 7fae07fff640 1 --2- 172.21.15.100:0/4121243436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5577bc17b7a0 0x5577bc189b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:36:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.200+0000 7fae0ed9d640 1 --2- 172.21.15.100:0/4121243436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577bc180910 0x5577bc18ab50 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fae08009ca0 tx=0x7fae0802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb13b5d809c3dbb4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:36:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.200+0000 7fadf5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:36:42.202 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.200+0000 7fadf5ffb640 1 -- 172.21.15.100:0/4121243436 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae080047a0 con 0x5577bc180910 2024-05-10T11:36:42.202 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.200+0000 7fae17577740 1 -- 172.21.15.100:0/4121243436 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5577bc0ed1e0 con 0x5577bc180910 2024-05-10T11:36:42.202 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.200+0000 7fadf5ffb640 1 -- 172.21.15.100:0/4121243436 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fae08032e00 con 0x5577bc180910 2024-05-10T11:36:42.202 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.200+0000 7fadf5ffb640 1 -- 172.21.15.100:0/4121243436 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fae08061050 con 0x5577bc180910 2024-05-10T11:36:42.202 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.200+0000 7fae17577740 10 client.19008 fetch_fsmap learned FSMap version 41 2024-05-10T11:36:42.202 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.200+0000 7fae17577740 1 -- 172.21.15.100:0/4121243436 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5577bc1fba10 con 0x5577bc180910 2024-05-10T11:36:42.203 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.201+0000 7fadf5ffb640 1 -- 172.21.15.100:0/4121243436 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x5577bc0ed1e0 con 0x5577bc180910 2024-05-10T11:36:42.203 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.201+0000 7fadf5ffb640 1 -- 172.21.15.100:0/4121243436 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fae08056020 con 0x5577bc180910 2024-05-10T11:36:42.203 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.201+0000 7fae17577740 10 client.19008 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:36:42.203 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.201+0000 7fae17577740 10 client.19008 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:36:42.203 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.201+0000 7fae17577740 20 client.19008 populate_metadata read hostname 'smithi100' 2024-05-10T11:36:42.203 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.201+0000 7fae17577740 1 --2- 172.21.15.100:0/4121243436 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5577bc1fc3f0 0x5577bc1fe810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:42.203 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.201+0000 7fae17577740 4 client.19008 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:36:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.201+0000 7fae17577740 1 -- 172.21.15.100:0/4121243436 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5577bc0e7050 con 0x5577bc1fc3f0 2024-05-10T11:36:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.201+0000 7fae0e59c640 1 --2- 172.21.15.100:0/4121243436 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5577bc1fc3f0 0x5577bc1fe810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.201+0000 7fadf5ffb640 10 client.19008 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:36:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.202+0000 7fae0e59c640 1 --2- 172.21.15.100:0/4121243436 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5577bc1fc3f0 0x5577bc1fe810 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=464bc9969a1197b5 server_cookie=c8320464f6b85209 in_seq=0 out_seq=0 2024-05-10T11:36:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.329+0000 7fadf5ffb640 1 -- 172.21.15.100:0/4121243436 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5577bc0e7050 con 0x5577bc1fc3f0 2024-05-10T11:36:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.329+0000 7fadf5ffb640 10 client.19008 handle_command_reply: tid=0 2024-05-10T11:36:42.331 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.329+0000 7fae17577740 1 client.19008 shutdown 2024-05-10T11:36:42.331 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.330+0000 7fae17577740 1 -- 172.21.15.100:0/4121243436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577bc180910 msgr2=0x5577bc18ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:42.331 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.330+0000 7fae17577740 1 --2- 172.21.15.100:0/4121243436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577bc180910 0x5577bc18ab50 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fae08009ca0 tx=0x7fae0802a8c0 comp rx=0 tx=0).stop 2024-05-10T11:36:42.331 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.330+0000 7fae17577740 1 -- 172.21.15.100:0/4121243436 shutdown_connections 2024-05-10T11:36:42.331 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.330+0000 7fae17577740 1 --2- 172.21.15.100:0/4121243436 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5577bc1fc3f0 0x5577bc1fe810 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:42.331 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.330+0000 7fae17577740 1 --2- 172.21.15.100:0/4121243436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577bc180910 0x5577bc18ab50 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:42.331 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.330+0000 7fae17577740 1 --2- 172.21.15.100:0/4121243436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5577bc17b7a0 0x5577bc189b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:42.331 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.330+0000 7fae17577740 1 --2- 172.21.15.100:0/4121243436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5577bc17adb0 0x5577bc189650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:42.331 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.330+0000 7fae17577740 1 -- 172.21.15.100:0/4121243436 >> 172.21.15.100:0/4121243436 conn(0x5577bc176420 msgr2=0x5577bc184c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:36:42.331 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.330+0000 7fadf5ffb640 0 client.19008 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:36:42.331 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.330+0000 7fae17577740 1 -- 172.21.15.100:0/4121243436 shutdown_connections 2024-05-10T11:36:42.331 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.330+0000 7fae17577740 1 -- 172.21.15.100:0/4121243436 wait complete. 2024-05-10T11:36:42.331 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.330+0000 7fae17577740 20 client.19008 trim_cache size 0 max 16384 2024-05-10T11:36:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:36:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:36:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:36:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:36:42.333 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.331+0000 7fae14862640 1 -- 172.21.15.100:0/2638167320 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fadfc0b2920 msgr2=0x7fadfc0b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:42.333 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.331+0000 7fae14862640 1 --2- 172.21.15.100:0/2638167320 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fadfc0b2920 0x7fadfc0b4de0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fae08009a60 tx=0x7fae080095c0 comp rx=0 tx=0).stop 2024-05-10T11:36:42.333 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.331+0000 7fae14862640 1 -- 172.21.15.100:0/2638167320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae10102a10 msgr2=0x7fae1019ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:42.333 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.331+0000 7fae14862640 1 --2- 172.21.15.100:0/2638167320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae10102a10 0x7fae1019ac50 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fae0000e9c0 tx=0x7fae0000ee80 comp rx=0 tx=0).stop 2024-05-10T11:36:42.333 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.331+0000 7fae14862640 1 -- 172.21.15.100:0/2638167320 shutdown_connections 2024-05-10T11:36:42.333 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.331+0000 7fae14862640 1 --2- 172.21.15.100:0/2638167320 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fadfc0b2920 0x7fadfc0b4de0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:42.333 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.331+0000 7fae14862640 1 --2- 172.21.15.100:0/2638167320 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae101045c0 0x7fae101a21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:42.333 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.331+0000 7fae14862640 1 --2- 172.21.15.100:0/2638167320 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae10103c00 0x7fae1019b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:42.333 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.331+0000 7fae14862640 1 --2- 172.21.15.100:0/2638167320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae10102a10 0x7fae1019ac50 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:42.333 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.331+0000 7fae14862640 1 -- 172.21.15.100:0/2638167320 >> 172.21.15.100:0/2638167320 conn(0x7fae100fdfa0 msgr2=0x7fae10106f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:36:42.334 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.331+0000 7fae14862640 1 -- 172.21.15.100:0/2638167320 shutdown_connections 2024-05-10T11:36:42.334 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:42.331+0000 7fae14862640 1 -- 172.21.15.100:0/2638167320 wait complete. 2024-05-10T11:36:42.931 INFO:tasks.workunit.client.0.smithi100.stdout: 10 46426 9.03 MB/sec execute 156 sec latency 119.820 ms 2024-05-10T11:36:43.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.479+0000 7fabca213700 1 -- 172.21.15.100:0/1694260755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabc406bb40 msgr2=0x7fabc4110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:43.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.479+0000 7fabca213700 1 --2- 172.21.15.100:0/1694260755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabc406bb40 0x7fabc4110330 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fabbc00b0a0 tx=0x7fabbc00b3b0 comp rx=0 tx=0).stop 2024-05-10T11:36:43.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.479+0000 7fabca213700 1 -- 172.21.15.100:0/1694260755 shutdown_connections 2024-05-10T11:36:43.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.479+0000 7fabca213700 1 --2- 172.21.15.100:0/1694260755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabc406bb40 0x7fabc4110330 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:43.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.479+0000 7fabca213700 1 --2- 172.21.15.100:0/1694260755 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fabc4109510 0x7fabc41098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:43.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.479+0000 7fabca213700 1 --2- 172.21.15.100:0/1694260755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabc406b180 0x7fabc406b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:43.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.479+0000 7fabca213700 1 -- 172.21.15.100:0/1694260755 >> 172.21.15.100:0/1694260755 conn(0x7fabc4079350 msgr2=0x7fabc4079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:36:43.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.480+0000 7fabca213700 1 -- 172.21.15.100:0/1694260755 shutdown_connections 2024-05-10T11:36:43.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.480+0000 7fabca213700 1 -- 172.21.15.100:0/1694260755 wait complete. 2024-05-10T11:36:43.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.482+0000 7fabca213700 1 Processor -- start 2024-05-10T11:36:43.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.482+0000 7fabca213700 1 -- start start 2024-05-10T11:36:43.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.482+0000 7fabca213700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fabc406b180 0x7fabc4102470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:43.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.482+0000 7fabca213700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabc406bb40 0x7fabc41029b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:43.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.483+0000 7fabca213700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabc4109510 0x7fabc4102ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:43.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.483+0000 7fabca213700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fabc4113520 con 0x7fabc4109510 2024-05-10T11:36:43.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.483+0000 7fabca213700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fabc41133a0 con 0x7fabc406b180 2024-05-10T11:36:43.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.483+0000 7fabca213700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fabc41136a0 con 0x7fabc406bb40 2024-05-10T11:36:43.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.483+0000 7fabc37fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fabc406b180 0x7fabc4102470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:43.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.483+0000 7fabc2ffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabc406bb40 0x7fabc41029b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:43.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.483+0000 7fabc3fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabc4109510 0x7fabc4102ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:43.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.483+0000 7fabc3fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabc4109510 0x7fabc4102ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45902/0 (socket says 172.21.15.100:45902) 2024-05-10T11:36:43.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.483+0000 7fabc2ffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabc406bb40 0x7fabc41029b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:41868/0 (socket says 172.21.15.100:41868) 2024-05-10T11:36:43.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.483+0000 7fabc3fff700 1 -- 172.21.15.100:0/708874525 learned_addr learned my addr 172.21.15.100:0/708874525 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:36:43.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.483+0000 7fabc3fff700 1 -- 172.21.15.100:0/708874525 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabc406bb40 msgr2=0x7fabc41029b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:43.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.483+0000 7fabc3fff700 1 --2- 172.21.15.100:0/708874525 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabc406bb40 0x7fabc41029b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:43.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.483+0000 7fabc3fff700 1 -- 172.21.15.100:0/708874525 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fabc406b180 msgr2=0x7fabc4102470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:43.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.483+0000 7fabc3fff700 1 --2- 172.21.15.100:0/708874525 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fabc406b180 0x7fabc4102470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:43.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.483+0000 7fabc3fff700 1 -- 172.21.15.100:0/708874525 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabc41a7e40 con 0x7fabc4109510 2024-05-10T11:36:43.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.483+0000 7fabc3fff700 1 --2- 172.21.15.100:0/708874525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabc4109510 0x7fabc4102ef0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fabbc009fd0 tx=0x7fabbc0095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:36:43.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.484+0000 7fabc0ff9700 1 -- 172.21.15.100:0/708874525 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabbc021070 con 0x7fabc4109510 2024-05-10T11:36:43.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.484+0000 7fabc0ff9700 1 -- 172.21.15.100:0/708874525 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fabbc007b80 con 0x7fabc4109510 2024-05-10T11:36:43.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.484+0000 7fabca213700 1 -- 172.21.15.100:0/708874525 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabc41a80d0 con 0x7fabc4109510 2024-05-10T11:36:43.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.484+0000 7fabc0ff9700 1 -- 172.21.15.100:0/708874525 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabbc004770 con 0x7fabc4109510 2024-05-10T11:36:43.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.484+0000 7fabca213700 1 -- 172.21.15.100:0/708874525 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabc41a84d0 con 0x7fabc4109510 2024-05-10T11:36:43.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.489+0000 7fabc0ff9700 1 -- 172.21.15.100:0/708874525 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fabbc01d450 con 0x7fabc4109510 2024-05-10T11:36:43.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.491+0000 7fabc0ff9700 1 --2- 172.21.15.100:0/708874525 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fabac0b2900 0x7fabac0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:43.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.492+0000 7fabc0ff9700 1 -- 172.21.15.100:0/708874525 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fabbc0e3690 con 0x7fabc4109510 2024-05-10T11:36:43.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.492+0000 7fabc37fe700 1 --2- 172.21.15.100:0/708874525 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fabac0b2900 0x7fabac0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:43.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.492+0000 7fabd4aa6b80 1 Processor -- start 2024-05-10T11:36:43.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.492+0000 7fabd4aa6b80 1 -- start start 2024-05-10T11:36:43.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.492+0000 7fabd4aa6b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e50071c1b0 0x55e50071c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:43.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.493+0000 7fabd4aa6b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e50071cad0 0x55e500716810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:43.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.493+0000 7fabd4aa6b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e500716d50 0x55e500717190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:43.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.493+0000 7fabd4aa6b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55e500668940 con 0x55e500716d50 2024-05-10T11:36:43.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.493+0000 7fabd4aa6b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e5005cdda0 con 0x55e50071c1b0 2024-05-10T11:36:43.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.493+0000 7fabd4aa6b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55e500664900 con 0x55e50071cad0 2024-05-10T11:36:43.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.493+0000 7fabc37fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e50071c1b0 0x55e50071c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:43.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.493+0000 7fabc37fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e50071c1b0 0x55e50071c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:34606/0 (socket says 172.21.15.100:34606) 2024-05-10T11:36:43.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.493+0000 7fabc37fe700 1 -- 172.21.15.100:0/1842600691 learned_addr learned my addr 172.21.15.100:0/1842600691 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:36:43.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.493+0000 7fabc2ffd700 1 --2- 172.21.15.100:0/1842600691 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e50071cad0 0x55e500716810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:43.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.493+0000 7fabc37fe700 1 --2- 172.21.15.100:0/708874525 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fabac0b2900 0x7fabac0b4dc0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fabb4009dc0 tx=0x7fabb4011720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:36:43.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.493+0000 7fabc37fe700 1 -- 172.21.15.100:0/1842600691 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e50071cad0 msgr2=0x55e500716810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:43.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.493+0000 7fabc37fe700 1 --2- 172.21.15.100:0/1842600691 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e50071cad0 0x55e500716810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:43.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.493+0000 7fabc37fe700 1 -- 172.21.15.100:0/1842600691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e500716d50 msgr2=0x55e500717190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:43.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.493+0000 7fabc37fe700 1 --2- 172.21.15.100:0/1842600691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e500716d50 0x55e500717190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:43.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.493+0000 7fabc37fe700 1 -- 172.21.15.100:0/1842600691 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5005b58d0 con 0x55e50071c1b0 2024-05-10T11:36:43.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.494+0000 7fabc37fe700 1 --2- 172.21.15.100:0/1842600691 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e50071c1b0 0x55e50071c590 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fabb401e560 tx=0x7fabb401d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ff14c2c55bb7ed7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:36:43.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.494+0000 7faba9ffb700 1 -- 172.21.15.100:0/1842600691 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb401d680 con 0x55e50071c1b0 2024-05-10T11:36:43.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.494+0000 7faba9ffb700 1 -- 172.21.15.100:0/1842600691 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fabb401dcb0 con 0x55e50071c1b0 2024-05-10T11:36:43.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.494+0000 7faba9ffb700 1 -- 172.21.15.100:0/1842600691 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb400fce0 con 0x55e50071c1b0 2024-05-10T11:36:43.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.496+0000 7fabd4aa6b80 1 -- 172.21.15.100:0/1842600691 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e50071c1b0 msgr2=0x55e50071c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:43.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.496+0000 7fabd4aa6b80 1 --2- 172.21.15.100:0/1842600691 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e50071c1b0 0x55e50071c590 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fabb401e560 tx=0x7fabb401d000 comp rx=0 tx=0).stop 2024-05-10T11:36:43.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.496+0000 7fabd4aa6b80 1 -- 172.21.15.100:0/1842600691 shutdown_connections 2024-05-10T11:36:43.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.496+0000 7fabd4aa6b80 1 --2- 172.21.15.100:0/1842600691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e500716d50 0x55e500717190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:43.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.496+0000 7fabd4aa6b80 1 --2- 172.21.15.100:0/1842600691 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e50071c1b0 0x55e50071c590 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:43.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.496+0000 7fabd4aa6b80 1 --2- 172.21.15.100:0/1842600691 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e50071cad0 0x55e500716810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:43.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.496+0000 7fabd4aa6b80 1 -- 172.21.15.100:0/1842600691 >> 172.21.15.100:0/1842600691 conn(0x55e500642b70 msgr2=0x55e5007112a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:36:43.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.496+0000 7fabd4aa6b80 1 -- 172.21.15.100:0/1842600691 shutdown_connections 2024-05-10T11:36:43.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.496+0000 7fabd4aa6b80 1 -- 172.21.15.100:0/1842600691 wait complete. 2024-05-10T11:36:43.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.497+0000 7fabd4aa6b80 1 -- start start 2024-05-10T11:36:43.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.497+0000 7fabd4aa6b80 1 Processor -- start 2024-05-10T11:36:43.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.498+0000 7fabd4aa6b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e500716d50 0x55e5007227d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:43.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.498+0000 7fabd4aa6b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e50071c1b0 0x55e50071fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:43.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.498+0000 7fabd4aa6b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e50071cad0 0x55e500720100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:43.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.498+0000 7fabd4aa6b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55e50072bc00 con 0x55e50071cad0 2024-05-10T11:36:43.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.498+0000 7fabd4aa6b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e500621ed0 con 0x55e500716d50 2024-05-10T11:36:43.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.498+0000 7fabd4aa6b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55e50072bd80 con 0x55e50071c1b0 2024-05-10T11:36:43.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.498+0000 7fabc37fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e500716d50 0x55e5007227d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:43.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.498+0000 7fabd4aa6b80 10 client.0 _ll_register_callbacks cb 0x55e5005fc5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:36:43.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.498+0000 7fabc3fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e50071cad0 0x55e500720100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:43.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.498+0000 7fabc2ffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e50071c1b0 0x55e50071fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:43.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.498+0000 7fabc37fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e500716d50 0x55e5007227d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:34610/0 (socket says 172.21.15.100:34610) 2024-05-10T11:36:43.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.498+0000 7fabc37fe700 1 -- 172.21.15.100:0/571690067 learned_addr learned my addr 172.21.15.100:0/571690067 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:36:43.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.499+0000 7fabc3fff700 1 -- 172.21.15.100:0/571690067 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e50071c1b0 msgr2=0x55e50071fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:43.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.499+0000 7fabc3fff700 1 --2- 172.21.15.100:0/571690067 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e50071c1b0 0x55e50071fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:43.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.499+0000 7fabc3fff700 1 -- 172.21.15.100:0/571690067 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e500716d50 msgr2=0x55e5007227d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:43.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.499+0000 7fabc3fff700 1 --2- 172.21.15.100:0/571690067 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e500716d50 0x55e5007227d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:43.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.499+0000 7fabc3fff700 1 -- 172.21.15.100:0/571690067 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e50061f760 con 0x55e50071cad0 2024-05-10T11:36:43.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.499+0000 7fabc37fe700 1 --2- 172.21.15.100:0/571690067 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e500716d50 0x55e5007227d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:36:43.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.499+0000 7fabc3fff700 1 --2- 172.21.15.100:0/571690067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e50071cad0 0x55e500720100 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fabbc0aa680 tx=0x7fabbc0aa2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c9367f790d10930 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:36:43.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.499+0000 7faba2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:36:43.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.499+0000 7faba2ffd700 1 -- 172.21.15.100:0/571690067 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabbc02a030 con 0x55e50071cad0 2024-05-10T11:36:43.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.499+0000 7faba2ffd700 1 -- 172.21.15.100:0/571690067 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fabbc007d30 con 0x55e50071cad0 2024-05-10T11:36:43.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.499+0000 7fabd4aa6b80 1 -- 172.21.15.100:0/571690067 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabb401dcb0 con 0x55e50071cad0 2024-05-10T11:36:43.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.499+0000 7faba2ffd700 1 -- 172.21.15.100:0/571690067 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fabbc0a6110 con 0x55e50071cad0 2024-05-10T11:36:43.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.499+0000 7fabd4aa6b80 10 client.19017 fetch_fsmap learned FSMap version 41 2024-05-10T11:36:43.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.499+0000 7fabd4aa6b80 1 -- 172.21.15.100:0/571690067 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55e500736370 con 0x55e50071cad0 2024-05-10T11:36:43.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.500+0000 7faba2ffd700 1 -- 172.21.15.100:0/571690067 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fabb401dcb0 con 0x55e50071cad0 2024-05-10T11:36:43.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.500+0000 7faba2ffd700 1 -- 172.21.15.100:0/571690067 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fabbc011070 con 0x55e50071cad0 2024-05-10T11:36:43.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.500+0000 7fabd4aa6b80 10 client.19017 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:36:43.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.500+0000 7fabd4aa6b80 10 client.19017 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-10T11:36:43.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.500+0000 7fabd4aa6b80 20 client.19017 populate_metadata read hostname 'smithi100' 2024-05-10T11:36:43.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.500+0000 7fabd4aa6b80 1 --2- 172.21.15.100:0/571690067 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55e500738890 0x55e50073ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:43.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.500+0000 7fabd4aa6b80 4 client.19017 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:36:43.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.500+0000 7fabd4aa6b80 1 -- 172.21.15.100:0/571690067 --> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e500676320 con 0x55e500738890 2024-05-10T11:36:43.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.501+0000 7fabc37fe700 1 --2- 172.21.15.100:0/571690067 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55e500738890 0x55e50073ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:43.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.501+0000 7faba2ffd700 10 client.19017 ms_handle_connect on v2:172.21.15.188:6836/1943342620 2024-05-10T11:36:43.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.501+0000 7fabc37fe700 1 --2- 172.21.15.100:0/571690067 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55e500738890 0x55e50073ac80 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bcf29c71c4325df5 server_cookie=3c83210797cf65af in_seq=0 out_seq=0 2024-05-10T11:36:43.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.503+0000 7faba2ffd700 1 -- 172.21.15.100:0/571690067 <== mds.2 v2:172.21.15.188:6836/1943342620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e500676320 con 0x55e500738890 2024-05-10T11:36:43.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.503+0000 7faba2ffd700 10 client.19017 handle_command_reply: tid=0 2024-05-10T11:36:43.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.503+0000 7fabd4aa6b80 1 client.19017 shutdown 2024-05-10T11:36:43.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.503+0000 7fabd4aa6b80 1 -- 172.21.15.100:0/571690067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e50071cad0 msgr2=0x55e500720100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:43.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.503+0000 7fabd4aa6b80 1 --2- 172.21.15.100:0/571690067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e50071cad0 0x55e500720100 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fabbc0aa680 tx=0x7fabbc0aa2d0 comp rx=0 tx=0).stop 2024-05-10T11:36:43.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.503+0000 7fabd4aa6b80 1 -- 172.21.15.100:0/571690067 shutdown_connections 2024-05-10T11:36:43.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.503+0000 7fabd4aa6b80 1 --2- 172.21.15.100:0/571690067 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55e500738890 0x55e50073ac80 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:43.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.503+0000 7faba2ffd700 0 client.19017 ms_handle_reset on v2:172.21.15.188:6836/1943342620 2024-05-10T11:36:43.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.503+0000 7fabd4aa6b80 1 --2- 172.21.15.100:0/571690067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e50071cad0 0x55e500720100 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:43.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.503+0000 7fabd4aa6b80 1 --2- 172.21.15.100:0/571690067 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e500716d50 0x55e5007227d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:43.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.503+0000 7fabd4aa6b80 1 --2- 172.21.15.100:0/571690067 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e50071c1b0 0x55e50071fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:43.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.503+0000 7fabd4aa6b80 1 -- 172.21.15.100:0/571690067 >> 172.21.15.100:0/571690067 conn(0x55e500642b70 msgr2=0x55e50070bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:36:43.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.504+0000 7fabd4aa6b80 1 -- 172.21.15.100:0/571690067 shutdown_connections 2024-05-10T11:36:43.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.504+0000 7fabd4aa6b80 1 -- 172.21.15.100:0/571690067 wait complete. 2024-05-10T11:36:43.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.505+0000 7fabd4aa6b80 20 client.19017 trim_cache size 0 max 16384 2024-05-10T11:36:43.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.530+0000 7fabd4aa6b80 1 Processor -- start 2024-05-10T11:36:43.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.530+0000 7fabd4aa6b80 1 -- start start 2024-05-10T11:36:43.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.531+0000 7fabd4aa6b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e500716d50 0x55e50071a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:43.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.531+0000 7fabd4aa6b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e50071c1b0 0x55e500734cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:43.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.531+0000 7fabd4aa6b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e50071cad0 0x55e50070bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:43.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.531+0000 7fabd4aa6b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fabc41af400 con 0x55e500716d50 2024-05-10T11:36:43.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.531+0000 7fabd4aa6b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fabc41acc90 con 0x55e50071cad0 2024-05-10T11:36:43.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.531+0000 7fabd4aa6b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fabc41aabb0 con 0x55e50071c1b0 2024-05-10T11:36:43.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.531+0000 7fabc37fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e500716d50 0x55e50071a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:43.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.531+0000 7fabc37fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e500716d50 0x55e50071a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45926/0 (socket says 172.21.15.100:45926) 2024-05-10T11:36:43.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.531+0000 7fabc37fe700 1 -- 172.21.15.100:0/1535841473 learned_addr learned my addr 172.21.15.100:0/1535841473 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:36:43.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.531+0000 7fabc2ffd700 1 --2- 172.21.15.100:0/1535841473 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e50071c1b0 0x55e500734cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:43.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.531+0000 7fabc3fff700 1 --2- 172.21.15.100:0/1535841473 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e50071cad0 0x55e50070bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:43.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.531+0000 7fabc37fe700 1 -- 172.21.15.100:0/1535841473 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e50071c1b0 msgr2=0x55e500734cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:43.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.531+0000 7fabc37fe700 1 --2- 172.21.15.100:0/1535841473 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e50071c1b0 0x55e500734cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:43.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.531+0000 7fabc37fe700 1 -- 172.21.15.100:0/1535841473 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e50071cad0 msgr2=0x55e50070bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:43.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.531+0000 7fabc37fe700 1 --2- 172.21.15.100:0/1535841473 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e50071cad0 0x55e50070bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:43.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.531+0000 7fabc37fe700 1 -- 172.21.15.100:0/1535841473 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e500735200 con 0x55e500716d50 2024-05-10T11:36:43.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.531+0000 7fabc37fe700 1 --2- 172.21.15.100:0/1535841473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e500716d50 0x55e50071a2c0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fabb4011bd0 tx=0x7fabb4029040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72e9e8ba69b67387 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:36:43.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.532+0000 7faba97fa700 1 -- 172.21.15.100:0/1535841473 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb4016070 con 0x55e500716d50 2024-05-10T11:36:43.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.532+0000 7faba97fa700 1 -- 172.21.15.100:0/1535841473 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fabb400f640 con 0x55e500716d50 2024-05-10T11:36:43.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.532+0000 7faba97fa700 1 -- 172.21.15.100:0/1535841473 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb4003680 con 0x55e500716d50 2024-05-10T11:36:43.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.532+0000 7fabd4aa6b80 1 -- 172.21.15.100:0/1535841473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e500716d50 msgr2=0x55e50071a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:43.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.532+0000 7fabd4aa6b80 1 --2- 172.21.15.100:0/1535841473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e500716d50 0x55e50071a2c0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fabb4011bd0 tx=0x7fabb4029040 comp rx=0 tx=0).stop 2024-05-10T11:36:43.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.533+0000 7fabd4aa6b80 1 -- 172.21.15.100:0/1535841473 shutdown_connections 2024-05-10T11:36:43.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.533+0000 7fabd4aa6b80 1 --2- 172.21.15.100:0/1535841473 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e50071cad0 0x55e50070bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:43.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.533+0000 7fabd4aa6b80 1 --2- 172.21.15.100:0/1535841473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e500716d50 0x55e50071a2c0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:43.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.533+0000 7fabd4aa6b80 1 --2- 172.21.15.100:0/1535841473 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e50071c1b0 0x55e500734cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:43.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.533+0000 7fabd4aa6b80 1 -- 172.21.15.100:0/1535841473 >> 172.21.15.100:0/1535841473 conn(0x55e500642b70 msgr2=0x55e50071cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:36:43.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.533+0000 7fabd4aa6b80 1 -- 172.21.15.100:0/1535841473 shutdown_connections 2024-05-10T11:36:43.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.533+0000 7fabd4aa6b80 1 -- 172.21.15.100:0/1535841473 wait complete. 2024-05-10T11:36:43.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.533+0000 7fabd4aa6b80 1 -- start start 2024-05-10T11:36:43.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.533+0000 7fabd4aa6b80 1 Processor -- start 2024-05-10T11:36:43.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.534+0000 7fabd4aa6b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e500716d50 0x55e500731290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:43.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.534+0000 7fabd4aa6b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e50071c1b0 0x55e50073f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:43.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.534+0000 7fabd4aa6b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e50071cad0 0x55e50073fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:43.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.534+0000 7fabd4aa6b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fabc41ab150 con 0x55e50071cad0 2024-05-10T11:36:43.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.534+0000 7fabd4aa6b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fabc41ac5c0 con 0x55e50071c1b0 2024-05-10T11:36:43.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.534+0000 7fabd4aa6b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fabc41ab820 con 0x55e500716d50 2024-05-10T11:36:43.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.534+0000 7fabc37fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e500716d50 0x55e500731290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:43.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.534+0000 7fabc37fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e500716d50 0x55e500731290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:41896/0 (socket says 172.21.15.100:41896) 2024-05-10T11:36:43.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.534+0000 7fabc37fe700 1 -- 172.21.15.100:0/1530253036 learned_addr learned my addr 172.21.15.100:0/1530253036 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:36:43.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.534+0000 7fabc2ffd700 1 --2- 172.21.15.100:0/1530253036 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e50071c1b0 0x55e50073f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:43.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.534+0000 7fabd4aa6b80 10 client.0 _ll_register_callbacks cb 0x55e5005fc5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:36:43.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.534+0000 7fabc3fff700 1 --2- 172.21.15.100:0/1530253036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e50071cad0 0x55e50073fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:43.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.534+0000 7fabc37fe700 1 -- 172.21.15.100:0/1530253036 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e50071c1b0 msgr2=0x55e50073f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:43.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.534+0000 7fabc37fe700 1 --2- 172.21.15.100:0/1530253036 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e50071c1b0 0x55e50073f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:43.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.534+0000 7fabc37fe700 1 -- 172.21.15.100:0/1530253036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e50071cad0 msgr2=0x55e50073fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:43.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.534+0000 7fabc37fe700 1 --2- 172.21.15.100:0/1530253036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e50071cad0 0x55e50073fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:43.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.534+0000 7fabc37fe700 1 -- 172.21.15.100:0/1530253036 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e500740180 con 0x55e500716d50 2024-05-10T11:36:43.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.534+0000 7fabc3fff700 1 --2- 172.21.15.100:0/1530253036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e50071cad0 0x55e50073fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:36:43.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.534+0000 7fabc2ffd700 1 --2- 172.21.15.100:0/1530253036 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e50071c1b0 0x55e50073f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:36:43.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.534+0000 7fabc37fe700 1 --2- 172.21.15.100:0/1530253036 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e500716d50 0x55e500731290 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fabb400f810 tx=0x7fabb400fe00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=708617ef2503f3c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:36:43.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.534+0000 7faba2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:36:43.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.535+0000 7faba2ffd700 1 -- 172.21.15.100:0/1530253036 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb4016040 con 0x55e500716d50 2024-05-10T11:36:43.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.535+0000 7faba2ffd700 1 -- 172.21.15.100:0/1530253036 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fabb4005260 con 0x55e500716d50 2024-05-10T11:36:43.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.535+0000 7fabd4aa6b80 1 -- 172.21.15.100:0/1530253036 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabb400f640 con 0x55e500716d50 2024-05-10T11:36:43.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.535+0000 7faba2ffd700 1 -- 172.21.15.100:0/1530253036 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fabb4005460 con 0x55e500716d50 2024-05-10T11:36:43.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.535+0000 7fabd4aa6b80 10 client.28429 fetch_fsmap learned FSMap version 41 2024-05-10T11:36:43.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.535+0000 7fabd4aa6b80 1 -- 172.21.15.100:0/1530253036 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55e5007a2d70 con 0x55e500716d50 2024-05-10T11:36:43.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.535+0000 7faba2ffd700 1 -- 172.21.15.100:0/1530253036 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fabb400f640 con 0x55e500716d50 2024-05-10T11:36:43.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.535+0000 7faba2ffd700 1 -- 172.21.15.100:0/1530253036 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fabb4030070 con 0x55e500716d50 2024-05-10T11:36:43.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.535+0000 7fabd4aa6b80 10 client.28429 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:36:43.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.535+0000 7fabd4aa6b80 10 client.28429 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-10T11:36:43.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.535+0000 7fabd4aa6b80 20 client.28429 populate_metadata read hostname 'smithi100' 2024-05-10T11:36:43.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.535+0000 7fabd4aa6b80 1 --2- 172.21.15.100:0/1530253036 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55e500738890 0x55e5007a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:43.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.535+0000 7fabd4aa6b80 4 client.28429 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:36:43.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.535+0000 7fabd4aa6b80 1 -- 172.21.15.100:0/1530253036 --> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e5006703c0 con 0x55e500738890 2024-05-10T11:36:43.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.536+0000 7fabc2ffd700 1 --2- 172.21.15.100:0/1530253036 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55e500738890 0x55e5007a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:43.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.536+0000 7faba2ffd700 10 client.28429 ms_handle_connect on v2:172.21.15.188:6836/1943342620 2024-05-10T11:36:43.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.536+0000 7fabc2ffd700 1 --2- 172.21.15.100:0/1530253036 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55e500738890 0x55e5007a50f0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a8767dfa54ba1241 server_cookie=90813a936656c8e8 in_seq=0 out_seq=0 2024-05-10T11:36:43.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.537+0000 7faba2ffd700 1 -- 172.21.15.100:0/1530253036 <== mds.2 v2:172.21.15.188:6836/1943342620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e5006703c0 con 0x55e500738890 2024-05-10T11:36:43.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.537+0000 7faba2ffd700 10 client.28429 handle_command_reply: tid=0 2024-05-10T11:36:43.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.537+0000 7fabd4aa6b80 1 client.28429 shutdown 2024-05-10T11:36:43.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.537+0000 7fabd4aa6b80 1 -- 172.21.15.100:0/1530253036 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e500716d50 msgr2=0x55e500731290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:43.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.537+0000 7fabd4aa6b80 1 --2- 172.21.15.100:0/1530253036 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e500716d50 0x55e500731290 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fabb400f810 tx=0x7fabb400fe00 comp rx=0 tx=0).stop 2024-05-10T11:36:43.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.537+0000 7fabd4aa6b80 1 -- 172.21.15.100:0/1530253036 shutdown_connections 2024-05-10T11:36:43.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.537+0000 7fabd4aa6b80 1 --2- 172.21.15.100:0/1530253036 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55e500738890 0x55e5007a50f0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:43.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.537+0000 7faba2ffd700 0 client.28429 ms_handle_reset on v2:172.21.15.188:6836/1943342620 2024-05-10T11:36:43.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.537+0000 7fabd4aa6b80 1 --2- 172.21.15.100:0/1530253036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e50071cad0 0x55e50073fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:43.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.537+0000 7fabd4aa6b80 1 --2- 172.21.15.100:0/1530253036 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e500716d50 0x55e500731290 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:43.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.537+0000 7fabd4aa6b80 1 --2- 172.21.15.100:0/1530253036 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e50071c1b0 0x55e50073f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:43.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.537+0000 7fabd4aa6b80 1 -- 172.21.15.100:0/1530253036 >> 172.21.15.100:0/1530253036 conn(0x55e500642b70 msgr2=0x55e50071e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:36:43.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.538+0000 7fabd4aa6b80 1 -- 172.21.15.100:0/1530253036 shutdown_connections 2024-05-10T11:36:43.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.538+0000 7fabd4aa6b80 1 -- 172.21.15.100:0/1530253036 wait complete. 2024-05-10T11:36:43.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.538+0000 7fabd4aa6b80 20 client.28429 trim_cache size 0 max 16384 2024-05-10T11:36:43.543 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:36:43.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.539+0000 7fabca213700 1 -- 172.21.15.100:0/708874525 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fabac0b2900 msgr2=0x7fabac0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:43.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.539+0000 7fabca213700 1 --2- 172.21.15.100:0/708874525 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fabac0b2900 0x7fabac0b4dc0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fabb4009dc0 tx=0x7fabb4011720 comp rx=0 tx=0).stop 2024-05-10T11:36:43.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.539+0000 7fabca213700 1 -- 172.21.15.100:0/708874525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabc4109510 msgr2=0x7fabc4102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:43.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.539+0000 7fabca213700 1 --2- 172.21.15.100:0/708874525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabc4109510 0x7fabc4102ef0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fabbc009fd0 tx=0x7fabbc0095a0 comp rx=0 tx=0).stop 2024-05-10T11:36:43.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.539+0000 7fabca213700 1 -- 172.21.15.100:0/708874525 shutdown_connections 2024-05-10T11:36:43.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.539+0000 7fabca213700 1 --2- 172.21.15.100:0/708874525 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fabac0b2900 0x7fabac0b4dc0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:43.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.539+0000 7fabca213700 1 --2- 172.21.15.100:0/708874525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabc4109510 0x7fabc4102ef0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:43.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.539+0000 7fabca213700 1 --2- 172.21.15.100:0/708874525 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fabc406b180 0x7fabc4102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:43.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.539+0000 7fabca213700 1 --2- 172.21.15.100:0/708874525 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabc406bb40 0x7fabc41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:43.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.539+0000 7fabca213700 1 -- 172.21.15.100:0/708874525 >> 172.21.15.100:0/708874525 conn(0x7fabc4079350 msgr2=0x7fabc4100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:36:43.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.539+0000 7fabca213700 1 -- 172.21.15.100:0/708874525 shutdown_connections 2024-05-10T11:36:43.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:43.539+0000 7fabca213700 1 -- 172.21.15.100:0/708874525 wait complete. 2024-05-10T11:36:43.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:36:43 smithi100 ceph-mon[30296]: pgmap v971: 129 pgs: 129 active+clean; 471 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 3.6 MiB/s wr, 239 op/s 2024-05-10T11:36:43.931 INFO:tasks.workunit.client.0.smithi100.stdout: 10 46605 9.05 MB/sec execute 157 sec latency 80.849 ms 2024-05-10T11:36:44.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:36:43 smithi178 ceph-mon[33359]: pgmap v971: 129 pgs: 129 active+clean; 471 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 3.6 MiB/s wr, 239 op/s 2024-05-10T11:36:44.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:36:43 smithi188 ceph-mon[34075]: pgmap v971: 129 pgs: 129 active+clean; 471 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 3.6 MiB/s wr, 239 op/s 2024-05-10T11:36:44.358 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-10T11:36:44.705 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:36:44.931 INFO:tasks.workunit.client.0.smithi100.stdout: 10 46763 9.03 MB/sec execute 158 sec latency 80.593 ms 2024-05-10T11:36:45.931 INFO:tasks.workunit.client.0.smithi100.stdout: 10 46894 9.02 MB/sec execute 159 sec latency 124.360 ms 2024-05-10T11:36:46.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:36:45 smithi178 ceph-mon[33359]: pgmap v972: 129 pgs: 129 active+clean; 471 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 3.6 MiB/s wr, 225 op/s 2024-05-10T11:36:46.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:36:45 smithi188 ceph-mon[34075]: pgmap v972: 129 pgs: 129 active+clean; 471 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 3.6 MiB/s wr, 225 op/s 2024-05-10T11:36:46.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:36:45 smithi100 ceph-mon[30296]: pgmap v972: 129 pgs: 129 active+clean; 471 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 3.6 MiB/s wr, 225 op/s 2024-05-10T11:36:46.932 INFO:tasks.workunit.client.0.smithi100.stdout: 10 47038 9.02 MB/sec execute 160 sec latency 168.975 ms 2024-05-10T11:36:47.932 INFO:tasks.workunit.client.0.smithi100.stdout: 10 47189 8.99 MB/sec execute 161 sec latency 223.803 ms 2024-05-10T11:36:48.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:36:47 smithi178 ceph-mon[33359]: pgmap v973: 129 pgs: 129 active+clean; 474 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 4.3 MiB/s wr, 297 op/s 2024-05-10T11:36:48.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:36:47 smithi188 ceph-mon[34075]: pgmap v973: 129 pgs: 129 active+clean; 474 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 4.3 MiB/s wr, 297 op/s 2024-05-10T11:36:48.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:36:47 smithi100 ceph-mon[30296]: pgmap v973: 129 pgs: 129 active+clean; 474 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 4.3 MiB/s wr, 297 op/s 2024-05-10T11:36:48.932 INFO:tasks.workunit.client.0.smithi100.stdout: 10 47347 8.99 MB/sec execute 162 sec latency 104.064 ms 2024-05-10T11:36:49.932 INFO:tasks.workunit.client.0.smithi100.stdout: 10 47500 8.97 MB/sec execute 163 sec latency 112.311 ms 2024-05-10T11:36:50.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:36:49 smithi178 ceph-mon[33359]: pgmap v974: 129 pgs: 129 active+clean; 468 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 4.1 MiB/s wr, 244 op/s 2024-05-10T11:36:50.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:36:49 smithi188 ceph-mon[34075]: pgmap v974: 129 pgs: 129 active+clean; 468 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 4.1 MiB/s wr, 244 op/s 2024-05-10T11:36:50.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:36:49 smithi100 ceph-mon[30296]: pgmap v974: 129 pgs: 129 active+clean; 468 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 4.1 MiB/s wr, 244 op/s 2024-05-10T11:36:50.932 INFO:tasks.workunit.client.0.smithi100.stdout: 10 47704 9.00 MB/sec execute 164 sec latency 64.232 ms 2024-05-10T11:36:51.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.126+0000 7fb704034700 1 -- 172.21.15.100:0/1554481809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6fc10b6e0 msgr2=0x7fb6fc10bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:51.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.126+0000 7fb704034700 1 --2- 172.21.15.100:0/1554481809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6fc10b6e0 0x7fb6fc10bac0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec0098f0 tx=0x7fb6ec009c00 comp rx=0 tx=0).stop 2024-05-10T11:36:51.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.126+0000 7fb704034700 1 -- 172.21.15.100:0/1554481809 shutdown_connections 2024-05-10T11:36:51.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.126+0000 7fb704034700 1 --2- 172.21.15.100:0/1554481809 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6fc1060a0 0x7fb6fc1125b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:51.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.126+0000 7fb704034700 1 --2- 172.21.15.100:0/1554481809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6fc10b6e0 0x7fb6fc10bac0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:51.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.126+0000 7fb704034700 1 --2- 172.21.15.100:0/1554481809 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6fc1056e0 0x7fb6fc105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:51.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.126+0000 7fb704034700 1 -- 172.21.15.100:0/1554481809 >> 172.21.15.100:0/1554481809 conn(0x7fb6fc1011d0 msgr2=0x7fb6fc1035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:36:51.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.126+0000 7fb704034700 1 -- 172.21.15.100:0/1554481809 shutdown_connections 2024-05-10T11:36:51.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.126+0000 7fb704034700 1 -- 172.21.15.100:0/1554481809 wait complete. 2024-05-10T11:36:51.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.127+0000 7fb704034700 1 Processor -- start 2024-05-10T11:36:51.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.127+0000 7fb704034700 1 -- start start 2024-05-10T11:36:51.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.128+0000 7fb704034700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6fc1056e0 0x7fb6fc1124e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:51.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.128+0000 7fb704034700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6fc1060a0 0x7fb6fc074b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:51.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.128+0000 7fb704034700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6fc10b6e0 0x7fb6fc075130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:51.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.128+0000 7fb704034700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb6fc06d370 con 0x7fb6fc10b6e0 2024-05-10T11:36:51.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.128+0000 7fb704034700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb6fc06d1f0 con 0x7fb6fc1060a0 2024-05-10T11:36:51.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.128+0000 7fb704034700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb6fc06d4f0 con 0x7fb6fc1056e0 2024-05-10T11:36:51.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.128+0000 7fb7015cf700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6fc1060a0 0x7fb6fc074b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:51.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.128+0000 7fb7015cf700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6fc1060a0 0x7fb6fc074b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:57518/0 (socket says 172.21.15.100:57518) 2024-05-10T11:36:51.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.128+0000 7fb7015cf700 1 -- 172.21.15.100:0/2963866276 learned_addr learned my addr 172.21.15.100:0/2963866276 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:36:51.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.128+0000 7fb7025d1700 1 --2- 172.21.15.100:0/2963866276 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6fc10b6e0 0x7fb6fc075130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:51.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.128+0000 7fb7015cf700 1 -- 172.21.15.100:0/2963866276 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6fc1056e0 msgr2=0x7fb6fc1124e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-10T11:36:51.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.128+0000 7fb7015cf700 1 --2- 172.21.15.100:0/2963866276 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6fc1056e0 0x7fb6fc1124e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:51.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.128+0000 7fb7015cf700 1 -- 172.21.15.100:0/2963866276 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6fc10b6e0 msgr2=0x7fb6fc075130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:51.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.128+0000 7fb7015cf700 1 --2- 172.21.15.100:0/2963866276 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6fc10b6e0 0x7fb6fc075130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:51.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.128+0000 7fb7015cf700 1 -- 172.21.15.100:0/2963866276 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6fc10fa70 con 0x7fb6fc1060a0 2024-05-10T11:36:51.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.129+0000 7fb7015cf700 1 --2- 172.21.15.100:0/2963866276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6fc1060a0 0x7fb6fc074b40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb6f000ec10 tx=0x7fb6f000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:36:51.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.130+0000 7fb6faffd700 1 -- 172.21.15.100:0/2963866276 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f000cd40 con 0x7fb6fc1060a0 2024-05-10T11:36:51.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.130+0000 7fb6faffd700 1 -- 172.21.15.100:0/2963866276 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb6f00044b0 con 0x7fb6fc1060a0 2024-05-10T11:36:51.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.130+0000 7fb6faffd700 1 -- 172.21.15.100:0/2963866276 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f0010430 con 0x7fb6fc1060a0 2024-05-10T11:36:51.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.130+0000 7fb704034700 1 -- 172.21.15.100:0/2963866276 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6fc10fd00 con 0x7fb6fc1060a0 2024-05-10T11:36:51.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.130+0000 7fb704034700 1 -- 172.21.15.100:0/2963866276 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6fc1101f0 con 0x7fb6fc1060a0 2024-05-10T11:36:51.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.133+0000 7fb6faffd700 1 -- 172.21.15.100:0/2963866276 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fb6f00105d0 con 0x7fb6fc1060a0 2024-05-10T11:36:51.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.134+0000 7fb6faffd700 1 --2- 172.21.15.100:0/2963866276 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fb6e40b29d0 0x7fb6e40b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:51.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.135+0000 7fb701dd0700 1 --2- 172.21.15.100:0/2963866276 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fb6e40b29d0 0x7fb6e40b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:51.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.135+0000 7fb6faffd700 1 -- 172.21.15.100:0/2963866276 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fb6f00e2810 con 0x7fb6fc1060a0 2024-05-10T11:36:51.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.135+0000 7fb70e8c7b80 1 Processor -- start 2024-05-10T11:36:51.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.135+0000 7fb70e8c7b80 1 -- start start 2024-05-10T11:36:51.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.135+0000 7fb701dd0700 1 --2- 172.21.15.100:0/2963866276 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fb6e40b29d0 0x7fb6e40b4e90 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec009680 tx=0x7fb6ec009f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:36:51.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.136+0000 7fb70e8c7b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56195aa904d0 0x56195aa908b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:51.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.136+0000 7fb70e8c7b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56195aa90df0 0x56195aa946f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:51.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.136+0000 7fb70e8c7b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56195aa94c80 0x56195aa95100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:51.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.136+0000 7fb70e8c7b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56195a9e2860 con 0x56195aa90df0 2024-05-10T11:36:51.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.136+0000 7fb70e8c7b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56195a9817c0 con 0x56195aa904d0 2024-05-10T11:36:51.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.136+0000 7fb70e8c7b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56195a9de820 con 0x56195aa94c80 2024-05-10T11:36:51.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.136+0000 7fb701dd0700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56195aa904d0 0x56195aa908b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:51.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.136+0000 7fb701dd0700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56195aa904d0 0x56195aa908b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:57528/0 (socket says 172.21.15.100:57528) 2024-05-10T11:36:51.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.136+0000 7fb701dd0700 1 -- 172.21.15.100:0/1032738580 learned_addr learned my addr 172.21.15.100:0/1032738580 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:36:51.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.136+0000 7fb7015cf700 1 --2- 172.21.15.100:0/1032738580 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56195aa90df0 0x56195aa946f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:51.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.136+0000 7fb7025d1700 1 --2- 172.21.15.100:0/1032738580 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56195aa94c80 0x56195aa95100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:51.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.136+0000 7fb701dd0700 1 -- 172.21.15.100:0/1032738580 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56195aa94c80 msgr2=0x56195aa95100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:51.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.136+0000 7fb701dd0700 1 --2- 172.21.15.100:0/1032738580 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56195aa94c80 0x56195aa95100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:51.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.136+0000 7fb701dd0700 1 -- 172.21.15.100:0/1032738580 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56195aa90df0 msgr2=0x56195aa946f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:51.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.136+0000 7fb701dd0700 1 --2- 172.21.15.100:0/1032738580 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56195aa90df0 0x56195aa946f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:51.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.136+0000 7fb701dd0700 1 -- 172.21.15.100:0/1032738580 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56195a92f8d0 con 0x56195aa904d0 2024-05-10T11:36:51.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.137+0000 7fb701dd0700 1 --2- 172.21.15.100:0/1032738580 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56195aa904d0 0x56195aa908b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec016000 tx=0x7fb6ec01c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83ac42adb5ec6142 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:36:51.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.137+0000 7fb6e3fff700 1 -- 172.21.15.100:0/1032738580 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec010ec0 con 0x56195aa904d0 2024-05-10T11:36:51.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.137+0000 7fb6e3fff700 1 -- 172.21.15.100:0/1032738580 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb6ec00f8f0 con 0x56195aa904d0 2024-05-10T11:36:51.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.137+0000 7fb6e3fff700 1 -- 172.21.15.100:0/1032738580 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec003cf0 con 0x56195aa904d0 2024-05-10T11:36:51.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.138+0000 7fb70e8c7b80 1 -- 172.21.15.100:0/1032738580 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56195aa904d0 msgr2=0x56195aa908b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:51.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.138+0000 7fb70e8c7b80 1 --2- 172.21.15.100:0/1032738580 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56195aa904d0 0x56195aa908b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec016000 tx=0x7fb6ec01c000 comp rx=0 tx=0).stop 2024-05-10T11:36:51.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.138+0000 7fb70e8c7b80 1 -- 172.21.15.100:0/1032738580 shutdown_connections 2024-05-10T11:36:51.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.138+0000 7fb70e8c7b80 1 --2- 172.21.15.100:0/1032738580 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56195aa94c80 0x56195aa95100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:51.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.138+0000 7fb70e8c7b80 1 --2- 172.21.15.100:0/1032738580 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56195aa904d0 0x56195aa908b0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:51.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.138+0000 7fb70e8c7b80 1 --2- 172.21.15.100:0/1032738580 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56195aa90df0 0x56195aa946f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:51.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.138+0000 7fb70e8c7b80 1 -- 172.21.15.100:0/1032738580 >> 172.21.15.100:0/1032738580 conn(0x56195a9bc100 msgr2=0x56195aa8ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:36:51.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.138+0000 7fb70e8c7b80 1 -- 172.21.15.100:0/1032738580 shutdown_connections 2024-05-10T11:36:51.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.138+0000 7fb70e8c7b80 1 -- 172.21.15.100:0/1032738580 wait complete. 2024-05-10T11:36:51.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.138+0000 7fb70e8c7b80 1 -- start start 2024-05-10T11:36:51.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.139+0000 7fb70e8c7b80 1 Processor -- start 2024-05-10T11:36:51.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.139+0000 7fb70e8c7b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56195aa904d0 0x56195aa9bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:51.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.139+0000 7fb70e8c7b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56195aa90df0 0x56195aa99350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:51.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.139+0000 7fb70e8c7b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56195aa94c80 0x56195aa99890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:51.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.139+0000 7fb70e8c7b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56195aa9ce30 con 0x56195aa904d0 2024-05-10T11:36:51.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.139+0000 7fb70e8c7b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56195a948c30 con 0x56195aa90df0 2024-05-10T11:36:51.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.139+0000 7fb70e8c7b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56195aa9cfb0 con 0x56195aa94c80 2024-05-10T11:36:51.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.139+0000 7fb7025d1700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56195aa94c80 0x56195aa99890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:51.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.139+0000 7fb7025d1700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56195aa94c80 0x56195aa99890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:59098/0 (socket says 172.21.15.100:59098) 2024-05-10T11:36:51.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.139+0000 7fb7025d1700 1 -- 172.21.15.100:0/466702167 learned_addr learned my addr 172.21.15.100:0/466702167 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:36:51.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.139+0000 7fb7015cf700 1 --2- 172.21.15.100:0/466702167 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56195aa90df0 0x56195aa99350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:51.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.139+0000 7fb701dd0700 1 --2- 172.21.15.100:0/466702167 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56195aa904d0 0x56195aa9bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:51.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.139+0000 7fb70e8c7b80 10 client.0 _ll_register_callbacks cb 0x56195a97a920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:36:51.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.140+0000 7fb7025d1700 1 -- 172.21.15.100:0/466702167 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56195aa90df0 msgr2=0x56195aa99350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:51.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.140+0000 7fb7025d1700 1 --2- 172.21.15.100:0/466702167 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56195aa90df0 0x56195aa99350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:51.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.140+0000 7fb7025d1700 1 -- 172.21.15.100:0/466702167 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56195aa904d0 msgr2=0x56195aa9bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:51.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.140+0000 7fb7025d1700 1 --2- 172.21.15.100:0/466702167 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56195aa904d0 0x56195aa9bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:51.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.140+0000 7fb7025d1700 1 -- 172.21.15.100:0/466702167 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56195a97e690 con 0x56195aa94c80 2024-05-10T11:36:51.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.140+0000 7fb701dd0700 1 --2- 172.21.15.100:0/466702167 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56195aa904d0 0x56195aa9bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:36:51.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.140+0000 7fb7025d1700 1 --2- 172.21.15.100:0/466702167 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56195aa94c80 0x56195aa99890 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb6f4011e10 tx=0x7fb6f4014600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d60ffd738f54c9f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:36:51.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.140+0000 7fb6e17fa700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:36:51.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.140+0000 7fb6e17fa700 1 -- 172.21.15.100:0/466702167 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f4003a00 con 0x56195aa94c80 2024-05-10T11:36:51.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.140+0000 7fb70e8c7b80 1 -- 172.21.15.100:0/466702167 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6ec00f8f0 con 0x56195aa94c80 2024-05-10T11:36:51.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.140+0000 7fb6e17fa700 1 -- 172.21.15.100:0/466702167 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fb6f4003ba0 con 0x56195aa94c80 2024-05-10T11:36:51.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.140+0000 7fb6e17fa700 1 -- 172.21.15.100:0/466702167 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fb6f4003da0 con 0x56195aa94c80 2024-05-10T11:36:51.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.141+0000 7fb70e8c7b80 10 client.28438 fetch_fsmap learned FSMap version 41 2024-05-10T11:36:51.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.141+0000 7fb70e8c7b80 1 -- 172.21.15.100:0/466702167 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x56195aaa7640 con 0x56195aa94c80 2024-05-10T11:36:51.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.141+0000 7fb6e17fa700 1 -- 172.21.15.100:0/466702167 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb6ec00f8f0 con 0x56195aa94c80 2024-05-10T11:36:51.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.141+0000 7fb6e17fa700 1 -- 172.21.15.100:0/466702167 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fb6f401e520 con 0x56195aa94c80 2024-05-10T11:36:51.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.141+0000 7fb70e8c7b80 10 client.28438 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:36:51.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.141+0000 7fb70e8c7b80 10 client.28438 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-10T11:36:51.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.141+0000 7fb70e8c7b80 20 client.28438 populate_metadata read hostname 'smithi100' 2024-05-10T11:36:51.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.141+0000 7fb70e8c7b80 1 --2- 172.21.15.100:0/466702167 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x56195aaa9cd0 0x56195aaac150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:51.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.141+0000 7fb70e8c7b80 4 client.28438 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:36:51.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.141+0000 7fb70e8c7b80 1 -- 172.21.15.100:0/466702167 --> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56195aa5c650 con 0x56195aaa9cd0 2024-05-10T11:36:51.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.142+0000 7fb701dd0700 1 --2- 172.21.15.100:0/466702167 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x56195aaa9cd0 0x56195aaac150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:51.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.142+0000 7fb701dd0700 1 --2- 172.21.15.100:0/466702167 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x56195aaa9cd0 0x56195aaac150 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=58598ffe5e943c4a server_cookie=dcd22082db63e1dd in_seq=0 out_seq=0 2024-05-10T11:36:51.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.142+0000 7fb6e17fa700 10 client.28438 ms_handle_connect on v2:172.21.15.178:6838/3760985404 2024-05-10T11:36:51.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.143+0000 7fb6e17fa700 1 -- 172.21.15.100:0/466702167 <== mds.1 v2:172.21.15.178:6838/3760985404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56195aa5c650 con 0x56195aaa9cd0 2024-05-10T11:36:51.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.143+0000 7fb6e17fa700 10 client.28438 handle_command_reply: tid=0 2024-05-10T11:36:51.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.143+0000 7fb70e8c7b80 1 client.28438 shutdown 2024-05-10T11:36:51.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.143+0000 7fb70e8c7b80 1 -- 172.21.15.100:0/466702167 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56195aa94c80 msgr2=0x56195aa99890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:51.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.143+0000 7fb70e8c7b80 1 --2- 172.21.15.100:0/466702167 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56195aa94c80 0x56195aa99890 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb6f4011e10 tx=0x7fb6f4014600 comp rx=0 tx=0).stop 2024-05-10T11:36:51.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.143+0000 7fb70e8c7b80 1 -- 172.21.15.100:0/466702167 shutdown_connections 2024-05-10T11:36:51.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.143+0000 7fb70e8c7b80 1 --2- 172.21.15.100:0/466702167 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x56195aaa9cd0 0x56195aaac150 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:51.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.144+0000 7fb70e8c7b80 1 --2- 172.21.15.100:0/466702167 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56195aa94c80 0x56195aa99890 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:51.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.144+0000 7fb6e17fa700 0 client.28438 ms_handle_reset on v2:172.21.15.178:6838/3760985404 2024-05-10T11:36:51.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.144+0000 7fb70e8c7b80 1 --2- 172.21.15.100:0/466702167 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56195aa904d0 0x56195aa9bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:51.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.144+0000 7fb70e8c7b80 1 --2- 172.21.15.100:0/466702167 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56195aa90df0 0x56195aa99350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:51.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.144+0000 7fb70e8c7b80 1 -- 172.21.15.100:0/466702167 >> 172.21.15.100:0/466702167 conn(0x56195a9bc100 msgr2=0x56195aa85720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:36:51.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.144+0000 7fb70e8c7b80 1 -- 172.21.15.100:0/466702167 shutdown_connections 2024-05-10T11:36:51.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.144+0000 7fb70e8c7b80 1 -- 172.21.15.100:0/466702167 wait complete. 2024-05-10T11:36:51.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.144+0000 7fb70e8c7b80 20 client.28438 trim_cache size 0 max 16384 2024-05-10T11:36:51.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.172+0000 7fb70e8c7b80 1 Processor -- start 2024-05-10T11:36:51.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.172+0000 7fb70e8c7b80 1 -- start start 2024-05-10T11:36:51.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.172+0000 7fb70e8c7b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56195aa904d0 0x56195aaa5f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:51.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.172+0000 7fb70e8c7b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56195aa90df0 0x56195aaa6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:51.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.172+0000 7fb70e8c7b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56195aa94c80 0x56195aa92520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:51.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.172+0000 7fb70e8c7b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e8011290 con 0x56195aa90df0 2024-05-10T11:36:51.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.172+0000 7fb70e8c7b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e800eb20 con 0x56195aa904d0 2024-05-10T11:36:51.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.172+0000 7fb7015cf700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56195aa90df0 0x56195aaa6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:51.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.172+0000 7fb701dd0700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56195aa904d0 0x56195aaa5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:51.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.172+0000 7fb7015cf700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56195aa90df0 0x56195aaa6490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55586/0 (socket says 172.21.15.100:55586) 2024-05-10T11:36:51.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.172+0000 7fb7015cf700 1 -- 172.21.15.100:0/1094250039 learned_addr learned my addr 172.21.15.100:0/1094250039 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:36:51.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.172+0000 7fb70e8c7b80 1 -- 172.21.15.100:0/1094250039 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e800c970 con 0x56195aa94c80 2024-05-10T11:36:51.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.172+0000 7fb7025d1700 1 --2- 172.21.15.100:0/1094250039 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56195aa94c80 0x56195aa92520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:51.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.173+0000 7fb701dd0700 1 -- 172.21.15.100:0/1094250039 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56195aa94c80 msgr2=0x56195aa92520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:51.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.173+0000 7fb701dd0700 1 --2- 172.21.15.100:0/1094250039 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56195aa94c80 0x56195aa92520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:51.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.173+0000 7fb701dd0700 1 -- 172.21.15.100:0/1094250039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56195aa90df0 msgr2=0x56195aaa6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:51.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.173+0000 7fb701dd0700 1 --2- 172.21.15.100:0/1094250039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56195aa90df0 0x56195aaa6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:51.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.173+0000 7fb701dd0700 1 -- 172.21.15.100:0/1094250039 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56195aa97900 con 0x56195aa904d0 2024-05-10T11:36:51.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.173+0000 7fb7015cf700 1 --2- 172.21.15.100:0/1094250039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56195aa90df0 0x56195aaa6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:36:51.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.173+0000 7fb701dd0700 1 --2- 172.21.15.100:0/1094250039 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56195aa904d0 0x56195aaa5f50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec01e040 tx=0x7fb6ec0095e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf44018f167a024 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:36:51.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.173+0000 7fb6e37fe700 1 -- 172.21.15.100:0/1094250039 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec03e070 con 0x56195aa904d0 2024-05-10T11:36:51.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.173+0000 7fb6e37fe700 1 -- 172.21.15.100:0/1094250039 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb6ec003d00 con 0x56195aa904d0 2024-05-10T11:36:51.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.173+0000 7fb6e37fe700 1 -- 172.21.15.100:0/1094250039 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec039420 con 0x56195aa904d0 2024-05-10T11:36:51.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.174+0000 7fb70e8c7b80 1 -- 172.21.15.100:0/1094250039 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56195aa904d0 msgr2=0x56195aaa5f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:51.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.174+0000 7fb70e8c7b80 1 --2- 172.21.15.100:0/1094250039 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56195aa904d0 0x56195aaa5f50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec01e040 tx=0x7fb6ec0095e0 comp rx=0 tx=0).stop 2024-05-10T11:36:51.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.174+0000 7fb70e8c7b80 1 -- 172.21.15.100:0/1094250039 shutdown_connections 2024-05-10T11:36:51.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.174+0000 7fb70e8c7b80 1 --2- 172.21.15.100:0/1094250039 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56195aa94c80 0x56195aa92520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:51.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.174+0000 7fb70e8c7b80 1 --2- 172.21.15.100:0/1094250039 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56195aa904d0 0x56195aaa5f50 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:51.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.174+0000 7fb70e8c7b80 1 --2- 172.21.15.100:0/1094250039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56195aa90df0 0x56195aaa6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:51.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.174+0000 7fb70e8c7b80 1 -- 172.21.15.100:0/1094250039 >> 172.21.15.100:0/1094250039 conn(0x56195a9bc100 msgr2=0x56195aa97c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:36:51.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.174+0000 7fb70e8c7b80 1 -- 172.21.15.100:0/1094250039 shutdown_connections 2024-05-10T11:36:51.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.174+0000 7fb70e8c7b80 1 -- 172.21.15.100:0/1094250039 wait complete. 2024-05-10T11:36:51.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.175+0000 7fb70e8c7b80 1 -- start start 2024-05-10T11:36:51.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.175+0000 7fb70e8c7b80 1 Processor -- start 2024-05-10T11:36:51.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.176+0000 7fb70e8c7b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56195aa904d0 0x56195aaa2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:51.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.176+0000 7fb70e8c7b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56195aa90df0 0x56195aaa2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:51.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.176+0000 7fb70e8c7b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56195aa94c80 0x56195aa9da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:51.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.176+0000 7fb70e8c7b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e800cfe0 con 0x56195aa94c80 2024-05-10T11:36:51.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.176+0000 7fb70e8c7b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e800e450 con 0x56195aa90df0 2024-05-10T11:36:51.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.176+0000 7fb70e8c7b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e800d6b0 con 0x56195aa904d0 2024-05-10T11:36:51.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.176+0000 7fb70e8c7b80 10 client.0 _ll_register_callbacks cb 0x56195a97a920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:36:51.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.176+0000 7fb701dd0700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56195aa904d0 0x56195aaa2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:51.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.176+0000 7fb701dd0700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56195aa904d0 0x56195aaa2430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:59110/0 (socket says 172.21.15.100:59110) 2024-05-10T11:36:51.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.176+0000 7fb701dd0700 1 -- 172.21.15.100:0/1127251433 learned_addr learned my addr 172.21.15.100:0/1127251433 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:36:51.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.176+0000 7fb7015cf700 1 --2- 172.21.15.100:0/1127251433 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56195aa90df0 0x56195aaa2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:51.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.176+0000 7fb7025d1700 1 --2- 172.21.15.100:0/1127251433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56195aa94c80 0x56195aa9da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:51.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.177+0000 7fb7025d1700 1 -- 172.21.15.100:0/1127251433 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56195aa904d0 msgr2=0x56195aaa2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:51.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.177+0000 7fb7025d1700 1 --2- 172.21.15.100:0/1127251433 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56195aa904d0 0x56195aaa2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:51.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.177+0000 7fb7025d1700 1 -- 172.21.15.100:0/1127251433 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56195aa90df0 msgr2=0x56195aaa2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:51.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.177+0000 7fb7025d1700 1 --2- 172.21.15.100:0/1127251433 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56195aa90df0 0x56195aaa2970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:51.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.177+0000 7fb7025d1700 1 -- 172.21.15.100:0/1127251433 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56195aa9e2b0 con 0x56195aa94c80 2024-05-10T11:36:51.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.177+0000 7fb7025d1700 1 --2- 172.21.15.100:0/1127251433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56195aa94c80 0x56195aa9da70 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fb6f400b010 tx=0x7fb6f4014950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42f0f48326c70221 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:36:51.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.178+0000 7fb6e17fa700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:36:51.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.178+0000 7fb6e17fa700 1 -- 172.21.15.100:0/1127251433 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f401ed70 con 0x56195aa94c80 2024-05-10T11:36:51.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.178+0000 7fb6e17fa700 1 -- 172.21.15.100:0/1127251433 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fb6f401e400 con 0x56195aa94c80 2024-05-10T11:36:51.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.178+0000 7fb70e8c7b80 1 -- 172.21.15.100:0/1127251433 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6ec003d00 con 0x56195aa94c80 2024-05-10T11:36:51.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.178+0000 7fb6e17fa700 1 -- 172.21.15.100:0/1127251433 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fb6f401e600 con 0x56195aa94c80 2024-05-10T11:36:51.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.178+0000 7fb70e8c7b80 10 client.19041 fetch_fsmap learned FSMap version 41 2024-05-10T11:36:51.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.178+0000 7fb70e8c7b80 1 -- 172.21.15.100:0/1127251433 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x56195ab14270 con 0x56195aa94c80 2024-05-10T11:36:51.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.179+0000 7fb6e17fa700 1 -- 172.21.15.100:0/1127251433 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb6ec003d00 con 0x56195aa94c80 2024-05-10T11:36:51.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.179+0000 7fb6e17fa700 1 -- 172.21.15.100:0/1127251433 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fb6f402c450 con 0x56195aa94c80 2024-05-10T11:36:51.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.179+0000 7fb70e8c7b80 10 client.19041 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:36:51.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.179+0000 7fb70e8c7b80 10 client.19041 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-10T11:36:51.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.179+0000 7fb70e8c7b80 20 client.19041 populate_metadata read hostname 'smithi100' 2024-05-10T11:36:51.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.179+0000 7fb70e8c7b80 1 --2- 172.21.15.100:0/1127251433 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x56195aaa9cd0 0x56195ab16910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:51.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.179+0000 7fb70e8c7b80 4 client.19041 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:36:51.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.179+0000 7fb70e8c7b80 1 -- 172.21.15.100:0/1127251433 --> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56195a9f0240 con 0x56195aaa9cd0 2024-05-10T11:36:51.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.180+0000 7fb701dd0700 1 --2- 172.21.15.100:0/1127251433 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x56195aaa9cd0 0x56195ab16910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:51.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.181+0000 7fb701dd0700 1 --2- 172.21.15.100:0/1127251433 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x56195aaa9cd0 0x56195ab16910 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e2857dc0c9f395a2 server_cookie=40b6a4a02b0dd31 in_seq=0 out_seq=0 2024-05-10T11:36:51.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.181+0000 7fb6e17fa700 10 client.19041 ms_handle_connect on v2:172.21.15.178:6838/3760985404 2024-05-10T11:36:51.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.181+0000 7fb6e17fa700 1 -- 172.21.15.100:0/1127251433 <== mds.1 v2:172.21.15.178:6838/3760985404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56195a9f0240 con 0x56195aaa9cd0 2024-05-10T11:36:51.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.181+0000 7fb6e17fa700 10 client.19041 handle_command_reply: tid=0 2024-05-10T11:36:51.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.181+0000 7fb70e8c7b80 1 client.19041 shutdown 2024-05-10T11:36:51.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.182+0000 7fb70e8c7b80 1 -- 172.21.15.100:0/1127251433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56195aa94c80 msgr2=0x56195aa9da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:51.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.182+0000 7fb70e8c7b80 1 --2- 172.21.15.100:0/1127251433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56195aa94c80 0x56195aa9da70 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fb6f400b010 tx=0x7fb6f4014950 comp rx=0 tx=0).stop 2024-05-10T11:36:51.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.182+0000 7fb70e8c7b80 1 -- 172.21.15.100:0/1127251433 shutdown_connections 2024-05-10T11:36:51.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.182+0000 7fb70e8c7b80 1 --2- 172.21.15.100:0/1127251433 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x56195aaa9cd0 0x56195ab16910 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:51.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.182+0000 7fb70e8c7b80 1 --2- 172.21.15.100:0/1127251433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56195aa94c80 0x56195aa9da70 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:51.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.182+0000 7fb70e8c7b80 1 --2- 172.21.15.100:0/1127251433 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56195aa904d0 0x56195aaa2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:51.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.182+0000 7fb70e8c7b80 1 --2- 172.21.15.100:0/1127251433 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56195aa90df0 0x56195aaa2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:51.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.182+0000 7fb70e8c7b80 1 -- 172.21.15.100:0/1127251433 >> 172.21.15.100:0/1127251433 conn(0x56195a9bc100 msgr2=0x56195aa93520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:36:51.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.183+0000 7fb6e17fa700 0 client.19041 ms_handle_reset on v2:172.21.15.178:6838/3760985404 2024-05-10T11:36:51.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.183+0000 7fb70e8c7b80 1 -- 172.21.15.100:0/1127251433 shutdown_connections 2024-05-10T11:36:51.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.183+0000 7fb70e8c7b80 1 -- 172.21.15.100:0/1127251433 wait complete. 2024-05-10T11:36:51.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.183+0000 7fb70e8c7b80 20 client.19041 trim_cache size 0 max 16384 2024-05-10T11:36:51.184 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:36:51.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.184+0000 7fb704034700 1 -- 172.21.15.100:0/2963866276 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fb6e40b29d0 msgr2=0x7fb6e40b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:51.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.184+0000 7fb704034700 1 --2- 172.21.15.100:0/2963866276 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fb6e40b29d0 0x7fb6e40b4e90 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec009680 tx=0x7fb6ec009f90 comp rx=0 tx=0).stop 2024-05-10T11:36:51.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.184+0000 7fb704034700 1 -- 172.21.15.100:0/2963866276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6fc1060a0 msgr2=0x7fb6fc074b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:51.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.184+0000 7fb704034700 1 --2- 172.21.15.100:0/2963866276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6fc1060a0 0x7fb6fc074b40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb6f000ec10 tx=0x7fb6f000ef20 comp rx=0 tx=0).stop 2024-05-10T11:36:51.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.184+0000 7fb704034700 1 -- 172.21.15.100:0/2963866276 shutdown_connections 2024-05-10T11:36:51.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.184+0000 7fb704034700 1 --2- 172.21.15.100:0/2963866276 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fb6e40b29d0 0x7fb6e40b4e90 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:51.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.184+0000 7fb704034700 1 --2- 172.21.15.100:0/2963866276 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6fc10b6e0 0x7fb6fc075130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:51.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.184+0000 7fb704034700 1 --2- 172.21.15.100:0/2963866276 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6fc1056e0 0x7fb6fc1124e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:51.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.184+0000 7fb704034700 1 --2- 172.21.15.100:0/2963866276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6fc1060a0 0x7fb6fc074b40 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:51.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.184+0000 7fb704034700 1 -- 172.21.15.100:0/2963866276 >> 172.21.15.100:0/2963866276 conn(0x7fb6fc1011d0 msgr2=0x7fb6fc101950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:36:51.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.185+0000 7fb704034700 1 -- 172.21.15.100:0/2963866276 shutdown_connections 2024-05-10T11:36:51.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:51.185+0000 7fb704034700 1 -- 172.21.15.100:0/2963866276 wait complete. 2024-05-10T11:36:51.932 INFO:tasks.workunit.client.0.smithi100.stdout: 10 47860 8.99 MB/sec execute 165 sec latency 66.720 ms 2024-05-10T11:36:52.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:36:51 smithi178 ceph-mon[33359]: pgmap v975: 129 pgs: 129 active+clean; 465 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 845 KiB/s rd, 4.5 MiB/s wr, 287 op/s 2024-05-10T11:36:52.004 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:36:51 smithi100 ceph-mon[30296]: pgmap v975: 129 pgs: 129 active+clean; 465 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 845 KiB/s rd, 4.5 MiB/s wr, 287 op/s 2024-05-10T11:36:52.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:36:51 smithi188 ceph-mon[34075]: pgmap v975: 129 pgs: 129 active+clean; 465 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 845 KiB/s rd, 4.5 MiB/s wr, 287 op/s 2024-05-10T11:36:52.039 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-10T11:36:52.379 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:36:52.932 INFO:tasks.workunit.client.0.smithi100.stdout: 10 48037 8.99 MB/sec execute 166 sec latency 69.818 ms 2024-05-10T11:36:53.932 INFO:tasks.workunit.client.0.smithi100.stdout: 10 48242 9.00 MB/sec execute 167 sec latency 67.444 ms 2024-05-10T11:36:53.935 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:36:53 smithi100 ceph-mon[30296]: pgmap v976: 129 pgs: 129 active+clean; 458 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 3.9 MiB/s wr, 273 op/s 2024-05-10T11:36:54.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:36:53 smithi178 ceph-mon[33359]: pgmap v976: 129 pgs: 129 active+clean; 458 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 3.9 MiB/s wr, 273 op/s 2024-05-10T11:36:54.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:36:53 smithi188 ceph-mon[34075]: pgmap v976: 129 pgs: 129 active+clean; 458 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 3.9 MiB/s wr, 273 op/s 2024-05-10T11:36:54.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.653+0000 7f36a7896700 1 -- 172.21.15.100:0/1334874216 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36a0109520 msgr2=0x7f36a0109900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:54.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.653+0000 7f36a7896700 1 --2- 172.21.15.100:0/1334874216 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36a0109520 0x7f36a0109900 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f369c0098f0 tx=0x7f369c009c00 comp rx=0 tx=0).stop 2024-05-10T11:36:54.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.653+0000 7f36a7896700 1 -- 172.21.15.100:0/1334874216 shutdown_connections 2024-05-10T11:36:54.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.653+0000 7f36a7896700 1 --2- 172.21.15.100:0/1334874216 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36a0102450 0x7f36a0110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:54.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.653+0000 7f36a7896700 1 --2- 172.21.15.100:0/1334874216 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36a0109520 0x7f36a0109900 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:54.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.653+0000 7f36a7896700 1 --2- 172.21.15.100:0/1334874216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36a0101a90 0x7f36a0101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:54.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.653+0000 7f36a7896700 1 -- 172.21.15.100:0/1334874216 >> 172.21.15.100:0/1334874216 conn(0x7f36a0078a10 msgr2=0x7f36a00ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:36:54.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.653+0000 7f36a7896700 1 -- 172.21.15.100:0/1334874216 shutdown_connections 2024-05-10T11:36:54.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.653+0000 7f36a7896700 1 -- 172.21.15.100:0/1334874216 wait complete. 2024-05-10T11:36:54.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.654+0000 7f36a7896700 1 Processor -- start 2024-05-10T11:36:54.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.654+0000 7f36a7896700 1 -- start start 2024-05-10T11:36:54.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.654+0000 7f36a7896700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36a0101a90 0x7f36a019f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:54.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.654+0000 7f36a7896700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36a0102450 0x7f36a019fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:54.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.654+0000 7f36a7896700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36a0109520 0x7f36a01998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:54.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.654+0000 7f36a4e31700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36a0102450 0x7f36a019fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:54.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.654+0000 7f36a4e31700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36a0102450 0x7f36a019fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55610/0 (socket says 172.21.15.100:55610) 2024-05-10T11:36:54.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.654+0000 7f36a7896700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f36a01134e0 con 0x7f36a0102450 2024-05-10T11:36:54.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.654+0000 7f36a4e31700 1 -- 172.21.15.100:0/579980539 learned_addr learned my addr 172.21.15.100:0/579980539 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:36:54.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.654+0000 7f36a7896700 1 -- 172.21.15.100:0/579980539 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f36a0113360 con 0x7f36a0109520 2024-05-10T11:36:54.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.654+0000 7f36a7896700 1 -- 172.21.15.100:0/579980539 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f36a0113660 con 0x7f36a0101a90 2024-05-10T11:36:54.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.655+0000 7f36a5e33700 1 --2- 172.21.15.100:0/579980539 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36a0109520 0x7f36a01998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:54.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.655+0000 7f36a5632700 1 --2- 172.21.15.100:0/579980539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36a0101a90 0x7f36a019f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:54.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.655+0000 7f36a4e31700 1 -- 172.21.15.100:0/579980539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36a0101a90 msgr2=0x7f36a019f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:54.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.655+0000 7f36a4e31700 1 --2- 172.21.15.100:0/579980539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36a0101a90 0x7f36a019f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:54.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.655+0000 7f36a4e31700 1 -- 172.21.15.100:0/579980539 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36a0109520 msgr2=0x7f36a01998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:54.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.655+0000 7f36a4e31700 1 --2- 172.21.15.100:0/579980539 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36a0109520 0x7f36a01998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:54.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.655+0000 7f36a4e31700 1 -- 172.21.15.100:0/579980539 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36a019a050 con 0x7f36a0102450 2024-05-10T11:36:54.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.655+0000 7f36a5632700 1 --2- 172.21.15.100:0/579980539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36a0101a90 0x7f36a019f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:36:54.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.655+0000 7f36a4e31700 1 --2- 172.21.15.100:0/579980539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36a0102450 0x7f36a019fc70 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f369000ec10 tx=0x7f369000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:36:54.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.655+0000 7f36967fc700 1 -- 172.21.15.100:0/579980539 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f369000cde0 con 0x7f36a0102450 2024-05-10T11:36:54.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.655+0000 7f36a7896700 1 -- 172.21.15.100:0/579980539 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36a019a340 con 0x7f36a0102450 2024-05-10T11:36:54.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.655+0000 7f36967fc700 1 -- 172.21.15.100:0/579980539 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f36900044b0 con 0x7f36a0102450 2024-05-10T11:36:54.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.656+0000 7f36a7896700 1 -- 172.21.15.100:0/579980539 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36a01a6600 con 0x7f36a0102450 2024-05-10T11:36:54.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.656+0000 7f36967fc700 1 -- 172.21.15.100:0/579980539 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3690010450 con 0x7f36a0102450 2024-05-10T11:36:54.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.659+0000 7f36967fc700 1 -- 172.21.15.100:0/579980539 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f3690010710 con 0x7f36a0102450 2024-05-10T11:36:54.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.660+0000 7f36967fc700 1 --2- 172.21.15.100:0/579980539 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f368c0b2880 0x7f368c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:54.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.660+0000 7f36b2129b80 1 Processor -- start 2024-05-10T11:36:54.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.660+0000 7f36b2129b80 1 -- start start 2024-05-10T11:36:54.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.660+0000 7f36a5632700 1 --2- 172.21.15.100:0/579980539 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f368c0b2880 0x7f368c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:54.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.660+0000 7f36967fc700 1 -- 172.21.15.100:0/579980539 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f36900e2a20 con 0x7f36a0102450 2024-05-10T11:36:54.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.661+0000 7f36b2129b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d7cb894d0 0x564d7cb898b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:54.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.661+0000 7f36b2129b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d7cb89df0 0x564d7cb8d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:54.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.661+0000 7f36b2129b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564d7cb8dc80 0x564d7cb8e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:54.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.661+0000 7f36b2129b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x564d7cadb860 con 0x564d7cb894d0 2024-05-10T11:36:54.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.661+0000 7f36b2129b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x564d7ca7a7c0 con 0x564d7cb8dc80 2024-05-10T11:36:54.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.661+0000 7f36b2129b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x564d7cad7820 con 0x564d7cb89df0 2024-05-10T11:36:54.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.661+0000 7f36a5632700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d7cb894d0 0x564d7cb898b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:54.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.661+0000 7f36a5632700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d7cb894d0 0x564d7cb898b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55614/0 (socket says 172.21.15.100:55614) 2024-05-10T11:36:54.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.661+0000 7f36a5632700 1 -- 172.21.15.100:0/735721667 learned_addr learned my addr 172.21.15.100:0/735721667 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:36:54.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.661+0000 7f36a5e33700 1 --2- 172.21.15.100:0/735721667 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564d7cb8dc80 0x564d7cb8e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:54.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.661+0000 7f36a4e31700 1 --2- 172.21.15.100:0/735721667 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d7cb89df0 0x564d7cb8d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:54.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.661+0000 7f36a5e33700 1 -- 172.21.15.100:0/735721667 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d7cb89df0 msgr2=0x564d7cb8d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:54.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.661+0000 7f36a5e33700 1 --2- 172.21.15.100:0/735721667 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d7cb89df0 0x564d7cb8d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:54.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.661+0000 7f36a5e33700 1 -- 172.21.15.100:0/735721667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d7cb894d0 msgr2=0x564d7cb898b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:54.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.661+0000 7f36a5e33700 1 --2- 172.21.15.100:0/735721667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d7cb894d0 0x564d7cb898b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:54.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.661+0000 7f36a5e33700 1 -- 172.21.15.100:0/735721667 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d7ca288d0 con 0x564d7cb8dc80 2024-05-10T11:36:54.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.661+0000 7f36a4e31700 1 --2- 172.21.15.100:0/735721667 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d7cb89df0 0x564d7cb8d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:36:54.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.661+0000 7f36a5e33700 1 --2- 172.21.15.100:0/735721667 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564d7cb8dc80 0x564d7cb8e100 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3698009310 tx=0x7f3698009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a87697af6211a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:36:54.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.661+0000 7f36a5632700 1 --2- 172.21.15.100:0/735721667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d7cb894d0 0x564d7cb898b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:36:54.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.662+0000 7f368b7fe700 1 -- 172.21.15.100:0/735721667 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3698019020 con 0x564d7cb8dc80 2024-05-10T11:36:54.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.662+0000 7f368b7fe700 1 -- 172.21.15.100:0/735721667 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3698015ad0 con 0x564d7cb8dc80 2024-05-10T11:36:54.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.662+0000 7f368b7fe700 1 -- 172.21.15.100:0/735721667 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3698014980 con 0x564d7cb8dc80 2024-05-10T11:36:54.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.663+0000 7f36b2129b80 1 -- 172.21.15.100:0/735721667 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564d7cb8dc80 msgr2=0x564d7cb8e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:54.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.663+0000 7f36b2129b80 1 --2- 172.21.15.100:0/735721667 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564d7cb8dc80 0x564d7cb8e100 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3698009310 tx=0x7f3698009920 comp rx=0 tx=0).stop 2024-05-10T11:36:54.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.663+0000 7f36a5632700 1 --2- 172.21.15.100:0/579980539 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f368c0b2880 0x7f368c0b4d40 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f369c005e90 tx=0x7f369c005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:36:54.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.663+0000 7f36b2129b80 1 -- 172.21.15.100:0/735721667 shutdown_connections 2024-05-10T11:36:54.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.663+0000 7f36b2129b80 1 --2- 172.21.15.100:0/735721667 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564d7cb8dc80 0x564d7cb8e100 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:54.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.663+0000 7f36b2129b80 1 --2- 172.21.15.100:0/735721667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d7cb894d0 0x564d7cb898b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:54.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.663+0000 7f36b2129b80 1 --2- 172.21.15.100:0/735721667 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d7cb89df0 0x564d7cb8d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:54.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.663+0000 7f36b2129b80 1 -- 172.21.15.100:0/735721667 >> 172.21.15.100:0/735721667 conn(0x564d7cab5100 msgr2=0x564d7cb83c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:36:54.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.663+0000 7f36b2129b80 1 -- 172.21.15.100:0/735721667 shutdown_connections 2024-05-10T11:36:54.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.663+0000 7f36b2129b80 1 -- 172.21.15.100:0/735721667 wait complete. 2024-05-10T11:36:54.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.664+0000 7f36b2129b80 1 -- start start 2024-05-10T11:36:54.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.664+0000 7f36b2129b80 1 Processor -- start 2024-05-10T11:36:54.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.665+0000 7f36b2129b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d7cb894d0 0x564d7cb95120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:54.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.665+0000 7f36b2129b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564d7cb89df0 0x564d7cb924c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:54.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.665+0000 7f36b2129b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d7cb8dc80 0x564d7cb92a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:54.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.665+0000 7f36b2129b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x564d7cb95e90 con 0x564d7cb894d0 2024-05-10T11:36:54.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.665+0000 7f36b2129b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x564d7ca41c30 con 0x564d7cb89df0 2024-05-10T11:36:54.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.665+0000 7f36b2129b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x564d7cb96010 con 0x564d7cb8dc80 2024-05-10T11:36:54.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.665+0000 7f36a4e31700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564d7cb89df0 0x564d7cb924c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:54.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.665+0000 7f36a4e31700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564d7cb89df0 0x564d7cb924c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:57602/0 (socket says 172.21.15.100:57602) 2024-05-10T11:36:54.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.665+0000 7f36a4e31700 1 -- 172.21.15.100:0/2236164881 learned_addr learned my addr 172.21.15.100:0/2236164881 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:36:54.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.665+0000 7f36b2129b80 10 client.0 _ll_register_callbacks cb 0x564d7ca73920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:36:54.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.665+0000 7f36a5e33700 1 --2- 172.21.15.100:0/2236164881 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d7cb8dc80 0x564d7cb92a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:54.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.665+0000 7f36a5632700 1 --2- 172.21.15.100:0/2236164881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d7cb894d0 0x564d7cb95120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:54.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.666+0000 7f36a4e31700 1 -- 172.21.15.100:0/2236164881 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d7cb8dc80 msgr2=0x564d7cb92a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:54.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.666+0000 7f36a4e31700 1 --2- 172.21.15.100:0/2236164881 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d7cb8dc80 0x564d7cb92a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:54.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.666+0000 7f36a4e31700 1 -- 172.21.15.100:0/2236164881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d7cb894d0 msgr2=0x564d7cb95120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:54.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.666+0000 7f36a4e31700 1 --2- 172.21.15.100:0/2236164881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d7cb894d0 0x564d7cb95120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:54.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.666+0000 7f36a4e31700 1 -- 172.21.15.100:0/2236164881 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d7ca77690 con 0x564d7cb89df0 2024-05-10T11:36:54.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.666+0000 7f36a4e31700 1 --2- 172.21.15.100:0/2236164881 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564d7cb89df0 0x564d7cb924c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f36900e0210 tx=0x7f36900039d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56c45f7c11a16d71 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:36:54.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.666+0000 7f3688ff9700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:36:54.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.667+0000 7f3688ff9700 1 -- 172.21.15.100:0/2236164881 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36900c43c0 con 0x564d7cb89df0 2024-05-10T11:36:54.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.667+0000 7f3688ff9700 1 -- 172.21.15.100:0/2236164881 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f36900a62d0 con 0x564d7cb89df0 2024-05-10T11:36:54.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.667+0000 7f36b2129b80 1 -- 172.21.15.100:0/2236164881 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3698015ad0 con 0x564d7cb89df0 2024-05-10T11:36:54.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.668+0000 7f3688ff9700 1 -- 172.21.15.100:0/2236164881 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f36900a9ce0 con 0x564d7cb89df0 2024-05-10T11:36:54.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.668+0000 7f36b2129b80 10 client.28355 fetch_fsmap learned FSMap version 41 2024-05-10T11:36:54.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.668+0000 7f36b2129b80 1 -- 172.21.15.100:0/2236164881 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x564d7cba0730 con 0x564d7cb89df0 2024-05-10T11:36:54.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.668+0000 7f3688ff9700 1 -- 172.21.15.100:0/2236164881 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3698015ad0 con 0x564d7cb89df0 2024-05-10T11:36:54.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.668+0000 7f3688ff9700 1 -- 172.21.15.100:0/2236164881 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f3690004040 con 0x564d7cb89df0 2024-05-10T11:36:54.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.668+0000 7f36b2129b80 10 client.28355 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:36:54.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.668+0000 7f36b2129b80 10 client.28355 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:36:54.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.668+0000 7f36b2129b80 20 client.28355 populate_metadata read hostname 'smithi100' 2024-05-10T11:36:54.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.668+0000 7f36b2129b80 1 --2- 172.21.15.100:0/2236164881 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x564d7cba2cc0 0x564d7cba5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:54.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.668+0000 7f36b2129b80 4 client.28355 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:36:54.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.668+0000 7f36b2129b80 1 -- 172.21.15.100:0/2236164881 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d7cb55650 con 0x564d7cba2cc0 2024-05-10T11:36:54.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.669+0000 7f36a5632700 1 --2- 172.21.15.100:0/2236164881 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x564d7cba2cc0 0x564d7cba5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:54.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.669+0000 7f3688ff9700 10 client.28355 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:36:54.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.669+0000 7f36a5632700 1 --2- 172.21.15.100:0/2236164881 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x564d7cba2cc0 0x564d7cba5100 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d74c455fc38f9b15 server_cookie=d580584ee44d5f53 in_seq=0 out_seq=0 2024-05-10T11:36:54.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.670+0000 7f3688ff9700 1 -- 172.21.15.100:0/2236164881 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564d7cb55650 con 0x564d7cba2cc0 2024-05-10T11:36:54.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.670+0000 7f3688ff9700 10 client.28355 handle_command_reply: tid=0 2024-05-10T11:36:54.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.670+0000 7f36b2129b80 1 client.28355 shutdown 2024-05-10T11:36:54.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.670+0000 7f36b2129b80 1 -- 172.21.15.100:0/2236164881 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564d7cb89df0 msgr2=0x564d7cb924c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:54.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.670+0000 7f36b2129b80 1 --2- 172.21.15.100:0/2236164881 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564d7cb89df0 0x564d7cb924c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f36900e0210 tx=0x7f36900039d0 comp rx=0 tx=0).stop 2024-05-10T11:36:54.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.670+0000 7f36b2129b80 1 -- 172.21.15.100:0/2236164881 shutdown_connections 2024-05-10T11:36:54.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.670+0000 7f36b2129b80 1 --2- 172.21.15.100:0/2236164881 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x564d7cba2cc0 0x564d7cba5100 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:54.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.670+0000 7f3688ff9700 0 client.28355 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:36:54.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.670+0000 7f36b2129b80 1 --2- 172.21.15.100:0/2236164881 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d7cb8dc80 0x564d7cb92a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:54.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.670+0000 7f36b2129b80 1 --2- 172.21.15.100:0/2236164881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d7cb894d0 0x564d7cb95120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:54.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.670+0000 7f36b2129b80 1 --2- 172.21.15.100:0/2236164881 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564d7cb89df0 0x564d7cb924c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:54.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.670+0000 7f36b2129b80 1 -- 172.21.15.100:0/2236164881 >> 172.21.15.100:0/2236164881 conn(0x564d7cab5100 msgr2=0x564d7cb8e080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:36:54.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.670+0000 7f36b2129b80 1 -- 172.21.15.100:0/2236164881 shutdown_connections 2024-05-10T11:36:54.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.671+0000 7f36b2129b80 1 -- 172.21.15.100:0/2236164881 wait complete. 2024-05-10T11:36:54.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.671+0000 7f36b2129b80 20 client.28355 trim_cache size 0 max 16384 2024-05-10T11:36:54.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.698+0000 7f36b2129b80 1 Processor -- start 2024-05-10T11:36:54.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.698+0000 7f36b2129b80 1 -- start start 2024-05-10T11:36:54.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.698+0000 7f36b2129b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d7cb894d0 0x564d7cb9ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:54.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.698+0000 7f36b2129b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d7cb89df0 0x564d7cb83ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:54.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.698+0000 7f36b2129b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564d7cb8dc80 0x564d7cbaa9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:54.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.699+0000 7f36b2129b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f36a01b3ce0 con 0x564d7cb89df0 2024-05-10T11:36:54.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.699+0000 7f36b2129b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f36a01b1570 con 0x564d7cb8dc80 2024-05-10T11:36:54.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.699+0000 7f36b2129b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f36a01af3c0 con 0x564d7cb894d0 2024-05-10T11:36:54.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.700+0000 7f36a5e33700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564d7cb8dc80 0x564d7cbaa9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:54.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.700+0000 7f36a5632700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d7cb894d0 0x564d7cb9ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:54.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.700+0000 7f36a5e33700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564d7cb8dc80 0x564d7cbaa9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:57604/0 (socket says 172.21.15.100:57604) 2024-05-10T11:36:54.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.700+0000 7f36a5e33700 1 -- 172.21.15.100:0/3173601135 learned_addr learned my addr 172.21.15.100:0/3173601135 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:36:54.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.700+0000 7f36a4e31700 1 --2- 172.21.15.100:0/3173601135 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d7cb89df0 0x564d7cb83ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:54.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.700+0000 7f36a5e33700 1 -- 172.21.15.100:0/3173601135 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d7cb894d0 msgr2=0x564d7cb9ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:54.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.700+0000 7f36a5e33700 1 --2- 172.21.15.100:0/3173601135 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d7cb894d0 0x564d7cb9ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:54.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.700+0000 7f36a5e33700 1 -- 172.21.15.100:0/3173601135 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d7cb89df0 msgr2=0x564d7cb83ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:54.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.700+0000 7f36a5e33700 1 --2- 172.21.15.100:0/3173601135 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d7cb89df0 0x564d7cb83ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:54.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.700+0000 7f36a5e33700 1 -- 172.21.15.100:0/3173601135 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d7cb93370 con 0x564d7cb8dc80 2024-05-10T11:36:54.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.700+0000 7f36a4e31700 1 --2- 172.21.15.100:0/3173601135 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d7cb89df0 0x564d7cb83ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:36:54.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.701+0000 7f36a5e33700 1 --2- 172.21.15.100:0/3173601135 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564d7cb8dc80 0x564d7cbaa9d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f369801a000 tx=0x7f3698009df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c23729425e128e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:36:54.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.701+0000 7f368affd700 1 -- 172.21.15.100:0/3173601135 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3698019070 con 0x564d7cb8dc80 2024-05-10T11:36:54.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.701+0000 7f368affd700 1 -- 172.21.15.100:0/3173601135 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f36980095e0 con 0x564d7cb8dc80 2024-05-10T11:36:54.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.701+0000 7f368affd700 1 -- 172.21.15.100:0/3173601135 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3698014880 con 0x564d7cb8dc80 2024-05-10T11:36:54.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.702+0000 7f36b2129b80 1 -- 172.21.15.100:0/3173601135 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564d7cb8dc80 msgr2=0x564d7cbaa9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:54.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.702+0000 7f36b2129b80 1 --2- 172.21.15.100:0/3173601135 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564d7cb8dc80 0x564d7cbaa9d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f369801a000 tx=0x7f3698009df0 comp rx=0 tx=0).stop 2024-05-10T11:36:54.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.702+0000 7f36b2129b80 1 -- 172.21.15.100:0/3173601135 shutdown_connections 2024-05-10T11:36:54.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.702+0000 7f36b2129b80 1 --2- 172.21.15.100:0/3173601135 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564d7cb8dc80 0x564d7cbaa9d0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:54.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.702+0000 7f36b2129b80 1 --2- 172.21.15.100:0/3173601135 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d7cb894d0 0x564d7cb9ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:54.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.702+0000 7f36b2129b80 1 --2- 172.21.15.100:0/3173601135 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d7cb89df0 0x564d7cb83ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:54.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.702+0000 7f36b2129b80 1 -- 172.21.15.100:0/3173601135 >> 172.21.15.100:0/3173601135 conn(0x564d7cab5100 msgr2=0x564d7cb94f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:36:54.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.702+0000 7f36b2129b80 1 -- 172.21.15.100:0/3173601135 shutdown_connections 2024-05-10T11:36:54.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.702+0000 7f36b2129b80 1 -- 172.21.15.100:0/3173601135 wait complete. 2024-05-10T11:36:54.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.703+0000 7f36b2129b80 1 -- start start 2024-05-10T11:36:54.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.703+0000 7f36b2129b80 1 Processor -- start 2024-05-10T11:36:54.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.703+0000 7f36b2129b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564d7cb894d0 0x564d7cb98530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:54.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.703+0000 7f36b2129b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d7cb89df0 0x564d7cb98a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:54.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.703+0000 7f36b2129b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d7cb8dc80 0x564d7cc0b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:54.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.703+0000 7f36a5632700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564d7cb894d0 0x564d7cb98530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:54.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.703+0000 7f36b2129b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f36a01afa30 con 0x564d7cb8dc80 2024-05-10T11:36:54.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.703+0000 7f36b2129b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f36a01b0ea0 con 0x564d7cb894d0 2024-05-10T11:36:54.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.703+0000 7f36a5632700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564d7cb894d0 0x564d7cb98530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:57616/0 (socket says 172.21.15.100:57616) 2024-05-10T11:36:54.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.703+0000 7f36a5632700 1 -- 172.21.15.100:0/3476749185 learned_addr learned my addr 172.21.15.100:0/3476749185 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:36:54.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.703+0000 7f36b2129b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f36a01b0100 con 0x564d7cb89df0 2024-05-10T11:36:54.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.704+0000 7f36b2129b80 10 client.0 _ll_register_callbacks cb 0x564d7ca73920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:36:54.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.704+0000 7f36a5e33700 1 --2- 172.21.15.100:0/3476749185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d7cb8dc80 0x564d7cc0b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:54.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.704+0000 7f36a4e31700 1 --2- 172.21.15.100:0/3476749185 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d7cb89df0 0x564d7cb98a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:54.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.704+0000 7f36a5632700 1 -- 172.21.15.100:0/3476749185 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d7cb89df0 msgr2=0x564d7cb98a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:54.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.704+0000 7f36a5632700 1 --2- 172.21.15.100:0/3476749185 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d7cb89df0 0x564d7cb98a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:54.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.704+0000 7f36a5632700 1 -- 172.21.15.100:0/3476749185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d7cb8dc80 msgr2=0x564d7cc0b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:54.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.704+0000 7f36a5632700 1 --2- 172.21.15.100:0/3476749185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d7cb8dc80 0x564d7cc0b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:54.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.704+0000 7f36a5632700 1 -- 172.21.15.100:0/3476749185 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d7cc0bac0 con 0x564d7cb894d0 2024-05-10T11:36:54.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.704+0000 7f36a5e33700 1 --2- 172.21.15.100:0/3476749185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d7cb8dc80 0x564d7cc0b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:36:54.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.704+0000 7f36a5632700 1 --2- 172.21.15.100:0/3476749185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564d7cb894d0 0x564d7cb98530 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f369c0098c0 tx=0x7f369c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8d2fd54e4a28458 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:36:54.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.704+0000 7f3688ff9700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:36:54.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.704+0000 7f3688ff9700 1 -- 172.21.15.100:0/3476749185 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f369c036030 con 0x564d7cb894d0 2024-05-10T11:36:54.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.704+0000 7f36b2129b80 1 -- 172.21.15.100:0/3476749185 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36980095e0 con 0x564d7cb894d0 2024-05-10T11:36:54.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.704+0000 7f3688ff9700 1 -- 172.21.15.100:0/3476749185 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f369c0323a0 con 0x564d7cb894d0 2024-05-10T11:36:54.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.704+0000 7f3688ff9700 1 -- 172.21.15.100:0/3476749185 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f369c043050 con 0x564d7cb894d0 2024-05-10T11:36:54.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.704+0000 7f36b2129b80 10 client.28361 fetch_fsmap learned FSMap version 41 2024-05-10T11:36:54.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.705+0000 7f36b2129b80 1 -- 172.21.15.100:0/3476749185 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x564d7cc0cde0 con 0x564d7cb894d0 2024-05-10T11:36:54.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.705+0000 7f3688ff9700 1 -- 172.21.15.100:0/3476749185 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f36980095e0 con 0x564d7cb894d0 2024-05-10T11:36:54.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.705+0000 7f3688ff9700 1 -- 172.21.15.100:0/3476749185 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f369c026040 con 0x564d7cb894d0 2024-05-10T11:36:54.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.705+0000 7f36b2129b80 10 client.28361 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:36:54.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.705+0000 7f36b2129b80 10 client.28361 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:36:54.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.705+0000 7f36b2129b80 20 client.28361 populate_metadata read hostname 'smithi100' 2024-05-10T11:36:54.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.705+0000 7f36b2129b80 1 --2- 172.21.15.100:0/3476749185 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x564d7cba2cc0 0x564d7cc0f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:54.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.705+0000 7f36b2129b80 4 client.28361 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:36:54.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.705+0000 7f36b2129b80 1 -- 172.21.15.100:0/3476749185 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564d7cae9240 con 0x564d7cba2cc0 2024-05-10T11:36:54.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.705+0000 7f36a4e31700 1 --2- 172.21.15.100:0/3476749185 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x564d7cba2cc0 0x564d7cc0f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:54.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.706+0000 7f3688ff9700 10 client.28361 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:36:54.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.706+0000 7f36a4e31700 1 --2- 172.21.15.100:0/3476749185 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x564d7cba2cc0 0x564d7cc0f4b0 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9aaf80d185a8db45 server_cookie=3a81ef3f66fd662b in_seq=0 out_seq=0 2024-05-10T11:36:54.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.707+0000 7f3688ff9700 1 -- 172.21.15.100:0/3476749185 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564d7cae9240 con 0x564d7cba2cc0 2024-05-10T11:36:54.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.707+0000 7f3688ff9700 10 client.28361 handle_command_reply: tid=0 2024-05-10T11:36:54.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.707+0000 7f36b2129b80 1 client.28361 shutdown 2024-05-10T11:36:54.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.707+0000 7f36b2129b80 1 -- 172.21.15.100:0/3476749185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564d7cb894d0 msgr2=0x564d7cb98530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:54.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.707+0000 7f36b2129b80 1 --2- 172.21.15.100:0/3476749185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564d7cb894d0 0x564d7cb98530 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f369c0098c0 tx=0x7f369c009260 comp rx=0 tx=0).stop 2024-05-10T11:36:54.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.707+0000 7f36b2129b80 1 -- 172.21.15.100:0/3476749185 shutdown_connections 2024-05-10T11:36:54.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.707+0000 7f36b2129b80 1 --2- 172.21.15.100:0/3476749185 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x564d7cba2cc0 0x564d7cc0f4b0 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:54.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.707+0000 7f36b2129b80 1 --2- 172.21.15.100:0/3476749185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d7cb8dc80 0x564d7cc0b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:54.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.707+0000 7f36b2129b80 1 --2- 172.21.15.100:0/3476749185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564d7cb894d0 0x564d7cb98530 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:54.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.707+0000 7f36b2129b80 1 --2- 172.21.15.100:0/3476749185 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d7cb89df0 0x564d7cb98a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:54.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.707+0000 7f36b2129b80 1 -- 172.21.15.100:0/3476749185 >> 172.21.15.100:0/3476749185 conn(0x564d7cab5100 msgr2=0x564d7cb90d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:36:54.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.707+0000 7f3688ff9700 0 client.28361 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:36:54.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.707+0000 7f36b2129b80 1 -- 172.21.15.100:0/3476749185 shutdown_connections 2024-05-10T11:36:54.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.707+0000 7f36b2129b80 1 -- 172.21.15.100:0/3476749185 wait complete. 2024-05-10T11:36:54.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.707+0000 7f36b2129b80 20 client.28361 trim_cache size 0 max 16384 2024-05-10T11:36:54.711 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:36:54.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.708+0000 7f36a7896700 1 -- 172.21.15.100:0/579980539 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f368c0b2880 msgr2=0x7f368c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:54.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.708+0000 7f36a7896700 1 --2- 172.21.15.100:0/579980539 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f368c0b2880 0x7f368c0b4d40 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f369c005e90 tx=0x7f369c005dc0 comp rx=0 tx=0).stop 2024-05-10T11:36:54.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.708+0000 7f36a7896700 1 -- 172.21.15.100:0/579980539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36a0102450 msgr2=0x7f36a019fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:54.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.708+0000 7f36a7896700 1 --2- 172.21.15.100:0/579980539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36a0102450 0x7f36a019fc70 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f369000ec10 tx=0x7f369000c2d0 comp rx=0 tx=0).stop 2024-05-10T11:36:54.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.708+0000 7f36a7896700 1 -- 172.21.15.100:0/579980539 shutdown_connections 2024-05-10T11:36:54.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.708+0000 7f36a7896700 1 --2- 172.21.15.100:0/579980539 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f368c0b2880 0x7f368c0b4d40 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:54.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.708+0000 7f36a7896700 1 --2- 172.21.15.100:0/579980539 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36a0109520 0x7f36a01998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:54.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.708+0000 7f36a7896700 1 --2- 172.21.15.100:0/579980539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36a0101a90 0x7f36a019f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:54.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.708+0000 7f36a7896700 1 --2- 172.21.15.100:0/579980539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36a0102450 0x7f36a019fc70 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:54.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.708+0000 7f36a7896700 1 -- 172.21.15.100:0/579980539 >> 172.21.15.100:0/579980539 conn(0x7f36a0078a10 msgr2=0x7f36a0100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:36:54.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.708+0000 7f36a7896700 1 -- 172.21.15.100:0/579980539 shutdown_connections 2024-05-10T11:36:54.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:54.708+0000 7f36a7896700 1 -- 172.21.15.100:0/579980539 wait complete. 2024-05-10T11:36:54.932 INFO:tasks.workunit.client.0.smithi100.stdout: 10 48440 9.00 MB/sec execute 168 sec latency 133.163 ms 2024-05-10T11:36:55.466 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.466+0000 7f11dbc62640 1 -- 172.21.15.100:0/3225439467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11d41034f0 msgr2=0x7f11d4109d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:55.467 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.466+0000 7f11dbc62640 1 --2- 172.21.15.100:0/3225439467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11d41034f0 0x7f11d4109d80 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f11cc009ca0 tx=0x7f11cc02a8c0 comp rx=0 tx=0).stop 2024-05-10T11:36:55.467 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.466+0000 7f11dbc62640 1 -- 172.21.15.100:0/3225439467 shutdown_connections 2024-05-10T11:36:55.467 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.466+0000 7f11dbc62640 1 --2- 172.21.15.100:0/3225439467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11d41034f0 0x7f11d4109d80 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:55.467 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.466+0000 7f11dbc62640 1 --2- 172.21.15.100:0/3225439467 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11d4102b30 0x7f11d4102fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:55.467 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.466+0000 7f11dbc62640 1 --2- 172.21.15.100:0/3225439467 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11d4101940 0x7f11d4101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:55.467 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.466+0000 7f11dbc62640 1 -- 172.21.15.100:0/3225439467 >> 172.21.15.100:0/3225439467 conn(0x7f11d40fceb0 msgr2=0x7f11d40ff2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:36:55.467 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.466+0000 7f11dbc62640 1 -- 172.21.15.100:0/3225439467 shutdown_connections 2024-05-10T11:36:55.467 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.466+0000 7f11dbc62640 1 -- 172.21.15.100:0/3225439467 wait complete. 2024-05-10T11:36:55.467 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.467+0000 7f11dbc62640 1 Processor -- start 2024-05-10T11:36:55.469 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.467+0000 7f11dbc62640 1 -- start start 2024-05-10T11:36:55.469 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.467+0000 7f11dbc62640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11d4101940 0x7f11d41979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:55.469 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.467+0000 7f11dbc62640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11d4102b30 0x7f11d4197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:55.469 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.467+0000 7f11dbc62640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11d41034f0 0x7f11d419ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:55.469 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.467+0000 7f11dbc62640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f11d410d370 con 0x7f11d4102b30 2024-05-10T11:36:55.469 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.467+0000 7f11dbc62640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f11d410d1f0 con 0x7f11d4101940 2024-05-10T11:36:55.469 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.467+0000 7f11dbc62640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f11d410d4f0 con 0x7f11d41034f0 2024-05-10T11:36:55.469 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.467+0000 7f11d99fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11d4101940 0x7f11d41979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:55.470 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.467+0000 7f11d91fd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11d4102b30 0x7f11d4197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:55.470 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.467+0000 7f11d99fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11d4101940 0x7f11d41979b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:57632/0 (socket says 172.21.15.100:57632) 2024-05-10T11:36:55.470 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.467+0000 7f11d99fe640 1 -- 172.21.15.100:0/893987298 learned_addr learned my addr 172.21.15.100:0/893987298 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:36:55.470 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.467+0000 7f11da1ff640 1 --2- 172.21.15.100:0/893987298 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11d41034f0 0x7f11d419ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:55.470 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.468+0000 7f11d99fe640 1 -- 172.21.15.100:0/893987298 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11d41034f0 msgr2=0x7f11d419ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:55.470 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.468+0000 7f11d99fe640 1 --2- 172.21.15.100:0/893987298 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11d41034f0 0x7f11d419ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:55.470 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.468+0000 7f11d99fe640 1 -- 172.21.15.100:0/893987298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11d4102b30 msgr2=0x7f11d4197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:55.470 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.468+0000 7f11d99fe640 1 --2- 172.21.15.100:0/893987298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11d4102b30 0x7f11d4197ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:55.470 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.468+0000 7f11d99fe640 1 -- 172.21.15.100:0/893987298 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11d419f5d0 con 0x7f11d4101940 2024-05-10T11:36:55.470 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.468+0000 7f11d91fd640 1 --2- 172.21.15.100:0/893987298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11d4102b30 0x7f11d4197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:36:55.470 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.468+0000 7f11d99fe640 1 --2- 172.21.15.100:0/893987298 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11d4101940 0x7f11d41979b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f11d000ead0 tx=0x7f11d000ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:36:55.470 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.468+0000 7f11caffd640 1 -- 172.21.15.100:0/893987298 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d000bbd0 con 0x7f11d4101940 2024-05-10T11:36:55.470 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.468+0000 7f11caffd640 1 -- 172.21.15.100:0/893987298 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f11d00044b0 con 0x7f11d4101940 2024-05-10T11:36:55.471 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.468+0000 7f11dbc62640 1 -- 172.21.15.100:0/893987298 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11d406b240 con 0x7f11d4101940 2024-05-10T11:36:55.471 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.468+0000 7f11caffd640 1 -- 172.21.15.100:0/893987298 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d0003de0 con 0x7f11d4101940 2024-05-10T11:36:55.471 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.468+0000 7f11dbc62640 1 -- 172.21.15.100:0/893987298 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11d406b790 con 0x7f11d4101940 2024-05-10T11:36:55.471 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.471+0000 7f11caffd640 1 -- 172.21.15.100:0/893987298 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f11d001e020 con 0x7f11d4101940 2024-05-10T11:36:55.471 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.471+0000 7f11de976740 1 Processor -- start 2024-05-10T11:36:55.472 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.472+0000 7f11de976740 1 -- start start 2024-05-10T11:36:55.472 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.472+0000 7f11de976740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55edd2db71c0 0x55edd2db75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:55.473 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.472+0000 7f11de976740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55edd2db7bb0 0x55edd2dbc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:55.473 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.472+0000 7f11de976740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55edd2dbcd20 0x55edd2dbf1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:55.473 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.472+0000 7f11de976740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55edd2d175d0 con 0x55edd2db71c0 2024-05-10T11:36:55.473 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.472+0000 7f11de976740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55edd2cd0890 con 0x55edd2dbcd20 2024-05-10T11:36:55.473 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.472+0000 7f11de976740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55edd2d13590 con 0x55edd2db7bb0 2024-05-10T11:36:55.473 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.472+0000 7f11d99fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55edd2db7bb0 0x55edd2dbc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:55.473 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.472+0000 7f11d91fd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55edd2dbcd20 0x55edd2dbf1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:55.473 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.472+0000 7f11d99fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55edd2db7bb0 0x55edd2dbc6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:59194/0 (socket says 172.21.15.100:59194) 2024-05-10T11:36:55.473 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.472+0000 7f11d99fe640 1 -- 172.21.15.100:0/2884053603 learned_addr learned my addr 172.21.15.100:0/2884053603 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:36:55.473 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.472+0000 7f11d91fd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55edd2dbcd20 0x55edd2dbf1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:57648/0 (socket says 172.21.15.100:57648) 2024-05-10T11:36:55.473 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.472+0000 7f11da1ff640 1 --2- 172.21.15.100:0/2884053603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55edd2db71c0 0x55edd2db75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:55.473 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.472+0000 7f11d99fe640 1 -- 172.21.15.100:0/2884053603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55edd2dbcd20 msgr2=0x55edd2dbf1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:55.473 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.472+0000 7f11d99fe640 1 --2- 172.21.15.100:0/2884053603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55edd2dbcd20 0x55edd2dbf1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:55.474 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.472+0000 7f11d99fe640 1 -- 172.21.15.100:0/2884053603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55edd2db71c0 msgr2=0x55edd2db75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:55.474 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.472+0000 7f11d99fe640 1 --2- 172.21.15.100:0/2884053603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55edd2db71c0 0x55edd2db75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:55.474 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.472+0000 7f11d99fe640 1 -- 172.21.15.100:0/2884053603 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edd2a29210 con 0x55edd2db7bb0 2024-05-10T11:36:55.474 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.473+0000 7f11d99fe640 1 --2- 172.21.15.100:0/2884053603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55edd2db7bb0 0x55edd2dbc6b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f11d00e59d0 tx=0x7f11d00e5ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcda1f39b50b8a44 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:36:55.475 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.473+0000 7f11bffff640 1 -- 172.21.15.100:0/2884053603 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d00f5070 con 0x55edd2db7bb0 2024-05-10T11:36:55.475 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.473+0000 7f11caffd640 1 --2- 172.21.15.100:0/893987298 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f11c00b28a0 0x7f11c00b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:55.475 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.473+0000 7f11bffff640 1 -- 172.21.15.100:0/2884053603 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f11d00f0910 con 0x55edd2db7bb0 2024-05-10T11:36:55.476 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.473+0000 7f11bffff640 1 -- 172.21.15.100:0/2884053603 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d00ea7b0 con 0x55edd2db7bb0 2024-05-10T11:36:55.476 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.473+0000 7f11da1ff640 1 --2- 172.21.15.100:0/893987298 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f11c00b28a0 0x7f11c00b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:55.476 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.473+0000 7f11caffd640 1 -- 172.21.15.100:0/893987298 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f11d00e3c50 con 0x7f11d4101940 2024-05-10T11:36:55.476 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.474+0000 7f11de976740 1 -- 172.21.15.100:0/2884053603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55edd2db7bb0 msgr2=0x55edd2dbc6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:55.477 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.474+0000 7f11de976740 1 --2- 172.21.15.100:0/2884053603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55edd2db7bb0 0x55edd2dbc6b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f11d00e59d0 tx=0x7f11d00e5ac0 comp rx=0 tx=0).stop 2024-05-10T11:36:55.477 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.474+0000 7f11da1ff640 1 --2- 172.21.15.100:0/893987298 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f11c00b28a0 0x7f11c00b4d60 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f11cc006010 tx=0x7f11cc009360 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:36:55.477 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.474+0000 7f11de976740 1 -- 172.21.15.100:0/2884053603 shutdown_connections 2024-05-10T11:36:55.477 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.474+0000 7f11de976740 1 --2- 172.21.15.100:0/2884053603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55edd2dbcd20 0x55edd2dbf1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:55.477 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.474+0000 7f11de976740 1 --2- 172.21.15.100:0/2884053603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55edd2db7bb0 0x55edd2dbc6b0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:55.477 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.474+0000 7f11de976740 1 --2- 172.21.15.100:0/2884053603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55edd2db71c0 0x55edd2db75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:55.477 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.474+0000 7f11de976740 1 -- 172.21.15.100:0/2884053603 >> 172.21.15.100:0/2884053603 conn(0x55edd2dae420 msgr2=0x55edd2dae820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:36:55.477 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.475+0000 7f11de976740 1 -- 172.21.15.100:0/2884053603 shutdown_connections 2024-05-10T11:36:55.477 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.475+0000 7f11de976740 1 -- 172.21.15.100:0/2884053603 wait complete. 2024-05-10T11:36:55.477 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.475+0000 7f11de976740 1 -- start start 2024-05-10T11:36:55.477 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.475+0000 7f11de976740 1 Processor -- start 2024-05-10T11:36:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.476+0000 7f11de976740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55edd2db71c0 0x55edd2dc1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.476+0000 7f11de976740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55edd2db7bb0 0x55edd2dc1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.476+0000 7f11de976740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55edd2dbcd20 0x55edd2dc8980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.476+0000 7f11de976740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55edd2dbfcd0 con 0x55edd2db71c0 2024-05-10T11:36:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.476+0000 7f11de976740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55edd2cd4370 con 0x55edd2dbcd20 2024-05-10T11:36:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.476+0000 7f11de976740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55edd2dbfe50 con 0x55edd2db7bb0 2024-05-10T11:36:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.476+0000 7f11de976740 10 client.0 _ll_register_callbacks cb 0x55edd2a28a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:36:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.476+0000 7f11d99fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55edd2db71c0 0x55edd2dc1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:55.483 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.476+0000 7f11d99fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55edd2db71c0 0x55edd2dc1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55664/0 (socket says 172.21.15.100:55664) 2024-05-10T11:36:55.483 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.476+0000 7f11d99fe640 1 -- 172.21.15.100:0/3936071869 learned_addr learned my addr 172.21.15.100:0/3936071869 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:36:55.483 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.476+0000 7f11da1ff640 1 --2- 172.21.15.100:0/3936071869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55edd2dbcd20 0x55edd2dc8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:55.483 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.476+0000 7f11d91fd640 1 --2- 172.21.15.100:0/3936071869 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55edd2db7bb0 0x55edd2dc1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:55.483 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.476+0000 7f11d99fe640 1 -- 172.21.15.100:0/3936071869 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55edd2db7bb0 msgr2=0x55edd2dc1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:55.483 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.476+0000 7f11d99fe640 1 --2- 172.21.15.100:0/3936071869 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55edd2db7bb0 0x55edd2dc1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:55.483 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.476+0000 7f11d99fe640 1 -- 172.21.15.100:0/3936071869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55edd2dbcd20 msgr2=0x55edd2dc8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:55.483 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.476+0000 7f11d99fe640 1 --2- 172.21.15.100:0/3936071869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55edd2dbcd20 0x55edd2dc8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:55.483 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.476+0000 7f11d99fe640 1 -- 172.21.15.100:0/3936071869 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edd2af5620 con 0x55edd2db71c0 2024-05-10T11:36:55.483 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.477+0000 7f11d99fe640 1 --2- 172.21.15.100:0/3936071869 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55edd2db71c0 0x55edd2dc1400 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f11d00aac10 tx=0x7f11d000bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eddebdd33d899e1f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:36:55.483 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.477+0000 7f11bd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:36:55.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.477+0000 7f11bd7fa640 1 -- 172.21.15.100:0/3936071869 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d00a6dd0 con 0x55edd2db71c0 2024-05-10T11:36:55.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.477+0000 7f11bd7fa640 1 -- 172.21.15.100:0/3936071869 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f11d00ab170 con 0x55edd2db71c0 2024-05-10T11:36:55.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.477+0000 7f11de976740 1 -- 172.21.15.100:0/3936071869 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11d00f0910 con 0x55edd2db71c0 2024-05-10T11:36:55.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.477+0000 7f11bd7fa640 1 -- 172.21.15.100:0/3936071869 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f11d00aa680 con 0x55edd2db71c0 2024-05-10T11:36:55.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.477+0000 7f11de976740 10 client.19071 fetch_fsmap learned FSMap version 41 2024-05-10T11:36:55.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.477+0000 7f11de976740 1 -- 172.21.15.100:0/3936071869 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55edd2dcb960 con 0x55edd2db71c0 2024-05-10T11:36:55.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.477+0000 7f11bd7fa640 1 -- 172.21.15.100:0/3936071869 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f11d00f0910 con 0x55edd2db71c0 2024-05-10T11:36:55.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.478+0000 7f11bd7fa640 1 -- 172.21.15.100:0/3936071869 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f11d00b38e0 con 0x55edd2db71c0 2024-05-10T11:36:55.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.478+0000 7f11de976740 10 client.19071 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:36:55.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.478+0000 7f11de976740 10 client.19071 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:36:55.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.478+0000 7f11de976740 20 client.19071 populate_metadata read hostname 'smithi100' 2024-05-10T11:36:55.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.478+0000 7f11de976740 1 --2- 172.21.15.100:0/3936071869 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55edd2dcccb0 0x55edd2dcf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:55.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.478+0000 7f11de976740 4 client.19071 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:36:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.478+0000 7f11de976740 1 -- 172.21.15.100:0/3936071869 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55edd2d251e0 con 0x55edd2dcccb0 2024-05-10T11:36:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.478+0000 7f11d91fd640 1 --2- 172.21.15.100:0/3936071869 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55edd2dcccb0 0x55edd2dcf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.478+0000 7f11bd7fa640 10 client.19071 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:36:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.479+0000 7f11d91fd640 1 --2- 172.21.15.100:0/3936071869 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55edd2dcccb0 0x55edd2dcf0a0 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49793ec6482cc932 server_cookie=48340be08120a10e in_seq=0 out_seq=0 2024-05-10T11:36:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.479+0000 7f11bd7fa640 1 -- 172.21.15.100:0/3936071869 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55edd2d251e0 con 0x55edd2dcccb0 2024-05-10T11:36:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.479+0000 7f11bd7fa640 10 client.19071 handle_command_reply: tid=0 2024-05-10T11:36:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.480+0000 7f11de976740 1 client.19071 shutdown 2024-05-10T11:36:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.480+0000 7f11de976740 1 -- 172.21.15.100:0/3936071869 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55edd2db71c0 msgr2=0x55edd2dc1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.480+0000 7f11de976740 1 --2- 172.21.15.100:0/3936071869 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55edd2db71c0 0x55edd2dc1400 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f11d00aac10 tx=0x7f11d000bf20 comp rx=0 tx=0).stop 2024-05-10T11:36:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.480+0000 7f11de976740 1 -- 172.21.15.100:0/3936071869 shutdown_connections 2024-05-10T11:36:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.480+0000 7f11de976740 1 --2- 172.21.15.100:0/3936071869 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55edd2dcccb0 0x55edd2dcf0a0 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.480+0000 7f11de976740 1 --2- 172.21.15.100:0/3936071869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55edd2dbcd20 0x55edd2dc8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.480+0000 7f11de976740 1 --2- 172.21.15.100:0/3936071869 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55edd2db7bb0 0x55edd2dc1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.480+0000 7f11de976740 1 --2- 172.21.15.100:0/3936071869 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55edd2db71c0 0x55edd2dc1400 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.480+0000 7f11de976740 1 -- 172.21.15.100:0/3936071869 >> 172.21.15.100:0/3936071869 conn(0x55edd2dae420 msgr2=0x55edd2dad580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:36:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.480+0000 7f11bd7fa640 0 client.19071 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:36:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.480+0000 7f11de976740 1 -- 172.21.15.100:0/3936071869 shutdown_connections 2024-05-10T11:36:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.480+0000 7f11de976740 1 -- 172.21.15.100:0/3936071869 wait complete. 2024-05-10T11:36:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.480+0000 7f11de976740 20 client.19071 trim_cache size 0 max 16384 2024-05-10T11:36:55.510 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.509+0000 7f11de976740 1 Processor -- start 2024-05-10T11:36:55.510 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.509+0000 7f11de976740 1 -- start start 2024-05-10T11:36:55.510 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.509+0000 7f11de976740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55edd2db71c0 0x55edd2db1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:55.511 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.509+0000 7f11de976740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55edd2db7bb0 0x55edd2dc8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:55.511 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.509+0000 7f11de976740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55edd2dbcd20 0x55edd2da93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:55.511 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.509+0000 7f11de976740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f11d41aba00 con 0x55edd2db7bb0 2024-05-10T11:36:55.511 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.509+0000 7f11de976740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f11d41a92d0 con 0x55edd2dbcd20 2024-05-10T11:36:55.511 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.509+0000 7f11de976740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f11d41a7120 con 0x55edd2db71c0 2024-05-10T11:36:55.511 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.509+0000 7f11d91fd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55edd2db7bb0 0x55edd2dc8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:55.511 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.509+0000 7f11d99fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55edd2db71c0 0x55edd2db1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:55.511 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.509+0000 7f11d91fd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55edd2db7bb0 0x55edd2dc8940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55668/0 (socket says 172.21.15.100:55668) 2024-05-10T11:36:55.511 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.509+0000 7f11d91fd640 1 -- 172.21.15.100:0/1580827857 learned_addr learned my addr 172.21.15.100:0/1580827857 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:36:55.511 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.509+0000 7f11da1ff640 1 --2- 172.21.15.100:0/1580827857 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55edd2dbcd20 0x55edd2da93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:55.511 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.509+0000 7f11d99fe640 1 -- 172.21.15.100:0/1580827857 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55edd2dbcd20 msgr2=0x55edd2da93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:55.511 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.509+0000 7f11d99fe640 1 --2- 172.21.15.100:0/1580827857 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55edd2dbcd20 0x55edd2da93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:55.512 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.510+0000 7f11d99fe640 1 -- 172.21.15.100:0/1580827857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55edd2db7bb0 msgr2=0x55edd2dc8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:55.512 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.510+0000 7f11d99fe640 1 --2- 172.21.15.100:0/1580827857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55edd2db7bb0 0x55edd2dc8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:55.512 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.510+0000 7f11d99fe640 1 -- 172.21.15.100:0/1580827857 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edd2dae250 con 0x55edd2db71c0 2024-05-10T11:36:55.512 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.510+0000 7f11d91fd640 1 --2- 172.21.15.100:0/1580827857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55edd2db7bb0 0x55edd2dc8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:36:55.512 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.510+0000 7f11d99fe640 1 --2- 172.21.15.100:0/1580827857 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55edd2db71c0 0x55edd2db1a80 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f11d00a6f90 tx=0x7f11d000bf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46d2c3ecd32ff71b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:36:55.512 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.510+0000 7f11bf7fe640 1 -- 172.21.15.100:0/1580827857 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d00aa860 con 0x55edd2db71c0 2024-05-10T11:36:55.512 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.510+0000 7f11bf7fe640 1 -- 172.21.15.100:0/1580827857 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f11d00035f0 con 0x55edd2db71c0 2024-05-10T11:36:55.512 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.510+0000 7f11bf7fe640 1 -- 172.21.15.100:0/1580827857 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d00aace0 con 0x55edd2db71c0 2024-05-10T11:36:55.516 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.511+0000 7f11de976740 1 -- 172.21.15.100:0/1580827857 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55edd2db71c0 msgr2=0x55edd2db1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:55.516 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.511+0000 7f11de976740 1 --2- 172.21.15.100:0/1580827857 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55edd2db71c0 0x55edd2db1a80 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f11d00a6f90 tx=0x7f11d000bf90 comp rx=0 tx=0).stop 2024-05-10T11:36:55.516 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.511+0000 7f11de976740 1 -- 172.21.15.100:0/1580827857 shutdown_connections 2024-05-10T11:36:55.516 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.511+0000 7f11de976740 1 --2- 172.21.15.100:0/1580827857 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55edd2dbcd20 0x55edd2da93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:55.516 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.511+0000 7f11de976740 1 --2- 172.21.15.100:0/1580827857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55edd2db7bb0 0x55edd2dc8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:55.516 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.511+0000 7f11de976740 1 --2- 172.21.15.100:0/1580827857 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55edd2db71c0 0x55edd2db1a80 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:55.516 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.511+0000 7f11de976740 1 -- 172.21.15.100:0/1580827857 >> 172.21.15.100:0/1580827857 conn(0x55edd2dae420 msgr2=0x55edd2db1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:36:55.516 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.511+0000 7f11de976740 1 -- 172.21.15.100:0/1580827857 shutdown_connections 2024-05-10T11:36:55.517 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.511+0000 7f11de976740 1 -- 172.21.15.100:0/1580827857 wait complete. 2024-05-10T11:36:55.517 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.511+0000 7f11de976740 1 -- start start 2024-05-10T11:36:55.517 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.511+0000 7f11de976740 1 Processor -- start 2024-05-10T11:36:55.517 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.512+0000 7f11de976740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55edd2db71c0 0x55edd2dc5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:55.517 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.512+0000 7f11de976740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55edd2db7bb0 0x55edd2dc6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:55.517 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.512+0000 7f11de976740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55edd2dbcd20 0x55edd2dc7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:55.517 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.512+0000 7f11de976740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f11d41a7790 con 0x55edd2db71c0 2024-05-10T11:36:55.517 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.512+0000 7f11de976740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f11d41a8c00 con 0x55edd2dbcd20 2024-05-10T11:36:55.517 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.512+0000 7f11de976740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f11d41a7e60 con 0x55edd2db7bb0 2024-05-10T11:36:55.517 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.512+0000 7f11de976740 10 client.0 _ll_register_callbacks cb 0x7f11d00e3f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:36:55.517 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.512+0000 7f11d99fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55edd2db71c0 0x55edd2dc5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:55.518 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.512+0000 7f11d99fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55edd2db71c0 0x55edd2dc5fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55674/0 (socket says 172.21.15.100:55674) 2024-05-10T11:36:55.518 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.512+0000 7f11d99fe640 1 -- 172.21.15.100:0/1909733762 learned_addr learned my addr 172.21.15.100:0/1909733762 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:36:55.518 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.512+0000 7f11da1ff640 1 --2- 172.21.15.100:0/1909733762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55edd2dbcd20 0x55edd2dc7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:55.518 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.512+0000 7f11d91fd640 1 --2- 172.21.15.100:0/1909733762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55edd2db7bb0 0x55edd2dc6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:55.518 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.512+0000 7f11d99fe640 1 -- 172.21.15.100:0/1909733762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55edd2db7bb0 msgr2=0x55edd2dc6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:55.518 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.512+0000 7f11d99fe640 1 --2- 172.21.15.100:0/1909733762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55edd2db7bb0 0x55edd2dc6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:55.518 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.512+0000 7f11d99fe640 1 -- 172.21.15.100:0/1909733762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55edd2dbcd20 msgr2=0x55edd2dc7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:55.518 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.512+0000 7f11d99fe640 1 --2- 172.21.15.100:0/1909733762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55edd2dbcd20 0x55edd2dc7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:55.518 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.512+0000 7f11d99fe640 1 -- 172.21.15.100:0/1909733762 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edd2dc7b10 con 0x55edd2db71c0 2024-05-10T11:36:55.518 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.512+0000 7f11d99fe640 1 --2- 172.21.15.100:0/1909733762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55edd2db71c0 0x55edd2dc5fd0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f11d000c2f0 tx=0x7f11d0003850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c9b111a7e95dcee server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:36:55.518 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.512+0000 7f11bd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:36:55.518 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.513+0000 7f11bd7fa640 1 -- 172.21.15.100:0/1909733762 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d00aace0 con 0x55edd2db71c0 2024-05-10T11:36:55.519 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.513+0000 7f11bd7fa640 1 -- 172.21.15.100:0/1909733762 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f11d00ab380 con 0x55edd2db71c0 2024-05-10T11:36:55.519 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.513+0000 7f11de976740 1 -- 172.21.15.100:0/1909733762 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55edd2d1f050 con 0x55edd2db71c0 2024-05-10T11:36:55.519 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.513+0000 7f11bd7fa640 1 -- 172.21.15.100:0/1909733762 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f11d00aa860 con 0x55edd2db71c0 2024-05-10T11:36:55.519 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.513+0000 7f11de976740 10 client.19077 fetch_fsmap learned FSMap version 41 2024-05-10T11:36:55.519 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.513+0000 7f11de976740 1 -- 172.21.15.100:0/1909733762 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55edd2dc7b10 con 0x55edd2db71c0 2024-05-10T11:36:55.519 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.513+0000 7f11bd7fa640 1 -- 172.21.15.100:0/1909733762 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55edd2d1f050 con 0x55edd2db71c0 2024-05-10T11:36:55.519 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.513+0000 7f11bd7fa640 1 -- 172.21.15.100:0/1909733762 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f11d00e52f0 con 0x55edd2db71c0 2024-05-10T11:36:55.519 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.513+0000 7f11de976740 10 client.19077 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:36:55.519 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.513+0000 7f11de976740 10 client.19077 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:36:55.519 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.513+0000 7f11de976740 20 client.19077 populate_metadata read hostname 'smithi100' 2024-05-10T11:36:55.519 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.513+0000 7f11de976740 1 --2- 172.21.15.100:0/1909733762 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55edd2e377d0 0x55edd2e39bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:55.519 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.513+0000 7f11de976740 4 client.19077 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:36:55.520 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.513+0000 7f11de976740 1 -- 172.21.15.100:0/1909733762 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55edd2c9a5f0 con 0x55edd2e377d0 2024-05-10T11:36:55.520 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.513+0000 7f11d91fd640 1 --2- 172.21.15.100:0/1909733762 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55edd2e377d0 0x55edd2e39bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:55.520 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.514+0000 7f11bd7fa640 10 client.19077 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:36:55.520 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.514+0000 7f11d91fd640 1 --2- 172.21.15.100:0/1909733762 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55edd2e377d0 0x55edd2e39bf0 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef5d13f9d224bda server_cookie=919265cd7e6bf6c4 in_seq=0 out_seq=0 2024-05-10T11:36:55.611 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.610+0000 7f11bd7fa640 1 -- 172.21.15.100:0/1909733762 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55edd2c9a5f0 con 0x55edd2e377d0 2024-05-10T11:36:55.611 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.610+0000 7f11bd7fa640 10 client.19077 handle_command_reply: tid=0 2024-05-10T11:36:55.611 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.610+0000 7f11de976740 1 client.19077 shutdown 2024-05-10T11:36:55.611 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.610+0000 7f11de976740 1 -- 172.21.15.100:0/1909733762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55edd2db71c0 msgr2=0x55edd2dc5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:55.611 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.610+0000 7f11de976740 1 --2- 172.21.15.100:0/1909733762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55edd2db71c0 0x55edd2dc5fd0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f11d000c2f0 tx=0x7f11d0003850 comp rx=0 tx=0).stop 2024-05-10T11:36:55.611 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.610+0000 7f11de976740 1 -- 172.21.15.100:0/1909733762 shutdown_connections 2024-05-10T11:36:55.611 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.610+0000 7f11de976740 1 --2- 172.21.15.100:0/1909733762 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55edd2e377d0 0x55edd2e39bf0 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:55.612 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.610+0000 7f11de976740 1 --2- 172.21.15.100:0/1909733762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55edd2dbcd20 0x55edd2dc7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:55.612 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.610+0000 7f11bd7fa640 0 client.19077 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:36:55.612 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.610+0000 7f11de976740 1 --2- 172.21.15.100:0/1909733762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55edd2db7bb0 0x55edd2dc6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:55.612 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.610+0000 7f11de976740 1 --2- 172.21.15.100:0/1909733762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55edd2db71c0 0x55edd2dc5fd0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:55.612 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.610+0000 7f11de976740 1 -- 172.21.15.100:0/1909733762 >> 172.21.15.100:0/1909733762 conn(0x55edd2dae420 msgr2=0x55edd2dc1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:36:55.612 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.610+0000 7f11de976740 1 -- 172.21.15.100:0/1909733762 shutdown_connections 2024-05-10T11:36:55.612 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.610+0000 7f11de976740 1 -- 172.21.15.100:0/1909733762 wait complete. 2024-05-10T11:36:55.612 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.610+0000 7f11de976740 20 client.19077 trim_cache size 0 max 16384 2024-05-10T11:36:55.612 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:36:55.612 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:36:55.612 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:36:55.613 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:36:55.613 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.611+0000 7f11dbc62640 1 -- 172.21.15.100:0/893987298 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f11c00b28a0 msgr2=0x7f11c00b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:55.614 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.611+0000 7f11dbc62640 1 --2- 172.21.15.100:0/893987298 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f11c00b28a0 0x7f11c00b4d60 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f11cc006010 tx=0x7f11cc009360 comp rx=0 tx=0).stop 2024-05-10T11:36:55.614 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.611+0000 7f11dbc62640 1 -- 172.21.15.100:0/893987298 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11d4101940 msgr2=0x7f11d41979b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:55.614 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.611+0000 7f11dbc62640 1 --2- 172.21.15.100:0/893987298 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11d4101940 0x7f11d41979b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f11d000ead0 tx=0x7f11d000ef90 comp rx=0 tx=0).stop 2024-05-10T11:36:55.614 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.611+0000 7f11dbc62640 1 -- 172.21.15.100:0/893987298 shutdown_connections 2024-05-10T11:36:55.614 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.611+0000 7f11dbc62640 1 --2- 172.21.15.100:0/893987298 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f11c00b28a0 0x7f11c00b4d60 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:55.614 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.611+0000 7f11dbc62640 1 --2- 172.21.15.100:0/893987298 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11d41034f0 0x7f11d419ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:55.614 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.611+0000 7f11dbc62640 1 --2- 172.21.15.100:0/893987298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11d4102b30 0x7f11d4197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:55.614 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.611+0000 7f11dbc62640 1 --2- 172.21.15.100:0/893987298 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11d4101940 0x7f11d41979b0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:55.614 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.611+0000 7f11dbc62640 1 -- 172.21.15.100:0/893987298 >> 172.21.15.100:0/893987298 conn(0x7f11d40fceb0 msgr2=0x7f11d4105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:36:55.614 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.611+0000 7f11dbc62640 1 -- 172.21.15.100:0/893987298 shutdown_connections 2024-05-10T11:36:55.614 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:36:55.611+0000 7f11dbc62640 1 -- 172.21.15.100:0/893987298 wait complete. 2024-05-10T11:36:55.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:36:55 smithi100 ceph-mon[30296]: pgmap v977: 129 pgs: 129 active+clean; 459 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 725 KiB/s rd, 3.8 MiB/s wr, 256 op/s 2024-05-10T11:36:55.932 INFO:tasks.workunit.client.0.smithi100.stdout: 10 48594 8.99 MB/sec execute 169 sec latency 132.992 ms 2024-05-10T11:36:56.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:36:55 smithi178 ceph-mon[33359]: pgmap v977: 129 pgs: 129 active+clean; 459 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 725 KiB/s rd, 3.8 MiB/s wr, 256 op/s 2024-05-10T11:36:56.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:36:55 smithi188 ceph-mon[34075]: pgmap v977: 129 pgs: 129 active+clean; 459 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 725 KiB/s rd, 3.8 MiB/s wr, 256 op/s 2024-05-10T11:36:56.932 INFO:tasks.workunit.client.0.smithi100.stdout: 10 48723 8.98 MB/sec execute 170 sec latency 158.624 ms 2024-05-10T11:36:57.020 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T11:36:57.020 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T11:36:57.373 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:36:57.918 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:36:57 smithi100 ceph-mon[30296]: pgmap v978: 129 pgs: 129 active+clean; 459 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.3 MiB/s wr, 317 op/s 2024-05-10T11:36:57.932 INFO:tasks.workunit.client.0.smithi100.stdout: 10 48912 9.03 MB/sec execute 171 sec latency 73.375 ms 2024-05-10T11:36:58.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:36:57 smithi178 ceph-mon[33359]: pgmap v978: 129 pgs: 129 active+clean; 459 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.3 MiB/s wr, 317 op/s 2024-05-10T11:36:58.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:36:57 smithi188 ceph-mon[34075]: pgmap v978: 129 pgs: 129 active+clean; 459 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.3 MiB/s wr, 317 op/s 2024-05-10T11:36:58.932 INFO:tasks.workunit.client.0.smithi100.stdout: 10 49159 9.06 MB/sec execute 172 sec latency 57.184 ms 2024-05-10T11:36:59.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.299+0000 7f98b0e2a700 1 -- 172.21.15.100:0/1501415405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98ac1043b0 msgr2=0x7f98ac104790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:59.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.299+0000 7f98b0e2a700 1 --2- 172.21.15.100:0/1501415405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98ac1043b0 0x7f98ac104790 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f989c009990 tx=0x7f989c009ca0 comp rx=0 tx=0).stop 2024-05-10T11:36:59.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.299+0000 7f98b0e2a700 1 -- 172.21.15.100:0/1501415405 shutdown_connections 2024-05-10T11:36:59.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.299+0000 7f98b0e2a700 1 --2- 172.21.15.100:0/1501415405 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98ac102530 0x7f98ac112220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:59.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.299+0000 7f98b0e2a700 1 --2- 172.21.15.100:0/1501415405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98ac1043b0 0x7f98ac104790 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:59.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.299+0000 7f98b0e2a700 1 --2- 172.21.15.100:0/1501415405 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f98ac101a90 0x7f98ac101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:59.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.299+0000 7f98b0e2a700 1 -- 172.21.15.100:0/1501415405 >> 172.21.15.100:0/1501415405 conn(0x7f98ac078a10 msgr2=0x7f98ac0ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:36:59.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.299+0000 7f98b0e2a700 1 -- 172.21.15.100:0/1501415405 shutdown_connections 2024-05-10T11:36:59.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.299+0000 7f98b0e2a700 1 -- 172.21.15.100:0/1501415405 wait complete. 2024-05-10T11:36:59.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.300+0000 7f98b0e2a700 1 Processor -- start 2024-05-10T11:36:59.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.300+0000 7f98b0e2a700 1 -- start start 2024-05-10T11:36:59.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.300+0000 7f98b0e2a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f98ac101a90 0x7f98ac1a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:59.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.300+0000 7f98b0e2a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98ac102530 0x7f98ac1a1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:59.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.301+0000 7f98b0e2a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98ac1043b0 0x7f98ac19b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:59.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.301+0000 7f98b0e2a700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f98ac115740 con 0x7f98ac1043b0 2024-05-10T11:36:59.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.301+0000 7f98b0e2a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f98ac1155c0 con 0x7f98ac102530 2024-05-10T11:36:59.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.301+0000 7f98b0e2a700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f98ac1158c0 con 0x7f98ac101a90 2024-05-10T11:36:59.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.301+0000 7f98a9d9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98ac102530 0x7f98ac1a1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:59.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.301+0000 7f98aad9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98ac1043b0 0x7f98ac19b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:59.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.301+0000 7f98a9d9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98ac102530 0x7f98ac1a1c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:39828/0 (socket says 172.21.15.100:39828) 2024-05-10T11:36:59.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.301+0000 7f98a9d9b700 1 -- 172.21.15.100:0/2861308631 learned_addr learned my addr 172.21.15.100:0/2861308631 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:36:59.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.301+0000 7f98aa59c700 1 --2- 172.21.15.100:0/2861308631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f98ac101a90 0x7f98ac1a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:59.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.301+0000 7f98aad9d700 1 -- 172.21.15.100:0/2861308631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f98ac101a90 msgr2=0x7f98ac1a1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:59.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.301+0000 7f98aad9d700 1 --2- 172.21.15.100:0/2861308631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f98ac101a90 0x7f98ac1a1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:59.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.301+0000 7f98aad9d700 1 -- 172.21.15.100:0/2861308631 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98ac102530 msgr2=0x7f98ac1a1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:59.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.301+0000 7f98aad9d700 1 --2- 172.21.15.100:0/2861308631 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98ac102530 0x7f98ac1a1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:59.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.301+0000 7f98aad9d700 1 -- 172.21.15.100:0/2861308631 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98ac19c040 con 0x7f98ac1043b0 2024-05-10T11:36:59.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.301+0000 7f98a9d9b700 1 --2- 172.21.15.100:0/2861308631 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98ac102530 0x7f98ac1a1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:36:59.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.301+0000 7f98aad9d700 1 --2- 172.21.15.100:0/2861308631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98ac1043b0 0x7f98ac19b8e0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f98a400efc0 tx=0x7f98a400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:36:59.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.302+0000 7f989b7fe700 1 -- 172.21.15.100:0/2861308631 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a40092e0 con 0x7f98ac1043b0 2024-05-10T11:36:59.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.302+0000 7f98b0e2a700 1 -- 172.21.15.100:0/2861308631 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98ac19c2d0 con 0x7f98ac1043b0 2024-05-10T11:36:59.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.302+0000 7f989b7fe700 1 -- 172.21.15.100:0/2861308631 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f98a400c9f0 con 0x7f98ac1043b0 2024-05-10T11:36:59.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.302+0000 7f98b0e2a700 1 -- 172.21.15.100:0/2861308631 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98ac1a85f0 con 0x7f98ac1043b0 2024-05-10T11:36:59.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.302+0000 7f989b7fe700 1 -- 172.21.15.100:0/2861308631 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a40092e0 con 0x7f98ac1043b0 2024-05-10T11:36:59.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.305+0000 7f989b7fe700 1 -- 172.21.15.100:0/2861308631 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f98a4012050 con 0x7f98ac1043b0 2024-05-10T11:36:59.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.307+0000 7f98bb6bdb80 1 Processor -- start 2024-05-10T11:36:59.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.307+0000 7f989b7fe700 1 --2- 172.21.15.100:0/2861308631 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f98940b2c70 0x7f98940b5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:59.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.307+0000 7f98bb6bdb80 1 -- start start 2024-05-10T11:36:59.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.307+0000 7f989b7fe700 1 -- 172.21.15.100:0/2861308631 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f98a4019070 con 0x7f98ac1043b0 2024-05-10T11:36:59.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.307+0000 7f98bb6bdb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564f25507550 0x564f25507930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:59.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.307+0000 7f98bb6bdb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f25507e70 0x564f2550b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:59.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.307+0000 7f98bb6bdb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f2550bd00 0x564f2550c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:59.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.307+0000 7f98bb6bdb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x564f254598e0 con 0x564f25507e70 2024-05-10T11:36:59.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.307+0000 7f98bb6bdb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x564f253f8840 con 0x564f25507550 2024-05-10T11:36:59.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.307+0000 7f98a9d9b700 1 --2- 172.21.15.100:0/2861308631 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f98940b2c70 0x7f98940b5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:59.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.307+0000 7f98bb6bdb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x564f254558a0 con 0x564f2550bd00 2024-05-10T11:36:59.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.307+0000 7f98aa59c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564f25507550 0x564f25507930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:59.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.307+0000 7f98aa59c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564f25507550 0x564f25507930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:39834/0 (socket says 172.21.15.100:39834) 2024-05-10T11:36:59.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.307+0000 7f98aa59c700 1 -- 172.21.15.100:0/714952038 learned_addr learned my addr 172.21.15.100:0/714952038 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:36:59.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.307+0000 7f98a9d9b700 1 --2- 172.21.15.100:0/714952038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f25507e70 0x564f2550b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:59.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.307+0000 7f98aad9d700 1 --2- 172.21.15.100:0/714952038 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f2550bd00 0x564f2550c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:59.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.308+0000 7f98aa59c700 1 -- 172.21.15.100:0/714952038 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f2550bd00 msgr2=0x564f2550c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:59.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.308+0000 7f98aa59c700 1 --2- 172.21.15.100:0/714952038 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f2550bd00 0x564f2550c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:59.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.308+0000 7f98aa59c700 1 -- 172.21.15.100:0/714952038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f25507e70 msgr2=0x564f2550b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:59.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.308+0000 7f98aa59c700 1 --2- 172.21.15.100:0/714952038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f25507e70 0x564f2550b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:59.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.308+0000 7f98aa59c700 1 -- 172.21.15.100:0/714952038 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f253a6950 con 0x564f25507550 2024-05-10T11:36:59.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.308+0000 7f98a9d9b700 1 --2- 172.21.15.100:0/714952038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f25507e70 0x564f2550b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:36:59.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.308+0000 7f98a9d9b700 1 --2- 172.21.15.100:0/2861308631 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f98940b2c70 0x7f98940b5130 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f98a00096b0 tx=0x7f98a00114d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:36:59.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.308+0000 7f98aa59c700 1 --2- 172.21.15.100:0/714952038 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564f25507550 0x564f25507930 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f989c015000 tx=0x7f989c00b610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0eeaac26661f877 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:36:59.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.308+0000 7f9898ff9700 1 -- 172.21.15.100:0/714952038 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f989c027070 con 0x564f25507550 2024-05-10T11:36:59.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.308+0000 7f9898ff9700 1 -- 172.21.15.100:0/714952038 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f989c00bd40 con 0x564f25507550 2024-05-10T11:36:59.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.308+0000 7f9898ff9700 1 -- 172.21.15.100:0/714952038 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f989c003d30 con 0x564f25507550 2024-05-10T11:36:59.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.309+0000 7f98bb6bdb80 1 -- 172.21.15.100:0/714952038 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564f25507550 msgr2=0x564f25507930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:59.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.309+0000 7f98bb6bdb80 1 --2- 172.21.15.100:0/714952038 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564f25507550 0x564f25507930 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f989c015000 tx=0x7f989c00b610 comp rx=0 tx=0).stop 2024-05-10T11:36:59.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.309+0000 7f98bb6bdb80 1 -- 172.21.15.100:0/714952038 shutdown_connections 2024-05-10T11:36:59.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.309+0000 7f98bb6bdb80 1 --2- 172.21.15.100:0/714952038 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f2550bd00 0x564f2550c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:59.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.309+0000 7f98bb6bdb80 1 --2- 172.21.15.100:0/714952038 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564f25507550 0x564f25507930 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:59.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.309+0000 7f98bb6bdb80 1 --2- 172.21.15.100:0/714952038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f25507e70 0x564f2550b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:59.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.309+0000 7f98bb6bdb80 1 -- 172.21.15.100:0/714952038 >> 172.21.15.100:0/714952038 conn(0x564f25433180 msgr2=0x564f25501cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:36:59.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.309+0000 7f98bb6bdb80 1 -- 172.21.15.100:0/714952038 shutdown_connections 2024-05-10T11:36:59.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.310+0000 7f98bb6bdb80 1 -- 172.21.15.100:0/714952038 wait complete. 2024-05-10T11:36:59.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.310+0000 7f98bb6bdb80 1 -- start start 2024-05-10T11:36:59.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.310+0000 7f98bb6bdb80 1 Processor -- start 2024-05-10T11:36:59.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.311+0000 7f98bb6bdb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f25507550 0x564f25513030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:59.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.311+0000 7f98bb6bdb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564f25507e70 0x564f255103d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:59.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.311+0000 7f98bb6bdb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f2550bd00 0x564f25510910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:59.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.311+0000 7f98bb6bdb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x564f25513f50 con 0x564f25507550 2024-05-10T11:36:59.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.311+0000 7f98aa59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f25507550 0x564f25513030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:59.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.311+0000 7f98bb6bdb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x564f253bfcb0 con 0x564f25507e70 2024-05-10T11:36:59.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.311+0000 7f98aa59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f25507550 0x564f25513030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33172/0 (socket says 172.21.15.100:33172) 2024-05-10T11:36:59.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.311+0000 7f98bb6bdb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x564f255140d0 con 0x564f2550bd00 2024-05-10T11:36:59.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.311+0000 7f98aa59c700 1 -- 172.21.15.100:0/2633969754 learned_addr learned my addr 172.21.15.100:0/2633969754 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:36:59.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.311+0000 7f98bb6bdb80 10 client.0 _ll_register_callbacks cb 0x564f253f19a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:36:59.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.311+0000 7f98a9d9b700 1 --2- 172.21.15.100:0/2633969754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564f25507e70 0x564f255103d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:59.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.311+0000 7f98aad9d700 1 --2- 172.21.15.100:0/2633969754 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f2550bd00 0x564f25510910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:59.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.311+0000 7f98a9d9b700 1 -- 172.21.15.100:0/2633969754 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f2550bd00 msgr2=0x564f25510910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:59.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.311+0000 7f98a9d9b700 1 --2- 172.21.15.100:0/2633969754 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f2550bd00 0x564f25510910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:59.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.311+0000 7f98a9d9b700 1 -- 172.21.15.100:0/2633969754 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f25507550 msgr2=0x564f25513030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:59.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.311+0000 7f98a9d9b700 1 --2- 172.21.15.100:0/2633969754 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f25507550 0x564f25513030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:59.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.311+0000 7f98a9d9b700 1 -- 172.21.15.100:0/2633969754 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f253f5710 con 0x564f25507e70 2024-05-10T11:36:59.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.311+0000 7f98a9d9b700 1 --2- 172.21.15.100:0/2633969754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564f25507e70 0x564f255103d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f98a0009880 tx=0x7f98a0009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=caade86894360818 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:36:59.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.311+0000 7f9891ffb700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:36:59.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.312+0000 7f9891ffb700 1 -- 172.21.15.100:0/2633969754 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a0024070 con 0x564f25507e70 2024-05-10T11:36:59.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.312+0000 7f98bb6bdb80 1 -- 172.21.15.100:0/2633969754 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f989c00bd40 con 0x564f25507e70 2024-05-10T11:36:59.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.312+0000 7f9891ffb700 1 -- 172.21.15.100:0/2633969754 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f98a001fde0 con 0x564f25507e70 2024-05-10T11:36:59.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.312+0000 7f9891ffb700 1 -- 172.21.15.100:0/2633969754 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f98a0009ca0 con 0x564f25507e70 2024-05-10T11:36:59.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.312+0000 7f98bb6bdb80 10 client.28391 fetch_fsmap learned FSMap version 41 2024-05-10T11:36:59.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.312+0000 7f98bb6bdb80 1 -- 172.21.15.100:0/2633969754 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x564f25526ce0 con 0x564f25507e70 2024-05-10T11:36:59.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.312+0000 7f9891ffb700 1 -- 172.21.15.100:0/2633969754 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f989c00bd40 con 0x564f25507e70 2024-05-10T11:36:59.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.312+0000 7f9891ffb700 1 -- 172.21.15.100:0/2633969754 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f98a00203f0 con 0x564f25507e70 2024-05-10T11:36:59.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.312+0000 7f98bb6bdb80 10 client.28391 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:36:59.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.312+0000 7f98bb6bdb80 10 client.28391 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:36:59.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.312+0000 7f98bb6bdb80 20 client.28391 populate_metadata read hostname 'smithi100' 2024-05-10T11:36:59.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.313+0000 7f98bb6bdb80 1 --2- 172.21.15.100:0/2633969754 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x564f25529270 0x564f2552b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:59.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.313+0000 7f98bb6bdb80 4 client.28391 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:36:59.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.313+0000 7f98bb6bdb80 1 -- 172.21.15.100:0/2633969754 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f254d36d0 con 0x564f25529270 2024-05-10T11:36:59.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.313+0000 7f98aa59c700 1 --2- 172.21.15.100:0/2633969754 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x564f25529270 0x564f2552b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:59.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.313+0000 7f9891ffb700 10 client.28391 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:36:59.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.313+0000 7f98aa59c700 1 --2- 172.21.15.100:0/2633969754 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x564f25529270 0x564f2552b6c0 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f4594e6a54fa181 server_cookie=61af6bf480f4fe9f in_seq=0 out_seq=0 2024-05-10T11:36:59.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.314+0000 7f9891ffb700 1 -- 172.21.15.100:0/2633969754 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564f254d36d0 con 0x564f25529270 2024-05-10T11:36:59.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.314+0000 7f9891ffb700 10 client.28391 handle_command_reply: tid=0 2024-05-10T11:36:59.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.314+0000 7f98bb6bdb80 1 client.28391 shutdown 2024-05-10T11:36:59.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.314+0000 7f98bb6bdb80 1 -- 172.21.15.100:0/2633969754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564f25507e70 msgr2=0x564f255103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:59.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.314+0000 7f98bb6bdb80 1 --2- 172.21.15.100:0/2633969754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564f25507e70 0x564f255103d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f98a0009880 tx=0x7f98a0009260 comp rx=0 tx=0).stop 2024-05-10T11:36:59.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.314+0000 7f98bb6bdb80 1 -- 172.21.15.100:0/2633969754 shutdown_connections 2024-05-10T11:36:59.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.314+0000 7f98bb6bdb80 1 --2- 172.21.15.100:0/2633969754 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x564f25529270 0x564f2552b6c0 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:59.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.314+0000 7f9891ffb700 0 client.28391 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:36:59.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.314+0000 7f98bb6bdb80 1 --2- 172.21.15.100:0/2633969754 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f2550bd00 0x564f25510910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:59.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.314+0000 7f98bb6bdb80 1 --2- 172.21.15.100:0/2633969754 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f25507550 0x564f25513030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:59.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.314+0000 7f98bb6bdb80 1 --2- 172.21.15.100:0/2633969754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564f25507e70 0x564f255103d0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:59.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.314+0000 7f98bb6bdb80 1 -- 172.21.15.100:0/2633969754 >> 172.21.15.100:0/2633969754 conn(0x564f25433180 msgr2=0x564f254fc7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:36:59.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.315+0000 7f98bb6bdb80 1 -- 172.21.15.100:0/2633969754 shutdown_connections 2024-05-10T11:36:59.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.315+0000 7f98bb6bdb80 1 -- 172.21.15.100:0/2633969754 wait complete. 2024-05-10T11:36:59.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.315+0000 7f98bb6bdb80 20 client.28391 trim_cache size 0 max 16384 2024-05-10T11:36:59.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.345+0000 7f98bb6bdb80 1 Processor -- start 2024-05-10T11:36:59.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.345+0000 7f98bb6bdb80 1 -- start start 2024-05-10T11:36:59.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.345+0000 7f98bb6bdb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f25507550 0x564f254fc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:59.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.345+0000 7f98bb6bdb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f25507e70 0x564f255095a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:59.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.345+0000 7f98bb6bdb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564f2550bd00 0x564f25509ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:59.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.345+0000 7f98bb6bdb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f98ac1af580 con 0x564f25507e70 2024-05-10T11:36:59.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.345+0000 7f98bb6bdb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f98ac1ace10 con 0x564f2550bd00 2024-05-10T11:36:59.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.345+0000 7f98bb6bdb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f98ac1aacb0 con 0x564f25507550 2024-05-10T11:36:59.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.345+0000 7f98a9d9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f25507e70 0x564f255095a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:59.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.345+0000 7f98a9d9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f25507e70 0x564f255095a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33176/0 (socket says 172.21.15.100:33176) 2024-05-10T11:36:59.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.345+0000 7f98a9d9b700 1 -- 172.21.15.100:0/3293074393 learned_addr learned my addr 172.21.15.100:0/3293074393 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:36:59.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.345+0000 7f98aa59c700 1 --2- 172.21.15.100:0/3293074393 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f25507550 0x564f254fc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:59.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.345+0000 7f98a9d9b700 1 -- 172.21.15.100:0/3293074393 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f25507550 msgr2=0x564f254fc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:59.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.345+0000 7f98a9d9b700 1 --2- 172.21.15.100:0/3293074393 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f25507550 0x564f254fc620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:59.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.345+0000 7f98a9d9b700 1 -- 172.21.15.100:0/3293074393 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564f2550bd00 msgr2=0x564f25509ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:59.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.345+0000 7f98a9d9b700 1 --2- 172.21.15.100:0/3293074393 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564f2550bd00 0x564f25509ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:59.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.346+0000 7f98a9d9b700 1 -- 172.21.15.100:0/3293074393 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f255035a0 con 0x564f25507e70 2024-05-10T11:36:59.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.346+0000 7f98a9d9b700 1 --2- 172.21.15.100:0/3293074393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f25507e70 0x564f255095a0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f98a000a010 tx=0x7f98a0020c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b243793e8e8ecb9c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:36:59.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.346+0000 7f9893fff700 1 -- 172.21.15.100:0/3293074393 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a0024070 con 0x564f25507e70 2024-05-10T11:36:59.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.346+0000 7f9893fff700 1 -- 172.21.15.100:0/3293074393 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f98a00046f0 con 0x564f25507e70 2024-05-10T11:36:59.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.346+0000 7f9893fff700 1 -- 172.21.15.100:0/3293074393 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a001ffe0 con 0x564f25507e70 2024-05-10T11:36:59.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.347+0000 7f98bb6bdb80 1 -- 172.21.15.100:0/3293074393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f25507e70 msgr2=0x564f255095a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:59.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.347+0000 7f98bb6bdb80 1 --2- 172.21.15.100:0/3293074393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f25507e70 0x564f255095a0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f98a000a010 tx=0x7f98a0020c90 comp rx=0 tx=0).stop 2024-05-10T11:36:59.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.347+0000 7f98bb6bdb80 1 -- 172.21.15.100:0/3293074393 shutdown_connections 2024-05-10T11:36:59.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.347+0000 7f98bb6bdb80 1 --2- 172.21.15.100:0/3293074393 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564f2550bd00 0x564f25509ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:59.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.347+0000 7f98bb6bdb80 1 --2- 172.21.15.100:0/3293074393 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f25507550 0x564f254fc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:59.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.347+0000 7f98bb6bdb80 1 --2- 172.21.15.100:0/3293074393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f25507e70 0x564f255095a0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:59.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.347+0000 7f98bb6bdb80 1 -- 172.21.15.100:0/3293074393 >> 172.21.15.100:0/3293074393 conn(0x564f25433180 msgr2=0x564f254fbf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:36:59.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.347+0000 7f98bb6bdb80 1 -- 172.21.15.100:0/3293074393 shutdown_connections 2024-05-10T11:36:59.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.347+0000 7f98bb6bdb80 1 -- 172.21.15.100:0/3293074393 wait complete. 2024-05-10T11:36:59.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.347+0000 7f98bb6bdb80 1 -- start start 2024-05-10T11:36:59.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.347+0000 7f98bb6bdb80 1 Processor -- start 2024-05-10T11:36:59.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.348+0000 7f98bb6bdb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f25507550 0x564f25521dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:59.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.348+0000 7f98bb6bdb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f25507e70 0x564f25522310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:59.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.348+0000 7f98bb6bdb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564f2550bd00 0x564f2551ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:59.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.348+0000 7f98bb6bdb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f98ac1ab2d0 con 0x564f25507550 2024-05-10T11:36:59.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.348+0000 7f98bb6bdb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f98ac1ac740 con 0x564f2550bd00 2024-05-10T11:36:59.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.348+0000 7f98bb6bdb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f98ac1ab9a0 con 0x564f25507e70 2024-05-10T11:36:59.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.348+0000 7f98aad9d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564f2550bd00 0x564f2551ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:59.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.348+0000 7f98bb6bdb80 10 client.0 _ll_register_callbacks cb 0x564f253f19a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:36:59.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.348+0000 7f98a9d9b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f25507e70 0x564f25522310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:59.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.348+0000 7f98aad9d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564f2550bd00 0x564f2551ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:39852/0 (socket says 172.21.15.100:39852) 2024-05-10T11:36:59.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.348+0000 7f98aad9d700 1 -- 172.21.15.100:0/1385756857 learned_addr learned my addr 172.21.15.100:0/1385756857 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:36:59.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.348+0000 7f98aa59c700 1 --2- 172.21.15.100:0/1385756857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f25507550 0x564f25521dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:59.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.349+0000 7f98aad9d700 1 -- 172.21.15.100:0/1385756857 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f25507e70 msgr2=0x564f25522310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:59.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.349+0000 7f98aad9d700 1 --2- 172.21.15.100:0/1385756857 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f25507e70 0x564f25522310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:59.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.349+0000 7f98aad9d700 1 -- 172.21.15.100:0/1385756857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f25507550 msgr2=0x564f25521dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:36:59.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.349+0000 7f98aad9d700 1 --2- 172.21.15.100:0/1385756857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f25507550 0x564f25521dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:59.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.349+0000 7f98aad9d700 1 -- 172.21.15.100:0/1385756857 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f2551f5b0 con 0x564f2550bd00 2024-05-10T11:36:59.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.349+0000 7f98aad9d700 1 --2- 172.21.15.100:0/1385756857 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564f2550bd00 0x564f2551ed70 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f98a40a8920 tx=0x7f98a40a87c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc5f49c3d64e00c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:36:59.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.349+0000 7f9891ffb700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:36:59.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.349+0000 7f9891ffb700 1 -- 172.21.15.100:0/1385756857 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a40cb2d0 con 0x564f2550bd00 2024-05-10T11:36:59.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.349+0000 7f9891ffb700 1 -- 172.21.15.100:0/1385756857 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f98a40a8bf0 con 0x564f2550bd00 2024-05-10T11:36:59.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.349+0000 7f98bb6bdb80 1 -- 172.21.15.100:0/1385756857 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98a00046f0 con 0x564f2550bd00 2024-05-10T11:36:59.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.349+0000 7f9891ffb700 1 -- 172.21.15.100:0/1385756857 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f98a40a8990 con 0x564f2550bd00 2024-05-10T11:36:59.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.349+0000 7f98bb6bdb80 10 client.28394 fetch_fsmap learned FSMap version 41 2024-05-10T11:36:59.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.349+0000 7f98bb6bdb80 1 -- 172.21.15.100:0/1385756857 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x564f255948b0 con 0x564f2550bd00 2024-05-10T11:36:59.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.350+0000 7f9891ffb700 1 -- 172.21.15.100:0/1385756857 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f98a00046f0 con 0x564f2550bd00 2024-05-10T11:36:59.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.350+0000 7f9891ffb700 1 -- 172.21.15.100:0/1385756857 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f98a4004bd0 con 0x564f2550bd00 2024-05-10T11:36:59.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.350+0000 7f98bb6bdb80 10 client.28394 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:36:59.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.350+0000 7f98bb6bdb80 10 client.28394 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:36:59.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.350+0000 7f98bb6bdb80 20 client.28394 populate_metadata read hostname 'smithi100' 2024-05-10T11:36:59.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.350+0000 7f98bb6bdb80 1 --2- 172.21.15.100:0/1385756857 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x564f25594fe0 0x564f25597400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:36:59.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.350+0000 7f98bb6bdb80 4 client.28394 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T11:36:59.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.350+0000 7f98bb6bdb80 1 -- 172.21.15.100:0/1385756857 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564f254672c0 con 0x564f25594fe0 2024-05-10T11:36:59.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.350+0000 7f98aa59c700 1 --2- 172.21.15.100:0/1385756857 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x564f25594fe0 0x564f25597400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:36:59.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.350+0000 7f98aa59c700 1 --2- 172.21.15.100:0/1385756857 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x564f25594fe0 0x564f25597400 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=278922c59a6fc971 server_cookie=c19d36086a1b0d66 in_seq=0 out_seq=0 2024-05-10T11:36:59.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.350+0000 7f9891ffb700 10 client.28394 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:36:59.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.351+0000 7f9891ffb700 1 -- 172.21.15.100:0/1385756857 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564f254672c0 con 0x564f25594fe0 2024-05-10T11:36:59.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.351+0000 7f9891ffb700 10 client.28394 handle_command_reply: tid=0 2024-05-10T11:36:59.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.352+0000 7f98bb6bdb80 1 client.28394 shutdown 2024-05-10T11:36:59.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.352+0000 7f98bb6bdb80 1 -- 172.21.15.100:0/1385756857 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564f2550bd00 msgr2=0x564f2551ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:59.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.352+0000 7f98bb6bdb80 1 --2- 172.21.15.100:0/1385756857 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564f2550bd00 0x564f2551ed70 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f98a40a8920 tx=0x7f98a40a87c0 comp rx=0 tx=0).stop 2024-05-10T11:36:59.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.352+0000 7f98bb6bdb80 1 -- 172.21.15.100:0/1385756857 shutdown_connections 2024-05-10T11:36:59.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.352+0000 7f98bb6bdb80 1 --2- 172.21.15.100:0/1385756857 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x564f25594fe0 0x564f25597400 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:59.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.352+0000 7f98bb6bdb80 1 --2- 172.21.15.100:0/1385756857 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564f2550bd00 0x564f2551ed70 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:59.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.352+0000 7f98bb6bdb80 1 --2- 172.21.15.100:0/1385756857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f25507550 0x564f25521dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:59.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.352+0000 7f98bb6bdb80 1 --2- 172.21.15.100:0/1385756857 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f25507e70 0x564f25522310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:59.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.352+0000 7f98bb6bdb80 1 -- 172.21.15.100:0/1385756857 >> 172.21.15.100:0/1385756857 conn(0x564f25433180 msgr2=0x564f2550e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:36:59.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.352+0000 7f9891ffb700 0 client.28394 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:36:59.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.352+0000 7f98bb6bdb80 1 -- 172.21.15.100:0/1385756857 shutdown_connections 2024-05-10T11:36:59.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.352+0000 7f98bb6bdb80 1 -- 172.21.15.100:0/1385756857 wait complete. 2024-05-10T11:36:59.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.352+0000 7f98bb6bdb80 20 client.28394 trim_cache size 0 max 16384 2024-05-10T11:36:59.357 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:36:59.357 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-05-10T11:36:59.357 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "c9b068d2-0ad9-440e-b804-c3de6c8852ba", 2024-05-10T11:36:59.357 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-05-10T11:36:59.357 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:36:59.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.353+0000 7f98b0e2a700 1 -- 172.21.15.100:0/2861308631 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f98940b2c70 msgr2=0x7f98940b5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:59.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.353+0000 7f98b0e2a700 1 --2- 172.21.15.100:0/2861308631 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f98940b2c70 0x7f98940b5130 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f98a00096b0 tx=0x7f98a00114d0 comp rx=0 tx=0).stop 2024-05-10T11:36:59.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.353+0000 7f98b0e2a700 1 -- 172.21.15.100:0/2861308631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98ac1043b0 msgr2=0x7f98ac19b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:36:59.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.353+0000 7f98b0e2a700 1 --2- 172.21.15.100:0/2861308631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98ac1043b0 0x7f98ac19b8e0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f98a400efc0 tx=0x7f98a400c3a0 comp rx=0 tx=0).stop 2024-05-10T11:36:59.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.353+0000 7f98b0e2a700 1 -- 172.21.15.100:0/2861308631 shutdown_connections 2024-05-10T11:36:59.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.353+0000 7f98b0e2a700 1 --2- 172.21.15.100:0/2861308631 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f98940b2c70 0x7f98940b5130 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:59.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.353+0000 7f98b0e2a700 1 --2- 172.21.15.100:0/2861308631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98ac1043b0 0x7f98ac19b8e0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:59.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.353+0000 7f98b0e2a700 1 --2- 172.21.15.100:0/2861308631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f98ac101a90 0x7f98ac1a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:59.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.353+0000 7f98b0e2a700 1 --2- 172.21.15.100:0/2861308631 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98ac102530 0x7f98ac1a1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:36:59.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.353+0000 7f98b0e2a700 1 -- 172.21.15.100:0/2861308631 >> 172.21.15.100:0/2861308631 conn(0x7f98ac078a10 msgr2=0x7f98ac0fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:36:59.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.353+0000 7f98b0e2a700 1 -- 172.21.15.100:0/2861308631 shutdown_connections 2024-05-10T11:36:59.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:36:59.353+0000 7f98b0e2a700 1 -- 172.21.15.100:0/2861308631 wait complete. 2024-05-10T11:36:59.670 INFO:journalctl@ceph.mds.i.smithi188.stdout:May 10 11:36:59 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-i[64554]: 2024-05-10T11:36:59.415+0000 7fd152433700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000004d2 [...2,head] /volumes/qa/sv_0/96765652-39ef-4cf3-b20a-a70f7e9d774d/client.0/tmp/clients/client8/~dmtmp/COREL/ auth{2=1} v8510 ap=2 f(v260 m2024-05-10T11:36:46.881233+0000 11=11+0) n(v15 rc2024-05-10T11:36:46.881233+0000 b2861056 12=11+1) (inest sync->mix g=2) caps={15207=pAsLsXsFs/-@5997} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x560801803700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000004d2:[<0x1000000041f/COREL v50>,<0x10000000410/~dmtmp v34>,<0x10000000007/client8 v1938>,<0x10000000006/clients v1743>,<0x10000000005/tmp v1642>,<0x10000000003/client.0 v1616>,<0x10000000002/96765652-39ef-4cf3-b20a-a70f7e9d774d v1609>,<0x10000000001/sv_0 v1620>,<0x10000000000/qa v1598>,<0x1/volumes v1566>]//[]","memoryvalue":"(2)0x100000004d2:[<0x1000000041f/COREL v8510>,<0x10000000410/~dmtmp v5009>,<0x10000000007/client8 v4939>,<0x10000000006/clients v2777>,<0x10000000005/tmp v2227>,<0x10000000003/client.0 v2192>,<0x10000000002/96765652-39ef-4cf3-b20a-a70f7e9d774d v2182>,<0x10000000001/sv_0 v2188>,<0x10000000000/qa v2162>,<0x1/volumes v2121>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-10T11:36:46.848801+0000 12=12+0)","ondisk_value.rstat":"n(v0 rc2024-05-10T11:36:46.881233+0000 b2861056 12=11+1)","memory_value.dirstat":"f(v260 m2024-05-10T11:36:46.881233+0000 11=11+0)","memory_value.rstat":"n(v15 rc2024-05-10T11:36:46.881233+0000 b2861056 12=11+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-05-10T11:36:59.670 INFO:journalctl@ceph.mds.i.smithi188.stdout:May 10 11:36:59 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-i[64554]: 2024-05-10T11:36:59.486+0000 7fd152433700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000004a2 [...2,head] /volumes/qa/sv_0/96765652-39ef-4cf3-b20a-a70f7e9d774d/client.0/tmp/clients/client7/~dmtmp/PWRPNT/ auth{2=1} v6911 ap=2 f(v270 m2024-05-10T11:36:36.037057+0000 7=7+0) n(v17 rc2024-05-10T11:36:36.037057+0000 b1462582 8=7+1) (inest sync->mix g=2) caps={15207=pAsLsXsFs/-@5645} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x5608017a8100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000004a2:[<0x10000000420/PWRPNT v220>,<0x10000000412/~dmtmp v34>,<0x10000000007/client7 v1940>,<0x10000000006/clients v1743>,<0x10000000005/tmp v1642>,<0x10000000003/client.0 v1616>,<0x10000000002/96765652-39ef-4cf3-b20a-a70f7e9d774d v1609>,<0x10000000001/sv_0 v1620>,<0x10000000000/qa v1598>,<0x1/volumes v1566>]//[]","memoryvalue":"(2)0x100000004a2:[<0x10000000420/PWRPNT v6911>,<0x10000000412/~dmtmp v5922>,<0x10000000007/client7 v4957>,<0x10000000006/clients v2777>,<0x10000000005/tmp v2227>,<0x10000000003/client.0 v2192>,<0x10000000002/96765652-39ef-4cf3-b20a-a70f7e9d774d v2182>,<0x10000000001/sv_0 v2188>,<0x10000000000/qa v2162>,<0x1/volumes v2121>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-10T11:36:35.893136+0000 8=8+0)","ondisk_value.rstat":"n(v0 rc2024-05-10T11:36:36.037057+0000 b1462582 8=7+1)","memory_value.dirstat":"f(v270 m2024-05-10T11:36:36.037057+0000 7=7+0)","memory_value.rstat":"n(v17 rc2024-05-10T11:36:36.037057+0000 b1462582 8=7+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-05-10T11:36:59.932 INFO:tasks.workunit.client.0.smithi100.stdout: 10 49284 9.04 MB/sec execute 173 sec latency 92.838 ms 2024-05-10T11:36:59.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:36:59 smithi100 ceph-mon[30296]: pgmap v979: 129 pgs: 129 active+clean; 460 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 3.5 MiB/s wr, 249 op/s 2024-05-10T11:37:00.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:36:59 smithi178 ceph-mon[33359]: pgmap v979: 129 pgs: 129 active+clean; 460 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 3.5 MiB/s wr, 249 op/s 2024-05-10T11:37:00.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:36:59 smithi188 ceph-mon[34075]: pgmap v979: 129 pgs: 129 active+clean; 460 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 3.5 MiB/s wr, 249 op/s 2024-05-10T11:37:00.288 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T11:37:00.634 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:37:00.932 INFO:tasks.workunit.client.0.smithi100.stdout: 10 49414 9.03 MB/sec execute 174 sec latency 72.197 ms 2024-05-10T11:37:00.933 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:37:00 smithi100 ceph-mon[30296]: scrub queued for path: / 2024-05-10T11:37:00.933 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:37:00 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:37:00.933 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:37:00 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:37:00.933 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:37:00 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:37:00.933 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:37:00 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:37:00.933 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:37:00 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:37:00.933 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:37:00 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:37:00.933 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:37:00 smithi100 ceph-mon[30296]: Scrub error on inode 0x100000004d2 (/volumes/qa/sv_0/96765652-39ef-4cf3-b20a-a70f7e9d774d/client.0/tmp/clients/client8/~dmtmp/COREL) see mds.i log and `damage ls` output for details 2024-05-10T11:37:00.933 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:37:00 smithi100 ceph-mon[30296]: Scrub error on inode 0x100000004a2 (/volumes/qa/sv_0/96765652-39ef-4cf3-b20a-a70f7e9d774d/client.0/tmp/clients/client7/~dmtmp/PWRPNT) see mds.i log and `damage ls` output for details 2024-05-10T11:37:00.934 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:37:00 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:37:00.934 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:37:00 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:37:00.934 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:37:00 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:37:01.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:37:00 smithi178 ceph-mon[33359]: scrub queued for path: / 2024-05-10T11:37:01.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:37:00 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:37:01.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:37:00 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:37:01.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:37:00 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:37:01.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:37:00 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:37:01.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:37:00 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:37:01.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:37:00 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:37:01.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:37:00 smithi178 ceph-mon[33359]: Scrub error on inode 0x100000004d2 (/volumes/qa/sv_0/96765652-39ef-4cf3-b20a-a70f7e9d774d/client.0/tmp/clients/client8/~dmtmp/COREL) see mds.i log and `damage ls` output for details 2024-05-10T11:37:01.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:37:00 smithi178 ceph-mon[33359]: Scrub error on inode 0x100000004a2 (/volumes/qa/sv_0/96765652-39ef-4cf3-b20a-a70f7e9d774d/client.0/tmp/clients/client7/~dmtmp/PWRPNT) see mds.i log and `damage ls` output for details 2024-05-10T11:37:01.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:37:00 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:37:01.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:37:00 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:37:01.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:37:00 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:37:01.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:37:00 smithi188 ceph-mon[34075]: scrub queued for path: / 2024-05-10T11:37:01.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:37:00 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:37:01.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:37:00 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:37:01.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:37:00 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:37:01.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:37:00 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:37:01.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:37:00 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:37:01.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:37:00 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:37:01.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:37:00 smithi188 ceph-mon[34075]: Scrub error on inode 0x100000004d2 (/volumes/qa/sv_0/96765652-39ef-4cf3-b20a-a70f7e9d774d/client.0/tmp/clients/client8/~dmtmp/COREL) see mds.i log and `damage ls` output for details 2024-05-10T11:37:01.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:37:00 smithi188 ceph-mon[34075]: Scrub error on inode 0x100000004a2 (/volumes/qa/sv_0/96765652-39ef-4cf3-b20a-a70f7e9d774d/client.0/tmp/clients/client7/~dmtmp/PWRPNT) see mds.i log and `damage ls` output for details 2024-05-10T11:37:01.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:37:00 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:37:01.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:37:00 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:37:01.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:37:00 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:37:01.932 INFO:tasks.workunit.client.0.smithi100.stdout: 10 49610 9.06 MB/sec execute 175 sec latency 67.356 ms 2024-05-10T11:37:02.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:37:01 smithi178 ceph-mon[33359]: pgmap v980: 129 pgs: 129 active+clean; 468 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 4.5 MiB/s wr, 315 op/s 2024-05-10T11:37:02.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:37:01 smithi188 ceph-mon[34075]: pgmap v980: 129 pgs: 129 active+clean; 468 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 4.5 MiB/s wr, 315 op/s 2024-05-10T11:37:02.034 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:37:01 smithi100 ceph-mon[30296]: pgmap v980: 129 pgs: 129 active+clean; 468 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 4.5 MiB/s wr, 315 op/s 2024-05-10T11:37:02.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.506+0000 7f0ba70cd700 1 -- 172.21.15.100:0/690792551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ba0103540 msgr2=0x7f0ba01039c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:02.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.506+0000 7f0ba70cd700 1 --2- 172.21.15.100:0/690792551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ba0103540 0x7f0ba01039c0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c009920 tx=0x7f0b8c009c30 comp rx=0 tx=0).stop 2024-05-10T11:37:02.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.506+0000 7f0ba70cd700 1 -- 172.21.15.100:0/690792551 shutdown_connections 2024-05-10T11:37:02.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.506+0000 7f0ba70cd700 1 --2- 172.21.15.100:0/690792551 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ba0103f00 0x7f0ba01103b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:02.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.506+0000 7f0ba70cd700 1 --2- 172.21.15.100:0/690792551 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ba01095e0 0x7f0ba01099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:02.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.506+0000 7f0ba70cd700 1 --2- 172.21.15.100:0/690792551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ba0103540 0x7f0ba01039c0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:02.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.506+0000 7f0ba70cd700 1 -- 172.21.15.100:0/690792551 >> 172.21.15.100:0/690792551 conn(0x7f0ba007a650 msgr2=0x7f0ba007aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:37:02.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.506+0000 7f0ba70cd700 1 -- 172.21.15.100:0/690792551 shutdown_connections 2024-05-10T11:37:02.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.506+0000 7f0ba70cd700 1 -- 172.21.15.100:0/690792551 wait complete. 2024-05-10T11:37:02.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.507+0000 7f0ba70cd700 1 Processor -- start 2024-05-10T11:37:02.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.507+0000 7f0ba70cd700 1 -- start start 2024-05-10T11:37:02.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.507+0000 7f0ba70cd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ba0103540 0x7f0ba00757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:02.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.507+0000 7f0ba70cd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ba0103f00 0x7f0ba0075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:02.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.507+0000 7f0ba70cd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ba01095e0 0x7f0ba0076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:02.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.507+0000 7f0ba70cd700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba01135f0 con 0x7f0ba0103f00 2024-05-10T11:37:02.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.507+0000 7f0ba70cd700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba0113470 con 0x7f0ba0103540 2024-05-10T11:37:02.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.507+0000 7f0ba70cd700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba0113770 con 0x7f0ba01095e0 2024-05-10T11:37:02.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.507+0000 7f0b9ffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ba0103f00 0x7f0ba0075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:02.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.507+0000 7f0b9ffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ba0103f00 0x7f0ba0075ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33192/0 (socket says 172.21.15.100:33192) 2024-05-10T11:37:02.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.507+0000 7f0ba4e69700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ba0103540 0x7f0ba00757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:02.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.507+0000 7f0b9ffff700 1 -- 172.21.15.100:0/3343789412 learned_addr learned my addr 172.21.15.100:0/3343789412 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:37:02.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.507+0000 7f0ba566a700 1 --2- 172.21.15.100:0/3343789412 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ba01095e0 0x7f0ba0076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:02.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.508+0000 7f0ba4e69700 1 -- 172.21.15.100:0/3343789412 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ba01095e0 msgr2=0x7f0ba0076220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:02.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.508+0000 7f0ba4e69700 1 --2- 172.21.15.100:0/3343789412 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ba01095e0 0x7f0ba0076220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:02.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.508+0000 7f0ba4e69700 1 -- 172.21.15.100:0/3343789412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ba0103f00 msgr2=0x7f0ba0075ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:02.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.508+0000 7f0ba4e69700 1 --2- 172.21.15.100:0/3343789412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ba0103f00 0x7f0ba0075ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:02.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.508+0000 7f0ba4e69700 1 -- 172.21.15.100:0/3343789412 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ba01a8050 con 0x7f0ba0103540 2024-05-10T11:37:02.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.508+0000 7f0b9ffff700 1 --2- 172.21.15.100:0/3343789412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ba0103f00 0x7f0ba0075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:37:02.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.508+0000 7f0ba566a700 1 --2- 172.21.15.100:0/3343789412 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ba01095e0 0x7f0ba0076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:37:02.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.508+0000 7f0ba4e69700 1 --2- 172.21.15.100:0/3343789412 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ba0103540 0x7f0ba00757a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0b9400e9f0 tx=0x7f0b9400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:37:02.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.508+0000 7f0b9dffb700 1 -- 172.21.15.100:0/3343789412 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b9400cb70 con 0x7f0ba0103540 2024-05-10T11:37:02.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.508+0000 7f0ba70cd700 1 -- 172.21.15.100:0/3343789412 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ba01a8340 con 0x7f0ba0103540 2024-05-10T11:37:02.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.508+0000 7f0b9dffb700 1 -- 172.21.15.100:0/3343789412 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0b940044b0 con 0x7f0ba0103540 2024-05-10T11:37:02.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.508+0000 7f0b9dffb700 1 -- 172.21.15.100:0/3343789412 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b94003ca0 con 0x7f0ba0103540 2024-05-10T11:37:02.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.508+0000 7f0ba70cd700 1 -- 172.21.15.100:0/3343789412 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ba01a88d0 con 0x7f0ba0103540 2024-05-10T11:37:02.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.511+0000 7f0b9dffb700 1 -- 172.21.15.100:0/3343789412 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f0b94003f60 con 0x7f0ba0103540 2024-05-10T11:37:02.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.512+0000 7f0b9dffb700 1 --2- 172.21.15.100:0/3343789412 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0b900b2900 0x7f0b900b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:02.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.512+0000 7f0b9dffb700 1 -- 172.21.15.100:0/3343789412 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f0b940e2720 con 0x7f0ba0103540 2024-05-10T11:37:02.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.512+0000 7f0b9ffff700 1 --2- 172.21.15.100:0/3343789412 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0b900b2900 0x7f0b900b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:02.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.513+0000 7f0b9ffff700 1 --2- 172.21.15.100:0/3343789412 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0b900b2900 0x7f0b900b4dc0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c009680 tx=0x7f0b8c019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:37:02.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.514+0000 7f0bb1960b80 1 Processor -- start 2024-05-10T11:37:02.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.514+0000 7f0bb1960b80 1 -- start start 2024-05-10T11:37:02.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.514+0000 7f0bb1960b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da4122c2b0 0x55da4122c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:02.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.514+0000 7f0bb1960b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da4122cbd0 0x55da41226b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:02.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.514+0000 7f0bb1960b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55da41227120 0x55da41227590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:02.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.514+0000 7f0bb1960b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55da41170860 con 0x55da4122cbd0 2024-05-10T11:37:02.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.514+0000 7f0bb1960b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55da4110f7c0 con 0x55da4122c2b0 2024-05-10T11:37:02.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.514+0000 7f0bb1960b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55da4116c820 con 0x55da41227120 2024-05-10T11:37:02.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.514+0000 7f0b9ffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da4122cbd0 0x55da41226b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:02.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.514+0000 7f0ba4e69700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da4122c2b0 0x55da4122c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:02.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.514+0000 7f0ba4e69700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da4122c2b0 0x55da4122c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:39882/0 (socket says 172.21.15.100:39882) 2024-05-10T11:37:02.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.514+0000 7f0ba4e69700 1 -- 172.21.15.100:0/4180319272 learned_addr learned my addr 172.21.15.100:0/4180319272 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:37:02.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.515+0000 7f0ba566a700 1 --2- 172.21.15.100:0/4180319272 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55da41227120 0x55da41227590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:02.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.515+0000 7f0ba4e69700 1 -- 172.21.15.100:0/4180319272 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55da41227120 msgr2=0x55da41227590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:37:02.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.515+0000 7f0ba4e69700 1 --2- 172.21.15.100:0/4180319272 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55da41227120 0x55da41227590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:02.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.515+0000 7f0ba4e69700 1 -- 172.21.15.100:0/4180319272 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da4122cbd0 msgr2=0x55da41226b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:37:02.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.515+0000 7f0ba4e69700 1 --2- 172.21.15.100:0/4180319272 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da4122cbd0 0x55da41226b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:02.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.515+0000 7f0ba4e69700 1 -- 172.21.15.100:0/4180319272 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da410bd8d0 con 0x55da4122c2b0 2024-05-10T11:37:02.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.515+0000 7f0ba4e69700 1 --2- 172.21.15.100:0/4180319272 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da4122c2b0 0x55da4122c690 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0b940a9970 tx=0x7f0b940a9150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c39eed8ddf836c8c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:37:02.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.515+0000 7f0b8affd700 1 -- 172.21.15.100:0/4180319272 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b940aa0e0 con 0x55da4122c2b0 2024-05-10T11:37:02.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.515+0000 7f0b8affd700 1 -- 172.21.15.100:0/4180319272 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0b940a99e0 con 0x55da4122c2b0 2024-05-10T11:37:02.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.515+0000 7f0b8affd700 1 -- 172.21.15.100:0/4180319272 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b94004250 con 0x55da4122c2b0 2024-05-10T11:37:02.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.516+0000 7f0bb1960b80 1 -- 172.21.15.100:0/4180319272 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da4122c2b0 msgr2=0x55da4122c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:02.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.516+0000 7f0bb1960b80 1 --2- 172.21.15.100:0/4180319272 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da4122c2b0 0x55da4122c690 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0b940a9970 tx=0x7f0b940a9150 comp rx=0 tx=0).stop 2024-05-10T11:37:02.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.516+0000 7f0bb1960b80 1 -- 172.21.15.100:0/4180319272 shutdown_connections 2024-05-10T11:37:02.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.516+0000 7f0bb1960b80 1 --2- 172.21.15.100:0/4180319272 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55da41227120 0x55da41227590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:02.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.516+0000 7f0bb1960b80 1 --2- 172.21.15.100:0/4180319272 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da4122c2b0 0x55da4122c690 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:02.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.516+0000 7f0bb1960b80 1 --2- 172.21.15.100:0/4180319272 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da4122cbd0 0x55da41226b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:02.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.516+0000 7f0bb1960b80 1 -- 172.21.15.100:0/4180319272 >> 172.21.15.100:0/4180319272 conn(0x55da4114a100 msgr2=0x55da41218c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:37:02.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.517+0000 7f0bb1960b80 1 -- 172.21.15.100:0/4180319272 shutdown_connections 2024-05-10T11:37:02.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.517+0000 7f0bb1960b80 1 -- 172.21.15.100:0/4180319272 wait complete. 2024-05-10T11:37:02.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.517+0000 7f0bb1960b80 1 -- start start 2024-05-10T11:37:02.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.517+0000 7f0bb1960b80 1 Processor -- start 2024-05-10T11:37:02.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.518+0000 7f0bb1960b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55da41227120 0x55da41232970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:02.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.518+0000 7f0bb1960b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da4122c2b0 0x55da4122fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:02.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.518+0000 7f0bb1960b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da4122cbd0 0x55da41230250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:02.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.518+0000 7f0bb1960b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55da41233680 con 0x55da4122cbd0 2024-05-10T11:37:02.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.518+0000 7f0bb1960b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55da410d6c30 con 0x55da4122c2b0 2024-05-10T11:37:02.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.518+0000 7f0bb1960b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55da41233800 con 0x55da41227120 2024-05-10T11:37:02.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.518+0000 7f0ba4e69700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55da41227120 0x55da41232970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:02.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.518+0000 7f0b9ffff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da4122c2b0 0x55da4122fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:02.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.518+0000 7f0ba4e69700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55da41227120 0x55da41232970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:53966/0 (socket says 172.21.15.100:53966) 2024-05-10T11:37:02.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.518+0000 7f0ba4e69700 1 -- 172.21.15.100:0/2428483554 learned_addr learned my addr 172.21.15.100:0/2428483554 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:37:02.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.518+0000 7f0b9ffff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da4122c2b0 0x55da4122fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:39898/0 (socket says 172.21.15.100:39898) 2024-05-10T11:37:02.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.518+0000 7f0ba566a700 1 --2- 172.21.15.100:0/2428483554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da4122cbd0 0x55da41230250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:02.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.518+0000 7f0bb1960b80 10 client.0 _ll_register_callbacks cb 0x55da41108920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:37:02.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.518+0000 7f0ba4e69700 1 -- 172.21.15.100:0/2428483554 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da4122c2b0 msgr2=0x55da4122fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:37:02.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.518+0000 7f0ba4e69700 1 --2- 172.21.15.100:0/2428483554 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da4122c2b0 0x55da4122fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:02.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.518+0000 7f0ba4e69700 1 -- 172.21.15.100:0/2428483554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da4122cbd0 msgr2=0x55da41230250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:37:02.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.518+0000 7f0ba4e69700 1 --2- 172.21.15.100:0/2428483554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da4122cbd0 0x55da41230250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:02.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.518+0000 7f0ba4e69700 1 -- 172.21.15.100:0/2428483554 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da4110c690 con 0x55da41227120 2024-05-10T11:37:02.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.518+0000 7f0ba4e69700 1 --2- 172.21.15.100:0/2428483554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55da41227120 0x55da41232970 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f0b94009230 tx=0x7f0b940036f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6de32f1184c34885 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:37:02.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.518+0000 7f0b7ffff700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:37:02.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.519+0000 7f0b7ffff700 1 -- 172.21.15.100:0/2428483554 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b940c2030 con 0x55da41227120 2024-05-10T11:37:02.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.519+0000 7f0b7ffff700 1 -- 172.21.15.100:0/2428483554 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f0b940a5fd0 con 0x55da41227120 2024-05-10T11:37:02.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.519+0000 7f0bb1960b80 1 -- 172.21.15.100:0/2428483554 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b940a99e0 con 0x55da41227120 2024-05-10T11:37:02.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.519+0000 7f0b7ffff700 1 -- 172.21.15.100:0/2428483554 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f0b940add90 con 0x55da41227120 2024-05-10T11:37:02.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.519+0000 7f0bb1960b80 10 client.28507 fetch_fsmap learned FSMap version 41 2024-05-10T11:37:02.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.519+0000 7f0bb1960b80 1 -- 172.21.15.100:0/2428483554 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55da41233680 con 0x55da41227120 2024-05-10T11:37:02.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.519+0000 7f0b7ffff700 1 -- 172.21.15.100:0/2428483554 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0b940a99e0 con 0x55da41227120 2024-05-10T11:37:02.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.519+0000 7f0b7ffff700 1 -- 172.21.15.100:0/2428483554 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f0b940cc0c0 con 0x55da41227120 2024-05-10T11:37:02.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.519+0000 7f0bb1960b80 10 client.28507 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:37:02.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.519+0000 7f0bb1960b80 10 client.28507 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:37:02.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.519+0000 7f0bb1960b80 20 client.28507 populate_metadata read hostname 'smithi100' 2024-05-10T11:37:02.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.519+0000 7f0bb1960b80 1 --2- 172.21.15.100:0/2428483554 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55da412401f0 0x55da412425e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:02.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.520+0000 7f0bb1960b80 4 client.28507 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:37:02.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.520+0000 7f0bb1960b80 1 -- 172.21.15.100:0/2428483554 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da411ea650 con 0x55da412401f0 2024-05-10T11:37:02.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.520+0000 7f0b9ffff700 1 --2- 172.21.15.100:0/2428483554 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55da412401f0 0x55da412425e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:02.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.520+0000 7f0b9ffff700 1 --2- 172.21.15.100:0/2428483554 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55da412401f0 0x55da412425e0 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a7cdfddcea80f77 server_cookie=a71c760edea6444b in_seq=0 out_seq=0 2024-05-10T11:37:02.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.520+0000 7f0b7ffff700 10 client.28507 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:37:02.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.521+0000 7f0b7ffff700 1 -- 172.21.15.100:0/2428483554 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55da411ea650 con 0x55da412401f0 2024-05-10T11:37:02.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.521+0000 7f0b7ffff700 10 client.28507 handle_command_reply: tid=0 2024-05-10T11:37:02.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.521+0000 7f0bb1960b80 1 client.28507 shutdown 2024-05-10T11:37:02.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.522+0000 7f0bb1960b80 1 -- 172.21.15.100:0/2428483554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55da41227120 msgr2=0x55da41232970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:02.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.522+0000 7f0bb1960b80 1 --2- 172.21.15.100:0/2428483554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55da41227120 0x55da41232970 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f0b94009230 tx=0x7f0b940036f0 comp rx=0 tx=0).stop 2024-05-10T11:37:02.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.522+0000 7f0bb1960b80 1 -- 172.21.15.100:0/2428483554 shutdown_connections 2024-05-10T11:37:02.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.522+0000 7f0bb1960b80 1 --2- 172.21.15.100:0/2428483554 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55da412401f0 0x55da412425e0 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:02.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.522+0000 7f0bb1960b80 1 --2- 172.21.15.100:0/2428483554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da4122cbd0 0x55da41230250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:02.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.522+0000 7f0bb1960b80 1 --2- 172.21.15.100:0/2428483554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55da41227120 0x55da41232970 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:02.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.522+0000 7f0bb1960b80 1 --2- 172.21.15.100:0/2428483554 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da4122c2b0 0x55da4122fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:02.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.522+0000 7f0bb1960b80 1 -- 172.21.15.100:0/2428483554 >> 172.21.15.100:0/2428483554 conn(0x55da4114a100 msgr2=0x55da41213720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:37:02.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.522+0000 7f0b7ffff700 0 client.28507 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:37:02.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.522+0000 7f0bb1960b80 1 -- 172.21.15.100:0/2428483554 shutdown_connections 2024-05-10T11:37:02.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.522+0000 7f0bb1960b80 1 -- 172.21.15.100:0/2428483554 wait complete. 2024-05-10T11:37:02.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.522+0000 7f0bb1960b80 20 client.28507 trim_cache size 0 max 16384 2024-05-10T11:37:02.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.554+0000 7f0bb1960b80 1 Processor -- start 2024-05-10T11:37:02.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.554+0000 7f0bb1960b80 1 -- start start 2024-05-10T11:37:02.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.555+0000 7f0bb1960b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da41227120 0x55da4122a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:02.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.555+0000 7f0bb1960b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da4122c2b0 0x55da4123c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:02.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.555+0000 7f0ba4e69700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da41227120 0x55da4122a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:02.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.555+0000 7f0ba4e69700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da41227120 0x55da4122a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:39906/0 (socket says 172.21.15.100:39906) 2024-05-10T11:37:02.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.555+0000 7f0ba4e69700 1 -- 172.21.15.100:0/4228101209 learned_addr learned my addr 172.21.15.100:0/4228101209 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:37:02.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.555+0000 7f0bb1960b80 1 --2- 172.21.15.100:0/4228101209 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55da4122cbd0 0x55da4122d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:02.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.555+0000 7f0bb1960b80 1 -- 172.21.15.100:0/4228101209 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba01af900 con 0x55da4122c2b0 2024-05-10T11:37:02.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.555+0000 7f0bb1960b80 1 -- 172.21.15.100:0/4228101209 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba01ad190 con 0x55da41227120 2024-05-10T11:37:02.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.555+0000 7f0bb1960b80 1 -- 172.21.15.100:0/4228101209 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba01ab000 con 0x55da4122cbd0 2024-05-10T11:37:02.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.555+0000 7f0ba566a700 1 --2- 172.21.15.100:0/4228101209 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55da4122cbd0 0x55da4122d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:02.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.555+0000 7f0b9ffff700 1 --2- 172.21.15.100:0/4228101209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da4122c2b0 0x55da4123c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:02.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.555+0000 7f0ba4e69700 1 -- 172.21.15.100:0/4228101209 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55da4122cbd0 msgr2=0x55da4122d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:37:02.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.555+0000 7f0ba4e69700 1 --2- 172.21.15.100:0/4228101209 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55da4122cbd0 0x55da4122d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:02.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.555+0000 7f0ba4e69700 1 -- 172.21.15.100:0/4228101209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da4122c2b0 msgr2=0x55da4123c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:37:02.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.555+0000 7f0ba4e69700 1 --2- 172.21.15.100:0/4228101209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da4122c2b0 0x55da4123c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:02.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.555+0000 7f0ba4e69700 1 -- 172.21.15.100:0/4228101209 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da41226d50 con 0x55da41227120 2024-05-10T11:37:02.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.555+0000 7f0ba566a700 1 --2- 172.21.15.100:0/4228101209 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55da4122cbd0 0x55da4122d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:37:02.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.556+0000 7f0ba4e69700 1 --2- 172.21.15.100:0/4228101209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da41227120 0x55da4122a470 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0b940e0230 tx=0x7f0b940a95b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=321b6464004ae79a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:37:02.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.556+0000 7f0b8a7fc700 1 -- 172.21.15.100:0/4228101209 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b940aa0e0 con 0x55da41227120 2024-05-10T11:37:02.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.556+0000 7f0b8a7fc700 1 -- 172.21.15.100:0/4228101209 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0b940a99e0 con 0x55da41227120 2024-05-10T11:37:02.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.556+0000 7f0b8a7fc700 1 -- 172.21.15.100:0/4228101209 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b940a5460 con 0x55da41227120 2024-05-10T11:37:02.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.557+0000 7f0bb1960b80 1 -- 172.21.15.100:0/4228101209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da41227120 msgr2=0x55da4122a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:02.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.557+0000 7f0bb1960b80 1 --2- 172.21.15.100:0/4228101209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da41227120 0x55da4122a470 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0b940e0230 tx=0x7f0b940a95b0 comp rx=0 tx=0).stop 2024-05-10T11:37:02.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.557+0000 7f0bb1960b80 1 -- 172.21.15.100:0/4228101209 shutdown_connections 2024-05-10T11:37:02.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.557+0000 7f0bb1960b80 1 --2- 172.21.15.100:0/4228101209 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55da4122cbd0 0x55da4122d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:02.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.557+0000 7f0bb1960b80 1 --2- 172.21.15.100:0/4228101209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da41227120 0x55da4122a470 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:02.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.557+0000 7f0bb1960b80 1 --2- 172.21.15.100:0/4228101209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da4122c2b0 0x55da4123c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:02.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.557+0000 7f0bb1960b80 1 -- 172.21.15.100:0/4228101209 >> 172.21.15.100:0/4228101209 conn(0x55da4114a100 msgr2=0x55da41212e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:37:02.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.557+0000 7f0bb1960b80 1 -- 172.21.15.100:0/4228101209 shutdown_connections 2024-05-10T11:37:02.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.557+0000 7f0bb1960b80 1 -- 172.21.15.100:0/4228101209 wait complete. 2024-05-10T11:37:02.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.558+0000 7f0bb1960b80 1 -- start start 2024-05-10T11:37:02.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.558+0000 7f0bb1960b80 1 Processor -- start 2024-05-10T11:37:02.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.558+0000 7f0bb1960b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da41227120 0x55da412345a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:02.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.558+0000 7f0bb1960b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55da4122c2b0 0x55da41234ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:02.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.559+0000 7f0bb1960b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da4122cbd0 0x55da41235020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:02.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.559+0000 7f0bb1960b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba01ab650 con 0x55da41227120 2024-05-10T11:37:02.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.559+0000 7f0bb1960b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba01acac0 con 0x55da4122cbd0 2024-05-10T11:37:02.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.559+0000 7f0bb1960b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba01abd20 con 0x55da4122c2b0 2024-05-10T11:37:02.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.559+0000 7f0bb1960b80 10 client.0 _ll_register_callbacks cb 0x55da41108920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:37:02.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.559+0000 7f0b9ffff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55da4122c2b0 0x55da41234ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:02.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.559+0000 7f0ba4e69700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da41227120 0x55da412345a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:02.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.559+0000 7f0b9ffff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55da4122c2b0 0x55da41234ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:53976/0 (socket says 172.21.15.100:53976) 2024-05-10T11:37:02.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.559+0000 7f0b9ffff700 1 -- 172.21.15.100:0/2630186141 learned_addr learned my addr 172.21.15.100:0/2630186141 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:37:02.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.559+0000 7f0ba566a700 1 --2- 172.21.15.100:0/2630186141 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da4122cbd0 0x55da41235020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:02.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.559+0000 7f0b9ffff700 1 -- 172.21.15.100:0/2630186141 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da4122cbd0 msgr2=0x55da41235020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:37:02.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.559+0000 7f0b9ffff700 1 --2- 172.21.15.100:0/2630186141 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da4122cbd0 0x55da41235020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:02.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.559+0000 7f0b9ffff700 1 -- 172.21.15.100:0/2630186141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da41227120 msgr2=0x55da412345a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:37:02.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.559+0000 7f0b9ffff700 1 --2- 172.21.15.100:0/2630186141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da41227120 0x55da412345a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:02.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.559+0000 7f0b9ffff700 1 -- 172.21.15.100:0/2630186141 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da412a98d0 con 0x55da4122c2b0 2024-05-10T11:37:02.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.559+0000 7f0b9ffff700 1 --2- 172.21.15.100:0/2630186141 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55da4122c2b0 0x55da41234ae0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c000c00 tx=0x7f0b8c009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5595dc27ecfb79f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:37:02.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.559+0000 7f0b7ffff700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:37:02.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.559+0000 7f0b7ffff700 1 -- 172.21.15.100:0/2630186141 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b8c03d070 con 0x55da4122c2b0 2024-05-10T11:37:02.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.559+0000 7f0b7ffff700 1 -- 172.21.15.100:0/2630186141 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f0b8c005450 con 0x55da4122c2b0 2024-05-10T11:37:02.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.560+0000 7f0bb1960b80 1 -- 172.21.15.100:0/2630186141 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b940a99e0 con 0x55da4122c2b0 2024-05-10T11:37:02.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.560+0000 7f0b7ffff700 1 -- 172.21.15.100:0/2630186141 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f0b8c045050 con 0x55da4122c2b0 2024-05-10T11:37:02.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.560+0000 7f0bb1960b80 10 client.28513 fetch_fsmap learned FSMap version 41 2024-05-10T11:37:02.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.560+0000 7f0bb1960b80 1 -- 172.21.15.100:0/2630186141 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55da412aab60 con 0x55da4122c2b0 2024-05-10T11:37:02.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.560+0000 7f0b7ffff700 1 -- 172.21.15.100:0/2630186141 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0b8c005650 con 0x55da4122c2b0 2024-05-10T11:37:02.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.560+0000 7f0b7ffff700 1 -- 172.21.15.100:0/2630186141 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f0b8c040030 con 0x55da4122c2b0 2024-05-10T11:37:02.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.560+0000 7f0bb1960b80 10 client.28513 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:37:02.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.560+0000 7f0bb1960b80 10 client.28513 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:37:02.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.560+0000 7f0bb1960b80 20 client.28513 populate_metadata read hostname 'smithi100' 2024-05-10T11:37:02.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.560+0000 7f0bb1960b80 1 --2- 172.21.15.100:0/2630186141 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55da412ab2d0 0x55da412ad6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:02.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.560+0000 7f0bb1960b80 4 client.28513 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T11:37:02.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.560+0000 7f0bb1960b80 1 -- 172.21.15.100:0/2630186141 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f0b940a99e0 con 0x55da412ab2d0 2024-05-10T11:37:02.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.561+0000 7f0ba4e69700 1 --2- 172.21.15.100:0/2630186141 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55da412ab2d0 0x55da412ad6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:02.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.561+0000 7f0ba4e69700 1 --2- 172.21.15.100:0/2630186141 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55da412ab2d0 0x55da412ad6f0 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea3b1dc47555eab3 server_cookie=723469655517253b in_seq=0 out_seq=0 2024-05-10T11:37:02.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.561+0000 7f0b7ffff700 10 client.28513 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:37:02.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.562+0000 7f0b7ffff700 1 -- 172.21.15.100:0/2630186141 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f0b940a99e0 con 0x55da412ab2d0 2024-05-10T11:37:02.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.562+0000 7f0b7ffff700 10 client.28513 handle_command_reply: tid=0 2024-05-10T11:37:02.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.562+0000 7f0bb1960b80 1 client.28513 shutdown 2024-05-10T11:37:02.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.562+0000 7f0bb1960b80 1 -- 172.21.15.100:0/2630186141 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55da4122c2b0 msgr2=0x55da41234ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:02.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.562+0000 7f0bb1960b80 1 --2- 172.21.15.100:0/2630186141 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55da4122c2b0 0x55da41234ae0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c000c00 tx=0x7f0b8c009260 comp rx=0 tx=0).stop 2024-05-10T11:37:02.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.562+0000 7f0bb1960b80 1 -- 172.21.15.100:0/2630186141 shutdown_connections 2024-05-10T11:37:02.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.562+0000 7f0bb1960b80 1 --2- 172.21.15.100:0/2630186141 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55da412ab2d0 0x55da412ad6f0 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:02.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.562+0000 7f0bb1960b80 1 --2- 172.21.15.100:0/2630186141 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da4122cbd0 0x55da41235020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:02.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.562+0000 7f0b7ffff700 0 client.28513 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:37:02.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.562+0000 7f0bb1960b80 1 --2- 172.21.15.100:0/2630186141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da41227120 0x55da412345a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:02.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.562+0000 7f0bb1960b80 1 --2- 172.21.15.100:0/2630186141 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55da4122c2b0 0x55da41234ae0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:02.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.562+0000 7f0bb1960b80 1 -- 172.21.15.100:0/2630186141 >> 172.21.15.100:0/2630186141 conn(0x55da4114a100 msgr2=0x55da41213720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:37:02.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.562+0000 7f0bb1960b80 1 -- 172.21.15.100:0/2630186141 shutdown_connections 2024-05-10T11:37:02.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.562+0000 7f0bb1960b80 1 -- 172.21.15.100:0/2630186141 wait complete. 2024-05-10T11:37:02.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.562+0000 7f0bb1960b80 20 client.28513 trim_cache size 0 max 16384 2024-05-10T11:37:02.567 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:37:02.567 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T11:37:02.567 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-05-10T11:37:02.567 INFO:teuthology.orchestra.run.smithi100.stdout: "c9b068d2-0ad9-440e-b804-c3de6c8852ba": { 2024-05-10T11:37:02.567 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-05-10T11:37:02.567 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "c9b068d2-0ad9-440e-b804-c3de6c8852ba", 2024-05-10T11:37:02.567 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-05-10T11:37:02.567 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:37:02.567 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:37:02.567 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:37:02.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.563+0000 7f0ba70cd700 1 -- 172.21.15.100:0/3343789412 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0b900b2900 msgr2=0x7f0b900b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:02.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.563+0000 7f0ba70cd700 1 --2- 172.21.15.100:0/3343789412 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0b900b2900 0x7f0b900b4dc0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c009680 tx=0x7f0b8c019040 comp rx=0 tx=0).stop 2024-05-10T11:37:02.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.563+0000 7f0ba70cd700 1 -- 172.21.15.100:0/3343789412 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ba0103540 msgr2=0x7f0ba00757a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:02.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.563+0000 7f0ba70cd700 1 --2- 172.21.15.100:0/3343789412 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ba0103540 0x7f0ba00757a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0b9400e9f0 tx=0x7f0b9400ed00 comp rx=0 tx=0).stop 2024-05-10T11:37:02.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.563+0000 7f0ba70cd700 1 -- 172.21.15.100:0/3343789412 shutdown_connections 2024-05-10T11:37:02.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.563+0000 7f0ba70cd700 1 --2- 172.21.15.100:0/3343789412 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0b900b2900 0x7f0b900b4dc0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:02.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.563+0000 7f0ba70cd700 1 --2- 172.21.15.100:0/3343789412 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ba01095e0 0x7f0ba0076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:02.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.563+0000 7f0ba70cd700 1 --2- 172.21.15.100:0/3343789412 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ba0103540 0x7f0ba00757a0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:02.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.563+0000 7f0ba70cd700 1 --2- 172.21.15.100:0/3343789412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ba0103f00 0x7f0ba0075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:02.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.563+0000 7f0ba70cd700 1 -- 172.21.15.100:0/3343789412 >> 172.21.15.100:0/3343789412 conn(0x7f0ba007a650 msgr2=0x7f0ba01023f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:37:02.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.563+0000 7f0ba70cd700 1 -- 172.21.15.100:0/3343789412 shutdown_connections 2024-05-10T11:37:02.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:02.563+0000 7f0ba70cd700 1 -- 172.21.15.100:0/3343789412 wait complete. 2024-05-10T11:37:02.932 INFO:tasks.workunit.client.0.smithi100.stdout: 10 49763 9.05 MB/sec execute 176 sec latency 205.794 ms 2024-05-10T11:37:03.311 INFO:tasks.cephfs.filesystem:scrub status for tag:c9b068d2-0ad9-440e-b804-c3de6c8852ba - {'path': '/', 'tag': 'c9b068d2-0ad9-440e-b804-c3de6c8852ba', 'options': 'recursive,force'} 2024-05-10T11:37:03.932 INFO:tasks.workunit.client.0.smithi100.stdout: 10 49921 9.04 MB/sec execute 177 sec latency 165.175 ms 2024-05-10T11:37:04.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:37:03 smithi178 ceph-mon[33359]: pgmap v981: 129 pgs: 129 active+clean; 471 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 822 KiB/s rd, 4.3 MiB/s wr, 322 op/s 2024-05-10T11:37:04.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:37:03 smithi188 ceph-mon[34075]: pgmap v981: 129 pgs: 129 active+clean; 471 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 822 KiB/s rd, 4.3 MiB/s wr, 322 op/s 2024-05-10T11:37:04.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:37:03 smithi100 ceph-mon[30296]: pgmap v981: 129 pgs: 129 active+clean; 471 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 822 KiB/s rd, 4.3 MiB/s wr, 322 op/s 2024-05-10T11:37:04.932 INFO:tasks.workunit.client.0.smithi100.stdout: 10 50082 9.01 MB/sec execute 178 sec latency 77.669 ms 2024-05-10T11:37:05.932 INFO:tasks.workunit.client.0.smithi100.stdout: 10 50253 9.02 MB/sec execute 179 sec latency 81.783 ms 2024-05-10T11:37:06.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:37:05 smithi178 ceph-mon[33359]: pgmap v982: 129 pgs: 129 active+clean; 473 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 877 KiB/s rd, 4.5 MiB/s wr, 327 op/s 2024-05-10T11:37:06.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:37:05 smithi188 ceph-mon[34075]: pgmap v982: 129 pgs: 129 active+clean; 473 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 877 KiB/s rd, 4.5 MiB/s wr, 327 op/s 2024-05-10T11:37:06.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:37:05 smithi100 ceph-mon[30296]: pgmap v982: 129 pgs: 129 active+clean; 473 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 877 KiB/s rd, 4.5 MiB/s wr, 327 op/s 2024-05-10T11:37:06.932 INFO:tasks.workunit.client.0.smithi100.stdout: 10 50425 9.01 MB/sec execute 180 sec latency 73.696 ms 2024-05-10T11:37:07.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:37:07 smithi100 ceph-mon[30296]: pgmap v983: 129 pgs: 129 active+clean; 474 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 5.0 MiB/s wr, 378 op/s 2024-05-10T11:37:07.932 INFO:tasks.workunit.client.0.smithi100.stdout: 10 50624 9.05 MB/sec execute 181 sec latency 102.498 ms 2024-05-10T11:37:08.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:37:07 smithi178 ceph-mon[33359]: pgmap v983: 129 pgs: 129 active+clean; 474 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 5.0 MiB/s wr, 378 op/s 2024-05-10T11:37:08.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:37:07 smithi188 ceph-mon[34075]: pgmap v983: 129 pgs: 129 active+clean; 474 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 5.0 MiB/s wr, 378 op/s 2024-05-10T11:37:08.742 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.741+0000 7f5624d9d640 1 -- 172.21.15.100:0/3909652346 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5620105dc0 msgr2=0x7f56201081b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:08.742 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.741+0000 7f5624d9d640 1 --2- 172.21.15.100:0/3909652346 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5620105dc0 0x7f56201081b0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f5614009900 tx=0x7f561402a3f0 comp rx=0 tx=0).stop 2024-05-10T11:37:08.742 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.742+0000 7f5624d9d640 1 -- 172.21.15.100:0/3909652346 shutdown_connections 2024-05-10T11:37:08.742 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.742+0000 7f5624d9d640 1 --2- 172.21.15.100:0/3909652346 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56201086f0 0x7f562010ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:08.743 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.742+0000 7f5624d9d640 1 --2- 172.21.15.100:0/3909652346 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5620105dc0 0x7f56201081b0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:08.743 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.742+0000 7f5624d9d640 1 --2- 172.21.15.100:0/3909652346 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5620103490 0x7f5620105880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:08.743 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.742+0000 7f5624d9d640 1 -- 172.21.15.100:0/3909652346 >> 172.21.15.100:0/3909652346 conn(0x7f56200fce80 msgr2=0x7f56200ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:37:08.743 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.742+0000 7f5624d9d640 1 -- 172.21.15.100:0/3909652346 shutdown_connections 2024-05-10T11:37:08.743 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.742+0000 7f5624d9d640 1 -- 172.21.15.100:0/3909652346 wait complete. 2024-05-10T11:37:08.743 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.743+0000 7f5624d9d640 1 Processor -- start 2024-05-10T11:37:08.744 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.743+0000 7f5624d9d640 1 -- start start 2024-05-10T11:37:08.744 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.743+0000 7f5624d9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5620103490 0x7f562019ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:08.744 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.743+0000 7f5624d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5620105dc0 0x7f562019b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:08.744 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.743+0000 7f5624d9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56201086f0 0x7f56201a22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:08.744 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.743+0000 7f5624d9d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5620102960 con 0x7f5620105dc0 2024-05-10T11:37:08.744 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.743+0000 7f5624d9d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f56201027e0 con 0x7f56201086f0 2024-05-10T11:37:08.744 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.743+0000 7f5624d9d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5620102ae0 con 0x7f5620103490 2024-05-10T11:37:08.744 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.744+0000 7f561dd9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5620105dc0 0x7f562019b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:08.744 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.744+0000 7f561e59c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5620103490 0x7f562019ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:08.744 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.744+0000 7f561dd9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5620105dc0 0x7f562019b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45034/0 (socket says 172.21.15.100:45034) 2024-05-10T11:37:08.744 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.744+0000 7f561e59c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5620103490 0x7f562019ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:60442/0 (socket says 172.21.15.100:60442) 2024-05-10T11:37:08.745 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.744+0000 7f561dd9b640 1 -- 172.21.15.100:0/2353965417 learned_addr learned my addr 172.21.15.100:0/2353965417 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:37:08.746 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.744+0000 7f561dd9b640 1 -- 172.21.15.100:0/2353965417 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5620103490 msgr2=0x7f562019ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:08.746 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.744+0000 7f561dd9b640 1 --2- 172.21.15.100:0/2353965417 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5620103490 0x7f562019ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:08.746 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.744+0000 7f561dd9b640 1 -- 172.21.15.100:0/2353965417 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56201086f0 msgr2=0x7f56201a22c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-10T11:37:08.746 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.744+0000 7f561dd9b640 1 --2- 172.21.15.100:0/2353965417 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56201086f0 0x7f56201a22c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:08.746 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.744+0000 7f561dd9b640 1 -- 172.21.15.100:0/2353965417 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56201a2960 con 0x7f5620105dc0 2024-05-10T11:37:08.746 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.744+0000 7f561dd9b640 1 --2- 172.21.15.100:0/2353965417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5620105dc0 0x7f562019b290 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f5614031460 tx=0x7f5614003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:37:08.746 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.745+0000 7f560f7fe640 1 -- 172.21.15.100:0/2353965417 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561403f070 con 0x7f5620105dc0 2024-05-10T11:37:08.746 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.745+0000 7f560f7fe640 1 -- 172.21.15.100:0/2353965417 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5614031ba0 con 0x7f5620105dc0 2024-05-10T11:37:08.746 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.745+0000 7f560f7fe640 1 -- 172.21.15.100:0/2353965417 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5614035030 con 0x7f5620105dc0 2024-05-10T11:37:08.746 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.745+0000 7f5624d9d640 1 -- 172.21.15.100:0/2353965417 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56201a2bf0 con 0x7f5620105dc0 2024-05-10T11:37:08.746 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.745+0000 7f5624d9d640 1 -- 172.21.15.100:0/2353965417 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56201a3090 con 0x7f5620105dc0 2024-05-10T11:37:08.748 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.748+0000 7f5627b94740 1 Processor -- start 2024-05-10T11:37:08.750 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.749+0000 7f5627b94740 1 -- start start 2024-05-10T11:37:08.750 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.749+0000 7f5627b94740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556312de9db0 0x556312dea1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:08.750 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.749+0000 7f5627b94740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556312dea7a0 0x556312def2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:08.750 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.749+0000 7f5627b94740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556312def910 0x556312df1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:08.751 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.749+0000 7f5627b94740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x556312d4e5d0 con 0x556312dea7a0 2024-05-10T11:37:08.751 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.749+0000 7f5627b94740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556312d07890 con 0x556312def910 2024-05-10T11:37:08.751 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.749+0000 7f5627b94740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556312d4a590 con 0x556312de9db0 2024-05-10T11:37:08.751 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.749+0000 7f561e59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556312dea7a0 0x556312def2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:08.751 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.749+0000 7f561ed9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556312de9db0 0x556312dea1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:08.751 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.749+0000 7f561e59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556312dea7a0 0x556312def2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45050/0 (socket says 172.21.15.100:45050) 2024-05-10T11:37:08.752 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.749+0000 7f561e59c640 1 -- 172.21.15.100:0/3233584282 learned_addr learned my addr 172.21.15.100:0/3233584282 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:37:08.752 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.750+0000 7f561e59c640 1 -- 172.21.15.100:0/3233584282 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556312de9db0 msgr2=0x556312dea1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:37:08.752 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.750+0000 7f561e59c640 1 --2- 172.21.15.100:0/3233584282 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556312de9db0 0x556312dea1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:08.752 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.750+0000 7f561e59c640 1 -- 172.21.15.100:0/3233584282 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556312def910 msgr2=0x556312df1da0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T11:37:08.752 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.750+0000 7f561e59c640 1 --2- 172.21.15.100:0/3233584282 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556312def910 0x556312df1da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:08.752 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.750+0000 7f561e59c640 1 -- 172.21.15.100:0/3233584282 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556312a60210 con 0x556312dea7a0 2024-05-10T11:37:08.757 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.750+0000 7f561e59c640 1 --2- 172.21.15.100:0/3233584282 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556312dea7a0 0x556312def2a0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f5610009ca0 tx=0x7f5610011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b64d032e1ae63d7b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:37:08.757 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.750+0000 7f560cff9640 1 -- 172.21.15.100:0/3233584282 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5610005220 con 0x556312dea7a0 2024-05-10T11:37:08.757 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.751+0000 7f560cff9640 1 -- 172.21.15.100:0/3233584282 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5610004c30 con 0x556312dea7a0 2024-05-10T11:37:08.757 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.751+0000 7f560cff9640 1 -- 172.21.15.100:0/3233584282 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5610019650 con 0x556312dea7a0 2024-05-10T11:37:08.757 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.751+0000 7f560f7fe640 1 -- 172.21.15.100:0/2353965417 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f5614042070 con 0x7f5620105dc0 2024-05-10T11:37:08.757 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.751+0000 7f5627b94740 1 -- 172.21.15.100:0/3233584282 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556312dea7a0 msgr2=0x556312def2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:08.757 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.751+0000 7f5627b94740 1 --2- 172.21.15.100:0/3233584282 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556312dea7a0 0x556312def2a0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f5610009ca0 tx=0x7f5610011810 comp rx=0 tx=0).stop 2024-05-10T11:37:08.757 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.752+0000 7f5627b94740 1 -- 172.21.15.100:0/3233584282 shutdown_connections 2024-05-10T11:37:08.757 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.752+0000 7f5627b94740 1 --2- 172.21.15.100:0/3233584282 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556312def910 0x556312df1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:08.758 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.752+0000 7f5627b94740 1 --2- 172.21.15.100:0/3233584282 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556312dea7a0 0x556312def2a0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:08.758 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.752+0000 7f5627b94740 1 --2- 172.21.15.100:0/3233584282 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556312de9db0 0x556312dea1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:08.758 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.752+0000 7f5627b94740 1 -- 172.21.15.100:0/3233584282 >> 172.21.15.100:0/3233584282 conn(0x556312de5420 msgr2=0x556312de5820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:37:08.758 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.752+0000 7f5627b94740 1 -- 172.21.15.100:0/3233584282 shutdown_connections 2024-05-10T11:37:08.758 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.752+0000 7f560f7fe640 1 --2- 172.21.15.100:0/2353965417 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f56080b27b0 0x7f56080b4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:08.758 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.752+0000 7f5627b94740 1 -- 172.21.15.100:0/3233584282 wait complete. 2024-05-10T11:37:08.758 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.752+0000 7f560f7fe640 1 -- 172.21.15.100:0/2353965417 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f5614102bb0 con 0x7f5620105dc0 2024-05-10T11:37:08.758 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.752+0000 7f561e59c640 1 --2- 172.21.15.100:0/2353965417 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f56080b27b0 0x7f56080b4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:08.758 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.752+0000 7f5627b94740 1 -- start start 2024-05-10T11:37:08.758 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.753+0000 7f5627b94740 1 Processor -- start 2024-05-10T11:37:08.758 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.753+0000 7f561e59c640 1 --2- 172.21.15.100:0/2353965417 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f56080b27b0 0x7f56080b4c70 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f561001b040 tx=0x7f56100096a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:37:08.758 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.753+0000 7f5627b94740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556312de9db0 0x556312dfc850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:08.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.753+0000 7f5627b94740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556312dea7a0 0x556312dfcd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:08.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.753+0000 7f5627b94740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556312def910 0x556312e03dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:08.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.753+0000 7f561e59c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556312de9db0 0x556312dfc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:08.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.753+0000 7f561e59c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556312de9db0 0x556312dfc850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:60458/0 (socket says 172.21.15.100:60458) 2024-05-10T11:37:08.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.753+0000 7f561dd9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556312dea7a0 0x556312dfcd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:08.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.753+0000 7f561e59c640 1 -- 172.21.15.100:0/3724154412 learned_addr learned my addr 172.21.15.100:0/3724154412 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:37:08.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.753+0000 7f5627b94740 1 -- 172.21.15.100:0/3724154412 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x556312df28c0 con 0x556312dea7a0 2024-05-10T11:37:08.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.753+0000 7f5627b94740 1 -- 172.21.15.100:0/3724154412 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556312d0b370 con 0x556312def910 2024-05-10T11:37:08.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.753+0000 7f5627b94740 1 -- 172.21.15.100:0/3724154412 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556312df2a40 con 0x556312de9db0 2024-05-10T11:37:08.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.754+0000 7f5627b94740 10 client.0 _ll_register_callbacks cb 0x556312a5fa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:37:08.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.754+0000 7f561ed9d640 1 --2- 172.21.15.100:0/3724154412 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556312def910 0x556312e03dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:08.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.754+0000 7f561e59c640 1 -- 172.21.15.100:0/3724154412 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556312def910 msgr2=0x556312e03dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:37:08.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.754+0000 7f561e59c640 1 --2- 172.21.15.100:0/3724154412 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556312def910 0x556312e03dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:08.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.754+0000 7f561e59c640 1 -- 172.21.15.100:0/3724154412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556312dea7a0 msgr2=0x556312dfcd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:37:08.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.754+0000 7f561e59c640 1 --2- 172.21.15.100:0/3724154412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556312dea7a0 0x556312dfcd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:08.760 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.754+0000 7f561e59c640 1 -- 172.21.15.100:0/3724154412 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556312b2c620 con 0x556312de9db0 2024-05-10T11:37:08.760 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.754+0000 7f561dd9b640 1 --2- 172.21.15.100:0/3724154412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556312dea7a0 0x556312dfcd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:37:08.760 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.754+0000 7f561e59c640 1 --2- 172.21.15.100:0/3724154412 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556312de9db0 0x556312dfc850 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5610009a30 tx=0x7f5610009830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7901dfc1f00cb872 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:37:08.760 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.754+0000 7f5605ffb640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:37:08.760 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.754+0000 7f5605ffb640 1 -- 172.21.15.100:0/3724154412 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5610019430 con 0x556312de9db0 2024-05-10T11:37:08.760 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.754+0000 7f5605ffb640 1 -- 172.21.15.100:0/3724154412 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f56100195d0 con 0x556312de9db0 2024-05-10T11:37:08.760 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.754+0000 7f5627b94740 1 -- 172.21.15.100:0/3724154412 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5610004c30 con 0x556312de9db0 2024-05-10T11:37:08.760 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.755+0000 7f5605ffb640 1 -- 172.21.15.100:0/3724154412 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f56100197d0 con 0x556312de9db0 2024-05-10T11:37:08.760 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.755+0000 7f5627b94740 10 client.28525 fetch_fsmap learned FSMap version 41 2024-05-10T11:37:08.760 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.755+0000 7f5627b94740 1 -- 172.21.15.100:0/3724154412 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x556312e06e20 con 0x556312de9db0 2024-05-10T11:37:08.760 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.755+0000 7f5605ffb640 1 -- 172.21.15.100:0/3724154412 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5610004c30 con 0x556312de9db0 2024-05-10T11:37:08.760 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.755+0000 7f5605ffb640 1 -- 172.21.15.100:0/3724154412 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f561002f920 con 0x556312de9db0 2024-05-10T11:37:08.760 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.755+0000 7f5627b94740 10 client.28525 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:37:08.760 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.755+0000 7f5627b94740 10 client.28525 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:37:08.761 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.755+0000 7f5627b94740 20 client.28525 populate_metadata read hostname 'smithi100' 2024-05-10T11:37:08.761 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.755+0000 7f5627b94740 1 --2- 172.21.15.100:0/3724154412 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556312e08300 0x556312e0a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:08.761 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.755+0000 7f5627b94740 4 client.28525 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:37:08.761 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.755+0000 7f5627b94740 1 -- 172.21.15.100:0/3724154412 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556312d5c1e0 con 0x556312e08300 2024-05-10T11:37:08.761 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.756+0000 7f561dd9b640 1 --2- 172.21.15.100:0/3724154412 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556312e08300 0x556312e0a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:08.761 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.756+0000 7f561dd9b640 1 --2- 172.21.15.100:0/3724154412 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556312e08300 0x556312e0a750 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1ace08492e01b5d server_cookie=74e2695a255d4227 in_seq=0 out_seq=0 2024-05-10T11:37:08.761 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.756+0000 7f5605ffb640 10 client.28525 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:37:08.762 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.758+0000 7f5605ffb640 1 -- 172.21.15.100:0/3724154412 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556312d5c1e0 con 0x556312e08300 2024-05-10T11:37:08.762 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.758+0000 7f5605ffb640 10 client.28525 handle_command_reply: tid=0 2024-05-10T11:37:08.762 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.758+0000 7f5627b94740 1 client.28525 shutdown 2024-05-10T11:37:08.763 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.758+0000 7f5627b94740 1 -- 172.21.15.100:0/3724154412 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556312de9db0 msgr2=0x556312dfc850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:08.763 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.758+0000 7f5627b94740 1 --2- 172.21.15.100:0/3724154412 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556312de9db0 0x556312dfc850 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5610009a30 tx=0x7f5610009830 comp rx=0 tx=0).stop 2024-05-10T11:37:08.763 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.758+0000 7f5627b94740 1 -- 172.21.15.100:0/3724154412 shutdown_connections 2024-05-10T11:37:08.763 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.758+0000 7f5627b94740 1 --2- 172.21.15.100:0/3724154412 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556312e08300 0x556312e0a750 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:08.763 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.758+0000 7f5627b94740 1 --2- 172.21.15.100:0/3724154412 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556312def910 0x556312e03dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:08.763 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.758+0000 7f5627b94740 1 --2- 172.21.15.100:0/3724154412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556312dea7a0 0x556312dfcd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:08.763 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.758+0000 7f5627b94740 1 --2- 172.21.15.100:0/3724154412 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556312de9db0 0x556312dfc850 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:08.763 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.758+0000 7f5605ffb640 0 client.28525 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:37:08.763 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.758+0000 7f5627b94740 1 -- 172.21.15.100:0/3724154412 >> 172.21.15.100:0/3724154412 conn(0x556312de5420 msgr2=0x556312de4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:37:08.763 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.758+0000 7f5627b94740 1 -- 172.21.15.100:0/3724154412 shutdown_connections 2024-05-10T11:37:08.763 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.758+0000 7f5627b94740 1 -- 172.21.15.100:0/3724154412 wait complete. 2024-05-10T11:37:08.763 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.758+0000 7f5627b94740 20 client.28525 trim_cache size 0 max 16384 2024-05-10T11:37:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.783+0000 7f5627b94740 1 Processor -- start 2024-05-10T11:37:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.783+0000 7f5627b94740 1 -- start start 2024-05-10T11:37:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.784+0000 7f5627b94740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556312de9db0 0x556312deba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.784+0000 7f5627b94740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556312dea7a0 0x556312debfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.784+0000 7f5627b94740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556312def910 0x556312dfc520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.784+0000 7f5627b94740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f56201aba90 con 0x556312de9db0 2024-05-10T11:37:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.784+0000 7f5627b94740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f56201a9360 con 0x556312def910 2024-05-10T11:37:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.784+0000 7f5627b94740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f56201a71b0 con 0x556312dea7a0 2024-05-10T11:37:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.784+0000 7f561e59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556312de9db0 0x556312deba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.784+0000 7f561e59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556312de9db0 0x556312deba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45054/0 (socket says 172.21.15.100:45054) 2024-05-10T11:37:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.784+0000 7f561e59c640 1 -- 172.21.15.100:0/1872828808 learned_addr learned my addr 172.21.15.100:0/1872828808 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:37:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.784+0000 7f561dd9b640 1 --2- 172.21.15.100:0/1872828808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556312dea7a0 0x556312debfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.784+0000 7f561ed9d640 1 --2- 172.21.15.100:0/1872828808 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556312def910 0x556312dfc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.784+0000 7f561e59c640 1 -- 172.21.15.100:0/1872828808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556312dea7a0 msgr2=0x556312debfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:37:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.784+0000 7f561e59c640 1 --2- 172.21.15.100:0/1872828808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556312dea7a0 0x556312debfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.784+0000 7f561e59c640 1 -- 172.21.15.100:0/1872828808 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556312def910 msgr2=0x556312dfc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:37:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.784+0000 7f561e59c640 1 --2- 172.21.15.100:0/1872828808 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556312def910 0x556312dfc520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.784+0000 7f561e59c640 1 -- 172.21.15.100:0/1872828808 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556312deab80 con 0x556312de9db0 2024-05-10T11:37:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.784+0000 7f561dd9b640 1 --2- 172.21.15.100:0/1872828808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556312dea7a0 0x556312debfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:37:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.784+0000 7f561e59c640 1 --2- 172.21.15.100:0/1872828808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556312de9db0 0x556312deba80 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f561002fde0 tx=0x7f5610004d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0a63696fdfdbea0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:37:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.784+0000 7f5607fff640 1 -- 172.21.15.100:0/1872828808 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5610047070 con 0x556312de9db0 2024-05-10T11:37:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.785+0000 7f5607fff640 1 -- 172.21.15.100:0/1872828808 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5610034070 con 0x556312de9db0 2024-05-10T11:37:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.785+0000 7f5607fff640 1 -- 172.21.15.100:0/1872828808 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561002f650 con 0x556312de9db0 2024-05-10T11:37:08.790 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.785+0000 7f5627b94740 1 -- 172.21.15.100:0/1872828808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556312de9db0 msgr2=0x556312deba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:08.790 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.785+0000 7f5627b94740 1 --2- 172.21.15.100:0/1872828808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556312de9db0 0x556312deba80 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f561002fde0 tx=0x7f5610004d40 comp rx=0 tx=0).stop 2024-05-10T11:37:08.790 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.785+0000 7f5627b94740 1 -- 172.21.15.100:0/1872828808 shutdown_connections 2024-05-10T11:37:08.790 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.785+0000 7f5627b94740 1 --2- 172.21.15.100:0/1872828808 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556312def910 0x556312dfc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:08.790 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.785+0000 7f5627b94740 1 --2- 172.21.15.100:0/1872828808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556312dea7a0 0x556312debfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:08.790 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.785+0000 7f5627b94740 1 --2- 172.21.15.100:0/1872828808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556312de9db0 0x556312deba80 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.785+0000 7f5627b94740 1 -- 172.21.15.100:0/1872828808 >> 172.21.15.100:0/1872828808 conn(0x556312de5420 msgr2=0x556312e0a230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:37:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.785+0000 7f5627b94740 1 -- 172.21.15.100:0/1872828808 shutdown_connections 2024-05-10T11:37:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.786+0000 7f5627b94740 1 -- 172.21.15.100:0/1872828808 wait complete. 2024-05-10T11:37:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.786+0000 7f5627b94740 1 -- start start 2024-05-10T11:37:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.786+0000 7f5627b94740 1 Processor -- start 2024-05-10T11:37:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.786+0000 7f5627b94740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556312de9db0 0x556312e00350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.786+0000 7f5627b94740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556312dea7a0 0x556312e00890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.786+0000 7f5627b94740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556312def910 0x556312e70c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.786+0000 7f5627b94740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f56201a7820 con 0x556312de9db0 2024-05-10T11:37:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.786+0000 7f5627b94740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f56201a8c90 con 0x556312dea7a0 2024-05-10T11:37:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.786+0000 7f5627b94740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f56201a7ef0 con 0x556312def910 2024-05-10T11:37:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.786+0000 7f561e59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556312de9db0 0x556312e00350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.786+0000 7f561e59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556312de9db0 0x556312e00350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45068/0 (socket says 172.21.15.100:45068) 2024-05-10T11:37:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.786+0000 7f561e59c640 1 -- 172.21.15.100:0/802663638 learned_addr learned my addr 172.21.15.100:0/802663638 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:37:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.786+0000 7f561dd9b640 1 --2- 172.21.15.100:0/802663638 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556312dea7a0 0x556312e00890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.786+0000 7f5627b94740 10 client.0 _ll_register_callbacks cb 0x7f56180098e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:37:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.786+0000 7f561ed9d640 1 --2- 172.21.15.100:0/802663638 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556312def910 0x556312e70c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.787+0000 7f561e59c640 1 -- 172.21.15.100:0/802663638 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556312def910 msgr2=0x556312e70c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:37:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.787+0000 7f561e59c640 1 --2- 172.21.15.100:0/802663638 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556312def910 0x556312e70c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.787+0000 7f561e59c640 1 -- 172.21.15.100:0/802663638 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556312dea7a0 msgr2=0x556312e00890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:37:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.787+0000 7f561e59c640 1 --2- 172.21.15.100:0/802663638 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556312dea7a0 0x556312e00890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.787+0000 7f561e59c640 1 -- 172.21.15.100:0/802663638 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556312e00f30 con 0x556312de9db0 2024-05-10T11:37:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.787+0000 7f561e59c640 1 --2- 172.21.15.100:0/802663638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556312de9db0 0x556312e00350 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f5610005fd0 tx=0x7f5610019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab299d65c5952c99 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:37:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.787+0000 7f5605ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:37:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.787+0000 7f5605ffb640 1 -- 172.21.15.100:0/802663638 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5610047040 con 0x556312de9db0 2024-05-10T11:37:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.787+0000 7f5605ffb640 1 -- 172.21.15.100:0/802663638 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f561002f3f0 con 0x556312de9db0 2024-05-10T11:37:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.787+0000 7f5627b94740 1 -- 172.21.15.100:0/802663638 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5610034070 con 0x556312de9db0 2024-05-10T11:37:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.787+0000 7f5605ffb640 1 -- 172.21.15.100:0/802663638 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f5610059050 con 0x556312de9db0 2024-05-10T11:37:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.787+0000 7f5627b94740 10 client.19131 fetch_fsmap learned FSMap version 41 2024-05-10T11:37:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.787+0000 7f5627b94740 1 -- 172.21.15.100:0/802663638 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x556312e72280 con 0x556312de9db0 2024-05-10T11:37:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.788+0000 7f5605ffb640 1 -- 172.21.15.100:0/802663638 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5610034070 con 0x556312de9db0 2024-05-10T11:37:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.788+0000 7f5605ffb640 1 -- 172.21.15.100:0/802663638 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f561004a050 con 0x556312de9db0 2024-05-10T11:37:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.788+0000 7f5627b94740 10 client.19131 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:37:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.788+0000 7f5627b94740 10 client.19131 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:37:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.788+0000 7f5627b94740 20 client.19131 populate_metadata read hostname 'smithi100' 2024-05-10T11:37:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.788+0000 7f5627b94740 1 --2- 172.21.15.100:0/802663638 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556312e72b70 0x556312e74f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.788+0000 7f5627b94740 4 client.19131 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:37:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.788+0000 7f5627b94740 1 -- 172.21.15.100:0/802663638 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556312d56050 con 0x556312e72b70 2024-05-10T11:37:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.788+0000 7f561dd9b640 1 --2- 172.21.15.100:0/802663638 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556312e72b70 0x556312e74f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.788+0000 7f561dd9b640 1 --2- 172.21.15.100:0/802663638 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556312e72b70 0x556312e74f90 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d83057293abd95b server_cookie=ef44cea9dfa882b5 in_seq=0 out_seq=0 2024-05-10T11:37:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.788+0000 7f5605ffb640 10 client.19131 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:37:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.895+0000 7f5605ffb640 1 -- 172.21.15.100:0/802663638 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556312d56050 con 0x556312e72b70 2024-05-10T11:37:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.895+0000 7f5605ffb640 10 client.19131 handle_command_reply: tid=0 2024-05-10T11:37:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.895+0000 7f5627b94740 1 client.19131 shutdown 2024-05-10T11:37:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.895+0000 7f5627b94740 1 -- 172.21.15.100:0/802663638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556312de9db0 msgr2=0x556312e00350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.895+0000 7f5627b94740 1 --2- 172.21.15.100:0/802663638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556312de9db0 0x556312e00350 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f5610005fd0 tx=0x7f5610019710 comp rx=0 tx=0).stop 2024-05-10T11:37:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.896+0000 7f5627b94740 1 -- 172.21.15.100:0/802663638 shutdown_connections 2024-05-10T11:37:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.896+0000 7f5627b94740 1 --2- 172.21.15.100:0/802663638 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556312e72b70 0x556312e74f90 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:08.897 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.896+0000 7f5605ffb640 0 client.19131 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:37:08.897 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.896+0000 7f5627b94740 1 --2- 172.21.15.100:0/802663638 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556312def910 0x556312e70c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:08.897 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.896+0000 7f5627b94740 1 --2- 172.21.15.100:0/802663638 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556312dea7a0 0x556312e00890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:08.897 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.896+0000 7f5627b94740 1 --2- 172.21.15.100:0/802663638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556312de9db0 0x556312e00350 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:08.897 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.896+0000 7f5627b94740 1 -- 172.21.15.100:0/802663638 >> 172.21.15.100:0/802663638 conn(0x556312de5420 msgr2=0x556312deba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:37:08.897 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.896+0000 7f5627b94740 1 -- 172.21.15.100:0/802663638 shutdown_connections 2024-05-10T11:37:08.897 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.896+0000 7f5627b94740 1 -- 172.21.15.100:0/802663638 wait complete. 2024-05-10T11:37:08.897 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.896+0000 7f5627b94740 20 client.19131 trim_cache size 0 max 16384 2024-05-10T11:37:08.897 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:37:08.897 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:37:08.898 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:37:08.898 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:37:08.898 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.896+0000 7f5624d9d640 1 -- 172.21.15.100:0/2353965417 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f56080b27b0 msgr2=0x7f56080b4c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:08.899 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.896+0000 7f5624d9d640 1 --2- 172.21.15.100:0/2353965417 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f56080b27b0 0x7f56080b4c70 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f561001b040 tx=0x7f56100096a0 comp rx=0 tx=0).stop 2024-05-10T11:37:08.899 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.896+0000 7f5624d9d640 1 -- 172.21.15.100:0/2353965417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5620105dc0 msgr2=0x7f562019b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:08.899 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.896+0000 7f5624d9d640 1 --2- 172.21.15.100:0/2353965417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5620105dc0 0x7f562019b290 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f5614031460 tx=0x7f5614003c10 comp rx=0 tx=0).stop 2024-05-10T11:37:08.899 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.897+0000 7f5624d9d640 1 -- 172.21.15.100:0/2353965417 shutdown_connections 2024-05-10T11:37:08.899 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.897+0000 7f5624d9d640 1 --2- 172.21.15.100:0/2353965417 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f56080b27b0 0x7f56080b4c70 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:08.899 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.897+0000 7f5624d9d640 1 --2- 172.21.15.100:0/2353965417 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56201086f0 0x7f56201a22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:08.899 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.897+0000 7f5624d9d640 1 --2- 172.21.15.100:0/2353965417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5620105dc0 0x7f562019b290 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:08.899 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.897+0000 7f5624d9d640 1 --2- 172.21.15.100:0/2353965417 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5620103490 0x7f562019ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:08.899 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.897+0000 7f5624d9d640 1 -- 172.21.15.100:0/2353965417 >> 172.21.15.100:0/2353965417 conn(0x7f56200fce80 msgr2=0x7f5620103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:37:08.899 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.897+0000 7f5624d9d640 1 -- 172.21.15.100:0/2353965417 shutdown_connections 2024-05-10T11:37:08.899 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:08.897+0000 7f5624d9d640 1 -- 172.21.15.100:0/2353965417 wait complete. 2024-05-10T11:37:08.932 INFO:tasks.workunit.client.0.smithi100.stdout: 10 50798 9.04 MB/sec execute 182 sec latency 131.076 ms 2024-05-10T11:37:09.932 INFO:tasks.workunit.client.0.smithi100.stdout: 10 50978 9.04 MB/sec execute 183 sec latency 82.765 ms 2024-05-10T11:37:10.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:37:09 smithi178 ceph-mon[33359]: pgmap v984: 129 pgs: 129 active+clean; 472 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 4.5 MiB/s wr, 315 op/s 2024-05-10T11:37:10.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:37:09 smithi188 ceph-mon[34075]: pgmap v984: 129 pgs: 129 active+clean; 472 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 4.5 MiB/s wr, 315 op/s 2024-05-10T11:37:10.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:37:09 smithi100 ceph-mon[30296]: pgmap v984: 129 pgs: 129 active+clean; 472 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 4.5 MiB/s wr, 315 op/s 2024-05-10T11:37:10.932 INFO:tasks.workunit.client.0.smithi100.stdout: 10 51135 9.00 MB/sec execute 184 sec latency 89.945 ms 2024-05-10T11:37:11.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:37:10 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:37:11.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:37:10 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:37:11.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:37:10 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:37:11.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:37:10 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:37:11.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:37:10 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:37:11.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:37:10 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:37:11.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:37:10 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:37:11.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:37:10 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:37:11.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:37:10 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:37:11.883 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:37:11 smithi100 ceph-mon[30296]: pgmap v985: 129 pgs: 129 active+clean; 470 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 974 KiB/s rd, 5.1 MiB/s wr, 370 op/s 2024-05-10T11:37:11.932 INFO:tasks.workunit.client.0.smithi100.stdout: 10 51303 8.98 MB/sec execute 185 sec latency 65.967 ms 2024-05-10T11:37:12.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:37:11 smithi178 ceph-mon[33359]: pgmap v985: 129 pgs: 129 active+clean; 470 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 974 KiB/s rd, 5.1 MiB/s wr, 370 op/s 2024-05-10T11:37:12.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:37:11 smithi188 ceph-mon[34075]: pgmap v985: 129 pgs: 129 active+clean; 470 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 974 KiB/s rd, 5.1 MiB/s wr, 370 op/s 2024-05-10T11:37:12.932 INFO:tasks.workunit.client.0.smithi100.stdout: 10 51465 8.96 MB/sec execute 186 sec latency 82.069 ms 2024-05-10T11:37:13.932 INFO:tasks.workunit.client.0.smithi100.stdout: 10 51630 8.96 MB/sec execute 187 sec latency 103.802 ms 2024-05-10T11:37:14.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:37:13 smithi178 ceph-mon[33359]: pgmap v986: 129 pgs: 129 active+clean; 470 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 4.0 MiB/s wr, 340 op/s 2024-05-10T11:37:14.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:37:13 smithi188 ceph-mon[34075]: pgmap v986: 129 pgs: 129 active+clean; 470 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 4.0 MiB/s wr, 340 op/s 2024-05-10T11:37:14.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:37:13 smithi100 ceph-mon[30296]: pgmap v986: 129 pgs: 129 active+clean; 470 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 4.0 MiB/s wr, 340 op/s 2024-05-10T11:37:14.932 INFO:tasks.workunit.client.0.smithi100.stdout: 10 51811 8.99 MB/sec execute 188 sec latency 73.703 ms 2024-05-10T11:37:15.932 INFO:tasks.workunit.client.0.smithi100.stdout: 10 52080 9.07 MB/sec execute 189 sec latency 73.786 ms 2024-05-10T11:37:16.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:37:15 smithi178 ceph-mon[33359]: pgmap v987: 129 pgs: 129 active+clean; 471 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 3.8 MiB/s wr, 312 op/s 2024-05-10T11:37:16.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:37:15 smithi188 ceph-mon[34075]: pgmap v987: 129 pgs: 129 active+clean; 471 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 3.8 MiB/s wr, 312 op/s 2024-05-10T11:37:16.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:37:15 smithi100 ceph-mon[30296]: pgmap v987: 129 pgs: 129 active+clean; 471 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 3.8 MiB/s wr, 312 op/s 2024-05-10T11:37:16.933 INFO:tasks.workunit.client.0.smithi100.stdout: 10 52324 9.09 MB/sec execute 190 sec latency 63.620 ms 2024-05-10T11:37:17.933 INFO:tasks.workunit.client.0.smithi100.stdout: 10 52421 9.08 MB/sec execute 191 sec latency 126.041 ms 2024-05-10T11:37:18.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:37:17 smithi178 ceph-mon[33359]: pgmap v988: 129 pgs: 129 active+clean; 476 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 875 KiB/s rd, 4.4 MiB/s wr, 305 op/s 2024-05-10T11:37:18.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:37:17 smithi188 ceph-mon[34075]: pgmap v988: 129 pgs: 129 active+clean; 476 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 875 KiB/s rd, 4.4 MiB/s wr, 305 op/s 2024-05-10T11:37:18.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:37:17 smithi100 ceph-mon[30296]: pgmap v988: 129 pgs: 129 active+clean; 476 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 875 KiB/s rd, 4.4 MiB/s wr, 305 op/s 2024-05-10T11:37:18.933 INFO:tasks.workunit.client.0.smithi100.stdout: 10 52546 9.07 MB/sec execute 192 sec latency 86.349 ms 2024-05-10T11:37:19.933 INFO:tasks.workunit.client.0.smithi100.stdout: 10 52761 9.12 MB/sec execute 193 sec latency 63.161 ms 2024-05-10T11:37:20.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:37:19 smithi178 ceph-mon[33359]: pgmap v989: 129 pgs: 129 active+clean; 479 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 3.9 MiB/s wr, 279 op/s 2024-05-10T11:37:20.030 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:37:19 smithi188 ceph-mon[34075]: pgmap v989: 129 pgs: 129 active+clean; 479 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 3.9 MiB/s wr, 279 op/s 2024-05-10T11:37:20.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:37:19 smithi100 ceph-mon[30296]: pgmap v989: 129 pgs: 129 active+clean; 479 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 3.9 MiB/s wr, 279 op/s 2024-05-10T11:37:20.933 INFO:tasks.workunit.client.0.smithi100.stdout: 10 52915 9.11 MB/sec execute 194 sec latency 57.816 ms 2024-05-10T11:37:21.933 INFO:tasks.workunit.client.0.smithi100.stdout: 10 53048 9.09 MB/sec execute 195 sec latency 64.730 ms 2024-05-10T11:37:22.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:37:21 smithi178 ceph-mon[33359]: pgmap v990: 129 pgs: 129 active+clean; 482 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 4.5 MiB/s wr, 298 op/s 2024-05-10T11:37:22.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:37:21 smithi188 ceph-mon[34075]: pgmap v990: 129 pgs: 129 active+clean; 482 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 4.5 MiB/s wr, 298 op/s 2024-05-10T11:37:22.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.036+0000 7f88aace1640 1 -- 172.21.15.100:0/1072273980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88a4111230 msgr2=0x7f88a4113620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:22.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.036+0000 7f88aace1640 1 --2- 172.21.15.100:0/1072273980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88a4111230 0x7f88a4113620 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f889c009be0 tx=0x7f889c02a6c0 comp rx=0 tx=0).stop 2024-05-10T11:37:22.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.036+0000 7f88aace1640 1 -- 172.21.15.100:0/1072273980 shutdown_connections 2024-05-10T11:37:22.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.036+0000 7f88aace1640 1 --2- 172.21.15.100:0/1072273980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88a4111230 0x7f88a4113620 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:22.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.036+0000 7f88aace1640 1 --2- 172.21.15.100:0/1072273980 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f88a410a310 0x7f88a4110cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:22.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.036+0000 7f88aace1640 1 --2- 172.21.15.100:0/1072273980 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88a41099b0 0x7f88a4109dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:22.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.036+0000 7f88aace1640 1 -- 172.21.15.100:0/1072273980 >> 172.21.15.100:0/1072273980 conn(0x7f88a406fa20 msgr2=0x7f88a4071e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:37:22.037 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.036+0000 7f88aace1640 1 -- 172.21.15.100:0/1072273980 shutdown_connections 2024-05-10T11:37:22.037 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.036+0000 7f88aace1640 1 -- 172.21.15.100:0/1072273980 wait complete. 2024-05-10T11:37:22.038 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.037+0000 7f88aace1640 1 Processor -- start 2024-05-10T11:37:22.038 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.037+0000 7f88aace1640 1 -- start start 2024-05-10T11:37:22.038 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.037+0000 7f88aace1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f88a41099b0 0x7f88a41a11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:22.038 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.037+0000 7f88aace1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88a410a310 0x7f88a41a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:22.038 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.037+0000 7f88aace1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88a4111230 0x7f88a41a8790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:22.038 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.037+0000 7f88aace1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f88a4116c00 con 0x7f88a410a310 2024-05-10T11:37:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.037+0000 7f88aace1640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f88a4116a80 con 0x7f88a41099b0 2024-05-10T11:37:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.037+0000 7f88aace1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f88a4116d80 con 0x7f88a4111230 2024-05-10T11:37:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.037+0000 7f88a8a7d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f88a41099b0 0x7f88a41a11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.037+0000 7f88a3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88a410a310 0x7f88a41a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.037+0000 7f88a3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88a410a310 0x7f88a41a1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34892/0 (socket says 172.21.15.100:34892) 2024-05-10T11:37:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.037+0000 7f88a3fff640 1 -- 172.21.15.100:0/3496189216 learned_addr learned my addr 172.21.15.100:0/3496189216 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:37:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.037+0000 7f88a8a7d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f88a41099b0 0x7f88a41a11e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:38996/0 (socket says 172.21.15.100:38996) 2024-05-10T11:37:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.037+0000 7f88a927e640 1 --2- 172.21.15.100:0/3496189216 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88a4111230 0x7f88a41a8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.038+0000 7f88a3fff640 1 -- 172.21.15.100:0/3496189216 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88a4111230 msgr2=0x7f88a41a8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:22.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:37:21 smithi100 ceph-mon[30296]: pgmap v990: 129 pgs: 129 active+clean; 482 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 4.5 MiB/s wr, 298 op/s 2024-05-10T11:37:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.038+0000 7f88a3fff640 1 --2- 172.21.15.100:0/3496189216 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88a4111230 0x7f88a41a8790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.038+0000 7f88a3fff640 1 -- 172.21.15.100:0/3496189216 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f88a41099b0 msgr2=0x7f88a41a11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.038+0000 7f88a3fff640 1 --2- 172.21.15.100:0/3496189216 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f88a41099b0 0x7f88a41a11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.038+0000 7f88a3fff640 1 -- 172.21.15.100:0/3496189216 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88a41a8e00 con 0x7f88a410a310 2024-05-10T11:37:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.038+0000 7f88a927e640 1 --2- 172.21.15.100:0/3496189216 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88a4111230 0x7f88a41a8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:37:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.038+0000 7f88a3fff640 1 --2- 172.21.15.100:0/3496189216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88a410a310 0x7f88a41a1720 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f889000d950 tx=0x7f889000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:37:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.038+0000 7f88a1ffb640 1 -- 172.21.15.100:0/3496189216 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8890017070 con 0x7f88a410a310 2024-05-10T11:37:22.041 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.038+0000 7f88a1ffb640 1 -- 172.21.15.100:0/3496189216 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8890009c30 con 0x7f88a410a310 2024-05-10T11:37:22.041 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.038+0000 7f88aace1640 1 -- 172.21.15.100:0/3496189216 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88a41a90f0 con 0x7f88a410a310 2024-05-10T11:37:22.041 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.038+0000 7f88a1ffb640 1 -- 172.21.15.100:0/3496189216 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f889000f620 con 0x7f88a410a310 2024-05-10T11:37:22.041 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.038+0000 7f88aace1640 1 -- 172.21.15.100:0/3496189216 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88a41a9690 con 0x7f88a410a310 2024-05-10T11:37:22.042 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.042+0000 7f88ada02740 1 Processor -- start 2024-05-10T11:37:22.042 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.042+0000 7f88ada02740 1 -- start start 2024-05-10T11:37:22.043 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.042+0000 7f88ada02740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db917f1db0 0x55db917f21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:22.043 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.042+0000 7f88ada02740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db917f27a0 0x55db917f72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.042+0000 7f88ada02740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db917f7910 0x55db917f9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.042+0000 7f88ada02740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55db917565d0 con 0x55db917f7910 2024-05-10T11:37:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.042+0000 7f88ada02740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55db9170f890 con 0x55db917f1db0 2024-05-10T11:37:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.042+0000 7f88ada02740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55db91752590 con 0x55db917f27a0 2024-05-10T11:37:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.042+0000 7f88a1ffb640 1 -- 172.21.15.100:0/3496189216 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f8890010460 con 0x7f88a410a310 2024-05-10T11:37:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.042+0000 7f88a927e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db917f1db0 0x55db917f21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.042+0000 7f88a8a7d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db917f27a0 0x55db917f72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.042+0000 7f88a8a7d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db917f27a0 0x55db917f72a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:45308/0 (socket says 172.21.15.100:45308) 2024-05-10T11:37:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.042+0000 7f88a927e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db917f1db0 0x55db917f21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:38998/0 (socket says 172.21.15.100:38998) 2024-05-10T11:37:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.042+0000 7f88a8a7d640 1 -- 172.21.15.100:0/2196883814 learned_addr learned my addr 172.21.15.100:0/2196883814 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:37:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.042+0000 7f88a3fff640 1 --2- 172.21.15.100:0/2196883814 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db917f7910 0x55db917f9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.043+0000 7f88a927e640 1 -- 172.21.15.100:0/2196883814 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db917f27a0 msgr2=0x55db917f72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:37:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.043+0000 7f88a927e640 1 --2- 172.21.15.100:0/2196883814 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db917f27a0 0x55db917f72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.043+0000 7f88a927e640 1 -- 172.21.15.100:0/2196883814 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db917f7910 msgr2=0x55db917f9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:37:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.043+0000 7f88a927e640 1 --2- 172.21.15.100:0/2196883814 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db917f7910 0x55db917f9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.043+0000 7f88a927e640 1 -- 172.21.15.100:0/2196883814 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db91468210 con 0x55db917f1db0 2024-05-10T11:37:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.043+0000 7f88a927e640 1 --2- 172.21.15.100:0/2196883814 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db917f1db0 0x55db917f21d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f889c02abe0 tx=0x7f889c0092f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=577fa33f1b6fcd8f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:37:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.043+0000 7f888effd640 1 -- 172.21.15.100:0/2196883814 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f889c040030 con 0x55db917f1db0 2024-05-10T11:37:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.043+0000 7f888effd640 1 -- 172.21.15.100:0/2196883814 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f889c037040 con 0x55db917f1db0 2024-05-10T11:37:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.043+0000 7f888effd640 1 -- 172.21.15.100:0/2196883814 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f889c036070 con 0x55db917f1db0 2024-05-10T11:37:22.050 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.044+0000 7f88a1ffb640 1 --2- 172.21.15.100:0/3496189216 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f88940b2920 0x7f88940b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:22.050 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.044+0000 7f88a8a7d640 1 --2- 172.21.15.100:0/3496189216 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f88940b2920 0x7f88940b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:22.050 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.044+0000 7f88a1ffb640 1 -- 172.21.15.100:0/3496189216 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f88900e2120 con 0x7f88a410a310 2024-05-10T11:37:22.050 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.044+0000 7f88ada02740 1 -- 172.21.15.100:0/2196883814 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db917f1db0 msgr2=0x55db917f21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:22.050 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.044+0000 7f88ada02740 1 --2- 172.21.15.100:0/2196883814 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db917f1db0 0x55db917f21d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f889c02abe0 tx=0x7f889c0092f0 comp rx=0 tx=0).stop 2024-05-10T11:37:22.050 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.044+0000 7f88ada02740 1 -- 172.21.15.100:0/2196883814 shutdown_connections 2024-05-10T11:37:22.051 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.044+0000 7f88ada02740 1 --2- 172.21.15.100:0/2196883814 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db917f7910 0x55db917f9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:22.051 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.044+0000 7f88ada02740 1 --2- 172.21.15.100:0/2196883814 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db917f27a0 0x55db917f72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:22.051 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.044+0000 7f88ada02740 1 --2- 172.21.15.100:0/2196883814 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db917f1db0 0x55db917f21d0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:22.051 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.044+0000 7f88ada02740 1 -- 172.21.15.100:0/2196883814 >> 172.21.15.100:0/2196883814 conn(0x55db917ed420 msgr2=0x55db917ed820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:37:22.051 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.044+0000 7f88a8a7d640 1 --2- 172.21.15.100:0/3496189216 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f88940b2920 0x7f88940b4de0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x55db917f2ba0 tx=0x7f8898009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:37:22.051 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.044+0000 7f88ada02740 1 -- 172.21.15.100:0/2196883814 shutdown_connections 2024-05-10T11:37:22.051 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.045+0000 7f88ada02740 1 -- 172.21.15.100:0/2196883814 wait complete. 2024-05-10T11:37:22.051 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.045+0000 7f88ada02740 1 -- start start 2024-05-10T11:37:22.051 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.045+0000 7f88ada02740 1 Processor -- start 2024-05-10T11:37:22.051 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.045+0000 7f88ada02740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db917f1db0 0x55db91800430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:22.051 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.045+0000 7f88ada02740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db917f27a0 0x55db91800970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:22.051 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.045+0000 7f88ada02740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db917f7910 0x55db918079b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:22.051 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.045+0000 7f88ada02740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55db917fec70 con 0x55db917f7910 2024-05-10T11:37:22.051 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.045+0000 7f88ada02740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55db91713370 con 0x55db917f27a0 2024-05-10T11:37:22.052 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.045+0000 7f88ada02740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55db917fedf0 con 0x55db917f1db0 2024-05-10T11:37:22.052 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.046+0000 7f88a8a7d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db917f1db0 0x55db91800430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:22.052 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.046+0000 7f88a8a7d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db917f1db0 0x55db91800430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:45314/0 (socket says 172.21.15.100:45314) 2024-05-10T11:37:22.052 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.046+0000 7f88a8a7d640 1 -- 172.21.15.100:0/4084488346 learned_addr learned my addr 172.21.15.100:0/4084488346 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:37:22.052 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.046+0000 7f88ada02740 10 client.0 _ll_register_callbacks cb 0x55db91467a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:37:22.052 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.046+0000 7f88a3fff640 1 --2- 172.21.15.100:0/4084488346 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db917f27a0 0x55db91800970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:22.052 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.046+0000 7f88a927e640 1 --2- 172.21.15.100:0/4084488346 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db917f7910 0x55db918079b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:22.052 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.046+0000 7f88a8a7d640 1 -- 172.21.15.100:0/4084488346 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db917f27a0 msgr2=0x55db91800970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:37:22.052 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.046+0000 7f88a8a7d640 1 --2- 172.21.15.100:0/4084488346 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db917f27a0 0x55db91800970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:22.052 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.046+0000 7f88a8a7d640 1 -- 172.21.15.100:0/4084488346 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db917f7910 msgr2=0x55db918079b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:37:22.052 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.046+0000 7f88a8a7d640 1 --2- 172.21.15.100:0/4084488346 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db917f7910 0x55db918079b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:22.052 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.046+0000 7f88a8a7d640 1 -- 172.21.15.100:0/4084488346 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db91534620 con 0x55db917f1db0 2024-05-10T11:37:22.052 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.046+0000 7f88a8a7d640 1 --2- 172.21.15.100:0/4084488346 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db917f1db0 0x55db91800430 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f889801d9c0 tx=0x7f8898004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3dedb6668b352c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:37:22.053 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.046+0000 7f8883fff640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:37:22.053 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.046+0000 7f8883fff640 1 -- 172.21.15.100:0/4084488346 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f889802d060 con 0x55db917f1db0 2024-05-10T11:37:22.053 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.046+0000 7f88ada02740 1 -- 172.21.15.100:0/4084488346 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f889c037040 con 0x55db917f1db0 2024-05-10T11:37:22.053 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.046+0000 7f8883fff640 1 -- 172.21.15.100:0/4084488346 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f8898018030 con 0x55db917f1db0 2024-05-10T11:37:22.053 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.047+0000 7f8883fff640 1 -- 172.21.15.100:0/4084488346 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f8898009af0 con 0x55db917f1db0 2024-05-10T11:37:22.053 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.047+0000 7f88ada02740 10 client.28543 fetch_fsmap learned FSMap version 41 2024-05-10T11:37:22.053 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.047+0000 7f88ada02740 1 -- 172.21.15.100:0/4084488346 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55db9180aa40 con 0x55db917f1db0 2024-05-10T11:37:22.053 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.047+0000 7f8883fff640 1 -- 172.21.15.100:0/4084488346 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f889c037040 con 0x55db917f1db0 2024-05-10T11:37:22.053 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.047+0000 7f8883fff640 1 -- 172.21.15.100:0/4084488346 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f8898031450 con 0x55db917f1db0 2024-05-10T11:37:22.053 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.047+0000 7f88ada02740 10 client.28543 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:37:22.053 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.047+0000 7f88ada02740 10 client.28543 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:37:22.053 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.047+0000 7f88ada02740 20 client.28543 populate_metadata read hostname 'smithi100' 2024-05-10T11:37:22.053 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.047+0000 7f88ada02740 1 --2- 172.21.15.100:0/4084488346 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55db9180c0c0 0x55db9180e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:22.054 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.047+0000 7f88ada02740 4 client.28543 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:37:22.054 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.047+0000 7f88ada02740 1 -- 172.21.15.100:0/4084488346 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db917641e0 con 0x55db9180c0c0 2024-05-10T11:37:22.054 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.047+0000 7f88a3fff640 1 --2- 172.21.15.100:0/4084488346 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55db9180c0c0 0x55db9180e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:22.054 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.048+0000 7f8883fff640 10 client.28543 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:37:22.054 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.048+0000 7f88a3fff640 1 --2- 172.21.15.100:0/4084488346 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55db9180c0c0 0x55db9180e4f0 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c313e6ae12cf10e3 server_cookie=5dc166c89181f7df in_seq=0 out_seq=0 2024-05-10T11:37:22.054 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.048+0000 7f8883fff640 1 -- 172.21.15.100:0/4084488346 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55db917641e0 con 0x55db9180c0c0 2024-05-10T11:37:22.054 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.048+0000 7f8883fff640 10 client.28543 handle_command_reply: tid=0 2024-05-10T11:37:22.054 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.048+0000 7f88ada02740 1 client.28543 shutdown 2024-05-10T11:37:22.054 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.049+0000 7f88ada02740 1 -- 172.21.15.100:0/4084488346 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db917f1db0 msgr2=0x55db91800430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:22.054 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.049+0000 7f88ada02740 1 --2- 172.21.15.100:0/4084488346 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db917f1db0 0x55db91800430 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f889801d9c0 tx=0x7f8898004730 comp rx=0 tx=0).stop 2024-05-10T11:37:22.054 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.049+0000 7f88ada02740 1 -- 172.21.15.100:0/4084488346 shutdown_connections 2024-05-10T11:37:22.054 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.049+0000 7f88ada02740 1 --2- 172.21.15.100:0/4084488346 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55db9180c0c0 0x55db9180e4f0 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:22.054 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.049+0000 7f8883fff640 0 client.28543 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:37:22.055 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.049+0000 7f88ada02740 1 --2- 172.21.15.100:0/4084488346 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db917f7910 0x55db918079b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:22.055 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.049+0000 7f88ada02740 1 --2- 172.21.15.100:0/4084488346 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db917f27a0 0x55db91800970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:22.055 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.049+0000 7f88ada02740 1 --2- 172.21.15.100:0/4084488346 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db917f1db0 0x55db91800430 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:22.055 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.049+0000 7f88ada02740 1 -- 172.21.15.100:0/4084488346 >> 172.21.15.100:0/4084488346 conn(0x55db917ed420 msgr2=0x55db917ec580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:37:22.055 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.049+0000 7f88ada02740 1 -- 172.21.15.100:0/4084488346 shutdown_connections 2024-05-10T11:37:22.055 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.049+0000 7f88ada02740 1 -- 172.21.15.100:0/4084488346 wait complete. 2024-05-10T11:37:22.055 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.049+0000 7f88ada02740 20 client.28543 trim_cache size 0 max 16384 2024-05-10T11:37:22.073 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.072+0000 7f88ada02740 1 Processor -- start 2024-05-10T11:37:22.073 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.072+0000 7f88ada02740 1 -- start start 2024-05-10T11:37:22.073 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.072+0000 7f88ada02740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db917ed420 0x55db91800040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:22.073 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.072+0000 7f88ada02740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db917f1db0 0x55db91800580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:22.073 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.072+0000 7f88ada02740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db917f27a0 0x55db917f0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:22.073 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.072+0000 7f88ada02740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f88a41b4200 con 0x55db917ed420 2024-05-10T11:37:22.073 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.072+0000 7f88ada02740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f88a41b1ad0 con 0x55db917f27a0 2024-05-10T11:37:22.074 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.072+0000 7f88ada02740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f88a41af920 con 0x55db917f1db0 2024-05-10T11:37:22.074 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.072+0000 7f88a8a7d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db917ed420 0x55db91800040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:22.074 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.072+0000 7f88a8a7d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db917ed420 0x55db91800040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34922/0 (socket says 172.21.15.100:34922) 2024-05-10T11:37:22.074 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.072+0000 7f88a3fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db917f1db0 0x55db91800580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:22.074 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.072+0000 7f88a8a7d640 1 -- 172.21.15.100:0/745409886 learned_addr learned my addr 172.21.15.100:0/745409886 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:37:22.074 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.073+0000 7f88a927e640 1 --2- 172.21.15.100:0/745409886 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db917f27a0 0x55db917f0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:22.074 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.073+0000 7f88a8a7d640 1 -- 172.21.15.100:0/745409886 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db917f1db0 msgr2=0x55db91800580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:37:22.074 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.073+0000 7f88a8a7d640 1 --2- 172.21.15.100:0/745409886 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db917f1db0 0x55db91800580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:22.074 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.073+0000 7f88a8a7d640 1 -- 172.21.15.100:0/745409886 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db917f27a0 msgr2=0x55db917f0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:37:22.074 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.073+0000 7f88a8a7d640 1 --2- 172.21.15.100:0/745409886 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db917f27a0 0x55db917f0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:22.074 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.073+0000 7f88a8a7d640 1 -- 172.21.15.100:0/745409886 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db917f19f0 con 0x55db917ed420 2024-05-10T11:37:22.074 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.073+0000 7f88a8a7d640 1 --2- 172.21.15.100:0/745409886 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db917ed420 0x55db91800040 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f889801df00 tx=0x7f8898012fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d51355827c62538 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:37:22.074 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.073+0000 7f888e7fc640 1 -- 172.21.15.100:0/745409886 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8898037080 con 0x55db917ed420 2024-05-10T11:37:22.074 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.073+0000 7f888e7fc640 1 -- 172.21.15.100:0/745409886 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8898031cf0 con 0x55db917ed420 2024-05-10T11:37:22.075 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.073+0000 7f888e7fc640 1 -- 172.21.15.100:0/745409886 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88980035f0 con 0x55db917ed420 2024-05-10T11:37:22.077 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.074+0000 7f88ada02740 1 -- 172.21.15.100:0/745409886 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db917ed420 msgr2=0x55db91800040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:22.077 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.074+0000 7f88ada02740 1 --2- 172.21.15.100:0/745409886 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db917ed420 0x55db91800040 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f889801df00 tx=0x7f8898012fd0 comp rx=0 tx=0).stop 2024-05-10T11:37:22.078 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.074+0000 7f88ada02740 1 -- 172.21.15.100:0/745409886 shutdown_connections 2024-05-10T11:37:22.078 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.074+0000 7f88ada02740 1 --2- 172.21.15.100:0/745409886 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db917f27a0 0x55db917f0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:22.078 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.074+0000 7f88ada02740 1 --2- 172.21.15.100:0/745409886 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db917f1db0 0x55db91800580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:22.078 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.074+0000 7f88ada02740 1 --2- 172.21.15.100:0/745409886 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db917ed420 0x55db91800040 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:22.078 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.074+0000 7f88ada02740 1 -- 172.21.15.100:0/745409886 >> 172.21.15.100:0/745409886 conn(0x55db917f73e0 msgr2=0x55db917e83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:37:22.078 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.074+0000 7f88ada02740 1 -- 172.21.15.100:0/745409886 shutdown_connections 2024-05-10T11:37:22.078 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.074+0000 7f88ada02740 1 -- 172.21.15.100:0/745409886 wait complete. 2024-05-10T11:37:22.078 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.074+0000 7f88ada02740 1 -- start start 2024-05-10T11:37:22.081 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.075+0000 7f88ada02740 1 Processor -- start 2024-05-10T11:37:22.081 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.075+0000 7f88ada02740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db917ed420 0x55db91804910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:22.081 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.075+0000 7f88ada02740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db917f1db0 0x55db91804e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:22.081 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.075+0000 7f88ada02740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db917f27a0 0x55db91805df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:22.081 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.075+0000 7f88ada02740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f88a41aff90 con 0x55db917ed420 2024-05-10T11:37:22.081 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.075+0000 7f88ada02740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f88a41b1400 con 0x55db917f1db0 2024-05-10T11:37:22.081 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.075+0000 7f88ada02740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f88a41b0660 con 0x55db917f27a0 2024-05-10T11:37:22.081 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.075+0000 7f88ada02740 10 client.0 _ll_register_callbacks cb 0x7f8898005a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:37:22.081 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.075+0000 7f88a8a7d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db917ed420 0x55db91804910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:22.081 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.075+0000 7f88a8a7d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db917ed420 0x55db91804910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34932/0 (socket says 172.21.15.100:34932) 2024-05-10T11:37:22.082 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.075+0000 7f88a8a7d640 1 -- 172.21.15.100:0/1121716120 learned_addr learned my addr 172.21.15.100:0/1121716120 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:37:22.082 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.075+0000 7f88a3fff640 1 --2- 172.21.15.100:0/1121716120 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db917f1db0 0x55db91804e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:22.082 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.075+0000 7f88a927e640 1 --2- 172.21.15.100:0/1121716120 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db917f27a0 0x55db91805df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:22.082 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.076+0000 7f88a8a7d640 1 -- 172.21.15.100:0/1121716120 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db917f27a0 msgr2=0x55db91805df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:37:22.082 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.076+0000 7f88a8a7d640 1 --2- 172.21.15.100:0/1121716120 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db917f27a0 0x55db91805df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:22.082 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.076+0000 7f88a8a7d640 1 -- 172.21.15.100:0/1121716120 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db917f1db0 msgr2=0x55db91804e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:37:22.082 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.076+0000 7f88a8a7d640 1 --2- 172.21.15.100:0/1121716120 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db917f1db0 0x55db91804e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:22.082 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.076+0000 7f88a8a7d640 1 -- 172.21.15.100:0/1121716120 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db917ee0f0 con 0x55db917ed420 2024-05-10T11:37:22.082 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.076+0000 7f88a8a7d640 1 --2- 172.21.15.100:0/1121716120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db917ed420 0x55db91804910 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f889801d9c0 tx=0x7f88980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dca200e92d4f32cd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:37:22.082 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.076+0000 7f8883fff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:37:22.082 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.076+0000 7f8883fff640 1 -- 172.21.15.100:0/1121716120 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8898040020 con 0x55db917ed420 2024-05-10T11:37:22.082 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.076+0000 7f8883fff640 1 -- 172.21.15.100:0/1121716120 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f8898005240 con 0x55db917ed420 2024-05-10T11:37:22.082 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.076+0000 7f88ada02740 1 -- 172.21.15.100:0/1121716120 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55db9175e050 con 0x55db917ed420 2024-05-10T11:37:22.082 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.076+0000 7f8883fff640 1 -- 172.21.15.100:0/1121716120 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f889804b050 con 0x55db917ed420 2024-05-10T11:37:22.082 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.076+0000 7f88ada02740 10 client.19149 fetch_fsmap learned FSMap version 41 2024-05-10T11:37:22.083 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.076+0000 7f88ada02740 1 -- 172.21.15.100:0/1121716120 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55db918769b0 con 0x55db917ed420 2024-05-10T11:37:22.083 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.077+0000 7f8883fff640 1 -- 172.21.15.100:0/1121716120 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55db9175e050 con 0x55db917ed420 2024-05-10T11:37:22.083 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.077+0000 7f8883fff640 1 -- 172.21.15.100:0/1121716120 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f88980035f0 con 0x55db917ed420 2024-05-10T11:37:22.083 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.077+0000 7f88ada02740 10 client.19149 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:37:22.083 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.077+0000 7f88ada02740 10 client.19149 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:37:22.083 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.077+0000 7f88ada02740 20 client.19149 populate_metadata read hostname 'smithi100' 2024-05-10T11:37:22.083 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.077+0000 7f88ada02740 1 --2- 172.21.15.100:0/1121716120 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55db917f7910 0x55db918792d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:22.083 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.077+0000 7f88ada02740 4 client.19149 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:37:22.083 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.077+0000 7f88ada02740 1 -- 172.21.15.100:0/1121716120 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55db916d95f0 con 0x55db917f7910 2024-05-10T11:37:22.083 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.077+0000 7f88a3fff640 1 --2- 172.21.15.100:0/1121716120 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55db917f7910 0x55db918792d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:22.083 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.077+0000 7f8883fff640 10 client.19149 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:37:22.083 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.077+0000 7f88a3fff640 1 --2- 172.21.15.100:0/1121716120 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55db917f7910 0x55db918792d0 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17fd3ca9053e5e1c server_cookie=277f792425d71c38 in_seq=0 out_seq=0 2024-05-10T11:37:22.181 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.181+0000 7f8883fff640 1 -- 172.21.15.100:0/1121716120 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55db916d95f0 con 0x55db917f7910 2024-05-10T11:37:22.181 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.181+0000 7f8883fff640 10 client.19149 handle_command_reply: tid=0 2024-05-10T11:37:22.181 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.181+0000 7f88ada02740 1 client.19149 shutdown 2024-05-10T11:37:22.183 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.182+0000 7f88ada02740 1 -- 172.21.15.100:0/1121716120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db917ed420 msgr2=0x55db91804910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:22.183 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.182+0000 7f88ada02740 1 --2- 172.21.15.100:0/1121716120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db917ed420 0x55db91804910 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f889801d9c0 tx=0x7f88980057f0 comp rx=0 tx=0).stop 2024-05-10T11:37:22.183 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.182+0000 7f88ada02740 1 -- 172.21.15.100:0/1121716120 shutdown_connections 2024-05-10T11:37:22.183 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.182+0000 7f88ada02740 1 --2- 172.21.15.100:0/1121716120 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55db917f7910 0x55db918792d0 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:22.183 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.182+0000 7f88ada02740 1 --2- 172.21.15.100:0/1121716120 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db917f27a0 0x55db91805df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:22.183 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.182+0000 7f88ada02740 1 --2- 172.21.15.100:0/1121716120 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db917f1db0 0x55db91804e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:22.183 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.182+0000 7f88ada02740 1 --2- 172.21.15.100:0/1121716120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db917ed420 0x55db91804910 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:22.184 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.182+0000 7f88ada02740 1 -- 172.21.15.100:0/1121716120 >> 172.21.15.100:0/1121716120 conn(0x55db917f73e0 msgr2=0x55db91800020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:37:22.184 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.183+0000 7f8883fff640 0 client.19149 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:37:22.184 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.183+0000 7f88ada02740 1 -- 172.21.15.100:0/1121716120 shutdown_connections 2024-05-10T11:37:22.184 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.183+0000 7f88ada02740 1 -- 172.21.15.100:0/1121716120 wait complete. 2024-05-10T11:37:22.184 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.183+0000 7f88ada02740 20 client.19149 trim_cache size 0 max 16384 2024-05-10T11:37:22.184 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:37:22.184 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:37:22.184 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:37:22.184 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:37:22.185 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.184+0000 7f88aace1640 1 -- 172.21.15.100:0/3496189216 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f88940b2920 msgr2=0x7f88940b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:22.185 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.184+0000 7f88aace1640 1 --2- 172.21.15.100:0/3496189216 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f88940b2920 0x7f88940b4de0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x55db917f2ba0 tx=0x7f8898009260 comp rx=0 tx=0).stop 2024-05-10T11:37:22.185 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.184+0000 7f88aace1640 1 -- 172.21.15.100:0/3496189216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88a410a310 msgr2=0x7f88a41a1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:22.185 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.184+0000 7f88aace1640 1 --2- 172.21.15.100:0/3496189216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88a410a310 0x7f88a41a1720 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f889000d950 tx=0x7f889000de10 comp rx=0 tx=0).stop 2024-05-10T11:37:22.185 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.184+0000 7f88aace1640 1 -- 172.21.15.100:0/3496189216 shutdown_connections 2024-05-10T11:37:22.185 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.184+0000 7f88aace1640 1 --2- 172.21.15.100:0/3496189216 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f88940b2920 0x7f88940b4de0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:22.186 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.184+0000 7f88aace1640 1 --2- 172.21.15.100:0/3496189216 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88a4111230 0x7f88a41a8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:22.186 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.184+0000 7f88aace1640 1 --2- 172.21.15.100:0/3496189216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88a410a310 0x7f88a41a1720 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:22.186 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.184+0000 7f88aace1640 1 --2- 172.21.15.100:0/3496189216 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f88a41099b0 0x7f88a41a11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:22.186 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.184+0000 7f88aace1640 1 -- 172.21.15.100:0/3496189216 >> 172.21.15.100:0/3496189216 conn(0x7f88a406fa20 msgr2=0x7f88a4071660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:37:22.186 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.184+0000 7f88aace1640 1 -- 172.21.15.100:0/3496189216 shutdown_connections 2024-05-10T11:37:22.186 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:22.184+0000 7f88aace1640 1 -- 172.21.15.100:0/3496189216 wait complete. 2024-05-10T11:37:22.933 INFO:tasks.workunit.client.0.smithi100.stdout: 10 53151 9.07 MB/sec execute 196 sec latency 159.591 ms 2024-05-10T11:37:23.933 INFO:tasks.workunit.client.0.smithi100.stdout: 10 53322 9.05 MB/sec execute 197 sec latency 93.273 ms 2024-05-10T11:37:24.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:37:23 smithi178 ceph-mon[33359]: pgmap v991: 129 pgs: 129 active+clean; 485 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 4.0 MiB/s wr, 253 op/s 2024-05-10T11:37:24.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:37:23 smithi188 ceph-mon[34075]: pgmap v991: 129 pgs: 129 active+clean; 485 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 4.0 MiB/s wr, 253 op/s 2024-05-10T11:37:24.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:37:23 smithi100 ceph-mon[30296]: pgmap v991: 129 pgs: 129 active+clean; 485 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 4.0 MiB/s wr, 253 op/s 2024-05-10T11:37:24.933 INFO:tasks.workunit.client.0.smithi100.stdout: 10 53485 9.02 MB/sec execute 198 sec latency 80.315 ms 2024-05-10T11:37:25.933 INFO:tasks.workunit.client.0.smithi100.stdout: 10 53649 9.01 MB/sec execute 199 sec latency 78.028 ms 2024-05-10T11:37:26.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:37:25 smithi178 ceph-mon[33359]: pgmap v992: 129 pgs: 129 active+clean; 488 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 4.6 MiB/s wr, 260 op/s 2024-05-10T11:37:26.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:37:25 smithi188 ceph-mon[34075]: pgmap v992: 129 pgs: 129 active+clean; 488 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 4.6 MiB/s wr, 260 op/s 2024-05-10T11:37:26.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:37:25 smithi100 ceph-mon[30296]: pgmap v992: 129 pgs: 129 active+clean; 488 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 4.6 MiB/s wr, 260 op/s 2024-05-10T11:37:26.933 INFO:tasks.workunit.client.0.smithi100.stdout: 10 53853 9.05 MB/sec execute 200 sec latency 111.611 ms 2024-05-10T11:37:27.933 INFO:tasks.workunit.client.0.smithi100.stdout: 10 54040 9.06 MB/sec execute 201 sec latency 82.904 ms 2024-05-10T11:37:28.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:37:27 smithi178 ceph-mon[33359]: pgmap v993: 129 pgs: 129 active+clean; 487 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 5.0 MiB/s wr, 264 op/s 2024-05-10T11:37:28.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:37:27 smithi188 ceph-mon[34075]: pgmap v993: 129 pgs: 129 active+clean; 487 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 5.0 MiB/s wr, 264 op/s 2024-05-10T11:37:28.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:37:27 smithi100 ceph-mon[30296]: pgmap v993: 129 pgs: 129 active+clean; 487 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 5.0 MiB/s wr, 264 op/s 2024-05-10T11:37:28.933 INFO:tasks.workunit.client.0.smithi100.stdout: 10 54206 9.03 MB/sec execute 202 sec latency 85.988 ms 2024-05-10T11:37:29.933 INFO:tasks.workunit.client.0.smithi100.stdout: 10 54406 9.04 MB/sec execute 203 sec latency 200.668 ms 2024-05-10T11:37:30.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:37:29 smithi178 ceph-mon[33359]: pgmap v994: 129 pgs: 129 active+clean; 489 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 4.5 MiB/s wr, 300 op/s 2024-05-10T11:37:30.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:37:29 smithi188 ceph-mon[34075]: pgmap v994: 129 pgs: 129 active+clean; 489 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 4.5 MiB/s wr, 300 op/s 2024-05-10T11:37:30.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:37:29 smithi100 ceph-mon[30296]: pgmap v994: 129 pgs: 129 active+clean; 489 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 4.5 MiB/s wr, 300 op/s 2024-05-10T11:37:30.933 INFO:tasks.workunit.client.0.smithi100.stdout: 10 54608 9.04 MB/sec execute 204 sec latency 69.793 ms 2024-05-10T11:37:31.933 INFO:tasks.workunit.client.0.smithi100.stdout: 10 54777 9.03 MB/sec execute 205 sec latency 70.775 ms 2024-05-10T11:37:32.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:37:31 smithi178 ceph-mon[33359]: pgmap v995: 129 pgs: 129 active+clean; 485 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 909 KiB/s rd, 4.6 MiB/s wr, 307 op/s 2024-05-10T11:37:32.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:37:31 smithi188 ceph-mon[34075]: pgmap v995: 129 pgs: 129 active+clean; 485 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 909 KiB/s rd, 4.6 MiB/s wr, 307 op/s 2024-05-10T11:37:32.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:37:31 smithi100 ceph-mon[30296]: pgmap v995: 129 pgs: 129 active+clean; 485 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 909 KiB/s rd, 4.6 MiB/s wr, 307 op/s 2024-05-10T11:37:32.933 INFO:tasks.workunit.client.0.smithi100.stdout: 10 54971 9.05 MB/sec execute 206 sec latency 72.759 ms 2024-05-10T11:37:33.311 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T11:37:33.897 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:37:33.933 INFO:tasks.workunit.client.0.smithi100.stdout: 10 55182 9.07 MB/sec execute 207 sec latency 83.427 ms 2024-05-10T11:37:34.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:37:33 smithi178 ceph-mon[33359]: pgmap v996: 129 pgs: 129 active+clean; 480 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 4.0 MiB/s wr, 291 op/s 2024-05-10T11:37:34.028 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:37:33 smithi100 ceph-mon[30296]: pgmap v996: 129 pgs: 129 active+clean; 480 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 4.0 MiB/s wr, 291 op/s 2024-05-10T11:37:34.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:37:33 smithi188 ceph-mon[34075]: pgmap v996: 129 pgs: 129 active+clean; 480 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 4.0 MiB/s wr, 291 op/s 2024-05-10T11:37:34.933 INFO:tasks.workunit.client.0.smithi100.stdout: 10 55333 9.06 MB/sec execute 208 sec latency 181.789 ms 2024-05-10T11:37:35.319 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.317+0000 7f0a0a6f8640 1 -- 172.21.15.100:0/1400683230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a04101910 msgr2=0x7f0a04101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:35.319 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.317+0000 7f0a0a6f8640 1 --2- 172.21.15.100:0/1400683230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a04101910 0x7f0a04101d30 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f09f4009900 tx=0x7f09f402a410 comp rx=0 tx=0).stop 2024-05-10T11:37:35.320 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.317+0000 7f0a0a6f8640 1 -- 172.21.15.100:0/1400683230 shutdown_connections 2024-05-10T11:37:35.320 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.317+0000 7f0a0a6f8640 1 --2- 172.21.15.100:0/1400683230 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a041034c0 0x7f0a04109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:35.320 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.317+0000 7f0a0a6f8640 1 --2- 172.21.15.100:0/1400683230 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a04102b00 0x7f0a04102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:35.320 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.317+0000 7f0a0a6f8640 1 --2- 172.21.15.100:0/1400683230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a04101910 0x7f0a04101d30 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:35.320 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.317+0000 7f0a0a6f8640 1 -- 172.21.15.100:0/1400683230 >> 172.21.15.100:0/1400683230 conn(0x7f0a040fce80 msgr2=0x7f0a040ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:37:35.320 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.317+0000 7f0a0a6f8640 1 -- 172.21.15.100:0/1400683230 shutdown_connections 2024-05-10T11:37:35.320 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.317+0000 7f0a0a6f8640 1 -- 172.21.15.100:0/1400683230 wait complete. 2024-05-10T11:37:35.320 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.318+0000 7f0a0a6f8640 1 Processor -- start 2024-05-10T11:37:35.320 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.318+0000 7f0a0a6f8640 1 -- start start 2024-05-10T11:37:35.320 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.319+0000 7f0a0a6f8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a04101910 0x7f0a04198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:35.320 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.319+0000 7f0a0a6f8640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a04102b00 0x7f0a04199070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:35.320 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.319+0000 7f0a0a6f8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a041034c0 0x7f0a041a00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:35.320 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.319+0000 7f0a0a6f8640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0a0410d2e0 con 0x7f0a041034c0 2024-05-10T11:37:35.320 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.319+0000 7f0a0a6f8640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0a0410d160 con 0x7f0a04102b00 2024-05-10T11:37:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.319+0000 7f0a0a6f8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0a0410d460 con 0x7f0a04101910 2024-05-10T11:37:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.319+0000 7f0a08c95640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a041034c0 0x7f0a041a00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.319+0000 7f0a08c95640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a041034c0 0x7f0a041a00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45180/0 (socket says 172.21.15.100:45180) 2024-05-10T11:37:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.319+0000 7f0a08c95640 1 -- 172.21.15.100:0/1426153535 learned_addr learned my addr 172.21.15.100:0/1426153535 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:37:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.319+0000 7f0a03fff640 1 --2- 172.21.15.100:0/1426153535 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a04101910 0x7f0a04198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.319+0000 7f0a037fe640 1 --2- 172.21.15.100:0/1426153535 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a04102b00 0x7f0a04199070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.319+0000 7f0a08c95640 1 -- 172.21.15.100:0/1426153535 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a04101910 msgr2=0x7f0a04198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.319+0000 7f0a08c95640 1 --2- 172.21.15.100:0/1426153535 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a04101910 0x7f0a04198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.319+0000 7f0a08c95640 1 -- 172.21.15.100:0/1426153535 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a04102b00 msgr2=0x7f0a04199070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.319+0000 7f0a08c95640 1 --2- 172.21.15.100:0/1426153535 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a04102b00 0x7f0a04199070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.319+0000 7f0a08c95640 1 -- 172.21.15.100:0/1426153535 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a041a0750 con 0x7f0a041034c0 2024-05-10T11:37:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.319+0000 7f0a037fe640 1 --2- 172.21.15.100:0/1426153535 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a04102b00 0x7f0a04199070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:37:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.319+0000 7f0a08c95640 1 --2- 172.21.15.100:0/1426153535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a041034c0 0x7f0a041a00e0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f09fc00eb10 tx=0x7f09fc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:37:35.322 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.320+0000 7f0a017fa640 1 -- 172.21.15.100:0/1426153535 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09fc004b20 con 0x7f0a041034c0 2024-05-10T11:37:35.322 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.320+0000 7f0a0a6f8640 1 -- 172.21.15.100:0/1426153535 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a041a0a40 con 0x7f0a041034c0 2024-05-10T11:37:35.322 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.320+0000 7f0a017fa640 1 -- 172.21.15.100:0/1426153535 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f09fc004cc0 con 0x7f0a041034c0 2024-05-10T11:37:35.322 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.320+0000 7f0a0a6f8640 1 -- 172.21.15.100:0/1426153535 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a041a0f90 con 0x7f0a041034c0 2024-05-10T11:37:35.322 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.320+0000 7f0a017fa640 1 -- 172.21.15.100:0/1426153535 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09fc010690 con 0x7f0a041034c0 2024-05-10T11:37:35.323 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.323+0000 7f0a0d40d740 1 Processor -- start 2024-05-10T11:37:35.323 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.324+0000 7f0a017fa640 1 -- 172.21.15.100:0/1426153535 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f09fc005020 con 0x7f0a041034c0 2024-05-10T11:37:35.324 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.324+0000 7f0a0d40d740 1 -- start start 2024-05-10T11:37:35.325 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.324+0000 7f0a0d40d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5636bdec0d30 0x5636bdec1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:35.325 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.324+0000 7f0a0d40d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5636bdec1720 0x5636bdec4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:35.325 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.324+0000 7f0a0d40d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636bdec4880 0x5636bded16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:35.326 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.324+0000 7f0a0d40d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5636bde255d0 con 0x5636bdec4880 2024-05-10T11:37:35.326 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.324+0000 7f0a0d40d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5636bddde890 con 0x5636bdec1720 2024-05-10T11:37:35.326 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.324+0000 7f0a0d40d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5636bde21590 con 0x5636bdec0d30 2024-05-10T11:37:35.326 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.324+0000 7f0a037fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5636bdec0d30 0x5636bdec1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:35.326 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.324+0000 7f0a03fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5636bdec1720 0x5636bdec4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:35.326 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.324+0000 7f0a037fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5636bdec0d30 0x5636bdec1150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:49032/0 (socket says 172.21.15.100:49032) 2024-05-10T11:37:35.326 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.324+0000 7f0a037fe640 1 -- 172.21.15.100:0/1903373479 learned_addr learned my addr 172.21.15.100:0/1903373479 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:37:35.326 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.324+0000 7f0a037fe640 1 --2- 172.21.15.100:0/1903373479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636bdec4880 0x5636bded16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:35.326 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.325+0000 7f0a03fff640 1 -- 172.21.15.100:0/1903373479 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5636bdec0d30 msgr2=0x5636bdec1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:37:35.326 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.325+0000 7f0a03fff640 1 --2- 172.21.15.100:0/1903373479 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5636bdec0d30 0x5636bdec1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:35.326 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.325+0000 7f0a03fff640 1 -- 172.21.15.100:0/1903373479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636bdec4880 msgr2=0x5636bded16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:37:35.326 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.325+0000 7f0a03fff640 1 --2- 172.21.15.100:0/1903373479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636bdec4880 0x5636bded16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:35.327 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.325+0000 7f0a03fff640 1 -- 172.21.15.100:0/1903373479 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636bdb37210 con 0x5636bdec1720 2024-05-10T11:37:35.327 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.325+0000 7f0a037fe640 1 --2- 172.21.15.100:0/1903373479 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5636bdec0d30 0x5636bdec1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:37:35.327 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.325+0000 7f0a017fa640 1 --2- 172.21.15.100:0/1426153535 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f09ec0b28a0 0x7f09ec0b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:35.327 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.325+0000 7f0a03fff640 1 --2- 172.21.15.100:0/1903373479 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5636bdec1720 0x5636bdec4210 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f09f4009230 tx=0x7f09f4005dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4aaffba924a78d0f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:37:35.327 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.325+0000 7f0a037fe640 1 --2- 172.21.15.100:0/1426153535 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f09ec0b28a0 0x7f09ec0b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:35.327 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.325+0000 7f0a017fa640 1 -- 172.21.15.100:0/1426153535 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f09fc0e2ac0 con 0x7f0a041034c0 2024-05-10T11:37:35.327 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.325+0000 7f09ea7fc640 1 -- 172.21.15.100:0/1903373479 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f4035030 con 0x5636bdec1720 2024-05-10T11:37:35.327 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.325+0000 7f09ea7fc640 1 -- 172.21.15.100:0/1903373479 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f09f4040070 con 0x5636bdec1720 2024-05-10T11:37:35.327 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.325+0000 7f09ea7fc640 1 -- 172.21.15.100:0/1903373479 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f4003850 con 0x5636bdec1720 2024-05-10T11:37:35.330 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.326+0000 7f0a0d40d740 1 -- 172.21.15.100:0/1903373479 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5636bdec1720 msgr2=0x5636bdec4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:35.330 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.326+0000 7f0a037fe640 1 --2- 172.21.15.100:0/1426153535 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f09ec0b28a0 0x7f09ec0b4d60 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x5636bdec1f80 tx=0x7f09f8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:37:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.326+0000 7f0a0d40d740 1 --2- 172.21.15.100:0/1903373479 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5636bdec1720 0x5636bdec4210 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f09f4009230 tx=0x7f09f4005dd0 comp rx=0 tx=0).stop 2024-05-10T11:37:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.326+0000 7f0a0d40d740 1 -- 172.21.15.100:0/1903373479 shutdown_connections 2024-05-10T11:37:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.326+0000 7f0a0d40d740 1 --2- 172.21.15.100:0/1903373479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636bdec4880 0x5636bded16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.326+0000 7f0a0d40d740 1 --2- 172.21.15.100:0/1903373479 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5636bdec1720 0x5636bdec4210 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.326+0000 7f0a0d40d740 1 --2- 172.21.15.100:0/1903373479 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5636bdec0d30 0x5636bdec1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.326+0000 7f0a0d40d740 1 -- 172.21.15.100:0/1903373479 >> 172.21.15.100:0/1903373479 conn(0x5636bdebc420 msgr2=0x5636bdebc820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:37:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.326+0000 7f0a0d40d740 1 -- 172.21.15.100:0/1903373479 shutdown_connections 2024-05-10T11:37:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.326+0000 7f0a0d40d740 1 -- 172.21.15.100:0/1903373479 wait complete. 2024-05-10T11:37:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.327+0000 7f0a0d40d740 1 -- start start 2024-05-10T11:37:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.327+0000 7f0a0d40d740 1 Processor -- start 2024-05-10T11:37:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.327+0000 7f0a0d40d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5636bdec0d30 0x5636bded37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.327+0000 7f0a0d40d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636bdec1720 0x5636bded3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.327+0000 7f0a0d40d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5636bdec4880 0x5636bdedad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.327+0000 7f0a0d40d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5636bded2090 con 0x5636bdec1720 2024-05-10T11:37:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.327+0000 7f0a037fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636bdec1720 0x5636bded3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.327+0000 7f0a03fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5636bdec0d30 0x5636bded37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.327+0000 7f0a037fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636bdec1720 0x5636bded3ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45200/0 (socket says 172.21.15.100:45200) 2024-05-10T11:37:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.327+0000 7f0a037fe640 1 -- 172.21.15.100:0/3271039542 learned_addr learned my addr 172.21.15.100:0/3271039542 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:37:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.327+0000 7f0a0d40d740 1 -- 172.21.15.100:0/3271039542 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5636bdde2370 con 0x5636bdec0d30 2024-05-10T11:37:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.327+0000 7f0a0d40d740 1 -- 172.21.15.100:0/3271039542 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5636bded2210 con 0x5636bdec4880 2024-05-10T11:37:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.328+0000 7f0a08c95640 1 --2- 172.21.15.100:0/3271039542 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5636bdec4880 0x5636bdedad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.328+0000 7f0a0d40d740 10 client.0 _ll_register_callbacks cb 0x5636bdb36a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:37:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.328+0000 7f0a03fff640 1 -- 172.21.15.100:0/3271039542 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5636bdec4880 msgr2=0x5636bdedad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:37:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.328+0000 7f0a03fff640 1 --2- 172.21.15.100:0/3271039542 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5636bdec4880 0x5636bdedad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.328+0000 7f0a03fff640 1 -- 172.21.15.100:0/3271039542 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636bdec1720 msgr2=0x5636bded3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:37:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.328+0000 7f0a03fff640 1 --2- 172.21.15.100:0/3271039542 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636bdec1720 0x5636bded3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.328+0000 7f0a03fff640 1 -- 172.21.15.100:0/3271039542 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636bdc03620 con 0x5636bdec0d30 2024-05-10T11:37:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.328+0000 7f0a08c95640 1 --2- 172.21.15.100:0/3271039542 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5636bdec4880 0x5636bdedad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:37:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.328+0000 7f0a03fff640 1 --2- 172.21.15.100:0/3271039542 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5636bdec0d30 0x5636bded37a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x5636bdec2190 tx=0x7f09f4005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f1f986b8a226955 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:37:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.328+0000 7f09e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:37:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.328+0000 7f09e37fe640 1 -- 172.21.15.100:0/3271039542 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f4051060 con 0x5636bdec0d30 2024-05-10T11:37:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.328+0000 7f0a0d40d740 1 -- 172.21.15.100:0/3271039542 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09f4040070 con 0x5636bdec0d30 2024-05-10T11:37:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.328+0000 7f09e37fe640 1 -- 172.21.15.100:0/3271039542 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f09f4031c70 con 0x5636bdec0d30 2024-05-10T11:37:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.328+0000 7f09e37fe640 1 -- 172.21.15.100:0/3271039542 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f09f4058050 con 0x5636bdec0d30 2024-05-10T11:37:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.328+0000 7f0a0d40d740 10 client.28454 fetch_fsmap learned FSMap version 41 2024-05-10T11:37:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.329+0000 7f0a0d40d740 1 -- 172.21.15.100:0/3271039542 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5636bdeddce0 con 0x5636bdec0d30 2024-05-10T11:37:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.329+0000 7f09e37fe640 1 -- 172.21.15.100:0/3271039542 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f09f4040070 con 0x5636bdec0d30 2024-05-10T11:37:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.329+0000 7f09e37fe640 1 -- 172.21.15.100:0/3271039542 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f09f403c440 con 0x5636bdec0d30 2024-05-10T11:37:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.329+0000 7f0a0d40d740 10 client.28454 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:37:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.329+0000 7f0a0d40d740 10 client.28454 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:37:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.329+0000 7f0a0d40d740 20 client.28454 populate_metadata read hostname 'smithi100' 2024-05-10T11:37:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.329+0000 7f0a0d40d740 1 --2- 172.21.15.100:0/3271039542 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5636bdedf4b0 0x5636bdee1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.329+0000 7f0a0d40d740 4 client.28454 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:37:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.329+0000 7f0a0d40d740 1 -- 172.21.15.100:0/3271039542 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636bde331e0 con 0x5636bdedf4b0 2024-05-10T11:37:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.329+0000 7f0a037fe640 1 --2- 172.21.15.100:0/3271039542 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5636bdedf4b0 0x5636bdee1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.330+0000 7f09e37fe640 10 client.28454 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:37:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.330+0000 7f0a037fe640 1 --2- 172.21.15.100:0/3271039542 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5636bdedf4b0 0x5636bdee1900 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7728880baf5739f1 server_cookie=f7bdd4c6c0c526c8 in_seq=0 out_seq=0 2024-05-10T11:37:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.331+0000 7f09e37fe640 1 -- 172.21.15.100:0/3271039542 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5636bde331e0 con 0x5636bdedf4b0 2024-05-10T11:37:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.331+0000 7f09e37fe640 10 client.28454 handle_command_reply: tid=0 2024-05-10T11:37:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.331+0000 7f0a0d40d740 1 client.28454 shutdown 2024-05-10T11:37:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.331+0000 7f0a0d40d740 1 -- 172.21.15.100:0/3271039542 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5636bdec0d30 msgr2=0x5636bded37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.331+0000 7f0a0d40d740 1 --2- 172.21.15.100:0/3271039542 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5636bdec0d30 0x5636bded37a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x5636bdec2190 tx=0x7f09f4005fd0 comp rx=0 tx=0).stop 2024-05-10T11:37:35.336 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.331+0000 7f0a0d40d740 1 -- 172.21.15.100:0/3271039542 shutdown_connections 2024-05-10T11:37:35.336 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.331+0000 7f0a0d40d740 1 --2- 172.21.15.100:0/3271039542 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5636bdedf4b0 0x5636bdee1900 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:35.336 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.331+0000 7f0a0d40d740 1 --2- 172.21.15.100:0/3271039542 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5636bdec4880 0x5636bdedad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:35.336 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.331+0000 7f09e37fe640 0 client.28454 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:37:35.336 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.331+0000 7f0a0d40d740 1 --2- 172.21.15.100:0/3271039542 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636bdec1720 0x5636bded3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:35.336 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.331+0000 7f0a0d40d740 1 --2- 172.21.15.100:0/3271039542 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5636bdec0d30 0x5636bded37a0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:35.336 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.331+0000 7f0a0d40d740 1 -- 172.21.15.100:0/3271039542 >> 172.21.15.100:0/3271039542 conn(0x5636bdebc420 msgr2=0x5636bdebb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:37:35.336 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.331+0000 7f0a0d40d740 1 -- 172.21.15.100:0/3271039542 shutdown_connections 2024-05-10T11:37:35.336 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.331+0000 7f0a0d40d740 1 -- 172.21.15.100:0/3271039542 wait complete. 2024-05-10T11:37:35.336 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.332+0000 7f0a0d40d740 20 client.28454 trim_cache size 0 max 16384 2024-05-10T11:37:35.359 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.357+0000 7f0a0d40d740 1 Processor -- start 2024-05-10T11:37:35.359 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.357+0000 7f0a0d40d740 1 -- start start 2024-05-10T11:37:35.359 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.358+0000 7f0a0d40d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5636bdec0d30 0x5636bdebacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:35.359 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.358+0000 7f0a0d40d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636bdec1720 0x5636bdecddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:35.359 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.358+0000 7f0a0d40d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5636bdec4880 0x5636bdece2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:35.359 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.358+0000 7f0a037fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636bdec1720 0x5636bdecddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:35.359 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.358+0000 7f0a03fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5636bdec0d30 0x5636bdebacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:35.359 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.358+0000 7f0a037fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636bdec1720 0x5636bdecddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45214/0 (socket says 172.21.15.100:45214) 2024-05-10T11:37:35.359 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.358+0000 7f0a03fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5636bdec0d30 0x5636bdebacf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:57542/0 (socket says 172.21.15.100:57542) 2024-05-10T11:37:35.359 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.358+0000 7f0a037fe640 1 -- 172.21.15.100:0/4200303578 learned_addr learned my addr 172.21.15.100:0/4200303578 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:37:35.359 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.358+0000 7f0a0d40d740 1 -- 172.21.15.100:0/4200303578 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0a041ab9b0 con 0x5636bdec1720 2024-05-10T11:37:35.360 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.358+0000 7f0a0d40d740 1 -- 172.21.15.100:0/4200303578 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0a041a9280 con 0x5636bdec0d30 2024-05-10T11:37:35.360 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.358+0000 7f0a0d40d740 1 -- 172.21.15.100:0/4200303578 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0a041a70d0 con 0x5636bdec4880 2024-05-10T11:37:35.360 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.358+0000 7f0a08c95640 1 --2- 172.21.15.100:0/4200303578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5636bdec4880 0x5636bdece2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:35.360 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.358+0000 7f0a037fe640 1 -- 172.21.15.100:0/4200303578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5636bdec4880 msgr2=0x5636bdece2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:37:35.360 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.358+0000 7f0a037fe640 1 --2- 172.21.15.100:0/4200303578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5636bdec4880 0x5636bdece2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:35.360 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.358+0000 7f0a037fe640 1 -- 172.21.15.100:0/4200303578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5636bdec0d30 msgr2=0x5636bdebacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:37:35.360 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.358+0000 7f0a037fe640 1 --2- 172.21.15.100:0/4200303578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5636bdec0d30 0x5636bdebacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:35.360 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.358+0000 7f0a037fe640 1 -- 172.21.15.100:0/4200303578 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636bdebc250 con 0x5636bdec1720 2024-05-10T11:37:35.360 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.359+0000 7f0a037fe640 1 --2- 172.21.15.100:0/4200303578 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636bdec1720 0x5636bdecddb0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f09f8009fd0 tx=0x7f09f8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbf4e3a3254945c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:37:35.360 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.359+0000 7f09e9ffb640 1 -- 172.21.15.100:0/4200303578 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f8010de0 con 0x5636bdec1720 2024-05-10T11:37:35.360 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.359+0000 7f09e9ffb640 1 -- 172.21.15.100:0/4200303578 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f09f8021d80 con 0x5636bdec1720 2024-05-10T11:37:35.360 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.359+0000 7f09e9ffb640 1 -- 172.21.15.100:0/4200303578 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f8005550 con 0x5636bdec1720 2024-05-10T11:37:35.363 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.360+0000 7f0a0d40d740 1 -- 172.21.15.100:0/4200303578 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636bdec1720 msgr2=0x5636bdecddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:35.363 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.360+0000 7f0a0d40d740 1 --2- 172.21.15.100:0/4200303578 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636bdec1720 0x5636bdecddb0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f09f8009fd0 tx=0x7f09f8016000 comp rx=0 tx=0).stop 2024-05-10T11:37:35.363 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.360+0000 7f0a0d40d740 1 -- 172.21.15.100:0/4200303578 shutdown_connections 2024-05-10T11:37:35.363 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.360+0000 7f0a0d40d740 1 --2- 172.21.15.100:0/4200303578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5636bdec4880 0x5636bdece2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:35.363 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.360+0000 7f0a0d40d740 1 --2- 172.21.15.100:0/4200303578 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636bdec1720 0x5636bdecddb0 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:35.363 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.360+0000 7f0a0d40d740 1 --2- 172.21.15.100:0/4200303578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5636bdec0d30 0x5636bdebacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:35.363 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.360+0000 7f0a0d40d740 1 -- 172.21.15.100:0/4200303578 >> 172.21.15.100:0/4200303578 conn(0x5636bdebc420 msgr2=0x5636bdeb73e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:37:35.363 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.360+0000 7f0a0d40d740 1 -- 172.21.15.100:0/4200303578 shutdown_connections 2024-05-10T11:37:35.364 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.360+0000 7f0a0d40d740 1 -- 172.21.15.100:0/4200303578 wait complete. 2024-05-10T11:37:35.364 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.360+0000 7f0a0d40d740 1 -- start start 2024-05-10T11:37:35.364 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.360+0000 7f0a0d40d740 1 Processor -- start 2024-05-10T11:37:35.364 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.361+0000 7f0a0d40d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636bdec0d30 0x5636bdee6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:35.364 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.361+0000 7f0a0d40d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5636bdec1720 0x5636bdee6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:35.364 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.361+0000 7f0a0d40d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5636bdec4880 0x5636bdf47c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:35.364 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.361+0000 7f0a0d40d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0a041a7740 con 0x5636bdec0d30 2024-05-10T11:37:35.364 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.361+0000 7f0a0d40d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0a041a8bb0 con 0x5636bdec1720 2024-05-10T11:37:35.364 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.361+0000 7f0a0d40d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0a041a7e10 con 0x5636bdec4880 2024-05-10T11:37:35.364 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.361+0000 7f0a03fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636bdec0d30 0x5636bdee6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:35.364 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.361+0000 7f0a03fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636bdec0d30 0x5636bdee6650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45220/0 (socket says 172.21.15.100:45220) 2024-05-10T11:37:35.364 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.361+0000 7f0a03fff640 1 -- 172.21.15.100:0/128811094 learned_addr learned my addr 172.21.15.100:0/128811094 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:37:35.364 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.361+0000 7f0a0d40d740 10 client.0 _ll_register_callbacks cb 0x7f09f80116d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:37:35.364 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.361+0000 7f0a03fff640 1 -- 172.21.15.100:0/128811094 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5636bdec4880 msgr2=0x5636bdf47c90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T11:37:35.365 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.361+0000 7f0a03fff640 1 --2- 172.21.15.100:0/128811094 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5636bdec4880 0x5636bdf47c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:35.365 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.361+0000 7f0a03fff640 1 -- 172.21.15.100:0/128811094 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5636bdec1720 msgr2=0x5636bdee6b90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T11:37:35.365 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.361+0000 7f0a03fff640 1 --2- 172.21.15.100:0/128811094 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5636bdec1720 0x5636bdee6b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:35.365 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.361+0000 7f0a03fff640 1 -- 172.21.15.100:0/128811094 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636bdf481d0 con 0x5636bdec0d30 2024-05-10T11:37:35.365 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.361+0000 7f0a03fff640 1 --2- 172.21.15.100:0/128811094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636bdec0d30 0x5636bdee6650 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f09f403c7e0 tx=0x7f09f403c6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa84951aa2a26542 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:37:35.365 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.361+0000 7f09e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:37:35.365 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.362+0000 7f09e37fe640 1 -- 172.21.15.100:0/128811094 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f4062060 con 0x5636bdec0d30 2024-05-10T11:37:35.365 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.362+0000 7f09e37fe640 1 -- 172.21.15.100:0/128811094 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f09f403ca50 con 0x5636bdec0d30 2024-05-10T11:37:35.365 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.362+0000 7f0a0d40d740 1 -- 172.21.15.100:0/128811094 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09f8021d80 con 0x5636bdec0d30 2024-05-10T11:37:35.365 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.362+0000 7f09e37fe640 1 -- 172.21.15.100:0/128811094 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f09f4051040 con 0x5636bdec0d30 2024-05-10T11:37:35.365 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.362+0000 7f0a0d40d740 10 client.19167 fetch_fsmap learned FSMap version 41 2024-05-10T11:37:35.365 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.362+0000 7f0a0d40d740 1 -- 172.21.15.100:0/128811094 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5636bdf49420 con 0x5636bdec0d30 2024-05-10T11:37:35.365 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.362+0000 7f09e37fe640 1 -- 172.21.15.100:0/128811094 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f09f8021d80 con 0x5636bdec0d30 2024-05-10T11:37:35.365 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.363+0000 7f09e37fe640 1 -- 172.21.15.100:0/128811094 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f09f4003780 con 0x5636bdec0d30 2024-05-10T11:37:35.365 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.363+0000 7f0a0d40d740 10 client.19167 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:37:35.366 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.363+0000 7f0a0d40d740 10 client.19167 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:37:35.366 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.363+0000 7f0a0d40d740 20 client.19167 populate_metadata read hostname 'smithi100' 2024-05-10T11:37:35.366 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.363+0000 7f0a0d40d740 1 --2- 172.21.15.100:0/128811094 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5636bdf49e00 0x5636bdf4c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:35.366 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.363+0000 7f0a0d40d740 4 client.19167 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:37:35.366 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.363+0000 7f0a0d40d740 1 -- 172.21.15.100:0/128811094 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5636bde2d050 con 0x5636bdf49e00 2024-05-10T11:37:35.366 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.363+0000 7f0a037fe640 1 --2- 172.21.15.100:0/128811094 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5636bdf49e00 0x5636bdf4c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:35.366 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.365+0000 7f0a037fe640 1 --2- 172.21.15.100:0/128811094 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5636bdf49e00 0x5636bdf4c220 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=364b63a5c5c97afa server_cookie=144426e3bb9a4059 in_seq=0 out_seq=0 2024-05-10T11:37:35.366 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.365+0000 7f09e37fe640 10 client.19167 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:37:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.506+0000 7f09e37fe640 1 -- 172.21.15.100:0/128811094 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5636bde2d050 con 0x5636bdf49e00 2024-05-10T11:37:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.506+0000 7f09e37fe640 10 client.19167 handle_command_reply: tid=0 2024-05-10T11:37:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.506+0000 7f0a0d40d740 1 client.19167 shutdown 2024-05-10T11:37:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.506+0000 7f0a0d40d740 1 -- 172.21.15.100:0/128811094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636bdec0d30 msgr2=0x5636bdee6650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.506+0000 7f0a0d40d740 1 --2- 172.21.15.100:0/128811094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636bdec0d30 0x5636bdee6650 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f09f403c7e0 tx=0x7f09f403c6f0 comp rx=0 tx=0).stop 2024-05-10T11:37:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.506+0000 7f0a0d40d740 1 -- 172.21.15.100:0/128811094 shutdown_connections 2024-05-10T11:37:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.506+0000 7f0a0d40d740 1 --2- 172.21.15.100:0/128811094 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5636bdf49e00 0x5636bdf4c220 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.506+0000 7f09e37fe640 0 client.19167 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:37:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.506+0000 7f0a0d40d740 1 --2- 172.21.15.100:0/128811094 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5636bdec4880 0x5636bdf47c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.506+0000 7f0a0d40d740 1 --2- 172.21.15.100:0/128811094 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5636bdec1720 0x5636bdee6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.506+0000 7f0a0d40d740 1 --2- 172.21.15.100:0/128811094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636bdec0d30 0x5636bdee6650 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.506+0000 7f0a0d40d740 1 -- 172.21.15.100:0/128811094 >> 172.21.15.100:0/128811094 conn(0x5636bdebc420 msgr2=0x5636bdecddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:37:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.507+0000 7f0a0d40d740 1 -- 172.21.15.100:0/128811094 shutdown_connections 2024-05-10T11:37:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.507+0000 7f0a0d40d740 1 -- 172.21.15.100:0/128811094 wait complete. 2024-05-10T11:37:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.507+0000 7f0a0d40d740 20 client.19167 trim_cache size 0 max 16384 2024-05-10T11:37:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:37:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:37:35.509 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:37:35.509 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:37:35.509 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.507+0000 7f0a0a6f8640 1 -- 172.21.15.100:0/1426153535 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f09ec0b28a0 msgr2=0x7f09ec0b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:35.509 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.508+0000 7f0a0a6f8640 1 --2- 172.21.15.100:0/1426153535 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f09ec0b28a0 0x7f09ec0b4d60 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x5636bdec1f80 tx=0x7f09f8005f90 comp rx=0 tx=0).stop 2024-05-10T11:37:35.509 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.508+0000 7f0a0a6f8640 1 -- 172.21.15.100:0/1426153535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a041034c0 msgr2=0x7f0a041a00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:35.510 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.508+0000 7f0a0a6f8640 1 --2- 172.21.15.100:0/1426153535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a041034c0 0x7f0a041a00e0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f09fc00eb10 tx=0x7f09fc00c2d0 comp rx=0 tx=0).stop 2024-05-10T11:37:35.510 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.508+0000 7f0a0a6f8640 1 -- 172.21.15.100:0/1426153535 shutdown_connections 2024-05-10T11:37:35.510 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.508+0000 7f0a0a6f8640 1 --2- 172.21.15.100:0/1426153535 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f09ec0b28a0 0x7f09ec0b4d60 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:35.510 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.508+0000 7f0a0a6f8640 1 --2- 172.21.15.100:0/1426153535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a041034c0 0x7f0a041a00e0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:35.510 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.508+0000 7f0a0a6f8640 1 --2- 172.21.15.100:0/1426153535 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a04102b00 0x7f0a04199070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:35.510 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.508+0000 7f0a0a6f8640 1 --2- 172.21.15.100:0/1426153535 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a04101910 0x7f0a04198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:35.510 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.508+0000 7f0a0a6f8640 1 -- 172.21.15.100:0/1426153535 >> 172.21.15.100:0/1426153535 conn(0x7f0a040fce80 msgr2=0x7f0a0406a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:37:35.510 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.508+0000 7f0a0a6f8640 1 -- 172.21.15.100:0/1426153535 shutdown_connections 2024-05-10T11:37:35.510 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:35.508+0000 7f0a0a6f8640 1 -- 172.21.15.100:0/1426153535 wait complete. 2024-05-10T11:37:35.933 INFO:tasks.workunit.client.0.smithi100.stdout: 10 55456 9.06 MB/sec execute 209 sec latency 148.949 ms 2024-05-10T11:37:35.998 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:37:35 smithi100 ceph-mon[30296]: pgmap v997: 129 pgs: 129 active+clean; 484 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 883 KiB/s rd, 4.1 MiB/s wr, 311 op/s 2024-05-10T11:37:36.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:37:35 smithi178 ceph-mon[33359]: pgmap v997: 129 pgs: 129 active+clean; 484 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 883 KiB/s rd, 4.1 MiB/s wr, 311 op/s 2024-05-10T11:37:36.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:37:35 smithi188 ceph-mon[34075]: pgmap v997: 129 pgs: 129 active+clean; 484 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 883 KiB/s rd, 4.1 MiB/s wr, 311 op/s 2024-05-10T11:37:36.933 INFO:tasks.workunit.client.0.smithi100.stdout: 10 55640 9.11 MB/sec execute 210 sec latency 113.189 ms 2024-05-10T11:37:37.933 INFO:tasks.workunit.client.0.smithi100.stdout: 10 55804 9.09 MB/sec execute 211 sec latency 130.895 ms 2024-05-10T11:37:38.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:37:37 smithi178 ceph-mon[33359]: pgmap v998: 129 pgs: 129 active+clean; 487 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 843 KiB/s rd, 4.1 MiB/s wr, 294 op/s 2024-05-10T11:37:38.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:37:37 smithi188 ceph-mon[34075]: pgmap v998: 129 pgs: 129 active+clean; 487 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 843 KiB/s rd, 4.1 MiB/s wr, 294 op/s 2024-05-10T11:37:38.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:37:37 smithi100 ceph-mon[30296]: pgmap v998: 129 pgs: 129 active+clean; 487 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 843 KiB/s rd, 4.1 MiB/s wr, 294 op/s 2024-05-10T11:37:38.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.824+0000 7f0262ef2700 1 -- 172.21.15.100:0/287771398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f025c06b180 msgr2=0x7f025c06b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:38.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.824+0000 7f0262ef2700 1 --2- 172.21.15.100:0/287771398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f025c06b180 0x7f025c06b600 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f0248009990 tx=0x7f0248009ca0 comp rx=0 tx=0).stop 2024-05-10T11:37:38.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.824+0000 7f0262ef2700 1 -- 172.21.15.100:0/287771398 shutdown_connections 2024-05-10T11:37:38.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.824+0000 7f0262ef2700 1 --2- 172.21.15.100:0/287771398 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f025c06bb40 0x7f025c110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:38.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.824+0000 7f0262ef2700 1 --2- 172.21.15.100:0/287771398 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f025c109510 0x7f025c1098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:38.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.824+0000 7f0262ef2700 1 --2- 172.21.15.100:0/287771398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f025c06b180 0x7f025c06b600 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:38.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.824+0000 7f0262ef2700 1 -- 172.21.15.100:0/287771398 >> 172.21.15.100:0/287771398 conn(0x7f025c079350 msgr2=0x7f025c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:37:38.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.824+0000 7f0262ef2700 1 -- 172.21.15.100:0/287771398 shutdown_connections 2024-05-10T11:37:38.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.824+0000 7f0262ef2700 1 -- 172.21.15.100:0/287771398 wait complete. 2024-05-10T11:37:38.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.825+0000 7f0262ef2700 1 Processor -- start 2024-05-10T11:37:38.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.825+0000 7f0262ef2700 1 -- start start 2024-05-10T11:37:38.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.825+0000 7f0262ef2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f025c06b180 0x7f025c19f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:38.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.825+0000 7f0262ef2700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f025c06bb40 0x7f025c19fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:38.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.825+0000 7f0262ef2700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f025c109510 0x7f025c1997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:38.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.825+0000 7f0262ef2700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f025c113540 con 0x7f025c06b180 2024-05-10T11:37:38.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.825+0000 7f0262ef2700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f025c1133c0 con 0x7f025c109510 2024-05-10T11:37:38.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.825+0000 7f0262ef2700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f025c1136c0 con 0x7f025c06bb40 2024-05-10T11:37:38.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.825+0000 7f0260c8e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f025c06b180 0x7f025c19f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:38.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.825+0000 7f0260c8e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f025c06b180 0x7f025c19f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38410/0 (socket says 172.21.15.100:38410) 2024-05-10T11:37:38.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.825+0000 7f0260c8e700 1 -- 172.21.15.100:0/747865313 learned_addr learned my addr 172.21.15.100:0/747865313 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:37:38.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.825+0000 7f026148f700 1 --2- 172.21.15.100:0/747865313 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f025c109510 0x7f025c1997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:38.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.825+0000 7f025bfff700 1 --2- 172.21.15.100:0/747865313 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f025c06bb40 0x7f025c19fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:38.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.826+0000 7f0260c8e700 1 -- 172.21.15.100:0/747865313 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f025c06bb40 msgr2=0x7f025c19fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:38.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.826+0000 7f0260c8e700 1 --2- 172.21.15.100:0/747865313 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f025c06bb40 0x7f025c19fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:38.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.826+0000 7f0260c8e700 1 -- 172.21.15.100:0/747865313 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f025c109510 msgr2=0x7f025c1997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:38.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.826+0000 7f0260c8e700 1 --2- 172.21.15.100:0/747865313 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f025c109510 0x7f025c1997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:38.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.826+0000 7f0260c8e700 1 -- 172.21.15.100:0/747865313 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f025c19a080 con 0x7f025c06b180 2024-05-10T11:37:38.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.826+0000 7f026148f700 1 --2- 172.21.15.100:0/747865313 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f025c109510 0x7f025c1997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:37:38.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.826+0000 7f0260c8e700 1 --2- 172.21.15.100:0/747865313 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f025c06b180 0x7f025c19f680 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f025000e9a0 tx=0x7f025000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:37:38.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.826+0000 7f0259ffb700 1 -- 172.21.15.100:0/747865313 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f025000ca10 con 0x7f025c06b180 2024-05-10T11:37:38.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.826+0000 7f0259ffb700 1 -- 172.21.15.100:0/747865313 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f025000cbb0 con 0x7f025c06b180 2024-05-10T11:37:38.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.826+0000 7f0259ffb700 1 -- 172.21.15.100:0/747865313 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0250018680 con 0x7f025c06b180 2024-05-10T11:37:38.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.826+0000 7f0262ef2700 1 -- 172.21.15.100:0/747865313 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f025c19a310 con 0x7f025c06b180 2024-05-10T11:37:38.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.827+0000 7f0262ef2700 1 -- 172.21.15.100:0/747865313 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f025c1023a0 con 0x7f025c06b180 2024-05-10T11:37:38.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.830+0000 7f0259ffb700 1 -- 172.21.15.100:0/747865313 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f0250010430 con 0x7f025c06b180 2024-05-10T11:37:38.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.830+0000 7f0259ffb700 1 --2- 172.21.15.100:0/747865313 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f024c0b2880 0x7f024c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:38.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.831+0000 7f025bfff700 1 --2- 172.21.15.100:0/747865313 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f024c0b2880 0x7f024c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:38.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.831+0000 7f0259ffb700 1 -- 172.21.15.100:0/747865313 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f02500e26b0 con 0x7f025c06b180 2024-05-10T11:37:38.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.831+0000 7f025bfff700 1 --2- 172.21.15.100:0/747865313 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f024c0b2880 0x7f024c0b4d40 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f025c19af10 tx=0x7f024800b610 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:37:38.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.832+0000 7f026d785b80 1 Processor -- start 2024-05-10T11:37:38.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.832+0000 7f026d785b80 1 -- start start 2024-05-10T11:37:38.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.832+0000 7f026d785b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e3a7d81b0 0x562e3a7d8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:38.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.832+0000 7f026d785b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e3a7d8ad0 0x562e3a7d2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:38.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.832+0000 7f026d785b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e3a7d2d50 0x562e3a7d3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:38.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.832+0000 7f026d785b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x562e3a724940 con 0x562e3a7d81b0 2024-05-10T11:37:38.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.832+0000 7f0260c8e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e3a7d81b0 0x562e3a7d8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:38.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.833+0000 7f026d785b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562e3a689da0 con 0x562e3a7d8ad0 2024-05-10T11:37:38.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.833+0000 7f0260c8e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e3a7d81b0 0x562e3a7d8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38418/0 (socket says 172.21.15.100:38418) 2024-05-10T11:37:38.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.833+0000 7f025bfff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e3a7d8ad0 0x562e3a7d2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:38.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.833+0000 7f0260c8e700 1 -- 172.21.15.100:0/3244140839 learned_addr learned my addr 172.21.15.100:0/3244140839 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:37:38.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.833+0000 7f026148f700 1 --2- 172.21.15.100:0/3244140839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e3a7d2d50 0x562e3a7d3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:38.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.833+0000 7f026d785b80 1 -- 172.21.15.100:0/3244140839 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x562e3a720900 con 0x562e3a7d2d50 2024-05-10T11:37:38.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.833+0000 7f025bfff700 1 -- 172.21.15.100:0/3244140839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e3a7d2d50 msgr2=0x562e3a7d3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:37:38.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.833+0000 7f025bfff700 1 --2- 172.21.15.100:0/3244140839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e3a7d2d50 0x562e3a7d3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:38.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.833+0000 7f025bfff700 1 -- 172.21.15.100:0/3244140839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e3a7d81b0 msgr2=0x562e3a7d8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:37:38.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.833+0000 7f025bfff700 1 --2- 172.21.15.100:0/3244140839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e3a7d81b0 0x562e3a7d8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:38.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.833+0000 7f025bfff700 1 -- 172.21.15.100:0/3244140839 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e3a6718d0 con 0x562e3a7d8ad0 2024-05-10T11:37:38.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.833+0000 7f0260c8e700 1 --2- 172.21.15.100:0/3244140839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e3a7d81b0 0x562e3a7d8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:37:38.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.833+0000 7f025bfff700 1 --2- 172.21.15.100:0/3244140839 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e3a7d8ad0 0x562e3a7d2810 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0248006010 tx=0x7f0248028000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=115d034f9d94f6f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:37:38.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.834+0000 7f0246ffd700 1 -- 172.21.15.100:0/3244140839 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0248003680 con 0x562e3a7d8ad0 2024-05-10T11:37:38.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.834+0000 7f0246ffd700 1 -- 172.21.15.100:0/3244140839 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0248003cb0 con 0x562e3a7d8ad0 2024-05-10T11:37:38.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.834+0000 7f0246ffd700 1 -- 172.21.15.100:0/3244140839 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0248039470 con 0x562e3a7d8ad0 2024-05-10T11:37:38.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.835+0000 7f026d785b80 1 -- 172.21.15.100:0/3244140839 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e3a7d8ad0 msgr2=0x562e3a7d2810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:38.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.835+0000 7f026d785b80 1 --2- 172.21.15.100:0/3244140839 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e3a7d8ad0 0x562e3a7d2810 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0248006010 tx=0x7f0248028000 comp rx=0 tx=0).stop 2024-05-10T11:37:38.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.835+0000 7f026d785b80 1 -- 172.21.15.100:0/3244140839 shutdown_connections 2024-05-10T11:37:38.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.835+0000 7f026d785b80 1 --2- 172.21.15.100:0/3244140839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e3a7d2d50 0x562e3a7d3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:38.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.835+0000 7f026d785b80 1 --2- 172.21.15.100:0/3244140839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e3a7d81b0 0x562e3a7d8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:38.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.835+0000 7f026d785b80 1 --2- 172.21.15.100:0/3244140839 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e3a7d8ad0 0x562e3a7d2810 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:38.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.835+0000 7f026d785b80 1 -- 172.21.15.100:0/3244140839 >> 172.21.15.100:0/3244140839 conn(0x562e3a6feb70 msgr2=0x562e3a7cd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:37:38.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.835+0000 7f026d785b80 1 -- 172.21.15.100:0/3244140839 shutdown_connections 2024-05-10T11:37:38.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.835+0000 7f026d785b80 1 -- 172.21.15.100:0/3244140839 wait complete. 2024-05-10T11:37:38.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.835+0000 7f026d785b80 1 -- start start 2024-05-10T11:37:38.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.836+0000 7f026d785b80 1 Processor -- start 2024-05-10T11:37:38.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.836+0000 7f026d785b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e3a7d2d50 0x562e3a7de840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:38.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.836+0000 7f026d785b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e3a7d81b0 0x562e3a7dbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:38.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.836+0000 7f026d785b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e3a7d8ad0 0x562e3a7dc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:38.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.836+0000 7f026d785b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x562e3a7df3f0 con 0x562e3a7d8ad0 2024-05-10T11:37:38.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.836+0000 7f026d785b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562e3a6dded0 con 0x562e3a7d81b0 2024-05-10T11:37:38.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.836+0000 7f026d785b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x562e3a7df570 con 0x562e3a7d2d50 2024-05-10T11:37:38.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.836+0000 7f026d785b80 10 client.0 _ll_register_callbacks cb 0x562e3a6b85b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:37:38.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.836+0000 7f0260c8e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e3a7d2d50 0x562e3a7de840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:38.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.836+0000 7f025bfff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e3a7d81b0 0x562e3a7dbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:38.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.837+0000 7f0260c8e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e3a7d2d50 0x562e3a7de840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:45246/0 (socket says 172.21.15.100:45246) 2024-05-10T11:37:38.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.837+0000 7f0260c8e700 1 -- 172.21.15.100:0/3408060569 learned_addr learned my addr 172.21.15.100:0/3408060569 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:37:38.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.837+0000 7f025bfff700 1 --2- 172.21.15.100:0/3408060569 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e3a7d81b0 0x562e3a7dbbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:59730/0 (socket says 172.21.15.100:59730) 2024-05-10T11:37:38.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.837+0000 7f026148f700 1 --2- 172.21.15.100:0/3408060569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e3a7d8ad0 0x562e3a7dc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:38.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.837+0000 7f025bfff700 1 -- 172.21.15.100:0/3408060569 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e3a7d2d50 msgr2=0x562e3a7de840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:37:38.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.837+0000 7f025bfff700 1 --2- 172.21.15.100:0/3408060569 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e3a7d2d50 0x562e3a7de840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:38.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.837+0000 7f025bfff700 1 -- 172.21.15.100:0/3408060569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e3a7d8ad0 msgr2=0x562e3a7dc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:37:38.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.837+0000 7f025bfff700 1 --2- 172.21.15.100:0/3408060569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e3a7d8ad0 0x562e3a7dc120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:38.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.838+0000 7f025bfff700 1 -- 172.21.15.100:0/3408060569 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e3a6db760 con 0x562e3a7d81b0 2024-05-10T11:37:38.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.838+0000 7f0260c8e700 1 --2- 172.21.15.100:0/3408060569 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e3a7d2d50 0x562e3a7de840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:37:38.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.838+0000 7f026148f700 1 --2- 172.21.15.100:0/3408060569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e3a7d8ad0 0x562e3a7dc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:37:38.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.838+0000 7f025bfff700 1 --2- 172.21.15.100:0/3408060569 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e3a7d81b0 0x562e3a7dbbe0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x562e3a7d8eb0 tx=0x7f024800b6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1c0a955d85b6cea server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:37:38.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.838+0000 7f023bfff700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:37:38.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.838+0000 7f023bfff700 1 -- 172.21.15.100:0/3408060569 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0248047060 con 0x562e3a7d81b0 2024-05-10T11:37:38.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.838+0000 7f026d785b80 1 -- 172.21.15.100:0/3408060569 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0248003cb0 con 0x562e3a7d81b0 2024-05-10T11:37:38.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.838+0000 7f023bfff700 1 -- 172.21.15.100:0/3408060569 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f0248003a90 con 0x562e3a7d81b0 2024-05-10T11:37:38.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.838+0000 7f023bfff700 1 -- 172.21.15.100:0/3408060569 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f024804e050 con 0x562e3a7d81b0 2024-05-10T11:37:38.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.838+0000 7f026d785b80 10 client.28469 fetch_fsmap learned FSMap version 41 2024-05-10T11:37:38.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.838+0000 7f026d785b80 1 -- 172.21.15.100:0/3408060569 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x562e3a7e9d90 con 0x562e3a7d81b0 2024-05-10T11:37:38.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.839+0000 7f023bfff700 1 -- 172.21.15.100:0/3408060569 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0248003cb0 con 0x562e3a7d81b0 2024-05-10T11:37:38.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.839+0000 7f023bfff700 1 -- 172.21.15.100:0/3408060569 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f024802d070 con 0x562e3a7d81b0 2024-05-10T11:37:38.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.839+0000 7f026d785b80 10 client.28469 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:37:38.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.839+0000 7f026d785b80 10 client.28469 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:37:38.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.839+0000 7f026d785b80 20 client.28469 populate_metadata read hostname 'smithi100' 2024-05-10T11:37:38.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.839+0000 7f026d785b80 1 --2- 172.21.15.100:0/3408060569 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562e3a7ec300 0x562e3a7ee6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:38.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.839+0000 7f026d785b80 4 client.28469 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:37:38.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.839+0000 7f026d785b80 1 -- 172.21.15.100:0/3408060569 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e3a732320 con 0x562e3a7ec300 2024-05-10T11:37:38.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.840+0000 7f0260c8e700 1 --2- 172.21.15.100:0/3408060569 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562e3a7ec300 0x562e3a7ee6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:38.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.841+0000 7f023bfff700 10 client.28469 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:37:38.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.841+0000 7f0260c8e700 1 --2- 172.21.15.100:0/3408060569 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562e3a7ec300 0x562e3a7ee6f0 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=918ec40621961c95 server_cookie=448e2fa7137d8831 in_seq=0 out_seq=0 2024-05-10T11:37:38.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.844+0000 7f023bfff700 1 -- 172.21.15.100:0/3408060569 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562e3a732320 con 0x562e3a7ec300 2024-05-10T11:37:38.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.844+0000 7f023bfff700 10 client.28469 handle_command_reply: tid=0 2024-05-10T11:37:38.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.844+0000 7f026d785b80 1 client.28469 shutdown 2024-05-10T11:37:38.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.844+0000 7f026d785b80 1 -- 172.21.15.100:0/3408060569 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e3a7d81b0 msgr2=0x562e3a7dbbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:38.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.844+0000 7f026d785b80 1 --2- 172.21.15.100:0/3408060569 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e3a7d81b0 0x562e3a7dbbe0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x562e3a7d8eb0 tx=0x7f024800b6b0 comp rx=0 tx=0).stop 2024-05-10T11:37:38.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.844+0000 7f026d785b80 1 -- 172.21.15.100:0/3408060569 shutdown_connections 2024-05-10T11:37:38.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.844+0000 7f026d785b80 1 --2- 172.21.15.100:0/3408060569 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562e3a7ec300 0x562e3a7ee6f0 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:38.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.844+0000 7f026d785b80 1 --2- 172.21.15.100:0/3408060569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e3a7d8ad0 0x562e3a7dc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:38.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.844+0000 7f026d785b80 1 --2- 172.21.15.100:0/3408060569 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e3a7d2d50 0x562e3a7de840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:38.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.844+0000 7f026d785b80 1 --2- 172.21.15.100:0/3408060569 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e3a7d81b0 0x562e3a7dbbe0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:38.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.844+0000 7f026d785b80 1 -- 172.21.15.100:0/3408060569 >> 172.21.15.100:0/3408060569 conn(0x562e3a6feb70 msgr2=0x562e3a7c7b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:37:38.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.844+0000 7f023bfff700 0 client.28469 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:37:38.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.845+0000 7f026d785b80 1 -- 172.21.15.100:0/3408060569 shutdown_connections 2024-05-10T11:37:38.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.845+0000 7f026d785b80 1 -- 172.21.15.100:0/3408060569 wait complete. 2024-05-10T11:37:38.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.845+0000 7f026d785b80 20 client.28469 trim_cache size 0 max 16384 2024-05-10T11:37:38.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.880+0000 7f026d785b80 1 Processor -- start 2024-05-10T11:37:38.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.882+0000 7f026d785b80 1 -- start start 2024-05-10T11:37:38.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.882+0000 7f026d785b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e3a7d2d50 0x562e3a7d2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:38.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.882+0000 7f026d785b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e3a7d81b0 0x562e3a7da840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:38.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.882+0000 7f026d785b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e3a7d8ad0 0x562e3a7d9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:38.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.882+0000 7f026d785b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f025c1af5c0 con 0x562e3a7d8ad0 2024-05-10T11:37:38.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.882+0000 7f026d785b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f025c1ace50 con 0x562e3a7d81b0 2024-05-10T11:37:38.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.882+0000 7f026d785b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f025c1aaca0 con 0x562e3a7d2d50 2024-05-10T11:37:38.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.882+0000 7f026148f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e3a7d8ad0 0x562e3a7d9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:38.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.882+0000 7f026148f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e3a7d8ad0 0x562e3a7d9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38430/0 (socket says 172.21.15.100:38430) 2024-05-10T11:37:38.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.882+0000 7f026148f700 1 -- 172.21.15.100:0/70045126 learned_addr learned my addr 172.21.15.100:0/70045126 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:37:38.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.883+0000 7f026148f700 1 -- 172.21.15.100:0/70045126 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e3a7d2d50 msgr2=0x562e3a7d2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:37:38.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.883+0000 7f0260c8e700 1 --2- 172.21.15.100:0/70045126 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e3a7d2d50 0x562e3a7d2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:38.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.883+0000 7f026148f700 1 --2- 172.21.15.100:0/70045126 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e3a7d2d50 0x562e3a7d2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:38.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.883+0000 7f026148f700 1 -- 172.21.15.100:0/70045126 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e3a7d81b0 msgr2=0x562e3a7da840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:37:38.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.883+0000 7f025bfff700 1 --2- 172.21.15.100:0/70045126 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e3a7d81b0 0x562e3a7da840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:38.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.883+0000 7f026148f700 1 --2- 172.21.15.100:0/70045126 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e3a7d81b0 0x562e3a7da840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:38.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.883+0000 7f026148f700 1 -- 172.21.15.100:0/70045126 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e3a7d87e0 con 0x562e3a7d8ad0 2024-05-10T11:37:38.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.883+0000 7f0260c8e700 1 --2- 172.21.15.100:0/70045126 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e3a7d2d50 0x562e3a7d2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:37:38.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.883+0000 7f026148f700 1 --2- 172.21.15.100:0/70045126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e3a7d8ad0 0x562e3a7d9770 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f025400e9c0 tx=0x7f0254009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbf8c6334358c574 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:37:38.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.884+0000 7f02467fc700 1 -- 172.21.15.100:0/70045126 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f025400eea0 con 0x562e3a7d8ad0 2024-05-10T11:37:38.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.884+0000 7f02467fc700 1 -- 172.21.15.100:0/70045126 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f025401b070 con 0x562e3a7d8ad0 2024-05-10T11:37:38.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.884+0000 7f02467fc700 1 -- 172.21.15.100:0/70045126 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0254020c60 con 0x562e3a7d8ad0 2024-05-10T11:37:38.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.885+0000 7f026d785b80 1 -- 172.21.15.100:0/70045126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e3a7d8ad0 msgr2=0x562e3a7d9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:38.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.885+0000 7f026d785b80 1 --2- 172.21.15.100:0/70045126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e3a7d8ad0 0x562e3a7d9770 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f025400e9c0 tx=0x7f0254009920 comp rx=0 tx=0).stop 2024-05-10T11:37:38.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.885+0000 7f026d785b80 1 -- 172.21.15.100:0/70045126 shutdown_connections 2024-05-10T11:37:38.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.885+0000 7f026d785b80 1 --2- 172.21.15.100:0/70045126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e3a7d8ad0 0x562e3a7d9770 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:38.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.885+0000 7f026d785b80 1 --2- 172.21.15.100:0/70045126 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e3a7d2d50 0x562e3a7d2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:38.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.885+0000 7f026d785b80 1 --2- 172.21.15.100:0/70045126 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e3a7d81b0 0x562e3a7da840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:38.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.886+0000 7f026d785b80 1 -- 172.21.15.100:0/70045126 >> 172.21.15.100:0/70045126 conn(0x562e3a6feb70 msgr2=0x562e3a7d60b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:37:38.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.886+0000 7f026d785b80 1 -- 172.21.15.100:0/70045126 shutdown_connections 2024-05-10T11:37:38.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.887+0000 7f026d785b80 1 -- 172.21.15.100:0/70045126 wait complete. 2024-05-10T11:37:38.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.887+0000 7f026d785b80 1 -- start start 2024-05-10T11:37:38.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.887+0000 7f026d785b80 1 Processor -- start 2024-05-10T11:37:38.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.889+0000 7f026d785b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e3a7d2d50 0x562e3a7e4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:38.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.889+0000 7f026d785b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e3a7d81b0 0x562e3a7e4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:38.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.889+0000 7f026d785b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e3a7d8ad0 0x562e3a7f2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:38.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.889+0000 7f026d785b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f025c1ab310 con 0x562e3a7d2d50 2024-05-10T11:37:38.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.889+0000 7f026d785b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f025c1ac780 con 0x562e3a7d8ad0 2024-05-10T11:37:38.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.889+0000 7f026d785b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f025c1ab9e0 con 0x562e3a7d81b0 2024-05-10T11:37:38.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.889+0000 7f026d785b80 10 client.0 _ll_register_callbacks cb 0x562e3a6b85b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:37:38.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.889+0000 7f026148f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e3a7d8ad0 0x562e3a7f2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:38.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.890+0000 7f026148f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e3a7d8ad0 0x562e3a7f2f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:59748/0 (socket says 172.21.15.100:59748) 2024-05-10T11:37:38.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.890+0000 7f026148f700 1 -- 172.21.15.100:0/3195908917 learned_addr learned my addr 172.21.15.100:0/3195908917 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:37:38.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.890+0000 7f025bfff700 1 --2- 172.21.15.100:0/3195908917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e3a7d81b0 0x562e3a7e4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:38.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.890+0000 7f0260c8e700 1 --2- 172.21.15.100:0/3195908917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e3a7d2d50 0x562e3a7e4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:38.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.891+0000 7f025bfff700 1 -- 172.21.15.100:0/3195908917 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e3a7d8ad0 msgr2=0x562e3a7f2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:37:38.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.891+0000 7f025bfff700 1 --2- 172.21.15.100:0/3195908917 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e3a7d8ad0 0x562e3a7f2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:38.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.891+0000 7f025bfff700 1 -- 172.21.15.100:0/3195908917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e3a7d2d50 msgr2=0x562e3a7e4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:37:38.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.891+0000 7f025bfff700 1 --2- 172.21.15.100:0/3195908917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e3a7d2d50 0x562e3a7e4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:38.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.891+0000 7f025bfff700 1 -- 172.21.15.100:0/3195908917 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e3a7f49d0 con 0x562e3a7d81b0 2024-05-10T11:37:38.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.891+0000 7f025bfff700 1 --2- 172.21.15.100:0/3195908917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e3a7d81b0 0x562e3a7e4d40 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x562e3a7c9d30 tx=0x7f024800fe50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc4c0902769a4445 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:37:38.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.891+0000 7f023bfff700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:37:38.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.891+0000 7f023bfff700 1 -- 172.21.15.100:0/3195908917 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0248047040 con 0x562e3a7d81b0 2024-05-10T11:37:38.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.891+0000 7f026d785b80 1 -- 172.21.15.100:0/3195908917 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f025401b070 con 0x562e3a7d81b0 2024-05-10T11:37:38.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.891+0000 7f023bfff700 1 -- 172.21.15.100:0/3195908917 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f0248042480 con 0x562e3a7d81b0 2024-05-10T11:37:38.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.893+0000 7f023bfff700 1 -- 172.21.15.100:0/3195908917 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f0248003e60 con 0x562e3a7d81b0 2024-05-10T11:37:38.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.896+0000 7f026d785b80 10 client.28579 fetch_fsmap learned FSMap version 41 2024-05-10T11:37:38.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.896+0000 7f026d785b80 1 -- 172.21.15.100:0/3195908917 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x562e3a866b50 con 0x562e3a7d81b0 2024-05-10T11:37:38.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.896+0000 7f023bfff700 1 -- 172.21.15.100:0/3195908917 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f025401b070 con 0x562e3a7d81b0 2024-05-10T11:37:38.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.897+0000 7f023bfff700 1 -- 172.21.15.100:0/3195908917 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f024802d070 con 0x562e3a7d81b0 2024-05-10T11:37:38.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.897+0000 7f026d785b80 10 client.28579 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:37:38.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.897+0000 7f026d785b80 10 client.28579 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:37:38.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.897+0000 7f026d785b80 20 client.28579 populate_metadata read hostname 'smithi100' 2024-05-10T11:37:38.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.897+0000 7f026d785b80 1 --2- 172.21.15.100:0/3195908917 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562e3a867080 0x562e3a8694a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:38.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.897+0000 7f026d785b80 4 client.28579 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T11:37:38.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.897+0000 7f026d785b80 1 -- 172.21.15.100:0/3195908917 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562e3a72c3c0 con 0x562e3a867080 2024-05-10T11:37:38.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.898+0000 7f0260c8e700 1 --2- 172.21.15.100:0/3195908917 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562e3a867080 0x562e3a8694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:38.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.900+0000 7f0260c8e700 1 --2- 172.21.15.100:0/3195908917 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562e3a867080 0x562e3a8694a0 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee80ffb8f4295d42 server_cookie=ca27f5d4e25c6e6c in_seq=0 out_seq=0 2024-05-10T11:37:38.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.900+0000 7f023bfff700 10 client.28579 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:37:38.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.900+0000 7f023bfff700 1 -- 172.21.15.100:0/3195908917 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562e3a72c3c0 con 0x562e3a867080 2024-05-10T11:37:38.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.900+0000 7f023bfff700 10 client.28579 handle_command_reply: tid=0 2024-05-10T11:37:38.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.900+0000 7f026d785b80 1 client.28579 shutdown 2024-05-10T11:37:38.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.901+0000 7f026d785b80 1 -- 172.21.15.100:0/3195908917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e3a7d81b0 msgr2=0x562e3a7e4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:38.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.901+0000 7f026d785b80 1 --2- 172.21.15.100:0/3195908917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e3a7d81b0 0x562e3a7e4d40 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x562e3a7c9d30 tx=0x7f024800fe50 comp rx=0 tx=0).stop 2024-05-10T11:37:38.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.901+0000 7f026d785b80 1 -- 172.21.15.100:0/3195908917 shutdown_connections 2024-05-10T11:37:38.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.901+0000 7f026d785b80 1 --2- 172.21.15.100:0/3195908917 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562e3a867080 0x562e3a8694a0 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:38.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.901+0000 7f026d785b80 1 --2- 172.21.15.100:0/3195908917 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e3a7d8ad0 0x562e3a7f2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:38.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.901+0000 7f026d785b80 1 --2- 172.21.15.100:0/3195908917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e3a7d2d50 0x562e3a7e4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:38.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.901+0000 7f026d785b80 1 --2- 172.21.15.100:0/3195908917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e3a7d81b0 0x562e3a7e4d40 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:38.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.901+0000 7f026d785b80 1 -- 172.21.15.100:0/3195908917 >> 172.21.15.100:0/3195908917 conn(0x562e3a6feb70 msgr2=0x562e3a7d6cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:37:38.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.901+0000 7f023bfff700 0 client.28579 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:37:38.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.902+0000 7f026d785b80 1 -- 172.21.15.100:0/3195908917 shutdown_connections 2024-05-10T11:37:38.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.902+0000 7f026d785b80 1 -- 172.21.15.100:0/3195908917 wait complete. 2024-05-10T11:37:38.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.902+0000 7f026d785b80 20 client.28579 trim_cache size 0 max 16384 2024-05-10T11:37:38.903 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:37:38.903 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-05-10T11:37:38.903 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-05-10T11:37:38.903 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:37:38.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.903+0000 7f0262ef2700 1 -- 172.21.15.100:0/747865313 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f024c0b2880 msgr2=0x7f024c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:38.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.903+0000 7f0262ef2700 1 --2- 172.21.15.100:0/747865313 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f024c0b2880 0x7f024c0b4d40 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f025c19af10 tx=0x7f024800b610 comp rx=0 tx=0).stop 2024-05-10T11:37:38.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.903+0000 7f0262ef2700 1 -- 172.21.15.100:0/747865313 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f025c06b180 msgr2=0x7f025c19f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:38.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.903+0000 7f0262ef2700 1 --2- 172.21.15.100:0/747865313 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f025c06b180 0x7f025c19f680 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f025000e9a0 tx=0x7f025000ecb0 comp rx=0 tx=0).stop 2024-05-10T11:37:38.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.903+0000 7f0262ef2700 1 -- 172.21.15.100:0/747865313 shutdown_connections 2024-05-10T11:37:38.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.903+0000 7f0262ef2700 1 --2- 172.21.15.100:0/747865313 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f024c0b2880 0x7f024c0b4d40 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:38.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.903+0000 7f0262ef2700 1 --2- 172.21.15.100:0/747865313 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f025c109510 0x7f025c1997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:38.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.904+0000 7f0262ef2700 1 --2- 172.21.15.100:0/747865313 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f025c06b180 0x7f025c19f680 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:38.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.904+0000 7f0262ef2700 1 --2- 172.21.15.100:0/747865313 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f025c06bb40 0x7f025c19fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:38.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.904+0000 7f0262ef2700 1 -- 172.21.15.100:0/747865313 >> 172.21.15.100:0/747865313 conn(0x7f025c079350 msgr2=0x7f025c100ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:37:38.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.904+0000 7f0262ef2700 1 -- 172.21.15.100:0/747865313 shutdown_connections 2024-05-10T11:37:38.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:38.904+0000 7f0262ef2700 1 -- 172.21.15.100:0/747865313 wait complete. 2024-05-10T11:37:38.933 INFO:tasks.workunit.client.0.smithi100.stdout: 10 55968 9.08 MB/sec execute 212 sec latency 87.561 ms 2024-05-10T11:37:39.933 INFO:tasks.workunit.client.0.smithi100.stdout: 10 56091 9.07 MB/sec execute 213 sec latency 91.972 ms 2024-05-10T11:37:40.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:37:39 smithi178 ceph-mon[33359]: pgmap v999: 129 pgs: 129 active+clean; 487 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 965 KiB/s rd, 4.2 MiB/s wr, 300 op/s 2024-05-10T11:37:40.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:37:39 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:37:40.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:37:39 smithi188 ceph-mon[34075]: pgmap v999: 129 pgs: 129 active+clean; 487 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 965 KiB/s rd, 4.2 MiB/s wr, 300 op/s 2024-05-10T11:37:40.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:37:39 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:37:40.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:37:39 smithi100 ceph-mon[30296]: pgmap v999: 129 pgs: 129 active+clean; 487 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 965 KiB/s rd, 4.2 MiB/s wr, 300 op/s 2024-05-10T11:37:40.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:37:39 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:37:40.274 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T11:37:40.275 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T11:37:40.934 INFO:tasks.workunit.client.0.smithi100.stdout: 10 56208 9.04 MB/sec execute 214 sec latency 108.916 ms 2024-05-10T11:37:41.360 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:37:41.770 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:37:41 smithi100 ceph-mon[30296]: pgmap v1000: 129 pgs: 129 active+clean; 493 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 909 KiB/s rd, 4.7 MiB/s wr, 294 op/s 2024-05-10T11:37:41.934 INFO:tasks.workunit.client.0.smithi100.stdout: 10 56378 9.06 MB/sec execute 215 sec latency 108.804 ms 2024-05-10T11:37:42.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:37:41 smithi178 ceph-mon[33359]: pgmap v1000: 129 pgs: 129 active+clean; 493 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 909 KiB/s rd, 4.7 MiB/s wr, 294 op/s 2024-05-10T11:37:42.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:37:41 smithi188 ceph-mon[34075]: pgmap v1000: 129 pgs: 129 active+clean; 493 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 909 KiB/s rd, 4.7 MiB/s wr, 294 op/s 2024-05-10T11:37:42.934 INFO:tasks.workunit.client.0.smithi100.stdout: 10 56543 9.05 MB/sec execute 216 sec latency 86.440 ms 2024-05-10T11:37:43.217 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:37:42 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:37:43.217 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:37:42 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:37:43.217 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:37:42 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:37:43.217 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:37:42 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:37:43.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:37:42 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:37:43.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:37:42 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:37:43.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:37:42 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:37:43.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:37:42 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:37:43.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:37:42 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:37:43.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:37:42 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:37:43.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:37:42 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:37:43.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:37:42 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:37:43.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:43.381+0000 7f8f4622e700 1 -- 172.21.15.100:0/2323353892 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f4011cc80 msgr2=0x7f8f4011f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:43.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:43.381+0000 7f8f4622e700 1 --2- 172.21.15.100:0/2323353892 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f4011cc80 0x7f8f4011f070 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f8f3800b0a0 tx=0x7f8f3800b3b0 comp rx=0 tx=0).stop 2024-05-10T11:37:43.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:43.381+0000 7f8f4622e700 1 -- 172.21.15.100:0/2323353892 shutdown_connections 2024-05-10T11:37:43.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:43.381+0000 7f8f4622e700 1 --2- 172.21.15.100:0/2323353892 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f4011cc80 0x7f8f4011f070 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:43.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:43.381+0000 7f8f4622e700 1 --2- 172.21.15.100:0/2323353892 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f40111c80 0x7f8f4010a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:43.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:43.381+0000 7f8f4622e700 1 --2- 172.21.15.100:0/2323353892 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f4010a9e0 0x7f8f4010ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:43.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:43.381+0000 7f8f4622e700 1 -- 172.21.15.100:0/2323353892 >> 172.21.15.100:0/2323353892 conn(0x7f8f4006e610 msgr2=0x7f8f4006ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:37:43.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:43.381+0000 7f8f4622e700 1 -- 172.21.15.100:0/2323353892 shutdown_connections 2024-05-10T11:37:43.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:43.382+0000 7f8f4622e700 1 -- 172.21.15.100:0/2323353892 wait complete. 2024-05-10T11:37:43.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:43.383+0000 7f8f4622e700 1 Processor -- start 2024-05-10T11:37:43.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:43.383+0000 7f8f4622e700 1 -- start start 2024-05-10T11:37:43.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:43.384+0000 7f8f4622e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f4010a9e0 0x7f8f401ae690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:43.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:43.384+0000 7f8f4622e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f40111c80 0x7f8f401aebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:43.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:43.384+0000 7f8f4622e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f4011cc80 0x7f8f401a8760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:43.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:43.384+0000 7f8f4622e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8f401223c0 con 0x7f8f40111c80 2024-05-10T11:37:43.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:43.384+0000 7f8f4622e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8f40122240 con 0x7f8f4011cc80 2024-05-10T11:37:43.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:43.384+0000 7f8f4622e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8f40122540 con 0x7f8f4010a9e0 2024-05-10T11:37:43.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:43.384+0000 7f8f3ffff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f4011cc80 0x7f8f401a8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:43.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:43.384+0000 7f8f3effd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f40111c80 0x7f8f401aebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:43.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:43.384+0000 7f8f3ffff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f4011cc80 0x7f8f401a8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:59762/0 (socket says 172.21.15.100:59762) 2024-05-10T11:37:43.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:43.384+0000 7f8f3ffff700 1 -- 172.21.15.100:0/1725856593 learned_addr learned my addr 172.21.15.100:0/1725856593 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:37:43.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:43.384+0000 7f8f3f7fe700 1 --2- 172.21.15.100:0/1725856593 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f4010a9e0 0x7f8f401ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:43.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:43.384+0000 7f8f3ffff700 1 -- 172.21.15.100:0/1725856593 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f4010a9e0 msgr2=0x7f8f401ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:43.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:43.384+0000 7f8f3ffff700 1 --2- 172.21.15.100:0/1725856593 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f4010a9e0 0x7f8f401ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:43.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:43.384+0000 7f8f3ffff700 1 -- 172.21.15.100:0/1725856593 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f40111c80 msgr2=0x7f8f401aebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:43.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:43.384+0000 7f8f3ffff700 1 --2- 172.21.15.100:0/1725856593 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f40111c80 0x7f8f401aebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:43.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:43.384+0000 7f8f3ffff700 1 -- 172.21.15.100:0/1725856593 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f401a9040 con 0x7f8f4011cc80 2024-05-10T11:37:43.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:43.385+0000 7f8f3ffff700 1 --2- 172.21.15.100:0/1725856593 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f4011cc80 0x7f8f401a8760 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8f38006070 tx=0x7f8f38004c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:37:43.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:43.385+0000 7f8f3cff9700 1 -- 172.21.15.100:0/1725856593 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f38021070 con 0x7f8f4011cc80 2024-05-10T11:37:43.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:43.386+0000 7f8f3cff9700 1 -- 172.21.15.100:0/1725856593 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8f38007b80 con 0x7f8f4011cc80 2024-05-10T11:37:43.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:43.386+0000 7f8f4622e700 1 -- 172.21.15.100:0/1725856593 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f401a92d0 con 0x7f8f4011cc80 2024-05-10T11:37:43.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:43.386+0000 7f8f4622e700 1 -- 172.21.15.100:0/1725856593 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f401b5610 con 0x7f8f4011cc80 2024-05-10T11:37:43.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:43.386+0000 7f8f3cff9700 1 -- 172.21.15.100:0/1725856593 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f38004730 con 0x7f8f4011cc80 2024-05-10T11:37:43.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:43.387+0000 7f8f4622e700 1 -- 172.21.15.100:0/1725856593 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f20005330 con 0x7f8f4011cc80 2024-05-10T11:37:43.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:43.390+0000 7f8f3cff9700 1 -- 172.21.15.100:0/1725856593 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f8f38003a00 con 0x7f8f4011cc80 2024-05-10T11:37:43.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:43.391+0000 7f8f3cff9700 1 --2- 172.21.15.100:0/1725856593 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8f2c0b2900 0x7f8f2c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:43.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:43.391+0000 7f8f3f7fe700 1 --2- 172.21.15.100:0/1725856593 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8f2c0b2900 0x7f8f2c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:43.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:43.391+0000 7f8f3cff9700 1 -- 172.21.15.100:0/1725856593 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f8f3800f020 con 0x7f8f4011cc80 2024-05-10T11:37:43.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:43.391+0000 7f8f3f7fe700 1 --2- 172.21.15.100:0/1725856593 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8f2c0b2900 0x7f8f2c0b4dc0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f8f30005950 tx=0x7f8f3000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:37:43.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:43.395+0000 7f8f3cff9700 1 -- 172.21.15.100:0/1725856593 <== mon.2 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f8f3802a020 con 0x7f8f4011cc80 2024-05-10T11:37:43.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:43.833+0000 7f8f4622e700 1 -- 172.21.15.100:0/1725856593 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8f20005620 con 0x7f8f4011cc80 2024-05-10T11:37:43.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:43.833+0000 7f8f3cff9700 1 -- 172.21.15.100:0/1725856593 <== mon.2 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+11897 (secure 0 0 0) 0x7f8f380a6020 con 0x7f8f4011cc80 2024-05-10T11:37:43.834 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:37:43.834 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2072225012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2072225012},{"type":"v1","addr":"172.21.15.100:6835","nonce":2072225012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/401939540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":401939540},{"type":"v1","addr":"172.21.15.100:6837","nonce":401939540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14571,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/3946311025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3946311025},{"type":"v1","addr":"172.21.15.100:6839","nonce":3946311025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3736371660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3736371660},{"type":"v1","addr":"172.21.15.100:6841","nonce":3736371660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/2741810791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2741810791},{"type":"v1","addr":"172.21.15.178:6833","nonce":2741810791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1704891041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1704891041},{"type":"v1","addr":"172.21.15.188:6833","nonce":1704891041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":41,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T11:14:27.242452+0000","modified":"2024-05-10T11:35:29.588072+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24439,"mds_1":24437,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/2124076312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2124076312},{"type":"v1","addr":"172.21.15.178:6835","nonce":2124076312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/3224358425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3224358425},{"type":"v1","addr":"172.21.15.178:6837","nonce":3224358425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/1943342620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1943342620},{"type":"v1","addr":"172.21.15.188:6837","nonce":1943342620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.178:6839/3760985404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3760985404},{"type":"v1","addr":"172.21.15.178:6839","nonce":3760985404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.188:6839/3194917070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":3194917070},{"type":"v1","addr":"172.21.15.188:6839","nonce":3194917070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27034":{"gid":27034,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/3139559450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3139559450},{"type":"v1","addr":"172.21.15.188:6835","nonce":3139559450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24439,"qdb_cluster":[24437,24439,24427]},"id":1}]} 2024-05-10T11:37:43.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:43.837+0000 7f8f4622e700 1 -- 172.21.15.100:0/1725856593 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8f2c0b2900 msgr2=0x7f8f2c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:43.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:43.837+0000 7f8f4622e700 1 --2- 172.21.15.100:0/1725856593 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8f2c0b2900 0x7f8f2c0b4dc0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f8f30005950 tx=0x7f8f3000b410 comp rx=0 tx=0).stop 2024-05-10T11:37:43.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:43.837+0000 7f8f4622e700 1 -- 172.21.15.100:0/1725856593 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f4011cc80 msgr2=0x7f8f401a8760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:43.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:43.837+0000 7f8f4622e700 1 --2- 172.21.15.100:0/1725856593 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f4011cc80 0x7f8f401a8760 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8f38006070 tx=0x7f8f38004c80 comp rx=0 tx=0).stop 2024-05-10T11:37:43.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:43.837+0000 7f8f4622e700 1 -- 172.21.15.100:0/1725856593 shutdown_connections 2024-05-10T11:37:43.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:43.837+0000 7f8f4622e700 1 --2- 172.21.15.100:0/1725856593 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8f2c0b2900 0x7f8f2c0b4dc0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:43.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:43.837+0000 7f8f4622e700 1 --2- 172.21.15.100:0/1725856593 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f4011cc80 0x7f8f401a8760 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:43.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:43.837+0000 7f8f4622e700 1 --2- 172.21.15.100:0/1725856593 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f4010a9e0 0x7f8f401ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:43.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:43.837+0000 7f8f4622e700 1 --2- 172.21.15.100:0/1725856593 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f40111c80 0x7f8f401aebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:43.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:43.837+0000 7f8f4622e700 1 -- 172.21.15.100:0/1725856593 >> 172.21.15.100:0/1725856593 conn(0x7f8f4006e610 msgr2=0x7f8f4011ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:37:43.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:43.837+0000 7f8f4622e700 1 -- 172.21.15.100:0/1725856593 shutdown_connections 2024-05-10T11:37:43.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:43.837+0000 7f8f4622e700 1 -- 172.21.15.100:0/1725856593 wait complete. 2024-05-10T11:37:43.838 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 41 2024-05-10T11:37:43.934 INFO:tasks.workunit.client.0.smithi100.stdout: 10 56707 9.08 MB/sec execute 217 sec latency 138.765 ms 2024-05-10T11:37:44.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:37:44 smithi178 ceph-mon[33359]: pgmap v1001: 129 pgs: 129 active+clean; 494 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 4.3 MiB/s wr, 260 op/s 2024-05-10T11:37:44.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:37:44 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:37:44.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:37:44 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:37:44.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:37:44 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:37:44.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:37:44 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:37:44.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:37:44 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:37:44.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:37:44 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:37:44.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:37:44 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/1725856593' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:37:44.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:37:44 smithi188 ceph-mon[34075]: pgmap v1001: 129 pgs: 129 active+clean; 494 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 4.3 MiB/s wr, 260 op/s 2024-05-10T11:37:44.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:37:44 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:37:44.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:37:44 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:37:44.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:37:44 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:37:44.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:37:44 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:37:44.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:37:44 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:37:44.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:37:44 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:37:44.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:37:44 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/1725856593' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:37:44.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:37:44 smithi100 ceph-mon[30296]: pgmap v1001: 129 pgs: 129 active+clean; 494 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 4.3 MiB/s wr, 260 op/s 2024-05-10T11:37:44.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:37:44 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:37:44.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:37:44 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:37:44.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:37:44 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:37:44.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:37:44 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:37:44.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:37:44 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:37:44.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:37:44 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:37:44.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:37:44 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/1725856593' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:37:44.661 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-10T11:37:44.934 INFO:tasks.workunit.client.0.smithi100.stdout: 10 56855 9.05 MB/sec execute 218 sec latency 122.466 ms 2024-05-10T11:37:45.029 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:37:45.934 INFO:tasks.workunit.client.0.smithi100.stdout: 10 56992 9.05 MB/sec execute 219 sec latency 130.876 ms 2024-05-10T11:37:46.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:37:46 smithi100 ceph-mon[30296]: pgmap v1002: 129 pgs: 129 active+clean; 493 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 909 KiB/s rd, 4.6 MiB/s wr, 265 op/s 2024-05-10T11:37:46.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:37:46 smithi178 ceph-mon[33359]: pgmap v1002: 129 pgs: 129 active+clean; 493 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 909 KiB/s rd, 4.6 MiB/s wr, 265 op/s 2024-05-10T11:37:46.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:37:46 smithi188 ceph-mon[34075]: pgmap v1002: 129 pgs: 129 active+clean; 493 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 909 KiB/s rd, 4.6 MiB/s wr, 265 op/s 2024-05-10T11:37:46.934 INFO:tasks.workunit.client.0.smithi100.stdout: 10 57143 9.03 MB/sec execute 220 sec latency 90.978 ms 2024-05-10T11:37:47.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:37:47 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:37:47.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:37:47 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:37:47.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:37:47 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:37:47.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:37:47 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:37:47.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:37:47 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:37:47.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:37:47 smithi100 ceph-mon[30296]: pgmap v1003: 129 pgs: 129 active+clean; 494 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 4.8 MiB/s wr, 282 op/s 2024-05-10T11:37:47.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:37:47 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:37:47.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:37:47 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:37:47.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:37:47 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:37:47.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:37:47 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:37:47.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:37:47 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:37:47.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:37:47 smithi178 ceph-mon[33359]: pgmap v1003: 129 pgs: 129 active+clean; 494 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 4.8 MiB/s wr, 282 op/s 2024-05-10T11:37:47.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:37:47 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:37:47.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:37:47 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:37:47.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:37:47 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:37:47.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:37:47 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:37:47.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:37:47 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:37:47.779 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:37:47 smithi188 ceph-mon[34075]: pgmap v1003: 129 pgs: 129 active+clean; 494 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 4.8 MiB/s wr, 282 op/s 2024-05-10T11:37:47.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.854+0000 7f5672d1f700 1 -- 172.21.15.100:0/3046606372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f566c1060a0 msgr2=0x7f566c1125b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:47.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.854+0000 7f5672d1f700 1 --2- 172.21.15.100:0/3046606372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f566c1060a0 0x7f566c1125b0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f566400b0a0 tx=0x7f566400b3b0 comp rx=0 tx=0).stop 2024-05-10T11:37:47.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.854+0000 7f5672d1f700 1 -- 172.21.15.100:0/3046606372 shutdown_connections 2024-05-10T11:37:47.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.854+0000 7f5672d1f700 1 --2- 172.21.15.100:0/3046606372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f566c1060a0 0x7f566c1125b0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:47.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.854+0000 7f5672d1f700 1 --2- 172.21.15.100:0/3046606372 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f566c10b6e0 0x7f566c10bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:47.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.854+0000 7f5672d1f700 1 --2- 172.21.15.100:0/3046606372 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f566c1056e0 0x7f566c105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:47.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.854+0000 7f5672d1f700 1 -- 172.21.15.100:0/3046606372 >> 172.21.15.100:0/3046606372 conn(0x7f566c1011d0 msgr2=0x7f566c1035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:37:47.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.854+0000 7f5672d1f700 1 -- 172.21.15.100:0/3046606372 shutdown_connections 2024-05-10T11:37:47.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.854+0000 7f5672d1f700 1 -- 172.21.15.100:0/3046606372 wait complete. 2024-05-10T11:37:47.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.855+0000 7f5672d1f700 1 Processor -- start 2024-05-10T11:37:47.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.855+0000 7f5672d1f700 1 -- start start 2024-05-10T11:37:47.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.855+0000 7f5672d1f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f566c1056e0 0x7f566c1a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:47.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.855+0000 7f5672d1f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f566c1060a0 0x7f566c1a1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:47.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.855+0000 7f5672d1f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f566c10b6e0 0x7f566c19ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:47.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.855+0000 7f5672d1f700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f566c1157d0 con 0x7f566c1056e0 2024-05-10T11:37:47.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.855+0000 7f5672d1f700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f566c115650 con 0x7f566c1060a0 2024-05-10T11:37:47.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.855+0000 7f5672d1f700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f566c115950 con 0x7f566c10b6e0 2024-05-10T11:37:47.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.856+0000 7f5670abb700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f566c1056e0 0x7f566c1a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:47.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.856+0000 7f566bfff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f566c1060a0 0x7f566c1a1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:47.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.856+0000 7f56712bc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f566c10b6e0 0x7f566c19ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:47.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.856+0000 7f566bfff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f566c1060a0 0x7f566c1a1df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:35154/0 (socket says 172.21.15.100:35154) 2024-05-10T11:37:47.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.856+0000 7f566bfff700 1 -- 172.21.15.100:0/3744021367 learned_addr learned my addr 172.21.15.100:0/3744021367 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:37:47.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.856+0000 7f56712bc700 1 --2- 172.21.15.100:0/3744021367 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f566c10b6e0 0x7f566c19ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:45638/0 (socket says 172.21.15.100:45638) 2024-05-10T11:37:47.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.856+0000 7f566bfff700 1 -- 172.21.15.100:0/3744021367 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f566c10b6e0 msgr2=0x7f566c19ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:47.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.856+0000 7f566bfff700 1 --2- 172.21.15.100:0/3744021367 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f566c10b6e0 0x7f566c19ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:47.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.856+0000 7f566bfff700 1 -- 172.21.15.100:0/3744021367 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f566c1056e0 msgr2=0x7f566c1a18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:47.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.856+0000 7f566bfff700 1 --2- 172.21.15.100:0/3744021367 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f566c1056e0 0x7f566c1a18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:47.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.856+0000 7f566bfff700 1 -- 172.21.15.100:0/3744021367 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f566c19c1d0 con 0x7f566c1060a0 2024-05-10T11:37:47.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.856+0000 7f5670abb700 1 --2- 172.21.15.100:0/3744021367 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f566c1056e0 0x7f566c1a18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:37:47.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.857+0000 7f566bfff700 1 --2- 172.21.15.100:0/3744021367 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f566c1060a0 0x7f566c1a1df0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f565800e9a0 tx=0x7f565800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:37:47.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.857+0000 7f5669ffb700 1 -- 172.21.15.100:0/3744021367 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565800cbd0 con 0x7f566c1060a0 2024-05-10T11:37:47.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.857+0000 7f5669ffb700 1 -- 172.21.15.100:0/3744021367 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f565800cd70 con 0x7f566c1060a0 2024-05-10T11:37:47.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.857+0000 7f5669ffb700 1 -- 172.21.15.100:0/3744021367 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5658018680 con 0x7f566c1060a0 2024-05-10T11:37:47.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.857+0000 7f5672d1f700 1 -- 172.21.15.100:0/3744021367 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f566c19c460 con 0x7f566c1060a0 2024-05-10T11:37:47.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.857+0000 7f5672d1f700 1 -- 172.21.15.100:0/3744021367 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f566c1a8780 con 0x7f566c1060a0 2024-05-10T11:37:47.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.861+0000 7f5669ffb700 1 -- 172.21.15.100:0/3744021367 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f5658010430 con 0x7f566c1060a0 2024-05-10T11:37:47.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.862+0000 7f5669ffb700 1 --2- 172.21.15.100:0/3744021367 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f565c0b29d0 0x7f565c0b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:47.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.862+0000 7f5670abb700 1 --2- 172.21.15.100:0/3744021367 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f565c0b29d0 0x7f565c0b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:47.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.862+0000 7f5669ffb700 1 -- 172.21.15.100:0/3744021367 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f56580e26b0 con 0x7f566c1060a0 2024-05-10T11:37:47.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.863+0000 7f567d5b2b80 1 Processor -- start 2024-05-10T11:37:47.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.863+0000 7f5670abb700 1 --2- 172.21.15.100:0/3744021367 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f565c0b29d0 0x7f565c0b4e90 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f566c19cbb0 tx=0x7f5660009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:37:47.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.863+0000 7f567d5b2b80 1 -- start start 2024-05-10T11:37:47.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.863+0000 7f567d5b2b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558712a0f1b0 0x558712a0f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:47.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.863+0000 7f567d5b2b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558712a0fad0 0x558712a09810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:47.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.863+0000 7f567d5b2b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558712a09d50 0x558712a0a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:47.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.864+0000 7f567d5b2b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55871295b940 con 0x558712a0fad0 2024-05-10T11:37:47.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.864+0000 7f567d5b2b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5587128c0da0 con 0x558712a09d50 2024-05-10T11:37:47.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.864+0000 7f567d5b2b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558712957900 con 0x558712a0f1b0 2024-05-10T11:37:47.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.864+0000 7f5670abb700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558712a0f1b0 0x558712a0f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:47.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.864+0000 7f56712bc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558712a09d50 0x558712a0a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:47.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.864+0000 7f56712bc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558712a09d50 0x558712a0a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:35160/0 (socket says 172.21.15.100:35160) 2024-05-10T11:37:47.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.864+0000 7f5670abb700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558712a0f1b0 0x558712a0f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:45646/0 (socket says 172.21.15.100:45646) 2024-05-10T11:37:47.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.864+0000 7f5670abb700 1 -- 172.21.15.100:0/2278055708 learned_addr learned my addr 172.21.15.100:0/2278055708 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:37:47.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.864+0000 7f566bfff700 1 --2- 172.21.15.100:0/2278055708 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558712a0fad0 0x558712a09810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:47.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.864+0000 7f56712bc700 1 -- 172.21.15.100:0/2278055708 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558712a0f1b0 msgr2=0x558712a0f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:37:47.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.864+0000 7f56712bc700 1 --2- 172.21.15.100:0/2278055708 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558712a0f1b0 0x558712a0f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:47.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.864+0000 7f56712bc700 1 -- 172.21.15.100:0/2278055708 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558712a0fad0 msgr2=0x558712a09810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:37:47.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.864+0000 7f56712bc700 1 --2- 172.21.15.100:0/2278055708 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558712a0fad0 0x558712a09810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:47.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.864+0000 7f56712bc700 1 -- 172.21.15.100:0/2278055708 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587128a88d0 con 0x558712a09d50 2024-05-10T11:37:47.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.864+0000 7f56712bc700 1 --2- 172.21.15.100:0/2278055708 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558712a09d50 0x558712a0a190 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f566400b070 tx=0x7f5664009df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72213549fbe002b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:37:47.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.864+0000 7f5656ffd700 1 -- 172.21.15.100:0/2278055708 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5664020070 con 0x558712a09d50 2024-05-10T11:37:47.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.865+0000 7f5656ffd700 1 -- 172.21.15.100:0/2278055708 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5664009460 con 0x558712a09d50 2024-05-10T11:37:47.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.865+0000 7f5656ffd700 1 -- 172.21.15.100:0/2278055708 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5664007aa0 con 0x558712a09d50 2024-05-10T11:37:47.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.865+0000 7f567d5b2b80 1 -- 172.21.15.100:0/2278055708 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558712a09d50 msgr2=0x558712a0a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:47.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.865+0000 7f567d5b2b80 1 --2- 172.21.15.100:0/2278055708 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558712a09d50 0x558712a0a190 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f566400b070 tx=0x7f5664009df0 comp rx=0 tx=0).stop 2024-05-10T11:37:47.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.865+0000 7f567d5b2b80 1 -- 172.21.15.100:0/2278055708 shutdown_connections 2024-05-10T11:37:47.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.866+0000 7f567d5b2b80 1 --2- 172.21.15.100:0/2278055708 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558712a09d50 0x558712a0a190 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:47.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.866+0000 7f567d5b2b80 1 --2- 172.21.15.100:0/2278055708 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558712a0f1b0 0x558712a0f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:47.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.866+0000 7f567d5b2b80 1 --2- 172.21.15.100:0/2278055708 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558712a0fad0 0x558712a09810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:47.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.866+0000 7f567d5b2b80 1 -- 172.21.15.100:0/2278055708 >> 172.21.15.100:0/2278055708 conn(0x558712935b70 msgr2=0x558712a042a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:37:47.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.866+0000 7f567d5b2b80 1 -- 172.21.15.100:0/2278055708 shutdown_connections 2024-05-10T11:37:47.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.866+0000 7f567d5b2b80 1 -- 172.21.15.100:0/2278055708 wait complete. 2024-05-10T11:37:47.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.866+0000 7f567d5b2b80 1 -- start start 2024-05-10T11:37:47.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.867+0000 7f567d5b2b80 1 Processor -- start 2024-05-10T11:37:47.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.867+0000 7f567d5b2b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558712a09d50 0x558712a15770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:47.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.867+0000 7f567d5b2b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558712a0f1b0 0x558712a12b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:47.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.867+0000 7f567d5b2b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558712a0fad0 0x558712a13050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:47.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.867+0000 7f567d5b2b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x558712a16490 con 0x558712a09d50 2024-05-10T11:37:47.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.867+0000 7f567d5b2b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558712914ed0 con 0x558712a0fad0 2024-05-10T11:37:47.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.867+0000 7f567d5b2b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558712a16610 con 0x558712a0f1b0 2024-05-10T11:37:47.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.867+0000 7f567d5b2b80 10 client.0 _ll_register_callbacks cb 0x5587128ef5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:37:47.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.867+0000 7f566bfff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558712a0f1b0 0x558712a12b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:47.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.867+0000 7f566bfff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558712a0f1b0 0x558712a12b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:45660/0 (socket says 172.21.15.100:45660) 2024-05-10T11:37:47.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.867+0000 7f566bfff700 1 -- 172.21.15.100:0/3624829140 learned_addr learned my addr 172.21.15.100:0/3624829140 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:37:47.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.867+0000 7f56712bc700 1 --2- 172.21.15.100:0/3624829140 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558712a0fad0 0x558712a13050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:47.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.868+0000 7f5670abb700 1 --2- 172.21.15.100:0/3624829140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558712a09d50 0x558712a15770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:47.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.868+0000 7f56712bc700 1 -- 172.21.15.100:0/3624829140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558712a0f1b0 msgr2=0x558712a12b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:37:47.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.868+0000 7f56712bc700 1 --2- 172.21.15.100:0/3624829140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558712a0f1b0 0x558712a12b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:47.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.868+0000 7f56712bc700 1 -- 172.21.15.100:0/3624829140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558712a09d50 msgr2=0x558712a15770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:37:47.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.868+0000 7f56712bc700 1 --2- 172.21.15.100:0/3624829140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558712a09d50 0x558712a15770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:47.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.868+0000 7f56712bc700 1 -- 172.21.15.100:0/3624829140 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558712912760 con 0x558712a0fad0 2024-05-10T11:37:47.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.868+0000 7f5670abb700 1 --2- 172.21.15.100:0/3624829140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558712a09d50 0x558712a15770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:37:47.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.868+0000 7f56712bc700 1 --2- 172.21.15.100:0/3624829140 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558712a0fad0 0x558712a13050 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5664009610 tx=0x7f5664003c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55bb9a86484edceb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:37:47.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.868+0000 7f564bfff700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:37:47.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.868+0000 7f564bfff700 1 -- 172.21.15.100:0/3624829140 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5664020040 con 0x558712a0fad0 2024-05-10T11:37:47.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.868+0000 7f564bfff700 1 -- 172.21.15.100:0/3624829140 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f56640078b0 con 0x558712a0fad0 2024-05-10T11:37:47.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.868+0000 7f567d5b2b80 1 -- 172.21.15.100:0/3624829140 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5664009460 con 0x558712a0fad0 2024-05-10T11:37:47.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.869+0000 7f564bfff700 1 -- 172.21.15.100:0/3624829140 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f566400cea0 con 0x558712a0fad0 2024-05-10T11:37:47.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.869+0000 7f567d5b2b80 10 client.28490 fetch_fsmap learned FSMap version 41 2024-05-10T11:37:47.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.869+0000 7f567d5b2b80 1 -- 172.21.15.100:0/3624829140 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x558712a29580 con 0x558712a0fad0 2024-05-10T11:37:47.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.869+0000 7f564bfff700 1 -- 172.21.15.100:0/3624829140 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5664028070 con 0x558712a0fad0 2024-05-10T11:37:47.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.869+0000 7f564bfff700 1 -- 172.21.15.100:0/3624829140 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f566402d070 con 0x558712a0fad0 2024-05-10T11:37:47.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.869+0000 7f567d5b2b80 10 client.28490 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:37:47.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.869+0000 7f567d5b2b80 10 client.28490 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-10T11:37:47.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.869+0000 7f567d5b2b80 20 client.28490 populate_metadata read hostname 'smithi100' 2024-05-10T11:37:47.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.869+0000 7f567d5b2b80 1 --2- 172.21.15.100:0/3624829140 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x558712a2ba50 0x558712a2de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:47.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.869+0000 7f567d5b2b80 4 client.28490 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:37:47.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.869+0000 7f567d5b2b80 1 -- 172.21.15.100:0/3624829140 --> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5664009460 con 0x558712a2ba50 2024-05-10T11:37:47.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.869+0000 7f5670abb700 1 --2- 172.21.15.100:0/3624829140 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x558712a2ba50 0x558712a2de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:47.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.870+0000 7f5670abb700 1 --2- 172.21.15.100:0/3624829140 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x558712a2ba50 0x558712a2de40 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6966e1c59d4707f3 server_cookie=400965d351d3b046 in_seq=0 out_seq=0 2024-05-10T11:37:47.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.870+0000 7f564bfff700 10 client.28490 ms_handle_connect on v2:172.21.15.188:6836/1943342620 2024-05-10T11:37:47.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.871+0000 7f564bfff700 1 -- 172.21.15.100:0/3624829140 <== mds.2 v2:172.21.15.188:6836/1943342620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f5664009460 con 0x558712a2ba50 2024-05-10T11:37:47.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.871+0000 7f564bfff700 10 client.28490 handle_command_reply: tid=0 2024-05-10T11:37:47.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.871+0000 7f567d5b2b80 1 client.28490 shutdown 2024-05-10T11:37:47.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.871+0000 7f567d5b2b80 1 -- 172.21.15.100:0/3624829140 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558712a0fad0 msgr2=0x558712a13050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:47.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.871+0000 7f567d5b2b80 1 --2- 172.21.15.100:0/3624829140 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558712a0fad0 0x558712a13050 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5664009610 tx=0x7f5664003c30 comp rx=0 tx=0).stop 2024-05-10T11:37:47.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.871+0000 7f567d5b2b80 1 -- 172.21.15.100:0/3624829140 shutdown_connections 2024-05-10T11:37:47.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.871+0000 7f567d5b2b80 1 --2- 172.21.15.100:0/3624829140 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x558712a2ba50 0x558712a2de40 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:47.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.871+0000 7f567d5b2b80 1 --2- 172.21.15.100:0/3624829140 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558712a0fad0 0x558712a13050 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:47.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.871+0000 7f567d5b2b80 1 --2- 172.21.15.100:0/3624829140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558712a09d50 0x558712a15770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:47.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.871+0000 7f564bfff700 0 client.28490 ms_handle_reset on v2:172.21.15.188:6836/1943342620 2024-05-10T11:37:47.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.871+0000 7f567d5b2b80 1 --2- 172.21.15.100:0/3624829140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558712a0f1b0 0x558712a12b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:47.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.871+0000 7f567d5b2b80 1 -- 172.21.15.100:0/3624829140 >> 172.21.15.100:0/3624829140 conn(0x558712935b70 msgr2=0x5587129feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:37:47.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.871+0000 7f567d5b2b80 1 -- 172.21.15.100:0/3624829140 shutdown_connections 2024-05-10T11:37:47.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.871+0000 7f567d5b2b80 1 -- 172.21.15.100:0/3624829140 wait complete. 2024-05-10T11:37:47.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.871+0000 7f567d5b2b80 20 client.28490 trim_cache size 0 max 16384 2024-05-10T11:37:47.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.896+0000 7f567d5b2b80 1 Processor -- start 2024-05-10T11:37:47.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.896+0000 7f567d5b2b80 1 -- start start 2024-05-10T11:37:47.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.896+0000 7f567d5b2b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558712a09d50 0x558712a09810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:47.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.896+0000 7f567d5b2b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558712a0f1b0 0x558712a10740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:47.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.897+0000 7f567d5b2b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558712a0fad0 0x558712a14fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:47.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.897+0000 7f567d5b2b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f566c1af680 con 0x558712a0fad0 2024-05-10T11:37:47.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.897+0000 7f567d5b2b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f566c1acf10 con 0x558712a0f1b0 2024-05-10T11:37:47.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.897+0000 7f567d5b2b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f566c1aad80 con 0x558712a09d50 2024-05-10T11:37:47.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.897+0000 7f566bfff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558712a0f1b0 0x558712a10740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:47.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.897+0000 7f566bfff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558712a0f1b0 0x558712a10740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:35188/0 (socket says 172.21.15.100:35188) 2024-05-10T11:37:47.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.897+0000 7f566bfff700 1 -- 172.21.15.100:0/1975656321 learned_addr learned my addr 172.21.15.100:0/1975656321 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:37:47.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.897+0000 7f56712bc700 1 --2- 172.21.15.100:0/1975656321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558712a0fad0 0x558712a14fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:47.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.897+0000 7f56712bc700 1 -- 172.21.15.100:0/1975656321 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558712a09d50 msgr2=0x558712a09810 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T11:37:47.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.897+0000 7f56712bc700 1 --2- 172.21.15.100:0/1975656321 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558712a09d50 0x558712a09810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:47.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.897+0000 7f56712bc700 1 -- 172.21.15.100:0/1975656321 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558712a0f1b0 msgr2=0x558712a10740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:37:47.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.897+0000 7f56712bc700 1 --2- 172.21.15.100:0/1975656321 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558712a0f1b0 0x558712a10740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:47.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.897+0000 7f56712bc700 1 -- 172.21.15.100:0/1975656321 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558712a11040 con 0x558712a0fad0 2024-05-10T11:37:47.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.898+0000 7f56712bc700 1 --2- 172.21.15.100:0/1975656321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558712a0fad0 0x558712a14fb0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f5664005970 tx=0x7f56640099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9e01411650d8d69 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:37:47.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.900+0000 7f56567fc700 1 -- 172.21.15.100:0/1975656321 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5664020040 con 0x558712a0fad0 2024-05-10T11:37:47.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.900+0000 7f56567fc700 1 -- 172.21.15.100:0/1975656321 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5664028070 con 0x558712a0fad0 2024-05-10T11:37:47.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.901+0000 7f56567fc700 1 -- 172.21.15.100:0/1975656321 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5664005ea0 con 0x558712a0fad0 2024-05-10T11:37:47.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.902+0000 7f567d5b2b80 1 -- 172.21.15.100:0/1975656321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558712a0fad0 msgr2=0x558712a14fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:47.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.902+0000 7f567d5b2b80 1 --2- 172.21.15.100:0/1975656321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558712a0fad0 0x558712a14fb0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f5664005970 tx=0x7f56640099e0 comp rx=0 tx=0).stop 2024-05-10T11:37:47.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.908+0000 7f567d5b2b80 1 -- 172.21.15.100:0/1975656321 shutdown_connections 2024-05-10T11:37:47.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.908+0000 7f567d5b2b80 1 --2- 172.21.15.100:0/1975656321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558712a0fad0 0x558712a14fb0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:47.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.908+0000 7f567d5b2b80 1 --2- 172.21.15.100:0/1975656321 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558712a09d50 0x558712a09810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:47.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.908+0000 7f567d5b2b80 1 --2- 172.21.15.100:0/1975656321 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558712a0f1b0 0x558712a10740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:47.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.908+0000 7f567d5b2b80 1 -- 172.21.15.100:0/1975656321 >> 172.21.15.100:0/1975656321 conn(0x558712935b70 msgr2=0x558712a11400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:37:47.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.908+0000 7f567d5b2b80 1 -- 172.21.15.100:0/1975656321 shutdown_connections 2024-05-10T11:37:47.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.908+0000 7f567d5b2b80 1 -- 172.21.15.100:0/1975656321 wait complete. 2024-05-10T11:37:47.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.909+0000 7f567d5b2b80 1 -- start start 2024-05-10T11:37:47.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.909+0000 7f567d5b2b80 1 Processor -- start 2024-05-10T11:37:47.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.910+0000 7f567d5b2b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558712a09d50 0x558712a2e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:47.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.910+0000 7f567d5b2b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558712a0f1b0 0x558712a2ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:47.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.910+0000 7f567d5b2b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558712a0fad0 0x558712a2f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:47.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.910+0000 7f567d5b2b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f566c1ab3d0 con 0x558712a0f1b0 2024-05-10T11:37:47.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.910+0000 7f567d5b2b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f566c1ac840 con 0x558712a09d50 2024-05-10T11:37:47.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.910+0000 7f56712bc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558712a0fad0 0x558712a2f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:47.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.910+0000 7f56712bc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558712a0fad0 0x558712a2f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:45678/0 (socket says 172.21.15.100:45678) 2024-05-10T11:37:47.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.910+0000 7f56712bc700 1 -- 172.21.15.100:0/3573843259 learned_addr learned my addr 172.21.15.100:0/3573843259 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:37:47.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.910+0000 7f567d5b2b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f566c1abaa0 con 0x558712a0fad0 2024-05-10T11:37:47.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.911+0000 7f567d5b2b80 10 client.0 _ll_register_callbacks cb 0x5587128ef5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:37:47.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.911+0000 7f566bfff700 1 --2- 172.21.15.100:0/3573843259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558712a0f1b0 0x558712a2ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:47.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.911+0000 7f56712bc700 1 -- 172.21.15.100:0/3573843259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558712a09d50 msgr2=0x558712a2e950 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T11:37:47.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.911+0000 7f56712bc700 1 --2- 172.21.15.100:0/3573843259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558712a09d50 0x558712a2e950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:47.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.911+0000 7f56712bc700 1 -- 172.21.15.100:0/3573843259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558712a0f1b0 msgr2=0x558712a2ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:37:47.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.911+0000 7f56712bc700 1 --2- 172.21.15.100:0/3573843259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558712a0f1b0 0x558712a2ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:47.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.911+0000 7f56712bc700 1 -- 172.21.15.100:0/3573843259 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558712a94f70 con 0x558712a0fad0 2024-05-10T11:37:47.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.912+0000 7f56712bc700 1 --2- 172.21.15.100:0/3573843259 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558712a0fad0 0x558712a2f3d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5664009090 tx=0x7f5664009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b7355b73435746b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:37:47.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.912+0000 7f564bfff700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:37:47.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.912+0000 7f564bfff700 1 -- 172.21.15.100:0/3573843259 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5664020040 con 0x558712a0fad0 2024-05-10T11:37:47.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.912+0000 7f567d5b2b80 1 -- 172.21.15.100:0/3573843259 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5664028070 con 0x558712a0fad0 2024-05-10T11:37:47.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.912+0000 7f564bfff700 1 -- 172.21.15.100:0/3573843259 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f56640052d0 con 0x558712a0fad0 2024-05-10T11:37:47.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.913+0000 7f564bfff700 1 -- 172.21.15.100:0/3573843259 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f56640054d0 con 0x558712a0fad0 2024-05-10T11:37:47.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.913+0000 7f567d5b2b80 10 client.28600 fetch_fsmap learned FSMap version 41 2024-05-10T11:37:47.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.913+0000 7f567d5b2b80 1 -- 172.21.15.100:0/3573843259 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x558712a961b0 con 0x558712a0fad0 2024-05-10T11:37:47.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.914+0000 7f564bfff700 1 -- 172.21.15.100:0/3573843259 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5664028070 con 0x558712a0fad0 2024-05-10T11:37:47.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.914+0000 7f564bfff700 1 -- 172.21.15.100:0/3573843259 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f566402d040 con 0x558712a0fad0 2024-05-10T11:37:47.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.914+0000 7f567d5b2b80 10 client.28600 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:37:47.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.914+0000 7f567d5b2b80 10 client.28600 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-10T11:37:47.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.914+0000 7f567d5b2b80 20 client.28600 populate_metadata read hostname 'smithi100' 2024-05-10T11:37:47.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.914+0000 7f567d5b2b80 1 --2- 172.21.15.100:0/3573843259 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x558712a96570 0x558712a98960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:47.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.914+0000 7f567d5b2b80 4 client.28600 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:37:47.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.914+0000 7f567d5b2b80 1 -- 172.21.15.100:0/3573843259 --> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558712969320 con 0x558712a96570 2024-05-10T11:37:47.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.914+0000 7f5670abb700 1 --2- 172.21.15.100:0/3573843259 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x558712a96570 0x558712a98960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:47.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.915+0000 7f564bfff700 10 client.28600 ms_handle_connect on v2:172.21.15.188:6836/1943342620 2024-05-10T11:37:47.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.915+0000 7f5670abb700 1 --2- 172.21.15.100:0/3573843259 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x558712a96570 0x558712a98960 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6d842b46ca5b901b server_cookie=b4085dd0526f95a1 in_seq=0 out_seq=0 2024-05-10T11:37:47.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.916+0000 7f564bfff700 1 -- 172.21.15.100:0/3573843259 <== mds.2 v2:172.21.15.188:6836/1943342620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558712969320 con 0x558712a96570 2024-05-10T11:37:47.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.916+0000 7f564bfff700 10 client.28600 handle_command_reply: tid=0 2024-05-10T11:37:47.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.916+0000 7f567d5b2b80 1 client.28600 shutdown 2024-05-10T11:37:47.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.916+0000 7f567d5b2b80 1 -- 172.21.15.100:0/3573843259 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558712a0fad0 msgr2=0x558712a2f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:47.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.916+0000 7f567d5b2b80 1 --2- 172.21.15.100:0/3573843259 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558712a0fad0 0x558712a2f3d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5664009090 tx=0x7f5664009b80 comp rx=0 tx=0).stop 2024-05-10T11:37:47.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.916+0000 7f567d5b2b80 1 -- 172.21.15.100:0/3573843259 shutdown_connections 2024-05-10T11:37:47.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.916+0000 7f567d5b2b80 1 --2- 172.21.15.100:0/3573843259 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x558712a96570 0x558712a98960 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:47.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.916+0000 7f567d5b2b80 1 --2- 172.21.15.100:0/3573843259 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558712a0fad0 0x558712a2f3d0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:47.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.916+0000 7f567d5b2b80 1 --2- 172.21.15.100:0/3573843259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558712a09d50 0x558712a2e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:47.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.916+0000 7f567d5b2b80 1 --2- 172.21.15.100:0/3573843259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558712a0f1b0 0x558712a2ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:47.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.916+0000 7f567d5b2b80 1 -- 172.21.15.100:0/3573843259 >> 172.21.15.100:0/3573843259 conn(0x558712935b70 msgr2=0x558712a0e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:37:47.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.916+0000 7f564bfff700 0 client.28600 ms_handle_reset on v2:172.21.15.188:6836/1943342620 2024-05-10T11:37:47.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.916+0000 7f567d5b2b80 1 -- 172.21.15.100:0/3573843259 shutdown_connections 2024-05-10T11:37:47.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.916+0000 7f567d5b2b80 1 -- 172.21.15.100:0/3573843259 wait complete. 2024-05-10T11:37:47.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.916+0000 7f567d5b2b80 20 client.28600 trim_cache size 0 max 16384 2024-05-10T11:37:47.917 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:37:47.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.917+0000 7f5672d1f700 1 -- 172.21.15.100:0/3744021367 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f565c0b29d0 msgr2=0x7f565c0b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:47.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.917+0000 7f5672d1f700 1 --2- 172.21.15.100:0/3744021367 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f565c0b29d0 0x7f565c0b4e90 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f566c19cbb0 tx=0x7f5660009260 comp rx=0 tx=0).stop 2024-05-10T11:37:47.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.917+0000 7f5672d1f700 1 -- 172.21.15.100:0/3744021367 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f566c1060a0 msgr2=0x7f566c1a1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:47.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.917+0000 7f5672d1f700 1 --2- 172.21.15.100:0/3744021367 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f566c1060a0 0x7f566c1a1df0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f565800e9a0 tx=0x7f565800ecb0 comp rx=0 tx=0).stop 2024-05-10T11:37:47.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.917+0000 7f5672d1f700 1 -- 172.21.15.100:0/3744021367 shutdown_connections 2024-05-10T11:37:47.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.917+0000 7f5672d1f700 1 --2- 172.21.15.100:0/3744021367 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f565c0b29d0 0x7f565c0b4e90 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:47.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.917+0000 7f5672d1f700 1 --2- 172.21.15.100:0/3744021367 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f566c10b6e0 0x7f566c19ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:47.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.917+0000 7f5672d1f700 1 --2- 172.21.15.100:0/3744021367 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f566c1056e0 0x7f566c1a18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:47.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.917+0000 7f5672d1f700 1 --2- 172.21.15.100:0/3744021367 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f566c1060a0 0x7f566c1a1df0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:47.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.917+0000 7f5672d1f700 1 -- 172.21.15.100:0/3744021367 >> 172.21.15.100:0/3744021367 conn(0x7f566c1011d0 msgr2=0x7f566c101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:37:47.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.917+0000 7f5672d1f700 1 -- 172.21.15.100:0/3744021367 shutdown_connections 2024-05-10T11:37:47.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:47.917+0000 7f5672d1f700 1 -- 172.21.15.100:0/3744021367 wait complete. 2024-05-10T11:37:47.934 INFO:tasks.workunit.client.0.smithi100.stdout: 10 57319 9.02 MB/sec execute 221 sec latency 89.033 ms 2024-05-10T11:37:48.638 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.638+0000 7ff829245640 1 -- 172.21.15.100:0/3191536073 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8241012e0 msgr2=0x7ff824101700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:48.638 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.638+0000 7ff829245640 1 --2- 172.21.15.100:0/3191536073 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8241012e0 0x7ff824101700 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7ff814009900 tx=0x7ff81402a410 comp rx=0 tx=0).stop 2024-05-10T11:37:48.638 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.638+0000 7ff829245640 1 -- 172.21.15.100:0/3191536073 shutdown_connections 2024-05-10T11:37:48.638 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.638+0000 7ff829245640 1 --2- 172.21.15.100:0/3191536073 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8241025e0 0x7ff824102a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:48.639 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.638+0000 7ff829245640 1 --2- 172.21.15.100:0/3191536073 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff824101c40 0x7ff8241020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:48.639 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.638+0000 7ff829245640 1 --2- 172.21.15.100:0/3191536073 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8241012e0 0x7ff824101700 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:48.639 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.638+0000 7ff829245640 1 -- 172.21.15.100:0/3191536073 >> 172.21.15.100:0/3191536073 conn(0x7ff8240fce80 msgr2=0x7ff8240ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:37:48.639 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.638+0000 7ff829245640 1 -- 172.21.15.100:0/3191536073 shutdown_connections 2024-05-10T11:37:48.639 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.638+0000 7ff829245640 1 -- 172.21.15.100:0/3191536073 wait complete. 2024-05-10T11:37:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.639+0000 7ff829245640 1 Processor -- start 2024-05-10T11:37:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.639+0000 7ff829245640 1 -- start start 2024-05-10T11:37:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.639+0000 7ff829245640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8241012e0 0x7ff82419ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.639+0000 7ff829245640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff824101c40 0x7ff82419b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.639+0000 7ff829245640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8241025e0 0x7ff8241a2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.639+0000 7ff829245640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff8241105d0 con 0x7ff8241025e0 2024-05-10T11:37:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.639+0000 7ff829245640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff824110450 con 0x7ff8241012e0 2024-05-10T11:37:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.639+0000 7ff829245640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff824110750 con 0x7ff824101c40 2024-05-10T11:37:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.639+0000 7ff82359e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8241025e0 0x7ff8241a2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.639+0000 7ff822d9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8241012e0 0x7ff82419ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.639+0000 7ff82359e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8241025e0 0x7ff8241a2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58044/0 (socket says 172.21.15.100:58044) 2024-05-10T11:37:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.639+0000 7ff81bfff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff824101c40 0x7ff82419b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.639+0000 7ff82359e640 1 -- 172.21.15.100:0/2534423404 learned_addr learned my addr 172.21.15.100:0/2534423404 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:37:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.640+0000 7ff82359e640 1 -- 172.21.15.100:0/2534423404 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff824101c40 msgr2=0x7ff82419b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.640+0000 7ff82359e640 1 --2- 172.21.15.100:0/2534423404 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff824101c40 0x7ff82419b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.640+0000 7ff82359e640 1 -- 172.21.15.100:0/2534423404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8241012e0 msgr2=0x7ff82419ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.640+0000 7ff82359e640 1 --2- 172.21.15.100:0/2534423404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8241012e0 0x7ff82419ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.640+0000 7ff82359e640 1 -- 172.21.15.100:0/2534423404 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8241a2910 con 0x7ff8241025e0 2024-05-10T11:37:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.640+0000 7ff822d9d640 1 --2- 172.21.15.100:0/2534423404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8241012e0 0x7ff82419ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:37:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.640+0000 7ff82359e640 1 --2- 172.21.15.100:0/2534423404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8241025e0 0x7ff8241a2270 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7ff81c00eb10 tx=0x7ff81c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:37:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.640+0000 7ff820d99640 1 -- 172.21.15.100:0/2534423404 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81c004b20 con 0x7ff8241025e0 2024-05-10T11:37:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.640+0000 7ff820d99640 1 -- 172.21.15.100:0/2534423404 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff81c004cc0 con 0x7ff8241025e0 2024-05-10T11:37:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.640+0000 7ff829245640 1 -- 172.21.15.100:0/2534423404 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8241a2c00 con 0x7ff8241025e0 2024-05-10T11:37:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.640+0000 7ff820d99640 1 -- 172.21.15.100:0/2534423404 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81c010690 con 0x7ff8241025e0 2024-05-10T11:37:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.640+0000 7ff829245640 1 -- 172.21.15.100:0/2534423404 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8241a3150 con 0x7ff8241025e0 2024-05-10T11:37:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.643+0000 7ff820d99640 1 -- 172.21.15.100:0/2534423404 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7ff81c005020 con 0x7ff8241025e0 2024-05-10T11:37:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.643+0000 7ff82c038740 1 Processor -- start 2024-05-10T11:37:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.644+0000 7ff82c038740 1 -- start start 2024-05-10T11:37:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.644+0000 7ff82c038740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56307a533db0 0x56307a5341d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.644+0000 7ff82c038740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56307a5347a0 0x56307a5392a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.644+0000 7ff82c038740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56307a539910 0x56307a53bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.644+0000 7ff82c038740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56307a4985d0 con 0x56307a533db0 2024-05-10T11:37:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.644+0000 7ff82c038740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56307a451890 con 0x56307a5347a0 2024-05-10T11:37:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.644+0000 7ff82c038740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56307a494590 con 0x56307a539910 2024-05-10T11:37:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.644+0000 7ff81bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56307a533db0 0x56307a5341d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:48.646 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.644+0000 7ff81bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56307a533db0 0x56307a5341d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58048/0 (socket says 172.21.15.100:58048) 2024-05-10T11:37:48.646 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.644+0000 7ff81bfff640 1 -- 172.21.15.100:0/1657230325 learned_addr learned my addr 172.21.15.100:0/1657230325 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:37:48.646 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.644+0000 7ff822d9d640 1 --2- 172.21.15.100:0/1657230325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56307a5347a0 0x56307a5392a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:48.646 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.644+0000 7ff81bfff640 1 --2- 172.21.15.100:0/1657230325 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56307a539910 0x56307a53bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:48.646 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.644+0000 7ff820d99640 1 --2- 172.21.15.100:0/2534423404 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff8040b28a0 0x7ff8040b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:48.646 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.644+0000 7ff82359e640 1 --2- 172.21.15.100:0/2534423404 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff8040b28a0 0x7ff8040b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:48.646 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.644+0000 7ff81bfff640 1 -- 172.21.15.100:0/1657230325 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56307a539910 msgr2=0x56307a53bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:37:48.646 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.644+0000 7ff81bfff640 1 --2- 172.21.15.100:0/1657230325 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56307a539910 0x56307a53bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:48.646 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.644+0000 7ff81bfff640 1 -- 172.21.15.100:0/1657230325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56307a5347a0 msgr2=0x56307a5392a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:37:48.646 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.644+0000 7ff81bfff640 1 --2- 172.21.15.100:0/1657230325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56307a5347a0 0x56307a5392a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:48.646 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.644+0000 7ff81bfff640 1 -- 172.21.15.100:0/1657230325 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56307a1aa210 con 0x56307a533db0 2024-05-10T11:37:48.646 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.645+0000 7ff820d99640 1 -- 172.21.15.100:0/2534423404 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7ff81c0e2b60 con 0x7ff8241025e0 2024-05-10T11:37:48.646 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.645+0000 7ff81bfff640 1 --2- 172.21.15.100:0/1657230325 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56307a533db0 0x56307a5341d0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7ff80c013850 tx=0x7ff80c0097c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7659b5b31deae062 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:37:48.646 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.645+0000 7ff82359e640 1 --2- 172.21.15.100:0/2534423404 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff8040b28a0 0x7ff8040b4d60 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7ff81c0a5e10 tx=0x7ff81c0a5c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:37:48.646 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.645+0000 7ff8197fa640 1 -- 172.21.15.100:0/1657230325 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff80c013dd0 con 0x56307a533db0 2024-05-10T11:37:48.647 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.645+0000 7ff8197fa640 1 -- 172.21.15.100:0/1657230325 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff80c0054b0 con 0x56307a533db0 2024-05-10T11:37:48.647 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.645+0000 7ff8197fa640 1 -- 172.21.15.100:0/1657230325 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff80c0117e0 con 0x56307a533db0 2024-05-10T11:37:48.649 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.646+0000 7ff82c038740 1 -- 172.21.15.100:0/1657230325 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56307a533db0 msgr2=0x56307a5341d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:48.650 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.646+0000 7ff82c038740 1 --2- 172.21.15.100:0/1657230325 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56307a533db0 0x56307a5341d0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7ff80c013850 tx=0x7ff80c0097c0 comp rx=0 tx=0).stop 2024-05-10T11:37:48.650 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.646+0000 7ff82c038740 1 -- 172.21.15.100:0/1657230325 shutdown_connections 2024-05-10T11:37:48.650 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.646+0000 7ff82c038740 1 --2- 172.21.15.100:0/1657230325 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56307a539910 0x56307a53bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:48.650 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.646+0000 7ff82c038740 1 --2- 172.21.15.100:0/1657230325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56307a5347a0 0x56307a5392a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:48.650 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.646+0000 7ff82c038740 1 --2- 172.21.15.100:0/1657230325 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56307a533db0 0x56307a5341d0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:48.650 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.646+0000 7ff82c038740 1 -- 172.21.15.100:0/1657230325 >> 172.21.15.100:0/1657230325 conn(0x56307a52f420 msgr2=0x56307a52f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:37:48.650 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.646+0000 7ff82c038740 1 -- 172.21.15.100:0/1657230325 shutdown_connections 2024-05-10T11:37:48.650 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.646+0000 7ff82c038740 1 -- 172.21.15.100:0/1657230325 wait complete. 2024-05-10T11:37:48.650 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.647+0000 7ff82c038740 1 -- start start 2024-05-10T11:37:48.650 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.647+0000 7ff82c038740 1 Processor -- start 2024-05-10T11:37:48.650 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.647+0000 7ff82c038740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56307a533db0 0x56307a53df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:48.650 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.647+0000 7ff82c038740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56307a5347a0 0x56307a53e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:48.650 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.647+0000 7ff82c038740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56307a539910 0x56307a545510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:48.650 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.647+0000 7ff81bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56307a533db0 0x56307a53df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:48.651 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.647+0000 7ff81bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56307a533db0 0x56307a53df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58056/0 (socket says 172.21.15.100:58056) 2024-05-10T11:37:48.651 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.647+0000 7ff822d9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56307a5347a0 0x56307a53e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:48.651 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.647+0000 7ff822d9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56307a5347a0 0x56307a53e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:45712/0 (socket says 172.21.15.100:45712) 2024-05-10T11:37:48.651 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.648+0000 7ff81bfff640 1 -- 172.21.15.100:0/775984813 learned_addr learned my addr 172.21.15.100:0/775984813 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:37:48.651 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.648+0000 7ff82c038740 1 -- 172.21.15.100:0/775984813 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56307a53c860 con 0x56307a533db0 2024-05-10T11:37:48.651 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.648+0000 7ff82c038740 1 -- 172.21.15.100:0/775984813 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56307a455370 con 0x56307a539910 2024-05-10T11:37:48.651 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.648+0000 7ff82c038740 1 -- 172.21.15.100:0/775984813 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56307a53c9e0 con 0x56307a5347a0 2024-05-10T11:37:48.651 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.648+0000 7ff82c038740 10 client.0 _ll_register_callbacks cb 0x56307a1a9a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:37:48.651 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.648+0000 7ff81bfff640 1 --2- 172.21.15.100:0/775984813 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56307a539910 0x56307a545510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:48.651 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.648+0000 7ff81bfff640 1 -- 172.21.15.100:0/775984813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56307a5347a0 msgr2=0x56307a53e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:37:48.651 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.648+0000 7ff81bfff640 1 --2- 172.21.15.100:0/775984813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56307a5347a0 0x56307a53e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:48.651 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.648+0000 7ff81bfff640 1 -- 172.21.15.100:0/775984813 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56307a539910 msgr2=0x56307a545510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:37:48.651 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.648+0000 7ff81bfff640 1 --2- 172.21.15.100:0/775984813 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56307a539910 0x56307a545510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:48.651 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.648+0000 7ff81bfff640 1 -- 172.21.15.100:0/775984813 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56307a276620 con 0x56307a533db0 2024-05-10T11:37:48.651 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.648+0000 7ff81bfff640 1 --2- 172.21.15.100:0/775984813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56307a533db0 0x56307a53df90 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7ff80c013660 tx=0x7ff80c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=941dfb0ca8245a96 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:37:48.652 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.648+0000 7ff8127fc640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:37:48.652 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.649+0000 7ff8127fc640 1 -- 172.21.15.100:0/775984813 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff80c004040 con 0x56307a533db0 2024-05-10T11:37:48.652 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.649+0000 7ff82c038740 1 -- 172.21.15.100:0/775984813 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff80c0054b0 con 0x56307a533db0 2024-05-10T11:37:48.652 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.649+0000 7ff8127fc640 1 -- 172.21.15.100:0/775984813 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7ff80c0041e0 con 0x56307a533db0 2024-05-10T11:37:48.652 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.649+0000 7ff8127fc640 1 -- 172.21.15.100:0/775984813 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7ff80c0043e0 con 0x56307a533db0 2024-05-10T11:37:48.652 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.649+0000 7ff82c038740 10 client.19221 fetch_fsmap learned FSMap version 41 2024-05-10T11:37:48.652 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.649+0000 7ff82c038740 1 -- 172.21.15.100:0/775984813 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x56307a5484f0 con 0x56307a533db0 2024-05-10T11:37:48.652 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.649+0000 7ff8127fc640 1 -- 172.21.15.100:0/775984813 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff80c0054b0 con 0x56307a533db0 2024-05-10T11:37:48.652 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.649+0000 7ff8127fc640 1 -- 172.21.15.100:0/775984813 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7ff80c003710 con 0x56307a533db0 2024-05-10T11:37:48.652 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.649+0000 7ff82c038740 10 client.19221 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:37:48.652 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.649+0000 7ff82c038740 10 client.19221 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:37:48.652 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.650+0000 7ff82c038740 20 client.19221 populate_metadata read hostname 'smithi100' 2024-05-10T11:37:48.652 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.650+0000 7ff82c038740 1 --2- 172.21.15.100:0/775984813 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56307a549ea0 0x56307a54c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:48.652 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.650+0000 7ff82c038740 4 client.19221 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:37:48.652 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.650+0000 7ff82c038740 1 -- 172.21.15.100:0/775984813 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56307a4a61e0 con 0x56307a549ea0 2024-05-10T11:37:48.653 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.650+0000 7ff822d9d640 1 --2- 172.21.15.100:0/775984813 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56307a549ea0 0x56307a54c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:48.654 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.650+0000 7ff822d9d640 1 --2- 172.21.15.100:0/775984813 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56307a549ea0 0x56307a54c2b0 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abfb5e0fd7ce1e3a server_cookie=8ca3ec48094d809e in_seq=0 out_seq=0 2024-05-10T11:37:48.654 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.650+0000 7ff8127fc640 10 client.19221 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:37:48.654 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.652+0000 7ff8127fc640 1 -- 172.21.15.100:0/775984813 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56307a4a61e0 con 0x56307a549ea0 2024-05-10T11:37:48.654 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.652+0000 7ff8127fc640 10 client.19221 handle_command_reply: tid=0 2024-05-10T11:37:48.654 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.652+0000 7ff82c038740 1 client.19221 shutdown 2024-05-10T11:37:48.654 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.652+0000 7ff82c038740 1 -- 172.21.15.100:0/775984813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56307a533db0 msgr2=0x56307a53df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:48.654 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.652+0000 7ff82c038740 1 --2- 172.21.15.100:0/775984813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56307a533db0 0x56307a53df90 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7ff80c013660 tx=0x7ff80c005f90 comp rx=0 tx=0).stop 2024-05-10T11:37:48.654 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.652+0000 7ff82c038740 1 -- 172.21.15.100:0/775984813 shutdown_connections 2024-05-10T11:37:48.654 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.652+0000 7ff82c038740 1 --2- 172.21.15.100:0/775984813 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56307a549ea0 0x56307a54c2b0 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:48.654 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.652+0000 7ff82c038740 1 --2- 172.21.15.100:0/775984813 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56307a539910 0x56307a545510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:48.654 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.652+0000 7ff82c038740 1 --2- 172.21.15.100:0/775984813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56307a5347a0 0x56307a53e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:48.655 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.652+0000 7ff8127fc640 0 client.19221 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:37:48.655 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.652+0000 7ff82c038740 1 --2- 172.21.15.100:0/775984813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56307a533db0 0x56307a53df90 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:48.655 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.652+0000 7ff82c038740 1 -- 172.21.15.100:0/775984813 >> 172.21.15.100:0/775984813 conn(0x56307a52f420 msgr2=0x56307a52e580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:37:48.655 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.652+0000 7ff82c038740 1 -- 172.21.15.100:0/775984813 shutdown_connections 2024-05-10T11:37:48.655 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.652+0000 7ff82c038740 1 -- 172.21.15.100:0/775984813 wait complete. 2024-05-10T11:37:48.655 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.652+0000 7ff82c038740 20 client.19221 trim_cache size 0 max 16384 2024-05-10T11:37:48.676 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.675+0000 7ff82c038740 1 Processor -- start 2024-05-10T11:37:48.676 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.675+0000 7ff82c038740 1 -- start start 2024-05-10T11:37:48.676 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.675+0000 7ff82c038740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56307a533db0 0x56307a53d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:48.676 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.675+0000 7ff82c038740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56307a5347a0 0x56307a53e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:48.676 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.675+0000 7ff82c038740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56307a539910 0x56307a532100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.675+0000 7ff82c038740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff8241afe90 con 0x56307a5347a0 2024-05-10T11:37:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.675+0000 7ff82c038740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff8241ad760 con 0x56307a539910 2024-05-10T11:37:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.675+0000 7ff82c038740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff8241ab5b0 con 0x56307a533db0 2024-05-10T11:37:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.675+0000 7ff822d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56307a5347a0 0x56307a53e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.675+0000 7ff822d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56307a5347a0 0x56307a53e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58068/0 (socket says 172.21.15.100:58068) 2024-05-10T11:37:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.675+0000 7ff81bfff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56307a533db0 0x56307a53d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.675+0000 7ff822d9d640 1 -- 172.21.15.100:0/239843167 learned_addr learned my addr 172.21.15.100:0/239843167 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:37:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.675+0000 7ff81bfff640 1 --2- 172.21.15.100:0/239843167 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56307a539910 0x56307a532100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.676+0000 7ff822d9d640 1 -- 172.21.15.100:0/239843167 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56307a533db0 msgr2=0x56307a53d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:37:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.676+0000 7ff822d9d640 1 --2- 172.21.15.100:0/239843167 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56307a533db0 0x56307a53d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.676+0000 7ff822d9d640 1 -- 172.21.15.100:0/239843167 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56307a539910 msgr2=0x56307a532100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:37:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.676+0000 7ff822d9d640 1 --2- 172.21.15.100:0/239843167 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56307a539910 0x56307a532100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.676+0000 7ff822d9d640 1 -- 172.21.15.100:0/239843167 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56307a52f250 con 0x56307a5347a0 2024-05-10T11:37:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.676+0000 7ff822d9d640 1 --2- 172.21.15.100:0/239843167 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56307a5347a0 0x56307a53e2b0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7ff81402a930 tx=0x7ff814009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e5aac2d0c532992 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:37:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.676+0000 7ff818ff9640 1 -- 172.21.15.100:0/239843167 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81404b070 con 0x56307a5347a0 2024-05-10T11:37:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.676+0000 7ff818ff9640 1 -- 172.21.15.100:0/239843167 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff8140044d0 con 0x56307a5347a0 2024-05-10T11:37:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.676+0000 7ff818ff9640 1 -- 172.21.15.100:0/239843167 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff814035040 con 0x56307a5347a0 2024-05-10T11:37:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.677+0000 7ff82c038740 1 -- 172.21.15.100:0/239843167 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56307a5347a0 msgr2=0x56307a53e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.677+0000 7ff82c038740 1 --2- 172.21.15.100:0/239843167 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56307a5347a0 0x56307a53e2b0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7ff81402a930 tx=0x7ff814009260 comp rx=0 tx=0).stop 2024-05-10T11:37:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.677+0000 7ff82c038740 1 -- 172.21.15.100:0/239843167 shutdown_connections 2024-05-10T11:37:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.677+0000 7ff82c038740 1 --2- 172.21.15.100:0/239843167 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56307a539910 0x56307a532100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.677+0000 7ff82c038740 1 --2- 172.21.15.100:0/239843167 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56307a5347a0 0x56307a53e2b0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.677+0000 7ff82c038740 1 --2- 172.21.15.100:0/239843167 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56307a533db0 0x56307a53d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.677+0000 7ff82c038740 1 -- 172.21.15.100:0/239843167 >> 172.21.15.100:0/239843167 conn(0x56307a52f420 msgr2=0x56307a53dba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:37:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.677+0000 7ff82c038740 1 -- 172.21.15.100:0/239843167 shutdown_connections 2024-05-10T11:37:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.677+0000 7ff82c038740 1 -- 172.21.15.100:0/239843167 wait complete. 2024-05-10T11:37:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.678+0000 7ff82c038740 1 -- start start 2024-05-10T11:37:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.678+0000 7ff82c038740 1 Processor -- start 2024-05-10T11:37:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.678+0000 7ff82c038740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56307a533db0 0x56307a541fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.678+0000 7ff82c038740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56307a5347a0 0x56307a5424f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.678+0000 7ff82c038740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56307a539910 0x56307a543500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.678+0000 7ff82c038740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff8241abc20 con 0x56307a5347a0 2024-05-10T11:37:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.678+0000 7ff82c038740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff8241ad090 con 0x56307a533db0 2024-05-10T11:37:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.678+0000 7ff82c038740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff8241ac2f0 con 0x56307a539910 2024-05-10T11:37:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.678+0000 7ff82c038740 10 client.0 _ll_register_callbacks cb 0x7ff81c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:37:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.678+0000 7ff81bfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56307a533db0 0x56307a541fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.678+0000 7ff81bfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56307a533db0 0x56307a541fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:35260/0 (socket says 172.21.15.100:35260) 2024-05-10T11:37:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.678+0000 7ff81bfff640 1 -- 172.21.15.100:0/4174021506 learned_addr learned my addr 172.21.15.100:0/4174021506 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:37:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.678+0000 7ff822d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56307a5347a0 0x56307a5424f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.678+0000 7ff81bfff640 1 --2- 172.21.15.100:0/4174021506 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56307a539910 0x56307a543500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.678+0000 7ff822d9d640 1 -- 172.21.15.100:0/4174021506 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56307a539910 msgr2=0x56307a543500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:37:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.678+0000 7ff822d9d640 1 --2- 172.21.15.100:0/4174021506 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56307a539910 0x56307a543500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.678+0000 7ff822d9d640 1 -- 172.21.15.100:0/4174021506 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56307a533db0 msgr2=0x56307a541fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:37:48.683 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.678+0000 7ff822d9d640 1 --2- 172.21.15.100:0/4174021506 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56307a533db0 0x56307a541fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:48.683 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.679+0000 7ff822d9d640 1 -- 172.21.15.100:0/4174021506 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56307a5339f0 con 0x56307a5347a0 2024-05-10T11:37:48.683 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.679+0000 7ff81bfff640 1 --2- 172.21.15.100:0/4174021506 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56307a539910 0x56307a543500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:37:48.683 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.679+0000 7ff822d9d640 1 --2- 172.21.15.100:0/4174021506 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56307a5347a0 0x56307a5424f0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7ff814009230 tx=0x7ff814031980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e10ab19e05feeaa5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:37:48.683 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.679+0000 7ff8127fc640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:37:48.683 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.679+0000 7ff8127fc640 1 -- 172.21.15.100:0/4174021506 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81404b040 con 0x56307a5347a0 2024-05-10T11:37:48.683 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.679+0000 7ff8127fc640 1 -- 172.21.15.100:0/4174021506 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7ff8140466c0 con 0x56307a5347a0 2024-05-10T11:37:48.683 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.679+0000 7ff82c038740 1 -- 172.21.15.100:0/4174021506 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8140044d0 con 0x56307a5347a0 2024-05-10T11:37:48.683 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.679+0000 7ff8127fc640 1 -- 172.21.15.100:0/4174021506 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7ff81405d050 con 0x56307a5347a0 2024-05-10T11:37:48.683 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.679+0000 7ff82c038740 10 client.19227 fetch_fsmap learned FSMap version 41 2024-05-10T11:37:48.683 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.679+0000 7ff82c038740 1 -- 172.21.15.100:0/4174021506 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x56307a544c30 con 0x56307a5347a0 2024-05-10T11:37:48.683 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.679+0000 7ff8127fc640 1 -- 172.21.15.100:0/4174021506 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff8140044d0 con 0x56307a5347a0 2024-05-10T11:37:48.683 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.680+0000 7ff8127fc640 1 -- 172.21.15.100:0/4174021506 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7ff81404e070 con 0x56307a5347a0 2024-05-10T11:37:48.683 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.680+0000 7ff82c038740 10 client.19227 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:37:48.683 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.680+0000 7ff82c038740 10 client.19227 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:37:48.684 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.680+0000 7ff82c038740 20 client.19227 populate_metadata read hostname 'smithi100' 2024-05-10T11:37:48.684 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.680+0000 7ff82c038740 1 --2- 172.21.15.100:0/4174021506 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56307a5b4c00 0x56307a5b7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:48.684 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.680+0000 7ff82c038740 4 client.19227 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:37:48.684 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.680+0000 7ff82c038740 1 -- 172.21.15.100:0/4174021506 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56307a4a0050 con 0x56307a5b4c00 2024-05-10T11:37:48.684 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.680+0000 7ff81bfff640 1 --2- 172.21.15.100:0/4174021506 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56307a5b4c00 0x56307a5b7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:48.684 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.680+0000 7ff81bfff640 1 --2- 172.21.15.100:0/4174021506 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56307a5b4c00 0x56307a5b7020 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd7b549ae1f4955b server_cookie=bdf34779e5e8e74c in_seq=0 out_seq=0 2024-05-10T11:37:48.685 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.680+0000 7ff8127fc640 10 client.19227 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:37:48.772 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.772+0000 7ff8127fc640 1 -- 172.21.15.100:0/4174021506 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56307a4a0050 con 0x56307a5b4c00 2024-05-10T11:37:48.772 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.772+0000 7ff8127fc640 10 client.19227 handle_command_reply: tid=0 2024-05-10T11:37:48.773 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.772+0000 7ff82c038740 1 client.19227 shutdown 2024-05-10T11:37:48.773 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.772+0000 7ff82c038740 1 -- 172.21.15.100:0/4174021506 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56307a5347a0 msgr2=0x56307a5424f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:48.773 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.772+0000 7ff82c038740 1 --2- 172.21.15.100:0/4174021506 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56307a5347a0 0x56307a5424f0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7ff814009230 tx=0x7ff814031980 comp rx=0 tx=0).stop 2024-05-10T11:37:48.773 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.772+0000 7ff82c038740 1 -- 172.21.15.100:0/4174021506 shutdown_connections 2024-05-10T11:37:48.773 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.772+0000 7ff82c038740 1 --2- 172.21.15.100:0/4174021506 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56307a5b4c00 0x56307a5b7020 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:48.773 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.772+0000 7ff82c038740 1 --2- 172.21.15.100:0/4174021506 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56307a539910 0x56307a543500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:48.773 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.772+0000 7ff8127fc640 0 client.19227 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:37:48.773 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.772+0000 7ff82c038740 1 --2- 172.21.15.100:0/4174021506 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56307a5347a0 0x56307a5424f0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:48.773 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.772+0000 7ff82c038740 1 --2- 172.21.15.100:0/4174021506 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56307a533db0 0x56307a541fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:48.773 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.772+0000 7ff82c038740 1 -- 172.21.15.100:0/4174021506 >> 172.21.15.100:0/4174021506 conn(0x56307a52f420 msgr2=0x56307a52dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:37:48.773 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.772+0000 7ff82c038740 1 -- 172.21.15.100:0/4174021506 shutdown_connections 2024-05-10T11:37:48.773 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.772+0000 7ff82c038740 1 -- 172.21.15.100:0/4174021506 wait complete. 2024-05-10T11:37:48.773 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.772+0000 7ff82c038740 20 client.19227 trim_cache size 0 max 16384 2024-05-10T11:37:48.774 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:37:48.774 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:37:48.774 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:37:48.774 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:37:48.775 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.773+0000 7ff829245640 1 -- 172.21.15.100:0/2534423404 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff8040b28a0 msgr2=0x7ff8040b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:48.775 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.773+0000 7ff829245640 1 --2- 172.21.15.100:0/2534423404 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff8040b28a0 0x7ff8040b4d60 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7ff81c0a5e10 tx=0x7ff81c0a5c50 comp rx=0 tx=0).stop 2024-05-10T11:37:48.775 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.773+0000 7ff829245640 1 -- 172.21.15.100:0/2534423404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8241025e0 msgr2=0x7ff8241a2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:48.775 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.773+0000 7ff829245640 1 --2- 172.21.15.100:0/2534423404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8241025e0 0x7ff8241a2270 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7ff81c00eb10 tx=0x7ff81c00c2d0 comp rx=0 tx=0).stop 2024-05-10T11:37:48.775 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.773+0000 7ff829245640 1 -- 172.21.15.100:0/2534423404 shutdown_connections 2024-05-10T11:37:48.775 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.773+0000 7ff829245640 1 --2- 172.21.15.100:0/2534423404 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff8040b28a0 0x7ff8040b4d60 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:48.775 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.773+0000 7ff829245640 1 --2- 172.21.15.100:0/2534423404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8241025e0 0x7ff8241a2270 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:48.775 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.773+0000 7ff829245640 1 --2- 172.21.15.100:0/2534423404 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff824101c40 0x7ff82419b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:48.775 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.773+0000 7ff829245640 1 --2- 172.21.15.100:0/2534423404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8241012e0 0x7ff82419ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:48.775 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.773+0000 7ff829245640 1 -- 172.21.15.100:0/2534423404 >> 172.21.15.100:0/2534423404 conn(0x7ff8240fce80 msgr2=0x7ff8241001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:37:48.775 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.773+0000 7ff829245640 1 -- 172.21.15.100:0/2534423404 shutdown_connections 2024-05-10T11:37:48.775 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:37:48.773+0000 7ff829245640 1 -- 172.21.15.100:0/2534423404 wait complete. 2024-05-10T11:37:48.899 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-10T11:37:48.934 INFO:tasks.workunit.client.0.smithi100.stdout: 10 57498 9.01 MB/sec execute 222 sec latency 111.625 ms 2024-05-10T11:37:49.264 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:37:49.679 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:37:49 smithi100 ceph-mon[30296]: pgmap v1004: 129 pgs: 129 active+clean; 491 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 4.2 MiB/s wr, 249 op/s 2024-05-10T11:37:49.934 INFO:tasks.workunit.client.0.smithi100.stdout: 10 57660 9.00 MB/sec execute 223 sec latency 94.774 ms 2024-05-10T11:37:50.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:37:49 smithi178 ceph-mon[33359]: pgmap v1004: 129 pgs: 129 active+clean; 491 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 4.2 MiB/s wr, 249 op/s 2024-05-10T11:37:50.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:37:49 smithi188 ceph-mon[34075]: pgmap v1004: 129 pgs: 129 active+clean; 491 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 4.2 MiB/s wr, 249 op/s 2024-05-10T11:37:50.934 INFO:tasks.workunit.client.0.smithi100.stdout: 10 57806 8.99 MB/sec execute 224 sec latency 94.045 ms 2024-05-10T11:37:51.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.672+0000 7f8135370700 1 -- 172.21.15.100:0/2635929395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8130108460 msgr2=0x7f8130108840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:51.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.672+0000 7f8135370700 1 --2- 172.21.15.100:0/2635929395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8130108460 0x7f8130108840 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f812800b0a0 tx=0x7f812800b3b0 comp rx=0 tx=0).stop 2024-05-10T11:37:51.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.672+0000 7f8135370700 1 -- 172.21.15.100:0/2635929395 shutdown_connections 2024-05-10T11:37:51.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.672+0000 7f8135370700 1 --2- 172.21.15.100:0/2635929395 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8130103ef0 0x7f81301103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:51.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.672+0000 7f8135370700 1 --2- 172.21.15.100:0/2635929395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8130108460 0x7f8130108840 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:51.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.672+0000 7f8135370700 1 --2- 172.21.15.100:0/2635929395 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8130103550 0x7f81301039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:51.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.672+0000 7f8135370700 1 -- 172.21.15.100:0/2635929395 >> 172.21.15.100:0/2635929395 conn(0x7f813007a600 msgr2=0x7f813007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:37:51.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.672+0000 7f8135370700 1 -- 172.21.15.100:0/2635929395 shutdown_connections 2024-05-10T11:37:51.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.672+0000 7f8135370700 1 -- 172.21.15.100:0/2635929395 wait complete. 2024-05-10T11:37:51.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.673+0000 7f8135370700 1 Processor -- start 2024-05-10T11:37:51.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.673+0000 7f8135370700 1 -- start start 2024-05-10T11:37:51.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.673+0000 7f8135370700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8130103550 0x7f81301101c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:51.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.673+0000 7f8135370700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8130103ef0 0x7f813010d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:51.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.673+0000 7f8135370700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8130108460 0x7f813010d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:51.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.673+0000 7f8135370700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8130077020 con 0x7f8130103550 2024-05-10T11:37:51.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.673+0000 7f8135370700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8130076ea0 con 0x7f8130103ef0 2024-05-10T11:37:51.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.673+0000 7f8135370700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f81300771a0 con 0x7f8130108460 2024-05-10T11:37:51.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.673+0000 7f812e7fc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8130103ef0 0x7f813010d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:51.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.673+0000 7f812effd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8130103550 0x7f81301101c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:51.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.673+0000 7f812e7fc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8130103ef0 0x7f813010d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:35278/0 (socket says 172.21.15.100:35278) 2024-05-10T11:37:51.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.673+0000 7f812f7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8130108460 0x7f813010d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:51.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.673+0000 7f812e7fc700 1 -- 172.21.15.100:0/459259783 learned_addr learned my addr 172.21.15.100:0/459259783 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:37:51.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.674+0000 7f812e7fc700 1 -- 172.21.15.100:0/459259783 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8130108460 msgr2=0x7f813010d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:51.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.674+0000 7f812e7fc700 1 --2- 172.21.15.100:0/459259783 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8130108460 0x7f813010d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:51.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.674+0000 7f812e7fc700 1 -- 172.21.15.100:0/459259783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8130103550 msgr2=0x7f81301101c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:51.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.674+0000 7f812e7fc700 1 --2- 172.21.15.100:0/459259783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8130103550 0x7f81301101c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:51.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.674+0000 7f812e7fc700 1 -- 172.21.15.100:0/459259783 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f813010e0d0 con 0x7f8130103ef0 2024-05-10T11:37:51.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.674+0000 7f812e7fc700 1 --2- 172.21.15.100:0/459259783 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8130103ef0 0x7f813010d2c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f811800e9a0 tx=0x7f811800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:37:51.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.674+0000 7f8127fff700 1 -- 172.21.15.100:0/459259783 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811800ca10 con 0x7f8130103ef0 2024-05-10T11:37:51.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.674+0000 7f8127fff700 1 -- 172.21.15.100:0/459259783 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f811800cbb0 con 0x7f8130103ef0 2024-05-10T11:37:51.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.674+0000 7f8135370700 1 -- 172.21.15.100:0/459259783 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81301ac650 con 0x7f8130103ef0 2024-05-10T11:37:51.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.674+0000 7f8135370700 1 -- 172.21.15.100:0/459259783 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81301acb60 con 0x7f8130103ef0 2024-05-10T11:37:51.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.674+0000 7f8127fff700 1 -- 172.21.15.100:0/459259783 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8118018680 con 0x7f8130103ef0 2024-05-10T11:37:51.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.678+0000 7f8127fff700 1 -- 172.21.15.100:0/459259783 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f8118010430 con 0x7f8130103ef0 2024-05-10T11:37:51.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.678+0000 7f8127fff700 1 --2- 172.21.15.100:0/459259783 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f811c0b2880 0x7f811c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:51.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.679+0000 7f8127fff700 1 -- 172.21.15.100:0/459259783 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f81180e27d0 con 0x7f8130103ef0 2024-05-10T11:37:51.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.679+0000 7f812effd700 1 --2- 172.21.15.100:0/459259783 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f811c0b2880 0x7f811c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:51.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.680+0000 7f813fc03b80 1 Processor -- start 2024-05-10T11:37:51.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.680+0000 7f812effd700 1 --2- 172.21.15.100:0/459259783 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f811c0b2880 0x7f811c0b4d40 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f81280062a0 tx=0x7f8128006210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:37:51.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.680+0000 7f813fc03b80 1 -- start start 2024-05-10T11:37:51.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.681+0000 7f813fc03b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5581ae7044d0 0x5581ae7048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:51.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.681+0000 7f813fc03b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581ae704df0 0x5581ae7086f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:51.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.681+0000 7f813fc03b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581ae708c80 0x5581ae709100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:51.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.681+0000 7f812effd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5581ae7044d0 0x5581ae7048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:51.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.681+0000 7f812effd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5581ae7044d0 0x5581ae7048b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:45764/0 (socket says 172.21.15.100:45764) 2024-05-10T11:37:51.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.681+0000 7f812effd700 1 -- 172.21.15.100:0/745786715 learned_addr learned my addr 172.21.15.100:0/745786715 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:37:51.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.681+0000 7f813fc03b80 1 -- 172.21.15.100:0/745786715 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5581ae656860 con 0x5581ae704df0 2024-05-10T11:37:51.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.681+0000 7f812e7fc700 1 --2- 172.21.15.100:0/745786715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581ae704df0 0x5581ae7086f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:51.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.681+0000 7f812f7fe700 1 --2- 172.21.15.100:0/745786715 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581ae708c80 0x5581ae709100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:51.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.681+0000 7f813fc03b80 1 -- 172.21.15.100:0/745786715 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5581ae5f57c0 con 0x5581ae708c80 2024-05-10T11:37:51.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.682+0000 7f813fc03b80 1 -- 172.21.15.100:0/745786715 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5581ae652820 con 0x5581ae7044d0 2024-05-10T11:37:51.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.682+0000 7f812effd700 1 -- 172.21.15.100:0/745786715 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581ae708c80 msgr2=0x5581ae709100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:37:51.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.682+0000 7f812effd700 1 --2- 172.21.15.100:0/745786715 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581ae708c80 0x5581ae709100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:51.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.682+0000 7f812effd700 1 -- 172.21.15.100:0/745786715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581ae704df0 msgr2=0x5581ae7086f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:37:51.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.682+0000 7f812effd700 1 --2- 172.21.15.100:0/745786715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581ae704df0 0x5581ae7086f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:51.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.682+0000 7f812effd700 1 -- 172.21.15.100:0/745786715 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581ae5a38d0 con 0x5581ae7044d0 2024-05-10T11:37:51.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.682+0000 7f812f7fe700 1 --2- 172.21.15.100:0/745786715 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581ae708c80 0x5581ae709100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:37:51.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.682+0000 7f812effd700 1 --2- 172.21.15.100:0/745786715 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5581ae7044d0 0x5581ae7048b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8128015000 tx=0x7f812801d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d24b80396c8513d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:37:51.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.682+0000 7f81257fa700 1 -- 172.21.15.100:0/745786715 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81280074e0 con 0x5581ae7044d0 2024-05-10T11:37:51.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.683+0000 7f81257fa700 1 -- 172.21.15.100:0/745786715 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f812802e070 con 0x5581ae7044d0 2024-05-10T11:37:51.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.683+0000 7f81257fa700 1 -- 172.21.15.100:0/745786715 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8128029820 con 0x5581ae7044d0 2024-05-10T11:37:51.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.683+0000 7f813fc03b80 1 -- 172.21.15.100:0/745786715 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5581ae7044d0 msgr2=0x5581ae7048b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:51.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.683+0000 7f813fc03b80 1 --2- 172.21.15.100:0/745786715 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5581ae7044d0 0x5581ae7048b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8128015000 tx=0x7f812801d000 comp rx=0 tx=0).stop 2024-05-10T11:37:51.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.684+0000 7f813fc03b80 1 -- 172.21.15.100:0/745786715 shutdown_connections 2024-05-10T11:37:51.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.684+0000 7f813fc03b80 1 --2- 172.21.15.100:0/745786715 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581ae708c80 0x5581ae709100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:51.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.684+0000 7f813fc03b80 1 --2- 172.21.15.100:0/745786715 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5581ae7044d0 0x5581ae7048b0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:51.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.684+0000 7f813fc03b80 1 --2- 172.21.15.100:0/745786715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581ae704df0 0x5581ae7086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:51.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.684+0000 7f813fc03b80 1 -- 172.21.15.100:0/745786715 >> 172.21.15.100:0/745786715 conn(0x5581ae630100 msgr2=0x5581ae6fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:37:51.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.684+0000 7f813fc03b80 1 -- 172.21.15.100:0/745786715 shutdown_connections 2024-05-10T11:37:51.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.684+0000 7f813fc03b80 1 -- 172.21.15.100:0/745786715 wait complete. 2024-05-10T11:37:51.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.684+0000 7f813fc03b80 1 -- start start 2024-05-10T11:37:51.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.684+0000 7f813fc03b80 1 Processor -- start 2024-05-10T11:37:51.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.685+0000 7f813fc03b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5581ae7044d0 0x5581ae710110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:51.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.685+0000 7f813fc03b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581ae704df0 0x5581ae70d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:51.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.685+0000 7f813fc03b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581ae708c80 0x5581ae70d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:51.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.685+0000 7f813fc03b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5581ae710e30 con 0x5581ae704df0 2024-05-10T11:37:51.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.685+0000 7f813fc03b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5581ae5bcc30 con 0x5581ae708c80 2024-05-10T11:37:51.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.685+0000 7f813fc03b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5581ae710fb0 con 0x5581ae7044d0 2024-05-10T11:37:51.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.685+0000 7f813fc03b80 10 client.0 _ll_register_callbacks cb 0x5581ae5ee920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:37:51.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.685+0000 7f812effd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5581ae7044d0 0x5581ae710110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:51.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.685+0000 7f812effd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5581ae7044d0 0x5581ae710110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:45778/0 (socket says 172.21.15.100:45778) 2024-05-10T11:37:51.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.685+0000 7f812effd700 1 -- 172.21.15.100:0/3837796308 learned_addr learned my addr 172.21.15.100:0/3837796308 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:37:51.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.685+0000 7f812f7fe700 1 --2- 172.21.15.100:0/3837796308 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581ae708c80 0x5581ae70d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:51.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.685+0000 7f812e7fc700 1 --2- 172.21.15.100:0/3837796308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581ae704df0 0x5581ae70d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:51.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.685+0000 7f812effd700 1 -- 172.21.15.100:0/3837796308 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581ae708c80 msgr2=0x5581ae70d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:37:51.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.685+0000 7f812effd700 1 --2- 172.21.15.100:0/3837796308 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581ae708c80 0x5581ae70d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:51.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.685+0000 7f812effd700 1 -- 172.21.15.100:0/3837796308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581ae704df0 msgr2=0x5581ae70d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:37:51.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.685+0000 7f812effd700 1 --2- 172.21.15.100:0/3837796308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581ae704df0 0x5581ae70d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:51.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.685+0000 7f812effd700 1 -- 172.21.15.100:0/3837796308 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581ae5f2690 con 0x5581ae7044d0 2024-05-10T11:37:51.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.685+0000 7f812e7fc700 1 --2- 172.21.15.100:0/3837796308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581ae704df0 0x5581ae70d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:37:51.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.686+0000 7f812f7fe700 1 --2- 172.21.15.100:0/3837796308 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581ae708c80 0x5581ae70d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:37:51.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.686+0000 7f812effd700 1 --2- 172.21.15.100:0/3837796308 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5581ae7044d0 0x5581ae710110 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f812800b3f0 tx=0x7f8128007970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4699d20d1ef5312 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:37:51.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.686+0000 7f81167fc700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:37:51.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.686+0000 7f81167fc700 1 -- 172.21.15.100:0/3837796308 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8128029680 con 0x5581ae7044d0 2024-05-10T11:37:51.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.686+0000 7f81167fc700 1 -- 172.21.15.100:0/3837796308 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f8128029820 con 0x5581ae7044d0 2024-05-10T11:37:51.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.686+0000 7f813fc03b80 1 -- 172.21.15.100:0/3837796308 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f812802e070 con 0x5581ae7044d0 2024-05-10T11:37:51.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.686+0000 7f81167fc700 1 -- 172.21.15.100:0/3837796308 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f8128029a20 con 0x5581ae7044d0 2024-05-10T11:37:51.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.686+0000 7f813fc03b80 10 client.28630 fetch_fsmap learned FSMap version 41 2024-05-10T11:37:51.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.686+0000 7f813fc03b80 1 -- 172.21.15.100:0/3837796308 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5581ae710e30 con 0x5581ae7044d0 2024-05-10T11:37:51.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.687+0000 7f81167fc700 1 -- 172.21.15.100:0/3837796308 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8128037070 con 0x5581ae7044d0 2024-05-10T11:37:51.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.687+0000 7f81167fc700 1 -- 172.21.15.100:0/3837796308 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f8128011070 con 0x5581ae7044d0 2024-05-10T11:37:51.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.687+0000 7f813fc03b80 10 client.28630 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:37:51.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.687+0000 7f813fc03b80 10 client.28630 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-10T11:37:51.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.687+0000 7f813fc03b80 20 client.28630 populate_metadata read hostname 'smithi100' 2024-05-10T11:37:51.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.687+0000 7f813fc03b80 1 --2- 172.21.15.100:0/3837796308 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x5581ae71dba0 0x5581ae71ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:51.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.687+0000 7f813fc03b80 4 client.28630 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:37:51.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.687+0000 7f813fc03b80 1 -- 172.21.15.100:0/3837796308 --> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f812802e070 con 0x5581ae71dba0 2024-05-10T11:37:51.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.687+0000 7f812e7fc700 1 --2- 172.21.15.100:0/3837796308 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x5581ae71dba0 0x5581ae71ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:51.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.688+0000 7f81167fc700 10 client.28630 ms_handle_connect on v2:172.21.15.178:6838/3760985404 2024-05-10T11:37:51.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.688+0000 7f812e7fc700 1 --2- 172.21.15.100:0/3837796308 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x5581ae71dba0 0x5581ae71ffc0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bea5b914699fce2c server_cookie=c76704a05b289346 in_seq=0 out_seq=0 2024-05-10T11:37:51.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.688+0000 7f81167fc700 1 -- 172.21.15.100:0/3837796308 <== mds.1 v2:172.21.15.178:6838/3760985404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f812802e070 con 0x5581ae71dba0 2024-05-10T11:37:51.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.688+0000 7f81167fc700 10 client.28630 handle_command_reply: tid=0 2024-05-10T11:37:51.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.689+0000 7f813fc03b80 1 client.28630 shutdown 2024-05-10T11:37:51.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.689+0000 7f813fc03b80 1 -- 172.21.15.100:0/3837796308 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5581ae7044d0 msgr2=0x5581ae710110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:51.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.689+0000 7f813fc03b80 1 --2- 172.21.15.100:0/3837796308 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5581ae7044d0 0x5581ae710110 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f812800b3f0 tx=0x7f8128007970 comp rx=0 tx=0).stop 2024-05-10T11:37:51.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.689+0000 7f813fc03b80 1 -- 172.21.15.100:0/3837796308 shutdown_connections 2024-05-10T11:37:51.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.689+0000 7f813fc03b80 1 --2- 172.21.15.100:0/3837796308 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x5581ae71dba0 0x5581ae71ffc0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:51.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.689+0000 7f81167fc700 0 client.28630 ms_handle_reset on v2:172.21.15.178:6838/3760985404 2024-05-10T11:37:51.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.689+0000 7f813fc03b80 1 --2- 172.21.15.100:0/3837796308 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581ae708c80 0x5581ae70d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:51.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.689+0000 7f813fc03b80 1 --2- 172.21.15.100:0/3837796308 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5581ae7044d0 0x5581ae710110 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:51.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.689+0000 7f813fc03b80 1 --2- 172.21.15.100:0/3837796308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581ae704df0 0x5581ae70d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:51.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.689+0000 7f813fc03b80 1 -- 172.21.15.100:0/3837796308 >> 172.21.15.100:0/3837796308 conn(0x5581ae630100 msgr2=0x5581ae6f9720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:37:51.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.689+0000 7f813fc03b80 1 -- 172.21.15.100:0/3837796308 shutdown_connections 2024-05-10T11:37:51.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.689+0000 7f813fc03b80 1 -- 172.21.15.100:0/3837796308 wait complete. 2024-05-10T11:37:51.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.689+0000 7f813fc03b80 20 client.28630 trim_cache size 0 max 16384 2024-05-10T11:37:51.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.713+0000 7f813fc03b80 1 Processor -- start 2024-05-10T11:37:51.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.713+0000 7f813fc03b80 1 -- start start 2024-05-10T11:37:51.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.713+0000 7f813fc03b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581ae7044d0 0x5581ae70c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:51.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.714+0000 7f813fc03b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5581ae704df0 0x5581ae719ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:51.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.714+0000 7f813fc03b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581ae708c80 0x5581ae725670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:51.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.714+0000 7f813fc03b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f81301b7f70 con 0x5581ae708c80 2024-05-10T11:37:51.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.714+0000 7f813fc03b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f81301b5800 con 0x5581ae7044d0 2024-05-10T11:37:51.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.714+0000 7f813fc03b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f81301b3650 con 0x5581ae704df0 2024-05-10T11:37:51.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.714+0000 7f812e7fc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5581ae704df0 0x5581ae719ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:51.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.714+0000 7f812e7fc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5581ae704df0 0x5581ae719ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:45786/0 (socket says 172.21.15.100:45786) 2024-05-10T11:37:51.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.714+0000 7f812e7fc700 1 -- 172.21.15.100:0/795805180 learned_addr learned my addr 172.21.15.100:0/795805180 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:37:51.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.714+0000 7f812effd700 1 --2- 172.21.15.100:0/795805180 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581ae7044d0 0x5581ae70c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:51.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.714+0000 7f812f7fe700 1 --2- 172.21.15.100:0/795805180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581ae708c80 0x5581ae725670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:51.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.714+0000 7f812e7fc700 1 -- 172.21.15.100:0/795805180 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581ae7044d0 msgr2=0x5581ae70c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:37:51.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.714+0000 7f812e7fc700 1 --2- 172.21.15.100:0/795805180 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581ae7044d0 0x5581ae70c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:51.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.714+0000 7f812e7fc700 1 -- 172.21.15.100:0/795805180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581ae708c80 msgr2=0x5581ae725670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:37:51.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.714+0000 7f812e7fc700 1 --2- 172.21.15.100:0/795805180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581ae708c80 0x5581ae725670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:51.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.714+0000 7f812e7fc700 1 -- 172.21.15.100:0/795805180 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581ae708ab0 con 0x5581ae704df0 2024-05-10T11:37:51.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.714+0000 7f812e7fc700 1 --2- 172.21.15.100:0/795805180 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5581ae704df0 0x5581ae719ec0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f81180a99c0 tx=0x7f81180dfde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf68ee3c8640e62a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:37:51.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.714+0000 7f8124ff9700 1 -- 172.21.15.100:0/795805180 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81180aa130 con 0x5581ae704df0 2024-05-10T11:37:51.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.715+0000 7f8124ff9700 1 -- 172.21.15.100:0/795805180 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8118010e50 con 0x5581ae704df0 2024-05-10T11:37:51.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.715+0000 7f8124ff9700 1 -- 172.21.15.100:0/795805180 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811800cd60 con 0x5581ae704df0 2024-05-10T11:37:51.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.715+0000 7f813fc03b80 1 -- 172.21.15.100:0/795805180 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5581ae704df0 msgr2=0x5581ae719ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:51.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.715+0000 7f813fc03b80 1 --2- 172.21.15.100:0/795805180 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5581ae704df0 0x5581ae719ec0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f81180a99c0 tx=0x7f81180dfde0 comp rx=0 tx=0).stop 2024-05-10T11:37:51.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.716+0000 7f813fc03b80 1 -- 172.21.15.100:0/795805180 shutdown_connections 2024-05-10T11:37:51.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.716+0000 7f813fc03b80 1 --2- 172.21.15.100:0/795805180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581ae708c80 0x5581ae725670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:51.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.716+0000 7f813fc03b80 1 --2- 172.21.15.100:0/795805180 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581ae7044d0 0x5581ae70c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:51.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.716+0000 7f813fc03b80 1 --2- 172.21.15.100:0/795805180 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5581ae704df0 0x5581ae719ec0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:51.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.716+0000 7f813fc03b80 1 -- 172.21.15.100:0/795805180 >> 172.21.15.100:0/795805180 conn(0x5581ae630100 msgr2=0x5581ae6f9720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:37:51.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.716+0000 7f813fc03b80 1 -- 172.21.15.100:0/795805180 shutdown_connections 2024-05-10T11:37:51.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.716+0000 7f813fc03b80 1 -- 172.21.15.100:0/795805180 wait complete. 2024-05-10T11:37:51.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.716+0000 7f813fc03b80 1 -- start start 2024-05-10T11:37:51.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.716+0000 7f813fc03b80 1 Processor -- start 2024-05-10T11:37:51.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.717+0000 7f813fc03b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581ae7044d0 0x5581ae716530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:51.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.717+0000 7f813fc03b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581ae704df0 0x5581ae711900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:51.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.717+0000 7f813fc03b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5581ae708c80 0x5581ae711e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:51.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.717+0000 7f813fc03b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f81301b3cc0 con 0x5581ae704df0 2024-05-10T11:37:51.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.717+0000 7f812e7fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581ae704df0 0x5581ae711900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:51.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.717+0000 7f813fc03b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f81301b5130 con 0x5581ae7044d0 2024-05-10T11:37:51.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.717+0000 7f813fc03b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f81301b4390 con 0x5581ae708c80 2024-05-10T11:37:51.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.717+0000 7f812effd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581ae7044d0 0x5581ae716530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:51.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.717+0000 7f812e7fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581ae704df0 0x5581ae711900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58144/0 (socket says 172.21.15.100:58144) 2024-05-10T11:37:51.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.717+0000 7f812e7fc700 1 -- 172.21.15.100:0/4042258285 learned_addr learned my addr 172.21.15.100:0/4042258285 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:37:51.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.717+0000 7f813fc03b80 10 client.0 _ll_register_callbacks cb 0x5581ae5ee920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:37:51.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.717+0000 7f812f7fe700 1 --2- 172.21.15.100:0/4042258285 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5581ae708c80 0x5581ae711e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:51.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.717+0000 7f812effd700 1 -- 172.21.15.100:0/4042258285 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5581ae708c80 msgr2=0x5581ae711e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:37:51.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.717+0000 7f812effd700 1 --2- 172.21.15.100:0/4042258285 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5581ae708c80 0x5581ae711e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:51.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.717+0000 7f812effd700 1 -- 172.21.15.100:0/4042258285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581ae704df0 msgr2=0x5581ae711900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:37:51.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.717+0000 7f812effd700 1 --2- 172.21.15.100:0/4042258285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581ae704df0 0x5581ae711900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:51.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.717+0000 7f812effd700 1 -- 172.21.15.100:0/4042258285 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581ae712400 con 0x5581ae7044d0 2024-05-10T11:37:51.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.717+0000 7f812e7fc700 1 --2- 172.21.15.100:0/4042258285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581ae704df0 0x5581ae711900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:37:51.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.717+0000 7f812effd700 1 --2- 172.21.15.100:0/4042258285 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581ae7044d0 0x5581ae716530 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8128000f80 tx=0x7f8128007a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5975eb88339e5152 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:37:51.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.717+0000 7f81167fc700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:37:51.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.718+0000 7f81167fc700 1 -- 172.21.15.100:0/4042258285 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8128020070 con 0x5581ae7044d0 2024-05-10T11:37:51.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.718+0000 7f81167fc700 1 -- 172.21.15.100:0/4042258285 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f81280093a0 con 0x5581ae7044d0 2024-05-10T11:37:51.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.718+0000 7f813fc03b80 1 -- 172.21.15.100:0/4042258285 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8118010e50 con 0x5581ae7044d0 2024-05-10T11:37:51.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.718+0000 7f81167fc700 1 -- 172.21.15.100:0/4042258285 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f8128029480 con 0x5581ae7044d0 2024-05-10T11:37:51.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.718+0000 7f813fc03b80 10 client.28529 fetch_fsmap learned FSMap version 41 2024-05-10T11:37:51.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.718+0000 7f813fc03b80 1 -- 172.21.15.100:0/4042258285 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5581ae787fd0 con 0x5581ae7044d0 2024-05-10T11:37:51.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.718+0000 7f81167fc700 1 -- 172.21.15.100:0/4042258285 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8118010e50 con 0x5581ae7044d0 2024-05-10T11:37:51.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.718+0000 7f81167fc700 1 -- 172.21.15.100:0/4042258285 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f8128011070 con 0x5581ae7044d0 2024-05-10T11:37:51.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.718+0000 7f813fc03b80 10 client.28529 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:37:51.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.718+0000 7f813fc03b80 10 client.28529 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-10T11:37:51.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.718+0000 7f813fc03b80 20 client.28529 populate_metadata read hostname 'smithi100' 2024-05-10T11:37:51.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.718+0000 7f813fc03b80 1 --2- 172.21.15.100:0/4042258285 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x5581ae788440 0x5581ae78a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:51.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.718+0000 7f813fc03b80 4 client.28529 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:37:51.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.718+0000 7f813fc03b80 1 -- 172.21.15.100:0/4042258285 --> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5581ae6d0650 con 0x5581ae788440 2024-05-10T11:37:51.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.719+0000 7f812e7fc700 1 --2- 172.21.15.100:0/4042258285 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x5581ae788440 0x5581ae78a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:51.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.719+0000 7f81167fc700 10 client.28529 ms_handle_connect on v2:172.21.15.178:6838/3760985404 2024-05-10T11:37:51.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.719+0000 7f812e7fc700 1 --2- 172.21.15.100:0/4042258285 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x5581ae788440 0x5581ae78a830 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d292ae376d902b07 server_cookie=861031e28a37a808 in_seq=0 out_seq=0 2024-05-10T11:37:51.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.719+0000 7f81167fc700 1 -- 172.21.15.100:0/4042258285 <== mds.1 v2:172.21.15.178:6838/3760985404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5581ae6d0650 con 0x5581ae788440 2024-05-10T11:37:51.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.719+0000 7f81167fc700 10 client.28529 handle_command_reply: tid=0 2024-05-10T11:37:51.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.719+0000 7f813fc03b80 1 client.28529 shutdown 2024-05-10T11:37:51.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.719+0000 7f813fc03b80 1 -- 172.21.15.100:0/4042258285 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581ae7044d0 msgr2=0x5581ae716530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:51.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.719+0000 7f813fc03b80 1 --2- 172.21.15.100:0/4042258285 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581ae7044d0 0x5581ae716530 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8128000f80 tx=0x7f8128007a90 comp rx=0 tx=0).stop 2024-05-10T11:37:51.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.719+0000 7f813fc03b80 1 -- 172.21.15.100:0/4042258285 shutdown_connections 2024-05-10T11:37:51.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.719+0000 7f813fc03b80 1 --2- 172.21.15.100:0/4042258285 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x5581ae788440 0x5581ae78a830 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:51.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.719+0000 7f813fc03b80 1 --2- 172.21.15.100:0/4042258285 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5581ae708c80 0x5581ae711e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:51.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.719+0000 7f813fc03b80 1 --2- 172.21.15.100:0/4042258285 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581ae7044d0 0x5581ae716530 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:51.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.719+0000 7f813fc03b80 1 --2- 172.21.15.100:0/4042258285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581ae704df0 0x5581ae711900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:51.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.719+0000 7f813fc03b80 1 -- 172.21.15.100:0/4042258285 >> 172.21.15.100:0/4042258285 conn(0x5581ae630100 msgr2=0x5581ae7032e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:37:51.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.719+0000 7f81167fc700 0 client.28529 ms_handle_reset on v2:172.21.15.178:6838/3760985404 2024-05-10T11:37:51.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.720+0000 7f813fc03b80 1 -- 172.21.15.100:0/4042258285 shutdown_connections 2024-05-10T11:37:51.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.720+0000 7f813fc03b80 1 -- 172.21.15.100:0/4042258285 wait complete. 2024-05-10T11:37:51.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.720+0000 7f813fc03b80 20 client.28529 trim_cache size 0 max 16384 2024-05-10T11:37:51.725 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:37:51.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.720+0000 7f8135370700 1 -- 172.21.15.100:0/459259783 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f811c0b2880 msgr2=0x7f811c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:51.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.720+0000 7f8135370700 1 --2- 172.21.15.100:0/459259783 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f811c0b2880 0x7f811c0b4d40 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f81280062a0 tx=0x7f8128006210 comp rx=0 tx=0).stop 2024-05-10T11:37:51.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.720+0000 7f8135370700 1 -- 172.21.15.100:0/459259783 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8130103ef0 msgr2=0x7f813010d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:51.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.720+0000 7f8135370700 1 --2- 172.21.15.100:0/459259783 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8130103ef0 0x7f813010d2c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f811800e9a0 tx=0x7f811800ecb0 comp rx=0 tx=0).stop 2024-05-10T11:37:51.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.720+0000 7f8135370700 1 -- 172.21.15.100:0/459259783 shutdown_connections 2024-05-10T11:37:51.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.720+0000 7f8135370700 1 --2- 172.21.15.100:0/459259783 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f811c0b2880 0x7f811c0b4d40 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:51.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.720+0000 7f8135370700 1 --2- 172.21.15.100:0/459259783 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8130108460 0x7f813010d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:51.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.720+0000 7f8135370700 1 --2- 172.21.15.100:0/459259783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8130103550 0x7f81301101c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:51.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.720+0000 7f8135370700 1 --2- 172.21.15.100:0/459259783 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8130103ef0 0x7f813010d2c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:51.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.720+0000 7f8135370700 1 -- 172.21.15.100:0/459259783 >> 172.21.15.100:0/459259783 conn(0x7f813007a600 msgr2=0x7f8130101640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:37:51.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.720+0000 7f8135370700 1 -- 172.21.15.100:0/459259783 shutdown_connections 2024-05-10T11:37:51.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:51.720+0000 7f8135370700 1 -- 172.21.15.100:0/459259783 wait complete. 2024-05-10T11:37:51.925 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:37:51 smithi100 ceph-mon[30296]: pgmap v1005: 129 pgs: 129 active+clean; 493 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 4.2 MiB/s wr, 295 op/s 2024-05-10T11:37:51.934 INFO:tasks.workunit.client.0.smithi100.stdout: 10 57995 9.02 MB/sec execute 225 sec latency 79.805 ms 2024-05-10T11:37:52.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:37:51 smithi178 ceph-mon[33359]: pgmap v1005: 129 pgs: 129 active+clean; 493 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 4.2 MiB/s wr, 295 op/s 2024-05-10T11:37:52.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:37:51 smithi188 ceph-mon[34075]: pgmap v1005: 129 pgs: 129 active+clean; 493 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 4.2 MiB/s wr, 295 op/s 2024-05-10T11:37:52.537 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-10T11:37:52.934 INFO:tasks.workunit.client.0.smithi100.stdout: 10 58251 9.05 MB/sec execute 226 sec latency 92.202 ms 2024-05-10T11:37:52.946 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:37:53.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:37:53 smithi100 ceph-mon[30296]: pgmap v1006: 129 pgs: 129 active+clean; 492 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 642 KiB/s rd, 3.3 MiB/s wr, 254 op/s 2024-05-10T11:37:53.934 INFO:tasks.workunit.client.0.smithi100.stdout: 10 58456 9.05 MB/sec execute 227 sec latency 63.768 ms 2024-05-10T11:37:54.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:37:53 smithi178 ceph-mon[33359]: pgmap v1006: 129 pgs: 129 active+clean; 492 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 642 KiB/s rd, 3.3 MiB/s wr, 254 op/s 2024-05-10T11:37:54.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:37:53 smithi188 ceph-mon[34075]: pgmap v1006: 129 pgs: 129 active+clean; 492 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 642 KiB/s rd, 3.3 MiB/s wr, 254 op/s 2024-05-10T11:37:54.934 INFO:tasks.workunit.client.0.smithi100.stdout: 10 58564 9.03 MB/sec execute 228 sec latency 102.973 ms 2024-05-10T11:37:55.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.001+0000 7fb30cc88700 1 -- 172.21.15.100:0/431677374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb308109500 msgr2=0x7fb3081098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:55.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.001+0000 7fb30cc88700 1 --2- 172.21.15.100:0/431677374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb308109500 0x7fb3081098e0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fb2f8009990 tx=0x7fb2f8009ca0 comp rx=0 tx=0).stop 2024-05-10T11:37:55.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.002+0000 7fb30cc88700 1 -- 172.21.15.100:0/431677374 shutdown_connections 2024-05-10T11:37:55.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.002+0000 7fb30cc88700 1 --2- 172.21.15.100:0/431677374 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb30806c3d0 0x7fb308110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:55.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.002+0000 7fb30cc88700 1 --2- 172.21.15.100:0/431677374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb308109500 0x7fb3081098e0 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:55.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.002+0000 7fb30cc88700 1 --2- 172.21.15.100:0/431677374 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb30806ba10 0x7fb30806be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:55.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.002+0000 7fb30cc88700 1 -- 172.21.15.100:0/431677374 >> 172.21.15.100:0/431677374 conn(0x7fb308079310 msgr2=0x7fb308079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:37:55.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.002+0000 7fb30cc88700 1 -- 172.21.15.100:0/431677374 shutdown_connections 2024-05-10T11:37:55.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.002+0000 7fb30cc88700 1 -- 172.21.15.100:0/431677374 wait complete. 2024-05-10T11:37:55.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.003+0000 7fb30cc88700 1 Processor -- start 2024-05-10T11:37:55.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.003+0000 7fb30cc88700 1 -- start start 2024-05-10T11:37:55.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.003+0000 7fb30cc88700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb30806ba10 0x7fb3081a1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:55.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.003+0000 7fb30cc88700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb30806c3d0 0x7fb3081a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:55.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.003+0000 7fb30cc88700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb308109500 0x7fb30819bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:55.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.003+0000 7fb30cc88700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb3081134c0 con 0x7fb30806ba10 2024-05-10T11:37:55.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.003+0000 7fb30cc88700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb308113340 con 0x7fb308109500 2024-05-10T11:37:55.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.003+0000 7fb30cc88700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb308113640 con 0x7fb30806c3d0 2024-05-10T11:37:55.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.003+0000 7fb305d9b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb30806c3d0 0x7fb3081a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:55.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.003+0000 7fb306d9d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb308109500 0x7fb30819bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:55.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.003+0000 7fb30659c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb30806ba10 0x7fb3081a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:55.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.003+0000 7fb306d9d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb308109500 0x7fb30819bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:35336/0 (socket says 172.21.15.100:35336) 2024-05-10T11:37:55.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.003+0000 7fb306d9d700 1 -- 172.21.15.100:0/3767470614 learned_addr learned my addr 172.21.15.100:0/3767470614 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:37:55.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.004+0000 7fb306d9d700 1 -- 172.21.15.100:0/3767470614 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb30806c3d0 msgr2=0x7fb3081a1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:55.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.004+0000 7fb306d9d700 1 --2- 172.21.15.100:0/3767470614 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb30806c3d0 0x7fb3081a1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:55.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.004+0000 7fb306d9d700 1 -- 172.21.15.100:0/3767470614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb30806ba10 msgr2=0x7fb3081a1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:55.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.004+0000 7fb306d9d700 1 --2- 172.21.15.100:0/3767470614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb30806ba10 0x7fb3081a1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:55.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.004+0000 7fb306d9d700 1 -- 172.21.15.100:0/3767470614 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb30819c260 con 0x7fb308109500 2024-05-10T11:37:55.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.004+0000 7fb305d9b700 1 --2- 172.21.15.100:0/3767470614 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb30806c3d0 0x7fb3081a1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:37:55.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.004+0000 7fb306d9d700 1 --2- 172.21.15.100:0/3767470614 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb308109500 0x7fb30819bb00 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb30000ef30 tx=0x7fb30000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:37:55.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.004+0000 7fb2f77fe700 1 -- 172.21.15.100:0/3767470614 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3000092e0 con 0x7fb308109500 2024-05-10T11:37:55.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.004+0000 7fb30cc88700 1 -- 172.21.15.100:0/3767470614 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb30819c4f0 con 0x7fb308109500 2024-05-10T11:37:55.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.004+0000 7fb2f77fe700 1 -- 172.21.15.100:0/3767470614 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb300004bb0 con 0x7fb308109500 2024-05-10T11:37:55.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.004+0000 7fb2f77fe700 1 -- 172.21.15.100:0/3767470614 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb300004020 con 0x7fb308109500 2024-05-10T11:37:55.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.005+0000 7fb30cc88700 1 -- 172.21.15.100:0/3767470614 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3081a8810 con 0x7fb308109500 2024-05-10T11:37:55.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.007+0000 7fb2f77fe700 1 -- 172.21.15.100:0/3767470614 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fb30001b030 con 0x7fb308109500 2024-05-10T11:37:55.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.009+0000 7fb2f77fe700 1 --2- 172.21.15.100:0/3767470614 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fb2f00b2900 0x7fb2f00b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:55.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.009+0000 7fb31751bb80 1 Processor -- start 2024-05-10T11:37:55.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.009+0000 7fb31751bb80 1 -- start start 2024-05-10T11:37:55.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.009+0000 7fb30659c700 1 --2- 172.21.15.100:0/3767470614 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fb2f00b2900 0x7fb2f00b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:55.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.009+0000 7fb2f77fe700 1 -- 172.21.15.100:0/3767470614 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fb3000e1f10 con 0x7fb308109500 2024-05-10T11:37:55.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.009+0000 7fb31751bb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6db64fce0 0x55c6db6500c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:55.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.009+0000 7fb31751bb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6db650600 0x55c6db653f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:55.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.009+0000 7fb31751bb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6db654440 0x55c6db6548a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:55.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.009+0000 7fb31751bb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55c6db599860 con 0x55c6db654440 2024-05-10T11:37:55.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.009+0000 7fb31751bb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c6db5387c0 con 0x55c6db650600 2024-05-10T11:37:55.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.009+0000 7fb31751bb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c6db595820 con 0x55c6db64fce0 2024-05-10T11:37:55.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.009+0000 7fb305d9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6db650600 0x55c6db653f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:55.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.009+0000 7fb305d9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6db650600 0x55c6db653f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:35350/0 (socket says 172.21.15.100:35350) 2024-05-10T11:37:55.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.009+0000 7fb305d9b700 1 -- 172.21.15.100:0/125809590 learned_addr learned my addr 172.21.15.100:0/125809590 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:37:55.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.010+0000 7fb30659c700 1 --2- 172.21.15.100:0/3767470614 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fb2f00b2900 0x7fb2f00b4dc0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fb2f800b2e0 tx=0x7fb2f8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:37:55.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.010+0000 7fb30659c700 1 --2- 172.21.15.100:0/125809590 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6db64fce0 0x55c6db6500c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:55.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.010+0000 7fb306d9d700 1 --2- 172.21.15.100:0/125809590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6db654440 0x55c6db6548a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:55.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.010+0000 7fb305d9b700 1 -- 172.21.15.100:0/125809590 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6db64fce0 msgr2=0x55c6db6500c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:37:55.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.010+0000 7fb305d9b700 1 --2- 172.21.15.100:0/125809590 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6db64fce0 0x55c6db6500c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:55.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.010+0000 7fb305d9b700 1 -- 172.21.15.100:0/125809590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6db654440 msgr2=0x55c6db6548a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:37:55.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.010+0000 7fb305d9b700 1 --2- 172.21.15.100:0/125809590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6db654440 0x55c6db6548a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:55.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.010+0000 7fb305d9b700 1 -- 172.21.15.100:0/125809590 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6db4e68d0 con 0x55c6db650600 2024-05-10T11:37:55.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.010+0000 7fb305d9b700 1 --2- 172.21.15.100:0/125809590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6db650600 0x55c6db653f00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc0115f0 tx=0x7fb2fc011900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30c04db27f598267 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:37:55.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.010+0000 7fb2f4ff9700 1 -- 172.21.15.100:0/125809590 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2fc004230 con 0x55c6db650600 2024-05-10T11:37:55.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.011+0000 7fb2f4ff9700 1 -- 172.21.15.100:0/125809590 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb2fc0043d0 con 0x55c6db650600 2024-05-10T11:37:55.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.011+0000 7fb2f4ff9700 1 -- 172.21.15.100:0/125809590 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2fc019830 con 0x55c6db650600 2024-05-10T11:37:55.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.011+0000 7fb31751bb80 1 -- 172.21.15.100:0/125809590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6db650600 msgr2=0x55c6db653f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:55.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.011+0000 7fb31751bb80 1 --2- 172.21.15.100:0/125809590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6db650600 0x55c6db653f00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc0115f0 tx=0x7fb2fc011900 comp rx=0 tx=0).stop 2024-05-10T11:37:55.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.011+0000 7fb31751bb80 1 -- 172.21.15.100:0/125809590 shutdown_connections 2024-05-10T11:37:55.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.011+0000 7fb31751bb80 1 --2- 172.21.15.100:0/125809590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6db654440 0x55c6db6548a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:55.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.011+0000 7fb31751bb80 1 --2- 172.21.15.100:0/125809590 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6db64fce0 0x55c6db6500c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:55.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.011+0000 7fb31751bb80 1 --2- 172.21.15.100:0/125809590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6db650600 0x55c6db653f00 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:55.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.011+0000 7fb31751bb80 1 -- 172.21.15.100:0/125809590 >> 172.21.15.100:0/125809590 conn(0x55c6db573100 msgr2=0x55c6db641c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:37:55.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.011+0000 7fb31751bb80 1 -- 172.21.15.100:0/125809590 shutdown_connections 2024-05-10T11:37:55.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.012+0000 7fb31751bb80 1 -- 172.21.15.100:0/125809590 wait complete. 2024-05-10T11:37:55.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.012+0000 7fb31751bb80 1 -- start start 2024-05-10T11:37:55.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.012+0000 7fb31751bb80 1 Processor -- start 2024-05-10T11:37:55.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.013+0000 7fb31751bb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6db64fce0 0x55c6db65bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:55.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.013+0000 7fb31751bb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6db654440 0x55c6db658db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:55.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.013+0000 7fb31751bb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6db6592f0 0x55c6db659770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:55.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.013+0000 7fb31751bb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55c6db65c6f0 con 0x55c6db6592f0 2024-05-10T11:37:55.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.013+0000 7fb31751bb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c6db4ffc30 con 0x55c6db64fce0 2024-05-10T11:37:55.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.013+0000 7fb31751bb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c6db65c870 con 0x55c6db654440 2024-05-10T11:37:55.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.013+0000 7fb31751bb80 10 client.0 _ll_register_callbacks cb 0x55c6db531920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:37:55.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.013+0000 7fb305d9b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6db654440 0x55c6db658db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:55.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.013+0000 7fb305d9b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6db654440 0x55c6db658db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:45824/0 (socket says 172.21.15.100:45824) 2024-05-10T11:37:55.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.013+0000 7fb305d9b700 1 -- 172.21.15.100:0/4218314680 learned_addr learned my addr 172.21.15.100:0/4218314680 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:37:55.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.013+0000 7fb30659c700 1 --2- 172.21.15.100:0/4218314680 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6db64fce0 0x55c6db65bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:55.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.013+0000 7fb306d9d700 1 --2- 172.21.15.100:0/4218314680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6db6592f0 0x55c6db659770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:55.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.013+0000 7fb305d9b700 1 -- 172.21.15.100:0/4218314680 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6db64fce0 msgr2=0x55c6db65bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:37:55.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.013+0000 7fb305d9b700 1 --2- 172.21.15.100:0/4218314680 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6db64fce0 0x55c6db65bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:55.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.013+0000 7fb305d9b700 1 -- 172.21.15.100:0/4218314680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6db6592f0 msgr2=0x55c6db659770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:37:55.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.013+0000 7fb305d9b700 1 --2- 172.21.15.100:0/4218314680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6db6592f0 0x55c6db659770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:55.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.013+0000 7fb305d9b700 1 -- 172.21.15.100:0/4218314680 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6db535690 con 0x55c6db654440 2024-05-10T11:37:55.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.013+0000 7fb306d9d700 1 --2- 172.21.15.100:0/4218314680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6db6592f0 0x55c6db659770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:37:55.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.013+0000 7fb305d9b700 1 --2- 172.21.15.100:0/4218314680 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6db654440 0x55c6db658db0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc00a010 tx=0x7fb2fc0038b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97493c8db52a91c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:37:55.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.014+0000 7fb2edffb700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:37:55.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.014+0000 7fb2edffb700 1 -- 172.21.15.100:0/4218314680 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2fc027070 con 0x55c6db654440 2024-05-10T11:37:55.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.014+0000 7fb31751bb80 1 -- 172.21.15.100:0/4218314680 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2fc0043d0 con 0x55c6db654440 2024-05-10T11:37:55.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.014+0000 7fb2edffb700 1 -- 172.21.15.100:0/4218314680 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fb2fc005240 con 0x55c6db654440 2024-05-10T11:37:55.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.014+0000 7fb2edffb700 1 -- 172.21.15.100:0/4218314680 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fb2fc033050 con 0x55c6db654440 2024-05-10T11:37:55.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.014+0000 7fb31751bb80 10 client.28648 fetch_fsmap learned FSMap version 41 2024-05-10T11:37:55.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.014+0000 7fb31751bb80 1 -- 172.21.15.100:0/4218314680 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55c6db667530 con 0x55c6db654440 2024-05-10T11:37:55.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.014+0000 7fb2edffb700 1 -- 172.21.15.100:0/4218314680 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb2fc0043d0 con 0x55c6db654440 2024-05-10T11:37:55.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.014+0000 7fb2edffb700 1 -- 172.21.15.100:0/4218314680 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fb2fc004be0 con 0x55c6db654440 2024-05-10T11:37:55.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.014+0000 7fb31751bb80 10 client.28648 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:37:55.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.014+0000 7fb31751bb80 10 client.28648 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:37:55.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.014+0000 7fb31751bb80 20 client.28648 populate_metadata read hostname 'smithi100' 2024-05-10T11:37:55.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.015+0000 7fb31751bb80 1 --2- 172.21.15.100:0/4218314680 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c6db669a60 0x55c6db66be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:55.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.015+0000 7fb31751bb80 4 client.28648 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:37:55.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.015+0000 7fb31751bb80 1 -- 172.21.15.100:0/4218314680 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6db613650 con 0x55c6db669a60 2024-05-10T11:37:55.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.015+0000 7fb30659c700 1 --2- 172.21.15.100:0/4218314680 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c6db669a60 0x55c6db66be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:55.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.015+0000 7fb30659c700 1 --2- 172.21.15.100:0/4218314680 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c6db669a60 0x55c6db66be50 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11b66a517722c3e1 server_cookie=e1d949b7798bef58 in_seq=0 out_seq=0 2024-05-10T11:37:55.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.015+0000 7fb2edffb700 10 client.28648 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:37:55.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.016+0000 7fb2edffb700 1 -- 172.21.15.100:0/4218314680 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c6db613650 con 0x55c6db669a60 2024-05-10T11:37:55.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.016+0000 7fb2edffb700 10 client.28648 handle_command_reply: tid=0 2024-05-10T11:37:55.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.016+0000 7fb31751bb80 1 client.28648 shutdown 2024-05-10T11:37:55.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.016+0000 7fb31751bb80 1 -- 172.21.15.100:0/4218314680 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6db654440 msgr2=0x55c6db658db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:55.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.016+0000 7fb31751bb80 1 --2- 172.21.15.100:0/4218314680 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6db654440 0x55c6db658db0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc00a010 tx=0x7fb2fc0038b0 comp rx=0 tx=0).stop 2024-05-10T11:37:55.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.016+0000 7fb31751bb80 1 -- 172.21.15.100:0/4218314680 shutdown_connections 2024-05-10T11:37:55.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.016+0000 7fb31751bb80 1 --2- 172.21.15.100:0/4218314680 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c6db669a60 0x55c6db66be50 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:55.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.016+0000 7fb31751bb80 1 --2- 172.21.15.100:0/4218314680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6db6592f0 0x55c6db659770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:55.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.016+0000 7fb31751bb80 1 --2- 172.21.15.100:0/4218314680 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6db64fce0 0x55c6db65bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:55.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.016+0000 7fb31751bb80 1 --2- 172.21.15.100:0/4218314680 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6db654440 0x55c6db658db0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:55.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.016+0000 7fb31751bb80 1 -- 172.21.15.100:0/4218314680 >> 172.21.15.100:0/4218314680 conn(0x55c6db573100 msgr2=0x55c6db63c720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:37:55.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.016+0000 7fb2edffb700 0 client.28648 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:37:55.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.017+0000 7fb31751bb80 1 -- 172.21.15.100:0/4218314680 shutdown_connections 2024-05-10T11:37:55.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.017+0000 7fb31751bb80 1 -- 172.21.15.100:0/4218314680 wait complete. 2024-05-10T11:37:55.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.017+0000 7fb31751bb80 20 client.28648 trim_cache size 0 max 16384 2024-05-10T11:37:55.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.037+0000 7fb31751bb80 1 Processor -- start 2024-05-10T11:37:55.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.037+0000 7fb31751bb80 1 -- start start 2024-05-10T11:37:55.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.037+0000 7fb31751bb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6db64fce0 0x55c6db665780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:55.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.038+0000 7fb31751bb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6db654440 0x55c6db671540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:55.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.038+0000 7fb31751bb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6db6592f0 0x55c6db671a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:55.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.038+0000 7fb31751bb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb3081af770 con 0x55c6db6592f0 2024-05-10T11:37:55.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.038+0000 7fb31751bb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb3081ad000 con 0x55c6db64fce0 2024-05-10T11:37:55.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.038+0000 7fb31751bb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb3081aaed0 con 0x55c6db654440 2024-05-10T11:37:55.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.038+0000 7fb30659c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6db64fce0 0x55c6db665780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:55.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.038+0000 7fb30659c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6db64fce0 0x55c6db665780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:35370/0 (socket says 172.21.15.100:35370) 2024-05-10T11:37:55.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.038+0000 7fb30659c700 1 -- 172.21.15.100:0/1867082362 learned_addr learned my addr 172.21.15.100:0/1867082362 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:37:55.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.038+0000 7fb305d9b700 1 --2- 172.21.15.100:0/1867082362 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6db654440 0x55c6db671540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:55.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.038+0000 7fb306d9d700 1 --2- 172.21.15.100:0/1867082362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6db6592f0 0x55c6db671a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:55.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.038+0000 7fb30659c700 1 -- 172.21.15.100:0/1867082362 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6db654440 msgr2=0x55c6db671540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:37:55.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.038+0000 7fb30659c700 1 --2- 172.21.15.100:0/1867082362 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6db654440 0x55c6db671540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:55.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.038+0000 7fb30659c700 1 -- 172.21.15.100:0/1867082362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6db6592f0 msgr2=0x55c6db671a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:37:55.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.038+0000 7fb30659c700 1 --2- 172.21.15.100:0/1867082362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6db6592f0 0x55c6db671a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:55.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.038+0000 7fb30659c700 1 -- 172.21.15.100:0/1867082362 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6db63e810 con 0x55c6db64fce0 2024-05-10T11:37:55.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.038+0000 7fb306d9d700 1 --2- 172.21.15.100:0/1867082362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6db6592f0 0x55c6db671a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:37:55.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.038+0000 7fb30659c700 1 --2- 172.21.15.100:0/1867082362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6db64fce0 0x55c6db665780 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb2f8000c00 tx=0x7fb2f801f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66d9be22273e0d92 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:37:55.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.039+0000 7fb2effff700 1 -- 172.21.15.100:0/1867082362 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f8034700 con 0x55c6db64fce0 2024-05-10T11:37:55.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.039+0000 7fb2effff700 1 -- 172.21.15.100:0/1867082362 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb2f8034e50 con 0x55c6db64fce0 2024-05-10T11:37:55.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.039+0000 7fb2effff700 1 -- 172.21.15.100:0/1867082362 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f800f6c0 con 0x55c6db64fce0 2024-05-10T11:37:55.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.040+0000 7fb31751bb80 1 -- 172.21.15.100:0/1867082362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6db64fce0 msgr2=0x55c6db665780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:55.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.040+0000 7fb31751bb80 1 --2- 172.21.15.100:0/1867082362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6db64fce0 0x55c6db665780 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb2f8000c00 tx=0x7fb2f801f000 comp rx=0 tx=0).stop 2024-05-10T11:37:55.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.040+0000 7fb31751bb80 1 -- 172.21.15.100:0/1867082362 shutdown_connections 2024-05-10T11:37:55.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.040+0000 7fb31751bb80 1 --2- 172.21.15.100:0/1867082362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6db6592f0 0x55c6db671a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:55.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.040+0000 7fb31751bb80 1 --2- 172.21.15.100:0/1867082362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6db64fce0 0x55c6db665780 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:55.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.040+0000 7fb31751bb80 1 --2- 172.21.15.100:0/1867082362 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6db654440 0x55c6db671540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:55.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.040+0000 7fb31751bb80 1 -- 172.21.15.100:0/1867082362 >> 172.21.15.100:0/1867082362 conn(0x55c6db573100 msgr2=0x55c6db641a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:37:55.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.040+0000 7fb31751bb80 1 -- 172.21.15.100:0/1867082362 shutdown_connections 2024-05-10T11:37:55.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.040+0000 7fb31751bb80 1 -- 172.21.15.100:0/1867082362 wait complete. 2024-05-10T11:37:55.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.040+0000 7fb31751bb80 1 -- start start 2024-05-10T11:37:55.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.040+0000 7fb31751bb80 1 Processor -- start 2024-05-10T11:37:55.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.041+0000 7fb31751bb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6db64fce0 0x55c6db65ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:55.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.041+0000 7fb31751bb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6db654440 0x55c6db65d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:55.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.041+0000 7fb31751bb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6db6592f0 0x55c6db6d20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:55.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.041+0000 7fb31751bb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb3081ab4c0 con 0x55c6db654440 2024-05-10T11:37:55.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.041+0000 7fb31751bb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb3081ac930 con 0x55c6db6592f0 2024-05-10T11:37:55.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.041+0000 7fb31751bb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb3081abb90 con 0x55c6db64fce0 2024-05-10T11:37:55.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.041+0000 7fb31751bb80 10 client.0 _ll_register_callbacks cb 0x55c6db531920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:37:55.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.041+0000 7fb30659c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6db64fce0 0x55c6db65ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:55.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.041+0000 7fb30659c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6db64fce0 0x55c6db65ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:45840/0 (socket says 172.21.15.100:45840) 2024-05-10T11:37:55.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.041+0000 7fb306d9d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6db6592f0 0x55c6db6d20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:55.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.041+0000 7fb30659c700 1 -- 172.21.15.100:0/2464496136 learned_addr learned my addr 172.21.15.100:0/2464496136 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:37:55.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.041+0000 7fb305d9b700 1 --2- 172.21.15.100:0/2464496136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6db654440 0x55c6db65d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:55.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.041+0000 7fb306d9d700 1 -- 172.21.15.100:0/2464496136 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6db64fce0 msgr2=0x55c6db65ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:37:55.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.041+0000 7fb306d9d700 1 --2- 172.21.15.100:0/2464496136 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6db64fce0 0x55c6db65ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:55.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.041+0000 7fb306d9d700 1 -- 172.21.15.100:0/2464496136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6db654440 msgr2=0x55c6db65d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:37:55.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.041+0000 7fb306d9d700 1 --2- 172.21.15.100:0/2464496136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6db654440 0x55c6db65d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:55.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.041+0000 7fb306d9d700 1 -- 172.21.15.100:0/2464496136 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6db6d26a0 con 0x55c6db6592f0 2024-05-10T11:37:55.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.041+0000 7fb305d9b700 1 --2- 172.21.15.100:0/2464496136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6db654440 0x55c6db65d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:37:55.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.041+0000 7fb306d9d700 1 --2- 172.21.15.100:0/2464496136 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6db6592f0 0x55c6db6d20e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb3000df680 tx=0x7fb300004f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18624027533b1c95 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:37:55.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.041+0000 7fb2edffb700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:37:55.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.042+0000 7fb2edffb700 1 -- 172.21.15.100:0/2464496136 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3000a9960 con 0x55c6db6592f0 2024-05-10T11:37:55.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.042+0000 7fb2edffb700 1 -- 172.21.15.100:0/2464496136 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fb3000b1db0 con 0x55c6db6592f0 2024-05-10T11:37:55.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.042+0000 7fb31751bb80 1 -- 172.21.15.100:0/2464496136 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2f8034e50 con 0x55c6db6592f0 2024-05-10T11:37:55.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.042+0000 7fb2edffb700 1 -- 172.21.15.100:0/2464496136 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fb3000a9260 con 0x55c6db6592f0 2024-05-10T11:37:55.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.042+0000 7fb31751bb80 10 client.28547 fetch_fsmap learned FSMap version 41 2024-05-10T11:37:55.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.042+0000 7fb31751bb80 1 -- 172.21.15.100:0/2464496136 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55c6db6d39e0 con 0x55c6db6592f0 2024-05-10T11:37:55.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.042+0000 7fb2edffb700 1 -- 172.21.15.100:0/2464496136 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb2f8034e50 con 0x55c6db6592f0 2024-05-10T11:37:55.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.042+0000 7fb2edffb700 1 -- 172.21.15.100:0/2464496136 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fb3000a84c0 con 0x55c6db6592f0 2024-05-10T11:37:55.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.042+0000 7fb31751bb80 10 client.28547 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:37:55.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.042+0000 7fb31751bb80 10 client.28547 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:37:55.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.042+0000 7fb31751bb80 20 client.28547 populate_metadata read hostname 'smithi100' 2024-05-10T11:37:55.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.042+0000 7fb31751bb80 1 --2- 172.21.15.100:0/2464496136 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c6db6d4070 0x55c6db6d6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:37:55.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.042+0000 7fb31751bb80 4 client.28547 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:37:55.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.042+0000 7fb31751bb80 1 -- 172.21.15.100:0/2464496136 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c6db5a7240 con 0x55c6db6d4070 2024-05-10T11:37:55.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.043+0000 7fb30659c700 1 --2- 172.21.15.100:0/2464496136 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c6db6d4070 0x55c6db6d6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:37:55.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.043+0000 7fb30659c700 1 --2- 172.21.15.100:0/2464496136 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c6db6d4070 0x55c6db6d6490 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf67c5160ddc9f98 server_cookie=ef7c8a1ef4b191c0 in_seq=0 out_seq=0 2024-05-10T11:37:55.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.043+0000 7fb2edffb700 10 client.28547 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:37:55.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.045+0000 7fb2edffb700 1 -- 172.21.15.100:0/2464496136 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c6db5a7240 con 0x55c6db6d4070 2024-05-10T11:37:55.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.045+0000 7fb2edffb700 10 client.28547 handle_command_reply: tid=0 2024-05-10T11:37:55.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.045+0000 7fb31751bb80 1 client.28547 shutdown 2024-05-10T11:37:55.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.046+0000 7fb31751bb80 1 -- 172.21.15.100:0/2464496136 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6db6592f0 msgr2=0x55c6db6d20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:55.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.046+0000 7fb31751bb80 1 --2- 172.21.15.100:0/2464496136 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6db6592f0 0x55c6db6d20e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb3000df680 tx=0x7fb300004f90 comp rx=0 tx=0).stop 2024-05-10T11:37:55.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.046+0000 7fb31751bb80 1 -- 172.21.15.100:0/2464496136 shutdown_connections 2024-05-10T11:37:55.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.046+0000 7fb31751bb80 1 --2- 172.21.15.100:0/2464496136 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c6db6d4070 0x55c6db6d6490 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:55.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.046+0000 7fb31751bb80 1 --2- 172.21.15.100:0/2464496136 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6db6592f0 0x55c6db6d20e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:55.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.046+0000 7fb31751bb80 1 --2- 172.21.15.100:0/2464496136 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6db64fce0 0x55c6db65ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:55.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.046+0000 7fb31751bb80 1 --2- 172.21.15.100:0/2464496136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6db654440 0x55c6db65d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:55.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.046+0000 7fb31751bb80 1 -- 172.21.15.100:0/2464496136 >> 172.21.15.100:0/2464496136 conn(0x55c6db573100 msgr2=0x55c6db63c720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:37:55.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.046+0000 7fb2edffb700 0 client.28547 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:37:55.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.046+0000 7fb31751bb80 1 -- 172.21.15.100:0/2464496136 shutdown_connections 2024-05-10T11:37:55.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.046+0000 7fb31751bb80 1 -- 172.21.15.100:0/2464496136 wait complete. 2024-05-10T11:37:55.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.046+0000 7fb31751bb80 20 client.28547 trim_cache size 0 max 16384 2024-05-10T11:37:55.047 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:37:55.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.047+0000 7fb30cc88700 1 -- 172.21.15.100:0/3767470614 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fb2f00b2900 msgr2=0x7fb2f00b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:55.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.047+0000 7fb30cc88700 1 --2- 172.21.15.100:0/3767470614 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fb2f00b2900 0x7fb2f00b4dc0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fb2f800b2e0 tx=0x7fb2f8005fb0 comp rx=0 tx=0).stop 2024-05-10T11:37:55.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.047+0000 7fb30cc88700 1 -- 172.21.15.100:0/3767470614 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb308109500 msgr2=0x7fb30819bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:37:55.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.047+0000 7fb30cc88700 1 --2- 172.21.15.100:0/3767470614 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb308109500 0x7fb30819bb00 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb30000ef30 tx=0x7fb30000c3a0 comp rx=0 tx=0).stop 2024-05-10T11:37:55.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.047+0000 7fb30cc88700 1 -- 172.21.15.100:0/3767470614 shutdown_connections 2024-05-10T11:37:55.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.047+0000 7fb30cc88700 1 --2- 172.21.15.100:0/3767470614 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fb2f00b2900 0x7fb2f00b4dc0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:55.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.047+0000 7fb30cc88700 1 --2- 172.21.15.100:0/3767470614 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb308109500 0x7fb30819bb00 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:55.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.047+0000 7fb30cc88700 1 --2- 172.21.15.100:0/3767470614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb30806ba10 0x7fb3081a1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:55.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.047+0000 7fb30cc88700 1 --2- 172.21.15.100:0/3767470614 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb30806c3d0 0x7fb3081a1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:37:55.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.047+0000 7fb30cc88700 1 -- 172.21.15.100:0/3767470614 >> 172.21.15.100:0/3767470614 conn(0x7fb308079310 msgr2=0x7fb30810e670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:37:55.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.048+0000 7fb30cc88700 1 -- 172.21.15.100:0/3767470614 shutdown_connections 2024-05-10T11:37:55.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:37:55.048+0000 7fb30cc88700 1 -- 172.21.15.100:0/3767470614 wait complete. 2024-05-10T11:37:55.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:37:55 smithi100 ceph-mon[30296]: pgmap v1007: 129 pgs: 129 active+clean; 495 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 3.9 MiB/s wr, 293 op/s 2024-05-10T11:37:55.934 INFO:tasks.workunit.client.0.smithi100.stdout: 10 58754 9.06 MB/sec execute 229 sec latency 76.081 ms 2024-05-10T11:37:56.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:37:55 smithi178 ceph-mon[33359]: pgmap v1007: 129 pgs: 129 active+clean; 495 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 3.9 MiB/s wr, 293 op/s 2024-05-10T11:37:56.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:37:55 smithi188 ceph-mon[34075]: pgmap v1007: 129 pgs: 129 active+clean; 495 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 3.9 MiB/s wr, 293 op/s 2024-05-10T11:37:56.874 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T11:37:56.874 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T11:37:56.935 INFO:tasks.workunit.client.0.smithi100.stdout: 10 58955 9.09 MB/sec execute 230 sec latency 172.886 ms 2024-05-10T11:37:57.222 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:37:57.935 INFO:tasks.workunit.client.0.smithi100.stdout: 10 59113 9.07 MB/sec execute 231 sec latency 196.470 ms 2024-05-10T11:37:58.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:37:57 smithi178 ceph-mon[33359]: pgmap v1008: 129 pgs: 129 active+clean; 499 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 4.2 MiB/s wr, 311 op/s 2024-05-10T11:37:58.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:37:57 smithi188 ceph-mon[34075]: pgmap v1008: 129 pgs: 129 active+clean; 499 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 4.2 MiB/s wr, 311 op/s 2024-05-10T11:37:58.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:37:57 smithi100 ceph-mon[30296]: pgmap v1008: 129 pgs: 129 active+clean; 499 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 4.2 MiB/s wr, 311 op/s 2024-05-10T11:37:58.935 INFO:tasks.workunit.client.0.smithi100.stdout: 10 59258 9.07 MB/sec execute 232 sec latency 83.198 ms 2024-05-10T11:37:59.935 INFO:tasks.workunit.client.0.smithi100.stdout: 10 59415 9.06 MB/sec execute 233 sec latency 114.640 ms 2024-05-10T11:38:00.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:37:59 smithi178 ceph-mon[33359]: pgmap v1009: 129 pgs: 129 active+clean; 501 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 4.1 MiB/s wr, 281 op/s 2024-05-10T11:38:00.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:37:59 smithi188 ceph-mon[34075]: pgmap v1009: 129 pgs: 129 active+clean; 501 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 4.1 MiB/s wr, 281 op/s 2024-05-10T11:38:00.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:37:59 smithi100 ceph-mon[30296]: pgmap v1009: 129 pgs: 129 active+clean; 501 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 4.1 MiB/s wr, 281 op/s 2024-05-10T11:38:00.937 INFO:tasks.workunit.client.0.smithi100.stdout: 10 59588 9.06 MB/sec execute 234 sec latency 165.169 ms 2024-05-10T11:38:01.909 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.905+0000 7f14bab38640 1 -- 172.21.15.100:0/2914738513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14b4101910 msgr2=0x7f14b4101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:01.909 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.905+0000 7f14bab38640 1 --2- 172.21.15.100:0/2914738513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14b4101910 0x7f14b4101d30 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f14a8009900 tx=0x7f14a802a410 comp rx=0 tx=0).stop 2024-05-10T11:38:01.909 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.908+0000 7f14bab38640 1 -- 172.21.15.100:0/2914738513 shutdown_connections 2024-05-10T11:38:01.909 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.908+0000 7f14bab38640 1 --2- 172.21.15.100:0/2914738513 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14b41034c0 0x7f14b4109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:01.909 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.908+0000 7f14bab38640 1 --2- 172.21.15.100:0/2914738513 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14b4102b00 0x7f14b4102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:01.909 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.908+0000 7f14bab38640 1 --2- 172.21.15.100:0/2914738513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14b4101910 0x7f14b4101d30 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:01.909 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.908+0000 7f14bab38640 1 -- 172.21.15.100:0/2914738513 >> 172.21.15.100:0/2914738513 conn(0x7f14b40fce80 msgr2=0x7f14b40ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:01.911 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.910+0000 7f14bab38640 1 -- 172.21.15.100:0/2914738513 shutdown_connections 2024-05-10T11:38:01.911 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.910+0000 7f14bab38640 1 -- 172.21.15.100:0/2914738513 wait complete. 2024-05-10T11:38:01.913 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.910+0000 7f14bab38640 1 Processor -- start 2024-05-10T11:38:01.913 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.911+0000 7f14bab38640 1 -- start start 2024-05-10T11:38:01.913 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.911+0000 7f14bab38640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14b4101910 0x7f14b419f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:01.913 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.911+0000 7f14bab38640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14b4102b00 0x7f14b419fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:01.913 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.911+0000 7f14bab38640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14b41034c0 0x7f14b41a6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:01.914 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.911+0000 7f14bab38640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f14b410d330 con 0x7f14b41034c0 2024-05-10T11:38:01.914 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.911+0000 7f14bab38640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f14b410d1b0 con 0x7f14b4102b00 2024-05-10T11:38:01.914 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.911+0000 7f14bab38640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f14b410d4b0 con 0x7f14b4101910 2024-05-10T11:38:01.914 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.911+0000 7f14b88d4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14b4101910 0x7f14b419f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:01.914 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.911+0000 7f14b88d4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14b4101910 0x7f14b419f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:33136/0 (socket says 172.21.15.100:33136) 2024-05-10T11:38:01.914 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.911+0000 7f14b88d4640 1 -- 172.21.15.100:0/3198514702 learned_addr learned my addr 172.21.15.100:0/3198514702 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:38:01.914 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.911+0000 7f14b90d5640 1 --2- 172.21.15.100:0/3198514702 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14b41034c0 0x7f14b41a6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:01.914 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.911+0000 7f14b3fff640 1 --2- 172.21.15.100:0/3198514702 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14b4102b00 0x7f14b419fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:01.914 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.912+0000 7f14b90d5640 1 -- 172.21.15.100:0/3198514702 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14b4101910 msgr2=0x7f14b419f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:01.914 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.912+0000 7f14b90d5640 1 --2- 172.21.15.100:0/3198514702 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14b4101910 0x7f14b419f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:01.914 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.912+0000 7f14b90d5640 1 -- 172.21.15.100:0/3198514702 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14b4102b00 msgr2=0x7f14b419fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:01.914 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.912+0000 7f14b90d5640 1 --2- 172.21.15.100:0/3198514702 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14b4102b00 0x7f14b419fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:01.914 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.912+0000 7f14b90d5640 1 -- 172.21.15.100:0/3198514702 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14b4196f20 con 0x7f14b41034c0 2024-05-10T11:38:01.914 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.912+0000 7f14b88d4640 1 --2- 172.21.15.100:0/3198514702 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14b4101910 0x7f14b419f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:38:01.914 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.912+0000 7f14b90d5640 1 --2- 172.21.15.100:0/3198514702 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14b41034c0 0x7f14b41a6700 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f14ac009fd0 tx=0x7f14ac00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:01.915 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.913+0000 7f14b1ffb640 1 -- 172.21.15.100:0/3198514702 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ac00ce20 con 0x7f14b41034c0 2024-05-10T11:38:01.915 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.913+0000 7f14b1ffb640 1 -- 172.21.15.100:0/3198514702 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f14ac005020 con 0x7f14b41034c0 2024-05-10T11:38:01.917 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.914+0000 7f14b1ffb640 1 -- 172.21.15.100:0/3198514702 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ac010690 con 0x7f14b41034c0 2024-05-10T11:38:01.917 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.914+0000 7f14bab38640 1 -- 172.21.15.100:0/3198514702 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14b41971b0 con 0x7f14b41034c0 2024-05-10T11:38:01.917 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.914+0000 7f14bab38640 1 -- 172.21.15.100:0/3198514702 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14b4197680 con 0x7f14b41034c0 2024-05-10T11:38:01.919 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.918+0000 7f14bd854740 1 Processor -- start 2024-05-10T11:38:01.919 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.919+0000 7f14b1ffb640 1 -- 172.21.15.100:0/3198514702 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f14ac0051d0 con 0x7f14b41034c0 2024-05-10T11:38:01.920 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.919+0000 7f14bd854740 1 -- start start 2024-05-10T11:38:01.921 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.920+0000 7f14bd854740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f3b001adb0 0x55f3b001b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:01.921 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.920+0000 7f14bd854740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f3b001b7a0 0x55f3b00202a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:01.921 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.920+0000 7f14bd854740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3b0020910 0x55f3b0022da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:01.921 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.920+0000 7f14bd854740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55f3aff7f5d0 con 0x55f3b0020910 2024-05-10T11:38:01.921 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.920+0000 7f14bd854740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f3aff38890 con 0x55f3b001adb0 2024-05-10T11:38:01.921 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.920+0000 7f14bd854740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55f3aff7b590 con 0x55f3b001b7a0 2024-05-10T11:38:01.921 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.920+0000 7f14b3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3b0020910 0x55f3b0022da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:01.921 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.920+0000 7f14b3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3b0020910 0x55f3b0022da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52650/0 (socket says 172.21.15.100:52650) 2024-05-10T11:38:01.921 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.920+0000 7f14b3fff640 1 -- 172.21.15.100:0/3455099895 learned_addr learned my addr 172.21.15.100:0/3455099895 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:38:01.921 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.920+0000 7f14b88d4640 1 --2- 172.21.15.100:0/3455099895 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f3b001b7a0 0x55f3b00202a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:01.921 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.920+0000 7f14b3fff640 1 --2- 172.21.15.100:0/3455099895 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f3b001adb0 0x55f3b001b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:01.922 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.920+0000 7f14b1ffb640 1 --2- 172.21.15.100:0/3198514702 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f14a40b28e0 0x7f14a40b4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:01.923 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.920+0000 7f14b3fff640 1 -- 172.21.15.100:0/3455099895 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f3b001b7a0 msgr2=0x55f3b00202a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:01.923 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.920+0000 7f14b3fff640 1 --2- 172.21.15.100:0/3455099895 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f3b001b7a0 0x55f3b00202a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:01.923 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.920+0000 7f14b3fff640 1 -- 172.21.15.100:0/3455099895 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f3b001adb0 msgr2=0x55f3b001b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:01.923 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.920+0000 7f14b3fff640 1 --2- 172.21.15.100:0/3455099895 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f3b001adb0 0x55f3b001b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:01.923 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.921+0000 7f14b3fff640 1 -- 172.21.15.100:0/3455099895 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3afc91210 con 0x55f3b0020910 2024-05-10T11:38:01.923 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.921+0000 7f14b1ffb640 1 -- 172.21.15.100:0/3198514702 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f14ac0e26d0 con 0x7f14b41034c0 2024-05-10T11:38:01.923 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.921+0000 7f14b90d5640 1 --2- 172.21.15.100:0/3198514702 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f14a40b28e0 0x7f14a40b4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:01.923 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.921+0000 7f14b3fff640 1 --2- 172.21.15.100:0/3455099895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3b0020910 0x55f3b0022da0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f14a0033850 tx=0x7f14a00097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=872712318f9cecfc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:01.923 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.921+0000 7f14b90d5640 1 --2- 172.21.15.100:0/3198514702 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f14a40b28e0 0x7f14a40b4da0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f14ac0a59b0 tx=0x7f14ac005f70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:01.923 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.921+0000 7f149effd640 1 -- 172.21.15.100:0/3455099895 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a00317d0 con 0x55f3b0020910 2024-05-10T11:38:01.923 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.921+0000 7f149effd640 1 -- 172.21.15.100:0/3455099895 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f14a0031e00 con 0x55f3b0020910 2024-05-10T11:38:01.923 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.921+0000 7f149effd640 1 -- 172.21.15.100:0/3455099895 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a0041cd0 con 0x55f3b0020910 2024-05-10T11:38:01.928 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.922+0000 7f14bd854740 1 -- 172.21.15.100:0/3455099895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3b0020910 msgr2=0x55f3b0022da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:01.928 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.922+0000 7f14bd854740 1 --2- 172.21.15.100:0/3455099895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3b0020910 0x55f3b0022da0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f14a0033850 tx=0x7f14a00097d0 comp rx=0 tx=0).stop 2024-05-10T11:38:01.928 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.922+0000 7f14bd854740 1 -- 172.21.15.100:0/3455099895 shutdown_connections 2024-05-10T11:38:01.928 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.922+0000 7f14bd854740 1 --2- 172.21.15.100:0/3455099895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3b0020910 0x55f3b0022da0 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:01.928 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.922+0000 7f14bd854740 1 --2- 172.21.15.100:0/3455099895 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f3b001b7a0 0x55f3b00202a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:01.928 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.922+0000 7f14bd854740 1 --2- 172.21.15.100:0/3455099895 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f3b001adb0 0x55f3b001b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:01.928 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.922+0000 7f14bd854740 1 -- 172.21.15.100:0/3455099895 >> 172.21.15.100:0/3455099895 conn(0x55f3b0016420 msgr2=0x55f3b0016820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:01.928 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.922+0000 7f14bd854740 1 -- 172.21.15.100:0/3455099895 shutdown_connections 2024-05-10T11:38:01.928 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.922+0000 7f14bd854740 1 -- 172.21.15.100:0/3455099895 wait complete. 2024-05-10T11:38:01.928 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.922+0000 7f14bd854740 1 -- start start 2024-05-10T11:38:01.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.923+0000 7f14bd854740 1 Processor -- start 2024-05-10T11:38:01.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.923+0000 7f14bd854740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3b001adb0 0x55f3b0025010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:01.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.923+0000 7f14bd854740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f3b001b7a0 0x55f3b0025550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:01.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.923+0000 7f14bd854740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f3b0020910 0x55f3b002c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:01.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.923+0000 7f14bd854740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55f3b00238c0 con 0x55f3b001adb0 2024-05-10T11:38:01.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.923+0000 7f14bd854740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f3aff3c370 con 0x55f3b0020910 2024-05-10T11:38:01.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.923+0000 7f14bd854740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55f3b0023a40 con 0x55f3b001b7a0 2024-05-10T11:38:01.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.923+0000 7f14bd854740 10 client.0 _ll_register_callbacks cb 0x55f3afc90a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:38:01.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.923+0000 7f14b3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3b001adb0 0x55f3b0025010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:01.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.923+0000 7f14b3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3b001adb0 0x55f3b0025010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52658/0 (socket says 172.21.15.100:52658) 2024-05-10T11:38:01.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.923+0000 7f14b3fff640 1 -- 172.21.15.100:0/2819335256 learned_addr learned my addr 172.21.15.100:0/2819335256 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:38:01.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.923+0000 7f14b88d4640 1 --2- 172.21.15.100:0/2819335256 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f3b001b7a0 0x55f3b0025550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:01.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.923+0000 7f14b3fff640 1 --2- 172.21.15.100:0/2819335256 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f3b0020910 0x55f3b002c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:01.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.923+0000 7f14b3fff640 1 -- 172.21.15.100:0/2819335256 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f3b001b7a0 msgr2=0x55f3b0025550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:01.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.923+0000 7f14b3fff640 1 --2- 172.21.15.100:0/2819335256 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f3b001b7a0 0x55f3b0025550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:01.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.923+0000 7f14b3fff640 1 -- 172.21.15.100:0/2819335256 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f3b0020910 msgr2=0x55f3b002c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:01.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.923+0000 7f14b3fff640 1 --2- 172.21.15.100:0/2819335256 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f3b0020910 0x55f3b002c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:01.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.923+0000 7f14b3fff640 1 -- 172.21.15.100:0/2819335256 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3afd5d620 con 0x55f3b001adb0 2024-05-10T11:38:01.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.924+0000 7f14b3fff640 1 --2- 172.21.15.100:0/2819335256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3b001adb0 0x55f3b0025010 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f14a0032010 tx=0x7f14a0001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2dd2432dd765aff server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:01.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.924+0000 7f1493fff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:38:01.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.924+0000 7f1493fff640 1 -- 172.21.15.100:0/2819335256 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a0052060 con 0x55f3b001adb0 2024-05-10T11:38:01.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.924+0000 7f1493fff640 1 -- 172.21.15.100:0/2819335256 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f14a0041440 con 0x55f3b001adb0 2024-05-10T11:38:01.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.924+0000 7f14bd854740 1 -- 172.21.15.100:0/2819335256 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14a0031e00 con 0x55f3b001adb0 2024-05-10T11:38:01.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.924+0000 7f1493fff640 1 -- 172.21.15.100:0/2819335256 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f14a0059050 con 0x55f3b001adb0 2024-05-10T11:38:01.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.924+0000 7f14bd854740 10 client.19272 fetch_fsmap learned FSMap version 41 2024-05-10T11:38:01.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.924+0000 7f14bd854740 1 -- 172.21.15.100:0/2819335256 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55f3b002f570 con 0x55f3b001adb0 2024-05-10T11:38:01.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.924+0000 7f1493fff640 1 -- 172.21.15.100:0/2819335256 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f14a0031e00 con 0x55f3b001adb0 2024-05-10T11:38:01.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.924+0000 7f1493fff640 1 -- 172.21.15.100:0/2819335256 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f14a004c030 con 0x55f3b001adb0 2024-05-10T11:38:01.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.924+0000 7f14bd854740 10 client.19272 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:38:01.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.924+0000 7f14bd854740 10 client.19272 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:38:01.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.924+0000 7f14bd854740 20 client.19272 populate_metadata read hostname 'smithi100' 2024-05-10T11:38:01.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.924+0000 7f14bd854740 1 --2- 172.21.15.100:0/2819335256 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55f3b00309a0 0x55f3b0032d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:01.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.924+0000 7f14bd854740 4 client.19272 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:38:01.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.924+0000 7f14bd854740 1 -- 172.21.15.100:0/2819335256 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3aff8d1e0 con 0x55f3b00309a0 2024-05-10T11:38:01.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.925+0000 7f14b88d4640 1 --2- 172.21.15.100:0/2819335256 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55f3b00309a0 0x55f3b0032d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:01.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.925+0000 7f14b88d4640 1 --2- 172.21.15.100:0/2819335256 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55f3b00309a0 0x55f3b0032d90 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=302e5814a478a825 server_cookie=526e2dfdb5635eed in_seq=0 out_seq=0 2024-05-10T11:38:01.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.925+0000 7f1493fff640 10 client.19272 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:38:01.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.925+0000 7f1493fff640 1 -- 172.21.15.100:0/2819335256 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f3aff8d1e0 con 0x55f3b00309a0 2024-05-10T11:38:01.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.925+0000 7f1493fff640 10 client.19272 handle_command_reply: tid=0 2024-05-10T11:38:01.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.926+0000 7f14bd854740 1 client.19272 shutdown 2024-05-10T11:38:01.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.926+0000 7f14bd854740 1 -- 172.21.15.100:0/2819335256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3b001adb0 msgr2=0x55f3b0025010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:01.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.926+0000 7f14bd854740 1 --2- 172.21.15.100:0/2819335256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3b001adb0 0x55f3b0025010 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f14a0032010 tx=0x7f14a0001e70 comp rx=0 tx=0).stop 2024-05-10T11:38:01.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.926+0000 7f14bd854740 1 -- 172.21.15.100:0/2819335256 shutdown_connections 2024-05-10T11:38:01.932 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.926+0000 7f14bd854740 1 --2- 172.21.15.100:0/2819335256 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55f3b00309a0 0x55f3b0032d90 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:01.932 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.926+0000 7f14bd854740 1 --2- 172.21.15.100:0/2819335256 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f3b0020910 0x55f3b002c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:01.932 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.926+0000 7f14bd854740 1 --2- 172.21.15.100:0/2819335256 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f3b001b7a0 0x55f3b0025550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:01.932 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.926+0000 7f14bd854740 1 --2- 172.21.15.100:0/2819335256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3b001adb0 0x55f3b0025010 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:01.932 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.926+0000 7f1493fff640 0 client.19272 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:38:01.932 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.926+0000 7f14bd854740 1 -- 172.21.15.100:0/2819335256 >> 172.21.15.100:0/2819335256 conn(0x55f3b0016420 msgr2=0x55f3b0015580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:01.932 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.926+0000 7f14bd854740 1 -- 172.21.15.100:0/2819335256 shutdown_connections 2024-05-10T11:38:01.932 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.926+0000 7f14bd854740 1 -- 172.21.15.100:0/2819335256 wait complete. 2024-05-10T11:38:01.932 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.926+0000 7f14bd854740 20 client.19272 trim_cache size 0 max 16384 2024-05-10T11:38:01.936 INFO:tasks.workunit.client.0.smithi100.stdout: 10 59700 9.04 MB/sec execute 235 sec latency 187.979 ms 2024-05-10T11:38:01.946 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.941+0000 7f14bd854740 1 Processor -- start 2024-05-10T11:38:01.946 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.941+0000 7f14bd854740 1 -- start start 2024-05-10T11:38:01.946 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.941+0000 7f14bd854740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f3b001adb0 0x55f3b0019410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:01.946 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.941+0000 7f14bd854740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f3b001b7a0 0x55f3b002c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:01.946 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.941+0000 7f14bd854740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3b0020910 0x55f3b0019950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:01.946 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.941+0000 7f14bd854740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f14b41ab830 con 0x55f3b0020910 2024-05-10T11:38:01.946 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.941+0000 7f14bd854740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f14b41a9100 con 0x55f3b001b7a0 2024-05-10T11:38:01.946 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.941+0000 7f14bd854740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f14b419f020 con 0x55f3b001adb0 2024-05-10T11:38:01.946 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.941+0000 7f14b3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3b0020910 0x55f3b0019950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:01.946 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.941+0000 7f14b3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3b0020910 0x55f3b0019950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52660/0 (socket says 172.21.15.100:52660) 2024-05-10T11:38:01.946 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.941+0000 7f14b3fff640 1 -- 172.21.15.100:0/4203390318 learned_addr learned my addr 172.21.15.100:0/4203390318 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:38:01.946 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.941+0000 7f14b3fff640 1 --2- 172.21.15.100:0/4203390318 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f3b001adb0 0x55f3b0019410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:01.947 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.941+0000 7f14b88d4640 1 --2- 172.21.15.100:0/4203390318 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f3b001b7a0 0x55f3b002c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:01.947 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.941+0000 7f14b3fff640 1 -- 172.21.15.100:0/4203390318 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f3b001adb0 msgr2=0x55f3b0019410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:01.947 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.941+0000 7f14b3fff640 1 --2- 172.21.15.100:0/4203390318 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f3b001adb0 0x55f3b0019410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:01.947 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.941+0000 7f14b3fff640 1 -- 172.21.15.100:0/4203390318 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f3b001b7a0 msgr2=0x55f3b002c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:01.947 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.941+0000 7f14b3fff640 1 --2- 172.21.15.100:0/4203390318 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f3b001b7a0 0x55f3b002c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:01.947 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.941+0000 7f14b3fff640 1 -- 172.21.15.100:0/4203390318 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3b0016250 con 0x55f3b0020910 2024-05-10T11:38:01.947 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.941+0000 7f14b3fff640 1 --2- 172.21.15.100:0/4203390318 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3b0020910 0x55f3b0019950 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f14a0033750 tx=0x7f14a0009cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b22581e8b05c3dc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:01.947 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.941+0000 7f149e7fc640 1 -- 172.21.15.100:0/4203390318 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a0052040 con 0x55f3b0020910 2024-05-10T11:38:01.947 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.941+0000 7f149e7fc640 1 -- 172.21.15.100:0/4203390318 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f14a0031db0 con 0x55f3b0020910 2024-05-10T11:38:01.947 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.941+0000 7f149e7fc640 1 -- 172.21.15.100:0/4203390318 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a0041440 con 0x55f3b0020910 2024-05-10T11:38:01.947 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.942+0000 7f14bd854740 1 -- 172.21.15.100:0/4203390318 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3b0020910 msgr2=0x55f3b0019950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:01.947 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.942+0000 7f14bd854740 1 --2- 172.21.15.100:0/4203390318 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3b0020910 0x55f3b0019950 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f14a0033750 tx=0x7f14a0009cc0 comp rx=0 tx=0).stop 2024-05-10T11:38:01.947 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.942+0000 7f14bd854740 1 -- 172.21.15.100:0/4203390318 shutdown_connections 2024-05-10T11:38:01.947 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.942+0000 7f14bd854740 1 --2- 172.21.15.100:0/4203390318 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3b0020910 0x55f3b0019950 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:01.947 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.942+0000 7f14bd854740 1 --2- 172.21.15.100:0/4203390318 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f3b001b7a0 0x55f3b002c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:01.948 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.942+0000 7f14bd854740 1 --2- 172.21.15.100:0/4203390318 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f3b001adb0 0x55f3b0019410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:01.948 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.942+0000 7f14bd854740 1 -- 172.21.15.100:0/4203390318 >> 172.21.15.100:0/4203390318 conn(0x55f3b0016420 msgr2=0x55f3b001ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:01.948 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.942+0000 7f14bd854740 1 -- 172.21.15.100:0/4203390318 shutdown_connections 2024-05-10T11:38:01.948 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.942+0000 7f14bd854740 1 -- 172.21.15.100:0/4203390318 wait complete. 2024-05-10T11:38:01.948 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.942+0000 7f14bd854740 1 -- start start 2024-05-10T11:38:01.948 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.942+0000 7f14bd854740 1 Processor -- start 2024-05-10T11:38:01.948 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.943+0000 7f14bd854740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3b001adb0 0x55f3b00291d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:01.948 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.943+0000 7f14bd854740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f3b001b7a0 0x55f3b0029710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:01.948 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.943+0000 7f14bd854740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f3b0020910 0x55f3b0099240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:01.948 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.943+0000 7f14bd854740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f14b41a75e0 con 0x55f3b001adb0 2024-05-10T11:38:01.948 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.943+0000 7f14bd854740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f14b41a8a30 con 0x55f3b001b7a0 2024-05-10T11:38:01.948 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.943+0000 7f14bd854740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f14b41a7c90 con 0x55f3b0020910 2024-05-10T11:38:01.948 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.943+0000 7f14bd854740 10 client.0 _ll_register_callbacks cb 0x7f14a0009ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:38:01.948 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.943+0000 7f14b3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3b001adb0 0x55f3b00291d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:01.948 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.943+0000 7f14b3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3b001adb0 0x55f3b00291d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52672/0 (socket says 172.21.15.100:52672) 2024-05-10T11:38:01.948 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.943+0000 7f14b3fff640 1 -- 172.21.15.100:0/3772159860 learned_addr learned my addr 172.21.15.100:0/3772159860 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:38:01.948 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.943+0000 7f14b3fff640 1 --2- 172.21.15.100:0/3772159860 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f3b0020910 0x55f3b0099240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:01.949 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.943+0000 7f14b3fff640 1 -- 172.21.15.100:0/3772159860 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f3b0020910 msgr2=0x55f3b0099240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:01.949 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.943+0000 7f14b3fff640 1 --2- 172.21.15.100:0/3772159860 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f3b0020910 0x55f3b0099240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:01.949 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.943+0000 7f14b3fff640 1 -- 172.21.15.100:0/3772159860 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f3b001b7a0 msgr2=0x55f3b0029710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:01.949 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.943+0000 7f14b3fff640 1 --2- 172.21.15.100:0/3772159860 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f3b001b7a0 0x55f3b0029710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:01.949 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.943+0000 7f14b3fff640 1 -- 172.21.15.100:0/3772159860 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3b002c790 con 0x55f3b001adb0 2024-05-10T11:38:01.949 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.943+0000 7f14b3fff640 1 --2- 172.21.15.100:0/3772159860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3b001adb0 0x55f3b00291d0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f14a00058c0 tx=0x7f14a0005420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e0e7e375ac7a57f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:01.949 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.943+0000 7f1493fff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:38:01.949 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.943+0000 7f1493fff640 1 -- 172.21.15.100:0/3772159860 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a0052040 con 0x55f3b001adb0 2024-05-10T11:38:01.949 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.943+0000 7f1493fff640 1 -- 172.21.15.100:0/3772159860 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f14a0005af0 con 0x55f3b001adb0 2024-05-10T11:38:01.949 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.943+0000 7f14bd854740 1 -- 172.21.15.100:0/3772159860 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14a0031db0 con 0x55f3b001adb0 2024-05-10T11:38:01.949 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.943+0000 7f1493fff640 1 -- 172.21.15.100:0/3772159860 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f14a0005d10 con 0x55f3b001adb0 2024-05-10T11:38:01.949 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.943+0000 7f14bd854740 10 client.19278 fetch_fsmap learned FSMap version 41 2024-05-10T11:38:01.949 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.943+0000 7f14bd854740 1 -- 172.21.15.100:0/3772159860 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55f3b009a760 con 0x55f3b001adb0 2024-05-10T11:38:01.949 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.943+0000 7f1493fff640 1 -- 172.21.15.100:0/3772159860 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f14a0031db0 con 0x55f3b001adb0 2024-05-10T11:38:01.950 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.944+0000 7f1493fff640 1 -- 172.21.15.100:0/3772159860 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f14a005c020 con 0x55f3b001adb0 2024-05-10T11:38:01.950 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.944+0000 7f14bd854740 10 client.19278 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:38:01.950 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.944+0000 7f14bd854740 10 client.19278 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:38:01.950 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.944+0000 7f14bd854740 20 client.19278 populate_metadata read hostname 'smithi100' 2024-05-10T11:38:01.950 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.944+0000 7f14bd854740 1 --2- 172.21.15.100:0/3772159860 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55f3b009af70 0x55f3b009d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:01.950 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.944+0000 7f14bd854740 4 client.19278 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:38:01.950 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.944+0000 7f14bd854740 1 -- 172.21.15.100:0/3772159860 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f3aff87050 con 0x55f3b009af70 2024-05-10T11:38:01.950 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.944+0000 7f14b88d4640 1 --2- 172.21.15.100:0/3772159860 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55f3b009af70 0x55f3b009d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:01.950 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.944+0000 7f1493fff640 10 client.19278 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:38:01.950 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:01.944+0000 7f14b88d4640 1 --2- 172.21.15.100:0/3772159860 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55f3b009af70 0x55f3b009d390 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a63d8196871a30b7 server_cookie=163a12e8a476c928 in_seq=0 out_seq=0 2024-05-10T11:38:02.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:01 smithi178 ceph-mon[33359]: pgmap v1010: 129 pgs: 129 active+clean; 501 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 4.6 MiB/s wr, 300 op/s 2024-05-10T11:38:02.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:01 smithi188 ceph-mon[34075]: pgmap v1010: 129 pgs: 129 active+clean; 501 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 4.6 MiB/s wr, 300 op/s 2024-05-10T11:38:02.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:01 smithi100 ceph-mon[30296]: pgmap v1010: 129 pgs: 129 active+clean; 501 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 4.6 MiB/s wr, 300 op/s 2024-05-10T11:38:02.054 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:02.052+0000 7f1493fff640 1 -- 172.21.15.100:0/3772159860 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f3aff87050 con 0x55f3b009af70 2024-05-10T11:38:02.054 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:02.052+0000 7f1493fff640 10 client.19278 handle_command_reply: tid=0 2024-05-10T11:38:02.054 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:02.052+0000 7f14bd854740 1 client.19278 shutdown 2024-05-10T11:38:02.054 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:02.052+0000 7f14bd854740 1 -- 172.21.15.100:0/3772159860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3b001adb0 msgr2=0x55f3b00291d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:02.054 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:02.052+0000 7f14bd854740 1 --2- 172.21.15.100:0/3772159860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3b001adb0 0x55f3b00291d0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f14a00058c0 tx=0x7f14a0005420 comp rx=0 tx=0).stop 2024-05-10T11:38:02.054 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:02.052+0000 7f14bd854740 1 -- 172.21.15.100:0/3772159860 shutdown_connections 2024-05-10T11:38:02.054 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:02.052+0000 7f14bd854740 1 --2- 172.21.15.100:0/3772159860 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55f3b009af70 0x55f3b009d390 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:02.054 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:02.052+0000 7f14bd854740 1 --2- 172.21.15.100:0/3772159860 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f3b0020910 0x55f3b0099240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:02.054 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:02.052+0000 7f14bd854740 1 --2- 172.21.15.100:0/3772159860 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f3b001b7a0 0x55f3b0029710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:02.054 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:02.052+0000 7f14bd854740 1 --2- 172.21.15.100:0/3772159860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3b001adb0 0x55f3b00291d0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:02.054 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:02.052+0000 7f14bd854740 1 -- 172.21.15.100:0/3772159860 >> 172.21.15.100:0/3772159860 conn(0x55f3b0016420 msgr2=0x55f3b0019410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:02.054 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:02.052+0000 7f1493fff640 0 client.19278 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:38:02.055 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:02.052+0000 7f14bd854740 1 -- 172.21.15.100:0/3772159860 shutdown_connections 2024-05-10T11:38:02.055 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:02.052+0000 7f14bd854740 1 -- 172.21.15.100:0/3772159860 wait complete. 2024-05-10T11:38:02.055 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:02.052+0000 7f14bd854740 20 client.19278 trim_cache size 0 max 16384 2024-05-10T11:38:02.055 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:38:02.055 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:38:02.055 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:38:02.055 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:38:02.056 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:02.053+0000 7f14bab38640 1 -- 172.21.15.100:0/3198514702 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f14a40b28e0 msgr2=0x7f14a40b4da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:02.056 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:02.053+0000 7f14bab38640 1 --2- 172.21.15.100:0/3198514702 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f14a40b28e0 0x7f14a40b4da0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f14ac0a59b0 tx=0x7f14ac005f70 comp rx=0 tx=0).stop 2024-05-10T11:38:02.056 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:02.053+0000 7f14bab38640 1 -- 172.21.15.100:0/3198514702 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14b41034c0 msgr2=0x7f14b41a6700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:02.056 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:02.053+0000 7f14bab38640 1 --2- 172.21.15.100:0/3198514702 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14b41034c0 0x7f14b41a6700 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f14ac009fd0 tx=0x7f14ac00ef10 comp rx=0 tx=0).stop 2024-05-10T11:38:02.056 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:02.053+0000 7f14bab38640 1 -- 172.21.15.100:0/3198514702 shutdown_connections 2024-05-10T11:38:02.056 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:02.053+0000 7f14bab38640 1 --2- 172.21.15.100:0/3198514702 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f14a40b28e0 0x7f14a40b4da0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:02.056 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:02.053+0000 7f14bab38640 1 --2- 172.21.15.100:0/3198514702 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14b41034c0 0x7f14b41a6700 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:02.056 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:02.053+0000 7f14bab38640 1 --2- 172.21.15.100:0/3198514702 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14b4102b00 0x7f14b419fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:02.056 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:02.053+0000 7f14bab38640 1 --2- 172.21.15.100:0/3198514702 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14b4101910 0x7f14b419f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:02.056 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:02.053+0000 7f14bab38640 1 -- 172.21.15.100:0/3198514702 >> 172.21.15.100:0/3198514702 conn(0x7f14b40fce80 msgr2=0x7f14b40fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:02.056 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:02.054+0000 7f14bab38640 1 -- 172.21.15.100:0/3198514702 shutdown_connections 2024-05-10T11:38:02.056 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:02.054+0000 7f14bab38640 1 -- 172.21.15.100:0/3198514702 wait complete. 2024-05-10T11:38:02.937 INFO:tasks.workunit.client.0.smithi100.stdout: 10 59804 9.02 MB/sec execute 236 sec latency 169.622 ms 2024-05-10T11:38:03.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:03 smithi100 ceph-mon[30296]: pgmap v1011: 129 pgs: 129 active+clean; 498 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 4.5 MiB/s wr, 243 op/s 2024-05-10T11:38:03.937 INFO:tasks.workunit.client.0.smithi100.stdout: 10 59947 9.01 MB/sec execute 237 sec latency 166.418 ms 2024-05-10T11:38:04.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:03 smithi178 ceph-mon[33359]: pgmap v1011: 129 pgs: 129 active+clean; 498 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 4.5 MiB/s wr, 243 op/s 2024-05-10T11:38:04.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:03 smithi188 ceph-mon[34075]: pgmap v1011: 129 pgs: 129 active+clean; 498 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 4.5 MiB/s wr, 243 op/s 2024-05-10T11:38:04.029 INFO:journalctl@ceph.mds.i.smithi188.stdout:May 10 11:38:03 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-i[64554]: 2024-05-10T11:38:03.715+0000 7fd152433700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000487 [...2,head] /volumes/qa/sv_0/96765652-39ef-4cf3-b20a-a70f7e9d774d/client.0/tmp/clients/client7/~dmtmp/EXCEL/ auth{2=1} v8521 ap=2 f(v456 m2024-05-10T11:37:52.689513+0000 2=2+0) n(v26 rc2024-05-10T11:37:52.689513+0000 b520192 3=2+1) (inest sync->mix g=2) caps={15207=pAsLsXsFs/-@7457} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x56080179c580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000487:[<0x10000000420/EXCEL v14>,<0x10000000412/~dmtmp v34>,<0x10000000007/client7 v1940>,<0x10000000006/clients v1743>,<0x10000000005/tmp v1642>,<0x10000000003/client.0 v1616>,<0x10000000002/96765652-39ef-4cf3-b20a-a70f7e9d774d v1609>,<0x10000000001/sv_0 v1620>,<0x10000000000/qa v1598>,<0x1/volumes v1566>]//[]","memoryvalue":"(2)0x10000000487:[<0x10000000420/EXCEL v8521>,<0x10000000412/~dmtmp v7314>,<0x10000000007/client7 v5583>,<0x10000000006/clients v2999>,<0x10000000005/tmp v2365>,<0x10000000003/client.0 v2330>,<0x10000000002/96765652-39ef-4cf3-b20a-a70f7e9d774d v2320>,<0x10000000001/sv_0 v2326>,<0x10000000000/qa v2300>,<0x1/volumes v2260>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-10T11:37:52.663131+0000 3=3+0)","ondisk_value.rstat":"n(v0 rc2024-05-10T11:37:52.689513+0000 b520192 3=2+1)","memory_value.dirstat":"f(v456 m2024-05-10T11:37:52.689513+0000 2=2+0)","memory_value.rstat":"n(v26 rc2024-05-10T11:37:52.689513+0000 b520192 3=2+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-05-10T11:38:04.029 INFO:journalctl@ceph.mds.i.smithi188.stdout:May 10 11:38:03 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-i[64554]: 2024-05-10T11:38:03.718+0000 7fd152433700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000544 [...2,head] /volumes/qa/sv_0/96765652-39ef-4cf3-b20a-a70f7e9d774d/client.0/tmp/clients/client7/~dmtmp/WORDPRO/ auth{2=1} v8527 ap=2 f(v420 m2024-05-10T11:37:53.042038+0000 3=3+0) n(v27 rc2024-05-10T11:37:53.042038+0000 b780288 4=3+1) (inest sync->mix g=2) caps={15207=pAsLsXsFs/-@7365} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x560801875080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000544:[<0x10000000420/WORDPRO v58>,<0x10000000412/~dmtmp v34>,<0x10000000007/client7 v1940>,<0x10000000006/clients v1743>,<0x10000000005/tmp v1642>,<0x10000000003/client.0 v1616>,<0x10000000002/96765652-39ef-4cf3-b20a-a70f7e9d774d v1609>,<0x10000000001/sv_0 v1620>,<0x10000000000/qa v1598>,<0x1/volumes v1566>]//[]","memoryvalue":"(2)0x10000000544:[<0x10000000420/WORDPRO v8527>,<0x10000000412/~dmtmp v7314>,<0x10000000007/client7 v5583>,<0x10000000006/clients v2999>,<0x10000000005/tmp v2365>,<0x10000000003/client.0 v2330>,<0x10000000002/96765652-39ef-4cf3-b20a-a70f7e9d774d v2320>,<0x10000000001/sv_0 v2326>,<0x10000000000/qa v2300>,<0x1/volumes v2260>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-10T11:37:53.016067+0000 4=4+0)","ondisk_value.rstat":"n(v0 rc2024-05-10T11:37:53.042038+0000 b780288 4=3+1)","memory_value.dirstat":"f(v420 m2024-05-10T11:37:53.042038+0000 3=3+0)","memory_value.rstat":"n(v27 rc2024-05-10T11:37:53.042038+0000 b780288 4=3+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-05-10T11:38:04.029 INFO:journalctl@ceph.mds.i.smithi188.stdout:May 10 11:38:03 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-i[64554]: 2024-05-10T11:38:03.801+0000 7fd152433700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000064b [...2,head] /volumes/qa/sv_0/96765652-39ef-4cf3-b20a-a70f7e9d774d/client.0/tmp/clients/client7/~dmtmp/ACCESS/ auth{2=1} v8533 ap=2 f(v168 m2024-05-10T11:37:53.317485+0000 1=1+0) n(v24 rc2024-05-10T11:37:53.317485+0000 b260096 2=1+1) (inest sync->mix g=2) caps={15207=pAsLsXsFs/-@6276} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x560801a31180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000064b:[<0x10000000420/ACCESS v112>,<0x10000000412/~dmtmp v34>,<0x10000000007/client7 v1940>,<0x10000000006/clients v1743>,<0x10000000005/tmp v1642>,<0x10000000003/client.0 v1616>,<0x10000000002/96765652-39ef-4cf3-b20a-a70f7e9d774d v1609>,<0x10000000001/sv_0 v1620>,<0x10000000000/qa v1598>,<0x1/volumes v1566>]//[]","memoryvalue":"(2)0x1000000064b:[<0x10000000420/ACCESS v8533>,<0x10000000412/~dmtmp v7326>,<0x10000000007/client7 v5583>,<0x10000000006/clients v2999>,<0x10000000005/tmp v2365>,<0x10000000003/client.0 v2330>,<0x10000000002/96765652-39ef-4cf3-b20a-a70f7e9d774d v2320>,<0x10000000001/sv_0 v2326>,<0x10000000000/qa v2300>,<0x1/volumes v2260>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-10T11:37:53.285840+0000 2=2+0)","ondisk_value.rstat":"n(v0 rc2024-05-10T11:37:53.317485+0000 b260096 2=1+1)","memory_value.dirstat":"f(v168 m2024-05-10T11:37:53.317485+0000 1=1+0)","memory_value.rstat":"n(v24 rc2024-05-10T11:37:53.317485+0000 b260096 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-05-10T11:38:04.029 INFO:journalctl@ceph.mds.i.smithi188.stdout:May 10 11:38:03 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-i[64554]: 2024-05-10T11:38:03.806+0000 7fd152433700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000566 [...2,head] /volumes/qa/sv_0/96765652-39ef-4cf3-b20a-a70f7e9d774d/client.0/tmp/clients/client9/~dmtmp/PM/ auth{2=1} v17814 ap=2 f(v1036 m2024-05-10T11:37:47.223222+0000 3=3+0) n(v30 rc2024-05-10T11:37:47.223222+0000 b780288 4=3+1) (inest sync->mix g=2) caps={15207=pAsLsXsFs/-@11074} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x5608018c1600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000566:[<0x1000000041e/PM v66>,<0x1000000040f/~dmtmp v32>,<0x10000000007/client9 v1920>,<0x10000000006/clients v1743>,<0x10000000005/tmp v1642>,<0x10000000003/client.0 v1616>,<0x10000000002/96765652-39ef-4cf3-b20a-a70f7e9d774d v1609>,<0x10000000001/sv_0 v1620>,<0x10000000000/qa v1598>,<0x1/volumes v1566>]//[]","memoryvalue":"(2)0x10000000566:[<0x1000000041e/PM v17814>,<0x1000000040f/~dmtmp v2841>,<0x10000000007/client9 v5587>,<0x10000000006/clients v2999>,<0x10000000005/tmp v2365>,<0x10000000003/client.0 v2330>,<0x10000000002/96765652-39ef-4cf3-b20a-a70f7e9d774d v2320>,<0x10000000001/sv_0 v2326>,<0x10000000000/qa v2300>,<0x1/volumes v2260>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-10T11:37:46.390348+0000 4=4+0)","ondisk_value.rstat":"n(v0 rc2024-05-10T11:37:47.223222+0000 b780288 4=3+1)","memory_value.dirstat":"f(v1036 m2024-05-10T11:37:47.223222+0000 3=3+0)","memory_value.rstat":"n(v30 rc2024-05-10T11:37:47.223222+0000 b780288 4=3+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-05-10T11:38:04.029 INFO:journalctl@ceph.mds.i.smithi188.stdout:May 10 11:38:03 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-i[64554]: 2024-05-10T11:38:03.809+0000 7fd152433700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000557 [...2,head] /volumes/qa/sv_0/96765652-39ef-4cf3-b20a-a70f7e9d774d/client.0/tmp/clients/client5/~dmtmp/PM/ auth{2=1} v17783 ap=2 f(v1036 m2024-05-10T11:37:47.671773+0000 3=3+0) n(v30 rc2024-05-10T11:37:47.671773+0000 b780288 4=3+1) (inest sync->mix g=2) caps={15207=pAsLsXsFs/-@11075} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x5608018a8000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000557:[<0x10000000416/PM v66>,<0x1000000040c/~dmtmp v34>,<0x10000000007/client5 v1936>,<0x10000000006/clients v1743>,<0x10000000005/tmp v1642>,<0x10000000003/client.0 v1616>,<0x10000000002/96765652-39ef-4cf3-b20a-a70f7e9d774d v1609>,<0x10000000001/sv_0 v1620>,<0x10000000000/qa v1598>,<0x1/volumes v1566>]//[]","memoryvalue":"(2)0x10000000557:[<0x10000000416/PM v17783>,<0x1000000040c/~dmtmp v2843>,<0x10000000007/client5 v5589>,<0x10000000006/clients v2999>,<0x10000000005/tmp v2365>,<0x10000000003/client.0 v2330>,<0x10000000002/96765652-39ef-4cf3-b20a-a70f7e9d774d v2320>,<0x10000000001/sv_0 v2326>,<0x10000000000/qa v2300>,<0x1/volumes v2260>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-10T11:37:46.740263+0000 4=4+0)","ondisk_value.rstat":"n(v0 rc2024-05-10T11:37:47.671773+0000 b780288 4=3+1)","memory_value.dirstat":"f(v1036 m2024-05-10T11:37:47.671773+0000 3=3+0)","memory_value.rstat":"n(v30 rc2024-05-10T11:37:47.671773+0000 b780288 4=3+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-05-10T11:38:04.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.552+0000 7f67b2a23700 1 -- 172.21.15.100:0/2626075674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67ac1023a0 msgr2=0x7f67ac102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:04.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.552+0000 7f67b2a23700 1 --2- 172.21.15.100:0/2626075674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67ac1023a0 0x7f67ac102820 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f67a0009990 tx=0x7f67a0009ca0 comp rx=0 tx=0).stop 2024-05-10T11:38:04.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.552+0000 7f67b2a23700 1 -- 172.21.15.100:0/2626075674 shutdown_connections 2024-05-10T11:38:04.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.552+0000 7f67b2a23700 1 --2- 172.21.15.100:0/2626075674 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67ac102d60 0x7f67ac10f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:04.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.552+0000 7f67b2a23700 1 --2- 172.21.15.100:0/2626075674 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67ac108410 0x7f67ac1087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:04.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.552+0000 7f67b2a23700 1 --2- 172.21.15.100:0/2626075674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67ac1023a0 0x7f67ac102820 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:04.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.552+0000 7f67b2a23700 1 -- 172.21.15.100:0/2626075674 >> 172.21.15.100:0/2626075674 conn(0x7f67ac079350 msgr2=0x7f67ac079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:04.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.552+0000 7f67b2a23700 1 -- 172.21.15.100:0/2626075674 shutdown_connections 2024-05-10T11:38:04.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.552+0000 7f67b2a23700 1 -- 172.21.15.100:0/2626075674 wait complete. 2024-05-10T11:38:04.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.553+0000 7f67b2a23700 1 Processor -- start 2024-05-10T11:38:04.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.553+0000 7f67b2a23700 1 -- start start 2024-05-10T11:38:04.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.553+0000 7f67b2a23700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67ac1023a0 0x7f67ac104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:04.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.553+0000 7f67b2a23700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67ac102d60 0x7f67ac1045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:04.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.553+0000 7f67b2a23700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67ac108410 0x7f67ac104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:04.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.553+0000 7f67b2a23700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f67ac1124b0 con 0x7f67ac108410 2024-05-10T11:38:04.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.553+0000 7f67b2a23700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f67ac112330 con 0x7f67ac102d60 2024-05-10T11:38:04.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.553+0000 7f67b2a23700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f67ac112630 con 0x7f67ac1023a0 2024-05-10T11:38:04.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.553+0000 7f67b0fc0700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67ac108410 0x7f67ac104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:04.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.553+0000 7f67abfff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67ac1023a0 0x7f67ac104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:04.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.553+0000 7f67ab7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67ac102d60 0x7f67ac1045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:04.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.553+0000 7f67b0fc0700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67ac108410 0x7f67ac104b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52698/0 (socket says 172.21.15.100:52698) 2024-05-10T11:38:04.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.553+0000 7f67b0fc0700 1 -- 172.21.15.100:0/537082232 learned_addr learned my addr 172.21.15.100:0/537082232 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:38:04.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.554+0000 7f67b0fc0700 1 -- 172.21.15.100:0/537082232 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67ac1023a0 msgr2=0x7f67ac104080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:04.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.554+0000 7f67b0fc0700 1 --2- 172.21.15.100:0/537082232 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67ac1023a0 0x7f67ac104080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:04.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.554+0000 7f67b0fc0700 1 -- 172.21.15.100:0/537082232 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67ac102d60 msgr2=0x7f67ac1045c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:04.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.554+0000 7f67b0fc0700 1 --2- 172.21.15.100:0/537082232 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67ac102d60 0x7f67ac1045c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:04.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.554+0000 7f67b0fc0700 1 -- 172.21.15.100:0/537082232 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67ac1a7ff0 con 0x7f67ac108410 2024-05-10T11:38:04.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.554+0000 7f67b0fc0700 1 --2- 172.21.15.100:0/537082232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67ac108410 0x7f67ac104b00 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f67a400b970 tx=0x7f67a400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:04.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.554+0000 7f67a97fa700 1 -- 172.21.15.100:0/537082232 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a400d860 con 0x7f67ac108410 2024-05-10T11:38:04.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.554+0000 7f67a97fa700 1 -- 172.21.15.100:0/537082232 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f67a400e430 con 0x7f67ac108410 2024-05-10T11:38:04.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.554+0000 7f67b2a23700 1 -- 172.21.15.100:0/537082232 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67ac1a82e0 con 0x7f67ac108410 2024-05-10T11:38:04.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.554+0000 7f67a97fa700 1 -- 172.21.15.100:0/537082232 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a4017660 con 0x7f67ac108410 2024-05-10T11:38:04.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.554+0000 7f67b2a23700 1 -- 172.21.15.100:0/537082232 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67ac1a8870 con 0x7f67ac108410 2024-05-10T11:38:04.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.557+0000 7f67a97fa700 1 -- 172.21.15.100:0/537082232 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f67a400da00 con 0x7f67ac108410 2024-05-10T11:38:04.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.557+0000 7f67a97fa700 1 --2- 172.21.15.100:0/537082232 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f67940b2900 0x7f67940b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:04.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.558+0000 7f67abfff700 1 --2- 172.21.15.100:0/537082232 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f67940b2900 0x7f67940b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:04.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.558+0000 7f67a97fa700 1 -- 172.21.15.100:0/537082232 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f67a40e2090 con 0x7f67ac108410 2024-05-10T11:38:04.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.558+0000 7f67bd2b6b80 1 Processor -- start 2024-05-10T11:38:04.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.558+0000 7f67bd2b6b80 1 -- start start 2024-05-10T11:38:04.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.558+0000 7f67abfff700 1 --2- 172.21.15.100:0/537082232 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f67940b2900 0x7f67940b4dc0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f679c00ac50 tx=0x7f679c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:04.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.558+0000 7f67bd2b6b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ac71b0550 0x558ac71b0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:04.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.558+0000 7f67bd2b6b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ac71b0e70 0x558ac71b4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:04.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.558+0000 7f67bd2b6b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558ac71b4d00 0x558ac71b5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:04.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.558+0000 7f67bd2b6b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x558ac71028e0 con 0x558ac71b4d00 2024-05-10T11:38:04.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.558+0000 7f67bd2b6b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558ac70a1840 con 0x558ac71b0550 2024-05-10T11:38:04.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.558+0000 7f67bd2b6b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558ac70fe8a0 con 0x558ac71b0e70 2024-05-10T11:38:04.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.558+0000 7f67b0fc0700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558ac71b4d00 0x558ac71b5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:04.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.558+0000 7f67b0fc0700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558ac71b4d00 0x558ac71b5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52700/0 (socket says 172.21.15.100:52700) 2024-05-10T11:38:04.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.558+0000 7f67b0fc0700 1 -- 172.21.15.100:0/976755251 learned_addr learned my addr 172.21.15.100:0/976755251 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:38:04.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.558+0000 7f67abfff700 1 --2- 172.21.15.100:0/976755251 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ac71b0550 0x558ac71b0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:04.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.558+0000 7f67ab7fe700 1 --2- 172.21.15.100:0/976755251 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ac71b0e70 0x558ac71b4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:04.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.558+0000 7f67b0fc0700 1 -- 172.21.15.100:0/976755251 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ac71b0e70 msgr2=0x558ac71b4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:04.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.558+0000 7f67b0fc0700 1 --2- 172.21.15.100:0/976755251 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ac71b0e70 0x558ac71b4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:04.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.558+0000 7f67b0fc0700 1 -- 172.21.15.100:0/976755251 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ac71b0550 msgr2=0x558ac71b0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:04.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.558+0000 7f67b0fc0700 1 --2- 172.21.15.100:0/976755251 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ac71b0550 0x558ac71b0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:04.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.558+0000 7f67b0fc0700 1 -- 172.21.15.100:0/976755251 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ac704f950 con 0x558ac71b4d00 2024-05-10T11:38:04.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.558+0000 7f67ab7fe700 1 --2- 172.21.15.100:0/976755251 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ac71b0e70 0x558ac71b4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:38:04.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.559+0000 7f67b0fc0700 1 --2- 172.21.15.100:0/976755251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558ac71b4d00 0x558ac71b5180 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f67a40a65c0 tx=0x7f67a40a9a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9888cc279805a1ec server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:04.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.559+0000 7f67927fc700 1 -- 172.21.15.100:0/976755251 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a40b21b0 con 0x558ac71b4d00 2024-05-10T11:38:04.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.559+0000 7f67927fc700 1 -- 172.21.15.100:0/976755251 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f67a400e8e0 con 0x558ac71b4d00 2024-05-10T11:38:04.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.559+0000 7f67927fc700 1 -- 172.21.15.100:0/976755251 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a401e2f0 con 0x558ac71b4d00 2024-05-10T11:38:04.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.560+0000 7f67bd2b6b80 1 -- 172.21.15.100:0/976755251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558ac71b4d00 msgr2=0x558ac71b5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:04.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.560+0000 7f67bd2b6b80 1 --2- 172.21.15.100:0/976755251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558ac71b4d00 0x558ac71b5180 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f67a40a65c0 tx=0x7f67a40a9a50 comp rx=0 tx=0).stop 2024-05-10T11:38:04.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.560+0000 7f67bd2b6b80 1 -- 172.21.15.100:0/976755251 shutdown_connections 2024-05-10T11:38:04.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.560+0000 7f67bd2b6b80 1 --2- 172.21.15.100:0/976755251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558ac71b4d00 0x558ac71b5180 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:04.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.560+0000 7f67bd2b6b80 1 --2- 172.21.15.100:0/976755251 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ac71b0550 0x558ac71b0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:04.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.560+0000 7f67bd2b6b80 1 --2- 172.21.15.100:0/976755251 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ac71b0e70 0x558ac71b4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:04.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.560+0000 7f67bd2b6b80 1 -- 172.21.15.100:0/976755251 >> 172.21.15.100:0/976755251 conn(0x558ac70dc180 msgr2=0x558ac71aacd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:04.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.560+0000 7f67bd2b6b80 1 -- 172.21.15.100:0/976755251 shutdown_connections 2024-05-10T11:38:04.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.560+0000 7f67bd2b6b80 1 -- 172.21.15.100:0/976755251 wait complete. 2024-05-10T11:38:04.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.561+0000 7f67bd2b6b80 1 -- start start 2024-05-10T11:38:04.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.561+0000 7f67bd2b6b80 1 Processor -- start 2024-05-10T11:38:04.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.561+0000 7f67bd2b6b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558ac71b0550 0x558ac71bc190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:04.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.561+0000 7f67bd2b6b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ac71b0e70 0x558ac71b9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:04.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.561+0000 7f67bd2b6b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ac71b4d00 0x558ac71b9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:04.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.561+0000 7f67bd2b6b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x558ac71bcf10 con 0x558ac71b0550 2024-05-10T11:38:04.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.561+0000 7f67bd2b6b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558ac7068cb0 con 0x558ac71b0e70 2024-05-10T11:38:04.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.561+0000 7f67bd2b6b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558ac71bd090 con 0x558ac71b4d00 2024-05-10T11:38:04.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.561+0000 7f67abfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558ac71b0550 0x558ac71bc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:04.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.561+0000 7f67bd2b6b80 10 client.0 _ll_register_callbacks cb 0x558ac709a9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:38:04.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.561+0000 7f67abfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558ac71b0550 0x558ac71bc190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52716/0 (socket says 172.21.15.100:52716) 2024-05-10T11:38:04.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.561+0000 7f67abfff700 1 -- 172.21.15.100:0/1869877554 learned_addr learned my addr 172.21.15.100:0/1869877554 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:38:04.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.561+0000 7f67b0fc0700 1 --2- 172.21.15.100:0/1869877554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ac71b4d00 0x558ac71b9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:04.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.561+0000 7f67ab7fe700 1 --2- 172.21.15.100:0/1869877554 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ac71b0e70 0x558ac71b9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:04.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.562+0000 7f67abfff700 1 -- 172.21.15.100:0/1869877554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ac71b4d00 msgr2=0x558ac71b9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:04.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.562+0000 7f67abfff700 1 --2- 172.21.15.100:0/1869877554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ac71b4d00 0x558ac71b9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:04.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.562+0000 7f67abfff700 1 -- 172.21.15.100:0/1869877554 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ac71b0e70 msgr2=0x558ac71b9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:04.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.562+0000 7f67abfff700 1 --2- 172.21.15.100:0/1869877554 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ac71b0e70 0x558ac71b9530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:04.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.562+0000 7f67abfff700 1 -- 172.21.15.100:0/1869877554 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ac709e710 con 0x558ac71b0550 2024-05-10T11:38:04.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.562+0000 7f67abfff700 1 --2- 172.21.15.100:0/1869877554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558ac71b0550 0x558ac71bc190 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f679c003930 tx=0x7f679c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=770f289eac2cdd55 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:04.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.562+0000 7f678b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:38:04.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.562+0000 7f678b7fe700 1 -- 172.21.15.100:0/1869877554 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f679c016070 con 0x558ac71b0550 2024-05-10T11:38:04.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.562+0000 7f678b7fe700 1 -- 172.21.15.100:0/1869877554 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f679c010940 con 0x558ac71b0550 2024-05-10T11:38:04.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.562+0000 7f67bd2b6b80 1 -- 172.21.15.100:0/1869877554 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67a400e8e0 con 0x558ac71b0550 2024-05-10T11:38:04.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.562+0000 7f678b7fe700 1 -- 172.21.15.100:0/1869877554 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f679c030050 con 0x558ac71b0550 2024-05-10T11:38:04.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.562+0000 7f67bd2b6b80 10 client.19290 fetch_fsmap learned FSMap version 41 2024-05-10T11:38:04.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.562+0000 7f67bd2b6b80 1 -- 172.21.15.100:0/1869877554 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x558ac71c7790 con 0x558ac71b0550 2024-05-10T11:38:04.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.563+0000 7f678b7fe700 1 -- 172.21.15.100:0/1869877554 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f67a400e8e0 con 0x558ac71b0550 2024-05-10T11:38:04.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.563+0000 7f678b7fe700 1 -- 172.21.15.100:0/1869877554 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f679c01ec30 con 0x558ac71b0550 2024-05-10T11:38:04.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.563+0000 7f67bd2b6b80 10 client.19290 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:38:04.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.563+0000 7f67bd2b6b80 10 client.19290 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:38:04.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.563+0000 7f67bd2b6b80 20 client.19290 populate_metadata read hostname 'smithi100' 2024-05-10T11:38:04.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.563+0000 7f67bd2b6b80 1 --2- 172.21.15.100:0/1869877554 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558ac71ca240 0x558ac71cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:04.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.563+0000 7f67bd2b6b80 4 client.19290 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:38:04.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.563+0000 7f67bd2b6b80 1 -- 172.21.15.100:0/1869877554 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ac717c6d0 con 0x558ac71ca240 2024-05-10T11:38:04.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.563+0000 7f67ab7fe700 1 --2- 172.21.15.100:0/1869877554 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558ac71ca240 0x558ac71cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:04.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.564+0000 7f67ab7fe700 1 --2- 172.21.15.100:0/1869877554 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558ac71ca240 0x558ac71cc6e0 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ec1adf5600240d9 server_cookie=5b72a57399cf93ef in_seq=0 out_seq=0 2024-05-10T11:38:04.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.564+0000 7f678b7fe700 10 client.19290 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:38:04.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.564+0000 7f678b7fe700 1 -- 172.21.15.100:0/1869877554 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558ac717c6d0 con 0x558ac71ca240 2024-05-10T11:38:04.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.564+0000 7f678b7fe700 10 client.19290 handle_command_reply: tid=0 2024-05-10T11:38:04.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.564+0000 7f67bd2b6b80 1 client.19290 shutdown 2024-05-10T11:38:04.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.565+0000 7f67bd2b6b80 1 -- 172.21.15.100:0/1869877554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558ac71b0550 msgr2=0x558ac71bc190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:04.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.565+0000 7f67bd2b6b80 1 --2- 172.21.15.100:0/1869877554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558ac71b0550 0x558ac71bc190 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f679c003930 tx=0x7f679c0047c0 comp rx=0 tx=0).stop 2024-05-10T11:38:04.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.565+0000 7f67bd2b6b80 1 -- 172.21.15.100:0/1869877554 shutdown_connections 2024-05-10T11:38:04.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.565+0000 7f67bd2b6b80 1 --2- 172.21.15.100:0/1869877554 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558ac71ca240 0x558ac71cc6e0 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:04.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.565+0000 7f67bd2b6b80 1 --2- 172.21.15.100:0/1869877554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ac71b4d00 0x558ac71b9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:04.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.565+0000 7f67bd2b6b80 1 --2- 172.21.15.100:0/1869877554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558ac71b0550 0x558ac71bc190 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:04.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.565+0000 7f67bd2b6b80 1 --2- 172.21.15.100:0/1869877554 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ac71b0e70 0x558ac71b9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:04.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.565+0000 7f67bd2b6b80 1 -- 172.21.15.100:0/1869877554 >> 172.21.15.100:0/1869877554 conn(0x558ac70dc180 msgr2=0x558ac71aac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:04.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.565+0000 7f678b7fe700 0 client.19290 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:38:04.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.565+0000 7f67bd2b6b80 1 -- 172.21.15.100:0/1869877554 shutdown_connections 2024-05-10T11:38:04.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.565+0000 7f67bd2b6b80 1 -- 172.21.15.100:0/1869877554 wait complete. 2024-05-10T11:38:04.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.565+0000 7f67bd2b6b80 20 client.19290 trim_cache size 0 max 16384 2024-05-10T11:38:04.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.600+0000 7f67bd2b6b80 1 Processor -- start 2024-05-10T11:38:04.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.600+0000 7f67bd2b6b80 1 -- start start 2024-05-10T11:38:04.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.600+0000 7f67bd2b6b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ac70dc180 0x558ac71c5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:04.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.601+0000 7f67bd2b6b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ac71b0550 0x558ac71d1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:04.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.601+0000 7f67bd2b6b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558ac71b0e70 0x558ac71d2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:04.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.601+0000 7f67abfff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ac70dc180 0x558ac71c5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:04.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.601+0000 7f67ab7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ac71b0550 0x558ac71d1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:04.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.601+0000 7f67abfff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ac70dc180 0x558ac71c5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:33180/0 (socket says 172.21.15.100:33180) 2024-05-10T11:38:04.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.601+0000 7f67abfff700 1 -- 172.21.15.100:0/2885728612 learned_addr learned my addr 172.21.15.100:0/2885728612 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:38:04.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.601+0000 7f67bd2b6b80 1 -- 172.21.15.100:0/2885728612 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f67ac1b3c40 con 0x558ac71b0e70 2024-05-10T11:38:04.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.601+0000 7f67bd2b6b80 1 -- 172.21.15.100:0/2885728612 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f67ac1b14d0 con 0x558ac71b0550 2024-05-10T11:38:04.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.601+0000 7f67b0fc0700 1 --2- 172.21.15.100:0/2885728612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558ac71b0e70 0x558ac71d2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:04.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.601+0000 7f67bd2b6b80 1 -- 172.21.15.100:0/2885728612 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f67ac1af320 con 0x558ac70dc180 2024-05-10T11:38:04.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.601+0000 7f67ab7fe700 1 -- 172.21.15.100:0/2885728612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ac70dc180 msgr2=0x558ac71c5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:04.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.601+0000 7f67ab7fe700 1 --2- 172.21.15.100:0/2885728612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ac70dc180 0x558ac71c5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:04.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.601+0000 7f67ab7fe700 1 -- 172.21.15.100:0/2885728612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558ac71b0e70 msgr2=0x558ac71d2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:04.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.601+0000 7f67ab7fe700 1 --2- 172.21.15.100:0/2885728612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558ac71b0e70 0x558ac71d2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:04.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.601+0000 7f67ab7fe700 1 -- 172.21.15.100:0/2885728612 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ac71a3350 con 0x558ac71b0550 2024-05-10T11:38:04.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.601+0000 7f67b0fc0700 1 --2- 172.21.15.100:0/2885728612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558ac71b0e70 0x558ac71d2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:38:04.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.601+0000 7f67ab7fe700 1 --2- 172.21.15.100:0/2885728612 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ac71b0550 0x558ac71d1ff0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f67a000b5c0 tx=0x7f67a0005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffa42bf2e7f5e59a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:04.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.602+0000 7f6791ffb700 1 -- 172.21.15.100:0/2885728612 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a0005510 con 0x558ac71b0550 2024-05-10T11:38:04.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.602+0000 7f6791ffb700 1 -- 172.21.15.100:0/2885728612 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f67a002d070 con 0x558ac71b0550 2024-05-10T11:38:04.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.602+0000 7f6791ffb700 1 -- 172.21.15.100:0/2885728612 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a000ba60 con 0x558ac71b0550 2024-05-10T11:38:04.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.602+0000 7f67bd2b6b80 1 -- 172.21.15.100:0/2885728612 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ac71b0550 msgr2=0x558ac71d1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:04.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.602+0000 7f67bd2b6b80 1 --2- 172.21.15.100:0/2885728612 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ac71b0550 0x558ac71d1ff0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f67a000b5c0 tx=0x7f67a0005fd0 comp rx=0 tx=0).stop 2024-05-10T11:38:04.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.603+0000 7f67bd2b6b80 1 -- 172.21.15.100:0/2885728612 shutdown_connections 2024-05-10T11:38:04.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.603+0000 7f67bd2b6b80 1 --2- 172.21.15.100:0/2885728612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558ac71b0e70 0x558ac71d2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:04.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.603+0000 7f67bd2b6b80 1 --2- 172.21.15.100:0/2885728612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ac70dc180 0x558ac71c5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:04.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.603+0000 7f67bd2b6b80 1 --2- 172.21.15.100:0/2885728612 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ac71b0550 0x558ac71d1ff0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:04.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.603+0000 7f67bd2b6b80 1 -- 172.21.15.100:0/2885728612 >> 172.21.15.100:0/2885728612 conn(0x558ac71a5b30 msgr2=0x558ac71afbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:04.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.603+0000 7f67bd2b6b80 1 -- 172.21.15.100:0/2885728612 shutdown_connections 2024-05-10T11:38:04.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.603+0000 7f67bd2b6b80 1 -- 172.21.15.100:0/2885728612 wait complete. 2024-05-10T11:38:04.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.603+0000 7f67bd2b6b80 1 -- start start 2024-05-10T11:38:04.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.603+0000 7f67bd2b6b80 1 Processor -- start 2024-05-10T11:38:04.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.603+0000 7f67bd2b6b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558ac70dc180 0x558ac71c0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:04.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.604+0000 7f67bd2b6b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ac71b0550 0x558ac71c13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:04.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.604+0000 7f67bd2b6b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ac71b0e70 0x558ac71c18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:04.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.604+0000 7f67bd2b6b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f67ac1af990 con 0x558ac70dc180 2024-05-10T11:38:04.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.604+0000 7f67bd2b6b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f67ac1b0e00 con 0x558ac71b0e70 2024-05-10T11:38:04.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.604+0000 7f67bd2b6b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f67ac1b0060 con 0x558ac71b0550 2024-05-10T11:38:04.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.604+0000 7f67bd2b6b80 10 client.0 _ll_register_callbacks cb 0x558ac709a9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:38:04.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.604+0000 7f67abfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558ac70dc180 0x558ac71c0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:04.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.604+0000 7f67ab7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ac71b0550 0x558ac71c13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:04.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.604+0000 7f67abfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558ac70dc180 0x558ac71c0e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52732/0 (socket says 172.21.15.100:52732) 2024-05-10T11:38:04.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.604+0000 7f67abfff700 1 -- 172.21.15.100:0/2895771699 learned_addr learned my addr 172.21.15.100:0/2895771699 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:38:04.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.604+0000 7f67b0fc0700 1 --2- 172.21.15.100:0/2895771699 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ac71b0e70 0x558ac71c18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:04.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.604+0000 7f67ab7fe700 1 -- 172.21.15.100:0/2895771699 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ac71b0e70 msgr2=0x558ac71c18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:04.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.604+0000 7f67ab7fe700 1 --2- 172.21.15.100:0/2895771699 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ac71b0e70 0x558ac71c18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:04.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.604+0000 7f67ab7fe700 1 -- 172.21.15.100:0/2895771699 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558ac70dc180 msgr2=0x558ac71c0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:04.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.604+0000 7f67ab7fe700 1 --2- 172.21.15.100:0/2895771699 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558ac70dc180 0x558ac71c0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:04.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.604+0000 7f67ab7fe700 1 -- 172.21.15.100:0/2895771699 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ac71b0b80 con 0x558ac71b0550 2024-05-10T11:38:04.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.604+0000 7f67b0fc0700 1 --2- 172.21.15.100:0/2895771699 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ac71b0e70 0x558ac71c18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:38:04.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.604+0000 7f67abfff700 1 --2- 172.21.15.100:0/2895771699 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558ac70dc180 0x558ac71c0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:38:04.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.604+0000 7f67ab7fe700 1 --2- 172.21.15.100:0/2895771699 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ac71b0550 0x558ac71c13a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x558ac70fe530 tx=0x7f67a0005540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3809162ad923349 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:04.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.604+0000 7f678b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:38:04.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.605+0000 7f678b7fe700 1 -- 172.21.15.100:0/2895771699 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a0003bd0 con 0x558ac71b0550 2024-05-10T11:38:04.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.605+0000 7f678b7fe700 1 -- 172.21.15.100:0/2895771699 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f67a0003d70 con 0x558ac71b0550 2024-05-10T11:38:04.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.605+0000 7f67bd2b6b80 1 -- 172.21.15.100:0/2895771699 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67a002d070 con 0x558ac71b0550 2024-05-10T11:38:04.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.605+0000 7f678b7fe700 1 -- 172.21.15.100:0/2895771699 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f67a0041050 con 0x558ac71b0550 2024-05-10T11:38:04.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.605+0000 7f67bd2b6b80 10 client.28684 fetch_fsmap learned FSMap version 41 2024-05-10T11:38:04.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.605+0000 7f67bd2b6b80 1 -- 172.21.15.100:0/2895771699 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x558ac7235c90 con 0x558ac71b0550 2024-05-10T11:38:04.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.605+0000 7f678b7fe700 1 -- 172.21.15.100:0/2895771699 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f67a002d070 con 0x558ac71b0550 2024-05-10T11:38:04.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.605+0000 7f678b7fe700 1 -- 172.21.15.100:0/2895771699 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f67a000f580 con 0x558ac71b0550 2024-05-10T11:38:04.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.605+0000 7f67bd2b6b80 10 client.28684 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:38:04.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.605+0000 7f67bd2b6b80 10 client.28684 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:38:04.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.605+0000 7f67bd2b6b80 20 client.28684 populate_metadata read hostname 'smithi100' 2024-05-10T11:38:04.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.606+0000 7f67bd2b6b80 1 --2- 172.21.15.100:0/2895771699 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558ac71b4d00 0x558ac7238380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:04.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.606+0000 7f67bd2b6b80 4 client.28684 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T11:38:04.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.606+0000 7f67bd2b6b80 1 -- 172.21.15.100:0/2895771699 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558ac71102c0 con 0x558ac71b4d00 2024-05-10T11:38:04.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.606+0000 7f67abfff700 1 --2- 172.21.15.100:0/2895771699 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558ac71b4d00 0x558ac7238380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:04.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.606+0000 7f678b7fe700 10 client.28684 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:38:04.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.606+0000 7f67abfff700 1 --2- 172.21.15.100:0/2895771699 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558ac71b4d00 0x558ac7238380 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1c03336bbc6b66b server_cookie=57ccbe81a6acf83d in_seq=0 out_seq=0 2024-05-10T11:38:04.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.607+0000 7f678b7fe700 1 -- 172.21.15.100:0/2895771699 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558ac71102c0 con 0x558ac71b4d00 2024-05-10T11:38:04.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.607+0000 7f678b7fe700 10 client.28684 handle_command_reply: tid=0 2024-05-10T11:38:04.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.607+0000 7f67bd2b6b80 1 client.28684 shutdown 2024-05-10T11:38:04.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.608+0000 7f67bd2b6b80 1 -- 172.21.15.100:0/2895771699 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ac71b0550 msgr2=0x558ac71c13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:04.449 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:38:04.449 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-05-10T11:38:04.449 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "41dc61d2-3dd1-440f-a5ec-1c06b918cd54", 2024-05-10T11:38:04.449 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-05-10T11:38:04.449 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:38:04.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.608+0000 7f67bd2b6b80 1 --2- 172.21.15.100:0/2895771699 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ac71b0550 0x558ac71c13a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x558ac70fe530 tx=0x7f67a0005540 comp rx=0 tx=0).stop 2024-05-10T11:38:04.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.608+0000 7f67bd2b6b80 1 -- 172.21.15.100:0/2895771699 shutdown_connections 2024-05-10T11:38:04.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.608+0000 7f67bd2b6b80 1 --2- 172.21.15.100:0/2895771699 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558ac71b4d00 0x558ac7238380 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:04.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.608+0000 7f67bd2b6b80 1 --2- 172.21.15.100:0/2895771699 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ac71b0e70 0x558ac71c18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:04.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.608+0000 7f67bd2b6b80 1 --2- 172.21.15.100:0/2895771699 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558ac70dc180 0x558ac71c0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:04.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.608+0000 7f67bd2b6b80 1 --2- 172.21.15.100:0/2895771699 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ac71b0550 0x558ac71c13a0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:04.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.608+0000 7f67bd2b6b80 1 -- 172.21.15.100:0/2895771699 >> 172.21.15.100:0/2895771699 conn(0x558ac71a5b30 msgr2=0x558ac71bab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:04.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.608+0000 7f678b7fe700 0 client.28684 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:38:04.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.608+0000 7f67bd2b6b80 1 -- 172.21.15.100:0/2895771699 shutdown_connections 2024-05-10T11:38:04.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.608+0000 7f67bd2b6b80 1 -- 172.21.15.100:0/2895771699 wait complete. 2024-05-10T11:38:04.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.608+0000 7f67bd2b6b80 20 client.28684 trim_cache size 0 max 16384 2024-05-10T11:38:04.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.609+0000 7f67b2a23700 1 -- 172.21.15.100:0/537082232 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f67940b2900 msgr2=0x7f67940b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:04.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.609+0000 7f67b2a23700 1 --2- 172.21.15.100:0/537082232 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f67940b2900 0x7f67940b4dc0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f679c00ac50 tx=0x7f679c00a380 comp rx=0 tx=0).stop 2024-05-10T11:38:04.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.609+0000 7f67b2a23700 1 -- 172.21.15.100:0/537082232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67ac108410 msgr2=0x7f67ac104b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:04.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.609+0000 7f67b2a23700 1 --2- 172.21.15.100:0/537082232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67ac108410 0x7f67ac104b00 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f67a400b970 tx=0x7f67a400bc80 comp rx=0 tx=0).stop 2024-05-10T11:38:04.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.609+0000 7f67b2a23700 1 -- 172.21.15.100:0/537082232 shutdown_connections 2024-05-10T11:38:04.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.609+0000 7f67b2a23700 1 --2- 172.21.15.100:0/537082232 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f67940b2900 0x7f67940b4dc0 secure :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f679c00ac50 tx=0x7f679c00a380 comp rx=0 tx=0).stop 2024-05-10T11:38:04.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.609+0000 7f67b2a23700 1 --2- 172.21.15.100:0/537082232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67ac108410 0x7f67ac104b00 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:04.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.609+0000 7f67b2a23700 1 --2- 172.21.15.100:0/537082232 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67ac1023a0 0x7f67ac104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:04.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.609+0000 7f67b2a23700 1 --2- 172.21.15.100:0/537082232 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67ac102d60 0x7f67ac1045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:04.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.609+0000 7f67b2a23700 1 -- 172.21.15.100:0/537082232 >> 172.21.15.100:0/537082232 conn(0x7f67ac079350 msgr2=0x7f67ac1016d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:04.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.609+0000 7f67b2a23700 1 -- 172.21.15.100:0/537082232 shutdown_connections 2024-05-10T11:38:04.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:03.609+0000 7f67b2a23700 1 -- 172.21.15.100:0/537082232 wait complete. 2024-05-10T11:38:04.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:04 smithi100 ceph-mon[30296]: scrub queued for path: / 2024-05-10T11:38:04.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:04 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:38:04.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:04 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:38:04.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:04 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:38:04.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:04 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:38:04.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:04 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:38:04.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:04 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:38:04.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:04 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:38:04.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:04 smithi100 ceph-mon[30296]: Scrub error on inode 0x10000000487 (/volumes/qa/sv_0/96765652-39ef-4cf3-b20a-a70f7e9d774d/client.0/tmp/clients/client7/~dmtmp/EXCEL) see mds.i log and `damage ls` output for details 2024-05-10T11:38:04.789 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:04 smithi100 ceph-mon[30296]: Scrub error on inode 0x10000000544 (/volumes/qa/sv_0/96765652-39ef-4cf3-b20a-a70f7e9d774d/client.0/tmp/clients/client7/~dmtmp/WORDPRO) see mds.i log and `damage ls` output for details 2024-05-10T11:38:04.789 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:04 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:38:04.789 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:04 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:38:04.789 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:04 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:38:04.789 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:04 smithi100 ceph-mon[30296]: Scrub error on inode 0x1000000064b (/volumes/qa/sv_0/96765652-39ef-4cf3-b20a-a70f7e9d774d/client.0/tmp/clients/client7/~dmtmp/ACCESS) see mds.i log and `damage ls` output for details 2024-05-10T11:38:04.789 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:04 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:38:04.789 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:04 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:38:04.789 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:04 smithi100 ceph-mon[30296]: Scrub error on inode 0x10000000566 (/volumes/qa/sv_0/96765652-39ef-4cf3-b20a-a70f7e9d774d/client.0/tmp/clients/client9/~dmtmp/PM) see mds.i log and `damage ls` output for details 2024-05-10T11:38:04.789 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:04 smithi100 ceph-mon[30296]: Scrub error on inode 0x10000000557 (/volumes/qa/sv_0/96765652-39ef-4cf3-b20a-a70f7e9d774d/client.0/tmp/clients/client5/~dmtmp/PM) see mds.i log and `damage ls` output for details 2024-05-10T11:38:04.937 INFO:tasks.workunit.client.0.smithi100.stdout: 10 60108 9.04 MB/sec execute 238 sec latency 165.439 ms 2024-05-10T11:38:05.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:04 smithi178 ceph-mon[33359]: scrub queued for path: / 2024-05-10T11:38:05.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:04 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:38:05.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:04 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:38:05.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:04 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:38:05.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:04 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:38:05.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:04 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:38:05.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:04 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:38:05.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:04 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:38:05.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:04 smithi178 ceph-mon[33359]: Scrub error on inode 0x10000000487 (/volumes/qa/sv_0/96765652-39ef-4cf3-b20a-a70f7e9d774d/client.0/tmp/clients/client7/~dmtmp/EXCEL) see mds.i log and `damage ls` output for details 2024-05-10T11:38:05.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:04 smithi178 ceph-mon[33359]: Scrub error on inode 0x10000000544 (/volumes/qa/sv_0/96765652-39ef-4cf3-b20a-a70f7e9d774d/client.0/tmp/clients/client7/~dmtmp/WORDPRO) see mds.i log and `damage ls` output for details 2024-05-10T11:38:05.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:04 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:38:05.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:04 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:38:05.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:04 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:38:05.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:04 smithi178 ceph-mon[33359]: Scrub error on inode 0x1000000064b (/volumes/qa/sv_0/96765652-39ef-4cf3-b20a-a70f7e9d774d/client.0/tmp/clients/client7/~dmtmp/ACCESS) see mds.i log and `damage ls` output for details 2024-05-10T11:38:05.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:04 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:38:05.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:04 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:38:05.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:04 smithi178 ceph-mon[33359]: Scrub error on inode 0x10000000566 (/volumes/qa/sv_0/96765652-39ef-4cf3-b20a-a70f7e9d774d/client.0/tmp/clients/client9/~dmtmp/PM) see mds.i log and `damage ls` output for details 2024-05-10T11:38:05.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:04 smithi178 ceph-mon[33359]: Scrub error on inode 0x10000000557 (/volumes/qa/sv_0/96765652-39ef-4cf3-b20a-a70f7e9d774d/client.0/tmp/clients/client5/~dmtmp/PM) see mds.i log and `damage ls` output for details 2024-05-10T11:38:05.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:04 smithi188 ceph-mon[34075]: scrub queued for path: / 2024-05-10T11:38:05.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:04 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:38:05.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:04 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:38:05.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:04 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:38:05.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:04 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:38:05.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:04 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:38:05.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:04 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:38:05.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:04 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:38:05.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:04 smithi188 ceph-mon[34075]: Scrub error on inode 0x10000000487 (/volumes/qa/sv_0/96765652-39ef-4cf3-b20a-a70f7e9d774d/client.0/tmp/clients/client7/~dmtmp/EXCEL) see mds.i log and `damage ls` output for details 2024-05-10T11:38:05.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:04 smithi188 ceph-mon[34075]: Scrub error on inode 0x10000000544 (/volumes/qa/sv_0/96765652-39ef-4cf3-b20a-a70f7e9d774d/client.0/tmp/clients/client7/~dmtmp/WORDPRO) see mds.i log and `damage ls` output for details 2024-05-10T11:38:05.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:04 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:38:05.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:04 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:38:05.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:04 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:38:05.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:04 smithi188 ceph-mon[34075]: Scrub error on inode 0x1000000064b (/volumes/qa/sv_0/96765652-39ef-4cf3-b20a-a70f7e9d774d/client.0/tmp/clients/client7/~dmtmp/ACCESS) see mds.i log and `damage ls` output for details 2024-05-10T11:38:05.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:04 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:38:05.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:04 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:38:05.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:04 smithi188 ceph-mon[34075]: Scrub error on inode 0x10000000566 (/volumes/qa/sv_0/96765652-39ef-4cf3-b20a-a70f7e9d774d/client.0/tmp/clients/client9/~dmtmp/PM) see mds.i log and `damage ls` output for details 2024-05-10T11:38:05.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:04 smithi188 ceph-mon[34075]: Scrub error on inode 0x10000000557 (/volumes/qa/sv_0/96765652-39ef-4cf3-b20a-a70f7e9d774d/client.0/tmp/clients/client5/~dmtmp/PM) see mds.i log and `damage ls` output for details 2024-05-10T11:38:05.937 INFO:tasks.workunit.client.0.smithi100.stdout: 10 60268 9.03 MB/sec execute 239 sec latency 96.538 ms 2024-05-10T11:38:06.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:05 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:38:06.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:05 smithi178 ceph-mon[33359]: pgmap v1012: 129 pgs: 129 active+clean; 499 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 4.8 MiB/s wr, 277 op/s 2024-05-10T11:38:06.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:05 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:38:06.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:05 smithi188 ceph-mon[34075]: pgmap v1012: 129 pgs: 129 active+clean; 499 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 4.8 MiB/s wr, 277 op/s 2024-05-10T11:38:06.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:05 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:38:06.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:05 smithi100 ceph-mon[30296]: pgmap v1012: 129 pgs: 129 active+clean; 499 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 4.8 MiB/s wr, 277 op/s 2024-05-10T11:38:06.937 INFO:tasks.workunit.client.0.smithi100.stdout: 10 60364 9.01 MB/sec execute 240 sec latency 277.282 ms 2024-05-10T11:38:07.716 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:07 smithi100 ceph-mon[30296]: pgmap v1013: 129 pgs: 129 active+clean; 491 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 808 KiB/s rd, 4.5 MiB/s wr, 298 op/s 2024-05-10T11:38:07.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:07 smithi178 ceph-mon[33359]: pgmap v1013: 129 pgs: 129 active+clean; 491 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 808 KiB/s rd, 4.5 MiB/s wr, 298 op/s 2024-05-10T11:38:07.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:07 smithi188 ceph-mon[34075]: pgmap v1013: 129 pgs: 129 active+clean; 491 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 808 KiB/s rd, 4.5 MiB/s wr, 298 op/s 2024-05-10T11:38:07.937 INFO:tasks.workunit.client.0.smithi100.stdout: 10 60491 8.98 MB/sec execute 241 sec latency 132.493 ms 2024-05-10T11:38:08.937 INFO:tasks.workunit.client.0.smithi100.stdout: 10 60598 8.95 MB/sec execute 242 sec latency 158.467 ms 2024-05-10T11:38:09.937 INFO:tasks.workunit.client.0.smithi100.stdout: 10 60801 8.98 MB/sec execute 243 sec latency 135.720 ms 2024-05-10T11:38:10.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:09 smithi178 ceph-mon[33359]: pgmap v1014: 129 pgs: 129 active+clean; 490 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 910 KiB/s rd, 3.9 MiB/s wr, 322 op/s 2024-05-10T11:38:10.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:09 smithi188 ceph-mon[34075]: pgmap v1014: 129 pgs: 129 active+clean; 490 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 910 KiB/s rd, 3.9 MiB/s wr, 322 op/s 2024-05-10T11:38:10.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:09 smithi100 ceph-mon[30296]: pgmap v1014: 129 pgs: 129 active+clean; 490 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 910 KiB/s rd, 3.9 MiB/s wr, 322 op/s 2024-05-10T11:38:10.937 INFO:tasks.workunit.client.0.smithi100.stdout: 10 60956 8.97 MB/sec execute 244 sec latency 132.813 ms 2024-05-10T11:38:11.596 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T11:38:11.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:11 smithi188 ceph-mon[34075]: pgmap v1015: 129 pgs: 129 active+clean; 491 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 3.9 MiB/s wr, 333 op/s 2024-05-10T11:38:11.840 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:11 smithi100 ceph-mon[30296]: pgmap v1015: 129 pgs: 129 active+clean; 491 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 3.9 MiB/s wr, 333 op/s 2024-05-10T11:38:11.937 INFO:tasks.workunit.client.0.smithi100.stdout: 10 61109 8.96 MB/sec execute 245 sec latency 168.872 ms 2024-05-10T11:38:11.974 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:38:12.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:11 smithi178 ceph-mon[33359]: pgmap v1015: 129 pgs: 129 active+clean; 491 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 3.9 MiB/s wr, 333 op/s 2024-05-10T11:38:12.937 INFO:tasks.workunit.client.0.smithi100.stdout: 10 61301 8.98 MB/sec execute 246 sec latency 154.674 ms 2024-05-10T11:38:13.937 INFO:tasks.workunit.client.0.smithi100.stdout: 10 61496 8.98 MB/sec execute 247 sec latency 88.984 ms 2024-05-10T11:38:13.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:13 smithi100 ceph-mon[30296]: pgmap v1016: 129 pgs: 129 active+clean; 492 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 805 KiB/s rd, 3.4 MiB/s wr, 323 op/s 2024-05-10T11:38:14.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:13 smithi178 ceph-mon[33359]: pgmap v1016: 129 pgs: 129 active+clean; 492 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 805 KiB/s rd, 3.4 MiB/s wr, 323 op/s 2024-05-10T11:38:14.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:13 smithi188 ceph-mon[34075]: pgmap v1016: 129 pgs: 129 active+clean; 492 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 805 KiB/s rd, 3.4 MiB/s wr, 323 op/s 2024-05-10T11:38:14.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.596+0000 7f5155f21700 1 -- 172.21.15.100:0/1426237120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5150102390 msgr2=0x7f5150102810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:14.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.596+0000 7f5155f21700 1 --2- 172.21.15.100:0/1426237120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5150102390 0x7f5150102810 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f5144009990 tx=0x7f5144009ca0 comp rx=0 tx=0).stop 2024-05-10T11:38:14.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.596+0000 7f5155f21700 1 -- 172.21.15.100:0/1426237120 shutdown_connections 2024-05-10T11:38:14.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.596+0000 7f5155f21700 1 --2- 172.21.15.100:0/1426237120 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5150102d50 0x7f515010f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:14.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.596+0000 7f5155f21700 1 --2- 172.21.15.100:0/1426237120 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5150108400 0x7f51501087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:14.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.596+0000 7f5155f21700 1 --2- 172.21.15.100:0/1426237120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5150102390 0x7f5150102810 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:14.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.596+0000 7f5155f21700 1 -- 172.21.15.100:0/1426237120 >> 172.21.15.100:0/1426237120 conn(0x7f5150079310 msgr2=0x7f5150079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:14.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.597+0000 7f5155f21700 1 -- 172.21.15.100:0/1426237120 shutdown_connections 2024-05-10T11:38:14.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.597+0000 7f5155f21700 1 -- 172.21.15.100:0/1426237120 wait complete. 2024-05-10T11:38:14.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.598+0000 7f5155f21700 1 Processor -- start 2024-05-10T11:38:14.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.598+0000 7f5155f21700 1 -- start start 2024-05-10T11:38:14.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.598+0000 7f5155f21700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5150102390 0x7f5150198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:14.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.598+0000 7f5155f21700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5150102d50 0x7f5150198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:14.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.598+0000 7f5155f21700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5150108400 0x7f515019d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:14.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.598+0000 7f5155f21700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5150112470 con 0x7f5150102d50 2024-05-10T11:38:14.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.598+0000 7f5155f21700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f51501122f0 con 0x7f5150108400 2024-05-10T11:38:14.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.598+0000 7f5155f21700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f51501125f0 con 0x7f5150102390 2024-05-10T11:38:14.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.598+0000 7f514f7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5150102390 0x7f5150198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:14.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.598+0000 7f514effd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5150102d50 0x7f5150198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:14.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.598+0000 7f514f7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5150102390 0x7f5150198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:51918/0 (socket says 172.21.15.100:51918) 2024-05-10T11:38:14.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.598+0000 7f514f7fe700 1 -- 172.21.15.100:0/2207748591 learned_addr learned my addr 172.21.15.100:0/2207748591 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:38:14.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.598+0000 7f514ffff700 1 --2- 172.21.15.100:0/2207748591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5150108400 0x7f515019d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:14.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.599+0000 7f514effd700 1 -- 172.21.15.100:0/2207748591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5150102390 msgr2=0x7f5150198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:14.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.599+0000 7f514effd700 1 --2- 172.21.15.100:0/2207748591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5150102390 0x7f5150198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:14.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.599+0000 7f514effd700 1 -- 172.21.15.100:0/2207748591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5150108400 msgr2=0x7f515019d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:14.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.599+0000 7f514effd700 1 --2- 172.21.15.100:0/2207748591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5150108400 0x7f515019d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:14.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.599+0000 7f514effd700 1 -- 172.21.15.100:0/2207748591 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f515019dcb0 con 0x7f5150102d50 2024-05-10T11:38:14.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.599+0000 7f514f7fe700 1 --2- 172.21.15.100:0/2207748591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5150102390 0x7f5150198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:38:14.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.599+0000 7f514effd700 1 --2- 172.21.15.100:0/2207748591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5150102d50 0x7f5150198c60 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f5144006010 tx=0x7f51440057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:14.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.599+0000 7f514cff9700 1 -- 172.21.15.100:0/2207748591 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5144026070 con 0x7f5150102d50 2024-05-10T11:38:14.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.599+0000 7f514cff9700 1 -- 172.21.15.100:0/2207748591 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f514400fb00 con 0x7f5150102d50 2024-05-10T11:38:14.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.599+0000 7f5155f21700 1 -- 172.21.15.100:0/2207748591 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f515019df40 con 0x7f5150102d50 2024-05-10T11:38:14.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.600+0000 7f5155f21700 1 -- 172.21.15.100:0/2207748591 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f515019e470 con 0x7f5150102d50 2024-05-10T11:38:14.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.600+0000 7f514cff9700 1 -- 172.21.15.100:0/2207748591 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5144021a50 con 0x7f5150102d50 2024-05-10T11:38:14.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.603+0000 7f514cff9700 1 -- 172.21.15.100:0/2207748591 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f5144029070 con 0x7f5150102d50 2024-05-10T11:38:14.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.604+0000 7f51607b4b80 1 Processor -- start 2024-05-10T11:38:14.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.604+0000 7f51607b4b80 1 -- start start 2024-05-10T11:38:14.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.605+0000 7f51607b4b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5632390964d0 0x5632390968b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:14.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.605+0000 7f51607b4b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563239096df0 0x56323909a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:14.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.605+0000 7f51607b4b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56323909ac80 0x56323909b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:14.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.605+0000 7f51607b4b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x563238fe8860 con 0x563239096df0 2024-05-10T11:38:14.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.605+0000 7f51607b4b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563238f877c0 con 0x5632390964d0 2024-05-10T11:38:14.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.605+0000 7f51607b4b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563238fe4820 con 0x56323909ac80 2024-05-10T11:38:14.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.605+0000 7f514ffff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5632390964d0 0x5632390968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:14.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.605+0000 7f514cff9700 1 --2- 172.21.15.100:0/2207748591 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f51380b29c0 0x7f51380b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:14.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.605+0000 7f514ffff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5632390964d0 0x5632390968b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:55254/0 (socket says 172.21.15.100:55254) 2024-05-10T11:38:14.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.605+0000 7f514ffff700 1 -- 172.21.15.100:0/2896994397 learned_addr learned my addr 172.21.15.100:0/2896994397 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:38:14.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.605+0000 7f514ffff700 1 --2- 172.21.15.100:0/2207748591 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f51380b29c0 0x7f51380b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:14.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.605+0000 7f514cff9700 1 -- 172.21.15.100:0/2207748591 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f51440e2d80 con 0x7f5150102d50 2024-05-10T11:38:14.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.605+0000 7f514effd700 1 --2- 172.21.15.100:0/2896994397 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56323909ac80 0x56323909b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:14.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.605+0000 7f514ffff700 1 -- 172.21.15.100:0/2896994397 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56323909ac80 msgr2=0x56323909b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:14.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.605+0000 7f514ffff700 1 --2- 172.21.15.100:0/2896994397 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56323909ac80 0x56323909b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:14.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.605+0000 7f514ffff700 1 -- 172.21.15.100:0/2896994397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563239096df0 msgr2=0x56323909a6f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T11:38:14.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.605+0000 7f514ffff700 1 --2- 172.21.15.100:0/2896994397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563239096df0 0x56323909a6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:14.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.605+0000 7f514ffff700 1 -- 172.21.15.100:0/2896994397 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563238f358d0 con 0x5632390964d0 2024-05-10T11:38:14.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.606+0000 7f514ffff700 1 --2- 172.21.15.100:0/2896994397 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5632390964d0 0x5632390968b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5148004070 tx=0x7f5148004380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31229deaead3a9e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:14.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.606+0000 7f5135ffb700 1 -- 172.21.15.100:0/2896994397 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514801e070 con 0x5632390964d0 2024-05-10T11:38:14.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.606+0000 7f5135ffb700 1 -- 172.21.15.100:0/2896994397 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5148007a50 con 0x5632390964d0 2024-05-10T11:38:14.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.606+0000 7f5135ffb700 1 -- 172.21.15.100:0/2896994397 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514801aae0 con 0x5632390964d0 2024-05-10T11:38:14.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.607+0000 7f514ffff700 1 --2- 172.21.15.100:0/2207748591 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f51380b29c0 0x7f51380b4e80 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f51480099d0 tx=0x7f514801ad60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:14.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.607+0000 7f51607b4b80 1 -- 172.21.15.100:0/2896994397 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5632390964d0 msgr2=0x5632390968b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:14.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.607+0000 7f51607b4b80 1 --2- 172.21.15.100:0/2896994397 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5632390964d0 0x5632390968b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5148004070 tx=0x7f5148004380 comp rx=0 tx=0).stop 2024-05-10T11:38:14.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.608+0000 7f51607b4b80 1 -- 172.21.15.100:0/2896994397 shutdown_connections 2024-05-10T11:38:14.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.608+0000 7f51607b4b80 1 --2- 172.21.15.100:0/2896994397 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56323909ac80 0x56323909b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:14.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.608+0000 7f51607b4b80 1 --2- 172.21.15.100:0/2896994397 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5632390964d0 0x5632390968b0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:14.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.608+0000 7f51607b4b80 1 --2- 172.21.15.100:0/2896994397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563239096df0 0x56323909a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:14.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.608+0000 7f51607b4b80 1 -- 172.21.15.100:0/2896994397 >> 172.21.15.100:0/2896994397 conn(0x563238fc2100 msgr2=0x563239090c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:14.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.608+0000 7f51607b4b80 1 -- 172.21.15.100:0/2896994397 shutdown_connections 2024-05-10T11:38:14.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.608+0000 7f51607b4b80 1 -- 172.21.15.100:0/2896994397 wait complete. 2024-05-10T11:38:14.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.609+0000 7f51607b4b80 1 -- start start 2024-05-10T11:38:14.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.609+0000 7f51607b4b80 1 Processor -- start 2024-05-10T11:38:14.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.609+0000 7f51607b4b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5632390964d0 0x5632390a2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:14.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.610+0000 7f51607b4b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563239096df0 0x56323909f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:14.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.610+0000 7f51607b4b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56323909ac80 0x56323909f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:14.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.610+0000 7f51607b4b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5632390a2e90 con 0x56323909ac80 2024-05-10T11:38:14.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.610+0000 7f51607b4b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563238f4ec30 con 0x5632390964d0 2024-05-10T11:38:14.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.610+0000 7f514f7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5632390964d0 0x5632390a2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:14.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.610+0000 7f514ffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56323909ac80 0x56323909f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:14.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.610+0000 7f514ffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56323909ac80 0x56323909f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35912/0 (socket says 172.21.15.100:35912) 2024-05-10T11:38:14.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.610+0000 7f514ffff700 1 -- 172.21.15.100:0/1021289622 learned_addr learned my addr 172.21.15.100:0/1021289622 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:38:14.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.610+0000 7f514effd700 1 --2- 172.21.15.100:0/1021289622 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563239096df0 0x56323909f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:14.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.610+0000 7f51607b4b80 1 -- 172.21.15.100:0/1021289622 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5632390a3010 con 0x563239096df0 2024-05-10T11:38:14.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.610+0000 7f51607b4b80 10 client.0 _ll_register_callbacks cb 0x563238f80920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:38:14.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.610+0000 7f514effd700 1 -- 172.21.15.100:0/1021289622 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5632390964d0 msgr2=0x5632390a2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:14.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.610+0000 7f514effd700 1 --2- 172.21.15.100:0/1021289622 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5632390964d0 0x5632390a2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:14.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.610+0000 7f514effd700 1 -- 172.21.15.100:0/1021289622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56323909ac80 msgr2=0x56323909f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:14.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.610+0000 7f514effd700 1 --2- 172.21.15.100:0/1021289622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56323909ac80 0x56323909f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:14.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.610+0000 7f514effd700 1 -- 172.21.15.100:0/1021289622 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563238f84690 con 0x563239096df0 2024-05-10T11:38:14.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.610+0000 7f514effd700 1 --2- 172.21.15.100:0/1021289622 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563239096df0 0x56323909f410 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x563239092710 tx=0x7f51440aa5b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9d9d48d10bf59c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:14.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.611+0000 7f512effd700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:38:14.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.611+0000 7f512effd700 1 -- 172.21.15.100:0/1021289622 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51440095c0 con 0x563239096df0 2024-05-10T11:38:14.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.611+0000 7f512effd700 1 -- 172.21.15.100:0/1021289622 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f514400f880 con 0x563239096df0 2024-05-10T11:38:14.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.611+0000 7f51607b4b80 1 -- 172.21.15.100:0/1021289622 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5148007a50 con 0x563239096df0 2024-05-10T11:38:14.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.611+0000 7f512effd700 1 -- 172.21.15.100:0/1021289622 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f51440a9b80 con 0x563239096df0 2024-05-10T11:38:14.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.611+0000 7f51607b4b80 10 client.28693 fetch_fsmap learned FSMap version 41 2024-05-10T11:38:14.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.611+0000 7f51607b4b80 1 -- 172.21.15.100:0/1021289622 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5632390a2e90 con 0x563239096df0 2024-05-10T11:38:14.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.612+0000 7f512effd700 1 -- 172.21.15.100:0/1021289622 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5148007a50 con 0x563239096df0 2024-05-10T11:38:14.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.612+0000 7f512effd700 1 -- 172.21.15.100:0/1021289622 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f51440050b0 con 0x563239096df0 2024-05-10T11:38:14.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.612+0000 7f51607b4b80 10 client.28693 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:38:14.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.612+0000 7f51607b4b80 10 client.28693 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:38:14.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.612+0000 7f51607b4b80 20 client.28693 populate_metadata read hostname 'smithi100' 2024-05-10T11:38:14.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.612+0000 7f51607b4b80 1 --2- 172.21.15.100:0/1021289622 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5632390afb30 0x5632390b1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:14.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.612+0000 7f51607b4b80 4 client.28693 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:38:14.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.612+0000 7f51607b4b80 1 -- 172.21.15.100:0/1021289622 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563239062650 con 0x5632390afb30 2024-05-10T11:38:14.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.612+0000 7f514f7fe700 1 --2- 172.21.15.100:0/1021289622 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5632390afb30 0x5632390b1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:14.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.613+0000 7f512effd700 10 client.28693 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:38:14.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.613+0000 7f514f7fe700 1 --2- 172.21.15.100:0/1021289622 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5632390afb30 0x5632390b1f50 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cc231d5afa44f9b server_cookie=6f41ba0c4b35e77b in_seq=0 out_seq=0 2024-05-10T11:38:14.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.614+0000 7f512effd700 1 -- 172.21.15.100:0/1021289622 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563239062650 con 0x5632390afb30 2024-05-10T11:38:14.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.614+0000 7f512effd700 10 client.28693 handle_command_reply: tid=0 2024-05-10T11:38:14.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.614+0000 7f51607b4b80 1 client.28693 shutdown 2024-05-10T11:38:14.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.615+0000 7f51607b4b80 1 -- 172.21.15.100:0/1021289622 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563239096df0 msgr2=0x56323909f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:14.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.615+0000 7f51607b4b80 1 --2- 172.21.15.100:0/1021289622 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563239096df0 0x56323909f410 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x563239092710 tx=0x7f51440aa5b0 comp rx=0 tx=0).stop 2024-05-10T11:38:14.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.615+0000 7f51607b4b80 1 -- 172.21.15.100:0/1021289622 shutdown_connections 2024-05-10T11:38:14.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.615+0000 7f51607b4b80 1 --2- 172.21.15.100:0/1021289622 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5632390afb30 0x5632390b1f50 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:14.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.615+0000 7f51607b4b80 1 --2- 172.21.15.100:0/1021289622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56323909ac80 0x56323909f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:14.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.615+0000 7f51607b4b80 1 --2- 172.21.15.100:0/1021289622 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5632390964d0 0x5632390a2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:14.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.615+0000 7f51607b4b80 1 --2- 172.21.15.100:0/1021289622 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563239096df0 0x56323909f410 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:14.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.615+0000 7f512effd700 0 client.28693 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:38:14.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.615+0000 7f51607b4b80 1 -- 172.21.15.100:0/1021289622 >> 172.21.15.100:0/1021289622 conn(0x563238fc2100 msgr2=0x56323908b720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:14.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.615+0000 7f51607b4b80 1 -- 172.21.15.100:0/1021289622 shutdown_connections 2024-05-10T11:38:14.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.615+0000 7f51607b4b80 1 -- 172.21.15.100:0/1021289622 wait complete. 2024-05-10T11:38:14.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.615+0000 7f51607b4b80 20 client.28693 trim_cache size 0 max 16384 2024-05-10T11:38:14.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.649+0000 7f51607b4b80 1 Processor -- start 2024-05-10T11:38:14.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.649+0000 7f51607b4b80 1 -- start start 2024-05-10T11:38:14.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.649+0000 7f51607b4b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5632390964d0 0x56323909e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:14.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.649+0000 7f51607b4b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563239096df0 0x5632390abfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:14.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.649+0000 7f51607b4b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56323909ac80 0x5632390b7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:14.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.649+0000 7f51607b4b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f513c0111f0 con 0x563239096df0 2024-05-10T11:38:14.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.649+0000 7f514f7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5632390964d0 0x56323909e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:14.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.650+0000 7f514f7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5632390964d0 0x56323909e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:51950/0 (socket says 172.21.15.100:51950) 2024-05-10T11:38:14.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.650+0000 7f514f7fe700 1 -- 172.21.15.100:0/2807360685 learned_addr learned my addr 172.21.15.100:0/2807360685 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:38:14.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.650+0000 7f514effd700 1 --2- 172.21.15.100:0/2807360685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563239096df0 0x5632390abfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:14.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.650+0000 7f51607b4b80 1 -- 172.21.15.100:0/2807360685 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f513c00ea80 con 0x56323909ac80 2024-05-10T11:38:14.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.650+0000 7f51607b4b80 1 -- 172.21.15.100:0/2807360685 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f513c00c8d0 con 0x5632390964d0 2024-05-10T11:38:14.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.650+0000 7f514ffff700 1 --2- 172.21.15.100:0/2807360685 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56323909ac80 0x5632390b7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:14.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.650+0000 7f514f7fe700 1 -- 172.21.15.100:0/2807360685 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56323909ac80 msgr2=0x5632390b7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:14.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.650+0000 7f514f7fe700 1 --2- 172.21.15.100:0/2807360685 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56323909ac80 0x5632390b7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:14.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.650+0000 7f514f7fe700 1 -- 172.21.15.100:0/2807360685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563239096df0 msgr2=0x5632390abfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:14.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.650+0000 7f514f7fe700 1 --2- 172.21.15.100:0/2807360685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563239096df0 0x5632390abfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:14.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.650+0000 7f514f7fe700 1 -- 172.21.15.100:0/2807360685 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632390a23e0 con 0x5632390964d0 2024-05-10T11:38:14.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.650+0000 7f514f7fe700 1 --2- 172.21.15.100:0/2807360685 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5632390964d0 0x56323909e0b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f5140013a40 tx=0x7f5140013d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3588534bbc08b22e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:14.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.651+0000 7f51357fa700 1 -- 172.21.15.100:0/2807360685 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5140023070 con 0x5632390964d0 2024-05-10T11:38:14.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.651+0000 7f51357fa700 1 -- 172.21.15.100:0/2807360685 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f51400044c0 con 0x5632390964d0 2024-05-10T11:38:14.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.651+0000 7f51357fa700 1 -- 172.21.15.100:0/2807360685 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5140003680 con 0x5632390964d0 2024-05-10T11:38:14.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.652+0000 7f51607b4b80 1 -- 172.21.15.100:0/2807360685 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5632390964d0 msgr2=0x56323909e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:14.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.652+0000 7f51607b4b80 1 --2- 172.21.15.100:0/2807360685 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5632390964d0 0x56323909e0b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f5140013a40 tx=0x7f5140013d50 comp rx=0 tx=0).stop 2024-05-10T11:38:14.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.652+0000 7f51607b4b80 1 -- 172.21.15.100:0/2807360685 shutdown_connections 2024-05-10T11:38:14.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.652+0000 7f51607b4b80 1 --2- 172.21.15.100:0/2807360685 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56323909ac80 0x5632390b7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:14.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.652+0000 7f51607b4b80 1 --2- 172.21.15.100:0/2807360685 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5632390964d0 0x56323909e0b0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:14.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.652+0000 7f51607b4b80 1 --2- 172.21.15.100:0/2807360685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563239096df0 0x5632390abfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:14.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.652+0000 7f51607b4b80 1 -- 172.21.15.100:0/2807360685 >> 172.21.15.100:0/2807360685 conn(0x563238fc2100 msgr2=0x56323908b720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:14.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.652+0000 7f51607b4b80 1 -- 172.21.15.100:0/2807360685 shutdown_connections 2024-05-10T11:38:14.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.652+0000 7f51607b4b80 1 -- 172.21.15.100:0/2807360685 wait complete. 2024-05-10T11:38:14.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.652+0000 7f51607b4b80 1 -- start start 2024-05-10T11:38:14.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.652+0000 7f51607b4b80 1 Processor -- start 2024-05-10T11:38:14.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.653+0000 7f51607b4b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632390964d0 0x5632390a88a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:14.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.653+0000 7f51607b4b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563239096df0 0x5632390a3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:14.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.653+0000 7f514f7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632390964d0 0x5632390a88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:14.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.653+0000 7f51607b4b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56323909ac80 0x5632390a41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:14.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.653+0000 7f514f7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632390964d0 0x5632390a88a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35942/0 (socket says 172.21.15.100:35942) 2024-05-10T11:38:14.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.653+0000 7f514f7fe700 1 -- 172.21.15.100:0/3378022603 learned_addr learned my addr 172.21.15.100:0/3378022603 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:38:14.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.653+0000 7f51607b4b80 1 -- 172.21.15.100:0/3378022603 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f513c00cf40 con 0x5632390964d0 2024-05-10T11:38:14.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.653+0000 7f51607b4b80 1 -- 172.21.15.100:0/3378022603 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f513c00e3b0 con 0x56323909ac80 2024-05-10T11:38:14.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.653+0000 7f51607b4b80 1 -- 172.21.15.100:0/3378022603 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f513c00d610 con 0x563239096df0 2024-05-10T11:38:14.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.653+0000 7f514effd700 1 --2- 172.21.15.100:0/3378022603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563239096df0 0x5632390a3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:14.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.653+0000 7f51607b4b80 10 client.0 _ll_register_callbacks cb 0x563238f80920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:38:14.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.653+0000 7f514ffff700 1 --2- 172.21.15.100:0/3378022603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56323909ac80 0x5632390a41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:14.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.653+0000 7f514effd700 1 -- 172.21.15.100:0/3378022603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56323909ac80 msgr2=0x5632390a41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:14.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.653+0000 7f514effd700 1 --2- 172.21.15.100:0/3378022603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56323909ac80 0x5632390a41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:14.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.653+0000 7f514effd700 1 -- 172.21.15.100:0/3378022603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632390964d0 msgr2=0x5632390a88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:14.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.653+0000 7f514effd700 1 --2- 172.21.15.100:0/3378022603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632390964d0 0x5632390a88a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:14.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.653+0000 7f514effd700 1 -- 172.21.15.100:0/3378022603 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632390a49f0 con 0x563239096df0 2024-05-10T11:38:14.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.653+0000 7f514effd700 1 --2- 172.21.15.100:0/3378022603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563239096df0 0x5632390a3c70 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f51440a60f0 tx=0x7f5144005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d89d7b48b39f6cc9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:14.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.654+0000 7f512effd700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:38:14.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.654+0000 7f512effd700 1 -- 172.21.15.100:0/3378022603 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51440095c0 con 0x563239096df0 2024-05-10T11:38:14.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.654+0000 7f51607b4b80 1 -- 172.21.15.100:0/3378022603 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51400044c0 con 0x563239096df0 2024-05-10T11:38:14.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.654+0000 7f512effd700 1 -- 172.21.15.100:0/3378022603 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f514400f880 con 0x563239096df0 2024-05-10T11:38:14.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.654+0000 7f512effd700 1 -- 172.21.15.100:0/3378022603 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f51440aa770 con 0x563239096df0 2024-05-10T11:38:14.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.654+0000 7f51607b4b80 10 client.28699 fetch_fsmap learned FSMap version 41 2024-05-10T11:38:14.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.654+0000 7f51607b4b80 1 -- 172.21.15.100:0/3378022603 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x56323911a470 con 0x563239096df0 2024-05-10T11:38:14.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.654+0000 7f512effd700 1 -- 172.21.15.100:0/3378022603 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f51400044c0 con 0x563239096df0 2024-05-10T11:38:14.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.654+0000 7f512effd700 1 -- 172.21.15.100:0/3378022603 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f51440a8d40 con 0x563239096df0 2024-05-10T11:38:14.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.655+0000 7f51607b4b80 10 client.28699 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:38:14.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.655+0000 7f51607b4b80 10 client.28699 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:38:14.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.655+0000 7f51607b4b80 20 client.28699 populate_metadata read hostname 'smithi100' 2024-05-10T11:38:14.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.655+0000 7f51607b4b80 1 --2- 172.21.15.100:0/3378022603 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56323911a9e0 0x56323911cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:14.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.655+0000 7f51607b4b80 4 client.28699 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T11:38:14.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.655+0000 7f51607b4b80 1 -- 172.21.15.100:0/3378022603 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563238ff6240 con 0x56323911a9e0 2024-05-10T11:38:14.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.655+0000 7f514f7fe700 1 --2- 172.21.15.100:0/3378022603 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56323911a9e0 0x56323911cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:14.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.655+0000 7f512effd700 10 client.28699 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:38:14.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.655+0000 7f514f7fe700 1 --2- 172.21.15.100:0/3378022603 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56323911a9e0 0x56323911cdd0 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94f7564020a412d8 server_cookie=ec82bc70c43bcec1 in_seq=0 out_seq=0 2024-05-10T11:38:14.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.655+0000 7f512effd700 1 -- 172.21.15.100:0/3378022603 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563238ff6240 con 0x56323911a9e0 2024-05-10T11:38:14.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.655+0000 7f512effd700 10 client.28699 handle_command_reply: tid=0 2024-05-10T11:38:14.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.655+0000 7f51607b4b80 1 client.28699 shutdown 2024-05-10T11:38:14.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.656+0000 7f51607b4b80 1 -- 172.21.15.100:0/3378022603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563239096df0 msgr2=0x5632390a3c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:14.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.656+0000 7f51607b4b80 1 --2- 172.21.15.100:0/3378022603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563239096df0 0x5632390a3c70 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f51440a60f0 tx=0x7f5144005c00 comp rx=0 tx=0).stop 2024-05-10T11:38:14.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.656+0000 7f51607b4b80 1 -- 172.21.15.100:0/3378022603 shutdown_connections 2024-05-10T11:38:14.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.656+0000 7f51607b4b80 1 --2- 172.21.15.100:0/3378022603 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56323911a9e0 0x56323911cdd0 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:14.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.656+0000 7f51607b4b80 1 --2- 172.21.15.100:0/3378022603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56323909ac80 0x5632390a41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:14.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.656+0000 7f51607b4b80 1 --2- 172.21.15.100:0/3378022603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632390964d0 0x5632390a88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:14.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.656+0000 7f51607b4b80 1 --2- 172.21.15.100:0/3378022603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563239096df0 0x5632390a3c70 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:14.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.656+0000 7f51607b4b80 1 -- 172.21.15.100:0/3378022603 >> 172.21.15.100:0/3378022603 conn(0x563238fc2100 msgr2=0x5632390952e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:14.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.656+0000 7f512effd700 0 client.28699 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:38:14.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.656+0000 7f51607b4b80 1 -- 172.21.15.100:0/3378022603 shutdown_connections 2024-05-10T11:38:14.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.656+0000 7f51607b4b80 1 -- 172.21.15.100:0/3378022603 wait complete. 2024-05-10T11:38:14.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.656+0000 7f51607b4b80 20 client.28699 trim_cache size 0 max 16384 2024-05-10T11:38:14.661 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:38:14.661 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-05-10T11:38:14.661 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-05-10T11:38:14.661 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:38:14.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.657+0000 7f5155f21700 1 -- 172.21.15.100:0/2207748591 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f51380b29c0 msgr2=0x7f51380b4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:14.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.657+0000 7f5155f21700 1 --2- 172.21.15.100:0/2207748591 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f51380b29c0 0x7f51380b4e80 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f51480099d0 tx=0x7f514801ad60 comp rx=0 tx=0).stop 2024-05-10T11:38:14.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.657+0000 7f5155f21700 1 -- 172.21.15.100:0/2207748591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5150102d50 msgr2=0x7f5150198c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:14.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.657+0000 7f5155f21700 1 --2- 172.21.15.100:0/2207748591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5150102d50 0x7f5150198c60 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f5144006010 tx=0x7f51440057c0 comp rx=0 tx=0).stop 2024-05-10T11:38:14.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.657+0000 7f5155f21700 1 -- 172.21.15.100:0/2207748591 shutdown_connections 2024-05-10T11:38:14.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.657+0000 7f5155f21700 1 --2- 172.21.15.100:0/2207748591 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f51380b29c0 0x7f51380b4e80 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:14.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.657+0000 7f5155f21700 1 --2- 172.21.15.100:0/2207748591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5150108400 0x7f515019d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:14.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.657+0000 7f5155f21700 1 --2- 172.21.15.100:0/2207748591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5150102390 0x7f5150198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:14.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.657+0000 7f5155f21700 1 --2- 172.21.15.100:0/2207748591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5150102d50 0x7f5150198c60 secure :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f5144006010 tx=0x7f51440057c0 comp rx=0 tx=0).stop 2024-05-10T11:38:14.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.657+0000 7f5155f21700 1 -- 172.21.15.100:0/2207748591 >> 172.21.15.100:0/2207748591 conn(0x7f5150079310 msgr2=0x7f51501010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:14.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.658+0000 7f5155f21700 1 -- 172.21.15.100:0/2207748591 shutdown_connections 2024-05-10T11:38:14.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:14.658+0000 7f5155f21700 1 -- 172.21.15.100:0/2207748591 wait complete. 2024-05-10T11:38:14.937 INFO:tasks.workunit.client.0.smithi100.stdout: 10 61641 8.97 MB/sec execute 248 sec latency 85.686 ms 2024-05-10T11:38:15.189 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.187+0000 7f0da3140640 1 -- 172.21.15.100:0/2700366984 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d9c101910 msgr2=0x7f0d9c101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:15.189 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.187+0000 7f0da3140640 1 --2- 172.21.15.100:0/2700366984 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d9c101910 0x7f0d9c101d30 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f0d9000d910 tx=0x7f0d9002a160 comp rx=0 tx=0).stop 2024-05-10T11:38:15.189 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.187+0000 7f0da3140640 1 -- 172.21.15.100:0/2700366984 shutdown_connections 2024-05-10T11:38:15.189 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.187+0000 7f0da3140640 1 --2- 172.21.15.100:0/2700366984 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d9c1034c0 0x7f0d9c109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:15.189 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.187+0000 7f0da3140640 1 --2- 172.21.15.100:0/2700366984 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d9c102b00 0x7f0d9c102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:15.189 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.187+0000 7f0da3140640 1 --2- 172.21.15.100:0/2700366984 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d9c101910 0x7f0d9c101d30 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:15.190 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.187+0000 7f0da3140640 1 -- 172.21.15.100:0/2700366984 >> 172.21.15.100:0/2700366984 conn(0x7f0d9c0fce80 msgr2=0x7f0d9c0ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:15.190 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.187+0000 7f0da3140640 1 -- 172.21.15.100:0/2700366984 shutdown_connections 2024-05-10T11:38:15.190 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.187+0000 7f0da3140640 1 -- 172.21.15.100:0/2700366984 wait complete. 2024-05-10T11:38:15.190 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.188+0000 7f0da3140640 1 Processor -- start 2024-05-10T11:38:15.190 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.188+0000 7f0da3140640 1 -- start start 2024-05-10T11:38:15.190 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.188+0000 7f0da3140640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d9c101910 0x7f0d9c1968f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:15.190 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.188+0000 7f0da3140640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d9c102b00 0x7f0d9c196e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:15.190 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.188+0000 7f0da3140640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d9c1034c0 0x7f0d9c19dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:15.190 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.188+0000 7f0da3140640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c10c240 con 0x7f0d9c1034c0 2024-05-10T11:38:15.190 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.188+0000 7f0da3140640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c10c0c0 con 0x7f0d9c101910 2024-05-10T11:38:15.190 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.188+0000 7f0da3140640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c10c3c0 con 0x7f0d9c102b00 2024-05-10T11:38:15.190 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.188+0000 7f0d9bfff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d9c102b00 0x7f0d9c196e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:15.191 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.188+0000 7f0da0edc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d9c101910 0x7f0d9c1968f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:15.191 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.188+0000 7f0d9bfff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d9c102b00 0x7f0d9c196e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:51980/0 (socket says 172.21.15.100:51980) 2024-05-10T11:38:15.191 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.188+0000 7f0da0edc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d9c101910 0x7f0d9c1968f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:55276/0 (socket says 172.21.15.100:55276) 2024-05-10T11:38:15.191 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.188+0000 7f0d9bfff640 1 -- 172.21.15.100:0/1457049603 learned_addr learned my addr 172.21.15.100:0/1457049603 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:38:15.191 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.188+0000 7f0da16dd640 1 --2- 172.21.15.100:0/1457049603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d9c1034c0 0x7f0d9c19dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:15.191 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.188+0000 7f0d9bfff640 1 -- 172.21.15.100:0/1457049603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d9c101910 msgr2=0x7f0d9c1968f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:15.191 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.188+0000 7f0d9bfff640 1 --2- 172.21.15.100:0/1457049603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d9c101910 0x7f0d9c1968f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:15.191 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.188+0000 7f0d9bfff640 1 -- 172.21.15.100:0/1457049603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d9c1034c0 msgr2=0x7f0d9c19dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:15.191 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.188+0000 7f0d9bfff640 1 --2- 172.21.15.100:0/1457049603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d9c1034c0 0x7f0d9c19dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:15.191 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.188+0000 7f0d9bfff640 1 -- 172.21.15.100:0/1457049603 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d9c19e510 con 0x7f0d9c102b00 2024-05-10T11:38:15.191 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.188+0000 7f0da0edc640 1 --2- 172.21.15.100:0/1457049603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d9c101910 0x7f0d9c1968f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:38:15.191 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.189+0000 7f0d9bfff640 1 --2- 172.21.15.100:0/1457049603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d9c102b00 0x7f0d9c196e30 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f0d88009ca0 tx=0x7f0d88007590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:15.191 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.189+0000 7f0d99ffb640 1 -- 172.21.15.100:0/1457049603 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d880195f0 con 0x7f0d9c102b00 2024-05-10T11:38:15.192 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.189+0000 7f0da3140640 1 -- 172.21.15.100:0/1457049603 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d9c19e800 con 0x7f0d9c102b00 2024-05-10T11:38:15.192 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.189+0000 7f0d99ffb640 1 -- 172.21.15.100:0/1457049603 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0d8801e070 con 0x7f0d9c102b00 2024-05-10T11:38:15.192 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.189+0000 7f0d99ffb640 1 -- 172.21.15.100:0/1457049603 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d880187b0 con 0x7f0d9c102b00 2024-05-10T11:38:15.192 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.189+0000 7f0da3140640 1 -- 172.21.15.100:0/1457049603 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d9c19ed50 con 0x7f0d9c102b00 2024-05-10T11:38:15.194 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.192+0000 7f0d99ffb640 1 -- 172.21.15.100:0/1457049603 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f0d88019790 con 0x7f0d9c102b00 2024-05-10T11:38:15.194 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.192+0000 7f0da5e5c740 1 Processor -- start 2024-05-10T11:38:15.194 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.193+0000 7f0da5e5c740 1 -- start start 2024-05-10T11:38:15.194 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.193+0000 7f0da5e5c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd4dcd6db0 0x55cd4dcd71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:15.194 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.193+0000 7f0da5e5c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cd4dcd77a0 0x55cd4dcdc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:15.195 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.193+0000 7f0da5e5c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd4dcdc910 0x55cd4dcdeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:15.195 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.193+0000 7f0da5e5c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55cd4dc3b5d0 con 0x55cd4dcd6db0 2024-05-10T11:38:15.195 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.193+0000 7f0da5e5c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55cd4dbf4890 con 0x55cd4dcd77a0 2024-05-10T11:38:15.195 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.193+0000 7f0da5e5c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55cd4dc37590 con 0x55cd4dcdc910 2024-05-10T11:38:15.195 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.193+0000 7f0da0edc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cd4dcd77a0 0x55cd4dcdc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:15.195 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.193+0000 7f0da16dd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd4dcd6db0 0x55cd4dcd71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:15.195 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.193+0000 7f0da0edc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cd4dcd77a0 0x55cd4dcdc2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:55278/0 (socket says 172.21.15.100:55278) 2024-05-10T11:38:15.195 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.193+0000 7f0da0edc640 1 -- 172.21.15.100:0/2076398607 learned_addr learned my addr 172.21.15.100:0/2076398607 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:38:15.195 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.193+0000 7f0d9bfff640 1 --2- 172.21.15.100:0/2076398607 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd4dcdc910 0x55cd4dcdeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:15.195 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.193+0000 7f0da0edc640 1 -- 172.21.15.100:0/2076398607 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd4dcdc910 msgr2=0x55cd4dcdeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:15.195 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.193+0000 7f0da0edc640 1 --2- 172.21.15.100:0/2076398607 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd4dcdc910 0x55cd4dcdeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:15.195 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.193+0000 7f0da0edc640 1 -- 172.21.15.100:0/2076398607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd4dcd6db0 msgr2=0x55cd4dcd71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:15.195 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.193+0000 7f0da0edc640 1 --2- 172.21.15.100:0/2076398607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd4dcd6db0 0x55cd4dcd71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:15.196 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.193+0000 7f0da0edc640 1 -- 172.21.15.100:0/2076398607 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd4d94d210 con 0x55cd4dcd77a0 2024-05-10T11:38:15.196 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.194+0000 7f0da16dd640 1 --2- 172.21.15.100:0/2076398607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd4dcd6db0 0x55cd4dcd71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:38:15.196 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.194+0000 7f0da0edc640 1 --2- 172.21.15.100:0/2076398607 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cd4dcd77a0 0x55cd4dcdc2a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0d90005ab0 tx=0x7f0d9000d540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d6a3f74df2f96a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:15.196 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.194+0000 7f0d99ffb640 1 --2- 172.21.15.100:0/1457049603 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0d8c0b28a0 0x7f0d8c0b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:15.196 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.194+0000 7f0da16dd640 1 --2- 172.21.15.100:0/1457049603 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0d8c0b28a0 0x7f0d8c0b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:15.196 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.194+0000 7f0d99ffb640 1 -- 172.21.15.100:0/1457049603 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f0d880e6180 con 0x7f0d9c102b00 2024-05-10T11:38:15.196 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.194+0000 7f0d86ffd640 1 -- 172.21.15.100:0/2076398607 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d90040070 con 0x55cd4dcd77a0 2024-05-10T11:38:15.196 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.194+0000 7f0d86ffd640 1 -- 172.21.15.100:0/2076398607 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0d9002e070 con 0x55cd4dcd77a0 2024-05-10T11:38:15.196 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.194+0000 7f0d86ffd640 1 -- 172.21.15.100:0/2076398607 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9003c430 con 0x55cd4dcd77a0 2024-05-10T11:38:15.201 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.195+0000 7f0da16dd640 1 --2- 172.21.15.100:0/1457049603 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0d8c0b28a0 0x7f0d8c0b4d60 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x55cd4dcd8000 tx=0x7f0d94009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:15.201 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.195+0000 7f0da5e5c740 1 -- 172.21.15.100:0/2076398607 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cd4dcd77a0 msgr2=0x55cd4dcdc2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:15.201 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.195+0000 7f0da5e5c740 1 --2- 172.21.15.100:0/2076398607 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cd4dcd77a0 0x55cd4dcdc2a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0d90005ab0 tx=0x7f0d9000d540 comp rx=0 tx=0).stop 2024-05-10T11:38:15.201 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.195+0000 7f0da5e5c740 1 -- 172.21.15.100:0/2076398607 shutdown_connections 2024-05-10T11:38:15.201 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.195+0000 7f0da5e5c740 1 --2- 172.21.15.100:0/2076398607 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd4dcdc910 0x55cd4dcdeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:15.201 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.195+0000 7f0da5e5c740 1 --2- 172.21.15.100:0/2076398607 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cd4dcd77a0 0x55cd4dcdc2a0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:15.201 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.195+0000 7f0da5e5c740 1 --2- 172.21.15.100:0/2076398607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd4dcd6db0 0x55cd4dcd71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:15.202 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.195+0000 7f0da5e5c740 1 -- 172.21.15.100:0/2076398607 >> 172.21.15.100:0/2076398607 conn(0x55cd4dcd2420 msgr2=0x55cd4dcd2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:15.202 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.195+0000 7f0da5e5c740 1 -- 172.21.15.100:0/2076398607 shutdown_connections 2024-05-10T11:38:15.202 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.195+0000 7f0da5e5c740 1 -- 172.21.15.100:0/2076398607 wait complete. 2024-05-10T11:38:15.202 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.195+0000 7f0da5e5c740 1 -- start start 2024-05-10T11:38:15.202 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.196+0000 7f0da5e5c740 1 Processor -- start 2024-05-10T11:38:15.202 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.196+0000 7f0da5e5c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cd4dcd6db0 0x55cd4dce0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:15.202 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.196+0000 7f0da5e5c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd4dcd77a0 0x55cd4dce1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:15.202 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.196+0000 7f0da5e5c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd4dcdc910 0x55cd4dce8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:15.202 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.196+0000 7f0da5e5c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55cd4dcdf860 con 0x55cd4dcd77a0 2024-05-10T11:38:15.202 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.196+0000 7f0da5e5c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55cd4dbf8370 con 0x55cd4dcd6db0 2024-05-10T11:38:15.202 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.196+0000 7f0da5e5c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55cd4dcdf9e0 con 0x55cd4dcdc910 2024-05-10T11:38:15.202 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.196+0000 7f0da0edc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cd4dcd6db0 0x55cd4dce0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.196+0000 7f0da16dd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd4dcdc910 0x55cd4dce8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.196+0000 7f0da16dd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd4dcdc910 0x55cd4dce8570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:51992/0 (socket says 172.21.15.100:51992) 2024-05-10T11:38:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.196+0000 7f0da16dd640 1 -- 172.21.15.100:0/623187047 learned_addr learned my addr 172.21.15.100:0/623187047 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:38:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.196+0000 7f0da5e5c740 10 client.0 _ll_register_callbacks cb 0x55cd4d94ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:38:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.196+0000 7f0d9bfff640 1 --2- 172.21.15.100:0/623187047 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd4dcd77a0 0x55cd4dce1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.197+0000 7f0da16dd640 1 -- 172.21.15.100:0/623187047 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cd4dcd6db0 msgr2=0x55cd4dce0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.197+0000 7f0da16dd640 1 --2- 172.21.15.100:0/623187047 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cd4dcd6db0 0x55cd4dce0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.197+0000 7f0da16dd640 1 -- 172.21.15.100:0/623187047 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd4dcd77a0 msgr2=0x55cd4dce1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.197+0000 7f0da16dd640 1 --2- 172.21.15.100:0/623187047 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd4dcd77a0 0x55cd4dce1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.197+0000 7f0da16dd640 1 -- 172.21.15.100:0/623187047 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd4da19620 con 0x55cd4dcdc910 2024-05-10T11:38:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.197+0000 7f0d9bfff640 1 --2- 172.21.15.100:0/623187047 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd4dcd77a0 0x55cd4dce1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:38:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.197+0000 7f0da16dd640 1 --2- 172.21.15.100:0/623187047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd4dcdc910 0x55cd4dce8570 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0d94003bb0 tx=0x7f0d94004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13103fba534f309e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.197+0000 7f0d7bfff640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:38:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.197+0000 7f0d7bfff640 1 -- 172.21.15.100:0/623187047 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d94004310 con 0x55cd4dcdc910 2024-05-10T11:38:15.204 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.197+0000 7f0d7bfff640 1 -- 172.21.15.100:0/623187047 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f0d940044b0 con 0x55cd4dcdc910 2024-05-10T11:38:15.204 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.197+0000 7f0da5e5c740 1 -- 172.21.15.100:0/623187047 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d9002e070 con 0x55cd4dcdc910 2024-05-10T11:38:15.204 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.198+0000 7f0d7bfff640 1 -- 172.21.15.100:0/623187047 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f0d940046b0 con 0x55cd4dcdc910 2024-05-10T11:38:15.204 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.198+0000 7f0da5e5c740 10 client.28711 fetch_fsmap learned FSMap version 41 2024-05-10T11:38:15.204 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.198+0000 7f0da5e5c740 1 -- 172.21.15.100:0/623187047 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55cd4dceb670 con 0x55cd4dcdc910 2024-05-10T11:38:15.204 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.198+0000 7f0d7bfff640 1 -- 172.21.15.100:0/623187047 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0d9002e070 con 0x55cd4dcdc910 2024-05-10T11:38:15.204 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.198+0000 7f0d7bfff640 1 -- 172.21.15.100:0/623187047 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f0d94026070 con 0x55cd4dcdc910 2024-05-10T11:38:15.204 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.198+0000 7f0da5e5c740 10 client.28711 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:38:15.204 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.198+0000 7f0da5e5c740 10 client.28711 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:38:15.204 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.198+0000 7f0da5e5c740 20 client.28711 populate_metadata read hostname 'smithi100' 2024-05-10T11:38:15.204 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.198+0000 7f0da5e5c740 1 --2- 172.21.15.100:0/623187047 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55cd4dcecdb0 0x55cd4dcef210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:15.204 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.198+0000 7f0da5e5c740 4 client.28711 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:38:15.204 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.198+0000 7f0da5e5c740 1 -- 172.21.15.100:0/623187047 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd4dc491e0 con 0x55cd4dcecdb0 2024-05-10T11:38:15.205 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.198+0000 7f0da0edc640 1 --2- 172.21.15.100:0/623187047 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55cd4dcecdb0 0x55cd4dcef210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:15.205 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.199+0000 7f0da0edc640 1 --2- 172.21.15.100:0/623187047 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55cd4dcecdb0 0x55cd4dcef210 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=523d73ecb8172477 server_cookie=7c8077a63a5e9195 in_seq=0 out_seq=0 2024-05-10T11:38:15.205 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.199+0000 7f0d7bfff640 10 client.28711 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:38:15.205 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.199+0000 7f0d7bfff640 1 -- 172.21.15.100:0/623187047 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55cd4dc491e0 con 0x55cd4dcecdb0 2024-05-10T11:38:15.205 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.199+0000 7f0d7bfff640 10 client.28711 handle_command_reply: tid=0 2024-05-10T11:38:15.205 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.199+0000 7f0da5e5c740 1 client.28711 shutdown 2024-05-10T11:38:15.205 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.200+0000 7f0da5e5c740 1 -- 172.21.15.100:0/623187047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd4dcdc910 msgr2=0x55cd4dce8570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:15.205 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.200+0000 7f0da5e5c740 1 --2- 172.21.15.100:0/623187047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd4dcdc910 0x55cd4dce8570 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0d94003bb0 tx=0x7f0d94004ab0 comp rx=0 tx=0).stop 2024-05-10T11:38:15.205 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.200+0000 7f0da5e5c740 1 -- 172.21.15.100:0/623187047 shutdown_connections 2024-05-10T11:38:15.205 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.200+0000 7f0da5e5c740 1 --2- 172.21.15.100:0/623187047 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55cd4dcecdb0 0x55cd4dcef210 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:15.205 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.200+0000 7f0da5e5c740 1 --2- 172.21.15.100:0/623187047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd4dcdc910 0x55cd4dce8570 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:15.205 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.200+0000 7f0da5e5c740 1 --2- 172.21.15.100:0/623187047 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd4dcd77a0 0x55cd4dce1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:15.205 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.200+0000 7f0da5e5c740 1 --2- 172.21.15.100:0/623187047 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cd4dcd6db0 0x55cd4dce0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:15.206 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.200+0000 7f0da5e5c740 1 -- 172.21.15.100:0/623187047 >> 172.21.15.100:0/623187047 conn(0x55cd4dcd2420 msgr2=0x55cd4dcd1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:15.206 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.200+0000 7f0d7bfff640 0 client.28711 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:38:15.206 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.200+0000 7f0da5e5c740 1 -- 172.21.15.100:0/623187047 shutdown_connections 2024-05-10T11:38:15.206 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.200+0000 7f0da5e5c740 1 -- 172.21.15.100:0/623187047 wait complete. 2024-05-10T11:38:15.206 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.200+0000 7f0da5e5c740 20 client.28711 trim_cache size 0 max 16384 2024-05-10T11:38:15.225 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.224+0000 7f0da5e5c740 1 Processor -- start 2024-05-10T11:38:15.225 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.224+0000 7f0da5e5c740 1 -- start start 2024-05-10T11:38:15.225 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.224+0000 7f0da5e5c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cd4dcd6db0 0x55cd4dce0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:15.225 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.224+0000 7f0da5e5c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd4dcd77a0 0x55cd4dce1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:15.225 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.224+0000 7f0da5e5c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd4dcdc910 0x55cd4dce8530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:15.225 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.224+0000 7f0da5e5c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c1aba10 con 0x55cd4dcdc910 2024-05-10T11:38:15.225 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.224+0000 7f0da5e5c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c1a92e0 con 0x55cd4dcd6db0 2024-05-10T11:38:15.225 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.224+0000 7f0da5e5c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c1a7130 con 0x55cd4dcd77a0 2024-05-10T11:38:15.225 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.224+0000 7f0da16dd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd4dcdc910 0x55cd4dce8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:15.226 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.224+0000 7f0da16dd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd4dcdc910 0x55cd4dce8530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35980/0 (socket says 172.21.15.100:35980) 2024-05-10T11:38:15.226 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.224+0000 7f0da16dd640 1 -- 172.21.15.100:0/3560913805 learned_addr learned my addr 172.21.15.100:0/3560913805 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:38:15.226 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.224+0000 7f0da0edc640 1 --2- 172.21.15.100:0/3560913805 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cd4dcd6db0 0x55cd4dce0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:15.226 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.224+0000 7f0d9bfff640 1 --2- 172.21.15.100:0/3560913805 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd4dcd77a0 0x55cd4dce1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:15.226 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.225+0000 7f0da0edc640 1 -- 172.21.15.100:0/3560913805 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd4dcd77a0 msgr2=0x55cd4dce1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:15.226 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.225+0000 7f0da0edc640 1 --2- 172.21.15.100:0/3560913805 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd4dcd77a0 0x55cd4dce1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:15.226 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.225+0000 7f0da0edc640 1 -- 172.21.15.100:0/3560913805 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd4dcdc910 msgr2=0x55cd4dce8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:15.226 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.225+0000 7f0da0edc640 1 --2- 172.21.15.100:0/3560913805 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd4dcdc910 0x55cd4dce8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:15.226 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.225+0000 7f0da0edc640 1 -- 172.21.15.100:0/3560913805 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd4dcd2250 con 0x55cd4dcd6db0 2024-05-10T11:38:15.226 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.225+0000 7f0da16dd640 1 --2- 172.21.15.100:0/3560913805 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd4dcdc910 0x55cd4dce8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:38:15.226 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.225+0000 7f0da0edc640 1 --2- 172.21.15.100:0/3560913805 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cd4dcd6db0 0x55cd4dce0ff0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x55cd4dcd7ba0 tx=0x7f0d9002a160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8abe05e5582016f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:15.226 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.225+0000 7f0d867fc640 1 -- 172.21.15.100:0/3560913805 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d90040040 con 0x55cd4dcd6db0 2024-05-10T11:38:15.227 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.225+0000 7f0d867fc640 1 -- 172.21.15.100:0/3560913805 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0d9002e040 con 0x55cd4dcd6db0 2024-05-10T11:38:15.227 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.225+0000 7f0d867fc640 1 -- 172.21.15.100:0/3560913805 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9000c7e0 con 0x55cd4dcd6db0 2024-05-10T11:38:15.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.226+0000 7f0da5e5c740 1 -- 172.21.15.100:0/3560913805 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cd4dcd6db0 msgr2=0x55cd4dce0ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:15.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.226+0000 7f0da5e5c740 1 --2- 172.21.15.100:0/3560913805 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cd4dcd6db0 0x55cd4dce0ff0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x55cd4dcd7ba0 tx=0x7f0d9002a160 comp rx=0 tx=0).stop 2024-05-10T11:38:15.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.226+0000 7f0da5e5c740 1 -- 172.21.15.100:0/3560913805 shutdown_connections 2024-05-10T11:38:15.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.226+0000 7f0da5e5c740 1 --2- 172.21.15.100:0/3560913805 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd4dcdc910 0x55cd4dce8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:15.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.226+0000 7f0da5e5c740 1 --2- 172.21.15.100:0/3560913805 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd4dcd77a0 0x55cd4dce1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:15.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.226+0000 7f0da5e5c740 1 --2- 172.21.15.100:0/3560913805 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cd4dcd6db0 0x55cd4dce0ff0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:15.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.226+0000 7f0da5e5c740 1 -- 172.21.15.100:0/3560913805 >> 172.21.15.100:0/3560913805 conn(0x55cd4dcd2420 msgr2=0x55cd4dccd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:15.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.226+0000 7f0da5e5c740 1 -- 172.21.15.100:0/3560913805 shutdown_connections 2024-05-10T11:38:15.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.226+0000 7f0da5e5c740 1 -- 172.21.15.100:0/3560913805 wait complete. 2024-05-10T11:38:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.226+0000 7f0da5e5c740 1 -- start start 2024-05-10T11:38:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.227+0000 7f0da5e5c740 1 Processor -- start 2024-05-10T11:38:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.227+0000 7f0da5e5c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cd4dcd6db0 0x55cd4dce5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.227+0000 7f0da5e5c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd4dcd77a0 0x55cd4dce5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.227+0000 7f0da5e5c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd4dcdc910 0x55cd4dd556c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.227+0000 7f0da5e5c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c1a77a0 con 0x55cd4dcd77a0 2024-05-10T11:38:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.227+0000 7f0da5e5c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c1a8c10 con 0x55cd4dcd6db0 2024-05-10T11:38:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.227+0000 7f0da5e5c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c1a7e70 con 0x55cd4dcdc910 2024-05-10T11:38:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.227+0000 7f0da5e5c740 10 client.0 _ll_register_callbacks cb 0x7f0d94005e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:38:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.227+0000 7f0da16dd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd4dcdc910 0x55cd4dd556c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.227+0000 7f0d9bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd4dcd77a0 0x55cd4dce5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.227+0000 7f0da0edc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cd4dcd6db0 0x55cd4dce5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.227+0000 7f0da16dd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd4dcdc910 0x55cd4dd556c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:52002/0 (socket says 172.21.15.100:52002) 2024-05-10T11:38:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.227+0000 7f0da16dd640 1 -- 172.21.15.100:0/1388411466 learned_addr learned my addr 172.21.15.100:0/1388411466 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:38:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.227+0000 7f0da0edc640 1 -- 172.21.15.100:0/1388411466 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd4dcdc910 msgr2=0x55cd4dd556c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.227+0000 7f0da0edc640 1 --2- 172.21.15.100:0/1388411466 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd4dcdc910 0x55cd4dd556c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.227+0000 7f0da0edc640 1 -- 172.21.15.100:0/1388411466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd4dcd77a0 msgr2=0x55cd4dce5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.227+0000 7f0da0edc640 1 --2- 172.21.15.100:0/1388411466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd4dcd77a0 0x55cd4dce5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.227+0000 7f0da0edc640 1 -- 172.21.15.100:0/1388411466 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd4dce00b0 con 0x55cd4dcd6db0 2024-05-10T11:38:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.228+0000 7f0da0edc640 1 --2- 172.21.15.100:0/1388411466 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cd4dcd6db0 0x55cd4dce5350 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0d9002d040 tx=0x7f0d9000ce90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9419aab8f1ca67ef server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.228+0000 7f0d7bfff640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:38:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.228+0000 7f0d7bfff640 1 -- 172.21.15.100:0/1388411466 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d90040040 con 0x55cd4dcd6db0 2024-05-10T11:38:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.228+0000 7f0da5e5c740 1 -- 172.21.15.100:0/1388411466 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d9002e040 con 0x55cd4dcd6db0 2024-05-10T11:38:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.228+0000 7f0d7bfff640 1 -- 172.21.15.100:0/1388411466 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f0d9000c560 con 0x55cd4dcd6db0 2024-05-10T11:38:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.228+0000 7f0d7bfff640 1 -- 172.21.15.100:0/1388411466 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f0d9002ae60 con 0x55cd4dcd6db0 2024-05-10T11:38:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.228+0000 7f0da5e5c740 10 client.28610 fetch_fsmap learned FSMap version 41 2024-05-10T11:38:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.228+0000 7f0da5e5c740 1 -- 172.21.15.100:0/1388411466 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55cd4dd56c90 con 0x55cd4dcd6db0 2024-05-10T11:38:15.233 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.228+0000 7f0d7bfff640 1 -- 172.21.15.100:0/1388411466 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0d9002e040 con 0x55cd4dcd6db0 2024-05-10T11:38:15.233 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.229+0000 7f0d7bfff640 1 -- 172.21.15.100:0/1388411466 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f0d94011d20 con 0x55cd4dcd6db0 2024-05-10T11:38:15.233 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.229+0000 7f0da5e5c740 10 client.28610 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:38:15.233 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.229+0000 7f0da5e5c740 10 client.28610 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:38:15.233 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.229+0000 7f0da5e5c740 20 client.28610 populate_metadata read hostname 'smithi100' 2024-05-10T11:38:15.233 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.229+0000 7f0da5e5c740 1 --2- 172.21.15.100:0/1388411466 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55cd4dd571e0 0x55cd4dd595d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:15.233 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.229+0000 7f0da5e5c740 4 client.28610 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:38:15.233 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.229+0000 7f0da5e5c740 1 -- 172.21.15.100:0/1388411466 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cd4dc43050 con 0x55cd4dd571e0 2024-05-10T11:38:15.233 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.229+0000 7f0d9bfff640 1 --2- 172.21.15.100:0/1388411466 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55cd4dd571e0 0x55cd4dd595d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:15.233 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.229+0000 7f0d7bfff640 10 client.28610 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:38:15.233 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.229+0000 7f0d9bfff640 1 --2- 172.21.15.100:0/1388411466 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55cd4dd571e0 0x55cd4dd595d0 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81f7dc51e765b4ea server_cookie=5c742e0d2d279dd6 in_seq=0 out_seq=0 2024-05-10T11:38:15.333 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.332+0000 7f0d7bfff640 1 -- 172.21.15.100:0/1388411466 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cd4dc43050 con 0x55cd4dd571e0 2024-05-10T11:38:15.333 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.332+0000 7f0d7bfff640 10 client.28610 handle_command_reply: tid=0 2024-05-10T11:38:15.333 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.332+0000 7f0da5e5c740 1 client.28610 shutdown 2024-05-10T11:38:15.333 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.333+0000 7f0da5e5c740 1 -- 172.21.15.100:0/1388411466 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cd4dcd6db0 msgr2=0x55cd4dce5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:15.333 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.333+0000 7f0da5e5c740 1 --2- 172.21.15.100:0/1388411466 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cd4dcd6db0 0x55cd4dce5350 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0d9002d040 tx=0x7f0d9000ce90 comp rx=0 tx=0).stop 2024-05-10T11:38:15.333 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.333+0000 7f0da5e5c740 1 -- 172.21.15.100:0/1388411466 shutdown_connections 2024-05-10T11:38:15.333 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.333+0000 7f0da5e5c740 1 --2- 172.21.15.100:0/1388411466 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55cd4dd571e0 0x55cd4dd595d0 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:15.334 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.333+0000 7f0da5e5c740 1 --2- 172.21.15.100:0/1388411466 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd4dcdc910 0x55cd4dd556c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:15.334 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.333+0000 7f0da5e5c740 1 --2- 172.21.15.100:0/1388411466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd4dcd77a0 0x55cd4dce5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:15.334 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.333+0000 7f0d7bfff640 0 client.28610 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:38:15.334 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.333+0000 7f0da5e5c740 1 --2- 172.21.15.100:0/1388411466 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cd4dcd6db0 0x55cd4dce5350 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:15.334 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.333+0000 7f0da5e5c740 1 -- 172.21.15.100:0/1388411466 >> 172.21.15.100:0/1388411466 conn(0x55cd4dcd2420 msgr2=0x55cd4dce0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:15.334 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.333+0000 7f0da5e5c740 1 -- 172.21.15.100:0/1388411466 shutdown_connections 2024-05-10T11:38:15.334 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.333+0000 7f0da5e5c740 1 -- 172.21.15.100:0/1388411466 wait complete. 2024-05-10T11:38:15.334 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.333+0000 7f0da5e5c740 20 client.28610 trim_cache size 0 max 16384 2024-05-10T11:38:15.335 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:38:15.335 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:38:15.335 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:38:15.335 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:38:15.336 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.334+0000 7f0da3140640 1 -- 172.21.15.100:0/1457049603 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0d8c0b28a0 msgr2=0x7f0d8c0b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:15.336 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.334+0000 7f0da3140640 1 --2- 172.21.15.100:0/1457049603 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0d8c0b28a0 0x7f0d8c0b4d60 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x55cd4dcd8000 tx=0x7f0d94009500 comp rx=0 tx=0).stop 2024-05-10T11:38:15.336 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.334+0000 7f0da3140640 1 -- 172.21.15.100:0/1457049603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d9c102b00 msgr2=0x7f0d9c196e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:15.336 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.334+0000 7f0da3140640 1 --2- 172.21.15.100:0/1457049603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d9c102b00 0x7f0d9c196e30 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f0d88009ca0 tx=0x7f0d88007590 comp rx=0 tx=0).stop 2024-05-10T11:38:15.336 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.334+0000 7f0da3140640 1 -- 172.21.15.100:0/1457049603 shutdown_connections 2024-05-10T11:38:15.336 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.334+0000 7f0da3140640 1 --2- 172.21.15.100:0/1457049603 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0d8c0b28a0 0x7f0d8c0b4d60 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:15.336 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.334+0000 7f0da3140640 1 --2- 172.21.15.100:0/1457049603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d9c1034c0 0x7f0d9c19dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:15.336 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.334+0000 7f0da3140640 1 --2- 172.21.15.100:0/1457049603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d9c102b00 0x7f0d9c196e30 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:15.336 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.334+0000 7f0da3140640 1 --2- 172.21.15.100:0/1457049603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d9c101910 0x7f0d9c1968f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:15.336 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.334+0000 7f0da3140640 1 -- 172.21.15.100:0/1457049603 >> 172.21.15.100:0/1457049603 conn(0x7f0d9c0fce80 msgr2=0x7f0d9c105f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:15.336 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.334+0000 7f0da3140640 1 -- 172.21.15.100:0/1457049603 shutdown_connections 2024-05-10T11:38:15.336 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:15.334+0000 7f0da3140640 1 -- 172.21.15.100:0/1457049603 wait complete. 2024-05-10T11:38:15.504 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T11:38:15.505 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T11:38:15.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:15 smithi100 ceph-mon[30296]: pgmap v1017: 129 pgs: 129 active+clean; 493 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 3.3 MiB/s wr, 327 op/s 2024-05-10T11:38:15.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:15 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:38:15.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:15 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:38:15.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:15 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:38:15.880 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:38:15.937 INFO:tasks.workunit.client.0.smithi100.stdout: 10 61802 8.99 MB/sec execute 249 sec latency 111.475 ms 2024-05-10T11:38:16.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:15 smithi178 ceph-mon[33359]: pgmap v1017: 129 pgs: 129 active+clean; 493 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 3.3 MiB/s wr, 327 op/s 2024-05-10T11:38:16.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:15 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:38:16.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:15 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:38:16.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:15 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:38:16.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:15 smithi188 ceph-mon[34075]: pgmap v1017: 129 pgs: 129 active+clean; 493 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 3.3 MiB/s wr, 327 op/s 2024-05-10T11:38:16.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:15 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:38:16.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:15 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:38:16.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:15 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:38:16.938 INFO:tasks.workunit.client.0.smithi100.stdout: 10 61999 9.01 MB/sec execute 250 sec latency 90.877 ms 2024-05-10T11:38:17.938 INFO:tasks.workunit.client.0.smithi100.stdout: 10 62158 9.00 MB/sec execute 251 sec latency 80.526 ms 2024-05-10T11:38:17.969 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:17 smithi100 ceph-mon[30296]: pgmap v1018: 129 pgs: 129 active+clean; 496 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 3.6 MiB/s wr, 315 op/s 2024-05-10T11:38:18.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:17 smithi178 ceph-mon[33359]: pgmap v1018: 129 pgs: 129 active+clean; 496 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 3.6 MiB/s wr, 315 op/s 2024-05-10T11:38:18.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:17 smithi188 ceph-mon[34075]: pgmap v1018: 129 pgs: 129 active+clean; 496 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 3.6 MiB/s wr, 315 op/s 2024-05-10T11:38:18.938 INFO:tasks.workunit.client.0.smithi100.stdout: 10 62283 8.98 MB/sec execute 252 sec latency 108.074 ms 2024-05-10T11:38:19.938 INFO:tasks.workunit.client.0.smithi100.stdout: 10 62469 8.99 MB/sec execute 253 sec latency 166.186 ms 2024-05-10T11:38:19.998 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:19 smithi100 ceph-mon[30296]: pgmap v1019: 129 pgs: 129 active+clean; 499 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 3.7 MiB/s wr, 301 op/s 2024-05-10T11:38:20.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:19 smithi178 ceph-mon[33359]: pgmap v1019: 129 pgs: 129 active+clean; 499 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 3.7 MiB/s wr, 301 op/s 2024-05-10T11:38:20.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:19 smithi188 ceph-mon[34075]: pgmap v1019: 129 pgs: 129 active+clean; 499 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 3.7 MiB/s wr, 301 op/s 2024-05-10T11:38:20.937 INFO:tasks.workunit.client.0.smithi100.stdout: 10 62627 8.98 MB/sec execute 254 sec latency 204.840 ms 2024-05-10T11:38:21.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:20 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/4267229553' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:38:21.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:20 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/4267229553' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:38:21.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:20 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/4267229553' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:38:21.083 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:38:21.083 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2072225012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2072225012},{"type":"v1","addr":"172.21.15.100:6835","nonce":2072225012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/401939540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":401939540},{"type":"v1","addr":"172.21.15.100:6837","nonce":401939540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14571,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/3946311025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3946311025},{"type":"v1","addr":"172.21.15.100:6839","nonce":3946311025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3736371660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3736371660},{"type":"v1","addr":"172.21.15.100:6841","nonce":3736371660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/2741810791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2741810791},{"type":"v1","addr":"172.21.15.178:6833","nonce":2741810791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1704891041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1704891041},{"type":"v1","addr":"172.21.15.188:6833","nonce":1704891041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":41,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T11:14:27.242452+0000","modified":"2024-05-10T11:35:29.588072+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24439,"mds_1":24437,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/2124076312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2124076312},{"type":"v1","addr":"172.21.15.178:6835","nonce":2124076312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/3224358425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3224358425},{"type":"v1","addr":"172.21.15.178:6837","nonce":3224358425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/1943342620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1943342620},{"type":"v1","addr":"172.21.15.188:6837","nonce":1943342620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.178:6839/3760985404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3760985404},{"type":"v1","addr":"172.21.15.178:6839","nonce":3760985404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.188:6839/3194917070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":3194917070},{"type":"v1","addr":"172.21.15.188:6839","nonce":3194917070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27034":{"gid":27034,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/3139559450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3139559450},{"type":"v1","addr":"172.21.15.188:6835","nonce":3139559450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24439,"qdb_cluster":[24437,24439,24427]},"id":1}]} 2024-05-10T11:38:21.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:20.156+0000 7f226e951700 1 -- 172.21.15.100:0/2545634100 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2268108410 msgr2=0x7f22681087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:21.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:20.156+0000 7f226e951700 1 --2- 172.21.15.100:0/2545634100 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2268108410 0x7f22681087f0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f2258009990 tx=0x7f2258009ca0 comp rx=0 tx=0).stop 2024-05-10T11:38:21.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:20.156+0000 7f226e951700 1 -- 172.21.15.100:0/2545634100 shutdown_connections 2024-05-10T11:38:21.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:20.156+0000 7f226e951700 1 --2- 172.21.15.100:0/2545634100 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2268102d60 0x7f226810f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:21.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:20.156+0000 7f226e951700 1 --2- 172.21.15.100:0/2545634100 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2268108410 0x7f22681087f0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:21.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:20.156+0000 7f226e951700 1 --2- 172.21.15.100:0/2545634100 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22681023a0 0x7f2268102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:21.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:20.156+0000 7f226e951700 1 -- 172.21.15.100:0/2545634100 >> 172.21.15.100:0/2545634100 conn(0x7f2268079350 msgr2=0x7f2268079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:21.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:20.156+0000 7f226e951700 1 -- 172.21.15.100:0/2545634100 shutdown_connections 2024-05-10T11:38:21.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:20.156+0000 7f226e951700 1 -- 172.21.15.100:0/2545634100 wait complete. 2024-05-10T11:38:21.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:20.156+0000 7f226e951700 1 Processor -- start 2024-05-10T11:38:21.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:20.157+0000 7f226e951700 1 -- start start 2024-05-10T11:38:21.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:20.157+0000 7f226e951700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f22681023a0 0x7f22681a1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:21.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:20.157+0000 7f226e951700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2268102d60 0x7f22681a1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:21.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:20.157+0000 7f226e951700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2268108410 0x7f226819bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:21.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:20.157+0000 7f226e951700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2268111320 con 0x7f2268108410 2024-05-10T11:38:21.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:20.157+0000 7f226e951700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f22681111a0 con 0x7f22681023a0 2024-05-10T11:38:21.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:20.157+0000 7f226e951700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f22681114a0 con 0x7f2268102d60 2024-05-10T11:38:21.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:20.157+0000 7f226ceee700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2268108410 0x7f226819bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:21.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:20.157+0000 7f226ceee700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2268108410 0x7f226819bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38422/0 (socket says 172.21.15.100:38422) 2024-05-10T11:38:21.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:20.157+0000 7f226ceee700 1 -- 172.21.15.100:0/4267229553 learned_addr learned my addr 172.21.15.100:0/4267229553 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:38:21.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:20.157+0000 7f2267fff700 1 --2- 172.21.15.100:0/4267229553 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f22681023a0 0x7f22681a1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:21.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:20.157+0000 7f22677fe700 1 --2- 172.21.15.100:0/4267229553 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2268102d60 0x7f22681a1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:21.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:20.158+0000 7f22677fe700 1 -- 172.21.15.100:0/4267229553 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f22681023a0 msgr2=0x7f22681a1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:21.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:20.158+0000 7f22677fe700 1 --2- 172.21.15.100:0/4267229553 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f22681023a0 0x7f22681a1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:21.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:20.158+0000 7f22677fe700 1 -- 172.21.15.100:0/4267229553 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2268108410 msgr2=0x7f226819bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:21.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:20.158+0000 7f22677fe700 1 --2- 172.21.15.100:0/4267229553 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2268108410 0x7f226819bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:21.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:20.158+0000 7f22677fe700 1 -- 172.21.15.100:0/4267229553 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f226819c300 con 0x7f2268102d60 2024-05-10T11:38:21.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:20.158+0000 7f226ceee700 1 --2- 172.21.15.100:0/4267229553 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2268108410 0x7f226819bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:38:21.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:20.158+0000 7f2267fff700 1 --2- 172.21.15.100:0/4267229553 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f22681023a0 0x7f22681a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:38:21.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:20.158+0000 7f22677fe700 1 --2- 172.21.15.100:0/4267229553 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2268102d60 0x7f22681a1ea0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f225c00e9f0 tx=0x7f225c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:21.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:20.158+0000 7f22657fa700 1 -- 172.21.15.100:0/4267229553 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f225c00c9c0 con 0x7f2268102d60 2024-05-10T11:38:21.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:20.158+0000 7f226e951700 1 -- 172.21.15.100:0/4267229553 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f226819c5f0 con 0x7f2268102d60 2024-05-10T11:38:21.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:20.158+0000 7f22657fa700 1 -- 172.21.15.100:0/4267229553 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f225c010430 con 0x7f2268102d60 2024-05-10T11:38:21.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:20.158+0000 7f226e951700 1 -- 172.21.15.100:0/4267229553 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22681a8970 con 0x7f2268102d60 2024-05-10T11:38:21.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:20.158+0000 7f22657fa700 1 -- 172.21.15.100:0/4267229553 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f225c018680 con 0x7f2268102d60 2024-05-10T11:38:21.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:20.159+0000 7f226e951700 1 -- 172.21.15.100:0/4267229553 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2254005330 con 0x7f2268102d60 2024-05-10T11:38:21.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:20.161+0000 7f22657fa700 1 -- 172.21.15.100:0/4267229553 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f225c00cb60 con 0x7f2268102d60 2024-05-10T11:38:21.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:20.162+0000 7f22657fa700 1 --2- 172.21.15.100:0/4267229553 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f22500b2980 0x7f22500b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:21.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:20.162+0000 7f2267fff700 1 --2- 172.21.15.100:0/4267229553 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f22500b2980 0x7f22500b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:21.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:20.162+0000 7f22657fa700 1 -- 172.21.15.100:0/4267229553 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f225c0e2860 con 0x7f2268102d60 2024-05-10T11:38:21.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:20.163+0000 7f2267fff700 1 --2- 172.21.15.100:0/4267229553 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f22500b2980 0x7f22500b4e40 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f225800b2e0 tx=0x7f2258005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:21.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:20.165+0000 7f22657fa700 1 -- 172.21.15.100:0/4267229553 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f225c0aa1c0 con 0x7f2268102d60 2024-05-10T11:38:21.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:20.540+0000 7f226e951700 1 -- 172.21.15.100:0/4267229553 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2254005620 con 0x7f2268102d60 2024-05-10T11:38:21.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:20.541+0000 7f22657fa700 1 -- 172.21.15.100:0/4267229553 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+11897 (secure 0 0 0) 0x7f225c0b2240 con 0x7f2268102d60 2024-05-10T11:38:21.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:20.545+0000 7f226e951700 1 -- 172.21.15.100:0/4267229553 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f22500b2980 msgr2=0x7f22500b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:21.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:20.545+0000 7f226e951700 1 --2- 172.21.15.100:0/4267229553 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f22500b2980 0x7f22500b4e40 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f225800b2e0 tx=0x7f2258005fb0 comp rx=0 tx=0).stop 2024-05-10T11:38:21.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:20.545+0000 7f226e951700 1 -- 172.21.15.100:0/4267229553 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2268102d60 msgr2=0x7f22681a1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:21.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:20.545+0000 7f226e951700 1 --2- 172.21.15.100:0/4267229553 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2268102d60 0x7f22681a1ea0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f225c00e9f0 tx=0x7f225c00ed00 comp rx=0 tx=0).stop 2024-05-10T11:38:21.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:20.545+0000 7f226e951700 1 -- 172.21.15.100:0/4267229553 shutdown_connections 2024-05-10T11:38:21.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:20.545+0000 7f226e951700 1 --2- 172.21.15.100:0/4267229553 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f22500b2980 0x7f22500b4e40 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:21.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:20.545+0000 7f226e951700 1 --2- 172.21.15.100:0/4267229553 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2268108410 0x7f226819bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:21.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:20.545+0000 7f226e951700 1 --2- 172.21.15.100:0/4267229553 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f22681023a0 0x7f22681a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:21.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:20.545+0000 7f226e951700 1 --2- 172.21.15.100:0/4267229553 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2268102d60 0x7f22681a1ea0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:21.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:20.545+0000 7f226e951700 1 -- 172.21.15.100:0/4267229553 >> 172.21.15.100:0/4267229553 conn(0x7f2268079350 msgr2=0x7f22681005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:21.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:20.545+0000 7f226e951700 1 -- 172.21.15.100:0/4267229553 shutdown_connections 2024-05-10T11:38:21.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:20.545+0000 7f226e951700 1 -- 172.21.15.100:0/4267229553 wait complete. 2024-05-10T11:38:21.087 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 41 2024-05-10T11:38:21.942 INFO:tasks.workunit.client.0.smithi100.stdout: 10 62760 8.96 MB/sec execute 255 sec latency 110.659 ms 2024-05-10T11:38:22.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:21 smithi178 ceph-mon[33359]: pgmap v1020: 129 pgs: 129 active+clean; 501 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 4.2 MiB/s wr, 271 op/s 2024-05-10T11:38:22.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:21 smithi188 ceph-mon[34075]: pgmap v1020: 129 pgs: 129 active+clean; 501 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 4.2 MiB/s wr, 271 op/s 2024-05-10T11:38:22.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:21 smithi100 ceph-mon[30296]: pgmap v1020: 129 pgs: 129 active+clean; 501 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 4.2 MiB/s wr, 271 op/s 2024-05-10T11:38:22.942 INFO:tasks.workunit.client.0.smithi100.stdout: 10 62888 8.95 MB/sec execute 256 sec latency 142.633 ms 2024-05-10T11:38:22.966 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-10T11:38:23.342 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:38:23.761 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:23 smithi100 ceph-mon[30296]: pgmap v1021: 129 pgs: 129 active+clean; 504 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 3.8 MiB/s wr, 235 op/s 2024-05-10T11:38:23.944 INFO:tasks.workunit.client.0.smithi100.stdout: 10 63067 8.96 MB/sec execute 257 sec latency 99.275 ms 2024-05-10T11:38:24.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:23 smithi178 ceph-mon[33359]: pgmap v1021: 129 pgs: 129 active+clean; 504 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 3.8 MiB/s wr, 235 op/s 2024-05-10T11:38:24.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:23 smithi188 ceph-mon[34075]: pgmap v1021: 129 pgs: 129 active+clean; 504 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 3.8 MiB/s wr, 235 op/s 2024-05-10T11:38:24.943 INFO:tasks.workunit.client.0.smithi100.stdout: 10 63293 8.99 MB/sec execute 258 sec latency 90.763 ms 2024-05-10T11:38:25.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.436+0000 7f1e299ca700 1 -- 172.21.15.100:0/656102483 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e240756d0 msgr2=0x7f1e24075ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:25.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.436+0000 7f1e299ca700 1 --2- 172.21.15.100:0/656102483 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e240756d0 0x7f1e24075ab0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f1e14009990 tx=0x7f1e14009ca0 comp rx=0 tx=0).stop 2024-05-10T11:38:25.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.436+0000 7f1e299ca700 1 -- 172.21.15.100:0/656102483 shutdown_connections 2024-05-10T11:38:25.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.436+0000 7f1e299ca700 1 --2- 172.21.15.100:0/656102483 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e24114670 0x7f1e24116a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:25.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.436+0000 7f1e299ca700 1 --2- 172.21.15.100:0/656102483 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e240756d0 0x7f1e24075ab0 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:25.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.436+0000 7f1e299ca700 1 --2- 172.21.15.100:0/656102483 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e24075ff0 0x7f1e24114130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:25.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.436+0000 7f1e299ca700 1 -- 172.21.15.100:0/656102483 >> 172.21.15.100:0/656102483 conn(0x7f1e2407a600 msgr2=0x7f1e2407aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:25.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.436+0000 7f1e299ca700 1 -- 172.21.15.100:0/656102483 shutdown_connections 2024-05-10T11:38:25.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.436+0000 7f1e299ca700 1 -- 172.21.15.100:0/656102483 wait complete. 2024-05-10T11:38:25.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.437+0000 7f1e299ca700 1 Processor -- start 2024-05-10T11:38:25.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.437+0000 7f1e299ca700 1 -- start start 2024-05-10T11:38:25.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.437+0000 7f1e299ca700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e240756d0 0x7f1e241a0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:25.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.437+0000 7f1e299ca700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e24075ff0 0x7f1e241a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:25.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.437+0000 7f1e299ca700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e24114670 0x7f1e241a4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:25.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.438+0000 7f1e299ca700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1e24119b00 con 0x7f1e24114670 2024-05-10T11:38:25.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.438+0000 7f1e299ca700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1e24119980 con 0x7f1e240756d0 2024-05-10T11:38:25.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.438+0000 7f1e299ca700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1e24119c80 con 0x7f1e24075ff0 2024-05-10T11:38:25.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.438+0000 7f1e237fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e24114670 0x7f1e241a4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:25.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.438+0000 7f1e237fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e24114670 0x7f1e241a4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38440/0 (socket says 172.21.15.100:38440) 2024-05-10T11:38:25.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.438+0000 7f1e237fe700 1 -- 172.21.15.100:0/1034446740 learned_addr learned my addr 172.21.15.100:0/1034446740 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:38:25.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.438+0000 7f1e227fc700 1 --2- 172.21.15.100:0/1034446740 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e24075ff0 0x7f1e241a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:25.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.438+0000 7f1e22ffd700 1 --2- 172.21.15.100:0/1034446740 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e240756d0 0x7f1e241a0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:25.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.438+0000 7f1e237fe700 1 -- 172.21.15.100:0/1034446740 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e24075ff0 msgr2=0x7f1e241a05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:25.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.438+0000 7f1e237fe700 1 --2- 172.21.15.100:0/1034446740 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e24075ff0 0x7f1e241a05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:25.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.438+0000 7f1e237fe700 1 -- 172.21.15.100:0/1034446740 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e240756d0 msgr2=0x7f1e241a0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:25.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.438+0000 7f1e237fe700 1 --2- 172.21.15.100:0/1034446740 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e240756d0 0x7f1e241a0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:25.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.438+0000 7f1e237fe700 1 -- 172.21.15.100:0/1034446740 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e241a50a0 con 0x7f1e24114670 2024-05-10T11:38:25.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.439+0000 7f1e237fe700 1 --2- 172.21.15.100:0/1034446740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e24114670 0x7f1e241a4960 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c007ec0 tx=0x7f1e1c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:25.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.439+0000 7f1e289c8700 1 -- 172.21.15.100:0/1034446740 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c00dbd0 con 0x7f1e24114670 2024-05-10T11:38:25.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.439+0000 7f1e289c8700 1 -- 172.21.15.100:0/1034446740 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1e1c00dd70 con 0x7f1e24114670 2024-05-10T11:38:25.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.439+0000 7f1e289c8700 1 -- 172.21.15.100:0/1034446740 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c004440 con 0x7f1e24114670 2024-05-10T11:38:25.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.439+0000 7f1e299ca700 1 -- 172.21.15.100:0/1034446740 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e241a5390 con 0x7f1e24114670 2024-05-10T11:38:25.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.440+0000 7f1e299ca700 1 -- 172.21.15.100:0/1034446740 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e241acc10 con 0x7f1e24114670 2024-05-10T11:38:25.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.443+0000 7f1e289c8700 1 -- 172.21.15.100:0/1034446740 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f1e1c01a030 con 0x7f1e24114670 2024-05-10T11:38:25.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.444+0000 7f1e289c8700 1 --2- 172.21.15.100:0/1034446740 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f1e0c0b2900 0x7f1e0c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:25.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.444+0000 7f1e22ffd700 1 --2- 172.21.15.100:0/1034446740 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f1e0c0b2900 0x7f1e0c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:25.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.444+0000 7f1e289c8700 1 -- 172.21.15.100:0/1034446740 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f1e1c0e1550 con 0x7f1e24114670 2024-05-10T11:38:25.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.445+0000 7f1e3425db80 1 Processor -- start 2024-05-10T11:38:25.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.446+0000 7f1e3425db80 1 -- start start 2024-05-10T11:38:25.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.446+0000 7f1e3425db80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb548f54d0 0x55bb548f58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:25.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.446+0000 7f1e3425db80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb548f5df0 0x55bb548f96f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:25.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.446+0000 7f1e3425db80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb548f9c80 0x55bb548fa100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:25.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.446+0000 7f1e227fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb548f5df0 0x55bb548f96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:25.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.446+0000 7f1e227fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb548f5df0 0x55bb548f96f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38448/0 (socket says 172.21.15.100:38448) 2024-05-10T11:38:25.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.446+0000 7f1e227fc700 1 -- 172.21.15.100:0/78467909 learned_addr learned my addr 172.21.15.100:0/78467909 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:38:25.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.446+0000 7f1e3425db80 1 -- 172.21.15.100:0/78467909 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55bb54847860 con 0x55bb548f5df0 2024-05-10T11:38:25.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.446+0000 7f1e3425db80 1 -- 172.21.15.100:0/78467909 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55bb547e67c0 con 0x55bb548f9c80 2024-05-10T11:38:25.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.446+0000 7f1e3425db80 1 -- 172.21.15.100:0/78467909 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55bb54843820 con 0x55bb548f54d0 2024-05-10T11:38:25.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.446+0000 7f1e237fe700 1 --2- 172.21.15.100:0/78467909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb548f9c80 0x55bb548fa100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:25.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.447+0000 7f1e22ffd700 1 --2- 172.21.15.100:0/1034446740 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f1e0c0b2900 0x7f1e0c0b4dc0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f1e1400b2e0 tx=0x7f1e1401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:25.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.447+0000 7f1e227fc700 1 -- 172.21.15.100:0/78467909 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb548f54d0 msgr2=0x55bb548f58b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T11:38:25.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.447+0000 7f1e227fc700 1 --2- 172.21.15.100:0/78467909 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb548f54d0 0x55bb548f58b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:25.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.447+0000 7f1e227fc700 1 -- 172.21.15.100:0/78467909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb548f9c80 msgr2=0x55bb548fa100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:25.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.447+0000 7f1e227fc700 1 --2- 172.21.15.100:0/78467909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb548f9c80 0x55bb548fa100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:25.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.447+0000 7f1e227fc700 1 -- 172.21.15.100:0/78467909 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb547948d0 con 0x55bb548f5df0 2024-05-10T11:38:25.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.447+0000 7f1e237fe700 1 --2- 172.21.15.100:0/78467909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb548f9c80 0x55bb548fa100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:38:25.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.447+0000 7f1e227fc700 1 --2- 172.21.15.100:0/78467909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb548f5df0 0x55bb548f96f0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f1e18009d20 tx=0x7f1e18011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cc384edbd8c5115 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:25.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.448+0000 7f1e09ffb700 1 -- 172.21.15.100:0/78467909 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e18004a30 con 0x55bb548f5df0 2024-05-10T11:38:25.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.448+0000 7f1e09ffb700 1 -- 172.21.15.100:0/78467909 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1e180054b0 con 0x55bb548f5df0 2024-05-10T11:38:25.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.448+0000 7f1e09ffb700 1 -- 172.21.15.100:0/78467909 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e180198a0 con 0x55bb548f5df0 2024-05-10T11:38:25.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.449+0000 7f1e3425db80 1 -- 172.21.15.100:0/78467909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb548f5df0 msgr2=0x55bb548f96f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:25.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.449+0000 7f1e3425db80 1 --2- 172.21.15.100:0/78467909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb548f5df0 0x55bb548f96f0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f1e18009d20 tx=0x7f1e18011720 comp rx=0 tx=0).stop 2024-05-10T11:38:25.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.449+0000 7f1e3425db80 1 -- 172.21.15.100:0/78467909 shutdown_connections 2024-05-10T11:38:25.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.449+0000 7f1e3425db80 1 --2- 172.21.15.100:0/78467909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb548f9c80 0x55bb548fa100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:25.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.449+0000 7f1e3425db80 1 --2- 172.21.15.100:0/78467909 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb548f54d0 0x55bb548f58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:25.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.449+0000 7f1e3425db80 1 --2- 172.21.15.100:0/78467909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb548f5df0 0x55bb548f96f0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:25.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.449+0000 7f1e3425db80 1 -- 172.21.15.100:0/78467909 >> 172.21.15.100:0/78467909 conn(0x55bb54821100 msgr2=0x55bb548efc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:25.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.449+0000 7f1e3425db80 1 -- 172.21.15.100:0/78467909 shutdown_connections 2024-05-10T11:38:25.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.449+0000 7f1e3425db80 1 -- 172.21.15.100:0/78467909 wait complete. 2024-05-10T11:38:25.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.450+0000 7f1e3425db80 1 -- start start 2024-05-10T11:38:25.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.450+0000 7f1e3425db80 1 Processor -- start 2024-05-10T11:38:25.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.450+0000 7f1e3425db80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb548f54d0 0x55bb54900f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:25.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.450+0000 7f1e3425db80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb548f5df0 0x55bb548fe2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:25.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.451+0000 7f1e3425db80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb548f9c80 0x55bb548fe830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:25.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.451+0000 7f1e3425db80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55bb54901f30 con 0x55bb548f5df0 2024-05-10T11:38:25.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.451+0000 7f1e3425db80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55bb547adc30 con 0x55bb548f54d0 2024-05-10T11:38:25.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.451+0000 7f1e3425db80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55bb549020b0 con 0x55bb548f9c80 2024-05-10T11:38:25.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.451+0000 7f1e3425db80 10 client.0 _ll_register_callbacks cb 0x55bb547df920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:38:25.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.451+0000 7f1e22ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb548f54d0 0x55bb54900f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:25.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.451+0000 7f1e227fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb548f5df0 0x55bb548fe2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:25.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.451+0000 7f1e22ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb548f54d0 0x55bb54900f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:38272/0 (socket says 172.21.15.100:38272) 2024-05-10T11:38:25.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.451+0000 7f1e227fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb548f5df0 0x55bb548fe2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38456/0 (socket says 172.21.15.100:38456) 2024-05-10T11:38:25.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.451+0000 7f1e22ffd700 1 -- 172.21.15.100:0/2422970627 learned_addr learned my addr 172.21.15.100:0/2422970627 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:38:25.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.451+0000 7f1e237fe700 1 --2- 172.21.15.100:0/2422970627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb548f9c80 0x55bb548fe830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:25.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.451+0000 7f1e22ffd700 1 -- 172.21.15.100:0/2422970627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb548f9c80 msgr2=0x55bb548fe830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:25.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.451+0000 7f1e22ffd700 1 --2- 172.21.15.100:0/2422970627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb548f9c80 0x55bb548fe830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:25.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.451+0000 7f1e22ffd700 1 -- 172.21.15.100:0/2422970627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb548f5df0 msgr2=0x55bb548fe2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:25.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.451+0000 7f1e22ffd700 1 --2- 172.21.15.100:0/2422970627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb548f5df0 0x55bb548fe2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:25.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.451+0000 7f1e22ffd700 1 -- 172.21.15.100:0/2422970627 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb547e3690 con 0x55bb548f54d0 2024-05-10T11:38:25.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.451+0000 7f1e237fe700 1 --2- 172.21.15.100:0/2422970627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb548f9c80 0x55bb548fe830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:38:25.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.451+0000 7f1e22ffd700 1 --2- 172.21.15.100:0/2422970627 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb548f54d0 0x55bb54900f50 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1e1400f6e0 tx=0x7f1e14009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=318c18c3832be922 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:25.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.451+0000 7f1e02ffd700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:38:25.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.452+0000 7f1e02ffd700 1 -- 172.21.15.100:0/2422970627 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e14032060 con 0x55bb548f54d0 2024-05-10T11:38:25.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.452+0000 7f1e02ffd700 1 -- 172.21.15.100:0/2422970627 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f1e14023610 con 0x55bb548f54d0 2024-05-10T11:38:25.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.452+0000 7f1e3425db80 1 -- 172.21.15.100:0/2422970627 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e180054b0 con 0x55bb548f54d0 2024-05-10T11:38:25.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.452+0000 7f1e02ffd700 1 -- 172.21.15.100:0/2422970627 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f1e140095c0 con 0x55bb548f54d0 2024-05-10T11:38:25.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.452+0000 7f1e3425db80 10 client.28628 fetch_fsmap learned FSMap version 41 2024-05-10T11:38:25.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.452+0000 7f1e3425db80 1 -- 172.21.15.100:0/2422970627 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55bb5490c680 con 0x55bb548f54d0 2024-05-10T11:38:25.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.452+0000 7f1e02ffd700 1 -- 172.21.15.100:0/2422970627 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1e180054b0 con 0x55bb548f54d0 2024-05-10T11:38:25.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.452+0000 7f1e02ffd700 1 -- 172.21.15.100:0/2422970627 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f1e14017880 con 0x55bb548f54d0 2024-05-10T11:38:25.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.452+0000 7f1e3425db80 10 client.28628 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:38:25.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.452+0000 7f1e3425db80 10 client.28628 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-10T11:38:25.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.452+0000 7f1e3425db80 20 client.28628 populate_metadata read hostname 'smithi100' 2024-05-10T11:38:25.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.452+0000 7f1e3425db80 1 --2- 172.21.15.100:0/2422970627 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55bb5490ed10 0x55bb54911140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:25.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.452+0000 7f1e3425db80 4 client.28628 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:38:25.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.452+0000 7f1e3425db80 1 -- 172.21.15.100:0/2422970627 --> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb548c1650 con 0x55bb5490ed10 2024-05-10T11:38:25.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.453+0000 7f1e227fc700 1 --2- 172.21.15.100:0/2422970627 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55bb5490ed10 0x55bb54911140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:25.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.453+0000 7f1e02ffd700 10 client.28628 ms_handle_connect on v2:172.21.15.188:6836/1943342620 2024-05-10T11:38:25.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.453+0000 7f1e227fc700 1 --2- 172.21.15.100:0/2422970627 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55bb5490ed10 0x55bb54911140 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=106d651fba5eac1b server_cookie=c5126c8d64e2e264 in_seq=0 out_seq=0 2024-05-10T11:38:25.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.454+0000 7f1e02ffd700 1 -- 172.21.15.100:0/2422970627 <== mds.2 v2:172.21.15.188:6836/1943342620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bb548c1650 con 0x55bb5490ed10 2024-05-10T11:38:25.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.454+0000 7f1e02ffd700 10 client.28628 handle_command_reply: tid=0 2024-05-10T11:38:25.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.454+0000 7f1e3425db80 1 client.28628 shutdown 2024-05-10T11:38:25.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.454+0000 7f1e3425db80 1 -- 172.21.15.100:0/2422970627 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb548f54d0 msgr2=0x55bb54900f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:25.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.454+0000 7f1e3425db80 1 --2- 172.21.15.100:0/2422970627 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb548f54d0 0x55bb54900f50 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1e1400f6e0 tx=0x7f1e14009310 comp rx=0 tx=0).stop 2024-05-10T11:38:25.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.454+0000 7f1e3425db80 1 -- 172.21.15.100:0/2422970627 shutdown_connections 2024-05-10T11:38:25.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.454+0000 7f1e3425db80 1 --2- 172.21.15.100:0/2422970627 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55bb5490ed10 0x55bb54911140 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:25.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.454+0000 7f1e3425db80 1 --2- 172.21.15.100:0/2422970627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb548f9c80 0x55bb548fe830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:25.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.454+0000 7f1e02ffd700 0 client.28628 ms_handle_reset on v2:172.21.15.188:6836/1943342620 2024-05-10T11:38:25.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.454+0000 7f1e3425db80 1 --2- 172.21.15.100:0/2422970627 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb548f54d0 0x55bb54900f50 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:25.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.455+0000 7f1e3425db80 1 --2- 172.21.15.100:0/2422970627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb548f5df0 0x55bb548fe2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:25.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.455+0000 7f1e3425db80 1 -- 172.21.15.100:0/2422970627 >> 172.21.15.100:0/2422970627 conn(0x55bb54821100 msgr2=0x55bb548ea720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:25.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.455+0000 7f1e3425db80 1 -- 172.21.15.100:0/2422970627 shutdown_connections 2024-05-10T11:38:25.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.455+0000 7f1e3425db80 1 -- 172.21.15.100:0/2422970627 wait complete. 2024-05-10T11:38:25.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.455+0000 7f1e3425db80 20 client.28628 trim_cache size 0 max 16384 2024-05-10T11:38:25.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.477+0000 7f1e3425db80 1 Processor -- start 2024-05-10T11:38:25.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.477+0000 7f1e3425db80 1 -- start start 2024-05-10T11:38:25.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.478+0000 7f1e3425db80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb548f54d0 0x55bb548f75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:25.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.478+0000 7f1e3425db80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb548f5df0 0x55bb548f7b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:25.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.478+0000 7f1e3425db80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb548f9c80 0x55bb548e9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:25.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.478+0000 7f1e3425db80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1e241b7f90 con 0x55bb548f9c80 2024-05-10T11:38:25.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.478+0000 7f1e3425db80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1e241b5820 con 0x55bb548f54d0 2024-05-10T11:38:25.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.478+0000 7f1e3425db80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1e241b3670 con 0x55bb548f5df0 2024-05-10T11:38:25.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.478+0000 7f1e22ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb548f54d0 0x55bb548f75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:25.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.478+0000 7f1e237fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb548f9c80 0x55bb548e9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:25.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.478+0000 7f1e22ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb548f54d0 0x55bb548f75f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:38282/0 (socket says 172.21.15.100:38282) 2024-05-10T11:38:25.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.478+0000 7f1e237fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb548f9c80 0x55bb548e9e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38458/0 (socket says 172.21.15.100:38458) 2024-05-10T11:38:25.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.478+0000 7f1e22ffd700 1 -- 172.21.15.100:0/2563561869 learned_addr learned my addr 172.21.15.100:0/2563561869 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:38:25.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.478+0000 7f1e227fc700 1 --2- 172.21.15.100:0/2563561869 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb548f5df0 0x55bb548f7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:25.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.478+0000 7f1e237fe700 1 -- 172.21.15.100:0/2563561869 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb548f5df0 msgr2=0x55bb548f7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:25.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.478+0000 7f1e237fe700 1 --2- 172.21.15.100:0/2563561869 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb548f5df0 0x55bb548f7b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:25.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.478+0000 7f1e237fe700 1 -- 172.21.15.100:0/2563561869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb548f54d0 msgr2=0x55bb548f75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:25.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.478+0000 7f1e237fe700 1 --2- 172.21.15.100:0/2563561869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb548f54d0 0x55bb548f75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:25.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.478+0000 7f1e237fe700 1 -- 172.21.15.100:0/2563561869 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb54901da0 con 0x55bb548f9c80 2024-05-10T11:38:25.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.478+0000 7f1e22ffd700 1 --2- 172.21.15.100:0/2563561869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb548f54d0 0x55bb548f75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:38:25.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.479+0000 7f1e237fe700 1 --2- 172.21.15.100:0/2563561869 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb548f9c80 0x55bb548e9e90 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c0a5b10 tx=0x7f1e1c0a4040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8896db1c2dbd01be server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:25.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.479+0000 7f1e097fa700 1 -- 172.21.15.100:0/2563561869 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c0a88a0 con 0x55bb548f9c80 2024-05-10T11:38:25.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.479+0000 7f1e097fa700 1 -- 172.21.15.100:0/2563561869 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1e1c0acc50 con 0x55bb548f9c80 2024-05-10T11:38:25.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.479+0000 7f1e097fa700 1 -- 172.21.15.100:0/2563561869 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c0a8b00 con 0x55bb548f9c80 2024-05-10T11:38:25.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.480+0000 7f1e3425db80 1 -- 172.21.15.100:0/2563561869 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb548f9c80 msgr2=0x55bb548e9e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:25.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.480+0000 7f1e3425db80 1 --2- 172.21.15.100:0/2563561869 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb548f9c80 0x55bb548e9e90 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c0a5b10 tx=0x7f1e1c0a4040 comp rx=0 tx=0).stop 2024-05-10T11:38:25.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.480+0000 7f1e3425db80 1 -- 172.21.15.100:0/2563561869 shutdown_connections 2024-05-10T11:38:25.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.480+0000 7f1e3425db80 1 --2- 172.21.15.100:0/2563561869 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb548f9c80 0x55bb548e9e90 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:25.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.480+0000 7f1e3425db80 1 --2- 172.21.15.100:0/2563561869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb548f54d0 0x55bb548f75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:25.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.480+0000 7f1e3425db80 1 --2- 172.21.15.100:0/2563561869 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb548f5df0 0x55bb548f7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:25.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.480+0000 7f1e3425db80 1 -- 172.21.15.100:0/2563561869 >> 172.21.15.100:0/2563561869 conn(0x55bb54821100 msgr2=0x55bb548fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:25.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.480+0000 7f1e3425db80 1 -- 172.21.15.100:0/2563561869 shutdown_connections 2024-05-10T11:38:25.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.480+0000 7f1e3425db80 1 -- 172.21.15.100:0/2563561869 wait complete. 2024-05-10T11:38:25.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.480+0000 7f1e3425db80 1 -- start start 2024-05-10T11:38:25.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.481+0000 7f1e3425db80 1 Processor -- start 2024-05-10T11:38:25.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.481+0000 7f1e3425db80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb548f54d0 0x55bb54907520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:25.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.481+0000 7f1e3425db80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb548f5df0 0x55bb54907a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:25.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.481+0000 7f1e3425db80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb548f9c80 0x55bb54915de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:25.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.481+0000 7f1e3425db80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1e241b3ce0 con 0x55bb548f54d0 2024-05-10T11:38:25.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.481+0000 7f1e3425db80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1e241b5150 con 0x55bb548f5df0 2024-05-10T11:38:25.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.481+0000 7f1e3425db80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1e241b43b0 con 0x55bb548f9c80 2024-05-10T11:38:25.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.481+0000 7f1e3425db80 10 client.0 _ll_register_callbacks cb 0x55bb547df920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:38:25.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.481+0000 7f1e227fc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb548f5df0 0x55bb54907a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:25.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.481+0000 7f1e227fc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb548f5df0 0x55bb54907a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:38286/0 (socket says 172.21.15.100:38286) 2024-05-10T11:38:25.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.481+0000 7f1e227fc700 1 -- 172.21.15.100:0/1033875766 learned_addr learned my addr 172.21.15.100:0/1033875766 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:38:25.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.481+0000 7f1e237fe700 1 --2- 172.21.15.100:0/1033875766 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb548f9c80 0x55bb54915de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:25.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.481+0000 7f1e22ffd700 1 --2- 172.21.15.100:0/1033875766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb548f54d0 0x55bb54907520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:25.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.481+0000 7f1e227fc700 1 -- 172.21.15.100:0/1033875766 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb548f9c80 msgr2=0x55bb54915de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:25.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.481+0000 7f1e227fc700 1 --2- 172.21.15.100:0/1033875766 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb548f9c80 0x55bb54915de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:25.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.481+0000 7f1e227fc700 1 -- 172.21.15.100:0/1033875766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb548f54d0 msgr2=0x55bb54907520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:25.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.481+0000 7f1e227fc700 1 --2- 172.21.15.100:0/1033875766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb548f54d0 0x55bb54907520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:25.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.481+0000 7f1e227fc700 1 -- 172.21.15.100:0/1033875766 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb549165f0 con 0x55bb548f5df0 2024-05-10T11:38:25.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.482+0000 7f1e227fc700 1 --2- 172.21.15.100:0/1033875766 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb548f5df0 0x55bb54907a60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1e18009c70 tx=0x7f1e18009910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfed104c89e19371 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:25.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.482+0000 7f1e02ffd700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:38:25.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.482+0000 7f1e02ffd700 1 -- 172.21.15.100:0/1033875766 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e180052e0 con 0x55bb548f5df0 2024-05-10T11:38:25.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.482+0000 7f1e3425db80 1 -- 172.21.15.100:0/1033875766 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e1c0acc50 con 0x55bb548f5df0 2024-05-10T11:38:25.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.482+0000 7f1e02ffd700 1 -- 172.21.15.100:0/1033875766 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f1e18003680 con 0x55bb548f5df0 2024-05-10T11:38:25.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.482+0000 7f1e02ffd700 1 -- 172.21.15.100:0/1033875766 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f1e1803b050 con 0x55bb548f5df0 2024-05-10T11:38:25.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.482+0000 7f1e3425db80 10 client.28634 fetch_fsmap learned FSMap version 41 2024-05-10T11:38:25.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.482+0000 7f1e3425db80 1 -- 172.21.15.100:0/1033875766 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55bb549792f0 con 0x55bb548f5df0 2024-05-10T11:38:25.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.482+0000 7f1e02ffd700 1 -- 172.21.15.100:0/1033875766 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1e1c0acc50 con 0x55bb548f5df0 2024-05-10T11:38:25.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.482+0000 7f1e02ffd700 1 -- 172.21.15.100:0/1033875766 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f1e1802b550 con 0x55bb548f5df0 2024-05-10T11:38:25.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.483+0000 7f1e3425db80 10 client.28634 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:38:25.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.483+0000 7f1e3425db80 10 client.28634 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-10T11:38:25.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.483+0000 7f1e3425db80 20 client.28634 populate_metadata read hostname 'smithi100' 2024-05-10T11:38:25.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.483+0000 7f1e3425db80 1 --2- 172.21.15.100:0/1033875766 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55bb5490ed10 0x55bb5497b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:25.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.483+0000 7f1e3425db80 4 client.28634 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:38:25.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.483+0000 7f1e3425db80 1 -- 172.21.15.100:0/1033875766 --> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bb54855240 con 0x55bb5490ed10 2024-05-10T11:38:25.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.483+0000 7f1e22ffd700 1 --2- 172.21.15.100:0/1033875766 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55bb5490ed10 0x55bb5497b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:25.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.483+0000 7f1e02ffd700 10 client.28634 ms_handle_connect on v2:172.21.15.188:6836/1943342620 2024-05-10T11:38:25.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.483+0000 7f1e22ffd700 1 --2- 172.21.15.100:0/1033875766 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55bb5490ed10 0x55bb5497b880 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=976451cf98df3b50 server_cookie=ded6bb9fcc189151 in_seq=0 out_seq=0 2024-05-10T11:38:25.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.485+0000 7f1e02ffd700 1 -- 172.21.15.100:0/1033875766 <== mds.2 v2:172.21.15.188:6836/1943342620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bb54855240 con 0x55bb5490ed10 2024-05-10T11:38:25.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.485+0000 7f1e02ffd700 10 client.28634 handle_command_reply: tid=0 2024-05-10T11:38:25.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.485+0000 7f1e3425db80 1 client.28634 shutdown 2024-05-10T11:38:25.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.485+0000 7f1e3425db80 1 -- 172.21.15.100:0/1033875766 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb548f5df0 msgr2=0x55bb54907a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:25.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.485+0000 7f1e3425db80 1 --2- 172.21.15.100:0/1033875766 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb548f5df0 0x55bb54907a60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1e18009c70 tx=0x7f1e18009910 comp rx=0 tx=0).stop 2024-05-10T11:38:25.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.485+0000 7f1e3425db80 1 -- 172.21.15.100:0/1033875766 shutdown_connections 2024-05-10T11:38:25.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.485+0000 7f1e3425db80 1 --2- 172.21.15.100:0/1033875766 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55bb5490ed10 0x55bb5497b880 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:25.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.485+0000 7f1e3425db80 1 --2- 172.21.15.100:0/1033875766 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb548f9c80 0x55bb54915de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:25.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.485+0000 7f1e3425db80 1 --2- 172.21.15.100:0/1033875766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb548f54d0 0x55bb54907520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:25.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.485+0000 7f1e3425db80 1 --2- 172.21.15.100:0/1033875766 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb548f5df0 0x55bb54907a60 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:25.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.485+0000 7f1e3425db80 1 -- 172.21.15.100:0/1033875766 >> 172.21.15.100:0/1033875766 conn(0x55bb54821100 msgr2=0x55bb548e9e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:25.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.485+0000 7f1e02ffd700 0 client.28634 ms_handle_reset on v2:172.21.15.188:6836/1943342620 2024-05-10T11:38:25.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.485+0000 7f1e3425db80 1 -- 172.21.15.100:0/1033875766 shutdown_connections 2024-05-10T11:38:25.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.485+0000 7f1e3425db80 1 -- 172.21.15.100:0/1033875766 wait complete. 2024-05-10T11:38:25.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.485+0000 7f1e3425db80 20 client.28634 trim_cache size 0 max 16384 2024-05-10T11:38:25.490 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:38:25.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.486+0000 7f1e299ca700 1 -- 172.21.15.100:0/1034446740 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f1e0c0b2900 msgr2=0x7f1e0c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:25.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.486+0000 7f1e299ca700 1 --2- 172.21.15.100:0/1034446740 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f1e0c0b2900 0x7f1e0c0b4dc0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f1e1400b2e0 tx=0x7f1e1401a040 comp rx=0 tx=0).stop 2024-05-10T11:38:25.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.486+0000 7f1e299ca700 1 -- 172.21.15.100:0/1034446740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e24114670 msgr2=0x7f1e241a4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:25.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.486+0000 7f1e299ca700 1 --2- 172.21.15.100:0/1034446740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e24114670 0x7f1e241a4960 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c007ec0 tx=0x7f1e1c00d3b0 comp rx=0 tx=0).stop 2024-05-10T11:38:25.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.486+0000 7f1e299ca700 1 -- 172.21.15.100:0/1034446740 shutdown_connections 2024-05-10T11:38:25.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.486+0000 7f1e299ca700 1 --2- 172.21.15.100:0/1034446740 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f1e0c0b2900 0x7f1e0c0b4dc0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:25.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.486+0000 7f1e299ca700 1 --2- 172.21.15.100:0/1034446740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e24114670 0x7f1e241a4960 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:25.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.486+0000 7f1e299ca700 1 --2- 172.21.15.100:0/1034446740 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e240756d0 0x7f1e241a0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:25.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.486+0000 7f1e299ca700 1 --2- 172.21.15.100:0/1034446740 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e24075ff0 0x7f1e241a05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:25.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.486+0000 7f1e299ca700 1 -- 172.21.15.100:0/1034446740 >> 172.21.15.100:0/1034446740 conn(0x7f1e2407a600 msgr2=0x7f1e24111570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:25.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.486+0000 7f1e299ca700 1 -- 172.21.15.100:0/1034446740 shutdown_connections 2024-05-10T11:38:25.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:25.486+0000 7f1e299ca700 1 -- 172.21.15.100:0/1034446740 wait complete. 2024-05-10T11:38:25.944 INFO:tasks.workunit.client.0.smithi100.stdout: 10 63470 9.00 MB/sec execute 259 sec latency 95.756 ms 2024-05-10T11:38:26.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:25 smithi178 ceph-mon[33359]: pgmap v1022: 129 pgs: 129 active+clean; 506 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 4.4 MiB/s wr, 277 op/s 2024-05-10T11:38:26.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:25 smithi188 ceph-mon[34075]: pgmap v1022: 129 pgs: 129 active+clean; 506 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 4.4 MiB/s wr, 277 op/s 2024-05-10T11:38:26.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:25 smithi100 ceph-mon[30296]: pgmap v1022: 129 pgs: 129 active+clean; 506 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 4.4 MiB/s wr, 277 op/s 2024-05-10T11:38:26.500 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-10T11:38:26.828 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:38:26.944 INFO:tasks.workunit.client.0.smithi100.stdout: 10 63609 8.98 MB/sec execute 260 sec latency 171.099 ms 2024-05-10T11:38:27.944 INFO:tasks.workunit.client.0.smithi100.stdout: 10 63784 8.99 MB/sec execute 261 sec latency 65.120 ms 2024-05-10T11:38:27.986 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:27 smithi100 ceph-mon[30296]: pgmap v1023: 129 pgs: 129 active+clean; 503 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 817 KiB/s rd, 4.6 MiB/s wr, 268 op/s 2024-05-10T11:38:28.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:27 smithi178 ceph-mon[33359]: pgmap v1023: 129 pgs: 129 active+clean; 503 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 817 KiB/s rd, 4.6 MiB/s wr, 268 op/s 2024-05-10T11:38:28.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:27 smithi188 ceph-mon[34075]: pgmap v1023: 129 pgs: 129 active+clean; 503 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 817 KiB/s rd, 4.6 MiB/s wr, 268 op/s 2024-05-10T11:38:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.470+0000 7f657bebd640 1 -- 172.21.15.100:0/3998483047 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6574102840 msgr2=0x7f6574102c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.470+0000 7f657bebd640 1 --2- 172.21.15.100:0/3998483047 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6574102840 0x7f6574102c60 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f6570009900 tx=0x7f657002a410 comp rx=0 tx=0).stop 2024-05-10T11:38:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.470+0000 7f657bebd640 1 -- 172.21.15.100:0/3998483047 shutdown_connections 2024-05-10T11:38:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.470+0000 7f657bebd640 1 --2- 172.21.15.100:0/3998483047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65741043f0 0x7f657410ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.470+0000 7f657bebd640 1 --2- 172.21.15.100:0/3998483047 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6574103a30 0x7f6574103eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.470+0000 7f657bebd640 1 --2- 172.21.15.100:0/3998483047 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6574102840 0x7f6574102c60 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.470+0000 7f657bebd640 1 -- 172.21.15.100:0/3998483047 >> 172.21.15.100:0/3998483047 conn(0x7f65740fddb0 msgr2=0x7f65741001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.470+0000 7f657bebd640 1 -- 172.21.15.100:0/3998483047 shutdown_connections 2024-05-10T11:38:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.470+0000 7f657bebd640 1 -- 172.21.15.100:0/3998483047 wait complete. 2024-05-10T11:38:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.471+0000 7f657bebd640 1 Processor -- start 2024-05-10T11:38:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.471+0000 7f657bebd640 1 -- start start 2024-05-10T11:38:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.471+0000 7f657bebd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6574103a30 0x7f657419f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.471+0000 7f657bebd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65741043f0 0x7f657419f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.471+0000 7f657bebd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65741a4790 0x7f65741a6b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.471+0000 7f657bebd640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f657410e220 con 0x7f65741043f0 2024-05-10T11:38:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.471+0000 7f657bebd640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f657410e0a0 con 0x7f6574103a30 2024-05-10T11:38:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.471+0000 7f657bebd640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f657410e3a0 con 0x7f65741a4790 2024-05-10T11:38:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.471+0000 7f6579458640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65741043f0 0x7f657419f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.471+0000 7f6579458640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65741043f0 0x7f657419f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47294/0 (socket says 172.21.15.100:47294) 2024-05-10T11:38:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.471+0000 7f6579458640 1 -- 172.21.15.100:0/3716383522 learned_addr learned my addr 172.21.15.100:0/3716383522 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:38:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.471+0000 7f6579c59640 1 --2- 172.21.15.100:0/3716383522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6574103a30 0x7f657419f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:28.474 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.471+0000 7f657a45a640 1 --2- 172.21.15.100:0/3716383522 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65741a4790 0x7f65741a6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:28.474 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.471+0000 7f6579458640 1 -- 172.21.15.100:0/3716383522 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65741a4790 msgr2=0x7f65741a6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:28.474 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.471+0000 7f6579458640 1 --2- 172.21.15.100:0/3716383522 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65741a4790 0x7f65741a6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:28.474 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.471+0000 7f6579458640 1 -- 172.21.15.100:0/3716383522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6574103a30 msgr2=0x7f657419f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:28.474 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.471+0000 7f6579458640 1 --2- 172.21.15.100:0/3716383522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6574103a30 0x7f657419f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:28.474 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.471+0000 7f6579458640 1 -- 172.21.15.100:0/3716383522 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65741a71f0 con 0x7f65741043f0 2024-05-10T11:38:28.474 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.472+0000 7f6579458640 1 --2- 172.21.15.100:0/3716383522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65741043f0 0x7f657419f730 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f656400e9f0 tx=0x7f656400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:28.475 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.472+0000 7f656affd640 1 -- 172.21.15.100:0/3716383522 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f656400cd80 con 0x7f65741043f0 2024-05-10T11:38:28.475 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.472+0000 7f657bebd640 1 -- 172.21.15.100:0/3716383522 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65741a74e0 con 0x7f65741043f0 2024-05-10T11:38:28.475 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.472+0000 7f656affd640 1 -- 172.21.15.100:0/3716383522 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f65640108c0 con 0x7f65741043f0 2024-05-10T11:38:28.475 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.472+0000 7f657bebd640 1 -- 172.21.15.100:0/3716383522 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65741a7a80 con 0x7f65741043f0 2024-05-10T11:38:28.475 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.472+0000 7f656affd640 1 -- 172.21.15.100:0/3716383522 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6564026060 con 0x7f65741043f0 2024-05-10T11:38:28.476 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.476+0000 7f657ebd5740 1 Processor -- start 2024-05-10T11:38:28.477 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.476+0000 7f657ebd5740 1 -- start start 2024-05-10T11:38:28.477 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.476+0000 7f656affd640 1 -- 172.21.15.100:0/3716383522 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f6564010430 con 0x7f65741043f0 2024-05-10T11:38:28.477 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.477+0000 7f657ebd5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615d0d5df90 0x5615d0d5e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:28.477 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.477+0000 7f657ebd5740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5615d0d5e980 0x5615d0d63480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:28.477 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.477+0000 7f657ebd5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615d0d63af0 0x5615d0d65f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:28.477 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.477+0000 7f657ebd5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5615d0cb15d0 con 0x5615d0d5df90 2024-05-10T11:38:28.477 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.477+0000 7f657ebd5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5615d0c6a890 con 0x5615d0d63af0 2024-05-10T11:38:28.477 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.477+0000 7f657ebd5740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5615d0cad590 con 0x5615d0d5e980 2024-05-10T11:38:28.478 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.477+0000 7f6579458640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615d0d63af0 0x5615d0d65f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:28.478 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.477+0000 7f657a45a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615d0d5df90 0x5615d0d5e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:28.478 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.477+0000 7f657a45a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615d0d5df90 0x5615d0d5e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47298/0 (socket says 172.21.15.100:47298) 2024-05-10T11:38:28.478 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.477+0000 7f657a45a640 1 -- 172.21.15.100:0/1146071490 learned_addr learned my addr 172.21.15.100:0/1146071490 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:38:28.478 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.477+0000 7f6579c59640 1 --2- 172.21.15.100:0/1146071490 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5615d0d5e980 0x5615d0d63480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.477+0000 7f657a45a640 1 -- 172.21.15.100:0/1146071490 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5615d0d5e980 msgr2=0x5615d0d63480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.477+0000 7f657a45a640 1 --2- 172.21.15.100:0/1146071490 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5615d0d5e980 0x5615d0d63480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.477+0000 7f657a45a640 1 -- 172.21.15.100:0/1146071490 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615d0d63af0 msgr2=0x5615d0d65f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.477+0000 7f657a45a640 1 --2- 172.21.15.100:0/1146071490 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615d0d63af0 0x5615d0d65f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.477+0000 7f657a45a640 1 -- 172.21.15.100:0/1146071490 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615d09c3210 con 0x5615d0d5df90 2024-05-10T11:38:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.478+0000 7f6579c59640 1 --2- 172.21.15.100:0/1146071490 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5615d0d5e980 0x5615d0d63480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:38:28.483 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.478+0000 7f656affd640 1 --2- 172.21.15.100:0/3716383522 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f65600b27e0 0x7f65600b4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:28.483 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.478+0000 7f657a45a640 1 --2- 172.21.15.100:0/1146071490 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615d0d5df90 0x5615d0d5e3b0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f656c00f7a0 tx=0x7f656c00a580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27eea19527aaa054 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:28.483 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.478+0000 7f656affd640 1 -- 172.21.15.100:0/3716383522 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f65640e28e0 con 0x7f65741043f0 2024-05-10T11:38:28.483 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.479+0000 7f655ffff640 1 -- 172.21.15.100:0/1146071490 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f656c00d970 con 0x5615d0d5df90 2024-05-10T11:38:28.483 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.479+0000 7f655ffff640 1 -- 172.21.15.100:0/1146071490 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f656c019430 con 0x5615d0d5df90 2024-05-10T11:38:28.483 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.479+0000 7f655ffff640 1 -- 172.21.15.100:0/1146071490 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f656c018640 con 0x5615d0d5df90 2024-05-10T11:38:28.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.479+0000 7f6579c59640 1 --2- 172.21.15.100:0/3716383522 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f65600b27e0 0x7f65600b4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:28.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.480+0000 7f657ebd5740 1 -- 172.21.15.100:0/1146071490 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615d0d5df90 msgr2=0x5615d0d5e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:28.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.480+0000 7f657ebd5740 1 --2- 172.21.15.100:0/1146071490 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615d0d5df90 0x5615d0d5e3b0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f656c00f7a0 tx=0x7f656c00a580 comp rx=0 tx=0).stop 2024-05-10T11:38:28.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.480+0000 7f657ebd5740 1 -- 172.21.15.100:0/1146071490 shutdown_connections 2024-05-10T11:38:28.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.480+0000 7f657ebd5740 1 --2- 172.21.15.100:0/1146071490 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615d0d63af0 0x5615d0d65f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:28.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.480+0000 7f657ebd5740 1 --2- 172.21.15.100:0/1146071490 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5615d0d5e980 0x5615d0d63480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:28.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.480+0000 7f657ebd5740 1 --2- 172.21.15.100:0/1146071490 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615d0d5df90 0x5615d0d5e3b0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:28.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.480+0000 7f657ebd5740 1 -- 172.21.15.100:0/1146071490 >> 172.21.15.100:0/1146071490 conn(0x5615d0d485f0 msgr2=0x5615d0d489f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:28.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.480+0000 7f6579c59640 1 --2- 172.21.15.100:0/3716383522 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f65600b27e0 0x7f65600b4ca0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f6570006010 tx=0x7f6570031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:28.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.480+0000 7f657ebd5740 1 -- 172.21.15.100:0/1146071490 shutdown_connections 2024-05-10T11:38:28.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.480+0000 7f657ebd5740 1 -- 172.21.15.100:0/1146071490 wait complete. 2024-05-10T11:38:28.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.481+0000 7f657ebd5740 1 -- start start 2024-05-10T11:38:28.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.481+0000 7f657ebd5740 1 Processor -- start 2024-05-10T11:38:28.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.481+0000 7f657ebd5740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5615d0d5df90 0x5615d0d682c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:28.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.481+0000 7f657ebd5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615d0d5e980 0x5615d0d68800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.481+0000 7f6579458640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615d0d5e980 0x5615d0d68800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.481+0000 7f6579458640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615d0d5e980 0x5615d0d68800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47310/0 (socket says 172.21.15.100:47310) 2024-05-10T11:38:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.481+0000 7f6579c59640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5615d0d5df90 0x5615d0d682c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.481+0000 7f6579458640 1 -- 172.21.15.100:0/3022680084 learned_addr learned my addr 172.21.15.100:0/3022680084 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:38:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.481+0000 7f657ebd5740 1 --2- 172.21.15.100:0/3022680084 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615d0d63af0 0x5615d0d6f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.481+0000 7f657ebd5740 1 -- 172.21.15.100:0/3022680084 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5615d0d66aa0 con 0x5615d0d5e980 2024-05-10T11:38:28.497 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.481+0000 7f657ebd5740 1 -- 172.21.15.100:0/3022680084 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5615d0c6e370 con 0x5615d0d63af0 2024-05-10T11:38:28.497 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.481+0000 7f657ebd5740 1 -- 172.21.15.100:0/3022680084 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5615d0d66c20 con 0x5615d0d5df90 2024-05-10T11:38:28.497 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.482+0000 7f657ebd5740 10 client.0 _ll_register_callbacks cb 0x5615d09c2a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:38:28.497 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.482+0000 7f6579c59640 1 -- 172.21.15.100:0/3022680084 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615d0d63af0 msgr2=0x5615d0d6f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:28.497 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.482+0000 7f657a45a640 1 --2- 172.21.15.100:0/3022680084 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615d0d63af0 0x5615d0d6f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:28.497 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.482+0000 7f6579c59640 1 --2- 172.21.15.100:0/3022680084 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615d0d63af0 0x5615d0d6f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:28.497 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.482+0000 7f6579c59640 1 -- 172.21.15.100:0/3022680084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615d0d5e980 msgr2=0x5615d0d68800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:28.497 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.482+0000 7f6579c59640 1 --2- 172.21.15.100:0/3022680084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615d0d5e980 0x5615d0d68800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:28.497 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.482+0000 7f6579c59640 1 -- 172.21.15.100:0/3022680084 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615d0a8f620 con 0x5615d0d5df90 2024-05-10T11:38:28.497 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.482+0000 7f6579458640 1 --2- 172.21.15.100:0/3022680084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615d0d5e980 0x5615d0d68800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:38:28.497 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.483+0000 7f657a45a640 1 --2- 172.21.15.100:0/3022680084 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615d0d63af0 0x5615d0d6f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:38:28.498 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.483+0000 7f6579c59640 1 --2- 172.21.15.100:0/3022680084 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5615d0d5df90 0x5615d0d682c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6574102380 tx=0x7f6570009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4380d645f0e4cc6d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:28.498 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.483+0000 7f655d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:38:28.498 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.483+0000 7f655d7fa640 1 -- 172.21.15.100:0/3022680084 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f657004c030 con 0x5615d0d5df90 2024-05-10T11:38:28.498 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.483+0000 7f657ebd5740 1 -- 172.21.15.100:0/3022680084 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f656c019430 con 0x5615d0d5df90 2024-05-10T11:38:28.498 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.483+0000 7f655d7fa640 1 -- 172.21.15.100:0/3022680084 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f6570004040 con 0x5615d0d5df90 2024-05-10T11:38:28.498 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.484+0000 7f655d7fa640 1 -- 172.21.15.100:0/3022680084 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f6570059050 con 0x5615d0d5df90 2024-05-10T11:38:28.498 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.484+0000 7f657ebd5740 10 client.28747 fetch_fsmap learned FSMap version 41 2024-05-10T11:38:28.498 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.484+0000 7f657ebd5740 1 -- 172.21.15.100:0/3022680084 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5615d0d72850 con 0x5615d0d5df90 2024-05-10T11:38:28.498 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.484+0000 7f655d7fa640 1 -- 172.21.15.100:0/3022680084 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6570004240 con 0x5615d0d5df90 2024-05-10T11:38:28.498 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.485+0000 7f655d7fa640 1 -- 172.21.15.100:0/3022680084 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f6570048440 con 0x5615d0d5df90 2024-05-10T11:38:28.498 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.485+0000 7f657ebd5740 10 client.28747 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:38:28.498 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.485+0000 7f657ebd5740 10 client.28747 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:38:28.498 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.485+0000 7f657ebd5740 20 client.28747 populate_metadata read hostname 'smithi100' 2024-05-10T11:38:28.499 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.485+0000 7f657ebd5740 1 --2- 172.21.15.100:0/3022680084 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5615d0d743a0 0x5615d0d767d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:28.499 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.485+0000 7f657ebd5740 4 client.28747 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:38:28.499 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.485+0000 7f657ebd5740 1 -- 172.21.15.100:0/3022680084 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f656c019430 con 0x5615d0d743a0 2024-05-10T11:38:28.499 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.485+0000 7f6579458640 1 --2- 172.21.15.100:0/3022680084 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5615d0d743a0 0x5615d0d767d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:28.499 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.485+0000 7f655d7fa640 10 client.28747 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:38:28.499 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.485+0000 7f6579458640 1 --2- 172.21.15.100:0/3022680084 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5615d0d743a0 0x5615d0d767d0 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2db090878effcfd5 server_cookie=b05dc29cf5b35a7b in_seq=0 out_seq=0 2024-05-10T11:38:28.499 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.486+0000 7f655d7fa640 1 -- 172.21.15.100:0/3022680084 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f656c019430 con 0x5615d0d743a0 2024-05-10T11:38:28.499 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.486+0000 7f655d7fa640 10 client.28747 handle_command_reply: tid=0 2024-05-10T11:38:28.499 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.486+0000 7f657ebd5740 1 client.28747 shutdown 2024-05-10T11:38:28.499 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.487+0000 7f657ebd5740 1 -- 172.21.15.100:0/3022680084 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5615d0d5df90 msgr2=0x5615d0d682c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:28.499 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.487+0000 7f657ebd5740 1 --2- 172.21.15.100:0/3022680084 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5615d0d5df90 0x5615d0d682c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6574102380 tx=0x7f6570009260 comp rx=0 tx=0).stop 2024-05-10T11:38:28.499 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.487+0000 7f657ebd5740 1 -- 172.21.15.100:0/3022680084 shutdown_connections 2024-05-10T11:38:28.499 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.487+0000 7f657ebd5740 1 --2- 172.21.15.100:0/3022680084 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5615d0d743a0 0x5615d0d767d0 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:28.499 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.487+0000 7f657ebd5740 1 --2- 172.21.15.100:0/3022680084 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615d0d63af0 0x5615d0d6f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:28.499 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.487+0000 7f657ebd5740 1 --2- 172.21.15.100:0/3022680084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615d0d5e980 0x5615d0d68800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:28.500 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.487+0000 7f657ebd5740 1 --2- 172.21.15.100:0/3022680084 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5615d0d5df90 0x5615d0d682c0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:28.500 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.487+0000 7f655d7fa640 0 client.28747 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:38:28.500 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.487+0000 7f657ebd5740 1 -- 172.21.15.100:0/3022680084 >> 172.21.15.100:0/3022680084 conn(0x5615d0d485f0 msgr2=0x5615d0d47750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:28.500 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.487+0000 7f657ebd5740 1 -- 172.21.15.100:0/3022680084 shutdown_connections 2024-05-10T11:38:28.500 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.487+0000 7f657ebd5740 1 -- 172.21.15.100:0/3022680084 wait complete. 2024-05-10T11:38:28.500 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.487+0000 7f657ebd5740 20 client.28747 trim_cache size 0 max 16384 2024-05-10T11:38:28.511 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.511+0000 7f657ebd5740 1 Processor -- start 2024-05-10T11:38:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.511+0000 7f657ebd5740 1 -- start start 2024-05-10T11:38:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.511+0000 7f657ebd5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615d0d5df90 0x5615d0d5fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.511+0000 7f657ebd5740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5615d0d5e980 0x5615d0d601b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.511+0000 7f657ebd5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615d0d63af0 0x5615d0d67ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.511+0000 7f657ebd5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f65741abed0 con 0x5615d0d63af0 2024-05-10T11:38:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.511+0000 7f657ebd5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f65741a97a0 con 0x5615d0d5df90 2024-05-10T11:38:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.511+0000 7f657ebd5740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f657419b960 con 0x5615d0d5e980 2024-05-10T11:38:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.511+0000 7f657a45a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615d0d63af0 0x5615d0d67ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.512+0000 7f657a45a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615d0d63af0 0x5615d0d67ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47316/0 (socket says 172.21.15.100:47316) 2024-05-10T11:38:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.512+0000 7f657a45a640 1 -- 172.21.15.100:0/719754684 learned_addr learned my addr 172.21.15.100:0/719754684 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:38:28.513 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.512+0000 7f6579c59640 1 --2- 172.21.15.100:0/719754684 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615d0d5df90 0x5615d0d5fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.512+0000 7f6579458640 1 --2- 172.21.15.100:0/719754684 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5615d0d5e980 0x5615d0d601b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.512+0000 7f657a45a640 1 -- 172.21.15.100:0/719754684 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5615d0d5e980 msgr2=0x5615d0d601b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.512+0000 7f657a45a640 1 --2- 172.21.15.100:0/719754684 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5615d0d5e980 0x5615d0d601b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.512+0000 7f657a45a640 1 -- 172.21.15.100:0/719754684 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615d0d5df90 msgr2=0x5615d0d5fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.512+0000 7f657a45a640 1 --2- 172.21.15.100:0/719754684 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615d0d5df90 0x5615d0d5fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.512+0000 7f657a45a640 1 -- 172.21.15.100:0/719754684 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615d0d43540 con 0x5615d0d63af0 2024-05-10T11:38:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.512+0000 7f6579c59640 1 --2- 172.21.15.100:0/719754684 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615d0d5df90 0x5615d0d5fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:38:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.512+0000 7f657a45a640 1 --2- 172.21.15.100:0/719754684 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615d0d63af0 0x5615d0d67ed0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f656c000dc0 tx=0x7f656c000b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0b7f76918caf166 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:28.516 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.512+0000 7f655f7fe640 1 -- 172.21.15.100:0/719754684 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f656c0012e0 con 0x5615d0d63af0 2024-05-10T11:38:28.516 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.512+0000 7f655f7fe640 1 -- 172.21.15.100:0/719754684 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f656c00de00 con 0x5615d0d63af0 2024-05-10T11:38:28.516 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.512+0000 7f655f7fe640 1 -- 172.21.15.100:0/719754684 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f656c019c40 con 0x5615d0d63af0 2024-05-10T11:38:28.516 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.513+0000 7f657ebd5740 1 -- 172.21.15.100:0/719754684 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615d0d63af0 msgr2=0x5615d0d67ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:28.516 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.513+0000 7f657ebd5740 1 --2- 172.21.15.100:0/719754684 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615d0d63af0 0x5615d0d67ed0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f656c000dc0 tx=0x7f656c000b60 comp rx=0 tx=0).stop 2024-05-10T11:38:28.516 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.513+0000 7f657ebd5740 1 -- 172.21.15.100:0/719754684 shutdown_connections 2024-05-10T11:38:28.516 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.513+0000 7f657ebd5740 1 --2- 172.21.15.100:0/719754684 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615d0d63af0 0x5615d0d67ed0 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:28.516 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.513+0000 7f657ebd5740 1 --2- 172.21.15.100:0/719754684 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5615d0d5e980 0x5615d0d601b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.513+0000 7f657ebd5740 1 --2- 172.21.15.100:0/719754684 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615d0d5df90 0x5615d0d5fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.513+0000 7f657ebd5740 1 -- 172.21.15.100:0/719754684 >> 172.21.15.100:0/719754684 conn(0x5615d0d485f0 msgr2=0x5615d0d489d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.513+0000 7f657ebd5740 1 -- 172.21.15.100:0/719754684 shutdown_connections 2024-05-10T11:38:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.513+0000 7f657ebd5740 1 -- 172.21.15.100:0/719754684 wait complete. 2024-05-10T11:38:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.514+0000 7f657ebd5740 1 -- start start 2024-05-10T11:38:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.514+0000 7f657ebd5740 1 Processor -- start 2024-05-10T11:38:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.514+0000 7f657ebd5740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5615d0d5df90 0x5615d0d6c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.514+0000 7f657ebd5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615d0d5e980 0x5615d0d6c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.514+0000 7f657ebd5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615d0d63af0 0x5615d0d6d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.514+0000 7f657ebd5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f65741a7da0 con 0x5615d0d63af0 2024-05-10T11:38:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.514+0000 7f657ebd5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f65741a90d0 con 0x5615d0d5e980 2024-05-10T11:38:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.514+0000 7f657ebd5740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f65741a8330 con 0x5615d0d5df90 2024-05-10T11:38:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.514+0000 7f6579458640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615d0d5e980 0x5615d0d6c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.514+0000 7f6579458640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615d0d5e980 0x5615d0d6c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:36764/0 (socket says 172.21.15.100:36764) 2024-05-10T11:38:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.514+0000 7f6579458640 1 -- 172.21.15.100:0/797945641 learned_addr learned my addr 172.21.15.100:0/797945641 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:38:28.518 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.514+0000 7f657ebd5740 10 client.0 _ll_register_callbacks cb 0x7f656c00a420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:38:28.518 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.514+0000 7f657a45a640 1 --2- 172.21.15.100:0/797945641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615d0d63af0 0x5615d0d6d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:28.518 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.514+0000 7f6579c59640 1 --2- 172.21.15.100:0/797945641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5615d0d5df90 0x5615d0d6c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:28.518 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.514+0000 7f6579458640 1 -- 172.21.15.100:0/797945641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5615d0d5df90 msgr2=0x5615d0d6c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:28.518 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.514+0000 7f6579458640 1 --2- 172.21.15.100:0/797945641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5615d0d5df90 0x5615d0d6c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:28.518 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.514+0000 7f6579458640 1 -- 172.21.15.100:0/797945641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615d0d63af0 msgr2=0x5615d0d6d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:28.518 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.514+0000 7f6579458640 1 --2- 172.21.15.100:0/797945641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615d0d63af0 0x5615d0d6d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:28.535 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.514+0000 7f6579458640 1 -- 172.21.15.100:0/797945641 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615d0d6df30 con 0x5615d0d5e980 2024-05-10T11:38:28.535 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.515+0000 7f6579458640 1 --2- 172.21.15.100:0/797945641 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615d0d5e980 0x5615d0d6c9f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f65640a6ec0 tx=0x7f65640cc040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b59d2643ea150cb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:28.535 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.515+0000 7f655d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:38:28.535 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.515+0000 7f655d7fa640 1 -- 172.21.15.100:0/797945641 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65640a95a0 con 0x5615d0d5e980 2024-05-10T11:38:28.535 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.515+0000 7f657ebd5740 1 -- 172.21.15.100:0/797945641 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f656c00de00 con 0x5615d0d5e980 2024-05-10T11:38:28.535 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.515+0000 7f655d7fa640 1 -- 172.21.15.100:0/797945641 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f65640c4230 con 0x5615d0d5e980 2024-05-10T11:38:28.535 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.515+0000 7f655d7fa640 1 -- 172.21.15.100:0/797945641 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f65640b2340 con 0x5615d0d5e980 2024-05-10T11:38:28.535 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.515+0000 7f657ebd5740 10 client.28649 fetch_fsmap learned FSMap version 41 2024-05-10T11:38:28.535 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.515+0000 7f657ebd5740 1 -- 172.21.15.100:0/797945641 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5615d0ddec90 con 0x5615d0d5e980 2024-05-10T11:38:28.535 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.515+0000 7f655d7fa640 1 -- 172.21.15.100:0/797945641 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f656c00de00 con 0x5615d0d5e980 2024-05-10T11:38:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.516+0000 7f655d7fa640 1 -- 172.21.15.100:0/797945641 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f65640a55c0 con 0x5615d0d5e980 2024-05-10T11:38:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.516+0000 7f657ebd5740 10 client.28649 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:38:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.516+0000 7f657ebd5740 10 client.28649 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:38:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.516+0000 7f657ebd5740 20 client.28649 populate_metadata read hostname 'smithi100' 2024-05-10T11:38:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.516+0000 7f657ebd5740 1 --2- 172.21.15.100:0/797945641 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5615d0ddf480 0x5615d0de18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.516+0000 7f657ebd5740 4 client.28649 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:38:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.516+0000 7f657ebd5740 1 -- 172.21.15.100:0/797945641 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5615d0cbf1e0 con 0x5615d0ddf480 2024-05-10T11:38:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.516+0000 7f6579c59640 1 --2- 172.21.15.100:0/797945641 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5615d0ddf480 0x5615d0de18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.516+0000 7f6579c59640 1 --2- 172.21.15.100:0/797945641 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5615d0ddf480 0x5615d0de18a0 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aac884acf9dd9d61 server_cookie=84988ca3e412a550 in_seq=0 out_seq=0 2024-05-10T11:38:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.516+0000 7f655d7fa640 10 client.28649 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:38:28.621 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.620+0000 7f655d7fa640 1 -- 172.21.15.100:0/797945641 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5615d0cbf1e0 con 0x5615d0ddf480 2024-05-10T11:38:28.621 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.620+0000 7f655d7fa640 10 client.28649 handle_command_reply: tid=0 2024-05-10T11:38:28.621 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.620+0000 7f657ebd5740 1 client.28649 shutdown 2024-05-10T11:38:28.621 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.620+0000 7f657ebd5740 1 -- 172.21.15.100:0/797945641 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615d0d5e980 msgr2=0x5615d0d6c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:28.621 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.620+0000 7f657ebd5740 1 --2- 172.21.15.100:0/797945641 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615d0d5e980 0x5615d0d6c9f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f65640a6ec0 tx=0x7f65640cc040 comp rx=0 tx=0).stop 2024-05-10T11:38:28.621 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.620+0000 7f657ebd5740 1 -- 172.21.15.100:0/797945641 shutdown_connections 2024-05-10T11:38:28.621 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.620+0000 7f657ebd5740 1 --2- 172.21.15.100:0/797945641 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5615d0ddf480 0x5615d0de18a0 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:28.621 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.620+0000 7f657ebd5740 1 --2- 172.21.15.100:0/797945641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615d0d63af0 0x5615d0d6d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:28.621 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.620+0000 7f657ebd5740 1 --2- 172.21.15.100:0/797945641 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615d0d5e980 0x5615d0d6c9f0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:28.621 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.620+0000 7f657ebd5740 1 --2- 172.21.15.100:0/797945641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5615d0d5df90 0x5615d0d6c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:28.622 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.620+0000 7f657ebd5740 1 -- 172.21.15.100:0/797945641 >> 172.21.15.100:0/797945641 conn(0x5615d0d485f0 msgr2=0x5615d0d5fc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:28.622 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.621+0000 7f655d7fa640 0 client.28649 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:38:28.622 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.621+0000 7f657ebd5740 1 -- 172.21.15.100:0/797945641 shutdown_connections 2024-05-10T11:38:28.622 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.621+0000 7f657ebd5740 1 -- 172.21.15.100:0/797945641 wait complete. 2024-05-10T11:38:28.622 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.621+0000 7f657ebd5740 20 client.28649 trim_cache size 0 max 16384 2024-05-10T11:38:28.622 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:38:28.622 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:38:28.622 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:38:28.623 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:38:28.623 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.622+0000 7f657bebd640 1 -- 172.21.15.100:0/3716383522 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f65600b27e0 msgr2=0x7f65600b4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:28.623 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.622+0000 7f657bebd640 1 --2- 172.21.15.100:0/3716383522 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f65600b27e0 0x7f65600b4ca0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f6570006010 tx=0x7f6570031400 comp rx=0 tx=0).stop 2024-05-10T11:38:28.623 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.622+0000 7f657bebd640 1 -- 172.21.15.100:0/3716383522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65741043f0 msgr2=0x7f657419f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:28.623 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.622+0000 7f657bebd640 1 --2- 172.21.15.100:0/3716383522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65741043f0 0x7f657419f730 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f656400e9f0 tx=0x7f656400eeb0 comp rx=0 tx=0).stop 2024-05-10T11:38:28.624 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.622+0000 7f657bebd640 1 -- 172.21.15.100:0/3716383522 shutdown_connections 2024-05-10T11:38:28.624 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.622+0000 7f657bebd640 1 --2- 172.21.15.100:0/3716383522 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f65600b27e0 0x7f65600b4ca0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:28.624 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.622+0000 7f657bebd640 1 --2- 172.21.15.100:0/3716383522 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65741a4790 0x7f65741a6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:28.624 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.622+0000 7f657bebd640 1 --2- 172.21.15.100:0/3716383522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65741043f0 0x7f657419f730 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:28.624 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.622+0000 7f657bebd640 1 --2- 172.21.15.100:0/3716383522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6574103a30 0x7f657419f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:28.624 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.622+0000 7f657bebd640 1 -- 172.21.15.100:0/3716383522 >> 172.21.15.100:0/3716383522 conn(0x7f65740fddb0 msgr2=0x7f6574104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:28.624 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.622+0000 7f657bebd640 1 -- 172.21.15.100:0/3716383522 shutdown_connections 2024-05-10T11:38:28.624 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:28.623+0000 7f657bebd640 1 -- 172.21.15.100:0/3716383522 wait complete. 2024-05-10T11:38:28.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.761+0000 7fd11e923700 1 -- 172.21.15.100:0/1212111611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd11806b180 msgr2=0x7fd11806b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:28.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.761+0000 7fd11e923700 1 --2- 172.21.15.100:0/1212111611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd11806b180 0x7fd11806b600 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fd108009990 tx=0x7fd108009ca0 comp rx=0 tx=0).stop 2024-05-10T11:38:28.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.761+0000 7fd11e923700 1 -- 172.21.15.100:0/1212111611 shutdown_connections 2024-05-10T11:38:28.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.761+0000 7fd11e923700 1 --2- 172.21.15.100:0/1212111611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd11806bb40 0x7fd118110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:28.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.761+0000 7fd11e923700 1 --2- 172.21.15.100:0/1212111611 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd118109510 0x7fd1181098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:28.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.761+0000 7fd11e923700 1 --2- 172.21.15.100:0/1212111611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd11806b180 0x7fd11806b600 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:28.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.761+0000 7fd11e923700 1 -- 172.21.15.100:0/1212111611 >> 172.21.15.100:0/1212111611 conn(0x7fd118079350 msgr2=0x7fd118079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:28.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.761+0000 7fd11e923700 1 -- 172.21.15.100:0/1212111611 shutdown_connections 2024-05-10T11:38:28.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.761+0000 7fd11e923700 1 -- 172.21.15.100:0/1212111611 wait complete. 2024-05-10T11:38:28.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.761+0000 7fd11e923700 1 Processor -- start 2024-05-10T11:38:28.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.762+0000 7fd11e923700 1 -- start start 2024-05-10T11:38:28.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.762+0000 7fd11e923700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd11806b180 0x7fd118102470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:28.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.762+0000 7fd11e923700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd11806bb40 0x7fd1181029b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:28.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.762+0000 7fd11e923700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd118109510 0x7fd118102ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:28.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.762+0000 7fd11e923700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd1181134d0 con 0x7fd11806b180 2024-05-10T11:38:28.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.762+0000 7fd11e923700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd118113350 con 0x7fd118109510 2024-05-10T11:38:28.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.762+0000 7fd11e923700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd118113650 con 0x7fd11806bb40 2024-05-10T11:38:28.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.762+0000 7fd117fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd11806b180 0x7fd118102470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:28.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.762+0000 7fd117fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd11806b180 0x7fd118102470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47340/0 (socket says 172.21.15.100:47340) 2024-05-10T11:38:28.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.762+0000 7fd117fff700 1 -- 172.21.15.100:0/1691189056 learned_addr learned my addr 172.21.15.100:0/1691189056 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:38:28.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.762+0000 7fd1177fe700 1 --2- 172.21.15.100:0/1691189056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd11806bb40 0x7fd1181029b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:28.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.762+0000 7fd11cec0700 1 --2- 172.21.15.100:0/1691189056 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd118109510 0x7fd118102ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:28.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.762+0000 7fd117fff700 1 -- 172.21.15.100:0/1691189056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd11806bb40 msgr2=0x7fd1181029b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:28.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.762+0000 7fd117fff700 1 --2- 172.21.15.100:0/1691189056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd11806bb40 0x7fd1181029b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:28.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.762+0000 7fd117fff700 1 -- 172.21.15.100:0/1691189056 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd118109510 msgr2=0x7fd118102ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:28.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.762+0000 7fd117fff700 1 --2- 172.21.15.100:0/1691189056 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd118109510 0x7fd118102ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:28.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.762+0000 7fd117fff700 1 -- 172.21.15.100:0/1691189056 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1181a7fb0 con 0x7fd11806b180 2024-05-10T11:38:28.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.763+0000 7fd117fff700 1 --2- 172.21.15.100:0/1691189056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd11806b180 0x7fd118102470 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fd10c00e9a0 tx=0x7fd10c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:28.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.763+0000 7fd1157fa700 1 -- 172.21.15.100:0/1691189056 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd10c00ca10 con 0x7fd11806b180 2024-05-10T11:38:28.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.763+0000 7fd1157fa700 1 -- 172.21.15.100:0/1691189056 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd10c00cbb0 con 0x7fd11806b180 2024-05-10T11:38:28.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.763+0000 7fd11e923700 1 -- 172.21.15.100:0/1691189056 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1181a82a0 con 0x7fd11806b180 2024-05-10T11:38:28.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.763+0000 7fd11e923700 1 -- 172.21.15.100:0/1691189056 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1181a8800 con 0x7fd11806b180 2024-05-10T11:38:28.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.763+0000 7fd1157fa700 1 -- 172.21.15.100:0/1691189056 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd10c018680 con 0x7fd11806b180 2024-05-10T11:38:28.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.768+0000 7fd1157fa700 1 -- 172.21.15.100:0/1691189056 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fd10c010430 con 0x7fd11806b180 2024-05-10T11:38:28.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.768+0000 7fd1291b6b80 1 Processor -- start 2024-05-10T11:38:28.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.768+0000 7fd1291b6b80 1 -- start start 2024-05-10T11:38:28.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.768+0000 7fd1291b6b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56301f3761b0 0x56301f376590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:28.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.768+0000 7fd1291b6b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56301f376ad0 0x56301f370810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:28.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.768+0000 7fd1291b6b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56301f370d50 0x56301f371190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:28.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.768+0000 7fd1291b6b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56301f2c2940 con 0x56301f370d50 2024-05-10T11:38:28.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.768+0000 7fd1291b6b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56301f227da0 con 0x56301f376ad0 2024-05-10T11:38:28.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.768+0000 7fd1291b6b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56301f2be900 con 0x56301f3761b0 2024-05-10T11:38:28.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.769+0000 7fd11cec0700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56301f3761b0 0x56301f376590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:28.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.769+0000 7fd11cec0700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56301f3761b0 0x56301f376590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:36168/0 (socket says 172.21.15.100:36168) 2024-05-10T11:38:28.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.769+0000 7fd117fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56301f376ad0 0x56301f370810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:28.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.769+0000 7fd11cec0700 1 -- 172.21.15.100:0/3212571465 learned_addr learned my addr 172.21.15.100:0/3212571465 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:38:28.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.769+0000 7fd1177fe700 1 --2- 172.21.15.100:0/3212571465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56301f370d50 0x56301f371190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:28.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.769+0000 7fd1157fa700 1 --2- 172.21.15.100:0/1691189056 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fd1000b2900 0x7fd1000b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:28.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.769+0000 7fd11cec0700 1 -- 172.21.15.100:0/3212571465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56301f376ad0 msgr2=0x56301f370810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:28.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.769+0000 7fd11cec0700 1 --2- 172.21.15.100:0/3212571465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56301f376ad0 0x56301f370810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:28.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.769+0000 7fd11cec0700 1 -- 172.21.15.100:0/3212571465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56301f370d50 msgr2=0x56301f371190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:28.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.769+0000 7fd11cec0700 1 --2- 172.21.15.100:0/3212571465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56301f370d50 0x56301f371190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:28.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.769+0000 7fd11cec0700 1 -- 172.21.15.100:0/3212571465 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56301f20f8d0 con 0x56301f3761b0 2024-05-10T11:38:28.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.769+0000 7fd1177fe700 1 --2- 172.21.15.100:0/3212571465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56301f370d50 0x56301f371190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:38:28.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.769+0000 7fd117fff700 1 --2- 172.21.15.100:0/3212571465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56301f376ad0 0x56301f370810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:38:28.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.769+0000 7fd1157fa700 1 -- 172.21.15.100:0/1691189056 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fd10c0e2510 con 0x7fd11806b180 2024-05-10T11:38:28.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.769+0000 7fd11cec0700 1 --2- 172.21.15.100:0/3212571465 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56301f3761b0 0x56301f376590 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd110011680 tx=0x7fd110009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=587bc910ebad3346 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:28.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.770+0000 7fd11cec0700 1 --2- 172.21.15.100:0/1691189056 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fd1000b2900 0x7fd1000b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:28.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.770+0000 7fd0fe7fc700 1 -- 172.21.15.100:0/3212571465 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd110019040 con 0x56301f3761b0 2024-05-10T11:38:28.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.770+0000 7fd0fe7fc700 1 -- 172.21.15.100:0/3212571465 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd11001a020 con 0x56301f3761b0 2024-05-10T11:38:28.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.770+0000 7fd0fe7fc700 1 -- 172.21.15.100:0/3212571465 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd110007b70 con 0x56301f3761b0 2024-05-10T11:38:28.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.770+0000 7fd1291b6b80 1 -- 172.21.15.100:0/3212571465 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56301f3761b0 msgr2=0x56301f376590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:28.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.770+0000 7fd1291b6b80 1 --2- 172.21.15.100:0/3212571465 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56301f3761b0 0x56301f376590 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd110011680 tx=0x7fd110009920 comp rx=0 tx=0).stop 2024-05-10T11:38:28.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.771+0000 7fd1291b6b80 1 -- 172.21.15.100:0/3212571465 shutdown_connections 2024-05-10T11:38:28.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.771+0000 7fd1291b6b80 1 --2- 172.21.15.100:0/3212571465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56301f370d50 0x56301f371190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:28.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.771+0000 7fd1291b6b80 1 --2- 172.21.15.100:0/3212571465 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56301f3761b0 0x56301f376590 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:28.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.771+0000 7fd1291b6b80 1 --2- 172.21.15.100:0/3212571465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56301f376ad0 0x56301f370810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:28.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.771+0000 7fd1291b6b80 1 -- 172.21.15.100:0/3212571465 >> 172.21.15.100:0/3212571465 conn(0x56301f29cb70 msgr2=0x56301f36b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:28.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.771+0000 7fd11cec0700 1 --2- 172.21.15.100:0/1691189056 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fd1000b2900 0x7fd1000b4dc0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fd110011300 tx=0x7fd110022740 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:28.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.771+0000 7fd1291b6b80 1 -- 172.21.15.100:0/3212571465 shutdown_connections 2024-05-10T11:38:28.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.771+0000 7fd1291b6b80 1 -- 172.21.15.100:0/3212571465 wait complete. 2024-05-10T11:38:28.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.771+0000 7fd1291b6b80 1 -- start start 2024-05-10T11:38:28.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.771+0000 7fd1291b6b80 1 Processor -- start 2024-05-10T11:38:28.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.771+0000 7fd1291b6b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56301f370d50 0x56301f37c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:28.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.771+0000 7fd1291b6b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56301f3761b0 0x56301f379900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:28.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.771+0000 7fd1291b6b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56301f376ad0 0x56301f379e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:28.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.771+0000 7fd1291b6b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56301f37d450 con 0x56301f376ad0 2024-05-10T11:38:28.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.771+0000 7fd1291b6b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56301f27bed0 con 0x56301f370d50 2024-05-10T11:38:28.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.771+0000 7fd1291b6b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56301f37d5d0 con 0x56301f3761b0 2024-05-10T11:38:28.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.771+0000 7fd1291b6b80 10 client.0 _ll_register_callbacks cb 0x56301f2565b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:38:28.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.772+0000 7fd117fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56301f370d50 0x56301f37c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:28.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.772+0000 7fd117fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56301f370d50 0x56301f37c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:36812/0 (socket says 172.21.15.100:36812) 2024-05-10T11:38:28.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.772+0000 7fd117fff700 1 -- 172.21.15.100:0/1152716975 learned_addr learned my addr 172.21.15.100:0/1152716975 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:38:28.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.772+0000 7fd11cec0700 1 --2- 172.21.15.100:0/1152716975 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56301f376ad0 0x56301f379e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:28.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.772+0000 7fd1177fe700 1 --2- 172.21.15.100:0/1152716975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56301f3761b0 0x56301f379900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:28.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.772+0000 7fd117fff700 1 -- 172.21.15.100:0/1152716975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56301f3761b0 msgr2=0x56301f379900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:28.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.772+0000 7fd117fff700 1 --2- 172.21.15.100:0/1152716975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56301f3761b0 0x56301f379900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:28.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.772+0000 7fd117fff700 1 -- 172.21.15.100:0/1152716975 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56301f376ad0 msgr2=0x56301f379e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:28.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.772+0000 7fd117fff700 1 --2- 172.21.15.100:0/1152716975 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56301f376ad0 0x56301f379e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:28.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.772+0000 7fd117fff700 1 -- 172.21.15.100:0/1152716975 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56301f279760 con 0x56301f370d50 2024-05-10T11:38:28.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.772+0000 7fd117fff700 1 --2- 172.21.15.100:0/1152716975 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56301f370d50 0x56301f37c560 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd10c010c60 tx=0x7fd10c0a9d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cbdff0a286e8815 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:28.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.772+0000 7fd0f77fe700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:38:28.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.772+0000 7fd0f77fe700 1 -- 172.21.15.100:0/1152716975 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd10c0e1020 con 0x56301f370d50 2024-05-10T11:38:28.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.772+0000 7fd0f77fe700 1 -- 172.21.15.100:0/1152716975 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fd10c091320 con 0x56301f370d50 2024-05-10T11:38:28.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.772+0000 7fd1291b6b80 1 -- 172.21.15.100:0/1152716975 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd11001a020 con 0x56301f370d50 2024-05-10T11:38:28.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.773+0000 7fd0f77fe700 1 -- 172.21.15.100:0/1152716975 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fd10c0a95a0 con 0x56301f370d50 2024-05-10T11:38:28.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.773+0000 7fd1291b6b80 10 client.28661 fetch_fsmap learned FSMap version 41 2024-05-10T11:38:28.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.773+0000 7fd1291b6b80 1 -- 172.21.15.100:0/1152716975 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x56301f3903a0 con 0x56301f370d50 2024-05-10T11:38:28.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.773+0000 7fd0f77fe700 1 -- 172.21.15.100:0/1152716975 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd11001a020 con 0x56301f370d50 2024-05-10T11:38:28.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.773+0000 7fd0f77fe700 1 -- 172.21.15.100:0/1152716975 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fd10c010720 con 0x56301f370d50 2024-05-10T11:38:28.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.773+0000 7fd1291b6b80 10 client.28661 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:38:28.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.773+0000 7fd1291b6b80 10 client.28661 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-10T11:38:28.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.773+0000 7fd1291b6b80 20 client.28661 populate_metadata read hostname 'smithi100' 2024-05-10T11:38:28.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.773+0000 7fd1291b6b80 1 --2- 172.21.15.100:0/1152716975 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x56301f3929d0 0x56301f394e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:28.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.773+0000 7fd1291b6b80 4 client.28661 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:38:28.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.773+0000 7fd1291b6b80 1 -- 172.21.15.100:0/1152716975 --> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56301f2d0320 con 0x56301f3929d0 2024-05-10T11:38:28.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.773+0000 7fd1177fe700 1 --2- 172.21.15.100:0/1152716975 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x56301f3929d0 0x56301f394e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:28.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.774+0000 7fd0f77fe700 10 client.28661 ms_handle_connect on v2:172.21.15.178:6838/3760985404 2024-05-10T11:38:28.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.774+0000 7fd1177fe700 1 --2- 172.21.15.100:0/1152716975 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x56301f3929d0 0x56301f394e20 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=56f707a60c5387c9 server_cookie=15f75992ec5675ad in_seq=0 out_seq=0 2024-05-10T11:38:28.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.775+0000 7fd0f77fe700 1 -- 172.21.15.100:0/1152716975 <== mds.1 v2:172.21.15.178:6838/3760985404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56301f2d0320 con 0x56301f3929d0 2024-05-10T11:38:28.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.775+0000 7fd0f77fe700 10 client.28661 handle_command_reply: tid=0 2024-05-10T11:38:28.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.775+0000 7fd1291b6b80 1 client.28661 shutdown 2024-05-10T11:38:28.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.775+0000 7fd1291b6b80 1 -- 172.21.15.100:0/1152716975 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56301f370d50 msgr2=0x56301f37c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:28.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.775+0000 7fd1291b6b80 1 --2- 172.21.15.100:0/1152716975 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56301f370d50 0x56301f37c560 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd10c010c60 tx=0x7fd10c0a9d10 comp rx=0 tx=0).stop 2024-05-10T11:38:28.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.775+0000 7fd1291b6b80 1 -- 172.21.15.100:0/1152716975 shutdown_connections 2024-05-10T11:38:28.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.775+0000 7fd1291b6b80 1 --2- 172.21.15.100:0/1152716975 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x56301f3929d0 0x56301f394e20 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:28.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.775+0000 7fd1291b6b80 1 --2- 172.21.15.100:0/1152716975 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56301f376ad0 0x56301f379e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:28.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.776+0000 7fd0f77fe700 0 client.28661 ms_handle_reset on v2:172.21.15.178:6838/3760985404 2024-05-10T11:38:28.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.776+0000 7fd1291b6b80 1 --2- 172.21.15.100:0/1152716975 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56301f370d50 0x56301f37c560 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:28.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.776+0000 7fd1291b6b80 1 --2- 172.21.15.100:0/1152716975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56301f3761b0 0x56301f379900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:28.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.776+0000 7fd1291b6b80 1 -- 172.21.15.100:0/1152716975 >> 172.21.15.100:0/1152716975 conn(0x56301f29cb70 msgr2=0x56301f365b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:28.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.776+0000 7fd1291b6b80 1 -- 172.21.15.100:0/1152716975 shutdown_connections 2024-05-10T11:38:28.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.776+0000 7fd1291b6b80 1 -- 172.21.15.100:0/1152716975 wait complete. 2024-05-10T11:38:28.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.776+0000 7fd1291b6b80 20 client.28661 trim_cache size 0 max 16384 2024-05-10T11:38:28.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.802+0000 7fd1291b6b80 1 Processor -- start 2024-05-10T11:38:28.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.802+0000 7fd1291b6b80 1 -- start start 2024-05-10T11:38:28.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.802+0000 7fd1291b6b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56301f370d50 0x56301f371130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:28.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.802+0000 7fd1291b6b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56301f3761b0 0x56301f365c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:28.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.802+0000 7fd1291b6b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56301f376ad0 0x56301f3791b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:28.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.802+0000 7fd1291b6b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd1040112b0 con 0x56301f3761b0 2024-05-10T11:38:28.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.802+0000 7fd117fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56301f370d50 0x56301f371130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:28.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.802+0000 7fd1291b6b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd10400eb40 con 0x56301f370d50 2024-05-10T11:38:28.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.802+0000 7fd1291b6b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd10400c990 con 0x56301f376ad0 2024-05-10T11:38:28.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.802+0000 7fd117fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56301f370d50 0x56301f371130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:36828/0 (socket says 172.21.15.100:36828) 2024-05-10T11:38:28.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.802+0000 7fd117fff700 1 -- 172.21.15.100:0/1157394352 learned_addr learned my addr 172.21.15.100:0/1157394352 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:38:28.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.802+0000 7fd11cec0700 1 --2- 172.21.15.100:0/1157394352 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56301f376ad0 0x56301f3791b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:28.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.803+0000 7fd1177fe700 1 --2- 172.21.15.100:0/1157394352 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56301f3761b0 0x56301f365c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:28.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.803+0000 7fd11cec0700 1 -- 172.21.15.100:0/1157394352 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56301f370d50 msgr2=0x56301f371130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:28.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.803+0000 7fd11cec0700 1 --2- 172.21.15.100:0/1157394352 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56301f370d50 0x56301f371130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:28.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.803+0000 7fd11cec0700 1 -- 172.21.15.100:0/1157394352 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56301f3761b0 msgr2=0x56301f365c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:28.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.803+0000 7fd11cec0700 1 --2- 172.21.15.100:0/1157394352 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56301f3761b0 0x56301f365c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:28.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.803+0000 7fd11cec0700 1 -- 172.21.15.100:0/1157394352 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56301f3954a0 con 0x56301f376ad0 2024-05-10T11:38:28.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.803+0000 7fd1177fe700 1 --2- 172.21.15.100:0/1157394352 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56301f3761b0 0x56301f365c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:38:28.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.803+0000 7fd117fff700 1 --2- 172.21.15.100:0/1157394352 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56301f370d50 0x56301f371130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:38:28.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.803+0000 7fd11cec0700 1 --2- 172.21.15.100:0/1157394352 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56301f376ad0 0x56301f3791b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd110009960 tx=0x7fd110025000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d7463b892aeeb88 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:28.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.803+0000 7fd0fdffb700 1 -- 172.21.15.100:0/1157394352 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd110031070 con 0x56301f376ad0 2024-05-10T11:38:28.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.803+0000 7fd0fdffb700 1 -- 172.21.15.100:0/1157394352 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd110009290 con 0x56301f376ad0 2024-05-10T11:38:28.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.804+0000 7fd0fdffb700 1 -- 172.21.15.100:0/1157394352 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd110007b60 con 0x56301f376ad0 2024-05-10T11:38:28.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.804+0000 7fd1291b6b80 1 -- 172.21.15.100:0/1157394352 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56301f376ad0 msgr2=0x56301f3791b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:28.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.804+0000 7fd1291b6b80 1 --2- 172.21.15.100:0/1157394352 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56301f376ad0 0x56301f3791b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd110009960 tx=0x7fd110025000 comp rx=0 tx=0).stop 2024-05-10T11:38:28.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.805+0000 7fd1291b6b80 1 -- 172.21.15.100:0/1157394352 shutdown_connections 2024-05-10T11:38:28.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.805+0000 7fd1291b6b80 1 --2- 172.21.15.100:0/1157394352 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56301f376ad0 0x56301f3791b0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:28.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.805+0000 7fd1291b6b80 1 --2- 172.21.15.100:0/1157394352 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56301f370d50 0x56301f371130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:28.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.805+0000 7fd1291b6b80 1 --2- 172.21.15.100:0/1157394352 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56301f3761b0 0x56301f365c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:28.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.805+0000 7fd1291b6b80 1 -- 172.21.15.100:0/1157394352 >> 172.21.15.100:0/1157394352 conn(0x56301f29cb70 msgr2=0x56301f3781f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:28.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.805+0000 7fd1291b6b80 1 -- 172.21.15.100:0/1157394352 shutdown_connections 2024-05-10T11:38:28.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.805+0000 7fd1291b6b80 1 -- 172.21.15.100:0/1157394352 wait complete. 2024-05-10T11:38:28.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.805+0000 7fd1291b6b80 1 -- start start 2024-05-10T11:38:28.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.805+0000 7fd1291b6b80 1 Processor -- start 2024-05-10T11:38:28.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.806+0000 7fd1291b6b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56301f370d50 0x56301f38b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:28.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.806+0000 7fd1291b6b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56301f3761b0 0x56301f38b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:28.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.806+0000 7fd1291b6b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56301f376ad0 0x56301f399760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:28.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.806+0000 7fd1291b6b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd10400d000 con 0x56301f3761b0 2024-05-10T11:38:28.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.806+0000 7fd1291b6b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd10400e470 con 0x56301f370d50 2024-05-10T11:38:28.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.806+0000 7fd1291b6b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd10400d6d0 con 0x56301f376ad0 2024-05-10T11:38:28.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.806+0000 7fd1291b6b80 10 client.0 _ll_register_callbacks cb 0x56301f2565b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:38:28.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.806+0000 7fd1177fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56301f3761b0 0x56301f38b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:28.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.806+0000 7fd117fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56301f370d50 0x56301f38b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:28.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.806+0000 7fd1177fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56301f3761b0 0x56301f38b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47378/0 (socket says 172.21.15.100:47378) 2024-05-10T11:38:28.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.806+0000 7fd1177fe700 1 -- 172.21.15.100:0/1489534348 learned_addr learned my addr 172.21.15.100:0/1489534348 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:38:28.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.806+0000 7fd11cec0700 1 --2- 172.21.15.100:0/1489534348 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56301f376ad0 0x56301f399760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:28.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.806+0000 7fd1177fe700 1 -- 172.21.15.100:0/1489534348 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56301f376ad0 msgr2=0x56301f399760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:28.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.806+0000 7fd1177fe700 1 --2- 172.21.15.100:0/1489534348 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56301f376ad0 0x56301f399760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:28.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.806+0000 7fd1177fe700 1 -- 172.21.15.100:0/1489534348 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56301f370d50 msgr2=0x56301f38b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:28.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.806+0000 7fd1177fe700 1 --2- 172.21.15.100:0/1489534348 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56301f370d50 0x56301f38b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:28.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.806+0000 7fd1177fe700 1 -- 172.21.15.100:0/1489534348 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56301f399f60 con 0x56301f3761b0 2024-05-10T11:38:28.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.807+0000 7fd1177fe700 1 --2- 172.21.15.100:0/1489534348 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56301f3761b0 0x56301f38b810 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fd108009990 tx=0x7fd108009910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e7dbd2448230d0a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:28.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.807+0000 7fd0f77fe700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:38:28.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.807+0000 7fd0f77fe700 1 -- 172.21.15.100:0/1489534348 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd108003a90 con 0x56301f3761b0 2024-05-10T11:38:28.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.807+0000 7fd0f77fe700 1 -- 172.21.15.100:0/1489534348 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fd108003c30 con 0x56301f3761b0 2024-05-10T11:38:28.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.807+0000 7fd1291b6b80 1 -- 172.21.15.100:0/1489534348 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd110009290 con 0x56301f3761b0 2024-05-10T11:38:28.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.807+0000 7fd0f77fe700 1 -- 172.21.15.100:0/1489534348 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fd108036050 con 0x56301f3761b0 2024-05-10T11:38:28.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.807+0000 7fd1291b6b80 10 client.19386 fetch_fsmap learned FSMap version 41 2024-05-10T11:38:28.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.807+0000 7fd1291b6b80 1 -- 172.21.15.100:0/1489534348 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x56301f40ddf0 con 0x56301f3761b0 2024-05-10T11:38:28.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.808+0000 7fd0f77fe700 1 -- 172.21.15.100:0/1489534348 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd110009290 con 0x56301f3761b0 2024-05-10T11:38:28.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.808+0000 7fd0f77fe700 1 -- 172.21.15.100:0/1489534348 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fd1080280e0 con 0x56301f3761b0 2024-05-10T11:38:28.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.809+0000 7fd1291b6b80 10 client.19386 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:38:28.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.809+0000 7fd1291b6b80 10 client.19386 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-10T11:38:28.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.809+0000 7fd1291b6b80 20 client.19386 populate_metadata read hostname 'smithi100' 2024-05-10T11:38:28.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.809+0000 7fd1291b6b80 1 --2- 172.21.15.100:0/1489534348 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x56301f40e530 0x56301f410950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:28.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.809+0000 7fd1291b6b80 4 client.19386 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:38:28.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.809+0000 7fd1291b6b80 1 -- 172.21.15.100:0/1489534348 --> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56301f2ca3c0 con 0x56301f40e530 2024-05-10T11:38:28.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.809+0000 7fd117fff700 1 --2- 172.21.15.100:0/1489534348 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x56301f40e530 0x56301f410950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:28.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.809+0000 7fd0f77fe700 10 client.19386 ms_handle_connect on v2:172.21.15.178:6838/3760985404 2024-05-10T11:38:28.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.809+0000 7fd117fff700 1 --2- 172.21.15.100:0/1489534348 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x56301f40e530 0x56301f410950 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3f626d723075feea server_cookie=77e3907e9aba4349 in_seq=0 out_seq=0 2024-05-10T11:38:28.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.810+0000 7fd0f77fe700 1 -- 172.21.15.100:0/1489534348 <== mds.1 v2:172.21.15.178:6838/3760985404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56301f2ca3c0 con 0x56301f40e530 2024-05-10T11:38:28.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.810+0000 7fd0f77fe700 10 client.19386 handle_command_reply: tid=0 2024-05-10T11:38:28.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.810+0000 7fd1291b6b80 1 client.19386 shutdown 2024-05-10T11:38:28.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.810+0000 7fd1291b6b80 1 -- 172.21.15.100:0/1489534348 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56301f3761b0 msgr2=0x56301f38b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:28.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.810+0000 7fd1291b6b80 1 --2- 172.21.15.100:0/1489534348 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56301f3761b0 0x56301f38b810 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fd108009990 tx=0x7fd108009910 comp rx=0 tx=0).stop 2024-05-10T11:38:28.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.810+0000 7fd1291b6b80 1 -- 172.21.15.100:0/1489534348 shutdown_connections 2024-05-10T11:38:28.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.810+0000 7fd1291b6b80 1 --2- 172.21.15.100:0/1489534348 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x56301f40e530 0x56301f410950 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:28.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.810+0000 7fd1291b6b80 1 --2- 172.21.15.100:0/1489534348 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56301f376ad0 0x56301f399760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:28.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.810+0000 7fd1291b6b80 1 --2- 172.21.15.100:0/1489534348 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56301f370d50 0x56301f38b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:28.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.810+0000 7fd1291b6b80 1 --2- 172.21.15.100:0/1489534348 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56301f3761b0 0x56301f38b810 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:28.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.810+0000 7fd1291b6b80 1 -- 172.21.15.100:0/1489534348 >> 172.21.15.100:0/1489534348 conn(0x56301f29cb70 msgr2=0x56301f36f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:28.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.810+0000 7fd0f77fe700 0 client.19386 ms_handle_reset on v2:172.21.15.178:6838/3760985404 2024-05-10T11:38:28.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.810+0000 7fd1291b6b80 1 -- 172.21.15.100:0/1489534348 shutdown_connections 2024-05-10T11:38:28.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.810+0000 7fd1291b6b80 1 -- 172.21.15.100:0/1489534348 wait complete. 2024-05-10T11:38:28.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.810+0000 7fd1291b6b80 20 client.19386 trim_cache size 0 max 16384 2024-05-10T11:38:28.814 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:38:28.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.811+0000 7fd11e923700 1 -- 172.21.15.100:0/1691189056 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fd1000b2900 msgr2=0x7fd1000b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:28.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.811+0000 7fd11e923700 1 --2- 172.21.15.100:0/1691189056 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fd1000b2900 0x7fd1000b4dc0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fd110011300 tx=0x7fd110022740 comp rx=0 tx=0).stop 2024-05-10T11:38:28.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.811+0000 7fd11e923700 1 -- 172.21.15.100:0/1691189056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd11806b180 msgr2=0x7fd118102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:28.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.811+0000 7fd11e923700 1 --2- 172.21.15.100:0/1691189056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd11806b180 0x7fd118102470 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fd10c00e9a0 tx=0x7fd10c00ecb0 comp rx=0 tx=0).stop 2024-05-10T11:38:28.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.811+0000 7fd11e923700 1 -- 172.21.15.100:0/1691189056 shutdown_connections 2024-05-10T11:38:28.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.811+0000 7fd11e923700 1 --2- 172.21.15.100:0/1691189056 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fd1000b2900 0x7fd1000b4dc0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:28.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.811+0000 7fd11e923700 1 --2- 172.21.15.100:0/1691189056 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd118109510 0x7fd118102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:28.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.811+0000 7fd11e923700 1 --2- 172.21.15.100:0/1691189056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd11806b180 0x7fd118102470 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:28.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.811+0000 7fd11e923700 1 --2- 172.21.15.100:0/1691189056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd11806bb40 0x7fd1181029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:28.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.811+0000 7fd11e923700 1 -- 172.21.15.100:0/1691189056 >> 172.21.15.100:0/1691189056 conn(0x7fd118079350 msgr2=0x7fd1181012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:28.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.811+0000 7fd11e923700 1 -- 172.21.15.100:0/1691189056 shutdown_connections 2024-05-10T11:38:28.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:28.811+0000 7fd11e923700 1 -- 172.21.15.100:0/1691189056 wait complete. 2024-05-10T11:38:28.944 INFO:tasks.workunit.client.0.smithi100.stdout: 10 63924 8.97 MB/sec execute 262 sec latency 148.014 ms 2024-05-10T11:38:29.546 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-10T11:38:29.789 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:29 smithi100 ceph-mon[30296]: pgmap v1024: 129 pgs: 129 active+clean; 505 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 933 KiB/s rd, 4.2 MiB/s wr, 268 op/s 2024-05-10T11:38:29.943 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:38:29.944 INFO:tasks.workunit.client.0.smithi100.stdout: 10 64087 8.98 MB/sec execute 263 sec latency 78.714 ms 2024-05-10T11:38:30.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:29 smithi178 ceph-mon[33359]: pgmap v1024: 129 pgs: 129 active+clean; 505 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 933 KiB/s rd, 4.2 MiB/s wr, 268 op/s 2024-05-10T11:38:30.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:29 smithi188 ceph-mon[34075]: pgmap v1024: 129 pgs: 129 active+clean; 505 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 933 KiB/s rd, 4.2 MiB/s wr, 268 op/s 2024-05-10T11:38:30.945 INFO:tasks.workunit.client.0.smithi100.stdout: 10 64279 9.00 MB/sec execute 264 sec latency 65.786 ms 2024-05-10T11:38:31.945 INFO:tasks.workunit.client.0.smithi100.stdout: 10 64502 9.00 MB/sec execute 265 sec latency 67.147 ms 2024-05-10T11:38:31.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:31 smithi100 ceph-mon[30296]: pgmap v1025: 129 pgs: 129 active+clean; 508 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 4.3 MiB/s wr, 273 op/s 2024-05-10T11:38:32.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:31 smithi178 ceph-mon[33359]: pgmap v1025: 129 pgs: 129 active+clean; 508 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 4.3 MiB/s wr, 273 op/s 2024-05-10T11:38:32.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:31 smithi188 ceph-mon[34075]: pgmap v1025: 129 pgs: 129 active+clean; 508 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 4.3 MiB/s wr, 273 op/s 2024-05-10T11:38:32.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.120+0000 7f6f8e591700 1 -- 172.21.15.100:0/78137157 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f800b7b00 msgr2=0x7f6f800b9ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:32.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.120+0000 7f6f8e591700 1 --2- 172.21.15.100:0/78137157 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f800b7b00 0x7f6f800b9ef0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f6f88069f30 tx=0x7f6f880522b0 comp rx=0 tx=0).stop 2024-05-10T11:38:32.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.121+0000 7f6f8e591700 1 -- 172.21.15.100:0/78137157 shutdown_connections 2024-05-10T11:38:32.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.121+0000 7f6f8e591700 1 --2- 172.21.15.100:0/78137157 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f800b7b00 0x7f6f800b9ef0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:32.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.121+0000 7f6f8e591700 1 --2- 172.21.15.100:0/78137157 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f800a4de0 0x7f6f800a51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:32.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.121+0000 7f6f8e591700 1 --2- 172.21.15.100:0/78137157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f800a5700 0x7f6f800b75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:32.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.121+0000 7f6f8e591700 1 -- 172.21.15.100:0/78137157 >> 172.21.15.100:0/78137157 conn(0x7f6f8001a6a0 msgr2=0x7f6f8001aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:32.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.122+0000 7f6f8e591700 1 -- 172.21.15.100:0/78137157 shutdown_connections 2024-05-10T11:38:32.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.122+0000 7f6f8e591700 1 -- 172.21.15.100:0/78137157 wait complete. 2024-05-10T11:38:32.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.123+0000 7f6f8e591700 1 Processor -- start 2024-05-10T11:38:32.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.123+0000 7f6f8e591700 1 -- start start 2024-05-10T11:38:32.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.124+0000 7f6f8e591700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f800a4de0 0x7f6f80149620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:32.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.124+0000 7f6f8e591700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f800a5700 0x7f6f80149b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:32.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.124+0000 7f6f8e591700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f800b7b00 0x7f6f801436f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:32.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.124+0000 7f6f8e591700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6f800bd310 con 0x7f6f800b7b00 2024-05-10T11:38:32.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.124+0000 7f6f8e591700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6f800bd190 con 0x7f6f800a4de0 2024-05-10T11:38:32.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.124+0000 7f6f8e591700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6f800bd490 con 0x7f6f800a5700 2024-05-10T11:38:32.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.124+0000 7f6f8d58f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f800a4de0 0x7f6f80149620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:32.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.124+0000 7f6f8d58f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f800a4de0 0x7f6f80149620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:36872/0 (socket says 172.21.15.100:36872) 2024-05-10T11:38:32.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.124+0000 7f6f8d58f700 1 -- 172.21.15.100:0/2484116668 learned_addr learned my addr 172.21.15.100:0/2484116668 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:38:32.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.124+0000 7f6f8cd8e700 1 --2- 172.21.15.100:0/2484116668 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f800a5700 0x7f6f80149b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:32.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.124+0000 7f6f8dd90700 1 --2- 172.21.15.100:0/2484116668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f800b7b00 0x7f6f801436f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:32.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.125+0000 7f6f8d58f700 1 -- 172.21.15.100:0/2484116668 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f800a5700 msgr2=0x7f6f80149b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:32.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.125+0000 7f6f8d58f700 1 --2- 172.21.15.100:0/2484116668 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f800a5700 0x7f6f80149b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:32.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.125+0000 7f6f8d58f700 1 -- 172.21.15.100:0/2484116668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f800b7b00 msgr2=0x7f6f801436f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:32.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.125+0000 7f6f8d58f700 1 --2- 172.21.15.100:0/2484116668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f800b7b00 0x7f6f801436f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:32.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.125+0000 7f6f8d58f700 1 -- 172.21.15.100:0/2484116668 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f80143fd0 con 0x7f6f800a4de0 2024-05-10T11:38:32.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.125+0000 7f6f8dd90700 1 --2- 172.21.15.100:0/2484116668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f800b7b00 0x7f6f801436f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:38:32.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.125+0000 7f6f8d58f700 1 --2- 172.21.15.100:0/2484116668 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f800a4de0 0x7f6f80149620 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6f8400e9c0 tx=0x7f6f8400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:32.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.125+0000 7f6f7e7fc700 1 -- 172.21.15.100:0/2484116668 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f8400cbd0 con 0x7f6f800a4de0 2024-05-10T11:38:32.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.126+0000 7f6f8e591700 1 -- 172.21.15.100:0/2484116668 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f801442c0 con 0x7f6f800a4de0 2024-05-10T11:38:32.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.126+0000 7f6f8e591700 1 -- 172.21.15.100:0/2484116668 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f801505a0 con 0x7f6f800a4de0 2024-05-10T11:38:32.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.126+0000 7f6f7e7fc700 1 -- 172.21.15.100:0/2484116668 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6f8400cd70 con 0x7f6f800a4de0 2024-05-10T11:38:32.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.126+0000 7f6f7e7fc700 1 -- 172.21.15.100:0/2484116668 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f840054e0 con 0x7f6f800a4de0 2024-05-10T11:38:32.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.129+0000 7f6f7e7fc700 1 -- 172.21.15.100:0/2484116668 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f6f84021440 con 0x7f6f800a4de0 2024-05-10T11:38:32.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.131+0000 7f6f7e7fc700 1 --2- 172.21.15.100:0/2484116668 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f6f740b29c0 0x7f6f740b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:32.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.131+0000 7f6f7e7fc700 1 -- 172.21.15.100:0/2484116668 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f6f840e2700 con 0x7f6f800a4de0 2024-05-10T11:38:32.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.131+0000 7f6f8cd8e700 1 --2- 172.21.15.100:0/2484116668 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f6f740b29c0 0x7f6f740b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:32.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.131+0000 7f6f9a086b80 1 Processor -- start 2024-05-10T11:38:32.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.131+0000 7f6f9a086b80 1 -- start start 2024-05-10T11:38:32.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.132+0000 7f6f9a086b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5c5abc290 0x55b5c5abc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:32.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.132+0000 7f6f8cd8e700 1 --2- 172.21.15.100:0/2484116668 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f6f740b29c0 0x7f6f740b4e80 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f6f80144eb0 tx=0x7f6f7800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:32.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.132+0000 7f6f9a086b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b5c5abcbb0 0x55b5c5ab6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:32.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.132+0000 7f6f9a086b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b5c5ab6d50 0x55b5c5ab7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:32.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.132+0000 7f6f9a086b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55b5c5a08940 con 0x55b5c5abc290 2024-05-10T11:38:32.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.132+0000 7f6f9a086b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b5c596dda0 con 0x55b5c5abcbb0 2024-05-10T11:38:32.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.132+0000 7f6f9a086b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b5c5a04900 con 0x55b5c5ab6d50 2024-05-10T11:38:32.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.132+0000 7f6f8d58f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5c5abc290 0x55b5c5abc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:32.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.132+0000 7f6f8d58f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5c5abc290 0x55b5c5abc670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47392/0 (socket says 172.21.15.100:47392) 2024-05-10T11:38:32.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.132+0000 7f6f8d58f700 1 -- 172.21.15.100:0/3475654540 learned_addr learned my addr 172.21.15.100:0/3475654540 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:38:32.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.132+0000 7f6f8cd8e700 1 --2- 172.21.15.100:0/3475654540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b5c5abcbb0 0x55b5c5ab6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:32.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.132+0000 7f6f8d58f700 1 -- 172.21.15.100:0/3475654540 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b5c5ab6d50 msgr2=0x55b5c5ab7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:32.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.132+0000 7f6f8d58f700 1 --2- 172.21.15.100:0/3475654540 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b5c5ab6d50 0x55b5c5ab7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:32.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.132+0000 7f6f8d58f700 1 -- 172.21.15.100:0/3475654540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b5c5abcbb0 msgr2=0x55b5c5ab6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:32.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.132+0000 7f6f8d58f700 1 --2- 172.21.15.100:0/3475654540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b5c5abcbb0 0x55b5c5ab6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:32.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.132+0000 7f6f8d58f700 1 -- 172.21.15.100:0/3475654540 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5c59558d0 con 0x55b5c5abc290 2024-05-10T11:38:32.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.133+0000 7f6f8d58f700 1 --2- 172.21.15.100:0/3475654540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5c5abc290 0x55b5c5abc670 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f6f840a98f0 tx=0x7f6f84009550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e51aebc5008c1ce server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:32.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.133+0000 7f6f737fe700 1 -- 172.21.15.100:0/3475654540 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f840b2110 con 0x55b5c5abc290 2024-05-10T11:38:32.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.133+0000 7f6f737fe700 1 -- 172.21.15.100:0/3475654540 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6f840aa060 con 0x55b5c5abc290 2024-05-10T11:38:32.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.133+0000 7f6f737fe700 1 -- 172.21.15.100:0/3475654540 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f840e2440 con 0x55b5c5abc290 2024-05-10T11:38:32.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.134+0000 7f6f9a086b80 1 -- 172.21.15.100:0/3475654540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5c5abc290 msgr2=0x55b5c5abc670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:32.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.134+0000 7f6f9a086b80 1 --2- 172.21.15.100:0/3475654540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5c5abc290 0x55b5c5abc670 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f6f840a98f0 tx=0x7f6f84009550 comp rx=0 tx=0).stop 2024-05-10T11:38:32.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.134+0000 7f6f9a086b80 1 -- 172.21.15.100:0/3475654540 shutdown_connections 2024-05-10T11:38:32.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.134+0000 7f6f9a086b80 1 --2- 172.21.15.100:0/3475654540 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b5c5ab6d50 0x55b5c5ab7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:32.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.134+0000 7f6f9a086b80 1 --2- 172.21.15.100:0/3475654540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5c5abc290 0x55b5c5abc670 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:32.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.134+0000 7f6f9a086b80 1 --2- 172.21.15.100:0/3475654540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b5c5abcbb0 0x55b5c5ab6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:32.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.134+0000 7f6f9a086b80 1 -- 172.21.15.100:0/3475654540 >> 172.21.15.100:0/3475654540 conn(0x55b5c59e2b70 msgr2=0x55b5c5ab12d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:32.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.134+0000 7f6f9a086b80 1 -- 172.21.15.100:0/3475654540 shutdown_connections 2024-05-10T11:38:32.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.134+0000 7f6f9a086b80 1 -- 172.21.15.100:0/3475654540 wait complete. 2024-05-10T11:38:32.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.135+0000 7f6f9a086b80 1 -- start start 2024-05-10T11:38:32.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.135+0000 7f6f9a086b80 1 Processor -- start 2024-05-10T11:38:32.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.136+0000 7f6f9a086b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b5c5ab6d50 0x55b5c5ac2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:32.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.136+0000 7f6f9a086b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b5c5abc290 0x55b5c5abfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:32.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.136+0000 7f6f9a086b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5c5abcbb0 0x55b5c5ac01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:32.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.136+0000 7f6f9a086b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55b5c5ac3580 con 0x55b5c5abcbb0 2024-05-10T11:38:32.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.136+0000 7f6f9a086b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b5c59c1ed0 con 0x55b5c5abc290 2024-05-10T11:38:32.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.136+0000 7f6f9a086b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b5c5ac3700 con 0x55b5c5ab6d50 2024-05-10T11:38:32.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.136+0000 7f6f8cd8e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b5c5abc290 0x55b5c5abfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:32.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.136+0000 7f6f8d58f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b5c5ab6d50 0x55b5c5ac2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:32.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.136+0000 7f6f8d58f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b5c5ab6d50 0x55b5c5ac2890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:36232/0 (socket says 172.21.15.100:36232) 2024-05-10T11:38:32.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.136+0000 7f6f8cd8e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b5c5abc290 0x55b5c5abfc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:36884/0 (socket says 172.21.15.100:36884) 2024-05-10T11:38:32.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.136+0000 7f6f8d58f700 1 -- 172.21.15.100:0/528480153 learned_addr learned my addr 172.21.15.100:0/528480153 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:38:32.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.136+0000 7f6f9a086b80 10 client.0 _ll_register_callbacks cb 0x55b5c599c5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:38:32.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.136+0000 7f6f8cd8e700 1 -- 172.21.15.100:0/528480153 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b5c5ab6d50 msgr2=0x55b5c5ac2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:32.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.136+0000 7f6f8cd8e700 1 --2- 172.21.15.100:0/528480153 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b5c5ab6d50 0x55b5c5ac2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:32.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.136+0000 7f6f8cd8e700 1 -- 172.21.15.100:0/528480153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5c5abcbb0 msgr2=0x55b5c5ac01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:32.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.136+0000 7f6f8cd8e700 1 --2- 172.21.15.100:0/528480153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5c5abcbb0 0x55b5c5ac01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:32.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.136+0000 7f6f8cd8e700 1 -- 172.21.15.100:0/528480153 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5c59bf760 con 0x55b5c5abc290 2024-05-10T11:38:32.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.136+0000 7f6f8cd8e700 1 --2- 172.21.15.100:0/528480153 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b5c5abc290 0x55b5c5abfc80 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6f80144f40 tx=0x7f6f780047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5af1c4ee01e0783 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:32.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.136+0000 7f6f70ff9700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:38:32.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.137+0000 7f6f70ff9700 1 -- 172.21.15.100:0/528480153 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f78004830 con 0x55b5c5abc290 2024-05-10T11:38:32.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.137+0000 7f6f70ff9700 1 -- 172.21.15.100:0/528480153 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f6f780049d0 con 0x55b5c5abc290 2024-05-10T11:38:32.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.137+0000 7f6f9a086b80 1 -- 172.21.15.100:0/528480153 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f840aa060 con 0x55b5c5abc290 2024-05-10T11:38:32.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.137+0000 7f6f70ff9700 1 -- 172.21.15.100:0/528480153 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f6f78030050 con 0x55b5c5abc290 2024-05-10T11:38:32.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.137+0000 7f6f9a086b80 10 client.28679 fetch_fsmap learned FSMap version 41 2024-05-10T11:38:32.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.137+0000 7f6f9a086b80 1 -- 172.21.15.100:0/528480153 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55b5c5acdc60 con 0x55b5c5abc290 2024-05-10T11:38:32.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.137+0000 7f6f70ff9700 1 -- 172.21.15.100:0/528480153 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6f78023070 con 0x55b5c5abc290 2024-05-10T11:38:32.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.137+0000 7f6f70ff9700 1 -- 172.21.15.100:0/528480153 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f6f7801f430 con 0x55b5c5abc290 2024-05-10T11:38:32.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.138+0000 7f6f9a086b80 10 client.28679 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:38:32.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.138+0000 7f6f9a086b80 10 client.28679 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:38:32.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.138+0000 7f6f9a086b80 20 client.28679 populate_metadata read hostname 'smithi100' 2024-05-10T11:38:32.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.138+0000 7f6f9a086b80 1 --2- 172.21.15.100:0/528480153 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55b5c5ad02f0 0x55b5c5ad2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:32.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.138+0000 7f6f9a086b80 4 client.28679 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:38:32.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.138+0000 7f6f9a086b80 1 -- 172.21.15.100:0/528480153 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6f840aa060 con 0x55b5c5ad02f0 2024-05-10T11:38:32.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.138+0000 7f6f8d58f700 1 --2- 172.21.15.100:0/528480153 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55b5c5ad02f0 0x55b5c5ad2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:32.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.138+0000 7f6f8d58f700 1 --2- 172.21.15.100:0/528480153 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55b5c5ad02f0 0x55b5c5ad2720 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44cc01e7c7f35095 server_cookie=f09e58fe9553be04 in_seq=0 out_seq=0 2024-05-10T11:38:32.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.138+0000 7f6f70ff9700 10 client.28679 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:38:32.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.139+0000 7f6f70ff9700 1 -- 172.21.15.100:0/528480153 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f6f840aa060 con 0x55b5c5ad02f0 2024-05-10T11:38:32.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.139+0000 7f6f70ff9700 10 client.28679 handle_command_reply: tid=0 2024-05-10T11:38:32.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.139+0000 7f6f9a086b80 1 client.28679 shutdown 2024-05-10T11:38:32.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.139+0000 7f6f9a086b80 1 -- 172.21.15.100:0/528480153 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b5c5abc290 msgr2=0x55b5c5abfc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:32.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.139+0000 7f6f9a086b80 1 --2- 172.21.15.100:0/528480153 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b5c5abc290 0x55b5c5abfc80 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6f80144f40 tx=0x7f6f780047c0 comp rx=0 tx=0).stop 2024-05-10T11:38:32.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.139+0000 7f6f9a086b80 1 -- 172.21.15.100:0/528480153 shutdown_connections 2024-05-10T11:38:32.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.139+0000 7f6f9a086b80 1 --2- 172.21.15.100:0/528480153 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55b5c5ad02f0 0x55b5c5ad2720 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:32.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.139+0000 7f6f70ff9700 0 client.28679 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:38:32.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.139+0000 7f6f9a086b80 1 --2- 172.21.15.100:0/528480153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5c5abcbb0 0x55b5c5ac01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:32.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.139+0000 7f6f9a086b80 1 --2- 172.21.15.100:0/528480153 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b5c5ab6d50 0x55b5c5ac2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:32.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.139+0000 7f6f9a086b80 1 --2- 172.21.15.100:0/528480153 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b5c5abc290 0x55b5c5abfc80 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:32.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.139+0000 7f6f9a086b80 1 -- 172.21.15.100:0/528480153 >> 172.21.15.100:0/528480153 conn(0x55b5c59e2b70 msgr2=0x55b5c5ab0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:32.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.139+0000 7f6f9a086b80 1 -- 172.21.15.100:0/528480153 shutdown_connections 2024-05-10T11:38:32.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.139+0000 7f6f9a086b80 1 -- 172.21.15.100:0/528480153 wait complete. 2024-05-10T11:38:32.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.140+0000 7f6f9a086b80 20 client.28679 trim_cache size 0 max 16384 2024-05-10T11:38:32.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.163+0000 7f6f9a086b80 1 Processor -- start 2024-05-10T11:38:32.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.164+0000 7f6f9a086b80 1 -- start start 2024-05-10T11:38:32.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.164+0000 7f6f9a086b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b5c5ab6d50 0x55b5c5acc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:32.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.164+0000 7f6f9a086b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b5c5abc290 0x55b5c5ab10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:32.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.164+0000 7f6f9a086b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5c5abcbb0 0x55b5c5ad7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:32.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.164+0000 7f6f8d58f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b5c5ab6d50 0x55b5c5acc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:32.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.164+0000 7f6f8d58f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b5c5ab6d50 0x55b5c5acc670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:36900/0 (socket says 172.21.15.100:36900) 2024-05-10T11:38:32.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.164+0000 7f6f9a086b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6f8015b830 con 0x55b5c5abcbb0 2024-05-10T11:38:32.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.164+0000 7f6f8cd8e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b5c5abc290 0x55b5c5ab10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:32.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.164+0000 7f6f9a086b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6f801590c0 con 0x55b5c5ab6d50 2024-05-10T11:38:32.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.164+0000 7f6f9a086b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6f80156f10 con 0x55b5c5abc290 2024-05-10T11:38:32.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.164+0000 7f6f8cd8e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b5c5abc290 0x55b5c5ab10e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:36246/0 (socket says 172.21.15.100:36246) 2024-05-10T11:38:32.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.164+0000 7f6f8cd8e700 1 -- 172.21.15.100:0/1080654243 learned_addr learned my addr 172.21.15.100:0/1080654243 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:38:32.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.165+0000 7f6f8dd90700 1 --2- 172.21.15.100:0/1080654243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5c5abcbb0 0x55b5c5ad7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:32.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.165+0000 7f6f8cd8e700 1 -- 172.21.15.100:0/1080654243 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b5c5ab6d50 msgr2=0x55b5c5acc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:32.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.165+0000 7f6f8cd8e700 1 --2- 172.21.15.100:0/1080654243 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b5c5ab6d50 0x55b5c5acc670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:32.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.165+0000 7f6f8cd8e700 1 -- 172.21.15.100:0/1080654243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5c5abcbb0 msgr2=0x55b5c5ad7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:32.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.165+0000 7f6f8cd8e700 1 --2- 172.21.15.100:0/1080654243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5c5abcbb0 0x55b5c5ad7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:32.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.165+0000 7f6f8cd8e700 1 -- 172.21.15.100:0/1080654243 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5c5ab76d0 con 0x55b5c5abc290 2024-05-10T11:38:32.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.165+0000 7f6f8d58f700 1 --2- 172.21.15.100:0/1080654243 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b5c5ab6d50 0x55b5c5acc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:38:32.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.165+0000 7f6f8cd8e700 1 --2- 172.21.15.100:0/1080654243 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b5c5abc290 0x55b5c5ab10e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6f7801f9c0 tx=0x7f6f78010b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97af693f9b9251fd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:32.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.165+0000 7f6f72ffd700 1 -- 172.21.15.100:0/1080654243 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f7801f430 con 0x55b5c5abc290 2024-05-10T11:38:32.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.166+0000 7f6f72ffd700 1 -- 172.21.15.100:0/1080654243 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6f78023070 con 0x55b5c5abc290 2024-05-10T11:38:32.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.166+0000 7f6f72ffd700 1 -- 172.21.15.100:0/1080654243 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f78005020 con 0x55b5c5abc290 2024-05-10T11:38:32.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.166+0000 7f6f9a086b80 1 -- 172.21.15.100:0/1080654243 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b5c5abc290 msgr2=0x55b5c5ab10e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:32.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.166+0000 7f6f9a086b80 1 --2- 172.21.15.100:0/1080654243 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b5c5abc290 0x55b5c5ab10e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6f7801f9c0 tx=0x7f6f78010b30 comp rx=0 tx=0).stop 2024-05-10T11:38:32.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.166+0000 7f6f9a086b80 1 -- 172.21.15.100:0/1080654243 shutdown_connections 2024-05-10T11:38:32.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.166+0000 7f6f9a086b80 1 --2- 172.21.15.100:0/1080654243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5c5abcbb0 0x55b5c5ad7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:32.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.166+0000 7f6f9a086b80 1 --2- 172.21.15.100:0/1080654243 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b5c5ab6d50 0x55b5c5acc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:32.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.166+0000 7f6f9a086b80 1 --2- 172.21.15.100:0/1080654243 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b5c5abc290 0x55b5c5ab10e0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:32.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.167+0000 7f6f9a086b80 1 -- 172.21.15.100:0/1080654243 >> 172.21.15.100:0/1080654243 conn(0x55b5c59e2b70 msgr2=0x55b5c5ab9d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:32.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.167+0000 7f6f9a086b80 1 -- 172.21.15.100:0/1080654243 shutdown_connections 2024-05-10T11:38:32.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.167+0000 7f6f9a086b80 1 -- 172.21.15.100:0/1080654243 wait complete. 2024-05-10T11:38:32.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.167+0000 7f6f9a086b80 1 -- start start 2024-05-10T11:38:32.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.167+0000 7f6f9a086b80 1 Processor -- start 2024-05-10T11:38:32.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.167+0000 7f6f9a086b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b5c5ab6d50 0x55b5c5ad7a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:32.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.167+0000 7f6f9a086b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b5c5abc290 0x55b5c5ad7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:32.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.167+0000 7f6f9a086b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5c5abcbb0 0x55b5c5ad84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:32.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.168+0000 7f6f8d58f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b5c5ab6d50 0x55b5c5ad7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:32.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.168+0000 7f6f9a086b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6f80157580 con 0x55b5c5abcbb0 2024-05-10T11:38:32.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.168+0000 7f6f9a086b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6f801589f0 con 0x55b5c5abc290 2024-05-10T11:38:32.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.168+0000 7f6f9a086b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6f80157c50 con 0x55b5c5ab6d50 2024-05-10T11:38:32.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.168+0000 7f6f8d58f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b5c5ab6d50 0x55b5c5ad7a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:36260/0 (socket says 172.21.15.100:36260) 2024-05-10T11:38:32.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.168+0000 7f6f8d58f700 1 -- 172.21.15.100:0/91369628 learned_addr learned my addr 172.21.15.100:0/91369628 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:38:32.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.168+0000 7f6f8cd8e700 1 --2- 172.21.15.100:0/91369628 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b5c5abc290 0x55b5c5ad7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:32.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.168+0000 7f6f9a086b80 10 client.0 _ll_register_callbacks cb 0x55b5c599c5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:38:32.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.168+0000 7f6f8dd90700 1 --2- 172.21.15.100:0/91369628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5c5abcbb0 0x55b5c5ad84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:32.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.168+0000 7f6f8d58f700 1 -- 172.21.15.100:0/91369628 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b5c5abc290 msgr2=0x55b5c5ad7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:32.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.168+0000 7f6f8d58f700 1 --2- 172.21.15.100:0/91369628 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b5c5abc290 0x55b5c5ad7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:32.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.168+0000 7f6f8d58f700 1 -- 172.21.15.100:0/91369628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5c5abcbb0 msgr2=0x55b5c5ad84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:32.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.168+0000 7f6f8d58f700 1 --2- 172.21.15.100:0/91369628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5c5abcbb0 0x55b5c5ad84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:32.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.168+0000 7f6f8d58f700 1 -- 172.21.15.100:0/91369628 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5c5b39e00 con 0x55b5c5ab6d50 2024-05-10T11:38:32.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.168+0000 7f6f8cd8e700 1 --2- 172.21.15.100:0/91369628 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b5c5abc290 0x55b5c5ad7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:38:32.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.168+0000 7f6f8d58f700 1 --2- 172.21.15.100:0/91369628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b5c5ab6d50 0x55b5c5ad7a40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6f840a9660 tx=0x7f6f8400efd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1b1ee72aa76ec2e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:32.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.168+0000 7f6f70ff9700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:38:32.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.168+0000 7f6f70ff9700 1 -- 172.21.15.100:0/91369628 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f840add10 con 0x55b5c5ab6d50 2024-05-10T11:38:32.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.168+0000 7f6f70ff9700 1 -- 172.21.15.100:0/91369628 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f6f840cc0b0 con 0x55b5c5ab6d50 2024-05-10T11:38:32.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.168+0000 7f6f9a086b80 1 -- 172.21.15.100:0/91369628 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f78023070 con 0x55b5c5ab6d50 2024-05-10T11:38:32.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.168+0000 7f6f70ff9700 1 -- 172.21.15.100:0/91369628 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f6f840cc2b0 con 0x55b5c5ab6d50 2024-05-10T11:38:32.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.168+0000 7f6f9a086b80 10 client.28786 fetch_fsmap learned FSMap version 41 2024-05-10T11:38:32.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.168+0000 7f6f9a086b80 1 -- 172.21.15.100:0/91369628 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55b5c5b3b0c0 con 0x55b5c5ab6d50 2024-05-10T11:38:32.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.169+0000 7f6f70ff9700 1 -- 172.21.15.100:0/91369628 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6f78023070 con 0x55b5c5ab6d50 2024-05-10T11:38:32.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.169+0000 7f6f70ff9700 1 -- 172.21.15.100:0/91369628 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f6f840c6070 con 0x55b5c5ab6d50 2024-05-10T11:38:32.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.169+0000 7f6f9a086b80 10 client.28786 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:38:32.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.169+0000 7f6f9a086b80 10 client.28786 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:38:32.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.169+0000 7f6f9a086b80 20 client.28786 populate_metadata read hostname 'smithi100' 2024-05-10T11:38:32.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.169+0000 7f6f9a086b80 1 --2- 172.21.15.100:0/91369628 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55b5c5ad02f0 0x55b5c5b3d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:32.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.169+0000 7f6f9a086b80 4 client.28786 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:38:32.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.169+0000 7f6f9a086b80 1 -- 172.21.15.100:0/91369628 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b5c5a16320 con 0x55b5c5ad02f0 2024-05-10T11:38:32.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.169+0000 7f6f8cd8e700 1 --2- 172.21.15.100:0/91369628 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55b5c5ad02f0 0x55b5c5b3d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:32.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.170+0000 7f6f70ff9700 10 client.28786 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:38:32.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.170+0000 7f6f8cd8e700 1 --2- 172.21.15.100:0/91369628 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55b5c5ad02f0 0x55b5c5b3d360 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd75c8d529d46fcf server_cookie=f1b42e7111dd09ba in_seq=0 out_seq=0 2024-05-10T11:38:32.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.171+0000 7f6f70ff9700 1 -- 172.21.15.100:0/91369628 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b5c5a16320 con 0x55b5c5ad02f0 2024-05-10T11:38:32.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.171+0000 7f6f70ff9700 10 client.28786 handle_command_reply: tid=0 2024-05-10T11:38:32.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.171+0000 7f6f9a086b80 1 client.28786 shutdown 2024-05-10T11:38:32.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.171+0000 7f6f9a086b80 1 -- 172.21.15.100:0/91369628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b5c5ab6d50 msgr2=0x55b5c5ad7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:32.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.171+0000 7f6f9a086b80 1 --2- 172.21.15.100:0/91369628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b5c5ab6d50 0x55b5c5ad7a40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6f840a9660 tx=0x7f6f8400efd0 comp rx=0 tx=0).stop 2024-05-10T11:38:32.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.171+0000 7f6f9a086b80 1 -- 172.21.15.100:0/91369628 shutdown_connections 2024-05-10T11:38:32.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.171+0000 7f6f9a086b80 1 --2- 172.21.15.100:0/91369628 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55b5c5ad02f0 0x55b5c5b3d360 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:32.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.171+0000 7f6f9a086b80 1 --2- 172.21.15.100:0/91369628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5c5abcbb0 0x55b5c5ad84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:32.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.171+0000 7f6f70ff9700 0 client.28786 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:38:32.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.171+0000 7f6f9a086b80 1 --2- 172.21.15.100:0/91369628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b5c5ab6d50 0x55b5c5ad7a40 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:32.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.171+0000 7f6f9a086b80 1 --2- 172.21.15.100:0/91369628 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b5c5abc290 0x55b5c5ad7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:32.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.171+0000 7f6f9a086b80 1 -- 172.21.15.100:0/91369628 >> 172.21.15.100:0/91369628 conn(0x55b5c59e2b70 msgr2=0x55b5c5abcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:32.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.171+0000 7f6f9a086b80 1 -- 172.21.15.100:0/91369628 shutdown_connections 2024-05-10T11:38:32.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.171+0000 7f6f9a086b80 1 -- 172.21.15.100:0/91369628 wait complete. 2024-05-10T11:38:32.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.171+0000 7f6f9a086b80 20 client.28786 trim_cache size 0 max 16384 2024-05-10T11:38:32.175 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:38:32.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.172+0000 7f6f8e591700 1 -- 172.21.15.100:0/2484116668 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f6f740b29c0 msgr2=0x7f6f740b4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:32.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.172+0000 7f6f8e591700 1 --2- 172.21.15.100:0/2484116668 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f6f740b29c0 0x7f6f740b4e80 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f6f80144eb0 tx=0x7f6f7800b410 comp rx=0 tx=0).stop 2024-05-10T11:38:32.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.172+0000 7f6f8e591700 1 -- 172.21.15.100:0/2484116668 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f800a4de0 msgr2=0x7f6f80149620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:32.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.172+0000 7f6f8e591700 1 --2- 172.21.15.100:0/2484116668 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f800a4de0 0x7f6f80149620 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6f8400e9c0 tx=0x7f6f8400ecd0 comp rx=0 tx=0).stop 2024-05-10T11:38:32.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.172+0000 7f6f8e591700 1 -- 172.21.15.100:0/2484116668 shutdown_connections 2024-05-10T11:38:32.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.173+0000 7f6f8e591700 1 --2- 172.21.15.100:0/2484116668 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f6f740b29c0 0x7f6f740b4e80 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:32.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.173+0000 7f6f8e591700 1 --2- 172.21.15.100:0/2484116668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f800b7b00 0x7f6f801436f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:32.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.173+0000 7f6f8e591700 1 --2- 172.21.15.100:0/2484116668 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f800a4de0 0x7f6f80149620 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:32.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.173+0000 7f6f8e591700 1 --2- 172.21.15.100:0/2484116668 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f800a5700 0x7f6f80149b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:32.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.173+0000 7f6f8e591700 1 -- 172.21.15.100:0/2484116668 >> 172.21.15.100:0/2484116668 conn(0x7f6f8001a6a0 msgr2=0x7f6f800b55b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:32.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.173+0000 7f6f8e591700 1 -- 172.21.15.100:0/2484116668 shutdown_connections 2024-05-10T11:38:32.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:32.173+0000 7f6f8e591700 1 -- 172.21.15.100:0/2484116668 wait complete. 2024-05-10T11:38:32.944 INFO:tasks.workunit.client.0.smithi100.stdout: 10 64629 8.99 MB/sec execute 266 sec latency 76.607 ms 2024-05-10T11:38:33.944 INFO:tasks.workunit.client.0.smithi100.stdout: 10 64773 8.97 MB/sec execute 267 sec latency 80.305 ms 2024-05-10T11:38:34.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:33 smithi178 ceph-mon[33359]: pgmap v1026: 129 pgs: 129 active+clean; 508 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 3.8 MiB/s wr, 256 op/s 2024-05-10T11:38:34.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:33 smithi188 ceph-mon[34075]: pgmap v1026: 129 pgs: 129 active+clean; 508 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 3.8 MiB/s wr, 256 op/s 2024-05-10T11:38:34.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:33 smithi100 ceph-mon[30296]: pgmap v1026: 129 pgs: 129 active+clean; 508 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 3.8 MiB/s wr, 256 op/s 2024-05-10T11:38:34.135 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T11:38:34.136 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T11:38:34.506 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:38:34.945 INFO:tasks.workunit.client.0.smithi100.stdout: 10 64960 8.98 MB/sec execute 268 sec latency 76.489 ms 2024-05-10T11:38:35.944 INFO:tasks.workunit.client.0.smithi100.stdout: 10 65133 8.99 MB/sec execute 269 sec latency 75.235 ms 2024-05-10T11:38:35.975 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:35 smithi100 ceph-mon[30296]: pgmap v1027: 129 pgs: 129 active+clean; 508 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 903 KiB/s rd, 4.2 MiB/s wr, 302 op/s 2024-05-10T11:38:36.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:35 smithi178 ceph-mon[33359]: pgmap v1027: 129 pgs: 129 active+clean; 508 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 903 KiB/s rd, 4.2 MiB/s wr, 302 op/s 2024-05-10T11:38:36.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:35 smithi188 ceph-mon[34075]: pgmap v1027: 129 pgs: 129 active+clean; 508 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 903 KiB/s rd, 4.2 MiB/s wr, 302 op/s 2024-05-10T11:38:36.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.657+0000 7f41599cd700 1 -- 172.21.15.100:0/703754838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4154101a90 msgr2=0x7f4154101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:36.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.657+0000 7f41599cd700 1 --2- 172.21.15.100:0/703754838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4154101a90 0x7f4154101f10 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f41480098f0 tx=0x7f4148009c00 comp rx=0 tx=0).stop 2024-05-10T11:38:36.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.657+0000 7f41599cd700 1 -- 172.21.15.100:0/703754838 shutdown_connections 2024-05-10T11:38:36.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.657+0000 7f41599cd700 1 --2- 172.21.15.100:0/703754838 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4154102450 0x7f4154110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:36.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.657+0000 7f41599cd700 1 --2- 172.21.15.100:0/703754838 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4154109520 0x7f4154109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:36.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.657+0000 7f41599cd700 1 --2- 172.21.15.100:0/703754838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4154101a90 0x7f4154101f10 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:36.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.657+0000 7f41599cd700 1 -- 172.21.15.100:0/703754838 >> 172.21.15.100:0/703754838 conn(0x7f4154078a10 msgr2=0x7f41540ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:36.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.657+0000 7f41599cd700 1 -- 172.21.15.100:0/703754838 shutdown_connections 2024-05-10T11:38:36.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.657+0000 7f41599cd700 1 -- 172.21.15.100:0/703754838 wait complete. 2024-05-10T11:38:36.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.659+0000 7f41599cd700 1 Processor -- start 2024-05-10T11:38:36.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.659+0000 7f41599cd700 1 -- start start 2024-05-10T11:38:36.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.659+0000 7f41599cd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4154101a90 0x7f4154102c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:36.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.660+0000 7f41599cd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4154102450 0x7f41541031b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:36.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.660+0000 7f41599cd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4154109520 0x7f41541036f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:36.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.660+0000 7f41599cd700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4154113550 con 0x7f4154101a90 2024-05-10T11:38:36.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.660+0000 7f41599cd700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f41541133d0 con 0x7f4154109520 2024-05-10T11:38:36.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.660+0000 7f41599cd700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f41541136d0 con 0x7f4154102450 2024-05-10T11:38:36.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.660+0000 7f4152ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4154101a90 0x7f4154102c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:36.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.660+0000 7f41537fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4154109520 0x7f41541036f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:36.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.660+0000 7f41527fc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4154102450 0x7f41541031b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:36.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.660+0000 7f4152ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4154101a90 0x7f4154102c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34434/0 (socket says 172.21.15.100:34434) 2024-05-10T11:38:36.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.660+0000 7f4152ffd700 1 -- 172.21.15.100:0/720001464 learned_addr learned my addr 172.21.15.100:0/720001464 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:38:36.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.660+0000 7f41527fc700 1 -- 172.21.15.100:0/720001464 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4154109520 msgr2=0x7f41541036f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:36.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.660+0000 7f41527fc700 1 --2- 172.21.15.100:0/720001464 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4154109520 0x7f41541036f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:36.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.660+0000 7f41527fc700 1 -- 172.21.15.100:0/720001464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4154101a90 msgr2=0x7f4154102c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:36.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.660+0000 7f41527fc700 1 --2- 172.21.15.100:0/720001464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4154101a90 0x7f4154102c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:36.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.660+0000 7f41527fc700 1 -- 172.21.15.100:0/720001464 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41541a8050 con 0x7f4154102450 2024-05-10T11:38:36.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.661+0000 7f41527fc700 1 --2- 172.21.15.100:0/720001464 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4154102450 0x7f41541031b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4148009fd0 tx=0x7f414800f990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:36.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.661+0000 7f41589cb700 1 -- 172.21.15.100:0/720001464 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f414800b5e0 con 0x7f4154102450 2024-05-10T11:38:36.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.661+0000 7f41599cd700 1 -- 172.21.15.100:0/720001464 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41541a82e0 con 0x7f4154102450 2024-05-10T11:38:36.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.661+0000 7f41599cd700 1 -- 172.21.15.100:0/720001464 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41541a8790 con 0x7f4154102450 2024-05-10T11:38:36.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.661+0000 7f41589cb700 1 -- 172.21.15.100:0/720001464 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4148005040 con 0x7f4154102450 2024-05-10T11:38:36.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.661+0000 7f41589cb700 1 -- 172.21.15.100:0/720001464 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41480038e0 con 0x7f4154102450 2024-05-10T11:38:36.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.665+0000 7f41589cb700 1 -- 172.21.15.100:0/720001464 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f4148003a80 con 0x7f4154102450 2024-05-10T11:38:36.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.666+0000 7f4164260b80 1 Processor -- start 2024-05-10T11:38:36.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.666+0000 7f4164260b80 1 -- start start 2024-05-10T11:38:36.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.666+0000 7f4164260b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560edd726230 0x560edd726610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:36.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.666+0000 7f4164260b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560edd726b50 0x560edd720890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:36.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.666+0000 7f4164260b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560edd720dd0 0x560edd721210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:36.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.666+0000 7f4164260b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x560edd6729c0 con 0x560edd720dd0 2024-05-10T11:38:36.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.666+0000 7f4164260b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560edd5d7e20 con 0x560edd726b50 2024-05-10T11:38:36.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.666+0000 7f4164260b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x560edd66e980 con 0x560edd726230 2024-05-10T11:38:36.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.666+0000 7f41527fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560edd720dd0 0x560edd721210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:36.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.666+0000 7f41527fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560edd720dd0 0x560edd721210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34450/0 (socket says 172.21.15.100:34450) 2024-05-10T11:38:36.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.666+0000 7f41527fc700 1 -- 172.21.15.100:0/2588885712 learned_addr learned my addr 172.21.15.100:0/2588885712 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:38:36.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.667+0000 7f4152ffd700 1 --2- 172.21.15.100:0/2588885712 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560edd726b50 0x560edd720890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:36.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.667+0000 7f41589cb700 1 --2- 172.21.15.100:0/720001464 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f413c0b2900 0x7f413c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:36.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.667+0000 7f41527fc700 1 -- 172.21.15.100:0/2588885712 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560edd726230 msgr2=0x560edd726610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:36.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.667+0000 7f41527fc700 1 --2- 172.21.15.100:0/2588885712 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560edd726230 0x560edd726610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:36.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.667+0000 7f41527fc700 1 -- 172.21.15.100:0/2588885712 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560edd726b50 msgr2=0x560edd720890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:36.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.667+0000 7f41527fc700 1 --2- 172.21.15.100:0/2588885712 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560edd726b50 0x560edd720890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:36.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.667+0000 7f41527fc700 1 -- 172.21.15.100:0/2588885712 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560edd5bf950 con 0x560edd720dd0 2024-05-10T11:38:36.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.667+0000 7f41589cb700 1 -- 172.21.15.100:0/720001464 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f41480e3270 con 0x7f4154102450 2024-05-10T11:38:36.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.667+0000 7f41537fe700 1 --2- 172.21.15.100:0/720001464 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f413c0b2900 0x7f413c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:36.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.667+0000 7f41527fc700 1 --2- 172.21.15.100:0/2588885712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560edd720dd0 0x560edd721210 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f414809ba90 tx=0x7f414806fb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d943541c41a4633 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:36.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.668+0000 7f4139ffb700 1 -- 172.21.15.100:0/2588885712 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41480aa390 con 0x560edd720dd0 2024-05-10T11:38:36.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.668+0000 7f4139ffb700 1 -- 172.21.15.100:0/2588885712 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4148021e60 con 0x560edd720dd0 2024-05-10T11:38:36.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.668+0000 7f4139ffb700 1 -- 172.21.15.100:0/2588885712 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41480a9d30 con 0x560edd720dd0 2024-05-10T11:38:36.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.669+0000 7f41537fe700 1 --2- 172.21.15.100:0/720001464 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f413c0b2900 0x7f413c0b4dc0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f414c0060b0 tx=0x7f414c009920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:36.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.669+0000 7f4164260b80 1 -- 172.21.15.100:0/2588885712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560edd720dd0 msgr2=0x560edd721210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:36.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.669+0000 7f4164260b80 1 --2- 172.21.15.100:0/2588885712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560edd720dd0 0x560edd721210 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f414809ba90 tx=0x7f414806fb00 comp rx=0 tx=0).stop 2024-05-10T11:38:36.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.669+0000 7f4164260b80 1 -- 172.21.15.100:0/2588885712 shutdown_connections 2024-05-10T11:38:36.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.669+0000 7f4164260b80 1 --2- 172.21.15.100:0/2588885712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560edd720dd0 0x560edd721210 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:36.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.669+0000 7f4164260b80 1 --2- 172.21.15.100:0/2588885712 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560edd726230 0x560edd726610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:36.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.669+0000 7f4164260b80 1 --2- 172.21.15.100:0/2588885712 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560edd726b50 0x560edd720890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:36.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.669+0000 7f4164260b80 1 -- 172.21.15.100:0/2588885712 >> 172.21.15.100:0/2588885712 conn(0x560edd64cbf0 msgr2=0x560edd71b320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:36.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.669+0000 7f4164260b80 1 -- 172.21.15.100:0/2588885712 shutdown_connections 2024-05-10T11:38:36.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.669+0000 7f4164260b80 1 -- 172.21.15.100:0/2588885712 wait complete. 2024-05-10T11:38:36.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.670+0000 7f4164260b80 1 -- start start 2024-05-10T11:38:36.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.670+0000 7f4164260b80 1 Processor -- start 2024-05-10T11:38:36.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.671+0000 7f4164260b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560edd720dd0 0x560edd72c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:36.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.671+0000 7f4164260b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560edd726230 0x560edd729b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:36.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.671+0000 7f4164260b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560edd726b50 0x560edd72a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:36.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.671+0000 7f4164260b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x560edd6729c0 con 0x560edd726230 2024-05-10T11:38:36.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.671+0000 7f41527fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560edd726230 0x560edd729b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:36.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.671+0000 7f4164260b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560edd62bf50 con 0x560edd720dd0 2024-05-10T11:38:36.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.671+0000 7f4164260b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x560edd72d290 con 0x560edd726b50 2024-05-10T11:38:36.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.671+0000 7f41537fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560edd726b50 0x560edd72a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:36.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.671+0000 7f41527fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560edd726230 0x560edd729b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34458/0 (socket says 172.21.15.100:34458) 2024-05-10T11:38:36.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.671+0000 7f41537fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560edd726b50 0x560edd72a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:37514/0 (socket says 172.21.15.100:37514) 2024-05-10T11:38:36.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.671+0000 7f41527fc700 1 -- 172.21.15.100:0/590691218 learned_addr learned my addr 172.21.15.100:0/590691218 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:38:36.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.671+0000 7f4164260b80 10 client.0 _ll_register_callbacks cb 0x560edd606630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:38:36.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.672+0000 7f41537fe700 1 -- 172.21.15.100:0/590691218 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560edd720dd0 msgr2=0x560edd72c780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T11:38:36.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.672+0000 7f41537fe700 1 --2- 172.21.15.100:0/590691218 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560edd720dd0 0x560edd72c780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:36.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.672+0000 7f41537fe700 1 -- 172.21.15.100:0/590691218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560edd726230 msgr2=0x560edd729b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:36.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.672+0000 7f41537fe700 1 --2- 172.21.15.100:0/590691218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560edd726230 0x560edd729b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:36.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.672+0000 7f41537fe700 1 -- 172.21.15.100:0/590691218 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560edd6297e0 con 0x560edd726b50 2024-05-10T11:38:36.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.672+0000 7f41537fe700 1 --2- 172.21.15.100:0/590691218 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560edd726b50 0x560edd72a060 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f414c004b10 tx=0x7f414c004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d57314d016eddf8f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:36.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.672+0000 7f4132ffd700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:38:36.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.672+0000 7f4132ffd700 1 -- 172.21.15.100:0/590691218 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f414c026060 con 0x560edd726b50 2024-05-10T11:38:36.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.672+0000 7f4132ffd700 1 -- 172.21.15.100:0/590691218 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f414c0147e0 con 0x560edd726b50 2024-05-10T11:38:36.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.672+0000 7f4164260b80 1 -- 172.21.15.100:0/590691218 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560edd6803a0 con 0x560edd726b50 2024-05-10T11:38:36.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.673+0000 7f4132ffd700 1 -- 172.21.15.100:0/590691218 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f414c02d050 con 0x560edd726b50 2024-05-10T11:38:36.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.673+0000 7f4164260b80 10 client.28795 fetch_fsmap learned FSMap version 41 2024-05-10T11:38:36.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.673+0000 7f4164260b80 1 -- 172.21.15.100:0/590691218 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x560edd737860 con 0x560edd726b50 2024-05-10T11:38:36.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.673+0000 7f4132ffd700 1 -- 172.21.15.100:0/590691218 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x560edd6803a0 con 0x560edd726b50 2024-05-10T11:38:36.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.673+0000 7f4132ffd700 1 -- 172.21.15.100:0/590691218 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f414c021aa0 con 0x560edd726b50 2024-05-10T11:38:36.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.673+0000 7f4164260b80 10 client.28795 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:38:36.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.673+0000 7f4164260b80 10 client.28795 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:38:36.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.673+0000 7f4164260b80 20 client.28795 populate_metadata read hostname 'smithi100' 2024-05-10T11:38:36.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.674+0000 7f4164260b80 1 --2- 172.21.15.100:0/590691218 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x560edd739ef0 0x560edd73c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:36.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.674+0000 7f4164260b80 4 client.28795 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:38:36.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.674+0000 7f4164260b80 1 -- 172.21.15.100:0/590691218 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560edd67a440 con 0x560edd739ef0 2024-05-10T11:38:36.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.674+0000 7f4152ffd700 1 --2- 172.21.15.100:0/590691218 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x560edd739ef0 0x560edd73c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:36.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.674+0000 7f4132ffd700 10 client.28795 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:38:36.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.674+0000 7f4152ffd700 1 --2- 172.21.15.100:0/590691218 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x560edd739ef0 0x560edd73c330 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e1c94df7473204c server_cookie=1c7c5fee610b2550 in_seq=0 out_seq=0 2024-05-10T11:38:36.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.675+0000 7f4132ffd700 1 -- 172.21.15.100:0/590691218 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560edd67a440 con 0x560edd739ef0 2024-05-10T11:38:36.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.675+0000 7f4132ffd700 10 client.28795 handle_command_reply: tid=0 2024-05-10T11:38:36.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.675+0000 7f4164260b80 1 client.28795 shutdown 2024-05-10T11:38:36.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.676+0000 7f4164260b80 1 -- 172.21.15.100:0/590691218 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560edd726b50 msgr2=0x560edd72a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:36.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.676+0000 7f4164260b80 1 --2- 172.21.15.100:0/590691218 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560edd726b50 0x560edd72a060 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f414c004b10 tx=0x7f414c004b40 comp rx=0 tx=0).stop 2024-05-10T11:38:36.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.676+0000 7f4164260b80 1 -- 172.21.15.100:0/590691218 shutdown_connections 2024-05-10T11:38:36.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.676+0000 7f4164260b80 1 --2- 172.21.15.100:0/590691218 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x560edd739ef0 0x560edd73c330 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:36.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.676+0000 7f4164260b80 1 --2- 172.21.15.100:0/590691218 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560edd726b50 0x560edd72a060 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:36.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.676+0000 7f4164260b80 1 --2- 172.21.15.100:0/590691218 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560edd720dd0 0x560edd72c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:36.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.676+0000 7f4164260b80 1 --2- 172.21.15.100:0/590691218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560edd726230 0x560edd729b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:36.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.676+0000 7f4164260b80 1 -- 172.21.15.100:0/590691218 >> 172.21.15.100:0/590691218 conn(0x560edd64cbf0 msgr2=0x560edd715bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:36.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.676+0000 7f4132ffd700 0 client.28795 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:38:36.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.676+0000 7f4164260b80 1 -- 172.21.15.100:0/590691218 shutdown_connections 2024-05-10T11:38:36.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.676+0000 7f4164260b80 1 -- 172.21.15.100:0/590691218 wait complete. 2024-05-10T11:38:36.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.676+0000 7f4164260b80 20 client.28795 trim_cache size 0 max 16384 2024-05-10T11:38:36.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.705+0000 7f4164260b80 1 Processor -- start 2024-05-10T11:38:36.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.706+0000 7f4164260b80 1 -- start start 2024-05-10T11:38:36.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.706+0000 7f4164260b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560edd720dd0 0x560edd720890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:36.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.706+0000 7f4164260b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560edd726230 0x560edd715cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:36.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.706+0000 7f4164260b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560edd726b50 0x560edd725360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:36.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.706+0000 7f4152ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560edd720dd0 0x560edd720890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:36.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.706+0000 7f4152ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560edd720dd0 0x560edd720890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:54546/0 (socket says 172.21.15.100:54546) 2024-05-10T11:38:36.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.706+0000 7f4164260b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f41400112e0 con 0x560edd726230 2024-05-10T11:38:36.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.706+0000 7f4164260b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f414000eb70 con 0x560edd720dd0 2024-05-10T11:38:36.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.706+0000 7f4164260b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f414000c9c0 con 0x560edd726b50 2024-05-10T11:38:36.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.706+0000 7f4152ffd700 1 -- 172.21.15.100:0/2875612222 learned_addr learned my addr 172.21.15.100:0/2875612222 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:38:36.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.706+0000 7f41527fc700 1 --2- 172.21.15.100:0/2875612222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560edd726230 0x560edd715cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:36.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.706+0000 7f41537fe700 1 --2- 172.21.15.100:0/2875612222 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560edd726b50 0x560edd725360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:36.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.707+0000 7f4152ffd700 1 -- 172.21.15.100:0/2875612222 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560edd726b50 msgr2=0x560edd725360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:36.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.707+0000 7f4152ffd700 1 --2- 172.21.15.100:0/2875612222 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560edd726b50 0x560edd725360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:36.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.707+0000 7f4152ffd700 1 -- 172.21.15.100:0/2875612222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560edd726230 msgr2=0x560edd715cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:36.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.707+0000 7f4152ffd700 1 --2- 172.21.15.100:0/2875612222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560edd726230 0x560edd715cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:36.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.707+0000 7f4152ffd700 1 -- 172.21.15.100:0/2875612222 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560edd726950 con 0x560edd720dd0 2024-05-10T11:38:36.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.707+0000 7f41537fe700 1 --2- 172.21.15.100:0/2875612222 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560edd726b50 0x560edd725360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:38:36.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.707+0000 7f4152ffd700 1 --2- 172.21.15.100:0/2875612222 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560edd720dd0 0x560edd720890 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x560edd71c0c0 tx=0x7f4144010710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa84eaa84e8288a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:36.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.707+0000 7f41397fa700 1 -- 172.21.15.100:0/2875612222 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4144010dc0 con 0x560edd720dd0 2024-05-10T11:38:36.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.707+0000 7f41397fa700 1 -- 172.21.15.100:0/2875612222 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f41440044e0 con 0x560edd720dd0 2024-05-10T11:38:36.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.707+0000 7f41397fa700 1 -- 172.21.15.100:0/2875612222 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4144003da0 con 0x560edd720dd0 2024-05-10T11:38:36.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.708+0000 7f4164260b80 1 -- 172.21.15.100:0/2875612222 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560edd720dd0 msgr2=0x560edd720890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:36.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.708+0000 7f4164260b80 1 --2- 172.21.15.100:0/2875612222 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560edd720dd0 0x560edd720890 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x560edd71c0c0 tx=0x7f4144010710 comp rx=0 tx=0).stop 2024-05-10T11:38:36.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.708+0000 7f4164260b80 1 -- 172.21.15.100:0/2875612222 shutdown_connections 2024-05-10T11:38:36.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.708+0000 7f4164260b80 1 --2- 172.21.15.100:0/2875612222 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560edd726b50 0x560edd725360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:36.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.708+0000 7f4164260b80 1 --2- 172.21.15.100:0/2875612222 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560edd720dd0 0x560edd720890 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:36.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.708+0000 7f4164260b80 1 --2- 172.21.15.100:0/2875612222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560edd726230 0x560edd715cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:36.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.708+0000 7f4164260b80 1 -- 172.21.15.100:0/2875612222 >> 172.21.15.100:0/2875612222 conn(0x560edd64cbf0 msgr2=0x560edd724c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:36.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.708+0000 7f4164260b80 1 -- 172.21.15.100:0/2875612222 shutdown_connections 2024-05-10T11:38:36.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.708+0000 7f4164260b80 1 -- 172.21.15.100:0/2875612222 wait complete. 2024-05-10T11:38:36.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.709+0000 7f4164260b80 1 -- start start 2024-05-10T11:38:36.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.709+0000 7f4164260b80 1 Processor -- start 2024-05-10T11:38:36.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.709+0000 7f4164260b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560edd720dd0 0x560edd7308a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:36.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.709+0000 7f4164260b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560edd726230 0x560edd730de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:36.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.709+0000 7f4164260b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560edd726b50 0x560edd7a3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:36.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.709+0000 7f4164260b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f414000d030 con 0x560edd726230 2024-05-10T11:38:36.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.709+0000 7f4164260b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f414000e4a0 con 0x560edd720dd0 2024-05-10T11:38:36.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.709+0000 7f4164260b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f414000d700 con 0x560edd726b50 2024-05-10T11:38:36.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.709+0000 7f41527fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560edd726230 0x560edd730de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:36.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.709+0000 7f4164260b80 10 client.0 _ll_register_callbacks cb 0x560edd606630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:38:36.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.709+0000 7f41527fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560edd726230 0x560edd730de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34476/0 (socket says 172.21.15.100:34476) 2024-05-10T11:38:36.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.709+0000 7f41527fc700 1 -- 172.21.15.100:0/1686435561 learned_addr learned my addr 172.21.15.100:0/1686435561 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:38:36.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.710+0000 7f41527fc700 1 -- 172.21.15.100:0/1686435561 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560edd726b50 msgr2=0x560edd7a3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:36.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.710+0000 7f41527fc700 1 --2- 172.21.15.100:0/1686435561 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560edd726b50 0x560edd7a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:36.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.710+0000 7f41527fc700 1 -- 172.21.15.100:0/1686435561 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560edd720dd0 msgr2=0x560edd7308a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T11:38:36.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.710+0000 7f4152ffd700 1 --2- 172.21.15.100:0/1686435561 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560edd720dd0 0x560edd7308a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:36.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.710+0000 7f41527fc700 1 --2- 172.21.15.100:0/1686435561 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560edd720dd0 0x560edd7308a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:36.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.710+0000 7f41527fc700 1 -- 172.21.15.100:0/1686435561 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560edd7a3d70 con 0x560edd726230 2024-05-10T11:38:36.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.710+0000 7f41527fc700 1 --2- 172.21.15.100:0/1686435561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560edd726230 0x560edd730de0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f41480a6640 tx=0x7f4148009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a23d2b78b2efedf0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:36.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.710+0000 7f4132ffd700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:38:36.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.711+0000 7f4132ffd700 1 -- 172.21.15.100:0/1686435561 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41480a9f10 con 0x560edd726230 2024-05-10T11:38:36.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.711+0000 7f4132ffd700 1 -- 172.21.15.100:0/1686435561 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f41480a5f30 con 0x560edd726230 2024-05-10T11:38:36.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.711+0000 7f4164260b80 1 -- 172.21.15.100:0/1686435561 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41440044e0 con 0x560edd726230 2024-05-10T11:38:36.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.711+0000 7f4132ffd700 1 -- 172.21.15.100:0/1686435561 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f41480aa0d0 con 0x560edd726230 2024-05-10T11:38:36.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.711+0000 7f4164260b80 10 client.19419 fetch_fsmap learned FSMap version 41 2024-05-10T11:38:36.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.711+0000 7f4164260b80 1 -- 172.21.15.100:0/1686435561 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x560edd7a50b0 con 0x560edd726230 2024-05-10T11:38:36.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.712+0000 7f4132ffd700 1 -- 172.21.15.100:0/1686435561 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f41480aacc0 con 0x560edd726230 2024-05-10T11:38:36.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.712+0000 7f4132ffd700 1 -- 172.21.15.100:0/1686435561 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f41480a4ce0 con 0x560edd726230 2024-05-10T11:38:36.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.712+0000 7f4164260b80 10 client.19419 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:38:36.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.712+0000 7f4164260b80 10 client.19419 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:38:36.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.712+0000 7f4164260b80 20 client.19419 populate_metadata read hostname 'smithi100' 2024-05-10T11:38:36.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.712+0000 7f4164260b80 1 --2- 172.21.15.100:0/1686435561 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x560edd7a5600 0x560edd7a7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:36.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.712+0000 7f4164260b80 4 client.19419 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T11:38:36.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.712+0000 7f4164260b80 1 -- 172.21.15.100:0/1686435561 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f41440044e0 con 0x560edd7a5600 2024-05-10T11:38:36.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.712+0000 7f4152ffd700 1 --2- 172.21.15.100:0/1686435561 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x560edd7a5600 0x560edd7a7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:36.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.713+0000 7f4132ffd700 10 client.19419 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:38:36.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.713+0000 7f4152ffd700 1 --2- 172.21.15.100:0/1686435561 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x560edd7a5600 0x560edd7a7a20 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da22dc1264b48517 server_cookie=bf410dcd12e0f746 in_seq=0 out_seq=0 2024-05-10T11:38:36.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.714+0000 7f4132ffd700 1 -- 172.21.15.100:0/1686435561 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f41440044e0 con 0x560edd7a5600 2024-05-10T11:38:36.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.714+0000 7f4132ffd700 10 client.19419 handle_command_reply: tid=0 2024-05-10T11:38:36.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.714+0000 7f4164260b80 1 client.19419 shutdown 2024-05-10T11:38:36.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.714+0000 7f4164260b80 1 -- 172.21.15.100:0/1686435561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560edd726230 msgr2=0x560edd730de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:36.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.714+0000 7f4164260b80 1 --2- 172.21.15.100:0/1686435561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560edd726230 0x560edd730de0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f41480a6640 tx=0x7f4148009c00 comp rx=0 tx=0).stop 2024-05-10T11:38:36.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.714+0000 7f4164260b80 1 -- 172.21.15.100:0/1686435561 shutdown_connections 2024-05-10T11:38:36.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.714+0000 7f4164260b80 1 --2- 172.21.15.100:0/1686435561 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x560edd7a5600 0x560edd7a7a20 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:36.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.714+0000 7f4164260b80 1 --2- 172.21.15.100:0/1686435561 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560edd726b50 0x560edd7a3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:36.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.714+0000 7f4164260b80 1 --2- 172.21.15.100:0/1686435561 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560edd720dd0 0x560edd7308a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:36.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.714+0000 7f4164260b80 1 --2- 172.21.15.100:0/1686435561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560edd726230 0x560edd730de0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:36.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.714+0000 7f4164260b80 1 -- 172.21.15.100:0/1686435561 >> 172.21.15.100:0/1686435561 conn(0x560edd64cbf0 msgr2=0x560edd727640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:36.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.714+0000 7f4132ffd700 0 client.19419 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:38:36.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.714+0000 7f4164260b80 1 -- 172.21.15.100:0/1686435561 shutdown_connections 2024-05-10T11:38:36.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.715+0000 7f4164260b80 1 -- 172.21.15.100:0/1686435561 wait complete. 2024-05-10T11:38:36.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.715+0000 7f4164260b80 20 client.19419 trim_cache size 0 max 16384 2024-05-10T11:38:36.718 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:38:36.718 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-05-10T11:38:36.718 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "a14e2bed-1997-4226-8cb0-6cb7eaad5456", 2024-05-10T11:38:36.718 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-05-10T11:38:36.718 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:38:36.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.715+0000 7f41599cd700 1 -- 172.21.15.100:0/720001464 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f413c0b2900 msgr2=0x7f413c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:36.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.715+0000 7f41599cd700 1 --2- 172.21.15.100:0/720001464 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f413c0b2900 0x7f413c0b4dc0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f414c0060b0 tx=0x7f414c009920 comp rx=0 tx=0).stop 2024-05-10T11:38:36.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.715+0000 7f41599cd700 1 -- 172.21.15.100:0/720001464 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4154102450 msgr2=0x7f41541031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:36.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.715+0000 7f41599cd700 1 --2- 172.21.15.100:0/720001464 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4154102450 0x7f41541031b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4148009fd0 tx=0x7f414800f990 comp rx=0 tx=0).stop 2024-05-10T11:38:36.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.716+0000 7f41599cd700 1 -- 172.21.15.100:0/720001464 shutdown_connections 2024-05-10T11:38:36.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.716+0000 7f41599cd700 1 --2- 172.21.15.100:0/720001464 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f413c0b2900 0x7f413c0b4dc0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:36.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.716+0000 7f41599cd700 1 --2- 172.21.15.100:0/720001464 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4154109520 0x7f41541036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:36.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.716+0000 7f41599cd700 1 --2- 172.21.15.100:0/720001464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4154101a90 0x7f4154102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:36.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.716+0000 7f41599cd700 1 --2- 172.21.15.100:0/720001464 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4154102450 0x7f41541031b0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:36.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.716+0000 7f41599cd700 1 -- 172.21.15.100:0/720001464 >> 172.21.15.100:0/720001464 conn(0x7f4154078a10 msgr2=0x7f4154100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:36.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.716+0000 7f41599cd700 1 -- 172.21.15.100:0/720001464 shutdown_connections 2024-05-10T11:38:36.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:36.716+0000 7f41599cd700 1 -- 172.21.15.100:0/720001464 wait complete. 2024-05-10T11:38:36.944 INFO:tasks.workunit.client.0.smithi100.stdout: 10 65277 8.98 MB/sec execute 270 sec latency 104.119 ms 2024-05-10T11:38:37.591 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T11:38:37.938 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:38:37.944 INFO:tasks.workunit.client.0.smithi100.stdout: 10 65479 9.00 MB/sec execute 271 sec latency 75.281 ms 2024-05-10T11:38:38.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:37 smithi178 ceph-mon[33359]: pgmap v1028: 129 pgs: 129 active+clean; 513 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 4.5 MiB/s wr, 302 op/s 2024-05-10T11:38:38.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:37 smithi178 ceph-mon[33359]: scrub queued for path: / 2024-05-10T11:38:38.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:37 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:38:38.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:37 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:38:38.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:37 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:38:38.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:37 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:38:38.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:37 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:38:38.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:37 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:38:38.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:37 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:38:38.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:37 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:38:38.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:37 smithi188 ceph-mon[34075]: pgmap v1028: 129 pgs: 129 active+clean; 513 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 4.5 MiB/s wr, 302 op/s 2024-05-10T11:38:38.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:37 smithi188 ceph-mon[34075]: scrub queued for path: / 2024-05-10T11:38:38.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:37 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:38:38.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:37 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:38:38.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:37 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:38:38.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:37 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:38:38.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:37 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:38:38.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:37 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:38:38.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:37 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:38:38.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:37 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:38:38.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:37 smithi100 ceph-mon[30296]: pgmap v1028: 129 pgs: 129 active+clean; 513 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 4.5 MiB/s wr, 302 op/s 2024-05-10T11:38:38.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:37 smithi100 ceph-mon[30296]: scrub queued for path: / 2024-05-10T11:38:38.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:37 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:38:38.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:37 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:38:38.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:37 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:38:38.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:37 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:38:38.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:37 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:38:38.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:37 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:38:38.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:37 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:38:38.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:37 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:38:38.683 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:38 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:38:38.945 INFO:tasks.workunit.client.0.smithi100.stdout: 10 65655 8.99 MB/sec execute 272 sec latency 82.925 ms 2024-05-10T11:38:39.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:38 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:38:39.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:38 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:38:39.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:39 smithi100 ceph-mon[30296]: pgmap v1029: 129 pgs: 129 active+clean; 511 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 761 KiB/s rd, 4.1 MiB/s wr, 306 op/s 2024-05-10T11:38:39.945 INFO:tasks.workunit.client.0.smithi100.stdout: 10 65800 8.96 MB/sec execute 273 sec latency 103.892 ms 2024-05-10T11:38:40.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:39 smithi178 ceph-mon[33359]: pgmap v1029: 129 pgs: 129 active+clean; 511 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 761 KiB/s rd, 4.1 MiB/s wr, 306 op/s 2024-05-10T11:38:40.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:39 smithi188 ceph-mon[34075]: pgmap v1029: 129 pgs: 129 active+clean; 511 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 761 KiB/s rd, 4.1 MiB/s wr, 306 op/s 2024-05-10T11:38:40.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.242+0000 7f06b1c18700 1 -- 172.21.15.100:0/451846281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06ac104100 msgr2=0x7f06ac108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:40.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.242+0000 7f06b1c18700 1 --2- 172.21.15.100:0/451846281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06ac104100 0x7f06ac108120 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f06a00099c0 tx=0x7f06a0009cd0 comp rx=0 tx=0).stop 2024-05-10T11:38:40.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.242+0000 7f06b1c18700 1 -- 172.21.15.100:0/451846281 shutdown_connections 2024-05-10T11:38:40.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.242+0000 7f06b1c18700 1 --2- 172.21.15.100:0/451846281 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06ac108850 0x7f06ac1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:40.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.242+0000 7f06b1c18700 1 --2- 172.21.15.100:0/451846281 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06ac103750 0x7f06ac103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:40.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.242+0000 7f06b1c18700 1 --2- 172.21.15.100:0/451846281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06ac104100 0x7f06ac108120 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:40.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.242+0000 7f06b1c18700 1 -- 172.21.15.100:0/451846281 >> 172.21.15.100:0/451846281 conn(0x7f06ac07a600 msgr2=0x7f06ac07aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:40.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.242+0000 7f06b1c18700 1 -- 172.21.15.100:0/451846281 shutdown_connections 2024-05-10T11:38:40.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.242+0000 7f06b1c18700 1 -- 172.21.15.100:0/451846281 wait complete. 2024-05-10T11:38:40.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.243+0000 7f06b1c18700 1 Processor -- start 2024-05-10T11:38:40.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.243+0000 7f06b1c18700 1 -- start start 2024-05-10T11:38:40.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.243+0000 7f06b1c18700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06ac103750 0x7f06ac1a0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:40.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.244+0000 7f06b1c18700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06ac104100 0x7f06ac1a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:40.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.244+0000 7f06b1c18700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06ac108850 0x7f06ac1a49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:40.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.244+0000 7f06b1c18700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f06ac119bb0 con 0x7f06ac108850 2024-05-10T11:38:40.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.244+0000 7f06b1c18700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f06ac119a30 con 0x7f06ac103750 2024-05-10T11:38:40.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.244+0000 7f06b1c18700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f06ac119d30 con 0x7f06ac104100 2024-05-10T11:38:40.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.244+0000 7f06ab7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06ac103750 0x7f06ac1a0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:40.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.244+0000 7f06ab7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06ac103750 0x7f06ac1a0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:54574/0 (socket says 172.21.15.100:54574) 2024-05-10T11:38:40.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.244+0000 7f06ab7fe700 1 -- 172.21.15.100:0/3016337083 learned_addr learned my addr 172.21.15.100:0/3016337083 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:38:40.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.244+0000 7f06aaffd700 1 --2- 172.21.15.100:0/3016337083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06ac104100 0x7f06ac1a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:40.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.244+0000 7f06abfff700 1 --2- 172.21.15.100:0/3016337083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06ac108850 0x7f06ac1a49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:40.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.244+0000 7f06aaffd700 1 -- 172.21.15.100:0/3016337083 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06ac103750 msgr2=0x7f06ac1a0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:40.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.244+0000 7f06aaffd700 1 --2- 172.21.15.100:0/3016337083 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06ac103750 0x7f06ac1a0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:40.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.244+0000 7f06aaffd700 1 -- 172.21.15.100:0/3016337083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06ac108850 msgr2=0x7f06ac1a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:40.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.244+0000 7f06aaffd700 1 --2- 172.21.15.100:0/3016337083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06ac108850 0x7f06ac1a49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:40.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.244+0000 7f06aaffd700 1 -- 172.21.15.100:0/3016337083 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06ac1a50e0 con 0x7f06ac104100 2024-05-10T11:38:40.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.244+0000 7f06abfff700 1 --2- 172.21.15.100:0/3016337083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06ac108850 0x7f06ac1a49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:38:40.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.244+0000 7f06aaffd700 1 --2- 172.21.15.100:0/3016337083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06ac104100 0x7f06ac1a0660 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f06a00098e0 tx=0x7f06a000bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:40.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.245+0000 7f06a8ff9700 1 -- 172.21.15.100:0/3016337083 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a0027070 con 0x7f06ac104100 2024-05-10T11:38:40.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.245+0000 7f06a8ff9700 1 -- 172.21.15.100:0/3016337083 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f06a0011e50 con 0x7f06ac104100 2024-05-10T11:38:40.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.245+0000 7f06b1c18700 1 -- 172.21.15.100:0/3016337083 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06ac1a5370 con 0x7f06ac104100 2024-05-10T11:38:40.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.245+0000 7f06a8ff9700 1 -- 172.21.15.100:0/3016337083 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a0003ae0 con 0x7f06ac104100 2024-05-10T11:38:40.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.245+0000 7f06b1c18700 1 -- 172.21.15.100:0/3016337083 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06ac1acc50 con 0x7f06ac104100 2024-05-10T11:38:40.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.248+0000 7f06a8ff9700 1 -- 172.21.15.100:0/3016337083 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f06a002a070 con 0x7f06ac104100 2024-05-10T11:38:40.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.249+0000 7f06a8ff9700 1 --2- 172.21.15.100:0/3016337083 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f06940b2900 0x7f06940b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:40.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.249+0000 7f06ab7fe700 1 --2- 172.21.15.100:0/3016337083 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f06940b2900 0x7f06940b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:40.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.249+0000 7f06a8ff9700 1 -- 172.21.15.100:0/3016337083 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f06a00e3410 con 0x7f06ac104100 2024-05-10T11:38:40.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.249+0000 7f06ab7fe700 1 --2- 172.21.15.100:0/3016337083 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f06940b2900 0x7f06940b4dc0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f069c009d60 tx=0x7f069c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:40.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.250+0000 7f06bc4abb80 1 Processor -- start 2024-05-10T11:38:40.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.250+0000 7f06bc4abb80 1 -- start start 2024-05-10T11:38:40.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.251+0000 7f06bc4abb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b098c94d0 0x562b098c98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:40.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.251+0000 7f06bc4abb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562b098c9df0 0x562b098cd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:40.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.251+0000 7f06bc4abb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b098cdc80 0x562b098ce100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:40.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.251+0000 7f06aaffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562b098c9df0 0x562b098cd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:40.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.251+0000 7f06bc4abb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x562b0981b860 con 0x562b098cdc80 2024-05-10T11:38:40.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.251+0000 7f06aaffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562b098c9df0 0x562b098cd6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:54586/0 (socket says 172.21.15.100:54586) 2024-05-10T11:38:40.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.251+0000 7f06aaffd700 1 -- 172.21.15.100:0/1950535554 learned_addr learned my addr 172.21.15.100:0/1950535554 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:38:40.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.251+0000 7f06bc4abb80 1 -- 172.21.15.100:0/1950535554 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562b097ba7c0 con 0x562b098c9df0 2024-05-10T11:38:40.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.251+0000 7f06bc4abb80 1 -- 172.21.15.100:0/1950535554 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x562b09817820 con 0x562b098c94d0 2024-05-10T11:38:40.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.251+0000 7f06ab7fe700 1 --2- 172.21.15.100:0/1950535554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b098c94d0 0x562b098c98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:40.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.251+0000 7f06abfff700 1 --2- 172.21.15.100:0/1950535554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b098cdc80 0x562b098ce100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:40.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.251+0000 7f06ab7fe700 1 -- 172.21.15.100:0/1950535554 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562b098c9df0 msgr2=0x562b098cd6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:40.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.251+0000 7f06ab7fe700 1 --2- 172.21.15.100:0/1950535554 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562b098c9df0 0x562b098cd6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:40.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.251+0000 7f06ab7fe700 1 -- 172.21.15.100:0/1950535554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b098cdc80 msgr2=0x562b098ce100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:40.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.251+0000 7f06ab7fe700 1 --2- 172.21.15.100:0/1950535554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b098cdc80 0x562b098ce100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:40.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.251+0000 7f06ab7fe700 1 -- 172.21.15.100:0/1950535554 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b097688d0 con 0x562b098c94d0 2024-05-10T11:38:40.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.252+0000 7f06ab7fe700 1 --2- 172.21.15.100:0/1950535554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b098c94d0 0x562b098c98b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f069c011ab0 tx=0x7f069c014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a41ec4352f73fc9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:40.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.252+0000 7f0691ffb700 1 -- 172.21.15.100:0/1950535554 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069c028480 con 0x562b098c94d0 2024-05-10T11:38:40.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.252+0000 7f0691ffb700 1 -- 172.21.15.100:0/1950535554 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f069c028ab0 con 0x562b098c94d0 2024-05-10T11:38:40.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.252+0000 7f0691ffb700 1 -- 172.21.15.100:0/1950535554 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069c01d980 con 0x562b098c94d0 2024-05-10T11:38:40.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.253+0000 7f06bc4abb80 1 -- 172.21.15.100:0/1950535554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b098c94d0 msgr2=0x562b098c98b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:40.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.253+0000 7f06bc4abb80 1 --2- 172.21.15.100:0/1950535554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b098c94d0 0x562b098c98b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f069c011ab0 tx=0x7f069c014000 comp rx=0 tx=0).stop 2024-05-10T11:38:40.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.253+0000 7f06bc4abb80 1 -- 172.21.15.100:0/1950535554 shutdown_connections 2024-05-10T11:38:40.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.253+0000 7f06bc4abb80 1 --2- 172.21.15.100:0/1950535554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b098cdc80 0x562b098ce100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:40.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.253+0000 7f06bc4abb80 1 --2- 172.21.15.100:0/1950535554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b098c94d0 0x562b098c98b0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:40.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.253+0000 7f06bc4abb80 1 --2- 172.21.15.100:0/1950535554 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562b098c9df0 0x562b098cd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:40.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.253+0000 7f06bc4abb80 1 -- 172.21.15.100:0/1950535554 >> 172.21.15.100:0/1950535554 conn(0x562b097f5100 msgr2=0x562b098c3c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:40.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.253+0000 7f06bc4abb80 1 -- 172.21.15.100:0/1950535554 shutdown_connections 2024-05-10T11:38:40.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.253+0000 7f06bc4abb80 1 -- 172.21.15.100:0/1950535554 wait complete. 2024-05-10T11:38:40.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.253+0000 7f06bc4abb80 1 -- start start 2024-05-10T11:38:40.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.253+0000 7f06bc4abb80 1 Processor -- start 2024-05-10T11:38:40.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.254+0000 7f06bc4abb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b098c94d0 0x562b098d5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:40.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.254+0000 7f06bc4abb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b098c9df0 0x562b098d2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:40.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.254+0000 7f06ab7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b098c94d0 0x562b098d5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:40.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.254+0000 7f06aaffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b098c9df0 0x562b098d2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:40.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.254+0000 7f06ab7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b098c94d0 0x562b098d5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:37564/0 (socket says 172.21.15.100:37564) 2024-05-10T11:38:40.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.254+0000 7f06aaffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b098c9df0 0x562b098d2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34504/0 (socket says 172.21.15.100:34504) 2024-05-10T11:38:40.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.254+0000 7f06bc4abb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562b098cdc80 0x562b098d2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:40.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.254+0000 7f06ab7fe700 1 -- 172.21.15.100:0/2479405436 learned_addr learned my addr 172.21.15.100:0/2479405436 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:38:40.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.254+0000 7f06bc4abb80 1 -- 172.21.15.100:0/2479405436 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x562b098d5e90 con 0x562b098c9df0 2024-05-10T11:38:40.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.254+0000 7f06bc4abb80 1 -- 172.21.15.100:0/2479405436 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562b09781c30 con 0x562b098cdc80 2024-05-10T11:38:40.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.254+0000 7f06bc4abb80 1 -- 172.21.15.100:0/2479405436 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x562b098d6010 con 0x562b098c94d0 2024-05-10T11:38:40.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.254+0000 7f06bc4abb80 10 client.0 _ll_register_callbacks cb 0x562b097b3920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:38:40.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.254+0000 7f06abfff700 1 --2- 172.21.15.100:0/2479405436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562b098cdc80 0x562b098d2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:40.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.254+0000 7f06ab7fe700 1 -- 172.21.15.100:0/2479405436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562b098cdc80 msgr2=0x562b098d2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:40.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.254+0000 7f06ab7fe700 1 --2- 172.21.15.100:0/2479405436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562b098cdc80 0x562b098d2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:40.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.254+0000 7f06ab7fe700 1 -- 172.21.15.100:0/2479405436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b098c9df0 msgr2=0x562b098d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:40.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.254+0000 7f06ab7fe700 1 --2- 172.21.15.100:0/2479405436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b098c9df0 0x562b098d2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:40.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.254+0000 7f06ab7fe700 1 -- 172.21.15.100:0/2479405436 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b097b7690 con 0x562b098c94d0 2024-05-10T11:38:40.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.255+0000 7f06abfff700 1 --2- 172.21.15.100:0/2479405436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562b098cdc80 0x562b098d2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:38:40.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.255+0000 7f06aaffd700 1 --2- 172.21.15.100:0/2479405436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b098c9df0 0x562b098d2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:38:40.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.255+0000 7f06ab7fe700 1 --2- 172.21.15.100:0/2479405436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b098c94d0 0x562b098d5180 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f069c009ee0 tx=0x7f069c009bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9625bbe336101fd2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:40.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.255+0000 7f068affd700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:38:40.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.255+0000 7f068affd700 1 -- 172.21.15.100:0/2479405436 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069c005240 con 0x562b098c94d0 2024-05-10T11:38:40.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.255+0000 7f068affd700 1 -- 172.21.15.100:0/2479405436 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f069c0053e0 con 0x562b098c94d0 2024-05-10T11:38:40.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.255+0000 7f06bc4abb80 1 -- 172.21.15.100:0/2479405436 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f069c028ab0 con 0x562b098c94d0 2024-05-10T11:38:40.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.255+0000 7f068affd700 1 -- 172.21.15.100:0/2479405436 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f069c03b050 con 0x562b098c94d0 2024-05-10T11:38:40.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.255+0000 7f06bc4abb80 10 client.28810 fetch_fsmap learned FSMap version 41 2024-05-10T11:38:40.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.255+0000 7f06bc4abb80 1 -- 172.21.15.100:0/2479405436 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x562b098e0770 con 0x562b098c94d0 2024-05-10T11:38:40.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.256+0000 7f068affd700 1 -- 172.21.15.100:0/2479405436 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f069c028ab0 con 0x562b098c94d0 2024-05-10T11:38:40.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.256+0000 7f068affd700 1 -- 172.21.15.100:0/2479405436 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f069c01d5c0 con 0x562b098c94d0 2024-05-10T11:38:40.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.256+0000 7f06bc4abb80 10 client.28810 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:38:40.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.256+0000 7f06bc4abb80 10 client.28810 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:38:40.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.256+0000 7f06bc4abb80 20 client.28810 populate_metadata read hostname 'smithi100' 2024-05-10T11:38:40.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.256+0000 7f06bc4abb80 1 --2- 172.21.15.100:0/2479405436 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562b098e2d50 0x562b098e5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:40.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.256+0000 7f06bc4abb80 4 client.28810 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:38:40.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.256+0000 7f06bc4abb80 1 -- 172.21.15.100:0/2479405436 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b09895650 con 0x562b098e2d50 2024-05-10T11:38:40.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.256+0000 7f06aaffd700 1 --2- 172.21.15.100:0/2479405436 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562b098e2d50 0x562b098e5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:40.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.256+0000 7f06aaffd700 1 --2- 172.21.15.100:0/2479405436 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562b098e2d50 0x562b098e5140 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=877ca4387212b85a server_cookie=14b334382f8658b8 in_seq=0 out_seq=0 2024-05-10T11:38:40.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.256+0000 7f068affd700 10 client.28810 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:38:40.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.257+0000 7f068affd700 1 -- 172.21.15.100:0/2479405436 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562b09895650 con 0x562b098e2d50 2024-05-10T11:38:40.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.257+0000 7f068affd700 10 client.28810 handle_command_reply: tid=0 2024-05-10T11:38:40.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.257+0000 7f06bc4abb80 1 client.28810 shutdown 2024-05-10T11:38:40.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.257+0000 7f06bc4abb80 1 -- 172.21.15.100:0/2479405436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b098c94d0 msgr2=0x562b098d5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:40.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.257+0000 7f06bc4abb80 1 --2- 172.21.15.100:0/2479405436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b098c94d0 0x562b098d5180 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f069c009ee0 tx=0x7f069c009bc0 comp rx=0 tx=0).stop 2024-05-10T11:38:40.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.257+0000 7f06bc4abb80 1 -- 172.21.15.100:0/2479405436 shutdown_connections 2024-05-10T11:38:40.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.257+0000 7f06bc4abb80 1 --2- 172.21.15.100:0/2479405436 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562b098e2d50 0x562b098e5140 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:40.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.257+0000 7f068affd700 0 client.28810 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:38:40.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.258+0000 7f06bc4abb80 1 --2- 172.21.15.100:0/2479405436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562b098cdc80 0x562b098d2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:40.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.258+0000 7f06bc4abb80 1 --2- 172.21.15.100:0/2479405436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b098c94d0 0x562b098d5180 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:40.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.258+0000 7f06bc4abb80 1 --2- 172.21.15.100:0/2479405436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b098c9df0 0x562b098d2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:40.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.258+0000 7f06bc4abb80 1 -- 172.21.15.100:0/2479405436 >> 172.21.15.100:0/2479405436 conn(0x562b097f5100 msgr2=0x562b098be720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:40.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.258+0000 7f06bc4abb80 1 -- 172.21.15.100:0/2479405436 shutdown_connections 2024-05-10T11:38:40.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.258+0000 7f06bc4abb80 1 -- 172.21.15.100:0/2479405436 wait complete. 2024-05-10T11:38:40.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.258+0000 7f06bc4abb80 20 client.28810 trim_cache size 0 max 16384 2024-05-10T11:38:40.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.288+0000 7f06bc4abb80 1 Processor -- start 2024-05-10T11:38:40.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.288+0000 7f06bc4abb80 1 -- start start 2024-05-10T11:38:40.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.288+0000 7f06bc4abb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b097f5100 0x562b098c89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:40.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.288+0000 7f06bc4abb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b098c94d0 0x562b098d3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:40.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.288+0000 7f06bc4abb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562b098c9df0 0x562b098d4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:40.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.288+0000 7f06bc4abb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f06ac1b7df0 con 0x562b097f5100 2024-05-10T11:38:40.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.288+0000 7f06bc4abb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f06ac1b5680 con 0x562b098c9df0 2024-05-10T11:38:40.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.288+0000 7f06bc4abb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f06ac1b34d0 con 0x562b098c94d0 2024-05-10T11:38:40.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.288+0000 7f06ab7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b097f5100 0x562b098c89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:40.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.288+0000 7f06ab7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b097f5100 0x562b098c89f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34520/0 (socket says 172.21.15.100:34520) 2024-05-10T11:38:40.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.288+0000 7f06ab7fe700 1 -- 172.21.15.100:0/222680347 learned_addr learned my addr 172.21.15.100:0/222680347 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:38:40.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.288+0000 7f06aaffd700 1 --2- 172.21.15.100:0/222680347 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b098c94d0 0x562b098d3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:40.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.288+0000 7f06abfff700 1 --2- 172.21.15.100:0/222680347 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562b098c9df0 0x562b098d4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:40.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.289+0000 7f06ab7fe700 1 -- 172.21.15.100:0/222680347 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b098c94d0 msgr2=0x562b098d3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:40.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.289+0000 7f06ab7fe700 1 --2- 172.21.15.100:0/222680347 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b098c94d0 0x562b098d3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:40.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.289+0000 7f06ab7fe700 1 -- 172.21.15.100:0/222680347 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562b098c9df0 msgr2=0x562b098d4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:40.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.289+0000 7f06ab7fe700 1 --2- 172.21.15.100:0/222680347 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562b098c9df0 0x562b098d4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:40.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.289+0000 7f06ab7fe700 1 -- 172.21.15.100:0/222680347 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b098c9b00 con 0x562b097f5100 2024-05-10T11:38:40.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.289+0000 7f06ab7fe700 1 --2- 172.21.15.100:0/222680347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b097f5100 0x562b098c89f0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f069c016040 tx=0x7f069c0285b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d9c9a8e925ceee6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:40.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.289+0000 7f06917fa700 1 -- 172.21.15.100:0/222680347 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069c01d6d0 con 0x562b097f5100 2024-05-10T11:38:40.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.289+0000 7f06917fa700 1 -- 172.21.15.100:0/222680347 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f069c0054d0 con 0x562b097f5100 2024-05-10T11:38:40.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.289+0000 7f06917fa700 1 -- 172.21.15.100:0/222680347 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069c004c20 con 0x562b097f5100 2024-05-10T11:38:40.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.290+0000 7f06bc4abb80 1 -- 172.21.15.100:0/222680347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b097f5100 msgr2=0x562b098c89f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:40.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.290+0000 7f06bc4abb80 1 --2- 172.21.15.100:0/222680347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b097f5100 0x562b098c89f0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f069c016040 tx=0x7f069c0285b0 comp rx=0 tx=0).stop 2024-05-10T11:38:40.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.290+0000 7f06bc4abb80 1 -- 172.21.15.100:0/222680347 shutdown_connections 2024-05-10T11:38:40.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.290+0000 7f06bc4abb80 1 --2- 172.21.15.100:0/222680347 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562b098c9df0 0x562b098d4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:40.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.290+0000 7f06bc4abb80 1 --2- 172.21.15.100:0/222680347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b097f5100 0x562b098c89f0 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:40.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.290+0000 7f06bc4abb80 1 --2- 172.21.15.100:0/222680347 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b098c94d0 0x562b098d3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:40.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.290+0000 7f06bc4abb80 1 -- 172.21.15.100:0/222680347 >> 172.21.15.100:0/222680347 conn(0x562b098beab0 msgr2=0x562b098c82e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:40.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.290+0000 7f06bc4abb80 1 -- 172.21.15.100:0/222680347 shutdown_connections 2024-05-10T11:38:40.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.291+0000 7f06bc4abb80 1 -- 172.21.15.100:0/222680347 wait complete. 2024-05-10T11:38:40.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.291+0000 7f06bc4abb80 1 -- start start 2024-05-10T11:38:40.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.291+0000 7f06bc4abb80 1 Processor -- start 2024-05-10T11:38:40.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.292+0000 7f06bc4abb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b097f5100 0x562b098e7690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:40.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.292+0000 7f06bc4abb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b098c94d0 0x562b098e7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:40.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.292+0000 7f06ab7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b097f5100 0x562b098e7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:40.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.292+0000 7f06ab7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b097f5100 0x562b098e7690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:37592/0 (socket says 172.21.15.100:37592) 2024-05-10T11:38:40.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.292+0000 7f06ab7fe700 1 -- 172.21.15.100:0/3882231394 learned_addr learned my addr 172.21.15.100:0/3882231394 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:38:40.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.292+0000 7f06bc4abb80 1 --2- 172.21.15.100:0/3882231394 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562b098c9df0 0x562b099452b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:40.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.292+0000 7f06bc4abb80 1 -- 172.21.15.100:0/3882231394 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f06ac1b3b40 con 0x562b098c94d0 2024-05-10T11:38:40.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.292+0000 7f06bc4abb80 1 -- 172.21.15.100:0/3882231394 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f06ac1b4fb0 con 0x562b098c9df0 2024-05-10T11:38:40.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.292+0000 7f06bc4abb80 1 -- 172.21.15.100:0/3882231394 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f06ac1b4210 con 0x562b097f5100 2024-05-10T11:38:40.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.292+0000 7f06bc4abb80 10 client.0 _ll_register_callbacks cb 0x562b097b3920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:38:40.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.292+0000 7f06aaffd700 1 --2- 172.21.15.100:0/3882231394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b098c94d0 0x562b098e7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:40.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.292+0000 7f06abfff700 1 --2- 172.21.15.100:0/3882231394 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562b098c9df0 0x562b099452b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:40.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.293+0000 7f06ab7fe700 1 -- 172.21.15.100:0/3882231394 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562b098c9df0 msgr2=0x562b099452b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:40.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.293+0000 7f06ab7fe700 1 --2- 172.21.15.100:0/3882231394 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562b098c9df0 0x562b099452b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:40.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.293+0000 7f06ab7fe700 1 -- 172.21.15.100:0/3882231394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b098c94d0 msgr2=0x562b098e7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:40.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.293+0000 7f06ab7fe700 1 --2- 172.21.15.100:0/3882231394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b098c94d0 0x562b098e7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:40.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.293+0000 7f06ab7fe700 1 -- 172.21.15.100:0/3882231394 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b09945920 con 0x562b097f5100 2024-05-10T11:38:40.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.293+0000 7f06aaffd700 1 --2- 172.21.15.100:0/3882231394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b098c94d0 0x562b098e7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:38:40.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.293+0000 7f06ab7fe700 1 --2- 172.21.15.100:0/3882231394 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b097f5100 0x562b098e7690 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f069c01cf50 tx=0x7f069c005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c307f377722e73 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:40.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.293+0000 7f068affd700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:38:40.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.293+0000 7f068affd700 1 -- 172.21.15.100:0/3882231394 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069c044070 con 0x562b097f5100 2024-05-10T11:38:40.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.293+0000 7f068affd700 1 -- 172.21.15.100:0/3882231394 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f069c0288c0 con 0x562b097f5100 2024-05-10T11:38:40.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.293+0000 7f06bc4abb80 1 -- 172.21.15.100:0/3882231394 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f069c0054d0 con 0x562b097f5100 2024-05-10T11:38:40.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.293+0000 7f068affd700 1 -- 172.21.15.100:0/3882231394 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f069c050050 con 0x562b097f5100 2024-05-10T11:38:40.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.293+0000 7f06bc4abb80 10 client.28816 fetch_fsmap learned FSMap version 41 2024-05-10T11:38:40.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.293+0000 7f06bc4abb80 1 -- 172.21.15.100:0/3882231394 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x562b0994ccf0 con 0x562b097f5100 2024-05-10T11:38:40.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.294+0000 7f068affd700 1 -- 172.21.15.100:0/3882231394 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f069c0054d0 con 0x562b097f5100 2024-05-10T11:38:40.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.294+0000 7f068affd700 1 -- 172.21.15.100:0/3882231394 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f069c004a80 con 0x562b097f5100 2024-05-10T11:38:40.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.294+0000 7f06bc4abb80 10 client.28816 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:38:40.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.294+0000 7f06bc4abb80 10 client.28816 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:38:40.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.294+0000 7f06bc4abb80 20 client.28816 populate_metadata read hostname 'smithi100' 2024-05-10T11:38:40.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.294+0000 7f06bc4abb80 1 --2- 172.21.15.100:0/3882231394 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562b098cdc80 0x562b0994f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:40.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.294+0000 7f06bc4abb80 4 client.28816 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T11:38:40.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.294+0000 7f06bc4abb80 1 -- 172.21.15.100:0/3882231394 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562b09829240 con 0x562b098cdc80 2024-05-10T11:38:40.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.294+0000 7f06aaffd700 1 --2- 172.21.15.100:0/3882231394 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562b098cdc80 0x562b0994f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:40.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.295+0000 7f06aaffd700 1 --2- 172.21.15.100:0/3882231394 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562b098cdc80 0x562b0994f2f0 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2c5707a15a4ad server_cookie=7760aba0f9c6c23c in_seq=0 out_seq=0 2024-05-10T11:38:40.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.295+0000 7f068affd700 10 client.28816 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:38:40.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.295+0000 7f068affd700 1 -- 172.21.15.100:0/3882231394 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x562b09829240 con 0x562b098cdc80 2024-05-10T11:38:40.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.295+0000 7f068affd700 10 client.28816 handle_command_reply: tid=0 2024-05-10T11:38:40.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.295+0000 7f06bc4abb80 1 client.28816 shutdown 2024-05-10T11:38:40.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.295+0000 7f06bc4abb80 1 -- 172.21.15.100:0/3882231394 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b097f5100 msgr2=0x562b098e7690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:40.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.295+0000 7f06bc4abb80 1 --2- 172.21.15.100:0/3882231394 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b097f5100 0x562b098e7690 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f069c01cf50 tx=0x7f069c005270 comp rx=0 tx=0).stop 2024-05-10T11:38:40.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.295+0000 7f06bc4abb80 1 -- 172.21.15.100:0/3882231394 shutdown_connections 2024-05-10T11:38:40.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.295+0000 7f06bc4abb80 1 --2- 172.21.15.100:0/3882231394 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562b098cdc80 0x562b0994f2f0 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:40.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.295+0000 7f06bc4abb80 1 --2- 172.21.15.100:0/3882231394 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562b098c9df0 0x562b099452b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:40.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.295+0000 7f06bc4abb80 1 --2- 172.21.15.100:0/3882231394 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b097f5100 0x562b098e7690 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:40.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.295+0000 7f06bc4abb80 1 --2- 172.21.15.100:0/3882231394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b098c94d0 0x562b098e7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:40.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.295+0000 7f06bc4abb80 1 -- 172.21.15.100:0/3882231394 >> 172.21.15.100:0/3882231394 conn(0x562b098beab0 msgr2=0x562b098c8b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:40.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.296+0000 7f068affd700 0 client.28816 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:38:40.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.296+0000 7f06bc4abb80 1 -- 172.21.15.100:0/3882231394 shutdown_connections 2024-05-10T11:38:40.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.296+0000 7f06bc4abb80 1 -- 172.21.15.100:0/3882231394 wait complete. 2024-05-10T11:38:40.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.296+0000 7f06bc4abb80 20 client.28816 trim_cache size 0 max 16384 2024-05-10T11:38:40.300 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:38:40.300 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T11:38:40.300 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-05-10T11:38:40.301 INFO:teuthology.orchestra.run.smithi100.stdout: "a14e2bed-1997-4226-8cb0-6cb7eaad5456": { 2024-05-10T11:38:40.301 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-05-10T11:38:40.301 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "a14e2bed-1997-4226-8cb0-6cb7eaad5456", 2024-05-10T11:38:40.301 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-05-10T11:38:40.301 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:38:40.301 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:38:40.301 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:38:40.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.297+0000 7f06b1c18700 1 -- 172.21.15.100:0/3016337083 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f06940b2900 msgr2=0x7f06940b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:40.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.297+0000 7f06b1c18700 1 --2- 172.21.15.100:0/3016337083 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f06940b2900 0x7f06940b4dc0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f069c009d60 tx=0x7f069c009260 comp rx=0 tx=0).stop 2024-05-10T11:38:40.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.297+0000 7f06b1c18700 1 -- 172.21.15.100:0/3016337083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06ac104100 msgr2=0x7f06ac1a0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:40.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.297+0000 7f06b1c18700 1 --2- 172.21.15.100:0/3016337083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06ac104100 0x7f06ac1a0660 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f06a00098e0 tx=0x7f06a000bf60 comp rx=0 tx=0).stop 2024-05-10T11:38:40.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.297+0000 7f06b1c18700 1 -- 172.21.15.100:0/3016337083 shutdown_connections 2024-05-10T11:38:40.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.297+0000 7f06b1c18700 1 --2- 172.21.15.100:0/3016337083 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f06940b2900 0x7f06940b4dc0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:40.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.297+0000 7f06b1c18700 1 --2- 172.21.15.100:0/3016337083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06ac108850 0x7f06ac1a49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:40.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.297+0000 7f06b1c18700 1 --2- 172.21.15.100:0/3016337083 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06ac103750 0x7f06ac1a0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:40.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.297+0000 7f06b1c18700 1 --2- 172.21.15.100:0/3016337083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06ac104100 0x7f06ac1a0660 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:40.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.297+0000 7f06b1c18700 1 -- 172.21.15.100:0/3016337083 >> 172.21.15.100:0/3016337083 conn(0x7f06ac07a600 msgr2=0x7f06ac101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:40.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.297+0000 7f06b1c18700 1 -- 172.21.15.100:0/3016337083 shutdown_connections 2024-05-10T11:38:40.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:38:40.297+0000 7f06b1c18700 1 -- 172.21.15.100:0/3016337083 wait complete. 2024-05-10T11:38:40.945 INFO:tasks.workunit.client.0.smithi100.stdout: 10 65967 8.96 MB/sec execute 274 sec latency 135.549 ms 2024-05-10T11:38:41.045 INFO:tasks.cephfs.filesystem:scrub status for tag:a14e2bed-1997-4226-8cb0-6cb7eaad5456 - {'path': '/', 'tag': 'a14e2bed-1997-4226-8cb0-6cb7eaad5456', 'options': 'recursive,force'} 2024-05-10T11:38:41.755 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.754+0000 7fdfbbc99640 1 -- 172.21.15.100:0/816262615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfb4101910 msgr2=0x7fdfb4101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:41.755 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.754+0000 7fdfbbc99640 1 --2- 172.21.15.100:0/816262615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfb4101910 0x7fdfb4101d30 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0009900 tx=0x7fdfb002a410 comp rx=0 tx=0).stop 2024-05-10T11:38:41.756 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.754+0000 7fdfbbc99640 1 -- 172.21.15.100:0/816262615 shutdown_connections 2024-05-10T11:38:41.756 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.754+0000 7fdfbbc99640 1 --2- 172.21.15.100:0/816262615 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdfb41034c0 0x7fdfb4109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:41.756 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.754+0000 7fdfbbc99640 1 --2- 172.21.15.100:0/816262615 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfb4102b00 0x7fdfb4102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:41.756 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.754+0000 7fdfbbc99640 1 --2- 172.21.15.100:0/816262615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfb4101910 0x7fdfb4101d30 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:41.756 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.754+0000 7fdfbbc99640 1 -- 172.21.15.100:0/816262615 >> 172.21.15.100:0/816262615 conn(0x7fdfb40fce80 msgr2=0x7fdfb40ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:41.756 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.754+0000 7fdfbbc99640 1 -- 172.21.15.100:0/816262615 shutdown_connections 2024-05-10T11:38:41.756 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.754+0000 7fdfbbc99640 1 -- 172.21.15.100:0/816262615 wait complete. 2024-05-10T11:38:41.758 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.755+0000 7fdfbbc99640 1 Processor -- start 2024-05-10T11:38:41.758 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.755+0000 7fdfbbc99640 1 -- start start 2024-05-10T11:38:41.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.756+0000 7fdfbbc99640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfb4102b00 0x7fdfb410cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:41.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.756+0000 7fdfbbc99640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdfb41034c0 0x7fdfb410b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:41.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.756+0000 7fdfbbc99640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfb419cac0 0x7fdfb410bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:41.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.756+0000 7fdfbbc99640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb410e510 con 0x7fdfb4102b00 2024-05-10T11:38:41.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.756+0000 7fdfbbc99640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb410e390 con 0x7fdfb41034c0 2024-05-10T11:38:41.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.756+0000 7fdfbbc99640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb410e690 con 0x7fdfb419cac0 2024-05-10T11:38:41.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.756+0000 7fdfba236640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfb419cac0 0x7fdfb410bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:41.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.756+0000 7fdfba236640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfb419cac0 0x7fdfb410bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:37616/0 (socket says 172.21.15.100:37616) 2024-05-10T11:38:41.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.756+0000 7fdfba236640 1 -- 172.21.15.100:0/1340804720 learned_addr learned my addr 172.21.15.100:0/1340804720 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:38:41.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.756+0000 7fdfb9234640 1 --2- 172.21.15.100:0/1340804720 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdfb41034c0 0x7fdfb410b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:41.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.756+0000 7fdfb9a35640 1 --2- 172.21.15.100:0/1340804720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfb4102b00 0x7fdfb410cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:41.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.757+0000 7fdfba236640 1 -- 172.21.15.100:0/1340804720 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdfb41034c0 msgr2=0x7fdfb410b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:41.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.757+0000 7fdfba236640 1 --2- 172.21.15.100:0/1340804720 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdfb41034c0 0x7fdfb410b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:41.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.757+0000 7fdfba236640 1 -- 172.21.15.100:0/1340804720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfb4102b00 msgr2=0x7fdfb410cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:41.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.757+0000 7fdfba236640 1 --2- 172.21.15.100:0/1340804720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfb4102b00 0x7fdfb410cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:41.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.757+0000 7fdfba236640 1 -- 172.21.15.100:0/1340804720 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfb410c390 con 0x7fdfb419cac0 2024-05-10T11:38:41.760 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.757+0000 7fdfba236640 1 --2- 172.21.15.100:0/1340804720 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfb419cac0 0x7fdfb410bb00 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fdfac00b970 tx=0x7fdfac00bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:41.760 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.757+0000 7fdfaaffd640 1 -- 172.21.15.100:0/1340804720 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfac019070 con 0x7fdfb419cac0 2024-05-10T11:38:41.760 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.757+0000 7fdfbbc99640 1 -- 172.21.15.100:0/1340804720 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfb41a7790 con 0x7fdfb419cac0 2024-05-10T11:38:41.760 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.758+0000 7fdfaaffd640 1 -- 172.21.15.100:0/1340804720 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdfac014440 con 0x7fdfb419cac0 2024-05-10T11:38:41.760 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.758+0000 7fdfbbc99640 1 -- 172.21.15.100:0/1340804720 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfb41a7d60 con 0x7fdfb419cac0 2024-05-10T11:38:41.760 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.758+0000 7fdfaaffd640 1 -- 172.21.15.100:0/1340804720 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfac01d5b0 con 0x7fdfb419cac0 2024-05-10T11:38:41.763 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.762+0000 7fdfbe9b8740 1 Processor -- start 2024-05-10T11:38:41.765 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.762+0000 7fdfbe9b8740 1 -- start start 2024-05-10T11:38:41.765 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.762+0000 7fdfbe9b8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a7fe78bdb0 0x55a7fe78c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:41.765 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.762+0000 7fdfbe9b8740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a7fe78c7a0 0x55a7fe7912a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:41.765 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.762+0000 7fdfbe9b8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7fe791910 0x55a7fe793da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:41.765 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.762+0000 7fdfbe9b8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55a7fe6f05d0 con 0x55a7fe78bdb0 2024-05-10T11:38:41.766 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.762+0000 7fdfbe9b8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a7fe6a9890 con 0x55a7fe791910 2024-05-10T11:38:41.766 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.762+0000 7fdfbe9b8740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55a7fe6ec590 con 0x55a7fe78c7a0 2024-05-10T11:38:41.766 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.763+0000 7fdfaaffd640 1 -- 172.21.15.100:0/1340804720 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fdfac01c070 con 0x7fdfb419cac0 2024-05-10T11:38:41.766 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.763+0000 7fdfb9234640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a7fe78bdb0 0x55a7fe78c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:41.766 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.763+0000 7fdfb9234640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a7fe78bdb0 0x55a7fe78c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34556/0 (socket says 172.21.15.100:34556) 2024-05-10T11:38:41.766 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.763+0000 7fdfb9234640 1 -- 172.21.15.100:0/788664015 learned_addr learned my addr 172.21.15.100:0/788664015 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:38:41.766 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.763+0000 7fdfb9234640 1 --2- 172.21.15.100:0/788664015 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7fe791910 0x55a7fe793da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:41.766 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.764+0000 7fdfb9a35640 1 --2- 172.21.15.100:0/788664015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a7fe78c7a0 0x55a7fe7912a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:41.766 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.764+0000 7fdfaaffd640 1 --2- 172.21.15.100:0/1340804720 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fdfa00b2920 0x7fdfa00b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:41.766 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.764+0000 7fdfb9234640 1 -- 172.21.15.100:0/788664015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a7fe78c7a0 msgr2=0x55a7fe7912a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:41.766 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.764+0000 7fdfb9234640 1 --2- 172.21.15.100:0/788664015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a7fe78c7a0 0x55a7fe7912a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:41.766 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.764+0000 7fdfb9234640 1 -- 172.21.15.100:0/788664015 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7fe791910 msgr2=0x55a7fe793da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:41.766 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.764+0000 7fdfb9234640 1 --2- 172.21.15.100:0/788664015 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7fe791910 0x55a7fe793da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:41.768 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.764+0000 7fdfba236640 1 --2- 172.21.15.100:0/1340804720 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fdfa00b2920 0x7fdfa00b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:41.768 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.764+0000 7fdfb9234640 1 -- 172.21.15.100:0/788664015 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7fe402210 con 0x55a7fe78bdb0 2024-05-10T11:38:41.768 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.764+0000 7fdfaaffd640 1 -- 172.21.15.100:0/1340804720 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fdfac0e1b90 con 0x7fdfb419cac0 2024-05-10T11:38:41.768 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.765+0000 7fdfb9234640 1 --2- 172.21.15.100:0/788664015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a7fe78bdb0 0x55a7fe78c1d0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fdfa4017770 tx=0x7fdfa4009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f9c7e3257d1063c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:41.768 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.765+0000 7fdfba236640 1 --2- 172.21.15.100:0/1340804720 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fdfa00b2920 0x7fdfa00b4de0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fdfac00bfd0 tx=0x7fdfac00e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:41.768 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.765+0000 7fdf9ffff640 1 -- 172.21.15.100:0/788664015 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa4015760 con 0x55a7fe78bdb0 2024-05-10T11:38:41.769 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.765+0000 7fdf9ffff640 1 -- 172.21.15.100:0/788664015 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdfa401d070 con 0x55a7fe78bdb0 2024-05-10T11:38:41.769 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.765+0000 7fdf9ffff640 1 -- 172.21.15.100:0/788664015 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa4018a10 con 0x55a7fe78bdb0 2024-05-10T11:38:41.769 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.766+0000 7fdfbe9b8740 1 -- 172.21.15.100:0/788664015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a7fe78bdb0 msgr2=0x55a7fe78c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:41.769 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.766+0000 7fdfbe9b8740 1 --2- 172.21.15.100:0/788664015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a7fe78bdb0 0x55a7fe78c1d0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fdfa4017770 tx=0x7fdfa4009520 comp rx=0 tx=0).stop 2024-05-10T11:38:41.769 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.766+0000 7fdfbe9b8740 1 -- 172.21.15.100:0/788664015 shutdown_connections 2024-05-10T11:38:41.769 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.766+0000 7fdfbe9b8740 1 --2- 172.21.15.100:0/788664015 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7fe791910 0x55a7fe793da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:41.769 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.766+0000 7fdfbe9b8740 1 --2- 172.21.15.100:0/788664015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a7fe78c7a0 0x55a7fe7912a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:41.769 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.766+0000 7fdfbe9b8740 1 --2- 172.21.15.100:0/788664015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a7fe78bdb0 0x55a7fe78c1d0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:41.769 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.766+0000 7fdfbe9b8740 1 -- 172.21.15.100:0/788664015 >> 172.21.15.100:0/788664015 conn(0x55a7fe787420 msgr2=0x55a7fe787820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:41.769 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.767+0000 7fdfbe9b8740 1 -- 172.21.15.100:0/788664015 shutdown_connections 2024-05-10T11:38:41.769 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.767+0000 7fdfbe9b8740 1 -- 172.21.15.100:0/788664015 wait complete. 2024-05-10T11:38:41.769 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.767+0000 7fdfbe9b8740 1 -- start start 2024-05-10T11:38:41.773 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.768+0000 7fdfbe9b8740 1 Processor -- start 2024-05-10T11:38:41.773 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.768+0000 7fdfbe9b8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7fe78bdb0 0x55a7fe7a6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:41.773 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.768+0000 7fdfbe9b8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a7fe78c7a0 0x55a7fe7a74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:41.773 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.768+0000 7fdfbe9b8740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a7fe791910 0x55a7fe7ae520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:41.773 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.768+0000 7fdfbe9b8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55a7fe7a5870 con 0x55a7fe78c7a0 2024-05-10T11:38:41.774 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.768+0000 7fdfbe9b8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a7fe6ad370 con 0x55a7fe78bdb0 2024-05-10T11:38:41.774 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.768+0000 7fdfbe9b8740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55a7fe7a59f0 con 0x55a7fe791910 2024-05-10T11:38:41.774 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.768+0000 7fdfbe9b8740 10 client.0 _ll_register_callbacks cb 0x55a7fe401a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:38:41.774 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.768+0000 7fdfb9234640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7fe78bdb0 0x55a7fe7a6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:41.774 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.768+0000 7fdfb9234640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7fe78bdb0 0x55a7fe7a6fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:54646/0 (socket says 172.21.15.100:54646) 2024-05-10T11:38:41.774 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.768+0000 7fdfb9234640 1 -- 172.21.15.100:0/3119016231 learned_addr learned my addr 172.21.15.100:0/3119016231 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:38:41.774 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.768+0000 7fdfb9234640 1 --2- 172.21.15.100:0/3119016231 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a7fe791910 0x55a7fe7ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:41.774 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.769+0000 7fdfb9a35640 1 --2- 172.21.15.100:0/3119016231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a7fe78c7a0 0x55a7fe7a74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:41.774 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.769+0000 7fdfb9234640 1 -- 172.21.15.100:0/3119016231 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a7fe791910 msgr2=0x55a7fe7ae520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:41.774 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.769+0000 7fdfb9234640 1 --2- 172.21.15.100:0/3119016231 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a7fe791910 0x55a7fe7ae520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:41.774 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.769+0000 7fdfb9234640 1 -- 172.21.15.100:0/3119016231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a7fe78c7a0 msgr2=0x55a7fe7a74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:41.774 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.769+0000 7fdfb9234640 1 --2- 172.21.15.100:0/3119016231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a7fe78c7a0 0x55a7fe7a74e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:41.774 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.769+0000 7fdfb9234640 1 -- 172.21.15.100:0/3119016231 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7fe4ce620 con 0x55a7fe78bdb0 2024-05-10T11:38:41.774 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.769+0000 7fdfb9a35640 1 --2- 172.21.15.100:0/3119016231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a7fe78c7a0 0x55a7fe7a74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:38:41.774 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.769+0000 7fdfb9234640 1 --2- 172.21.15.100:0/3119016231 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7fe78bdb0 0x55a7fe7a6fa0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fdfa4009330 tx=0x7fdfa4008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=849feb5b2babece3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:41.775 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.769+0000 7fdf9d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:38:41.775 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.769+0000 7fdf9d7fa640 1 -- 172.21.15.100:0/3119016231 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa4015da0 con 0x55a7fe78bdb0 2024-05-10T11:38:41.775 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.769+0000 7fdf9d7fa640 1 -- 172.21.15.100:0/3119016231 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fdfa4018790 con 0x55a7fe78bdb0 2024-05-10T11:38:41.775 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.769+0000 7fdfbe9b8740 1 -- 172.21.15.100:0/3119016231 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfa401d070 con 0x55a7fe78bdb0 2024-05-10T11:38:41.775 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.769+0000 7fdf9d7fa640 1 -- 172.21.15.100:0/3119016231 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fdfa4033050 con 0x55a7fe78bdb0 2024-05-10T11:38:41.775 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.769+0000 7fdfbe9b8740 10 client.28718 fetch_fsmap learned FSMap version 41 2024-05-10T11:38:41.775 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.769+0000 7fdfbe9b8740 1 -- 172.21.15.100:0/3119016231 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55a7fe7b1500 con 0x55a7fe78bdb0 2024-05-10T11:38:41.775 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.770+0000 7fdf9d7fa640 1 -- 172.21.15.100:0/3119016231 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdfa401d070 con 0x55a7fe78bdb0 2024-05-10T11:38:41.775 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.770+0000 7fdf9d7fa640 1 -- 172.21.15.100:0/3119016231 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fdfa402e030 con 0x55a7fe78bdb0 2024-05-10T11:38:41.775 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.770+0000 7fdfbe9b8740 10 client.28718 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:38:41.775 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.770+0000 7fdfbe9b8740 10 client.28718 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:38:41.775 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.770+0000 7fdfbe9b8740 20 client.28718 populate_metadata read hostname 'smithi100' 2024-05-10T11:38:41.775 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.770+0000 7fdfbe9b8740 1 --2- 172.21.15.100:0/3119016231 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55a7fe7b2910 0x55a7fe7b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:41.775 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.770+0000 7fdfbe9b8740 4 client.28718 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:38:41.775 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.770+0000 7fdfbe9b8740 1 -- 172.21.15.100:0/3119016231 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7fe6fe1e0 con 0x55a7fe7b2910 2024-05-10T11:38:41.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.770+0000 7fdfb9a35640 1 --2- 172.21.15.100:0/3119016231 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55a7fe7b2910 0x55a7fe7b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:41.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.771+0000 7fdf9d7fa640 10 client.28718 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:38:41.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.771+0000 7fdfb9a35640 1 --2- 172.21.15.100:0/3119016231 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55a7fe7b2910 0x55a7fe7b4d00 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1de482f613125f12 server_cookie=57a5bc73255e58 in_seq=0 out_seq=0 2024-05-10T11:38:41.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.771+0000 7fdf9d7fa640 1 -- 172.21.15.100:0/3119016231 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a7fe6fe1e0 con 0x55a7fe7b2910 2024-05-10T11:38:41.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.771+0000 7fdf9d7fa640 10 client.28718 handle_command_reply: tid=0 2024-05-10T11:38:41.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.771+0000 7fdfbe9b8740 1 client.28718 shutdown 2024-05-10T11:38:41.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.772+0000 7fdfbe9b8740 1 -- 172.21.15.100:0/3119016231 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7fe78bdb0 msgr2=0x55a7fe7a6fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:41.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.772+0000 7fdfbe9b8740 1 --2- 172.21.15.100:0/3119016231 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7fe78bdb0 0x55a7fe7a6fa0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fdfa4009330 tx=0x7fdfa4008040 comp rx=0 tx=0).stop 2024-05-10T11:38:41.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.772+0000 7fdfbe9b8740 1 -- 172.21.15.100:0/3119016231 shutdown_connections 2024-05-10T11:38:41.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.772+0000 7fdfbe9b8740 1 --2- 172.21.15.100:0/3119016231 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55a7fe7b2910 0x55a7fe7b4d00 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:41.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.772+0000 7fdf9d7fa640 0 client.28718 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:38:41.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.772+0000 7fdfbe9b8740 1 --2- 172.21.15.100:0/3119016231 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a7fe791910 0x55a7fe7ae520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:41.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.772+0000 7fdfbe9b8740 1 --2- 172.21.15.100:0/3119016231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a7fe78c7a0 0x55a7fe7a74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:41.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.772+0000 7fdfbe9b8740 1 --2- 172.21.15.100:0/3119016231 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7fe78bdb0 0x55a7fe7a6fa0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:41.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.772+0000 7fdfbe9b8740 1 -- 172.21.15.100:0/3119016231 >> 172.21.15.100:0/3119016231 conn(0x55a7fe787420 msgr2=0x55a7fe786580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:41.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.772+0000 7fdfbe9b8740 1 -- 172.21.15.100:0/3119016231 shutdown_connections 2024-05-10T11:38:41.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.772+0000 7fdfbe9b8740 1 -- 172.21.15.100:0/3119016231 wait complete. 2024-05-10T11:38:41.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.772+0000 7fdfbe9b8740 20 client.28718 trim_cache size 0 max 16384 2024-05-10T11:38:41.794 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.792+0000 7fdfbe9b8740 1 Processor -- start 2024-05-10T11:38:41.794 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.792+0000 7fdfbe9b8740 1 -- start start 2024-05-10T11:38:41.794 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.792+0000 7fdfbe9b8740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a7fe78bdb0 0x55a7fe7823e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:41.794 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.792+0000 7fdfbe9b8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a7fe78c7a0 0x55a7fe7a6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.792+0000 7fdfbe9b8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7fe791910 0x55a7fe7a70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.792+0000 7fdfbe9b8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb41ac250 con 0x55a7fe78c7a0 2024-05-10T11:38:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.792+0000 7fdfbe9b8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb41a9b20 con 0x55a7fe791910 2024-05-10T11:38:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.792+0000 7fdfbe9b8740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb41a5060 con 0x55a7fe78bdb0 2024-05-10T11:38:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.792+0000 7fdfb9a35640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a7fe78c7a0 0x55a7fe7a6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.792+0000 7fdfb9a35640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a7fe78c7a0 0x55a7fe7a6bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34576/0 (socket says 172.21.15.100:34576) 2024-05-10T11:38:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.792+0000 7fdfb9a35640 1 -- 172.21.15.100:0/505269673 learned_addr learned my addr 172.21.15.100:0/505269673 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:38:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.792+0000 7fdfb9234640 1 --2- 172.21.15.100:0/505269673 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a7fe78bdb0 0x55a7fe7823e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.793+0000 7fdfb9234640 1 --2- 172.21.15.100:0/505269673 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7fe791910 0x55a7fe7a70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.793+0000 7fdfb9a35640 1 -- 172.21.15.100:0/505269673 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a7fe78bdb0 msgr2=0x55a7fe7823e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.793+0000 7fdfb9a35640 1 --2- 172.21.15.100:0/505269673 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a7fe78bdb0 0x55a7fe7823e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.793+0000 7fdfb9a35640 1 -- 172.21.15.100:0/505269673 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7fe791910 msgr2=0x55a7fe7a70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.793+0000 7fdfb9a35640 1 --2- 172.21.15.100:0/505269673 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7fe791910 0x55a7fe7a70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.793+0000 7fdfb9a35640 1 -- 172.21.15.100:0/505269673 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7fe787250 con 0x55a7fe78c7a0 2024-05-10T11:38:41.796 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.793+0000 7fdfb9a35640 1 --2- 172.21.15.100:0/505269673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a7fe78c7a0 0x55a7fe7a6bb0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fdfb00057c0 tx=0x7fdfb0031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e623d6b6dd413fb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:41.796 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.793+0000 7fdf9f7fe640 1 -- 172.21.15.100:0/505269673 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb0035040 con 0x55a7fe78c7a0 2024-05-10T11:38:41.796 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.793+0000 7fdf9f7fe640 1 -- 172.21.15.100:0/505269673 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdfb00447a0 con 0x55a7fe78c7a0 2024-05-10T11:38:41.796 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.793+0000 7fdf9f7fe640 1 -- 172.21.15.100:0/505269673 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb0003c30 con 0x55a7fe78c7a0 2024-05-10T11:38:41.799 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.794+0000 7fdfbe9b8740 1 -- 172.21.15.100:0/505269673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a7fe78c7a0 msgr2=0x55a7fe7a6bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:41.799 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.794+0000 7fdfbe9b8740 1 --2- 172.21.15.100:0/505269673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a7fe78c7a0 0x55a7fe7a6bb0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fdfb00057c0 tx=0x7fdfb0031400 comp rx=0 tx=0).stop 2024-05-10T11:38:41.799 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.794+0000 7fdfbe9b8740 1 -- 172.21.15.100:0/505269673 shutdown_connections 2024-05-10T11:38:41.799 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.794+0000 7fdfbe9b8740 1 --2- 172.21.15.100:0/505269673 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7fe791910 0x55a7fe7a70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:41.799 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.794+0000 7fdfbe9b8740 1 --2- 172.21.15.100:0/505269673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a7fe78c7a0 0x55a7fe7a6bb0 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:41.799 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.794+0000 7fdfbe9b8740 1 --2- 172.21.15.100:0/505269673 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a7fe78bdb0 0x55a7fe7823e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:41.800 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.794+0000 7fdfbe9b8740 1 -- 172.21.15.100:0/505269673 >> 172.21.15.100:0/505269673 conn(0x55a7fe787420 msgr2=0x55a7fe78ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:41.800 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.794+0000 7fdfbe9b8740 1 -- 172.21.15.100:0/505269673 shutdown_connections 2024-05-10T11:38:41.800 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.794+0000 7fdfbe9b8740 1 -- 172.21.15.100:0/505269673 wait complete. 2024-05-10T11:38:41.800 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.794+0000 7fdfbe9b8740 1 -- start start 2024-05-10T11:38:41.800 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.794+0000 7fdfbe9b8740 1 Processor -- start 2024-05-10T11:38:41.800 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.795+0000 7fdfbe9b8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7fe78bdb0 0x55a7fe7aaaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:41.800 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.795+0000 7fdfbe9b8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a7fe78c7a0 0x55a7fe7ab030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:41.800 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.795+0000 7fdfbe9b8740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a7fe791910 0x55a7fe81b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:41.800 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.795+0000 7fdfb9234640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7fe78bdb0 0x55a7fe7aaaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:41.800 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.795+0000 7fdfbe9b8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb41a8010 con 0x55a7fe78c7a0 2024-05-10T11:38:41.800 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.795+0000 7fdfbe9b8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb41a9450 con 0x55a7fe78bdb0 2024-05-10T11:38:41.800 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.795+0000 7fdfb9234640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7fe78bdb0 0x55a7fe7aaaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:54662/0 (socket says 172.21.15.100:54662) 2024-05-10T11:38:41.800 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.795+0000 7fdfb9234640 1 -- 172.21.15.100:0/3283313845 learned_addr learned my addr 172.21.15.100:0/3283313845 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:38:41.800 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.795+0000 7fdfbe9b8740 1 -- 172.21.15.100:0/3283313845 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb41a86b0 con 0x55a7fe791910 2024-05-10T11:38:41.801 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.795+0000 7fdfb9a35640 1 --2- 172.21.15.100:0/3283313845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a7fe78c7a0 0x55a7fe7ab030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:41.801 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.796+0000 7fdfbe9b8740 10 client.0 _ll_register_callbacks cb 0x7fdfa4009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:38:41.801 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.796+0000 7fdfb9a35640 1 -- 172.21.15.100:0/3283313845 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a7fe791910 msgr2=0x55a7fe81b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:41.801 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.796+0000 7fdfb9a35640 1 --2- 172.21.15.100:0/3283313845 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a7fe791910 0x55a7fe81b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:41.801 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.796+0000 7fdfb9a35640 1 -- 172.21.15.100:0/3283313845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7fe78bdb0 msgr2=0x55a7fe7aaaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:41.801 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.796+0000 7fdfb9a35640 1 --2- 172.21.15.100:0/3283313845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7fe78bdb0 0x55a7fe7aaaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:41.801 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.796+0000 7fdfb9a35640 1 -- 172.21.15.100:0/3283313845 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7fe81b600 con 0x55a7fe78c7a0 2024-05-10T11:38:41.801 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.796+0000 7fdfb9a35640 1 --2- 172.21.15.100:0/3283313845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a7fe78c7a0 0x55a7fe7ab030 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0009850 tx=0x7fdfb0003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28bc36ff168d0fcb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:41.801 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.796+0000 7fdf9d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:38:41.801 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.796+0000 7fdf9d7fa640 1 -- 172.21.15.100:0/3283313845 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb0054060 con 0x55a7fe78c7a0 2024-05-10T11:38:41.801 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.796+0000 7fdf9d7fa640 1 -- 172.21.15.100:0/3283313845 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fdfb0044310 con 0x55a7fe78c7a0 2024-05-10T11:38:41.801 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.796+0000 7fdfbe9b8740 1 -- 172.21.15.100:0/3283313845 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfb00447a0 con 0x55a7fe78c7a0 2024-05-10T11:38:41.801 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.797+0000 7fdf9d7fa640 1 -- 172.21.15.100:0/3283313845 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fdfb0035030 con 0x55a7fe78c7a0 2024-05-10T11:38:41.801 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.797+0000 7fdfbe9b8740 10 client.19452 fetch_fsmap learned FSMap version 41 2024-05-10T11:38:41.802 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.797+0000 7fdfbe9b8740 1 -- 172.21.15.100:0/3283313845 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55a7fe81c810 con 0x55a7fe78c7a0 2024-05-10T11:38:41.802 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.797+0000 7fdf9d7fa640 1 -- 172.21.15.100:0/3283313845 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdfb00447a0 con 0x55a7fe78c7a0 2024-05-10T11:38:41.802 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.797+0000 7fdf9d7fa640 1 -- 172.21.15.100:0/3283313845 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fdfb0044ce0 con 0x55a7fe78c7a0 2024-05-10T11:38:41.802 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.797+0000 7fdfbe9b8740 10 client.19452 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:38:41.802 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.797+0000 7fdfbe9b8740 10 client.19452 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:38:41.802 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.797+0000 7fdfbe9b8740 20 client.19452 populate_metadata read hostname 'smithi100' 2024-05-10T11:38:41.802 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.797+0000 7fdfbe9b8740 1 --2- 172.21.15.100:0/3283313845 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55a7fe7b2910 0x55a7fe81f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:41.802 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.797+0000 7fdfbe9b8740 4 client.19452 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:38:41.802 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.797+0000 7fdfbe9b8740 1 -- 172.21.15.100:0/3283313845 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a7fe6f8050 con 0x55a7fe7b2910 2024-05-10T11:38:41.802 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.797+0000 7fdfb9234640 1 --2- 172.21.15.100:0/3283313845 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55a7fe7b2910 0x55a7fe81f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:41.802 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.798+0000 7fdf9d7fa640 10 client.19452 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:38:41.802 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.798+0000 7fdfb9234640 1 --2- 172.21.15.100:0/3283313845 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55a7fe7b2910 0x55a7fe81f160 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95dbddebd4204f7b server_cookie=17a7dcda3c8a0094 in_seq=0 out_seq=0 2024-05-10T11:38:41.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.890+0000 7fdf9d7fa640 1 -- 172.21.15.100:0/3283313845 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a7fe6f8050 con 0x55a7fe7b2910 2024-05-10T11:38:41.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.890+0000 7fdf9d7fa640 10 client.19452 handle_command_reply: tid=0 2024-05-10T11:38:41.892 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.890+0000 7fdfbe9b8740 1 client.19452 shutdown 2024-05-10T11:38:41.892 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.890+0000 7fdfbe9b8740 1 -- 172.21.15.100:0/3283313845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a7fe78c7a0 msgr2=0x55a7fe7ab030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:41.892 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.890+0000 7fdfbe9b8740 1 --2- 172.21.15.100:0/3283313845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a7fe78c7a0 0x55a7fe7ab030 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0009850 tx=0x7fdfb0003680 comp rx=0 tx=0).stop 2024-05-10T11:38:41.892 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.890+0000 7fdfbe9b8740 1 -- 172.21.15.100:0/3283313845 shutdown_connections 2024-05-10T11:38:41.892 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.890+0000 7fdfbe9b8740 1 --2- 172.21.15.100:0/3283313845 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55a7fe7b2910 0x55a7fe81f160 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:41.892 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.890+0000 7fdf9d7fa640 0 client.19452 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:38:41.892 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.890+0000 7fdfbe9b8740 1 --2- 172.21.15.100:0/3283313845 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a7fe791910 0x55a7fe81b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:41.892 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.890+0000 7fdfbe9b8740 1 --2- 172.21.15.100:0/3283313845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a7fe78c7a0 0x55a7fe7ab030 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:41.892 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.890+0000 7fdfbe9b8740 1 --2- 172.21.15.100:0/3283313845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7fe78bdb0 0x55a7fe7aaaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:41.892 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.890+0000 7fdfbe9b8740 1 -- 172.21.15.100:0/3283313845 >> 172.21.15.100:0/3283313845 conn(0x55a7fe787420 msgr2=0x55a7fe7823e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:41.892 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.890+0000 7fdfbe9b8740 1 -- 172.21.15.100:0/3283313845 shutdown_connections 2024-05-10T11:38:41.892 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.890+0000 7fdfbe9b8740 1 -- 172.21.15.100:0/3283313845 wait complete. 2024-05-10T11:38:41.893 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.890+0000 7fdfbe9b8740 20 client.19452 trim_cache size 0 max 16384 2024-05-10T11:38:41.893 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:38:41.893 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:38:41.893 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:38:41.893 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:38:41.894 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.891+0000 7fdfbbc99640 1 -- 172.21.15.100:0/1340804720 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fdfa00b2920 msgr2=0x7fdfa00b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:41.894 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.891+0000 7fdfbbc99640 1 --2- 172.21.15.100:0/1340804720 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fdfa00b2920 0x7fdfa00b4de0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fdfac00bfd0 tx=0x7fdfac00e040 comp rx=0 tx=0).stop 2024-05-10T11:38:41.894 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.891+0000 7fdfbbc99640 1 -- 172.21.15.100:0/1340804720 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfb419cac0 msgr2=0x7fdfb410bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:41.894 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.891+0000 7fdfbbc99640 1 --2- 172.21.15.100:0/1340804720 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfb419cac0 0x7fdfb410bb00 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fdfac00b970 tx=0x7fdfac00bd70 comp rx=0 tx=0).stop 2024-05-10T11:38:41.894 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.891+0000 7fdfbbc99640 1 -- 172.21.15.100:0/1340804720 shutdown_connections 2024-05-10T11:38:41.894 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.891+0000 7fdfbbc99640 1 --2- 172.21.15.100:0/1340804720 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fdfa00b2920 0x7fdfa00b4de0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:41.894 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.891+0000 7fdfbbc99640 1 --2- 172.21.15.100:0/1340804720 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfb419cac0 0x7fdfb410bb00 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:41.894 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.891+0000 7fdfbbc99640 1 --2- 172.21.15.100:0/1340804720 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdfb41034c0 0x7fdfb410b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:41.895 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.891+0000 7fdfbbc99640 1 --2- 172.21.15.100:0/1340804720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfb4102b00 0x7fdfb410cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:41.895 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.891+0000 7fdfbbc99640 1 -- 172.21.15.100:0/1340804720 >> 172.21.15.100:0/1340804720 conn(0x7fdfb40fce80 msgr2=0x7fdfb4105fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:41.895 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.891+0000 7fdfbbc99640 1 -- 172.21.15.100:0/1340804720 shutdown_connections 2024-05-10T11:38:41.895 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:41.891+0000 7fdfbbc99640 1 -- 172.21.15.100:0/1340804720 wait complete. 2024-05-10T11:38:41.945 INFO:tasks.workunit.client.0.smithi100.stdout: 10 66168 8.97 MB/sec execute 275 sec latency 113.948 ms 2024-05-10T11:38:42.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:41 smithi178 ceph-mon[33359]: pgmap v1030: 129 pgs: 129 active+clean; 510 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 4.5 MiB/s wr, 366 op/s 2024-05-10T11:38:42.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:41 smithi188 ceph-mon[34075]: pgmap v1030: 129 pgs: 129 active+clean; 510 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 4.5 MiB/s wr, 366 op/s 2024-05-10T11:38:42.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:41 smithi100 ceph-mon[30296]: pgmap v1030: 129 pgs: 129 active+clean; 510 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 4.5 MiB/s wr, 366 op/s 2024-05-10T11:38:42.945 INFO:tasks.workunit.client.0.smithi100.stdout: 10 66349 8.99 MB/sec execute 276 sec latency 69.574 ms 2024-05-10T11:38:43.945 INFO:tasks.workunit.client.0.smithi100.stdout: 10 66491 8.97 MB/sec execute 277 sec latency 99.972 ms 2024-05-10T11:38:44.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:43 smithi178 ceph-mon[33359]: pgmap v1031: 129 pgs: 129 active+clean; 509 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 4.0 MiB/s wr, 337 op/s 2024-05-10T11:38:44.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:43 smithi188 ceph-mon[34075]: pgmap v1031: 129 pgs: 129 active+clean; 509 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 4.0 MiB/s wr, 337 op/s 2024-05-10T11:38:44.033 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:43 smithi100 ceph-mon[30296]: pgmap v1031: 129 pgs: 129 active+clean; 509 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 4.0 MiB/s wr, 337 op/s 2024-05-10T11:38:44.945 INFO:tasks.workunit.client.0.smithi100.stdout: 10 66688 8.98 MB/sec execute 278 sec latency 59.124 ms 2024-05-10T11:38:45.945 INFO:tasks.workunit.client.0.smithi100.stdout: 10 66908 9.00 MB/sec execute 279 sec latency 72.578 ms 2024-05-10T11:38:46.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:45 smithi178 ceph-mon[33359]: pgmap v1032: 129 pgs: 129 active+clean; 509 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 4.4 MiB/s wr, 358 op/s 2024-05-10T11:38:46.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:45 smithi188 ceph-mon[34075]: pgmap v1032: 129 pgs: 129 active+clean; 509 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 4.4 MiB/s wr, 358 op/s 2024-05-10T11:38:46.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:45 smithi100 ceph-mon[30296]: pgmap v1032: 129 pgs: 129 active+clean; 509 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 4.4 MiB/s wr, 358 op/s 2024-05-10T11:38:46.945 INFO:tasks.workunit.client.0.smithi100.stdout: 10 67112 9.00 MB/sec execute 280 sec latency 79.703 ms 2024-05-10T11:38:47.945 INFO:tasks.workunit.client.0.smithi100.stdout: 10 67218 8.99 MB/sec execute 281 sec latency 148.173 ms 2024-05-10T11:38:48.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:47 smithi178 ceph-mon[33359]: pgmap v1033: 129 pgs: 129 active+clean; 516 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 976 KiB/s rd, 4.8 MiB/s wr, 367 op/s 2024-05-10T11:38:48.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:47 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:38:48.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:47 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:38:48.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:47 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:38:48.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:47 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:38:48.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:47 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:38:48.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:47 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:38:48.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:47 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:38:48.020 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:47 smithi100 ceph-mon[30296]: pgmap v1033: 129 pgs: 129 active+clean; 516 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 976 KiB/s rd, 4.8 MiB/s wr, 367 op/s 2024-05-10T11:38:48.021 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:47 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:38:48.021 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:47 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:38:48.021 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:47 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:38:48.021 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:47 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:38:48.021 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:47 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:38:48.021 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:47 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:38:48.021 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:47 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:38:48.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:47 smithi188 ceph-mon[34075]: pgmap v1033: 129 pgs: 129 active+clean; 516 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 976 KiB/s rd, 4.8 MiB/s wr, 367 op/s 2024-05-10T11:38:48.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:47 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:38:48.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:47 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:38:48.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:47 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:38:48.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:47 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:38:48.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:47 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:38:48.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:47 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:38:48.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:47 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:38:48.945 INFO:tasks.workunit.client.0.smithi100.stdout: 10 67355 9.00 MB/sec execute 282 sec latency 137.729 ms 2024-05-10T11:38:49.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:48 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:38:49.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:48 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:38:49.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:48 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:38:49.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:48 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:38:49.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:48 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:38:49.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:48 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:38:49.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:48 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:38:49.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:48 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:38:49.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:48 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:38:49.945 INFO:tasks.workunit.client.0.smithi100.stdout: 10 67595 9.03 MB/sec execute 283 sec latency 149.447 ms 2024-05-10T11:38:50.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:49 smithi178 ceph-mon[33359]: pgmap v1034: 129 pgs: 129 active+clean; 518 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 4.1 MiB/s wr, 332 op/s 2024-05-10T11:38:50.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:49 smithi188 ceph-mon[34075]: pgmap v1034: 129 pgs: 129 active+clean; 518 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 4.1 MiB/s wr, 332 op/s 2024-05-10T11:38:50.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:49 smithi100 ceph-mon[30296]: pgmap v1034: 129 pgs: 129 active+clean; 518 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 4.1 MiB/s wr, 332 op/s 2024-05-10T11:38:50.945 INFO:tasks.workunit.client.0.smithi100.stdout: 10 67763 9.04 MB/sec execute 284 sec latency 78.268 ms 2024-05-10T11:38:51.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:50 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:38:51.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:50 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:38:51.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:50 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:38:51.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:50 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:38:51.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:50 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:38:51.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:50 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:38:51.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:50 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:38:51.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:50 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:38:51.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:50 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:38:51.945 INFO:tasks.workunit.client.0.smithi100.stdout: 10 67892 9.03 MB/sec execute 285 sec latency 87.149 ms 2024-05-10T11:38:52.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:51 smithi178 ceph-mon[33359]: pgmap v1035: 129 pgs: 129 active+clean; 523 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.5 MiB/s wr, 344 op/s 2024-05-10T11:38:52.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:51 smithi188 ceph-mon[34075]: pgmap v1035: 129 pgs: 129 active+clean; 523 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.5 MiB/s wr, 344 op/s 2024-05-10T11:38:52.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:51 smithi100 ceph-mon[30296]: pgmap v1035: 129 pgs: 129 active+clean; 523 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.5 MiB/s wr, 344 op/s 2024-05-10T11:38:52.945 INFO:tasks.workunit.client.0.smithi100.stdout: 10 68068 9.05 MB/sec execute 286 sec latency 55.798 ms 2024-05-10T11:38:53.945 INFO:tasks.workunit.client.0.smithi100.stdout: 10 68206 9.03 MB/sec execute 287 sec latency 135.726 ms 2024-05-10T11:38:54.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:53 smithi178 ceph-mon[33359]: pgmap v1036: 129 pgs: 129 active+clean; 525 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 4.3 MiB/s wr, 268 op/s 2024-05-10T11:38:54.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:53 smithi188 ceph-mon[34075]: pgmap v1036: 129 pgs: 129 active+clean; 525 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 4.3 MiB/s wr, 268 op/s 2024-05-10T11:38:54.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:53 smithi100 ceph-mon[30296]: pgmap v1036: 129 pgs: 129 active+clean; 525 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 4.3 MiB/s wr, 268 op/s 2024-05-10T11:38:54.945 INFO:tasks.workunit.client.0.smithi100.stdout: 10 68342 9.01 MB/sec execute 288 sec latency 71.607 ms 2024-05-10T11:38:55.032 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.031+0000 7fa167938640 1 -- 172.21.15.100:0/2302424620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa16010c320 msgr2=0x7fa16010e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:55.033 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.031+0000 7fa167938640 1 --2- 172.21.15.100:0/2302424620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa16010c320 0x7fa16010e790 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fa150009900 tx=0x7fa15002a420 comp rx=0 tx=0).stop 2024-05-10T11:38:55.035 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.035+0000 7fa167938640 1 -- 172.21.15.100:0/2302424620 shutdown_connections 2024-05-10T11:38:55.035 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.035+0000 7fa167938640 1 --2- 172.21.15.100:0/2302424620 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa16010ecd0 0x7fa16010f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:55.035 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.035+0000 7fa167938640 1 --2- 172.21.15.100:0/2302424620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa16010c320 0x7fa16010e790 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:55.035 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.035+0000 7fa167938640 1 --2- 172.21.15.100:0/2302424620 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa1601099b0 0x7fa16010bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:55.035 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.035+0000 7fa167938640 1 -- 172.21.15.100:0/2302424620 >> 172.21.15.100:0/2302424620 conn(0x7fa16006fa20 msgr2=0x7fa160071e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:55.037 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.037+0000 7fa167938640 1 -- 172.21.15.100:0/2302424620 shutdown_connections 2024-05-10T11:38:55.037 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.037+0000 7fa167938640 1 -- 172.21.15.100:0/2302424620 wait complete. 2024-05-10T11:38:55.038 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.038+0000 7fa167938640 1 Processor -- start 2024-05-10T11:38:55.038 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.039+0000 7fa167938640 1 -- start start 2024-05-10T11:38:55.039 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.039+0000 7fa167938640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1601099b0 0x7fa1601a78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:55.039 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.039+0000 7fa167938640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa16010c320 0x7fa1601a7df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:55.039 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.039+0000 7fa167938640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa16010ecd0 0x7fa1601aee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:55.039 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.039+0000 7fa167938640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa16011d290 con 0x7fa16010c320 2024-05-10T11:38:55.039 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.039+0000 7fa167938640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa16011d110 con 0x7fa16010ecd0 2024-05-10T11:38:55.039 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.039+0000 7fa167938640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa16011d410 con 0x7fa1601099b0 2024-05-10T11:38:55.040 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.039+0000 7fa165ed5640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa16010ecd0 0x7fa1601aee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:55.040 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.039+0000 7fa165ed5640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa16010ecd0 0x7fa1601aee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:49554/0 (socket says 172.21.15.100:49554) 2024-05-10T11:38:55.040 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.039+0000 7fa165ed5640 1 -- 172.21.15.100:0/1102128128 learned_addr learned my addr 172.21.15.100:0/1102128128 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:38:55.040 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.040+0000 7fa1656d4640 1 --2- 172.21.15.100:0/1102128128 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1601099b0 0x7fa1601a78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:55.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.042+0000 7fa164ed3640 1 --2- 172.21.15.100:0/1102128128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa16010c320 0x7fa1601a7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:55.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.043+0000 7fa165ed5640 1 -- 172.21.15.100:0/1102128128 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1601099b0 msgr2=0x7fa1601a78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:55.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.043+0000 7fa165ed5640 1 --2- 172.21.15.100:0/1102128128 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1601099b0 0x7fa1601a78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:55.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.043+0000 7fa165ed5640 1 -- 172.21.15.100:0/1102128128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa16010c320 msgr2=0x7fa1601a7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:55.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.043+0000 7fa165ed5640 1 --2- 172.21.15.100:0/1102128128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa16010c320 0x7fa1601a7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:55.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.043+0000 7fa165ed5640 1 -- 172.21.15.100:0/1102128128 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1601af4d0 con 0x7fa16010ecd0 2024-05-10T11:38:55.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.044+0000 7fa165ed5640 1 --2- 172.21.15.100:0/1102128128 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa16010ecd0 0x7fa1601aee60 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa15c00d750 tx=0x7fa15c00dc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:55.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.044+0000 7fa1567fc640 1 -- 172.21.15.100:0/1102128128 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa15c017070 con 0x7fa16010ecd0 2024-05-10T11:38:55.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.044+0000 7fa167938640 1 -- 172.21.15.100:0/1102128128 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1601af7c0 con 0x7fa16010ecd0 2024-05-10T11:38:55.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.044+0000 7fa167938640 1 -- 172.21.15.100:0/1102128128 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1601afd10 con 0x7fa16010ecd0 2024-05-10T11:38:55.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.044+0000 7fa1567fc640 1 -- 172.21.15.100:0/1102128128 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa15c0054b0 con 0x7fa16010ecd0 2024-05-10T11:38:55.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.044+0000 7fa1567fc640 1 -- 172.21.15.100:0/1102128128 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa15c017070 con 0x7fa16010ecd0 2024-05-10T11:38:55.049 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.049+0000 7fa16a655740 1 Processor -- start 2024-05-10T11:38:55.050 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.050+0000 7fa1567fc640 1 -- 172.21.15.100:0/1102128128 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fa15c004970 con 0x7fa16010ecd0 2024-05-10T11:38:55.050 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.050+0000 7fa16a655740 1 -- start start 2024-05-10T11:38:55.050 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.050+0000 7fa16a655740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ed7928db0 0x561ed79291d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:55.050 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.050+0000 7fa16a655740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ed79297a0 0x561ed792e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:55.050 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.050+0000 7fa16a655740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561ed792e910 0x561ed7930da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:55.050 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.050+0000 7fa16a655740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x561ed788d5d0 con 0x561ed79297a0 2024-05-10T11:38:55.051 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.050+0000 7fa16a655740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561ed7846890 con 0x561ed7928db0 2024-05-10T11:38:55.051 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.050+0000 7fa16a655740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x561ed7889590 con 0x561ed792e910 2024-05-10T11:38:55.051 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.051+0000 7fa1567fc640 1 --2- 172.21.15.100:0/1102128128 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa14c0b2d60 0x7fa14c0b5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:55.051 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.052+0000 7fa1567fc640 1 -- 172.21.15.100:0/1102128128 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fa15c0099d0 con 0x7fa16010ecd0 2024-05-10T11:38:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.052+0000 7fa165ed5640 1 --2- 172.21.15.100:0/1102128128 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa14c0b2d60 0x7fa14c0b5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.052+0000 7fa165ed5640 1 --2- 172.21.15.100:0/1102128128 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa14c0b2d60 0x7fa14c0b5220 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fa15c0a8b70 tx=0x7fa15c0e19d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.053+0000 7fa1656d4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ed79297a0 0x561ed792e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.053+0000 7fa1656d4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ed79297a0 0x561ed792e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56938/0 (socket says 172.21.15.100:56938) 2024-05-10T11:38:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.053+0000 7fa1656d4640 1 -- 172.21.15.100:0/1508794743 learned_addr learned my addr 172.21.15.100:0/1508794743 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:38:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.053+0000 7fa164ed3640 1 --2- 172.21.15.100:0/1508794743 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ed7928db0 0x561ed79291d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.053+0000 7fa164ed3640 1 --2- 172.21.15.100:0/1508794743 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561ed792e910 0x561ed7930da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:55.054 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.054+0000 7fa1656d4640 1 -- 172.21.15.100:0/1508794743 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561ed792e910 msgr2=0x561ed7930da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:55.054 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.054+0000 7fa1656d4640 1 --2- 172.21.15.100:0/1508794743 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561ed792e910 0x561ed7930da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:55.054 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.054+0000 7fa1656d4640 1 -- 172.21.15.100:0/1508794743 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ed7928db0 msgr2=0x561ed79291d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:55.054 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.054+0000 7fa1656d4640 1 --2- 172.21.15.100:0/1508794743 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ed7928db0 0x561ed79291d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:55.055 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.054+0000 7fa1656d4640 1 -- 172.21.15.100:0/1508794743 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ed759f210 con 0x561ed79297a0 2024-05-10T11:38:55.055 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.056+0000 7fa1656d4640 1 --2- 172.21.15.100:0/1508794743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ed79297a0 0x561ed792e2a0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fa15800a5b0 tx=0x7fa1580096d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90e2318620bc42e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:55.056 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.056+0000 7fa14b7fe640 1 -- 172.21.15.100:0/1508794743 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa158018020 con 0x561ed79297a0 2024-05-10T11:38:55.056 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.056+0000 7fa14b7fe640 1 -- 172.21.15.100:0/1508794743 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa158014ca0 con 0x561ed79297a0 2024-05-10T11:38:55.056 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.056+0000 7fa14b7fe640 1 -- 172.21.15.100:0/1508794743 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa158013ce0 con 0x561ed79297a0 2024-05-10T11:38:55.058 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.058+0000 7fa16a655740 1 -- 172.21.15.100:0/1508794743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ed79297a0 msgr2=0x561ed792e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:55.058 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.058+0000 7fa16a655740 1 --2- 172.21.15.100:0/1508794743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ed79297a0 0x561ed792e2a0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fa15800a5b0 tx=0x7fa1580096d0 comp rx=0 tx=0).stop 2024-05-10T11:38:55.058 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.058+0000 7fa16a655740 1 -- 172.21.15.100:0/1508794743 shutdown_connections 2024-05-10T11:38:55.058 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.058+0000 7fa16a655740 1 --2- 172.21.15.100:0/1508794743 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561ed792e910 0x561ed7930da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:55.058 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.058+0000 7fa16a655740 1 --2- 172.21.15.100:0/1508794743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ed79297a0 0x561ed792e2a0 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:55.058 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.058+0000 7fa16a655740 1 --2- 172.21.15.100:0/1508794743 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ed7928db0 0x561ed79291d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:55.058 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.058+0000 7fa16a655740 1 -- 172.21.15.100:0/1508794743 >> 172.21.15.100:0/1508794743 conn(0x561ed7924420 msgr2=0x561ed7924820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:55.062 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.060+0000 7fa16a655740 1 -- 172.21.15.100:0/1508794743 shutdown_connections 2024-05-10T11:38:55.062 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.060+0000 7fa16a655740 1 -- 172.21.15.100:0/1508794743 wait complete. 2024-05-10T11:38:55.062 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.060+0000 7fa16a655740 1 -- start start 2024-05-10T11:38:55.062 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.060+0000 7fa16a655740 1 Processor -- start 2024-05-10T11:38:55.062 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.061+0000 7fa16a655740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ed7928db0 0x561ed79330c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:55.062 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.061+0000 7fa16a655740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561ed79297a0 0x561ed7933600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:55.062 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.061+0000 7fa16a655740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ed792e910 0x561ed793a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:55.062 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.061+0000 7fa16a655740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x561ed79318c0 con 0x561ed792e910 2024-05-10T11:38:55.062 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.061+0000 7fa16a655740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561ed784a370 con 0x561ed7928db0 2024-05-10T11:38:55.062 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.061+0000 7fa16a655740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x561ed7931a40 con 0x561ed79297a0 2024-05-10T11:38:55.062 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.061+0000 7fa164ed3640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ed7928db0 0x561ed79330c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:55.063 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.061+0000 7fa164ed3640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ed7928db0 0x561ed79330c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:49578/0 (socket says 172.21.15.100:49578) 2024-05-10T11:38:55.063 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.061+0000 7fa164ed3640 1 -- 172.21.15.100:0/713421650 learned_addr learned my addr 172.21.15.100:0/713421650 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:38:55.063 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.062+0000 7fa16a655740 10 client.0 _ll_register_callbacks cb 0x561ed759ea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:38:55.063 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.062+0000 7fa164ed3640 1 --2- 172.21.15.100:0/713421650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ed792e910 0x561ed793a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.062+0000 7fa164ed3640 1 -- 172.21.15.100:0/713421650 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561ed79297a0 msgr2=0x561ed7933600 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T11:38:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.062+0000 7fa164ed3640 1 --2- 172.21.15.100:0/713421650 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561ed79297a0 0x561ed7933600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.062+0000 7fa164ed3640 1 -- 172.21.15.100:0/713421650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ed792e910 msgr2=0x561ed793a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.062+0000 7fa164ed3640 1 --2- 172.21.15.100:0/713421650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ed792e910 0x561ed793a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.062+0000 7fa164ed3640 1 -- 172.21.15.100:0/713421650 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ed766b620 con 0x561ed7928db0 2024-05-10T11:38:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.063+0000 7fa164ed3640 1 --2- 172.21.15.100:0/713421650 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ed7928db0 0x561ed79330c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa150031970 tx=0x7fa150009640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0e41b19bbd19ab0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.063+0000 7fa148ff9640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:38:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.063+0000 7fa148ff9640 1 -- 172.21.15.100:0/713421650 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa150042030 con 0x561ed7928db0 2024-05-10T11:38:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.064+0000 7fa148ff9640 1 -- 172.21.15.100:0/713421650 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fa150049040 con 0x561ed7928db0 2024-05-10T11:38:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.065+0000 7fa16a655740 1 -- 172.21.15.100:0/713421650 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa158014ca0 con 0x561ed7928db0 2024-05-10T11:38:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.065+0000 7fa148ff9640 1 -- 172.21.15.100:0/713421650 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fa150053050 con 0x561ed7928db0 2024-05-10T11:38:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.067+0000 7fa16a655740 10 client.28733 fetch_fsmap learned FSMap version 41 2024-05-10T11:38:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.067+0000 7fa16a655740 1 -- 172.21.15.100:0/713421650 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x561ed793d690 con 0x561ed7928db0 2024-05-10T11:38:55.068 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.068+0000 7fa148ff9640 1 -- 172.21.15.100:0/713421650 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa158014ca0 con 0x561ed7928db0 2024-05-10T11:38:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.068+0000 7fa148ff9640 1 -- 172.21.15.100:0/713421650 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fa15004c070 con 0x561ed7928db0 2024-05-10T11:38:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.068+0000 7fa16a655740 10 client.28733 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:38:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.068+0000 7fa16a655740 10 client.28733 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:38:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.068+0000 7fa16a655740 20 client.28733 populate_metadata read hostname 'smithi100' 2024-05-10T11:38:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.068+0000 7fa16a655740 1 --2- 172.21.15.100:0/713421650 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x561ed793f260 0x561ed7941720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.069+0000 7fa16a655740 4 client.28733 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:38:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.069+0000 7fa16a655740 1 -- 172.21.15.100:0/713421650 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ed789b1e0 con 0x561ed793f260 2024-05-10T11:38:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.069+0000 7fa1656d4640 1 --2- 172.21.15.100:0/713421650 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x561ed793f260 0x561ed7941720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:55.070 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.070+0000 7fa1656d4640 1 --2- 172.21.15.100:0/713421650 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x561ed793f260 0x561ed7941720 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52c862ef788e833e server_cookie=a81a31317ec1466e in_seq=0 out_seq=0 2024-05-10T11:38:55.070 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.070+0000 7fa148ff9640 10 client.28733 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:38:55.071 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.071+0000 7fa148ff9640 1 -- 172.21.15.100:0/713421650 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561ed789b1e0 con 0x561ed793f260 2024-05-10T11:38:55.071 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.071+0000 7fa148ff9640 10 client.28733 handle_command_reply: tid=0 2024-05-10T11:38:55.071 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.071+0000 7fa16a655740 1 client.28733 shutdown 2024-05-10T11:38:55.072 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.072+0000 7fa16a655740 1 -- 172.21.15.100:0/713421650 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ed7928db0 msgr2=0x561ed79330c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:55.072 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.072+0000 7fa16a655740 1 --2- 172.21.15.100:0/713421650 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ed7928db0 0x561ed79330c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa150031970 tx=0x7fa150009640 comp rx=0 tx=0).stop 2024-05-10T11:38:55.072 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.072+0000 7fa16a655740 1 -- 172.21.15.100:0/713421650 shutdown_connections 2024-05-10T11:38:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.072+0000 7fa16a655740 1 --2- 172.21.15.100:0/713421650 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x561ed793f260 0x561ed7941720 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.072+0000 7fa16a655740 1 --2- 172.21.15.100:0/713421650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ed792e910 0x561ed793a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.072+0000 7fa16a655740 1 --2- 172.21.15.100:0/713421650 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561ed79297a0 0x561ed7933600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.072+0000 7fa16a655740 1 --2- 172.21.15.100:0/713421650 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ed7928db0 0x561ed79330c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.072+0000 7fa16a655740 1 -- 172.21.15.100:0/713421650 >> 172.21.15.100:0/713421650 conn(0x561ed7924420 msgr2=0x561ed7923580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.072+0000 7fa148ff9640 0 client.28733 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:38:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.072+0000 7fa16a655740 1 -- 172.21.15.100:0/713421650 shutdown_connections 2024-05-10T11:38:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.072+0000 7fa16a655740 1 -- 172.21.15.100:0/713421650 wait complete. 2024-05-10T11:38:55.074 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.073+0000 7fa16a655740 20 client.28733 trim_cache size 0 max 16384 2024-05-10T11:38:55.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.100+0000 7fa16a655740 1 Processor -- start 2024-05-10T11:38:55.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.100+0000 7fa16a655740 1 -- start start 2024-05-10T11:38:55.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.100+0000 7fa16a655740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ed7928db0 0x561ed7927330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:55.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.100+0000 7fa16a655740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ed79297a0 0x561ed793a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:55.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.100+0000 7fa16a655740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561ed792e910 0x561ed7927870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:55.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.100+0000 7fa16a655740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa1440124a0 con 0x561ed79297a0 2024-05-10T11:38:55.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.100+0000 7fa1656d4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ed79297a0 0x561ed793a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:55.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.100+0000 7fa16a655740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa14400fd70 con 0x561ed7928db0 2024-05-10T11:38:55.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.100+0000 7fa164ed3640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ed7928db0 0x561ed7927330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:55.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.100+0000 7fa1656d4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ed79297a0 0x561ed793a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56956/0 (socket says 172.21.15.100:56956) 2024-05-10T11:38:55.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.100+0000 7fa1656d4640 1 -- 172.21.15.100:0/425097789 learned_addr learned my addr 172.21.15.100:0/425097789 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:38:55.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.100+0000 7fa16a655740 1 -- 172.21.15.100:0/425097789 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa14400dbc0 con 0x561ed792e910 2024-05-10T11:38:55.102 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.101+0000 7fa164ed3640 1 --2- 172.21.15.100:0/425097789 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561ed792e910 0x561ed7927870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:55.103 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.101+0000 7fa164ed3640 1 -- 172.21.15.100:0/425097789 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561ed792e910 msgr2=0x561ed7927870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:55.103 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.101+0000 7fa164ed3640 1 --2- 172.21.15.100:0/425097789 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561ed792e910 0x561ed7927870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:55.103 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.101+0000 7fa164ed3640 1 -- 172.21.15.100:0/425097789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ed79297a0 msgr2=0x561ed793a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:55.103 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.101+0000 7fa164ed3640 1 --2- 172.21.15.100:0/425097789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ed79297a0 0x561ed793a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:55.103 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.101+0000 7fa164ed3640 1 -- 172.21.15.100:0/425097789 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ed7924250 con 0x561ed7928db0 2024-05-10T11:38:55.103 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.101+0000 7fa1656d4640 1 --2- 172.21.15.100:0/425097789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ed79297a0 0x561ed793a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:38:55.103 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.101+0000 7fa164ed3640 1 --2- 172.21.15.100:0/425097789 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ed7928db0 0x561ed7927330 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa15003d800 tx=0x7fa15002a420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fddf506428b48119 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:55.103 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.101+0000 7fa14affd640 1 -- 172.21.15.100:0/425097789 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa150042040 con 0x561ed7928db0 2024-05-10T11:38:55.104 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.102+0000 7fa14affd640 1 -- 172.21.15.100:0/425097789 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa1500044b0 con 0x561ed7928db0 2024-05-10T11:38:55.104 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.102+0000 7fa14affd640 1 -- 172.21.15.100:0/425097789 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa150035040 con 0x561ed7928db0 2024-05-10T11:38:55.104 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.102+0000 7fa16a655740 1 -- 172.21.15.100:0/425097789 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ed7928db0 msgr2=0x561ed7927330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:55.104 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.102+0000 7fa16a655740 1 --2- 172.21.15.100:0/425097789 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ed7928db0 0x561ed7927330 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa15003d800 tx=0x7fa15002a420 comp rx=0 tx=0).stop 2024-05-10T11:38:55.104 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.102+0000 7fa16a655740 1 -- 172.21.15.100:0/425097789 shutdown_connections 2024-05-10T11:38:55.104 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.102+0000 7fa16a655740 1 --2- 172.21.15.100:0/425097789 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561ed792e910 0x561ed7927870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:55.104 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.102+0000 7fa16a655740 1 --2- 172.21.15.100:0/425097789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ed79297a0 0x561ed793a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:55.104 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.102+0000 7fa16a655740 1 --2- 172.21.15.100:0/425097789 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ed7928db0 0x561ed7927330 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:55.104 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.102+0000 7fa16a655740 1 -- 172.21.15.100:0/425097789 >> 172.21.15.100:0/425097789 conn(0x561ed7924420 msgr2=0x561ed792aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:55.104 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.103+0000 7fa16a655740 1 -- 172.21.15.100:0/425097789 shutdown_connections 2024-05-10T11:38:55.104 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.103+0000 7fa16a655740 1 -- 172.21.15.100:0/425097789 wait complete. 2024-05-10T11:38:55.104 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.103+0000 7fa16a655740 1 -- start start 2024-05-10T11:38:55.104 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.103+0000 7fa16a655740 1 Processor -- start 2024-05-10T11:38:55.107 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.104+0000 7fa16a655740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ed7928db0 0x561ed79375f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:55.107 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.104+0000 7fa16a655740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ed79297a0 0x561ed7937b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:55.107 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.104+0000 7fa16a655740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561ed792e910 0x561ed79a7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:55.108 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.104+0000 7fa16a655740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa14400e230 con 0x561ed79297a0 2024-05-10T11:38:55.108 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.104+0000 7fa1656d4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ed79297a0 0x561ed7937b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:55.108 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.104+0000 7fa164ed3640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ed7928db0 0x561ed79375f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:55.108 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.104+0000 7fa16a655740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa14400f6a0 con 0x561ed7928db0 2024-05-10T11:38:55.108 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.104+0000 7fa1656d4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ed79297a0 0x561ed7937b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56960/0 (socket says 172.21.15.100:56960) 2024-05-10T11:38:55.108 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.104+0000 7fa1656d4640 1 -- 172.21.15.100:0/1191692977 learned_addr learned my addr 172.21.15.100:0/1191692977 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:38:55.108 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.104+0000 7fa16a655740 1 -- 172.21.15.100:0/1191692977 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa14400e900 con 0x561ed792e910 2024-05-10T11:38:55.108 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.104+0000 7fa16a655740 10 client.0 _ll_register_callbacks cb 0x7fa15003d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:38:55.108 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.104+0000 7fa164ed3640 1 --2- 172.21.15.100:0/1191692977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561ed792e910 0x561ed79a7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:55.108 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.104+0000 7fa1656d4640 1 -- 172.21.15.100:0/1191692977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561ed792e910 msgr2=0x561ed79a7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:55.108 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.104+0000 7fa1656d4640 1 --2- 172.21.15.100:0/1191692977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561ed792e910 0x561ed79a7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:55.108 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.104+0000 7fa1656d4640 1 -- 172.21.15.100:0/1191692977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ed7928db0 msgr2=0x561ed79375f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:38:55.108 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.104+0000 7fa1656d4640 1 --2- 172.21.15.100:0/1191692977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ed7928db0 0x561ed79375f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:55.108 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.104+0000 7fa1656d4640 1 -- 172.21.15.100:0/1191692977 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ed79a8140 con 0x561ed79297a0 2024-05-10T11:38:55.108 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.104+0000 7fa164ed3640 1 --2- 172.21.15.100:0/1191692977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ed7928db0 0x561ed79375f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:38:55.109 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.104+0000 7fa1656d4640 1 --2- 172.21.15.100:0/1191692977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ed79297a0 0x561ed7937b30 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fa15800a580 tx=0x7fa158009ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3914a52a907d3ba2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:38:55.109 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.104+0000 7fa148ff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:38:55.109 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.105+0000 7fa148ff9640 1 -- 172.21.15.100:0/1191692977 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa158018040 con 0x561ed79297a0 2024-05-10T11:38:55.109 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.105+0000 7fa148ff9640 1 -- 172.21.15.100:0/1191692977 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fa158003c10 con 0x561ed79297a0 2024-05-10T11:38:55.109 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.105+0000 7fa16a655740 1 -- 172.21.15.100:0/1191692977 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1500044b0 con 0x561ed79297a0 2024-05-10T11:38:55.109 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.105+0000 7fa148ff9640 1 -- 172.21.15.100:0/1191692977 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fa158003e10 con 0x561ed79297a0 2024-05-10T11:38:55.109 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.105+0000 7fa16a655740 10 client.19470 fetch_fsmap learned FSMap version 41 2024-05-10T11:38:55.109 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.105+0000 7fa16a655740 1 -- 172.21.15.100:0/1191692977 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x561ed79a9410 con 0x561ed79297a0 2024-05-10T11:38:55.109 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.106+0000 7fa148ff9640 1 -- 172.21.15.100:0/1191692977 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa1500044b0 con 0x561ed79297a0 2024-05-10T11:38:55.109 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.106+0000 7fa148ff9640 1 -- 172.21.15.100:0/1191692977 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fa158026ce0 con 0x561ed79297a0 2024-05-10T11:38:55.109 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.106+0000 7fa16a655740 10 client.19470 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:38:55.109 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.106+0000 7fa16a655740 10 client.19470 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:38:55.109 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.106+0000 7fa16a655740 20 client.19470 populate_metadata read hostname 'smithi100' 2024-05-10T11:38:55.110 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.106+0000 7fa16a655740 1 --2- 172.21.15.100:0/1191692977 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x561ed79a9b80 0x561ed79abfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:38:55.110 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.106+0000 7fa16a655740 4 client.19470 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:38:55.110 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.106+0000 7fa16a655740 1 -- 172.21.15.100:0/1191692977 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561ed7895050 con 0x561ed79a9b80 2024-05-10T11:38:55.110 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.106+0000 7fa164ed3640 1 --2- 172.21.15.100:0/1191692977 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x561ed79a9b80 0x561ed79abfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:38:55.110 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.106+0000 7fa164ed3640 1 --2- 172.21.15.100:0/1191692977 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x561ed79a9b80 0x561ed79abfa0 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68cefcbeb72c3e51 server_cookie=138705b43cf3595c in_seq=0 out_seq=0 2024-05-10T11:38:55.110 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.106+0000 7fa148ff9640 10 client.19470 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:38:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.334+0000 7fa148ff9640 1 -- 172.21.15.100:0/1191692977 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561ed7895050 con 0x561ed79a9b80 2024-05-10T11:38:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.334+0000 7fa148ff9640 10 client.19470 handle_command_reply: tid=0 2024-05-10T11:38:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.334+0000 7fa16a655740 1 client.19470 shutdown 2024-05-10T11:38:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.334+0000 7fa16a655740 1 -- 172.21.15.100:0/1191692977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ed79297a0 msgr2=0x561ed7937b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.334+0000 7fa16a655740 1 --2- 172.21.15.100:0/1191692977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ed79297a0 0x561ed7937b30 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fa15800a580 tx=0x7fa158009ce0 comp rx=0 tx=0).stop 2024-05-10T11:38:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.334+0000 7fa16a655740 1 -- 172.21.15.100:0/1191692977 shutdown_connections 2024-05-10T11:38:55.336 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.334+0000 7fa16a655740 1 --2- 172.21.15.100:0/1191692977 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x561ed79a9b80 0x561ed79abfa0 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:55.336 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.334+0000 7fa16a655740 1 --2- 172.21.15.100:0/1191692977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561ed792e910 0x561ed79a7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:55.336 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.334+0000 7fa16a655740 1 --2- 172.21.15.100:0/1191692977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ed79297a0 0x561ed7937b30 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:55.336 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.334+0000 7fa148ff9640 0 client.19470 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:38:55.336 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.334+0000 7fa16a655740 1 --2- 172.21.15.100:0/1191692977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ed7928db0 0x561ed79375f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:55.336 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.334+0000 7fa16a655740 1 -- 172.21.15.100:0/1191692977 >> 172.21.15.100:0/1191692977 conn(0x561ed7924420 msgr2=0x561ed7927330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:55.336 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.335+0000 7fa16a655740 1 -- 172.21.15.100:0/1191692977 shutdown_connections 2024-05-10T11:38:55.336 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.335+0000 7fa16a655740 1 -- 172.21.15.100:0/1191692977 wait complete. 2024-05-10T11:38:55.336 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.335+0000 7fa16a655740 20 client.19470 trim_cache size 0 max 16384 2024-05-10T11:38:55.336 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:38:55.337 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:38:55.337 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:38:55.337 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:38:55.338 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.335+0000 7fa167938640 1 -- 172.21.15.100:0/1102128128 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa14c0b2d60 msgr2=0x7fa14c0b5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:55.338 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.335+0000 7fa167938640 1 --2- 172.21.15.100:0/1102128128 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa14c0b2d60 0x7fa14c0b5220 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fa15c0a8b70 tx=0x7fa15c0e19d0 comp rx=0 tx=0).stop 2024-05-10T11:38:55.338 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.336+0000 7fa167938640 1 -- 172.21.15.100:0/1102128128 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa16010ecd0 msgr2=0x7fa1601aee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:38:55.338 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.336+0000 7fa167938640 1 --2- 172.21.15.100:0/1102128128 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa16010ecd0 0x7fa1601aee60 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa15c00d750 tx=0x7fa15c00dc10 comp rx=0 tx=0).stop 2024-05-10T11:38:55.338 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.336+0000 7fa167938640 1 -- 172.21.15.100:0/1102128128 shutdown_connections 2024-05-10T11:38:55.338 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.336+0000 7fa167938640 1 --2- 172.21.15.100:0/1102128128 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa14c0b2d60 0x7fa14c0b5220 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:55.338 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.336+0000 7fa167938640 1 --2- 172.21.15.100:0/1102128128 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa16010ecd0 0x7fa1601aee60 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.336+0000 7fa167938640 1 --2- 172.21.15.100:0/1102128128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa16010c320 0x7fa1601a7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.336+0000 7fa167938640 1 --2- 172.21.15.100:0/1102128128 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1601099b0 0x7fa1601a78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:38:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.336+0000 7fa167938640 1 -- 172.21.15.100:0/1102128128 >> 172.21.15.100:0/1102128128 conn(0x7fa16006fa20 msgr2=0x7fa160109dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:38:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.338+0000 7fa167938640 1 -- 172.21.15.100:0/1102128128 shutdown_connections 2024-05-10T11:38:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:38:55.338+0000 7fa167938640 1 -- 172.21.15.100:0/1102128128 wait complete. 2024-05-10T11:38:55.945 INFO:tasks.workunit.client.0.smithi100.stdout: 10 68494 8.99 MB/sec execute 289 sec latency 104.146 ms 2024-05-10T11:38:56.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:55 smithi178 ceph-mon[33359]: pgmap v1037: 129 pgs: 129 active+clean; 526 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 811 KiB/s rd, 4.8 MiB/s wr, 260 op/s 2024-05-10T11:38:56.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:55 smithi188 ceph-mon[34075]: pgmap v1037: 129 pgs: 129 active+clean; 526 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 811 KiB/s rd, 4.8 MiB/s wr, 260 op/s 2024-05-10T11:38:56.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:55 smithi100 ceph-mon[30296]: pgmap v1037: 129 pgs: 129 active+clean; 526 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 811 KiB/s rd, 4.8 MiB/s wr, 260 op/s 2024-05-10T11:38:56.945 INFO:tasks.workunit.client.0.smithi100.stdout: 10 68658 8.97 MB/sec execute 290 sec latency 76.943 ms 2024-05-10T11:38:57.945 INFO:tasks.workunit.client.0.smithi100.stdout: 10 68835 8.97 MB/sec execute 291 sec latency 87.882 ms 2024-05-10T11:38:58.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:57 smithi178 ceph-mon[33359]: pgmap v1038: 129 pgs: 129 active+clean; 527 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 5.0 MiB/s wr, 300 op/s 2024-05-10T11:38:58.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:57 smithi188 ceph-mon[34075]: pgmap v1038: 129 pgs: 129 active+clean; 527 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 5.0 MiB/s wr, 300 op/s 2024-05-10T11:38:58.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:57 smithi100 ceph-mon[30296]: pgmap v1038: 129 pgs: 129 active+clean; 527 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 5.0 MiB/s wr, 300 op/s 2024-05-10T11:38:58.945 INFO:tasks.workunit.client.0.smithi100.stdout: 10 69007 8.97 MB/sec execute 292 sec latency 114.930 ms 2024-05-10T11:38:59.945 INFO:tasks.workunit.client.0.smithi100.stdout: 10 69212 8.99 MB/sec execute 293 sec latency 68.422 ms 2024-05-10T11:39:00.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:38:59 smithi178 ceph-mon[33359]: pgmap v1039: 129 pgs: 129 active+clean; 530 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 4.1 MiB/s wr, 276 op/s 2024-05-10T11:39:00.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:38:59 smithi188 ceph-mon[34075]: pgmap v1039: 129 pgs: 129 active+clean; 530 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 4.1 MiB/s wr, 276 op/s 2024-05-10T11:39:00.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:38:59 smithi100 ceph-mon[30296]: pgmap v1039: 129 pgs: 129 active+clean; 530 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 4.1 MiB/s wr, 276 op/s 2024-05-10T11:39:00.945 INFO:tasks.workunit.client.0.smithi100.stdout: 10 69394 8.99 MB/sec execute 294 sec latency 63.076 ms 2024-05-10T11:39:01.946 INFO:tasks.workunit.client.0.smithi100.stdout: 10 69571 8.98 MB/sec execute 295 sec latency 68.079 ms 2024-05-10T11:39:02.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:39:01 smithi178 ceph-mon[33359]: pgmap v1040: 129 pgs: 129 active+clean; 520 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 850 KiB/s rd, 4.5 MiB/s wr, 294 op/s 2024-05-10T11:39:02.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:39:01 smithi188 ceph-mon[34075]: pgmap v1040: 129 pgs: 129 active+clean; 520 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 850 KiB/s rd, 4.5 MiB/s wr, 294 op/s 2024-05-10T11:39:02.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:39:01 smithi100 ceph-mon[30296]: pgmap v1040: 129 pgs: 129 active+clean; 520 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 850 KiB/s rd, 4.5 MiB/s wr, 294 op/s 2024-05-10T11:39:02.945 INFO:tasks.workunit.client.0.smithi100.stdout: 10 69781 8.99 MB/sec execute 296 sec latency 65.111 ms 2024-05-10T11:39:03.945 INFO:tasks.workunit.client.0.smithi100.stdout: 10 69958 8.98 MB/sec execute 297 sec latency 112.576 ms 2024-05-10T11:39:04.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:39:03 smithi178 ceph-mon[33359]: pgmap v1041: 129 pgs: 129 active+clean; 515 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 880 KiB/s rd, 4.2 MiB/s wr, 314 op/s 2024-05-10T11:39:04.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:39:03 smithi188 ceph-mon[34075]: pgmap v1041: 129 pgs: 129 active+clean; 515 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 880 KiB/s rd, 4.2 MiB/s wr, 314 op/s 2024-05-10T11:39:04.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:39:03 smithi100 ceph-mon[30296]: pgmap v1041: 129 pgs: 129 active+clean; 515 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 880 KiB/s rd, 4.2 MiB/s wr, 314 op/s 2024-05-10T11:39:04.946 INFO:tasks.workunit.client.0.smithi100.stdout: 10 70156 8.99 MB/sec execute 298 sec latency 64.332 ms 2024-05-10T11:39:05.946 INFO:tasks.workunit.client.0.smithi100.stdout: 10 70315 8.99 MB/sec execute 299 sec latency 58.913 ms 2024-05-10T11:39:06.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:39:05 smithi178 ceph-mon[33359]: pgmap v1042: 129 pgs: 129 active+clean; 518 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 4.0 MiB/s wr, 309 op/s 2024-05-10T11:39:06.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:39:05 smithi188 ceph-mon[34075]: pgmap v1042: 129 pgs: 129 active+clean; 518 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 4.0 MiB/s wr, 309 op/s 2024-05-10T11:39:06.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:39:05 smithi100 ceph-mon[30296]: pgmap v1042: 129 pgs: 129 active+clean; 518 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 4.0 MiB/s wr, 309 op/s 2024-05-10T11:39:06.946 INFO:tasks.workunit.client.0.smithi100.stdout: 10 70525 9.03 MB/sec execute 300 sec latency 55.603 ms 2024-05-10T11:39:07.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:39:07 smithi178 ceph-mon[33359]: pgmap v1043: 129 pgs: 129 active+clean; 518 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 4.0 MiB/s wr, 323 op/s 2024-05-10T11:39:07.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:39:07 smithi188 ceph-mon[34075]: pgmap v1043: 129 pgs: 129 active+clean; 518 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 4.0 MiB/s wr, 323 op/s 2024-05-10T11:39:07.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:39:07 smithi100 ceph-mon[30296]: pgmap v1043: 129 pgs: 129 active+clean; 518 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 4.0 MiB/s wr, 323 op/s 2024-05-10T11:39:07.946 INFO:tasks.workunit.client.0.smithi100.stdout: 10 70757 9.04 MB/sec execute 301 sec latency 62.336 ms 2024-05-10T11:39:08.476 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.476+0000 7f3dcec72640 1 -- 172.21.15.100:0/1717981303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3dc8105e30 msgr2=0x7f3dc81062b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:08.477 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.476+0000 7f3dcec72640 1 --2- 172.21.15.100:0/1717981303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3dc8105e30 0x7f3dc81062b0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f3db8009900 tx=0x7f3db802a410 comp rx=0 tx=0).stop 2024-05-10T11:39:08.477 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.476+0000 7f3dcec72640 1 -- 172.21.15.100:0/1717981303 shutdown_connections 2024-05-10T11:39:08.477 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.476+0000 7f3dcec72640 1 --2- 172.21.15.100:0/1717981303 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3dc81067f0 0x7f3dc810d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:08.477 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.476+0000 7f3dcec72640 1 --2- 172.21.15.100:0/1717981303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3dc8105e30 0x7f3dc81062b0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:08.477 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.476+0000 7f3dcec72640 1 --2- 172.21.15.100:0/1717981303 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3dc8104c40 0x7f3dc8105060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:08.477 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.476+0000 7f3dcec72640 1 -- 172.21.15.100:0/1717981303 >> 172.21.15.100:0/1717981303 conn(0x7f3dc81001d0 msgr2=0x7f3dc81025f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:39:08.477 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.476+0000 7f3dcec72640 1 -- 172.21.15.100:0/1717981303 shutdown_connections 2024-05-10T11:39:08.477 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.476+0000 7f3dcec72640 1 -- 172.21.15.100:0/1717981303 wait complete. 2024-05-10T11:39:08.479 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.477+0000 7f3dcec72640 1 Processor -- start 2024-05-10T11:39:08.479 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.477+0000 7f3dcec72640 1 -- start start 2024-05-10T11:39:08.479 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.477+0000 7f3dcec72640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3dc8104c40 0x7f3dc819ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:08.479 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.477+0000 7f3dcec72640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3dc8105e30 0x7f3dc819b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:08.479 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.477+0000 7f3dcec72640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3dc81067f0 0x7f3dc81a22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:08.479 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.477+0000 7f3dcec72640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc81106a0 con 0x7f3dc8105e30 2024-05-10T11:39:08.479 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.477+0000 7f3dcec72640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc8110520 con 0x7f3dc8104c40 2024-05-10T11:39:08.480 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.477+0000 7f3dcec72640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc8110820 con 0x7f3dc81067f0 2024-05-10T11:39:08.480 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.477+0000 7f3dcca0e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3dc8104c40 0x7f3dc819ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:08.480 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.477+0000 7f3dbffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3dc8105e30 0x7f3dc819b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:08.480 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.477+0000 7f3dcca0e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3dc8104c40 0x7f3dc819ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:40710/0 (socket says 172.21.15.100:40710) 2024-05-10T11:39:08.480 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.477+0000 7f3dcca0e640 1 -- 172.21.15.100:0/2138245539 learned_addr learned my addr 172.21.15.100:0/2138245539 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:39:08.480 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.478+0000 7f3dcd20f640 1 --2- 172.21.15.100:0/2138245539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3dc81067f0 0x7f3dc81a22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:08.480 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.478+0000 7f3dbffff640 1 -- 172.21.15.100:0/2138245539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3dc81067f0 msgr2=0x7f3dc81a22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:08.480 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.478+0000 7f3dbffff640 1 --2- 172.21.15.100:0/2138245539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3dc81067f0 0x7f3dc81a22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:08.480 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.478+0000 7f3dbffff640 1 -- 172.21.15.100:0/2138245539 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3dc8104c40 msgr2=0x7f3dc819ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:08.480 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.478+0000 7f3dbffff640 1 --2- 172.21.15.100:0/2138245539 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3dc8104c40 0x7f3dc819ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:08.480 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.478+0000 7f3dbffff640 1 -- 172.21.15.100:0/2138245539 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dc81a2950 con 0x7f3dc8105e30 2024-05-10T11:39:08.480 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.478+0000 7f3dcd20f640 1 --2- 172.21.15.100:0/2138245539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3dc81067f0 0x7f3dc81a22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:39:08.480 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.478+0000 7f3dbffff640 1 --2- 172.21.15.100:0/2138245539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3dc8105e30 0x7f3dc819b280 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f3db8009230 tx=0x7f3db80040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:39:08.480 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.478+0000 7f3dbdffb640 1 -- 172.21.15.100:0/2138245539 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db803f030 con 0x7f3dc8105e30 2024-05-10T11:39:08.481 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.478+0000 7f3dcec72640 1 -- 172.21.15.100:0/2138245539 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dc81a2be0 con 0x7f3dc8105e30 2024-05-10T11:39:08.481 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.478+0000 7f3dbdffb640 1 -- 172.21.15.100:0/2138245539 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3db80044c0 con 0x7f3dc8105e30 2024-05-10T11:39:08.481 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.478+0000 7f3dbdffb640 1 -- 172.21.15.100:0/2138245539 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db8035070 con 0x7f3dc8105e30 2024-05-10T11:39:08.481 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.479+0000 7f3dcec72640 1 -- 172.21.15.100:0/2138245539 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dc81a30d0 con 0x7f3dc8105e30 2024-05-10T11:39:08.482 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.482+0000 7f3dbdffb640 1 -- 172.21.15.100:0/2138245539 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f3db8042030 con 0x7f3dc8105e30 2024-05-10T11:39:08.482 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.482+0000 7f3dd1992740 1 Processor -- start 2024-05-10T11:39:08.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.482+0000 7f3dd1992740 1 -- start start 2024-05-10T11:39:08.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.482+0000 7f3dd1992740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c2711edb0 0x558c2711f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:08.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.482+0000 7f3dd1992740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c2711f7a0 0x558c271242a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:08.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.482+0000 7f3dd1992740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558c27124910 0x558c27126da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:08.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.482+0000 7f3dd1992740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x558c270835d0 con 0x558c2711edb0 2024-05-10T11:39:08.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.482+0000 7f3dd1992740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558c2703c890 con 0x558c2711f7a0 2024-05-10T11:39:08.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.483+0000 7f3dd1992740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558c2707f590 con 0x558c27124910 2024-05-10T11:39:08.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.483+0000 7f3dcd20f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c2711edb0 0x558c2711f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:08.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.483+0000 7f3dcca0e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c2711f7a0 0x558c271242a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:08.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.483+0000 7f3dcd20f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c2711edb0 0x558c2711f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43560/0 (socket says 172.21.15.100:43560) 2024-05-10T11:39:08.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.483+0000 7f3dcd20f640 1 -- 172.21.15.100:0/4241418537 learned_addr learned my addr 172.21.15.100:0/4241418537 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:39:08.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.483+0000 7f3dbffff640 1 --2- 172.21.15.100:0/4241418537 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558c27124910 0x558c27126da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.483+0000 7f3dcca0e640 1 -- 172.21.15.100:0/4241418537 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558c27124910 msgr2=0x558c27126da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.483+0000 7f3dcca0e640 1 --2- 172.21.15.100:0/4241418537 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558c27124910 0x558c27126da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.483+0000 7f3dcca0e640 1 -- 172.21.15.100:0/4241418537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c2711edb0 msgr2=0x558c2711f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.483+0000 7f3dcca0e640 1 --2- 172.21.15.100:0/4241418537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c2711edb0 0x558c2711f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.483+0000 7f3dcca0e640 1 -- 172.21.15.100:0/4241418537 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c26d95210 con 0x558c2711f7a0 2024-05-10T11:39:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.483+0000 7f3dcd20f640 1 --2- 172.21.15.100:0/4241418537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c2711edb0 0x558c2711f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:39:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.483+0000 7f3dcca0e640 1 --2- 172.21.15.100:0/4241418537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c2711f7a0 0x558c271242a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f3dc4005f40 tx=0x7f3dc4009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9da4169508ff3350 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:39:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.483+0000 7f3dbdffb640 1 --2- 172.21.15.100:0/2138245539 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f3db00b28a0 0x7f3db00b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.484+0000 7f3dcd20f640 1 --2- 172.21.15.100:0/2138245539 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f3db00b28a0 0x7f3db00b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.484+0000 7f3daeffd640 1 -- 172.21.15.100:0/4241418537 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc4003970 con 0x558c2711f7a0 2024-05-10T11:39:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.484+0000 7f3daeffd640 1 -- 172.21.15.100:0/4241418537 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3dc4004020 con 0x558c2711f7a0 2024-05-10T11:39:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.484+0000 7f3daeffd640 1 -- 172.21.15.100:0/4241418537 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc400f7e0 con 0x558c2711f7a0 2024-05-10T11:39:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.484+0000 7f3dbdffb640 1 -- 172.21.15.100:0/2138245539 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f3db8102af0 con 0x7f3dc8105e30 2024-05-10T11:39:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.485+0000 7f3dd1992740 1 -- 172.21.15.100:0/4241418537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c2711f7a0 msgr2=0x558c271242a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.485+0000 7f3dd1992740 1 --2- 172.21.15.100:0/4241418537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c2711f7a0 0x558c271242a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f3dc4005f40 tx=0x7f3dc4009f90 comp rx=0 tx=0).stop 2024-05-10T11:39:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.485+0000 7f3dcd20f640 1 --2- 172.21.15.100:0/2138245539 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f3db00b28a0 0x7f3db00b4d60 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f3dc0006fd0 tx=0x7f3dc0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:39:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.485+0000 7f3dd1992740 1 -- 172.21.15.100:0/4241418537 shutdown_connections 2024-05-10T11:39:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.485+0000 7f3dd1992740 1 --2- 172.21.15.100:0/4241418537 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558c27124910 0x558c27126da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.485+0000 7f3dd1992740 1 --2- 172.21.15.100:0/4241418537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c2711f7a0 0x558c271242a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.485+0000 7f3dd1992740 1 --2- 172.21.15.100:0/4241418537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c2711edb0 0x558c2711f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.485+0000 7f3dd1992740 1 -- 172.21.15.100:0/4241418537 >> 172.21.15.100:0/4241418537 conn(0x558c2711a420 msgr2=0x558c2711a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:39:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.485+0000 7f3dd1992740 1 -- 172.21.15.100:0/4241418537 shutdown_connections 2024-05-10T11:39:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.485+0000 7f3dd1992740 1 -- 172.21.15.100:0/4241418537 wait complete. 2024-05-10T11:39:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.485+0000 7f3dd1992740 1 -- start start 2024-05-10T11:39:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.485+0000 7f3dd1992740 1 Processor -- start 2024-05-10T11:39:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.486+0000 7f3dd1992740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c2711edb0 0x558c27128f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.486+0000 7f3dd1992740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c2711f7a0 0x558c27129480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.486+0000 7f3dd1992740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558c27124910 0x558c271304c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.486+0000 7f3dd1992740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x558c271278c0 con 0x558c2711edb0 2024-05-10T11:39:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.486+0000 7f3dcca0e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c2711edb0 0x558c27128f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.486+0000 7f3dcca0e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c2711edb0 0x558c27128f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43562/0 (socket says 172.21.15.100:43562) 2024-05-10T11:39:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.486+0000 7f3dcca0e640 1 -- 172.21.15.100:0/4133120494 learned_addr learned my addr 172.21.15.100:0/4133120494 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:39:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.486+0000 7f3dd1992740 1 -- 172.21.15.100:0/4133120494 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558c27040370 con 0x558c2711f7a0 2024-05-10T11:39:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.486+0000 7f3dd1992740 1 -- 172.21.15.100:0/4133120494 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558c27127a40 con 0x558c27124910 2024-05-10T11:39:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.486+0000 7f3dbffff640 1 --2- 172.21.15.100:0/4133120494 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c2711f7a0 0x558c27129480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.486+0000 7f3dcd20f640 1 --2- 172.21.15.100:0/4133120494 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558c27124910 0x558c271304c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.486+0000 7f3dd1992740 10 client.0 _ll_register_callbacks cb 0x558c26d94a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:39:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.486+0000 7f3dcca0e640 1 -- 172.21.15.100:0/4133120494 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558c27124910 msgr2=0x558c271304c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.486+0000 7f3dcca0e640 1 --2- 172.21.15.100:0/4133120494 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558c27124910 0x558c271304c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.486+0000 7f3dcca0e640 1 -- 172.21.15.100:0/4133120494 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c2711f7a0 msgr2=0x558c27129480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.486+0000 7f3dcca0e640 1 --2- 172.21.15.100:0/4133120494 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c2711f7a0 0x558c27129480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:08.492 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.486+0000 7f3dcca0e640 1 -- 172.21.15.100:0/4133120494 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c26e61620 con 0x558c2711edb0 2024-05-10T11:39:08.492 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.486+0000 7f3dbffff640 1 --2- 172.21.15.100:0/4133120494 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c2711f7a0 0x558c27129480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:39:08.492 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.486+0000 7f3dcca0e640 1 --2- 172.21.15.100:0/4133120494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c2711edb0 0x558c27128f40 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f3dc4005c40 tx=0x7f3dc4003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b86da36b6a5f942 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:39:08.492 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.486+0000 7f3da7fff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:39:08.492 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.487+0000 7f3da7fff640 1 -- 172.21.15.100:0/4133120494 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc4028020 con 0x558c2711edb0 2024-05-10T11:39:08.492 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.487+0000 7f3da7fff640 1 -- 172.21.15.100:0/4133120494 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f3dc4016040 con 0x558c2711edb0 2024-05-10T11:39:08.492 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.487+0000 7f3dd1992740 1 -- 172.21.15.100:0/4133120494 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3dc4004020 con 0x558c2711edb0 2024-05-10T11:39:08.492 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.487+0000 7f3da7fff640 1 -- 172.21.15.100:0/4133120494 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f3dc4033050 con 0x558c2711edb0 2024-05-10T11:39:08.492 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.487+0000 7f3dd1992740 10 client.19482 fetch_fsmap learned FSMap version 41 2024-05-10T11:39:08.492 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.487+0000 7f3dd1992740 1 -- 172.21.15.100:0/4133120494 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x558c27133480 con 0x558c2711edb0 2024-05-10T11:39:08.492 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.487+0000 7f3da7fff640 1 -- 172.21.15.100:0/4133120494 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3dc400fa30 con 0x558c2711edb0 2024-05-10T11:39:08.492 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.487+0000 7f3da7fff640 1 -- 172.21.15.100:0/4133120494 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f3dc4023450 con 0x558c2711edb0 2024-05-10T11:39:08.492 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.487+0000 7f3dd1992740 10 client.19482 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:39:08.492 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.487+0000 7f3dd1992740 10 client.19482 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:39:08.493 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.487+0000 7f3dd1992740 20 client.19482 populate_metadata read hostname 'smithi100' 2024-05-10T11:39:08.493 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.487+0000 7f3dd1992740 1 --2- 172.21.15.100:0/4133120494 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558c27134940 0x558c27136d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:08.493 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.488+0000 7f3dd1992740 4 client.19482 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:39:08.493 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.488+0000 7f3dd1992740 1 -- 172.21.15.100:0/4133120494 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f3dc4004020 con 0x558c27134940 2024-05-10T11:39:08.493 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.488+0000 7f3dbffff640 1 --2- 172.21.15.100:0/4133120494 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558c27134940 0x558c27136d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:08.493 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.488+0000 7f3dbffff640 1 --2- 172.21.15.100:0/4133120494 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558c27134940 0x558c27136d70 crc :-1 s=READY pgs=395 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c2cfd8e2ac29bd7 server_cookie=486dbc1b16fb81b2 in_seq=0 out_seq=0 2024-05-10T11:39:08.493 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.488+0000 7f3da7fff640 10 client.19482 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:39:08.493 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.489+0000 7f3da7fff640 1 -- 172.21.15.100:0/4133120494 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f3dc4004020 con 0x558c27134940 2024-05-10T11:39:08.493 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.489+0000 7f3da7fff640 10 client.19482 handle_command_reply: tid=0 2024-05-10T11:39:08.493 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.489+0000 7f3dd1992740 1 client.19482 shutdown 2024-05-10T11:39:08.493 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.489+0000 7f3dd1992740 1 -- 172.21.15.100:0/4133120494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c2711edb0 msgr2=0x558c27128f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:08.493 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.489+0000 7f3dd1992740 1 --2- 172.21.15.100:0/4133120494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c2711edb0 0x558c27128f40 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f3dc4005c40 tx=0x7f3dc4003ba0 comp rx=0 tx=0).stop 2024-05-10T11:39:08.493 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.489+0000 7f3dd1992740 1 -- 172.21.15.100:0/4133120494 shutdown_connections 2024-05-10T11:39:08.493 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.489+0000 7f3dd1992740 1 --2- 172.21.15.100:0/4133120494 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558c27134940 0x558c27136d70 crc :-1 s=READY pgs=395 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:08.493 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.489+0000 7f3dd1992740 1 --2- 172.21.15.100:0/4133120494 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558c27124910 0x558c271304c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:08.494 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.489+0000 7f3dd1992740 1 --2- 172.21.15.100:0/4133120494 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c2711f7a0 0x558c27129480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:08.494 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.489+0000 7f3dd1992740 1 --2- 172.21.15.100:0/4133120494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c2711edb0 0x558c27128f40 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:08.494 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.489+0000 7f3dd1992740 1 -- 172.21.15.100:0/4133120494 >> 172.21.15.100:0/4133120494 conn(0x558c2711a420 msgr2=0x558c27119580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:39:08.494 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.489+0000 7f3da7fff640 0 client.19482 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:39:08.494 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.489+0000 7f3dd1992740 1 -- 172.21.15.100:0/4133120494 shutdown_connections 2024-05-10T11:39:08.494 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.489+0000 7f3dd1992740 1 -- 172.21.15.100:0/4133120494 wait complete. 2024-05-10T11:39:08.494 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.489+0000 7f3dd1992740 20 client.19482 trim_cache size 0 max 16384 2024-05-10T11:39:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.511+0000 7f3dd1992740 1 Processor -- start 2024-05-10T11:39:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.511+0000 7f3dd1992740 1 -- start start 2024-05-10T11:39:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.511+0000 7f3dd1992740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c2711edb0 0x558c2711d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.511+0000 7f3dd1992740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c2711f7a0 0x558c27130480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.512+0000 7f3dd1992740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558c27124910 0x558c2711d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.512+0000 7f3dd1992740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc81afe60 con 0x558c2711f7a0 2024-05-10T11:39:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.512+0000 7f3dd1992740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc81ad730 con 0x558c2711edb0 2024-05-10T11:39:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.512+0000 7f3dcd20f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558c27124910 0x558c2711d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.512+0000 7f3dcca0e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c2711edb0 0x558c2711d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:08.513 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.512+0000 7f3dbffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c2711f7a0 0x558c27130480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:08.513 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.512+0000 7f3dcca0e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c2711edb0 0x558c2711d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:40716/0 (socket says 172.21.15.100:40716) 2024-05-10T11:39:08.513 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.512+0000 7f3dcca0e640 1 -- 172.21.15.100:0/2589188961 learned_addr learned my addr 172.21.15.100:0/2589188961 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:39:08.513 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.512+0000 7f3dd1992740 1 -- 172.21.15.100:0/2589188961 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc81ab580 con 0x558c27124910 2024-05-10T11:39:08.513 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.512+0000 7f3dbffff640 1 -- 172.21.15.100:0/2589188961 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558c27124910 msgr2=0x558c2711d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:08.514 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.512+0000 7f3dbffff640 1 --2- 172.21.15.100:0/2589188961 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558c27124910 0x558c2711d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:08.514 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.512+0000 7f3dbffff640 1 -- 172.21.15.100:0/2589188961 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c2711edb0 msgr2=0x558c2711d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:08.514 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.512+0000 7f3dbffff640 1 --2- 172.21.15.100:0/2589188961 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c2711edb0 0x558c2711d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:08.514 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.512+0000 7f3dbffff640 1 -- 172.21.15.100:0/2589188961 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c27123b90 con 0x558c2711f7a0 2024-05-10T11:39:08.514 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.513+0000 7f3dcca0e640 1 --2- 172.21.15.100:0/2589188961 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c2711edb0 0x558c2711d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:39:08.514 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.513+0000 7f3dbffff640 1 --2- 172.21.15.100:0/2589188961 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c2711f7a0 0x558c27130480 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f3db8000c00 tx=0x7f3db8104f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66a4ed0110f5b6d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:39:08.514 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.513+0000 7f3dae7fc640 1 -- 172.21.15.100:0/2589188961 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db80ae070 con 0x558c2711f7a0 2024-05-10T11:39:08.514 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.513+0000 7f3dae7fc640 1 -- 172.21.15.100:0/2589188961 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3db80c9860 con 0x558c2711f7a0 2024-05-10T11:39:08.514 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.513+0000 7f3dae7fc640 1 -- 172.21.15.100:0/2589188961 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db8049040 con 0x558c2711f7a0 2024-05-10T11:39:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.514+0000 7f3dd1992740 1 -- 172.21.15.100:0/2589188961 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c2711f7a0 msgr2=0x558c27130480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.514+0000 7f3dd1992740 1 --2- 172.21.15.100:0/2589188961 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c2711f7a0 0x558c27130480 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f3db8000c00 tx=0x7f3db8104f40 comp rx=0 tx=0).stop 2024-05-10T11:39:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.514+0000 7f3dd1992740 1 -- 172.21.15.100:0/2589188961 shutdown_connections 2024-05-10T11:39:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.514+0000 7f3dd1992740 1 --2- 172.21.15.100:0/2589188961 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558c27124910 0x558c2711d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.514+0000 7f3dd1992740 1 --2- 172.21.15.100:0/2589188961 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c2711f7a0 0x558c27130480 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.514+0000 7f3dd1992740 1 --2- 172.21.15.100:0/2589188961 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c2711edb0 0x558c2711d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.514+0000 7f3dd1992740 1 -- 172.21.15.100:0/2589188961 >> 172.21.15.100:0/2589188961 conn(0x558c2711a420 msgr2=0x558c27120920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:39:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.514+0000 7f3dd1992740 1 -- 172.21.15.100:0/2589188961 shutdown_connections 2024-05-10T11:39:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.514+0000 7f3dd1992740 1 -- 172.21.15.100:0/2589188961 wait complete. 2024-05-10T11:39:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.515+0000 7f3dd1992740 1 -- start start 2024-05-10T11:39:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.515+0000 7f3dd1992740 1 Processor -- start 2024-05-10T11:39:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.515+0000 7f3dd1992740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c2711edb0 0x558c2712d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.515+0000 7f3dd1992740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558c2711f7a0 0x558c2712da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.515+0000 7f3dd1992740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c27124910 0x558c2712ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.515+0000 7f3dd1992740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc81abbf0 con 0x558c2711edb0 2024-05-10T11:39:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.515+0000 7f3dd1992740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc81ad060 con 0x558c27124910 2024-05-10T11:39:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.515+0000 7f3dd1992740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc81ac2c0 con 0x558c2711f7a0 2024-05-10T11:39:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.515+0000 7f3dd1992740 10 client.0 _ll_register_callbacks cb 0x7f3dc4004570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:39:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.516+0000 7f3dcd20f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c27124910 0x558c2712ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.516+0000 7f3dcca0e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c2711edb0 0x558c2712d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.516+0000 7f3dcd20f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c27124910 0x558c2712ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:40728/0 (socket says 172.21.15.100:40728) 2024-05-10T11:39:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.516+0000 7f3dcd20f640 1 -- 172.21.15.100:0/2314572982 learned_addr learned my addr 172.21.15.100:0/2314572982 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:39:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.516+0000 7f3dbffff640 1 --2- 172.21.15.100:0/2314572982 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558c2711f7a0 0x558c2712da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.516+0000 7f3dcd20f640 1 -- 172.21.15.100:0/2314572982 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558c2711f7a0 msgr2=0x558c2712da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.516+0000 7f3dcd20f640 1 --2- 172.21.15.100:0/2314572982 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558c2711f7a0 0x558c2712da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.516+0000 7f3dcd20f640 1 -- 172.21.15.100:0/2314572982 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c2711edb0 msgr2=0x558c2712d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.516+0000 7f3dcd20f640 1 --2- 172.21.15.100:0/2314572982 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c2711edb0 0x558c2712d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.516+0000 7f3dcd20f640 1 -- 172.21.15.100:0/2314572982 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c2712f050 con 0x558c27124910 2024-05-10T11:39:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.516+0000 7f3dbffff640 1 --2- 172.21.15.100:0/2314572982 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558c2711f7a0 0x558c2712da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:39:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.516+0000 7f3dcd20f640 1 --2- 172.21.15.100:0/2314572982 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c27124910 0x558c2712ea60 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3dc0016730 tx=0x7f3dc00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30892855b74a1e4c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:39:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.516+0000 7f3da7fff640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:39:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.516+0000 7f3da7fff640 1 -- 172.21.15.100:0/2314572982 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc001c040 con 0x558c27124910 2024-05-10T11:39:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.517+0000 7f3da7fff640 1 -- 172.21.15.100:0/2314572982 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f3dc0014840 con 0x558c27124910 2024-05-10T11:39:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.517+0000 7f3dd1992740 1 -- 172.21.15.100:0/2314572982 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3db80c9860 con 0x558c27124910 2024-05-10T11:39:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.517+0000 7f3da7fff640 1 -- 172.21.15.100:0/2314572982 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f3dc0035050 con 0x558c27124910 2024-05-10T11:39:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.517+0000 7f3dd1992740 10 client.28754 fetch_fsmap learned FSMap version 41 2024-05-10T11:39:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.517+0000 7f3dd1992740 1 -- 172.21.15.100:0/2314572982 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x558c271303d0 con 0x558c27124910 2024-05-10T11:39:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.517+0000 7f3da7fff640 1 -- 172.21.15.100:0/2314572982 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3db80c9860 con 0x558c27124910 2024-05-10T11:39:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.517+0000 7f3da7fff640 1 -- 172.21.15.100:0/2314572982 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f3dc0024b30 con 0x558c27124910 2024-05-10T11:39:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.517+0000 7f3dd1992740 10 client.28754 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:39:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.518+0000 7f3dd1992740 10 client.28754 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:39:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.518+0000 7f3dd1992740 20 client.28754 populate_metadata read hostname 'smithi100' 2024-05-10T11:39:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.518+0000 7f3dd1992740 1 --2- 172.21.15.100:0/2314572982 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558c2719f670 0x558c271a1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.518+0000 7f3dd1992740 4 client.28754 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:39:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.518+0000 7f3dd1992740 1 -- 172.21.15.100:0/2314572982 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558c270911e0 con 0x558c2719f670 2024-05-10T11:39:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.518+0000 7f3dcca0e640 1 --2- 172.21.15.100:0/2314572982 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558c2719f670 0x558c271a1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.518+0000 7f3da7fff640 10 client.28754 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:39:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.518+0000 7f3dcca0e640 1 --2- 172.21.15.100:0/2314572982 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558c2719f670 0x558c271a1a90 crc :-1 s=READY pgs=396 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7946f2ea5cd1fbee server_cookie=aedd2c5739a216b4 in_seq=0 out_seq=0 2024-05-10T11:39:08.660 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.659+0000 7f3da7fff640 1 -- 172.21.15.100:0/2314572982 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558c270911e0 con 0x558c2719f670 2024-05-10T11:39:08.660 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.659+0000 7f3da7fff640 10 client.28754 handle_command_reply: tid=0 2024-05-10T11:39:08.661 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.659+0000 7f3dd1992740 1 client.28754 shutdown 2024-05-10T11:39:08.661 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.660+0000 7f3dd1992740 1 -- 172.21.15.100:0/2314572982 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c27124910 msgr2=0x558c2712ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:08.661 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.660+0000 7f3dd1992740 1 --2- 172.21.15.100:0/2314572982 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c27124910 0x558c2712ea60 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3dc0016730 tx=0x7f3dc00046c0 comp rx=0 tx=0).stop 2024-05-10T11:39:08.661 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.660+0000 7f3dd1992740 1 -- 172.21.15.100:0/2314572982 shutdown_connections 2024-05-10T11:39:08.661 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.660+0000 7f3dd1992740 1 --2- 172.21.15.100:0/2314572982 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558c2719f670 0x558c271a1a90 crc :-1 s=READY pgs=396 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:08.661 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.660+0000 7f3dd1992740 1 --2- 172.21.15.100:0/2314572982 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c27124910 0x558c2712ea60 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:08.661 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.660+0000 7f3dd1992740 1 --2- 172.21.15.100:0/2314572982 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558c2711f7a0 0x558c2712da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:08.661 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.660+0000 7f3da7fff640 0 client.28754 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:39:08.661 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.660+0000 7f3dd1992740 1 --2- 172.21.15.100:0/2314572982 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c2711edb0 0x558c2712d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:08.661 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.660+0000 7f3dd1992740 1 -- 172.21.15.100:0/2314572982 >> 172.21.15.100:0/2314572982 conn(0x558c2711a420 msgr2=0x558c2711d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:39:08.661 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.660+0000 7f3dd1992740 1 -- 172.21.15.100:0/2314572982 shutdown_connections 2024-05-10T11:39:08.661 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.660+0000 7f3dd1992740 1 -- 172.21.15.100:0/2314572982 wait complete. 2024-05-10T11:39:08.661 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.660+0000 7f3dd1992740 20 client.28754 trim_cache size 0 max 16384 2024-05-10T11:39:08.662 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:39:08.662 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:39:08.662 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:39:08.662 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:39:08.663 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.661+0000 7f3dcec72640 1 -- 172.21.15.100:0/2138245539 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f3db00b28a0 msgr2=0x7f3db00b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:08.663 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.661+0000 7f3dcec72640 1 --2- 172.21.15.100:0/2138245539 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f3db00b28a0 0x7f3db00b4d60 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f3dc0006fd0 tx=0x7f3dc0008040 comp rx=0 tx=0).stop 2024-05-10T11:39:08.663 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.661+0000 7f3dcec72640 1 -- 172.21.15.100:0/2138245539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3dc8105e30 msgr2=0x7f3dc819b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:08.663 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.661+0000 7f3dcec72640 1 --2- 172.21.15.100:0/2138245539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3dc8105e30 0x7f3dc819b280 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f3db8009230 tx=0x7f3db80040c0 comp rx=0 tx=0).stop 2024-05-10T11:39:08.663 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.661+0000 7f3dcec72640 1 -- 172.21.15.100:0/2138245539 shutdown_connections 2024-05-10T11:39:08.663 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.661+0000 7f3dcec72640 1 --2- 172.21.15.100:0/2138245539 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f3db00b28a0 0x7f3db00b4d60 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:08.663 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.661+0000 7f3dcec72640 1 --2- 172.21.15.100:0/2138245539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3dc81067f0 0x7f3dc81a22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:08.663 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.661+0000 7f3dcec72640 1 --2- 172.21.15.100:0/2138245539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3dc8105e30 0x7f3dc819b280 secure :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f3db8009230 tx=0x7f3db80040c0 comp rx=0 tx=0).stop 2024-05-10T11:39:08.663 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.661+0000 7f3dcec72640 1 --2- 172.21.15.100:0/2138245539 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3dc8104c40 0x7f3dc819ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:08.663 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.661+0000 7f3dcec72640 1 -- 172.21.15.100:0/2138245539 >> 172.21.15.100:0/2138245539 conn(0x7f3dc81001d0 msgr2=0x7f3dc81091a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:39:08.663 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.661+0000 7f3dcec72640 1 -- 172.21.15.100:0/2138245539 shutdown_connections 2024-05-10T11:39:08.663 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:08.661+0000 7f3dcec72640 1 -- 172.21.15.100:0/2138245539 wait complete. 2024-05-10T11:39:08.946 INFO:tasks.workunit.client.0.smithi100.stdout: 10 70930 9.05 MB/sec execute 302 sec latency 117.138 ms 2024-05-10T11:39:09.946 INFO:tasks.workunit.client.0.smithi100.stdout: 10 71051 9.05 MB/sec execute 303 sec latency 72.727 ms 2024-05-10T11:39:10.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:39:09 smithi178 ceph-mon[33359]: pgmap v1044: 129 pgs: 129 active+clean; 516 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 920 KiB/s rd, 3.9 MiB/s wr, 294 op/s 2024-05-10T11:39:10.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:39:09 smithi188 ceph-mon[34075]: pgmap v1044: 129 pgs: 129 active+clean; 516 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 920 KiB/s rd, 3.9 MiB/s wr, 294 op/s 2024-05-10T11:39:10.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:39:09 smithi100 ceph-mon[30296]: pgmap v1044: 129 pgs: 129 active+clean; 516 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 920 KiB/s rd, 3.9 MiB/s wr, 294 op/s 2024-05-10T11:39:10.946 INFO:tasks.workunit.client.0.smithi100.stdout: 10 71244 9.06 MB/sec execute 304 sec latency 60.692 ms 2024-05-10T11:39:11.046 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T11:39:11.406 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:39:11.868 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:39:11 smithi100 ceph-mon[30296]: pgmap v1045: 129 pgs: 129 active+clean; 520 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 4.4 MiB/s wr, 278 op/s 2024-05-10T11:39:11.946 INFO:tasks.workunit.client.0.smithi100.stdout: 10 71418 9.08 MB/sec execute 305 sec latency 69.266 ms 2024-05-10T11:39:12.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:39:11 smithi178 ceph-mon[33359]: pgmap v1045: 129 pgs: 129 active+clean; 520 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 4.4 MiB/s wr, 278 op/s 2024-05-10T11:39:12.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:39:11 smithi188 ceph-mon[34075]: pgmap v1045: 129 pgs: 129 active+clean; 520 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 4.4 MiB/s wr, 278 op/s 2024-05-10T11:39:12.946 INFO:tasks.workunit.client.0.smithi100.stdout: 10 71547 9.07 MB/sec execute 306 sec latency 78.419 ms 2024-05-10T11:39:13.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.584+0000 7f8fc5149700 1 -- 172.21.15.100:0/340923357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fc0104100 msgr2=0x7f8fc0108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:13.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.584+0000 7f8fc5149700 1 --2- 172.21.15.100:0/340923357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fc0104100 0x7f8fc0108120 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4009990 tx=0x7f8fb4009ca0 comp rx=0 tx=0).stop 2024-05-10T11:39:13.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.584+0000 7f8fc5149700 1 -- 172.21.15.100:0/340923357 shutdown_connections 2024-05-10T11:39:13.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.584+0000 7f8fc5149700 1 --2- 172.21.15.100:0/340923357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8fc0108850 0x7f8fc01103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:13.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.584+0000 7f8fc5149700 1 --2- 172.21.15.100:0/340923357 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8fc0103750 0x7f8fc0103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:13.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.584+0000 7f8fc5149700 1 --2- 172.21.15.100:0/340923357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fc0104100 0x7f8fc0108120 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:13.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.584+0000 7f8fc5149700 1 -- 172.21.15.100:0/340923357 >> 172.21.15.100:0/340923357 conn(0x7f8fc007a600 msgr2=0x7f8fc007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:39:13.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.584+0000 7f8fc5149700 1 -- 172.21.15.100:0/340923357 shutdown_connections 2024-05-10T11:39:13.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.584+0000 7f8fc5149700 1 -- 172.21.15.100:0/340923357 wait complete. 2024-05-10T11:39:13.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.585+0000 7f8fc5149700 1 Processor -- start 2024-05-10T11:39:13.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.585+0000 7f8fc5149700 1 -- start start 2024-05-10T11:39:13.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.586+0000 7f8fc5149700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8fc0103750 0x7f8fc019f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:13.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.586+0000 7f8fc5149700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fc0104100 0x7f8fc019fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:13.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.586+0000 7f8fc5149700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8fc0108850 0x7f8fc0199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:13.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.586+0000 7f8fc5149700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc0113510 con 0x7f8fc0103750 2024-05-10T11:39:13.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.586+0000 7f8fc5149700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc0113390 con 0x7f8fc0108850 2024-05-10T11:39:13.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.586+0000 7f8fc5149700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc0113690 con 0x7f8fc0104100 2024-05-10T11:39:13.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.586+0000 7f8fbed9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8fc0103750 0x7f8fc019f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:13.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.586+0000 7f8fbf59e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8fc0108850 0x7f8fc0199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:13.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.586+0000 7f8fbed9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8fc0103750 0x7f8fc019f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43604/0 (socket says 172.21.15.100:43604) 2024-05-10T11:39:13.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.586+0000 7f8fbed9d700 1 -- 172.21.15.100:0/1595540348 learned_addr learned my addr 172.21.15.100:0/1595540348 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:39:13.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.586+0000 7f8fbe59c700 1 --2- 172.21.15.100:0/1595540348 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fc0104100 0x7f8fc019fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:13.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.586+0000 7f8fbf59e700 1 -- 172.21.15.100:0/1595540348 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fc0104100 msgr2=0x7f8fc019fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:13.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.586+0000 7f8fbf59e700 1 --2- 172.21.15.100:0/1595540348 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fc0104100 0x7f8fc019fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:13.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.586+0000 7f8fbf59e700 1 -- 172.21.15.100:0/1595540348 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8fc0103750 msgr2=0x7f8fc019f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:13.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.586+0000 7f8fbf59e700 1 --2- 172.21.15.100:0/1595540348 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8fc0103750 0x7f8fc019f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:13.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.586+0000 7f8fbf59e700 1 -- 172.21.15.100:0/1595540348 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fc0199ff0 con 0x7f8fc0108850 2024-05-10T11:39:13.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.587+0000 7f8fbf59e700 1 --2- 172.21.15.100:0/1595540348 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8fc0108850 0x7f8fc0199890 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8fb800efc0 tx=0x7f8fb800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:39:13.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.587+0000 7f8fa7fff700 1 -- 172.21.15.100:0/1595540348 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb80092e0 con 0x7f8fc0108850 2024-05-10T11:39:13.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.587+0000 7f8fc5149700 1 -- 172.21.15.100:0/1595540348 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fc019a280 con 0x7f8fc0108850 2024-05-10T11:39:13.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.587+0000 7f8fc5149700 1 -- 172.21.15.100:0/1595540348 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fc01a65a0 con 0x7f8fc0108850 2024-05-10T11:39:13.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.587+0000 7f8fa7fff700 1 -- 172.21.15.100:0/1595540348 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8fb8004bb0 con 0x7f8fc0108850 2024-05-10T11:39:13.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.587+0000 7f8fa7fff700 1 -- 172.21.15.100:0/1595540348 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb8004020 con 0x7f8fc0108850 2024-05-10T11:39:13.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.591+0000 7f8fa7fff700 1 -- 172.21.15.100:0/1595540348 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f8fb801b030 con 0x7f8fc0108850 2024-05-10T11:39:13.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.591+0000 7f8fcf9dcb80 1 Processor -- start 2024-05-10T11:39:13.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.591+0000 7f8fcf9dcb80 1 -- start start 2024-05-10T11:39:13.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.591+0000 7f8fcf9dcb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56359ee2c1b0 0x56359ee2c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:13.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.592+0000 7f8fcf9dcb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56359ee2cad0 0x56359ee26810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:13.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.592+0000 7f8fcf9dcb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56359ee26d50 0x56359ee27190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:13.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.592+0000 7f8fbe59c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56359ee2c1b0 0x56359ee2c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:13.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.592+0000 7f8fbe59c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56359ee2c1b0 0x56359ee2c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:53896/0 (socket says 172.21.15.100:53896) 2024-05-10T11:39:13.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.592+0000 7f8fbe59c700 1 -- 172.21.15.100:0/1414125081 learned_addr learned my addr 172.21.15.100:0/1414125081 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:39:13.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.592+0000 7f8fbed9d700 1 --2- 172.21.15.100:0/1414125081 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56359ee2cad0 0x56359ee26810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:13.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.592+0000 7f8fcf9dcb80 1 -- 172.21.15.100:0/1414125081 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56359ed78940 con 0x56359ee26d50 2024-05-10T11:39:13.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.592+0000 7f8fcf9dcb80 1 -- 172.21.15.100:0/1414125081 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56359ecddda0 con 0x56359ee2cad0 2024-05-10T11:39:13.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.592+0000 7f8fcf9dcb80 1 -- 172.21.15.100:0/1414125081 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56359ed74900 con 0x56359ee2c1b0 2024-05-10T11:39:13.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.592+0000 7f8fbe59c700 1 --2- 172.21.15.100:0/1414125081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56359ee26d50 0x56359ee27190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:13.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.592+0000 7f8fbe59c700 1 -- 172.21.15.100:0/1414125081 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56359ee2cad0 msgr2=0x56359ee26810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:13.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.592+0000 7f8fbe59c700 1 --2- 172.21.15.100:0/1414125081 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56359ee2cad0 0x56359ee26810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:13.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.592+0000 7f8fbe59c700 1 -- 172.21.15.100:0/1414125081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56359ee26d50 msgr2=0x56359ee27190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:13.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.592+0000 7f8fbe59c700 1 --2- 172.21.15.100:0/1414125081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56359ee26d50 0x56359ee27190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:13.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.592+0000 7f8fbe59c700 1 -- 172.21.15.100:0/1414125081 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56359ecc58d0 con 0x56359ee2c1b0 2024-05-10T11:39:13.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.593+0000 7f8fbe59c700 1 --2- 172.21.15.100:0/1414125081 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56359ee2c1b0 0x56359ee2c590 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4006010 tx=0x7f8fb4005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b78577a758d2165b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:39:13.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.593+0000 7f8fa57fa700 1 -- 172.21.15.100:0/1414125081 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb4021040 con 0x56359ee2c1b0 2024-05-10T11:39:13.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.593+0000 7f8fa57fa700 1 -- 172.21.15.100:0/1414125081 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8fb400bb40 con 0x56359ee2c1b0 2024-05-10T11:39:13.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.593+0000 7f8fa57fa700 1 -- 172.21.15.100:0/1414125081 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb400f970 con 0x56359ee2c1b0 2024-05-10T11:39:13.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.593+0000 7f8fa7fff700 1 --2- 172.21.15.100:0/1595540348 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8fa80b2900 0x7f8fa80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:13.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.593+0000 7f8fbed9d700 1 --2- 172.21.15.100:0/1595540348 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8fa80b2900 0x7f8fa80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:13.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.593+0000 7f8fa7fff700 1 -- 172.21.15.100:0/1595540348 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f8fb80e1f60 con 0x7f8fc0108850 2024-05-10T11:39:13.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.594+0000 7f8fbed9d700 1 --2- 172.21.15.100:0/1595540348 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8fa80b2900 0x7f8fa80b4dc0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x56359ee23450 tx=0x7f8fb000b500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:39:13.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.594+0000 7f8fcf9dcb80 1 -- 172.21.15.100:0/1414125081 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56359ee2c1b0 msgr2=0x56359ee2c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:13.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.594+0000 7f8fcf9dcb80 1 --2- 172.21.15.100:0/1414125081 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56359ee2c1b0 0x56359ee2c590 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4006010 tx=0x7f8fb4005e80 comp rx=0 tx=0).stop 2024-05-10T11:39:13.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.594+0000 7f8fcf9dcb80 1 -- 172.21.15.100:0/1414125081 shutdown_connections 2024-05-10T11:39:13.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.594+0000 7f8fcf9dcb80 1 --2- 172.21.15.100:0/1414125081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56359ee26d50 0x56359ee27190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:13.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.594+0000 7f8fcf9dcb80 1 --2- 172.21.15.100:0/1414125081 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56359ee2c1b0 0x56359ee2c590 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:13.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.594+0000 7f8fcf9dcb80 1 --2- 172.21.15.100:0/1414125081 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56359ee2cad0 0x56359ee26810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:13.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.594+0000 7f8fcf9dcb80 1 -- 172.21.15.100:0/1414125081 >> 172.21.15.100:0/1414125081 conn(0x56359ed52b70 msgr2=0x56359ee212a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:39:13.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.594+0000 7f8fcf9dcb80 1 -- 172.21.15.100:0/1414125081 shutdown_connections 2024-05-10T11:39:13.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.594+0000 7f8fcf9dcb80 1 -- 172.21.15.100:0/1414125081 wait complete. 2024-05-10T11:39:13.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.595+0000 7f8fcf9dcb80 1 -- start start 2024-05-10T11:39:13.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.595+0000 7f8fcf9dcb80 1 Processor -- start 2024-05-10T11:39:13.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.596+0000 7f8fcf9dcb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56359ee26d50 0x56359ee32700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:13.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.596+0000 7f8fcf9dcb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56359ee2c1b0 0x56359ee2fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:13.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.596+0000 7f8fbed9d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56359ee26d50 0x56359ee32700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:13.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.596+0000 7f8fbed9d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56359ee26d50 0x56359ee32700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:53900/0 (socket says 172.21.15.100:53900) 2024-05-10T11:39:13.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.596+0000 7f8fbe59c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56359ee2c1b0 0x56359ee2fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:13.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.596+0000 7f8fbe59c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56359ee2c1b0 0x56359ee2fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:40756/0 (socket says 172.21.15.100:40756) 2024-05-10T11:39:13.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.596+0000 7f8fbed9d700 1 -- 172.21.15.100:0/1926355627 learned_addr learned my addr 172.21.15.100:0/1926355627 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:39:13.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.596+0000 7f8fcf9dcb80 1 --2- 172.21.15.100:0/1926355627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56359ee2cad0 0x56359ee30050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:13.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.596+0000 7f8fcf9dcb80 1 -- 172.21.15.100:0/1926355627 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56359ee37800 con 0x56359ee2cad0 2024-05-10T11:39:13.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.596+0000 7f8fcf9dcb80 1 -- 172.21.15.100:0/1926355627 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56359ed31ed0 con 0x56359ee2c1b0 2024-05-10T11:39:13.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.596+0000 7f8fcf9dcb80 1 -- 172.21.15.100:0/1926355627 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56359ee37980 con 0x56359ee26d50 2024-05-10T11:39:13.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.596+0000 7f8fbf59e700 1 --2- 172.21.15.100:0/1926355627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56359ee2cad0 0x56359ee30050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:13.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.596+0000 7f8fbed9d700 1 -- 172.21.15.100:0/1926355627 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56359ee2c1b0 msgr2=0x56359ee2fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:13.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.596+0000 7f8fbed9d700 1 --2- 172.21.15.100:0/1926355627 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56359ee2c1b0 0x56359ee2fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:13.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.596+0000 7f8fbed9d700 1 -- 172.21.15.100:0/1926355627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56359ee2cad0 msgr2=0x56359ee30050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:13.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.596+0000 7f8fbed9d700 1 --2- 172.21.15.100:0/1926355627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56359ee2cad0 0x56359ee30050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:13.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.597+0000 7f8fbed9d700 1 -- 172.21.15.100:0/1926355627 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56359ed2f760 con 0x56359ee26d50 2024-05-10T11:39:13.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.597+0000 7f8fbf59e700 1 --2- 172.21.15.100:0/1926355627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56359ee2cad0 0x56359ee30050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:39:13.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.597+0000 7f8fcf9dcb80 10 client.0 _ll_register_callbacks cb 0x56359ed0c5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:39:13.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.597+0000 7f8fcf9dcb80 1 -- 172.21.15.100:0/1926355627 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fb400bb40 con 0x56359ee26d50 2024-05-10T11:39:13.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.597+0000 7f8fbed9d700 1 --2- 172.21.15.100:0/1926355627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56359ee26d50 0x56359ee32700 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0005950 tx=0x7f8fb00047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1dbdb06caa45698 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:39:13.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.597+0000 7f8fa2ff5700 10 client.28873 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:39:13.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.597+0000 7f8fa2ff5700 1 -- 172.21.15.100:0/1926355627 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb001fd60 con 0x56359ee26d50 2024-05-10T11:39:13.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.597+0000 7f8fa2ff5700 1 -- 172.21.15.100:0/1926355627 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f8fb0017030 con 0x56359ee26d50 2024-05-10T11:39:13.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.597+0000 7f8fa2ff5700 1 -- 172.21.15.100:0/1926355627 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f8fb0004020 con 0x56359ee26d50 2024-05-10T11:39:13.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.597+0000 7f8fcf9dcb80 10 client.28873 fetch_fsmap learned FSMap version 41 2024-05-10T11:39:13.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.598+0000 7f8fcf9dcb80 1 -- 172.21.15.100:0/1926355627 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({fsmap=41}) -- 0x56359ee52e00 con 0x56359ee26d50 2024-05-10T11:39:13.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.598+0000 7f8fa2ff5700 1 -- 172.21.15.100:0/1926355627 <== mon.1 v2:172.21.15.188:3300/0 4 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f8fb00041c0 con 0x56359ee26d50 2024-05-10T11:39:13.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.598+0000 7f8fcf9dcb80 10 client.28873 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:39:13.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.598+0000 7f8fcf9dcb80 10 client.28873 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:39:13.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.598+0000 7f8fcf9dcb80 20 client.28873 populate_metadata read hostname 'smithi100' 2024-05-10T11:39:13.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.598+0000 7f8fcf9dcb80 1 --2- 172.21.15.100:0/1926355627 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56359ee553c0 0x56359ee57880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:13.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.598+0000 7f8fcf9dcb80 4 client.28873 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:39:13.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.598+0000 7f8fcf9dcb80 1 -- 172.21.15.100:0/1926355627 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56359ed86320 con 0x56359ee553c0 2024-05-10T11:39:13.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.598+0000 7f8fbe59c700 1 --2- 172.21.15.100:0/1926355627 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56359ee553c0 0x56359ee57880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:13.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.599+0000 7f8fbe59c700 1 --2- 172.21.15.100:0/1926355627 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56359ee553c0 0x56359ee57880 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0618f13086d6a51 server_cookie=6c758abbfffa05d7 in_seq=0 out_seq=0 2024-05-10T11:39:13.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.599+0000 7f8fa2ff5700 10 client.28873 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:39:13.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.599+0000 7f8fa2ff5700 1 -- 172.21.15.100:0/1926355627 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56359ed86320 con 0x56359ee553c0 2024-05-10T11:39:13.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.600+0000 7f8fa2ff5700 10 client.28873 handle_command_reply: tid=0 2024-05-10T11:39:13.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.600+0000 7f8fcf9dcb80 1 client.28873 shutdown 2024-05-10T11:39:13.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.600+0000 7f8fcf9dcb80 1 -- 172.21.15.100:0/1926355627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56359ee26d50 msgr2=0x56359ee32700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:13.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.600+0000 7f8fcf9dcb80 1 --2- 172.21.15.100:0/1926355627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56359ee26d50 0x56359ee32700 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0005950 tx=0x7f8fb00047c0 comp rx=0 tx=0).stop 2024-05-10T11:39:13.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.600+0000 7f8fcf9dcb80 1 -- 172.21.15.100:0/1926355627 shutdown_connections 2024-05-10T11:39:13.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.600+0000 7f8fcf9dcb80 1 --2- 172.21.15.100:0/1926355627 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56359ee553c0 0x56359ee57880 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:13.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.600+0000 7f8fcf9dcb80 1 --2- 172.21.15.100:0/1926355627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56359ee2cad0 0x56359ee30050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:13.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.600+0000 7f8fcf9dcb80 1 --2- 172.21.15.100:0/1926355627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56359ee26d50 0x56359ee32700 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:13.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.600+0000 7f8fcf9dcb80 1 --2- 172.21.15.100:0/1926355627 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56359ee2c1b0 0x56359ee2fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:13.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.600+0000 7f8fcf9dcb80 1 -- 172.21.15.100:0/1926355627 >> 172.21.15.100:0/1926355627 conn(0x56359ed52b70 msgr2=0x56359ee1bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:39:13.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.600+0000 7f8fa2ff5700 0 client.28873 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:39:13.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.600+0000 7f8fcf9dcb80 1 -- 172.21.15.100:0/1926355627 shutdown_connections 2024-05-10T11:39:13.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.600+0000 7f8fcf9dcb80 1 -- 172.21.15.100:0/1926355627 wait complete. 2024-05-10T11:39:13.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.600+0000 7f8fcf9dcb80 20 client.28873 trim_cache size 0 max 16384 2024-05-10T11:39:13.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.626+0000 7f8fcf9dcb80 1 Processor -- start 2024-05-10T11:39:13.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.626+0000 7f8fcf9dcb80 1 -- start start 2024-05-10T11:39:13.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.626+0000 7f8fcf9dcb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56359ee26d50 0x56359ee2d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:13.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.626+0000 7f8fcf9dcb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56359ee2c1b0 0x56359ee40920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:13.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.626+0000 7f8fcf9dcb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56359ee2cad0 0x56359ee2f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:13.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.626+0000 7f8fcf9dcb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8fac011200 con 0x56359ee2cad0 2024-05-10T11:39:13.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.626+0000 7f8fcf9dcb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8fac00ea90 con 0x56359ee2c1b0 2024-05-10T11:39:13.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.626+0000 7f8fcf9dcb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8fac00c8e0 con 0x56359ee26d50 2024-05-10T11:39:13.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.626+0000 7f8fbed9d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56359ee26d50 0x56359ee2d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:13.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.626+0000 7f8fbe59c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56359ee2c1b0 0x56359ee40920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:13.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.626+0000 7f8fbf59e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56359ee2cad0 0x56359ee2f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:13.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.627+0000 7f8fbe59c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56359ee2c1b0 0x56359ee40920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:40760/0 (socket says 172.21.15.100:40760) 2024-05-10T11:39:13.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.627+0000 7f8fbed9d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56359ee26d50 0x56359ee2d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:53902/0 (socket says 172.21.15.100:53902) 2024-05-10T11:39:13.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.627+0000 7f8fbe59c700 1 -- 172.21.15.100:0/1959177237 learned_addr learned my addr 172.21.15.100:0/1959177237 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:39:13.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.627+0000 7f8fbe59c700 1 -- 172.21.15.100:0/1959177237 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56359ee26d50 msgr2=0x56359ee2d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:13.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.627+0000 7f8fbe59c700 1 --2- 172.21.15.100:0/1959177237 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56359ee26d50 0x56359ee2d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:13.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.627+0000 7f8fbe59c700 1 -- 172.21.15.100:0/1959177237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56359ee2cad0 msgr2=0x56359ee2f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:13.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.627+0000 7f8fbe59c700 1 --2- 172.21.15.100:0/1959177237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56359ee2cad0 0x56359ee2f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:13.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.627+0000 7f8fbe59c700 1 -- 172.21.15.100:0/1959177237 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56359ee2a320 con 0x56359ee2c1b0 2024-05-10T11:39:13.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.627+0000 7f8fbed9d700 1 --2- 172.21.15.100:0/1959177237 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56359ee26d50 0x56359ee2d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:39:13.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.627+0000 7f8fbf59e700 1 --2- 172.21.15.100:0/1959177237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56359ee2cad0 0x56359ee2f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:39:13.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.627+0000 7f8fbe59c700 1 --2- 172.21.15.100:0/1959177237 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56359ee2c1b0 0x56359ee40920 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4006010 tx=0x7f8fb4005260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44c1df7095627525 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:39:13.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.627+0000 7f8fa4ff9700 1 -- 172.21.15.100:0/1959177237 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb4038070 con 0x56359ee2c1b0 2024-05-10T11:39:13.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.627+0000 7f8fa4ff9700 1 -- 172.21.15.100:0/1959177237 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8fb4005980 con 0x56359ee2c1b0 2024-05-10T11:39:13.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.627+0000 7f8fa4ff9700 1 -- 172.21.15.100:0/1959177237 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb40038e0 con 0x56359ee2c1b0 2024-05-10T11:39:13.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.628+0000 7f8fcf9dcb80 1 -- 172.21.15.100:0/1959177237 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56359ee2c1b0 msgr2=0x56359ee40920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:13.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.628+0000 7f8fcf9dcb80 1 --2- 172.21.15.100:0/1959177237 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56359ee2c1b0 0x56359ee40920 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4006010 tx=0x7f8fb4005260 comp rx=0 tx=0).stop 2024-05-10T11:39:13.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.628+0000 7f8fcf9dcb80 1 -- 172.21.15.100:0/1959177237 shutdown_connections 2024-05-10T11:39:13.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.628+0000 7f8fcf9dcb80 1 --2- 172.21.15.100:0/1959177237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56359ee2cad0 0x56359ee2f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:13.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.628+0000 7f8fcf9dcb80 1 --2- 172.21.15.100:0/1959177237 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56359ee26d50 0x56359ee2d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:13.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.628+0000 7f8fcf9dcb80 1 --2- 172.21.15.100:0/1959177237 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56359ee2c1b0 0x56359ee40920 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:13.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.628+0000 7f8fcf9dcb80 1 -- 172.21.15.100:0/1959177237 >> 172.21.15.100:0/1959177237 conn(0x56359ed52b70 msgr2=0x56359ee2abf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:39:13.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.628+0000 7f8fcf9dcb80 1 -- 172.21.15.100:0/1959177237 shutdown_connections 2024-05-10T11:39:13.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.628+0000 7f8fcf9dcb80 1 -- 172.21.15.100:0/1959177237 wait complete. 2024-05-10T11:39:13.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.629+0000 7f8fcf9dcb80 1 -- start start 2024-05-10T11:39:13.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.629+0000 7f8fcf9dcb80 1 Processor -- start 2024-05-10T11:39:13.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.629+0000 7f8fcf9dcb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56359ee26d50 0x56359ee5cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:13.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.629+0000 7f8fcf9dcb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56359ee2c1b0 0x56359ee5d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:13.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.629+0000 7f8fcf9dcb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56359ee2cad0 0x56359ee5d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:13.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.629+0000 7f8fcf9dcb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8fac00cf50 con 0x56359ee26d50 2024-05-10T11:39:13.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.629+0000 7f8fcf9dcb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8fac00e3c0 con 0x56359ee2c1b0 2024-05-10T11:39:13.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.629+0000 7f8fcf9dcb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8fac00d620 con 0x56359ee2cad0 2024-05-10T11:39:13.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.629+0000 7f8fcf9dcb80 10 client.0 _ll_register_callbacks cb 0x56359ed0c5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:39:13.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.629+0000 7f8fbe59c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56359ee2c1b0 0x56359ee5d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:13.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.629+0000 7f8fbe59c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56359ee2c1b0 0x56359ee5d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:40768/0 (socket says 172.21.15.100:40768) 2024-05-10T11:39:13.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.629+0000 7f8fbe59c700 1 -- 172.21.15.100:0/287213073 learned_addr learned my addr 172.21.15.100:0/287213073 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:39:13.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.630+0000 7f8fbed9d700 1 --2- 172.21.15.100:0/287213073 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56359ee26d50 0x56359ee5cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:13.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.630+0000 7f8fbf59e700 1 --2- 172.21.15.100:0/287213073 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56359ee2cad0 0x56359ee5d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:13.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.630+0000 7f8fbe59c700 1 -- 172.21.15.100:0/287213073 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56359ee2cad0 msgr2=0x56359ee5d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:13.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.630+0000 7f8fbe59c700 1 --2- 172.21.15.100:0/287213073 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56359ee2cad0 0x56359ee5d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:13.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.630+0000 7f8fbe59c700 1 -- 172.21.15.100:0/287213073 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56359ee26d50 msgr2=0x56359ee5cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:13.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.630+0000 7f8fbe59c700 1 --2- 172.21.15.100:0/287213073 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56359ee26d50 0x56359ee5cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:13.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.630+0000 7f8fbe59c700 1 -- 172.21.15.100:0/287213073 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56359ee363a0 con 0x56359ee2c1b0 2024-05-10T11:39:13.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.630+0000 7f8fbed9d700 1 --2- 172.21.15.100:0/287213073 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56359ee26d50 0x56359ee5cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:39:13.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.630+0000 7f8fbe59c700 1 --2- 172.21.15.100:0/287213073 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56359ee2c1b0 0x56359ee5d040 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4005d20 tx=0x7f8fb400bdf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=617b29ee34109508 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:39:13.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.630+0000 7f8fa2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:39:13.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.630+0000 7f8fa2ff5700 1 -- 172.21.15.100:0/287213073 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb4038040 con 0x56359ee2c1b0 2024-05-10T11:39:13.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.630+0000 7f8fcf9dcb80 1 -- 172.21.15.100:0/287213073 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fb4005980 con 0x56359ee2c1b0 2024-05-10T11:39:13.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.630+0000 7f8fa2ff5700 1 -- 172.21.15.100:0/287213073 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f8fb400b8b0 con 0x56359ee2c1b0 2024-05-10T11:39:13.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.631+0000 7f8fa2ff5700 1 -- 172.21.15.100:0/287213073 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f8fb4033450 con 0x56359ee2c1b0 2024-05-10T11:39:13.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.631+0000 7f8fcf9dcb80 10 client.28772 fetch_fsmap learned FSMap version 41 2024-05-10T11:39:13.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.631+0000 7f8fcf9dcb80 1 -- 172.21.15.100:0/287213073 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x56359eec4590 con 0x56359ee2c1b0 2024-05-10T11:39:13.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.631+0000 7f8fa2ff5700 1 -- 172.21.15.100:0/287213073 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8fb4005980 con 0x56359ee2c1b0 2024-05-10T11:39:13.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.631+0000 7f8fa2ff5700 1 -- 172.21.15.100:0/287213073 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f8fb403b070 con 0x56359ee2c1b0 2024-05-10T11:39:13.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.631+0000 7f8fcf9dcb80 10 client.28772 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:39:13.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.631+0000 7f8fcf9dcb80 10 client.28772 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:39:13.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.631+0000 7f8fcf9dcb80 20 client.28772 populate_metadata read hostname 'smithi100' 2024-05-10T11:39:13.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.631+0000 7f8fcf9dcb80 1 --2- 172.21.15.100:0/287213073 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56359eec47c0 0x56359eec6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:13.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.631+0000 7f8fcf9dcb80 4 client.28772 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T11:39:13.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.631+0000 7f8fcf9dcb80 1 -- 172.21.15.100:0/287213073 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56359ed803c0 con 0x56359eec47c0 2024-05-10T11:39:13.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.631+0000 7f8fbed9d700 1 --2- 172.21.15.100:0/287213073 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56359eec47c0 0x56359eec6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:13.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.632+0000 7f8fa2ff5700 10 client.28772 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:39:13.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.632+0000 7f8fbed9d700 1 --2- 172.21.15.100:0/287213073 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56359eec47c0 0x56359eec6bb0 crc :-1 s=READY pgs=398 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f81206cdeeb53224 server_cookie=d8fbe4051e822927 in_seq=0 out_seq=0 2024-05-10T11:39:13.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.632+0000 7f8fa2ff5700 1 -- 172.21.15.100:0/287213073 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56359ed803c0 con 0x56359eec47c0 2024-05-10T11:39:13.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.632+0000 7f8fa2ff5700 10 client.28772 handle_command_reply: tid=0 2024-05-10T11:39:13.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.633+0000 7f8fcf9dcb80 1 client.28772 shutdown 2024-05-10T11:39:13.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.633+0000 7f8fcf9dcb80 1 -- 172.21.15.100:0/287213073 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56359ee2c1b0 msgr2=0x56359ee5d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:13.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.633+0000 7f8fcf9dcb80 1 --2- 172.21.15.100:0/287213073 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56359ee2c1b0 0x56359ee5d040 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4005d20 tx=0x7f8fb400bdf0 comp rx=0 tx=0).stop 2024-05-10T11:39:13.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.633+0000 7f8fcf9dcb80 1 -- 172.21.15.100:0/287213073 shutdown_connections 2024-05-10T11:39:13.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.633+0000 7f8fcf9dcb80 1 --2- 172.21.15.100:0/287213073 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56359eec47c0 0x56359eec6bb0 crc :-1 s=READY pgs=398 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:13.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.633+0000 7f8fcf9dcb80 1 --2- 172.21.15.100:0/287213073 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56359ee2cad0 0x56359ee5d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:13.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.633+0000 7f8fa2ff5700 0 client.28772 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:39:13.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.633+0000 7f8fcf9dcb80 1 --2- 172.21.15.100:0/287213073 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56359ee26d50 0x56359ee5cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:13.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.633+0000 7f8fcf9dcb80 1 --2- 172.21.15.100:0/287213073 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56359ee2c1b0 0x56359ee5d040 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:13.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.633+0000 7f8fcf9dcb80 1 -- 172.21.15.100:0/287213073 >> 172.21.15.100:0/287213073 conn(0x56359ed52b70 msgr2=0x56359ee2b480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:39:13.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.633+0000 7f8fcf9dcb80 1 -- 172.21.15.100:0/287213073 shutdown_connections 2024-05-10T11:39:13.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.633+0000 7f8fcf9dcb80 1 -- 172.21.15.100:0/287213073 wait complete. 2024-05-10T11:39:13.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.633+0000 7f8fcf9dcb80 20 client.28772 trim_cache size 0 max 16384 2024-05-10T11:39:13.637 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:39:13.637 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-05-10T11:39:13.637 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-05-10T11:39:13.638 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:39:13.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.634+0000 7f8fc5149700 1 -- 172.21.15.100:0/1595540348 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8fa80b2900 msgr2=0x7f8fa80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:13.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.634+0000 7f8fc5149700 1 --2- 172.21.15.100:0/1595540348 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8fa80b2900 0x7f8fa80b4dc0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x56359ee23450 tx=0x7f8fb000b500 comp rx=0 tx=0).stop 2024-05-10T11:39:13.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.634+0000 7f8fc5149700 1 -- 172.21.15.100:0/1595540348 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8fc0108850 msgr2=0x7f8fc0199890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:13.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.634+0000 7f8fc5149700 1 --2- 172.21.15.100:0/1595540348 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8fc0108850 0x7f8fc0199890 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8fb800efc0 tx=0x7f8fb800c3a0 comp rx=0 tx=0).stop 2024-05-10T11:39:13.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.634+0000 7f8fc5149700 1 -- 172.21.15.100:0/1595540348 shutdown_connections 2024-05-10T11:39:13.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.634+0000 7f8fc5149700 1 --2- 172.21.15.100:0/1595540348 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8fa80b2900 0x7f8fa80b4dc0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:13.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.634+0000 7f8fc5149700 1 --2- 172.21.15.100:0/1595540348 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8fc0108850 0x7f8fc0199890 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:13.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.634+0000 7f8fc5149700 1 --2- 172.21.15.100:0/1595540348 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8fc0103750 0x7f8fc019f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:13.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.634+0000 7f8fc5149700 1 --2- 172.21.15.100:0/1595540348 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fc0104100 0x7f8fc019fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:13.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.634+0000 7f8fc5149700 1 -- 172.21.15.100:0/1595540348 >> 172.21.15.100:0/1595540348 conn(0x7f8fc007a600 msgr2=0x7f8fc01019a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:39:13.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.634+0000 7f8fc5149700 1 -- 172.21.15.100:0/1595540348 shutdown_connections 2024-05-10T11:39:13.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:13.635+0000 7f8fc5149700 1 -- 172.21.15.100:0/1595540348 wait complete. 2024-05-10T11:39:13.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:39:13 smithi100 ceph-mon[30296]: pgmap v1046: 129 pgs: 129 active+clean; 524 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 4.6 MiB/s wr, 277 op/s 2024-05-10T11:39:13.946 INFO:tasks.workunit.client.0.smithi100.stdout: 10 71682 9.05 MB/sec execute 307 sec latency 142.418 ms 2024-05-10T11:39:14.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:39:13 smithi178 ceph-mon[33359]: pgmap v1046: 129 pgs: 129 active+clean; 524 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 4.6 MiB/s wr, 277 op/s 2024-05-10T11:39:14.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:39:13 smithi188 ceph-mon[34075]: pgmap v1046: 129 pgs: 129 active+clean; 524 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 4.6 MiB/s wr, 277 op/s 2024-05-10T11:39:14.669 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T11:39:14.669 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T11:39:14.946 INFO:tasks.workunit.client.0.smithi100.stdout: 10 71817 9.03 MB/sec execute 308 sec latency 161.584 ms 2024-05-10T11:39:15.054 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:39:15.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:39:15 smithi100 ceph-mon[30296]: pgmap v1047: 129 pgs: 129 active+clean; 525 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 5.0 MiB/s wr, 269 op/s 2024-05-10T11:39:15.946 INFO:tasks.workunit.client.0.smithi100.stdout: 10 71966 9.01 MB/sec execute 309 sec latency 99.466 ms 2024-05-10T11:39:16.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:39:15 smithi178 ceph-mon[33359]: pgmap v1047: 129 pgs: 129 active+clean; 525 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 5.0 MiB/s wr, 269 op/s 2024-05-10T11:39:16.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:39:15 smithi188 ceph-mon[34075]: pgmap v1047: 129 pgs: 129 active+clean; 525 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 5.0 MiB/s wr, 269 op/s 2024-05-10T11:39:16.946 INFO:tasks.workunit.client.0.smithi100.stdout: 10 72115 9.01 MB/sec execute 310 sec latency 119.888 ms 2024-05-10T11:39:17.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:17.557+0000 7f0c05327700 1 -- 172.21.15.100:0/3972054682 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c00103f00 msgr2=0x7f0c001103b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:17.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:17.557+0000 7f0c05327700 1 --2- 172.21.15.100:0/3972054682 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c00103f00 0x7f0c001103b0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f0bf800b0d0 tx=0x7f0bf800b3e0 comp rx=0 tx=0).stop 2024-05-10T11:39:17.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:17.557+0000 7f0c05327700 1 -- 172.21.15.100:0/3972054682 shutdown_connections 2024-05-10T11:39:17.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:17.557+0000 7f0c05327700 1 --2- 172.21.15.100:0/3972054682 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c00103f00 0x7f0c001103b0 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:17.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:17.557+0000 7f0c05327700 1 --2- 172.21.15.100:0/3972054682 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c001095e0 0x7f0c001099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:17.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:17.557+0000 7f0c05327700 1 --2- 172.21.15.100:0/3972054682 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c00103540 0x7f0c001039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:17.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:17.557+0000 7f0c05327700 1 -- 172.21.15.100:0/3972054682 >> 172.21.15.100:0/3972054682 conn(0x7f0c0007a650 msgr2=0x7f0c0007aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:39:17.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:17.557+0000 7f0c05327700 1 -- 172.21.15.100:0/3972054682 shutdown_connections 2024-05-10T11:39:17.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:17.558+0000 7f0c05327700 1 -- 172.21.15.100:0/3972054682 wait complete. 2024-05-10T11:39:17.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:17.558+0000 7f0c05327700 1 Processor -- start 2024-05-10T11:39:17.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:17.559+0000 7f0c05327700 1 -- start start 2024-05-10T11:39:17.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:17.559+0000 7f0c05327700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c00103540 0x7f0c000757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:17.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:17.559+0000 7f0c05327700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c00103f00 0x7f0c00075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:17.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:17.559+0000 7f0c05327700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c001095e0 0x7f0c00076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:17.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:17.559+0000 7f0c05327700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0c001135b0 con 0x7f0c00103540 2024-05-10T11:39:17.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:17.559+0000 7f0c05327700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0c00113430 con 0x7f0c00103f00 2024-05-10T11:39:17.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:17.559+0000 7f0c05327700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0c00113730 con 0x7f0c001095e0 2024-05-10T11:39:17.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:17.559+0000 7f0bfeffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c00103540 0x7f0c000757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:17.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:17.559+0000 7f0bfe7fc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c00103f00 0x7f0c00075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:17.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:17.559+0000 7f0bfeffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c00103540 0x7f0c000757a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41094/0 (socket says 172.21.15.100:41094) 2024-05-10T11:39:17.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:17.559+0000 7f0bfeffd700 1 -- 172.21.15.100:0/408936169 learned_addr learned my addr 172.21.15.100:0/408936169 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:39:17.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:17.559+0000 7f0bff7fe700 1 --2- 172.21.15.100:0/408936169 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c001095e0 0x7f0c00076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:17.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:17.560+0000 7f0bfeffd700 1 -- 172.21.15.100:0/408936169 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c001095e0 msgr2=0x7f0c00076220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:17.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:17.560+0000 7f0bfeffd700 1 --2- 172.21.15.100:0/408936169 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c001095e0 0x7f0c00076220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:17.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:17.560+0000 7f0bfeffd700 1 -- 172.21.15.100:0/408936169 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c00103f00 msgr2=0x7f0c00075ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:17.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:17.560+0000 7f0bfeffd700 1 --2- 172.21.15.100:0/408936169 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c00103f00 0x7f0c00075ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:17.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:17.560+0000 7f0bfeffd700 1 -- 172.21.15.100:0/408936169 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c001a7f30 con 0x7f0c00103540 2024-05-10T11:39:17.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:17.560+0000 7f0bff7fe700 1 --2- 172.21.15.100:0/408936169 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c001095e0 0x7f0c00076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:39:17.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:17.560+0000 7f0bfe7fc700 1 --2- 172.21.15.100:0/408936169 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c00103f00 0x7f0c00075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:39:17.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:17.560+0000 7f0bfeffd700 1 --2- 172.21.15.100:0/408936169 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c00103540 0x7f0c000757a0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f0bf000eb30 tx=0x7f0bf000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:39:17.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:17.560+0000 7f0be7fff700 1 -- 172.21.15.100:0/408936169 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf000cbd0 con 0x7f0c00103540 2024-05-10T11:39:17.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:17.560+0000 7f0c05327700 1 -- 172.21.15.100:0/408936169 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c001a8220 con 0x7f0c00103540 2024-05-10T11:39:17.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:17.560+0000 7f0c05327700 1 -- 172.21.15.100:0/408936169 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c001a87b0 con 0x7f0c00103540 2024-05-10T11:39:17.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:17.560+0000 7f0be7fff700 1 -- 172.21.15.100:0/408936169 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0bf00044b0 con 0x7f0c00103540 2024-05-10T11:39:17.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:17.560+0000 7f0be7fff700 1 -- 172.21.15.100:0/408936169 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf0003dc0 con 0x7f0c00103540 2024-05-10T11:39:17.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:17.561+0000 7f0c05327700 1 -- 172.21.15.100:0/408936169 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c00104c50 con 0x7f0c00103540 2024-05-10T11:39:17.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:17.563+0000 7f0be7fff700 1 -- 172.21.15.100:0/408936169 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f0bf0003f60 con 0x7f0c00103540 2024-05-10T11:39:17.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:17.564+0000 7f0be7fff700 1 --2- 172.21.15.100:0/408936169 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0be80b2900 0x7f0be80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:17.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:17.564+0000 7f0bfe7fc700 1 --2- 172.21.15.100:0/408936169 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0be80b2900 0x7f0be80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:17.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:17.564+0000 7f0be7fff700 1 -- 172.21.15.100:0/408936169 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f0bf00e2800 con 0x7f0c00103540 2024-05-10T11:39:17.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:17.565+0000 7f0bfe7fc700 1 --2- 172.21.15.100:0/408936169 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0be80b2900 0x7f0be80b4dc0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f0c0007abe0 tx=0x7f0bf4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:39:17.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:17.571+0000 7f0be7fff700 1 -- 172.21.15.100:0/408936169 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f0bf00aa250 con 0x7f0c00103540 2024-05-10T11:39:17.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:39:17 smithi100 ceph-mon[30296]: pgmap v1048: 129 pgs: 129 active+clean; 526 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 5.1 MiB/s wr, 297 op/s 2024-05-10T11:39:17.946 INFO:tasks.workunit.client.0.smithi100.stdout: 10 72279 9.00 MB/sec execute 311 sec latency 107.210 ms 2024-05-10T11:39:17.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:17.987+0000 7f0c05327700 1 -- 172.21.15.100:0/408936169 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0c00076cd0 con 0x7f0c00103540 2024-05-10T11:39:17.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:17.988+0000 7f0be7fff700 1 -- 172.21.15.100:0/408936169 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+11897 (secure 0 0 0) 0x7f0bf00b2240 con 0x7f0c00103540 2024-05-10T11:39:17.988 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:39:17.989 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2072225012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2072225012},{"type":"v1","addr":"172.21.15.100:6835","nonce":2072225012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/401939540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":401939540},{"type":"v1","addr":"172.21.15.100:6837","nonce":401939540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14571,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/3946311025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3946311025},{"type":"v1","addr":"172.21.15.100:6839","nonce":3946311025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3736371660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3736371660},{"type":"v1","addr":"172.21.15.100:6841","nonce":3736371660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/2741810791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2741810791},{"type":"v1","addr":"172.21.15.178:6833","nonce":2741810791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1704891041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1704891041},{"type":"v1","addr":"172.21.15.188:6833","nonce":1704891041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":41,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T11:14:27.242452+0000","modified":"2024-05-10T11:35:29.588072+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24439,"mds_1":24437,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/2124076312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2124076312},{"type":"v1","addr":"172.21.15.178:6835","nonce":2124076312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/3224358425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3224358425},{"type":"v1","addr":"172.21.15.178:6837","nonce":3224358425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/1943342620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1943342620},{"type":"v1","addr":"172.21.15.188:6837","nonce":1943342620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.178:6839/3760985404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3760985404},{"type":"v1","addr":"172.21.15.178:6839","nonce":3760985404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.188:6839/3194917070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":3194917070},{"type":"v1","addr":"172.21.15.188:6839","nonce":3194917070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27034":{"gid":27034,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/3139559450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3139559450},{"type":"v1","addr":"172.21.15.188:6835","nonce":3139559450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24439,"qdb_cluster":[24437,24439,24427]},"id":1}]} 2024-05-10T11:39:17.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:17.992+0000 7f0c05327700 1 -- 172.21.15.100:0/408936169 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0be80b2900 msgr2=0x7f0be80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:17.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:17.992+0000 7f0c05327700 1 --2- 172.21.15.100:0/408936169 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0be80b2900 0x7f0be80b4dc0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f0c0007abe0 tx=0x7f0bf4009260 comp rx=0 tx=0).stop 2024-05-10T11:39:17.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:17.992+0000 7f0c05327700 1 -- 172.21.15.100:0/408936169 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c00103540 msgr2=0x7f0c000757a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:17.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:17.992+0000 7f0c05327700 1 --2- 172.21.15.100:0/408936169 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c00103540 0x7f0c000757a0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f0bf000eb30 tx=0x7f0bf000ee40 comp rx=0 tx=0).stop 2024-05-10T11:39:17.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:17.992+0000 7f0c05327700 1 -- 172.21.15.100:0/408936169 shutdown_connections 2024-05-10T11:39:17.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:17.992+0000 7f0c05327700 1 --2- 172.21.15.100:0/408936169 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0be80b2900 0x7f0be80b4dc0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:17.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:17.992+0000 7f0c05327700 1 --2- 172.21.15.100:0/408936169 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c001095e0 0x7f0c00076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:17.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:17.992+0000 7f0c05327700 1 --2- 172.21.15.100:0/408936169 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c00103540 0x7f0c000757a0 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:17.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:17.992+0000 7f0c05327700 1 --2- 172.21.15.100:0/408936169 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c00103f00 0x7f0c00075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:17.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:17.992+0000 7f0c05327700 1 -- 172.21.15.100:0/408936169 >> 172.21.15.100:0/408936169 conn(0x7f0c0007a650 msgr2=0x7f0c00102120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:39:17.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:17.992+0000 7f0c05327700 1 -- 172.21.15.100:0/408936169 shutdown_connections 2024-05-10T11:39:17.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:17.992+0000 7f0c05327700 1 -- 172.21.15.100:0/408936169 wait complete. 2024-05-10T11:39:17.993 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 41 2024-05-10T11:39:18.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:39:17 smithi178 ceph-mon[33359]: pgmap v1048: 129 pgs: 129 active+clean; 526 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 5.1 MiB/s wr, 297 op/s 2024-05-10T11:39:18.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:39:17 smithi188 ceph-mon[34075]: pgmap v1048: 129 pgs: 129 active+clean; 526 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 5.1 MiB/s wr, 297 op/s 2024-05-10T11:39:18.647 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-10T11:39:18.946 INFO:tasks.workunit.client.0.smithi100.stdout: 10 72456 9.00 MB/sec execute 312 sec latency 70.556 ms 2024-05-10T11:39:18.990 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:39:19.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:39:18 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/408936169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:39:19.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:39:18 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/408936169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:39:19.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:39:18 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/408936169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:39:19.946 INFO:tasks.workunit.client.0.smithi100.stdout: 10 72625 9.00 MB/sec execute 313 sec latency 81.024 ms 2024-05-10T11:39:20.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:39:19 smithi178 ceph-mon[33359]: pgmap v1049: 129 pgs: 129 active+clean; 524 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 4.8 MiB/s wr, 298 op/s 2024-05-10T11:39:20.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:39:19 smithi188 ceph-mon[34075]: pgmap v1049: 129 pgs: 129 active+clean; 524 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 4.8 MiB/s wr, 298 op/s 2024-05-10T11:39:20.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:39:19 smithi100 ceph-mon[30296]: pgmap v1049: 129 pgs: 129 active+clean; 524 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 4.8 MiB/s wr, 298 op/s 2024-05-10T11:39:20.946 INFO:tasks.workunit.client.0.smithi100.stdout: 10 72799 8.98 MB/sec execute 314 sec latency 75.798 ms 2024-05-10T11:39:21.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.004+0000 7facbc055700 1 -- 172.21.15.100:0/2593791406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7facb4108300 msgr2=0x7facb41086e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:21.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.004+0000 7facbc055700 1 --2- 172.21.15.100:0/2593791406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7facb4108300 0x7facb41086e0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7facb0009990 tx=0x7facb0009ca0 comp rx=0 tx=0).stop 2024-05-10T11:39:21.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.005+0000 7facbc055700 1 -- 172.21.15.100:0/2593791406 shutdown_connections 2024-05-10T11:39:21.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.005+0000 7facbc055700 1 --2- 172.21.15.100:0/2593791406 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7facb41076e0 0x7facb410f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:21.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.005+0000 7facbc055700 1 --2- 172.21.15.100:0/2593791406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7facb4108300 0x7facb41086e0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:21.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.005+0000 7facbc055700 1 --2- 172.21.15.100:0/2593791406 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7facb4108cb0 0x7facb4107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:21.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.005+0000 7facbc055700 1 -- 172.21.15.100:0/2593791406 >> 172.21.15.100:0/2593791406 conn(0x7facb40785a0 msgr2=0x7facb40ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:39:21.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.005+0000 7facbc055700 1 -- 172.21.15.100:0/2593791406 shutdown_connections 2024-05-10T11:39:21.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.005+0000 7facbc055700 1 -- 172.21.15.100:0/2593791406 wait complete. 2024-05-10T11:39:21.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.006+0000 7facbc055700 1 Processor -- start 2024-05-10T11:39:21.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.006+0000 7facbc055700 1 -- start start 2024-05-10T11:39:21.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.006+0000 7facbc055700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7facb41076e0 0x7facb41976d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:21.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.006+0000 7facbc055700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7facb4108300 0x7facb4197c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:21.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.006+0000 7facbc055700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7facb4108cb0 0x7facb419bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:21.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.006+0000 7facbc055700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7facb41113c0 con 0x7facb4108cb0 2024-05-10T11:39:21.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.006+0000 7facbc055700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7facb4111240 con 0x7facb4108300 2024-05-10T11:39:21.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.006+0000 7facbc055700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7facb4111540 con 0x7facb41076e0 2024-05-10T11:39:21.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.006+0000 7facb9df1700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7facb41076e0 0x7facb41976d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:21.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.006+0000 7facb95f0700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7facb4108300 0x7facb4197c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:21.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.006+0000 7facb9df1700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7facb41076e0 0x7facb41976d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:47052/0 (socket says 172.21.15.100:47052) 2024-05-10T11:39:21.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.006+0000 7facb9df1700 1 -- 172.21.15.100:0/2179517559 learned_addr learned my addr 172.21.15.100:0/2179517559 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:39:21.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.006+0000 7facba5f2700 1 --2- 172.21.15.100:0/2179517559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7facb4108cb0 0x7facb419bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:21.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.007+0000 7facba5f2700 1 -- 172.21.15.100:0/2179517559 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7facb41076e0 msgr2=0x7facb41976d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:21.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.007+0000 7facba5f2700 1 --2- 172.21.15.100:0/2179517559 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7facb41076e0 0x7facb41976d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:21.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.007+0000 7facba5f2700 1 -- 172.21.15.100:0/2179517559 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7facb4108300 msgr2=0x7facb4197c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:21.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.007+0000 7facba5f2700 1 --2- 172.21.15.100:0/2179517559 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7facb4108300 0x7facb4197c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:21.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.007+0000 7facba5f2700 1 -- 172.21.15.100:0/2179517559 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facb419c6e0 con 0x7facb4108cb0 2024-05-10T11:39:21.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.007+0000 7facb95f0700 1 --2- 172.21.15.100:0/2179517559 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7facb4108300 0x7facb4197c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:39:21.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.007+0000 7facba5f2700 1 --2- 172.21.15.100:0/2179517559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7facb4108cb0 0x7facb419bfa0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7facac007760 tx=0x7facac007a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:39:21.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.007+0000 7facaaffd700 1 -- 172.21.15.100:0/2179517559 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facac004bb0 con 0x7facb4108cb0 2024-05-10T11:39:21.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.008+0000 7facaaffd700 1 -- 172.21.15.100:0/2179517559 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7facac004d50 con 0x7facb4108cb0 2024-05-10T11:39:21.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.008+0000 7facbc055700 1 -- 172.21.15.100:0/2179517559 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facb419c9d0 con 0x7facb4108cb0 2024-05-10T11:39:21.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.008+0000 7facaaffd700 1 -- 172.21.15.100:0/2179517559 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facac009970 con 0x7facb4108cb0 2024-05-10T11:39:21.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.008+0000 7facbc055700 1 -- 172.21.15.100:0/2179517559 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facb41a4250 con 0x7facb4108cb0 2024-05-10T11:39:21.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.012+0000 7facaaffd700 1 -- 172.21.15.100:0/2179517559 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7facac009b10 con 0x7facb4108cb0 2024-05-10T11:39:21.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.013+0000 7facc68e8b80 1 Processor -- start 2024-05-10T11:39:21.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.013+0000 7facaaffd700 1 --2- 172.21.15.100:0/2179517559 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7faca00b2900 0x7faca00b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:21.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.014+0000 7facc68e8b80 1 -- start start 2024-05-10T11:39:21.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.014+0000 7facb95f0700 1 --2- 172.21.15.100:0/2179517559 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7faca00b2900 0x7faca00b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:21.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.014+0000 7facaaffd700 1 -- 172.21.15.100:0/2179517559 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7facac0e6010 con 0x7facb4108cb0 2024-05-10T11:39:21.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.014+0000 7facb95f0700 1 --2- 172.21.15.100:0/2179517559 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7faca00b2900 0x7faca00b4dc0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7facb4106f40 tx=0x7faca4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:39:21.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.015+0000 7facc68e8b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563729276380 0x563729276760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:21.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.015+0000 7facc68e8b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563729276ca0 0x563729270b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:21.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.015+0000 7facc68e8b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563729271090 0x563729271510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:21.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.015+0000 7facc68e8b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5637291ba860 con 0x563729276ca0 2024-05-10T11:39:21.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.015+0000 7facc68e8b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5637291597c0 con 0x563729271090 2024-05-10T11:39:21.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.015+0000 7facc68e8b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5637291b6820 con 0x563729276380 2024-05-10T11:39:21.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.015+0000 7facba5f2700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563729271090 0x563729271510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:21.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.015+0000 7facba5f2700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563729271090 0x563729271510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:43322/0 (socket says 172.21.15.100:43322) 2024-05-10T11:39:21.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.015+0000 7facba5f2700 1 -- 172.21.15.100:0/1418979156 learned_addr learned my addr 172.21.15.100:0/1418979156 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:39:21.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.015+0000 7facb95f0700 1 --2- 172.21.15.100:0/1418979156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563729276ca0 0x563729270b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:21.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.015+0000 7facba5f2700 1 -- 172.21.15.100:0/1418979156 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563729276380 msgr2=0x563729276760 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T11:39:21.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.015+0000 7facba5f2700 1 --2- 172.21.15.100:0/1418979156 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563729276380 0x563729276760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:21.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.015+0000 7facba5f2700 1 -- 172.21.15.100:0/1418979156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563729276ca0 msgr2=0x563729270b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:21.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.015+0000 7facba5f2700 1 --2- 172.21.15.100:0/1418979156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563729276ca0 0x563729270b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:21.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.015+0000 7facba5f2700 1 -- 172.21.15.100:0/1418979156 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637291078d0 con 0x563729271090 2024-05-10T11:39:21.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.015+0000 7facb95f0700 1 --2- 172.21.15.100:0/1418979156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563729276ca0 0x563729270b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:39:21.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.016+0000 7facba5f2700 1 --2- 172.21.15.100:0/1418979156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563729271090 0x563729271510 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7facac007730 tx=0x7facac0acb40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de70cb7a04bdb3ec server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:39:21.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.016+0000 7fac9ffff700 1 -- 172.21.15.100:0/1418979156 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facac01e040 con 0x563729271090 2024-05-10T11:39:21.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.016+0000 7fac9ffff700 1 -- 172.21.15.100:0/1418979156 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7facac0ad2d0 con 0x563729271090 2024-05-10T11:39:21.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.016+0000 7fac9ffff700 1 -- 172.21.15.100:0/1418979156 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facac0b5f00 con 0x563729271090 2024-05-10T11:39:21.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.017+0000 7facc68e8b80 1 -- 172.21.15.100:0/1418979156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563729271090 msgr2=0x563729271510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:21.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.017+0000 7facc68e8b80 1 --2- 172.21.15.100:0/1418979156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563729271090 0x563729271510 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7facac007730 tx=0x7facac0acb40 comp rx=0 tx=0).stop 2024-05-10T11:39:21.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.017+0000 7facc68e8b80 1 -- 172.21.15.100:0/1418979156 shutdown_connections 2024-05-10T11:39:21.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.017+0000 7facc68e8b80 1 --2- 172.21.15.100:0/1418979156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563729271090 0x563729271510 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:21.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.017+0000 7facc68e8b80 1 --2- 172.21.15.100:0/1418979156 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563729276380 0x563729276760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:21.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.017+0000 7facc68e8b80 1 --2- 172.21.15.100:0/1418979156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563729276ca0 0x563729270b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:21.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.017+0000 7facc68e8b80 1 -- 172.21.15.100:0/1418979156 >> 172.21.15.100:0/1418979156 conn(0x563729194100 msgr2=0x563729262c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:39:21.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.017+0000 7facc68e8b80 1 -- 172.21.15.100:0/1418979156 shutdown_connections 2024-05-10T11:39:21.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.017+0000 7facc68e8b80 1 -- 172.21.15.100:0/1418979156 wait complete. 2024-05-10T11:39:21.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.017+0000 7facc68e8b80 1 -- start start 2024-05-10T11:39:21.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.017+0000 7facc68e8b80 1 Processor -- start 2024-05-10T11:39:21.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.018+0000 7facc68e8b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563729271090 0x56372927c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:21.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.018+0000 7facc68e8b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563729276380 0x563729279ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:21.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.018+0000 7facc68e8b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563729276ca0 0x56372927a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:21.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.018+0000 7facc68e8b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56372927d700 con 0x563729276ca0 2024-05-10T11:39:21.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.018+0000 7facc68e8b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563729120c30 con 0x563729271090 2024-05-10T11:39:21.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.018+0000 7facb95f0700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563729276380 0x563729279ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:21.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.018+0000 7facb95f0700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563729276380 0x563729279ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:47070/0 (socket says 172.21.15.100:47070) 2024-05-10T11:39:21.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.018+0000 7facb95f0700 1 -- 172.21.15.100:0/194562775 learned_addr learned my addr 172.21.15.100:0/194562775 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:39:21.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.018+0000 7facb9df1700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563729271090 0x56372927c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:21.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.018+0000 7facc68e8b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56372927d880 con 0x563729276380 2024-05-10T11:39:21.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.018+0000 7facba5f2700 1 --2- 172.21.15.100:0/194562775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563729276ca0 0x56372927a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:21.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.018+0000 7facc68e8b80 10 client.0 _ll_register_callbacks cb 0x563729152920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:39:21.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.019+0000 7facb95f0700 1 -- 172.21.15.100:0/194562775 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563729271090 msgr2=0x56372927c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:21.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.019+0000 7facb95f0700 1 --2- 172.21.15.100:0/194562775 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563729271090 0x56372927c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:21.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.019+0000 7facb95f0700 1 -- 172.21.15.100:0/194562775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563729276ca0 msgr2=0x56372927a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:21.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.019+0000 7facb95f0700 1 --2- 172.21.15.100:0/194562775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563729276ca0 0x56372927a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:21.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.019+0000 7facb95f0700 1 -- 172.21.15.100:0/194562775 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563729156690 con 0x563729276380 2024-05-10T11:39:21.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.019+0000 7facb95f0700 1 --2- 172.21.15.100:0/194562775 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563729276380 0x563729279ca0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7faca400eb10 tx=0x7faca40047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=809e6f65c0cf7350 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:39:21.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.019+0000 7fac9d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:39:21.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.019+0000 7fac9d7fa700 1 -- 172.21.15.100:0/194562775 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca4004460 con 0x563729276380 2024-05-10T11:39:21.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.019+0000 7facc68e8b80 1 -- 172.21.15.100:0/194562775 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facac0ad2d0 con 0x563729276380 2024-05-10T11:39:21.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.019+0000 7fac9d7fa700 1 -- 172.21.15.100:0/194562775 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7faca401e820 con 0x563729276380 2024-05-10T11:39:21.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.020+0000 7fac9d7fa700 1 -- 172.21.15.100:0/194562775 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7faca401ea20 con 0x563729276380 2024-05-10T11:39:21.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.020+0000 7facc68e8b80 10 client.28894 fetch_fsmap learned FSMap version 41 2024-05-10T11:39:21.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.020+0000 7facc68e8b80 1 -- 172.21.15.100:0/194562775 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x563729287dd0 con 0x563729276380 2024-05-10T11:39:21.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.020+0000 7fac9d7fa700 1 -- 172.21.15.100:0/194562775 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faca4031070 con 0x563729276380 2024-05-10T11:39:21.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.021+0000 7fac9d7fa700 1 -- 172.21.15.100:0/194562775 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7faca4003ca0 con 0x563729276380 2024-05-10T11:39:21.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.021+0000 7facc68e8b80 10 client.28894 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:39:21.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.021+0000 7facc68e8b80 10 client.28894 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-10T11:39:21.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.021+0000 7facc68e8b80 20 client.28894 populate_metadata read hostname 'smithi100' 2024-05-10T11:39:21.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.021+0000 7facc68e8b80 1 --2- 172.21.15.100:0/194562775 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x56372928a840 0x56372928cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:21.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.021+0000 7facc68e8b80 4 client.28894 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:39:21.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.021+0000 7facc68e8b80 1 -- 172.21.15.100:0/194562775 --> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563729234650 con 0x56372928a840 2024-05-10T11:39:21.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.022+0000 7facb9df1700 1 --2- 172.21.15.100:0/194562775 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x56372928a840 0x56372928cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:21.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.023+0000 7facb9df1700 1 --2- 172.21.15.100:0/194562775 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x56372928a840 0x56372928cca0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3d16011eeaeff1eb server_cookie=5e7bde24b27c8d36 in_seq=0 out_seq=0 2024-05-10T11:39:21.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.023+0000 7fac9d7fa700 10 client.28894 ms_handle_connect on v2:172.21.15.188:6836/1943342620 2024-05-10T11:39:21.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.024+0000 7fac9d7fa700 1 -- 172.21.15.100:0/194562775 <== mds.2 v2:172.21.15.188:6836/1943342620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563729234650 con 0x56372928a840 2024-05-10T11:39:21.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.024+0000 7fac9d7fa700 10 client.28894 handle_command_reply: tid=0 2024-05-10T11:39:21.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.024+0000 7facc68e8b80 1 client.28894 shutdown 2024-05-10T11:39:21.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.024+0000 7facc68e8b80 1 -- 172.21.15.100:0/194562775 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563729276380 msgr2=0x563729279ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:21.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.024+0000 7facc68e8b80 1 --2- 172.21.15.100:0/194562775 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563729276380 0x563729279ca0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7faca400eb10 tx=0x7faca40047c0 comp rx=0 tx=0).stop 2024-05-10T11:39:21.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.025+0000 7facc68e8b80 1 -- 172.21.15.100:0/194562775 shutdown_connections 2024-05-10T11:39:21.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.025+0000 7facc68e8b80 1 --2- 172.21.15.100:0/194562775 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x56372928a840 0x56372928cca0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:21.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.025+0000 7facc68e8b80 1 --2- 172.21.15.100:0/194562775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563729276ca0 0x56372927a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:21.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.025+0000 7facc68e8b80 1 --2- 172.21.15.100:0/194562775 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563729271090 0x56372927c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:21.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.025+0000 7facc68e8b80 1 --2- 172.21.15.100:0/194562775 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563729276380 0x563729279ca0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:21.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.025+0000 7facc68e8b80 1 -- 172.21.15.100:0/194562775 >> 172.21.15.100:0/194562775 conn(0x563729194100 msgr2=0x56372925d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:39:21.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.025+0000 7fac9d7fa700 0 client.28894 ms_handle_reset on v2:172.21.15.188:6836/1943342620 2024-05-10T11:39:21.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.025+0000 7facc68e8b80 1 -- 172.21.15.100:0/194562775 shutdown_connections 2024-05-10T11:39:21.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.025+0000 7facc68e8b80 1 -- 172.21.15.100:0/194562775 wait complete. 2024-05-10T11:39:21.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.025+0000 7facc68e8b80 20 client.28894 trim_cache size 0 max 16384 2024-05-10T11:39:21.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.051+0000 7facc68e8b80 1 Processor -- start 2024-05-10T11:39:21.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.051+0000 7facc68e8b80 1 -- start start 2024-05-10T11:39:21.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.051+0000 7facc68e8b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563729271090 0x5637292867c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:21.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.051+0000 7facc68e8b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563729276380 0x5637292785b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:21.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.051+0000 7facc68e8b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563729276ca0 0x563729278af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:21.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.051+0000 7facc68e8b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7facb41af5a0 con 0x563729276380 2024-05-10T11:39:21.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.051+0000 7facc68e8b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7facb41ace30 con 0x563729276ca0 2024-05-10T11:39:21.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.051+0000 7facc68e8b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7facb41aac80 con 0x563729271090 2024-05-10T11:39:21.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.051+0000 7facb9df1700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563729271090 0x5637292867c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:21.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.052+0000 7facb9df1700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563729271090 0x5637292867c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:47078/0 (socket says 172.21.15.100:47078) 2024-05-10T11:39:21.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.052+0000 7facb9df1700 1 -- 172.21.15.100:0/1626510186 learned_addr learned my addr 172.21.15.100:0/1626510186 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:39:21.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.052+0000 7facb95f0700 1 --2- 172.21.15.100:0/1626510186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563729276380 0x5637292785b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:21.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.052+0000 7facb9df1700 1 -- 172.21.15.100:0/1626510186 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563729276ca0 msgr2=0x563729278af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:21.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.052+0000 7facb9df1700 1 --2- 172.21.15.100:0/1626510186 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563729276ca0 0x563729278af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:21.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.052+0000 7facb9df1700 1 -- 172.21.15.100:0/1626510186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563729276380 msgr2=0x5637292785b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:21.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.052+0000 7facb9df1700 1 --2- 172.21.15.100:0/1626510186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563729276380 0x5637292785b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:21.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.052+0000 7facb9df1700 1 -- 172.21.15.100:0/1626510186 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563729278230 con 0x563729271090 2024-05-10T11:39:21.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.052+0000 7facb95f0700 1 --2- 172.21.15.100:0/1626510186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563729276380 0x5637292785b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:39:21.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.052+0000 7facb9df1700 1 --2- 172.21.15.100:0/1626510186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563729271090 0x5637292867c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7facb0005910 tx=0x7facb000b610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93b91eaa6e37868 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:39:21.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.053+0000 7fac9f7fe700 1 -- 172.21.15.100:0/1626510186 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb002d070 con 0x563729271090 2024-05-10T11:39:21.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.053+0000 7fac9f7fe700 1 -- 172.21.15.100:0/1626510186 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7facb00055c0 con 0x563729271090 2024-05-10T11:39:21.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.054+0000 7fac9f7fe700 1 -- 172.21.15.100:0/1626510186 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb000ba30 con 0x563729271090 2024-05-10T11:39:21.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.054+0000 7facc68e8b80 1 -- 172.21.15.100:0/1626510186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563729271090 msgr2=0x5637292867c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:21.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.054+0000 7facc68e8b80 1 --2- 172.21.15.100:0/1626510186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563729271090 0x5637292867c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7facb0005910 tx=0x7facb000b610 comp rx=0 tx=0).stop 2024-05-10T11:39:21.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.054+0000 7facc68e8b80 1 -- 172.21.15.100:0/1626510186 shutdown_connections 2024-05-10T11:39:21.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.054+0000 7facc68e8b80 1 --2- 172.21.15.100:0/1626510186 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563729276ca0 0x563729278af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:21.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.054+0000 7facc68e8b80 1 --2- 172.21.15.100:0/1626510186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563729271090 0x5637292867c0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:21.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.054+0000 7facc68e8b80 1 --2- 172.21.15.100:0/1626510186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563729276380 0x5637292785b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:21.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.054+0000 7facc68e8b80 1 -- 172.21.15.100:0/1626510186 >> 172.21.15.100:0/1626510186 conn(0x563729194100 msgr2=0x563729273d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:39:21.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.054+0000 7facc68e8b80 1 -- 172.21.15.100:0/1626510186 shutdown_connections 2024-05-10T11:39:21.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.055+0000 7facc68e8b80 1 -- 172.21.15.100:0/1626510186 wait complete. 2024-05-10T11:39:21.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.055+0000 7facc68e8b80 1 -- start start 2024-05-10T11:39:21.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.055+0000 7facc68e8b80 1 Processor -- start 2024-05-10T11:39:21.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.056+0000 7facc68e8b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563729271090 0x563729282b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:21.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.056+0000 7facc68e8b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563729276380 0x563729283080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:21.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.056+0000 7facc68e8b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563729276ca0 0x5637292f0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:21.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.056+0000 7facc68e8b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7facb41ab2f0 con 0x563729276380 2024-05-10T11:39:21.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.056+0000 7facc68e8b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7facb41ac760 con 0x563729276ca0 2024-05-10T11:39:21.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.056+0000 7facc68e8b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7facb41ab9c0 con 0x563729271090 2024-05-10T11:39:21.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.056+0000 7facc68e8b80 10 client.0 _ll_register_callbacks cb 0x563729152920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:39:21.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.056+0000 7facb9df1700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563729271090 0x563729282b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:21.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.056+0000 7facb9df1700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563729271090 0x563729282b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:47086/0 (socket says 172.21.15.100:47086) 2024-05-10T11:39:21.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.056+0000 7facb9df1700 1 -- 172.21.15.100:0/193920230 learned_addr learned my addr 172.21.15.100:0/193920230 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:39:21.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.056+0000 7facb95f0700 1 --2- 172.21.15.100:0/193920230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563729276380 0x563729283080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:21.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.056+0000 7facba5f2700 1 --2- 172.21.15.100:0/193920230 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563729276ca0 0x5637292f0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:21.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.057+0000 7facb9df1700 1 -- 172.21.15.100:0/193920230 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563729276ca0 msgr2=0x5637292f0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:21.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.057+0000 7facb9df1700 1 --2- 172.21.15.100:0/193920230 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563729276ca0 0x5637292f0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:21.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.057+0000 7facb9df1700 1 -- 172.21.15.100:0/193920230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563729276380 msgr2=0x563729283080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:21.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.057+0000 7facb9df1700 1 --2- 172.21.15.100:0/193920230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563729276380 0x563729283080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:21.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.057+0000 7facb9df1700 1 -- 172.21.15.100:0/193920230 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637292f0740 con 0x563729271090 2024-05-10T11:39:21.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.057+0000 7facb9df1700 1 --2- 172.21.15.100:0/193920230 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563729271090 0x563729282b40 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7facb000b2e0 tx=0x7facb00038b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9253a4119924c332 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:39:21.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.057+0000 7fac9d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:39:21.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.058+0000 7fac9d7fa700 1 -- 172.21.15.100:0/193920230 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb002d040 con 0x563729271090 2024-05-10T11:39:21.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.058+0000 7facc68e8b80 1 -- 172.21.15.100:0/193920230 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facb00055c0 con 0x563729271090 2024-05-10T11:39:21.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.058+0000 7fac9d7fa700 1 -- 172.21.15.100:0/193920230 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7facb0003ad0 con 0x563729271090 2024-05-10T11:39:21.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.058+0000 7fac9d7fa700 1 -- 172.21.15.100:0/193920230 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7facb0003cd0 con 0x563729271090 2024-05-10T11:39:21.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.060+0000 7facc68e8b80 10 client.28900 fetch_fsmap learned FSMap version 41 2024-05-10T11:39:21.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.060+0000 7facc68e8b80 1 -- 172.21.15.100:0/193920230 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x563729291db0 con 0x563729271090 2024-05-10T11:39:21.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.060+0000 7fac9d7fa700 1 -- 172.21.15.100:0/193920230 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7facb00055c0 con 0x563729271090 2024-05-10T11:39:21.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.060+0000 7fac9d7fa700 1 -- 172.21.15.100:0/193920230 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7facb000f770 con 0x563729271090 2024-05-10T11:39:21.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.060+0000 7facc68e8b80 10 client.28900 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:39:21.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.060+0000 7facc68e8b80 10 client.28900 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-10T11:39:21.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.060+0000 7facc68e8b80 20 client.28900 populate_metadata read hostname 'smithi100' 2024-05-10T11:39:21.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.060+0000 7facc68e8b80 1 --2- 172.21.15.100:0/193920230 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x563729306130 0x563729308520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:21.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.060+0000 7facc68e8b80 4 client.28900 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:39:21.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.060+0000 7facc68e8b80 1 -- 172.21.15.100:0/193920230 --> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5637291c8240 con 0x563729306130 2024-05-10T11:39:21.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.061+0000 7facb95f0700 1 --2- 172.21.15.100:0/193920230 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x563729306130 0x563729308520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:21.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.062+0000 7facb95f0700 1 --2- 172.21.15.100:0/193920230 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x563729306130 0x563729308520 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4db7c28b7facf524 server_cookie=9a8231ae491111f9 in_seq=0 out_seq=0 2024-05-10T11:39:21.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.062+0000 7fac9d7fa700 10 client.28900 ms_handle_connect on v2:172.21.15.188:6836/1943342620 2024-05-10T11:39:21.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.062+0000 7fac9d7fa700 1 -- 172.21.15.100:0/193920230 <== mds.2 v2:172.21.15.188:6836/1943342620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5637291c8240 con 0x563729306130 2024-05-10T11:39:21.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.062+0000 7fac9d7fa700 10 client.28900 handle_command_reply: tid=0 2024-05-10T11:39:21.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.062+0000 7facc68e8b80 1 client.28900 shutdown 2024-05-10T11:39:21.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.062+0000 7facc68e8b80 1 -- 172.21.15.100:0/193920230 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563729271090 msgr2=0x563729282b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:21.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.062+0000 7facc68e8b80 1 --2- 172.21.15.100:0/193920230 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563729271090 0x563729282b40 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7facb000b2e0 tx=0x7facb00038b0 comp rx=0 tx=0).stop 2024-05-10T11:39:21.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.063+0000 7facc68e8b80 1 -- 172.21.15.100:0/193920230 shutdown_connections 2024-05-10T11:39:21.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.063+0000 7facc68e8b80 1 --2- 172.21.15.100:0/193920230 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x563729306130 0x563729308520 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:21.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.063+0000 7facc68e8b80 1 --2- 172.21.15.100:0/193920230 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563729276ca0 0x5637292f0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:21.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.063+0000 7facc68e8b80 1 --2- 172.21.15.100:0/193920230 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563729271090 0x563729282b40 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:21.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.063+0000 7facc68e8b80 1 --2- 172.21.15.100:0/193920230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563729276380 0x563729283080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:21.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.063+0000 7facc68e8b80 1 -- 172.21.15.100:0/193920230 >> 172.21.15.100:0/193920230 conn(0x563729194100 msgr2=0x563729273d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:39:21.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.063+0000 7fac9d7fa700 0 client.28900 ms_handle_reset on v2:172.21.15.188:6836/1943342620 2024-05-10T11:39:21.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.064+0000 7facc68e8b80 1 -- 172.21.15.100:0/193920230 shutdown_connections 2024-05-10T11:39:21.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.064+0000 7facc68e8b80 1 -- 172.21.15.100:0/193920230 wait complete. 2024-05-10T11:39:21.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.064+0000 7facc68e8b80 20 client.28900 trim_cache size 0 max 16384 2024-05-10T11:39:21.066 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:39:21.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.065+0000 7facbc055700 1 -- 172.21.15.100:0/2179517559 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7faca00b2900 msgr2=0x7faca00b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:21.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.065+0000 7facbc055700 1 --2- 172.21.15.100:0/2179517559 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7faca00b2900 0x7faca00b4dc0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7facb4106f40 tx=0x7faca4009260 comp rx=0 tx=0).stop 2024-05-10T11:39:21.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.065+0000 7facbc055700 1 -- 172.21.15.100:0/2179517559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7facb4108cb0 msgr2=0x7facb419bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:21.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.065+0000 7facbc055700 1 --2- 172.21.15.100:0/2179517559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7facb4108cb0 0x7facb419bfa0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7facac007760 tx=0x7facac007a70 comp rx=0 tx=0).stop 2024-05-10T11:39:21.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.066+0000 7facbc055700 1 -- 172.21.15.100:0/2179517559 shutdown_connections 2024-05-10T11:39:21.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.066+0000 7facbc055700 1 --2- 172.21.15.100:0/2179517559 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7faca00b2900 0x7faca00b4dc0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:21.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.066+0000 7facbc055700 1 --2- 172.21.15.100:0/2179517559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7facb4108cb0 0x7facb419bfa0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:21.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.066+0000 7facbc055700 1 --2- 172.21.15.100:0/2179517559 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7facb41076e0 0x7facb41976d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:21.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.066+0000 7facbc055700 1 --2- 172.21.15.100:0/2179517559 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7facb4108300 0x7facb4197c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:21.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.066+0000 7facbc055700 1 -- 172.21.15.100:0/2179517559 >> 172.21.15.100:0/2179517559 conn(0x7facb40785a0 msgr2=0x7facb4100000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:39:21.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.066+0000 7facbc055700 1 -- 172.21.15.100:0/2179517559 shutdown_connections 2024-05-10T11:39:21.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:21.066+0000 7facbc055700 1 -- 172.21.15.100:0/2179517559 wait complete. 2024-05-10T11:39:21.767 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-10T11:39:21.789 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.788+0000 7fecbcf9d640 1 -- 172.21.15.100:0/166675039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fecb8103560 msgr2=0x7fecb8109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:21.790 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.788+0000 7fecbcf9d640 1 --2- 172.21.15.100:0/166675039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fecb8103560 0x7fecb8109df0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fecac009900 tx=0x7fecac02a3f0 comp rx=0 tx=0).stop 2024-05-10T11:39:21.790 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.789+0000 7fecbcf9d640 1 -- 172.21.15.100:0/166675039 shutdown_connections 2024-05-10T11:39:21.790 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.789+0000 7fecbcf9d640 1 --2- 172.21.15.100:0/166675039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fecb8103560 0x7fecb8109df0 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:21.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.789+0000 7fecbcf9d640 1 --2- 172.21.15.100:0/166675039 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fecb8102ba0 0x7fecb8103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:21.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.789+0000 7fecbcf9d640 1 --2- 172.21.15.100:0/166675039 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fecb81019b0 0x7fecb8101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:21.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.789+0000 7fecbcf9d640 1 -- 172.21.15.100:0/166675039 >> 172.21.15.100:0/166675039 conn(0x7fecb80fcf20 msgr2=0x7fecb80ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:39:21.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.789+0000 7fecbcf9d640 1 -- 172.21.15.100:0/166675039 shutdown_connections 2024-05-10T11:39:21.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.789+0000 7fecbcf9d640 1 -- 172.21.15.100:0/166675039 wait complete. 2024-05-10T11:39:21.793 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.790+0000 7fecbcf9d640 1 Processor -- start 2024-05-10T11:39:21.793 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.790+0000 7fecbcf9d640 1 -- start start 2024-05-10T11:39:21.793 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.790+0000 7fecbcf9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fecb81019b0 0x7fecb819ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:21.793 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.790+0000 7fecbcf9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fecb8102ba0 0x7fecb819b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:21.793 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.790+0000 7fecbcf9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fecb8103560 0x7fecb81a21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:21.793 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.790+0000 7fecbcf9d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fecb810c210 con 0x7fecb8103560 2024-05-10T11:39:21.793 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.790+0000 7fecbcf9d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fecb810c090 con 0x7fecb81019b0 2024-05-10T11:39:21.793 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.790+0000 7fecbcf9d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fecb810c390 con 0x7fecb8102ba0 2024-05-10T11:39:21.793 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.791+0000 7fecb659c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fecb81019b0 0x7fecb819ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:21.793 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.791+0000 7fecb5d9b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fecb8102ba0 0x7fecb819b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:21.793 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.791+0000 7fecb5d9b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fecb8102ba0 0x7fecb819b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:47094/0 (socket says 172.21.15.100:47094) 2024-05-10T11:39:21.793 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.791+0000 7fecb659c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fecb81019b0 0x7fecb819ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:43358/0 (socket says 172.21.15.100:43358) 2024-05-10T11:39:21.794 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.791+0000 7fecb5d9b640 1 -- 172.21.15.100:0/598483137 learned_addr learned my addr 172.21.15.100:0/598483137 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:39:21.794 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.791+0000 7fecb6d9d640 1 --2- 172.21.15.100:0/598483137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fecb8103560 0x7fecb81a21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:21.794 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.791+0000 7fecb5d9b640 1 -- 172.21.15.100:0/598483137 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fecb81019b0 msgr2=0x7fecb819ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:21.794 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.791+0000 7fecb5d9b640 1 --2- 172.21.15.100:0/598483137 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fecb81019b0 0x7fecb819ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:21.794 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.791+0000 7fecb5d9b640 1 -- 172.21.15.100:0/598483137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fecb8103560 msgr2=0x7fecb81a21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:21.794 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.791+0000 7fecb5d9b640 1 --2- 172.21.15.100:0/598483137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fecb8103560 0x7fecb81a21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:21.794 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.791+0000 7fecb5d9b640 1 -- 172.21.15.100:0/598483137 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecb81a2860 con 0x7fecb8102ba0 2024-05-10T11:39:21.794 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.791+0000 7fecb5d9b640 1 --2- 172.21.15.100:0/598483137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fecb8102ba0 0x7fecb819b190 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fecb000eb90 tx=0x7fecb000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:39:21.794 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.791+0000 7feca77fe640 1 -- 172.21.15.100:0/598483137 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb0004d20 con 0x7fecb8102ba0 2024-05-10T11:39:21.794 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.791+0000 7feca77fe640 1 -- 172.21.15.100:0/598483137 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fecb000ee90 con 0x7fecb8102ba0 2024-05-10T11:39:21.794 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.792+0000 7fecbcf9d640 1 -- 172.21.15.100:0/598483137 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecb81a2b50 con 0x7fecb8102ba0 2024-05-10T11:39:21.794 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.792+0000 7feca77fe640 1 -- 172.21.15.100:0/598483137 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb0010690 con 0x7fecb8102ba0 2024-05-10T11:39:21.794 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.792+0000 7fecbcf9d640 1 -- 172.21.15.100:0/598483137 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecb81a30a0 con 0x7fecb8102ba0 2024-05-10T11:39:21.796 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.795+0000 7fecbfd94740 1 Processor -- start 2024-05-10T11:39:21.796 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.795+0000 7feca77fe640 1 -- 172.21.15.100:0/598483137 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fecb0005020 con 0x7fecb8102ba0 2024-05-10T11:39:21.796 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.795+0000 7fecbfd94740 1 -- start start 2024-05-10T11:39:21.797 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.796+0000 7fecbfd94740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624f00a4db0 0x5624f00a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:21.797 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.796+0000 7fecbfd94740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5624f00a57a0 0x5624f00aa2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.796+0000 7fecbfd94740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5624f00aa910 0x5624f00acda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.796+0000 7fecbfd94740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5624f00095d0 con 0x5624f00a4db0 2024-05-10T11:39:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.796+0000 7fecbfd94740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5624effc2890 con 0x5624f00a57a0 2024-05-10T11:39:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.796+0000 7fecbfd94740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5624f0005590 con 0x5624f00aa910 2024-05-10T11:39:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.796+0000 7fecb6d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624f00a4db0 0x5624f00a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.796+0000 7fecb6d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624f00a4db0 0x5624f00a51d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41190/0 (socket says 172.21.15.100:41190) 2024-05-10T11:39:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.796+0000 7fecb6d9d640 1 -- 172.21.15.100:0/1790594341 learned_addr learned my addr 172.21.15.100:0/1790594341 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:39:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.796+0000 7fecb659c640 1 --2- 172.21.15.100:0/1790594341 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5624f00a57a0 0x5624f00aa2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.796+0000 7fecb5d9b640 1 --2- 172.21.15.100:0/1790594341 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5624f00aa910 0x5624f00acda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.797+0000 7feca77fe640 1 --2- 172.21.15.100:0/598483137 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7feca00b29a0 0x7feca00b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:21.799 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.797+0000 7fecb6d9d640 1 -- 172.21.15.100:0/1790594341 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5624f00aa910 msgr2=0x5624f00acda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:21.799 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.797+0000 7fecb6d9d640 1 --2- 172.21.15.100:0/1790594341 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5624f00aa910 0x5624f00acda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:21.799 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.797+0000 7fecb6d9d640 1 -- 172.21.15.100:0/1790594341 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5624f00a57a0 msgr2=0x5624f00aa2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:21.800 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.797+0000 7fecb6d9d640 1 --2- 172.21.15.100:0/1790594341 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5624f00a57a0 0x5624f00aa2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:21.800 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.797+0000 7fecb6d9d640 1 -- 172.21.15.100:0/1790594341 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624efd1b210 con 0x5624f00a4db0 2024-05-10T11:39:21.800 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.797+0000 7feca77fe640 1 -- 172.21.15.100:0/598483137 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fecb00e2bf0 con 0x7fecb8102ba0 2024-05-10T11:39:21.800 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.798+0000 7fecb6d9d640 1 --2- 172.21.15.100:0/1790594341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624f00a4db0 0x5624f00a51d0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fecac009230 tx=0x7fecac031500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aaafee6b5de40c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:39:21.800 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.798+0000 7fecb6d9d640 1 --2- 172.21.15.100:0/598483137 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7feca00b29a0 0x7feca00b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:21.800 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.798+0000 7feca4ff9640 1 -- 172.21.15.100:0/1790594341 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecac035030 con 0x5624f00a4db0 2024-05-10T11:39:21.800 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.798+0000 7feca4ff9640 1 -- 172.21.15.100:0/1790594341 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fecac042070 con 0x5624f00a4db0 2024-05-10T11:39:21.800 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.798+0000 7feca4ff9640 1 -- 172.21.15.100:0/1790594341 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecac003df0 con 0x5624f00a4db0 2024-05-10T11:39:21.801 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.799+0000 7fecbfd94740 1 -- 172.21.15.100:0/1790594341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624f00a4db0 msgr2=0x5624f00a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:21.801 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.799+0000 7fecbfd94740 1 --2- 172.21.15.100:0/1790594341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624f00a4db0 0x5624f00a51d0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fecac009230 tx=0x7fecac031500 comp rx=0 tx=0).stop 2024-05-10T11:39:21.801 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.799+0000 7fecb6d9d640 1 --2- 172.21.15.100:0/598483137 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7feca00b29a0 0x7feca00b4e60 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fecac005ec0 tx=0x7fecac046000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:39:21.801 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.800+0000 7fecbfd94740 1 -- 172.21.15.100:0/1790594341 shutdown_connections 2024-05-10T11:39:21.801 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.800+0000 7fecbfd94740 1 --2- 172.21.15.100:0/1790594341 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5624f00aa910 0x5624f00acda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:21.801 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.800+0000 7fecbfd94740 1 --2- 172.21.15.100:0/1790594341 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5624f00a57a0 0x5624f00aa2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:21.801 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.800+0000 7fecbfd94740 1 --2- 172.21.15.100:0/1790594341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624f00a4db0 0x5624f00a51d0 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:21.801 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.800+0000 7fecbfd94740 1 -- 172.21.15.100:0/1790594341 >> 172.21.15.100:0/1790594341 conn(0x5624f00a0420 msgr2=0x5624f00a0820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:39:21.802 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.800+0000 7fecbfd94740 1 -- 172.21.15.100:0/1790594341 shutdown_connections 2024-05-10T11:39:21.802 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.800+0000 7fecbfd94740 1 -- 172.21.15.100:0/1790594341 wait complete. 2024-05-10T11:39:21.802 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.801+0000 7fecbfd94740 1 -- start start 2024-05-10T11:39:21.802 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.801+0000 7fecbfd94740 1 Processor -- start 2024-05-10T11:39:21.802 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.801+0000 7fecbfd94740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624f00a4db0 0x5624f00aef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:21.805 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.802+0000 7fecbfd94740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5624f00a57a0 0x5624f00af4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:21.805 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.802+0000 7fecbfd94740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5624f00aa910 0x5624f00b64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:21.805 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.802+0000 7fecb5d9b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5624f00a57a0 0x5624f00af4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:21.805 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.802+0000 7fecb659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624f00a4db0 0x5624f00aef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:21.805 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.802+0000 7fecb5d9b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5624f00a57a0 0x5624f00af4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:43376/0 (socket says 172.21.15.100:43376) 2024-05-10T11:39:21.805 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.802+0000 7fecb659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624f00a4db0 0x5624f00aef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41206/0 (socket says 172.21.15.100:41206) 2024-05-10T11:39:21.805 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.802+0000 7fecbfd94740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5624f00ad860 con 0x5624f00a4db0 2024-05-10T11:39:21.805 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.802+0000 7fecb5d9b640 1 -- 172.21.15.100:0/850092082 learned_addr learned my addr 172.21.15.100:0/850092082 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:39:21.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.802+0000 7fecbfd94740 1 -- 172.21.15.100:0/850092082 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5624effc6370 con 0x5624f00a57a0 2024-05-10T11:39:21.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.802+0000 7fecbfd94740 1 -- 172.21.15.100:0/850092082 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5624f00ad9e0 con 0x5624f00aa910 2024-05-10T11:39:21.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.802+0000 7fecbfd94740 10 client.0 _ll_register_callbacks cb 0x5624efd1aa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:39:21.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.802+0000 7fecb6d9d640 1 --2- 172.21.15.100:0/850092082 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5624f00aa910 0x5624f00b64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:21.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.802+0000 7fecb5d9b640 1 -- 172.21.15.100:0/850092082 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5624f00aa910 msgr2=0x5624f00b64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:21.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.802+0000 7fecb5d9b640 1 --2- 172.21.15.100:0/850092082 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5624f00aa910 0x5624f00b64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:21.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.802+0000 7fecb5d9b640 1 -- 172.21.15.100:0/850092082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624f00a4db0 msgr2=0x5624f00aef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:21.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.802+0000 7fecb5d9b640 1 --2- 172.21.15.100:0/850092082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624f00a4db0 0x5624f00aef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:21.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.802+0000 7fecb5d9b640 1 -- 172.21.15.100:0/850092082 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624efde7620 con 0x5624f00a57a0 2024-05-10T11:39:21.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.802+0000 7fecb659c640 1 --2- 172.21.15.100:0/850092082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624f00a4db0 0x5624f00aef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:39:21.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.803+0000 7fecb5d9b640 1 --2- 172.21.15.100:0/850092082 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5624f00a57a0 0x5624f00af4b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fecb00049d0 tx=0x7fecb00a9ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4fcf321818881b66 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:39:21.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.803+0000 7fec9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:39:21.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.803+0000 7fec9dffb640 1 -- 172.21.15.100:0/850092082 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb00ea040 con 0x5624f00a57a0 2024-05-10T11:39:21.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.803+0000 7fec9dffb640 1 -- 172.21.15.100:0/850092082 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fecb00b2a30 con 0x5624f00a57a0 2024-05-10T11:39:21.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.803+0000 7fecbfd94740 1 -- 172.21.15.100:0/850092082 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fecac042070 con 0x5624f00a57a0 2024-05-10T11:39:21.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.803+0000 7fec9dffb640 1 -- 172.21.15.100:0/850092082 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fecb00aa550 con 0x5624f00a57a0 2024-05-10T11:39:21.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.803+0000 7fecbfd94740 10 client.28805 fetch_fsmap learned FSMap version 41 2024-05-10T11:39:21.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.803+0000 7fecbfd94740 1 -- 172.21.15.100:0/850092082 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5624f00b9590 con 0x5624f00a57a0 2024-05-10T11:39:21.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.804+0000 7fec9dffb640 1 -- 172.21.15.100:0/850092082 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fecac042070 con 0x5624f00a57a0 2024-05-10T11:39:21.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.804+0000 7fec9dffb640 1 -- 172.21.15.100:0/850092082 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fecb00a7da0 con 0x5624f00a57a0 2024-05-10T11:39:21.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.804+0000 7fecbfd94740 10 client.28805 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:39:21.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.804+0000 7fecbfd94740 10 client.28805 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:39:21.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.804+0000 7fecbfd94740 20 client.28805 populate_metadata read hostname 'smithi100' 2024-05-10T11:39:21.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.804+0000 7fecbfd94740 1 --2- 172.21.15.100:0/850092082 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5624f00bab60 0x5624f00bcfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:21.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.804+0000 7fecbfd94740 4 client.28805 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:39:21.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.804+0000 7fecbfd94740 1 -- 172.21.15.100:0/850092082 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624f00171e0 con 0x5624f00bab60 2024-05-10T11:39:21.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.804+0000 7fecb659c640 1 --2- 172.21.15.100:0/850092082 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5624f00bab60 0x5624f00bcfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:21.808 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.805+0000 7fec9dffb640 10 client.28805 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:39:21.809 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.805+0000 7fecb659c640 1 --2- 172.21.15.100:0/850092082 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5624f00bab60 0x5624f00bcfd0 crc :-1 s=READY pgs=399 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65077a0f07a0a536 server_cookie=e389e776575205a4 in_seq=0 out_seq=0 2024-05-10T11:39:21.809 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.806+0000 7fec9dffb640 1 -- 172.21.15.100:0/850092082 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5624f00171e0 con 0x5624f00bab60 2024-05-10T11:39:21.809 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.806+0000 7fec9dffb640 10 client.28805 handle_command_reply: tid=0 2024-05-10T11:39:21.809 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.806+0000 7fecbfd94740 1 client.28805 shutdown 2024-05-10T11:39:21.809 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.806+0000 7fecbfd94740 1 -- 172.21.15.100:0/850092082 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5624f00a57a0 msgr2=0x5624f00af4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:21.809 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.806+0000 7fecbfd94740 1 --2- 172.21.15.100:0/850092082 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5624f00a57a0 0x5624f00af4b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fecb00049d0 tx=0x7fecb00a9ce0 comp rx=0 tx=0).stop 2024-05-10T11:39:21.809 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.806+0000 7fecbfd94740 1 -- 172.21.15.100:0/850092082 shutdown_connections 2024-05-10T11:39:21.809 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.806+0000 7fecbfd94740 1 --2- 172.21.15.100:0/850092082 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5624f00bab60 0x5624f00bcfd0 crc :-1 s=READY pgs=399 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:21.809 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.806+0000 7fecbfd94740 1 --2- 172.21.15.100:0/850092082 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5624f00aa910 0x5624f00b64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:21.809 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.806+0000 7fecbfd94740 1 --2- 172.21.15.100:0/850092082 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5624f00a57a0 0x5624f00af4b0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:21.809 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.806+0000 7fecbfd94740 1 --2- 172.21.15.100:0/850092082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624f00a4db0 0x5624f00aef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:21.809 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.806+0000 7fec9dffb640 0 client.28805 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:39:21.809 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.806+0000 7fecbfd94740 1 -- 172.21.15.100:0/850092082 >> 172.21.15.100:0/850092082 conn(0x5624f00a0420 msgr2=0x5624f009f580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:39:21.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.806+0000 7fecbfd94740 1 -- 172.21.15.100:0/850092082 shutdown_connections 2024-05-10T11:39:21.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.806+0000 7fecbfd94740 1 -- 172.21.15.100:0/850092082 wait complete. 2024-05-10T11:39:21.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.807+0000 7fecbfd94740 20 client.28805 trim_cache size 0 max 16384 2024-05-10T11:39:21.837 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.835+0000 7fecbfd94740 1 Processor -- start 2024-05-10T11:39:21.837 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.835+0000 7fecbfd94740 1 -- start start 2024-05-10T11:39:21.837 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.836+0000 7fecbfd94740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5624f00a4db0 0x5624f009ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:21.837 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.836+0000 7fecbfd94740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624f00a57a0 0x5624f009f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.836+0000 7fecbfd94740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5624f00aa910 0x5624f009f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.836+0000 7fecbfd94740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fecb81afe40 con 0x5624f00a57a0 2024-05-10T11:39:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.836+0000 7fecbfd94740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fecb81ad710 con 0x5624f00a4db0 2024-05-10T11:39:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.836+0000 7fecbfd94740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fecb81ab560 con 0x5624f00aa910 2024-05-10T11:39:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.836+0000 7fecb6d9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5624f00aa910 0x5624f009f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.836+0000 7fecb5d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624f00a57a0 0x5624f009f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.836+0000 7fecb6d9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5624f00aa910 0x5624f009f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:47106/0 (socket says 172.21.15.100:47106) 2024-05-10T11:39:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.836+0000 7fecb6d9d640 1 -- 172.21.15.100:0/1630909220 learned_addr learned my addr 172.21.15.100:0/1630909220 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:39:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.836+0000 7fecb659c640 1 --2- 172.21.15.100:0/1630909220 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5624f00a4db0 0x5624f009ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.836+0000 7fecb5d9b640 1 -- 172.21.15.100:0/1630909220 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5624f00aa910 msgr2=0x5624f009f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.836+0000 7fecb5d9b640 1 --2- 172.21.15.100:0/1630909220 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5624f00aa910 0x5624f009f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.836+0000 7fecb5d9b640 1 -- 172.21.15.100:0/1630909220 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5624f00a4db0 msgr2=0x5624f009ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.836+0000 7fecb5d9b640 1 --2- 172.21.15.100:0/1630909220 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5624f00a4db0 0x5624f009ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.836+0000 7fecb5d9b640 1 -- 172.21.15.100:0/1630909220 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624f00bd650 con 0x5624f00a57a0 2024-05-10T11:39:21.839 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.836+0000 7fecb6d9d640 1 --2- 172.21.15.100:0/1630909220 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5624f00aa910 0x5624f009f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:39:21.839 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.836+0000 7fecb659c640 1 --2- 172.21.15.100:0/1630909220 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5624f00a4db0 0x5624f009ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:39:21.841 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.837+0000 7fecb5d9b640 1 --2- 172.21.15.100:0/1630909220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624f00a57a0 0x5624f009f230 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fecb00a57d0 tx=0x7fecb0009a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=609ee162bb443953 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:39:21.841 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.837+0000 7fec9ffff640 1 -- 172.21.15.100:0/1630909220 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb0089070 con 0x5624f00a57a0 2024-05-10T11:39:21.841 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.837+0000 7fec9ffff640 1 -- 172.21.15.100:0/1630909220 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fecb00ac070 con 0x5624f00a57a0 2024-05-10T11:39:21.841 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.837+0000 7fec9ffff640 1 -- 172.21.15.100:0/1630909220 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb00ae200 con 0x5624f00a57a0 2024-05-10T11:39:21.841 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.838+0000 7fecbfd94740 1 -- 172.21.15.100:0/1630909220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624f00a57a0 msgr2=0x5624f009f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:21.841 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.838+0000 7fecbfd94740 1 --2- 172.21.15.100:0/1630909220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624f00a57a0 0x5624f009f230 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fecb00a57d0 tx=0x7fecb0009a40 comp rx=0 tx=0).stop 2024-05-10T11:39:21.841 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.838+0000 7fecbfd94740 1 -- 172.21.15.100:0/1630909220 shutdown_connections 2024-05-10T11:39:21.841 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.838+0000 7fecbfd94740 1 --2- 172.21.15.100:0/1630909220 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5624f00aa910 0x5624f009f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.838+0000 7fecbfd94740 1 --2- 172.21.15.100:0/1630909220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624f00a57a0 0x5624f009f230 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.838+0000 7fecbfd94740 1 --2- 172.21.15.100:0/1630909220 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5624f00a4db0 0x5624f009ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.838+0000 7fecbfd94740 1 -- 172.21.15.100:0/1630909220 >> 172.21.15.100:0/1630909220 conn(0x5624f00a0420 msgr2=0x5624f00a6980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:39:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.838+0000 7fecbfd94740 1 -- 172.21.15.100:0/1630909220 shutdown_connections 2024-05-10T11:39:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.839+0000 7fecbfd94740 1 -- 172.21.15.100:0/1630909220 wait complete. 2024-05-10T11:39:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.839+0000 7fecbfd94740 1 -- start start 2024-05-10T11:39:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.839+0000 7fecbfd94740 1 Processor -- start 2024-05-10T11:39:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.839+0000 7fecbfd94740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5624f00a4db0 0x5624f00b3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.839+0000 7fecbfd94740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5624f00a57a0 0x5624f00b3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.839+0000 7fecbfd94740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624f00aa910 0x5624f00b4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.839+0000 7fecbfd94740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fecb81abbd0 con 0x5624f00aa910 2024-05-10T11:39:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.839+0000 7fecbfd94740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fecb81ad040 con 0x5624f00a4db0 2024-05-10T11:39:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.839+0000 7fecbfd94740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fecb81ac2a0 con 0x5624f00a57a0 2024-05-10T11:39:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.839+0000 7fecb6d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624f00aa910 0x5624f00b4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.839+0000 7fecb5d9b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5624f00a57a0 0x5624f00b3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.839+0000 7fecb6d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624f00aa910 0x5624f00b4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41232/0 (socket says 172.21.15.100:41232) 2024-05-10T11:39:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.839+0000 7fecb659c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5624f00a4db0 0x5624f00b3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.839+0000 7fecb6d9d640 1 -- 172.21.15.100:0/1889925328 learned_addr learned my addr 172.21.15.100:0/1889925328 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:39:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.840+0000 7fecbfd94740 10 client.0 _ll_register_callbacks cb 0x7fecb00e60e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:39:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.840+0000 7fecb6d9d640 1 -- 172.21.15.100:0/1889925328 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5624f00a57a0 msgr2=0x5624f00b3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.840+0000 7fecb6d9d640 1 --2- 172.21.15.100:0/1889925328 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5624f00a57a0 0x5624f00b3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.840+0000 7fecb6d9d640 1 -- 172.21.15.100:0/1889925328 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5624f00a4db0 msgr2=0x5624f00b3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.840+0000 7fecb6d9d640 1 --2- 172.21.15.100:0/1889925328 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5624f00a4db0 0x5624f00b3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.840+0000 7fecb6d9d640 1 -- 172.21.15.100:0/1889925328 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624f00a0250 con 0x5624f00aa910 2024-05-10T11:39:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.840+0000 7fecb5d9b640 1 --2- 172.21.15.100:0/1889925328 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5624f00a57a0 0x5624f00b3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:39:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.840+0000 7fecb6d9d640 1 --2- 172.21.15.100:0/1889925328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624f00aa910 0x5624f00b4c90 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fecac005c40 tx=0x7fecac031bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c6944cfb4a5c9ed server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:39:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.840+0000 7fec9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:39:21.845 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.840+0000 7fec9dffb640 1 -- 172.21.15.100:0/1889925328 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecac05f060 con 0x5624f00aa910 2024-05-10T11:39:21.845 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.840+0000 7fec9dffb640 1 -- 172.21.15.100:0/1889925328 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fecac0035f0 con 0x5624f00aa910 2024-05-10T11:39:21.845 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.841+0000 7fecbfd94740 1 -- 172.21.15.100:0/1889925328 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fecb00ac070 con 0x5624f00aa910 2024-05-10T11:39:21.845 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.841+0000 7fec9dffb640 1 -- 172.21.15.100:0/1889925328 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fecac035050 con 0x5624f00aa910 2024-05-10T11:39:21.846 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.841+0000 7fecbfd94740 10 client.19545 fetch_fsmap learned FSMap version 41 2024-05-10T11:39:21.846 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.841+0000 7fecbfd94740 1 -- 172.21.15.100:0/1889925328 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5624f00a6470 con 0x5624f00aa910 2024-05-10T11:39:21.846 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.842+0000 7fec9dffb640 1 -- 172.21.15.100:0/1889925328 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fecb00ac070 con 0x5624f00aa910 2024-05-10T11:39:21.846 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.842+0000 7fec9dffb640 1 -- 172.21.15.100:0/1889925328 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fecac059070 con 0x5624f00aa910 2024-05-10T11:39:21.846 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.842+0000 7fecbfd94740 10 client.19545 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:39:21.846 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.842+0000 7fecbfd94740 10 client.19545 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:39:21.846 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.842+0000 7fecbfd94740 20 client.19545 populate_metadata read hostname 'smithi100' 2024-05-10T11:39:21.846 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.842+0000 7fecbfd94740 1 --2- 172.21.15.100:0/1889925328 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5624f0125bd0 0x5624f0127fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:21.846 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.843+0000 7fecbfd94740 4 client.19545 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:39:21.846 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.843+0000 7fecbfd94740 1 -- 172.21.15.100:0/1889925328 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5624f0011050 con 0x5624f0125bd0 2024-05-10T11:39:21.846 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.843+0000 7fecb659c640 1 --2- 172.21.15.100:0/1889925328 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5624f0125bd0 0x5624f0127fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:21.847 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.845+0000 7fecb659c640 1 --2- 172.21.15.100:0/1889925328 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5624f0125bd0 0x5624f0127fc0 crc :-1 s=READY pgs=400 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1fb23b5f5e1385d server_cookie=3ca2a1b7bfc4a8af in_seq=0 out_seq=0 2024-05-10T11:39:21.847 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.845+0000 7fec9dffb640 10 client.19545 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:39:21.946 INFO:tasks.workunit.client.0.smithi100.stdout: 10 73011 9.00 MB/sec execute 315 sec latency 123.417 ms 2024-05-10T11:39:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.999+0000 7fec9dffb640 1 -- 172.21.15.100:0/1889925328 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5624f0011050 con 0x5624f0125bd0 2024-05-10T11:39:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.999+0000 7fec9dffb640 10 client.19545 handle_command_reply: tid=0 2024-05-10T11:39:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:21.999+0000 7fecbfd94740 1 client.19545 shutdown 2024-05-10T11:39:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:22.000+0000 7fecbfd94740 1 -- 172.21.15.100:0/1889925328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624f00aa910 msgr2=0x5624f00b4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:22.000+0000 7fecbfd94740 1 --2- 172.21.15.100:0/1889925328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624f00aa910 0x5624f00b4c90 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fecac005c40 tx=0x7fecac031bc0 comp rx=0 tx=0).stop 2024-05-10T11:39:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:22.000+0000 7fecbfd94740 1 -- 172.21.15.100:0/1889925328 shutdown_connections 2024-05-10T11:39:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:22.000+0000 7fecbfd94740 1 --2- 172.21.15.100:0/1889925328 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5624f0125bd0 0x5624f0127fc0 crc :-1 s=READY pgs=400 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:22.000+0000 7fecbfd94740 1 --2- 172.21.15.100:0/1889925328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624f00aa910 0x5624f00b4c90 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:22.000+0000 7fecbfd94740 1 --2- 172.21.15.100:0/1889925328 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5624f00a57a0 0x5624f00b3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:22.000+0000 7fec9dffb640 0 client.19545 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:39:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:22.000+0000 7fecbfd94740 1 --2- 172.21.15.100:0/1889925328 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5624f00a4db0 0x5624f00b3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:22.000+0000 7fecbfd94740 1 -- 172.21.15.100:0/1889925328 >> 172.21.15.100:0/1889925328 conn(0x5624f00a0420 msgr2=0x5624f00a31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:39:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:22.000+0000 7fecbfd94740 1 -- 172.21.15.100:0/1889925328 shutdown_connections 2024-05-10T11:39:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:22.000+0000 7fecbfd94740 1 -- 172.21.15.100:0/1889925328 wait complete. 2024-05-10T11:39:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:22.000+0000 7fecbfd94740 20 client.19545 trim_cache size 0 max 16384 2024-05-10T11:39:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:39:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:39:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:39:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:39:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:22.001+0000 7fecbcf9d640 1 -- 172.21.15.100:0/598483137 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7feca00b29a0 msgr2=0x7feca00b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:22.001+0000 7fecbcf9d640 1 --2- 172.21.15.100:0/598483137 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7feca00b29a0 0x7feca00b4e60 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fecac005ec0 tx=0x7fecac046000 comp rx=0 tx=0).stop 2024-05-10T11:39:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:22.001+0000 7fecbcf9d640 1 -- 172.21.15.100:0/598483137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fecb8102ba0 msgr2=0x7fecb819b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:22.001+0000 7fecbcf9d640 1 --2- 172.21.15.100:0/598483137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fecb8102ba0 0x7fecb819b190 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fecb000eb90 tx=0x7fecb000c6a0 comp rx=0 tx=0).stop 2024-05-10T11:39:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:22.001+0000 7fecbcf9d640 1 -- 172.21.15.100:0/598483137 shutdown_connections 2024-05-10T11:39:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:22.001+0000 7fecbcf9d640 1 --2- 172.21.15.100:0/598483137 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7feca00b29a0 0x7feca00b4e60 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:22.001+0000 7fecbcf9d640 1 --2- 172.21.15.100:0/598483137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fecb8103560 0x7fecb81a21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:22.001+0000 7fecbcf9d640 1 --2- 172.21.15.100:0/598483137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fecb8102ba0 0x7fecb819b190 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:22.001+0000 7fecbcf9d640 1 --2- 172.21.15.100:0/598483137 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fecb81019b0 0x7fecb819ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:22.001+0000 7fecbcf9d640 1 -- 172.21.15.100:0/598483137 >> 172.21.15.100:0/598483137 conn(0x7fecb80fcf20 msgr2=0x7fecb8105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:39:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:22.001+0000 7fecbcf9d640 1 -- 172.21.15.100:0/598483137 shutdown_connections 2024-05-10T11:39:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:22.001+0000 7fecbcf9d640 1 -- 172.21.15.100:0/598483137 wait complete. 2024-05-10T11:39:22.005 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:39:21 smithi178 ceph-mon[33359]: pgmap v1050: 129 pgs: 129 active+clean; 521 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 871 KiB/s rd, 4.8 MiB/s wr, 307 op/s 2024-05-10T11:39:22.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:39:21 smithi188 ceph-mon[34075]: pgmap v1050: 129 pgs: 129 active+clean; 521 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 871 KiB/s rd, 4.8 MiB/s wr, 307 op/s 2024-05-10T11:39:22.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:39:21 smithi100 ceph-mon[30296]: pgmap v1050: 129 pgs: 129 active+clean; 521 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 871 KiB/s rd, 4.8 MiB/s wr, 307 op/s 2024-05-10T11:39:22.160 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:39:22.946 INFO:tasks.workunit.client.0.smithi100.stdout: 10 73182 9.00 MB/sec execute 316 sec latency 126.530 ms 2024-05-10T11:39:23.946 INFO:tasks.workunit.client.0.smithi100.stdout: 10 73295 8.98 MB/sec execute 317 sec latency 159.955 ms 2024-05-10T11:39:24.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:39:23 smithi178 ceph-mon[33359]: pgmap v1051: 129 pgs: 129 active+clean; 515 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 4.1 MiB/s wr, 299 op/s 2024-05-10T11:39:24.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:39:23 smithi188 ceph-mon[34075]: pgmap v1051: 129 pgs: 129 active+clean; 515 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 4.1 MiB/s wr, 299 op/s 2024-05-10T11:39:24.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:39:23 smithi100 ceph-mon[30296]: pgmap v1051: 129 pgs: 129 active+clean; 515 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 4.1 MiB/s wr, 299 op/s 2024-05-10T11:39:24.946 INFO:tasks.workunit.client.0.smithi100.stdout: 10 73465 8.99 MB/sec execute 318 sec latency 160.782 ms 2024-05-10T11:39:25.947 INFO:tasks.workunit.client.0.smithi100.stdout: 10 73687 9.02 MB/sec execute 319 sec latency 120.611 ms 2024-05-10T11:39:26.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:39:25 smithi178 ceph-mon[33359]: pgmap v1052: 129 pgs: 129 active+clean; 517 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 3.8 MiB/s wr, 298 op/s 2024-05-10T11:39:26.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:39:25 smithi188 ceph-mon[34075]: pgmap v1052: 129 pgs: 129 active+clean; 517 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 3.8 MiB/s wr, 298 op/s 2024-05-10T11:39:26.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:39:25 smithi100 ceph-mon[30296]: pgmap v1052: 129 pgs: 129 active+clean; 517 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 3.8 MiB/s wr, 298 op/s 2024-05-10T11:39:26.946 INFO:tasks.workunit.client.0.smithi100.stdout: 10 73861 9.02 MB/sec execute 320 sec latency 148.058 ms 2024-05-10T11:39:27.946 INFO:tasks.workunit.client.0.smithi100.stdout: 10 73950 9.00 MB/sec execute 321 sec latency 123.763 ms 2024-05-10T11:39:28.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:39:27 smithi178 ceph-mon[33359]: pgmap v1053: 129 pgs: 129 active+clean; 522 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 3.9 MiB/s wr, 309 op/s 2024-05-10T11:39:28.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:39:27 smithi188 ceph-mon[34075]: pgmap v1053: 129 pgs: 129 active+clean; 522 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 3.9 MiB/s wr, 309 op/s 2024-05-10T11:39:28.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:39:27 smithi100 ceph-mon[30296]: pgmap v1053: 129 pgs: 129 active+clean; 522 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 3.9 MiB/s wr, 309 op/s 2024-05-10T11:39:28.947 INFO:tasks.workunit.client.0.smithi100.stdout: 10 74062 9.00 MB/sec execute 322 sec latency 166.119 ms 2024-05-10T11:39:29.947 INFO:tasks.workunit.client.0.smithi100.stdout: 10 74213 9.00 MB/sec execute 323 sec latency 153.608 ms 2024-05-10T11:39:30.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:39:29 smithi178 ceph-mon[33359]: pgmap v1054: 129 pgs: 129 active+clean; 526 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 3.6 MiB/s wr, 276 op/s 2024-05-10T11:39:30.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:39:29 smithi188 ceph-mon[34075]: pgmap v1054: 129 pgs: 129 active+clean; 526 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 3.6 MiB/s wr, 276 op/s 2024-05-10T11:39:30.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:39:29 smithi100 ceph-mon[30296]: pgmap v1054: 129 pgs: 129 active+clean; 526 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 3.6 MiB/s wr, 276 op/s 2024-05-10T11:39:30.947 INFO:tasks.workunit.client.0.smithi100.stdout: 10 74355 9.02 MB/sec execute 324 sec latency 150.551 ms 2024-05-10T11:39:31.947 INFO:tasks.workunit.client.0.smithi100.stdout: 10 74503 9.01 MB/sec execute 325 sec latency 129.978 ms 2024-05-10T11:39:32.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:39:31 smithi178 ceph-mon[33359]: pgmap v1055: 129 pgs: 129 active+clean; 532 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 4.0 MiB/s wr, 288 op/s 2024-05-10T11:39:32.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:39:31 smithi188 ceph-mon[34075]: pgmap v1055: 129 pgs: 129 active+clean; 532 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 4.0 MiB/s wr, 288 op/s 2024-05-10T11:39:32.033 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:39:31 smithi100 ceph-mon[30296]: pgmap v1055: 129 pgs: 129 active+clean; 532 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 4.0 MiB/s wr, 288 op/s 2024-05-10T11:39:32.947 INFO:tasks.workunit.client.0.smithi100.stdout: 10 74665 9.00 MB/sec execute 326 sec latency 141.737 ms 2024-05-10T11:39:33.947 INFO:tasks.workunit.client.0.smithi100.stdout: 10 74761 8.98 MB/sec execute 327 sec latency 162.622 ms 2024-05-10T11:39:34.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:39:33 smithi178 ceph-mon[33359]: pgmap v1056: 129 pgs: 129 active+clean; 533 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 3.6 MiB/s wr, 233 op/s 2024-05-10T11:39:34.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:39:33 smithi188 ceph-mon[34075]: pgmap v1056: 129 pgs: 129 active+clean; 533 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 3.6 MiB/s wr, 233 op/s 2024-05-10T11:39:34.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:39:33 smithi100 ceph-mon[30296]: pgmap v1056: 129 pgs: 129 active+clean; 533 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 3.6 MiB/s wr, 233 op/s 2024-05-10T11:39:34.947 INFO:tasks.workunit.client.0.smithi100.stdout: 10 74877 8.96 MB/sec execute 328 sec latency 112.756 ms 2024-05-10T11:39:35.137 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.135+0000 7f5caa551640 1 -- 172.21.15.100:0/1008746407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ca41034c0 msgr2=0x7f5ca4109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:35.137 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.135+0000 7f5caa551640 1 --2- 172.21.15.100:0/1008746407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ca41034c0 0x7f5ca4109d50 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c009ca0 tx=0x7f5c9c02a8c0 comp rx=0 tx=0).stop 2024-05-10T11:39:35.137 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.135+0000 7f5caa551640 1 -- 172.21.15.100:0/1008746407 shutdown_connections 2024-05-10T11:39:35.137 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.135+0000 7f5caa551640 1 --2- 172.21.15.100:0/1008746407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ca41034c0 0x7f5ca4109d50 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:35.137 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.135+0000 7f5caa551640 1 --2- 172.21.15.100:0/1008746407 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ca4102b00 0x7f5ca4102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:35.137 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.135+0000 7f5caa551640 1 --2- 172.21.15.100:0/1008746407 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ca4101910 0x7f5ca4101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:35.137 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.135+0000 7f5caa551640 1 -- 172.21.15.100:0/1008746407 >> 172.21.15.100:0/1008746407 conn(0x7f5ca40fce80 msgr2=0x7f5ca40ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:39:35.137 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.136+0000 7f5caa551640 1 -- 172.21.15.100:0/1008746407 shutdown_connections 2024-05-10T11:39:35.137 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.136+0000 7f5caa551640 1 -- 172.21.15.100:0/1008746407 wait complete. 2024-05-10T11:39:35.137 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.136+0000 7f5caa551640 1 Processor -- start 2024-05-10T11:39:35.137 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.136+0000 7f5caa551640 1 -- start start 2024-05-10T11:39:35.138 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.137+0000 7f5caa551640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ca4101910 0x7f5ca410c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:35.138 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.137+0000 7f5caa551640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ca4102b00 0x7f5ca410a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:35.139 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.137+0000 7f5caa551640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ca41034c0 0x7f5ca410ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:35.139 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.137+0000 7f5ca3fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ca4101910 0x7f5ca410c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:35.139 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.137+0000 7f5caa551640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca410e470 con 0x7f5ca4102b00 2024-05-10T11:39:35.139 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.137+0000 7f5caa551640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca410e2f0 con 0x7f5ca41034c0 2024-05-10T11:39:35.139 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.137+0000 7f5caa551640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca410e5f0 con 0x7f5ca4101910 2024-05-10T11:39:35.139 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.137+0000 7f5ca3fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ca4101910 0x7f5ca410c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:58566/0 (socket says 172.21.15.100:58566) 2024-05-10T11:39:35.140 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.137+0000 7f5ca3fff640 1 -- 172.21.15.100:0/635317173 learned_addr learned my addr 172.21.15.100:0/635317173 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:39:35.140 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.137+0000 7f5c9bfff640 1 --2- 172.21.15.100:0/635317173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ca4102b00 0x7f5ca410a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:35.140 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.137+0000 7f5ca8aee640 1 --2- 172.21.15.100:0/635317173 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ca41034c0 0x7f5ca410ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:35.140 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.137+0000 7f5ca3fff640 1 -- 172.21.15.100:0/635317173 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ca41034c0 msgr2=0x7f5ca410ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:35.140 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.137+0000 7f5ca3fff640 1 --2- 172.21.15.100:0/635317173 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ca41034c0 0x7f5ca410ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:35.140 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.137+0000 7f5ca3fff640 1 -- 172.21.15.100:0/635317173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ca4102b00 msgr2=0x7f5ca410a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:35.140 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.137+0000 7f5ca3fff640 1 --2- 172.21.15.100:0/635317173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ca4102b00 0x7f5ca410a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:35.140 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.137+0000 7f5ca3fff640 1 -- 172.21.15.100:0/635317173 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ca410b6a0 con 0x7f5ca4101910 2024-05-10T11:39:35.140 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.137+0000 7f5ca8aee640 1 --2- 172.21.15.100:0/635317173 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ca41034c0 0x7f5ca410ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:39:35.140 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.137+0000 7f5c9bfff640 1 --2- 172.21.15.100:0/635317173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ca4102b00 0x7f5ca410a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:39:35.140 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.138+0000 7f5ca3fff640 1 --2- 172.21.15.100:0/635317173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ca4101910 0x7f5ca410c290 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5c9400e9c0 tx=0x7f5c9400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:39:35.141 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.138+0000 7f5ca1ffb640 1 -- 172.21.15.100:0/635317173 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9400cd80 con 0x7f5ca4101910 2024-05-10T11:39:35.141 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.138+0000 7f5ca1ffb640 1 -- 172.21.15.100:0/635317173 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5c940044b0 con 0x7f5ca4101910 2024-05-10T11:39:35.141 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.138+0000 7f5ca1ffb640 1 -- 172.21.15.100:0/635317173 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c94010430 con 0x7f5ca4101910 2024-05-10T11:39:35.141 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.138+0000 7f5caa551640 1 -- 172.21.15.100:0/635317173 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ca41a2ba0 con 0x7f5ca4101910 2024-05-10T11:39:35.141 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.138+0000 7f5caa551640 1 -- 172.21.15.100:0/635317173 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ca41a30a0 con 0x7f5ca4101910 2024-05-10T11:39:35.143 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.141+0000 7f5cad26d740 1 Processor -- start 2024-05-10T11:39:35.143 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.141+0000 7f5cad26d740 1 -- start start 2024-05-10T11:39:35.143 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.141+0000 7f5ca1ffb640 1 -- 172.21.15.100:0/635317173 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f5c94003f00 con 0x7f5ca4101910 2024-05-10T11:39:35.143 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.142+0000 7f5cad26d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599bc66cdb0 0x5599bc66d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:35.143 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.142+0000 7f5cad26d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599bc66d7a0 0x5599bc6722a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:35.143 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.142+0000 7f5cad26d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599bc672910 0x5599bc674da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:35.143 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.142+0000 7f5cad26d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5599bc5d15d0 con 0x5599bc66cdb0 2024-05-10T11:39:35.144 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.142+0000 7f5cad26d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5599bc58a890 con 0x5599bc672910 2024-05-10T11:39:35.144 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.142+0000 7f5cad26d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5599bc5cd590 con 0x5599bc66d7a0 2024-05-10T11:39:35.144 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.142+0000 7f5ca8aee640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599bc66cdb0 0x5599bc66d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:35.144 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.142+0000 7f5ca8aee640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599bc66cdb0 0x5599bc66d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49130/0 (socket says 172.21.15.100:49130) 2024-05-10T11:39:35.144 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.142+0000 7f5ca8aee640 1 -- 172.21.15.100:0/797847521 learned_addr learned my addr 172.21.15.100:0/797847521 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:39:35.144 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.142+0000 7f5ca3fff640 1 --2- 172.21.15.100:0/797847521 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599bc66d7a0 0x5599bc6722a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:35.144 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.142+0000 7f5c9bfff640 1 --2- 172.21.15.100:0/797847521 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599bc672910 0x5599bc674da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:35.144 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.142+0000 7f5ca8aee640 1 -- 172.21.15.100:0/797847521 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599bc66d7a0 msgr2=0x5599bc6722a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:35.144 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.142+0000 7f5ca8aee640 1 --2- 172.21.15.100:0/797847521 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599bc66d7a0 0x5599bc6722a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:35.144 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.142+0000 7f5ca8aee640 1 -- 172.21.15.100:0/797847521 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599bc672910 msgr2=0x5599bc674da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:35.144 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.142+0000 7f5ca8aee640 1 --2- 172.21.15.100:0/797847521 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599bc672910 0x5599bc674da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:35.144 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.142+0000 7f5ca8aee640 1 -- 172.21.15.100:0/797847521 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599bc2e3210 con 0x5599bc66cdb0 2024-05-10T11:39:35.144 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.143+0000 7f5ca8aee640 1 --2- 172.21.15.100:0/797847521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599bc66cdb0 0x5599bc66d1d0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c005e30 tx=0x7f5c9c009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b2d87d407ac12fe server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:39:35.144 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.143+0000 7f5ca1ffb640 1 --2- 172.21.15.100:0/635317173 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f5c900b2920 0x7f5c900b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:35.145 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.143+0000 7f5ca3fff640 1 --2- 172.21.15.100:0/635317173 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f5c900b2920 0x7f5c900b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:35.145 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.143+0000 7f5c9a7fc640 1 -- 172.21.15.100:0/797847521 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c031a80 con 0x5599bc66cdb0 2024-05-10T11:39:35.145 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.143+0000 7f5c9a7fc640 1 -- 172.21.15.100:0/797847521 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5c9c036040 con 0x5599bc66cdb0 2024-05-10T11:39:35.145 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.143+0000 7f5c9a7fc640 1 -- 172.21.15.100:0/797847521 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c0044e0 con 0x5599bc66cdb0 2024-05-10T11:39:35.145 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.143+0000 7f5ca1ffb640 1 -- 172.21.15.100:0/635317173 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f5c940e29d0 con 0x7f5ca4101910 2024-05-10T11:39:35.148 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.144+0000 7f5ca3fff640 1 --2- 172.21.15.100:0/635317173 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f5c900b2920 0x7f5c900b4de0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f5c940e04e0 tx=0x7f5c940e8550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:39:35.149 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.144+0000 7f5cad26d740 1 -- 172.21.15.100:0/797847521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599bc66cdb0 msgr2=0x5599bc66d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:35.149 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.144+0000 7f5cad26d740 1 --2- 172.21.15.100:0/797847521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599bc66cdb0 0x5599bc66d1d0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c005e30 tx=0x7f5c9c009a20 comp rx=0 tx=0).stop 2024-05-10T11:39:35.149 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.144+0000 7f5cad26d740 1 -- 172.21.15.100:0/797847521 shutdown_connections 2024-05-10T11:39:35.149 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.144+0000 7f5cad26d740 1 --2- 172.21.15.100:0/797847521 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599bc672910 0x5599bc674da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:35.149 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.144+0000 7f5cad26d740 1 --2- 172.21.15.100:0/797847521 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599bc66d7a0 0x5599bc6722a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:35.149 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.144+0000 7f5cad26d740 1 --2- 172.21.15.100:0/797847521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599bc66cdb0 0x5599bc66d1d0 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:35.149 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.144+0000 7f5cad26d740 1 -- 172.21.15.100:0/797847521 >> 172.21.15.100:0/797847521 conn(0x5599bc668420 msgr2=0x5599bc668820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:39:35.149 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.144+0000 7f5cad26d740 1 -- 172.21.15.100:0/797847521 shutdown_connections 2024-05-10T11:39:35.149 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.144+0000 7f5cad26d740 1 -- 172.21.15.100:0/797847521 wait complete. 2024-05-10T11:39:35.149 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.145+0000 7f5cad26d740 1 -- start start 2024-05-10T11:39:35.149 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.145+0000 7f5cad26d740 1 Processor -- start 2024-05-10T11:39:35.149 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.145+0000 7f5cad26d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599bc66cdb0 0x5599bc676fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:35.149 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.145+0000 7f5cad26d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599bc66d7a0 0x5599bc677500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:35.150 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.146+0000 7f5cad26d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599bc672910 0x5599bc67e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:35.150 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.146+0000 7f5cad26d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5599bc675860 con 0x5599bc66d7a0 2024-05-10T11:39:35.150 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.146+0000 7f5ca8aee640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599bc66cdb0 0x5599bc676fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:35.150 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.146+0000 7f5ca8aee640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599bc66cdb0 0x5599bc676fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:52692/0 (socket says 172.21.15.100:52692) 2024-05-10T11:39:35.150 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.146+0000 7f5ca8aee640 1 -- 172.21.15.100:0/1409829244 learned_addr learned my addr 172.21.15.100:0/1409829244 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:39:35.150 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.146+0000 7f5c9bfff640 1 --2- 172.21.15.100:0/1409829244 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599bc66d7a0 0x5599bc677500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:35.150 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.146+0000 7f5cad26d740 1 -- 172.21.15.100:0/1409829244 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5599bc58e370 con 0x5599bc66cdb0 2024-05-10T11:39:35.150 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.146+0000 7f5cad26d740 1 -- 172.21.15.100:0/1409829244 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5599bc6759e0 con 0x5599bc672910 2024-05-10T11:39:35.150 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.146+0000 7f5ca8aee640 1 --2- 172.21.15.100:0/1409829244 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599bc672910 0x5599bc67e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:35.150 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.146+0000 7f5ca8aee640 1 -- 172.21.15.100:0/1409829244 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599bc672910 msgr2=0x5599bc67e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:35.150 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.146+0000 7f5ca8aee640 1 --2- 172.21.15.100:0/1409829244 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599bc672910 0x5599bc67e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:35.150 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.146+0000 7f5ca8aee640 1 -- 172.21.15.100:0/1409829244 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599bc66d7a0 msgr2=0x5599bc677500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:35.150 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.146+0000 7f5ca8aee640 1 --2- 172.21.15.100:0/1409829244 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599bc66d7a0 0x5599bc677500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:35.151 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.146+0000 7f5ca8aee640 1 -- 172.21.15.100:0/1409829244 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599bc3af620 con 0x5599bc66cdb0 2024-05-10T11:39:35.151 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.146+0000 7f5cad26d740 10 client.0 _ll_register_callbacks cb 0x5599bc2e2a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:39:35.151 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.146+0000 7f5cad26d740 1 -- 172.21.15.100:0/1409829244 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c9c036040 con 0x5599bc66cdb0 2024-05-10T11:39:35.151 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.147+0000 7f5ca8aee640 1 --2- 172.21.15.100:0/1409829244 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599bc66cdb0 0x5599bc676fc0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c009620 tx=0x7f5c9c005bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a6a68e6f672e14c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:39:35.151 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.147+0000 7f5c837fe640 10 client.28823 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:39:35.151 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.147+0000 7f5c837fe640 1 -- 172.21.15.100:0/1409829244 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c04a040 con 0x5599bc66cdb0 2024-05-10T11:39:35.151 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.147+0000 7f5c837fe640 1 -- 172.21.15.100:0/1409829244 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f5c9c003970 con 0x5599bc66cdb0 2024-05-10T11:39:35.151 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.147+0000 7f5c837fe640 1 -- 172.21.15.100:0/1409829244 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f5c9c003b70 con 0x5599bc66cdb0 2024-05-10T11:39:35.151 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.147+0000 7f5cad26d740 10 client.28823 fetch_fsmap learned FSMap version 41 2024-05-10T11:39:35.151 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.147+0000 7f5cad26d740 1 -- 172.21.15.100:0/1409829244 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({fsmap=41}) -- 0x5599bc6758d0 con 0x5599bc66cdb0 2024-05-10T11:39:35.151 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.148+0000 7f5c837fe640 1 -- 172.21.15.100:0/1409829244 <== mon.2 v2:172.21.15.178:3300/0 4 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f5c9c050070 con 0x5599bc66cdb0 2024-05-10T11:39:35.151 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.148+0000 7f5cad26d740 10 client.28823 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:39:35.151 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.148+0000 7f5cad26d740 10 client.28823 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:39:35.151 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.148+0000 7f5cad26d740 20 client.28823 populate_metadata read hostname 'smithi100' 2024-05-10T11:39:35.151 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.148+0000 7f5cad26d740 1 --2- 172.21.15.100:0/1409829244 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5599bc682500 0x5599bc684970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:35.152 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.148+0000 7f5cad26d740 4 client.28823 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:39:35.152 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.148+0000 7f5cad26d740 1 -- 172.21.15.100:0/1409829244 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599bc5df1e0 con 0x5599bc682500 2024-05-10T11:39:35.152 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.149+0000 7f5c9bfff640 1 --2- 172.21.15.100:0/1409829244 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5599bc682500 0x5599bc684970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:35.153 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.149+0000 7f5c837fe640 10 client.28823 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:39:35.153 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.149+0000 7f5c9bfff640 1 --2- 172.21.15.100:0/1409829244 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5599bc682500 0x5599bc684970 crc :-1 s=READY pgs=401 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41cd50bd95ab995f server_cookie=c4e852064d242233 in_seq=0 out_seq=0 2024-05-10T11:39:35.153 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.150+0000 7f5c837fe640 1 -- 172.21.15.100:0/1409829244 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5599bc5df1e0 con 0x5599bc682500 2024-05-10T11:39:35.153 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.150+0000 7f5c837fe640 10 client.28823 handle_command_reply: tid=0 2024-05-10T11:39:35.153 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.150+0000 7f5cad26d740 1 client.28823 shutdown 2024-05-10T11:39:35.153 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.150+0000 7f5cad26d740 1 -- 172.21.15.100:0/1409829244 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599bc66cdb0 msgr2=0x5599bc676fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:35.153 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.150+0000 7f5cad26d740 1 --2- 172.21.15.100:0/1409829244 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599bc66cdb0 0x5599bc676fc0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c009620 tx=0x7f5c9c005bf0 comp rx=0 tx=0).stop 2024-05-10T11:39:35.154 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.150+0000 7f5cad26d740 1 -- 172.21.15.100:0/1409829244 shutdown_connections 2024-05-10T11:39:35.154 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.150+0000 7f5cad26d740 1 --2- 172.21.15.100:0/1409829244 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5599bc682500 0x5599bc684970 crc :-1 s=READY pgs=401 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:35.154 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.150+0000 7f5cad26d740 1 --2- 172.21.15.100:0/1409829244 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599bc672910 0x5599bc67e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:35.154 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.150+0000 7f5cad26d740 1 --2- 172.21.15.100:0/1409829244 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599bc66d7a0 0x5599bc677500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:35.154 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.150+0000 7f5cad26d740 1 --2- 172.21.15.100:0/1409829244 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599bc66cdb0 0x5599bc676fc0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:35.154 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.150+0000 7f5cad26d740 1 -- 172.21.15.100:0/1409829244 >> 172.21.15.100:0/1409829244 conn(0x5599bc668420 msgr2=0x5599bc667580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:39:35.154 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.150+0000 7f5c837fe640 0 client.28823 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:39:35.154 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.150+0000 7f5cad26d740 1 -- 172.21.15.100:0/1409829244 shutdown_connections 2024-05-10T11:39:35.154 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.150+0000 7f5cad26d740 1 -- 172.21.15.100:0/1409829244 wait complete. 2024-05-10T11:39:35.154 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.150+0000 7f5cad26d740 20 client.28823 trim_cache size 0 max 16384 2024-05-10T11:39:35.178 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.178+0000 7f5cad26d740 1 Processor -- start 2024-05-10T11:39:35.178 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.178+0000 7f5cad26d740 1 -- start start 2024-05-10T11:39:35.179 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.178+0000 7f5cad26d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599bc66cdb0 0x5599bc66ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:35.179 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.178+0000 7f5cad26d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599bc66d7a0 0x5599bc6765e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:35.179 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.178+0000 7f5cad26d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599bc672910 0x5599bc676b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.179+0000 7f5cad26d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca41afe10 con 0x5599bc66d7a0 2024-05-10T11:39:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.179+0000 7f5cad26d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca41ad6e0 con 0x5599bc66cdb0 2024-05-10T11:39:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.179+0000 7f5cad26d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca41ab530 con 0x5599bc672910 2024-05-10T11:39:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.179+0000 7f5ca8aee640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599bc66cdb0 0x5599bc66ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.179+0000 7f5c9bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599bc66d7a0 0x5599bc6765e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.179+0000 7f5c9bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599bc66d7a0 0x5599bc6765e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49160/0 (socket says 172.21.15.100:49160) 2024-05-10T11:39:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.179+0000 7f5c9bfff640 1 -- 172.21.15.100:0/3380408998 learned_addr learned my addr 172.21.15.100:0/3380408998 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:39:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.179+0000 7f5ca8aee640 1 --2- 172.21.15.100:0/3380408998 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599bc672910 0x5599bc676b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.179+0000 7f5c9bfff640 1 -- 172.21.15.100:0/3380408998 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599bc672910 msgr2=0x5599bc676b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.179+0000 7f5c9bfff640 1 --2- 172.21.15.100:0/3380408998 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599bc672910 0x5599bc676b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.179+0000 7f5c9bfff640 1 -- 172.21.15.100:0/3380408998 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599bc66cdb0 msgr2=0x5599bc66ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.179+0000 7f5c9bfff640 1 --2- 172.21.15.100:0/3380408998 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599bc66cdb0 0x5599bc66ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.179+0000 7f5c9bfff640 1 -- 172.21.15.100:0/3380408998 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599bc668250 con 0x5599bc66d7a0 2024-05-10T11:39:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.180+0000 7f5c9bfff640 1 --2- 172.21.15.100:0/3380408998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599bc66d7a0 0x5599bc6765e0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x5599bc6692f0 tx=0x7f5c8c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cb379ede347d0f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:39:35.181 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.180+0000 7f5c99ffb640 1 -- 172.21.15.100:0/3380408998 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c004020 con 0x5599bc66d7a0 2024-05-10T11:39:35.182 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.180+0000 7f5c99ffb640 1 -- 172.21.15.100:0/3380408998 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5c8c025070 con 0x5599bc66d7a0 2024-05-10T11:39:35.182 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.180+0000 7f5c99ffb640 1 -- 172.21.15.100:0/3380408998 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c0210e0 con 0x5599bc66d7a0 2024-05-10T11:39:35.182 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.181+0000 7f5cad26d740 1 -- 172.21.15.100:0/3380408998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599bc66d7a0 msgr2=0x5599bc6765e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:35.182 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.181+0000 7f5cad26d740 1 --2- 172.21.15.100:0/3380408998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599bc66d7a0 0x5599bc6765e0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x5599bc6692f0 tx=0x7f5c8c005f90 comp rx=0 tx=0).stop 2024-05-10T11:39:35.182 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.181+0000 7f5cad26d740 1 -- 172.21.15.100:0/3380408998 shutdown_connections 2024-05-10T11:39:35.182 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.181+0000 7f5cad26d740 1 --2- 172.21.15.100:0/3380408998 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599bc672910 0x5599bc676b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:35.182 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.181+0000 7f5cad26d740 1 --2- 172.21.15.100:0/3380408998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599bc66d7a0 0x5599bc6765e0 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:35.182 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.181+0000 7f5cad26d740 1 --2- 172.21.15.100:0/3380408998 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599bc66cdb0 0x5599bc66ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:35.182 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.181+0000 7f5cad26d740 1 -- 172.21.15.100:0/3380408998 >> 172.21.15.100:0/3380408998 conn(0x5599bc668420 msgr2=0x5599bc66b330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:39:35.182 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.181+0000 7f5cad26d740 1 -- 172.21.15.100:0/3380408998 shutdown_connections 2024-05-10T11:39:35.182 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.181+0000 7f5cad26d740 1 -- 172.21.15.100:0/3380408998 wait complete. 2024-05-10T11:39:35.182 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.182+0000 7f5cad26d740 1 -- start start 2024-05-10T11:39:35.183 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.182+0000 7f5cad26d740 1 Processor -- start 2024-05-10T11:39:35.183 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.182+0000 7f5cad26d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599bc66cdb0 0x5599bc67b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:35.183 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.182+0000 7f5cad26d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599bc672910 0x5599bc67b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:35.183 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.182+0000 7f5cad26d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599bc67c670 0x5599bc67caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:35.183 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.183+0000 7f5ca8aee640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599bc66cdb0 0x5599bc67b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:35.183 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.183+0000 7f5ca8aee640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599bc66cdb0 0x5599bc67b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:58608/0 (socket says 172.21.15.100:58608) 2024-05-10T11:39:35.183 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.183+0000 7f5ca8aee640 1 -- 172.21.15.100:0/2281396742 learned_addr learned my addr 172.21.15.100:0/2281396742 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:39:35.184 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.183+0000 7f5cad26d740 1 -- 172.21.15.100:0/2281396742 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca41abba0 con 0x5599bc67c670 2024-05-10T11:39:35.184 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.183+0000 7f5cad26d740 1 -- 172.21.15.100:0/2281396742 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca41ad010 con 0x5599bc672910 2024-05-10T11:39:35.184 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.183+0000 7f5cad26d740 1 -- 172.21.15.100:0/2281396742 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca41ac270 con 0x5599bc66cdb0 2024-05-10T11:39:35.184 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.183+0000 7f5ca8aee640 1 --2- 172.21.15.100:0/2281396742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599bc67c670 0x5599bc67caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:35.184 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.183+0000 7f5c9bfff640 1 --2- 172.21.15.100:0/2281396742 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599bc672910 0x5599bc67b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:35.184 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.183+0000 7f5cad26d740 10 client.0 _ll_register_callbacks cb 0x7f5c8c009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:39:35.185 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.183+0000 7f5ca8aee640 1 -- 172.21.15.100:0/2281396742 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599bc672910 msgr2=0x5599bc67b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:35.185 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.183+0000 7f5ca8aee640 1 --2- 172.21.15.100:0/2281396742 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599bc672910 0x5599bc67b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:35.185 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.183+0000 7f5ca8aee640 1 -- 172.21.15.100:0/2281396742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599bc67c670 msgr2=0x5599bc67caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:35.185 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.183+0000 7f5ca8aee640 1 --2- 172.21.15.100:0/2281396742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599bc67c670 0x5599bc67caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:35.185 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.183+0000 7f5ca8aee640 1 -- 172.21.15.100:0/2281396742 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599bc67d030 con 0x5599bc66cdb0 2024-05-10T11:39:35.185 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.183+0000 7f5ca8aee640 1 --2- 172.21.15.100:0/2281396742 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599bc66cdb0 0x5599bc67b170 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c000f80 tx=0x7f5c9c04d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b684557e7a1c9db server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:39:35.185 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.185+0000 7f5c837fe640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:39:35.186 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.185+0000 7f5c837fe640 1 -- 172.21.15.100:0/2281396742 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c065040 con 0x5599bc66cdb0 2024-05-10T11:39:35.186 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.185+0000 7f5cad26d740 1 -- 172.21.15.100:0/2281396742 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c8c025070 con 0x5599bc66cdb0 2024-05-10T11:39:35.186 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.185+0000 7f5c837fe640 1 -- 172.21.15.100:0/2281396742 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f5c9c0305c0 con 0x5599bc66cdb0 2024-05-10T11:39:35.186 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.185+0000 7f5c837fe640 1 -- 172.21.15.100:0/2281396742 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f5c9c0307c0 con 0x5599bc66cdb0 2024-05-10T11:39:35.186 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.185+0000 7f5cad26d740 10 client.28936 fetch_fsmap learned FSMap version 41 2024-05-10T11:39:35.187 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.185+0000 7f5cad26d740 1 -- 172.21.15.100:0/2281396742 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5599bc6ecf20 con 0x5599bc66cdb0 2024-05-10T11:39:35.187 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.186+0000 7f5c837fe640 1 -- 172.21.15.100:0/2281396742 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5c9c066040 con 0x5599bc66cdb0 2024-05-10T11:39:35.187 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.186+0000 7f5c837fe640 1 -- 172.21.15.100:0/2281396742 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f5c9c05d6a0 con 0x5599bc66cdb0 2024-05-10T11:39:35.187 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.186+0000 7f5cad26d740 10 client.28936 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:39:35.187 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.186+0000 7f5cad26d740 10 client.28936 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:39:35.187 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.186+0000 7f5cad26d740 20 client.28936 populate_metadata read hostname 'smithi100' 2024-05-10T11:39:35.187 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.186+0000 7f5cad26d740 1 --2- 172.21.15.100:0/2281396742 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5599bc6ed7f0 0x5599bc6efc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:35.187 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.186+0000 7f5cad26d740 4 client.28936 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:39:35.187 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.186+0000 7f5cad26d740 1 -- 172.21.15.100:0/2281396742 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f5c8c025070 con 0x5599bc6ed7f0 2024-05-10T11:39:35.188 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.187+0000 7f5c9bfff640 1 --2- 172.21.15.100:0/2281396742 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5599bc6ed7f0 0x5599bc6efc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:35.188 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.188+0000 7f5c837fe640 10 client.28936 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:39:35.188 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.188+0000 7f5c9bfff640 1 --2- 172.21.15.100:0/2281396742 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5599bc6ed7f0 0x5599bc6efc10 crc :-1 s=READY pgs=402 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4334ff4c0bf6fb81 server_cookie=9ab2bae2ad4e98d8 in_seq=0 out_seq=0 2024-05-10T11:39:35.368 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.368+0000 7f5c837fe640 1 -- 172.21.15.100:0/2281396742 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f5c8c025070 con 0x5599bc6ed7f0 2024-05-10T11:39:35.368 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.368+0000 7f5c837fe640 10 client.28936 handle_command_reply: tid=0 2024-05-10T11:39:35.369 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.368+0000 7f5cad26d740 1 client.28936 shutdown 2024-05-10T11:39:35.370 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.369+0000 7f5cad26d740 1 -- 172.21.15.100:0/2281396742 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599bc66cdb0 msgr2=0x5599bc67b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:35.370 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.369+0000 7f5cad26d740 1 --2- 172.21.15.100:0/2281396742 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599bc66cdb0 0x5599bc67b170 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c000f80 tx=0x7f5c9c04d000 comp rx=0 tx=0).stop 2024-05-10T11:39:35.370 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.369+0000 7f5cad26d740 1 -- 172.21.15.100:0/2281396742 shutdown_connections 2024-05-10T11:39:35.371 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.369+0000 7f5cad26d740 1 --2- 172.21.15.100:0/2281396742 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5599bc6ed7f0 0x5599bc6efc10 crc :-1 s=READY pgs=402 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:35.371 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.369+0000 7f5cad26d740 1 --2- 172.21.15.100:0/2281396742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599bc67c670 0x5599bc67caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:35.371 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.369+0000 7f5c837fe640 0 client.28936 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:39:35.371 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.369+0000 7f5cad26d740 1 --2- 172.21.15.100:0/2281396742 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599bc672910 0x5599bc67b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:35.371 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.369+0000 7f5cad26d740 1 --2- 172.21.15.100:0/2281396742 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599bc66cdb0 0x5599bc67b170 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:35.371 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.369+0000 7f5cad26d740 1 -- 172.21.15.100:0/2281396742 >> 172.21.15.100:0/2281396742 conn(0x5599bc668420 msgr2=0x5599bc666cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:39:35.371 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.369+0000 7f5cad26d740 1 -- 172.21.15.100:0/2281396742 shutdown_connections 2024-05-10T11:39:35.371 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.369+0000 7f5cad26d740 1 -- 172.21.15.100:0/2281396742 wait complete. 2024-05-10T11:39:35.371 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.369+0000 7f5cad26d740 20 client.28936 trim_cache size 0 max 16384 2024-05-10T11:39:35.371 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:39:35.372 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:39:35.372 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:39:35.372 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:39:35.373 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.373+0000 7f5caa551640 1 -- 172.21.15.100:0/635317173 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f5c900b2920 msgr2=0x7f5c900b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:35.373 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.373+0000 7f5caa551640 1 --2- 172.21.15.100:0/635317173 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f5c900b2920 0x7f5c900b4de0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f5c940e04e0 tx=0x7f5c940e8550 comp rx=0 tx=0).stop 2024-05-10T11:39:35.373 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.373+0000 7f5caa551640 1 -- 172.21.15.100:0/635317173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ca4101910 msgr2=0x7f5ca410c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:35.373 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.373+0000 7f5caa551640 1 --2- 172.21.15.100:0/635317173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ca4101910 0x7f5ca410c290 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5c9400e9c0 tx=0x7f5c9400ee80 comp rx=0 tx=0).stop 2024-05-10T11:39:35.374 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.374+0000 7f5caa551640 1 -- 172.21.15.100:0/635317173 shutdown_connections 2024-05-10T11:39:35.374 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.374+0000 7f5caa551640 1 --2- 172.21.15.100:0/635317173 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f5c900b2920 0x7f5c900b4de0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:35.374 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.374+0000 7f5caa551640 1 --2- 172.21.15.100:0/635317173 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ca41034c0 0x7f5ca410ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:35.374 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.374+0000 7f5caa551640 1 --2- 172.21.15.100:0/635317173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ca4102b00 0x7f5ca410a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:35.374 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.374+0000 7f5caa551640 1 --2- 172.21.15.100:0/635317173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ca4101910 0x7f5ca410c290 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:35.374 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.374+0000 7f5caa551640 1 -- 172.21.15.100:0/635317173 >> 172.21.15.100:0/635317173 conn(0x7f5ca40fce80 msgr2=0x7f5ca4105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:39:35.374 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.374+0000 7f5caa551640 1 -- 172.21.15.100:0/635317173 shutdown_connections 2024-05-10T11:39:35.375 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:35.374+0000 7f5caa551640 1 -- 172.21.15.100:0/635317173 wait complete. 2024-05-10T11:39:35.947 INFO:tasks.workunit.client.0.smithi100.stdout: 10 75002 8.97 MB/sec execute 329 sec latency 225.579 ms 2024-05-10T11:39:36.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:39:35 smithi178 ceph-mon[33359]: pgmap v1057: 129 pgs: 129 active+clean; 536 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 717 KiB/s rd, 4.0 MiB/s wr, 232 op/s 2024-05-10T11:39:36.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:39:35 smithi188 ceph-mon[34075]: pgmap v1057: 129 pgs: 129 active+clean; 536 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 717 KiB/s rd, 4.0 MiB/s wr, 232 op/s 2024-05-10T11:39:36.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:39:35 smithi100 ceph-mon[30296]: pgmap v1057: 129 pgs: 129 active+clean; 536 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 717 KiB/s rd, 4.0 MiB/s wr, 232 op/s 2024-05-10T11:39:36.274 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:39:36.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.328+0000 7f53e2f1b700 1 -- 172.21.15.100:0/1015955968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53dc109540 msgr2=0x7f53dc109920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:36.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.328+0000 7f53e2f1b700 1 --2- 172.21.15.100:0/1015955968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53dc109540 0x7f53dc109920 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f53d00099c0 tx=0x7f53d0009cd0 comp rx=0 tx=0).stop 2024-05-10T11:39:36.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.328+0000 7f53e2f1b700 1 -- 172.21.15.100:0/1015955968 shutdown_connections 2024-05-10T11:39:36.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.328+0000 7f53e2f1b700 1 --2- 172.21.15.100:0/1015955968 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53dc103f20 0x7f53dc110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:36.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.328+0000 7f53e2f1b700 1 --2- 172.21.15.100:0/1015955968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53dc109540 0x7f53dc109920 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:36.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.328+0000 7f53e2f1b700 1 --2- 172.21.15.100:0/1015955968 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53dc103560 0x7f53dc1039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:36.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.328+0000 7f53e2f1b700 1 -- 172.21.15.100:0/1015955968 >> 172.21.15.100:0/1015955968 conn(0x7f53dc07a660 msgr2=0x7f53dc07aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:39:36.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.328+0000 7f53e2f1b700 1 -- 172.21.15.100:0/1015955968 shutdown_connections 2024-05-10T11:39:36.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.328+0000 7f53e2f1b700 1 -- 172.21.15.100:0/1015955968 wait complete. 2024-05-10T11:39:36.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.328+0000 7f53e2f1b700 1 Processor -- start 2024-05-10T11:39:36.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.328+0000 7f53e2f1b700 1 -- start start 2024-05-10T11:39:36.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.329+0000 7f53e2f1b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53dc103560 0x7f53dc19f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:36.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.329+0000 7f53e2f1b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53dc103f20 0x7f53dc19fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:36.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.329+0000 7f53e2f1b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53dc109540 0x7f53dc199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:36.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.329+0000 7f53e2f1b700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f53dc1134d0 con 0x7f53dc109540 2024-05-10T11:39:36.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.329+0000 7f53e2f1b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f53dc113350 con 0x7f53dc103f20 2024-05-10T11:39:36.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.329+0000 7f53e2f1b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f53dc113650 con 0x7f53dc103560 2024-05-10T11:39:36.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.329+0000 7f53e14b8700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53dc109540 0x7f53dc199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:36.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.329+0000 7f53e14b8700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53dc109540 0x7f53dc199830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49190/0 (socket says 172.21.15.100:49190) 2024-05-10T11:39:36.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.329+0000 7f53e14b8700 1 -- 172.21.15.100:0/2057573578 learned_addr learned my addr 172.21.15.100:0/2057573578 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:39:36.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.329+0000 7f53e0cb7700 1 --2- 172.21.15.100:0/2057573578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53dc103560 0x7f53dc19f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:36.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.329+0000 7f53dbfff700 1 --2- 172.21.15.100:0/2057573578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53dc103f20 0x7f53dc19fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:36.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.329+0000 7f53e14b8700 1 -- 172.21.15.100:0/2057573578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53dc103560 msgr2=0x7f53dc19f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:36.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.329+0000 7f53e14b8700 1 --2- 172.21.15.100:0/2057573578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53dc103560 0x7f53dc19f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:36.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.329+0000 7f53e14b8700 1 -- 172.21.15.100:0/2057573578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53dc103f20 msgr2=0x7f53dc19fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:36.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.329+0000 7f53e14b8700 1 --2- 172.21.15.100:0/2057573578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53dc103f20 0x7f53dc19fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:36.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.329+0000 7f53e14b8700 1 -- 172.21.15.100:0/2057573578 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53dc19a010 con 0x7f53dc109540 2024-05-10T11:39:36.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.329+0000 7f53e14b8700 1 --2- 172.21.15.100:0/2057573578 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53dc109540 0x7f53dc199830 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f53d400ada0 tx=0x7f53d400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:39:36.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.329+0000 7f53d9ffb700 1 -- 172.21.15.100:0/2057573578 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d4017020 con 0x7f53dc109540 2024-05-10T11:39:36.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.329+0000 7f53d9ffb700 1 -- 172.21.15.100:0/2057573578 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f53d400cd60 con 0x7f53dc109540 2024-05-10T11:39:36.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.329+0000 7f53e2f1b700 1 -- 172.21.15.100:0/2057573578 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53dc19a300 con 0x7f53dc109540 2024-05-10T11:39:36.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.329+0000 7f53d9ffb700 1 -- 172.21.15.100:0/2057573578 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d4005280 con 0x7f53dc109540 2024-05-10T11:39:36.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.330+0000 7f53e2f1b700 1 -- 172.21.15.100:0/2057573578 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53dc1a6510 con 0x7f53dc109540 2024-05-10T11:39:36.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.332+0000 7f53d9ffb700 1 -- 172.21.15.100:0/2057573578 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f53d400c8d0 con 0x7f53dc109540 2024-05-10T11:39:36.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.332+0000 7f53d9ffb700 1 --2- 172.21.15.100:0/2057573578 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f53cc0b2880 0x7f53cc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:36.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.333+0000 7f53e0cb7700 1 --2- 172.21.15.100:0/2057573578 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f53cc0b2880 0x7f53cc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:36.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.333+0000 7f53d9ffb700 1 -- 172.21.15.100:0/2057573578 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f53d40e1d60 con 0x7f53dc109540 2024-05-10T11:39:36.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.333+0000 7f53e0cb7700 1 --2- 172.21.15.100:0/2057573578 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f53cc0b2880 0x7f53cc0b4d40 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f53d0000c00 tx=0x7f53d0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:39:36.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.333+0000 7f53ed7aeb80 1 Processor -- start 2024-05-10T11:39:36.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.333+0000 7f53ed7aeb80 1 -- start start 2024-05-10T11:39:36.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.334+0000 7f53ed7aeb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5556da56a590 0x5556da56a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:36.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.334+0000 7f53ed7aeb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556da56aeb0 0x5556da564c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:36.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.334+0000 7f53ed7aeb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5556da5651b0 0x5556da565610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:36.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.334+0000 7f53ed7aeb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5556da4b2940 con 0x5556da56a590 2024-05-10T11:39:36.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.334+0000 7f53ed7aeb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5556da417da0 con 0x5556da56aeb0 2024-05-10T11:39:36.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.334+0000 7f53ed7aeb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5556da4ae900 con 0x5556da5651b0 2024-05-10T11:39:36.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.334+0000 7f53e0cb7700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5556da56a590 0x5556da56a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:36.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.334+0000 7f53e0cb7700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5556da56a590 0x5556da56a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49198/0 (socket says 172.21.15.100:49198) 2024-05-10T11:39:36.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.334+0000 7f53e0cb7700 1 -- 172.21.15.100:0/2373098008 learned_addr learned my addr 172.21.15.100:0/2373098008 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:39:36.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.334+0000 7f53dbfff700 1 --2- 172.21.15.100:0/2373098008 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556da56aeb0 0x5556da564c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:36.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.334+0000 7f53e14b8700 1 --2- 172.21.15.100:0/2373098008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5556da5651b0 0x5556da565610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:36.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.334+0000 7f53e0cb7700 1 -- 172.21.15.100:0/2373098008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5556da5651b0 msgr2=0x5556da565610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:36.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.334+0000 7f53e0cb7700 1 --2- 172.21.15.100:0/2373098008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5556da5651b0 0x5556da565610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:36.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.334+0000 7f53e0cb7700 1 -- 172.21.15.100:0/2373098008 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556da56aeb0 msgr2=0x5556da564c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:36.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.334+0000 7f53e0cb7700 1 --2- 172.21.15.100:0/2373098008 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556da56aeb0 0x5556da564c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:36.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.334+0000 7f53e0cb7700 1 -- 172.21.15.100:0/2373098008 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556da3ff8d0 con 0x5556da56a590 2024-05-10T11:39:36.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.337+0000 7f53e0cb7700 1 --2- 172.21.15.100:0/2373098008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5556da56a590 0x5556da56a970 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f53d0009fd0 tx=0x7f53d001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e3e522d2212b2a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:39:36.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.338+0000 7f53c6ffd700 1 -- 172.21.15.100:0/2373098008 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d000bbf0 con 0x5556da56a590 2024-05-10T11:39:36.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.338+0000 7f53c6ffd700 1 -- 172.21.15.100:0/2373098008 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f53d000bd90 con 0x5556da56a590 2024-05-10T11:39:36.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.338+0000 7f53c6ffd700 1 -- 172.21.15.100:0/2373098008 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d0030970 con 0x5556da56a590 2024-05-10T11:39:36.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.339+0000 7f53ed7aeb80 1 -- 172.21.15.100:0/2373098008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5556da56a590 msgr2=0x5556da56a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:36.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.339+0000 7f53ed7aeb80 1 --2- 172.21.15.100:0/2373098008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5556da56a590 0x5556da56a970 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f53d0009fd0 tx=0x7f53d001f000 comp rx=0 tx=0).stop 2024-05-10T11:39:36.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.340+0000 7f53ed7aeb80 1 -- 172.21.15.100:0/2373098008 shutdown_connections 2024-05-10T11:39:36.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.340+0000 7f53ed7aeb80 1 --2- 172.21.15.100:0/2373098008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5556da5651b0 0x5556da565610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:36.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.340+0000 7f53ed7aeb80 1 --2- 172.21.15.100:0/2373098008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5556da56a590 0x5556da56a970 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:36.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.340+0000 7f53ed7aeb80 1 --2- 172.21.15.100:0/2373098008 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556da56aeb0 0x5556da564c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:36.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.340+0000 7f53ed7aeb80 1 -- 172.21.15.100:0/2373098008 >> 172.21.15.100:0/2373098008 conn(0x5556da48cb70 msgr2=0x5556da55b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:39:36.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.340+0000 7f53ed7aeb80 1 -- 172.21.15.100:0/2373098008 shutdown_connections 2024-05-10T11:39:36.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.340+0000 7f53ed7aeb80 1 -- 172.21.15.100:0/2373098008 wait complete. 2024-05-10T11:39:36.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.341+0000 7f53ed7aeb80 1 -- start start 2024-05-10T11:39:36.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.341+0000 7f53ed7aeb80 1 Processor -- start 2024-05-10T11:39:36.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.341+0000 7f53ed7aeb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5556da5651b0 0x5556da570c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:36.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.341+0000 7f53ed7aeb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556da56aeb0 0x5556da56e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:36.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.341+0000 7f53ed7aeb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5556da56e700 0x5556da56eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:36.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.342+0000 7f53ed7aeb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5556da5718d0 con 0x5556da5651b0 2024-05-10T11:39:36.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.342+0000 7f53ed7aeb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5556da46bed0 con 0x5556da56aeb0 2024-05-10T11:39:36.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.342+0000 7f53ed7aeb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5556da571a50 con 0x5556da56e700 2024-05-10T11:39:36.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.342+0000 7f53dbfff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556da56aeb0 0x5556da56e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:36.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.342+0000 7f53e0cb7700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5556da5651b0 0x5556da570c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:36.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.342+0000 7f53e0cb7700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5556da5651b0 0x5556da570c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49204/0 (socket says 172.21.15.100:49204) 2024-05-10T11:39:36.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.342+0000 7f53e0cb7700 1 -- 172.21.15.100:0/1801681426 learned_addr learned my addr 172.21.15.100:0/1801681426 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:39:36.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.342+0000 7f53ed7aeb80 10 client.0 _ll_register_callbacks cb 0x5556da4465b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:39:36.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.342+0000 7f53e14b8700 1 --2- 172.21.15.100:0/1801681426 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5556da56e700 0x5556da56eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:36.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.342+0000 7f53dbfff700 1 -- 172.21.15.100:0/1801681426 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5556da56e700 msgr2=0x5556da56eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:36.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.342+0000 7f53dbfff700 1 --2- 172.21.15.100:0/1801681426 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5556da56e700 0x5556da56eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:36.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.342+0000 7f53dbfff700 1 -- 172.21.15.100:0/1801681426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5556da5651b0 msgr2=0x5556da570c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:36.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.342+0000 7f53dbfff700 1 --2- 172.21.15.100:0/1801681426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5556da5651b0 0x5556da570c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:36.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.342+0000 7f53dbfff700 1 -- 172.21.15.100:0/1801681426 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556da469760 con 0x5556da56aeb0 2024-05-10T11:39:36.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.342+0000 7f53e14b8700 1 --2- 172.21.15.100:0/1801681426 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5556da56e700 0x5556da56eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:39:36.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.343+0000 7f53dbfff700 1 --2- 172.21.15.100:0/1801681426 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556da56aeb0 0x5556da56e1c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x5556da55cb40 tx=0x7f53c800be20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95eadbdf8469526d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:39:36.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.343+0000 7f53bbfff700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:39:36.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.343+0000 7f53bbfff700 1 -- 172.21.15.100:0/1801681426 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c8003cd0 con 0x5556da56aeb0 2024-05-10T11:39:36.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.343+0000 7f53bbfff700 1 -- 172.21.15.100:0/1801681426 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f53c8004830 con 0x5556da56aeb0 2024-05-10T11:39:36.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.343+0000 7f53ed7aeb80 1 -- 172.21.15.100:0/1801681426 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53d000bd90 con 0x5556da56aeb0 2024-05-10T11:39:36.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.343+0000 7f53bbfff700 1 -- 172.21.15.100:0/1801681426 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f53c8024050 con 0x5556da56aeb0 2024-05-10T11:39:36.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.343+0000 7f53ed7aeb80 10 client.28838 fetch_fsmap learned FSMap version 41 2024-05-10T11:39:36.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.343+0000 7f53ed7aeb80 1 -- 172.21.15.100:0/1801681426 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5556da580d60 con 0x5556da56aeb0 2024-05-10T11:39:36.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.343+0000 7f53bbfff700 1 -- 172.21.15.100:0/1801681426 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f53d000bd90 con 0x5556da56aeb0 2024-05-10T11:39:36.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.343+0000 7f53bbfff700 1 -- 172.21.15.100:0/1801681426 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f53c8011440 con 0x5556da56aeb0 2024-05-10T11:39:36.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.344+0000 7f53ed7aeb80 10 client.28838 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:39:36.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.344+0000 7f53ed7aeb80 10 client.28838 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-10T11:39:36.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.344+0000 7f53ed7aeb80 20 client.28838 populate_metadata read hostname 'smithi100' 2024-05-10T11:39:36.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.344+0000 7f53ed7aeb80 1 --2- 172.21.15.100:0/1801681426 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x5556da5833a0 0x5556da585790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:36.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.344+0000 7f53ed7aeb80 4 client.28838 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:39:36.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.344+0000 7f53ed7aeb80 1 -- 172.21.15.100:0/1801681426 --> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5556da4c0320 con 0x5556da5833a0 2024-05-10T11:39:36.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.344+0000 7f53e0cb7700 1 --2- 172.21.15.100:0/1801681426 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x5556da5833a0 0x5556da585790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:36.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.344+0000 7f53bbfff700 10 client.28838 ms_handle_connect on v2:172.21.15.178:6838/3760985404 2024-05-10T11:39:36.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.344+0000 7f53e0cb7700 1 --2- 172.21.15.100:0/1801681426 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x5556da5833a0 0x5556da585790 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5f46fc34dd956ea6 server_cookie=4bc9def5b57367cd in_seq=0 out_seq=0 2024-05-10T11:39:36.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.345+0000 7f53bbfff700 1 -- 172.21.15.100:0/1801681426 <== mds.1 v2:172.21.15.178:6838/3760985404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5556da4c0320 con 0x5556da5833a0 2024-05-10T11:39:36.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.345+0000 7f53bbfff700 10 client.28838 handle_command_reply: tid=0 2024-05-10T11:39:36.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.345+0000 7f53ed7aeb80 1 client.28838 shutdown 2024-05-10T11:39:36.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.345+0000 7f53ed7aeb80 1 -- 172.21.15.100:0/1801681426 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556da56aeb0 msgr2=0x5556da56e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:36.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.345+0000 7f53ed7aeb80 1 --2- 172.21.15.100:0/1801681426 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556da56aeb0 0x5556da56e1c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x5556da55cb40 tx=0x7f53c800be20 comp rx=0 tx=0).stop 2024-05-10T11:39:36.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.345+0000 7f53ed7aeb80 1 -- 172.21.15.100:0/1801681426 shutdown_connections 2024-05-10T11:39:36.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.345+0000 7f53ed7aeb80 1 --2- 172.21.15.100:0/1801681426 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x5556da5833a0 0x5556da585790 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:36.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.345+0000 7f53ed7aeb80 1 --2- 172.21.15.100:0/1801681426 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5556da56e700 0x5556da56eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:36.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.345+0000 7f53ed7aeb80 1 --2- 172.21.15.100:0/1801681426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5556da5651b0 0x5556da570c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:36.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.345+0000 7f53ed7aeb80 1 --2- 172.21.15.100:0/1801681426 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556da56aeb0 0x5556da56e1c0 secure :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0x5556da55cb40 tx=0x7f53c800be20 comp rx=0 tx=0).stop 2024-05-10T11:39:36.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.345+0000 7f53ed7aeb80 1 -- 172.21.15.100:0/1801681426 >> 172.21.15.100:0/1801681426 conn(0x5556da48cb70 msgr2=0x5556da555b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:39:36.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.345+0000 7f53bbfff700 0 client.28838 ms_handle_reset on v2:172.21.15.178:6838/3760985404 2024-05-10T11:39:36.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.346+0000 7f53ed7aeb80 1 -- 172.21.15.100:0/1801681426 shutdown_connections 2024-05-10T11:39:36.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.346+0000 7f53ed7aeb80 1 -- 172.21.15.100:0/1801681426 wait complete. 2024-05-10T11:39:36.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.346+0000 7f53ed7aeb80 20 client.28838 trim_cache size 0 max 16384 2024-05-10T11:39:36.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.363+0000 7f53ed7aeb80 1 Processor -- start 2024-05-10T11:39:36.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.364+0000 7f53ed7aeb80 1 -- start start 2024-05-10T11:39:36.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.364+0000 7f53ed7aeb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5556da5651b0 0x5556da56e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:36.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.364+0000 7f53ed7aeb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556da56e700 0x5556da555c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:36.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.364+0000 7f53ed7aeb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5556da570ec0 0x5556da58aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:36.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.364+0000 7f53ed7aeb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f53dc1af640 con 0x5556da5651b0 2024-05-10T11:39:36.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.364+0000 7f53ed7aeb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f53dc1aced0 con 0x5556da56e700 2024-05-10T11:39:36.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.364+0000 7f53ed7aeb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f53dc1aad20 con 0x5556da570ec0 2024-05-10T11:39:36.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.364+0000 7f53e14b8700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5556da570ec0 0x5556da58aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:36.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.364+0000 7f53e14b8700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5556da570ec0 0x5556da58aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:58650/0 (socket says 172.21.15.100:58650) 2024-05-10T11:39:36.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.364+0000 7f53e14b8700 1 -- 172.21.15.100:0/920273794 learned_addr learned my addr 172.21.15.100:0/920273794 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:39:36.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.364+0000 7f53e14b8700 1 -- 172.21.15.100:0/920273794 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556da56e700 msgr2=0x5556da555c40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T11:39:36.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.364+0000 7f53e14b8700 1 --2- 172.21.15.100:0/920273794 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556da56e700 0x5556da555c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:36.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.364+0000 7f53e14b8700 1 -- 172.21.15.100:0/920273794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5556da5651b0 msgr2=0x5556da56e1a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T11:39:36.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.364+0000 7f53e14b8700 1 --2- 172.21.15.100:0/920273794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5556da5651b0 0x5556da56e1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:36.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.364+0000 7f53e14b8700 1 -- 172.21.15.100:0/920273794 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556da5537d0 con 0x5556da570ec0 2024-05-10T11:39:36.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.365+0000 7f53e14b8700 1 --2- 172.21.15.100:0/920273794 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5556da570ec0 0x5556da58aff0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f53d40df550 tx=0x7f53d4004b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4d8336c08d27b9f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:39:36.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.365+0000 7f53c67fc700 1 -- 172.21.15.100:0/920273794 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d40c3070 con 0x5556da570ec0 2024-05-10T11:39:36.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.366+0000 7f53c67fc700 1 -- 172.21.15.100:0/920273794 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f53d40a90b0 con 0x5556da570ec0 2024-05-10T11:39:36.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.366+0000 7f53c67fc700 1 -- 172.21.15.100:0/920273794 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d40a9550 con 0x5556da570ec0 2024-05-10T11:39:36.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.367+0000 7f53ed7aeb80 1 -- 172.21.15.100:0/920273794 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5556da570ec0 msgr2=0x5556da58aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:36.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.367+0000 7f53ed7aeb80 1 --2- 172.21.15.100:0/920273794 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5556da570ec0 0x5556da58aff0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f53d40df550 tx=0x7f53d4004b50 comp rx=0 tx=0).stop 2024-05-10T11:39:36.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.368+0000 7f53ed7aeb80 1 -- 172.21.15.100:0/920273794 shutdown_connections 2024-05-10T11:39:36.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.368+0000 7f53ed7aeb80 1 --2- 172.21.15.100:0/920273794 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5556da570ec0 0x5556da58aff0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:36.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.368+0000 7f53ed7aeb80 1 --2- 172.21.15.100:0/920273794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5556da5651b0 0x5556da56e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:36.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.368+0000 7f53ed7aeb80 1 --2- 172.21.15.100:0/920273794 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556da56e700 0x5556da555c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:36.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.368+0000 7f53ed7aeb80 1 -- 172.21.15.100:0/920273794 >> 172.21.15.100:0/920273794 conn(0x5556da48cb70 msgr2=0x5556da56da90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:39:36.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.368+0000 7f53ed7aeb80 1 -- 172.21.15.100:0/920273794 shutdown_connections 2024-05-10T11:39:36.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.368+0000 7f53ed7aeb80 1 -- 172.21.15.100:0/920273794 wait complete. 2024-05-10T11:39:36.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.369+0000 7f53ed7aeb80 1 -- start start 2024-05-10T11:39:36.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.369+0000 7f53ed7aeb80 1 Processor -- start 2024-05-10T11:39:36.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.370+0000 7f53ed7aeb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5556da5651b0 0x5556da5e6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:36.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.370+0000 7f53ed7aeb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5556da56e700 0x5556da5e6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:36.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.370+0000 7f53ed7aeb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556da57e570 0x5556da57e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:36.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.370+0000 7f53ed7aeb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f53dc1ab390 con 0x5556da56e700 2024-05-10T11:39:36.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.370+0000 7f53ed7aeb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f53dc1ac800 con 0x5556da57e570 2024-05-10T11:39:36.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.370+0000 7f53ed7aeb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f53dc1aba60 con 0x5556da5651b0 2024-05-10T11:39:36.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.370+0000 7f53ed7aeb80 10 client.0 _ll_register_callbacks cb 0x5556da4465b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:39:36.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.370+0000 7f53e0cb7700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5556da5651b0 0x5556da5e6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:36.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.370+0000 7f53e0cb7700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5556da5651b0 0x5556da5e6320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:58658/0 (socket says 172.21.15.100:58658) 2024-05-10T11:39:36.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.370+0000 7f53e0cb7700 1 -- 172.21.15.100:0/240099405 learned_addr learned my addr 172.21.15.100:0/240099405 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:39:36.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.370+0000 7f53e14b8700 1 --2- 172.21.15.100:0/240099405 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556da57e570 0x5556da57e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:36.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.371+0000 7f53e0cb7700 1 -- 172.21.15.100:0/240099405 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556da57e570 msgr2=0x5556da57e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:36.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.371+0000 7f53e0cb7700 1 --2- 172.21.15.100:0/240099405 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556da57e570 0x5556da57e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:36.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.371+0000 7f53e0cb7700 1 -- 172.21.15.100:0/240099405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5556da56e700 msgr2=0x5556da5e6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:36.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.371+0000 7f53e0cb7700 1 --2- 172.21.15.100:0/240099405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5556da56e700 0x5556da5e6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:36.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.371+0000 7f53e0cb7700 1 -- 172.21.15.100:0/240099405 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556da57ef30 con 0x5556da5651b0 2024-05-10T11:39:36.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.371+0000 7f53e0cb7700 1 --2- 172.21.15.100:0/240099405 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5556da5651b0 0x5556da5e6320 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f53d0009990 tx=0x7f53d0009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8194cd291a019950 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:39:36.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.371+0000 7f53bbfff700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:39:36.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.371+0000 7f53bbfff700 1 -- 172.21.15.100:0/240099405 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d003f060 con 0x5556da5651b0 2024-05-10T11:39:36.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.371+0000 7f53bbfff700 1 -- 172.21.15.100:0/240099405 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f53d003a340 con 0x5556da5651b0 2024-05-10T11:39:36.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.371+0000 7f53ed7aeb80 1 -- 172.21.15.100:0/240099405 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53d40a90b0 con 0x5556da5651b0 2024-05-10T11:39:36.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.371+0000 7f53bbfff700 1 -- 172.21.15.100:0/240099405 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f53d0046050 con 0x5556da5651b0 2024-05-10T11:39:36.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.371+0000 7f53ed7aeb80 10 client.28954 fetch_fsmap learned FSMap version 41 2024-05-10T11:39:36.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.371+0000 7f53ed7aeb80 1 -- 172.21.15.100:0/240099405 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5556da580230 con 0x5556da5651b0 2024-05-10T11:39:36.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.372+0000 7f53bbfff700 1 -- 172.21.15.100:0/240099405 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f53d40a90b0 con 0x5556da5651b0 2024-05-10T11:39:36.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.373+0000 7f53bbfff700 1 -- 172.21.15.100:0/240099405 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f53d0003a60 con 0x5556da5651b0 2024-05-10T11:39:36.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.373+0000 7f53ed7aeb80 10 client.28954 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:39:36.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.373+0000 7f53ed7aeb80 10 client.28954 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-10T11:39:36.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.373+0000 7f53ed7aeb80 20 client.28954 populate_metadata read hostname 'smithi100' 2024-05-10T11:39:36.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.373+0000 7f53ed7aeb80 1 --2- 172.21.15.100:0/240099405 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x5556da5febb0 0x5556da600fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:36.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.373+0000 7f53ed7aeb80 4 client.28954 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:39:36.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.373+0000 7f53ed7aeb80 1 -- 172.21.15.100:0/240099405 --> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5556da4ba3c0 con 0x5556da5febb0 2024-05-10T11:39:36.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.373+0000 7f53dbfff700 1 --2- 172.21.15.100:0/240099405 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x5556da5febb0 0x5556da600fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:36.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.374+0000 7f53dbfff700 1 --2- 172.21.15.100:0/240099405 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x5556da5febb0 0x5556da600fa0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e05d2c8708bbac4d server_cookie=1ea65490f95b0601 in_seq=0 out_seq=0 2024-05-10T11:39:36.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.374+0000 7f53bbfff700 10 client.28954 ms_handle_connect on v2:172.21.15.178:6838/3760985404 2024-05-10T11:39:36.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.374+0000 7f53bbfff700 1 -- 172.21.15.100:0/240099405 <== mds.1 v2:172.21.15.178:6838/3760985404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5556da4ba3c0 con 0x5556da5febb0 2024-05-10T11:39:36.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.374+0000 7f53bbfff700 10 client.28954 handle_command_reply: tid=0 2024-05-10T11:39:36.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.374+0000 7f53ed7aeb80 1 client.28954 shutdown 2024-05-10T11:39:36.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.374+0000 7f53ed7aeb80 1 -- 172.21.15.100:0/240099405 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5556da5651b0 msgr2=0x5556da5e6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:36.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.374+0000 7f53ed7aeb80 1 --2- 172.21.15.100:0/240099405 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5556da5651b0 0x5556da5e6320 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f53d0009990 tx=0x7f53d0009580 comp rx=0 tx=0).stop 2024-05-10T11:39:36.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.375+0000 7f53ed7aeb80 1 -- 172.21.15.100:0/240099405 shutdown_connections 2024-05-10T11:39:36.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.375+0000 7f53ed7aeb80 1 --2- 172.21.15.100:0/240099405 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x5556da5febb0 0x5556da600fa0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:36.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.375+0000 7f53ed7aeb80 1 --2- 172.21.15.100:0/240099405 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556da57e570 0x5556da57e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:36.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.375+0000 7f53ed7aeb80 1 --2- 172.21.15.100:0/240099405 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5556da5651b0 0x5556da5e6320 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:36.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.375+0000 7f53ed7aeb80 1 --2- 172.21.15.100:0/240099405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5556da56e700 0x5556da5e6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:36.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.375+0000 7f53bbfff700 0 client.28954 ms_handle_reset on v2:172.21.15.178:6838/3760985404 2024-05-10T11:39:36.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.375+0000 7f53ed7aeb80 1 -- 172.21.15.100:0/240099405 >> 172.21.15.100:0/240099405 conn(0x5556da48cb70 msgr2=0x5556da568060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:39:36.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.377+0000 7f53ed7aeb80 1 -- 172.21.15.100:0/240099405 shutdown_connections 2024-05-10T11:39:36.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.377+0000 7f53ed7aeb80 1 -- 172.21.15.100:0/240099405 wait complete. 2024-05-10T11:39:36.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.378+0000 7f53ed7aeb80 20 client.28954 trim_cache size 0 max 16384 2024-05-10T11:39:36.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.383+0000 7f53e2f1b700 1 -- 172.21.15.100:0/2057573578 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f53cc0b2880 msgr2=0x7f53cc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:36.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.383+0000 7f53e2f1b700 1 --2- 172.21.15.100:0/2057573578 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f53cc0b2880 0x7f53cc0b4d40 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f53d0000c00 tx=0x7f53d0005fb0 comp rx=0 tx=0).stop 2024-05-10T11:39:36.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.383+0000 7f53e2f1b700 1 -- 172.21.15.100:0/2057573578 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53dc109540 msgr2=0x7f53dc199830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:36.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.383+0000 7f53e2f1b700 1 --2- 172.21.15.100:0/2057573578 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53dc109540 0x7f53dc199830 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f53d400ada0 tx=0x7f53d400c330 comp rx=0 tx=0).stop 2024-05-10T11:39:36.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.383+0000 7f53e2f1b700 1 -- 172.21.15.100:0/2057573578 shutdown_connections 2024-05-10T11:39:36.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.383+0000 7f53e2f1b700 1 --2- 172.21.15.100:0/2057573578 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f53cc0b2880 0x7f53cc0b4d40 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:36.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.383+0000 7f53e2f1b700 1 --2- 172.21.15.100:0/2057573578 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53dc109540 0x7f53dc199830 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:36.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.383+0000 7f53e2f1b700 1 --2- 172.21.15.100:0/2057573578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53dc103560 0x7f53dc19f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:36.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.383+0000 7f53e2f1b700 1 --2- 172.21.15.100:0/2057573578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53dc103f20 0x7f53dc19fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:36.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.383+0000 7f53e2f1b700 1 -- 172.21.15.100:0/2057573578 >> 172.21.15.100:0/2057573578 conn(0x7f53dc07a660 msgr2=0x7f53dc102360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:39:36.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.384+0000 7f53e2f1b700 1 -- 172.21.15.100:0/2057573578 shutdown_connections 2024-05-10T11:39:36.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:35.384+0000 7f53e2f1b700 1 -- 172.21.15.100:0/2057573578 wait complete. 2024-05-10T11:39:36.947 INFO:tasks.workunit.client.0.smithi100.stdout: 10 75172 8.97 MB/sec execute 330 sec latency 117.521 ms 2024-05-10T11:39:37.019 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-10T11:39:37.446 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:39:37.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:39:37 smithi100 ceph-mon[30296]: pgmap v1058: 129 pgs: 129 active+clean; 538 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 809 KiB/s rd, 4.3 MiB/s wr, 245 op/s 2024-05-10T11:39:37.947 INFO:tasks.workunit.client.0.smithi100.stdout: 10 75340 8.97 MB/sec execute 331 sec latency 72.709 ms 2024-05-10T11:39:38.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:39:37 smithi178 ceph-mon[33359]: pgmap v1058: 129 pgs: 129 active+clean; 538 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 809 KiB/s rd, 4.3 MiB/s wr, 245 op/s 2024-05-10T11:39:38.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:39:37 smithi188 ceph-mon[34075]: pgmap v1058: 129 pgs: 129 active+clean; 538 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 809 KiB/s rd, 4.3 MiB/s wr, 245 op/s 2024-05-10T11:39:38.947 INFO:tasks.workunit.client.0.smithi100.stdout: 10 75542 8.99 MB/sec execute 332 sec latency 57.491 ms 2024-05-10T11:39:39.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.801+0000 7f7824d0c700 1 -- 172.21.15.100:0/3073216633 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f782006bb70 msgr2=0x7f782006bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:39.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.801+0000 7f7824d0c700 1 --2- 172.21.15.100:0/3073216633 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f782006bb70 0x7f782006bff0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f7810009990 tx=0x7f7810009ca0 comp rx=0 tx=0).stop 2024-05-10T11:39:39.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.801+0000 7f7824d0c700 1 -- 172.21.15.100:0/3073216633 shutdown_connections 2024-05-10T11:39:39.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.801+0000 7f7824d0c700 1 --2- 172.21.15.100:0/3073216633 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f782006c530 0x7f7820110350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:39.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.801+0000 7f7824d0c700 1 --2- 172.21.15.100:0/3073216633 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7820109530 0x7f7820109910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:39.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.801+0000 7f7824d0c700 1 --2- 172.21.15.100:0/3073216633 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f782006bb70 0x7f782006bff0 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:39.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.801+0000 7f7824d0c700 1 -- 172.21.15.100:0/3073216633 >> 172.21.15.100:0/3073216633 conn(0x7f7820079350 msgr2=0x7f7820079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:39:39.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.801+0000 7f7824d0c700 1 -- 172.21.15.100:0/3073216633 shutdown_connections 2024-05-10T11:39:39.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.801+0000 7f7824d0c700 1 -- 172.21.15.100:0/3073216633 wait complete. 2024-05-10T11:39:39.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.802+0000 7f7824d0c700 1 Processor -- start 2024-05-10T11:39:39.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.802+0000 7f7824d0c700 1 -- start start 2024-05-10T11:39:39.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.803+0000 7f7824d0c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f782006bb70 0x7f78201a4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:39.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.803+0000 7f7824d0c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f782006c530 0x7f78201a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:39.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.803+0000 7f7824d0c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7820109530 0x7f782010f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:39.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.803+0000 7f7824d0c700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7820113560 con 0x7f782006bb70 2024-05-10T11:39:39.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.803+0000 7f7824d0c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f78201133e0 con 0x7f7820109530 2024-05-10T11:39:39.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.803+0000 7f7824d0c700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f78201136e0 con 0x7f782006c530 2024-05-10T11:39:39.923 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:39:39 smithi100 ceph-mon[30296]: pgmap v1059: 129 pgs: 129 active+clean; 530 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 3.7 MiB/s wr, 198 op/s 2024-05-10T11:39:39.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.803+0000 7f781ed9d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7820109530 0x7f782010f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:39.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.803+0000 7f781dd9b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f782006c530 0x7f78201a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:39.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.803+0000 7f781e59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f782006bb70 0x7f78201a4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:39.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.803+0000 7f781ed9d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7820109530 0x7f782010f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:58212/0 (socket says 172.21.15.100:58212) 2024-05-10T11:39:39.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.803+0000 7f781ed9d700 1 -- 172.21.15.100:0/1724789434 learned_addr learned my addr 172.21.15.100:0/1724789434 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:39:39.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.804+0000 7f781dd9b700 1 -- 172.21.15.100:0/1724789434 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7820109530 msgr2=0x7f782010f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:39.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.804+0000 7f781dd9b700 1 --2- 172.21.15.100:0/1724789434 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7820109530 0x7f782010f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:39.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.804+0000 7f781dd9b700 1 -- 172.21.15.100:0/1724789434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f782006bb70 msgr2=0x7f78201a4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:39.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.804+0000 7f781dd9b700 1 --2- 172.21.15.100:0/1724789434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f782006bb70 0x7f78201a4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:39.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.804+0000 7f781dd9b700 1 -- 172.21.15.100:0/1724789434 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f782010fce0 con 0x7f782006c530 2024-05-10T11:39:39.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.804+0000 7f781dd9b700 1 --2- 172.21.15.100:0/1724789434 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f782006c530 0x7f78201a5230 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f78100098e0 tx=0x7f78100047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:39:39.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.805+0000 7f780f7fe700 1 -- 172.21.15.100:0/1724789434 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7810026070 con 0x7f782006c530 2024-05-10T11:39:39.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.805+0000 7f780f7fe700 1 -- 172.21.15.100:0/1724789434 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7810021410 con 0x7f782006c530 2024-05-10T11:39:39.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.805+0000 7f780f7fe700 1 -- 172.21.15.100:0/1724789434 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7810026070 con 0x7f782006c530 2024-05-10T11:39:39.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.805+0000 7f7824d0c700 1 -- 172.21.15.100:0/1724789434 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f782010ff70 con 0x7f782006c530 2024-05-10T11:39:39.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.805+0000 7f7824d0c700 1 -- 172.21.15.100:0/1724789434 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78201acc20 con 0x7f782006c530 2024-05-10T11:39:39.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.809+0000 7f780f7fe700 1 -- 172.21.15.100:0/1724789434 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f7810029070 con 0x7f782006c530 2024-05-10T11:39:39.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.810+0000 7f782f59fb80 1 Processor -- start 2024-05-10T11:39:39.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.811+0000 7f782f59fb80 1 -- start start 2024-05-10T11:39:39.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.811+0000 7f780f7fe700 1 --2- 172.21.15.100:0/1724789434 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f78080b2c70 0x7f78080b5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:39.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.811+0000 7f782f59fb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575a1eb81b0 0x5575a1eb8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:39.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.811+0000 7f782f59fb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5575a1eb8ad0 0x5575a1eb2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:39.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.811+0000 7f782f59fb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5575a1eb2d50 0x5575a1eb3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:39.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.811+0000 7f782f59fb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5575a1e04940 con 0x5575a1eb81b0 2024-05-10T11:39:39.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.811+0000 7f782f59fb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5575a1d69da0 con 0x5575a1eb8ad0 2024-05-10T11:39:39.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.811+0000 7f782f59fb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5575a1e00900 con 0x5575a1eb2d50 2024-05-10T11:39:39.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.811+0000 7f781e59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575a1eb81b0 0x5575a1eb8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:39.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.811+0000 7f781e59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575a1eb81b0 0x5575a1eb8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54558/0 (socket says 172.21.15.100:54558) 2024-05-10T11:39:39.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.811+0000 7f781e59c700 1 -- 172.21.15.100:0/3376820657 learned_addr learned my addr 172.21.15.100:0/3376820657 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:39:39.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.812+0000 7f780f7fe700 1 -- 172.21.15.100:0/1724789434 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f781001e070 con 0x7f782006c530 2024-05-10T11:39:39.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.812+0000 7f781e59c700 1 -- 172.21.15.100:0/3376820657 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5575a1eb2d50 msgr2=0x5575a1eb3190 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T11:39:39.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.812+0000 7f781e59c700 1 --2- 172.21.15.100:0/3376820657 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5575a1eb2d50 0x5575a1eb3190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:39.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.812+0000 7f781e59c700 1 -- 172.21.15.100:0/3376820657 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5575a1eb8ad0 msgr2=0x5575a1eb2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:39.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.812+0000 7f781ed9d700 1 --2- 172.21.15.100:0/1724789434 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f78080b2c70 0x7f78080b5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:39.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.812+0000 7f781dd9b700 1 --2- 172.21.15.100:0/3376820657 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5575a1eb8ad0 0x5575a1eb2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:39.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.812+0000 7f781e59c700 1 --2- 172.21.15.100:0/3376820657 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5575a1eb8ad0 0x5575a1eb2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:39.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.812+0000 7f781e59c700 1 -- 172.21.15.100:0/3376820657 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575a1d518d0 con 0x5575a1eb81b0 2024-05-10T11:39:39.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.813+0000 7f781e59c700 1 --2- 172.21.15.100:0/3376820657 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575a1eb81b0 0x5575a1eb8590 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f7814009cb0 tx=0x7f7814011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f34a237b439a3fe4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:39:39.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.813+0000 7f780cff9700 1 -- 172.21.15.100:0/3376820657 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7814011e30 con 0x5575a1eb81b0 2024-05-10T11:39:39.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.813+0000 7f780cff9700 1 -- 172.21.15.100:0/3376820657 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f78140044c0 con 0x5575a1eb81b0 2024-05-10T11:39:39.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.813+0000 7f780cff9700 1 -- 172.21.15.100:0/3376820657 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7814019420 con 0x5575a1eb81b0 2024-05-10T11:39:39.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.813+0000 7f781ed9d700 1 --2- 172.21.15.100:0/1724789434 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f78080b2c70 0x7f78080b5130 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f7818009d30 tx=0x7f781800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:39:39.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.814+0000 7f782f59fb80 1 -- 172.21.15.100:0/3376820657 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575a1eb81b0 msgr2=0x5575a1eb8590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:39.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.814+0000 7f782f59fb80 1 --2- 172.21.15.100:0/3376820657 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575a1eb81b0 0x5575a1eb8590 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f7814009cb0 tx=0x7f7814011440 comp rx=0 tx=0).stop 2024-05-10T11:39:39.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.814+0000 7f782f59fb80 1 -- 172.21.15.100:0/3376820657 shutdown_connections 2024-05-10T11:39:39.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.814+0000 7f782f59fb80 1 --2- 172.21.15.100:0/3376820657 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5575a1eb2d50 0x5575a1eb3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:39.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.814+0000 7f782f59fb80 1 --2- 172.21.15.100:0/3376820657 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575a1eb81b0 0x5575a1eb8590 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:39.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.814+0000 7f782f59fb80 1 --2- 172.21.15.100:0/3376820657 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5575a1eb8ad0 0x5575a1eb2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:39.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.814+0000 7f782f59fb80 1 -- 172.21.15.100:0/3376820657 >> 172.21.15.100:0/3376820657 conn(0x5575a1ddeb70 msgr2=0x5575a1ead2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:39:39.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.814+0000 7f782f59fb80 1 -- 172.21.15.100:0/3376820657 shutdown_connections 2024-05-10T11:39:39.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.814+0000 7f782f59fb80 1 -- 172.21.15.100:0/3376820657 wait complete. 2024-05-10T11:39:39.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.815+0000 7f782f59fb80 1 -- start start 2024-05-10T11:39:39.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.815+0000 7f782f59fb80 1 Processor -- start 2024-05-10T11:39:39.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.815+0000 7f782f59fb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5575a1eb2d50 0x5575a1ebe700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:39.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.815+0000 7f782f59fb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575a1eb81b0 0x5575a1ebbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:39.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.815+0000 7f782f59fb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5575a1eb8ad0 0x5575a1ebbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:39.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.815+0000 7f782f59fb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5575a1ed0460 con 0x5575a1eb81b0 2024-05-10T11:39:39.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.815+0000 7f782f59fb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5575a1dbded0 con 0x5575a1eb8ad0 2024-05-10T11:39:39.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.815+0000 7f782f59fb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5575a1ed05e0 con 0x5575a1eb2d50 2024-05-10T11:39:39.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.815+0000 7f782f59fb80 10 client.0 _ll_register_callbacks cb 0x5575a1d985b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:39:39.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.815+0000 7f781e59c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5575a1eb2d50 0x5575a1ebe700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:39.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.815+0000 7f781e59c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5575a1eb2d50 0x5575a1ebe700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:39640/0 (socket says 172.21.15.100:39640) 2024-05-10T11:39:39.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.815+0000 7f781e59c700 1 -- 172.21.15.100:0/559447029 learned_addr learned my addr 172.21.15.100:0/559447029 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:39:39.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.815+0000 7f781dd9b700 1 --2- 172.21.15.100:0/559447029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575a1eb81b0 0x5575a1ebbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:39.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.815+0000 7f781ed9d700 1 --2- 172.21.15.100:0/559447029 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5575a1eb8ad0 0x5575a1ebbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:39.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.816+0000 7f781dd9b700 1 -- 172.21.15.100:0/559447029 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5575a1eb2d50 msgr2=0x5575a1ebe700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:39.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.816+0000 7f781dd9b700 1 --2- 172.21.15.100:0/559447029 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5575a1eb2d50 0x5575a1ebe700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:39.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.816+0000 7f781dd9b700 1 -- 172.21.15.100:0/559447029 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5575a1eb8ad0 msgr2=0x5575a1ebbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:39.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.816+0000 7f781dd9b700 1 --2- 172.21.15.100:0/559447029 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5575a1eb8ad0 0x5575a1ebbfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:39.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.816+0000 7f781dd9b700 1 -- 172.21.15.100:0/559447029 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575a1dbb760 con 0x5575a1eb81b0 2024-05-10T11:39:39.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.816+0000 7f781e59c700 1 --2- 172.21.15.100:0/559447029 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5575a1eb2d50 0x5575a1ebe700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:39:39.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.816+0000 7f781dd9b700 1 --2- 172.21.15.100:0/559447029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575a1eb81b0 0x5575a1ebbaa0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x5575a1eae040 tx=0x7f78100aa4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27462bd2c2b60dee server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:39:39.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.816+0000 7f7805ffb700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:39:39.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.816+0000 7f7805ffb700 1 -- 172.21.15.100:0/559447029 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7810026040 con 0x5575a1eb81b0 2024-05-10T11:39:39.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.816+0000 7f7805ffb700 1 -- 172.21.15.100:0/559447029 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f78100ae160 con 0x5575a1eb81b0 2024-05-10T11:39:39.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.816+0000 7f782f59fb80 1 -- 172.21.15.100:0/559447029 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78140044c0 con 0x5575a1eb81b0 2024-05-10T11:39:39.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.816+0000 7f7805ffb700 1 -- 172.21.15.100:0/559447029 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f7810027ba0 con 0x5575a1eb81b0 2024-05-10T11:39:39.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.816+0000 7f782f59fb80 10 client.19587 fetch_fsmap learned FSMap version 41 2024-05-10T11:39:39.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.816+0000 7f782f59fb80 1 -- 172.21.15.100:0/559447029 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5575a1edacd0 con 0x5575a1eb81b0 2024-05-10T11:39:39.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.817+0000 7f7805ffb700 1 -- 172.21.15.100:0/559447029 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f78100095d0 con 0x5575a1eb81b0 2024-05-10T11:39:39.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.817+0000 7f7805ffb700 1 -- 172.21.15.100:0/559447029 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f7810092050 con 0x5575a1eb81b0 2024-05-10T11:39:39.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.817+0000 7f782f59fb80 10 client.19587 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:39:39.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.817+0000 7f782f59fb80 10 client.19587 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:39:39.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.817+0000 7f782f59fb80 20 client.19587 populate_metadata read hostname 'smithi100' 2024-05-10T11:39:39.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.817+0000 7f782f59fb80 1 --2- 172.21.15.100:0/559447029 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5575a1edd2e0 0x5575a1edf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:39.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.817+0000 7f782f59fb80 4 client.19587 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:39:39.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.817+0000 7f782f59fb80 1 -- 172.21.15.100:0/559447029 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f78140044c0 con 0x5575a1edd2e0 2024-05-10T11:39:39.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.817+0000 7f781e59c700 1 --2- 172.21.15.100:0/559447029 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5575a1edd2e0 0x5575a1edf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:39.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.818+0000 7f781e59c700 1 --2- 172.21.15.100:0/559447029 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5575a1edd2e0 0x5575a1edf720 crc :-1 s=READY pgs=403 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a72d5d7e8a1c268 server_cookie=b6bcd3f8731037f7 in_seq=0 out_seq=0 2024-05-10T11:39:39.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.818+0000 7f7805ffb700 10 client.19587 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:39:39.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.819+0000 7f7805ffb700 1 -- 172.21.15.100:0/559447029 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f78140044c0 con 0x5575a1edd2e0 2024-05-10T11:39:39.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.819+0000 7f7805ffb700 10 client.19587 handle_command_reply: tid=0 2024-05-10T11:39:39.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.819+0000 7f782f59fb80 1 client.19587 shutdown 2024-05-10T11:39:39.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.819+0000 7f782f59fb80 1 -- 172.21.15.100:0/559447029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575a1eb81b0 msgr2=0x5575a1ebbaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:39.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.819+0000 7f782f59fb80 1 --2- 172.21.15.100:0/559447029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575a1eb81b0 0x5575a1ebbaa0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x5575a1eae040 tx=0x7f78100aa4f0 comp rx=0 tx=0).stop 2024-05-10T11:39:39.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.819+0000 7f782f59fb80 1 -- 172.21.15.100:0/559447029 shutdown_connections 2024-05-10T11:39:39.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.819+0000 7f782f59fb80 1 --2- 172.21.15.100:0/559447029 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5575a1edd2e0 0x5575a1edf720 crc :-1 s=READY pgs=403 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:39.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.819+0000 7f782f59fb80 1 --2- 172.21.15.100:0/559447029 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5575a1eb8ad0 0x5575a1ebbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:39.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.819+0000 7f782f59fb80 1 --2- 172.21.15.100:0/559447029 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5575a1eb2d50 0x5575a1ebe700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:39.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.819+0000 7f782f59fb80 1 --2- 172.21.15.100:0/559447029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575a1eb81b0 0x5575a1ebbaa0 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:39.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.819+0000 7f782f59fb80 1 -- 172.21.15.100:0/559447029 >> 172.21.15.100:0/559447029 conn(0x5575a1ddeb70 msgr2=0x5575a1ea7b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:39:39.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.819+0000 7f7805ffb700 0 client.19587 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:39:39.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.819+0000 7f782f59fb80 1 -- 172.21.15.100:0/559447029 shutdown_connections 2024-05-10T11:39:39.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.819+0000 7f782f59fb80 1 -- 172.21.15.100:0/559447029 wait complete. 2024-05-10T11:39:39.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.820+0000 7f782f59fb80 20 client.19587 trim_cache size 0 max 16384 2024-05-10T11:39:39.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.846+0000 7f782f59fb80 1 Processor -- start 2024-05-10T11:39:39.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.846+0000 7f782f59fb80 1 -- start start 2024-05-10T11:39:39.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.846+0000 7f782f59fb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5575a1ddeb70 0x5575a1eb9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:39.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.846+0000 7f782f59fb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5575a1eb2d50 0x5575a1ed94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:39.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.846+0000 7f782f59fb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575a1eb81b0 0x5575a1ea7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:39.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.846+0000 7f782f59fb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f78201af630 con 0x5575a1eb81b0 2024-05-10T11:39:39.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.846+0000 7f782f59fb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f78201acec0 con 0x5575a1eb2d50 2024-05-10T11:39:39.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.846+0000 7f782f59fb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f78200735e0 con 0x5575a1ddeb70 2024-05-10T11:39:39.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.846+0000 7f781e59c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5575a1ddeb70 0x5575a1eb9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:39.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.846+0000 7f781e59c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5575a1ddeb70 0x5575a1eb9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:39652/0 (socket says 172.21.15.100:39652) 2024-05-10T11:39:39.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.846+0000 7f781e59c700 1 -- 172.21.15.100:0/2561522006 learned_addr learned my addr 172.21.15.100:0/2561522006 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:39:39.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.846+0000 7f781dd9b700 1 --2- 172.21.15.100:0/2561522006 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5575a1eb2d50 0x5575a1ed94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:39.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.847+0000 7f781dd9b700 1 -- 172.21.15.100:0/2561522006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5575a1ddeb70 msgr2=0x5575a1eb9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:39.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.847+0000 7f781dd9b700 1 --2- 172.21.15.100:0/2561522006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5575a1ddeb70 0x5575a1eb9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:39.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.847+0000 7f781dd9b700 1 -- 172.21.15.100:0/2561522006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575a1eb81b0 msgr2=0x5575a1ea7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:39.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.847+0000 7f781dd9b700 1 --2- 172.21.15.100:0/2561522006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575a1eb81b0 0x5575a1ea7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:39.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.847+0000 7f781dd9b700 1 -- 172.21.15.100:0/2561522006 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575a1ed9fc0 con 0x5575a1eb2d50 2024-05-10T11:39:39.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.847+0000 7f781dd9b700 1 --2- 172.21.15.100:0/2561522006 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5575a1eb2d50 0x5575a1ed94f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f78100aad10 tx=0x7f781000b2e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f17e0c8ecee0982f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:39:39.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.847+0000 7f7807fff700 1 -- 172.21.15.100:0/2561522006 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7810026070 con 0x5575a1eb2d50 2024-05-10T11:39:39.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.847+0000 7f7807fff700 1 -- 172.21.15.100:0/2561522006 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f781000a250 con 0x5575a1eb2d50 2024-05-10T11:39:39.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.847+0000 7f7807fff700 1 -- 172.21.15.100:0/2561522006 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78100ae160 con 0x5575a1eb2d50 2024-05-10T11:39:39.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.848+0000 7f782f59fb80 1 -- 172.21.15.100:0/2561522006 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5575a1eb2d50 msgr2=0x5575a1ed94f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:39.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.848+0000 7f782f59fb80 1 --2- 172.21.15.100:0/2561522006 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5575a1eb2d50 0x5575a1ed94f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f78100aad10 tx=0x7f781000b2e0 comp rx=0 tx=0).stop 2024-05-10T11:39:39.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.849+0000 7f782f59fb80 1 -- 172.21.15.100:0/2561522006 shutdown_connections 2024-05-10T11:39:39.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.849+0000 7f782f59fb80 1 --2- 172.21.15.100:0/2561522006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575a1eb81b0 0x5575a1ea7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:39.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.849+0000 7f782f59fb80 1 --2- 172.21.15.100:0/2561522006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5575a1ddeb70 0x5575a1eb9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:39.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.849+0000 7f782f59fb80 1 --2- 172.21.15.100:0/2561522006 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5575a1eb2d50 0x5575a1ed94f0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:39.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.849+0000 7f782f59fb80 1 -- 172.21.15.100:0/2561522006 >> 172.21.15.100:0/2561522006 conn(0x5575a1eb3440 msgr2=0x5575a1eba390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:39:39.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.849+0000 7f782f59fb80 1 -- 172.21.15.100:0/2561522006 shutdown_connections 2024-05-10T11:39:39.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.849+0000 7f782f59fb80 1 -- 172.21.15.100:0/2561522006 wait complete. 2024-05-10T11:39:39.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.850+0000 7f782f59fb80 1 -- start start 2024-05-10T11:39:39.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.850+0000 7f782f59fb80 1 Processor -- start 2024-05-10T11:39:39.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.850+0000 7f782f59fb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5575a1ddeb70 0x5575a1ed5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:39.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.850+0000 7f782f59fb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5575a1eb2d50 0x5575a1ee4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:39.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.850+0000 7f782f59fb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575a1eb81b0 0x5575a1ee4e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:39.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.850+0000 7f782f59fb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7820073c50 con 0x5575a1eb81b0 2024-05-10T11:39:39.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.850+0000 7f782f59fb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f78200750c0 con 0x5575a1eb2d50 2024-05-10T11:39:39.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.850+0000 7f782f59fb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7820074320 con 0x5575a1ddeb70 2024-05-10T11:39:39.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.850+0000 7f782f59fb80 10 client.0 _ll_register_callbacks cb 0x5575a1d985b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:39:39.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.850+0000 7f781ed9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575a1eb81b0 0x5575a1ee4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:39.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.851+0000 7f781ed9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575a1eb81b0 0x5575a1ee4e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54592/0 (socket says 172.21.15.100:54592) 2024-05-10T11:39:39.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.851+0000 7f781ed9d700 1 -- 172.21.15.100:0/256980577 learned_addr learned my addr 172.21.15.100:0/256980577 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:39:39.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.851+0000 7f781dd9b700 1 --2- 172.21.15.100:0/256980577 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5575a1eb2d50 0x5575a1ee4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:39.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.851+0000 7f781e59c700 1 --2- 172.21.15.100:0/256980577 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5575a1ddeb70 0x5575a1ed5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:39.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.851+0000 7f781ed9d700 1 -- 172.21.15.100:0/256980577 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5575a1ddeb70 msgr2=0x5575a1ed5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:39.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.851+0000 7f781ed9d700 1 --2- 172.21.15.100:0/256980577 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5575a1ddeb70 0x5575a1ed5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:39.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.851+0000 7f781ed9d700 1 -- 172.21.15.100:0/256980577 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5575a1eb2d50 msgr2=0x5575a1ee4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:39.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.851+0000 7f781ed9d700 1 --2- 172.21.15.100:0/256980577 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5575a1eb2d50 0x5575a1ee4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:39.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.851+0000 7f781ed9d700 1 -- 172.21.15.100:0/256980577 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575a1f46be0 con 0x5575a1eb81b0 2024-05-10T11:39:39.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.851+0000 7f781ed9d700 1 --2- 172.21.15.100:0/256980577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575a1eb81b0 0x5575a1ee4e00 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f7818008a90 tx=0x7f7818009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10e2357422be2280 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:39:39.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.851+0000 7f7805ffb700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:39:39.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.852+0000 7f7805ffb700 1 -- 172.21.15.100:0/256980577 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7818026030 con 0x5575a1eb81b0 2024-05-10T11:39:39.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.852+0000 7f7805ffb700 1 -- 172.21.15.100:0/256980577 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f7818004bb0 con 0x5575a1eb81b0 2024-05-10T11:39:39.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.852+0000 7f782f59fb80 1 -- 172.21.15.100:0/256980577 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5575a1e12320 con 0x5575a1eb81b0 2024-05-10T11:39:39.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.852+0000 7f7805ffb700 1 -- 172.21.15.100:0/256980577 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f7818030050 con 0x5575a1eb81b0 2024-05-10T11:39:39.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.852+0000 7f782f59fb80 10 client.19590 fetch_fsmap learned FSMap version 41 2024-05-10T11:39:39.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.852+0000 7f782f59fb80 1 -- 172.21.15.100:0/256980577 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5575a1f47e20 con 0x5575a1eb81b0 2024-05-10T11:39:39.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.853+0000 7f7805ffb700 1 -- 172.21.15.100:0/256980577 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7818008e50 con 0x5575a1eb81b0 2024-05-10T11:39:39.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.853+0000 7f7805ffb700 1 -- 172.21.15.100:0/256980577 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f78180229e0 con 0x5575a1eb81b0 2024-05-10T11:39:39.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.853+0000 7f782f59fb80 10 client.19590 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:39:39.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.853+0000 7f782f59fb80 10 client.19590 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:39:39.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.853+0000 7f782f59fb80 20 client.19590 populate_metadata read hostname 'smithi100' 2024-05-10T11:39:39.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.853+0000 7f782f59fb80 1 --2- 172.21.15.100:0/256980577 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5575a1eb8ad0 0x5575a1f4a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:39.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.853+0000 7f782f59fb80 4 client.19590 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:39:39.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.853+0000 7f782f59fb80 1 -- 172.21.15.100:0/256980577 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5575a1e12320 con 0x5575a1eb8ad0 2024-05-10T11:39:39.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.853+0000 7f781e59c700 1 --2- 172.21.15.100:0/256980577 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5575a1eb8ad0 0x5575a1f4a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:39.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.853+0000 7f7805ffb700 10 client.19590 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:39:39.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.853+0000 7f781e59c700 1 --2- 172.21.15.100:0/256980577 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5575a1eb8ad0 0x5575a1f4a5a0 crc :-1 s=READY pgs=404 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=baa7ec74e28d0579 server_cookie=8810aebd5e3f90be in_seq=0 out_seq=0 2024-05-10T11:39:39.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.854+0000 7f7805ffb700 1 -- 172.21.15.100:0/256980577 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5575a1e12320 con 0x5575a1eb8ad0 2024-05-10T11:39:39.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.854+0000 7f7805ffb700 10 client.19590 handle_command_reply: tid=0 2024-05-10T11:39:39.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.854+0000 7f782f59fb80 1 client.19590 shutdown 2024-05-10T11:39:39.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.854+0000 7f782f59fb80 1 -- 172.21.15.100:0/256980577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575a1eb81b0 msgr2=0x5575a1ee4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:39.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.854+0000 7f782f59fb80 1 --2- 172.21.15.100:0/256980577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575a1eb81b0 0x5575a1ee4e00 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f7818008a90 tx=0x7f7818009920 comp rx=0 tx=0).stop 2024-05-10T11:39:39.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.854+0000 7f782f59fb80 1 -- 172.21.15.100:0/256980577 shutdown_connections 2024-05-10T11:39:39.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.854+0000 7f782f59fb80 1 --2- 172.21.15.100:0/256980577 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5575a1eb8ad0 0x5575a1f4a5a0 crc :-1 s=READY pgs=404 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:39.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.854+0000 7f782f59fb80 1 --2- 172.21.15.100:0/256980577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575a1eb81b0 0x5575a1ee4e00 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:39.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.854+0000 7f7805ffb700 0 client.19590 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:39:39.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.854+0000 7f782f59fb80 1 --2- 172.21.15.100:0/256980577 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5575a1ddeb70 0x5575a1ed5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:39.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.854+0000 7f782f59fb80 1 --2- 172.21.15.100:0/256980577 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5575a1eb2d50 0x5575a1ee4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:39.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.854+0000 7f782f59fb80 1 -- 172.21.15.100:0/256980577 >> 172.21.15.100:0/256980577 conn(0x5575a1eb3440 msgr2=0x5575a1eb95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:39:39.937 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:39:39.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.855+0000 7f782f59fb80 1 -- 172.21.15.100:0/256980577 shutdown_connections 2024-05-10T11:39:39.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.855+0000 7f782f59fb80 1 -- 172.21.15.100:0/256980577 wait complete. 2024-05-10T11:39:39.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.855+0000 7f782f59fb80 20 client.19590 trim_cache size 0 max 16384 2024-05-10T11:39:39.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.856+0000 7f7824d0c700 1 -- 172.21.15.100:0/1724789434 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f78080b2c70 msgr2=0x7f78080b5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:39.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.856+0000 7f7824d0c700 1 --2- 172.21.15.100:0/1724789434 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f78080b2c70 0x7f78080b5130 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f7818009d30 tx=0x7f781800b040 comp rx=0 tx=0).stop 2024-05-10T11:39:39.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.856+0000 7f7824d0c700 1 -- 172.21.15.100:0/1724789434 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f782006c530 msgr2=0x7f78201a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:39.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.856+0000 7f7824d0c700 1 --2- 172.21.15.100:0/1724789434 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f782006c530 0x7f78201a5230 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f78100098e0 tx=0x7f78100047b0 comp rx=0 tx=0).stop 2024-05-10T11:39:39.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.856+0000 7f7824d0c700 1 -- 172.21.15.100:0/1724789434 shutdown_connections 2024-05-10T11:39:39.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.856+0000 7f7824d0c700 1 --2- 172.21.15.100:0/1724789434 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f78080b2c70 0x7f78080b5130 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:39.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.856+0000 7f7824d0c700 1 --2- 172.21.15.100:0/1724789434 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7820109530 0x7f782010f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:39.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.856+0000 7f7824d0c700 1 --2- 172.21.15.100:0/1724789434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f782006bb70 0x7f78201a4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:39.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.856+0000 7f7824d0c700 1 --2- 172.21.15.100:0/1724789434 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f782006c530 0x7f78201a5230 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:39.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.856+0000 7f7824d0c700 1 -- 172.21.15.100:0/1724789434 >> 172.21.15.100:0/1724789434 conn(0x7f7820079350 msgr2=0x7f7820101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:39:39.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.857+0000 7f7824d0c700 1 -- 172.21.15.100:0/1724789434 shutdown_connections 2024-05-10T11:39:39.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:39.857+0000 7f7824d0c700 1 -- 172.21.15.100:0/1724789434 wait complete. 2024-05-10T11:39:39.947 INFO:tasks.workunit.client.0.smithi100.stdout: 10 75670 8.97 MB/sec execute 333 sec latency 91.865 ms 2024-05-10T11:39:40.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:39:39 smithi178 ceph-mon[33359]: pgmap v1059: 129 pgs: 129 active+clean; 530 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 3.7 MiB/s wr, 198 op/s 2024-05-10T11:39:40.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:39:39 smithi188 ceph-mon[34075]: pgmap v1059: 129 pgs: 129 active+clean; 530 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 3.7 MiB/s wr, 198 op/s 2024-05-10T11:39:40.947 INFO:tasks.workunit.client.0.smithi100.stdout: 10 75815 8.95 MB/sec execute 334 sec latency 97.980 ms 2024-05-10T11:39:41.947 INFO:tasks.workunit.client.0.smithi100.stdout: 10 75982 8.94 MB/sec execute 335 sec latency 66.541 ms 2024-05-10T11:39:42.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:39:41 smithi178 ceph-mon[33359]: pgmap v1060: 129 pgs: 129 active+clean; 530 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 4.4 MiB/s wr, 272 op/s 2024-05-10T11:39:42.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:39:41 smithi188 ceph-mon[34075]: pgmap v1060: 129 pgs: 129 active+clean; 530 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 4.4 MiB/s wr, 272 op/s 2024-05-10T11:39:42.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:39:41 smithi100 ceph-mon[30296]: pgmap v1060: 129 pgs: 129 active+clean; 530 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 4.4 MiB/s wr, 272 op/s 2024-05-10T11:39:42.055 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T11:39:42.055 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T11:39:42.423 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:39:42.947 INFO:tasks.workunit.client.0.smithi100.stdout: 10 76145 8.93 MB/sec execute 336 sec latency 75.009 ms 2024-05-10T11:39:43.948 INFO:tasks.workunit.client.0.smithi100.stdout: 10 76294 8.93 MB/sec execute 337 sec latency 102.254 ms 2024-05-10T11:39:44.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:39:43 smithi178 ceph-mon[33359]: pgmap v1061: 129 pgs: 129 active+clean; 529 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 3.6 MiB/s wr, 233 op/s 2024-05-10T11:39:44.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:39:43 smithi188 ceph-mon[34075]: pgmap v1061: 129 pgs: 129 active+clean; 529 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 3.6 MiB/s wr, 233 op/s 2024-05-10T11:39:44.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:39:43 smithi100 ceph-mon[30296]: pgmap v1061: 129 pgs: 129 active+clean; 529 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 3.6 MiB/s wr, 233 op/s 2024-05-10T11:39:44.947 INFO:tasks.workunit.client.0.smithi100.stdout: 10 76439 8.93 MB/sec execute 338 sec latency 116.636 ms 2024-05-10T11:39:45.947 INFO:tasks.workunit.client.0.smithi100.stdout: 10 76621 8.94 MB/sec execute 339 sec latency 157.105 ms 2024-05-10T11:39:46.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:39:45 smithi178 ceph-mon[33359]: pgmap v1062: 129 pgs: 129 active+clean; 532 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 3.6 MiB/s wr, 247 op/s 2024-05-10T11:39:46.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:39:45 smithi188 ceph-mon[34075]: pgmap v1062: 129 pgs: 129 active+clean; 532 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 3.6 MiB/s wr, 247 op/s 2024-05-10T11:39:46.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:39:45 smithi100 ceph-mon[30296]: pgmap v1062: 129 pgs: 129 active+clean; 532 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 3.6 MiB/s wr, 247 op/s 2024-05-10T11:39:46.947 INFO:tasks.workunit.client.0.smithi100.stdout: 10 76828 8.98 MB/sec execute 340 sec latency 97.272 ms 2024-05-10T11:39:47.934 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:39:47 smithi100 ceph-mon[30296]: pgmap v1063: 129 pgs: 129 active+clean; 533 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 814 KiB/s rd, 3.9 MiB/s wr, 300 op/s 2024-05-10T11:39:47.947 INFO:tasks.workunit.client.0.smithi100.stdout: 10 76991 8.97 MB/sec execute 341 sec latency 66.937 ms 2024-05-10T11:39:48.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:39:47 smithi178 ceph-mon[33359]: pgmap v1063: 129 pgs: 129 active+clean; 533 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 814 KiB/s rd, 3.9 MiB/s wr, 300 op/s 2024-05-10T11:39:48.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:39:47 smithi188 ceph-mon[34075]: pgmap v1063: 129 pgs: 129 active+clean; 533 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 814 KiB/s rd, 3.9 MiB/s wr, 300 op/s 2024-05-10T11:39:48.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.254+0000 7f7319868700 1 -- 172.21.15.100:0/2802840280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7314109500 msgr2=0x7f73141098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:48.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.254+0000 7f7319868700 1 --2- 172.21.15.100:0/2802840280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7314109500 0x7f73141098e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f73040098f0 tx=0x7f7304009c00 comp rx=0 tx=0).stop 2024-05-10T11:39:48.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.256+0000 7f7319868700 1 -- 172.21.15.100:0/2802840280 shutdown_connections 2024-05-10T11:39:48.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.256+0000 7f7319868700 1 --2- 172.21.15.100:0/2802840280 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f731406c3d0 0x7f7314110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:48.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.256+0000 7f7319868700 1 --2- 172.21.15.100:0/2802840280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7314109500 0x7f73141098e0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:48.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.256+0000 7f7319868700 1 --2- 172.21.15.100:0/2802840280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f731406ba10 0x7f731406be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:48.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.256+0000 7f7319868700 1 -- 172.21.15.100:0/2802840280 >> 172.21.15.100:0/2802840280 conn(0x7f7314079310 msgr2=0x7f7314079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:39:48.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.256+0000 7f7319868700 1 -- 172.21.15.100:0/2802840280 shutdown_connections 2024-05-10T11:39:48.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.256+0000 7f7319868700 1 -- 172.21.15.100:0/2802840280 wait complete. 2024-05-10T11:39:48.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.257+0000 7f7319868700 1 Processor -- start 2024-05-10T11:39:48.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.258+0000 7f7319868700 1 -- start start 2024-05-10T11:39:48.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.258+0000 7f7319868700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f731406ba10 0x7f731410f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:48.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.258+0000 7f7319868700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f731406c3d0 0x7f731410fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:48.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.258+0000 7f7319868700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7314109500 0x7f73141a4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:48.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.258+0000 7f7319868700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7314113510 con 0x7f731406ba10 2024-05-10T11:39:48.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.258+0000 7f7319868700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7314113390 con 0x7f7314109500 2024-05-10T11:39:48.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.258+0000 7f7319868700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7314113690 con 0x7f731406c3d0 2024-05-10T11:39:48.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.258+0000 7f73127fc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f731406c3d0 0x7f731410fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:48.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.258+0000 7f73127fc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f731406c3d0 0x7f731410fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:55422/0 (socket says 172.21.15.100:55422) 2024-05-10T11:39:48.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.258+0000 7f73127fc700 1 -- 172.21.15.100:0/637770888 learned_addr learned my addr 172.21.15.100:0/637770888 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:39:48.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.258+0000 7f73137fe700 1 --2- 172.21.15.100:0/637770888 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7314109500 0x7f73141a4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:48.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.258+0000 7f7312ffd700 1 --2- 172.21.15.100:0/637770888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f731406ba10 0x7f731410f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:48.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.259+0000 7f73127fc700 1 -- 172.21.15.100:0/637770888 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7314109500 msgr2=0x7f73141a4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:48.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.259+0000 7f73127fc700 1 --2- 172.21.15.100:0/637770888 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7314109500 0x7f73141a4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:48.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.259+0000 7f73127fc700 1 -- 172.21.15.100:0/637770888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f731406ba10 msgr2=0x7f731410f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:48.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.259+0000 7f73127fc700 1 --2- 172.21.15.100:0/637770888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f731406ba10 0x7f731410f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:48.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.259+0000 7f73127fc700 1 -- 172.21.15.100:0/637770888 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73141a5160 con 0x7f731406c3d0 2024-05-10T11:39:48.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.259+0000 7f73127fc700 1 --2- 172.21.15.100:0/637770888 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f731406c3d0 0x7f731410fd80 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f730800e9c0 tx=0x7f730800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:39:48.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.259+0000 7f7318866700 1 -- 172.21.15.100:0/637770888 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7308004a30 con 0x7f731406c3d0 2024-05-10T11:39:48.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.259+0000 7f7318866700 1 -- 172.21.15.100:0/637770888 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f730800cc50 con 0x7f731406c3d0 2024-05-10T11:39:48.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.259+0000 7f7318866700 1 -- 172.21.15.100:0/637770888 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7308005420 con 0x7f731406c3d0 2024-05-10T11:39:48.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.259+0000 7f7319868700 1 -- 172.21.15.100:0/637770888 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73141a5450 con 0x7f731406c3d0 2024-05-10T11:39:48.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.260+0000 7f7319868700 1 -- 172.21.15.100:0/637770888 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73141acd00 con 0x7f731406c3d0 2024-05-10T11:39:48.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.263+0000 7f7318866700 1 -- 172.21.15.100:0/637770888 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f73080055c0 con 0x7f731406c3d0 2024-05-10T11:39:48.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.264+0000 7f7318866700 1 --2- 172.21.15.100:0/637770888 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f72fc0b2900 0x7f72fc0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:48.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.265+0000 7f7318866700 1 -- 172.21.15.100:0/637770888 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f73080e25d0 con 0x7f731406c3d0 2024-05-10T11:39:48.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.265+0000 7f73240fbb80 1 Processor -- start 2024-05-10T11:39:48.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.265+0000 7f7312ffd700 1 --2- 172.21.15.100:0/637770888 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f72fc0b2900 0x7f72fc0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:48.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.265+0000 7f73240fbb80 1 -- start start 2024-05-10T11:39:48.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.265+0000 7f73240fbb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56445e810550 0x56445e810930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:48.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.265+0000 7f73240fbb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56445e810e70 0x56445e814770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:48.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.266+0000 7f73240fbb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56445e814d00 0x56445e815180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:48.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.266+0000 7f73240fbb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56445e7628e0 con 0x56445e814d00 2024-05-10T11:39:48.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.266+0000 7f73240fbb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56445e701840 con 0x56445e810e70 2024-05-10T11:39:48.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.266+0000 7f73240fbb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56445e75e8a0 con 0x56445e810550 2024-05-10T11:39:48.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.266+0000 7f73137fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56445e814d00 0x56445e815180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:48.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.266+0000 7f73137fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56445e814d00 0x56445e815180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39920/0 (socket says 172.21.15.100:39920) 2024-05-10T11:39:48.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.266+0000 7f73137fe700 1 -- 172.21.15.100:0/3043873926 learned_addr learned my addr 172.21.15.100:0/3043873926 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:39:48.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.266+0000 7f73127fc700 1 --2- 172.21.15.100:0/3043873926 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56445e810e70 0x56445e814770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:48.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.266+0000 7f73137fe700 1 -- 172.21.15.100:0/3043873926 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56445e810550 msgr2=0x56445e810930 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T11:39:48.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.266+0000 7f7312ffd700 1 --2- 172.21.15.100:0/3043873926 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56445e810550 0x56445e810930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:48.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.266+0000 7f73137fe700 1 --2- 172.21.15.100:0/3043873926 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56445e810550 0x56445e810930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:48.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.267+0000 7f73137fe700 1 -- 172.21.15.100:0/3043873926 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56445e810e70 msgr2=0x56445e814770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:48.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.267+0000 7f73137fe700 1 --2- 172.21.15.100:0/3043873926 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56445e810e70 0x56445e814770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:48.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.267+0000 7f73137fe700 1 -- 172.21.15.100:0/3043873926 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56445e6af950 con 0x56445e814d00 2024-05-10T11:39:48.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.267+0000 7f7312ffd700 1 --2- 172.21.15.100:0/3043873926 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56445e810550 0x56445e810930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:39:48.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.267+0000 7f73127fc700 1 --2- 172.21.15.100:0/3043873926 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56445e810e70 0x56445e814770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:39:48.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.267+0000 7f7312ffd700 1 --2- 172.21.15.100:0/637770888 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f72fc0b2900 0x7f72fc0b4dc0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f73040098c0 tx=0x7f7304019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:39:48.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.267+0000 7f73137fe700 1 --2- 172.21.15.100:0/3043873926 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56445e814d00 0x56445e815180 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f730c011680 tx=0x7f730c009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c12a8e169d05eca2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:39:48.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.267+0000 7f72f9ffb700 1 -- 172.21.15.100:0/3043873926 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f730c019030 con 0x56445e814d00 2024-05-10T11:39:48.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.267+0000 7f72f9ffb700 1 -- 172.21.15.100:0/3043873926 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f730c0092e0 con 0x56445e814d00 2024-05-10T11:39:48.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.267+0000 7f72f9ffb700 1 -- 172.21.15.100:0/3043873926 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f730c007500 con 0x56445e814d00 2024-05-10T11:39:48.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.268+0000 7f73240fbb80 1 -- 172.21.15.100:0/3043873926 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56445e814d00 msgr2=0x56445e815180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:48.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.268+0000 7f73240fbb80 1 --2- 172.21.15.100:0/3043873926 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56445e814d00 0x56445e815180 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f730c011680 tx=0x7f730c009920 comp rx=0 tx=0).stop 2024-05-10T11:39:48.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.268+0000 7f73240fbb80 1 -- 172.21.15.100:0/3043873926 shutdown_connections 2024-05-10T11:39:48.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.268+0000 7f73240fbb80 1 --2- 172.21.15.100:0/3043873926 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56445e814d00 0x56445e815180 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:48.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.268+0000 7f73240fbb80 1 --2- 172.21.15.100:0/3043873926 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56445e810550 0x56445e810930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:48.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.268+0000 7f73240fbb80 1 --2- 172.21.15.100:0/3043873926 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56445e810e70 0x56445e814770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:48.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.268+0000 7f73240fbb80 1 -- 172.21.15.100:0/3043873926 >> 172.21.15.100:0/3043873926 conn(0x56445e73c180 msgr2=0x56445e80acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:39:48.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.269+0000 7f73240fbb80 1 -- 172.21.15.100:0/3043873926 shutdown_connections 2024-05-10T11:39:48.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.269+0000 7f73240fbb80 1 -- 172.21.15.100:0/3043873926 wait complete. 2024-05-10T11:39:48.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.269+0000 7f73240fbb80 1 -- start start 2024-05-10T11:39:48.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.269+0000 7f73240fbb80 1 Processor -- start 2024-05-10T11:39:48.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.270+0000 7f73240fbb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56445e810550 0x56445e81c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:48.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.270+0000 7f73240fbb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56445e810e70 0x56445e819560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:48.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.270+0000 7f73240fbb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56445e814d00 0x56445e819aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:48.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.270+0000 7f7312ffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56445e810550 0x56445e81c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:48.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.270+0000 7f73240fbb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56445e81cf10 con 0x56445e814d00 2024-05-10T11:39:48.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.270+0000 7f7312ffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56445e810550 0x56445e81c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:55440/0 (socket says 172.21.15.100:55440) 2024-05-10T11:39:48.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.270+0000 7f73240fbb80 1 -- 172.21.15.100:0/2239092720 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56445e6c8cb0 con 0x56445e810e70 2024-05-10T11:39:48.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.270+0000 7f7312ffd700 1 -- 172.21.15.100:0/2239092720 learned_addr learned my addr 172.21.15.100:0/2239092720 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:39:48.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.270+0000 7f73240fbb80 1 -- 172.21.15.100:0/2239092720 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56445e81d090 con 0x56445e810550 2024-05-10T11:39:48.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.270+0000 7f73240fbb80 10 client.0 _ll_register_callbacks cb 0x56445e6fa9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:39:48.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.271+0000 7f73127fc700 1 --2- 172.21.15.100:0/2239092720 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56445e810e70 0x56445e819560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:48.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.271+0000 7f73137fe700 1 --2- 172.21.15.100:0/2239092720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56445e814d00 0x56445e819aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:48.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.271+0000 7f7312ffd700 1 -- 172.21.15.100:0/2239092720 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56445e810e70 msgr2=0x56445e819560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:48.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.271+0000 7f7312ffd700 1 --2- 172.21.15.100:0/2239092720 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56445e810e70 0x56445e819560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:48.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.271+0000 7f7312ffd700 1 -- 172.21.15.100:0/2239092720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56445e814d00 msgr2=0x56445e819aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:48.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.271+0000 7f7312ffd700 1 --2- 172.21.15.100:0/2239092720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56445e814d00 0x56445e819aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:48.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.271+0000 7f7312ffd700 1 -- 172.21.15.100:0/2239092720 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56445e6fe710 con 0x56445e810550 2024-05-10T11:39:48.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.271+0000 7f7312ffd700 1 --2- 172.21.15.100:0/2239092720 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56445e810550 0x56445e81c1c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f73040098f0 tx=0x7f7304009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a4c9b6fedafed10 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:39:48.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.271+0000 7f72f2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:39:48.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.272+0000 7f72f2ffd700 1 -- 172.21.15.100:0/2239092720 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7304032060 con 0x56445e810550 2024-05-10T11:39:48.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.272+0000 7f73240fbb80 1 -- 172.21.15.100:0/2239092720 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f730c0092e0 con 0x56445e810550 2024-05-10T11:39:48.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.272+0000 7f72f2ffd700 1 -- 172.21.15.100:0/2239092720 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f7304003680 con 0x56445e810550 2024-05-10T11:39:48.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.272+0000 7f72f2ffd700 1 -- 172.21.15.100:0/2239092720 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f7304029040 con 0x56445e810550 2024-05-10T11:39:48.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.272+0000 7f73240fbb80 10 client.28975 fetch_fsmap learned FSMap version 41 2024-05-10T11:39:48.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.272+0000 7f73240fbb80 1 -- 172.21.15.100:0/2239092720 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x56445e827540 con 0x56445e810550 2024-05-10T11:39:48.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.272+0000 7f72f2ffd700 1 -- 172.21.15.100:0/2239092720 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f730c0092e0 con 0x56445e810550 2024-05-10T11:39:48.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.273+0000 7f72f2ffd700 1 -- 172.21.15.100:0/2239092720 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f730400b700 con 0x56445e810550 2024-05-10T11:39:48.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.273+0000 7f73240fbb80 10 client.28975 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:39:48.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.273+0000 7f73240fbb80 10 client.28975 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:39:48.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.273+0000 7f73240fbb80 20 client.28975 populate_metadata read hostname 'smithi100' 2024-05-10T11:39:48.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.273+0000 7f73240fbb80 1 --2- 172.21.15.100:0/2239092720 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56445e829ed0 0x56445e82c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:48.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.273+0000 7f73240fbb80 4 client.28975 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:39:48.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.273+0000 7f73240fbb80 1 -- 172.21.15.100:0/2239092720 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56445e7dc6d0 con 0x56445e829ed0 2024-05-10T11:39:48.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.274+0000 7f73127fc700 1 --2- 172.21.15.100:0/2239092720 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56445e829ed0 0x56445e82c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:48.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.274+0000 7f73127fc700 1 --2- 172.21.15.100:0/2239092720 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56445e829ed0 0x56445e82c330 crc :-1 s=READY pgs=405 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b6875d0e07c5f2e server_cookie=4df8b9d1d0fd8348 in_seq=0 out_seq=0 2024-05-10T11:39:48.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.274+0000 7f72f2ffd700 10 client.28975 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:39:48.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.275+0000 7f72f2ffd700 1 -- 172.21.15.100:0/2239092720 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56445e7dc6d0 con 0x56445e829ed0 2024-05-10T11:39:48.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.275+0000 7f72f2ffd700 10 client.28975 handle_command_reply: tid=0 2024-05-10T11:39:48.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.275+0000 7f73240fbb80 1 client.28975 shutdown 2024-05-10T11:39:48.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.278+0000 7f73240fbb80 1 -- 172.21.15.100:0/2239092720 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56445e810550 msgr2=0x56445e81c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:48.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.278+0000 7f73240fbb80 1 --2- 172.21.15.100:0/2239092720 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56445e810550 0x56445e81c1c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f73040098f0 tx=0x7f7304009260 comp rx=0 tx=0).stop 2024-05-10T11:39:48.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.279+0000 7f73240fbb80 1 -- 172.21.15.100:0/2239092720 shutdown_connections 2024-05-10T11:39:48.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.279+0000 7f73240fbb80 1 --2- 172.21.15.100:0/2239092720 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56445e829ed0 0x56445e82c330 crc :-1 s=READY pgs=405 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:48.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.279+0000 7f73240fbb80 1 --2- 172.21.15.100:0/2239092720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56445e814d00 0x56445e819aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:48.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.279+0000 7f72f2ffd700 0 client.28975 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:39:48.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.279+0000 7f73240fbb80 1 --2- 172.21.15.100:0/2239092720 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56445e810550 0x56445e81c1c0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:48.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.279+0000 7f73240fbb80 1 --2- 172.21.15.100:0/2239092720 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56445e810e70 0x56445e819560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:48.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.279+0000 7f73240fbb80 1 -- 172.21.15.100:0/2239092720 >> 172.21.15.100:0/2239092720 conn(0x56445e73c180 msgr2=0x56445e80ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:39:48.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.280+0000 7f73240fbb80 1 -- 172.21.15.100:0/2239092720 shutdown_connections 2024-05-10T11:39:48.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.280+0000 7f73240fbb80 1 -- 172.21.15.100:0/2239092720 wait complete. 2024-05-10T11:39:48.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.280+0000 7f73240fbb80 20 client.28975 trim_cache size 0 max 16384 2024-05-10T11:39:48.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.307+0000 7f73240fbb80 1 Processor -- start 2024-05-10T11:39:48.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.308+0000 7f73240fbb80 1 -- start start 2024-05-10T11:39:48.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.308+0000 7f73240fbb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56445e810550 0x56445e819ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:48.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.308+0000 7f73240fbb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56445e810e70 0x56445e825fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:48.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.308+0000 7f73240fbb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56445e814d00 0x56445e831c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:48.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.308+0000 7f73240fbb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f73141b8180 con 0x56445e810550 2024-05-10T11:39:48.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.308+0000 7f73240fbb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f73141b5a10 con 0x56445e810e70 2024-05-10T11:39:48.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.308+0000 7f73240fbb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f73141b3860 con 0x56445e814d00 2024-05-10T11:39:48.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.308+0000 7f73137fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56445e814d00 0x56445e831c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:48.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.308+0000 7f7312ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56445e810550 0x56445e819ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:48.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.308+0000 7f7312ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56445e810550 0x56445e819ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39938/0 (socket says 172.21.15.100:39938) 2024-05-10T11:39:48.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.308+0000 7f7312ffd700 1 -- 172.21.15.100:0/3883712958 learned_addr learned my addr 172.21.15.100:0/3883712958 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:39:48.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.308+0000 7f73137fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56445e814d00 0x56445e831c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:55448/0 (socket says 172.21.15.100:55448) 2024-05-10T11:39:48.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.308+0000 7f73127fc700 1 --2- 172.21.15.100:0/3883712958 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56445e810e70 0x56445e825fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:48.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.308+0000 7f7312ffd700 1 -- 172.21.15.100:0/3883712958 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56445e814d00 msgr2=0x56445e831c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:48.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.308+0000 7f7312ffd700 1 --2- 172.21.15.100:0/3883712958 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56445e814d00 0x56445e831c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:48.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.308+0000 7f7312ffd700 1 -- 172.21.15.100:0/3883712958 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56445e810e70 msgr2=0x56445e825fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:48.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.308+0000 7f7312ffd700 1 --2- 172.21.15.100:0/3883712958 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56445e810e70 0x56445e825fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:48.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.308+0000 7f7312ffd700 1 -- 172.21.15.100:0/3883712958 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56445e81a460 con 0x56445e810550 2024-05-10T11:39:48.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.309+0000 7f7312ffd700 1 --2- 172.21.15.100:0/3883712958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56445e810550 0x56445e819ad0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f7304005b80 tx=0x7f7304000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88455be4e2151685 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:39:48.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.309+0000 7f72f97fa700 1 -- 172.21.15.100:0/3883712958 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7304032070 con 0x56445e810550 2024-05-10T11:39:48.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.309+0000 7f72f97fa700 1 -- 172.21.15.100:0/3883712958 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7304042070 con 0x56445e810550 2024-05-10T11:39:48.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.309+0000 7f72f97fa700 1 -- 172.21.15.100:0/3883712958 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f730402dac0 con 0x56445e810550 2024-05-10T11:39:48.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.310+0000 7f73240fbb80 1 -- 172.21.15.100:0/3883712958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56445e810550 msgr2=0x56445e819ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:48.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.310+0000 7f73240fbb80 1 --2- 172.21.15.100:0/3883712958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56445e810550 0x56445e819ad0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f7304005b80 tx=0x7f7304000bc0 comp rx=0 tx=0).stop 2024-05-10T11:39:48.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.310+0000 7f73240fbb80 1 -- 172.21.15.100:0/3883712958 shutdown_connections 2024-05-10T11:39:48.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.310+0000 7f73240fbb80 1 --2- 172.21.15.100:0/3883712958 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56445e814d00 0x56445e831c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:48.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.310+0000 7f73240fbb80 1 --2- 172.21.15.100:0/3883712958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56445e810550 0x56445e819ad0 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:48.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.310+0000 7f73240fbb80 1 --2- 172.21.15.100:0/3883712958 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56445e810e70 0x56445e825fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:48.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.310+0000 7f73240fbb80 1 -- 172.21.15.100:0/3883712958 >> 172.21.15.100:0/3883712958 conn(0x56445e73c180 msgr2=0x56445e80fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:39:48.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.310+0000 7f73240fbb80 1 -- 172.21.15.100:0/3883712958 shutdown_connections 2024-05-10T11:39:48.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.311+0000 7f73240fbb80 1 -- 172.21.15.100:0/3883712958 wait complete. 2024-05-10T11:39:48.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.311+0000 7f73240fbb80 1 -- start start 2024-05-10T11:39:48.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.311+0000 7f73240fbb80 1 Processor -- start 2024-05-10T11:39:48.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.312+0000 7f73240fbb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56445e810550 0x56445e8224d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:48.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.312+0000 7f73240fbb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56445e810e70 0x56445e822a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:48.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.312+0000 7f7312ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56445e810550 0x56445e8224d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:48.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.312+0000 7f73240fbb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56445e814d00 0x56445e81ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:48.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.312+0000 7f7312ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56445e810550 0x56445e8224d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:45252/0 (socket says 172.21.15.100:45252) 2024-05-10T11:39:48.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.312+0000 7f73240fbb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f73141b3ed0 con 0x56445e810e70 2024-05-10T11:39:48.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.312+0000 7f73240fbb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f73141b5340 con 0x56445e810550 2024-05-10T11:39:48.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.312+0000 7f73127fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56445e810e70 0x56445e822a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:48.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.312+0000 7f73240fbb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f73141b45a0 con 0x56445e814d00 2024-05-10T11:39:48.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.312+0000 7f7312ffd700 1 -- 172.21.15.100:0/977540640 learned_addr learned my addr 172.21.15.100:0/977540640 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:39:48.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.312+0000 7f73240fbb80 10 client.0 _ll_register_callbacks cb 0x56445e6fa9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:39:48.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.312+0000 7f73137fe700 1 --2- 172.21.15.100:0/977540640 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56445e814d00 0x56445e81ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:48.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.313+0000 7f7312ffd700 1 -- 172.21.15.100:0/977540640 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56445e814d00 msgr2=0x56445e81ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:48.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.313+0000 7f7312ffd700 1 --2- 172.21.15.100:0/977540640 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56445e814d00 0x56445e81ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:48.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.313+0000 7f7312ffd700 1 -- 172.21.15.100:0/977540640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56445e810e70 msgr2=0x56445e822a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:48.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.313+0000 7f7312ffd700 1 --2- 172.21.15.100:0/977540640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56445e810e70 0x56445e822a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:48.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.313+0000 7f7312ffd700 1 -- 172.21.15.100:0/977540640 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56445e81f270 con 0x56445e810550 2024-05-10T11:39:48.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.313+0000 7f73127fc700 1 --2- 172.21.15.100:0/977540640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56445e810e70 0x56445e822a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:39:48.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.313+0000 7f7312ffd700 1 --2- 172.21.15.100:0/977540640 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56445e810550 0x56445e8224d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7304009810 tx=0x7f730400b7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9bd42e309f350ec server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:39:48.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.313+0000 7f72f2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:39:48.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.313+0000 7f72f2ffd700 1 -- 172.21.15.100:0/977540640 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7304032070 con 0x56445e810550 2024-05-10T11:39:48.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.313+0000 7f73240fbb80 1 -- 172.21.15.100:0/977540640 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7304042070 con 0x56445e810550 2024-05-10T11:39:48.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.313+0000 7f72f2ffd700 1 -- 172.21.15.100:0/977540640 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f730402db70 con 0x56445e810550 2024-05-10T11:39:48.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.314+0000 7f72f2ffd700 1 -- 172.21.15.100:0/977540640 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f7304054050 con 0x56445e810550 2024-05-10T11:39:48.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.314+0000 7f73240fbb80 10 client.28871 fetch_fsmap learned FSMap version 41 2024-05-10T11:39:48.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.314+0000 7f73240fbb80 1 -- 172.21.15.100:0/977540640 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x56445e8954c0 con 0x56445e810550 2024-05-10T11:39:48.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.314+0000 7f72f2ffd700 1 -- 172.21.15.100:0/977540640 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7304042070 con 0x56445e810550 2024-05-10T11:39:48.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.314+0000 7f72f2ffd700 1 -- 172.21.15.100:0/977540640 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f7304003bf0 con 0x56445e810550 2024-05-10T11:39:48.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.315+0000 7f73240fbb80 10 client.28871 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:39:48.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.315+0000 7f73240fbb80 10 client.28871 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:39:48.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.315+0000 7f73240fbb80 20 client.28871 populate_metadata read hostname 'smithi100' 2024-05-10T11:39:48.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.315+0000 7f73240fbb80 1 --2- 172.21.15.100:0/977540640 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56445e829ed0 0x56445e897b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:48.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.315+0000 7f73240fbb80 4 client.28871 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T11:39:48.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.315+0000 7f73240fbb80 1 -- 172.21.15.100:0/977540640 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56445e7702c0 con 0x56445e829ed0 2024-05-10T11:39:48.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.315+0000 7f73127fc700 1 --2- 172.21.15.100:0/977540640 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56445e829ed0 0x56445e897b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:48.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.315+0000 7f72f2ffd700 10 client.28871 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:39:48.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.315+0000 7f73127fc700 1 --2- 172.21.15.100:0/977540640 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56445e829ed0 0x56445e897b30 crc :-1 s=READY pgs=406 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50d9c42d4e57572f server_cookie=862662f2ebbd4864 in_seq=0 out_seq=0 2024-05-10T11:39:48.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.317+0000 7f72f2ffd700 1 -- 172.21.15.100:0/977540640 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56445e7702c0 con 0x56445e829ed0 2024-05-10T11:39:48.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.317+0000 7f72f2ffd700 10 client.28871 handle_command_reply: tid=0 2024-05-10T11:39:48.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.317+0000 7f73240fbb80 1 client.28871 shutdown 2024-05-10T11:39:48.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.317+0000 7f73240fbb80 1 -- 172.21.15.100:0/977540640 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56445e810550 msgr2=0x56445e8224d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:48.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.317+0000 7f73240fbb80 1 --2- 172.21.15.100:0/977540640 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56445e810550 0x56445e8224d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7304009810 tx=0x7f730400b7f0 comp rx=0 tx=0).stop 2024-05-10T11:39:48.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.317+0000 7f73240fbb80 1 -- 172.21.15.100:0/977540640 shutdown_connections 2024-05-10T11:39:48.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.317+0000 7f73240fbb80 1 --2- 172.21.15.100:0/977540640 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56445e829ed0 0x56445e897b30 crc :-1 s=READY pgs=406 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:48.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.317+0000 7f73240fbb80 1 --2- 172.21.15.100:0/977540640 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56445e814d00 0x56445e81ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:48.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.317+0000 7f73240fbb80 1 --2- 172.21.15.100:0/977540640 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56445e810550 0x56445e8224d0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:48.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.317+0000 7f73240fbb80 1 --2- 172.21.15.100:0/977540640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56445e810e70 0x56445e822a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:48.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.317+0000 7f73240fbb80 1 -- 172.21.15.100:0/977540640 >> 172.21.15.100:0/977540640 conn(0x56445e73c180 msgr2=0x56445e8135a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:39:48.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.317+0000 7f72f2ffd700 0 client.28871 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:39:48.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.318+0000 7f73240fbb80 1 -- 172.21.15.100:0/977540640 shutdown_connections 2024-05-10T11:39:48.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.318+0000 7f73240fbb80 1 -- 172.21.15.100:0/977540640 wait complete. 2024-05-10T11:39:48.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.318+0000 7f73240fbb80 20 client.28871 trim_cache size 0 max 16384 2024-05-10T11:39:48.319 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:39:48.319 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-05-10T11:39:48.319 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "73adf975-1f4a-40e6-9a7c-f44ef7f24e5a", 2024-05-10T11:39:48.319 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-05-10T11:39:48.319 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:39:48.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.319+0000 7f7319868700 1 -- 172.21.15.100:0/637770888 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f72fc0b2900 msgr2=0x7f72fc0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:48.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.319+0000 7f7319868700 1 --2- 172.21.15.100:0/637770888 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f72fc0b2900 0x7f72fc0b4dc0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f73040098c0 tx=0x7f7304019040 comp rx=0 tx=0).stop 2024-05-10T11:39:48.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.319+0000 7f7319868700 1 -- 172.21.15.100:0/637770888 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f731406c3d0 msgr2=0x7f731410fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:48.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.319+0000 7f7319868700 1 --2- 172.21.15.100:0/637770888 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f731406c3d0 0x7f731410fd80 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f730800e9c0 tx=0x7f730800ecd0 comp rx=0 tx=0).stop 2024-05-10T11:39:48.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.319+0000 7f7319868700 1 -- 172.21.15.100:0/637770888 shutdown_connections 2024-05-10T11:39:48.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.319+0000 7f7319868700 1 --2- 172.21.15.100:0/637770888 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f72fc0b2900 0x7f72fc0b4dc0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:48.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.319+0000 7f7319868700 1 --2- 172.21.15.100:0/637770888 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7314109500 0x7f73141a4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:48.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.319+0000 7f7319868700 1 --2- 172.21.15.100:0/637770888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f731406ba10 0x7f731410f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:48.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.319+0000 7f7319868700 1 --2- 172.21.15.100:0/637770888 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f731406c3d0 0x7f731410fd80 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:48.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.319+0000 7f7319868700 1 -- 172.21.15.100:0/637770888 >> 172.21.15.100:0/637770888 conn(0x7f7314079310 msgr2=0x7f7314101100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:39:48.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.319+0000 7f7319868700 1 -- 172.21.15.100:0/637770888 shutdown_connections 2024-05-10T11:39:48.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:48.319+0000 7f7319868700 1 -- 172.21.15.100:0/637770888 wait complete. 2024-05-10T11:39:48.526 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.521+0000 7f6129b4e640 1 -- 172.21.15.100:0/945901364 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6124102f00 msgr2=0x7f6124103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:48.526 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.521+0000 7f6129b4e640 1 --2- 172.21.15.100:0/945901364 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6124102f00 0x7f6124103320 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f6114009930 tx=0x7f611402a420 comp rx=0 tx=0).stop 2024-05-10T11:39:48.526 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.521+0000 7f6129b4e640 1 -- 172.21.15.100:0/945901364 shutdown_connections 2024-05-10T11:39:48.526 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.521+0000 7f6129b4e640 1 --2- 172.21.15.100:0/945901364 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6124102560 0x7f6124109e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:48.526 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.521+0000 7f6129b4e640 1 --2- 172.21.15.100:0/945901364 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6124101ba0 0x7f6124102020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:48.527 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.521+0000 7f6129b4e640 1 --2- 172.21.15.100:0/945901364 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6124102f00 0x7f6124103320 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:48.527 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.521+0000 7f6129b4e640 1 -- 172.21.15.100:0/945901364 >> 172.21.15.100:0/945901364 conn(0x7f61240fc610 msgr2=0x7f61240fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:39:48.527 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.522+0000 7f6129b4e640 1 -- 172.21.15.100:0/945901364 shutdown_connections 2024-05-10T11:39:48.527 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.522+0000 7f6129b4e640 1 -- 172.21.15.100:0/945901364 wait complete. 2024-05-10T11:39:48.527 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.522+0000 7f6129b4e640 1 Processor -- start 2024-05-10T11:39:48.527 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.523+0000 7f6129b4e640 1 -- start start 2024-05-10T11:39:48.528 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.523+0000 7f6129b4e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6124101ba0 0x7f6124101610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:48.528 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.523+0000 7f6129b4e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6124102560 0x7f612419d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:48.528 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.523+0000 7f6129b4e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6124102f00 0x7f612419fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:48.528 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.523+0000 7f6129b4e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f612410d2d0 con 0x7f6124101ba0 2024-05-10T11:39:48.528 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.523+0000 7f6129b4e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f612410d150 con 0x7f6124102f00 2024-05-10T11:39:48.528 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.523+0000 7f6129b4e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f612410d450 con 0x7f6124102560 2024-05-10T11:39:48.529 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.523+0000 7f61237fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6124101ba0 0x7f6124101610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:48.529 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.523+0000 7f6122ffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6124102560 0x7f612419d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:48.529 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.523+0000 7f61237fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6124101ba0 0x7f6124101610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39974/0 (socket says 172.21.15.100:39974) 2024-05-10T11:39:48.529 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.523+0000 7f6122ffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6124102560 0x7f612419d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:55494/0 (socket says 172.21.15.100:55494) 2024-05-10T11:39:48.529 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.523+0000 7f61237fe640 1 -- 172.21.15.100:0/2478917860 learned_addr learned my addr 172.21.15.100:0/2478917860 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:39:48.529 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.523+0000 7f6123fff640 1 --2- 172.21.15.100:0/2478917860 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6124102f00 0x7f612419fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:48.530 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.523+0000 7f6123fff640 1 -- 172.21.15.100:0/2478917860 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6124102560 msgr2=0x7f612419d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:48.530 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.523+0000 7f6123fff640 1 --2- 172.21.15.100:0/2478917860 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6124102560 0x7f612419d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:48.530 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.523+0000 7f6123fff640 1 -- 172.21.15.100:0/2478917860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6124101ba0 msgr2=0x7f6124101610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:48.530 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.523+0000 7f6123fff640 1 --2- 172.21.15.100:0/2478917860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6124101ba0 0x7f6124101610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:48.530 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.523+0000 7f6123fff640 1 -- 172.21.15.100:0/2478917860 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61241a0750 con 0x7f6124102f00 2024-05-10T11:39:48.530 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.524+0000 7f6123fff640 1 --2- 172.21.15.100:0/2478917860 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6124102f00 0x7f612419fe90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f611c009fd0 tx=0x7f611c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:39:48.531 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.524+0000 7f6120ff9640 1 -- 172.21.15.100:0/2478917860 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611c00ce20 con 0x7f6124102f00 2024-05-10T11:39:48.531 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.524+0000 7f6129b4e640 1 -- 172.21.15.100:0/2478917860 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61241a0a40 con 0x7f6124102f00 2024-05-10T11:39:48.531 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.524+0000 7f6120ff9640 1 -- 172.21.15.100:0/2478917860 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f611c005020 con 0x7f6124102f00 2024-05-10T11:39:48.531 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.524+0000 7f6120ff9640 1 -- 172.21.15.100:0/2478917860 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611c010690 con 0x7f6124102f00 2024-05-10T11:39:48.531 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.524+0000 7f6129b4e640 1 -- 172.21.15.100:0/2478917860 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61241a0f90 con 0x7f6124102f00 2024-05-10T11:39:48.544 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.527+0000 7f6120ff9640 1 -- 172.21.15.100:0/2478917860 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f611c004a40 con 0x7f6124102f00 2024-05-10T11:39:48.544 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.528+0000 7f612c860740 1 Processor -- start 2024-05-10T11:39:48.545 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.528+0000 7f612c860740 1 -- start start 2024-05-10T11:39:48.545 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.528+0000 7f6120ff9640 1 --2- 172.21.15.100:0/2478917860 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f610c0b2920 0x7f610c0b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:48.545 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.528+0000 7f6122ffd640 1 --2- 172.21.15.100:0/2478917860 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f610c0b2920 0x7f610c0b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:48.545 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.528+0000 7f612c860740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c44a5a6db0 0x55c44a5a71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:48.546 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.528+0000 7f612c860740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c44a5a77a0 0x55c44a5ac2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:48.546 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.528+0000 7f6120ff9640 1 -- 172.21.15.100:0/2478917860 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f611c0e2ba0 con 0x7f6124102f00 2024-05-10T11:39:48.546 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.528+0000 7f612c860740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c44a5ac910 0x55c44a5aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:48.546 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.528+0000 7f612c860740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55c44a50b5d0 con 0x55c44a5a6db0 2024-05-10T11:39:48.546 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.528+0000 7f612c860740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c44a4c4890 con 0x55c44a5a77a0 2024-05-10T11:39:48.547 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.528+0000 7f612c860740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c44a507590 con 0x55c44a5ac910 2024-05-10T11:39:48.547 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.529+0000 7f6122ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c44a5a77a0 0x55c44a5ac2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:48.547 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.529+0000 7f6122ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c44a5a77a0 0x55c44a5ac2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:45276/0 (socket says 172.21.15.100:45276) 2024-05-10T11:39:48.547 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.529+0000 7f6122ffd640 1 -- 172.21.15.100:0/641819584 learned_addr learned my addr 172.21.15.100:0/641819584 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:39:48.547 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.529+0000 7f61237fe640 1 --2- 172.21.15.100:0/641819584 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c44a5a6db0 0x55c44a5a71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:48.547 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.529+0000 7f6123fff640 1 --2- 172.21.15.100:0/641819584 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c44a5ac910 0x55c44a5aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:48.548 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.529+0000 7f6122ffd640 1 -- 172.21.15.100:0/641819584 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c44a5ac910 msgr2=0x55c44a5aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:48.548 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.529+0000 7f6122ffd640 1 --2- 172.21.15.100:0/641819584 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c44a5ac910 0x55c44a5aeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:48.548 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.529+0000 7f6122ffd640 1 -- 172.21.15.100:0/641819584 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c44a5a6db0 msgr2=0x55c44a5a71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:48.548 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.529+0000 7f6122ffd640 1 --2- 172.21.15.100:0/641819584 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c44a5a6db0 0x55c44a5a71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:48.549 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.529+0000 7f6122ffd640 1 -- 172.21.15.100:0/641819584 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c44a21d210 con 0x55c44a5a77a0 2024-05-10T11:39:48.549 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.529+0000 7f6122ffd640 1 --2- 172.21.15.100:0/2478917860 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f610c0b2920 0x7f610c0b4de0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f6118013990 tx=0x7f61180098a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:39:48.549 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.529+0000 7f6122ffd640 1 --2- 172.21.15.100:0/641819584 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c44a5a77a0 0x55c44a5ac2a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f61180097a0 tx=0x7f611801d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a130b1fd032a9caa server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:39:48.549 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.529+0000 7f6109ffb640 1 -- 172.21.15.100:0/641819584 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6118013170 con 0x55c44a5a77a0 2024-05-10T11:39:48.549 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.530+0000 7f6109ffb640 1 -- 172.21.15.100:0/641819584 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f61180054b0 con 0x55c44a5a77a0 2024-05-10T11:39:48.549 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.530+0000 7f6109ffb640 1 -- 172.21.15.100:0/641819584 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611800eca0 con 0x55c44a5a77a0 2024-05-10T11:39:48.550 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.531+0000 7f612c860740 1 -- 172.21.15.100:0/641819584 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c44a5a77a0 msgr2=0x55c44a5ac2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:48.550 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.531+0000 7f612c860740 1 --2- 172.21.15.100:0/641819584 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c44a5a77a0 0x55c44a5ac2a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f61180097a0 tx=0x7f611801d000 comp rx=0 tx=0).stop 2024-05-10T11:39:48.550 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.531+0000 7f612c860740 1 -- 172.21.15.100:0/641819584 shutdown_connections 2024-05-10T11:39:48.550 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.531+0000 7f612c860740 1 --2- 172.21.15.100:0/641819584 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c44a5ac910 0x55c44a5aeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:48.551 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.531+0000 7f612c860740 1 --2- 172.21.15.100:0/641819584 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c44a5a77a0 0x55c44a5ac2a0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:48.551 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.531+0000 7f612c860740 1 --2- 172.21.15.100:0/641819584 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c44a5a6db0 0x55c44a5a71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:48.551 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.531+0000 7f612c860740 1 -- 172.21.15.100:0/641819584 >> 172.21.15.100:0/641819584 conn(0x55c44a5a2420 msgr2=0x55c44a5a2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:39:48.551 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.531+0000 7f612c860740 1 -- 172.21.15.100:0/641819584 shutdown_connections 2024-05-10T11:39:48.551 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.531+0000 7f612c860740 1 -- 172.21.15.100:0/641819584 wait complete. 2024-05-10T11:39:48.551 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.531+0000 7f612c860740 1 -- start start 2024-05-10T11:39:48.552 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.532+0000 7f612c860740 1 Processor -- start 2024-05-10T11:39:48.552 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.532+0000 7f612c860740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c44a5a6db0 0x55c44a5b1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:48.552 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.532+0000 7f612c860740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c44a5a77a0 0x55c44a5b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:48.552 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.532+0000 7f612c860740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c44a5ac910 0x55c44a5b85f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:48.552 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.532+0000 7f612c860740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55c44a5af860 con 0x55c44a5a77a0 2024-05-10T11:39:48.552 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.532+0000 7f61237fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c44a5a6db0 0x55c44a5b1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:48.553 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.532+0000 7f61237fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c44a5a6db0 0x55c44a5b1070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:45280/0 (socket says 172.21.15.100:45280) 2024-05-10T11:39:48.553 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.532+0000 7f61237fe640 1 -- 172.21.15.100:0/7191874 learned_addr learned my addr 172.21.15.100:0/7191874 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:39:48.553 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.532+0000 7f6122ffd640 1 --2- 172.21.15.100:0/7191874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c44a5a77a0 0x55c44a5b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:48.553 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.533+0000 7f6123fff640 1 --2- 172.21.15.100:0/7191874 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c44a5ac910 0x55c44a5b85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:48.554 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.533+0000 7f612c860740 1 -- 172.21.15.100:0/7191874 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c44a4c8370 con 0x55c44a5a6db0 2024-05-10T11:39:48.554 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.533+0000 7f612c860740 1 -- 172.21.15.100:0/7191874 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c44a5af9e0 con 0x55c44a5ac910 2024-05-10T11:39:48.554 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.533+0000 7f612c860740 10 client.0 _ll_register_callbacks cb 0x55c44a21ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:39:48.554 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.533+0000 7f6122ffd640 1 -- 172.21.15.100:0/7191874 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c44a5ac910 msgr2=0x55c44a5b85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:48.554 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.533+0000 7f6122ffd640 1 --2- 172.21.15.100:0/7191874 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c44a5ac910 0x55c44a5b85f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:48.555 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.533+0000 7f6122ffd640 1 -- 172.21.15.100:0/7191874 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c44a5a6db0 msgr2=0x55c44a5b1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:48.555 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.533+0000 7f6122ffd640 1 --2- 172.21.15.100:0/7191874 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c44a5a6db0 0x55c44a5b1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:48.555 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.533+0000 7f6122ffd640 1 -- 172.21.15.100:0/7191874 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c44a2e9620 con 0x55c44a5a77a0 2024-05-10T11:39:48.555 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.533+0000 7f61237fe640 1 --2- 172.21.15.100:0/7191874 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c44a5a6db0 0x55c44a5b1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:39:48.555 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.533+0000 7f612c860740 1 -- 172.21.15.100:0/7191874 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61180054b0 con 0x55c44a5a77a0 2024-05-10T11:39:48.555 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.533+0000 7f6122ffd640 1 --2- 172.21.15.100:0/7191874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c44a5a77a0 0x55c44a5b15b0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f6118023040 tx=0x7f61180133a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22e4cb4a2a2a487b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:39:48.556 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.533+0000 7f6102ffd640 10 client.19620 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:39:48.556 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.534+0000 7f6102ffd640 1 -- 172.21.15.100:0/7191874 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611800ec30 con 0x55c44a5a77a0 2024-05-10T11:39:48.556 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.534+0000 7f6102ffd640 1 -- 172.21.15.100:0/7191874 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f611800edd0 con 0x55c44a5a77a0 2024-05-10T11:39:48.556 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.534+0000 7f6102ffd640 1 -- 172.21.15.100:0/7191874 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f61180050d0 con 0x55c44a5a77a0 2024-05-10T11:39:48.556 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.534+0000 7f612c860740 10 client.19620 fetch_fsmap learned FSMap version 41 2024-05-10T11:39:48.556 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.534+0000 7f612c860740 1 -- 172.21.15.100:0/7191874 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({fsmap=41}) -- 0x55c44a5bb550 con 0x55c44a5a77a0 2024-05-10T11:39:48.557 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.534+0000 7f6102ffd640 1 -- 172.21.15.100:0/7191874 <== mon.0 v2:172.21.15.100:3300/0 4 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f6118020050 con 0x55c44a5a77a0 2024-05-10T11:39:48.557 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.535+0000 7f612c860740 10 client.19620 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:39:48.557 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.535+0000 7f612c860740 10 client.19620 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:39:48.557 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.535+0000 7f612c860740 20 client.19620 populate_metadata read hostname 'smithi100' 2024-05-10T11:39:48.557 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.535+0000 7f612c860740 1 --2- 172.21.15.100:0/7191874 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c44a5bc940 0x55c44a5bed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:48.557 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.535+0000 7f612c860740 4 client.19620 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:39:48.558 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.535+0000 7f612c860740 1 -- 172.21.15.100:0/7191874 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c44a5191e0 con 0x55c44a5bc940 2024-05-10T11:39:48.558 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.535+0000 7f61237fe640 1 --2- 172.21.15.100:0/7191874 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c44a5bc940 0x55c44a5bed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:48.558 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.536+0000 7f61237fe640 1 --2- 172.21.15.100:0/7191874 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c44a5bc940 0x55c44a5bed30 crc :-1 s=READY pgs=407 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67cb9a9b30fafcd server_cookie=d49a4b2de915afdd in_seq=0 out_seq=0 2024-05-10T11:39:48.558 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.536+0000 7f6102ffd640 10 client.19620 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:39:48.558 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.537+0000 7f6102ffd640 1 -- 172.21.15.100:0/7191874 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c44a5191e0 con 0x55c44a5bc940 2024-05-10T11:39:48.559 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.537+0000 7f6102ffd640 10 client.19620 handle_command_reply: tid=0 2024-05-10T11:39:48.559 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.537+0000 7f612c860740 1 client.19620 shutdown 2024-05-10T11:39:48.559 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.537+0000 7f612c860740 1 -- 172.21.15.100:0/7191874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c44a5a77a0 msgr2=0x55c44a5b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:48.559 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.537+0000 7f612c860740 1 --2- 172.21.15.100:0/7191874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c44a5a77a0 0x55c44a5b15b0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f6118023040 tx=0x7f61180133a0 comp rx=0 tx=0).stop 2024-05-10T11:39:48.559 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.537+0000 7f612c860740 1 -- 172.21.15.100:0/7191874 shutdown_connections 2024-05-10T11:39:48.559 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.537+0000 7f612c860740 1 --2- 172.21.15.100:0/7191874 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c44a5bc940 0x55c44a5bed30 crc :-1 s=READY pgs=407 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.537+0000 7f612c860740 1 --2- 172.21.15.100:0/7191874 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c44a5ac910 0x55c44a5b85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.537+0000 7f612c860740 1 --2- 172.21.15.100:0/7191874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c44a5a77a0 0x55c44a5b15b0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.537+0000 7f612c860740 1 --2- 172.21.15.100:0/7191874 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c44a5a6db0 0x55c44a5b1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.537+0000 7f6102ffd640 0 client.19620 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:39:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.538+0000 7f612c860740 1 -- 172.21.15.100:0/7191874 >> 172.21.15.100:0/7191874 conn(0x55c44a5a2420 msgr2=0x55c44a5a1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:39:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.538+0000 7f612c860740 1 -- 172.21.15.100:0/7191874 shutdown_connections 2024-05-10T11:39:48.561 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.538+0000 7f612c860740 1 -- 172.21.15.100:0/7191874 wait complete. 2024-05-10T11:39:48.561 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.538+0000 7f612c860740 20 client.19620 trim_cache size 0 max 16384 2024-05-10T11:39:48.565 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.563+0000 7f612c860740 1 Processor -- start 2024-05-10T11:39:48.565 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.563+0000 7f612c860740 1 -- start start 2024-05-10T11:39:48.565 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.563+0000 7f612c860740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c44a5a6db0 0x55c44a5ac050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:48.565 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.563+0000 7f612c860740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c44a5a77a0 0x55c44a5a8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:48.565 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.563+0000 7f612c860740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c44a5ac910 0x55c44a5a8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:48.565 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.563+0000 7f612c860740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f61241adbc0 con 0x55c44a5ac910 2024-05-10T11:39:48.566 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.563+0000 7f612c860740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f61241ab490 con 0x55c44a5a77a0 2024-05-10T11:39:48.566 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.563+0000 7f612c860740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f61241a92c0 con 0x55c44a5a6db0 2024-05-10T11:39:48.566 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.564+0000 7f61237fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c44a5a6db0 0x55c44a5ac050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:48.566 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.564+0000 7f6122ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c44a5a77a0 0x55c44a5a8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:48.566 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.564+0000 7f61237fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c44a5a6db0 0x55c44a5ac050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:55526/0 (socket says 172.21.15.100:55526) 2024-05-10T11:39:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.564+0000 7f61237fe640 1 -- 172.21.15.100:0/1487092861 learned_addr learned my addr 172.21.15.100:0/1487092861 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:39:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.564+0000 7f6123fff640 1 --2- 172.21.15.100:0/1487092861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c44a5ac910 0x55c44a5a8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.564+0000 7f6123fff640 1 -- 172.21.15.100:0/1487092861 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c44a5a6db0 msgr2=0x55c44a5ac050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.564+0000 7f6123fff640 1 --2- 172.21.15.100:0/1487092861 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c44a5a6db0 0x55c44a5ac050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.564+0000 7f6123fff640 1 -- 172.21.15.100:0/1487092861 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c44a5a77a0 msgr2=0x55c44a5a8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.564+0000 7f6123fff640 1 --2- 172.21.15.100:0/1487092861 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c44a5a77a0 0x55c44a5a8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.564+0000 7f6123fff640 1 -- 172.21.15.100:0/1487092861 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c44a5bf0a0 con 0x55c44a5ac910 2024-05-10T11:39:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.564+0000 7f61237fe640 1 --2- 172.21.15.100:0/1487092861 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c44a5a6db0 0x55c44a5ac050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:39:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.564+0000 7f6123fff640 1 --2- 172.21.15.100:0/1487092861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c44a5ac910 0x55c44a5a8f60 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f611c0a9b60 tx=0x7f611c0a5da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8949570136bb01a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:39:48.574 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.565+0000 7f61097fa640 1 -- 172.21.15.100:0/1487092861 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611c0054d0 con 0x55c44a5ac910 2024-05-10T11:39:48.574 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.565+0000 7f61097fa640 1 -- 172.21.15.100:0/1487092861 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f611c0051d0 con 0x55c44a5ac910 2024-05-10T11:39:48.574 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.565+0000 7f61097fa640 1 -- 172.21.15.100:0/1487092861 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611c0b2a90 con 0x55c44a5ac910 2024-05-10T11:39:48.574 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.566+0000 7f612c860740 1 -- 172.21.15.100:0/1487092861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c44a5ac910 msgr2=0x55c44a5a8f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:48.574 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.566+0000 7f612c860740 1 --2- 172.21.15.100:0/1487092861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c44a5ac910 0x55c44a5a8f60 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f611c0a9b60 tx=0x7f611c0a5da0 comp rx=0 tx=0).stop 2024-05-10T11:39:48.574 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.566+0000 7f612c860740 1 -- 172.21.15.100:0/1487092861 shutdown_connections 2024-05-10T11:39:48.575 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.566+0000 7f612c860740 1 --2- 172.21.15.100:0/1487092861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c44a5ac910 0x55c44a5a8f60 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:48.575 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.566+0000 7f612c860740 1 --2- 172.21.15.100:0/1487092861 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c44a5a77a0 0x55c44a5a8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:48.575 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.566+0000 7f612c860740 1 --2- 172.21.15.100:0/1487092861 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c44a5a6db0 0x55c44a5ac050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:48.575 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.566+0000 7f612c860740 1 -- 172.21.15.100:0/1487092861 >> 172.21.15.100:0/1487092861 conn(0x55c44a5a2420 msgr2=0x55c44a5a2800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:39:48.576 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.567+0000 7f612c860740 1 -- 172.21.15.100:0/1487092861 shutdown_connections 2024-05-10T11:39:48.576 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.567+0000 7f612c860740 1 -- 172.21.15.100:0/1487092861 wait complete. 2024-05-10T11:39:48.576 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.567+0000 7f612c860740 1 -- start start 2024-05-10T11:39:48.576 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.568+0000 7f612c860740 1 Processor -- start 2024-05-10T11:39:48.576 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.568+0000 7f612c860740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c44a5a6db0 0x55c44a5aaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:48.576 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.568+0000 7f612c860740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c44a5a77a0 0x55c44a5aafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:48.577 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.568+0000 7f612c860740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c44a5ac910 0x55c44a629730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:48.577 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.568+0000 7f61237fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c44a5a6db0 0x55c44a5aaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:48.577 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.568+0000 7f61237fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c44a5a6db0 0x55c44a5aaa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:55528/0 (socket says 172.21.15.100:55528) 2024-05-10T11:39:48.577 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.568+0000 7f61237fe640 1 -- 172.21.15.100:0/1747591142 learned_addr learned my addr 172.21.15.100:0/1747591142 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:39:48.577 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.568+0000 7f6122ffd640 1 --2- 172.21.15.100:0/1747591142 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c44a5a77a0 0x55c44a5aafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:48.577 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.569+0000 7f612c860740 1 -- 172.21.15.100:0/1747591142 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f61241a9950 con 0x55c44a5ac910 2024-05-10T11:39:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.569+0000 7f612c860740 1 -- 172.21.15.100:0/1747591142 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f61241aadc0 con 0x55c44a5a77a0 2024-05-10T11:39:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.569+0000 7f612c860740 1 -- 172.21.15.100:0/1747591142 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f61241aa020 con 0x55c44a5a6db0 2024-05-10T11:39:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.569+0000 7f612c860740 10 client.0 _ll_register_callbacks cb 0x7f6118009c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:39:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.569+0000 7f61237fe640 1 -- 172.21.15.100:0/1747591142 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c44a5a77a0 msgr2=0x55c44a5aafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.569+0000 7f61237fe640 1 --2- 172.21.15.100:0/1747591142 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c44a5a77a0 0x55c44a5aafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:48.579 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.569+0000 7f61237fe640 1 -- 172.21.15.100:0/1747591142 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c44a5ac910 msgr2=0x55c44a629730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:48.579 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.569+0000 7f61237fe640 1 --2- 172.21.15.100:0/1747591142 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c44a5ac910 0x55c44a629730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:48.579 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.569+0000 7f61237fe640 1 -- 172.21.15.100:0/1747591142 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c44a629d20 con 0x55c44a5a6db0 2024-05-10T11:39:48.579 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.569+0000 7f6122ffd640 1 --2- 172.21.15.100:0/1747591142 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c44a5a77a0 0x55c44a5aafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:39:48.579 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.569+0000 7f61237fe640 1 --2- 172.21.15.100:0/1747591142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c44a5a6db0 0x55c44a5aaa70 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6114009900 tx=0x7f6114004340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3580ed6870c1d3e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:39:48.579 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.569+0000 7f6102ffd640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:39:48.579 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.570+0000 7f6102ffd640 1 -- 172.21.15.100:0/1747591142 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6114035040 con 0x55c44a5a6db0 2024-05-10T11:39:48.580 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.570+0000 7f6102ffd640 1 -- 172.21.15.100:0/1747591142 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f6114031b90 con 0x55c44a5a6db0 2024-05-10T11:39:48.580 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.570+0000 7f612c860740 1 -- 172.21.15.100:0/1747591142 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f611c0051d0 con 0x55c44a5a6db0 2024-05-10T11:39:48.580 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.570+0000 7f6102ffd640 1 -- 172.21.15.100:0/1747591142 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f6114054050 con 0x55c44a5a6db0 2024-05-10T11:39:48.580 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.570+0000 7f612c860740 10 client.28999 fetch_fsmap learned FSMap version 41 2024-05-10T11:39:48.580 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.570+0000 7f612c860740 1 -- 172.21.15.100:0/1747591142 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55c44a62b080 con 0x55c44a5a6db0 2024-05-10T11:39:48.580 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.571+0000 7f6102ffd640 1 -- 172.21.15.100:0/1747591142 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f61140044b0 con 0x55c44a5a6db0 2024-05-10T11:39:48.581 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.571+0000 7f6102ffd640 1 -- 172.21.15.100:0/1747591142 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f6114003790 con 0x55c44a5a6db0 2024-05-10T11:39:48.581 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.571+0000 7f612c860740 10 client.28999 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:39:48.581 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.571+0000 7f612c860740 10 client.28999 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:39:48.581 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.571+0000 7f612c860740 20 client.28999 populate_metadata read hostname 'smithi100' 2024-05-10T11:39:48.581 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.571+0000 7f612c860740 1 --2- 172.21.15.100:0/1747591142 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c44a62bae0 0x55c44a62df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:48.581 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.571+0000 7f612c860740 4 client.28999 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:39:48.581 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.571+0000 7f612c860740 1 -- 172.21.15.100:0/1747591142 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f611c0051d0 con 0x55c44a62bae0 2024-05-10T11:39:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.572+0000 7f6122ffd640 1 --2- 172.21.15.100:0/1747591142 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c44a62bae0 0x55c44a62df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.572+0000 7f6122ffd640 1 --2- 172.21.15.100:0/1747591142 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c44a62bae0 0x55c44a62df00 crc :-1 s=READY pgs=408 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6518fd7082eb83f7 server_cookie=2369ce50e1297c64 in_seq=0 out_seq=0 2024-05-10T11:39:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.572+0000 7f6102ffd640 10 client.28999 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:39:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.733+0000 7f6102ffd640 1 -- 172.21.15.100:0/1747591142 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f611c0051d0 con 0x55c44a62bae0 2024-05-10T11:39:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.733+0000 7f6102ffd640 10 client.28999 handle_command_reply: tid=0 2024-05-10T11:39:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.733+0000 7f612c860740 1 client.28999 shutdown 2024-05-10T11:39:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.733+0000 7f612c860740 1 -- 172.21.15.100:0/1747591142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c44a5a6db0 msgr2=0x55c44a5aaa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.733+0000 7f612c860740 1 --2- 172.21.15.100:0/1747591142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c44a5a6db0 0x55c44a5aaa70 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6114009900 tx=0x7f6114004340 comp rx=0 tx=0).stop 2024-05-10T11:39:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.733+0000 7f612c860740 1 -- 172.21.15.100:0/1747591142 shutdown_connections 2024-05-10T11:39:48.736 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.733+0000 7f612c860740 1 --2- 172.21.15.100:0/1747591142 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c44a62bae0 0x55c44a62df00 crc :-1 s=READY pgs=408 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:48.736 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.733+0000 7f612c860740 1 --2- 172.21.15.100:0/1747591142 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c44a5ac910 0x55c44a629730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:48.736 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.733+0000 7f612c860740 1 --2- 172.21.15.100:0/1747591142 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c44a5a77a0 0x55c44a5aafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:48.736 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.733+0000 7f612c860740 1 --2- 172.21.15.100:0/1747591142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c44a5a6db0 0x55c44a5aaa70 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:48.736 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.733+0000 7f612c860740 1 -- 172.21.15.100:0/1747591142 >> 172.21.15.100:0/1747591142 conn(0x55c44a5a2420 msgr2=0x55c44a5a8a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:39:48.736 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.733+0000 7f6102ffd640 0 client.28999 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:39:48.736 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.733+0000 7f612c860740 1 -- 172.21.15.100:0/1747591142 shutdown_connections 2024-05-10T11:39:48.736 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.734+0000 7f612c860740 1 -- 172.21.15.100:0/1747591142 wait complete. 2024-05-10T11:39:48.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.734+0000 7f612c860740 20 client.28999 trim_cache size 0 max 16384 2024-05-10T11:39:48.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:39:48.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:39:48.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:39:48.738 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:39:48.739 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.734+0000 7f6129b4e640 1 -- 172.21.15.100:0/2478917860 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f610c0b2920 msgr2=0x7f610c0b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:48.739 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.734+0000 7f6129b4e640 1 --2- 172.21.15.100:0/2478917860 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f610c0b2920 0x7f610c0b4de0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f6118013990 tx=0x7f61180098a0 comp rx=0 tx=0).stop 2024-05-10T11:39:48.739 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.734+0000 7f6129b4e640 1 -- 172.21.15.100:0/2478917860 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6124102f00 msgr2=0x7f612419fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:48.739 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.734+0000 7f6129b4e640 1 --2- 172.21.15.100:0/2478917860 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6124102f00 0x7f612419fe90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f611c009fd0 tx=0x7f611c00ef10 comp rx=0 tx=0).stop 2024-05-10T11:39:48.739 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.735+0000 7f6129b4e640 1 -- 172.21.15.100:0/2478917860 shutdown_connections 2024-05-10T11:39:48.739 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.735+0000 7f6129b4e640 1 --2- 172.21.15.100:0/2478917860 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f610c0b2920 0x7f610c0b4de0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:48.740 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.735+0000 7f6129b4e640 1 --2- 172.21.15.100:0/2478917860 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6124102f00 0x7f612419fe90 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:48.740 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.735+0000 7f6129b4e640 1 --2- 172.21.15.100:0/2478917860 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6124102560 0x7f612419d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:48.740 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.735+0000 7f6129b4e640 1 --2- 172.21.15.100:0/2478917860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6124101ba0 0x7f6124101610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:48.740 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.735+0000 7f6129b4e640 1 -- 172.21.15.100:0/2478917860 >> 172.21.15.100:0/2478917860 conn(0x7f61240fc610 msgr2=0x7f6124106020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:39:48.740 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.735+0000 7f6129b4e640 1 -- 172.21.15.100:0/2478917860 shutdown_connections 2024-05-10T11:39:48.740 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:39:48.735+0000 7f6129b4e640 1 -- 172.21.15.100:0/2478917860 wait complete. 2024-05-10T11:39:48.948 INFO:tasks.workunit.client.0.smithi100.stdout: 10 77095 8.96 MB/sec execute 342 sec latency 128.790 ms 2024-05-10T11:39:49.398 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T11:39:49.717 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:39:49 smithi100 ceph-mon[30296]: scrub queued for path: / 2024-05-10T11:39:49.717 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:39:49 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:39:49.717 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:39:49 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:39:49.717 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:39:49 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:39:49.717 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:39:49 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:39:49.717 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:39:49 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:39:49.717 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:39:49 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:39:49.717 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:39:49 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:39:49.718 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:39:49 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:39:49.718 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:39:49 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:39:49.718 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:39:49 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:39:49.718 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:39:49 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:39:49.718 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:39:49 smithi100 ceph-mon[30296]: pgmap v1064: 129 pgs: 129 active+clean; 534 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 3.4 MiB/s wr, 255 op/s 2024-05-10T11:39:49.718 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:39:49 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:39:49.718 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:39:49 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:39:49.718 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:39:49 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:39:49.718 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:39:49 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:39:49.771 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:39:49.948 INFO:tasks.workunit.client.0.smithi100.stdout: 10 77235 8.96 MB/sec execute 343 sec latency 125.495 ms 2024-05-10T11:39:50.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:39:49 smithi178 ceph-mon[33359]: scrub queued for path: / 2024-05-10T11:39:50.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:39:49 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:39:50.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:39:49 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:39:50.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:39:49 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:39:50.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:39:49 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:39:50.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:39:49 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:39:50.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:39:49 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:39:50.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:39:49 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:39:50.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:39:49 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:39:50.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:39:49 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:39:50.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:39:49 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:39:50.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:39:49 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:39:50.004 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:39:49 smithi178 ceph-mon[33359]: pgmap v1064: 129 pgs: 129 active+clean; 534 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 3.4 MiB/s wr, 255 op/s 2024-05-10T11:39:50.004 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:39:49 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:39:50.004 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:39:49 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:39:50.004 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:39:49 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:39:50.004 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:39:49 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:39:50.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:39:49 smithi188 ceph-mon[34075]: scrub queued for path: / 2024-05-10T11:39:50.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:39:49 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:39:50.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:39:49 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:39:50.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:39:49 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:39:50.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:39:49 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:39:50.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:39:49 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:39:50.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:39:49 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:39:50.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:39:49 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:39:50.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:39:49 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:39:50.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:39:49 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:39:50.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:39:49 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:39:50.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:39:49 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:39:50.030 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:39:49 smithi188 ceph-mon[34075]: pgmap v1064: 129 pgs: 129 active+clean; 534 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 3.4 MiB/s wr, 255 op/s 2024-05-10T11:39:50.030 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:39:49 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:39:50.030 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:39:49 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:39:50.030 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:39:49 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:39:50.030 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:39:49 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:39:50.948 INFO:tasks.workunit.client.0.smithi100.stdout: 10 77419 8.97 MB/sec execute 344 sec latency 77.885 ms 2024-05-10T11:39:51.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.918+0000 7faf8ef4e700 1 -- 172.21.15.100:0/2349923237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf88103eb0 msgr2=0x7faf88110390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:51.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.918+0000 7faf8ef4e700 1 --2- 172.21.15.100:0/2349923237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf88103eb0 0x7faf88110390 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7faf8000b0a0 tx=0x7faf8000b3b0 comp rx=0 tx=0).stop 2024-05-10T11:39:51.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.918+0000 7faf8ef4e700 1 -- 172.21.15.100:0/2349923237 shutdown_connections 2024-05-10T11:39:51.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.918+0000 7faf8ef4e700 1 --2- 172.21.15.100:0/2349923237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf88103eb0 0x7faf88110390 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:51.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.918+0000 7faf8ef4e700 1 --2- 172.21.15.100:0/2349923237 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf88109600 0x7faf881099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:51.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.918+0000 7faf8ef4e700 1 --2- 172.21.15.100:0/2349923237 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf881034f0 0x7faf88103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:51.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.918+0000 7faf8ef4e700 1 -- 172.21.15.100:0/2349923237 >> 172.21.15.100:0/2349923237 conn(0x7faf8807a650 msgr2=0x7faf8807aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:39:51.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.918+0000 7faf8ef4e700 1 -- 172.21.15.100:0/2349923237 shutdown_connections 2024-05-10T11:39:51.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.918+0000 7faf8ef4e700 1 -- 172.21.15.100:0/2349923237 wait complete. 2024-05-10T11:39:51.920 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:39:51 smithi100 ceph-mon[30296]: pgmap v1065: 129 pgs: 129 active+clean; 541 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1021 KiB/s rd, 4.3 MiB/s wr, 342 op/s 2024-05-10T11:39:51.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.919+0000 7faf8ef4e700 1 Processor -- start 2024-05-10T11:39:51.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.919+0000 7faf8ef4e700 1 -- start start 2024-05-10T11:39:51.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.919+0000 7faf8ef4e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf881034f0 0x7faf8819f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:51.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.919+0000 7faf8ef4e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf88103eb0 0x7faf8819fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:51.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.919+0000 7faf8ef4e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf88109600 0x7faf88199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:51.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.919+0000 7faf8ef4e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7faf88113500 con 0x7faf881034f0 2024-05-10T11:39:51.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.919+0000 7faf8ef4e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7faf88113380 con 0x7faf88109600 2024-05-10T11:39:51.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.919+0000 7faf8ef4e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7faf88113680 con 0x7faf88103eb0 2024-05-10T11:39:51.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.919+0000 7faf87fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf88103eb0 0x7faf8819fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:51.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.919+0000 7faf87fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf88103eb0 0x7faf8819fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:55548/0 (socket says 172.21.15.100:55548) 2024-05-10T11:39:51.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.919+0000 7faf87fff700 1 -- 172.21.15.100:0/3539935407 learned_addr learned my addr 172.21.15.100:0/3539935407 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:39:51.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.919+0000 7faf8ccea700 1 --2- 172.21.15.100:0/3539935407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf881034f0 0x7faf8819f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:51.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.919+0000 7faf8d4eb700 1 --2- 172.21.15.100:0/3539935407 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf88109600 0x7faf88199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:51.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.920+0000 7faf87fff700 1 -- 172.21.15.100:0/3539935407 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf88109600 msgr2=0x7faf88199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:51.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.920+0000 7faf87fff700 1 --2- 172.21.15.100:0/3539935407 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf88109600 0x7faf88199840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:51.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.920+0000 7faf87fff700 1 -- 172.21.15.100:0/3539935407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf881034f0 msgr2=0x7faf8819f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:51.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.920+0000 7faf87fff700 1 --2- 172.21.15.100:0/3539935407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf881034f0 0x7faf8819f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:51.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.920+0000 7faf87fff700 1 -- 172.21.15.100:0/3539935407 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf8819a020 con 0x7faf88103eb0 2024-05-10T11:39:51.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.920+0000 7faf8ccea700 1 --2- 172.21.15.100:0/3539935407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf881034f0 0x7faf8819f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:39:51.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.920+0000 7faf87fff700 1 --2- 172.21.15.100:0/3539935407 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf88103eb0 0x7faf8819fbb0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7faf7400e9a0 tx=0x7faf7400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:39:51.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.920+0000 7faf85ffb700 1 -- 172.21.15.100:0/3539935407 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7400ca10 con 0x7faf88103eb0 2024-05-10T11:39:51.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.920+0000 7faf8ef4e700 1 -- 172.21.15.100:0/3539935407 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf8819a2b0 con 0x7faf88103eb0 2024-05-10T11:39:51.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.920+0000 7faf85ffb700 1 -- 172.21.15.100:0/3539935407 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faf7400cbb0 con 0x7faf88103eb0 2024-05-10T11:39:51.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.921+0000 7faf85ffb700 1 -- 172.21.15.100:0/3539935407 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf74018680 con 0x7faf88103eb0 2024-05-10T11:39:51.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.921+0000 7faf8ef4e700 1 -- 172.21.15.100:0/3539935407 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf880756d0 con 0x7faf88103eb0 2024-05-10T11:39:51.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.923+0000 7faf85ffb700 1 -- 172.21.15.100:0/3539935407 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7faf74018820 con 0x7faf88103eb0 2024-05-10T11:39:51.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.924+0000 7faf85ffb700 1 --2- 172.21.15.100:0/3539935407 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7faf780bb110 0x7faf780bd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:51.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.925+0000 7faf8ccea700 1 --2- 172.21.15.100:0/3539935407 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7faf780bb110 0x7faf780bd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:51.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.925+0000 7faf85ffb700 1 -- 172.21.15.100:0/3539935407 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7faf740e2700 con 0x7faf88103eb0 2024-05-10T11:39:51.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.925+0000 7faf997e1b80 1 Processor -- start 2024-05-10T11:39:51.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.925+0000 7faf997e1b80 1 -- start start 2024-05-10T11:39:51.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.925+0000 7faf997e1b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55661f15b4d0 0x55661f15b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:51.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.925+0000 7faf997e1b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55661f15bdf0 0x55661f15f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:51.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.925+0000 7faf8ccea700 1 --2- 172.21.15.100:0/3539935407 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7faf780bb110 0x7faf780bd5d0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7faf7c011490 tx=0x7faf7c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:39:51.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.925+0000 7faf997e1b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55661f15fc80 0x55661f160100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:51.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.925+0000 7faf997e1b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55661f0ad860 con 0x55661f15bdf0 2024-05-10T11:39:51.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.925+0000 7faf997e1b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55661f04c7c0 con 0x55661f15b4d0 2024-05-10T11:39:51.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.925+0000 7faf997e1b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55661f0a9820 con 0x55661f15fc80 2024-05-10T11:39:51.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.926+0000 7faf8ccea700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55661f15b4d0 0x55661f15b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:51.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.926+0000 7faf8ccea700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55661f15b4d0 0x55661f15b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:45336/0 (socket says 172.21.15.100:45336) 2024-05-10T11:39:51.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.926+0000 7faf8ccea700 1 -- 172.21.15.100:0/2179344401 learned_addr learned my addr 172.21.15.100:0/2179344401 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:39:51.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.926+0000 7faf8d4eb700 1 --2- 172.21.15.100:0/2179344401 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55661f15fc80 0x55661f160100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:51.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.926+0000 7faf87fff700 1 --2- 172.21.15.100:0/2179344401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55661f15bdf0 0x55661f15f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:51.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.926+0000 7faf8ccea700 1 -- 172.21.15.100:0/2179344401 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55661f15fc80 msgr2=0x55661f160100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:51.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.926+0000 7faf8ccea700 1 --2- 172.21.15.100:0/2179344401 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55661f15fc80 0x55661f160100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:51.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.926+0000 7faf8ccea700 1 -- 172.21.15.100:0/2179344401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55661f15bdf0 msgr2=0x55661f15f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:51.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.926+0000 7faf8ccea700 1 --2- 172.21.15.100:0/2179344401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55661f15bdf0 0x55661f15f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:51.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.926+0000 7faf8ccea700 1 -- 172.21.15.100:0/2179344401 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55661effa8d0 con 0x55661f15b4d0 2024-05-10T11:39:51.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.926+0000 7faf87fff700 1 --2- 172.21.15.100:0/2179344401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55661f15bdf0 0x55661f15f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:39:51.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.926+0000 7faf8ccea700 1 --2- 172.21.15.100:0/2179344401 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55661f15b4d0 0x55661f15b8b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7faf7c0043c0 tx=0x7faf7c014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d68aa08ef45179a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:39:51.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.926+0000 7faf72ffd700 1 -- 172.21.15.100:0/2179344401 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7c003d10 con 0x55661f15b4d0 2024-05-10T11:39:51.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.926+0000 7faf72ffd700 1 -- 172.21.15.100:0/2179344401 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faf7c011e50 con 0x55661f15b4d0 2024-05-10T11:39:51.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.926+0000 7faf72ffd700 1 -- 172.21.15.100:0/2179344401 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7c01e650 con 0x55661f15b4d0 2024-05-10T11:39:51.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.927+0000 7faf997e1b80 1 -- 172.21.15.100:0/2179344401 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55661f15b4d0 msgr2=0x55661f15b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:51.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.927+0000 7faf997e1b80 1 --2- 172.21.15.100:0/2179344401 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55661f15b4d0 0x55661f15b8b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7faf7c0043c0 tx=0x7faf7c014000 comp rx=0 tx=0).stop 2024-05-10T11:39:51.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.927+0000 7faf997e1b80 1 -- 172.21.15.100:0/2179344401 shutdown_connections 2024-05-10T11:39:51.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.927+0000 7faf997e1b80 1 --2- 172.21.15.100:0/2179344401 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55661f15fc80 0x55661f160100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:51.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.927+0000 7faf997e1b80 1 --2- 172.21.15.100:0/2179344401 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55661f15b4d0 0x55661f15b8b0 secure :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7faf7c0043c0 tx=0x7faf7c014000 comp rx=0 tx=0).stop 2024-05-10T11:39:51.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.927+0000 7faf997e1b80 1 --2- 172.21.15.100:0/2179344401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55661f15bdf0 0x55661f15f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:51.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.927+0000 7faf997e1b80 1 -- 172.21.15.100:0/2179344401 >> 172.21.15.100:0/2179344401 conn(0x55661f087100 msgr2=0x55661f155c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:39:51.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.927+0000 7faf997e1b80 1 -- 172.21.15.100:0/2179344401 shutdown_connections 2024-05-10T11:39:51.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.927+0000 7faf997e1b80 1 -- 172.21.15.100:0/2179344401 wait complete. 2024-05-10T11:39:51.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.927+0000 7faf997e1b80 1 -- start start 2024-05-10T11:39:51.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.928+0000 7faf997e1b80 1 Processor -- start 2024-05-10T11:39:51.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.928+0000 7faf997e1b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55661f15bdf0 0x55661f1672f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:51.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.928+0000 7faf997e1b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55661f15fc80 0x55661f1645f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:51.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.928+0000 7faf997e1b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55661f164b30 0x55661f164fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:51.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.928+0000 7faf997e1b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55661f167ee0 con 0x55661f15bdf0 2024-05-10T11:39:51.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.928+0000 7faf997e1b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55661f013c30 con 0x55661f15fc80 2024-05-10T11:39:51.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.928+0000 7faf997e1b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55661f168060 con 0x55661f164b30 2024-05-10T11:39:51.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.928+0000 7faf997e1b80 10 client.0 _ll_register_callbacks cb 0x55661f045920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:39:51.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.928+0000 7faf8ccea700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55661f15bdf0 0x55661f1672f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:51.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.928+0000 7faf8d4eb700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55661f164b30 0x55661f164fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:51.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.928+0000 7faf8d4eb700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55661f164b30 0x55661f164fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:55566/0 (socket says 172.21.15.100:55566) 2024-05-10T11:39:51.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.928+0000 7faf8d4eb700 1 -- 172.21.15.100:0/2104293788 learned_addr learned my addr 172.21.15.100:0/2104293788 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:39:51.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.928+0000 7faf87fff700 1 --2- 172.21.15.100:0/2104293788 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55661f15fc80 0x55661f1645f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:51.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.929+0000 7faf8d4eb700 1 -- 172.21.15.100:0/2104293788 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55661f15fc80 msgr2=0x55661f1645f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:51.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.929+0000 7faf8d4eb700 1 --2- 172.21.15.100:0/2104293788 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55661f15fc80 0x55661f1645f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:51.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.929+0000 7faf8d4eb700 1 -- 172.21.15.100:0/2104293788 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55661f15bdf0 msgr2=0x55661f1672f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:51.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.929+0000 7faf8d4eb700 1 --2- 172.21.15.100:0/2104293788 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55661f15bdf0 0x55661f1672f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:51.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.929+0000 7faf8d4eb700 1 -- 172.21.15.100:0/2104293788 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55661f049690 con 0x55661f164b30 2024-05-10T11:39:51.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.929+0000 7faf8ccea700 1 --2- 172.21.15.100:0/2104293788 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55661f15bdf0 0x55661f1672f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:39:51.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.929+0000 7faf8d4eb700 1 --2- 172.21.15.100:0/2104293788 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55661f164b30 0x55661f164fb0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7faf80015000 tx=0x7faf8000bd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f55d8df8ccb542cc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:39:51.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.929+0000 7faf67fff700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:39:51.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.929+0000 7faf67fff700 1 -- 172.21.15.100:0/2104293788 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf800074e0 con 0x55661f164b30 2024-05-10T11:39:51.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.930+0000 7faf67fff700 1 -- 172.21.15.100:0/2104293788 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7faf80007680 con 0x55661f164b30 2024-05-10T11:39:51.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.930+0000 7faf997e1b80 1 -- 172.21.15.100:0/2104293788 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf7c011e50 con 0x55661f164b30 2024-05-10T11:39:51.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.930+0000 7faf67fff700 1 -- 172.21.15.100:0/2104293788 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7faf80007880 con 0x55661f164b30 2024-05-10T11:39:51.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.930+0000 7faf997e1b80 10 client.29011 fetch_fsmap learned FSMap version 41 2024-05-10T11:39:51.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.930+0000 7faf997e1b80 1 -- 172.21.15.100:0/2104293788 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55661f172d40 con 0x55661f164b30 2024-05-10T11:39:51.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.930+0000 7faf67fff700 1 -- 172.21.15.100:0/2104293788 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faf7c011e50 con 0x55661f164b30 2024-05-10T11:39:51.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.930+0000 7faf67fff700 1 -- 172.21.15.100:0/2104293788 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7faf80011070 con 0x55661f164b30 2024-05-10T11:39:51.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.930+0000 7faf997e1b80 10 client.29011 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:39:51.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.930+0000 7faf997e1b80 10 client.29011 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:39:51.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.930+0000 7faf997e1b80 20 client.29011 populate_metadata read hostname 'smithi100' 2024-05-10T11:39:51.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.930+0000 7faf997e1b80 1 --2- 172.21.15.100:0/2104293788 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55661f175380 0x55661f177770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:51.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.930+0000 7faf997e1b80 4 client.29011 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:39:51.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.930+0000 7faf997e1b80 1 -- 172.21.15.100:0/2104293788 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55661f127650 con 0x55661f175380 2024-05-10T11:39:51.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.931+0000 7faf8ccea700 1 --2- 172.21.15.100:0/2104293788 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55661f175380 0x55661f177770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:51.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.931+0000 7faf8ccea700 1 --2- 172.21.15.100:0/2104293788 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55661f175380 0x55661f177770 crc :-1 s=READY pgs=409 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b67cd7b4a27a800d server_cookie=825350e5ce7d2410 in_seq=0 out_seq=0 2024-05-10T11:39:51.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.931+0000 7faf67fff700 10 client.29011 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:39:51.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.932+0000 7faf67fff700 1 -- 172.21.15.100:0/2104293788 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55661f127650 con 0x55661f175380 2024-05-10T11:39:51.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.932+0000 7faf67fff700 10 client.29011 handle_command_reply: tid=0 2024-05-10T11:39:51.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.932+0000 7faf997e1b80 1 client.29011 shutdown 2024-05-10T11:39:51.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.932+0000 7faf997e1b80 1 -- 172.21.15.100:0/2104293788 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55661f164b30 msgr2=0x55661f164fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:51.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.932+0000 7faf997e1b80 1 --2- 172.21.15.100:0/2104293788 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55661f164b30 0x55661f164fb0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7faf80015000 tx=0x7faf8000bd50 comp rx=0 tx=0).stop 2024-05-10T11:39:51.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.932+0000 7faf997e1b80 1 -- 172.21.15.100:0/2104293788 shutdown_connections 2024-05-10T11:39:51.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.932+0000 7faf997e1b80 1 --2- 172.21.15.100:0/2104293788 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55661f175380 0x55661f177770 crc :-1 s=READY pgs=409 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:51.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.932+0000 7faf997e1b80 1 --2- 172.21.15.100:0/2104293788 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55661f164b30 0x55661f164fb0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:51.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.932+0000 7faf997e1b80 1 --2- 172.21.15.100:0/2104293788 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55661f15bdf0 0x55661f1672f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:51.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.932+0000 7faf67fff700 0 client.29011 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:39:51.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.932+0000 7faf997e1b80 1 --2- 172.21.15.100:0/2104293788 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55661f15fc80 0x55661f1645f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:51.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.932+0000 7faf997e1b80 1 -- 172.21.15.100:0/2104293788 >> 172.21.15.100:0/2104293788 conn(0x55661f087100 msgr2=0x55661f150720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:39:51.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.932+0000 7faf997e1b80 1 -- 172.21.15.100:0/2104293788 shutdown_connections 2024-05-10T11:39:51.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.932+0000 7faf997e1b80 1 -- 172.21.15.100:0/2104293788 wait complete. 2024-05-10T11:39:51.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.932+0000 7faf997e1b80 20 client.29011 trim_cache size 0 max 16384 2024-05-10T11:39:51.948 INFO:tasks.workunit.client.0.smithi100.stdout: 10 77585 8.97 MB/sec execute 345 sec latency 70.251 ms 2024-05-10T11:39:51.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.964+0000 7faf997e1b80 1 Processor -- start 2024-05-10T11:39:51.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.964+0000 7faf997e1b80 1 -- start start 2024-05-10T11:39:51.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.965+0000 7faf997e1b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55661f15bdf0 0x55661f17d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:51.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.965+0000 7faf997e1b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55661f15fc80 0x55661f17d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:51.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.965+0000 7faf997e1b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55661f164b30 0x55661f16d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:51.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.965+0000 7faf997e1b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7faf881af6b0 con 0x55661f164b30 2024-05-10T11:39:51.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.965+0000 7faf997e1b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7faf881acf40 con 0x55661f15fc80 2024-05-10T11:39:51.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.965+0000 7faf997e1b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7faf881aad90 con 0x55661f15bdf0 2024-05-10T11:39:51.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.965+0000 7faf8d4eb700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55661f164b30 0x55661f16d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:51.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.965+0000 7faf8d4eb700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55661f164b30 0x55661f16d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40066/0 (socket says 172.21.15.100:40066) 2024-05-10T11:39:51.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.965+0000 7faf8d4eb700 1 -- 172.21.15.100:0/779726408 learned_addr learned my addr 172.21.15.100:0/779726408 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:39:51.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.965+0000 7faf87fff700 1 --2- 172.21.15.100:0/779726408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55661f15fc80 0x55661f17d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:51.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.965+0000 7faf8ccea700 1 --2- 172.21.15.100:0/779726408 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55661f15bdf0 0x55661f17d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:51.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.965+0000 7faf8d4eb700 1 -- 172.21.15.100:0/779726408 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55661f15bdf0 msgr2=0x55661f17d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:51.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.965+0000 7faf8d4eb700 1 --2- 172.21.15.100:0/779726408 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55661f15bdf0 0x55661f17d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:51.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.965+0000 7faf8d4eb700 1 -- 172.21.15.100:0/779726408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55661f15fc80 msgr2=0x55661f17d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:51.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.965+0000 7faf8d4eb700 1 --2- 172.21.15.100:0/779726408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55661f15fc80 0x55661f17d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:51.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.965+0000 7faf8d4eb700 1 -- 172.21.15.100:0/779726408 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55661f177ae0 con 0x55661f164b30 2024-05-10T11:39:51.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.965+0000 7faf8d4eb700 1 --2- 172.21.15.100:0/779726408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55661f164b30 0x55661f16d2e0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7faf80009e60 tx=0x7faf80027420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bb242a7f9a3fd18 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:39:51.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.966+0000 7faf727fc700 1 -- 172.21.15.100:0/779726408 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf80005930 con 0x55661f164b30 2024-05-10T11:39:51.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.966+0000 7faf727fc700 1 -- 172.21.15.100:0/779726408 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faf80004ca0 con 0x55661f164b30 2024-05-10T11:39:51.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.966+0000 7faf727fc700 1 -- 172.21.15.100:0/779726408 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf80005c60 con 0x55661f164b30 2024-05-10T11:39:51.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.967+0000 7faf997e1b80 1 -- 172.21.15.100:0/779726408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55661f164b30 msgr2=0x55661f16d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:51.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.967+0000 7faf997e1b80 1 --2- 172.21.15.100:0/779726408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55661f164b30 0x55661f16d2e0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7faf80009e60 tx=0x7faf80027420 comp rx=0 tx=0).stop 2024-05-10T11:39:51.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.967+0000 7faf997e1b80 1 -- 172.21.15.100:0/779726408 shutdown_connections 2024-05-10T11:39:51.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.967+0000 7faf997e1b80 1 --2- 172.21.15.100:0/779726408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55661f164b30 0x55661f16d2e0 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:51.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.967+0000 7faf997e1b80 1 --2- 172.21.15.100:0/779726408 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55661f15bdf0 0x55661f17d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:51.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.967+0000 7faf997e1b80 1 --2- 172.21.15.100:0/779726408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55661f15fc80 0x55661f17d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:51.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.967+0000 7faf997e1b80 1 -- 172.21.15.100:0/779726408 >> 172.21.15.100:0/779726408 conn(0x55661f087100 msgr2=0x55661f155b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:39:51.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.967+0000 7faf997e1b80 1 -- 172.21.15.100:0/779726408 shutdown_connections 2024-05-10T11:39:51.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.967+0000 7faf997e1b80 1 -- 172.21.15.100:0/779726408 wait complete. 2024-05-10T11:39:51.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.967+0000 7faf997e1b80 1 -- start start 2024-05-10T11:39:51.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.967+0000 7faf997e1b80 1 Processor -- start 2024-05-10T11:39:51.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.968+0000 7faf997e1b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55661f15bdf0 0x55661f16a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:51.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.968+0000 7faf997e1b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55661f15fc80 0x55661f16adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:51.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.968+0000 7faf997e1b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55661f164b30 0x55661f1d7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:51.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.968+0000 7faf997e1b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7faf881ab400 con 0x55661f15bdf0 2024-05-10T11:39:51.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.968+0000 7faf997e1b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7faf881ac870 con 0x55661f164b30 2024-05-10T11:39:51.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.968+0000 7faf997e1b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7faf881abad0 con 0x55661f15fc80 2024-05-10T11:39:51.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.968+0000 7faf87fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55661f15fc80 0x55661f16adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:51.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.968+0000 7faf87fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55661f15fc80 0x55661f16adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:55580/0 (socket says 172.21.15.100:55580) 2024-05-10T11:39:51.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.968+0000 7faf87fff700 1 -- 172.21.15.100:0/2428525910 learned_addr learned my addr 172.21.15.100:0/2428525910 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:39:51.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.968+0000 7faf997e1b80 10 client.0 _ll_register_callbacks cb 0x55661f045920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:39:51.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.968+0000 7faf8ccea700 1 --2- 172.21.15.100:0/2428525910 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55661f15bdf0 0x55661f16a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:51.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.968+0000 7faf8d4eb700 1 --2- 172.21.15.100:0/2428525910 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55661f164b30 0x55661f1d7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:51.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.968+0000 7faf87fff700 1 -- 172.21.15.100:0/2428525910 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55661f164b30 msgr2=0x55661f1d7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:51.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.968+0000 7faf87fff700 1 --2- 172.21.15.100:0/2428525910 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55661f164b30 0x55661f1d7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:51.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.968+0000 7faf87fff700 1 -- 172.21.15.100:0/2428525910 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55661f15bdf0 msgr2=0x55661f16a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:39:51.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.968+0000 7faf87fff700 1 --2- 172.21.15.100:0/2428525910 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55661f15bdf0 0x55661f16a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:51.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.968+0000 7faf87fff700 1 -- 172.21.15.100:0/2428525910 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55661f1d8270 con 0x55661f15fc80 2024-05-10T11:39:51.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.969+0000 7faf87fff700 1 --2- 172.21.15.100:0/2428525910 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55661f15fc80 0x55661f16adc0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7faf740a6c30 tx=0x7faf740b22b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f50886d4317d9ed server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:39:51.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.969+0000 7faf67fff700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:39:51.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.969+0000 7faf67fff700 1 -- 172.21.15.100:0/2428525910 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf740a99c0 con 0x55661f15fc80 2024-05-10T11:39:51.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.969+0000 7faf997e1b80 1 -- 172.21.15.100:0/2428525910 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf80004ca0 con 0x55661f15fc80 2024-05-10T11:39:51.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.969+0000 7faf67fff700 1 -- 172.21.15.100:0/2428525910 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7faf740a5fb0 con 0x55661f15fc80 2024-05-10T11:39:51.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.969+0000 7faf67fff700 1 -- 172.21.15.100:0/2428525910 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7faf740add70 con 0x55661f15fc80 2024-05-10T11:39:51.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.969+0000 7faf997e1b80 10 client.29017 fetch_fsmap learned FSMap version 41 2024-05-10T11:39:51.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.969+0000 7faf997e1b80 1 -- 172.21.15.100:0/2428525910 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55661f1df6d0 con 0x55661f15fc80 2024-05-10T11:39:51.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.969+0000 7faf67fff700 1 -- 172.21.15.100:0/2428525910 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faf740b2820 con 0x55661f15fc80 2024-05-10T11:39:51.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.970+0000 7faf67fff700 1 -- 172.21.15.100:0/2428525910 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7faf740a5330 con 0x55661f15fc80 2024-05-10T11:39:51.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.970+0000 7faf997e1b80 10 client.29017 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:39:51.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.970+0000 7faf997e1b80 10 client.29017 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:39:51.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.970+0000 7faf997e1b80 20 client.29017 populate_metadata read hostname 'smithi100' 2024-05-10T11:39:51.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.970+0000 7faf997e1b80 1 --2- 172.21.15.100:0/2428525910 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55661f1dfe10 0x55661f1e2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:39:51.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.970+0000 7faf997e1b80 4 client.29017 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T11:39:51.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.970+0000 7faf997e1b80 1 -- 172.21.15.100:0/2428525910 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7faf80004ca0 con 0x55661f1dfe10 2024-05-10T11:39:51.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.970+0000 7faf8ccea700 1 --2- 172.21.15.100:0/2428525910 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55661f1dfe10 0x55661f1e2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:39:51.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.970+0000 7faf8ccea700 1 --2- 172.21.15.100:0/2428525910 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55661f1dfe10 0x55661f1e2230 crc :-1 s=READY pgs=410 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a95f6a1848e1addc server_cookie=4c04a14857e00d08 in_seq=0 out_seq=0 2024-05-10T11:39:51.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.970+0000 7faf67fff700 10 client.29017 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:39:51.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.971+0000 7faf67fff700 1 -- 172.21.15.100:0/2428525910 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7faf7c034e50 con 0x55661f1dfe10 2024-05-10T11:39:51.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.971+0000 7faf67fff700 10 client.29017 handle_command_reply: tid=0 2024-05-10T11:39:51.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.971+0000 7faf997e1b80 1 client.29017 shutdown 2024-05-10T11:39:51.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.971+0000 7faf997e1b80 1 -- 172.21.15.100:0/2428525910 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55661f15fc80 msgr2=0x55661f16adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:51.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.971+0000 7faf997e1b80 1 --2- 172.21.15.100:0/2428525910 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55661f15fc80 0x55661f16adc0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7faf740a6c30 tx=0x7faf740b22b0 comp rx=0 tx=0).stop 2024-05-10T11:39:51.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.971+0000 7faf997e1b80 1 -- 172.21.15.100:0/2428525910 shutdown_connections 2024-05-10T11:39:51.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.971+0000 7faf997e1b80 1 --2- 172.21.15.100:0/2428525910 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55661f1dfe10 0x55661f1e2230 crc :-1 s=READY pgs=410 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:51.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.971+0000 7faf997e1b80 1 --2- 172.21.15.100:0/2428525910 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55661f164b30 0x55661f1d7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:51.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.971+0000 7faf997e1b80 1 --2- 172.21.15.100:0/2428525910 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55661f15bdf0 0x55661f16a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:51.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.971+0000 7faf997e1b80 1 --2- 172.21.15.100:0/2428525910 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55661f15fc80 0x55661f16adc0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:51.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.971+0000 7faf997e1b80 1 -- 172.21.15.100:0/2428525910 >> 172.21.15.100:0/2428525910 conn(0x55661f087100 msgr2=0x55661f150720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:39:51.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.971+0000 7faf67fff700 0 client.29017 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:39:51.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.972+0000 7faf997e1b80 1 -- 172.21.15.100:0/2428525910 shutdown_connections 2024-05-10T11:39:51.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.972+0000 7faf997e1b80 1 -- 172.21.15.100:0/2428525910 wait complete. 2024-05-10T11:39:51.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.972+0000 7faf997e1b80 20 client.29017 trim_cache size 0 max 16384 2024-05-10T11:39:51.976 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:39:51.976 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T11:39:51.977 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-05-10T11:39:51.977 INFO:teuthology.orchestra.run.smithi100.stdout: "73adf975-1f4a-40e6-9a7c-f44ef7f24e5a": { 2024-05-10T11:39:51.977 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-05-10T11:39:51.977 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "73adf975-1f4a-40e6-9a7c-f44ef7f24e5a", 2024-05-10T11:39:51.977 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-05-10T11:39:51.977 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:39:51.977 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:39:51.977 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:39:51.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.973+0000 7faf8ef4e700 1 -- 172.21.15.100:0/3539935407 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7faf780bb110 msgr2=0x7faf780bd5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:51.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.973+0000 7faf8ef4e700 1 --2- 172.21.15.100:0/3539935407 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7faf780bb110 0x7faf780bd5d0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7faf7c011490 tx=0x7faf7c009260 comp rx=0 tx=0).stop 2024-05-10T11:39:51.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.973+0000 7faf8ef4e700 1 -- 172.21.15.100:0/3539935407 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf88103eb0 msgr2=0x7faf8819fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:39:51.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.973+0000 7faf8ef4e700 1 --2- 172.21.15.100:0/3539935407 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf88103eb0 0x7faf8819fbb0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7faf7400e9a0 tx=0x7faf7400ecb0 comp rx=0 tx=0).stop 2024-05-10T11:39:51.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.973+0000 7faf8ef4e700 1 -- 172.21.15.100:0/3539935407 shutdown_connections 2024-05-10T11:39:51.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.973+0000 7faf8ef4e700 1 --2- 172.21.15.100:0/3539935407 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7faf780bb110 0x7faf780bd5d0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:51.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.973+0000 7faf8ef4e700 1 --2- 172.21.15.100:0/3539935407 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf88109600 0x7faf88199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:51.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.973+0000 7faf8ef4e700 1 --2- 172.21.15.100:0/3539935407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf881034f0 0x7faf8819f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:51.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.973+0000 7faf8ef4e700 1 --2- 172.21.15.100:0/3539935407 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf88103eb0 0x7faf8819fbb0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:39:51.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.973+0000 7faf8ef4e700 1 -- 172.21.15.100:0/3539935407 >> 172.21.15.100:0/3539935407 conn(0x7faf8807a650 msgr2=0x7faf88102180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:39:51.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.973+0000 7faf8ef4e700 1 -- 172.21.15.100:0/3539935407 shutdown_connections 2024-05-10T11:39:51.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:39:51.973+0000 7faf8ef4e700 1 -- 172.21.15.100:0/3539935407 wait complete. 2024-05-10T11:39:52.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:39:51 smithi178 ceph-mon[33359]: pgmap v1065: 129 pgs: 129 active+clean; 541 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1021 KiB/s rd, 4.3 MiB/s wr, 342 op/s 2024-05-10T11:39:52.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:39:51 smithi188 ceph-mon[34075]: pgmap v1065: 129 pgs: 129 active+clean; 541 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1021 KiB/s rd, 4.3 MiB/s wr, 342 op/s 2024-05-10T11:39:52.906 INFO:tasks.cephfs.filesystem:scrub status for tag:73adf975-1f4a-40e6-9a7c-f44ef7f24e5a - {'path': '/', 'tag': '73adf975-1f4a-40e6-9a7c-f44ef7f24e5a', 'options': 'recursive,force'} 2024-05-10T11:39:52.948 INFO:tasks.workunit.client.0.smithi100.stdout: 10 77715 8.95 MB/sec execute 346 sec latency 70.759 ms 2024-05-10T11:39:53.948 INFO:tasks.workunit.client.0.smithi100.stdout: 10 77853 8.93 MB/sec execute 347 sec latency 132.207 ms 2024-05-10T11:39:54.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:39:53 smithi178 ceph-mon[33359]: pgmap v1066: 129 pgs: 129 active+clean; 542 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 3.5 MiB/s wr, 284 op/s 2024-05-10T11:39:54.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:39:53 smithi188 ceph-mon[34075]: pgmap v1066: 129 pgs: 129 active+clean; 542 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 3.5 MiB/s wr, 284 op/s 2024-05-10T11:39:54.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:39:53 smithi100 ceph-mon[30296]: pgmap v1066: 129 pgs: 129 active+clean; 542 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 3.5 MiB/s wr, 284 op/s 2024-05-10T11:39:54.948 INFO:tasks.workunit.client.0.smithi100.stdout: 10 77975 8.93 MB/sec execute 348 sec latency 633.186 ms 2024-05-10T11:39:55.948 INFO:tasks.workunit.client.0.smithi100.stdout: 10 78155 8.93 MB/sec execute 349 sec latency 125.833 ms 2024-05-10T11:39:56.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:39:55 smithi178 ceph-mon[33359]: pgmap v1067: 129 pgs: 129 active+clean; 544 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 875 KiB/s rd, 4.2 MiB/s wr, 313 op/s 2024-05-10T11:39:56.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:39:55 smithi188 ceph-mon[34075]: pgmap v1067: 129 pgs: 129 active+clean; 544 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 875 KiB/s rd, 4.2 MiB/s wr, 313 op/s 2024-05-10T11:39:56.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:39:55 smithi100 ceph-mon[30296]: pgmap v1067: 129 pgs: 129 active+clean; 544 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 875 KiB/s rd, 4.2 MiB/s wr, 313 op/s 2024-05-10T11:39:56.949 INFO:tasks.workunit.client.0.smithi100.stdout: 10 78328 8.92 MB/sec execute 350 sec latency 99.293 ms 2024-05-10T11:39:57.949 INFO:tasks.workunit.client.0.smithi100.stdout: 10 78497 8.93 MB/sec execute 351 sec latency 67.916 ms 2024-05-10T11:39:58.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:39:57 smithi178 ceph-mon[33359]: pgmap v1068: 129 pgs: 129 active+clean; 547 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1006 KiB/s rd, 4.7 MiB/s wr, 348 op/s 2024-05-10T11:39:58.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:39:57 smithi188 ceph-mon[34075]: pgmap v1068: 129 pgs: 129 active+clean; 547 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1006 KiB/s rd, 4.7 MiB/s wr, 348 op/s 2024-05-10T11:39:58.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:39:57 smithi100 ceph-mon[30296]: pgmap v1068: 129 pgs: 129 active+clean; 547 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1006 KiB/s rd, 4.7 MiB/s wr, 348 op/s 2024-05-10T11:39:58.949 INFO:tasks.workunit.client.0.smithi100.stdout: 10 78692 8.95 MB/sec execute 352 sec latency 72.870 ms 2024-05-10T11:39:59.949 INFO:tasks.workunit.client.0.smithi100.stdout: 10 78874 8.95 MB/sec execute 353 sec latency 111.217 ms 2024-05-10T11:40:00.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:39:59 smithi178 ceph-mon[33359]: pgmap v1069: 129 pgs: 129 active+clean; 544 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 4.0 MiB/s wr, 291 op/s 2024-05-10T11:40:00.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:39:59 smithi188 ceph-mon[34075]: pgmap v1069: 129 pgs: 129 active+clean; 544 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 4.0 MiB/s wr, 291 op/s 2024-05-10T11:40:00.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:39:59 smithi100 ceph-mon[30296]: pgmap v1069: 129 pgs: 129 active+clean; 544 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 4.0 MiB/s wr, 291 op/s 2024-05-10T11:40:00.949 INFO:tasks.workunit.client.0.smithi100.stdout: 10 79071 8.94 MB/sec execute 354 sec latency 68.424 ms 2024-05-10T11:40:01.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:40:00 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:40:01.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:40:00 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:40:01.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:40:00 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:40:01.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:40:00 smithi178 ceph-mon[33359]: overall HEALTH_OK 2024-05-10T11:40:01.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:40:00 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:40:01.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:40:00 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:40:01.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:40:00 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:40:01.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:40:00 smithi188 ceph-mon[34075]: overall HEALTH_OK 2024-05-10T11:40:01.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:40:00 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:40:01.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:40:00 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:40:01.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:40:00 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:40:01.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:40:00 smithi100 ceph-mon[30296]: overall HEALTH_OK 2024-05-10T11:40:01.870 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.869+0000 7ff9d274d640 1 -- 172.21.15.100:0/2244293304 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9cc10c320 msgr2=0x7ff9cc10e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:01.871 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.869+0000 7ff9d274d640 1 --2- 172.21.15.100:0/2244293304 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9cc10c320 0x7ff9cc10e790 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc009860 tx=0x7ff9bc02a340 comp rx=0 tx=0).stop 2024-05-10T11:40:01.871 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.869+0000 7ff9d274d640 1 -- 172.21.15.100:0/2244293304 shutdown_connections 2024-05-10T11:40:01.871 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.869+0000 7ff9d274d640 1 --2- 172.21.15.100:0/2244293304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9cc10ecd0 0x7ff9cc10f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:01.871 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.869+0000 7ff9d274d640 1 --2- 172.21.15.100:0/2244293304 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9cc10c320 0x7ff9cc10e790 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:01.871 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.869+0000 7ff9d274d640 1 --2- 172.21.15.100:0/2244293304 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9cc1099b0 0x7ff9cc10bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:01.871 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.869+0000 7ff9d274d640 1 -- 172.21.15.100:0/2244293304 >> 172.21.15.100:0/2244293304 conn(0x7ff9cc06fa20 msgr2=0x7ff9cc071e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:40:01.871 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.869+0000 7ff9d274d640 1 -- 172.21.15.100:0/2244293304 shutdown_connections 2024-05-10T11:40:01.871 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.869+0000 7ff9d274d640 1 -- 172.21.15.100:0/2244293304 wait complete. 2024-05-10T11:40:01.871 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.869+0000 7ff9d274d640 1 Processor -- start 2024-05-10T11:40:01.871 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.870+0000 7ff9d274d640 1 -- start start 2024-05-10T11:40:01.873 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.870+0000 7ff9d274d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9cc1099b0 0x7ff9cc1a78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:01.873 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.870+0000 7ff9d274d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9cc10c320 0x7ff9cc1a7de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:01.873 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.870+0000 7ff9d274d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9cc10ecd0 0x7ff9cc1aee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:01.873 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.870+0000 7ff9d274d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff9cc11d200 con 0x7ff9cc10ecd0 2024-05-10T11:40:01.873 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.870+0000 7ff9d274d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff9cc11d080 con 0x7ff9cc1099b0 2024-05-10T11:40:01.873 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.870+0000 7ff9d274d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff9cc11d380 con 0x7ff9cc10c320 2024-05-10T11:40:01.873 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.870+0000 7ff9d0cea640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9cc10ecd0 0x7ff9cc1aee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:01.873 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.870+0000 7ff9cb7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9cc10c320 0x7ff9cc1a7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:01.873 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.870+0000 7ff9cbfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9cc1099b0 0x7ff9cc1a78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:01.873 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.870+0000 7ff9cb7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9cc10c320 0x7ff9cc1a7de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:51930/0 (socket says 172.21.15.100:51930) 2024-05-10T11:40:01.874 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.870+0000 7ff9cb7fe640 1 -- 172.21.15.100:0/3725875161 learned_addr learned my addr 172.21.15.100:0/3725875161 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:40:01.874 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.870+0000 7ff9cbfff640 1 -- 172.21.15.100:0/3725875161 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9cc10c320 msgr2=0x7ff9cc1a7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:01.874 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.870+0000 7ff9cbfff640 1 --2- 172.21.15.100:0/3725875161 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9cc10c320 0x7ff9cc1a7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:01.874 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.870+0000 7ff9cbfff640 1 -- 172.21.15.100:0/3725875161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9cc10ecd0 msgr2=0x7ff9cc1aee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:01.874 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.870+0000 7ff9cbfff640 1 --2- 172.21.15.100:0/3725875161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9cc10ecd0 0x7ff9cc1aee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:01.874 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.870+0000 7ff9cbfff640 1 -- 172.21.15.100:0/3725875161 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9cc1af4c0 con 0x7ff9cc1099b0 2024-05-10T11:40:01.874 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.871+0000 7ff9cbfff640 1 --2- 172.21.15.100:0/3725875161 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9cc1099b0 0x7ff9cc1a78a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff9b400e9f0 tx=0x7ff9b400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:40:01.874 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.871+0000 7ff9c97fa640 1 -- 172.21.15.100:0/3725875161 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b400ccc0 con 0x7ff9cc1099b0 2024-05-10T11:40:01.874 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.871+0000 7ff9d274d640 1 -- 172.21.15.100:0/3725875161 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9cc1af7b0 con 0x7ff9cc1099b0 2024-05-10T11:40:01.874 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.871+0000 7ff9c97fa640 1 -- 172.21.15.100:0/3725875161 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff9b40108c0 con 0x7ff9cc1099b0 2024-05-10T11:40:01.874 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.871+0000 7ff9c97fa640 1 -- 172.21.15.100:0/3725875161 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b40187c0 con 0x7ff9cc1099b0 2024-05-10T11:40:01.874 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.871+0000 7ff9d274d640 1 -- 172.21.15.100:0/3725875161 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9cc1afd50 con 0x7ff9cc1099b0 2024-05-10T11:40:01.876 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.874+0000 7ff9d5462740 1 Processor -- start 2024-05-10T11:40:01.876 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.874+0000 7ff9d5462740 1 -- start start 2024-05-10T11:40:01.876 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.874+0000 7ff9d5462740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556882b23db0 0x556882b241d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:01.876 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.874+0000 7ff9d5462740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556882b247a0 0x556882b292a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:01.877 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.874+0000 7ff9d5462740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556882b29910 0x556882b2bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:01.877 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.874+0000 7ff9d5462740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x556882a885d0 con 0x556882b23db0 2024-05-10T11:40:01.877 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.874+0000 7ff9d5462740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556882a41890 con 0x556882b247a0 2024-05-10T11:40:01.877 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.874+0000 7ff9d5462740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556882a84590 con 0x556882b29910 2024-05-10T11:40:01.877 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.874+0000 7ff9c97fa640 1 -- 172.21.15.100:0/3725875161 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7ff9b4010430 con 0x7ff9cc1099b0 2024-05-10T11:40:01.877 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.874+0000 7ff9d0cea640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556882b23db0 0x556882b241d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:01.877 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.874+0000 7ff9d0cea640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556882b23db0 0x556882b241d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42700/0 (socket says 172.21.15.100:42700) 2024-05-10T11:40:01.877 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.874+0000 7ff9d0cea640 1 -- 172.21.15.100:0/3636464664 learned_addr learned my addr 172.21.15.100:0/3636464664 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:40:01.877 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.874+0000 7ff9cb7fe640 1 --2- 172.21.15.100:0/3636464664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556882b29910 0x556882b2bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:01.877 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.875+0000 7ff9cbfff640 1 --2- 172.21.15.100:0/3636464664 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556882b247a0 0x556882b292a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:01.877 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.875+0000 7ff9cb7fe640 1 -- 172.21.15.100:0/3636464664 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556882b247a0 msgr2=0x556882b292a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:01.877 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.875+0000 7ff9cb7fe640 1 --2- 172.21.15.100:0/3636464664 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556882b247a0 0x556882b292a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:01.877 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.875+0000 7ff9cb7fe640 1 -- 172.21.15.100:0/3636464664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556882b23db0 msgr2=0x556882b241d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:01.877 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.875+0000 7ff9cb7fe640 1 --2- 172.21.15.100:0/3636464664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556882b23db0 0x556882b241d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:01.877 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.875+0000 7ff9cb7fe640 1 -- 172.21.15.100:0/3636464664 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55688279a210 con 0x556882b29910 2024-05-10T11:40:01.878 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.875+0000 7ff9d0cea640 1 --2- 172.21.15.100:0/3636464664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556882b23db0 0x556882b241d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:40:01.878 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.875+0000 7ff9cb7fe640 1 --2- 172.21.15.100:0/3636464664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556882b29910 0x556882b2bda0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc005cd0 tx=0x7ff9bc005b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=299da54e33296088 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:40:01.878 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.875+0000 7ff9c27fc640 1 -- 172.21.15.100:0/3636464664 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc037040 con 0x556882b29910 2024-05-10T11:40:01.878 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.875+0000 7ff9c27fc640 1 -- 172.21.15.100:0/3636464664 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff9bc0306b0 con 0x556882b29910 2024-05-10T11:40:01.878 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.875+0000 7ff9c27fc640 1 -- 172.21.15.100:0/3636464664 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc005260 con 0x556882b29910 2024-05-10T11:40:01.878 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.876+0000 7ff9c97fa640 1 --2- 172.21.15.100:0/3725875161 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff9b80b2920 0x7ff9b80b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:01.882 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.876+0000 7ff9d0cea640 1 --2- 172.21.15.100:0/3725875161 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff9b80b2920 0x7ff9b80b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:01.883 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.876+0000 7ff9c97fa640 1 -- 172.21.15.100:0/3725875161 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7ff9b40e27a0 con 0x7ff9cc1099b0 2024-05-10T11:40:01.883 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.876+0000 7ff9d5462740 1 -- 172.21.15.100:0/3636464664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556882b29910 msgr2=0x556882b2bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:01.883 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.876+0000 7ff9d0cea640 1 --2- 172.21.15.100:0/3725875161 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff9b80b2920 0x7ff9b80b4de0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x556882b25000 tx=0x7ff9c4010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:40:01.883 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.876+0000 7ff9d5462740 1 --2- 172.21.15.100:0/3636464664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556882b29910 0x556882b2bda0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc005cd0 tx=0x7ff9bc005b10 comp rx=0 tx=0).stop 2024-05-10T11:40:01.883 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.876+0000 7ff9d5462740 1 -- 172.21.15.100:0/3636464664 shutdown_connections 2024-05-10T11:40:01.883 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.876+0000 7ff9d5462740 1 --2- 172.21.15.100:0/3636464664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556882b29910 0x556882b2bda0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:01.883 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.876+0000 7ff9d5462740 1 --2- 172.21.15.100:0/3636464664 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556882b247a0 0x556882b292a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:01.883 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.876+0000 7ff9d5462740 1 --2- 172.21.15.100:0/3636464664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556882b23db0 0x556882b241d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:01.883 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.876+0000 7ff9d5462740 1 -- 172.21.15.100:0/3636464664 >> 172.21.15.100:0/3636464664 conn(0x556882b1f420 msgr2=0x556882b1f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:40:01.883 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.876+0000 7ff9d5462740 1 -- 172.21.15.100:0/3636464664 shutdown_connections 2024-05-10T11:40:01.883 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.877+0000 7ff9d5462740 1 -- 172.21.15.100:0/3636464664 wait complete. 2024-05-10T11:40:01.883 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.877+0000 7ff9d5462740 1 -- start start 2024-05-10T11:40:01.883 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.877+0000 7ff9d5462740 1 Processor -- start 2024-05-10T11:40:01.884 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.877+0000 7ff9d5462740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556882b23db0 0x556882b2df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:01.884 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.877+0000 7ff9d5462740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556882b247a0 0x556882b2e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:01.884 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.877+0000 7ff9d5462740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556882b29910 0x556882b35480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:01.884 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.877+0000 7ff9d5462740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x556882b2c860 con 0x556882b247a0 2024-05-10T11:40:01.884 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.877+0000 7ff9d5462740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556882a45370 con 0x556882b29910 2024-05-10T11:40:01.884 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.877+0000 7ff9d5462740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556882b2c9e0 con 0x556882b23db0 2024-05-10T11:40:01.884 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.877+0000 7ff9d5462740 10 client.0 _ll_register_callbacks cb 0x556882799a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:40:01.884 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.878+0000 7ff9d0cea640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556882b29910 0x556882b35480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:01.884 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.878+0000 7ff9cbfff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556882b23db0 0x556882b2df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:01.884 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.878+0000 7ff9d0cea640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556882b29910 0x556882b35480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:60798/0 (socket says 172.21.15.100:60798) 2024-05-10T11:40:01.884 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.878+0000 7ff9d0cea640 1 -- 172.21.15.100:0/102500842 learned_addr learned my addr 172.21.15.100:0/102500842 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:40:01.884 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.878+0000 7ff9cb7fe640 1 --2- 172.21.15.100:0/102500842 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556882b247a0 0x556882b2e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:01.884 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.878+0000 7ff9d0cea640 1 -- 172.21.15.100:0/102500842 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556882b23db0 msgr2=0x556882b2df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:01.884 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.878+0000 7ff9d0cea640 1 --2- 172.21.15.100:0/102500842 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556882b23db0 0x556882b2df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:01.884 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.878+0000 7ff9d0cea640 1 -- 172.21.15.100:0/102500842 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556882b247a0 msgr2=0x556882b2e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:01.885 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.878+0000 7ff9d0cea640 1 --2- 172.21.15.100:0/102500842 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556882b247a0 0x556882b2e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:01.885 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.878+0000 7ff9d0cea640 1 -- 172.21.15.100:0/102500842 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556882866620 con 0x556882b29910 2024-05-10T11:40:01.885 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.878+0000 7ff9d0cea640 1 --2- 172.21.15.100:0/102500842 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556882b29910 0x556882b35480 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff9c401a6d0 tx=0x7ff9c4004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8075a4cb558ef57 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:40:01.885 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.878+0000 7ff9ab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:40:01.885 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.878+0000 7ff9ab7fe640 1 -- 172.21.15.100:0/102500842 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c4020040 con 0x556882b29910 2024-05-10T11:40:01.885 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.878+0000 7ff9d5462740 1 -- 172.21.15.100:0/102500842 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9bc0306b0 con 0x556882b29910 2024-05-10T11:40:01.885 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.878+0000 7ff9ab7fe640 1 -- 172.21.15.100:0/102500842 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7ff9c4018820 con 0x556882b29910 2024-05-10T11:40:01.885 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.879+0000 7ff9ab7fe640 1 -- 172.21.15.100:0/102500842 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7ff9c4032050 con 0x556882b29910 2024-05-10T11:40:01.885 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.879+0000 7ff9d5462740 10 client.28916 fetch_fsmap learned FSMap version 41 2024-05-10T11:40:01.885 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.879+0000 7ff9d5462740 1 -- 172.21.15.100:0/102500842 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x556882b384d0 con 0x556882b29910 2024-05-10T11:40:01.885 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.879+0000 7ff9ab7fe640 1 -- 172.21.15.100:0/102500842 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff9bc0306b0 con 0x556882b29910 2024-05-10T11:40:01.885 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.879+0000 7ff9ab7fe640 1 -- 172.21.15.100:0/102500842 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7ff9c400b040 con 0x556882b29910 2024-05-10T11:40:01.885 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.879+0000 7ff9d5462740 10 client.28916 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:40:01.885 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.879+0000 7ff9d5462740 10 client.28916 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:40:01.886 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.879+0000 7ff9d5462740 20 client.28916 populate_metadata read hostname 'smithi100' 2024-05-10T11:40:01.886 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.879+0000 7ff9d5462740 1 --2- 172.21.15.100:0/102500842 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556882b39bd0 0x556882b3c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:01.886 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.879+0000 7ff9d5462740 4 client.28916 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:40:01.886 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.879+0000 7ff9d5462740 1 -- 172.21.15.100:0/102500842 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556882a961e0 con 0x556882b39bd0 2024-05-10T11:40:01.886 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.879+0000 7ff9cbfff640 1 --2- 172.21.15.100:0/102500842 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556882b39bd0 0x556882b3c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:01.886 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.880+0000 7ff9ab7fe640 10 client.28916 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:40:01.886 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.880+0000 7ff9cbfff640 1 --2- 172.21.15.100:0/102500842 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556882b39bd0 0x556882b3c040 crc :-1 s=READY pgs=411 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfd48d3cb7b9dc98 server_cookie=81fcd46073c4d4a0 in_seq=0 out_seq=0 2024-05-10T11:40:01.886 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.881+0000 7ff9ab7fe640 1 -- 172.21.15.100:0/102500842 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556882a961e0 con 0x556882b39bd0 2024-05-10T11:40:01.886 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.881+0000 7ff9ab7fe640 10 client.28916 handle_command_reply: tid=0 2024-05-10T11:40:01.886 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.881+0000 7ff9d5462740 1 client.28916 shutdown 2024-05-10T11:40:01.886 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.881+0000 7ff9d5462740 1 -- 172.21.15.100:0/102500842 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556882b29910 msgr2=0x556882b35480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:01.887 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.881+0000 7ff9d5462740 1 --2- 172.21.15.100:0/102500842 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556882b29910 0x556882b35480 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff9c401a6d0 tx=0x7ff9c4004ab0 comp rx=0 tx=0).stop 2024-05-10T11:40:01.887 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.881+0000 7ff9d5462740 1 -- 172.21.15.100:0/102500842 shutdown_connections 2024-05-10T11:40:01.887 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.881+0000 7ff9d5462740 1 --2- 172.21.15.100:0/102500842 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556882b39bd0 0x556882b3c040 crc :-1 s=READY pgs=411 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:01.887 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.881+0000 7ff9ab7fe640 0 client.28916 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:40:01.887 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.881+0000 7ff9d5462740 1 --2- 172.21.15.100:0/102500842 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556882b29910 0x556882b35480 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:01.887 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.881+0000 7ff9d5462740 1 --2- 172.21.15.100:0/102500842 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556882b247a0 0x556882b2e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:01.887 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.881+0000 7ff9d5462740 1 --2- 172.21.15.100:0/102500842 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556882b23db0 0x556882b2df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:01.887 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.881+0000 7ff9d5462740 1 -- 172.21.15.100:0/102500842 >> 172.21.15.100:0/102500842 conn(0x556882b1f420 msgr2=0x556882b1e580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:40:01.887 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.881+0000 7ff9d5462740 1 -- 172.21.15.100:0/102500842 shutdown_connections 2024-05-10T11:40:01.887 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.881+0000 7ff9d5462740 1 -- 172.21.15.100:0/102500842 wait complete. 2024-05-10T11:40:01.887 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.881+0000 7ff9d5462740 20 client.28916 trim_cache size 0 max 16384 2024-05-10T11:40:01.905 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.903+0000 7ff9d5462740 1 Processor -- start 2024-05-10T11:40:01.905 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.903+0000 7ff9d5462740 1 -- start start 2024-05-10T11:40:01.905 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.904+0000 7ff9d5462740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556882b23db0 0x556882b22330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:01.905 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.904+0000 7ff9d5462740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556882b247a0 0x556882b22870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:01.905 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.904+0000 7ff9d5462740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556882b29910 0x556882b40940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:01.905 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.904+0000 7ff9cbfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556882b23db0 0x556882b22330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:01.905 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.904+0000 7ff9cbfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556882b23db0 0x556882b22330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42720/0 (socket says 172.21.15.100:42720) 2024-05-10T11:40:01.906 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.904+0000 7ff9cbfff640 1 -- 172.21.15.100:0/2552539819 learned_addr learned my addr 172.21.15.100:0/2552539819 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:40:01.906 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.904+0000 7ff9d5462740 1 -- 172.21.15.100:0/2552539819 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff9cc1bc960 con 0x556882b23db0 2024-05-10T11:40:01.906 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.904+0000 7ff9d5462740 1 -- 172.21.15.100:0/2552539819 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff9cc1ba230 con 0x556882b29910 2024-05-10T11:40:01.906 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.904+0000 7ff9d5462740 1 -- 172.21.15.100:0/2552539819 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff9cc1b8080 con 0x556882b247a0 2024-05-10T11:40:01.906 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.904+0000 7ff9cb7fe640 1 --2- 172.21.15.100:0/2552539819 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556882b247a0 0x556882b22870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:01.906 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.904+0000 7ff9cb7fe640 1 -- 172.21.15.100:0/2552539819 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556882b29910 msgr2=0x556882b40940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:01.906 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.904+0000 7ff9cb7fe640 1 --2- 172.21.15.100:0/2552539819 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556882b29910 0x556882b40940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:01.906 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.904+0000 7ff9cb7fe640 1 -- 172.21.15.100:0/2552539819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556882b23db0 msgr2=0x556882b22330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:01.906 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.904+0000 7ff9cb7fe640 1 --2- 172.21.15.100:0/2552539819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556882b23db0 0x556882b22330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:01.906 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.904+0000 7ff9cb7fe640 1 -- 172.21.15.100:0/2552539819 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556882b1f250 con 0x556882b247a0 2024-05-10T11:40:01.906 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.904+0000 7ff9cbfff640 1 --2- 172.21.15.100:0/2552539819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556882b23db0 0x556882b22330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:40:01.910 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.905+0000 7ff9cb7fe640 1 --2- 172.21.15.100:0/2552539819 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556882b247a0 0x556882b22870 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x556882af4e50 tx=0x7ff9bc02ae00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5ddc05e6b8cae25 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:40:01.910 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.905+0000 7ff9c1ffb640 1 -- 172.21.15.100:0/2552539819 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc04e070 con 0x556882b247a0 2024-05-10T11:40:01.910 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.905+0000 7ff9c1ffb640 1 -- 172.21.15.100:0/2552539819 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff9bc0306b0 con 0x556882b247a0 2024-05-10T11:40:01.910 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.905+0000 7ff9c1ffb640 1 -- 172.21.15.100:0/2552539819 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc037040 con 0x556882b247a0 2024-05-10T11:40:01.910 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.906+0000 7ff9d5462740 1 -- 172.21.15.100:0/2552539819 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556882b247a0 msgr2=0x556882b22870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:01.910 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.906+0000 7ff9d5462740 1 --2- 172.21.15.100:0/2552539819 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556882b247a0 0x556882b22870 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x556882af4e50 tx=0x7ff9bc02ae00 comp rx=0 tx=0).stop 2024-05-10T11:40:01.910 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.906+0000 7ff9d5462740 1 -- 172.21.15.100:0/2552539819 shutdown_connections 2024-05-10T11:40:01.910 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.906+0000 7ff9d5462740 1 --2- 172.21.15.100:0/2552539819 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556882b29910 0x556882b40940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:01.910 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.906+0000 7ff9d5462740 1 --2- 172.21.15.100:0/2552539819 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556882b247a0 0x556882b22870 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:01.910 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.906+0000 7ff9d5462740 1 --2- 172.21.15.100:0/2552539819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556882b23db0 0x556882b22330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:01.910 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.906+0000 7ff9d5462740 1 -- 172.21.15.100:0/2552539819 >> 172.21.15.100:0/2552539819 conn(0x556882b1f420 msgr2=0x556882b25980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:40:01.910 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.906+0000 7ff9d5462740 1 -- 172.21.15.100:0/2552539819 shutdown_connections 2024-05-10T11:40:01.910 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.906+0000 7ff9d5462740 1 -- 172.21.15.100:0/2552539819 wait complete. 2024-05-10T11:40:01.911 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.907+0000 7ff9d5462740 1 -- start start 2024-05-10T11:40:01.911 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.907+0000 7ff9d5462740 1 Processor -- start 2024-05-10T11:40:01.911 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.907+0000 7ff9d5462740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556882b23db0 0x556882b326b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:01.911 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.907+0000 7ff9d5462740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556882b247a0 0x556882b32bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:01.911 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.907+0000 7ff9d5462740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556882b29910 0x556882b33c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:01.911 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.907+0000 7ff9d5462740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff9cc1b86f0 con 0x556882b23db0 2024-05-10T11:40:01.911 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.907+0000 7ff9d5462740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff9cc1b9b60 con 0x556882b29910 2024-05-10T11:40:01.911 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.907+0000 7ff9d5462740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff9cc1b8dc0 con 0x556882b247a0 2024-05-10T11:40:01.911 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.907+0000 7ff9d5462740 10 client.0 _ll_register_callbacks cb 0x7ff9c401d040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:40:01.911 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.907+0000 7ff9d0cea640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556882b29910 0x556882b33c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:01.911 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.907+0000 7ff9d0cea640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556882b29910 0x556882b33c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:60802/0 (socket says 172.21.15.100:60802) 2024-05-10T11:40:01.911 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.907+0000 7ff9cb7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556882b247a0 0x556882b32bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:01.911 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.907+0000 7ff9d0cea640 1 -- 172.21.15.100:0/3806784748 learned_addr learned my addr 172.21.15.100:0/3806784748 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:40:01.911 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.908+0000 7ff9cbfff640 1 --2- 172.21.15.100:0/3806784748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556882b23db0 0x556882b326b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:01.911 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.908+0000 7ff9cb7fe640 1 -- 172.21.15.100:0/3806784748 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556882b29910 msgr2=0x556882b33c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:01.912 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.908+0000 7ff9cb7fe640 1 --2- 172.21.15.100:0/3806784748 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556882b29910 0x556882b33c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:01.912 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.908+0000 7ff9cb7fe640 1 -- 172.21.15.100:0/3806784748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556882b23db0 msgr2=0x556882b326b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:01.912 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.908+0000 7ff9cb7fe640 1 --2- 172.21.15.100:0/3806784748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556882b23db0 0x556882b326b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:01.912 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.908+0000 7ff9cb7fe640 1 -- 172.21.15.100:0/3806784748 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556882b25470 con 0x556882b247a0 2024-05-10T11:40:01.912 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.908+0000 7ff9cb7fe640 1 --2- 172.21.15.100:0/3806784748 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556882b247a0 0x556882b32bf0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc02a860 tx=0x7ff9bc000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e571a2c97935b41 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:40:01.912 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.908+0000 7ff9ab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:40:01.912 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.908+0000 7ff9ab7fe640 1 -- 172.21.15.100:0/3806784748 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc037040 con 0x556882b247a0 2024-05-10T11:40:01.912 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.908+0000 7ff9d5462740 1 -- 172.21.15.100:0/3806784748 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9bc0306b0 con 0x556882b247a0 2024-05-10T11:40:01.912 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.909+0000 7ff9ab7fe640 1 -- 172.21.15.100:0/3806784748 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7ff9bc030ab0 con 0x556882b247a0 2024-05-10T11:40:01.912 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.909+0000 7ff9ab7fe640 1 -- 172.21.15.100:0/3806784748 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7ff9bc04e040 con 0x556882b247a0 2024-05-10T11:40:01.912 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.909+0000 7ff9d5462740 10 client.29035 fetch_fsmap learned FSMap version 41 2024-05-10T11:40:01.912 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.909+0000 7ff9d5462740 1 -- 172.21.15.100:0/3806784748 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x556882ba47c0 con 0x556882b247a0 2024-05-10T11:40:01.913 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.909+0000 7ff9ab7fe640 1 -- 172.21.15.100:0/3806784748 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff9bc0306b0 con 0x556882b247a0 2024-05-10T11:40:01.913 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.909+0000 7ff9ab7fe640 1 -- 172.21.15.100:0/3806784748 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7ff9bc0048c0 con 0x556882b247a0 2024-05-10T11:40:01.913 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.909+0000 7ff9d5462740 10 client.29035 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:40:01.914 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.909+0000 7ff9d5462740 10 client.29035 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:40:01.914 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.909+0000 7ff9d5462740 20 client.29035 populate_metadata read hostname 'smithi100' 2024-05-10T11:40:01.914 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.909+0000 7ff9d5462740 1 --2- 172.21.15.100:0/3806784748 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556882ba4dc0 0x556882ba71b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:01.914 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.909+0000 7ff9d5462740 4 client.29035 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:40:01.914 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.909+0000 7ff9d5462740 1 -- 172.21.15.100:0/3806784748 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556882a90050 con 0x556882ba4dc0 2024-05-10T11:40:01.914 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.910+0000 7ff9cbfff640 1 --2- 172.21.15.100:0/3806784748 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556882ba4dc0 0x556882ba71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:01.914 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.910+0000 7ff9ab7fe640 10 client.29035 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:40:01.914 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.910+0000 7ff9cbfff640 1 --2- 172.21.15.100:0/3806784748 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556882ba4dc0 0x556882ba71b0 crc :-1 s=READY pgs=412 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f839b1dc46b50ffe server_cookie=988348fc444eb679 in_seq=0 out_seq=0 2024-05-10T11:40:01.949 INFO:tasks.workunit.client.0.smithi100.stdout: 10 79222 8.92 MB/sec execute 355 sec latency 64.122 ms 2024-05-10T11:40:01.999 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.998+0000 7ff9ab7fe640 1 -- 172.21.15.100:0/3806784748 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556882a90050 con 0x556882ba4dc0 2024-05-10T11:40:01.999 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.998+0000 7ff9ab7fe640 10 client.29035 handle_command_reply: tid=0 2024-05-10T11:40:01.999 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.998+0000 7ff9d5462740 1 client.29035 shutdown 2024-05-10T11:40:02.001 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.999+0000 7ff9d5462740 1 -- 172.21.15.100:0/3806784748 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556882b247a0 msgr2=0x556882b32bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:02.001 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.999+0000 7ff9d5462740 1 --2- 172.21.15.100:0/3806784748 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556882b247a0 0x556882b32bf0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc02a860 tx=0x7ff9bc000bc0 comp rx=0 tx=0).stop 2024-05-10T11:40:02.001 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.999+0000 7ff9d5462740 1 -- 172.21.15.100:0/3806784748 shutdown_connections 2024-05-10T11:40:02.001 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.999+0000 7ff9d5462740 1 --2- 172.21.15.100:0/3806784748 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556882ba4dc0 0x556882ba71b0 crc :-1 s=READY pgs=412 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:02.001 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.999+0000 7ff9d5462740 1 --2- 172.21.15.100:0/3806784748 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556882b29910 0x556882b33c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:02.001 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.999+0000 7ff9d5462740 1 --2- 172.21.15.100:0/3806784748 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556882b247a0 0x556882b32bf0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:02.001 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.999+0000 7ff9ab7fe640 0 client.29035 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:40:02.001 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.999+0000 7ff9d5462740 1 --2- 172.21.15.100:0/3806784748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556882b23db0 0x556882b326b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:02.001 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.999+0000 7ff9d5462740 1 -- 172.21.15.100:0/3806784748 >> 172.21.15.100:0/3806784748 conn(0x556882b1f420 msgr2=0x556882b22330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:40:02.001 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.999+0000 7ff9d5462740 1 -- 172.21.15.100:0/3806784748 shutdown_connections 2024-05-10T11:40:02.001 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:01.999+0000 7ff9d5462740 1 -- 172.21.15.100:0/3806784748 wait complete. 2024-05-10T11:40:02.002 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:02.000+0000 7ff9d5462740 20 client.29035 trim_cache size 0 max 16384 2024-05-10T11:40:02.002 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:40:02.002 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:40:02.002 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:40:02.002 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:40:02.003 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:02.000+0000 7ff9d274d640 1 -- 172.21.15.100:0/3725875161 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff9b80b2920 msgr2=0x7ff9b80b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:02.003 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:02.000+0000 7ff9d274d640 1 --2- 172.21.15.100:0/3725875161 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff9b80b2920 0x7ff9b80b4de0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x556882b25000 tx=0x7ff9c4010040 comp rx=0 tx=0).stop 2024-05-10T11:40:02.003 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:02.000+0000 7ff9d274d640 1 -- 172.21.15.100:0/3725875161 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9cc1099b0 msgr2=0x7ff9cc1a78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:02.003 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:02.000+0000 7ff9d274d640 1 --2- 172.21.15.100:0/3725875161 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9cc1099b0 0x7ff9cc1a78a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff9b400e9f0 tx=0x7ff9b400eeb0 comp rx=0 tx=0).stop 2024-05-10T11:40:02.003 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:02.001+0000 7ff9d274d640 1 -- 172.21.15.100:0/3725875161 shutdown_connections 2024-05-10T11:40:02.003 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:02.001+0000 7ff9d274d640 1 --2- 172.21.15.100:0/3725875161 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7ff9b80b2920 0x7ff9b80b4de0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:02.003 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:02.001+0000 7ff9d274d640 1 --2- 172.21.15.100:0/3725875161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9cc10ecd0 0x7ff9cc1aee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:02.003 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:02.001+0000 7ff9d274d640 1 --2- 172.21.15.100:0/3725875161 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9cc10c320 0x7ff9cc1a7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:02.003 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:02.001+0000 7ff9d274d640 1 --2- 172.21.15.100:0/3725875161 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9cc1099b0 0x7ff9cc1a78a0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:02.003 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:02.001+0000 7ff9d274d640 1 -- 172.21.15.100:0/3725875161 >> 172.21.15.100:0/3725875161 conn(0x7ff9cc06fa20 msgr2=0x7ff9cc109dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:40:02.003 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:02.001+0000 7ff9d274d640 1 -- 172.21.15.100:0/3725875161 shutdown_connections 2024-05-10T11:40:02.003 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:02.001+0000 7ff9d274d640 1 -- 172.21.15.100:0/3725875161 wait complete. 2024-05-10T11:40:02.004 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:40:01 smithi178 ceph-mon[33359]: pgmap v1070: 129 pgs: 129 active+clean; 541 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1019 KiB/s rd, 4.5 MiB/s wr, 345 op/s 2024-05-10T11:40:02.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:40:01 smithi188 ceph-mon[34075]: pgmap v1070: 129 pgs: 129 active+clean; 541 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1019 KiB/s rd, 4.5 MiB/s wr, 345 op/s 2024-05-10T11:40:02.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:40:01 smithi100 ceph-mon[30296]: pgmap v1070: 129 pgs: 129 active+clean; 541 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1019 KiB/s rd, 4.5 MiB/s wr, 345 op/s 2024-05-10T11:40:02.949 INFO:tasks.workunit.client.0.smithi100.stdout: 10 79380 8.91 MB/sec execute 356 sec latency 117.088 ms 2024-05-10T11:40:03.950 INFO:tasks.workunit.client.0.smithi100.stdout: 10 79568 8.92 MB/sec execute 357 sec latency 67.612 ms 2024-05-10T11:40:04.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:40:03 smithi178 ceph-mon[33359]: pgmap v1071: 129 pgs: 129 active+clean; 540 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 3.6 MiB/s wr, 278 op/s 2024-05-10T11:40:04.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:40:03 smithi188 ceph-mon[34075]: pgmap v1071: 129 pgs: 129 active+clean; 540 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 3.6 MiB/s wr, 278 op/s 2024-05-10T11:40:04.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:40:03 smithi100 ceph-mon[30296]: pgmap v1071: 129 pgs: 129 active+clean; 540 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 3.6 MiB/s wr, 278 op/s 2024-05-10T11:40:04.950 INFO:tasks.workunit.client.0.smithi100.stdout: 10 79791 8.95 MB/sec execute 358 sec latency 61.669 ms 2024-05-10T11:40:05.950 INFO:tasks.workunit.client.0.smithi100.stdout: 10 80013 8.97 MB/sec execute 359 sec latency 65.678 ms 2024-05-10T11:40:06.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:40:05 smithi178 ceph-mon[33359]: pgmap v1072: 129 pgs: 129 active+clean; 543 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 3.9 MiB/s wr, 273 op/s 2024-05-10T11:40:06.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:40:05 smithi188 ceph-mon[34075]: pgmap v1072: 129 pgs: 129 active+clean; 543 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 3.9 MiB/s wr, 273 op/s 2024-05-10T11:40:06.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:40:05 smithi100 ceph-mon[30296]: pgmap v1072: 129 pgs: 129 active+clean; 543 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 3.9 MiB/s wr, 273 op/s 2024-05-10T11:40:06.952 INFO:tasks.workunit.client.0.smithi100.stdout: 10 80231 8.98 MB/sec execute 360 sec latency 101.887 ms 2024-05-10T11:40:07.950 INFO:tasks.workunit.client.0.smithi100.stdout: 10 80396 8.99 MB/sec execute 361 sec latency 57.343 ms 2024-05-10T11:40:08.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:40:07 smithi178 ceph-mon[33359]: pgmap v1073: 129 pgs: 129 active+clean; 548 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 879 KiB/s rd, 4.4 MiB/s wr, 326 op/s 2024-05-10T11:40:08.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:40:07 smithi188 ceph-mon[34075]: pgmap v1073: 129 pgs: 129 active+clean; 548 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 879 KiB/s rd, 4.4 MiB/s wr, 326 op/s 2024-05-10T11:40:08.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:40:07 smithi100 ceph-mon[30296]: pgmap v1073: 129 pgs: 129 active+clean; 548 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 879 KiB/s rd, 4.4 MiB/s wr, 326 op/s 2024-05-10T11:40:08.950 INFO:tasks.workunit.client.0.smithi100.stdout: 10 80568 9.00 MB/sec execute 362 sec latency 65.960 ms 2024-05-10T11:40:09.950 INFO:tasks.workunit.client.0.smithi100.stdout: 10 80716 9.00 MB/sec execute 363 sec latency 168.937 ms 2024-05-10T11:40:10.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:40:09 smithi178 ceph-mon[33359]: pgmap v1074: 129 pgs: 129 active+clean; 552 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 673 KiB/s rd, 4.2 MiB/s wr, 286 op/s 2024-05-10T11:40:10.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:40:09 smithi188 ceph-mon[34075]: pgmap v1074: 129 pgs: 129 active+clean; 552 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 673 KiB/s rd, 4.2 MiB/s wr, 286 op/s 2024-05-10T11:40:10.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:40:09 smithi100 ceph-mon[30296]: pgmap v1074: 129 pgs: 129 active+clean; 552 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 673 KiB/s rd, 4.2 MiB/s wr, 286 op/s 2024-05-10T11:40:10.950 INFO:tasks.workunit.client.0.smithi100.stdout: 10 80880 9.00 MB/sec execute 364 sec latency 57.299 ms 2024-05-10T11:40:11.950 INFO:tasks.workunit.client.0.smithi100.stdout: 10 81028 8.98 MB/sec execute 365 sec latency 81.647 ms 2024-05-10T11:40:12.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:40:11 smithi178 ceph-mon[33359]: pgmap v1075: 129 pgs: 129 active+clean; 555 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 4.9 MiB/s wr, 309 op/s 2024-05-10T11:40:12.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:40:11 smithi188 ceph-mon[34075]: pgmap v1075: 129 pgs: 129 active+clean; 555 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 4.9 MiB/s wr, 309 op/s 2024-05-10T11:40:12.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:40:11 smithi100 ceph-mon[30296]: pgmap v1075: 129 pgs: 129 active+clean; 555 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 4.9 MiB/s wr, 309 op/s 2024-05-10T11:40:12.950 INFO:tasks.workunit.client.0.smithi100.stdout: 10 81181 8.97 MB/sec execute 366 sec latency 71.729 ms 2024-05-10T11:40:13.950 INFO:tasks.workunit.client.0.smithi100.stdout: 10 81328 8.95 MB/sec execute 367 sec latency 123.124 ms 2024-05-10T11:40:14.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:40:13 smithi178 ceph-mon[33359]: pgmap v1076: 129 pgs: 129 active+clean; 557 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 836 KiB/s rd, 4.8 MiB/s wr, 306 op/s 2024-05-10T11:40:14.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:40:13 smithi188 ceph-mon[34075]: pgmap v1076: 129 pgs: 129 active+clean; 557 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 836 KiB/s rd, 4.8 MiB/s wr, 306 op/s 2024-05-10T11:40:14.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:40:13 smithi100 ceph-mon[30296]: pgmap v1076: 129 pgs: 129 active+clean; 557 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 836 KiB/s rd, 4.8 MiB/s wr, 306 op/s 2024-05-10T11:40:14.950 INFO:tasks.workunit.client.0.smithi100.stdout: 10 81494 8.95 MB/sec execute 368 sec latency 92.966 ms 2024-05-10T11:40:15.129 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.124+0000 7fa39db58640 1 -- 172.21.15.100:0/1444205488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa398103bf0 msgr2=0x7fa398106060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:15.129 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.124+0000 7fa39db58640 1 --2- 172.21.15.100:0/1444205488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa398103bf0 0x7fa398106060 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fa38c009900 tx=0x7fa38c02a410 comp rx=0 tx=0).stop 2024-05-10T11:40:15.129 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.124+0000 7fa39db58640 1 -- 172.21.15.100:0/1444205488 shutdown_connections 2024-05-10T11:40:15.130 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.124+0000 7fa39db58640 1 --2- 172.21.15.100:0/1444205488 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3981065a0 0x7fa398106a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:15.130 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.124+0000 7fa39db58640 1 --2- 172.21.15.100:0/1444205488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa398103bf0 0x7fa398106060 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:15.130 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.124+0000 7fa39db58640 1 --2- 172.21.15.100:0/1444205488 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa398101280 0x7fa3981036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:15.130 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.124+0000 7fa39db58640 1 -- 172.21.15.100:0/1444205488 >> 172.21.15.100:0/1444205488 conn(0x7fa3980fce80 msgr2=0x7fa3980ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:40:15.130 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.124+0000 7fa39db58640 1 -- 172.21.15.100:0/1444205488 shutdown_connections 2024-05-10T11:40:15.131 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.124+0000 7fa39db58640 1 -- 172.21.15.100:0/1444205488 wait complete. 2024-05-10T11:40:15.131 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.125+0000 7fa39db58640 1 Processor -- start 2024-05-10T11:40:15.131 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.125+0000 7fa39db58640 1 -- start start 2024-05-10T11:40:15.131 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.126+0000 7fa39db58640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa398101280 0x7fa39819f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:15.131 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.126+0000 7fa39db58640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa398103bf0 0x7fa39819f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:15.131 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.126+0000 7fa39db58640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3981065a0 0x7fa3981a6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:15.132 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.126+0000 7fa39db58640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa398114a20 con 0x7fa3981065a0 2024-05-10T11:40:15.132 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.126+0000 7fa39db58640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa3981148a0 con 0x7fa398101280 2024-05-10T11:40:15.132 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.126+0000 7fa39db58640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa398114ba0 con 0x7fa398103bf0 2024-05-10T11:40:15.132 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.126+0000 7fa396ffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa398103bf0 0x7fa39819f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:15.132 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.126+0000 7fa396ffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa398103bf0 0x7fa39819f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:37698/0 (socket says 172.21.15.100:37698) 2024-05-10T11:40:15.133 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.126+0000 7fa397fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3981065a0 0x7fa3981a6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:15.133 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.126+0000 7fa396ffd640 1 -- 172.21.15.100:0/1737870957 learned_addr learned my addr 172.21.15.100:0/1737870957 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:40:15.133 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.126+0000 7fa3977fe640 1 --2- 172.21.15.100:0/1737870957 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa398101280 0x7fa39819f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:15.133 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.126+0000 7fa396ffd640 1 -- 172.21.15.100:0/1737870957 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa398101280 msgr2=0x7fa39819f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:15.133 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.126+0000 7fa396ffd640 1 --2- 172.21.15.100:0/1737870957 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa398101280 0x7fa39819f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:15.133 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.126+0000 7fa396ffd640 1 -- 172.21.15.100:0/1737870957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3981065a0 msgr2=0x7fa3981a6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:15.134 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.126+0000 7fa396ffd640 1 --2- 172.21.15.100:0/1737870957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3981065a0 0x7fa3981a6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:15.134 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.126+0000 7fa396ffd640 1 -- 172.21.15.100:0/1737870957 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3981a6cc0 con 0x7fa398103bf0 2024-05-10T11:40:15.134 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.127+0000 7fa396ffd640 1 --2- 172.21.15.100:0/1737870957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa398103bf0 0x7fa39819f5e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa38c031460 tx=0x7fa38c003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:40:15.134 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.127+0000 7fa394ff9640 1 -- 172.21.15.100:0/1737870957 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa38c03f070 con 0x7fa398103bf0 2024-05-10T11:40:15.135 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.127+0000 7fa39db58640 1 -- 172.21.15.100:0/1737870957 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3981a6f50 con 0x7fa398103bf0 2024-05-10T11:40:15.135 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.127+0000 7fa394ff9640 1 -- 172.21.15.100:0/1737870957 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa38c031ba0 con 0x7fa398103bf0 2024-05-10T11:40:15.135 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.127+0000 7fa394ff9640 1 -- 172.21.15.100:0/1737870957 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa38c035030 con 0x7fa398103bf0 2024-05-10T11:40:15.135 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.127+0000 7fa39db58640 1 -- 172.21.15.100:0/1737870957 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3981a73c0 con 0x7fa398103bf0 2024-05-10T11:40:15.144 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.131+0000 7fa3a0873740 1 Processor -- start 2024-05-10T11:40:15.144 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.131+0000 7fa394ff9640 1 -- 172.21.15.100:0/1737870957 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fa38c042070 con 0x7fa398103bf0 2024-05-10T11:40:15.144 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.131+0000 7fa3a0873740 1 -- start start 2024-05-10T11:40:15.144 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.131+0000 7fa3a0873740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cdd1e5db0 0x558cdd1e61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:15.144 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.131+0000 7fa3a0873740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558cdd1e67a0 0x558cdd1eb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:15.144 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.131+0000 7fa3a0873740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558cdd1eb910 0x558cdd1edda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:15.145 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.131+0000 7fa3a0873740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x558cdd14a5d0 con 0x558cdd1eb910 2024-05-10T11:40:15.145 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.131+0000 7fa3a0873740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558cdd103890 con 0x558cdd1e67a0 2024-05-10T11:40:15.145 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.131+0000 7fa3a0873740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558cdd146590 con 0x558cdd1e5db0 2024-05-10T11:40:15.145 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.131+0000 7fa3977fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558cdd1e67a0 0x558cdd1eb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:15.145 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.131+0000 7fa3977fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558cdd1e67a0 0x558cdd1eb2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:46162/0 (socket says 172.21.15.100:46162) 2024-05-10T11:40:15.145 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.131+0000 7fa397fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cdd1e5db0 0x558cdd1e61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:15.145 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.131+0000 7fa3977fe640 1 -- 172.21.15.100:0/507374970 learned_addr learned my addr 172.21.15.100:0/507374970 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:40:15.145 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.131+0000 7fa396ffd640 1 --2- 172.21.15.100:0/507374970 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558cdd1eb910 0x558cdd1edda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:15.145 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.132+0000 7fa397fff640 1 -- 172.21.15.100:0/507374970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558cdd1e67a0 msgr2=0x558cdd1eb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:15.145 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.132+0000 7fa397fff640 1 --2- 172.21.15.100:0/507374970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558cdd1e67a0 0x558cdd1eb2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:15.145 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.132+0000 7fa397fff640 1 -- 172.21.15.100:0/507374970 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558cdd1eb910 msgr2=0x558cdd1edda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:15.145 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.132+0000 7fa397fff640 1 --2- 172.21.15.100:0/507374970 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558cdd1eb910 0x558cdd1edda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:15.146 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.132+0000 7fa397fff640 1 -- 172.21.15.100:0/507374970 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cdce5c210 con 0x558cdd1e5db0 2024-05-10T11:40:15.146 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.132+0000 7fa397fff640 1 --2- 172.21.15.100:0/507374970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cdd1e5db0 0x558cdd1e61d0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa390009f00 tx=0x7fa390011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e80bafb60e4eacc4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:40:15.146 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.132+0000 7fa37dffb640 1 -- 172.21.15.100:0/507374970 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa390003c30 con 0x558cdd1e5db0 2024-05-10T11:40:15.146 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.132+0000 7fa37dffb640 1 -- 172.21.15.100:0/507374970 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa390003dd0 con 0x558cdd1e5db0 2024-05-10T11:40:15.146 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.132+0000 7fa37dffb640 1 -- 172.21.15.100:0/507374970 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa39000f800 con 0x558cdd1e5db0 2024-05-10T11:40:15.146 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.132+0000 7fa394ff9640 1 --2- 172.21.15.100:0/1737870957 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa3800b2920 0x7fa3800b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:15.146 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.132+0000 7fa3977fe640 1 --2- 172.21.15.100:0/1737870957 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa3800b2920 0x7fa3800b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:15.146 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.132+0000 7fa394ff9640 1 -- 172.21.15.100:0/1737870957 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fa38c102cb0 con 0x7fa398103bf0 2024-05-10T11:40:15.146 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.133+0000 7fa3a0873740 1 -- 172.21.15.100:0/507374970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cdd1e5db0 msgr2=0x558cdd1e61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:15.146 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.133+0000 7fa3977fe640 1 --2- 172.21.15.100:0/1737870957 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa3800b2920 0x7fa3800b4de0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x558cdd1e6ba0 tx=0x7fa388005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:40:15.146 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.133+0000 7fa3a0873740 1 --2- 172.21.15.100:0/507374970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cdd1e5db0 0x558cdd1e61d0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa390009f00 tx=0x7fa390011810 comp rx=0 tx=0).stop 2024-05-10T11:40:15.146 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.133+0000 7fa3a0873740 1 -- 172.21.15.100:0/507374970 shutdown_connections 2024-05-10T11:40:15.146 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.133+0000 7fa3a0873740 1 --2- 172.21.15.100:0/507374970 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558cdd1eb910 0x558cdd1edda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:15.146 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.133+0000 7fa3a0873740 1 --2- 172.21.15.100:0/507374970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558cdd1e67a0 0x558cdd1eb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:15.147 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.133+0000 7fa3a0873740 1 --2- 172.21.15.100:0/507374970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cdd1e5db0 0x558cdd1e61d0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:15.147 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.134+0000 7fa3a0873740 1 -- 172.21.15.100:0/507374970 >> 172.21.15.100:0/507374970 conn(0x558cdd1e1420 msgr2=0x558cdd1e1820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:40:15.147 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.134+0000 7fa3a0873740 1 -- 172.21.15.100:0/507374970 shutdown_connections 2024-05-10T11:40:15.147 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.134+0000 7fa3a0873740 1 -- 172.21.15.100:0/507374970 wait complete. 2024-05-10T11:40:15.147 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.134+0000 7fa3a0873740 1 -- start start 2024-05-10T11:40:15.147 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.134+0000 7fa3a0873740 1 Processor -- start 2024-05-10T11:40:15.147 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.135+0000 7fa3a0873740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cdd1e5db0 0x558cdd1f0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:15.147 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.135+0000 7fa3a0873740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558cdd1e67a0 0x558cdd1f05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:15.147 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.135+0000 7fa3a0873740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558cdd1eb910 0x558cdd1f75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:15.147 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.135+0000 7fa3a0873740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x558cdd1ee8c0 con 0x558cdd1eb910 2024-05-10T11:40:15.147 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.135+0000 7fa3a0873740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558cdd107370 con 0x558cdd1e67a0 2024-05-10T11:40:15.147 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.135+0000 7fa3a0873740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558cdd1eea40 con 0x558cdd1e5db0 2024-05-10T11:40:15.147 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.135+0000 7fa3a0873740 10 client.0 _ll_register_callbacks cb 0x558cdce5ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:40:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.135+0000 7fa3977fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cdd1e5db0 0x558cdd1f0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.135+0000 7fa396ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558cdd1e67a0 0x558cdd1f05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.135+0000 7fa3977fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cdd1e5db0 0x558cdd1f0060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:37718/0 (socket says 172.21.15.100:37718) 2024-05-10T11:40:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.135+0000 7fa397fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558cdd1eb910 0x558cdd1f75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.135+0000 7fa3977fe640 1 -- 172.21.15.100:0/315494093 learned_addr learned my addr 172.21.15.100:0/315494093 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:40:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.135+0000 7fa3977fe640 1 -- 172.21.15.100:0/315494093 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558cdd1e67a0 msgr2=0x558cdd1f05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.135+0000 7fa3977fe640 1 --2- 172.21.15.100:0/315494093 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558cdd1e67a0 0x558cdd1f05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.135+0000 7fa3977fe640 1 -- 172.21.15.100:0/315494093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558cdd1eb910 msgr2=0x558cdd1f75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.135+0000 7fa3977fe640 1 --2- 172.21.15.100:0/315494093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558cdd1eb910 0x558cdd1f75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.135+0000 7fa3977fe640 1 -- 172.21.15.100:0/315494093 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cdcf28620 con 0x558cdd1e5db0 2024-05-10T11:40:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.135+0000 7fa397fff640 1 --2- 172.21.15.100:0/315494093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558cdd1eb910 0x558cdd1f75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:40:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.136+0000 7fa3977fe640 1 --2- 172.21.15.100:0/315494093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cdd1e5db0 0x558cdd1f0060 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa38801ea00 tx=0x7fa388004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6f015b17c8386c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:40:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.136+0000 7fa376ffd640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:40:15.149 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.136+0000 7fa376ffd640 1 -- 172.21.15.100:0/315494093 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa38802e060 con 0x558cdd1e5db0 2024-05-10T11:40:15.149 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.136+0000 7fa3a0873740 1 -- 172.21.15.100:0/315494093 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa390003dd0 con 0x558cdd1e5db0 2024-05-10T11:40:15.149 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.136+0000 7fa376ffd640 1 -- 172.21.15.100:0/315494093 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fa388018030 con 0x558cdd1e5db0 2024-05-10T11:40:15.149 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.136+0000 7fa376ffd640 1 -- 172.21.15.100:0/315494093 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fa388035050 con 0x558cdd1e5db0 2024-05-10T11:40:15.149 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.136+0000 7fa3a0873740 10 client.29047 fetch_fsmap learned FSMap version 41 2024-05-10T11:40:15.149 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.136+0000 7fa3a0873740 1 -- 172.21.15.100:0/315494093 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x558cdd1fa6e0 con 0x558cdd1e5db0 2024-05-10T11:40:15.149 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.136+0000 7fa376ffd640 1 -- 172.21.15.100:0/315494093 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa390003dd0 con 0x558cdd1e5db0 2024-05-10T11:40:15.149 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.136+0000 7fa376ffd640 1 -- 172.21.15.100:0/315494093 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fa38800f3c0 con 0x558cdd1e5db0 2024-05-10T11:40:15.149 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.137+0000 7fa3a0873740 10 client.29047 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:40:15.149 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.137+0000 7fa3a0873740 10 client.29047 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:40:15.149 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.137+0000 7fa3a0873740 20 client.29047 populate_metadata read hostname 'smithi100' 2024-05-10T11:40:15.150 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.137+0000 7fa3a0873740 1 --2- 172.21.15.100:0/315494093 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558cdd1fbdb0 0x558cdd1fe1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:15.150 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.137+0000 7fa3a0873740 4 client.29047 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:40:15.150 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.137+0000 7fa3a0873740 1 -- 172.21.15.100:0/315494093 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558cdd1581e0 con 0x558cdd1fbdb0 2024-05-10T11:40:15.150 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.137+0000 7fa396ffd640 1 --2- 172.21.15.100:0/315494093 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558cdd1fbdb0 0x558cdd1fe1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:15.150 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.137+0000 7fa396ffd640 1 --2- 172.21.15.100:0/315494093 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558cdd1fbdb0 0x558cdd1fe1e0 crc :-1 s=READY pgs=413 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2312f0ee54c41612 server_cookie=de92315f258998e6 in_seq=0 out_seq=0 2024-05-10T11:40:15.150 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.137+0000 7fa376ffd640 10 client.29047 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:40:15.150 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.141+0000 7fa376ffd640 1 -- 172.21.15.100:0/315494093 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558cdd1581e0 con 0x558cdd1fbdb0 2024-05-10T11:40:15.150 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.141+0000 7fa376ffd640 10 client.29047 handle_command_reply: tid=0 2024-05-10T11:40:15.150 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.141+0000 7fa3a0873740 1 client.29047 shutdown 2024-05-10T11:40:15.150 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.141+0000 7fa3a0873740 1 -- 172.21.15.100:0/315494093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cdd1e5db0 msgr2=0x558cdd1f0060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:15.150 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.141+0000 7fa3a0873740 1 --2- 172.21.15.100:0/315494093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cdd1e5db0 0x558cdd1f0060 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa38801ea00 tx=0x7fa388004730 comp rx=0 tx=0).stop 2024-05-10T11:40:15.150 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.141+0000 7fa3a0873740 1 -- 172.21.15.100:0/315494093 shutdown_connections 2024-05-10T11:40:15.150 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.141+0000 7fa3a0873740 1 --2- 172.21.15.100:0/315494093 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558cdd1fbdb0 0x558cdd1fe1e0 crc :-1 s=READY pgs=413 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.141+0000 7fa3a0873740 1 --2- 172.21.15.100:0/315494093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558cdd1eb910 0x558cdd1f75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.141+0000 7fa3a0873740 1 --2- 172.21.15.100:0/315494093 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558cdd1e67a0 0x558cdd1f05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.141+0000 7fa3a0873740 1 --2- 172.21.15.100:0/315494093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cdd1e5db0 0x558cdd1f0060 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.141+0000 7fa3a0873740 1 -- 172.21.15.100:0/315494093 >> 172.21.15.100:0/315494093 conn(0x558cdd1e1420 msgr2=0x558cdd1e0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:40:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.141+0000 7fa376ffd640 0 client.29047 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:40:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.141+0000 7fa3a0873740 1 -- 172.21.15.100:0/315494093 shutdown_connections 2024-05-10T11:40:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.142+0000 7fa3a0873740 1 -- 172.21.15.100:0/315494093 wait complete. 2024-05-10T11:40:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.142+0000 7fa3a0873740 20 client.29047 trim_cache size 0 max 16384 2024-05-10T11:40:15.170 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.168+0000 7fa3a0873740 1 Processor -- start 2024-05-10T11:40:15.170 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.168+0000 7fa3a0873740 1 -- start start 2024-05-10T11:40:15.170 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.168+0000 7fa3a0873740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558cdd1e5db0 0x558cdd1f0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:15.170 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.168+0000 7fa3a0873740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cdd1e67a0 0x558cdd1f7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:15.170 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.168+0000 7fa3a0873740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558cdd1eb910 0x558cdd1e4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:15.170 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.168+0000 7fa3a0873740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa3981b4150 con 0x558cdd1eb910 2024-05-10T11:40:15.170 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.168+0000 7fa3a0873740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa3981b1a20 con 0x558cdd1e5db0 2024-05-10T11:40:15.171 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.168+0000 7fa3a0873740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa3981af870 con 0x558cdd1e67a0 2024-05-10T11:40:15.171 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.168+0000 7fa397fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558cdd1eb910 0x558cdd1e4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:15.171 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.168+0000 7fa397fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558cdd1eb910 0x558cdd1e4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34142/0 (socket says 172.21.15.100:34142) 2024-05-10T11:40:15.171 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.168+0000 7fa397fff640 1 -- 172.21.15.100:0/602764588 learned_addr learned my addr 172.21.15.100:0/602764588 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:40:15.171 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.169+0000 7fa397fff640 1 -- 172.21.15.100:0/602764588 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cdd1e67a0 msgr2=0x558cdd1f7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:15.171 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.169+0000 7fa397fff640 1 --2- 172.21.15.100:0/602764588 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cdd1e67a0 0x558cdd1f7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:15.171 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.169+0000 7fa397fff640 1 -- 172.21.15.100:0/602764588 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558cdd1e5db0 msgr2=0x558cdd1f0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:15.171 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.169+0000 7fa397fff640 1 --2- 172.21.15.100:0/602764588 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558cdd1e5db0 0x558cdd1f0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:15.171 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.169+0000 7fa397fff640 1 -- 172.21.15.100:0/602764588 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cdd1fa500 con 0x558cdd1eb910 2024-05-10T11:40:15.172 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.169+0000 7fa397fff640 1 --2- 172.21.15.100:0/602764588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558cdd1eb910 0x558cdd1e4810 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fa390009f00 tx=0x7fa390009d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9eda07c77d3791b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:40:15.173 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.170+0000 7fa37d7fa640 1 -- 172.21.15.100:0/602764588 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa39003b020 con 0x558cdd1eb910 2024-05-10T11:40:15.173 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.171+0000 7fa37d7fa640 1 -- 172.21.15.100:0/602764588 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa3900056d0 con 0x558cdd1eb910 2024-05-10T11:40:15.173 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.171+0000 7fa37d7fa640 1 -- 172.21.15.100:0/602764588 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa39001f040 con 0x558cdd1eb910 2024-05-10T11:40:15.173 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.172+0000 7fa3a0873740 1 -- 172.21.15.100:0/602764588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558cdd1eb910 msgr2=0x558cdd1e4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:15.174 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.172+0000 7fa3a0873740 1 --2- 172.21.15.100:0/602764588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558cdd1eb910 0x558cdd1e4810 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fa390009f00 tx=0x7fa390009d00 comp rx=0 tx=0).stop 2024-05-10T11:40:15.174 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.172+0000 7fa3a0873740 1 -- 172.21.15.100:0/602764588 shutdown_connections 2024-05-10T11:40:15.174 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.172+0000 7fa3a0873740 1 --2- 172.21.15.100:0/602764588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558cdd1eb910 0x558cdd1e4810 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:15.174 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.172+0000 7fa3a0873740 1 --2- 172.21.15.100:0/602764588 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cdd1e67a0 0x558cdd1f7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:15.174 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.172+0000 7fa3a0873740 1 --2- 172.21.15.100:0/602764588 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558cdd1e5db0 0x558cdd1f0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:15.174 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.172+0000 7fa3a0873740 1 -- 172.21.15.100:0/602764588 >> 172.21.15.100:0/602764588 conn(0x558cdd1e1420 msgr2=0x558cdd1e4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:40:15.174 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.172+0000 7fa3a0873740 1 -- 172.21.15.100:0/602764588 shutdown_connections 2024-05-10T11:40:15.174 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.173+0000 7fa3a0873740 1 -- 172.21.15.100:0/602764588 wait complete. 2024-05-10T11:40:15.174 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.173+0000 7fa3a0873740 1 -- start start 2024-05-10T11:40:15.174 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.174+0000 7fa3a0873740 1 Processor -- start 2024-05-10T11:40:15.175 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.175+0000 7fa3a0873740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558cdd1e5db0 0x558cdd1f4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:15.175 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.175+0000 7fa3a0873740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558cdd1e67a0 0x558cdd1f4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:15.175 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.175+0000 7fa3a0873740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cdd1eb910 0x558cdd1f5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:15.175 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.175+0000 7fa3a0873740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa3981afee0 con 0x558cdd1e67a0 2024-05-10T11:40:15.175 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.175+0000 7fa3a0873740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa3981b1350 con 0x558cdd1e5db0 2024-05-10T11:40:15.175 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.175+0000 7fa3a0873740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa3981b05b0 con 0x558cdd1eb910 2024-05-10T11:40:15.176 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.175+0000 7fa3a0873740 10 client.0 _ll_register_callbacks cb 0x7fa388012f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:40:15.176 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.175+0000 7fa397fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cdd1eb910 0x558cdd1f5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:15.176 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.175+0000 7fa396ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558cdd1e67a0 0x558cdd1f4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:15.176 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.175+0000 7fa397fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cdd1eb910 0x558cdd1f5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:37742/0 (socket says 172.21.15.100:37742) 2024-05-10T11:40:15.176 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.175+0000 7fa396ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558cdd1e67a0 0x558cdd1f4ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34150/0 (socket says 172.21.15.100:34150) 2024-05-10T11:40:15.176 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.175+0000 7fa397fff640 1 -- 172.21.15.100:0/3128197552 learned_addr learned my addr 172.21.15.100:0/3128197552 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:40:15.177 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.176+0000 7fa397fff640 1 -- 172.21.15.100:0/3128197552 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558cdd1e5db0 msgr2=0x558cdd1f4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:15.177 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.176+0000 7fa397fff640 1 --2- 172.21.15.100:0/3128197552 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558cdd1e5db0 0x558cdd1f4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:15.177 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.176+0000 7fa397fff640 1 -- 172.21.15.100:0/3128197552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558cdd1e67a0 msgr2=0x558cdd1f4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:15.178 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.176+0000 7fa397fff640 1 --2- 172.21.15.100:0/3128197552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558cdd1e67a0 0x558cdd1f4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:15.178 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.176+0000 7fa397fff640 1 -- 172.21.15.100:0/3128197552 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cdd1f6480 con 0x558cdd1eb910 2024-05-10T11:40:15.178 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.176+0000 7fa397fff640 1 --2- 172.21.15.100:0/3128197552 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cdd1eb910 0x558cdd1f5e90 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa390005f40 tx=0x7fa390005910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5937788f2cf76a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:40:15.178 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.176+0000 7fa376ffd640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:40:15.178 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.177+0000 7fa376ffd640 1 -- 172.21.15.100:0/3128197552 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa39003b030 con 0x558cdd1eb910 2024-05-10T11:40:15.178 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.177+0000 7fa3a0873740 1 -- 172.21.15.100:0/3128197552 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3900056d0 con 0x558cdd1eb910 2024-05-10T11:40:15.178 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.177+0000 7fa376ffd640 1 -- 172.21.15.100:0/3128197552 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fa390005270 con 0x558cdd1eb910 2024-05-10T11:40:15.178 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.177+0000 7fa376ffd640 1 -- 172.21.15.100:0/3128197552 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fa390050050 con 0x558cdd1eb910 2024-05-10T11:40:15.178 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.177+0000 7fa3a0873740 10 client.29050 fetch_fsmap learned FSMap version 41 2024-05-10T11:40:15.178 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.177+0000 7fa3a0873740 1 -- 172.21.15.100:0/3128197552 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x558cdd1f7790 con 0x558cdd1eb910 2024-05-10T11:40:15.179 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.178+0000 7fa376ffd640 1 -- 172.21.15.100:0/3128197552 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa3900056d0 con 0x558cdd1eb910 2024-05-10T11:40:15.180 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.178+0000 7fa376ffd640 1 -- 172.21.15.100:0/3128197552 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fa390018070 con 0x558cdd1eb910 2024-05-10T11:40:15.180 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.178+0000 7fa3a0873740 10 client.29050 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:40:15.180 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.178+0000 7fa3a0873740 10 client.29050 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:40:15.180 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.178+0000 7fa3a0873740 20 client.29050 populate_metadata read hostname 'smithi100' 2024-05-10T11:40:15.180 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.178+0000 7fa3a0873740 1 --2- 172.21.15.100:0/3128197552 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558cdd266b30 0x558cdd268f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:15.180 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.178+0000 7fa3a0873740 4 client.29050 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:40:15.180 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.178+0000 7fa3a0873740 1 -- 172.21.15.100:0/3128197552 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558cdd152050 con 0x558cdd266b30 2024-05-10T11:40:15.180 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.178+0000 7fa3977fe640 1 --2- 172.21.15.100:0/3128197552 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558cdd266b30 0x558cdd268f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:15.180 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.179+0000 7fa376ffd640 10 client.29050 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:40:15.180 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.179+0000 7fa3977fe640 1 --2- 172.21.15.100:0/3128197552 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558cdd266b30 0x558cdd268f50 crc :-1 s=READY pgs=414 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7f70373d42ec98c server_cookie=4a1feacb4261124f in_seq=0 out_seq=0 2024-05-10T11:40:15.348 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.347+0000 7fa376ffd640 1 -- 172.21.15.100:0/3128197552 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558cdd152050 con 0x558cdd266b30 2024-05-10T11:40:15.348 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.348+0000 7fa376ffd640 10 client.29050 handle_command_reply: tid=0 2024-05-10T11:40:15.349 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.348+0000 7fa3a0873740 1 client.29050 shutdown 2024-05-10T11:40:15.349 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.348+0000 7fa3a0873740 1 -- 172.21.15.100:0/3128197552 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cdd1eb910 msgr2=0x558cdd1f5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:15.349 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.348+0000 7fa3a0873740 1 --2- 172.21.15.100:0/3128197552 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cdd1eb910 0x558cdd1f5e90 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa390005f40 tx=0x7fa390005910 comp rx=0 tx=0).stop 2024-05-10T11:40:15.349 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.348+0000 7fa3a0873740 1 -- 172.21.15.100:0/3128197552 shutdown_connections 2024-05-10T11:40:15.349 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.348+0000 7fa3a0873740 1 --2- 172.21.15.100:0/3128197552 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558cdd266b30 0x558cdd268f50 crc :-1 s=READY pgs=414 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:15.349 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.348+0000 7fa3a0873740 1 --2- 172.21.15.100:0/3128197552 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cdd1eb910 0x558cdd1f5e90 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:15.349 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.348+0000 7fa3a0873740 1 --2- 172.21.15.100:0/3128197552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558cdd1e67a0 0x558cdd1f4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:15.349 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.348+0000 7fa3a0873740 1 --2- 172.21.15.100:0/3128197552 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558cdd1e5db0 0x558cdd1f4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:15.349 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.348+0000 7fa3a0873740 1 -- 172.21.15.100:0/3128197552 >> 172.21.15.100:0/3128197552 conn(0x558cdd1e1420 msgr2=0x558cdd1f02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:40:15.349 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.348+0000 7fa376ffd640 0 client.29050 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:40:15.349 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.348+0000 7fa3a0873740 1 -- 172.21.15.100:0/3128197552 shutdown_connections 2024-05-10T11:40:15.349 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.348+0000 7fa3a0873740 1 -- 172.21.15.100:0/3128197552 wait complete. 2024-05-10T11:40:15.349 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.348+0000 7fa3a0873740 20 client.29050 trim_cache size 0 max 16384 2024-05-10T11:40:15.350 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:40:15.350 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:40:15.350 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:40:15.350 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:40:15.351 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.349+0000 7fa39db58640 1 -- 172.21.15.100:0/1737870957 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa3800b2920 msgr2=0x7fa3800b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:15.351 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.349+0000 7fa39db58640 1 --2- 172.21.15.100:0/1737870957 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa3800b2920 0x7fa3800b4de0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x558cdd1e6ba0 tx=0x7fa388005f90 comp rx=0 tx=0).stop 2024-05-10T11:40:15.351 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.349+0000 7fa39db58640 1 -- 172.21.15.100:0/1737870957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa398103bf0 msgr2=0x7fa39819f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:15.351 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.349+0000 7fa39db58640 1 --2- 172.21.15.100:0/1737870957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa398103bf0 0x7fa39819f5e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa38c031460 tx=0x7fa38c003c10 comp rx=0 tx=0).stop 2024-05-10T11:40:15.351 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.349+0000 7fa39db58640 1 -- 172.21.15.100:0/1737870957 shutdown_connections 2024-05-10T11:40:15.351 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.349+0000 7fa39db58640 1 --2- 172.21.15.100:0/1737870957 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa3800b2920 0x7fa3800b4de0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:15.352 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.349+0000 7fa39db58640 1 --2- 172.21.15.100:0/1737870957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3981065a0 0x7fa3981a6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:15.352 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.349+0000 7fa39db58640 1 --2- 172.21.15.100:0/1737870957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa398103bf0 0x7fa39819f5e0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:15.352 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.349+0000 7fa39db58640 1 --2- 172.21.15.100:0/1737870957 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa398101280 0x7fa39819f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:15.352 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.349+0000 7fa39db58640 1 -- 172.21.15.100:0/1737870957 >> 172.21.15.100:0/1737870957 conn(0x7fa3980fce80 msgr2=0x7fa398101d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:40:15.352 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.349+0000 7fa39db58640 1 -- 172.21.15.100:0/1737870957 shutdown_connections 2024-05-10T11:40:15.352 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:15.349+0000 7fa39db58640 1 -- 172.21.15.100:0/1737870957 wait complete. 2024-05-10T11:40:15.950 INFO:tasks.workunit.client.0.smithi100.stdout: 10 81670 8.96 MB/sec execute 369 sec latency 167.133 ms 2024-05-10T11:40:16.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:40:15 smithi178 ceph-mon[33359]: pgmap v1077: 129 pgs: 129 active+clean; 557 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 4.9 MiB/s wr, 276 op/s 2024-05-10T11:40:16.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:40:15 smithi188 ceph-mon[34075]: pgmap v1077: 129 pgs: 129 active+clean; 557 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 4.9 MiB/s wr, 276 op/s 2024-05-10T11:40:16.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:40:15 smithi100 ceph-mon[30296]: pgmap v1077: 129 pgs: 129 active+clean; 557 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 4.9 MiB/s wr, 276 op/s 2024-05-10T11:40:16.950 INFO:tasks.workunit.client.0.smithi100.stdout: 10 81858 8.97 MB/sec execute 370 sec latency 109.146 ms 2024-05-10T11:40:17.950 INFO:tasks.workunit.client.0.smithi100.stdout: 10 82043 8.97 MB/sec execute 371 sec latency 71.513 ms 2024-05-10T11:40:18.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:40:17 smithi178 ceph-mon[33359]: pgmap v1078: 129 pgs: 129 active+clean; 552 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.2 MiB/s wr, 338 op/s 2024-05-10T11:40:18.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:40:17 smithi188 ceph-mon[34075]: pgmap v1078: 129 pgs: 129 active+clean; 552 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.2 MiB/s wr, 338 op/s 2024-05-10T11:40:18.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:40:17 smithi100 ceph-mon[30296]: pgmap v1078: 129 pgs: 129 active+clean; 552 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.2 MiB/s wr, 338 op/s 2024-05-10T11:40:18.950 INFO:tasks.workunit.client.0.smithi100.stdout: 10 82212 8.96 MB/sec execute 372 sec latency 71.840 ms 2024-05-10T11:40:19.950 INFO:tasks.workunit.client.0.smithi100.stdout: 10 82376 8.96 MB/sec execute 373 sec latency 88.044 ms 2024-05-10T11:40:20.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:40:19 smithi188 ceph-mon[34075]: pgmap v1079: 129 pgs: 129 active+clean; 549 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 4.4 MiB/s wr, 270 op/s 2024-05-10T11:40:20.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:40:19 smithi100 ceph-mon[30296]: pgmap v1079: 129 pgs: 129 active+clean; 549 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 4.4 MiB/s wr, 270 op/s 2024-05-10T11:40:20.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:40:19 smithi178 ceph-mon[33359]: pgmap v1079: 129 pgs: 129 active+clean; 549 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 4.4 MiB/s wr, 270 op/s 2024-05-10T11:40:20.950 INFO:tasks.workunit.client.0.smithi100.stdout: 10 82569 8.96 MB/sec execute 374 sec latency 86.142 ms 2024-05-10T11:40:21.950 INFO:tasks.workunit.client.0.smithi100.stdout: 10 82732 8.95 MB/sec execute 375 sec latency 54.344 ms 2024-05-10T11:40:22.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:40:21 smithi188 ceph-mon[34075]: pgmap v1080: 129 pgs: 129 active+clean; 541 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 4.2 MiB/s wr, 311 op/s 2024-05-10T11:40:22.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:40:21 smithi100 ceph-mon[30296]: pgmap v1080: 129 pgs: 129 active+clean; 541 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 4.2 MiB/s wr, 311 op/s 2024-05-10T11:40:22.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:40:21 smithi178 ceph-mon[33359]: pgmap v1080: 129 pgs: 129 active+clean; 541 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 4.2 MiB/s wr, 311 op/s 2024-05-10T11:40:22.907 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T11:40:22.950 INFO:tasks.workunit.client.0.smithi100.stdout: 10 82917 8.95 MB/sec execute 376 sec latency 56.569 ms 2024-05-10T11:40:23.694 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:40:23.950 INFO:tasks.workunit.client.0.smithi100.stdout: 10 83128 8.97 MB/sec execute 377 sec latency 75.610 ms 2024-05-10T11:40:24.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:40:23 smithi188 ceph-mon[34075]: pgmap v1081: 129 pgs: 129 active+clean; 531 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1011 KiB/s rd, 3.6 MiB/s wr, 323 op/s 2024-05-10T11:40:24.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:40:23 smithi100 ceph-mon[30296]: pgmap v1081: 129 pgs: 129 active+clean; 531 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1011 KiB/s rd, 3.6 MiB/s wr, 323 op/s 2024-05-10T11:40:24.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:40:23 smithi178 ceph-mon[33359]: pgmap v1081: 129 pgs: 129 active+clean; 531 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1011 KiB/s rd, 3.6 MiB/s wr, 323 op/s 2024-05-10T11:40:24.950 INFO:tasks.workunit.client.0.smithi100.stdout: 10 83267 8.97 MB/sec execute 378 sec latency 136.451 ms 2024-05-10T11:40:25.950 INFO:tasks.workunit.client.0.smithi100.stdout: 10 83418 8.97 MB/sec execute 379 sec latency 90.708 ms 2024-05-10T11:40:26.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:40:25 smithi188 ceph-mon[34075]: pgmap v1082: 129 pgs: 129 active+clean; 532 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 3.5 MiB/s wr, 271 op/s 2024-05-10T11:40:26.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:40:25 smithi100 ceph-mon[30296]: pgmap v1082: 129 pgs: 129 active+clean; 532 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 3.5 MiB/s wr, 271 op/s 2024-05-10T11:40:26.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:40:25 smithi178 ceph-mon[33359]: pgmap v1082: 129 pgs: 129 active+clean; 532 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 3.5 MiB/s wr, 271 op/s 2024-05-10T11:40:26.951 INFO:tasks.workunit.client.0.smithi100.stdout: 10 83567 8.97 MB/sec execute 380 sec latency 208.179 ms 2024-05-10T11:40:27.950 INFO:tasks.workunit.client.0.smithi100.stdout: 10 83781 8.99 MB/sec execute 381 sec latency 111.160 ms 2024-05-10T11:40:27.964 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:40:27 smithi100 ceph-mon[30296]: pgmap v1083: 129 pgs: 129 active+clean; 543 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 4.3 MiB/s wr, 345 op/s 2024-05-10T11:40:28.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:40:27 smithi188 ceph-mon[34075]: pgmap v1083: 129 pgs: 129 active+clean; 543 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 4.3 MiB/s wr, 345 op/s 2024-05-10T11:40:28.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.113+0000 7fd4ebfff700 1 -- 172.21.15.100:0/1447991132 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f0017400 con 0x7fd4f4108400 2024-05-10T11:40:28.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.114+0000 7fd4fb2cc700 1 -- 172.21.15.100:0/1447991132 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4f4108400 msgr2=0x7fd4f41087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:28.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.114+0000 7fd4fb2cc700 1 --2- 172.21.15.100:0/1447991132 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4f4108400 0x7fd4f41087e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd4f00098f0 tx=0x7fd4f0009c00 comp rx=0 tx=0).stop 2024-05-10T11:40:28.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.114+0000 7fd4fb2cc700 1 -- 172.21.15.100:0/1447991132 shutdown_connections 2024-05-10T11:40:28.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.114+0000 7fd4fb2cc700 1 --2- 172.21.15.100:0/1447991132 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4f4102d50 0x7fd4f410f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:28.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.114+0000 7fd4fb2cc700 1 --2- 172.21.15.100:0/1447991132 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4f4108400 0x7fd4f41087e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:28.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.114+0000 7fd4fb2cc700 1 --2- 172.21.15.100:0/1447991132 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4f4102390 0x7fd4f4102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:28.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.114+0000 7fd4fb2cc700 1 -- 172.21.15.100:0/1447991132 >> 172.21.15.100:0/1447991132 conn(0x7fd4f4079310 msgr2=0x7fd4f4079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:40:28.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.114+0000 7fd4fb2cc700 1 -- 172.21.15.100:0/1447991132 shutdown_connections 2024-05-10T11:40:28.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.114+0000 7fd4fb2cc700 1 -- 172.21.15.100:0/1447991132 wait complete. 2024-05-10T11:40:28.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.115+0000 7fd4fb2cc700 1 Processor -- start 2024-05-10T11:40:28.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.115+0000 7fd4fb2cc700 1 -- start start 2024-05-10T11:40:28.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.115+0000 7fd4fb2cc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4f4102390 0x7fd4f410fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:28.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.115+0000 7fd4fb2cc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4f4102d50 0x7fd4f41103d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:28.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.115+0000 7fd4fb2cc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4f4108400 0x7fd4f4110910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:28.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.115+0000 7fd4fb2cc700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f41134d0 con 0x7fd4f4102390 2024-05-10T11:40:28.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.115+0000 7fd4fb2cc700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f4113350 con 0x7fd4f4102d50 2024-05-10T11:40:28.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.115+0000 7fd4fb2cc700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f4113650 con 0x7fd4f4108400 2024-05-10T11:40:28.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.115+0000 7fd4f8867700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4f4102d50 0x7fd4f41103d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:28.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.115+0000 7fd4f9869700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4f4108400 0x7fd4f4110910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:28.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.115+0000 7fd4f8867700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4f4102d50 0x7fd4f41103d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:51762/0 (socket says 172.21.15.100:51762) 2024-05-10T11:40:28.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.115+0000 7fd4f8867700 1 -- 172.21.15.100:0/1162161539 learned_addr learned my addr 172.21.15.100:0/1162161539 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:40:28.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.115+0000 7fd4f9068700 1 --2- 172.21.15.100:0/1162161539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4f4102390 0x7fd4f410fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:28.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.116+0000 7fd4f8867700 1 -- 172.21.15.100:0/1162161539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4f4108400 msgr2=0x7fd4f4110910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:28.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.116+0000 7fd4f8867700 1 --2- 172.21.15.100:0/1162161539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4f4108400 0x7fd4f4110910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:28.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.116+0000 7fd4f8867700 1 -- 172.21.15.100:0/1162161539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4f4102390 msgr2=0x7fd4f410fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:28.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.116+0000 7fd4f8867700 1 --2- 172.21.15.100:0/1162161539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4f4102390 0x7fd4f410fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:28.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.116+0000 7fd4f8867700 1 -- 172.21.15.100:0/1162161539 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4f40740a0 con 0x7fd4f4102d50 2024-05-10T11:40:28.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.116+0000 7fd4f8867700 1 --2- 172.21.15.100:0/1162161539 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4f4102d50 0x7fd4f41103d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd4e400ec10 tx=0x7fd4e400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:40:28.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.116+0000 7fd4ea7fc700 1 -- 172.21.15.100:0/1162161539 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e400cc90 con 0x7fd4f4102d50 2024-05-10T11:40:28.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.116+0000 7fd4ea7fc700 1 -- 172.21.15.100:0/1162161539 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd4e400ce30 con 0x7fd4f4102d50 2024-05-10T11:40:28.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.116+0000 7fd4ea7fc700 1 -- 172.21.15.100:0/1162161539 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e4010430 con 0x7fd4f4102d50 2024-05-10T11:40:28.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.117+0000 7fd4fb2cc700 1 -- 172.21.15.100:0/1162161539 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4f4074390 con 0x7fd4f4102d50 2024-05-10T11:40:28.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.117+0000 7fd4fb2cc700 1 -- 172.21.15.100:0/1162161539 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4f4074880 con 0x7fd4f4102d50 2024-05-10T11:40:28.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.120+0000 7fd4ea7fc700 1 -- 172.21.15.100:0/1162161539 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fd4e4003f30 con 0x7fd4f4102d50 2024-05-10T11:40:28.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.121+0000 7fd4ea7fc700 1 --2- 172.21.15.100:0/1162161539 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fd4e00b2900 0x7fd4e00b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:28.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.121+0000 7fd4ea7fc700 1 -- 172.21.15.100:0/1162161539 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fd4e40e2840 con 0x7fd4f4102d50 2024-05-10T11:40:28.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.121+0000 7fd4f9068700 1 --2- 172.21.15.100:0/1162161539 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fd4e00b2900 0x7fd4e00b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:28.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.122+0000 7fd4f9068700 1 --2- 172.21.15.100:0/1162161539 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fd4e00b2900 0x7fd4e00b4dc0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0009810 tx=0x7fd4f0019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:40:28.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.124+0000 7fd505b5fb80 1 Processor -- start 2024-05-10T11:40:28.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.124+0000 7fd505b5fb80 1 -- start start 2024-05-10T11:40:28.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.124+0000 7fd505b5fb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5629fb5964d0 0x5629fb5968b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:28.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.124+0000 7fd505b5fb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629fb596df0 0x5629fb59a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:28.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.124+0000 7fd505b5fb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629fb59ac80 0x5629fb59b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:28.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.124+0000 7fd505b5fb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5629fb4e8860 con 0x5629fb596df0 2024-05-10T11:40:28.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.124+0000 7fd505b5fb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5629fb4877c0 con 0x5629fb59ac80 2024-05-10T11:40:28.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.124+0000 7fd4f9068700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5629fb5964d0 0x5629fb5968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:28.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.124+0000 7fd4f9068700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5629fb5964d0 0x5629fb5968b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:43498/0 (socket says 172.21.15.100:43498) 2024-05-10T11:40:28.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.124+0000 7fd4f9068700 1 -- 172.21.15.100:0/2963479105 learned_addr learned my addr 172.21.15.100:0/2963479105 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:40:28.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.124+0000 7fd4f9869700 1 --2- 172.21.15.100:0/2963479105 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629fb59ac80 0x5629fb59b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:28.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.125+0000 7fd4f8867700 1 --2- 172.21.15.100:0/2963479105 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629fb596df0 0x5629fb59a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:28.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.125+0000 7fd505b5fb80 1 -- 172.21.15.100:0/2963479105 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5629fb4e4820 con 0x5629fb5964d0 2024-05-10T11:40:28.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.125+0000 7fd4f9068700 1 -- 172.21.15.100:0/2963479105 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629fb59ac80 msgr2=0x5629fb59b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:28.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.125+0000 7fd4f9068700 1 --2- 172.21.15.100:0/2963479105 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629fb59ac80 0x5629fb59b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:28.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.125+0000 7fd4f9068700 1 -- 172.21.15.100:0/2963479105 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629fb596df0 msgr2=0x5629fb59a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:28.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.125+0000 7fd4f9068700 1 --2- 172.21.15.100:0/2963479105 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629fb596df0 0x5629fb59a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:28.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.125+0000 7fd4f9068700 1 -- 172.21.15.100:0/2963479105 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629fb4358d0 con 0x5629fb5964d0 2024-05-10T11:40:28.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.125+0000 7fd4f8867700 1 --2- 172.21.15.100:0/2963479105 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629fb596df0 0x5629fb59a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:40:28.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.125+0000 7fd4f9068700 1 --2- 172.21.15.100:0/2963479105 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5629fb5964d0 0x5629fb5968b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0009680 tx=0x7fd4f001e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf595a1aa8f30056 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:40:28.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.126+0000 7fd4df7fe700 1 -- 172.21.15.100:0/2963479105 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f000f7e0 con 0x5629fb5964d0 2024-05-10T11:40:28.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.126+0000 7fd4df7fe700 1 -- 172.21.15.100:0/2963479105 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd4f00054d0 con 0x5629fb5964d0 2024-05-10T11:40:28.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.126+0000 7fd4df7fe700 1 -- 172.21.15.100:0/2963479105 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f000b560 con 0x5629fb5964d0 2024-05-10T11:40:28.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.127+0000 7fd505b5fb80 1 -- 172.21.15.100:0/2963479105 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5629fb5964d0 msgr2=0x5629fb5968b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:28.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.127+0000 7fd505b5fb80 1 --2- 172.21.15.100:0/2963479105 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5629fb5964d0 0x5629fb5968b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0009680 tx=0x7fd4f001e000 comp rx=0 tx=0).stop 2024-05-10T11:40:28.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.127+0000 7fd505b5fb80 1 -- 172.21.15.100:0/2963479105 shutdown_connections 2024-05-10T11:40:28.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.127+0000 7fd505b5fb80 1 --2- 172.21.15.100:0/2963479105 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629fb59ac80 0x5629fb59b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:28.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.127+0000 7fd505b5fb80 1 --2- 172.21.15.100:0/2963479105 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5629fb5964d0 0x5629fb5968b0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:28.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.127+0000 7fd505b5fb80 1 --2- 172.21.15.100:0/2963479105 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629fb596df0 0x5629fb59a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:28.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.127+0000 7fd505b5fb80 1 -- 172.21.15.100:0/2963479105 >> 172.21.15.100:0/2963479105 conn(0x5629fb4c2100 msgr2=0x5629fb590c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:40:28.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.127+0000 7fd505b5fb80 1 -- 172.21.15.100:0/2963479105 shutdown_connections 2024-05-10T11:40:28.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.127+0000 7fd505b5fb80 1 -- 172.21.15.100:0/2963479105 wait complete. 2024-05-10T11:40:28.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.127+0000 7fd505b5fb80 1 -- start start 2024-05-10T11:40:28.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.128+0000 7fd505b5fb80 1 Processor -- start 2024-05-10T11:40:28.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.128+0000 7fd505b5fb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5629fb5964d0 0x5629fb5a2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:28.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.128+0000 7fd505b5fb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629fb596df0 0x5629fb59f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:28.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.129+0000 7fd505b5fb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629fb59ac80 0x5629fb59f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:28.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.129+0000 7fd505b5fb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5629fb5a2e30 con 0x5629fb596df0 2024-05-10T11:40:28.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.129+0000 7fd505b5fb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5629fb44ec30 con 0x5629fb59ac80 2024-05-10T11:40:28.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.129+0000 7fd505b5fb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5629fb5a2fb0 con 0x5629fb5964d0 2024-05-10T11:40:28.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.129+0000 7fd505b5fb80 10 client.0 _ll_register_callbacks cb 0x5629fb480920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:40:28.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.129+0000 7fd4f8867700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629fb596df0 0x5629fb59f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:28.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.129+0000 7fd4f8867700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629fb596df0 0x5629fb59f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56452/0 (socket says 172.21.15.100:56452) 2024-05-10T11:40:28.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.129+0000 7fd4f8867700 1 -- 172.21.15.100:0/3093585615 learned_addr learned my addr 172.21.15.100:0/3093585615 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:40:28.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.129+0000 7fd4f9068700 1 --2- 172.21.15.100:0/3093585615 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5629fb5964d0 0x5629fb5a2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:28.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.129+0000 7fd4f8867700 1 -- 172.21.15.100:0/3093585615 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5629fb5964d0 msgr2=0x5629fb5a2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:28.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.129+0000 7fd4f8867700 1 --2- 172.21.15.100:0/3093585615 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5629fb5964d0 0x5629fb5a2080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:28.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.129+0000 7fd4f8867700 1 -- 172.21.15.100:0/3093585615 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629fb59ac80 msgr2=0x5629fb59f9b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T11:40:28.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.129+0000 7fd4f8867700 1 --2- 172.21.15.100:0/3093585615 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629fb59ac80 0x5629fb59f9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:28.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.129+0000 7fd4f8867700 1 -- 172.21.15.100:0/3093585615 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629fb484690 con 0x5629fb596df0 2024-05-10T11:40:28.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.129+0000 7fd4f8867700 1 --2- 172.21.15.100:0/3093585615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629fb596df0 0x5629fb59f470 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fd4e40a6d10 tx=0x7fd4e40a5230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9307ae4dee6caee8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:40:28.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.129+0000 7fd4dcff9700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:40:28.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.129+0000 7fd4dcff9700 1 -- 172.21.15.100:0/3093585615 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e40b2240 con 0x5629fb596df0 2024-05-10T11:40:28.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.129+0000 7fd505b5fb80 1 -- 172.21.15.100:0/3093585615 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4f00054d0 con 0x5629fb596df0 2024-05-10T11:40:28.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.130+0000 7fd4dcff9700 1 -- 172.21.15.100:0/3093585615 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fd4e40a6090 con 0x5629fb596df0 2024-05-10T11:40:28.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.130+0000 7fd4dcff9700 1 -- 172.21.15.100:0/3093585615 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fd4e40a95b0 con 0x5629fb596df0 2024-05-10T11:40:28.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.130+0000 7fd505b5fb80 10 client.19686 fetch_fsmap learned FSMap version 41 2024-05-10T11:40:28.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.130+0000 7fd505b5fb80 1 -- 172.21.15.100:0/3093585615 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5629fb5ad720 con 0x5629fb596df0 2024-05-10T11:40:28.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.131+0000 7fd4dcff9700 1 -- 172.21.15.100:0/3093585615 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd4e40c3070 con 0x5629fb596df0 2024-05-10T11:40:28.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.131+0000 7fd4dcff9700 1 -- 172.21.15.100:0/3093585615 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fd4e4004220 con 0x5629fb596df0 2024-05-10T11:40:28.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.131+0000 7fd505b5fb80 10 client.19686 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:40:28.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.131+0000 7fd505b5fb80 10 client.19686 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:40:28.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.131+0000 7fd505b5fb80 20 client.19686 populate_metadata read hostname 'smithi100' 2024-05-10T11:40:28.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.131+0000 7fd505b5fb80 1 --2- 172.21.15.100:0/3093585615 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5629fb5afcb0 0x5629fb5b20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:28.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.131+0000 7fd505b5fb80 4 client.19686 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:40:28.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.131+0000 7fd505b5fb80 1 -- 172.21.15.100:0/3093585615 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd4f00054d0 con 0x5629fb5afcb0 2024-05-10T11:40:28.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.131+0000 7fd4f9068700 1 --2- 172.21.15.100:0/3093585615 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5629fb5afcb0 0x5629fb5b20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:28.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.132+0000 7fd4dcff9700 10 client.19686 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:40:28.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.133+0000 7fd4f9068700 1 --2- 172.21.15.100:0/3093585615 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5629fb5afcb0 0x5629fb5b20a0 crc :-1 s=READY pgs=415 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a47b9c05cc949fff server_cookie=b75d119420e89a0e in_seq=0 out_seq=0 2024-05-10T11:40:28.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.133+0000 7fd4dcff9700 1 -- 172.21.15.100:0/3093585615 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fd4f00054d0 con 0x5629fb5afcb0 2024-05-10T11:40:28.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.133+0000 7fd4dcff9700 10 client.19686 handle_command_reply: tid=0 2024-05-10T11:40:28.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.133+0000 7fd505b5fb80 1 client.19686 shutdown 2024-05-10T11:40:28.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.134+0000 7fd505b5fb80 1 -- 172.21.15.100:0/3093585615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629fb596df0 msgr2=0x5629fb59f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:28.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.134+0000 7fd505b5fb80 1 --2- 172.21.15.100:0/3093585615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629fb596df0 0x5629fb59f470 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fd4e40a6d10 tx=0x7fd4e40a5230 comp rx=0 tx=0).stop 2024-05-10T11:40:28.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.134+0000 7fd505b5fb80 1 -- 172.21.15.100:0/3093585615 shutdown_connections 2024-05-10T11:40:28.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.134+0000 7fd505b5fb80 1 --2- 172.21.15.100:0/3093585615 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5629fb5afcb0 0x5629fb5b20a0 crc :-1 s=READY pgs=415 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:28.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.134+0000 7fd505b5fb80 1 --2- 172.21.15.100:0/3093585615 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629fb59ac80 0x5629fb59f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:28.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.134+0000 7fd4dcff9700 0 client.19686 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:40:28.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.134+0000 7fd505b5fb80 1 --2- 172.21.15.100:0/3093585615 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5629fb5964d0 0x5629fb5a2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:28.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.134+0000 7fd505b5fb80 1 --2- 172.21.15.100:0/3093585615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629fb596df0 0x5629fb59f470 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:28.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.134+0000 7fd505b5fb80 1 -- 172.21.15.100:0/3093585615 >> 172.21.15.100:0/3093585615 conn(0x5629fb4c2100 msgr2=0x5629fb58b720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:40:28.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.134+0000 7fd505b5fb80 1 -- 172.21.15.100:0/3093585615 shutdown_connections 2024-05-10T11:40:28.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.134+0000 7fd505b5fb80 1 -- 172.21.15.100:0/3093585615 wait complete. 2024-05-10T11:40:28.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.135+0000 7fd505b5fb80 20 client.19686 trim_cache size 0 max 16384 2024-05-10T11:40:28.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.165+0000 7fd505b5fb80 1 Processor -- start 2024-05-10T11:40:28.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.165+0000 7fd505b5fb80 1 -- start start 2024-05-10T11:40:28.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.165+0000 7fd505b5fb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629fb5964d0 0x5629fb5abec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:28.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.165+0000 7fd505b5fb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629fb596df0 0x5629fb58b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:28.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.165+0000 7fd505b5fb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5629fb59ac80 0x5629fb5b7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:28.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.165+0000 7fd505b5fb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f41c0770 con 0x5629fb596df0 2024-05-10T11:40:28.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.165+0000 7fd505b5fb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f41be000 con 0x5629fb5964d0 2024-05-10T11:40:28.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.165+0000 7fd505b5fb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f41bbe50 con 0x5629fb59ac80 2024-05-10T11:40:28.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.166+0000 7fd4f8867700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629fb596df0 0x5629fb58b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:28.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.166+0000 7fd4f8867700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629fb596df0 0x5629fb58b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56454/0 (socket says 172.21.15.100:56454) 2024-05-10T11:40:28.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.166+0000 7fd4f8867700 1 -- 172.21.15.100:0/2347584146 learned_addr learned my addr 172.21.15.100:0/2347584146 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:40:28.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.166+0000 7fd4f9869700 1 --2- 172.21.15.100:0/2347584146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5629fb59ac80 0x5629fb5b7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:28.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.166+0000 7fd4f9068700 1 --2- 172.21.15.100:0/2347584146 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629fb5964d0 0x5629fb5abec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:28.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.166+0000 7fd4f8867700 1 -- 172.21.15.100:0/2347584146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5629fb59ac80 msgr2=0x5629fb5b7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:28.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.166+0000 7fd4f8867700 1 --2- 172.21.15.100:0/2347584146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5629fb59ac80 0x5629fb5b7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:28.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.166+0000 7fd4f8867700 1 -- 172.21.15.100:0/2347584146 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629fb5964d0 msgr2=0x5629fb5abec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:28.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.166+0000 7fd4f8867700 1 --2- 172.21.15.100:0/2347584146 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629fb5964d0 0x5629fb5abec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:28.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.166+0000 7fd4f8867700 1 -- 172.21.15.100:0/2347584146 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629fb59da20 con 0x5629fb596df0 2024-05-10T11:40:28.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.166+0000 7fd4f8867700 1 --2- 172.21.15.100:0/2347584146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629fb596df0 0x5629fb58b5a0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4005cb0 tx=0x7fd4e400ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da7792add6a2e0e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:40:28.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.166+0000 7fd4deffd700 1 -- 172.21.15.100:0/2347584146 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e40aa1a0 con 0x5629fb596df0 2024-05-10T11:40:28.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.166+0000 7fd4deffd700 1 -- 172.21.15.100:0/2347584146 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd4e40c3030 con 0x5629fb596df0 2024-05-10T11:40:28.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.166+0000 7fd4deffd700 1 -- 172.21.15.100:0/2347584146 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e40a9aa0 con 0x5629fb596df0 2024-05-10T11:40:28.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.167+0000 7fd505b5fb80 1 -- 172.21.15.100:0/2347584146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629fb596df0 msgr2=0x5629fb58b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:28.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.167+0000 7fd505b5fb80 1 --2- 172.21.15.100:0/2347584146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629fb596df0 0x5629fb58b5a0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4005cb0 tx=0x7fd4e400ef90 comp rx=0 tx=0).stop 2024-05-10T11:40:28.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.168+0000 7fd505b5fb80 1 -- 172.21.15.100:0/2347584146 shutdown_connections 2024-05-10T11:40:28.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.168+0000 7fd505b5fb80 1 --2- 172.21.15.100:0/2347584146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5629fb59ac80 0x5629fb5b7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:28.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.168+0000 7fd505b5fb80 1 --2- 172.21.15.100:0/2347584146 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629fb5964d0 0x5629fb5abec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:28.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.168+0000 7fd505b5fb80 1 --2- 172.21.15.100:0/2347584146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629fb596df0 0x5629fb58b5a0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:28.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.168+0000 7fd505b5fb80 1 -- 172.21.15.100:0/2347584146 >> 172.21.15.100:0/2347584146 conn(0x5629fb4c2100 msgr2=0x5629fb58ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:40:28.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.168+0000 7fd505b5fb80 1 -- 172.21.15.100:0/2347584146 shutdown_connections 2024-05-10T11:40:28.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.168+0000 7fd505b5fb80 1 -- 172.21.15.100:0/2347584146 wait complete. 2024-05-10T11:40:28.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.168+0000 7fd505b5fb80 1 -- start start 2024-05-10T11:40:28.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.168+0000 7fd505b5fb80 1 Processor -- start 2024-05-10T11:40:28.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.169+0000 7fd505b5fb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5629fb5964d0 0x5629fb5a6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:28.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.169+0000 7fd505b5fb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629fb596df0 0x5629fb5a6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:28.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.169+0000 7fd505b5fb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629fb59ac80 0x5629fb6134c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:28.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.169+0000 7fd505b5fb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f41bc4c0 con 0x5629fb59ac80 2024-05-10T11:40:28.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.169+0000 7fd505b5fb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f41bd930 con 0x5629fb596df0 2024-05-10T11:40:28.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.169+0000 7fd505b5fb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f41bcb90 con 0x5629fb5964d0 2024-05-10T11:40:28.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.169+0000 7fd505b5fb80 10 client.0 _ll_register_callbacks cb 0x5629fb480920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:40:28.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.169+0000 7fd4f8867700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629fb596df0 0x5629fb5a6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:28.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.169+0000 7fd4f9068700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5629fb5964d0 0x5629fb5a6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:28.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.169+0000 7fd4f8867700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629fb596df0 0x5629fb5a6ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:51796/0 (socket says 172.21.15.100:51796) 2024-05-10T11:40:28.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.169+0000 7fd4f9068700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5629fb5964d0 0x5629fb5a6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:43534/0 (socket says 172.21.15.100:43534) 2024-05-10T11:40:28.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.169+0000 7fd4f8867700 1 -- 172.21.15.100:0/944479230 learned_addr learned my addr 172.21.15.100:0/944479230 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:40:28.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.169+0000 7fd4f9869700 1 --2- 172.21.15.100:0/944479230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629fb59ac80 0x5629fb6134c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:28.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.169+0000 7fd4f9068700 1 -- 172.21.15.100:0/944479230 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629fb596df0 msgr2=0x5629fb5a6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:28.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.169+0000 7fd4f9068700 1 --2- 172.21.15.100:0/944479230 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629fb596df0 0x5629fb5a6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:28.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.169+0000 7fd4f9068700 1 -- 172.21.15.100:0/944479230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629fb59ac80 msgr2=0x5629fb6134c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:28.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.169+0000 7fd4f9068700 1 --2- 172.21.15.100:0/944479230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629fb59ac80 0x5629fb6134c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:28.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.169+0000 7fd4f9068700 1 -- 172.21.15.100:0/944479230 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629fb613ab0 con 0x5629fb5964d0 2024-05-10T11:40:28.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.169+0000 7fd4f8867700 1 --2- 172.21.15.100:0/944479230 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629fb596df0 0x5629fb5a6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:40:28.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.169+0000 7fd4f9869700 1 --2- 172.21.15.100:0/944479230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629fb59ac80 0x5629fb6134c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:40:28.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.169+0000 7fd4f9068700 1 --2- 172.21.15.100:0/944479230 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5629fb5964d0 0x5629fb5a6660 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x5629fb5b28d0 tx=0x7fd4f0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=804a05bc76572b7a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:40:28.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.169+0000 7fd4dcff9700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:40:28.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.170+0000 7fd4dcff9700 1 -- 172.21.15.100:0/944479230 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f000f4e0 con 0x5629fb5964d0 2024-05-10T11:40:28.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.170+0000 7fd4dcff9700 1 -- 172.21.15.100:0/944479230 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fd4f000f680 con 0x5629fb5964d0 2024-05-10T11:40:28.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.170+0000 7fd505b5fb80 1 -- 172.21.15.100:0/944479230 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4e40c3030 con 0x5629fb5964d0 2024-05-10T11:40:28.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.170+0000 7fd4dcff9700 1 -- 172.21.15.100:0/944479230 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fd4f0048050 con 0x5629fb5964d0 2024-05-10T11:40:28.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.170+0000 7fd505b5fb80 10 client.29065 fetch_fsmap learned FSMap version 41 2024-05-10T11:40:28.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.170+0000 7fd505b5fb80 1 -- 172.21.15.100:0/944479230 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5629fb61ae20 con 0x5629fb5964d0 2024-05-10T11:40:28.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.170+0000 7fd4dcff9700 1 -- 172.21.15.100:0/944479230 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd4f003f070 con 0x5629fb5964d0 2024-05-10T11:40:28.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.170+0000 7fd4dcff9700 1 -- 172.21.15.100:0/944479230 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fd4f003a4b0 con 0x5629fb5964d0 2024-05-10T11:40:28.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.170+0000 7fd505b5fb80 10 client.29065 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:40:28.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.170+0000 7fd505b5fb80 10 client.29065 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:40:28.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.170+0000 7fd505b5fb80 20 client.29065 populate_metadata read hostname 'smithi100' 2024-05-10T11:40:28.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.170+0000 7fd505b5fb80 1 --2- 172.21.15.100:0/944479230 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5629fb5afcb0 0x5629fb61d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:28.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.170+0000 7fd505b5fb80 4 client.29065 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T11:40:28.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.170+0000 7fd505b5fb80 1 -- 172.21.15.100:0/944479230 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fd4e40c3030 con 0x5629fb5afcb0 2024-05-10T11:40:28.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.171+0000 7fd4f8867700 1 --2- 172.21.15.100:0/944479230 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5629fb5afcb0 0x5629fb61d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:28.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.171+0000 7fd4dcff9700 10 client.29065 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:40:28.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.171+0000 7fd4f8867700 1 --2- 172.21.15.100:0/944479230 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5629fb5afcb0 0x5629fb61d060 crc :-1 s=READY pgs=416 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=273a1a24922ce796 server_cookie=13eaf5b57da73d0c in_seq=0 out_seq=0 2024-05-10T11:40:28.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.172+0000 7fd4dcff9700 1 -- 172.21.15.100:0/944479230 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7fd4e40c3030 con 0x5629fb5afcb0 2024-05-10T11:40:28.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.172+0000 7fd4dcff9700 10 client.29065 handle_command_reply: tid=0 2024-05-10T11:40:28.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.172+0000 7fd505b5fb80 1 client.29065 shutdown 2024-05-10T11:40:28.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.172+0000 7fd505b5fb80 1 -- 172.21.15.100:0/944479230 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5629fb5964d0 msgr2=0x5629fb5a6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:28.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.172+0000 7fd505b5fb80 1 --2- 172.21.15.100:0/944479230 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5629fb5964d0 0x5629fb5a6660 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x5629fb5b28d0 tx=0x7fd4f0009c00 comp rx=0 tx=0).stop 2024-05-10T11:40:28.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.173+0000 7fd505b5fb80 1 -- 172.21.15.100:0/944479230 shutdown_connections 2024-05-10T11:40:28.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.173+0000 7fd505b5fb80 1 --2- 172.21.15.100:0/944479230 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5629fb5afcb0 0x5629fb61d060 crc :-1 s=READY pgs=416 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:28.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.173+0000 7fd505b5fb80 1 --2- 172.21.15.100:0/944479230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629fb59ac80 0x5629fb6134c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:28.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.173+0000 7fd505b5fb80 1 --2- 172.21.15.100:0/944479230 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5629fb5964d0 0x5629fb5a6660 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:28.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.173+0000 7fd505b5fb80 1 --2- 172.21.15.100:0/944479230 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629fb596df0 0x5629fb5a6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:28.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.173+0000 7fd505b5fb80 1 -- 172.21.15.100:0/944479230 >> 172.21.15.100:0/944479230 conn(0x5629fb4c2100 msgr2=0x5629fb58b720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:40:28.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.173+0000 7fd4dcff9700 0 client.29065 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:40:28.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.173+0000 7fd505b5fb80 1 -- 172.21.15.100:0/944479230 shutdown_connections 2024-05-10T11:40:28.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.173+0000 7fd505b5fb80 1 -- 172.21.15.100:0/944479230 wait complete. 2024-05-10T11:40:28.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.173+0000 7fd505b5fb80 20 client.29065 trim_cache size 0 max 16384 2024-05-10T11:40:28.177 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:40:28.177 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-05-10T11:40:28.177 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-05-10T11:40:28.177 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:40:28.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.174+0000 7fd4fb2cc700 1 -- 172.21.15.100:0/1162161539 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fd4e00b2900 msgr2=0x7fd4e00b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:28.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.174+0000 7fd4fb2cc700 1 --2- 172.21.15.100:0/1162161539 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fd4e00b2900 0x7fd4e00b4dc0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0009810 tx=0x7fd4f0019040 comp rx=0 tx=0).stop 2024-05-10T11:40:28.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.174+0000 7fd4fb2cc700 1 -- 172.21.15.100:0/1162161539 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4f4102d50 msgr2=0x7fd4f41103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:28.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.174+0000 7fd4fb2cc700 1 --2- 172.21.15.100:0/1162161539 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4f4102d50 0x7fd4f41103d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd4e400ec10 tx=0x7fd4e400ef20 comp rx=0 tx=0).stop 2024-05-10T11:40:28.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.174+0000 7fd4fb2cc700 1 -- 172.21.15.100:0/1162161539 shutdown_connections 2024-05-10T11:40:28.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.174+0000 7fd4fb2cc700 1 --2- 172.21.15.100:0/1162161539 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fd4e00b2900 0x7fd4e00b4dc0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:28.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.174+0000 7fd4fb2cc700 1 --2- 172.21.15.100:0/1162161539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4f4108400 0x7fd4f4110910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:28.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.174+0000 7fd4fb2cc700 1 --2- 172.21.15.100:0/1162161539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4f4102390 0x7fd4f410fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:28.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.174+0000 7fd4fb2cc700 1 --2- 172.21.15.100:0/1162161539 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4f4102d50 0x7fd4f41103d0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:28.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.174+0000 7fd4fb2cc700 1 -- 172.21.15.100:0/1162161539 >> 172.21.15.100:0/1162161539 conn(0x7fd4f4079310 msgr2=0x7fd4f4101490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:40:28.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.174+0000 7fd4fb2cc700 1 -- 172.21.15.100:0/1162161539 shutdown_connections 2024-05-10T11:40:28.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:28.174+0000 7fd4fb2cc700 1 -- 172.21.15.100:0/1162161539 wait complete. 2024-05-10T11:40:28.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:40:27 smithi178 ceph-mon[33359]: pgmap v1083: 129 pgs: 129 active+clean; 543 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 4.3 MiB/s wr, 345 op/s 2024-05-10T11:40:28.476 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.475+0000 7f856fb89640 1 -- 172.21.15.100:0/3532631302 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85681034c0 msgr2=0x7f8568109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:28.476 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.475+0000 7f856fb89640 1 --2- 172.21.15.100:0/3532631302 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85681034c0 0x7f8568109d50 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f8560009ca0 tx=0x7f856002a8c0 comp rx=0 tx=0).stop 2024-05-10T11:40:28.476 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.475+0000 7f856fb89640 1 -- 172.21.15.100:0/3532631302 shutdown_connections 2024-05-10T11:40:28.476 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.475+0000 7f856fb89640 1 --2- 172.21.15.100:0/3532631302 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85681034c0 0x7f8568109d50 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:28.476 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.475+0000 7f856fb89640 1 --2- 172.21.15.100:0/3532631302 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8568102b00 0x7f8568102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:28.476 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.475+0000 7f856fb89640 1 --2- 172.21.15.100:0/3532631302 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8568101910 0x7f8568101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:28.476 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.475+0000 7f856fb89640 1 -- 172.21.15.100:0/3532631302 >> 172.21.15.100:0/3532631302 conn(0x7f85680fce80 msgr2=0x7f85680ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:40:28.476 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.475+0000 7f856fb89640 1 -- 172.21.15.100:0/3532631302 shutdown_connections 2024-05-10T11:40:28.477 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.475+0000 7f856fb89640 1 -- 172.21.15.100:0/3532631302 wait complete. 2024-05-10T11:40:28.477 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.476+0000 7f856fb89640 1 Processor -- start 2024-05-10T11:40:28.477 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.476+0000 7f856fb89640 1 -- start start 2024-05-10T11:40:28.478 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.476+0000 7f856fb89640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8568101910 0x7f856806abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:28.478 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.476+0000 7f856fb89640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8568102b00 0x7f856819d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:28.478 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.476+0000 7f856fb89640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85681034c0 0x7f856819fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.477+0000 7f856fb89640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f856810d340 con 0x7f85681034c0 2024-05-10T11:40:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.477+0000 7f856fb89640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f856810d1c0 con 0x7f8568102b00 2024-05-10T11:40:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.477+0000 7f856fb89640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f856810d4c0 con 0x7f8568101910 2024-05-10T11:40:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.477+0000 7f856d124640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8568102b00 0x7f856819d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.477+0000 7f856d925640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8568101910 0x7f856806abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.477+0000 7f856d925640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8568101910 0x7f856806abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:43556/0 (socket says 172.21.15.100:43556) 2024-05-10T11:40:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.477+0000 7f856d124640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8568102b00 0x7f856819d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:51802/0 (socket says 172.21.15.100:51802) 2024-05-10T11:40:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.477+0000 7f856d925640 1 -- 172.21.15.100:0/1835524012 learned_addr learned my addr 172.21.15.100:0/1835524012 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:40:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.477+0000 7f856e126640 1 --2- 172.21.15.100:0/1835524012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85681034c0 0x7f856819fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.477+0000 7f856d925640 1 -- 172.21.15.100:0/1835524012 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8568102b00 msgr2=0x7f856819d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.477+0000 7f856d925640 1 --2- 172.21.15.100:0/1835524012 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8568102b00 0x7f856819d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.477+0000 7f856d925640 1 -- 172.21.15.100:0/1835524012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85681034c0 msgr2=0x7f856819fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.477+0000 7f856d925640 1 --2- 172.21.15.100:0/1835524012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85681034c0 0x7f856819fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.477+0000 7f856d925640 1 -- 172.21.15.100:0/1835524012 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85681a0730 con 0x7f8568101910 2024-05-10T11:40:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.477+0000 7f856d925640 1 --2- 172.21.15.100:0/1835524012 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8568101910 0x7f856806abd0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f856400e9c0 tx=0x7f856400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:40:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.478+0000 7f855effd640 1 -- 172.21.15.100:0/1835524012 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f856400cd80 con 0x7f8568101910 2024-05-10T11:40:28.480 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.478+0000 7f856fb89640 1 -- 172.21.15.100:0/1835524012 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85681a09c0 con 0x7f8568101910 2024-05-10T11:40:28.480 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.478+0000 7f855effd640 1 -- 172.21.15.100:0/1835524012 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f85640035f0 con 0x7f8568101910 2024-05-10T11:40:28.480 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.478+0000 7f855effd640 1 -- 172.21.15.100:0/1835524012 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85640044c0 con 0x7f8568101910 2024-05-10T11:40:28.480 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.478+0000 7f856fb89640 1 -- 172.21.15.100:0/1835524012 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85681a0f10 con 0x7f8568101910 2024-05-10T11:40:28.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.482+0000 7f857297d740 1 Processor -- start 2024-05-10T11:40:28.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.483+0000 7f857297d740 1 -- start start 2024-05-10T11:40:28.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.483+0000 7f855effd640 1 -- 172.21.15.100:0/1835524012 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f8564004000 con 0x7f8568101910 2024-05-10T11:40:28.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.483+0000 7f857297d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561797cc01c0 0x561797cc05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:28.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.483+0000 7f857297d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561797cc0bb0 0x561797cc56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:28.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.483+0000 7f857297d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561797cc5d20 0x561797cc81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:28.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.483+0000 7f857297d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x561797c205d0 con 0x561797cc5d20 2024-05-10T11:40:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.483+0000 7f857297d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561797bd9890 con 0x561797cc0bb0 2024-05-10T11:40:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.483+0000 7f857297d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x561797c1c590 con 0x561797cc01c0 2024-05-10T11:40:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.483+0000 7f856d925640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561797cc0bb0 0x561797cc56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.483+0000 7f856d925640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561797cc0bb0 0x561797cc56b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:51808/0 (socket says 172.21.15.100:51808) 2024-05-10T11:40:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.483+0000 7f856d925640 1 -- 172.21.15.100:0/1446193731 learned_addr learned my addr 172.21.15.100:0/1446193731 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:40:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.483+0000 7f856e126640 1 --2- 172.21.15.100:0/1446193731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561797cc01c0 0x561797cc05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.483+0000 7f856d124640 1 --2- 172.21.15.100:0/1446193731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561797cc5d20 0x561797cc81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.483+0000 7f856d124640 1 -- 172.21.15.100:0/1446193731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561797cc01c0 msgr2=0x561797cc05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.483+0000 7f856d124640 1 --2- 172.21.15.100:0/1446193731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561797cc01c0 0x561797cc05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.483+0000 7f856d124640 1 -- 172.21.15.100:0/1446193731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561797cc0bb0 msgr2=0x561797cc56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.483+0000 7f856d124640 1 --2- 172.21.15.100:0/1446193731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561797cc0bb0 0x561797cc56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.483+0000 7f856d124640 1 -- 172.21.15.100:0/1446193731 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561797932210 con 0x561797cc5d20 2024-05-10T11:40:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.483+0000 7f856d925640 1 --2- 172.21.15.100:0/1446193731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561797cc0bb0 0x561797cc56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:40:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.484+0000 7f856d124640 1 --2- 172.21.15.100:0/1446193731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561797cc5d20 0x561797cc81b0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f8558009bf0 tx=0x7f8558011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96b2d3fb922d5f52 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:40:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.484+0000 7f8553fff640 1 -- 172.21.15.100:0/1446193731 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8558004050 con 0x561797cc5d20 2024-05-10T11:40:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.484+0000 7f8553fff640 1 -- 172.21.15.100:0/1446193731 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8558005020 con 0x561797cc5d20 2024-05-10T11:40:28.486 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.484+0000 7f8553fff640 1 -- 172.21.15.100:0/1446193731 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8558004c00 con 0x561797cc5d20 2024-05-10T11:40:28.486 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.484+0000 7f855effd640 1 --2- 172.21.15.100:0/1835524012 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f85540b28a0 0x7f85540b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:28.486 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.484+0000 7f856e126640 1 --2- 172.21.15.100:0/1835524012 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f85540b28a0 0x7f85540b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:28.486 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.484+0000 7f855effd640 1 -- 172.21.15.100:0/1835524012 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f85640e28f0 con 0x7f8568101910 2024-05-10T11:40:28.489 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.485+0000 7f857297d740 1 -- 172.21.15.100:0/1446193731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561797cc5d20 msgr2=0x561797cc81b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:28.489 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.485+0000 7f856e126640 1 --2- 172.21.15.100:0/1835524012 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f85540b28a0 0x7f85540b4d60 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f8560006010 tx=0x7f8560009360 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:40:28.489 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.485+0000 7f857297d740 1 --2- 172.21.15.100:0/1446193731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561797cc5d20 0x561797cc81b0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f8558009bf0 tx=0x7f8558011810 comp rx=0 tx=0).stop 2024-05-10T11:40:28.489 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.485+0000 7f857297d740 1 -- 172.21.15.100:0/1446193731 shutdown_connections 2024-05-10T11:40:28.490 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.485+0000 7f857297d740 1 --2- 172.21.15.100:0/1446193731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561797cc5d20 0x561797cc81b0 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:28.490 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.485+0000 7f857297d740 1 --2- 172.21.15.100:0/1446193731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561797cc0bb0 0x561797cc56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:28.490 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.485+0000 7f857297d740 1 --2- 172.21.15.100:0/1446193731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561797cc01c0 0x561797cc05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:28.490 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.485+0000 7f857297d740 1 -- 172.21.15.100:0/1446193731 >> 172.21.15.100:0/1446193731 conn(0x561797cb7420 msgr2=0x561797cb7820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:40:28.490 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.485+0000 7f857297d740 1 -- 172.21.15.100:0/1446193731 shutdown_connections 2024-05-10T11:40:28.490 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.485+0000 7f857297d740 1 -- 172.21.15.100:0/1446193731 wait complete. 2024-05-10T11:40:28.490 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.485+0000 7f857297d740 1 -- start start 2024-05-10T11:40:28.490 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.486+0000 7f857297d740 1 Processor -- start 2024-05-10T11:40:28.490 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.486+0000 7f857297d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561797cc01c0 0x561797cc3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:28.490 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.486+0000 7f857297d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561797cc0bb0 0x561797cc3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:28.490 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.486+0000 7f857297d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561797cc5d20 0x561797cc4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:28.490 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.486+0000 7f857297d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x561797cbbc90 con 0x561797cc01c0 2024-05-10T11:40:28.490 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.486+0000 7f857297d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561797bdd370 con 0x561797cc0bb0 2024-05-10T11:40:28.490 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.486+0000 7f857297d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x561797cbbe10 con 0x561797cc5d20 2024-05-10T11:40:28.491 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.486+0000 7f857297d740 10 client.0 _ll_register_callbacks cb 0x561797931a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:40:28.491 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.486+0000 7f856d925640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561797cc01c0 0x561797cc3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:28.491 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.486+0000 7f856d925640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561797cc01c0 0x561797cc3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56520/0 (socket says 172.21.15.100:56520) 2024-05-10T11:40:28.491 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.486+0000 7f856d925640 1 -- 172.21.15.100:0/2365647928 learned_addr learned my addr 172.21.15.100:0/2365647928 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:40:28.491 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.486+0000 7f856d124640 1 --2- 172.21.15.100:0/2365647928 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561797cc0bb0 0x561797cc3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:28.491 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.486+0000 7f856e126640 1 --2- 172.21.15.100:0/2365647928 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561797cc5d20 0x561797cc4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:28.491 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.487+0000 7f856d925640 1 -- 172.21.15.100:0/2365647928 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561797cc5d20 msgr2=0x561797cc4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:28.491 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.487+0000 7f856d925640 1 --2- 172.21.15.100:0/2365647928 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561797cc5d20 0x561797cc4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:28.491 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.487+0000 7f856d925640 1 -- 172.21.15.100:0/2365647928 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561797cc0bb0 msgr2=0x561797cc3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:28.491 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.487+0000 7f856d925640 1 --2- 172.21.15.100:0/2365647928 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561797cc0bb0 0x561797cc3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:28.491 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.487+0000 7f856d925640 1 -- 172.21.15.100:0/2365647928 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617979fe620 con 0x561797cc01c0 2024-05-10T11:40:28.491 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.487+0000 7f856d124640 1 --2- 172.21.15.100:0/2365647928 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561797cc0bb0 0x561797cc3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:40:28.491 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.487+0000 7f856e126640 1 --2- 172.21.15.100:0/2365647928 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561797cc5d20 0x561797cc4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:40:28.491 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.487+0000 7f856d925640 1 --2- 172.21.15.100:0/2365647928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561797cc01c0 0x561797cc3140 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f85640058c0 tx=0x7f856400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ccc6bedae53c50e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:40:28.491 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.487+0000 7f85517fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:40:28.492 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.487+0000 7f85517fa640 1 -- 172.21.15.100:0/2365647928 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85640cae60 con 0x561797cc01c0 2024-05-10T11:40:28.492 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.487+0000 7f857297d740 1 -- 172.21.15.100:0/2365647928 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8558005020 con 0x561797cc01c0 2024-05-10T11:40:28.492 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.487+0000 7f85517fa640 1 -- 172.21.15.100:0/2365647928 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f8564003aa0 con 0x561797cc01c0 2024-05-10T11:40:28.492 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.488+0000 7f85517fa640 1 -- 172.21.15.100:0/2365647928 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f85640a9500 con 0x561797cc01c0 2024-05-10T11:40:28.492 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.488+0000 7f857297d740 10 client.19704 fetch_fsmap learned FSMap version 41 2024-05-10T11:40:28.492 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.488+0000 7f857297d740 1 -- 172.21.15.100:0/2365647928 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x561797cd05c0 con 0x561797cc01c0 2024-05-10T11:40:28.492 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.488+0000 7f85517fa640 1 -- 172.21.15.100:0/2365647928 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8558005020 con 0x561797cc01c0 2024-05-10T11:40:28.492 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.488+0000 7f85517fa640 1 -- 172.21.15.100:0/2365647928 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f85640c40d0 con 0x561797cc01c0 2024-05-10T11:40:28.492 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.488+0000 7f857297d740 10 client.19704 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:40:28.492 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.488+0000 7f857297d740 10 client.19704 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:40:28.492 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.488+0000 7f857297d740 20 client.19704 populate_metadata read hostname 'smithi100' 2024-05-10T11:40:28.492 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.488+0000 7f857297d740 1 --2- 172.21.15.100:0/2365647928 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x561797cd2110 0x561797cd45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:28.492 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.488+0000 7f857297d740 4 client.19704 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:40:28.492 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.488+0000 7f857297d740 1 -- 172.21.15.100:0/2365647928 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561797c2e1e0 con 0x561797cd2110 2024-05-10T11:40:28.492 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.488+0000 7f856d124640 1 --2- 172.21.15.100:0/2365647928 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x561797cd2110 0x561797cd45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:28.492 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.489+0000 7f856d124640 1 --2- 172.21.15.100:0/2365647928 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x561797cd2110 0x561797cd45a0 crc :-1 s=READY pgs=417 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44511af8c0404201 server_cookie=33675c5dc0258533 in_seq=0 out_seq=0 2024-05-10T11:40:28.493 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.489+0000 7f85517fa640 10 client.19704 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:40:28.494 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.490+0000 7f85517fa640 1 -- 172.21.15.100:0/2365647928 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561797c2e1e0 con 0x561797cd2110 2024-05-10T11:40:28.494 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.490+0000 7f85517fa640 10 client.19704 handle_command_reply: tid=0 2024-05-10T11:40:28.494 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.490+0000 7f857297d740 1 client.19704 shutdown 2024-05-10T11:40:28.494 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.491+0000 7f857297d740 1 -- 172.21.15.100:0/2365647928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561797cc01c0 msgr2=0x561797cc3140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:28.494 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.491+0000 7f857297d740 1 --2- 172.21.15.100:0/2365647928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561797cc01c0 0x561797cc3140 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f85640058c0 tx=0x7f856400cf60 comp rx=0 tx=0).stop 2024-05-10T11:40:28.494 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.491+0000 7f857297d740 1 -- 172.21.15.100:0/2365647928 shutdown_connections 2024-05-10T11:40:28.494 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.491+0000 7f857297d740 1 --2- 172.21.15.100:0/2365647928 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x561797cd2110 0x561797cd45a0 crc :-1 s=READY pgs=417 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:28.494 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.491+0000 7f857297d740 1 --2- 172.21.15.100:0/2365647928 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561797cc5d20 0x561797cc4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:28.494 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.491+0000 7f85517fa640 0 client.19704 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:40:28.494 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.491+0000 7f857297d740 1 --2- 172.21.15.100:0/2365647928 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561797cc0bb0 0x561797cc3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:28.494 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.491+0000 7f857297d740 1 --2- 172.21.15.100:0/2365647928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561797cc01c0 0x561797cc3140 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:28.495 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.491+0000 7f857297d740 1 -- 172.21.15.100:0/2365647928 >> 172.21.15.100:0/2365647928 conn(0x561797cb7420 msgr2=0x561797cb6580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:40:28.495 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.491+0000 7f857297d740 1 -- 172.21.15.100:0/2365647928 shutdown_connections 2024-05-10T11:40:28.495 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.492+0000 7f857297d740 1 -- 172.21.15.100:0/2365647928 wait complete. 2024-05-10T11:40:28.495 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.492+0000 7f857297d740 20 client.19704 trim_cache size 0 max 16384 2024-05-10T11:40:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.525+0000 7f857297d740 1 Processor -- start 2024-05-10T11:40:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.526+0000 7f857297d740 1 -- start start 2024-05-10T11:40:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.527+0000 7f857297d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561797cc01c0 0x561797cba330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.527+0000 7f857297d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561797cc0bb0 0x561797cba870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.527+0000 7f857297d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561797cc5d20 0x561797cc2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.527+0000 7f857297d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f85681aba50 con 0x561797cc01c0 2024-05-10T11:40:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.527+0000 7f857297d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f85681a9320 con 0x561797cc5d20 2024-05-10T11:40:28.528 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.527+0000 7f857297d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f85681a7170 con 0x561797cc0bb0 2024-05-10T11:40:28.533 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.530+0000 7f856e126640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561797cc5d20 0x561797cc2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:28.534 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.530+0000 7f856e126640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561797cc5d20 0x561797cc2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:51816/0 (socket says 172.21.15.100:51816) 2024-05-10T11:40:28.534 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.530+0000 7f856e126640 1 -- 172.21.15.100:0/3847453591 learned_addr learned my addr 172.21.15.100:0/3847453591 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:40:28.534 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.531+0000 7f856d124640 1 --2- 172.21.15.100:0/3847453591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561797cc0bb0 0x561797cba870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:28.534 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.531+0000 7f856e126640 1 -- 172.21.15.100:0/3847453591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561797cc0bb0 msgr2=0x561797cba870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:28.534 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.531+0000 7f856e126640 1 --2- 172.21.15.100:0/3847453591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561797cc0bb0 0x561797cba870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:28.534 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.531+0000 7f856e126640 1 -- 172.21.15.100:0/3847453591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561797cc01c0 msgr2=0x561797cba330 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T11:40:28.534 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.531+0000 7f856e126640 1 --2- 172.21.15.100:0/3847453591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561797cc01c0 0x561797cba330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:28.534 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.531+0000 7f856e126640 1 -- 172.21.15.100:0/3847453591 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561797cb7250 con 0x561797cc5d20 2024-05-10T11:40:28.534 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.532+0000 7f856e126640 1 --2- 172.21.15.100:0/3847453591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561797cc5d20 0x561797cc2d50 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8560035000 tx=0x7f8560049000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7b92a2bbc1e32e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:40:28.534 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.532+0000 7f85537fe640 1 -- 172.21.15.100:0/3847453591 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8560056070 con 0x561797cc5d20 2024-05-10T11:40:28.534 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.532+0000 7f85537fe640 1 -- 172.21.15.100:0/3847453591 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8560036050 con 0x561797cc5d20 2024-05-10T11:40:28.535 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.532+0000 7f85537fe640 1 -- 172.21.15.100:0/3847453591 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8560004040 con 0x561797cc5d20 2024-05-10T11:40:28.535 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.535+0000 7f857297d740 1 -- 172.21.15.100:0/3847453591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561797cc5d20 msgr2=0x561797cc2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:28.535 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.535+0000 7f857297d740 1 --2- 172.21.15.100:0/3847453591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561797cc5d20 0x561797cc2d50 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8560035000 tx=0x7f8560049000 comp rx=0 tx=0).stop 2024-05-10T11:40:28.538 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.538+0000 7f857297d740 1 -- 172.21.15.100:0/3847453591 shutdown_connections 2024-05-10T11:40:28.538 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.538+0000 7f857297d740 1 --2- 172.21.15.100:0/3847453591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561797cc5d20 0x561797cc2d50 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:28.538 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.538+0000 7f857297d740 1 --2- 172.21.15.100:0/3847453591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561797cc0bb0 0x561797cba870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:28.538 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.538+0000 7f857297d740 1 --2- 172.21.15.100:0/3847453591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561797cc01c0 0x561797cba330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:28.538 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.538+0000 7f857297d740 1 -- 172.21.15.100:0/3847453591 >> 172.21.15.100:0/3847453591 conn(0x561797cb7420 msgr2=0x561797cb23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:40:28.540 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.541+0000 7f857297d740 1 -- 172.21.15.100:0/3847453591 shutdown_connections 2024-05-10T11:40:28.542 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.542+0000 7f857297d740 1 -- 172.21.15.100:0/3847453591 wait complete. 2024-05-10T11:40:28.543 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.543+0000 7f857297d740 1 -- start start 2024-05-10T11:40:28.543 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.543+0000 7f857297d740 1 Processor -- start 2024-05-10T11:40:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.544+0000 7f857297d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561797cc01c0 0x561797ccaa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.544+0000 7f857297d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561797cc0bb0 0x561797cccf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.544+0000 7f857297d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561797cc5d20 0x561797ccd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.544+0000 7f857297d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f85681a77e0 con 0x561797cc0bb0 2024-05-10T11:40:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.544+0000 7f857297d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f85681a8c50 con 0x561797cc5d20 2024-05-10T11:40:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.544+0000 7f857297d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f85681a7eb0 con 0x561797cc01c0 2024-05-10T11:40:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.544+0000 7f857297d740 10 client.0 _ll_register_callbacks cb 0x7f8564003d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:40:28.548 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.547+0000 7f856e126640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561797cc5d20 0x561797ccd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:28.549 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.547+0000 7f856e126640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561797cc5d20 0x561797ccd4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:51826/0 (socket says 172.21.15.100:51826) 2024-05-10T11:40:28.549 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.547+0000 7f856e126640 1 -- 172.21.15.100:0/3654886973 learned_addr learned my addr 172.21.15.100:0/3654886973 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:40:28.549 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.547+0000 7f856d925640 1 --2- 172.21.15.100:0/3654886973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561797cc01c0 0x561797ccaa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:28.549 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.548+0000 7f856d124640 1 --2- 172.21.15.100:0/3654886973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561797cc0bb0 0x561797cccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:28.549 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.548+0000 7f856d124640 1 -- 172.21.15.100:0/3654886973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561797cc01c0 msgr2=0x561797ccaa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:28.549 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.548+0000 7f856d124640 1 --2- 172.21.15.100:0/3654886973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561797cc01c0 0x561797ccaa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:28.549 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.548+0000 7f856d124640 1 -- 172.21.15.100:0/3654886973 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561797cc5d20 msgr2=0x561797ccd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:28.549 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.548+0000 7f856d124640 1 --2- 172.21.15.100:0/3654886973 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561797cc5d20 0x561797ccd4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:28.549 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.548+0000 7f856d124640 1 -- 172.21.15.100:0/3654886973 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561797ccda10 con 0x561797cc0bb0 2024-05-10T11:40:28.551 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.549+0000 7f856d124640 1 --2- 172.21.15.100:0/3654886973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561797cc0bb0 0x561797cccf90 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f8558000c00 tx=0x7f8558011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72acb2c7856c95e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:40:28.551 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.549+0000 7f85517fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:40:28.551 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.549+0000 7f85517fa640 1 -- 172.21.15.100:0/3654886973 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8558004050 con 0x561797cc0bb0 2024-05-10T11:40:28.551 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.549+0000 7f857297d740 1 -- 172.21.15.100:0/3654886973 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8560036050 con 0x561797cc0bb0 2024-05-10T11:40:28.551 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.550+0000 7f85517fa640 1 -- 172.21.15.100:0/3654886973 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f85580041f0 con 0x561797cc0bb0 2024-05-10T11:40:28.551 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.550+0000 7f85517fa640 1 -- 172.21.15.100:0/3654886973 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f855803c050 con 0x561797cc0bb0 2024-05-10T11:40:28.551 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.550+0000 7f857297d740 10 client.19707 fetch_fsmap learned FSMap version 41 2024-05-10T11:40:28.551 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.550+0000 7f857297d740 1 -- 172.21.15.100:0/3654886973 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x561797d4dfd0 con 0x561797cc0bb0 2024-05-10T11:40:28.551 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.550+0000 7f85517fa640 1 -- 172.21.15.100:0/3654886973 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8560036050 con 0x561797cc0bb0 2024-05-10T11:40:28.552 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.550+0000 7f85517fa640 1 -- 172.21.15.100:0/3654886973 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f85580047a0 con 0x561797cc0bb0 2024-05-10T11:40:28.552 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.551+0000 7f857297d740 10 client.19707 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:40:28.552 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.551+0000 7f857297d740 10 client.19707 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:40:28.552 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.551+0000 7f857297d740 20 client.19707 populate_metadata read hostname 'smithi100' 2024-05-10T11:40:28.552 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.551+0000 7f857297d740 1 --2- 172.21.15.100:0/3654886973 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x561797d4e960 0x561797d50d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:28.552 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.551+0000 7f857297d740 4 client.19707 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:40:28.552 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.551+0000 7f857297d740 1 -- 172.21.15.100:0/3654886973 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561797c28050 con 0x561797d4e960 2024-05-10T11:40:28.552 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.551+0000 7f856d925640 1 --2- 172.21.15.100:0/3654886973 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x561797d4e960 0x561797d50d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:28.553 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.552+0000 7f856d925640 1 --2- 172.21.15.100:0/3654886973 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x561797d4e960 0x561797d50d80 crc :-1 s=READY pgs=418 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44958762d781d6eb server_cookie=3eb8b5e1a677fcf7 in_seq=0 out_seq=0 2024-05-10T11:40:28.554 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.554+0000 7f85517fa640 10 client.19707 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:40:28.658 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.657+0000 7f85517fa640 1 -- 172.21.15.100:0/3654886973 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561797c28050 con 0x561797d4e960 2024-05-10T11:40:28.658 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.657+0000 7f85517fa640 10 client.19707 handle_command_reply: tid=0 2024-05-10T11:40:28.658 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.657+0000 7f857297d740 1 client.19707 shutdown 2024-05-10T11:40:28.658 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.657+0000 7f857297d740 1 -- 172.21.15.100:0/3654886973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561797cc0bb0 msgr2=0x561797cccf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:28.658 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.657+0000 7f857297d740 1 --2- 172.21.15.100:0/3654886973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561797cc0bb0 0x561797cccf90 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f8558000c00 tx=0x7f8558011e20 comp rx=0 tx=0).stop 2024-05-10T11:40:28.658 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.657+0000 7f857297d740 1 -- 172.21.15.100:0/3654886973 shutdown_connections 2024-05-10T11:40:28.658 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.657+0000 7f857297d740 1 --2- 172.21.15.100:0/3654886973 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x561797d4e960 0x561797d50d80 crc :-1 s=READY pgs=418 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:28.658 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.657+0000 7f857297d740 1 --2- 172.21.15.100:0/3654886973 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561797cc5d20 0x561797ccd4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:28.658 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.657+0000 7f857297d740 1 --2- 172.21.15.100:0/3654886973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561797cc0bb0 0x561797cccf90 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:28.658 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.657+0000 7f85517fa640 0 client.19707 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:40:28.658 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.657+0000 7f857297d740 1 --2- 172.21.15.100:0/3654886973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561797cc01c0 0x561797ccaa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:28.658 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.657+0000 7f857297d740 1 -- 172.21.15.100:0/3654886973 >> 172.21.15.100:0/3654886973 conn(0x561797cb7420 msgr2=0x561797cba330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:40:28.659 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.657+0000 7f857297d740 1 -- 172.21.15.100:0/3654886973 shutdown_connections 2024-05-10T11:40:28.659 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.657+0000 7f857297d740 1 -- 172.21.15.100:0/3654886973 wait complete. 2024-05-10T11:40:28.659 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.657+0000 7f857297d740 20 client.19707 trim_cache size 0 max 16384 2024-05-10T11:40:28.659 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:40:28.659 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:40:28.659 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:40:28.659 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:40:28.660 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.658+0000 7f856fb89640 1 -- 172.21.15.100:0/1835524012 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f85540b28a0 msgr2=0x7f85540b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:28.660 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.658+0000 7f856fb89640 1 --2- 172.21.15.100:0/1835524012 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f85540b28a0 0x7f85540b4d60 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f8560006010 tx=0x7f8560009360 comp rx=0 tx=0).stop 2024-05-10T11:40:28.660 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.658+0000 7f856fb89640 1 -- 172.21.15.100:0/1835524012 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8568101910 msgr2=0x7f856806abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:28.660 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.658+0000 7f856fb89640 1 --2- 172.21.15.100:0/1835524012 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8568101910 0x7f856806abd0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f856400e9c0 tx=0x7f856400ee80 comp rx=0 tx=0).stop 2024-05-10T11:40:28.660 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.658+0000 7f856fb89640 1 -- 172.21.15.100:0/1835524012 shutdown_connections 2024-05-10T11:40:28.660 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.658+0000 7f856fb89640 1 --2- 172.21.15.100:0/1835524012 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f85540b28a0 0x7f85540b4d60 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:28.660 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.658+0000 7f856fb89640 1 --2- 172.21.15.100:0/1835524012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85681034c0 0x7f856819fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:28.660 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.658+0000 7f856fb89640 1 --2- 172.21.15.100:0/1835524012 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8568102b00 0x7f856819d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:28.661 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.658+0000 7f856fb89640 1 --2- 172.21.15.100:0/1835524012 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8568101910 0x7f856806abd0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:28.661 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.658+0000 7f856fb89640 1 -- 172.21.15.100:0/1835524012 >> 172.21.15.100:0/1835524012 conn(0x7f85680fce80 msgr2=0x7f8568105e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:40:28.661 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.658+0000 7f856fb89640 1 -- 172.21.15.100:0/1835524012 shutdown_connections 2024-05-10T11:40:28.661 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:28.658+0000 7f856fb89640 1 -- 172.21.15.100:0/1835524012 wait complete. 2024-05-10T11:40:28.950 INFO:tasks.workunit.client.0.smithi100.stdout: 10 83920 8.99 MB/sec execute 382 sec latency 132.878 ms 2024-05-10T11:40:29.157 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T11:40:29.157 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T11:40:29.567 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:40:29.950 INFO:tasks.workunit.client.0.smithi100.stdout: 10 84080 8.99 MB/sec execute 383 sec latency 61.248 ms 2024-05-10T11:40:30.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:40:29 smithi188 ceph-mon[34075]: pgmap v1084: 129 pgs: 129 active+clean; 546 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 4.0 MiB/s wr, 261 op/s 2024-05-10T11:40:30.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:40:29 smithi100 ceph-mon[30296]: pgmap v1084: 129 pgs: 129 active+clean; 546 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 4.0 MiB/s wr, 261 op/s 2024-05-10T11:40:30.054 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:40:29 smithi178 ceph-mon[33359]: pgmap v1084: 129 pgs: 129 active+clean; 546 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 4.0 MiB/s wr, 261 op/s 2024-05-10T11:40:30.950 INFO:tasks.workunit.client.0.smithi100.stdout: 10 84248 9.00 MB/sec execute 384 sec latency 71.699 ms 2024-05-10T11:40:31.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:31.673+0000 7f8a1e0a4700 1 -- 172.21.15.100:0/3957386581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a181034f0 msgr2=0x7f8a18103970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:31.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:31.673+0000 7f8a1e0a4700 1 --2- 172.21.15.100:0/3957386581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a181034f0 0x7f8a18103970 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c009990 tx=0x7f8a0c009ca0 comp rx=0 tx=0).stop 2024-05-10T11:40:31.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:31.673+0000 7f8a1e0a4700 1 -- 172.21.15.100:0/3957386581 shutdown_connections 2024-05-10T11:40:31.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:31.673+0000 7f8a1e0a4700 1 --2- 172.21.15.100:0/3957386581 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a18103eb0 0x7f8a18110390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:31.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:31.673+0000 7f8a1e0a4700 1 --2- 172.21.15.100:0/3957386581 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a18109600 0x7f8a181099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:31.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:31.673+0000 7f8a1e0a4700 1 --2- 172.21.15.100:0/3957386581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a181034f0 0x7f8a18103970 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:31.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:31.673+0000 7f8a1e0a4700 1 -- 172.21.15.100:0/3957386581 >> 172.21.15.100:0/3957386581 conn(0x7f8a1807a650 msgr2=0x7f8a1807aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:40:31.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:31.673+0000 7f8a1e0a4700 1 -- 172.21.15.100:0/3957386581 shutdown_connections 2024-05-10T11:40:31.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:31.673+0000 7f8a1e0a4700 1 -- 172.21.15.100:0/3957386581 wait complete. 2024-05-10T11:40:31.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:31.674+0000 7f8a1e0a4700 1 Processor -- start 2024-05-10T11:40:31.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:31.674+0000 7f8a1e0a4700 1 -- start start 2024-05-10T11:40:31.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:31.674+0000 7f8a1e0a4700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a18103eb0 0x7f8a1810c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:31.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:31.675+0000 7f8a1e0a4700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a18109600 0x7f8a1810cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:31.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:31.675+0000 7f8a1e0a4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a1810d1a0 0x7f8a181a3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:31.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:31.675+0000 7f8a1e0a4700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8a18076210 con 0x7f8a1810d1a0 2024-05-10T11:40:31.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:31.675+0000 7f8a1e0a4700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8a18076090 con 0x7f8a18109600 2024-05-10T11:40:31.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:31.675+0000 7f8a1e0a4700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8a18076390 con 0x7f8a18103eb0 2024-05-10T11:40:31.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:31.675+0000 7f8a16ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a18109600 0x7f8a1810cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:31.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:31.675+0000 7f8a17fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a1810d1a0 0x7f8a181a3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:31.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:31.675+0000 7f8a17fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a1810d1a0 0x7f8a181a3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56554/0 (socket says 172.21.15.100:56554) 2024-05-10T11:40:31.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:31.675+0000 7f8a17fff700 1 -- 172.21.15.100:0/3732654709 learned_addr learned my addr 172.21.15.100:0/3732654709 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:40:31.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:31.675+0000 7f8a177fe700 1 --2- 172.21.15.100:0/3732654709 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a18103eb0 0x7f8a1810c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:31.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:31.676+0000 7f8a16ffd700 1 -- 172.21.15.100:0/3732654709 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a18103eb0 msgr2=0x7f8a1810c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:31.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:31.676+0000 7f8a16ffd700 1 --2- 172.21.15.100:0/3732654709 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a18103eb0 0x7f8a1810c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:31.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:31.676+0000 7f8a16ffd700 1 -- 172.21.15.100:0/3732654709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a1810d1a0 msgr2=0x7f8a181a3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:31.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:31.676+0000 7f8a16ffd700 1 --2- 172.21.15.100:0/3732654709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a1810d1a0 0x7f8a181a3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:31.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:31.676+0000 7f8a16ffd700 1 -- 172.21.15.100:0/3732654709 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a181b5320 con 0x7f8a18109600 2024-05-10T11:40:31.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:31.676+0000 7f8a16ffd700 1 --2- 172.21.15.100:0/3732654709 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a18109600 0x7f8a1810cc60 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c008000 tx=0x7f8a0c0047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:40:31.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:31.676+0000 7f8a14ff9700 1 -- 172.21.15.100:0/3732654709 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c026070 con 0x7f8a18109600 2024-05-10T11:40:31.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:31.676+0000 7f8a14ff9700 1 -- 172.21.15.100:0/3732654709 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a0c021410 con 0x7f8a18109600 2024-05-10T11:40:31.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:31.676+0000 7f8a14ff9700 1 -- 172.21.15.100:0/3732654709 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c021e60 con 0x7f8a18109600 2024-05-10T11:40:31.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:31.676+0000 7f8a1e0a4700 1 -- 172.21.15.100:0/3732654709 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a181b54f0 con 0x7f8a18109600 2024-05-10T11:40:31.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:31.677+0000 7f8a1e0a4700 1 -- 172.21.15.100:0/3732654709 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a181b5890 con 0x7f8a18109600 2024-05-10T11:40:31.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:31.678+0000 7f8a1e0a4700 1 -- 172.21.15.100:0/3732654709 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a18104c10 con 0x7f8a18109600 2024-05-10T11:40:31.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:31.679+0000 7f8a14ff9700 1 -- 172.21.15.100:0/3732654709 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f8a0c029070 con 0x7f8a18109600 2024-05-10T11:40:31.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:31.680+0000 7f8a14ff9700 1 --2- 172.21.15.100:0/3732654709 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8a000b2900 0x7f8a000b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:31.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:31.680+0000 7f8a14ff9700 1 -- 172.21.15.100:0/3732654709 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f8a0c0e2be0 con 0x7f8a18109600 2024-05-10T11:40:31.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:31.680+0000 7f8a177fe700 1 --2- 172.21.15.100:0/3732654709 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8a000b2900 0x7f8a000b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:31.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:31.681+0000 7f8a177fe700 1 --2- 172.21.15.100:0/3732654709 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8a000b2900 0x7f8a000b4dc0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f8a08009be0 tx=0x7f8a08009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:40:31.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:31.685+0000 7f8a14ff9700 1 -- 172.21.15.100:0/3732654709 <== mon.2 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f8a0c013070 con 0x7f8a18109600 2024-05-10T11:40:31.950 INFO:tasks.workunit.client.0.smithi100.stdout: 10 84400 8.99 MB/sec execute 385 sec latency 86.960 ms 2024-05-10T11:40:32.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:40:31 smithi188 ceph-mon[34075]: pgmap v1085: 129 pgs: 129 active+clean; 549 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 4.5 MiB/s wr, 274 op/s 2024-05-10T11:40:32.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:40:31 smithi100 ceph-mon[30296]: pgmap v1085: 129 pgs: 129 active+clean; 549 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 4.5 MiB/s wr, 274 op/s 2024-05-10T11:40:32.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:32.111+0000 7f8a1e0a4700 1 -- 172.21.15.100:0/3732654709 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8a181b5b50 con 0x7f8a18109600 2024-05-10T11:40:32.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:32.111+0000 7f8a14ff9700 1 -- 172.21.15.100:0/3732654709 <== mon.2 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+11897 (secure 0 0 0) 0x7f8a0c0aa5d0 con 0x7f8a18109600 2024-05-10T11:40:32.112 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:40:32.112 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2072225012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2072225012},{"type":"v1","addr":"172.21.15.100:6835","nonce":2072225012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/401939540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":401939540},{"type":"v1","addr":"172.21.15.100:6837","nonce":401939540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14571,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/3946311025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3946311025},{"type":"v1","addr":"172.21.15.100:6839","nonce":3946311025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3736371660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3736371660},{"type":"v1","addr":"172.21.15.100:6841","nonce":3736371660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/2741810791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2741810791},{"type":"v1","addr":"172.21.15.178:6833","nonce":2741810791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1704891041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1704891041},{"type":"v1","addr":"172.21.15.188:6833","nonce":1704891041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":41,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T11:14:27.242452+0000","modified":"2024-05-10T11:35:29.588072+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24439,"mds_1":24437,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/2124076312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2124076312},{"type":"v1","addr":"172.21.15.178:6835","nonce":2124076312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/3224358425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3224358425},{"type":"v1","addr":"172.21.15.178:6837","nonce":3224358425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/1943342620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1943342620},{"type":"v1","addr":"172.21.15.188:6837","nonce":1943342620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.178:6839/3760985404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3760985404},{"type":"v1","addr":"172.21.15.178:6839","nonce":3760985404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.188:6839/3194917070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":3194917070},{"type":"v1","addr":"172.21.15.188:6839","nonce":3194917070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27034":{"gid":27034,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/3139559450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3139559450},{"type":"v1","addr":"172.21.15.188:6835","nonce":3139559450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24439,"qdb_cluster":[24437,24439,24427]},"id":1}]} 2024-05-10T11:40:32.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:32.115+0000 7f8a1e0a4700 1 -- 172.21.15.100:0/3732654709 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8a000b2900 msgr2=0x7f8a000b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:32.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:32.115+0000 7f8a1e0a4700 1 --2- 172.21.15.100:0/3732654709 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8a000b2900 0x7f8a000b4dc0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f8a08009be0 tx=0x7f8a08009260 comp rx=0 tx=0).stop 2024-05-10T11:40:32.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:32.115+0000 7f8a1e0a4700 1 -- 172.21.15.100:0/3732654709 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a18109600 msgr2=0x7f8a1810cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:32.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:32.115+0000 7f8a1e0a4700 1 --2- 172.21.15.100:0/3732654709 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a18109600 0x7f8a1810cc60 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c008000 tx=0x7f8a0c0047b0 comp rx=0 tx=0).stop 2024-05-10T11:40:32.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:32.115+0000 7f8a1e0a4700 1 -- 172.21.15.100:0/3732654709 shutdown_connections 2024-05-10T11:40:32.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:32.115+0000 7f8a1e0a4700 1 --2- 172.21.15.100:0/3732654709 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8a000b2900 0x7f8a000b4dc0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:32.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:32.115+0000 7f8a1e0a4700 1 --2- 172.21.15.100:0/3732654709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a1810d1a0 0x7f8a181a3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:32.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:32.115+0000 7f8a1e0a4700 1 --2- 172.21.15.100:0/3732654709 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a18103eb0 0x7f8a1810c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:32.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:32.115+0000 7f8a1e0a4700 1 --2- 172.21.15.100:0/3732654709 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a18109600 0x7f8a1810cc60 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:32.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:32.115+0000 7f8a1e0a4700 1 -- 172.21.15.100:0/3732654709 >> 172.21.15.100:0/3732654709 conn(0x7f8a1807a650 msgr2=0x7f8a18101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:40:32.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:32.115+0000 7f8a1e0a4700 1 -- 172.21.15.100:0/3732654709 shutdown_connections 2024-05-10T11:40:32.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:32.115+0000 7f8a1e0a4700 1 -- 172.21.15.100:0/3732654709 wait complete. 2024-05-10T11:40:32.116 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 41 2024-05-10T11:40:32.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:40:31 smithi178 ceph-mon[33359]: pgmap v1085: 129 pgs: 129 active+clean; 549 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 4.5 MiB/s wr, 274 op/s 2024-05-10T11:40:32.925 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-10T11:40:32.950 INFO:tasks.workunit.client.0.smithi100.stdout: 10 84549 8.99 MB/sec execute 386 sec latency 94.329 ms 2024-05-10T11:40:33.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:40:32 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/3732654709' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:40:33.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:40:32 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/3732654709' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:40:33.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:40:32 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/3732654709' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:40:33.305 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:40:33.866 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:40:33 smithi100 ceph-mon[30296]: pgmap v1086: 129 pgs: 129 active+clean; 553 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 4.7 MiB/s wr, 267 op/s 2024-05-10T11:40:33.951 INFO:tasks.workunit.client.0.smithi100.stdout: 10 84682 8.97 MB/sec execute 387 sec latency 92.967 ms 2024-05-10T11:40:34.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:40:33 smithi188 ceph-mon[34075]: pgmap v1086: 129 pgs: 129 active+clean; 553 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 4.7 MiB/s wr, 267 op/s 2024-05-10T11:40:34.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:40:33 smithi178 ceph-mon[33359]: pgmap v1086: 129 pgs: 129 active+clean; 553 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 4.7 MiB/s wr, 267 op/s 2024-05-10T11:40:34.953 INFO:tasks.workunit.client.0.smithi100.stdout: 10 84829 8.96 MB/sec execute 388 sec latency 89.544 ms 2024-05-10T11:40:35.811 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:40:35 smithi100 ceph-mon[30296]: pgmap v1087: 129 pgs: 129 active+clean; 552 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 4.8 MiB/s wr, 241 op/s 2024-05-10T11:40:35.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.872+0000 7f41ec28f700 1 -- 172.21.15.100:0/3088040443 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41e41034f0 msgr2=0x7f41e4103970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:35.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.872+0000 7f41ec28f700 1 --2- 172.21.15.100:0/3088040443 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41e41034f0 0x7f41e4103970 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f41d4009990 tx=0x7f41d4009ca0 comp rx=0 tx=0).stop 2024-05-10T11:40:35.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.873+0000 7f41ec28f700 1 -- 172.21.15.100:0/3088040443 shutdown_connections 2024-05-10T11:40:35.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.873+0000 7f41ec28f700 1 --2- 172.21.15.100:0/3088040443 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41e4103eb0 0x7f41e4110390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:35.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.873+0000 7f41ec28f700 1 --2- 172.21.15.100:0/3088040443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41e4109600 0x7f41e41099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:35.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.873+0000 7f41ec28f700 1 --2- 172.21.15.100:0/3088040443 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41e41034f0 0x7f41e4103970 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:35.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.873+0000 7f41ec28f700 1 -- 172.21.15.100:0/3088040443 >> 172.21.15.100:0/3088040443 conn(0x7f41e407a650 msgr2=0x7f41e407aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:40:35.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.873+0000 7f41ec28f700 1 -- 172.21.15.100:0/3088040443 shutdown_connections 2024-05-10T11:40:35.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.873+0000 7f41ec28f700 1 -- 172.21.15.100:0/3088040443 wait complete. 2024-05-10T11:40:35.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.873+0000 7f41ec28f700 1 Processor -- start 2024-05-10T11:40:35.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.874+0000 7f41ec28f700 1 -- start start 2024-05-10T11:40:35.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.874+0000 7f41ec28f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41e41034f0 0x7f41e4077390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:35.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.874+0000 7f41ec28f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41e4103eb0 0x7f41e40757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:35.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.874+0000 7f41ec28f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41e4109600 0x7f41e4075e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:35.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.874+0000 7f41ea02b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41e41034f0 0x7f41e4077390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:35.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.874+0000 7f41ec28f700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f41e4113570 con 0x7f41e41034f0 2024-05-10T11:40:35.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.874+0000 7f41ea02b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41e41034f0 0x7f41e4077390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56584/0 (socket says 172.21.15.100:56584) 2024-05-10T11:40:35.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.874+0000 7f41ea02b700 1 -- 172.21.15.100:0/506504429 learned_addr learned my addr 172.21.15.100:0/506504429 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:40:35.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.874+0000 7f41ec28f700 1 -- 172.21.15.100:0/506504429 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f41e41133f0 con 0x7f41e4109600 2024-05-10T11:40:35.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.874+0000 7f41e982a700 1 --2- 172.21.15.100:0/506504429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41e4103eb0 0x7f41e40757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:35.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.874+0000 7f41ec28f700 1 -- 172.21.15.100:0/506504429 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f41e41136f0 con 0x7f41e4103eb0 2024-05-10T11:40:35.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.875+0000 7f41ea82c700 1 --2- 172.21.15.100:0/506504429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41e4109600 0x7f41e4075e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:35.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.875+0000 7f41e982a700 1 -- 172.21.15.100:0/506504429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41e4109600 msgr2=0x7f41e4075e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:35.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.875+0000 7f41e982a700 1 --2- 172.21.15.100:0/506504429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41e4109600 0x7f41e4075e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:35.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.875+0000 7f41e982a700 1 -- 172.21.15.100:0/506504429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41e41034f0 msgr2=0x7f41e4077390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:35.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.875+0000 7f41e982a700 1 --2- 172.21.15.100:0/506504429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41e41034f0 0x7f41e4077390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:35.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.875+0000 7f41e982a700 1 -- 172.21.15.100:0/506504429 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41e41ac490 con 0x7f41e4103eb0 2024-05-10T11:40:35.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.875+0000 7f41e982a700 1 --2- 172.21.15.100:0/506504429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41e4103eb0 0x7f41e40757a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f41d40098e0 tx=0x7f41d40047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:40:35.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.875+0000 7f41db7fe700 1 -- 172.21.15.100:0/506504429 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d4026070 con 0x7f41e4103eb0 2024-05-10T11:40:35.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.875+0000 7f41db7fe700 1 -- 172.21.15.100:0/506504429 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f41d4021410 con 0x7f41e4103eb0 2024-05-10T11:40:35.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.876+0000 7f41db7fe700 1 -- 172.21.15.100:0/506504429 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d4017650 con 0x7f41e4103eb0 2024-05-10T11:40:35.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.876+0000 7f41ec28f700 1 -- 172.21.15.100:0/506504429 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41e41ac6c0 con 0x7f41e4103eb0 2024-05-10T11:40:35.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.876+0000 7f41ec28f700 1 -- 172.21.15.100:0/506504429 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41e41aca60 con 0x7f41e4103eb0 2024-05-10T11:40:35.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.879+0000 7f41db7fe700 1 -- 172.21.15.100:0/506504429 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f41d4029070 con 0x7f41e4103eb0 2024-05-10T11:40:35.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.880+0000 7f41db7fe700 1 --2- 172.21.15.100:0/506504429 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f41d00b2980 0x7f41d00b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:35.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.881+0000 7f41db7fe700 1 -- 172.21.15.100:0/506504429 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f41d40e3230 con 0x7f41e4103eb0 2024-05-10T11:40:35.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.881+0000 7f41ea02b700 1 --2- 172.21.15.100:0/506504429 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f41d00b2980 0x7f41d00b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:35.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.881+0000 7f41ea02b700 1 --2- 172.21.15.100:0/506504429 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f41d00b2980 0x7f41d00b4e40 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f41e0009ce0 tx=0x7f41e0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:40:35.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.882+0000 7f41f6b22b80 1 Processor -- start 2024-05-10T11:40:35.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.882+0000 7f41f6b22b80 1 -- start start 2024-05-10T11:40:35.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.882+0000 7f41f6b22b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564719c141b0 0x564719c14590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:35.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.882+0000 7f41f6b22b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564719c14ad0 0x564719c0e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:35.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.882+0000 7f41f6b22b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564719c0ed50 0x564719c0f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:35.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.883+0000 7f41f6b22b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x564719b60940 con 0x564719c14ad0 2024-05-10T11:40:35.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.883+0000 7f41f6b22b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x564719ac5da0 con 0x564719c0ed50 2024-05-10T11:40:35.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.883+0000 7f41f6b22b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x564719b5c900 con 0x564719c141b0 2024-05-10T11:40:35.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.883+0000 7f41ea02b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564719c141b0 0x564719c14590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:35.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.883+0000 7f41ea82c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564719c0ed50 0x564719c0f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:35.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.883+0000 7f41e982a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564719c14ad0 0x564719c0e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:35.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.883+0000 7f41ea02b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564719c141b0 0x564719c14590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:43638/0 (socket says 172.21.15.100:43638) 2024-05-10T11:40:35.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.883+0000 7f41ea02b700 1 -- 172.21.15.100:0/2458820182 learned_addr learned my addr 172.21.15.100:0/2458820182 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:40:35.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.883+0000 7f41ea02b700 1 -- 172.21.15.100:0/2458820182 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564719c0ed50 msgr2=0x564719c0f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:35.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.883+0000 7f41ea02b700 1 --2- 172.21.15.100:0/2458820182 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564719c0ed50 0x564719c0f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:35.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.883+0000 7f41ea02b700 1 -- 172.21.15.100:0/2458820182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564719c14ad0 msgr2=0x564719c0e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:35.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.883+0000 7f41ea02b700 1 --2- 172.21.15.100:0/2458820182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564719c14ad0 0x564719c0e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:35.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.883+0000 7f41ea02b700 1 -- 172.21.15.100:0/2458820182 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564719aad8d0 con 0x564719c141b0 2024-05-10T11:40:35.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.884+0000 7f41ea02b700 1 --2- 172.21.15.100:0/2458820182 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564719c141b0 0x564719c14590 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f41e000f550 tx=0x7f41e0014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=215bf2dbb1db2b77 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:40:35.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.884+0000 7f41d8d70700 1 -- 172.21.15.100:0/2458820182 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e000fcf0 con 0x564719c141b0 2024-05-10T11:40:35.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.884+0000 7f41d8d70700 1 -- 172.21.15.100:0/2458820182 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f41e000fe90 con 0x564719c141b0 2024-05-10T11:40:35.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.884+0000 7f41d8d70700 1 -- 172.21.15.100:0/2458820182 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e001c7b0 con 0x564719c141b0 2024-05-10T11:40:35.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.885+0000 7f41f6b22b80 1 -- 172.21.15.100:0/2458820182 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564719c141b0 msgr2=0x564719c14590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:35.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.885+0000 7f41f6b22b80 1 --2- 172.21.15.100:0/2458820182 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564719c141b0 0x564719c14590 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f41e000f550 tx=0x7f41e0014000 comp rx=0 tx=0).stop 2024-05-10T11:40:35.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.885+0000 7f41f6b22b80 1 -- 172.21.15.100:0/2458820182 shutdown_connections 2024-05-10T11:40:35.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.885+0000 7f41f6b22b80 1 --2- 172.21.15.100:0/2458820182 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564719c0ed50 0x564719c0f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:35.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.885+0000 7f41f6b22b80 1 --2- 172.21.15.100:0/2458820182 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564719c141b0 0x564719c14590 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:35.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.885+0000 7f41f6b22b80 1 --2- 172.21.15.100:0/2458820182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564719c14ad0 0x564719c0e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:35.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.885+0000 7f41f6b22b80 1 -- 172.21.15.100:0/2458820182 >> 172.21.15.100:0/2458820182 conn(0x564719b3ab70 msgr2=0x564719c092a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:40:35.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.885+0000 7f41f6b22b80 1 -- 172.21.15.100:0/2458820182 shutdown_connections 2024-05-10T11:40:35.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.885+0000 7f41f6b22b80 1 -- 172.21.15.100:0/2458820182 wait complete. 2024-05-10T11:40:35.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.886+0000 7f41f6b22b80 1 -- start start 2024-05-10T11:40:35.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.886+0000 7f41f6b22b80 1 Processor -- start 2024-05-10T11:40:35.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.886+0000 7f41f6b22b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564719c0ed50 0x564719c1a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:35.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.886+0000 7f41f6b22b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564719c141b0 0x564719c17af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:35.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.886+0000 7f41f6b22b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564719c14ad0 0x564719c18030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:35.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.886+0000 7f41f6b22b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x564719c1b450 con 0x564719c141b0 2024-05-10T11:40:35.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.886+0000 7f41f6b22b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x564719b19ed0 con 0x564719c14ad0 2024-05-10T11:40:35.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.886+0000 7f41f6b22b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x564719c1b5d0 con 0x564719c0ed50 2024-05-10T11:40:35.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.886+0000 7f41e982a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564719c141b0 0x564719c17af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:35.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.886+0000 7f41e982a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564719c141b0 0x564719c17af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56596/0 (socket says 172.21.15.100:56596) 2024-05-10T11:40:35.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.886+0000 7f41ea02b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564719c0ed50 0x564719c1a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:35.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.886+0000 7f41e982a700 1 -- 172.21.15.100:0/609256181 learned_addr learned my addr 172.21.15.100:0/609256181 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:40:35.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.886+0000 7f41f6b22b80 10 client.0 _ll_register_callbacks cb 0x564719af45b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:40:35.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.886+0000 7f41ea82c700 1 --2- 172.21.15.100:0/609256181 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564719c14ad0 0x564719c18030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:35.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.886+0000 7f41e982a700 1 -- 172.21.15.100:0/609256181 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564719c0ed50 msgr2=0x564719c1a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:35.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.886+0000 7f41e982a700 1 --2- 172.21.15.100:0/609256181 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564719c0ed50 0x564719c1a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:35.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.887+0000 7f41e982a700 1 -- 172.21.15.100:0/609256181 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564719c14ad0 msgr2=0x564719c18030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:35.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.887+0000 7f41e982a700 1 --2- 172.21.15.100:0/609256181 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564719c14ad0 0x564719c18030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:35.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.887+0000 7f41e982a700 1 -- 172.21.15.100:0/609256181 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564719b17760 con 0x564719c141b0 2024-05-10T11:40:35.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.887+0000 7f41ea82c700 1 --2- 172.21.15.100:0/609256181 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564719c14ad0 0x564719c18030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:40:35.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.887+0000 7f41e982a700 1 --2- 172.21.15.100:0/609256181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564719c141b0 0x564719c17af0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f41d4000c00 tx=0x7f41d40a9ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4550a8644faf69c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:40:35.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.887+0000 7f41cdffb700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:40:35.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.887+0000 7f41cdffb700 1 -- 172.21.15.100:0/609256181 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d40aa520 con 0x564719c141b0 2024-05-10T11:40:35.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.887+0000 7f41f6b22b80 1 -- 172.21.15.100:0/609256181 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41e000fe90 con 0x564719c141b0 2024-05-10T11:40:35.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.887+0000 7f41cdffb700 1 -- 172.21.15.100:0/609256181 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f41d40920e0 con 0x564719c141b0 2024-05-10T11:40:35.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.887+0000 7f41cdffb700 1 -- 172.21.15.100:0/609256181 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f41d40ae8d0 con 0x564719c141b0 2024-05-10T11:40:35.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.887+0000 7f41f6b22b80 10 client.19725 fetch_fsmap learned FSMap version 41 2024-05-10T11:40:35.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.887+0000 7f41f6b22b80 1 -- 172.21.15.100:0/609256181 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x564719c25d00 con 0x564719c141b0 2024-05-10T11:40:35.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.888+0000 7f41cdffb700 1 -- 172.21.15.100:0/609256181 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f41e000fe90 con 0x564719c141b0 2024-05-10T11:40:35.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.888+0000 7f41cdffb700 1 -- 172.21.15.100:0/609256181 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f41d40b2e60 con 0x564719c141b0 2024-05-10T11:40:35.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.888+0000 7f41f6b22b80 10 client.19725 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:40:35.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.888+0000 7f41f6b22b80 10 client.19725 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-10T11:40:35.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.888+0000 7f41f6b22b80 20 client.19725 populate_metadata read hostname 'smithi100' 2024-05-10T11:40:35.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.888+0000 7f41f6b22b80 1 --2- 172.21.15.100:0/609256181 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x564719c282c0 0x564719c2a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:35.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.888+0000 7f41f6b22b80 4 client.19725 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:40:35.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.888+0000 7f41f6b22b80 1 -- 172.21.15.100:0/609256181 --> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564719b6e320 con 0x564719c282c0 2024-05-10T11:40:35.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.888+0000 7f41ea02b700 1 --2- 172.21.15.100:0/609256181 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x564719c282c0 0x564719c2a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:35.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.889+0000 7f41ea02b700 1 --2- 172.21.15.100:0/609256181 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x564719c282c0 0x564719c2a700 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=331e67b6d5dd57a2 server_cookie=ecbd13e92fdbc152 in_seq=0 out_seq=0 2024-05-10T11:40:35.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.889+0000 7f41cdffb700 10 client.19725 ms_handle_connect on v2:172.21.15.188:6836/1943342620 2024-05-10T11:40:35.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.890+0000 7f41cdffb700 1 -- 172.21.15.100:0/609256181 <== mds.2 v2:172.21.15.188:6836/1943342620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564719b6e320 con 0x564719c282c0 2024-05-10T11:40:35.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.890+0000 7f41cdffb700 10 client.19725 handle_command_reply: tid=0 2024-05-10T11:40:35.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.890+0000 7f41f6b22b80 1 client.19725 shutdown 2024-05-10T11:40:35.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.890+0000 7f41f6b22b80 1 -- 172.21.15.100:0/609256181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564719c141b0 msgr2=0x564719c17af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:35.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.890+0000 7f41f6b22b80 1 --2- 172.21.15.100:0/609256181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564719c141b0 0x564719c17af0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f41d4000c00 tx=0x7f41d40a9ef0 comp rx=0 tx=0).stop 2024-05-10T11:40:35.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.890+0000 7f41f6b22b80 1 -- 172.21.15.100:0/609256181 shutdown_connections 2024-05-10T11:40:35.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.890+0000 7f41f6b22b80 1 --2- 172.21.15.100:0/609256181 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x564719c282c0 0x564719c2a700 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:35.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.890+0000 7f41f6b22b80 1 --2- 172.21.15.100:0/609256181 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564719c14ad0 0x564719c18030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:35.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.890+0000 7f41f6b22b80 1 --2- 172.21.15.100:0/609256181 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564719c0ed50 0x564719c1a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:35.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.890+0000 7f41f6b22b80 1 --2- 172.21.15.100:0/609256181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564719c141b0 0x564719c17af0 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:35.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.890+0000 7f41f6b22b80 1 -- 172.21.15.100:0/609256181 >> 172.21.15.100:0/609256181 conn(0x564719b3ab70 msgr2=0x564719c03b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:40:35.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.890+0000 7f41cdffb700 0 client.19725 ms_handle_reset on v2:172.21.15.188:6836/1943342620 2024-05-10T11:40:35.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.890+0000 7f41f6b22b80 1 -- 172.21.15.100:0/609256181 shutdown_connections 2024-05-10T11:40:35.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.890+0000 7f41f6b22b80 1 -- 172.21.15.100:0/609256181 wait complete. 2024-05-10T11:40:35.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.891+0000 7f41f6b22b80 20 client.19725 trim_cache size 0 max 16384 2024-05-10T11:40:35.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.915+0000 7f41f6b22b80 1 Processor -- start 2024-05-10T11:40:35.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.915+0000 7f41f6b22b80 1 -- start start 2024-05-10T11:40:35.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.915+0000 7f41f6b22b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564719c0ed50 0x564719c0e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:35.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.915+0000 7f41f6b22b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564719c141b0 0x564719c244e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:35.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.915+0000 7f41f6b22b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564719c14ad0 0x564719c03c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:35.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.915+0000 7f41f6b22b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f41e41b7ec0 con 0x564719c14ad0 2024-05-10T11:40:35.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.915+0000 7f41f6b22b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f41e41b5750 con 0x564719c0ed50 2024-05-10T11:40:35.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.915+0000 7f41f6b22b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f41e41b35a0 con 0x564719c141b0 2024-05-10T11:40:35.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.916+0000 7f41e982a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564719c141b0 0x564719c244e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:35.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.916+0000 7f41ea82c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564719c14ad0 0x564719c03c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:35.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.916+0000 7f41e982a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564719c141b0 0x564719c244e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:43644/0 (socket says 172.21.15.100:43644) 2024-05-10T11:40:35.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.916+0000 7f41e982a700 1 -- 172.21.15.100:0/4069248106 learned_addr learned my addr 172.21.15.100:0/4069248106 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:40:35.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.916+0000 7f41ea02b700 1 --2- 172.21.15.100:0/4069248106 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564719c0ed50 0x564719c0e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:35.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.916+0000 7f41e982a700 1 -- 172.21.15.100:0/4069248106 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564719c0ed50 msgr2=0x564719c0e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:35.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.916+0000 7f41e982a700 1 --2- 172.21.15.100:0/4069248106 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564719c0ed50 0x564719c0e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:35.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.916+0000 7f41e982a700 1 -- 172.21.15.100:0/4069248106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564719c14ad0 msgr2=0x564719c03c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:35.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.916+0000 7f41e982a700 1 --2- 172.21.15.100:0/4069248106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564719c14ad0 0x564719c03c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:35.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.916+0000 7f41e982a700 1 -- 172.21.15.100:0/4069248106 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564719c0f650 con 0x564719c141b0 2024-05-10T11:40:35.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.916+0000 7f41e982a700 1 --2- 172.21.15.100:0/4069248106 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564719c141b0 0x564719c244e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f41d40a6ac0 tx=0x7f41d40e3160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ba9b87378742291 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:40:35.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.916+0000 7f41cffff700 1 -- 172.21.15.100:0/4069248106 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d40aa520 con 0x564719c141b0 2024-05-10T11:40:35.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.917+0000 7f41cffff700 1 -- 172.21.15.100:0/4069248106 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f41d40b32f0 con 0x564719c141b0 2024-05-10T11:40:35.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.917+0000 7f41cffff700 1 -- 172.21.15.100:0/4069248106 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d40a9a90 con 0x564719c141b0 2024-05-10T11:40:35.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.917+0000 7f41f6b22b80 1 -- 172.21.15.100:0/4069248106 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564719c141b0 msgr2=0x564719c244e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:35.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.917+0000 7f41f6b22b80 1 --2- 172.21.15.100:0/4069248106 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564719c141b0 0x564719c244e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f41d40a6ac0 tx=0x7f41d40e3160 comp rx=0 tx=0).stop 2024-05-10T11:40:35.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.918+0000 7f41f6b22b80 1 -- 172.21.15.100:0/4069248106 shutdown_connections 2024-05-10T11:40:35.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.918+0000 7f41f6b22b80 1 --2- 172.21.15.100:0/4069248106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564719c14ad0 0x564719c03c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:35.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.918+0000 7f41f6b22b80 1 --2- 172.21.15.100:0/4069248106 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564719c0ed50 0x564719c0e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:35.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.918+0000 7f41f6b22b80 1 --2- 172.21.15.100:0/4069248106 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564719c141b0 0x564719c244e0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:35.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.918+0000 7f41f6b22b80 1 -- 172.21.15.100:0/4069248106 >> 172.21.15.100:0/4069248106 conn(0x564719b3ab70 msgr2=0x564719c14eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:40:35.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.918+0000 7f41f6b22b80 1 -- 172.21.15.100:0/4069248106 shutdown_connections 2024-05-10T11:40:35.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.918+0000 7f41f6b22b80 1 -- 172.21.15.100:0/4069248106 wait complete. 2024-05-10T11:40:35.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.918+0000 7f41f6b22b80 1 -- start start 2024-05-10T11:40:35.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.918+0000 7f41f6b22b80 1 Processor -- start 2024-05-10T11:40:35.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.919+0000 7f41f6b22b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564719c0ed50 0x564719c305b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:35.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.919+0000 7f41f6b22b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564719c141b0 0x564719c20860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:35.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.919+0000 7f41ea02b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564719c0ed50 0x564719c305b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:35.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.919+0000 7f41ea02b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564719c0ed50 0x564719c305b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:51902/0 (socket says 172.21.15.100:51902) 2024-05-10T11:40:35.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.919+0000 7f41e982a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564719c141b0 0x564719c20860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:35.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.919+0000 7f41e982a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564719c141b0 0x564719c20860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:43660/0 (socket says 172.21.15.100:43660) 2024-05-10T11:40:35.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.919+0000 7f41ea02b700 1 -- 172.21.15.100:0/292001013 learned_addr learned my addr 172.21.15.100:0/292001013 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:40:35.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.919+0000 7f41f6b22b80 1 --2- 172.21.15.100:0/292001013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564719c14ad0 0x564719c20e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:35.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.919+0000 7f41f6b22b80 1 -- 172.21.15.100:0/292001013 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f41e41b3c10 con 0x564719c14ad0 2024-05-10T11:40:35.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.919+0000 7f41f6b22b80 1 -- 172.21.15.100:0/292001013 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f41e41b5080 con 0x564719c0ed50 2024-05-10T11:40:35.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.919+0000 7f41f6b22b80 1 -- 172.21.15.100:0/292001013 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f41e41b42e0 con 0x564719c141b0 2024-05-10T11:40:35.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.920+0000 7f41ea82c700 1 --2- 172.21.15.100:0/292001013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564719c14ad0 0x564719c20e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:35.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.920+0000 7f41ea02b700 1 -- 172.21.15.100:0/292001013 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564719c141b0 msgr2=0x564719c20860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:35.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.920+0000 7f41ea02b700 1 --2- 172.21.15.100:0/292001013 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564719c141b0 0x564719c20860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:35.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.920+0000 7f41ea02b700 1 -- 172.21.15.100:0/292001013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564719c14ad0 msgr2=0x564719c20e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:35.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.920+0000 7f41ea02b700 1 --2- 172.21.15.100:0/292001013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564719c14ad0 0x564719c20e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:35.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.920+0000 7f41ea02b700 1 -- 172.21.15.100:0/292001013 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564719c2f1a0 con 0x564719c0ed50 2024-05-10T11:40:35.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.920+0000 7f41f6b22b80 10 client.0 _ll_register_callbacks cb 0x564719af45b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:40:35.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.920+0000 7f41f6b22b80 1 -- 172.21.15.100:0/292001013 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41d40b32f0 con 0x564719c0ed50 2024-05-10T11:40:35.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.920+0000 7f41ea02b700 1 --2- 172.21.15.100:0/292001013 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564719c0ed50 0x564719c305b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f41e000fb40 tx=0x7f41e000ff20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=441c46a0a851d85b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:40:35.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.921+0000 7f41cdffb700 10 client.28985 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:40:35.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.921+0000 7f41cdffb700 1 -- 172.21.15.100:0/292001013 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e0005310 con 0x564719c0ed50 2024-05-10T11:40:35.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.921+0000 7f41cdffb700 1 -- 172.21.15.100:0/292001013 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f41e00054b0 con 0x564719c0ed50 2024-05-10T11:40:35.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.921+0000 7f41cdffb700 1 -- 172.21.15.100:0/292001013 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f41e0034440 con 0x564719c0ed50 2024-05-10T11:40:35.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.921+0000 7f41f6b22b80 10 client.28985 fetch_fsmap learned FSMap version 41 2024-05-10T11:40:35.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.921+0000 7f41f6b22b80 1 -- 172.21.15.100:0/292001013 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({fsmap=41}) -- 0x564719c91ef0 con 0x564719c0ed50 2024-05-10T11:40:35.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.921+0000 7f41cdffb700 1 -- 172.21.15.100:0/292001013 <== mon.2 v2:172.21.15.178:3300/0 4 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f41e00345e0 con 0x564719c0ed50 2024-05-10T11:40:35.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.922+0000 7f41f6b22b80 10 client.28985 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:40:35.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.922+0000 7f41f6b22b80 10 client.28985 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-10T11:40:35.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.922+0000 7f41f6b22b80 20 client.28985 populate_metadata read hostname 'smithi100' 2024-05-10T11:40:35.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.922+0000 7f41f6b22b80 1 --2- 172.21.15.100:0/292001013 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x564719c92210 0x564719c94630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:35.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.922+0000 7f41f6b22b80 4 client.28985 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:40:35.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.922+0000 7f41f6b22b80 1 -- 172.21.15.100:0/292001013 --> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564719b683c0 con 0x564719c92210 2024-05-10T11:40:35.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.922+0000 7f41e982a700 1 --2- 172.21.15.100:0/292001013 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x564719c92210 0x564719c94630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:35.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.923+0000 7f41e982a700 1 --2- 172.21.15.100:0/292001013 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x564719c92210 0x564719c94630 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b6037c7e65b97ff5 server_cookie=9809281ba22e5397 in_seq=0 out_seq=0 2024-05-10T11:40:35.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.923+0000 7f41cdffb700 10 client.28985 ms_handle_connect on v2:172.21.15.188:6836/1943342620 2024-05-10T11:40:35.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.923+0000 7f41cdffb700 1 -- 172.21.15.100:0/292001013 <== mds.2 v2:172.21.15.188:6836/1943342620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564719b683c0 con 0x564719c92210 2024-05-10T11:40:35.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.923+0000 7f41cdffb700 10 client.28985 handle_command_reply: tid=0 2024-05-10T11:40:35.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.923+0000 7f41f6b22b80 1 client.28985 shutdown 2024-05-10T11:40:35.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.924+0000 7f41f6b22b80 1 -- 172.21.15.100:0/292001013 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564719c0ed50 msgr2=0x564719c305b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:35.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.924+0000 7f41f6b22b80 1 --2- 172.21.15.100:0/292001013 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564719c0ed50 0x564719c305b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f41e000fb40 tx=0x7f41e000ff20 comp rx=0 tx=0).stop 2024-05-10T11:40:35.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.924+0000 7f41f6b22b80 1 -- 172.21.15.100:0/292001013 shutdown_connections 2024-05-10T11:40:35.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.924+0000 7f41f6b22b80 1 --2- 172.21.15.100:0/292001013 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x564719c92210 0x564719c94630 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:35.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.924+0000 7f41f6b22b80 1 --2- 172.21.15.100:0/292001013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564719c14ad0 0x564719c20e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:35.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.924+0000 7f41f6b22b80 1 --2- 172.21.15.100:0/292001013 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564719c0ed50 0x564719c305b0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:35.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.924+0000 7f41f6b22b80 1 --2- 172.21.15.100:0/292001013 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564719c141b0 0x564719c20860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:35.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.924+0000 7f41f6b22b80 1 -- 172.21.15.100:0/292001013 >> 172.21.15.100:0/292001013 conn(0x564719b3ab70 msgr2=0x564719c12bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:40:35.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.925+0000 7f41cdffb700 0 client.28985 ms_handle_reset on v2:172.21.15.188:6836/1943342620 2024-05-10T11:40:35.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.925+0000 7f41f6b22b80 1 -- 172.21.15.100:0/292001013 shutdown_connections 2024-05-10T11:40:35.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.925+0000 7f41f6b22b80 1 -- 172.21.15.100:0/292001013 wait complete. 2024-05-10T11:40:35.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.925+0000 7f41f6b22b80 20 client.28985 trim_cache size 0 max 16384 2024-05-10T11:40:35.927 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:40:35.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.926+0000 7f41ec28f700 1 -- 172.21.15.100:0/506504429 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f41d00b2980 msgr2=0x7f41d00b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:35.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.926+0000 7f41ec28f700 1 --2- 172.21.15.100:0/506504429 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f41d00b2980 0x7f41d00b4e40 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f41e0009ce0 tx=0x7f41e0009260 comp rx=0 tx=0).stop 2024-05-10T11:40:35.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.926+0000 7f41ec28f700 1 -- 172.21.15.100:0/506504429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41e4103eb0 msgr2=0x7f41e40757a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:35.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.926+0000 7f41ec28f700 1 --2- 172.21.15.100:0/506504429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41e4103eb0 0x7f41e40757a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f41d40098e0 tx=0x7f41d40047b0 comp rx=0 tx=0).stop 2024-05-10T11:40:35.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.926+0000 7f41ec28f700 1 -- 172.21.15.100:0/506504429 shutdown_connections 2024-05-10T11:40:35.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.926+0000 7f41ec28f700 1 --2- 172.21.15.100:0/506504429 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f41d00b2980 0x7f41d00b4e40 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:35.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.926+0000 7f41ec28f700 1 --2- 172.21.15.100:0/506504429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41e4109600 0x7f41e4075e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:35.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.926+0000 7f41ec28f700 1 --2- 172.21.15.100:0/506504429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41e41034f0 0x7f41e4077390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:35.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.926+0000 7f41ec28f700 1 --2- 172.21.15.100:0/506504429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41e4103eb0 0x7f41e40757a0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:35.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.926+0000 7f41ec28f700 1 -- 172.21.15.100:0/506504429 >> 172.21.15.100:0/506504429 conn(0x7f41e407a650 msgr2=0x7f41e41021f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:40:35.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.927+0000 7f41ec28f700 1 -- 172.21.15.100:0/506504429 shutdown_connections 2024-05-10T11:40:35.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:35.927+0000 7f41ec28f700 1 -- 172.21.15.100:0/506504429 wait complete. 2024-05-10T11:40:35.953 INFO:tasks.workunit.client.0.smithi100.stdout: 10 84981 8.96 MB/sec execute 389 sec latency 90.232 ms 2024-05-10T11:40:36.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:40:35 smithi188 ceph-mon[34075]: pgmap v1087: 129 pgs: 129 active+clean; 552 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 4.8 MiB/s wr, 241 op/s 2024-05-10T11:40:36.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:40:35 smithi178 ceph-mon[33359]: pgmap v1087: 129 pgs: 129 active+clean; 552 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 4.8 MiB/s wr, 241 op/s 2024-05-10T11:40:36.807 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-10T11:40:36.953 INFO:tasks.workunit.client.0.smithi100.stdout: 10 85123 8.95 MB/sec execute 390 sec latency 96.589 ms 2024-05-10T11:40:37.229 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:40:37.953 INFO:tasks.workunit.client.0.smithi100.stdout: 10 85285 8.95 MB/sec execute 391 sec latency 122.105 ms 2024-05-10T11:40:38.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:40:37 smithi100 ceph-mon[30296]: pgmap v1088: 129 pgs: 129 active+clean; 550 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 996 KiB/s rd, 5.0 MiB/s wr, 291 op/s 2024-05-10T11:40:38.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:40:37 smithi178 ceph-mon[33359]: pgmap v1088: 129 pgs: 129 active+clean; 550 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 996 KiB/s rd, 5.0 MiB/s wr, 291 op/s 2024-05-10T11:40:38.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:40:37 smithi188 ceph-mon[34075]: pgmap v1088: 129 pgs: 129 active+clean; 550 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 996 KiB/s rd, 5.0 MiB/s wr, 291 op/s 2024-05-10T11:40:38.953 INFO:tasks.workunit.client.0.smithi100.stdout: 10 85465 8.95 MB/sec execute 392 sec latency 130.864 ms 2024-05-10T11:40:39.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.191+0000 7f702effd700 1 -- 172.21.15.100:0/1131756479 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70281023a0 msgr2=0x7f7028102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:39.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.191+0000 7f702effd700 1 --2- 172.21.15.100:0/1131756479 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70281023a0 0x7f7028102820 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f70200098f0 tx=0x7f7020009c00 comp rx=0 tx=0).stop 2024-05-10T11:40:39.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.191+0000 7f702effd700 1 -- 172.21.15.100:0/1131756479 shutdown_connections 2024-05-10T11:40:39.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.191+0000 7f702effd700 1 --2- 172.21.15.100:0/1131756479 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7028102d60 0x7f702810f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:39.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.191+0000 7f702effd700 1 --2- 172.21.15.100:0/1131756479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7028108410 0x7f70281087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:39.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.191+0000 7f702effd700 1 --2- 172.21.15.100:0/1131756479 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70281023a0 0x7f7028102820 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:39.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.191+0000 7f702effd700 1 -- 172.21.15.100:0/1131756479 >> 172.21.15.100:0/1131756479 conn(0x7f7028079350 msgr2=0x7f7028079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:40:39.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.192+0000 7f702effd700 1 -- 172.21.15.100:0/1131756479 shutdown_connections 2024-05-10T11:40:39.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.192+0000 7f702effd700 1 -- 172.21.15.100:0/1131756479 wait complete. 2024-05-10T11:40:39.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.193+0000 7f702effd700 1 Processor -- start 2024-05-10T11:40:39.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.193+0000 7f702effd700 1 -- start start 2024-05-10T11:40:39.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.193+0000 7f702effd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70281023a0 0x7f70281a4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:39.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.193+0000 7f702effd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7028102d60 0x7f70281a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:39.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.194+0000 7f702effd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7028108410 0x7f70281a0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:39.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.194+0000 7f702effd700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7028112480 con 0x7f7028108410 2024-05-10T11:40:39.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.194+0000 7f702effd700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7028112300 con 0x7f7028102d60 2024-05-10T11:40:39.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.194+0000 7f702effd700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7028112600 con 0x7f70281023a0 2024-05-10T11:40:39.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.194+0000 7f7027fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7028102d60 0x7f70281a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:39.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.194+0000 7f7027fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7028102d60 0x7f70281a5220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:54882/0 (socket says 172.21.15.100:54882) 2024-05-10T11:40:39.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.194+0000 7f702d59a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7028108410 0x7f70281a0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:39.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.194+0000 7f7027fff700 1 -- 172.21.15.100:0/3542172952 learned_addr learned my addr 172.21.15.100:0/3542172952 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:40:39.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.194+0000 7f7027fff700 1 -- 172.21.15.100:0/3542172952 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70281023a0 msgr2=0x7f70281a4ce0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-10T11:40:39.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.194+0000 7f7027fff700 1 --2- 172.21.15.100:0/3542172952 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70281023a0 0x7f70281a4ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:39.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.194+0000 7f7027fff700 1 -- 172.21.15.100:0/3542172952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7028108410 msgr2=0x7f70281a0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:39.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.194+0000 7f7027fff700 1 --2- 172.21.15.100:0/3542172952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7028108410 0x7f70281a0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:39.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.194+0000 7f7027fff700 1 -- 172.21.15.100:0/3542172952 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70281a08e0 con 0x7f7028102d60 2024-05-10T11:40:39.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.195+0000 7f7027fff700 1 --2- 172.21.15.100:0/3542172952 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7028102d60 0x7f70281a5220 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7020009fd0 tx=0x7f702000fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:40:39.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.195+0000 7f7025ffb700 1 -- 172.21.15.100:0/3542172952 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f702000b710 con 0x7f7028102d60 2024-05-10T11:40:39.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.195+0000 7f7025ffb700 1 -- 172.21.15.100:0/3542172952 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7020005040 con 0x7f7028102d60 2024-05-10T11:40:39.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.195+0000 7f702effd700 1 -- 172.21.15.100:0/3542172952 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70281a0b70 con 0x7f7028102d60 2024-05-10T11:40:39.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.195+0000 7f702effd700 1 -- 172.21.15.100:0/3542172952 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70281acc60 con 0x7f7028102d60 2024-05-10T11:40:39.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.196+0000 7f7025ffb700 1 -- 172.21.15.100:0/3542172952 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7020003af0 con 0x7f7028102d60 2024-05-10T11:40:39.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.201+0000 7f7039890b80 1 Processor -- start 2024-05-10T11:40:39.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.201+0000 7f7039890b80 1 -- start start 2024-05-10T11:40:39.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.201+0000 7f7039890b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564ce6b61b0 0x5564ce6b6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:39.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.201+0000 7f7039890b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564ce6b6ad0 0x5564ce6b0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:39.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.201+0000 7f7039890b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564ce6b0d50 0x5564ce6b1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:39.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.201+0000 7f7039890b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5564ce602940 con 0x5564ce6b6ad0 2024-05-10T11:40:39.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.201+0000 7f7039890b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5564ce567da0 con 0x5564ce6b0d50 2024-05-10T11:40:39.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.201+0000 7f7039890b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5564ce5fe900 con 0x5564ce6b61b0 2024-05-10T11:40:39.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.202+0000 7f702d59a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564ce6b61b0 0x5564ce6b6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:39.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.202+0000 7f702d59a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564ce6b61b0 0x5564ce6b6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:45476/0 (socket says 172.21.15.100:45476) 2024-05-10T11:40:39.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.202+0000 7f702d59a700 1 -- 172.21.15.100:0/940096401 learned_addr learned my addr 172.21.15.100:0/940096401 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:40:39.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.202+0000 7f702cd99700 1 --2- 172.21.15.100:0/940096401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564ce6b6ad0 0x5564ce6b0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:39.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.202+0000 7f7025ffb700 1 -- 172.21.15.100:0/3542172952 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f7020003c90 con 0x7f7028102d60 2024-05-10T11:40:39.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.202+0000 7f702cd99700 1 -- 172.21.15.100:0/940096401 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564ce6b61b0 msgr2=0x5564ce6b6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:39.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.202+0000 7f702cd99700 1 --2- 172.21.15.100:0/940096401 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564ce6b61b0 0x5564ce6b6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:39.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.202+0000 7f702cd99700 1 -- 172.21.15.100:0/940096401 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564ce6b0d50 msgr2=0x5564ce6b1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:39.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.202+0000 7f702cd99700 1 --2- 172.21.15.100:0/940096401 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564ce6b0d50 0x5564ce6b1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:39.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.202+0000 7f702cd99700 1 -- 172.21.15.100:0/940096401 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564ce54f8d0 con 0x5564ce6b6ad0 2024-05-10T11:40:39.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.203+0000 7f702cd99700 1 --2- 172.21.15.100:0/940096401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564ce6b6ad0 0x5564ce6b0810 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f7010005fd0 tx=0x7f7010009cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55142af2e9d0211c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:40:39.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.203+0000 7f7025ffb700 1 --2- 172.21.15.100:0/3542172952 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f70140b2a10 0x7f70140b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:39.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.203+0000 7f701effd700 1 -- 172.21.15.100:0/940096401 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7010004230 con 0x5564ce6b6ad0 2024-05-10T11:40:39.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.204+0000 7f701effd700 1 -- 172.21.15.100:0/940096401 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f70100043d0 con 0x5564ce6b6ad0 2024-05-10T11:40:39.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.204+0000 7f702d59a700 1 --2- 172.21.15.100:0/3542172952 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f70140b2a10 0x7f70140b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:39.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.204+0000 7f7025ffb700 1 -- 172.21.15.100:0/3542172952 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f70200e3330 con 0x7f7028102d60 2024-05-10T11:40:39.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.204+0000 7f701effd700 1 -- 172.21.15.100:0/940096401 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701000ee10 con 0x5564ce6b6ad0 2024-05-10T11:40:39.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.204+0000 7f7039890b80 1 -- 172.21.15.100:0/940096401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564ce6b6ad0 msgr2=0x5564ce6b0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:39.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.205+0000 7f7039890b80 1 --2- 172.21.15.100:0/940096401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564ce6b6ad0 0x5564ce6b0810 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f7010005fd0 tx=0x7f7010009cf0 comp rx=0 tx=0).stop 2024-05-10T11:40:39.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.205+0000 7f7039890b80 1 -- 172.21.15.100:0/940096401 shutdown_connections 2024-05-10T11:40:39.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.205+0000 7f7039890b80 1 --2- 172.21.15.100:0/940096401 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564ce6b0d50 0x5564ce6b1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:39.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.205+0000 7f7039890b80 1 --2- 172.21.15.100:0/940096401 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564ce6b61b0 0x5564ce6b6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:39.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.205+0000 7f7039890b80 1 --2- 172.21.15.100:0/940096401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564ce6b6ad0 0x5564ce6b0810 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:39.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.205+0000 7f7039890b80 1 -- 172.21.15.100:0/940096401 >> 172.21.15.100:0/940096401 conn(0x5564ce5dcb70 msgr2=0x5564ce6ab2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:40:39.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.205+0000 7f7039890b80 1 -- 172.21.15.100:0/940096401 shutdown_connections 2024-05-10T11:40:39.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.207+0000 7f702d59a700 1 --2- 172.21.15.100:0/3542172952 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f70140b2a10 0x7f70140b4ed0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f70180075c0 tx=0x7f701800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:40:39.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.210+0000 7f7039890b80 1 -- 172.21.15.100:0/940096401 wait complete. 2024-05-10T11:40:39.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.210+0000 7f7039890b80 1 -- start start 2024-05-10T11:40:39.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.210+0000 7f7039890b80 1 Processor -- start 2024-05-10T11:40:39.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.211+0000 7f7039890b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564ce6b0d50 0x5564ce6bc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:39.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.212+0000 7f7039890b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564ce6b61b0 0x5564ce6b9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:39.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.212+0000 7f7039890b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564ce6b6ad0 0x5564ce6ba050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:39.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.212+0000 7f7039890b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5564ce6bd4a0 con 0x5564ce6b0d50 2024-05-10T11:40:39.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.212+0000 7f7039890b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5564ce5bbed0 con 0x5564ce6b6ad0 2024-05-10T11:40:39.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.212+0000 7f7039890b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5564ce6bd620 con 0x5564ce6b61b0 2024-05-10T11:40:39.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.212+0000 7f702cd99700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564ce6b0d50 0x5564ce6bc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:39.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.212+0000 7f702cd99700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564ce6b0d50 0x5564ce6bc770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49982/0 (socket says 172.21.15.100:49982) 2024-05-10T11:40:39.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.212+0000 7f702cd99700 1 -- 172.21.15.100:0/2467731389 learned_addr learned my addr 172.21.15.100:0/2467731389 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:40:39.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.212+0000 7f7039890b80 10 client.0 _ll_register_callbacks cb 0x5564ce5965b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:40:39.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.212+0000 7f702d59a700 1 --2- 172.21.15.100:0/2467731389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564ce6b6ad0 0x5564ce6ba050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:39.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.213+0000 7f7027fff700 1 --2- 172.21.15.100:0/2467731389 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564ce6b61b0 0x5564ce6b9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:39.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.213+0000 7f702cd99700 1 -- 172.21.15.100:0/2467731389 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564ce6b61b0 msgr2=0x5564ce6b9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:39.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.213+0000 7f702cd99700 1 --2- 172.21.15.100:0/2467731389 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564ce6b61b0 0x5564ce6b9b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:39.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.213+0000 7f702cd99700 1 -- 172.21.15.100:0/2467731389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564ce6b6ad0 msgr2=0x5564ce6ba050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:39.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.213+0000 7f702cd99700 1 --2- 172.21.15.100:0/2467731389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564ce6b6ad0 0x5564ce6ba050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:39.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.213+0000 7f702cd99700 1 -- 172.21.15.100:0/2467731389 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564ce5b9760 con 0x5564ce6b0d50 2024-05-10T11:40:39.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.213+0000 7f702cd99700 1 --2- 172.21.15.100:0/2467731389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564ce6b0d50 0x5564ce6bc770 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x5564ce6ac040 tx=0x7f7010004a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44b52928906eec48 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:40:39.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.213+0000 7f7007fff700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:40:39.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.214+0000 7f7007fff700 1 -- 172.21.15.100:0/2467731389 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7010028060 con 0x5564ce6b0d50 2024-05-10T11:40:39.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.214+0000 7f7039890b80 1 -- 172.21.15.100:0/2467731389 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70100043d0 con 0x5564ce6b0d50 2024-05-10T11:40:39.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.214+0000 7f7007fff700 1 -- 172.21.15.100:0/2467731389 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f7010005200 con 0x5564ce6b0d50 2024-05-10T11:40:39.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.214+0000 7f7007fff700 1 -- 172.21.15.100:0/2467731389 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f7010005400 con 0x5564ce6b0d50 2024-05-10T11:40:39.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.214+0000 7f7039890b80 10 client.19743 fetch_fsmap learned FSMap version 41 2024-05-10T11:40:39.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.214+0000 7f7039890b80 1 -- 172.21.15.100:0/2467731389 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5564ce6c7c30 con 0x5564ce6b0d50 2024-05-10T11:40:39.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.214+0000 7f7007fff700 1 -- 172.21.15.100:0/2467731389 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f70100043d0 con 0x5564ce6b0d50 2024-05-10T11:40:39.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.215+0000 7f7007fff700 1 -- 172.21.15.100:0/2467731389 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f7010023900 con 0x5564ce6b0d50 2024-05-10T11:40:39.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.215+0000 7f7039890b80 10 client.19743 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:40:39.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.215+0000 7f7039890b80 10 client.19743 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-10T11:40:39.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.215+0000 7f7039890b80 20 client.19743 populate_metadata read hostname 'smithi100' 2024-05-10T11:40:39.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.215+0000 7f7039890b80 1 --2- 172.21.15.100:0/2467731389 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x5564ce6ca430 0x5564ce6cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:39.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.215+0000 7f7039890b80 4 client.19743 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:40:39.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.215+0000 7f7039890b80 1 -- 172.21.15.100:0/2467731389 --> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5564ce610320 con 0x5564ce6ca430 2024-05-10T11:40:39.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.216+0000 7f7027fff700 1 --2- 172.21.15.100:0/2467731389 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x5564ce6ca430 0x5564ce6cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:39.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.216+0000 7f7027fff700 1 --2- 172.21.15.100:0/2467731389 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x5564ce6ca430 0x5564ce6cc8d0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8cd3f7972cfa32a5 server_cookie=56faabf06a7324d6 in_seq=0 out_seq=0 2024-05-10T11:40:39.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.216+0000 7f7007fff700 10 client.19743 ms_handle_connect on v2:172.21.15.178:6838/3760985404 2024-05-10T11:40:39.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.217+0000 7f7007fff700 1 -- 172.21.15.100:0/2467731389 <== mds.1 v2:172.21.15.178:6838/3760985404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5564ce610320 con 0x5564ce6ca430 2024-05-10T11:40:39.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.217+0000 7f7007fff700 10 client.19743 handle_command_reply: tid=0 2024-05-10T11:40:39.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.217+0000 7f7039890b80 1 client.19743 shutdown 2024-05-10T11:40:39.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.218+0000 7f7039890b80 1 -- 172.21.15.100:0/2467731389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564ce6b0d50 msgr2=0x5564ce6bc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:39.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.218+0000 7f7039890b80 1 --2- 172.21.15.100:0/2467731389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564ce6b0d50 0x5564ce6bc770 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x5564ce6ac040 tx=0x7f7010004a30 comp rx=0 tx=0).stop 2024-05-10T11:40:39.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.218+0000 7f7039890b80 1 -- 172.21.15.100:0/2467731389 shutdown_connections 2024-05-10T11:40:39.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.218+0000 7f7039890b80 1 --2- 172.21.15.100:0/2467731389 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x5564ce6ca430 0x5564ce6cc8d0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:39.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.218+0000 7f7039890b80 1 --2- 172.21.15.100:0/2467731389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564ce6b6ad0 0x5564ce6ba050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:39.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.218+0000 7f7039890b80 1 --2- 172.21.15.100:0/2467731389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564ce6b0d50 0x5564ce6bc770 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:39.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.218+0000 7f7039890b80 1 --2- 172.21.15.100:0/2467731389 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564ce6b61b0 0x5564ce6b9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:39.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.219+0000 7f7039890b80 1 -- 172.21.15.100:0/2467731389 >> 172.21.15.100:0/2467731389 conn(0x5564ce5dcb70 msgr2=0x5564ce6a5b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:40:39.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.219+0000 7f7007fff700 0 client.19743 ms_handle_reset on v2:172.21.15.178:6838/3760985404 2024-05-10T11:40:39.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.219+0000 7f7039890b80 1 -- 172.21.15.100:0/2467731389 shutdown_connections 2024-05-10T11:40:39.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.220+0000 7f7039890b80 1 -- 172.21.15.100:0/2467731389 wait complete. 2024-05-10T11:40:39.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.221+0000 7f7039890b80 20 client.19743 trim_cache size 0 max 16384 2024-05-10T11:40:39.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.246+0000 7f7039890b80 1 Processor -- start 2024-05-10T11:40:39.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.246+0000 7f7039890b80 1 -- start start 2024-05-10T11:40:39.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.246+0000 7f7039890b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564ce5dcb70 0x5564ce6c6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:39.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.246+0000 7f7039890b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564ce6b0d50 0x5564ce6ba0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:39.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.246+0000 7f7039890b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564ce6b61b0 0x5564ce6b75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:39.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.246+0000 7f7039890b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7008011230 con 0x5564ce6b61b0 2024-05-10T11:40:39.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.246+0000 7f7039890b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f700800eac0 con 0x5564ce6b0d50 2024-05-10T11:40:39.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.246+0000 7f7039890b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f700800c910 con 0x5564ce5dcb70 2024-05-10T11:40:39.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.247+0000 7f7027fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564ce6b0d50 0x5564ce6ba0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:39.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.247+0000 7f7027fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564ce6b0d50 0x5564ce6ba0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:54924/0 (socket says 172.21.15.100:54924) 2024-05-10T11:40:39.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.247+0000 7f7027fff700 1 -- 172.21.15.100:0/400412770 learned_addr learned my addr 172.21.15.100:0/400412770 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:40:39.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.247+0000 7f702cd99700 1 --2- 172.21.15.100:0/400412770 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564ce5dcb70 0x5564ce6c6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:39.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.247+0000 7f702d59a700 1 --2- 172.21.15.100:0/400412770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564ce6b61b0 0x5564ce6b75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:39.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.247+0000 7f7027fff700 1 -- 172.21.15.100:0/400412770 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564ce5dcb70 msgr2=0x5564ce6c6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:39.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.247+0000 7f7027fff700 1 --2- 172.21.15.100:0/400412770 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564ce5dcb70 0x5564ce6c6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:39.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.247+0000 7f7027fff700 1 -- 172.21.15.100:0/400412770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564ce6b61b0 msgr2=0x5564ce6b75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:39.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.247+0000 7f7027fff700 1 --2- 172.21.15.100:0/400412770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564ce6b61b0 0x5564ce6b75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:39.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.247+0000 7f7027fff700 1 -- 172.21.15.100:0/400412770 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564ce6abde0 con 0x5564ce6b0d50 2024-05-10T11:40:39.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.247+0000 7f702cd99700 1 --2- 172.21.15.100:0/400412770 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564ce5dcb70 0x5564ce6c6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:40:39.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.247+0000 7f7027fff700 1 --2- 172.21.15.100:0/400412770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564ce6b0d50 0x5564ce6ba0a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f70200a6700 tx=0x7f70200cd040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18e7ea81e00aaac7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:40:39.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.247+0000 7f701e7fc700 1 -- 172.21.15.100:0/400412770 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70200aad80 con 0x5564ce6b0d50 2024-05-10T11:40:39.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.247+0000 7f701e7fc700 1 -- 172.21.15.100:0/400412770 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f70200aa680 con 0x5564ce6b0d50 2024-05-10T11:40:39.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.247+0000 7f701e7fc700 1 -- 172.21.15.100:0/400412770 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70200e0be0 con 0x5564ce6b0d50 2024-05-10T11:40:39.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.248+0000 7f7039890b80 1 -- 172.21.15.100:0/400412770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564ce6b0d50 msgr2=0x5564ce6ba0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:39.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.248+0000 7f7039890b80 1 --2- 172.21.15.100:0/400412770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564ce6b0d50 0x5564ce6ba0a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f70200a6700 tx=0x7f70200cd040 comp rx=0 tx=0).stop 2024-05-10T11:40:39.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.248+0000 7f7039890b80 1 -- 172.21.15.100:0/400412770 shutdown_connections 2024-05-10T11:40:39.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.248+0000 7f7039890b80 1 --2- 172.21.15.100:0/400412770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564ce6b61b0 0x5564ce6b75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:39.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.248+0000 7f7039890b80 1 --2- 172.21.15.100:0/400412770 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564ce5dcb70 0x5564ce6c6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:39.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.248+0000 7f7039890b80 1 --2- 172.21.15.100:0/400412770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564ce6b0d50 0x5564ce6ba0a0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:39.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.248+0000 7f7039890b80 1 -- 172.21.15.100:0/400412770 >> 172.21.15.100:0/400412770 conn(0x5564ce6b1440 msgr2=0x5564ce6b6eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:40:39.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.249+0000 7f7039890b80 1 -- 172.21.15.100:0/400412770 shutdown_connections 2024-05-10T11:40:39.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.249+0000 7f7039890b80 1 -- 172.21.15.100:0/400412770 wait complete. 2024-05-10T11:40:39.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.249+0000 7f7039890b80 1 -- start start 2024-05-10T11:40:39.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.249+0000 7f7039890b80 1 Processor -- start 2024-05-10T11:40:39.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.250+0000 7f7039890b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564ce5dcb70 0x5564ce6c2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:39.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.250+0000 7f7039890b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564ce6b0d50 0x5564ce6c2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:39.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.250+0000 7f7039890b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564ce6b61b0 0x5564ce6d11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:39.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.250+0000 7f7039890b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f700800cf80 con 0x5564ce6b0d50 2024-05-10T11:40:39.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.250+0000 7f7039890b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f700800e3f0 con 0x5564ce6b61b0 2024-05-10T11:40:39.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.250+0000 7f7039890b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f700800d650 con 0x5564ce5dcb70 2024-05-10T11:40:39.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.250+0000 7f7027fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564ce6b0d50 0x5564ce6c2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:39.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.250+0000 7f7039890b80 10 client.0 _ll_register_callbacks cb 0x5564ce5965b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:40:39.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.250+0000 7f7027fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564ce6b0d50 0x5564ce6c2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49996/0 (socket says 172.21.15.100:49996) 2024-05-10T11:40:39.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.250+0000 7f7027fff700 1 -- 172.21.15.100:0/3239224020 learned_addr learned my addr 172.21.15.100:0/3239224020 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:40:39.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.250+0000 7f702d59a700 1 --2- 172.21.15.100:0/3239224020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564ce6b61b0 0x5564ce6d11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:39.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.250+0000 7f702cd99700 1 --2- 172.21.15.100:0/3239224020 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564ce5dcb70 0x5564ce6c2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:39.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.250+0000 7f7027fff700 1 -- 172.21.15.100:0/3239224020 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564ce5dcb70 msgr2=0x5564ce6c2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:39.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.250+0000 7f7027fff700 1 --2- 172.21.15.100:0/3239224020 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564ce5dcb70 0x5564ce6c2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:39.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.250+0000 7f7027fff700 1 -- 172.21.15.100:0/3239224020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564ce6b61b0 msgr2=0x5564ce6d11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:39.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.250+0000 7f7027fff700 1 --2- 172.21.15.100:0/3239224020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564ce6b61b0 0x5564ce6d11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:39.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.250+0000 7f7027fff700 1 -- 172.21.15.100:0/3239224020 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564ce6d1710 con 0x5564ce6b0d50 2024-05-10T11:40:39.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.250+0000 7f702cd99700 1 --2- 172.21.15.100:0/3239224020 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564ce5dcb70 0x5564ce6c2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:40:39.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.251+0000 7f702d59a700 1 --2- 172.21.15.100:0/3239224020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564ce6b61b0 0x5564ce6d11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:40:39.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.251+0000 7f7027fff700 1 --2- 172.21.15.100:0/3239224020 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564ce6b0d50 0x5564ce6c2ed0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f7020005e90 tx=0x7f70200e3280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d6567afa35fcb27 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:40:39.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.251+0000 7f7007fff700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:40:39.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.251+0000 7f7007fff700 1 -- 172.21.15.100:0/3239224020 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70200aad80 con 0x5564ce6b0d50 2024-05-10T11:40:39.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.251+0000 7f7007fff700 1 -- 172.21.15.100:0/3239224020 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f70200a6c70 con 0x5564ce6b0d50 2024-05-10T11:40:39.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.251+0000 7f7039890b80 1 -- 172.21.15.100:0/3239224020 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70200aa680 con 0x5564ce6b0d50 2024-05-10T11:40:39.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.251+0000 7f7007fff700 1 -- 172.21.15.100:0/3239224020 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f70200ba050 con 0x5564ce6b0d50 2024-05-10T11:40:39.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.251+0000 7f7039890b80 10 client.19749 fetch_fsmap learned FSMap version 41 2024-05-10T11:40:39.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.251+0000 7f7039890b80 1 -- 172.21.15.100:0/3239224020 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5564ce734410 con 0x5564ce6b0d50 2024-05-10T11:40:39.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.252+0000 7f7007fff700 1 -- 172.21.15.100:0/3239224020 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f70200aa680 con 0x5564ce6b0d50 2024-05-10T11:40:39.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.252+0000 7f7007fff700 1 -- 172.21.15.100:0/3239224020 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f70200b2fc0 con 0x5564ce6b0d50 2024-05-10T11:40:39.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.252+0000 7f7039890b80 10 client.19749 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:40:39.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.252+0000 7f7039890b80 10 client.19749 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-10T11:40:39.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.252+0000 7f7039890b80 20 client.19749 populate_metadata read hostname 'smithi100' 2024-05-10T11:40:39.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.252+0000 7f7039890b80 1 --2- 172.21.15.100:0/3239224020 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x5564ce6b6ad0 0x5564ce736a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:39.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.252+0000 7f7039890b80 4 client.19749 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:40:39.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.252+0000 7f7039890b80 1 -- 172.21.15.100:0/3239224020 --> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5564ce60a3c0 con 0x5564ce6b6ad0 2024-05-10T11:40:39.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.252+0000 7f702cd99700 1 --2- 172.21.15.100:0/3239224020 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x5564ce6b6ad0 0x5564ce736a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:39.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.253+0000 7f702cd99700 1 --2- 172.21.15.100:0/3239224020 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x5564ce6b6ad0 0x5564ce736a30 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=12da7fcbc28ea9f6 server_cookie=8214ff56d8bd637a in_seq=0 out_seq=0 2024-05-10T11:40:39.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.253+0000 7f7007fff700 10 client.19749 ms_handle_connect on v2:172.21.15.178:6838/3760985404 2024-05-10T11:40:39.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.253+0000 7f7007fff700 1 -- 172.21.15.100:0/3239224020 <== mds.1 v2:172.21.15.178:6838/3760985404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5564ce60a3c0 con 0x5564ce6b6ad0 2024-05-10T11:40:39.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.253+0000 7f7007fff700 10 client.19749 handle_command_reply: tid=0 2024-05-10T11:40:39.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.253+0000 7f7039890b80 1 client.19749 shutdown 2024-05-10T11:40:39.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.253+0000 7f7039890b80 1 -- 172.21.15.100:0/3239224020 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564ce6b0d50 msgr2=0x5564ce6c2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:39.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.253+0000 7f7039890b80 1 --2- 172.21.15.100:0/3239224020 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564ce6b0d50 0x5564ce6c2ed0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f7020005e90 tx=0x7f70200e3280 comp rx=0 tx=0).stop 2024-05-10T11:40:39.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.254+0000 7f7039890b80 1 -- 172.21.15.100:0/3239224020 shutdown_connections 2024-05-10T11:40:39.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.254+0000 7f7039890b80 1 --2- 172.21.15.100:0/3239224020 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x5564ce6b6ad0 0x5564ce736a30 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:39.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.254+0000 7f7039890b80 1 --2- 172.21.15.100:0/3239224020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564ce6b61b0 0x5564ce6d11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:39.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.254+0000 7f7039890b80 1 --2- 172.21.15.100:0/3239224020 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564ce5dcb70 0x5564ce6c2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:39.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.254+0000 7f7039890b80 1 --2- 172.21.15.100:0/3239224020 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564ce6b0d50 0x5564ce6c2ed0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:39.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.254+0000 7f7039890b80 1 -- 172.21.15.100:0/3239224020 >> 172.21.15.100:0/3239224020 conn(0x5564ce6b1440 msgr2=0x5564ce6b4c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:40:39.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.254+0000 7f7007fff700 0 client.19749 ms_handle_reset on v2:172.21.15.178:6838/3760985404 2024-05-10T11:40:39.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.254+0000 7f7039890b80 1 -- 172.21.15.100:0/3239224020 shutdown_connections 2024-05-10T11:40:39.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.254+0000 7f7039890b80 1 -- 172.21.15.100:0/3239224020 wait complete. 2024-05-10T11:40:39.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.254+0000 7f7039890b80 20 client.19749 trim_cache size 0 max 16384 2024-05-10T11:40:39.261 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:40:39.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.255+0000 7f702effd700 1 -- 172.21.15.100:0/3542172952 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f70140b2a10 msgr2=0x7f70140b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:39.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.255+0000 7f702effd700 1 --2- 172.21.15.100:0/3542172952 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f70140b2a10 0x7f70140b4ed0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f70180075c0 tx=0x7f701800b040 comp rx=0 tx=0).stop 2024-05-10T11:40:39.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.255+0000 7f702effd700 1 -- 172.21.15.100:0/3542172952 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7028102d60 msgr2=0x7f70281a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:39.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.255+0000 7f702effd700 1 --2- 172.21.15.100:0/3542172952 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7028102d60 0x7f70281a5220 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7020009fd0 tx=0x7f702000fa00 comp rx=0 tx=0).stop 2024-05-10T11:40:39.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.255+0000 7f702effd700 1 -- 172.21.15.100:0/3542172952 shutdown_connections 2024-05-10T11:40:39.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.255+0000 7f702effd700 1 --2- 172.21.15.100:0/3542172952 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f70140b2a10 0x7f70140b4ed0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:39.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.255+0000 7f702effd700 1 --2- 172.21.15.100:0/3542172952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7028108410 0x7f70281a0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:39.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.255+0000 7f702effd700 1 --2- 172.21.15.100:0/3542172952 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70281023a0 0x7f70281a4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:39.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.255+0000 7f702effd700 1 --2- 172.21.15.100:0/3542172952 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7028102d60 0x7f70281a5220 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:39.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.255+0000 7f702effd700 1 -- 172.21.15.100:0/3542172952 >> 172.21.15.100:0/3542172952 conn(0x7f7028079350 msgr2=0x7f7028100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:40:39.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.255+0000 7f702effd700 1 -- 172.21.15.100:0/3542172952 shutdown_connections 2024-05-10T11:40:39.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:39.255+0000 7f702effd700 1 -- 172.21.15.100:0/3542172952 wait complete. 2024-05-10T11:40:39.953 INFO:tasks.workunit.client.0.smithi100.stdout: 10 85639 8.95 MB/sec execute 393 sec latency 107.581 ms 2024-05-10T11:40:40.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:40:39 smithi100 ceph-mon[30296]: pgmap v1089: 129 pgs: 129 active+clean; 548 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 3.9 MiB/s wr, 221 op/s 2024-05-10T11:40:40.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:40:39 smithi178 ceph-mon[33359]: pgmap v1089: 129 pgs: 129 active+clean; 548 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 3.9 MiB/s wr, 221 op/s 2024-05-10T11:40:40.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:40:39 smithi188 ceph-mon[34075]: pgmap v1089: 129 pgs: 129 active+clean; 548 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 3.9 MiB/s wr, 221 op/s 2024-05-10T11:40:40.684 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-10T11:40:40.953 INFO:tasks.workunit.client.0.smithi100.stdout: 10 85787 8.94 MB/sec execute 394 sec latency 123.239 ms 2024-05-10T11:40:41.043 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:40:41.798 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.797+0000 7f4009a15640 1 -- 172.21.15.100:0/4137015870 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4004103c20 msgr2=0x7f40041040a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:41.798 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.797+0000 7f4009a15640 1 --2- 172.21.15.100:0/4137015870 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4004103c20 0x7f40041040a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f3ff4009900 tx=0x7f3ff402a3f0 comp rx=0 tx=0).stop 2024-05-10T11:40:41.798 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.797+0000 7f4009a15640 1 -- 172.21.15.100:0/4137015870 shutdown_connections 2024-05-10T11:40:41.798 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.797+0000 7f4009a15640 1 --2- 172.21.15.100:0/4137015870 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40041045e0 0x7f400410ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:41.798 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.797+0000 7f4009a15640 1 --2- 172.21.15.100:0/4137015870 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4004103c20 0x7f40041040a0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:41.798 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.797+0000 7f4009a15640 1 --2- 172.21.15.100:0/4137015870 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4004102a30 0x7f4004102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:41.798 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.797+0000 7f4009a15640 1 -- 172.21.15.100:0/4137015870 >> 172.21.15.100:0/4137015870 conn(0x7f40040fdfa0 msgr2=0x7f40041003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:40:41.798 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.797+0000 7f4009a15640 1 -- 172.21.15.100:0/4137015870 shutdown_connections 2024-05-10T11:40:41.798 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.798+0000 7f4009a15640 1 -- 172.21.15.100:0/4137015870 wait complete. 2024-05-10T11:40:41.800 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.798+0000 7f4009a15640 1 Processor -- start 2024-05-10T11:40:41.800 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.798+0000 7f4009a15640 1 -- start start 2024-05-10T11:40:41.800 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.798+0000 7f4009a15640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4004102a30 0x7f4004076760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:41.800 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.798+0000 7f4009a15640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4004103c20 0x7f4004074da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:41.800 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.799+0000 7f4009a15640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40041045e0 0x7f40040752e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:41.800 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.799+0000 7f4009a15640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f400410e460 con 0x7f40041045e0 2024-05-10T11:40:41.801 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.799+0000 7f4009a15640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f400410e2e0 con 0x7f4004103c20 2024-05-10T11:40:41.801 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.799+0000 7f4009a15640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f400410e5e0 con 0x7f4004102a30 2024-05-10T11:40:41.801 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.799+0000 7f3ffbfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4004103c20 0x7f4004074da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:41.801 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.799+0000 7f3ffbfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4004103c20 0x7f4004074da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:54944/0 (socket says 172.21.15.100:54944) 2024-05-10T11:40:41.801 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.799+0000 7f4002ffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4004102a30 0x7f4004076760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:41.801 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.799+0000 7f3ffbfff640 1 -- 172.21.15.100:0/140997665 learned_addr learned my addr 172.21.15.100:0/140997665 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:40:41.801 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.799+0000 7f40037fe640 1 --2- 172.21.15.100:0/140997665 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40041045e0 0x7f40040752e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:41.801 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.799+0000 7f3ffbfff640 1 -- 172.21.15.100:0/140997665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4004102a30 msgr2=0x7f4004076760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:41.801 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.799+0000 7f3ffbfff640 1 --2- 172.21.15.100:0/140997665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4004102a30 0x7f4004076760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:41.801 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.799+0000 7f3ffbfff640 1 -- 172.21.15.100:0/140997665 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40041045e0 msgr2=0x7f40040752e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:41.801 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.799+0000 7f3ffbfff640 1 --2- 172.21.15.100:0/140997665 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40041045e0 0x7f40040752e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:41.801 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.799+0000 7f3ffbfff640 1 -- 172.21.15.100:0/140997665 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4004075820 con 0x7f4004103c20 2024-05-10T11:40:41.801 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.799+0000 7f4002ffd640 1 --2- 172.21.15.100:0/140997665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4004102a30 0x7f4004076760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:40:41.801 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.799+0000 7f40037fe640 1 --2- 172.21.15.100:0/140997665 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40041045e0 0x7f40040752e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:40:41.802 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.799+0000 7f3ffbfff640 1 --2- 172.21.15.100:0/140997665 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4004103c20 0x7f4004074da0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3ff4009850 tx=0x7f3ff4003ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:40:41.802 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.800+0000 7f4008a13640 1 -- 172.21.15.100:0/140997665 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff403f070 con 0x7f4004103c20 2024-05-10T11:40:41.802 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.800+0000 7f4008a13640 1 -- 172.21.15.100:0/140997665 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3ff4031d10 con 0x7f4004103c20 2024-05-10T11:40:41.802 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.800+0000 7f4008a13640 1 -- 172.21.15.100:0/140997665 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff4035030 con 0x7f4004103c20 2024-05-10T11:40:41.802 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.800+0000 7f4009a15640 1 -- 172.21.15.100:0/140997665 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4004075ab0 con 0x7f4004103c20 2024-05-10T11:40:41.802 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.800+0000 7f4009a15640 1 -- 172.21.15.100:0/140997665 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40041a7380 con 0x7f4004103c20 2024-05-10T11:40:41.804 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.804+0000 7f4008a13640 1 -- 172.21.15.100:0/140997665 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f3ff4042070 con 0x7f4004103c20 2024-05-10T11:40:41.804 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.804+0000 7f400c806740 1 Processor -- start 2024-05-10T11:40:41.804 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.805+0000 7f400c806740 1 -- start start 2024-05-10T11:40:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.805+0000 7f400c806740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556046b1adb0 0x556046b1b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.805+0000 7f400c806740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556046b1b7a0 0x556046b202a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.805+0000 7f400c806740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556046b20910 0x556046b22da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.805+0000 7f400c806740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x556046a7f5d0 con 0x556046b20910 2024-05-10T11:40:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.805+0000 7f400c806740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556046a38890 con 0x556046b1b7a0 2024-05-10T11:40:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.805+0000 7f400c806740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556046a7b590 con 0x556046b1adb0 2024-05-10T11:40:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.805+0000 7f40037fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556046b1adb0 0x556046b1b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.805+0000 7f40037fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556046b1adb0 0x556046b1b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:45508/0 (socket says 172.21.15.100:45508) 2024-05-10T11:40:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.805+0000 7f40037fe640 1 -- 172.21.15.100:0/2637393295 learned_addr learned my addr 172.21.15.100:0/2637393295 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:40:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.805+0000 7f4002ffd640 1 --2- 172.21.15.100:0/2637393295 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556046b1b7a0 0x556046b202a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.805+0000 7f3ffbfff640 1 --2- 172.21.15.100:0/2637393295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556046b20910 0x556046b22da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.805+0000 7f40037fe640 1 -- 172.21.15.100:0/2637393295 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556046b1b7a0 msgr2=0x556046b202a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.805+0000 7f40037fe640 1 --2- 172.21.15.100:0/2637393295 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556046b1b7a0 0x556046b202a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.805+0000 7f40037fe640 1 -- 172.21.15.100:0/2637393295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556046b20910 msgr2=0x556046b22da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.806+0000 7f40037fe640 1 --2- 172.21.15.100:0/2637393295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556046b20910 0x556046b22da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.806+0000 7f40037fe640 1 -- 172.21.15.100:0/2637393295 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556046791210 con 0x556046b1adb0 2024-05-10T11:40:41.808 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.806+0000 7f4002ffd640 1 --2- 172.21.15.100:0/2637393295 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556046b1b7a0 0x556046b202a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:40:41.808 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.806+0000 7f3ffbfff640 1 --2- 172.21.15.100:0/2637393295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556046b20910 0x556046b22da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:40:41.808 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.806+0000 7f40037fe640 1 --2- 172.21.15.100:0/2637393295 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556046b1adb0 0x556046b1b1d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc011810 tx=0x7f3ffc011cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c7b5321f7322559 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:40:41.808 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.806+0000 7f3ff9ffb640 1 -- 172.21.15.100:0/2637393295 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ffc011310 con 0x556046b1adb0 2024-05-10T11:40:41.808 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.806+0000 7f3ff9ffb640 1 -- 172.21.15.100:0/2637393295 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3ffc00f9a0 con 0x556046b1adb0 2024-05-10T11:40:41.808 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.806+0000 7f3ff9ffb640 1 -- 172.21.15.100:0/2637393295 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ffc005600 con 0x556046b1adb0 2024-05-10T11:40:41.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.807+0000 7f4008a13640 1 --2- 172.21.15.100:0/140997665 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f3ff00b28a0 0x7f3ff00b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:41.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.807+0000 7f400c806740 1 -- 172.21.15.100:0/2637393295 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556046b1adb0 msgr2=0x556046b1b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:41.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.807+0000 7f400c806740 1 --2- 172.21.15.100:0/2637393295 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556046b1adb0 0x556046b1b1d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc011810 tx=0x7f3ffc011cd0 comp rx=0 tx=0).stop 2024-05-10T11:40:41.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.807+0000 7f400c806740 1 -- 172.21.15.100:0/2637393295 shutdown_connections 2024-05-10T11:40:41.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.807+0000 7f4002ffd640 1 --2- 172.21.15.100:0/140997665 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f3ff00b28a0 0x7f3ff00b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.807+0000 7f400c806740 1 --2- 172.21.15.100:0/2637393295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556046b20910 0x556046b22da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.807+0000 7f400c806740 1 --2- 172.21.15.100:0/2637393295 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556046b1b7a0 0x556046b202a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.807+0000 7f400c806740 1 --2- 172.21.15.100:0/2637393295 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556046b1adb0 0x556046b1b1d0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.807+0000 7f400c806740 1 -- 172.21.15.100:0/2637393295 >> 172.21.15.100:0/2637393295 conn(0x556046b16420 msgr2=0x556046b16820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:40:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.807+0000 7f400c806740 1 -- 172.21.15.100:0/2637393295 shutdown_connections 2024-05-10T11:40:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.807+0000 7f400c806740 1 -- 172.21.15.100:0/2637393295 wait complete. 2024-05-10T11:40:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.808+0000 7f4008a13640 1 -- 172.21.15.100:0/140997665 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f3ff4102eb0 con 0x7f4004103c20 2024-05-10T11:40:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.808+0000 7f400c806740 1 -- start start 2024-05-10T11:40:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.808+0000 7f4002ffd640 1 --2- 172.21.15.100:0/140997665 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f3ff00b28a0 0x7f3ff00b4d60 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x556046b172f0 tx=0x7f3fec005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:40:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.808+0000 7f400c806740 1 Processor -- start 2024-05-10T11:40:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.809+0000 7f400c806740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556046b1adb0 0x556046b25080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.809+0000 7f400c806740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556046b1b7a0 0x556046b255c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.809+0000 7f3ffbfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556046b1b7a0 0x556046b255c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.809+0000 7f3ffbfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556046b1b7a0 0x556046b255c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50024/0 (socket says 172.21.15.100:50024) 2024-05-10T11:40:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.809+0000 7f3ffbfff640 1 -- 172.21.15.100:0/861299925 learned_addr learned my addr 172.21.15.100:0/861299925 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:40:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.809+0000 7f400c806740 1 --2- 172.21.15.100:0/861299925 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556046b20910 0x556046b2c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.809+0000 7f400c806740 1 -- 172.21.15.100:0/861299925 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x556046b238c0 con 0x556046b1b7a0 2024-05-10T11:40:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.809+0000 7f400c806740 1 -- 172.21.15.100:0/861299925 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556046a3c370 con 0x556046b1adb0 2024-05-10T11:40:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.809+0000 7f400c806740 1 -- 172.21.15.100:0/861299925 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556046b23a40 con 0x556046b20910 2024-05-10T11:40:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.809+0000 7f400c806740 10 client.0 _ll_register_callbacks cb 0x556046790a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:40:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.809+0000 7f4002ffd640 1 --2- 172.21.15.100:0/861299925 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556046b1adb0 0x556046b25080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.809+0000 7f40037fe640 1 --2- 172.21.15.100:0/861299925 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556046b20910 0x556046b2c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.809+0000 7f3ffbfff640 1 -- 172.21.15.100:0/861299925 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556046b20910 msgr2=0x556046b2c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.809+0000 7f3ffbfff640 1 --2- 172.21.15.100:0/861299925 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556046b20910 0x556046b2c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.809+0000 7f3ffbfff640 1 -- 172.21.15.100:0/861299925 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556046b1adb0 msgr2=0x556046b25080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.809+0000 7f3ffbfff640 1 --2- 172.21.15.100:0/861299925 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556046b1adb0 0x556046b25080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.809+0000 7f3ffbfff640 1 -- 172.21.15.100:0/861299925 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55604685d620 con 0x556046b1b7a0 2024-05-10T11:40:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.809+0000 7f4002ffd640 1 --2- 172.21.15.100:0/861299925 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556046b1adb0 0x556046b25080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:40:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.810+0000 7f400c806740 1 -- 172.21.15.100:0/861299925 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ffc00f9a0 con 0x556046b1b7a0 2024-05-10T11:40:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.810+0000 7f3ffbfff640 1 --2- 172.21.15.100:0/861299925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556046b1b7a0 0x556046b255c0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f3ff40c7380 tx=0x7f3ff4005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5520a96b5224597e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:40:41.813 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.810+0000 7f3fe2ffd640 10 client.19758 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:40:41.876 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.875+0000 7f3fe2ffd640 1 -- 172.21.15.100:0/861299925 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff4049070 con 0x556046b1b7a0 2024-05-10T11:40:41.877 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.875+0000 7f3fe2ffd640 1 -- 172.21.15.100:0/861299925 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f3ff40c63a0 con 0x556046b1b7a0 2024-05-10T11:40:41.877 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.875+0000 7f3fe2ffd640 1 -- 172.21.15.100:0/861299925 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f3ff40ca810 con 0x556046b1b7a0 2024-05-10T11:40:41.877 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.875+0000 7f400c806740 10 client.19758 fetch_fsmap learned FSMap version 41 2024-05-10T11:40:41.877 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.875+0000 7f400c806740 1 -- 172.21.15.100:0/861299925 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({fsmap=41}) -- 0x556046b2f490 con 0x556046b1b7a0 2024-05-10T11:40:41.877 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.876+0000 7f3fe2ffd640 1 -- 172.21.15.100:0/861299925 <== mon.0 v2:172.21.15.100:3300/0 4 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f3ff40ce120 con 0x556046b1b7a0 2024-05-10T11:40:41.877 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.876+0000 7f400c806740 10 client.19758 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:40:41.877 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.876+0000 7f400c806740 10 client.19758 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:40:41.877 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.876+0000 7f400c806740 20 client.19758 populate_metadata read hostname 'smithi100' 2024-05-10T11:40:41.877 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.876+0000 7f400c806740 1 --2- 172.21.15.100:0/861299925 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556046b30bf0 0x556046b33030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:41.877 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.876+0000 7f400c806740 4 client.19758 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:40:41.877 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.876+0000 7f400c806740 1 -- 172.21.15.100:0/861299925 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556046a8d1e0 con 0x556046b30bf0 2024-05-10T11:40:41.877 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.876+0000 7f4002ffd640 1 --2- 172.21.15.100:0/861299925 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556046b30bf0 0x556046b33030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:41.878 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.876+0000 7f3fe2ffd640 10 client.19758 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:40:41.878 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.876+0000 7f4002ffd640 1 --2- 172.21.15.100:0/861299925 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556046b30bf0 0x556046b33030 crc :-1 s=READY pgs=419 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb0f1d30ce51503b server_cookie=d9703a95486edf3e in_seq=0 out_seq=0 2024-05-10T11:40:41.879 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.878+0000 7f3fe2ffd640 1 -- 172.21.15.100:0/861299925 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556046a8d1e0 con 0x556046b30bf0 2024-05-10T11:40:41.879 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.878+0000 7f3fe2ffd640 10 client.19758 handle_command_reply: tid=0 2024-05-10T11:40:41.879 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.878+0000 7f400c806740 1 client.19758 shutdown 2024-05-10T11:40:41.879 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.878+0000 7f400c806740 1 -- 172.21.15.100:0/861299925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556046b1b7a0 msgr2=0x556046b255c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:41.879 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.878+0000 7f400c806740 1 --2- 172.21.15.100:0/861299925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556046b1b7a0 0x556046b255c0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f3ff40c7380 tx=0x7f3ff4005de0 comp rx=0 tx=0).stop 2024-05-10T11:40:41.879 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.878+0000 7f400c806740 1 -- 172.21.15.100:0/861299925 shutdown_connections 2024-05-10T11:40:41.880 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.878+0000 7f400c806740 1 --2- 172.21.15.100:0/861299925 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556046b30bf0 0x556046b33030 crc :-1 s=READY pgs=419 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:41.880 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.878+0000 7f400c806740 1 --2- 172.21.15.100:0/861299925 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556046b20910 0x556046b2c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:41.880 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.878+0000 7f3fe2ffd640 0 client.19758 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:40:41.880 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.878+0000 7f400c806740 1 --2- 172.21.15.100:0/861299925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556046b1b7a0 0x556046b255c0 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:41.880 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.878+0000 7f400c806740 1 --2- 172.21.15.100:0/861299925 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556046b1adb0 0x556046b25080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:41.880 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.878+0000 7f400c806740 1 -- 172.21.15.100:0/861299925 >> 172.21.15.100:0/861299925 conn(0x556046b16420 msgr2=0x556046b15580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:40:41.880 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.878+0000 7f400c806740 1 -- 172.21.15.100:0/861299925 shutdown_connections 2024-05-10T11:40:41.880 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.878+0000 7f400c806740 1 -- 172.21.15.100:0/861299925 wait complete. 2024-05-10T11:40:41.880 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.878+0000 7f400c806740 20 client.19758 trim_cache size 0 max 16384 2024-05-10T11:40:41.907 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.904+0000 7f400c806740 1 Processor -- start 2024-05-10T11:40:41.907 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.904+0000 7f400c806740 1 -- start start 2024-05-10T11:40:41.907 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.904+0000 7f400c806740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556046b1adb0 0x556046b19330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:41.907 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.904+0000 7f400c806740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556046b1b7a0 0x556046b19870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:41.907 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.904+0000 7f400c806740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556046b20910 0x556046b37930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:41.907 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.904+0000 7f400c806740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f40041b3fc0 con 0x556046b1adb0 2024-05-10T11:40:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.904+0000 7f4002ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556046b1adb0 0x556046b19330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.904+0000 7f400c806740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f40041b1890 con 0x556046b1b7a0 2024-05-10T11:40:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.904+0000 7f400c806740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f40041af6e0 con 0x556046b20910 2024-05-10T11:40:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.904+0000 7f4002ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556046b1adb0 0x556046b19330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50038/0 (socket says 172.21.15.100:50038) 2024-05-10T11:40:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.904+0000 7f4002ffd640 1 -- 172.21.15.100:0/2200149422 learned_addr learned my addr 172.21.15.100:0/2200149422 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:40:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.904+0000 7f3ffbfff640 1 --2- 172.21.15.100:0/2200149422 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556046b1b7a0 0x556046b19870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.904+0000 7f40037fe640 1 --2- 172.21.15.100:0/2200149422 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556046b20910 0x556046b37930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.905+0000 7f4002ffd640 1 -- 172.21.15.100:0/2200149422 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556046b20910 msgr2=0x556046b37930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.905+0000 7f4002ffd640 1 --2- 172.21.15.100:0/2200149422 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556046b20910 0x556046b37930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.905+0000 7f4002ffd640 1 -- 172.21.15.100:0/2200149422 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556046b1b7a0 msgr2=0x556046b19870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.905+0000 7f4002ffd640 1 --2- 172.21.15.100:0/2200149422 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556046b1b7a0 0x556046b19870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.905+0000 7f4002ffd640 1 -- 172.21.15.100:0/2200149422 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556046b25910 con 0x556046b1adb0 2024-05-10T11:40:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.905+0000 7f4002ffd640 1 --2- 172.21.15.100:0/2200149422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556046b1adb0 0x556046b19330 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x556046b1c210 tx=0x7f3fec01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6e14636040ea5fa server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:40:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.905+0000 7f3ff97fa640 1 -- 172.21.15.100:0/2200149422 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fec031070 con 0x556046b1adb0 2024-05-10T11:40:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.905+0000 7f3ff97fa640 1 -- 172.21.15.100:0/2200149422 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3fec02b430 con 0x556046b1adb0 2024-05-10T11:40:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.905+0000 7f3ff97fa640 1 -- 172.21.15.100:0/2200149422 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fec029680 con 0x556046b1adb0 2024-05-10T11:40:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.906+0000 7f400c806740 1 -- 172.21.15.100:0/2200149422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556046b1adb0 msgr2=0x556046b19330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.906+0000 7f400c806740 1 --2- 172.21.15.100:0/2200149422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556046b1adb0 0x556046b19330 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x556046b1c210 tx=0x7f3fec01b000 comp rx=0 tx=0).stop 2024-05-10T11:40:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.906+0000 7f400c806740 1 -- 172.21.15.100:0/2200149422 shutdown_connections 2024-05-10T11:40:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.906+0000 7f400c806740 1 --2- 172.21.15.100:0/2200149422 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556046b20910 0x556046b37930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.906+0000 7f400c806740 1 --2- 172.21.15.100:0/2200149422 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556046b1b7a0 0x556046b19870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.906+0000 7f400c806740 1 --2- 172.21.15.100:0/2200149422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556046b1adb0 0x556046b19330 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.906+0000 7f400c806740 1 -- 172.21.15.100:0/2200149422 >> 172.21.15.100:0/2200149422 conn(0x556046b16420 msgr2=0x556046b1ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:40:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.907+0000 7f400c806740 1 -- 172.21.15.100:0/2200149422 shutdown_connections 2024-05-10T11:40:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.907+0000 7f400c806740 1 -- 172.21.15.100:0/2200149422 wait complete. 2024-05-10T11:40:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.907+0000 7f400c806740 1 -- start start 2024-05-10T11:40:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.907+0000 7f400c806740 1 Processor -- start 2024-05-10T11:40:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.908+0000 7f400c806740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556046b1adb0 0x556046b297d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.908+0000 7f400c806740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556046b1b7a0 0x556046b29d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.908+0000 7f400c806740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556046b20910 0x556046b2acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.908+0000 7f4002ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556046b1adb0 0x556046b297d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.908+0000 7f4002ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556046b1adb0 0x556046b297d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:54994/0 (socket says 172.21.15.100:54994) 2024-05-10T11:40:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.908+0000 7f4002ffd640 1 -- 172.21.15.100:0/759982005 learned_addr learned my addr 172.21.15.100:0/759982005 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:40:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.908+0000 7f400c806740 1 -- 172.21.15.100:0/759982005 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f40041afd50 con 0x556046b20910 2024-05-10T11:40:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.908+0000 7f400c806740 1 -- 172.21.15.100:0/759982005 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f40041b11c0 con 0x556046b1adb0 2024-05-10T11:40:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.908+0000 7f400c806740 1 -- 172.21.15.100:0/759982005 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f40041b0420 con 0x556046b1b7a0 2024-05-10T11:40:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.908+0000 7f400c806740 10 client.0 _ll_register_callbacks cb 0x7f3ff4005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:40:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.908+0000 7f40037fe640 1 --2- 172.21.15.100:0/759982005 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556046b20910 0x556046b2acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.909+0000 7f3ffbfff640 1 --2- 172.21.15.100:0/759982005 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556046b1b7a0 0x556046b29d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.909+0000 7f40037fe640 1 -- 172.21.15.100:0/759982005 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556046b1b7a0 msgr2=0x556046b29d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.909+0000 7f40037fe640 1 --2- 172.21.15.100:0/759982005 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556046b1b7a0 0x556046b29d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.909+0000 7f40037fe640 1 -- 172.21.15.100:0/759982005 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556046b1adb0 msgr2=0x556046b297d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.909+0000 7f40037fe640 1 --2- 172.21.15.100:0/759982005 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556046b1adb0 0x556046b297d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.909+0000 7f40037fe640 1 -- 172.21.15.100:0/759982005 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556046b2b2c0 con 0x556046b20910 2024-05-10T11:40:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.909+0000 7f40037fe640 1 --2- 172.21.15.100:0/759982005 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556046b20910 0x556046b2acd0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc000f80 tx=0x7f3ffc00fd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3760a91878aebf4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:40:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.909+0000 7f3fe2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:40:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.909+0000 7f3fe2ffd640 1 -- 172.21.15.100:0/759982005 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ffc009810 con 0x556046b20910 2024-05-10T11:40:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.910+0000 7f400c806740 1 -- 172.21.15.100:0/759982005 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fec02b430 con 0x556046b20910 2024-05-10T11:40:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.910+0000 7f3fe2ffd640 1 -- 172.21.15.100:0/759982005 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f3ffc01e030 con 0x556046b20910 2024-05-10T11:40:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.910+0000 7f3fe2ffd640 1 -- 172.21.15.100:0/759982005 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f3ffc036050 con 0x556046b20910 2024-05-10T11:40:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.910+0000 7f400c806740 10 client.19764 fetch_fsmap learned FSMap version 41 2024-05-10T11:40:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.910+0000 7f400c806740 1 -- 172.21.15.100:0/759982005 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x556046b9b880 con 0x556046b20910 2024-05-10T11:40:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.910+0000 7f3fe2ffd640 1 -- 172.21.15.100:0/759982005 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3ffc02b020 con 0x556046b20910 2024-05-10T11:40:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.910+0000 7f3fe2ffd640 1 -- 172.21.15.100:0/759982005 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f3ffc018070 con 0x556046b20910 2024-05-10T11:40:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.910+0000 7f400c806740 10 client.19764 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:40:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.910+0000 7f400c806740 10 client.19764 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:40:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.910+0000 7f400c806740 20 client.19764 populate_metadata read hostname 'smithi100' 2024-05-10T11:40:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.910+0000 7f400c806740 1 --2- 172.21.15.100:0/759982005 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556046b30bf0 0x556046b9e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.911+0000 7f400c806740 4 client.19764 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:40:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.911+0000 7f400c806740 1 -- 172.21.15.100:0/759982005 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f3fec02b430 con 0x556046b30bf0 2024-05-10T11:40:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.911+0000 7f4002ffd640 1 --2- 172.21.15.100:0/759982005 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556046b30bf0 0x556046b9e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.911+0000 7f3fe2ffd640 10 client.19764 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:40:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:41.911+0000 7f4002ffd640 1 --2- 172.21.15.100:0/759982005 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556046b30bf0 0x556046b9e070 crc :-1 s=READY pgs=420 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e27aec8c5ea2428 server_cookie=b03f2af94542da99 in_seq=0 out_seq=0 2024-05-10T11:40:41.953 INFO:tasks.workunit.client.0.smithi100.stdout: 10 85939 8.94 MB/sec execute 395 sec latency 129.697 ms 2024-05-10T11:40:42.011 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:42.011+0000 7f3fe2ffd640 1 -- 172.21.15.100:0/759982005 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f3fec02b430 con 0x556046b30bf0 2024-05-10T11:40:42.011 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:42.011+0000 7f3fe2ffd640 10 client.19764 handle_command_reply: tid=0 2024-05-10T11:40:42.012 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:42.011+0000 7f400c806740 1 client.19764 shutdown 2024-05-10T11:40:42.013 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:42.012+0000 7f400c806740 1 -- 172.21.15.100:0/759982005 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556046b20910 msgr2=0x556046b2acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:42.014 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:42.012+0000 7f400c806740 1 --2- 172.21.15.100:0/759982005 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556046b20910 0x556046b2acd0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc000f80 tx=0x7f3ffc00fd80 comp rx=0 tx=0).stop 2024-05-10T11:40:42.014 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:42.012+0000 7f400c806740 1 -- 172.21.15.100:0/759982005 shutdown_connections 2024-05-10T11:40:42.014 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:42.012+0000 7f400c806740 1 --2- 172.21.15.100:0/759982005 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x556046b30bf0 0x556046b9e070 crc :-1 s=READY pgs=420 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:42.014 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:42.012+0000 7f400c806740 1 --2- 172.21.15.100:0/759982005 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556046b20910 0x556046b2acd0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:42.014 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:42.012+0000 7f3fe2ffd640 0 client.19764 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:40:42.014 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:42.012+0000 7f400c806740 1 --2- 172.21.15.100:0/759982005 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556046b1b7a0 0x556046b29d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:42.015 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:42.012+0000 7f400c806740 1 --2- 172.21.15.100:0/759982005 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556046b1adb0 0x556046b297d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:42.015 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:42.012+0000 7f400c806740 1 -- 172.21.15.100:0/759982005 >> 172.21.15.100:0/759982005 conn(0x556046b16420 msgr2=0x556046b19330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:40:42.015 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:42.012+0000 7f400c806740 1 -- 172.21.15.100:0/759982005 shutdown_connections 2024-05-10T11:40:42.015 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:42.012+0000 7f400c806740 1 -- 172.21.15.100:0/759982005 wait complete. 2024-05-10T11:40:42.015 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:42.013+0000 7f400c806740 20 client.19764 trim_cache size 0 max 16384 2024-05-10T11:40:42.016 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:40:42.016 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:40:42.016 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:40:42.016 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:40:42.017 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:42.013+0000 7f4009a15640 1 -- 172.21.15.100:0/140997665 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f3ff00b28a0 msgr2=0x7f3ff00b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:42.017 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:42.013+0000 7f4009a15640 1 --2- 172.21.15.100:0/140997665 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f3ff00b28a0 0x7f3ff00b4d60 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x556046b172f0 tx=0x7f3fec005f90 comp rx=0 tx=0).stop 2024-05-10T11:40:42.017 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:42.013+0000 7f4009a15640 1 -- 172.21.15.100:0/140997665 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4004103c20 msgr2=0x7f4004074da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:42.018 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:42.013+0000 7f4009a15640 1 --2- 172.21.15.100:0/140997665 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4004103c20 0x7f4004074da0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3ff4009850 tx=0x7f3ff4003ed0 comp rx=0 tx=0).stop 2024-05-10T11:40:42.018 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:42.013+0000 7f4009a15640 1 -- 172.21.15.100:0/140997665 shutdown_connections 2024-05-10T11:40:42.018 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:42.013+0000 7f4009a15640 1 --2- 172.21.15.100:0/140997665 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f3ff00b28a0 0x7f3ff00b4d60 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:42.018 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:42.013+0000 7f4009a15640 1 --2- 172.21.15.100:0/140997665 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40041045e0 0x7f40040752e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:42.018 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:42.013+0000 7f4009a15640 1 --2- 172.21.15.100:0/140997665 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4004103c20 0x7f4004074da0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:42.018 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:42.013+0000 7f4009a15640 1 --2- 172.21.15.100:0/140997665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4004102a30 0x7f4004076760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:42.018 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:42.014+0000 7f4009a15640 1 -- 172.21.15.100:0/140997665 >> 172.21.15.100:0/140997665 conn(0x7f40040fdfa0 msgr2=0x7f4004106e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:40:42.018 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:42.014+0000 7f4009a15640 1 -- 172.21.15.100:0/140997665 shutdown_connections 2024-05-10T11:40:42.019 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:42.014+0000 7f4009a15640 1 -- 172.21.15.100:0/140997665 wait complete. 2024-05-10T11:40:42.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:40:41 smithi100 ceph-mon[30296]: pgmap v1090: 129 pgs: 129 active+clean; 549 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.1 MiB/s wr, 289 op/s 2024-05-10T11:40:42.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:40:41 smithi178 ceph-mon[33359]: pgmap v1090: 129 pgs: 129 active+clean; 549 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.1 MiB/s wr, 289 op/s 2024-05-10T11:40:42.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:40:41 smithi188 ceph-mon[34075]: pgmap v1090: 129 pgs: 129 active+clean; 549 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.1 MiB/s wr, 289 op/s 2024-05-10T11:40:42.954 INFO:tasks.workunit.client.0.smithi100.stdout: 10 86111 8.96 MB/sec execute 396 sec latency 154.034 ms 2024-05-10T11:40:43.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.565+0000 7f9611ed3700 1 -- 172.21.15.100:0/1674859356 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f960c06b180 msgr2=0x7f960c06b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:43.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.565+0000 7f9611ed3700 1 --2- 172.21.15.100:0/1674859356 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f960c06b180 0x7f960c06b600 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f96000098f0 tx=0x7f9600009c00 comp rx=0 tx=0).stop 2024-05-10T11:40:43.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.565+0000 7f9611ed3700 1 -- 172.21.15.100:0/1674859356 shutdown_connections 2024-05-10T11:40:43.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.565+0000 7f9611ed3700 1 --2- 172.21.15.100:0/1674859356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f960c06bb40 0x7f960c110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:43.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.565+0000 7f9611ed3700 1 --2- 172.21.15.100:0/1674859356 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f960c109510 0x7f960c1098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:43.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.565+0000 7f9611ed3700 1 --2- 172.21.15.100:0/1674859356 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f960c06b180 0x7f960c06b600 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:43.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.565+0000 7f9611ed3700 1 -- 172.21.15.100:0/1674859356 >> 172.21.15.100:0/1674859356 conn(0x7f960c079350 msgr2=0x7f960c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:40:43.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.565+0000 7f9611ed3700 1 -- 172.21.15.100:0/1674859356 shutdown_connections 2024-05-10T11:40:43.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.565+0000 7f9611ed3700 1 -- 172.21.15.100:0/1674859356 wait complete. 2024-05-10T11:40:43.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.566+0000 7f9611ed3700 1 Processor -- start 2024-05-10T11:40:43.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.566+0000 7f9611ed3700 1 -- start start 2024-05-10T11:40:43.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.567+0000 7f9611ed3700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f960c06b180 0x7f960c19d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:43.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.567+0000 7f9611ed3700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f960c06bb40 0x7f960c19db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:43.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.567+0000 7f9611ed3700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f960c109510 0x7f960c197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:43.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.567+0000 7f9611ed3700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f960c111360 con 0x7f960c06b180 2024-05-10T11:40:43.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.567+0000 7f960affd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f960c06bb40 0x7f960c19db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:43.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.567+0000 7f960b7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f960c06b180 0x7f960c19d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:43.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.567+0000 7f960b7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f960c06b180 0x7f960c19d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50056/0 (socket says 172.21.15.100:50056) 2024-05-10T11:40:43.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.567+0000 7f960affd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f960c06bb40 0x7f960c19db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:55012/0 (socket says 172.21.15.100:55012) 2024-05-10T11:40:43.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.567+0000 7f960b7fe700 1 -- 172.21.15.100:0/21662342 learned_addr learned my addr 172.21.15.100:0/21662342 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:40:43.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.567+0000 7f960bfff700 1 --2- 172.21.15.100:0/21662342 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f960c109510 0x7f960c197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:43.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.567+0000 7f9611ed3700 1 -- 172.21.15.100:0/21662342 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f960c1111e0 con 0x7f960c06bb40 2024-05-10T11:40:43.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.567+0000 7f9611ed3700 1 -- 172.21.15.100:0/21662342 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f960c1114e0 con 0x7f960c109510 2024-05-10T11:40:43.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.568+0000 7f960b7fe700 1 -- 172.21.15.100:0/21662342 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f960c109510 msgr2=0x7f960c197710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:43.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.568+0000 7f960b7fe700 1 --2- 172.21.15.100:0/21662342 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f960c109510 0x7f960c197710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:43.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.568+0000 7f960b7fe700 1 -- 172.21.15.100:0/21662342 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f960c06bb40 msgr2=0x7f960c19db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:43.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.568+0000 7f960b7fe700 1 --2- 172.21.15.100:0/21662342 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f960c06bb40 0x7f960c19db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:43.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.568+0000 7f960b7fe700 1 -- 172.21.15.100:0/21662342 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f960c197ff0 con 0x7f960c06b180 2024-05-10T11:40:43.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.568+0000 7f960b7fe700 1 --2- 172.21.15.100:0/21662342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f960c06b180 0x7f960c19d640 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f95fc00e9c0 tx=0x7f95fc00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:40:43.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.568+0000 7f9608ff9700 1 -- 172.21.15.100:0/21662342 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95fc004a30 con 0x7f960c06b180 2024-05-10T11:40:43.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.568+0000 7f9608ff9700 1 -- 172.21.15.100:0/21662342 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f95fc00cc50 con 0x7f960c06b180 2024-05-10T11:40:43.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.568+0000 7f9608ff9700 1 -- 172.21.15.100:0/21662342 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95fc005420 con 0x7f960c06b180 2024-05-10T11:40:43.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.569+0000 7f9611ed3700 1 -- 172.21.15.100:0/21662342 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f960c1982e0 con 0x7f960c06b180 2024-05-10T11:40:43.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.569+0000 7f9611ed3700 1 -- 172.21.15.100:0/21662342 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f960c1a45c0 con 0x7f960c06b180 2024-05-10T11:40:43.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.573+0000 7f9608ff9700 1 -- 172.21.15.100:0/21662342 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f95fc0055c0 con 0x7f960c06b180 2024-05-10T11:40:43.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.574+0000 7f961c766b80 1 Processor -- start 2024-05-10T11:40:43.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.574+0000 7f961c766b80 1 -- start start 2024-05-10T11:40:43.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.575+0000 7f9608ff9700 1 --2- 172.21.15.100:0/21662342 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f95f40b29d0 0x7f95f40b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:43.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.575+0000 7f961c766b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eea1fab4d0 0x55eea1fab8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:43.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.575+0000 7f961c766b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eea1fabdf0 0x55eea1faf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:43.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.575+0000 7f961c766b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eea1fafc80 0x55eea1fb0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:43.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.575+0000 7f961c766b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55eea1efd860 con 0x55eea1fab4d0 2024-05-10T11:40:43.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.575+0000 7f961c766b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55eea1e9c7c0 con 0x55eea1fabdf0 2024-05-10T11:40:43.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.575+0000 7f9608ff9700 1 -- 172.21.15.100:0/21662342 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f95fc0e2520 con 0x7f960c06b180 2024-05-10T11:40:43.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.575+0000 7f961c766b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55eea1ef9820 con 0x55eea1fafc80 2024-05-10T11:40:43.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.575+0000 7f960affd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eea1fabdf0 0x55eea1faf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:43.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.575+0000 7f960affd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eea1fabdf0 0x55eea1faf6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:55020/0 (socket says 172.21.15.100:55020) 2024-05-10T11:40:43.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.575+0000 7f960affd700 1 -- 172.21.15.100:0/1438369830 learned_addr learned my addr 172.21.15.100:0/1438369830 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:40:43.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.575+0000 7f960bfff700 1 --2- 172.21.15.100:0/21662342 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f95f40b29d0 0x7f95f40b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:43.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.575+0000 7f960b7fe700 1 --2- 172.21.15.100:0/1438369830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eea1fab4d0 0x55eea1fab8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:43.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.575+0000 7f960affd700 1 -- 172.21.15.100:0/1438369830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eea1fafc80 msgr2=0x55eea1fb0100 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T11:40:43.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.575+0000 7f960affd700 1 --2- 172.21.15.100:0/1438369830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eea1fafc80 0x55eea1fb0100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:43.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.575+0000 7f960affd700 1 -- 172.21.15.100:0/1438369830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eea1fab4d0 msgr2=0x55eea1fab8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:43.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.575+0000 7f960affd700 1 --2- 172.21.15.100:0/1438369830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eea1fab4d0 0x55eea1fab8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:43.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.575+0000 7f960affd700 1 -- 172.21.15.100:0/1438369830 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eea1e4a8d0 con 0x55eea1fabdf0 2024-05-10T11:40:43.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.575+0000 7f960b7fe700 1 --2- 172.21.15.100:0/1438369830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eea1fab4d0 0x55eea1fab8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:40:43.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.576+0000 7f960affd700 1 --2- 172.21.15.100:0/1438369830 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eea1fabdf0 0x55eea1faf6f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9600009230 tx=0x7f9600005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a854f78c235fbf6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:40:43.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.576+0000 7f960bfff700 1 --2- 172.21.15.100:0/21662342 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f95f40b29d0 0x7f95f40b4e90 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x55eea1fa7710 tx=0x7f9604011040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:40:43.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.576+0000 7f95f1ffb700 1 -- 172.21.15.100:0/1438369830 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f960000f460 con 0x55eea1fabdf0 2024-05-10T11:40:43.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.576+0000 7f95f1ffb700 1 -- 172.21.15.100:0/1438369830 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9600005040 con 0x55eea1fabdf0 2024-05-10T11:40:43.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.576+0000 7f95f1ffb700 1 -- 172.21.15.100:0/1438369830 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96000039b0 con 0x55eea1fabdf0 2024-05-10T11:40:43.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.577+0000 7f961c766b80 1 -- 172.21.15.100:0/1438369830 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eea1fabdf0 msgr2=0x55eea1faf6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:43.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.577+0000 7f961c766b80 1 --2- 172.21.15.100:0/1438369830 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eea1fabdf0 0x55eea1faf6f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9600009230 tx=0x7f9600005dc0 comp rx=0 tx=0).stop 2024-05-10T11:40:43.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.577+0000 7f961c766b80 1 -- 172.21.15.100:0/1438369830 shutdown_connections 2024-05-10T11:40:43.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.577+0000 7f961c766b80 1 --2- 172.21.15.100:0/1438369830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eea1fafc80 0x55eea1fb0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:43.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.577+0000 7f961c766b80 1 --2- 172.21.15.100:0/1438369830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eea1fab4d0 0x55eea1fab8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:43.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.577+0000 7f961c766b80 1 --2- 172.21.15.100:0/1438369830 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eea1fabdf0 0x55eea1faf6f0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:43.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.577+0000 7f961c766b80 1 -- 172.21.15.100:0/1438369830 >> 172.21.15.100:0/1438369830 conn(0x55eea1ed7100 msgr2=0x55eea1fa5c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:40:43.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.577+0000 7f961c766b80 1 -- 172.21.15.100:0/1438369830 shutdown_connections 2024-05-10T11:40:43.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.577+0000 7f961c766b80 1 -- 172.21.15.100:0/1438369830 wait complete. 2024-05-10T11:40:43.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.578+0000 7f961c766b80 1 -- start start 2024-05-10T11:40:43.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.578+0000 7f961c766b80 1 Processor -- start 2024-05-10T11:40:43.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.578+0000 7f961c766b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eea1fab4d0 0x55eea1fb7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:43.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.578+0000 7f961c766b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eea1fabdf0 0x55eea1fb4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:43.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.578+0000 7f961c766b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eea1fafc80 0x55eea1fb4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:43.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.578+0000 7f961c766b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55eea1fb7c70 con 0x55eea1fafc80 2024-05-10T11:40:43.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.578+0000 7f961c766b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55eea1e63c30 con 0x55eea1fab4d0 2024-05-10T11:40:43.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.578+0000 7f961c766b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55eea1fb7df0 con 0x55eea1fabdf0 2024-05-10T11:40:43.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.578+0000 7f961c766b80 10 client.0 _ll_register_callbacks cb 0x55eea1e95920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:40:43.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.579+0000 7f960affd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eea1fabdf0 0x55eea1fb4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:43.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.579+0000 7f960affd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eea1fabdf0 0x55eea1fb4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:45564/0 (socket says 172.21.15.100:45564) 2024-05-10T11:40:43.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.579+0000 7f960affd700 1 -- 172.21.15.100:0/1215366664 learned_addr learned my addr 172.21.15.100:0/1215366664 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:40:43.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.579+0000 7f960b7fe700 1 --2- 172.21.15.100:0/1215366664 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eea1fab4d0 0x55eea1fb7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:43.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.579+0000 7f960bfff700 1 --2- 172.21.15.100:0/1215366664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eea1fafc80 0x55eea1fb4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:43.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.579+0000 7f960affd700 1 -- 172.21.15.100:0/1215366664 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eea1fab4d0 msgr2=0x55eea1fb7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:43.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.579+0000 7f960affd700 1 --2- 172.21.15.100:0/1215366664 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eea1fab4d0 0x55eea1fb7070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:43.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.579+0000 7f960affd700 1 -- 172.21.15.100:0/1215366664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eea1fafc80 msgr2=0x55eea1fb4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:43.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.579+0000 7f960affd700 1 --2- 172.21.15.100:0/1215366664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eea1fafc80 0x55eea1fb4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:43.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.579+0000 7f960affd700 1 -- 172.21.15.100:0/1215366664 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eea1e99690 con 0x55eea1fabdf0 2024-05-10T11:40:43.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.579+0000 7f960b7fe700 1 --2- 172.21.15.100:0/1215366664 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eea1fab4d0 0x55eea1fb7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:40:43.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.580+0000 7f960affd700 1 --2- 172.21.15.100:0/1215366664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eea1fabdf0 0x55eea1fb4410 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f9600009810 tx=0x7f960000f8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49124543f8bb2335 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:40:43.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.580+0000 7f95eaffd700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:40:43.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.580+0000 7f95eaffd700 1 -- 172.21.15.100:0/1215366664 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f960002f070 con 0x55eea1fabdf0 2024-05-10T11:40:43.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.580+0000 7f95eaffd700 1 -- 172.21.15.100:0/1215366664 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f9600003680 con 0x55eea1fabdf0 2024-05-10T11:40:43.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.580+0000 7f961c766b80 1 -- 172.21.15.100:0/1215366664 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9600005040 con 0x55eea1fabdf0 2024-05-10T11:40:43.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.580+0000 7f95eaffd700 1 -- 172.21.15.100:0/1215366664 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f9600003880 con 0x55eea1fabdf0 2024-05-10T11:40:43.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.580+0000 7f961c766b80 10 client.29143 fetch_fsmap learned FSMap version 41 2024-05-10T11:40:43.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.580+0000 7f961c766b80 1 -- 172.21.15.100:0/1215366664 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55eea1fb7c70 con 0x55eea1fabdf0 2024-05-10T11:40:43.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.580+0000 7f95eaffd700 1 -- 172.21.15.100:0/1215366664 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9600005040 con 0x55eea1fabdf0 2024-05-10T11:40:43.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.581+0000 7f95eaffd700 1 -- 172.21.15.100:0/1215366664 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f960000bbb0 con 0x55eea1fabdf0 2024-05-10T11:40:43.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.581+0000 7f961c766b80 10 client.29143 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:40:43.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.581+0000 7f961c766b80 10 client.29143 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:40:43.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.582+0000 7f961c766b80 20 client.29143 populate_metadata read hostname 'smithi100' 2024-05-10T11:40:43.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.582+0000 7f961c766b80 1 --2- 172.21.15.100:0/1215366664 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55eea1fc4740 0x55eea1fc6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:43.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.582+0000 7f961c766b80 4 client.29143 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:40:43.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.582+0000 7f961c766b80 1 -- 172.21.15.100:0/1215366664 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eea1f77650 con 0x55eea1fc4740 2024-05-10T11:40:43.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.582+0000 7f960b7fe700 1 --2- 172.21.15.100:0/1215366664 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55eea1fc4740 0x55eea1fc6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:43.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.584+0000 7f960b7fe700 1 --2- 172.21.15.100:0/1215366664 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55eea1fc4740 0x55eea1fc6b70 crc :-1 s=READY pgs=421 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbc05f13364543ea server_cookie=a6d08b6a6fa28496 in_seq=0 out_seq=0 2024-05-10T11:40:43.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.584+0000 7f95eaffd700 10 client.29143 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:40:43.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.585+0000 7f95eaffd700 1 -- 172.21.15.100:0/1215366664 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55eea1f77650 con 0x55eea1fc4740 2024-05-10T11:40:43.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.585+0000 7f95eaffd700 10 client.29143 handle_command_reply: tid=0 2024-05-10T11:40:43.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.585+0000 7f961c766b80 1 client.29143 shutdown 2024-05-10T11:40:43.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.586+0000 7f961c766b80 1 -- 172.21.15.100:0/1215366664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eea1fabdf0 msgr2=0x55eea1fb4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:43.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.586+0000 7f961c766b80 1 --2- 172.21.15.100:0/1215366664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eea1fabdf0 0x55eea1fb4410 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f9600009810 tx=0x7f960000f8d0 comp rx=0 tx=0).stop 2024-05-10T11:40:43.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.586+0000 7f961c766b80 1 -- 172.21.15.100:0/1215366664 shutdown_connections 2024-05-10T11:40:43.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.586+0000 7f961c766b80 1 --2- 172.21.15.100:0/1215366664 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55eea1fc4740 0x55eea1fc6b70 crc :-1 s=READY pgs=421 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:43.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.586+0000 7f95eaffd700 0 client.29143 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:40:43.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.586+0000 7f961c766b80 1 --2- 172.21.15.100:0/1215366664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eea1fafc80 0x55eea1fb4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:43.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.586+0000 7f961c766b80 1 --2- 172.21.15.100:0/1215366664 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eea1fab4d0 0x55eea1fb7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:43.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.586+0000 7f961c766b80 1 --2- 172.21.15.100:0/1215366664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eea1fabdf0 0x55eea1fb4410 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:43.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.586+0000 7f961c766b80 1 -- 172.21.15.100:0/1215366664 >> 172.21.15.100:0/1215366664 conn(0x55eea1ed7100 msgr2=0x55eea1fa0720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:40:43.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.586+0000 7f961c766b80 1 -- 172.21.15.100:0/1215366664 shutdown_connections 2024-05-10T11:40:43.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.586+0000 7f961c766b80 1 -- 172.21.15.100:0/1215366664 wait complete. 2024-05-10T11:40:43.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.586+0000 7f961c766b80 20 client.29143 trim_cache size 0 max 16384 2024-05-10T11:40:43.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.611+0000 7f961c766b80 1 Processor -- start 2024-05-10T11:40:43.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.611+0000 7f961c766b80 1 -- start start 2024-05-10T11:40:43.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.611+0000 7f961c766b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eea1fab4d0 0x55eea1fa7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:43.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.611+0000 7f961c766b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eea1fabdf0 0x55eea1fc0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:43.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.611+0000 7f961c766b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eea1fafc80 0x55eea1fa5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:43.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.611+0000 7f961c766b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f960c1c8f10 con 0x55eea1fab4d0 2024-05-10T11:40:43.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.611+0000 7f961c766b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f960c1c67a0 con 0x55eea1fafc80 2024-05-10T11:40:43.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.611+0000 7f961c766b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f960c073520 con 0x55eea1fabdf0 2024-05-10T11:40:43.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.612+0000 7f960b7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eea1fab4d0 0x55eea1fa7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:43.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.612+0000 7f960b7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eea1fab4d0 0x55eea1fa7740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50084/0 (socket says 172.21.15.100:50084) 2024-05-10T11:40:43.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.612+0000 7f960b7fe700 1 -- 172.21.15.100:0/2058998909 learned_addr learned my addr 172.21.15.100:0/2058998909 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:40:43.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.612+0000 7f960affd700 1 --2- 172.21.15.100:0/2058998909 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eea1fabdf0 0x55eea1fc0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:43.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.612+0000 7f960b7fe700 1 -- 172.21.15.100:0/2058998909 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eea1fabdf0 msgr2=0x55eea1fc0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:43.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.612+0000 7f960b7fe700 1 --2- 172.21.15.100:0/2058998909 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eea1fabdf0 0x55eea1fc0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:43.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.612+0000 7f960b7fe700 1 -- 172.21.15.100:0/2058998909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eea1fafc80 msgr2=0x55eea1fa5bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T11:40:43.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.612+0000 7f960b7fe700 1 --2- 172.21.15.100:0/2058998909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eea1fafc80 0x55eea1fa5bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:43.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.612+0000 7f960b7fe700 1 -- 172.21.15.100:0/2058998909 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eea1fa7520 con 0x55eea1fab4d0 2024-05-10T11:40:43.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.612+0000 7f960affd700 1 --2- 172.21.15.100:0/2058998909 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eea1fabdf0 0x55eea1fc0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:40:43.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.613+0000 7f960b7fe700 1 --2- 172.21.15.100:0/2058998909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eea1fab4d0 0x55eea1fa7740 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f95fc0dfd10 tx=0x7f95fc009550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4ed700f4cfe205a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:40:43.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.613+0000 7f95f17fa700 1 -- 172.21.15.100:0/2058998909 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95fc0a9ee0 con 0x55eea1fab4d0 2024-05-10T11:40:43.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.614+0000 7f95f17fa700 1 -- 172.21.15.100:0/2058998909 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f95fc0e1040 con 0x55eea1fab4d0 2024-05-10T11:40:43.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.614+0000 7f95f17fa700 1 -- 172.21.15.100:0/2058998909 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95fc0e2260 con 0x55eea1fab4d0 2024-05-10T11:40:43.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.614+0000 7f961c766b80 1 -- 172.21.15.100:0/2058998909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eea1fab4d0 msgr2=0x55eea1fa7740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:43.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.614+0000 7f961c766b80 1 --2- 172.21.15.100:0/2058998909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eea1fab4d0 0x55eea1fa7740 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f95fc0dfd10 tx=0x7f95fc009550 comp rx=0 tx=0).stop 2024-05-10T11:40:43.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.615+0000 7f961c766b80 1 -- 172.21.15.100:0/2058998909 shutdown_connections 2024-05-10T11:40:43.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.615+0000 7f961c766b80 1 --2- 172.21.15.100:0/2058998909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eea1fafc80 0x55eea1fa5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:43.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.615+0000 7f961c766b80 1 --2- 172.21.15.100:0/2058998909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eea1fab4d0 0x55eea1fa7740 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:43.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.615+0000 7f961c766b80 1 --2- 172.21.15.100:0/2058998909 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eea1fabdf0 0x55eea1fc0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:43.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.615+0000 7f961c766b80 1 -- 172.21.15.100:0/2058998909 >> 172.21.15.100:0/2058998909 conn(0x55eea1ed7100 msgr2=0x55eea1f9f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:40:43.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.615+0000 7f961c766b80 1 -- 172.21.15.100:0/2058998909 shutdown_connections 2024-05-10T11:40:43.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.616+0000 7f961c766b80 1 -- 172.21.15.100:0/2058998909 wait complete. 2024-05-10T11:40:43.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.616+0000 7f961c766b80 1 -- start start 2024-05-10T11:40:43.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.616+0000 7f961c766b80 1 Processor -- start 2024-05-10T11:40:43.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.617+0000 7f961c766b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eea1fab4d0 0x55eea1fd5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:43.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.617+0000 7f961c766b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eea1fabdf0 0x55eea1fd6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:43.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.617+0000 7f961c766b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eea1fafc80 0x55eea1fd65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:43.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.617+0000 7f960b7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eea1fab4d0 0x55eea1fd5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:43.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.617+0000 7f960b7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eea1fab4d0 0x55eea1fd5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:45594/0 (socket says 172.21.15.100:45594) 2024-05-10T11:40:43.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.617+0000 7f960b7fe700 1 -- 172.21.15.100:0/280543683 learned_addr learned my addr 172.21.15.100:0/280543683 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:40:43.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.617+0000 7f961c766b80 1 -- 172.21.15.100:0/280543683 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f960c073b90 con 0x55eea1fafc80 2024-05-10T11:40:43.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.617+0000 7f960bfff700 1 --2- 172.21.15.100:0/280543683 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eea1fafc80 0x55eea1fd65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:43.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.617+0000 7f961c766b80 1 -- 172.21.15.100:0/280543683 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f960c075000 con 0x55eea1fabdf0 2024-05-10T11:40:43.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.617+0000 7f960affd700 1 --2- 172.21.15.100:0/280543683 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eea1fabdf0 0x55eea1fd6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:43.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.617+0000 7f961c766b80 1 -- 172.21.15.100:0/280543683 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f960c074260 con 0x55eea1fab4d0 2024-05-10T11:40:43.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.617+0000 7f961c766b80 10 client.0 _ll_register_callbacks cb 0x55eea1e95920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:40:43.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.618+0000 7f960b7fe700 1 -- 172.21.15.100:0/280543683 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eea1fabdf0 msgr2=0x55eea1fd6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:43.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.618+0000 7f960b7fe700 1 --2- 172.21.15.100:0/280543683 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eea1fabdf0 0x55eea1fd6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:43.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.618+0000 7f960b7fe700 1 -- 172.21.15.100:0/280543683 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eea1fafc80 msgr2=0x55eea1fd65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:43.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.618+0000 7f960b7fe700 1 --2- 172.21.15.100:0/280543683 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eea1fafc80 0x55eea1fd65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:43.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.618+0000 7f960b7fe700 1 -- 172.21.15.100:0/280543683 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eea1fbb3a0 con 0x55eea1fab4d0 2024-05-10T11:40:43.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.618+0000 7f960bfff700 1 --2- 172.21.15.100:0/280543683 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eea1fafc80 0x55eea1fd65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:40:43.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.618+0000 7f960affd700 1 --2- 172.21.15.100:0/280543683 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eea1fabdf0 0x55eea1fd6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:40:43.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.618+0000 7f960b7fe700 1 --2- 172.21.15.100:0/280543683 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eea1fab4d0 0x55eea1fd5b40 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f95fc0a9770 tx=0x7f95fc0a8fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad1080abf89ff46e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:40:43.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.618+0000 7f95eaffd700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:40:43.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.618+0000 7f95eaffd700 1 -- 172.21.15.100:0/280543683 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95fc0a9ee0 con 0x55eea1fab4d0 2024-05-10T11:40:43.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.618+0000 7f95eaffd700 1 -- 172.21.15.100:0/280543683 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f95fc0e2260 con 0x55eea1fab4d0 2024-05-10T11:40:43.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.618+0000 7f961c766b80 1 -- 172.21.15.100:0/280543683 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95fc0e1040 con 0x55eea1fab4d0 2024-05-10T11:40:43.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.618+0000 7f95eaffd700 1 -- 172.21.15.100:0/280543683 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f95fc0c6020 con 0x55eea1fab4d0 2024-05-10T11:40:43.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.618+0000 7f961c766b80 10 client.29149 fetch_fsmap learned FSMap version 41 2024-05-10T11:40:43.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.618+0000 7f961c766b80 1 -- 172.21.15.100:0/280543683 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55eea2037240 con 0x55eea1fab4d0 2024-05-10T11:40:43.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.619+0000 7f95eaffd700 1 -- 172.21.15.100:0/280543683 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f95fc0e1040 con 0x55eea1fab4d0 2024-05-10T11:40:43.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.619+0000 7f95eaffd700 1 -- 172.21.15.100:0/280543683 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f95fc0ca070 con 0x55eea1fab4d0 2024-05-10T11:40:43.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.619+0000 7f961c766b80 10 client.29149 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:40:43.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.619+0000 7f961c766b80 10 client.29149 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:40:43.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.619+0000 7f961c766b80 20 client.29149 populate_metadata read hostname 'smithi100' 2024-05-10T11:40:43.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.619+0000 7f961c766b80 1 --2- 172.21.15.100:0/280543683 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55eea1fc4740 0x55eea20397e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:43.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.619+0000 7f961c766b80 4 client.29149 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:40:43.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.619+0000 7f961c766b80 1 -- 172.21.15.100:0/280543683 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55eea1f0b240 con 0x55eea1fc4740 2024-05-10T11:40:43.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.619+0000 7f960affd700 1 --2- 172.21.15.100:0/280543683 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55eea1fc4740 0x55eea20397e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:43.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.620+0000 7f95eaffd700 10 client.29149 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:40:43.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.620+0000 7f960affd700 1 --2- 172.21.15.100:0/280543683 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55eea1fc4740 0x55eea20397e0 crc :-1 s=READY pgs=422 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fce7a39023ac0ed3 server_cookie=5e7db6a5a4cf2582 in_seq=0 out_seq=0 2024-05-10T11:40:43.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.620+0000 7f95eaffd700 1 -- 172.21.15.100:0/280543683 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55eea1f0b240 con 0x55eea1fc4740 2024-05-10T11:40:43.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.620+0000 7f95eaffd700 10 client.29149 handle_command_reply: tid=0 2024-05-10T11:40:43.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.620+0000 7f961c766b80 1 client.29149 shutdown 2024-05-10T11:40:43.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.621+0000 7f961c766b80 1 -- 172.21.15.100:0/280543683 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eea1fab4d0 msgr2=0x55eea1fd5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:43.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.621+0000 7f961c766b80 1 --2- 172.21.15.100:0/280543683 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eea1fab4d0 0x55eea1fd5b40 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f95fc0a9770 tx=0x7f95fc0a8fd0 comp rx=0 tx=0).stop 2024-05-10T11:40:43.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.621+0000 7f961c766b80 1 -- 172.21.15.100:0/280543683 shutdown_connections 2024-05-10T11:40:43.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.621+0000 7f961c766b80 1 --2- 172.21.15.100:0/280543683 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55eea1fc4740 0x55eea20397e0 crc :-1 s=READY pgs=422 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:43.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.621+0000 7f961c766b80 1 --2- 172.21.15.100:0/280543683 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eea1fafc80 0x55eea1fd65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:43.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.621+0000 7f95eaffd700 0 client.29149 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:40:43.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.621+0000 7f961c766b80 1 --2- 172.21.15.100:0/280543683 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eea1fab4d0 0x55eea1fd5b40 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:43.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.621+0000 7f961c766b80 1 --2- 172.21.15.100:0/280543683 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eea1fabdf0 0x55eea1fd6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:43.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.621+0000 7f961c766b80 1 -- 172.21.15.100:0/280543683 >> 172.21.15.100:0/280543683 conn(0x55eea1ed7100 msgr2=0x55eea1fa5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:40:43.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.621+0000 7f961c766b80 1 -- 172.21.15.100:0/280543683 shutdown_connections 2024-05-10T11:40:43.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.621+0000 7f961c766b80 1 -- 172.21.15.100:0/280543683 wait complete. 2024-05-10T11:40:43.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.621+0000 7f961c766b80 20 client.29149 trim_cache size 0 max 16384 2024-05-10T11:40:43.624 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:40:43.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.622+0000 7f9611ed3700 1 -- 172.21.15.100:0/21662342 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f95f40b29d0 msgr2=0x7f95f40b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:43.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.622+0000 7f9611ed3700 1 --2- 172.21.15.100:0/21662342 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f95f40b29d0 0x7f95f40b4e90 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x55eea1fa7710 tx=0x7f9604011040 comp rx=0 tx=0).stop 2024-05-10T11:40:43.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.622+0000 7f9611ed3700 1 -- 172.21.15.100:0/21662342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f960c06b180 msgr2=0x7f960c19d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:43.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.622+0000 7f9611ed3700 1 --2- 172.21.15.100:0/21662342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f960c06b180 0x7f960c19d640 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f95fc00e9c0 tx=0x7f95fc00ecd0 comp rx=0 tx=0).stop 2024-05-10T11:40:43.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.622+0000 7f9611ed3700 1 -- 172.21.15.100:0/21662342 shutdown_connections 2024-05-10T11:40:43.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.622+0000 7f9611ed3700 1 --2- 172.21.15.100:0/21662342 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f95f40b29d0 0x7f95f40b4e90 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:43.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.622+0000 7f9611ed3700 1 --2- 172.21.15.100:0/21662342 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f960c109510 0x7f960c197710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:43.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.622+0000 7f9611ed3700 1 --2- 172.21.15.100:0/21662342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f960c06b180 0x7f960c19d640 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:43.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.622+0000 7f9611ed3700 1 --2- 172.21.15.100:0/21662342 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f960c06bb40 0x7f960c19db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:43.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.622+0000 7f9611ed3700 1 -- 172.21.15.100:0/21662342 >> 172.21.15.100:0/21662342 conn(0x7f960c079350 msgr2=0x7f960c1012e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:40:43.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.622+0000 7f9611ed3700 1 -- 172.21.15.100:0/21662342 shutdown_connections 2024-05-10T11:40:43.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:43.622+0000 7f9611ed3700 1 -- 172.21.15.100:0/21662342 wait complete. 2024-05-10T11:40:43.954 INFO:tasks.workunit.client.0.smithi100.stdout: 10 86287 8.96 MB/sec execute 397 sec latency 92.770 ms 2024-05-10T11:40:44.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:40:43 smithi100 ceph-mon[30296]: pgmap v1091: 129 pgs: 129 active+clean; 546 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 3.6 MiB/s wr, 268 op/s 2024-05-10T11:40:44.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:40:43 smithi178 ceph-mon[33359]: pgmap v1091: 129 pgs: 129 active+clean; 546 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 3.6 MiB/s wr, 268 op/s 2024-05-10T11:40:44.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:40:43 smithi188 ceph-mon[34075]: pgmap v1091: 129 pgs: 129 active+clean; 546 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 3.6 MiB/s wr, 268 op/s 2024-05-10T11:40:44.954 INFO:tasks.workunit.client.0.smithi100.stdout: 10 86478 8.96 MB/sec execute 398 sec latency 60.081 ms 2024-05-10T11:40:45.454 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T11:40:45.454 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T11:40:45.808 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:40:45.954 INFO:tasks.workunit.client.0.smithi100.stdout: 10 86600 8.95 MB/sec execute 399 sec latency 64.951 ms 2024-05-10T11:40:46.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:40:45 smithi100 ceph-mon[30296]: pgmap v1092: 129 pgs: 129 active+clean; 548 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 840 KiB/s rd, 3.6 MiB/s wr, 271 op/s 2024-05-10T11:40:46.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:40:45 smithi178 ceph-mon[33359]: pgmap v1092: 129 pgs: 129 active+clean; 548 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 840 KiB/s rd, 3.6 MiB/s wr, 271 op/s 2024-05-10T11:40:46.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:40:45 smithi188 ceph-mon[34075]: pgmap v1092: 129 pgs: 129 active+clean; 548 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 840 KiB/s rd, 3.6 MiB/s wr, 271 op/s 2024-05-10T11:40:46.954 INFO:tasks.workunit.client.0.smithi100.stdout: 10 86771 8.96 MB/sec execute 400 sec latency 102.621 ms 2024-05-10T11:40:47.955 INFO:tasks.workunit.client.0.smithi100.stdout: 10 86940 8.97 MB/sec execute 401 sec latency 121.090 ms 2024-05-10T11:40:48.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:40:47 smithi178 ceph-mon[33359]: pgmap v1093: 129 pgs: 129 active+clean; 550 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 3.9 MiB/s wr, 289 op/s 2024-05-10T11:40:48.253 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:40:47 smithi100 ceph-mon[30296]: pgmap v1093: 129 pgs: 129 active+clean; 550 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 3.9 MiB/s wr, 289 op/s 2024-05-10T11:40:48.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:40:47 smithi188 ceph-mon[34075]: pgmap v1093: 129 pgs: 129 active+clean; 550 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 3.9 MiB/s wr, 289 op/s 2024-05-10T11:40:48.954 INFO:tasks.workunit.client.0.smithi100.stdout: 10 87065 8.95 MB/sec execute 402 sec latency 112.510 ms 2024-05-10T11:40:49.954 INFO:tasks.workunit.client.0.smithi100.stdout: 10 87201 8.94 MB/sec execute 403 sec latency 111.344 ms 2024-05-10T11:40:50.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:40:49 smithi100 ceph-mon[30296]: pgmap v1094: 129 pgs: 129 active+clean; 552 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 3.7 MiB/s wr, 242 op/s 2024-05-10T11:40:50.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:40:49 smithi178 ceph-mon[33359]: pgmap v1094: 129 pgs: 129 active+clean; 552 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 3.7 MiB/s wr, 242 op/s 2024-05-10T11:40:50.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:40:49 smithi188 ceph-mon[34075]: pgmap v1094: 129 pgs: 129 active+clean; 552 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 3.7 MiB/s wr, 242 op/s 2024-05-10T11:40:50.954 INFO:tasks.workunit.client.0.smithi100.stdout: 10 87319 8.94 MB/sec execute 404 sec latency 165.493 ms 2024-05-10T11:40:51.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:40:50 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:40:51.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.390+0000 7f4af8d9f700 1 -- 172.21.15.100:0/2509782133 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4af410a4a0 msgr2=0x7f4af410a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:51.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.390+0000 7f4af8d9f700 1 --2- 172.21.15.100:0/2509782133 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4af410a4a0 0x7f4af410a880 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8009990 tx=0x7f4ae8009ca0 comp rx=0 tx=0).stop 2024-05-10T11:40:51.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.391+0000 7f4af8d9f700 1 -- 172.21.15.100:0/2509782133 shutdown_connections 2024-05-10T11:40:51.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.391+0000 7f4af8d9f700 1 --2- 172.21.15.100:0/2509782133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4af410efc0 0x7f4af410f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:51.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.391+0000 7f4af8d9f700 1 --2- 172.21.15.100:0/2509782133 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4af410a4a0 0x7f4af410a880 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:51.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.391+0000 7f4af8d9f700 1 --2- 172.21.15.100:0/2509782133 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4af410adc0 0x7f4af410ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:51.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.391+0000 7f4af8d9f700 1 -- 172.21.15.100:0/2509782133 >> 172.21.15.100:0/2509782133 conn(0x7f4af406e8f0 msgr2=0x7f4af406ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:40:51.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.391+0000 7f4af8d9f700 1 -- 172.21.15.100:0/2509782133 shutdown_connections 2024-05-10T11:40:51.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.391+0000 7f4af8d9f700 1 -- 172.21.15.100:0/2509782133 wait complete. 2024-05-10T11:40:51.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.393+0000 7f4af8d9f700 1 Processor -- start 2024-05-10T11:40:51.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.394+0000 7f4af8d9f700 1 -- start start 2024-05-10T11:40:51.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.395+0000 7f4af8d9f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4af410a4a0 0x7f4af4111ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:51.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.395+0000 7f4af8d9f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4af410adc0 0x7f4af4112420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:51.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.395+0000 7f4af8d9f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4af410efc0 0x7f4af4112960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:51.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.395+0000 7f4af8d9f700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4af411bdb0 con 0x7f4af410adc0 2024-05-10T11:40:51.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.395+0000 7f4af8d9f700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4af411bc30 con 0x7f4af410a4a0 2024-05-10T11:40:51.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.395+0000 7f4af8d9f700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4af411bf30 con 0x7f4af410efc0 2024-05-10T11:40:51.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.398+0000 7f4af3fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4af410efc0 0x7f4af4112960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:51.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.398+0000 7f4af37fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4af410a4a0 0x7f4af4111ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:51.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.398+0000 7f4af37fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4af410a4a0 0x7f4af4111ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:35964/0 (socket says 172.21.15.100:35964) 2024-05-10T11:40:51.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.398+0000 7f4af37fe700 1 -- 172.21.15.100:0/1575377025 learned_addr learned my addr 172.21.15.100:0/1575377025 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:40:51.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.399+0000 7f4af37fe700 1 -- 172.21.15.100:0/1575377025 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4af410efc0 msgr2=0x7f4af4112960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:51.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.399+0000 7f4af37fe700 1 --2- 172.21.15.100:0/1575377025 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4af410efc0 0x7f4af4112960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:51.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.399+0000 7f4af37fe700 1 -- 172.21.15.100:0/1575377025 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4af410adc0 msgr2=0x7f4af4112420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:51.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.399+0000 7f4af37fe700 1 --2- 172.21.15.100:0/1575377025 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4af410adc0 0x7f4af4112420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:51.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.399+0000 7f4af37fe700 1 -- 172.21.15.100:0/1575377025 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4af41a64c0 con 0x7f4af410a4a0 2024-05-10T11:40:51.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.399+0000 7f4af37fe700 1 --2- 172.21.15.100:0/1575377025 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4af410a4a0 0x7f4af4111ee0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8005950 tx=0x7f4ae800f510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:40:51.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.400+0000 7f4af0ff9700 1 -- 172.21.15.100:0/1575377025 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae800f900 con 0x7f4af410a4a0 2024-05-10T11:40:51.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.400+0000 7f4af0ff9700 1 -- 172.21.15.100:0/1575377025 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4ae8004830 con 0x7f4af410a4a0 2024-05-10T11:40:51.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.400+0000 7f4af0ff9700 1 -- 172.21.15.100:0/1575377025 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae8005480 con 0x7f4af410a4a0 2024-05-10T11:40:51.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.400+0000 7f4af8d9f700 1 -- 172.21.15.100:0/1575377025 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4af41a6750 con 0x7f4af410a4a0 2024-05-10T11:40:51.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.400+0000 7f4af8d9f700 1 -- 172.21.15.100:0/1575377025 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4af41a6b80 con 0x7f4af410a4a0 2024-05-10T11:40:51.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.408+0000 7f4b04894b80 1 Processor -- start 2024-05-10T11:40:51.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.408+0000 7f4af0ff9700 1 -- 172.21.15.100:0/1575377025 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f4ae8021410 con 0x7f4af410a4a0 2024-05-10T11:40:51.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.409+0000 7f4af0ff9700 1 --2- 172.21.15.100:0/1575377025 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4adc0b29c0 0x7f4adc0b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:51.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.410+0000 7f4af0ff9700 1 -- 172.21.15.100:0/1575377025 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f4ae80e3290 con 0x7f4af410a4a0 2024-05-10T11:40:51.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.411+0000 7f4b04894b80 1 -- start start 2024-05-10T11:40:51.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.411+0000 7f4af3fff700 1 --2- 172.21.15.100:0/1575377025 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4adc0b29c0 0x7f4adc0b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:51.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.411+0000 7f4b04894b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c7a13e900 0x559c7a13ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:51.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.411+0000 7f4b04894b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c7a13f220 0x559c7a142910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:51.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.411+0000 7f4b04894b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c7a142e50 0x559c7a1432b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:51.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.411+0000 7f4b04894b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x559c7a0908e0 con 0x559c7a13e900 2024-05-10T11:40:51.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.411+0000 7f4b04894b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559c7a02f840 con 0x559c7a142e50 2024-05-10T11:40:51.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.411+0000 7f4b04894b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x559c7a08c8a0 con 0x559c7a13f220 2024-05-10T11:40:51.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.411+0000 7f4af37fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c7a13e900 0x559c7a13ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:51.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.411+0000 7f4af37fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c7a13e900 0x559c7a13ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42816/0 (socket says 172.21.15.100:42816) 2024-05-10T11:40:51.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.411+0000 7f4af37fe700 1 -- 172.21.15.100:0/3114337221 learned_addr learned my addr 172.21.15.100:0/3114337221 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:40:51.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.411+0000 7f4af2ffd700 1 --2- 172.21.15.100:0/3114337221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c7a13f220 0x559c7a142910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:51.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.411+0000 7f4af3fff700 1 --2- 172.21.15.100:0/3114337221 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c7a142e50 0x559c7a1432b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:51.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.412+0000 7f4af3fff700 1 --2- 172.21.15.100:0/1575377025 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4adc0b29c0 0x7f4adc0b4e80 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f4aec009ef0 tx=0x7f4aec00c420 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:40:51.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.412+0000 7f4af3fff700 1 -- 172.21.15.100:0/3114337221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c7a13f220 msgr2=0x559c7a142910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:51.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.412+0000 7f4af3fff700 1 --2- 172.21.15.100:0/3114337221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c7a13f220 0x559c7a142910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:51.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.412+0000 7f4af3fff700 1 -- 172.21.15.100:0/3114337221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c7a13e900 msgr2=0x559c7a13ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:51.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.412+0000 7f4af3fff700 1 --2- 172.21.15.100:0/3114337221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c7a13e900 0x559c7a13ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:51.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.412+0000 7f4af3fff700 1 -- 172.21.15.100:0/3114337221 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c79fdd950 con 0x559c7a142e50 2024-05-10T11:40:51.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.412+0000 7f4af3fff700 1 --2- 172.21.15.100:0/3114337221 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c7a142e50 0x559c7a1432b0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4aec009fd0 tx=0x7f4aec019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af01978166ffd0a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:40:51.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.413+0000 7f4ad9ffb700 1 -- 172.21.15.100:0/3114337221 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aec026070 con 0x559c7a142e50 2024-05-10T11:40:51.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.413+0000 7f4ad9ffb700 1 -- 172.21.15.100:0/3114337221 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4aec00ce30 con 0x559c7a142e50 2024-05-10T11:40:51.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.414+0000 7f4ad9ffb700 1 -- 172.21.15.100:0/3114337221 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aec020720 con 0x559c7a142e50 2024-05-10T11:40:51.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.414+0000 7f4b04894b80 1 -- 172.21.15.100:0/3114337221 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c7a142e50 msgr2=0x559c7a1432b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:51.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.414+0000 7f4b04894b80 1 --2- 172.21.15.100:0/3114337221 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c7a142e50 0x559c7a1432b0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4aec009fd0 tx=0x7f4aec019000 comp rx=0 tx=0).stop 2024-05-10T11:40:51.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.414+0000 7f4b04894b80 1 -- 172.21.15.100:0/3114337221 shutdown_connections 2024-05-10T11:40:51.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.414+0000 7f4b04894b80 1 --2- 172.21.15.100:0/3114337221 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c7a142e50 0x559c7a1432b0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:51.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.414+0000 7f4b04894b80 1 --2- 172.21.15.100:0/3114337221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c7a13e900 0x559c7a13ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:51.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.414+0000 7f4b04894b80 1 --2- 172.21.15.100:0/3114337221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c7a13f220 0x559c7a142910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:51.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.414+0000 7f4b04894b80 1 -- 172.21.15.100:0/3114337221 >> 172.21.15.100:0/3114337221 conn(0x559c7a06a180 msgr2=0x559c7a138f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:40:51.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.417+0000 7f4b04894b80 1 -- 172.21.15.100:0/3114337221 shutdown_connections 2024-05-10T11:40:51.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.417+0000 7f4b04894b80 1 -- 172.21.15.100:0/3114337221 wait complete. 2024-05-10T11:40:51.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.418+0000 7f4b04894b80 1 -- start start 2024-05-10T11:40:51.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.418+0000 7f4b04894b80 1 Processor -- start 2024-05-10T11:40:51.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.418+0000 7f4b04894b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c7a13e900 0x559c7a14a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:51.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.418+0000 7f4b04894b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c7a13f220 0x559c7a147560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:51.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.418+0000 7f4b04894b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c7a142e50 0x559c7a147aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:51.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.418+0000 7f4b04894b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x559c7a14b0d0 con 0x559c7a142e50 2024-05-10T11:40:51.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.418+0000 7f4b04894b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559c79ff6cb0 con 0x559c7a13f220 2024-05-10T11:40:51.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.418+0000 7f4b04894b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x559c7a14b250 con 0x559c7a13e900 2024-05-10T11:40:51.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.419+0000 7f4b04894b80 10 client.0 _ll_register_callbacks cb 0x559c7a0289a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:40:51.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.419+0000 7f4af2ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c7a13f220 0x559c7a147560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:51.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.419+0000 7f4af3fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c7a142e50 0x559c7a147aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:51.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.419+0000 7f4af3fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c7a142e50 0x559c7a147aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42824/0 (socket says 172.21.15.100:42824) 2024-05-10T11:40:51.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.419+0000 7f4af3fff700 1 -- 172.21.15.100:0/2738617428 learned_addr learned my addr 172.21.15.100:0/2738617428 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:40:51.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.421+0000 7f4af2ffd700 1 -- 172.21.15.100:0/2738617428 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c7a13e900 msgr2=0x559c7a14a1c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T11:40:51.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.421+0000 7f4af2ffd700 1 --2- 172.21.15.100:0/2738617428 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c7a13e900 0x559c7a14a1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:51.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.421+0000 7f4af2ffd700 1 -- 172.21.15.100:0/2738617428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c7a142e50 msgr2=0x559c7a147aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:51.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.421+0000 7f4af2ffd700 1 --2- 172.21.15.100:0/2738617428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c7a142e50 0x559c7a147aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:51.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.421+0000 7f4af2ffd700 1 -- 172.21.15.100:0/2738617428 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c7a02c710 con 0x559c7a13f220 2024-05-10T11:40:51.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.422+0000 7f4af2ffd700 1 --2- 172.21.15.100:0/2738617428 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c7a13f220 0x559c7a147560 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f4af41a6490 tx=0x7f4ae400e730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df45bfe93b616b82 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:40:51.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.422+0000 7f4ad2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:40:51.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.422+0000 7f4ad2ffd700 1 -- 172.21.15.100:0/2738617428 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae4017070 con 0x559c7a13f220 2024-05-10T11:40:51.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.423+0000 7f4b04894b80 1 -- 172.21.15.100:0/2738617428 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4aec00ce30 con 0x559c7a13f220 2024-05-10T11:40:51.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.423+0000 7f4ad2ffd700 1 -- 172.21.15.100:0/2738617428 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f4ae4013470 con 0x559c7a13f220 2024-05-10T11:40:51.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.423+0000 7f4ad2ffd700 1 -- 172.21.15.100:0/2738617428 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f4ae4023050 con 0x559c7a13f220 2024-05-10T11:40:51.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.423+0000 7f4b04894b80 10 client.29042 fetch_fsmap learned FSMap version 41 2024-05-10T11:40:51.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.423+0000 7f4b04894b80 1 -- 172.21.15.100:0/2738617428 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x559c7a1559d0 con 0x559c7a13f220 2024-05-10T11:40:51.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.423+0000 7f4ad2ffd700 1 -- 172.21.15.100:0/2738617428 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4aec00ce30 con 0x559c7a13f220 2024-05-10T11:40:51.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.424+0000 7f4ad2ffd700 1 -- 172.21.15.100:0/2738617428 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f4ae4012670 con 0x559c7a13f220 2024-05-10T11:40:51.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.424+0000 7f4b04894b80 10 client.29042 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:40:51.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.424+0000 7f4b04894b80 10 client.29042 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:40:51.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.424+0000 7f4b04894b80 20 client.29042 populate_metadata read hostname 'smithi100' 2024-05-10T11:40:51.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.424+0000 7f4b04894b80 1 --2- 172.21.15.100:0/2738617428 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x559c7a158300 0x559c7a15a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:51.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.424+0000 7f4b04894b80 4 client.29042 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:40:51.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.424+0000 7f4b04894b80 1 -- 172.21.15.100:0/2738617428 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c7a10a6d0 con 0x559c7a158300 2024-05-10T11:40:51.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.424+0000 7f4af37fe700 1 --2- 172.21.15.100:0/2738617428 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x559c7a158300 0x559c7a15a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:51.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.425+0000 7f4af37fe700 1 --2- 172.21.15.100:0/2738617428 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x559c7a158300 0x559c7a15a7a0 crc :-1 s=READY pgs=423 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f9230f3ef4f6491 server_cookie=e8c694b36782974b in_seq=0 out_seq=0 2024-05-10T11:40:51.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.425+0000 7f4ad2ffd700 10 client.29042 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:40:51.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.426+0000 7f4ad2ffd700 1 -- 172.21.15.100:0/2738617428 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559c7a10a6d0 con 0x559c7a158300 2024-05-10T11:40:51.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.426+0000 7f4ad2ffd700 10 client.29042 handle_command_reply: tid=0 2024-05-10T11:40:51.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.426+0000 7f4b04894b80 1 client.29042 shutdown 2024-05-10T11:40:51.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.426+0000 7f4b04894b80 1 -- 172.21.15.100:0/2738617428 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c7a13f220 msgr2=0x559c7a147560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:51.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.426+0000 7f4b04894b80 1 --2- 172.21.15.100:0/2738617428 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c7a13f220 0x559c7a147560 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f4af41a6490 tx=0x7f4ae400e730 comp rx=0 tx=0).stop 2024-05-10T11:40:51.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.427+0000 7f4b04894b80 1 -- 172.21.15.100:0/2738617428 shutdown_connections 2024-05-10T11:40:51.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.427+0000 7f4b04894b80 1 --2- 172.21.15.100:0/2738617428 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x559c7a158300 0x559c7a15a7a0 crc :-1 s=READY pgs=423 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:51.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.427+0000 7f4b04894b80 1 --2- 172.21.15.100:0/2738617428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c7a142e50 0x559c7a147aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:51.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.427+0000 7f4ad2ffd700 0 client.29042 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:40:51.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.427+0000 7f4b04894b80 1 --2- 172.21.15.100:0/2738617428 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c7a13e900 0x559c7a14a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:51.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.427+0000 7f4b04894b80 1 --2- 172.21.15.100:0/2738617428 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c7a13f220 0x559c7a147560 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:51.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.427+0000 7f4b04894b80 1 -- 172.21.15.100:0/2738617428 >> 172.21.15.100:0/2738617428 conn(0x559c7a06a180 msgr2=0x559c7a1442b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:40:51.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.427+0000 7f4b04894b80 1 -- 172.21.15.100:0/2738617428 shutdown_connections 2024-05-10T11:40:51.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.427+0000 7f4b04894b80 1 -- 172.21.15.100:0/2738617428 wait complete. 2024-05-10T11:40:51.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.427+0000 7f4b04894b80 20 client.29042 trim_cache size 0 max 16384 2024-05-10T11:40:51.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.456+0000 7f4b04894b80 1 Processor -- start 2024-05-10T11:40:51.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.456+0000 7f4b04894b80 1 -- start start 2024-05-10T11:40:51.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.457+0000 7f4b04894b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c7a13e900 0x559c7a1600b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:51.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.457+0000 7f4b04894b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c7a13f220 0x559c7a1605f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:51.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.457+0000 7f4b04894b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c7a142e50 0x559c7a13d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:51.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.457+0000 7f4af3fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c7a142e50 0x559c7a13d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:51.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.457+0000 7f4af3fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c7a142e50 0x559c7a13d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42832/0 (socket says 172.21.15.100:42832) 2024-05-10T11:40:51.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.457+0000 7f4af3fff700 1 -- 172.21.15.100:0/4264063609 learned_addr learned my addr 172.21.15.100:0/4264063609 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:40:51.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.457+0000 7f4af2ffd700 1 --2- 172.21.15.100:0/4264063609 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c7a13f220 0x559c7a1605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:51.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.457+0000 7f4b04894b80 1 -- 172.21.15.100:0/4264063609 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae0011280 con 0x559c7a142e50 2024-05-10T11:40:51.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.457+0000 7f4b04894b80 1 -- 172.21.15.100:0/4264063609 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae000eb10 con 0x559c7a13f220 2024-05-10T11:40:51.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.457+0000 7f4b04894b80 1 -- 172.21.15.100:0/4264063609 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae000c960 con 0x559c7a13e900 2024-05-10T11:40:51.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.457+0000 7f4af37fe700 1 --2- 172.21.15.100:0/4264063609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c7a13e900 0x559c7a1600b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:51.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.458+0000 7f4af2ffd700 1 -- 172.21.15.100:0/4264063609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c7a13e900 msgr2=0x559c7a1600b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:51.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.458+0000 7f4af2ffd700 1 --2- 172.21.15.100:0/4264063609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c7a13e900 0x559c7a1600b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:51.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.458+0000 7f4af2ffd700 1 -- 172.21.15.100:0/4264063609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c7a142e50 msgr2=0x559c7a13d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:51.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.458+0000 7f4af2ffd700 1 --2- 172.21.15.100:0/4264063609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c7a142e50 0x559c7a13d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:51.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.458+0000 7f4af2ffd700 1 -- 172.21.15.100:0/4264063609 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c7a14a9c0 con 0x559c7a13f220 2024-05-10T11:40:51.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.458+0000 7f4af37fe700 1 --2- 172.21.15.100:0/4264063609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c7a13e900 0x559c7a1600b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:40:51.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.458+0000 7f4af2ffd700 1 --2- 172.21.15.100:0/4264063609 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c7a13f220 0x559c7a1605f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x559c7a148b30 tx=0x7f4ae4006e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1f566076427c657 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:40:51.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.458+0000 7f4ad97fa700 1 -- 172.21.15.100:0/4264063609 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae402c070 con 0x559c7a13f220 2024-05-10T11:40:51.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.458+0000 7f4ad97fa700 1 -- 172.21.15.100:0/4264063609 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4ae4020070 con 0x559c7a13f220 2024-05-10T11:40:51.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.458+0000 7f4ad97fa700 1 -- 172.21.15.100:0/4264063609 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae4017070 con 0x559c7a13f220 2024-05-10T11:40:51.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.459+0000 7f4b04894b80 1 -- 172.21.15.100:0/4264063609 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c7a13f220 msgr2=0x559c7a1605f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:51.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.459+0000 7f4b04894b80 1 --2- 172.21.15.100:0/4264063609 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c7a13f220 0x559c7a1605f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x559c7a148b30 tx=0x7f4ae4006e10 comp rx=0 tx=0).stop 2024-05-10T11:40:51.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.460+0000 7f4b04894b80 1 -- 172.21.15.100:0/4264063609 shutdown_connections 2024-05-10T11:40:51.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.460+0000 7f4b04894b80 1 --2- 172.21.15.100:0/4264063609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c7a142e50 0x559c7a13d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:51.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.460+0000 7f4b04894b80 1 --2- 172.21.15.100:0/4264063609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c7a13e900 0x559c7a1600b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:51.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.460+0000 7f4b04894b80 1 --2- 172.21.15.100:0/4264063609 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c7a13f220 0x559c7a1605f0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:51.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.460+0000 7f4b04894b80 1 -- 172.21.15.100:0/4264063609 >> 172.21.15.100:0/4264063609 conn(0x559c7a06a180 msgr2=0x559c7a145e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:40:51.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.460+0000 7f4b04894b80 1 -- 172.21.15.100:0/4264063609 shutdown_connections 2024-05-10T11:40:51.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.460+0000 7f4b04894b80 1 -- 172.21.15.100:0/4264063609 wait complete. 2024-05-10T11:40:51.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.461+0000 7f4b04894b80 1 -- start start 2024-05-10T11:40:51.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.461+0000 7f4b04894b80 1 Processor -- start 2024-05-10T11:40:51.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.461+0000 7f4b04894b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c7a13e900 0x559c7a141100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:51.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.461+0000 7f4b04894b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c7a13f220 0x559c7a1505c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:51.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.461+0000 7f4b04894b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c7a142e50 0x559c7a13f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:51.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.461+0000 7f4b04894b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae000cfd0 con 0x559c7a13e900 2024-05-10T11:40:51.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.461+0000 7f4b04894b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae000e440 con 0x559c7a13f220 2024-05-10T11:40:51.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.461+0000 7f4b04894b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae000d6a0 con 0x559c7a142e50 2024-05-10T11:40:51.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.461+0000 7f4b04894b80 10 client.0 _ll_register_callbacks cb 0x559c7a0289a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:40:51.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.462+0000 7f4af37fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c7a13e900 0x559c7a141100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:51.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.462+0000 7f4af37fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c7a13e900 0x559c7a141100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42848/0 (socket says 172.21.15.100:42848) 2024-05-10T11:40:51.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.462+0000 7f4af37fe700 1 -- 172.21.15.100:0/2300593479 learned_addr learned my addr 172.21.15.100:0/2300593479 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:40:51.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.462+0000 7f4af3fff700 1 --2- 172.21.15.100:0/2300593479 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c7a142e50 0x559c7a13f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:51.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.462+0000 7f4af2ffd700 1 --2- 172.21.15.100:0/2300593479 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c7a13f220 0x559c7a1505c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:51.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.462+0000 7f4af37fe700 1 -- 172.21.15.100:0/2300593479 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c7a142e50 msgr2=0x559c7a13f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:51.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.462+0000 7f4af37fe700 1 --2- 172.21.15.100:0/2300593479 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c7a142e50 0x559c7a13f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:51.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.462+0000 7f4af37fe700 1 -- 172.21.15.100:0/2300593479 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c7a13f220 msgr2=0x559c7a1505c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:51.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.462+0000 7f4af37fe700 1 --2- 172.21.15.100:0/2300593479 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c7a13f220 0x559c7a1505c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:51.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.462+0000 7f4af37fe700 1 -- 172.21.15.100:0/2300593479 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c7a150fd0 con 0x559c7a13e900 2024-05-10T11:40:51.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.462+0000 7f4af37fe700 1 --2- 172.21.15.100:0/2300593479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c7a13e900 0x559c7a141100 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f4ae80e3200 tx=0x7f4ae801a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d97cc6a637bc0f35 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:40:51.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.462+0000 7f4ad2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:40:51.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.462+0000 7f4ad2ffd700 1 -- 172.21.15.100:0/2300593479 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae80ae930 con 0x559c7a13e900 2024-05-10T11:40:51.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.462+0000 7f4b04894b80 1 -- 172.21.15.100:0/2300593479 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ae4020070 con 0x559c7a13e900 2024-05-10T11:40:51.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.463+0000 7f4ad2ffd700 1 -- 172.21.15.100:0/2300593479 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f4ae8004ce0 con 0x559c7a13e900 2024-05-10T11:40:51.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.463+0000 7f4ad2ffd700 1 -- 172.21.15.100:0/2300593479 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f4ae80b3350 con 0x559c7a13e900 2024-05-10T11:40:51.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.463+0000 7f4b04894b80 10 client.19791 fetch_fsmap learned FSMap version 41 2024-05-10T11:40:51.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.463+0000 7f4b04894b80 1 -- 172.21.15.100:0/2300593479 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x559c7a1c41a0 con 0x559c7a13e900 2024-05-10T11:40:51.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.463+0000 7f4ad2ffd700 1 -- 172.21.15.100:0/2300593479 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4ae4020070 con 0x559c7a13e900 2024-05-10T11:40:51.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.465+0000 7f4ad2ffd700 1 -- 172.21.15.100:0/2300593479 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f4ae80cb030 con 0x559c7a13e900 2024-05-10T11:40:51.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.465+0000 7f4b04894b80 10 client.19791 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:40:51.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.465+0000 7f4b04894b80 10 client.19791 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:40:51.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.465+0000 7f4b04894b80 20 client.19791 populate_metadata read hostname 'smithi100' 2024-05-10T11:40:51.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.465+0000 7f4b04894b80 1 --2- 172.21.15.100:0/2300593479 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x559c7a1c48b0 0x559c7a1c6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:51.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.465+0000 7f4b04894b80 4 client.19791 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T11:40:51.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.465+0000 7f4b04894b80 1 -- 172.21.15.100:0/2300593479 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559c7a09e2c0 con 0x559c7a1c48b0 2024-05-10T11:40:51.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.465+0000 7f4af2ffd700 1 --2- 172.21.15.100:0/2300593479 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x559c7a1c48b0 0x559c7a1c6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:51.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.466+0000 7f4ad2ffd700 10 client.19791 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:40:51.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.466+0000 7f4af2ffd700 1 --2- 172.21.15.100:0/2300593479 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x559c7a1c48b0 0x559c7a1c6cd0 crc :-1 s=READY pgs=424 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5d05762aaf3b34e server_cookie=9c62529684b14ef5 in_seq=0 out_seq=0 2024-05-10T11:40:51.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.468+0000 7f4ad2ffd700 1 -- 172.21.15.100:0/2300593479 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559c7a09e2c0 con 0x559c7a1c48b0 2024-05-10T11:40:51.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.468+0000 7f4ad2ffd700 10 client.19791 handle_command_reply: tid=0 2024-05-10T11:40:51.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.468+0000 7f4b04894b80 1 client.19791 shutdown 2024-05-10T11:40:51.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.468+0000 7f4b04894b80 1 -- 172.21.15.100:0/2300593479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c7a13e900 msgr2=0x559c7a141100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:51.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.468+0000 7f4b04894b80 1 --2- 172.21.15.100:0/2300593479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c7a13e900 0x559c7a141100 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f4ae80e3200 tx=0x7f4ae801a040 comp rx=0 tx=0).stop 2024-05-10T11:40:51.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.468+0000 7f4b04894b80 1 -- 172.21.15.100:0/2300593479 shutdown_connections 2024-05-10T11:40:51.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.468+0000 7f4b04894b80 1 --2- 172.21.15.100:0/2300593479 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x559c7a1c48b0 0x559c7a1c6cd0 crc :-1 s=READY pgs=424 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:51.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.468+0000 7f4b04894b80 1 --2- 172.21.15.100:0/2300593479 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c7a142e50 0x559c7a13f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:51.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.468+0000 7f4b04894b80 1 --2- 172.21.15.100:0/2300593479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c7a13e900 0x559c7a141100 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:51.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.468+0000 7f4b04894b80 1 --2- 172.21.15.100:0/2300593479 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c7a13f220 0x559c7a1505c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:51.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.468+0000 7f4b04894b80 1 -- 172.21.15.100:0/2300593479 >> 172.21.15.100:0/2300593479 conn(0x559c7a06a180 msgr2=0x559c7a13d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:40:51.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.468+0000 7f4ad2ffd700 0 client.19791 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:40:51.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.468+0000 7f4b04894b80 1 -- 172.21.15.100:0/2300593479 shutdown_connections 2024-05-10T11:40:51.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.469+0000 7f4b04894b80 1 -- 172.21.15.100:0/2300593479 wait complete. 2024-05-10T11:40:51.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.469+0000 7f4b04894b80 20 client.19791 trim_cache size 0 max 16384 2024-05-10T11:40:51.271 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:40:51.271 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-05-10T11:40:51.271 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "3c8fea61-b040-48ee-91e1-f5e5c8d221f6", 2024-05-10T11:40:51.272 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-05-10T11:40:51.272 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:40:51.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.470+0000 7f4af8d9f700 1 -- 172.21.15.100:0/1575377025 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4adc0b29c0 msgr2=0x7f4adc0b4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:51.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.470+0000 7f4af8d9f700 1 --2- 172.21.15.100:0/1575377025 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4adc0b29c0 0x7f4adc0b4e80 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f4aec009ef0 tx=0x7f4aec00c420 comp rx=0 tx=0).stop 2024-05-10T11:40:51.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.470+0000 7f4af8d9f700 1 -- 172.21.15.100:0/1575377025 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4af410a4a0 msgr2=0x7f4af4111ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:51.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.470+0000 7f4af8d9f700 1 --2- 172.21.15.100:0/1575377025 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4af410a4a0 0x7f4af4111ee0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8005950 tx=0x7f4ae800f510 comp rx=0 tx=0).stop 2024-05-10T11:40:51.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.470+0000 7f4af8d9f700 1 -- 172.21.15.100:0/1575377025 shutdown_connections 2024-05-10T11:40:51.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.470+0000 7f4af8d9f700 1 --2- 172.21.15.100:0/1575377025 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f4adc0b29c0 0x7f4adc0b4e80 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:51.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.470+0000 7f4af8d9f700 1 --2- 172.21.15.100:0/1575377025 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4af410efc0 0x7f4af4112960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:51.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.470+0000 7f4af8d9f700 1 --2- 172.21.15.100:0/1575377025 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4af410a4a0 0x7f4af4111ee0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:51.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.470+0000 7f4af8d9f700 1 --2- 172.21.15.100:0/1575377025 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4af410adc0 0x7f4af4112420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:51.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.470+0000 7f4af8d9f700 1 -- 172.21.15.100:0/1575377025 >> 172.21.15.100:0/1575377025 conn(0x7f4af406e8f0 msgr2=0x7f4af410c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:40:51.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.470+0000 7f4af8d9f700 1 -- 172.21.15.100:0/1575377025 shutdown_connections 2024-05-10T11:40:51.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:40:50.470+0000 7f4af8d9f700 1 -- 172.21.15.100:0/1575377025 wait complete. 2024-05-10T11:40:51.273 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:40:50 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:40:51.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:40:50 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:40:51.954 INFO:tasks.workunit.client.0.smithi100.stdout: 10 87458 8.93 MB/sec execute 405 sec latency 195.414 ms 2024-05-10T11:40:52.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:40:51 smithi178 ceph-mon[33359]: scrub queued for path: / 2024-05-10T11:40:52.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:40:51 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:40:52.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:40:51 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:40:52.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:40:51 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:40:52.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:40:51 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:40:52.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:40:51 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:40:52.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:40:51 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:40:52.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:40:51 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:40:52.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:40:51 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:40:52.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:40:51 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:40:52.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:40:51 smithi178 ceph-mon[33359]: pgmap v1095: 129 pgs: 129 active+clean; 559 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 865 KiB/s rd, 4.5 MiB/s wr, 312 op/s 2024-05-10T11:40:52.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:40:51 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:40:52.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:40:51 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:40:52.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:40:51 smithi188 ceph-mon[34075]: scrub queued for path: / 2024-05-10T11:40:52.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:40:51 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:40:52.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:40:51 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:40:52.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:40:51 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:40:52.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:40:51 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:40:52.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:40:51 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:40:52.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:40:51 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:40:52.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:40:51 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:40:52.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:40:51 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:40:52.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:40:51 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:40:52.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:40:51 smithi188 ceph-mon[34075]: pgmap v1095: 129 pgs: 129 active+clean; 559 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 865 KiB/s rd, 4.5 MiB/s wr, 312 op/s 2024-05-10T11:40:52.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:40:51 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:40:52.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:40:51 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:40:52.284 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:40:51 smithi100 ceph-mon[30296]: scrub queued for path: / 2024-05-10T11:40:52.284 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:40:51 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:40:52.284 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:40:51 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:40:52.284 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:40:51 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:40:52.284 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:40:51 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:40:52.284 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:40:51 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:40:52.284 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:40:51 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:40:52.284 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:40:51 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:40:52.284 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:40:51 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:40:52.284 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:40:51 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:40:52.284 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:40:51 smithi100 ceph-mon[30296]: pgmap v1095: 129 pgs: 129 active+clean; 559 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 865 KiB/s rd, 4.5 MiB/s wr, 312 op/s 2024-05-10T11:40:52.285 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:40:51 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:40:52.285 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:40:51 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:40:52.954 INFO:tasks.workunit.client.0.smithi100.stdout: 10 87623 8.94 MB/sec execute 406 sec latency 169.083 ms 2024-05-10T11:40:53.954 INFO:tasks.workunit.client.0.smithi100.stdout: 10 87751 8.93 MB/sec execute 407 sec latency 195.041 ms 2024-05-10T11:40:54.087 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:40:53 smithi100 ceph-mon[30296]: pgmap v1096: 129 pgs: 129 active+clean; 560 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 3.9 MiB/s wr, 265 op/s 2024-05-10T11:40:54.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:40:53 smithi178 ceph-mon[33359]: pgmap v1096: 129 pgs: 129 active+clean; 560 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 3.9 MiB/s wr, 265 op/s 2024-05-10T11:40:54.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:40:53 smithi188 ceph-mon[34075]: pgmap v1096: 129 pgs: 129 active+clean; 560 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 3.9 MiB/s wr, 265 op/s 2024-05-10T11:40:54.954 INFO:tasks.workunit.client.0.smithi100.stdout: 10 87904 8.95 MB/sec execute 408 sec latency 138.060 ms 2024-05-10T11:40:55.124 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.124+0000 7fed97654640 1 -- 172.21.15.100:0/1576641673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed901034d0 msgr2=0x7fed901058c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:55.125 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.124+0000 7fed97654640 1 --2- 172.21.15.100:0/1576641673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed901034d0 0x7fed901058c0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fed8c009900 tx=0x7fed8c02a410 comp rx=0 tx=0).stop 2024-05-10T11:40:55.125 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.124+0000 7fed97654640 1 -- 172.21.15.100:0/1576641673 shutdown_connections 2024-05-10T11:40:55.125 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.124+0000 7fed97654640 1 --2- 172.21.15.100:0/1576641673 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed90108730 0x7fed9010ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:55.125 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.124+0000 7fed97654640 1 --2- 172.21.15.100:0/1576641673 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed90105e00 0x7fed901081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:55.125 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.124+0000 7fed97654640 1 --2- 172.21.15.100:0/1576641673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed901034d0 0x7fed901058c0 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:55.125 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.124+0000 7fed97654640 1 -- 172.21.15.100:0/1576641673 >> 172.21.15.100:0/1576641673 conn(0x7fed900fce80 msgr2=0x7fed900ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:40:55.125 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.125+0000 7fed97654640 1 -- 172.21.15.100:0/1576641673 shutdown_connections 2024-05-10T11:40:55.125 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.125+0000 7fed97654640 1 -- 172.21.15.100:0/1576641673 wait complete. 2024-05-10T11:40:55.126 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.126+0000 7fed97654640 1 Processor -- start 2024-05-10T11:40:55.127 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.126+0000 7fed97654640 1 -- start start 2024-05-10T11:40:55.127 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.126+0000 7fed97654640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed90105e00 0x7fed90102e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:55.127 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.126+0000 7fed97654640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed90108730 0x7fed90101480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:55.127 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.126+0000 7fed97654640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed901019c0 0x7fed90101e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:55.127 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.126+0000 7fed97654640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fed9010e460 con 0x7fed90108730 2024-05-10T11:40:55.127 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.126+0000 7fed97654640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fed9010e2e0 con 0x7fed90105e00 2024-05-10T11:40:55.127 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.126+0000 7fed97654640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fed9010e5e0 con 0x7fed901019c0 2024-05-10T11:40:55.127 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.126+0000 7fed953f0640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed90105e00 0x7fed90102e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:55.127 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.126+0000 7fed94bef640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed90108730 0x7fed90101480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:55.127 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.126+0000 7fed95bf1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed901019c0 0x7fed90101e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:55.127 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.126+0000 7fed953f0640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed90105e00 0x7fed90102e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:36022/0 (socket says 172.21.15.100:36022) 2024-05-10T11:40:55.127 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.126+0000 7fed953f0640 1 -- 172.21.15.100:0/1674284108 learned_addr learned my addr 172.21.15.100:0/1674284108 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:40:55.127 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.126+0000 7fed953f0640 1 -- 172.21.15.100:0/1674284108 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed901019c0 msgr2=0x7fed90101e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:55.128 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.126+0000 7fed953f0640 1 --2- 172.21.15.100:0/1674284108 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed901019c0 0x7fed90101e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:55.128 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.126+0000 7fed953f0640 1 -- 172.21.15.100:0/1674284108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed90108730 msgr2=0x7fed90101480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:55.128 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.126+0000 7fed953f0640 1 --2- 172.21.15.100:0/1674284108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed90108730 0x7fed90101480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:55.128 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.126+0000 7fed953f0640 1 -- 172.21.15.100:0/1674284108 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed901a2f80 con 0x7fed90105e00 2024-05-10T11:40:55.128 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.126+0000 7fed94bef640 1 --2- 172.21.15.100:0/1674284108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed90108730 0x7fed90101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:40:55.128 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.127+0000 7fed953f0640 1 --2- 172.21.15.100:0/1674284108 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed90105e00 0x7fed90102e40 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fed8c02a980 tx=0x7fed8c003cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:40:55.128 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.127+0000 7fed867fc640 1 -- 172.21.15.100:0/1674284108 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed8c03f020 con 0x7fed90105e00 2024-05-10T11:40:55.129 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.127+0000 7fed867fc640 1 -- 172.21.15.100:0/1674284108 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fed8c031b90 con 0x7fed90105e00 2024-05-10T11:40:55.129 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.127+0000 7fed867fc640 1 -- 172.21.15.100:0/1674284108 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed8c035070 con 0x7fed90105e00 2024-05-10T11:40:55.129 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.127+0000 7fed97654640 1 -- 172.21.15.100:0/1674284108 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed901a3210 con 0x7fed90105e00 2024-05-10T11:40:55.129 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.127+0000 7fed97654640 1 -- 172.21.15.100:0/1674284108 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed901a3680 con 0x7fed90105e00 2024-05-10T11:40:55.131 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.131+0000 7fed9a369740 1 Processor -- start 2024-05-10T11:40:55.131 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.131+0000 7fed867fc640 1 -- 172.21.15.100:0/1674284108 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fed8c042020 con 0x7fed90105e00 2024-05-10T11:40:55.131 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.131+0000 7fed9a369740 1 -- start start 2024-05-10T11:40:55.131 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.131+0000 7fed9a369740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558063e685c0 0x558063e689e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:55.132 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.131+0000 7fed9a369740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558063e68fb0 0x558063e6dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:55.132 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.131+0000 7fed9a369740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558063e6e120 0x558063e705b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:55.132 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.131+0000 7fed9a369740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x558063dc45d0 con 0x558063e68fb0 2024-05-10T11:40:55.132 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.131+0000 7fed9a369740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558063d7d890 con 0x558063e685c0 2024-05-10T11:40:55.132 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.131+0000 7fed9a369740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558063dc0590 con 0x558063e6e120 2024-05-10T11:40:55.132 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.131+0000 7fed95bf1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558063e685c0 0x558063e689e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:55.132 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.131+0000 7fed95bf1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558063e685c0 0x558063e689e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:36028/0 (socket says 172.21.15.100:36028) 2024-05-10T11:40:55.132 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.131+0000 7fed95bf1640 1 -- 172.21.15.100:0/3172123825 learned_addr learned my addr 172.21.15.100:0/3172123825 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:40:55.132 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.131+0000 7fed953f0640 1 --2- 172.21.15.100:0/3172123825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558063e68fb0 0x558063e6dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:55.132 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.132+0000 7fed95bf1640 1 -- 172.21.15.100:0/3172123825 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558063e6e120 msgr2=0x558063e705b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T11:40:55.132 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.132+0000 7fed95bf1640 1 --2- 172.21.15.100:0/3172123825 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558063e6e120 0x558063e705b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:55.132 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.132+0000 7fed95bf1640 1 -- 172.21.15.100:0/3172123825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558063e68fb0 msgr2=0x558063e6dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:55.132 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.132+0000 7fed95bf1640 1 --2- 172.21.15.100:0/3172123825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558063e68fb0 0x558063e6dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:55.132 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.132+0000 7fed95bf1640 1 -- 172.21.15.100:0/3172123825 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558063ad6210 con 0x558063e685c0 2024-05-10T11:40:55.133 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.132+0000 7fed953f0640 1 --2- 172.21.15.100:0/3172123825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558063e68fb0 0x558063e6dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:40:55.135 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.132+0000 7fed95bf1640 1 --2- 172.21.15.100:0/3172123825 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558063e685c0 0x558063e689e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fed880039f0 tx=0x7fed88003eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10a3082fbc50fef2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:40:55.135 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.132+0000 7fed7b7fe640 1 -- 172.21.15.100:0/3172123825 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed8801a470 con 0x558063e685c0 2024-05-10T11:40:55.135 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.132+0000 7fed7b7fe640 1 -- 172.21.15.100:0/3172123825 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fed88015070 con 0x558063e685c0 2024-05-10T11:40:55.135 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.132+0000 7fed7b7fe640 1 -- 172.21.15.100:0/3172123825 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed88010770 con 0x558063e685c0 2024-05-10T11:40:55.135 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.133+0000 7fed867fc640 1 --2- 172.21.15.100:0/1674284108 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fed7c0b29f0 0x7fed7c0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:55.135 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.133+0000 7fed953f0640 1 --2- 172.21.15.100:0/1674284108 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fed7c0b29f0 0x7fed7c0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:55.135 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.133+0000 7fed9a369740 1 -- 172.21.15.100:0/3172123825 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558063e685c0 msgr2=0x558063e689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:55.135 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.133+0000 7fed9a369740 1 --2- 172.21.15.100:0/3172123825 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558063e685c0 0x558063e689e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fed880039f0 tx=0x7fed88003eb0 comp rx=0 tx=0).stop 2024-05-10T11:40:55.135 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.133+0000 7fed9a369740 1 -- 172.21.15.100:0/3172123825 shutdown_connections 2024-05-10T11:40:55.135 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.133+0000 7fed9a369740 1 --2- 172.21.15.100:0/3172123825 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558063e6e120 0x558063e705b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:55.135 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.133+0000 7fed9a369740 1 --2- 172.21.15.100:0/3172123825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558063e68fb0 0x558063e6dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:55.136 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.133+0000 7fed9a369740 1 --2- 172.21.15.100:0/3172123825 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558063e685c0 0x558063e689e0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:55.136 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.133+0000 7fed9a369740 1 -- 172.21.15.100:0/3172123825 >> 172.21.15.100:0/3172123825 conn(0x558063e63c30 msgr2=0x558063e64030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:40:55.136 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.133+0000 7fed9a369740 1 -- 172.21.15.100:0/3172123825 shutdown_connections 2024-05-10T11:40:55.136 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.134+0000 7fed867fc640 1 -- 172.21.15.100:0/1674284108 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fed8c102dd0 con 0x7fed90105e00 2024-05-10T11:40:55.136 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.134+0000 7fed9a369740 1 -- 172.21.15.100:0/3172123825 wait complete. 2024-05-10T11:40:55.136 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.134+0000 7fed953f0640 1 --2- 172.21.15.100:0/1674284108 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fed7c0b29f0 0x7fed7c0b4eb0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fed8c106530 tx=0x7fed8c1064c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:40:55.136 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.134+0000 7fed9a369740 1 -- start start 2024-05-10T11:40:55.136 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.134+0000 7fed9a369740 1 Processor -- start 2024-05-10T11:40:55.136 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.135+0000 7fed9a369740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558063e685c0 0x558063e72810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:55.136 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.135+0000 7fed9a369740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558063e68fb0 0x558063e72d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:55.136 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.135+0000 7fed9a369740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558063e6e120 0x558063e79d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:55.136 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.135+0000 7fed95bf1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558063e685c0 0x558063e72810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:55.136 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.135+0000 7fed95bf1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558063e685c0 0x558063e72810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:52616/0 (socket says 172.21.15.100:52616) 2024-05-10T11:40:55.136 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.135+0000 7fed95bf1640 1 -- 172.21.15.100:0/2674052372 learned_addr learned my addr 172.21.15.100:0/2674052372 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:40:55.136 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.135+0000 7fed9a369740 1 -- 172.21.15.100:0/2674052372 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x558063e710d0 con 0x558063e6e120 2024-05-10T11:40:55.137 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.135+0000 7fed9a369740 1 -- 172.21.15.100:0/2674052372 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558063d81370 con 0x558063e68fb0 2024-05-10T11:40:55.137 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.135+0000 7fed9a369740 1 -- 172.21.15.100:0/2674052372 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558063e71250 con 0x558063e685c0 2024-05-10T11:40:55.137 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.135+0000 7fed94bef640 1 --2- 172.21.15.100:0/2674052372 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558063e68fb0 0x558063e72d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:55.137 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.135+0000 7fed9a369740 10 client.0 _ll_register_callbacks cb 0x558063ad5a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:40:55.137 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.135+0000 7fed95bf1640 1 --2- 172.21.15.100:0/2674052372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558063e6e120 0x558063e79d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:55.139 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.135+0000 7fed95bf1640 1 -- 172.21.15.100:0/2674052372 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558063e68fb0 msgr2=0x558063e72d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:55.139 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.135+0000 7fed95bf1640 1 --2- 172.21.15.100:0/2674052372 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558063e68fb0 0x558063e72d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:55.139 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.135+0000 7fed95bf1640 1 -- 172.21.15.100:0/2674052372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558063e6e120 msgr2=0x558063e79d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:55.139 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.135+0000 7fed95bf1640 1 --2- 172.21.15.100:0/2674052372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558063e6e120 0x558063e79d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:55.139 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.135+0000 7fed95bf1640 1 -- 172.21.15.100:0/2674052372 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558063ba2620 con 0x558063e685c0 2024-05-10T11:40:55.139 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.136+0000 7fed95bf1640 1 --2- 172.21.15.100:0/2674052372 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558063e685c0 0x558063e72810 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fed88009ab0 tx=0x7fed88009a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d82ab576573785fa server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:40:55.139 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.136+0000 7fed78ff9640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:40:55.139 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.136+0000 7fed78ff9640 1 -- 172.21.15.100:0/2674052372 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed88010920 con 0x558063e685c0 2024-05-10T11:40:55.139 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.136+0000 7fed78ff9640 1 -- 172.21.15.100:0/2674052372 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fed88010ac0 con 0x558063e685c0 2024-05-10T11:40:55.139 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.136+0000 7fed9a369740 1 -- 172.21.15.100:0/2674052372 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed88015070 con 0x558063e685c0 2024-05-10T11:40:55.139 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.136+0000 7fed78ff9640 1 -- 172.21.15.100:0/2674052372 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fed88010cc0 con 0x558063e685c0 2024-05-10T11:40:55.139 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.136+0000 7fed9a369740 10 client.29170 fetch_fsmap learned FSMap version 41 2024-05-10T11:40:55.140 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.136+0000 7fed9a369740 1 -- 172.21.15.100:0/2674052372 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x558063e7cb40 con 0x558063e685c0 2024-05-10T11:40:55.140 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.137+0000 7fed78ff9640 1 -- 172.21.15.100:0/2674052372 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fed88015070 con 0x558063e685c0 2024-05-10T11:40:55.140 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.137+0000 7fed78ff9640 1 -- 172.21.15.100:0/2674052372 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fed8800f3c0 con 0x558063e685c0 2024-05-10T11:40:55.140 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.137+0000 7fed9a369740 10 client.29170 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:40:55.140 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.137+0000 7fed9a369740 10 client.29170 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:40:55.140 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.137+0000 7fed9a369740 20 client.29170 populate_metadata read hostname 'smithi100' 2024-05-10T11:40:55.140 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.137+0000 7fed9a369740 1 --2- 172.21.15.100:0/2674052372 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558063e7df50 0x558063e80340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:55.140 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.137+0000 7fed9a369740 4 client.29170 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:40:55.140 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.137+0000 7fed9a369740 1 -- 172.21.15.100:0/2674052372 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558063dd21e0 con 0x558063e7df50 2024-05-10T11:40:55.140 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.137+0000 7fed94bef640 1 --2- 172.21.15.100:0/2674052372 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558063e7df50 0x558063e80340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:55.140 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.137+0000 7fed78ff9640 10 client.29170 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:40:55.140 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.137+0000 7fed94bef640 1 --2- 172.21.15.100:0/2674052372 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558063e7df50 0x558063e80340 crc :-1 s=READY pgs=425 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da8cbd46b18d7c31 server_cookie=2117a23ca8da803b in_seq=0 out_seq=0 2024-05-10T11:40:55.141 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.140+0000 7fed78ff9640 1 -- 172.21.15.100:0/2674052372 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558063dd21e0 con 0x558063e7df50 2024-05-10T11:40:55.141 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.141+0000 7fed78ff9640 10 client.29170 handle_command_reply: tid=0 2024-05-10T11:40:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.141+0000 7fed9a369740 1 client.29170 shutdown 2024-05-10T11:40:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.141+0000 7fed9a369740 1 -- 172.21.15.100:0/2674052372 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558063e685c0 msgr2=0x558063e72810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.141+0000 7fed9a369740 1 --2- 172.21.15.100:0/2674052372 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558063e685c0 0x558063e72810 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fed88009ab0 tx=0x7fed88009a00 comp rx=0 tx=0).stop 2024-05-10T11:40:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.141+0000 7fed9a369740 1 -- 172.21.15.100:0/2674052372 shutdown_connections 2024-05-10T11:40:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.141+0000 7fed9a369740 1 --2- 172.21.15.100:0/2674052372 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558063e7df50 0x558063e80340 crc :-1 s=READY pgs=425 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.141+0000 7fed9a369740 1 --2- 172.21.15.100:0/2674052372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558063e6e120 0x558063e79d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.141+0000 7fed9a369740 1 --2- 172.21.15.100:0/2674052372 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558063e68fb0 0x558063e72d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.141+0000 7fed78ff9640 0 client.29170 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:40:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.141+0000 7fed9a369740 1 --2- 172.21.15.100:0/2674052372 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558063e685c0 0x558063e72810 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.141+0000 7fed9a369740 1 -- 172.21.15.100:0/2674052372 >> 172.21.15.100:0/2674052372 conn(0x558063e63c30 msgr2=0x558063e5a580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:40:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.141+0000 7fed9a369740 1 -- 172.21.15.100:0/2674052372 shutdown_connections 2024-05-10T11:40:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.141+0000 7fed9a369740 1 -- 172.21.15.100:0/2674052372 wait complete. 2024-05-10T11:40:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.141+0000 7fed9a369740 20 client.29170 trim_cache size 0 max 16384 2024-05-10T11:40:55.166 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.165+0000 7fed9a369740 1 Processor -- start 2024-05-10T11:40:55.166 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.165+0000 7fed9a369740 1 -- start start 2024-05-10T11:40:55.166 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.165+0000 7fed9a369740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558063e685c0 0x558063e66b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:55.166 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.165+0000 7fed9a369740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558063e68fb0 0x558063e67080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:55.166 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.165+0000 7fed9a369740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558063e6e120 0x558063e84b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:55.166 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.165+0000 7fed9a369740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fed901abff0 con 0x558063e6e120 2024-05-10T11:40:55.166 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.165+0000 7fed9a369740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fed901a98c0 con 0x558063e68fb0 2024-05-10T11:40:55.166 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.165+0000 7fed9a369740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fed901a7710 con 0x558063e685c0 2024-05-10T11:40:55.167 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.165+0000 7fed95bf1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558063e6e120 0x558063e84b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:55.167 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.165+0000 7fed94bef640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558063e68fb0 0x558063e67080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:55.167 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.165+0000 7fed95bf1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558063e6e120 0x558063e84b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42900/0 (socket says 172.21.15.100:42900) 2024-05-10T11:40:55.167 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.165+0000 7fed94bef640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558063e68fb0 0x558063e67080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:36046/0 (socket says 172.21.15.100:36046) 2024-05-10T11:40:55.167 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.165+0000 7fed95bf1640 1 -- 172.21.15.100:0/3388344628 learned_addr learned my addr 172.21.15.100:0/3388344628 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:40:55.167 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.165+0000 7fed95bf1640 1 --2- 172.21.15.100:0/3388344628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558063e685c0 0x558063e66b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:55.167 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.165+0000 7fed94bef640 1 -- 172.21.15.100:0/3388344628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558063e685c0 msgr2=0x558063e66b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:55.167 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.165+0000 7fed94bef640 1 --2- 172.21.15.100:0/3388344628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558063e685c0 0x558063e66b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:55.167 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.165+0000 7fed94bef640 1 -- 172.21.15.100:0/3388344628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558063e6e120 msgr2=0x558063e84b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:55.167 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.165+0000 7fed94bef640 1 --2- 172.21.15.100:0/3388344628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558063e6e120 0x558063e84b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:55.167 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.165+0000 7fed94bef640 1 -- 172.21.15.100:0/3388344628 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558063e63a60 con 0x558063e68fb0 2024-05-10T11:40:55.167 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.166+0000 7fed95bf1640 1 --2- 172.21.15.100:0/3388344628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558063e6e120 0x558063e84b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:40:55.167 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.166+0000 7fed94bef640 1 --2- 172.21.15.100:0/3388344628 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558063e68fb0 0x558063e67080 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x558063e80a90 tx=0x7fed800123e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=544103821f97e699 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:40:55.167 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.166+0000 7fed7affd640 1 -- 172.21.15.100:0/3388344628 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed8001f640 con 0x558063e68fb0 2024-05-10T11:40:55.168 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.166+0000 7fed7affd640 1 -- 172.21.15.100:0/3388344628 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fed80023030 con 0x558063e68fb0 2024-05-10T11:40:55.168 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.166+0000 7fed7affd640 1 -- 172.21.15.100:0/3388344628 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed80010c10 con 0x558063e68fb0 2024-05-10T11:40:55.171 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.167+0000 7fed9a369740 1 -- 172.21.15.100:0/3388344628 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558063e68fb0 msgr2=0x558063e67080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:55.171 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.167+0000 7fed9a369740 1 --2- 172.21.15.100:0/3388344628 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558063e68fb0 0x558063e67080 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x558063e80a90 tx=0x7fed800123e0 comp rx=0 tx=0).stop 2024-05-10T11:40:55.171 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.167+0000 7fed9a369740 1 -- 172.21.15.100:0/3388344628 shutdown_connections 2024-05-10T11:40:55.171 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.167+0000 7fed9a369740 1 --2- 172.21.15.100:0/3388344628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558063e6e120 0x558063e84b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:55.171 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.167+0000 7fed9a369740 1 --2- 172.21.15.100:0/3388344628 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558063e68fb0 0x558063e67080 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:55.171 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.167+0000 7fed9a369740 1 --2- 172.21.15.100:0/3388344628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558063e685c0 0x558063e66b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:55.171 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.167+0000 7fed9a369740 1 -- 172.21.15.100:0/3388344628 >> 172.21.15.100:0/3388344628 conn(0x558063e63c30 msgr2=0x558063e6a230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:40:55.171 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.167+0000 7fed9a369740 1 -- 172.21.15.100:0/3388344628 shutdown_connections 2024-05-10T11:40:55.171 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.167+0000 7fed9a369740 1 -- 172.21.15.100:0/3388344628 wait complete. 2024-05-10T11:40:55.171 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.167+0000 7fed9a369740 1 -- start start 2024-05-10T11:40:55.171 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.167+0000 7fed9a369740 1 Processor -- start 2024-05-10T11:40:55.171 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.168+0000 7fed9a369740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558063e685c0 0x558063e77230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:55.171 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.168+0000 7fed9a369740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558063e68fb0 0x558063e77770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:55.171 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.168+0000 7fed9a369740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558063e6e120 0x558063e786c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:55.172 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.168+0000 7fed9a369740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fed901a7d80 con 0x558063e6e120 2024-05-10T11:40:55.172 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.168+0000 7fed9a369740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fed901a91f0 con 0x558063e685c0 2024-05-10T11:40:55.172 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.168+0000 7fed9a369740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fed901a8450 con 0x558063e68fb0 2024-05-10T11:40:55.172 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.168+0000 7fed9a369740 10 client.0 _ll_register_callbacks cb 0x7fed88009eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:40:55.172 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.168+0000 7fed94bef640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558063e68fb0 0x558063e77770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:55.172 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.168+0000 7fed95bf1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558063e6e120 0x558063e786c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:55.172 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.168+0000 7fed95bf1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558063e685c0 0x558063e77230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:55.172 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.168+0000 7fed94bef640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558063e68fb0 0x558063e77770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:52630/0 (socket says 172.21.15.100:52630) 2024-05-10T11:40:55.172 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.168+0000 7fed94bef640 1 -- 172.21.15.100:0/2701673691 learned_addr learned my addr 172.21.15.100:0/2701673691 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:40:55.172 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.168+0000 7fed95bf1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558063e685c0 0x558063e77230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:36062/0 (socket says 172.21.15.100:36062) 2024-05-10T11:40:55.172 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.168+0000 7fed94bef640 1 -- 172.21.15.100:0/2701673691 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558063e685c0 msgr2=0x558063e77230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:55.172 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.168+0000 7fed94bef640 1 --2- 172.21.15.100:0/2701673691 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558063e685c0 0x558063e77230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:55.172 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.168+0000 7fed94bef640 1 -- 172.21.15.100:0/2701673691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558063e6e120 msgr2=0x558063e786c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:40:55.172 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.168+0000 7fed94bef640 1 --2- 172.21.15.100:0/2701673691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558063e6e120 0x558063e786c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:55.173 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.168+0000 7fed94bef640 1 -- 172.21.15.100:0/2701673691 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558063e78cb0 con 0x558063e68fb0 2024-05-10T11:40:55.173 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.168+0000 7fed95bf1640 1 --2- 172.21.15.100:0/2701673691 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558063e685c0 0x558063e77230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:40:55.173 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.169+0000 7fed94bef640 1 --2- 172.21.15.100:0/2701673691 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558063e68fb0 0x558063e77770 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fed80014e60 tx=0x7fed80014e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c753e2ed0b623c75 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:40:55.173 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.169+0000 7fed78ff9640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:40:55.173 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.169+0000 7fed78ff9640 1 -- 172.21.15.100:0/2701673691 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed80010990 con 0x558063e68fb0 2024-05-10T11:40:55.173 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.169+0000 7fed78ff9640 1 -- 172.21.15.100:0/2701673691 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fed80010b30 con 0x558063e68fb0 2024-05-10T11:40:55.173 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.169+0000 7fed9a369740 1 -- 172.21.15.100:0/2701673691 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed80023030 con 0x558063e68fb0 2024-05-10T11:40:55.173 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.169+0000 7fed78ff9640 1 -- 172.21.15.100:0/2701673691 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fed80039050 con 0x558063e68fb0 2024-05-10T11:40:55.173 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.169+0000 7fed9a369740 10 client.29176 fetch_fsmap learned FSMap version 41 2024-05-10T11:40:55.173 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.169+0000 7fed9a369740 1 -- 172.21.15.100:0/2701673691 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x558063ee8d30 con 0x558063e68fb0 2024-05-10T11:40:55.173 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.169+0000 7fed78ff9640 1 -- 172.21.15.100:0/2701673691 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fed80023030 con 0x558063e68fb0 2024-05-10T11:40:55.173 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.169+0000 7fed78ff9640 1 -- 172.21.15.100:0/2701673691 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fed8001f600 con 0x558063e68fb0 2024-05-10T11:40:55.173 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.170+0000 7fed9a369740 10 client.29176 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:40:55.173 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.170+0000 7fed9a369740 10 client.29176 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:40:55.173 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.170+0000 7fed9a369740 20 client.29176 populate_metadata read hostname 'smithi100' 2024-05-10T11:40:55.173 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.170+0000 7fed9a369740 1 --2- 172.21.15.100:0/2701673691 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558063ee9630 0x558063eeba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:40:55.174 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.170+0000 7fed9a369740 4 client.29176 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:40:55.174 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.170+0000 7fed9a369740 1 -- 172.21.15.100:0/2701673691 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558063dcc050 con 0x558063ee9630 2024-05-10T11:40:55.174 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.170+0000 7fed95bf1640 1 --2- 172.21.15.100:0/2701673691 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558063ee9630 0x558063eeba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:40:55.174 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.170+0000 7fed78ff9640 10 client.29176 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:40:55.174 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.170+0000 7fed95bf1640 1 --2- 172.21.15.100:0/2701673691 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558063ee9630 0x558063eeba50 crc :-1 s=READY pgs=426 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90250df220db3d5a server_cookie=25b10c57fe7f4366 in_seq=0 out_seq=0 2024-05-10T11:40:55.295 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.294+0000 7fed78ff9640 1 -- 172.21.15.100:0/2701673691 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558063dcc050 con 0x558063ee9630 2024-05-10T11:40:55.295 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.294+0000 7fed78ff9640 10 client.29176 handle_command_reply: tid=0 2024-05-10T11:40:55.295 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.294+0000 7fed9a369740 1 client.29176 shutdown 2024-05-10T11:40:55.295 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.294+0000 7fed9a369740 1 -- 172.21.15.100:0/2701673691 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558063e68fb0 msgr2=0x558063e77770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:55.295 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.294+0000 7fed9a369740 1 --2- 172.21.15.100:0/2701673691 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558063e68fb0 0x558063e77770 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fed80014e60 tx=0x7fed80014e90 comp rx=0 tx=0).stop 2024-05-10T11:40:55.295 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.294+0000 7fed9a369740 1 -- 172.21.15.100:0/2701673691 shutdown_connections 2024-05-10T11:40:55.295 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.294+0000 7fed9a369740 1 --2- 172.21.15.100:0/2701673691 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x558063ee9630 0x558063eeba50 crc :-1 s=READY pgs=426 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:55.295 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.294+0000 7fed9a369740 1 --2- 172.21.15.100:0/2701673691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558063e6e120 0x558063e786c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:55.295 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.294+0000 7fed78ff9640 0 client.29176 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:40:55.296 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.294+0000 7fed9a369740 1 --2- 172.21.15.100:0/2701673691 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558063e68fb0 0x558063e77770 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:55.296 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.294+0000 7fed9a369740 1 --2- 172.21.15.100:0/2701673691 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558063e685c0 0x558063e77230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:55.296 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.294+0000 7fed9a369740 1 -- 172.21.15.100:0/2701673691 >> 172.21.15.100:0/2701673691 conn(0x558063e63c30 msgr2=0x558063e66b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:40:55.296 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.294+0000 7fed9a369740 1 -- 172.21.15.100:0/2701673691 shutdown_connections 2024-05-10T11:40:55.296 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.294+0000 7fed9a369740 1 -- 172.21.15.100:0/2701673691 wait complete. 2024-05-10T11:40:55.296 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.294+0000 7fed9a369740 20 client.29176 trim_cache size 0 max 16384 2024-05-10T11:40:55.296 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:40:55.296 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:40:55.296 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:40:55.297 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:40:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.295+0000 7fed97654640 1 -- 172.21.15.100:0/1674284108 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fed7c0b29f0 msgr2=0x7fed7c0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.295+0000 7fed97654640 1 --2- 172.21.15.100:0/1674284108 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fed7c0b29f0 0x7fed7c0b4eb0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fed8c106530 tx=0x7fed8c1064c0 comp rx=0 tx=0).stop 2024-05-10T11:40:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.295+0000 7fed97654640 1 -- 172.21.15.100:0/1674284108 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed90105e00 msgr2=0x7fed90102e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:40:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.295+0000 7fed97654640 1 --2- 172.21.15.100:0/1674284108 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed90105e00 0x7fed90102e40 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fed8c02a980 tx=0x7fed8c003cd0 comp rx=0 tx=0).stop 2024-05-10T11:40:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.295+0000 7fed97654640 1 -- 172.21.15.100:0/1674284108 shutdown_connections 2024-05-10T11:40:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.295+0000 7fed97654640 1 --2- 172.21.15.100:0/1674284108 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fed7c0b29f0 0x7fed7c0b4eb0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.295+0000 7fed97654640 1 --2- 172.21.15.100:0/1674284108 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed901019c0 0x7fed90101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.295+0000 7fed97654640 1 --2- 172.21.15.100:0/1674284108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed90108730 0x7fed90101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.295+0000 7fed97654640 1 --2- 172.21.15.100:0/1674284108 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed90105e00 0x7fed90102e40 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:40:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.295+0000 7fed97654640 1 -- 172.21.15.100:0/1674284108 >> 172.21.15.100:0/1674284108 conn(0x7fed900fce80 msgr2=0x7fed90105320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:40:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.295+0000 7fed97654640 1 -- 172.21.15.100:0/1674284108 shutdown_connections 2024-05-10T11:40:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:40:55.295+0000 7fed97654640 1 -- 172.21.15.100:0/1674284108 wait complete. 2024-05-10T11:40:55.954 INFO:tasks.workunit.client.0.smithi100.stdout: 10 88053 8.95 MB/sec execute 409 sec latency 174.246 ms 2024-05-10T11:40:56.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:40:55 smithi178 ceph-mon[33359]: pgmap v1097: 129 pgs: 129 active+clean; 563 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 684 KiB/s rd, 4.4 MiB/s wr, 295 op/s 2024-05-10T11:40:56.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:40:55 smithi188 ceph-mon[34075]: pgmap v1097: 129 pgs: 129 active+clean; 563 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 684 KiB/s rd, 4.4 MiB/s wr, 295 op/s 2024-05-10T11:40:56.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:40:55 smithi100 ceph-mon[30296]: pgmap v1097: 129 pgs: 129 active+clean; 563 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 684 KiB/s rd, 4.4 MiB/s wr, 295 op/s 2024-05-10T11:40:56.954 INFO:tasks.workunit.client.0.smithi100.stdout: 10 88166 8.93 MB/sec execute 410 sec latency 195.234 ms 2024-05-10T11:40:57.954 INFO:tasks.workunit.client.0.smithi100.stdout: 10 88278 8.92 MB/sec execute 411 sec latency 147.716 ms 2024-05-10T11:40:58.080 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:40:57 smithi188 ceph-mon[34075]: pgmap v1098: 129 pgs: 129 active+clean; 557 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 4.5 MiB/s wr, 321 op/s 2024-05-10T11:40:58.080 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:40:57 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:40:58.081 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:40:57 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:40:58.081 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:40:57 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:40:58.143 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:40:57 smithi100 ceph-mon[30296]: pgmap v1098: 129 pgs: 129 active+clean; 557 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 4.5 MiB/s wr, 321 op/s 2024-05-10T11:40:58.144 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:40:57 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:40:58.144 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:40:57 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:40:58.144 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:40:57 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:40:58.177 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T11:40:58.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:40:57 smithi178 ceph-mon[33359]: pgmap v1098: 129 pgs: 129 active+clean; 557 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 4.5 MiB/s wr, 321 op/s 2024-05-10T11:40:58.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:40:57 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:40:58.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:40:57 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:40:58.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:40:57 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:40:58.539 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:40:58.954 INFO:tasks.workunit.client.0.smithi100.stdout: 10 88441 8.91 MB/sec execute 412 sec latency 82.282 ms 2024-05-10T11:40:59.954 INFO:tasks.workunit.client.0.smithi100.stdout: 10 88553 8.89 MB/sec execute 413 sec latency 164.807 ms 2024-05-10T11:41:00.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:40:59 smithi100 ceph-mon[30296]: pgmap v1099: 129 pgs: 129 active+clean; 553 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 3.8 MiB/s wr, 295 op/s 2024-05-10T11:41:00.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:40:59 smithi178 ceph-mon[33359]: pgmap v1099: 129 pgs: 129 active+clean; 553 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 3.8 MiB/s wr, 295 op/s 2024-05-10T11:41:00.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:40:59 smithi188 ceph-mon[34075]: pgmap v1099: 129 pgs: 129 active+clean; 553 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 3.8 MiB/s wr, 295 op/s 2024-05-10T11:41:00.954 INFO:tasks.workunit.client.0.smithi100.stdout: 10 88699 8.89 MB/sec execute 414 sec latency 143.220 ms 2024-05-10T11:41:01.954 INFO:tasks.workunit.client.0.smithi100.stdout: 10 88863 8.89 MB/sec execute 415 sec latency 73.104 ms 2024-05-10T11:41:02.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:41:01 smithi178 ceph-mon[33359]: pgmap v1100: 129 pgs: 129 active+clean; 552 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.8 MiB/s wr, 344 op/s 2024-05-10T11:41:02.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:41:01 smithi188 ceph-mon[34075]: pgmap v1100: 129 pgs: 129 active+clean; 552 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.8 MiB/s wr, 344 op/s 2024-05-10T11:41:02.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:41:01 smithi100 ceph-mon[30296]: pgmap v1100: 129 pgs: 129 active+clean; 552 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.8 MiB/s wr, 344 op/s 2024-05-10T11:41:02.954 INFO:tasks.workunit.client.0.smithi100.stdout: 10 89061 8.91 MB/sec execute 416 sec latency 70.020 ms 2024-05-10T11:41:03.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.262+0000 7f5bbd330700 1 -- 172.21.15.100:0/1035679635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5bb009ad20 msgr2=0x7f5bb00047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:03.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.262+0000 7f5bbd330700 1 --2- 172.21.15.100:0/1035679635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5bb009ad20 0x7f5bb00047d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5bac009990 tx=0x7f5bac009ca0 comp rx=0 tx=0).stop 2024-05-10T11:41:03.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.264+0000 7f5bbd330700 1 -- 172.21.15.100:0/1035679635 shutdown_connections 2024-05-10T11:41:03.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.264+0000 7f5bbd330700 1 --2- 172.21.15.100:0/1035679635 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5bb0098720 0x7f5bb0098ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:03.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.264+0000 7f5bbd330700 1 --2- 172.21.15.100:0/1035679635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5bb009ad20 0x7f5bb00047d0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:03.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.264+0000 7f5bbd330700 1 --2- 172.21.15.100:0/1035679635 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5bb0004d10 0x7f5bb0005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:03.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.264+0000 7f5bbd330700 1 -- 172.21.15.100:0/1035679635 >> 172.21.15.100:0/1035679635 conn(0x7f5bb000a7a0 msgr2=0x7f5bb000abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:41:03.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.267+0000 7f5bbd330700 1 -- 172.21.15.100:0/1035679635 shutdown_connections 2024-05-10T11:41:03.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.268+0000 7f5bbd330700 1 -- 172.21.15.100:0/1035679635 wait complete. 2024-05-10T11:41:03.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.268+0000 7f5bbd330700 1 Processor -- start 2024-05-10T11:41:03.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.269+0000 7f5bbd330700 1 -- start start 2024-05-10T11:41:03.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.269+0000 7f5bbd330700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5bb0004d10 0x7f5bb0138570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:03.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.269+0000 7f5bbd330700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5bb0098720 0x7f5bb0138ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:03.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.269+0000 7f5bbd330700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5bb009ad20 0x7f5bb0132640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:03.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.269+0000 7f5bbd330700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb00ac2a0 con 0x7f5bb009ad20 2024-05-10T11:41:03.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.269+0000 7f5bbd330700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb00ac120 con 0x7f5bb0098720 2024-05-10T11:41:03.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.269+0000 7f5bbd330700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb00ac420 con 0x7f5bb0004d10 2024-05-10T11:41:03.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.270+0000 7f5bb7fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5bb0004d10 0x7f5bb0138570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:03.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.270+0000 7f5bb7fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5bb0004d10 0x7f5bb0138570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:60244/0 (socket says 172.21.15.100:60244) 2024-05-10T11:41:03.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.270+0000 7f5bb7fff700 1 -- 172.21.15.100:0/1683195042 learned_addr learned my addr 172.21.15.100:0/1683195042 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:41:03.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.270+0000 7f5bb7fff700 1 -- 172.21.15.100:0/1683195042 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5bb0098720 msgr2=0x7f5bb0138ab0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T11:41:03.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.270+0000 7f5bb7fff700 1 --2- 172.21.15.100:0/1683195042 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5bb0098720 0x7f5bb0138ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:03.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.270+0000 7f5bb7fff700 1 -- 172.21.15.100:0/1683195042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5bb009ad20 msgr2=0x7f5bb0132640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:03.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.270+0000 7f5bb7fff700 1 --2- 172.21.15.100:0/1683195042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5bb009ad20 0x7f5bb0132640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:03.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.270+0000 7f5bb7fff700 1 -- 172.21.15.100:0/1683195042 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bb0132e70 con 0x7f5bb0004d10 2024-05-10T11:41:03.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.273+0000 7f5bb7fff700 1 --2- 172.21.15.100:0/1683195042 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5bb0004d10 0x7f5bb0138570 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f5bac0057f0 tx=0x7f5bac009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:41:03.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.273+0000 7f5bb57fa700 1 -- 172.21.15.100:0/1683195042 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bac027070 con 0x7f5bb0004d10 2024-05-10T11:41:03.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.273+0000 7f5bbd330700 1 -- 172.21.15.100:0/1683195042 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bb0133160 con 0x7f5bb0004d10 2024-05-10T11:41:03.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.273+0000 7f5bbd330700 1 -- 172.21.15.100:0/1683195042 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bb009b6d0 con 0x7f5bb0004d10 2024-05-10T11:41:03.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.273+0000 7f5bb57fa700 1 -- 172.21.15.100:0/1683195042 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5bac009510 con 0x7f5bb0004d10 2024-05-10T11:41:03.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.274+0000 7f5bb57fa700 1 -- 172.21.15.100:0/1683195042 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bac027070 con 0x7f5bb0004d10 2024-05-10T11:41:03.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.278+0000 7f5bb57fa700 1 -- 172.21.15.100:0/1683195042 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f5bac005080 con 0x7f5bb0004d10 2024-05-10T11:41:03.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.279+0000 7f5bb57fa700 1 --2- 172.21.15.100:0/1683195042 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f5ba80b2a80 0x7f5ba80b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:03.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.280+0000 7f5bb57fa700 1 -- 172.21.15.100:0/1683195042 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f5bac0a6450 con 0x7f5bb0004d10 2024-05-10T11:41:03.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.280+0000 7f5bb77fe700 1 --2- 172.21.15.100:0/1683195042 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f5ba80b2a80 0x7f5ba80b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:03.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.281+0000 7f5bc8e25b80 1 Processor -- start 2024-05-10T11:41:03.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.281+0000 7f5bc8e25b80 1 -- start start 2024-05-10T11:41:03.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.281+0000 7f5bb77fe700 1 --2- 172.21.15.100:0/1683195042 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f5ba80b2a80 0x7f5ba80b4f40 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4005fd0 tx=0x7f5ba4005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:41:03.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.282+0000 7f5bc8e25b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7c16e1880 0x55b7c16e1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:03.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.282+0000 7f5bc8e25b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b7c16e21a0 0x55b7c16e5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:03.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.282+0000 7f5bc8e25b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7c16e5dd0 0x55b7c16e6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:03.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.282+0000 7f5bc8e25b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55b7c1633860 con 0x55b7c16e5dd0 2024-05-10T11:41:03.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.282+0000 7f5bc8e25b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b7c15d27c0 con 0x55b7c16e1880 2024-05-10T11:41:03.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.282+0000 7f5bb7fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7c16e1880 0x55b7c16e1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:03.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.282+0000 7f5bc8e25b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b7c162f820 con 0x55b7c16e21a0 2024-05-10T11:41:03.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.282+0000 7f5bb7fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7c16e1880 0x55b7c16e1c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:42520/0 (socket says 172.21.15.100:42520) 2024-05-10T11:41:03.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.282+0000 7f5bb7fff700 1 -- 172.21.15.100:0/3589503191 learned_addr learned my addr 172.21.15.100:0/3589503191 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:41:03.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.282+0000 7f5bb77fe700 1 --2- 172.21.15.100:0/3589503191 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b7c16e21a0 0x55b7c16e5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:03.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.282+0000 7f5bbcb2f700 1 --2- 172.21.15.100:0/3589503191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7c16e5dd0 0x55b7c16e6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:03.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.283+0000 7f5bb77fe700 1 -- 172.21.15.100:0/3589503191 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7c16e1880 msgr2=0x55b7c16e1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:03.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.283+0000 7f5bb77fe700 1 --2- 172.21.15.100:0/3589503191 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7c16e1880 0x55b7c16e1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:03.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.283+0000 7f5bb77fe700 1 -- 172.21.15.100:0/3589503191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7c16e5dd0 msgr2=0x55b7c16e6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:03.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.283+0000 7f5bb77fe700 1 --2- 172.21.15.100:0/3589503191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7c16e5dd0 0x55b7c16e6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:03.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.283+0000 7f5bb77fe700 1 -- 172.21.15.100:0/3589503191 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7c15808d0 con 0x55b7c16e21a0 2024-05-10T11:41:03.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.283+0000 7f5bbcb2f700 1 --2- 172.21.15.100:0/3589503191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7c16e5dd0 0x55b7c16e6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:41:03.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.286+0000 7f5bb77fe700 1 --2- 172.21.15.100:0/3589503191 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b7c16e21a0 0x55b7c16e5890 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5ba40098a0 tx=0x7f5ba4010000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6396b032e5ba747e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:41:03.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.287+0000 7f5ba27fc700 1 -- 172.21.15.100:0/3589503191 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba40108c0 con 0x55b7c16e21a0 2024-05-10T11:41:03.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.287+0000 7f5ba27fc700 1 -- 172.21.15.100:0/3589503191 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5ba4022070 con 0x55b7c16e21a0 2024-05-10T11:41:03.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.288+0000 7f5bc8e25b80 1 -- 172.21.15.100:0/3589503191 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b7c16e21a0 msgr2=0x55b7c16e5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:03.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.288+0000 7f5bc8e25b80 1 --2- 172.21.15.100:0/3589503191 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b7c16e21a0 0x55b7c16e5890 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5ba40098a0 tx=0x7f5ba4010000 comp rx=0 tx=0).stop 2024-05-10T11:41:03.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.288+0000 7f5bc8e25b80 1 -- 172.21.15.100:0/3589503191 shutdown_connections 2024-05-10T11:41:03.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.288+0000 7f5bc8e25b80 1 --2- 172.21.15.100:0/3589503191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7c16e5dd0 0x55b7c16e6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:03.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.288+0000 7f5bc8e25b80 1 --2- 172.21.15.100:0/3589503191 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7c16e1880 0x55b7c16e1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:03.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.288+0000 7f5bc8e25b80 1 --2- 172.21.15.100:0/3589503191 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b7c16e21a0 0x55b7c16e5890 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:03.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.288+0000 7f5bc8e25b80 1 -- 172.21.15.100:0/3589503191 >> 172.21.15.100:0/3589503191 conn(0x55b7c160d100 msgr2=0x55b7c16dbe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:41:03.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.291+0000 7f5bc8e25b80 1 -- 172.21.15.100:0/3589503191 shutdown_connections 2024-05-10T11:41:03.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.291+0000 7f5bc8e25b80 1 -- 172.21.15.100:0/3589503191 wait complete. 2024-05-10T11:41:03.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.292+0000 7f5bc8e25b80 1 -- start start 2024-05-10T11:41:03.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.292+0000 7f5bc8e25b80 1 Processor -- start 2024-05-10T11:41:03.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.293+0000 7f5bc8e25b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7c16e1880 0x55b7c16ed200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:03.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.293+0000 7f5bc8e25b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b7c16e21a0 0x55b7c16ea5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:03.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.293+0000 7f5bc8e25b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7c16e5dd0 0x55b7c16eaae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:03.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.293+0000 7f5bc8e25b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55b7c16edff0 con 0x55b7c16e5dd0 2024-05-10T11:41:03.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.293+0000 7f5bc8e25b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b7c1599c30 con 0x55b7c16e1880 2024-05-10T11:41:03.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.293+0000 7f5bc8e25b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b7c16ee170 con 0x55b7c16e21a0 2024-05-10T11:41:03.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.293+0000 7f5bc8e25b80 10 client.0 _ll_register_callbacks cb 0x55b7c15cb920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:41:03.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.293+0000 7f5bb7fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7c16e1880 0x55b7c16ed200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:03.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.293+0000 7f5bbcb2f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7c16e5dd0 0x55b7c16eaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:03.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.293+0000 7f5bb7fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7c16e1880 0x55b7c16ed200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:42530/0 (socket says 172.21.15.100:42530) 2024-05-10T11:41:03.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.293+0000 7f5bb7fff700 1 -- 172.21.15.100:0/2289152588 learned_addr learned my addr 172.21.15.100:0/2289152588 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:41:03.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.294+0000 7f5bbcb2f700 1 -- 172.21.15.100:0/2289152588 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b7c16e21a0 msgr2=0x55b7c16ea5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:03.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.294+0000 7f5bbcb2f700 1 --2- 172.21.15.100:0/2289152588 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b7c16e21a0 0x55b7c16ea5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:03.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.294+0000 7f5bbcb2f700 1 -- 172.21.15.100:0/2289152588 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7c16e1880 msgr2=0x55b7c16ed200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:03.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.294+0000 7f5bbcb2f700 1 --2- 172.21.15.100:0/2289152588 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7c16e1880 0x55b7c16ed200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:03.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.294+0000 7f5bbcb2f700 1 -- 172.21.15.100:0/2289152588 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7c15cf690 con 0x55b7c16e5dd0 2024-05-10T11:41:03.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.294+0000 7f5bbcb2f700 1 --2- 172.21.15.100:0/2289152588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7c16e5dd0 0x55b7c16eaae0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8069f30 tx=0x7f5bb80759b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9531dc3c241ec1ec server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:41:03.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.294+0000 7f5b977fe700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:41:03.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.295+0000 7f5b977fe700 1 -- 172.21.15.100:0/2289152588 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb8086060 con 0x55b7c16e5dd0 2024-05-10T11:41:03.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.295+0000 7f5b977fe700 1 -- 172.21.15.100:0/2289152588 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f5bb8081670 con 0x55b7c16e5dd0 2024-05-10T11:41:03.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.296+0000 7f5bc8e25b80 1 -- 172.21.15.100:0/2289152588 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b7c16ad650 con 0x55b7c16e5dd0 2024-05-10T11:41:03.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.296+0000 7f5b977fe700 1 -- 172.21.15.100:0/2289152588 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f5bb806a6b0 con 0x55b7c16e5dd0 2024-05-10T11:41:03.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.296+0000 7f5bc8e25b80 10 client.19815 fetch_fsmap learned FSMap version 41 2024-05-10T11:41:03.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.296+0000 7f5bc8e25b80 1 -- 172.21.15.100:0/2289152588 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55b7c16f8930 con 0x55b7c16e5dd0 2024-05-10T11:41:03.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.297+0000 7f5b977fe700 1 -- 172.21.15.100:0/2289152588 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55b7c16ad650 con 0x55b7c16e5dd0 2024-05-10T11:41:03.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.297+0000 7f5b977fe700 1 -- 172.21.15.100:0/2289152588 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f5bb807c070 con 0x55b7c16e5dd0 2024-05-10T11:41:03.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.298+0000 7f5bc8e25b80 10 client.19815 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:41:03.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.298+0000 7f5bc8e25b80 10 client.19815 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:41:03.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.298+0000 7f5bc8e25b80 20 client.19815 populate_metadata read hostname 'smithi100' 2024-05-10T11:41:03.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.298+0000 7f5bc8e25b80 1 --2- 172.21.15.100:0/2289152588 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55b7c16fb320 0x55b7c16fd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:03.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.298+0000 7f5bc8e25b80 4 client.19815 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:41:03.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.298+0000 7f5bc8e25b80 1 -- 172.21.15.100:0/2289152588 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b7c1641240 con 0x55b7c16fb320 2024-05-10T11:41:03.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.299+0000 7f5bb7fff700 1 --2- 172.21.15.100:0/2289152588 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55b7c16fb320 0x55b7c16fd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:03.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.300+0000 7f5bb7fff700 1 --2- 172.21.15.100:0/2289152588 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55b7c16fb320 0x55b7c16fd760 crc :-1 s=READY pgs=427 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19f82895fe80d9a6 server_cookie=c8d9968ae6f203b1 in_seq=0 out_seq=0 2024-05-10T11:41:03.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.300+0000 7f5b977fe700 10 client.19815 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:41:03.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.301+0000 7f5b977fe700 1 -- 172.21.15.100:0/2289152588 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b7c1641240 con 0x55b7c16fb320 2024-05-10T11:41:03.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.301+0000 7f5b977fe700 10 client.19815 handle_command_reply: tid=0 2024-05-10T11:41:03.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.301+0000 7f5bc8e25b80 1 client.19815 shutdown 2024-05-10T11:41:03.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.301+0000 7f5bc8e25b80 1 -- 172.21.15.100:0/2289152588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7c16e5dd0 msgr2=0x55b7c16eaae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:03.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.301+0000 7f5bc8e25b80 1 --2- 172.21.15.100:0/2289152588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7c16e5dd0 0x55b7c16eaae0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8069f30 tx=0x7f5bb80759b0 comp rx=0 tx=0).stop 2024-05-10T11:41:03.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.301+0000 7f5bc8e25b80 1 -- 172.21.15.100:0/2289152588 shutdown_connections 2024-05-10T11:41:03.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.301+0000 7f5bc8e25b80 1 --2- 172.21.15.100:0/2289152588 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55b7c16fb320 0x55b7c16fd760 crc :-1 s=READY pgs=427 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:03.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.302+0000 7f5bc8e25b80 1 --2- 172.21.15.100:0/2289152588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7c16e5dd0 0x55b7c16eaae0 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:03.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.302+0000 7f5bc8e25b80 1 --2- 172.21.15.100:0/2289152588 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7c16e1880 0x55b7c16ed200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:03.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.302+0000 7f5bc8e25b80 1 --2- 172.21.15.100:0/2289152588 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b7c16e21a0 0x55b7c16ea5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:03.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.302+0000 7f5bc8e25b80 1 -- 172.21.15.100:0/2289152588 >> 172.21.15.100:0/2289152588 conn(0x55b7c160d100 msgr2=0x55b7c16e7230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:41:03.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.302+0000 7f5b977fe700 0 client.19815 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:41:03.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.302+0000 7f5bc8e25b80 1 -- 172.21.15.100:0/2289152588 shutdown_connections 2024-05-10T11:41:03.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.302+0000 7f5bc8e25b80 1 -- 172.21.15.100:0/2289152588 wait complete. 2024-05-10T11:41:03.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.302+0000 7f5bc8e25b80 20 client.19815 trim_cache size 0 max 16384 2024-05-10T11:41:03.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.331+0000 7f5bc8e25b80 1 Processor -- start 2024-05-10T11:41:03.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.332+0000 7f5bc8e25b80 1 -- start start 2024-05-10T11:41:03.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.332+0000 7f5bc8e25b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7c16e1880 0x55b7c16f7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:03.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.332+0000 7f5bc8e25b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7c16e21a0 0x55b7c16e9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:03.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.332+0000 7f5bc8e25b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b7c16e5dd0 0x55b7c16db3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:03.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.332+0000 7f5bc8e25b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c0113f0 con 0x55b7c16e21a0 2024-05-10T11:41:03.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.332+0000 7f5bb7fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7c16e1880 0x55b7c16f7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:03.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.332+0000 7f5bb7fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7c16e1880 0x55b7c16f7160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:42536/0 (socket says 172.21.15.100:42536) 2024-05-10T11:41:03.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.332+0000 7f5bb7fff700 1 -- 172.21.15.100:0/1116899597 learned_addr learned my addr 172.21.15.100:0/1116899597 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:41:03.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.332+0000 7f5bb77fe700 1 --2- 172.21.15.100:0/1116899597 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7c16e21a0 0x55b7c16e9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:03.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.332+0000 7f5bc8e25b80 1 -- 172.21.15.100:0/1116899597 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c00ec80 con 0x55b7c16e1880 2024-05-10T11:41:03.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.332+0000 7f5bc8e25b80 1 -- 172.21.15.100:0/1116899597 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c00cad0 con 0x55b7c16e5dd0 2024-05-10T11:41:03.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.332+0000 7f5bbcb2f700 1 --2- 172.21.15.100:0/1116899597 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b7c16e5dd0 0x55b7c16db3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:03.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.333+0000 7f5bbcb2f700 1 -- 172.21.15.100:0/1116899597 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7c16e1880 msgr2=0x55b7c16f7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:03.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.333+0000 7f5bbcb2f700 1 --2- 172.21.15.100:0/1116899597 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7c16e1880 0x55b7c16f7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:03.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.333+0000 7f5bbcb2f700 1 -- 172.21.15.100:0/1116899597 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7c16e21a0 msgr2=0x55b7c16e9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:03.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.333+0000 7f5bbcb2f700 1 --2- 172.21.15.100:0/1116899597 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7c16e21a0 0x55b7c16e9520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:03.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.333+0000 7f5bbcb2f700 1 -- 172.21.15.100:0/1116899597 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7c16e8b30 con 0x55b7c16e5dd0 2024-05-10T11:41:03.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.333+0000 7f5bb7fff700 1 --2- 172.21.15.100:0/1116899597 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7c16e1880 0x55b7c16f7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:41:03.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.333+0000 7f5bbcb2f700 1 --2- 172.21.15.100:0/1116899597 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b7c16e5dd0 0x55b7c16db3b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f5bb80524a0 tx=0x7f5bb8052260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=874cb951df049c62 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:41:03.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.333+0000 7f5ba1ffb700 1 -- 172.21.15.100:0/1116899597 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb8080c60 con 0x55b7c16e5dd0 2024-05-10T11:41:03.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.333+0000 7f5ba1ffb700 1 -- 172.21.15.100:0/1116899597 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5bb8081470 con 0x55b7c16e5dd0 2024-05-10T11:41:03.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.333+0000 7f5ba1ffb700 1 -- 172.21.15.100:0/1116899597 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb8092660 con 0x55b7c16e5dd0 2024-05-10T11:41:03.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.334+0000 7f5bc8e25b80 1 -- 172.21.15.100:0/1116899597 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b7c16e5dd0 msgr2=0x55b7c16db3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:03.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.334+0000 7f5bc8e25b80 1 --2- 172.21.15.100:0/1116899597 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b7c16e5dd0 0x55b7c16db3b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f5bb80524a0 tx=0x7f5bb8052260 comp rx=0 tx=0).stop 2024-05-10T11:41:03.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.334+0000 7f5bc8e25b80 1 -- 172.21.15.100:0/1116899597 shutdown_connections 2024-05-10T11:41:03.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.334+0000 7f5bc8e25b80 1 --2- 172.21.15.100:0/1116899597 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b7c16e5dd0 0x55b7c16db3b0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:03.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.334+0000 7f5bc8e25b80 1 --2- 172.21.15.100:0/1116899597 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7c16e1880 0x55b7c16f7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:03.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.334+0000 7f5bc8e25b80 1 --2- 172.21.15.100:0/1116899597 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7c16e21a0 0x55b7c16e9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:03.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.334+0000 7f5bc8e25b80 1 -- 172.21.15.100:0/1116899597 >> 172.21.15.100:0/1116899597 conn(0x55b7c160d100 msgr2=0x55b7c16daca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:41:03.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.334+0000 7f5bc8e25b80 1 -- 172.21.15.100:0/1116899597 shutdown_connections 2024-05-10T11:41:03.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.334+0000 7f5bc8e25b80 1 -- 172.21.15.100:0/1116899597 wait complete. 2024-05-10T11:41:03.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.335+0000 7f5bc8e25b80 1 -- start start 2024-05-10T11:41:03.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.335+0000 7f5bc8e25b80 1 Processor -- start 2024-05-10T11:41:03.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.336+0000 7f5bc8e25b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7c16e1880 0x55b7c16f3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:03.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.336+0000 7f5bc8e25b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b7c16e21a0 0x55b7c16e37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:03.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.336+0000 7f5bc8e25b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7c16e5dd0 0x55b7c16e3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:03.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.336+0000 7f5bc8e25b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c00d140 con 0x55b7c16e1880 2024-05-10T11:41:03.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.336+0000 7f5bc8e25b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c00e5b0 con 0x55b7c16e5dd0 2024-05-10T11:41:03.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.336+0000 7f5bc8e25b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c00d810 con 0x55b7c16e21a0 2024-05-10T11:41:03.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.336+0000 7f5bbcb2f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7c16e5dd0 0x55b7c16e3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:03.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.336+0000 7f5bb77fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b7c16e21a0 0x55b7c16e37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:03.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.336+0000 7f5bbcb2f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7c16e5dd0 0x55b7c16e3d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:42542/0 (socket says 172.21.15.100:42542) 2024-05-10T11:41:03.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.336+0000 7f5bb7fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7c16e1880 0x55b7c16f3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:03.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.336+0000 7f5bb77fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b7c16e21a0 0x55b7c16e37e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:60280/0 (socket says 172.21.15.100:60280) 2024-05-10T11:41:03.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.336+0000 7f5bbcb2f700 1 -- 172.21.15.100:0/3256890748 learned_addr learned my addr 172.21.15.100:0/3256890748 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:41:03.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.336+0000 7f5bc8e25b80 10 client.0 _ll_register_callbacks cb 0x55b7c15cb920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:41:03.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.336+0000 7f5bb77fe700 1 -- 172.21.15.100:0/3256890748 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7c16e5dd0 msgr2=0x55b7c16e3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:03.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.336+0000 7f5bb77fe700 1 --2- 172.21.15.100:0/3256890748 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7c16e5dd0 0x55b7c16e3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:03.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.336+0000 7f5bb77fe700 1 -- 172.21.15.100:0/3256890748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7c16e1880 msgr2=0x55b7c16f3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:03.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.336+0000 7f5bb77fe700 1 --2- 172.21.15.100:0/3256890748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7c16e1880 0x55b7c16f3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:03.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.336+0000 7f5bb77fe700 1 -- 172.21.15.100:0/3256890748 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7c1702620 con 0x55b7c16e21a0 2024-05-10T11:41:03.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.336+0000 7f5bb77fe700 1 --2- 172.21.15.100:0/3256890748 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b7c16e21a0 0x55b7c16e37e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5ba40093b0 tx=0x7f5ba4010970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebf4c1aa73ddff0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:41:03.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.336+0000 7f5b977fe700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:41:03.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.337+0000 7f5b977fe700 1 -- 172.21.15.100:0/3256890748 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba4010ce0 con 0x55b7c16e21a0 2024-05-10T11:41:03.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.337+0000 7f5b977fe700 1 -- 172.21.15.100:0/3256890748 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f5ba401d440 con 0x55b7c16e21a0 2024-05-10T11:41:03.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.337+0000 7f5bc8e25b80 1 -- 172.21.15.100:0/3256890748 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5bb8081470 con 0x55b7c16e21a0 2024-05-10T11:41:03.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.337+0000 7f5b977fe700 1 -- 172.21.15.100:0/3256890748 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f5ba401d680 con 0x55b7c16e21a0 2024-05-10T11:41:03.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.337+0000 7f5bc8e25b80 10 client.29191 fetch_fsmap learned FSMap version 41 2024-05-10T11:41:03.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.337+0000 7f5bc8e25b80 1 -- 172.21.15.100:0/3256890748 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55b7c1765ce0 con 0x55b7c16e21a0 2024-05-10T11:41:03.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.337+0000 7f5b977fe700 1 -- 172.21.15.100:0/3256890748 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5bb8081470 con 0x55b7c16e21a0 2024-05-10T11:41:03.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.338+0000 7f5b977fe700 1 -- 172.21.15.100:0/3256890748 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f5ba40046f0 con 0x55b7c16e21a0 2024-05-10T11:41:03.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.338+0000 7f5bc8e25b80 10 client.29191 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:41:03.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.338+0000 7f5bc8e25b80 10 client.29191 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:41:03.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.338+0000 7f5bc8e25b80 20 client.29191 populate_metadata read hostname 'smithi100' 2024-05-10T11:41:03.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.338+0000 7f5bc8e25b80 1 --2- 172.21.15.100:0/3256890748 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55b7c1766230 0x55b7c1768650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:03.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.338+0000 7f5bc8e25b80 4 client.29191 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T11:41:03.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.338+0000 7f5bc8e25b80 1 -- 172.21.15.100:0/3256890748 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b7c163b2e0 con 0x55b7c1766230 2024-05-10T11:41:03.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.338+0000 7f5bb7fff700 1 --2- 172.21.15.100:0/3256890748 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55b7c1766230 0x55b7c1768650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:03.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.338+0000 7f5b977fe700 10 client.29191 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:41:03.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.338+0000 7f5bb7fff700 1 --2- 172.21.15.100:0/3256890748 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55b7c1766230 0x55b7c1768650 crc :-1 s=READY pgs=428 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2e5d2d67155cc43 server_cookie=6a4cc748042d2818 in_seq=0 out_seq=0 2024-05-10T11:41:03.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.339+0000 7f5b977fe700 1 -- 172.21.15.100:0/3256890748 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55b7c163b2e0 con 0x55b7c1766230 2024-05-10T11:41:03.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.339+0000 7f5b977fe700 10 client.29191 handle_command_reply: tid=0 2024-05-10T11:41:03.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.339+0000 7f5bc8e25b80 1 client.29191 shutdown 2024-05-10T11:41:03.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.339+0000 7f5bc8e25b80 1 -- 172.21.15.100:0/3256890748 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b7c16e21a0 msgr2=0x55b7c16e37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:03.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.339+0000 7f5bc8e25b80 1 --2- 172.21.15.100:0/3256890748 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b7c16e21a0 0x55b7c16e37e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5ba40093b0 tx=0x7f5ba4010970 comp rx=0 tx=0).stop 2024-05-10T11:41:03.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.339+0000 7f5bc8e25b80 1 -- 172.21.15.100:0/3256890748 shutdown_connections 2024-05-10T11:41:03.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.339+0000 7f5bc8e25b80 1 --2- 172.21.15.100:0/3256890748 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55b7c1766230 0x55b7c1768650 crc :-1 s=READY pgs=428 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:03.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.339+0000 7f5bc8e25b80 1 --2- 172.21.15.100:0/3256890748 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7c16e5dd0 0x55b7c16e3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:03.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.339+0000 7f5bc8e25b80 1 --2- 172.21.15.100:0/3256890748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7c16e1880 0x55b7c16f3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:03.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.339+0000 7f5bc8e25b80 1 --2- 172.21.15.100:0/3256890748 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b7c16e21a0 0x55b7c16e37e0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:03.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.339+0000 7f5bc8e25b80 1 -- 172.21.15.100:0/3256890748 >> 172.21.15.100:0/3256890748 conn(0x55b7c160d100 msgr2=0x55b7c16e0690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:41:03.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.339+0000 7f5b977fe700 0 client.29191 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:41:03.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.339+0000 7f5bc8e25b80 1 -- 172.21.15.100:0/3256890748 shutdown_connections 2024-05-10T11:41:03.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.339+0000 7f5bc8e25b80 1 -- 172.21.15.100:0/3256890748 wait complete. 2024-05-10T11:41:03.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.339+0000 7f5bc8e25b80 20 client.29191 trim_cache size 0 max 16384 2024-05-10T11:41:03.342 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:41:03.343 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T11:41:03.343 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-05-10T11:41:03.343 INFO:teuthology.orchestra.run.smithi100.stdout: "3c8fea61-b040-48ee-91e1-f5e5c8d221f6": { 2024-05-10T11:41:03.343 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-05-10T11:41:03.343 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "3c8fea61-b040-48ee-91e1-f5e5c8d221f6", 2024-05-10T11:41:03.343 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-05-10T11:41:03.343 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:41:03.343 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:41:03.343 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:41:03.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.340+0000 7f5bbd330700 1 -- 172.21.15.100:0/1683195042 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f5ba80b2a80 msgr2=0x7f5ba80b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:03.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.340+0000 7f5bbd330700 1 --2- 172.21.15.100:0/1683195042 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f5ba80b2a80 0x7f5ba80b4f40 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4005fd0 tx=0x7f5ba4005f60 comp rx=0 tx=0).stop 2024-05-10T11:41:03.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.340+0000 7f5bbd330700 1 -- 172.21.15.100:0/1683195042 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5bb0004d10 msgr2=0x7f5bb0138570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:03.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.340+0000 7f5bbd330700 1 --2- 172.21.15.100:0/1683195042 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5bb0004d10 0x7f5bb0138570 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f5bac0057f0 tx=0x7f5bac009f90 comp rx=0 tx=0).stop 2024-05-10T11:41:03.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.340+0000 7f5bbd330700 1 -- 172.21.15.100:0/1683195042 shutdown_connections 2024-05-10T11:41:03.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.340+0000 7f5bbd330700 1 --2- 172.21.15.100:0/1683195042 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f5ba80b2a80 0x7f5ba80b4f40 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:03.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.340+0000 7f5bbd330700 1 --2- 172.21.15.100:0/1683195042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5bb009ad20 0x7f5bb0132640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:03.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.340+0000 7f5bbd330700 1 --2- 172.21.15.100:0/1683195042 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5bb0004d10 0x7f5bb0138570 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:03.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.340+0000 7f5bbd330700 1 --2- 172.21.15.100:0/1683195042 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5bb0098720 0x7f5bb0138ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:03.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.340+0000 7f5bbd330700 1 -- 172.21.15.100:0/1683195042 >> 172.21.15.100:0/1683195042 conn(0x7f5bb000a7a0 msgr2=0x7f5bb0095c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:41:03.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.341+0000 7f5bbd330700 1 -- 172.21.15.100:0/1683195042 shutdown_connections 2024-05-10T11:41:03.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:03.341+0000 7f5bbd330700 1 -- 172.21.15.100:0/1683195042 wait complete. 2024-05-10T11:41:03.954 INFO:tasks.workunit.client.0.smithi100.stdout: 10 89250 8.92 MB/sec execute 417 sec latency 96.159 ms 2024-05-10T11:41:04.079 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:41:03 smithi100 ceph-mon[30296]: pgmap v1101: 129 pgs: 129 active+clean; 547 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 2.9 MiB/s wr, 273 op/s 2024-05-10T11:41:04.081 INFO:tasks.cephfs.filesystem:scrub status for tag:3c8fea61-b040-48ee-91e1-f5e5c8d221f6 - {'path': '/', 'tag': '3c8fea61-b040-48ee-91e1-f5e5c8d221f6', 'options': 'recursive,force'} 2024-05-10T11:41:04.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:41:03 smithi178 ceph-mon[33359]: pgmap v1101: 129 pgs: 129 active+clean; 547 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 2.9 MiB/s wr, 273 op/s 2024-05-10T11:41:04.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:41:03 smithi188 ceph-mon[34075]: pgmap v1101: 129 pgs: 129 active+clean; 547 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 2.9 MiB/s wr, 273 op/s 2024-05-10T11:41:04.954 INFO:tasks.workunit.client.0.smithi100.stdout: 10 89462 8.92 MB/sec execute 418 sec latency 94.806 ms 2024-05-10T11:41:05.954 INFO:tasks.workunit.client.0.smithi100.stdout: 10 89616 8.92 MB/sec execute 419 sec latency 51.176 ms 2024-05-10T11:41:06.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:41:05 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:41:06.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:41:05 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:41:06.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:41:05 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:41:06.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:41:05 smithi178 ceph-mon[33359]: pgmap v1102: 129 pgs: 129 active+clean; 546 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 3.2 MiB/s wr, 262 op/s 2024-05-10T11:41:06.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:41:05 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:41:06.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:41:05 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:41:06.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:41:05 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:41:06.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:41:05 smithi188 ceph-mon[34075]: pgmap v1102: 129 pgs: 129 active+clean; 546 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 3.2 MiB/s wr, 262 op/s 2024-05-10T11:41:06.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:41:05 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:41:06.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:41:05 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:41:06.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:41:05 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:41:06.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:41:05 smithi100 ceph-mon[30296]: pgmap v1102: 129 pgs: 129 active+clean; 546 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 3.2 MiB/s wr, 262 op/s 2024-05-10T11:41:06.954 INFO:tasks.workunit.client.0.smithi100.stdout: 10 89758 8.91 MB/sec execute 420 sec latency 82.495 ms 2024-05-10T11:41:07.954 INFO:tasks.workunit.client.0.smithi100.stdout: 10 89959 8.93 MB/sec execute 421 sec latency 107.115 ms 2024-05-10T11:41:08.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:41:07 smithi178 ceph-mon[33359]: pgmap v1103: 129 pgs: 129 active+clean; 557 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 3.9 MiB/s wr, 295 op/s 2024-05-10T11:41:08.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:41:07 smithi188 ceph-mon[34075]: pgmap v1103: 129 pgs: 129 active+clean; 557 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 3.9 MiB/s wr, 295 op/s 2024-05-10T11:41:08.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:41:07 smithi100 ceph-mon[30296]: pgmap v1103: 129 pgs: 129 active+clean; 557 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 3.9 MiB/s wr, 295 op/s 2024-05-10T11:41:08.420 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.419+0000 7fd06ae45640 1 -- 172.21.15.100:0/1971758933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0641034c0 msgr2=0x7fd064109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:08.420 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.419+0000 7fd06ae45640 1 --2- 172.21.15.100:0/1971758933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0641034c0 0x7fd064109d50 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fd05c009ca0 tx=0x7fd05c02a8c0 comp rx=0 tx=0).stop 2024-05-10T11:41:08.420 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.420+0000 7fd06ae45640 1 -- 172.21.15.100:0/1971758933 shutdown_connections 2024-05-10T11:41:08.420 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.420+0000 7fd06ae45640 1 --2- 172.21.15.100:0/1971758933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0641034c0 0x7fd064109d50 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:08.421 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.420+0000 7fd06ae45640 1 --2- 172.21.15.100:0/1971758933 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd064102b00 0x7fd064102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:08.421 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.420+0000 7fd06ae45640 1 --2- 172.21.15.100:0/1971758933 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd064101910 0x7fd064101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:08.421 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.420+0000 7fd06ae45640 1 -- 172.21.15.100:0/1971758933 >> 172.21.15.100:0/1971758933 conn(0x7fd0640fce60 msgr2=0x7fd0640ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:41:08.421 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.420+0000 7fd06ae45640 1 -- 172.21.15.100:0/1971758933 shutdown_connections 2024-05-10T11:41:08.421 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.420+0000 7fd06ae45640 1 -- 172.21.15.100:0/1971758933 wait complete. 2024-05-10T11:41:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.421+0000 7fd06ae45640 1 Processor -- start 2024-05-10T11:41:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.421+0000 7fd06ae45640 1 -- start start 2024-05-10T11:41:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.421+0000 7fd06ae45640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd064101910 0x7fd06419acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.421+0000 7fd06ae45640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd064102b00 0x7fd06419b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.421+0000 7fd06ae45640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd0641034c0 0x7fd0641a21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.421+0000 7fd063fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd064102b00 0x7fd06419b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.421+0000 7fd063fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd064102b00 0x7fd06419b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58882/0 (socket says 172.21.15.100:58882) 2024-05-10T11:41:08.424 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.421+0000 7fd068be1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd064101910 0x7fd06419acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:08.424 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.421+0000 7fd068be1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd064101910 0x7fd06419acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:53746/0 (socket says 172.21.15.100:53746) 2024-05-10T11:41:08.424 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.421+0000 7fd06ae45640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd0641105d0 con 0x7fd064102b00 2024-05-10T11:41:08.424 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.421+0000 7fd06ae45640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd064110450 con 0x7fd0641034c0 2024-05-10T11:41:08.424 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.421+0000 7fd06ae45640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd064110750 con 0x7fd064101910 2024-05-10T11:41:08.424 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.421+0000 7fd063fff640 1 -- 172.21.15.100:0/701728939 learned_addr learned my addr 172.21.15.100:0/701728939 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:41:08.424 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.422+0000 7fd0693e2640 1 --2- 172.21.15.100:0/701728939 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd0641034c0 0x7fd0641a21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:08.424 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.422+0000 7fd063fff640 1 -- 172.21.15.100:0/701728939 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd064101910 msgr2=0x7fd06419acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:08.424 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.422+0000 7fd063fff640 1 --2- 172.21.15.100:0/701728939 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd064101910 0x7fd06419acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:08.424 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.422+0000 7fd063fff640 1 -- 172.21.15.100:0/701728939 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd0641034c0 msgr2=0x7fd0641a21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:08.424 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.422+0000 7fd063fff640 1 --2- 172.21.15.100:0/701728939 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd0641034c0 0x7fd0641a21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:08.424 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.422+0000 7fd063fff640 1 -- 172.21.15.100:0/701728939 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0641a2890 con 0x7fd064102b00 2024-05-10T11:41:08.424 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.422+0000 7fd0693e2640 1 --2- 172.21.15.100:0/701728939 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd0641034c0 0x7fd0641a21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:41:08.424 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.422+0000 7fd063fff640 1 --2- 172.21.15.100:0/701728939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd064102b00 0x7fd06419b210 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fd0500097d0 tx=0x7fd050009c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:41:08.425 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.423+0000 7fd061ffb640 1 -- 172.21.15.100:0/701728939 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05000eab0 con 0x7fd064102b00 2024-05-10T11:41:08.425 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.423+0000 7fd061ffb640 1 -- 172.21.15.100:0/701728939 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd050019070 con 0x7fd064102b00 2024-05-10T11:41:08.425 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.423+0000 7fd06ae45640 1 -- 172.21.15.100:0/701728939 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0641a2b80 con 0x7fd064102b00 2024-05-10T11:41:08.425 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.423+0000 7fd061ffb640 1 -- 172.21.15.100:0/701728939 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd050013380 con 0x7fd064102b00 2024-05-10T11:41:08.425 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.423+0000 7fd06ae45640 1 -- 172.21.15.100:0/701728939 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0641a30d0 con 0x7fd064102b00 2024-05-10T11:41:08.427 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.426+0000 7fd061ffb640 1 -- 172.21.15.100:0/701728939 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fd050014420 con 0x7fd064102b00 2024-05-10T11:41:08.427 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.427+0000 7fd06dc3a740 1 Processor -- start 2024-05-10T11:41:08.427 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.427+0000 7fd06dc3a740 1 -- start start 2024-05-10T11:41:08.427 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.427+0000 7fd06dc3a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c7030bdb0 0x561c7030c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:08.427 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.427+0000 7fd06dc3a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561c7030c7a0 0x561c703112a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:08.427 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.427+0000 7fd06dc3a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c70311910 0x561c70313da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:08.428 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.427+0000 7fd06dc3a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x561c702705d0 con 0x561c7030bdb0 2024-05-10T11:41:08.428 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.427+0000 7fd06dc3a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561c70229890 con 0x561c70311910 2024-05-10T11:41:08.428 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.427+0000 7fd06dc3a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x561c7026c590 con 0x561c7030c7a0 2024-05-10T11:41:08.428 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.427+0000 7fd0693e2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c7030bdb0 0x561c7030c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:08.428 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.427+0000 7fd068be1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561c7030c7a0 0x561c703112a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:08.428 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.427+0000 7fd0693e2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c7030bdb0 0x561c7030c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58890/0 (socket says 172.21.15.100:58890) 2024-05-10T11:41:08.428 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.427+0000 7fd068be1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561c7030c7a0 0x561c703112a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:53758/0 (socket says 172.21.15.100:53758) 2024-05-10T11:41:08.428 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.427+0000 7fd0693e2640 1 -- 172.21.15.100:0/3358946727 learned_addr learned my addr 172.21.15.100:0/3358946727 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:41:08.428 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.427+0000 7fd063fff640 1 --2- 172.21.15.100:0/3358946727 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c70311910 0x561c70313da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:08.428 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.427+0000 7fd061ffb640 1 --2- 172.21.15.100:0/701728939 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fd0540b28f0 0x7fd0540b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.428+0000 7fd068be1640 1 -- 172.21.15.100:0/3358946727 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c70311910 msgr2=0x561c70313da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.428+0000 7fd068be1640 1 --2- 172.21.15.100:0/3358946727 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c70311910 0x561c70313da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.428+0000 7fd068be1640 1 -- 172.21.15.100:0/3358946727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c7030bdb0 msgr2=0x561c7030c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.428+0000 7fd068be1640 1 --2- 172.21.15.100:0/3358946727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c7030bdb0 0x561c7030c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.428+0000 7fd063fff640 1 --2- 172.21.15.100:0/3358946727 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c70311910 0x561c70313da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:41:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.428+0000 7fd068be1640 1 -- 172.21.15.100:0/3358946727 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c6ff82210 con 0x561c7030c7a0 2024-05-10T11:41:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.428+0000 7fd0693e2640 1 --2- 172.21.15.100:0/3358946727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c7030bdb0 0x561c7030c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:41:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.428+0000 7fd0693e2640 1 --2- 172.21.15.100:0/701728939 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fd0540b28f0 0x7fd0540b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.428+0000 7fd061ffb640 1 -- 172.21.15.100:0/701728939 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fd0500e24d0 con 0x7fd064102b00 2024-05-10T11:41:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.428+0000 7fd068be1640 1 --2- 172.21.15.100:0/3358946727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561c7030c7a0 0x561c703112a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd064102960 tx=0x7fd058011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e12ca4086edb3b6b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:41:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.428+0000 7fd04effd640 1 -- 172.21.15.100:0/3358946727 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd058009d70 con 0x561c7030c7a0 2024-05-10T11:41:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.428+0000 7fd04effd640 1 -- 172.21.15.100:0/3358946727 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd058005020 con 0x561c7030c7a0 2024-05-10T11:41:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.428+0000 7fd04effd640 1 -- 172.21.15.100:0/3358946727 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd058004a00 con 0x561c7030c7a0 2024-05-10T11:41:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.429+0000 7fd0693e2640 1 --2- 172.21.15.100:0/701728939 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fd0540b28f0 0x7fd0540b4db0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x561c7030d000 tx=0x7fd05c0095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:41:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.429+0000 7fd06dc3a740 1 -- 172.21.15.100:0/3358946727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561c7030c7a0 msgr2=0x561c703112a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.429+0000 7fd06dc3a740 1 --2- 172.21.15.100:0/3358946727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561c7030c7a0 0x561c703112a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd064102960 tx=0x7fd058011810 comp rx=0 tx=0).stop 2024-05-10T11:41:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.429+0000 7fd06dc3a740 1 -- 172.21.15.100:0/3358946727 shutdown_connections 2024-05-10T11:41:08.431 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.429+0000 7fd06dc3a740 1 --2- 172.21.15.100:0/3358946727 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c70311910 0x561c70313da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:08.431 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.429+0000 7fd06dc3a740 1 --2- 172.21.15.100:0/3358946727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561c7030c7a0 0x561c703112a0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:08.431 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.429+0000 7fd06dc3a740 1 --2- 172.21.15.100:0/3358946727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c7030bdb0 0x561c7030c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:08.431 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.429+0000 7fd06dc3a740 1 -- 172.21.15.100:0/3358946727 >> 172.21.15.100:0/3358946727 conn(0x561c70307420 msgr2=0x561c70307820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:41:08.431 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.429+0000 7fd06dc3a740 1 -- 172.21.15.100:0/3358946727 shutdown_connections 2024-05-10T11:41:08.431 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.429+0000 7fd06dc3a740 1 -- 172.21.15.100:0/3358946727 wait complete. 2024-05-10T11:41:08.431 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.430+0000 7fd06dc3a740 1 -- start start 2024-05-10T11:41:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.430+0000 7fd06dc3a740 1 Processor -- start 2024-05-10T11:41:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.430+0000 7fd06dc3a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c7030bdb0 0x561c7031e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.431+0000 7fd06dc3a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c7030c7a0 0x561c7031ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.431+0000 7fd06dc3a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561c70311910 0x561c70325d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.431+0000 7fd068be1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c7030bdb0 0x561c7031e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.431+0000 7fd068be1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c7030bdb0 0x561c7031e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:50282/0 (socket says 172.21.15.100:50282) 2024-05-10T11:41:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.431+0000 7fd068be1640 1 -- 172.21.15.100:0/1069442868 learned_addr learned my addr 172.21.15.100:0/1069442868 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:41:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.431+0000 7fd06dc3a740 1 -- 172.21.15.100:0/1069442868 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x561c703148c0 con 0x561c7030c7a0 2024-05-10T11:41:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.431+0000 7fd06dc3a740 1 -- 172.21.15.100:0/1069442868 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561c7022d370 con 0x561c7030bdb0 2024-05-10T11:41:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.431+0000 7fd06dc3a740 1 -- 172.21.15.100:0/1069442868 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x561c70314a40 con 0x561c70311910 2024-05-10T11:41:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.431+0000 7fd0693e2640 1 --2- 172.21.15.100:0/1069442868 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561c70311910 0x561c70325d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.431+0000 7fd063fff640 1 --2- 172.21.15.100:0/1069442868 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c7030c7a0 0x561c7031ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.431+0000 7fd06dc3a740 10 client.0 _ll_register_callbacks cb 0x561c6ff81a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:41:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.431+0000 7fd068be1640 1 -- 172.21.15.100:0/1069442868 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561c70311910 msgr2=0x561c70325d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.431+0000 7fd068be1640 1 --2- 172.21.15.100:0/1069442868 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561c70311910 0x561c70325d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:08.435 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.431+0000 7fd068be1640 1 -- 172.21.15.100:0/1069442868 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c7030c7a0 msgr2=0x561c7031ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:08.435 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.431+0000 7fd068be1640 1 --2- 172.21.15.100:0/1069442868 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c7030c7a0 0x561c7031ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:08.435 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.431+0000 7fd068be1640 1 -- 172.21.15.100:0/1069442868 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c7004e620 con 0x561c7030bdb0 2024-05-10T11:41:08.435 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.431+0000 7fd068be1640 1 --2- 172.21.15.100:0/1069442868 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c7030bdb0 0x561c7031e7d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd05801b040 tx=0x7fd058004970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5dffa88dff28ca10 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:41:08.435 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.431+0000 7fd043fff640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:41:08.435 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.431+0000 7fd043fff640 1 -- 172.21.15.100:0/1069442868 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd058029020 con 0x561c7030bdb0 2024-05-10T11:41:08.435 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.432+0000 7fd043fff640 1 -- 172.21.15.100:0/1069442868 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fd058004d70 con 0x561c7030bdb0 2024-05-10T11:41:08.435 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.432+0000 7fd06dc3a740 1 -- 172.21.15.100:0/1069442868 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd058005020 con 0x561c7030bdb0 2024-05-10T11:41:08.435 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.432+0000 7fd043fff640 1 -- 172.21.15.100:0/1069442868 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fd058034050 con 0x561c7030bdb0 2024-05-10T11:41:08.435 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.432+0000 7fd06dc3a740 10 client.29087 fetch_fsmap learned FSMap version 41 2024-05-10T11:41:08.435 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.432+0000 7fd06dc3a740 1 -- 172.21.15.100:0/1069442868 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x561c70328a70 con 0x561c7030bdb0 2024-05-10T11:41:08.435 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.432+0000 7fd043fff640 1 -- 172.21.15.100:0/1069442868 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd058005c50 con 0x561c7030bdb0 2024-05-10T11:41:08.435 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.432+0000 7fd043fff640 1 -- 172.21.15.100:0/1069442868 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fd058019570 con 0x561c7030bdb0 2024-05-10T11:41:08.435 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.433+0000 7fd06dc3a740 10 client.29087 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:41:08.435 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.433+0000 7fd06dc3a740 10 client.29087 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:41:08.435 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.433+0000 7fd06dc3a740 20 client.29087 populate_metadata read hostname 'smithi100' 2024-05-10T11:41:08.436 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.433+0000 7fd06dc3a740 1 --2- 172.21.15.100:0/1069442868 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x561c70329f40 0x561c7032c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:08.436 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.433+0000 7fd06dc3a740 4 client.29087 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:41:08.436 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.433+0000 7fd06dc3a740 1 -- 172.21.15.100:0/1069442868 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd058005020 con 0x561c70329f40 2024-05-10T11:41:08.436 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.433+0000 7fd063fff640 1 --2- 172.21.15.100:0/1069442868 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x561c70329f40 0x561c7032c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:08.436 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.433+0000 7fd063fff640 1 --2- 172.21.15.100:0/1069442868 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x561c70329f40 0x561c7032c350 crc :-1 s=READY pgs=429 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84d43873fe3e0153 server_cookie=278e9aede8813eac in_seq=0 out_seq=0 2024-05-10T11:41:08.436 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.433+0000 7fd043fff640 10 client.29087 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:41:08.437 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.434+0000 7fd043fff640 1 -- 172.21.15.100:0/1069442868 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fd058005020 con 0x561c70329f40 2024-05-10T11:41:08.437 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.434+0000 7fd043fff640 10 client.29087 handle_command_reply: tid=0 2024-05-10T11:41:08.437 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.434+0000 7fd06dc3a740 1 client.29087 shutdown 2024-05-10T11:41:08.437 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.434+0000 7fd06dc3a740 1 -- 172.21.15.100:0/1069442868 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c7030bdb0 msgr2=0x561c7031e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:08.437 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.434+0000 7fd06dc3a740 1 --2- 172.21.15.100:0/1069442868 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c7030bdb0 0x561c7031e7d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd05801b040 tx=0x7fd058004970 comp rx=0 tx=0).stop 2024-05-10T11:41:08.437 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.435+0000 7fd06dc3a740 1 -- 172.21.15.100:0/1069442868 shutdown_connections 2024-05-10T11:41:08.438 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.435+0000 7fd06dc3a740 1 --2- 172.21.15.100:0/1069442868 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x561c70329f40 0x561c7032c350 crc :-1 s=READY pgs=429 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:08.438 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.435+0000 7fd06dc3a740 1 --2- 172.21.15.100:0/1069442868 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561c70311910 0x561c70325d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:08.438 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.435+0000 7fd06dc3a740 1 --2- 172.21.15.100:0/1069442868 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c7030c7a0 0x561c7031ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:08.438 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.435+0000 7fd06dc3a740 1 --2- 172.21.15.100:0/1069442868 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c7030bdb0 0x561c7031e7d0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:08.438 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.435+0000 7fd06dc3a740 1 -- 172.21.15.100:0/1069442868 >> 172.21.15.100:0/1069442868 conn(0x561c70307420 msgr2=0x561c70306580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:41:08.438 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.435+0000 7fd043fff640 0 client.29087 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:41:08.438 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.435+0000 7fd06dc3a740 1 -- 172.21.15.100:0/1069442868 shutdown_connections 2024-05-10T11:41:08.438 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.435+0000 7fd06dc3a740 1 -- 172.21.15.100:0/1069442868 wait complete. 2024-05-10T11:41:08.438 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.435+0000 7fd06dc3a740 20 client.29087 trim_cache size 0 max 16384 2024-05-10T11:41:08.463 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.461+0000 7fd06dc3a740 1 Processor -- start 2024-05-10T11:41:08.463 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.461+0000 7fd06dc3a740 1 -- start start 2024-05-10T11:41:08.463 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.462+0000 7fd06dc3a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561c7030bdb0 0x561c7030d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:08.463 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.462+0000 7fd06dc3a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c7030c7a0 0x561c7030de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:08.465 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.462+0000 7fd06dc3a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c70311910 0x561c7030a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:08.465 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.462+0000 7fd06dc3a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd0641aba80 con 0x561c7030c7a0 2024-05-10T11:41:08.466 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.462+0000 7fd06dc3a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd0641a9350 con 0x561c70311910 2024-05-10T11:41:08.466 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.462+0000 7fd06dc3a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd0641a71c0 con 0x561c7030bdb0 2024-05-10T11:41:08.466 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.462+0000 7fd063fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c7030c7a0 0x561c7030de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:08.466 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.462+0000 7fd063fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c7030c7a0 0x561c7030de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58908/0 (socket says 172.21.15.100:58908) 2024-05-10T11:41:08.466 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.462+0000 7fd063fff640 1 -- 172.21.15.100:0/2862831385 learned_addr learned my addr 172.21.15.100:0/2862831385 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:41:08.466 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.462+0000 7fd068be1640 1 --2- 172.21.15.100:0/2862831385 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561c7030bdb0 0x561c7030d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:08.466 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.462+0000 7fd0693e2640 1 --2- 172.21.15.100:0/2862831385 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c70311910 0x561c7030a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:08.466 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.462+0000 7fd063fff640 1 -- 172.21.15.100:0/2862831385 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561c7030bdb0 msgr2=0x561c7030d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:08.466 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.462+0000 7fd063fff640 1 --2- 172.21.15.100:0/2862831385 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561c7030bdb0 0x561c7030d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:08.466 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.462+0000 7fd063fff640 1 -- 172.21.15.100:0/2862831385 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c70311910 msgr2=0x561c7030a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:08.466 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.462+0000 7fd063fff640 1 --2- 172.21.15.100:0/2862831385 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c70311910 0x561c7030a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:08.466 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.462+0000 7fd063fff640 1 -- 172.21.15.100:0/2862831385 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c70307250 con 0x561c7030c7a0 2024-05-10T11:41:08.466 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.463+0000 7fd063fff640 1 --2- 172.21.15.100:0/2862831385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c7030c7a0 0x561c7030de60 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fd05000b3c0 tx=0x7fd0500e49e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79ec5accb0caeb91 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:41:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.463+0000 7fd04e7fc640 1 -- 172.21.15.100:0/2862831385 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0500cc0c0 con 0x561c7030c7a0 2024-05-10T11:41:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.463+0000 7fd04e7fc640 1 -- 172.21.15.100:0/2862831385 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd0500a9790 con 0x561c7030c7a0 2024-05-10T11:41:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.463+0000 7fd04e7fc640 1 -- 172.21.15.100:0/2862831385 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0500a5990 con 0x561c7030c7a0 2024-05-10T11:41:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.464+0000 7fd06dc3a740 1 -- 172.21.15.100:0/2862831385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c7030c7a0 msgr2=0x561c7030de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.464+0000 7fd06dc3a740 1 --2- 172.21.15.100:0/2862831385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c7030c7a0 0x561c7030de60 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fd05000b3c0 tx=0x7fd0500e49e0 comp rx=0 tx=0).stop 2024-05-10T11:41:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.464+0000 7fd06dc3a740 1 -- 172.21.15.100:0/2862831385 shutdown_connections 2024-05-10T11:41:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.464+0000 7fd06dc3a740 1 --2- 172.21.15.100:0/2862831385 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c70311910 0x561c7030a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.464+0000 7fd06dc3a740 1 --2- 172.21.15.100:0/2862831385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c7030c7a0 0x561c7030de60 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.464+0000 7fd06dc3a740 1 --2- 172.21.15.100:0/2862831385 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561c7030bdb0 0x561c7030d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.464+0000 7fd06dc3a740 1 -- 172.21.15.100:0/2862831385 >> 172.21.15.100:0/2862831385 conn(0x561c70307420 msgr2=0x561c703023e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:41:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.464+0000 7fd06dc3a740 1 -- 172.21.15.100:0/2862831385 shutdown_connections 2024-05-10T11:41:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.464+0000 7fd06dc3a740 1 -- 172.21.15.100:0/2862831385 wait complete. 2024-05-10T11:41:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.464+0000 7fd06dc3a740 1 -- start start 2024-05-10T11:41:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.464+0000 7fd06dc3a740 1 Processor -- start 2024-05-10T11:41:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.465+0000 7fd06dc3a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c7030bdb0 0x561c703229b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.465+0000 7fd06dc3a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c7030c7a0 0x561c70322ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.465+0000 7fd06dc3a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561c70311910 0x561c70323f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.465+0000 7fd06dc3a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd0641a7810 con 0x561c7030bdb0 2024-05-10T11:41:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.465+0000 7fd06dc3a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd0641a8c80 con 0x561c7030c7a0 2024-05-10T11:41:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.465+0000 7fd06dc3a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd0641a7ee0 con 0x561c70311910 2024-05-10T11:41:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.465+0000 7fd068be1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c7030bdb0 0x561c703229b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.465+0000 7fd06dc3a740 10 client.0 _ll_register_callbacks cb 0x7fd058005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:41:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.465+0000 7fd068be1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c7030bdb0 0x561c703229b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58920/0 (socket says 172.21.15.100:58920) 2024-05-10T11:41:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.465+0000 7fd068be1640 1 -- 172.21.15.100:0/3787604869 learned_addr learned my addr 172.21.15.100:0/3787604869 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:41:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.465+0000 7fd063fff640 1 --2- 172.21.15.100:0/3787604869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c7030c7a0 0x561c70322ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.465+0000 7fd0693e2640 1 --2- 172.21.15.100:0/3787604869 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561c70311910 0x561c70323f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.465+0000 7fd063fff640 1 -- 172.21.15.100:0/3787604869 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561c70311910 msgr2=0x561c70323f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.465+0000 7fd063fff640 1 --2- 172.21.15.100:0/3787604869 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561c70311910 0x561c70323f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.465+0000 7fd063fff640 1 -- 172.21.15.100:0/3787604869 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c7030bdb0 msgr2=0x561c703229b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.465+0000 7fd063fff640 1 --2- 172.21.15.100:0/3787604869 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c7030bdb0 0x561c703229b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.465+0000 7fd063fff640 1 -- 172.21.15.100:0/3787604869 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c703244f0 con 0x561c7030c7a0 2024-05-10T11:41:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.465+0000 7fd068be1640 1 --2- 172.21.15.100:0/3787604869 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c7030bdb0 0x561c703229b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:41:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.466+0000 7fd063fff640 1 --2- 172.21.15.100:0/3787604869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c7030c7a0 0x561c70322ef0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd050014960 tx=0x7fd0500a5660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b38ae06d76ffad82 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:41:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.466+0000 7fd043fff640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:41:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.466+0000 7fd043fff640 1 -- 172.21.15.100:0/3787604869 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0500adb40 con 0x561c7030c7a0 2024-05-10T11:41:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.466+0000 7fd043fff640 1 -- 172.21.15.100:0/3787604869 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fd0500e2210 con 0x561c7030c7a0 2024-05-10T11:41:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.466+0000 7fd06dc3a740 1 -- 172.21.15.100:0/3787604869 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0500a9790 con 0x561c7030c7a0 2024-05-10T11:41:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.466+0000 7fd043fff640 1 -- 172.21.15.100:0/3787604869 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fd0500a5990 con 0x561c7030c7a0 2024-05-10T11:41:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.466+0000 7fd06dc3a740 10 client.29093 fetch_fsmap learned FSMap version 41 2024-05-10T11:41:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.466+0000 7fd06dc3a740 1 -- 172.21.15.100:0/3787604869 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x561c70394810 con 0x561c7030c7a0 2024-05-10T11:41:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.466+0000 7fd043fff640 1 -- 172.21.15.100:0/3787604869 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd0500a9790 con 0x561c7030c7a0 2024-05-10T11:41:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.467+0000 7fd043fff640 1 -- 172.21.15.100:0/3787604869 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fd0500b21f0 con 0x561c7030c7a0 2024-05-10T11:41:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.467+0000 7fd06dc3a740 10 client.29093 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:41:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.467+0000 7fd06dc3a740 10 client.29093 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:41:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.467+0000 7fd06dc3a740 20 client.29093 populate_metadata read hostname 'smithi100' 2024-05-10T11:41:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.467+0000 7fd06dc3a740 1 --2- 172.21.15.100:0/3787604869 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x561c70394d60 0x561c70397180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.467+0000 7fd06dc3a740 4 client.29093 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:41:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.467+0000 7fd06dc3a740 1 -- 172.21.15.100:0/3787604869 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561c7027e1e0 con 0x561c70394d60 2024-05-10T11:41:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.467+0000 7fd068be1640 1 --2- 172.21.15.100:0/3787604869 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x561c70394d60 0x561c70397180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.467+0000 7fd043fff640 10 client.29093 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:41:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.467+0000 7fd068be1640 1 --2- 172.21.15.100:0/3787604869 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x561c70394d60 0x561c70397180 crc :-1 s=READY pgs=430 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea65e8f8e7b0858c server_cookie=16b6a01604b5e14f in_seq=0 out_seq=0 2024-05-10T11:41:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.626+0000 7fd043fff640 1 -- 172.21.15.100:0/3787604869 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561c7027e1e0 con 0x561c70394d60 2024-05-10T11:41:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.626+0000 7fd043fff640 10 client.29093 handle_command_reply: tid=0 2024-05-10T11:41:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.626+0000 7fd06dc3a740 1 client.29093 shutdown 2024-05-10T11:41:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.626+0000 7fd06dc3a740 1 -- 172.21.15.100:0/3787604869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c7030c7a0 msgr2=0x561c70322ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.626+0000 7fd06dc3a740 1 --2- 172.21.15.100:0/3787604869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c7030c7a0 0x561c70322ef0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd050014960 tx=0x7fd0500a5660 comp rx=0 tx=0).stop 2024-05-10T11:41:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.627+0000 7fd06dc3a740 1 -- 172.21.15.100:0/3787604869 shutdown_connections 2024-05-10T11:41:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.627+0000 7fd06dc3a740 1 --2- 172.21.15.100:0/3787604869 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x561c70394d60 0x561c70397180 crc :-1 s=READY pgs=430 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.627+0000 7fd06dc3a740 1 --2- 172.21.15.100:0/3787604869 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561c70311910 0x561c70323f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.627+0000 7fd06dc3a740 1 --2- 172.21.15.100:0/3787604869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c7030c7a0 0x561c70322ef0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.627+0000 7fd06dc3a740 1 --2- 172.21.15.100:0/3787604869 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c7030bdb0 0x561c703229b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.627+0000 7fd043fff640 0 client.29093 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:41:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.627+0000 7fd06dc3a740 1 -- 172.21.15.100:0/3787604869 >> 172.21.15.100:0/3787604869 conn(0x561c70307420 msgr2=0x561c7030d920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:41:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.627+0000 7fd06dc3a740 1 -- 172.21.15.100:0/3787604869 shutdown_connections 2024-05-10T11:41:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.627+0000 7fd06dc3a740 1 -- 172.21.15.100:0/3787604869 wait complete. 2024-05-10T11:41:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.627+0000 7fd06dc3a740 20 client.29093 trim_cache size 0 max 16384 2024-05-10T11:41:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:41:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:41:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:41:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:41:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.628+0000 7fd06ae45640 1 -- 172.21.15.100:0/701728939 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fd0540b28f0 msgr2=0x7fd0540b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.628+0000 7fd06ae45640 1 --2- 172.21.15.100:0/701728939 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fd0540b28f0 0x7fd0540b4db0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x561c7030d000 tx=0x7fd05c0095c0 comp rx=0 tx=0).stop 2024-05-10T11:41:08.630 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.628+0000 7fd06ae45640 1 -- 172.21.15.100:0/701728939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd064102b00 msgr2=0x7fd06419b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:08.630 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.628+0000 7fd06ae45640 1 --2- 172.21.15.100:0/701728939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd064102b00 0x7fd06419b210 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fd0500097d0 tx=0x7fd050009c90 comp rx=0 tx=0).stop 2024-05-10T11:41:08.630 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.628+0000 7fd06ae45640 1 -- 172.21.15.100:0/701728939 shutdown_connections 2024-05-10T11:41:08.630 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.628+0000 7fd06ae45640 1 --2- 172.21.15.100:0/701728939 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fd0540b28f0 0x7fd0540b4db0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:08.630 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.628+0000 7fd06ae45640 1 --2- 172.21.15.100:0/701728939 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd0641034c0 0x7fd0641a21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:08.630 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.628+0000 7fd06ae45640 1 --2- 172.21.15.100:0/701728939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd064102b00 0x7fd06419b210 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:08.630 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.628+0000 7fd06ae45640 1 --2- 172.21.15.100:0/701728939 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd064101910 0x7fd06419acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:08.630 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.628+0000 7fd06ae45640 1 -- 172.21.15.100:0/701728939 >> 172.21.15.100:0/701728939 conn(0x7fd0640fce60 msgr2=0x7fd064105e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:41:08.630 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.629+0000 7fd06ae45640 1 -- 172.21.15.100:0/701728939 shutdown_connections 2024-05-10T11:41:08.630 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:08.629+0000 7fd06ae45640 1 -- 172.21.15.100:0/701728939 wait complete. 2024-05-10T11:41:08.955 INFO:tasks.workunit.client.0.smithi100.stdout: 10 90115 8.93 MB/sec execute 422 sec latency 141.607 ms 2024-05-10T11:41:09.954 INFO:tasks.workunit.client.0.smithi100.stdout: 10 90269 8.92 MB/sec execute 423 sec latency 68.481 ms 2024-05-10T11:41:10.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:41:09 smithi178 ceph-mon[33359]: pgmap v1104: 129 pgs: 129 active+clean; 556 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 3.5 MiB/s wr, 227 op/s 2024-05-10T11:41:10.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:41:09 smithi188 ceph-mon[34075]: pgmap v1104: 129 pgs: 129 active+clean; 556 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 3.5 MiB/s wr, 227 op/s 2024-05-10T11:41:10.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:41:09 smithi100 ceph-mon[30296]: pgmap v1104: 129 pgs: 129 active+clean; 556 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 3.5 MiB/s wr, 227 op/s 2024-05-10T11:41:10.955 INFO:tasks.workunit.client.0.smithi100.stdout: 10 90466 8.93 MB/sec execute 424 sec latency 75.340 ms 2024-05-10T11:41:11.955 INFO:tasks.workunit.client.0.smithi100.stdout: 10 90626 8.92 MB/sec execute 425 sec latency 77.523 ms 2024-05-10T11:41:12.090 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:41:11 smithi100 ceph-mon[30296]: pgmap v1105: 129 pgs: 129 active+clean; 555 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 4.5 MiB/s wr, 302 op/s 2024-05-10T11:41:12.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:41:11 smithi178 ceph-mon[33359]: pgmap v1105: 129 pgs: 129 active+clean; 555 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 4.5 MiB/s wr, 302 op/s 2024-05-10T11:41:12.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:41:11 smithi188 ceph-mon[34075]: pgmap v1105: 129 pgs: 129 active+clean; 555 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 4.5 MiB/s wr, 302 op/s 2024-05-10T11:41:12.955 INFO:tasks.workunit.client.0.smithi100.stdout: 10 90772 8.91 MB/sec execute 426 sec latency 101.906 ms 2024-05-10T11:41:13.955 INFO:tasks.workunit.client.0.smithi100.stdout: 10 90969 8.92 MB/sec execute 427 sec latency 126.490 ms 2024-05-10T11:41:14.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:41:13 smithi178 ceph-mon[33359]: pgmap v1106: 129 pgs: 129 active+clean; 556 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 4.2 MiB/s wr, 262 op/s 2024-05-10T11:41:14.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:41:13 smithi188 ceph-mon[34075]: pgmap v1106: 129 pgs: 129 active+clean; 556 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 4.2 MiB/s wr, 262 op/s 2024-05-10T11:41:14.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:41:13 smithi100 ceph-mon[30296]: pgmap v1106: 129 pgs: 129 active+clean; 556 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 4.2 MiB/s wr, 262 op/s 2024-05-10T11:41:14.955 INFO:tasks.workunit.client.0.smithi100.stdout: 10 91173 8.93 MB/sec execute 428 sec latency 92.691 ms 2024-05-10T11:41:15.955 INFO:tasks.workunit.client.0.smithi100.stdout: 10 91331 8.92 MB/sec execute 429 sec latency 74.991 ms 2024-05-10T11:41:16.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:41:15 smithi178 ceph-mon[33359]: pgmap v1107: 129 pgs: 129 active+clean; 558 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 4.8 MiB/s wr, 288 op/s 2024-05-10T11:41:16.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:41:15 smithi188 ceph-mon[34075]: pgmap v1107: 129 pgs: 129 active+clean; 558 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 4.8 MiB/s wr, 288 op/s 2024-05-10T11:41:16.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:41:15 smithi100 ceph-mon[30296]: pgmap v1107: 129 pgs: 129 active+clean; 558 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 4.8 MiB/s wr, 288 op/s 2024-05-10T11:41:16.955 INFO:tasks.workunit.client.0.smithi100.stdout: 10 91496 8.91 MB/sec execute 430 sec latency 67.969 ms 2024-05-10T11:41:17.955 INFO:tasks.workunit.client.0.smithi100.stdout: 10 91680 8.92 MB/sec execute 431 sec latency 78.405 ms 2024-05-10T11:41:18.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:41:17 smithi178 ceph-mon[33359]: pgmap v1108: 129 pgs: 129 active+clean; 553 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 5.0 MiB/s wr, 346 op/s 2024-05-10T11:41:18.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:41:17 smithi188 ceph-mon[34075]: pgmap v1108: 129 pgs: 129 active+clean; 553 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 5.0 MiB/s wr, 346 op/s 2024-05-10T11:41:18.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:41:17 smithi100 ceph-mon[30296]: pgmap v1108: 129 pgs: 129 active+clean; 553 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 5.0 MiB/s wr, 346 op/s 2024-05-10T11:41:18.955 INFO:tasks.workunit.client.0.smithi100.stdout: 10 91835 8.90 MB/sec execute 432 sec latency 67.323 ms 2024-05-10T11:41:19.955 INFO:tasks.workunit.client.0.smithi100.stdout: 10 92014 8.91 MB/sec execute 433 sec latency 79.309 ms 2024-05-10T11:41:20.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:41:19 smithi178 ceph-mon[33359]: pgmap v1109: 129 pgs: 129 active+clean; 554 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 822 KiB/s rd, 4.2 MiB/s wr, 309 op/s 2024-05-10T11:41:20.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:41:19 smithi188 ceph-mon[34075]: pgmap v1109: 129 pgs: 129 active+clean; 554 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 822 KiB/s rd, 4.2 MiB/s wr, 309 op/s 2024-05-10T11:41:20.290 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:41:19 smithi100 ceph-mon[30296]: pgmap v1109: 129 pgs: 129 active+clean; 554 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 822 KiB/s rd, 4.2 MiB/s wr, 309 op/s 2024-05-10T11:41:20.955 INFO:tasks.workunit.client.0.smithi100.stdout: 10 92193 8.91 MB/sec execute 434 sec latency 76.682 ms 2024-05-10T11:41:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.759+0000 7fde8039e640 1 -- 172.21.15.100:0/1177649535 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde781034b0 msgr2=0x7fde78109d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:21.760 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.759+0000 7fde8039e640 1 --2- 172.21.15.100:0/1177649535 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde781034b0 0x7fde78109d40 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fde70009be0 tx=0x7fde7002a6c0 comp rx=0 tx=0).stop 2024-05-10T11:41:21.760 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.759+0000 7fde8039e640 1 -- 172.21.15.100:0/1177649535 shutdown_connections 2024-05-10T11:41:21.760 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.759+0000 7fde8039e640 1 --2- 172.21.15.100:0/1177649535 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde781034b0 0x7fde78109d40 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:21.760 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.759+0000 7fde8039e640 1 --2- 172.21.15.100:0/1177649535 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde78102af0 0x7fde78102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:21.760 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.759+0000 7fde8039e640 1 --2- 172.21.15.100:0/1177649535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde78101900 0x7fde78101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:21.760 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.759+0000 7fde8039e640 1 -- 172.21.15.100:0/1177649535 >> 172.21.15.100:0/1177649535 conn(0x7fde780fce70 msgr2=0x7fde780ff2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:41:21.760 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.759+0000 7fde8039e640 1 -- 172.21.15.100:0/1177649535 shutdown_connections 2024-05-10T11:41:21.760 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.759+0000 7fde8039e640 1 -- 172.21.15.100:0/1177649535 wait complete. 2024-05-10T11:41:21.762 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.760+0000 7fde8039e640 1 Processor -- start 2024-05-10T11:41:21.762 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.760+0000 7fde8039e640 1 -- start start 2024-05-10T11:41:21.762 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.760+0000 7fde8039e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde78101900 0x7fde7819ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:21.763 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.760+0000 7fde8039e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde78102af0 0x7fde7819b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:21.763 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.760+0000 7fde8039e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde781034b0 0x7fde781a21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:21.763 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.760+0000 7fde8039e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fde7810c1b0 con 0x7fde78101900 2024-05-10T11:41:21.763 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.760+0000 7fde8039e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fde7810c030 con 0x7fde781034b0 2024-05-10T11:41:21.763 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.760+0000 7fde8039e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fde7810c330 con 0x7fde78102af0 2024-05-10T11:41:21.763 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.760+0000 7fde7e93b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde781034b0 0x7fde781a21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:21.763 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.760+0000 7fde7d939640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde78102af0 0x7fde7819b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:21.763 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.760+0000 7fde7e93b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde781034b0 0x7fde781a21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:49286/0 (socket says 172.21.15.100:49286) 2024-05-10T11:41:21.763 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.760+0000 7fde7e93b640 1 -- 172.21.15.100:0/3229035594 learned_addr learned my addr 172.21.15.100:0/3229035594 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:41:21.763 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.760+0000 7fde7d939640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde78102af0 0x7fde7819b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:56526/0 (socket says 172.21.15.100:56526) 2024-05-10T11:41:21.763 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.760+0000 7fde7e13a640 1 --2- 172.21.15.100:0/3229035594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde78101900 0x7fde7819ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:21.763 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.761+0000 7fde7e93b640 1 -- 172.21.15.100:0/3229035594 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde78102af0 msgr2=0x7fde7819b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:21.763 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.761+0000 7fde7e93b640 1 --2- 172.21.15.100:0/3229035594 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde78102af0 0x7fde7819b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:21.763 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.761+0000 7fde7e93b640 1 -- 172.21.15.100:0/3229035594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde78101900 msgr2=0x7fde7819ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:21.763 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.761+0000 7fde7e93b640 1 --2- 172.21.15.100:0/3229035594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde78101900 0x7fde7819ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:21.764 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.761+0000 7fde7e93b640 1 -- 172.21.15.100:0/3229035594 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde781a2870 con 0x7fde781034b0 2024-05-10T11:41:21.764 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.761+0000 7fde7e93b640 1 --2- 172.21.15.100:0/3229035594 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde781034b0 0x7fde781a21d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fde70034040 tx=0x7fde70030e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:41:21.764 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.761+0000 7fde6f7fe640 1 -- 172.21.15.100:0/3229035594 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde70041020 con 0x7fde781034b0 2024-05-10T11:41:21.764 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.761+0000 7fde8039e640 1 -- 172.21.15.100:0/3229035594 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde78103de0 con 0x7fde781034b0 2024-05-10T11:41:21.764 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.761+0000 7fde6f7fe640 1 -- 172.21.15.100:0/3229035594 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fde70037040 con 0x7fde781034b0 2024-05-10T11:41:21.764 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.762+0000 7fde6f7fe640 1 -- 172.21.15.100:0/3229035594 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde70036070 con 0x7fde781034b0 2024-05-10T11:41:21.764 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.762+0000 7fde8039e640 1 -- 172.21.15.100:0/3229035594 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde781042d0 con 0x7fde781034b0 2024-05-10T11:41:21.765 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.765+0000 7fde6f7fe640 1 -- 172.21.15.100:0/3229035594 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fde70044020 con 0x7fde781034b0 2024-05-10T11:41:21.765 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.765+0000 7fde8318b740 1 Processor -- start 2024-05-10T11:41:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.766+0000 7fde8318b740 1 -- start start 2024-05-10T11:41:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.766+0000 7fde8318b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d963408db0 0x55d9634091d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.766+0000 7fde8318b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d9634097a0 0x55d96340e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.766+0000 7fde8318b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d96340e910 0x55d963410da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.766+0000 7fde8318b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55d96336d5d0 con 0x55d96340e910 2024-05-10T11:41:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.766+0000 7fde8318b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d963326890 con 0x55d9634097a0 2024-05-10T11:41:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.766+0000 7fde8318b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d963369590 con 0x55d963408db0 2024-05-10T11:41:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.766+0000 7fde7e13a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d9634097a0 0x55d96340e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.766+0000 7fde7e13a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d9634097a0 0x55d96340e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:49296/0 (socket says 172.21.15.100:49296) 2024-05-10T11:41:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.766+0000 7fde7e13a640 1 -- 172.21.15.100:0/2384573218 learned_addr learned my addr 172.21.15.100:0/2384573218 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:41:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.766+0000 7fde7d939640 1 --2- 172.21.15.100:0/2384573218 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d963408db0 0x55d9634091d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.766+0000 7fde7d939640 1 --2- 172.21.15.100:0/2384573218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d96340e910 0x55d963410da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.766+0000 7fde7e13a640 1 -- 172.21.15.100:0/2384573218 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d963408db0 msgr2=0x55d9634091d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.766+0000 7fde7e13a640 1 --2- 172.21.15.100:0/2384573218 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d963408db0 0x55d9634091d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.766+0000 7fde7e13a640 1 -- 172.21.15.100:0/2384573218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d96340e910 msgr2=0x55d963410da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.766+0000 7fde7e13a640 1 --2- 172.21.15.100:0/2384573218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d96340e910 0x55d963410da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.767+0000 7fde7e13a640 1 -- 172.21.15.100:0/2384573218 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d96307f210 con 0x55d9634097a0 2024-05-10T11:41:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.767+0000 7fde7d939640 1 --2- 172.21.15.100:0/2384573218 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d963408db0 0x55d9634091d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:41:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.767+0000 7fde7e13a640 1 --2- 172.21.15.100:0/2384573218 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d9634097a0 0x55d96340e2a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fde74014f30 tx=0x7fde74009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=677d43e603f604e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:41:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.767+0000 7fde6cbfe640 1 -- 172.21.15.100:0/2384573218 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde74022030 con 0x55d9634097a0 2024-05-10T11:41:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.767+0000 7fde6cbfe640 1 -- 172.21.15.100:0/2384573218 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fde74009da0 con 0x55d9634097a0 2024-05-10T11:41:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.767+0000 7fde6cbfe640 1 -- 172.21.15.100:0/2384573218 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde740047a0 con 0x55d9634097a0 2024-05-10T11:41:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.767+0000 7fde6f7fe640 1 --2- 172.21.15.100:0/3229035594 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fde640b28a0 0x7fde640b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:21.772 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.768+0000 7fde7d939640 1 --2- 172.21.15.100:0/3229035594 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fde640b28a0 0x7fde640b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:21.772 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.768+0000 7fde6f7fe640 1 -- 172.21.15.100:0/3229035594 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fde70103110 con 0x7fde781034b0 2024-05-10T11:41:21.772 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.768+0000 7fde7d939640 1 --2- 172.21.15.100:0/3229035594 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fde640b28a0 0x7fde640b4d60 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fde6800dfd0 tx=0x7fde68005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:41:21.772 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.768+0000 7fde8318b740 1 -- 172.21.15.100:0/2384573218 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d9634097a0 msgr2=0x55d96340e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:21.772 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.768+0000 7fde8318b740 1 --2- 172.21.15.100:0/2384573218 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d9634097a0 0x55d96340e2a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fde74014f30 tx=0x7fde74009260 comp rx=0 tx=0).stop 2024-05-10T11:41:21.772 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.768+0000 7fde8318b740 1 -- 172.21.15.100:0/2384573218 shutdown_connections 2024-05-10T11:41:21.772 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.768+0000 7fde8318b740 1 --2- 172.21.15.100:0/2384573218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d96340e910 0x55d963410da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:21.772 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.768+0000 7fde8318b740 1 --2- 172.21.15.100:0/2384573218 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d9634097a0 0x55d96340e2a0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:21.772 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.768+0000 7fde8318b740 1 --2- 172.21.15.100:0/2384573218 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d963408db0 0x55d9634091d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:21.772 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.768+0000 7fde8318b740 1 -- 172.21.15.100:0/2384573218 >> 172.21.15.100:0/2384573218 conn(0x55d963404420 msgr2=0x55d963404820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:41:21.772 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.768+0000 7fde8318b740 1 -- 172.21.15.100:0/2384573218 shutdown_connections 2024-05-10T11:41:21.772 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.768+0000 7fde8318b740 1 -- 172.21.15.100:0/2384573218 wait complete. 2024-05-10T11:41:21.773 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.769+0000 7fde8318b740 1 -- start start 2024-05-10T11:41:21.773 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.769+0000 7fde8318b740 1 Processor -- start 2024-05-10T11:41:21.773 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.769+0000 7fde8318b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d963408db0 0x55d963412ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:21.773 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.769+0000 7fde8318b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d9634097a0 0x55d963413530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:21.773 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.769+0000 7fde8318b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d96340e910 0x55d96341a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:21.773 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.769+0000 7fde7e13a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d963408db0 0x55d963412ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:21.773 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.769+0000 7fde7e13a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d963408db0 0x55d963412ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:49302/0 (socket says 172.21.15.100:49302) 2024-05-10T11:41:21.773 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.769+0000 7fde7e13a640 1 -- 172.21.15.100:0/2996383834 learned_addr learned my addr 172.21.15.100:0/2996383834 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:41:21.773 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.770+0000 7fde7e93b640 1 --2- 172.21.15.100:0/2996383834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d96340e910 0x55d96341a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:21.773 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.770+0000 7fde7d939640 1 --2- 172.21.15.100:0/2996383834 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d9634097a0 0x55d963413530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:21.773 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.769+0000 7fde8318b740 1 -- 172.21.15.100:0/2996383834 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55d9634118c0 con 0x55d96340e910 2024-05-10T11:41:21.773 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.770+0000 7fde8318b740 1 -- 172.21.15.100:0/2996383834 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d96332a370 con 0x55d963408db0 2024-05-10T11:41:21.773 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.770+0000 7fde8318b740 1 -- 172.21.15.100:0/2996383834 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d963411a40 con 0x55d9634097a0 2024-05-10T11:41:21.773 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.770+0000 7fde8318b740 10 client.0 _ll_register_callbacks cb 0x55d96307ea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:41:21.773 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.770+0000 7fde7e13a640 1 -- 172.21.15.100:0/2996383834 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d9634097a0 msgr2=0x55d963413530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.770+0000 7fde7e13a640 1 --2- 172.21.15.100:0/2996383834 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d9634097a0 0x55d963413530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.770+0000 7fde7e13a640 1 -- 172.21.15.100:0/2996383834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d96340e910 msgr2=0x55d96341a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.770+0000 7fde7e13a640 1 --2- 172.21.15.100:0/2996383834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d96340e910 0x55d96341a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.770+0000 7fde7e13a640 1 -- 172.21.15.100:0/2996383834 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d96314b620 con 0x55d963408db0 2024-05-10T11:41:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.770+0000 7fde7e13a640 1 --2- 172.21.15.100:0/2996383834 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d963408db0 0x55d963412ff0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fde74014b70 tx=0x7fde7400bad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61ccfc3e2a56507c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:41:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.770+0000 7fde61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:41:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.770+0000 7fde61ffb640 1 -- 172.21.15.100:0/2996383834 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde74028660 con 0x55d963408db0 2024-05-10T11:41:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.770+0000 7fde8318b740 1 -- 172.21.15.100:0/2996383834 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde74009da0 con 0x55d963408db0 2024-05-10T11:41:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.771+0000 7fde61ffb640 1 -- 172.21.15.100:0/2996383834 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fde74028800 con 0x55d963408db0 2024-05-10T11:41:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.771+0000 7fde61ffb640 1 -- 172.21.15.100:0/2996383834 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fde74014bc0 con 0x55d963408db0 2024-05-10T11:41:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.771+0000 7fde8318b740 10 client.29105 fetch_fsmap learned FSMap version 41 2024-05-10T11:41:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.771+0000 7fde8318b740 1 -- 172.21.15.100:0/2996383834 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55d96341d5a0 con 0x55d963408db0 2024-05-10T11:41:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.771+0000 7fde61ffb640 1 -- 172.21.15.100:0/2996383834 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fde74009da0 con 0x55d963408db0 2024-05-10T11:41:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.771+0000 7fde61ffb640 1 -- 172.21.15.100:0/2996383834 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fde7402f070 con 0x55d963408db0 2024-05-10T11:41:21.775 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.771+0000 7fde8318b740 10 client.29105 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:41:21.775 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.771+0000 7fde8318b740 10 client.29105 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:41:21.775 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.771+0000 7fde8318b740 20 client.29105 populate_metadata read hostname 'smithi100' 2024-05-10T11:41:21.775 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.771+0000 7fde8318b740 1 --2- 172.21.15.100:0/2996383834 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d96341e9b0 0x55d963420da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:21.775 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.771+0000 7fde8318b740 4 client.29105 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:41:21.775 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.771+0000 7fde8318b740 1 -- 172.21.15.100:0/2996383834 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d96337b1e0 con 0x55d96341e9b0 2024-05-10T11:41:21.775 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.772+0000 7fde7d939640 1 --2- 172.21.15.100:0/2996383834 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d96341e9b0 0x55d963420da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:21.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.773+0000 7fde61ffb640 10 client.29105 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:41:21.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.773+0000 7fde7d939640 1 --2- 172.21.15.100:0/2996383834 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d96341e9b0 0x55d963420da0 crc :-1 s=READY pgs=431 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bedb5b536b26f199 server_cookie=bf95b86cfea9e837 in_seq=0 out_seq=0 2024-05-10T11:41:21.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.774+0000 7fde61ffb640 1 -- 172.21.15.100:0/2996383834 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d96337b1e0 con 0x55d96341e9b0 2024-05-10T11:41:21.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.774+0000 7fde61ffb640 10 client.29105 handle_command_reply: tid=0 2024-05-10T11:41:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.774+0000 7fde8318b740 1 client.29105 shutdown 2024-05-10T11:41:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.774+0000 7fde8318b740 1 -- 172.21.15.100:0/2996383834 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d963408db0 msgr2=0x55d963412ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.774+0000 7fde8318b740 1 --2- 172.21.15.100:0/2996383834 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d963408db0 0x55d963412ff0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fde74014b70 tx=0x7fde7400bad0 comp rx=0 tx=0).stop 2024-05-10T11:41:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.774+0000 7fde8318b740 1 -- 172.21.15.100:0/2996383834 shutdown_connections 2024-05-10T11:41:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.774+0000 7fde8318b740 1 --2- 172.21.15.100:0/2996383834 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d96341e9b0 0x55d963420da0 crc :-1 s=READY pgs=431 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.774+0000 7fde8318b740 1 --2- 172.21.15.100:0/2996383834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d96340e910 0x55d96341a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.774+0000 7fde8318b740 1 --2- 172.21.15.100:0/2996383834 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d9634097a0 0x55d963413530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.774+0000 7fde8318b740 1 --2- 172.21.15.100:0/2996383834 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d963408db0 0x55d963412ff0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.774+0000 7fde61ffb640 0 client.29105 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:41:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.774+0000 7fde8318b740 1 -- 172.21.15.100:0/2996383834 >> 172.21.15.100:0/2996383834 conn(0x55d963404420 msgr2=0x55d963403580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:41:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.775+0000 7fde8318b740 1 -- 172.21.15.100:0/2996383834 shutdown_connections 2024-05-10T11:41:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.775+0000 7fde8318b740 1 -- 172.21.15.100:0/2996383834 wait complete. 2024-05-10T11:41:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.775+0000 7fde8318b740 20 client.29105 trim_cache size 0 max 16384 2024-05-10T11:41:21.800 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.798+0000 7fde8318b740 1 Processor -- start 2024-05-10T11:41:21.800 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.798+0000 7fde8318b740 1 -- start start 2024-05-10T11:41:21.800 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.798+0000 7fde8318b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d963408db0 0x55d963403400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:21.800 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.798+0000 7fde8318b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d9634097a0 0x55d96341a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:21.800 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.798+0000 7fde8318b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d96340e910 0x55d963407000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:21.800 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.798+0000 7fde8318b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fde781ab8f0 con 0x55d96340e910 2024-05-10T11:41:21.800 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.798+0000 7fde8318b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fde781a91c0 con 0x55d963408db0 2024-05-10T11:41:21.800 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.798+0000 7fde8318b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fde781a7010 con 0x55d9634097a0 2024-05-10T11:41:21.800 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.799+0000 7fde7d939640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d9634097a0 0x55d96341a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:21.800 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.799+0000 7fde7d939640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d9634097a0 0x55d96341a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:56566/0 (socket says 172.21.15.100:56566) 2024-05-10T11:41:21.800 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.799+0000 7fde7e13a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d963408db0 0x55d963403400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:21.801 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.799+0000 7fde7d939640 1 -- 172.21.15.100:0/1583325069 learned_addr learned my addr 172.21.15.100:0/1583325069 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:41:21.801 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.799+0000 7fde7e93b640 1 --2- 172.21.15.100:0/1583325069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d96340e910 0x55d963407000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:21.801 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.799+0000 7fde7d939640 1 -- 172.21.15.100:0/1583325069 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d963408db0 msgr2=0x55d963403400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:21.801 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.799+0000 7fde7d939640 1 --2- 172.21.15.100:0/1583325069 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d963408db0 0x55d963403400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:21.801 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.799+0000 7fde7d939640 1 -- 172.21.15.100:0/1583325069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d96340e910 msgr2=0x55d963407000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:21.801 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.799+0000 7fde7d939640 1 --2- 172.21.15.100:0/1583325069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d96340e910 0x55d963407000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:21.801 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.799+0000 7fde7d939640 1 -- 172.21.15.100:0/1583325069 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d963409540 con 0x55d9634097a0 2024-05-10T11:41:21.801 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.799+0000 7fde7d939640 1 --2- 172.21.15.100:0/1583325069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d9634097a0 0x55d96341a490 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fde6800d550 tx=0x7fde680046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46ff13d50ce309fe server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:41:21.801 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.799+0000 7fde63fff640 1 -- 172.21.15.100:0/1583325069 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6801fca0 con 0x55d9634097a0 2024-05-10T11:41:21.801 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.799+0000 7fde63fff640 1 -- 172.21.15.100:0/1583325069 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fde68032030 con 0x55d9634097a0 2024-05-10T11:41:21.801 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.799+0000 7fde63fff640 1 -- 172.21.15.100:0/1583325069 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde680047a0 con 0x55d9634097a0 2024-05-10T11:41:21.804 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.800+0000 7fde8318b740 1 -- 172.21.15.100:0/1583325069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d9634097a0 msgr2=0x55d96341a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:21.805 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.800+0000 7fde8318b740 1 --2- 172.21.15.100:0/1583325069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d9634097a0 0x55d96341a490 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fde6800d550 tx=0x7fde680046c0 comp rx=0 tx=0).stop 2024-05-10T11:41:21.805 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.800+0000 7fde8318b740 1 -- 172.21.15.100:0/1583325069 shutdown_connections 2024-05-10T11:41:21.805 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.800+0000 7fde8318b740 1 --2- 172.21.15.100:0/1583325069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d96340e910 0x55d963407000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:21.805 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.800+0000 7fde8318b740 1 --2- 172.21.15.100:0/1583325069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d9634097a0 0x55d96341a490 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:21.805 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.800+0000 7fde8318b740 1 --2- 172.21.15.100:0/1583325069 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d963408db0 0x55d963403400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:21.805 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.800+0000 7fde8318b740 1 -- 172.21.15.100:0/1583325069 >> 172.21.15.100:0/1583325069 conn(0x55d963404420 msgr2=0x55d963402cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:41:21.805 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.801+0000 7fde8318b740 1 -- 172.21.15.100:0/1583325069 shutdown_connections 2024-05-10T11:41:21.805 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.801+0000 7fde8318b740 1 -- 172.21.15.100:0/1583325069 wait complete. 2024-05-10T11:41:21.805 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.801+0000 7fde8318b740 1 -- start start 2024-05-10T11:41:21.805 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.801+0000 7fde8318b740 1 Processor -- start 2024-05-10T11:41:21.805 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.801+0000 7fde8318b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d963408db0 0x55d963417140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:21.805 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.801+0000 7fde8318b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d9634097a0 0x55d963417680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:21.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.801+0000 7fde8318b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d96340e910 0x55d963487230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:21.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.801+0000 7fde8318b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fde781a7680 con 0x55d9634097a0 2024-05-10T11:41:21.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.801+0000 7fde8318b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fde781a8af0 con 0x55d96340e910 2024-05-10T11:41:21.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.801+0000 7fde8318b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fde781a7d50 con 0x55d963408db0 2024-05-10T11:41:21.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.801+0000 7fde8318b740 10 client.0 _ll_register_callbacks cb 0x7fde6800dd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:41:21.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.801+0000 7fde7d939640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d9634097a0 0x55d963417680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:21.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.801+0000 7fde7e13a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d963408db0 0x55d963417140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:21.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.801+0000 7fde7d939640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d9634097a0 0x55d963417680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47932/0 (socket says 172.21.15.100:47932) 2024-05-10T11:41:21.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.801+0000 7fde7d939640 1 -- 172.21.15.100:0/2888719834 learned_addr learned my addr 172.21.15.100:0/2888719834 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:41:21.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.801+0000 7fde7e93b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d96340e910 0x55d963487230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:21.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.802+0000 7fde7e13a640 1 -- 172.21.15.100:0/2888719834 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d96340e910 msgr2=0x55d963487230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:21.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.802+0000 7fde7e13a640 1 --2- 172.21.15.100:0/2888719834 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d96340e910 0x55d963487230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:21.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.802+0000 7fde7e13a640 1 -- 172.21.15.100:0/2888719834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d9634097a0 msgr2=0x55d963417680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:21.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.802+0000 7fde7e13a640 1 --2- 172.21.15.100:0/2888719834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d9634097a0 0x55d963417680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:21.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.802+0000 7fde7e13a640 1 -- 172.21.15.100:0/2888719834 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d96341a750 con 0x55d963408db0 2024-05-10T11:41:21.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.802+0000 7fde7e93b640 1 --2- 172.21.15.100:0/2888719834 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d96340e910 0x55d963487230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:41:21.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.802+0000 7fde7d939640 1 --2- 172.21.15.100:0/2888719834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d9634097a0 0x55d963417680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:41:21.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.802+0000 7fde7e13a640 1 --2- 172.21.15.100:0/2888719834 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d963408db0 0x55d963417140 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fde74004fb0 tx=0x7fde7400b740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7308ceac424fc19b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:41:21.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.802+0000 7fde61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:41:21.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.802+0000 7fde61ffb640 1 -- 172.21.15.100:0/2888719834 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde74022070 con 0x55d963408db0 2024-05-10T11:41:21.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.802+0000 7fde61ffb640 1 -- 172.21.15.100:0/2888719834 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fde74004850 con 0x55d963408db0 2024-05-10T11:41:21.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.802+0000 7fde8318b740 1 -- 172.21.15.100:0/2888719834 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde68032030 con 0x55d963408db0 2024-05-10T11:41:21.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.802+0000 7fde61ffb640 1 -- 172.21.15.100:0/2888719834 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fde74014bc0 con 0x55d963408db0 2024-05-10T11:41:21.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.802+0000 7fde8318b740 10 client.29227 fetch_fsmap learned FSMap version 41 2024-05-10T11:41:21.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.802+0000 7fde8318b740 1 -- 172.21.15.100:0/2888719834 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55d9634887b0 con 0x55d963408db0 2024-05-10T11:41:21.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.803+0000 7fde61ffb640 1 -- 172.21.15.100:0/2888719834 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fde68032030 con 0x55d963408db0 2024-05-10T11:41:21.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.803+0000 7fde61ffb640 1 -- 172.21.15.100:0/2888719834 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fde7403d060 con 0x55d963408db0 2024-05-10T11:41:21.808 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.803+0000 7fde8318b740 10 client.29227 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:41:21.808 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.803+0000 7fde8318b740 10 client.29227 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:41:21.808 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.803+0000 7fde8318b740 20 client.29227 populate_metadata read hostname 'smithi100' 2024-05-10T11:41:21.808 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.803+0000 7fde8318b740 1 --2- 172.21.15.100:0/2888719834 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d963489200 0x55d96348b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:21.808 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.803+0000 7fde8318b740 4 client.29227 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:41:21.808 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.803+0000 7fde8318b740 1 -- 172.21.15.100:0/2888719834 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d963375050 con 0x55d963489200 2024-05-10T11:41:21.808 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.803+0000 7fde7d939640 1 --2- 172.21.15.100:0/2888719834 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d963489200 0x55d96348b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:21.808 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.804+0000 7fde61ffb640 10 client.29227 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:41:21.808 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.804+0000 7fde7d939640 1 --2- 172.21.15.100:0/2888719834 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d963489200 0x55d96348b620 crc :-1 s=READY pgs=432 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8c22f95c67f54a9 server_cookie=875f78ddfc74e093 in_seq=0 out_seq=0 2024-05-10T11:41:21.906 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.905+0000 7fde61ffb640 1 -- 172.21.15.100:0/2888719834 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d963375050 con 0x55d963489200 2024-05-10T11:41:21.906 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.905+0000 7fde61ffb640 10 client.29227 handle_command_reply: tid=0 2024-05-10T11:41:21.906 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.905+0000 7fde8318b740 1 client.29227 shutdown 2024-05-10T11:41:21.906 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.905+0000 7fde8318b740 1 -- 172.21.15.100:0/2888719834 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d963408db0 msgr2=0x55d963417140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:21.907 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.906+0000 7fde8318b740 1 --2- 172.21.15.100:0/2888719834 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d963408db0 0x55d963417140 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fde74004fb0 tx=0x7fde7400b740 comp rx=0 tx=0).stop 2024-05-10T11:41:21.907 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.906+0000 7fde8318b740 1 -- 172.21.15.100:0/2888719834 shutdown_connections 2024-05-10T11:41:21.907 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.906+0000 7fde8318b740 1 --2- 172.21.15.100:0/2888719834 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d963489200 0x55d96348b620 crc :-1 s=READY pgs=432 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:21.907 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.906+0000 7fde8318b740 1 --2- 172.21.15.100:0/2888719834 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d96340e910 0x55d963487230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:21.907 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.906+0000 7fde8318b740 1 --2- 172.21.15.100:0/2888719834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d9634097a0 0x55d963417680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:21.907 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.906+0000 7fde61ffb640 0 client.29227 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:41:21.907 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.906+0000 7fde8318b740 1 --2- 172.21.15.100:0/2888719834 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d963408db0 0x55d963417140 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:21.907 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.906+0000 7fde8318b740 1 -- 172.21.15.100:0/2888719834 >> 172.21.15.100:0/2888719834 conn(0x55d963404420 msgr2=0x55d963402cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:41:21.907 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.906+0000 7fde8318b740 1 -- 172.21.15.100:0/2888719834 shutdown_connections 2024-05-10T11:41:21.907 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.906+0000 7fde8318b740 1 -- 172.21.15.100:0/2888719834 wait complete. 2024-05-10T11:41:21.907 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.906+0000 7fde8318b740 20 client.29227 trim_cache size 0 max 16384 2024-05-10T11:41:21.908 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:41:21.908 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:41:21.908 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:41:21.908 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:41:21.909 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.907+0000 7fde8039e640 1 -- 172.21.15.100:0/3229035594 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fde640b28a0 msgr2=0x7fde640b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:21.909 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.907+0000 7fde8039e640 1 --2- 172.21.15.100:0/3229035594 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fde640b28a0 0x7fde640b4d60 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fde6800dfd0 tx=0x7fde68005f90 comp rx=0 tx=0).stop 2024-05-10T11:41:21.909 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.907+0000 7fde8039e640 1 -- 172.21.15.100:0/3229035594 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde781034b0 msgr2=0x7fde781a21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:21.909 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.907+0000 7fde8039e640 1 --2- 172.21.15.100:0/3229035594 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde781034b0 0x7fde781a21d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fde70034040 tx=0x7fde70030e50 comp rx=0 tx=0).stop 2024-05-10T11:41:21.909 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.907+0000 7fde8039e640 1 -- 172.21.15.100:0/3229035594 shutdown_connections 2024-05-10T11:41:21.909 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.907+0000 7fde8039e640 1 --2- 172.21.15.100:0/3229035594 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fde640b28a0 0x7fde640b4d60 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:21.909 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.907+0000 7fde8039e640 1 --2- 172.21.15.100:0/3229035594 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde781034b0 0x7fde781a21d0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:21.909 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.907+0000 7fde8039e640 1 --2- 172.21.15.100:0/3229035594 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde78102af0 0x7fde7819b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:21.909 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.907+0000 7fde8039e640 1 --2- 172.21.15.100:0/3229035594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde78101900 0x7fde7819ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:21.909 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.907+0000 7fde8039e640 1 -- 172.21.15.100:0/3229035594 >> 172.21.15.100:0/3229035594 conn(0x7fde780fce70 msgr2=0x7fde78191890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:41:21.910 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.907+0000 7fde8039e640 1 -- 172.21.15.100:0/3229035594 shutdown_connections 2024-05-10T11:41:21.910 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:21.907+0000 7fde8039e640 1 -- 172.21.15.100:0/3229035594 wait complete. 2024-05-10T11:41:21.955 INFO:tasks.workunit.client.0.smithi100.stdout: 10 92442 8.94 MB/sec execute 435 sec latency 125.948 ms 2024-05-10T11:41:22.145 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:41:21 smithi100 ceph-mon[30296]: pgmap v1110: 129 pgs: 129 active+clean; 554 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 4.2 MiB/s wr, 331 op/s 2024-05-10T11:41:22.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:41:21 smithi178 ceph-mon[33359]: pgmap v1110: 129 pgs: 129 active+clean; 554 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 4.2 MiB/s wr, 331 op/s 2024-05-10T11:41:22.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:41:21 smithi188 ceph-mon[34075]: pgmap v1110: 129 pgs: 129 active+clean; 554 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 4.2 MiB/s wr, 331 op/s 2024-05-10T11:41:22.955 INFO:tasks.workunit.client.0.smithi100.stdout: 10 92602 8.93 MB/sec execute 436 sec latency 386.893 ms 2024-05-10T11:41:23.955 INFO:tasks.workunit.client.0.smithi100.stdout: 10 92726 8.93 MB/sec execute 437 sec latency 796.057 ms 2024-05-10T11:41:24.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:41:23 smithi178 ceph-mon[33359]: pgmap v1111: 129 pgs: 129 active+clean; 556 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 3.7 MiB/s wr, 282 op/s 2024-05-10T11:41:24.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:41:23 smithi188 ceph-mon[34075]: pgmap v1111: 129 pgs: 129 active+clean; 556 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 3.7 MiB/s wr, 282 op/s 2024-05-10T11:41:24.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:41:23 smithi100 ceph-mon[30296]: pgmap v1111: 129 pgs: 129 active+clean; 556 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 3.7 MiB/s wr, 282 op/s 2024-05-10T11:41:24.958 INFO:tasks.workunit.client.0.smithi100.stdout: 10 92902 8.94 MB/sec execute 438 sec latency 101.008 ms 2024-05-10T11:41:25.955 INFO:tasks.workunit.client.0.smithi100.stdout: 10 93115 8.96 MB/sec execute 439 sec latency 67.303 ms 2024-05-10T11:41:26.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:41:25 smithi178 ceph-mon[33359]: pgmap v1112: 129 pgs: 129 active+clean; 561 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 4.2 MiB/s wr, 292 op/s 2024-05-10T11:41:26.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:41:25 smithi188 ceph-mon[34075]: pgmap v1112: 129 pgs: 129 active+clean; 561 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 4.2 MiB/s wr, 292 op/s 2024-05-10T11:41:26.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:41:25 smithi100 ceph-mon[30296]: pgmap v1112: 129 pgs: 129 active+clean; 561 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 4.2 MiB/s wr, 292 op/s 2024-05-10T11:41:26.955 INFO:tasks.workunit.client.0.smithi100.stdout: 10 93282 8.95 MB/sec execute 440 sec latency 67.203 ms 2024-05-10T11:41:27.955 INFO:tasks.workunit.client.0.smithi100.stdout: 10 93408 8.94 MB/sec execute 441 sec latency 68.759 ms 2024-05-10T11:41:28.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:41:27 smithi178 ceph-mon[33359]: pgmap v1113: 129 pgs: 129 active+clean; 568 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 923 KiB/s rd, 4.6 MiB/s wr, 315 op/s 2024-05-10T11:41:28.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:41:27 smithi188 ceph-mon[34075]: pgmap v1113: 129 pgs: 129 active+clean; 568 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 923 KiB/s rd, 4.6 MiB/s wr, 315 op/s 2024-05-10T11:41:28.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:41:27 smithi100 ceph-mon[30296]: pgmap v1113: 129 pgs: 129 active+clean; 568 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 923 KiB/s rd, 4.6 MiB/s wr, 315 op/s 2024-05-10T11:41:28.955 INFO:tasks.workunit.client.0.smithi100.stdout: 10 93570 8.94 MB/sec execute 442 sec latency 73.109 ms 2024-05-10T11:41:29.955 INFO:tasks.workunit.client.0.smithi100.stdout: 10 93731 8.94 MB/sec execute 443 sec latency 108.132 ms 2024-05-10T11:41:30.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:41:29 smithi178 ceph-mon[33359]: pgmap v1114: 129 pgs: 129 active+clean; 569 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 4.5 MiB/s wr, 269 op/s 2024-05-10T11:41:30.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:41:29 smithi188 ceph-mon[34075]: pgmap v1114: 129 pgs: 129 active+clean; 569 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 4.5 MiB/s wr, 269 op/s 2024-05-10T11:41:30.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:41:29 smithi100 ceph-mon[30296]: pgmap v1114: 129 pgs: 129 active+clean; 569 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 4.5 MiB/s wr, 269 op/s 2024-05-10T11:41:30.957 INFO:tasks.workunit.client.0.smithi100.stdout: 10 93898 8.94 MB/sec execute 444 sec latency 53.397 ms 2024-05-10T11:41:31.956 INFO:tasks.workunit.client.0.smithi100.stdout: 10 94056 8.94 MB/sec execute 445 sec latency 208.557 ms 2024-05-10T11:41:32.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:41:31 smithi178 ceph-mon[33359]: pgmap v1115: 129 pgs: 129 active+clean; 570 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 4.6 MiB/s wr, 270 op/s 2024-05-10T11:41:32.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:41:31 smithi188 ceph-mon[34075]: pgmap v1115: 129 pgs: 129 active+clean; 570 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 4.6 MiB/s wr, 270 op/s 2024-05-10T11:41:32.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:41:31 smithi100 ceph-mon[30296]: pgmap v1115: 129 pgs: 129 active+clean; 570 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 4.6 MiB/s wr, 270 op/s 2024-05-10T11:41:32.956 INFO:tasks.workunit.client.0.smithi100.stdout: 10 94230 8.94 MB/sec execute 446 sec latency 89.492 ms 2024-05-10T11:41:33.956 INFO:tasks.workunit.client.0.smithi100.stdout: 10 94376 8.93 MB/sec execute 447 sec latency 104.674 ms 2024-05-10T11:41:34.082 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T11:41:34.115 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:41:33 smithi100 ceph-mon[30296]: pgmap v1116: 129 pgs: 129 active+clean; 572 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 4.7 MiB/s wr, 271 op/s 2024-05-10T11:41:34.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:41:33 smithi178 ceph-mon[33359]: pgmap v1116: 129 pgs: 129 active+clean; 572 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 4.7 MiB/s wr, 271 op/s 2024-05-10T11:41:34.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:41:33 smithi188 ceph-mon[34075]: pgmap v1116: 129 pgs: 129 active+clean; 572 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 4.7 MiB/s wr, 271 op/s 2024-05-10T11:41:34.637 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:41:34.956 INFO:tasks.workunit.client.0.smithi100.stdout: 10 94517 8.92 MB/sec execute 448 sec latency 93.100 ms 2024-05-10T11:41:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.038+0000 7f4127647640 1 -- 172.21.15.100:0/2272803528 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4120101930 msgr2=0x7f4120101d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.038+0000 7f4127647640 1 --2- 172.21.15.100:0/2272803528 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4120101930 0x7f4120101d50 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f411c009900 tx=0x7f411c02a410 comp rx=0 tx=0).stop 2024-05-10T11:41:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.038+0000 7f4127647640 1 -- 172.21.15.100:0/2272803528 shutdown_connections 2024-05-10T11:41:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.038+0000 7f4127647640 1 --2- 172.21.15.100:0/2272803528 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41201034e0 0x7f4120109d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.038+0000 7f4127647640 1 --2- 172.21.15.100:0/2272803528 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4120102b20 0x7f4120102fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:35.040 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.038+0000 7f4127647640 1 --2- 172.21.15.100:0/2272803528 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4120101930 0x7f4120101d50 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:35.040 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.038+0000 7f4127647640 1 -- 172.21.15.100:0/2272803528 >> 172.21.15.100:0/2272803528 conn(0x7f41200fcec0 msgr2=0x7f41200ff2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:41:35.040 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.039+0000 7f4127647640 1 -- 172.21.15.100:0/2272803528 shutdown_connections 2024-05-10T11:41:35.040 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.039+0000 7f4127647640 1 -- 172.21.15.100:0/2272803528 wait complete. 2024-05-10T11:41:35.040 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.039+0000 7f4127647640 1 Processor -- start 2024-05-10T11:41:35.040 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.039+0000 7f4127647640 1 -- start start 2024-05-10T11:41:35.041 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.040+0000 7f4127647640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4120101930 0x7f4120198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:35.042 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.040+0000 7f4127647640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4120102b20 0x7f4120199020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:35.042 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.040+0000 7f4127647640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41201034e0 0x7f41201a0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:35.042 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.040+0000 7f4127647640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f412010d310 con 0x7f41201034e0 2024-05-10T11:41:35.042 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.040+0000 7f4127647640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f412010d190 con 0x7f4120102b20 2024-05-10T11:41:35.042 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.040+0000 7f4127647640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f412010d490 con 0x7f4120101930 2024-05-10T11:41:35.042 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.040+0000 7f41253e3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4120101930 0x7f4120198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:35.042 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.040+0000 7f41253e3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4120101930 0x7f4120198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:36006/0 (socket says 172.21.15.100:36006) 2024-05-10T11:41:35.042 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.040+0000 7f41253e3640 1 -- 172.21.15.100:0/3803646598 learned_addr learned my addr 172.21.15.100:0/3803646598 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:41:35.042 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.040+0000 7f4125be4640 1 --2- 172.21.15.100:0/3803646598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41201034e0 0x7f41201a0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:35.042 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.040+0000 7f4124be2640 1 --2- 172.21.15.100:0/3803646598 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4120102b20 0x7f4120199020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:35.042 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.040+0000 7f41253e3640 1 -- 172.21.15.100:0/3803646598 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4120102b20 msgr2=0x7f4120199020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:35.042 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.040+0000 7f41253e3640 1 --2- 172.21.15.100:0/3803646598 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4120102b20 0x7f4120199020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:35.043 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.040+0000 7f41253e3640 1 -- 172.21.15.100:0/3803646598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41201034e0 msgr2=0x7f41201a0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:35.043 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.040+0000 7f41253e3640 1 --2- 172.21.15.100:0/3803646598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41201034e0 0x7f41201a0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:35.043 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.040+0000 7f41253e3640 1 -- 172.21.15.100:0/3803646598 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41201a0700 con 0x7f4120101930 2024-05-10T11:41:35.043 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.040+0000 7f4125be4640 1 --2- 172.21.15.100:0/3803646598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41201034e0 0x7f41201a0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:41:35.043 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.040+0000 7f41253e3640 1 --2- 172.21.15.100:0/3803646598 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4120101930 0x7f4120198ae0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f411c005ab0 tx=0x7f411c0042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:41:35.043 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.041+0000 7f41167fc640 1 -- 172.21.15.100:0/3803646598 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f411c03f030 con 0x7f4120101930 2024-05-10T11:41:35.043 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.041+0000 7f41167fc640 1 -- 172.21.15.100:0/3803646598 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f411c032e10 con 0x7f4120101930 2024-05-10T11:41:35.043 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.041+0000 7f41167fc640 1 -- 172.21.15.100:0/3803646598 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f411c035070 con 0x7f4120101930 2024-05-10T11:41:35.043 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.041+0000 7f4127647640 1 -- 172.21.15.100:0/3803646598 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41201a0990 con 0x7f4120101930 2024-05-10T11:41:35.043 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.041+0000 7f4127647640 1 -- 172.21.15.100:0/3803646598 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41201a0e30 con 0x7f4120101930 2024-05-10T11:41:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.045+0000 7f41167fc640 1 -- 172.21.15.100:0/3803646598 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f411c042030 con 0x7f4120101930 2024-05-10T11:41:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.045+0000 7f412a363740 1 Processor -- start 2024-05-10T11:41:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.045+0000 7f412a363740 1 -- start start 2024-05-10T11:41:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.045+0000 7f412a363740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c932cfa1c0 0x55c932cfa5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.045+0000 7f412a363740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c932cfabb0 0x55c932cff6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:35.046 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.045+0000 7f412a363740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c932cffd20 0x55c932d021b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:35.046 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.045+0000 7f412a363740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55c932c5a5d0 con 0x55c932cfabb0 2024-05-10T11:41:35.046 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.045+0000 7f412a363740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c932c13890 con 0x55c932cffd20 2024-05-10T11:41:35.046 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.045+0000 7f412a363740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c932c56590 con 0x55c932cfa1c0 2024-05-10T11:41:35.046 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.045+0000 7f4124be2640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c932cffd20 0x55c932d021b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:35.046 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.045+0000 7f4124be2640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c932cffd20 0x55c932d021b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:38848/0 (socket says 172.21.15.100:38848) 2024-05-10T11:41:35.046 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.045+0000 7f4124be2640 1 -- 172.21.15.100:0/2343501712 learned_addr learned my addr 172.21.15.100:0/2343501712 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:41:35.046 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.045+0000 7f41253e3640 1 --2- 172.21.15.100:0/2343501712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c932cfabb0 0x55c932cff6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:35.046 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.045+0000 7f4125be4640 1 --2- 172.21.15.100:0/2343501712 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c932cfa1c0 0x55c932cfa5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:35.047 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.046+0000 7f4124be2640 1 -- 172.21.15.100:0/2343501712 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c932cfa1c0 msgr2=0x55c932cfa5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:35.047 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.046+0000 7f4124be2640 1 --2- 172.21.15.100:0/2343501712 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c932cfa1c0 0x55c932cfa5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:35.047 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.046+0000 7f4124be2640 1 -- 172.21.15.100:0/2343501712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c932cfabb0 msgr2=0x55c932cff6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:35.047 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.046+0000 7f4124be2640 1 --2- 172.21.15.100:0/2343501712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c932cfabb0 0x55c932cff6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:35.047 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.046+0000 7f4124be2640 1 -- 172.21.15.100:0/2343501712 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c93296c210 con 0x55c932cffd20 2024-05-10T11:41:35.047 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.046+0000 7f41167fc640 1 --2- 172.21.15.100:0/3803646598 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f410c0b29a0 0x7f410c0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:35.047 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.046+0000 7f4124be2640 1 --2- 172.21.15.100:0/2343501712 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c932cffd20 0x55c932d021b0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4110009c90 tx=0x7f4110011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf29015ebd90d3bc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:41:35.047 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.046+0000 7f4125be4640 1 --2- 172.21.15.100:0/3803646598 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f410c0b29a0 0x7f410c0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:35.047 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.046+0000 7f41167fc640 1 -- 172.21.15.100:0/3803646598 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f411c102970 con 0x7f4120101930 2024-05-10T11:41:35.049 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.047+0000 7f410b7fe640 1 -- 172.21.15.100:0/2343501712 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f411001e070 con 0x55c932cffd20 2024-05-10T11:41:35.049 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.047+0000 7f410b7fe640 1 -- 172.21.15.100:0/2343501712 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4110019ac0 con 0x55c932cffd20 2024-05-10T11:41:35.049 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.047+0000 7f410b7fe640 1 -- 172.21.15.100:0/2343501712 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f411000e930 con 0x55c932cffd20 2024-05-10T11:41:35.053 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.047+0000 7f412a363740 1 -- 172.21.15.100:0/2343501712 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c932cffd20 msgr2=0x55c932d021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:35.053 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.047+0000 7f4125be4640 1 --2- 172.21.15.100:0/3803646598 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f410c0b29a0 0x7f410c0b4e60 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x55c932cfb410 tx=0x7f4118009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:41:35.053 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.047+0000 7f412a363740 1 --2- 172.21.15.100:0/2343501712 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c932cffd20 0x55c932d021b0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4110009c90 tx=0x7f4110011810 comp rx=0 tx=0).stop 2024-05-10T11:41:35.053 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.047+0000 7f412a363740 1 -- 172.21.15.100:0/2343501712 shutdown_connections 2024-05-10T11:41:35.053 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.047+0000 7f412a363740 1 --2- 172.21.15.100:0/2343501712 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c932cffd20 0x55c932d021b0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:35.053 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.047+0000 7f412a363740 1 --2- 172.21.15.100:0/2343501712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c932cfabb0 0x55c932cff6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:35.053 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.047+0000 7f412a363740 1 --2- 172.21.15.100:0/2343501712 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c932cfa1c0 0x55c932cfa5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:35.054 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.047+0000 7f412a363740 1 -- 172.21.15.100:0/2343501712 >> 172.21.15.100:0/2343501712 conn(0x55c932cf5830 msgr2=0x55c932cf5c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:41:35.054 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.048+0000 7f412a363740 1 -- 172.21.15.100:0/2343501712 shutdown_connections 2024-05-10T11:41:35.054 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.048+0000 7f412a363740 1 -- 172.21.15.100:0/2343501712 wait complete. 2024-05-10T11:41:35.054 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.048+0000 7f412a363740 1 -- start start 2024-05-10T11:41:35.054 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.048+0000 7f412a363740 1 Processor -- start 2024-05-10T11:41:35.054 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.049+0000 7f412a363740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c932cfa1c0 0x55c932d04400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:35.054 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.049+0000 7f412a363740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c932cfabb0 0x55c932d04940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:35.054 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.049+0000 7f412a363740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c932cffd20 0x55c932d0b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:35.054 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.049+0000 7f412a363740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55c932d02cd0 con 0x55c932cffd20 2024-05-10T11:41:35.054 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.049+0000 7f412a363740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c932c17370 con 0x55c932cfa1c0 2024-05-10T11:41:35.054 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.049+0000 7f412a363740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c932d02e50 con 0x55c932cfabb0 2024-05-10T11:41:35.054 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.049+0000 7f412a363740 10 client.0 _ll_register_callbacks cb 0x55c93296ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:41:35.055 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.049+0000 7f4124be2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c932cfabb0 0x55c932d04940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:35.055 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.049+0000 7f41253e3640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c932cfa1c0 0x55c932d04400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:35.055 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.049+0000 7f4124be2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c932cfabb0 0x55c932d04940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:36028/0 (socket says 172.21.15.100:36028) 2024-05-10T11:41:35.055 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.049+0000 7f4124be2640 1 -- 172.21.15.100:0/2951989662 learned_addr learned my addr 172.21.15.100:0/2951989662 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:41:35.055 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.049+0000 7f4125be4640 1 --2- 172.21.15.100:0/2951989662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c932cffd20 0x55c932d0b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:35.055 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.049+0000 7f4124be2640 1 -- 172.21.15.100:0/2951989662 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c932cfa1c0 msgr2=0x55c932d04400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:35.055 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.049+0000 7f4124be2640 1 --2- 172.21.15.100:0/2951989662 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c932cfa1c0 0x55c932d04400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:35.055 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.049+0000 7f4124be2640 1 -- 172.21.15.100:0/2951989662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c932cffd20 msgr2=0x55c932d0b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:35.055 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.049+0000 7f4124be2640 1 --2- 172.21.15.100:0/2951989662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c932cffd20 0x55c932d0b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:35.055 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.049+0000 7f4124be2640 1 -- 172.21.15.100:0/2951989662 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c932a38620 con 0x55c932cfabb0 2024-05-10T11:41:35.055 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.049+0000 7f41253e3640 1 --2- 172.21.15.100:0/2951989662 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c932cfa1c0 0x55c932d04400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:41:35.055 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.049+0000 7f4124be2640 1 --2- 172.21.15.100:0/2951989662 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c932cfabb0 0x55c932d04940 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f411001afd0 tx=0x7f4110019660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4964984c31043bd9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:41:35.055 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.049+0000 7f4108ff9640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:41:35.055 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.050+0000 7f4108ff9640 1 -- 172.21.15.100:0/2951989662 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4110029030 con 0x55c932cfabb0 2024-05-10T11:41:35.055 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.050+0000 7f412a363740 1 -- 172.21.15.100:0/2951989662 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4110019ac0 con 0x55c932cfabb0 2024-05-10T11:41:35.056 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.050+0000 7f4108ff9640 1 -- 172.21.15.100:0/2951989662 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f4110019880 con 0x55c932cfabb0 2024-05-10T11:41:35.056 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.050+0000 7f4108ff9640 1 -- 172.21.15.100:0/2951989662 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f4110033050 con 0x55c932cfabb0 2024-05-10T11:41:35.056 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.050+0000 7f412a363740 10 client.29239 fetch_fsmap learned FSMap version 41 2024-05-10T11:41:35.056 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.050+0000 7f412a363740 1 -- 172.21.15.100:0/2951989662 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55c932d0e9b0 con 0x55c932cfabb0 2024-05-10T11:41:35.056 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.050+0000 7f4108ff9640 1 -- 172.21.15.100:0/2951989662 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4110019ac0 con 0x55c932cfabb0 2024-05-10T11:41:35.056 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.050+0000 7f4108ff9640 1 -- 172.21.15.100:0/2951989662 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f41100048f0 con 0x55c932cfabb0 2024-05-10T11:41:35.056 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.050+0000 7f412a363740 10 client.29239 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:41:35.056 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.050+0000 7f412a363740 10 client.29239 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:41:35.056 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.050+0000 7f412a363740 20 client.29239 populate_metadata read hostname 'smithi100' 2024-05-10T11:41:35.056 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.050+0000 7f412a363740 1 --2- 172.21.15.100:0/2951989662 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c932d0fe00 0x55c932d12230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:35.056 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.050+0000 7f412a363740 4 client.29239 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:41:35.056 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.050+0000 7f412a363740 1 -- 172.21.15.100:0/2951989662 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c932c681e0 con 0x55c932d0fe00 2024-05-10T11:41:35.056 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.050+0000 7f41253e3640 1 --2- 172.21.15.100:0/2951989662 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c932d0fe00 0x55c932d12230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:35.056 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.051+0000 7f4108ff9640 10 client.29239 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:41:35.057 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.051+0000 7f41253e3640 1 --2- 172.21.15.100:0/2951989662 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c932d0fe00 0x55c932d12230 crc :-1 s=READY pgs=433 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f017511b5567bd9 server_cookie=4a1551be52ff2d9 in_seq=0 out_seq=0 2024-05-10T11:41:35.057 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.052+0000 7f4108ff9640 1 -- 172.21.15.100:0/2951989662 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c932c681e0 con 0x55c932d0fe00 2024-05-10T11:41:35.057 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.052+0000 7f4108ff9640 10 client.29239 handle_command_reply: tid=0 2024-05-10T11:41:35.057 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.052+0000 7f412a363740 1 client.29239 shutdown 2024-05-10T11:41:35.057 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.052+0000 7f412a363740 1 -- 172.21.15.100:0/2951989662 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c932cfabb0 msgr2=0x55c932d04940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:35.057 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.052+0000 7f412a363740 1 --2- 172.21.15.100:0/2951989662 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c932cfabb0 0x55c932d04940 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f411001afd0 tx=0x7f4110019660 comp rx=0 tx=0).stop 2024-05-10T11:41:35.057 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.052+0000 7f412a363740 1 -- 172.21.15.100:0/2951989662 shutdown_connections 2024-05-10T11:41:35.057 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.052+0000 7f412a363740 1 --2- 172.21.15.100:0/2951989662 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c932d0fe00 0x55c932d12230 crc :-1 s=READY pgs=433 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:35.057 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.053+0000 7f412a363740 1 --2- 172.21.15.100:0/2951989662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c932cffd20 0x55c932d0b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:35.057 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.053+0000 7f412a363740 1 --2- 172.21.15.100:0/2951989662 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c932cfabb0 0x55c932d04940 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:35.057 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.053+0000 7f412a363740 1 --2- 172.21.15.100:0/2951989662 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c932cfa1c0 0x55c932d04400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:35.057 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.053+0000 7f412a363740 1 -- 172.21.15.100:0/2951989662 >> 172.21.15.100:0/2951989662 conn(0x55c932cf5830 msgr2=0x55c932cf0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:41:35.057 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.053+0000 7f4108ff9640 0 client.29239 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:41:35.057 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.053+0000 7f412a363740 1 -- 172.21.15.100:0/2951989662 shutdown_connections 2024-05-10T11:41:35.058 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.053+0000 7f412a363740 1 -- 172.21.15.100:0/2951989662 wait complete. 2024-05-10T11:41:35.058 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.053+0000 7f412a363740 20 client.29239 trim_cache size 0 max 16384 2024-05-10T11:41:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.082+0000 7f412a363740 1 Processor -- start 2024-05-10T11:41:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.082+0000 7f412a363740 1 -- start start 2024-05-10T11:41:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.082+0000 7f412a363740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c932cfa1c0 0x55c932d031f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.082+0000 7f412a363740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c932cfabb0 0x55c932d0b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.083+0000 7f41253e3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c932cfa1c0 0x55c932d031f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.083+0000 7f41253e3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c932cfa1c0 0x55c932d031f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:36030/0 (socket says 172.21.15.100:36030) 2024-05-10T11:41:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.084+0000 7f41253e3640 1 -- 172.21.15.100:0/2492598432 learned_addr learned my addr 172.21.15.100:0/2492598432 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:41:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.084+0000 7f412a363740 1 --2- 172.21.15.100:0/2492598432 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c932cffd20 0x55c932cec3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.084+0000 7f412a363740 1 -- 172.21.15.100:0/2492598432 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f41201ab9a0 con 0x55c932cffd20 2024-05-10T11:41:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.084+0000 7f4125be4640 1 --2- 172.21.15.100:0/2492598432 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c932cffd20 0x55c932cec3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.084+0000 7f412a363740 1 -- 172.21.15.100:0/2492598432 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f41201a9270 con 0x55c932cfabb0 2024-05-10T11:41:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.084+0000 7f412a363740 1 -- 172.21.15.100:0/2492598432 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f41201a70c0 con 0x55c932cfa1c0 2024-05-10T11:41:35.085 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.085+0000 7f4125be4640 1 -- 172.21.15.100:0/2492598432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c932cfa1c0 msgr2=0x55c932d031f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:35.085 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.085+0000 7f4125be4640 1 --2- 172.21.15.100:0/2492598432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c932cfa1c0 0x55c932d031f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:35.085 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.085+0000 7f4125be4640 1 -- 172.21.15.100:0/2492598432 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c932cfabb0 msgr2=0x55c932d0b940 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T11:41:35.085 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.085+0000 7f4124be2640 1 --2- 172.21.15.100:0/2492598432 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c932cfabb0 0x55c932d0b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.085+0000 7f4125be4640 1 --2- 172.21.15.100:0/2492598432 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c932cfabb0 0x55c932d0b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.085+0000 7f4125be4640 1 -- 172.21.15.100:0/2492598432 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c932cf5660 con 0x55c932cffd20 2024-05-10T11:41:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.085+0000 7f4124be2640 1 --2- 172.21.15.100:0/2492598432 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c932cfabb0 0x55c932d0b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:41:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.085+0000 7f4125be4640 1 --2- 172.21.15.100:0/2492598432 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c932cffd20 0x55c932cec3e0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f4118005fd0 tx=0x7f411801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=130136801d3d9625 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:41:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.086+0000 7f410affd640 1 -- 172.21.15.100:0/2492598432 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4118003d10 con 0x55c932cffd20 2024-05-10T11:41:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.086+0000 7f410affd640 1 -- 172.21.15.100:0/2492598432 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f41180045c0 con 0x55c932cffd20 2024-05-10T11:41:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.086+0000 7f410affd640 1 -- 172.21.15.100:0/2492598432 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f411801f430 con 0x55c932cffd20 2024-05-10T11:41:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.087+0000 7f412a363740 1 -- 172.21.15.100:0/2492598432 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c932cffd20 msgr2=0x55c932cec3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.087+0000 7f412a363740 1 --2- 172.21.15.100:0/2492598432 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c932cffd20 0x55c932cec3e0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f4118005fd0 tx=0x7f411801b000 comp rx=0 tx=0).stop 2024-05-10T11:41:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.087+0000 7f412a363740 1 -- 172.21.15.100:0/2492598432 shutdown_connections 2024-05-10T11:41:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.087+0000 7f412a363740 1 --2- 172.21.15.100:0/2492598432 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c932cffd20 0x55c932cec3e0 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.087+0000 7f412a363740 1 --2- 172.21.15.100:0/2492598432 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c932cfabb0 0x55c932d0b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.087+0000 7f412a363740 1 --2- 172.21.15.100:0/2492598432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c932cfa1c0 0x55c932d031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.087+0000 7f412a363740 1 -- 172.21.15.100:0/2492598432 >> 172.21.15.100:0/2492598432 conn(0x55c932cf5830 msgr2=0x55c932cf5c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:41:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.087+0000 7f412a363740 1 -- 172.21.15.100:0/2492598432 shutdown_connections 2024-05-10T11:41:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.088+0000 7f412a363740 1 -- 172.21.15.100:0/2492598432 wait complete. 2024-05-10T11:41:35.091 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.088+0000 7f412a363740 1 -- start start 2024-05-10T11:41:35.091 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.088+0000 7f412a363740 1 Processor -- start 2024-05-10T11:41:35.091 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.089+0000 7f412a363740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c932cfa1c0 0x55c932d08590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:35.091 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.089+0000 7f412a363740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c932cfabb0 0x55c932d08ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:35.091 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.089+0000 7f412a363740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c932cffd20 0x55c932d786e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.089+0000 7f412a363740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f41201a7730 con 0x55c932cfabb0 2024-05-10T11:41:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.089+0000 7f412a363740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f41201a8ba0 con 0x55c932cfa1c0 2024-05-10T11:41:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.089+0000 7f412a363740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f41201a7e00 con 0x55c932cffd20 2024-05-10T11:41:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.089+0000 7f412a363740 10 client.0 _ll_register_callbacks cb 0x7f411001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:41:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.089+0000 7f4124be2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c932cfabb0 0x55c932d08ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.089+0000 7f4124be2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c932cfabb0 0x55c932d08ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49414/0 (socket says 172.21.15.100:49414) 2024-05-10T11:41:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.089+0000 7f4124be2640 1 -- 172.21.15.100:0/2152929831 learned_addr learned my addr 172.21.15.100:0/2152929831 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:41:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.089+0000 7f4125be4640 1 --2- 172.21.15.100:0/2152929831 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c932cffd20 0x55c932d786e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.089+0000 7f41253e3640 1 --2- 172.21.15.100:0/2152929831 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c932cfa1c0 0x55c932d08590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.089+0000 7f4124be2640 1 -- 172.21.15.100:0/2152929831 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c932cffd20 msgr2=0x55c932d786e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.089+0000 7f4124be2640 1 --2- 172.21.15.100:0/2152929831 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c932cffd20 0x55c932d786e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.089+0000 7f4124be2640 1 -- 172.21.15.100:0/2152929831 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c932cfa1c0 msgr2=0x55c932d08590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.089+0000 7f4124be2640 1 --2- 172.21.15.100:0/2152929831 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c932cfa1c0 0x55c932d08590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.089+0000 7f4124be2640 1 -- 172.21.15.100:0/2152929831 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c932d0bba0 con 0x55c932cfabb0 2024-05-10T11:41:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.089+0000 7f41253e3640 1 --2- 172.21.15.100:0/2152929831 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c932cfa1c0 0x55c932d08590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:41:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.090+0000 7f4124be2640 1 --2- 172.21.15.100:0/2152929831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c932cfabb0 0x55c932d08ad0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f41100058c0 tx=0x7f4110005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ec2c56a4f5353f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:41:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.090+0000 7f4108ff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:41:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.090+0000 7f4108ff9640 1 -- 172.21.15.100:0/2152929831 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4110015020 con 0x55c932cfabb0 2024-05-10T11:41:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.090+0000 7f4108ff9640 1 -- 172.21.15.100:0/2152929831 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f411000e620 con 0x55c932cfabb0 2024-05-10T11:41:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.090+0000 7f412a363740 1 -- 172.21.15.100:0/2152929831 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41180045c0 con 0x55c932cfabb0 2024-05-10T11:41:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.090+0000 7f4108ff9640 1 -- 172.21.15.100:0/2152929831 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f411003e050 con 0x55c932cfabb0 2024-05-10T11:41:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.090+0000 7f412a363740 10 client.19875 fetch_fsmap learned FSMap version 41 2024-05-10T11:41:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.090+0000 7f412a363740 1 -- 172.21.15.100:0/2152929831 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55c932d79c10 con 0x55c932cfabb0 2024-05-10T11:41:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.090+0000 7f4108ff9640 1 -- 172.21.15.100:0/2152929831 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f41180045c0 con 0x55c932cfabb0 2024-05-10T11:41:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.090+0000 7f4108ff9640 1 -- 172.21.15.100:0/2152929831 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f41100053e0 con 0x55c932cfabb0 2024-05-10T11:41:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.090+0000 7f412a363740 10 client.19875 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:41:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.090+0000 7f412a363740 10 client.19875 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:41:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.090+0000 7f412a363740 20 client.19875 populate_metadata read hostname 'smithi100' 2024-05-10T11:41:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.090+0000 7f412a363740 1 --2- 172.21.15.100:0/2152929831 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c932d7a610 0x55c932d7ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.091+0000 7f412a363740 4 client.19875 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:41:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.091+0000 7f412a363740 1 -- 172.21.15.100:0/2152929831 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c932c62050 con 0x55c932d7a610 2024-05-10T11:41:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.091+0000 7f41253e3640 1 --2- 172.21.15.100:0/2152929831 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c932d7a610 0x55c932d7ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.091+0000 7f4108ff9640 10 client.19875 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:41:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.091+0000 7f41253e3640 1 --2- 172.21.15.100:0/2152929831 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c932d7a610 0x55c932d7ca30 crc :-1 s=READY pgs=434 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31bcec840da4a079 server_cookie=6c73997600f37599 in_seq=0 out_seq=0 2024-05-10T11:41:35.203 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.203+0000 7f4108ff9640 1 -- 172.21.15.100:0/2152929831 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c932c62050 con 0x55c932d7a610 2024-05-10T11:41:35.203 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.203+0000 7f4108ff9640 10 client.19875 handle_command_reply: tid=0 2024-05-10T11:41:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.203+0000 7f412a363740 1 client.19875 shutdown 2024-05-10T11:41:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.203+0000 7f412a363740 1 -- 172.21.15.100:0/2152929831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c932cfabb0 msgr2=0x55c932d08ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.203+0000 7f412a363740 1 --2- 172.21.15.100:0/2152929831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c932cfabb0 0x55c932d08ad0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f41100058c0 tx=0x7f4110005dc0 comp rx=0 tx=0).stop 2024-05-10T11:41:35.205 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.204+0000 7f412a363740 1 -- 172.21.15.100:0/2152929831 shutdown_connections 2024-05-10T11:41:35.205 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.204+0000 7f412a363740 1 --2- 172.21.15.100:0/2152929831 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c932d7a610 0x55c932d7ca30 crc :-1 s=READY pgs=434 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:35.205 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.204+0000 7f412a363740 1 --2- 172.21.15.100:0/2152929831 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c932cffd20 0x55c932d786e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:35.205 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.204+0000 7f4108ff9640 0 client.19875 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:41:35.205 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.204+0000 7f412a363740 1 --2- 172.21.15.100:0/2152929831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c932cfabb0 0x55c932d08ad0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:35.205 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.204+0000 7f412a363740 1 --2- 172.21.15.100:0/2152929831 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c932cfa1c0 0x55c932d08590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:35.205 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.205+0000 7f412a363740 1 -- 172.21.15.100:0/2152929831 >> 172.21.15.100:0/2152929831 conn(0x55c932cf5830 msgr2=0x55c932d031f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:41:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.207+0000 7f412a363740 1 -- 172.21.15.100:0/2152929831 shutdown_connections 2024-05-10T11:41:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.207+0000 7f412a363740 1 -- 172.21.15.100:0/2152929831 wait complete. 2024-05-10T11:41:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.207+0000 7f412a363740 20 client.19875 trim_cache size 0 max 16384 2024-05-10T11:41:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:41:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:41:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:41:35.208 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:41:35.211 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.210+0000 7f4127647640 1 -- 172.21.15.100:0/3803646598 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f410c0b29a0 msgr2=0x7f410c0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:35.211 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.210+0000 7f4127647640 1 --2- 172.21.15.100:0/3803646598 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f410c0b29a0 0x7f410c0b4e60 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x55c932cfb410 tx=0x7f4118009500 comp rx=0 tx=0).stop 2024-05-10T11:41:35.211 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.211+0000 7f4127647640 1 -- 172.21.15.100:0/3803646598 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4120101930 msgr2=0x7f4120198ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:35.211 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.211+0000 7f4127647640 1 --2- 172.21.15.100:0/3803646598 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4120101930 0x7f4120198ae0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f411c005ab0 tx=0x7f411c0042f0 comp rx=0 tx=0).stop 2024-05-10T11:41:35.213 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.211+0000 7f4127647640 1 -- 172.21.15.100:0/3803646598 shutdown_connections 2024-05-10T11:41:35.213 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.211+0000 7f4127647640 1 --2- 172.21.15.100:0/3803646598 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f410c0b29a0 0x7f410c0b4e60 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:35.213 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.211+0000 7f4127647640 1 --2- 172.21.15.100:0/3803646598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41201034e0 0x7f41201a0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:35.213 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.211+0000 7f4127647640 1 --2- 172.21.15.100:0/3803646598 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4120102b20 0x7f4120199020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:35.213 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.211+0000 7f4127647640 1 --2- 172.21.15.100:0/3803646598 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4120101930 0x7f4120198ae0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:35.213 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.211+0000 7f4127647640 1 -- 172.21.15.100:0/3803646598 >> 172.21.15.100:0/3803646598 conn(0x7f41200fcec0 msgr2=0x7f41200fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:41:35.213 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.212+0000 7f4127647640 1 -- 172.21.15.100:0/3803646598 shutdown_connections 2024-05-10T11:41:35.213 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:35.212+0000 7f4127647640 1 -- 172.21.15.100:0/3803646598 wait complete. 2024-05-10T11:41:35.956 INFO:tasks.workunit.client.0.smithi100.stdout: 10 94661 8.92 MB/sec execute 449 sec latency 186.374 ms 2024-05-10T11:41:36.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:41:35 smithi178 ceph-mon[33359]: pgmap v1117: 129 pgs: 129 active+clean; 570 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 4.7 MiB/s wr, 272 op/s 2024-05-10T11:41:36.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:41:35 smithi188 ceph-mon[34075]: pgmap v1117: 129 pgs: 129 active+clean; 570 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 4.7 MiB/s wr, 272 op/s 2024-05-10T11:41:36.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:41:35 smithi100 ceph-mon[30296]: pgmap v1117: 129 pgs: 129 active+clean; 570 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 4.7 MiB/s wr, 272 op/s 2024-05-10T11:41:36.956 INFO:tasks.workunit.client.0.smithi100.stdout: 10 94799 8.91 MB/sec execute 450 sec latency 155.386 ms 2024-05-10T11:41:37.956 INFO:tasks.workunit.client.0.smithi100.stdout: 10 94943 8.91 MB/sec execute 451 sec latency 139.214 ms 2024-05-10T11:41:38.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:41:37 smithi178 ceph-mon[33359]: pgmap v1118: 129 pgs: 129 active+clean; 567 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 845 KiB/s rd, 4.6 MiB/s wr, 310 op/s 2024-05-10T11:41:38.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:41:37 smithi188 ceph-mon[34075]: pgmap v1118: 129 pgs: 129 active+clean; 567 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 845 KiB/s rd, 4.6 MiB/s wr, 310 op/s 2024-05-10T11:41:38.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:41:37 smithi100 ceph-mon[30296]: pgmap v1118: 129 pgs: 129 active+clean; 567 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 845 KiB/s rd, 4.6 MiB/s wr, 310 op/s 2024-05-10T11:41:38.956 INFO:tasks.workunit.client.0.smithi100.stdout: 10 95111 8.92 MB/sec execute 452 sec latency 145.651 ms 2024-05-10T11:41:39.956 INFO:tasks.workunit.client.0.smithi100.stdout: 10 95303 8.93 MB/sec execute 453 sec latency 103.644 ms 2024-05-10T11:41:40.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:41:39 smithi178 ceph-mon[33359]: pgmap v1119: 129 pgs: 129 active+clean; 557 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 4.0 MiB/s wr, 278 op/s 2024-05-10T11:41:40.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:41:39 smithi188 ceph-mon[34075]: pgmap v1119: 129 pgs: 129 active+clean; 557 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 4.0 MiB/s wr, 278 op/s 2024-05-10T11:41:40.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:41:39 smithi100 ceph-mon[30296]: pgmap v1119: 129 pgs: 129 active+clean; 557 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 4.0 MiB/s wr, 278 op/s 2024-05-10T11:41:40.956 INFO:tasks.workunit.client.0.smithi100.stdout: 10 95404 8.93 MB/sec execute 454 sec latency 178.191 ms 2024-05-10T11:41:41.957 INFO:tasks.workunit.client.0.smithi100.stdout: 10 95594 8.92 MB/sec execute 455 sec latency 83.961 ms 2024-05-10T11:41:42.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:41:41 smithi100 ceph-mon[30296]: pgmap v1120: 129 pgs: 129 active+clean; 557 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 4.1 MiB/s wr, 309 op/s 2024-05-10T11:41:42.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:41:41 smithi178 ceph-mon[33359]: pgmap v1120: 129 pgs: 129 active+clean; 557 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 4.1 MiB/s wr, 309 op/s 2024-05-10T11:41:42.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:41:41 smithi188 ceph-mon[34075]: pgmap v1120: 129 pgs: 129 active+clean; 557 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 4.1 MiB/s wr, 309 op/s 2024-05-10T11:41:42.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.383+0000 7f42cfdc1700 1 -- 172.21.15.100:0/606290324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42c00958a0 msgr2=0x7f42c00a1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:42.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.383+0000 7f42cfdc1700 1 --2- 172.21.15.100:0/606290324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42c00958a0 0x7f42c00a1ac0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f42c8051460 tx=0x7f42c8069ab0 comp rx=0 tx=0).stop 2024-05-10T11:41:42.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.383+0000 7f42cfdc1700 1 -- 172.21.15.100:0/606290324 shutdown_connections 2024-05-10T11:41:42.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.383+0000 7f42cfdc1700 1 --2- 172.21.15.100:0/606290324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42c00958a0 0x7f42c00a1ac0 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:42.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.383+0000 7f42cfdc1700 1 --2- 172.21.15.100:0/606290324 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f42c009aca0 0x7f42c009b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:42.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.383+0000 7f42cfdc1700 1 --2- 172.21.15.100:0/606290324 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42c0094ee0 0x7f42c0095360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:42.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.383+0000 7f42cfdc1700 1 -- 172.21.15.100:0/606290324 >> 172.21.15.100:0/606290324 conn(0x7f42c000a7a0 msgr2=0x7f42c000abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:41:42.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.383+0000 7f42cfdc1700 1 -- 172.21.15.100:0/606290324 shutdown_connections 2024-05-10T11:41:42.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.383+0000 7f42cfdc1700 1 -- 172.21.15.100:0/606290324 wait complete. 2024-05-10T11:41:42.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.384+0000 7f42cfdc1700 1 Processor -- start 2024-05-10T11:41:42.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.384+0000 7f42cfdc1700 1 -- start start 2024-05-10T11:41:42.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.384+0000 7f42cfdc1700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f42c0094ee0 0x7f42c009bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:42.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.384+0000 7f42cfdc1700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42c00958a0 0x7f42c009bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:42.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.384+0000 7f42cfdc1700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42c009aca0 0x7f42c009c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:42.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.385+0000 7f42cfdc1700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f42c00a4c30 con 0x7f42c009aca0 2024-05-10T11:41:42.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.385+0000 7f42cfdc1700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f42c00a4ab0 con 0x7f42c0094ee0 2024-05-10T11:41:42.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.385+0000 7f42cfdc1700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f42c00a4db0 con 0x7f42c00958a0 2024-05-10T11:41:42.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.385+0000 7f42ce35e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42c009aca0 0x7f42c009c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:42.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.385+0000 7f42cd35c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42c00958a0 0x7f42c009bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:42.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.385+0000 7f42ce35e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42c009aca0 0x7f42c009c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51390/0 (socket says 172.21.15.100:51390) 2024-05-10T11:41:42.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.385+0000 7f42ce35e700 1 -- 172.21.15.100:0/2137941173 learned_addr learned my addr 172.21.15.100:0/2137941173 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:41:42.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.385+0000 7f42cdb5d700 1 --2- 172.21.15.100:0/2137941173 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f42c0094ee0 0x7f42c009bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:42.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.385+0000 7f42cd35c700 1 -- 172.21.15.100:0/2137941173 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f42c0094ee0 msgr2=0x7f42c009bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:42.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.385+0000 7f42cd35c700 1 --2- 172.21.15.100:0/2137941173 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f42c0094ee0 0x7f42c009bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:42.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.385+0000 7f42cd35c700 1 -- 172.21.15.100:0/2137941173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42c009aca0 msgr2=0x7f42c009c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:42.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.385+0000 7f42cd35c700 1 --2- 172.21.15.100:0/2137941173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42c009aca0 0x7f42c009c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:42.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.385+0000 7f42cd35c700 1 -- 172.21.15.100:0/2137941173 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42c013ea60 con 0x7f42c00958a0 2024-05-10T11:41:42.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.385+0000 7f42cd35c700 1 --2- 172.21.15.100:0/2137941173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42c00958a0 0x7f42c009bff0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f42b800ea20 tx=0x7f42b800ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:41:42.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.385+0000 7f42beffd700 1 -- 172.21.15.100:0/2137941173 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b800cbd0 con 0x7f42c00958a0 2024-05-10T11:41:42.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.385+0000 7f42beffd700 1 -- 172.21.15.100:0/2137941173 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f42b800cd70 con 0x7f42c00958a0 2024-05-10T11:41:42.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.385+0000 7f42cfdc1700 1 -- 172.21.15.100:0/2137941173 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42c013ed50 con 0x7f42c00958a0 2024-05-10T11:41:42.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.386+0000 7f42beffd700 1 -- 172.21.15.100:0/2137941173 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b8018680 con 0x7f42c00958a0 2024-05-10T11:41:42.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.386+0000 7f42cfdc1700 1 -- 172.21.15.100:0/2137941173 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42c013f1e0 con 0x7f42c00958a0 2024-05-10T11:41:42.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.389+0000 7f42beffd700 1 -- 172.21.15.100:0/2137941173 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f42b8010430 con 0x7f42c00958a0 2024-05-10T11:41:42.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.390+0000 7f42beffd700 1 --2- 172.21.15.100:0/2137941173 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f42b40b2880 0x7f42b40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:42.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.391+0000 7f42beffd700 1 -- 172.21.15.100:0/2137941173 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f42b80e2790 con 0x7f42c00958a0 2024-05-10T11:41:42.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.391+0000 7f42cdb5d700 1 --2- 172.21.15.100:0/2137941173 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f42b40b2880 0x7f42b40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:42.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.391+0000 7f42da654b80 1 Processor -- start 2024-05-10T11:41:42.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.391+0000 7f42da654b80 1 -- start start 2024-05-10T11:41:42.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.392+0000 7f42cdb5d700 1 --2- 172.21.15.100:0/2137941173 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f42b40b2880 0x7f42b40b4d40 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f42c4009750 tx=0x7f42c4009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:41:42.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.392+0000 7f42da654b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595fd090290 0x5595fd090670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:42.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.392+0000 7f42da654b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595fd090bb0 0x5595fd08a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:42.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.392+0000 7f42da654b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5595fd08ad50 0x5595fd08b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:42.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.392+0000 7f42da654b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5595fcfdc940 con 0x5595fd090290 2024-05-10T11:41:42.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.392+0000 7f42da654b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5595fcf41da0 con 0x5595fd08ad50 2024-05-10T11:41:42.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.392+0000 7f42da654b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5595fcfd8900 con 0x5595fd090bb0 2024-05-10T11:41:42.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.392+0000 7f42ce35e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5595fd08ad50 0x5595fd08b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:42.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.392+0000 7f42ce35e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5595fd08ad50 0x5595fd08b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:38056/0 (socket says 172.21.15.100:38056) 2024-05-10T11:41:42.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.392+0000 7f42ce35e700 1 -- 172.21.15.100:0/3056451946 learned_addr learned my addr 172.21.15.100:0/3056451946 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:41:42.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.392+0000 7f42cd35c700 1 --2- 172.21.15.100:0/3056451946 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595fd090bb0 0x5595fd08a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:42.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.392+0000 7f42ce35e700 1 -- 172.21.15.100:0/3056451946 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595fd090bb0 msgr2=0x5595fd08a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:42.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.392+0000 7f42ce35e700 1 --2- 172.21.15.100:0/3056451946 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595fd090bb0 0x5595fd08a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:42.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.392+0000 7f42ce35e700 1 -- 172.21.15.100:0/3056451946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595fd090290 msgr2=0x5595fd090670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:42.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.392+0000 7f42ce35e700 1 --2- 172.21.15.100:0/3056451946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595fd090290 0x5595fd090670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:42.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.392+0000 7f42ce35e700 1 -- 172.21.15.100:0/3056451946 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595fcf298d0 con 0x5595fd08ad50 2024-05-10T11:41:42.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.392+0000 7f42ce35e700 1 --2- 172.21.15.100:0/3056451946 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5595fd08ad50 0x5595fd08b170 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f42c8068f20 tx=0x7f42c8069b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ee389fc3d632e5c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:41:42.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.393+0000 7f42b3fff700 1 -- 172.21.15.100:0/3056451946 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c8085040 con 0x5595fd08ad50 2024-05-10T11:41:42.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.393+0000 7f42b3fff700 1 -- 172.21.15.100:0/3056451946 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f42c8075c20 con 0x5595fd08ad50 2024-05-10T11:41:42.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.393+0000 7f42b3fff700 1 -- 172.21.15.100:0/3056451946 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c807fc30 con 0x5595fd08ad50 2024-05-10T11:41:42.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.394+0000 7f42da654b80 1 -- 172.21.15.100:0/3056451946 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5595fd08ad50 msgr2=0x5595fd08b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:42.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.394+0000 7f42da654b80 1 --2- 172.21.15.100:0/3056451946 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5595fd08ad50 0x5595fd08b170 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f42c8068f20 tx=0x7f42c8069b80 comp rx=0 tx=0).stop 2024-05-10T11:41:42.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.394+0000 7f42da654b80 1 -- 172.21.15.100:0/3056451946 shutdown_connections 2024-05-10T11:41:42.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.394+0000 7f42da654b80 1 --2- 172.21.15.100:0/3056451946 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5595fd08ad50 0x5595fd08b170 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:42.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.394+0000 7f42da654b80 1 --2- 172.21.15.100:0/3056451946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595fd090290 0x5595fd090670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:42.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.394+0000 7f42da654b80 1 --2- 172.21.15.100:0/3056451946 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595fd090bb0 0x5595fd08a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:42.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.394+0000 7f42da654b80 1 -- 172.21.15.100:0/3056451946 >> 172.21.15.100:0/3056451946 conn(0x5595fcfb6b70 msgr2=0x5595fd0852d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:41:42.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.394+0000 7f42da654b80 1 -- 172.21.15.100:0/3056451946 shutdown_connections 2024-05-10T11:41:42.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.394+0000 7f42da654b80 1 -- 172.21.15.100:0/3056451946 wait complete. 2024-05-10T11:41:42.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.394+0000 7f42da654b80 1 -- start start 2024-05-10T11:41:42.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.395+0000 7f42da654b80 1 Processor -- start 2024-05-10T11:41:42.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.395+0000 7f42da654b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595fd08ad50 0x5595fd096800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:42.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.395+0000 7f42da654b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595fd090290 0x5595fd093ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:42.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.395+0000 7f42cdb5d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595fd08ad50 0x5595fd096800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:42.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.395+0000 7f42cdb5d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595fd08ad50 0x5595fd096800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51410/0 (socket says 172.21.15.100:51410) 2024-05-10T11:41:42.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.395+0000 7f42cdb5d700 1 -- 172.21.15.100:0/732895677 learned_addr learned my addr 172.21.15.100:0/732895677 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:41:42.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.395+0000 7f42da654b80 1 --2- 172.21.15.100:0/732895677 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5595fd090bb0 0x5595fd0940e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:42.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.395+0000 7f42da654b80 1 -- 172.21.15.100:0/732895677 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5595fd0975e0 con 0x5595fd08ad50 2024-05-10T11:41:42.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.395+0000 7f42da654b80 1 -- 172.21.15.100:0/732895677 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5595fcf95ed0 con 0x5595fd090bb0 2024-05-10T11:41:42.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.395+0000 7f42da654b80 1 -- 172.21.15.100:0/732895677 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5595fd097760 con 0x5595fd090290 2024-05-10T11:41:42.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.395+0000 7f42da654b80 10 client.0 _ll_register_callbacks cb 0x5595fcf705b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:41:42.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.395+0000 7f42cd35c700 1 --2- 172.21.15.100:0/732895677 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595fd090290 0x5595fd093ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:42.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.396+0000 7f42ce35e700 1 --2- 172.21.15.100:0/732895677 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5595fd090bb0 0x5595fd0940e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:42.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.396+0000 7f42ce35e700 1 -- 172.21.15.100:0/732895677 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595fd090290 msgr2=0x5595fd093ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:42.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.396+0000 7f42ce35e700 1 --2- 172.21.15.100:0/732895677 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595fd090290 0x5595fd093ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:42.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.396+0000 7f42ce35e700 1 -- 172.21.15.100:0/732895677 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595fd08ad50 msgr2=0x5595fd096800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:42.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.396+0000 7f42ce35e700 1 --2- 172.21.15.100:0/732895677 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595fd08ad50 0x5595fd096800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:42.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.396+0000 7f42ce35e700 1 -- 172.21.15.100:0/732895677 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595fcf93760 con 0x5595fd090bb0 2024-05-10T11:41:42.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.396+0000 7f42ce35e700 1 --2- 172.21.15.100:0/732895677 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5595fd090bb0 0x5595fd0940e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f42c80524a0 tx=0x7f42c8080920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a801a1f76666ae9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:41:42.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.396+0000 7f42b17fa700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:41:42.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.396+0000 7f42b17fa700 1 -- 172.21.15.100:0/732895677 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c809a070 con 0x5595fd090bb0 2024-05-10T11:41:42.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.397+0000 7f42da654b80 1 -- 172.21.15.100:0/732895677 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42c8075c20 con 0x5595fd090bb0 2024-05-10T11:41:42.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.397+0000 7f42b17fa700 1 -- 172.21.15.100:0/732895677 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f42c8080d60 con 0x5595fd090bb0 2024-05-10T11:41:42.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.397+0000 7f42b17fa700 1 -- 172.21.15.100:0/732895677 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f42c80a6050 con 0x5595fd090bb0 2024-05-10T11:41:42.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.397+0000 7f42da654b80 10 client.29138 fetch_fsmap learned FSMap version 41 2024-05-10T11:41:42.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.397+0000 7f42da654b80 1 -- 172.21.15.100:0/732895677 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5595fd0a1c10 con 0x5595fd090bb0 2024-05-10T11:41:42.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.397+0000 7f42b17fa700 1 -- 172.21.15.100:0/732895677 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f42c8075dd0 con 0x5595fd090bb0 2024-05-10T11:41:42.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.398+0000 7f42b17fa700 1 -- 172.21.15.100:0/732895677 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f42c808b040 con 0x5595fd090bb0 2024-05-10T11:41:42.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.398+0000 7f42da654b80 10 client.29138 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:41:42.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.398+0000 7f42da654b80 10 client.29138 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:41:42.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.398+0000 7f42da654b80 20 client.29138 populate_metadata read hostname 'smithi100' 2024-05-10T11:41:42.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.398+0000 7f42da654b80 1 --2- 172.21.15.100:0/732895677 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5595fd0a40e0 0x5595fd0a64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:42.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.398+0000 7f42da654b80 4 client.29138 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:41:42.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.398+0000 7f42da654b80 1 -- 172.21.15.100:0/732895677 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f42c8075c20 con 0x5595fd0a40e0 2024-05-10T11:41:42.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.398+0000 7f42cdb5d700 1 --2- 172.21.15.100:0/732895677 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5595fd0a40e0 0x5595fd0a64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:42.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.399+0000 7f42cdb5d700 1 --2- 172.21.15.100:0/732895677 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5595fd0a40e0 0x5595fd0a64d0 crc :-1 s=READY pgs=435 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c520205d6bdc8e38 server_cookie=d02fdff1ad72b658 in_seq=0 out_seq=0 2024-05-10T11:41:42.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.399+0000 7f42b17fa700 10 client.29138 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:41:42.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.399+0000 7f42b17fa700 1 -- 172.21.15.100:0/732895677 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f42c8075c20 con 0x5595fd0a40e0 2024-05-10T11:41:42.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.399+0000 7f42b17fa700 10 client.29138 handle_command_reply: tid=0 2024-05-10T11:41:42.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.399+0000 7f42da654b80 1 client.29138 shutdown 2024-05-10T11:41:42.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.400+0000 7f42da654b80 1 -- 172.21.15.100:0/732895677 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5595fd090bb0 msgr2=0x5595fd0940e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:42.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.400+0000 7f42da654b80 1 --2- 172.21.15.100:0/732895677 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5595fd090bb0 0x5595fd0940e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f42c80524a0 tx=0x7f42c8080920 comp rx=0 tx=0).stop 2024-05-10T11:41:42.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.400+0000 7f42da654b80 1 -- 172.21.15.100:0/732895677 shutdown_connections 2024-05-10T11:41:42.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.400+0000 7f42da654b80 1 --2- 172.21.15.100:0/732895677 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5595fd0a40e0 0x5595fd0a64d0 crc :-1 s=READY pgs=435 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:42.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.400+0000 7f42da654b80 1 --2- 172.21.15.100:0/732895677 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5595fd090bb0 0x5595fd0940e0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:42.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.400+0000 7f42da654b80 1 --2- 172.21.15.100:0/732895677 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595fd08ad50 0x5595fd096800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:42.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.400+0000 7f42da654b80 1 --2- 172.21.15.100:0/732895677 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595fd090290 0x5595fd093ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:42.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.400+0000 7f42da654b80 1 -- 172.21.15.100:0/732895677 >> 172.21.15.100:0/732895677 conn(0x5595fcfb6b70 msgr2=0x5595fd084960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:41:42.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.400+0000 7f42b17fa700 0 client.29138 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:41:42.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.400+0000 7f42da654b80 1 -- 172.21.15.100:0/732895677 shutdown_connections 2024-05-10T11:41:42.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.400+0000 7f42da654b80 1 -- 172.21.15.100:0/732895677 wait complete. 2024-05-10T11:41:42.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.400+0000 7f42da654b80 20 client.29138 trim_cache size 0 max 16384 2024-05-10T11:41:42.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.437+0000 7f42da654b80 1 Processor -- start 2024-05-10T11:41:42.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.437+0000 7f42da654b80 1 -- start start 2024-05-10T11:41:42.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.437+0000 7f42da654b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595fd08ad50 0x5595fd0a0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:42.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.437+0000 7f42da654b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595fd090290 0x5595fd0934a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:42.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.437+0000 7f42da654b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5595fd090bb0 0x5595fd0850e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:42.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.437+0000 7f42da654b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f42c014a4f0 con 0x5595fd090290 2024-05-10T11:41:42.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.437+0000 7f42da654b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f42c0147d80 con 0x5595fd090bb0 2024-05-10T11:41:42.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.437+0000 7f42da654b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f42c0145bd0 con 0x5595fd08ad50 2024-05-10T11:41:42.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.437+0000 7f42ce35e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5595fd090bb0 0x5595fd0850e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:42.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.438+0000 7f42ce35e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5595fd090bb0 0x5595fd0850e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:38068/0 (socket says 172.21.15.100:38068) 2024-05-10T11:41:42.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.438+0000 7f42ce35e700 1 -- 172.21.15.100:0/2321595211 learned_addr learned my addr 172.21.15.100:0/2321595211 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:41:42.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.439+0000 7f42cd35c700 1 --2- 172.21.15.100:0/2321595211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595fd090290 0x5595fd0934a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:42.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.439+0000 7f42cdb5d700 1 --2- 172.21.15.100:0/2321595211 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595fd08ad50 0x5595fd0a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:42.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.439+0000 7f42cd35c700 1 -- 172.21.15.100:0/2321595211 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595fd08ad50 msgr2=0x5595fd0a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:42.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.439+0000 7f42cd35c700 1 --2- 172.21.15.100:0/2321595211 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595fd08ad50 0x5595fd0a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:42.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.439+0000 7f42cd35c700 1 -- 172.21.15.100:0/2321595211 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5595fd090bb0 msgr2=0x5595fd0850e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:42.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.439+0000 7f42cd35c700 1 --2- 172.21.15.100:0/2321595211 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5595fd090bb0 0x5595fd0850e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:42.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.439+0000 7f42cd35c700 1 -- 172.21.15.100:0/2321595211 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595fd096fe0 con 0x5595fd090290 2024-05-10T11:41:42.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.440+0000 7f42cd35c700 1 --2- 172.21.15.100:0/2321595211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595fd090290 0x5595fd0934a0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f42b80a96f0 tx=0x7f42b8009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caf39c955391f69 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:41:42.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.444+0000 7f42b37fe700 1 -- 172.21.15.100:0/2321595211 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b80c3070 con 0x5595fd090290 2024-05-10T11:41:42.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.444+0000 7f42b37fe700 1 -- 172.21.15.100:0/2321595211 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f42b80b2190 con 0x5595fd090290 2024-05-10T11:41:42.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.445+0000 7f42da654b80 1 -- 172.21.15.100:0/2321595211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595fd090290 msgr2=0x5595fd0934a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:42.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.445+0000 7f42da654b80 1 --2- 172.21.15.100:0/2321595211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595fd090290 0x5595fd0934a0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f42b80a96f0 tx=0x7f42b8009f50 comp rx=0 tx=0).stop 2024-05-10T11:41:42.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.446+0000 7f42da654b80 1 -- 172.21.15.100:0/2321595211 shutdown_connections 2024-05-10T11:41:42.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.446+0000 7f42da654b80 1 --2- 172.21.15.100:0/2321595211 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5595fd090bb0 0x5595fd0850e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:42.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.446+0000 7f42da654b80 1 --2- 172.21.15.100:0/2321595211 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595fd08ad50 0x5595fd0a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:42.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.446+0000 7f42da654b80 1 --2- 172.21.15.100:0/2321595211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595fd090290 0x5595fd0934a0 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:42.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.446+0000 7f42da654b80 1 -- 172.21.15.100:0/2321595211 >> 172.21.15.100:0/2321595211 conn(0x5595fcfb6b70 msgr2=0x5595fd08ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:41:42.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.449+0000 7f42da654b80 1 -- 172.21.15.100:0/2321595211 shutdown_connections 2024-05-10T11:41:42.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.449+0000 7f42da654b80 1 -- 172.21.15.100:0/2321595211 wait complete. 2024-05-10T11:41:42.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.449+0000 7f42da654b80 1 -- start start 2024-05-10T11:41:42.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.450+0000 7f42da654b80 1 Processor -- start 2024-05-10T11:41:42.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.450+0000 7f42da654b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595fd08ad50 0x5595fd0a9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:42.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.450+0000 7f42da654b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595fd090290 0x5595fd0a7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:42.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.450+0000 7f42da654b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5595fd090bb0 0x5595fd0a8330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:42.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.450+0000 7f42da654b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f42c0146240 con 0x5595fd090290 2024-05-10T11:41:42.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.450+0000 7f42da654b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f42c01476b0 con 0x5595fd090bb0 2024-05-10T11:41:42.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.450+0000 7f42da654b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f42c0146910 con 0x5595fd08ad50 2024-05-10T11:41:42.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.450+0000 7f42da654b80 10 client.0 _ll_register_callbacks cb 0x5595fcf705b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:41:42.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.451+0000 7f42ce35e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5595fd090bb0 0x5595fd0a8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:42.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.451+0000 7f42ce35e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5595fd090bb0 0x5595fd0a8330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:38074/0 (socket says 172.21.15.100:38074) 2024-05-10T11:41:42.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.451+0000 7f42ce35e700 1 -- 172.21.15.100:0/1938942595 learned_addr learned my addr 172.21.15.100:0/1938942595 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:41:42.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.451+0000 7f42ce35e700 1 -- 172.21.15.100:0/1938942595 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595fd08ad50 msgr2=0x5595fd0a9a40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T11:41:42.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.451+0000 7f42ce35e700 1 --2- 172.21.15.100:0/1938942595 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595fd08ad50 0x5595fd0a9a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:42.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.451+0000 7f42ce35e700 1 -- 172.21.15.100:0/1938942595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595fd090290 msgr2=0x5595fd0a7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:42.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.451+0000 7f42ce35e700 1 --2- 172.21.15.100:0/1938942595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595fd090290 0x5595fd0a7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:42.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.451+0000 7f42ce35e700 1 -- 172.21.15.100:0/1938942595 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595fd0a8b70 con 0x5595fd090bb0 2024-05-10T11:41:42.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.452+0000 7f42ce35e700 1 --2- 172.21.15.100:0/1938942595 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5595fd090bb0 0x5595fd0a8330 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f42c80516d0 tx=0x7f42c8080f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d94d2d07ce5d8bb3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:41:42.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.452+0000 7f42b17fa700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:41:42.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.452+0000 7f42b17fa700 1 -- 172.21.15.100:0/1938942595 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c809a070 con 0x5595fd090bb0 2024-05-10T11:41:42.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.452+0000 7f42b17fa700 1 -- 172.21.15.100:0/1938942595 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f42c8075d20 con 0x5595fd090bb0 2024-05-10T11:41:42.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.453+0000 7f42da654b80 1 -- 172.21.15.100:0/1938942595 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5595fcfea320 con 0x5595fd090bb0 2024-05-10T11:41:42.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.453+0000 7f42b17fa700 1 -- 172.21.15.100:0/1938942595 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f42c8085040 con 0x5595fd090bb0 2024-05-10T11:41:42.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.455+0000 7f42da654b80 10 client.29144 fetch_fsmap learned FSMap version 41 2024-05-10T11:41:42.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.455+0000 7f42da654b80 1 -- 172.21.15.100:0/1938942595 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5595fd109600 con 0x5595fd090bb0 2024-05-10T11:41:42.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.455+0000 7f42b17fa700 1 -- 172.21.15.100:0/1938942595 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x5595fcfea320 con 0x5595fd090bb0 2024-05-10T11:41:42.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.455+0000 7f42b17fa700 1 -- 172.21.15.100:0/1938942595 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f42c808b040 con 0x5595fd090bb0 2024-05-10T11:41:42.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.455+0000 7f42da654b80 10 client.29144 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:41:42.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.455+0000 7f42da654b80 10 client.29144 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:41:42.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.455+0000 7f42da654b80 20 client.29144 populate_metadata read hostname 'smithi100' 2024-05-10T11:41:42.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.455+0000 7f42da654b80 1 --2- 172.21.15.100:0/1938942595 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5595fd109c10 0x5595fd10a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:42.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.455+0000 7f42da654b80 4 client.29144 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T11:41:42.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.455+0000 7f42da654b80 1 -- 172.21.15.100:0/1938942595 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5595fcfe43c0 con 0x5595fd109c10 2024-05-10T11:41:42.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.456+0000 7f42cdb5d700 1 --2- 172.21.15.100:0/1938942595 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5595fd109c10 0x5595fd10a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:42.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.459+0000 7f42cdb5d700 1 --2- 172.21.15.100:0/1938942595 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5595fd109c10 0x5595fd10a060 crc :-1 s=READY pgs=436 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d95a17e26e473e69 server_cookie=49929b7dc9f37cd7 in_seq=0 out_seq=0 2024-05-10T11:41:42.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.459+0000 7f42b17fa700 10 client.29144 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:41:42.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.460+0000 7f42b17fa700 1 -- 172.21.15.100:0/1938942595 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5595fcfe43c0 con 0x5595fd109c10 2024-05-10T11:41:42.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.460+0000 7f42b17fa700 10 client.29144 handle_command_reply: tid=0 2024-05-10T11:41:42.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.460+0000 7f42da654b80 1 client.29144 shutdown 2024-05-10T11:41:42.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.460+0000 7f42da654b80 1 -- 172.21.15.100:0/1938942595 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5595fd090bb0 msgr2=0x5595fd0a8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:42.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.460+0000 7f42da654b80 1 --2- 172.21.15.100:0/1938942595 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5595fd090bb0 0x5595fd0a8330 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f42c80516d0 tx=0x7f42c8080f90 comp rx=0 tx=0).stop 2024-05-10T11:41:42.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.461+0000 7f42da654b80 1 -- 172.21.15.100:0/1938942595 shutdown_connections 2024-05-10T11:41:42.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.461+0000 7f42da654b80 1 --2- 172.21.15.100:0/1938942595 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5595fd109c10 0x5595fd10a060 crc :-1 s=READY pgs=436 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:42.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.461+0000 7f42da654b80 1 --2- 172.21.15.100:0/1938942595 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5595fd090bb0 0x5595fd0a8330 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:42.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.461+0000 7f42da654b80 1 --2- 172.21.15.100:0/1938942595 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595fd08ad50 0x5595fd0a9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:42.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.461+0000 7f42da654b80 1 --2- 172.21.15.100:0/1938942595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595fd090290 0x5595fd0a7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:42.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.461+0000 7f42da654b80 1 -- 172.21.15.100:0/1938942595 >> 172.21.15.100:0/1938942595 conn(0x5595fcfb6b70 msgr2=0x5595fd08f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:41:42.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.461+0000 7f42b17fa700 0 client.29144 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:41:42.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.461+0000 7f42da654b80 1 -- 172.21.15.100:0/1938942595 shutdown_connections 2024-05-10T11:41:42.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.461+0000 7f42da654b80 1 -- 172.21.15.100:0/1938942595 wait complete. 2024-05-10T11:41:42.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.461+0000 7f42da654b80 20 client.29144 trim_cache size 0 max 16384 2024-05-10T11:41:42.462 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:41:42.462 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-05-10T11:41:42.462 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-05-10T11:41:42.463 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:41:42.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.462+0000 7f42cfdc1700 1 -- 172.21.15.100:0/2137941173 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f42b40b2880 msgr2=0x7f42b40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:42.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.462+0000 7f42cfdc1700 1 --2- 172.21.15.100:0/2137941173 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f42b40b2880 0x7f42b40b4d40 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f42c4009750 tx=0x7f42c4009310 comp rx=0 tx=0).stop 2024-05-10T11:41:42.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.462+0000 7f42cfdc1700 1 -- 172.21.15.100:0/2137941173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42c00958a0 msgr2=0x7f42c009bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:42.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.462+0000 7f42cfdc1700 1 --2- 172.21.15.100:0/2137941173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42c00958a0 0x7f42c009bff0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f42b800ea20 tx=0x7f42b800ed30 comp rx=0 tx=0).stop 2024-05-10T11:41:42.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.463+0000 7f42cfdc1700 1 -- 172.21.15.100:0/2137941173 shutdown_connections 2024-05-10T11:41:42.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.463+0000 7f42cfdc1700 1 --2- 172.21.15.100:0/2137941173 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f42b40b2880 0x7f42b40b4d40 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:42.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.463+0000 7f42cfdc1700 1 --2- 172.21.15.100:0/2137941173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42c009aca0 0x7f42c009c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:42.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.463+0000 7f42cfdc1700 1 --2- 172.21.15.100:0/2137941173 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f42c0094ee0 0x7f42c009bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:42.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.463+0000 7f42cfdc1700 1 --2- 172.21.15.100:0/2137941173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42c00958a0 0x7f42c009bff0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:42.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.463+0000 7f42cfdc1700 1 -- 172.21.15.100:0/2137941173 >> 172.21.15.100:0/2137941173 conn(0x7f42c000a7a0 msgr2=0x7f42c00a5930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:41:42.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.463+0000 7f42cfdc1700 1 -- 172.21.15.100:0/2137941173 shutdown_connections 2024-05-10T11:41:42.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:42.463+0000 7f42cfdc1700 1 -- 172.21.15.100:0/2137941173 wait complete. 2024-05-10T11:41:42.959 INFO:tasks.workunit.client.0.smithi100.stdout: 10 95736 8.91 MB/sec execute 456 sec latency 102.761 ms 2024-05-10T11:41:43.472 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T11:41:43.472 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T11:41:43.901 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:41:43.957 INFO:tasks.workunit.client.0.smithi100.stdout: 10 95895 8.91 MB/sec execute 457 sec latency 99.734 ms 2024-05-10T11:41:44.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:41:43 smithi178 ceph-mon[33359]: pgmap v1121: 129 pgs: 129 active+clean; 559 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 829 KiB/s rd, 3.9 MiB/s wr, 279 op/s 2024-05-10T11:41:44.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:41:43 smithi188 ceph-mon[34075]: pgmap v1121: 129 pgs: 129 active+clean; 559 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 829 KiB/s rd, 3.9 MiB/s wr, 279 op/s 2024-05-10T11:41:44.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:41:43 smithi100 ceph-mon[30296]: pgmap v1121: 129 pgs: 129 active+clean; 559 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 829 KiB/s rd, 3.9 MiB/s wr, 279 op/s 2024-05-10T11:41:44.957 INFO:tasks.workunit.client.0.smithi100.stdout: 10 96056 8.91 MB/sec execute 458 sec latency 99.357 ms 2024-05-10T11:41:45.957 INFO:tasks.workunit.client.0.smithi100.stdout: 10 96200 8.91 MB/sec execute 459 sec latency 85.407 ms 2024-05-10T11:41:46.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:41:45 smithi178 ceph-mon[33359]: pgmap v1122: 129 pgs: 129 active+clean; 561 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 3.7 MiB/s wr, 261 op/s 2024-05-10T11:41:46.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:41:45 smithi188 ceph-mon[34075]: pgmap v1122: 129 pgs: 129 active+clean; 561 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 3.7 MiB/s wr, 261 op/s 2024-05-10T11:41:46.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:41:45 smithi100 ceph-mon[30296]: pgmap v1122: 129 pgs: 129 active+clean; 561 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 3.7 MiB/s wr, 261 op/s 2024-05-10T11:41:46.957 INFO:tasks.workunit.client.0.smithi100.stdout: 10 96317 8.90 MB/sec execute 460 sec latency 165.726 ms 2024-05-10T11:41:47.957 INFO:tasks.workunit.client.0.smithi100.stdout: 10 96452 8.89 MB/sec execute 461 sec latency 145.717 ms 2024-05-10T11:41:48.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:41:47 smithi178 ceph-mon[33359]: pgmap v1123: 129 pgs: 129 active+clean; 560 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 3.8 MiB/s wr, 284 op/s 2024-05-10T11:41:48.264 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:41:47 smithi100 ceph-mon[30296]: pgmap v1123: 129 pgs: 129 active+clean; 560 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 3.8 MiB/s wr, 284 op/s 2024-05-10T11:41:48.264 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:41:48.265 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2072225012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2072225012},{"type":"v1","addr":"172.21.15.100:6835","nonce":2072225012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/401939540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":401939540},{"type":"v1","addr":"172.21.15.100:6837","nonce":401939540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14571,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/3946311025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3946311025},{"type":"v1","addr":"172.21.15.100:6839","nonce":3946311025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3736371660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3736371660},{"type":"v1","addr":"172.21.15.100:6841","nonce":3736371660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/2741810791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2741810791},{"type":"v1","addr":"172.21.15.178:6833","nonce":2741810791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1704891041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1704891041},{"type":"v1","addr":"172.21.15.188:6833","nonce":1704891041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":41,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T11:14:27.242452+0000","modified":"2024-05-10T11:35:29.588072+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24439,"mds_1":24437,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/2124076312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2124076312},{"type":"v1","addr":"172.21.15.178:6835","nonce":2124076312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/3224358425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3224358425},{"type":"v1","addr":"172.21.15.178:6837","nonce":3224358425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/1943342620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1943342620},{"type":"v1","addr":"172.21.15.188:6837","nonce":1943342620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.178:6839/3760985404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3760985404},{"type":"v1","addr":"172.21.15.178:6839","nonce":3760985404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.188:6839/3194917070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":3194917070},{"type":"v1","addr":"172.21.15.188:6839","nonce":3194917070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27034":{"gid":27034,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/3139559450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3139559450},{"type":"v1","addr":"172.21.15.188:6835","nonce":3139559450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24439,"qdb_cluster":[24437,24439,24427]},"id":1}]} 2024-05-10T11:41:48.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:47.595+0000 7f6d4edae700 1 -- 172.21.15.100:0/1089487613 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d4811cc80 msgr2=0x7f6d4811f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:48.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:47.595+0000 7f6d4edae700 1 --2- 172.21.15.100:0/1089487613 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d4811cc80 0x7f6d4811f070 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f6d38009990 tx=0x7f6d38009ca0 comp rx=0 tx=0).stop 2024-05-10T11:41:48.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:47.596+0000 7f6d4edae700 1 -- 172.21.15.100:0/1089487613 shutdown_connections 2024-05-10T11:41:48.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:47.596+0000 7f6d4edae700 1 --2- 172.21.15.100:0/1089487613 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d4811cc80 0x7f6d4811f070 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:48.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:47.596+0000 7f6d4edae700 1 --2- 172.21.15.100:0/1089487613 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d48111c80 0x7f6d4810a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:48.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:47.596+0000 7f6d4edae700 1 --2- 172.21.15.100:0/1089487613 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d4810a9e0 0x7f6d4810ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:48.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:47.596+0000 7f6d4edae700 1 -- 172.21.15.100:0/1089487613 >> 172.21.15.100:0/1089487613 conn(0x7f6d4806e610 msgr2=0x7f6d4806ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:41:48.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:47.597+0000 7f6d4edae700 1 -- 172.21.15.100:0/1089487613 shutdown_connections 2024-05-10T11:41:48.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:47.597+0000 7f6d4edae700 1 -- 172.21.15.100:0/1089487613 wait complete. 2024-05-10T11:41:48.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:47.598+0000 7f6d4edae700 1 Processor -- start 2024-05-10T11:41:48.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:47.598+0000 7f6d4edae700 1 -- start start 2024-05-10T11:41:48.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:47.598+0000 7f6d4edae700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d4810a9e0 0x7f6d481ae620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:48.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:47.598+0000 7f6d4edae700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d48111c80 0x7f6d481aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:48.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:47.598+0000 7f6d4edae700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d4811cc80 0x7f6d481a87a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:48.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:47.598+0000 7f6d4edae700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6d48122500 con 0x7f6d4811cc80 2024-05-10T11:41:48.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:47.598+0000 7f6d4edae700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6d48122380 con 0x7f6d48111c80 2024-05-10T11:41:48.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:47.598+0000 7f6d4edae700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6d48122680 con 0x7f6d4810a9e0 2024-05-10T11:41:48.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:47.599+0000 7f6d4cb4a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d4810a9e0 0x7f6d481ae620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:48.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:47.599+0000 7f6d47fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d48111c80 0x7f6d481aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:48.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:47.599+0000 7f6d47fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d48111c80 0x7f6d481aeb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:58486/0 (socket says 172.21.15.100:58486) 2024-05-10T11:41:48.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:47.599+0000 7f6d47fff700 1 -- 172.21.15.100:0/2775213420 learned_addr learned my addr 172.21.15.100:0/2775213420 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:41:48.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:47.600+0000 7f6d4d34b700 1 --2- 172.21.15.100:0/2775213420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d4811cc80 0x7f6d481a87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:48.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:47.600+0000 7f6d47fff700 1 -- 172.21.15.100:0/2775213420 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d4810a9e0 msgr2=0x7f6d481ae620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:48.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:47.600+0000 7f6d47fff700 1 --2- 172.21.15.100:0/2775213420 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d4810a9e0 0x7f6d481ae620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:48.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:47.600+0000 7f6d47fff700 1 -- 172.21.15.100:0/2775213420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d4811cc80 msgr2=0x7f6d481a87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:48.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:47.600+0000 7f6d47fff700 1 --2- 172.21.15.100:0/2775213420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d4811cc80 0x7f6d481a87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:48.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:47.600+0000 7f6d47fff700 1 -- 172.21.15.100:0/2775213420 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d481a8f80 con 0x7f6d48111c80 2024-05-10T11:41:48.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:47.601+0000 7f6d47fff700 1 --2- 172.21.15.100:0/2775213420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d48111c80 0x7f6d481aeb60 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c00b550 tx=0x7f6d3c00b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:41:48.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:47.601+0000 7f6d45ffb700 1 -- 172.21.15.100:0/2775213420 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c015020 con 0x7f6d48111c80 2024-05-10T11:41:48.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:47.601+0000 7f6d45ffb700 1 -- 172.21.15.100:0/2775213420 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d3c011cc0 con 0x7f6d48111c80 2024-05-10T11:41:48.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:47.601+0000 7f6d45ffb700 1 -- 172.21.15.100:0/2775213420 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c010c70 con 0x7f6d48111c80 2024-05-10T11:41:48.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:47.603+0000 7f6d4edae700 1 -- 172.21.15.100:0/2775213420 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d481a9270 con 0x7f6d48111c80 2024-05-10T11:41:48.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:47.603+0000 7f6d4edae700 1 -- 172.21.15.100:0/2775213420 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d4807b700 con 0x7f6d48111c80 2024-05-10T11:41:48.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:47.605+0000 7f6d4edae700 1 -- 172.21.15.100:0/2775213420 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d4806fba0 con 0x7f6d48111c80 2024-05-10T11:41:48.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:47.608+0000 7f6d45ffb700 1 -- 172.21.15.100:0/2775213420 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f6d3c018020 con 0x7f6d48111c80 2024-05-10T11:41:48.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:47.609+0000 7f6d45ffb700 1 --2- 172.21.15.100:0/2775213420 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f6d300b2a10 0x7f6d300b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:48.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:47.609+0000 7f6d45ffb700 1 -- 172.21.15.100:0/2775213420 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f6d3c0e2270 con 0x7f6d48111c80 2024-05-10T11:41:48.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:47.609+0000 7f6d4cb4a700 1 --2- 172.21.15.100:0/2775213420 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f6d300b2a10 0x7f6d300b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:48.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:47.609+0000 7f6d4cb4a700 1 --2- 172.21.15.100:0/2775213420 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f6d300b2a10 0x7f6d300b4ed0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f6d40009550 tx=0x7f6d400091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:41:48.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:47.611+0000 7f6d45ffb700 1 -- 172.21.15.100:0/2775213420 <== mon.2 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f6d3c0a9cb0 con 0x7f6d48111c80 2024-05-10T11:41:48.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:47.997+0000 7f6d4edae700 1 -- 172.21.15.100:0/2775213420 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6d4806a190 con 0x7f6d48111c80 2024-05-10T11:41:48.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:47.997+0000 7f6d45ffb700 1 -- 172.21.15.100:0/2775213420 <== mon.2 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+11897 (secure 0 0 0) 0x7f6d3c0a95b0 con 0x7f6d48111c80 2024-05-10T11:41:48.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:48.001+0000 7f6d4edae700 1 -- 172.21.15.100:0/2775213420 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f6d300b2a10 msgr2=0x7f6d300b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:48.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:48.001+0000 7f6d4edae700 1 --2- 172.21.15.100:0/2775213420 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f6d300b2a10 0x7f6d300b4ed0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f6d40009550 tx=0x7f6d400091b0 comp rx=0 tx=0).stop 2024-05-10T11:41:48.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:48.001+0000 7f6d4edae700 1 -- 172.21.15.100:0/2775213420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d48111c80 msgr2=0x7f6d481aeb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:48.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:48.001+0000 7f6d4edae700 1 --2- 172.21.15.100:0/2775213420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d48111c80 0x7f6d481aeb60 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c00b550 tx=0x7f6d3c00b860 comp rx=0 tx=0).stop 2024-05-10T11:41:48.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:48.001+0000 7f6d4edae700 1 -- 172.21.15.100:0/2775213420 shutdown_connections 2024-05-10T11:41:48.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:48.001+0000 7f6d4edae700 1 --2- 172.21.15.100:0/2775213420 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f6d300b2a10 0x7f6d300b4ed0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:48.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:48.001+0000 7f6d4edae700 1 --2- 172.21.15.100:0/2775213420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d4811cc80 0x7f6d481a87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:48.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:48.001+0000 7f6d4edae700 1 --2- 172.21.15.100:0/2775213420 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d4810a9e0 0x7f6d481ae620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:48.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:48.001+0000 7f6d4edae700 1 --2- 172.21.15.100:0/2775213420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d48111c80 0x7f6d481aeb60 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:48.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:48.001+0000 7f6d4edae700 1 -- 172.21.15.100:0/2775213420 >> 172.21.15.100:0/2775213420 conn(0x7f6d4806e610 msgr2=0x7f6d4810f860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:41:48.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:48.001+0000 7f6d4edae700 1 -- 172.21.15.100:0/2775213420 shutdown_connections 2024-05-10T11:41:48.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:48.001+0000 7f6d4edae700 1 -- 172.21.15.100:0/2775213420 wait complete. 2024-05-10T11:41:48.269 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 41 2024-05-10T11:41:48.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:41:47 smithi188 ceph-mon[34075]: pgmap v1123: 129 pgs: 129 active+clean; 560 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 3.8 MiB/s wr, 284 op/s 2024-05-10T11:41:48.343 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.341+0000 7f5792264640 1 -- 172.21.15.100:0/3747320255 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f578c1034c0 msgr2=0x7f578c109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:48.343 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.341+0000 7f5792264640 1 --2- 172.21.15.100:0/3747320255 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f578c1034c0 0x7f578c109d50 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f5784009ca0 tx=0x7f578402a8c0 comp rx=0 tx=0).stop 2024-05-10T11:41:48.343 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.341+0000 7f5792264640 1 -- 172.21.15.100:0/3747320255 shutdown_connections 2024-05-10T11:41:48.343 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.341+0000 7f5792264640 1 --2- 172.21.15.100:0/3747320255 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f578c1034c0 0x7f578c109d50 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:48.343 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.341+0000 7f5792264640 1 --2- 172.21.15.100:0/3747320255 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f578c102b00 0x7f578c102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:48.343 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.341+0000 7f5792264640 1 --2- 172.21.15.100:0/3747320255 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f578c101910 0x7f578c101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:48.343 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.341+0000 7f5792264640 1 -- 172.21.15.100:0/3747320255 >> 172.21.15.100:0/3747320255 conn(0x7f578c0fce80 msgr2=0x7f578c0ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:41:48.343 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.341+0000 7f5792264640 1 -- 172.21.15.100:0/3747320255 shutdown_connections 2024-05-10T11:41:48.343 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.341+0000 7f5792264640 1 -- 172.21.15.100:0/3747320255 wait complete. 2024-05-10T11:41:48.343 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.342+0000 7f5792264640 1 Processor -- start 2024-05-10T11:41:48.344 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.342+0000 7f5792264640 1 -- start start 2024-05-10T11:41:48.344 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.342+0000 7f5792264640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f578c101910 0x7f578c198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:48.344 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.342+0000 7f5792264640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f578c102b00 0x7f578c198fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:48.344 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.342+0000 7f5792264640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f578c1034c0 0x7f578c1a0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:48.344 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.342+0000 7f5792264640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f578c10e470 con 0x7f578c101910 2024-05-10T11:41:48.344 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.342+0000 7f5792264640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f578c10e2f0 con 0x7f578c1034c0 2024-05-10T11:41:48.344 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.342+0000 7f5792264640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f578c10e5f0 con 0x7f578c102b00 2024-05-10T11:41:48.344 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.342+0000 7f578bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f578c101910 0x7f578c198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:48.344 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.342+0000 7f578bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f578c101910 0x7f578c198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36948/0 (socket says 172.21.15.100:36948) 2024-05-10T11:41:48.344 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.342+0000 7f578bfff640 1 -- 172.21.15.100:0/189217868 learned_addr learned my addr 172.21.15.100:0/189217868 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:41:48.344 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.342+0000 7f5783fff640 1 --2- 172.21.15.100:0/189217868 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f578c102b00 0x7f578c198fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:48.344 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.342+0000 7f578bfff640 1 -- 172.21.15.100:0/189217868 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f578c102b00 msgr2=0x7f578c198fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:48.344 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.342+0000 7f578bfff640 1 --2- 172.21.15.100:0/189217868 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f578c102b00 0x7f578c198fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:48.344 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.342+0000 7f578bfff640 1 -- 172.21.15.100:0/189217868 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f578c1034c0 msgr2=0x7f578c1a0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:48.344 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.342+0000 7f578bfff640 1 --2- 172.21.15.100:0/189217868 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f578c1034c0 0x7f578c1a0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:48.345 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.342+0000 7f578bfff640 1 -- 172.21.15.100:0/189217868 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f578c1a06c0 con 0x7f578c101910 2024-05-10T11:41:48.345 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.342+0000 7f578bfff640 1 --2- 172.21.15.100:0/189217868 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f578c101910 0x7f578c198aa0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f577c00e9c0 tx=0x7f577c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:41:48.345 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.342+0000 7f5789ffb640 1 -- 172.21.15.100:0/189217868 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577c00cd80 con 0x7f578c101910 2024-05-10T11:41:48.345 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.342+0000 7f5789ffb640 1 -- 172.21.15.100:0/189217868 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f577c0035f0 con 0x7f578c101910 2024-05-10T11:41:48.345 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.342+0000 7f5789ffb640 1 -- 172.21.15.100:0/189217868 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577c0044c0 con 0x7f578c101910 2024-05-10T11:41:48.345 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.342+0000 7f5792264640 1 -- 172.21.15.100:0/189217868 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f578c1a09b0 con 0x7f578c101910 2024-05-10T11:41:48.345 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.343+0000 7f5792264640 1 -- 172.21.15.100:0/189217868 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f578c1a0eb0 con 0x7f578c101910 2024-05-10T11:41:48.350 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.345+0000 7f5789ffb640 1 -- 172.21.15.100:0/189217868 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f577c004000 con 0x7f578c101910 2024-05-10T11:41:48.350 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.345+0000 7f5794f80740 1 Processor -- start 2024-05-10T11:41:48.350 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.345+0000 7f5794f80740 1 -- start start 2024-05-10T11:41:48.350 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.345+0000 7f5794f80740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db94eb11c0 0x55db94eb15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:48.350 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.345+0000 7f5794f80740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db94eb1bb0 0x55db94eb66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:48.350 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.345+0000 7f5794f80740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db94eb6d20 0x55db94eb91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:48.350 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.345+0000 7f5794f80740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55db94e115d0 con 0x55db94eb6d20 2024-05-10T11:41:48.350 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.345+0000 7f5794f80740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55db94dca890 con 0x55db94eb1bb0 2024-05-10T11:41:48.351 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.345+0000 7f5794f80740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55db94e0d590 con 0x55db94eb11c0 2024-05-10T11:41:48.351 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.345+0000 7f5790801640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db94eb11c0 0x55db94eb15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:48.351 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.345+0000 7f5790801640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db94eb11c0 0x55db94eb15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:57798/0 (socket says 172.21.15.100:57798) 2024-05-10T11:41:48.351 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.345+0000 7f5790801640 1 -- 172.21.15.100:0/2724207825 learned_addr learned my addr 172.21.15.100:0/2724207825 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:41:48.351 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.346+0000 7f578bfff640 1 --2- 172.21.15.100:0/2724207825 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db94eb1bb0 0x55db94eb66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:48.351 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.346+0000 7f5783fff640 1 --2- 172.21.15.100:0/2724207825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db94eb6d20 0x55db94eb91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:48.351 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.346+0000 7f5790801640 1 -- 172.21.15.100:0/2724207825 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db94eb1bb0 msgr2=0x55db94eb66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:48.351 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.346+0000 7f5790801640 1 --2- 172.21.15.100:0/2724207825 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db94eb1bb0 0x55db94eb66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:48.351 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.346+0000 7f5790801640 1 -- 172.21.15.100:0/2724207825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db94eb6d20 msgr2=0x55db94eb91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:48.351 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.346+0000 7f5790801640 1 --2- 172.21.15.100:0/2724207825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db94eb6d20 0x55db94eb91b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:48.351 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.346+0000 7f5790801640 1 -- 172.21.15.100:0/2724207825 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db94b23210 con 0x55db94eb11c0 2024-05-10T11:41:48.351 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.346+0000 7f5789ffb640 1 --2- 172.21.15.100:0/189217868 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f57780b28a0 0x7f57780b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:48.351 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.346+0000 7f578bfff640 1 --2- 172.21.15.100:0/189217868 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f57780b28a0 0x7f57780b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:48.351 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.346+0000 7f5790801640 1 --2- 172.21.15.100:0/2724207825 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db94eb11c0 0x55db94eb15e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f578402aeb0 tx=0x7f5784009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88d320ab96c20f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:41:48.351 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.346+0000 7f5789ffb640 1 -- 172.21.15.100:0/189217868 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f577c0e2a50 con 0x7f578c101910 2024-05-10T11:41:48.352 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.346+0000 7f57827fc640 1 -- 172.21.15.100:0/2724207825 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57840316d0 con 0x55db94eb11c0 2024-05-10T11:41:48.352 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.346+0000 7f57827fc640 1 -- 172.21.15.100:0/2724207825 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f578403f070 con 0x55db94eb11c0 2024-05-10T11:41:48.352 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.346+0000 7f57827fc640 1 -- 172.21.15.100:0/2724207825 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5784004330 con 0x55db94eb11c0 2024-05-10T11:41:48.352 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.347+0000 7f5794f80740 1 -- 172.21.15.100:0/2724207825 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db94eb11c0 msgr2=0x55db94eb15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:48.352 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.347+0000 7f578bfff640 1 --2- 172.21.15.100:0/189217868 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f57780b28a0 0x7f57780b4d60 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f577c0e0560 tx=0x7f577c0e7040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:41:48.352 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.347+0000 7f5794f80740 1 --2- 172.21.15.100:0/2724207825 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db94eb11c0 0x55db94eb15e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f578402aeb0 tx=0x7f5784009a20 comp rx=0 tx=0).stop 2024-05-10T11:41:48.352 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.347+0000 7f5794f80740 1 -- 172.21.15.100:0/2724207825 shutdown_connections 2024-05-10T11:41:48.352 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.347+0000 7f5794f80740 1 --2- 172.21.15.100:0/2724207825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db94eb6d20 0x55db94eb91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:48.352 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.347+0000 7f5794f80740 1 --2- 172.21.15.100:0/2724207825 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db94eb1bb0 0x55db94eb66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:48.352 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.347+0000 7f5794f80740 1 --2- 172.21.15.100:0/2724207825 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db94eb11c0 0x55db94eb15e0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:48.352 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.347+0000 7f5794f80740 1 -- 172.21.15.100:0/2724207825 >> 172.21.15.100:0/2724207825 conn(0x55db94ea8420 msgr2=0x55db94ea8820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:41:48.352 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.347+0000 7f5794f80740 1 -- 172.21.15.100:0/2724207825 shutdown_connections 2024-05-10T11:41:48.352 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.347+0000 7f5794f80740 1 -- 172.21.15.100:0/2724207825 wait complete. 2024-05-10T11:41:48.352 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.347+0000 7f5794f80740 1 -- start start 2024-05-10T11:41:48.352 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.348+0000 7f5794f80740 1 Processor -- start 2024-05-10T11:41:48.352 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.348+0000 7f5794f80740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db94eb11c0 0x55db94ebb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:48.353 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.348+0000 7f5794f80740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db94eb1bb0 0x55db94ebb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:48.353 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.348+0000 7f5794f80740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db94eb6d20 0x55db94ec2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:48.353 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.348+0000 7f5794f80740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55db94eb9c70 con 0x55db94eb1bb0 2024-05-10T11:41:48.353 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.348+0000 7f5794f80740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55db94dce370 con 0x55db94eb6d20 2024-05-10T11:41:48.353 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.348+0000 7f5794f80740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55db94eb9df0 con 0x55db94eb11c0 2024-05-10T11:41:48.353 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.348+0000 7f5794f80740 10 client.0 _ll_register_callbacks cb 0x55db94b22a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:41:48.353 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.348+0000 7f5783fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db94eb1bb0 0x55db94ebb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:48.353 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.348+0000 7f5783fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db94eb1bb0 0x55db94ebb8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36970/0 (socket says 172.21.15.100:36970) 2024-05-10T11:41:48.353 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.348+0000 7f5783fff640 1 -- 172.21.15.100:0/3142976389 learned_addr learned my addr 172.21.15.100:0/3142976389 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:41:48.354 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.348+0000 7f5790801640 1 --2- 172.21.15.100:0/3142976389 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db94eb11c0 0x55db94ebb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:48.354 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.348+0000 7f5790801640 1 --2- 172.21.15.100:0/3142976389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db94eb6d20 0x55db94ec2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:48.354 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.348+0000 7f5783fff640 1 -- 172.21.15.100:0/3142976389 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db94eb11c0 msgr2=0x55db94ebb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:48.354 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.348+0000 7f5783fff640 1 --2- 172.21.15.100:0/3142976389 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db94eb11c0 0x55db94ebb3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:48.354 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.348+0000 7f5783fff640 1 -- 172.21.15.100:0/3142976389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db94eb6d20 msgr2=0x55db94ec2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:48.354 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.348+0000 7f5783fff640 1 --2- 172.21.15.100:0/3142976389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db94eb6d20 0x55db94ec2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:48.354 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.348+0000 7f5783fff640 1 -- 172.21.15.100:0/3142976389 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db94bef620 con 0x55db94eb1bb0 2024-05-10T11:41:48.354 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.349+0000 7f5783fff640 1 --2- 172.21.15.100:0/3142976389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db94eb1bb0 0x55db94ebb8f0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f5774013490 tx=0x7f5774013950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2497ecb7520cbabb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:41:48.354 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.349+0000 7f576b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:41:48.354 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.349+0000 7f576b7fe640 1 -- 172.21.15.100:0/3142976389 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5774004020 con 0x55db94eb1bb0 2024-05-10T11:41:48.354 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.349+0000 7f576b7fe640 1 -- 172.21.15.100:0/3142976389 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f57740041c0 con 0x55db94eb1bb0 2024-05-10T11:41:48.354 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.349+0000 7f5794f80740 1 -- 172.21.15.100:0/3142976389 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f578403f070 con 0x55db94eb1bb0 2024-05-10T11:41:48.354 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.349+0000 7f576b7fe640 1 -- 172.21.15.100:0/3142976389 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f5774025050 con 0x55db94eb1bb0 2024-05-10T11:41:48.355 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.349+0000 7f5794f80740 10 client.19905 fetch_fsmap learned FSMap version 41 2024-05-10T11:41:48.355 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.349+0000 7f5794f80740 1 -- 172.21.15.100:0/3142976389 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55db94ec59e0 con 0x55db94eb1bb0 2024-05-10T11:41:48.355 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.349+0000 7f576b7fe640 1 -- 172.21.15.100:0/3142976389 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f578403f070 con 0x55db94eb1bb0 2024-05-10T11:41:48.355 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.349+0000 7f576b7fe640 1 -- 172.21.15.100:0/3142976389 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f577400f040 con 0x55db94eb1bb0 2024-05-10T11:41:48.355 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.349+0000 7f5794f80740 10 client.19905 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:41:48.355 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.349+0000 7f5794f80740 10 client.19905 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:41:48.355 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.349+0000 7f5794f80740 20 client.19905 populate_metadata read hostname 'smithi100' 2024-05-10T11:41:48.355 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.349+0000 7f5794f80740 1 --2- 172.21.15.100:0/3142976389 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55db94ec7170 0x55db94ec9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:48.356 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.349+0000 7f5794f80740 4 client.19905 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:41:48.356 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.349+0000 7f5794f80740 1 -- 172.21.15.100:0/3142976389 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db94e1f1e0 con 0x55db94ec7170 2024-05-10T11:41:48.356 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.350+0000 7f5790801640 1 --2- 172.21.15.100:0/3142976389 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55db94ec7170 0x55db94ec9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:48.356 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.350+0000 7f576b7fe640 10 client.19905 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:41:48.356 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.350+0000 7f5790801640 1 --2- 172.21.15.100:0/3142976389 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55db94ec7170 0x55db94ec9580 crc :-1 s=READY pgs=437 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1643aaef103fb45 server_cookie=f235116dab45dcbb in_seq=0 out_seq=0 2024-05-10T11:41:48.357 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.351+0000 7f576b7fe640 1 -- 172.21.15.100:0/3142976389 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55db94e1f1e0 con 0x55db94ec7170 2024-05-10T11:41:48.357 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.351+0000 7f576b7fe640 10 client.19905 handle_command_reply: tid=0 2024-05-10T11:41:48.357 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.351+0000 7f5794f80740 1 client.19905 shutdown 2024-05-10T11:41:48.357 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.351+0000 7f5794f80740 1 -- 172.21.15.100:0/3142976389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db94eb1bb0 msgr2=0x55db94ebb8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:48.357 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.351+0000 7f5794f80740 1 --2- 172.21.15.100:0/3142976389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db94eb1bb0 0x55db94ebb8f0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f5774013490 tx=0x7f5774013950 comp rx=0 tx=0).stop 2024-05-10T11:41:48.357 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.351+0000 7f5794f80740 1 -- 172.21.15.100:0/3142976389 shutdown_connections 2024-05-10T11:41:48.357 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.351+0000 7f5794f80740 1 --2- 172.21.15.100:0/3142976389 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55db94ec7170 0x55db94ec9580 crc :-1 s=READY pgs=437 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:48.357 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.351+0000 7f576b7fe640 0 client.19905 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:41:48.357 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.351+0000 7f5794f80740 1 --2- 172.21.15.100:0/3142976389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db94eb6d20 0x55db94ec2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:48.357 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.351+0000 7f5794f80740 1 --2- 172.21.15.100:0/3142976389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db94eb1bb0 0x55db94ebb8f0 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:48.358 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.351+0000 7f5794f80740 1 --2- 172.21.15.100:0/3142976389 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db94eb11c0 0x55db94ebb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:48.358 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.351+0000 7f5794f80740 1 -- 172.21.15.100:0/3142976389 >> 172.21.15.100:0/3142976389 conn(0x55db94ea8420 msgr2=0x55db94ea7580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:41:48.358 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.351+0000 7f5794f80740 1 -- 172.21.15.100:0/3142976389 shutdown_connections 2024-05-10T11:41:48.358 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.351+0000 7f5794f80740 1 -- 172.21.15.100:0/3142976389 wait complete. 2024-05-10T11:41:48.358 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.352+0000 7f5794f80740 20 client.19905 trim_cache size 0 max 16384 2024-05-10T11:41:48.384 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.384+0000 7f5794f80740 1 Processor -- start 2024-05-10T11:41:48.385 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.385+0000 7f5794f80740 1 -- start start 2024-05-10T11:41:48.386 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.385+0000 7f5794f80740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db94eb11c0 0x55db94eb0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:48.386 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.385+0000 7f5794f80740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db94eb1bb0 0x55db94eb2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:48.386 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.385+0000 7f5794f80740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db94eb6d20 0x55db94eb3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:48.386 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.385+0000 7f5794f80740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f578c1aba10 con 0x55db94eb6d20 2024-05-10T11:41:48.386 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.385+0000 7f5794f80740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f578c1a92e0 con 0x55db94eb1bb0 2024-05-10T11:41:48.386 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.385+0000 7f5794f80740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f578c1a7130 con 0x55db94eb11c0 2024-05-10T11:41:48.389 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.389+0000 7f5790801640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db94eb11c0 0x55db94eb0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:48.390 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.389+0000 7f5790801640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db94eb11c0 0x55db94eb0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:57822/0 (socket says 172.21.15.100:57822) 2024-05-10T11:41:48.390 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.389+0000 7f5790801640 1 -- 172.21.15.100:0/3806392942 learned_addr learned my addr 172.21.15.100:0/3806392942 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:41:48.390 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.389+0000 7f5783fff640 1 --2- 172.21.15.100:0/3806392942 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db94eb1bb0 0x55db94eb2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:48.390 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.390+0000 7f5783fff640 1 -- 172.21.15.100:0/3806392942 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db94eb11c0 msgr2=0x55db94eb0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:48.390 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.390+0000 7f5783fff640 1 --2- 172.21.15.100:0/3806392942 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db94eb11c0 0x55db94eb0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:48.390 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.390+0000 7f5783fff640 1 -- 172.21.15.100:0/3806392942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db94eb6d20 msgr2=0x55db94eb3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:48.390 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.390+0000 7f5783fff640 1 --2- 172.21.15.100:0/3806392942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db94eb6d20 0x55db94eb3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:48.390 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.390+0000 7f5783fff640 1 -- 172.21.15.100:0/3806392942 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db94ea8250 con 0x55db94eb1bb0 2024-05-10T11:41:48.391 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.390+0000 7f5783fff640 1 --2- 172.21.15.100:0/3806392942 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db94eb1bb0 0x55db94eb2dd0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5774000c00 tx=0x7f5774009280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea98d12c0339bd7e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:41:48.391 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.391+0000 7f5781ffb640 1 -- 172.21.15.100:0/3806392942 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5774005d10 con 0x55db94eb1bb0 2024-05-10T11:41:48.391 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.392+0000 7f5781ffb640 1 -- 172.21.15.100:0/3806392942 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5774013e50 con 0x55db94eb1bb0 2024-05-10T11:41:48.392 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.392+0000 7f5781ffb640 1 -- 172.21.15.100:0/3806392942 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5774005d10 con 0x55db94eb1bb0 2024-05-10T11:41:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.393+0000 7f5794f80740 1 -- 172.21.15.100:0/3806392942 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db94eb1bb0 msgr2=0x55db94eb2dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.393+0000 7f5794f80740 1 --2- 172.21.15.100:0/3806392942 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db94eb1bb0 0x55db94eb2dd0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5774000c00 tx=0x7f5774009280 comp rx=0 tx=0).stop 2024-05-10T11:41:48.394 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.394+0000 7f5794f80740 1 -- 172.21.15.100:0/3806392942 shutdown_connections 2024-05-10T11:41:48.395 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.395+0000 7f5794f80740 1 --2- 172.21.15.100:0/3806392942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db94eb6d20 0x55db94eb3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:48.395 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.395+0000 7f5794f80740 1 --2- 172.21.15.100:0/3806392942 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db94eb1bb0 0x55db94eb2dd0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:48.395 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.395+0000 7f5794f80740 1 --2- 172.21.15.100:0/3806392942 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db94eb11c0 0x55db94eb0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:48.395 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.395+0000 7f5794f80740 1 -- 172.21.15.100:0/3806392942 >> 172.21.15.100:0/3806392942 conn(0x55db94ea8420 msgr2=0x55db94ea33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:41:48.395 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.395+0000 7f5794f80740 1 -- 172.21.15.100:0/3806392942 shutdown_connections 2024-05-10T11:41:48.396 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.395+0000 7f5794f80740 1 -- 172.21.15.100:0/3806392942 wait complete. 2024-05-10T11:41:48.396 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.396+0000 7f5794f80740 1 -- start start 2024-05-10T11:41:48.397 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.398+0000 7f5794f80740 1 Processor -- start 2024-05-10T11:41:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.398+0000 7f5794f80740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db94eb11c0 0x55db94eb0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.398+0000 7f5794f80740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db94eb1bb0 0x55db94ec08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.398+0000 7f5790801640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db94eb11c0 0x55db94eb0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.399+0000 7f5790801640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db94eb11c0 0x55db94eb0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:58538/0 (socket says 172.21.15.100:58538) 2024-05-10T11:41:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.399+0000 7f5790801640 1 -- 172.21.15.100:0/2686771056 learned_addr learned my addr 172.21.15.100:0/2686771056 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:41:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.399+0000 7f5794f80740 1 --2- 172.21.15.100:0/2686771056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db94eb6d20 0x55db94f34220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.399+0000 7f5794f80740 1 -- 172.21.15.100:0/2686771056 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f578c1a77a0 con 0x55db94eb1bb0 2024-05-10T11:41:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.399+0000 7f5794f80740 1 -- 172.21.15.100:0/2686771056 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f578c1a8c10 con 0x55db94eb11c0 2024-05-10T11:41:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.399+0000 7f5794f80740 1 -- 172.21.15.100:0/2686771056 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f578c1a7e70 con 0x55db94eb6d20 2024-05-10T11:41:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.399+0000 7f5794f80740 10 client.0 _ll_register_callbacks cb 0x7f5774005a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:41:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.399+0000 7f5783fff640 1 --2- 172.21.15.100:0/2686771056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db94eb1bb0 0x55db94ec08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.400+0000 7f5783fff640 1 -- 172.21.15.100:0/2686771056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db94eb6d20 msgr2=0x55db94f34220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T11:41:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.400+0000 7f5783fff640 1 --2- 172.21.15.100:0/2686771056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db94eb6d20 0x55db94f34220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.400+0000 7f5783fff640 1 -- 172.21.15.100:0/2686771056 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db94eb11c0 msgr2=0x55db94eb0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.400+0000 7f5783fff640 1 --2- 172.21.15.100:0/2686771056 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db94eb11c0 0x55db94eb0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.400+0000 7f5783fff640 1 -- 172.21.15.100:0/2686771056 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db94f34960 con 0x55db94eb1bb0 2024-05-10T11:41:48.401 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.401+0000 7f5783fff640 1 --2- 172.21.15.100:0/2686771056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db94eb1bb0 0x55db94ec08e0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f5774003900 tx=0x7f5774003d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e0e9392c5bb5263 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:41:48.401 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.401+0000 7f576b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:41:48.401 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.401+0000 7f576b7fe640 1 -- 172.21.15.100:0/2686771056 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5774036030 con 0x55db94eb1bb0 2024-05-10T11:41:48.401 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.401+0000 7f576b7fe640 1 -- 172.21.15.100:0/2686771056 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f57740035f0 con 0x55db94eb1bb0 2024-05-10T11:41:48.401 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.401+0000 7f5790801640 1 --2- 172.21.15.100:0/2686771056 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db94eb11c0 0x55db94eb0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:41:48.402 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.402+0000 7f5794f80740 1 -- 172.21.15.100:0/2686771056 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5774013e50 con 0x55db94eb1bb0 2024-05-10T11:41:48.402 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.402+0000 7f576b7fe640 1 -- 172.21.15.100:0/2686771056 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f5774041050 con 0x55db94eb1bb0 2024-05-10T11:41:48.402 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.402+0000 7f5794f80740 10 client.19908 fetch_fsmap learned FSMap version 41 2024-05-10T11:41:48.402 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.402+0000 7f5794f80740 1 -- 172.21.15.100:0/2686771056 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55db94f35c50 con 0x55db94eb1bb0 2024-05-10T11:41:48.404 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.404+0000 7f576b7fe640 1 -- 172.21.15.100:0/2686771056 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5774011da0 con 0x55db94eb1bb0 2024-05-10T11:41:48.404 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.404+0000 7f576b7fe640 1 -- 172.21.15.100:0/2686771056 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f5774032850 con 0x55db94eb1bb0 2024-05-10T11:41:48.404 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.404+0000 7f5794f80740 10 client.19908 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:41:48.404 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.404+0000 7f5794f80740 10 client.19908 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:41:48.405 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.404+0000 7f5794f80740 20 client.19908 populate_metadata read hostname 'smithi100' 2024-05-10T11:41:48.405 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.404+0000 7f5794f80740 1 --2- 172.21.15.100:0/2686771056 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55db94ec7170 0x55db94f385b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:48.405 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.404+0000 7f5794f80740 4 client.19908 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:41:48.405 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.404+0000 7f5794f80740 1 -- 172.21.15.100:0/2686771056 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f5774013e50 con 0x55db94ec7170 2024-05-10T11:41:48.406 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.406+0000 7f5790801640 1 --2- 172.21.15.100:0/2686771056 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55db94ec7170 0x55db94f385b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:48.412 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.412+0000 7f5790801640 1 --2- 172.21.15.100:0/2686771056 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55db94ec7170 0x55db94f385b0 crc :-1 s=READY pgs=438 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46614c42a2276183 server_cookie=c6ed6a7cfe221bac in_seq=0 out_seq=0 2024-05-10T11:41:48.412 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.412+0000 7f576b7fe640 10 client.19908 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:41:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.598+0000 7f576b7fe640 1 -- 172.21.15.100:0/2686771056 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f5774013e50 con 0x55db94ec7170 2024-05-10T11:41:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.598+0000 7f576b7fe640 10 client.19908 handle_command_reply: tid=0 2024-05-10T11:41:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.598+0000 7f5794f80740 1 client.19908 shutdown 2024-05-10T11:41:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.599+0000 7f5794f80740 1 -- 172.21.15.100:0/2686771056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db94eb1bb0 msgr2=0x55db94ec08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.600+0000 7f5794f80740 1 --2- 172.21.15.100:0/2686771056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db94eb1bb0 0x55db94ec08e0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f5774003900 tx=0x7f5774003d20 comp rx=0 tx=0).stop 2024-05-10T11:41:48.602 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.601+0000 7f5794f80740 1 -- 172.21.15.100:0/2686771056 shutdown_connections 2024-05-10T11:41:48.602 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.601+0000 7f5794f80740 1 --2- 172.21.15.100:0/2686771056 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55db94ec7170 0x55db94f385b0 crc :-1 s=READY pgs=438 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:48.602 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.601+0000 7f5794f80740 1 --2- 172.21.15.100:0/2686771056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db94eb6d20 0x55db94f34220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:48.602 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.601+0000 7f576b7fe640 0 client.19908 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:41:48.602 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.602+0000 7f5794f80740 1 --2- 172.21.15.100:0/2686771056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55db94eb1bb0 0x55db94ec08e0 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:48.602 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.602+0000 7f5794f80740 1 --2- 172.21.15.100:0/2686771056 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db94eb11c0 0x55db94eb0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:48.602 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.602+0000 7f5794f80740 1 -- 172.21.15.100:0/2686771056 >> 172.21.15.100:0/2686771056 conn(0x55db94ea8420 msgr2=0x55db94eb2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:41:48.603 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.603+0000 7f5794f80740 1 -- 172.21.15.100:0/2686771056 shutdown_connections 2024-05-10T11:41:48.603 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.603+0000 7f5794f80740 1 -- 172.21.15.100:0/2686771056 wait complete. 2024-05-10T11:41:48.605 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.606+0000 7f5794f80740 20 client.19908 trim_cache size 0 max 16384 2024-05-10T11:41:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:41:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:41:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:41:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:41:48.611 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.611+0000 7f5792264640 1 -- 172.21.15.100:0/189217868 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f57780b28a0 msgr2=0x7f57780b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:48.611 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.611+0000 7f5792264640 1 --2- 172.21.15.100:0/189217868 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f57780b28a0 0x7f57780b4d60 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f577c0e0560 tx=0x7f577c0e7040 comp rx=0 tx=0).stop 2024-05-10T11:41:48.611 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.611+0000 7f5792264640 1 -- 172.21.15.100:0/189217868 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f578c101910 msgr2=0x7f578c198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:48.611 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.611+0000 7f5792264640 1 --2- 172.21.15.100:0/189217868 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f578c101910 0x7f578c198aa0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f577c00e9c0 tx=0x7f577c00ee80 comp rx=0 tx=0).stop 2024-05-10T11:41:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.612+0000 7f5792264640 1 -- 172.21.15.100:0/189217868 shutdown_connections 2024-05-10T11:41:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.612+0000 7f5792264640 1 --2- 172.21.15.100:0/189217868 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f57780b28a0 0x7f57780b4d60 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.612+0000 7f5792264640 1 --2- 172.21.15.100:0/189217868 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f578c1034c0 0x7f578c1a0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.612+0000 7f5792264640 1 --2- 172.21.15.100:0/189217868 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f578c102b00 0x7f578c198fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.612+0000 7f5792264640 1 --2- 172.21.15.100:0/189217868 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f578c101910 0x7f578c198aa0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.612+0000 7f5792264640 1 -- 172.21.15.100:0/189217868 >> 172.21.15.100:0/189217868 conn(0x7f578c0fce80 msgr2=0x7f578c105e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:41:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.613+0000 7f5792264640 1 -- 172.21.15.100:0/189217868 shutdown_connections 2024-05-10T11:41:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:41:48.613+0000 7f5792264640 1 -- 172.21.15.100:0/189217868 wait complete. 2024-05-10T11:41:48.957 INFO:tasks.workunit.client.0.smithi100.stdout: 10 96624 8.90 MB/sec execute 462 sec latency 215.021 ms 2024-05-10T11:41:49.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:41:48 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/2775213420' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:41:49.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:41:48 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/2775213420' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:41:49.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:41:48 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/2775213420' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:41:49.957 INFO:tasks.workunit.client.0.smithi100.stdout: 10 96839 8.90 MB/sec execute 463 sec latency 87.083 ms 2024-05-10T11:41:50.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:41:49 smithi178 ceph-mon[33359]: pgmap v1124: 129 pgs: 129 active+clean; 563 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 3.7 MiB/s wr, 241 op/s 2024-05-10T11:41:50.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:41:49 smithi188 ceph-mon[34075]: pgmap v1124: 129 pgs: 129 active+clean; 563 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 3.7 MiB/s wr, 241 op/s 2024-05-10T11:41:50.287 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:41:49 smithi100 ceph-mon[30296]: pgmap v1124: 129 pgs: 129 active+clean; 563 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 3.7 MiB/s wr, 241 op/s 2024-05-10T11:41:50.509 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-10T11:41:50.869 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:41:50.957 INFO:tasks.workunit.client.0.smithi100.stdout: 10 97009 8.90 MB/sec execute 464 sec latency 107.706 ms 2024-05-10T11:41:51.957 INFO:tasks.workunit.client.0.smithi100.stdout: 10 97177 8.91 MB/sec execute 465 sec latency 91.264 ms 2024-05-10T11:41:52.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:41:51 smithi178 ceph-mon[33359]: pgmap v1125: 129 pgs: 129 active+clean; 562 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 959 KiB/s rd, 3.8 MiB/s wr, 269 op/s 2024-05-10T11:41:52.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:41:51 smithi188 ceph-mon[34075]: pgmap v1125: 129 pgs: 129 active+clean; 562 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 959 KiB/s rd, 3.8 MiB/s wr, 269 op/s 2024-05-10T11:41:52.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:41:51 smithi100 ceph-mon[30296]: pgmap v1125: 129 pgs: 129 active+clean; 562 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 959 KiB/s rd, 3.8 MiB/s wr, 269 op/s 2024-05-10T11:41:52.957 INFO:tasks.workunit.client.0.smithi100.stdout: 10 97344 8.92 MB/sec execute 466 sec latency 79.327 ms 2024-05-10T11:41:53.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.245+0000 7fe0676be700 1 -- 172.21.15.100:0/1307249979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0600779d0 msgr2=0x7fe060077db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:53.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.245+0000 7fe0676be700 1 --2- 172.21.15.100:0/1307249979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0600779d0 0x7fe060077db0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fe05c009990 tx=0x7fe05c009ca0 comp rx=0 tx=0).stop 2024-05-10T11:41:53.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.245+0000 7fe0676be700 1 -- 172.21.15.100:0/1307249979 shutdown_connections 2024-05-10T11:41:53.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.245+0000 7fe0676be700 1 --2- 172.21.15.100:0/1307249979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe060114380 0x7fe060116770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:53.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.245+0000 7fe0676be700 1 --2- 172.21.15.100:0/1307249979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0600779d0 0x7fe060077db0 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:53.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.245+0000 7fe0676be700 1 --2- 172.21.15.100:0/1307249979 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0600782f0 0x7fe060113e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:53.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.245+0000 7fe0676be700 1 -- 172.21.15.100:0/1307249979 >> 172.21.15.100:0/1307249979 conn(0x7fe0601011d0 msgr2=0x7fe0601035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:41:53.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.246+0000 7fe0676be700 1 -- 172.21.15.100:0/1307249979 shutdown_connections 2024-05-10T11:41:53.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.246+0000 7fe0676be700 1 -- 172.21.15.100:0/1307249979 wait complete. 2024-05-10T11:41:53.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.247+0000 7fe0676be700 1 Processor -- start 2024-05-10T11:41:53.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.247+0000 7fe0676be700 1 -- start start 2024-05-10T11:41:53.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.247+0000 7fe0676be700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0600779d0 0x7fe0601a0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:53.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.247+0000 7fe0676be700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0600782f0 0x7fe0601a05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:53.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.247+0000 7fe0676be700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe060114380 0x7fe0601a4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:53.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.247+0000 7fe0676be700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe060119ce0 con 0x7fe060114380 2024-05-10T11:41:53.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.247+0000 7fe0676be700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe060119b60 con 0x7fe0600782f0 2024-05-10T11:41:53.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.247+0000 7fe0676be700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe060119e60 con 0x7fe0600779d0 2024-05-10T11:41:53.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.247+0000 7fe06545a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0600779d0 0x7fe0601a0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:53.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.248+0000 7fe06545a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0600779d0 0x7fe0601a0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:57838/0 (socket says 172.21.15.100:57838) 2024-05-10T11:41:53.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.248+0000 7fe065c5b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe060114380 0x7fe0601a4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:53.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.248+0000 7fe064c59700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0600782f0 0x7fe0601a05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:53.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.248+0000 7fe065c5b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe060114380 0x7fe0601a4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37008/0 (socket says 172.21.15.100:37008) 2024-05-10T11:41:53.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.248+0000 7fe06545a700 1 -- 172.21.15.100:0/2229331748 learned_addr learned my addr 172.21.15.100:0/2229331748 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:41:53.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.248+0000 7fe065c5b700 1 -- 172.21.15.100:0/2229331748 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0600779d0 msgr2=0x7fe0601a0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:53.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.248+0000 7fe065c5b700 1 --2- 172.21.15.100:0/2229331748 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0600779d0 0x7fe0601a0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:53.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.248+0000 7fe065c5b700 1 -- 172.21.15.100:0/2229331748 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0600782f0 msgr2=0x7fe0601a05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:53.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.248+0000 7fe065c5b700 1 --2- 172.21.15.100:0/2229331748 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0600782f0 0x7fe0601a05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:53.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.248+0000 7fe065c5b700 1 -- 172.21.15.100:0/2229331748 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0601a5080 con 0x7fe060114380 2024-05-10T11:41:53.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.248+0000 7fe06545a700 1 --2- 172.21.15.100:0/2229331748 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0600779d0 0x7fe0601a0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:41:53.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.249+0000 7fe065c5b700 1 --2- 172.21.15.100:0/2229331748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe060114380 0x7fe0601a4940 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fe06007bcc0 tx=0x7fe05800ae20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:41:53.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.249+0000 7fe0567fc700 1 -- 172.21.15.100:0/2229331748 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe058017020 con 0x7fe060114380 2024-05-10T11:41:53.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.249+0000 7fe0676be700 1 -- 172.21.15.100:0/2229331748 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0601a5370 con 0x7fe060114380 2024-05-10T11:41:53.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.249+0000 7fe0567fc700 1 -- 172.21.15.100:0/2229331748 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe058004bb0 con 0x7fe060114380 2024-05-10T11:41:53.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.249+0000 7fe0567fc700 1 -- 172.21.15.100:0/2229331748 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe058005280 con 0x7fe060114380 2024-05-10T11:41:53.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.249+0000 7fe0676be700 1 -- 172.21.15.100:0/2229331748 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0601acbf0 con 0x7fe060114380 2024-05-10T11:41:53.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.252+0000 7fe0567fc700 1 -- 172.21.15.100:0/2229331748 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fe058005420 con 0x7fe060114380 2024-05-10T11:41:53.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.253+0000 7fe071f51b80 1 Processor -- start 2024-05-10T11:41:53.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.254+0000 7fe071f51b80 1 -- start start 2024-05-10T11:41:53.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.254+0000 7fe0567fc700 1 --2- 172.21.15.100:0/2229331748 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe04c0b2940 0x7fe04c0b4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:53.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.254+0000 7fe071f51b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609724f61b0 0x5609724f6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:53.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.254+0000 7fe071f51b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609724f6ad0 0x5609724f0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:53.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.254+0000 7fe071f51b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5609724f0d50 0x5609724f1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:53.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.254+0000 7fe071f51b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x560972442940 con 0x5609724f61b0 2024-05-10T11:41:53.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.254+0000 7fe071f51b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5609723a7da0 con 0x5609724f6ad0 2024-05-10T11:41:53.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.254+0000 7fe071f51b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56097243e900 con 0x5609724f0d50 2024-05-10T11:41:53.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.254+0000 7fe0567fc700 1 -- 172.21.15.100:0/2229331748 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fe0580e1c00 con 0x7fe060114380 2024-05-10T11:41:53.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.254+0000 7fe06545a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609724f61b0 0x5609724f6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:53.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.254+0000 7fe06545a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609724f61b0 0x5609724f6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37012/0 (socket says 172.21.15.100:37012) 2024-05-10T11:41:53.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.254+0000 7fe06545a700 1 -- 172.21.15.100:0/1242412650 learned_addr learned my addr 172.21.15.100:0/1242412650 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:41:53.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.254+0000 7fe065c5b700 1 --2- 172.21.15.100:0/1242412650 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5609724f0d50 0x5609724f1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:53.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.255+0000 7fe064c59700 1 --2- 172.21.15.100:0/1242412650 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609724f6ad0 0x5609724f0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:53.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.255+0000 7fe064c59700 1 --2- 172.21.15.100:0/2229331748 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe04c0b2940 0x7fe04c0b4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:53.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.255+0000 7fe065c5b700 1 -- 172.21.15.100:0/1242412650 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609724f6ad0 msgr2=0x5609724f0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:53.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.255+0000 7fe065c5b700 1 --2- 172.21.15.100:0/1242412650 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609724f6ad0 0x5609724f0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:53.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.255+0000 7fe065c5b700 1 -- 172.21.15.100:0/1242412650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609724f61b0 msgr2=0x5609724f6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:53.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.255+0000 7fe065c5b700 1 --2- 172.21.15.100:0/1242412650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609724f61b0 0x5609724f6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:53.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.255+0000 7fe065c5b700 1 -- 172.21.15.100:0/1242412650 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56097238f8d0 con 0x5609724f0d50 2024-05-10T11:41:53.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.255+0000 7fe065c5b700 1 --2- 172.21.15.100:0/1242412650 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5609724f0d50 0x5609724f1190 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe0580a4f70 tx=0x7fe0580a8910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eaa749a16f0cb004 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:41:53.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.255+0000 7fe04b7fe700 1 -- 172.21.15.100:0/1242412650 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0580a8a00 con 0x5609724f0d50 2024-05-10T11:41:53.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.255+0000 7fe04b7fe700 1 -- 172.21.15.100:0/1242412650 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe0580a95f0 con 0x5609724f0d50 2024-05-10T11:41:53.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.255+0000 7fe04b7fe700 1 -- 172.21.15.100:0/1242412650 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe058005710 con 0x5609724f0d50 2024-05-10T11:41:53.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.256+0000 7fe064c59700 1 --2- 172.21.15.100:0/2229331748 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe04c0b2940 0x7fe04c0b4e00 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x5609724ed450 tx=0x7fe050009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:41:53.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.256+0000 7fe071f51b80 1 -- 172.21.15.100:0/1242412650 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5609724f0d50 msgr2=0x5609724f1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:53.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.256+0000 7fe071f51b80 1 --2- 172.21.15.100:0/1242412650 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5609724f0d50 0x5609724f1190 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe0580a4f70 tx=0x7fe0580a8910 comp rx=0 tx=0).stop 2024-05-10T11:41:53.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.256+0000 7fe071f51b80 1 -- 172.21.15.100:0/1242412650 shutdown_connections 2024-05-10T11:41:53.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.256+0000 7fe071f51b80 1 --2- 172.21.15.100:0/1242412650 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5609724f0d50 0x5609724f1190 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:53.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.256+0000 7fe071f51b80 1 --2- 172.21.15.100:0/1242412650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609724f61b0 0x5609724f6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:53.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.256+0000 7fe071f51b80 1 --2- 172.21.15.100:0/1242412650 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609724f6ad0 0x5609724f0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:53.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.256+0000 7fe071f51b80 1 -- 172.21.15.100:0/1242412650 >> 172.21.15.100:0/1242412650 conn(0x56097241cb70 msgr2=0x5609724eb2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:41:53.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.256+0000 7fe071f51b80 1 -- 172.21.15.100:0/1242412650 shutdown_connections 2024-05-10T11:41:53.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.256+0000 7fe071f51b80 1 -- 172.21.15.100:0/1242412650 wait complete. 2024-05-10T11:41:53.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.257+0000 7fe071f51b80 1 -- start start 2024-05-10T11:41:53.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.257+0000 7fe071f51b80 1 Processor -- start 2024-05-10T11:41:53.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.257+0000 7fe071f51b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609724f0d50 0x5609724fc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:53.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.257+0000 7fe071f51b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609724f61b0 0x5609724f9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:53.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.257+0000 7fe071f51b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5609724f6ad0 0x5609724fa050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:53.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.257+0000 7fe071f51b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5609724fd3f0 con 0x5609724f61b0 2024-05-10T11:41:53.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.257+0000 7fe071f51b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5609723fbed0 con 0x5609724f0d50 2024-05-10T11:41:53.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.257+0000 7fe071f51b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5609724fd570 con 0x5609724f6ad0 2024-05-10T11:41:53.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.257+0000 7fe071f51b80 10 client.0 _ll_register_callbacks cb 0x5609723d65b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:41:53.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.258+0000 7fe064c59700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609724f61b0 0x5609724f9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:53.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.258+0000 7fe064c59700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609724f61b0 0x5609724f9b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37016/0 (socket says 172.21.15.100:37016) 2024-05-10T11:41:53.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.258+0000 7fe064c59700 1 -- 172.21.15.100:0/731128467 learned_addr learned my addr 172.21.15.100:0/731128467 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:41:53.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.258+0000 7fe065c5b700 1 --2- 172.21.15.100:0/731128467 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5609724f6ad0 0x5609724fa050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:53.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.258+0000 7fe06545a700 1 --2- 172.21.15.100:0/731128467 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609724f0d50 0x5609724fc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:53.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.258+0000 7fe065c5b700 1 -- 172.21.15.100:0/731128467 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609724f0d50 msgr2=0x5609724fc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:53.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.258+0000 7fe065c5b700 1 --2- 172.21.15.100:0/731128467 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609724f0d50 0x5609724fc770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:53.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.258+0000 7fe065c5b700 1 -- 172.21.15.100:0/731128467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609724f61b0 msgr2=0x5609724f9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:53.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.258+0000 7fe065c5b700 1 --2- 172.21.15.100:0/731128467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609724f61b0 0x5609724f9b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:53.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.258+0000 7fe065c5b700 1 -- 172.21.15.100:0/731128467 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609723f9760 con 0x5609724f6ad0 2024-05-10T11:41:53.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.258+0000 7fe064c59700 1 --2- 172.21.15.100:0/731128467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609724f61b0 0x5609724f9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:41:53.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.258+0000 7fe065c5b700 1 --2- 172.21.15.100:0/731128467 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5609724f6ad0 0x5609724fa050 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fe0580ad460 tx=0x7fe0580a8a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8b4e97877c7f794 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:41:53.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.258+0000 7fe048ff9700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:41:53.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.258+0000 7fe048ff9700 1 -- 172.21.15.100:0/731128467 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0580ad2a0 con 0x5609724f6ad0 2024-05-10T11:41:53.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.258+0000 7fe048ff9700 1 -- 172.21.15.100:0/731128467 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fe0580a54e0 con 0x5609724f6ad0 2024-05-10T11:41:53.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.259+0000 7fe071f51b80 1 -- 172.21.15.100:0/731128467 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0580a95f0 con 0x5609724f6ad0 2024-05-10T11:41:53.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.259+0000 7fe048ff9700 1 -- 172.21.15.100:0/731128467 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fe0580b1cc0 con 0x5609724f6ad0 2024-05-10T11:41:53.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.259+0000 7fe071f51b80 10 client.29284 fetch_fsmap learned FSMap version 41 2024-05-10T11:41:53.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.259+0000 7fe071f51b80 1 -- 172.21.15.100:0/731128467 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5609724fd3f0 con 0x5609724f6ad0 2024-05-10T11:41:53.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.259+0000 7fe048ff9700 1 -- 172.21.15.100:0/731128467 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe0580a95f0 con 0x5609724f6ad0 2024-05-10T11:41:53.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.259+0000 7fe048ff9700 1 -- 172.21.15.100:0/731128467 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fe058012070 con 0x5609724f6ad0 2024-05-10T11:41:53.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.259+0000 7fe071f51b80 10 client.29284 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:41:53.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.259+0000 7fe071f51b80 10 client.29284 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-10T11:41:53.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.259+0000 7fe071f51b80 20 client.29284 populate_metadata read hostname 'smithi100' 2024-05-10T11:41:53.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.259+0000 7fe071f51b80 1 --2- 172.21.15.100:0/731128467 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x560972509fd0 0x56097250c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:53.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.259+0000 7fe071f51b80 4 client.29284 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:41:53.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.259+0000 7fe071f51b80 1 -- 172.21.15.100:0/731128467 --> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560972450320 con 0x560972509fd0 2024-05-10T11:41:53.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.260+0000 7fe06545a700 1 --2- 172.21.15.100:0/731128467 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x560972509fd0 0x56097250c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:53.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.260+0000 7fe048ff9700 10 client.29284 ms_handle_connect on v2:172.21.15.188:6836/1943342620 2024-05-10T11:41:53.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.260+0000 7fe06545a700 1 --2- 172.21.15.100:0/731128467 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x560972509fd0 0x56097250c3c0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1f5217c40fe4ce15 server_cookie=381db52fec4eacbc in_seq=0 out_seq=0 2024-05-10T11:41:53.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.261+0000 7fe048ff9700 1 -- 172.21.15.100:0/731128467 <== mds.2 v2:172.21.15.188:6836/1943342620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560972450320 con 0x560972509fd0 2024-05-10T11:41:53.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.261+0000 7fe048ff9700 10 client.29284 handle_command_reply: tid=0 2024-05-10T11:41:53.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.261+0000 7fe071f51b80 1 client.29284 shutdown 2024-05-10T11:41:53.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.261+0000 7fe071f51b80 1 -- 172.21.15.100:0/731128467 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5609724f6ad0 msgr2=0x5609724fa050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:53.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.261+0000 7fe071f51b80 1 --2- 172.21.15.100:0/731128467 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5609724f6ad0 0x5609724fa050 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fe0580ad460 tx=0x7fe0580a8a70 comp rx=0 tx=0).stop 2024-05-10T11:41:53.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.261+0000 7fe071f51b80 1 -- 172.21.15.100:0/731128467 shutdown_connections 2024-05-10T11:41:53.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.261+0000 7fe071f51b80 1 --2- 172.21.15.100:0/731128467 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x560972509fd0 0x56097250c3c0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:53.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.261+0000 7fe071f51b80 1 --2- 172.21.15.100:0/731128467 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5609724f6ad0 0x5609724fa050 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:53.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.261+0000 7fe071f51b80 1 --2- 172.21.15.100:0/731128467 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609724f0d50 0x5609724fc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:53.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.261+0000 7fe048ff9700 0 client.29284 ms_handle_reset on v2:172.21.15.188:6836/1943342620 2024-05-10T11:41:53.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.261+0000 7fe071f51b80 1 --2- 172.21.15.100:0/731128467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609724f61b0 0x5609724f9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:53.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.261+0000 7fe071f51b80 1 -- 172.21.15.100:0/731128467 >> 172.21.15.100:0/731128467 conn(0x56097241cb70 msgr2=0x5609724e5b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:41:53.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.262+0000 7fe071f51b80 1 -- 172.21.15.100:0/731128467 shutdown_connections 2024-05-10T11:41:53.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.262+0000 7fe071f51b80 1 -- 172.21.15.100:0/731128467 wait complete. 2024-05-10T11:41:53.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.262+0000 7fe071f51b80 20 client.29284 trim_cache size 0 max 16384 2024-05-10T11:41:53.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.281+0000 7fe071f51b80 1 Processor -- start 2024-05-10T11:41:53.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.281+0000 7fe071f51b80 1 -- start start 2024-05-10T11:41:53.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.281+0000 7fe071f51b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609724f0d50 0x5609724f0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:53.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.281+0000 7fe071f51b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5609724f61b0 0x560972506480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:53.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.281+0000 7fe071f51b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609724f6ad0 0x5609724f8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:53.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.281+0000 7fe071f51b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe0601b7df0 con 0x5609724f0d50 2024-05-10T11:41:53.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.281+0000 7fe071f51b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe0601b5680 con 0x5609724f6ad0 2024-05-10T11:41:53.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.281+0000 7fe071f51b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe0601b34d0 con 0x5609724f61b0 2024-05-10T11:41:53.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.281+0000 7fe06545a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609724f0d50 0x5609724f0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:53.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.282+0000 7fe06545a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609724f0d50 0x5609724f0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37024/0 (socket says 172.21.15.100:37024) 2024-05-10T11:41:53.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.282+0000 7fe06545a700 1 -- 172.21.15.100:0/108719655 learned_addr learned my addr 172.21.15.100:0/108719655 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:41:53.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.282+0000 7fe064c59700 1 --2- 172.21.15.100:0/108719655 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5609724f61b0 0x560972506480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:53.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.282+0000 7fe065c5b700 1 --2- 172.21.15.100:0/108719655 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609724f6ad0 0x5609724f8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:53.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.282+0000 7fe06545a700 1 -- 172.21.15.100:0/108719655 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5609724f61b0 msgr2=0x560972506480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:53.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.282+0000 7fe06545a700 1 --2- 172.21.15.100:0/108719655 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5609724f61b0 0x560972506480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:53.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.282+0000 7fe06545a700 1 -- 172.21.15.100:0/108719655 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609724f6ad0 msgr2=0x5609724f8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:53.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.282+0000 7fe06545a700 1 --2- 172.21.15.100:0/108719655 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609724f6ad0 0x5609724f8cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:53.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.282+0000 7fe06545a700 1 -- 172.21.15.100:0/108719655 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609724f67e0 con 0x5609724f0d50 2024-05-10T11:41:53.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.282+0000 7fe06545a700 1 --2- 172.21.15.100:0/108719655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609724f0d50 0x5609724f0810 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fe05c006010 tx=0x7fe05c01a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d9f67c9223390a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:41:53.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.282+0000 7fe04affd700 1 -- 172.21.15.100:0/108719655 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe05c005980 con 0x5609724f0d50 2024-05-10T11:41:53.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.283+0000 7fe04affd700 1 -- 172.21.15.100:0/108719655 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe05c005560 con 0x5609724f0d50 2024-05-10T11:41:53.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.283+0000 7fe04affd700 1 -- 172.21.15.100:0/108719655 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe05c00f460 con 0x5609724f0d50 2024-05-10T11:41:53.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.284+0000 7fe071f51b80 1 -- 172.21.15.100:0/108719655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609724f0d50 msgr2=0x5609724f0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:53.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.284+0000 7fe071f51b80 1 --2- 172.21.15.100:0/108719655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609724f0d50 0x5609724f0810 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fe05c006010 tx=0x7fe05c01a040 comp rx=0 tx=0).stop 2024-05-10T11:41:53.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.284+0000 7fe071f51b80 1 -- 172.21.15.100:0/108719655 shutdown_connections 2024-05-10T11:41:53.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.284+0000 7fe071f51b80 1 --2- 172.21.15.100:0/108719655 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609724f6ad0 0x5609724f8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:53.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.284+0000 7fe071f51b80 1 --2- 172.21.15.100:0/108719655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609724f0d50 0x5609724f0810 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:53.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.284+0000 7fe071f51b80 1 --2- 172.21.15.100:0/108719655 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5609724f61b0 0x560972506480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:53.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.284+0000 7fe071f51b80 1 -- 172.21.15.100:0/108719655 >> 172.21.15.100:0/108719655 conn(0x56097241cb70 msgr2=0x5609724f4c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:41:53.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.284+0000 7fe071f51b80 1 -- 172.21.15.100:0/108719655 shutdown_connections 2024-05-10T11:41:53.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.284+0000 7fe071f51b80 1 -- 172.21.15.100:0/108719655 wait complete. 2024-05-10T11:41:53.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.284+0000 7fe071f51b80 1 -- start start 2024-05-10T11:41:53.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.284+0000 7fe071f51b80 1 Processor -- start 2024-05-10T11:41:53.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.285+0000 7fe071f51b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609724f0d50 0x560972502c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:53.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.285+0000 7fe071f51b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609724f61b0 0x560972510de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:53.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.285+0000 7fe071f51b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5609724f6ad0 0x560972511320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:53.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.285+0000 7fe071f51b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe0601b3b40 con 0x5609724f0d50 2024-05-10T11:41:53.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.285+0000 7fe071f51b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe0601b4fb0 con 0x5609724f61b0 2024-05-10T11:41:53.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.285+0000 7fe071f51b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe0601b4210 con 0x5609724f6ad0 2024-05-10T11:41:53.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.285+0000 7fe06545a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609724f0d50 0x560972502c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:53.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.285+0000 7fe071f51b80 10 client.0 _ll_register_callbacks cb 0x5609723d65b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:41:53.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.285+0000 7fe06545a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609724f0d50 0x560972502c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37030/0 (socket says 172.21.15.100:37030) 2024-05-10T11:41:53.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.285+0000 7fe064c59700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609724f61b0 0x560972510de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:53.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.285+0000 7fe06545a700 1 -- 172.21.15.100:0/835703216 learned_addr learned my addr 172.21.15.100:0/835703216 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:41:53.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.285+0000 7fe065c5b700 1 --2- 172.21.15.100:0/835703216 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5609724f6ad0 0x560972511320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:53.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.285+0000 7fe06545a700 1 -- 172.21.15.100:0/835703216 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5609724f6ad0 msgr2=0x560972511320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:53.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.285+0000 7fe06545a700 1 --2- 172.21.15.100:0/835703216 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5609724f6ad0 0x560972511320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:53.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.285+0000 7fe06545a700 1 -- 172.21.15.100:0/835703216 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609724f61b0 msgr2=0x560972510de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:53.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.285+0000 7fe06545a700 1 --2- 172.21.15.100:0/835703216 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609724f61b0 0x560972510de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:53.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.285+0000 7fe06545a700 1 -- 172.21.15.100:0/835703216 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560972511990 con 0x5609724f0d50 2024-05-10T11:41:53.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.286+0000 7fe064c59700 1 --2- 172.21.15.100:0/835703216 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609724f61b0 0x560972510de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:41:53.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.286+0000 7fe06545a700 1 --2- 172.21.15.100:0/835703216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609724f0d50 0x560972502c00 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fe05c005920 tx=0x7fe05c0039f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92a6529b005b93c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:41:53.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.286+0000 7fe048ff9700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:41:53.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.286+0000 7fe048ff9700 1 -- 172.21.15.100:0/835703216 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe05c034040 con 0x5609724f0d50 2024-05-10T11:41:53.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.286+0000 7fe071f51b80 1 -- 172.21.15.100:0/835703216 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe05c005560 con 0x5609724f0d50 2024-05-10T11:41:53.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.286+0000 7fe048ff9700 1 -- 172.21.15.100:0/835703216 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fe05c0050d0 con 0x5609724f0d50 2024-05-10T11:41:53.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.286+0000 7fe048ff9700 1 -- 172.21.15.100:0/835703216 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fe05c005980 con 0x5609724f0d50 2024-05-10T11:41:53.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.286+0000 7fe071f51b80 10 client.19926 fetch_fsmap learned FSMap version 41 2024-05-10T11:41:53.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.286+0000 7fe071f51b80 1 -- 172.21.15.100:0/835703216 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x560972574430 con 0x5609724f0d50 2024-05-10T11:41:53.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.286+0000 7fe048ff9700 1 -- 172.21.15.100:0/835703216 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe05c005560 con 0x5609724f0d50 2024-05-10T11:41:53.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.287+0000 7fe048ff9700 1 -- 172.21.15.100:0/835703216 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fe05c040030 con 0x5609724f0d50 2024-05-10T11:41:53.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.287+0000 7fe071f51b80 10 client.19926 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:41:53.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.287+0000 7fe071f51b80 10 client.19926 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-10T11:41:53.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.287+0000 7fe071f51b80 20 client.19926 populate_metadata read hostname 'smithi100' 2024-05-10T11:41:53.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.287+0000 7fe071f51b80 1 --2- 172.21.15.100:0/835703216 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x560972509fd0 0x560972576aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:53.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.287+0000 7fe071f51b80 4 client.19926 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:41:53.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.287+0000 7fe071f51b80 1 -- 172.21.15.100:0/835703216 --> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56097244a3c0 con 0x560972509fd0 2024-05-10T11:41:53.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.287+0000 7fe064c59700 1 --2- 172.21.15.100:0/835703216 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x560972509fd0 0x560972576aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:53.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.287+0000 7fe048ff9700 10 client.19926 ms_handle_connect on v2:172.21.15.188:6836/1943342620 2024-05-10T11:41:53.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.288+0000 7fe064c59700 1 --2- 172.21.15.100:0/835703216 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x560972509fd0 0x560972576aa0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=803466265f6d4808 server_cookie=975950a9741a7449 in_seq=0 out_seq=0 2024-05-10T11:41:53.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.288+0000 7fe048ff9700 1 -- 172.21.15.100:0/835703216 <== mds.2 v2:172.21.15.188:6836/1943342620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56097244a3c0 con 0x560972509fd0 2024-05-10T11:41:53.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.288+0000 7fe048ff9700 10 client.19926 handle_command_reply: tid=0 2024-05-10T11:41:53.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.288+0000 7fe071f51b80 1 client.19926 shutdown 2024-05-10T11:41:53.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.288+0000 7fe071f51b80 1 -- 172.21.15.100:0/835703216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609724f0d50 msgr2=0x560972502c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:53.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.288+0000 7fe071f51b80 1 --2- 172.21.15.100:0/835703216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609724f0d50 0x560972502c00 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fe05c005920 tx=0x7fe05c0039f0 comp rx=0 tx=0).stop 2024-05-10T11:41:53.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.288+0000 7fe071f51b80 1 -- 172.21.15.100:0/835703216 shutdown_connections 2024-05-10T11:41:53.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.288+0000 7fe071f51b80 1 --2- 172.21.15.100:0/835703216 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x560972509fd0 0x560972576aa0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:53.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.288+0000 7fe071f51b80 1 --2- 172.21.15.100:0/835703216 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5609724f6ad0 0x560972511320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:53.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.288+0000 7fe071f51b80 1 --2- 172.21.15.100:0/835703216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609724f0d50 0x560972502c00 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:53.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.288+0000 7fe071f51b80 1 --2- 172.21.15.100:0/835703216 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609724f61b0 0x560972510de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:53.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.288+0000 7fe071f51b80 1 -- 172.21.15.100:0/835703216 >> 172.21.15.100:0/835703216 conn(0x56097241cb70 msgr2=0x5609724f8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:41:53.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.288+0000 7fe048ff9700 0 client.19926 ms_handle_reset on v2:172.21.15.188:6836/1943342620 2024-05-10T11:41:53.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.289+0000 7fe071f51b80 1 -- 172.21.15.100:0/835703216 shutdown_connections 2024-05-10T11:41:53.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.289+0000 7fe071f51b80 1 -- 172.21.15.100:0/835703216 wait complete. 2024-05-10T11:41:53.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.289+0000 7fe071f51b80 20 client.19926 trim_cache size 0 max 16384 2024-05-10T11:41:53.293 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:41:53.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.289+0000 7fe0676be700 1 -- 172.21.15.100:0/2229331748 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe04c0b2940 msgr2=0x7fe04c0b4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:53.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.289+0000 7fe0676be700 1 --2- 172.21.15.100:0/2229331748 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe04c0b2940 0x7fe04c0b4e00 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x5609724ed450 tx=0x7fe050009190 comp rx=0 tx=0).stop 2024-05-10T11:41:53.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.289+0000 7fe0676be700 1 -- 172.21.15.100:0/2229331748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe060114380 msgr2=0x7fe0601a4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:53.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.290+0000 7fe0676be700 1 --2- 172.21.15.100:0/2229331748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe060114380 0x7fe0601a4940 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fe06007bcc0 tx=0x7fe05800ae20 comp rx=0 tx=0).stop 2024-05-10T11:41:53.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.290+0000 7fe0676be700 1 -- 172.21.15.100:0/2229331748 shutdown_connections 2024-05-10T11:41:53.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.290+0000 7fe0676be700 1 --2- 172.21.15.100:0/2229331748 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe04c0b2940 0x7fe04c0b4e00 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:53.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.290+0000 7fe0676be700 1 --2- 172.21.15.100:0/2229331748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe060114380 0x7fe0601a4940 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:53.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.290+0000 7fe0676be700 1 --2- 172.21.15.100:0/2229331748 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0600779d0 0x7fe0601a0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:53.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.290+0000 7fe0676be700 1 --2- 172.21.15.100:0/2229331748 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0600782f0 0x7fe0601a05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:53.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.290+0000 7fe0676be700 1 -- 172.21.15.100:0/2229331748 >> 172.21.15.100:0/2229331748 conn(0x7fe0601011d0 msgr2=0x7fe0601035c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:41:53.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.290+0000 7fe0676be700 1 -- 172.21.15.100:0/2229331748 shutdown_connections 2024-05-10T11:41:53.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:53.290+0000 7fe0676be700 1 -- 172.21.15.100:0/2229331748 wait complete. 2024-05-10T11:41:53.957 INFO:tasks.workunit.client.0.smithi100.stdout: 10 97529 8.92 MB/sec execute 467 sec latency 80.173 ms 2024-05-10T11:41:54.013 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:41:53 smithi100 ceph-mon[30296]: pgmap v1126: 129 pgs: 129 active+clean; 561 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 3.5 MiB/s wr, 232 op/s 2024-05-10T11:41:54.050 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-10T11:41:54.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:41:53 smithi178 ceph-mon[33359]: pgmap v1126: 129 pgs: 129 active+clean; 561 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 3.5 MiB/s wr, 232 op/s 2024-05-10T11:41:54.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:41:53 smithi188 ceph-mon[34075]: pgmap v1126: 129 pgs: 129 active+clean; 561 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 3.5 MiB/s wr, 232 op/s 2024-05-10T11:41:54.409 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:41:54.957 INFO:tasks.workunit.client.0.smithi100.stdout: 10 97725 8.93 MB/sec execute 468 sec latency 64.957 ms 2024-05-10T11:41:55.957 INFO:tasks.workunit.client.0.smithi100.stdout: 10 97894 8.92 MB/sec execute 469 sec latency 64.468 ms 2024-05-10T11:41:56.231 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:41:55 smithi100 ceph-mon[30296]: pgmap v1127: 129 pgs: 129 active+clean; 561 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 3.8 MiB/s wr, 252 op/s 2024-05-10T11:41:56.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:41:55 smithi178 ceph-mon[33359]: pgmap v1127: 129 pgs: 129 active+clean; 561 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 3.8 MiB/s wr, 252 op/s 2024-05-10T11:41:56.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:41:55 smithi188 ceph-mon[34075]: pgmap v1127: 129 pgs: 129 active+clean; 561 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 3.8 MiB/s wr, 252 op/s 2024-05-10T11:41:56.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.515+0000 7f0f8f03b700 1 -- 172.21.15.100:0/1889438805 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f88108400 msgr2=0x7f0f881087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:56.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.515+0000 7f0f8f03b700 1 --2- 172.21.15.100:0/1889438805 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f88108400 0x7f0f881087e0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c009990 tx=0x7f0f7c009ca0 comp rx=0 tx=0).stop 2024-05-10T11:41:56.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.516+0000 7f0f8f03b700 1 -- 172.21.15.100:0/1889438805 shutdown_connections 2024-05-10T11:41:56.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.516+0000 7f0f8f03b700 1 --2- 172.21.15.100:0/1889438805 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f88102d50 0x7f0f8810f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:56.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.516+0000 7f0f8f03b700 1 --2- 172.21.15.100:0/1889438805 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f88108400 0x7f0f881087e0 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:56.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.516+0000 7f0f8f03b700 1 --2- 172.21.15.100:0/1889438805 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f88102390 0x7f0f88102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:56.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.516+0000 7f0f8f03b700 1 -- 172.21.15.100:0/1889438805 >> 172.21.15.100:0/1889438805 conn(0x7f0f88079310 msgr2=0x7f0f88079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:41:56.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.516+0000 7f0f8f03b700 1 -- 172.21.15.100:0/1889438805 shutdown_connections 2024-05-10T11:41:56.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.516+0000 7f0f8f03b700 1 -- 172.21.15.100:0/1889438805 wait complete. 2024-05-10T11:41:56.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.517+0000 7f0f8f03b700 1 Processor -- start 2024-05-10T11:41:56.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.517+0000 7f0f8f03b700 1 -- start start 2024-05-10T11:41:56.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.517+0000 7f0f8f03b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f88102390 0x7f0f8819f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:56.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.517+0000 7f0f8f03b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f88102d50 0x7f0f8819fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:56.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.517+0000 7f0f8f03b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f88108400 0x7f0f88199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:56.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.517+0000 7f0f8f03b700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0f88112420 con 0x7f0f88102d50 2024-05-10T11:41:56.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.517+0000 7f0f8f03b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0f881122a0 con 0x7f0f88108400 2024-05-10T11:41:56.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.517+0000 7f0f8f03b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0f881125a0 con 0x7f0f88102390 2024-05-10T11:41:56.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.517+0000 7f0f87fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f88102d50 0x7f0f8819fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:56.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.517+0000 7f0f8cdd7700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f88102390 0x7f0f8819f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:56.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.517+0000 7f0f8d5d8700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f88108400 0x7f0f88199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:56.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.517+0000 7f0f8cdd7700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f88102390 0x7f0f8819f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:47316/0 (socket says 172.21.15.100:47316) 2024-05-10T11:41:56.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.517+0000 7f0f8d5d8700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f88108400 0x7f0f88199930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:59424/0 (socket says 172.21.15.100:59424) 2024-05-10T11:41:56.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.517+0000 7f0f8cdd7700 1 -- 172.21.15.100:0/435073492 learned_addr learned my addr 172.21.15.100:0/435073492 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:41:56.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.518+0000 7f0f8cdd7700 1 -- 172.21.15.100:0/435073492 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f88108400 msgr2=0x7f0f88199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:56.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.518+0000 7f0f8cdd7700 1 --2- 172.21.15.100:0/435073492 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f88108400 0x7f0f88199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:56.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.518+0000 7f0f8cdd7700 1 -- 172.21.15.100:0/435073492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f88102d50 msgr2=0x7f0f8819fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:56.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.518+0000 7f0f8cdd7700 1 --2- 172.21.15.100:0/435073492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f88102d50 0x7f0f8819fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:56.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.518+0000 7f0f8cdd7700 1 -- 172.21.15.100:0/435073492 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f8819a090 con 0x7f0f88102390 2024-05-10T11:41:56.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.518+0000 7f0f87fff700 1 --2- 172.21.15.100:0/435073492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f88102d50 0x7f0f8819fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:41:56.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.518+0000 7f0f8cdd7700 1 --2- 172.21.15.100:0/435073492 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f88102390 0x7f0f8819f720 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c00b650 tx=0x7f0f7c0047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:41:56.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.519+0000 7f0f85ffb700 1 -- 172.21.15.100:0/435073492 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c027070 con 0x7f0f88102390 2024-05-10T11:41:56.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.519+0000 7f0f8f03b700 1 -- 172.21.15.100:0/435073492 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f8819a320 con 0x7f0f88102390 2024-05-10T11:41:56.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.519+0000 7f0f85ffb700 1 -- 172.21.15.100:0/435073492 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0f7c022470 con 0x7f0f88102390 2024-05-10T11:41:56.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.519+0000 7f0f85ffb700 1 -- 172.21.15.100:0/435073492 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c017650 con 0x7f0f88102390 2024-05-10T11:41:56.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.519+0000 7f0f8f03b700 1 -- 172.21.15.100:0/435073492 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f88109210 con 0x7f0f88102390 2024-05-10T11:41:56.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.523+0000 7f0f85ffb700 1 -- 172.21.15.100:0/435073492 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f0f7c009d10 con 0x7f0f88102390 2024-05-10T11:41:56.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.524+0000 7f0f85ffb700 1 --2- 172.21.15.100:0/435073492 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0f780b2880 0x7f0f780b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:56.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.524+0000 7f0f998ceb80 1 Processor -- start 2024-05-10T11:41:56.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.524+0000 7f0f998ceb80 1 -- start start 2024-05-10T11:41:56.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.524+0000 7f0f85ffb700 1 -- 172.21.15.100:0/435073492 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f0f7c0e30e0 con 0x7f0f88102390 2024-05-10T11:41:56.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.524+0000 7f0f87fff700 1 --2- 172.21.15.100:0/435073492 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0f780b2880 0x7f0f780b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:56.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.525+0000 7f0f998ceb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ba3e58e1b0 0x55ba3e58e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:56.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.525+0000 7f0f998ceb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba3e58ead0 0x55ba3e588810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:56.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.525+0000 7f0f998ceb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba3e588d50 0x55ba3e589190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:56.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.525+0000 7f0f8cdd7700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ba3e58e1b0 0x55ba3e58e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:56.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.525+0000 7f0f87fff700 1 --2- 172.21.15.100:0/435073492 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0f780b2880 0x7f0f780b4d40 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f0f88079840 tx=0x7f0f74009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:41:56.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.525+0000 7f0f8cdd7700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ba3e58e1b0 0x55ba3e58e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:47318/0 (socket says 172.21.15.100:47318) 2024-05-10T11:41:56.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.525+0000 7f0f87fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba3e58ead0 0x55ba3e588810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:56.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.525+0000 7f0f87fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba3e58ead0 0x55ba3e588810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57322/0 (socket says 172.21.15.100:57322) 2024-05-10T11:41:56.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.525+0000 7f0f8cdd7700 1 -- 172.21.15.100:0/2552756606 learned_addr learned my addr 172.21.15.100:0/2552756606 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:41:56.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.525+0000 7f0f998ceb80 1 -- 172.21.15.100:0/2552756606 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55ba3e4da940 con 0x55ba3e58ead0 2024-05-10T11:41:56.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.525+0000 7f0f998ceb80 1 -- 172.21.15.100:0/2552756606 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ba3e43fda0 con 0x55ba3e588d50 2024-05-10T11:41:56.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.525+0000 7f0f998ceb80 1 -- 172.21.15.100:0/2552756606 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55ba3e4d6900 con 0x55ba3e58e1b0 2024-05-10T11:41:56.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.525+0000 7f0f8d5d8700 1 --2- 172.21.15.100:0/2552756606 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba3e588d50 0x55ba3e589190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:56.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.526+0000 7f0f8cdd7700 1 -- 172.21.15.100:0/2552756606 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba3e588d50 msgr2=0x55ba3e589190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:56.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.526+0000 7f0f8cdd7700 1 --2- 172.21.15.100:0/2552756606 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba3e588d50 0x55ba3e589190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:56.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.526+0000 7f0f8cdd7700 1 -- 172.21.15.100:0/2552756606 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba3e58ead0 msgr2=0x55ba3e588810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:56.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.526+0000 7f0f8cdd7700 1 --2- 172.21.15.100:0/2552756606 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba3e58ead0 0x55ba3e588810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:56.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.526+0000 7f0f8cdd7700 1 -- 172.21.15.100:0/2552756606 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba3e4278d0 con 0x55ba3e58e1b0 2024-05-10T11:41:56.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.526+0000 7f0f8d5d8700 1 --2- 172.21.15.100:0/2552756606 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba3e588d50 0x55ba3e589190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:41:56.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.526+0000 7f0f87fff700 1 --2- 172.21.15.100:0/2552756606 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba3e58ead0 0x55ba3e588810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:41:56.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.526+0000 7f0f8cdd7700 1 --2- 172.21.15.100:0/2552756606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ba3e58e1b0 0x55ba3e58e590 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c0a6450 tx=0x7f0f7c0a9da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e30354365335de server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:41:56.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.526+0000 7f0f72ffd700 1 -- 172.21.15.100:0/2552756606 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c0aaad0 con 0x55ba3e58e1b0 2024-05-10T11:41:56.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.526+0000 7f0f72ffd700 1 -- 172.21.15.100:0/2552756606 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0f7c0aa3d0 con 0x55ba3e58e1b0 2024-05-10T11:41:56.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.527+0000 7f0f72ffd700 1 -- 172.21.15.100:0/2552756606 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c0aa890 con 0x55ba3e58e1b0 2024-05-10T11:41:56.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.527+0000 7f0f998ceb80 1 -- 172.21.15.100:0/2552756606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ba3e58e1b0 msgr2=0x55ba3e58e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:56.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.527+0000 7f0f998ceb80 1 --2- 172.21.15.100:0/2552756606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ba3e58e1b0 0x55ba3e58e590 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c0a6450 tx=0x7f0f7c0a9da0 comp rx=0 tx=0).stop 2024-05-10T11:41:56.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.527+0000 7f0f998ceb80 1 -- 172.21.15.100:0/2552756606 shutdown_connections 2024-05-10T11:41:56.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.527+0000 7f0f998ceb80 1 --2- 172.21.15.100:0/2552756606 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba3e588d50 0x55ba3e589190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:56.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.528+0000 7f0f998ceb80 1 --2- 172.21.15.100:0/2552756606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ba3e58e1b0 0x55ba3e58e590 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:56.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.528+0000 7f0f998ceb80 1 --2- 172.21.15.100:0/2552756606 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba3e58ead0 0x55ba3e588810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:56.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.528+0000 7f0f998ceb80 1 -- 172.21.15.100:0/2552756606 >> 172.21.15.100:0/2552756606 conn(0x55ba3e4b4b70 msgr2=0x55ba3e5832a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:41:56.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.528+0000 7f0f998ceb80 1 -- 172.21.15.100:0/2552756606 shutdown_connections 2024-05-10T11:41:56.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.528+0000 7f0f998ceb80 1 -- 172.21.15.100:0/2552756606 wait complete. 2024-05-10T11:41:56.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.528+0000 7f0f998ceb80 1 -- start start 2024-05-10T11:41:56.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.529+0000 7f0f998ceb80 1 Processor -- start 2024-05-10T11:41:56.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.529+0000 7f0f998ceb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba3e588d50 0x55ba3e594650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:56.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.529+0000 7f0f998ceb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba3e58e1b0 0x55ba3e5919f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:56.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.529+0000 7f0f998ceb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ba3e58ead0 0x55ba3e591f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:56.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.529+0000 7f0f8cdd7700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba3e588d50 0x55ba3e594650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:56.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.530+0000 7f0f8cdd7700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba3e588d50 0x55ba3e594650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:59438/0 (socket says 172.21.15.100:59438) 2024-05-10T11:41:56.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.530+0000 7f0f8cdd7700 1 -- 172.21.15.100:0/3639696215 learned_addr learned my addr 172.21.15.100:0/3639696215 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:41:56.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.530+0000 7f0f87fff700 1 --2- 172.21.15.100:0/3639696215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba3e58e1b0 0x55ba3e5919f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:56.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.530+0000 7f0f998ceb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55ba3e595210 con 0x55ba3e58e1b0 2024-05-10T11:41:56.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.530+0000 7f0f8d5d8700 1 --2- 172.21.15.100:0/3639696215 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ba3e58ead0 0x55ba3e591f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:56.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.530+0000 7f0f998ceb80 1 -- 172.21.15.100:0/3639696215 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ba3e493ed0 con 0x55ba3e588d50 2024-05-10T11:41:56.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.530+0000 7f0f998ceb80 1 -- 172.21.15.100:0/3639696215 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55ba3e595390 con 0x55ba3e58ead0 2024-05-10T11:41:56.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.530+0000 7f0f998ceb80 10 client.0 _ll_register_callbacks cb 0x55ba3e46e5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:41:56.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.530+0000 7f0f8cdd7700 1 -- 172.21.15.100:0/3639696215 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ba3e58ead0 msgr2=0x55ba3e591f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:56.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.530+0000 7f0f8cdd7700 1 --2- 172.21.15.100:0/3639696215 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ba3e58ead0 0x55ba3e591f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:56.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.530+0000 7f0f8cdd7700 1 -- 172.21.15.100:0/3639696215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba3e58e1b0 msgr2=0x55ba3e5919f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:56.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.530+0000 7f0f8cdd7700 1 --2- 172.21.15.100:0/3639696215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba3e58e1b0 0x55ba3e5919f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:56.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.530+0000 7f0f8cdd7700 1 -- 172.21.15.100:0/3639696215 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba3e491760 con 0x55ba3e588d50 2024-05-10T11:41:56.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.530+0000 7f0f998ceb80 1 -- 172.21.15.100:0/3639696215 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f7c0aa3d0 con 0x55ba3e588d50 2024-05-10T11:41:56.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.530+0000 7f0f8cdd7700 1 --2- 172.21.15.100:0/3639696215 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba3e588d50 0x55ba3e594650 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c0aa0d0 tx=0x7f0f7c0b2fc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c9551992bec8870 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:41:56.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.530+0000 7f0f67fff700 10 client.29186 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:41:56.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.530+0000 7f0f8d5d8700 1 --2- 172.21.15.100:0/3639696215 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ba3e58ead0 0x55ba3e591f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:41:56.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.531+0000 7f0f67fff700 1 -- 172.21.15.100:0/3639696215 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c0b31a0 con 0x55ba3e588d50 2024-05-10T11:41:56.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.531+0000 7f0f67fff700 1 -- 172.21.15.100:0/3639696215 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f0f7c0aa840 con 0x55ba3e588d50 2024-05-10T11:41:56.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.531+0000 7f0f67fff700 1 -- 172.21.15.100:0/3639696215 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f0f7c0aaad0 con 0x55ba3e588d50 2024-05-10T11:41:56.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.531+0000 7f0f998ceb80 10 client.29186 fetch_fsmap learned FSMap version 41 2024-05-10T11:41:56.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.531+0000 7f0f998ceb80 1 -- 172.21.15.100:0/3639696215 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({fsmap=41}) -- 0x55ba3e595210 con 0x55ba3e588d50 2024-05-10T11:41:56.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.531+0000 7f0f67fff700 1 -- 172.21.15.100:0/3639696215 <== mon.2 v2:172.21.15.178:3300/0 4 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f0f7c0e0ce0 con 0x55ba3e588d50 2024-05-10T11:41:56.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.531+0000 7f0f998ceb80 10 client.29186 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:41:56.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.531+0000 7f0f998ceb80 10 client.29186 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-10T11:41:56.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.531+0000 7f0f998ceb80 20 client.29186 populate_metadata read hostname 'smithi100' 2024-05-10T11:41:56.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.531+0000 7f0f998ceb80 1 --2- 172.21.15.100:0/3639696215 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55ba3e5a1c10 0x55ba3e5a4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:56.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.531+0000 7f0f998ceb80 4 client.29186 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:41:56.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.531+0000 7f0f998ceb80 1 -- 172.21.15.100:0/3639696215 --> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba3e4e8320 con 0x55ba3e5a1c10 2024-05-10T11:41:56.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.532+0000 7f0f87fff700 1 --2- 172.21.15.100:0/3639696215 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55ba3e5a1c10 0x55ba3e5a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:56.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.532+0000 7f0f67fff700 10 client.29186 ms_handle_connect on v2:172.21.15.178:6838/3760985404 2024-05-10T11:41:56.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.532+0000 7f0f87fff700 1 --2- 172.21.15.100:0/3639696215 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55ba3e5a1c10 0x55ba3e5a4000 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c2e2c30a5f5e5aa server_cookie=8afd5477cf75eb27 in_seq=0 out_seq=0 2024-05-10T11:41:56.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.533+0000 7f0f67fff700 1 -- 172.21.15.100:0/3639696215 <== mds.1 v2:172.21.15.178:6838/3760985404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ba3e4e8320 con 0x55ba3e5a1c10 2024-05-10T11:41:56.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.533+0000 7f0f67fff700 10 client.29186 handle_command_reply: tid=0 2024-05-10T11:41:56.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.534+0000 7f0f998ceb80 1 client.29186 shutdown 2024-05-10T11:41:56.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.534+0000 7f0f998ceb80 1 -- 172.21.15.100:0/3639696215 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba3e588d50 msgr2=0x55ba3e594650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:56.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.534+0000 7f0f998ceb80 1 --2- 172.21.15.100:0/3639696215 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba3e588d50 0x55ba3e594650 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c0aa0d0 tx=0x7f0f7c0b2fc0 comp rx=0 tx=0).stop 2024-05-10T11:41:56.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.534+0000 7f0f998ceb80 1 -- 172.21.15.100:0/3639696215 shutdown_connections 2024-05-10T11:41:56.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.534+0000 7f0f998ceb80 1 --2- 172.21.15.100:0/3639696215 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55ba3e5a1c10 0x55ba3e5a4000 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:56.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.534+0000 7f0f998ceb80 1 --2- 172.21.15.100:0/3639696215 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ba3e58ead0 0x55ba3e591f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:56.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.534+0000 7f0f67fff700 0 client.29186 ms_handle_reset on v2:172.21.15.178:6838/3760985404 2024-05-10T11:41:56.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.534+0000 7f0f998ceb80 1 --2- 172.21.15.100:0/3639696215 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba3e588d50 0x55ba3e594650 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:56.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.534+0000 7f0f998ceb80 1 --2- 172.21.15.100:0/3639696215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba3e58e1b0 0x55ba3e5919f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:56.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.534+0000 7f0f998ceb80 1 -- 172.21.15.100:0/3639696215 >> 172.21.15.100:0/3639696215 conn(0x55ba3e4b4b70 msgr2=0x55ba3e57db30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:41:56.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.534+0000 7f0f998ceb80 1 -- 172.21.15.100:0/3639696215 shutdown_connections 2024-05-10T11:41:56.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.535+0000 7f0f998ceb80 1 -- 172.21.15.100:0/3639696215 wait complete. 2024-05-10T11:41:56.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.535+0000 7f0f998ceb80 20 client.29186 trim_cache size 0 max 16384 2024-05-10T11:41:56.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.554+0000 7f0f998ceb80 1 Processor -- start 2024-05-10T11:41:56.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.554+0000 7f0f998ceb80 1 -- start start 2024-05-10T11:41:56.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.554+0000 7f0f998ceb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ba3e588d50 0x55ba3e59e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:56.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.554+0000 7f0f998ceb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba3e58e1b0 0x55ba3e57dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:56.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.554+0000 7f0f998ceb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba3e58ead0 0x55ba3e5909f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:56.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.554+0000 7f0f998ceb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0f881af700 con 0x55ba3e58ead0 2024-05-10T11:41:56.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.554+0000 7f0f998ceb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0f881acf90 con 0x55ba3e58e1b0 2024-05-10T11:41:56.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.554+0000 7f0f998ceb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0f881aade0 con 0x55ba3e588d50 2024-05-10T11:41:56.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.555+0000 7f0f8cdd7700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ba3e588d50 0x55ba3e59e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:56.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.555+0000 7f0f87fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba3e58e1b0 0x55ba3e57dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:56.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.555+0000 7f0f87fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba3e58e1b0 0x55ba3e57dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:59446/0 (socket says 172.21.15.100:59446) 2024-05-10T11:41:56.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.555+0000 7f0f87fff700 1 -- 172.21.15.100:0/3684882840 learned_addr learned my addr 172.21.15.100:0/3684882840 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:41:56.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.555+0000 7f0f8d5d8700 1 --2- 172.21.15.100:0/3684882840 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba3e58ead0 0x55ba3e5909f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:56.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.555+0000 7f0f87fff700 1 -- 172.21.15.100:0/3684882840 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ba3e588d50 msgr2=0x55ba3e59e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:56.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.555+0000 7f0f87fff700 1 --2- 172.21.15.100:0/3684882840 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ba3e588d50 0x55ba3e59e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:56.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.555+0000 7f0f87fff700 1 -- 172.21.15.100:0/3684882840 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba3e58ead0 msgr2=0x55ba3e5909f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:56.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.555+0000 7f0f87fff700 1 --2- 172.21.15.100:0/3684882840 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba3e58ead0 0x55ba3e5909f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:56.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.555+0000 7f0f87fff700 1 -- 172.21.15.100:0/3684882840 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba3e584a90 con 0x55ba3e58e1b0 2024-05-10T11:41:56.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.555+0000 7f0f8d5d8700 1 --2- 172.21.15.100:0/3684882840 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba3e58ead0 0x55ba3e5909f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:41:56.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.555+0000 7f0f87fff700 1 --2- 172.21.15.100:0/3684882840 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba3e58e1b0 0x55ba3e57dc40 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f0f74009680 tx=0x7f0f7401b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd8500e6d7ec4df5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:41:56.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.556+0000 7f0f727fc700 1 -- 172.21.15.100:0/3684882840 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f74036070 con 0x55ba3e58e1b0 2024-05-10T11:41:56.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.556+0000 7f0f727fc700 1 -- 172.21.15.100:0/3684882840 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0f740044b0 con 0x55ba3e58e1b0 2024-05-10T11:41:56.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.556+0000 7f0f727fc700 1 -- 172.21.15.100:0/3684882840 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f74030030 con 0x55ba3e58e1b0 2024-05-10T11:41:56.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.557+0000 7f0f998ceb80 1 -- 172.21.15.100:0/3684882840 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba3e58e1b0 msgr2=0x55ba3e57dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:56.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.557+0000 7f0f998ceb80 1 --2- 172.21.15.100:0/3684882840 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba3e58e1b0 0x55ba3e57dc40 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f0f74009680 tx=0x7f0f7401b000 comp rx=0 tx=0).stop 2024-05-10T11:41:56.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.557+0000 7f0f998ceb80 1 -- 172.21.15.100:0/3684882840 shutdown_connections 2024-05-10T11:41:56.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.557+0000 7f0f998ceb80 1 --2- 172.21.15.100:0/3684882840 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba3e58ead0 0x55ba3e5909f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:56.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.557+0000 7f0f998ceb80 1 --2- 172.21.15.100:0/3684882840 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ba3e588d50 0x55ba3e59e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:56.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.557+0000 7f0f998ceb80 1 --2- 172.21.15.100:0/3684882840 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba3e58e1b0 0x55ba3e57dc40 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:56.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.557+0000 7f0f998ceb80 1 -- 172.21.15.100:0/3684882840 >> 172.21.15.100:0/3684882840 conn(0x55ba3e4b4b70 msgr2=0x55ba3e5902e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:41:56.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.557+0000 7f0f998ceb80 1 -- 172.21.15.100:0/3684882840 shutdown_connections 2024-05-10T11:41:56.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.557+0000 7f0f998ceb80 1 -- 172.21.15.100:0/3684882840 wait complete. 2024-05-10T11:41:56.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.557+0000 7f0f998ceb80 1 -- start start 2024-05-10T11:41:56.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.557+0000 7f0f998ceb80 1 Processor -- start 2024-05-10T11:41:56.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.558+0000 7f0f998ceb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ba3e588d50 0x55ba3e59a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:56.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.558+0000 7f0f998ceb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba3e58e1b0 0x55ba3e59abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:56.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.558+0000 7f0f998ceb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba3e58ead0 0x55ba3e590700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:56.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.558+0000 7f0f87fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba3e58e1b0 0x55ba3e59abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:56.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.558+0000 7f0f998ceb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0f881ab450 con 0x55ba3e58e1b0 2024-05-10T11:41:56.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.558+0000 7f0f998ceb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0f881ac8c0 con 0x55ba3e58ead0 2024-05-10T11:41:56.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.558+0000 7f0f998ceb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0f881abb20 con 0x55ba3e588d50 2024-05-10T11:41:56.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.558+0000 7f0f87fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba3e58e1b0 0x55ba3e59abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57354/0 (socket says 172.21.15.100:57354) 2024-05-10T11:41:56.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.558+0000 7f0f8cdd7700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ba3e588d50 0x55ba3e59a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:56.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.558+0000 7f0f87fff700 1 -- 172.21.15.100:0/2748589704 learned_addr learned my addr 172.21.15.100:0/2748589704 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:41:56.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.558+0000 7f0f998ceb80 10 client.0 _ll_register_callbacks cb 0x55ba3e46e5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:41:56.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.558+0000 7f0f8d5d8700 1 --2- 172.21.15.100:0/2748589704 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba3e58ead0 0x55ba3e590700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:56.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.559+0000 7f0f87fff700 1 -- 172.21.15.100:0/2748589704 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ba3e588d50 msgr2=0x55ba3e59a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:56.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.559+0000 7f0f87fff700 1 --2- 172.21.15.100:0/2748589704 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ba3e588d50 0x55ba3e59a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:56.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.559+0000 7f0f87fff700 1 -- 172.21.15.100:0/2748589704 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba3e58ead0 msgr2=0x55ba3e590700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:56.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.559+0000 7f0f87fff700 1 --2- 172.21.15.100:0/2748589704 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba3e58ead0 0x55ba3e590700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:56.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.559+0000 7f0f87fff700 1 -- 172.21.15.100:0/2748589704 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba3e590df0 con 0x55ba3e58e1b0 2024-05-10T11:41:56.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.559+0000 7f0f8cdd7700 1 --2- 172.21.15.100:0/2748589704 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ba3e588d50 0x55ba3e59a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:41:56.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.559+0000 7f0f87fff700 1 --2- 172.21.15.100:0/2748589704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba3e58e1b0 0x55ba3e59abc0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f0f74009b80 tx=0x7f0f74005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29a8bb83e2bb6e30 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:41:56.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.559+0000 7f0f67fff700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:41:56.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.559+0000 7f0f67fff700 1 -- 172.21.15.100:0/2748589704 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7403f060 con 0x55ba3e58e1b0 2024-05-10T11:41:56.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.559+0000 7f0f67fff700 1 -- 172.21.15.100:0/2748589704 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f0f7401fde0 con 0x55ba3e58e1b0 2024-05-10T11:41:56.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.559+0000 7f0f998ceb80 1 -- 172.21.15.100:0/2748589704 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f740044b0 con 0x55ba3e58e1b0 2024-05-10T11:41:56.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.559+0000 7f0f67fff700 1 -- 172.21.15.100:0/2748589704 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f0f74046050 con 0x55ba3e58e1b0 2024-05-10T11:41:56.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.559+0000 7f0f998ceb80 10 client.19941 fetch_fsmap learned FSMap version 41 2024-05-10T11:41:56.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.559+0000 7f0f998ceb80 1 -- 172.21.15.100:0/2748589704 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55ba3e60ba70 con 0x55ba3e58e1b0 2024-05-10T11:41:56.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.560+0000 7f0f67fff700 1 -- 172.21.15.100:0/2748589704 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0f740044b0 con 0x55ba3e58e1b0 2024-05-10T11:41:56.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.560+0000 7f0f67fff700 1 -- 172.21.15.100:0/2748589704 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f0f7401d520 con 0x55ba3e58e1b0 2024-05-10T11:41:56.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.560+0000 7f0f998ceb80 10 client.19941 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:41:56.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.560+0000 7f0f998ceb80 10 client.19941 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-10T11:41:56.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.560+0000 7f0f998ceb80 20 client.19941 populate_metadata read hostname 'smithi100' 2024-05-10T11:41:56.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.560+0000 7f0f998ceb80 1 --2- 172.21.15.100:0/2748589704 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55ba3e60c020 0x55ba3e60e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:56.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.560+0000 7f0f998ceb80 4 client.19941 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:41:56.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.560+0000 7f0f998ceb80 1 -- 172.21.15.100:0/2748589704 --> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ba3e4e23c0 con 0x55ba3e60c020 2024-05-10T11:41:56.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.560+0000 7f0f8cdd7700 1 --2- 172.21.15.100:0/2748589704 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55ba3e60c020 0x55ba3e60e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:56.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.561+0000 7f0f67fff700 10 client.19941 ms_handle_connect on v2:172.21.15.178:6838/3760985404 2024-05-10T11:41:56.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.561+0000 7f0f8cdd7700 1 --2- 172.21.15.100:0/2748589704 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55ba3e60c020 0x55ba3e60e410 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d7f8ecf54f398d7d server_cookie=b271b17942b13dda in_seq=0 out_seq=0 2024-05-10T11:41:56.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.561+0000 7f0f67fff700 1 -- 172.21.15.100:0/2748589704 <== mds.1 v2:172.21.15.178:6838/3760985404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ba3e4e23c0 con 0x55ba3e60c020 2024-05-10T11:41:56.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.561+0000 7f0f67fff700 10 client.19941 handle_command_reply: tid=0 2024-05-10T11:41:56.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.561+0000 7f0f998ceb80 1 client.19941 shutdown 2024-05-10T11:41:56.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.561+0000 7f0f998ceb80 1 -- 172.21.15.100:0/2748589704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba3e58e1b0 msgr2=0x55ba3e59abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:56.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.561+0000 7f0f998ceb80 1 --2- 172.21.15.100:0/2748589704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba3e58e1b0 0x55ba3e59abc0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f0f74009b80 tx=0x7f0f74005760 comp rx=0 tx=0).stop 2024-05-10T11:41:56.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.561+0000 7f0f998ceb80 1 -- 172.21.15.100:0/2748589704 shutdown_connections 2024-05-10T11:41:56.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.561+0000 7f0f998ceb80 1 --2- 172.21.15.100:0/2748589704 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x55ba3e60c020 0x55ba3e60e410 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:56.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.561+0000 7f0f998ceb80 1 --2- 172.21.15.100:0/2748589704 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba3e58ead0 0x55ba3e590700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:56.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.561+0000 7f0f998ceb80 1 --2- 172.21.15.100:0/2748589704 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ba3e588d50 0x55ba3e59a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:56.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.561+0000 7f0f998ceb80 1 --2- 172.21.15.100:0/2748589704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba3e58e1b0 0x55ba3e59abc0 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:56.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.561+0000 7f0f67fff700 0 client.19941 ms_handle_reset on v2:172.21.15.178:6838/3760985404 2024-05-10T11:41:56.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.561+0000 7f0f998ceb80 1 -- 172.21.15.100:0/2748589704 >> 172.21.15.100:0/2748589704 conn(0x55ba3e4b4b70 msgr2=0x55ba3e588090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:41:56.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.561+0000 7f0f998ceb80 1 -- 172.21.15.100:0/2748589704 shutdown_connections 2024-05-10T11:41:56.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.561+0000 7f0f998ceb80 1 -- 172.21.15.100:0/2748589704 wait complete. 2024-05-10T11:41:56.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.562+0000 7f0f998ceb80 20 client.19941 trim_cache size 0 max 16384 2024-05-10T11:41:56.566 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:41:56.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.562+0000 7f0f8f03b700 1 -- 172.21.15.100:0/435073492 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0f780b2880 msgr2=0x7f0f780b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:56.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.562+0000 7f0f8f03b700 1 --2- 172.21.15.100:0/435073492 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0f780b2880 0x7f0f780b4d40 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f0f88079840 tx=0x7f0f74009260 comp rx=0 tx=0).stop 2024-05-10T11:41:56.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.562+0000 7f0f8f03b700 1 -- 172.21.15.100:0/435073492 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f88102390 msgr2=0x7f0f8819f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:56.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.562+0000 7f0f8f03b700 1 --2- 172.21.15.100:0/435073492 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f88102390 0x7f0f8819f720 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c00b650 tx=0x7f0f7c0047b0 comp rx=0 tx=0).stop 2024-05-10T11:41:56.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.562+0000 7f0f8f03b700 1 -- 172.21.15.100:0/435073492 shutdown_connections 2024-05-10T11:41:56.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.563+0000 7f0f8f03b700 1 --2- 172.21.15.100:0/435073492 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0f780b2880 0x7f0f780b4d40 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:56.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.563+0000 7f0f8f03b700 1 --2- 172.21.15.100:0/435073492 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f88108400 0x7f0f88199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:56.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.563+0000 7f0f8f03b700 1 --2- 172.21.15.100:0/435073492 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f88102390 0x7f0f8819f720 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:56.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.563+0000 7f0f8f03b700 1 --2- 172.21.15.100:0/435073492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f88102d50 0x7f0f8819fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:56.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.563+0000 7f0f8f03b700 1 -- 172.21.15.100:0/435073492 >> 172.21.15.100:0/435073492 conn(0x7f0f88079310 msgr2=0x7f0f88100340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:41:56.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.563+0000 7f0f8f03b700 1 -- 172.21.15.100:0/435073492 shutdown_connections 2024-05-10T11:41:56.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:56.563+0000 7f0f8f03b700 1 -- 172.21.15.100:0/435073492 wait complete. 2024-05-10T11:41:56.957 INFO:tasks.workunit.client.0.smithi100.stdout: 10 98018 8.91 MB/sec execute 470 sec latency 100.730 ms 2024-05-10T11:41:57.406 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-10T11:41:57.788 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:41:57.957 INFO:tasks.workunit.client.0.smithi100.stdout: 10 98157 8.91 MB/sec execute 471 sec latency 85.438 ms 2024-05-10T11:41:58.165 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:41:57 smithi100 ceph-mon[30296]: pgmap v1128: 129 pgs: 129 active+clean; 566 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 4.2 MiB/s wr, 286 op/s 2024-05-10T11:41:58.165 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:41:57 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:41:58.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:41:57 smithi178 ceph-mon[33359]: pgmap v1128: 129 pgs: 129 active+clean; 566 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 4.2 MiB/s wr, 286 op/s 2024-05-10T11:41:58.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:41:57 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:41:58.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:41:57 smithi188 ceph-mon[34075]: pgmap v1128: 129 pgs: 129 active+clean; 566 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 4.2 MiB/s wr, 286 op/s 2024-05-10T11:41:58.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:41:57 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:41:58.957 INFO:tasks.workunit.client.0.smithi100.stdout: 10 98363 8.93 MB/sec execute 472 sec latency 71.397 ms 2024-05-10T11:41:59.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.918+0000 7f743bac2700 1 -- 172.21.15.100:0/588140314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7434103ee0 msgr2=0x7f7434110370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:59.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.918+0000 7f743bac2700 1 --2- 172.21.15.100:0/588140314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7434103ee0 0x7f7434110370 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f742c00b0a0 tx=0x7f742c00b3b0 comp rx=0 tx=0).stop 2024-05-10T11:41:59.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.918+0000 7f743bac2700 1 -- 172.21.15.100:0/588140314 shutdown_connections 2024-05-10T11:41:59.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.918+0000 7f743bac2700 1 --2- 172.21.15.100:0/588140314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7434103ee0 0x7f7434110370 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:59.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.918+0000 7f743bac2700 1 --2- 172.21.15.100:0/588140314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7434109530 0x7f7434109910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:59.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.918+0000 7f743bac2700 1 --2- 172.21.15.100:0/588140314 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7434103520 0x7f74341039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:59.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.918+0000 7f743bac2700 1 -- 172.21.15.100:0/588140314 >> 172.21.15.100:0/588140314 conn(0x7f743407a570 msgr2=0x7f743407a980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:41:59.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.918+0000 7f743bac2700 1 -- 172.21.15.100:0/588140314 shutdown_connections 2024-05-10T11:41:59.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.918+0000 7f743bac2700 1 -- 172.21.15.100:0/588140314 wait complete. 2024-05-10T11:41:59.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.919+0000 7f743bac2700 1 Processor -- start 2024-05-10T11:41:59.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.919+0000 7f743bac2700 1 -- start start 2024-05-10T11:41:59.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.919+0000 7f743bac2700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7434103520 0x7f74340757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:59.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.919+0000 7f743bac2700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7434103ee0 0x7f7434075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:59.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.919+0000 7f743bac2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7434109530 0x7f7434076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:59.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.919+0000 7f743bac2700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7434113530 con 0x7f7434109530 2024-05-10T11:41:59.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.919+0000 7f743bac2700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f74341133b0 con 0x7f7434103520 2024-05-10T11:41:59.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.920+0000 7f743bac2700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f74341136b0 con 0x7f7434103ee0 2024-05-10T11:41:59.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.920+0000 7f743905d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7434103ee0 0x7f7434075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:59.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.920+0000 7f743985e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7434103520 0x7f74340757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:59.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.920+0000 7f743905d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7434103ee0 0x7f7434075ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:47354/0 (socket says 172.21.15.100:47354) 2024-05-10T11:41:59.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.920+0000 7f743905d700 1 -- 172.21.15.100:0/1884166735 learned_addr learned my addr 172.21.15.100:0/1884166735 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:41:59.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.920+0000 7f743a05f700 1 --2- 172.21.15.100:0/1884166735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7434109530 0x7f7434076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:59.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.920+0000 7f743905d700 1 -- 172.21.15.100:0/1884166735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7434103520 msgr2=0x7f74340757a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:59.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.920+0000 7f743905d700 1 --2- 172.21.15.100:0/1884166735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7434103520 0x7f74340757a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:59.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.920+0000 7f743905d700 1 -- 172.21.15.100:0/1884166735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7434109530 msgr2=0x7f7434076220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:59.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.920+0000 7f743905d700 1 --2- 172.21.15.100:0/1884166735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7434109530 0x7f7434076220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:59.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.920+0000 7f743905d700 1 -- 172.21.15.100:0/1884166735 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74341a7f50 con 0x7f7434103ee0 2024-05-10T11:41:59.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.920+0000 7f743905d700 1 --2- 172.21.15.100:0/1884166735 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7434103ee0 0x7f7434075ce0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f742400d980 tx=0x7f742400dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:41:59.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.920+0000 7f742affd700 1 -- 172.21.15.100:0/1884166735 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7424016070 con 0x7f7434103ee0 2024-05-10T11:41:59.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.920+0000 7f743bac2700 1 -- 172.21.15.100:0/1884166735 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7434199760 con 0x7f7434103ee0 2024-05-10T11:41:59.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.920+0000 7f742affd700 1 -- 172.21.15.100:0/1884166735 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7424004830 con 0x7f7434103ee0 2024-05-10T11:41:59.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.920+0000 7f742affd700 1 -- 172.21.15.100:0/1884166735 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74240054c0 con 0x7f7434103ee0 2024-05-10T11:41:59.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.920+0000 7f743bac2700 1 -- 172.21.15.100:0/1884166735 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7434199cf0 con 0x7f7434103ee0 2024-05-10T11:41:59.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.924+0000 7f742affd700 1 -- 172.21.15.100:0/1884166735 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f7424004220 con 0x7f7434103ee0 2024-05-10T11:41:59.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.925+0000 7f742affd700 1 --2- 172.21.15.100:0/1884166735 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f74200b2880 0x7f74200b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:59.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.925+0000 7f743985e700 1 --2- 172.21.15.100:0/1884166735 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f74200b2880 0x7f74200b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:59.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.925+0000 7f7446355b80 1 Processor -- start 2024-05-10T11:41:59.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.925+0000 7f742affd700 1 -- 172.21.15.100:0/1884166735 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f74240e22b0 con 0x7f7434103ee0 2024-05-10T11:41:59.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.926+0000 7f7446355b80 1 -- start start 2024-05-10T11:41:59.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.926+0000 7f7446355b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5572896e01b0 0x5572896e0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:59.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.926+0000 7f7446355b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572896e0ad0 0x5572896da810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:59.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.926+0000 7f7446355b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5572896dad50 0x5572896db190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:59.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.926+0000 7f7446355b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55728962c940 con 0x5572896e0ad0 2024-05-10T11:41:59.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.926+0000 7f7446355b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x557289591da0 con 0x5572896dad50 2024-05-10T11:41:59.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.926+0000 7f7446355b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x557289628900 con 0x5572896e01b0 2024-05-10T11:41:59.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.926+0000 7f743a05f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5572896dad50 0x5572896db190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:59.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.926+0000 7f743a05f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5572896dad50 0x5572896db190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:59486/0 (socket says 172.21.15.100:59486) 2024-05-10T11:41:59.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.926+0000 7f743a05f700 1 -- 172.21.15.100:0/2342285591 learned_addr learned my addr 172.21.15.100:0/2342285591 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:41:59.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.926+0000 7f743985e700 1 --2- 172.21.15.100:0/1884166735 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f74200b2880 0x7f74200b4d40 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f7430004150 tx=0x7f7430018040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:41:59.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.926+0000 7f743985e700 1 --2- 172.21.15.100:0/2342285591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5572896e01b0 0x5572896e0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:59.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.926+0000 7f743905d700 1 --2- 172.21.15.100:0/2342285591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572896e0ad0 0x5572896da810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:59.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.926+0000 7f743a05f700 1 -- 172.21.15.100:0/2342285591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5572896e01b0 msgr2=0x5572896e0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:59.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.926+0000 7f743a05f700 1 --2- 172.21.15.100:0/2342285591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5572896e01b0 0x5572896e0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:59.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.926+0000 7f743a05f700 1 -- 172.21.15.100:0/2342285591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572896e0ad0 msgr2=0x5572896da810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:59.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.926+0000 7f743a05f700 1 --2- 172.21.15.100:0/2342285591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572896e0ad0 0x5572896da810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:59.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.926+0000 7f743a05f700 1 -- 172.21.15.100:0/2342285591 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572895798d0 con 0x5572896dad50 2024-05-10T11:41:59.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.926+0000 7f743985e700 1 --2- 172.21.15.100:0/2342285591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5572896e01b0 0x5572896e0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:41:59.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.927+0000 7f743a05f700 1 --2- 172.21.15.100:0/2342285591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5572896dad50 0x5572896db190 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f742c009390 tx=0x7f742c009e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66d88f538bf2da60 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:41:59.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.927+0000 7f741ffff700 1 -- 172.21.15.100:0/2342285591 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f742c020070 con 0x5572896dad50 2024-05-10T11:41:59.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.927+0000 7f741ffff700 1 -- 172.21.15.100:0/2342285591 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f742c00bcb0 con 0x5572896dad50 2024-05-10T11:41:59.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.927+0000 7f741ffff700 1 -- 172.21.15.100:0/2342285591 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f742c007cf0 con 0x5572896dad50 2024-05-10T11:41:59.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.928+0000 7f7446355b80 1 -- 172.21.15.100:0/2342285591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5572896dad50 msgr2=0x5572896db190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:59.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.928+0000 7f7446355b80 1 --2- 172.21.15.100:0/2342285591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5572896dad50 0x5572896db190 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f742c009390 tx=0x7f742c009e20 comp rx=0 tx=0).stop 2024-05-10T11:41:59.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.928+0000 7f7446355b80 1 -- 172.21.15.100:0/2342285591 shutdown_connections 2024-05-10T11:41:59.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.928+0000 7f7446355b80 1 --2- 172.21.15.100:0/2342285591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5572896dad50 0x5572896db190 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:59.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.928+0000 7f7446355b80 1 --2- 172.21.15.100:0/2342285591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5572896e01b0 0x5572896e0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:59.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.928+0000 7f7446355b80 1 --2- 172.21.15.100:0/2342285591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572896e0ad0 0x5572896da810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:59.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.928+0000 7f7446355b80 1 -- 172.21.15.100:0/2342285591 >> 172.21.15.100:0/2342285591 conn(0x557289606b70 msgr2=0x5572896d52a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:41:59.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.928+0000 7f7446355b80 1 -- 172.21.15.100:0/2342285591 shutdown_connections 2024-05-10T11:41:59.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.929+0000 7f7446355b80 1 -- 172.21.15.100:0/2342285591 wait complete. 2024-05-10T11:41:59.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.929+0000 7f7446355b80 1 -- start start 2024-05-10T11:41:59.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.929+0000 7f7446355b80 1 Processor -- start 2024-05-10T11:41:59.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.930+0000 7f7446355b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572896dad50 0x5572896e6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:59.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.930+0000 7f7446355b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5572896e01b0 0x5572896e3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:59.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.930+0000 7f7446355b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5572896e0ad0 0x5572896e4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:59.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.930+0000 7f7446355b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5572896e73f0 con 0x5572896dad50 2024-05-10T11:41:59.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.930+0000 7f7446355b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5572895e5ed0 con 0x5572896e01b0 2024-05-10T11:41:59.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.930+0000 7f7446355b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5572896e7570 con 0x5572896e0ad0 2024-05-10T11:41:59.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.930+0000 7f743985e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572896dad50 0x5572896e6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:59.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.930+0000 7f7446355b80 10 client.0 _ll_register_callbacks cb 0x5572895c05b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:41:59.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.930+0000 7f743a05f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5572896e0ad0 0x5572896e4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:59.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.930+0000 7f743985e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572896dad50 0x5572896e6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57388/0 (socket says 172.21.15.100:57388) 2024-05-10T11:41:59.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.930+0000 7f743985e700 1 -- 172.21.15.100:0/3469251777 learned_addr learned my addr 172.21.15.100:0/3469251777 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:41:59.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.930+0000 7f743905d700 1 --2- 172.21.15.100:0/3469251777 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5572896e01b0 0x5572896e3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:59.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.930+0000 7f743a05f700 1 -- 172.21.15.100:0/3469251777 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5572896e01b0 msgr2=0x5572896e3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:59.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.930+0000 7f743a05f700 1 --2- 172.21.15.100:0/3469251777 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5572896e01b0 0x5572896e3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:59.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.930+0000 7f743a05f700 1 -- 172.21.15.100:0/3469251777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572896dad50 msgr2=0x5572896e6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:59.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.930+0000 7f743a05f700 1 --2- 172.21.15.100:0/3469251777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572896dad50 0x5572896e6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:59.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.930+0000 7f743a05f700 1 -- 172.21.15.100:0/3469251777 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572895e3760 con 0x5572896e0ad0 2024-05-10T11:41:59.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.930+0000 7f743905d700 1 --2- 172.21.15.100:0/3469251777 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5572896e01b0 0x5572896e3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:41:59.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.930+0000 7f743985e700 1 --2- 172.21.15.100:0/3469251777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572896dad50 0x5572896e6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:41:59.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.930+0000 7f743a05f700 1 --2- 172.21.15.100:0/3469251777 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5572896e0ad0 0x5572896e4040 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f742c009390 tx=0x7f742c00afe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1dc8187a805e231c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:41:59.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.930+0000 7f741d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:41:59.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.931+0000 7f741d7fa700 1 -- 172.21.15.100:0/3469251777 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f742c020040 con 0x5572896e0ad0 2024-05-10T11:41:59.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.931+0000 7f7446355b80 1 -- 172.21.15.100:0/3469251777 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f742c00bcb0 con 0x5572896e0ad0 2024-05-10T11:41:59.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.931+0000 7f741d7fa700 1 -- 172.21.15.100:0/3469251777 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f742c007b00 con 0x5572896e0ad0 2024-05-10T11:41:59.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.931+0000 7f741d7fa700 1 -- 172.21.15.100:0/3469251777 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f742c00be60 con 0x5572896e0ad0 2024-05-10T11:41:59.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.931+0000 7f7446355b80 10 client.29320 fetch_fsmap learned FSMap version 41 2024-05-10T11:41:59.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.931+0000 7f7446355b80 1 -- 172.21.15.100:0/3469251777 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5572896e73f0 con 0x5572896e0ad0 2024-05-10T11:41:59.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.931+0000 7f741d7fa700 1 -- 172.21.15.100:0/3469251777 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f742c00bcb0 con 0x5572896e0ad0 2024-05-10T11:41:59.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.931+0000 7f741d7fa700 1 -- 172.21.15.100:0/3469251777 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f742c011070 con 0x5572896e0ad0 2024-05-10T11:41:59.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.931+0000 7f7446355b80 10 client.29320 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:41:59.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.931+0000 7f7446355b80 10 client.29320 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:41:59.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.931+0000 7f7446355b80 20 client.29320 populate_metadata read hostname 'smithi100' 2024-05-10T11:41:59.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.931+0000 7f7446355b80 1 --2- 172.21.15.100:0/3469251777 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5572896f3db0 0x5572896f61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:59.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.932+0000 7f7446355b80 4 client.29320 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:41:59.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.932+0000 7f7446355b80 1 -- 172.21.15.100:0/3469251777 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55728963a320 con 0x5572896f3db0 2024-05-10T11:41:59.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.932+0000 7f743985e700 1 --2- 172.21.15.100:0/3469251777 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5572896f3db0 0x5572896f61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:59.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.932+0000 7f743985e700 1 --2- 172.21.15.100:0/3469251777 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5572896f3db0 0x5572896f61a0 crc :-1 s=READY pgs=439 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a438b4aafcda7012 server_cookie=5c0c2420520077e9 in_seq=0 out_seq=0 2024-05-10T11:41:59.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.932+0000 7f741d7fa700 10 client.29320 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:41:59.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.933+0000 7f741d7fa700 1 -- 172.21.15.100:0/3469251777 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55728963a320 con 0x5572896f3db0 2024-05-10T11:41:59.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.933+0000 7f741d7fa700 10 client.29320 handle_command_reply: tid=0 2024-05-10T11:41:59.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.933+0000 7f7446355b80 1 client.29320 shutdown 2024-05-10T11:41:59.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.933+0000 7f7446355b80 1 -- 172.21.15.100:0/3469251777 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5572896e0ad0 msgr2=0x5572896e4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:59.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.933+0000 7f7446355b80 1 --2- 172.21.15.100:0/3469251777 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5572896e0ad0 0x5572896e4040 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f742c009390 tx=0x7f742c00afe0 comp rx=0 tx=0).stop 2024-05-10T11:41:59.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.933+0000 7f7446355b80 1 -- 172.21.15.100:0/3469251777 shutdown_connections 2024-05-10T11:41:59.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.933+0000 7f7446355b80 1 --2- 172.21.15.100:0/3469251777 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5572896f3db0 0x5572896f61a0 crc :-1 s=READY pgs=439 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:59.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.933+0000 7f7446355b80 1 --2- 172.21.15.100:0/3469251777 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5572896e0ad0 0x5572896e4040 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:59.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.933+0000 7f7446355b80 1 --2- 172.21.15.100:0/3469251777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572896dad50 0x5572896e6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:59.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.933+0000 7f7446355b80 1 --2- 172.21.15.100:0/3469251777 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5572896e01b0 0x5572896e3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:59.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.933+0000 7f741d7fa700 0 client.29320 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:41:59.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.933+0000 7f7446355b80 1 -- 172.21.15.100:0/3469251777 >> 172.21.15.100:0/3469251777 conn(0x557289606b70 msgr2=0x5572896cfb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:41:59.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.934+0000 7f7446355b80 1 -- 172.21.15.100:0/3469251777 shutdown_connections 2024-05-10T11:41:59.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.934+0000 7f7446355b80 1 -- 172.21.15.100:0/3469251777 wait complete. 2024-05-10T11:41:59.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.934+0000 7f7446355b80 20 client.29320 trim_cache size 0 max 16384 2024-05-10T11:41:59.958 INFO:tasks.workunit.client.0.smithi100.stdout: 10 98535 8.93 MB/sec execute 473 sec latency 121.921 ms 2024-05-10T11:41:59.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.963+0000 7f7446355b80 1 Processor -- start 2024-05-10T11:41:59.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.963+0000 7f7446355b80 1 -- start start 2024-05-10T11:41:59.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.963+0000 7f7446355b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5572896dad50 0x5572896da810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:59.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.963+0000 7f7446355b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572896e01b0 0x5572896f0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:59.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.963+0000 7f7446355b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5572896e0ad0 0x5572896e15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:59.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.964+0000 7f7446355b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f74341af5f0 con 0x5572896e01b0 2024-05-10T11:41:59.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.964+0000 7f7446355b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f74341ace80 con 0x5572896dad50 2024-05-10T11:41:59.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.964+0000 7f743905d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572896e01b0 0x5572896f0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:59.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.964+0000 7f7446355b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f74341aacd0 con 0x5572896e0ad0 2024-05-10T11:41:59.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.964+0000 7f743985e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5572896dad50 0x5572896da810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:59.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.964+0000 7f743985e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5572896dad50 0x5572896da810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:59498/0 (socket says 172.21.15.100:59498) 2024-05-10T11:41:59.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.964+0000 7f743905d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572896e01b0 0x5572896f0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57392/0 (socket says 172.21.15.100:57392) 2024-05-10T11:41:59.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.964+0000 7f743985e700 1 -- 172.21.15.100:0/3010120811 learned_addr learned my addr 172.21.15.100:0/3010120811 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:41:59.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.964+0000 7f743a05f700 1 --2- 172.21.15.100:0/3010120811 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5572896e0ad0 0x5572896e15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:59.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.964+0000 7f743985e700 1 -- 172.21.15.100:0/3010120811 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5572896e0ad0 msgr2=0x5572896e15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:59.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.964+0000 7f743985e700 1 --2- 172.21.15.100:0/3010120811 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5572896e0ad0 0x5572896e15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:59.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.964+0000 7f743985e700 1 -- 172.21.15.100:0/3010120811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572896e01b0 msgr2=0x5572896f0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:59.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.964+0000 7f743985e700 1 --2- 172.21.15.100:0/3010120811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572896e01b0 0x5572896f0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:59.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.964+0000 7f743985e700 1 -- 172.21.15.100:0/3010120811 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572896e38d0 con 0x5572896dad50 2024-05-10T11:41:59.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.964+0000 7f743905d700 1 --2- 172.21.15.100:0/3010120811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572896e01b0 0x5572896f0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:41:59.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.964+0000 7f743985e700 1 --2- 172.21.15.100:0/3010120811 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5572896dad50 0x5572896da810 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f7430009ad0 tx=0x7f743001c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a396195718718dc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:41:59.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.964+0000 7f741f7fe700 1 -- 172.21.15.100:0/3010120811 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7430003d70 con 0x5572896dad50 2024-05-10T11:41:59.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.964+0000 7f741f7fe700 1 -- 172.21.15.100:0/3010120811 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7430004cc0 con 0x5572896dad50 2024-05-10T11:41:59.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.965+0000 7f741f7fe700 1 -- 172.21.15.100:0/3010120811 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74300056a0 con 0x5572896dad50 2024-05-10T11:41:59.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.965+0000 7f7446355b80 1 -- 172.21.15.100:0/3010120811 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5572896dad50 msgr2=0x5572896da810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:59.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.965+0000 7f7446355b80 1 --2- 172.21.15.100:0/3010120811 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5572896dad50 0x5572896da810 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f7430009ad0 tx=0x7f743001c000 comp rx=0 tx=0).stop 2024-05-10T11:41:59.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.965+0000 7f7446355b80 1 -- 172.21.15.100:0/3010120811 shutdown_connections 2024-05-10T11:41:59.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.965+0000 7f7446355b80 1 --2- 172.21.15.100:0/3010120811 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5572896e0ad0 0x5572896e15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:59.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.965+0000 7f7446355b80 1 --2- 172.21.15.100:0/3010120811 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5572896dad50 0x5572896da810 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:59.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.965+0000 7f7446355b80 1 --2- 172.21.15.100:0/3010120811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572896e01b0 0x5572896f0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:59.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.965+0000 7f7446355b80 1 -- 172.21.15.100:0/3010120811 >> 172.21.15.100:0/3010120811 conn(0x557289606b70 msgr2=0x5572896e0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:41:59.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.966+0000 7f7446355b80 1 -- 172.21.15.100:0/3010120811 shutdown_connections 2024-05-10T11:41:59.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.966+0000 7f7446355b80 1 -- 172.21.15.100:0/3010120811 wait complete. 2024-05-10T11:41:59.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.966+0000 7f7446355b80 1 -- start start 2024-05-10T11:41:59.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.966+0000 7f7446355b80 1 Processor -- start 2024-05-10T11:41:59.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.967+0000 7f7446355b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5572896dad50 0x5572896ec800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:59.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.967+0000 7f7446355b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572896e01b0 0x5572896ecd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:59.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.967+0000 7f7446355b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5572896e0ad0 0x5572896fa7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:59.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.967+0000 7f7446355b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f74341ab340 con 0x5572896e01b0 2024-05-10T11:41:59.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.967+0000 7f743905d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572896e01b0 0x5572896ecd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:59.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.967+0000 7f743905d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572896e01b0 0x5572896ecd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57398/0 (socket says 172.21.15.100:57398) 2024-05-10T11:41:59.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.967+0000 7f743905d700 1 -- 172.21.15.100:0/2093396600 learned_addr learned my addr 172.21.15.100:0/2093396600 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:41:59.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.967+0000 7f743985e700 1 --2- 172.21.15.100:0/2093396600 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5572896dad50 0x5572896ec800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:59.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.967+0000 7f743a05f700 1 --2- 172.21.15.100:0/2093396600 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5572896e0ad0 0x5572896fa7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:59.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.967+0000 7f7446355b80 1 -- 172.21.15.100:0/2093396600 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f74341ac7b0 con 0x5572896e0ad0 2024-05-10T11:41:59.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.967+0000 7f7446355b80 1 -- 172.21.15.100:0/2093396600 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f74341aba10 con 0x5572896dad50 2024-05-10T11:41:59.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.967+0000 7f7446355b80 10 client.0 _ll_register_callbacks cb 0x5572895c05b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:41:59.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.968+0000 7f743905d700 1 -- 172.21.15.100:0/2093396600 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5572896dad50 msgr2=0x5572896ec800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:59.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.968+0000 7f743905d700 1 --2- 172.21.15.100:0/2093396600 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5572896dad50 0x5572896ec800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:59.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.968+0000 7f743905d700 1 -- 172.21.15.100:0/2093396600 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5572896e0ad0 msgr2=0x5572896fa7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:41:59.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.968+0000 7f743905d700 1 --2- 172.21.15.100:0/2093396600 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5572896e0ad0 0x5572896fa7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:59.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.968+0000 7f743905d700 1 -- 172.21.15.100:0/2093396600 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572896fae30 con 0x5572896e01b0 2024-05-10T11:41:59.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.968+0000 7f743985e700 1 --2- 172.21.15.100:0/2093396600 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5572896dad50 0x5572896ec800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:41:59.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.968+0000 7f743a05f700 1 --2- 172.21.15.100:0/2093396600 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5572896e0ad0 0x5572896fa7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:41:59.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.968+0000 7f743905d700 1 --2- 172.21.15.100:0/2093396600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572896e01b0 0x5572896ecd40 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f74240a55f0 tx=0x7f74240a9c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85d43d54cabe4cbe server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:41:59.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.968+0000 7f741d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:41:59.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.968+0000 7f741d7fa700 1 -- 172.21.15.100:0/2093396600 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74240c3060 con 0x5572896e01b0 2024-05-10T11:41:59.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.968+0000 7f741d7fa700 1 -- 172.21.15.100:0/2093396600 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f74240049e0 con 0x5572896e01b0 2024-05-10T11:41:59.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.968+0000 7f7446355b80 1 -- 172.21.15.100:0/2093396600 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7430004cc0 con 0x5572896e01b0 2024-05-10T11:41:59.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.969+0000 7f741d7fa700 1 -- 172.21.15.100:0/2093396600 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f74240c3e60 con 0x5572896e01b0 2024-05-10T11:41:59.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.969+0000 7f7446355b80 10 client.19959 fetch_fsmap learned FSMap version 41 2024-05-10T11:41:59.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.969+0000 7f7446355b80 1 -- 172.21.15.100:0/2093396600 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55728975dc50 con 0x5572896e01b0 2024-05-10T11:41:59.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.969+0000 7f741d7fa700 1 -- 172.21.15.100:0/2093396600 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7430004cc0 con 0x5572896e01b0 2024-05-10T11:41:59.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.969+0000 7f741d7fa700 1 -- 172.21.15.100:0/2093396600 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f74240b1f40 con 0x5572896e01b0 2024-05-10T11:41:59.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.969+0000 7f7446355b80 10 client.19959 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:41:59.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.969+0000 7f7446355b80 10 client.19959 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:41:59.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.969+0000 7f7446355b80 20 client.19959 populate_metadata read hostname 'smithi100' 2024-05-10T11:41:59.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.969+0000 7f7446355b80 1 --2- 172.21.15.100:0/2093396600 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55728975e360 0x557289760780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:41:59.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.969+0000 7f7446355b80 4 client.19959 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:41:59.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.969+0000 7f7446355b80 1 -- 172.21.15.100:0/2093396600 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5572896343c0 con 0x55728975e360 2024-05-10T11:41:59.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.970+0000 7f743985e700 1 --2- 172.21.15.100:0/2093396600 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55728975e360 0x557289760780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:41:59.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.970+0000 7f743985e700 1 --2- 172.21.15.100:0/2093396600 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55728975e360 0x557289760780 crc :-1 s=READY pgs=440 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c93da396368a6a45 server_cookie=78c87ffd83757bea in_seq=0 out_seq=0 2024-05-10T11:41:59.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.970+0000 7f741d7fa700 10 client.19959 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:41:59.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.970+0000 7f741d7fa700 1 -- 172.21.15.100:0/2093396600 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5572896343c0 con 0x55728975e360 2024-05-10T11:41:59.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.970+0000 7f741d7fa700 10 client.19959 handle_command_reply: tid=0 2024-05-10T11:41:59.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.970+0000 7f7446355b80 1 client.19959 shutdown 2024-05-10T11:41:59.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.970+0000 7f7446355b80 1 -- 172.21.15.100:0/2093396600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572896e01b0 msgr2=0x5572896ecd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:59.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.970+0000 7f7446355b80 1 --2- 172.21.15.100:0/2093396600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572896e01b0 0x5572896ecd40 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f74240a55f0 tx=0x7f74240a9c70 comp rx=0 tx=0).stop 2024-05-10T11:41:59.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.971+0000 7f7446355b80 1 -- 172.21.15.100:0/2093396600 shutdown_connections 2024-05-10T11:41:59.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.971+0000 7f7446355b80 1 --2- 172.21.15.100:0/2093396600 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55728975e360 0x557289760780 crc :-1 s=READY pgs=440 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:59.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.971+0000 7f7446355b80 1 --2- 172.21.15.100:0/2093396600 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5572896e0ad0 0x5572896fa7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:59.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.971+0000 7f7446355b80 1 --2- 172.21.15.100:0/2093396600 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5572896dad50 0x5572896ec800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:59.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.971+0000 7f7446355b80 1 --2- 172.21.15.100:0/2093396600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572896e01b0 0x5572896ecd40 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:59.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.971+0000 7f7446355b80 1 -- 172.21.15.100:0/2093396600 >> 172.21.15.100:0/2093396600 conn(0x557289606b70 msgr2=0x5572896dec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:41:59.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.971+0000 7f741d7fa700 0 client.19959 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:41:59.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.971+0000 7f7446355b80 1 -- 172.21.15.100:0/2093396600 shutdown_connections 2024-05-10T11:41:59.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.971+0000 7f7446355b80 1 -- 172.21.15.100:0/2093396600 wait complete. 2024-05-10T11:41:59.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.971+0000 7f7446355b80 20 client.19959 trim_cache size 0 max 16384 2024-05-10T11:41:59.976 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:41:59.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.972+0000 7f743bac2700 1 -- 172.21.15.100:0/1884166735 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f74200b2880 msgr2=0x7f74200b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:59.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.972+0000 7f743bac2700 1 --2- 172.21.15.100:0/1884166735 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f74200b2880 0x7f74200b4d40 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f7430004150 tx=0x7f7430018040 comp rx=0 tx=0).stop 2024-05-10T11:41:59.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.972+0000 7f743bac2700 1 -- 172.21.15.100:0/1884166735 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7434103ee0 msgr2=0x7f7434075ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:41:59.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.972+0000 7f743bac2700 1 --2- 172.21.15.100:0/1884166735 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7434103ee0 0x7f7434075ce0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f742400d980 tx=0x7f742400dc90 comp rx=0 tx=0).stop 2024-05-10T11:41:59.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.972+0000 7f743bac2700 1 -- 172.21.15.100:0/1884166735 shutdown_connections 2024-05-10T11:41:59.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.972+0000 7f743bac2700 1 --2- 172.21.15.100:0/1884166735 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f74200b2880 0x7f74200b4d40 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:59.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.972+0000 7f743bac2700 1 --2- 172.21.15.100:0/1884166735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7434109530 0x7f7434076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:59.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.972+0000 7f743bac2700 1 --2- 172.21.15.100:0/1884166735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7434103520 0x7f74340757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:59.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.972+0000 7f743bac2700 1 --2- 172.21.15.100:0/1884166735 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7434103ee0 0x7f7434075ce0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:41:59.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.972+0000 7f743bac2700 1 -- 172.21.15.100:0/1884166735 >> 172.21.15.100:0/1884166735 conn(0x7f743407a570 msgr2=0x7f7434102200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:41:59.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.972+0000 7f743bac2700 1 -- 172.21.15.100:0/1884166735 shutdown_connections 2024-05-10T11:41:59.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:41:59.972+0000 7f743bac2700 1 -- 172.21.15.100:0/1884166735 wait complete. 2024-05-10T11:42:00.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:41:59 smithi100 ceph-mon[30296]: pgmap v1129: 129 pgs: 129 active+clean; 571 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 4.3 MiB/s wr, 276 op/s 2024-05-10T11:42:00.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:41:59 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:42:00.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:41:59 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:42:00.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:41:59 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:42:00.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:41:59 smithi178 ceph-mon[33359]: pgmap v1129: 129 pgs: 129 active+clean; 571 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 4.3 MiB/s wr, 276 op/s 2024-05-10T11:42:00.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:41:59 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:42:00.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:41:59 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:42:00.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:41:59 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:42:00.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:41:59 smithi188 ceph-mon[34075]: pgmap v1129: 129 pgs: 129 active+clean; 571 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 4.3 MiB/s wr, 276 op/s 2024-05-10T11:42:00.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:41:59 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:42:00.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:41:59 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:42:00.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:41:59 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:42:00.958 INFO:tasks.workunit.client.0.smithi100.stdout: 10 98681 8.92 MB/sec execute 474 sec latency 128.646 ms 2024-05-10T11:42:01.758 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.758+0000 7fc1989af640 1 -- 172.21.15.100:0/935298542 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc194101950 msgr2=0x7fc194101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.758+0000 7fc1989af640 1 --2- 172.21.15.100:0/935298542 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc194101950 0x7fc194101d70 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fc184009900 tx=0x7fc18402a410 comp rx=0 tx=0).stop 2024-05-10T11:42:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.758+0000 7fc1989af640 1 -- 172.21.15.100:0/935298542 shutdown_connections 2024-05-10T11:42:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.758+0000 7fc1989af640 1 --2- 172.21.15.100:0/935298542 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc194103500 0x7fc194109d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.758+0000 7fc1989af640 1 --2- 172.21.15.100:0/935298542 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc194102b40 0x7fc194102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.758+0000 7fc1989af640 1 --2- 172.21.15.100:0/935298542 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc194101950 0x7fc194101d70 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.758+0000 7fc1989af640 1 -- 172.21.15.100:0/935298542 >> 172.21.15.100:0/935298542 conn(0x7fc1940fcec0 msgr2=0x7fc1940ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:42:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.758+0000 7fc1989af640 1 -- 172.21.15.100:0/935298542 shutdown_connections 2024-05-10T11:42:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.758+0000 7fc1989af640 1 -- 172.21.15.100:0/935298542 wait complete. 2024-05-10T11:42:01.761 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.759+0000 7fc1989af640 1 Processor -- start 2024-05-10T11:42:01.761 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.759+0000 7fc1989af640 1 -- start start 2024-05-10T11:42:01.761 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.759+0000 7fc1989af640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc194101950 0x7fc194198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:01.761 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.759+0000 7fc1989af640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc194102b40 0x7fc194199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.759+0000 7fc1989af640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc194103500 0x7fc1941a00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.759+0000 7fc1989af640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc19410d320 con 0x7fc194102b40 2024-05-10T11:42:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.759+0000 7fc1989af640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc19410d1a0 con 0x7fc194101950 2024-05-10T11:42:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.759+0000 7fc1989af640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc19410d4a0 con 0x7fc194103500 2024-05-10T11:42:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.759+0000 7fc19259c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc194101950 0x7fc194198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.759+0000 7fc18bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc194102b40 0x7fc194199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.759+0000 7fc19259c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc194101950 0x7fc194198bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:59526/0 (socket says 172.21.15.100:59526) 2024-05-10T11:42:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.759+0000 7fc19259c640 1 -- 172.21.15.100:0/2954952363 learned_addr learned my addr 172.21.15.100:0/2954952363 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:42:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.760+0000 7fc192d9d640 1 --2- 172.21.15.100:0/2954952363 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc194103500 0x7fc1941a00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.760+0000 7fc19259c640 1 -- 172.21.15.100:0/2954952363 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc194103500 msgr2=0x7fc1941a00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.760+0000 7fc19259c640 1 --2- 172.21.15.100:0/2954952363 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc194103500 0x7fc1941a00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.760+0000 7fc19259c640 1 -- 172.21.15.100:0/2954952363 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc194102b40 msgr2=0x7fc194199110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.760+0000 7fc19259c640 1 --2- 172.21.15.100:0/2954952363 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc194102b40 0x7fc194199110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.760+0000 7fc19259c640 1 -- 172.21.15.100:0/2954952363 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1941a0750 con 0x7fc194101950 2024-05-10T11:42:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.760+0000 7fc192d9d640 1 --2- 172.21.15.100:0/2954952363 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc194103500 0x7fc1941a00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:42:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.760+0000 7fc18bfff640 1 --2- 172.21.15.100:0/2954952363 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc194102b40 0x7fc194199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:42:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.760+0000 7fc19259c640 1 --2- 172.21.15.100:0/2954952363 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc194101950 0x7fc194198bd0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fc184005ab0 tx=0x7fc1840042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:42:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.760+0000 7fc18b7fe640 1 -- 172.21.15.100:0/2954952363 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18403f030 con 0x7fc194101950 2024-05-10T11:42:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.760+0000 7fc18b7fe640 1 -- 172.21.15.100:0/2954952363 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc184032e10 con 0x7fc194101950 2024-05-10T11:42:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.760+0000 7fc1989af640 1 -- 172.21.15.100:0/2954952363 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1941a09e0 con 0x7fc194101950 2024-05-10T11:42:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.761+0000 7fc1989af640 1 -- 172.21.15.100:0/2954952363 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1941a0ed0 con 0x7fc194101950 2024-05-10T11:42:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.761+0000 7fc18b7fe640 1 -- 172.21.15.100:0/2954952363 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc184035070 con 0x7fc194101950 2024-05-10T11:42:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.764+0000 7fc18b7fe640 1 -- 172.21.15.100:0/2954952363 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fc184042030 con 0x7fc194101950 2024-05-10T11:42:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.764+0000 7fc19b7a4740 1 Processor -- start 2024-05-10T11:42:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.764+0000 7fc19b7a4740 1 -- start start 2024-05-10T11:42:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.765+0000 7fc19b7a4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55654e79b1c0 0x55654e79b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.765+0000 7fc19b7a4740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55654e79bbb0 0x55654e7a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.765+0000 7fc192d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55654e79b1c0 0x55654e79b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.765+0000 7fc192d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55654e79b1c0 0x55654e79b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57422/0 (socket says 172.21.15.100:57422) 2024-05-10T11:42:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.765+0000 7fc19b7a4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55654e7a0d20 0x55654e7a31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.765+0000 7fc192d9d640 1 -- 172.21.15.100:0/682969362 learned_addr learned my addr 172.21.15.100:0/682969362 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:42:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.765+0000 7fc19b7a4740 1 -- 172.21.15.100:0/682969362 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55654e6fb5d0 con 0x55654e79b1c0 2024-05-10T11:42:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.765+0000 7fc19b7a4740 1 -- 172.21.15.100:0/682969362 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55654e6b4890 con 0x55654e7a0d20 2024-05-10T11:42:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.765+0000 7fc19b7a4740 1 -- 172.21.15.100:0/682969362 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55654e6f7590 con 0x55654e79bbb0 2024-05-10T11:42:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.765+0000 7fc19259c640 1 --2- 172.21.15.100:0/682969362 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55654e79bbb0 0x55654e7a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.765+0000 7fc18bfff640 1 --2- 172.21.15.100:0/682969362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55654e7a0d20 0x55654e7a31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.766+0000 7fc192d9d640 1 -- 172.21.15.100:0/682969362 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55654e79bbb0 msgr2=0x55654e7a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.766+0000 7fc192d9d640 1 --2- 172.21.15.100:0/682969362 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55654e79bbb0 0x55654e7a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.766+0000 7fc192d9d640 1 -- 172.21.15.100:0/682969362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55654e7a0d20 msgr2=0x55654e7a31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.766+0000 7fc192d9d640 1 --2- 172.21.15.100:0/682969362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55654e7a0d20 0x55654e7a31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.766+0000 7fc192d9d640 1 -- 172.21.15.100:0/682969362 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55654e40d210 con 0x55654e79b1c0 2024-05-10T11:42:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.766+0000 7fc19259c640 1 --2- 172.21.15.100:0/682969362 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55654e79bbb0 0x55654e7a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:42:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.767+0000 7fc18bfff640 1 --2- 172.21.15.100:0/682969362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55654e7a0d20 0x55654e7a31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:42:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.767+0000 7fc192d9d640 1 --2- 172.21.15.100:0/682969362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55654e79b1c0 0x55654e79b5e0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fc18c009fd0 tx=0x7fc18c0108e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3415e0f4c0711e8c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:42:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.767+0000 7fc18b7fe640 1 --2- 172.21.15.100:0/2954952363 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc1800b2a70 0x7fc1800b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.767+0000 7fc18b7fe640 1 -- 172.21.15.100:0/2954952363 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fc184102980 con 0x7fc194101950 2024-05-10T11:42:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.768+0000 7fc188ff9640 1 -- 172.21.15.100:0/682969362 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c011760 con 0x55654e79b1c0 2024-05-10T11:42:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.768+0000 7fc188ff9640 1 -- 172.21.15.100:0/682969362 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc18c011d90 con 0x55654e79b1c0 2024-05-10T11:42:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.768+0000 7fc188ff9640 1 -- 172.21.15.100:0/682969362 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c00e3c0 con 0x55654e79b1c0 2024-05-10T11:42:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.769+0000 7fc19b7a4740 1 -- 172.21.15.100:0/682969362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55654e79b1c0 msgr2=0x55654e79b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.769+0000 7fc19b7a4740 1 --2- 172.21.15.100:0/682969362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55654e79b1c0 0x55654e79b5e0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fc18c009fd0 tx=0x7fc18c0108e0 comp rx=0 tx=0).stop 2024-05-10T11:42:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.769+0000 7fc19b7a4740 1 -- 172.21.15.100:0/682969362 shutdown_connections 2024-05-10T11:42:01.770 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.769+0000 7fc19b7a4740 1 --2- 172.21.15.100:0/682969362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55654e7a0d20 0x55654e7a31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:01.770 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.769+0000 7fc19b7a4740 1 --2- 172.21.15.100:0/682969362 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55654e79bbb0 0x55654e7a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:01.770 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.769+0000 7fc19b7a4740 1 --2- 172.21.15.100:0/682969362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55654e79b1c0 0x55654e79b5e0 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:01.770 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.769+0000 7fc19b7a4740 1 -- 172.21.15.100:0/682969362 >> 172.21.15.100:0/682969362 conn(0x55654e796830 msgr2=0x55654e796c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:42:01.771 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.771+0000 7fc19259c640 1 --2- 172.21.15.100:0/2954952363 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc1800b2a70 0x7fc1800b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:01.771 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.771+0000 7fc19b7a4740 1 -- 172.21.15.100:0/682969362 shutdown_connections 2024-05-10T11:42:01.771 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.771+0000 7fc19b7a4740 1 -- 172.21.15.100:0/682969362 wait complete. 2024-05-10T11:42:01.771 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.771+0000 7fc19b7a4740 1 -- start start 2024-05-10T11:42:01.772 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.771+0000 7fc19b7a4740 1 Processor -- start 2024-05-10T11:42:01.772 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.772+0000 7fc19b7a4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55654e79b1c0 0x55654e7a5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.772+0000 7fc19b7a4740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55654e7a0d20 0x55654e7a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.772+0000 7fc19b7a4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55654e7aab10 0x55654e7acf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.772+0000 7fc19b7a4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55654e7a3cd0 con 0x55654e79b1c0 2024-05-10T11:42:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.772+0000 7fc19b7a4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55654e6b8370 con 0x55654e7aab10 2024-05-10T11:42:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.772+0000 7fc19b7a4740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55654e7a3e50 con 0x55654e7a0d20 2024-05-10T11:42:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.772+0000 7fc192d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55654e79b1c0 0x55654e7a5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.772+0000 7fc192d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55654e79b1c0 0x55654e7a5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57430/0 (socket says 172.21.15.100:57430) 2024-05-10T11:42:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.772+0000 7fc192d9d640 1 -- 172.21.15.100:0/3803516575 learned_addr learned my addr 172.21.15.100:0/3803516575 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:42:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.772+0000 7fc19b7a4740 10 client.0 _ll_register_callbacks cb 0x55654e40ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:42:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.772+0000 7fc192d9d640 1 --2- 172.21.15.100:0/3803516575 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55654e7aab10 0x55654e7acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.772+0000 7fc19259c640 1 --2- 172.21.15.100:0/2954952363 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc1800b2a70 0x7fc1800b4f30 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x55654e79c070 tx=0x7fc18403b460 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:42:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.773+0000 7fc192d9d640 1 -- 172.21.15.100:0/3803516575 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55654e7a0d20 msgr2=0x55654e7a5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.773+0000 7fc192d9d640 1 --2- 172.21.15.100:0/3803516575 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55654e7a0d20 0x55654e7a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.773+0000 7fc192d9d640 1 -- 172.21.15.100:0/3803516575 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55654e7aab10 msgr2=0x55654e7acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.773+0000 7fc192d9d640 1 --2- 172.21.15.100:0/3803516575 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55654e7aab10 0x55654e7acf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.773+0000 7fc192d9d640 1 -- 172.21.15.100:0/3803516575 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55654e4d9620 con 0x55654e79b1c0 2024-05-10T11:42:01.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.773+0000 7fc192d9d640 1 --2- 172.21.15.100:0/3803516575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55654e79b1c0 0x55654e7a5610 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fc18c009e10 tx=0x7fc18c010f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9ad3401adb18ec3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:42:01.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.773+0000 7fc179ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:42:01.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.773+0000 7fc179ffb640 1 -- 172.21.15.100:0/3803516575 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c02f060 con 0x55654e79b1c0 2024-05-10T11:42:01.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.773+0000 7fc179ffb640 1 -- 172.21.15.100:0/3803516575 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fc18c023070 con 0x55654e79b1c0 2024-05-10T11:42:01.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.773+0000 7fc19b7a4740 1 -- 172.21.15.100:0/3803516575 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc18c011d90 con 0x55654e79b1c0 2024-05-10T11:42:01.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.773+0000 7fc179ffb640 1 -- 172.21.15.100:0/3803516575 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fc18c036050 con 0x55654e79b1c0 2024-05-10T11:42:01.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.773+0000 7fc19b7a4740 10 client.19971 fetch_fsmap learned FSMap version 41 2024-05-10T11:42:01.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.773+0000 7fc19b7a4740 1 -- 172.21.15.100:0/3803516575 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55654e7afe50 con 0x55654e79b1c0 2024-05-10T11:42:01.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.774+0000 7fc179ffb640 1 -- 172.21.15.100:0/3803516575 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc18c011d90 con 0x55654e79b1c0 2024-05-10T11:42:01.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.774+0000 7fc179ffb640 1 -- 172.21.15.100:0/3803516575 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fc18c026050 con 0x55654e79b1c0 2024-05-10T11:42:01.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.774+0000 7fc19b7a4740 10 client.19971 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:42:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.774+0000 7fc19b7a4740 10 client.19971 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:42:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.774+0000 7fc19b7a4740 20 client.19971 populate_metadata read hostname 'smithi100' 2024-05-10T11:42:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.774+0000 7fc19b7a4740 1 --2- 172.21.15.100:0/3803516575 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55654e7b1260 0x55654e7b3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.774+0000 7fc19b7a4740 4 client.19971 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:42:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.774+0000 7fc19b7a4740 1 -- 172.21.15.100:0/3803516575 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55654e7091e0 con 0x55654e7b1260 2024-05-10T11:42:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.774+0000 7fc18bfff640 1 --2- 172.21.15.100:0/3803516575 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55654e7b1260 0x55654e7b3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.775+0000 7fc18bfff640 1 --2- 172.21.15.100:0/3803516575 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55654e7b1260 0x55654e7b3650 crc :-1 s=READY pgs=441 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e665c702ae4aadb server_cookie=f33831ba8fb49b07 in_seq=0 out_seq=0 2024-05-10T11:42:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.775+0000 7fc179ffb640 10 client.19971 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:42:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.778+0000 7fc179ffb640 1 -- 172.21.15.100:0/3803516575 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55654e7091e0 con 0x55654e7b1260 2024-05-10T11:42:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.778+0000 7fc179ffb640 10 client.19971 handle_command_reply: tid=0 2024-05-10T11:42:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.778+0000 7fc19b7a4740 1 client.19971 shutdown 2024-05-10T11:42:01.779 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.778+0000 7fc19b7a4740 1 -- 172.21.15.100:0/3803516575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55654e79b1c0 msgr2=0x55654e7a5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:01.779 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.778+0000 7fc19b7a4740 1 --2- 172.21.15.100:0/3803516575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55654e79b1c0 0x55654e7a5610 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fc18c009e10 tx=0x7fc18c010f90 comp rx=0 tx=0).stop 2024-05-10T11:42:01.779 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.778+0000 7fc19b7a4740 1 -- 172.21.15.100:0/3803516575 shutdown_connections 2024-05-10T11:42:01.779 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.778+0000 7fc19b7a4740 1 --2- 172.21.15.100:0/3803516575 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55654e7b1260 0x55654e7b3650 crc :-1 s=READY pgs=441 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:01.779 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.778+0000 7fc19b7a4740 1 --2- 172.21.15.100:0/3803516575 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55654e7aab10 0x55654e7acf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:01.779 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.778+0000 7fc19b7a4740 1 --2- 172.21.15.100:0/3803516575 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55654e7a0d20 0x55654e7a5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:01.779 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.778+0000 7fc19b7a4740 1 --2- 172.21.15.100:0/3803516575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55654e79b1c0 0x55654e7a5610 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:01.779 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.778+0000 7fc19b7a4740 1 -- 172.21.15.100:0/3803516575 >> 172.21.15.100:0/3803516575 conn(0x55654e796830 msgr2=0x55654e791580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:42:01.779 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.778+0000 7fc179ffb640 0 client.19971 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:42:01.780 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.778+0000 7fc19b7a4740 1 -- 172.21.15.100:0/3803516575 shutdown_connections 2024-05-10T11:42:01.780 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.778+0000 7fc19b7a4740 1 -- 172.21.15.100:0/3803516575 wait complete. 2024-05-10T11:42:01.780 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.778+0000 7fc19b7a4740 20 client.19971 trim_cache size 0 max 16384 2024-05-10T11:42:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.802+0000 7fc19b7a4740 1 Processor -- start 2024-05-10T11:42:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.802+0000 7fc19b7a4740 1 -- start start 2024-05-10T11:42:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.802+0000 7fc19b7a4740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55654e79b1c0 0x55654e7a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.802+0000 7fc19b7a4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55654e7a0d20 0x55654e7a3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.802+0000 7fc19b7a4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55654e7aab10 0x55654e7a5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.802+0000 7fc19b7a4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc1941ab9f0 con 0x55654e7a0d20 2024-05-10T11:42:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.802+0000 7fc19b7a4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc1941a92c0 con 0x55654e7aab10 2024-05-10T11:42:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.802+0000 7fc19b7a4740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc1941a7110 con 0x55654e79b1c0 2024-05-10T11:42:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.803+0000 7fc18bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55654e7a0d20 0x55654e7a3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.803+0000 7fc18bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55654e7a0d20 0x55654e7a3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57446/0 (socket says 172.21.15.100:57446) 2024-05-10T11:42:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.803+0000 7fc18bfff640 1 -- 172.21.15.100:0/128334474 learned_addr learned my addr 172.21.15.100:0/128334474 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:42:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.803+0000 7fc192d9d640 1 --2- 172.21.15.100:0/128334474 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55654e7aab10 0x55654e7a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.803+0000 7fc192d9d640 1 --2- 172.21.15.100:0/128334474 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55654e79b1c0 0x55654e7a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.803+0000 7fc18bfff640 1 -- 172.21.15.100:0/128334474 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55654e79b1c0 msgr2=0x55654e7a05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.803+0000 7fc18bfff640 1 --2- 172.21.15.100:0/128334474 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55654e79b1c0 0x55654e7a05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.803+0000 7fc18bfff640 1 -- 172.21.15.100:0/128334474 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55654e7aab10 msgr2=0x55654e7a5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.803+0000 7fc18bfff640 1 --2- 172.21.15.100:0/128334474 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55654e7aab10 0x55654e7a5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.803+0000 7fc18bfff640 1 -- 172.21.15.100:0/128334474 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55654e797500 con 0x55654e7a0d20 2024-05-10T11:42:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.803+0000 7fc192d9d640 1 --2- 172.21.15.100:0/128334474 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55654e7aab10 0x55654e7a5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:42:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.803+0000 7fc18bfff640 1 --2- 172.21.15.100:0/128334474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55654e7a0d20 0x55654e7a3110 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fc17c009be0 tx=0x7fc17c009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7fe773d6695ba33 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:42:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.803+0000 7fc17bfff640 1 -- 172.21.15.100:0/128334474 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17c005020 con 0x55654e7a0d20 2024-05-10T11:42:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.804+0000 7fc17bfff640 1 -- 172.21.15.100:0/128334474 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc17c01f030 con 0x55654e7a0d20 2024-05-10T11:42:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.804+0000 7fc17bfff640 1 -- 172.21.15.100:0/128334474 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17c01b390 con 0x55654e7a0d20 2024-05-10T11:42:01.809 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.804+0000 7fc19b7a4740 1 -- 172.21.15.100:0/128334474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55654e7a0d20 msgr2=0x55654e7a3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:01.809 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.804+0000 7fc19b7a4740 1 --2- 172.21.15.100:0/128334474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55654e7a0d20 0x55654e7a3110 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fc17c009be0 tx=0x7fc17c009ca0 comp rx=0 tx=0).stop 2024-05-10T11:42:01.809 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.805+0000 7fc19b7a4740 1 -- 172.21.15.100:0/128334474 shutdown_connections 2024-05-10T11:42:01.809 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.805+0000 7fc19b7a4740 1 --2- 172.21.15.100:0/128334474 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55654e7aab10 0x55654e7a5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:01.809 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.805+0000 7fc19b7a4740 1 --2- 172.21.15.100:0/128334474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55654e7a0d20 0x55654e7a3110 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:01.809 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.805+0000 7fc19b7a4740 1 --2- 172.21.15.100:0/128334474 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55654e79b1c0 0x55654e7a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.805+0000 7fc19b7a4740 1 -- 172.21.15.100:0/128334474 >> 172.21.15.100:0/128334474 conn(0x55654e796830 msgr2=0x55654e78d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:42:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.805+0000 7fc19b7a4740 1 -- 172.21.15.100:0/128334474 shutdown_connections 2024-05-10T11:42:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.805+0000 7fc19b7a4740 1 -- 172.21.15.100:0/128334474 wait complete. 2024-05-10T11:42:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.805+0000 7fc19b7a4740 1 -- start start 2024-05-10T11:42:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.805+0000 7fc19b7a4740 1 Processor -- start 2024-05-10T11:42:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.805+0000 7fc19b7a4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55654e79b1c0 0x55654e7c20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.806+0000 7fc19b7a4740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55654e7a0d20 0x55654e7c25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.806+0000 7fc19b7a4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55654e7aab10 0x55654e7c3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.806+0000 7fc192d9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55654e79b1c0 0x55654e7c20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.806+0000 7fc192d9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55654e79b1c0 0x55654e7c20a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:59562/0 (socket says 172.21.15.100:59562) 2024-05-10T11:42:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.806+0000 7fc18bfff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55654e7a0d20 0x55654e7c25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.806+0000 7fc192d9d640 1 -- 172.21.15.100:0/3908194308 learned_addr learned my addr 172.21.15.100:0/3908194308 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:42:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.806+0000 7fc19b7a4740 1 -- 172.21.15.100:0/3908194308 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc1941a7780 con 0x55654e7aab10 2024-05-10T11:42:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.806+0000 7fc19b7a4740 1 -- 172.21.15.100:0/3908194308 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc1941a8bf0 con 0x55654e79b1c0 2024-05-10T11:42:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.806+0000 7fc19b7a4740 1 -- 172.21.15.100:0/3908194308 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc1941a7e50 con 0x55654e7a0d20 2024-05-10T11:42:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.806+0000 7fc192d9d640 1 --2- 172.21.15.100:0/3908194308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55654e7aab10 0x55654e7c3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.806+0000 7fc19b7a4740 10 client.0 _ll_register_callbacks cb 0x7fc18c009ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:42:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.806+0000 7fc192d9d640 1 -- 172.21.15.100:0/3908194308 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55654e7a0d20 msgr2=0x55654e7c25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.806+0000 7fc192d9d640 1 --2- 172.21.15.100:0/3908194308 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55654e7a0d20 0x55654e7c25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.806+0000 7fc192d9d640 1 -- 172.21.15.100:0/3908194308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55654e7aab10 msgr2=0x55654e7c3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.806+0000 7fc192d9d640 1 --2- 172.21.15.100:0/3908194308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55654e7aab10 0x55654e7c3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.806+0000 7fc192d9d640 1 -- 172.21.15.100:0/3908194308 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55654e7c3b70 con 0x55654e79b1c0 2024-05-10T11:42:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.806+0000 7fc192d9d640 1 --2- 172.21.15.100:0/3908194308 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55654e79b1c0 0x55654e7c20a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc18c0053b0 tx=0x7fc18c010f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54489d718592434 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:42:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.806+0000 7fc179ffb640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:42:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.807+0000 7fc179ffb640 1 -- 172.21.15.100:0/3908194308 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c02f040 con 0x55654e79b1c0 2024-05-10T11:42:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.807+0000 7fc19b7a4740 1 -- 172.21.15.100:0/3908194308 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc17c01f030 con 0x55654e79b1c0 2024-05-10T11:42:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.807+0000 7fc179ffb640 1 -- 172.21.15.100:0/3908194308 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fc18c005960 con 0x55654e79b1c0 2024-05-10T11:42:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.807+0000 7fc179ffb640 1 -- 172.21.15.100:0/3908194308 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fc18c01b730 con 0x55654e79b1c0 2024-05-10T11:42:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.807+0000 7fc19b7a4740 10 client.29225 fetch_fsmap learned FSMap version 41 2024-05-10T11:42:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.807+0000 7fc19b7a4740 1 -- 172.21.15.100:0/3908194308 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55654e7c4e40 con 0x55654e79b1c0 2024-05-10T11:42:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.807+0000 7fc179ffb640 1 -- 172.21.15.100:0/3908194308 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc17c01f030 con 0x55654e79b1c0 2024-05-10T11:42:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.807+0000 7fc179ffb640 1 -- 172.21.15.100:0/3908194308 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fc18c026040 con 0x55654e79b1c0 2024-05-10T11:42:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.807+0000 7fc19b7a4740 10 client.29225 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:42:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.807+0000 7fc19b7a4740 10 client.29225 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:42:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.807+0000 7fc19b7a4740 20 client.29225 populate_metadata read hostname 'smithi100' 2024-05-10T11:42:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.807+0000 7fc19b7a4740 1 --2- 172.21.15.100:0/3908194308 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55654e792180 0x55654e7945a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.807+0000 7fc19b7a4740 4 client.29225 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:42:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.807+0000 7fc19b7a4740 1 -- 172.21.15.100:0/3908194308 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55654e703050 con 0x55654e792180 2024-05-10T11:42:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.807+0000 7fc18bfff640 1 --2- 172.21.15.100:0/3908194308 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55654e792180 0x55654e7945a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.808+0000 7fc179ffb640 10 client.29225 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:42:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.808+0000 7fc18bfff640 1 --2- 172.21.15.100:0/3908194308 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55654e792180 0x55654e7945a0 crc :-1 s=READY pgs=442 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c2b543af0a13aa9 server_cookie=f41653d0fb5bf9ec in_seq=0 out_seq=0 2024-05-10T11:42:01.928 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.927+0000 7fc179ffb640 1 -- 172.21.15.100:0/3908194308 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55654e703050 con 0x55654e792180 2024-05-10T11:42:01.928 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.927+0000 7fc179ffb640 10 client.29225 handle_command_reply: tid=0 2024-05-10T11:42:01.928 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.927+0000 7fc19b7a4740 1 client.29225 shutdown 2024-05-10T11:42:01.928 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.927+0000 7fc19b7a4740 1 -- 172.21.15.100:0/3908194308 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55654e79b1c0 msgr2=0x55654e7c20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:01.928 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.927+0000 7fc19b7a4740 1 --2- 172.21.15.100:0/3908194308 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55654e79b1c0 0x55654e7c20a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc18c0053b0 tx=0x7fc18c010f90 comp rx=0 tx=0).stop 2024-05-10T11:42:01.928 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.927+0000 7fc19b7a4740 1 -- 172.21.15.100:0/3908194308 shutdown_connections 2024-05-10T11:42:01.928 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.927+0000 7fc19b7a4740 1 --2- 172.21.15.100:0/3908194308 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55654e792180 0x55654e7945a0 crc :-1 s=READY pgs=442 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:01.928 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.927+0000 7fc19b7a4740 1 --2- 172.21.15.100:0/3908194308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55654e7aab10 0x55654e7c3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:01.928 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.927+0000 7fc19b7a4740 1 --2- 172.21.15.100:0/3908194308 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55654e7a0d20 0x55654e7c25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:01.928 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.927+0000 7fc179ffb640 0 client.29225 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:42:01.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.927+0000 7fc19b7a4740 1 --2- 172.21.15.100:0/3908194308 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55654e79b1c0 0x55654e7c20a0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:01.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.927+0000 7fc19b7a4740 1 -- 172.21.15.100:0/3908194308 >> 172.21.15.100:0/3908194308 conn(0x55654e796830 msgr2=0x55654e78d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:42:01.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.927+0000 7fc19b7a4740 1 -- 172.21.15.100:0/3908194308 shutdown_connections 2024-05-10T11:42:01.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.927+0000 7fc19b7a4740 1 -- 172.21.15.100:0/3908194308 wait complete. 2024-05-10T11:42:01.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.927+0000 7fc19b7a4740 20 client.29225 trim_cache size 0 max 16384 2024-05-10T11:42:01.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:42:01.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:42:01.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:42:01.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:42:01.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.928+0000 7fc1989af640 1 -- 172.21.15.100:0/2954952363 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc1800b2a70 msgr2=0x7fc1800b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:01.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.928+0000 7fc1989af640 1 --2- 172.21.15.100:0/2954952363 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc1800b2a70 0x7fc1800b4f30 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x55654e79c070 tx=0x7fc18403b460 comp rx=0 tx=0).stop 2024-05-10T11:42:01.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.928+0000 7fc1989af640 1 -- 172.21.15.100:0/2954952363 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc194101950 msgr2=0x7fc194198bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:01.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.928+0000 7fc1989af640 1 --2- 172.21.15.100:0/2954952363 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc194101950 0x7fc194198bd0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fc184005ab0 tx=0x7fc1840042f0 comp rx=0 tx=0).stop 2024-05-10T11:42:01.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.928+0000 7fc1989af640 1 -- 172.21.15.100:0/2954952363 shutdown_connections 2024-05-10T11:42:01.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.928+0000 7fc1989af640 1 --2- 172.21.15.100:0/2954952363 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc1800b2a70 0x7fc1800b4f30 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:01.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.928+0000 7fc1989af640 1 --2- 172.21.15.100:0/2954952363 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc194103500 0x7fc1941a00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:01.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.928+0000 7fc1989af640 1 --2- 172.21.15.100:0/2954952363 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc194102b40 0x7fc194199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:01.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.928+0000 7fc1989af640 1 --2- 172.21.15.100:0/2954952363 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc194101950 0x7fc194198bd0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:01.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.928+0000 7fc1989af640 1 -- 172.21.15.100:0/2954952363 >> 172.21.15.100:0/2954952363 conn(0x7fc1940fcec0 msgr2=0x7fc1940fea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:42:01.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.928+0000 7fc1989af640 1 -- 172.21.15.100:0/2954952363 shutdown_connections 2024-05-10T11:42:01.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:01.928+0000 7fc1989af640 1 -- 172.21.15.100:0/2954952363 wait complete. 2024-05-10T11:42:01.958 INFO:tasks.workunit.client.0.smithi100.stdout: 10 98818 8.92 MB/sec execute 475 sec latency 103.001 ms 2024-05-10T11:42:02.050 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T11:42:02.050 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T11:42:02.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:42:01 smithi178 ceph-mon[33359]: pgmap v1130: 129 pgs: 129 active+clean; 576 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 857 KiB/s rd, 4.6 MiB/s wr, 290 op/s 2024-05-10T11:42:02.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:42:01 smithi188 ceph-mon[34075]: pgmap v1130: 129 pgs: 129 active+clean; 576 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 857 KiB/s rd, 4.6 MiB/s wr, 290 op/s 2024-05-10T11:42:02.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:42:01 smithi100 ceph-mon[30296]: pgmap v1130: 129 pgs: 129 active+clean; 576 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 857 KiB/s rd, 4.6 MiB/s wr, 290 op/s 2024-05-10T11:42:02.403 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:42:02.958 INFO:tasks.workunit.client.0.smithi100.stdout: 10 98984 8.92 MB/sec execute 476 sec latency 91.915 ms 2024-05-10T11:42:03.958 INFO:tasks.workunit.client.0.smithi100.stdout: 10 99116 8.93 MB/sec execute 477 sec latency 455.801 ms 2024-05-10T11:42:04.213 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:42:03 smithi100 ceph-mon[30296]: pgmap v1131: 129 pgs: 129 active+clean; 577 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 616 KiB/s rd, 4.3 MiB/s wr, 245 op/s 2024-05-10T11:42:04.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:42:03 smithi178 ceph-mon[33359]: pgmap v1131: 129 pgs: 129 active+clean; 577 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 616 KiB/s rd, 4.3 MiB/s wr, 245 op/s 2024-05-10T11:42:04.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:42:03 smithi188 ceph-mon[34075]: pgmap v1131: 129 pgs: 129 active+clean; 577 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 616 KiB/s rd, 4.3 MiB/s wr, 245 op/s 2024-05-10T11:42:04.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.368+0000 7fe13272b700 1 -- 172.21.15.100:0/917981159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe12c102390 msgr2=0x7fe12c102810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:04.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.368+0000 7fe13272b700 1 --2- 172.21.15.100:0/917981159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe12c102390 0x7fe12c102810 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fe114009990 tx=0x7fe114009ca0 comp rx=0 tx=0).stop 2024-05-10T11:42:04.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.368+0000 7fe13272b700 1 -- 172.21.15.100:0/917981159 shutdown_connections 2024-05-10T11:42:04.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.368+0000 7fe13272b700 1 --2- 172.21.15.100:0/917981159 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe12c102d50 0x7fe12c10f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:04.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.368+0000 7fe13272b700 1 --2- 172.21.15.100:0/917981159 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe12c108400 0x7fe12c1087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:04.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.368+0000 7fe13272b700 1 --2- 172.21.15.100:0/917981159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe12c102390 0x7fe12c102810 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:04.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.368+0000 7fe13272b700 1 -- 172.21.15.100:0/917981159 >> 172.21.15.100:0/917981159 conn(0x7fe12c079310 msgr2=0x7fe12c079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:42:04.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.368+0000 7fe13272b700 1 -- 172.21.15.100:0/917981159 shutdown_connections 2024-05-10T11:42:04.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.368+0000 7fe13272b700 1 -- 172.21.15.100:0/917981159 wait complete. 2024-05-10T11:42:04.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.369+0000 7fe13272b700 1 Processor -- start 2024-05-10T11:42:04.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.369+0000 7fe13272b700 1 -- start start 2024-05-10T11:42:04.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.369+0000 7fe13272b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe12c102390 0x7fe12c1a18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:04.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.369+0000 7fe13272b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe12c102d50 0x7fe12c1a1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:04.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.369+0000 7fe13272b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe12c108400 0x7fe12c19ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:04.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.370+0000 7fe13272b700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe12c111390 con 0x7fe12c102d50 2024-05-10T11:42:04.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.370+0000 7fe13272b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe12c111210 con 0x7fe12c108400 2024-05-10T11:42:04.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.370+0000 7fe13272b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe12c111510 con 0x7fe12c102390 2024-05-10T11:42:04.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.370+0000 7fe123fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe12c102d50 0x7fe12c1a1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:04.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.370+0000 7fe123fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe12c102d50 0x7fe12c1a1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57466/0 (socket says 172.21.15.100:57466) 2024-05-10T11:42:04.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.370+0000 7fe123fff700 1 -- 172.21.15.100:0/3563877332 learned_addr learned my addr 172.21.15.100:0/3563877332 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:42:04.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.370+0000 7fe12bfff700 1 --2- 172.21.15.100:0/3563877332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe12c102390 0x7fe12c1a18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:04.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.370+0000 7fe123fff700 1 -- 172.21.15.100:0/3563877332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe12c102390 msgr2=0x7fe12c1a18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:04.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.370+0000 7fe123fff700 1 --2- 172.21.15.100:0/3563877332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe12c102390 0x7fe12c1a18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:04.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.370+0000 7fe123fff700 1 -- 172.21.15.100:0/3563877332 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe12c108400 msgr2=0x7fe12c19ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:04.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.370+0000 7fe123fff700 1 --2- 172.21.15.100:0/3563877332 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe12c108400 0x7fe12c19ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:04.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.370+0000 7fe123fff700 1 -- 172.21.15.100:0/3563877332 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe12c19c1e0 con 0x7fe12c102d50 2024-05-10T11:42:04.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.370+0000 7fe123fff700 1 --2- 172.21.15.100:0/3563877332 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe12c102d50 0x7fe12c1a1e00 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fe11400b650 tx=0x7fe1140047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:42:04.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.370+0000 7fe129ffb700 1 -- 172.21.15.100:0/3563877332 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe114027070 con 0x7fe12c102d50 2024-05-10T11:42:04.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.371+0000 7fe13272b700 1 -- 172.21.15.100:0/3563877332 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe12c19c470 con 0x7fe12c102d50 2024-05-10T11:42:04.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.371+0000 7fe129ffb700 1 -- 172.21.15.100:0/3563877332 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe114022470 con 0x7fe12c102d50 2024-05-10T11:42:04.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.371+0000 7fe129ffb700 1 -- 172.21.15.100:0/3563877332 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe114017650 con 0x7fe12c102d50 2024-05-10T11:42:04.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.371+0000 7fe13272b700 1 -- 172.21.15.100:0/3563877332 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe12c1a8790 con 0x7fe12c102d50 2024-05-10T11:42:04.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.374+0000 7fe129ffb700 1 -- 172.21.15.100:0/3563877332 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fe11402a070 con 0x7fe12c102d50 2024-05-10T11:42:04.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.375+0000 7fe129ffb700 1 --2- 172.21.15.100:0/3563877332 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe1180b2900 0x7fe1180b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:04.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.375+0000 7fe12bfff700 1 --2- 172.21.15.100:0/3563877332 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe1180b2900 0x7fe1180b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:04.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.375+0000 7fe129ffb700 1 -- 172.21.15.100:0/3563877332 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fe1140e3310 con 0x7fe12c102d50 2024-05-10T11:42:04.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.376+0000 7fe12bfff700 1 --2- 172.21.15.100:0/3563877332 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe1180b2900 0x7fe1180b4dc0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe12c19cbc0 tx=0x7fe11c00d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:42:04.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.376+0000 7fe13cfbeb80 1 Processor -- start 2024-05-10T11:42:04.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.377+0000 7fe13cfbeb80 1 -- start start 2024-05-10T11:42:04.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.377+0000 7fe13cfbeb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d18ae92230 0x55d18ae92610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:04.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.377+0000 7fe13cfbeb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d18ae92b50 0x55d18ae8c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:04.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.377+0000 7fe13cfbeb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d18ae8cdd0 0x55d18ae8d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:04.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.377+0000 7fe13cfbeb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55d18adde9c0 con 0x55d18ae8cdd0 2024-05-10T11:42:04.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.377+0000 7fe13cfbeb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d18ad43e20 con 0x55d18ae92230 2024-05-10T11:42:04.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.377+0000 7fe13cfbeb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d18adda980 con 0x55d18ae92b50 2024-05-10T11:42:04.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.377+0000 7fe123fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d18ae92b50 0x55d18ae8c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:04.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.377+0000 7fe12bfff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d18ae92230 0x55d18ae92610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:04.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.377+0000 7fe123fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d18ae92b50 0x55d18ae8c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:47456/0 (socket says 172.21.15.100:47456) 2024-05-10T11:42:04.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.377+0000 7fe123fff700 1 -- 172.21.15.100:0/3891921621 learned_addr learned my addr 172.21.15.100:0/3891921621 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:42:04.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.377+0000 7fe130cc8700 1 --2- 172.21.15.100:0/3891921621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d18ae8cdd0 0x55d18ae8d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:04.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.378+0000 7fe130cc8700 1 -- 172.21.15.100:0/3891921621 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d18ae92b50 msgr2=0x55d18ae8c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:04.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.378+0000 7fe130cc8700 1 --2- 172.21.15.100:0/3891921621 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d18ae92b50 0x55d18ae8c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:04.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.378+0000 7fe130cc8700 1 -- 172.21.15.100:0/3891921621 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d18ae92230 msgr2=0x55d18ae92610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:04.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.378+0000 7fe130cc8700 1 --2- 172.21.15.100:0/3891921621 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d18ae92230 0x55d18ae92610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:04.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.378+0000 7fe130cc8700 1 -- 172.21.15.100:0/3891921621 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d18ad2b950 con 0x55d18ae8cdd0 2024-05-10T11:42:04.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.378+0000 7fe12bfff700 1 --2- 172.21.15.100:0/3891921621 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d18ae92230 0x55d18ae92610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:42:04.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.378+0000 7fe130cc8700 1 --2- 172.21.15.100:0/3891921621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d18ae8cdd0 0x55d18ae8d210 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fe12400e490 tx=0x7fe12400e7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=febd6f2f12ab49ab server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:42:04.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.378+0000 7fe1227fc700 1 -- 172.21.15.100:0/3891921621 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe124018020 con 0x55d18ae8cdd0 2024-05-10T11:42:04.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.378+0000 7fe1227fc700 1 -- 172.21.15.100:0/3891921621 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe124004db0 con 0x55d18ae8cdd0 2024-05-10T11:42:04.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.378+0000 7fe1227fc700 1 -- 172.21.15.100:0/3891921621 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1240043f0 con 0x55d18ae8cdd0 2024-05-10T11:42:04.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.379+0000 7fe13cfbeb80 1 -- 172.21.15.100:0/3891921621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d18ae8cdd0 msgr2=0x55d18ae8d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:04.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.379+0000 7fe13cfbeb80 1 --2- 172.21.15.100:0/3891921621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d18ae8cdd0 0x55d18ae8d210 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fe12400e490 tx=0x7fe12400e7a0 comp rx=0 tx=0).stop 2024-05-10T11:42:04.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.379+0000 7fe13cfbeb80 1 -- 172.21.15.100:0/3891921621 shutdown_connections 2024-05-10T11:42:04.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.379+0000 7fe13cfbeb80 1 --2- 172.21.15.100:0/3891921621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d18ae8cdd0 0x55d18ae8d210 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:04.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.379+0000 7fe13cfbeb80 1 --2- 172.21.15.100:0/3891921621 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d18ae92230 0x55d18ae92610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:04.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.379+0000 7fe13cfbeb80 1 --2- 172.21.15.100:0/3891921621 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d18ae92b50 0x55d18ae8c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:04.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.379+0000 7fe13cfbeb80 1 -- 172.21.15.100:0/3891921621 >> 172.21.15.100:0/3891921621 conn(0x55d18adb8bf0 msgr2=0x55d18ae87320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:42:04.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.379+0000 7fe13cfbeb80 1 -- 172.21.15.100:0/3891921621 shutdown_connections 2024-05-10T11:42:04.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.379+0000 7fe13cfbeb80 1 -- 172.21.15.100:0/3891921621 wait complete. 2024-05-10T11:42:04.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.380+0000 7fe13cfbeb80 1 -- start start 2024-05-10T11:42:04.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.380+0000 7fe13cfbeb80 1 Processor -- start 2024-05-10T11:42:04.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.381+0000 7fe13cfbeb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d18ae8cdd0 0x55d18ae98680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:04.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.381+0000 7fe13cfbeb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d18ae92230 0x55d18ae95ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:04.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.381+0000 7fe13cfbeb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d18ae92b50 0x55d18ae96020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:04.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.381+0000 7fe12bfff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d18ae8cdd0 0x55d18ae98680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:04.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.381+0000 7fe12bfff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d18ae8cdd0 0x55d18ae98680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:59616/0 (socket says 172.21.15.100:59616) 2024-05-10T11:42:04.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.381+0000 7fe12bfff700 1 -- 172.21.15.100:0/2799966161 learned_addr learned my addr 172.21.15.100:0/2799966161 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:42:04.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.381+0000 7fe13cfbeb80 1 -- 172.21.15.100:0/2799966161 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55d18ae99520 con 0x55d18ae92b50 2024-05-10T11:42:04.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.381+0000 7fe13cfbeb80 1 -- 172.21.15.100:0/2799966161 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d18ad97f50 con 0x55d18ae8cdd0 2024-05-10T11:42:04.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.381+0000 7fe13cfbeb80 1 -- 172.21.15.100:0/2799966161 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d18ae996a0 con 0x55d18ae92230 2024-05-10T11:42:04.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.381+0000 7fe13cfbeb80 10 client.0 _ll_register_callbacks cb 0x55d18ad72630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:42:04.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.381+0000 7fe123fff700 1 --2- 172.21.15.100:0/2799966161 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d18ae92230 0x55d18ae95ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:04.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.381+0000 7fe130cc8700 1 --2- 172.21.15.100:0/2799966161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d18ae92b50 0x55d18ae96020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:04.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.381+0000 7fe12bfff700 1 -- 172.21.15.100:0/2799966161 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d18ae92230 msgr2=0x55d18ae95ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:04.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.381+0000 7fe12bfff700 1 --2- 172.21.15.100:0/2799966161 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d18ae92230 0x55d18ae95ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:04.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.381+0000 7fe12bfff700 1 -- 172.21.15.100:0/2799966161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d18ae92b50 msgr2=0x55d18ae96020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:04.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.381+0000 7fe12bfff700 1 --2- 172.21.15.100:0/2799966161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d18ae92b50 0x55d18ae96020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:04.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.381+0000 7fe12bfff700 1 -- 172.21.15.100:0/2799966161 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d18ad957e0 con 0x55d18ae8cdd0 2024-05-10T11:42:04.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.381+0000 7fe130cc8700 1 --2- 172.21.15.100:0/2799966161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d18ae92b50 0x55d18ae96020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:42:04.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.381+0000 7fe123fff700 1 --2- 172.21.15.100:0/2799966161 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d18ae92230 0x55d18ae95ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:42:04.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.381+0000 7fe12bfff700 1 --2- 172.21.15.100:0/2799966161 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d18ae8cdd0 0x55d18ae98680 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe11c018730 tx=0x7fe11c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54157ca34cff0e9c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:42:04.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.381+0000 7fe10b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:42:04.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.382+0000 7fe10b7fe700 1 -- 172.21.15.100:0/2799966161 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe11c016640 con 0x55d18ae8cdd0 2024-05-10T11:42:04.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.382+0000 7fe10b7fe700 1 -- 172.21.15.100:0/2799966161 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fe11c009050 con 0x55d18ae8cdd0 2024-05-10T11:42:04.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.382+0000 7fe13cfbeb80 1 -- 172.21.15.100:0/2799966161 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe124004db0 con 0x55d18ae8cdd0 2024-05-10T11:42:04.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.382+0000 7fe10b7fe700 1 -- 172.21.15.100:0/2799966161 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fe11c032050 con 0x55d18ae8cdd0 2024-05-10T11:42:04.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.382+0000 7fe13cfbeb80 10 client.29234 fetch_fsmap learned FSMap version 41 2024-05-10T11:42:04.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.382+0000 7fe13cfbeb80 1 -- 172.21.15.100:0/2799966161 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55d18aea3d70 con 0x55d18ae8cdd0 2024-05-10T11:42:04.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.382+0000 7fe10b7fe700 1 -- 172.21.15.100:0/2799966161 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe124004db0 con 0x55d18ae8cdd0 2024-05-10T11:42:04.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.383+0000 7fe10b7fe700 1 -- 172.21.15.100:0/2799966161 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fe11c02d750 con 0x55d18ae8cdd0 2024-05-10T11:42:04.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.383+0000 7fe13cfbeb80 10 client.29234 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:42:04.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.383+0000 7fe13cfbeb80 10 client.29234 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:42:04.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.383+0000 7fe13cfbeb80 20 client.29234 populate_metadata read hostname 'smithi100' 2024-05-10T11:42:04.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.383+0000 7fe13cfbeb80 1 --2- 172.21.15.100:0/2799966161 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d18aea63b0 0x55d18aea87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:04.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.383+0000 7fe13cfbeb80 4 client.29234 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:42:04.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.383+0000 7fe13cfbeb80 1 -- 172.21.15.100:0/2799966161 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d18adec3a0 con 0x55d18aea63b0 2024-05-10T11:42:04.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.383+0000 7fe123fff700 1 --2- 172.21.15.100:0/2799966161 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d18aea63b0 0x55d18aea87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:04.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.383+0000 7fe10b7fe700 10 client.29234 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:42:04.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.384+0000 7fe123fff700 1 --2- 172.21.15.100:0/2799966161 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d18aea63b0 0x55d18aea87e0 crc :-1 s=READY pgs=443 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb62e4c545ab0596 server_cookie=f15e136c6925ec95 in_seq=0 out_seq=0 2024-05-10T11:42:04.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.385+0000 7fe10b7fe700 1 -- 172.21.15.100:0/2799966161 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d18adec3a0 con 0x55d18aea63b0 2024-05-10T11:42:04.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.385+0000 7fe10b7fe700 10 client.29234 handle_command_reply: tid=0 2024-05-10T11:42:04.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.385+0000 7fe13cfbeb80 1 client.29234 shutdown 2024-05-10T11:42:04.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.385+0000 7fe13cfbeb80 1 -- 172.21.15.100:0/2799966161 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d18ae8cdd0 msgr2=0x55d18ae98680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:04.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.385+0000 7fe13cfbeb80 1 --2- 172.21.15.100:0/2799966161 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d18ae8cdd0 0x55d18ae98680 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe11c018730 tx=0x7fe11c004750 comp rx=0 tx=0).stop 2024-05-10T11:42:04.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.385+0000 7fe13cfbeb80 1 -- 172.21.15.100:0/2799966161 shutdown_connections 2024-05-10T11:42:04.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.385+0000 7fe13cfbeb80 1 --2- 172.21.15.100:0/2799966161 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d18aea63b0 0x55d18aea87e0 crc :-1 s=READY pgs=443 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:04.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.385+0000 7fe13cfbeb80 1 --2- 172.21.15.100:0/2799966161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d18ae92b50 0x55d18ae96020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:04.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.385+0000 7fe10b7fe700 0 client.29234 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:42:04.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.385+0000 7fe13cfbeb80 1 --2- 172.21.15.100:0/2799966161 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d18ae8cdd0 0x55d18ae98680 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:04.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.385+0000 7fe13cfbeb80 1 --2- 172.21.15.100:0/2799966161 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d18ae92230 0x55d18ae95ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:04.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.385+0000 7fe13cfbeb80 1 -- 172.21.15.100:0/2799966161 >> 172.21.15.100:0/2799966161 conn(0x55d18adb8bf0 msgr2=0x55d18ae81bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:42:04.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.385+0000 7fe13cfbeb80 1 -- 172.21.15.100:0/2799966161 shutdown_connections 2024-05-10T11:42:04.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.385+0000 7fe13cfbeb80 1 -- 172.21.15.100:0/2799966161 wait complete. 2024-05-10T11:42:04.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.386+0000 7fe13cfbeb80 20 client.29234 trim_cache size 0 max 16384 2024-05-10T11:42:04.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.419+0000 7fe13cfbeb80 1 Processor -- start 2024-05-10T11:42:04.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.419+0000 7fe13cfbeb80 1 -- start start 2024-05-10T11:42:04.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.419+0000 7fe13cfbeb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d18ae8cdd0 0x55d18ae8c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:04.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.419+0000 7fe13cfbeb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d18ae92230 0x55d18ae81cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:04.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.419+0000 7fe13cfbeb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d18ae92b50 0x55d18aeae0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:04.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.419+0000 7fe13cfbeb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe12c1b39e0 con 0x55d18ae92230 2024-05-10T11:42:04.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.419+0000 7fe13cfbeb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe12c1b1270 con 0x55d18ae92b50 2024-05-10T11:42:04.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.419+0000 7fe13cfbeb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe12c1af0c0 con 0x55d18ae8cdd0 2024-05-10T11:42:04.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.420+0000 7fe130cc8700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d18ae92b50 0x55d18aeae0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:04.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.420+0000 7fe123fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d18ae92230 0x55d18ae81cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:04.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.420+0000 7fe130cc8700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d18ae92b50 0x55d18aeae0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:59622/0 (socket says 172.21.15.100:59622) 2024-05-10T11:42:04.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.420+0000 7fe123fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d18ae92230 0x55d18ae81cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57486/0 (socket says 172.21.15.100:57486) 2024-05-10T11:42:04.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.420+0000 7fe130cc8700 1 -- 172.21.15.100:0/1897334977 learned_addr learned my addr 172.21.15.100:0/1897334977 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:42:04.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.420+0000 7fe130cc8700 1 -- 172.21.15.100:0/1897334977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d18ae8cdd0 msgr2=0x55d18ae8c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:04.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.420+0000 7fe130cc8700 1 --2- 172.21.15.100:0/1897334977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d18ae8cdd0 0x55d18ae8c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:04.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.420+0000 7fe130cc8700 1 -- 172.21.15.100:0/1897334977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d18ae92230 msgr2=0x55d18ae81cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:04.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.420+0000 7fe130cc8700 1 --2- 172.21.15.100:0/1897334977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d18ae92230 0x55d18ae81cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:04.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.420+0000 7fe130cc8700 1 -- 172.21.15.100:0/1897334977 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d18ae82200 con 0x55d18ae92b50 2024-05-10T11:42:04.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.420+0000 7fe130cc8700 1 --2- 172.21.15.100:0/1897334977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d18ae92b50 0x55d18aeae0b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe124009960 tx=0x7fe124004d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3b717fea437f731 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:42:04.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.421+0000 7fe121ffb700 1 -- 172.21.15.100:0/1897334977 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe124022020 con 0x55d18ae92b50 2024-05-10T11:42:04.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.421+0000 7fe121ffb700 1 -- 172.21.15.100:0/1897334977 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe1240092d0 con 0x55d18ae92b50 2024-05-10T11:42:04.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.421+0000 7fe121ffb700 1 -- 172.21.15.100:0/1897334977 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe124018070 con 0x55d18ae92b50 2024-05-10T11:42:04.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.422+0000 7fe13cfbeb80 1 -- 172.21.15.100:0/1897334977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d18ae92b50 msgr2=0x55d18aeae0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:04.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.422+0000 7fe13cfbeb80 1 --2- 172.21.15.100:0/1897334977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d18ae92b50 0x55d18aeae0b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe124009960 tx=0x7fe124004d40 comp rx=0 tx=0).stop 2024-05-10T11:42:04.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.422+0000 7fe13cfbeb80 1 -- 172.21.15.100:0/1897334977 shutdown_connections 2024-05-10T11:42:04.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.422+0000 7fe13cfbeb80 1 --2- 172.21.15.100:0/1897334977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d18ae92b50 0x55d18aeae0b0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:04.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.422+0000 7fe13cfbeb80 1 --2- 172.21.15.100:0/1897334977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d18ae8cdd0 0x55d18ae8c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:04.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.422+0000 7fe13cfbeb80 1 --2- 172.21.15.100:0/1897334977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d18ae92230 0x55d18ae81cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:04.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.422+0000 7fe13cfbeb80 1 -- 172.21.15.100:0/1897334977 >> 172.21.15.100:0/1897334977 conn(0x55d18adb8bf0 msgr2=0x55d18ae914c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:42:04.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.422+0000 7fe13cfbeb80 1 -- 172.21.15.100:0/1897334977 shutdown_connections 2024-05-10T11:42:04.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.424+0000 7fe13cfbeb80 1 -- 172.21.15.100:0/1897334977 wait complete. 2024-05-10T11:42:04.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.425+0000 7fe13cfbeb80 1 -- start start 2024-05-10T11:42:04.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.425+0000 7fe13cfbeb80 1 Processor -- start 2024-05-10T11:42:04.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.425+0000 7fe13cfbeb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d18ae8cdd0 0x55d18ae9cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:04.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.426+0000 7fe13cfbeb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d18ae92230 0x55d18ae9d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:04.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.426+0000 7fe13cfbeb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d18ae92b50 0x55d18af124c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:04.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.426+0000 7fe13cfbeb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe12c1af730 con 0x55d18ae8cdd0 2024-05-10T11:42:04.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.426+0000 7fe13cfbeb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe12c1b0ba0 con 0x55d18ae92230 2024-05-10T11:42:04.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.426+0000 7fe123fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d18ae92230 0x55d18ae9d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:04.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.426+0000 7fe123fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d18ae92230 0x55d18ae9d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:59638/0 (socket says 172.21.15.100:59638) 2024-05-10T11:42:04.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.426+0000 7fe123fff700 1 -- 172.21.15.100:0/2307168451 learned_addr learned my addr 172.21.15.100:0/2307168451 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:42:04.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.426+0000 7fe13cfbeb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe12c1afe00 con 0x55d18ae92b50 2024-05-10T11:42:04.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.426+0000 7fe13cfbeb80 10 client.0 _ll_register_callbacks cb 0x55d18ad72630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:42:04.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.426+0000 7fe123fff700 1 -- 172.21.15.100:0/2307168451 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d18ae92b50 msgr2=0x55d18af124c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:04.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.426+0000 7fe123fff700 1 --2- 172.21.15.100:0/2307168451 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d18ae92b50 0x55d18af124c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:04.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.426+0000 7fe123fff700 1 -- 172.21.15.100:0/2307168451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d18ae8cdd0 msgr2=0x55d18ae9cea0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T11:42:04.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.426+0000 7fe123fff700 1 --2- 172.21.15.100:0/2307168451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d18ae8cdd0 0x55d18ae9cea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:04.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.426+0000 7fe123fff700 1 -- 172.21.15.100:0/2307168451 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d18af12b00 con 0x55d18ae92230 2024-05-10T11:42:04.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.427+0000 7fe123fff700 1 --2- 172.21.15.100:0/2307168451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d18ae92230 0x55d18ae9d3e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe1140aa400 tx=0x7fe114092040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc583263134b503e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:42:04.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.427+0000 7fe10b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:42:04.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.427+0000 7fe10b7fe700 1 -- 172.21.15.100:0/2307168451 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1140c3070 con 0x55d18ae92230 2024-05-10T11:42:04.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.427+0000 7fe10b7fe700 1 -- 172.21.15.100:0/2307168451 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fe1140920b0 con 0x55d18ae92230 2024-05-10T11:42:04.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.428+0000 7fe13cfbeb80 1 -- 172.21.15.100:0/2307168451 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1240092d0 con 0x55d18ae92230 2024-05-10T11:42:04.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.429+0000 7fe10b7fe700 1 -- 172.21.15.100:0/2307168451 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fe1140aa700 con 0x55d18ae92230 2024-05-10T11:42:04.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.429+0000 7fe13cfbeb80 10 client.29240 fetch_fsmap learned FSMap version 41 2024-05-10T11:42:04.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.429+0000 7fe13cfbeb80 1 -- 172.21.15.100:0/2307168451 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55d18af19ef0 con 0x55d18ae92230 2024-05-10T11:42:04.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.429+0000 7fe10b7fe700 1 -- 172.21.15.100:0/2307168451 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe1240092d0 con 0x55d18ae92230 2024-05-10T11:42:04.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.429+0000 7fe10b7fe700 1 -- 172.21.15.100:0/2307168451 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fe1140aa960 con 0x55d18ae92230 2024-05-10T11:42:04.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.429+0000 7fe13cfbeb80 10 client.29240 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:42:04.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.429+0000 7fe13cfbeb80 10 client.29240 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:42:04.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.429+0000 7fe13cfbeb80 20 client.29240 populate_metadata read hostname 'smithi100' 2024-05-10T11:42:04.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.429+0000 7fe13cfbeb80 1 --2- 172.21.15.100:0/2307168451 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d18aea63b0 0x55d18af1c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:04.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.429+0000 7fe13cfbeb80 4 client.29240 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T11:42:04.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.429+0000 7fe13cfbeb80 1 -- 172.21.15.100:0/2307168451 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d18ade6440 con 0x55d18aea63b0 2024-05-10T11:42:04.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.432+0000 7fe12bfff700 1 --2- 172.21.15.100:0/2307168451 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d18aea63b0 0x55d18af1c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:04.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.435+0000 7fe10b7fe700 10 client.29240 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:42:04.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.435+0000 7fe12bfff700 1 --2- 172.21.15.100:0/2307168451 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d18aea63b0 0x55d18af1c6e0 crc :-1 s=READY pgs=444 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fcd2004a8b03fe8 server_cookie=2aadfca6de952c5a in_seq=0 out_seq=0 2024-05-10T11:42:04.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.436+0000 7fe10b7fe700 1 -- 172.21.15.100:0/2307168451 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d18ade6440 con 0x55d18aea63b0 2024-05-10T11:42:04.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.436+0000 7fe10b7fe700 10 client.29240 handle_command_reply: tid=0 2024-05-10T11:42:04.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.436+0000 7fe13cfbeb80 1 client.29240 shutdown 2024-05-10T11:42:04.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.437+0000 7fe13cfbeb80 1 -- 172.21.15.100:0/2307168451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d18ae92230 msgr2=0x55d18ae9d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:04.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.437+0000 7fe13cfbeb80 1 --2- 172.21.15.100:0/2307168451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d18ae92230 0x55d18ae9d3e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe1140aa400 tx=0x7fe114092040 comp rx=0 tx=0).stop 2024-05-10T11:42:04.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.437+0000 7fe13cfbeb80 1 -- 172.21.15.100:0/2307168451 shutdown_connections 2024-05-10T11:42:04.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.437+0000 7fe13cfbeb80 1 --2- 172.21.15.100:0/2307168451 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d18aea63b0 0x55d18af1c6e0 crc :-1 s=READY pgs=444 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:04.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.437+0000 7fe13cfbeb80 1 --2- 172.21.15.100:0/2307168451 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d18ae92b50 0x55d18af124c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:04.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.437+0000 7fe13cfbeb80 1 --2- 172.21.15.100:0/2307168451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d18ae8cdd0 0x55d18ae9cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:04.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.437+0000 7fe10b7fe700 0 client.29240 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:42:04.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.437+0000 7fe13cfbeb80 1 --2- 172.21.15.100:0/2307168451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d18ae92230 0x55d18ae9d3e0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:04.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.437+0000 7fe13cfbeb80 1 -- 172.21.15.100:0/2307168451 >> 172.21.15.100:0/2307168451 conn(0x55d18adb8bf0 msgr2=0x55d18ae8b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:42:04.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.437+0000 7fe13cfbeb80 1 -- 172.21.15.100:0/2307168451 shutdown_connections 2024-05-10T11:42:04.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.437+0000 7fe13cfbeb80 1 -- 172.21.15.100:0/2307168451 wait complete. 2024-05-10T11:42:04.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.437+0000 7fe13cfbeb80 20 client.29240 trim_cache size 0 max 16384 2024-05-10T11:42:04.439 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:42:04.439 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-05-10T11:42:04.439 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "db294da5-100f-42a6-bc9a-4905ccb97ab6", 2024-05-10T11:42:04.440 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-05-10T11:42:04.440 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:42:04.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.438+0000 7fe13272b700 1 -- 172.21.15.100:0/3563877332 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe1180b2900 msgr2=0x7fe1180b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:04.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.438+0000 7fe13272b700 1 --2- 172.21.15.100:0/3563877332 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe1180b2900 0x7fe1180b4dc0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe12c19cbc0 tx=0x7fe11c00d040 comp rx=0 tx=0).stop 2024-05-10T11:42:04.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.438+0000 7fe13272b700 1 -- 172.21.15.100:0/3563877332 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe12c102d50 msgr2=0x7fe12c1a1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:04.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.438+0000 7fe13272b700 1 --2- 172.21.15.100:0/3563877332 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe12c102d50 0x7fe12c1a1e00 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fe11400b650 tx=0x7fe1140047b0 comp rx=0 tx=0).stop 2024-05-10T11:42:04.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.439+0000 7fe13272b700 1 -- 172.21.15.100:0/3563877332 shutdown_connections 2024-05-10T11:42:04.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.439+0000 7fe13272b700 1 --2- 172.21.15.100:0/3563877332 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe1180b2900 0x7fe1180b4dc0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:04.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.439+0000 7fe13272b700 1 --2- 172.21.15.100:0/3563877332 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe12c108400 0x7fe12c19ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:04.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.439+0000 7fe13272b700 1 --2- 172.21.15.100:0/3563877332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe12c102390 0x7fe12c1a18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:04.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.439+0000 7fe13272b700 1 --2- 172.21.15.100:0/3563877332 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe12c102d50 0x7fe12c1a1e00 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:04.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.439+0000 7fe13272b700 1 -- 172.21.15.100:0/3563877332 >> 172.21.15.100:0/3563877332 conn(0x7fe12c079310 msgr2=0x7fe12c101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:42:04.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.439+0000 7fe13272b700 1 -- 172.21.15.100:0/3563877332 shutdown_connections 2024-05-10T11:42:04.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:04.439+0000 7fe13272b700 1 -- 172.21.15.100:0/3563877332 wait complete. 2024-05-10T11:42:04.958 INFO:tasks.workunit.client.0.smithi100.stdout: 10 99246 8.91 MB/sec execute 478 sec latency 127.077 ms 2024-05-10T11:42:05.817 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T11:42:05.958 INFO:tasks.workunit.client.0.smithi100.stdout: 10 99400 8.90 MB/sec execute 479 sec latency 94.902 ms 2024-05-10T11:42:06.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:42:05 smithi100 ceph-mon[30296]: scrub queued for path: / 2024-05-10T11:42:06.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:42:05 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:42:06.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:42:05 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:42:06.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:42:05 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:42:06.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:42:05 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:42:06.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:42:05 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:42:06.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:42:05 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:42:06.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:42:05 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:42:06.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:42:05 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:42:06.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:42:05 smithi100 ceph-mon[30296]: pgmap v1132: 129 pgs: 129 active+clean; 577 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 839 KiB/s rd, 4.8 MiB/s wr, 277 op/s 2024-05-10T11:42:06.039 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:42:05 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:42:06.159 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:42:06.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:42:05 smithi178 ceph-mon[33359]: scrub queued for path: / 2024-05-10T11:42:06.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:42:05 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:42:06.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:42:05 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:42:06.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:42:05 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:42:06.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:42:05 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:42:06.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:42:05 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:42:06.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:42:05 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:42:06.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:42:05 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:42:06.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:42:05 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:42:06.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:42:05 smithi178 ceph-mon[33359]: pgmap v1132: 129 pgs: 129 active+clean; 577 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 839 KiB/s rd, 4.8 MiB/s wr, 277 op/s 2024-05-10T11:42:06.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:42:05 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:42:06.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:42:05 smithi188 ceph-mon[34075]: scrub queued for path: / 2024-05-10T11:42:06.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:42:05 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:42:06.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:42:05 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:42:06.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:42:05 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:42:06.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:42:05 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:42:06.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:42:05 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:42:06.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:42:05 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:42:06.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:42:05 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:42:06.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:42:05 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:42:06.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:42:05 smithi188 ceph-mon[34075]: pgmap v1132: 129 pgs: 129 active+clean; 577 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 839 KiB/s rd, 4.8 MiB/s wr, 277 op/s 2024-05-10T11:42:06.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:42:05 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:42:06.958 INFO:tasks.workunit.client.0.smithi100.stdout: 10 99568 8.89 MB/sec execute 480 sec latency 82.390 ms 2024-05-10T11:42:07.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:42:07 smithi178 ceph-mon[33359]: pgmap v1133: 129 pgs: 129 active+clean; 565 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 974 KiB/s rd, 4.7 MiB/s wr, 334 op/s 2024-05-10T11:42:07.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:42:07 smithi188 ceph-mon[34075]: pgmap v1133: 129 pgs: 129 active+clean; 565 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 974 KiB/s rd, 4.7 MiB/s wr, 334 op/s 2024-05-10T11:42:07.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:42:07 smithi100 ceph-mon[30296]: pgmap v1133: 129 pgs: 129 active+clean; 565 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 974 KiB/s rd, 4.7 MiB/s wr, 334 op/s 2024-05-10T11:42:07.958 INFO:tasks.workunit.client.0.smithi100.stdout: 10 99733 8.89 MB/sec execute 481 sec latency 135.225 ms 2024-05-10T11:42:08.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.793+0000 7fc799b17700 1 -- 172.21.15.100:0/116788565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc794108410 msgr2=0x7fc7941087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:08.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.793+0000 7fc799b17700 1 --2- 172.21.15.100:0/116788565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc794108410 0x7fc7941087f0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fc784009990 tx=0x7fc784009ca0 comp rx=0 tx=0).stop 2024-05-10T11:42:08.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.793+0000 7fc799b17700 1 -- 172.21.15.100:0/116788565 shutdown_connections 2024-05-10T11:42:08.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.793+0000 7fc799b17700 1 --2- 172.21.15.100:0/116788565 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc794102d60 0x7fc79410f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:08.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.793+0000 7fc799b17700 1 --2- 172.21.15.100:0/116788565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc794108410 0x7fc7941087f0 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:08.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.793+0000 7fc799b17700 1 --2- 172.21.15.100:0/116788565 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7941023a0 0x7fc794102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:08.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.793+0000 7fc799b17700 1 -- 172.21.15.100:0/116788565 >> 172.21.15.100:0/116788565 conn(0x7fc794079350 msgr2=0x7fc794079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:42:08.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.793+0000 7fc799b17700 1 -- 172.21.15.100:0/116788565 shutdown_connections 2024-05-10T11:42:08.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.794+0000 7fc799b17700 1 -- 172.21.15.100:0/116788565 wait complete. 2024-05-10T11:42:08.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.794+0000 7fc799b17700 1 Processor -- start 2024-05-10T11:42:08.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.795+0000 7fc799b17700 1 -- start start 2024-05-10T11:42:08.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.795+0000 7fc799b17700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7941023a0 0x7fc7941092f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:08.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.795+0000 7fc799b17700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc794102d60 0x7fc794109830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:08.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.795+0000 7fc799b17700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc794108410 0x7fc794109d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:08.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.795+0000 7fc799b17700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc794112430 con 0x7fc794102d60 2024-05-10T11:42:08.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.795+0000 7fc799b17700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc7941122b0 con 0x7fc794108410 2024-05-10T11:42:08.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.795+0000 7fc799b17700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc7941125b0 con 0x7fc7941023a0 2024-05-10T11:42:08.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.795+0000 7fc7937fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7941023a0 0x7fc7941092f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:08.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.795+0000 7fc793fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc794108410 0x7fc794109d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:08.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.795+0000 7fc7937fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7941023a0 0x7fc7941092f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:52566/0 (socket says 172.21.15.100:52566) 2024-05-10T11:42:08.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.795+0000 7fc793fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc794108410 0x7fc794109d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:34490/0 (socket says 172.21.15.100:34490) 2024-05-10T11:42:08.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.795+0000 7fc7937fe700 1 -- 172.21.15.100:0/993536312 learned_addr learned my addr 172.21.15.100:0/993536312 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:42:08.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.796+0000 7fc793fff700 1 -- 172.21.15.100:0/993536312 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7941023a0 msgr2=0x7fc7941092f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:08.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.796+0000 7fc793fff700 1 --2- 172.21.15.100:0/993536312 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7941023a0 0x7fc7941092f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:08.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.796+0000 7fc793fff700 1 -- 172.21.15.100:0/993536312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc794102d60 msgr2=0x7fc794109830 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-10T11:42:08.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.796+0000 7fc792ffd700 1 --2- 172.21.15.100:0/993536312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc794102d60 0x7fc794109830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:08.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.796+0000 7fc793fff700 1 --2- 172.21.15.100:0/993536312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc794102d60 0x7fc794109830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:08.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.796+0000 7fc793fff700 1 -- 172.21.15.100:0/993536312 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7941a8020 con 0x7fc794108410 2024-05-10T11:42:08.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.796+0000 7fc793fff700 1 --2- 172.21.15.100:0/993536312 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc794108410 0x7fc794109d70 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc78c00efc0 tx=0x7fc78c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:42:08.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.796+0000 7fc790ff9700 1 -- 172.21.15.100:0/993536312 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78c0092e0 con 0x7fc794108410 2024-05-10T11:42:08.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.796+0000 7fc790ff9700 1 -- 172.21.15.100:0/993536312 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc78c004bb0 con 0x7fc794108410 2024-05-10T11:42:08.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.796+0000 7fc799b17700 1 -- 172.21.15.100:0/993536312 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7941a82b0 con 0x7fc794108410 2024-05-10T11:42:08.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.796+0000 7fc799b17700 1 -- 172.21.15.100:0/993536312 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7941a87a0 con 0x7fc794108410 2024-05-10T11:42:08.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.796+0000 7fc790ff9700 1 -- 172.21.15.100:0/993536312 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78c004020 con 0x7fc794108410 2024-05-10T11:42:08.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.799+0000 7fc790ff9700 1 -- 172.21.15.100:0/993536312 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fc78c01b030 con 0x7fc794108410 2024-05-10T11:42:08.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.801+0000 7fc790ff9700 1 --2- 172.21.15.100:0/993536312 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc77c0b2900 0x7fc77c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:08.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.801+0000 7fc7a43aab80 1 Processor -- start 2024-05-10T11:42:08.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.801+0000 7fc790ff9700 1 -- 172.21.15.100:0/993536312 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fc78c0e20d0 con 0x7fc794108410 2024-05-10T11:42:08.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.801+0000 7fc7937fe700 1 --2- 172.21.15.100:0/993536312 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc77c0b2900 0x7fc77c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:08.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.801+0000 7fc7a43aab80 1 -- start start 2024-05-10T11:42:08.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.802+0000 7fc7a43aab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d6428821b0 0x55d642882590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:08.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.802+0000 7fc7a43aab80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d642882ad0 0x55d64287c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:08.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.802+0000 7fc7a43aab80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d64287cd50 0x55d64287d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:08.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.802+0000 7fc7a43aab80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55d6427ce940 con 0x55d642882ad0 2024-05-10T11:42:08.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.802+0000 7fc7937fe700 1 --2- 172.21.15.100:0/993536312 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc77c0b2900 0x7fc77c0b4dc0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fc784006010 tx=0x7fc78400b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:42:08.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.802+0000 7fc7a43aab80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d642733da0 con 0x55d64287cd50 2024-05-10T11:42:08.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.802+0000 7fc7a43aab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d6427ca900 con 0x55d6428821b0 2024-05-10T11:42:08.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.802+0000 7fc7937fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d6428821b0 0x55d642882590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:08.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.802+0000 7fc792ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d642882ad0 0x55d64287c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:08.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.802+0000 7fc7937fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d6428821b0 0x55d642882590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:52572/0 (socket says 172.21.15.100:52572) 2024-05-10T11:42:08.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.802+0000 7fc7937fe700 1 -- 172.21.15.100:0/912935541 learned_addr learned my addr 172.21.15.100:0/912935541 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:42:08.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.802+0000 7fc793fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d64287cd50 0x55d64287d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:08.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.802+0000 7fc793fff700 1 -- 172.21.15.100:0/912935541 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d6428821b0 msgr2=0x55d642882590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:08.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.802+0000 7fc793fff700 1 --2- 172.21.15.100:0/912935541 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d6428821b0 0x55d642882590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:08.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.802+0000 7fc793fff700 1 -- 172.21.15.100:0/912935541 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d642882ad0 msgr2=0x55d64287c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:08.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.803+0000 7fc793fff700 1 --2- 172.21.15.100:0/912935541 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d642882ad0 0x55d64287c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:08.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.803+0000 7fc793fff700 1 -- 172.21.15.100:0/912935541 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d64271b8d0 con 0x55d64287cd50 2024-05-10T11:42:08.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.803+0000 7fc792ffd700 1 --2- 172.21.15.100:0/912935541 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d642882ad0 0x55d64287c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:42:08.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.803+0000 7fc793fff700 1 --2- 172.21.15.100:0/912935541 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d64287cd50 0x55d64287d190 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc78c0df810 tx=0x7fc78c0a8c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e1abc8a0236d713 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:42:08.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.803+0000 7fc779ffb700 1 -- 172.21.15.100:0/912935541 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78c0a93c0 con 0x55d64287cd50 2024-05-10T11:42:08.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.803+0000 7fc779ffb700 1 -- 172.21.15.100:0/912935541 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc78c0ad770 con 0x55d64287cd50 2024-05-10T11:42:08.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.803+0000 7fc779ffb700 1 -- 172.21.15.100:0/912935541 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78c0b1f60 con 0x55d64287cd50 2024-05-10T11:42:08.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.804+0000 7fc7a43aab80 1 -- 172.21.15.100:0/912935541 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d64287cd50 msgr2=0x55d64287d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:08.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.804+0000 7fc7a43aab80 1 --2- 172.21.15.100:0/912935541 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d64287cd50 0x55d64287d190 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc78c0df810 tx=0x7fc78c0a8c00 comp rx=0 tx=0).stop 2024-05-10T11:42:08.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.804+0000 7fc7a43aab80 1 -- 172.21.15.100:0/912935541 shutdown_connections 2024-05-10T11:42:08.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.804+0000 7fc7a43aab80 1 --2- 172.21.15.100:0/912935541 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d64287cd50 0x55d64287d190 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:08.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.804+0000 7fc7a43aab80 1 --2- 172.21.15.100:0/912935541 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d6428821b0 0x55d642882590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:08.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.804+0000 7fc7a43aab80 1 --2- 172.21.15.100:0/912935541 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d642882ad0 0x55d64287c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:08.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.804+0000 7fc7a43aab80 1 -- 172.21.15.100:0/912935541 >> 172.21.15.100:0/912935541 conn(0x55d6427a8b70 msgr2=0x55d6428772a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:42:08.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.804+0000 7fc7a43aab80 1 -- 172.21.15.100:0/912935541 shutdown_connections 2024-05-10T11:42:08.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.804+0000 7fc7a43aab80 1 -- 172.21.15.100:0/912935541 wait complete. 2024-05-10T11:42:08.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.805+0000 7fc7a43aab80 1 -- start start 2024-05-10T11:42:08.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.805+0000 7fc7a43aab80 1 Processor -- start 2024-05-10T11:42:08.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.806+0000 7fc7a43aab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d64287cd50 0x55d6428887d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:08.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.806+0000 7fc7a43aab80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d6428821b0 0x55d642885b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:08.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.806+0000 7fc7a43aab80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d642882ad0 0x55d6428860b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:08.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.806+0000 7fc7a43aab80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55d642889450 con 0x55d642882ad0 2024-05-10T11:42:08.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.806+0000 7fc7a43aab80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d642787ed0 con 0x55d6428821b0 2024-05-10T11:42:08.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.806+0000 7fc7937fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d64287cd50 0x55d6428887d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:08.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.806+0000 7fc7a43aab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d6428895d0 con 0x55d64287cd50 2024-05-10T11:42:08.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.806+0000 7fc7937fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d64287cd50 0x55d6428887d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:52584/0 (socket says 172.21.15.100:52584) 2024-05-10T11:42:08.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.806+0000 7fc792ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d6428821b0 0x55d642885b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:08.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.806+0000 7fc7937fe700 1 -- 172.21.15.100:0/191966408 learned_addr learned my addr 172.21.15.100:0/191966408 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:42:08.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.806+0000 7fc793fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d642882ad0 0x55d6428860b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:08.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.806+0000 7fc7a43aab80 10 client.0 _ll_register_callbacks cb 0x55d6427625b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:42:08.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.806+0000 7fc7937fe700 1 -- 172.21.15.100:0/191966408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d6428821b0 msgr2=0x55d642885b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:08.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.806+0000 7fc7937fe700 1 --2- 172.21.15.100:0/191966408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d6428821b0 0x55d642885b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:08.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.806+0000 7fc7937fe700 1 -- 172.21.15.100:0/191966408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d642882ad0 msgr2=0x55d6428860b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:08.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.806+0000 7fc7937fe700 1 --2- 172.21.15.100:0/191966408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d642882ad0 0x55d6428860b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:08.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.806+0000 7fc7937fe700 1 -- 172.21.15.100:0/191966408 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d642785760 con 0x55d64287cd50 2024-05-10T11:42:08.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.806+0000 7fc793fff700 1 --2- 172.21.15.100:0/191966408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d642882ad0 0x55d6428860b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:42:08.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.807+0000 7fc7937fe700 1 --2- 172.21.15.100:0/191966408 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d64287cd50 0x55d6428887d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc784003db0 tx=0x7fc784009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebeed10fe80762c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:42:08.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.807+0000 7fc772ffd700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:42:08.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.807+0000 7fc772ffd700 1 -- 172.21.15.100:0/191966408 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78400bcd0 con 0x55d64287cd50 2024-05-10T11:42:08.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.807+0000 7fc7a43aab80 1 -- 172.21.15.100:0/191966408 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc78c0ad770 con 0x55d64287cd50 2024-05-10T11:42:08.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.807+0000 7fc772ffd700 1 -- 172.21.15.100:0/191966408 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fc78400f460 con 0x55d64287cd50 2024-05-10T11:42:08.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.807+0000 7fc772ffd700 1 -- 172.21.15.100:0/191966408 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fc78400be70 con 0x55d64287cd50 2024-05-10T11:42:08.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.807+0000 7fc7a43aab80 10 client.29362 fetch_fsmap learned FSMap version 41 2024-05-10T11:42:08.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.807+0000 7fc7a43aab80 1 -- 172.21.15.100:0/191966408 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55d642893dd0 con 0x55d64287cd50 2024-05-10T11:42:08.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.807+0000 7fc772ffd700 1 -- 172.21.15.100:0/191966408 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc78c0ad770 con 0x55d64287cd50 2024-05-10T11:42:08.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.808+0000 7fc772ffd700 1 -- 172.21.15.100:0/191966408 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fc78402f020 con 0x55d64287cd50 2024-05-10T11:42:08.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.808+0000 7fc7a43aab80 10 client.29362 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:42:08.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.808+0000 7fc7a43aab80 10 client.29362 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:42:08.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.808+0000 7fc7a43aab80 20 client.29362 populate_metadata read hostname 'smithi100' 2024-05-10T11:42:08.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.808+0000 7fc7a43aab80 1 --2- 172.21.15.100:0/191966408 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d642896360 0x55d642898750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:08.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.808+0000 7fc7a43aab80 4 client.29362 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:42:08.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.808+0000 7fc7a43aab80 1 -- 172.21.15.100:0/191966408 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d6427dc320 con 0x55d642896360 2024-05-10T11:42:08.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.808+0000 7fc792ffd700 1 --2- 172.21.15.100:0/191966408 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d642896360 0x55d642898750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:08.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.808+0000 7fc792ffd700 1 --2- 172.21.15.100:0/191966408 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d642896360 0x55d642898750 crc :-1 s=READY pgs=445 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b04bb1fab1f763ef server_cookie=4f93c6451f2df72b in_seq=0 out_seq=0 2024-05-10T11:42:08.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.808+0000 7fc772ffd700 10 client.29362 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:42:08.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.809+0000 7fc772ffd700 1 -- 172.21.15.100:0/191966408 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d6427dc320 con 0x55d642896360 2024-05-10T11:42:08.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.809+0000 7fc772ffd700 10 client.29362 handle_command_reply: tid=0 2024-05-10T11:42:08.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.809+0000 7fc7a43aab80 1 client.29362 shutdown 2024-05-10T11:42:08.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.809+0000 7fc7a43aab80 1 -- 172.21.15.100:0/191966408 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d64287cd50 msgr2=0x55d6428887d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:08.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.809+0000 7fc7a43aab80 1 --2- 172.21.15.100:0/191966408 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d64287cd50 0x55d6428887d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc784003db0 tx=0x7fc784009310 comp rx=0 tx=0).stop 2024-05-10T11:42:08.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.809+0000 7fc7a43aab80 1 -- 172.21.15.100:0/191966408 shutdown_connections 2024-05-10T11:42:08.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.809+0000 7fc7a43aab80 1 --2- 172.21.15.100:0/191966408 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d642896360 0x55d642898750 crc :-1 s=READY pgs=445 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:08.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.809+0000 7fc7a43aab80 1 --2- 172.21.15.100:0/191966408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d642882ad0 0x55d6428860b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:08.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.809+0000 7fc7a43aab80 1 --2- 172.21.15.100:0/191966408 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d64287cd50 0x55d6428887d0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:08.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.809+0000 7fc7a43aab80 1 --2- 172.21.15.100:0/191966408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d6428821b0 0x55d642885b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:08.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.809+0000 7fc7a43aab80 1 -- 172.21.15.100:0/191966408 >> 172.21.15.100:0/191966408 conn(0x55d6427a8b70 msgr2=0x55d642871b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:42:08.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.810+0000 7fc772ffd700 0 client.29362 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:42:08.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.810+0000 7fc7a43aab80 1 -- 172.21.15.100:0/191966408 shutdown_connections 2024-05-10T11:42:08.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.810+0000 7fc7a43aab80 1 -- 172.21.15.100:0/191966408 wait complete. 2024-05-10T11:42:08.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.810+0000 7fc7a43aab80 20 client.29362 trim_cache size 0 max 16384 2024-05-10T11:42:08.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.844+0000 7fc7a43aab80 1 Processor -- start 2024-05-10T11:42:08.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.844+0000 7fc7a43aab80 1 -- start start 2024-05-10T11:42:08.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.845+0000 7fc7a43aab80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d64287cd50 0x55d642888010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:08.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.845+0000 7fc7a43aab80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6428821b0 0x55d642883770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:08.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.845+0000 7fc7a43aab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d642882ad0 0x55d6428924e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:08.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.845+0000 7fc7a43aab80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc7941b3c10 con 0x55d6428821b0 2024-05-10T11:42:08.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.845+0000 7fc7a43aab80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc7941b14a0 con 0x55d64287cd50 2024-05-10T11:42:08.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.845+0000 7fc7937fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d64287cd50 0x55d642888010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:08.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.845+0000 7fc7a43aab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc7941af2f0 con 0x55d642882ad0 2024-05-10T11:42:08.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.845+0000 7fc7937fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d64287cd50 0x55d642888010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:34528/0 (socket says 172.21.15.100:34528) 2024-05-10T11:42:08.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.845+0000 7fc7937fe700 1 -- 172.21.15.100:0/434352526 learned_addr learned my addr 172.21.15.100:0/434352526 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:42:08.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.845+0000 7fc793fff700 1 --2- 172.21.15.100:0/434352526 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d642882ad0 0x55d6428924e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:08.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.845+0000 7fc792ffd700 1 --2- 172.21.15.100:0/434352526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6428821b0 0x55d642883770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:08.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.846+0000 7fc793fff700 1 -- 172.21.15.100:0/434352526 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d64287cd50 msgr2=0x55d642888010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:08.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.846+0000 7fc793fff700 1 --2- 172.21.15.100:0/434352526 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d64287cd50 0x55d642888010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:08.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.846+0000 7fc793fff700 1 -- 172.21.15.100:0/434352526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6428821b0 msgr2=0x55d642883770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:08.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.846+0000 7fc793fff700 1 --2- 172.21.15.100:0/434352526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6428821b0 0x55d642883770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:08.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.846+0000 7fc793fff700 1 -- 172.21.15.100:0/434352526 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6428827e0 con 0x55d642882ad0 2024-05-10T11:42:08.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.846+0000 7fc793fff700 1 --2- 172.21.15.100:0/434352526 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d642882ad0 0x55d6428924e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc78c00ef90 tx=0x7fc78c0a5280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70c5ca5a4fba8995 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:42:08.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.846+0000 7fc7797fa700 1 -- 172.21.15.100:0/434352526 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78c0a93c0 con 0x55d642882ad0 2024-05-10T11:42:08.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.846+0000 7fc7797fa700 1 -- 172.21.15.100:0/434352526 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc78c0df890 con 0x55d642882ad0 2024-05-10T11:42:08.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.846+0000 7fc7797fa700 1 -- 172.21.15.100:0/434352526 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78c0a3ae0 con 0x55d642882ad0 2024-05-10T11:42:08.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.847+0000 7fc7a43aab80 1 -- 172.21.15.100:0/434352526 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d642882ad0 msgr2=0x55d6428924e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:08.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.847+0000 7fc7a43aab80 1 --2- 172.21.15.100:0/434352526 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d642882ad0 0x55d6428924e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc78c00ef90 tx=0x7fc78c0a5280 comp rx=0 tx=0).stop 2024-05-10T11:42:08.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.847+0000 7fc7a43aab80 1 -- 172.21.15.100:0/434352526 shutdown_connections 2024-05-10T11:42:08.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.847+0000 7fc7a43aab80 1 --2- 172.21.15.100:0/434352526 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d642882ad0 0x55d6428924e0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:08.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.847+0000 7fc7a43aab80 1 --2- 172.21.15.100:0/434352526 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d64287cd50 0x55d642888010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:08.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.847+0000 7fc7a43aab80 1 --2- 172.21.15.100:0/434352526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6428821b0 0x55d642883770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:08.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.847+0000 7fc7a43aab80 1 -- 172.21.15.100:0/434352526 >> 172.21.15.100:0/434352526 conn(0x55d6427a8b70 msgr2=0x55d642880cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:42:08.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.847+0000 7fc7a43aab80 1 -- 172.21.15.100:0/434352526 shutdown_connections 2024-05-10T11:42:08.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.847+0000 7fc7a43aab80 1 -- 172.21.15.100:0/434352526 wait complete. 2024-05-10T11:42:08.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.847+0000 7fc7a43aab80 1 -- start start 2024-05-10T11:42:08.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.848+0000 7fc7a43aab80 1 Processor -- start 2024-05-10T11:42:08.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.848+0000 7fc7a43aab80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d64287cd50 0x55d64288ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:08.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.848+0000 7fc7a43aab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d6428821b0 0x55d64289d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:08.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.848+0000 7fc7a43aab80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d642882ad0 0x55d64289daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:08.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.848+0000 7fc7a43aab80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc7941af960 con 0x55d642882ad0 2024-05-10T11:42:08.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.848+0000 7fc7a43aab80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc7941b0dd0 con 0x55d64287cd50 2024-05-10T11:42:08.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.848+0000 7fc7a43aab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc7941b0030 con 0x55d6428821b0 2024-05-10T11:42:08.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.848+0000 7fc7937fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d64287cd50 0x55d64288ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:08.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.848+0000 7fc7a43aab80 10 client.0 _ll_register_callbacks cb 0x55d6427625b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:42:08.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.848+0000 7fc7937fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d64287cd50 0x55d64288ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:34536/0 (socket says 172.21.15.100:34536) 2024-05-10T11:42:08.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.848+0000 7fc7937fe700 1 -- 172.21.15.100:0/2455462837 learned_addr learned my addr 172.21.15.100:0/2455462837 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:42:08.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.848+0000 7fc792ffd700 1 --2- 172.21.15.100:0/2455462837 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d6428821b0 0x55d64289d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:08.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.848+0000 7fc793fff700 1 --2- 172.21.15.100:0/2455462837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d642882ad0 0x55d64289daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:08.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.849+0000 7fc7937fe700 1 -- 172.21.15.100:0/2455462837 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d6428821b0 msgr2=0x55d64289d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:08.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.849+0000 7fc7937fe700 1 --2- 172.21.15.100:0/2455462837 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d6428821b0 0x55d64289d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:08.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.849+0000 7fc7937fe700 1 -- 172.21.15.100:0/2455462837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d642882ad0 msgr2=0x55d64289daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:08.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.849+0000 7fc7937fe700 1 --2- 172.21.15.100:0/2455462837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d642882ad0 0x55d64289daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:08.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.849+0000 7fc7937fe700 1 -- 172.21.15.100:0/2455462837 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d642892a20 con 0x55d64287cd50 2024-05-10T11:42:08.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.849+0000 7fc793fff700 1 --2- 172.21.15.100:0/2455462837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d642882ad0 0x55d64289daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:42:08.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.849+0000 7fc7937fe700 1 --2- 172.21.15.100:0/2455462837 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d64287cd50 0x55d64288ee50 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc784003db0 tx=0x7fc784005700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b12aad978d2d7736 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:42:08.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.849+0000 7fc772ffd700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:42:08.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.849+0000 7fc772ffd700 1 -- 172.21.15.100:0/2455462837 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc784041020 con 0x55d64287cd50 2024-05-10T11:42:08.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.849+0000 7fc772ffd700 1 -- 172.21.15.100:0/2455462837 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fc78400f870 con 0x55d64287cd50 2024-05-10T11:42:08.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.849+0000 7fc7a43aab80 1 -- 172.21.15.100:0/2455462837 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc78c0df890 con 0x55d64287cd50 2024-05-10T11:42:08.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.849+0000 7fc772ffd700 1 -- 172.21.15.100:0/2455462837 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fc78404c050 con 0x55d64287cd50 2024-05-10T11:42:08.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.850+0000 7fc7a43aab80 10 client.29258 fetch_fsmap learned FSMap version 41 2024-05-10T11:42:08.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.850+0000 7fc7a43aab80 1 -- 172.21.15.100:0/2455462837 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55d6429009f0 con 0x55d64287cd50 2024-05-10T11:42:08.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.850+0000 7fc772ffd700 1 -- 172.21.15.100:0/2455462837 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc784031440 con 0x55d64287cd50 2024-05-10T11:42:08.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.851+0000 7fc772ffd700 1 -- 172.21.15.100:0/2455462837 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fc78402f040 con 0x55d64287cd50 2024-05-10T11:42:08.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.851+0000 7fc7a43aab80 10 client.29258 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:42:08.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.851+0000 7fc7a43aab80 10 client.29258 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:42:08.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.851+0000 7fc7a43aab80 20 client.29258 populate_metadata read hostname 'smithi100' 2024-05-10T11:42:08.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.851+0000 7fc7a43aab80 1 --2- 172.21.15.100:0/2455462837 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d642900f60 0x55d642903380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:08.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.851+0000 7fc7a43aab80 4 client.29258 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T11:42:08.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.851+0000 7fc7a43aab80 1 -- 172.21.15.100:0/2455462837 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fc78c0df890 con 0x55d642900f60 2024-05-10T11:42:08.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.851+0000 7fc792ffd700 1 --2- 172.21.15.100:0/2455462837 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d642900f60 0x55d642903380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:08.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.852+0000 7fc792ffd700 1 --2- 172.21.15.100:0/2455462837 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d642900f60 0x55d642903380 crc :-1 s=READY pgs=446 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34d29b7e0a7325e7 server_cookie=965ccb9af3059195 in_seq=0 out_seq=0 2024-05-10T11:42:08.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.852+0000 7fc772ffd700 10 client.29258 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:42:08.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.853+0000 7fc772ffd700 1 -- 172.21.15.100:0/2455462837 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7fc78c0df890 con 0x55d642900f60 2024-05-10T11:42:08.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.853+0000 7fc772ffd700 10 client.29258 handle_command_reply: tid=0 2024-05-10T11:42:08.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.853+0000 7fc7a43aab80 1 client.29258 shutdown 2024-05-10T11:42:08.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.854+0000 7fc7a43aab80 1 -- 172.21.15.100:0/2455462837 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d64287cd50 msgr2=0x55d64288ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:08.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.854+0000 7fc7a43aab80 1 --2- 172.21.15.100:0/2455462837 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d64287cd50 0x55d64288ee50 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc784003db0 tx=0x7fc784005700 comp rx=0 tx=0).stop 2024-05-10T11:42:08.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.854+0000 7fc7a43aab80 1 -- 172.21.15.100:0/2455462837 shutdown_connections 2024-05-10T11:42:08.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.854+0000 7fc7a43aab80 1 --2- 172.21.15.100:0/2455462837 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55d642900f60 0x55d642903380 crc :-1 s=READY pgs=446 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:08.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.854+0000 7fc7a43aab80 1 --2- 172.21.15.100:0/2455462837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d642882ad0 0x55d64289daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:08.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.854+0000 7fc772ffd700 0 client.29258 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:42:08.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.854+0000 7fc7a43aab80 1 --2- 172.21.15.100:0/2455462837 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d64287cd50 0x55d64288ee50 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:08.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.854+0000 7fc7a43aab80 1 --2- 172.21.15.100:0/2455462837 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d6428821b0 0x55d64289d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:08.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.854+0000 7fc7a43aab80 1 -- 172.21.15.100:0/2455462837 >> 172.21.15.100:0/2455462837 conn(0x55d6427a8b70 msgr2=0x55d6428835f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:42:08.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.855+0000 7fc7a43aab80 1 -- 172.21.15.100:0/2455462837 shutdown_connections 2024-05-10T11:42:08.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.855+0000 7fc7a43aab80 1 -- 172.21.15.100:0/2455462837 wait complete. 2024-05-10T11:42:08.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.855+0000 7fc7a43aab80 20 client.29258 trim_cache size 0 max 16384 2024-05-10T11:42:08.857 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:42:08.857 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T11:42:08.857 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-05-10T11:42:08.858 INFO:teuthology.orchestra.run.smithi100.stdout: "db294da5-100f-42a6-bc9a-4905ccb97ab6": { 2024-05-10T11:42:08.858 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-05-10T11:42:08.858 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "db294da5-100f-42a6-bc9a-4905ccb97ab6", 2024-05-10T11:42:08.858 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-05-10T11:42:08.858 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:42:08.858 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:42:08.858 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:42:08.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.856+0000 7fc799b17700 1 -- 172.21.15.100:0/993536312 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc77c0b2900 msgr2=0x7fc77c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:08.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.856+0000 7fc799b17700 1 --2- 172.21.15.100:0/993536312 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc77c0b2900 0x7fc77c0b4dc0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fc784006010 tx=0x7fc78400b260 comp rx=0 tx=0).stop 2024-05-10T11:42:08.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.856+0000 7fc799b17700 1 -- 172.21.15.100:0/993536312 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc794108410 msgr2=0x7fc794109d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:08.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.856+0000 7fc799b17700 1 --2- 172.21.15.100:0/993536312 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc794108410 0x7fc794109d70 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc78c00efc0 tx=0x7fc78c00c3a0 comp rx=0 tx=0).stop 2024-05-10T11:42:08.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.856+0000 7fc799b17700 1 -- 172.21.15.100:0/993536312 shutdown_connections 2024-05-10T11:42:08.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.856+0000 7fc799b17700 1 --2- 172.21.15.100:0/993536312 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc77c0b2900 0x7fc77c0b4dc0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:08.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.856+0000 7fc799b17700 1 --2- 172.21.15.100:0/993536312 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc794108410 0x7fc794109d70 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:08.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.856+0000 7fc799b17700 1 --2- 172.21.15.100:0/993536312 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7941023a0 0x7fc7941092f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:08.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.856+0000 7fc799b17700 1 --2- 172.21.15.100:0/993536312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc794102d60 0x7fc794109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:08.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.856+0000 7fc799b17700 1 -- 172.21.15.100:0/993536312 >> 172.21.15.100:0/993536312 conn(0x7fc794079350 msgr2=0x7fc7940fffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:42:08.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.856+0000 7fc799b17700 1 -- 172.21.15.100:0/993536312 shutdown_connections 2024-05-10T11:42:08.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:08.857+0000 7fc799b17700 1 -- 172.21.15.100:0/993536312 wait complete. 2024-05-10T11:42:08.958 INFO:tasks.workunit.client.0.smithi100.stdout: 10 99903 8.89 MB/sec execute 482 sec latency 123.452 ms 2024-05-10T11:42:09.616 INFO:tasks.cephfs.filesystem:scrub status for tag:db294da5-100f-42a6-bc9a-4905ccb97ab6 - {'path': '/', 'tag': 'db294da5-100f-42a6-bc9a-4905ccb97ab6', 'options': 'recursive,force'} 2024-05-10T11:42:09.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:42:09 smithi100 ceph-mon[30296]: pgmap v1134: 129 pgs: 129 active+clean; 557 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.3 MiB/s wr, 351 op/s 2024-05-10T11:42:09.958 INFO:tasks.workunit.client.0.smithi100.stdout: 10 100055 8.88 MB/sec execute 483 sec latency 100.604 ms 2024-05-10T11:42:10.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:42:09 smithi178 ceph-mon[33359]: pgmap v1134: 129 pgs: 129 active+clean; 557 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.3 MiB/s wr, 351 op/s 2024-05-10T11:42:10.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:42:09 smithi188 ceph-mon[34075]: pgmap v1134: 129 pgs: 129 active+clean; 557 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.3 MiB/s wr, 351 op/s 2024-05-10T11:42:10.958 INFO:tasks.workunit.client.0.smithi100.stdout: 10 100259 8.89 MB/sec execute 484 sec latency 62.986 ms 2024-05-10T11:42:11.958 INFO:tasks.workunit.client.0.smithi100.stdout: 10 100471 8.90 MB/sec execute 485 sec latency 78.317 ms 2024-05-10T11:42:12.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:42:11 smithi178 ceph-mon[33359]: pgmap v1135: 129 pgs: 129 active+clean; 557 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 4.1 MiB/s wr, 339 op/s 2024-05-10T11:42:12.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:42:11 smithi188 ceph-mon[34075]: pgmap v1135: 129 pgs: 129 active+clean; 557 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 4.1 MiB/s wr, 339 op/s 2024-05-10T11:42:12.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:42:11 smithi100 ceph-mon[30296]: pgmap v1135: 129 pgs: 129 active+clean; 557 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 4.1 MiB/s wr, 339 op/s 2024-05-10T11:42:12.959 INFO:tasks.workunit.client.0.smithi100.stdout: 10 100691 8.90 MB/sec execute 486 sec latency 73.353 ms 2024-05-10T11:42:13.958 INFO:tasks.workunit.client.0.smithi100.stdout: 10 100833 8.89 MB/sec execute 487 sec latency 66.566 ms 2024-05-10T11:42:14.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:42:13 smithi178 ceph-mon[33359]: pgmap v1136: 129 pgs: 129 active+clean; 557 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 3.6 MiB/s wr, 336 op/s 2024-05-10T11:42:14.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:42:13 smithi188 ceph-mon[34075]: pgmap v1136: 129 pgs: 129 active+clean; 557 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 3.6 MiB/s wr, 336 op/s 2024-05-10T11:42:14.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:42:13 smithi100 ceph-mon[30296]: pgmap v1136: 129 pgs: 129 active+clean; 557 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 3.6 MiB/s wr, 336 op/s 2024-05-10T11:42:14.958 INFO:tasks.workunit.client.0.smithi100.stdout: 10 100984 8.89 MB/sec execute 488 sec latency 73.252 ms 2024-05-10T11:42:15.059 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.059+0000 7f9995462640 1 -- 172.21.15.100:0/88761603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f999010c5e0 msgr2=0x7f999010ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:15.059 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.059+0000 7f9995462640 1 --2- 172.21.15.100:0/88761603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f999010c5e0 0x7f999010ea30 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f998000b9f0 tx=0x7f998002c0c0 comp rx=0 tx=0).stop 2024-05-10T11:42:15.059 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.059+0000 7f9995462640 1 -- 172.21.15.100:0/88761603 shutdown_connections 2024-05-10T11:42:15.059 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.059+0000 7f9995462640 1 --2- 172.21.15.100:0/88761603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f999010f0c0 0x7f9990111580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:15.059 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.059+0000 7f9995462640 1 --2- 172.21.15.100:0/88761603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f999010c5e0 0x7f999010ea30 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:15.059 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.059+0000 7f9995462640 1 --2- 172.21.15.100:0/88761603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9990109c20 0x7f999010c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:15.059 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.059+0000 7f9995462640 1 -- 172.21.15.100:0/88761603 >> 172.21.15.100:0/88761603 conn(0x7f999006fbe0 msgr2=0x7f9990072020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:42:15.060 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.059+0000 7f9995462640 1 -- 172.21.15.100:0/88761603 shutdown_connections 2024-05-10T11:42:15.060 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.059+0000 7f9995462640 1 -- 172.21.15.100:0/88761603 wait complete. 2024-05-10T11:42:15.061 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.059+0000 7f9995462640 1 Processor -- start 2024-05-10T11:42:15.062 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.060+0000 7f9995462640 1 -- start start 2024-05-10T11:42:15.062 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.060+0000 7f9995462640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9990109c20 0x7f99901a3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:15.062 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.060+0000 7f9995462640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f999010c5e0 0x7f99901a3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:15.062 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.060+0000 7f9995462640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f999010f0c0 0x7f99901aab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:15.062 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.060+0000 7f9995462640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9990118e70 con 0x7f9990109c20 2024-05-10T11:42:15.062 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.060+0000 7f9995462640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9990118cf0 con 0x7f999010c5e0 2024-05-10T11:42:15.062 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.060+0000 7f9995462640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9990118ff0 con 0x7f999010f0c0 2024-05-10T11:42:15.062 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.060+0000 7f998f7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f999010f0c0 0x7f99901aab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:15.062 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.060+0000 7f998effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9990109c20 0x7f99901a3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:15.062 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.060+0000 7f998f7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f999010f0c0 0x7f99901aab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:52624/0 (socket says 172.21.15.100:52624) 2024-05-10T11:42:15.062 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.060+0000 7f998f7fe640 1 -- 172.21.15.100:0/1884969471 learned_addr learned my addr 172.21.15.100:0/1884969471 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:42:15.062 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.060+0000 7f998e7fc640 1 --2- 172.21.15.100:0/1884969471 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f999010c5e0 0x7f99901a3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:15.063 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.061+0000 7f998f7fe640 1 -- 172.21.15.100:0/1884969471 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f999010c5e0 msgr2=0x7f99901a3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:15.063 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.061+0000 7f998f7fe640 1 --2- 172.21.15.100:0/1884969471 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f999010c5e0 0x7f99901a3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:15.063 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.061+0000 7f998f7fe640 1 -- 172.21.15.100:0/1884969471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9990109c20 msgr2=0x7f99901a3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:15.063 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.061+0000 7f998f7fe640 1 --2- 172.21.15.100:0/1884969471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9990109c20 0x7f99901a3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:15.063 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.061+0000 7f998f7fe640 1 -- 172.21.15.100:0/1884969471 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99901ab1a0 con 0x7f999010f0c0 2024-05-10T11:42:15.063 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.061+0000 7f998e7fc640 1 --2- 172.21.15.100:0/1884969471 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f999010c5e0 0x7f99901a3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:42:15.063 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.061+0000 7f998effd640 1 --2- 172.21.15.100:0/1884969471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9990109c20 0x7f99901a3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:42:15.063 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.061+0000 7f998f7fe640 1 --2- 172.21.15.100:0/1884969471 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f999010f0c0 0x7f99901aab30 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f99840087d0 tx=0x7f9984008c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:42:15.063 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.061+0000 7f9977fff640 1 -- 172.21.15.100:0/1884969471 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f998400eb20 con 0x7f999010f0c0 2024-05-10T11:42:15.063 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.061+0000 7f9995462640 1 -- 172.21.15.100:0/1884969471 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99901ab490 con 0x7f999010f0c0 2024-05-10T11:42:15.063 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.061+0000 7f9977fff640 1 -- 172.21.15.100:0/1884969471 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f998401a070 con 0x7f999010f0c0 2024-05-10T11:42:15.063 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.061+0000 7f9977fff640 1 -- 172.21.15.100:0/1884969471 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99840133b0 con 0x7f999010f0c0 2024-05-10T11:42:15.063 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.061+0000 7f9995462640 1 -- 172.21.15.100:0/1884969471 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99901aba30 con 0x7f999010f0c0 2024-05-10T11:42:15.064 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.064+0000 7f9998177740 1 Processor -- start 2024-05-10T11:42:15.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.064+0000 7f9977fff640 1 -- 172.21.15.100:0/1884969471 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f9984014070 con 0x7f999010f0c0 2024-05-10T11:42:15.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.065+0000 7f9998177740 1 -- start start 2024-05-10T11:42:15.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.065+0000 7f9998177740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56033e5afdb0 0x56033e5b01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:15.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.065+0000 7f9998177740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56033e5b07a0 0x56033e5b52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:15.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.065+0000 7f9998177740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56033e5b5910 0x56033e5b7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:15.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.065+0000 7f9998177740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56033e5145d0 con 0x56033e5afdb0 2024-05-10T11:42:15.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.065+0000 7f9998177740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56033e4cd890 con 0x56033e5b07a0 2024-05-10T11:42:15.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.065+0000 7f9998177740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56033e510590 con 0x56033e5b5910 2024-05-10T11:42:15.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.065+0000 7f998effd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56033e5b07a0 0x56033e5b52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:15.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.065+0000 7f998e7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56033e5afdb0 0x56033e5b01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:15.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.065+0000 7f998effd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56033e5b07a0 0x56033e5b52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:34564/0 (socket says 172.21.15.100:34564) 2024-05-10T11:42:15.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.065+0000 7f998effd640 1 -- 172.21.15.100:0/4130539803 learned_addr learned my addr 172.21.15.100:0/4130539803 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:42:15.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.065+0000 7f998e7fc640 1 --2- 172.21.15.100:0/4130539803 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56033e5b5910 0x56033e5b7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:15.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.065+0000 7f998effd640 1 -- 172.21.15.100:0/4130539803 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56033e5b5910 msgr2=0x56033e5b7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:15.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.065+0000 7f998effd640 1 --2- 172.21.15.100:0/4130539803 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56033e5b5910 0x56033e5b7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:15.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.065+0000 7f998effd640 1 -- 172.21.15.100:0/4130539803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56033e5afdb0 msgr2=0x56033e5b01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:15.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.065+0000 7f998effd640 1 --2- 172.21.15.100:0/4130539803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56033e5afdb0 0x56033e5b01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:15.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.066+0000 7f998effd640 1 -- 172.21.15.100:0/4130539803 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56033e226210 con 0x56033e5b07a0 2024-05-10T11:42:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.066+0000 7f998e7fc640 1 --2- 172.21.15.100:0/4130539803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56033e5afdb0 0x56033e5b01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:42:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.066+0000 7f9977fff640 1 --2- 172.21.15.100:0/1884969471 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f99780b28a0 0x7f99780b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.066+0000 7f998effd640 1 --2- 172.21.15.100:0/4130539803 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56033e5b07a0 0x56033e5b52a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f998800c450 tx=0x7f998800c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e99e6601960002b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:42:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.066+0000 7f998e7fc640 1 --2- 172.21.15.100:0/1884969471 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f99780b28a0 0x7f99780b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.066+0000 7f9977fff640 1 -- 172.21.15.100:0/1884969471 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f99840e1bf0 con 0x7f999010f0c0 2024-05-10T11:42:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.066+0000 7f99757fa640 1 -- 172.21.15.100:0/4130539803 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f998801a070 con 0x56033e5b07a0 2024-05-10T11:42:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.066+0000 7f99757fa640 1 -- 172.21.15.100:0/4130539803 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f99880078c0 con 0x56033e5b07a0 2024-05-10T11:42:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.066+0000 7f99757fa640 1 -- 172.21.15.100:0/4130539803 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9988015750 con 0x56033e5b07a0 2024-05-10T11:42:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.067+0000 7f9998177740 1 -- 172.21.15.100:0/4130539803 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56033e5b07a0 msgr2=0x56033e5b52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.067+0000 7f998e7fc640 1 --2- 172.21.15.100:0/1884969471 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f99780b28a0 0x7f99780b4d60 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f9980000c00 tx=0x7f9980005af0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:42:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.067+0000 7f9998177740 1 --2- 172.21.15.100:0/4130539803 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56033e5b07a0 0x56033e5b52a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f998800c450 tx=0x7f998800c910 comp rx=0 tx=0).stop 2024-05-10T11:42:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.067+0000 7f9998177740 1 -- 172.21.15.100:0/4130539803 shutdown_connections 2024-05-10T11:42:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.067+0000 7f9998177740 1 --2- 172.21.15.100:0/4130539803 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56033e5b5910 0x56033e5b7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.067+0000 7f9998177740 1 --2- 172.21.15.100:0/4130539803 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56033e5b07a0 0x56033e5b52a0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.067+0000 7f9998177740 1 --2- 172.21.15.100:0/4130539803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56033e5afdb0 0x56033e5b01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.067+0000 7f9998177740 1 -- 172.21.15.100:0/4130539803 >> 172.21.15.100:0/4130539803 conn(0x56033e5ab420 msgr2=0x56033e5ab820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:42:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.067+0000 7f9998177740 1 -- 172.21.15.100:0/4130539803 shutdown_connections 2024-05-10T11:42:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.067+0000 7f9998177740 1 -- 172.21.15.100:0/4130539803 wait complete. 2024-05-10T11:42:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.067+0000 7f9998177740 1 -- start start 2024-05-10T11:42:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.067+0000 7f9998177740 1 Processor -- start 2024-05-10T11:42:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.068+0000 7f9998177740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56033e5afdb0 0x56033e5b9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.068+0000 7f9998177740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56033e5b07a0 0x56033e5ba4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.068+0000 7f9998177740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56033e5b5910 0x56033e5c1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.068+0000 7f9998177740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56033e5b8910 con 0x56033e5b5910 2024-05-10T11:42:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.068+0000 7f9998177740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56033e4d1370 con 0x56033e5b07a0 2024-05-10T11:42:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.068+0000 7f9998177740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56033e5b8a90 con 0x56033e5afdb0 2024-05-10T11:42:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.068+0000 7f9998177740 10 client.0 _ll_register_callbacks cb 0x56033e225a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:42:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.068+0000 7f998effd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56033e5afdb0 0x56033e5b9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.068+0000 7f998f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56033e5b5910 0x56033e5c1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.068+0000 7f998effd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56033e5afdb0 0x56033e5b9f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:52646/0 (socket says 172.21.15.100:52646) 2024-05-10T11:42:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.068+0000 7f998effd640 1 -- 172.21.15.100:0/1423358151 learned_addr learned my addr 172.21.15.100:0/1423358151 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:42:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.068+0000 7f998e7fc640 1 --2- 172.21.15.100:0/1423358151 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56033e5b07a0 0x56033e5ba4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.068+0000 7f998effd640 1 -- 172.21.15.100:0/1423358151 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56033e5b07a0 msgr2=0x56033e5ba4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.068+0000 7f998effd640 1 --2- 172.21.15.100:0/1423358151 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56033e5b07a0 0x56033e5ba4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.068+0000 7f998effd640 1 -- 172.21.15.100:0/1423358151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56033e5b5910 msgr2=0x56033e5c1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.068+0000 7f998effd640 1 --2- 172.21.15.100:0/1423358151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56033e5b5910 0x56033e5c1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.068+0000 7f998effd640 1 -- 172.21.15.100:0/1423358151 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56033e2f2620 con 0x56033e5afdb0 2024-05-10T11:42:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.068+0000 7f998f7fe640 1 --2- 172.21.15.100:0/1423358151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56033e5b5910 0x56033e5c1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:42:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.069+0000 7f998effd640 1 --2- 172.21.15.100:0/1423358151 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56033e5afdb0 0x56033e5b9f90 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f998800c320 tx=0x7f9988007ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d99aee182ea2cc48 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:42:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.069+0000 7f9972ff5640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:42:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.069+0000 7f9972ff5640 1 -- 172.21.15.100:0/1423358151 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9988022020 con 0x56033e5afdb0 2024-05-10T11:42:15.076 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.069+0000 7f9972ff5640 1 -- 172.21.15.100:0/1423358151 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f99880154f0 con 0x56033e5afdb0 2024-05-10T11:42:15.076 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.069+0000 7f9998177740 1 -- 172.21.15.100:0/1423358151 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99880078c0 con 0x56033e5afdb0 2024-05-10T11:42:15.076 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.069+0000 7f9972ff5640 1 -- 172.21.15.100:0/1423358151 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f998802d050 con 0x56033e5afdb0 2024-05-10T11:42:15.076 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.069+0000 7f9998177740 10 client.29380 fetch_fsmap learned FSMap version 41 2024-05-10T11:42:15.076 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.069+0000 7f9998177740 1 -- 172.21.15.100:0/1423358151 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x56033e5c4540 con 0x56033e5afdb0 2024-05-10T11:42:15.076 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.069+0000 7f9972ff5640 1 -- 172.21.15.100:0/1423358151 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f99880078c0 con 0x56033e5afdb0 2024-05-10T11:42:15.076 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.070+0000 7f9972ff5640 1 -- 172.21.15.100:0/1423358151 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f998801d030 con 0x56033e5afdb0 2024-05-10T11:42:15.076 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.070+0000 7f9998177740 10 client.29380 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:42:15.076 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.070+0000 7f9998177740 10 client.29380 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:42:15.076 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.070+0000 7f9998177740 20 client.29380 populate_metadata read hostname 'smithi100' 2024-05-10T11:42:15.076 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.070+0000 7f9998177740 1 --2- 172.21.15.100:0/1423358151 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56033e5c59a0 0x56033e5c7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:15.076 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.070+0000 7f9998177740 4 client.29380 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:42:15.076 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.070+0000 7f9998177740 1 -- 172.21.15.100:0/1423358151 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56033e5221e0 con 0x56033e5c59a0 2024-05-10T11:42:15.077 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.070+0000 7f998e7fc640 1 --2- 172.21.15.100:0/1423358151 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56033e5c59a0 0x56033e5c7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:15.077 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.070+0000 7f9972ff5640 10 client.29380 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:42:15.077 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.070+0000 7f998e7fc640 1 --2- 172.21.15.100:0/1423358151 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56033e5c59a0 0x56033e5c7d90 crc :-1 s=READY pgs=447 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a81b1773f4d7dd28 server_cookie=3d61f3c75e10506e in_seq=0 out_seq=0 2024-05-10T11:42:15.077 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.071+0000 7f9972ff5640 1 -- 172.21.15.100:0/1423358151 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56033e5221e0 con 0x56033e5c59a0 2024-05-10T11:42:15.077 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.071+0000 7f9972ff5640 10 client.29380 handle_command_reply: tid=0 2024-05-10T11:42:15.077 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.071+0000 7f9998177740 1 client.29380 shutdown 2024-05-10T11:42:15.077 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.071+0000 7f9998177740 1 -- 172.21.15.100:0/1423358151 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56033e5afdb0 msgr2=0x56033e5b9f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:15.077 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.071+0000 7f9998177740 1 --2- 172.21.15.100:0/1423358151 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56033e5afdb0 0x56033e5b9f90 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f998800c320 tx=0x7f9988007ca0 comp rx=0 tx=0).stop 2024-05-10T11:42:15.077 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.071+0000 7f9998177740 1 -- 172.21.15.100:0/1423358151 shutdown_connections 2024-05-10T11:42:15.077 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.071+0000 7f9998177740 1 --2- 172.21.15.100:0/1423358151 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56033e5c59a0 0x56033e5c7d90 crc :-1 s=READY pgs=447 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:15.077 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.071+0000 7f9998177740 1 --2- 172.21.15.100:0/1423358151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56033e5b5910 0x56033e5c1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:15.078 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.071+0000 7f9998177740 1 --2- 172.21.15.100:0/1423358151 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56033e5b07a0 0x56033e5ba4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:15.078 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.071+0000 7f9972ff5640 0 client.29380 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:42:15.078 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.071+0000 7f9998177740 1 --2- 172.21.15.100:0/1423358151 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56033e5afdb0 0x56033e5b9f90 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:15.078 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.071+0000 7f9998177740 1 -- 172.21.15.100:0/1423358151 >> 172.21.15.100:0/1423358151 conn(0x56033e5ab420 msgr2=0x56033e5aa580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:42:15.078 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.072+0000 7f9998177740 1 -- 172.21.15.100:0/1423358151 shutdown_connections 2024-05-10T11:42:15.078 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.072+0000 7f9998177740 1 -- 172.21.15.100:0/1423358151 wait complete. 2024-05-10T11:42:15.078 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.072+0000 7f9998177740 20 client.29380 trim_cache size 0 max 16384 2024-05-10T11:42:15.095 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.095+0000 7f9998177740 1 Processor -- start 2024-05-10T11:42:15.097 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.095+0000 7f9998177740 1 -- start start 2024-05-10T11:42:15.097 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.096+0000 7f9998177740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56033e5afdb0 0x56033e5ae940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:15.097 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.096+0000 7f9998177740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56033e5b07a0 0x56033e5c14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:15.097 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.096+0000 7f9998177740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56033e5b5910 0x56033e5cc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:15.097 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.096+0000 7f9998177740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f99901b8620 con 0x56033e5b5910 2024-05-10T11:42:15.097 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.096+0000 7f9998177740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f99901b5ef0 con 0x56033e5b07a0 2024-05-10T11:42:15.097 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.096+0000 7f9998177740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f99901b3d40 con 0x56033e5afdb0 2024-05-10T11:42:15.097 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.096+0000 7f998f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56033e5b5910 0x56033e5cc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:15.097 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.096+0000 7f998f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56033e5b5910 0x56033e5cc5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34970/0 (socket says 172.21.15.100:34970) 2024-05-10T11:42:15.097 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.096+0000 7f998f7fe640 1 -- 172.21.15.100:0/1041172783 learned_addr learned my addr 172.21.15.100:0/1041172783 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:42:15.097 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.096+0000 7f998e7fc640 1 --2- 172.21.15.100:0/1041172783 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56033e5b07a0 0x56033e5c14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:15.098 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.096+0000 7f998effd640 1 --2- 172.21.15.100:0/1041172783 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56033e5afdb0 0x56033e5ae940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:15.098 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.096+0000 7f998f7fe640 1 -- 172.21.15.100:0/1041172783 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56033e5afdb0 msgr2=0x56033e5ae940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:15.098 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.096+0000 7f998f7fe640 1 --2- 172.21.15.100:0/1041172783 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56033e5afdb0 0x56033e5ae940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:15.098 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.096+0000 7f998f7fe640 1 -- 172.21.15.100:0/1041172783 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56033e5b07a0 msgr2=0x56033e5c14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:15.098 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.096+0000 7f998f7fe640 1 --2- 172.21.15.100:0/1041172783 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56033e5b07a0 0x56033e5c14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:15.098 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.096+0000 7f998f7fe640 1 -- 172.21.15.100:0/1041172783 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56033e5b9060 con 0x56033e5b5910 2024-05-10T11:42:15.098 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.096+0000 7f998e7fc640 1 --2- 172.21.15.100:0/1041172783 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56033e5b07a0 0x56033e5c14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:42:15.098 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.096+0000 7f998f7fe640 1 --2- 172.21.15.100:0/1041172783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56033e5b5910 0x56033e5cc5d0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f99840df3e0 tx=0x7f99840a8b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e99dca144608678c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:42:15.098 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.097+0000 7f9974ff9640 1 -- 172.21.15.100:0/1041172783 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99840b1d10 con 0x56033e5b5910 2024-05-10T11:42:15.100 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.097+0000 7f9974ff9640 1 -- 172.21.15.100:0/1041172783 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f99840df4a0 con 0x56033e5b5910 2024-05-10T11:42:15.100 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.097+0000 7f9974ff9640 1 -- 172.21.15.100:0/1041172783 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99840a7de0 con 0x56033e5b5910 2024-05-10T11:42:15.100 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.098+0000 7f9998177740 1 -- 172.21.15.100:0/1041172783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56033e5b5910 msgr2=0x56033e5cc5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:15.100 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.098+0000 7f9998177740 1 --2- 172.21.15.100:0/1041172783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56033e5b5910 0x56033e5cc5d0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f99840df3e0 tx=0x7f99840a8b00 comp rx=0 tx=0).stop 2024-05-10T11:42:15.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.098+0000 7f9998177740 1 -- 172.21.15.100:0/1041172783 shutdown_connections 2024-05-10T11:42:15.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.098+0000 7f9998177740 1 --2- 172.21.15.100:0/1041172783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56033e5b5910 0x56033e5cc5d0 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:15.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.098+0000 7f9998177740 1 --2- 172.21.15.100:0/1041172783 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56033e5b07a0 0x56033e5c14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:15.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.098+0000 7f9998177740 1 --2- 172.21.15.100:0/1041172783 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56033e5afdb0 0x56033e5ae940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:15.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.098+0000 7f9998177740 1 -- 172.21.15.100:0/1041172783 >> 172.21.15.100:0/1041172783 conn(0x56033e5ab420 msgr2=0x56033e5aa580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:42:15.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.098+0000 7f9998177740 1 -- 172.21.15.100:0/1041172783 shutdown_connections 2024-05-10T11:42:15.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.098+0000 7f9998177740 1 -- 172.21.15.100:0/1041172783 wait complete. 2024-05-10T11:42:15.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.098+0000 7f9998177740 1 -- start start 2024-05-10T11:42:15.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.098+0000 7f9998177740 1 Processor -- start 2024-05-10T11:42:15.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.098+0000 7f9998177740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56033e5afdb0 0x56033e5be9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:15.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.098+0000 7f9998177740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56033e5b07a0 0x56033e5bef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:15.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.098+0000 7f9998177740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56033e5b5910 0x56033e5bff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:15.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.098+0000 7f9998177740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f99901b43b0 con 0x56033e5b07a0 2024-05-10T11:42:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.098+0000 7f9998177740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f99901b5820 con 0x56033e5b5910 2024-05-10T11:42:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.098+0000 7f9998177740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f99901b4a80 con 0x56033e5afdb0 2024-05-10T11:42:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.098+0000 7f9998177740 10 client.0 _ll_register_callbacks cb 0x7f9988018070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:42:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.098+0000 7f998e7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56033e5b07a0 0x56033e5bef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.098+0000 7f998effd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56033e5afdb0 0x56033e5be9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.099+0000 7f998e7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56033e5b07a0 0x56033e5bef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34976/0 (socket says 172.21.15.100:34976) 2024-05-10T11:42:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.099+0000 7f998e7fc640 1 -- 172.21.15.100:0/1343857651 learned_addr learned my addr 172.21.15.100:0/1343857651 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:42:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.099+0000 7f998f7fe640 1 --2- 172.21.15.100:0/1343857651 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56033e5b5910 0x56033e5bff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.099+0000 7f998e7fc640 1 -- 172.21.15.100:0/1343857651 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56033e5afdb0 msgr2=0x56033e5be9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.099+0000 7f998e7fc640 1 --2- 172.21.15.100:0/1343857651 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56033e5afdb0 0x56033e5be9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.099+0000 7f998e7fc640 1 -- 172.21.15.100:0/1343857651 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56033e5b5910 msgr2=0x56033e5bff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:15.104 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.099+0000 7f998e7fc640 1 --2- 172.21.15.100:0/1343857651 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56033e5b5910 0x56033e5bff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:15.104 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.099+0000 7f998e7fc640 1 -- 172.21.15.100:0/1343857651 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56033e5c0510 con 0x56033e5b07a0 2024-05-10T11:42:15.104 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.099+0000 7f998f7fe640 1 --2- 172.21.15.100:0/1343857651 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56033e5b5910 0x56033e5bff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:42:15.104 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.099+0000 7f998e7fc640 1 --2- 172.21.15.100:0/1343857651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56033e5b07a0 0x56033e5bef10 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f9990074b40 tx=0x7f998002c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8c57374f32c1022 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:42:15.104 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.099+0000 7f9972ff5640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:42:15.104 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.099+0000 7f9972ff5640 1 -- 172.21.15.100:0/1343857651 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9980039040 con 0x56033e5b07a0 2024-05-10T11:42:15.104 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.099+0000 7f9972ff5640 1 -- 172.21.15.100:0/1343857651 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f9980003c40 con 0x56033e5b07a0 2024-05-10T11:42:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.099+0000 7f9998177740 1 -- 172.21.15.100:0/1343857651 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99840df4a0 con 0x56033e5b07a0 2024-05-10T11:42:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.099+0000 7f9972ff5640 1 -- 172.21.15.100:0/1343857651 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f9980067050 con 0x56033e5b07a0 2024-05-10T11:42:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.099+0000 7f9998177740 10 client.20025 fetch_fsmap learned FSMap version 41 2024-05-10T11:42:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.100+0000 7f9998177740 1 -- 172.21.15.100:0/1343857651 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x56033e630190 con 0x56033e5b07a0 2024-05-10T11:42:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.100+0000 7f9972ff5640 1 -- 172.21.15.100:0/1343857651 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f998003a040 con 0x56033e5b07a0 2024-05-10T11:42:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.100+0000 7f9972ff5640 1 -- 172.21.15.100:0/1343857651 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f9980056ce0 con 0x56033e5b07a0 2024-05-10T11:42:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.100+0000 7f9998177740 10 client.20025 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:42:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.100+0000 7f9998177740 10 client.20025 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:42:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.100+0000 7f9998177740 20 client.20025 populate_metadata read hostname 'smithi100' 2024-05-10T11:42:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.100+0000 7f9998177740 1 --2- 172.21.15.100:0/1343857651 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56033e630a70 0x56033e632e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.100+0000 7f9998177740 4 client.20025 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:42:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.100+0000 7f9998177740 1 -- 172.21.15.100:0/1343857651 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f99840df4a0 con 0x56033e630a70 2024-05-10T11:42:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.100+0000 7f998effd640 1 --2- 172.21.15.100:0/1343857651 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56033e630a70 0x56033e632e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.101+0000 7f9972ff5640 10 client.20025 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:42:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.101+0000 7f998effd640 1 --2- 172.21.15.100:0/1343857651 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56033e630a70 0x56033e632e90 crc :-1 s=READY pgs=448 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d73442e5f35a75b8 server_cookie=7a8766b5b4b4988a in_seq=0 out_seq=0 2024-05-10T11:42:15.228 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.227+0000 7f9972ff5640 1 -- 172.21.15.100:0/1343857651 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f99840df4a0 con 0x56033e630a70 2024-05-10T11:42:15.228 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.227+0000 7f9972ff5640 10 client.20025 handle_command_reply: tid=0 2024-05-10T11:42:15.228 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.227+0000 7f9998177740 1 client.20025 shutdown 2024-05-10T11:42:15.228 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.228+0000 7f9998177740 1 -- 172.21.15.100:0/1343857651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56033e5b07a0 msgr2=0x56033e5bef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:15.228 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.228+0000 7f9998177740 1 --2- 172.21.15.100:0/1343857651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56033e5b07a0 0x56033e5bef10 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f9990074b40 tx=0x7f998002c0c0 comp rx=0 tx=0).stop 2024-05-10T11:42:15.228 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.228+0000 7f9998177740 1 -- 172.21.15.100:0/1343857651 shutdown_connections 2024-05-10T11:42:15.228 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.228+0000 7f9998177740 1 --2- 172.21.15.100:0/1343857651 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x56033e630a70 0x56033e632e90 crc :-1 s=READY pgs=448 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:15.228 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.228+0000 7f9998177740 1 --2- 172.21.15.100:0/1343857651 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56033e5b5910 0x56033e5bff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:15.228 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.228+0000 7f9998177740 1 --2- 172.21.15.100:0/1343857651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56033e5b07a0 0x56033e5bef10 secure :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f9990074b40 tx=0x7f998002c0c0 comp rx=0 tx=0).stop 2024-05-10T11:42:15.228 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.228+0000 7f9972ff5640 0 client.20025 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:42:15.229 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.228+0000 7f9998177740 1 --2- 172.21.15.100:0/1343857651 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56033e5afdb0 0x56033e5be9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:15.229 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.228+0000 7f9998177740 1 -- 172.21.15.100:0/1343857651 >> 172.21.15.100:0/1343857651 conn(0x56033e5ab420 msgr2=0x56033e5b9580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:42:15.229 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.228+0000 7f9998177740 1 -- 172.21.15.100:0/1343857651 shutdown_connections 2024-05-10T11:42:15.229 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.228+0000 7f9998177740 1 -- 172.21.15.100:0/1343857651 wait complete. 2024-05-10T11:42:15.229 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.228+0000 7f9998177740 20 client.20025 trim_cache size 0 max 16384 2024-05-10T11:42:15.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:42:15.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:42:15.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:42:15.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:42:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.229+0000 7f9995462640 1 -- 172.21.15.100:0/1884969471 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f99780b28a0 msgr2=0x7f99780b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.229+0000 7f9995462640 1 --2- 172.21.15.100:0/1884969471 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f99780b28a0 0x7f99780b4d60 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f9980000c00 tx=0x7f9980005af0 comp rx=0 tx=0).stop 2024-05-10T11:42:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.229+0000 7f9995462640 1 -- 172.21.15.100:0/1884969471 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f999010f0c0 msgr2=0x7f99901aab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.229+0000 7f9995462640 1 --2- 172.21.15.100:0/1884969471 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f999010f0c0 0x7f99901aab30 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f99840087d0 tx=0x7f9984008c90 comp rx=0 tx=0).stop 2024-05-10T11:42:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.229+0000 7f9995462640 1 -- 172.21.15.100:0/1884969471 shutdown_connections 2024-05-10T11:42:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.229+0000 7f9995462640 1 --2- 172.21.15.100:0/1884969471 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f99780b28a0 0x7f99780b4d60 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.229+0000 7f9995462640 1 --2- 172.21.15.100:0/1884969471 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f999010f0c0 0x7f99901aab30 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.229+0000 7f9995462640 1 --2- 172.21.15.100:0/1884969471 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f999010c5e0 0x7f99901a3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.229+0000 7f9995462640 1 --2- 172.21.15.100:0/1884969471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9990109c20 0x7f99901a3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.229+0000 7f9995462640 1 -- 172.21.15.100:0/1884969471 >> 172.21.15.100:0/1884969471 conn(0x7f999006fbe0 msgr2=0x7f999010a560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:42:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.229+0000 7f9995462640 1 -- 172.21.15.100:0/1884969471 shutdown_connections 2024-05-10T11:42:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:15.230+0000 7f9995462640 1 -- 172.21.15.100:0/1884969471 wait complete. 2024-05-10T11:42:15.958 INFO:tasks.workunit.client.0.smithi100.stdout: 10 101200 8.92 MB/sec execute 489 sec latency 122.939 ms 2024-05-10T11:42:16.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:42:15 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:42:16.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:42:15 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:42:16.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:42:15 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:42:16.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:42:15 smithi178 ceph-mon[33359]: pgmap v1137: 129 pgs: 129 active+clean; 557 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 959 KiB/s rd, 3.7 MiB/s wr, 352 op/s 2024-05-10T11:42:16.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:42:15 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:42:16.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:42:15 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:42:16.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:42:15 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:42:16.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:42:15 smithi188 ceph-mon[34075]: pgmap v1137: 129 pgs: 129 active+clean; 557 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 959 KiB/s rd, 3.7 MiB/s wr, 352 op/s 2024-05-10T11:42:16.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:42:15 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:42:16.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:42:15 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:42:16.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:42:15 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:42:16.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:42:15 smithi100 ceph-mon[30296]: pgmap v1137: 129 pgs: 129 active+clean; 557 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 959 KiB/s rd, 3.7 MiB/s wr, 352 op/s 2024-05-10T11:42:16.958 INFO:tasks.workunit.client.0.smithi100.stdout: 10 101433 8.93 MB/sec execute 490 sec latency 77.225 ms 2024-05-10T11:42:17.958 INFO:tasks.workunit.client.0.smithi100.stdout: 10 101588 8.93 MB/sec execute 491 sec latency 99.227 ms 2024-05-10T11:42:18.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:42:17 smithi178 ceph-mon[33359]: pgmap v1138: 129 pgs: 129 active+clean; 562 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 875 KiB/s rd, 3.7 MiB/s wr, 352 op/s 2024-05-10T11:42:18.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:42:17 smithi188 ceph-mon[34075]: pgmap v1138: 129 pgs: 129 active+clean; 562 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 875 KiB/s rd, 3.7 MiB/s wr, 352 op/s 2024-05-10T11:42:18.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:42:17 smithi100 ceph-mon[30296]: pgmap v1138: 129 pgs: 129 active+clean; 562 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 875 KiB/s rd, 3.7 MiB/s wr, 352 op/s 2024-05-10T11:42:18.958 INFO:tasks.workunit.client.0.smithi100.stdout: 10 101749 8.94 MB/sec execute 492 sec latency 57.428 ms 2024-05-10T11:42:19.958 INFO:tasks.workunit.client.0.smithi100.stdout: 10 101875 8.93 MB/sec execute 493 sec latency 124.549 ms 2024-05-10T11:42:20.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:42:19 smithi178 ceph-mon[33359]: pgmap v1139: 129 pgs: 129 active+clean; 564 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 4.0 MiB/s wr, 305 op/s 2024-05-10T11:42:20.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:42:19 smithi188 ceph-mon[34075]: pgmap v1139: 129 pgs: 129 active+clean; 564 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 4.0 MiB/s wr, 305 op/s 2024-05-10T11:42:20.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:42:19 smithi100 ceph-mon[30296]: pgmap v1139: 129 pgs: 129 active+clean; 564 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 4.0 MiB/s wr, 305 op/s 2024-05-10T11:42:20.959 INFO:tasks.workunit.client.0.smithi100.stdout: 10 102046 8.94 MB/sec execute 494 sec latency 52.636 ms 2024-05-10T11:42:21.959 INFO:tasks.workunit.client.0.smithi100.stdout: 10 102215 8.94 MB/sec execute 495 sec latency 69.885 ms 2024-05-10T11:42:22.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:42:21 smithi178 ceph-mon[33359]: pgmap v1140: 129 pgs: 129 active+clean; 569 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 627 KiB/s rd, 4.4 MiB/s wr, 275 op/s 2024-05-10T11:42:22.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:42:21 smithi188 ceph-mon[34075]: pgmap v1140: 129 pgs: 129 active+clean; 569 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 627 KiB/s rd, 4.4 MiB/s wr, 275 op/s 2024-05-10T11:42:22.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:42:21 smithi100 ceph-mon[30296]: pgmap v1140: 129 pgs: 129 active+clean; 569 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 627 KiB/s rd, 4.4 MiB/s wr, 275 op/s 2024-05-10T11:42:22.958 INFO:tasks.workunit.client.0.smithi100.stdout: 10 102365 8.93 MB/sec execute 496 sec latency 162.261 ms 2024-05-10T11:42:23.959 INFO:tasks.workunit.client.0.smithi100.stdout: 10 102506 8.93 MB/sec execute 497 sec latency 95.288 ms 2024-05-10T11:42:24.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:42:23 smithi178 ceph-mon[33359]: pgmap v1141: 129 pgs: 129 active+clean; 573 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 4.5 MiB/s wr, 256 op/s 2024-05-10T11:42:24.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:42:23 smithi188 ceph-mon[34075]: pgmap v1141: 129 pgs: 129 active+clean; 573 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 4.5 MiB/s wr, 256 op/s 2024-05-10T11:42:24.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:42:23 smithi100 ceph-mon[30296]: pgmap v1141: 129 pgs: 129 active+clean; 573 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 4.5 MiB/s wr, 256 op/s 2024-05-10T11:42:24.958 INFO:tasks.workunit.client.0.smithi100.stdout: 10 102656 8.92 MB/sec execute 498 sec latency 94.501 ms 2024-05-10T11:42:25.959 INFO:tasks.workunit.client.0.smithi100.stdout: 10 102827 8.92 MB/sec execute 499 sec latency 106.089 ms 2024-05-10T11:42:26.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:42:25 smithi178 ceph-mon[33359]: pgmap v1142: 129 pgs: 129 active+clean; 575 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 4.8 MiB/s wr, 261 op/s 2024-05-10T11:42:26.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:42:25 smithi188 ceph-mon[34075]: pgmap v1142: 129 pgs: 129 active+clean; 575 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 4.8 MiB/s wr, 261 op/s 2024-05-10T11:42:26.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:42:25 smithi100 ceph-mon[30296]: pgmap v1142: 129 pgs: 129 active+clean; 575 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 4.8 MiB/s wr, 261 op/s 2024-05-10T11:42:26.959 INFO:tasks.workunit.client.0.smithi100.stdout: 10 102983 8.92 MB/sec execute 500 sec latency 98.158 ms 2024-05-10T11:42:27.959 INFO:tasks.workunit.client.0.smithi100.stdout: 10 103150 8.91 MB/sec execute 501 sec latency 134.014 ms 2024-05-10T11:42:28.006 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:42:27 smithi178 ceph-mon[33359]: pgmap v1143: 129 pgs: 129 active+clean; 573 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 884 KiB/s rd, 5.3 MiB/s wr, 294 op/s 2024-05-10T11:42:28.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:42:27 smithi188 ceph-mon[34075]: pgmap v1143: 129 pgs: 129 active+clean; 573 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 884 KiB/s rd, 5.3 MiB/s wr, 294 op/s 2024-05-10T11:42:28.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:42:27 smithi100 ceph-mon[30296]: pgmap v1143: 129 pgs: 129 active+clean; 573 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 884 KiB/s rd, 5.3 MiB/s wr, 294 op/s 2024-05-10T11:42:28.367 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.366+0000 7fdbc0b46640 1 -- 172.21.15.100:0/3416632988 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdbbc102b00 msgr2=0x7fdbbc102f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:28.367 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.366+0000 7fdbc0b46640 1 --2- 172.21.15.100:0/3416632988 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdbbc102b00 0x7fdbbc102f80 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0009860 tx=0x7fdbb002a340 comp rx=0 tx=0).stop 2024-05-10T11:42:28.367 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.366+0000 7fdbc0b46640 1 -- 172.21.15.100:0/3416632988 shutdown_connections 2024-05-10T11:42:28.367 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.366+0000 7fdbc0b46640 1 --2- 172.21.15.100:0/3416632988 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbbc1034c0 0x7fdbbc109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:28.367 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.366+0000 7fdbc0b46640 1 --2- 172.21.15.100:0/3416632988 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdbbc102b00 0x7fdbbc102f80 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:28.367 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.366+0000 7fdbc0b46640 1 --2- 172.21.15.100:0/3416632988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdbbc101910 0x7fdbbc101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:28.367 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.366+0000 7fdbc0b46640 1 -- 172.21.15.100:0/3416632988 >> 172.21.15.100:0/3416632988 conn(0x7fdbbc0fce80 msgr2=0x7fdbbc0ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:42:28.368 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.366+0000 7fdbc0b46640 1 -- 172.21.15.100:0/3416632988 shutdown_connections 2024-05-10T11:42:28.368 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.366+0000 7fdbc0b46640 1 -- 172.21.15.100:0/3416632988 wait complete. 2024-05-10T11:42:28.370 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.367+0000 7fdbc0b46640 1 Processor -- start 2024-05-10T11:42:28.370 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.367+0000 7fdbc0b46640 1 -- start start 2024-05-10T11:42:28.370 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.367+0000 7fdbc0b46640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdbbc101910 0x7fdbbc198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:28.370 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.367+0000 7fdbc0b46640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbbc102b00 0x7fdbbc199030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:28.370 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.367+0000 7fdbc0b46640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdbbc1034c0 0x7fdbbc1a00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:28.370 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.367+0000 7fdbc0b46640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdbbc10e410 con 0x7fdbbc101910 2024-05-10T11:42:28.370 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.367+0000 7fdbc0b46640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdbbc10e290 con 0x7fdbbc102b00 2024-05-10T11:42:28.370 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.368+0000 7fdbc0b46640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fdbbc10e590 con 0x7fdbbc1034c0 2024-05-10T11:42:28.370 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.368+0000 7fdbba59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdbbc101910 0x7fdbbc198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:28.370 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.368+0000 7fdbba59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdbbc101910 0x7fdbbc198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60452/0 (socket says 172.21.15.100:60452) 2024-05-10T11:42:28.370 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.368+0000 7fdbba59c640 1 -- 172.21.15.100:0/875326159 learned_addr learned my addr 172.21.15.100:0/875326159 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:42:28.371 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.368+0000 7fdbb9d9b640 1 --2- 172.21.15.100:0/875326159 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbbc102b00 0x7fdbbc199030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:28.371 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.368+0000 7fdbbad9d640 1 --2- 172.21.15.100:0/875326159 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdbbc1034c0 0x7fdbbc1a00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:28.371 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.368+0000 7fdbb9d9b640 1 -- 172.21.15.100:0/875326159 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdbbc1034c0 msgr2=0x7fdbbc1a00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:28.371 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.368+0000 7fdbb9d9b640 1 --2- 172.21.15.100:0/875326159 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdbbc1034c0 0x7fdbbc1a00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:28.371 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.368+0000 7fdbb9d9b640 1 -- 172.21.15.100:0/875326159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdbbc101910 msgr2=0x7fdbbc198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:28.371 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.368+0000 7fdbb9d9b640 1 --2- 172.21.15.100:0/875326159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdbbc101910 0x7fdbbc198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:28.371 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.368+0000 7fdbb9d9b640 1 -- 172.21.15.100:0/875326159 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbbc1a0710 con 0x7fdbbc102b00 2024-05-10T11:42:28.371 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.368+0000 7fdbb9d9b640 1 --2- 172.21.15.100:0/875326159 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbbc102b00 0x7fdbbc199030 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fdbb002a860 tx=0x7fdbb0030760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:42:28.371 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.368+0000 7fdbab7fe640 1 -- 172.21.15.100:0/875326159 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb0037040 con 0x7fdbbc102b00 2024-05-10T11:42:28.371 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.369+0000 7fdbc0b46640 1 -- 172.21.15.100:0/875326159 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbbc1a09a0 con 0x7fdbbc102b00 2024-05-10T11:42:28.371 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.369+0000 7fdbc0b46640 1 -- 172.21.15.100:0/875326159 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbbc1a0e80 con 0x7fdbbc102b00 2024-05-10T11:42:28.372 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.369+0000 7fdbab7fe640 1 -- 172.21.15.100:0/875326159 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdbb0032df0 con 0x7fdbbc102b00 2024-05-10T11:42:28.372 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.369+0000 7fdbab7fe640 1 -- 172.21.15.100:0/875326159 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb0004430 con 0x7fdbbc102b00 2024-05-10T11:42:28.372 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.372+0000 7fdbc3867740 1 Processor -- start 2024-05-10T11:42:28.372 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.373+0000 7fdbc3867740 1 -- start start 2024-05-10T11:42:28.374 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.373+0000 7fdbc3867740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e051001c0 0x564e051005e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:28.374 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.373+0000 7fdbc3867740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e05100bb0 0x564e051056b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:28.374 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.373+0000 7fdbc3867740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564e05105d20 0x564e051081b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:28.374 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.373+0000 7fdbc3867740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x564e050605d0 con 0x564e05100bb0 2024-05-10T11:42:28.375 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.373+0000 7fdbc3867740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x564e05019890 con 0x564e051001c0 2024-05-10T11:42:28.375 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.373+0000 7fdbc3867740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x564e0505c590 con 0x564e05105d20 2024-05-10T11:42:28.375 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.373+0000 7fdbbad9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e051001c0 0x564e051005e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:28.375 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.373+0000 7fdbbad9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e051001c0 0x564e051005e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:54698/0 (socket says 172.21.15.100:54698) 2024-05-10T11:42:28.375 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.373+0000 7fdbbad9d640 1 -- 172.21.15.100:0/1513627195 learned_addr learned my addr 172.21.15.100:0/1513627195 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:42:28.375 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.373+0000 7fdbab7fe640 1 -- 172.21.15.100:0/875326159 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fdbb0043030 con 0x7fdbbc102b00 2024-05-10T11:42:28.375 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.373+0000 7fdbba59c640 1 --2- 172.21.15.100:0/1513627195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e05100bb0 0x564e051056b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:28.375 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.373+0000 7fdbb9d9b640 1 --2- 172.21.15.100:0/1513627195 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564e05105d20 0x564e051081b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:28.376 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.373+0000 7fdbbad9d640 1 -- 172.21.15.100:0/1513627195 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564e05105d20 msgr2=0x564e051081b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:28.376 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.373+0000 7fdbbad9d640 1 --2- 172.21.15.100:0/1513627195 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564e05105d20 0x564e051081b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:28.376 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.373+0000 7fdbbad9d640 1 -- 172.21.15.100:0/1513627195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e05100bb0 msgr2=0x564e051056b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:28.376 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.373+0000 7fdbbad9d640 1 --2- 172.21.15.100:0/1513627195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e05100bb0 0x564e051056b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:28.376 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.373+0000 7fdbbad9d640 1 -- 172.21.15.100:0/1513627195 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e04d72210 con 0x564e051001c0 2024-05-10T11:42:28.376 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.374+0000 7fdbbad9d640 1 --2- 172.21.15.100:0/1513627195 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e051001c0 0x564e051005e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fdbb4009df0 tx=0x7fdbb400f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b41e3361851f576 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:42:28.376 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.374+0000 7fdba8ff9640 1 -- 172.21.15.100:0/1513627195 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb4003970 con 0x564e051001c0 2024-05-10T11:42:28.376 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.374+0000 7fdba8ff9640 1 -- 172.21.15.100:0/1513627195 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdbb4004030 con 0x564e051001c0 2024-05-10T11:42:28.377 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.374+0000 7fdba8ff9640 1 -- 172.21.15.100:0/1513627195 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb4011820 con 0x564e051001c0 2024-05-10T11:42:28.380 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.375+0000 7fdbab7fe640 1 --2- 172.21.15.100:0/875326159 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fdba40b2920 0x7fdba40b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:28.380 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.375+0000 7fdbba59c640 1 --2- 172.21.15.100:0/875326159 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fdba40b2920 0x7fdba40b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:28.380 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.375+0000 7fdbab7fe640 1 -- 172.21.15.100:0/875326159 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fdbb01019e0 con 0x7fdbbc102b00 2024-05-10T11:42:28.380 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.375+0000 7fdbc3867740 1 -- 172.21.15.100:0/1513627195 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e051001c0 msgr2=0x564e051005e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:28.380 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.375+0000 7fdbc3867740 1 --2- 172.21.15.100:0/1513627195 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e051001c0 0x564e051005e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fdbb4009df0 tx=0x7fdbb400f730 comp rx=0 tx=0).stop 2024-05-10T11:42:28.380 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.375+0000 7fdbba59c640 1 --2- 172.21.15.100:0/875326159 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fdba40b2920 0x7fdba40b4de0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x564e05100fb0 tx=0x7fdbac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:42:28.380 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.375+0000 7fdbc3867740 1 -- 172.21.15.100:0/1513627195 shutdown_connections 2024-05-10T11:42:28.380 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.375+0000 7fdbc3867740 1 --2- 172.21.15.100:0/1513627195 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564e05105d20 0x564e051081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:28.381 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.375+0000 7fdbc3867740 1 --2- 172.21.15.100:0/1513627195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e05100bb0 0x564e051056b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:28.381 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.375+0000 7fdbc3867740 1 --2- 172.21.15.100:0/1513627195 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e051001c0 0x564e051005e0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:28.381 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.375+0000 7fdbc3867740 1 -- 172.21.15.100:0/1513627195 >> 172.21.15.100:0/1513627195 conn(0x564e050fb830 msgr2=0x564e050fbc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:42:28.381 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.376+0000 7fdbc3867740 1 -- 172.21.15.100:0/1513627195 shutdown_connections 2024-05-10T11:42:28.381 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.376+0000 7fdbc3867740 1 -- 172.21.15.100:0/1513627195 wait complete. 2024-05-10T11:42:28.381 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.376+0000 7fdbc3867740 1 -- start start 2024-05-10T11:42:28.381 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.376+0000 7fdbc3867740 1 Processor -- start 2024-05-10T11:42:28.381 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.376+0000 7fdbc3867740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e051001c0 0x564e0510a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:28.381 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.376+0000 7fdbc3867740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564e05100bb0 0x564e0510a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:28.381 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.376+0000 7fdbc3867740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e05105d20 0x564e051119e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:28.381 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.376+0000 7fdbc3867740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x564e05108cd0 con 0x564e05105d20 2024-05-10T11:42:28.381 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.376+0000 7fdbc3867740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x564e0501d370 con 0x564e051001c0 2024-05-10T11:42:28.381 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.376+0000 7fdbc3867740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x564e05108e50 con 0x564e05100bb0 2024-05-10T11:42:28.381 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.376+0000 7fdbc3867740 10 client.0 _ll_register_callbacks cb 0x564e04d71a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:42:28.381 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.376+0000 7fdbba59c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e051001c0 0x564e0510a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:28.382 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.376+0000 7fdbbad9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e05105d20 0x564e051119e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:28.382 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.376+0000 7fdbba59c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e051001c0 0x564e0510a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:54714/0 (socket says 172.21.15.100:54714) 2024-05-10T11:42:28.382 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.377+0000 7fdbba59c640 1 -- 172.21.15.100:0/3876393027 learned_addr learned my addr 172.21.15.100:0/3876393027 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:42:28.382 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.377+0000 7fdbb9d9b640 1 --2- 172.21.15.100:0/3876393027 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564e05100bb0 0x564e0510a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:28.382 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.377+0000 7fdbbad9d640 1 -- 172.21.15.100:0/3876393027 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564e05100bb0 msgr2=0x564e0510a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:28.382 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.377+0000 7fdbbad9d640 1 --2- 172.21.15.100:0/3876393027 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564e05100bb0 0x564e0510a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:28.382 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.377+0000 7fdbbad9d640 1 -- 172.21.15.100:0/3876393027 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e051001c0 msgr2=0x564e0510a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:28.382 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.377+0000 7fdbbad9d640 1 --2- 172.21.15.100:0/3876393027 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e051001c0 0x564e0510a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:28.382 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.377+0000 7fdbbad9d640 1 -- 172.21.15.100:0/3876393027 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e04e3e620 con 0x564e05105d20 2024-05-10T11:42:28.382 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.377+0000 7fdbbad9d640 1 --2- 172.21.15.100:0/3876393027 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e05105d20 0x564e051119e0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fdbb4000f80 tx=0x7fdbb40043f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc45ede434f57fdd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:42:28.382 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.377+0000 7fdba1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:42:28.382 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.377+0000 7fdbb9d9b640 1 --2- 172.21.15.100:0/3876393027 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564e05100bb0 0x564e0510a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:42:28.382 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.377+0000 7fdba1ffb640 1 -- 172.21.15.100:0/3876393027 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb4029060 con 0x564e05105d20 2024-05-10T11:42:28.382 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.378+0000 7fdba1ffb640 1 -- 172.21.15.100:0/3876393027 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fdbb4004a40 con 0x564e05105d20 2024-05-10T11:42:28.383 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.378+0000 7fdbc3867740 1 -- 172.21.15.100:0/3876393027 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbb4004030 con 0x564e05105d20 2024-05-10T11:42:28.383 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.378+0000 7fdba1ffb640 1 -- 172.21.15.100:0/3876393027 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fdbb4030050 con 0x564e05105d20 2024-05-10T11:42:28.383 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.378+0000 7fdbc3867740 10 client.20037 fetch_fsmap learned FSMap version 41 2024-05-10T11:42:28.383 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.378+0000 7fdbc3867740 1 -- 172.21.15.100:0/3876393027 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x564e050f8620 con 0x564e05105d20 2024-05-10T11:42:28.383 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.378+0000 7fdba1ffb640 1 -- 172.21.15.100:0/3876393027 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdbb4004030 con 0x564e05105d20 2024-05-10T11:42:28.383 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.379+0000 7fdba1ffb640 1 -- 172.21.15.100:0/3876393027 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fdbb4011620 con 0x564e05105d20 2024-05-10T11:42:28.383 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.379+0000 7fdbc3867740 10 client.20037 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:42:28.383 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.379+0000 7fdbc3867740 10 client.20037 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:42:28.383 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.379+0000 7fdbc3867740 20 client.20037 populate_metadata read hostname 'smithi100' 2024-05-10T11:42:28.383 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.379+0000 7fdbc3867740 1 --2- 172.21.15.100:0/3876393027 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x564e050fa0c0 0x564e050fa4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:28.383 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.380+0000 7fdbc3867740 4 client.20037 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:42:28.383 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.380+0000 7fdbc3867740 1 -- 172.21.15.100:0/3876393027 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e0506e1e0 con 0x564e050fa0c0 2024-05-10T11:42:28.383 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.380+0000 7fdbba59c640 1 --2- 172.21.15.100:0/3876393027 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x564e050fa0c0 0x564e050fa4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:28.383 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.380+0000 7fdbba59c640 1 --2- 172.21.15.100:0/3876393027 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x564e050fa0c0 0x564e050fa4c0 crc :-1 s=READY pgs=449 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3e60b36b8402f37 server_cookie=6fc8133fb1b0c9c2 in_seq=0 out_seq=0 2024-05-10T11:42:28.384 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.380+0000 7fdba1ffb640 10 client.20037 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:42:28.384 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.381+0000 7fdba1ffb640 1 -- 172.21.15.100:0/3876393027 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564e0506e1e0 con 0x564e050fa0c0 2024-05-10T11:42:28.385 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.381+0000 7fdba1ffb640 10 client.20037 handle_command_reply: tid=0 2024-05-10T11:42:28.385 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.381+0000 7fdbc3867740 1 client.20037 shutdown 2024-05-10T11:42:28.385 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.381+0000 7fdbc3867740 1 -- 172.21.15.100:0/3876393027 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e05105d20 msgr2=0x564e051119e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:28.385 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.381+0000 7fdbc3867740 1 --2- 172.21.15.100:0/3876393027 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e05105d20 0x564e051119e0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fdbb4000f80 tx=0x7fdbb40043f0 comp rx=0 tx=0).stop 2024-05-10T11:42:28.385 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.381+0000 7fdbc3867740 1 -- 172.21.15.100:0/3876393027 shutdown_connections 2024-05-10T11:42:28.385 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.381+0000 7fdbc3867740 1 --2- 172.21.15.100:0/3876393027 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x564e050fa0c0 0x564e050fa4c0 crc :-1 s=READY pgs=449 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:28.385 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.381+0000 7fdbc3867740 1 --2- 172.21.15.100:0/3876393027 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e05105d20 0x564e051119e0 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:28.385 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.381+0000 7fdbc3867740 1 --2- 172.21.15.100:0/3876393027 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564e05100bb0 0x564e0510a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:28.385 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.381+0000 7fdba1ffb640 0 client.20037 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:42:28.385 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.381+0000 7fdbc3867740 1 --2- 172.21.15.100:0/3876393027 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e051001c0 0x564e0510a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:28.385 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.381+0000 7fdbc3867740 1 -- 172.21.15.100:0/3876393027 >> 172.21.15.100:0/3876393027 conn(0x564e050fb830 msgr2=0x564e050f6580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:42:28.385 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.382+0000 7fdbc3867740 1 -- 172.21.15.100:0/3876393027 shutdown_connections 2024-05-10T11:42:28.385 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.382+0000 7fdbc3867740 1 -- 172.21.15.100:0/3876393027 wait complete. 2024-05-10T11:42:28.385 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.382+0000 7fdbc3867740 20 client.20037 trim_cache size 0 max 16384 2024-05-10T11:42:28.405 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.403+0000 7fdbc3867740 1 Processor -- start 2024-05-10T11:42:28.405 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.403+0000 7fdbc3867740 1 -- start start 2024-05-10T11:42:28.405 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.404+0000 7fdbc3867740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e051001c0 0x564e0510a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:28.405 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.404+0000 7fdbc3867740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564e05100bb0 0x564e051119a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:28.405 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.404+0000 7fdbc3867740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e05105d20 0x564e0510a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:28.405 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.404+0000 7fdbc3867740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9c012380 con 0x564e051001c0 2024-05-10T11:42:28.405 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.404+0000 7fdbc3867740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9c00fc50 con 0x564e05105d20 2024-05-10T11:42:28.405 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.404+0000 7fdbc3867740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9c00daa0 con 0x564e05100bb0 2024-05-10T11:42:28.405 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.404+0000 7fdbba59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e051001c0 0x564e0510a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:28.406 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.404+0000 7fdbba59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e051001c0 0x564e0510a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60464/0 (socket says 172.21.15.100:60464) 2024-05-10T11:42:28.406 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.404+0000 7fdbba59c640 1 -- 172.21.15.100:0/3933498307 learned_addr learned my addr 172.21.15.100:0/3933498307 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:42:28.406 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.404+0000 7fdbb9d9b640 1 --2- 172.21.15.100:0/3933498307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564e05100bb0 0x564e051119a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:28.406 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.404+0000 7fdbbad9d640 1 --2- 172.21.15.100:0/3933498307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e05105d20 0x564e0510a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:28.406 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.404+0000 7fdbba59c640 1 -- 172.21.15.100:0/3933498307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564e05100bb0 msgr2=0x564e051119a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:28.406 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.404+0000 7fdbba59c640 1 --2- 172.21.15.100:0/3933498307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564e05100bb0 0x564e051119a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:28.406 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.404+0000 7fdbba59c640 1 -- 172.21.15.100:0/3933498307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e05105d20 msgr2=0x564e0510a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:28.406 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.404+0000 7fdbba59c640 1 --2- 172.21.15.100:0/3933498307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e05105d20 0x564e0510a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:28.406 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.404+0000 7fdbba59c640 1 -- 172.21.15.100:0/3933498307 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e050fab40 con 0x564e051001c0 2024-05-10T11:42:28.407 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.404+0000 7fdbba59c640 1 --2- 172.21.15.100:0/3933498307 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e051001c0 0x564e0510a460 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fdbac005ab0 tx=0x7fdbac01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81cae62858719f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:42:28.407 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.405+0000 7fdba3fff640 1 -- 172.21.15.100:0/3933498307 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbac01ddf0 con 0x564e051001c0 2024-05-10T11:42:28.407 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.405+0000 7fdba3fff640 1 -- 172.21.15.100:0/3933498307 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdbac0054b0 con 0x564e051001c0 2024-05-10T11:42:28.407 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.405+0000 7fdba3fff640 1 -- 172.21.15.100:0/3933498307 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbac02c0f0 con 0x564e051001c0 2024-05-10T11:42:28.410 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.406+0000 7fdbc3867740 1 -- 172.21.15.100:0/3933498307 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e051001c0 msgr2=0x564e0510a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:28.410 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.406+0000 7fdbc3867740 1 --2- 172.21.15.100:0/3933498307 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e051001c0 0x564e0510a460 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fdbac005ab0 tx=0x7fdbac01b000 comp rx=0 tx=0).stop 2024-05-10T11:42:28.410 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.406+0000 7fdbc3867740 1 -- 172.21.15.100:0/3933498307 shutdown_connections 2024-05-10T11:42:28.410 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.406+0000 7fdbc3867740 1 --2- 172.21.15.100:0/3933498307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e05105d20 0x564e0510a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:28.410 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.406+0000 7fdbc3867740 1 --2- 172.21.15.100:0/3933498307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564e05100bb0 0x564e051119a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:28.410 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.406+0000 7fdbc3867740 1 --2- 172.21.15.100:0/3933498307 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e051001c0 0x564e0510a460 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:28.410 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.406+0000 7fdbc3867740 1 -- 172.21.15.100:0/3933498307 >> 172.21.15.100:0/3933498307 conn(0x564e050fb830 msgr2=0x564e050fbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:42:28.410 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.406+0000 7fdbc3867740 1 -- 172.21.15.100:0/3933498307 shutdown_connections 2024-05-10T11:42:28.410 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.406+0000 7fdbc3867740 1 -- 172.21.15.100:0/3933498307 wait complete. 2024-05-10T11:42:28.414 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.406+0000 7fdbc3867740 1 -- start start 2024-05-10T11:42:28.414 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.406+0000 7fdbc3867740 1 Processor -- start 2024-05-10T11:42:28.414 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.407+0000 7fdbc3867740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e051001c0 0x564e05111620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:28.414 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.407+0000 7fdbc3867740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e05100bb0 0x564e0510e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:28.414 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.407+0000 7fdbc3867740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564e05105d20 0x564e0510e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:28.414 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.407+0000 7fdbc3867740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9c00e110 con 0x564e051001c0 2024-05-10T11:42:28.414 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.407+0000 7fdbc3867740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9c00f580 con 0x564e05100bb0 2024-05-10T11:42:28.414 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.407+0000 7fdbc3867740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9c00e7e0 con 0x564e05105d20 2024-05-10T11:42:28.414 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.407+0000 7fdbba59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e051001c0 0x564e05111620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:28.414 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.407+0000 7fdbc3867740 10 client.0 _ll_register_callbacks cb 0x7fdbb4009c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:42:28.414 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.407+0000 7fdbba59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e051001c0 0x564e05111620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60472/0 (socket says 172.21.15.100:60472) 2024-05-10T11:42:28.414 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.407+0000 7fdbba59c640 1 -- 172.21.15.100:0/3322348024 learned_addr learned my addr 172.21.15.100:0/3322348024 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:42:28.415 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.407+0000 7fdbbad9d640 1 --2- 172.21.15.100:0/3322348024 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564e05105d20 0x564e0510e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:28.415 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.407+0000 7fdbb9d9b640 1 --2- 172.21.15.100:0/3322348024 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e05100bb0 0x564e0510e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:28.415 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.407+0000 7fdbba59c640 1 -- 172.21.15.100:0/3322348024 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564e05105d20 msgr2=0x564e0510e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:28.415 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.407+0000 7fdbba59c640 1 --2- 172.21.15.100:0/3322348024 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564e05105d20 0x564e0510e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:28.415 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.407+0000 7fdbba59c640 1 -- 172.21.15.100:0/3322348024 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e05100bb0 msgr2=0x564e0510e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:28.415 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.407+0000 7fdbba59c640 1 --2- 172.21.15.100:0/3322348024 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e05100bb0 0x564e0510e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:28.415 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.407+0000 7fdbba59c640 1 -- 172.21.15.100:0/3322348024 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e05111c50 con 0x564e051001c0 2024-05-10T11:42:28.415 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.407+0000 7fdbba59c640 1 --2- 172.21.15.100:0/3322348024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e051001c0 0x564e05111620 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fdbac0096c0 tx=0x7fdbac0056f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f0140b03a787cfb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:42:28.415 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.407+0000 7fdba1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:42:28.415 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.408+0000 7fdba1ffb640 1 -- 172.21.15.100:0/3322348024 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbac0049c0 con 0x564e051001c0 2024-05-10T11:42:28.415 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.408+0000 7fdba1ffb640 1 -- 172.21.15.100:0/3322348024 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fdbac004d50 con 0x564e051001c0 2024-05-10T11:42:28.415 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.408+0000 7fdbc3867740 1 -- 172.21.15.100:0/3322348024 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbac0054b0 con 0x564e051001c0 2024-05-10T11:42:28.415 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.408+0000 7fdba1ffb640 1 -- 172.21.15.100:0/3322348024 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fdbac045050 con 0x564e051001c0 2024-05-10T11:42:28.415 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.408+0000 7fdbc3867740 10 client.20043 fetch_fsmap learned FSMap version 41 2024-05-10T11:42:28.416 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.408+0000 7fdbc3867740 1 -- 172.21.15.100:0/3322348024 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x564e05184490 con 0x564e051001c0 2024-05-10T11:42:28.416 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.408+0000 7fdba1ffb640 1 -- 172.21.15.100:0/3322348024 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdbac02c0f0 con 0x564e051001c0 2024-05-10T11:42:28.416 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.409+0000 7fdba1ffb640 1 -- 172.21.15.100:0/3322348024 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fdbac03f4d0 con 0x564e051001c0 2024-05-10T11:42:28.416 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.409+0000 7fdbc3867740 10 client.20043 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:42:28.416 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.409+0000 7fdbc3867740 10 client.20043 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:42:28.416 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.409+0000 7fdbc3867740 20 client.20043 populate_metadata read hostname 'smithi100' 2024-05-10T11:42:28.416 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.409+0000 7fdbc3867740 1 --2- 172.21.15.100:0/3322348024 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x564e05184dc0 0x564e051871e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:28.416 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.409+0000 7fdbc3867740 4 client.20043 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:42:28.416 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.409+0000 7fdbc3867740 1 -- 172.21.15.100:0/3322348024 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fdbac0054b0 con 0x564e05184dc0 2024-05-10T11:42:28.416 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.409+0000 7fdbb9d9b640 1 --2- 172.21.15.100:0/3322348024 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x564e05184dc0 0x564e051871e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:28.416 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.409+0000 7fdbb9d9b640 1 --2- 172.21.15.100:0/3322348024 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x564e05184dc0 0x564e051871e0 crc :-1 s=READY pgs=450 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7abb0cb931966042 server_cookie=462d205fe09d88af in_seq=0 out_seq=0 2024-05-10T11:42:28.416 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.409+0000 7fdba1ffb640 10 client.20043 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:42:28.544 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.543+0000 7fdba1ffb640 1 -- 172.21.15.100:0/3322348024 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fdbac0054b0 con 0x564e05184dc0 2024-05-10T11:42:28.544 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.543+0000 7fdba1ffb640 10 client.20043 handle_command_reply: tid=0 2024-05-10T11:42:28.544 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.543+0000 7fdbc3867740 1 client.20043 shutdown 2024-05-10T11:42:28.544 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.543+0000 7fdbc3867740 1 -- 172.21.15.100:0/3322348024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e051001c0 msgr2=0x564e05111620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:28.544 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.543+0000 7fdbc3867740 1 --2- 172.21.15.100:0/3322348024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e051001c0 0x564e05111620 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fdbac0096c0 tx=0x7fdbac0056f0 comp rx=0 tx=0).stop 2024-05-10T11:42:28.545 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.543+0000 7fdbc3867740 1 -- 172.21.15.100:0/3322348024 shutdown_connections 2024-05-10T11:42:28.545 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.543+0000 7fdbc3867740 1 --2- 172.21.15.100:0/3322348024 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x564e05184dc0 0x564e051871e0 crc :-1 s=READY pgs=450 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:28.545 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.543+0000 7fdbc3867740 1 --2- 172.21.15.100:0/3322348024 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564e05105d20 0x564e0510e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:28.545 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.543+0000 7fdbc3867740 1 --2- 172.21.15.100:0/3322348024 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e05100bb0 0x564e0510e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:28.545 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.543+0000 7fdbc3867740 1 --2- 172.21.15.100:0/3322348024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564e051001c0 0x564e05111620 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:28.545 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.543+0000 7fdba1ffb640 0 client.20043 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:42:28.545 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.543+0000 7fdbc3867740 1 -- 172.21.15.100:0/3322348024 >> 172.21.15.100:0/3322348024 conn(0x564e050fb830 msgr2=0x564e050f5cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:42:28.545 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.544+0000 7fdbc3867740 1 -- 172.21.15.100:0/3322348024 shutdown_connections 2024-05-10T11:42:28.545 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.544+0000 7fdbc3867740 1 -- 172.21.15.100:0/3322348024 wait complete. 2024-05-10T11:42:28.545 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.544+0000 7fdbc3867740 20 client.20043 trim_cache size 0 max 16384 2024-05-10T11:42:28.546 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:42:28.546 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:42:28.546 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:42:28.546 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:42:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.544+0000 7fdbc0b46640 1 -- 172.21.15.100:0/875326159 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fdba40b2920 msgr2=0x7fdba40b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.544+0000 7fdbc0b46640 1 --2- 172.21.15.100:0/875326159 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fdba40b2920 0x7fdba40b4de0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x564e05100fb0 tx=0x7fdbac009260 comp rx=0 tx=0).stop 2024-05-10T11:42:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.544+0000 7fdbc0b46640 1 -- 172.21.15.100:0/875326159 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbbc102b00 msgr2=0x7fdbbc199030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.544+0000 7fdbc0b46640 1 --2- 172.21.15.100:0/875326159 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbbc102b00 0x7fdbbc199030 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fdbb002a860 tx=0x7fdbb0030760 comp rx=0 tx=0).stop 2024-05-10T11:42:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.544+0000 7fdbc0b46640 1 -- 172.21.15.100:0/875326159 shutdown_connections 2024-05-10T11:42:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.544+0000 7fdbc0b46640 1 --2- 172.21.15.100:0/875326159 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fdba40b2920 0x7fdba40b4de0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.544+0000 7fdbc0b46640 1 --2- 172.21.15.100:0/875326159 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdbbc1034c0 0x7fdbbc1a00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.544+0000 7fdbc0b46640 1 --2- 172.21.15.100:0/875326159 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbbc102b00 0x7fdbbc199030 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.544+0000 7fdbc0b46640 1 --2- 172.21.15.100:0/875326159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdbbc101910 0x7fdbbc198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.545+0000 7fdbc0b46640 1 -- 172.21.15.100:0/875326159 >> 172.21.15.100:0/875326159 conn(0x7fdbbc0fce80 msgr2=0x7fdbbc105f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:42:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.545+0000 7fdbc0b46640 1 -- 172.21.15.100:0/875326159 shutdown_connections 2024-05-10T11:42:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:28.545+0000 7fdbc0b46640 1 -- 172.21.15.100:0/875326159 wait complete. 2024-05-10T11:42:28.959 INFO:tasks.workunit.client.0.smithi100.stdout: 10 103312 8.91 MB/sec execute 502 sec latency 153.057 ms 2024-05-10T11:42:29.959 INFO:tasks.workunit.client.0.smithi100.stdout: 10 103469 8.90 MB/sec execute 503 sec latency 77.882 ms 2024-05-10T11:42:30.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:42:29 smithi178 ceph-mon[33359]: pgmap v1144: 129 pgs: 129 active+clean; 571 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 901 KiB/s rd, 4.8 MiB/s wr, 284 op/s 2024-05-10T11:42:30.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:42:29 smithi188 ceph-mon[34075]: pgmap v1144: 129 pgs: 129 active+clean; 571 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 901 KiB/s rd, 4.8 MiB/s wr, 284 op/s 2024-05-10T11:42:30.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:42:29 smithi100 ceph-mon[30296]: pgmap v1144: 129 pgs: 129 active+clean; 571 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 901 KiB/s rd, 4.8 MiB/s wr, 284 op/s 2024-05-10T11:42:30.959 INFO:tasks.workunit.client.0.smithi100.stdout: 10 103651 8.89 MB/sec execute 504 sec latency 111.902 ms 2024-05-10T11:42:31.959 INFO:tasks.workunit.client.0.smithi100.stdout: 10 103826 8.89 MB/sec execute 505 sec latency 89.034 ms 2024-05-10T11:42:32.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:42:31 smithi178 ceph-mon[33359]: pgmap v1145: 129 pgs: 129 active+clean; 567 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.2 MiB/s wr, 298 op/s 2024-05-10T11:42:32.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:42:31 smithi188 ceph-mon[34075]: pgmap v1145: 129 pgs: 129 active+clean; 567 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.2 MiB/s wr, 298 op/s 2024-05-10T11:42:32.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:42:31 smithi100 ceph-mon[30296]: pgmap v1145: 129 pgs: 129 active+clean; 567 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.2 MiB/s wr, 298 op/s 2024-05-10T11:42:32.959 INFO:tasks.workunit.client.0.smithi100.stdout: 10 103996 8.89 MB/sec execute 506 sec latency 102.014 ms 2024-05-10T11:42:33.959 INFO:tasks.workunit.client.0.smithi100.stdout: 10 104187 8.90 MB/sec execute 507 sec latency 94.698 ms 2024-05-10T11:42:34.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:42:33 smithi178 ceph-mon[33359]: pgmap v1146: 129 pgs: 129 active+clean; 566 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 3.7 MiB/s wr, 285 op/s 2024-05-10T11:42:34.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:42:33 smithi188 ceph-mon[34075]: pgmap v1146: 129 pgs: 129 active+clean; 566 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 3.7 MiB/s wr, 285 op/s 2024-05-10T11:42:34.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:42:33 smithi100 ceph-mon[30296]: pgmap v1146: 129 pgs: 129 active+clean; 566 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 3.7 MiB/s wr, 285 op/s 2024-05-10T11:42:34.959 INFO:tasks.workunit.client.0.smithi100.stdout: 10 104429 8.93 MB/sec execute 508 sec latency 68.676 ms 2024-05-10T11:42:35.959 INFO:tasks.workunit.client.0.smithi100.stdout: 10 104615 8.93 MB/sec execute 509 sec latency 73.678 ms 2024-05-10T11:42:36.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:42:35 smithi178 ceph-mon[33359]: pgmap v1147: 129 pgs: 129 active+clean; 569 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1017 KiB/s rd, 3.8 MiB/s wr, 316 op/s 2024-05-10T11:42:36.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:42:35 smithi188 ceph-mon[34075]: pgmap v1147: 129 pgs: 129 active+clean; 569 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1017 KiB/s rd, 3.8 MiB/s wr, 316 op/s 2024-05-10T11:42:36.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:42:35 smithi100 ceph-mon[30296]: pgmap v1147: 129 pgs: 129 active+clean; 569 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1017 KiB/s rd, 3.8 MiB/s wr, 316 op/s 2024-05-10T11:42:36.959 INFO:tasks.workunit.client.0.smithi100.stdout: 10 104805 8.93 MB/sec execute 510 sec latency 65.890 ms 2024-05-10T11:42:37.959 INFO:tasks.workunit.client.0.smithi100.stdout: 10 104963 8.94 MB/sec execute 511 sec latency 77.492 ms 2024-05-10T11:42:38.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:42:37 smithi178 ceph-mon[33359]: pgmap v1148: 129 pgs: 129 active+clean; 575 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 933 KiB/s rd, 4.2 MiB/s wr, 317 op/s 2024-05-10T11:42:38.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:42:37 smithi188 ceph-mon[34075]: pgmap v1148: 129 pgs: 129 active+clean; 575 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 933 KiB/s rd, 4.2 MiB/s wr, 317 op/s 2024-05-10T11:42:38.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:42:37 smithi100 ceph-mon[30296]: pgmap v1148: 129 pgs: 129 active+clean; 575 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 933 KiB/s rd, 4.2 MiB/s wr, 317 op/s 2024-05-10T11:42:38.959 INFO:tasks.workunit.client.0.smithi100.stdout: 10 105130 8.94 MB/sec execute 512 sec latency 87.540 ms 2024-05-10T11:42:39.617 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T11:42:39.960 INFO:tasks.workunit.client.0.smithi100.stdout: 10 105285 8.94 MB/sec execute 513 sec latency 64.292 ms 2024-05-10T11:42:39.996 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:42:40.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:42:39 smithi178 ceph-mon[33359]: pgmap v1149: 129 pgs: 129 active+clean; 579 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 4.1 MiB/s wr, 278 op/s 2024-05-10T11:42:40.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:42:39 smithi188 ceph-mon[34075]: pgmap v1149: 129 pgs: 129 active+clean; 579 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 4.1 MiB/s wr, 278 op/s 2024-05-10T11:42:40.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:42:39 smithi100 ceph-mon[30296]: pgmap v1149: 129 pgs: 129 active+clean; 579 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 4.1 MiB/s wr, 278 op/s 2024-05-10T11:42:40.959 INFO:tasks.workunit.client.0.smithi100.stdout: 10 105428 8.94 MB/sec execute 514 sec latency 100.142 ms 2024-05-10T11:42:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.679+0000 7f9704aa7640 1 -- 172.21.15.100:0/1900613219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9700104b00 msgr2=0x7f9700106ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.679+0000 7f9704aa7640 1 --2- 172.21.15.100:0/1900613219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9700104b00 0x7f9700106ef0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f96f4009860 tx=0x7f96f402a340 comp rx=0 tx=0).stop 2024-05-10T11:42:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.679+0000 7f9704aa7640 1 -- 172.21.15.100:0/1900613219 shutdown_connections 2024-05-10T11:42:41.681 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.679+0000 7f9704aa7640 1 --2- 172.21.15.100:0/1900613219 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9700107510 0x7f97001099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:41.682 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.679+0000 7f9704aa7640 1 --2- 172.21.15.100:0/1900613219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9700104b00 0x7f9700106ef0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:41.682 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.679+0000 7f9704aa7640 1 --2- 172.21.15.100:0/1900613219 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97001021d0 0x7f97001045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:41.682 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.680+0000 7f9704aa7640 1 -- 172.21.15.100:0/1900613219 >> 172.21.15.100:0/1900613219 conn(0x7f97000fc1d0 msgr2=0x7f97000fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:42:41.682 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.680+0000 7f9704aa7640 1 -- 172.21.15.100:0/1900613219 shutdown_connections 2024-05-10T11:42:41.682 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.680+0000 7f9704aa7640 1 -- 172.21.15.100:0/1900613219 wait complete. 2024-05-10T11:42:41.682 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.680+0000 7f9704aa7640 1 Processor -- start 2024-05-10T11:42:41.682 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.680+0000 7f9704aa7640 1 -- start start 2024-05-10T11:42:41.683 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.681+0000 7f9704aa7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97001021d0 0x7f9700101a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:41.683 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.681+0000 7f9704aa7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9700104b00 0x7f970019fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:41.684 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.681+0000 7f9704aa7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9700107510 0x7f97001a21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:41.684 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.681+0000 7f9704aa7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f970010d330 con 0x7f9700107510 2024-05-10T11:42:41.684 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.681+0000 7f9704aa7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f970010d1b0 con 0x7f9700104b00 2024-05-10T11:42:41.684 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.681+0000 7f9704aa7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f970010d4b0 con 0x7f97001021d0 2024-05-10T11:42:41.684 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.681+0000 7f96fdd9b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9700104b00 0x7f970019fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:41.684 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.681+0000 7f96fdd9b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9700104b00 0x7f970019fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:44790/0 (socket says 172.21.15.100:44790) 2024-05-10T11:42:41.684 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.681+0000 7f96fdd9b640 1 -- 172.21.15.100:0/2575742580 learned_addr learned my addr 172.21.15.100:0/2575742580 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:42:41.684 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.681+0000 7f96fe59c640 1 --2- 172.21.15.100:0/2575742580 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97001021d0 0x7f9700101a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:41.684 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.681+0000 7f96fed9d640 1 --2- 172.21.15.100:0/2575742580 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9700107510 0x7f97001a21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:41.684 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.681+0000 7f96fdd9b640 1 -- 172.21.15.100:0/2575742580 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97001021d0 msgr2=0x7f9700101a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:41.684 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.681+0000 7f96fdd9b640 1 --2- 172.21.15.100:0/2575742580 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97001021d0 0x7f9700101a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:41.684 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.681+0000 7f96fdd9b640 1 -- 172.21.15.100:0/2575742580 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9700107510 msgr2=0x7f97001a21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:41.684 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.681+0000 7f96fdd9b640 1 --2- 172.21.15.100:0/2575742580 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9700107510 0x7f97001a21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:41.684 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.681+0000 7f96fdd9b640 1 -- 172.21.15.100:0/2575742580 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97001a29e0 con 0x7f9700104b00 2024-05-10T11:42:41.685 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.682+0000 7f96fdd9b640 1 --2- 172.21.15.100:0/2575742580 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9700104b00 0x7f970019fca0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f96f4009780 tx=0x7f96f40306e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:42:41.685 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.682+0000 7f96ef7fe640 1 -- 172.21.15.100:0/2575742580 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f4037040 con 0x7f9700104b00 2024-05-10T11:42:41.685 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.682+0000 7f96ef7fe640 1 -- 172.21.15.100:0/2575742580 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f96f4032df0 con 0x7f9700104b00 2024-05-10T11:42:41.685 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.682+0000 7f96ef7fe640 1 -- 172.21.15.100:0/2575742580 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f4003970 con 0x7f9700104b00 2024-05-10T11:42:41.686 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.682+0000 7f9704aa7640 1 -- 172.21.15.100:0/2575742580 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97001a2c10 con 0x7f9700104b00 2024-05-10T11:42:41.686 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.682+0000 7f9704aa7640 1 -- 172.21.15.100:0/2575742580 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97001a2ff0 con 0x7f9700104b00 2024-05-10T11:42:41.686 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.685+0000 7f96ef7fe640 1 -- 172.21.15.100:0/2575742580 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f96f4043020 con 0x7f9700104b00 2024-05-10T11:42:41.686 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.686+0000 7f97077c8740 1 Processor -- start 2024-05-10T11:42:41.687 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.687+0000 7f96ef7fe640 1 --2- 172.21.15.100:0/2575742580 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f96e80b2af0 0x7f96e80b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:41.687 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.687+0000 7f96ef7fe640 1 -- 172.21.15.100:0/2575742580 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f96f4101af0 con 0x7f9700104b00 2024-05-10T11:42:41.687 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.687+0000 7f96fed9d640 1 --2- 172.21.15.100:0/2575742580 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f96e80b2af0 0x7f96e80b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:41.688 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.688+0000 7f97077c8740 1 -- start start 2024-05-10T11:42:41.688 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.688+0000 7f97077c8740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650a9962db0 0x5650a99631d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:41.688 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.688+0000 7f97077c8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5650a99637a0 0x5650a99682a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:41.688 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.688+0000 7f97077c8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650a9968910 0x5650a996ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:41.688 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.688+0000 7f96fed9d640 1 --2- 172.21.15.100:0/2575742580 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f96e80b2af0 0x7f96e80b4fb0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f96f8005fd0 tx=0x7f96f80095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:42:41.688 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.688+0000 7f97077c8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5650a98c75d0 con 0x5650a9968910 2024-05-10T11:42:41.689 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.688+0000 7f97077c8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5650a9880890 con 0x5650a99637a0 2024-05-10T11:42:41.689 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.688+0000 7f97077c8740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5650a98c3590 con 0x5650a9962db0 2024-05-10T11:42:41.689 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.689+0000 7f96fed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650a9968910 0x5650a996ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:41.689 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.689+0000 7f96fed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650a9968910 0x5650a996ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50946/0 (socket says 172.21.15.100:50946) 2024-05-10T11:42:41.689 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.689+0000 7f96fed9d640 1 -- 172.21.15.100:0/1863943052 learned_addr learned my addr 172.21.15.100:0/1863943052 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:42:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.690+0000 7f96fe59c640 1 --2- 172.21.15.100:0/1863943052 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650a9962db0 0x5650a99631d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.690+0000 7f96fdd9b640 1 --2- 172.21.15.100:0/1863943052 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5650a99637a0 0x5650a99682a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.691+0000 7f96fe59c640 1 -- 172.21.15.100:0/1863943052 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5650a99637a0 msgr2=0x5650a99682a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.691+0000 7f96fe59c640 1 --2- 172.21.15.100:0/1863943052 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5650a99637a0 0x5650a99682a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.691+0000 7f96fe59c640 1 -- 172.21.15.100:0/1863943052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650a9968910 msgr2=0x5650a996ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.691+0000 7f96fe59c640 1 --2- 172.21.15.100:0/1863943052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650a9968910 0x5650a996ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.691+0000 7f96fe59c640 1 -- 172.21.15.100:0/1863943052 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650a95d9210 con 0x5650a9962db0 2024-05-10T11:42:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.691+0000 7f96fed9d640 1 --2- 172.21.15.100:0/1863943052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650a9968910 0x5650a996ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:42:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.691+0000 7f96fe59c640 1 --2- 172.21.15.100:0/1863943052 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650a9962db0 0x5650a99631d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f96f000bbc0 tx=0x7f96f00107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da89a33e9a55ddde server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:42:41.692 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.691+0000 7f96ecff9640 1 -- 172.21.15.100:0/1863943052 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f0004030 con 0x5650a9962db0 2024-05-10T11:42:41.692 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.691+0000 7f96ecff9640 1 -- 172.21.15.100:0/1863943052 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f96f00035f0 con 0x5650a9962db0 2024-05-10T11:42:41.692 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.691+0000 7f96ecff9640 1 -- 172.21.15.100:0/1863943052 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f00116a0 con 0x5650a9962db0 2024-05-10T11:42:41.693 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.693+0000 7f97077c8740 1 -- 172.21.15.100:0/1863943052 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650a9962db0 msgr2=0x5650a99631d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:41.693 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.693+0000 7f97077c8740 1 --2- 172.21.15.100:0/1863943052 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650a9962db0 0x5650a99631d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f96f000bbc0 tx=0x7f96f00107a0 comp rx=0 tx=0).stop 2024-05-10T11:42:41.693 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.693+0000 7f97077c8740 1 -- 172.21.15.100:0/1863943052 shutdown_connections 2024-05-10T11:42:41.693 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.693+0000 7f97077c8740 1 --2- 172.21.15.100:0/1863943052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650a9968910 0x5650a996ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:41.693 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.693+0000 7f97077c8740 1 --2- 172.21.15.100:0/1863943052 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5650a99637a0 0x5650a99682a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:41.693 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.693+0000 7f97077c8740 1 --2- 172.21.15.100:0/1863943052 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650a9962db0 0x5650a99631d0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:41.693 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.693+0000 7f97077c8740 1 -- 172.21.15.100:0/1863943052 >> 172.21.15.100:0/1863943052 conn(0x5650a995e420 msgr2=0x5650a995e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:42:41.693 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.693+0000 7f97077c8740 1 -- 172.21.15.100:0/1863943052 shutdown_connections 2024-05-10T11:42:41.693 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.693+0000 7f97077c8740 1 -- 172.21.15.100:0/1863943052 wait complete. 2024-05-10T11:42:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.693+0000 7f97077c8740 1 -- start start 2024-05-10T11:42:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.693+0000 7f97077c8740 1 Processor -- start 2024-05-10T11:42:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.694+0000 7f97077c8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650a9962db0 0x5650a996d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.694+0000 7f96fe59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650a9962db0 0x5650a996d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.694+0000 7f96fe59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650a9962db0 0x5650a996d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50952/0 (socket says 172.21.15.100:50952) 2024-05-10T11:42:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.694+0000 7f96fe59c640 1 -- 172.21.15.100:0/3825052916 learned_addr learned my addr 172.21.15.100:0/3825052916 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:42:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.694+0000 7f97077c8740 1 --2- 172.21.15.100:0/3825052916 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650a99637a0 0x5650a996d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.694+0000 7f97077c8740 1 --2- 172.21.15.100:0/3825052916 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5650a9968910 0x5650a99745d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.694+0000 7f97077c8740 1 -- 172.21.15.100:0/3825052916 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5650a996b8c0 con 0x5650a9962db0 2024-05-10T11:42:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.694+0000 7f97077c8740 1 -- 172.21.15.100:0/3825052916 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5650a9884370 con 0x5650a9968910 2024-05-10T11:42:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.694+0000 7f97077c8740 1 -- 172.21.15.100:0/3825052916 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5650a996ba40 con 0x5650a99637a0 2024-05-10T11:42:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.694+0000 7f96fdd9b640 1 --2- 172.21.15.100:0/3825052916 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650a99637a0 0x5650a996d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.694+0000 7f96fed9d640 1 --2- 172.21.15.100:0/3825052916 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5650a9968910 0x5650a99745d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.694+0000 7f97077c8740 10 client.0 _ll_register_callbacks cb 0x5650a95d8a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:42:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.694+0000 7f96fe59c640 1 -- 172.21.15.100:0/3825052916 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650a99637a0 msgr2=0x5650a996d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.694+0000 7f96fe59c640 1 --2- 172.21.15.100:0/3825052916 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650a99637a0 0x5650a996d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.694+0000 7f96fe59c640 1 -- 172.21.15.100:0/3825052916 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5650a9968910 msgr2=0x5650a99745d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.694+0000 7f96fe59c640 1 --2- 172.21.15.100:0/3825052916 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5650a9968910 0x5650a99745d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.695+0000 7f96fe59c640 1 -- 172.21.15.100:0/3825052916 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650a96a5620 con 0x5650a9962db0 2024-05-10T11:42:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.695+0000 7f96fe59c640 1 --2- 172.21.15.100:0/3825052916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650a9962db0 0x5650a996d050 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f96f00044f0 tx=0x7f96f0004520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a40144cd8e95c152 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:42:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.695+0000 7f96e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:42:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.695+0000 7f96e5ffb640 1 -- 172.21.15.100:0/3825052916 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f002a060 con 0x5650a9962db0 2024-05-10T11:42:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.695+0000 7f96e5ffb640 1 -- 172.21.15.100:0/3825052916 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f96f00039b0 con 0x5650a9962db0 2024-05-10T11:42:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.695+0000 7f97077c8740 1 -- 172.21.15.100:0/3825052916 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96f00035f0 con 0x5650a9962db0 2024-05-10T11:42:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.695+0000 7f96e5ffb640 1 -- 172.21.15.100:0/3825052916 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f96f0003bb0 con 0x5650a9962db0 2024-05-10T11:42:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.695+0000 7f97077c8740 10 client.20055 fetch_fsmap learned FSMap version 41 2024-05-10T11:42:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.695+0000 7f97077c8740 1 -- 172.21.15.100:0/3825052916 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5650a9977360 con 0x5650a9962db0 2024-05-10T11:42:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.696+0000 7f96e5ffb640 1 -- 172.21.15.100:0/3825052916 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f96f00035f0 con 0x5650a9962db0 2024-05-10T11:42:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.696+0000 7f96e5ffb640 1 -- 172.21.15.100:0/3825052916 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f96f00049f0 con 0x5650a9962db0 2024-05-10T11:42:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.696+0000 7f97077c8740 10 client.20055 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:42:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.696+0000 7f97077c8740 10 client.20055 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:42:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.696+0000 7f97077c8740 20 client.20055 populate_metadata read hostname 'smithi100' 2024-05-10T11:42:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.696+0000 7f97077c8740 1 --2- 172.21.15.100:0/3825052916 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5650a9978710 0x5650a997ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.696+0000 7f97077c8740 4 client.20055 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:42:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.696+0000 7f97077c8740 1 -- 172.21.15.100:0/3825052916 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650a98d51e0 con 0x5650a9978710 2024-05-10T11:42:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.696+0000 7f96fdd9b640 1 --2- 172.21.15.100:0/3825052916 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5650a9978710 0x5650a997ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.697+0000 7f96e5ffb640 10 client.20055 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:42:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.697+0000 7f96fdd9b640 1 --2- 172.21.15.100:0/3825052916 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5650a9978710 0x5650a997ab00 crc :-1 s=READY pgs=451 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fef9070d22d4b9 server_cookie=93f98556755e31d5 in_seq=0 out_seq=0 2024-05-10T11:42:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.698+0000 7f96e5ffb640 1 -- 172.21.15.100:0/3825052916 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5650a98d51e0 con 0x5650a9978710 2024-05-10T11:42:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.698+0000 7f96e5ffb640 10 client.20055 handle_command_reply: tid=0 2024-05-10T11:42:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.698+0000 7f97077c8740 1 client.20055 shutdown 2024-05-10T11:42:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.698+0000 7f97077c8740 1 -- 172.21.15.100:0/3825052916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650a9962db0 msgr2=0x5650a996d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.698+0000 7f97077c8740 1 --2- 172.21.15.100:0/3825052916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650a9962db0 0x5650a996d050 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f96f00044f0 tx=0x7f96f0004520 comp rx=0 tx=0).stop 2024-05-10T11:42:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.698+0000 7f97077c8740 1 -- 172.21.15.100:0/3825052916 shutdown_connections 2024-05-10T11:42:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.698+0000 7f97077c8740 1 --2- 172.21.15.100:0/3825052916 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5650a9978710 0x5650a997ab00 crc :-1 s=READY pgs=451 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:41.702 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.698+0000 7f97077c8740 1 --2- 172.21.15.100:0/3825052916 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5650a9968910 0x5650a99745d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:41.702 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.698+0000 7f97077c8740 1 --2- 172.21.15.100:0/3825052916 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650a99637a0 0x5650a996d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:41.702 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.698+0000 7f97077c8740 1 --2- 172.21.15.100:0/3825052916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650a9962db0 0x5650a996d050 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:41.702 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.698+0000 7f97077c8740 1 -- 172.21.15.100:0/3825052916 >> 172.21.15.100:0/3825052916 conn(0x5650a995e420 msgr2=0x5650a995d580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:42:41.702 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.698+0000 7f96e5ffb640 0 client.20055 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:42:41.702 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.698+0000 7f97077c8740 1 -- 172.21.15.100:0/3825052916 shutdown_connections 2024-05-10T11:42:41.702 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.699+0000 7f97077c8740 1 -- 172.21.15.100:0/3825052916 wait complete. 2024-05-10T11:42:41.702 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.699+0000 7f97077c8740 20 client.20055 trim_cache size 0 max 16384 2024-05-10T11:42:41.729 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.728+0000 7f97077c8740 1 Processor -- start 2024-05-10T11:42:41.729 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.728+0000 7f97077c8740 1 -- start start 2024-05-10T11:42:41.730 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.728+0000 7f97077c8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5650a99637a0 0x5650a9961300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:41.730 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.728+0000 7f97077c8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650a9968910 0x5650a9961840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:41.730 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.728+0000 7f97077c8740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650a9961d80 0x5650a996c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:41.730 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.728+0000 7f97077c8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f97001afdd0 con 0x5650a9968910 2024-05-10T11:42:41.730 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.728+0000 7f97077c8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f97001ad6a0 con 0x5650a99637a0 2024-05-10T11:42:41.730 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.728+0000 7f97077c8740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f97001ab4f0 con 0x5650a9961d80 2024-05-10T11:42:41.730 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.728+0000 7f96fdd9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650a9968910 0x5650a9961840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:41.730 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.728+0000 7f96fdd9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650a9968910 0x5650a9961840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50966/0 (socket says 172.21.15.100:50966) 2024-05-10T11:42:41.730 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.728+0000 7f96fdd9b640 1 -- 172.21.15.100:0/2066714196 learned_addr learned my addr 172.21.15.100:0/2066714196 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:42:41.730 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.728+0000 7f96fed9d640 1 --2- 172.21.15.100:0/2066714196 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650a9961d80 0x5650a996c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:41.730 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.728+0000 7f96fe59c640 1 --2- 172.21.15.100:0/2066714196 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5650a99637a0 0x5650a9961300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:41.730 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.729+0000 7f96fdd9b640 1 -- 172.21.15.100:0/2066714196 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650a9961d80 msgr2=0x5650a996c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:41.730 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.729+0000 7f96fdd9b640 1 --2- 172.21.15.100:0/2066714196 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650a9961d80 0x5650a996c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:41.730 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.729+0000 7f96fdd9b640 1 -- 172.21.15.100:0/2066714196 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5650a99637a0 msgr2=0x5650a9961300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:41.731 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.729+0000 7f96fdd9b640 1 --2- 172.21.15.100:0/2066714196 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5650a99637a0 0x5650a9961300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:41.731 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.729+0000 7f96fdd9b640 1 -- 172.21.15.100:0/2066714196 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650a9964510 con 0x5650a9968910 2024-05-10T11:42:41.731 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.729+0000 7f96fdd9b640 1 --2- 172.21.15.100:0/2066714196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650a9968910 0x5650a9961840 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f96f40c4dd0 tx=0x7f96f40ff100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7dfd2abef8457a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:42:41.731 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.729+0000 7f96e7fff640 1 -- 172.21.15.100:0/2066714196 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f40c8860 con 0x5650a9968910 2024-05-10T11:42:41.731 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.729+0000 7f96e7fff640 1 -- 172.21.15.100:0/2066714196 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f96f40c9450 con 0x5650a9968910 2024-05-10T11:42:41.731 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.729+0000 7f96e7fff640 1 -- 172.21.15.100:0/2066714196 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f40ff340 con 0x5650a9968910 2024-05-10T11:42:41.735 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.730+0000 7f97077c8740 1 -- 172.21.15.100:0/2066714196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650a9968910 msgr2=0x5650a9961840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:41.735 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.730+0000 7f97077c8740 1 --2- 172.21.15.100:0/2066714196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650a9968910 0x5650a9961840 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f96f40c4dd0 tx=0x7f96f40ff100 comp rx=0 tx=0).stop 2024-05-10T11:42:41.735 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.730+0000 7f97077c8740 1 -- 172.21.15.100:0/2066714196 shutdown_connections 2024-05-10T11:42:41.735 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.730+0000 7f97077c8740 1 --2- 172.21.15.100:0/2066714196 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650a9961d80 0x5650a996c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:41.735 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.730+0000 7f97077c8740 1 --2- 172.21.15.100:0/2066714196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650a9968910 0x5650a9961840 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:41.735 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.730+0000 7f97077c8740 1 --2- 172.21.15.100:0/2066714196 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5650a99637a0 0x5650a9961300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:41.735 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.730+0000 7f97077c8740 1 -- 172.21.15.100:0/2066714196 >> 172.21.15.100:0/2066714196 conn(0x5650a995e420 msgr2=0x5650a99593e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:42:41.735 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.730+0000 7f97077c8740 1 -- 172.21.15.100:0/2066714196 shutdown_connections 2024-05-10T11:42:41.735 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.730+0000 7f97077c8740 1 -- 172.21.15.100:0/2066714196 wait complete. 2024-05-10T11:42:41.735 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.731+0000 7f97077c8740 1 -- start start 2024-05-10T11:42:41.735 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.731+0000 7f97077c8740 1 Processor -- start 2024-05-10T11:42:41.735 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.731+0000 7f97077c8740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650a9961d80 0x5650a9970f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:41.735 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.731+0000 7f97077c8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650a99637a0 0x5650a9971460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:41.736 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.731+0000 7f97077c8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5650a9972470 0x5650a99728f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:41.736 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.731+0000 7f97077c8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f97001abb60 con 0x5650a99637a0 2024-05-10T11:42:41.736 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.731+0000 7f97077c8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f97001acfd0 con 0x5650a9972470 2024-05-10T11:42:41.736 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.731+0000 7f97077c8740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f97001ac230 con 0x5650a9961d80 2024-05-10T11:42:41.736 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.731+0000 7f96fe59c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650a9961d80 0x5650a9970f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:41.736 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.731+0000 7f96fe59c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650a9961d80 0x5650a9970f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:48124/0 (socket says 172.21.15.100:48124) 2024-05-10T11:42:41.736 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.731+0000 7f96fe59c640 1 -- 172.21.15.100:0/3001271250 learned_addr learned my addr 172.21.15.100:0/3001271250 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:42:41.736 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.731+0000 7f96fdd9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650a99637a0 0x5650a9971460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:41.736 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.732+0000 7f97077c8740 10 client.0 _ll_register_callbacks cb 0x7f96f800fef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:42:41.736 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.732+0000 7f96fed9d640 1 --2- 172.21.15.100:0/3001271250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5650a9972470 0x5650a99728f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:41.736 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.732+0000 7f96fe59c640 1 -- 172.21.15.100:0/3001271250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5650a9972470 msgr2=0x5650a99728f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:41.736 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.732+0000 7f96fe59c640 1 --2- 172.21.15.100:0/3001271250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5650a9972470 0x5650a99728f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:41.736 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.732+0000 7f96fe59c640 1 -- 172.21.15.100:0/3001271250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650a99637a0 msgr2=0x5650a9971460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:41.736 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.732+0000 7f96fe59c640 1 --2- 172.21.15.100:0/3001271250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650a99637a0 0x5650a9971460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:41.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.732+0000 7f96fe59c640 1 -- 172.21.15.100:0/3001271250 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650a9972e30 con 0x5650a9961d80 2024-05-10T11:42:41.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.732+0000 7f96fdd9b640 1 --2- 172.21.15.100:0/3001271250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650a99637a0 0x5650a9971460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:42:41.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.732+0000 7f96fe59c640 1 --2- 172.21.15.100:0/3001271250 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650a9961d80 0x5650a9970f20 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x5650a996def0 tx=0x7f96f0011e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b3c6a84a9faf22a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:42:41.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.732+0000 7f96e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:42:41.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.732+0000 7f96e5ffb640 1 -- 172.21.15.100:0/3001271250 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f003a070 con 0x5650a9961d80 2024-05-10T11:42:41.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.732+0000 7f96e5ffb640 1 -- 172.21.15.100:0/3001271250 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f96f00039b0 con 0x5650a9961d80 2024-05-10T11:42:41.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.732+0000 7f97077c8740 1 -- 172.21.15.100:0/3001271250 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96f40c9450 con 0x5650a9961d80 2024-05-10T11:42:41.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.733+0000 7f96e5ffb640 1 -- 172.21.15.100:0/3001271250 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f96f0043050 con 0x5650a9961d80 2024-05-10T11:42:41.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.733+0000 7f97077c8740 10 client.29419 fetch_fsmap learned FSMap version 41 2024-05-10T11:42:41.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.733+0000 7f97077c8740 1 -- 172.21.15.100:0/3001271250 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5650a99e2eb0 con 0x5650a9961d80 2024-05-10T11:42:41.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.733+0000 7f96e5ffb640 1 -- 172.21.15.100:0/3001271250 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f96f40c9450 con 0x5650a9961d80 2024-05-10T11:42:41.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.733+0000 7f96e5ffb640 1 -- 172.21.15.100:0/3001271250 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f96f003d020 con 0x5650a9961d80 2024-05-10T11:42:41.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.733+0000 7f97077c8740 10 client.29419 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:42:41.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.733+0000 7f97077c8740 10 client.29419 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:42:41.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.733+0000 7f97077c8740 20 client.29419 populate_metadata read hostname 'smithi100' 2024-05-10T11:42:41.738 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.733+0000 7f97077c8740 1 --2- 172.21.15.100:0/3001271250 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5650a99e3760 0x5650a99e5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:41.738 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.733+0000 7f97077c8740 4 client.29419 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:42:41.738 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.733+0000 7f97077c8740 1 -- 172.21.15.100:0/3001271250 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5650a98cf050 con 0x5650a99e3760 2024-05-10T11:42:41.738 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.733+0000 7f96fdd9b640 1 --2- 172.21.15.100:0/3001271250 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5650a99e3760 0x5650a99e5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:41.738 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.734+0000 7f96fdd9b640 1 --2- 172.21.15.100:0/3001271250 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5650a99e3760 0x5650a99e5b80 crc :-1 s=READY pgs=452 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f5ced747e60992e server_cookie=173ffbdc7660fcbd in_seq=0 out_seq=0 2024-05-10T11:42:41.738 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.734+0000 7f96e5ffb640 10 client.29419 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:42:41.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:42:41 smithi100 ceph-mon[30296]: pgmap v1150: 129 pgs: 129 active+clean; 582 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 4.4 MiB/s wr, 271 op/s 2024-05-10T11:42:41.879 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.879+0000 7f96e5ffb640 1 -- 172.21.15.100:0/3001271250 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5650a98cf050 con 0x5650a99e3760 2024-05-10T11:42:41.880 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.879+0000 7f96e5ffb640 10 client.29419 handle_command_reply: tid=0 2024-05-10T11:42:41.880 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.879+0000 7f97077c8740 1 client.29419 shutdown 2024-05-10T11:42:41.881 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.880+0000 7f97077c8740 1 -- 172.21.15.100:0/3001271250 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650a9961d80 msgr2=0x5650a9970f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:41.881 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.880+0000 7f97077c8740 1 --2- 172.21.15.100:0/3001271250 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650a9961d80 0x5650a9970f20 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x5650a996def0 tx=0x7f96f0011e80 comp rx=0 tx=0).stop 2024-05-10T11:42:41.881 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.881+0000 7f97077c8740 1 -- 172.21.15.100:0/3001271250 shutdown_connections 2024-05-10T11:42:41.882 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.881+0000 7f97077c8740 1 --2- 172.21.15.100:0/3001271250 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5650a99e3760 0x5650a99e5b80 crc :-1 s=READY pgs=452 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:41.882 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.881+0000 7f97077c8740 1 --2- 172.21.15.100:0/3001271250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5650a9972470 0x5650a99728f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:41.882 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.881+0000 7f97077c8740 1 --2- 172.21.15.100:0/3001271250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650a99637a0 0x5650a9971460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:41.882 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.881+0000 7f97077c8740 1 --2- 172.21.15.100:0/3001271250 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650a9961d80 0x5650a9970f20 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:41.882 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.881+0000 7f96e5ffb640 0 client.29419 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:42:41.882 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.881+0000 7f97077c8740 1 -- 172.21.15.100:0/3001271250 >> 172.21.15.100:0/3001271250 conn(0x5650a995e420 msgr2=0x5650a9964a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:42:41.882 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.881+0000 7f97077c8740 1 -- 172.21.15.100:0/3001271250 shutdown_connections 2024-05-10T11:42:41.882 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.881+0000 7f97077c8740 1 -- 172.21.15.100:0/3001271250 wait complete. 2024-05-10T11:42:41.882 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.881+0000 7f97077c8740 20 client.29419 trim_cache size 0 max 16384 2024-05-10T11:42:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:42:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:42:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:42:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:42:41.884 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.882+0000 7f9704aa7640 1 -- 172.21.15.100:0/2575742580 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f96e80b2af0 msgr2=0x7f96e80b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:41.884 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.882+0000 7f9704aa7640 1 --2- 172.21.15.100:0/2575742580 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f96e80b2af0 0x7f96e80b4fb0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f96f8005fd0 tx=0x7f96f80095c0 comp rx=0 tx=0).stop 2024-05-10T11:42:41.884 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.882+0000 7f9704aa7640 1 -- 172.21.15.100:0/2575742580 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9700104b00 msgr2=0x7f970019fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:41.884 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.882+0000 7f9704aa7640 1 --2- 172.21.15.100:0/2575742580 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9700104b00 0x7f970019fca0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f96f4009780 tx=0x7f96f40306e0 comp rx=0 tx=0).stop 2024-05-10T11:42:41.884 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.882+0000 7f9704aa7640 1 -- 172.21.15.100:0/2575742580 shutdown_connections 2024-05-10T11:42:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.882+0000 7f9704aa7640 1 --2- 172.21.15.100:0/2575742580 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f96e80b2af0 0x7f96e80b4fb0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.882+0000 7f9704aa7640 1 --2- 172.21.15.100:0/2575742580 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9700107510 0x7f97001a21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.882+0000 7f9704aa7640 1 --2- 172.21.15.100:0/2575742580 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9700104b00 0x7f970019fca0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.882+0000 7f9704aa7640 1 --2- 172.21.15.100:0/2575742580 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97001021d0 0x7f9700101a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.882+0000 7f9704aa7640 1 -- 172.21.15.100:0/2575742580 >> 172.21.15.100:0/2575742580 conn(0x7f97000fc1d0 msgr2=0x7f9700102b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:42:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.882+0000 7f9704aa7640 1 -- 172.21.15.100:0/2575742580 shutdown_connections 2024-05-10T11:42:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:41.883+0000 7f9704aa7640 1 -- 172.21.15.100:0/2575742580 wait complete. 2024-05-10T11:42:41.959 INFO:tasks.workunit.client.0.smithi100.stdout: 10 105560 8.93 MB/sec execute 515 sec latency 132.060 ms 2024-05-10T11:42:42.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:42:41 smithi178 ceph-mon[33359]: pgmap v1150: 129 pgs: 129 active+clean; 582 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 4.4 MiB/s wr, 271 op/s 2024-05-10T11:42:42.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:42:41 smithi188 ceph-mon[34075]: pgmap v1150: 129 pgs: 129 active+clean; 582 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 4.4 MiB/s wr, 271 op/s 2024-05-10T11:42:42.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.208+0000 7f7df7bb5700 1 -- 172.21.15.100:0/962484853 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7df006bb40 msgr2=0x7f7df0110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:42.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.208+0000 7f7df7bb5700 1 --2- 172.21.15.100:0/962484853 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7df006bb40 0x7f7df0110330 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7de8009f90 tx=0x7f7de800b330 comp rx=0 tx=0).stop 2024-05-10T11:42:42.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.208+0000 7f7df7bb5700 1 -- 172.21.15.100:0/962484853 shutdown_connections 2024-05-10T11:42:42.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.208+0000 7f7df7bb5700 1 --2- 172.21.15.100:0/962484853 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7df006bb40 0x7f7df0110330 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:42.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.208+0000 7f7df7bb5700 1 --2- 172.21.15.100:0/962484853 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7df0109510 0x7f7df01098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:42.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.208+0000 7f7df7bb5700 1 --2- 172.21.15.100:0/962484853 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7df006b180 0x7f7df006b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:42.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.208+0000 7f7df7bb5700 1 -- 172.21.15.100:0/962484853 >> 172.21.15.100:0/962484853 conn(0x7f7df0079350 msgr2=0x7f7df0079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:42:42.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.208+0000 7f7df7bb5700 1 -- 172.21.15.100:0/962484853 shutdown_connections 2024-05-10T11:42:42.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.208+0000 7f7df7bb5700 1 -- 172.21.15.100:0/962484853 wait complete. 2024-05-10T11:42:42.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.209+0000 7f7df7bb5700 1 Processor -- start 2024-05-10T11:42:42.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.209+0000 7f7df7bb5700 1 -- start start 2024-05-10T11:42:42.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.210+0000 7f7df7bb5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7df006b180 0x7f7df0103e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:42.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.210+0000 7f7df7bb5700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7df006bb40 0x7f7df0102420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:42.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.210+0000 7f7df7bb5700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7df0109510 0x7f7df0102960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:42.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.210+0000 7f7df7bb5700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7df01134d0 con 0x7f7df0109510 2024-05-10T11:42:42.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.210+0000 7f7df7bb5700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7df0113350 con 0x7f7df006b180 2024-05-10T11:42:42.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.210+0000 7f7df7bb5700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7df0113650 con 0x7f7df006bb40 2024-05-10T11:42:42.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.210+0000 7f7df6152700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7df0109510 0x7f7df0102960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:42.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.210+0000 7f7df6152700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7df0109510 0x7f7df0102960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50986/0 (socket says 172.21.15.100:50986) 2024-05-10T11:42:42.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.210+0000 7f7df6152700 1 -- 172.21.15.100:0/3986149318 learned_addr learned my addr 172.21.15.100:0/3986149318 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:42:42.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.210+0000 7f7df5150700 1 --2- 172.21.15.100:0/3986149318 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7df006bb40 0x7f7df0102420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:42.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.210+0000 7f7df5951700 1 --2- 172.21.15.100:0/3986149318 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7df006b180 0x7f7df0103e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:42.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.210+0000 7f7df5150700 1 -- 172.21.15.100:0/3986149318 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7df006b180 msgr2=0x7f7df0103e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:42.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.210+0000 7f7df5150700 1 --2- 172.21.15.100:0/3986149318 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7df006b180 0x7f7df0103e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:42.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.210+0000 7f7df5150700 1 -- 172.21.15.100:0/3986149318 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7df0109510 msgr2=0x7f7df0102960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:42.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.210+0000 7f7df5150700 1 --2- 172.21.15.100:0/3986149318 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7df0109510 0x7f7df0102960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:42.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.210+0000 7f7df5150700 1 -- 172.21.15.100:0/3986149318 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7df0103120 con 0x7f7df006bb40 2024-05-10T11:42:42.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.211+0000 7f7df5150700 1 --2- 172.21.15.100:0/3986149318 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7df006bb40 0x7f7df0102420 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f7de000e9f0 tx=0x7f7de000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:42:42.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.211+0000 7f7de6ffd700 1 -- 172.21.15.100:0/3986149318 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de000cbd0 con 0x7f7df006bb40 2024-05-10T11:42:42.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.211+0000 7f7de6ffd700 1 -- 172.21.15.100:0/3986149318 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7de000cd70 con 0x7f7df006bb40 2024-05-10T11:42:42.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.211+0000 7f7de6ffd700 1 -- 172.21.15.100:0/3986149318 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de0018680 con 0x7f7df006bb40 2024-05-10T11:42:42.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.211+0000 7f7df7bb5700 1 -- 172.21.15.100:0/3986149318 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7df01b09f0 con 0x7f7df006bb40 2024-05-10T11:42:42.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.212+0000 7f7df7bb5700 1 -- 172.21.15.100:0/3986149318 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7df01b0eb0 con 0x7f7df006bb40 2024-05-10T11:42:42.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.215+0000 7f7de6ffd700 1 -- 172.21.15.100:0/3986149318 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f7de0010430 con 0x7f7df006bb40 2024-05-10T11:42:42.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.216+0000 7f7de6ffd700 1 --2- 172.21.15.100:0/3986149318 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f7ddc0b2900 0x7f7ddc0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:42.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.216+0000 7f7de6ffd700 1 -- 172.21.15.100:0/3986149318 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f7de00e2660 con 0x7f7df006bb40 2024-05-10T11:42:42.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.216+0000 7f7df5951700 1 --2- 172.21.15.100:0/3986149318 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f7ddc0b2900 0x7f7ddc0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:42.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.217+0000 7f7df5951700 1 --2- 172.21.15.100:0/3986149318 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f7ddc0b2900 0x7f7ddc0b4dc0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f7dec005fd0 tx=0x7f7dec009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:42:42.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.217+0000 7f7e02448b80 1 Processor -- start 2024-05-10T11:42:42.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.217+0000 7f7e02448b80 1 -- start start 2024-05-10T11:42:42.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.217+0000 7f7e02448b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e165bc64d0 0x55e165bc68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:42.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.217+0000 7f7e02448b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e165bc6df0 0x55e165bca6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:42.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.217+0000 7f7e02448b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e165bcac80 0x55e165bcb100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:42.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.218+0000 7f7e02448b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55e165b18860 con 0x55e165bc64d0 2024-05-10T11:42:42.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.218+0000 7f7df5951700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e165bc64d0 0x55e165bc68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:42.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.218+0000 7f7df5951700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e165bc64d0 0x55e165bc68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50998/0 (socket says 172.21.15.100:50998) 2024-05-10T11:42:42.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.218+0000 7f7df5951700 1 -- 172.21.15.100:0/2271935145 learned_addr learned my addr 172.21.15.100:0/2271935145 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:42:42.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.218+0000 7f7df5150700 1 --2- 172.21.15.100:0/2271935145 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e165bc6df0 0x55e165bca6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:42.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.218+0000 7f7e02448b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e165ab77c0 con 0x55e165bc6df0 2024-05-10T11:42:42.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.218+0000 7f7e02448b80 1 -- 172.21.15.100:0/2271935145 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55e165b14820 con 0x55e165bcac80 2024-05-10T11:42:42.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.218+0000 7f7df6152700 1 --2- 172.21.15.100:0/2271935145 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e165bcac80 0x55e165bcb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:42.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.218+0000 7f7df5150700 1 -- 172.21.15.100:0/2271935145 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e165bcac80 msgr2=0x55e165bcb100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:42.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.218+0000 7f7df5150700 1 --2- 172.21.15.100:0/2271935145 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e165bcac80 0x55e165bcb100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:42.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.218+0000 7f7df5150700 1 -- 172.21.15.100:0/2271935145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e165bc64d0 msgr2=0x55e165bc68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:42.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.218+0000 7f7df5150700 1 --2- 172.21.15.100:0/2271935145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e165bc64d0 0x55e165bc68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:42.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.218+0000 7f7df5150700 1 -- 172.21.15.100:0/2271935145 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e165a658d0 con 0x55e165bc6df0 2024-05-10T11:42:42.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.218+0000 7f7df5951700 1 --2- 172.21.15.100:0/2271935145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e165bc64d0 0x55e165bc68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:42:42.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.218+0000 7f7df5150700 1 --2- 172.21.15.100:0/2271935145 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e165bc6df0 0x55e165bca6f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7de0000c00 tx=0x7f7de00a9090 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5588d6fa3f91b23f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:42:42.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.219+0000 7f7ddbfff700 1 -- 172.21.15.100:0/2271935145 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de00a9430 con 0x55e165bc6df0 2024-05-10T11:42:42.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.219+0000 7f7ddbfff700 1 -- 172.21.15.100:0/2271935145 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7de00aa020 con 0x55e165bc6df0 2024-05-10T11:42:42.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.219+0000 7f7ddbfff700 1 -- 172.21.15.100:0/2271935145 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de00cc0d0 con 0x55e165bc6df0 2024-05-10T11:42:42.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.220+0000 7f7e02448b80 1 -- 172.21.15.100:0/2271935145 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e165bc6df0 msgr2=0x55e165bca6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:42.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.220+0000 7f7e02448b80 1 --2- 172.21.15.100:0/2271935145 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e165bc6df0 0x55e165bca6f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7de0000c00 tx=0x7f7de00a9090 comp rx=0 tx=0).stop 2024-05-10T11:42:42.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.220+0000 7f7e02448b80 1 -- 172.21.15.100:0/2271935145 shutdown_connections 2024-05-10T11:42:42.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.220+0000 7f7e02448b80 1 --2- 172.21.15.100:0/2271935145 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e165bcac80 0x55e165bcb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:42.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.220+0000 7f7e02448b80 1 --2- 172.21.15.100:0/2271935145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e165bc64d0 0x55e165bc68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:42.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.220+0000 7f7e02448b80 1 --2- 172.21.15.100:0/2271935145 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e165bc6df0 0x55e165bca6f0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:42.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.220+0000 7f7e02448b80 1 -- 172.21.15.100:0/2271935145 >> 172.21.15.100:0/2271935145 conn(0x55e165af2100 msgr2=0x55e165bc0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:42:42.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.220+0000 7f7e02448b80 1 -- 172.21.15.100:0/2271935145 shutdown_connections 2024-05-10T11:42:42.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.220+0000 7f7e02448b80 1 -- 172.21.15.100:0/2271935145 wait complete. 2024-05-10T11:42:42.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.220+0000 7f7e02448b80 1 -- start start 2024-05-10T11:42:42.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.220+0000 7f7e02448b80 1 Processor -- start 2024-05-10T11:42:42.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.221+0000 7f7e02448b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e165bc64d0 0x55e165bd2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:42.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.221+0000 7f7e02448b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e165bc6df0 0x55e165bcf490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:42.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.221+0000 7f7e02448b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e165bcac80 0x55e165bcf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:42.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.221+0000 7f7e02448b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55e165bd2e30 con 0x55e165bc6df0 2024-05-10T11:42:42.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.221+0000 7f7e02448b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e165a7ec30 con 0x55e165bc64d0 2024-05-10T11:42:42.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.221+0000 7f7e02448b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55e165bd2fb0 con 0x55e165bcac80 2024-05-10T11:42:42.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.221+0000 7f7df5951700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e165bc64d0 0x55e165bd2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:42.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.221+0000 7f7e02448b80 10 client.0 _ll_register_callbacks cb 0x55e165ab0920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:42:42.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.221+0000 7f7df5951700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e165bc64d0 0x55e165bd2080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:44860/0 (socket says 172.21.15.100:44860) 2024-05-10T11:42:42.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.221+0000 7f7df5951700 1 -- 172.21.15.100:0/1477235352 learned_addr learned my addr 172.21.15.100:0/1477235352 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:42:42.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.221+0000 7f7df6152700 1 --2- 172.21.15.100:0/1477235352 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e165bcac80 0x55e165bcf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:42.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.221+0000 7f7df5150700 1 --2- 172.21.15.100:0/1477235352 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e165bc6df0 0x55e165bcf490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:42.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.222+0000 7f7df5951700 1 -- 172.21.15.100:0/1477235352 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e165bcac80 msgr2=0x55e165bcf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:42.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.222+0000 7f7df5951700 1 --2- 172.21.15.100:0/1477235352 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e165bcac80 0x55e165bcf9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:42.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.222+0000 7f7df5951700 1 -- 172.21.15.100:0/1477235352 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e165bc6df0 msgr2=0x55e165bcf490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:42.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.222+0000 7f7df5951700 1 --2- 172.21.15.100:0/1477235352 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e165bc6df0 0x55e165bcf490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:42.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.222+0000 7f7df5951700 1 -- 172.21.15.100:0/1477235352 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e165ab4690 con 0x55e165bc64d0 2024-05-10T11:42:42.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.222+0000 7f7df5951700 1 --2- 172.21.15.100:0/1477235352 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e165bc64d0 0x55e165bd2080 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7dec0124a0 tx=0x7f7dec004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c197fd1b3c7751df server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:42:42.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.222+0000 7f7dd97fa700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:42:42.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.222+0000 7f7dd97fa700 1 -- 172.21.15.100:0/1477235352 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dec029670 con 0x55e165bc64d0 2024-05-10T11:42:42.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.222+0000 7f7e02448b80 1 -- 172.21.15.100:0/1477235352 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7de00aa020 con 0x55e165bc64d0 2024-05-10T11:42:42.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.222+0000 7f7dd97fa700 1 -- 172.21.15.100:0/1477235352 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f7dec029810 con 0x55e165bc64d0 2024-05-10T11:42:42.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.222+0000 7f7dd97fa700 1 -- 172.21.15.100:0/1477235352 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f7dec012e50 con 0x55e165bc64d0 2024-05-10T11:42:42.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.222+0000 7f7e02448b80 10 client.29315 fetch_fsmap learned FSMap version 41 2024-05-10T11:42:42.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.223+0000 7f7e02448b80 1 -- 172.21.15.100:0/1477235352 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55e165bdd780 con 0x55e165bc64d0 2024-05-10T11:42:42.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.223+0000 7f7dd97fa700 1 -- 172.21.15.100:0/1477235352 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7de00aa020 con 0x55e165bc64d0 2024-05-10T11:42:42.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.223+0000 7f7dd97fa700 1 -- 172.21.15.100:0/1477235352 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f7dec00f040 con 0x55e165bc64d0 2024-05-10T11:42:42.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.223+0000 7f7e02448b80 10 client.29315 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:42:42.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.223+0000 7f7e02448b80 10 client.29315 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:42:42.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.223+0000 7f7e02448b80 20 client.29315 populate_metadata read hostname 'smithi100' 2024-05-10T11:42:42.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.223+0000 7f7e02448b80 1 --2- 172.21.15.100:0/1477235352 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55e165bdfd10 0x55e165be2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:42.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.223+0000 7f7e02448b80 4 client.29315 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:42:42.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.223+0000 7f7e02448b80 1 -- 172.21.15.100:0/1477235352 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e165b92650 con 0x55e165bdfd10 2024-05-10T11:42:42.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.223+0000 7f7df5150700 1 --2- 172.21.15.100:0/1477235352 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55e165bdfd10 0x55e165be2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:42.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.224+0000 7f7dd97fa700 10 client.29315 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:42:42.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.224+0000 7f7df5150700 1 --2- 172.21.15.100:0/1477235352 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55e165bdfd10 0x55e165be2100 crc :-1 s=READY pgs=453 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4a07de196d23246 server_cookie=2f015418c46bcdbe in_seq=0 out_seq=0 2024-05-10T11:42:42.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.225+0000 7f7dd97fa700 1 -- 172.21.15.100:0/1477235352 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e165b92650 con 0x55e165bdfd10 2024-05-10T11:42:42.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.225+0000 7f7dd97fa700 10 client.29315 handle_command_reply: tid=0 2024-05-10T11:42:42.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.225+0000 7f7e02448b80 1 client.29315 shutdown 2024-05-10T11:42:42.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.225+0000 7f7e02448b80 1 -- 172.21.15.100:0/1477235352 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e165bc64d0 msgr2=0x55e165bd2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:42.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.225+0000 7f7e02448b80 1 --2- 172.21.15.100:0/1477235352 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e165bc64d0 0x55e165bd2080 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7dec0124a0 tx=0x7f7dec004750 comp rx=0 tx=0).stop 2024-05-10T11:42:42.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.226+0000 7f7e02448b80 1 -- 172.21.15.100:0/1477235352 shutdown_connections 2024-05-10T11:42:42.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.226+0000 7f7e02448b80 1 --2- 172.21.15.100:0/1477235352 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55e165bdfd10 0x55e165be2100 crc :-1 s=READY pgs=453 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:42.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.226+0000 7f7e02448b80 1 --2- 172.21.15.100:0/1477235352 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e165bcac80 0x55e165bcf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:42.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.226+0000 7f7e02448b80 1 --2- 172.21.15.100:0/1477235352 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e165bc64d0 0x55e165bd2080 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:42.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.226+0000 7f7dd97fa700 0 client.29315 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:42:42.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.226+0000 7f7e02448b80 1 --2- 172.21.15.100:0/1477235352 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e165bc6df0 0x55e165bcf490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:42.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.226+0000 7f7e02448b80 1 -- 172.21.15.100:0/1477235352 >> 172.21.15.100:0/1477235352 conn(0x55e165af2100 msgr2=0x55e165bbb720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:42:42.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.226+0000 7f7e02448b80 1 -- 172.21.15.100:0/1477235352 shutdown_connections 2024-05-10T11:42:42.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.226+0000 7f7e02448b80 1 -- 172.21.15.100:0/1477235352 wait complete. 2024-05-10T11:42:42.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.226+0000 7f7e02448b80 20 client.29315 trim_cache size 0 max 16384 2024-05-10T11:42:42.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.252+0000 7f7e02448b80 1 Processor -- start 2024-05-10T11:42:42.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.252+0000 7f7e02448b80 1 -- start start 2024-05-10T11:42:42.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.252+0000 7f7e02448b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e165bc64d0 0x55e165bdbec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:42.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.252+0000 7f7e02448b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e165bc6df0 0x55e165bc59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:42.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.252+0000 7f7e02448b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e165bcac80 0x55e165be79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:42.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.252+0000 7f7e02448b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7df01bc430 con 0x55e165bcac80 2024-05-10T11:42:42.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.252+0000 7f7e02448b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7df01b9cc0 con 0x55e165bc6df0 2024-05-10T11:42:42.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.252+0000 7f7e02448b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7df01b7b10 con 0x55e165bc64d0 2024-05-10T11:42:42.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.252+0000 7f7df5150700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e165bc6df0 0x55e165bc59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:42.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.252+0000 7f7df6152700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e165bcac80 0x55e165be79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:42.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.253+0000 7f7df5951700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e165bc64d0 0x55e165bdbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:42.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.253+0000 7f7df5150700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e165bc6df0 0x55e165bc59f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:44872/0 (socket says 172.21.15.100:44872) 2024-05-10T11:42:42.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.253+0000 7f7df5150700 1 -- 172.21.15.100:0/3328170377 learned_addr learned my addr 172.21.15.100:0/3328170377 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:42:42.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.253+0000 7f7df5150700 1 -- 172.21.15.100:0/3328170377 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e165bc64d0 msgr2=0x55e165bdbec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:42.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.253+0000 7f7df5150700 1 --2- 172.21.15.100:0/3328170377 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e165bc64d0 0x55e165bdbec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:42.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.253+0000 7f7df5150700 1 -- 172.21.15.100:0/3328170377 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e165bcac80 msgr2=0x55e165be79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:42.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.253+0000 7f7df5150700 1 --2- 172.21.15.100:0/3328170377 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e165bcac80 0x55e165be79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:42.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.253+0000 7f7df5150700 1 -- 172.21.15.100:0/3328170377 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e165bd0480 con 0x55e165bc6df0 2024-05-10T11:42:42.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.253+0000 7f7df5150700 1 --2- 172.21.15.100:0/3328170377 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e165bc6df0 0x55e165bc59f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f7de0000c00 tx=0x7f7de0091040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a32d181d3e7db2b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:42:42.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.253+0000 7f7ddb7fe700 1 -- 172.21.15.100:0/3328170377 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de008e070 con 0x55e165bc6df0 2024-05-10T11:42:42.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.253+0000 7f7ddb7fe700 1 -- 172.21.15.100:0/3328170377 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7de00aa020 con 0x55e165bc6df0 2024-05-10T11:42:42.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.253+0000 7f7ddb7fe700 1 -- 172.21.15.100:0/3328170377 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de00cc0d0 con 0x55e165bc6df0 2024-05-10T11:42:42.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.254+0000 7f7e02448b80 1 -- 172.21.15.100:0/3328170377 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e165bc6df0 msgr2=0x55e165bc59f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:42.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.254+0000 7f7e02448b80 1 --2- 172.21.15.100:0/3328170377 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e165bc6df0 0x55e165bc59f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f7de0000c00 tx=0x7f7de0091040 comp rx=0 tx=0).stop 2024-05-10T11:42:42.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.254+0000 7f7e02448b80 1 -- 172.21.15.100:0/3328170377 shutdown_connections 2024-05-10T11:42:42.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.254+0000 7f7e02448b80 1 --2- 172.21.15.100:0/3328170377 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e165bcac80 0x55e165be79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:42.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.254+0000 7f7e02448b80 1 --2- 172.21.15.100:0/3328170377 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e165bc64d0 0x55e165bdbec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:42.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.254+0000 7f7e02448b80 1 --2- 172.21.15.100:0/3328170377 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e165bc6df0 0x55e165bc59f0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:42.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.254+0000 7f7e02448b80 1 -- 172.21.15.100:0/3328170377 >> 172.21.15.100:0/3328170377 conn(0x55e165af2100 msgr2=0x55e165bc52e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:42:42.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.254+0000 7f7e02448b80 1 -- 172.21.15.100:0/3328170377 shutdown_connections 2024-05-10T11:42:42.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.254+0000 7f7e02448b80 1 -- 172.21.15.100:0/3328170377 wait complete. 2024-05-10T11:42:42.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.255+0000 7f7e02448b80 1 -- start start 2024-05-10T11:42:42.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.255+0000 7f7e02448b80 1 Processor -- start 2024-05-10T11:42:42.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.255+0000 7f7e02448b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e165bc64d0 0x55e165bd80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:42.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.256+0000 7f7e02448b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e165bc6df0 0x55e165bd85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:42.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.256+0000 7f7e02448b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e165bcac80 0x55e165bd55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:42.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.256+0000 7f7e02448b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7df01b8180 con 0x55e165bcac80 2024-05-10T11:42:42.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.256+0000 7f7e02448b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7df01b95f0 con 0x55e165bc64d0 2024-05-10T11:42:42.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.256+0000 7f7e02448b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7df01b8850 con 0x55e165bc6df0 2024-05-10T11:42:42.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.256+0000 7f7e02448b80 10 client.0 _ll_register_callbacks cb 0x55e165ab0920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:42:42.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.256+0000 7f7df5150700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e165bc6df0 0x55e165bd85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:42.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.256+0000 7f7df5951700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e165bc64d0 0x55e165bd80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:42.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.256+0000 7f7df5150700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e165bc6df0 0x55e165bd85e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:48202/0 (socket says 172.21.15.100:48202) 2024-05-10T11:42:42.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.256+0000 7f7df5150700 1 -- 172.21.15.100:0/3219976230 learned_addr learned my addr 172.21.15.100:0/3219976230 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:42:42.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.256+0000 7f7df6152700 1 --2- 172.21.15.100:0/3219976230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e165bcac80 0x55e165bd55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:42.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.256+0000 7f7df5951700 1 -- 172.21.15.100:0/3219976230 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e165bc6df0 msgr2=0x55e165bd85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:42.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.256+0000 7f7df5951700 1 --2- 172.21.15.100:0/3219976230 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e165bc6df0 0x55e165bd85e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:42.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.256+0000 7f7df5951700 1 -- 172.21.15.100:0/3219976230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e165bcac80 msgr2=0x55e165bd55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:42.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.256+0000 7f7df5951700 1 --2- 172.21.15.100:0/3219976230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e165bcac80 0x55e165bd55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:42.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.256+0000 7f7df5951700 1 -- 172.21.15.100:0/3219976230 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e165bd5cc0 con 0x55e165bc64d0 2024-05-10T11:42:42.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.256+0000 7f7df6152700 1 --2- 172.21.15.100:0/3219976230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e165bcac80 0x55e165bd55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:42:42.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.256+0000 7f7df5951700 1 --2- 172.21.15.100:0/3219976230 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e165bc64d0 0x55e165bd80a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7dec015040 tx=0x7f7dec005260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b220b7343b5d48f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:42:42.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.256+0000 7f7dd97fa700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:42:42.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.256+0000 7f7dd97fa700 1 -- 172.21.15.100:0/3219976230 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dec03a070 con 0x55e165bc64d0 2024-05-10T11:42:42.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.257+0000 7f7e02448b80 1 -- 172.21.15.100:0/3219976230 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7de00aa020 con 0x55e165bc64d0 2024-05-10T11:42:42.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.257+0000 7f7dd97fa700 1 -- 172.21.15.100:0/3219976230 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f7dec029a10 con 0x55e165bc64d0 2024-05-10T11:42:42.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.257+0000 7f7dd97fa700 1 -- 172.21.15.100:0/3219976230 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f7dec046050 con 0x55e165bc64d0 2024-05-10T11:42:42.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.257+0000 7f7e02448b80 10 client.29321 fetch_fsmap learned FSMap version 41 2024-05-10T11:42:42.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.257+0000 7f7e02448b80 1 -- 172.21.15.100:0/3219976230 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55e165c52870 con 0x55e165bc64d0 2024-05-10T11:42:42.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.257+0000 7f7dd97fa700 1 -- 172.21.15.100:0/3219976230 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7de00aa020 con 0x55e165bc64d0 2024-05-10T11:42:42.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.257+0000 7f7dd97fa700 1 -- 172.21.15.100:0/3219976230 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f7dec0209e0 con 0x55e165bc64d0 2024-05-10T11:42:42.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.257+0000 7f7e02448b80 10 client.29321 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:42:42.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.257+0000 7f7e02448b80 10 client.29321 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:42:42.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.257+0000 7f7e02448b80 20 client.29321 populate_metadata read hostname 'smithi100' 2024-05-10T11:42:42.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.257+0000 7f7e02448b80 1 --2- 172.21.15.100:0/3219976230 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55e165c52e30 0x55e165c55250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:42.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.257+0000 7f7e02448b80 4 client.29321 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T11:42:42.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.257+0000 7f7e02448b80 1 -- 172.21.15.100:0/3219976230 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e165b26240 con 0x55e165c52e30 2024-05-10T11:42:42.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.258+0000 7f7df5150700 1 --2- 172.21.15.100:0/3219976230 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55e165c52e30 0x55e165c55250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:42.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.259+0000 7f7df5150700 1 --2- 172.21.15.100:0/3219976230 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55e165c52e30 0x55e165c55250 crc :-1 s=READY pgs=454 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fffb2d9086f037a0 server_cookie=3ebc347bf143637e in_seq=0 out_seq=0 2024-05-10T11:42:42.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.259+0000 7f7dd97fa700 10 client.29321 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:42:42.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.260+0000 7f7dd97fa700 1 -- 172.21.15.100:0/3219976230 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e165b26240 con 0x55e165c52e30 2024-05-10T11:42:42.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.260+0000 7f7dd97fa700 10 client.29321 handle_command_reply: tid=0 2024-05-10T11:42:42.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.260+0000 7f7e02448b80 1 client.29321 shutdown 2024-05-10T11:42:42.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.261+0000 7f7e02448b80 1 -- 172.21.15.100:0/3219976230 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e165bc64d0 msgr2=0x55e165bd80a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:42.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.261+0000 7f7e02448b80 1 --2- 172.21.15.100:0/3219976230 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e165bc64d0 0x55e165bd80a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7dec015040 tx=0x7f7dec005260 comp rx=0 tx=0).stop 2024-05-10T11:42:42.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.261+0000 7f7e02448b80 1 -- 172.21.15.100:0/3219976230 shutdown_connections 2024-05-10T11:42:42.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.261+0000 7f7e02448b80 1 --2- 172.21.15.100:0/3219976230 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55e165c52e30 0x55e165c55250 crc :-1 s=READY pgs=454 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:42.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.261+0000 7f7e02448b80 1 --2- 172.21.15.100:0/3219976230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e165bcac80 0x55e165bd55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:42.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.261+0000 7f7e02448b80 1 --2- 172.21.15.100:0/3219976230 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e165bc64d0 0x55e165bd80a0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:42.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.261+0000 7f7e02448b80 1 --2- 172.21.15.100:0/3219976230 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e165bc6df0 0x55e165bd85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:42.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.261+0000 7f7dd97fa700 0 client.29321 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:42:42.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.261+0000 7f7e02448b80 1 -- 172.21.15.100:0/3219976230 >> 172.21.15.100:0/3219976230 conn(0x55e165af2100 msgr2=0x55e165bc5b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:42:42.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.261+0000 7f7e02448b80 1 -- 172.21.15.100:0/3219976230 shutdown_connections 2024-05-10T11:42:42.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.261+0000 7f7e02448b80 1 -- 172.21.15.100:0/3219976230 wait complete. 2024-05-10T11:42:42.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.261+0000 7f7e02448b80 20 client.29321 trim_cache size 0 max 16384 2024-05-10T11:42:42.264 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:42:42.264 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-05-10T11:42:42.264 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-05-10T11:42:42.264 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:42:42.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.262+0000 7f7df7bb5700 1 -- 172.21.15.100:0/3986149318 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f7ddc0b2900 msgr2=0x7f7ddc0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:42.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.262+0000 7f7df7bb5700 1 --2- 172.21.15.100:0/3986149318 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f7ddc0b2900 0x7f7ddc0b4dc0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f7dec005fd0 tx=0x7f7dec009310 comp rx=0 tx=0).stop 2024-05-10T11:42:42.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.262+0000 7f7df7bb5700 1 -- 172.21.15.100:0/3986149318 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7df006bb40 msgr2=0x7f7df0102420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:42.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.262+0000 7f7df7bb5700 1 --2- 172.21.15.100:0/3986149318 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7df006bb40 0x7f7df0102420 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f7de000e9f0 tx=0x7f7de000ed00 comp rx=0 tx=0).stop 2024-05-10T11:42:42.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.262+0000 7f7df7bb5700 1 -- 172.21.15.100:0/3986149318 shutdown_connections 2024-05-10T11:42:42.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.262+0000 7f7df7bb5700 1 --2- 172.21.15.100:0/3986149318 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f7ddc0b2900 0x7f7ddc0b4dc0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:42.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.262+0000 7f7df7bb5700 1 --2- 172.21.15.100:0/3986149318 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7df0109510 0x7f7df0102960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:42.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.262+0000 7f7df7bb5700 1 --2- 172.21.15.100:0/3986149318 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7df006b180 0x7f7df0103e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:42.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.262+0000 7f7df7bb5700 1 --2- 172.21.15.100:0/3986149318 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7df006bb40 0x7f7df0102420 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:42.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.262+0000 7f7df7bb5700 1 -- 172.21.15.100:0/3986149318 >> 172.21.15.100:0/3986149318 conn(0x7f7df0079350 msgr2=0x7f7df0101220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:42:42.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.262+0000 7f7df7bb5700 1 -- 172.21.15.100:0/3986149318 shutdown_connections 2024-05-10T11:42:42.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:42.262+0000 7f7df7bb5700 1 -- 172.21.15.100:0/3986149318 wait complete. 2024-05-10T11:42:42.959 INFO:tasks.workunit.client.0.smithi100.stdout: 10 105706 8.93 MB/sec execute 516 sec latency 85.503 ms 2024-05-10T11:42:43.096 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T11:42:43.096 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T11:42:43.520 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:42:43.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:42:43 smithi100 ceph-mon[30296]: pgmap v1151: 129 pgs: 129 active+clean; 586 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 4.5 MiB/s wr, 255 op/s 2024-05-10T11:42:43.959 INFO:tasks.workunit.client.0.smithi100.stdout: 10 105867 8.93 MB/sec execute 517 sec latency 121.317 ms 2024-05-10T11:42:44.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:42:43 smithi178 ceph-mon[33359]: pgmap v1151: 129 pgs: 129 active+clean; 586 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 4.5 MiB/s wr, 255 op/s 2024-05-10T11:42:44.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:42:43 smithi188 ceph-mon[34075]: pgmap v1151: 129 pgs: 129 active+clean; 586 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 4.5 MiB/s wr, 255 op/s 2024-05-10T11:42:44.959 INFO:tasks.workunit.client.0.smithi100.stdout: 10 106021 8.92 MB/sec execute 518 sec latency 117.383 ms 2024-05-10T11:42:45.959 INFO:tasks.workunit.client.0.smithi100.stdout: 10 106177 8.92 MB/sec execute 519 sec latency 96.794 ms 2024-05-10T11:42:46.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:42:45 smithi178 ceph-mon[33359]: pgmap v1152: 129 pgs: 129 active+clean; 589 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 4.8 MiB/s wr, 266 op/s 2024-05-10T11:42:46.015 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:42:45 smithi100 ceph-mon[30296]: pgmap v1152: 129 pgs: 129 active+clean; 589 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 4.8 MiB/s wr, 266 op/s 2024-05-10T11:42:46.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:42:45 smithi188 ceph-mon[34075]: pgmap v1152: 129 pgs: 129 active+clean; 589 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 4.8 MiB/s wr, 266 op/s 2024-05-10T11:42:46.743 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:42:46.743 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2072225012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2072225012},{"type":"v1","addr":"172.21.15.100:6835","nonce":2072225012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/401939540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":401939540},{"type":"v1","addr":"172.21.15.100:6837","nonce":401939540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14571,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/3946311025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3946311025},{"type":"v1","addr":"172.21.15.100:6839","nonce":3946311025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3736371660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3736371660},{"type":"v1","addr":"172.21.15.100:6841","nonce":3736371660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/2741810791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2741810791},{"type":"v1","addr":"172.21.15.178:6833","nonce":2741810791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1704891041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1704891041},{"type":"v1","addr":"172.21.15.188:6833","nonce":1704891041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":41,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T11:14:27.242452+0000","modified":"2024-05-10T11:35:29.588072+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24439,"mds_1":24437,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/2124076312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2124076312},{"type":"v1","addr":"172.21.15.178:6835","nonce":2124076312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/3224358425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3224358425},{"type":"v1","addr":"172.21.15.178:6837","nonce":3224358425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/1943342620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1943342620},{"type":"v1","addr":"172.21.15.188:6837","nonce":1943342620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.178:6839/3760985404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3760985404},{"type":"v1","addr":"172.21.15.178:6839","nonce":3760985404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.188:6839/3194917070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":3194917070},{"type":"v1","addr":"172.21.15.188:6839","nonce":3194917070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27034":{"gid":27034,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/3139559450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3139559450},{"type":"v1","addr":"172.21.15.188:6835","nonce":3139559450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24439,"qdb_cluster":[24437,24439,24427]},"id":1}]} 2024-05-10T11:42:46.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:46.160+0000 7f930c9d5700 1 -- 172.21.15.100:0/2397615816 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9308102d60 msgr2=0x7f930810f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:46.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:46.160+0000 7f930c9d5700 1 --2- 172.21.15.100:0/2397615816 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9308102d60 0x7f930810f250 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f930000b0a0 tx=0x7f930000b3b0 comp rx=0 tx=0).stop 2024-05-10T11:42:46.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:46.161+0000 7f930c9d5700 1 -- 172.21.15.100:0/2397615816 shutdown_connections 2024-05-10T11:42:46.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:46.161+0000 7f930c9d5700 1 --2- 172.21.15.100:0/2397615816 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9308102d60 0x7f930810f250 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:46.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:46.161+0000 7f930c9d5700 1 --2- 172.21.15.100:0/2397615816 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9308108410 0x7f93081087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:46.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:46.161+0000 7f930c9d5700 1 --2- 172.21.15.100:0/2397615816 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93081023a0 0x7f9308102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:46.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:46.161+0000 7f930c9d5700 1 -- 172.21.15.100:0/2397615816 >> 172.21.15.100:0/2397615816 conn(0x7f9308079350 msgr2=0x7f9308079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:42:46.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:46.161+0000 7f930c9d5700 1 -- 172.21.15.100:0/2397615816 shutdown_connections 2024-05-10T11:42:46.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:46.161+0000 7f930c9d5700 1 -- 172.21.15.100:0/2397615816 wait complete. 2024-05-10T11:42:46.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:46.162+0000 7f930c9d5700 1 Processor -- start 2024-05-10T11:42:46.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:46.162+0000 7f930c9d5700 1 -- start start 2024-05-10T11:42:46.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:46.162+0000 7f930c9d5700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93081023a0 0x7f9308104380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:46.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:46.162+0000 7f930c9d5700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9308102d60 0x7f93081048c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:46.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:46.162+0000 7f930c9d5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9308108410 0x7f9308104e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:46.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:46.162+0000 7f930c9d5700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9308112410 con 0x7f93081023a0 2024-05-10T11:42:46.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:46.162+0000 7f930c9d5700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9308112290 con 0x7f9308108410 2024-05-10T11:42:46.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:46.162+0000 7f930c9d5700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9308112590 con 0x7f9308102d60 2024-05-10T11:42:46.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:46.162+0000 7f9305d9b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9308102d60 0x7f93081048c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:46.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:46.162+0000 7f930659c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93081023a0 0x7f9308104380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:46.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:46.162+0000 7f9305d9b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9308102d60 0x7f93081048c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:48216/0 (socket says 172.21.15.100:48216) 2024-05-10T11:42:46.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:46.162+0000 7f9305d9b700 1 -- 172.21.15.100:0/3944774901 learned_addr learned my addr 172.21.15.100:0/3944774901 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:42:46.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:46.163+0000 7f9306d9d700 1 --2- 172.21.15.100:0/3944774901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9308108410 0x7f9308104e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:46.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:46.163+0000 7f9305d9b700 1 -- 172.21.15.100:0/3944774901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9308108410 msgr2=0x7f9308104e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:46.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:46.163+0000 7f9305d9b700 1 --2- 172.21.15.100:0/3944774901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9308108410 0x7f9308104e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:46.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:46.163+0000 7f9305d9b700 1 -- 172.21.15.100:0/3944774901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93081023a0 msgr2=0x7f9308104380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:46.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:46.163+0000 7f9305d9b700 1 --2- 172.21.15.100:0/3944774901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93081023a0 0x7f9308104380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:46.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:46.163+0000 7f9305d9b700 1 -- 172.21.15.100:0/3944774901 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93081ac3c0 con 0x7f9308102d60 2024-05-10T11:42:46.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:46.163+0000 7f9305d9b700 1 --2- 172.21.15.100:0/3944774901 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9308102d60 0x7f93081048c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f92fc00e9c0 tx=0x7f92fc00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:42:46.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:46.163+0000 7f92f77fe700 1 -- 172.21.15.100:0/3944774901 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92fc00cbd0 con 0x7f9308102d60 2024-05-10T11:42:46.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:46.163+0000 7f930c9d5700 1 -- 172.21.15.100:0/3944774901 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93081ac5f0 con 0x7f9308102d60 2024-05-10T11:42:46.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:46.163+0000 7f92f77fe700 1 -- 172.21.15.100:0/3944774901 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f92fc00ee60 con 0x7f9308102d60 2024-05-10T11:42:46.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:46.163+0000 7f930c9d5700 1 -- 172.21.15.100:0/3944774901 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93081acb30 con 0x7f9308102d60 2024-05-10T11:42:46.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:46.164+0000 7f92f77fe700 1 -- 172.21.15.100:0/3944774901 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92fc0054d0 con 0x7f9308102d60 2024-05-10T11:42:46.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:46.165+0000 7f930c9d5700 1 -- 172.21.15.100:0/3944774901 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92e8005330 con 0x7f9308102d60 2024-05-10T11:42:46.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:46.167+0000 7f92f77fe700 1 -- 172.21.15.100:0/3944774901 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f92fc048a00 con 0x7f9308102d60 2024-05-10T11:42:46.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:46.168+0000 7f92f77fe700 1 --2- 172.21.15.100:0/3944774901 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f92f00b2880 0x7f92f00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:46.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:46.168+0000 7f930659c700 1 --2- 172.21.15.100:0/3944774901 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f92f00b2880 0x7f92f00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:46.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:46.168+0000 7f92f77fe700 1 -- 172.21.15.100:0/3944774901 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f92fc0e2af0 con 0x7f9308102d60 2024-05-10T11:42:46.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:46.169+0000 7f930659c700 1 --2- 172.21.15.100:0/3944774901 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f92f00b2880 0x7f92f00b4d40 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f92f8009680 tx=0x7f92f8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:42:46.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:46.172+0000 7f92f77fe700 1 -- 172.21.15.100:0/3944774901 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f92fc0aa4b0 con 0x7f9308102d60 2024-05-10T11:42:46.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:46.626+0000 7f930c9d5700 1 -- 172.21.15.100:0/3944774901 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f92e8005620 con 0x7f9308102d60 2024-05-10T11:42:46.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:46.627+0000 7f92f77fe700 1 -- 172.21.15.100:0/3944774901 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+11897 (secure 0 0 0) 0x7f92fc0a9db0 con 0x7f9308102d60 2024-05-10T11:42:46.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:46.630+0000 7f930c9d5700 1 -- 172.21.15.100:0/3944774901 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f92f00b2880 msgr2=0x7f92f00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:46.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:46.630+0000 7f930c9d5700 1 --2- 172.21.15.100:0/3944774901 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f92f00b2880 0x7f92f00b4d40 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f92f8009680 tx=0x7f92f8009260 comp rx=0 tx=0).stop 2024-05-10T11:42:46.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:46.630+0000 7f930c9d5700 1 -- 172.21.15.100:0/3944774901 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9308102d60 msgr2=0x7f93081048c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:46.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:46.630+0000 7f930c9d5700 1 --2- 172.21.15.100:0/3944774901 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9308102d60 0x7f93081048c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f92fc00e9c0 tx=0x7f92fc00ecd0 comp rx=0 tx=0).stop 2024-05-10T11:42:46.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:46.631+0000 7f930c9d5700 1 -- 172.21.15.100:0/3944774901 shutdown_connections 2024-05-10T11:42:46.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:46.631+0000 7f930c9d5700 1 --2- 172.21.15.100:0/3944774901 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f92f00b2880 0x7f92f00b4d40 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:46.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:46.631+0000 7f930c9d5700 1 --2- 172.21.15.100:0/3944774901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9308108410 0x7f9308104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:46.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:46.631+0000 7f930c9d5700 1 --2- 172.21.15.100:0/3944774901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93081023a0 0x7f9308104380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:46.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:46.631+0000 7f930c9d5700 1 --2- 172.21.15.100:0/3944774901 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9308102d60 0x7f93081048c0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:46.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:46.631+0000 7f930c9d5700 1 -- 172.21.15.100:0/3944774901 >> 172.21.15.100:0/3944774901 conn(0x7f9308079350 msgr2=0x7f9308106620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:42:46.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:46.631+0000 7f930c9d5700 1 -- 172.21.15.100:0/3944774901 shutdown_connections 2024-05-10T11:42:46.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:46.631+0000 7f930c9d5700 1 -- 172.21.15.100:0/3944774901 wait complete. 2024-05-10T11:42:46.749 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 41 2024-05-10T11:42:46.959 INFO:tasks.workunit.client.0.smithi100.stdout: 10 106326 8.92 MB/sec execute 520 sec latency 97.765 ms 2024-05-10T11:42:47.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:42:46 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/3944774901' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:42:47.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:42:46 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/3944774901' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:42:47.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:42:46 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/3944774901' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:42:47.674 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-10T11:42:47.959 INFO:tasks.workunit.client.0.smithi100.stdout: 10 106481 8.92 MB/sec execute 521 sec latency 68.451 ms 2024-05-10T11:42:48.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:42:47 smithi188 ceph-mon[34075]: pgmap v1153: 129 pgs: 129 active+clean; 583 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 4.8 MiB/s wr, 268 op/s 2024-05-10T11:42:48.036 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:42:48.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:42:47 smithi100 ceph-mon[30296]: pgmap v1153: 129 pgs: 129 active+clean; 583 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 4.8 MiB/s wr, 268 op/s 2024-05-10T11:42:48.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:42:47 smithi178 ceph-mon[33359]: pgmap v1153: 129 pgs: 129 active+clean; 583 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 4.8 MiB/s wr, 268 op/s 2024-05-10T11:42:48.959 INFO:tasks.workunit.client.0.smithi100.stdout: 10 106666 8.93 MB/sec execute 522 sec latency 68.068 ms 2024-05-10T11:42:49.959 INFO:tasks.workunit.client.0.smithi100.stdout: 10 106798 8.92 MB/sec execute 523 sec latency 86.511 ms 2024-05-10T11:42:50.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:42:49 smithi188 ceph-mon[34075]: pgmap v1154: 129 pgs: 129 active+clean; 571 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 4.3 MiB/s wr, 268 op/s 2024-05-10T11:42:50.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:42:49 smithi100 ceph-mon[30296]: pgmap v1154: 129 pgs: 129 active+clean; 571 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 4.3 MiB/s wr, 268 op/s 2024-05-10T11:42:50.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:42:49 smithi178 ceph-mon[33359]: pgmap v1154: 129 pgs: 129 active+clean; 571 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 4.3 MiB/s wr, 268 op/s 2024-05-10T11:42:50.959 INFO:tasks.workunit.client.0.smithi100.stdout: 10 106931 8.90 MB/sec execute 524 sec latency 130.450 ms 2024-05-10T11:42:51.960 INFO:tasks.workunit.client.0.smithi100.stdout: 10 107095 8.90 MB/sec execute 525 sec latency 145.488 ms 2024-05-10T11:42:52.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:42:51 smithi188 ceph-mon[34075]: pgmap v1155: 129 pgs: 129 active+clean; 565 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 4.0 MiB/s wr, 283 op/s 2024-05-10T11:42:52.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:42:51 smithi100 ceph-mon[30296]: pgmap v1155: 129 pgs: 129 active+clean; 565 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 4.0 MiB/s wr, 283 op/s 2024-05-10T11:42:52.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:42:51 smithi178 ceph-mon[33359]: pgmap v1155: 129 pgs: 129 active+clean; 565 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 4.0 MiB/s wr, 283 op/s 2024-05-10T11:42:52.960 INFO:tasks.workunit.client.0.smithi100.stdout: 10 107256 8.90 MB/sec execute 526 sec latency 121.285 ms 2024-05-10T11:42:53.960 INFO:tasks.workunit.client.0.smithi100.stdout: 10 107364 8.89 MB/sec execute 527 sec latency 191.014 ms 2024-05-10T11:42:54.027 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:42:53 smithi100 ceph-mon[30296]: pgmap v1156: 129 pgs: 129 active+clean; 567 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 908 KiB/s rd, 3.7 MiB/s wr, 293 op/s 2024-05-10T11:42:54.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:42:53 smithi188 ceph-mon[34075]: pgmap v1156: 129 pgs: 129 active+clean; 567 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 908 KiB/s rd, 3.7 MiB/s wr, 293 op/s 2024-05-10T11:42:54.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:42:53 smithi178 ceph-mon[33359]: pgmap v1156: 129 pgs: 129 active+clean; 567 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 908 KiB/s rd, 3.7 MiB/s wr, 293 op/s 2024-05-10T11:42:54.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.175+0000 7f6f29511700 1 -- 172.21.15.100:0/653749911 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f24103390 msgr2=0x7f6f24107000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:54.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.175+0000 7f6f29511700 1 --2- 172.21.15.100:0/653749911 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f24103390 0x7f6f24107000 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f6f18009920 tx=0x7f6f18009c30 comp rx=0 tx=0).stop 2024-05-10T11:42:54.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.175+0000 7f6f29511700 1 -- 172.21.15.100:0/653749911 shutdown_connections 2024-05-10T11:42:54.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.175+0000 7f6f29511700 1 --2- 172.21.15.100:0/653749911 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f24107730 0x7f6f2410f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:54.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.175+0000 7f6f29511700 1 --2- 172.21.15.100:0/653749911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f241029e0 0x7f6f24102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:54.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.175+0000 7f6f29511700 1 --2- 172.21.15.100:0/653749911 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f24103390 0x7f6f24107000 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:54.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.175+0000 7f6f29511700 1 -- 172.21.15.100:0/653749911 >> 172.21.15.100:0/653749911 conn(0x7f6f24079350 msgr2=0x7f6f24079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:42:54.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.175+0000 7f6f29511700 1 -- 172.21.15.100:0/653749911 shutdown_connections 2024-05-10T11:42:54.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.176+0000 7f6f29511700 1 -- 172.21.15.100:0/653749911 wait complete. 2024-05-10T11:42:54.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.177+0000 7f6f29511700 1 Processor -- start 2024-05-10T11:42:54.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.177+0000 7f6f29511700 1 -- start start 2024-05-10T11:42:54.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.177+0000 7f6f29511700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f241029e0 0x7f6f2419bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:54.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.177+0000 7f6f29511700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f24103390 0x7f6f2419c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:54.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.178+0000 7f6f29511700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f24107730 0x7f6f241a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:54.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.178+0000 7f6f29511700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6f24112450 con 0x7f6f24103390 2024-05-10T11:42:54.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.178+0000 7f6f29511700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6f241122d0 con 0x7f6f24107730 2024-05-10T11:42:54.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.178+0000 7f6f237fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f24107730 0x7f6f241a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:54.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.178+0000 7f6f237fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f24107730 0x7f6f241a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:32956/0 (socket says 172.21.15.100:32956) 2024-05-10T11:42:54.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.178+0000 7f6f237fe700 1 -- 172.21.15.100:0/2868353960 learned_addr learned my addr 172.21.15.100:0/2868353960 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:42:54.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.178+0000 7f6f29511700 1 -- 172.21.15.100:0/2868353960 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6f241125d0 con 0x7f6f241029e0 2024-05-10T11:42:54.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.178+0000 7f6f22ffd700 1 --2- 172.21.15.100:0/2868353960 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f241029e0 0x7f6f2419bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:54.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.179+0000 7f6f237fe700 1 -- 172.21.15.100:0/2868353960 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f241029e0 msgr2=0x7f6f2419bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:54.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.179+0000 7f6f237fe700 1 --2- 172.21.15.100:0/2868353960 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f241029e0 0x7f6f2419bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:54.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.179+0000 7f6f237fe700 1 -- 172.21.15.100:0/2868353960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f24103390 msgr2=0x7f6f2419c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:54.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.179+0000 7f6f237fe700 1 --2- 172.21.15.100:0/2868353960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f24103390 0x7f6f2419c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:54.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.179+0000 7f6f237fe700 1 -- 172.21.15.100:0/2868353960 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f241a0db0 con 0x7f6f24107730 2024-05-10T11:42:54.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.179+0000 7f6f237fe700 1 --2- 172.21.15.100:0/2868353960 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f24107730 0x7f6f241a0670 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c00c370 tx=0x7f6f1c00c730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:42:54.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.179+0000 7f6f0bfff700 1 -- 172.21.15.100:0/2868353960 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f1c0090d0 con 0x7f6f24107730 2024-05-10T11:42:54.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.180+0000 7f6f0bfff700 1 -- 172.21.15.100:0/2868353960 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6f1c009270 con 0x7f6f24107730 2024-05-10T11:42:54.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.180+0000 7f6f0bfff700 1 -- 172.21.15.100:0/2868353960 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f1c004790 con 0x7f6f24107730 2024-05-10T11:42:54.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.180+0000 7f6f29511700 1 -- 172.21.15.100:0/2868353960 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f241a1040 con 0x7f6f24107730 2024-05-10T11:42:54.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.180+0000 7f6f29511700 1 -- 172.21.15.100:0/2868353960 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f241a8920 con 0x7f6f24107730 2024-05-10T11:42:54.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.184+0000 7f6f0bfff700 1 -- 172.21.15.100:0/2868353960 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f6f1c01a020 con 0x7f6f24107730 2024-05-10T11:42:54.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.186+0000 7f6f0bfff700 1 --2- 172.21.15.100:0/2868353960 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f6f0c0b2900 0x7f6f0c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:54.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.186+0000 7f6f33da4b80 1 Processor -- start 2024-05-10T11:42:54.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.186+0000 7f6f22ffd700 1 --2- 172.21.15.100:0/2868353960 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f6f0c0b2900 0x7f6f0c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:54.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.186+0000 7f6f33da4b80 1 -- start start 2024-05-10T11:42:54.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.186+0000 7f6f33da4b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dcac4814d0 0x55dcac4818b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:54.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.187+0000 7f6f33da4b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dcac481df0 0x55dcac4856f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:54.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.187+0000 7f6f33da4b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dcac485c80 0x55dcac486100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:54.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.187+0000 7f6f0bfff700 1 -- 172.21.15.100:0/2868353960 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f6f1c0e1f20 con 0x7f6f24107730 2024-05-10T11:42:54.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.187+0000 7f6f22ffd700 1 --2- 172.21.15.100:0/2868353960 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f6f0c0b2900 0x7f6f0c0b4dc0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f6f14005fd0 tx=0x7f6f14009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:42:54.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.187+0000 7f6f33da4b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55dcac3d3860 con 0x55dcac4814d0 2024-05-10T11:42:54.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.187+0000 7f6f33da4b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55dcac3727c0 con 0x55dcac481df0 2024-05-10T11:42:54.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.187+0000 7f6f33da4b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55dcac3cf820 con 0x55dcac485c80 2024-05-10T11:42:54.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.187+0000 7f6f22ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dcac4814d0 0x55dcac4818b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:54.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.187+0000 7f6f227fc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dcac481df0 0x55dcac4856f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:54.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.187+0000 7f6f237fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dcac485c80 0x55dcac486100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:54.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.187+0000 7f6f22ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dcac4814d0 0x55dcac4818b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45766/0 (socket says 172.21.15.100:45766) 2024-05-10T11:42:54.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.187+0000 7f6f22ffd700 1 -- 172.21.15.100:0/103106830 learned_addr learned my addr 172.21.15.100:0/103106830 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:42:54.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.187+0000 7f6f227fc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dcac481df0 0x55dcac4856f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:32972/0 (socket says 172.21.15.100:32972) 2024-05-10T11:42:54.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.187+0000 7f6f227fc700 1 -- 172.21.15.100:0/103106830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dcac485c80 msgr2=0x55dcac486100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:54.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.187+0000 7f6f227fc700 1 --2- 172.21.15.100:0/103106830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dcac485c80 0x55dcac486100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:54.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.187+0000 7f6f227fc700 1 -- 172.21.15.100:0/103106830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dcac4814d0 msgr2=0x55dcac4818b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:54.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.187+0000 7f6f227fc700 1 --2- 172.21.15.100:0/103106830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dcac4814d0 0x55dcac4818b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:54.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.187+0000 7f6f227fc700 1 -- 172.21.15.100:0/103106830 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcac3208d0 con 0x55dcac481df0 2024-05-10T11:42:54.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.188+0000 7f6f22ffd700 1 --2- 172.21.15.100:0/103106830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dcac4814d0 0x55dcac4818b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:42:54.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.188+0000 7f6f227fc700 1 --2- 172.21.15.100:0/103106830 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dcac481df0 0x55dcac4856f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f6f2406cc50 tx=0x7f6f18009590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9940537a37899dc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:42:54.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.188+0000 7f6f097fa700 1 -- 172.21.15.100:0/103106830 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f18038070 con 0x55dcac481df0 2024-05-10T11:42:54.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.189+0000 7f6f097fa700 1 -- 172.21.15.100:0/103106830 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6f1800f8f0 con 0x55dcac481df0 2024-05-10T11:42:54.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.189+0000 7f6f097fa700 1 -- 172.21.15.100:0/103106830 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f180046d0 con 0x55dcac481df0 2024-05-10T11:42:54.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.189+0000 7f6f33da4b80 1 -- 172.21.15.100:0/103106830 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dcac481df0 msgr2=0x55dcac4856f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:54.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.189+0000 7f6f33da4b80 1 --2- 172.21.15.100:0/103106830 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dcac481df0 0x55dcac4856f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f6f2406cc50 tx=0x7f6f18009590 comp rx=0 tx=0).stop 2024-05-10T11:42:54.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.190+0000 7f6f33da4b80 1 -- 172.21.15.100:0/103106830 shutdown_connections 2024-05-10T11:42:54.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.190+0000 7f6f33da4b80 1 --2- 172.21.15.100:0/103106830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dcac485c80 0x55dcac486100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:54.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.190+0000 7f6f33da4b80 1 --2- 172.21.15.100:0/103106830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dcac4814d0 0x55dcac4818b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:54.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.190+0000 7f6f33da4b80 1 --2- 172.21.15.100:0/103106830 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dcac481df0 0x55dcac4856f0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:54.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.190+0000 7f6f33da4b80 1 -- 172.21.15.100:0/103106830 >> 172.21.15.100:0/103106830 conn(0x55dcac3ad100 msgr2=0x55dcac47bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:42:54.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.190+0000 7f6f33da4b80 1 -- 172.21.15.100:0/103106830 shutdown_connections 2024-05-10T11:42:54.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.190+0000 7f6f33da4b80 1 -- 172.21.15.100:0/103106830 wait complete. 2024-05-10T11:42:54.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.190+0000 7f6f33da4b80 1 -- start start 2024-05-10T11:42:54.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.190+0000 7f6f33da4b80 1 Processor -- start 2024-05-10T11:42:54.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.191+0000 7f6f33da4b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dcac4814d0 0x55dcac48d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:54.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.191+0000 7f6f33da4b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dcac481df0 0x55dcac48a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:54.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.191+0000 7f6f33da4b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dcac485c80 0x55dcac48a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:54.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.191+0000 7f6f22ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dcac4814d0 0x55dcac48d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:54.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.191+0000 7f6f22ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dcac4814d0 0x55dcac48d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:32976/0 (socket says 172.21.15.100:32976) 2024-05-10T11:42:54.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.191+0000 7f6f33da4b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55dcac48de80 con 0x55dcac485c80 2024-05-10T11:42:54.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.191+0000 7f6f33da4b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55dcac339c30 con 0x55dcac4814d0 2024-05-10T11:42:54.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.191+0000 7f6f33da4b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55dcac48e000 con 0x55dcac481df0 2024-05-10T11:42:54.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.191+0000 7f6f237fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dcac485c80 0x55dcac48a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:54.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.191+0000 7f6f33da4b80 10 client.0 _ll_register_callbacks cb 0x55dcac36b920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:42:54.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.191+0000 7f6f227fc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dcac481df0 0x55dcac48a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:54.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.191+0000 7f6f227fc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dcac481df0 0x55dcac48a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:47106/0 (socket says 172.21.15.100:47106) 2024-05-10T11:42:54.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.191+0000 7f6f227fc700 1 -- 172.21.15.100:0/4276601750 learned_addr learned my addr 172.21.15.100:0/4276601750 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:42:54.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.192+0000 7f6f227fc700 1 -- 172.21.15.100:0/4276601750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dcac4814d0 msgr2=0x55dcac48d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:54.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.192+0000 7f6f227fc700 1 --2- 172.21.15.100:0/4276601750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dcac4814d0 0x55dcac48d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:54.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.192+0000 7f6f227fc700 1 -- 172.21.15.100:0/4276601750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dcac485c80 msgr2=0x55dcac48a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:54.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.192+0000 7f6f227fc700 1 --2- 172.21.15.100:0/4276601750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dcac485c80 0x55dcac48a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:54.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.192+0000 7f6f227fc700 1 -- 172.21.15.100:0/4276601750 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcac36f690 con 0x55dcac481df0 2024-05-10T11:42:54.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.192+0000 7f6f22ffd700 1 --2- 172.21.15.100:0/4276601750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dcac4814d0 0x55dcac48d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:42:54.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.192+0000 7f6f227fc700 1 --2- 172.21.15.100:0/4276601750 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dcac481df0 0x55dcac48a430 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f6f18010dc0 tx=0x7f6f18010df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16521ed8f9864e46 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:42:54.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.192+0000 7f6f06ff5700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:42:54.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.192+0000 7f6f06ff5700 1 -- 172.21.15.100:0/4276601750 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f18041060 con 0x55dcac481df0 2024-05-10T11:42:54.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.192+0000 7f6f06ff5700 1 -- 172.21.15.100:0/4276601750 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f6f180054c0 con 0x55dcac481df0 2024-05-10T11:42:54.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.193+0000 7f6f33da4b80 1 -- 172.21.15.100:0/4276601750 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f1800f8f0 con 0x55dcac481df0 2024-05-10T11:42:54.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.193+0000 7f6f06ff5700 1 -- 172.21.15.100:0/4276601750 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f6f18048050 con 0x55dcac481df0 2024-05-10T11:42:54.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.193+0000 7f6f33da4b80 10 client.29455 fetch_fsmap learned FSMap version 41 2024-05-10T11:42:54.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.193+0000 7f6f33da4b80 1 -- 172.21.15.100:0/4276601750 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55dcac48de80 con 0x55dcac481df0 2024-05-10T11:42:54.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.193+0000 7f6f06ff5700 1 -- 172.21.15.100:0/4276601750 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6f1800f8f0 con 0x55dcac481df0 2024-05-10T11:42:54.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.193+0000 7f6f06ff5700 1 -- 172.21.15.100:0/4276601750 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f6f1800b720 con 0x55dcac481df0 2024-05-10T11:42:54.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.193+0000 7f6f33da4b80 10 client.29455 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:42:54.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.193+0000 7f6f33da4b80 10 client.29455 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-10T11:42:54.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.193+0000 7f6f33da4b80 20 client.29455 populate_metadata read hostname 'smithi100' 2024-05-10T11:42:54.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.193+0000 7f6f33da4b80 1 --2- 172.21.15.100:0/4276601750 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55dcac49a790 0x55dcac49cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:54.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.193+0000 7f6f33da4b80 4 client.29455 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:42:54.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.193+0000 7f6f33da4b80 1 -- 172.21.15.100:0/4276601750 --> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dcac44d650 con 0x55dcac49a790 2024-05-10T11:42:54.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.194+0000 7f6f22ffd700 1 --2- 172.21.15.100:0/4276601750 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55dcac49a790 0x55dcac49cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:54.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.194+0000 7f6f22ffd700 1 --2- 172.21.15.100:0/4276601750 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55dcac49a790 0x55dcac49cb80 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=23312436d1bd02a6 server_cookie=af0c938e092a25a in_seq=0 out_seq=0 2024-05-10T11:42:54.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.194+0000 7f6f06ff5700 10 client.29455 ms_handle_connect on v2:172.21.15.188:6836/1943342620 2024-05-10T11:42:54.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.195+0000 7f6f06ff5700 1 -- 172.21.15.100:0/4276601750 <== mds.2 v2:172.21.15.188:6836/1943342620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dcac44d650 con 0x55dcac49a790 2024-05-10T11:42:54.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.195+0000 7f6f06ff5700 10 client.29455 handle_command_reply: tid=0 2024-05-10T11:42:54.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.195+0000 7f6f33da4b80 1 client.29455 shutdown 2024-05-10T11:42:54.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.195+0000 7f6f33da4b80 1 -- 172.21.15.100:0/4276601750 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dcac481df0 msgr2=0x55dcac48a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:54.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.195+0000 7f6f33da4b80 1 --2- 172.21.15.100:0/4276601750 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dcac481df0 0x55dcac48a430 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f6f18010dc0 tx=0x7f6f18010df0 comp rx=0 tx=0).stop 2024-05-10T11:42:54.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.195+0000 7f6f33da4b80 1 -- 172.21.15.100:0/4276601750 shutdown_connections 2024-05-10T11:42:54.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.195+0000 7f6f33da4b80 1 --2- 172.21.15.100:0/4276601750 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55dcac49a790 0x55dcac49cb80 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:54.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.195+0000 7f6f33da4b80 1 --2- 172.21.15.100:0/4276601750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dcac485c80 0x55dcac48a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:54.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.195+0000 7f6f06ff5700 0 client.29455 ms_handle_reset on v2:172.21.15.188:6836/1943342620 2024-05-10T11:42:54.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.195+0000 7f6f33da4b80 1 --2- 172.21.15.100:0/4276601750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dcac4814d0 0x55dcac48d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:54.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.195+0000 7f6f33da4b80 1 --2- 172.21.15.100:0/4276601750 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dcac481df0 0x55dcac48a430 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:54.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.195+0000 7f6f33da4b80 1 -- 172.21.15.100:0/4276601750 >> 172.21.15.100:0/4276601750 conn(0x55dcac3ad100 msgr2=0x55dcac476720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:42:54.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.195+0000 7f6f33da4b80 1 -- 172.21.15.100:0/4276601750 shutdown_connections 2024-05-10T11:42:54.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.195+0000 7f6f33da4b80 1 -- 172.21.15.100:0/4276601750 wait complete. 2024-05-10T11:42:54.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.196+0000 7f6f33da4b80 20 client.29455 trim_cache size 0 max 16384 2024-05-10T11:42:54.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.217+0000 7f6f33da4b80 1 Processor -- start 2024-05-10T11:42:54.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.217+0000 7f6f33da4b80 1 -- start start 2024-05-10T11:42:54.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.217+0000 7f6f33da4b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dcac4814d0 0x55dcac496f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:54.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.217+0000 7f6f33da4b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dcac481df0 0x55dcac489470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:54.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.217+0000 7f6f33da4b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dcac485c80 0x55dcac4a2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:54.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.217+0000 7f6f33da4b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6f241af6f0 con 0x55dcac481df0 2024-05-10T11:42:54.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.217+0000 7f6f33da4b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6f241acf80 con 0x55dcac485c80 2024-05-10T11:42:54.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.217+0000 7f6f33da4b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6f241aaf20 con 0x55dcac4814d0 2024-05-10T11:42:54.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.218+0000 7f6f227fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dcac481df0 0x55dcac489470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:54.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.218+0000 7f6f22ffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dcac4814d0 0x55dcac496f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:54.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.218+0000 7f6f227fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dcac481df0 0x55dcac489470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45774/0 (socket says 172.21.15.100:45774) 2024-05-10T11:42:54.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.218+0000 7f6f227fc700 1 -- 172.21.15.100:0/351651233 learned_addr learned my addr 172.21.15.100:0/351651233 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:42:54.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.218+0000 7f6f22ffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dcac4814d0 0x55dcac496f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:47120/0 (socket says 172.21.15.100:47120) 2024-05-10T11:42:54.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.218+0000 7f6f237fe700 1 --2- 172.21.15.100:0/351651233 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dcac485c80 0x55dcac4a2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:54.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.218+0000 7f6f22ffd700 1 -- 172.21.15.100:0/351651233 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dcac485c80 msgr2=0x55dcac4a2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:54.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.218+0000 7f6f22ffd700 1 --2- 172.21.15.100:0/351651233 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dcac485c80 0x55dcac4a2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:54.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.218+0000 7f6f22ffd700 1 -- 172.21.15.100:0/351651233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dcac481df0 msgr2=0x55dcac489470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:54.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.218+0000 7f6f22ffd700 1 --2- 172.21.15.100:0/351651233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dcac481df0 0x55dcac489470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:54.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.218+0000 7f6f22ffd700 1 -- 172.21.15.100:0/351651233 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcac48a200 con 0x55dcac4814d0 2024-05-10T11:42:54.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.218+0000 7f6f227fc700 1 --2- 172.21.15.100:0/351651233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dcac481df0 0x55dcac489470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:42:54.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.218+0000 7f6f22ffd700 1 --2- 172.21.15.100:0/351651233 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dcac4814d0 0x55dcac496f10 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f6f14011770 tx=0x7f6f14014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b4b8a7ecbb6cc6d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:42:54.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.219+0000 7f6f08ff9700 1 -- 172.21.15.100:0/351651233 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f1402b670 con 0x55dcac4814d0 2024-05-10T11:42:54.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.219+0000 7f6f08ff9700 1 -- 172.21.15.100:0/351651233 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6f14030070 con 0x55dcac4814d0 2024-05-10T11:42:54.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.219+0000 7f6f08ff9700 1 -- 172.21.15.100:0/351651233 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f14020c80 con 0x55dcac4814d0 2024-05-10T11:42:54.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.219+0000 7f6f33da4b80 1 -- 172.21.15.100:0/351651233 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dcac4814d0 msgr2=0x55dcac496f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:54.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.219+0000 7f6f33da4b80 1 --2- 172.21.15.100:0/351651233 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dcac4814d0 0x55dcac496f10 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f6f14011770 tx=0x7f6f14014000 comp rx=0 tx=0).stop 2024-05-10T11:42:54.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.220+0000 7f6f33da4b80 1 -- 172.21.15.100:0/351651233 shutdown_connections 2024-05-10T11:42:54.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.220+0000 7f6f33da4b80 1 --2- 172.21.15.100:0/351651233 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dcac485c80 0x55dcac4a2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:54.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.220+0000 7f6f33da4b80 1 --2- 172.21.15.100:0/351651233 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dcac4814d0 0x55dcac496f10 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:54.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.220+0000 7f6f33da4b80 1 --2- 172.21.15.100:0/351651233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dcac481df0 0x55dcac489470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:54.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.220+0000 7f6f33da4b80 1 -- 172.21.15.100:0/351651233 >> 172.21.15.100:0/351651233 conn(0x55dcac3ad100 msgr2=0x55dcac488d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:42:54.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.220+0000 7f6f33da4b80 1 -- 172.21.15.100:0/351651233 shutdown_connections 2024-05-10T11:42:54.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.220+0000 7f6f33da4b80 1 -- 172.21.15.100:0/351651233 wait complete. 2024-05-10T11:42:54.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.220+0000 7f6f33da4b80 1 -- start start 2024-05-10T11:42:54.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.220+0000 7f6f33da4b80 1 Processor -- start 2024-05-10T11:42:54.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.221+0000 7f6f33da4b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dcac4814d0 0x55dcac48e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:54.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.221+0000 7f6f33da4b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dcac481df0 0x55dcac48e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:54.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.221+0000 7f6f33da4b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dcac485c80 0x55dcac502e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:54.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.221+0000 7f6f33da4b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6f241ab440 con 0x55dcac481df0 2024-05-10T11:42:54.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.221+0000 7f6f22ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dcac4814d0 0x55dcac48e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:54.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.221+0000 7f6f227fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dcac481df0 0x55dcac48e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:54.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.221+0000 7f6f22ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dcac4814d0 0x55dcac48e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:32992/0 (socket says 172.21.15.100:32992) 2024-05-10T11:42:54.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.221+0000 7f6f22ffd700 1 -- 172.21.15.100:0/1455164495 learned_addr learned my addr 172.21.15.100:0/1455164495 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:42:54.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.221+0000 7f6f237fe700 1 --2- 172.21.15.100:0/1455164495 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dcac485c80 0x55dcac502e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:54.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.221+0000 7f6f33da4b80 1 -- 172.21.15.100:0/1455164495 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6f241ac8b0 con 0x55dcac4814d0 2024-05-10T11:42:54.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.221+0000 7f6f33da4b80 1 -- 172.21.15.100:0/1455164495 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6f241abb10 con 0x55dcac485c80 2024-05-10T11:42:54.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.221+0000 7f6f33da4b80 10 client.0 _ll_register_callbacks cb 0x55dcac36b920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:42:54.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.222+0000 7f6f237fe700 1 -- 172.21.15.100:0/1455164495 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dcac4814d0 msgr2=0x55dcac48e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:54.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.222+0000 7f6f237fe700 1 --2- 172.21.15.100:0/1455164495 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dcac4814d0 0x55dcac48e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:54.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.222+0000 7f6f237fe700 1 -- 172.21.15.100:0/1455164495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dcac481df0 msgr2=0x55dcac48e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:54.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.222+0000 7f6f237fe700 1 --2- 172.21.15.100:0/1455164495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dcac481df0 0x55dcac48e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:54.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.222+0000 7f6f237fe700 1 -- 172.21.15.100:0/1455164495 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcac503350 con 0x55dcac485c80 2024-05-10T11:42:54.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.222+0000 7f6f227fc700 1 --2- 172.21.15.100:0/1455164495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dcac481df0 0x55dcac48e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:42:54.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.222+0000 7f6f237fe700 1 --2- 172.21.15.100:0/1455164495 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dcac485c80 0x55dcac502e10 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c0a63f0 tx=0x7f6f1c0578d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4428bd420ca0c797 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:42:54.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.222+0000 7f6f06ff5700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:42:54.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.222+0000 7f6f06ff5700 1 -- 172.21.15.100:0/1455164495 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f1c0a9180 con 0x55dcac485c80 2024-05-10T11:42:54.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.222+0000 7f6f33da4b80 1 -- 172.21.15.100:0/1455164495 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f14030070 con 0x55dcac485c80 2024-05-10T11:42:54.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.222+0000 7f6f06ff5700 1 -- 172.21.15.100:0/1455164495 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f6f1c0ad120 con 0x55dcac485c80 2024-05-10T11:42:54.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.222+0000 7f6f06ff5700 1 -- 172.21.15.100:0/1455164495 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f6f1c0ad320 con 0x55dcac485c80 2024-05-10T11:42:54.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.222+0000 7f6f33da4b80 10 client.29461 fetch_fsmap learned FSMap version 41 2024-05-10T11:42:54.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.222+0000 7f6f33da4b80 1 -- 172.21.15.100:0/1455164495 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55dcac5045c0 con 0x55dcac485c80 2024-05-10T11:42:54.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.223+0000 7f6f06ff5700 1 -- 172.21.15.100:0/1455164495 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6f1c0ad530 con 0x55dcac485c80 2024-05-10T11:42:54.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.223+0000 7f6f06ff5700 1 -- 172.21.15.100:0/1455164495 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f6f1c0b1b50 con 0x55dcac485c80 2024-05-10T11:42:54.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.223+0000 7f6f33da4b80 10 client.29461 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:42:54.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.223+0000 7f6f33da4b80 10 client.29461 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-10T11:42:54.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.223+0000 7f6f33da4b80 20 client.29461 populate_metadata read hostname 'smithi100' 2024-05-10T11:42:54.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.223+0000 7f6f33da4b80 1 --2- 172.21.15.100:0/1455164495 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55dcac504c70 0x55dcac507090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:54.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.223+0000 7f6f33da4b80 4 client.29461 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:42:54.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.223+0000 7f6f33da4b80 1 -- 172.21.15.100:0/1455164495 --> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f6f14030070 con 0x55dcac504c70 2024-05-10T11:42:54.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.223+0000 7f6f22ffd700 1 --2- 172.21.15.100:0/1455164495 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55dcac504c70 0x55dcac507090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:54.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.224+0000 7f6f22ffd700 1 --2- 172.21.15.100:0/1455164495 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55dcac504c70 0x55dcac507090 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2afe9cbf37cfbb9c server_cookie=fdabfda6d6879d9d in_seq=0 out_seq=0 2024-05-10T11:42:54.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.224+0000 7f6f06ff5700 10 client.29461 ms_handle_connect on v2:172.21.15.188:6836/1943342620 2024-05-10T11:42:54.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.226+0000 7f6f06ff5700 1 -- 172.21.15.100:0/1455164495 <== mds.2 v2:172.21.15.188:6836/1943342620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f6f14030070 con 0x55dcac504c70 2024-05-10T11:42:54.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.226+0000 7f6f06ff5700 10 client.29461 handle_command_reply: tid=0 2024-05-10T11:42:54.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.226+0000 7f6f33da4b80 1 client.29461 shutdown 2024-05-10T11:42:54.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.226+0000 7f6f33da4b80 1 -- 172.21.15.100:0/1455164495 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dcac485c80 msgr2=0x55dcac502e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:54.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.226+0000 7f6f33da4b80 1 --2- 172.21.15.100:0/1455164495 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dcac485c80 0x55dcac502e10 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c0a63f0 tx=0x7f6f1c0578d0 comp rx=0 tx=0).stop 2024-05-10T11:42:54.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.226+0000 7f6f33da4b80 1 -- 172.21.15.100:0/1455164495 shutdown_connections 2024-05-10T11:42:54.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.226+0000 7f6f33da4b80 1 --2- 172.21.15.100:0/1455164495 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x55dcac504c70 0x55dcac507090 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:54.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.226+0000 7f6f33da4b80 1 --2- 172.21.15.100:0/1455164495 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dcac485c80 0x55dcac502e10 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:54.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.226+0000 7f6f06ff5700 0 client.29461 ms_handle_reset on v2:172.21.15.188:6836/1943342620 2024-05-10T11:42:54.963 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:42:54.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.226+0000 7f6f33da4b80 1 --2- 172.21.15.100:0/1455164495 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dcac4814d0 0x55dcac48e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:54.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.226+0000 7f6f33da4b80 1 --2- 172.21.15.100:0/1455164495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dcac481df0 0x55dcac48e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:54.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.226+0000 7f6f33da4b80 1 -- 172.21.15.100:0/1455164495 >> 172.21.15.100:0/1455164495 conn(0x55dcac3ad100 msgr2=0x55dcac475e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:42:54.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.226+0000 7f6f33da4b80 1 -- 172.21.15.100:0/1455164495 shutdown_connections 2024-05-10T11:42:54.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.226+0000 7f6f33da4b80 1 -- 172.21.15.100:0/1455164495 wait complete. 2024-05-10T11:42:54.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.227+0000 7f6f33da4b80 20 client.29461 trim_cache size 0 max 16384 2024-05-10T11:42:54.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.227+0000 7f6f29511700 1 -- 172.21.15.100:0/2868353960 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f6f0c0b2900 msgr2=0x7f6f0c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:54.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.227+0000 7f6f29511700 1 --2- 172.21.15.100:0/2868353960 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f6f0c0b2900 0x7f6f0c0b4dc0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f6f14005fd0 tx=0x7f6f14009310 comp rx=0 tx=0).stop 2024-05-10T11:42:54.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.227+0000 7f6f29511700 1 -- 172.21.15.100:0/2868353960 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f24107730 msgr2=0x7f6f241a0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:54.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.227+0000 7f6f29511700 1 --2- 172.21.15.100:0/2868353960 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f24107730 0x7f6f241a0670 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c00c370 tx=0x7f6f1c00c730 comp rx=0 tx=0).stop 2024-05-10T11:42:54.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.227+0000 7f6f29511700 1 -- 172.21.15.100:0/2868353960 shutdown_connections 2024-05-10T11:42:54.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.227+0000 7f6f29511700 1 --2- 172.21.15.100:0/2868353960 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f6f0c0b2900 0x7f6f0c0b4dc0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:54.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.227+0000 7f6f29511700 1 --2- 172.21.15.100:0/2868353960 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f24107730 0x7f6f241a0670 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:54.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.227+0000 7f6f29511700 1 --2- 172.21.15.100:0/2868353960 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f241029e0 0x7f6f2419bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:54.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.227+0000 7f6f29511700 1 --2- 172.21.15.100:0/2868353960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f24103390 0x7f6f2419c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:54.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.228+0000 7f6f29511700 1 -- 172.21.15.100:0/2868353960 >> 172.21.15.100:0/2868353960 conn(0x7f6f24079350 msgr2=0x7f6f2406bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:42:54.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.228+0000 7f6f29511700 1 -- 172.21.15.100:0/2868353960 shutdown_connections 2024-05-10T11:42:54.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:42:54.228+0000 7f6f29511700 1 -- 172.21.15.100:0/2868353960 wait complete. 2024-05-10T11:42:54.965 INFO:tasks.workunit.client.0.smithi100.stdout: 10 107484 8.89 MB/sec execute 528 sec latency 168.895 ms 2024-05-10T11:42:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.018+0000 7f95a159a640 1 -- 172.21.15.100:0/3771673363 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959c031d90 con 0x7f95a406e550 2024-05-10T11:42:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.018+0000 7f95a8ae1640 1 -- 172.21.15.100:0/3771673363 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95a406e550 msgr2=0x7f95a4072e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.018+0000 7f95a8ae1640 1 --2- 172.21.15.100:0/3771673363 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95a406e550 0x7f95a4072e10 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f959c009ca0 tx=0x7f959c02a8d0 comp rx=0 tx=0).stop 2024-05-10T11:42:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.019+0000 7f95a8ae1640 1 -- 172.21.15.100:0/3771673363 shutdown_connections 2024-05-10T11:42:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.019+0000 7f95a8ae1640 1 --2- 172.21.15.100:0/3771673363 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95a406e550 0x7f95a4072e10 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.019+0000 7f95a8ae1640 1 --2- 172.21.15.100:0/3771673363 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95a406dbb0 0x7f95a406e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.019+0000 7f95a8ae1640 1 --2- 172.21.15.100:0/3771673363 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95a406c970 0x7f95a406cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.019+0000 7f95a8ae1640 1 -- 172.21.15.100:0/3771673363 >> 172.21.15.100:0/3771673363 conn(0x7f95a40fce80 msgr2=0x7f95a40ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:42:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.019+0000 7f95a8ae1640 1 -- 172.21.15.100:0/3771673363 shutdown_connections 2024-05-10T11:42:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.019+0000 7f95a8ae1640 1 -- 172.21.15.100:0/3771673363 wait complete. 2024-05-10T11:42:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.020+0000 7f95a8ae1640 1 Processor -- start 2024-05-10T11:42:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.020+0000 7f95a8ae1640 1 -- start start 2024-05-10T11:42:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.020+0000 7f95a8ae1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95a406c970 0x7f95a41a7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.020+0000 7f95a8ae1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95a406dbb0 0x7f95a41a7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.020+0000 7f95a8ae1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95a406e550 0x7f95a41aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.020+0000 7f95a8ae1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f95a4118eb0 con 0x7f95a406c970 2024-05-10T11:42:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.020+0000 7f95a8ae1640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f95a4118d30 con 0x7f95a406dbb0 2024-05-10T11:42:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.020+0000 7f95a8ae1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f95a4119030 con 0x7f95a406e550 2024-05-10T11:42:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.020+0000 7f95a2d9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95a406e550 0x7f95a41aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.020+0000 7f95a2d9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95a406e550 0x7f95a41aeed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:47156/0 (socket says 172.21.15.100:47156) 2024-05-10T11:42:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.020+0000 7f95a2d9d640 1 -- 172.21.15.100:0/3380900254 learned_addr learned my addr 172.21.15.100:0/3380900254 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:42:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.020+0000 7f95a259c640 1 --2- 172.21.15.100:0/3380900254 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95a406c970 0x7f95a41a7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.021+0000 7f95a1d9b640 1 --2- 172.21.15.100:0/3380900254 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95a406dbb0 0x7f95a41a7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.021+0000 7f95a1d9b640 1 -- 172.21.15.100:0/3380900254 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95a406e550 msgr2=0x7f95a41aeed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.021+0000 7f95a1d9b640 1 --2- 172.21.15.100:0/3380900254 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95a406e550 0x7f95a41aeed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.021+0000 7f95a1d9b640 1 -- 172.21.15.100:0/3380900254 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95a406c970 msgr2=0x7f95a41a7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.021+0000 7f95a1d9b640 1 --2- 172.21.15.100:0/3380900254 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95a406c970 0x7f95a41a7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.021+0000 7f95a1d9b640 1 -- 172.21.15.100:0/3380900254 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95a41af540 con 0x7f95a406dbb0 2024-05-10T11:42:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.021+0000 7f95a2d9d640 1 --2- 172.21.15.100:0/3380900254 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95a406e550 0x7f95a41aeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:42:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.021+0000 7f95a259c640 1 --2- 172.21.15.100:0/3380900254 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95a406c970 0x7f95a41a7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:42:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.021+0000 7f95a1d9b640 1 --2- 172.21.15.100:0/3380900254 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95a406dbb0 0x7f95a41a7e60 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f959800e9f0 tx=0x7f959800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:42:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.021+0000 7f95937fe640 1 -- 172.21.15.100:0/3380900254 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959800ccc0 con 0x7f95a406dbb0 2024-05-10T11:42:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.022+0000 7f95a8ae1640 1 -- 172.21.15.100:0/3380900254 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95a41af830 con 0x7f95a406dbb0 2024-05-10T11:42:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.022+0000 7f95937fe640 1 -- 172.21.15.100:0/3380900254 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f95980108c0 con 0x7f95a406dbb0 2024-05-10T11:42:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.022+0000 7f95937fe640 1 -- 172.21.15.100:0/3380900254 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95980187e0 con 0x7f95a406dbb0 2024-05-10T11:42:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.022+0000 7f95a8ae1640 1 -- 172.21.15.100:0/3380900254 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95a41afe50 con 0x7f95a406dbb0 2024-05-10T11:42:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.025+0000 7f95937fe640 1 -- 172.21.15.100:0/3380900254 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f9598018980 con 0x7f95a406dbb0 2024-05-10T11:42:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.025+0000 7f95ab800740 1 Processor -- start 2024-05-10T11:42:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.025+0000 7f95ab800740 1 -- start start 2024-05-10T11:42:55.027 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.025+0000 7f95ab800740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca7e85adb0 0x55ca7e85b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:55.027 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.025+0000 7f95ab800740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7e85b7a0 0x55ca7e8602a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:55.027 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.026+0000 7f95ab800740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ca7e860910 0x55ca7e862da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:55.027 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.026+0000 7f95ab800740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55ca7e7bf5d0 con 0x55ca7e85b7a0 2024-05-10T11:42:55.027 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.026+0000 7f95ab800740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ca7e778890 con 0x55ca7e860910 2024-05-10T11:42:55.027 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.026+0000 7f95ab800740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55ca7e7bb590 con 0x55ca7e85adb0 2024-05-10T11:42:55.027 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.026+0000 7f95a259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7e85b7a0 0x55ca7e8602a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:55.027 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.026+0000 7f95a259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7e85b7a0 0x55ca7e8602a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45808/0 (socket says 172.21.15.100:45808) 2024-05-10T11:42:55.027 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.026+0000 7f95a2d9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca7e85adb0 0x55ca7e85b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:55.027 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.026+0000 7f95a259c640 1 -- 172.21.15.100:0/1038620737 learned_addr learned my addr 172.21.15.100:0/1038620737 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:42:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.026+0000 7f95a1d9b640 1 --2- 172.21.15.100:0/1038620737 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ca7e860910 0x55ca7e862da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.026+0000 7f95a259c640 1 -- 172.21.15.100:0/1038620737 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca7e85adb0 msgr2=0x55ca7e85b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.026+0000 7f95a259c640 1 --2- 172.21.15.100:0/1038620737 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca7e85adb0 0x55ca7e85b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.026+0000 7f95a259c640 1 -- 172.21.15.100:0/1038620737 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ca7e860910 msgr2=0x55ca7e862da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.026+0000 7f95a259c640 1 --2- 172.21.15.100:0/1038620737 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ca7e860910 0x55ca7e862da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.026+0000 7f95a259c640 1 -- 172.21.15.100:0/1038620737 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca7e4d1210 con 0x55ca7e85b7a0 2024-05-10T11:42:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.026+0000 7f95a259c640 1 --2- 172.21.15.100:0/1038620737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7e85b7a0 0x55ca7e8602a0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f9594009df0 tx=0x7f959400e5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce5a6a4dea8c08b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:42:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.026+0000 7f95937fe640 1 --2- 172.21.15.100:0/3380900254 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f958c0b2920 0x7f958c0b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.027+0000 7f9590ff9640 1 -- 172.21.15.100:0/1038620737 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9594017070 con 0x55ca7e85b7a0 2024-05-10T11:42:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.027+0000 7f95a2d9d640 1 --2- 172.21.15.100:0/3380900254 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f958c0b2920 0x7f958c0b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.027+0000 7f9590ff9640 1 -- 172.21.15.100:0/1038620737 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9594013900 con 0x55ca7e85b7a0 2024-05-10T11:42:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.027+0000 7f95937fe640 1 -- 172.21.15.100:0/3380900254 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f95980e2880 con 0x7f95a406dbb0 2024-05-10T11:42:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.027+0000 7f9590ff9640 1 -- 172.21.15.100:0/1038620737 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95940126e0 con 0x55ca7e85b7a0 2024-05-10T11:42:55.032 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.027+0000 7f95ab800740 1 -- 172.21.15.100:0/1038620737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7e85b7a0 msgr2=0x55ca7e8602a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:55.033 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.027+0000 7f95a2d9d640 1 --2- 172.21.15.100:0/3380900254 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f958c0b2920 0x7f958c0b4de0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x55ca7e85c000 tx=0x7f959c031990 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:42:55.033 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.027+0000 7f95ab800740 1 --2- 172.21.15.100:0/1038620737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7e85b7a0 0x55ca7e8602a0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f9594009df0 tx=0x7f959400e5a0 comp rx=0 tx=0).stop 2024-05-10T11:42:55.033 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.028+0000 7f95ab800740 1 -- 172.21.15.100:0/1038620737 shutdown_connections 2024-05-10T11:42:55.033 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.028+0000 7f95ab800740 1 --2- 172.21.15.100:0/1038620737 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ca7e860910 0x55ca7e862da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:55.033 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.028+0000 7f95ab800740 1 --2- 172.21.15.100:0/1038620737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7e85b7a0 0x55ca7e8602a0 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:55.033 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.028+0000 7f95ab800740 1 --2- 172.21.15.100:0/1038620737 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca7e85adb0 0x55ca7e85b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:55.033 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.028+0000 7f95ab800740 1 -- 172.21.15.100:0/1038620737 >> 172.21.15.100:0/1038620737 conn(0x55ca7e856420 msgr2=0x55ca7e856820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:42:55.033 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.028+0000 7f95ab800740 1 -- 172.21.15.100:0/1038620737 shutdown_connections 2024-05-10T11:42:55.033 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.028+0000 7f95ab800740 1 -- 172.21.15.100:0/1038620737 wait complete. 2024-05-10T11:42:55.033 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.028+0000 7f95ab800740 1 -- start start 2024-05-10T11:42:55.033 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.028+0000 7f95ab800740 1 Processor -- start 2024-05-10T11:42:55.033 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.028+0000 7f95ab800740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ca7e85adb0 0x55ca7e865040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:55.033 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.029+0000 7f95ab800740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca7e85b7a0 0x55ca7e865580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:55.034 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.029+0000 7f95ab800740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7e860910 0x55ca7e86c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:55.034 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.029+0000 7f95ab800740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55ca7e863910 con 0x55ca7e860910 2024-05-10T11:42:55.034 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.029+0000 7f95ab800740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ca7e77c370 con 0x55ca7e85adb0 2024-05-10T11:42:55.034 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.029+0000 7f95ab800740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55ca7e863a90 con 0x55ca7e85b7a0 2024-05-10T11:42:55.034 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.029+0000 7f95ab800740 10 client.0 _ll_register_callbacks cb 0x55ca7e4d0a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:42:55.034 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.029+0000 7f95a2d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7e860910 0x55ca7e86c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:55.034 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.029+0000 7f95a259c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ca7e85adb0 0x55ca7e865040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:55.034 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.029+0000 7f95a259c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ca7e85adb0 0x55ca7e865040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:33010/0 (socket says 172.21.15.100:33010) 2024-05-10T11:42:55.034 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.029+0000 7f95a2d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7e860910 0x55ca7e86c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45824/0 (socket says 172.21.15.100:45824) 2024-05-10T11:42:55.034 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.029+0000 7f95a259c640 1 -- 172.21.15.100:0/2856347846 learned_addr learned my addr 172.21.15.100:0/2856347846 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:42:55.034 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.029+0000 7f95a1d9b640 1 --2- 172.21.15.100:0/2856347846 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca7e85b7a0 0x55ca7e865580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:55.034 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.029+0000 7f95a259c640 1 -- 172.21.15.100:0/2856347846 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca7e85b7a0 msgr2=0x55ca7e865580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:55.034 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.029+0000 7f95a259c640 1 --2- 172.21.15.100:0/2856347846 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca7e85b7a0 0x55ca7e865580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:55.034 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.029+0000 7f95a259c640 1 -- 172.21.15.100:0/2856347846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7e860910 msgr2=0x55ca7e86c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:55.034 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.029+0000 7f95a259c640 1 --2- 172.21.15.100:0/2856347846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7e860910 0x55ca7e86c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:55.035 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.029+0000 7f95a259c640 1 -- 172.21.15.100:0/2856347846 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca7e59d620 con 0x55ca7e85adb0 2024-05-10T11:42:55.035 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.029+0000 7f95a1d9b640 1 --2- 172.21.15.100:0/2856347846 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca7e85b7a0 0x55ca7e865580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:42:55.035 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.029+0000 7f95a259c640 1 --2- 172.21.15.100:0/2856347846 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ca7e85adb0 0x55ca7e865040 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9594006fd0 tx=0x7f9594013d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75c766a5952c1d65 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:42:55.035 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.029+0000 7f9589ffb640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:42:55.035 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.030+0000 7f9589ffb640 1 -- 172.21.15.100:0/2856347846 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9594017040 con 0x55ca7e85adb0 2024-05-10T11:42:55.035 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.030+0000 7f9589ffb640 1 -- 172.21.15.100:0/2856347846 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f959401c860 con 0x55ca7e85adb0 2024-05-10T11:42:55.035 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.030+0000 7f95ab800740 1 -- 172.21.15.100:0/2856347846 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9594013900 con 0x55ca7e85adb0 2024-05-10T11:42:55.035 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.030+0000 7f9589ffb640 1 -- 172.21.15.100:0/2856347846 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f9594006d10 con 0x55ca7e85adb0 2024-05-10T11:42:55.035 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.030+0000 7f95ab800740 10 client.29357 fetch_fsmap learned FSMap version 41 2024-05-10T11:42:55.035 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.030+0000 7f95ab800740 1 -- 172.21.15.100:0/2856347846 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55ca7e86f5f0 con 0x55ca7e85adb0 2024-05-10T11:42:55.035 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.030+0000 7f9589ffb640 1 -- 172.21.15.100:0/2856347846 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9594013900 con 0x55ca7e85adb0 2024-05-10T11:42:55.035 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.030+0000 7f9589ffb640 1 -- 172.21.15.100:0/2856347846 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f9594026990 con 0x55ca7e85adb0 2024-05-10T11:42:55.035 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.030+0000 7f95ab800740 10 client.29357 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:42:55.035 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.030+0000 7f95ab800740 10 client.29357 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:42:55.035 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.030+0000 7f95ab800740 20 client.29357 populate_metadata read hostname 'smithi100' 2024-05-10T11:42:55.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.030+0000 7f95ab800740 1 --2- 172.21.15.100:0/2856347846 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ca7e8709a0 0x55ca7e872d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:55.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.030+0000 7f95ab800740 4 client.29357 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:42:55.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.030+0000 7f95ab800740 1 -- 172.21.15.100:0/2856347846 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca7e7cd1e0 con 0x55ca7e8709a0 2024-05-10T11:42:55.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.031+0000 7f95a1d9b640 1 --2- 172.21.15.100:0/2856347846 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ca7e8709a0 0x55ca7e872d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:55.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.031+0000 7f9589ffb640 10 client.29357 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:42:55.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.031+0000 7f95a1d9b640 1 --2- 172.21.15.100:0/2856347846 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ca7e8709a0 0x55ca7e872d90 crc :-1 s=READY pgs=455 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e534cbea45aaf71 server_cookie=f58ea715fac05c60 in_seq=0 out_seq=0 2024-05-10T11:42:55.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.032+0000 7f9589ffb640 1 -- 172.21.15.100:0/2856347846 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ca7e7cd1e0 con 0x55ca7e8709a0 2024-05-10T11:42:55.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.032+0000 7f9589ffb640 10 client.29357 handle_command_reply: tid=0 2024-05-10T11:42:55.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.032+0000 7f95ab800740 1 client.29357 shutdown 2024-05-10T11:42:55.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.032+0000 7f95ab800740 1 -- 172.21.15.100:0/2856347846 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ca7e85adb0 msgr2=0x55ca7e865040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:55.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.032+0000 7f95ab800740 1 --2- 172.21.15.100:0/2856347846 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ca7e85adb0 0x55ca7e865040 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9594006fd0 tx=0x7f9594013d60 comp rx=0 tx=0).stop 2024-05-10T11:42:55.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.032+0000 7f95ab800740 1 -- 172.21.15.100:0/2856347846 shutdown_connections 2024-05-10T11:42:55.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.032+0000 7f95ab800740 1 --2- 172.21.15.100:0/2856347846 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ca7e8709a0 0x55ca7e872d90 crc :-1 s=READY pgs=455 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:55.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.032+0000 7f95ab800740 1 --2- 172.21.15.100:0/2856347846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7e860910 0x55ca7e86c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:55.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.032+0000 7f95ab800740 1 --2- 172.21.15.100:0/2856347846 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca7e85b7a0 0x55ca7e865580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:55.037 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.032+0000 7f95ab800740 1 --2- 172.21.15.100:0/2856347846 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ca7e85adb0 0x55ca7e865040 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:55.037 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.032+0000 7f95ab800740 1 -- 172.21.15.100:0/2856347846 >> 172.21.15.100:0/2856347846 conn(0x55ca7e856420 msgr2=0x55ca7e855580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:42:55.037 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.032+0000 7f9589ffb640 0 client.29357 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:42:55.037 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.032+0000 7f95ab800740 1 -- 172.21.15.100:0/2856347846 shutdown_connections 2024-05-10T11:42:55.037 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.032+0000 7f95ab800740 1 -- 172.21.15.100:0/2856347846 wait complete. 2024-05-10T11:42:55.037 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.033+0000 7f95ab800740 20 client.29357 trim_cache size 0 max 16384 2024-05-10T11:42:55.059 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.058+0000 7f95ab800740 1 Processor -- start 2024-05-10T11:42:55.059 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.058+0000 7f95ab800740 1 -- start start 2024-05-10T11:42:55.059 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.058+0000 7f95ab800740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ca7e85adb0 0x55ca7e859100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:55.059 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.058+0000 7f95ab800740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7e85b7a0 0x55ca7e86c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:55.059 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.058+0000 7f95ab800740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca7e860910 0x55ca7e859640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:55.059 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.058+0000 7f95ab800740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f95a41bcbc0 con 0x55ca7e85b7a0 2024-05-10T11:42:55.059 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.058+0000 7f95ab800740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f95a41ba490 con 0x55ca7e85adb0 2024-05-10T11:42:55.059 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.058+0000 7f95ab800740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f95a41b82e0 con 0x55ca7e860910 2024-05-10T11:42:55.060 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.058+0000 7f95a1d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7e85b7a0 0x55ca7e86c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:55.060 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.058+0000 7f95a1d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7e85b7a0 0x55ca7e86c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45836/0 (socket says 172.21.15.100:45836) 2024-05-10T11:42:55.060 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.058+0000 7f95a1d9b640 1 -- 172.21.15.100:0/4175325020 learned_addr learned my addr 172.21.15.100:0/4175325020 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:42:55.060 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.058+0000 7f95a2d9d640 1 --2- 172.21.15.100:0/4175325020 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca7e860910 0x55ca7e859640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:55.060 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.058+0000 7f95a259c640 1 --2- 172.21.15.100:0/4175325020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ca7e85adb0 0x55ca7e859100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:55.060 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.058+0000 7f95a1d9b640 1 -- 172.21.15.100:0/4175325020 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca7e860910 msgr2=0x55ca7e859640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:55.060 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.058+0000 7f95a1d9b640 1 --2- 172.21.15.100:0/4175325020 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca7e860910 0x55ca7e859640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:55.060 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.058+0000 7f95a1d9b640 1 -- 172.21.15.100:0/4175325020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ca7e85adb0 msgr2=0x55ca7e859100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:55.060 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.058+0000 7f95a1d9b640 1 --2- 172.21.15.100:0/4175325020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ca7e85adb0 0x55ca7e859100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:55.060 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.058+0000 7f95a1d9b640 1 -- 172.21.15.100:0/4175325020 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca7e851f70 con 0x55ca7e85b7a0 2024-05-10T11:42:55.060 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.058+0000 7f95a259c640 1 --2- 172.21.15.100:0/4175325020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ca7e85adb0 0x55ca7e859100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:42:55.060 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.059+0000 7f95a1d9b640 1 --2- 172.21.15.100:0/4175325020 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7e85b7a0 0x55ca7e86c580 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f95980e0020 tx=0x7f95980e4d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=566863f36553de1e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:42:55.060 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.059+0000 7f958bfff640 1 -- 172.21.15.100:0/4175325020 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95980a5380 con 0x55ca7e85b7a0 2024-05-10T11:42:55.060 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.059+0000 7f958bfff640 1 -- 172.21.15.100:0/4175325020 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f95980b22e0 con 0x55ca7e85b7a0 2024-05-10T11:42:55.061 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.059+0000 7f958bfff640 1 -- 172.21.15.100:0/4175325020 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9598010a70 con 0x55ca7e85b7a0 2024-05-10T11:42:55.063 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.060+0000 7f95ab800740 1 -- 172.21.15.100:0/4175325020 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7e85b7a0 msgr2=0x55ca7e86c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:55.063 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.060+0000 7f95ab800740 1 --2- 172.21.15.100:0/4175325020 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7e85b7a0 0x55ca7e86c580 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f95980e0020 tx=0x7f95980e4d90 comp rx=0 tx=0).stop 2024-05-10T11:42:55.063 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.060+0000 7f95ab800740 1 -- 172.21.15.100:0/4175325020 shutdown_connections 2024-05-10T11:42:55.063 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.060+0000 7f95ab800740 1 --2- 172.21.15.100:0/4175325020 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca7e860910 0x55ca7e859640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.060+0000 7f95ab800740 1 --2- 172.21.15.100:0/4175325020 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7e85b7a0 0x55ca7e86c580 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.060+0000 7f95ab800740 1 --2- 172.21.15.100:0/4175325020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ca7e85adb0 0x55ca7e859100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.060+0000 7f95ab800740 1 -- 172.21.15.100:0/4175325020 >> 172.21.15.100:0/4175325020 conn(0x55ca7e856420 msgr2=0x55ca7e85ffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:42:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.060+0000 7f95ab800740 1 -- 172.21.15.100:0/4175325020 shutdown_connections 2024-05-10T11:42:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.060+0000 7f95ab800740 1 -- 172.21.15.100:0/4175325020 wait complete. 2024-05-10T11:42:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.060+0000 7f95ab800740 1 -- start start 2024-05-10T11:42:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.061+0000 7f95ab800740 1 Processor -- start 2024-05-10T11:42:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.061+0000 7f95ab800740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7e85adb0 0x55ca7e86c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.061+0000 7f95ab800740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ca7e85b7a0 0x55ca7e868f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.061+0000 7f95ab800740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca7e860910 0x55ca7e8694e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.061+0000 7f95ab800740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f95a41b8950 con 0x55ca7e85adb0 2024-05-10T11:42:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.061+0000 7f95ab800740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f95a41b9dc0 con 0x55ca7e85b7a0 2024-05-10T11:42:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.061+0000 7f95ab800740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f95a41b9020 con 0x55ca7e860910 2024-05-10T11:42:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.061+0000 7f95ab800740 10 client.0 _ll_register_callbacks cb 0x7f95980e8420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:42:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.061+0000 7f95a1d9b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ca7e85b7a0 0x55ca7e868f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.061+0000 7f95a1d9b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ca7e85b7a0 0x55ca7e868f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:33016/0 (socket says 172.21.15.100:33016) 2024-05-10T11:42:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.061+0000 7f95a1d9b640 1 -- 172.21.15.100:0/824478575 learned_addr learned my addr 172.21.15.100:0/824478575 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:42:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.061+0000 7f95a259c640 1 --2- 172.21.15.100:0/824478575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7e85adb0 0x55ca7e86c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.062+0000 7f95a2d9d640 1 --2- 172.21.15.100:0/824478575 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca7e860910 0x55ca7e8694e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.062+0000 7f95a1d9b640 1 -- 172.21.15.100:0/824478575 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca7e860910 msgr2=0x55ca7e8694e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.062+0000 7f95a1d9b640 1 --2- 172.21.15.100:0/824478575 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca7e860910 0x55ca7e8694e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.062+0000 7f95a1d9b640 1 -- 172.21.15.100:0/824478575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7e85adb0 msgr2=0x55ca7e86c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:42:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.062+0000 7f95a1d9b640 1 --2- 172.21.15.100:0/824478575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7e85adb0 0x55ca7e86c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.062+0000 7f95a1d9b640 1 -- 172.21.15.100:0/824478575 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca7e8d9180 con 0x55ca7e85b7a0 2024-05-10T11:42:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.062+0000 7f95a259c640 1 --2- 172.21.15.100:0/824478575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7e85adb0 0x55ca7e86c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:42:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.062+0000 7f95a1d9b640 1 --2- 172.21.15.100:0/824478575 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ca7e85b7a0 0x55ca7e868f60 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f959800cfd0 tx=0x7f959801f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b599d3de35eddb0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:42:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.062+0000 7f9589ffb640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:42:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.062+0000 7f9589ffb640 1 -- 172.21.15.100:0/824478575 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959808f060 con 0x55ca7e85b7a0 2024-05-10T11:42:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.062+0000 7f9589ffb640 1 -- 172.21.15.100:0/824478575 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f95980a61e0 con 0x55ca7e85b7a0 2024-05-10T11:42:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.063+0000 7f95ab800740 1 -- 172.21.15.100:0/824478575 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ca7e7c7050 con 0x55ca7e85b7a0 2024-05-10T11:42:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.063+0000 7f9589ffb640 1 -- 172.21.15.100:0/824478575 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f95980a5380 con 0x55ca7e85b7a0 2024-05-10T11:42:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.063+0000 7f95ab800740 10 client.29363 fetch_fsmap learned FSMap version 41 2024-05-10T11:42:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.063+0000 7f95ab800740 1 -- 172.21.15.100:0/824478575 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55ca7e8da4c0 con 0x55ca7e85b7a0 2024-05-10T11:42:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.063+0000 7f9589ffb640 1 -- 172.21.15.100:0/824478575 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55ca7e7c7050 con 0x55ca7e85b7a0 2024-05-10T11:42:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.064+0000 7f9589ffb640 1 -- 172.21.15.100:0/824478575 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f95980a8e50 con 0x55ca7e85b7a0 2024-05-10T11:42:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.064+0000 7f95ab800740 10 client.29363 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:42:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.064+0000 7f95ab800740 10 client.29363 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:42:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.064+0000 7f95ab800740 20 client.29363 populate_metadata read hostname 'smithi100' 2024-05-10T11:42:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.064+0000 7f95ab800740 1 --2- 172.21.15.100:0/824478575 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ca7e8dadd0 0x55ca7e8dd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:42:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.064+0000 7f95ab800740 4 client.29363 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:42:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.064+0000 7f95ab800740 1 -- 172.21.15.100:0/824478575 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ca7e7425f0 con 0x55ca7e8dadd0 2024-05-10T11:42:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.064+0000 7f95a259c640 1 --2- 172.21.15.100:0/824478575 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ca7e8dadd0 0x55ca7e8dd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:42:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.065+0000 7f95a259c640 1 --2- 172.21.15.100:0/824478575 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ca7e8dadd0 0x55ca7e8dd1f0 crc :-1 s=READY pgs=456 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2ed3058184fc629 server_cookie=3c65fb2ea50469d7 in_seq=0 out_seq=0 2024-05-10T11:42:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.065+0000 7f9589ffb640 10 client.29363 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:42:55.190 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.190+0000 7f9589ffb640 1 -- 172.21.15.100:0/824478575 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ca7e7425f0 con 0x55ca7e8dadd0 2024-05-10T11:42:55.191 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.190+0000 7f9589ffb640 10 client.29363 handle_command_reply: tid=0 2024-05-10T11:42:55.191 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.190+0000 7f95ab800740 1 client.29363 shutdown 2024-05-10T11:42:55.191 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.190+0000 7f95ab800740 1 -- 172.21.15.100:0/824478575 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ca7e85b7a0 msgr2=0x55ca7e868f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:55.191 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.190+0000 7f95ab800740 1 --2- 172.21.15.100:0/824478575 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ca7e85b7a0 0x55ca7e868f60 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f959800cfd0 tx=0x7f959801f000 comp rx=0 tx=0).stop 2024-05-10T11:42:55.191 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.190+0000 7f95ab800740 1 -- 172.21.15.100:0/824478575 shutdown_connections 2024-05-10T11:42:55.191 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.190+0000 7f95ab800740 1 --2- 172.21.15.100:0/824478575 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ca7e8dadd0 0x55ca7e8dd1f0 crc :-1 s=READY pgs=456 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:55.191 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.190+0000 7f95ab800740 1 --2- 172.21.15.100:0/824478575 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca7e860910 0x55ca7e8694e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:55.191 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.190+0000 7f95ab800740 1 --2- 172.21.15.100:0/824478575 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ca7e85b7a0 0x55ca7e868f60 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:55.191 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.190+0000 7f9589ffb640 0 client.29363 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:42:55.191 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.190+0000 7f95ab800740 1 --2- 172.21.15.100:0/824478575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7e85adb0 0x55ca7e86c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:55.191 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.190+0000 7f95ab800740 1 -- 172.21.15.100:0/824478575 >> 172.21.15.100:0/824478575 conn(0x55ca7e856420 msgr2=0x55ca7e854cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:42:55.191 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.190+0000 7f95ab800740 1 -- 172.21.15.100:0/824478575 shutdown_connections 2024-05-10T11:42:55.191 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.190+0000 7f95ab800740 1 -- 172.21.15.100:0/824478575 wait complete. 2024-05-10T11:42:55.191 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.190+0000 7f95ab800740 20 client.29363 trim_cache size 0 max 16384 2024-05-10T11:42:55.192 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:42:55.192 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:42:55.192 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:42:55.192 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:42:55.193 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.190+0000 7f95a8ae1640 1 -- 172.21.15.100:0/3380900254 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f958c0b2920 msgr2=0x7f958c0b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:55.193 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.190+0000 7f95a8ae1640 1 --2- 172.21.15.100:0/3380900254 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f958c0b2920 0x7f958c0b4de0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x55ca7e85c000 tx=0x7f959c031990 comp rx=0 tx=0).stop 2024-05-10T11:42:55.193 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.190+0000 7f95a8ae1640 1 -- 172.21.15.100:0/3380900254 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95a406dbb0 msgr2=0x7f95a41a7e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:42:55.193 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.190+0000 7f95a8ae1640 1 --2- 172.21.15.100:0/3380900254 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95a406dbb0 0x7f95a41a7e60 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f959800e9f0 tx=0x7f959800eeb0 comp rx=0 tx=0).stop 2024-05-10T11:42:55.193 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.191+0000 7f95a8ae1640 1 -- 172.21.15.100:0/3380900254 shutdown_connections 2024-05-10T11:42:55.193 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.191+0000 7f95a8ae1640 1 --2- 172.21.15.100:0/3380900254 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f958c0b2920 0x7f958c0b4de0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:55.193 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.191+0000 7f95a8ae1640 1 --2- 172.21.15.100:0/3380900254 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95a406e550 0x7f95a41aeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:55.193 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.191+0000 7f95a8ae1640 1 --2- 172.21.15.100:0/3380900254 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95a406dbb0 0x7f95a41a7e60 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:55.193 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.191+0000 7f95a8ae1640 1 --2- 172.21.15.100:0/3380900254 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95a406c970 0x7f95a41a7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:42:55.193 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.191+0000 7f95a8ae1640 1 -- 172.21.15.100:0/3380900254 >> 172.21.15.100:0/3380900254 conn(0x7f95a40fce80 msgr2=0x7f95a40fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:42:55.193 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.191+0000 7f95a8ae1640 1 -- 172.21.15.100:0/3380900254 shutdown_connections 2024-05-10T11:42:55.193 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:42:55.191+0000 7f95a8ae1640 1 -- 172.21.15.100:0/3380900254 wait complete. 2024-05-10T11:42:55.960 INFO:tasks.workunit.client.0.smithi100.stdout: 10 107618 8.88 MB/sec execute 529 sec latency 147.001 ms 2024-05-10T11:42:56.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:42:55 smithi188 ceph-mon[34075]: pgmap v1157: 129 pgs: 129 active+clean; 570 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 761 KiB/s rd, 3.5 MiB/s wr, 271 op/s 2024-05-10T11:42:56.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:42:55 smithi100 ceph-mon[30296]: pgmap v1157: 129 pgs: 129 active+clean; 570 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 761 KiB/s rd, 3.5 MiB/s wr, 271 op/s 2024-05-10T11:42:56.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:42:55 smithi178 ceph-mon[33359]: pgmap v1157: 129 pgs: 129 active+clean; 570 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 761 KiB/s rd, 3.5 MiB/s wr, 271 op/s 2024-05-10T11:42:56.960 INFO:tasks.workunit.client.0.smithi100.stdout: 10 107794 8.89 MB/sec execute 530 sec latency 95.499 ms 2024-05-10T11:42:57.960 INFO:tasks.workunit.client.0.smithi100.stdout: 10 107981 8.91 MB/sec execute 531 sec latency 89.436 ms 2024-05-10T11:42:58.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:42:57 smithi188 ceph-mon[34075]: pgmap v1158: 129 pgs: 129 active+clean; 571 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 3.5 MiB/s wr, 275 op/s 2024-05-10T11:42:58.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:42:57 smithi100 ceph-mon[30296]: pgmap v1158: 129 pgs: 129 active+clean; 571 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 3.5 MiB/s wr, 275 op/s 2024-05-10T11:42:58.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:42:57 smithi178 ceph-mon[33359]: pgmap v1158: 129 pgs: 129 active+clean; 571 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 3.5 MiB/s wr, 275 op/s 2024-05-10T11:42:58.960 INFO:tasks.workunit.client.0.smithi100.stdout: 10 108129 8.91 MB/sec execute 532 sec latency 195.053 ms 2024-05-10T11:42:59.960 INFO:tasks.workunit.client.0.smithi100.stdout: 10 108272 8.91 MB/sec execute 533 sec latency 136.755 ms 2024-05-10T11:43:00.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:42:59 smithi188 ceph-mon[34075]: pgmap v1159: 129 pgs: 129 active+clean; 576 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 3.8 MiB/s wr, 271 op/s 2024-05-10T11:43:00.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:42:59 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:43:00.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:42:59 smithi100 ceph-mon[30296]: pgmap v1159: 129 pgs: 129 active+clean; 576 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 3.8 MiB/s wr, 271 op/s 2024-05-10T11:43:00.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:42:59 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:43:00.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:42:59 smithi178 ceph-mon[33359]: pgmap v1159: 129 pgs: 129 active+clean; 576 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 3.8 MiB/s wr, 271 op/s 2024-05-10T11:43:00.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:42:59 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:43:00.960 INFO:tasks.workunit.client.0.smithi100.stdout: 10 108372 8.90 MB/sec execute 534 sec latency 149.465 ms 2024-05-10T11:43:01.862 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-10T11:43:01.900 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:43:01 smithi178 ceph-mon[33359]: pgmap v1160: 129 pgs: 129 active+clean; 575 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 3.8 MiB/s wr, 253 op/s 2024-05-10T11:43:01.960 INFO:tasks.workunit.client.0.smithi100.stdout: 10 108475 8.89 MB/sec execute 535 sec latency 148.812 ms 2024-05-10T11:43:02.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:43:01 smithi188 ceph-mon[34075]: pgmap v1160: 129 pgs: 129 active+clean; 575 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 3.8 MiB/s wr, 253 op/s 2024-05-10T11:43:02.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:43:01 smithi100 ceph-mon[30296]: pgmap v1160: 129 pgs: 129 active+clean; 575 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 3.8 MiB/s wr, 253 op/s 2024-05-10T11:43:02.960 INFO:tasks.workunit.client.0.smithi100.stdout: 10 108631 8.89 MB/sec execute 536 sec latency 104.042 ms 2024-05-10T11:43:03.879 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:43:03 smithi178 ceph-mon[33359]: pgmap v1161: 129 pgs: 129 active+clean; 577 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 3.7 MiB/s wr, 257 op/s 2024-05-10T11:43:03.950 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:43:03.960 INFO:tasks.workunit.client.0.smithi100.stdout: 10 108789 8.89 MB/sec execute 537 sec latency 125.204 ms 2024-05-10T11:43:04.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:43:03 smithi188 ceph-mon[34075]: pgmap v1161: 129 pgs: 129 active+clean; 577 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 3.7 MiB/s wr, 257 op/s 2024-05-10T11:43:04.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:43:03 smithi100 ceph-mon[30296]: pgmap v1161: 129 pgs: 129 active+clean; 577 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 3.7 MiB/s wr, 257 op/s 2024-05-10T11:43:04.962 INFO:tasks.workunit.client.0.smithi100.stdout: 10 108926 8.88 MB/sec execute 538 sec latency 82.397 ms 2024-05-10T11:43:05.960 INFO:tasks.workunit.client.0.smithi100.stdout: 10 109076 8.88 MB/sec execute 539 sec latency 80.465 ms 2024-05-10T11:43:06.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:43:06 smithi178 ceph-mon[33359]: pgmap v1162: 129 pgs: 129 active+clean; 577 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 3.7 MiB/s wr, 213 op/s 2024-05-10T11:43:06.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:43:06 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:43:06.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:43:06 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:43:06.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:43:06 smithi188 ceph-mon[34075]: pgmap v1162: 129 pgs: 129 active+clean; 577 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 3.7 MiB/s wr, 213 op/s 2024-05-10T11:43:06.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:43:06 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:43:06.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:43:06 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:43:06.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:43:06 smithi100 ceph-mon[30296]: pgmap v1162: 129 pgs: 129 active+clean; 577 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 3.7 MiB/s wr, 213 op/s 2024-05-10T11:43:06.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:43:06 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:43:06.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:43:06 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:43:06.960 INFO:tasks.workunit.client.0.smithi100.stdout: 10 109216 8.88 MB/sec execute 540 sec latency 158.053 ms 2024-05-10T11:43:07.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.507+0000 7f7b37339700 1 -- 172.21.15.100:0/3485034497 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b30101a90 msgr2=0x7f7b30101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:07.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.507+0000 7f7b37339700 1 --2- 172.21.15.100:0/3485034497 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b30101a90 0x7f7b30101f10 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c009990 tx=0x7f7b2c009ca0 comp rx=0 tx=0).stop 2024-05-10T11:43:07.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.509+0000 7f7b37339700 1 -- 172.21.15.100:0/3485034497 shutdown_connections 2024-05-10T11:43:07.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.509+0000 7f7b37339700 1 --2- 172.21.15.100:0/3485034497 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b30102530 0x7f7b30112220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:07.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.509+0000 7f7b37339700 1 --2- 172.21.15.100:0/3485034497 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b301043b0 0x7f7b30104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:07.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.509+0000 7f7b37339700 1 --2- 172.21.15.100:0/3485034497 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b30101a90 0x7f7b30101f10 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:07.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.509+0000 7f7b37339700 1 -- 172.21.15.100:0/3485034497 >> 172.21.15.100:0/3485034497 conn(0x7f7b30078a10 msgr2=0x7f7b300ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:43:07.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.509+0000 7f7b37339700 1 -- 172.21.15.100:0/3485034497 shutdown_connections 2024-05-10T11:43:07.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.509+0000 7f7b37339700 1 -- 172.21.15.100:0/3485034497 wait complete. 2024-05-10T11:43:07.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.510+0000 7f7b37339700 1 Processor -- start 2024-05-10T11:43:07.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.510+0000 7f7b37339700 1 -- start start 2024-05-10T11:43:07.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.511+0000 7f7b37339700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b30101a90 0x7f7b30111e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:07.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.511+0000 7f7b37339700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b30102530 0x7f7b3010aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:07.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.511+0000 7f7b37339700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b301043b0 0x7f7b3010b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:07.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.511+0000 7f7b348d4700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b30102530 0x7f7b3010aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:07.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.511+0000 7f7b348d4700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b30102530 0x7f7b3010aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:47632/0 (socket says 172.21.15.100:47632) 2024-05-10T11:43:07.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.511+0000 7f7b358d6700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b301043b0 0x7f7b3010b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:07.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.511+0000 7f7b358d6700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b301043b0 0x7f7b3010b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41908/0 (socket says 172.21.15.100:41908) 2024-05-10T11:43:07.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.511+0000 7f7b358d6700 1 -- 172.21.15.100:0/3601874052 learned_addr learned my addr 172.21.15.100:0/3601874052 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:43:07.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.511+0000 7f7b37339700 1 -- 172.21.15.100:0/3601874052 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7b30115810 con 0x7f7b301043b0 2024-05-10T11:43:07.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.511+0000 7f7b37339700 1 -- 172.21.15.100:0/3601874052 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7b30115690 con 0x7f7b30101a90 2024-05-10T11:43:07.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.511+0000 7f7b37339700 1 -- 172.21.15.100:0/3601874052 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7b30115990 con 0x7f7b30102530 2024-05-10T11:43:07.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.512+0000 7f7b350d5700 1 --2- 172.21.15.100:0/3601874052 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b30101a90 0x7f7b30111e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:07.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.512+0000 7f7b358d6700 1 -- 172.21.15.100:0/3601874052 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b30102530 msgr2=0x7f7b3010aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:07.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.512+0000 7f7b358d6700 1 --2- 172.21.15.100:0/3601874052 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b30102530 0x7f7b3010aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:07.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.512+0000 7f7b358d6700 1 -- 172.21.15.100:0/3601874052 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b30101a90 msgr2=0x7f7b30111e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:07.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.512+0000 7f7b358d6700 1 --2- 172.21.15.100:0/3601874052 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b30101a90 0x7f7b30111e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:07.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.512+0000 7f7b358d6700 1 -- 172.21.15.100:0/3601874052 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b3010ba40 con 0x7f7b301043b0 2024-05-10T11:43:07.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.513+0000 7f7b358d6700 1 --2- 172.21.15.100:0/3601874052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b301043b0 0x7f7b3010b400 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f7b2000b550 tx=0x7f7b2000b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:43:07.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.513+0000 7f7b267fc700 1 -- 172.21.15.100:0/3601874052 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b20015020 con 0x7f7b301043b0 2024-05-10T11:43:07.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.514+0000 7f7b267fc700 1 -- 172.21.15.100:0/3601874052 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b20011cc0 con 0x7f7b301043b0 2024-05-10T11:43:07.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.514+0000 7f7b37339700 1 -- 172.21.15.100:0/3601874052 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b3010f960 con 0x7f7b301043b0 2024-05-10T11:43:07.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.514+0000 7f7b37339700 1 -- 172.21.15.100:0/3601874052 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b3010fe10 con 0x7f7b301043b0 2024-05-10T11:43:07.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.515+0000 7f7b267fc700 1 -- 172.21.15.100:0/3601874052 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b20010cb0 con 0x7f7b301043b0 2024-05-10T11:43:07.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.521+0000 7f7b267fc700 1 -- 172.21.15.100:0/3601874052 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f7b20018020 con 0x7f7b301043b0 2024-05-10T11:43:07.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.522+0000 7f7b267fc700 1 --2- 172.21.15.100:0/3601874052 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f7b1c0b28b0 0x7f7b1c0b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:07.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.522+0000 7f7b267fc700 1 -- 172.21.15.100:0/3601874052 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f7b200e1c50 con 0x7f7b301043b0 2024-05-10T11:43:07.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.523+0000 7f7b350d5700 1 --2- 172.21.15.100:0/3601874052 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f7b1c0b28b0 0x7f7b1c0b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:07.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.523+0000 7f7b350d5700 1 --2- 172.21.15.100:0/3601874052 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f7b1c0b28b0 0x7f7b1c0b4d70 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f7b28009300 tx=0x7f7b2800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:43:07.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.526+0000 7f7b41bccb80 1 Processor -- start 2024-05-10T11:43:07.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.526+0000 7f7b41bccb80 1 -- start start 2024-05-10T11:43:07.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.526+0000 7f7b41bccb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a3b0ed1b0 0x563a3b0ed590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:07.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.526+0000 7f7b41bccb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563a3b0edad0 0x563a3b0e7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:07.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.526+0000 7f7b41bccb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a3b0e7d50 0x563a3b0e8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:07.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.526+0000 7f7b41bccb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x563a3b039940 con 0x563a3b0ed1b0 2024-05-10T11:43:07.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.526+0000 7f7b41bccb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563a3af9eda0 con 0x563a3b0e7d50 2024-05-10T11:43:07.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.526+0000 7f7b41bccb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563a3b035900 con 0x563a3b0edad0 2024-05-10T11:43:07.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.526+0000 7f7b348d4700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563a3b0edad0 0x563a3b0e7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:07.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.526+0000 7f7b348d4700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563a3b0edad0 0x563a3b0e7810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:47636/0 (socket says 172.21.15.100:47636) 2024-05-10T11:43:07.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.526+0000 7f7b348d4700 1 -- 172.21.15.100:0/2109351910 learned_addr learned my addr 172.21.15.100:0/2109351910 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:43:07.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.526+0000 7f7b358d6700 1 --2- 172.21.15.100:0/2109351910 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a3b0e7d50 0x563a3b0e8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:07.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.527+0000 7f7b348d4700 1 -- 172.21.15.100:0/2109351910 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a3b0e7d50 msgr2=0x563a3b0e8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:43:07.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.527+0000 7f7b348d4700 1 --2- 172.21.15.100:0/2109351910 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a3b0e7d50 0x563a3b0e8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:07.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.527+0000 7f7b348d4700 1 -- 172.21.15.100:0/2109351910 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a3b0ed1b0 msgr2=0x563a3b0ed590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:43:07.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.527+0000 7f7b348d4700 1 --2- 172.21.15.100:0/2109351910 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a3b0ed1b0 0x563a3b0ed590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:07.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.527+0000 7f7b348d4700 1 -- 172.21.15.100:0/2109351910 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a3af868d0 con 0x563a3b0edad0 2024-05-10T11:43:07.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.527+0000 7f7b348d4700 1 --2- 172.21.15.100:0/2109351910 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563a3b0edad0 0x563a3b0e7810 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c000c00 tx=0x7f7b2c005040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7270334a7ee7981 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:43:07.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.528+0000 7f7b1b7fe700 1 -- 172.21.15.100:0/2109351910 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b2c027070 con 0x563a3b0edad0 2024-05-10T11:43:07.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.528+0000 7f7b1b7fe700 1 -- 172.21.15.100:0/2109351910 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b2c00bbe0 con 0x563a3b0edad0 2024-05-10T11:43:07.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.528+0000 7f7b1b7fe700 1 -- 172.21.15.100:0/2109351910 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b2c003d60 con 0x563a3b0edad0 2024-05-10T11:43:07.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.529+0000 7f7b41bccb80 1 -- 172.21.15.100:0/2109351910 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563a3b0edad0 msgr2=0x563a3b0e7810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:07.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.529+0000 7f7b41bccb80 1 --2- 172.21.15.100:0/2109351910 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563a3b0edad0 0x563a3b0e7810 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c000c00 tx=0x7f7b2c005040 comp rx=0 tx=0).stop 2024-05-10T11:43:07.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.530+0000 7f7b41bccb80 1 -- 172.21.15.100:0/2109351910 shutdown_connections 2024-05-10T11:43:07.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.530+0000 7f7b41bccb80 1 --2- 172.21.15.100:0/2109351910 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a3b0e7d50 0x563a3b0e8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:07.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.530+0000 7f7b41bccb80 1 --2- 172.21.15.100:0/2109351910 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a3b0ed1b0 0x563a3b0ed590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:07.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.530+0000 7f7b41bccb80 1 --2- 172.21.15.100:0/2109351910 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563a3b0edad0 0x563a3b0e7810 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:07.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.530+0000 7f7b41bccb80 1 -- 172.21.15.100:0/2109351910 >> 172.21.15.100:0/2109351910 conn(0x563a3b013b70 msgr2=0x563a3b0e22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:43:07.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.531+0000 7f7b41bccb80 1 -- 172.21.15.100:0/2109351910 shutdown_connections 2024-05-10T11:43:07.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.531+0000 7f7b41bccb80 1 -- 172.21.15.100:0/2109351910 wait complete. 2024-05-10T11:43:07.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.532+0000 7f7b41bccb80 1 -- start start 2024-05-10T11:43:07.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.532+0000 7f7b41bccb80 1 Processor -- start 2024-05-10T11:43:07.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.533+0000 7f7b41bccb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563a3b0e7d50 0x563a3b0f3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:07.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.533+0000 7f7b41bccb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a3b0ed1b0 0x563a3b0f0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:07.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.533+0000 7f7b41bccb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a3b0edad0 0x563a3b0f1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:07.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.533+0000 7f7b41bccb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x563a3b0f43f0 con 0x563a3b0edad0 2024-05-10T11:43:07.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.533+0000 7f7b41bccb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563a3aff2ed0 con 0x563a3b0ed1b0 2024-05-10T11:43:07.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.533+0000 7f7b350d5700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563a3b0e7d50 0x563a3b0f3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:07.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.533+0000 7f7b350d5700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563a3b0e7d50 0x563a3b0f3780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:47638/0 (socket says 172.21.15.100:47638) 2024-05-10T11:43:07.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.533+0000 7f7b350d5700 1 -- 172.21.15.100:0/2147020273 learned_addr learned my addr 172.21.15.100:0/2147020273 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:43:07.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.533+0000 7f7b348d4700 1 --2- 172.21.15.100:0/2147020273 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a3b0ed1b0 0x563a3b0f0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:07.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.533+0000 7f7b41bccb80 1 -- 172.21.15.100:0/2147020273 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563a3b0f4570 con 0x563a3b0e7d50 2024-05-10T11:43:07.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.534+0000 7f7b41bccb80 10 client.0 _ll_register_callbacks cb 0x563a3afcd5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:43:07.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.534+0000 7f7b350d5700 1 -- 172.21.15.100:0/2147020273 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a3b0ed1b0 msgr2=0x563a3b0f0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:43:07.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.534+0000 7f7b350d5700 1 --2- 172.21.15.100:0/2147020273 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a3b0ed1b0 0x563a3b0f0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:07.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.534+0000 7f7b350d5700 1 -- 172.21.15.100:0/2147020273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a3b0edad0 msgr2=0x563a3b0f1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:43:07.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.534+0000 7f7b350d5700 1 --2- 172.21.15.100:0/2147020273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a3b0edad0 0x563a3b0f1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:07.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.534+0000 7f7b350d5700 1 -- 172.21.15.100:0/2147020273 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a3aff0760 con 0x563a3b0e7d50 2024-05-10T11:43:07.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.535+0000 7f7b350d5700 1 --2- 172.21.15.100:0/2147020273 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563a3b0e7d50 0x563a3b0f3780 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f7b280169a0 tx=0x7f7b28009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58b2f8f639237c56 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:43:07.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.535+0000 7f7b18ff9700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:43:07.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.535+0000 7f7b18ff9700 1 -- 172.21.15.100:0/2147020273 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b28026060 con 0x563a3b0e7d50 2024-05-10T11:43:07.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.535+0000 7f7b41bccb80 1 -- 172.21.15.100:0/2147020273 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b2c00bbe0 con 0x563a3b0e7d50 2024-05-10T11:43:07.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.535+0000 7f7b18ff9700 1 -- 172.21.15.100:0/2147020273 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f7b280149e0 con 0x563a3b0e7d50 2024-05-10T11:43:07.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.536+0000 7f7b18ff9700 1 -- 172.21.15.100:0/2147020273 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f7b2802d050 con 0x563a3b0e7d50 2024-05-10T11:43:07.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.536+0000 7f7b41bccb80 10 client.29485 fetch_fsmap learned FSMap version 41 2024-05-10T11:43:07.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.536+0000 7f7b41bccb80 1 -- 172.21.15.100:0/2147020273 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x563a3b107570 con 0x563a3b0e7d50 2024-05-10T11:43:07.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.537+0000 7f7b18ff9700 1 -- 172.21.15.100:0/2147020273 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b2c00bbe0 con 0x563a3b0e7d50 2024-05-10T11:43:07.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.537+0000 7f7b18ff9700 1 -- 172.21.15.100:0/2147020273 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f7b28020b70 con 0x563a3b0e7d50 2024-05-10T11:43:07.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.537+0000 7f7b41bccb80 10 client.29485 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:43:07.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.537+0000 7f7b41bccb80 10 client.29485 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-10T11:43:07.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.537+0000 7f7b41bccb80 20 client.29485 populate_metadata read hostname 'smithi100' 2024-05-10T11:43:07.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.537+0000 7f7b41bccb80 1 --2- 172.21.15.100:0/2147020273 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x563a3b109bb0 0x563a3b10bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:07.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.537+0000 7f7b41bccb80 4 client.29485 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:43:07.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.537+0000 7f7b41bccb80 1 -- 172.21.15.100:0/2147020273 --> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a3b047320 con 0x563a3b109bb0 2024-05-10T11:43:07.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.537+0000 7f7b348d4700 1 --2- 172.21.15.100:0/2147020273 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x563a3b109bb0 0x563a3b10bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:07.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.538+0000 7f7b348d4700 1 --2- 172.21.15.100:0/2147020273 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x563a3b109bb0 0x563a3b10bfa0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a9b1f7929f24431a server_cookie=5d2b9a00be6a6d75 in_seq=0 out_seq=0 2024-05-10T11:43:07.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.538+0000 7f7b18ff9700 10 client.29485 ms_handle_connect on v2:172.21.15.178:6838/3760985404 2024-05-10T11:43:07.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.540+0000 7f7b18ff9700 1 -- 172.21.15.100:0/2147020273 <== mds.1 v2:172.21.15.178:6838/3760985404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563a3b047320 con 0x563a3b109bb0 2024-05-10T11:43:07.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.540+0000 7f7b18ff9700 10 client.29485 handle_command_reply: tid=0 2024-05-10T11:43:07.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.540+0000 7f7b41bccb80 1 client.29485 shutdown 2024-05-10T11:43:07.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.540+0000 7f7b41bccb80 1 -- 172.21.15.100:0/2147020273 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563a3b0e7d50 msgr2=0x563a3b0f3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:07.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.540+0000 7f7b41bccb80 1 --2- 172.21.15.100:0/2147020273 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563a3b0e7d50 0x563a3b0f3780 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f7b280169a0 tx=0x7f7b28009920 comp rx=0 tx=0).stop 2024-05-10T11:43:07.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.540+0000 7f7b41bccb80 1 -- 172.21.15.100:0/2147020273 shutdown_connections 2024-05-10T11:43:07.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.540+0000 7f7b41bccb80 1 --2- 172.21.15.100:0/2147020273 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x563a3b109bb0 0x563a3b10bfa0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:07.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.541+0000 7f7b41bccb80 1 --2- 172.21.15.100:0/2147020273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a3b0edad0 0x563a3b0f1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:07.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.541+0000 7f7b41bccb80 1 --2- 172.21.15.100:0/2147020273 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563a3b0e7d50 0x563a3b0f3780 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:07.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.541+0000 7f7b41bccb80 1 --2- 172.21.15.100:0/2147020273 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a3b0ed1b0 0x563a3b0f0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:07.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.541+0000 7f7b41bccb80 1 -- 172.21.15.100:0/2147020273 >> 172.21.15.100:0/2147020273 conn(0x563a3b013b70 msgr2=0x563a3b0dcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:43:07.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.541+0000 7f7b18ff9700 0 client.29485 ms_handle_reset on v2:172.21.15.178:6838/3760985404 2024-05-10T11:43:07.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.541+0000 7f7b41bccb80 1 -- 172.21.15.100:0/2147020273 shutdown_connections 2024-05-10T11:43:07.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.541+0000 7f7b41bccb80 1 -- 172.21.15.100:0/2147020273 wait complete. 2024-05-10T11:43:07.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.541+0000 7f7b41bccb80 20 client.29485 trim_cache size 0 max 16384 2024-05-10T11:43:07.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.572+0000 7f7b41bccb80 1 Processor -- start 2024-05-10T11:43:07.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.572+0000 7f7b41bccb80 1 -- start start 2024-05-10T11:43:07.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.573+0000 7f7b41bccb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a3b0e7d50 0x563a3b0f10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:07.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.573+0000 7f7b41bccb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563a3b0ed1b0 0x563a3b0ee770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:07.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.573+0000 7f7b41bccb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a3b0edad0 0x563a3b0f2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:07.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.573+0000 7f7b348d4700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563a3b0ed1b0 0x563a3b0ee770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:07.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.573+0000 7f7b348d4700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563a3b0ed1b0 0x563a3b0ee770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:47650/0 (socket says 172.21.15.100:47650) 2024-05-10T11:43:07.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.573+0000 7f7b348d4700 1 -- 172.21.15.100:0/3925116185 learned_addr learned my addr 172.21.15.100:0/3925116185 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:43:07.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.573+0000 7f7b358d6700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a3b0edad0 0x563a3b0f2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:07.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.573+0000 7f7b350d5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a3b0e7d50 0x563a3b0f10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:07.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.573+0000 7f7b41bccb80 1 -- 172.21.15.100:0/3925116185 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7b301b7d50 con 0x563a3b0edad0 2024-05-10T11:43:07.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.574+0000 7f7b41bccb80 1 -- 172.21.15.100:0/3925116185 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7b301b55e0 con 0x563a3b0e7d50 2024-05-10T11:43:07.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.574+0000 7f7b41bccb80 1 -- 172.21.15.100:0/3925116185 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7b30108da0 con 0x563a3b0ed1b0 2024-05-10T11:43:07.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.574+0000 7f7b350d5700 1 -- 172.21.15.100:0/3925116185 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563a3b0ed1b0 msgr2=0x563a3b0ee770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:43:07.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.574+0000 7f7b350d5700 1 --2- 172.21.15.100:0/3925116185 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563a3b0ed1b0 0x563a3b0ee770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:07.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.574+0000 7f7b350d5700 1 -- 172.21.15.100:0/3925116185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a3b0edad0 msgr2=0x563a3b0f2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:43:07.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.574+0000 7f7b350d5700 1 --2- 172.21.15.100:0/3925116185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a3b0edad0 0x563a3b0f2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:07.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.574+0000 7f7b350d5700 1 -- 172.21.15.100:0/3925116185 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a3b0dec80 con 0x563a3b0e7d50 2024-05-10T11:43:07.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.575+0000 7f7b350d5700 1 --2- 172.21.15.100:0/3925116185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a3b0e7d50 0x563a3b0f10b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f7b28013420 tx=0x7f7b28016930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b150596f5f13643 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:43:07.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.575+0000 7f7b1affd700 1 -- 172.21.15.100:0/3925116185 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b28035020 con 0x563a3b0e7d50 2024-05-10T11:43:07.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.575+0000 7f7b1affd700 1 -- 172.21.15.100:0/3925116185 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b28004db0 con 0x563a3b0e7d50 2024-05-10T11:43:07.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.575+0000 7f7b1affd700 1 -- 172.21.15.100:0/3925116185 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b2802faf0 con 0x563a3b0e7d50 2024-05-10T11:43:07.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.576+0000 7f7b41bccb80 1 -- 172.21.15.100:0/3925116185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a3b0e7d50 msgr2=0x563a3b0f10b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:07.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.576+0000 7f7b41bccb80 1 --2- 172.21.15.100:0/3925116185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a3b0e7d50 0x563a3b0f10b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f7b28013420 tx=0x7f7b28016930 comp rx=0 tx=0).stop 2024-05-10T11:43:07.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.576+0000 7f7b41bccb80 1 -- 172.21.15.100:0/3925116185 shutdown_connections 2024-05-10T11:43:07.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.576+0000 7f7b41bccb80 1 --2- 172.21.15.100:0/3925116185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a3b0edad0 0x563a3b0f2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:07.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.576+0000 7f7b41bccb80 1 --2- 172.21.15.100:0/3925116185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a3b0e7d50 0x563a3b0f10b0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:07.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.576+0000 7f7b41bccb80 1 --2- 172.21.15.100:0/3925116185 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563a3b0ed1b0 0x563a3b0ee770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:07.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.576+0000 7f7b41bccb80 1 -- 172.21.15.100:0/3925116185 >> 172.21.15.100:0/3925116185 conn(0x563a3b013b70 msgr2=0x563a3b0eb0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:43:07.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.576+0000 7f7b41bccb80 1 -- 172.21.15.100:0/3925116185 shutdown_connections 2024-05-10T11:43:07.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.576+0000 7f7b41bccb80 1 -- 172.21.15.100:0/3925116185 wait complete. 2024-05-10T11:43:07.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.577+0000 7f7b41bccb80 1 -- start start 2024-05-10T11:43:07.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.577+0000 7f7b41bccb80 1 Processor -- start 2024-05-10T11:43:07.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.578+0000 7f7b41bccb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a3b0e7d50 0x563a3b111e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:07.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.578+0000 7f7b41bccb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563a3b0ed1b0 0x563a3b1020c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:07.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.578+0000 7f7b41bccb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a3b0edad0 0x563a3b102740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:07.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.578+0000 7f7b41bccb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7b301b3aa0 con 0x563a3b0e7d50 2024-05-10T11:43:07.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.578+0000 7f7b41bccb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7b301b4f10 con 0x563a3b0edad0 2024-05-10T11:43:07.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.578+0000 7f7b41bccb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7b301b4170 con 0x563a3b0ed1b0 2024-05-10T11:43:07.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.578+0000 7f7b358d6700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a3b0edad0 0x563a3b102740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:07.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.578+0000 7f7b358d6700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a3b0edad0 0x563a3b102740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:56554/0 (socket says 172.21.15.100:56554) 2024-05-10T11:43:07.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.578+0000 7f7b358d6700 1 -- 172.21.15.100:0/586636644 learned_addr learned my addr 172.21.15.100:0/586636644 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:43:07.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.578+0000 7f7b350d5700 1 --2- 172.21.15.100:0/586636644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a3b0e7d50 0x563a3b111e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:07.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.578+0000 7f7b41bccb80 10 client.0 _ll_register_callbacks cb 0x563a3afcd5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:43:07.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.578+0000 7f7b348d4700 1 --2- 172.21.15.100:0/586636644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563a3b0ed1b0 0x563a3b1020c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:07.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.579+0000 7f7b358d6700 1 -- 172.21.15.100:0/586636644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563a3b0ed1b0 msgr2=0x563a3b1020c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:43:07.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.579+0000 7f7b358d6700 1 --2- 172.21.15.100:0/586636644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563a3b0ed1b0 0x563a3b1020c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:07.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.579+0000 7f7b358d6700 1 -- 172.21.15.100:0/586636644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a3b0e7d50 msgr2=0x563a3b111e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:43:07.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.579+0000 7f7b358d6700 1 --2- 172.21.15.100:0/586636644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a3b0e7d50 0x563a3b111e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:07.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.579+0000 7f7b358d6700 1 -- 172.21.15.100:0/586636644 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a3b110aa0 con 0x563a3b0edad0 2024-05-10T11:43:07.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.579+0000 7f7b350d5700 1 --2- 172.21.15.100:0/586636644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a3b0e7d50 0x563a3b111e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:43:07.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.579+0000 7f7b348d4700 1 --2- 172.21.15.100:0/586636644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563a3b0ed1b0 0x563a3b1020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:43:07.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.579+0000 7f7b358d6700 1 --2- 172.21.15.100:0/586636644 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a3b0edad0 0x563a3b102740 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7b200a4f30 tx=0x7f7b20011a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6de41217796b722 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:43:07.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.579+0000 7f7b18ff9700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:43:07.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.580+0000 7f7b18ff9700 1 -- 172.21.15.100:0/586636644 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b20009280 con 0x563a3b0edad0 2024-05-10T11:43:07.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.580+0000 7f7b18ff9700 1 -- 172.21.15.100:0/586636644 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f7b200a9110 con 0x563a3b0edad0 2024-05-10T11:43:07.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.580+0000 7f7b41bccb80 1 -- 172.21.15.100:0/586636644 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b28004db0 con 0x563a3b0edad0 2024-05-10T11:43:07.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.580+0000 7f7b18ff9700 1 -- 172.21.15.100:0/586636644 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f7b200a8eb0 con 0x563a3b0edad0 2024-05-10T11:43:07.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.580+0000 7f7b41bccb80 10 client.29378 fetch_fsmap learned FSMap version 41 2024-05-10T11:43:07.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.580+0000 7f7b41bccb80 1 -- 172.21.15.100:0/586636644 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x563a3b173990 con 0x563a3b0edad0 2024-05-10T11:43:07.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.580+0000 7f7b18ff9700 1 -- 172.21.15.100:0/586636644 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b28004db0 con 0x563a3b0edad0 2024-05-10T11:43:07.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.580+0000 7f7b18ff9700 1 -- 172.21.15.100:0/586636644 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f7b200a4820 con 0x563a3b0edad0 2024-05-10T11:43:07.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.580+0000 7f7b41bccb80 10 client.29378 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:43:07.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.580+0000 7f7b41bccb80 10 client.29378 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-10T11:43:07.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.580+0000 7f7b41bccb80 20 client.29378 populate_metadata read hostname 'smithi100' 2024-05-10T11:43:07.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.580+0000 7f7b41bccb80 1 --2- 172.21.15.100:0/586636644 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x563a3b174110 0x563a3b176530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:07.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.580+0000 7f7b41bccb80 4 client.29378 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:43:07.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.580+0000 7f7b41bccb80 1 -- 172.21.15.100:0/586636644 --> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563a3b0efeb0 con 0x563a3b174110 2024-05-10T11:43:07.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.581+0000 7f7b350d5700 1 --2- 172.21.15.100:0/586636644 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x563a3b174110 0x563a3b176530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:07.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.581+0000 7f7b18ff9700 10 client.29378 ms_handle_connect on v2:172.21.15.178:6838/3760985404 2024-05-10T11:43:07.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.581+0000 7f7b350d5700 1 --2- 172.21.15.100:0/586636644 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x563a3b174110 0x563a3b176530 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e49ad3a7cf13a18b server_cookie=43faf63212880cc8 in_seq=0 out_seq=0 2024-05-10T11:43:07.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.582+0000 7f7b18ff9700 1 -- 172.21.15.100:0/586636644 <== mds.1 v2:172.21.15.178:6838/3760985404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563a3b0efeb0 con 0x563a3b174110 2024-05-10T11:43:07.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.582+0000 7f7b18ff9700 10 client.29378 handle_command_reply: tid=0 2024-05-10T11:43:07.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.582+0000 7f7b41bccb80 1 client.29378 shutdown 2024-05-10T11:43:07.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.582+0000 7f7b41bccb80 1 -- 172.21.15.100:0/586636644 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a3b0edad0 msgr2=0x563a3b102740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:07.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.582+0000 7f7b41bccb80 1 --2- 172.21.15.100:0/586636644 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a3b0edad0 0x563a3b102740 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7b200a4f30 tx=0x7f7b20011a90 comp rx=0 tx=0).stop 2024-05-10T11:43:07.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.582+0000 7f7b41bccb80 1 -- 172.21.15.100:0/586636644 shutdown_connections 2024-05-10T11:43:07.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.582+0000 7f7b41bccb80 1 --2- 172.21.15.100:0/586636644 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x563a3b174110 0x563a3b176530 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:07.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.582+0000 7f7b41bccb80 1 --2- 172.21.15.100:0/586636644 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a3b0edad0 0x563a3b102740 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:07.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.582+0000 7f7b41bccb80 1 --2- 172.21.15.100:0/586636644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a3b0e7d50 0x563a3b111e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:07.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.582+0000 7f7b41bccb80 1 --2- 172.21.15.100:0/586636644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563a3b0ed1b0 0x563a3b1020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:07.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.582+0000 7f7b18ff9700 0 client.29378 ms_handle_reset on v2:172.21.15.178:6838/3760985404 2024-05-10T11:43:07.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.582+0000 7f7b41bccb80 1 -- 172.21.15.100:0/586636644 >> 172.21.15.100:0/586636644 conn(0x563a3b013b70 msgr2=0x563a3b0ebcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:43:07.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.583+0000 7f7b41bccb80 1 -- 172.21.15.100:0/586636644 shutdown_connections 2024-05-10T11:43:07.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.583+0000 7f7b41bccb80 1 -- 172.21.15.100:0/586636644 wait complete. 2024-05-10T11:43:07.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.583+0000 7f7b41bccb80 20 client.29378 trim_cache size 0 max 16384 2024-05-10T11:43:07.586 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:43:07.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.583+0000 7f7b37339700 1 -- 172.21.15.100:0/3601874052 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f7b1c0b28b0 msgr2=0x7f7b1c0b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:07.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.583+0000 7f7b37339700 1 --2- 172.21.15.100:0/3601874052 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f7b1c0b28b0 0x7f7b1c0b4d70 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f7b28009300 tx=0x7f7b2800b040 comp rx=0 tx=0).stop 2024-05-10T11:43:07.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.583+0000 7f7b37339700 1 -- 172.21.15.100:0/3601874052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b301043b0 msgr2=0x7f7b3010b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:07.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.583+0000 7f7b37339700 1 --2- 172.21.15.100:0/3601874052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b301043b0 0x7f7b3010b400 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f7b2000b550 tx=0x7f7b2000b860 comp rx=0 tx=0).stop 2024-05-10T11:43:07.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.584+0000 7f7b37339700 1 -- 172.21.15.100:0/3601874052 shutdown_connections 2024-05-10T11:43:07.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.584+0000 7f7b37339700 1 --2- 172.21.15.100:0/3601874052 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f7b1c0b28b0 0x7f7b1c0b4d70 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:07.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.584+0000 7f7b37339700 1 --2- 172.21.15.100:0/3601874052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b301043b0 0x7f7b3010b400 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:07.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.584+0000 7f7b37339700 1 --2- 172.21.15.100:0/3601874052 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b30101a90 0x7f7b30111e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:07.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.584+0000 7f7b37339700 1 --2- 172.21.15.100:0/3601874052 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b30102530 0x7f7b3010aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:07.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.584+0000 7f7b37339700 1 -- 172.21.15.100:0/3601874052 >> 172.21.15.100:0/3601874052 conn(0x7f7b30078a10 msgr2=0x7f7b300ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:43:07.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.584+0000 7f7b37339700 1 -- 172.21.15.100:0/3601874052 shutdown_connections 2024-05-10T11:43:07.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:07.584+0000 7f7b37339700 1 -- 172.21.15.100:0/3601874052 wait complete. 2024-05-10T11:43:07.960 INFO:tasks.workunit.client.0.smithi100.stdout: 10 109356 8.87 MB/sec execute 541 sec latency 103.989 ms 2024-05-10T11:43:08.265 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:43:08 smithi100 ceph-mon[30296]: pgmap v1163: 129 pgs: 129 active+clean; 575 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 4.1 MiB/s wr, 250 op/s 2024-05-10T11:43:08.314 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-10T11:43:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.322+0000 7fbc961d4640 1 -- 172.21.15.100:0/2097386271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc90103c10 msgr2=0x7fbc90104090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.322+0000 7fbc961d4640 1 --2- 172.21.15.100:0/2097386271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc90103c10 0x7fbc90104090 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fbc84009900 tx=0x7fbc8402a410 comp rx=0 tx=0).stop 2024-05-10T11:43:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.322+0000 7fbc961d4640 1 -- 172.21.15.100:0/2097386271 shutdown_connections 2024-05-10T11:43:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.322+0000 7fbc961d4640 1 --2- 172.21.15.100:0/2097386271 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc901045d0 0x7fbc9010ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.322+0000 7fbc961d4640 1 --2- 172.21.15.100:0/2097386271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc90103c10 0x7fbc90104090 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.322+0000 7fbc961d4640 1 --2- 172.21.15.100:0/2097386271 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc90102a20 0x7fbc90102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.322+0000 7fbc961d4640 1 -- 172.21.15.100:0/2097386271 >> 172.21.15.100:0/2097386271 conn(0x7fbc900fdf90 msgr2=0x7fbc901003d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:43:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.322+0000 7fbc961d4640 1 -- 172.21.15.100:0/2097386271 shutdown_connections 2024-05-10T11:43:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.322+0000 7fbc961d4640 1 -- 172.21.15.100:0/2097386271 wait complete. 2024-05-10T11:43:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.323+0000 7fbc961d4640 1 Processor -- start 2024-05-10T11:43:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.323+0000 7fbc961d4640 1 -- start start 2024-05-10T11:43:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.323+0000 7fbc961d4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc90102a20 0x7fbc9019acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.323+0000 7fbc961d4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc90103c10 0x7fbc9019b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.323+0000 7fbc961d4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc901045d0 0x7fbc901a2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.323+0000 7fbc961d4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9010e3e0 con 0x7fbc90102a20 2024-05-10T11:43:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.323+0000 7fbc961d4640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9010e260 con 0x7fbc901045d0 2024-05-10T11:43:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.323+0000 7fbc961d4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9010e560 con 0x7fbc90103c10 2024-05-10T11:43:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.323+0000 7fbc8effd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc90103c10 0x7fbc9019b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.323+0000 7fbc8f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc90102a20 0x7fbc9019acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.323+0000 7fbc8effd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc90103c10 0x7fbc9019b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:47678/0 (socket says 172.21.15.100:47678) 2024-05-10T11:43:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.323+0000 7fbc8effd640 1 -- 172.21.15.100:0/3816415653 learned_addr learned my addr 172.21.15.100:0/3816415653 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:43:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.323+0000 7fbc8f7fe640 1 --2- 172.21.15.100:0/3816415653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc90102a20 0x7fbc9019acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41972/0 (socket says 172.21.15.100:41972) 2024-05-10T11:43:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.323+0000 7fbc8ffff640 1 --2- 172.21.15.100:0/3816415653 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc901045d0 0x7fbc901a2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.324+0000 7fbc8effd640 1 -- 172.21.15.100:0/3816415653 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc901045d0 msgr2=0x7fbc901a2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.324+0000 7fbc8effd640 1 --2- 172.21.15.100:0/3816415653 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc901045d0 0x7fbc901a2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.324+0000 7fbc8effd640 1 -- 172.21.15.100:0/3816415653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc90102a20 msgr2=0x7fbc9019acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.324+0000 7fbc8effd640 1 --2- 172.21.15.100:0/3816415653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc90102a20 0x7fbc9019acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.324+0000 7fbc8effd640 1 -- 172.21.15.100:0/3816415653 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc901a2900 con 0x7fbc90103c10 2024-05-10T11:43:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.324+0000 7fbc8ffff640 1 --2- 172.21.15.100:0/3816415653 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc901045d0 0x7fbc901a2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:43:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.324+0000 7fbc8effd640 1 --2- 172.21.15.100:0/3816415653 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc90103c10 0x7fbc9019b230 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fbc84009230 tx=0x7fbc840042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:43:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.324+0000 7fbc8cff9640 1 -- 172.21.15.100:0/3816415653 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc8403f030 con 0x7fbc90103c10 2024-05-10T11:43:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.324+0000 7fbc8cff9640 1 -- 172.21.15.100:0/3816415653 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbc84032e10 con 0x7fbc90103c10 2024-05-10T11:43:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.324+0000 7fbc961d4640 1 -- 172.21.15.100:0/3816415653 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc901a2b90 con 0x7fbc90103c10 2024-05-10T11:43:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.324+0000 7fbc961d4640 1 -- 172.21.15.100:0/3816415653 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc901a30d0 con 0x7fbc90103c10 2024-05-10T11:43:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.324+0000 7fbc8cff9640 1 -- 172.21.15.100:0/3816415653 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc84035070 con 0x7fbc90103c10 2024-05-10T11:43:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.327+0000 7fbc98fcd740 1 Processor -- start 2024-05-10T11:43:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.327+0000 7fbc98fcd740 1 -- start start 2024-05-10T11:43:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.328+0000 7fbc98fcd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c08f9bd5c0 0x55c08f9bd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.328+0000 7fbc98fcd740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c08f9bdfb0 0x55c08f9c2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.328+0000 7fbc98fcd740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c08f9c3120 0x55c08f9c55b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.328+0000 7fbc98fcd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55c08f9195d0 con 0x55c08f9bd5c0 2024-05-10T11:43:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.328+0000 7fbc98fcd740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c08f8d2890 con 0x55c08f9c3120 2024-05-10T11:43:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.328+0000 7fbc98fcd740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c08f915590 con 0x55c08f9bdfb0 2024-05-10T11:43:08.329 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.328+0000 7fbc8cff9640 1 -- 172.21.15.100:0/3816415653 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fbc84042030 con 0x7fbc90103c10 2024-05-10T11:43:08.329 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.328+0000 7fbc8f7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c08f9bdfb0 0x55c08f9c2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:08.329 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.328+0000 7fbc8f7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c08f9bdfb0 0x55c08f9c2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:47680/0 (socket says 172.21.15.100:47680) 2024-05-10T11:43:08.329 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.328+0000 7fbc8f7fe640 1 -- 172.21.15.100:0/237725320 learned_addr learned my addr 172.21.15.100:0/237725320 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:43:08.329 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.328+0000 7fbc8ffff640 1 --2- 172.21.15.100:0/237725320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c08f9bd5c0 0x55c08f9bd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:08.329 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.328+0000 7fbc8effd640 1 --2- 172.21.15.100:0/237725320 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c08f9c3120 0x55c08f9c55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:08.329 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.328+0000 7fbc8f7fe640 1 -- 172.21.15.100:0/237725320 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c08f9c3120 msgr2=0x55c08f9c55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:43:08.332 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.328+0000 7fbc8f7fe640 1 --2- 172.21.15.100:0/237725320 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c08f9c3120 0x55c08f9c55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:08.332 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.328+0000 7fbc8f7fe640 1 -- 172.21.15.100:0/237725320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c08f9bd5c0 msgr2=0x55c08f9bd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:43:08.332 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.328+0000 7fbc8f7fe640 1 --2- 172.21.15.100:0/237725320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c08f9bd5c0 0x55c08f9bd9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:08.332 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.328+0000 7fbc8f7fe640 1 -- 172.21.15.100:0/237725320 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c08f62b210 con 0x55c08f9bdfb0 2024-05-10T11:43:08.332 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.328+0000 7fbc8ffff640 1 --2- 172.21.15.100:0/237725320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c08f9bd5c0 0x55c08f9bd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:43:08.332 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.329+0000 7fbc8f7fe640 1 --2- 172.21.15.100:0/237725320 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c08f9bdfb0 0x55c08f9c2ab0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fbc80009c60 tx=0x7fbc80011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c76f42d19563c81 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:43:08.332 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.329+0000 7fbc75ffb640 1 -- 172.21.15.100:0/237725320 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc80004220 con 0x55c08f9bdfb0 2024-05-10T11:43:08.332 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.329+0000 7fbc75ffb640 1 -- 172.21.15.100:0/237725320 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbc800043c0 con 0x55c08f9bdfb0 2024-05-10T11:43:08.332 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.329+0000 7fbc75ffb640 1 -- 172.21.15.100:0/237725320 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc80004ce0 con 0x55c08f9bdfb0 2024-05-10T11:43:08.332 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.329+0000 7fbc8cff9640 1 --2- 172.21.15.100:0/3816415653 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fbc780b2920 0x7fbc780b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:08.332 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.330+0000 7fbc98fcd740 1 -- 172.21.15.100:0/237725320 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c08f9bdfb0 msgr2=0x55c08f9c2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:08.332 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.330+0000 7fbc98fcd740 1 --2- 172.21.15.100:0/237725320 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c08f9bdfb0 0x55c08f9c2ab0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fbc80009c60 tx=0x7fbc80011810 comp rx=0 tx=0).stop 2024-05-10T11:43:08.332 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.330+0000 7fbc8ffff640 1 --2- 172.21.15.100:0/3816415653 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fbc780b2920 0x7fbc780b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:08.333 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.330+0000 7fbc98fcd740 1 -- 172.21.15.100:0/237725320 shutdown_connections 2024-05-10T11:43:08.333 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.330+0000 7fbc98fcd740 1 --2- 172.21.15.100:0/237725320 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c08f9c3120 0x55c08f9c55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:08.333 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.330+0000 7fbc98fcd740 1 --2- 172.21.15.100:0/237725320 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c08f9bdfb0 0x55c08f9c2ab0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:08.333 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.330+0000 7fbc8cff9640 1 -- 172.21.15.100:0/3816415653 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fbc84102b80 con 0x7fbc90103c10 2024-05-10T11:43:08.333 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.330+0000 7fbc98fcd740 1 --2- 172.21.15.100:0/237725320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c08f9bd5c0 0x55c08f9bd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:08.333 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.330+0000 7fbc98fcd740 1 -- 172.21.15.100:0/237725320 >> 172.21.15.100:0/237725320 conn(0x55c08f9b8c30 msgr2=0x55c08f9b9030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:43:08.333 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.330+0000 7fbc98fcd740 1 -- 172.21.15.100:0/237725320 shutdown_connections 2024-05-10T11:43:08.333 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.330+0000 7fbc8ffff640 1 --2- 172.21.15.100:0/3816415653 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fbc780b2920 0x7fbc780b4de0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x55c08f9be810 tx=0x7fbc88009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:43:08.333 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.330+0000 7fbc98fcd740 1 -- 172.21.15.100:0/237725320 wait complete. 2024-05-10T11:43:08.333 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.331+0000 7fbc98fcd740 1 -- start start 2024-05-10T11:43:08.333 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.331+0000 7fbc98fcd740 1 Processor -- start 2024-05-10T11:43:08.333 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.331+0000 7fbc98fcd740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c08f9bd5c0 0x55c08f9c7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:08.333 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.331+0000 7fbc98fcd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c08f9bdfb0 0x55c08f9c7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:08.333 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.331+0000 7fbc98fcd740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c08f9c3120 0x55c08f9ced80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:08.333 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.331+0000 7fbc98fcd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55c08f9c6120 con 0x55c08f9bdfb0 2024-05-10T11:43:08.334 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.331+0000 7fbc98fcd740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c08f8d6370 con 0x55c08f9bd5c0 2024-05-10T11:43:08.334 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.331+0000 7fbc98fcd740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c08f9c62a0 con 0x55c08f9c3120 2024-05-10T11:43:08.334 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.331+0000 7fbc98fcd740 10 client.0 _ll_register_callbacks cb 0x55c08f62aa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:43:08.334 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.331+0000 7fbc8ffff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c08f9c3120 0x55c08f9ced80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:08.334 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.331+0000 7fbc8ffff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c08f9c3120 0x55c08f9ced80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:47690/0 (socket says 172.21.15.100:47690) 2024-05-10T11:43:08.334 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.331+0000 7fbc8effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c08f9bdfb0 0x55c08f9c7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:08.334 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.331+0000 7fbc8ffff640 1 -- 172.21.15.100:0/503565085 learned_addr learned my addr 172.21.15.100:0/503565085 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:43:08.334 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.332+0000 7fbc8f7fe640 1 --2- 172.21.15.100:0/503565085 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c08f9bd5c0 0x55c08f9c7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:08.334 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.332+0000 7fbc8ffff640 1 -- 172.21.15.100:0/503565085 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c08f9bd5c0 msgr2=0x55c08f9c7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:43:08.334 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.332+0000 7fbc8ffff640 1 --2- 172.21.15.100:0/503565085 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c08f9bd5c0 0x55c08f9c7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:08.334 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.332+0000 7fbc8ffff640 1 -- 172.21.15.100:0/503565085 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c08f9bdfb0 msgr2=0x55c08f9c7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:43:08.334 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.332+0000 7fbc8ffff640 1 --2- 172.21.15.100:0/503565085 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c08f9bdfb0 0x55c08f9c7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:08.334 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.332+0000 7fbc8ffff640 1 -- 172.21.15.100:0/503565085 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c08f6f7620 con 0x55c08f9c3120 2024-05-10T11:43:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.332+0000 7fbc8ffff640 1 --2- 172.21.15.100:0/503565085 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c08f9c3120 0x55c08f9ced80 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fbc88011ad0 tx=0x7fbc88004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0c0d7d5263314a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:43:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.332+0000 7fbc6effd640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:43:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.332+0000 7fbc6effd640 1 -- 172.21.15.100:0/503565085 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc8802d060 con 0x55c08f9c3120 2024-05-10T11:43:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.332+0000 7fbc98fcd740 1 -- 172.21.15.100:0/503565085 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc800043c0 con 0x55c08f9c3120 2024-05-10T11:43:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.332+0000 7fbc6effd640 1 -- 172.21.15.100:0/503565085 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fbc88017030 con 0x55c08f9c3120 2024-05-10T11:43:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.333+0000 7fbc6effd640 1 -- 172.21.15.100:0/503565085 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fbc88034050 con 0x55c08f9c3120 2024-05-10T11:43:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.333+0000 7fbc98fcd740 10 client.29503 fetch_fsmap learned FSMap version 41 2024-05-10T11:43:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.333+0000 7fbc98fcd740 1 -- 172.21.15.100:0/503565085 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55c08f9d1e80 con 0x55c08f9c3120 2024-05-10T11:43:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.333+0000 7fbc6effd640 1 -- 172.21.15.100:0/503565085 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbc800043c0 con 0x55c08f9c3120 2024-05-10T11:43:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.333+0000 7fbc6effd640 1 -- 172.21.15.100:0/503565085 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fbc88028720 con 0x55c08f9c3120 2024-05-10T11:43:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.333+0000 7fbc98fcd740 10 client.29503 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:43:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.333+0000 7fbc98fcd740 10 client.29503 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:43:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.333+0000 7fbc98fcd740 20 client.29503 populate_metadata read hostname 'smithi100' 2024-05-10T11:43:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.333+0000 7fbc98fcd740 1 --2- 172.21.15.100:0/503565085 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c08f9d3570 0x55c08f9d59a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.333+0000 7fbc98fcd740 4 client.29503 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:43:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.333+0000 7fbc98fcd740 1 -- 172.21.15.100:0/503565085 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c08f9271e0 con 0x55c08f9d3570 2024-05-10T11:43:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.333+0000 7fbc8f7fe640 1 --2- 172.21.15.100:0/503565085 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c08f9d3570 0x55c08f9d59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.334+0000 7fbc6effd640 10 client.29503 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:43:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.334+0000 7fbc8f7fe640 1 --2- 172.21.15.100:0/503565085 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c08f9d3570 0x55c08f9d59a0 crc :-1 s=READY pgs=457 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=102d7b93f362c707 server_cookie=6b68172d4ff4b22 in_seq=0 out_seq=0 2024-05-10T11:43:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.335+0000 7fbc6effd640 1 -- 172.21.15.100:0/503565085 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c08f9271e0 con 0x55c08f9d3570 2024-05-10T11:43:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.335+0000 7fbc6effd640 10 client.29503 handle_command_reply: tid=0 2024-05-10T11:43:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.335+0000 7fbc98fcd740 1 client.29503 shutdown 2024-05-10T11:43:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.335+0000 7fbc98fcd740 1 -- 172.21.15.100:0/503565085 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c08f9c3120 msgr2=0x55c08f9ced80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.335+0000 7fbc98fcd740 1 --2- 172.21.15.100:0/503565085 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c08f9c3120 0x55c08f9ced80 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fbc88011ad0 tx=0x7fbc88004ab0 comp rx=0 tx=0).stop 2024-05-10T11:43:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.335+0000 7fbc98fcd740 1 -- 172.21.15.100:0/503565085 shutdown_connections 2024-05-10T11:43:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.335+0000 7fbc98fcd740 1 --2- 172.21.15.100:0/503565085 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c08f9d3570 0x55c08f9d59a0 crc :-1 s=READY pgs=457 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.335+0000 7fbc98fcd740 1 --2- 172.21.15.100:0/503565085 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c08f9c3120 0x55c08f9ced80 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.335+0000 7fbc98fcd740 1 --2- 172.21.15.100:0/503565085 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c08f9bdfb0 0x55c08f9c7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.335+0000 7fbc98fcd740 1 --2- 172.21.15.100:0/503565085 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c08f9bd5c0 0x55c08f9c7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.335+0000 7fbc98fcd740 1 -- 172.21.15.100:0/503565085 >> 172.21.15.100:0/503565085 conn(0x55c08f9b8c30 msgr2=0x55c08f9af580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:43:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.335+0000 7fbc6effd640 0 client.29503 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:43:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.335+0000 7fbc98fcd740 1 -- 172.21.15.100:0/503565085 shutdown_connections 2024-05-10T11:43:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.335+0000 7fbc98fcd740 1 -- 172.21.15.100:0/503565085 wait complete. 2024-05-10T11:43:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.335+0000 7fbc98fcd740 20 client.29503 trim_cache size 0 max 16384 2024-05-10T11:43:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.365+0000 7fbc98fcd740 1 Processor -- start 2024-05-10T11:43:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.365+0000 7fbc98fcd740 1 -- start start 2024-05-10T11:43:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.365+0000 7fbc98fcd740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c08f9bd5c0 0x55c08f9bbb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.365+0000 7fbc98fcd740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c08f9bdfb0 0x55c08f9ced40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.365+0000 7fbc98fcd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c08f9c3120 0x55c08f9bc080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.366+0000 7fbc98fcd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbc901ab950 con 0x55c08f9c3120 2024-05-10T11:43:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.366+0000 7fbc98fcd740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbc901a9220 con 0x55c08f9bdfb0 2024-05-10T11:43:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.366+0000 7fbc98fcd740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbc901a7090 con 0x55c08f9bd5c0 2024-05-10T11:43:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.366+0000 7fbc8f7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c08f9bd5c0 0x55c08f9bbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.366+0000 7fbc8ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c08f9c3120 0x55c08f9bc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.366+0000 7fbc8effd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c08f9bdfb0 0x55c08f9ced40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.366+0000 7fbc8f7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c08f9bd5c0 0x55c08f9bbb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:47704/0 (socket says 172.21.15.100:47704) 2024-05-10T11:43:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.366+0000 7fbc8f7fe640 1 -- 172.21.15.100:0/1163912460 learned_addr learned my addr 172.21.15.100:0/1163912460 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:43:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.366+0000 7fbc8f7fe640 1 -- 172.21.15.100:0/1163912460 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c08f9bdfb0 msgr2=0x55c08f9ced40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:43:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.366+0000 7fbc8f7fe640 1 --2- 172.21.15.100:0/1163912460 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c08f9bdfb0 0x55c08f9ced40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.366+0000 7fbc8f7fe640 1 -- 172.21.15.100:0/1163912460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c08f9c3120 msgr2=0x55c08f9bc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:43:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.366+0000 7fbc8f7fe640 1 --2- 172.21.15.100:0/1163912460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c08f9c3120 0x55c08f9bc080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.366+0000 7fbc8f7fe640 1 -- 172.21.15.100:0/1163912460 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c08f9d1ca0 con 0x55c08f9bd5c0 2024-05-10T11:43:08.368 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.366+0000 7fbc8ffff640 1 --2- 172.21.15.100:0/1163912460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c08f9c3120 0x55c08f9bc080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:43:08.368 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.366+0000 7fbc8f7fe640 1 --2- 172.21.15.100:0/1163912460 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c08f9bd5c0 0x55c08f9bbb40 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fbc80011d10 tx=0x7fbc80011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85cad1d07600ac8d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:43:08.369 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.367+0000 7fbc757fa640 1 -- 172.21.15.100:0/1163912460 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc80030070 con 0x55c08f9bd5c0 2024-05-10T11:43:08.369 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.367+0000 7fbc757fa640 1 -- 172.21.15.100:0/1163912460 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbc8002b330 con 0x55c08f9bd5c0 2024-05-10T11:43:08.369 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.367+0000 7fbc757fa640 1 -- 172.21.15.100:0/1163912460 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc8001d040 con 0x55c08f9bd5c0 2024-05-10T11:43:08.369 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.368+0000 7fbc98fcd740 1 -- 172.21.15.100:0/1163912460 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c08f9bd5c0 msgr2=0x55c08f9bbb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:08.369 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.368+0000 7fbc98fcd740 1 --2- 172.21.15.100:0/1163912460 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c08f9bd5c0 0x55c08f9bbb40 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fbc80011d10 tx=0x7fbc80011810 comp rx=0 tx=0).stop 2024-05-10T11:43:08.370 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.368+0000 7fbc98fcd740 1 -- 172.21.15.100:0/1163912460 shutdown_connections 2024-05-10T11:43:08.370 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.368+0000 7fbc98fcd740 1 --2- 172.21.15.100:0/1163912460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c08f9c3120 0x55c08f9bc080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:08.370 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.368+0000 7fbc98fcd740 1 --2- 172.21.15.100:0/1163912460 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c08f9bdfb0 0x55c08f9ced40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:08.370 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.368+0000 7fbc98fcd740 1 --2- 172.21.15.100:0/1163912460 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c08f9bd5c0 0x55c08f9bbb40 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:08.370 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.368+0000 7fbc98fcd740 1 -- 172.21.15.100:0/1163912460 >> 172.21.15.100:0/1163912460 conn(0x55c08f9b8c30 msgr2=0x55c08f9bf130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:43:08.370 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.368+0000 7fbc98fcd740 1 -- 172.21.15.100:0/1163912460 shutdown_connections 2024-05-10T11:43:08.370 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.368+0000 7fbc98fcd740 1 -- 172.21.15.100:0/1163912460 wait complete. 2024-05-10T11:43:08.370 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.369+0000 7fbc98fcd740 1 -- start start 2024-05-10T11:43:08.372 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.370+0000 7fbc98fcd740 1 Processor -- start 2024-05-10T11:43:08.373 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.370+0000 7fbc98fcd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c08f9bd5c0 0x55c08f9cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:08.373 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.370+0000 7fbc98fcd740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c08f9bdfb0 0x55c08f9cc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:08.373 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.370+0000 7fbc98fcd740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c08f9c3120 0x55c08f9cd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:08.373 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.370+0000 7fbc8f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c08f9bd5c0 0x55c08f9cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:08.373 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.370+0000 7fbc8f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c08f9bd5c0 0x55c08f9cbf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42016/0 (socket says 172.21.15.100:42016) 2024-05-10T11:43:08.373 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.370+0000 7fbc8f7fe640 1 -- 172.21.15.100:0/1414431777 learned_addr learned my addr 172.21.15.100:0/1414431777 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:43:08.373 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.370+0000 7fbc98fcd740 1 -- 172.21.15.100:0/1414431777 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbc901a76e0 con 0x55c08f9bd5c0 2024-05-10T11:43:08.373 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.370+0000 7fbc98fcd740 1 -- 172.21.15.100:0/1414431777 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbc901a8b50 con 0x55c08f9bdfb0 2024-05-10T11:43:08.373 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.370+0000 7fbc98fcd740 1 -- 172.21.15.100:0/1414431777 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbc901a7db0 con 0x55c08f9c3120 2024-05-10T11:43:08.373 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.370+0000 7fbc98fcd740 10 client.0 _ll_register_callbacks cb 0x7fbc88003970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:43:08.373 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.371+0000 7fbc8ffff640 1 --2- 172.21.15.100:0/1414431777 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c08f9c3120 0x55c08f9cd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:08.374 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.371+0000 7fbc8effd640 1 --2- 172.21.15.100:0/1414431777 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c08f9bdfb0 0x55c08f9cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:08.374 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.371+0000 7fbc8f7fe640 1 -- 172.21.15.100:0/1414431777 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c08f9c3120 msgr2=0x55c08f9cd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:43:08.374 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.371+0000 7fbc8f7fe640 1 --2- 172.21.15.100:0/1414431777 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c08f9c3120 0x55c08f9cd490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:08.374 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.371+0000 7fbc8f7fe640 1 -- 172.21.15.100:0/1414431777 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c08f9bdfb0 msgr2=0x55c08f9cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:43:08.374 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.371+0000 7fbc8f7fe640 1 --2- 172.21.15.100:0/1414431777 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c08f9bdfb0 0x55c08f9cc480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:08.374 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.371+0000 7fbc8f7fe640 1 -- 172.21.15.100:0/1414431777 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c08f9cda80 con 0x55c08f9bd5c0 2024-05-10T11:43:08.374 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.371+0000 7fbc8effd640 1 --2- 172.21.15.100:0/1414431777 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c08f9bdfb0 0x55c08f9cc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:43:08.374 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.371+0000 7fbc8f7fe640 1 --2- 172.21.15.100:0/1414431777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c08f9bd5c0 0x55c08f9cbf40 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fbc8002bcb0 tx=0x7fbc80005cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f565fdafa04ff115 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:43:08.374 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.371+0000 7fbc6effd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:43:08.374 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.371+0000 7fbc6effd640 1 -- 172.21.15.100:0/1414431777 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc8001d030 con 0x55c08f9bd5c0 2024-05-10T11:43:08.374 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.371+0000 7fbc6effd640 1 -- 172.21.15.100:0/1414431777 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fbc8002b8a0 con 0x55c08f9bd5c0 2024-05-10T11:43:08.374 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.372+0000 7fbc98fcd740 1 -- 172.21.15.100:0/1414431777 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc8002b330 con 0x55c08f9bd5c0 2024-05-10T11:43:08.374 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.372+0000 7fbc6effd640 1 -- 172.21.15.100:0/1414431777 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fbc80030040 con 0x55c08f9bd5c0 2024-05-10T11:43:08.375 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.372+0000 7fbc98fcd740 10 client.20145 fetch_fsmap learned FSMap version 41 2024-05-10T11:43:08.375 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.372+0000 7fbc98fcd740 1 -- 172.21.15.100:0/1414431777 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55c08f9ced90 con 0x55c08f9bd5c0 2024-05-10T11:43:08.375 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.372+0000 7fbc6effd640 1 -- 172.21.15.100:0/1414431777 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbc8002b330 con 0x55c08f9bd5c0 2024-05-10T11:43:08.375 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.372+0000 7fbc6effd640 1 -- 172.21.15.100:0/1414431777 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fbc8002bce0 con 0x55c08f9bd5c0 2024-05-10T11:43:08.375 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.372+0000 7fbc98fcd740 10 client.20145 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:43:08.375 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.372+0000 7fbc98fcd740 10 client.20145 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:43:08.375 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.373+0000 7fbc98fcd740 20 client.20145 populate_metadata read hostname 'smithi100' 2024-05-10T11:43:08.375 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.373+0000 7fbc98fcd740 1 --2- 172.21.15.100:0/1414431777 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c08fa3e100 0x55c08fa40520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:08.375 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.373+0000 7fbc98fcd740 4 client.20145 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:43:08.375 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.373+0000 7fbc98fcd740 1 -- 172.21.15.100:0/1414431777 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c08f921050 con 0x55c08fa3e100 2024-05-10T11:43:08.375 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.373+0000 7fbc8effd640 1 --2- 172.21.15.100:0/1414431777 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c08fa3e100 0x55c08fa40520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:08.376 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.373+0000 7fbc8effd640 1 --2- 172.21.15.100:0/1414431777 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c08fa3e100 0x55c08fa40520 crc :-1 s=READY pgs=458 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a777a0bc9fecedd server_cookie=7df4818b86f99e44 in_seq=0 out_seq=0 2024-05-10T11:43:08.376 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.373+0000 7fbc6effd640 10 client.20145 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:43:08.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:43:08 smithi178 ceph-mon[33359]: pgmap v1163: 129 pgs: 129 active+clean; 575 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 4.1 MiB/s wr, 250 op/s 2024-05-10T11:43:08.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:43:08 smithi188 ceph-mon[34075]: pgmap v1163: 129 pgs: 129 active+clean; 575 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 4.1 MiB/s wr, 250 op/s 2024-05-10T11:43:08.532 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.532+0000 7fbc6effd640 1 -- 172.21.15.100:0/1414431777 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c08f921050 con 0x55c08fa3e100 2024-05-10T11:43:08.532 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.532+0000 7fbc6effd640 10 client.20145 handle_command_reply: tid=0 2024-05-10T11:43:08.532 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.532+0000 7fbc98fcd740 1 client.20145 shutdown 2024-05-10T11:43:08.536 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.536+0000 7fbc98fcd740 1 -- 172.21.15.100:0/1414431777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c08f9bd5c0 msgr2=0x55c08f9cbf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:08.536 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.536+0000 7fbc98fcd740 1 --2- 172.21.15.100:0/1414431777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c08f9bd5c0 0x55c08f9cbf40 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fbc8002bcb0 tx=0x7fbc80005cd0 comp rx=0 tx=0).stop 2024-05-10T11:43:08.536 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.536+0000 7fbc98fcd740 1 -- 172.21.15.100:0/1414431777 shutdown_connections 2024-05-10T11:43:08.536 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.536+0000 7fbc98fcd740 1 --2- 172.21.15.100:0/1414431777 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c08fa3e100 0x55c08fa40520 crc :-1 s=READY pgs=458 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:08.536 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.536+0000 7fbc98fcd740 1 --2- 172.21.15.100:0/1414431777 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c08f9c3120 0x55c08f9cd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:08.536 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.536+0000 7fbc98fcd740 1 --2- 172.21.15.100:0/1414431777 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c08f9bdfb0 0x55c08f9cc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:08.536 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.536+0000 7fbc98fcd740 1 --2- 172.21.15.100:0/1414431777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c08f9bd5c0 0x55c08f9cbf40 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:08.536 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.536+0000 7fbc98fcd740 1 -- 172.21.15.100:0/1414431777 >> 172.21.15.100:0/1414431777 conn(0x55c08f9b8c30 msgr2=0x55c08f9aecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:43:08.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.536+0000 7fbc6effd640 0 client.20145 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:43:08.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.536+0000 7fbc98fcd740 1 -- 172.21.15.100:0/1414431777 shutdown_connections 2024-05-10T11:43:08.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.536+0000 7fbc98fcd740 1 -- 172.21.15.100:0/1414431777 wait complete. 2024-05-10T11:43:08.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.537+0000 7fbc98fcd740 20 client.20145 trim_cache size 0 max 16384 2024-05-10T11:43:08.537 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:43:08.538 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:43:08.538 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:43:08.538 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:43:08.539 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.539+0000 7fbc961d4640 1 -- 172.21.15.100:0/3816415653 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fbc780b2920 msgr2=0x7fbc780b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:08.540 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.539+0000 7fbc961d4640 1 --2- 172.21.15.100:0/3816415653 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fbc780b2920 0x7fbc780b4de0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x55c08f9be810 tx=0x7fbc88009500 comp rx=0 tx=0).stop 2024-05-10T11:43:08.540 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.539+0000 7fbc961d4640 1 -- 172.21.15.100:0/3816415653 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc90103c10 msgr2=0x7fbc9019b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:08.540 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.539+0000 7fbc961d4640 1 --2- 172.21.15.100:0/3816415653 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc90103c10 0x7fbc9019b230 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fbc84009230 tx=0x7fbc840042f0 comp rx=0 tx=0).stop 2024-05-10T11:43:08.540 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.539+0000 7fbc961d4640 1 -- 172.21.15.100:0/3816415653 shutdown_connections 2024-05-10T11:43:08.540 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.539+0000 7fbc961d4640 1 --2- 172.21.15.100:0/3816415653 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fbc780b2920 0x7fbc780b4de0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:08.540 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.539+0000 7fbc961d4640 1 --2- 172.21.15.100:0/3816415653 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc901045d0 0x7fbc901a2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:08.540 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.539+0000 7fbc961d4640 1 --2- 172.21.15.100:0/3816415653 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc90103c10 0x7fbc9019b230 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:08.540 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.540+0000 7fbc961d4640 1 --2- 172.21.15.100:0/3816415653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc90102a20 0x7fbc9019acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:08.540 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.540+0000 7fbc961d4640 1 -- 172.21.15.100:0/3816415653 >> 172.21.15.100:0/3816415653 conn(0x7fbc900fdf90 msgr2=0x7fbc901093e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:43:08.541 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.541+0000 7fbc961d4640 1 -- 172.21.15.100:0/3816415653 shutdown_connections 2024-05-10T11:43:08.541 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:08.541+0000 7fbc961d4640 1 -- 172.21.15.100:0/3816415653 wait complete. 2024-05-10T11:43:08.960 INFO:tasks.workunit.client.0.smithi100.stdout: 10 109512 8.87 MB/sec execute 542 sec latency 153.783 ms 2024-05-10T11:43:09.960 INFO:tasks.workunit.client.0.smithi100.stdout: 10 109652 8.87 MB/sec execute 543 sec latency 130.702 ms 2024-05-10T11:43:10.196 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:43:10 smithi188 ceph-mon[34075]: pgmap v1164: 129 pgs: 129 active+clean; 557 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 4.1 MiB/s wr, 237 op/s 2024-05-10T11:43:10.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:43:10 smithi178 ceph-mon[33359]: pgmap v1164: 129 pgs: 129 active+clean; 557 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 4.1 MiB/s wr, 237 op/s 2024-05-10T11:43:10.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:43:10 smithi100 ceph-mon[30296]: pgmap v1164: 129 pgs: 129 active+clean; 557 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 4.1 MiB/s wr, 237 op/s 2024-05-10T11:43:10.960 INFO:tasks.workunit.client.0.smithi100.stdout: 10 109815 8.86 MB/sec execute 544 sec latency 103.408 ms 2024-05-10T11:43:11.834 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:43:11.961 INFO:tasks.workunit.client.0.smithi100.stdout: 10 109963 8.85 MB/sec execute 545 sec latency 142.369 ms 2024-05-10T11:43:12.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:43:12 smithi100 ceph-mon[30296]: pgmap v1165: 129 pgs: 129 active+clean; 558 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 756 KiB/s rd, 3.7 MiB/s wr, 239 op/s 2024-05-10T11:43:12.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:43:12 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:43:12.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:43:12 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:43:12.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:43:12 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:43:12.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:43:12 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:43:12.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:43:12 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:43:12.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:43:12 smithi178 ceph-mon[33359]: pgmap v1165: 129 pgs: 129 active+clean; 558 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 756 KiB/s rd, 3.7 MiB/s wr, 239 op/s 2024-05-10T11:43:12.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:43:12 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:43:12.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:43:12 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:43:12.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:43:12 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:43:12.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:43:12 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:43:12.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:43:12 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:43:12.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:43:12 smithi188 ceph-mon[34075]: pgmap v1165: 129 pgs: 129 active+clean; 558 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 756 KiB/s rd, 3.7 MiB/s wr, 239 op/s 2024-05-10T11:43:12.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:43:12 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:43:12.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:43:12 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:43:12.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:43:12 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:43:12.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:43:12 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:43:12.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:43:12 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:43:12.960 INFO:tasks.workunit.client.0.smithi100.stdout: 10 110093 8.85 MB/sec execute 546 sec latency 150.641 ms 2024-05-10T11:43:13.962 INFO:tasks.workunit.client.0.smithi100.stdout: 10 110246 8.86 MB/sec execute 547 sec latency 159.051 ms 2024-05-10T11:43:14.483 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:43:14 smithi100 ceph-mon[30296]: pgmap v1166: 129 pgs: 129 active+clean; 560 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 3.4 MiB/s wr, 257 op/s 2024-05-10T11:43:14.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:43:14 smithi178 ceph-mon[33359]: pgmap v1166: 129 pgs: 129 active+clean; 560 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 3.4 MiB/s wr, 257 op/s 2024-05-10T11:43:14.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:43:14 smithi188 ceph-mon[34075]: pgmap v1166: 129 pgs: 129 active+clean; 560 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 3.4 MiB/s wr, 257 op/s 2024-05-10T11:43:14.960 INFO:tasks.workunit.client.0.smithi100.stdout: 10 110437 8.87 MB/sec execute 548 sec latency 74.725 ms 2024-05-10T11:43:15.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.684+0000 7fbb3d560700 1 -- 172.21.15.100:0/2703287054 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb3806b180 msgr2=0x7fbb3806b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:15.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.684+0000 7fbb3d560700 1 --2- 172.21.15.100:0/2703287054 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb3806b180 0x7fbb3806b600 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fbb200098f0 tx=0x7fbb20009c00 comp rx=0 tx=0).stop 2024-05-10T11:43:15.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.684+0000 7fbb3d560700 1 -- 172.21.15.100:0/2703287054 shutdown_connections 2024-05-10T11:43:15.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.684+0000 7fbb3d560700 1 --2- 172.21.15.100:0/2703287054 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb3806bb40 0x7fbb38110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:15.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.684+0000 7fbb3d560700 1 --2- 172.21.15.100:0/2703287054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb38109510 0x7fbb381098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:15.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.684+0000 7fbb3d560700 1 --2- 172.21.15.100:0/2703287054 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb3806b180 0x7fbb3806b600 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:15.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.684+0000 7fbb3d560700 1 -- 172.21.15.100:0/2703287054 >> 172.21.15.100:0/2703287054 conn(0x7fbb38079350 msgr2=0x7fbb38079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:43:15.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.685+0000 7fbb3d560700 1 -- 172.21.15.100:0/2703287054 shutdown_connections 2024-05-10T11:43:15.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.685+0000 7fbb3d560700 1 -- 172.21.15.100:0/2703287054 wait complete. 2024-05-10T11:43:15.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.685+0000 7fbb3d560700 1 Processor -- start 2024-05-10T11:43:15.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.686+0000 7fbb3d560700 1 -- start start 2024-05-10T11:43:15.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.686+0000 7fbb3d560700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb3806b180 0x7fbb3819f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:15.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.686+0000 7fbb3d560700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb3806bb40 0x7fbb3819fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:15.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.686+0000 7fbb3d560700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb38109510 0x7fbb38199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:15.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.686+0000 7fbb3d560700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbb38113570 con 0x7fbb3806b180 2024-05-10T11:43:15.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.686+0000 7fbb3d560700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbb381133f0 con 0x7fbb38109510 2024-05-10T11:43:15.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.686+0000 7fbb3d560700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbb381136f0 con 0x7fbb3806bb40 2024-05-10T11:43:15.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.686+0000 7fbb2e7fc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb3806bb40 0x7fbb3819fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:15.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.686+0000 7fbb2e7fc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb3806bb40 0x7fbb3819fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:47740/0 (socket says 172.21.15.100:47740) 2024-05-10T11:43:15.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.686+0000 7fbb2e7fc700 1 -- 172.21.15.100:0/781660744 learned_addr learned my addr 172.21.15.100:0/781660744 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:43:15.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.686+0000 7fbb377fe700 1 --2- 172.21.15.100:0/781660744 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb38109510 0x7fbb38199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:15.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.686+0000 7fbb2e7fc700 1 -- 172.21.15.100:0/781660744 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb38109510 msgr2=0x7fbb38199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:15.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.687+0000 7fbb2e7fc700 1 --2- 172.21.15.100:0/781660744 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb38109510 0x7fbb38199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:15.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.687+0000 7fbb2e7fc700 1 -- 172.21.15.100:0/781660744 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb3806b180 msgr2=0x7fbb3819f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:15.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.687+0000 7fbb2e7fc700 1 --2- 172.21.15.100:0/781660744 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb3806b180 0x7fbb3819f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:15.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.687+0000 7fbb2e7fc700 1 -- 172.21.15.100:0/781660744 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb3819a0b0 con 0x7fbb3806bb40 2024-05-10T11:43:15.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.687+0000 7fbb2e7fc700 1 --2- 172.21.15.100:0/781660744 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb3806bb40 0x7fbb3819fcd0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fbb2000c010 tx=0x7fbb2000b560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:43:15.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.687+0000 7fbb34ff9700 1 -- 172.21.15.100:0/781660744 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb2000b780 con 0x7fbb3806bb40 2024-05-10T11:43:15.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.687+0000 7fbb3d560700 1 -- 172.21.15.100:0/781660744 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb3819a340 con 0x7fbb3806bb40 2024-05-10T11:43:15.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.687+0000 7fbb3d560700 1 -- 172.21.15.100:0/781660744 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb381023a0 con 0x7fbb3806bb40 2024-05-10T11:43:15.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.687+0000 7fbb34ff9700 1 -- 172.21.15.100:0/781660744 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbb200054d0 con 0x7fbb3806bb40 2024-05-10T11:43:15.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.687+0000 7fbb34ff9700 1 -- 172.21.15.100:0/781660744 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb20003b20 con 0x7fbb3806bb40 2024-05-10T11:43:15.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.692+0000 7fbb34ff9700 1 -- 172.21.15.100:0/781660744 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fbb20003cc0 con 0x7fbb3806bb40 2024-05-10T11:43:15.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.693+0000 7fbb47df3b80 1 Processor -- start 2024-05-10T11:43:15.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.693+0000 7fbb47df3b80 1 -- start start 2024-05-10T11:43:15.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.693+0000 7fbb47df3b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604c48d4200 0x5604c48d45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:15.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.694+0000 7fbb47df3b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5604c48d4b20 0x5604c48d8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:15.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.694+0000 7fbb47df3b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5604c48d8aa0 0x5604c48d8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:15.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.694+0000 7fbb47df3b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5604c481d940 con 0x5604c48d4b20 2024-05-10T11:43:15.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.694+0000 7fbb47df3b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5604c4782da0 con 0x5604c48d4200 2024-05-10T11:43:15.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.694+0000 7fbb47df3b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5604c4819900 con 0x5604c48d8aa0 2024-05-10T11:43:15.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.695+0000 7fbb2e7fc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5604c48d8aa0 0x5604c48d8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:15.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.695+0000 7fbb2e7fc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5604c48d8aa0 0x5604c48d8f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:47752/0 (socket says 172.21.15.100:47752) 2024-05-10T11:43:15.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.695+0000 7fbb2e7fc700 1 -- 172.21.15.100:0/633068523 learned_addr learned my addr 172.21.15.100:0/633068523 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:43:15.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.695+0000 7fbb34ff9700 1 --2- 172.21.15.100:0/781660744 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fbb240b29c0 0x7fbb240b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:15.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.695+0000 7fbb2e7fc700 1 -- 172.21.15.100:0/633068523 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604c48d4200 msgr2=0x5604c48d45e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T11:43:15.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.695+0000 7fbb2e7fc700 1 --2- 172.21.15.100:0/633068523 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604c48d4200 0x5604c48d45e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:15.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.695+0000 7fbb2e7fc700 1 -- 172.21.15.100:0/633068523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5604c48d4b20 msgr2=0x5604c48d8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:43:15.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.695+0000 7fbb2e7fc700 1 --2- 172.21.15.100:0/633068523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5604c48d4b20 0x5604c48d8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:15.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.695+0000 7fbb2e7fc700 1 -- 172.21.15.100:0/633068523 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604c476a8d0 con 0x5604c48d8aa0 2024-05-10T11:43:15.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.696+0000 7fbb2e7fc700 1 --2- 172.21.15.100:0/633068523 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5604c48d8aa0 0x5604c48d8f00 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fbb2008e1c0 tx=0x7fbb20005810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24bb2346e125eeda server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:43:15.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.696+0000 7fbb2d7fa700 1 -- 172.21.15.100:0/633068523 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb200aa520 con 0x5604c48d8aa0 2024-05-10T11:43:15.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.696+0000 7fbb2d7fa700 1 -- 172.21.15.100:0/633068523 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbb200aa260 con 0x5604c48d8aa0 2024-05-10T11:43:15.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.696+0000 7fbb2d7fa700 1 -- 172.21.15.100:0/633068523 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb2002c8f0 con 0x5604c48d8aa0 2024-05-10T11:43:15.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.696+0000 7fbb34ff9700 1 -- 172.21.15.100:0/781660744 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fbb200e3460 con 0x7fbb3806bb40 2024-05-10T11:43:15.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.698+0000 7fbb47df3b80 1 -- 172.21.15.100:0/633068523 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5604c48d8aa0 msgr2=0x5604c48d8f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:15.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.698+0000 7fbb47df3b80 1 --2- 172.21.15.100:0/633068523 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5604c48d8aa0 0x5604c48d8f00 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fbb2008e1c0 tx=0x7fbb20005810 comp rx=0 tx=0).stop 2024-05-10T11:43:15.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.698+0000 7fbb377fe700 1 --2- 172.21.15.100:0/781660744 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fbb240b29c0 0x7fbb240b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:15.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.699+0000 7fbb47df3b80 1 -- 172.21.15.100:0/633068523 shutdown_connections 2024-05-10T11:43:15.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.699+0000 7fbb47df3b80 1 --2- 172.21.15.100:0/633068523 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5604c48d8aa0 0x5604c48d8f00 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:15.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.699+0000 7fbb47df3b80 1 --2- 172.21.15.100:0/633068523 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604c48d4200 0x5604c48d45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:15.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.699+0000 7fbb47df3b80 1 --2- 172.21.15.100:0/633068523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5604c48d4b20 0x5604c48d8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:15.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.699+0000 7fbb47df3b80 1 -- 172.21.15.100:0/633068523 >> 172.21.15.100:0/633068523 conn(0x5604c47f7b70 msgr2=0x5604c48c62a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:43:15.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.700+0000 7fbb377fe700 1 --2- 172.21.15.100:0/781660744 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fbb240b29c0 0x7fbb240b4e80 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fbb30009550 tx=0x7fbb30009100 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:43:15.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.700+0000 7fbb47df3b80 1 -- 172.21.15.100:0/633068523 shutdown_connections 2024-05-10T11:43:15.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.701+0000 7fbb47df3b80 1 -- 172.21.15.100:0/633068523 wait complete. 2024-05-10T11:43:15.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.702+0000 7fbb47df3b80 1 -- start start 2024-05-10T11:43:15.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.702+0000 7fbb47df3b80 1 Processor -- start 2024-05-10T11:43:15.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.703+0000 7fbb47df3b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604c48d4200 0x5604c48dfdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:15.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.703+0000 7fbb47df3b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5604c48d4b20 0x5604c48dd170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:15.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.703+0000 7fbb47df3b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5604c48d8aa0 0x5604c48dd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:15.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.703+0000 7fbb47df3b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5604c4819900 con 0x5604c48d8aa0 2024-05-10T11:43:15.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.703+0000 7fbb47df3b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5604c47d6ed0 con 0x5604c48d4200 2024-05-10T11:43:15.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.703+0000 7fbb47df3b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5604c48e09c0 con 0x5604c48d4b20 2024-05-10T11:43:15.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.703+0000 7fbb47df3b80 10 client.0 _ll_register_callbacks cb 0x5604c47b15b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:43:15.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.704+0000 7fbb2e7fc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5604c48d4b20 0x5604c48dd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:15.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.704+0000 7fbb2e7fc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5604c48d4b20 0x5604c48dd170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:47754/0 (socket says 172.21.15.100:47754) 2024-05-10T11:43:15.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.704+0000 7fbb2e7fc700 1 -- 172.21.15.100:0/1582471221 learned_addr learned my addr 172.21.15.100:0/1582471221 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:43:15.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.704+0000 7fbb377fe700 1 --2- 172.21.15.100:0/1582471221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5604c48d8aa0 0x5604c48dd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:15.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.704+0000 7fbb36ffd700 1 --2- 172.21.15.100:0/1582471221 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604c48d4200 0x5604c48dfdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:15.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.704+0000 7fbb2e7fc700 1 -- 172.21.15.100:0/1582471221 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604c48d4200 msgr2=0x5604c48dfdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:43:15.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.704+0000 7fbb2e7fc700 1 --2- 172.21.15.100:0/1582471221 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604c48d4200 0x5604c48dfdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:15.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.704+0000 7fbb2e7fc700 1 -- 172.21.15.100:0/1582471221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5604c48d8aa0 msgr2=0x5604c48dd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:43:15.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.704+0000 7fbb2e7fc700 1 --2- 172.21.15.100:0/1582471221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5604c48d8aa0 0x5604c48dd6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:15.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.704+0000 7fbb2e7fc700 1 -- 172.21.15.100:0/1582471221 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604c47d4760 con 0x5604c48d4b20 2024-05-10T11:43:15.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.704+0000 7fbb36ffd700 1 --2- 172.21.15.100:0/1582471221 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604c48d4200 0x5604c48dfdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:43:15.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.704+0000 7fbb2e7fc700 1 --2- 172.21.15.100:0/1582471221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5604c48d4b20 0x5604c48dd170 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fbb20003fd0 tx=0x7fbb200e6da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2cf79c8db4eda4d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:43:15.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.704+0000 7fbb1e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:43:15.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.706+0000 7fbb1e7fc700 1 -- 172.21.15.100:0/1582471221 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb20024cf0 con 0x5604c48d4b20 2024-05-10T11:43:15.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.706+0000 7fbb1e7fc700 1 -- 172.21.15.100:0/1582471221 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fbb200a60c0 con 0x5604c48d4b20 2024-05-10T11:43:15.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.706+0000 7fbb47df3b80 1 -- 172.21.15.100:0/1582471221 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb200aa260 con 0x5604c48d4b20 2024-05-10T11:43:15.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.706+0000 7fbb1e7fc700 1 -- 172.21.15.100:0/1582471221 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fbb20017400 con 0x5604c48d4b20 2024-05-10T11:43:15.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.706+0000 7fbb47df3b80 10 client.29521 fetch_fsmap learned FSMap version 41 2024-05-10T11:43:15.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.706+0000 7fbb47df3b80 1 -- 172.21.15.100:0/1582471221 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5604c48eafa0 con 0x5604c48d4b20 2024-05-10T11:43:15.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.707+0000 7fbb1e7fc700 1 -- 172.21.15.100:0/1582471221 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbb200aae50 con 0x5604c48d4b20 2024-05-10T11:43:15.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.707+0000 7fbb1e7fc700 1 -- 172.21.15.100:0/1582471221 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fbb200e30d0 con 0x5604c48d4b20 2024-05-10T11:43:15.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.707+0000 7fbb47df3b80 10 client.29521 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:43:15.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.707+0000 7fbb47df3b80 10 client.29521 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:43:15.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.707+0000 7fbb47df3b80 20 client.29521 populate_metadata read hostname 'smithi100' 2024-05-10T11:43:15.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.707+0000 7fbb47df3b80 1 --2- 172.21.15.100:0/1582471221 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5604c48ed470 0x5604c48ef890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:15.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.707+0000 7fbb47df3b80 4 client.29521 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:43:15.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.707+0000 7fbb47df3b80 1 -- 172.21.15.100:0/1582471221 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fbb200aa260 con 0x5604c48ed470 2024-05-10T11:43:15.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.708+0000 7fbb36ffd700 1 --2- 172.21.15.100:0/1582471221 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5604c48ed470 0x5604c48ef890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:15.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.708+0000 7fbb36ffd700 1 --2- 172.21.15.100:0/1582471221 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5604c48ed470 0x5604c48ef890 crc :-1 s=READY pgs=459 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa5b695ea55be567 server_cookie=860f1d187c4447a3 in_seq=0 out_seq=0 2024-05-10T11:43:15.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.708+0000 7fbb1e7fc700 10 client.29521 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:43:15.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.709+0000 7fbb1e7fc700 1 -- 172.21.15.100:0/1582471221 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fbb200aa260 con 0x5604c48ed470 2024-05-10T11:43:15.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.710+0000 7fbb1e7fc700 10 client.29521 handle_command_reply: tid=0 2024-05-10T11:43:15.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.710+0000 7fbb47df3b80 1 client.29521 shutdown 2024-05-10T11:43:15.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.710+0000 7fbb47df3b80 1 -- 172.21.15.100:0/1582471221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5604c48d4b20 msgr2=0x5604c48dd170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:15.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.710+0000 7fbb47df3b80 1 --2- 172.21.15.100:0/1582471221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5604c48d4b20 0x5604c48dd170 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fbb20003fd0 tx=0x7fbb200e6da0 comp rx=0 tx=0).stop 2024-05-10T11:43:15.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.711+0000 7fbb47df3b80 1 -- 172.21.15.100:0/1582471221 shutdown_connections 2024-05-10T11:43:15.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.711+0000 7fbb47df3b80 1 --2- 172.21.15.100:0/1582471221 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5604c48ed470 0x5604c48ef890 crc :-1 s=READY pgs=459 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:15.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.711+0000 7fbb47df3b80 1 --2- 172.21.15.100:0/1582471221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5604c48d8aa0 0x5604c48dd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:15.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.711+0000 7fbb47df3b80 1 --2- 172.21.15.100:0/1582471221 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604c48d4200 0x5604c48dfdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:15.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.711+0000 7fbb47df3b80 1 --2- 172.21.15.100:0/1582471221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5604c48d4b20 0x5604c48dd170 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:15.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.711+0000 7fbb47df3b80 1 -- 172.21.15.100:0/1582471221 >> 172.21.15.100:0/1582471221 conn(0x5604c47f7b70 msgr2=0x5604c48c0b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:43:15.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.711+0000 7fbb1e7fc700 0 client.29521 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:43:15.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.711+0000 7fbb47df3b80 1 -- 172.21.15.100:0/1582471221 shutdown_connections 2024-05-10T11:43:15.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.711+0000 7fbb47df3b80 1 -- 172.21.15.100:0/1582471221 wait complete. 2024-05-10T11:43:15.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.711+0000 7fbb47df3b80 20 client.29521 trim_cache size 0 max 16384 2024-05-10T11:43:15.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.741+0000 7fbb47df3b80 1 Processor -- start 2024-05-10T11:43:15.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.741+0000 7fbb47df3b80 1 -- start start 2024-05-10T11:43:15.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.742+0000 7fbb47df3b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604c48d4200 0x5604c48d8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:15.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.742+0000 7fbb47df3b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5604c48d4b20 0x5604c48c0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:15.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.742+0000 7fbb47df3b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5604c48d8aa0 0x5604c48dc170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:15.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.742+0000 7fbb47df3b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbb18011360 con 0x5604c48d8aa0 2024-05-10T11:43:15.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.742+0000 7fbb47df3b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1800ebf0 con 0x5604c48d4200 2024-05-10T11:43:15.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.742+0000 7fbb47df3b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1800ca40 con 0x5604c48d4b20 2024-05-10T11:43:15.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.743+0000 7fbb36ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604c48d4200 0x5604c48d8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:15.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.743+0000 7fbb36ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604c48d4200 0x5604c48d8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:56666/0 (socket says 172.21.15.100:56666) 2024-05-10T11:43:15.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.744+0000 7fbb36ffd700 1 -- 172.21.15.100:0/3213524788 learned_addr learned my addr 172.21.15.100:0/3213524788 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:43:15.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.745+0000 7fbb36ffd700 1 -- 172.21.15.100:0/3213524788 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5604c48d4b20 msgr2=0x5604c48c0c40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T11:43:15.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.745+0000 7fbb36ffd700 1 --2- 172.21.15.100:0/3213524788 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5604c48d4b20 0x5604c48c0c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:15.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.745+0000 7fbb36ffd700 1 -- 172.21.15.100:0/3213524788 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5604c48d8aa0 msgr2=0x5604c48dc170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T11:43:15.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.745+0000 7fbb36ffd700 1 --2- 172.21.15.100:0/3213524788 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5604c48d8aa0 0x5604c48dc170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:15.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.745+0000 7fbb36ffd700 1 -- 172.21.15.100:0/3213524788 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604c48dd9c0 con 0x5604c48d4200 2024-05-10T11:43:15.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.746+0000 7fbb36ffd700 1 --2- 172.21.15.100:0/3213524788 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604c48d4200 0x5604c48d8ea0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fbb28005b20 tx=0x7fbb280097b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c5652c15d4f5c57 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:43:15.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.746+0000 7fbb2cff9700 1 -- 172.21.15.100:0/3213524788 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb28009ea0 con 0x5604c48d4200 2024-05-10T11:43:15.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.746+0000 7fbb2cff9700 1 -- 172.21.15.100:0/3213524788 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbb28013de0 con 0x5604c48d4200 2024-05-10T11:43:15.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.747+0000 7fbb2cff9700 1 -- 172.21.15.100:0/3213524788 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb2801b650 con 0x5604c48d4200 2024-05-10T11:43:15.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.747+0000 7fbb47df3b80 1 -- 172.21.15.100:0/3213524788 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604c48d4200 msgr2=0x5604c48d8ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:15.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.747+0000 7fbb47df3b80 1 --2- 172.21.15.100:0/3213524788 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604c48d4200 0x5604c48d8ea0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fbb28005b20 tx=0x7fbb280097b0 comp rx=0 tx=0).stop 2024-05-10T11:43:15.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.748+0000 7fbb47df3b80 1 -- 172.21.15.100:0/3213524788 shutdown_connections 2024-05-10T11:43:15.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.748+0000 7fbb47df3b80 1 --2- 172.21.15.100:0/3213524788 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5604c48d8aa0 0x5604c48dc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:15.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.748+0000 7fbb47df3b80 1 --2- 172.21.15.100:0/3213524788 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604c48d4200 0x5604c48d8ea0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:15.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.748+0000 7fbb47df3b80 1 --2- 172.21.15.100:0/3213524788 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5604c48d4b20 0x5604c48c0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:15.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.748+0000 7fbb47df3b80 1 -- 172.21.15.100:0/3213524788 >> 172.21.15.100:0/3213524788 conn(0x5604c47f7b70 msgr2=0x5604c48dba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:43:15.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.749+0000 7fbb47df3b80 1 -- 172.21.15.100:0/3213524788 shutdown_connections 2024-05-10T11:43:15.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.749+0000 7fbb47df3b80 1 -- 172.21.15.100:0/3213524788 wait complete. 2024-05-10T11:43:15.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.749+0000 7fbb47df3b80 1 -- start start 2024-05-10T11:43:15.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.749+0000 7fbb47df3b80 1 Processor -- start 2024-05-10T11:43:15.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.750+0000 7fbb47df3b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5604c48d4200 0x5604c4954ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:15.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.750+0000 7fbb47df3b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604c48d4b20 0x5604c4955410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:15.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.750+0000 7fbb47df3b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5604c48d8aa0 0x5604c48e5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:15.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.750+0000 7fbb47df3b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1800d0b0 con 0x5604c48d4200 2024-05-10T11:43:15.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.750+0000 7fbb47df3b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1800e520 con 0x5604c48d4b20 2024-05-10T11:43:15.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.750+0000 7fbb47df3b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1800d780 con 0x5604c48d8aa0 2024-05-10T11:43:15.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.750+0000 7fbb47df3b80 10 client.0 _ll_register_callbacks cb 0x5604c48de1b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:43:15.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.750+0000 7fbb36ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5604c48d4200 0x5604c4954ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:15.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.750+0000 7fbb36ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5604c48d4200 0x5604c4954ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42088/0 (socket says 172.21.15.100:42088) 2024-05-10T11:43:15.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.750+0000 7fbb36ffd700 1 -- 172.21.15.100:0/1718386923 learned_addr learned my addr 172.21.15.100:0/1718386923 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:43:15.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.751+0000 7fbb377fe700 1 --2- 172.21.15.100:0/1718386923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5604c48d8aa0 0x5604c48e5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:15.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.751+0000 7fbb2e7fc700 1 --2- 172.21.15.100:0/1718386923 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604c48d4b20 0x5604c4955410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:15.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.751+0000 7fbb36ffd700 1 -- 172.21.15.100:0/1718386923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5604c48d8aa0 msgr2=0x5604c48e5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:43:15.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.751+0000 7fbb36ffd700 1 --2- 172.21.15.100:0/1718386923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5604c48d8aa0 0x5604c48e5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:15.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.751+0000 7fbb36ffd700 1 -- 172.21.15.100:0/1718386923 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604c48d4b20 msgr2=0x5604c4955410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:43:15.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.751+0000 7fbb36ffd700 1 --2- 172.21.15.100:0/1718386923 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604c48d4b20 0x5604c4955410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:15.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.751+0000 7fbb36ffd700 1 -- 172.21.15.100:0/1718386923 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604c48e61e0 con 0x5604c48d4200 2024-05-10T11:43:15.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.751+0000 7fbb2e7fc700 1 --2- 172.21.15.100:0/1718386923 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604c48d4b20 0x5604c4955410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:43:15.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.751+0000 7fbb36ffd700 1 --2- 172.21.15.100:0/1718386923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5604c48d4200 0x5604c4954ed0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fbb28006010 tx=0x7fbb28009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44c5ad4408cf2ecc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:43:15.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.751+0000 7fbb1e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:43:15.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.752+0000 7fbb1e7fc700 1 -- 172.21.15.100:0/1718386923 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb28004830 con 0x5604c48d4200 2024-05-10T11:43:15.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.752+0000 7fbb47df3b80 1 -- 172.21.15.100:0/1718386923 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb28013de0 con 0x5604c48d4200 2024-05-10T11:43:15.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.752+0000 7fbb1e7fc700 1 -- 172.21.15.100:0/1718386923 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fbb28004240 con 0x5604c48d4200 2024-05-10T11:43:15.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.753+0000 7fbb1e7fc700 1 -- 172.21.15.100:0/1718386923 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fbb28035050 con 0x5604c48d4200 2024-05-10T11:43:15.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.753+0000 7fbb47df3b80 10 client.20151 fetch_fsmap learned FSMap version 41 2024-05-10T11:43:15.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.753+0000 7fbb47df3b80 1 -- 172.21.15.100:0/1718386923 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5604c48da4d0 con 0x5604c48d4200 2024-05-10T11:43:15.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.754+0000 7fbb1e7fc700 1 -- 172.21.15.100:0/1718386923 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbb28013de0 con 0x5604c48d4200 2024-05-10T11:43:15.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.754+0000 7fbb1e7fc700 1 -- 172.21.15.100:0/1718386923 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fbb28003920 con 0x5604c48d4200 2024-05-10T11:43:15.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.754+0000 7fbb47df3b80 10 client.20151 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:43:15.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.754+0000 7fbb47df3b80 10 client.20151 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:43:15.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.754+0000 7fbb47df3b80 20 client.20151 populate_metadata read hostname 'smithi100' 2024-05-10T11:43:15.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.754+0000 7fbb47df3b80 1 --2- 172.21.15.100:0/1718386923 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5604c4968f40 0x5604c496b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:15.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.754+0000 7fbb47df3b80 4 client.20151 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:43:15.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.754+0000 7fbb47df3b80 1 -- 172.21.15.100:0/1718386923 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5604c482b320 con 0x5604c4968f40 2024-05-10T11:43:15.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.754+0000 7fbb2e7fc700 1 --2- 172.21.15.100:0/1718386923 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5604c4968f40 0x5604c496b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:15.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.756+0000 7fbb1e7fc700 10 client.20151 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:43:15.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.756+0000 7fbb2e7fc700 1 --2- 172.21.15.100:0/1718386923 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5604c4968f40 0x5604c496b360 crc :-1 s=READY pgs=460 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1ca85e20711237b server_cookie=3eae3133418856fb in_seq=0 out_seq=0 2024-05-10T11:43:15.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.757+0000 7fbb1e7fc700 1 -- 172.21.15.100:0/1718386923 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5604c482b320 con 0x5604c4968f40 2024-05-10T11:43:15.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.757+0000 7fbb1e7fc700 10 client.20151 handle_command_reply: tid=0 2024-05-10T11:43:15.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.757+0000 7fbb47df3b80 1 client.20151 shutdown 2024-05-10T11:43:15.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.757+0000 7fbb47df3b80 1 -- 172.21.15.100:0/1718386923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5604c48d4200 msgr2=0x5604c4954ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:15.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.757+0000 7fbb47df3b80 1 --2- 172.21.15.100:0/1718386923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5604c48d4200 0x5604c4954ed0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fbb28006010 tx=0x7fbb28009f30 comp rx=0 tx=0).stop 2024-05-10T11:43:15.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.757+0000 7fbb47df3b80 1 -- 172.21.15.100:0/1718386923 shutdown_connections 2024-05-10T11:43:15.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.757+0000 7fbb47df3b80 1 --2- 172.21.15.100:0/1718386923 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5604c4968f40 0x5604c496b360 crc :-1 s=READY pgs=460 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:15.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.757+0000 7fbb47df3b80 1 --2- 172.21.15.100:0/1718386923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5604c48d8aa0 0x5604c48e5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:15.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.757+0000 7fbb47df3b80 1 --2- 172.21.15.100:0/1718386923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5604c48d4200 0x5604c4954ed0 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:15.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.757+0000 7fbb47df3b80 1 --2- 172.21.15.100:0/1718386923 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604c48d4b20 0x5604c4955410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:15.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.757+0000 7fbb47df3b80 1 -- 172.21.15.100:0/1718386923 >> 172.21.15.100:0/1718386923 conn(0x5604c47f7b70 msgr2=0x5604c48d7250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:43:15.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.757+0000 7fbb1e7fc700 0 client.20151 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:43:15.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.757+0000 7fbb47df3b80 1 -- 172.21.15.100:0/1718386923 shutdown_connections 2024-05-10T11:43:15.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.757+0000 7fbb47df3b80 1 -- 172.21.15.100:0/1718386923 wait complete. 2024-05-10T11:43:15.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.757+0000 7fbb47df3b80 20 client.20151 trim_cache size 0 max 16384 2024-05-10T11:43:15.760 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:43:15.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.758+0000 7fbb3d560700 1 -- 172.21.15.100:0/781660744 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fbb240b29c0 msgr2=0x7fbb240b4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:15.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.758+0000 7fbb3d560700 1 --2- 172.21.15.100:0/781660744 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fbb240b29c0 0x7fbb240b4e80 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fbb30009550 tx=0x7fbb30009100 comp rx=0 tx=0).stop 2024-05-10T11:43:15.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.758+0000 7fbb3d560700 1 -- 172.21.15.100:0/781660744 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb3806bb40 msgr2=0x7fbb3819fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:15.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.758+0000 7fbb3d560700 1 --2- 172.21.15.100:0/781660744 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb3806bb40 0x7fbb3819fcd0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fbb2000c010 tx=0x7fbb2000b560 comp rx=0 tx=0).stop 2024-05-10T11:43:15.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.758+0000 7fbb3d560700 1 -- 172.21.15.100:0/781660744 shutdown_connections 2024-05-10T11:43:15.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.758+0000 7fbb3d560700 1 --2- 172.21.15.100:0/781660744 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fbb240b29c0 0x7fbb240b4e80 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:15.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.758+0000 7fbb3d560700 1 --2- 172.21.15.100:0/781660744 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb38109510 0x7fbb38199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:15.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.758+0000 7fbb3d560700 1 --2- 172.21.15.100:0/781660744 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb3806b180 0x7fbb3819f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:15.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.758+0000 7fbb3d560700 1 --2- 172.21.15.100:0/781660744 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb3806bb40 0x7fbb3819fcd0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:15.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.758+0000 7fbb3d560700 1 -- 172.21.15.100:0/781660744 >> 172.21.15.100:0/781660744 conn(0x7fbb38079350 msgr2=0x7fbb38101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:43:15.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.759+0000 7fbb3d560700 1 -- 172.21.15.100:0/781660744 shutdown_connections 2024-05-10T11:43:15.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:15.759+0000 7fbb3d560700 1 -- 172.21.15.100:0/781660744 wait complete. 2024-05-10T11:43:15.960 INFO:tasks.workunit.client.0.smithi100.stdout: 10 110618 8.87 MB/sec execute 549 sec latency 139.486 ms 2024-05-10T11:43:16.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:43:16 smithi178 ceph-mon[33359]: pgmap v1167: 129 pgs: 129 active+clean; 557 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 642 KiB/s rd, 3.4 MiB/s wr, 236 op/s 2024-05-10T11:43:16.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:43:16 smithi188 ceph-mon[34075]: pgmap v1167: 129 pgs: 129 active+clean; 557 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 642 KiB/s rd, 3.4 MiB/s wr, 236 op/s 2024-05-10T11:43:16.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:43:16 smithi100 ceph-mon[30296]: pgmap v1167: 129 pgs: 129 active+clean; 557 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 642 KiB/s rd, 3.4 MiB/s wr, 236 op/s 2024-05-10T11:43:16.960 INFO:tasks.workunit.client.0.smithi100.stdout: 10 110800 8.87 MB/sec execute 550 sec latency 64.370 ms 2024-05-10T11:43:17.614 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T11:43:17.614 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T11:43:17.960 INFO:tasks.workunit.client.0.smithi100.stdout: 10 110959 8.87 MB/sec execute 551 sec latency 60.441 ms 2024-05-10T11:43:18.005 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:43:18.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:43:18 smithi178 ceph-mon[33359]: pgmap v1168: 129 pgs: 129 active+clean; 561 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 933 KiB/s rd, 4.1 MiB/s wr, 298 op/s 2024-05-10T11:43:18.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:43:18 smithi188 ceph-mon[34075]: pgmap v1168: 129 pgs: 129 active+clean; 561 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 933 KiB/s rd, 4.1 MiB/s wr, 298 op/s 2024-05-10T11:43:18.530 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:43:18 smithi100 ceph-mon[30296]: pgmap v1168: 129 pgs: 129 active+clean; 561 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 933 KiB/s rd, 4.1 MiB/s wr, 298 op/s 2024-05-10T11:43:18.961 INFO:tasks.workunit.client.0.smithi100.stdout: 10 111125 8.87 MB/sec execute 552 sec latency 82.495 ms 2024-05-10T11:43:19.961 INFO:tasks.workunit.client.0.smithi100.stdout: 10 111288 8.88 MB/sec execute 553 sec latency 100.910 ms 2024-05-10T11:43:20.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.972+0000 7f77aa27c700 1 -- 172.21.15.100:0/4024415753 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77a4102230 msgr2=0x7f77a4109280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:20.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.972+0000 7f77aa27c700 1 --2- 172.21.15.100:0/4024415753 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77a4102230 0x7f77a4109280 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f77940099c0 tx=0x7f7794009cd0 comp rx=0 tx=0).stop 2024-05-10T11:43:20.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.972+0000 7f77aa27c700 1 -- 172.21.15.100:0/4024415753 shutdown_connections 2024-05-10T11:43:20.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.972+0000 7f77aa27c700 1 --2- 172.21.15.100:0/4024415753 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77a4109960 0x7f77a4110390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:20.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.972+0000 7f77aa27c700 1 --2- 172.21.15.100:0/4024415753 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77a4101880 0x7f77a4101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:20.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.972+0000 7f77aa27c700 1 --2- 172.21.15.100:0/4024415753 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77a4102230 0x7f77a4109280 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:20.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.972+0000 7f77aa27c700 1 -- 172.21.15.100:0/4024415753 >> 172.21.15.100:0/4024415753 conn(0x7f77a40785a0 msgr2=0x7f77a40ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:43:20.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.972+0000 7f77aa27c700 1 -- 172.21.15.100:0/4024415753 shutdown_connections 2024-05-10T11:43:20.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.972+0000 7f77aa27c700 1 -- 172.21.15.100:0/4024415753 wait complete. 2024-05-10T11:43:20.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.973+0000 7f77aa27c700 1 Processor -- start 2024-05-10T11:43:20.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.973+0000 7f77aa27c700 1 -- start start 2024-05-10T11:43:20.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.973+0000 7f77aa27c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77a4101880 0x7f77a41a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:20.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.973+0000 7f77aa27c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77a4102230 0x7f77a41a1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:20.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.973+0000 7f77aa27c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77a4109960 0x7f77a419bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:20.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.973+0000 7f77aa27c700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f77a4104190 con 0x7f77a4102230 2024-05-10T11:43:20.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.973+0000 7f77aa27c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f77a4104010 con 0x7f77a4109960 2024-05-10T11:43:20.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.973+0000 7f77aa27c700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f77a4104310 con 0x7f77a4101880 2024-05-10T11:43:20.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.973+0000 7f77a37fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77a4102230 0x7f77a41a1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:20.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.973+0000 7f77a3fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77a4101880 0x7f77a41a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:20.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.973+0000 7f77a8819700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77a4109960 0x7f77a419bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:20.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.973+0000 7f77a3fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77a4101880 0x7f77a41a1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:56802/0 (socket says 172.21.15.100:56802) 2024-05-10T11:43:20.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.973+0000 7f77a3fff700 1 -- 172.21.15.100:0/817490582 learned_addr learned my addr 172.21.15.100:0/817490582 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:43:20.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.974+0000 7f77a8819700 1 -- 172.21.15.100:0/817490582 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77a4101880 msgr2=0x7f77a41a1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:20.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.974+0000 7f77a8819700 1 --2- 172.21.15.100:0/817490582 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77a4101880 0x7f77a41a1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:20.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.974+0000 7f77a8819700 1 -- 172.21.15.100:0/817490582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77a4102230 msgr2=0x7f77a41a1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:20.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.974+0000 7f77a8819700 1 --2- 172.21.15.100:0/817490582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77a4102230 0x7f77a41a1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:20.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.974+0000 7f77a8819700 1 -- 172.21.15.100:0/817490582 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77a419c410 con 0x7f77a4109960 2024-05-10T11:43:20.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.974+0000 7f77a37fe700 1 --2- 172.21.15.100:0/817490582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77a4102230 0x7f77a41a1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:43:20.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.974+0000 7f77a8819700 1 --2- 172.21.15.100:0/817490582 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77a4109960 0x7f77a419bb30 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f779c00efc0 tx=0x7f779c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:43:20.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.974+0000 7f77a17fa700 1 -- 172.21.15.100:0/817490582 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779c0092e0 con 0x7f77a4109960 2024-05-10T11:43:20.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.974+0000 7f77aa27c700 1 -- 172.21.15.100:0/817490582 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77a419c6a0 con 0x7f77a4109960 2024-05-10T11:43:20.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.974+0000 7f77aa27c700 1 -- 172.21.15.100:0/817490582 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77a41a89e0 con 0x7f77a4109960 2024-05-10T11:43:20.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.974+0000 7f77a17fa700 1 -- 172.21.15.100:0/817490582 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f779c004bb0 con 0x7f77a4109960 2024-05-10T11:43:20.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.975+0000 7f77a17fa700 1 -- 172.21.15.100:0/817490582 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779c004020 con 0x7f77a4109960 2024-05-10T11:43:20.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.978+0000 7f77a17fa700 1 -- 172.21.15.100:0/817490582 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f779c01b030 con 0x7f77a4109960 2024-05-10T11:43:20.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.979+0000 7f77a17fa700 1 --2- 172.21.15.100:0/817490582 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f778c0b2900 0x7f778c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:20.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.979+0000 7f77a3fff700 1 --2- 172.21.15.100:0/817490582 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f778c0b2900 0x7f778c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:20.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.979+0000 7f77a17fa700 1 -- 172.21.15.100:0/817490582 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f779c0e1e30 con 0x7f77a4109960 2024-05-10T11:43:20.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.979+0000 7f77b4b0fb80 1 Processor -- start 2024-05-10T11:43:20.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.979+0000 7f77b4b0fb80 1 -- start start 2024-05-10T11:43:20.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.980+0000 7f77b4b0fb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55685dd63330 0x55685dd63710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:20.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.980+0000 7f77b4b0fb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55685dd63c50 0x55685dd5db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:20.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.980+0000 7f77b4b0fb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55685dd5e1a0 0x55685dd5e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:20.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.980+0000 7f77a3fff700 1 --2- 172.21.15.100:0/817490582 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f778c0b2900 0x7f778c0b4dc0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f7798009680 tx=0x7f7798009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:43:20.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.980+0000 7f77b4b0fb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55685dca78e0 con 0x55685dd63c50 2024-05-10T11:43:20.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.980+0000 7f77b4b0fb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55685dc46840 con 0x55685dd5e1a0 2024-05-10T11:43:20.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.980+0000 7f77b4b0fb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55685dca38a0 con 0x55685dd63330 2024-05-10T11:43:20.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.980+0000 7f77a37fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55685dd63c50 0x55685dd5db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:20.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.980+0000 7f77a37fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55685dd63c50 0x55685dd5db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35620/0 (socket says 172.21.15.100:35620) 2024-05-10T11:43:20.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.980+0000 7f77a37fe700 1 -- 172.21.15.100:0/1269711492 learned_addr learned my addr 172.21.15.100:0/1269711492 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:43:20.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.980+0000 7f77a3fff700 1 --2- 172.21.15.100:0/1269711492 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55685dd63330 0x55685dd63710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:20.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.980+0000 7f77a8819700 1 --2- 172.21.15.100:0/1269711492 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55685dd5e1a0 0x55685dd5e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:20.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.980+0000 7f77a37fe700 1 -- 172.21.15.100:0/1269711492 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55685dd63330 msgr2=0x55685dd63710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:43:20.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.980+0000 7f77a37fe700 1 --2- 172.21.15.100:0/1269711492 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55685dd63330 0x55685dd63710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:20.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.980+0000 7f77a37fe700 1 -- 172.21.15.100:0/1269711492 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55685dd5e1a0 msgr2=0x55685dd5e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:43:20.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.980+0000 7f77a37fe700 1 --2- 172.21.15.100:0/1269711492 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55685dd5e1a0 0x55685dd5e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:20.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.980+0000 7f77a37fe700 1 -- 172.21.15.100:0/1269711492 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55685dbf4950 con 0x55685dd63c50 2024-05-10T11:43:20.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.980+0000 7f77a3fff700 1 --2- 172.21.15.100:0/1269711492 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55685dd63330 0x55685dd63710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:43:20.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.980+0000 7f77a37fe700 1 --2- 172.21.15.100:0/1269711492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55685dd63c50 0x55685dd5db80 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f77940099c0 tx=0x7f77940036e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6028e06f36ab57f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:43:20.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.981+0000 7f778a7fc700 1 -- 172.21.15.100:0/1269711492 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7794027070 con 0x55685dd63c50 2024-05-10T11:43:20.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.981+0000 7f778a7fc700 1 -- 172.21.15.100:0/1269711492 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f779400bcf0 con 0x55685dd63c50 2024-05-10T11:43:20.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.981+0000 7f778a7fc700 1 -- 172.21.15.100:0/1269711492 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7794003bc0 con 0x55685dd63c50 2024-05-10T11:43:20.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.982+0000 7f77b4b0fb80 1 -- 172.21.15.100:0/1269711492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55685dd63c50 msgr2=0x55685dd5db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:20.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.982+0000 7f77b4b0fb80 1 --2- 172.21.15.100:0/1269711492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55685dd63c50 0x55685dd5db80 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f77940099c0 tx=0x7f77940036e0 comp rx=0 tx=0).stop 2024-05-10T11:43:20.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.982+0000 7f77b4b0fb80 1 -- 172.21.15.100:0/1269711492 shutdown_connections 2024-05-10T11:43:20.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.982+0000 7f77b4b0fb80 1 --2- 172.21.15.100:0/1269711492 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55685dd5e1a0 0x55685dd5e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:20.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.982+0000 7f77b4b0fb80 1 --2- 172.21.15.100:0/1269711492 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55685dd63330 0x55685dd63710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:20.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.982+0000 7f77b4b0fb80 1 --2- 172.21.15.100:0/1269711492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55685dd63c50 0x55685dd5db80 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:20.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.982+0000 7f77b4b0fb80 1 -- 172.21.15.100:0/1269711492 >> 172.21.15.100:0/1269711492 conn(0x55685dc81180 msgr2=0x55685dd4fcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:43:20.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.982+0000 7f77b4b0fb80 1 -- 172.21.15.100:0/1269711492 shutdown_connections 2024-05-10T11:43:20.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.982+0000 7f77b4b0fb80 1 -- 172.21.15.100:0/1269711492 wait complete. 2024-05-10T11:43:20.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.982+0000 7f77b4b0fb80 1 -- start start 2024-05-10T11:43:20.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.983+0000 7f77b4b0fb80 1 Processor -- start 2024-05-10T11:43:20.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.983+0000 7f77b4b0fb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55685dd5e1a0 0x55685dd69900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:20.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.983+0000 7f77b4b0fb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55685dd63330 0x55685dd66ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:20.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.983+0000 7f77b4b0fb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55685dd63c50 0x55685dd671e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:20.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.983+0000 7f77b4b0fb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55685dd6a6a0 con 0x55685dd63c50 2024-05-10T11:43:20.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.983+0000 7f77b4b0fb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55685dc0dcb0 con 0x55685dd5e1a0 2024-05-10T11:43:20.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.983+0000 7f77b4b0fb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55685dd6a820 con 0x55685dd63330 2024-05-10T11:43:20.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.983+0000 7f77b4b0fb80 10 client.0 _ll_register_callbacks cb 0x55685dc3f9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:43:20.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.983+0000 7f77a37fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55685dd63330 0x55685dd66ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:20.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.983+0000 7f77a3fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55685dd5e1a0 0x55685dd69900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:20.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.983+0000 7f77a37fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55685dd63330 0x55685dd66ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:56818/0 (socket says 172.21.15.100:56818) 2024-05-10T11:43:20.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.983+0000 7f77a37fe700 1 -- 172.21.15.100:0/3641569223 learned_addr learned my addr 172.21.15.100:0/3641569223 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:43:20.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.983+0000 7f77a8819700 1 --2- 172.21.15.100:0/3641569223 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55685dd63c50 0x55685dd671e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:20.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.983+0000 7f77a37fe700 1 -- 172.21.15.100:0/3641569223 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55685dd5e1a0 msgr2=0x55685dd69900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:43:20.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.983+0000 7f77a37fe700 1 --2- 172.21.15.100:0/3641569223 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55685dd5e1a0 0x55685dd69900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:20.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.983+0000 7f77a37fe700 1 -- 172.21.15.100:0/3641569223 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55685dd63c50 msgr2=0x55685dd671e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:43:20.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.983+0000 7f77a37fe700 1 --2- 172.21.15.100:0/3641569223 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55685dd63c50 0x55685dd671e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:20.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.983+0000 7f77a37fe700 1 -- 172.21.15.100:0/3641569223 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55685dc43710 con 0x55685dd63330 2024-05-10T11:43:20.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.984+0000 7f77a37fe700 1 --2- 172.21.15.100:0/3641569223 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55685dd63330 0x55685dd66ca0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f7794009990 tx=0x7f779400fa70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0c71705fa577c41 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:43:20.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.984+0000 7f77837fe700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:43:20.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.984+0000 7f77837fe700 1 -- 172.21.15.100:0/3641569223 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7794027040 con 0x55685dd63330 2024-05-10T11:43:20.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.984+0000 7f77837fe700 1 -- 172.21.15.100:0/3641569223 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f779400fc90 con 0x55685dd63330 2024-05-10T11:43:20.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.984+0000 7f77b4b0fb80 1 -- 172.21.15.100:0/3641569223 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f779400bcf0 con 0x55685dd63330 2024-05-10T11:43:20.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.984+0000 7f77837fe700 1 -- 172.21.15.100:0/3641569223 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f779400bea0 con 0x55685dd63330 2024-05-10T11:43:20.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.984+0000 7f77b4b0fb80 10 client.29536 fetch_fsmap learned FSMap version 41 2024-05-10T11:43:20.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.984+0000 7f77b4b0fb80 1 -- 172.21.15.100:0/3641569223 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55685dd6a6a0 con 0x55685dd63330 2024-05-10T11:43:20.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.985+0000 7f77837fe700 1 -- 172.21.15.100:0/3641569223 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f77940058e0 con 0x55685dd63330 2024-05-10T11:43:20.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.985+0000 7f77837fe700 1 -- 172.21.15.100:0/3641569223 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f7794033030 con 0x55685dd63330 2024-05-10T11:43:20.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.985+0000 7f77b4b0fb80 10 client.29536 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:43:20.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.985+0000 7f77b4b0fb80 10 client.29536 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:43:20.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.985+0000 7f77b4b0fb80 20 client.29536 populate_metadata read hostname 'smithi100' 2024-05-10T11:43:20.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.985+0000 7f77b4b0fb80 1 --2- 172.21.15.100:0/3641569223 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55685dd771f0 0x55685dd795e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:20.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.985+0000 7f77b4b0fb80 4 client.29536 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:43:20.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.985+0000 7f77b4b0fb80 1 -- 172.21.15.100:0/3641569223 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f779400bcf0 con 0x55685dd771f0 2024-05-10T11:43:20.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.985+0000 7f77a3fff700 1 --2- 172.21.15.100:0/3641569223 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55685dd771f0 0x55685dd795e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:20.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.986+0000 7f77837fe700 10 client.29536 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:43:20.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.986+0000 7f77a3fff700 1 --2- 172.21.15.100:0/3641569223 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55685dd771f0 0x55685dd795e0 crc :-1 s=READY pgs=461 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a8d35c50f570f7c server_cookie=f69b120fa314f612 in_seq=0 out_seq=0 2024-05-10T11:43:20.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.987+0000 7f77837fe700 1 -- 172.21.15.100:0/3641569223 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f779400bcf0 con 0x55685dd771f0 2024-05-10T11:43:20.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.987+0000 7f77837fe700 10 client.29536 handle_command_reply: tid=0 2024-05-10T11:43:20.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.987+0000 7f77b4b0fb80 1 client.29536 shutdown 2024-05-10T11:43:20.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.987+0000 7f77b4b0fb80 1 -- 172.21.15.100:0/3641569223 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55685dd63330 msgr2=0x55685dd66ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:20.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.987+0000 7f77b4b0fb80 1 --2- 172.21.15.100:0/3641569223 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55685dd63330 0x55685dd66ca0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f7794009990 tx=0x7f779400fa70 comp rx=0 tx=0).stop 2024-05-10T11:43:20.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.987+0000 7f77b4b0fb80 1 -- 172.21.15.100:0/3641569223 shutdown_connections 2024-05-10T11:43:20.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.987+0000 7f77b4b0fb80 1 --2- 172.21.15.100:0/3641569223 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55685dd771f0 0x55685dd795e0 crc :-1 s=READY pgs=461 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:20.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.987+0000 7f77b4b0fb80 1 --2- 172.21.15.100:0/3641569223 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55685dd63c50 0x55685dd671e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:20.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.987+0000 7f77b4b0fb80 1 --2- 172.21.15.100:0/3641569223 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55685dd5e1a0 0x55685dd69900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:20.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.987+0000 7f77b4b0fb80 1 --2- 172.21.15.100:0/3641569223 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55685dd63330 0x55685dd66ca0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:20.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.987+0000 7f77b4b0fb80 1 -- 172.21.15.100:0/3641569223 >> 172.21.15.100:0/3641569223 conn(0x55685dc81180 msgr2=0x55685dd4a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:43:20.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.987+0000 7f77837fe700 0 client.29536 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:43:20.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.988+0000 7f77b4b0fb80 1 -- 172.21.15.100:0/3641569223 shutdown_connections 2024-05-10T11:43:20.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.988+0000 7f77b4b0fb80 1 -- 172.21.15.100:0/3641569223 wait complete. 2024-05-10T11:43:20.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:19.988+0000 7f77b4b0fb80 20 client.29536 trim_cache size 0 max 16384 2024-05-10T11:43:20.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.020+0000 7f77b4b0fb80 1 Processor -- start 2024-05-10T11:43:20.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.021+0000 7f77b4b0fb80 1 -- start start 2024-05-10T11:43:20.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.021+0000 7f77b4b0fb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55685dd5e1a0 0x55685dd737c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:20.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.021+0000 7f77b4b0fb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55685dd63330 0x55685dd7ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:20.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.021+0000 7f77b4b0fb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55685dd63c50 0x55685dd7f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:20.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.021+0000 7f77b4b0fb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7790011290 con 0x55685dd63c50 2024-05-10T11:43:20.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.021+0000 7f77b4b0fb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f779000eb20 con 0x55685dd63330 2024-05-10T11:43:20.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.021+0000 7f77b4b0fb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f779000c970 con 0x55685dd5e1a0 2024-05-10T11:43:20.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.021+0000 7f77a37fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55685dd63330 0x55685dd7ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:20.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.021+0000 7f77a3fff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55685dd5e1a0 0x55685dd737c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:20.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.021+0000 7f77a37fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55685dd63330 0x55685dd7ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:34468/0 (socket says 172.21.15.100:34468) 2024-05-10T11:43:20.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.021+0000 7f77a37fe700 1 -- 172.21.15.100:0/1613178010 learned_addr learned my addr 172.21.15.100:0/1613178010 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:43:20.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.021+0000 7f77a8819700 1 --2- 172.21.15.100:0/1613178010 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55685dd63c50 0x55685dd7f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:20.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.021+0000 7f77a37fe700 1 -- 172.21.15.100:0/1613178010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55685dd5e1a0 msgr2=0x55685dd737c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:43:20.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.021+0000 7f77a37fe700 1 --2- 172.21.15.100:0/1613178010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55685dd5e1a0 0x55685dd737c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:20.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.021+0000 7f77a37fe700 1 -- 172.21.15.100:0/1613178010 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55685dd63c50 msgr2=0x55685dd7f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:43:20.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.021+0000 7f77a37fe700 1 --2- 172.21.15.100:0/1613178010 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55685dd63c50 0x55685dd7f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:20.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.021+0000 7f77a37fe700 1 -- 172.21.15.100:0/1613178010 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55685dd62e30 con 0x55685dd63330 2024-05-10T11:43:20.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.022+0000 7f77a37fe700 1 --2- 172.21.15.100:0/1613178010 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55685dd63330 0x55685dd7ec10 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f7794000c00 tx=0x7f7794009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e1988556425dbcd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:43:20.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.022+0000 7f7789ffb700 1 -- 172.21.15.100:0/1613178010 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7794027040 con 0x55685dd63330 2024-05-10T11:43:20.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.022+0000 7f7789ffb700 1 -- 172.21.15.100:0/1613178010 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f77940058e0 con 0x55685dd63330 2024-05-10T11:43:20.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.022+0000 7f7789ffb700 1 -- 172.21.15.100:0/1613178010 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7794005370 con 0x55685dd63330 2024-05-10T11:43:20.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.023+0000 7f77b4b0fb80 1 -- 172.21.15.100:0/1613178010 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55685dd63330 msgr2=0x55685dd7ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:20.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.023+0000 7f77b4b0fb80 1 --2- 172.21.15.100:0/1613178010 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55685dd63330 0x55685dd7ec10 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f7794000c00 tx=0x7f7794009310 comp rx=0 tx=0).stop 2024-05-10T11:43:20.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.023+0000 7f77b4b0fb80 1 -- 172.21.15.100:0/1613178010 shutdown_connections 2024-05-10T11:43:20.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.023+0000 7f77b4b0fb80 1 --2- 172.21.15.100:0/1613178010 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55685dd63c50 0x55685dd7f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:20.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.023+0000 7f77b4b0fb80 1 --2- 172.21.15.100:0/1613178010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55685dd5e1a0 0x55685dd737c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:20.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.023+0000 7f77b4b0fb80 1 --2- 172.21.15.100:0/1613178010 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55685dd63330 0x55685dd7ec10 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:20.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.023+0000 7f77b4b0fb80 1 -- 172.21.15.100:0/1613178010 >> 172.21.15.100:0/1613178010 conn(0x55685dc81180 msgr2=0x55685dd655b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:43:20.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.023+0000 7f77b4b0fb80 1 -- 172.21.15.100:0/1613178010 shutdown_connections 2024-05-10T11:43:20.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.023+0000 7f77b4b0fb80 1 -- 172.21.15.100:0/1613178010 wait complete. 2024-05-10T11:43:20.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.023+0000 7f77b4b0fb80 1 -- start start 2024-05-10T11:43:20.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.024+0000 7f77b4b0fb80 1 Processor -- start 2024-05-10T11:43:20.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.024+0000 7f77b4b0fb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55685dd5e1a0 0x55685dd7e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:20.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.024+0000 7f77b4b0fb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55685dd63330 0x55685dd7e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:20.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.024+0000 7f77b4b0fb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55685dd63c50 0x55685dd7ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:20.269 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:43:20.269 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-05-10T11:43:20.269 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "8f368896-e12a-4ef6-a300-4d80f2c0c4bc", 2024-05-10T11:43:20.269 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-05-10T11:43:20.270 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:43:20.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.024+0000 7f77b4b0fb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f779000cfe0 con 0x55685dd5e1a0 2024-05-10T11:43:20.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.024+0000 7f77b4b0fb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f779000e450 con 0x55685dd63330 2024-05-10T11:43:20.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.024+0000 7f77b4b0fb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f779000d6b0 con 0x55685dd63c50 2024-05-10T11:43:20.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.024+0000 7f77a3fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55685dd5e1a0 0x55685dd7e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:20.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.024+0000 7f77a8819700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55685dd63c50 0x55685dd7ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:20.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.024+0000 7f77a8819700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55685dd63c50 0x55685dd7ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:56832/0 (socket says 172.21.15.100:56832) 2024-05-10T11:43:20.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.024+0000 7f77a8819700 1 -- 172.21.15.100:0/1058241059 learned_addr learned my addr 172.21.15.100:0/1058241059 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:43:20.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.024+0000 7f77b4b0fb80 10 client.0 _ll_register_callbacks cb 0x55685dc3f9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:43:20.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.024+0000 7f77a37fe700 1 --2- 172.21.15.100:0/1058241059 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55685dd63330 0x55685dd7e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:20.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.025+0000 7f77a8819700 1 -- 172.21.15.100:0/1058241059 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55685dd63330 msgr2=0x55685dd7e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:43:20.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.025+0000 7f77a8819700 1 --2- 172.21.15.100:0/1058241059 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55685dd63330 0x55685dd7e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:20.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.025+0000 7f77a8819700 1 -- 172.21.15.100:0/1058241059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55685dd5e1a0 msgr2=0x55685dd7e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:43:20.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.025+0000 7f77a8819700 1 --2- 172.21.15.100:0/1058241059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55685dd5e1a0 0x55685dd7e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:20.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.025+0000 7f77a8819700 1 -- 172.21.15.100:0/1058241059 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55685dde0870 con 0x55685dd63c50 2024-05-10T11:43:20.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.025+0000 7f77a8819700 1 --2- 172.21.15.100:0/1058241059 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55685dd63c50 0x55685dd7ece0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f779c00c9f0 tx=0x7f779c0a96c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d91355ea24914471 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:43:20.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.025+0000 7f77837fe700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:43:20.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.025+0000 7f77837fe700 1 -- 172.21.15.100:0/1058241059 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779c0a9880 con 0x55685dd63c50 2024-05-10T11:43:20.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.025+0000 7f77837fe700 1 -- 172.21.15.100:0/1058241059 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f779c0a83e0 con 0x55685dd63c50 2024-05-10T11:43:20.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.025+0000 7f77b4b0fb80 1 -- 172.21.15.100:0/1058241059 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77940058e0 con 0x55685dd63c50 2024-05-10T11:43:20.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.025+0000 7f77837fe700 1 -- 172.21.15.100:0/1058241059 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f779c0a9180 con 0x55685dd63c50 2024-05-10T11:43:20.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.025+0000 7f77b4b0fb80 10 client.29542 fetch_fsmap learned FSMap version 41 2024-05-10T11:43:20.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.025+0000 7f77b4b0fb80 1 -- 172.21.15.100:0/1058241059 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55685dde1bb0 con 0x55685dd63c50 2024-05-10T11:43:20.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.025+0000 7f77837fe700 1 -- 172.21.15.100:0/1058241059 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f77940058e0 con 0x55685dd63c50 2024-05-10T11:43:20.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.026+0000 7f77837fe700 1 -- 172.21.15.100:0/1058241059 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f779c0a4af0 con 0x55685dd63c50 2024-05-10T11:43:20.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.026+0000 7f77b4b0fb80 10 client.29542 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:43:20.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.026+0000 7f77b4b0fb80 10 client.29542 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:43:20.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.026+0000 7f77b4b0fb80 20 client.29542 populate_metadata read hostname 'smithi100' 2024-05-10T11:43:20.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.026+0000 7f77b4b0fb80 1 --2- 172.21.15.100:0/1058241059 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55685dde2260 0x55685dde4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:20.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.026+0000 7f77b4b0fb80 4 client.29542 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T11:43:20.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.026+0000 7f77b4b0fb80 1 -- 172.21.15.100:0/1058241059 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55685dd216d0 con 0x55685dde2260 2024-05-10T11:43:20.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.026+0000 7f77a3fff700 1 --2- 172.21.15.100:0/1058241059 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55685dde2260 0x55685dde4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:20.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.026+0000 7f77837fe700 10 client.29542 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:43:20.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.026+0000 7f77a3fff700 1 --2- 172.21.15.100:0/1058241059 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55685dde2260 0x55685dde4680 crc :-1 s=READY pgs=462 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69948341841a29fa server_cookie=40f19c1268b59c6a in_seq=0 out_seq=0 2024-05-10T11:43:20.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.028+0000 7f77837fe700 1 -- 172.21.15.100:0/1058241059 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55685dd216d0 con 0x55685dde2260 2024-05-10T11:43:20.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.028+0000 7f77837fe700 10 client.29542 handle_command_reply: tid=0 2024-05-10T11:43:20.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.028+0000 7f77b4b0fb80 1 client.29542 shutdown 2024-05-10T11:43:20.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.028+0000 7f77b4b0fb80 1 -- 172.21.15.100:0/1058241059 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55685dd63c50 msgr2=0x55685dd7ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:20.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.028+0000 7f77b4b0fb80 1 --2- 172.21.15.100:0/1058241059 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55685dd63c50 0x55685dd7ece0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f779c00c9f0 tx=0x7f779c0a96c0 comp rx=0 tx=0).stop 2024-05-10T11:43:20.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.028+0000 7f77b4b0fb80 1 -- 172.21.15.100:0/1058241059 shutdown_connections 2024-05-10T11:43:20.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.028+0000 7f77b4b0fb80 1 --2- 172.21.15.100:0/1058241059 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55685dde2260 0x55685dde4680 crc :-1 s=READY pgs=462 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:20.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.028+0000 7f77b4b0fb80 1 --2- 172.21.15.100:0/1058241059 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55685dd63c50 0x55685dd7ece0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:20.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.028+0000 7f77837fe700 0 client.29542 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:43:20.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.028+0000 7f77b4b0fb80 1 --2- 172.21.15.100:0/1058241059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55685dd5e1a0 0x55685dd7e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:20.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.028+0000 7f77b4b0fb80 1 --2- 172.21.15.100:0/1058241059 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55685dd63330 0x55685dd7e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:20.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.028+0000 7f77b4b0fb80 1 -- 172.21.15.100:0/1058241059 >> 172.21.15.100:0/1058241059 conn(0x55685dc81180 msgr2=0x55685dd64030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:43:20.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.029+0000 7f77b4b0fb80 1 -- 172.21.15.100:0/1058241059 shutdown_connections 2024-05-10T11:43:20.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.029+0000 7f77b4b0fb80 1 -- 172.21.15.100:0/1058241059 wait complete. 2024-05-10T11:43:20.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.029+0000 7f77b4b0fb80 20 client.29542 trim_cache size 0 max 16384 2024-05-10T11:43:20.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.029+0000 7f77aa27c700 1 -- 172.21.15.100:0/817490582 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f778c0b2900 msgr2=0x7f778c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:20.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.029+0000 7f77aa27c700 1 --2- 172.21.15.100:0/817490582 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f778c0b2900 0x7f778c0b4dc0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f7798009680 tx=0x7f7798009260 comp rx=0 tx=0).stop 2024-05-10T11:43:20.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.029+0000 7f77aa27c700 1 -- 172.21.15.100:0/817490582 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77a4109960 msgr2=0x7f77a419bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:20.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.029+0000 7f77aa27c700 1 --2- 172.21.15.100:0/817490582 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77a4109960 0x7f77a419bb30 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f779c00efc0 tx=0x7f779c00c3a0 comp rx=0 tx=0).stop 2024-05-10T11:43:20.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.030+0000 7f77aa27c700 1 -- 172.21.15.100:0/817490582 shutdown_connections 2024-05-10T11:43:20.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.030+0000 7f77aa27c700 1 --2- 172.21.15.100:0/817490582 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f778c0b2900 0x7f778c0b4dc0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:20.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.030+0000 7f77aa27c700 1 --2- 172.21.15.100:0/817490582 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77a4109960 0x7f77a419bb30 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:20.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.030+0000 7f77aa27c700 1 --2- 172.21.15.100:0/817490582 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77a4101880 0x7f77a41a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:20.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.030+0000 7f77aa27c700 1 --2- 172.21.15.100:0/817490582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77a4102230 0x7f77a41a1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:20.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.030+0000 7f77aa27c700 1 -- 172.21.15.100:0/817490582 >> 172.21.15.100:0/817490582 conn(0x7f77a40785a0 msgr2=0x7f77a40ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:43:20.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.030+0000 7f77aa27c700 1 -- 172.21.15.100:0/817490582 shutdown_connections 2024-05-10T11:43:20.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:20.030+0000 7f77aa27c700 1 -- 172.21.15.100:0/817490582 wait complete. 2024-05-10T11:43:20.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:43:20 smithi178 ceph-mon[33359]: pgmap v1169: 129 pgs: 129 active+clean; 565 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 3.9 MiB/s wr, 254 op/s 2024-05-10T11:43:20.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:43:20 smithi188 ceph-mon[34075]: pgmap v1169: 129 pgs: 129 active+clean; 565 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 3.9 MiB/s wr, 254 op/s 2024-05-10T11:43:20.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:43:20 smithi100 ceph-mon[30296]: pgmap v1169: 129 pgs: 129 active+clean; 565 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 3.9 MiB/s wr, 254 op/s 2024-05-10T11:43:20.961 INFO:tasks.workunit.client.0.smithi100.stdout: 10 111441 8.88 MB/sec execute 554 sec latency 113.925 ms 2024-05-10T11:43:21.064 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T11:43:21.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:43:21 smithi100 ceph-mon[30296]: scrub queued for path: / 2024-05-10T11:43:21.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:43:21 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:43:21.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:43:21 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:43:21.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:43:21 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:43:21.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:43:21 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:43:21.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:43:21 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:43:21.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:43:21 smithi100 ceph-mon[30296]: scrub summary: active paths [/] 2024-05-10T11:43:21.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:43:21 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:43:21.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:43:21 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:43:21.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:43:21 smithi100 ceph-mon[30296]: scrub summary: idle+waiting paths [/] 2024-05-10T11:43:21.431 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:43:21.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:43:21 smithi178 ceph-mon[33359]: scrub queued for path: / 2024-05-10T11:43:21.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:43:21 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:43:21.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:43:21 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:43:21.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:43:21 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:43:21.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:43:21 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:43:21.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:43:21 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:43:21.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:43:21 smithi178 ceph-mon[33359]: scrub summary: active paths [/] 2024-05-10T11:43:21.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:43:21 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:43:21.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:43:21 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:43:21.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:43:21 smithi178 ceph-mon[33359]: scrub summary: idle+waiting paths [/] 2024-05-10T11:43:21.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:43:21 smithi188 ceph-mon[34075]: scrub queued for path: / 2024-05-10T11:43:21.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:43:21 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:43:21.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:43:21 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:43:21.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:43:21 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:43:21.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:43:21 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:43:21.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:43:21 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:43:21.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:43:21 smithi188 ceph-mon[34075]: scrub summary: active paths [/] 2024-05-10T11:43:21.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:43:21 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:43:21.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:43:21 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:43:21.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:43:21 smithi188 ceph-mon[34075]: scrub summary: idle+waiting paths [/] 2024-05-10T11:43:21.693 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.692+0000 7f566fb89640 1 -- 172.21.15.100:0/3681452151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5668102b00 msgr2=0x7f5668102f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:21.693 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.692+0000 7f566fb89640 1 --2- 172.21.15.100:0/3681452151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5668102b00 0x7f5668102f80 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f5664009900 tx=0x7f566402a410 comp rx=0 tx=0).stop 2024-05-10T11:43:21.693 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.692+0000 7f566fb89640 1 -- 172.21.15.100:0/3681452151 shutdown_connections 2024-05-10T11:43:21.693 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.692+0000 7f566fb89640 1 --2- 172.21.15.100:0/3681452151 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56681034c0 0x7f5668109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:21.694 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.692+0000 7f566fb89640 1 --2- 172.21.15.100:0/3681452151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5668102b00 0x7f5668102f80 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:21.694 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.692+0000 7f566fb89640 1 --2- 172.21.15.100:0/3681452151 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5668101910 0x7f5668101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:21.694 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.692+0000 7f566fb89640 1 -- 172.21.15.100:0/3681452151 >> 172.21.15.100:0/3681452151 conn(0x7f56680fce80 msgr2=0x7f56680ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:43:21.694 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.693+0000 7f566fb89640 1 -- 172.21.15.100:0/3681452151 shutdown_connections 2024-05-10T11:43:21.694 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.693+0000 7f566fb89640 1 -- 172.21.15.100:0/3681452151 wait complete. 2024-05-10T11:43:21.694 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.693+0000 7f566fb89640 1 Processor -- start 2024-05-10T11:43:21.694 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.693+0000 7f566fb89640 1 -- start start 2024-05-10T11:43:21.696 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.694+0000 7f566fb89640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5668101910 0x7f566819ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:21.696 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.694+0000 7f566fb89640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5668102b00 0x7f566819b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:21.696 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.694+0000 7f566fb89640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56681034c0 0x7f56681a22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:21.696 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.694+0000 7f566fb89640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f566810e430 con 0x7f5668102b00 2024-05-10T11:43:21.696 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.694+0000 7f566fb89640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f566810e2b0 con 0x7f5668101910 2024-05-10T11:43:21.696 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.694+0000 7f566fb89640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f566810e5b0 con 0x7f56681034c0 2024-05-10T11:43:21.696 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.694+0000 7f566d124640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5668102b00 0x7f566819b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:21.696 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.694+0000 7f566d925640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5668101910 0x7f566819ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:21.696 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.694+0000 7f566d124640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5668102b00 0x7f566819b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35650/0 (socket says 172.21.15.100:35650) 2024-05-10T11:43:21.696 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.694+0000 7f566d925640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5668101910 0x7f566819ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:34506/0 (socket says 172.21.15.100:34506) 2024-05-10T11:43:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.694+0000 7f566d124640 1 -- 172.21.15.100:0/790001866 learned_addr learned my addr 172.21.15.100:0/790001866 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:43:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.694+0000 7f566e126640 1 --2- 172.21.15.100:0/790001866 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56681034c0 0x7f56681a22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.694+0000 7f566d124640 1 -- 172.21.15.100:0/790001866 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56681034c0 msgr2=0x7f56681a22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.694+0000 7f566d124640 1 --2- 172.21.15.100:0/790001866 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56681034c0 0x7f56681a22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.694+0000 7f566d124640 1 -- 172.21.15.100:0/790001866 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5668101910 msgr2=0x7f566819ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.694+0000 7f566d124640 1 --2- 172.21.15.100:0/790001866 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5668101910 0x7f566819ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.694+0000 7f566d124640 1 -- 172.21.15.100:0/790001866 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56681a2960 con 0x7f5668102b00 2024-05-10T11:43:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.694+0000 7f566d925640 1 --2- 172.21.15.100:0/790001866 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5668101910 0x7f566819ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:43:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.695+0000 7f566d124640 1 --2- 172.21.15.100:0/790001866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5668102b00 0x7f566819b280 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f5664031460 tx=0x7f5664003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:43:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.695+0000 7f565affd640 1 -- 172.21.15.100:0/790001866 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566403f070 con 0x7f5668102b00 2024-05-10T11:43:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.695+0000 7f566fb89640 1 -- 172.21.15.100:0/790001866 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56681a2bf0 con 0x7f5668102b00 2024-05-10T11:43:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.695+0000 7f565affd640 1 -- 172.21.15.100:0/790001866 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5664031ba0 con 0x7f5668102b00 2024-05-10T11:43:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.695+0000 7f565affd640 1 -- 172.21.15.100:0/790001866 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5664035030 con 0x7f5668102b00 2024-05-10T11:43:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.695+0000 7f566fb89640 1 -- 172.21.15.100:0/790001866 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56681a3130 con 0x7f5668102b00 2024-05-10T11:43:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.698+0000 7f567297c740 1 Processor -- start 2024-05-10T11:43:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.698+0000 7f567297c740 1 -- start start 2024-05-10T11:43:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.698+0000 7f567297c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607b94375c0 0x5607b94379e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.698+0000 7f567297c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607b9437fb0 0x5607b943cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.698+0000 7f567297c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607b943d120 0x5607b943f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.698+0000 7f567297c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5607b93935d0 con 0x5607b94375c0 2024-05-10T11:43:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.698+0000 7f567297c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5607b934c890 con 0x5607b943d120 2024-05-10T11:43:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.698+0000 7f567297c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5607b938f590 con 0x5607b9437fb0 2024-05-10T11:43:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.699+0000 7f565affd640 1 -- 172.21.15.100:0/790001866 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f5664042070 con 0x7f5668102b00 2024-05-10T11:43:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.699+0000 7f566e126640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607b94375c0 0x5607b94379e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.699+0000 7f566d925640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607b9437fb0 0x5607b943cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:21.701 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.699+0000 7f566e126640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607b94375c0 0x5607b94379e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35664/0 (socket says 172.21.15.100:35664) 2024-05-10T11:43:21.701 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.699+0000 7f566d925640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607b9437fb0 0x5607b943cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:56866/0 (socket says 172.21.15.100:56866) 2024-05-10T11:43:21.701 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.699+0000 7f566e126640 1 -- 172.21.15.100:0/3157773817 learned_addr learned my addr 172.21.15.100:0/3157773817 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:43:21.701 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.699+0000 7f566d124640 1 --2- 172.21.15.100:0/3157773817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607b943d120 0x5607b943f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:21.701 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.699+0000 7f566e126640 1 -- 172.21.15.100:0/3157773817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607b9437fb0 msgr2=0x5607b943cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:43:21.701 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.699+0000 7f566e126640 1 --2- 172.21.15.100:0/3157773817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607b9437fb0 0x5607b943cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:21.701 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.699+0000 7f566e126640 1 -- 172.21.15.100:0/3157773817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607b943d120 msgr2=0x5607b943f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:43:21.701 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.699+0000 7f566e126640 1 --2- 172.21.15.100:0/3157773817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607b943d120 0x5607b943f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:21.701 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.699+0000 7f566e126640 1 -- 172.21.15.100:0/3157773817 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607b90a5210 con 0x5607b94375c0 2024-05-10T11:43:21.701 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.699+0000 7f566d124640 1 --2- 172.21.15.100:0/3157773817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607b943d120 0x5607b943f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:43:21.701 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.699+0000 7f566e126640 1 --2- 172.21.15.100:0/3157773817 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607b94375c0 0x5607b94379e0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f565c00bc70 tx=0x7f565c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=292160d4f833aa0f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:43:21.701 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.700+0000 7f5653fff640 1 -- 172.21.15.100:0/3157773817 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565c015070 con 0x5607b94375c0 2024-05-10T11:43:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.700+0000 7f565affd640 1 --2- 172.21.15.100:0/790001866 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f56540b2920 0x7f56540b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.700+0000 7f5653fff640 1 -- 172.21.15.100:0/3157773817 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f565c01a8f0 con 0x5607b94375c0 2024-05-10T11:43:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.700+0000 7f566d925640 1 --2- 172.21.15.100:0/790001866 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f56540b2920 0x7f56540b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.700+0000 7f565affd640 1 -- 172.21.15.100:0/790001866 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f56641031c0 con 0x7f5668102b00 2024-05-10T11:43:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.700+0000 7f5653fff640 1 -- 172.21.15.100:0/3157773817 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565c011810 con 0x5607b94375c0 2024-05-10T11:43:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.701+0000 7f567297c740 1 -- 172.21.15.100:0/3157773817 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607b94375c0 msgr2=0x5607b94379e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.701+0000 7f566d925640 1 --2- 172.21.15.100:0/790001866 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f56540b2920 0x7f56540b4de0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x5607b94383b0 tx=0x7f5660009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:43:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.701+0000 7f567297c740 1 --2- 172.21.15.100:0/3157773817 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607b94375c0 0x5607b94379e0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f565c00bc70 tx=0x7f565c00f730 comp rx=0 tx=0).stop 2024-05-10T11:43:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.701+0000 7f567297c740 1 -- 172.21.15.100:0/3157773817 shutdown_connections 2024-05-10T11:43:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.701+0000 7f567297c740 1 --2- 172.21.15.100:0/3157773817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607b943d120 0x5607b943f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.701+0000 7f567297c740 1 --2- 172.21.15.100:0/3157773817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607b9437fb0 0x5607b943cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.701+0000 7f567297c740 1 --2- 172.21.15.100:0/3157773817 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607b94375c0 0x5607b94379e0 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.701+0000 7f567297c740 1 -- 172.21.15.100:0/3157773817 >> 172.21.15.100:0/3157773817 conn(0x5607b9432c30 msgr2=0x5607b9433030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:43:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.701+0000 7f567297c740 1 -- 172.21.15.100:0/3157773817 shutdown_connections 2024-05-10T11:43:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.701+0000 7f567297c740 1 -- 172.21.15.100:0/3157773817 wait complete. 2024-05-10T11:43:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.701+0000 7f567297c740 1 -- start start 2024-05-10T11:43:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.701+0000 7f567297c740 1 Processor -- start 2024-05-10T11:43:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.702+0000 7f567297c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607b94375c0 0x5607b94418e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.702+0000 7f567297c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607b9437fb0 0x5607b9441e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.702+0000 7f567297c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607b943d120 0x5607b9448e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.702+0000 7f566d124640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607b9437fb0 0x5607b9441e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.702+0000 7f566d925640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607b94375c0 0x5607b94418e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.702+0000 7f566d124640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607b9437fb0 0x5607b9441e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:34526/0 (socket says 172.21.15.100:34526) 2024-05-10T11:43:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.702+0000 7f566d925640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607b94375c0 0x5607b94418e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:56882/0 (socket says 172.21.15.100:56882) 2024-05-10T11:43:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.702+0000 7f566d124640 1 -- 172.21.15.100:0/1840976900 learned_addr learned my addr 172.21.15.100:0/1840976900 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:43:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.702+0000 7f566e126640 1 --2- 172.21.15.100:0/1840976900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607b943d120 0x5607b9448e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.703+0000 7f567297c740 1 -- 172.21.15.100:0/1840976900 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5607b9440120 con 0x5607b943d120 2024-05-10T11:43:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.703+0000 7f567297c740 1 -- 172.21.15.100:0/1840976900 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5607b9350370 con 0x5607b9437fb0 2024-05-10T11:43:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.703+0000 7f567297c740 1 -- 172.21.15.100:0/1840976900 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5607b94402a0 con 0x5607b94375c0 2024-05-10T11:43:21.708 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.703+0000 7f567297c740 10 client.0 _ll_register_callbacks cb 0x5607b90a4a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:43:21.708 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.703+0000 7f566d124640 1 -- 172.21.15.100:0/1840976900 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607b94375c0 msgr2=0x5607b94418e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:43:21.708 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.703+0000 7f566d124640 1 --2- 172.21.15.100:0/1840976900 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607b94375c0 0x5607b94418e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:21.708 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.703+0000 7f566d124640 1 -- 172.21.15.100:0/1840976900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607b943d120 msgr2=0x5607b9448e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:43:21.708 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.703+0000 7f566d124640 1 --2- 172.21.15.100:0/1840976900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607b943d120 0x5607b9448e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:21.708 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.703+0000 7f566d124640 1 -- 172.21.15.100:0/1840976900 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607b9171620 con 0x5607b9437fb0 2024-05-10T11:43:21.708 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.703+0000 7f566d124640 1 --2- 172.21.15.100:0/1840976900 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607b9437fb0 0x5607b9441e20 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f56640ca120 tx=0x7f56640c9e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b582bc15de0c67bc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:43:21.708 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.703+0000 7f56517fa640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:43:21.708 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.704+0000 7f56517fa640 1 -- 172.21.15.100:0/1840976900 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5664049050 con 0x5607b9437fb0 2024-05-10T11:43:21.708 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.704+0000 7f567297c740 1 -- 172.21.15.100:0/1840976900 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f565c01a8f0 con 0x5607b9437fb0 2024-05-10T11:43:21.708 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.704+0000 7f56517fa640 1 -- 172.21.15.100:0/1840976900 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f56640c66b0 con 0x5607b9437fb0 2024-05-10T11:43:21.708 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.704+0000 7f56517fa640 1 -- 172.21.15.100:0/1840976900 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f56640cab20 con 0x5607b9437fb0 2024-05-10T11:43:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.704+0000 7f567297c740 10 client.29435 fetch_fsmap learned FSMap version 41 2024-05-10T11:43:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.704+0000 7f567297c740 1 -- 172.21.15.100:0/1840976900 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5607b944be20 con 0x5607b9437fb0 2024-05-10T11:43:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.704+0000 7f56517fa640 1 -- 172.21.15.100:0/1840976900 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f565c01a8f0 con 0x5607b9437fb0 2024-05-10T11:43:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.705+0000 7f56517fa640 1 -- 172.21.15.100:0/1840976900 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f566403b460 con 0x5607b9437fb0 2024-05-10T11:43:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.705+0000 7f567297c740 10 client.29435 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:43:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.705+0000 7f567297c740 10 client.29435 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:43:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.705+0000 7f567297c740 20 client.29435 populate_metadata read hostname 'smithi100' 2024-05-10T11:43:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.705+0000 7f567297c740 1 --2- 172.21.15.100:0/1840976900 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5607b944d6e0 0x5607b944fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.705+0000 7f567297c740 4 client.29435 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:43:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.705+0000 7f567297c740 1 -- 172.21.15.100:0/1840976900 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607b93a11e0 con 0x5607b944d6e0 2024-05-10T11:43:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.705+0000 7f566d925640 1 --2- 172.21.15.100:0/1840976900 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5607b944d6e0 0x5607b944fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.706+0000 7f566d925640 1 --2- 172.21.15.100:0/1840976900 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5607b944d6e0 0x5607b944fb30 crc :-1 s=READY pgs=463 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34fa03da403ea64d server_cookie=a509231e619f3725 in_seq=0 out_seq=0 2024-05-10T11:43:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.706+0000 7f56517fa640 10 client.29435 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:43:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.706+0000 7f56517fa640 1 -- 172.21.15.100:0/1840976900 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5607b93a11e0 con 0x5607b944d6e0 2024-05-10T11:43:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.706+0000 7f56517fa640 10 client.29435 handle_command_reply: tid=0 2024-05-10T11:43:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.706+0000 7f567297c740 1 client.29435 shutdown 2024-05-10T11:43:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.707+0000 7f567297c740 1 -- 172.21.15.100:0/1840976900 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607b9437fb0 msgr2=0x5607b9441e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.707+0000 7f567297c740 1 --2- 172.21.15.100:0/1840976900 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607b9437fb0 0x5607b9441e20 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f56640ca120 tx=0x7f56640c9e30 comp rx=0 tx=0).stop 2024-05-10T11:43:21.712 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.707+0000 7f567297c740 1 -- 172.21.15.100:0/1840976900 shutdown_connections 2024-05-10T11:43:21.712 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.707+0000 7f567297c740 1 --2- 172.21.15.100:0/1840976900 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5607b944d6e0 0x5607b944fb30 crc :-1 s=READY pgs=463 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:21.712 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.707+0000 7f567297c740 1 --2- 172.21.15.100:0/1840976900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607b943d120 0x5607b9448e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:21.712 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.707+0000 7f56517fa640 0 client.29435 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:43:21.712 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.707+0000 7f567297c740 1 --2- 172.21.15.100:0/1840976900 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607b9437fb0 0x5607b9441e20 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:21.712 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.707+0000 7f567297c740 1 --2- 172.21.15.100:0/1840976900 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607b94375c0 0x5607b94418e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:21.712 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.707+0000 7f567297c740 1 -- 172.21.15.100:0/1840976900 >> 172.21.15.100:0/1840976900 conn(0x5607b9432c30 msgr2=0x5607b9429580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:43:21.712 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.707+0000 7f567297c740 1 -- 172.21.15.100:0/1840976900 shutdown_connections 2024-05-10T11:43:21.712 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.707+0000 7f567297c740 1 -- 172.21.15.100:0/1840976900 wait complete. 2024-05-10T11:43:21.712 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.708+0000 7f567297c740 20 client.29435 trim_cache size 0 max 16384 2024-05-10T11:43:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.738+0000 7f567297c740 1 Processor -- start 2024-05-10T11:43:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.738+0000 7f567297c740 1 -- start start 2024-05-10T11:43:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.738+0000 7f567297c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607b9432c30 0x5607b943f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.739+0000 7f567297c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607b94375c0 0x5607b9439230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.739+0000 7f567297c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607b9437fb0 0x5607b9439770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.739+0000 7f567297c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f56681abaa0 con 0x5607b9432c30 2024-05-10T11:43:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.739+0000 7f567297c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f56681a9370 con 0x5607b94375c0 2024-05-10T11:43:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.739+0000 7f567297c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f56681a71c0 con 0x5607b9437fb0 2024-05-10T11:43:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.739+0000 7f566d124640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607b94375c0 0x5607b9439230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:21.740 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.739+0000 7f566d124640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607b94375c0 0x5607b9439230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:34532/0 (socket says 172.21.15.100:34532) 2024-05-10T11:43:21.740 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.739+0000 7f566d124640 1 -- 172.21.15.100:0/2982588956 learned_addr learned my addr 172.21.15.100:0/2982588956 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:43:21.740 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.739+0000 7f566d925640 1 --2- 172.21.15.100:0/2982588956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607b9432c30 0x5607b943f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:21.740 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.740+0000 7f566d925640 1 -- 172.21.15.100:0/2982588956 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607b9437fb0 msgr2=0x5607b9439770 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T11:43:21.740 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.740+0000 7f566d925640 1 --2- 172.21.15.100:0/2982588956 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607b9437fb0 0x5607b9439770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:21.740 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.740+0000 7f566d925640 1 -- 172.21.15.100:0/2982588956 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607b94375c0 msgr2=0x5607b9439230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:43:21.740 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.740+0000 7f566d925640 1 --2- 172.21.15.100:0/2982588956 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607b94375c0 0x5607b9439230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:21.740 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.740+0000 7f566d925640 1 -- 172.21.15.100:0/2982588956 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607b943c500 con 0x5607b9432c30 2024-05-10T11:43:21.741 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.741+0000 7f566d925640 1 --2- 172.21.15.100:0/2982588956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607b9432c30 0x5607b943f510 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x5607b9433b00 tx=0x7f566001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49e4e8b5ff41b901 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:43:21.742 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.742+0000 7f56537fe640 1 -- 172.21.15.100:0/2982588956 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5660030070 con 0x5607b9432c30 2024-05-10T11:43:21.742 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.742+0000 7f56537fe640 1 -- 172.21.15.100:0/2982588956 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f566000fd90 con 0x5607b9432c30 2024-05-10T11:43:21.742 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.742+0000 7f56537fe640 1 -- 172.21.15.100:0/2982588956 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56600044a0 con 0x5607b9432c30 2024-05-10T11:43:21.745 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.745+0000 7f567297c740 1 -- 172.21.15.100:0/2982588956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607b9432c30 msgr2=0x5607b943f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:21.746 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.745+0000 7f567297c740 1 --2- 172.21.15.100:0/2982588956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607b9432c30 0x5607b943f510 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x5607b9433b00 tx=0x7f566001a000 comp rx=0 tx=0).stop 2024-05-10T11:43:21.746 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.745+0000 7f567297c740 1 -- 172.21.15.100:0/2982588956 shutdown_connections 2024-05-10T11:43:21.746 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.745+0000 7f567297c740 1 --2- 172.21.15.100:0/2982588956 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607b9437fb0 0x5607b9439770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:21.746 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.745+0000 7f567297c740 1 --2- 172.21.15.100:0/2982588956 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607b94375c0 0x5607b9439230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:21.746 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.745+0000 7f567297c740 1 --2- 172.21.15.100:0/2982588956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607b9432c30 0x5607b943f510 secure :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x5607b9433b00 tx=0x7f566001a000 comp rx=0 tx=0).stop 2024-05-10T11:43:21.746 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.745+0000 7f567297c740 1 -- 172.21.15.100:0/2982588956 >> 172.21.15.100:0/2982588956 conn(0x5607b943cbf0 msgr2=0x5607b94253e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:43:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.747+0000 7f567297c740 1 -- 172.21.15.100:0/2982588956 shutdown_connections 2024-05-10T11:43:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.749+0000 7f567297c740 1 -- 172.21.15.100:0/2982588956 wait complete. 2024-05-10T11:43:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.750+0000 7f567297c740 1 -- start start 2024-05-10T11:43:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.750+0000 7f567297c740 1 Processor -- start 2024-05-10T11:43:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.750+0000 7f567297c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607b94375c0 0x5607b9442fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.750+0000 7f567297c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607b9437fb0 0x5607b94484c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.750+0000 7f567297c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607b943d120 0x5607b9443540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.750+0000 7f567297c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f56681a7830 con 0x5607b9437fb0 2024-05-10T11:43:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.750+0000 7f567297c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f56681a8ca0 con 0x5607b94375c0 2024-05-10T11:43:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.750+0000 7f567297c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f56681a7f00 con 0x5607b943d120 2024-05-10T11:43:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.750+0000 7f567297c740 10 client.0 _ll_register_callbacks cb 0x7f56640c6350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:43:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.751+0000 7f566d124640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607b9437fb0 0x5607b94484c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.751+0000 7f566d124640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607b9437fb0 0x5607b94484c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35696/0 (socket says 172.21.15.100:35696) 2024-05-10T11:43:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.751+0000 7f566d124640 1 -- 172.21.15.100:0/1159847012 learned_addr learned my addr 172.21.15.100:0/1159847012 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:43:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.751+0000 7f566d925640 1 --2- 172.21.15.100:0/1159847012 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607b94375c0 0x5607b9442fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.752+0000 7f566d124640 1 -- 172.21.15.100:0/1159847012 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607b943d120 msgr2=0x5607b9443540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:43:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.752+0000 7f566d124640 1 --2- 172.21.15.100:0/1159847012 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607b943d120 0x5607b9443540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.752+0000 7f566d124640 1 -- 172.21.15.100:0/1159847012 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607b94375c0 msgr2=0x5607b9442fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:43:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.752+0000 7f566d124640 1 --2- 172.21.15.100:0/1159847012 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607b94375c0 0x5607b9442fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.752+0000 7f566d124640 1 -- 172.21.15.100:0/1159847012 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607b942ad30 con 0x5607b9437fb0 2024-05-10T11:43:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.752+0000 7f566d124640 1 --2- 172.21.15.100:0/1159847012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607b9437fb0 0x5607b94484c0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f56640c7690 tx=0x7f56640c9cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64fef1452729bcc9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:43:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.752+0000 7f56517fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:43:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.752+0000 7f56517fa640 1 -- 172.21.15.100:0/1159847012 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5664049070 con 0x5607b9437fb0 2024-05-10T11:43:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.753+0000 7f567297c740 1 -- 172.21.15.100:0/1159847012 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f566000fd90 con 0x5607b9437fb0 2024-05-10T11:43:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.753+0000 7f56517fa640 1 -- 172.21.15.100:0/1159847012 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f5664031710 con 0x5607b9437fb0 2024-05-10T11:43:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.753+0000 7f56517fa640 1 -- 172.21.15.100:0/1159847012 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f56640ca8f0 con 0x5607b9437fb0 2024-05-10T11:43:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.753+0000 7f567297c740 10 client.20187 fetch_fsmap learned FSMap version 41 2024-05-10T11:43:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.753+0000 7f567297c740 1 -- 172.21.15.100:0/1159847012 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5607b942c020 con 0x5607b9437fb0 2024-05-10T11:43:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.754+0000 7f56517fa640 1 -- 172.21.15.100:0/1159847012 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f566000fd90 con 0x5607b9437fb0 2024-05-10T11:43:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.754+0000 7f56517fa640 1 -- 172.21.15.100:0/1159847012 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f56640eb280 con 0x5607b9437fb0 2024-05-10T11:43:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.754+0000 7f567297c740 10 client.20187 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:43:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.754+0000 7f567297c740 10 client.20187 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:43:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.754+0000 7f567297c740 20 client.20187 populate_metadata read hostname 'smithi100' 2024-05-10T11:43:21.755 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.754+0000 7f567297c740 1 --2- 172.21.15.100:0/1159847012 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5607b944d6e0 0x5607b942e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:21.755 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.754+0000 7f567297c740 4 client.20187 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:43:21.755 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.754+0000 7f567297c740 1 -- 172.21.15.100:0/1159847012 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5607b939b050 con 0x5607b944d6e0 2024-05-10T11:43:21.755 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.754+0000 7f566d925640 1 --2- 172.21.15.100:0/1159847012 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5607b944d6e0 0x5607b942e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:21.755 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.755+0000 7f566d925640 1 --2- 172.21.15.100:0/1159847012 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5607b944d6e0 0x5607b942e760 crc :-1 s=READY pgs=464 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc0f64325cbc2fe7 server_cookie=225b4c17a654abd5 in_seq=0 out_seq=0 2024-05-10T11:43:21.755 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.755+0000 7f56517fa640 10 client.20187 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:43:21.862 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.861+0000 7f56517fa640 1 -- 172.21.15.100:0/1159847012 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5607b939b050 con 0x5607b944d6e0 2024-05-10T11:43:21.862 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.861+0000 7f56517fa640 10 client.20187 handle_command_reply: tid=0 2024-05-10T11:43:21.862 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.861+0000 7f567297c740 1 client.20187 shutdown 2024-05-10T11:43:21.862 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.861+0000 7f567297c740 1 -- 172.21.15.100:0/1159847012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607b9437fb0 msgr2=0x5607b94484c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:21.862 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.861+0000 7f567297c740 1 --2- 172.21.15.100:0/1159847012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607b9437fb0 0x5607b94484c0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f56640c7690 tx=0x7f56640c9cc0 comp rx=0 tx=0).stop 2024-05-10T11:43:21.862 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.861+0000 7f567297c740 1 -- 172.21.15.100:0/1159847012 shutdown_connections 2024-05-10T11:43:21.862 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.861+0000 7f567297c740 1 --2- 172.21.15.100:0/1159847012 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5607b944d6e0 0x5607b942e760 crc :-1 s=READY pgs=464 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:21.862 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.861+0000 7f567297c740 1 --2- 172.21.15.100:0/1159847012 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607b943d120 0x5607b9443540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:21.862 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.861+0000 7f567297c740 1 --2- 172.21.15.100:0/1159847012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607b9437fb0 0x5607b94484c0 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:21.862 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.861+0000 7f567297c740 1 --2- 172.21.15.100:0/1159847012 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607b94375c0 0x5607b9442fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:21.863 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.861+0000 7f56517fa640 0 client.20187 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:43:21.863 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.861+0000 7f567297c740 1 -- 172.21.15.100:0/1159847012 >> 172.21.15.100:0/1159847012 conn(0x5607b943cbf0 msgr2=0x5607b9441830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:43:21.863 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.861+0000 7f567297c740 1 -- 172.21.15.100:0/1159847012 shutdown_connections 2024-05-10T11:43:21.863 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.861+0000 7f567297c740 1 -- 172.21.15.100:0/1159847012 wait complete. 2024-05-10T11:43:21.863 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.861+0000 7f567297c740 20 client.20187 trim_cache size 0 max 16384 2024-05-10T11:43:21.863 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:43:21.863 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:43:21.863 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:43:21.863 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:43:21.864 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.862+0000 7f566fb89640 1 -- 172.21.15.100:0/790001866 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f56540b2920 msgr2=0x7f56540b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:21.864 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.862+0000 7f566fb89640 1 --2- 172.21.15.100:0/790001866 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f56540b2920 0x7f56540b4de0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x5607b94383b0 tx=0x7f5660009500 comp rx=0 tx=0).stop 2024-05-10T11:43:21.864 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.862+0000 7f566fb89640 1 -- 172.21.15.100:0/790001866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5668102b00 msgr2=0x7f566819b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:21.864 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.862+0000 7f566fb89640 1 --2- 172.21.15.100:0/790001866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5668102b00 0x7f566819b280 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f5664031460 tx=0x7f5664003c10 comp rx=0 tx=0).stop 2024-05-10T11:43:21.864 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.862+0000 7f566fb89640 1 -- 172.21.15.100:0/790001866 shutdown_connections 2024-05-10T11:43:21.864 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.862+0000 7f566fb89640 1 --2- 172.21.15.100:0/790001866 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f56540b2920 0x7f56540b4de0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:21.865 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.862+0000 7f566fb89640 1 --2- 172.21.15.100:0/790001866 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56681034c0 0x7f56681a22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:21.865 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.862+0000 7f566fb89640 1 --2- 172.21.15.100:0/790001866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5668102b00 0x7f566819b280 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:21.865 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.862+0000 7f566fb89640 1 --2- 172.21.15.100:0/790001866 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5668101910 0x7f566819ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:21.865 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.862+0000 7f566fb89640 1 -- 172.21.15.100:0/790001866 >> 172.21.15.100:0/790001866 conn(0x7f56680fce80 msgr2=0x7f566810b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:43:21.865 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.862+0000 7f566fb89640 1 -- 172.21.15.100:0/790001866 shutdown_connections 2024-05-10T11:43:21.865 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:21.862+0000 7f566fb89640 1 -- 172.21.15.100:0/790001866 wait complete. 2024-05-10T11:43:21.961 INFO:tasks.workunit.client.0.smithi100.stdout: 10 111575 8.87 MB/sec execute 555 sec latency 136.097 ms 2024-05-10T11:43:22.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:43:22 smithi100 ceph-mon[30296]: pgmap v1170: 129 pgs: 129 active+clean; 569 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 4.0 MiB/s wr, 283 op/s 2024-05-10T11:43:22.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:43:22 smithi178 ceph-mon[33359]: pgmap v1170: 129 pgs: 129 active+clean; 569 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 4.0 MiB/s wr, 283 op/s 2024-05-10T11:43:22.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:43:22 smithi188 ceph-mon[34075]: pgmap v1170: 129 pgs: 129 active+clean; 569 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 4.0 MiB/s wr, 283 op/s 2024-05-10T11:43:22.961 INFO:tasks.workunit.client.0.smithi100.stdout: 10 111770 8.88 MB/sec execute 556 sec latency 78.639 ms 2024-05-10T11:43:23.962 INFO:tasks.workunit.client.0.smithi100.stdout: 10 111928 8.87 MB/sec execute 557 sec latency 108.444 ms 2024-05-10T11:43:24.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.037+0000 7fa885bed700 1 -- 172.21.15.100:0/2124070283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa880108460 msgr2=0x7fa880108840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:24.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.037+0000 7fa885bed700 1 --2- 172.21.15.100:0/2124070283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa880108460 0x7fa880108840 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fa868009990 tx=0x7fa868009ca0 comp rx=0 tx=0).stop 2024-05-10T11:43:24.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.037+0000 7fa885bed700 1 -- 172.21.15.100:0/2124070283 shutdown_connections 2024-05-10T11:43:24.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.037+0000 7fa885bed700 1 --2- 172.21.15.100:0/2124070283 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa880103ef0 0x7fa8801103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:24.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.038+0000 7fa885bed700 1 --2- 172.21.15.100:0/2124070283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa880108460 0x7fa880108840 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:24.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.038+0000 7fa885bed700 1 --2- 172.21.15.100:0/2124070283 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa880103550 0x7fa8801039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:24.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.038+0000 7fa885bed700 1 -- 172.21.15.100:0/2124070283 >> 172.21.15.100:0/2124070283 conn(0x7fa88007a600 msgr2=0x7fa88007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:43:24.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.038+0000 7fa885bed700 1 -- 172.21.15.100:0/2124070283 shutdown_connections 2024-05-10T11:43:24.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.038+0000 7fa885bed700 1 -- 172.21.15.100:0/2124070283 wait complete. 2024-05-10T11:43:24.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.039+0000 7fa885bed700 1 Processor -- start 2024-05-10T11:43:24.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.039+0000 7fa885bed700 1 -- start start 2024-05-10T11:43:24.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.039+0000 7fa885bed700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa880103550 0x7fa8801a1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:24.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.039+0000 7fa885bed700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa880103ef0 0x7fa8801a1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:24.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.039+0000 7fa885bed700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa880108460 0x7fa88019b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:24.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.039+0000 7fa885bed700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa880115840 con 0x7fa880108460 2024-05-10T11:43:24.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.039+0000 7fa876ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa880103ef0 0x7fa8801a1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:24.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.039+0000 7fa87f7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa880103550 0x7fa8801a1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:24.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.039+0000 7fa87ffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa880108460 0x7fa88019b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:24.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.039+0000 7fa87f7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa880103550 0x7fa8801a1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:56920/0 (socket says 172.21.15.100:56920) 2024-05-10T11:43:24.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.039+0000 7fa876ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa880103ef0 0x7fa8801a1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:34568/0 (socket says 172.21.15.100:34568) 2024-05-10T11:43:24.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.039+0000 7fa87f7fe700 1 -- 172.21.15.100:0/29686398 learned_addr learned my addr 172.21.15.100:0/29686398 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:43:24.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.039+0000 7fa885bed700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa8801156c0 con 0x7fa880103ef0 2024-05-10T11:43:24.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.040+0000 7fa885bed700 1 -- 172.21.15.100:0/29686398 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa8801159c0 con 0x7fa880103550 2024-05-10T11:43:24.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.040+0000 7fa87f7fe700 1 -- 172.21.15.100:0/29686398 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa880103ef0 msgr2=0x7fa8801a1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:24.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.040+0000 7fa87f7fe700 1 --2- 172.21.15.100:0/29686398 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa880103ef0 0x7fa8801a1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:24.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.040+0000 7fa87f7fe700 1 -- 172.21.15.100:0/29686398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa880108460 msgr2=0x7fa88019b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:24.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.040+0000 7fa87f7fe700 1 --2- 172.21.15.100:0/29686398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa880108460 0x7fa88019b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:24.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.040+0000 7fa87f7fe700 1 -- 172.21.15.100:0/29686398 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa88019c140 con 0x7fa880103550 2024-05-10T11:43:24.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.040+0000 7fa87f7fe700 1 --2- 172.21.15.100:0/29686398 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa880103550 0x7fa8801a1820 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fa86800b2e0 tx=0x7fa86800f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:43:24.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.040+0000 7fa87d7fa700 1 -- 172.21.15.100:0/29686398 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa868026070 con 0x7fa880103550 2024-05-10T11:43:24.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.040+0000 7fa885bed700 1 -- 172.21.15.100:0/29686398 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa88019c3d0 con 0x7fa880103550 2024-05-10T11:43:24.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.040+0000 7fa87d7fa700 1 -- 172.21.15.100:0/29686398 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa86800be60 con 0x7fa880103550 2024-05-10T11:43:24.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.040+0000 7fa87d7fa700 1 -- 172.21.15.100:0/29686398 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa868017650 con 0x7fa880103550 2024-05-10T11:43:24.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.040+0000 7fa885bed700 1 -- 172.21.15.100:0/29686398 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8801a86f0 con 0x7fa880103550 2024-05-10T11:43:24.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.044+0000 7fa87d7fa700 1 -- 172.21.15.100:0/29686398 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fa868029070 con 0x7fa880103550 2024-05-10T11:43:24.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.045+0000 7fa87d7fa700 1 --2- 172.21.15.100:0/29686398 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa86c0b2900 0x7fa86c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:24.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.045+0000 7fa890480b80 1 Processor -- start 2024-05-10T11:43:24.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.045+0000 7fa87d7fa700 1 -- 172.21.15.100:0/29686398 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fa8680e3660 con 0x7fa880103550 2024-05-10T11:43:24.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.045+0000 7fa876ffd700 1 --2- 172.21.15.100:0/29686398 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa86c0b2900 0x7fa86c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:24.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.045+0000 7fa890480b80 1 -- start start 2024-05-10T11:43:24.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.046+0000 7fa890480b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e450cf5ce0 0x55e450cf60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:24.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.046+0000 7fa890480b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e450cf6600 0x55e450cf9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:24.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.046+0000 7fa890480b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e450cfa440 0x55e450cfa8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:24.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.046+0000 7fa890480b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55e450c3f860 con 0x55e450cf6600 2024-05-10T11:43:24.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.046+0000 7fa890480b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e450bde7c0 con 0x55e450cfa440 2024-05-10T11:43:24.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.046+0000 7fa890480b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55e450c3b820 con 0x55e450cf5ce0 2024-05-10T11:43:24.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.046+0000 7fa87f7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e450cf5ce0 0x55e450cf60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:24.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.046+0000 7fa87f7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e450cf5ce0 0x55e450cf60c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:56932/0 (socket says 172.21.15.100:56932) 2024-05-10T11:43:24.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.046+0000 7fa87f7fe700 1 -- 172.21.15.100:0/1438994666 learned_addr learned my addr 172.21.15.100:0/1438994666 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:43:24.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.046+0000 7fa876ffd700 1 --2- 172.21.15.100:0/1438994666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e450cf6600 0x55e450cf9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:24.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.046+0000 7fa876ffd700 1 --2- 172.21.15.100:0/29686398 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa86c0b2900 0x7fa86c0b4dc0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fa88007ab90 tx=0x7fa870011720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:43:24.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.046+0000 7fa87f7fe700 1 -- 172.21.15.100:0/1438994666 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e450cfa440 msgr2=0x55e450cfa8a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T11:43:24.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.046+0000 7fa87ffff700 1 --2- 172.21.15.100:0/1438994666 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e450cfa440 0x55e450cfa8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:24.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.046+0000 7fa87f7fe700 1 --2- 172.21.15.100:0/1438994666 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e450cfa440 0x55e450cfa8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:24.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.046+0000 7fa87f7fe700 1 -- 172.21.15.100:0/1438994666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e450cf6600 msgr2=0x55e450cf9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:43:24.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.046+0000 7fa87f7fe700 1 --2- 172.21.15.100:0/1438994666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e450cf6600 0x55e450cf9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:24.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.046+0000 7fa87f7fe700 1 -- 172.21.15.100:0/1438994666 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e450b8c8d0 con 0x55e450cf5ce0 2024-05-10T11:43:24.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.046+0000 7fa876ffd700 1 --2- 172.21.15.100:0/1438994666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e450cf6600 0x55e450cf9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:43:24.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.046+0000 7fa87f7fe700 1 --2- 172.21.15.100:0/1438994666 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e450cf5ce0 0x55e450cf60c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa8680a7c20 tx=0x7fa868009290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c1a5c9edd04e3e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:43:24.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.047+0000 7fa875ffb700 1 -- 172.21.15.100:0/1438994666 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8680aa4c0 con 0x55e450cf5ce0 2024-05-10T11:43:24.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.047+0000 7fa875ffb700 1 -- 172.21.15.100:0/1438994666 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa8680ab0b0 con 0x55e450cf5ce0 2024-05-10T11:43:24.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.047+0000 7fa875ffb700 1 -- 172.21.15.100:0/1438994666 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8680cd0e0 con 0x55e450cf5ce0 2024-05-10T11:43:24.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.048+0000 7fa890480b80 1 -- 172.21.15.100:0/1438994666 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e450cf5ce0 msgr2=0x55e450cf60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:24.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.048+0000 7fa890480b80 1 --2- 172.21.15.100:0/1438994666 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e450cf5ce0 0x55e450cf60c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa8680a7c20 tx=0x7fa868009290 comp rx=0 tx=0).stop 2024-05-10T11:43:24.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.048+0000 7fa890480b80 1 -- 172.21.15.100:0/1438994666 shutdown_connections 2024-05-10T11:43:24.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.048+0000 7fa890480b80 1 --2- 172.21.15.100:0/1438994666 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e450cfa440 0x55e450cfa8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:24.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.048+0000 7fa890480b80 1 --2- 172.21.15.100:0/1438994666 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e450cf5ce0 0x55e450cf60c0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:24.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.048+0000 7fa890480b80 1 --2- 172.21.15.100:0/1438994666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e450cf6600 0x55e450cf9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:24.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.048+0000 7fa890480b80 1 -- 172.21.15.100:0/1438994666 >> 172.21.15.100:0/1438994666 conn(0x55e450c19100 msgr2=0x55e450ce7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:43:24.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.048+0000 7fa890480b80 1 -- 172.21.15.100:0/1438994666 shutdown_connections 2024-05-10T11:43:24.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.048+0000 7fa890480b80 1 -- 172.21.15.100:0/1438994666 wait complete. 2024-05-10T11:43:24.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.048+0000 7fa890480b80 1 -- start start 2024-05-10T11:43:24.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.049+0000 7fa890480b80 1 Processor -- start 2024-05-10T11:43:24.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.049+0000 7fa890480b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e450cf5ce0 0x55e450d01900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:24.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.050+0000 7fa890480b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e450cf6600 0x55e450cfeca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:24.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.050+0000 7fa890480b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e450cfa440 0x55e450cff1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:24.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.050+0000 7fa890480b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55e450d02410 con 0x55e450cf5ce0 2024-05-10T11:43:24.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.050+0000 7fa890480b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e450ba5c30 con 0x55e450cfa440 2024-05-10T11:43:24.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.050+0000 7fa890480b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55e450d02590 con 0x55e450cf6600 2024-05-10T11:43:24.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.050+0000 7fa890480b80 10 client.0 _ll_register_callbacks cb 0x55e450bd7920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:43:24.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.050+0000 7fa87f7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e450cf5ce0 0x55e450d01900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:24.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.050+0000 7fa87f7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e450cf5ce0 0x55e450d01900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35740/0 (socket says 172.21.15.100:35740) 2024-05-10T11:43:24.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.050+0000 7fa87f7fe700 1 -- 172.21.15.100:0/4140790172 learned_addr learned my addr 172.21.15.100:0/4140790172 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:43:24.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.050+0000 7fa876ffd700 1 --2- 172.21.15.100:0/4140790172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e450cf6600 0x55e450cfeca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:24.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.051+0000 7fa87f7fe700 1 -- 172.21.15.100:0/4140790172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e450cf6600 msgr2=0x55e450cfeca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:43:24.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.051+0000 7fa87f7fe700 1 --2- 172.21.15.100:0/4140790172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e450cf6600 0x55e450cfeca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:24.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.051+0000 7fa87f7fe700 1 -- 172.21.15.100:0/4140790172 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e450cfa440 msgr2=0x55e450cff1e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T11:43:24.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.051+0000 7fa87f7fe700 1 --2- 172.21.15.100:0/4140790172 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e450cfa440 0x55e450cff1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:24.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.051+0000 7fa87f7fe700 1 -- 172.21.15.100:0/4140790172 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e450bdb690 con 0x55e450cf5ce0 2024-05-10T11:43:24.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.051+0000 7fa87f7fe700 1 --2- 172.21.15.100:0/4140790172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e450cf5ce0 0x55e450d01900 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fa8680a69e0 tx=0x7fa868005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d356c8cfffdbe565 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:43:24.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.051+0000 7fa85effd700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:43:24.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.051+0000 7fa85effd700 1 -- 172.21.15.100:0/4140790172 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8680b3780 con 0x55e450cf5ce0 2024-05-10T11:43:24.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.051+0000 7fa85effd700 1 -- 172.21.15.100:0/4140790172 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fa8680b3050 con 0x55e450cf5ce0 2024-05-10T11:43:24.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.051+0000 7fa890480b80 1 -- 172.21.15.100:0/4140790172 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8680ab0b0 con 0x55e450cf5ce0 2024-05-10T11:43:24.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.052+0000 7fa85effd700 1 -- 172.21.15.100:0/4140790172 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fa8680aa9b0 con 0x55e450cf5ce0 2024-05-10T11:43:24.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.052+0000 7fa890480b80 10 client.20199 fetch_fsmap learned FSMap version 41 2024-05-10T11:43:24.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.052+0000 7fa890480b80 1 -- 172.21.15.100:0/4140790172 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55e450d02410 con 0x55e450cf5ce0 2024-05-10T11:43:24.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.052+0000 7fa85effd700 1 -- 172.21.15.100:0/4140790172 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa8680ab0b0 con 0x55e450cf5ce0 2024-05-10T11:43:24.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.052+0000 7fa85effd700 1 -- 172.21.15.100:0/4140790172 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fa8680abce0 con 0x55e450cf5ce0 2024-05-10T11:43:24.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.052+0000 7fa890480b80 10 client.20199 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:43:24.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.052+0000 7fa890480b80 10 client.20199 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:43:24.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.052+0000 7fa890480b80 20 client.20199 populate_metadata read hostname 'smithi100' 2024-05-10T11:43:24.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.052+0000 7fa890480b80 1 --2- 172.21.15.100:0/4140790172 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55e450d0eeb0 0x55e450d112a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:24.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.052+0000 7fa890480b80 4 client.20199 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:43:24.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.052+0000 7fa890480b80 1 -- 172.21.15.100:0/4140790172 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e450cb9650 con 0x55e450d0eeb0 2024-05-10T11:43:24.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.053+0000 7fa876ffd700 1 --2- 172.21.15.100:0/4140790172 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55e450d0eeb0 0x55e450d112a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:24.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.053+0000 7fa85effd700 10 client.20199 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:43:24.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.053+0000 7fa876ffd700 1 --2- 172.21.15.100:0/4140790172 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55e450d0eeb0 0x55e450d112a0 crc :-1 s=READY pgs=465 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eda99e50ff0efc55 server_cookie=e5fd3cbb914b3ebf in_seq=0 out_seq=0 2024-05-10T11:43:24.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.056+0000 7fa85effd700 1 -- 172.21.15.100:0/4140790172 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e450cb9650 con 0x55e450d0eeb0 2024-05-10T11:43:24.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.056+0000 7fa85effd700 10 client.20199 handle_command_reply: tid=0 2024-05-10T11:43:24.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.056+0000 7fa890480b80 1 client.20199 shutdown 2024-05-10T11:43:24.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.056+0000 7fa890480b80 1 -- 172.21.15.100:0/4140790172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e450cf5ce0 msgr2=0x55e450d01900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:24.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.056+0000 7fa890480b80 1 --2- 172.21.15.100:0/4140790172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e450cf5ce0 0x55e450d01900 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fa8680a69e0 tx=0x7fa868005c00 comp rx=0 tx=0).stop 2024-05-10T11:43:24.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.056+0000 7fa890480b80 1 -- 172.21.15.100:0/4140790172 shutdown_connections 2024-05-10T11:43:24.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.056+0000 7fa890480b80 1 --2- 172.21.15.100:0/4140790172 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55e450d0eeb0 0x55e450d112a0 crc :-1 s=READY pgs=465 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:24.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.056+0000 7fa890480b80 1 --2- 172.21.15.100:0/4140790172 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e450cfa440 0x55e450cff1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:24.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.056+0000 7fa890480b80 1 --2- 172.21.15.100:0/4140790172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e450cf5ce0 0x55e450d01900 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:24.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.056+0000 7fa890480b80 1 --2- 172.21.15.100:0/4140790172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e450cf6600 0x55e450cfeca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:24.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.056+0000 7fa85effd700 0 client.20199 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:43:24.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.056+0000 7fa890480b80 1 -- 172.21.15.100:0/4140790172 >> 172.21.15.100:0/4140790172 conn(0x55e450c19100 msgr2=0x55e450ce2720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:43:24.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.056+0000 7fa890480b80 1 -- 172.21.15.100:0/4140790172 shutdown_connections 2024-05-10T11:43:24.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.056+0000 7fa890480b80 1 -- 172.21.15.100:0/4140790172 wait complete. 2024-05-10T11:43:24.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.057+0000 7fa890480b80 20 client.20199 trim_cache size 0 max 16384 2024-05-10T11:43:24.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.087+0000 7fa890480b80 1 Processor -- start 2024-05-10T11:43:24.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.087+0000 7fa890480b80 1 -- start start 2024-05-10T11:43:24.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.087+0000 7fa890480b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e450cf5ce0 0x55e450d0b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:24.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.087+0000 7fa890480b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e450cf6600 0x55e450cfb820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:24.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.087+0000 7fa890480b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e450cfa440 0x55e450cfdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:24.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.087+0000 7fa87f7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e450cf5ce0 0x55e450d0b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:24.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.087+0000 7fa890480b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa880073c10 con 0x55e450cf5ce0 2024-05-10T11:43:24.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.087+0000 7fa87f7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e450cf5ce0 0x55e450d0b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35752/0 (socket says 172.21.15.100:35752) 2024-05-10T11:43:24.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.087+0000 7fa87f7fe700 1 -- 172.21.15.100:0/2236817494 learned_addr learned my addr 172.21.15.100:0/2236817494 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:43:24.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.088+0000 7fa890480b80 1 -- 172.21.15.100:0/2236817494 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa8800714a0 con 0x55e450cfa440 2024-05-10T11:43:24.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.088+0000 7fa890480b80 1 -- 172.21.15.100:0/2236817494 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa88006f2f0 con 0x55e450cf6600 2024-05-10T11:43:24.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.088+0000 7fa87ffff700 1 --2- 172.21.15.100:0/2236817494 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e450cfa440 0x55e450cfdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:24.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.088+0000 7fa876ffd700 1 --2- 172.21.15.100:0/2236817494 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e450cf6600 0x55e450cfb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:24.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.088+0000 7fa87ffff700 1 -- 172.21.15.100:0/2236817494 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e450cf6600 msgr2=0x55e450cfb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:43:24.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.088+0000 7fa87ffff700 1 --2- 172.21.15.100:0/2236817494 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e450cf6600 0x55e450cfb820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:24.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.088+0000 7fa87ffff700 1 -- 172.21.15.100:0/2236817494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e450cf5ce0 msgr2=0x55e450d0b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:43:24.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.088+0000 7fa87ffff700 1 --2- 172.21.15.100:0/2236817494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e450cf5ce0 0x55e450d0b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:24.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.088+0000 7fa876ffd700 1 --2- 172.21.15.100:0/2236817494 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e450cf6600 0x55e450cfb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:43:24.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.088+0000 7fa87ffff700 1 -- 172.21.15.100:0/2236817494 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e450ce4810 con 0x55e450cfa440 2024-05-10T11:43:24.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.088+0000 7fa87f7fe700 1 --2- 172.21.15.100:0/2236817494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e450cf5ce0 0x55e450d0b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:43:24.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.088+0000 7fa87ffff700 1 --2- 172.21.15.100:0/2236817494 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e450cfa440 0x55e450cfdcc0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fa878009e40 tx=0x7fa87800a6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=613fd9e348f5d09d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:43:24.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.089+0000 7fa8757fa700 1 -- 172.21.15.100:0/2236817494 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87801a070 con 0x55e450cfa440 2024-05-10T11:43:24.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.089+0000 7fa8757fa700 1 -- 172.21.15.100:0/2236817494 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa878015880 con 0x55e450cfa440 2024-05-10T11:43:24.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.089+0000 7fa8757fa700 1 -- 172.21.15.100:0/2236817494 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8780147b0 con 0x55e450cfa440 2024-05-10T11:43:24.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.090+0000 7fa890480b80 1 -- 172.21.15.100:0/2236817494 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e450cfa440 msgr2=0x55e450cfdcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:24.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.090+0000 7fa890480b80 1 --2- 172.21.15.100:0/2236817494 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e450cfa440 0x55e450cfdcc0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fa878009e40 tx=0x7fa87800a6e0 comp rx=0 tx=0).stop 2024-05-10T11:43:24.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.090+0000 7fa890480b80 1 -- 172.21.15.100:0/2236817494 shutdown_connections 2024-05-10T11:43:24.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.090+0000 7fa890480b80 1 --2- 172.21.15.100:0/2236817494 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e450cfa440 0x55e450cfdcc0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:24.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.090+0000 7fa890480b80 1 --2- 172.21.15.100:0/2236817494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e450cf5ce0 0x55e450d0b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:24.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.090+0000 7fa890480b80 1 --2- 172.21.15.100:0/2236817494 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e450cf6600 0x55e450cfb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:24.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.090+0000 7fa890480b80 1 -- 172.21.15.100:0/2236817494 >> 172.21.15.100:0/2236817494 conn(0x55e450c19100 msgr2=0x55e450cfd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:43:24.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.090+0000 7fa890480b80 1 -- 172.21.15.100:0/2236817494 shutdown_connections 2024-05-10T11:43:24.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.090+0000 7fa890480b80 1 -- 172.21.15.100:0/2236817494 wait complete. 2024-05-10T11:43:24.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.090+0000 7fa890480b80 1 -- start start 2024-05-10T11:43:24.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.091+0000 7fa890480b80 1 Processor -- start 2024-05-10T11:43:24.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.091+0000 7fa890480b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e450cf5ce0 0x55e450d07600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:24.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.091+0000 7fa890480b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e450cf6600 0x55e450d07b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:24.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.091+0000 7fa890480b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e450cfa440 0x55e450d045d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:24.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.091+0000 7fa87f7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e450cf5ce0 0x55e450d07600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:24.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.091+0000 7fa87ffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e450cfa440 0x55e450d045d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:24.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.091+0000 7fa87ffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e450cfa440 0x55e450d045d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35766/0 (socket says 172.21.15.100:35766) 2024-05-10T11:43:24.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.091+0000 7fa87ffff700 1 -- 172.21.15.100:0/1172352189 learned_addr learned my addr 172.21.15.100:0/1172352189 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:43:24.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.091+0000 7fa890480b80 1 -- 172.21.15.100:0/1172352189 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa88006f960 con 0x55e450cfa440 2024-05-10T11:43:24.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.091+0000 7fa890480b80 1 -- 172.21.15.100:0/1172352189 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa880070dd0 con 0x55e450cf5ce0 2024-05-10T11:43:24.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.091+0000 7fa890480b80 1 -- 172.21.15.100:0/1172352189 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa880070030 con 0x55e450cf6600 2024-05-10T11:43:24.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.091+0000 7fa890480b80 10 client.0 _ll_register_callbacks cb 0x55e450bd7920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:43:24.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.091+0000 7fa876ffd700 1 --2- 172.21.15.100:0/1172352189 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e450cf6600 0x55e450d07b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:24.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.092+0000 7fa87f7fe700 1 -- 172.21.15.100:0/1172352189 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e450cf6600 msgr2=0x55e450d07b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:43:24.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.092+0000 7fa87f7fe700 1 --2- 172.21.15.100:0/1172352189 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e450cf6600 0x55e450d07b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:24.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.092+0000 7fa87f7fe700 1 -- 172.21.15.100:0/1172352189 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e450cfa440 msgr2=0x55e450d045d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:43:24.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.092+0000 7fa87f7fe700 1 --2- 172.21.15.100:0/1172352189 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e450cfa440 0x55e450d045d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:24.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.092+0000 7fa87f7fe700 1 -- 172.21.15.100:0/1172352189 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e450d04b10 con 0x55e450cf5ce0 2024-05-10T11:43:24.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.092+0000 7fa87f7fe700 1 --2- 172.21.15.100:0/1172352189 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e450cf5ce0 0x55e450d07600 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa8680a7c20 tx=0x7fa868009290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e547e7741247847 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:43:24.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.092+0000 7fa85effd700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:43:24.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.092+0000 7fa85effd700 1 -- 172.21.15.100:0/1172352189 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa868088070 con 0x55e450cf5ce0 2024-05-10T11:43:24.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.092+0000 7fa85effd700 1 -- 172.21.15.100:0/1172352189 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fa8680a9e20 con 0x55e450cf5ce0 2024-05-10T11:43:24.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.092+0000 7fa890480b80 1 -- 172.21.15.100:0/1172352189 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa878015880 con 0x55e450cf5ce0 2024-05-10T11:43:24.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.093+0000 7fa85effd700 1 -- 172.21.15.100:0/1172352189 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fa8680b30c0 con 0x55e450cf5ce0 2024-05-10T11:43:24.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.093+0000 7fa890480b80 10 client.29450 fetch_fsmap learned FSMap version 41 2024-05-10T11:43:24.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.093+0000 7fa890480b80 1 -- 172.21.15.100:0/1172352189 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55e450d04b10 con 0x55e450cf5ce0 2024-05-10T11:43:24.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.093+0000 7fa85effd700 1 -- 172.21.15.100:0/1172352189 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa878015880 con 0x55e450cf5ce0 2024-05-10T11:43:24.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.093+0000 7fa85effd700 1 -- 172.21.15.100:0/1172352189 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fa86808f050 con 0x55e450cf5ce0 2024-05-10T11:43:24.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.093+0000 7fa890480b80 10 client.29450 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:43:24.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.093+0000 7fa890480b80 10 client.29450 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:43:24.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.093+0000 7fa890480b80 20 client.29450 populate_metadata read hostname 'smithi100' 2024-05-10T11:43:24.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.093+0000 7fa890480b80 1 --2- 172.21.15.100:0/1172352189 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55e450d7a080 0x55e450d7c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:24.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.093+0000 7fa890480b80 4 client.29450 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T11:43:24.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.093+0000 7fa890480b80 1 -- 172.21.15.100:0/1172352189 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e450c4d240 con 0x55e450d7a080 2024-05-10T11:43:24.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.093+0000 7fa876ffd700 1 --2- 172.21.15.100:0/1172352189 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55e450d7a080 0x55e450d7c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:24.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.094+0000 7fa85effd700 10 client.29450 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:43:24.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.094+0000 7fa876ffd700 1 --2- 172.21.15.100:0/1172352189 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55e450d7a080 0x55e450d7c470 crc :-1 s=READY pgs=466 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97cd426fea56eba1 server_cookie=94391fc593c7a32c in_seq=0 out_seq=0 2024-05-10T11:43:24.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.094+0000 7fa85effd700 1 -- 172.21.15.100:0/1172352189 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e450c4d240 con 0x55e450d7a080 2024-05-10T11:43:24.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.094+0000 7fa85effd700 10 client.29450 handle_command_reply: tid=0 2024-05-10T11:43:24.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.094+0000 7fa890480b80 1 client.29450 shutdown 2024-05-10T11:43:24.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.094+0000 7fa890480b80 1 -- 172.21.15.100:0/1172352189 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e450cf5ce0 msgr2=0x55e450d07600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:24.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.094+0000 7fa890480b80 1 --2- 172.21.15.100:0/1172352189 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e450cf5ce0 0x55e450d07600 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa8680a7c20 tx=0x7fa868009290 comp rx=0 tx=0).stop 2024-05-10T11:43:24.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.094+0000 7fa890480b80 1 -- 172.21.15.100:0/1172352189 shutdown_connections 2024-05-10T11:43:24.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.094+0000 7fa890480b80 1 --2- 172.21.15.100:0/1172352189 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55e450d7a080 0x55e450d7c470 crc :-1 s=READY pgs=466 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:24.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.094+0000 7fa890480b80 1 --2- 172.21.15.100:0/1172352189 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e450cfa440 0x55e450d045d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:24.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.094+0000 7fa890480b80 1 --2- 172.21.15.100:0/1172352189 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e450cf5ce0 0x55e450d07600 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:24.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.094+0000 7fa85effd700 0 client.29450 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:43:24.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.094+0000 7fa890480b80 1 --2- 172.21.15.100:0/1172352189 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e450cf6600 0x55e450d07b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:24.231 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:43:24.231 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T11:43:24.231 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-05-10T11:43:24.232 INFO:teuthology.orchestra.run.smithi100.stdout: "8f368896-e12a-4ef6-a300-4d80f2c0c4bc": { 2024-05-10T11:43:24.232 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-05-10T11:43:24.232 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "8f368896-e12a-4ef6-a300-4d80f2c0c4bc", 2024-05-10T11:43:24.232 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-05-10T11:43:24.232 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:43:24.232 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:43:24.232 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:43:24.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.094+0000 7fa890480b80 1 -- 172.21.15.100:0/1172352189 >> 172.21.15.100:0/1172352189 conn(0x55e450c19100 msgr2=0x55e450cfb510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:43:24.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.094+0000 7fa890480b80 1 -- 172.21.15.100:0/1172352189 shutdown_connections 2024-05-10T11:43:24.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.095+0000 7fa890480b80 1 -- 172.21.15.100:0/1172352189 wait complete. 2024-05-10T11:43:24.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.095+0000 7fa890480b80 20 client.29450 trim_cache size 0 max 16384 2024-05-10T11:43:24.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.095+0000 7fa885bed700 1 -- 172.21.15.100:0/29686398 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa86c0b2900 msgr2=0x7fa86c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:24.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.095+0000 7fa885bed700 1 --2- 172.21.15.100:0/29686398 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa86c0b2900 0x7fa86c0b4dc0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fa88007ab90 tx=0x7fa870011720 comp rx=0 tx=0).stop 2024-05-10T11:43:24.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.095+0000 7fa885bed700 1 -- 172.21.15.100:0/29686398 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa880103550 msgr2=0x7fa8801a1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:24.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.095+0000 7fa885bed700 1 --2- 172.21.15.100:0/29686398 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa880103550 0x7fa8801a1820 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fa86800b2e0 tx=0x7fa86800f690 comp rx=0 tx=0).stop 2024-05-10T11:43:24.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.096+0000 7fa885bed700 1 -- 172.21.15.100:0/29686398 shutdown_connections 2024-05-10T11:43:24.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.096+0000 7fa885bed700 1 --2- 172.21.15.100:0/29686398 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa86c0b2900 0x7fa86c0b4dc0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:24.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.096+0000 7fa885bed700 1 --2- 172.21.15.100:0/29686398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa880108460 0x7fa88019b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:24.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.096+0000 7fa885bed700 1 --2- 172.21.15.100:0/29686398 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa880103550 0x7fa8801a1820 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:24.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.096+0000 7fa885bed700 1 --2- 172.21.15.100:0/29686398 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa880103ef0 0x7fa8801a1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:24.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.096+0000 7fa885bed700 1 -- 172.21.15.100:0/29686398 >> 172.21.15.100:0/29686398 conn(0x7fa88007a600 msgr2=0x7fa880101520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:43:24.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.096+0000 7fa885bed700 1 -- 172.21.15.100:0/29686398 shutdown_connections 2024-05-10T11:43:24.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:24.096+0000 7fa885bed700 1 -- 172.21.15.100:0/29686398 wait complete. 2024-05-10T11:43:24.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:43:24 smithi178 ceph-mon[33359]: pgmap v1171: 129 pgs: 129 active+clean; 571 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 3.7 MiB/s wr, 278 op/s 2024-05-10T11:43:24.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:43:24 smithi188 ceph-mon[34075]: pgmap v1171: 129 pgs: 129 active+clean; 571 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 3.7 MiB/s wr, 278 op/s 2024-05-10T11:43:24.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:43:24 smithi100 ceph-mon[30296]: pgmap v1171: 129 pgs: 129 active+clean; 571 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 3.7 MiB/s wr, 278 op/s 2024-05-10T11:43:24.961 INFO:tasks.workunit.client.0.smithi100.stdout: 10 112079 8.87 MB/sec execute 558 sec latency 147.540 ms 2024-05-10T11:43:25.146 INFO:tasks.cephfs.filesystem:scrub status for tag:8f368896-e12a-4ef6-a300-4d80f2c0c4bc - {'path': '/', 'tag': '8f368896-e12a-4ef6-a300-4d80f2c0c4bc', 'options': 'recursive,force'} 2024-05-10T11:43:25.961 INFO:tasks.workunit.client.0.smithi100.stdout: 10 112238 8.86 MB/sec execute 559 sec latency 69.049 ms 2024-05-10T11:43:26.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:43:26 smithi178 ceph-mon[33359]: pgmap v1172: 129 pgs: 129 active+clean; 573 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 4.1 MiB/s wr, 288 op/s 2024-05-10T11:43:26.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:43:26 smithi188 ceph-mon[34075]: pgmap v1172: 129 pgs: 129 active+clean; 573 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 4.1 MiB/s wr, 288 op/s 2024-05-10T11:43:26.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:43:26 smithi100 ceph-mon[30296]: pgmap v1172: 129 pgs: 129 active+clean; 573 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 4.1 MiB/s wr, 288 op/s 2024-05-10T11:43:26.961 INFO:tasks.workunit.client.0.smithi100.stdout: 10 112425 8.88 MB/sec execute 560 sec latency 120.362 ms 2024-05-10T11:43:27.961 INFO:tasks.workunit.client.0.smithi100.stdout: 10 112600 8.88 MB/sec execute 561 sec latency 99.751 ms 2024-05-10T11:43:28.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:43:28 smithi178 ceph-mon[33359]: pgmap v1173: 129 pgs: 129 active+clean; 571 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1000 KiB/s rd, 4.7 MiB/s wr, 347 op/s 2024-05-10T11:43:28.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:43:28 smithi188 ceph-mon[34075]: pgmap v1173: 129 pgs: 129 active+clean; 571 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1000 KiB/s rd, 4.7 MiB/s wr, 347 op/s 2024-05-10T11:43:28.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:43:28 smithi100 ceph-mon[30296]: pgmap v1173: 129 pgs: 129 active+clean; 571 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1000 KiB/s rd, 4.7 MiB/s wr, 347 op/s 2024-05-10T11:43:28.961 INFO:tasks.workunit.client.0.smithi100.stdout: 10 112764 8.87 MB/sec execute 562 sec latency 94.241 ms 2024-05-10T11:43:29.962 INFO:tasks.workunit.client.0.smithi100.stdout: 10 112945 8.86 MB/sec execute 563 sec latency 67.414 ms 2024-05-10T11:43:30.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:43:30 smithi178 ceph-mon[33359]: pgmap v1174: 129 pgs: 129 active+clean; 569 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 4.3 MiB/s wr, 314 op/s 2024-05-10T11:43:30.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:43:30 smithi188 ceph-mon[34075]: pgmap v1174: 129 pgs: 129 active+clean; 569 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 4.3 MiB/s wr, 314 op/s 2024-05-10T11:43:30.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:43:30 smithi100 ceph-mon[30296]: pgmap v1174: 129 pgs: 129 active+clean; 569 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 4.3 MiB/s wr, 314 op/s 2024-05-10T11:43:30.961 INFO:tasks.workunit.client.0.smithi100.stdout: 10 113131 8.86 MB/sec execute 564 sec latency 99.876 ms 2024-05-10T11:43:31.961 INFO:tasks.workunit.client.0.smithi100.stdout: 10 113301 8.86 MB/sec execute 565 sec latency 109.818 ms 2024-05-10T11:43:32.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:43:32 smithi178 ceph-mon[33359]: pgmap v1175: 129 pgs: 129 active+clean; 566 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 4.2 MiB/s wr, 328 op/s 2024-05-10T11:43:32.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:43:32 smithi188 ceph-mon[34075]: pgmap v1175: 129 pgs: 129 active+clean; 566 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 4.2 MiB/s wr, 328 op/s 2024-05-10T11:43:32.539 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:43:32 smithi100 ceph-mon[30296]: pgmap v1175: 129 pgs: 129 active+clean; 566 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 4.2 MiB/s wr, 328 op/s 2024-05-10T11:43:32.961 INFO:tasks.workunit.client.0.smithi100.stdout: 10 113462 8.86 MB/sec execute 566 sec latency 93.555 ms 2024-05-10T11:43:33.961 INFO:tasks.workunit.client.0.smithi100.stdout: 10 113695 8.88 MB/sec execute 567 sec latency 87.489 ms 2024-05-10T11:43:34.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:43:34 smithi178 ceph-mon[33359]: pgmap v1176: 129 pgs: 129 active+clean; 567 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 3.9 MiB/s wr, 317 op/s 2024-05-10T11:43:34.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:43:34 smithi188 ceph-mon[34075]: pgmap v1176: 129 pgs: 129 active+clean; 567 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 3.9 MiB/s wr, 317 op/s 2024-05-10T11:43:34.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:43:34 smithi100 ceph-mon[30296]: pgmap v1176: 129 pgs: 129 active+clean; 567 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 3.9 MiB/s wr, 317 op/s 2024-05-10T11:43:34.961 INFO:tasks.workunit.client.0.smithi100.stdout: 10 113877 8.88 MB/sec execute 568 sec latency 100.356 ms 2024-05-10T11:43:34.993 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:34.991+0000 7fd167cfc640 1 -- 172.21.15.100:0/1672635032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd160101910 msgr2=0x7fd160101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:34.994 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:34.991+0000 7fd167cfc640 1 --2- 172.21.15.100:0/1672635032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd160101910 0x7fd160101d30 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fd15c009900 tx=0x7fd15c02a410 comp rx=0 tx=0).stop 2024-05-10T11:43:34.994 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:34.991+0000 7fd167cfc640 1 -- 172.21.15.100:0/1672635032 shutdown_connections 2024-05-10T11:43:34.994 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:34.991+0000 7fd167cfc640 1 --2- 172.21.15.100:0/1672635032 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1601034c0 0x7fd160109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:34.994 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:34.991+0000 7fd167cfc640 1 --2- 172.21.15.100:0/1672635032 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd160102b00 0x7fd160102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:34.994 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:34.991+0000 7fd167cfc640 1 --2- 172.21.15.100:0/1672635032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd160101910 0x7fd160101d30 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:34.994 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:34.991+0000 7fd167cfc640 1 -- 172.21.15.100:0/1672635032 >> 172.21.15.100:0/1672635032 conn(0x7fd1600fce80 msgr2=0x7fd1600ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:43:34.994 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:34.991+0000 7fd167cfc640 1 -- 172.21.15.100:0/1672635032 shutdown_connections 2024-05-10T11:43:34.994 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:34.991+0000 7fd167cfc640 1 -- 172.21.15.100:0/1672635032 wait complete. 2024-05-10T11:43:34.994 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:34.992+0000 7fd167cfc640 1 Processor -- start 2024-05-10T11:43:34.994 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:34.992+0000 7fd167cfc640 1 -- start start 2024-05-10T11:43:34.994 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:34.992+0000 7fd167cfc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd160101910 0x7fd160196900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:34.994 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:34.992+0000 7fd167cfc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd160102b00 0x7fd160196e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:34.995 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:34.992+0000 7fd167cfc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1601034c0 0x7fd16019deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:34.995 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:34.992+0000 7fd167cfc640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd16010c1f0 con 0x7fd160102b00 2024-05-10T11:43:34.995 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:34.992+0000 7fd167cfc640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd16010c070 con 0x7fd1601034c0 2024-05-10T11:43:34.995 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:34.992+0000 7fd167cfc640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd16010c370 con 0x7fd160101910 2024-05-10T11:43:34.995 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:34.992+0000 7fd165a98640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd160101910 0x7fd160196900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:34.995 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:34.992+0000 7fd165297640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd160102b00 0x7fd160196e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:34.995 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:34.992+0000 7fd165297640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd160102b00 0x7fd160196e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46980/0 (socket says 172.21.15.100:46980) 2024-05-10T11:43:34.995 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:34.992+0000 7fd165297640 1 -- 172.21.15.100:0/442267185 learned_addr learned my addr 172.21.15.100:0/442267185 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:43:34.995 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:34.992+0000 7fd166299640 1 --2- 172.21.15.100:0/442267185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1601034c0 0x7fd16019deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:34.995 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:34.993+0000 7fd165297640 1 -- 172.21.15.100:0/442267185 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd160101910 msgr2=0x7fd160196900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:34.995 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:34.993+0000 7fd165297640 1 --2- 172.21.15.100:0/442267185 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd160101910 0x7fd160196900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:34.995 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:34.993+0000 7fd165297640 1 -- 172.21.15.100:0/442267185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1601034c0 msgr2=0x7fd16019deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:34.995 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:34.993+0000 7fd165297640 1 --2- 172.21.15.100:0/442267185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1601034c0 0x7fd16019deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:34.995 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:34.993+0000 7fd165297640 1 -- 172.21.15.100:0/442267185 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd16019e520 con 0x7fd160102b00 2024-05-10T11:43:34.996 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:34.993+0000 7fd166299640 1 --2- 172.21.15.100:0/442267185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1601034c0 0x7fd16019deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:43:34.996 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:34.993+0000 7fd165297640 1 --2- 172.21.15.100:0/442267185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd160102b00 0x7fd160196e40 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fd15000e9f0 tx=0x7fd15000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:43:34.996 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:34.993+0000 7fd156ffd640 1 -- 172.21.15.100:0/442267185 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd15000cd80 con 0x7fd160102b00 2024-05-10T11:43:34.996 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:34.993+0000 7fd167cfc640 1 -- 172.21.15.100:0/442267185 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd16019e810 con 0x7fd160102b00 2024-05-10T11:43:34.996 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:34.993+0000 7fd156ffd640 1 -- 172.21.15.100:0/442267185 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd1500108c0 con 0x7fd160102b00 2024-05-10T11:43:34.996 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:34.993+0000 7fd156ffd640 1 -- 172.21.15.100:0/442267185 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1500187c0 con 0x7fd160102b00 2024-05-10T11:43:34.996 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:34.993+0000 7fd167cfc640 1 -- 172.21.15.100:0/442267185 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd16019ed60 con 0x7fd160102b00 2024-05-10T11:43:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:34.996+0000 7fd16aa13740 1 Processor -- start 2024-05-10T11:43:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:34.996+0000 7fd16aa13740 1 -- start start 2024-05-10T11:43:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:34.997+0000 7fd16aa13740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55865c6c11c0 0x55865c6c15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:34.997+0000 7fd16aa13740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55865c6c1bb0 0x55865c6c66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:34.997+0000 7fd16aa13740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55865c6c6d20 0x55865c6c91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:34.997+0000 7fd16aa13740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55865c6215d0 con 0x55865c6c1bb0 2024-05-10T11:43:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:34.997+0000 7fd16aa13740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55865c5da890 con 0x55865c6c6d20 2024-05-10T11:43:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:34.997+0000 7fd16aa13740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55865c61d590 con 0x55865c6c11c0 2024-05-10T11:43:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:34.997+0000 7fd166299640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55865c6c11c0 0x55865c6c15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:34.997+0000 7fd166299640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55865c6c11c0 0x55865c6c15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:53380/0 (socket says 172.21.15.100:53380) 2024-05-10T11:43:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:34.997+0000 7fd166299640 1 -- 172.21.15.100:0/177482930 learned_addr learned my addr 172.21.15.100:0/177482930 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:43:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:34.997+0000 7fd165a98640 1 --2- 172.21.15.100:0/177482930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55865c6c1bb0 0x55865c6c66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:34.997+0000 7fd156ffd640 1 -- 172.21.15.100:0/442267185 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fd150010430 con 0x7fd160102b00 2024-05-10T11:43:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:34.997+0000 7fd166299640 1 -- 172.21.15.100:0/177482930 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55865c6c6d20 msgr2=0x55865c6c91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:43:34.999 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:34.997+0000 7fd166299640 1 --2- 172.21.15.100:0/177482930 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55865c6c6d20 0x55865c6c91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:34.999 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:34.997+0000 7fd166299640 1 -- 172.21.15.100:0/177482930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55865c6c1bb0 msgr2=0x55865c6c66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:43:34.999 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:34.997+0000 7fd166299640 1 --2- 172.21.15.100:0/177482930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55865c6c1bb0 0x55865c6c66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:34.999 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:34.997+0000 7fd166299640 1 -- 172.21.15.100:0/177482930 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55865c333210 con 0x55865c6c11c0 2024-05-10T11:43:34.999 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:34.998+0000 7fd166299640 1 --2- 172.21.15.100:0/177482930 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55865c6c11c0 0x55865c6c15e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd1580117e0 tx=0x7fd158011ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=210aa9128199fc0a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:43:34.999 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:34.998+0000 7fd14bfff640 1 -- 172.21.15.100:0/177482930 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd158011310 con 0x55865c6c11c0 2024-05-10T11:43:34.999 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:34.998+0000 7fd14bfff640 1 -- 172.21.15.100:0/177482930 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd158003e00 con 0x55865c6c11c0 2024-05-10T11:43:34.999 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:34.998+0000 7fd14bfff640 1 -- 172.21.15.100:0/177482930 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd158004430 con 0x55865c6c11c0 2024-05-10T11:43:35.000 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:34.998+0000 7fd156ffd640 1 --2- 172.21.15.100:0/442267185 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fd14c0b28b0 0x7fd14c0b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:35.000 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:34.998+0000 7fd156ffd640 1 -- 172.21.15.100:0/442267185 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fd1500e2900 con 0x7fd160102b00 2024-05-10T11:43:35.000 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:34.998+0000 7fd165a98640 1 --2- 172.21.15.100:0/442267185 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fd14c0b28b0 0x7fd14c0b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:35.000 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:34.999+0000 7fd16aa13740 1 -- 172.21.15.100:0/177482930 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55865c6c11c0 msgr2=0x55865c6c15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:35.000 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:34.999+0000 7fd16aa13740 1 --2- 172.21.15.100:0/177482930 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55865c6c11c0 0x55865c6c15e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd1580117e0 tx=0x7fd158011ca0 comp rx=0 tx=0).stop 2024-05-10T11:43:35.000 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:34.999+0000 7fd16aa13740 1 -- 172.21.15.100:0/177482930 shutdown_connections 2024-05-10T11:43:35.000 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:34.999+0000 7fd16aa13740 1 --2- 172.21.15.100:0/177482930 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55865c6c6d20 0x55865c6c91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:35.000 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:34.999+0000 7fd16aa13740 1 --2- 172.21.15.100:0/177482930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55865c6c1bb0 0x55865c6c66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:35.000 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:34.999+0000 7fd16aa13740 1 --2- 172.21.15.100:0/177482930 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55865c6c11c0 0x55865c6c15e0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:35.001 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:34.999+0000 7fd16aa13740 1 -- 172.21.15.100:0/177482930 >> 172.21.15.100:0/177482930 conn(0x55865c6b8420 msgr2=0x55865c6b8820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:43:35.001 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:34.999+0000 7fd165a98640 1 --2- 172.21.15.100:0/442267185 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fd14c0b28b0 0x7fd14c0b4d70 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fd15c0057c0 tx=0x7fd15c031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:43:35.001 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.000+0000 7fd16aa13740 1 -- 172.21.15.100:0/177482930 shutdown_connections 2024-05-10T11:43:35.001 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.000+0000 7fd16aa13740 1 -- 172.21.15.100:0/177482930 wait complete. 2024-05-10T11:43:35.001 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.000+0000 7fd16aa13740 1 -- start start 2024-05-10T11:43:35.001 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.000+0000 7fd16aa13740 1 Processor -- start 2024-05-10T11:43:35.003 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.000+0000 7fd16aa13740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55865c6c11c0 0x55865c6cb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:35.003 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.001+0000 7fd16aa13740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55865c6c1bb0 0x55865c6cb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:35.003 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.001+0000 7fd16aa13740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55865c6c6d20 0x55865c6d29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:35.004 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.001+0000 7fd16aa13740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55865c6c9cd0 con 0x55865c6c6d20 2024-05-10T11:43:35.004 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.001+0000 7fd16aa13740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55865c5de370 con 0x55865c6c11c0 2024-05-10T11:43:35.004 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.001+0000 7fd16aa13740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55865c6c9e50 con 0x55865c6c1bb0 2024-05-10T11:43:35.004 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.001+0000 7fd16aa13740 10 client.0 _ll_register_callbacks cb 0x55865c332a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:43:35.004 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.001+0000 7fd165a98640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55865c6c11c0 0x55865c6cb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:35.004 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.001+0000 7fd165a98640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55865c6c11c0 0x55865c6cb440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:35384/0 (socket says 172.21.15.100:35384) 2024-05-10T11:43:35.004 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.001+0000 7fd165a98640 1 -- 172.21.15.100:0/3712445009 learned_addr learned my addr 172.21.15.100:0/3712445009 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:43:35.004 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.001+0000 7fd166299640 1 --2- 172.21.15.100:0/3712445009 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55865c6c6d20 0x55865c6d29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:35.004 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.001+0000 7fd165297640 1 --2- 172.21.15.100:0/3712445009 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55865c6c1bb0 0x55865c6cb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:35.004 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.001+0000 7fd166299640 1 -- 172.21.15.100:0/3712445009 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55865c6c1bb0 msgr2=0x55865c6cb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:43:35.004 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.001+0000 7fd166299640 1 --2- 172.21.15.100:0/3712445009 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55865c6c1bb0 0x55865c6cb980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:35.004 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.001+0000 7fd166299640 1 -- 172.21.15.100:0/3712445009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55865c6c11c0 msgr2=0x55865c6cb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:43:35.004 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.001+0000 7fd166299640 1 --2- 172.21.15.100:0/3712445009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55865c6c11c0 0x55865c6cb440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:35.004 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.001+0000 7fd166299640 1 -- 172.21.15.100:0/3712445009 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55865c3ff620 con 0x55865c6c6d20 2024-05-10T11:43:35.005 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.001+0000 7fd165a98640 1 --2- 172.21.15.100:0/3712445009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55865c6c11c0 0x55865c6cb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:43:35.005 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.001+0000 7fd166299640 1 --2- 172.21.15.100:0/3712445009 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55865c6c6d20 0x55865c6d29c0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fd15800a010 tx=0x7fd158011480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ffcc318d6ea090 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:43:35.005 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.001+0000 7fd1497fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:43:35.005 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.002+0000 7fd1497fa640 1 -- 172.21.15.100:0/3712445009 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd15802a060 con 0x55865c6c6d20 2024-05-10T11:43:35.005 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.002+0000 7fd1497fa640 1 -- 172.21.15.100:0/3712445009 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fd15800f9a0 con 0x55865c6c6d20 2024-05-10T11:43:35.005 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.002+0000 7fd16aa13740 1 -- 172.21.15.100:0/3712445009 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd158003e00 con 0x55865c6c6d20 2024-05-10T11:43:35.005 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.002+0000 7fd1497fa640 1 -- 172.21.15.100:0/3712445009 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fd158031050 con 0x55865c6c6d20 2024-05-10T11:43:35.005 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.002+0000 7fd16aa13740 10 client.20217 fetch_fsmap learned FSMap version 41 2024-05-10T11:43:35.005 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.002+0000 7fd16aa13740 1 -- 172.21.15.100:0/3712445009 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55865c6d59a0 con 0x55865c6c6d20 2024-05-10T11:43:35.005 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.002+0000 7fd1497fa640 1 -- 172.21.15.100:0/3712445009 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd158003e00 con 0x55865c6c6d20 2024-05-10T11:43:35.005 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.002+0000 7fd1497fa640 1 -- 172.21.15.100:0/3712445009 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fd158018020 con 0x55865c6c6d20 2024-05-10T11:43:35.005 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.002+0000 7fd16aa13740 10 client.20217 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:43:35.005 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.002+0000 7fd16aa13740 10 client.20217 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:43:35.005 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.002+0000 7fd16aa13740 20 client.20217 populate_metadata read hostname 'smithi100' 2024-05-10T11:43:35.005 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.003+0000 7fd16aa13740 1 --2- 172.21.15.100:0/3712445009 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55865c6d6dd0 0x55865c6d91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:35.006 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.003+0000 7fd16aa13740 4 client.20217 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:43:35.006 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.003+0000 7fd16aa13740 1 -- 172.21.15.100:0/3712445009 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55865c62f1e0 con 0x55865c6d6dd0 2024-05-10T11:43:35.006 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.003+0000 7fd165a98640 1 --2- 172.21.15.100:0/3712445009 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55865c6d6dd0 0x55865c6d91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:35.006 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.003+0000 7fd165a98640 1 --2- 172.21.15.100:0/3712445009 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55865c6d6dd0 0x55865c6d91c0 crc :-1 s=READY pgs=467 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71e0237c9cf4b48a server_cookie=ce75870552143c5b in_seq=0 out_seq=0 2024-05-10T11:43:35.006 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.003+0000 7fd1497fa640 10 client.20217 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:43:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.005+0000 7fd1497fa640 1 -- 172.21.15.100:0/3712445009 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55865c62f1e0 con 0x55865c6d6dd0 2024-05-10T11:43:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.005+0000 7fd1497fa640 10 client.20217 handle_command_reply: tid=0 2024-05-10T11:43:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.005+0000 7fd16aa13740 1 client.20217 shutdown 2024-05-10T11:43:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.005+0000 7fd16aa13740 1 -- 172.21.15.100:0/3712445009 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55865c6c6d20 msgr2=0x55865c6d29c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.005+0000 7fd16aa13740 1 --2- 172.21.15.100:0/3712445009 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55865c6c6d20 0x55865c6d29c0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fd15800a010 tx=0x7fd158011480 comp rx=0 tx=0).stop 2024-05-10T11:43:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.005+0000 7fd16aa13740 1 -- 172.21.15.100:0/3712445009 shutdown_connections 2024-05-10T11:43:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.005+0000 7fd16aa13740 1 --2- 172.21.15.100:0/3712445009 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55865c6d6dd0 0x55865c6d91c0 crc :-1 s=READY pgs=467 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.005+0000 7fd16aa13740 1 --2- 172.21.15.100:0/3712445009 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55865c6c6d20 0x55865c6d29c0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.005+0000 7fd1497fa640 0 client.20217 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:43:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.005+0000 7fd16aa13740 1 --2- 172.21.15.100:0/3712445009 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55865c6c1bb0 0x55865c6cb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.005+0000 7fd16aa13740 1 --2- 172.21.15.100:0/3712445009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55865c6c11c0 0x55865c6cb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.005+0000 7fd16aa13740 1 -- 172.21.15.100:0/3712445009 >> 172.21.15.100:0/3712445009 conn(0x55865c6b8420 msgr2=0x55865c6b7580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:43:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.005+0000 7fd16aa13740 1 -- 172.21.15.100:0/3712445009 shutdown_connections 2024-05-10T11:43:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.005+0000 7fd16aa13740 1 -- 172.21.15.100:0/3712445009 wait complete. 2024-05-10T11:43:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.006+0000 7fd16aa13740 20 client.20217 trim_cache size 0 max 16384 2024-05-10T11:43:35.031 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.030+0000 7fd16aa13740 1 Processor -- start 2024-05-10T11:43:35.032 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.030+0000 7fd16aa13740 1 -- start start 2024-05-10T11:43:35.032 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.030+0000 7fd16aa13740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55865c6c11c0 0x55865c6bb810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:35.032 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.031+0000 7fd16aa13740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55865c6c1bb0 0x55865c6bbd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:35.032 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.031+0000 7fd16aa13740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55865c6c6d20 0x55865c6d2980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:35.032 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.031+0000 7fd16aa13740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd1601abab0 con 0x55865c6c11c0 2024-05-10T11:43:35.032 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.031+0000 7fd16aa13740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd1601a9380 con 0x55865c6c6d20 2024-05-10T11:43:35.032 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.031+0000 7fd16aa13740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd1601a71d0 con 0x55865c6c1bb0 2024-05-10T11:43:35.032 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.031+0000 7fd165a98640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55865c6c11c0 0x55865c6bb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:35.032 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.031+0000 7fd165a98640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55865c6c11c0 0x55865c6bb810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47004/0 (socket says 172.21.15.100:47004) 2024-05-10T11:43:35.032 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.031+0000 7fd165a98640 1 -- 172.21.15.100:0/3962385397 learned_addr learned my addr 172.21.15.100:0/3962385397 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:43:35.032 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.031+0000 7fd165297640 1 --2- 172.21.15.100:0/3962385397 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55865c6c1bb0 0x55865c6bbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:35.032 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.031+0000 7fd166299640 1 --2- 172.21.15.100:0/3962385397 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55865c6c6d20 0x55865c6d2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:35.032 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.031+0000 7fd165a98640 1 -- 172.21.15.100:0/3962385397 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55865c6c1bb0 msgr2=0x55865c6bbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:43:35.032 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.031+0000 7fd165a98640 1 --2- 172.21.15.100:0/3962385397 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55865c6c1bb0 0x55865c6bbd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:35.033 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.031+0000 7fd165a98640 1 -- 172.21.15.100:0/3962385397 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55865c6c6d20 msgr2=0x55865c6d2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:43:35.033 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.031+0000 7fd165a98640 1 --2- 172.21.15.100:0/3962385397 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55865c6c6d20 0x55865c6d2980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:35.033 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.031+0000 7fd165a98640 1 -- 172.21.15.100:0/3962385397 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55865c6c60f0 con 0x55865c6c11c0 2024-05-10T11:43:35.033 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.031+0000 7fd165a98640 1 --2- 172.21.15.100:0/3962385397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55865c6c11c0 0x55865c6bb810 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fd15c009230 tx=0x7fd15c040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=673f021e164da8d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:43:35.033 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.031+0000 7fd14b7fe640 1 -- 172.21.15.100:0/3962385397 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd15c035040 con 0x55865c6c11c0 2024-05-10T11:43:35.033 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.032+0000 7fd14b7fe640 1 -- 172.21.15.100:0/3962385397 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd15c004240 con 0x55865c6c11c0 2024-05-10T11:43:35.033 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.032+0000 7fd14b7fe640 1 -- 172.21.15.100:0/3962385397 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd15c03d6b0 con 0x55865c6c11c0 2024-05-10T11:43:35.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.032+0000 7fd16aa13740 1 -- 172.21.15.100:0/3962385397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55865c6c11c0 msgr2=0x55865c6bb810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:35.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.032+0000 7fd16aa13740 1 --2- 172.21.15.100:0/3962385397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55865c6c11c0 0x55865c6bb810 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fd15c009230 tx=0x7fd15c040000 comp rx=0 tx=0).stop 2024-05-10T11:43:35.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.033+0000 7fd16aa13740 1 -- 172.21.15.100:0/3962385397 shutdown_connections 2024-05-10T11:43:35.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.033+0000 7fd16aa13740 1 --2- 172.21.15.100:0/3962385397 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55865c6c6d20 0x55865c6d2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:35.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.033+0000 7fd16aa13740 1 --2- 172.21.15.100:0/3962385397 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55865c6c1bb0 0x55865c6bbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:35.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.033+0000 7fd16aa13740 1 --2- 172.21.15.100:0/3962385397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55865c6c11c0 0x55865c6bb810 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:35.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.033+0000 7fd16aa13740 1 -- 172.21.15.100:0/3962385397 >> 172.21.15.100:0/3962385397 conn(0x55865c6b8420 msgr2=0x55865c6bb100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:43:35.037 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.033+0000 7fd16aa13740 1 -- 172.21.15.100:0/3962385397 shutdown_connections 2024-05-10T11:43:35.037 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.033+0000 7fd16aa13740 1 -- 172.21.15.100:0/3962385397 wait complete. 2024-05-10T11:43:35.037 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.033+0000 7fd16aa13740 1 -- start start 2024-05-10T11:43:35.037 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.033+0000 7fd16aa13740 1 Processor -- start 2024-05-10T11:43:35.037 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.034+0000 7fd16aa13740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55865c6c1bb0 0x55865c6cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:35.037 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.034+0000 7fd16aa13740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55865c6c6d20 0x55865c6d0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:35.037 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.034+0000 7fd16aa13740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55865c6d1090 0x55865c6d1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:35.037 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.034+0000 7fd16aa13740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd1601a7840 con 0x55865c6c1bb0 2024-05-10T11:43:35.037 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.034+0000 7fd16aa13740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd1601a8cb0 con 0x55865c6c6d20 2024-05-10T11:43:35.037 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.034+0000 7fd16aa13740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd1601a7f10 con 0x55865c6d1090 2024-05-10T11:43:35.037 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.034+0000 7fd16aa13740 10 client.0 _ll_register_callbacks cb 0x7fd1580093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:43:35.038 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.034+0000 7fd165a98640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55865c6c1bb0 0x55865c6cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:35.038 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.034+0000 7fd166299640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55865c6d1090 0x55865c6d1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:35.038 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.034+0000 7fd165a98640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55865c6c1bb0 0x55865c6cfb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47014/0 (socket says 172.21.15.100:47014) 2024-05-10T11:43:35.038 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.034+0000 7fd165297640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55865c6c6d20 0x55865c6d0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:35.038 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.034+0000 7fd165a98640 1 -- 172.21.15.100:0/980276719 learned_addr learned my addr 172.21.15.100:0/980276719 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:43:35.038 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.034+0000 7fd165a98640 1 -- 172.21.15.100:0/980276719 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55865c6d1090 msgr2=0x55865c6d1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:43:35.038 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.034+0000 7fd165a98640 1 --2- 172.21.15.100:0/980276719 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55865c6d1090 0x55865c6d1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:35.038 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.034+0000 7fd165a98640 1 -- 172.21.15.100:0/980276719 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55865c6c6d20 msgr2=0x55865c6d0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:43:35.038 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.034+0000 7fd165a98640 1 --2- 172.21.15.100:0/980276719 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55865c6c6d20 0x55865c6d0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:35.038 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.034+0000 7fd165a98640 1 -- 172.21.15.100:0/980276719 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55865c6d1a50 con 0x55865c6c1bb0 2024-05-10T11:43:35.038 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.035+0000 7fd165a98640 1 --2- 172.21.15.100:0/980276719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55865c6c1bb0 0x55865c6cfb40 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fd15c02a930 tx=0x7fd15c031ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfc0d4f09ab7b28c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:43:35.038 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.035+0000 7fd1497fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:43:35.038 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.035+0000 7fd1497fa640 1 -- 172.21.15.100:0/980276719 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd15c05e020 con 0x55865c6c1bb0 2024-05-10T11:43:35.038 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.035+0000 7fd16aa13740 1 -- 172.21.15.100:0/980276719 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd15c004240 con 0x55865c6c1bb0 2024-05-10T11:43:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.035+0000 7fd1497fa640 1 -- 172.21.15.100:0/980276719 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fd15c0035f0 con 0x55865c6c1bb0 2024-05-10T11:43:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.035+0000 7fd1497fa640 1 -- 172.21.15.100:0/980276719 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fd15c035040 con 0x55865c6c1bb0 2024-05-10T11:43:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.035+0000 7fd16aa13740 10 client.20223 fetch_fsmap learned FSMap version 41 2024-05-10T11:43:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.035+0000 7fd16aa13740 1 -- 172.21.15.100:0/980276719 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55865c6d2db0 con 0x55865c6c1bb0 2024-05-10T11:43:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.035+0000 7fd1497fa640 1 -- 172.21.15.100:0/980276719 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd15c05a480 con 0x55865c6c1bb0 2024-05-10T11:43:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.035+0000 7fd1497fa640 1 -- 172.21.15.100:0/980276719 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fd15c044040 con 0x55865c6c1bb0 2024-05-10T11:43:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.036+0000 7fd16aa13740 10 client.20223 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:43:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.036+0000 7fd16aa13740 10 client.20223 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:43:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.036+0000 7fd16aa13740 20 client.20223 populate_metadata read hostname 'smithi100' 2024-05-10T11:43:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.036+0000 7fd16aa13740 1 --2- 172.21.15.100:0/980276719 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55865c741e20 0x55865c744240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.036+0000 7fd16aa13740 4 client.20223 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:43:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.036+0000 7fd16aa13740 1 -- 172.21.15.100:0/980276719 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fd15c004240 con 0x55865c741e20 2024-05-10T11:43:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.036+0000 7fd165297640 1 --2- 172.21.15.100:0/980276719 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55865c741e20 0x55865c744240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.036+0000 7fd165297640 1 --2- 172.21.15.100:0/980276719 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55865c741e20 0x55865c744240 crc :-1 s=READY pgs=468 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d82ead7f6049d398 server_cookie=8cdc6dfe04032174 in_seq=0 out_seq=0 2024-05-10T11:43:35.040 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.036+0000 7fd1497fa640 10 client.20223 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:43:35.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.230+0000 7fd1497fa640 1 -- 172.21.15.100:0/980276719 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fd15c004240 con 0x55865c741e20 2024-05-10T11:43:35.232 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.230+0000 7fd1497fa640 10 client.20223 handle_command_reply: tid=0 2024-05-10T11:43:35.232 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.230+0000 7fd16aa13740 1 client.20223 shutdown 2024-05-10T11:43:35.232 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.230+0000 7fd16aa13740 1 -- 172.21.15.100:0/980276719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55865c6c1bb0 msgr2=0x55865c6cfb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:35.232 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.230+0000 7fd16aa13740 1 --2- 172.21.15.100:0/980276719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55865c6c1bb0 0x55865c6cfb40 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fd15c02a930 tx=0x7fd15c031ef0 comp rx=0 tx=0).stop 2024-05-10T11:43:35.232 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.231+0000 7fd16aa13740 1 -- 172.21.15.100:0/980276719 shutdown_connections 2024-05-10T11:43:35.232 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.231+0000 7fd16aa13740 1 --2- 172.21.15.100:0/980276719 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55865c741e20 0x55865c744240 crc :-1 s=READY pgs=468 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:35.232 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.231+0000 7fd16aa13740 1 --2- 172.21.15.100:0/980276719 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55865c6d1090 0x55865c6d1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:35.232 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.231+0000 7fd1497fa640 0 client.20223 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:43:35.232 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.231+0000 7fd16aa13740 1 --2- 172.21.15.100:0/980276719 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55865c6c6d20 0x55865c6d0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:35.232 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.231+0000 7fd16aa13740 1 --2- 172.21.15.100:0/980276719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55865c6c1bb0 0x55865c6cfb40 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:35.232 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.231+0000 7fd16aa13740 1 -- 172.21.15.100:0/980276719 >> 172.21.15.100:0/980276719 conn(0x55865c6b8420 msgr2=0x55865c6bb990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:43:35.232 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.231+0000 7fd16aa13740 1 -- 172.21.15.100:0/980276719 shutdown_connections 2024-05-10T11:43:35.233 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.231+0000 7fd16aa13740 1 -- 172.21.15.100:0/980276719 wait complete. 2024-05-10T11:43:35.233 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.231+0000 7fd16aa13740 20 client.20223 trim_cache size 0 max 16384 2024-05-10T11:43:35.233 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:43:35.233 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:43:35.233 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:43:35.233 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:43:35.234 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.232+0000 7fd167cfc640 1 -- 172.21.15.100:0/442267185 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fd14c0b28b0 msgr2=0x7fd14c0b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:35.234 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.232+0000 7fd167cfc640 1 --2- 172.21.15.100:0/442267185 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fd14c0b28b0 0x7fd14c0b4d70 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fd15c0057c0 tx=0x7fd15c031400 comp rx=0 tx=0).stop 2024-05-10T11:43:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.232+0000 7fd167cfc640 1 -- 172.21.15.100:0/442267185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd160102b00 msgr2=0x7fd160196e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.232+0000 7fd167cfc640 1 --2- 172.21.15.100:0/442267185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd160102b00 0x7fd160196e40 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fd15000e9f0 tx=0x7fd15000eeb0 comp rx=0 tx=0).stop 2024-05-10T11:43:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.232+0000 7fd167cfc640 1 -- 172.21.15.100:0/442267185 shutdown_connections 2024-05-10T11:43:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.232+0000 7fd167cfc640 1 --2- 172.21.15.100:0/442267185 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fd14c0b28b0 0x7fd14c0b4d70 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.232+0000 7fd167cfc640 1 --2- 172.21.15.100:0/442267185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1601034c0 0x7fd16019deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.232+0000 7fd167cfc640 1 --2- 172.21.15.100:0/442267185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd160102b00 0x7fd160196e40 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.232+0000 7fd167cfc640 1 --2- 172.21.15.100:0/442267185 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd160101910 0x7fd160196900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.232+0000 7fd167cfc640 1 -- 172.21.15.100:0/442267185 >> 172.21.15.100:0/442267185 conn(0x7fd1600fce80 msgr2=0x7fd160105fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:43:35.237 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.232+0000 7fd167cfc640 1 -- 172.21.15.100:0/442267185 shutdown_connections 2024-05-10T11:43:35.237 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:35.232+0000 7fd167cfc640 1 -- 172.21.15.100:0/442267185 wait complete. 2024-05-10T11:43:35.511 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:43:35 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:43:35.511 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:43:35 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:43:35.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:43:35 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:43:35.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:43:35 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:43:35.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:43:35 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:43:35.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:43:35 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:43:35.961 INFO:tasks.workunit.client.0.smithi100.stdout: 10 114064 8.88 MB/sec execute 569 sec latency 147.065 ms 2024-05-10T11:43:36.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:43:36 smithi178 ceph-mon[33359]: scrub summary: idle 2024-05-10T11:43:36.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:43:36 smithi178 ceph-mon[33359]: pgmap v1177: 129 pgs: 129 active+clean; 570 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 4.2 MiB/s wr, 315 op/s 2024-05-10T11:43:36.530 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:43:36 smithi188 ceph-mon[34075]: scrub summary: idle 2024-05-10T11:43:36.530 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:43:36 smithi188 ceph-mon[34075]: pgmap v1177: 129 pgs: 129 active+clean; 570 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 4.2 MiB/s wr, 315 op/s 2024-05-10T11:43:36.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:43:36 smithi100 ceph-mon[30296]: scrub summary: idle 2024-05-10T11:43:36.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:43:36 smithi100 ceph-mon[30296]: pgmap v1177: 129 pgs: 129 active+clean; 570 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 4.2 MiB/s wr, 315 op/s 2024-05-10T11:43:36.962 INFO:tasks.workunit.client.0.smithi100.stdout: 10 114232 8.89 MB/sec execute 570 sec latency 65.714 ms 2024-05-10T11:43:37.961 INFO:tasks.workunit.client.0.smithi100.stdout: 10 114396 8.89 MB/sec execute 571 sec latency 72.491 ms 2024-05-10T11:43:38.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:43:38 smithi178 ceph-mon[33359]: pgmap v1178: 129 pgs: 129 active+clean; 576 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1014 KiB/s rd, 4.8 MiB/s wr, 316 op/s 2024-05-10T11:43:38.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:43:38 smithi188 ceph-mon[34075]: pgmap v1178: 129 pgs: 129 active+clean; 576 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1014 KiB/s rd, 4.8 MiB/s wr, 316 op/s 2024-05-10T11:43:38.540 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:43:38 smithi100 ceph-mon[30296]: pgmap v1178: 129 pgs: 129 active+clean; 576 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1014 KiB/s rd, 4.8 MiB/s wr, 316 op/s 2024-05-10T11:43:38.961 INFO:tasks.workunit.client.0.smithi100.stdout: 10 114565 8.89 MB/sec execute 572 sec latency 70.508 ms 2024-05-10T11:43:39.961 INFO:tasks.workunit.client.0.smithi100.stdout: 10 114741 8.89 MB/sec execute 573 sec latency 71.541 ms 2024-05-10T11:43:40.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:43:40 smithi178 ceph-mon[33359]: pgmap v1179: 129 pgs: 129 active+clean; 576 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 4.5 MiB/s wr, 283 op/s 2024-05-10T11:43:40.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:43:40 smithi188 ceph-mon[34075]: pgmap v1179: 129 pgs: 129 active+clean; 576 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 4.5 MiB/s wr, 283 op/s 2024-05-10T11:43:40.541 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:43:40 smithi100 ceph-mon[30296]: pgmap v1179: 129 pgs: 129 active+clean; 576 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 4.5 MiB/s wr, 283 op/s 2024-05-10T11:43:40.961 INFO:tasks.workunit.client.0.smithi100.stdout: 10 114882 8.88 MB/sec execute 574 sec latency 104.283 ms 2024-05-10T11:43:41.961 INFO:tasks.workunit.client.0.smithi100.stdout: 10 115065 8.88 MB/sec execute 575 sec latency 74.829 ms 2024-05-10T11:43:42.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:43:42 smithi178 ceph-mon[33359]: pgmap v1180: 129 pgs: 129 active+clean; 579 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 873 KiB/s rd, 4.4 MiB/s wr, 284 op/s 2024-05-10T11:43:42.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:43:42 smithi188 ceph-mon[34075]: pgmap v1180: 129 pgs: 129 active+clean; 579 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 873 KiB/s rd, 4.4 MiB/s wr, 284 op/s 2024-05-10T11:43:42.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:43:42 smithi100 ceph-mon[30296]: pgmap v1180: 129 pgs: 129 active+clean; 579 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 873 KiB/s rd, 4.4 MiB/s wr, 284 op/s 2024-05-10T11:43:42.961 INFO:tasks.workunit.client.0.smithi100.stdout: 10 115230 8.88 MB/sec execute 576 sec latency 104.694 ms 2024-05-10T11:43:43.961 INFO:tasks.workunit.client.0.smithi100.stdout: 10 115369 8.88 MB/sec execute 577 sec latency 159.417 ms 2024-05-10T11:43:44.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:43:44 smithi178 ceph-mon[33359]: pgmap v1181: 129 pgs: 129 active+clean; 582 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 902 KiB/s rd, 4.3 MiB/s wr, 288 op/s 2024-05-10T11:43:44.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:43:44 smithi188 ceph-mon[34075]: pgmap v1181: 129 pgs: 129 active+clean; 582 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 902 KiB/s rd, 4.3 MiB/s wr, 288 op/s 2024-05-10T11:43:44.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:43:44 smithi100 ceph-mon[30296]: pgmap v1181: 129 pgs: 129 active+clean; 582 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 902 KiB/s rd, 4.3 MiB/s wr, 288 op/s 2024-05-10T11:43:44.962 INFO:tasks.workunit.client.0.smithi100.stdout: 10 115561 8.89 MB/sec execute 578 sec latency 102.412 ms 2024-05-10T11:43:45.962 INFO:tasks.workunit.client.0.smithi100.stdout: 10 115788 8.90 MB/sec execute 579 sec latency 71.137 ms 2024-05-10T11:43:46.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:43:46 smithi178 ceph-mon[33359]: pgmap v1182: 129 pgs: 129 active+clean; 583 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 4.7 MiB/s wr, 285 op/s 2024-05-10T11:43:46.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:43:46 smithi188 ceph-mon[34075]: pgmap v1182: 129 pgs: 129 active+clean; 583 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 4.7 MiB/s wr, 285 op/s 2024-05-10T11:43:46.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:43:46 smithi100 ceph-mon[30296]: pgmap v1182: 129 pgs: 129 active+clean; 583 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 4.7 MiB/s wr, 285 op/s 2024-05-10T11:43:46.962 INFO:tasks.workunit.client.0.smithi100.stdout: 10 115919 8.89 MB/sec execute 580 sec latency 135.528 ms 2024-05-10T11:43:47.962 INFO:tasks.workunit.client.0.smithi100.stdout: 10 116050 8.88 MB/sec execute 581 sec latency 95.590 ms 2024-05-10T11:43:48.347 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.346+0000 7fa6872e8640 1 -- 172.21.15.100:0/2185674554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa680102a20 msgr2=0x7fa680102e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:48.347 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.346+0000 7fa6872e8640 1 --2- 172.21.15.100:0/2185674554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa680102a20 0x7fa680102e40 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fa67c009900 tx=0x7fa67c02a3f0 comp rx=0 tx=0).stop 2024-05-10T11:43:48.347 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.346+0000 7fa6872e8640 1 -- 172.21.15.100:0/2185674554 shutdown_connections 2024-05-10T11:43:48.347 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.346+0000 7fa6872e8640 1 --2- 172.21.15.100:0/2185674554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6801045d0 0x7fa68010ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:48.347 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.346+0000 7fa6872e8640 1 --2- 172.21.15.100:0/2185674554 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa680103c10 0x7fa680104090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:48.347 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.346+0000 7fa6872e8640 1 --2- 172.21.15.100:0/2185674554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa680102a20 0x7fa680102e40 secure :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fa67c009900 tx=0x7fa67c02a3f0 comp rx=0 tx=0).stop 2024-05-10T11:43:48.347 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.346+0000 7fa6872e8640 1 -- 172.21.15.100:0/2185674554 >> 172.21.15.100:0/2185674554 conn(0x7fa6800fdf90 msgr2=0x7fa6801003d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:43:48.347 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.346+0000 7fa6872e8640 1 -- 172.21.15.100:0/2185674554 shutdown_connections 2024-05-10T11:43:48.348 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.346+0000 7fa6872e8640 1 -- 172.21.15.100:0/2185674554 wait complete. 2024-05-10T11:43:48.349 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.347+0000 7fa6872e8640 1 Processor -- start 2024-05-10T11:43:48.349 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.347+0000 7fa6872e8640 1 -- start start 2024-05-10T11:43:48.349 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.348+0000 7fa6872e8640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa680103c10 0x7fa68019f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:48.349 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.348+0000 7fa6872e8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6801045d0 0x7fa68019f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:48.349 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.348+0000 7fa6872e8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6801a4870 0x7fa6801a6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:48.349 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.348+0000 7fa6872e8640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa68010e420 con 0x7fa6801a4870 2024-05-10T11:43:48.349 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.348+0000 7fa6872e8640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa68010e2a0 con 0x7fa680103c10 2024-05-10T11:43:48.349 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.348+0000 7fa6872e8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa68010e5a0 con 0x7fa6801045d0 2024-05-10T11:43:48.350 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.348+0000 7fa684883640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6801045d0 0x7fa68019f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:48.350 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.348+0000 7fa684883640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6801045d0 0x7fa68019f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:49686/0 (socket says 172.21.15.100:49686) 2024-05-10T11:43:48.350 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.348+0000 7fa684883640 1 -- 172.21.15.100:0/21863617 learned_addr learned my addr 172.21.15.100:0/21863617 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:43:48.350 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.348+0000 7fa685885640 1 --2- 172.21.15.100:0/21863617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6801a4870 0x7fa6801a6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:48.350 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.349+0000 7fa684883640 1 -- 172.21.15.100:0/21863617 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa680103c10 msgr2=0x7fa68019f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:48.350 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.349+0000 7fa684883640 1 --2- 172.21.15.100:0/21863617 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa680103c10 0x7fa68019f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:48.350 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.349+0000 7fa684883640 1 -- 172.21.15.100:0/21863617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6801a4870 msgr2=0x7fa6801a6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:48.350 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.349+0000 7fa684883640 1 --2- 172.21.15.100:0/21863617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6801a4870 0x7fa6801a6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:48.350 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.349+0000 7fa684883640 1 -- 172.21.15.100:0/21863617 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6801a72d0 con 0x7fa6801045d0 2024-05-10T11:43:48.351 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.349+0000 7fa685885640 1 --2- 172.21.15.100:0/21863617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6801a4870 0x7fa6801a6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:43:48.351 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.349+0000 7fa684883640 1 --2- 172.21.15.100:0/21863617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6801045d0 0x7fa68019f810 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fa67000e9f0 tx=0x7fa67000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:43:48.351 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.349+0000 7fa6767fc640 1 -- 172.21.15.100:0/21863617 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa67000cd80 con 0x7fa6801045d0 2024-05-10T11:43:48.352 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.349+0000 7fa6767fc640 1 -- 172.21.15.100:0/21863617 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa6700108c0 con 0x7fa6801045d0 2024-05-10T11:43:48.352 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.349+0000 7fa6767fc640 1 -- 172.21.15.100:0/21863617 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6700187c0 con 0x7fa6801045d0 2024-05-10T11:43:48.352 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.349+0000 7fa6872e8640 1 -- 172.21.15.100:0/21863617 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6801a75c0 con 0x7fa6801045d0 2024-05-10T11:43:48.352 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.349+0000 7fa6872e8640 1 -- 172.21.15.100:0/21863617 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6801a7ac0 con 0x7fa6801045d0 2024-05-10T11:43:48.357 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.354+0000 7fa68a00a740 1 Processor -- start 2024-05-10T11:43:48.357 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.354+0000 7fa68a00a740 1 -- start start 2024-05-10T11:43:48.357 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.354+0000 7fa6767fc640 1 -- 172.21.15.100:0/21863617 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fa670010430 con 0x7fa6801045d0 2024-05-10T11:43:48.358 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.354+0000 7fa68a00a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562892c11dc0 0x562892c121e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:48.358 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.354+0000 7fa68a00a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562892c127b0 0x562892c172b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:48.358 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.354+0000 7fa68a00a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562892c17920 0x562892c19db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:48.358 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.354+0000 7fa68a00a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x562892b655d0 con 0x562892c11dc0 2024-05-10T11:43:48.358 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.354+0000 7fa68a00a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562892b1e890 con 0x562892c127b0 2024-05-10T11:43:48.358 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.354+0000 7fa68a00a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x562892b61590 con 0x562892c17920 2024-05-10T11:43:48.358 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.354+0000 7fa685885640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562892c11dc0 0x562892c121e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:48.358 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.354+0000 7fa685084640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562892c127b0 0x562892c172b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:48.358 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.354+0000 7fa685885640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562892c11dc0 0x562892c121e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42828/0 (socket says 172.21.15.100:42828) 2024-05-10T11:43:48.358 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.354+0000 7fa685084640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562892c127b0 0x562892c172b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:33448/0 (socket says 172.21.15.100:33448) 2024-05-10T11:43:48.359 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.354+0000 7fa685885640 1 -- 172.21.15.100:0/2247829242 learned_addr learned my addr 172.21.15.100:0/2247829242 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:43:48.359 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.355+0000 7fa684883640 1 --2- 172.21.15.100:0/2247829242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562892c17920 0x562892c19db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:48.359 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.355+0000 7fa685084640 1 -- 172.21.15.100:0/2247829242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562892c17920 msgr2=0x562892c19db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:43:48.359 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.355+0000 7fa685084640 1 --2- 172.21.15.100:0/2247829242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562892c17920 0x562892c19db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:48.359 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.355+0000 7fa685084640 1 -- 172.21.15.100:0/2247829242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562892c11dc0 msgr2=0x562892c121e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:43:48.359 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.355+0000 7fa685084640 1 --2- 172.21.15.100:0/2247829242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562892c11dc0 0x562892c121e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:48.359 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.355+0000 7fa685084640 1 -- 172.21.15.100:0/2247829242 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562892877210 con 0x562892c127b0 2024-05-10T11:43:48.359 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.355+0000 7fa685885640 1 --2- 172.21.15.100:0/2247829242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562892c11dc0 0x562892c121e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:43:48.359 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.355+0000 7fa684883640 1 --2- 172.21.15.100:0/2247829242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562892c17920 0x562892c19db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:43:48.360 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.355+0000 7fa685084640 1 --2- 172.21.15.100:0/2247829242 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562892c127b0 0x562892c172b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa680102560 tx=0x7fa67c004130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf7b2eb802870e80 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:43:48.360 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.356+0000 7fa66b7fe640 1 -- 172.21.15.100:0/2247829242 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa67c03f020 con 0x562892c127b0 2024-05-10T11:43:48.360 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.356+0000 7fa6767fc640 1 --2- 172.21.15.100:0/21863617 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa66c0b2970 0x7fa66c0b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:48.360 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.356+0000 7fa66b7fe640 1 -- 172.21.15.100:0/2247829242 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa67c0044a0 con 0x562892c127b0 2024-05-10T11:43:48.360 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.356+0000 7fa66b7fe640 1 -- 172.21.15.100:0/2247829242 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa67c035070 con 0x562892c127b0 2024-05-10T11:43:48.360 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.356+0000 7fa685885640 1 --2- 172.21.15.100:0/21863617 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa66c0b2970 0x7fa66c0b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:48.360 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.356+0000 7fa6767fc640 1 -- 172.21.15.100:0/21863617 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fa6700e2a10 con 0x7fa6801045d0 2024-05-10T11:43:48.360 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.357+0000 7fa68a00a740 1 -- 172.21.15.100:0/2247829242 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562892c127b0 msgr2=0x562892c172b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:48.360 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.357+0000 7fa68a00a740 1 --2- 172.21.15.100:0/2247829242 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562892c127b0 0x562892c172b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa680102560 tx=0x7fa67c004130 comp rx=0 tx=0).stop 2024-05-10T11:43:48.361 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.357+0000 7fa685885640 1 --2- 172.21.15.100:0/21863617 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa66c0b2970 0x7fa66c0b4e30 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x562892c13010 tx=0x7fa678009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:43:48.361 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.357+0000 7fa68a00a740 1 -- 172.21.15.100:0/2247829242 shutdown_connections 2024-05-10T11:43:48.361 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.357+0000 7fa68a00a740 1 --2- 172.21.15.100:0/2247829242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562892c17920 0x562892c19db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:48.361 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.357+0000 7fa68a00a740 1 --2- 172.21.15.100:0/2247829242 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562892c127b0 0x562892c172b0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:48.361 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.357+0000 7fa68a00a740 1 --2- 172.21.15.100:0/2247829242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562892c11dc0 0x562892c121e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:48.361 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.357+0000 7fa68a00a740 1 -- 172.21.15.100:0/2247829242 >> 172.21.15.100:0/2247829242 conn(0x562892bfc420 msgr2=0x562892bfc820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:43:48.361 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.357+0000 7fa68a00a740 1 -- 172.21.15.100:0/2247829242 shutdown_connections 2024-05-10T11:43:48.361 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.357+0000 7fa68a00a740 1 -- 172.21.15.100:0/2247829242 wait complete. 2024-05-10T11:43:48.361 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.357+0000 7fa68a00a740 1 -- start start 2024-05-10T11:43:48.362 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.357+0000 7fa68a00a740 1 Processor -- start 2024-05-10T11:43:48.365 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.358+0000 7fa68a00a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562892c11dc0 0x562892c02470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:48.365 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.358+0000 7fa68a00a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562892c127b0 0x562892c029b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:48.365 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.358+0000 7fa68a00a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562892c17920 0x562892c099f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:48.366 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.358+0000 7fa68a00a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x562892c00c30 con 0x562892c17920 2024-05-10T11:43:48.366 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.358+0000 7fa68a00a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562892b22370 con 0x562892c11dc0 2024-05-10T11:43:48.366 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.358+0000 7fa68a00a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x562892c00db0 con 0x562892c127b0 2024-05-10T11:43:48.366 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.358+0000 7fa684883640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562892c127b0 0x562892c029b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:48.366 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.358+0000 7fa685885640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562892c17920 0x562892c099f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:48.366 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.358+0000 7fa684883640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562892c127b0 0x562892c029b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:49704/0 (socket says 172.21.15.100:49704) 2024-05-10T11:43:48.366 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.358+0000 7fa684883640 1 -- 172.21.15.100:0/4243189450 learned_addr learned my addr 172.21.15.100:0/4243189450 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:43:48.366 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.358+0000 7fa685084640 1 --2- 172.21.15.100:0/4243189450 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562892c11dc0 0x562892c02470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:48.366 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.358+0000 7fa68a00a740 10 client.0 _ll_register_callbacks cb 0x562892876a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:43:48.366 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.358+0000 7fa684883640 1 -- 172.21.15.100:0/4243189450 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562892c11dc0 msgr2=0x562892c02470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:43:48.367 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.358+0000 7fa684883640 1 --2- 172.21.15.100:0/4243189450 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562892c11dc0 0x562892c02470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:48.367 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.358+0000 7fa684883640 1 -- 172.21.15.100:0/4243189450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562892c17920 msgr2=0x562892c099f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:43:48.367 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.358+0000 7fa684883640 1 --2- 172.21.15.100:0/4243189450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562892c17920 0x562892c099f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:48.367 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.358+0000 7fa684883640 1 -- 172.21.15.100:0/4243189450 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562892943620 con 0x562892c127b0 2024-05-10T11:43:48.367 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.359+0000 7fa684883640 1 --2- 172.21.15.100:0/4243189450 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562892c127b0 0x562892c029b0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa6700a6f40 tx=0x7fa6700a97e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ba0ac9fd4c97ce5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:43:48.367 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.359+0000 7fa668ff9640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:43:48.367 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.359+0000 7fa668ff9640 1 -- 172.21.15.100:0/4243189450 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6700a9cd0 con 0x562892c127b0 2024-05-10T11:43:48.367 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.359+0000 7fa68a00a740 1 -- 172.21.15.100:0/4243189450 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa67c0044a0 con 0x562892c127b0 2024-05-10T11:43:48.367 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.359+0000 7fa668ff9640 1 -- 172.21.15.100:0/4243189450 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fa670010a70 con 0x562892c127b0 2024-05-10T11:43:48.367 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.359+0000 7fa668ff9640 1 -- 172.21.15.100:0/4243189450 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fa6700ae080 con 0x562892c127b0 2024-05-10T11:43:48.367 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.359+0000 7fa68a00a740 10 client.29599 fetch_fsmap learned FSMap version 41 2024-05-10T11:43:48.368 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.359+0000 7fa68a00a740 1 -- 172.21.15.100:0/4243189450 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x562892c0c9b0 con 0x562892c127b0 2024-05-10T11:43:48.368 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.359+0000 7fa668ff9640 1 -- 172.21.15.100:0/4243189450 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa67c0044a0 con 0x562892c127b0 2024-05-10T11:43:48.368 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.359+0000 7fa668ff9640 1 -- 172.21.15.100:0/4243189450 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fa6700a5840 con 0x562892c127b0 2024-05-10T11:43:48.368 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.359+0000 7fa68a00a740 10 client.29599 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:43:48.368 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.359+0000 7fa68a00a740 10 client.29599 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:43:48.368 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.359+0000 7fa68a00a740 20 client.29599 populate_metadata read hostname 'smithi100' 2024-05-10T11:43:48.368 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.359+0000 7fa68a00a740 1 --2- 172.21.15.100:0/4243189450 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562892c0e0d0 0x562892c10550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:48.368 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.359+0000 7fa68a00a740 4 client.29599 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:43:48.368 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.359+0000 7fa68a00a740 1 -- 172.21.15.100:0/4243189450 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562892b731e0 con 0x562892c0e0d0 2024-05-10T11:43:48.368 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.359+0000 7fa685084640 1 --2- 172.21.15.100:0/4243189450 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562892c0e0d0 0x562892c10550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:48.368 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.360+0000 7fa668ff9640 10 client.29599 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:43:48.369 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.360+0000 7fa685084640 1 --2- 172.21.15.100:0/4243189450 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562892c0e0d0 0x562892c10550 crc :-1 s=READY pgs=469 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac80a817b339aa5 server_cookie=de44351ca6e8a5dd in_seq=0 out_seq=0 2024-05-10T11:43:48.369 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.361+0000 7fa668ff9640 1 -- 172.21.15.100:0/4243189450 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562892b731e0 con 0x562892c0e0d0 2024-05-10T11:43:48.369 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.361+0000 7fa668ff9640 10 client.29599 handle_command_reply: tid=0 2024-05-10T11:43:48.369 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.361+0000 7fa68a00a740 1 client.29599 shutdown 2024-05-10T11:43:48.369 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.361+0000 7fa68a00a740 1 -- 172.21.15.100:0/4243189450 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562892c127b0 msgr2=0x562892c029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:48.369 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.361+0000 7fa68a00a740 1 --2- 172.21.15.100:0/4243189450 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562892c127b0 0x562892c029b0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa6700a6f40 tx=0x7fa6700a97e0 comp rx=0 tx=0).stop 2024-05-10T11:43:48.369 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.361+0000 7fa68a00a740 1 -- 172.21.15.100:0/4243189450 shutdown_connections 2024-05-10T11:43:48.369 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.361+0000 7fa68a00a740 1 --2- 172.21.15.100:0/4243189450 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562892c0e0d0 0x562892c10550 crc :-1 s=READY pgs=469 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:48.369 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.361+0000 7fa68a00a740 1 --2- 172.21.15.100:0/4243189450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562892c17920 0x562892c099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:48.369 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.361+0000 7fa68a00a740 1 --2- 172.21.15.100:0/4243189450 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562892c127b0 0x562892c029b0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:48.370 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.361+0000 7fa68a00a740 1 --2- 172.21.15.100:0/4243189450 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562892c11dc0 0x562892c02470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:48.370 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.361+0000 7fa68a00a740 1 -- 172.21.15.100:0/4243189450 >> 172.21.15.100:0/4243189450 conn(0x562892bfc420 msgr2=0x562892bfb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:43:48.370 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.361+0000 7fa668ff9640 0 client.29599 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:43:48.370 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.361+0000 7fa68a00a740 1 -- 172.21.15.100:0/4243189450 shutdown_connections 2024-05-10T11:43:48.370 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.361+0000 7fa68a00a740 1 -- 172.21.15.100:0/4243189450 wait complete. 2024-05-10T11:43:48.370 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.361+0000 7fa68a00a740 20 client.29599 trim_cache size 0 max 16384 2024-05-10T11:43:48.391 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.390+0000 7fa68a00a740 1 Processor -- start 2024-05-10T11:43:48.392 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.390+0000 7fa68a00a740 1 -- start start 2024-05-10T11:43:48.392 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.390+0000 7fa68a00a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562892c11dc0 0x562892c10df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:48.392 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.390+0000 7fa68a00a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562892c127b0 0x562892c11330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:48.392 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.390+0000 7fa68a00a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562892c17920 0x562892bff330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:48.392 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.390+0000 7fa68a00a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa6801abfb0 con 0x562892c11dc0 2024-05-10T11:43:48.392 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.390+0000 7fa68a00a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa6801a9880 con 0x562892c17920 2024-05-10T11:43:48.392 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.390+0000 7fa68a00a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa6800744f0 con 0x562892c127b0 2024-05-10T11:43:48.392 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.390+0000 7fa685084640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562892c11dc0 0x562892c10df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:48.392 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.390+0000 7fa685084640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562892c11dc0 0x562892c10df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42852/0 (socket says 172.21.15.100:42852) 2024-05-10T11:43:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.390+0000 7fa685084640 1 -- 172.21.15.100:0/3080676890 learned_addr learned my addr 172.21.15.100:0/3080676890 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:43:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.390+0000 7fa684883640 1 --2- 172.21.15.100:0/3080676890 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562892c127b0 0x562892c11330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.390+0000 7fa685885640 1 --2- 172.21.15.100:0/3080676890 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562892c17920 0x562892bff330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.391+0000 7fa685084640 1 -- 172.21.15.100:0/3080676890 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562892c127b0 msgr2=0x562892c11330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:43:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.391+0000 7fa685084640 1 --2- 172.21.15.100:0/3080676890 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562892c127b0 0x562892c11330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.391+0000 7fa685084640 1 -- 172.21.15.100:0/3080676890 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562892c17920 msgr2=0x562892bff330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:43:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.391+0000 7fa685084640 1 --2- 172.21.15.100:0/3080676890 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562892c17920 0x562892bff330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.391+0000 7fa685084640 1 -- 172.21.15.100:0/3080676890 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562892bfc250 con 0x562892c11dc0 2024-05-10T11:43:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.391+0000 7fa685885640 1 --2- 172.21.15.100:0/3080676890 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562892c17920 0x562892bff330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:43:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.391+0000 7fa685084640 1 --2- 172.21.15.100:0/3080676890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562892c11dc0 0x562892c10df0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fa67c006010 tx=0x7fa67c004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ea33fa8a4c4a567 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:43:48.394 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.391+0000 7fa66affd640 1 -- 172.21.15.100:0/3080676890 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa67c054070 con 0x562892c11dc0 2024-05-10T11:43:48.394 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.391+0000 7fa66affd640 1 -- 172.21.15.100:0/3080676890 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa67c003c80 con 0x562892c11dc0 2024-05-10T11:43:48.394 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.391+0000 7fa66affd640 1 -- 172.21.15.100:0/3080676890 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa67c03f070 con 0x562892c11dc0 2024-05-10T11:43:48.397 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.392+0000 7fa68a00a740 1 -- 172.21.15.100:0/3080676890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562892c11dc0 msgr2=0x562892c10df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:48.397 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.392+0000 7fa68a00a740 1 --2- 172.21.15.100:0/3080676890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562892c11dc0 0x562892c10df0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fa67c006010 tx=0x7fa67c004570 comp rx=0 tx=0).stop 2024-05-10T11:43:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.392+0000 7fa68a00a740 1 -- 172.21.15.100:0/3080676890 shutdown_connections 2024-05-10T11:43:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.392+0000 7fa68a00a740 1 --2- 172.21.15.100:0/3080676890 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562892c17920 0x562892bff330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.392+0000 7fa68a00a740 1 --2- 172.21.15.100:0/3080676890 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562892c127b0 0x562892c11330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.392+0000 7fa68a00a740 1 --2- 172.21.15.100:0/3080676890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562892c11dc0 0x562892c10df0 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.392+0000 7fa68a00a740 1 -- 172.21.15.100:0/3080676890 >> 172.21.15.100:0/3080676890 conn(0x562892bfc420 msgr2=0x562892c13ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:43:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.392+0000 7fa68a00a740 1 -- 172.21.15.100:0/3080676890 shutdown_connections 2024-05-10T11:43:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.392+0000 7fa68a00a740 1 -- 172.21.15.100:0/3080676890 wait complete. 2024-05-10T11:43:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.393+0000 7fa68a00a740 1 -- start start 2024-05-10T11:43:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.393+0000 7fa68a00a740 1 Processor -- start 2024-05-10T11:43:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.393+0000 7fa68a00a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562892c127b0 0x562892c1fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.393+0000 7fa68a00a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562892c17920 0x562892c1e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.393+0000 7fa68a00a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562892c1e720 0x562892c1eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.394+0000 7fa684883640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562892c17920 0x562892c1e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.394+0000 7fa685084640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562892c127b0 0x562892c1fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.394+0000 7fa684883640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562892c17920 0x562892c1e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:33464/0 (socket says 172.21.15.100:33464) 2024-05-10T11:43:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.394+0000 7fa684883640 1 -- 172.21.15.100:0/3823437815 learned_addr learned my addr 172.21.15.100:0/3823437815 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:43:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.394+0000 7fa68a00a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa6801a7de0 con 0x562892c1e720 2024-05-10T11:43:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.394+0000 7fa68a00a740 1 -- 172.21.15.100:0/3823437815 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa6801a91b0 con 0x562892c17920 2024-05-10T11:43:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.394+0000 7fa68a00a740 1 -- 172.21.15.100:0/3823437815 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa6801a8410 con 0x562892c127b0 2024-05-10T11:43:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.394+0000 7fa685885640 1 --2- 172.21.15.100:0/3823437815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562892c1e720 0x562892c1eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.394+0000 7fa68a00a740 10 client.0 _ll_register_callbacks cb 0x7fa670005ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:43:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.394+0000 7fa685084640 1 -- 172.21.15.100:0/3823437815 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562892c17920 msgr2=0x562892c1e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:43:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.394+0000 7fa685084640 1 --2- 172.21.15.100:0/3823437815 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562892c17920 0x562892c1e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.394+0000 7fa685084640 1 -- 172.21.15.100:0/3823437815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562892c1e720 msgr2=0x562892c1eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:43:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.394+0000 7fa685084640 1 --2- 172.21.15.100:0/3823437815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562892c1e720 0x562892c1eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.394+0000 7fa685084640 1 -- 172.21.15.100:0/3823437815 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562892c135c0 con 0x562892c127b0 2024-05-10T11:43:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.394+0000 7fa685885640 1 --2- 172.21.15.100:0/3823437815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562892c1e720 0x562892c1eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:43:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.394+0000 7fa685084640 1 --2- 172.21.15.100:0/3823437815 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562892c127b0 0x562892c1fb80 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x562892c0c880 tx=0x7fa67c0058e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ccde86131717d12 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:43:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.394+0000 7fa668ff9640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:43:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.394+0000 7fa668ff9640 1 -- 172.21.15.100:0/3823437815 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa67c03f030 con 0x562892c127b0 2024-05-10T11:43:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.395+0000 7fa668ff9640 1 -- 172.21.15.100:0/3823437815 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fa67c0035f0 con 0x562892c127b0 2024-05-10T11:43:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.395+0000 7fa68a00a740 1 -- 172.21.15.100:0/3823437815 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa67c003c80 con 0x562892c127b0 2024-05-10T11:43:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.395+0000 7fa668ff9640 1 -- 172.21.15.100:0/3823437815 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fa67c054040 con 0x562892c127b0 2024-05-10T11:43:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.395+0000 7fa68a00a740 10 client.29605 fetch_fsmap learned FSMap version 41 2024-05-10T11:43:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.395+0000 7fa68a00a740 1 -- 172.21.15.100:0/3823437815 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x562892c81350 con 0x562892c127b0 2024-05-10T11:43:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.395+0000 7fa668ff9640 1 -- 172.21.15.100:0/3823437815 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa67c003c80 con 0x562892c127b0 2024-05-10T11:43:48.401 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.396+0000 7fa668ff9640 1 -- 172.21.15.100:0/3823437815 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fa67c057070 con 0x562892c127b0 2024-05-10T11:43:48.401 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.396+0000 7fa68a00a740 10 client.29605 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:43:48.401 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.396+0000 7fa68a00a740 10 client.29605 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:43:48.401 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.396+0000 7fa68a00a740 20 client.29605 populate_metadata read hostname 'smithi100' 2024-05-10T11:43:48.401 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.396+0000 7fa68a00a740 1 --2- 172.21.15.100:0/3823437815 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562892c81720 0x562892c83b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:48.401 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.396+0000 7fa68a00a740 4 client.29605 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:43:48.401 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.396+0000 7fa68a00a740 1 -- 172.21.15.100:0/3823437815 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562892c02260 con 0x562892c81720 2024-05-10T11:43:48.401 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.396+0000 7fa684883640 1 --2- 172.21.15.100:0/3823437815 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562892c81720 0x562892c83b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:48.401 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.397+0000 7fa684883640 1 --2- 172.21.15.100:0/3823437815 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562892c81720 0x562892c83b10 crc :-1 s=READY pgs=470 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7a47e77a24774c9 server_cookie=9ec3fb502e43452b in_seq=0 out_seq=0 2024-05-10T11:43:48.401 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.397+0000 7fa668ff9640 10 client.29605 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:43:48.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:43:48 smithi178 ceph-mon[33359]: pgmap v1183: 129 pgs: 129 active+clean; 580 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 5.0 MiB/s wr, 290 op/s 2024-05-10T11:43:48.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:43:48 smithi188 ceph-mon[34075]: pgmap v1183: 129 pgs: 129 active+clean; 580 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 5.0 MiB/s wr, 290 op/s 2024-05-10T11:43:48.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:43:48 smithi100 ceph-mon[30296]: pgmap v1183: 129 pgs: 129 active+clean; 580 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 5.0 MiB/s wr, 290 op/s 2024-05-10T11:43:48.577 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.577+0000 7fa668ff9640 1 -- 172.21.15.100:0/3823437815 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562892c02260 con 0x562892c81720 2024-05-10T11:43:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.577+0000 7fa668ff9640 10 client.29605 handle_command_reply: tid=0 2024-05-10T11:43:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.578+0000 7fa68a00a740 1 client.29605 shutdown 2024-05-10T11:43:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.578+0000 7fa68a00a740 1 -- 172.21.15.100:0/3823437815 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562892c127b0 msgr2=0x562892c1fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:48.579 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.578+0000 7fa68a00a740 1 --2- 172.21.15.100:0/3823437815 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562892c127b0 0x562892c1fb80 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x562892c0c880 tx=0x7fa67c0058e0 comp rx=0 tx=0).stop 2024-05-10T11:43:48.579 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.578+0000 7fa68a00a740 1 -- 172.21.15.100:0/3823437815 shutdown_connections 2024-05-10T11:43:48.579 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.578+0000 7fa68a00a740 1 --2- 172.21.15.100:0/3823437815 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x562892c81720 0x562892c83b10 crc :-1 s=READY pgs=470 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:48.579 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.578+0000 7fa68a00a740 1 --2- 172.21.15.100:0/3823437815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562892c1e720 0x562892c1eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:48.579 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.578+0000 7fa68a00a740 1 --2- 172.21.15.100:0/3823437815 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562892c17920 0x562892c1e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:48.579 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.578+0000 7fa68a00a740 1 --2- 172.21.15.100:0/3823437815 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562892c127b0 0x562892c1fb80 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:48.579 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.578+0000 7fa68a00a740 1 -- 172.21.15.100:0/3823437815 >> 172.21.15.100:0/3823437815 conn(0x562892bfc420 msgr2=0x562892c10d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:43:48.579 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.578+0000 7fa668ff9640 0 client.29605 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:43:48.580 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.579+0000 7fa68a00a740 1 -- 172.21.15.100:0/3823437815 shutdown_connections 2024-05-10T11:43:48.580 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.579+0000 7fa68a00a740 1 -- 172.21.15.100:0/3823437815 wait complete. 2024-05-10T11:43:48.580 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.579+0000 7fa68a00a740 20 client.29605 trim_cache size 0 max 16384 2024-05-10T11:43:48.580 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:43:48.580 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:43:48.580 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:43:48.580 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:43:48.581 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.580+0000 7fa6872e8640 1 -- 172.21.15.100:0/21863617 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa66c0b2970 msgr2=0x7fa66c0b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:48.581 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.580+0000 7fa6872e8640 1 --2- 172.21.15.100:0/21863617 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa66c0b2970 0x7fa66c0b4e30 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x562892c13010 tx=0x7fa678009500 comp rx=0 tx=0).stop 2024-05-10T11:43:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.580+0000 7fa6872e8640 1 -- 172.21.15.100:0/21863617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6801045d0 msgr2=0x7fa68019f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.580+0000 7fa6872e8640 1 --2- 172.21.15.100:0/21863617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6801045d0 0x7fa68019f810 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fa67000e9f0 tx=0x7fa67000eeb0 comp rx=0 tx=0).stop 2024-05-10T11:43:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.580+0000 7fa6872e8640 1 -- 172.21.15.100:0/21863617 shutdown_connections 2024-05-10T11:43:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.580+0000 7fa6872e8640 1 --2- 172.21.15.100:0/21863617 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fa66c0b2970 0x7fa66c0b4e30 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.580+0000 7fa6872e8640 1 --2- 172.21.15.100:0/21863617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6801a4870 0x7fa6801a6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.580+0000 7fa6872e8640 1 --2- 172.21.15.100:0/21863617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6801045d0 0x7fa68019f810 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.580+0000 7fa6872e8640 1 --2- 172.21.15.100:0/21863617 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa680103c10 0x7fa68019f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.580+0000 7fa6872e8640 1 -- 172.21.15.100:0/21863617 >> 172.21.15.100:0/21863617 conn(0x7fa6800fdf90 msgr2=0x7fa6801093e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:43:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.581+0000 7fa6872e8640 1 -- 172.21.15.100:0/21863617 shutdown_connections 2024-05-10T11:43:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:43:48.581+0000 7fa6872e8640 1 -- 172.21.15.100:0/21863617 wait complete. 2024-05-10T11:43:48.962 INFO:tasks.workunit.client.0.smithi100.stdout: 10 116209 8.88 MB/sec execute 582 sec latency 140.532 ms 2024-05-10T11:43:49.962 INFO:tasks.workunit.client.0.smithi100.stdout: 10 116389 8.88 MB/sec execute 583 sec latency 81.430 ms 2024-05-10T11:43:50.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:43:50 smithi178 ceph-mon[33359]: pgmap v1184: 129 pgs: 129 active+clean; 582 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 4.2 MiB/s wr, 270 op/s 2024-05-10T11:43:50.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:43:50 smithi188 ceph-mon[34075]: pgmap v1184: 129 pgs: 129 active+clean; 582 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 4.2 MiB/s wr, 270 op/s 2024-05-10T11:43:50.537 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:43:50 smithi100 ceph-mon[30296]: pgmap v1184: 129 pgs: 129 active+clean; 582 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 4.2 MiB/s wr, 270 op/s 2024-05-10T11:43:50.962 INFO:tasks.workunit.client.0.smithi100.stdout: 10 116557 8.88 MB/sec execute 584 sec latency 79.439 ms 2024-05-10T11:43:51.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:43:51 smithi178 ceph-mon[33359]: pgmap v1185: 129 pgs: 129 active+clean; 584 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 929 KiB/s rd, 4.2 MiB/s wr, 284 op/s 2024-05-10T11:43:51.778 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:43:51 smithi188 ceph-mon[34075]: pgmap v1185: 129 pgs: 129 active+clean; 584 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 929 KiB/s rd, 4.2 MiB/s wr, 284 op/s 2024-05-10T11:43:51.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:43:51 smithi100 ceph-mon[30296]: pgmap v1185: 129 pgs: 129 active+clean; 584 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 929 KiB/s rd, 4.2 MiB/s wr, 284 op/s 2024-05-10T11:43:51.962 INFO:tasks.workunit.client.0.smithi100.stdout: 10 116775 8.90 MB/sec execute 585 sec latency 99.816 ms 2024-05-10T11:43:52.962 INFO:tasks.workunit.client.0.smithi100.stdout: 10 116961 8.90 MB/sec execute 586 sec latency 84.299 ms 2024-05-10T11:43:53.962 INFO:tasks.workunit.client.0.smithi100.stdout: 10 117097 8.90 MB/sec execute 587 sec latency 126.077 ms 2024-05-10T11:43:54.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:43:53 smithi178 ceph-mon[33359]: pgmap v1186: 129 pgs: 129 active+clean; 586 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 812 KiB/s rd, 3.9 MiB/s wr, 271 op/s 2024-05-10T11:43:54.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:43:53 smithi188 ceph-mon[34075]: pgmap v1186: 129 pgs: 129 active+clean; 586 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 812 KiB/s rd, 3.9 MiB/s wr, 271 op/s 2024-05-10T11:43:54.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:43:53 smithi100 ceph-mon[30296]: pgmap v1186: 129 pgs: 129 active+clean; 586 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 812 KiB/s rd, 3.9 MiB/s wr, 271 op/s 2024-05-10T11:43:54.962 INFO:tasks.workunit.client.0.smithi100.stdout: 10 117260 8.90 MB/sec execute 588 sec latency 100.381 ms 2024-05-10T11:43:55.147 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-10T11:43:55.878 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:43:55.962 INFO:tasks.workunit.client.0.smithi100.stdout: 10 117393 8.90 MB/sec execute 589 sec latency 557.826 ms 2024-05-10T11:43:56.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:43:55 smithi178 ceph-mon[33359]: pgmap v1187: 129 pgs: 129 active+clean; 586 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 4.3 MiB/s wr, 285 op/s 2024-05-10T11:43:56.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:43:55 smithi188 ceph-mon[34075]: pgmap v1187: 129 pgs: 129 active+clean; 586 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 4.3 MiB/s wr, 285 op/s 2024-05-10T11:43:56.031 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:43:55 smithi100 ceph-mon[30296]: pgmap v1187: 129 pgs: 129 active+clean; 586 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 4.3 MiB/s wr, 285 op/s 2024-05-10T11:43:56.962 INFO:tasks.workunit.client.0.smithi100.stdout: 10 117516 8.89 MB/sec execute 590 sec latency 558.867 ms 2024-05-10T11:43:57.962 INFO:tasks.workunit.client.0.smithi100.stdout: 10 117655 8.89 MB/sec execute 591 sec latency 167.222 ms 2024-05-10T11:43:58.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:43:57 smithi178 ceph-mon[33359]: pgmap v1188: 129 pgs: 129 active+clean; 590 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 4.6 MiB/s wr, 302 op/s 2024-05-10T11:43:58.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:43:57 smithi188 ceph-mon[34075]: pgmap v1188: 129 pgs: 129 active+clean; 590 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 4.6 MiB/s wr, 302 op/s 2024-05-10T11:43:58.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:43:57 smithi100 ceph-mon[30296]: pgmap v1188: 129 pgs: 129 active+clean; 590 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 4.6 MiB/s wr, 302 op/s 2024-05-10T11:43:58.962 INFO:tasks.workunit.client.0.smithi100.stdout: 10 117824 8.89 MB/sec execute 592 sec latency 97.449 ms 2024-05-10T11:43:59.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.091+0000 7f0b7aeb2700 1 -- 172.21.15.100:0/445535462 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b74108880 msgr2=0x7f0b74110400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:59.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.091+0000 7f0b7aeb2700 1 --2- 172.21.15.100:0/445535462 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b74108880 0x7f0b74110400 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c009f90 tx=0x7f0b6c00b330 comp rx=0 tx=0).stop 2024-05-10T11:43:59.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.091+0000 7f0b7aeb2700 1 -- 172.21.15.100:0/445535462 shutdown_connections 2024-05-10T11:43:59.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.091+0000 7f0b7aeb2700 1 --2- 172.21.15.100:0/445535462 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b74108880 0x7f0b74110400 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:59.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.091+0000 7f0b7aeb2700 1 --2- 172.21.15.100:0/445535462 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b741037b0 0x7f0b74103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:59.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.091+0000 7f0b7aeb2700 1 --2- 172.21.15.100:0/445535462 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b74104160 0x7f0b74108150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:59.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.091+0000 7f0b7aeb2700 1 -- 172.21.15.100:0/445535462 >> 172.21.15.100:0/445535462 conn(0x7f0b7407a530 msgr2=0x7f0b7407a940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:43:59.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.092+0000 7f0b7aeb2700 1 -- 172.21.15.100:0/445535462 shutdown_connections 2024-05-10T11:43:59.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.092+0000 7f0b7aeb2700 1 -- 172.21.15.100:0/445535462 wait complete. 2024-05-10T11:43:59.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.092+0000 7f0b7aeb2700 1 Processor -- start 2024-05-10T11:43:59.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.093+0000 7f0b7aeb2700 1 -- start start 2024-05-10T11:43:59.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.093+0000 7f0b7aeb2700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b741037b0 0x7f0b7419f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:59.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.093+0000 7f0b7aeb2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b74104160 0x7f0b7419fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:59.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.093+0000 7f0b7aeb2700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b74108880 0x7f0b741997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:59.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.093+0000 7f0b78c4e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b741037b0 0x7f0b7419f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:59.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.093+0000 7f0b7aeb2700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0b74113520 con 0x7f0b74104160 2024-05-10T11:43:59.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.093+0000 7f0b73fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b74104160 0x7f0b7419fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:59.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.093+0000 7f0b73fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b74104160 0x7f0b7419fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43560/0 (socket says 172.21.15.100:43560) 2024-05-10T11:43:59.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.093+0000 7f0b78c4e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b741037b0 0x7f0b7419f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:43552/0 (socket says 172.21.15.100:43552) 2024-05-10T11:43:59.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.093+0000 7f0b73fff700 1 -- 172.21.15.100:0/1579148829 learned_addr learned my addr 172.21.15.100:0/1579148829 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:43:59.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.093+0000 7f0b7aeb2700 1 -- 172.21.15.100:0/1579148829 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0b741133a0 con 0x7f0b741037b0 2024-05-10T11:43:59.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.093+0000 7f0b7aeb2700 1 -- 172.21.15.100:0/1579148829 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0b741136a0 con 0x7f0b74108880 2024-05-10T11:43:59.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.093+0000 7f0b7944f700 1 --2- 172.21.15.100:0/1579148829 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b74108880 0x7f0b741997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:59.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.093+0000 7f0b73fff700 1 -- 172.21.15.100:0/1579148829 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b74108880 msgr2=0x7f0b741997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:59.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.094+0000 7f0b73fff700 1 --2- 172.21.15.100:0/1579148829 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b74108880 0x7f0b741997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:59.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.094+0000 7f0b73fff700 1 -- 172.21.15.100:0/1579148829 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b741037b0 msgr2=0x7f0b7419f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:59.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.094+0000 7f0b73fff700 1 --2- 172.21.15.100:0/1579148829 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b741037b0 0x7f0b7419f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:59.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.094+0000 7f0b73fff700 1 -- 172.21.15.100:0/1579148829 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b74199fa0 con 0x7f0b74104160 2024-05-10T11:43:59.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.094+0000 7f0b78c4e700 1 --2- 172.21.15.100:0/1579148829 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b741037b0 0x7f0b7419f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:43:59.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.094+0000 7f0b73fff700 1 --2- 172.21.15.100:0/1579148829 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b74104160 0x7f0b7419fb80 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f0b6000ec10 tx=0x7f0b6000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:43:59.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.094+0000 7f0b71ffb700 1 -- 172.21.15.100:0/1579148829 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6000cd40 con 0x7f0b74104160 2024-05-10T11:43:59.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.094+0000 7f0b71ffb700 1 -- 172.21.15.100:0/1579148829 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0b600044b0 con 0x7f0b74104160 2024-05-10T11:43:59.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.094+0000 7f0b7aeb2700 1 -- 172.21.15.100:0/1579148829 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b7419a290 con 0x7f0b74104160 2024-05-10T11:43:59.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.094+0000 7f0b71ffb700 1 -- 172.21.15.100:0/1579148829 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b60010430 con 0x7f0b74104160 2024-05-10T11:43:59.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.094+0000 7f0b7aeb2700 1 -- 172.21.15.100:0/1579148829 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b740756d0 con 0x7f0b74104160 2024-05-10T11:43:59.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.097+0000 7f0b71ffb700 1 -- 172.21.15.100:0/1579148829 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f0b600105d0 con 0x7f0b74104160 2024-05-10T11:43:59.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.098+0000 7f0b71ffb700 1 --2- 172.21.15.100:0/1579148829 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0b640b2900 0x7f0b640b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:59.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.098+0000 7f0b71ffb700 1 -- 172.21.15.100:0/1579148829 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f0b600e3860 con 0x7f0b74104160 2024-05-10T11:43:59.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.098+0000 7f0b78c4e700 1 --2- 172.21.15.100:0/1579148829 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0b640b2900 0x7f0b640b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:59.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.099+0000 7f0b78c4e700 1 --2- 172.21.15.100:0/1579148829 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0b640b2900 0x7f0b640b4dc0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f0b68011490 tx=0x7f0b68009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:43:59.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.099+0000 7f0b85745b80 1 Processor -- start 2024-05-10T11:43:59.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.099+0000 7f0b85745b80 1 -- start start 2024-05-10T11:43:59.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.100+0000 7f0b85745b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef6dccb4d0 0x55ef6dccb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:59.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.100+0000 7f0b85745b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ef6dccbdf0 0x55ef6dccf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:59.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.100+0000 7f0b73fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ef6dccbdf0 0x55ef6dccf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:59.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.100+0000 7f0b73fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ef6dccbdf0 0x55ef6dccf6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43562/0 (socket says 172.21.15.100:43562) 2024-05-10T11:43:59.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.100+0000 7f0b78c4e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef6dccb4d0 0x55ef6dccb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:59.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.100+0000 7f0b78c4e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef6dccb4d0 0x55ef6dccb8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:43556/0 (socket says 172.21.15.100:43556) 2024-05-10T11:43:59.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.100+0000 7f0b85745b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef6dccfc80 0x55ef6dcd0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:59.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.100+0000 7f0b73fff700 1 -- 172.21.15.100:0/3635805007 learned_addr learned my addr 172.21.15.100:0/3635805007 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:43:59.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.100+0000 7f0b85745b80 1 -- 172.21.15.100:0/3635805007 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55ef6dc1d860 con 0x55ef6dccbdf0 2024-05-10T11:43:59.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.100+0000 7f0b85745b80 1 -- 172.21.15.100:0/3635805007 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ef6dbbc7c0 con 0x55ef6dccb4d0 2024-05-10T11:43:59.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.100+0000 7f0b85745b80 1 -- 172.21.15.100:0/3635805007 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55ef6dc19820 con 0x55ef6dccfc80 2024-05-10T11:43:59.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.100+0000 7f0b7944f700 1 --2- 172.21.15.100:0/3635805007 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef6dccfc80 0x55ef6dcd0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:59.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.101+0000 7f0b73fff700 1 -- 172.21.15.100:0/3635805007 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef6dccfc80 msgr2=0x55ef6dcd0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:43:59.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.101+0000 7f0b73fff700 1 --2- 172.21.15.100:0/3635805007 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef6dccfc80 0x55ef6dcd0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:59.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.101+0000 7f0b73fff700 1 -- 172.21.15.100:0/3635805007 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef6dccb4d0 msgr2=0x55ef6dccb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:43:59.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.101+0000 7f0b73fff700 1 --2- 172.21.15.100:0/3635805007 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef6dccb4d0 0x55ef6dccb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:59.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.101+0000 7f0b73fff700 1 -- 172.21.15.100:0/3635805007 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef6db6a8d0 con 0x55ef6dccbdf0 2024-05-10T11:43:59.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.101+0000 7f0b73fff700 1 --2- 172.21.15.100:0/3635805007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ef6dccbdf0 0x55ef6dccf6f0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f0b600aa7c0 tx=0x7f0b60004660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25a6236ee628735b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:43:59.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.101+0000 7f0b5effd700 1 -- 172.21.15.100:0/3635805007 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b600aa230 con 0x55ef6dccbdf0 2024-05-10T11:43:59.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.101+0000 7f0b5effd700 1 -- 172.21.15.100:0/3635805007 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0b600b3270 con 0x55ef6dccbdf0 2024-05-10T11:43:59.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.102+0000 7f0b5effd700 1 -- 172.21.15.100:0/3635805007 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b600a6250 con 0x55ef6dccbdf0 2024-05-10T11:43:59.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.102+0000 7f0b85745b80 1 -- 172.21.15.100:0/3635805007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ef6dccbdf0 msgr2=0x55ef6dccf6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:59.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.102+0000 7f0b85745b80 1 --2- 172.21.15.100:0/3635805007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ef6dccbdf0 0x55ef6dccf6f0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f0b600aa7c0 tx=0x7f0b60004660 comp rx=0 tx=0).stop 2024-05-10T11:43:59.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.103+0000 7f0b85745b80 1 -- 172.21.15.100:0/3635805007 shutdown_connections 2024-05-10T11:43:59.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.103+0000 7f0b85745b80 1 --2- 172.21.15.100:0/3635805007 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef6dccfc80 0x55ef6dcd0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:59.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.103+0000 7f0b85745b80 1 --2- 172.21.15.100:0/3635805007 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef6dccb4d0 0x55ef6dccb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:59.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.103+0000 7f0b85745b80 1 --2- 172.21.15.100:0/3635805007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ef6dccbdf0 0x55ef6dccf6f0 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:59.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.103+0000 7f0b85745b80 1 -- 172.21.15.100:0/3635805007 >> 172.21.15.100:0/3635805007 conn(0x55ef6dbf7100 msgr2=0x55ef6dcc5c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:43:59.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.103+0000 7f0b85745b80 1 -- 172.21.15.100:0/3635805007 shutdown_connections 2024-05-10T11:43:59.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.103+0000 7f0b85745b80 1 -- 172.21.15.100:0/3635805007 wait complete. 2024-05-10T11:43:59.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.103+0000 7f0b85745b80 1 -- start start 2024-05-10T11:43:59.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.104+0000 7f0b85745b80 1 Processor -- start 2024-05-10T11:43:59.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.105+0000 7f0b85745b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ef6dccb4d0 0x55ef6dcd6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:59.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.105+0000 7f0b85745b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef6dccbdf0 0x55ef6dcd4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:59.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.105+0000 7f0b78c4e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ef6dccb4d0 0x55ef6dcd6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:59.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.105+0000 7f0b78c4e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ef6dccb4d0 0x55ef6dcd6fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43566/0 (socket says 172.21.15.100:43566) 2024-05-10T11:43:59.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.105+0000 7f0b78c4e700 1 -- 172.21.15.100:0/1471640849 learned_addr learned my addr 172.21.15.100:0/1471640849 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:43:59.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.105+0000 7f0b85745b80 1 --2- 172.21.15.100:0/1471640849 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef6dccfc80 0x55ef6dcd48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:59.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.105+0000 7f0b73fff700 1 --2- 172.21.15.100:0/1471640849 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef6dccbdf0 0x55ef6dcd4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:59.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.105+0000 7f0b85745b80 1 -- 172.21.15.100:0/1471640849 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55ef6dcd7f30 con 0x55ef6dccb4d0 2024-05-10T11:43:59.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.105+0000 7f0b85745b80 1 -- 172.21.15.100:0/1471640849 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ef6db83c30 con 0x55ef6dccbdf0 2024-05-10T11:43:59.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.105+0000 7f0b85745b80 1 -- 172.21.15.100:0/1471640849 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55ef6dcd80b0 con 0x55ef6dccfc80 2024-05-10T11:43:59.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.105+0000 7f0b7944f700 1 --2- 172.21.15.100:0/1471640849 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef6dccfc80 0x55ef6dcd48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:59.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.105+0000 7f0b85745b80 10 client.0 _ll_register_callbacks cb 0x55ef6dbb5920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:43:59.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.106+0000 7f0b73fff700 1 -- 172.21.15.100:0/1471640849 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef6dccfc80 msgr2=0x55ef6dcd48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:43:59.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.106+0000 7f0b73fff700 1 --2- 172.21.15.100:0/1471640849 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef6dccfc80 0x55ef6dcd48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:59.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.106+0000 7f0b73fff700 1 -- 172.21.15.100:0/1471640849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ef6dccb4d0 msgr2=0x55ef6dcd6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:43:59.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.106+0000 7f0b73fff700 1 --2- 172.21.15.100:0/1471640849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ef6dccb4d0 0x55ef6dcd6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:59.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.106+0000 7f0b73fff700 1 -- 172.21.15.100:0/1471640849 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef6dbb9690 con 0x55ef6dccbdf0 2024-05-10T11:43:59.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.106+0000 7f0b7944f700 1 --2- 172.21.15.100:0/1471640849 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef6dccfc80 0x55ef6dcd48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:43:59.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.106+0000 7f0b78c4e700 1 --2- 172.21.15.100:0/1471640849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ef6dccb4d0 0x55ef6dcd6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:43:59.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.106+0000 7f0b73fff700 1 --2- 172.21.15.100:0/1471640849 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef6dccbdf0 0x55ef6dcd4390 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0b600aa7c0 tx=0x7f0b600aa5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2d3fdfaac415e88 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:43:59.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.106+0000 7f0b53fff700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:43:59.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.106+0000 7f0b53fff700 1 -- 172.21.15.100:0/1471640849 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b600c5050 con 0x55ef6dccbdf0 2024-05-10T11:43:59.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.106+0000 7f0b85745b80 1 -- 172.21.15.100:0/1471640849 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b600b3270 con 0x55ef6dccbdf0 2024-05-10T11:43:59.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.107+0000 7f0b53fff700 1 -- 172.21.15.100:0/1471640849 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f0b60003710 con 0x55ef6dccbdf0 2024-05-10T11:43:59.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.107+0000 7f0b53fff700 1 -- 172.21.15.100:0/1471640849 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f0b600aa230 con 0x55ef6dccbdf0 2024-05-10T11:43:59.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.107+0000 7f0b85745b80 10 client.29492 fetch_fsmap learned FSMap version 41 2024-05-10T11:43:59.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.107+0000 7f0b85745b80 1 -- 172.21.15.100:0/1471640849 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55ef6dce2680 con 0x55ef6dccbdf0 2024-05-10T11:43:59.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.107+0000 7f0b53fff700 1 -- 172.21.15.100:0/1471640849 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0b600b3270 con 0x55ef6dccbdf0 2024-05-10T11:43:59.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.107+0000 7f0b53fff700 1 -- 172.21.15.100:0/1471640849 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f0b600b3490 con 0x55ef6dccbdf0 2024-05-10T11:43:59.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.108+0000 7f0b85745b80 10 client.29492 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:43:59.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.108+0000 7f0b85745b80 10 client.29492 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:43:59.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.108+0000 7f0b85745b80 20 client.29492 populate_metadata read hostname 'smithi100' 2024-05-10T11:43:59.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.108+0000 7f0b85745b80 1 --2- 172.21.15.100:0/1471640849 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ef6dce4b50 0x55ef6dce6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:59.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.108+0000 7f0b85745b80 4 client.29492 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:43:59.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.108+0000 7f0b85745b80 1 -- 172.21.15.100:0/1471640849 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef6dc97650 con 0x55ef6dce4b50 2024-05-10T11:43:59.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.108+0000 7f0b78c4e700 1 --2- 172.21.15.100:0/1471640849 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ef6dce4b50 0x55ef6dce6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:59.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.109+0000 7f0b53fff700 10 client.29492 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:43:59.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.109+0000 7f0b78c4e700 1 --2- 172.21.15.100:0/1471640849 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ef6dce4b50 0x55ef6dce6f40 crc :-1 s=READY pgs=471 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25a690b49e32db98 server_cookie=21ff4948f64beea0 in_seq=0 out_seq=0 2024-05-10T11:43:59.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.110+0000 7f0b53fff700 1 -- 172.21.15.100:0/1471640849 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ef6dc97650 con 0x55ef6dce4b50 2024-05-10T11:43:59.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.110+0000 7f0b53fff700 10 client.29492 handle_command_reply: tid=0 2024-05-10T11:43:59.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.110+0000 7f0b85745b80 1 client.29492 shutdown 2024-05-10T11:43:59.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.110+0000 7f0b85745b80 1 -- 172.21.15.100:0/1471640849 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef6dccbdf0 msgr2=0x55ef6dcd4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:59.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.110+0000 7f0b85745b80 1 --2- 172.21.15.100:0/1471640849 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef6dccbdf0 0x55ef6dcd4390 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0b600aa7c0 tx=0x7f0b600aa5d0 comp rx=0 tx=0).stop 2024-05-10T11:43:59.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.110+0000 7f0b85745b80 1 -- 172.21.15.100:0/1471640849 shutdown_connections 2024-05-10T11:43:59.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.110+0000 7f0b85745b80 1 --2- 172.21.15.100:0/1471640849 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ef6dce4b50 0x55ef6dce6f40 crc :-1 s=READY pgs=471 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:59.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.110+0000 7f0b85745b80 1 --2- 172.21.15.100:0/1471640849 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef6dccfc80 0x55ef6dcd48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:59.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.110+0000 7f0b85745b80 1 --2- 172.21.15.100:0/1471640849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ef6dccb4d0 0x55ef6dcd6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:59.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.110+0000 7f0b85745b80 1 --2- 172.21.15.100:0/1471640849 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef6dccbdf0 0x55ef6dcd4390 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:59.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.110+0000 7f0b85745b80 1 -- 172.21.15.100:0/1471640849 >> 172.21.15.100:0/1471640849 conn(0x55ef6dbf7100 msgr2=0x55ef6dcc0720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:43:59.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.110+0000 7f0b53fff700 0 client.29492 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:43:59.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.110+0000 7f0b85745b80 1 -- 172.21.15.100:0/1471640849 shutdown_connections 2024-05-10T11:43:59.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.110+0000 7f0b85745b80 1 -- 172.21.15.100:0/1471640849 wait complete. 2024-05-10T11:43:59.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.110+0000 7f0b85745b80 20 client.29492 trim_cache size 0 max 16384 2024-05-10T11:43:59.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.141+0000 7f0b85745b80 1 Processor -- start 2024-05-10T11:43:59.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.141+0000 7f0b85745b80 1 -- start start 2024-05-10T11:43:59.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.141+0000 7f0b85745b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef6dccb4d0 0x55ef6dccd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:59.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.141+0000 7f0b85745b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef6dccbdf0 0x55ef6dccdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:59.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.141+0000 7f0b85745b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ef6dccfc80 0x55ef6dcd2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:59.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.141+0000 7f0b85745b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0b741b3b70 con 0x55ef6dccfc80 2024-05-10T11:43:59.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.141+0000 7f0b85745b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0b741b1400 con 0x55ef6dccb4d0 2024-05-10T11:43:59.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.141+0000 7f0b85745b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0b741af250 con 0x55ef6dccbdf0 2024-05-10T11:43:59.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.141+0000 7f0b7944f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ef6dccfc80 0x55ef6dcd2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:59.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.141+0000 7f0b7944f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ef6dccfc80 0x55ef6dcd2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43582/0 (socket says 172.21.15.100:43582) 2024-05-10T11:43:59.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.141+0000 7f0b7944f700 1 -- 172.21.15.100:0/2905130053 learned_addr learned my addr 172.21.15.100:0/2905130053 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:43:59.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.141+0000 7f0b78c4e700 1 --2- 172.21.15.100:0/2905130053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef6dccb4d0 0x55ef6dccd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:59.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.141+0000 7f0b73fff700 1 --2- 172.21.15.100:0/2905130053 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef6dccbdf0 0x55ef6dccdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:59.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.142+0000 7f0b7944f700 1 -- 172.21.15.100:0/2905130053 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef6dccbdf0 msgr2=0x55ef6dccdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:43:59.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.142+0000 7f0b7944f700 1 --2- 172.21.15.100:0/2905130053 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef6dccbdf0 0x55ef6dccdb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:59.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.142+0000 7f0b7944f700 1 -- 172.21.15.100:0/2905130053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef6dccb4d0 msgr2=0x55ef6dccd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:43:59.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.142+0000 7f0b7944f700 1 --2- 172.21.15.100:0/2905130053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef6dccb4d0 0x55ef6dccd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:59.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.142+0000 7f0b7944f700 1 -- 172.21.15.100:0/2905130053 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef6dcd7d70 con 0x55ef6dccfc80 2024-05-10T11:43:59.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.142+0000 7f0b7944f700 1 --2- 172.21.15.100:0/2905130053 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ef6dccfc80 0x55ef6dcd2c80 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c003c40 tx=0x7f0b6c003c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea10a6c2ae297df3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:43:59.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.142+0000 7f0b5e7fc700 1 -- 172.21.15.100:0/2905130053 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c00bbf0 con 0x55ef6dccfc80 2024-05-10T11:43:59.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.142+0000 7f0b5e7fc700 1 -- 172.21.15.100:0/2905130053 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0b6c00bd90 con 0x55ef6dccfc80 2024-05-10T11:43:59.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.142+0000 7f0b5e7fc700 1 -- 172.21.15.100:0/2905130053 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c005520 con 0x55ef6dccfc80 2024-05-10T11:43:59.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.143+0000 7f0b85745b80 1 -- 172.21.15.100:0/2905130053 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ef6dccfc80 msgr2=0x55ef6dcd2c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:59.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.143+0000 7f0b85745b80 1 --2- 172.21.15.100:0/2905130053 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ef6dccfc80 0x55ef6dcd2c80 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c003c40 tx=0x7f0b6c003c70 comp rx=0 tx=0).stop 2024-05-10T11:43:59.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.143+0000 7f0b85745b80 1 -- 172.21.15.100:0/2905130053 shutdown_connections 2024-05-10T11:43:59.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.143+0000 7f0b85745b80 1 --2- 172.21.15.100:0/2905130053 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ef6dccfc80 0x55ef6dcd2c80 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:59.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.143+0000 7f0b85745b80 1 --2- 172.21.15.100:0/2905130053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef6dccb4d0 0x55ef6dccd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:59.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.143+0000 7f0b85745b80 1 --2- 172.21.15.100:0/2905130053 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef6dccbdf0 0x55ef6dccdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:59.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.143+0000 7f0b85745b80 1 -- 172.21.15.100:0/2905130053 >> 172.21.15.100:0/2905130053 conn(0x55ef6dbf7100 msgr2=0x55ef6dcc5b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:43:59.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.143+0000 7f0b85745b80 1 -- 172.21.15.100:0/2905130053 shutdown_connections 2024-05-10T11:43:59.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.143+0000 7f0b85745b80 1 -- 172.21.15.100:0/2905130053 wait complete. 2024-05-10T11:43:59.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.144+0000 7f0b85745b80 1 -- start start 2024-05-10T11:43:59.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.144+0000 7f0b85745b80 1 Processor -- start 2024-05-10T11:43:59.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.144+0000 7f0b85745b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef6dccb4d0 0x55ef6dce6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:59.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.144+0000 7f0b85745b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef6dccbdf0 0x55ef6dce6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:59.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.144+0000 7f0b85745b80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ef6dccfc80 0x55ef6dd4d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:59.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.144+0000 7f0b85745b80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0b741af8c0 con 0x55ef6dccfc80 2024-05-10T11:43:59.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.144+0000 7f0b85745b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0b741b0d30 con 0x55ef6dccbdf0 2024-05-10T11:43:59.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.144+0000 7f0b85745b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0b741aff90 con 0x55ef6dccb4d0 2024-05-10T11:43:59.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.144+0000 7f0b85745b80 10 client.0 _ll_register_callbacks cb 0x55ef6dbb5920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:43:59.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.144+0000 7f0b7944f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ef6dccfc80 0x55ef6dd4d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:59.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.144+0000 7f0b7944f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ef6dccfc80 0x55ef6dd4d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43590/0 (socket says 172.21.15.100:43590) 2024-05-10T11:43:59.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.144+0000 7f0b7944f700 1 -- 172.21.15.100:0/1998250915 learned_addr learned my addr 172.21.15.100:0/1998250915 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:43:59.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.145+0000 7f0b73fff700 1 --2- 172.21.15.100:0/1998250915 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef6dccbdf0 0x55ef6dce6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:59.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.145+0000 7f0b7944f700 1 -- 172.21.15.100:0/1998250915 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef6dccb4d0 msgr2=0x55ef6dce6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:43:59.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.145+0000 7f0b7944f700 1 --2- 172.21.15.100:0/1998250915 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef6dccb4d0 0x55ef6dce6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:59.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.145+0000 7f0b7944f700 1 -- 172.21.15.100:0/1998250915 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef6dccbdf0 msgr2=0x55ef6dce6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:43:59.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.145+0000 7f0b7944f700 1 --2- 172.21.15.100:0/1998250915 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef6dccbdf0 0x55ef6dce6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:59.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.145+0000 7f0b7944f700 1 -- 172.21.15.100:0/1998250915 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef6dce7830 con 0x55ef6dccfc80 2024-05-10T11:43:59.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.145+0000 7f0b73fff700 1 --2- 172.21.15.100:0/1998250915 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef6dccbdf0 0x55ef6dce6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:43:59.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.145+0000 7f0b7944f700 1 --2- 172.21.15.100:0/1998250915 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ef6dccfc80 0x55ef6dd4d060 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c009f90 tx=0x7f0b6c0076c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f8c995d3120e84 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:43:59.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.145+0000 7f0b53fff700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:43:59.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.145+0000 7f0b53fff700 1 -- 172.21.15.100:0/1998250915 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c0092e0 con 0x55ef6dccfc80 2024-05-10T11:43:59.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.145+0000 7f0b53fff700 1 -- 172.21.15.100:0/1998250915 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f0b6c0050e0 con 0x55ef6dccfc80 2024-05-10T11:43:59.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.145+0000 7f0b85745b80 1 -- 172.21.15.100:0/1998250915 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b6c00bd90 con 0x55ef6dccfc80 2024-05-10T11:43:59.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.146+0000 7f0b53fff700 1 -- 172.21.15.100:0/1998250915 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f0b6c0052e0 con 0x55ef6dccfc80 2024-05-10T11:43:59.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.146+0000 7f0b85745b80 10 client.20259 fetch_fsmap learned FSMap version 41 2024-05-10T11:43:59.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.146+0000 7f0b85745b80 1 -- 172.21.15.100:0/1998250915 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55ef6dd4e680 con 0x55ef6dccfc80 2024-05-10T11:43:59.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.146+0000 7f0b53fff700 1 -- 172.21.15.100:0/1998250915 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0b6c00bd90 con 0x55ef6dccfc80 2024-05-10T11:43:59.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.146+0000 7f0b53fff700 1 -- 172.21.15.100:0/1998250915 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f0b6c02e070 con 0x55ef6dccfc80 2024-05-10T11:43:59.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.146+0000 7f0b85745b80 10 client.20259 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:43:59.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.146+0000 7f0b85745b80 10 client.20259 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:43:59.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.146+0000 7f0b85745b80 20 client.20259 populate_metadata read hostname 'smithi100' 2024-05-10T11:43:59.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.146+0000 7f0b85745b80 1 --2- 172.21.15.100:0/1998250915 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ef6dd4eca0 0x55ef6dd510e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:43:59.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.146+0000 7f0b85745b80 4 client.20259 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T11:43:59.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.146+0000 7f0b85745b80 1 -- 172.21.15.100:0/1998250915 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ef6dc2b240 con 0x55ef6dd4eca0 2024-05-10T11:43:59.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.146+0000 7f0b78c4e700 1 --2- 172.21.15.100:0/1998250915 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ef6dd4eca0 0x55ef6dd510e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:43:59.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.147+0000 7f0b53fff700 10 client.20259 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:43:59.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.147+0000 7f0b78c4e700 1 --2- 172.21.15.100:0/1998250915 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ef6dd4eca0 0x55ef6dd510e0 crc :-1 s=READY pgs=472 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7db1096ded32c7e server_cookie=60351536f934b02e in_seq=0 out_seq=0 2024-05-10T11:43:59.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.147+0000 7f0b53fff700 1 -- 172.21.15.100:0/1998250915 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ef6dc2b240 con 0x55ef6dd4eca0 2024-05-10T11:43:59.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.147+0000 7f0b53fff700 10 client.20259 handle_command_reply: tid=0 2024-05-10T11:43:59.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.147+0000 7f0b85745b80 1 client.20259 shutdown 2024-05-10T11:43:59.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.148+0000 7f0b85745b80 1 -- 172.21.15.100:0/1998250915 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ef6dccfc80 msgr2=0x55ef6dd4d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:59.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.148+0000 7f0b85745b80 1 --2- 172.21.15.100:0/1998250915 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ef6dccfc80 0x55ef6dd4d060 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c009f90 tx=0x7f0b6c0076c0 comp rx=0 tx=0).stop 2024-05-10T11:43:59.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.148+0000 7f0b85745b80 1 -- 172.21.15.100:0/1998250915 shutdown_connections 2024-05-10T11:43:59.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.148+0000 7f0b85745b80 1 --2- 172.21.15.100:0/1998250915 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55ef6dd4eca0 0x55ef6dd510e0 crc :-1 s=READY pgs=472 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:59.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.148+0000 7f0b85745b80 1 --2- 172.21.15.100:0/1998250915 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ef6dccfc80 0x55ef6dd4d060 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:59.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.148+0000 7f0b85745b80 1 --2- 172.21.15.100:0/1998250915 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef6dccb4d0 0x55ef6dce6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:59.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.148+0000 7f0b85745b80 1 --2- 172.21.15.100:0/1998250915 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef6dccbdf0 0x55ef6dce6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:59.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.148+0000 7f0b85745b80 1 -- 172.21.15.100:0/1998250915 >> 172.21.15.100:0/1998250915 conn(0x55ef6dbf7100 msgr2=0x55ef6dcce520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:43:59.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.148+0000 7f0b53fff700 0 client.20259 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:43:59.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.148+0000 7f0b85745b80 1 -- 172.21.15.100:0/1998250915 shutdown_connections 2024-05-10T11:43:59.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.148+0000 7f0b85745b80 1 -- 172.21.15.100:0/1998250915 wait complete. 2024-05-10T11:43:59.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.148+0000 7f0b85745b80 20 client.20259 trim_cache size 0 max 16384 2024-05-10T11:43:59.155 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:43:59.155 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-05-10T11:43:59.155 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-05-10T11:43:59.155 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:43:59.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.149+0000 7f0b7aeb2700 1 -- 172.21.15.100:0/1579148829 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0b640b2900 msgr2=0x7f0b640b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:59.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.149+0000 7f0b7aeb2700 1 --2- 172.21.15.100:0/1579148829 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0b640b2900 0x7f0b640b4dc0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f0b68011490 tx=0x7f0b68009260 comp rx=0 tx=0).stop 2024-05-10T11:43:59.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.149+0000 7f0b7aeb2700 1 -- 172.21.15.100:0/1579148829 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b74104160 msgr2=0x7f0b7419fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:43:59.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.149+0000 7f0b7aeb2700 1 --2- 172.21.15.100:0/1579148829 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b74104160 0x7f0b7419fb80 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f0b6000ec10 tx=0x7f0b6000ef20 comp rx=0 tx=0).stop 2024-05-10T11:43:59.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.149+0000 7f0b7aeb2700 1 -- 172.21.15.100:0/1579148829 shutdown_connections 2024-05-10T11:43:59.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.149+0000 7f0b7aeb2700 1 --2- 172.21.15.100:0/1579148829 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0b640b2900 0x7f0b640b4dc0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:59.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.149+0000 7f0b7aeb2700 1 --2- 172.21.15.100:0/1579148829 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b74108880 0x7f0b741997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:59.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.149+0000 7f0b7aeb2700 1 --2- 172.21.15.100:0/1579148829 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b741037b0 0x7f0b7419f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:59.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.149+0000 7f0b7aeb2700 1 --2- 172.21.15.100:0/1579148829 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b74104160 0x7f0b7419fb80 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:43:59.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.149+0000 7f0b7aeb2700 1 -- 172.21.15.100:0/1579148829 >> 172.21.15.100:0/1579148829 conn(0x7f0b7407a530 msgr2=0x7f0b74101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:43:59.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.150+0000 7f0b7aeb2700 1 -- 172.21.15.100:0/1579148829 shutdown_connections 2024-05-10T11:43:59.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:43:59.150+0000 7f0b7aeb2700 1 -- 172.21.15.100:0/1579148829 wait complete. 2024-05-10T11:43:59.788 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:43:59 smithi100 ceph-mon[30296]: pgmap v1189: 129 pgs: 129 active+clean; 589 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 4.2 MiB/s wr, 283 op/s 2024-05-10T11:43:59.947 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T11:43:59.947 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T11:43:59.963 INFO:tasks.workunit.client.0.smithi100.stdout: 10 117954 8.88 MB/sec execute 593 sec latency 137.476 ms 2024-05-10T11:44:00.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:43:59 smithi178 ceph-mon[33359]: pgmap v1189: 129 pgs: 129 active+clean; 589 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 4.2 MiB/s wr, 283 op/s 2024-05-10T11:44:00.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:43:59 smithi188 ceph-mon[34075]: pgmap v1189: 129 pgs: 129 active+clean; 589 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 4.2 MiB/s wr, 283 op/s 2024-05-10T11:44:00.332 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:44:00.962 INFO:tasks.workunit.client.0.smithi100.stdout: 10 118110 8.87 MB/sec execute 594 sec latency 83.970 ms 2024-05-10T11:44:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.757+0000 7f657f59e640 1 -- 172.21.15.100:0/2590460929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65780a6710 msgr2=0x7f65780aafd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.757+0000 7f657f59e640 1 --2- 172.21.15.100:0/2590460929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65780a6710 0x7f65780aafd0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f6570009900 tx=0x7f657002a420 comp rx=0 tx=0).stop 2024-05-10T11:44:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.757+0000 7f657f59e640 1 -- 172.21.15.100:0/2590460929 shutdown_connections 2024-05-10T11:44:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.757+0000 7f657f59e640 1 --2- 172.21.15.100:0/2590460929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65780a6710 0x7f65780aafd0 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.757+0000 7f657f59e640 1 --2- 172.21.15.100:0/2590460929 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f65780a5d70 0x7f65780a61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.757+0000 7f657f59e640 1 --2- 172.21.15.100:0/2590460929 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65780a4c40 0x7f65780a5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.757+0000 7f657f59e640 1 -- 172.21.15.100:0/2590460929 >> 172.21.15.100:0/2590460929 conn(0x7f65780a00d0 msgr2=0x7f65780a2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:44:01.758 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.757+0000 7f657f59e640 1 -- 172.21.15.100:0/2590460929 shutdown_connections 2024-05-10T11:44:01.758 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.757+0000 7f657f59e640 1 -- 172.21.15.100:0/2590460929 wait complete. 2024-05-10T11:44:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.758+0000 7f657f59e640 1 Processor -- start 2024-05-10T11:44:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.758+0000 7f657f59e640 1 -- start start 2024-05-10T11:44:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.758+0000 7f657f59e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65780a4c40 0x7f6578143480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.758+0000 7f657f59e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65780a5d70 0x7f65781439c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.758+0000 7f657f59e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f65780a6710 0x7f657814aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.758+0000 7f657f59e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f65780b8de0 con 0x7f65780a4c40 2024-05-10T11:44:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.758+0000 7f657f59e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f65780b8c60 con 0x7f65780a6710 2024-05-10T11:44:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.758+0000 7f657f59e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f65780b8f60 con 0x7f65780a5d70 2024-05-10T11:44:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.758+0000 7f657ed9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f65780a6710 0x7f657814aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.759+0000 7f657ed9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f65780a6710 0x7f657814aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:43594/0 (socket says 172.21.15.100:43594) 2024-05-10T11:44:01.760 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.759+0000 7f657ed9d640 1 -- 172.21.15.100:0/3487243142 learned_addr learned my addr 172.21.15.100:0/3487243142 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:44:01.760 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.759+0000 7f657dd9b640 1 --2- 172.21.15.100:0/3487243142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65780a5d70 0x7f65781439c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:01.760 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.759+0000 7f657dd9b640 1 -- 172.21.15.100:0/3487243142 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f65780a6710 msgr2=0x7f657814aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:01.760 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.759+0000 7f657dd9b640 1 --2- 172.21.15.100:0/3487243142 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f65780a6710 0x7f657814aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:01.760 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.759+0000 7f657dd9b640 1 -- 172.21.15.100:0/3487243142 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65780a4c40 msgr2=0x7f6578143480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:01.760 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.759+0000 7f657dd9b640 1 --2- 172.21.15.100:0/3487243142 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65780a4c40 0x7f6578143480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:01.760 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.759+0000 7f657dd9b640 1 -- 172.21.15.100:0/3487243142 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f657814b0a0 con 0x7f65780a5d70 2024-05-10T11:44:01.761 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.759+0000 7f657dd9b640 1 --2- 172.21.15.100:0/3487243142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65780a5d70 0x7f65781439c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f657400c970 tx=0x7f657400ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:44:01.761 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.760+0000 7f656f7fe640 1 -- 172.21.15.100:0/3487243142 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6574007bb0 con 0x7f65780a5d70 2024-05-10T11:44:01.761 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.760+0000 7f656f7fe640 1 -- 172.21.15.100:0/3487243142 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6574007d50 con 0x7f65780a5d70 2024-05-10T11:44:01.761 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.760+0000 7f656f7fe640 1 -- 172.21.15.100:0/3487243142 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6574004d20 con 0x7f65780a5d70 2024-05-10T11:44:01.761 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.760+0000 7f657f59e640 1 -- 172.21.15.100:0/3487243142 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f657814b390 con 0x7f65780a5d70 2024-05-10T11:44:01.761 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.760+0000 7f657f59e640 1 -- 172.21.15.100:0/3487243142 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f657814b910 con 0x7f65780a5d70 2024-05-10T11:44:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.764+0000 7f6587c4b740 1 Processor -- start 2024-05-10T11:44:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.764+0000 7f6587c4b740 1 -- start start 2024-05-10T11:44:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.764+0000 7f656f7fe640 1 -- 172.21.15.100:0/3487243142 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f657400f450 con 0x7f65780a5d70 2024-05-10T11:44:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.764+0000 7f6587c4b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599cc135f80 0x5599cc1363a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.764+0000 7f6587c4b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599cc136970 0x5599cc13b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.764+0000 7f6587c4b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599cc13bae0 0x5599cc13df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.764+0000 7f6587c4b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5599cc09a5d0 con 0x5599cc136970 2024-05-10T11:44:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.764+0000 7f6587c4b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5599cc053890 con 0x5599cc13bae0 2024-05-10T11:44:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.764+0000 7f6587c4b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5599cc096590 con 0x5599cc135f80 2024-05-10T11:44:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.764+0000 7f657dd9b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599cc13bae0 0x5599cc13df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.764+0000 7f657e59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599cc136970 0x5599cc13b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.764+0000 7f657e59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599cc136970 0x5599cc13b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43612/0 (socket says 172.21.15.100:43612) 2024-05-10T11:44:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.764+0000 7f657e59c640 1 -- 172.21.15.100:0/1345846741 learned_addr learned my addr 172.21.15.100:0/1345846741 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:44:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.764+0000 7f657ed9d640 1 --2- 172.21.15.100:0/1345846741 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599cc135f80 0x5599cc1363a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.765+0000 7f657e59c640 1 -- 172.21.15.100:0/1345846741 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599cc135f80 msgr2=0x5599cc1363a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:44:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.765+0000 7f657e59c640 1 --2- 172.21.15.100:0/1345846741 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599cc135f80 0x5599cc1363a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.765+0000 7f657e59c640 1 -- 172.21.15.100:0/1345846741 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599cc13bae0 msgr2=0x5599cc13df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:44:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.765+0000 7f657e59c640 1 --2- 172.21.15.100:0/1345846741 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599cc13bae0 0x5599cc13df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.765+0000 7f657e59c640 1 -- 172.21.15.100:0/1345846741 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599cbdac210 con 0x5599cc136970 2024-05-10T11:44:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.765+0000 7f657e59c640 1 --2- 172.21.15.100:0/1345846741 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599cc136970 0x5599cc13b470 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f6580068120 tx=0x7f65800517c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3fcd6b748f971e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:44:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.765+0000 7f656cff9640 1 -- 172.21.15.100:0/1345846741 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f658007c070 con 0x5599cc136970 2024-05-10T11:44:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.765+0000 7f656cff9640 1 -- 172.21.15.100:0/1345846741 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6580075ba0 con 0x5599cc136970 2024-05-10T11:44:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.765+0000 7f656cff9640 1 -- 172.21.15.100:0/1345846741 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6580077b30 con 0x5599cc136970 2024-05-10T11:44:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.766+0000 7f6587c4b740 1 -- 172.21.15.100:0/1345846741 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599cc136970 msgr2=0x5599cc13b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.766+0000 7f6587c4b740 1 --2- 172.21.15.100:0/1345846741 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599cc136970 0x5599cc13b470 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f6580068120 tx=0x7f65800517c0 comp rx=0 tx=0).stop 2024-05-10T11:44:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.766+0000 7f6587c4b740 1 -- 172.21.15.100:0/1345846741 shutdown_connections 2024-05-10T11:44:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.766+0000 7f6587c4b740 1 --2- 172.21.15.100:0/1345846741 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599cc13bae0 0x5599cc13df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.766+0000 7f6587c4b740 1 --2- 172.21.15.100:0/1345846741 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599cc136970 0x5599cc13b470 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.766+0000 7f6587c4b740 1 --2- 172.21.15.100:0/1345846741 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599cc135f80 0x5599cc1363a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.766+0000 7f6587c4b740 1 -- 172.21.15.100:0/1345846741 >> 172.21.15.100:0/1345846741 conn(0x5599cc1315f0 msgr2=0x5599cc1319f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:44:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.766+0000 7f656f7fe640 1 --2- 172.21.15.100:0/3487243142 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f65680b2870 0x7f65680b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.767+0000 7f657e59c640 1 --2- 172.21.15.100:0/3487243142 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f65680b2870 0x7f65680b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.767+0000 7f656f7fe640 1 -- 172.21.15.100:0/3487243142 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f65740e2170 con 0x7f65780a5d70 2024-05-10T11:44:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.767+0000 7f657e59c640 1 --2- 172.21.15.100:0/3487243142 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f65680b2870 0x7f65680b4d30 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f6580050a10 tx=0x7f6580068190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:44:01.770 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.770+0000 7f6587c4b740 1 -- 172.21.15.100:0/1345846741 shutdown_connections 2024-05-10T11:44:01.772 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.772+0000 7f6587c4b740 1 -- 172.21.15.100:0/1345846741 wait complete. 2024-05-10T11:44:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.773+0000 7f6587c4b740 1 -- start start 2024-05-10T11:44:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.773+0000 7f6587c4b740 1 Processor -- start 2024-05-10T11:44:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.774+0000 7f6587c4b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599cc135f80 0x5599cc140080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.774+0000 7f6587c4b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599cc136970 0x5599cc1405c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.774+0000 7f6587c4b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599cc13bae0 0x5599cc147600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.774+0000 7f6587c4b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5599cc13ea90 con 0x5599cc136970 2024-05-10T11:44:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.774+0000 7f6587c4b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5599cc057370 con 0x5599cc135f80 2024-05-10T11:44:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.774+0000 7f6587c4b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5599cc13ec10 con 0x5599cc13bae0 2024-05-10T11:44:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.775+0000 7f657dd9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599cc136970 0x5599cc1405c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.775+0000 7f657e59c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599cc135f80 0x5599cc140080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.775+0000 7f657e59c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599cc135f80 0x5599cc140080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:43612/0 (socket says 172.21.15.100:43612) 2024-05-10T11:44:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.775+0000 7f657e59c640 1 -- 172.21.15.100:0/1294578557 learned_addr learned my addr 172.21.15.100:0/1294578557 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:44:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.775+0000 7f657ed9d640 1 --2- 172.21.15.100:0/1294578557 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599cc13bae0 0x5599cc147600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.775+0000 7f6587c4b740 10 client.0 _ll_register_callbacks cb 0x5599cbdaba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:44:01.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.775+0000 7f657dd9b640 1 -- 172.21.15.100:0/1294578557 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599cc13bae0 msgr2=0x5599cc147600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:44:01.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.775+0000 7f657dd9b640 1 --2- 172.21.15.100:0/1294578557 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599cc13bae0 0x5599cc147600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:01.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.775+0000 7f657dd9b640 1 -- 172.21.15.100:0/1294578557 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599cc135f80 msgr2=0x5599cc140080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:44:01.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.775+0000 7f657dd9b640 1 --2- 172.21.15.100:0/1294578557 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599cc135f80 0x5599cc140080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:01.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.775+0000 7f657dd9b640 1 -- 172.21.15.100:0/1294578557 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599cbe78620 con 0x5599cc136970 2024-05-10T11:44:01.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.775+0000 7f657ed9d640 1 --2- 172.21.15.100:0/1294578557 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599cc13bae0 0x5599cc147600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:44:01.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.776+0000 7f657dd9b640 1 --2- 172.21.15.100:0/1294578557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599cc136970 0x5599cc1405c0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f65740df960 tx=0x7f657400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=390555d8367957df server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:44:01.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.776+0000 7f6565ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:44:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.776+0000 7f6565ffb640 1 -- 172.21.15.100:0/1294578557 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f657401d040 con 0x5599cc136970 2024-05-10T11:44:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.777+0000 7f6565ffb640 1 -- 172.21.15.100:0/1294578557 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f65740a5a20 con 0x5599cc136970 2024-05-10T11:44:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.777+0000 7f6587c4b740 1 -- 172.21.15.100:0/1294578557 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6580075ba0 con 0x5599cc136970 2024-05-10T11:44:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.777+0000 7f6565ffb640 1 -- 172.21.15.100:0/1294578557 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f65740a8ba0 con 0x5599cc136970 2024-05-10T11:44:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.777+0000 7f6587c4b740 10 client.20268 fetch_fsmap learned FSMap version 41 2024-05-10T11:44:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.777+0000 7f6587c4b740 1 -- 172.21.15.100:0/1294578557 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5599cc14a440 con 0x5599cc136970 2024-05-10T11:44:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.777+0000 7f6565ffb640 1 -- 172.21.15.100:0/1294578557 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6580075ba0 con 0x5599cc136970 2024-05-10T11:44:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.777+0000 7f6565ffb640 1 -- 172.21.15.100:0/1294578557 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f65740a4da0 con 0x5599cc136970 2024-05-10T11:44:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.778+0000 7f6587c4b740 10 client.20268 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:44:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.778+0000 7f6587c4b740 10 client.20268 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:44:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.778+0000 7f6587c4b740 20 client.20268 populate_metadata read hostname 'smithi100' 2024-05-10T11:44:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.778+0000 7f6587c4b740 1 --2- 172.21.15.100:0/1294578557 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5599cc14bb10 0x5599cc14df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.778+0000 7f6587c4b740 4 client.20268 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:44:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.778+0000 7f6587c4b740 1 -- 172.21.15.100:0/1294578557 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599cc0a81e0 con 0x5599cc14bb10 2024-05-10T11:44:01.779 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.778+0000 7f657e59c640 1 --2- 172.21.15.100:0/1294578557 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5599cc14bb10 0x5599cc14df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:01.779 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.779+0000 7f657e59c640 1 --2- 172.21.15.100:0/1294578557 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5599cc14bb10 0x5599cc14df80 crc :-1 s=READY pgs=473 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e9cf42cea536620 server_cookie=cbb227048b79f63b in_seq=0 out_seq=0 2024-05-10T11:44:01.779 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.779+0000 7f6565ffb640 10 client.20268 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:44:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.780+0000 7f6565ffb640 1 -- 172.21.15.100:0/1294578557 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5599cc0a81e0 con 0x5599cc14bb10 2024-05-10T11:44:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.780+0000 7f6565ffb640 10 client.20268 handle_command_reply: tid=0 2024-05-10T11:44:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.780+0000 7f6587c4b740 1 client.20268 shutdown 2024-05-10T11:44:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.780+0000 7f6587c4b740 1 -- 172.21.15.100:0/1294578557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599cc136970 msgr2=0x5599cc1405c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.780+0000 7f6587c4b740 1 --2- 172.21.15.100:0/1294578557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599cc136970 0x5599cc1405c0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f65740df960 tx=0x7f657400cf60 comp rx=0 tx=0).stop 2024-05-10T11:44:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.780+0000 7f6587c4b740 1 -- 172.21.15.100:0/1294578557 shutdown_connections 2024-05-10T11:44:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.780+0000 7f6587c4b740 1 --2- 172.21.15.100:0/1294578557 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5599cc14bb10 0x5599cc14df80 crc :-1 s=READY pgs=473 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.781+0000 7f6565ffb640 0 client.20268 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:44:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.781+0000 7f6587c4b740 1 --2- 172.21.15.100:0/1294578557 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599cc13bae0 0x5599cc147600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.781+0000 7f6587c4b740 1 --2- 172.21.15.100:0/1294578557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599cc136970 0x5599cc1405c0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.781+0000 7f6587c4b740 1 --2- 172.21.15.100:0/1294578557 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599cc135f80 0x5599cc140080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.781+0000 7f6587c4b740 1 -- 172.21.15.100:0/1294578557 >> 172.21.15.100:0/1294578557 conn(0x5599cc1315f0 msgr2=0x5599cc130750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:44:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.781+0000 7f6587c4b740 1 -- 172.21.15.100:0/1294578557 shutdown_connections 2024-05-10T11:44:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.781+0000 7f6587c4b740 1 -- 172.21.15.100:0/1294578557 wait complete. 2024-05-10T11:44:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.781+0000 7f6587c4b740 20 client.20268 trim_cache size 0 max 16384 2024-05-10T11:44:01.787 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:44:01 smithi100 ceph-mon[30296]: pgmap v1190: 129 pgs: 129 active+clean; 582 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 4.3 MiB/s wr, 296 op/s 2024-05-10T11:44:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.804+0000 7f6587c4b740 1 Processor -- start 2024-05-10T11:44:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.804+0000 7f6587c4b740 1 -- start start 2024-05-10T11:44:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.804+0000 7f6587c4b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599cc135f80 0x5599cc137a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.804+0000 7f6587c4b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599cc136970 0x5599cc1475c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.804+0000 7f6587c4b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599cc13bae0 0x5599cc137f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.804+0000 7f6587c4b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6560012400 con 0x5599cc136970 2024-05-10T11:44:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.804+0000 7f6587c4b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f656000fcd0 con 0x5599cc135f80 2024-05-10T11:44:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.804+0000 7f6587c4b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f656000db20 con 0x5599cc13bae0 2024-05-10T11:44:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.804+0000 7f657e59c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599cc135f80 0x5599cc137a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.804+0000 7f657dd9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599cc136970 0x5599cc1475c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.804+0000 7f657e59c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599cc135f80 0x5599cc137a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:43620/0 (socket says 172.21.15.100:43620) 2024-05-10T11:44:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.804+0000 7f657e59c640 1 -- 172.21.15.100:0/2389532901 learned_addr learned my addr 172.21.15.100:0/2389532901 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:44:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.804+0000 7f657ed9d640 1 --2- 172.21.15.100:0/2389532901 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599cc13bae0 0x5599cc137f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.805+0000 7f657e59c640 1 -- 172.21.15.100:0/2389532901 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599cc13bae0 msgr2=0x5599cc137f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:44:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.805+0000 7f657e59c640 1 --2- 172.21.15.100:0/2389532901 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599cc13bae0 0x5599cc137f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.805+0000 7f657e59c640 1 -- 172.21.15.100:0/2389532901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599cc136970 msgr2=0x5599cc1475c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:44:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.805+0000 7f657e59c640 1 --2- 172.21.15.100:0/2389532901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599cc136970 0x5599cc1475c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.805+0000 7f657e59c640 1 -- 172.21.15.100:0/2389532901 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599cc12c540 con 0x5599cc135f80 2024-05-10T11:44:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.805+0000 7f657dd9b640 1 --2- 172.21.15.100:0/2389532901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599cc136970 0x5599cc1475c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:44:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.805+0000 7f657e59c640 1 --2- 172.21.15.100:0/2389532901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599cc135f80 0x5599cc137a50 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f6580068120 tx=0x7f6580080000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9965554631f022c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:44:01.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.805+0000 7f6567fff640 1 -- 172.21.15.100:0/2389532901 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f658007c040 con 0x5599cc135f80 2024-05-10T11:44:01.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.805+0000 7f6567fff640 1 -- 172.21.15.100:0/2389532901 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6580075510 con 0x5599cc135f80 2024-05-10T11:44:01.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.805+0000 7f6567fff640 1 -- 172.21.15.100:0/2389532901 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65800816a0 con 0x5599cc135f80 2024-05-10T11:44:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.806+0000 7f6587c4b740 1 -- 172.21.15.100:0/2389532901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599cc135f80 msgr2=0x5599cc137a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.806+0000 7f6587c4b740 1 --2- 172.21.15.100:0/2389532901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599cc135f80 0x5599cc137a50 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f6580068120 tx=0x7f6580080000 comp rx=0 tx=0).stop 2024-05-10T11:44:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.806+0000 7f6587c4b740 1 -- 172.21.15.100:0/2389532901 shutdown_connections 2024-05-10T11:44:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.806+0000 7f6587c4b740 1 --2- 172.21.15.100:0/2389532901 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599cc13bae0 0x5599cc137f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.806+0000 7f6587c4b740 1 --2- 172.21.15.100:0/2389532901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599cc136970 0x5599cc1475c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.806+0000 7f6587c4b740 1 --2- 172.21.15.100:0/2389532901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599cc135f80 0x5599cc137a50 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.806+0000 7f6587c4b740 1 -- 172.21.15.100:0/2389532901 >> 172.21.15.100:0/2389532901 conn(0x5599cc1315f0 msgr2=0x5599cc1319d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:44:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.806+0000 7f6587c4b740 1 -- 172.21.15.100:0/2389532901 shutdown_connections 2024-05-10T11:44:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.806+0000 7f6587c4b740 1 -- 172.21.15.100:0/2389532901 wait complete. 2024-05-10T11:44:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.807+0000 7f6587c4b740 1 -- start start 2024-05-10T11:44:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.807+0000 7f6587c4b740 1 Processor -- start 2024-05-10T11:44:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.807+0000 7f6587c4b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599cc135f80 0x5599cc144de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.807+0000 7f6587c4b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599cc136970 0x5599cc145320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.807+0000 7f6587c4b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599cc13bae0 0x5599cc146330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.807+0000 7f6587c4b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f656000e190 con 0x5599cc136970 2024-05-10T11:44:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.807+0000 7f6587c4b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f656000f600 con 0x5599cc13bae0 2024-05-10T11:44:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.807+0000 7f6587c4b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f656000e860 con 0x5599cc135f80 2024-05-10T11:44:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.807+0000 7f657dd9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599cc136970 0x5599cc145320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.807+0000 7f6587c4b740 10 client.0 _ll_register_callbacks cb 0x7f65740046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:44:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.807+0000 7f657dd9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599cc136970 0x5599cc145320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43640/0 (socket says 172.21.15.100:43640) 2024-05-10T11:44:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.807+0000 7f657dd9b640 1 -- 172.21.15.100:0/620454138 learned_addr learned my addr 172.21.15.100:0/620454138 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:44:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.807+0000 7f657e59c640 1 --2- 172.21.15.100:0/620454138 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599cc135f80 0x5599cc144de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.807+0000 7f657ed9d640 1 --2- 172.21.15.100:0/620454138 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599cc13bae0 0x5599cc146330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.808+0000 7f657e59c640 1 -- 172.21.15.100:0/620454138 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599cc13bae0 msgr2=0x5599cc146330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:44:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.808+0000 7f657e59c640 1 --2- 172.21.15.100:0/620454138 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599cc13bae0 0x5599cc146330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.808+0000 7f657e59c640 1 -- 172.21.15.100:0/620454138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599cc136970 msgr2=0x5599cc145320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:44:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.808+0000 7f657e59c640 1 --2- 172.21.15.100:0/620454138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599cc136970 0x5599cc145320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.808+0000 7f657e59c640 1 -- 172.21.15.100:0/620454138 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599cc146920 con 0x5599cc135f80 2024-05-10T11:44:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.808+0000 7f657dd9b640 1 --2- 172.21.15.100:0/620454138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599cc136970 0x5599cc145320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:44:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.808+0000 7f657e59c640 1 --2- 172.21.15.100:0/620454138 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599cc135f80 0x5599cc144de0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f65800686c0 tx=0x7f6580077730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b1e0763b478b990 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:44:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.808+0000 7f6565ffb640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:44:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.808+0000 7f6565ffb640 1 -- 172.21.15.100:0/620454138 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6580077860 con 0x5599cc135f80 2024-05-10T11:44:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.808+0000 7f6587c4b740 1 -- 172.21.15.100:0/620454138 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6580075510 con 0x5599cc135f80 2024-05-10T11:44:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.808+0000 7f6565ffb640 1 -- 172.21.15.100:0/620454138 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f6580077a00 con 0x5599cc135f80 2024-05-10T11:44:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.808+0000 7f6565ffb640 1 -- 172.21.15.100:0/620454138 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f65800a6050 con 0x5599cc135f80 2024-05-10T11:44:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.808+0000 7f6587c4b740 10 client.29635 fetch_fsmap learned FSMap version 41 2024-05-10T11:44:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.808+0000 7f6587c4b740 1 -- 172.21.15.100:0/620454138 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5599cc1b64a0 con 0x5599cc135f80 2024-05-10T11:44:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.809+0000 7f6565ffb640 1 -- 172.21.15.100:0/620454138 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6580075510 con 0x5599cc135f80 2024-05-10T11:44:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.809+0000 7f6565ffb640 1 -- 172.21.15.100:0/620454138 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f6580092420 con 0x5599cc135f80 2024-05-10T11:44:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.809+0000 7f6587c4b740 10 client.29635 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:44:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.809+0000 7f6587c4b740 10 client.29635 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-10T11:44:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.809+0000 7f6587c4b740 20 client.29635 populate_metadata read hostname 'smithi100' 2024-05-10T11:44:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.809+0000 7f6587c4b740 1 --2- 172.21.15.100:0/620454138 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5599cc1b6c20 0x5599cc1b9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.809+0000 7f6587c4b740 4 client.29635 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T11:44:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.809+0000 7f6587c4b740 1 -- 172.21.15.100:0/620454138 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5599cc0a2050 con 0x5599cc1b6c20 2024-05-10T11:44:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.809+0000 7f657dd9b640 1 --2- 172.21.15.100:0/620454138 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5599cc1b6c20 0x5599cc1b9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.810+0000 7f6565ffb640 10 client.29635 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:44:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.810+0000 7f657dd9b640 1 --2- 172.21.15.100:0/620454138 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5599cc1b6c20 0x5599cc1b9040 crc :-1 s=READY pgs=474 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c58d148a4c538d5 server_cookie=b5b6e0461ba36dd1 in_seq=0 out_seq=0 2024-05-10T11:44:01.962 INFO:tasks.workunit.client.0.smithi100.stdout: 10 118280 8.87 MB/sec execute 595 sec latency 134.604 ms 2024-05-10T11:44:01.972 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.971+0000 7f6565ffb640 1 -- 172.21.15.100:0/620454138 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5599cc0a2050 con 0x5599cc1b6c20 2024-05-10T11:44:01.972 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.971+0000 7f6565ffb640 10 client.29635 handle_command_reply: tid=0 2024-05-10T11:44:01.972 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.971+0000 7f6587c4b740 1 client.29635 shutdown 2024-05-10T11:44:01.972 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.972+0000 7f6587c4b740 1 -- 172.21.15.100:0/620454138 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599cc135f80 msgr2=0x5599cc144de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:01.972 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.972+0000 7f6587c4b740 1 --2- 172.21.15.100:0/620454138 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599cc135f80 0x5599cc144de0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f65800686c0 tx=0x7f6580077730 comp rx=0 tx=0).stop 2024-05-10T11:44:01.973 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.972+0000 7f6587c4b740 1 -- 172.21.15.100:0/620454138 shutdown_connections 2024-05-10T11:44:01.973 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.972+0000 7f6587c4b740 1 --2- 172.21.15.100:0/620454138 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x5599cc1b6c20 0x5599cc1b9040 crc :-1 s=READY pgs=474 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:01.973 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.972+0000 7f6587c4b740 1 --2- 172.21.15.100:0/620454138 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599cc13bae0 0x5599cc146330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:01.973 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.972+0000 7f6587c4b740 1 --2- 172.21.15.100:0/620454138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599cc136970 0x5599cc145320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:01.973 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.972+0000 7f6565ffb640 0 client.29635 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:44:01.973 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.972+0000 7f6587c4b740 1 --2- 172.21.15.100:0/620454138 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599cc135f80 0x5599cc144de0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:01.973 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.972+0000 7f6587c4b740 1 -- 172.21.15.100:0/620454138 >> 172.21.15.100:0/620454138 conn(0x5599cc1315f0 msgr2=0x5599cc137a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:44:01.973 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.972+0000 7f6587c4b740 1 -- 172.21.15.100:0/620454138 shutdown_connections 2024-05-10T11:44:01.973 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.972+0000 7f6587c4b740 1 -- 172.21.15.100:0/620454138 wait complete. 2024-05-10T11:44:01.973 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.972+0000 7f6587c4b740 20 client.29635 trim_cache size 0 max 16384 2024-05-10T11:44:01.973 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-05-10T11:44:01.974 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-05-10T11:44:01.974 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-05-10T11:44:01.974 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-05-10T11:44:01.975 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.973+0000 7f657f59e640 1 -- 172.21.15.100:0/3487243142 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f65680b2870 msgr2=0x7f65680b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:01.975 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.973+0000 7f657f59e640 1 --2- 172.21.15.100:0/3487243142 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f65680b2870 0x7f65680b4d30 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f6580050a10 tx=0x7f6580068190 comp rx=0 tx=0).stop 2024-05-10T11:44:01.975 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.973+0000 7f657f59e640 1 -- 172.21.15.100:0/3487243142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65780a5d70 msgr2=0x7f65781439c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:01.975 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.973+0000 7f657f59e640 1 --2- 172.21.15.100:0/3487243142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65780a5d70 0x7f65781439c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f657400c970 tx=0x7f657400ce30 comp rx=0 tx=0).stop 2024-05-10T11:44:01.975 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.973+0000 7f657f59e640 1 -- 172.21.15.100:0/3487243142 shutdown_connections 2024-05-10T11:44:01.975 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.973+0000 7f657f59e640 1 --2- 172.21.15.100:0/3487243142 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f65680b2870 0x7f65680b4d30 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:01.975 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.973+0000 7f657f59e640 1 --2- 172.21.15.100:0/3487243142 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f65780a6710 0x7f657814aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:01.975 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.973+0000 7f657f59e640 1 --2- 172.21.15.100:0/3487243142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65780a5d70 0x7f65781439c0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:01.975 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.973+0000 7f657f59e640 1 --2- 172.21.15.100:0/3487243142 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65780a4c40 0x7f6578143480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:01.975 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.973+0000 7f657f59e640 1 -- 172.21.15.100:0/3487243142 >> 172.21.15.100:0/3487243142 conn(0x7f65780a00d0 msgr2=0x7f65780a0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:44:01.975 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.973+0000 7f657f59e640 1 -- 172.21.15.100:0/3487243142 shutdown_connections 2024-05-10T11:44:01.975 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-05-10T11:44:01.973+0000 7f657f59e640 1 -- 172.21.15.100:0/3487243142 wait complete. 2024-05-10T11:44:02.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:44:01 smithi178 ceph-mon[33359]: pgmap v1190: 129 pgs: 129 active+clean; 582 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 4.3 MiB/s wr, 296 op/s 2024-05-10T11:44:02.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:44:01 smithi188 ceph-mon[34075]: pgmap v1190: 129 pgs: 129 active+clean; 582 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 4.3 MiB/s wr, 296 op/s 2024-05-10T11:44:02.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:02.714+0000 7fc28d065700 1 -- 172.21.15.100:0/2598152336 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc288109630 msgr2=0x7fc288109a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:02.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:02.714+0000 7fc28d065700 1 --2- 172.21.15.100:0/2598152336 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc288109630 0x7fc288109a10 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fc278009990 tx=0x7fc278009ca0 comp rx=0 tx=0).stop 2024-05-10T11:44:02.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:02.714+0000 7fc28d065700 1 -- 172.21.15.100:0/2598152336 shutdown_connections 2024-05-10T11:44:02.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:02.714+0000 7fc28d065700 1 --2- 172.21.15.100:0/2598152336 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc288103ee0 0x7fc2881103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:02.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:02.714+0000 7fc28d065700 1 --2- 172.21.15.100:0/2598152336 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc288109630 0x7fc288109a10 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:02.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:02.714+0000 7fc28d065700 1 --2- 172.21.15.100:0/2598152336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc288103520 0x7fc2881039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:02.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:02.714+0000 7fc28d065700 1 -- 172.21.15.100:0/2598152336 >> 172.21.15.100:0/2598152336 conn(0x7fc28807a650 msgr2=0x7fc28807aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:44:02.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:02.714+0000 7fc28d065700 1 -- 172.21.15.100:0/2598152336 shutdown_connections 2024-05-10T11:44:02.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:02.714+0000 7fc28d065700 1 -- 172.21.15.100:0/2598152336 wait complete. 2024-05-10T11:44:02.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:02.715+0000 7fc28d065700 1 Processor -- start 2024-05-10T11:44:02.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:02.716+0000 7fc28d065700 1 -- start start 2024-05-10T11:44:02.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:02.716+0000 7fc28d065700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc288103520 0x7fc2881a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:02.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:02.716+0000 7fc28d065700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc288103ee0 0x7fc2881a1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:02.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:02.716+0000 7fc28d065700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc288109630 0x7fc28819b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:02.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:02.716+0000 7fc28d065700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc2881135b0 con 0x7fc288103520 2024-05-10T11:44:02.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:02.717+0000 7fc286d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc288103520 0x7fc2881a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:02.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:02.717+0000 7fc28659c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc288103ee0 0x7fc2881a1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:02.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:02.717+0000 7fc286d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc288103520 0x7fc2881a16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43666/0 (socket says 172.21.15.100:43666) 2024-05-10T11:44:02.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:02.717+0000 7fc286d9d700 1 -- 172.21.15.100:0/4005858173 learned_addr learned my addr 172.21.15.100:0/4005858173 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:44:02.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:02.717+0000 7fc28759e700 1 --2- 172.21.15.100:0/4005858173 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc288109630 0x7fc28819b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:02.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:02.717+0000 7fc28d065700 1 -- 172.21.15.100:0/4005858173 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc288113430 con 0x7fc288109630 2024-05-10T11:44:02.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:02.717+0000 7fc28d065700 1 -- 172.21.15.100:0/4005858173 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc288113730 con 0x7fc288103ee0 2024-05-10T11:44:02.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:02.717+0000 7fc28759e700 1 -- 172.21.15.100:0/4005858173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc288103ee0 msgr2=0x7fc2881a1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:02.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:02.717+0000 7fc28759e700 1 --2- 172.21.15.100:0/4005858173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc288103ee0 0x7fc2881a1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:02.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:02.717+0000 7fc28759e700 1 -- 172.21.15.100:0/4005858173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc288103520 msgr2=0x7fc2881a16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:02.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:02.717+0000 7fc28759e700 1 --2- 172.21.15.100:0/4005858173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc288103520 0x7fc2881a16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:02.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:02.717+0000 7fc28759e700 1 -- 172.21.15.100:0/4005858173 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc28819c010 con 0x7fc288109630 2024-05-10T11:44:02.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:02.717+0000 7fc286d9d700 1 --2- 172.21.15.100:0/4005858173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc288103520 0x7fc2881a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:44:02.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:02.717+0000 7fc28759e700 1 --2- 172.21.15.100:0/4005858173 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc288109630 0x7fc28819b8b0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc28000efc0 tx=0x7fc28000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:44:02.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:02.718+0000 7fc26ffff700 1 -- 172.21.15.100:0/4005858173 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2800092e0 con 0x7fc288109630 2024-05-10T11:44:02.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:02.718+0000 7fc28d065700 1 -- 172.21.15.100:0/4005858173 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc28819c2a0 con 0x7fc288109630 2024-05-10T11:44:02.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:02.718+0000 7fc26ffff700 1 -- 172.21.15.100:0/4005858173 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc280004bb0 con 0x7fc288109630 2024-05-10T11:44:02.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:02.718+0000 7fc28d065700 1 -- 172.21.15.100:0/4005858173 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2881a85c0 con 0x7fc288109630 2024-05-10T11:44:02.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:02.718+0000 7fc26ffff700 1 -- 172.21.15.100:0/4005858173 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc280004020 con 0x7fc288109630 2024-05-10T11:44:02.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:02.719+0000 7fc28d065700 1 -- 172.21.15.100:0/4005858173 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc28806a190 con 0x7fc288109630 2024-05-10T11:44:02.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:02.721+0000 7fc26ffff700 1 -- 172.21.15.100:0/4005858173 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fc28001b030 con 0x7fc288109630 2024-05-10T11:44:02.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:02.721+0000 7fc26ffff700 1 --2- 172.21.15.100:0/4005858173 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc2700b2900 0x7fc2700b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:02.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:02.722+0000 7fc286d9d700 1 --2- 172.21.15.100:0/4005858173 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc2700b2900 0x7fc2700b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:02.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:02.722+0000 7fc26ffff700 1 -- 172.21.15.100:0/4005858173 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fc2800e1f20 con 0x7fc288109630 2024-05-10T11:44:02.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:02.722+0000 7fc286d9d700 1 --2- 172.21.15.100:0/4005858173 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc2700b2900 0x7fc2700b4dc0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fc278009350 tx=0x7fc27801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:44:02.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:02.728+0000 7fc26ffff700 1 -- 172.21.15.100:0/4005858173 <== mon.2 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fc280022020 con 0x7fc288109630 2024-05-10T11:44:02.962 INFO:tasks.workunit.client.0.smithi100.stdout: 10 118472 8.87 MB/sec execute 596 sec latency 114.826 ms 2024-05-10T11:44:03.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:03.143+0000 7fc28d065700 1 -- 172.21.15.100:0/4005858173 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc2881a8790 con 0x7fc288109630 2024-05-10T11:44:03.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:03.144+0000 7fc26ffff700 1 -- 172.21.15.100:0/4005858173 <== mon.2 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+11897 (secure 0 0 0) 0x7fc280010040 con 0x7fc288109630 2024-05-10T11:44:03.144 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:44:03.144 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2072225012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2072225012},{"type":"v1","addr":"172.21.15.100:6835","nonce":2072225012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/401939540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":401939540},{"type":"v1","addr":"172.21.15.100:6837","nonce":401939540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14571,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/3946311025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3946311025},{"type":"v1","addr":"172.21.15.100:6839","nonce":3946311025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3736371660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3736371660},{"type":"v1","addr":"172.21.15.100:6841","nonce":3736371660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/2741810791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2741810791},{"type":"v1","addr":"172.21.15.178:6833","nonce":2741810791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1704891041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1704891041},{"type":"v1","addr":"172.21.15.188:6833","nonce":1704891041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":41,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T11:14:27.242452+0000","modified":"2024-05-10T11:35:29.588072+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24439,"mds_1":24437,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/2124076312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2124076312},{"type":"v1","addr":"172.21.15.178:6835","nonce":2124076312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/3224358425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3224358425},{"type":"v1","addr":"172.21.15.178:6837","nonce":3224358425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/1943342620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1943342620},{"type":"v1","addr":"172.21.15.188:6837","nonce":1943342620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.178:6839/3760985404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3760985404},{"type":"v1","addr":"172.21.15.178:6839","nonce":3760985404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.188:6839/3194917070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":3194917070},{"type":"v1","addr":"172.21.15.188:6839","nonce":3194917070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27034":{"gid":27034,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/3139559450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3139559450},{"type":"v1","addr":"172.21.15.188:6835","nonce":3139559450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24439,"qdb_cluster":[24437,24439,24427]},"id":1}]} 2024-05-10T11:44:03.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:03.147+0000 7fc28d065700 1 -- 172.21.15.100:0/4005858173 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc2700b2900 msgr2=0x7fc2700b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:03.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:03.147+0000 7fc28d065700 1 --2- 172.21.15.100:0/4005858173 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc2700b2900 0x7fc2700b4dc0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fc278009350 tx=0x7fc27801a040 comp rx=0 tx=0).stop 2024-05-10T11:44:03.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:03.147+0000 7fc28d065700 1 -- 172.21.15.100:0/4005858173 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc288109630 msgr2=0x7fc28819b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:03.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:03.147+0000 7fc28d065700 1 --2- 172.21.15.100:0/4005858173 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc288109630 0x7fc28819b8b0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc28000efc0 tx=0x7fc28000c3a0 comp rx=0 tx=0).stop 2024-05-10T11:44:03.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:03.147+0000 7fc28d065700 1 -- 172.21.15.100:0/4005858173 shutdown_connections 2024-05-10T11:44:03.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:03.147+0000 7fc28d065700 1 --2- 172.21.15.100:0/4005858173 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fc2700b2900 0x7fc2700b4dc0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:03.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:03.147+0000 7fc28d065700 1 --2- 172.21.15.100:0/4005858173 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc288109630 0x7fc28819b8b0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:03.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:03.147+0000 7fc28d065700 1 --2- 172.21.15.100:0/4005858173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc288103520 0x7fc2881a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:03.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:03.147+0000 7fc28d065700 1 --2- 172.21.15.100:0/4005858173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc288103ee0 0x7fc2881a1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:03.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:03.147+0000 7fc28d065700 1 -- 172.21.15.100:0/4005858173 >> 172.21.15.100:0/4005858173 conn(0x7fc28807a650 msgr2=0x7fc2880756d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:44:03.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:03.147+0000 7fc28d065700 1 -- 172.21.15.100:0/4005858173 shutdown_connections 2024-05-10T11:44:03.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:03.147+0000 7fc28d065700 1 -- 172.21.15.100:0/4005858173 wait complete. 2024-05-10T11:44:03.148 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 41 2024-05-10T11:44:03.829 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-10T11:44:03.962 INFO:tasks.workunit.client.0.smithi100.stdout: 10 118660 8.87 MB/sec execute 597 sec latency 90.538 ms 2024-05-10T11:44:04.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:44:03 smithi188 ceph-mon[34075]: pgmap v1191: 129 pgs: 129 active+clean; 577 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 4.2 MiB/s wr, 262 op/s 2024-05-10T11:44:04.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:44:03 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/4005858173' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:44:04.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:44:03 smithi100 ceph-mon[30296]: pgmap v1191: 129 pgs: 129 active+clean; 577 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 4.2 MiB/s wr, 262 op/s 2024-05-10T11:44:04.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:44:03 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/4005858173' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:44:04.218 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:44:04.254 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:44:03 smithi178 ceph-mon[33359]: pgmap v1191: 129 pgs: 129 active+clean; 577 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 4.2 MiB/s wr, 262 op/s 2024-05-10T11:44:04.254 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:44:03 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/4005858173' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:44:04.962 INFO:tasks.workunit.client.0.smithi100.stdout: 10 118809 8.87 MB/sec execute 598 sec latency 123.895 ms 2024-05-10T11:44:05.962 INFO:tasks.workunit.client.0.smithi100.stdout: 10 118961 8.86 MB/sec execute 599 sec latency 101.833 ms 2024-05-10T11:44:06.008 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:44:05 smithi100 ceph-mon[30296]: pgmap v1192: 129 pgs: 129 active+clean; 567 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 4.2 MiB/s wr, 275 op/s 2024-05-10T11:44:06.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:44:05 smithi188 ceph-mon[34075]: pgmap v1192: 129 pgs: 129 active+clean; 567 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 4.2 MiB/s wr, 275 op/s 2024-05-10T11:44:06.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.068+0000 7fdbf74d8700 1 -- 172.21.15.100:0/1548686771 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdbf0102390 msgr2=0x7fdbf0102810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:06.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.068+0000 7fdbf74d8700 1 --2- 172.21.15.100:0/1548686771 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdbf0102390 0x7fdbf0102810 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fdbe0009990 tx=0x7fdbe0009ca0 comp rx=0 tx=0).stop 2024-05-10T11:44:06.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.068+0000 7fdbf74d8700 1 -- 172.21.15.100:0/1548686771 shutdown_connections 2024-05-10T11:44:06.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.068+0000 7fdbf74d8700 1 --2- 172.21.15.100:0/1548686771 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbf0102d50 0x7fdbf010f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:06.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.068+0000 7fdbf74d8700 1 --2- 172.21.15.100:0/1548686771 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdbf0108400 0x7fdbf01087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:06.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.068+0000 7fdbf74d8700 1 --2- 172.21.15.100:0/1548686771 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdbf0102390 0x7fdbf0102810 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:06.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.068+0000 7fdbf74d8700 1 -- 172.21.15.100:0/1548686771 >> 172.21.15.100:0/1548686771 conn(0x7fdbf0079310 msgr2=0x7fdbf0079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:44:06.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.068+0000 7fdbf74d8700 1 -- 172.21.15.100:0/1548686771 shutdown_connections 2024-05-10T11:44:06.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.068+0000 7fdbf74d8700 1 -- 172.21.15.100:0/1548686771 wait complete. 2024-05-10T11:44:06.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.069+0000 7fdbf74d8700 1 Processor -- start 2024-05-10T11:44:06.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.069+0000 7fdbf74d8700 1 -- start start 2024-05-10T11:44:06.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.069+0000 7fdbf74d8700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdbf0102390 0x7fdbf0199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:06.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.069+0000 7fdbf74d8700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbf0102d50 0x7fdbf019a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:06.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.069+0000 7fdbf74d8700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdbf0108400 0x7fdbf019e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:06.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.069+0000 7fdbf74d8700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf01123d0 con 0x7fdbf0102390 2024-05-10T11:44:06.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.069+0000 7fdbf74d8700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf0112250 con 0x7fdbf0102d50 2024-05-10T11:44:06.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.069+0000 7fdbf74d8700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf0112550 con 0x7fdbf0108400 2024-05-10T11:44:06.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.069+0000 7fdbf5274700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdbf0102390 0x7fdbf0199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:06.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.069+0000 7fdbf4a73700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbf0102d50 0x7fdbf019a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:06.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.069+0000 7fdbf5274700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdbf0102390 0x7fdbf0199b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43690/0 (socket says 172.21.15.100:43690) 2024-05-10T11:44:06.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.069+0000 7fdbf4a73700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbf0102d50 0x7fdbf019a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:43648/0 (socket says 172.21.15.100:43648) 2024-05-10T11:44:06.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.069+0000 7fdbf5274700 1 -- 172.21.15.100:0/577634659 learned_addr learned my addr 172.21.15.100:0/577634659 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:44:06.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.069+0000 7fdbf5a75700 1 --2- 172.21.15.100:0/577634659 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdbf0108400 0x7fdbf019e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:06.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.070+0000 7fdbf5274700 1 -- 172.21.15.100:0/577634659 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdbf0108400 msgr2=0x7fdbf019e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:06.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.070+0000 7fdbf5274700 1 --2- 172.21.15.100:0/577634659 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdbf0108400 0x7fdbf019e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:06.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.070+0000 7fdbf5274700 1 -- 172.21.15.100:0/577634659 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbf0102d50 msgr2=0x7fdbf019a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:06.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.070+0000 7fdbf5274700 1 --2- 172.21.15.100:0/577634659 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbf0102d50 0x7fdbf019a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:06.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.070+0000 7fdbf5274700 1 -- 172.21.15.100:0/577634659 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbf019eb10 con 0x7fdbf0102390 2024-05-10T11:44:06.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.070+0000 7fdbf5274700 1 --2- 172.21.15.100:0/577634659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdbf0102390 0x7fdbf0199b10 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fdbec00eb30 tx=0x7fdbec00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:44:06.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.070+0000 7fdbe67fc700 1 -- 172.21.15.100:0/577634659 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbec00cbd0 con 0x7fdbf0102390 2024-05-10T11:44:06.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.070+0000 7fdbe67fc700 1 -- 172.21.15.100:0/577634659 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdbec0044b0 con 0x7fdbf0102390 2024-05-10T11:44:06.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.070+0000 7fdbe67fc700 1 -- 172.21.15.100:0/577634659 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbec003dc0 con 0x7fdbf0102390 2024-05-10T11:44:06.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.070+0000 7fdbf74d8700 1 -- 172.21.15.100:0/577634659 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbf019ee00 con 0x7fdbf0102390 2024-05-10T11:44:06.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.070+0000 7fdbf74d8700 1 -- 172.21.15.100:0/577634659 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbf01a66c0 con 0x7fdbf0102390 2024-05-10T11:44:06.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.074+0000 7fdbe67fc700 1 -- 172.21.15.100:0/577634659 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fdbec003f60 con 0x7fdbf0102390 2024-05-10T11:44:06.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.075+0000 7fdbe67fc700 1 --2- 172.21.15.100:0/577634659 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fdbdc0b2900 0x7fdbdc0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:06.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.075+0000 7fdbe67fc700 1 -- 172.21.15.100:0/577634659 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fdbec0e2740 con 0x7fdbf0102390 2024-05-10T11:44:06.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.075+0000 7fdbf4a73700 1 --2- 172.21.15.100:0/577634659 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fdbdc0b2900 0x7fdbdc0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:06.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.075+0000 7fdc01d6bb80 1 Processor -- start 2024-05-10T11:44:06.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.075+0000 7fdc01d6bb80 1 -- start start 2024-05-10T11:44:06.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.076+0000 7fdbf4a73700 1 --2- 172.21.15.100:0/577634659 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fdbdc0b2900 0x7fdbdc0b4dc0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fdbf019aa80 tx=0x7fdbe0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:44:06.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.076+0000 7fdc01d6bb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583d3426a60 0x5583d3426e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:06.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.076+0000 7fdc01d6bb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5583d3427380 0x5583d3418810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:06.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.076+0000 7fdc01d6bb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583d3418d50 0x5583d3419190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:06.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.076+0000 7fdc01d6bb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5583d336a940 con 0x5583d3426a60 2024-05-10T11:44:06.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.076+0000 7fdc01d6bb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5583d32cfda0 con 0x5583d3418d50 2024-05-10T11:44:06.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.076+0000 7fdc01d6bb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5583d3366900 con 0x5583d3427380 2024-05-10T11:44:06.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.076+0000 7fdbf5a75700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583d3418d50 0x5583d3419190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:06.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.076+0000 7fdbf5274700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583d3426a60 0x5583d3426e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:06.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.076+0000 7fdbf4a73700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5583d3427380 0x5583d3418810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:06.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.076+0000 7fdbf5a75700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583d3418d50 0x5583d3419190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:43652/0 (socket says 172.21.15.100:43652) 2024-05-10T11:44:06.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.076+0000 7fdbf5a75700 1 -- 172.21.15.100:0/1869929237 learned_addr learned my addr 172.21.15.100:0/1869929237 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:44:06.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.076+0000 7fdbf4a73700 1 -- 172.21.15.100:0/1869929237 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583d3418d50 msgr2=0x5583d3419190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:44:06.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.076+0000 7fdbf4a73700 1 --2- 172.21.15.100:0/1869929237 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583d3418d50 0x5583d3419190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:06.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.076+0000 7fdbf4a73700 1 -- 172.21.15.100:0/1869929237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583d3426a60 msgr2=0x5583d3426e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:44:06.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.077+0000 7fdbf4a73700 1 --2- 172.21.15.100:0/1869929237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583d3426a60 0x5583d3426e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:06.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.077+0000 7fdbf4a73700 1 -- 172.21.15.100:0/1869929237 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583d32b78d0 con 0x5583d3427380 2024-05-10T11:44:06.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.077+0000 7fdbf5a75700 1 --2- 172.21.15.100:0/1869929237 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583d3418d50 0x5583d3419190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:44:06.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.077+0000 7fdbf5274700 1 --2- 172.21.15.100:0/1869929237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583d3426a60 0x5583d3426e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:44:06.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.077+0000 7fdbf4a73700 1 --2- 172.21.15.100:0/1869929237 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5583d3427380 0x5583d3418810 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fdbe00098e0 tx=0x7fdbe001f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13de182ee493609e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:44:06.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.077+0000 7fdbdb7fe700 1 -- 172.21.15.100:0/1869929237 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe000b6b0 con 0x5583d3427380 2024-05-10T11:44:06.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.077+0000 7fdbdb7fe700 1 -- 172.21.15.100:0/1869929237 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdbe000bce0 con 0x5583d3427380 2024-05-10T11:44:06.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.077+0000 7fdbdb7fe700 1 -- 172.21.15.100:0/1869929237 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe000fca0 con 0x5583d3427380 2024-05-10T11:44:06.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.078+0000 7fdc01d6bb80 1 -- 172.21.15.100:0/1869929237 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5583d3427380 msgr2=0x5583d3418810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:06.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.078+0000 7fdc01d6bb80 1 --2- 172.21.15.100:0/1869929237 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5583d3427380 0x5583d3418810 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fdbe00098e0 tx=0x7fdbe001f000 comp rx=0 tx=0).stop 2024-05-10T11:44:06.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.078+0000 7fdc01d6bb80 1 -- 172.21.15.100:0/1869929237 shutdown_connections 2024-05-10T11:44:06.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.078+0000 7fdc01d6bb80 1 --2- 172.21.15.100:0/1869929237 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583d3418d50 0x5583d3419190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:06.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.078+0000 7fdc01d6bb80 1 --2- 172.21.15.100:0/1869929237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583d3426a60 0x5583d3426e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:06.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.078+0000 7fdc01d6bb80 1 --2- 172.21.15.100:0/1869929237 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5583d3427380 0x5583d3418810 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:06.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.078+0000 7fdc01d6bb80 1 -- 172.21.15.100:0/1869929237 >> 172.21.15.100:0/1869929237 conn(0x5583d3344b70 msgr2=0x5583d34132a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:44:06.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.078+0000 7fdc01d6bb80 1 -- 172.21.15.100:0/1869929237 shutdown_connections 2024-05-10T11:44:06.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.078+0000 7fdc01d6bb80 1 -- 172.21.15.100:0/1869929237 wait complete. 2024-05-10T11:44:06.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.079+0000 7fdc01d6bb80 1 -- start start 2024-05-10T11:44:06.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.079+0000 7fdc01d6bb80 1 Processor -- start 2024-05-10T11:44:06.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.079+0000 7fdc01d6bb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5583d3418d50 0x5583d342cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:06.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.079+0000 7fdc01d6bb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583d3426a60 0x5583d342a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:06.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.079+0000 7fdc01d6bb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583d3427380 0x5583d342a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:06.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.079+0000 7fdc01d6bb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5583d342dbd0 con 0x5583d3427380 2024-05-10T11:44:06.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.079+0000 7fdc01d6bb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5583d3323ed0 con 0x5583d3426a60 2024-05-10T11:44:06.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.079+0000 7fdc01d6bb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5583d342dd50 con 0x5583d3418d50 2024-05-10T11:44:06.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.080+0000 7fdc01d6bb80 10 client.0 _ll_register_callbacks cb 0x5583d32fe5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:44:06.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.080+0000 7fdbf5274700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5583d3418d50 0x5583d342cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:06.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.080+0000 7fdbf4a73700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583d3426a60 0x5583d342a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:06.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.080+0000 7fdbf5274700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5583d3418d50 0x5583d342cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:38204/0 (socket says 172.21.15.100:38204) 2024-05-10T11:44:06.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.080+0000 7fdbf5274700 1 -- 172.21.15.100:0/1634722300 learned_addr learned my addr 172.21.15.100:0/1634722300 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:44:06.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.080+0000 7fdbf5a75700 1 --2- 172.21.15.100:0/1634722300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583d3427380 0x5583d342a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:06.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.080+0000 7fdbf5274700 1 -- 172.21.15.100:0/1634722300 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583d3426a60 msgr2=0x5583d342a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:44:06.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.080+0000 7fdbf5274700 1 --2- 172.21.15.100:0/1634722300 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583d3426a60 0x5583d342a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:06.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.080+0000 7fdbf5274700 1 -- 172.21.15.100:0/1634722300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583d3427380 msgr2=0x5583d342a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:44:06.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.080+0000 7fdbf5274700 1 --2- 172.21.15.100:0/1634722300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583d3427380 0x5583d342a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:06.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.080+0000 7fdbf5274700 1 -- 172.21.15.100:0/1634722300 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583d3321760 con 0x5583d3418d50 2024-05-10T11:44:06.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.080+0000 7fdbf5274700 1 --2- 172.21.15.100:0/1634722300 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5583d3418d50 0x5583d342cec0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fdbec0dff30 tx=0x7fdbec0a9470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e2b088dbdd28a3d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:44:06.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.080+0000 7fdbd8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:44:06.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.080+0000 7fdbd8ff9700 1 -- 172.21.15.100:0/1634722300 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbec0c3060 con 0x5583d3418d50 2024-05-10T11:44:06.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.080+0000 7fdbd8ff9700 1 -- 172.21.15.100:0/1634722300 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fdbec0cc0e0 con 0x5583d3418d50 2024-05-10T11:44:06.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.080+0000 7fdc01d6bb80 1 -- 172.21.15.100:0/1634722300 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbe000bce0 con 0x5583d3418d50 2024-05-10T11:44:06.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.081+0000 7fdbd8ff9700 1 -- 172.21.15.100:0/1634722300 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fdbec00ce90 con 0x5583d3418d50 2024-05-10T11:44:06.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.081+0000 7fdc01d6bb80 10 client.29653 fetch_fsmap learned FSMap version 41 2024-05-10T11:44:06.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.081+0000 7fdc01d6bb80 1 -- 172.21.15.100:0/1634722300 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5583d3438430 con 0x5583d3418d50 2024-05-10T11:44:06.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.081+0000 7fdbd8ff9700 1 -- 172.21.15.100:0/1634722300 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdbe000bce0 con 0x5583d3418d50 2024-05-10T11:44:06.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.081+0000 7fdbd8ff9700 1 -- 172.21.15.100:0/1634722300 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fdbec0c5020 con 0x5583d3418d50 2024-05-10T11:44:06.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.081+0000 7fdc01d6bb80 10 client.29653 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:44:06.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.081+0000 7fdc01d6bb80 10 client.29653 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-10T11:44:06.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.081+0000 7fdc01d6bb80 20 client.29653 populate_metadata read hostname 'smithi100' 2024-05-10T11:44:06.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.081+0000 7fdc01d6bb80 1 --2- 172.21.15.100:0/1634722300 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x5583d343aa70 0x5583d343ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:06.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.081+0000 7fdc01d6bb80 4 client.29653 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:44:06.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.081+0000 7fdc01d6bb80 1 -- 172.21.15.100:0/1634722300 --> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583d3378320 con 0x5583d343aa70 2024-05-10T11:44:06.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.081+0000 7fdbf4a73700 1 --2- 172.21.15.100:0/1634722300 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x5583d343aa70 0x5583d343ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:06.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.082+0000 7fdbf4a73700 1 --2- 172.21.15.100:0/1634722300 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x5583d343aa70 0x5583d343ce60 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2a2fefc4398d493a server_cookie=6ad7875f2c94744c in_seq=0 out_seq=0 2024-05-10T11:44:06.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.082+0000 7fdbd8ff9700 10 client.29653 ms_handle_connect on v2:172.21.15.188:6836/1943342620 2024-05-10T11:44:06.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.083+0000 7fdbd8ff9700 1 -- 172.21.15.100:0/1634722300 <== mds.2 v2:172.21.15.188:6836/1943342620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5583d3378320 con 0x5583d343aa70 2024-05-10T11:44:06.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.083+0000 7fdbd8ff9700 10 client.29653 handle_command_reply: tid=0 2024-05-10T11:44:06.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.083+0000 7fdc01d6bb80 1 client.29653 shutdown 2024-05-10T11:44:06.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.083+0000 7fdc01d6bb80 1 -- 172.21.15.100:0/1634722300 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5583d3418d50 msgr2=0x5583d342cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:06.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.083+0000 7fdc01d6bb80 1 --2- 172.21.15.100:0/1634722300 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5583d3418d50 0x5583d342cec0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fdbec0dff30 tx=0x7fdbec0a9470 comp rx=0 tx=0).stop 2024-05-10T11:44:06.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.083+0000 7fdc01d6bb80 1 -- 172.21.15.100:0/1634722300 shutdown_connections 2024-05-10T11:44:06.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.083+0000 7fdc01d6bb80 1 --2- 172.21.15.100:0/1634722300 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x5583d343aa70 0x5583d343ce60 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:06.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.084+0000 7fdbd8ff9700 0 client.29653 ms_handle_reset on v2:172.21.15.188:6836/1943342620 2024-05-10T11:44:06.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.084+0000 7fdc01d6bb80 1 --2- 172.21.15.100:0/1634722300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583d3427380 0x5583d342a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:06.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.084+0000 7fdc01d6bb80 1 --2- 172.21.15.100:0/1634722300 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5583d3418d50 0x5583d342cec0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:06.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.084+0000 7fdc01d6bb80 1 --2- 172.21.15.100:0/1634722300 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583d3426a60 0x5583d342a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:06.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.084+0000 7fdc01d6bb80 1 -- 172.21.15.100:0/1634722300 >> 172.21.15.100:0/1634722300 conn(0x5583d3344b70 msgr2=0x5583d34131b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:44:06.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.084+0000 7fdc01d6bb80 1 -- 172.21.15.100:0/1634722300 shutdown_connections 2024-05-10T11:44:06.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.084+0000 7fdc01d6bb80 1 -- 172.21.15.100:0/1634722300 wait complete. 2024-05-10T11:44:06.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.084+0000 7fdc01d6bb80 20 client.29653 trim_cache size 0 max 16384 2024-05-10T11:44:06.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.109+0000 7fdc01d6bb80 1 Processor -- start 2024-05-10T11:44:06.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.109+0000 7fdc01d6bb80 1 -- start start 2024-05-10T11:44:06.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.109+0000 7fdc01d6bb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583d3418d50 0x5583d3418810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:06.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.109+0000 7fdc01d6bb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5583d3426a60 0x5583d34291c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:06.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.109+0000 7fdc01d6bb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583d3427380 0x5583d3436c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:06.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.109+0000 7fdc01d6bb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf01af760 con 0x5583d3418d50 2024-05-10T11:44:06.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.109+0000 7fdc01d6bb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf01acff0 con 0x5583d3427380 2024-05-10T11:44:06.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.109+0000 7fdc01d6bb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf01aae60 con 0x5583d3426a60 2024-05-10T11:44:06.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.109+0000 7fdbf5274700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583d3418d50 0x5583d3418810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:06.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.109+0000 7fdbf5274700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583d3418d50 0x5583d3418810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43710/0 (socket says 172.21.15.100:43710) 2024-05-10T11:44:06.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.109+0000 7fdbf4a73700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5583d3426a60 0x5583d34291c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:06.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.109+0000 7fdbf5274700 1 -- 172.21.15.100:0/1840019684 learned_addr learned my addr 172.21.15.100:0/1840019684 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:44:06.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.109+0000 7fdbf5a75700 1 --2- 172.21.15.100:0/1840019684 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583d3427380 0x5583d3436c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:06.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.109+0000 7fdbf5a75700 1 -- 172.21.15.100:0/1840019684 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5583d3426a60 msgr2=0x5583d34291c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:44:06.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.109+0000 7fdbf5a75700 1 --2- 172.21.15.100:0/1840019684 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5583d3426a60 0x5583d34291c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:06.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.110+0000 7fdbf5a75700 1 -- 172.21.15.100:0/1840019684 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583d3418d50 msgr2=0x5583d3418810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:44:06.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.110+0000 7fdbf5a75700 1 --2- 172.21.15.100:0/1840019684 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583d3418d50 0x5583d3418810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:06.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.110+0000 7fdbf5a75700 1 -- 172.21.15.100:0/1840019684 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583d342d230 con 0x5583d3427380 2024-05-10T11:44:06.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.110+0000 7fdbf5a75700 1 --2- 172.21.15.100:0/1840019684 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583d3427380 0x5583d3436c60 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x5583d3415340 tx=0x7fdbe8009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24d0b917f30d7200 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:44:06.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.110+0000 7fdbf4a73700 1 --2- 172.21.15.100:0/1840019684 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5583d3426a60 0x5583d34291c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:44:06.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.110+0000 7fdbf5274700 1 --2- 172.21.15.100:0/1840019684 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583d3418d50 0x5583d3418810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:44:06.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.110+0000 7fdbdaffd700 1 -- 172.21.15.100:0/1840019684 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe8017230 con 0x5583d3427380 2024-05-10T11:44:06.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.110+0000 7fdbdaffd700 1 -- 172.21.15.100:0/1840019684 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdbe8004050 con 0x5583d3427380 2024-05-10T11:44:06.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.110+0000 7fdbdaffd700 1 -- 172.21.15.100:0/1840019684 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe80093e0 con 0x5583d3427380 2024-05-10T11:44:06.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.111+0000 7fdc01d6bb80 1 -- 172.21.15.100:0/1840019684 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583d3427380 msgr2=0x5583d3436c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:06.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.111+0000 7fdc01d6bb80 1 --2- 172.21.15.100:0/1840019684 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583d3427380 0x5583d3436c60 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x5583d3415340 tx=0x7fdbe8009920 comp rx=0 tx=0).stop 2024-05-10T11:44:06.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.111+0000 7fdc01d6bb80 1 -- 172.21.15.100:0/1840019684 shutdown_connections 2024-05-10T11:44:06.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.111+0000 7fdc01d6bb80 1 --2- 172.21.15.100:0/1840019684 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583d3427380 0x5583d3436c60 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:06.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.111+0000 7fdc01d6bb80 1 --2- 172.21.15.100:0/1840019684 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583d3418d50 0x5583d3418810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:06.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.111+0000 7fdc01d6bb80 1 --2- 172.21.15.100:0/1840019684 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5583d3426a60 0x5583d34291c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:06.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.111+0000 7fdc01d6bb80 1 -- 172.21.15.100:0/1840019684 >> 172.21.15.100:0/1840019684 conn(0x5583d3344b70 msgr2=0x5583d3429b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:44:06.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.111+0000 7fdc01d6bb80 1 -- 172.21.15.100:0/1840019684 shutdown_connections 2024-05-10T11:44:06.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.111+0000 7fdc01d6bb80 1 -- 172.21.15.100:0/1840019684 wait complete. 2024-05-10T11:44:06.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.111+0000 7fdc01d6bb80 1 -- start start 2024-05-10T11:44:06.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.111+0000 7fdc01d6bb80 1 Processor -- start 2024-05-10T11:44:06.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.112+0000 7fdc01d6bb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583d3418d50 0x5583d3441e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:06.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.112+0000 7fdc01d6bb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5583d3426a60 0x5583d3442340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:06.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.112+0000 7fdbf5274700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583d3418d50 0x5583d3441e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:06.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.112+0000 7fdbf5274700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583d3418d50 0x5583d3441e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43722/0 (socket says 172.21.15.100:43722) 2024-05-10T11:44:06.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.112+0000 7fdc01d6bb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583d3427380 0x5583d3442880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:06.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.112+0000 7fdbf5274700 1 -- 172.21.15.100:0/850396263 learned_addr learned my addr 172.21.15.100:0/850396263 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:44:06.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.112+0000 7fdc01d6bb80 1 -- 172.21.15.100:0/850396263 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf01ab4b0 con 0x5583d3418d50 2024-05-10T11:44:06.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.112+0000 7fdc01d6bb80 1 -- 172.21.15.100:0/850396263 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf01ac920 con 0x5583d3427380 2024-05-10T11:44:06.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.112+0000 7fdc01d6bb80 1 -- 172.21.15.100:0/850396263 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf01abb80 con 0x5583d3426a60 2024-05-10T11:44:06.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.112+0000 7fdc01d6bb80 10 client.0 _ll_register_callbacks cb 0x5583d32fe5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:44:06.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.112+0000 7fdbf4a73700 1 --2- 172.21.15.100:0/850396263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5583d3426a60 0x5583d3442340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:06.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.112+0000 7fdbf5a75700 1 --2- 172.21.15.100:0/850396263 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583d3427380 0x5583d3442880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:06.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.112+0000 7fdbf5274700 1 -- 172.21.15.100:0/850396263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5583d3426a60 msgr2=0x5583d3442340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:44:06.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.112+0000 7fdbf5274700 1 --2- 172.21.15.100:0/850396263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5583d3426a60 0x5583d3442340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:06.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.112+0000 7fdbf5274700 1 -- 172.21.15.100:0/850396263 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583d3427380 msgr2=0x5583d3442880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:44:06.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.112+0000 7fdbf5274700 1 --2- 172.21.15.100:0/850396263 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583d3427380 0x5583d3442880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:06.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.112+0000 7fdbf5274700 1 -- 172.21.15.100:0/850396263 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583d34a41f0 con 0x5583d3418d50 2024-05-10T11:44:06.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.113+0000 7fdbf5274700 1 --2- 172.21.15.100:0/850396263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583d3418d50 0x5583d3441e00 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fdbec0a6c70 tx=0x7fdbec0cc2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=271fae6e856ba2df server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:44:06.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.113+0000 7fdbd8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:44:06.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.113+0000 7fdbd8ff9700 1 -- 172.21.15.100:0/850396263 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbec00ce90 con 0x5583d3418d50 2024-05-10T11:44:06.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.113+0000 7fdc01d6bb80 1 -- 172.21.15.100:0/850396263 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbe8004050 con 0x5583d3418d50 2024-05-10T11:44:06.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.113+0000 7fdbd8ff9700 1 -- 172.21.15.100:0/850396263 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fdbec0a5ff0 con 0x5583d3418d50 2024-05-10T11:44:06.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.113+0000 7fdbd8ff9700 1 -- 172.21.15.100:0/850396263 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fdbec0a9510 con 0x5583d3418d50 2024-05-10T11:44:06.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.113+0000 7fdc01d6bb80 10 client.20298 fetch_fsmap learned FSMap version 41 2024-05-10T11:44:06.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.113+0000 7fdc01d6bb80 1 -- 172.21.15.100:0/850396263 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5583d34a54c0 con 0x5583d3418d50 2024-05-10T11:44:06.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.113+0000 7fdbd8ff9700 1 -- 172.21.15.100:0/850396263 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdbe8004050 con 0x5583d3418d50 2024-05-10T11:44:06.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.114+0000 7fdbd8ff9700 1 -- 172.21.15.100:0/850396263 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fdbec0c5040 con 0x5583d3418d50 2024-05-10T11:44:06.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.114+0000 7fdc01d6bb80 10 client.20298 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:44:06.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.114+0000 7fdc01d6bb80 10 client.20298 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-10T11:44:06.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.114+0000 7fdc01d6bb80 20 client.20298 populate_metadata read hostname 'smithi100' 2024-05-10T11:44:06.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.114+0000 7fdc01d6bb80 1 --2- 172.21.15.100:0/850396263 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x5583d34a5bf0 0x5583d34a8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:06.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.114+0000 7fdc01d6bb80 4 client.20298 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:44:06.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.114+0000 7fdc01d6bb80 1 -- 172.21.15.100:0/850396263 --> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5583d33723c0 con 0x5583d34a5bf0 2024-05-10T11:44:06.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.114+0000 7fdbf4a73700 1 --2- 172.21.15.100:0/850396263 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x5583d34a5bf0 0x5583d34a8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:06.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.114+0000 7fdbf4a73700 1 --2- 172.21.15.100:0/850396263 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x5583d34a5bf0 0x5583d34a8010 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8689c69b64770f59 server_cookie=9bcef198ef758c97 in_seq=0 out_seq=0 2024-05-10T11:44:06.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.114+0000 7fdbd8ff9700 10 client.20298 ms_handle_connect on v2:172.21.15.188:6836/1943342620 2024-05-10T11:44:06.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.115+0000 7fdbd8ff9700 1 -- 172.21.15.100:0/850396263 <== mds.2 v2:172.21.15.188:6836/1943342620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5583d33723c0 con 0x5583d34a5bf0 2024-05-10T11:44:06.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.115+0000 7fdbd8ff9700 10 client.20298 handle_command_reply: tid=0 2024-05-10T11:44:06.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.115+0000 7fdc01d6bb80 1 client.20298 shutdown 2024-05-10T11:44:06.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.116+0000 7fdc01d6bb80 1 -- 172.21.15.100:0/850396263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583d3418d50 msgr2=0x5583d3441e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:06.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.116+0000 7fdc01d6bb80 1 --2- 172.21.15.100:0/850396263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583d3418d50 0x5583d3441e00 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fdbec0a6c70 tx=0x7fdbec0cc2c0 comp rx=0 tx=0).stop 2024-05-10T11:44:06.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.116+0000 7fdc01d6bb80 1 -- 172.21.15.100:0/850396263 shutdown_connections 2024-05-10T11:44:06.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.116+0000 7fdc01d6bb80 1 --2- 172.21.15.100:0/850396263 >> [v2:172.21.15.188:6836/1943342620,v1:172.21.15.188:6837/1943342620] conn(0x5583d34a5bf0 0x5583d34a8010 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:06.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.116+0000 7fdc01d6bb80 1 --2- 172.21.15.100:0/850396263 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583d3427380 0x5583d3442880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:06.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.116+0000 7fdc01d6bb80 1 --2- 172.21.15.100:0/850396263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583d3418d50 0x5583d3441e00 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:06.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.116+0000 7fdc01d6bb80 1 --2- 172.21.15.100:0/850396263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5583d3426a60 0x5583d3442340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:06.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.116+0000 7fdc01d6bb80 1 -- 172.21.15.100:0/850396263 >> 172.21.15.100:0/850396263 conn(0x5583d3344b70 msgr2=0x5583d342a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:44:06.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.116+0000 7fdbd8ff9700 0 client.20298 ms_handle_reset on v2:172.21.15.188:6836/1943342620 2024-05-10T11:44:06.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.116+0000 7fdc01d6bb80 1 -- 172.21.15.100:0/850396263 shutdown_connections 2024-05-10T11:44:06.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.116+0000 7fdc01d6bb80 1 -- 172.21.15.100:0/850396263 wait complete. 2024-05-10T11:44:06.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.116+0000 7fdc01d6bb80 20 client.20298 trim_cache size 0 max 16384 2024-05-10T11:44:06.120 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:44:06.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.117+0000 7fdbf74d8700 1 -- 172.21.15.100:0/577634659 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fdbdc0b2900 msgr2=0x7fdbdc0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:06.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.117+0000 7fdbf74d8700 1 --2- 172.21.15.100:0/577634659 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fdbdc0b2900 0x7fdbdc0b4dc0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fdbf019aa80 tx=0x7fdbe0005fb0 comp rx=0 tx=0).stop 2024-05-10T11:44:06.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.117+0000 7fdbf74d8700 1 -- 172.21.15.100:0/577634659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdbf0102390 msgr2=0x7fdbf0199b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:06.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.117+0000 7fdbf74d8700 1 --2- 172.21.15.100:0/577634659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdbf0102390 0x7fdbf0199b10 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fdbec00eb30 tx=0x7fdbec00ee40 comp rx=0 tx=0).stop 2024-05-10T11:44:06.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.117+0000 7fdbf74d8700 1 -- 172.21.15.100:0/577634659 shutdown_connections 2024-05-10T11:44:06.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.117+0000 7fdbf74d8700 1 --2- 172.21.15.100:0/577634659 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fdbdc0b2900 0x7fdbdc0b4dc0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:06.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.117+0000 7fdbf74d8700 1 --2- 172.21.15.100:0/577634659 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdbf0108400 0x7fdbf019e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:06.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.117+0000 7fdbf74d8700 1 --2- 172.21.15.100:0/577634659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdbf0102390 0x7fdbf0199b10 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:06.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.117+0000 7fdbf74d8700 1 --2- 172.21.15.100:0/577634659 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbf0102d50 0x7fdbf019a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:06.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.117+0000 7fdbf74d8700 1 -- 172.21.15.100:0/577634659 >> 172.21.15.100:0/577634659 conn(0x7fdbf0079310 msgr2=0x7fdbf0106760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:44:06.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.117+0000 7fdbf74d8700 1 -- 172.21.15.100:0/577634659 shutdown_connections 2024-05-10T11:44:06.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:06.117+0000 7fdbf74d8700 1 -- 172.21.15.100:0/577634659 wait complete. 2024-05-10T11:44:06.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:44:05 smithi178 ceph-mon[33359]: pgmap v1192: 129 pgs: 129 active+clean; 567 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 4.2 MiB/s wr, 275 op/s 2024-05-10T11:44:06.897 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-10T11:44:06.962 INFO:tasks.workunit.client.0.smithi100.stdout: 10 cleanup 600 sec 2024-05-10T11:44:07.229 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:44:07.962 INFO:tasks.workunit.client.0.smithi100.stdout: 4 cleanup 601 sec 2024-05-10T11:44:08.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:44:07 smithi188 ceph-mon[34075]: pgmap v1193: 129 pgs: 129 active+clean; 566 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 4.0 MiB/s wr, 302 op/s 2024-05-10T11:44:08.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:44:07 smithi100 ceph-mon[30296]: pgmap v1193: 129 pgs: 129 active+clean; 566 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 4.0 MiB/s wr, 302 op/s 2024-05-10T11:44:08.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:44:07 smithi178 ceph-mon[33359]: pgmap v1193: 129 pgs: 129 active+clean; 566 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 4.0 MiB/s wr, 302 op/s 2024-05-10T11:44:08.648 INFO:tasks.workunit.client.0.smithi100.stdout: 0 cleanup 602 sec 2024-05-10T11:44:08.648 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-05-10T11:44:08.649 INFO:tasks.workunit.client.0.smithi100.stdout: Operation Count AvgLat MaxLat 2024-05-10T11:44:08.649 INFO:tasks.workunit.client.0.smithi100.stdout: ---------------------------------------- 2024-05-10T11:44:08.649 INFO:tasks.workunit.client.0.smithi100.stdout: NTCreateX 170809 9.964 711.853 2024-05-10T11:44:08.649 INFO:tasks.workunit.client.0.smithi100.stdout: Close 125469 0.192 94.103 2024-05-10T11:44:08.649 INFO:tasks.workunit.client.0.smithi100.stdout: Rename 7225 21.693 479.709 2024-05-10T11:44:08.649 INFO:tasks.workunit.client.0.smithi100.stdout: Unlink 34492 9.004 604.495 2024-05-10T11:44:08.649 INFO:tasks.workunit.client.0.smithi100.stdout: Qpathinfo 154904 6.090 524.538 2024-05-10T11:44:08.649 INFO:tasks.workunit.client.0.smithi100.stdout: Qfileinfo 26999 0.354 78.546 2024-05-10T11:44:08.649 INFO:tasks.workunit.client.0.smithi100.stdout: Qfsinfo 28374 0.998 105.081 2024-05-10T11:44:08.649 INFO:tasks.workunit.client.0.smithi100.stdout: Sfileinfo 13922 2.989 154.829 2024-05-10T11:44:08.649 INFO:tasks.workunit.client.0.smithi100.stdout: Find 59856 43.560 796.054 2024-05-10T11:44:08.649 INFO:tasks.workunit.client.0.smithi100.stdout: WriteX 84357 0.644 131.570 2024-05-10T11:44:08.649 INFO:tasks.workunit.client.0.smithi100.stdout: ReadX 267921 0.009 119.378 2024-05-10T11:44:08.649 INFO:tasks.workunit.client.0.smithi100.stdout: LockX 554 2.236 54.631 2024-05-10T11:44:08.650 INFO:tasks.workunit.client.0.smithi100.stdout: UnlockX 554 1.814 24.019 2024-05-10T11:44:08.650 INFO:tasks.workunit.client.0.smithi100.stdout: Flush 11961 9.534 517.838 2024-05-10T11:44:08.650 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-05-10T11:44:08.650 INFO:tasks.workunit.client.0.smithi100.stdout:Throughput 8.86034 MB/sec 10 clients 10 procs max_latency=796.057 ms 2024-05-10T11:44:08.650 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-10T11:44:08.650 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-10T11:44:09.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.220+0000 7f7af4df9700 1 -- 172.21.15.100:0/2486919326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7af01034c0 msgr2=0x7f7af010f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:09.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.220+0000 7f7af4df9700 1 --2- 172.21.15.100:0/2486919326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7af01034c0 0x7f7af010f240 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f7ae800b0a0 tx=0x7f7ae800b3b0 comp rx=0 tx=0).stop 2024-05-10T11:44:09.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.220+0000 7f7af4df9700 1 -- 172.21.15.100:0/2486919326 shutdown_connections 2024-05-10T11:44:09.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.220+0000 7f7af4df9700 1 --2- 172.21.15.100:0/2486919326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7af01034c0 0x7f7af010f240 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:09.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.220+0000 7f7af4df9700 1 --2- 172.21.15.100:0/2486919326 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7af0108470 0x7f7af0108850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:09.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.220+0000 7f7af4df9700 1 --2- 172.21.15.100:0/2486919326 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7af0102b00 0x7f7af0102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:09.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.220+0000 7f7af4df9700 1 -- 172.21.15.100:0/2486919326 >> 172.21.15.100:0/2486919326 conn(0x7f7af00789a0 msgr2=0x7f7af00ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:44:09.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.221+0000 7f7af4df9700 1 -- 172.21.15.100:0/2486919326 shutdown_connections 2024-05-10T11:44:09.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.221+0000 7f7af4df9700 1 -- 172.21.15.100:0/2486919326 wait complete. 2024-05-10T11:44:09.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.221+0000 7f7af4df9700 1 Processor -- start 2024-05-10T11:44:09.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.221+0000 7f7af4df9700 1 -- start start 2024-05-10T11:44:09.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.222+0000 7f7af4df9700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7af0102b00 0x7f7af019e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:09.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.222+0000 7f7af4df9700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7af01034c0 0x7f7af019ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:09.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.222+0000 7f7af4df9700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7af0108470 0x7f7af01987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:09.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.222+0000 7f7af4df9700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7af0112430 con 0x7f7af0102b00 2024-05-10T11:44:09.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.222+0000 7f7af4df9700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7af01122b0 con 0x7f7af0108470 2024-05-10T11:44:09.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.222+0000 7f7af4df9700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7af01125b0 con 0x7f7af01034c0 2024-05-10T11:44:09.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.222+0000 7f7aee59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7af0102b00 0x7f7af019e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:09.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.222+0000 7f7aee59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7af0102b00 0x7f7af019e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34528/0 (socket says 172.21.15.100:34528) 2024-05-10T11:44:09.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.222+0000 7f7aee59c700 1 -- 172.21.15.100:0/2231732993 learned_addr learned my addr 172.21.15.100:0/2231732993 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:44:09.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.222+0000 7f7aeed9d700 1 --2- 172.21.15.100:0/2231732993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7af0108470 0x7f7af01987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:09.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.222+0000 7f7aedd9b700 1 --2- 172.21.15.100:0/2231732993 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7af01034c0 0x7f7af019ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:09.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.222+0000 7f7aee59c700 1 -- 172.21.15.100:0/2231732993 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7af01034c0 msgr2=0x7f7af019ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:09.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.222+0000 7f7aee59c700 1 --2- 172.21.15.100:0/2231732993 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7af01034c0 0x7f7af019ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:09.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.222+0000 7f7aee59c700 1 -- 172.21.15.100:0/2231732993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7af0108470 msgr2=0x7f7af01987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:09.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.222+0000 7f7aee59c700 1 --2- 172.21.15.100:0/2231732993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7af0108470 0x7f7af01987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:09.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.222+0000 7f7aee59c700 1 -- 172.21.15.100:0/2231732993 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7af0198fd0 con 0x7f7af0102b00 2024-05-10T11:44:09.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.223+0000 7f7aee59c700 1 --2- 172.21.15.100:0/2231732993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7af0102b00 0x7f7af019e670 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4007910 tx=0x7f7ae4007c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:44:09.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.223+0000 7f7adf7fe700 1 -- 172.21.15.100:0/2231732993 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4004020 con 0x7f7af0102b00 2024-05-10T11:44:09.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.223+0000 7f7adf7fe700 1 -- 172.21.15.100:0/2231732993 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7ae4004830 con 0x7f7af0102b00 2024-05-10T11:44:09.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.223+0000 7f7adf7fe700 1 -- 172.21.15.100:0/2231732993 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4005480 con 0x7f7af0102b00 2024-05-10T11:44:09.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.223+0000 7f7af4df9700 1 -- 172.21.15.100:0/2231732993 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7af0199260 con 0x7f7af0102b00 2024-05-10T11:44:09.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.223+0000 7f7af4df9700 1 -- 172.21.15.100:0/2231732993 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7af01019f0 con 0x7f7af0102b00 2024-05-10T11:44:09.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.226+0000 7f7adf7fe700 1 -- 172.21.15.100:0/2231732993 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f7ae401f020 con 0x7f7af0102b00 2024-05-10T11:44:09.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.226+0000 7f7adf7fe700 1 --2- 172.21.15.100:0/2231732993 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f7ad80b2900 0x7f7ad80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:09.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.226+0000 7f7aedd9b700 1 --2- 172.21.15.100:0/2231732993 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f7ad80b2900 0x7f7ad80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:09.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.226+0000 7f7adf7fe700 1 -- 172.21.15.100:0/2231732993 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f7ae40e3540 con 0x7f7af0102b00 2024-05-10T11:44:09.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.227+0000 7f7aff68cb80 1 Processor -- start 2024-05-10T11:44:09.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.227+0000 7f7aedd9b700 1 --2- 172.21.15.100:0/2231732993 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f7ad80b2900 0x7f7ad80b4dc0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f7af0107bd0 tx=0x7f7ae0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:44:09.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.227+0000 7f7aff68cb80 1 -- start start 2024-05-10T11:44:09.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.227+0000 7f7aff68cb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5648b028b1b0 0x5648b028b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:09.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.227+0000 7f7aff68cb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5648b028bad0 0x5648b0285810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:09.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.227+0000 7f7aff68cb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648b0285d50 0x5648b0286190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:09.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.227+0000 7f7aff68cb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5648b01d7940 con 0x5648b028b1b0 2024-05-10T11:44:09.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.227+0000 7f7aff68cb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5648b013cda0 con 0x5648b0285d50 2024-05-10T11:44:09.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.227+0000 7f7aff68cb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5648b01d3900 con 0x5648b028bad0 2024-05-10T11:44:09.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.227+0000 7f7aee59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5648b028b1b0 0x5648b028b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:09.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.227+0000 7f7aee59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5648b028b1b0 0x5648b028b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34534/0 (socket says 172.21.15.100:34534) 2024-05-10T11:44:09.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.227+0000 7f7aee59c700 1 -- 172.21.15.100:0/3941572889 learned_addr learned my addr 172.21.15.100:0/3941572889 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:44:09.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.227+0000 7f7aedd9b700 1 --2- 172.21.15.100:0/3941572889 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5648b028bad0 0x5648b0285810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:09.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.227+0000 7f7aeed9d700 1 --2- 172.21.15.100:0/3941572889 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648b0285d50 0x5648b0286190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:09.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.228+0000 7f7aee59c700 1 -- 172.21.15.100:0/3941572889 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5648b028bad0 msgr2=0x5648b0285810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:44:09.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.228+0000 7f7aee59c700 1 --2- 172.21.15.100:0/3941572889 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5648b028bad0 0x5648b0285810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:09.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.228+0000 7f7aee59c700 1 -- 172.21.15.100:0/3941572889 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648b0285d50 msgr2=0x5648b0286190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:44:09.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.228+0000 7f7aee59c700 1 --2- 172.21.15.100:0/3941572889 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648b0285d50 0x5648b0286190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:09.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.228+0000 7f7aee59c700 1 -- 172.21.15.100:0/3941572889 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648b01248d0 con 0x5648b028b1b0 2024-05-10T11:44:09.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.228+0000 7f7aee59c700 1 --2- 172.21.15.100:0/3941572889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5648b028b1b0 0x5648b028b590 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f7ae40a7a70 tx=0x7f7ae40aa3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed392abcd368d71a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:44:09.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.228+0000 7f7adcff9700 1 -- 172.21.15.100:0/3941572889 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae401c070 con 0x5648b028b1b0 2024-05-10T11:44:09.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.228+0000 7f7adcff9700 1 -- 172.21.15.100:0/3941572889 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7ae40aebb0 con 0x5648b028b1b0 2024-05-10T11:44:09.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.228+0000 7f7adcff9700 1 -- 172.21.15.100:0/3941572889 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae40a6a90 con 0x5648b028b1b0 2024-05-10T11:44:09.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.229+0000 7f7aff68cb80 1 -- 172.21.15.100:0/3941572889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5648b028b1b0 msgr2=0x5648b028b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:09.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.229+0000 7f7aff68cb80 1 --2- 172.21.15.100:0/3941572889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5648b028b1b0 0x5648b028b590 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f7ae40a7a70 tx=0x7f7ae40aa3e0 comp rx=0 tx=0).stop 2024-05-10T11:44:09.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.229+0000 7f7aff68cb80 1 -- 172.21.15.100:0/3941572889 shutdown_connections 2024-05-10T11:44:09.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.229+0000 7f7aff68cb80 1 --2- 172.21.15.100:0/3941572889 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648b0285d50 0x5648b0286190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:09.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.229+0000 7f7aff68cb80 1 --2- 172.21.15.100:0/3941572889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5648b028b1b0 0x5648b028b590 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:09.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.229+0000 7f7aff68cb80 1 --2- 172.21.15.100:0/3941572889 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5648b028bad0 0x5648b0285810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:09.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.229+0000 7f7aff68cb80 1 -- 172.21.15.100:0/3941572889 >> 172.21.15.100:0/3941572889 conn(0x5648b01b1b70 msgr2=0x5648b02802a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:44:09.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.229+0000 7f7aff68cb80 1 -- 172.21.15.100:0/3941572889 shutdown_connections 2024-05-10T11:44:09.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.229+0000 7f7aff68cb80 1 -- 172.21.15.100:0/3941572889 wait complete. 2024-05-10T11:44:09.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.229+0000 7f7aff68cb80 1 -- start start 2024-05-10T11:44:09.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.229+0000 7f7aff68cb80 1 Processor -- start 2024-05-10T11:44:09.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.230+0000 7f7aff68cb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648b0285d50 0x5648b02917f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:09.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.230+0000 7f7aff68cb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5648b028b1b0 0x5648b028eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:09.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.230+0000 7f7aff68cb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5648b028bad0 0x5648b028f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:09.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.230+0000 7f7aff68cb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5648b0292450 con 0x5648b028bad0 2024-05-10T11:44:09.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.230+0000 7f7aff68cb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5648b0190ed0 con 0x5648b0285d50 2024-05-10T11:44:09.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.230+0000 7f7aff68cb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5648b02925d0 con 0x5648b028b1b0 2024-05-10T11:44:09.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.230+0000 7f7aff68cb80 10 client.0 _ll_register_callbacks cb 0x5648b016b5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:44:09.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.230+0000 7f7aeed9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5648b028bad0 0x5648b028f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:09.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.230+0000 7f7aee59c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648b0285d50 0x5648b02917f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:09.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.230+0000 7f7aeed9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5648b028bad0 0x5648b028f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34550/0 (socket says 172.21.15.100:34550) 2024-05-10T11:44:09.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.230+0000 7f7aee59c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648b0285d50 0x5648b02917f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:56646/0 (socket says 172.21.15.100:56646) 2024-05-10T11:44:09.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.230+0000 7f7aeed9d700 1 -- 172.21.15.100:0/3657616410 learned_addr learned my addr 172.21.15.100:0/3657616410 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:44:09.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.230+0000 7f7aedd9b700 1 --2- 172.21.15.100:0/3657616410 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5648b028b1b0 0x5648b028eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:09.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.230+0000 7f7aeed9d700 1 -- 172.21.15.100:0/3657616410 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5648b028b1b0 msgr2=0x5648b028eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:44:09.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.230+0000 7f7aeed9d700 1 --2- 172.21.15.100:0/3657616410 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5648b028b1b0 0x5648b028eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:09.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.230+0000 7f7aeed9d700 1 -- 172.21.15.100:0/3657616410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648b0285d50 msgr2=0x5648b02917f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:44:09.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.230+0000 7f7aeed9d700 1 --2- 172.21.15.100:0/3657616410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648b0285d50 0x5648b02917f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:09.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.230+0000 7f7aeed9d700 1 -- 172.21.15.100:0/3657616410 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648b018e760 con 0x5648b028bad0 2024-05-10T11:44:09.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.231+0000 7f7aeed9d700 1 --2- 172.21.15.100:0/3657616410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5648b028bad0 0x5648b028f0d0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f7ae800be30 tx=0x7f7ae800be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62eaabc4adbce202 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:44:09.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.231+0000 7f7ad5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:44:09.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.231+0000 7f7ad5ffb700 1 -- 172.21.15.100:0/3657616410 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae8004da0 con 0x5648b028bad0 2024-05-10T11:44:09.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.231+0000 7f7ad5ffb700 1 -- 172.21.15.100:0/3657616410 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f7ae801f7b0 con 0x5648b028bad0 2024-05-10T11:44:09.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.231+0000 7f7aff68cb80 1 -- 172.21.15.100:0/3657616410 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ae40aebb0 con 0x5648b028bad0 2024-05-10T11:44:09.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.231+0000 7f7ad5ffb700 1 -- 172.21.15.100:0/3657616410 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f7ae801f9b0 con 0x5648b028bad0 2024-05-10T11:44:09.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.231+0000 7f7aff68cb80 10 client.20310 fetch_fsmap learned FSMap version 41 2024-05-10T11:44:09.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.231+0000 7f7aff68cb80 1 -- 172.21.15.100:0/3657616410 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5648b029ce50 con 0x5648b028bad0 2024-05-10T11:44:09.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.231+0000 7f7ad5ffb700 1 -- 172.21.15.100:0/3657616410 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7ae40aebb0 con 0x5648b028bad0 2024-05-10T11:44:09.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.231+0000 7f7ad5ffb700 1 -- 172.21.15.100:0/3657616410 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f7ae8011070 con 0x5648b028bad0 2024-05-10T11:44:09.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.231+0000 7f7aff68cb80 10 client.20310 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:44:09.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.231+0000 7f7aff68cb80 10 client.20310 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-10T11:44:09.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.232+0000 7f7aff68cb80 20 client.20310 populate_metadata read hostname 'smithi100' 2024-05-10T11:44:09.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.232+0000 7f7aff68cb80 1 --2- 172.21.15.100:0/3657616410 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x5648b029f4e0 0x5648b02a1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:09.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.232+0000 7f7aff68cb80 4 client.20310 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:44:09.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.232+0000 7f7aff68cb80 1 -- 172.21.15.100:0/3657616410 --> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648b01e5320 con 0x5648b029f4e0 2024-05-10T11:44:09.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.232+0000 7f7aee59c700 1 --2- 172.21.15.100:0/3657616410 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x5648b029f4e0 0x5648b02a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:09.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.232+0000 7f7aee59c700 1 --2- 172.21.15.100:0/3657616410 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x5648b029f4e0 0x5648b02a1910 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fd933e6d2585ea65 server_cookie=bf9becb62d36a1fd in_seq=0 out_seq=0 2024-05-10T11:44:09.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.232+0000 7f7ad5ffb700 10 client.20310 ms_handle_connect on v2:172.21.15.178:6838/3760985404 2024-05-10T11:44:09.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.233+0000 7f7ad5ffb700 1 -- 172.21.15.100:0/3657616410 <== mds.1 v2:172.21.15.178:6838/3760985404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5648b01e5320 con 0x5648b029f4e0 2024-05-10T11:44:09.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.233+0000 7f7ad5ffb700 10 client.20310 handle_command_reply: tid=0 2024-05-10T11:44:09.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.233+0000 7f7aff68cb80 1 client.20310 shutdown 2024-05-10T11:44:09.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.233+0000 7f7aff68cb80 1 -- 172.21.15.100:0/3657616410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5648b028bad0 msgr2=0x5648b028f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:09.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.233+0000 7f7aff68cb80 1 --2- 172.21.15.100:0/3657616410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5648b028bad0 0x5648b028f0d0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f7ae800be30 tx=0x7f7ae800be60 comp rx=0 tx=0).stop 2024-05-10T11:44:09.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.233+0000 7f7aff68cb80 1 -- 172.21.15.100:0/3657616410 shutdown_connections 2024-05-10T11:44:09.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.233+0000 7f7aff68cb80 1 --2- 172.21.15.100:0/3657616410 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x5648b029f4e0 0x5648b02a1910 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:09.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.233+0000 7f7aff68cb80 1 --2- 172.21.15.100:0/3657616410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5648b028bad0 0x5648b028f0d0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:09.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.233+0000 7f7aff68cb80 1 --2- 172.21.15.100:0/3657616410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648b0285d50 0x5648b02917f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:09.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.233+0000 7f7aff68cb80 1 --2- 172.21.15.100:0/3657616410 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5648b028b1b0 0x5648b028eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:09.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.233+0000 7f7aff68cb80 1 -- 172.21.15.100:0/3657616410 >> 172.21.15.100:0/3657616410 conn(0x5648b01b1b70 msgr2=0x5648b027ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:44:09.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.233+0000 7f7ad5ffb700 0 client.20310 ms_handle_reset on v2:172.21.15.178:6838/3760985404 2024-05-10T11:44:09.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.233+0000 7f7aff68cb80 1 -- 172.21.15.100:0/3657616410 shutdown_connections 2024-05-10T11:44:09.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.233+0000 7f7aff68cb80 1 -- 172.21.15.100:0/3657616410 wait complete. 2024-05-10T11:44:09.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.233+0000 7f7aff68cb80 20 client.20310 trim_cache size 0 max 16384 2024-05-10T11:44:09.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.248+0000 7f7aff68cb80 1 Processor -- start 2024-05-10T11:44:09.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.248+0000 7f7aff68cb80 1 -- start start 2024-05-10T11:44:09.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.248+0000 7f7aff68cb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648b0285d50 0x5648b0285810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:09.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.248+0000 7f7aff68cb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5648b028b1b0 0x5648b027ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:09.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.248+0000 7f7aff68cb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5648b028bad0 0x5648b029b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:09.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.248+0000 7f7aff68cb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7af01af700 con 0x5648b028b1b0 2024-05-10T11:44:09.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.248+0000 7f7aff68cb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7af01acf90 con 0x5648b0285d50 2024-05-10T11:44:09.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.248+0000 7f7aff68cb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7af01aade0 con 0x5648b028bad0 2024-05-10T11:44:09.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.249+0000 7f7aedd9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5648b028b1b0 0x5648b027ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:09.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.249+0000 7f7aedd9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5648b028b1b0 0x5648b027ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34560/0 (socket says 172.21.15.100:34560) 2024-05-10T11:44:09.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.249+0000 7f7aedd9b700 1 -- 172.21.15.100:0/4030025556 learned_addr learned my addr 172.21.15.100:0/4030025556 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:44:09.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.249+0000 7f7aee59c700 1 --2- 172.21.15.100:0/4030025556 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648b0285d50 0x5648b0285810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:09.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.249+0000 7f7aeed9d700 1 --2- 172.21.15.100:0/4030025556 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5648b028bad0 0x5648b029b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:09.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.249+0000 7f7aedd9b700 1 -- 172.21.15.100:0/4030025556 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5648b028bad0 msgr2=0x5648b029b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:44:09.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.249+0000 7f7aedd9b700 1 --2- 172.21.15.100:0/4030025556 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5648b028bad0 0x5648b029b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:09.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.249+0000 7f7aedd9b700 1 -- 172.21.15.100:0/4030025556 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648b0285d50 msgr2=0x5648b0285810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:44:09.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.249+0000 7f7aedd9b700 1 --2- 172.21.15.100:0/4030025556 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648b0285d50 0x5648b0285810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:09.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.249+0000 7f7aedd9b700 1 -- 172.21.15.100:0/4030025556 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648b0286650 con 0x5648b028b1b0 2024-05-10T11:44:09.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.249+0000 7f7aedd9b700 1 --2- 172.21.15.100:0/4030025556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5648b028b1b0 0x5648b027ac40 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x5648b0281040 tx=0x7f7ae0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a480efc507f9d3e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:44:09.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.249+0000 7f7ad7fff700 1 -- 172.21.15.100:0/4030025556 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae0003680 con 0x5648b028b1b0 2024-05-10T11:44:09.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.249+0000 7f7ad7fff700 1 -- 172.21.15.100:0/4030025556 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7ae0003cb0 con 0x5648b028b1b0 2024-05-10T11:44:09.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.249+0000 7f7ad7fff700 1 -- 172.21.15.100:0/4030025556 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae001fd30 con 0x5648b028b1b0 2024-05-10T11:44:09.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.250+0000 7f7aff68cb80 1 -- 172.21.15.100:0/4030025556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5648b028b1b0 msgr2=0x5648b027ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:09.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.250+0000 7f7aff68cb80 1 --2- 172.21.15.100:0/4030025556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5648b028b1b0 0x5648b027ac40 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x5648b0281040 tx=0x7f7ae0014000 comp rx=0 tx=0).stop 2024-05-10T11:44:09.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.250+0000 7f7aff68cb80 1 -- 172.21.15.100:0/4030025556 shutdown_connections 2024-05-10T11:44:09.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.250+0000 7f7aff68cb80 1 --2- 172.21.15.100:0/4030025556 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5648b028bad0 0x5648b029b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:09.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.250+0000 7f7aff68cb80 1 --2- 172.21.15.100:0/4030025556 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648b0285d50 0x5648b0285810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:09.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.250+0000 7f7aff68cb80 1 --2- 172.21.15.100:0/4030025556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5648b028b1b0 0x5648b027ac40 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:09.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.250+0000 7f7aff68cb80 1 -- 172.21.15.100:0/4030025556 >> 172.21.15.100:0/4030025556 conn(0x5648b01b1b70 msgr2=0x5648b028d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:44:09.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.250+0000 7f7aff68cb80 1 -- 172.21.15.100:0/4030025556 shutdown_connections 2024-05-10T11:44:09.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.250+0000 7f7aff68cb80 1 -- 172.21.15.100:0/4030025556 wait complete. 2024-05-10T11:44:09.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.251+0000 7f7aff68cb80 1 -- start start 2024-05-10T11:44:09.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.251+0000 7f7aff68cb80 1 Processor -- start 2024-05-10T11:44:09.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.251+0000 7f7aff68cb80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5648b0285d50 0x5648b0297890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:09.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.251+0000 7f7aff68cb80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5648b028bad0 0x5648b0297dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:09.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.251+0000 7f7aff68cb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648b02a61d0 0x5648b02a6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:09.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.251+0000 7f7aff68cb80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7af01ab450 con 0x5648b028bad0 2024-05-10T11:44:09.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.251+0000 7f7aff68cb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7af01ac8c0 con 0x5648b02a61d0 2024-05-10T11:44:09.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.251+0000 7f7aff68cb80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7af01abb20 con 0x5648b0285d50 2024-05-10T11:44:09.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.251+0000 7f7aff68cb80 10 client.0 _ll_register_callbacks cb 0x5648b016b5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:44:09.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.251+0000 7f7aedd9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5648b028bad0 0x5648b0297dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:09.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.251+0000 7f7aee59c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5648b0285d50 0x5648b0297890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:09.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.251+0000 7f7aedd9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5648b028bad0 0x5648b0297dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34572/0 (socket says 172.21.15.100:34572) 2024-05-10T11:44:09.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.251+0000 7f7aedd9b700 1 -- 172.21.15.100:0/872652892 learned_addr learned my addr 172.21.15.100:0/872652892 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:44:09.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.251+0000 7f7aeed9d700 1 --2- 172.21.15.100:0/872652892 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648b02a61d0 0x5648b02a6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:09.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.252+0000 7f7aedd9b700 1 -- 172.21.15.100:0/872652892 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5648b0285d50 msgr2=0x5648b0297890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:44:09.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.252+0000 7f7aedd9b700 1 --2- 172.21.15.100:0/872652892 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5648b0285d50 0x5648b0297890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:09.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.252+0000 7f7aedd9b700 1 -- 172.21.15.100:0/872652892 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648b02a61d0 msgr2=0x5648b02a6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:44:09.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.252+0000 7f7aedd9b700 1 --2- 172.21.15.100:0/872652892 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648b02a61d0 0x5648b02a6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:09.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.252+0000 7f7aedd9b700 1 -- 172.21.15.100:0/872652892 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648b02a6c90 con 0x5648b028bad0 2024-05-10T11:44:09.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.252+0000 7f7aedd9b700 1 --2- 172.21.15.100:0/872652892 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5648b028bad0 0x5648b0297dd0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f7ae0026000 tx=0x7f7ae0003ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fa782b1be4df41c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:44:09.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.252+0000 7f7ad5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-05-10T11:44:09.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.252+0000 7f7ad5ffb700 1 -- 172.21.15.100:0/872652892 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae0029640 con 0x5648b028bad0 2024-05-10T11:44:09.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.252+0000 7f7aff68cb80 1 -- 172.21.15.100:0/872652892 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ae0003cb0 con 0x5648b028bad0 2024-05-10T11:44:09.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.252+0000 7f7ad5ffb700 1 -- 172.21.15.100:0/872652892 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f7ae00297e0 con 0x5648b028bad0 2024-05-10T11:44:09.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.252+0000 7f7ad5ffb700 1 -- 172.21.15.100:0/872652892 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f7ae003c050 con 0x5648b028bad0 2024-05-10T11:44:09.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.252+0000 7f7aff68cb80 10 client.20316 fetch_fsmap learned FSMap version 41 2024-05-10T11:44:09.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.252+0000 7f7aff68cb80 1 -- 172.21.15.100:0/872652892 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5648b0309b00 con 0x5648b028bad0 2024-05-10T11:44:09.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.252+0000 7f7ad5ffb700 1 -- 172.21.15.100:0/872652892 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7ae0003cb0 con 0x5648b028bad0 2024-05-10T11:44:09.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.253+0000 7f7ad5ffb700 1 -- 172.21.15.100:0/872652892 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f7ae001f910 con 0x5648b028bad0 2024-05-10T11:44:09.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.253+0000 7f7aff68cb80 10 client.20316 fetch_fsmap finished waiting for FSMap version 41 2024-05-10T11:44:09.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.253+0000 7f7aff68cb80 10 client.20316 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-10T11:44:09.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.253+0000 7f7aff68cb80 20 client.20316 populate_metadata read hostname 'smithi100' 2024-05-10T11:44:09.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.253+0000 7f7aff68cb80 1 --2- 172.21.15.100:0/872652892 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x5648b030a210 0x5648b030c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:09.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.253+0000 7f7aff68cb80 4 client.20316 mds_command: new command op to 24437 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:44:09.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.253+0000 7f7aff68cb80 1 -- 172.21.15.100:0/872652892 --> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5648b01df3c0 con 0x5648b030a210 2024-05-10T11:44:09.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.253+0000 7f7aee59c700 1 --2- 172.21.15.100:0/872652892 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x5648b030a210 0x5648b030c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:09.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.253+0000 7f7aee59c700 1 --2- 172.21.15.100:0/872652892 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x5648b030a210 0x5648b030c630 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fb6a26317e3e32d5 server_cookie=a8525d494d8e0bf2 in_seq=0 out_seq=0 2024-05-10T11:44:09.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.253+0000 7f7ad5ffb700 10 client.20316 ms_handle_connect on v2:172.21.15.178:6838/3760985404 2024-05-10T11:44:09.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.253+0000 7f7ad5ffb700 1 -- 172.21.15.100:0/872652892 <== mds.1 v2:172.21.15.178:6838/3760985404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5648b01df3c0 con 0x5648b030a210 2024-05-10T11:44:09.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.253+0000 7f7ad5ffb700 10 client.20316 handle_command_reply: tid=0 2024-05-10T11:44:09.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.253+0000 7f7aff68cb80 1 client.20316 shutdown 2024-05-10T11:44:09.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.253+0000 7f7aff68cb80 1 -- 172.21.15.100:0/872652892 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5648b028bad0 msgr2=0x5648b0297dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:09.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.253+0000 7f7aff68cb80 1 --2- 172.21.15.100:0/872652892 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5648b028bad0 0x5648b0297dd0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f7ae0026000 tx=0x7f7ae0003ad0 comp rx=0 tx=0).stop 2024-05-10T11:44:09.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.254+0000 7f7aff68cb80 1 -- 172.21.15.100:0/872652892 shutdown_connections 2024-05-10T11:44:09.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.254+0000 7f7aff68cb80 1 --2- 172.21.15.100:0/872652892 >> [v2:172.21.15.178:6838/3760985404,v1:172.21.15.178:6839/3760985404] conn(0x5648b030a210 0x5648b030c630 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:09.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.254+0000 7f7aff68cb80 1 --2- 172.21.15.100:0/872652892 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648b02a61d0 0x5648b02a6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:09.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.254+0000 7f7aff68cb80 1 --2- 172.21.15.100:0/872652892 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5648b0285d50 0x5648b0297890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:09.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.254+0000 7f7aff68cb80 1 --2- 172.21.15.100:0/872652892 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5648b028bad0 0x5648b0297dd0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:09.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.254+0000 7f7aff68cb80 1 -- 172.21.15.100:0/872652892 >> 172.21.15.100:0/872652892 conn(0x5648b01b1b70 msgr2=0x5648b028beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:44:09.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.254+0000 7f7ad5ffb700 0 client.20316 ms_handle_reset on v2:172.21.15.178:6838/3760985404 2024-05-10T11:44:09.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.254+0000 7f7aff68cb80 1 -- 172.21.15.100:0/872652892 shutdown_connections 2024-05-10T11:44:09.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.254+0000 7f7aff68cb80 1 -- 172.21.15.100:0/872652892 wait complete. 2024-05-10T11:44:09.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.254+0000 7f7aff68cb80 20 client.20316 trim_cache size 0 max 16384 2024-05-10T11:44:09.259 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:44:09.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.254+0000 7f7af4df9700 1 -- 172.21.15.100:0/2231732993 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f7ad80b2900 msgr2=0x7f7ad80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:09.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.254+0000 7f7af4df9700 1 --2- 172.21.15.100:0/2231732993 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f7ad80b2900 0x7f7ad80b4dc0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f7af0107bd0 tx=0x7f7ae0009260 comp rx=0 tx=0).stop 2024-05-10T11:44:09.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.254+0000 7f7af4df9700 1 -- 172.21.15.100:0/2231732993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7af0102b00 msgr2=0x7f7af019e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:09.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.254+0000 7f7af4df9700 1 --2- 172.21.15.100:0/2231732993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7af0102b00 0x7f7af019e670 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4007910 tx=0x7f7ae4007c20 comp rx=0 tx=0).stop 2024-05-10T11:44:09.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.254+0000 7f7af4df9700 1 -- 172.21.15.100:0/2231732993 shutdown_connections 2024-05-10T11:44:09.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.254+0000 7f7af4df9700 1 --2- 172.21.15.100:0/2231732993 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f7ad80b2900 0x7f7ad80b4dc0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:09.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.254+0000 7f7af4df9700 1 --2- 172.21.15.100:0/2231732993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7af0108470 0x7f7af01987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:09.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.254+0000 7f7af4df9700 1 --2- 172.21.15.100:0/2231732993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7af0102b00 0x7f7af019e670 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:09.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.254+0000 7f7af4df9700 1 --2- 172.21.15.100:0/2231732993 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7af01034c0 0x7f7af019ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:09.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.255+0000 7f7af4df9700 1 -- 172.21.15.100:0/2231732993 >> 172.21.15.100:0/2231732993 conn(0x7f7af00789a0 msgr2=0x7f7af01001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:44:09.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.255+0000 7f7af4df9700 1 -- 172.21.15.100:0/2231732993 shutdown_connections 2024-05-10T11:44:09.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:09.255+0000 7f7af4df9700 1 -- 172.21.15.100:0/2231732993 wait complete. 2024-05-10T11:44:09.964 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:44:09 smithi100 ceph-mon[30296]: pgmap v1194: 129 pgs: 129 active+clean; 566 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 3.4 MiB/s wr, 260 op/s 2024-05-10T11:44:10.013 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-10T11:44:10.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:44:09 smithi188 ceph-mon[34075]: pgmap v1194: 129 pgs: 129 active+clean; 566 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 3.4 MiB/s wr, 260 op/s 2024-05-10T11:44:10.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:44:09 smithi178 ceph-mon[33359]: pgmap v1194: 129 pgs: 129 active+clean; 566 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 3.4 MiB/s wr, 260 op/s 2024-05-10T11:44:10.457 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:44:10.524 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-05-10T11:44:10.524 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-05-10T11:44:11.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:44:10 smithi100 ceph-mon[30296]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T11:44:11.089 DEBUG:teuthology.parallel:result is None 2024-05-10T11:44:11.089 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-05-10T11:44:11.123 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-05-10T11:44:11.123 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-05-10T11:44:11.347 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi100.front.sepia.ceph.com 2024-05-10T11:44:11.348 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:44:10 smithi188 ceph-mon[34075]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T11:44:11.348 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:44:10 smithi178 ceph-mon[33359]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T11:44:11.357 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-05-10T11:44:11.529 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-05-10T11:44:11.875 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:44:11 smithi100 ceph-mon[30296]: pgmap v1195: 129 pgs: 129 active+clean; 514 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 3.5 MiB/s wr, 312 op/s 2024-05-10T11:44:11.875 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:44:11 smithi100 ceph-mon[30296]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T11:44:12.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:44:11 smithi178 ceph-mon[33359]: pgmap v1195: 129 pgs: 129 active+clean; 514 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 3.5 MiB/s wr, 312 op/s 2024-05-10T11:44:12.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:44:11 smithi178 ceph-mon[33359]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T11:44:12.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:44:11 smithi188 ceph-mon[34075]: pgmap v1195: 129 pgs: 129 active+clean; 514 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 3.5 MiB/s wr, 312 op/s 2024-05-10T11:44:12.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:44:11 smithi188 ceph-mon[34075]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T11:44:12.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.445+0000 7f343f407700 1 -- 172.21.15.100:0/2530454723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f343806b180 msgr2=0x7f343806b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:12.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.445+0000 7f343f407700 1 --2- 172.21.15.100:0/2530454723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f343806b180 0x7f343806b600 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f3428009990 tx=0x7f3428009ca0 comp rx=0 tx=0).stop 2024-05-10T11:44:12.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.445+0000 7f343f407700 1 -- 172.21.15.100:0/2530454723 shutdown_connections 2024-05-10T11:44:12.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.445+0000 7f343f407700 1 --2- 172.21.15.100:0/2530454723 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f343806bb40 0x7f3438110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:12.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.445+0000 7f343f407700 1 --2- 172.21.15.100:0/2530454723 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3438109510 0x7f34381098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:12.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.445+0000 7f343f407700 1 --2- 172.21.15.100:0/2530454723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f343806b180 0x7f343806b600 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:12.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.445+0000 7f343f407700 1 -- 172.21.15.100:0/2530454723 >> 172.21.15.100:0/2530454723 conn(0x7f3438079350 msgr2=0x7f3438079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:44:12.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.445+0000 7f343f407700 1 -- 172.21.15.100:0/2530454723 shutdown_connections 2024-05-10T11:44:12.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.445+0000 7f343f407700 1 -- 172.21.15.100:0/2530454723 wait complete. 2024-05-10T11:44:12.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.446+0000 7f343f407700 1 Processor -- start 2024-05-10T11:44:12.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.446+0000 7f343f407700 1 -- start start 2024-05-10T11:44:12.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.446+0000 7f343f407700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f343806b180 0x7f3438102470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:12.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.446+0000 7f343f407700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f343806bb40 0x7f34381029b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:12.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.446+0000 7f343f407700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3438109510 0x7f3438102ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:12.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.446+0000 7f343f407700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f34381134d0 con 0x7f3438109510 2024-05-10T11:44:12.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.446+0000 7f343f407700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3438113350 con 0x7f343806b180 2024-05-10T11:44:12.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.446+0000 7f343f407700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3438113650 con 0x7f343806bb40 2024-05-10T11:44:12.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.446+0000 7f343d9a4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3438109510 0x7f3438102ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:12.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.447+0000 7f343d9a4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3438109510 0x7f3438102ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34592/0 (socket says 172.21.15.100:34592) 2024-05-10T11:44:12.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.447+0000 7f343d1a3700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f343806b180 0x7f3438102470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:12.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.447+0000 7f343d9a4700 1 -- 172.21.15.100:0/1258424468 learned_addr learned my addr 172.21.15.100:0/1258424468 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:44:12.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.447+0000 7f343c9a2700 1 --2- 172.21.15.100:0/1258424468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f343806bb40 0x7f34381029b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:12.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.447+0000 7f343d1a3700 1 -- 172.21.15.100:0/1258424468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f343806bb40 msgr2=0x7f34381029b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:12.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.447+0000 7f343d1a3700 1 --2- 172.21.15.100:0/1258424468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f343806bb40 0x7f34381029b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:12.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.447+0000 7f343d1a3700 1 -- 172.21.15.100:0/1258424468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3438109510 msgr2=0x7f3438102ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:12.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.447+0000 7f343d1a3700 1 --2- 172.21.15.100:0/1258424468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3438109510 0x7f3438102ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:12.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.447+0000 7f343d1a3700 1 -- 172.21.15.100:0/1258424468 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34381a7fb0 con 0x7f343806b180 2024-05-10T11:44:12.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.447+0000 7f343d9a4700 1 --2- 172.21.15.100:0/1258424468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3438109510 0x7f3438102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T11:44:12.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.447+0000 7f343d1a3700 1 --2- 172.21.15.100:0/1258424468 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f343806b180 0x7f3438102470 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f343400e9f0 tx=0x7f343400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:44:12.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.447+0000 7f342e7fc700 1 -- 172.21.15.100:0/1258424468 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343400cb70 con 0x7f343806b180 2024-05-10T11:44:12.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.447+0000 7f342e7fc700 1 -- 172.21.15.100:0/1258424468 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f343400ee90 con 0x7f343806b180 2024-05-10T11:44:12.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.447+0000 7f342e7fc700 1 -- 172.21.15.100:0/1258424468 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3434003c30 con 0x7f343806b180 2024-05-10T11:44:12.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.447+0000 7f343f407700 1 -- 172.21.15.100:0/1258424468 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34381a82a0 con 0x7f343806b180 2024-05-10T11:44:12.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.448+0000 7f343f407700 1 -- 172.21.15.100:0/1258424468 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34381a8760 con 0x7f343806b180 2024-05-10T11:44:12.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.450+0000 7f342e7fc700 1 -- 172.21.15.100:0/1258424468 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f3434003dd0 con 0x7f343806b180 2024-05-10T11:44:12.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.451+0000 7f342e7fc700 1 --2- 172.21.15.100:0/1258424468 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f34240b2880 0x7f34240b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:12.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.451+0000 7f343c9a2700 1 --2- 172.21.15.100:0/1258424468 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f34240b2880 0x7f34240b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:12.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.451+0000 7f342e7fc700 1 -- 172.21.15.100:0/1258424468 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f34340e26c0 con 0x7f343806b180 2024-05-10T11:44:12.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.451+0000 7f343c9a2700 1 --2- 172.21.15.100:0/1258424468 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f34240b2880 0x7f34240b4d40 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f3438103f00 tx=0x7f3428005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:44:12.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.453+0000 7f3449c9ab80 1 Processor -- start 2024-05-10T11:44:12.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.453+0000 7f3449c9ab80 1 -- start start 2024-05-10T11:44:12.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.453+0000 7f3449c9ab80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6f07ae4d0 0x55c6f07ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:12.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.453+0000 7f3449c9ab80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f07aedf0 0x55c6f07b26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:12.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.453+0000 7f3449c9ab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6f07b2c80 0x55c6f07b3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:12.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.453+0000 7f3449c9ab80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55c6f0700860 con 0x55c6f07aedf0 2024-05-10T11:44:12.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.453+0000 7f3449c9ab80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c6f069f7c0 con 0x55c6f07ae4d0 2024-05-10T11:44:12.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.453+0000 7f3449c9ab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c6f06fc820 con 0x55c6f07b2c80 2024-05-10T11:44:12.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.454+0000 7f343d1a3700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6f07ae4d0 0x55c6f07ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:12.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.454+0000 7f343c9a2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f07aedf0 0x55c6f07b26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:12.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.454+0000 7f343d1a3700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6f07ae4d0 0x55c6f07ae8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.100:56708/0 (socket says 172.21.15.100:56708) 2024-05-10T11:44:12.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.454+0000 7f343d1a3700 1 -- 172.21.15.100:0/2262020326 learned_addr learned my addr 172.21.15.100:0/2262020326 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:44:12.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.454+0000 7f343d9a4700 1 --2- 172.21.15.100:0/2262020326 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6f07b2c80 0x55c6f07b3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:12.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.454+0000 7f343d1a3700 1 -- 172.21.15.100:0/2262020326 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6f07b2c80 msgr2=0x55c6f07b3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:44:12.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.454+0000 7f343d1a3700 1 --2- 172.21.15.100:0/2262020326 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6f07b2c80 0x55c6f07b3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:12.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.454+0000 7f343d1a3700 1 -- 172.21.15.100:0/2262020326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f07aedf0 msgr2=0x55c6f07b26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:44:12.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.454+0000 7f343d1a3700 1 --2- 172.21.15.100:0/2262020326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f07aedf0 0x55c6f07b26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:12.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.454+0000 7f343d1a3700 1 -- 172.21.15.100:0/2262020326 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6f064d8d0 con 0x55c6f07ae4d0 2024-05-10T11:44:12.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.454+0000 7f343d1a3700 1 --2- 172.21.15.100:0/2262020326 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6f07ae4d0 0x55c6f07ae8b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f3434009230 tx=0x7f343400cf70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=493668b2014eab26 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:44:12.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.454+0000 7f34237fe700 1 -- 172.21.15.100:0/2262020326 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34340b20b0 con 0x55c6f07ae4d0 2024-05-10T11:44:12.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.454+0000 7f34237fe700 1 -- 172.21.15.100:0/2262020326 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f34340adcd0 con 0x55c6f07ae4d0 2024-05-10T11:44:12.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.454+0000 7f34237fe700 1 -- 172.21.15.100:0/2262020326 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34340a9b80 con 0x55c6f07ae4d0 2024-05-10T11:44:12.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.455+0000 7f3449c9ab80 1 -- 172.21.15.100:0/2262020326 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6f07ae4d0 msgr2=0x55c6f07ae8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:12.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.455+0000 7f3449c9ab80 1 --2- 172.21.15.100:0/2262020326 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6f07ae4d0 0x55c6f07ae8b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f3434009230 tx=0x7f343400cf70 comp rx=0 tx=0).stop 2024-05-10T11:44:12.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.455+0000 7f3449c9ab80 1 -- 172.21.15.100:0/2262020326 shutdown_connections 2024-05-10T11:44:12.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.455+0000 7f3449c9ab80 1 --2- 172.21.15.100:0/2262020326 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6f07b2c80 0x55c6f07b3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:12.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.455+0000 7f3449c9ab80 1 --2- 172.21.15.100:0/2262020326 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6f07ae4d0 0x55c6f07ae8b0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:12.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.455+0000 7f3449c9ab80 1 --2- 172.21.15.100:0/2262020326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f07aedf0 0x55c6f07b26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:12.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.455+0000 7f3449c9ab80 1 -- 172.21.15.100:0/2262020326 >> 172.21.15.100:0/2262020326 conn(0x55c6f06da100 msgr2=0x55c6f07a8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:44:12.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.455+0000 7f3449c9ab80 1 -- 172.21.15.100:0/2262020326 shutdown_connections 2024-05-10T11:44:12.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.455+0000 7f3449c9ab80 1 -- 172.21.15.100:0/2262020326 wait complete. 2024-05-10T11:44:12.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.456+0000 7f3449c9ab80 1 -- start start 2024-05-10T11:44:12.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.456+0000 7f3449c9ab80 1 Processor -- start 2024-05-10T11:44:12.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.456+0000 7f3449c9ab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6f07ae4d0 0x55c6f07ba110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:12.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.456+0000 7f3449c9ab80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f07aedf0 0x55c6f07b74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:12.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.457+0000 7f343c9a2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f07aedf0 0x55c6f07b74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:12.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.457+0000 7f343c9a2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f07aedf0 0x55c6f07b74b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34618/0 (socket says 172.21.15.100:34618) 2024-05-10T11:44:12.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.457+0000 7f3449c9ab80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6f07b2c80 0x55c6f07b79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:12.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.457+0000 7f343d1a3700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6f07ae4d0 0x55c6f07ba110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:12.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.457+0000 7f343c9a2700 1 -- 172.21.15.100:0/2564313494 learned_addr learned my addr 172.21.15.100:0/2564313494 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:44:12.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.457+0000 7f3449c9ab80 1 -- 172.21.15.100:0/2564313494 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55c6f07bae90 con 0x55c6f07aedf0 2024-05-10T11:44:12.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.457+0000 7f3449c9ab80 1 -- 172.21.15.100:0/2564313494 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c6f0666c30 con 0x55c6f07b2c80 2024-05-10T11:44:12.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.457+0000 7f3449c9ab80 1 -- 172.21.15.100:0/2564313494 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c6f07bb010 con 0x55c6f07ae4d0 2024-05-10T11:44:12.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.457+0000 7f3449c9ab80 10 client.0 _ll_register_callbacks cb 0x55c6f0698920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:44:12.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.457+0000 7f343d9a4700 1 --2- 172.21.15.100:0/2564313494 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6f07b2c80 0x55c6f07b79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:12.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.457+0000 7f343d9a4700 1 -- 172.21.15.100:0/2564313494 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6f07ae4d0 msgr2=0x55c6f07ba110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:44:12.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.457+0000 7f343d9a4700 1 --2- 172.21.15.100:0/2564313494 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6f07ae4d0 0x55c6f07ba110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:12.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.457+0000 7f343d9a4700 1 -- 172.21.15.100:0/2564313494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f07aedf0 msgr2=0x55c6f07b74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:44:12.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.457+0000 7f343d9a4700 1 --2- 172.21.15.100:0/2564313494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f07aedf0 0x55c6f07b74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:12.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.457+0000 7f343d9a4700 1 -- 172.21.15.100:0/2564313494 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6f069c690 con 0x55c6f07b2c80 2024-05-10T11:44:12.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.458+0000 7f343d9a4700 1 --2- 172.21.15.100:0/2564313494 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6f07b2c80 0x55c6f07b79f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3430009c10 tx=0x7f3430009510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=896f7f7d487a29a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:44:12.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.458+0000 7f3420ff9700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-05-10T11:44:12.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.458+0000 7f3420ff9700 1 -- 172.21.15.100:0/2564313494 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343001a070 con 0x55c6f07b2c80 2024-05-10T11:44:12.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.458+0000 7f3449c9ab80 1 -- 172.21.15.100:0/2564313494 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34340adcd0 con 0x55c6f07b2c80 2024-05-10T11:44:12.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.458+0000 7f3420ff9700 1 -- 172.21.15.100:0/2564313494 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f3430004db0 con 0x55c6f07b2c80 2024-05-10T11:44:12.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.458+0000 7f3420ff9700 1 -- 172.21.15.100:0/2564313494 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f3430026050 con 0x55c6f07b2c80 2024-05-10T11:44:12.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.458+0000 7f3449c9ab80 10 client.29564 fetch_fsmap learned FSMap version 43 2024-05-10T11:44:12.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.458+0000 7f3449c9ab80 1 -- 172.21.15.100:0/2564313494 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55c6f07c57f0 con 0x55c6f07b2c80 2024-05-10T11:44:12.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.458+0000 7f3420ff9700 1 -- 172.21.15.100:0/2564313494 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f34340adcd0 con 0x55c6f07b2c80 2024-05-10T11:44:12.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.459+0000 7f3420ff9700 1 -- 172.21.15.100:0/2564313494 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 43) ==== 8358+0+0 (secure 0 0 0) 0x7f3430016400 con 0x55c6f07b2c80 2024-05-10T11:44:12.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.459+0000 7f3449c9ab80 10 client.29564 fetch_fsmap finished waiting for FSMap version 43 2024-05-10T11:44:12.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.459+0000 7f3449c9ab80 10 client.29564 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:44:12.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.459+0000 7f3449c9ab80 20 client.29564 populate_metadata read hostname 'smithi100' 2024-05-10T11:44:12.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.459+0000 7f3449c9ab80 1 --2- 172.21.15.100:0/2564313494 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c6f07c8200 0x55c6f07ca620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:12.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.459+0000 7f3449c9ab80 4 client.29564 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T11:44:12.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.459+0000 7f3449c9ab80 1 -- 172.21.15.100:0/2564313494 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6f077a650 con 0x55c6f07c8200 2024-05-10T11:44:12.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.459+0000 7f343d1a3700 1 --2- 172.21.15.100:0/2564313494 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c6f07c8200 0x55c6f07ca620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:12.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.459+0000 7f3420ff9700 10 client.29564 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:44:12.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.459+0000 7f343d1a3700 1 --2- 172.21.15.100:0/2564313494 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c6f07c8200 0x55c6f07ca620 crc :-1 s=READY pgs=475 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d758dbecb856ebc server_cookie=782c873e8615d714 in_seq=0 out_seq=0 2024-05-10T11:44:12.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.460+0000 7f3420ff9700 1 -- 172.21.15.100:0/2564313494 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c6f077a650 con 0x55c6f07c8200 2024-05-10T11:44:12.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.460+0000 7f3420ff9700 10 client.29564 handle_command_reply: tid=0 2024-05-10T11:44:12.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.460+0000 7f3449c9ab80 1 client.29564 shutdown 2024-05-10T11:44:12.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.460+0000 7f3449c9ab80 1 -- 172.21.15.100:0/2564313494 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6f07b2c80 msgr2=0x55c6f07b79f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:12.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.460+0000 7f3449c9ab80 1 --2- 172.21.15.100:0/2564313494 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6f07b2c80 0x55c6f07b79f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3430009c10 tx=0x7f3430009510 comp rx=0 tx=0).stop 2024-05-10T11:44:12.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.460+0000 7f3449c9ab80 1 -- 172.21.15.100:0/2564313494 shutdown_connections 2024-05-10T11:44:12.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.460+0000 7f3449c9ab80 1 --2- 172.21.15.100:0/2564313494 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c6f07c8200 0x55c6f07ca620 crc :-1 s=READY pgs=475 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:12.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.460+0000 7f3449c9ab80 1 --2- 172.21.15.100:0/2564313494 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6f07b2c80 0x55c6f07b79f0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:12.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.460+0000 7f3420ff9700 0 client.29564 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:44:12.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.460+0000 7f3449c9ab80 1 --2- 172.21.15.100:0/2564313494 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6f07ae4d0 0x55c6f07ba110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:12.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.460+0000 7f3449c9ab80 1 --2- 172.21.15.100:0/2564313494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f07aedf0 0x55c6f07b74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:12.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.460+0000 7f3449c9ab80 1 -- 172.21.15.100:0/2564313494 >> 172.21.15.100:0/2564313494 conn(0x55c6f06da100 msgr2=0x55c6f07a3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:44:12.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.461+0000 7f3449c9ab80 1 -- 172.21.15.100:0/2564313494 shutdown_connections 2024-05-10T11:44:12.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.461+0000 7f3449c9ab80 1 -- 172.21.15.100:0/2564313494 wait complete. 2024-05-10T11:44:12.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.461+0000 7f3449c9ab80 20 client.29564 trim_cache size 0 max 16384 2024-05-10T11:44:12.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.478+0000 7f3449c9ab80 1 Processor -- start 2024-05-10T11:44:12.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.479+0000 7f3449c9ab80 1 -- start start 2024-05-10T11:44:12.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.479+0000 7f3449c9ab80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6f07ae4d0 0x55c6f07b7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:12.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.479+0000 7f3449c9ab80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f07aedf0 0x55c6f07c3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:12.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.479+0000 7f3449c9ab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6f07b2c80 0x55c6f07b6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:12.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.479+0000 7f3449c9ab80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f34381b3cf0 con 0x55c6f07aedf0 2024-05-10T11:44:12.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.479+0000 7f3449c9ab80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f34381b1520 con 0x55c6f07ae4d0 2024-05-10T11:44:12.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.479+0000 7f3449c9ab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f34381af370 con 0x55c6f07b2c80 2024-05-10T11:44:12.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.479+0000 7f343c9a2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f07aedf0 0x55c6f07c3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:12.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.479+0000 7f343c9a2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f07aedf0 0x55c6f07c3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34634/0 (socket says 172.21.15.100:34634) 2024-05-10T11:44:12.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.479+0000 7f343c9a2700 1 -- 172.21.15.100:0/616087686 learned_addr learned my addr 172.21.15.100:0/616087686 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:44:12.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.479+0000 7f343d1a3700 1 --2- 172.21.15.100:0/616087686 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6f07ae4d0 0x55c6f07b7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:12.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.479+0000 7f343d9a4700 1 --2- 172.21.15.100:0/616087686 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6f07b2c80 0x55c6f07b6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:12.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.479+0000 7f343c9a2700 1 -- 172.21.15.100:0/616087686 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6f07b2c80 msgr2=0x55c6f07b6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:44:12.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.479+0000 7f343c9a2700 1 --2- 172.21.15.100:0/616087686 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6f07b2c80 0x55c6f07b6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:12.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.479+0000 7f343c9a2700 1 -- 172.21.15.100:0/616087686 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6f07ae4d0 msgr2=0x55c6f07b7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:44:12.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.479+0000 7f343c9a2700 1 --2- 172.21.15.100:0/616087686 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6f07ae4d0 0x55c6f07b7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:12.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.479+0000 7f343c9a2700 1 -- 172.21.15.100:0/616087686 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6f07bafc0 con 0x55c6f07aedf0 2024-05-10T11:44:12.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.480+0000 7f343c9a2700 1 --2- 172.21.15.100:0/616087686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f07aedf0 0x55c6f07c3f20 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f3428009990 tx=0x7f3428022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e8dfff19f05435c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:44:12.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.480+0000 7f3422ffd700 1 -- 172.21.15.100:0/616087686 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342800b8b0 con 0x55c6f07aedf0 2024-05-10T11:44:12.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.480+0000 7f3422ffd700 1 -- 172.21.15.100:0/616087686 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f342800f460 con 0x55c6f07aedf0 2024-05-10T11:44:12.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.480+0000 7f3422ffd700 1 -- 172.21.15.100:0/616087686 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3428024670 con 0x55c6f07aedf0 2024-05-10T11:44:12.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.481+0000 7f3449c9ab80 1 -- 172.21.15.100:0/616087686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f07aedf0 msgr2=0x55c6f07c3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:12.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.481+0000 7f3449c9ab80 1 --2- 172.21.15.100:0/616087686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f07aedf0 0x55c6f07c3f20 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f3428009990 tx=0x7f3428022000 comp rx=0 tx=0).stop 2024-05-10T11:44:12.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.481+0000 7f3449c9ab80 1 -- 172.21.15.100:0/616087686 shutdown_connections 2024-05-10T11:44:12.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.481+0000 7f3449c9ab80 1 --2- 172.21.15.100:0/616087686 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6f07b2c80 0x55c6f07b6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:12.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.481+0000 7f3449c9ab80 1 --2- 172.21.15.100:0/616087686 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6f07ae4d0 0x55c6f07b7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:12.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.481+0000 7f3449c9ab80 1 --2- 172.21.15.100:0/616087686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f07aedf0 0x55c6f07c3f20 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:12.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.481+0000 7f3449c9ab80 1 -- 172.21.15.100:0/616087686 >> 172.21.15.100:0/616087686 conn(0x55c6f06da100 msgr2=0x55c6f07a2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:44:12.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.481+0000 7f3449c9ab80 1 -- 172.21.15.100:0/616087686 shutdown_connections 2024-05-10T11:44:12.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.481+0000 7f3449c9ab80 1 -- 172.21.15.100:0/616087686 wait complete. 2024-05-10T11:44:12.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.481+0000 7f3449c9ab80 1 -- start start 2024-05-10T11:44:12.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.481+0000 7f3449c9ab80 1 Processor -- start 2024-05-10T11:44:12.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.482+0000 7f3449c9ab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6f07ae4d0 0x55c6f07c00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:12.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.482+0000 7f3449c9ab80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6f07aedf0 0x55c6f07c0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:12.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.482+0000 7f3449c9ab80 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f07b2c80 0x55c6f07bb660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:12.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.482+0000 7f3449c9ab80 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f34381af9e0 con 0x55c6f07b2c80 2024-05-10T11:44:12.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.482+0000 7f3449c9ab80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f34381b0e50 con 0x55c6f07aedf0 2024-05-10T11:44:12.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.482+0000 7f3449c9ab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f34381b00b0 con 0x55c6f07ae4d0 2024-05-10T11:44:12.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.482+0000 7f3449c9ab80 10 client.0 _ll_register_callbacks cb 0x55c6f0698920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T11:44:12.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.482+0000 7f343d1a3700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6f07ae4d0 0x55c6f07c00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:12.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.482+0000 7f343d1a3700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6f07ae4d0 0x55c6f07c00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.100:47868/0 (socket says 172.21.15.100:47868) 2024-05-10T11:44:12.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.482+0000 7f343d1a3700 1 -- 172.21.15.100:0/2033217191 learned_addr learned my addr 172.21.15.100:0/2033217191 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:44:12.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.482+0000 7f343c9a2700 1 --2- 172.21.15.100:0/2033217191 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6f07aedf0 0x55c6f07c0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:12.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.482+0000 7f343d9a4700 1 --2- 172.21.15.100:0/2033217191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f07b2c80 0x55c6f07bb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:12.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.482+0000 7f343d1a3700 1 -- 172.21.15.100:0/2033217191 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6f07aedf0 msgr2=0x55c6f07c0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:44:12.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.482+0000 7f343d1a3700 1 --2- 172.21.15.100:0/2033217191 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6f07aedf0 0x55c6f07c0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:12.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.482+0000 7f343d1a3700 1 -- 172.21.15.100:0/2033217191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f07b2c80 msgr2=0x55c6f07bb660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T11:44:12.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.482+0000 7f343d1a3700 1 --2- 172.21.15.100:0/2033217191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f07b2c80 0x55c6f07bb660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:12.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.482+0000 7f343d1a3700 1 -- 172.21.15.100:0/2033217191 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6f07bbe70 con 0x55c6f07ae4d0 2024-05-10T11:44:12.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.482+0000 7f343d1a3700 1 --2- 172.21.15.100:0/2033217191 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6f07ae4d0 0x55c6f07c00f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f34340cc3f0 tx=0x7f34340a9820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54d210f6036826b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:44:12.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.482+0000 7f3420ff9700 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-10T11:44:12.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.483+0000 7f3420ff9700 1 -- 172.21.15.100:0/2033217191 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34340a50b0 con 0x55c6f07ae4d0 2024-05-10T11:44:12.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.483+0000 7f3420ff9700 1 -- 172.21.15.100:0/2033217191 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f34340e2400 con 0x55c6f07ae4d0 2024-05-10T11:44:12.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.483+0000 7f3449c9ab80 1 -- 172.21.15.100:0/2033217191 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f342800f460 con 0x55c6f07ae4d0 2024-05-10T11:44:12.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.483+0000 7f3420ff9700 1 -- 172.21.15.100:0/2033217191 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f34340a57e0 con 0x55c6f07ae4d0 2024-05-10T11:44:12.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.483+0000 7f3449c9ab80 10 client.29695 fetch_fsmap learned FSMap version 43 2024-05-10T11:44:12.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.483+0000 7f3449c9ab80 1 -- 172.21.15.100:0/2033217191 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55c6f0832500 con 0x55c6f07ae4d0 2024-05-10T11:44:12.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.483+0000 7f3420ff9700 1 -- 172.21.15.100:0/2033217191 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f342800f460 con 0x55c6f07ae4d0 2024-05-10T11:44:12.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.483+0000 7f3420ff9700 1 -- 172.21.15.100:0/2033217191 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 43) ==== 8358+0+0 (secure 0 0 0) 0x7f34340a9c10 con 0x55c6f07ae4d0 2024-05-10T11:44:12.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.483+0000 7f3449c9ab80 10 client.29695 fetch_fsmap finished waiting for FSMap version 43 2024-05-10T11:44:12.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.483+0000 7f3449c9ab80 10 client.29695 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-10T11:44:12.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.483+0000 7f3449c9ab80 20 client.29695 populate_metadata read hostname 'smithi100' 2024-05-10T11:44:12.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.483+0000 7f3449c9ab80 1 --2- 172.21.15.100:0/2033217191 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c6f08327f0 0x55c6f0834be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:12.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.483+0000 7f3449c9ab80 4 client.29695 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T11:44:12.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.483+0000 7f3449c9ab80 1 -- 172.21.15.100:0/2033217191 --> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c6f070e240 con 0x55c6f08327f0 2024-05-10T11:44:12.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.483+0000 7f343c9a2700 1 --2- 172.21.15.100:0/2033217191 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c6f08327f0 0x55c6f0834be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:12.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.484+0000 7f343c9a2700 1 --2- 172.21.15.100:0/2033217191 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c6f08327f0 0x55c6f0834be0 crc :-1 s=READY pgs=476 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ac504a70e573792 server_cookie=f2e32973f3e422cf in_seq=0 out_seq=0 2024-05-10T11:44:12.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.484+0000 7f3420ff9700 10 client.29695 ms_handle_connect on v2:172.21.15.188:6838/3194917070 2024-05-10T11:44:12.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.484+0000 7f3420ff9700 1 -- 172.21.15.100:0/2033217191 <== mds.0 v2:172.21.15.188:6838/3194917070 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c6f070e240 con 0x55c6f08327f0 2024-05-10T11:44:12.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.484+0000 7f3420ff9700 10 client.29695 handle_command_reply: tid=0 2024-05-10T11:44:12.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.484+0000 7f3449c9ab80 1 client.29695 shutdown 2024-05-10T11:44:12.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.484+0000 7f3449c9ab80 1 -- 172.21.15.100:0/2033217191 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6f07ae4d0 msgr2=0x55c6f07c00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:12.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.484+0000 7f3449c9ab80 1 --2- 172.21.15.100:0/2033217191 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6f07ae4d0 0x55c6f07c00f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f34340cc3f0 tx=0x7f34340a9820 comp rx=0 tx=0).stop 2024-05-10T11:44:12.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.484+0000 7f3449c9ab80 1 -- 172.21.15.100:0/2033217191 shutdown_connections 2024-05-10T11:44:12.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.484+0000 7f3449c9ab80 1 --2- 172.21.15.100:0/2033217191 >> [v2:172.21.15.188:6838/3194917070,v1:172.21.15.188:6839/3194917070] conn(0x55c6f08327f0 0x55c6f0834be0 crc :-1 s=READY pgs=476 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:12.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.484+0000 7f3420ff9700 0 client.29695 ms_handle_reset on v2:172.21.15.188:6838/3194917070 2024-05-10T11:44:12.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.484+0000 7f3449c9ab80 1 --2- 172.21.15.100:0/2033217191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f07b2c80 0x55c6f07bb660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:12.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.484+0000 7f3449c9ab80 1 --2- 172.21.15.100:0/2033217191 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c6f07ae4d0 0x55c6f07c00f0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:12.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.484+0000 7f3449c9ab80 1 --2- 172.21.15.100:0/2033217191 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6f07aedf0 0x55c6f07c0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:12.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.484+0000 7f3449c9ab80 1 -- 172.21.15.100:0/2033217191 >> 172.21.15.100:0/2033217191 conn(0x55c6f06da100 msgr2=0x55c6f07a3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:44:12.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.484+0000 7f3449c9ab80 1 -- 172.21.15.100:0/2033217191 shutdown_connections 2024-05-10T11:44:12.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.484+0000 7f3449c9ab80 1 -- 172.21.15.100:0/2033217191 wait complete. 2024-05-10T11:44:12.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.485+0000 7f3449c9ab80 20 client.29695 trim_cache size 0 max 16384 2024-05-10T11:44:12.490 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-05-10T11:44:12.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.485+0000 7f343f407700 1 -- 172.21.15.100:0/1258424468 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f34240b2880 msgr2=0x7f34240b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:12.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.485+0000 7f343f407700 1 --2- 172.21.15.100:0/1258424468 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f34240b2880 0x7f34240b4d40 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f3438103f00 tx=0x7f3428005fb0 comp rx=0 tx=0).stop 2024-05-10T11:44:12.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.485+0000 7f343f407700 1 -- 172.21.15.100:0/1258424468 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f343806b180 msgr2=0x7f3438102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:12.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.485+0000 7f343f407700 1 --2- 172.21.15.100:0/1258424468 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f343806b180 0x7f3438102470 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f343400e9f0 tx=0x7f343400ed00 comp rx=0 tx=0).stop 2024-05-10T11:44:12.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.485+0000 7f343f407700 1 -- 172.21.15.100:0/1258424468 shutdown_connections 2024-05-10T11:44:12.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.485+0000 7f343f407700 1 --2- 172.21.15.100:0/1258424468 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f34240b2880 0x7f34240b4d40 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:12.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.485+0000 7f343f407700 1 --2- 172.21.15.100:0/1258424468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3438109510 0x7f3438102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:12.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.485+0000 7f343f407700 1 --2- 172.21.15.100:0/1258424468 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f343806b180 0x7f3438102470 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:12.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.485+0000 7f343f407700 1 --2- 172.21.15.100:0/1258424468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f343806bb40 0x7f34381029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:12.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.485+0000 7f343f407700 1 -- 172.21.15.100:0/1258424468 >> 172.21.15.100:0/1258424468 conn(0x7f3438079350 msgr2=0x7f34381012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:44:12.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.485+0000 7f343f407700 1 -- 172.21.15.100:0/1258424468 shutdown_connections 2024-05-10T11:44:12.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:12.485+0000 7f343f407700 1 -- 172.21.15.100:0/1258424468 wait complete. 2024-05-10T11:44:13.119 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:44:12 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:44:13.120 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:44:12 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:44:13.120 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:44:12 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:44:13.120 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:44:12 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:44:13.120 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:44:12 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:44:13.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:44:12 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:44:13.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:44:12 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:44:13.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:44:12 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:44:13.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:44:12 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:44:13.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:44:12 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:44:13.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:44:12 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:44:13.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:44:12 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:44:13.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:44:12 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:44:13.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:44:12 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:44:13.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:44:12 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:44:13.704 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_d39fa08867836ac070c47cb66c782fa559fd480b/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_d39fa08867836ac070c47cb66c782fa559fd480b/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph-c_d39fa08867836ac070c47cb66c782fa559fd480b/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-05-10T11:44:13.704 INFO:tasks.fwd_scrub:done joining 2024-05-10T11:44:13.704 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-05-10T11:44:13.715 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T11:44:13.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:13.821+0000 7f1daa01d640 1 -- 172.21.15.100:0/3117634756 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1da41018d0 msgr2=0x7f1da4101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:13.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:13.821+0000 7f1daa01d640 1 --2- 172.21.15.100:0/3117634756 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1da41018d0 0x7f1da4101cf0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f1d94009900 tx=0x7f1d9402a410 comp rx=0 tx=0).stop 2024-05-10T11:44:13.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:13.821+0000 7f1daa01d640 1 -- 172.21.15.100:0/3117634756 shutdown_connections 2024-05-10T11:44:13.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:13.821+0000 7f1daa01d640 1 --2- 172.21.15.100:0/3117634756 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1da4103480 0x7f1da4109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:13.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:13.821+0000 7f1daa01d640 1 --2- 172.21.15.100:0/3117634756 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1da4102ac0 0x7f1da4102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:13.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:13.821+0000 7f1daa01d640 1 --2- 172.21.15.100:0/3117634756 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1da41018d0 0x7f1da4101cf0 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:13.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:13.821+0000 7f1daa01d640 1 -- 172.21.15.100:0/3117634756 >> 172.21.15.100:0/3117634756 conn(0x7f1da40fce40 msgr2=0x7f1da40ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:44:13.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:13.821+0000 7f1daa01d640 1 -- 172.21.15.100:0/3117634756 shutdown_connections 2024-05-10T11:44:13.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:13.821+0000 7f1daa01d640 1 -- 172.21.15.100:0/3117634756 wait complete. 2024-05-10T11:44:13.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:13.821+0000 7f1daa01d640 1 Processor -- start 2024-05-10T11:44:13.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:13.822+0000 7f1daa01d640 1 -- start start 2024-05-10T11:44:13.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:13.822+0000 7f1daa01d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1da41018d0 0x7f1da406a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:13.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:13.822+0000 7f1daa01d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1da4102ac0 0x7f1da419d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:13.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:13.822+0000 7f1daa01d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1da4103480 0x7f1da419fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:13.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:13.822+0000 7f1daa01d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1da410d280 con 0x7f1da41018d0 2024-05-10T11:44:13.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:13.822+0000 7f1daa01d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1da410d100 con 0x7f1da4103480 2024-05-10T11:44:13.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:13.822+0000 7f1daa01d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1da410d400 con 0x7f1da4102ac0 2024-05-10T11:44:13.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:13.822+0000 7f1da37fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1da41018d0 0x7f1da406a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:13.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:13.822+0000 7f1da37fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1da41018d0 0x7f1da406a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34656/0 (socket says 172.21.15.100:34656) 2024-05-10T11:44:13.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:13.822+0000 7f1da37fe640 1 -- 172.21.15.100:0/2109162353 learned_addr learned my addr 172.21.15.100:0/2109162353 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:44:13.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:13.822+0000 7f1da2ffd640 1 --2- 172.21.15.100:0/2109162353 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1da4102ac0 0x7f1da419d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:13.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:13.822+0000 7f1da3fff640 1 --2- 172.21.15.100:0/2109162353 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1da4103480 0x7f1da419fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:13.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:13.822+0000 7f1da37fe640 1 -- 172.21.15.100:0/2109162353 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1da4102ac0 msgr2=0x7f1da419d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:13.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:13.822+0000 7f1da37fe640 1 --2- 172.21.15.100:0/2109162353 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1da4102ac0 0x7f1da419d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:13.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:13.822+0000 7f1da37fe640 1 -- 172.21.15.100:0/2109162353 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1da4103480 msgr2=0x7f1da419fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:13.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:13.822+0000 7f1da37fe640 1 --2- 172.21.15.100:0/2109162353 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1da4103480 0x7f1da419fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:13.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:13.822+0000 7f1da37fe640 1 -- 172.21.15.100:0/2109162353 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1da41a06f0 con 0x7f1da41018d0 2024-05-10T11:44:13.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:13.822+0000 7f1da2ffd640 1 --2- 172.21.15.100:0/2109162353 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1da4102ac0 0x7f1da419d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:44:13.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:13.822+0000 7f1da37fe640 1 --2- 172.21.15.100:0/2109162353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1da41018d0 0x7f1da406a6d0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f1d94005ab0 tx=0x7f1d940042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:44:13.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:13.823+0000 7f1da0ff9640 1 -- 172.21.15.100:0/2109162353 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9403f030 con 0x7f1da41018d0 2024-05-10T11:44:13.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:13.823+0000 7f1da0ff9640 1 -- 172.21.15.100:0/2109162353 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1d94003c90 con 0x7f1da41018d0 2024-05-10T11:44:13.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:13.823+0000 7f1daa01d640 1 -- 172.21.15.100:0/2109162353 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1da41a0980 con 0x7f1da41018d0 2024-05-10T11:44:13.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:13.823+0000 7f1da0ff9640 1 -- 172.21.15.100:0/2109162353 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d94035070 con 0x7f1da41018d0 2024-05-10T11:44:13.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:13.823+0000 7f1daa01d640 1 -- 172.21.15.100:0/2109162353 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1da41a0e70 con 0x7f1da41018d0 2024-05-10T11:44:13.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:13.824+0000 7f1daa01d640 1 -- 172.21.15.100:0/2109162353 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1da4108d10 con 0x7f1da41018d0 2024-05-10T11:44:13.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:13.825+0000 7f1da0ff9640 1 -- 172.21.15.100:0/2109162353 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f1d94042030 con 0x7f1da41018d0 2024-05-10T11:44:13.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:13.826+0000 7f1da0ff9640 1 --2- 172.21.15.100:0/2109162353 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f1d8c0b27f0 0x7f1d8c0b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:13.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:13.826+0000 7f1da2ffd640 1 --2- 172.21.15.100:0/2109162353 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f1d8c0b27f0 0x7f1d8c0b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:13.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:13.827+0000 7f1da0ff9640 1 -- 172.21.15.100:0/2109162353 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f1d94103180 con 0x7f1da41018d0 2024-05-10T11:44:13.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:13.827+0000 7f1da2ffd640 1 --2- 172.21.15.100:0/2109162353 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f1d8c0b27f0 0x7f1d8c0b4cb0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f1d98005d00 tx=0x7f1d98005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:44:13.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:13.830+0000 7f1da0ff9640 1 -- 172.21.15.100:0/2109162353 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f1d940cab40 con 0x7f1da41018d0 2024-05-10T11:44:14.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.021+0000 7f1daa01d640 1 -- 172.21.15.100:0/2109162353 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1da402dd50 con 0x7f1da41018d0 2024-05-10T11:44:14.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.023+0000 7f1da0ff9640 1 -- 172.21.15.100:0/2109162353 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24737 (secure 0 0 0) 0x7f1d940ca440 con 0x7f1da41018d0 2024-05-10T11:44:14.023 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:44:14.023 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":90,"fsid":"20a542cc-0ebd-11ef-bc98-c7b262605968","created":"2024-05-10T11:06:05.454446+0000","modified":"2024-05-10T11:29:07.126022+0000","last_up_change":"2024-05-10T11:13:18.185471+0000","last_in_change":"2024-05-10T11:12:57.268678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T11:09:26.176441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T11:14:21.004285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T11:14:24.179414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T11:14:33.308517+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5f031418-da23-4e85-8669-160960dadd09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":643028278},{"type":"v1","addr":"172.21.15.100:6803","nonce":643028278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":643028278},{"type":"v1","addr":"172.21.15.100:6805","nonce":643028278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":643028278},{"type":"v1","addr":"172.21.15.100:6809","nonce":643028278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":643028278},{"type":"v1","addr":"172.21.15.100:6807","nonce":643028278}]},"public_addr":"172.21.15.100:6803/643028278","cluster_addr":"172.21.15.100:6805/643028278","heartbeat_back_addr":"172.21.15.100:6809/643028278","heartbeat_front_addr":"172.21.15.100:6807/643028278","state":["exists","up"]},{"osd":1,"uuid":"a908cdc9-001e-4b65-9ab0-0dd08dba20f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":3103987017},{"type":"v1","addr":"172.21.15.178:6801","nonce":3103987017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":3103987017},{"type":"v1","addr":"172.21.15.178:6803","nonce":3103987017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":3103987017},{"type":"v1","addr":"172.21.15.178:6807","nonce":3103987017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":3103987017},{"type":"v1","addr":"172.21.15.178:6805","nonce":3103987017}]},"public_addr":"172.21.15.178:6801/3103987017","cluster_addr":"172.21.15.178:6803/3103987017","heartbeat_back_addr":"172.21.15.178:6807/3103987017","heartbeat_front_addr":"172.21.15.178:6805/3103987017","state":["exists","up"]},{"osd":2,"uuid":"4d8f89ae-9424-46e9-9c80-17c47cc5e8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3991466468},{"type":"v1","addr":"172.21.15.188:6801","nonce":3991466468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3991466468},{"type":"v1","addr":"172.21.15.188:6803","nonce":3991466468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3991466468},{"type":"v1","addr":"172.21.15.188:6807","nonce":3991466468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3991466468},{"type":"v1","addr":"172.21.15.188:6805","nonce":3991466468}]},"public_addr":"172.21.15.188:6801/3991466468","cluster_addr":"172.21.15.188:6803/3991466468","heartbeat_back_addr":"172.21.15.188:6807/3991466468","heartbeat_front_addr":"172.21.15.188:6805/3991466468","state":["exists","up"]},{"osd":3,"uuid":"4c8865aa-fef4-4710-b766-1385adb9462c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1713357192},{"type":"v1","addr":"172.21.15.100:6811","nonce":1713357192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1713357192},{"type":"v1","addr":"172.21.15.100:6813","nonce":1713357192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1713357192},{"type":"v1","addr":"172.21.15.100:6817","nonce":1713357192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1713357192},{"type":"v1","addr":"172.21.15.100:6815","nonce":1713357192}]},"public_addr":"172.21.15.100:6811/1713357192","cluster_addr":"172.21.15.100:6813/1713357192","heartbeat_back_addr":"172.21.15.100:6817/1713357192","heartbeat_front_addr":"172.21.15.100:6815/1713357192","state":["exists","up"]},{"osd":4,"uuid":"a9653800-da57-43eb-8781-3188e8a3949f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":2222814192},{"type":"v1","addr":"172.21.15.178:6809","nonce":2222814192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":2222814192},{"type":"v1","addr":"172.21.15.178:6811","nonce":2222814192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":2222814192},{"type":"v1","addr":"172.21.15.178:6815","nonce":2222814192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":2222814192},{"type":"v1","addr":"172.21.15.178:6813","nonce":2222814192}]},"public_addr":"172.21.15.178:6809/2222814192","cluster_addr":"172.21.15.178:6811/2222814192","heartbeat_back_addr":"172.21.15.178:6815/2222814192","heartbeat_front_addr":"172.21.15.178:6813/2222814192","state":["exists","up"]},{"osd":5,"uuid":"fb5f8af2-1eb0-4d8a-a7a3-ed6dd68e692e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3517375935},{"type":"v1","addr":"172.21.15.188:6809","nonce":3517375935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3517375935},{"type":"v1","addr":"172.21.15.188:6811","nonce":3517375935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3517375935},{"type":"v1","addr":"172.21.15.188:6815","nonce":3517375935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3517375935},{"type":"v1","addr":"172.21.15.188:6813","nonce":3517375935}]},"public_addr":"172.21.15.188:6809/3517375935","cluster_addr":"172.21.15.188:6811/3517375935","heartbeat_back_addr":"172.21.15.188:6815/3517375935","heartbeat_front_addr":"172.21.15.188:6813/3517375935","state":["exists","up"]},{"osd":6,"uuid":"84b57387-0a94-4e49-b099-493d329b87dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1410593862},{"type":"v1","addr":"172.21.15.100:6819","nonce":1410593862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1410593862},{"type":"v1","addr":"172.21.15.100:6821","nonce":1410593862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1410593862},{"type":"v1","addr":"172.21.15.100:6825","nonce":1410593862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1410593862},{"type":"v1","addr":"172.21.15.100:6823","nonce":1410593862}]},"public_addr":"172.21.15.100:6819/1410593862","cluster_addr":"172.21.15.100:6821/1410593862","heartbeat_back_addr":"172.21.15.100:6825/1410593862","heartbeat_front_addr":"172.21.15.100:6823/1410593862","state":["exists","up"]},{"osd":7,"uuid":"02ada28f-ac4f-47cf-983f-8a6da1140abe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2458690455},{"type":"v1","addr":"172.21.15.178:6817","nonce":2458690455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2458690455},{"type":"v1","addr":"172.21.15.178:6819","nonce":2458690455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2458690455},{"type":"v1","addr":"172.21.15.178:6823","nonce":2458690455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2458690455},{"type":"v1","addr":"172.21.15.178:6821","nonce":2458690455}]},"public_addr":"172.21.15.178:6817/2458690455","cluster_addr":"172.21.15.178:6819/2458690455","heartbeat_back_addr":"172.21.15.178:6823/2458690455","heartbeat_front_addr":"172.21.15.178:6821/2458690455","state":["exists","up"]},{"osd":8,"uuid":"e7e489bc-15fc-4a4c-b297-d1bb626513ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1523872826},{"type":"v1","addr":"172.21.15.188:6817","nonce":1523872826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1523872826},{"type":"v1","addr":"172.21.15.188:6819","nonce":1523872826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1523872826},{"type":"v1","addr":"172.21.15.188:6823","nonce":1523872826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1523872826},{"type":"v1","addr":"172.21.15.188:6821","nonce":1523872826}]},"public_addr":"172.21.15.188:6817/1523872826","cluster_addr":"172.21.15.188:6819/1523872826","heartbeat_back_addr":"172.21.15.188:6823/1523872826","heartbeat_front_addr":"172.21.15.188:6821/1523872826","state":["exists","up"]},{"osd":9,"uuid":"72328326-cd16-45d9-bfd9-40f6c5c20f5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3585761978},{"type":"v1","addr":"172.21.15.100:6827","nonce":3585761978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3585761978},{"type":"v1","addr":"172.21.15.100:6829","nonce":3585761978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3585761978},{"type":"v1","addr":"172.21.15.100:6833","nonce":3585761978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3585761978},{"type":"v1","addr":"172.21.15.100:6831","nonce":3585761978}]},"public_addr":"172.21.15.100:6827/3585761978","cluster_addr":"172.21.15.100:6829/3585761978","heartbeat_back_addr":"172.21.15.100:6833/3585761978","heartbeat_front_addr":"172.21.15.100:6831/3585761978","state":["exists","up"]},{"osd":10,"uuid":"a355f9a8-91a3-41d4-814d-5a9c9b953706","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2985841725},{"type":"v1","addr":"172.21.15.178:6825","nonce":2985841725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2985841725},{"type":"v1","addr":"172.21.15.178:6827","nonce":2985841725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2985841725},{"type":"v1","addr":"172.21.15.178:6831","nonce":2985841725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2985841725},{"type":"v1","addr":"172.21.15.178:6829","nonce":2985841725}]},"public_addr":"172.21.15.178:6825/2985841725","cluster_addr":"172.21.15.178:6827/2985841725","heartbeat_back_addr":"172.21.15.178:6831/2985841725","heartbeat_front_addr":"172.21.15.178:6829/2985841725","state":["exists","up"]},{"osd":11,"uuid":"76a5e13c-7e38-42f9-bfa7-7fedec7c7184","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2574955118},{"type":"v1","addr":"172.21.15.188:6825","nonce":2574955118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2574955118},{"type":"v1","addr":"172.21.15.188:6827","nonce":2574955118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2574955118},{"type":"v1","addr":"172.21.15.188:6831","nonce":2574955118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2574955118},{"type":"v1","addr":"172.21.15.188:6829","nonce":2574955118}]},"public_addr":"172.21.15.188:6825/2574955118","cluster_addr":"172.21.15.188:6827/2574955118","heartbeat_back_addr":"172.21.15.188:6831/2574955118","heartbeat_front_addr":"172.21.15.188:6829/2574955118","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:08:36.086637+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:08:58.948031+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:09:22.142517+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:09:47.655748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:10:12.853999+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:10:37.911383+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:11:04.448815+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:11:30.331740+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:11:55.819278+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:12:23.204612+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:12:49.484835+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:13:15.783531+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:6800/1597433518":"2024-05-11T11:07:00.092732+0000","172.21.15.100:6800/1086450256":"2024-05-11T11:06:33.114176+0000","172.21.15.100:0/767964410":"2024-05-11T11:07:00.092732+0000","172.21.15.100:6801/1597433518":"2024-05-11T11:07:00.092732+0000","172.21.15.100:6801/1086450256":"2024-05-11T11:06:33.114176+0000","172.21.15.100:0/2770572070":"2024-05-11T11:06:33.114176+0000","172.21.15.100:0/1379340513":"2024-05-11T11:06:33.114176+0000","172.21.15.100:0/3025646953":"2024-05-11T11:07:00.092732+0000","172.21.15.100:0/4099258930":"2024-05-11T11:07:00.092732+0000","172.21.15.100:0/4267693567":"2024-05-11T11:06:33.114176+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T11:44:14.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.025+0000 7f1daa01d640 1 -- 172.21.15.100:0/2109162353 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f1d8c0b27f0 msgr2=0x7f1d8c0b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:14.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.025+0000 7f1daa01d640 1 --2- 172.21.15.100:0/2109162353 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f1d8c0b27f0 0x7f1d8c0b4cb0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f1d98005d00 tx=0x7f1d98005c70 comp rx=0 tx=0).stop 2024-05-10T11:44:14.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.025+0000 7f1daa01d640 1 -- 172.21.15.100:0/2109162353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1da41018d0 msgr2=0x7f1da406a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:14.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.025+0000 7f1daa01d640 1 --2- 172.21.15.100:0/2109162353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1da41018d0 0x7f1da406a6d0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f1d94005ab0 tx=0x7f1d940042f0 comp rx=0 tx=0).stop 2024-05-10T11:44:14.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.025+0000 7f1daa01d640 1 -- 172.21.15.100:0/2109162353 shutdown_connections 2024-05-10T11:44:14.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.025+0000 7f1daa01d640 1 --2- 172.21.15.100:0/2109162353 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f1d8c0b27f0 0x7f1d8c0b4cb0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:14.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.025+0000 7f1daa01d640 1 --2- 172.21.15.100:0/2109162353 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1da4103480 0x7f1da419fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:14.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.025+0000 7f1daa01d640 1 --2- 172.21.15.100:0/2109162353 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1da4102ac0 0x7f1da419d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:14.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.025+0000 7f1daa01d640 1 --2- 172.21.15.100:0/2109162353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1da41018d0 0x7f1da406a6d0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:14.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.025+0000 7f1daa01d640 1 -- 172.21.15.100:0/2109162353 >> 172.21.15.100:0/2109162353 conn(0x7f1da40fce40 msgr2=0x7f1da4105ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:44:14.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.025+0000 7f1daa01d640 1 -- 172.21.15.100:0/2109162353 shutdown_connections 2024-05-10T11:44:14.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.025+0000 7f1daa01d640 1 -- 172.21.15.100:0/2109162353 wait complete. 2024-05-10T11:44:14.035 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:44:13 smithi100 ceph-mon[30296]: pgmap v1196: 129 pgs: 129 active+clean; 439 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 3.0 MiB/s wr, 315 op/s 2024-05-10T11:44:14.035 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:44:13 smithi100 ceph-mon[30296]: Adjusting osd_memory_target on smithi100 to 176.1M 2024-05-10T11:44:14.036 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:44:13 smithi100 ceph-mon[30296]: Unable to set osd_memory_target on smithi100 to 184709324: error parsing value: Value '184709324' is below minimum 939524096 2024-05-10T11:44:14.036 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:44:13 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:44:14.036 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:44:13 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:44:14.036 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:44:13 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:44:14.036 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:44:13 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:44:14.036 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:44:13 smithi100 ceph-mon[30296]: Adjusting osd_memory_target on smithi178 to 176.1M 2024-05-10T11:44:14.036 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:44:13 smithi100 ceph-mon[30296]: Unable to set osd_memory_target on smithi178 to 184736563: error parsing value: Value '184736563' is below minimum 939524096 2024-05-10T11:44:14.037 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T11:09:26.176441+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T11:14:21.004285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T11:14:24.179414+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-10T11:14:33.308517+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-10T11:44:14.038 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T11:44:14.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.141+0000 7f0d80ba4640 1 -- 172.21.15.100:0/1927340813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d7c1018f0 msgr2=0x7f0d7c101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:14.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.141+0000 7f0d80ba4640 1 --2- 172.21.15.100:0/1927340813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d7c1018f0 0x7f0d7c101d10 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c009900 tx=0x7f0d6c02a410 comp rx=0 tx=0).stop 2024-05-10T11:44:14.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.141+0000 7f0d80ba4640 1 -- 172.21.15.100:0/1927340813 shutdown_connections 2024-05-10T11:44:14.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.141+0000 7f0d80ba4640 1 --2- 172.21.15.100:0/1927340813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d7c1034a0 0x7f0d7c109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:14.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.141+0000 7f0d80ba4640 1 --2- 172.21.15.100:0/1927340813 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d7c102ae0 0x7f0d7c102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:14.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.141+0000 7f0d80ba4640 1 --2- 172.21.15.100:0/1927340813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d7c1018f0 0x7f0d7c101d10 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:14.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.141+0000 7f0d80ba4640 1 -- 172.21.15.100:0/1927340813 >> 172.21.15.100:0/1927340813 conn(0x7f0d7c0fce60 msgr2=0x7f0d7c0ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:44:14.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.141+0000 7f0d80ba4640 1 -- 172.21.15.100:0/1927340813 shutdown_connections 2024-05-10T11:44:14.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.141+0000 7f0d80ba4640 1 -- 172.21.15.100:0/1927340813 wait complete. 2024-05-10T11:44:14.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.142+0000 7f0d80ba4640 1 Processor -- start 2024-05-10T11:44:14.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.142+0000 7f0d80ba4640 1 -- start start 2024-05-10T11:44:14.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.142+0000 7f0d80ba4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d7c1018f0 0x7f0d7c06a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:14.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.142+0000 7f0d80ba4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d7c102ae0 0x7f0d7c19db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:14.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.142+0000 7f0d80ba4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d7c1034a0 0x7f0d7c1a0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:14.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.143+0000 7f0d80ba4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7c10d3a0 con 0x7f0d7c102ae0 2024-05-10T11:44:14.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.143+0000 7f0d80ba4640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7c10d220 con 0x7f0d7c1018f0 2024-05-10T11:44:14.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.143+0000 7f0d80ba4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7c10d520 con 0x7f0d7c1034a0 2024-05-10T11:44:14.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.143+0000 7f0d71d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d7c102ae0 0x7f0d7c19db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:14.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.143+0000 7f0d71d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d7c102ae0 0x7f0d7c19db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34678/0 (socket says 172.21.15.100:34678) 2024-05-10T11:44:14.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.143+0000 7f0d71d9b640 1 -- 172.21.15.100:0/3120015333 learned_addr learned my addr 172.21.15.100:0/3120015333 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:44:14.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.143+0000 7f0d7a59c640 1 --2- 172.21.15.100:0/3120015333 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d7c1018f0 0x7f0d7c06a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:14.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.143+0000 7f0d7ad9d640 1 --2- 172.21.15.100:0/3120015333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d7c1034a0 0x7f0d7c1a0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:14.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.143+0000 7f0d71d9b640 1 -- 172.21.15.100:0/3120015333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d7c1034a0 msgr2=0x7f0d7c1a0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:14.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.143+0000 7f0d71d9b640 1 --2- 172.21.15.100:0/3120015333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d7c1034a0 0x7f0d7c1a0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:14.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.143+0000 7f0d71d9b640 1 -- 172.21.15.100:0/3120015333 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d7c1018f0 msgr2=0x7f0d7c06a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:14.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.143+0000 7f0d71d9b640 1 --2- 172.21.15.100:0/3120015333 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d7c1018f0 0x7f0d7c06a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:14.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.143+0000 7f0d71d9b640 1 -- 172.21.15.100:0/3120015333 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d7c1a0930 con 0x7f0d7c102ae0 2024-05-10T11:44:14.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.143+0000 7f0d7ad9d640 1 --2- 172.21.15.100:0/3120015333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d7c1034a0 0x7f0d7c1a0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:44:14.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.143+0000 7f0d71d9b640 1 --2- 172.21.15.100:0/3120015333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d7c102ae0 0x7f0d7c19db00 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f0d6400b4a0 tx=0x7f0d6400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:44:14.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.143+0000 7f0d73fff640 1 -- 172.21.15.100:0/3120015333 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d64014070 con 0x7f0d7c102ae0 2024-05-10T11:44:14.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.143+0000 7f0d80ba4640 1 -- 172.21.15.100:0/3120015333 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d7c1a0c20 con 0x7f0d7c102ae0 2024-05-10T11:44:14.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.143+0000 7f0d73fff640 1 -- 172.21.15.100:0/3120015333 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0d64010aa0 con 0x7f0d7c102ae0 2024-05-10T11:44:14.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.143+0000 7f0d73fff640 1 -- 172.21.15.100:0/3120015333 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d64017b70 con 0x7f0d7c102ae0 2024-05-10T11:44:14.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.143+0000 7f0d80ba4640 1 -- 172.21.15.100:0/3120015333 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d7c1a1170 con 0x7f0d7c102ae0 2024-05-10T11:44:14.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.144+0000 7f0d80ba4640 1 -- 172.21.15.100:0/3120015333 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d7c108e20 con 0x7f0d7c102ae0 2024-05-10T11:44:14.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.146+0000 7f0d73fff640 1 -- 172.21.15.100:0/3120015333 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f0d64017070 con 0x7f0d7c102ae0 2024-05-10T11:44:14.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.146+0000 7f0d73fff640 1 --2- 172.21.15.100:0/3120015333 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0d680b2870 0x7f0d680b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:14.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.147+0000 7f0d7a59c640 1 --2- 172.21.15.100:0/3120015333 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0d680b2870 0x7f0d680b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:14.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.147+0000 7f0d73fff640 1 -- 172.21.15.100:0/3120015333 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f0d640e1cb0 con 0x7f0d7c102ae0 2024-05-10T11:44:14.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.147+0000 7f0d7a59c640 1 --2- 172.21.15.100:0/3120015333 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0d680b2870 0x7f0d680b4d30 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c0057c0 tx=0x7f0d6c031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:44:14.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.151+0000 7f0d73fff640 1 -- 172.21.15.100:0/3120015333 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f0d640a9610 con 0x7f0d7c102ae0 2024-05-10T11:44:14.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:44:13 smithi178 ceph-mon[33359]: pgmap v1196: 129 pgs: 129 active+clean; 439 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 3.0 MiB/s wr, 315 op/s 2024-05-10T11:44:14.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:44:13 smithi178 ceph-mon[33359]: Adjusting osd_memory_target on smithi100 to 176.1M 2024-05-10T11:44:14.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:44:13 smithi178 ceph-mon[33359]: Unable to set osd_memory_target on smithi100 to 184709324: error parsing value: Value '184709324' is below minimum 939524096 2024-05-10T11:44:14.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:44:13 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:44:14.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:44:13 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:44:14.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:44:13 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:44:14.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:44:13 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:44:14.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:44:13 smithi178 ceph-mon[33359]: Adjusting osd_memory_target on smithi178 to 176.1M 2024-05-10T11:44:14.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:44:13 smithi178 ceph-mon[33359]: Unable to set osd_memory_target on smithi178 to 184736563: error parsing value: Value '184736563' is below minimum 939524096 2024-05-10T11:44:14.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:44:13 smithi188 ceph-mon[34075]: pgmap v1196: 129 pgs: 129 active+clean; 439 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 3.0 MiB/s wr, 315 op/s 2024-05-10T11:44:14.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:44:13 smithi188 ceph-mon[34075]: Adjusting osd_memory_target on smithi100 to 176.1M 2024-05-10T11:44:14.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:44:13 smithi188 ceph-mon[34075]: Unable to set osd_memory_target on smithi100 to 184709324: error parsing value: Value '184709324' is below minimum 939524096 2024-05-10T11:44:14.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:44:13 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:44:14.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:44:13 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:44:14.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:44:13 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:44:14.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:44:13 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:44:14.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:44:13 smithi188 ceph-mon[34075]: Adjusting osd_memory_target on smithi178 to 176.1M 2024-05-10T11:44:14.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:44:13 smithi188 ceph-mon[34075]: Unable to set osd_memory_target on smithi178 to 184736563: error parsing value: Value '184736563' is below minimum 939524096 2024-05-10T11:44:14.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.337+0000 7f0d80ba4640 1 -- 172.21.15.100:0/3120015333 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0d7c0635e0 con 0x7f0d7c102ae0 2024-05-10T11:44:14.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.338+0000 7f0d73fff640 1 -- 172.21.15.100:0/3120015333 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f0d640a8f10 con 0x7f0d7c102ae0 2024-05-10T11:44:14.338 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-05-10T11:44:14.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.340+0000 7f0d80ba4640 1 -- 172.21.15.100:0/3120015333 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0d680b2870 msgr2=0x7f0d680b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:14.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.340+0000 7f0d80ba4640 1 --2- 172.21.15.100:0/3120015333 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0d680b2870 0x7f0d680b4d30 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c0057c0 tx=0x7f0d6c031400 comp rx=0 tx=0).stop 2024-05-10T11:44:14.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.340+0000 7f0d80ba4640 1 -- 172.21.15.100:0/3120015333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d7c102ae0 msgr2=0x7f0d7c19db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:14.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.340+0000 7f0d80ba4640 1 --2- 172.21.15.100:0/3120015333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d7c102ae0 0x7f0d7c19db00 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f0d6400b4a0 tx=0x7f0d6400b960 comp rx=0 tx=0).stop 2024-05-10T11:44:14.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.340+0000 7f0d80ba4640 1 -- 172.21.15.100:0/3120015333 shutdown_connections 2024-05-10T11:44:14.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.340+0000 7f0d80ba4640 1 --2- 172.21.15.100:0/3120015333 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f0d680b2870 0x7f0d680b4d30 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:14.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.340+0000 7f0d80ba4640 1 --2- 172.21.15.100:0/3120015333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d7c1034a0 0x7f0d7c1a0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:14.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.340+0000 7f0d80ba4640 1 --2- 172.21.15.100:0/3120015333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d7c102ae0 0x7f0d7c19db00 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:14.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.340+0000 7f0d80ba4640 1 --2- 172.21.15.100:0/3120015333 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d7c1018f0 0x7f0d7c06a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:14.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.340+0000 7f0d80ba4640 1 -- 172.21.15.100:0/3120015333 >> 172.21.15.100:0/3120015333 conn(0x7f0d7c0fce60 msgr2=0x7f0d7c105ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:44:14.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.340+0000 7f0d80ba4640 1 -- 172.21.15.100:0/3120015333 shutdown_connections 2024-05-10T11:44:14.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.340+0000 7f0d80ba4640 1 -- 172.21.15.100:0/3120015333 wait complete. 2024-05-10T11:44:14.351 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T11:44:14.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.455+0000 7f2e64fc1640 1 -- 172.21.15.100:0/2940516106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e601034a0 msgr2=0x7f2e60109d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:14.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.455+0000 7f2e64fc1640 1 --2- 172.21.15.100:0/2940516106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e601034a0 0x7f2e60109d30 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f2e58009900 tx=0x7f2e5802a410 comp rx=0 tx=0).stop 2024-05-10T11:44:14.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.456+0000 7f2e64fc1640 1 -- 172.21.15.100:0/2940516106 shutdown_connections 2024-05-10T11:44:14.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.456+0000 7f2e64fc1640 1 --2- 172.21.15.100:0/2940516106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e601034a0 0x7f2e60109d30 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:14.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.456+0000 7f2e64fc1640 1 --2- 172.21.15.100:0/2940516106 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e60102ae0 0x7f2e60102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:14.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.456+0000 7f2e64fc1640 1 --2- 172.21.15.100:0/2940516106 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2e601018f0 0x7f2e60101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:14.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.456+0000 7f2e64fc1640 1 -- 172.21.15.100:0/2940516106 >> 172.21.15.100:0/2940516106 conn(0x7f2e600fce60 msgr2=0x7f2e600ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:44:14.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.456+0000 7f2e64fc1640 1 -- 172.21.15.100:0/2940516106 shutdown_connections 2024-05-10T11:44:14.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.456+0000 7f2e64fc1640 1 -- 172.21.15.100:0/2940516106 wait complete. 2024-05-10T11:44:14.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.456+0000 7f2e64fc1640 1 Processor -- start 2024-05-10T11:44:14.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.456+0000 7f2e64fc1640 1 -- start start 2024-05-10T11:44:14.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.457+0000 7f2e64fc1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e601018f0 0x7f2e60197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:14.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.457+0000 7f2e64fc1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2e60102ae0 0x7f2e60197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:14.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.457+0000 7f2e64fc1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e601034a0 0x7f2e6019efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:14.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.457+0000 7f2e64fc1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2e6010d330 con 0x7f2e601034a0 2024-05-10T11:44:14.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.457+0000 7f2e64fc1640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2e6010d1b0 con 0x7f2e601018f0 2024-05-10T11:44:14.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.457+0000 7f2e64fc1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2e6010d4b0 con 0x7f2e60102ae0 2024-05-10T11:44:14.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.457+0000 7f2e5ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e601034a0 0x7f2e6019efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:14.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.457+0000 7f2e5ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e601034a0 0x7f2e6019efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34704/0 (socket says 172.21.15.100:34704) 2024-05-10T11:44:14.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.457+0000 7f2e5ed9d640 1 -- 172.21.15.100:0/3016306620 learned_addr learned my addr 172.21.15.100:0/3016306620 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:44:14.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.457+0000 7f2e5dd9b640 1 --2- 172.21.15.100:0/3016306620 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2e60102ae0 0x7f2e60197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:14.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.457+0000 7f2e5e59c640 1 --2- 172.21.15.100:0/3016306620 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e601018f0 0x7f2e60197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:14.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.457+0000 7f2e5ed9d640 1 -- 172.21.15.100:0/3016306620 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2e60102ae0 msgr2=0x7f2e60197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:14.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.457+0000 7f2e5ed9d640 1 --2- 172.21.15.100:0/3016306620 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2e60102ae0 0x7f2e60197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:14.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.457+0000 7f2e5ed9d640 1 -- 172.21.15.100:0/3016306620 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e601018f0 msgr2=0x7f2e60197a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:14.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.457+0000 7f2e5ed9d640 1 --2- 172.21.15.100:0/3016306620 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e601018f0 0x7f2e60197a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:14.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.457+0000 7f2e5ed9d640 1 -- 172.21.15.100:0/3016306620 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e6019f650 con 0x7f2e601034a0 2024-05-10T11:44:14.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.457+0000 7f2e5dd9b640 1 --2- 172.21.15.100:0/3016306620 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2e60102ae0 0x7f2e60197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:44:14.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.457+0000 7f2e5e59c640 1 --2- 172.21.15.100:0/3016306620 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e601018f0 0x7f2e60197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:44:14.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.457+0000 7f2e5ed9d640 1 --2- 172.21.15.100:0/3016306620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e601034a0 0x7f2e6019efe0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f2e58005940 tx=0x7f2e580042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:44:14.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.457+0000 7f2e577fe640 1 -- 172.21.15.100:0/3016306620 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e5803f030 con 0x7f2e601034a0 2024-05-10T11:44:14.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.457+0000 7f2e577fe640 1 -- 172.21.15.100:0/3016306620 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2e58003c90 con 0x7f2e601034a0 2024-05-10T11:44:14.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.457+0000 7f2e64fc1640 1 -- 172.21.15.100:0/3016306620 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e6019f8e0 con 0x7f2e601034a0 2024-05-10T11:44:14.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.457+0000 7f2e577fe640 1 -- 172.21.15.100:0/3016306620 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e58035070 con 0x7f2e601034a0 2024-05-10T11:44:14.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.457+0000 7f2e64fc1640 1 -- 172.21.15.100:0/3016306620 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e6019fea0 con 0x7f2e601034a0 2024-05-10T11:44:14.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.458+0000 7f2e64fc1640 1 -- 172.21.15.100:0/3016306620 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e60108ce0 con 0x7f2e601034a0 2024-05-10T11:44:14.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.460+0000 7f2e577fe640 1 -- 172.21.15.100:0/3016306620 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f2e58042030 con 0x7f2e601034a0 2024-05-10T11:44:14.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.461+0000 7f2e577fe640 1 --2- 172.21.15.100:0/3016306620 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2e4c0b2870 0x7f2e4c0b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:14.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.461+0000 7f2e5e59c640 1 --2- 172.21.15.100:0/3016306620 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2e4c0b2870 0x7f2e4c0b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:14.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.461+0000 7f2e577fe640 1 -- 172.21.15.100:0/3016306620 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f2e58103250 con 0x7f2e601034a0 2024-05-10T11:44:14.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.461+0000 7f2e5e59c640 1 --2- 172.21.15.100:0/3016306620 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2e4c0b2870 0x7f2e4c0b4d30 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f2e60102940 tx=0x7f2e50008370 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:44:14.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.464+0000 7f2e577fe640 1 -- 172.21.15.100:0/3016306620 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f2e580cabb0 con 0x7f2e601034a0 2024-05-10T11:44:14.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.649+0000 7f2e64fc1640 1 -- 172.21.15.100:0/3016306620 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2e6002db50 con 0x7f2e601034a0 2024-05-10T11:44:14.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.649+0000 7f2e577fe640 1 -- 172.21.15.100:0/3016306620 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f2e580ca4b0 con 0x7f2e601034a0 2024-05-10T11:44:14.649 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-05-10T11:44:14.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.651+0000 7f2e64fc1640 1 -- 172.21.15.100:0/3016306620 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2e4c0b2870 msgr2=0x7f2e4c0b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:14.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.651+0000 7f2e64fc1640 1 --2- 172.21.15.100:0/3016306620 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2e4c0b2870 0x7f2e4c0b4d30 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f2e60102940 tx=0x7f2e50008370 comp rx=0 tx=0).stop 2024-05-10T11:44:14.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.651+0000 7f2e64fc1640 1 -- 172.21.15.100:0/3016306620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e601034a0 msgr2=0x7f2e6019efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:14.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.651+0000 7f2e64fc1640 1 --2- 172.21.15.100:0/3016306620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e601034a0 0x7f2e6019efe0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f2e58005940 tx=0x7f2e580042f0 comp rx=0 tx=0).stop 2024-05-10T11:44:14.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.651+0000 7f2e64fc1640 1 -- 172.21.15.100:0/3016306620 shutdown_connections 2024-05-10T11:44:14.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.651+0000 7f2e64fc1640 1 --2- 172.21.15.100:0/3016306620 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f2e4c0b2870 0x7f2e4c0b4d30 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:14.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.651+0000 7f2e64fc1640 1 --2- 172.21.15.100:0/3016306620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e601034a0 0x7f2e6019efe0 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:14.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.651+0000 7f2e64fc1640 1 --2- 172.21.15.100:0/3016306620 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2e60102ae0 0x7f2e60197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:14.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.651+0000 7f2e64fc1640 1 --2- 172.21.15.100:0/3016306620 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e601018f0 0x7f2e60197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:14.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.651+0000 7f2e64fc1640 1 -- 172.21.15.100:0/3016306620 >> 172.21.15.100:0/3016306620 conn(0x7f2e600fce60 msgr2=0x7f2e60105eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:44:14.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.651+0000 7f2e64fc1640 1 -- 172.21.15.100:0/3016306620 shutdown_connections 2024-05-10T11:44:14.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.651+0000 7f2e64fc1640 1 -- 172.21.15.100:0/3016306620 wait complete. 2024-05-10T11:44:14.663 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T11:44:14.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.767+0000 7f809ead9640 1 -- 172.21.15.100:0/4140719622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8098103d50 msgr2=0x7f80981041d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:14.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.767+0000 7f809ead9640 1 --2- 172.21.15.100:0/4140719622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8098103d50 0x7f80981041d0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f8084009930 tx=0x7f808402a420 comp rx=0 tx=0).stop 2024-05-10T11:44:14.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.767+0000 7f809ead9640 1 -- 172.21.15.100:0/4140719622 shutdown_connections 2024-05-10T11:44:14.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.767+0000 7f809ead9640 1 --2- 172.21.15.100:0/4140719622 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8098104710 0x7f809810afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:14.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.767+0000 7f809ead9640 1 --2- 172.21.15.100:0/4140719622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8098103d50 0x7f80981041d0 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:14.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.767+0000 7f809ead9640 1 --2- 172.21.15.100:0/4140719622 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8098100ce0 0x7f8098101100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:14.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.767+0000 7f809ead9640 1 -- 172.21.15.100:0/4140719622 >> 172.21.15.100:0/4140719622 conn(0x7f80980fc5f0 msgr2=0x7f80980fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:44:14.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.767+0000 7f809ead9640 1 -- 172.21.15.100:0/4140719622 shutdown_connections 2024-05-10T11:44:14.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.767+0000 7f809ead9640 1 -- 172.21.15.100:0/4140719622 wait complete. 2024-05-10T11:44:14.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.768+0000 7f809ead9640 1 Processor -- start 2024-05-10T11:44:14.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.768+0000 7f809ead9640 1 -- start start 2024-05-10T11:44:14.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.768+0000 7f809ead9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8098100ce0 0x7f8098198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:14.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.768+0000 7f809ead9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8098103d50 0x7f80981990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:14.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.768+0000 7f809ead9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8098104710 0x7f80981a0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:14.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.768+0000 7f809ead9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f809810e3f0 con 0x7f8098103d50 2024-05-10T11:44:14.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.768+0000 7f809ead9640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f809810e270 con 0x7f8098104710 2024-05-10T11:44:14.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.768+0000 7f809ead9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f809810e570 con 0x7f8098100ce0 2024-05-10T11:44:14.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.768+0000 7f8097fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8098103d50 0x7f80981990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:14.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.768+0000 7f8097fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8098103d50 0x7f80981990c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34716/0 (socket says 172.21.15.100:34716) 2024-05-10T11:44:14.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.768+0000 7f8097fff640 1 -- 172.21.15.100:0/2267235106 learned_addr learned my addr 172.21.15.100:0/2267235106 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:44:14.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.768+0000 7f809d076640 1 --2- 172.21.15.100:0/2267235106 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8098104710 0x7f80981a0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:14.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.768+0000 7f809c875640 1 --2- 172.21.15.100:0/2267235106 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8098100ce0 0x7f8098198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:14.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.768+0000 7f8097fff640 1 -- 172.21.15.100:0/2267235106 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8098100ce0 msgr2=0x7f8098198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:14.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.768+0000 7f8097fff640 1 --2- 172.21.15.100:0/2267235106 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8098100ce0 0x7f8098198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:14.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.768+0000 7f8097fff640 1 -- 172.21.15.100:0/2267235106 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8098104710 msgr2=0x7f80981a0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:14.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.768+0000 7f8097fff640 1 --2- 172.21.15.100:0/2267235106 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8098104710 0x7f80981a0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:14.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.768+0000 7f8097fff640 1 -- 172.21.15.100:0/2267235106 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80981a07a0 con 0x7f8098103d50 2024-05-10T11:44:14.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.768+0000 7f8097fff640 1 --2- 172.21.15.100:0/2267235106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8098103d50 0x7f80981990c0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f8084009900 tx=0x7f80840040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:44:14.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.769+0000 7f8095ffb640 1 -- 172.21.15.100:0/2267235106 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f808403f030 con 0x7f8098103d50 2024-05-10T11:44:14.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.769+0000 7f8095ffb640 1 -- 172.21.15.100:0/2267235106 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f80840044c0 con 0x7f8098103d50 2024-05-10T11:44:14.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.769+0000 7f8095ffb640 1 -- 172.21.15.100:0/2267235106 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8084035070 con 0x7f8098103d50 2024-05-10T11:44:14.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.769+0000 7f809ead9640 1 -- 172.21.15.100:0/2267235106 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80981a0a30 con 0x7f8098103d50 2024-05-10T11:44:14.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.769+0000 7f809ead9640 1 -- 172.21.15.100:0/2267235106 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80981a0f20 con 0x7f8098103d50 2024-05-10T11:44:14.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.770+0000 7f809ead9640 1 -- 172.21.15.100:0/2267235106 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8098109ff0 con 0x7f8098103d50 2024-05-10T11:44:14.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.771+0000 7f8095ffb640 1 -- 172.21.15.100:0/2267235106 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f8084042030 con 0x7f8098103d50 2024-05-10T11:44:14.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.772+0000 7f8095ffb640 1 --2- 172.21.15.100:0/2267235106 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f80880b27f0 0x7f80880b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:14.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.772+0000 7f809c875640 1 --2- 172.21.15.100:0/2267235106 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f80880b27f0 0x7f80880b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:14.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.772+0000 7f8095ffb640 1 -- 172.21.15.100:0/2267235106 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f8084102a40 con 0x7f8098103d50 2024-05-10T11:44:14.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.773+0000 7f809c875640 1 --2- 172.21.15.100:0/2267235106 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f80880b27f0 0x7f80880b4cb0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f808c005fd0 tx=0x7f808c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:44:14.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.777+0000 7f8095ffb640 1 -- 172.21.15.100:0/2267235106 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f80840c9db0 con 0x7f8098103d50 2024-05-10T11:44:14.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.964+0000 7f809ead9640 1 -- 172.21.15.100:0/2267235106 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f80980635b0 con 0x7f8098103d50 2024-05-10T11:44:14.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.964+0000 7f8095ffb640 1 -- 172.21.15.100:0/2267235106 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7f80840ce0b0 con 0x7f8098103d50 2024-05-10T11:44:14.965 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 64 2024-05-10T11:44:14.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.966+0000 7f809ead9640 1 -- 172.21.15.100:0/2267235106 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f80880b27f0 msgr2=0x7f80880b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:14.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.966+0000 7f809ead9640 1 --2- 172.21.15.100:0/2267235106 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f80880b27f0 0x7f80880b4cb0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f808c005fd0 tx=0x7f808c00b390 comp rx=0 tx=0).stop 2024-05-10T11:44:14.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.966+0000 7f809ead9640 1 -- 172.21.15.100:0/2267235106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8098103d50 msgr2=0x7f80981990c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:14.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.966+0000 7f809ead9640 1 --2- 172.21.15.100:0/2267235106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8098103d50 0x7f80981990c0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f8084009900 tx=0x7f80840040c0 comp rx=0 tx=0).stop 2024-05-10T11:44:14.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.966+0000 7f809ead9640 1 -- 172.21.15.100:0/2267235106 shutdown_connections 2024-05-10T11:44:14.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.966+0000 7f809ead9640 1 --2- 172.21.15.100:0/2267235106 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f80880b27f0 0x7f80880b4cb0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:14.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.966+0000 7f809ead9640 1 --2- 172.21.15.100:0/2267235106 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8098104710 0x7f80981a0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:14.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.966+0000 7f809ead9640 1 --2- 172.21.15.100:0/2267235106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8098103d50 0x7f80981990c0 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:14.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.966+0000 7f809ead9640 1 --2- 172.21.15.100:0/2267235106 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8098100ce0 0x7f8098198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:14.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.966+0000 7f809ead9640 1 -- 172.21.15.100:0/2267235106 >> 172.21.15.100:0/2267235106 conn(0x7f80980fc5f0 msgr2=0x7f80981071c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:44:14.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.967+0000 7f809ead9640 1 -- 172.21.15.100:0/2267235106 shutdown_connections 2024-05-10T11:44:14.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:14.967+0000 7f809ead9640 1 -- 172.21.15.100:0/2267235106 wait complete. 2024-05-10T11:44:14.977 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:44:14 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/2109162353' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T11:44:14.977 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:44:14 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/3120015333' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T11:44:14.977 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:44:14 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/3016306620' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T11:44:14.978 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-10T11:44:15.028 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:44:14 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/2109162353' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T11:44:15.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:44:14 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/3120015333' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T11:44:15.029 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:44:14 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/3016306620' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T11:44:15.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.082+0000 7f82100dc640 1 -- 172.21.15.100:0/1554670713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8208102520 msgr2=0x7f8208109dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:15.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.082+0000 7f82100dc640 1 --2- 172.21.15.100:0/1554670713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8208102520 0x7f8208109dc0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f8200009ca0 tx=0x7f820002a8c0 comp rx=0 tx=0).stop 2024-05-10T11:44:15.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.082+0000 7f82100dc640 1 -- 172.21.15.100:0/1554670713 shutdown_connections 2024-05-10T11:44:15.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.082+0000 7f82100dc640 1 --2- 172.21.15.100:0/1554670713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8208102520 0x7f8208109dc0 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:15.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.082+0000 7f82100dc640 1 --2- 172.21.15.100:0/1554670713 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8208101b60 0x7f8208101fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:15.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.082+0000 7f82100dc640 1 --2- 172.21.15.100:0/1554670713 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8208102f10 0x7f8208103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:15.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.082+0000 7f82100dc640 1 -- 172.21.15.100:0/1554670713 >> 172.21.15.100:0/1554670713 conn(0x7f82080fc5f0 msgr2=0x7f82080fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:44:15.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.082+0000 7f82100dc640 1 -- 172.21.15.100:0/1554670713 shutdown_connections 2024-05-10T11:44:15.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.082+0000 7f82100dc640 1 -- 172.21.15.100:0/1554670713 wait complete. 2024-05-10T11:44:15.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.083+0000 7f82100dc640 1 Processor -- start 2024-05-10T11:44:15.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.083+0000 7f82100dc640 1 -- start start 2024-05-10T11:44:15.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.083+0000 7f82100dc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8208101b60 0x7f8208198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:15.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.083+0000 7f82100dc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8208102520 0x7f8208199020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:15.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.083+0000 7f82100dc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8208102f10 0x7f82081a0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:15.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.083+0000 7f82100dc640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f820810d280 con 0x7f8208102520 2024-05-10T11:44:15.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.083+0000 7f82100dc640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f820810d100 con 0x7f8208102f10 2024-05-10T11:44:15.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.083+0000 7f82100dc640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f820810d400 con 0x7f8208101b60 2024-05-10T11:44:15.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.083+0000 7f820d677640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8208102520 0x7f8208199020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:15.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.083+0000 7f820d677640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8208102520 0x7f8208199020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34736/0 (socket says 172.21.15.100:34736) 2024-05-10T11:44:15.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.083+0000 7f820d677640 1 -- 172.21.15.100:0/1009135002 learned_addr learned my addr 172.21.15.100:0/1009135002 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:44:15.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.083+0000 7f820de78640 1 --2- 172.21.15.100:0/1009135002 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8208101b60 0x7f8208198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:15.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.083+0000 7f820e679640 1 --2- 172.21.15.100:0/1009135002 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8208102f10 0x7f82081a0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:15.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.083+0000 7f820d677640 1 -- 172.21.15.100:0/1009135002 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8208101b60 msgr2=0x7f8208198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:15.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.083+0000 7f820d677640 1 --2- 172.21.15.100:0/1009135002 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8208101b60 0x7f8208198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:15.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.083+0000 7f820d677640 1 -- 172.21.15.100:0/1009135002 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8208102f10 msgr2=0x7f82081a0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:15.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.083+0000 7f820d677640 1 --2- 172.21.15.100:0/1009135002 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8208102f10 0x7f82081a0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:15.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.083+0000 7f820d677640 1 -- 172.21.15.100:0/1009135002 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82081a0700 con 0x7f8208102520 2024-05-10T11:44:15.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.083+0000 7f820d677640 1 --2- 172.21.15.100:0/1009135002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8208102520 0x7f8208199020 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f820400d770 tx=0x7f820400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:44:15.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.083+0000 7f81faffd640 1 -- 172.21.15.100:0/1009135002 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8204017070 con 0x7f8208102520 2024-05-10T11:44:15.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.083+0000 7f81faffd640 1 -- 172.21.15.100:0/1009135002 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8204009c30 con 0x7f8208102520 2024-05-10T11:44:15.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.083+0000 7f82100dc640 1 -- 172.21.15.100:0/1009135002 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82081a09f0 con 0x7f8208102520 2024-05-10T11:44:15.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.083+0000 7f81faffd640 1 -- 172.21.15.100:0/1009135002 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8204003c50 con 0x7f8208102520 2024-05-10T11:44:15.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.084+0000 7f82100dc640 1 -- 172.21.15.100:0/1009135002 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82081a0f40 con 0x7f8208102520 2024-05-10T11:44:15.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.085+0000 7f82100dc640 1 -- 172.21.15.100:0/1009135002 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82081a11d0 con 0x7f8208102520 2024-05-10T11:44:15.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.086+0000 7f81faffd640 1 -- 172.21.15.100:0/1009135002 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f8204003df0 con 0x7f8208102520 2024-05-10T11:44:15.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.087+0000 7f81faffd640 1 --2- 172.21.15.100:0/1009135002 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f81f40b27f0 0x7f81f40b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:15.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.087+0000 7f820de78640 1 --2- 172.21.15.100:0/1009135002 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f81f40b27f0 0x7f81f40b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:15.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.087+0000 7f81faffd640 1 -- 172.21.15.100:0/1009135002 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f82040e2030 con 0x7f8208102520 2024-05-10T11:44:15.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.087+0000 7f820de78640 1 --2- 172.21.15.100:0/1009135002 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f81f40b27f0 0x7f81f40b4cb0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f81fc005fd0 tx=0x7f81fc005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:44:15.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.091+0000 7f81faffd640 1 -- 172.21.15.100:0/1009135002 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f82040a99f0 con 0x7f8208102520 2024-05-10T11:44:15.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:44:14 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/2109162353' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T11:44:15.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:44:14 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/3120015333' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T11:44:15.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:44:14 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/3016306620' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T11:44:15.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.274+0000 7f82100dc640 1 -- 172.21.15.100:0/1009135002 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f82081a13a0 con 0x7f8208102520 2024-05-10T11:44:15.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.274+0000 7f81faffd640 1 -- 172.21.15.100:0/1009135002 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) ==== 103+0+11 (secure 0 0 0) 0x7f8204009de0 con 0x7f8208102520 2024-05-10T11:44:15.275 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-05-10T11:44:15.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.276+0000 7f82100dc640 1 -- 172.21.15.100:0/1009135002 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f81f40b27f0 msgr2=0x7f81f40b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:15.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.276+0000 7f82100dc640 1 --2- 172.21.15.100:0/1009135002 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f81f40b27f0 0x7f81f40b4cb0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f81fc005fd0 tx=0x7f81fc005850 comp rx=0 tx=0).stop 2024-05-10T11:44:15.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.276+0000 7f82100dc640 1 -- 172.21.15.100:0/1009135002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8208102520 msgr2=0x7f8208199020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:15.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.276+0000 7f82100dc640 1 --2- 172.21.15.100:0/1009135002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8208102520 0x7f8208199020 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f820400d770 tx=0x7f820400dc30 comp rx=0 tx=0).stop 2024-05-10T11:44:15.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.276+0000 7f82100dc640 1 -- 172.21.15.100:0/1009135002 shutdown_connections 2024-05-10T11:44:15.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.276+0000 7f82100dc640 1 --2- 172.21.15.100:0/1009135002 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f81f40b27f0 0x7f81f40b4cb0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:15.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.276+0000 7f82100dc640 1 --2- 172.21.15.100:0/1009135002 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8208102f10 0x7f82081a0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:15.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.276+0000 7f82100dc640 1 --2- 172.21.15.100:0/1009135002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8208102520 0x7f8208199020 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:15.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.276+0000 7f82100dc640 1 --2- 172.21.15.100:0/1009135002 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8208101b60 0x7f8208198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:15.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.276+0000 7f82100dc640 1 -- 172.21.15.100:0/1009135002 >> 172.21.15.100:0/1009135002 conn(0x7f82080fc5f0 msgr2=0x7f8208100a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:44:15.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.277+0000 7f82100dc640 1 -- 172.21.15.100:0/1009135002 shutdown_connections 2024-05-10T11:44:15.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.277+0000 7f82100dc640 1 -- 172.21.15.100:0/1009135002 wait complete. 2024-05-10T11:44:15.288 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-05-10T11:44:15.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.390+0000 7fd6f1bf7640 1 -- 172.21.15.100:0/1622092473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6ec102a00 msgr2=0x7fd6ec102e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:15.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.390+0000 7fd6f1bf7640 1 --2- 172.21.15.100:0/1622092473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6ec102a00 0x7fd6ec102e20 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc009900 tx=0x7fd6dc02a410 comp rx=0 tx=0).stop 2024-05-10T11:44:15.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.390+0000 7fd6f1bf7640 1 -- 172.21.15.100:0/1622092473 shutdown_connections 2024-05-10T11:44:15.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.390+0000 7fd6f1bf7640 1 --2- 172.21.15.100:0/1622092473 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd6ec1045b0 0x7fd6ec10ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:15.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.390+0000 7fd6f1bf7640 1 --2- 172.21.15.100:0/1622092473 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6ec103bf0 0x7fd6ec104070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:15.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.390+0000 7fd6f1bf7640 1 --2- 172.21.15.100:0/1622092473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6ec102a00 0x7fd6ec102e20 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:15.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.390+0000 7fd6f1bf7640 1 -- 172.21.15.100:0/1622092473 >> 172.21.15.100:0/1622092473 conn(0x7fd6ec0fdf90 msgr2=0x7fd6ec1003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:44:15.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.390+0000 7fd6f1bf7640 1 -- 172.21.15.100:0/1622092473 shutdown_connections 2024-05-10T11:44:15.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.390+0000 7fd6f1bf7640 1 -- 172.21.15.100:0/1622092473 wait complete. 2024-05-10T11:44:15.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.391+0000 7fd6f1bf7640 1 Processor -- start 2024-05-10T11:44:15.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.391+0000 7fd6f1bf7640 1 -- start start 2024-05-10T11:44:15.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.391+0000 7fd6f1bf7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6ec102a00 0x7fd6ec198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:15.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.391+0000 7fd6f1bf7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd6ec103bf0 0x7fd6ec199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:15.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.391+0000 7fd6f1bf7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6ec1045b0 0x7fd6ec1a0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:15.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.391+0000 7fd6f1bf7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd6ec10e3d0 con 0x7fd6ec1045b0 2024-05-10T11:44:15.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.391+0000 7fd6f1bf7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd6ec10e250 con 0x7fd6ec103bf0 2024-05-10T11:44:15.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.391+0000 7fd6f1bf7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd6ec10e550 con 0x7fd6ec102a00 2024-05-10T11:44:15.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.391+0000 7fd6ebfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6ec1045b0 0x7fd6ec1a0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:15.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.391+0000 7fd6ebfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6ec1045b0 0x7fd6ec1a0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34748/0 (socket says 172.21.15.100:34748) 2024-05-10T11:44:15.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.391+0000 7fd6ebfff640 1 -- 172.21.15.100:0/290447203 learned_addr learned my addr 172.21.15.100:0/290447203 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:44:15.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.391+0000 7fd6e3fff640 1 --2- 172.21.15.100:0/290447203 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd6ec103bf0 0x7fd6ec199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:15.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.391+0000 7fd6eb7fe640 1 --2- 172.21.15.100:0/290447203 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6ec102a00 0x7fd6ec198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:15.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.391+0000 7fd6ebfff640 1 -- 172.21.15.100:0/290447203 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6ec102a00 msgr2=0x7fd6ec198bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:15.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.391+0000 7fd6ebfff640 1 --2- 172.21.15.100:0/290447203 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6ec102a00 0x7fd6ec198bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:15.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.391+0000 7fd6ebfff640 1 -- 172.21.15.100:0/290447203 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd6ec103bf0 msgr2=0x7fd6ec199100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:15.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.391+0000 7fd6ebfff640 1 --2- 172.21.15.100:0/290447203 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd6ec103bf0 0x7fd6ec199100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:15.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.391+0000 7fd6ebfff640 1 -- 172.21.15.100:0/290447203 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6ec1a07e0 con 0x7fd6ec1045b0 2024-05-10T11:44:15.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.391+0000 7fd6ebfff640 1 --2- 172.21.15.100:0/290447203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6ec1045b0 0x7fd6ec1a0170 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4009fd0 tx=0x7fd6e400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:44:15.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.392+0000 7fd6e97fa640 1 -- 172.21.15.100:0/290447203 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e4017070 con 0x7fd6ec1045b0 2024-05-10T11:44:15.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.392+0000 7fd6e97fa640 1 -- 172.21.15.100:0/290447203 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd6e4003e00 con 0x7fd6ec1045b0 2024-05-10T11:44:15.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.392+0000 7fd6e97fa640 1 -- 172.21.15.100:0/290447203 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e4010460 con 0x7fd6ec1045b0 2024-05-10T11:44:15.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.392+0000 7fd6f1bf7640 1 -- 172.21.15.100:0/290447203 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6ec1a0ad0 con 0x7fd6ec1045b0 2024-05-10T11:44:15.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.392+0000 7fd6f1bf7640 1 -- 172.21.15.100:0/290447203 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6ec1a0fd0 con 0x7fd6ec1045b0 2024-05-10T11:44:15.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.393+0000 7fd6f1bf7640 1 -- 172.21.15.100:0/290447203 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6ec109e60 con 0x7fd6ec1045b0 2024-05-10T11:44:15.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.394+0000 7fd6e97fa640 1 -- 172.21.15.100:0/290447203 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fd6e4003970 con 0x7fd6ec1045b0 2024-05-10T11:44:15.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.395+0000 7fd6e97fa640 1 --2- 172.21.15.100:0/290447203 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fd6d80b2870 0x7fd6d80b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:15.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.395+0000 7fd6eb7fe640 1 --2- 172.21.15.100:0/290447203 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fd6d80b2870 0x7fd6d80b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:15.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.395+0000 7fd6e97fa640 1 -- 172.21.15.100:0/290447203 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fd6e40e2640 con 0x7fd6ec1045b0 2024-05-10T11:44:15.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.396+0000 7fd6eb7fe640 1 --2- 172.21.15.100:0/290447203 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fd6d80b2870 0x7fd6d80b4d30 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc006010 tx=0x7fd6dc0095f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:44:15.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.399+0000 7fd6e97fa640 1 -- 172.21.15.100:0/290447203 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fd6e40aa080 con 0x7fd6ec1045b0 2024-05-10T11:44:15.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.634+0000 7fd6f1bf7640 1 -- 172.21.15.100:0/290447203 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fd6ec063650 con 0x7fd6ec1045b0 2024-05-10T11:44:15.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.642+0000 7fd6e97fa640 1 -- 172.21.15.100:0/290447203 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) ==== 81+0+815651 (secure 0 0 0) 0x7fd6e40a9980 con 0x7fd6ec1045b0 2024-05-10T11:44:15.654 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:44:15.654 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:44:15.654 INFO:teuthology.orchestra.run.smithi100.stdout: "epoch": 18, 2024-05-10T11:44:15.654 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.654 INFO:teuthology.orchestra.run.smithi100.stdout: "active_gid": 14152, 2024-05-10T11:44:15.654 INFO:teuthology.orchestra.run.smithi100.stdout: "active_name": "x", 2024-05-10T11:44:15.655 INFO:teuthology.orchestra.run.smithi100.stdout: "active_addrs": { 2024-05-10T11:44:15.655 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-05-10T11:44:15.655 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:15.655 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-05-10T11:44:15.655 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6800", 2024-05-10T11:44:15.655 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2507287356 2024-05-10T11:44:15.655 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.655 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:15.655 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-05-10T11:44:15.655 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6801", 2024-05-10T11:44:15.655 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2507287356 2024-05-10T11:44:15.655 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:15.655 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-05-10T11:44:15.655 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.656 INFO:teuthology.orchestra.run.smithi100.stdout: "active_addr": "172.21.15.100:6801/2507287356", 2024-05-10T11:44:15.656 INFO:teuthology.orchestra.run.smithi100.stdout: "active_change": "2024-05-10T11:07:00.092843+0000", 2024-05-10T11:44:15.656 INFO:teuthology.orchestra.run.smithi100.stdout: "active_mgr_features": 4540701547738038271, 2024-05-10T11:44:15.656 INFO:teuthology.orchestra.run.smithi100.stdout: "available": true, 2024-05-10T11:44:15.656 INFO:teuthology.orchestra.run.smithi100.stdout: "standbys": [ 2024-05-10T11:44:15.656 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:15.656 INFO:teuthology.orchestra.run.smithi100.stdout: "gid": 24109, 2024-05-10T11:44:15.656 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "z", 2024-05-10T11:44:15.656 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr_features": 4540701547738038271, 2024-05-10T11:44:15.656 INFO:teuthology.orchestra.run.smithi100.stdout: "available_modules": [ 2024-05-10T11:44:15.656 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:15.656 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "alerts", 2024-05-10T11:44:15.656 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:15.656 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:15.656 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:15.657 INFO:teuthology.orchestra.run.smithi100.stdout: "interval": { 2024-05-10T11:44:15.657 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "interval", 2024-05-10T11:44:15.657 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-05-10T11:44:15.657 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.657 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.657 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "60", 2024-05-10T11:44:15.657 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.657 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.657 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.657 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "How frequently to reexamine health status", 2024-05-10T11:44:15.657 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.657 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.657 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.657 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.658 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:15.658 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:15.658 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.658 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.658 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.658 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:15.658 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.658 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.658 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:15.658 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:15.658 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:15.658 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:15.658 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:15.658 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:15.659 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:15.659 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:15.659 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.659 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.659 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.659 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.659 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.659 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:15.659 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:15.659 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.659 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.659 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.659 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:15.660 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.660 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.660 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.660 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.660 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.660 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.660 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.660 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.660 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:15.660 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:15.661 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.661 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.661 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.661 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:15.661 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.661 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.661 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:15.661 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:15.661 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:15.661 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:15.661 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:15.661 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:15.661 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:15.661 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:15.662 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.662 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.662 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.662 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.662 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.662 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:15.662 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:15.662 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.662 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.662 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.662 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:15.662 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.662 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.662 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.663 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.663 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.663 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.663 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.663 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.663 INFO:teuthology.orchestra.run.smithi100.stdout: "smtp_destination": { 2024-05-10T11:44:15.663 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smtp_destination", 2024-05-10T11:44:15.663 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.663 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.663 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.663 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:15.664 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.664 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.664 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.664 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Email address to send alerts to", 2024-05-10T11:44:15.664 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.664 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.664 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.664 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.664 INFO:teuthology.orchestra.run.smithi100.stdout: "smtp_from_name": { 2024-05-10T11:44:15.664 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smtp_from_name", 2024-05-10T11:44:15.664 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.664 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.664 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.664 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "Ceph", 2024-05-10T11:44:15.664 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.665 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.665 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.665 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Email From: name", 2024-05-10T11:44:15.665 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.665 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.665 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.665 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.665 INFO:teuthology.orchestra.run.smithi100.stdout: "smtp_host": { 2024-05-10T11:44:15.665 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smtp_host", 2024-05-10T11:44:15.665 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.665 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.665 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.665 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:15.665 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.666 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.666 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.666 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "SMTP server", 2024-05-10T11:44:15.666 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.666 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.666 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.666 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.666 INFO:teuthology.orchestra.run.smithi100.stdout: "smtp_password": { 2024-05-10T11:44:15.666 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smtp_password", 2024-05-10T11:44:15.666 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.666 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.666 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.666 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:15.666 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.667 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.667 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.667 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Password to authenticate with", 2024-05-10T11:44:15.667 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.667 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.667 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.667 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.667 INFO:teuthology.orchestra.run.smithi100.stdout: "smtp_port": { 2024-05-10T11:44:15.667 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smtp_port", 2024-05-10T11:44:15.667 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:15.667 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.667 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.667 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "465", 2024-05-10T11:44:15.667 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.667 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.668 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.668 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "SMTP port", 2024-05-10T11:44:15.668 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.668 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.668 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.668 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.668 INFO:teuthology.orchestra.run.smithi100.stdout: "smtp_sender": { 2024-05-10T11:44:15.668 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smtp_sender", 2024-05-10T11:44:15.668 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.668 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.668 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.668 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:15.668 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.668 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.668 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.669 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "SMTP envelope sender", 2024-05-10T11:44:15.669 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.669 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.669 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.669 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.669 INFO:teuthology.orchestra.run.smithi100.stdout: "smtp_ssl": { 2024-05-10T11:44:15.669 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smtp_ssl", 2024-05-10T11:44:15.669 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.669 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.670 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.670 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:15.670 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.670 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.670 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.670 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-10T11:44:15.670 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.670 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.670 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.670 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.670 INFO:teuthology.orchestra.run.smithi100.stdout: "smtp_user": { 2024-05-10T11:44:15.671 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smtp_user", 2024-05-10T11:44:15.671 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.671 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.671 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.671 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:15.671 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.671 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.671 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.671 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "User to authenticate as", 2024-05-10T11:44:15.671 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.671 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.671 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.671 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.671 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:15.672 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:15.672 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:15.672 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:15.672 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.672 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:15.672 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.672 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.672 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.672 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.672 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.672 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.672 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.672 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:15.672 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:15.672 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.673 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:15.673 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "balancer", 2024-05-10T11:44:15.673 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:15.673 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:15.673 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:15.673 INFO:teuthology.orchestra.run.smithi100.stdout: "active": { 2024-05-10T11:44:15.673 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "active", 2024-05-10T11:44:15.673 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.673 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.673 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.673 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:15.673 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.673 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.673 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.674 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "automatically balance PGs across cluster", 2024-05-10T11:44:15.674 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.674 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.674 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.674 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.674 INFO:teuthology.orchestra.run.smithi100.stdout: "begin_time": { 2024-05-10T11:44:15.674 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "begin_time", 2024-05-10T11:44:15.674 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.674 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.674 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.674 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0000", 2024-05-10T11:44:15.674 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.674 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.675 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.675 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "beginning time of day to automatically balance", 2024-05-10T11:44:15.675 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-10T11:44:15.675 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.675 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.675 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.675 INFO:teuthology.orchestra.run.smithi100.stdout: "begin_weekday": { 2024-05-10T11:44:15.675 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "begin_weekday", 2024-05-10T11:44:15.675 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "uint", 2024-05-10T11:44:15.675 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.675 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.675 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:15.675 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "0", 2024-05-10T11:44:15.675 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "6", 2024-05-10T11:44:15.675 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.676 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-10T11:44:15.676 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-10T11:44:15.676 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.676 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.676 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.676 INFO:teuthology.orchestra.run.smithi100.stdout: "crush_compat_max_iterations": { 2024-05-10T11:44:15.676 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "crush_compat_max_iterations", 2024-05-10T11:44:15.676 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "uint", 2024-05-10T11:44:15.676 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.676 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.676 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "25", 2024-05-10T11:44:15.676 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "1", 2024-05-10T11:44:15.676 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "250", 2024-05-10T11:44:15.676 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.676 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-10T11:44:15.677 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.677 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.677 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.677 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.677 INFO:teuthology.orchestra.run.smithi100.stdout: "crush_compat_metrics": { 2024-05-10T11:44:15.677 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "crush_compat_metrics", 2024-05-10T11:44:15.677 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.677 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.677 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.677 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "pgs,objects,bytes", 2024-05-10T11:44:15.677 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.678 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.678 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.678 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-10T11:44:15.678 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-10T11:44:15.678 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.678 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.678 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.678 INFO:teuthology.orchestra.run.smithi100.stdout: "crush_compat_step": { 2024-05-10T11:44:15.678 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "crush_compat_step", 2024-05-10T11:44:15.678 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "float", 2024-05-10T11:44:15.678 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.678 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.678 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0.5", 2024-05-10T11:44:15.678 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "0.001", 2024-05-10T11:44:15.679 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "0.999", 2024-05-10T11:44:15.679 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.679 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "aggressiveness of optimization", 2024-05-10T11:44:15.679 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-10T11:44:15.679 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.679 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.679 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.679 INFO:teuthology.orchestra.run.smithi100.stdout: "end_time": { 2024-05-10T11:44:15.679 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "end_time", 2024-05-10T11:44:15.679 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.679 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.679 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.679 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "2359", 2024-05-10T11:44:15.679 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.679 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.680 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.680 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "ending time of day to automatically balance", 2024-05-10T11:44:15.680 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-10T11:44:15.680 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.680 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.680 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.680 INFO:teuthology.orchestra.run.smithi100.stdout: "end_weekday": { 2024-05-10T11:44:15.680 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "end_weekday", 2024-05-10T11:44:15.680 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "uint", 2024-05-10T11:44:15.680 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.681 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.681 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:15.681 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "0", 2024-05-10T11:44:15.681 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "6", 2024-05-10T11:44:15.681 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.681 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-10T11:44:15.681 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-10T11:44:15.681 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.681 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.681 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.681 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:15.681 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:15.681 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.681 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.681 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.682 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:15.682 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.682 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.682 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:15.682 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:15.682 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:15.682 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:15.682 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:15.682 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:15.682 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:15.682 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:15.682 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.683 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.683 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.683 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.683 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.683 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:15.683 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:15.683 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.683 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.683 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.683 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:15.683 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.683 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.683 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.683 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.683 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.684 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.684 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.684 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.684 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:15.684 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:15.684 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.684 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.684 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.684 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:15.684 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.684 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.685 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:15.685 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:15.685 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:15.685 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:15.685 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:15.685 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:15.685 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:15.685 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:15.685 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.685 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.685 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.685 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.685 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.686 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:15.686 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:15.686 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.686 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.686 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.686 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:15.686 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.686 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.686 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.686 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.686 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.686 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.686 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.686 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.687 INFO:teuthology.orchestra.run.smithi100.stdout: "min_score": { 2024-05-10T11:44:15.687 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "min_score", 2024-05-10T11:44:15.687 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "float", 2024-05-10T11:44:15.687 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.687 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.687 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:15.687 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.687 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.687 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.687 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-10T11:44:15.687 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.687 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.687 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.687 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.687 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": { 2024-05-10T11:44:15.688 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "mode", 2024-05-10T11:44:15.688 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.688 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.688 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.688 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "upmap", 2024-05-10T11:44:15.688 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.688 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.688 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:15.688 INFO:teuthology.orchestra.run.smithi100.stdout: "crush-compat", 2024-05-10T11:44:15.688 INFO:teuthology.orchestra.run.smithi100.stdout: "none", 2024-05-10T11:44:15.688 INFO:teuthology.orchestra.run.smithi100.stdout: "read", 2024-05-10T11:44:15.688 INFO:teuthology.orchestra.run.smithi100.stdout: "upmap", 2024-05-10T11:44:15.688 INFO:teuthology.orchestra.run.smithi100.stdout: "upmap-read" 2024-05-10T11:44:15.688 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:15.689 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Balancer mode", 2024-05-10T11:44:15.689 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.689 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.689 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.689 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.689 INFO:teuthology.orchestra.run.smithi100.stdout: "pool_ids": { 2024-05-10T11:44:15.689 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "pool_ids", 2024-05-10T11:44:15.689 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.689 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.689 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.689 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:15.689 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.689 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.689 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.689 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-10T11:44:15.690 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.690 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.690 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.690 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.690 INFO:teuthology.orchestra.run.smithi100.stdout: "sleep_interval": { 2024-05-10T11:44:15.690 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sleep_interval", 2024-05-10T11:44:15.690 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-05-10T11:44:15.690 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.690 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.690 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "60", 2024-05-10T11:44:15.690 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.690 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.690 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.690 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-10T11:44:15.691 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.691 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.691 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.691 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.691 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:15.691 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:15.691 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:15.691 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:15.691 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.691 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:15.691 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.691 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.691 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.691 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.691 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.692 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.692 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.692 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.692 INFO:teuthology.orchestra.run.smithi100.stdout: "upmap_max_deviation": { 2024-05-10T11:44:15.692 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "upmap_max_deviation", 2024-05-10T11:44:15.692 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:15.692 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.692 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.692 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "5", 2024-05-10T11:44:15.692 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "1", 2024-05-10T11:44:15.692 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.692 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.692 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-10T11:44:15.692 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-10T11:44:15.692 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.693 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.693 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.693 INFO:teuthology.orchestra.run.smithi100.stdout: "upmap_max_optimizations": { 2024-05-10T11:44:15.693 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "upmap_max_optimizations", 2024-05-10T11:44:15.693 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "uint", 2024-05-10T11:44:15.693 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.693 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.693 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10", 2024-05-10T11:44:15.693 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.693 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.693 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.693 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-10T11:44:15.693 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.693 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.693 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.694 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:15.694 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:15.694 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.694 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:15.694 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "cephadm", 2024-05-10T11:44:15.694 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:15.694 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:15.694 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:15.694 INFO:teuthology.orchestra.run.smithi100.stdout: "agent_down_multiplier": { 2024-05-10T11:44:15.694 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "agent_down_multiplier", 2024-05-10T11:44:15.694 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "float", 2024-05-10T11:44:15.694 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.694 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.694 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "3.0", 2024-05-10T11:44:15.695 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.695 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.695 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.695 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-10T11:44:15.695 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.695 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.695 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.695 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.695 INFO:teuthology.orchestra.run.smithi100.stdout: "agent_refresh_rate": { 2024-05-10T11:44:15.695 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "agent_refresh_rate", 2024-05-10T11:44:15.695 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-05-10T11:44:15.695 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.695 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.696 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "20", 2024-05-10T11:44:15.696 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.696 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.696 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.696 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-10T11:44:15.696 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.696 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.696 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.696 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.696 INFO:teuthology.orchestra.run.smithi100.stdout: "agent_starting_port": { 2024-05-10T11:44:15.696 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "agent_starting_port", 2024-05-10T11:44:15.696 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:15.696 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.696 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.696 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "4721", 2024-05-10T11:44:15.697 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.697 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.697 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.697 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-10T11:44:15.697 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.697 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.697 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.697 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.697 INFO:teuthology.orchestra.run.smithi100.stdout: "allow_ptrace": { 2024-05-10T11:44:15.697 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "allow_ptrace", 2024-05-10T11:44:15.697 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.697 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.697 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.698 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:15.698 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.698 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.698 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.698 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-10T11:44:15.698 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-10T11:44:15.698 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.698 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.698 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.698 INFO:teuthology.orchestra.run.smithi100.stdout: "autotune_interval": { 2024-05-10T11:44:15.698 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "autotune_interval", 2024-05-10T11:44:15.699 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-05-10T11:44:15.699 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.699 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.699 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "600", 2024-05-10T11:44:15.699 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.699 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.699 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.699 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-10T11:44:15.699 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.699 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.699 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.699 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.699 INFO:teuthology.orchestra.run.smithi100.stdout: "autotune_memory_target_ratio": { 2024-05-10T11:44:15.699 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "autotune_memory_target_ratio", 2024-05-10T11:44:15.700 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "float", 2024-05-10T11:44:15.700 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.700 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.700 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0.7", 2024-05-10T11:44:15.700 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.700 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.700 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.700 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-10T11:44:15.700 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.700 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.701 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.701 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.701 INFO:teuthology.orchestra.run.smithi100.stdout: "cephadm_log_destination": { 2024-05-10T11:44:15.701 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "cephadm_log_destination", 2024-05-10T11:44:15.701 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.701 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.701 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.701 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:15.701 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.701 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.701 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:15.701 INFO:teuthology.orchestra.run.smithi100.stdout: "file", 2024-05-10T11:44:15.701 INFO:teuthology.orchestra.run.smithi100.stdout: "file,syslog", 2024-05-10T11:44:15.701 INFO:teuthology.orchestra.run.smithi100.stdout: "syslog" 2024-05-10T11:44:15.702 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:15.702 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-10T11:44:15.702 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.702 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.702 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.702 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.702 INFO:teuthology.orchestra.run.smithi100.stdout: "cgroups_split": { 2024-05-10T11:44:15.702 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "cgroups_split", 2024-05-10T11:44:15.702 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.702 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.702 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.702 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:15.702 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.702 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.703 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.703 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-10T11:44:15.703 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.703 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.703 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.703 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.703 INFO:teuthology.orchestra.run.smithi100.stdout: "config_checks_enabled": { 2024-05-10T11:44:15.703 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "config_checks_enabled", 2024-05-10T11:44:15.703 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.703 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.703 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.703 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:15.703 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.703 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.704 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.704 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-10T11:44:15.704 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.704 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.704 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.704 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.704 INFO:teuthology.orchestra.run.smithi100.stdout: "config_dashboard": { 2024-05-10T11:44:15.705 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "config_dashboard", 2024-05-10T11:44:15.705 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.705 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.705 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.705 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:15.705 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.705 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.705 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.705 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-10T11:44:15.705 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.705 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.706 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.706 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.706 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_alertmanager": { 2024-05-10T11:44:15.706 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_alertmanager", 2024-05-10T11:44:15.706 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.706 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.706 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.706 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-10T11:44:15.706 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.706 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.706 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.706 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Prometheus container image", 2024-05-10T11:44:15.706 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.706 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.707 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.707 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.707 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_base": { 2024-05-10T11:44:15.707 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_base", 2024-05-10T11:44:15.707 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.707 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.707 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.707 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-10T11:44:15.707 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.707 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.707 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.707 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Container image name, without the tag", 2024-05-10T11:44:15.707 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.707 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.707 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.708 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.708 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_elasticsearch": { 2024-05-10T11:44:15.708 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_elasticsearch", 2024-05-10T11:44:15.708 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.708 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.708 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.708 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-10T11:44:15.708 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.708 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.708 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.708 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "elasticsearch container image", 2024-05-10T11:44:15.708 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.709 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.709 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.709 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.709 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_grafana": { 2024-05-10T11:44:15.709 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_grafana", 2024-05-10T11:44:15.709 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.709 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.709 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.709 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-10T11:44:15.709 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.709 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.709 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.709 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Prometheus container image", 2024-05-10T11:44:15.709 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.710 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.710 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.710 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.710 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_haproxy": { 2024-05-10T11:44:15.710 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_haproxy", 2024-05-10T11:44:15.710 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.710 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.710 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.710 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-10T11:44:15.710 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.710 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.710 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.710 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "HAproxy container image", 2024-05-10T11:44:15.711 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.711 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.711 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.711 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.711 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_jaeger_agent": { 2024-05-10T11:44:15.711 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_jaeger_agent", 2024-05-10T11:44:15.711 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.711 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.711 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.711 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-10T11:44:15.711 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.711 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.711 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.711 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Jaeger agent container image", 2024-05-10T11:44:15.712 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.712 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.712 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.712 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.712 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_jaeger_collector": { 2024-05-10T11:44:15.712 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_jaeger_collector", 2024-05-10T11:44:15.712 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.712 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.712 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.712 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-10T11:44:15.712 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.712 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.712 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.712 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Jaeger collector container image", 2024-05-10T11:44:15.713 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.713 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.713 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.713 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.713 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_jaeger_query": { 2024-05-10T11:44:15.713 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_jaeger_query", 2024-05-10T11:44:15.713 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.713 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.713 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.713 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-10T11:44:15.713 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.713 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.713 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.713 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Jaeger query container image", 2024-05-10T11:44:15.714 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.714 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.714 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.714 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.714 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_keepalived": { 2024-05-10T11:44:15.714 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_keepalived", 2024-05-10T11:44:15.714 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.714 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.714 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.714 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-10T11:44:15.714 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.714 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.714 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.714 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Keepalived container image", 2024-05-10T11:44:15.715 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.715 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.715 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.715 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.715 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_loki": { 2024-05-10T11:44:15.715 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_loki", 2024-05-10T11:44:15.715 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.715 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.715 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.715 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-10T11:44:15.715 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.715 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.715 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.715 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Loki container image", 2024-05-10T11:44:15.715 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.716 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.716 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.716 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.716 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_node_exporter": { 2024-05-10T11:44:15.716 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_node_exporter", 2024-05-10T11:44:15.716 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.716 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.716 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.716 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-10T11:44:15.716 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.716 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.716 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.716 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Prometheus container image", 2024-05-10T11:44:15.716 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.716 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.717 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.717 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.717 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_nvmeof": { 2024-05-10T11:44:15.717 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_nvmeof", 2024-05-10T11:44:15.717 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.717 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.717 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.717 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-10T11:44:15.717 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.717 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.717 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.717 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Nvme-of container image", 2024-05-10T11:44:15.717 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.717 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.718 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.718 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.718 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_prometheus": { 2024-05-10T11:44:15.718 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_prometheus", 2024-05-10T11:44:15.718 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.718 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.718 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.718 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-10T11:44:15.718 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.718 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.718 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.718 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Prometheus container image", 2024-05-10T11:44:15.718 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.718 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.719 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.719 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.719 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_promtail": { 2024-05-10T11:44:15.719 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_promtail", 2024-05-10T11:44:15.719 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.719 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.719 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.719 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-10T11:44:15.719 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.719 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.719 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.719 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Promtail container image", 2024-05-10T11:44:15.719 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.719 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.719 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.720 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.720 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_samba": { 2024-05-10T11:44:15.720 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_samba", 2024-05-10T11:44:15.720 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.720 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.720 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.720 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-10T11:44:15.720 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.720 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.720 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.720 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Samba/SMB container image", 2024-05-10T11:44:15.721 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.721 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.721 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.721 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.721 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_snmp_gateway": { 2024-05-10T11:44:15.721 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_snmp_gateway", 2024-05-10T11:44:15.721 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.721 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.721 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.721 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-10T11:44:15.721 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.721 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.721 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.722 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "SNMP Gateway container image", 2024-05-10T11:44:15.722 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.722 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.722 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.722 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.722 INFO:teuthology.orchestra.run.smithi100.stdout: "container_init": { 2024-05-10T11:44:15.722 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_init", 2024-05-10T11:44:15.722 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.722 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.722 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.722 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:15.722 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.722 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.722 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.723 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Run podman/docker with `--init`", 2024-05-10T11:44:15.723 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.723 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.723 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.723 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.723 INFO:teuthology.orchestra.run.smithi100.stdout: "daemon_cache_timeout": { 2024-05-10T11:44:15.723 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "daemon_cache_timeout", 2024-05-10T11:44:15.723 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-05-10T11:44:15.723 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.723 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.723 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "600", 2024-05-10T11:44:15.723 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.723 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.723 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.724 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-10T11:44:15.724 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.724 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.724 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.724 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.724 INFO:teuthology.orchestra.run.smithi100.stdout: "default_cephadm_command_timeout": { 2024-05-10T11:44:15.724 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "default_cephadm_command_timeout", 2024-05-10T11:44:15.724 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:15.724 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.724 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.724 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "900", 2024-05-10T11:44:15.724 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.724 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.724 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.725 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-10T11:44:15.725 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.725 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.725 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.725 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.725 INFO:teuthology.orchestra.run.smithi100.stdout: "default_registry": { 2024-05-10T11:44:15.725 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "default_registry", 2024-05-10T11:44:15.725 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.725 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.725 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.725 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "docker.io", 2024-05-10T11:44:15.725 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.725 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.725 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.726 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-10T11:44:15.726 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.726 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.726 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.726 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.726 INFO:teuthology.orchestra.run.smithi100.stdout: "device_cache_timeout": { 2024-05-10T11:44:15.726 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "device_cache_timeout", 2024-05-10T11:44:15.726 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-05-10T11:44:15.726 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.726 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.726 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "1800", 2024-05-10T11:44:15.726 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.727 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.727 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.727 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "seconds to cache device inventory", 2024-05-10T11:44:15.727 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.727 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.727 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.727 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.727 INFO:teuthology.orchestra.run.smithi100.stdout: "device_enhanced_scan": { 2024-05-10T11:44:15.727 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "device_enhanced_scan", 2024-05-10T11:44:15.727 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.727 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.728 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.728 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:15.728 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.728 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.728 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.728 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-10T11:44:15.728 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.729 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.729 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.729 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.729 INFO:teuthology.orchestra.run.smithi100.stdout: "facts_cache_timeout": { 2024-05-10T11:44:15.729 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "facts_cache_timeout", 2024-05-10T11:44:15.729 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-05-10T11:44:15.729 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.729 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.729 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "60", 2024-05-10T11:44:15.729 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.729 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.729 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.729 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "seconds to cache host facts data", 2024-05-10T11:44:15.729 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.729 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.730 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.730 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.730 INFO:teuthology.orchestra.run.smithi100.stdout: "grafana_dashboards_path": { 2024-05-10T11:44:15.730 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "grafana_dashboards_path", 2024-05-10T11:44:15.730 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.730 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.730 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.730 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-10T11:44:15.730 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.730 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.730 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.730 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-10T11:44:15.730 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.731 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.731 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.731 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.731 INFO:teuthology.orchestra.run.smithi100.stdout: "host_check_interval": { 2024-05-10T11:44:15.731 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "host_check_interval", 2024-05-10T11:44:15.731 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-05-10T11:44:15.731 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.731 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.731 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "600", 2024-05-10T11:44:15.731 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.731 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.731 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.731 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "how frequently to perform a host check", 2024-05-10T11:44:15.731 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.731 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.732 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.732 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.732 INFO:teuthology.orchestra.run.smithi100.stdout: "hw_monitoring": { 2024-05-10T11:44:15.732 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "hw_monitoring", 2024-05-10T11:44:15.732 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.732 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.732 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.732 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:15.732 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.732 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.732 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.732 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-10T11:44:15.732 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.732 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.732 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.733 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.733 INFO:teuthology.orchestra.run.smithi100.stdout: "inventory_list_all": { 2024-05-10T11:44:15.733 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "inventory_list_all", 2024-05-10T11:44:15.733 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.733 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.733 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.733 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:15.733 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.733 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.733 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.733 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-10T11:44:15.733 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.733 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.734 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.734 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.734 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:15.734 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:15.734 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.734 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.734 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.734 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:15.734 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.734 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.734 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:15.734 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:15.734 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:15.734 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:15.735 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:15.735 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:15.735 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:15.735 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:15.735 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.735 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.735 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.735 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.735 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.735 INFO:teuthology.orchestra.run.smithi100.stdout: "log_refresh_metadata": { 2024-05-10T11:44:15.735 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_refresh_metadata", 2024-05-10T11:44:15.735 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.735 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.735 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.735 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:15.735 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.736 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.736 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.736 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-10T11:44:15.736 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.736 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.736 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.736 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.736 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:15.736 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:15.736 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.736 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.736 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.736 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:15.736 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.737 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.737 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.737 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-10T11:44:15.737 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.737 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.737 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.737 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.737 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:15.737 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:15.737 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.737 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.737 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.737 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:15.737 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.738 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.738 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:15.738 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:15.738 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:15.738 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:15.738 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:15.738 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:15.738 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:15.738 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:15.738 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.738 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.738 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.738 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.738 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.738 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:15.739 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:15.739 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.739 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.739 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.739 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:15.739 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.739 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.739 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.739 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.739 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.739 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.739 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.739 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.739 INFO:teuthology.orchestra.run.smithi100.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-10T11:44:15.740 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-10T11:44:15.740 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.740 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.740 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.740 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:15.740 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.740 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.740 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.740 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-10T11:44:15.740 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.740 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.740 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.740 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.740 INFO:teuthology.orchestra.run.smithi100.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-10T11:44:15.740 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-10T11:44:15.740 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.741 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.741 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.741 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "*", 2024-05-10T11:44:15.741 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.741 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.741 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.741 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-10T11:44:15.741 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.741 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.741 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.741 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.741 INFO:teuthology.orchestra.run.smithi100.stdout: "max_count_per_host": { 2024-05-10T11:44:15.741 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "max_count_per_host", 2024-05-10T11:44:15.741 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:15.741 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.742 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.742 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10", 2024-05-10T11:44:15.742 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.742 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.742 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.742 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "max number of daemons per service per host", 2024-05-10T11:44:15.742 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.742 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.742 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.742 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.742 INFO:teuthology.orchestra.run.smithi100.stdout: "max_osd_draining_count": { 2024-05-10T11:44:15.742 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "max_osd_draining_count", 2024-05-10T11:44:15.742 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:15.742 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.742 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.743 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10", 2024-05-10T11:44:15.743 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.743 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.743 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.743 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-10T11:44:15.743 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.743 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.743 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.743 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.743 INFO:teuthology.orchestra.run.smithi100.stdout: "migration_current": { 2024-05-10T11:44:15.743 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "migration_current", 2024-05-10T11:44:15.743 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:15.743 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.743 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.743 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:15.744 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.744 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.744 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.744 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "internal - do not modify", 2024-05-10T11:44:15.744 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.744 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.744 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.744 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.744 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": { 2024-05-10T11:44:15.744 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "mode", 2024-05-10T11:44:15.744 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.744 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.744 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.744 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "root", 2024-05-10T11:44:15.745 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.745 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.745 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:15.745 INFO:teuthology.orchestra.run.smithi100.stdout: "cephadm-package", 2024-05-10T11:44:15.745 INFO:teuthology.orchestra.run.smithi100.stdout: "root" 2024-05-10T11:44:15.745 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:15.745 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "mode for remote execution of cephadm", 2024-05-10T11:44:15.745 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.745 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.745 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.745 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.745 INFO:teuthology.orchestra.run.smithi100.stdout: "oob_default_addr": { 2024-05-10T11:44:15.745 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "oob_default_addr", 2024-05-10T11:44:15.745 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.745 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.745 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.746 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "169.254.1.1", 2024-05-10T11:44:15.746 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.746 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.746 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.746 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-10T11:44:15.746 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.746 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.746 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.746 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.746 INFO:teuthology.orchestra.run.smithi100.stdout: "prometheus_alerts_path": { 2024-05-10T11:44:15.746 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "prometheus_alerts_path", 2024-05-10T11:44:15.746 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.746 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.747 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.747 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-10T11:44:15.747 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.747 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.747 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.747 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-10T11:44:15.747 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.747 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.747 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.747 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.747 INFO:teuthology.orchestra.run.smithi100.stdout: "registry_insecure": { 2024-05-10T11:44:15.747 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "registry_insecure", 2024-05-10T11:44:15.747 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.747 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.747 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.748 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:15.748 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.748 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.748 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.748 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-10T11:44:15.748 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.748 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.748 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.748 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.748 INFO:teuthology.orchestra.run.smithi100.stdout: "registry_password": { 2024-05-10T11:44:15.748 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "registry_password", 2024-05-10T11:44:15.748 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.748 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.748 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.749 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:15.749 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.749 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.749 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.749 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-10T11:44:15.749 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.749 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.749 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.749 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.749 INFO:teuthology.orchestra.run.smithi100.stdout: "registry_url": { 2024-05-10T11:44:15.749 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "registry_url", 2024-05-10T11:44:15.749 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.749 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.749 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.749 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:15.750 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.750 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.750 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.750 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-10T11:44:15.750 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.750 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.750 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.750 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.750 INFO:teuthology.orchestra.run.smithi100.stdout: "registry_username": { 2024-05-10T11:44:15.750 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "registry_username", 2024-05-10T11:44:15.750 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.750 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.750 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.750 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:15.751 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.751 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.751 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.751 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-10T11:44:15.751 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.751 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.751 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.751 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.751 INFO:teuthology.orchestra.run.smithi100.stdout: "secure_monitoring_stack": { 2024-05-10T11:44:15.751 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "secure_monitoring_stack", 2024-05-10T11:44:15.751 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.751 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.751 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.751 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:15.751 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.751 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.752 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.752 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-10T11:44:15.752 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.752 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.752 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.752 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.752 INFO:teuthology.orchestra.run.smithi100.stdout: "service_discovery_port": { 2024-05-10T11:44:15.752 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "service_discovery_port", 2024-05-10T11:44:15.752 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:15.752 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.752 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.752 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "8765", 2024-05-10T11:44:15.752 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.752 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.752 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.753 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "cephadm service discovery port", 2024-05-10T11:44:15.753 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.753 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.753 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.753 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.753 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:15.753 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:15.753 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:15.753 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:15.753 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.753 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:15.753 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.753 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.753 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.753 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.754 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.754 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.754 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.754 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.754 INFO:teuthology.orchestra.run.smithi100.stdout: "ssh_config_file": { 2024-05-10T11:44:15.754 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ssh_config_file", 2024-05-10T11:44:15.754 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.754 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.754 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.754 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:15.754 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.754 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.754 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.754 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-10T11:44:15.755 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.755 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.755 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.755 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.755 INFO:teuthology.orchestra.run.smithi100.stdout: "use_agent": { 2024-05-10T11:44:15.755 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "use_agent", 2024-05-10T11:44:15.755 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.755 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.755 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.755 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:15.755 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.755 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.755 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.755 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-10T11:44:15.756 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.756 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.756 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.756 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.756 INFO:teuthology.orchestra.run.smithi100.stdout: "use_repo_digest": { 2024-05-10T11:44:15.756 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "use_repo_digest", 2024-05-10T11:44:15.756 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.756 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.756 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.756 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:15.756 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.756 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.756 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.756 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-10T11:44:15.756 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.757 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.757 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.757 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.757 INFO:teuthology.orchestra.run.smithi100.stdout: "warn_on_failed_host_check": { 2024-05-10T11:44:15.757 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "warn_on_failed_host_check", 2024-05-10T11:44:15.757 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.757 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.757 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.757 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:15.757 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.757 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.757 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.757 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "raise a health warning if the host check fails", 2024-05-10T11:44:15.757 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.758 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.758 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.758 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.758 INFO:teuthology.orchestra.run.smithi100.stdout: "warn_on_stray_daemons": { 2024-05-10T11:44:15.758 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "warn_on_stray_daemons", 2024-05-10T11:44:15.758 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.758 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.758 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.758 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:15.758 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.758 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.758 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.758 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-10T11:44:15.759 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.759 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.759 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.759 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.759 INFO:teuthology.orchestra.run.smithi100.stdout: "warn_on_stray_hosts": { 2024-05-10T11:44:15.759 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "warn_on_stray_hosts", 2024-05-10T11:44:15.759 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.759 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.759 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.759 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:15.759 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.759 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.759 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.759 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-10T11:44:15.760 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.760 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.760 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.760 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:15.760 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:15.760 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.760 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:15.760 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "crash", 2024-05-10T11:44:15.760 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:15.760 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:15.760 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:15.760 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:15.760 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:15.760 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.761 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.761 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.761 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:15.761 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.761 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.761 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:15.761 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:15.761 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:15.761 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:15.761 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:15.761 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:15.761 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:15.761 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:15.761 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.762 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.762 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.762 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.762 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.762 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:15.762 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:15.762 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.762 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.762 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.762 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:15.762 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.762 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.762 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.762 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.762 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.763 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.763 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.763 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.763 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:15.763 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:15.763 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.763 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.763 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.763 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:15.763 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.763 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.763 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:15.763 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:15.763 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:15.763 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:15.764 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:15.764 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:15.764 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:15.764 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:15.764 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.764 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.764 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.764 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.764 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.764 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:15.764 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:15.764 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.764 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.764 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.765 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:15.765 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.765 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.765 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.765 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.765 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.765 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.765 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.765 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.765 INFO:teuthology.orchestra.run.smithi100.stdout: "retain_interval": { 2024-05-10T11:44:15.765 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "retain_interval", 2024-05-10T11:44:15.765 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-05-10T11:44:15.765 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.765 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.766 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "31536000", 2024-05-10T11:44:15.766 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.766 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.766 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.766 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-10T11:44:15.766 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.766 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.766 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.766 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.766 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:15.766 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:15.766 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:15.766 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:15.766 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.767 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:15.767 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.767 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.767 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.767 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.767 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.767 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.767 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.767 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.767 INFO:teuthology.orchestra.run.smithi100.stdout: "warn_recent_interval": { 2024-05-10T11:44:15.767 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "warn_recent_interval", 2024-05-10T11:44:15.767 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-05-10T11:44:15.767 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.767 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.767 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "1209600", 2024-05-10T11:44:15.768 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.768 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.768 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.768 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-10T11:44:15.768 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.768 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.768 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.768 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:15.768 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:15.768 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.768 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:15.768 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "dashboard", 2024-05-10T11:44:15.768 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:15.768 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:15.768 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:15.768 INFO:teuthology.orchestra.run.smithi100.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-10T11:44:15.769 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-10T11:44:15.769 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:15.769 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.769 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.769 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10", 2024-05-10T11:44:15.769 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.769 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.769 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.769 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.769 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.769 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.769 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.769 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.769 INFO:teuthology.orchestra.run.smithi100.stdout: "ALERTMANAGER_API_HOST": { 2024-05-10T11:44:15.769 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-10T11:44:15.769 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.769 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.770 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.770 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:15.770 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.770 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.770 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.770 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.770 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.770 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.770 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.770 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.770 INFO:teuthology.orchestra.run.smithi100.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-10T11:44:15.770 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-10T11:44:15.770 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.771 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.771 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.771 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:15.771 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.771 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.771 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.771 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.771 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.771 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.771 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.771 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.771 INFO:teuthology.orchestra.run.smithi100.stdout: "AUDIT_API_ENABLED": { 2024-05-10T11:44:15.771 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "AUDIT_API_ENABLED", 2024-05-10T11:44:15.771 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.771 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.771 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.772 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:15.772 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.772 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.772 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.772 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.772 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.772 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.772 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.772 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.772 INFO:teuthology.orchestra.run.smithi100.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-10T11:44:15.772 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-10T11:44:15.772 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.772 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.772 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.772 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:15.773 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.773 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.773 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.773 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.773 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.773 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.773 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.773 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.773 INFO:teuthology.orchestra.run.smithi100.stdout: "ENABLE_BROWSABLE_API": { 2024-05-10T11:44:15.773 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-10T11:44:15.773 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.773 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.773 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.773 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:15.774 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.774 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.774 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.774 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.774 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.774 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.774 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.774 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.774 INFO:teuthology.orchestra.run.smithi100.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-10T11:44:15.774 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-10T11:44:15.774 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.774 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.774 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.774 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:15.775 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.775 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.775 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.775 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.775 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.775 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.775 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.775 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.775 INFO:teuthology.orchestra.run.smithi100.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-10T11:44:15.775 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-10T11:44:15.775 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.775 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.775 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.775 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:15.775 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.775 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.776 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.776 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.776 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.776 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.776 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.776 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.776 INFO:teuthology.orchestra.run.smithi100.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-10T11:44:15.776 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-10T11:44:15.776 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.776 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.776 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.776 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:15.776 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.776 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.776 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.777 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.777 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.777 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.777 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.777 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.777 INFO:teuthology.orchestra.run.smithi100.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-10T11:44:15.777 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-10T11:44:15.777 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.777 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.777 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.777 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:15.777 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.777 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.777 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.777 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.778 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.778 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.778 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.778 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.778 INFO:teuthology.orchestra.run.smithi100.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-10T11:44:15.778 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-10T11:44:15.778 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.778 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.778 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.778 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:15.778 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.778 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.778 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.778 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.778 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.779 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.779 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.779 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.779 INFO:teuthology.orchestra.run.smithi100.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-10T11:44:15.779 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-10T11:44:15.779 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.779 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.779 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.779 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:15.779 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.779 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.779 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.779 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.779 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.779 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.780 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.780 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.780 INFO:teuthology.orchestra.run.smithi100.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-10T11:44:15.780 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-10T11:44:15.780 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.780 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.780 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.780 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:15.780 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.780 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.780 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.780 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.780 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.780 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.780 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.781 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.781 INFO:teuthology.orchestra.run.smithi100.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-10T11:44:15.781 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-10T11:44:15.781 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.781 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.781 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.781 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:15.781 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.781 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.781 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.781 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.781 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.781 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.781 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.781 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.782 INFO:teuthology.orchestra.run.smithi100.stdout: "GRAFANA_API_PASSWORD": { 2024-05-10T11:44:15.782 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-10T11:44:15.782 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.782 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.782 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.782 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "admin", 2024-05-10T11:44:15.782 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.782 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.782 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.782 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.782 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.782 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.782 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.782 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.782 INFO:teuthology.orchestra.run.smithi100.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-10T11:44:15.783 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-10T11:44:15.783 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.783 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.783 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.783 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:15.783 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.783 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.783 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.783 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.783 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.783 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.783 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.784 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.784 INFO:teuthology.orchestra.run.smithi100.stdout: "GRAFANA_API_URL": { 2024-05-10T11:44:15.784 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "GRAFANA_API_URL", 2024-05-10T11:44:15.784 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.784 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.784 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.784 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:15.784 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.784 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.784 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.784 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.784 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.784 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.784 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.784 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.785 INFO:teuthology.orchestra.run.smithi100.stdout: "GRAFANA_API_USERNAME": { 2024-05-10T11:44:15.785 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-10T11:44:15.785 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.785 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.785 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.785 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "admin", 2024-05-10T11:44:15.785 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.785 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.785 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.785 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.785 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.785 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.785 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.785 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.785 INFO:teuthology.orchestra.run.smithi100.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-10T11:44:15.785 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-10T11:44:15.786 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.786 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.786 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.786 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:15.786 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.786 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.786 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.786 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.786 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.786 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.786 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.786 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.786 INFO:teuthology.orchestra.run.smithi100.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-10T11:44:15.786 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-10T11:44:15.787 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.787 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.787 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.787 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:15.787 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.787 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.787 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.787 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.787 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.787 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.787 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.787 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.787 INFO:teuthology.orchestra.run.smithi100.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-10T11:44:15.787 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-10T11:44:15.788 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.788 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.788 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.788 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:15.788 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.788 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.788 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.788 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.788 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.788 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.788 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.788 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.788 INFO:teuthology.orchestra.run.smithi100.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-10T11:44:15.788 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-10T11:44:15.788 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.788 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.789 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.789 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:15.789 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.789 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.789 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.789 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.789 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.789 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.789 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.789 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.789 INFO:teuthology.orchestra.run.smithi100.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-10T11:44:15.789 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-10T11:44:15.789 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.789 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.789 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.790 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "[]", 2024-05-10T11:44:15.790 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.790 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.790 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.790 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.790 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.790 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.790 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.790 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.790 INFO:teuthology.orchestra.run.smithi100.stdout: "MULTICLUSTER_CONFIG": { 2024-05-10T11:44:15.790 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-10T11:44:15.790 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.790 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.790 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.790 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "{}", 2024-05-10T11:44:15.791 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.791 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.791 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.791 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.791 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.791 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.791 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.791 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.791 INFO:teuthology.orchestra.run.smithi100.stdout: "PROMETHEUS_API_HOST": { 2024-05-10T11:44:15.791 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-10T11:44:15.791 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.791 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.791 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.791 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:15.791 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.792 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.792 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.792 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.792 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.792 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.792 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.792 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.792 INFO:teuthology.orchestra.run.smithi100.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-10T11:44:15.792 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-10T11:44:15.792 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.792 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.792 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.792 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:15.792 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.792 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.793 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.793 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.793 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.793 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.793 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.793 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.793 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-10T11:44:15.793 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-10T11:44:15.793 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.793 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.793 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.793 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:15.793 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.793 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.793 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.794 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.794 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.794 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.794 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.794 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.794 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-10T11:44:15.794 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-10T11:44:15.794 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.794 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.794 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.794 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:15.794 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.794 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.794 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.794 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.795 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.795 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.795 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.795 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.795 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-10T11:44:15.795 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-10T11:44:15.795 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.795 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.795 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.795 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:15.795 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.795 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.795 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.796 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.796 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.796 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.796 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.796 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.796 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-10T11:44:15.796 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-10T11:44:15.796 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.796 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.796 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.796 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:15.796 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.796 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.796 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.796 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.797 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.797 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.797 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.797 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.797 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-10T11:44:15.797 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-10T11:44:15.797 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.797 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.797 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.797 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:15.797 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.797 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.797 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.797 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.797 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.798 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.798 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.798 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.798 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-10T11:44:15.798 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-10T11:44:15.798 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.798 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.798 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.798 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:15.798 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.798 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.798 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.798 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.798 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.799 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.799 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.799 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.799 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-10T11:44:15.799 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-10T11:44:15.799 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.799 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.799 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.799 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:15.799 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.799 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.799 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.799 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.799 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.799 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.800 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.800 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.800 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_ENABLED": { 2024-05-10T11:44:15.800 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-10T11:44:15.800 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.800 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.800 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.800 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:15.800 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.800 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.800 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.800 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.800 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.800 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.800 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.801 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.801 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-10T11:44:15.801 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-10T11:44:15.801 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.801 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.801 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.801 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-10T11:44:15.801 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.801 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.801 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.801 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.801 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.801 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.801 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.801 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.802 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-10T11:44:15.802 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-10T11:44:15.802 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:15.802 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.802 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.802 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10", 2024-05-10T11:44:15.802 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.802 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.802 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.802 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.802 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.802 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.802 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.802 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.803 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-10T11:44:15.803 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-10T11:44:15.803 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:15.803 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.803 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.803 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "8", 2024-05-10T11:44:15.803 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.803 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.803 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.803 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.803 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.803 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.803 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.803 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.803 INFO:teuthology.orchestra.run.smithi100.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-10T11:44:15.804 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-10T11:44:15.804 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:15.804 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.804 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.804 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "45", 2024-05-10T11:44:15.804 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.804 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.804 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.804 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.804 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.804 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.804 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.804 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.804 INFO:teuthology.orchestra.run.smithi100.stdout: "RGW_API_ACCESS_KEY": { 2024-05-10T11:44:15.804 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-10T11:44:15.804 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.805 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.805 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.805 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:15.805 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.805 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.805 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.805 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.805 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.805 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.805 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.805 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.805 INFO:teuthology.orchestra.run.smithi100.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-10T11:44:15.805 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-10T11:44:15.805 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.805 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.806 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.806 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "admin", 2024-05-10T11:44:15.806 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.806 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.806 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.806 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.806 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.806 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.806 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.807 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.807 INFO:teuthology.orchestra.run.smithi100.stdout: "RGW_API_SECRET_KEY": { 2024-05-10T11:44:15.807 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-10T11:44:15.807 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.807 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.807 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.807 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:15.807 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.807 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.807 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.807 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.807 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.807 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.808 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.808 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.808 INFO:teuthology.orchestra.run.smithi100.stdout: "RGW_API_SSL_VERIFY": { 2024-05-10T11:44:15.808 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-10T11:44:15.808 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.808 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.808 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.808 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:15.808 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.808 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.808 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.808 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.808 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.809 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.809 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.809 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.809 INFO:teuthology.orchestra.run.smithi100.stdout: "UNSAFE_TLS_v1_2": { 2024-05-10T11:44:15.809 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-10T11:44:15.809 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.809 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.809 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.809 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:15.809 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.809 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.809 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.809 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.809 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.809 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.810 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.810 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.810 INFO:teuthology.orchestra.run.smithi100.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-10T11:44:15.810 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-10T11:44:15.810 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:15.810 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.810 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.810 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:15.810 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.810 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.810 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.810 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.810 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.810 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.810 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.811 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.811 INFO:teuthology.orchestra.run.smithi100.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-10T11:44:15.811 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-10T11:44:15.811 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:15.811 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.811 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.811 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10", 2024-05-10T11:44:15.811 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.811 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.811 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.811 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.811 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.811 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.811 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.811 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.812 INFO:teuthology.orchestra.run.smithi100.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-10T11:44:15.812 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-10T11:44:15.812 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:15.812 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.812 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.812 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "5", 2024-05-10T11:44:15.812 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.812 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.812 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.812 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.812 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.812 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.812 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.812 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.813 INFO:teuthology.orchestra.run.smithi100.stdout: "cross_origin_url": { 2024-05-10T11:44:15.813 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "cross_origin_url", 2024-05-10T11:44:15.813 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.813 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.813 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.813 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:15.813 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.813 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.813 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.813 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.813 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.813 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.813 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.813 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.813 INFO:teuthology.orchestra.run.smithi100.stdout: "crt_file": { 2024-05-10T11:44:15.814 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "crt_file", 2024-05-10T11:44:15.814 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.814 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.814 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.814 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:15.814 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.814 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.814 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.814 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.814 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.814 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.814 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.814 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.814 INFO:teuthology.orchestra.run.smithi100.stdout: "debug": { 2024-05-10T11:44:15.814 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "debug", 2024-05-10T11:44:15.815 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.815 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.815 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.815 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:15.815 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.815 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.815 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.815 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Enable/disable debug options", 2024-05-10T11:44:15.815 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.815 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.815 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.815 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.815 INFO:teuthology.orchestra.run.smithi100.stdout: "jwt_token_ttl": { 2024-05-10T11:44:15.815 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "jwt_token_ttl", 2024-05-10T11:44:15.815 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:15.816 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.816 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.816 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "28800", 2024-05-10T11:44:15.816 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.816 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.816 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.816 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.816 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.816 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.816 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.816 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.816 INFO:teuthology.orchestra.run.smithi100.stdout: "key_file": { 2024-05-10T11:44:15.816 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "key_file", 2024-05-10T11:44:15.816 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.816 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.817 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.817 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:15.817 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.817 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.817 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.817 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.817 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.817 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.817 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.817 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.817 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:15.817 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:15.817 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.817 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.817 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.818 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:15.818 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.818 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.818 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:15.818 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:15.818 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:15.818 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:15.818 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:15.818 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:15.818 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:15.818 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:15.818 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.818 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.819 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.819 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.819 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.819 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:15.819 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:15.819 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.819 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.819 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.819 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:15.819 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.819 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.819 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.819 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.819 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.820 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.820 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.820 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.820 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:15.820 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:15.820 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.820 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.820 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.820 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:15.820 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.821 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.821 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:15.821 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:15.821 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:15.821 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:15.821 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:15.821 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:15.821 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:15.821 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:15.821 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.821 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.821 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.821 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.822 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.822 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:15.822 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:15.822 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.822 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.822 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.822 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:15.822 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.822 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.822 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.822 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.822 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.822 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.822 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.823 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.823 INFO:teuthology.orchestra.run.smithi100.stdout: "motd": { 2024-05-10T11:44:15.823 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "motd", 2024-05-10T11:44:15.823 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.823 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.823 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.823 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:15.823 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.823 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.823 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.823 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "The message of the day", 2024-05-10T11:44:15.823 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.823 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.823 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.823 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.824 INFO:teuthology.orchestra.run.smithi100.stdout: "redirect_resolve_ip_addr": { 2024-05-10T11:44:15.824 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "redirect_resolve_ip_addr", 2024-05-10T11:44:15.824 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.824 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.824 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.824 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:15.824 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.824 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.824 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.824 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.824 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.824 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.824 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.824 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.825 INFO:teuthology.orchestra.run.smithi100.stdout: "server_addr": { 2024-05-10T11:44:15.825 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "server_addr", 2024-05-10T11:44:15.825 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.825 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.825 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.825 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "::", 2024-05-10T11:44:15.825 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.825 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.825 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.825 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.825 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.825 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.825 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.825 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.825 INFO:teuthology.orchestra.run.smithi100.stdout: "server_port": { 2024-05-10T11:44:15.825 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "server_port", 2024-05-10T11:44:15.826 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:15.826 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.826 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.826 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "8080", 2024-05-10T11:44:15.826 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.826 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.826 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.826 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.826 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.826 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.826 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.826 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.826 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:15.827 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:15.827 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:15.827 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:15.827 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.827 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:15.827 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.827 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.827 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.827 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.827 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.827 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.827 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.827 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.827 INFO:teuthology.orchestra.run.smithi100.stdout: "ssl": { 2024-05-10T11:44:15.828 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ssl", 2024-05-10T11:44:15.828 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.828 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.828 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.828 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:15.828 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.828 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.828 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.828 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.828 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.828 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.828 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.828 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.828 INFO:teuthology.orchestra.run.smithi100.stdout: "ssl_server_port": { 2024-05-10T11:44:15.828 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ssl_server_port", 2024-05-10T11:44:15.828 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:15.829 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.829 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.829 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "8443", 2024-05-10T11:44:15.829 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.829 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.829 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.829 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.829 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.829 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.829 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.829 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.829 INFO:teuthology.orchestra.run.smithi100.stdout: "standby_behaviour": { 2024-05-10T11:44:15.829 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "standby_behaviour", 2024-05-10T11:44:15.829 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.829 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.830 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.830 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "redirect", 2024-05-10T11:44:15.830 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.830 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.830 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:15.830 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:15.830 INFO:teuthology.orchestra.run.smithi100.stdout: "redirect" 2024-05-10T11:44:15.830 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:15.830 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.830 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.830 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.830 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.830 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.830 INFO:teuthology.orchestra.run.smithi100.stdout: "standby_error_status_code": { 2024-05-10T11:44:15.831 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "standby_error_status_code", 2024-05-10T11:44:15.831 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:15.831 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.831 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.831 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "500", 2024-05-10T11:44:15.831 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "400", 2024-05-10T11:44:15.831 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "599", 2024-05-10T11:44:15.831 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.831 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.831 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.831 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.831 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.831 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.831 INFO:teuthology.orchestra.run.smithi100.stdout: "url_prefix": { 2024-05-10T11:44:15.831 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "url_prefix", 2024-05-10T11:44:15.832 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.832 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.832 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.832 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:15.832 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.832 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.832 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.832 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.832 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.832 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.832 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.832 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:15.832 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:15.832 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.832 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:15.833 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "devicehealth", 2024-05-10T11:44:15.833 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:15.833 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:15.833 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:15.833 INFO:teuthology.orchestra.run.smithi100.stdout: "enable_monitoring": { 2024-05-10T11:44:15.833 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "enable_monitoring", 2024-05-10T11:44:15.833 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.833 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.833 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.833 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:15.833 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.833 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.833 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.833 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "monitor device health metrics", 2024-05-10T11:44:15.833 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.834 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.834 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.834 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.834 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:15.834 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:15.834 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.834 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.834 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.834 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:15.834 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.834 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.834 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:15.834 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:15.834 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:15.835 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:15.835 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:15.835 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:15.835 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:15.835 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:15.835 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.835 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.835 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.835 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.835 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.835 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:15.835 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:15.835 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.835 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.835 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.836 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:15.836 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.836 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.836 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.836 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.836 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.836 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.836 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.836 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.836 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:15.836 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:15.836 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.836 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.836 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.837 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:15.837 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.837 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.837 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:15.837 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:15.837 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:15.837 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:15.837 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:15.837 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:15.837 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:15.837 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:15.837 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.837 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.837 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.837 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.838 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.838 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:15.838 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:15.838 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.838 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.838 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.838 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:15.838 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.838 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.838 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.838 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.838 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.838 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.838 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.838 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.839 INFO:teuthology.orchestra.run.smithi100.stdout: "mark_out_threshold": { 2024-05-10T11:44:15.839 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "mark_out_threshold", 2024-05-10T11:44:15.839 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-05-10T11:44:15.839 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.839 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.839 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "2419200", 2024-05-10T11:44:15.839 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.839 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.839 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.839 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-10T11:44:15.839 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.839 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.839 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.839 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.839 INFO:teuthology.orchestra.run.smithi100.stdout: "pool_name": { 2024-05-10T11:44:15.840 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "pool_name", 2024-05-10T11:44:15.840 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.840 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.840 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.840 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "device_health_metrics", 2024-05-10T11:44:15.840 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.840 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.840 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.840 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-10T11:44:15.840 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.840 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.840 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.840 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.840 INFO:teuthology.orchestra.run.smithi100.stdout: "retention_period": { 2024-05-10T11:44:15.840 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "retention_period", 2024-05-10T11:44:15.841 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-05-10T11:44:15.841 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.841 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.841 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "15552000", 2024-05-10T11:44:15.841 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.841 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.841 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.841 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "how long to retain device health metrics", 2024-05-10T11:44:15.841 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.841 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.841 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.841 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.841 INFO:teuthology.orchestra.run.smithi100.stdout: "scrape_frequency": { 2024-05-10T11:44:15.841 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "scrape_frequency", 2024-05-10T11:44:15.841 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-05-10T11:44:15.842 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.842 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.842 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "86400", 2024-05-10T11:44:15.842 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.842 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.842 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.842 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-10T11:44:15.842 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.842 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.842 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.842 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.842 INFO:teuthology.orchestra.run.smithi100.stdout: "self_heal": { 2024-05-10T11:44:15.843 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "self_heal", 2024-05-10T11:44:15.843 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.843 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.843 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.843 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:15.843 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.843 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.843 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.843 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-10T11:44:15.843 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.843 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.843 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.843 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.843 INFO:teuthology.orchestra.run.smithi100.stdout: "sleep_interval": { 2024-05-10T11:44:15.843 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sleep_interval", 2024-05-10T11:44:15.844 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-05-10T11:44:15.844 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.844 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.844 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "600", 2024-05-10T11:44:15.844 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.844 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.844 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.844 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "how frequently to wake up and check device health", 2024-05-10T11:44:15.844 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.844 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.844 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.844 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.844 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:15.844 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:15.844 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:15.845 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:15.845 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.845 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:15.845 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.845 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.845 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.845 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.845 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.845 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.845 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.845 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.845 INFO:teuthology.orchestra.run.smithi100.stdout: "warn_threshold": { 2024-05-10T11:44:15.845 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "warn_threshold", 2024-05-10T11:44:15.845 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-05-10T11:44:15.845 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.846 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.846 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "7257600", 2024-05-10T11:44:15.846 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.846 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.846 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.846 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-10T11:44:15.846 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.846 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.846 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.846 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:15.846 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:15.846 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.846 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:15.846 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "diskprediction_local", 2024-05-10T11:44:15.847 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:15.847 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:15.847 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:15.847 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:15.847 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:15.847 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.847 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.847 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.847 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:15.847 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.847 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.847 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:15.847 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:15.847 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:15.848 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:15.848 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:15.848 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:15.848 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:15.848 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:15.848 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.848 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.848 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.848 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.848 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.848 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:15.848 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:15.849 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.849 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.849 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.849 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:15.849 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.849 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.849 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.849 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.849 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.849 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.849 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.849 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.849 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:15.849 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:15.849 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.850 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.850 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.850 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:15.850 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.850 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.850 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:15.850 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:15.850 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:15.850 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:15.850 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:15.850 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:15.850 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:15.850 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:15.850 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.850 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.851 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.851 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.851 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.851 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:15.851 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:15.851 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.851 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.851 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.851 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:15.851 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.851 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.851 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.851 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.852 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.852 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.852 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.852 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.852 INFO:teuthology.orchestra.run.smithi100.stdout: "predict_interval": { 2024-05-10T11:44:15.852 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "predict_interval", 2024-05-10T11:44:15.852 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.852 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.852 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.852 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "86400", 2024-05-10T11:44:15.852 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.852 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.852 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.852 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.852 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.853 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.853 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.853 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.853 INFO:teuthology.orchestra.run.smithi100.stdout: "predictor_model": { 2024-05-10T11:44:15.853 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "predictor_model", 2024-05-10T11:44:15.853 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.853 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.853 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.853 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "prophetstor", 2024-05-10T11:44:15.853 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.853 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.853 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.853 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.853 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.853 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.854 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.854 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.854 INFO:teuthology.orchestra.run.smithi100.stdout: "sleep_interval": { 2024-05-10T11:44:15.854 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sleep_interval", 2024-05-10T11:44:15.854 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.854 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.854 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.854 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "600", 2024-05-10T11:44:15.854 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.854 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.854 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.854 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.854 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.854 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.855 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.855 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.855 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:15.855 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:15.855 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:15.855 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:15.855 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.855 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:15.855 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.855 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.855 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.855 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.855 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.855 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.855 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.856 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:15.856 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:15.856 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.856 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:15.856 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "influx", 2024-05-10T11:44:15.856 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": false, 2024-05-10T11:44:15.856 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "influxdb python module not found", 2024-05-10T11:44:15.856 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:15.856 INFO:teuthology.orchestra.run.smithi100.stdout: "batch_size": { 2024-05-10T11:44:15.856 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "batch_size", 2024-05-10T11:44:15.856 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:15.856 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.856 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.856 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "5000", 2024-05-10T11:44:15.856 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.857 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.857 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.857 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-10T11:44:15.857 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.857 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.857 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.857 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.857 INFO:teuthology.orchestra.run.smithi100.stdout: "database": { 2024-05-10T11:44:15.857 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "database", 2024-05-10T11:44:15.857 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.857 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.857 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.857 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "ceph", 2024-05-10T11:44:15.857 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.857 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.857 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.858 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-10T11:44:15.858 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.858 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.858 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.858 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.858 INFO:teuthology.orchestra.run.smithi100.stdout: "hostname": { 2024-05-10T11:44:15.858 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "hostname", 2024-05-10T11:44:15.858 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.858 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.858 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.858 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:15.858 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.858 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.858 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.859 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "InfluxDB server hostname", 2024-05-10T11:44:15.859 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.859 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.859 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.859 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.859 INFO:teuthology.orchestra.run.smithi100.stdout: "interval": { 2024-05-10T11:44:15.859 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "interval", 2024-05-10T11:44:15.859 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-05-10T11:44:15.859 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.859 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.859 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "30", 2024-05-10T11:44:15.859 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "5", 2024-05-10T11:44:15.859 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.859 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.859 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-10T11:44:15.860 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.860 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.860 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.860 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.860 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:15.860 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:15.860 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.860 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.860 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.860 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:15.860 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.860 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.860 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:15.860 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:15.861 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:15.861 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:15.861 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:15.861 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:15.861 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:15.861 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:15.861 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.861 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.861 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.861 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.861 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.861 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:15.861 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:15.862 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.862 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.862 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.862 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:15.862 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.862 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.862 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.862 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.862 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.862 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.862 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.862 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.862 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:15.862 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:15.862 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.863 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.863 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.863 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:15.863 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.863 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.863 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:15.863 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:15.863 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:15.863 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:15.863 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:15.863 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:15.863 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:15.863 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:15.863 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.864 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.864 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.864 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.864 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.864 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:15.864 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:15.864 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.864 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.864 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.864 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:15.864 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.864 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.864 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.865 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.865 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.865 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.865 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.865 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.865 INFO:teuthology.orchestra.run.smithi100.stdout: "password": { 2024-05-10T11:44:15.865 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "password", 2024-05-10T11:44:15.865 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.865 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.865 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.865 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:15.865 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.865 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.865 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.865 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "password of InfluxDB server user", 2024-05-10T11:44:15.866 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.866 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.866 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.866 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.866 INFO:teuthology.orchestra.run.smithi100.stdout: "port": { 2024-05-10T11:44:15.866 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "port", 2024-05-10T11:44:15.866 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:15.866 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.866 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.866 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "8086", 2024-05-10T11:44:15.866 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.866 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.866 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.866 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "InfluxDB server port", 2024-05-10T11:44:15.867 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.867 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.867 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.867 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.867 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:15.867 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:15.867 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:15.867 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:15.867 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.867 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:15.867 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.867 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.867 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.867 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.867 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.868 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.868 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.868 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.868 INFO:teuthology.orchestra.run.smithi100.stdout: "ssl": { 2024-05-10T11:44:15.868 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ssl", 2024-05-10T11:44:15.868 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.868 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.868 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.868 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "false", 2024-05-10T11:44:15.868 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.868 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.868 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.868 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-10T11:44:15.869 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.869 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.869 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.869 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.869 INFO:teuthology.orchestra.run.smithi100.stdout: "threads": { 2024-05-10T11:44:15.869 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "threads", 2024-05-10T11:44:15.869 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:15.869 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.869 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.869 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "5", 2024-05-10T11:44:15.869 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "1", 2024-05-10T11:44:15.869 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "32", 2024-05-10T11:44:15.869 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.869 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-10T11:44:15.869 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.870 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.870 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.870 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.870 INFO:teuthology.orchestra.run.smithi100.stdout: "username": { 2024-05-10T11:44:15.870 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "username", 2024-05-10T11:44:15.870 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.870 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.870 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.870 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:15.870 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.870 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.870 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.870 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "username of InfluxDB server user", 2024-05-10T11:44:15.870 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.871 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.871 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.871 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.871 INFO:teuthology.orchestra.run.smithi100.stdout: "verify_ssl": { 2024-05-10T11:44:15.871 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "verify_ssl", 2024-05-10T11:44:15.871 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.871 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.871 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.871 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "true", 2024-05-10T11:44:15.871 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.871 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.871 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.871 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-10T11:44:15.871 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.871 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.872 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.872 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:15.872 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:15.872 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.872 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:15.872 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "insights", 2024-05-10T11:44:15.872 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:15.872 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:15.872 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:15.872 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:15.872 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:15.872 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.872 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.872 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.873 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:15.873 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.873 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.873 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:15.873 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:15.873 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:15.873 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:15.873 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:15.873 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:15.873 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:15.873 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:15.873 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.873 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.874 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.874 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.874 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.874 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:15.874 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:15.874 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.874 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.874 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.874 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:15.874 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.874 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.874 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.874 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.875 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.875 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.875 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.875 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.875 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:15.875 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:15.875 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.875 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.875 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.875 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:15.875 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.875 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.875 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:15.875 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:15.875 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:15.876 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:15.876 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:15.876 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:15.876 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:15.876 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:15.876 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.876 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.876 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.876 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.876 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.876 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:15.876 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:15.876 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.876 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.876 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.877 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:15.877 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.877 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.877 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.877 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.877 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.877 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.877 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.877 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.877 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:15.877 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:15.877 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:15.877 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:15.877 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.877 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:15.878 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.878 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.878 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.878 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.878 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.878 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.878 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.878 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:15.878 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:15.878 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.878 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:15.878 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "iostat", 2024-05-10T11:44:15.878 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:15.878 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:15.878 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:15.879 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:15.879 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:15.879 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.879 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.879 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.879 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:15.879 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.879 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.879 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:15.879 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:15.879 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:15.879 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:15.879 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:15.879 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:15.880 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:15.880 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:15.880 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.880 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.880 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.880 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.880 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.880 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:15.880 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:15.880 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.880 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.880 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.880 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:15.880 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.880 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.880 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.881 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.881 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.881 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.881 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.881 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.881 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:15.881 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:15.881 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.881 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.881 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.881 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:15.881 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.881 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.881 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:15.881 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:15.882 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:15.882 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:15.882 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:15.882 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:15.882 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:15.882 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:15.882 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.882 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.882 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.882 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.882 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.882 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:15.882 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:15.883 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.883 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.883 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.883 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:15.883 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.883 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.883 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.883 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.883 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.883 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.883 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.883 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.883 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:15.883 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:15.883 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:15.883 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:15.884 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.884 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:15.884 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.884 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.884 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.884 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.884 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.884 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.884 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.884 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:15.884 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:15.884 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.884 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:15.884 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "k8sevents", 2024-05-10T11:44:15.884 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:15.885 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:15.885 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:15.885 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_event_retention_days": { 2024-05-10T11:44:15.885 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ceph_event_retention_days", 2024-05-10T11:44:15.885 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:15.885 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.885 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.885 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "7", 2024-05-10T11:44:15.885 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.885 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.885 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.885 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-10T11:44:15.885 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.885 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.885 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.886 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.886 INFO:teuthology.orchestra.run.smithi100.stdout: "config_check_secs": { 2024-05-10T11:44:15.886 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "config_check_secs", 2024-05-10T11:44:15.886 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:15.886 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.886 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.886 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10", 2024-05-10T11:44:15.886 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "10", 2024-05-10T11:44:15.886 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.886 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.886 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-10T11:44:15.886 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.886 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.886 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.887 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.887 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:15.887 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:15.887 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.887 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.887 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.887 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:15.887 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.887 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.887 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:15.887 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:15.887 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:15.887 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:15.887 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:15.888 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:15.888 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:15.888 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:15.888 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.888 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.888 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.888 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.888 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.888 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:15.888 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:15.888 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.888 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.888 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.888 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:15.889 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.889 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.889 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.889 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.889 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.889 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.889 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.889 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.889 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:15.889 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:15.889 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.889 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.889 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.889 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:15.889 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.890 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.890 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:15.890 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:15.890 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:15.890 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:15.890 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:15.890 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:15.890 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:15.890 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:15.890 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.890 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.890 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.890 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.890 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.891 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:15.891 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:15.891 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.891 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.891 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.891 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:15.891 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.891 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.891 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.891 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.891 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.891 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.891 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.891 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.891 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:15.892 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:15.892 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:15.892 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:15.892 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.892 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:15.892 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.892 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.892 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.892 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.892 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.892 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.892 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.892 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:15.892 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:15.892 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.893 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:15.893 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "localpool", 2024-05-10T11:44:15.893 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:15.893 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:15.893 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:15.893 INFO:teuthology.orchestra.run.smithi100.stdout: "failure_domain": { 2024-05-10T11:44:15.893 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "failure_domain", 2024-05-10T11:44:15.893 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.893 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.893 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.893 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "host", 2024-05-10T11:44:15.893 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.893 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.893 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.893 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "failure domain for any created local pool", 2024-05-10T11:44:15.894 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-10T11:44:15.894 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.894 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.894 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.894 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:15.894 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:15.894 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.894 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.894 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.894 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:15.894 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.894 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.894 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:15.894 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:15.895 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:15.895 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:15.895 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:15.895 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:15.895 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:15.895 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:15.895 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.895 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.895 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.895 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.895 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.895 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:15.895 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:15.895 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.895 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.896 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.896 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:15.896 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.896 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.896 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.896 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.896 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.896 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.896 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.896 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.896 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:15.896 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:15.896 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.896 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.896 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.897 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:15.897 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.897 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.897 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:15.897 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:15.897 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:15.897 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:15.897 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:15.897 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:15.897 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:15.897 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:15.897 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.897 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.897 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.897 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.898 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.898 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:15.898 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:15.898 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.898 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.898 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.898 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:15.898 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.898 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.898 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.898 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.898 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.898 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.898 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.899 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.899 INFO:teuthology.orchestra.run.smithi100.stdout: "min_size": { 2024-05-10T11:44:15.899 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "min_size", 2024-05-10T11:44:15.899 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:15.899 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.899 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.899 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:15.899 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.899 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.899 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.899 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "default min_size for any created local pool", 2024-05-10T11:44:15.899 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-10T11:44:15.899 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.899 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.899 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.900 INFO:teuthology.orchestra.run.smithi100.stdout: "num_rep": { 2024-05-10T11:44:15.900 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "num_rep", 2024-05-10T11:44:15.900 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:15.900 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.900 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.900 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "3", 2024-05-10T11:44:15.900 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.900 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.900 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.900 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "default replica count for any created local pool", 2024-05-10T11:44:15.900 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.901 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.901 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.901 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.901 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num": { 2024-05-10T11:44:15.901 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "pg_num", 2024-05-10T11:44:15.901 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:15.901 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.901 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.901 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "128", 2024-05-10T11:44:15.901 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.901 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.901 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.901 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "default pg_num for any created local pool", 2024-05-10T11:44:15.901 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.901 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.902 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.902 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.902 INFO:teuthology.orchestra.run.smithi100.stdout: "prefix": { 2024-05-10T11:44:15.902 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "prefix", 2024-05-10T11:44:15.902 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.902 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.902 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.902 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:15.906 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.906 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.906 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.906 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "name prefix for any created local pool", 2024-05-10T11:44:15.906 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.906 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.906 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.906 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.906 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:15.907 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:15.907 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:15.907 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:15.907 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.907 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:15.907 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.907 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.907 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.907 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.907 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.907 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.907 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.907 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.907 INFO:teuthology.orchestra.run.smithi100.stdout: "subtree": { 2024-05-10T11:44:15.907 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "subtree", 2024-05-10T11:44:15.908 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.908 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.908 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.908 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "rack", 2024-05-10T11:44:15.908 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.908 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.908 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.908 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-10T11:44:15.908 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-10T11:44:15.908 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.908 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.908 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:15.909 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:15.909 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.909 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:15.909 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "mds_autoscaler", 2024-05-10T11:44:15.909 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:15.909 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:15.909 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:15.909 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:15.909 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:15.909 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.909 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.909 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.909 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:15.910 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.910 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.910 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:15.910 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:15.910 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:15.910 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:15.910 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:15.910 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:15.910 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:15.910 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:15.910 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.910 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.910 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.910 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.911 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.911 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:15.911 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:15.911 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.911 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.911 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.911 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:15.911 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.911 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.911 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.911 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.911 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.911 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.911 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.911 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.912 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:15.912 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:15.912 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.912 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.912 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.912 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:15.912 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.912 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.912 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:15.912 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:15.912 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:15.912 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:15.912 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:15.912 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:15.912 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:15.913 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:15.913 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.913 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.913 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.913 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.913 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.913 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:15.913 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:15.913 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.913 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.913 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.913 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:15.913 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.913 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.914 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.914 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.914 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.914 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.914 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.914 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.914 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:15.914 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:15.914 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:15.914 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:15.914 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.914 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:15.914 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.914 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.915 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.915 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.915 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.915 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.915 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.915 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:15.915 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:15.915 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.915 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:15.915 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "mirroring", 2024-05-10T11:44:15.915 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:15.915 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:15.915 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:15.915 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:15.915 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:15.916 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.916 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.916 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.916 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:15.916 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.916 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.916 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:15.916 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:15.916 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:15.916 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:15.916 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:15.916 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:15.916 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:15.916 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:15.916 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.916 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.917 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.917 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.917 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.917 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:15.917 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:15.917 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.917 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.917 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.917 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:15.917 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.917 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.917 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.918 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.918 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.918 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.918 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.918 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.918 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:15.918 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:15.918 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.918 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.918 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.918 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:15.918 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.918 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.918 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:15.919 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:15.919 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:15.919 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:15.919 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:15.919 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:15.919 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:15.919 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:15.919 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.919 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.919 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.919 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.919 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.919 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:15.919 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:15.920 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.920 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.920 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.920 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:15.920 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.920 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.920 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.920 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.920 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.920 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.920 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.920 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.920 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:15.920 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:15.921 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:15.921 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:15.921 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.921 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:15.921 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.921 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.921 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.921 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.921 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.921 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.921 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.921 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:15.921 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:15.921 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.921 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:15.921 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "nfs", 2024-05-10T11:44:15.922 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:15.922 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:15.922 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:15.922 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:15.922 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:15.922 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.922 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.922 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.922 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:15.922 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.922 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.922 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:15.922 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:15.923 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:15.923 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:15.923 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:15.923 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:15.923 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:15.923 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:15.923 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.923 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.923 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.923 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.923 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.923 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:15.923 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:15.923 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.924 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.924 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.924 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:15.924 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.924 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.924 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.924 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.924 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.924 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.924 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.924 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.924 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:15.924 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:15.924 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.924 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.925 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.925 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:15.925 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.925 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.925 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:15.925 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:15.925 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:15.925 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:15.925 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:15.925 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:15.925 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:15.925 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:15.925 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.925 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.926 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.926 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.926 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.926 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:15.926 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:15.926 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.926 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.926 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.926 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:15.926 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.926 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.926 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.926 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.926 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.926 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.927 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.927 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.927 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:15.927 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:15.927 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:15.927 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:15.927 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.927 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:15.927 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.927 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.927 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.927 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.927 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.927 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.928 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.928 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:15.928 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:15.928 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.928 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:15.928 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "orchestrator", 2024-05-10T11:44:15.928 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:15.928 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:15.928 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:15.928 INFO:teuthology.orchestra.run.smithi100.stdout: "fail_fs": { 2024-05-10T11:44:15.928 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "fail_fs", 2024-05-10T11:44:15.928 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.928 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.928 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.928 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:15.928 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.929 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.929 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.929 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-10T11:44:15.929 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.929 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.929 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.929 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.929 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:15.929 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:15.929 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.929 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.929 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.929 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:15.929 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.929 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.930 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:15.930 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:15.930 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:15.930 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:15.930 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:15.930 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:15.930 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:15.930 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:15.930 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.930 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.930 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.930 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.930 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.930 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:15.930 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:15.931 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.931 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.931 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.931 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:15.931 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.931 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.931 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.931 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.931 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.931 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.931 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.931 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.931 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:15.931 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:15.932 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.932 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.932 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.932 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:15.932 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.932 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.932 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:15.932 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:15.932 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:15.932 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:15.932 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:15.932 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:15.932 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:15.932 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:15.932 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.933 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.933 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.933 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.933 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.933 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:15.933 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:15.933 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.933 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.933 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.933 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:15.933 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.933 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.933 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.933 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.933 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.933 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.934 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.934 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.934 INFO:teuthology.orchestra.run.smithi100.stdout: "orchestrator": { 2024-05-10T11:44:15.934 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "orchestrator", 2024-05-10T11:44:15.934 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.934 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.934 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.934 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:15.934 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.934 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.934 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:15.934 INFO:teuthology.orchestra.run.smithi100.stdout: "cephadm", 2024-05-10T11:44:15.934 INFO:teuthology.orchestra.run.smithi100.stdout: "rook", 2024-05-10T11:44:15.934 INFO:teuthology.orchestra.run.smithi100.stdout: "test_orchestrator" 2024-05-10T11:44:15.935 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:15.935 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Orchestrator backend", 2024-05-10T11:44:15.935 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.935 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.935 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.935 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.935 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:15.935 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:15.935 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:15.935 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:15.935 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.935 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:15.935 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.935 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.935 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.935 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.936 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.936 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.936 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.936 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:15.936 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:15.936 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.936 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:15.936 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "osd_perf_query", 2024-05-10T11:44:15.936 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:15.936 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:15.936 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:15.936 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:15.936 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:15.936 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.936 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.937 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.937 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:15.937 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.937 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.937 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:15.937 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:15.937 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:15.937 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:15.937 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:15.937 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:15.937 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:15.937 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:15.937 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.937 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.937 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.938 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.938 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.938 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:15.938 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:15.938 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.938 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.938 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.938 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:15.938 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.938 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.938 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.938 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.938 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.938 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.938 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.939 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.939 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:15.939 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:15.939 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.939 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.939 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.939 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:15.939 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.939 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.939 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:15.939 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:15.939 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:15.939 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:15.939 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:15.939 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:15.940 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:15.940 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:15.940 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.940 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.940 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.940 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.940 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.940 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:15.940 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:15.940 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.940 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.940 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.940 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:15.940 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.940 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.941 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.941 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.941 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.941 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.941 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.941 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.941 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:15.941 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:15.941 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:15.941 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:15.941 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.941 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:15.941 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.941 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.942 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.942 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.942 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.942 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.942 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.942 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:15.942 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:15.942 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.942 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:15.942 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "osd_support", 2024-05-10T11:44:15.942 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:15.942 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:15.942 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:15.942 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:15.942 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:15.943 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.943 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.943 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.943 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:15.943 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.943 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.943 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:15.943 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:15.943 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:15.943 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:15.943 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:15.943 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:15.943 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:15.943 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:15.944 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.944 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.944 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.944 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.944 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.944 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:15.944 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:15.944 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.944 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.944 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.944 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:15.944 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.944 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.944 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.945 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.945 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.945 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.945 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.945 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.945 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:15.945 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:15.945 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.945 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.945 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.945 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:15.945 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.945 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.945 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:15.946 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:15.946 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:15.946 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:15.946 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:15.946 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:15.946 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:15.946 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:15.946 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.946 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.946 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.946 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.946 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.946 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:15.946 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:15.947 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.947 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.947 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.947 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:15.947 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.947 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.947 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.947 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.947 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.947 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.947 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.947 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.947 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:15.948 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:15.948 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:15.948 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:15.948 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.948 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:15.948 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.948 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.948 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.948 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.948 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.948 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.948 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.948 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:15.948 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:15.949 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.949 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:15.949 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "pg_autoscaler", 2024-05-10T11:44:15.949 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:15.949 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:15.949 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:15.949 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:15.949 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:15.949 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.949 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.949 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.949 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:15.949 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.949 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.949 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:15.950 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:15.950 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:15.950 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:15.950 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:15.950 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:15.950 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:15.950 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:15.950 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.950 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.950 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.950 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.950 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.950 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:15.950 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:15.950 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.951 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.951 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.951 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:15.951 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.951 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.951 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.951 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.951 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.951 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.951 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.951 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.951 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:15.951 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:15.951 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.951 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.952 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.952 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:15.952 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.952 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.952 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:15.952 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:15.952 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:15.952 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:15.952 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:15.952 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:15.952 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:15.952 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:15.952 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.952 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.952 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.953 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.953 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.953 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:15.953 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:15.953 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.953 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.953 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.953 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:15.953 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.953 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.953 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.953 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.953 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.953 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.953 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.954 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.954 INFO:teuthology.orchestra.run.smithi100.stdout: "sleep_interval": { 2024-05-10T11:44:15.954 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sleep_interval", 2024-05-10T11:44:15.954 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-05-10T11:44:15.954 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.954 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.954 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "60", 2024-05-10T11:44:15.954 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.954 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.954 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.954 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.954 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.954 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.954 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.954 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.955 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:15.955 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:15.955 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:15.955 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:15.955 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.955 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:15.955 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.955 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.955 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.955 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.955 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.955 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.955 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.955 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.955 INFO:teuthology.orchestra.run.smithi100.stdout: "threshold": { 2024-05-10T11:44:15.956 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "threshold", 2024-05-10T11:44:15.956 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "float", 2024-05-10T11:44:15.956 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.956 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.956 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "3.0", 2024-05-10T11:44:15.956 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "1.0", 2024-05-10T11:44:15.956 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.956 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.956 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "scaling threshold", 2024-05-10T11:44:15.956 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-10T11:44:15.956 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.956 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.956 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:15.956 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:15.957 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.957 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:15.957 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "progress", 2024-05-10T11:44:15.957 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:15.957 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:15.957 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:15.957 INFO:teuthology.orchestra.run.smithi100.stdout: "allow_pg_recovery_event": { 2024-05-10T11:44:15.957 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "allow_pg_recovery_event", 2024-05-10T11:44:15.957 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.957 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.957 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.957 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:15.957 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.957 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.958 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.958 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-10T11:44:15.958 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.958 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.958 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.958 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.958 INFO:teuthology.orchestra.run.smithi100.stdout: "enabled": { 2024-05-10T11:44:15.958 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "enabled", 2024-05-10T11:44:15.958 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.958 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.958 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.958 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:15.958 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.958 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.959 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.959 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.959 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.959 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.959 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.959 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.959 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:15.959 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:15.959 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.959 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.959 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.959 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:15.959 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.959 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.959 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:15.960 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:15.960 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:15.960 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:15.960 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:15.960 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:15.960 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:15.960 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:15.960 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.960 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.960 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.960 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.960 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.960 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:15.960 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:15.960 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.961 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.961 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.961 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:15.961 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.961 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.961 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.961 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.961 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.961 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.961 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.961 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.961 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:15.961 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:15.961 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.961 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.962 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.962 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:15.962 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.962 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.962 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:15.962 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:15.962 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:15.962 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:15.962 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:15.962 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:15.962 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:15.962 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:15.962 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.962 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.962 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.963 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.963 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.963 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:15.963 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:15.963 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.963 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.963 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.963 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:15.963 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.963 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.963 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.963 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.963 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.963 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.963 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.964 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.964 INFO:teuthology.orchestra.run.smithi100.stdout: "max_completed_events": { 2024-05-10T11:44:15.964 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "max_completed_events", 2024-05-10T11:44:15.964 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:15.964 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.964 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.964 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "50", 2024-05-10T11:44:15.964 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.964 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.964 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.964 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "number of past completed events to remember", 2024-05-10T11:44:15.964 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.964 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.964 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.964 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.965 INFO:teuthology.orchestra.run.smithi100.stdout: "sleep_interval": { 2024-05-10T11:44:15.965 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sleep_interval", 2024-05-10T11:44:15.965 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-05-10T11:44:15.965 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.965 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.965 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "5", 2024-05-10T11:44:15.965 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.965 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.965 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.965 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "how long the module is going to sleep", 2024-05-10T11:44:15.965 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.965 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.965 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.965 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.965 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:15.965 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:15.966 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:15.966 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:15.966 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.966 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:15.966 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.966 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.966 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.966 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.966 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.966 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.966 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.966 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:15.966 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:15.966 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.967 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:15.967 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "prometheus", 2024-05-10T11:44:15.967 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:15.967 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:15.967 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:15.967 INFO:teuthology.orchestra.run.smithi100.stdout: "cache": { 2024-05-10T11:44:15.967 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "cache", 2024-05-10T11:44:15.967 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.967 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.967 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.967 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:15.967 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.967 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.967 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.967 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.968 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.968 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.968 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.968 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.968 INFO:teuthology.orchestra.run.smithi100.stdout: "exclude_perf_counters": { 2024-05-10T11:44:15.968 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "exclude_perf_counters", 2024-05-10T11:44:15.968 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.968 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.968 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.968 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:15.968 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.968 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.968 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.968 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-10T11:44:15.968 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-10T11:44:15.968 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.969 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.969 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.969 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:15.969 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:15.969 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.969 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.969 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.969 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:15.969 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.969 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.969 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:15.969 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:15.969 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:15.969 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:15.969 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:15.970 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:15.970 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:15.970 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:15.970 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.970 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.970 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.970 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.970 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.970 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:15.970 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:15.971 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.971 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.971 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.971 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:15.971 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.971 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.971 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.971 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.971 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.971 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.971 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.971 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.972 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:15.972 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:15.972 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.972 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.972 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.972 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:15.972 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.972 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.972 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:15.972 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:15.972 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:15.972 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:15.973 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:15.973 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:15.973 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:15.973 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:15.973 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.973 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.973 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.973 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.973 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.973 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:15.973 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:15.973 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.973 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.973 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.974 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:15.974 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.974 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.974 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.974 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.974 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.974 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.974 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.974 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.974 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_stats_pools": { 2024-05-10T11:44:15.974 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rbd_stats_pools", 2024-05-10T11:44:15.974 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.974 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.974 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.975 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:15.975 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.975 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.975 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.975 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.975 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.975 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.975 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.975 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.975 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-10T11:44:15.975 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-10T11:44:15.975 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:15.975 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.975 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.976 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "300", 2024-05-10T11:44:15.976 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.976 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.976 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.976 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.976 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.976 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.976 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.976 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.976 INFO:teuthology.orchestra.run.smithi100.stdout: "scrape_interval": { 2024-05-10T11:44:15.976 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "scrape_interval", 2024-05-10T11:44:15.976 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "float", 2024-05-10T11:44:15.976 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.976 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.976 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "15.0", 2024-05-10T11:44:15.977 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.977 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.977 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.977 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.977 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.977 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.977 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.977 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.977 INFO:teuthology.orchestra.run.smithi100.stdout: "server_addr": { 2024-05-10T11:44:15.977 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "server_addr", 2024-05-10T11:44:15.977 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.977 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.977 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.977 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "::", 2024-05-10T11:44:15.977 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.977 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.978 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.978 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-10T11:44:15.978 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.978 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.978 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.978 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.978 INFO:teuthology.orchestra.run.smithi100.stdout: "server_port": { 2024-05-10T11:44:15.978 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "server_port", 2024-05-10T11:44:15.978 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:15.978 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.978 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.978 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "9283", 2024-05-10T11:44:15.978 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.978 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.979 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.979 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-10T11:44:15.979 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.979 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.979 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.979 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.979 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:15.979 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:15.979 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:15.979 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:15.979 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.979 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:15.979 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.979 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.979 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.980 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.980 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.980 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.980 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.980 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.980 INFO:teuthology.orchestra.run.smithi100.stdout: "stale_cache_strategy": { 2024-05-10T11:44:15.980 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "stale_cache_strategy", 2024-05-10T11:44:15.980 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.980 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.980 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.980 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "log", 2024-05-10T11:44:15.980 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.980 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.980 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.980 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.981 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.981 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.981 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.981 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.981 INFO:teuthology.orchestra.run.smithi100.stdout: "standby_behaviour": { 2024-05-10T11:44:15.981 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "standby_behaviour", 2024-05-10T11:44:15.981 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.981 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.981 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.981 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "default", 2024-05-10T11:44:15.981 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.981 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.981 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:15.981 INFO:teuthology.orchestra.run.smithi100.stdout: "default", 2024-05-10T11:44:15.981 INFO:teuthology.orchestra.run.smithi100.stdout: "error" 2024-05-10T11:44:15.981 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:15.982 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.982 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.982 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.982 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.982 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.982 INFO:teuthology.orchestra.run.smithi100.stdout: "standby_error_status_code": { 2024-05-10T11:44:15.982 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "standby_error_status_code", 2024-05-10T11:44:15.982 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:15.982 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.982 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.982 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "500", 2024-05-10T11:44:15.982 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "400", 2024-05-10T11:44:15.982 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "599", 2024-05-10T11:44:15.982 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.982 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.983 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.983 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.983 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.983 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:15.983 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:15.983 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.983 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:15.983 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rbd_support", 2024-05-10T11:44:15.983 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:15.983 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:15.983 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:15.984 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:15.984 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:15.984 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.984 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.984 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.984 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:15.984 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.984 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.984 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:15.984 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:15.984 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:15.984 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:15.984 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:15.984 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:15.985 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:15.985 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:15.985 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.985 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.985 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.985 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.985 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.985 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:15.985 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:15.985 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.985 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.985 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.985 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:15.985 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.985 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.986 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.986 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.986 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.986 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.986 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.986 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.986 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:15.986 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:15.986 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.986 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.986 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.986 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:15.986 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.986 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.986 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:15.987 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:15.987 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:15.987 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:15.987 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:15.987 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:15.987 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:15.987 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:15.987 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.987 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.987 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.987 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.987 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.987 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:15.987 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:15.987 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.988 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.988 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.988 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:15.988 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.988 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.988 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.988 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.988 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.988 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.988 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.988 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.988 INFO:teuthology.orchestra.run.smithi100.stdout: "max_concurrent_snap_create": { 2024-05-10T11:44:15.988 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "max_concurrent_snap_create", 2024-05-10T11:44:15.988 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:15.988 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.989 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.989 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10", 2024-05-10T11:44:15.989 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.989 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.989 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.989 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.989 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.989 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.989 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.989 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.989 INFO:teuthology.orchestra.run.smithi100.stdout: "mirror_snapshot_schedule": { 2024-05-10T11:44:15.989 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "mirror_snapshot_schedule", 2024-05-10T11:44:15.989 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.989 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.990 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.990 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:15.990 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.990 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.990 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.990 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.990 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.990 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.990 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.990 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.990 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:15.990 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:15.990 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:15.990 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:15.990 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.991 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:15.991 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.991 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.991 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.991 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.991 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.991 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.991 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.991 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.991 INFO:teuthology.orchestra.run.smithi100.stdout: "trash_purge_schedule": { 2024-05-10T11:44:15.991 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "trash_purge_schedule", 2024-05-10T11:44:15.991 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.991 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.991 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.991 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:15.992 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.992 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.992 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.992 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.992 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.992 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.992 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.992 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:15.992 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:15.992 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.992 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:15.992 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "restful", 2024-05-10T11:44:15.992 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:15.992 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:15.992 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:15.992 INFO:teuthology.orchestra.run.smithi100.stdout: "enable_auth": { 2024-05-10T11:44:15.993 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "enable_auth", 2024-05-10T11:44:15.993 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.993 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.993 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.993 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:15.993 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.993 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.993 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.993 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.993 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.993 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.993 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.993 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.993 INFO:teuthology.orchestra.run.smithi100.stdout: "key_file": { 2024-05-10T11:44:15.993 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "key_file", 2024-05-10T11:44:15.994 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.994 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.994 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.994 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:15.994 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.994 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.994 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.994 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.994 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.994 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.994 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.994 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.994 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:15.994 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:15.995 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.995 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.995 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.995 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:15.995 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.995 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.995 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:15.995 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:15.995 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:15.995 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:15.995 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:15.995 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:15.995 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:15.995 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:15.995 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.995 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.996 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.996 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.996 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.996 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:15.996 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:15.996 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.996 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.996 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.996 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:15.996 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.996 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.996 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.996 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.996 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.997 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.997 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.997 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.997 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:15.997 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:15.997 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.997 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.997 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.997 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:15.997 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.997 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.997 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:15.997 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:15.997 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:15.997 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:15.998 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:15.998 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:15.998 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:15.998 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:15.998 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.998 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.998 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.998 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.998 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.998 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:15.998 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:15.998 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:15.998 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.998 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:15.998 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:15.999 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:15.999 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:15.999 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:15.999 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:15.999 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:15.999 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:15.999 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:15.999 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:15.999 INFO:teuthology.orchestra.run.smithi100.stdout: "server_addr": { 2024-05-10T11:44:15.999 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "server_addr", 2024-05-10T11:44:15.999 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:15.999 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:15.999 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:15.999 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:15.999 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.000 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.000 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.000 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.000 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.000 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.000 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.000 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.000 INFO:teuthology.orchestra.run.smithi100.stdout: "server_port": { 2024-05-10T11:44:16.000 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "server_port", 2024-05-10T11:44:16.000 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.000 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.000 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.000 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.000 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.000 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.001 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.001 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.001 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.001 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.001 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.001 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.001 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.001 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.001 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.001 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.001 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.001 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.001 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.001 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.001 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.002 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.002 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.002 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.002 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.002 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.002 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.002 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.002 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.002 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rgw", 2024-05-10T11:44:16.002 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.002 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.002 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.002 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.002 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.002 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.003 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.003 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.003 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.003 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.003 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.003 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.003 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.003 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.003 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.003 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.003 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.003 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.003 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.003 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.003 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.004 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.004 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.004 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.004 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.004 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.004 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.004 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.004 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.004 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.004 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.004 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.004 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.004 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.004 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.005 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.005 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.005 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.005 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.005 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.005 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.005 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.005 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.005 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.005 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.005 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.005 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.005 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.005 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.006 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.006 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.006 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.006 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.006 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.006 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.006 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.006 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.006 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.006 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.006 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.006 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.006 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.006 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.006 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.007 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.007 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.007 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.007 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.007 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.007 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.007 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.007 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.007 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.007 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.007 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.007 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.007 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.007 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.007 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.008 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.008 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.008 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.008 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.008 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.008 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.008 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.008 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.008 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.008 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.008 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.008 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rook", 2024-05-10T11:44:16.008 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.008 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.009 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.009 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.009 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.009 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.009 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.009 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.009 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.009 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.009 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.009 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.009 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.009 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.009 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.010 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.010 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.010 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.010 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.010 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.010 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.010 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.010 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.010 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.010 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.010 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.010 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.011 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.011 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.011 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.011 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.011 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.011 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.011 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.011 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.011 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.011 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.011 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.011 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.011 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.011 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.012 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.012 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.012 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.012 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.012 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.012 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.012 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.012 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.012 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.012 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.012 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.012 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.012 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.013 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.013 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.013 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.013 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.013 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.013 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.013 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.013 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.013 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.013 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.013 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.013 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.013 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.014 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.014 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.014 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.014 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.014 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.014 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.014 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.014 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.014 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.014 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.014 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.014 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.015 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.015 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.015 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.015 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.015 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.015 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.015 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.015 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.015 INFO:teuthology.orchestra.run.smithi100.stdout: "storage_class": { 2024-05-10T11:44:16.015 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "storage_class", 2024-05-10T11:44:16.015 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.015 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.015 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.015 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "local", 2024-05-10T11:44:16.016 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.016 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.016 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.016 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-10T11:44:16.016 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.016 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.016 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.016 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.016 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.016 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.016 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.016 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "selftest", 2024-05-10T11:44:16.016 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.017 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.017 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.017 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.017 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.017 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.017 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.017 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.017 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.017 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.017 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.017 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.017 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.017 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.017 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.017 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.018 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.018 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.018 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.018 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.018 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.018 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.018 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.018 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.018 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.018 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.018 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.018 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.018 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.018 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.019 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.019 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.019 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.019 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.019 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.019 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.019 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.019 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.019 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.019 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.019 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.019 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.019 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.019 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.019 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.019 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.020 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.020 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.020 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.020 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.020 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.020 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.020 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.020 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.020 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.020 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.020 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.020 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.020 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.020 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.020 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.021 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.021 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.021 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.021 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.021 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.021 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.021 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.021 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.021 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.021 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.021 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.021 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.021 INFO:teuthology.orchestra.run.smithi100.stdout: "roption1": { 2024-05-10T11:44:16.021 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "roption1", 2024-05-10T11:44:16.022 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.022 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.022 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.022 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.022 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.022 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.022 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.022 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.022 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.022 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.022 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.022 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.022 INFO:teuthology.orchestra.run.smithi100.stdout: "roption2": { 2024-05-10T11:44:16.022 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "roption2", 2024-05-10T11:44:16.023 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.023 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.023 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.023 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "xyz", 2024-05-10T11:44:16.023 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.023 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.023 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.023 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.023 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.023 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.023 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.023 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.023 INFO:teuthology.orchestra.run.smithi100.stdout: "rwoption1": { 2024-05-10T11:44:16.023 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rwoption1", 2024-05-10T11:44:16.023 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.024 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.024 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.024 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.024 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.024 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.024 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.024 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.024 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.024 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.024 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.024 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.024 INFO:teuthology.orchestra.run.smithi100.stdout: "rwoption2": { 2024-05-10T11:44:16.024 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rwoption2", 2024-05-10T11:44:16.024 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.025 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.025 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.025 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.025 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.025 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.025 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.025 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.025 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.025 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.025 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.025 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.025 INFO:teuthology.orchestra.run.smithi100.stdout: "rwoption3": { 2024-05-10T11:44:16.025 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rwoption3", 2024-05-10T11:44:16.025 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "float", 2024-05-10T11:44:16.026 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.026 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.026 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.026 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.026 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.026 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.026 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.026 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.026 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.026 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.026 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.026 INFO:teuthology.orchestra.run.smithi100.stdout: "rwoption4": { 2024-05-10T11:44:16.026 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rwoption4", 2024-05-10T11:44:16.027 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.027 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.027 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.027 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.027 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.027 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.027 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.027 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.027 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.027 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.027 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.027 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.027 INFO:teuthology.orchestra.run.smithi100.stdout: "rwoption5": { 2024-05-10T11:44:16.027 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rwoption5", 2024-05-10T11:44:16.028 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.028 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.028 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.028 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.028 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.028 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.028 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.028 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.028 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.028 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.028 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.028 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.028 INFO:teuthology.orchestra.run.smithi100.stdout: "rwoption6": { 2024-05-10T11:44:16.028 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rwoption6", 2024-05-10T11:44:16.029 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.029 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.029 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.029 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.029 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.029 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.029 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.029 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.029 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.029 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.029 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.029 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.029 INFO:teuthology.orchestra.run.smithi100.stdout: "rwoption7": { 2024-05-10T11:44:16.029 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rwoption7", 2024-05-10T11:44:16.029 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.029 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.030 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.030 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.030 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "1", 2024-05-10T11:44:16.030 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "42", 2024-05-10T11:44:16.030 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.030 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.030 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.030 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.030 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.030 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.030 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.030 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.030 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.030 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.031 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.031 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.031 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.031 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.031 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.031 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.031 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.031 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.031 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.031 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.031 INFO:teuthology.orchestra.run.smithi100.stdout: "testkey": { 2024-05-10T11:44:16.031 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "testkey", 2024-05-10T11:44:16.031 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.031 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.032 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.032 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.032 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.032 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.032 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.032 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.032 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.032 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.032 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.032 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.032 INFO:teuthology.orchestra.run.smithi100.stdout: "testlkey": { 2024-05-10T11:44:16.032 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "testlkey", 2024-05-10T11:44:16.032 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.032 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.033 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.033 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.033 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.033 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.033 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.033 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.033 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.033 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.033 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.033 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.033 INFO:teuthology.orchestra.run.smithi100.stdout: "testnewline": { 2024-05-10T11:44:16.033 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "testnewline", 2024-05-10T11:44:16.033 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.033 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.033 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.034 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.034 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.034 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.034 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.034 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.034 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.034 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.034 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.034 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.034 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.034 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.034 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.034 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smb", 2024-05-10T11:44:16.034 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.035 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.035 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.035 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.035 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.035 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.035 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.035 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.035 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.035 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.035 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.035 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.035 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.035 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.035 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.035 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.035 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.036 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.036 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.036 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.036 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.036 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.036 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.036 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.036 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.036 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.036 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.036 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.036 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.036 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.036 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.036 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.036 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.037 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.037 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.037 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.037 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.037 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.037 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.037 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.037 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.037 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.037 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.037 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.037 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.037 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.038 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.038 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.038 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.038 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.038 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.038 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.038 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.038 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.038 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.038 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.038 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.038 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.038 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.038 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.039 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.039 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.039 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.039 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.039 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.039 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.039 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.039 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.039 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.039 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.039 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.039 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.039 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.039 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.039 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.040 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.040 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.040 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.040 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.040 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.040 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.040 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.040 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.040 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.040 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.040 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.040 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.040 INFO:teuthology.orchestra.run.smithi100.stdout: "update_orchestration": { 2024-05-10T11:44:16.040 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "update_orchestration", 2024-05-10T11:44:16.040 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.041 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.041 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.041 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.041 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.041 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.041 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.041 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-10T11:44:16.041 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.041 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.041 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.041 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.041 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.041 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.041 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.042 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "snap_schedule", 2024-05-10T11:44:16.042 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.042 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.042 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.042 INFO:teuthology.orchestra.run.smithi100.stdout: "allow_m_granularity": { 2024-05-10T11:44:16.042 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "allow_m_granularity", 2024-05-10T11:44:16.042 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.042 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.042 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.042 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.042 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.042 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.042 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.042 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "allow minute scheduled snapshots", 2024-05-10T11:44:16.043 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.043 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.043 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.043 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.043 INFO:teuthology.orchestra.run.smithi100.stdout: "dump_on_update": { 2024-05-10T11:44:16.043 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "dump_on_update", 2024-05-10T11:44:16.043 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.043 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.043 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.043 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.043 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.043 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.043 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.043 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "dump database to debug log on update", 2024-05-10T11:44:16.044 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.044 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.044 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.044 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.044 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.044 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.044 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.044 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.044 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.044 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.044 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.044 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.044 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.044 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.045 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.045 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.045 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.045 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.045 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.045 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.045 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.045 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.045 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.045 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.045 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.045 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.045 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.045 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.045 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.046 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.046 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.046 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.046 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.046 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.046 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.046 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.046 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.046 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.046 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.046 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.046 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.046 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.046 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.046 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.047 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.047 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.047 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.047 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.047 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.047 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.047 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.047 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.047 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.047 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.047 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.047 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.047 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.047 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.048 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.048 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.048 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.048 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.048 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.048 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.048 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.048 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.048 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.048 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.048 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.048 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.048 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.048 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.048 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.048 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.049 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.049 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.049 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.049 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.049 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.049 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.049 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.049 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.049 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.049 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.049 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.049 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.049 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.049 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.049 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.050 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.050 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.050 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "stats", 2024-05-10T11:44:16.050 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.050 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.050 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.050 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.050 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.050 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.050 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.050 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.050 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.050 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.050 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.050 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.051 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.051 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.051 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.051 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.051 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.051 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.051 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.051 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.051 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.051 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.051 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.051 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.051 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.051 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.052 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.052 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.052 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.052 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.052 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.052 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.052 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.052 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.052 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.052 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.052 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.052 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.052 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.052 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.052 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.053 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.053 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.053 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.053 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.053 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.053 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.053 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.053 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.053 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.053 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.053 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.053 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.053 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.053 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.053 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.054 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.054 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.054 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.054 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.054 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.054 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.054 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.054 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.054 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.054 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.054 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.054 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.054 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.054 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.054 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.055 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.055 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.055 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.055 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.055 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.055 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.055 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.055 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.055 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.055 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.055 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.055 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.055 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.055 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.055 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.056 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.056 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.056 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.056 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.056 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "status", 2024-05-10T11:44:16.056 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.056 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.056 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.056 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.056 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.056 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.056 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.056 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.056 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.057 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.057 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.057 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.057 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.057 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.057 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.057 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.057 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.057 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.057 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.057 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.057 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.057 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.057 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.057 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.058 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.058 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.058 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.058 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.058 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.058 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.058 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.058 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.058 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.058 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.058 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.058 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.058 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.058 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.058 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.059 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.059 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.059 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.059 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.059 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.059 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.059 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.059 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.059 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.059 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.059 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.059 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.059 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.059 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.059 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.060 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.060 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.060 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.060 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.060 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.060 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.060 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.060 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.060 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.060 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.060 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.060 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.060 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.060 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.060 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.061 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.061 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.061 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.061 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.061 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.061 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.061 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.061 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.061 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.061 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.061 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.061 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.061 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.061 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.061 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.061 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.062 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.062 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.062 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.062 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.062 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.062 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "telegraf", 2024-05-10T11:44:16.062 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.062 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.062 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.062 INFO:teuthology.orchestra.run.smithi100.stdout: "address": { 2024-05-10T11:44:16.062 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "address", 2024-05-10T11:44:16.062 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.062 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.062 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.062 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-10T11:44:16.063 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.063 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.063 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.063 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.063 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.063 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.063 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.063 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.063 INFO:teuthology.orchestra.run.smithi100.stdout: "interval": { 2024-05-10T11:44:16.063 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "interval", 2024-05-10T11:44:16.063 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-05-10T11:44:16.063 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.063 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.063 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "15", 2024-05-10T11:44:16.064 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.064 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.064 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.064 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.064 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.064 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.064 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.064 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.064 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.064 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.064 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.064 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.064 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.065 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.065 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.065 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.065 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.065 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.065 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.065 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.065 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.065 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.065 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.065 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.065 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.065 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.065 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.066 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.066 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.066 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.066 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.066 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.066 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.066 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.066 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.066 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.066 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.066 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.066 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.066 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.066 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.066 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.067 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.067 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.067 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.067 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.067 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.067 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.067 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.067 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.067 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.067 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.067 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.067 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.067 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.067 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.067 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.068 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.068 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.068 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.068 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.068 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.068 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.068 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.068 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.068 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.068 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.068 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.068 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.068 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.068 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.068 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.069 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.069 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.069 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.069 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.069 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.069 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.069 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.069 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.069 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.069 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.069 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.069 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.069 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.069 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.070 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.070 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.070 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.070 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.070 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.070 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.070 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.070 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.070 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.070 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "telemetry", 2024-05-10T11:44:16.070 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.071 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.071 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.071 INFO:teuthology.orchestra.run.smithi100.stdout: "channel_basic": { 2024-05-10T11:44:16.071 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "channel_basic", 2024-05-10T11:44:16.071 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.071 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.071 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.071 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.071 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.071 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.071 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.071 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-10T11:44:16.071 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.071 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.071 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.072 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.072 INFO:teuthology.orchestra.run.smithi100.stdout: "channel_crash": { 2024-05-10T11:44:16.072 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "channel_crash", 2024-05-10T11:44:16.072 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.072 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.072 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.072 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.072 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.072 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.072 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.072 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-10T11:44:16.072 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.072 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.072 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.073 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.073 INFO:teuthology.orchestra.run.smithi100.stdout: "channel_device": { 2024-05-10T11:44:16.073 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "channel_device", 2024-05-10T11:44:16.073 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.073 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.073 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.073 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.073 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.073 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.073 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.073 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-10T11:44:16.073 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.073 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.073 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.074 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.074 INFO:teuthology.orchestra.run.smithi100.stdout: "channel_ident": { 2024-05-10T11:44:16.074 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "channel_ident", 2024-05-10T11:44:16.074 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.074 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.074 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.074 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.074 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.074 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.074 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.074 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-10T11:44:16.074 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.074 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.075 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.075 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.075 INFO:teuthology.orchestra.run.smithi100.stdout: "channel_perf": { 2024-05-10T11:44:16.075 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "channel_perf", 2024-05-10T11:44:16.075 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.075 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.075 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.075 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.075 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.075 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.075 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.075 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-10T11:44:16.075 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.075 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.075 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.076 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.076 INFO:teuthology.orchestra.run.smithi100.stdout: "contact": { 2024-05-10T11:44:16.076 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "contact", 2024-05-10T11:44:16.076 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.076 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.076 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.076 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.076 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.076 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.076 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.076 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.076 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.076 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.076 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.076 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.076 INFO:teuthology.orchestra.run.smithi100.stdout: "description": { 2024-05-10T11:44:16.077 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "description", 2024-05-10T11:44:16.077 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.077 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.077 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.077 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.077 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.077 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.077 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.077 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.077 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.077 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.077 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.077 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.078 INFO:teuthology.orchestra.run.smithi100.stdout: "device_url": { 2024-05-10T11:44:16.078 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "device_url", 2024-05-10T11:44:16.078 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.078 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.078 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.078 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-10T11:44:16.078 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.078 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.078 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.078 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.078 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.078 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.078 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.078 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.079 INFO:teuthology.orchestra.run.smithi100.stdout: "enabled": { 2024-05-10T11:44:16.079 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "enabled", 2024-05-10T11:44:16.079 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.079 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.079 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.079 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.079 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.079 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.079 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.079 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.079 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.079 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.079 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.079 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.079 INFO:teuthology.orchestra.run.smithi100.stdout: "interval": { 2024-05-10T11:44:16.080 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "interval", 2024-05-10T11:44:16.080 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.080 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.080 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.080 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "24", 2024-05-10T11:44:16.080 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "8", 2024-05-10T11:44:16.080 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.080 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.080 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.080 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.080 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.080 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.080 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.080 INFO:teuthology.orchestra.run.smithi100.stdout: "last_opt_revision": { 2024-05-10T11:44:16.080 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "last_opt_revision", 2024-05-10T11:44:16.081 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.081 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.081 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.081 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "1", 2024-05-10T11:44:16.081 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.081 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.081 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.081 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.081 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.081 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.081 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.081 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.081 INFO:teuthology.orchestra.run.smithi100.stdout: "leaderboard": { 2024-05-10T11:44:16.081 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "leaderboard", 2024-05-10T11:44:16.081 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.082 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.082 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.082 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.082 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.082 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.082 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.082 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.082 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.082 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.082 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.082 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.082 INFO:teuthology.orchestra.run.smithi100.stdout: "leaderboard_description": { 2024-05-10T11:44:16.082 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "leaderboard_description", 2024-05-10T11:44:16.082 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.083 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.083 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.083 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.083 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.083 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.083 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.083 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.083 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.083 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.083 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.083 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.083 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.083 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.083 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.083 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.084 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.084 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.084 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.084 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.084 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.084 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.084 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.084 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.084 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.084 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.084 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.084 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.084 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.084 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.084 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.085 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.085 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.085 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.085 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.085 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.085 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.085 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.085 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.085 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.085 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.085 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.085 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.085 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.085 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.085 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.085 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.086 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.086 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.086 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.086 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.086 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.086 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.086 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.086 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.086 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.086 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.086 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.086 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.086 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.086 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.086 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.087 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.087 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.087 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.087 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.087 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.087 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.087 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.087 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.087 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.087 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.087 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.087 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.087 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.087 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.087 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.088 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.088 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.088 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.088 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.088 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.088 INFO:teuthology.orchestra.run.smithi100.stdout: "organization": { 2024-05-10T11:44:16.088 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "organization", 2024-05-10T11:44:16.088 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.088 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.088 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.088 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.088 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.088 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.088 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.088 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.089 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.089 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.089 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.089 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.089 INFO:teuthology.orchestra.run.smithi100.stdout: "proxy": { 2024-05-10T11:44:16.089 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "proxy", 2024-05-10T11:44:16.089 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.089 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.089 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.089 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.089 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.089 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.089 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.089 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.089 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.089 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.090 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.090 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.090 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.090 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.090 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.090 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.090 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.090 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.090 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.090 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.090 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.090 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.090 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.090 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.091 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.091 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.091 INFO:teuthology.orchestra.run.smithi100.stdout: "url": { 2024-05-10T11:44:16.091 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "url", 2024-05-10T11:44:16.091 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.091 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.091 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.091 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-10T11:44:16.091 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.091 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.091 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.091 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.091 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.091 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.091 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.092 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.092 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.092 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.092 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.092 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "test_orchestrator", 2024-05-10T11:44:16.092 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.092 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.092 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.092 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.092 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.092 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.092 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.092 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.092 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.092 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.093 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.093 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.093 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.093 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.093 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.093 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.093 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.093 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.093 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.093 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.093 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.093 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.093 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.093 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.093 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.094 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.094 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.094 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.094 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.094 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.094 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.094 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.094 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.094 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.094 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.094 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.094 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.094 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.094 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.095 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.095 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.095 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.095 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.095 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.095 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.095 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.095 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.095 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.095 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.095 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.095 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.095 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.095 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.095 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.095 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.096 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.096 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.096 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.096 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.096 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.096 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.096 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.096 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.096 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.096 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.096 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.096 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.096 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.096 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.096 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.097 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.097 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.097 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.097 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.097 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.097 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.097 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.097 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.097 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.097 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.097 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.097 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.097 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.097 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.097 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.098 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.098 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.098 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.098 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.098 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.098 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "volumes", 2024-05-10T11:44:16.098 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.098 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.098 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.098 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.098 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.098 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.098 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.098 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.099 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.099 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.099 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.099 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.099 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.099 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.099 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.099 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.099 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.099 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.099 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.099 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.099 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.099 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.099 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.100 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.100 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.100 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.100 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.100 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.100 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.100 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.100 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.100 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.100 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.100 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.100 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.100 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.100 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.100 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.100 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.101 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.101 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.101 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.101 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.101 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.101 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.101 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.101 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.101 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.101 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.101 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.101 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.101 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.101 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.101 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.101 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.102 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.102 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.102 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.102 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.102 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.102 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.102 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.102 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.102 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.102 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.102 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.102 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.102 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.102 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.103 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.103 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.103 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.103 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.103 INFO:teuthology.orchestra.run.smithi100.stdout: "max_concurrent_clones": { 2024-05-10T11:44:16.103 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "max_concurrent_clones", 2024-05-10T11:44:16.103 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.103 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.103 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.103 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "4", 2024-05-10T11:44:16.103 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.103 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.103 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.104 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-10T11:44:16.104 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.104 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.104 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.104 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.104 INFO:teuthology.orchestra.run.smithi100.stdout: "periodic_async_work": { 2024-05-10T11:44:16.104 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "periodic_async_work", 2024-05-10T11:44:16.104 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.104 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.104 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.104 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.104 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.104 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.104 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.104 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Periodically check for async work", 2024-05-10T11:44:16.104 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.105 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.105 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.105 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.105 INFO:teuthology.orchestra.run.smithi100.stdout: "snapshot_clone_delay": { 2024-05-10T11:44:16.105 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "snapshot_clone_delay", 2024-05-10T11:44:16.105 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.105 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.105 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.105 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.105 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.105 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.105 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.105 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-10T11:44:16.105 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.106 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.106 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.106 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.106 INFO:teuthology.orchestra.run.smithi100.stdout: "snapshot_clone_no_wait": { 2024-05-10T11:44:16.106 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "snapshot_clone_no_wait", 2024-05-10T11:44:16.106 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.106 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.106 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.106 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.106 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.106 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.106 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.106 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-10T11:44:16.106 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.107 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.107 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.107 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.107 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.107 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.107 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.107 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.107 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.107 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.107 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.107 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.107 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.107 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.107 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.107 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.108 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.108 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.108 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.108 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.108 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.108 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "zabbix", 2024-05-10T11:44:16.108 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.108 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.108 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.108 INFO:teuthology.orchestra.run.smithi100.stdout: "discovery_interval": { 2024-05-10T11:44:16.108 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "discovery_interval", 2024-05-10T11:44:16.108 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "uint", 2024-05-10T11:44:16.108 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.108 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.108 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "100", 2024-05-10T11:44:16.109 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.109 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.109 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.109 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.109 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.109 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.109 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.109 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.109 INFO:teuthology.orchestra.run.smithi100.stdout: "identifier": { 2024-05-10T11:44:16.109 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "identifier", 2024-05-10T11:44:16.109 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.109 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.109 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.109 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.109 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.109 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.110 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.110 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.110 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.110 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.110 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.110 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.110 INFO:teuthology.orchestra.run.smithi100.stdout: "interval": { 2024-05-10T11:44:16.110 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "interval", 2024-05-10T11:44:16.110 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-05-10T11:44:16.110 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.110 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.110 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "60", 2024-05-10T11:44:16.110 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.110 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.110 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.111 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.111 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.111 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.111 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.111 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.111 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.111 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.111 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.111 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.111 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.111 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.111 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.111 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.111 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.111 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.111 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.112 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.112 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.112 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.112 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.112 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.112 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.112 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.112 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.112 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.112 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.112 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.112 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.112 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.112 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.112 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.113 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.113 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.113 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.113 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.113 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.113 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.113 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.113 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.113 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.113 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.113 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.113 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.113 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.113 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.113 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.114 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.114 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.114 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.114 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.114 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.114 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.114 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.114 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.114 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.114 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.114 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.114 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.114 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.114 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.115 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.115 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.115 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.115 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.115 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.115 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.115 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.115 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.115 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.115 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.115 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.115 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.115 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.115 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.115 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.116 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.116 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.116 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.116 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.116 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.116 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.116 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.116 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.116 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.116 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.116 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.116 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.116 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.116 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.117 INFO:teuthology.orchestra.run.smithi100.stdout: "zabbix_host": { 2024-05-10T11:44:16.117 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "zabbix_host", 2024-05-10T11:44:16.117 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.117 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.117 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.117 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.117 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.117 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.117 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.117 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.117 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.117 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.117 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.117 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.117 INFO:teuthology.orchestra.run.smithi100.stdout: "zabbix_port": { 2024-05-10T11:44:16.118 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "zabbix_port", 2024-05-10T11:44:16.118 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.118 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.118 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.118 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10051", 2024-05-10T11:44:16.118 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.118 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.118 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.118 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.118 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.118 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.118 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.118 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.118 INFO:teuthology.orchestra.run.smithi100.stdout: "zabbix_sender": { 2024-05-10T11:44:16.118 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "zabbix_sender", 2024-05-10T11:44:16.119 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.119 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.119 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.119 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-10T11:44:16.119 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.119 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.119 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.119 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.119 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.119 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.119 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.119 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.119 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.119 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.120 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-05-10T11:44:16.120 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.120 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.120 INFO:teuthology.orchestra.run.smithi100.stdout: "gid": 24116, 2024-05-10T11:44:16.120 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "y", 2024-05-10T11:44:16.120 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr_features": 4540701547738038271, 2024-05-10T11:44:16.120 INFO:teuthology.orchestra.run.smithi100.stdout: "available_modules": [ 2024-05-10T11:44:16.120 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.120 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "alerts", 2024-05-10T11:44:16.120 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.120 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.120 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.120 INFO:teuthology.orchestra.run.smithi100.stdout: "interval": { 2024-05-10T11:44:16.120 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "interval", 2024-05-10T11:44:16.120 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-05-10T11:44:16.121 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.121 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.121 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "60", 2024-05-10T11:44:16.121 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.121 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.121 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.121 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "How frequently to reexamine health status", 2024-05-10T11:44:16.121 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.121 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.121 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.121 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.121 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.121 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.121 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.121 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.122 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.122 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.122 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.122 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.122 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.122 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.122 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.122 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.122 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.122 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.122 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.122 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.122 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.122 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.123 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.123 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.123 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.123 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.123 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.123 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.123 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.123 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.123 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.123 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.123 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.123 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.123 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.123 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.123 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.124 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.124 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.124 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.124 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.124 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.124 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.124 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.124 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.124 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.124 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.124 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.124 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.124 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.124 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.124 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.125 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.125 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.125 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.125 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.125 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.125 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.125 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.125 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.125 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.125 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.125 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.125 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.125 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.125 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.125 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.126 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.126 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.126 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.126 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.126 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.126 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.126 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.126 INFO:teuthology.orchestra.run.smithi100.stdout: "smtp_destination": { 2024-05-10T11:44:16.126 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smtp_destination", 2024-05-10T11:44:16.126 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.126 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.126 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.126 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.126 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.127 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.127 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.127 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Email address to send alerts to", 2024-05-10T11:44:16.127 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.127 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.127 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.127 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.127 INFO:teuthology.orchestra.run.smithi100.stdout: "smtp_from_name": { 2024-05-10T11:44:16.127 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smtp_from_name", 2024-05-10T11:44:16.127 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.127 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.127 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.127 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "Ceph", 2024-05-10T11:44:16.127 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.127 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.128 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.128 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Email From: name", 2024-05-10T11:44:16.128 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.128 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.128 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.128 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.128 INFO:teuthology.orchestra.run.smithi100.stdout: "smtp_host": { 2024-05-10T11:44:16.128 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smtp_host", 2024-05-10T11:44:16.128 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.128 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.128 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.128 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.128 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.128 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.129 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.129 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "SMTP server", 2024-05-10T11:44:16.129 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.129 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.129 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.129 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.129 INFO:teuthology.orchestra.run.smithi100.stdout: "smtp_password": { 2024-05-10T11:44:16.129 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smtp_password", 2024-05-10T11:44:16.129 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.129 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.129 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.129 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.129 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.129 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.130 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.130 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Password to authenticate with", 2024-05-10T11:44:16.130 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.130 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.130 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.130 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.130 INFO:teuthology.orchestra.run.smithi100.stdout: "smtp_port": { 2024-05-10T11:44:16.130 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smtp_port", 2024-05-10T11:44:16.130 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.130 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.130 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.130 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "465", 2024-05-10T11:44:16.130 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.130 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.131 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.131 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "SMTP port", 2024-05-10T11:44:16.131 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.131 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.131 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.131 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.131 INFO:teuthology.orchestra.run.smithi100.stdout: "smtp_sender": { 2024-05-10T11:44:16.131 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smtp_sender", 2024-05-10T11:44:16.131 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.131 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.131 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.131 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.131 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.131 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.131 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.131 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "SMTP envelope sender", 2024-05-10T11:44:16.132 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.132 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.132 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.132 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.132 INFO:teuthology.orchestra.run.smithi100.stdout: "smtp_ssl": { 2024-05-10T11:44:16.132 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smtp_ssl", 2024-05-10T11:44:16.132 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.132 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.132 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.132 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.132 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.132 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.132 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.132 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-10T11:44:16.132 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.133 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.133 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.133 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.133 INFO:teuthology.orchestra.run.smithi100.stdout: "smtp_user": { 2024-05-10T11:44:16.133 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smtp_user", 2024-05-10T11:44:16.133 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.133 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.133 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.133 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.133 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.133 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.133 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.133 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "User to authenticate as", 2024-05-10T11:44:16.133 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.133 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.134 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.134 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.134 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.134 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.134 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.134 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.134 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.134 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.134 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.134 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.134 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.134 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.134 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.134 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.134 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.135 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.135 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.135 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.135 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.135 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "balancer", 2024-05-10T11:44:16.135 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.135 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.135 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.135 INFO:teuthology.orchestra.run.smithi100.stdout: "active": { 2024-05-10T11:44:16.135 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "active", 2024-05-10T11:44:16.135 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.135 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.135 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.135 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.135 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.136 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.136 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.136 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "automatically balance PGs across cluster", 2024-05-10T11:44:16.136 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.136 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.136 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.136 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.136 INFO:teuthology.orchestra.run.smithi100.stdout: "begin_time": { 2024-05-10T11:44:16.136 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "begin_time", 2024-05-10T11:44:16.136 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.136 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.136 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.136 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0000", 2024-05-10T11:44:16.136 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.136 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.137 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.137 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "beginning time of day to automatically balance", 2024-05-10T11:44:16.137 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-10T11:44:16.137 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.137 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.137 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.137 INFO:teuthology.orchestra.run.smithi100.stdout: "begin_weekday": { 2024-05-10T11:44:16.137 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "begin_weekday", 2024-05-10T11:44:16.137 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "uint", 2024-05-10T11:44:16.137 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.137 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.137 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.137 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "0", 2024-05-10T11:44:16.137 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "6", 2024-05-10T11:44:16.138 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.138 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-10T11:44:16.138 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-10T11:44:16.138 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.138 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.138 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.138 INFO:teuthology.orchestra.run.smithi100.stdout: "crush_compat_max_iterations": { 2024-05-10T11:44:16.138 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "crush_compat_max_iterations", 2024-05-10T11:44:16.138 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "uint", 2024-05-10T11:44:16.138 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.138 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.138 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "25", 2024-05-10T11:44:16.138 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "1", 2024-05-10T11:44:16.138 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "250", 2024-05-10T11:44:16.139 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.139 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-10T11:44:16.139 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.139 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.139 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.139 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.139 INFO:teuthology.orchestra.run.smithi100.stdout: "crush_compat_metrics": { 2024-05-10T11:44:16.139 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "crush_compat_metrics", 2024-05-10T11:44:16.139 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.139 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.139 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.139 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "pgs,objects,bytes", 2024-05-10T11:44:16.139 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.139 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.139 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.140 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-10T11:44:16.140 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-10T11:44:16.140 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.140 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.140 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.140 INFO:teuthology.orchestra.run.smithi100.stdout: "crush_compat_step": { 2024-05-10T11:44:16.140 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "crush_compat_step", 2024-05-10T11:44:16.140 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "float", 2024-05-10T11:44:16.140 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.140 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.140 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0.5", 2024-05-10T11:44:16.140 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "0.001", 2024-05-10T11:44:16.140 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "0.999", 2024-05-10T11:44:16.141 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.141 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "aggressiveness of optimization", 2024-05-10T11:44:16.141 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-10T11:44:16.141 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.141 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.141 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.141 INFO:teuthology.orchestra.run.smithi100.stdout: "end_time": { 2024-05-10T11:44:16.141 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "end_time", 2024-05-10T11:44:16.141 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.141 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.141 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.141 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "2359", 2024-05-10T11:44:16.141 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.141 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.142 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.142 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "ending time of day to automatically balance", 2024-05-10T11:44:16.142 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-10T11:44:16.142 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.142 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.142 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.142 INFO:teuthology.orchestra.run.smithi100.stdout: "end_weekday": { 2024-05-10T11:44:16.142 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "end_weekday", 2024-05-10T11:44:16.142 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "uint", 2024-05-10T11:44:16.142 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.142 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.142 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.142 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "0", 2024-05-10T11:44:16.143 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "6", 2024-05-10T11:44:16.143 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.143 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-10T11:44:16.143 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-10T11:44:16.143 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.143 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.143 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.143 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.143 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.143 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.143 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.143 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.143 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.143 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.143 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.144 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.144 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.144 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.144 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.144 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.144 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.144 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.144 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.144 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.144 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.144 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.144 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.144 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.144 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.144 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.145 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.145 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.145 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.145 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.145 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.145 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.145 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.145 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.145 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.145 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.145 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.145 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.145 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.145 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.146 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.146 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.146 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.146 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.146 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.146 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.146 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.146 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.146 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.146 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.146 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.146 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.146 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.146 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.146 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.147 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.147 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.147 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.147 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.147 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.147 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.147 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.147 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.147 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.147 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.147 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.147 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.147 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.147 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.147 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.148 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.148 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.148 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.148 INFO:teuthology.orchestra.run.smithi100.stdout: "min_score": { 2024-05-10T11:44:16.148 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "min_score", 2024-05-10T11:44:16.148 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "float", 2024-05-10T11:44:16.148 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.148 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.148 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.148 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.148 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.148 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.148 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-10T11:44:16.148 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.148 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.149 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.149 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.149 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": { 2024-05-10T11:44:16.149 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "mode", 2024-05-10T11:44:16.149 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.149 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.149 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.149 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "upmap", 2024-05-10T11:44:16.149 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.149 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.149 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.149 INFO:teuthology.orchestra.run.smithi100.stdout: "crush-compat", 2024-05-10T11:44:16.149 INFO:teuthology.orchestra.run.smithi100.stdout: "none", 2024-05-10T11:44:16.149 INFO:teuthology.orchestra.run.smithi100.stdout: "read", 2024-05-10T11:44:16.149 INFO:teuthology.orchestra.run.smithi100.stdout: "upmap", 2024-05-10T11:44:16.150 INFO:teuthology.orchestra.run.smithi100.stdout: "upmap-read" 2024-05-10T11:44:16.150 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.150 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Balancer mode", 2024-05-10T11:44:16.150 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.150 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.150 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.150 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.150 INFO:teuthology.orchestra.run.smithi100.stdout: "pool_ids": { 2024-05-10T11:44:16.150 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "pool_ids", 2024-05-10T11:44:16.150 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.150 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.151 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.151 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.151 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.151 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.151 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.151 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-10T11:44:16.151 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.151 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.151 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.151 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.151 INFO:teuthology.orchestra.run.smithi100.stdout: "sleep_interval": { 2024-05-10T11:44:16.151 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sleep_interval", 2024-05-10T11:44:16.151 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-05-10T11:44:16.151 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.151 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.152 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "60", 2024-05-10T11:44:16.152 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.152 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.152 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.152 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-10T11:44:16.152 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.152 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.152 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.152 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.152 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.152 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.152 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.152 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.152 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.152 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.153 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.153 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.153 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.153 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.153 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.153 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.153 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.153 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.153 INFO:teuthology.orchestra.run.smithi100.stdout: "upmap_max_deviation": { 2024-05-10T11:44:16.153 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "upmap_max_deviation", 2024-05-10T11:44:16.153 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.153 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.153 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.153 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "5", 2024-05-10T11:44:16.153 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "1", 2024-05-10T11:44:16.154 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.154 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.154 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-10T11:44:16.154 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-10T11:44:16.154 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.154 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.154 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.154 INFO:teuthology.orchestra.run.smithi100.stdout: "upmap_max_optimizations": { 2024-05-10T11:44:16.154 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "upmap_max_optimizations", 2024-05-10T11:44:16.154 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "uint", 2024-05-10T11:44:16.154 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.154 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.154 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10", 2024-05-10T11:44:16.154 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.155 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.155 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.155 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-10T11:44:16.155 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.155 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.155 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.155 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.155 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.155 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.155 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.155 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "cephadm", 2024-05-10T11:44:16.155 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.155 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.155 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.156 INFO:teuthology.orchestra.run.smithi100.stdout: "agent_down_multiplier": { 2024-05-10T11:44:16.156 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "agent_down_multiplier", 2024-05-10T11:44:16.156 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "float", 2024-05-10T11:44:16.156 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.156 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.156 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "3.0", 2024-05-10T11:44:16.156 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.156 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.156 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.156 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-10T11:44:16.156 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.156 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.156 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.156 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.156 INFO:teuthology.orchestra.run.smithi100.stdout: "agent_refresh_rate": { 2024-05-10T11:44:16.157 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "agent_refresh_rate", 2024-05-10T11:44:16.157 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-05-10T11:44:16.157 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.157 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.157 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "20", 2024-05-10T11:44:16.157 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.157 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.157 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.157 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-10T11:44:16.157 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.157 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.157 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.157 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.157 INFO:teuthology.orchestra.run.smithi100.stdout: "agent_starting_port": { 2024-05-10T11:44:16.158 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "agent_starting_port", 2024-05-10T11:44:16.158 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.158 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.158 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.158 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "4721", 2024-05-10T11:44:16.158 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.158 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.158 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.158 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-10T11:44:16.158 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.158 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.158 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.158 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.158 INFO:teuthology.orchestra.run.smithi100.stdout: "allow_ptrace": { 2024-05-10T11:44:16.159 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "allow_ptrace", 2024-05-10T11:44:16.159 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.159 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.159 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.159 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.159 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.159 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.159 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.159 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-10T11:44:16.159 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-10T11:44:16.159 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.159 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.159 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.159 INFO:teuthology.orchestra.run.smithi100.stdout: "autotune_interval": { 2024-05-10T11:44:16.159 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "autotune_interval", 2024-05-10T11:44:16.160 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-05-10T11:44:16.160 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.160 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.160 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "600", 2024-05-10T11:44:16.160 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.160 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.160 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.160 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-10T11:44:16.160 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.160 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.160 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.160 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.160 INFO:teuthology.orchestra.run.smithi100.stdout: "autotune_memory_target_ratio": { 2024-05-10T11:44:16.160 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "autotune_memory_target_ratio", 2024-05-10T11:44:16.160 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "float", 2024-05-10T11:44:16.161 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.161 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.161 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0.7", 2024-05-10T11:44:16.161 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.161 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.161 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.161 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-10T11:44:16.161 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.161 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.161 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.161 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.161 INFO:teuthology.orchestra.run.smithi100.stdout: "cephadm_log_destination": { 2024-05-10T11:44:16.161 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "cephadm_log_destination", 2024-05-10T11:44:16.161 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.161 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.162 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.162 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.162 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.162 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.162 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.162 INFO:teuthology.orchestra.run.smithi100.stdout: "file", 2024-05-10T11:44:16.162 INFO:teuthology.orchestra.run.smithi100.stdout: "file,syslog", 2024-05-10T11:44:16.162 INFO:teuthology.orchestra.run.smithi100.stdout: "syslog" 2024-05-10T11:44:16.162 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.162 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-10T11:44:16.162 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.162 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.162 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.162 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.163 INFO:teuthology.orchestra.run.smithi100.stdout: "cgroups_split": { 2024-05-10T11:44:16.163 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "cgroups_split", 2024-05-10T11:44:16.163 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.163 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.163 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.163 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.163 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.163 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.163 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.163 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-10T11:44:16.163 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.163 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.163 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.164 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.164 INFO:teuthology.orchestra.run.smithi100.stdout: "config_checks_enabled": { 2024-05-10T11:44:16.164 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "config_checks_enabled", 2024-05-10T11:44:16.164 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.164 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.164 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.164 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.164 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.164 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.164 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.164 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-10T11:44:16.164 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.164 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.164 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.164 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.165 INFO:teuthology.orchestra.run.smithi100.stdout: "config_dashboard": { 2024-05-10T11:44:16.165 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "config_dashboard", 2024-05-10T11:44:16.165 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.165 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.165 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.165 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.165 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.165 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.165 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.165 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-10T11:44:16.165 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.165 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.165 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.165 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.165 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_alertmanager": { 2024-05-10T11:44:16.166 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_alertmanager", 2024-05-10T11:44:16.166 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.166 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.166 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.166 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-10T11:44:16.166 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.166 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.166 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.166 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Prometheus container image", 2024-05-10T11:44:16.166 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.166 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.166 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.166 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.167 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_base": { 2024-05-10T11:44:16.167 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_base", 2024-05-10T11:44:16.167 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.167 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.167 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.167 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-10T11:44:16.167 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.167 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.167 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.167 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Container image name, without the tag", 2024-05-10T11:44:16.167 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.167 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.167 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.167 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.167 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_elasticsearch": { 2024-05-10T11:44:16.168 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_elasticsearch", 2024-05-10T11:44:16.168 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.168 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.168 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.168 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-10T11:44:16.168 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.168 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.168 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.168 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "elasticsearch container image", 2024-05-10T11:44:16.168 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.168 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.168 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.168 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.168 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_grafana": { 2024-05-10T11:44:16.168 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_grafana", 2024-05-10T11:44:16.169 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.169 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.169 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.169 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-10T11:44:16.169 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.169 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.169 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.169 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Prometheus container image", 2024-05-10T11:44:16.169 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.169 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.169 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.169 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.169 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_haproxy": { 2024-05-10T11:44:16.169 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_haproxy", 2024-05-10T11:44:16.169 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.170 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.170 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.170 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-10T11:44:16.170 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.170 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.170 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.170 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "HAproxy container image", 2024-05-10T11:44:16.170 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.170 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.170 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.170 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.170 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_jaeger_agent": { 2024-05-10T11:44:16.170 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_jaeger_agent", 2024-05-10T11:44:16.170 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.171 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.171 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.171 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-10T11:44:16.171 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.171 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.171 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.171 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Jaeger agent container image", 2024-05-10T11:44:16.171 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.171 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.171 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.171 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.171 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_jaeger_collector": { 2024-05-10T11:44:16.171 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_jaeger_collector", 2024-05-10T11:44:16.171 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.171 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.172 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.172 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-10T11:44:16.172 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.172 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.172 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.172 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Jaeger collector container image", 2024-05-10T11:44:16.172 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.172 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.172 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.172 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.172 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_jaeger_query": { 2024-05-10T11:44:16.172 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_jaeger_query", 2024-05-10T11:44:16.172 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.172 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.172 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.172 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-10T11:44:16.172 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.173 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.173 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.173 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Jaeger query container image", 2024-05-10T11:44:16.173 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.173 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.173 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.173 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.173 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_keepalived": { 2024-05-10T11:44:16.173 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_keepalived", 2024-05-10T11:44:16.173 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.173 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.173 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.173 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-10T11:44:16.173 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.174 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.174 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.174 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Keepalived container image", 2024-05-10T11:44:16.174 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.174 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.174 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.174 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.174 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_loki": { 2024-05-10T11:44:16.174 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_loki", 2024-05-10T11:44:16.174 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.174 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.174 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.174 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-10T11:44:16.175 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.175 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.175 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.175 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Loki container image", 2024-05-10T11:44:16.175 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.175 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.175 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.175 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.175 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_node_exporter": { 2024-05-10T11:44:16.175 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_node_exporter", 2024-05-10T11:44:16.175 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.175 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.175 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.175 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-10T11:44:16.175 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.176 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.176 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.176 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Prometheus container image", 2024-05-10T11:44:16.176 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.176 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.176 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.176 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.176 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_nvmeof": { 2024-05-10T11:44:16.176 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_nvmeof", 2024-05-10T11:44:16.176 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.176 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.176 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.176 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-10T11:44:16.176 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.176 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.177 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.177 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Nvme-of container image", 2024-05-10T11:44:16.177 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.177 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.177 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.177 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.177 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_prometheus": { 2024-05-10T11:44:16.177 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_prometheus", 2024-05-10T11:44:16.177 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.177 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.177 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.177 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-10T11:44:16.177 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.178 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.178 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.178 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Prometheus container image", 2024-05-10T11:44:16.178 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.178 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.178 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.178 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.178 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_promtail": { 2024-05-10T11:44:16.178 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_promtail", 2024-05-10T11:44:16.178 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.178 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.178 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.178 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-10T11:44:16.179 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.179 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.179 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.179 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Promtail container image", 2024-05-10T11:44:16.179 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.179 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.179 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.179 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.179 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_samba": { 2024-05-10T11:44:16.179 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_samba", 2024-05-10T11:44:16.179 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.179 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.179 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.179 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-10T11:44:16.179 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.180 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.180 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.180 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Samba/SMB container image", 2024-05-10T11:44:16.180 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.180 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.180 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.180 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.180 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_snmp_gateway": { 2024-05-10T11:44:16.180 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_snmp_gateway", 2024-05-10T11:44:16.180 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.180 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.180 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.180 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-10T11:44:16.180 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.180 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.181 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.181 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "SNMP Gateway container image", 2024-05-10T11:44:16.181 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.181 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.181 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.181 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.181 INFO:teuthology.orchestra.run.smithi100.stdout: "container_init": { 2024-05-10T11:44:16.181 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_init", 2024-05-10T11:44:16.181 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.181 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.181 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.181 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.181 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.181 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.181 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.182 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Run podman/docker with `--init`", 2024-05-10T11:44:16.182 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.182 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.182 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.182 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.182 INFO:teuthology.orchestra.run.smithi100.stdout: "daemon_cache_timeout": { 2024-05-10T11:44:16.182 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "daemon_cache_timeout", 2024-05-10T11:44:16.182 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-05-10T11:44:16.182 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.182 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.182 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "600", 2024-05-10T11:44:16.182 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.182 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.183 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.183 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-10T11:44:16.183 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.183 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.183 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.183 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.183 INFO:teuthology.orchestra.run.smithi100.stdout: "default_cephadm_command_timeout": { 2024-05-10T11:44:16.183 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "default_cephadm_command_timeout", 2024-05-10T11:44:16.183 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.183 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.183 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.183 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "900", 2024-05-10T11:44:16.183 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.183 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.184 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.184 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-10T11:44:16.184 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.184 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.184 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.184 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.184 INFO:teuthology.orchestra.run.smithi100.stdout: "default_registry": { 2024-05-10T11:44:16.184 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "default_registry", 2024-05-10T11:44:16.184 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.184 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.184 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.184 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "docker.io", 2024-05-10T11:44:16.184 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.184 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.184 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.185 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-10T11:44:16.185 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.185 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.185 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.185 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.185 INFO:teuthology.orchestra.run.smithi100.stdout: "device_cache_timeout": { 2024-05-10T11:44:16.185 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "device_cache_timeout", 2024-05-10T11:44:16.185 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-05-10T11:44:16.185 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.185 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.185 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "1800", 2024-05-10T11:44:16.185 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.185 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.185 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.185 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "seconds to cache device inventory", 2024-05-10T11:44:16.186 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.186 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.186 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.186 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.186 INFO:teuthology.orchestra.run.smithi100.stdout: "device_enhanced_scan": { 2024-05-10T11:44:16.186 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "device_enhanced_scan", 2024-05-10T11:44:16.186 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.186 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.186 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.186 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.186 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.186 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.186 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.186 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-10T11:44:16.187 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.187 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.187 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.187 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.187 INFO:teuthology.orchestra.run.smithi100.stdout: "facts_cache_timeout": { 2024-05-10T11:44:16.187 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "facts_cache_timeout", 2024-05-10T11:44:16.187 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-05-10T11:44:16.187 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.187 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.187 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "60", 2024-05-10T11:44:16.187 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.187 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.187 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.187 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "seconds to cache host facts data", 2024-05-10T11:44:16.188 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.188 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.188 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.188 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.188 INFO:teuthology.orchestra.run.smithi100.stdout: "grafana_dashboards_path": { 2024-05-10T11:44:16.188 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "grafana_dashboards_path", 2024-05-10T11:44:16.188 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.188 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.188 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.188 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-10T11:44:16.188 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.188 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.188 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.188 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-10T11:44:16.188 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.188 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.189 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.189 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.189 INFO:teuthology.orchestra.run.smithi100.stdout: "host_check_interval": { 2024-05-10T11:44:16.189 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "host_check_interval", 2024-05-10T11:44:16.189 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-05-10T11:44:16.189 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.189 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.189 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "600", 2024-05-10T11:44:16.189 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.189 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.189 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.189 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "how frequently to perform a host check", 2024-05-10T11:44:16.189 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.189 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.190 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.190 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.190 INFO:teuthology.orchestra.run.smithi100.stdout: "hw_monitoring": { 2024-05-10T11:44:16.190 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "hw_monitoring", 2024-05-10T11:44:16.190 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.190 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.190 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.190 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.190 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.190 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.190 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.190 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-10T11:44:16.190 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.190 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.191 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.191 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.191 INFO:teuthology.orchestra.run.smithi100.stdout: "inventory_list_all": { 2024-05-10T11:44:16.191 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "inventory_list_all", 2024-05-10T11:44:16.191 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.191 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.191 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.191 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.191 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.191 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.191 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.191 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-10T11:44:16.191 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.191 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.191 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.192 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.192 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.192 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.192 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.192 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.192 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.192 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.192 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.192 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.192 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.192 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.192 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.192 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.192 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.193 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.193 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.193 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.193 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.193 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.193 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.193 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.193 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.193 INFO:teuthology.orchestra.run.smithi100.stdout: "log_refresh_metadata": { 2024-05-10T11:44:16.193 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_refresh_metadata", 2024-05-10T11:44:16.193 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.193 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.193 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.193 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.194 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.194 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.194 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.194 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-10T11:44:16.194 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.194 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.194 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.194 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.194 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.194 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.194 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.194 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.194 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.194 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.195 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.195 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.195 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.195 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-10T11:44:16.195 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.195 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.195 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.195 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.195 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.195 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.195 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.195 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.195 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.196 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.196 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.196 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.196 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.196 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.196 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.196 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.196 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.196 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.196 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.196 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.196 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.196 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.197 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.197 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.197 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.197 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.197 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.197 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.197 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.197 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.197 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.197 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.197 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.197 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.197 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.197 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.197 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.198 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.198 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.198 INFO:teuthology.orchestra.run.smithi100.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-10T11:44:16.198 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-10T11:44:16.198 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.198 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.198 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.198 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.198 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.198 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.198 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.198 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-10T11:44:16.198 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.198 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.199 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.199 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.199 INFO:teuthology.orchestra.run.smithi100.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-10T11:44:16.199 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-10T11:44:16.199 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.199 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.199 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.199 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "*", 2024-05-10T11:44:16.199 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.199 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.199 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.199 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-10T11:44:16.199 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.199 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.199 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.200 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.200 INFO:teuthology.orchestra.run.smithi100.stdout: "max_count_per_host": { 2024-05-10T11:44:16.200 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "max_count_per_host", 2024-05-10T11:44:16.200 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.200 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.200 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.200 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10", 2024-05-10T11:44:16.200 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.200 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.200 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.200 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "max number of daemons per service per host", 2024-05-10T11:44:16.200 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.200 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.200 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.200 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.201 INFO:teuthology.orchestra.run.smithi100.stdout: "max_osd_draining_count": { 2024-05-10T11:44:16.201 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "max_osd_draining_count", 2024-05-10T11:44:16.201 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.201 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.201 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.201 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10", 2024-05-10T11:44:16.201 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.201 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.201 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.201 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-10T11:44:16.201 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.201 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.201 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.201 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.202 INFO:teuthology.orchestra.run.smithi100.stdout: "migration_current": { 2024-05-10T11:44:16.202 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "migration_current", 2024-05-10T11:44:16.202 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.202 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.202 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.202 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.202 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.202 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.202 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.202 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "internal - do not modify", 2024-05-10T11:44:16.202 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.202 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.202 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.202 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.202 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": { 2024-05-10T11:44:16.203 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "mode", 2024-05-10T11:44:16.203 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.203 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.203 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.203 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "root", 2024-05-10T11:44:16.203 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.203 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.203 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.203 INFO:teuthology.orchestra.run.smithi100.stdout: "cephadm-package", 2024-05-10T11:44:16.203 INFO:teuthology.orchestra.run.smithi100.stdout: "root" 2024-05-10T11:44:16.203 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.203 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "mode for remote execution of cephadm", 2024-05-10T11:44:16.203 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.203 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.204 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.204 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.204 INFO:teuthology.orchestra.run.smithi100.stdout: "oob_default_addr": { 2024-05-10T11:44:16.204 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "oob_default_addr", 2024-05-10T11:44:16.204 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.204 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.204 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.204 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "169.254.1.1", 2024-05-10T11:44:16.204 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.204 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.204 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.204 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-10T11:44:16.204 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.204 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.204 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.205 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.205 INFO:teuthology.orchestra.run.smithi100.stdout: "prometheus_alerts_path": { 2024-05-10T11:44:16.205 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "prometheus_alerts_path", 2024-05-10T11:44:16.205 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.205 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.205 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.205 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-10T11:44:16.205 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.205 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.205 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.205 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-10T11:44:16.205 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.205 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.205 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.206 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.206 INFO:teuthology.orchestra.run.smithi100.stdout: "registry_insecure": { 2024-05-10T11:44:16.206 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "registry_insecure", 2024-05-10T11:44:16.206 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.206 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.206 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.206 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.206 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.206 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.206 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.206 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-10T11:44:16.206 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.206 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.207 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.207 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.207 INFO:teuthology.orchestra.run.smithi100.stdout: "registry_password": { 2024-05-10T11:44:16.207 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "registry_password", 2024-05-10T11:44:16.207 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.207 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.207 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.207 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.207 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.207 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.207 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.207 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-10T11:44:16.207 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.207 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.207 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.208 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.208 INFO:teuthology.orchestra.run.smithi100.stdout: "registry_url": { 2024-05-10T11:44:16.208 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "registry_url", 2024-05-10T11:44:16.208 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.208 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.208 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.208 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.208 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.208 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.208 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.208 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-10T11:44:16.208 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.208 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.208 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.208 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.209 INFO:teuthology.orchestra.run.smithi100.stdout: "registry_username": { 2024-05-10T11:44:16.209 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "registry_username", 2024-05-10T11:44:16.209 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.209 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.209 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.209 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.209 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.209 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.209 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.209 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-10T11:44:16.209 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.209 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.209 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.209 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.209 INFO:teuthology.orchestra.run.smithi100.stdout: "secure_monitoring_stack": { 2024-05-10T11:44:16.210 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "secure_monitoring_stack", 2024-05-10T11:44:16.210 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.210 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.210 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.210 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.210 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.210 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.210 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.210 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-10T11:44:16.210 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.210 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.210 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.210 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.210 INFO:teuthology.orchestra.run.smithi100.stdout: "service_discovery_port": { 2024-05-10T11:44:16.211 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "service_discovery_port", 2024-05-10T11:44:16.211 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.211 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.211 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.211 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "8765", 2024-05-10T11:44:16.211 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.211 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.211 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.211 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "cephadm service discovery port", 2024-05-10T11:44:16.211 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.211 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.211 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.211 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.211 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.211 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.212 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.212 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.212 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.212 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.212 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.212 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.212 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.212 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.212 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.212 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.212 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.212 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.212 INFO:teuthology.orchestra.run.smithi100.stdout: "ssh_config_file": { 2024-05-10T11:44:16.213 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ssh_config_file", 2024-05-10T11:44:16.213 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.213 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.213 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.213 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.213 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.213 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.213 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.213 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-10T11:44:16.213 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.213 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.213 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.213 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.213 INFO:teuthology.orchestra.run.smithi100.stdout: "use_agent": { 2024-05-10T11:44:16.213 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "use_agent", 2024-05-10T11:44:16.214 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.214 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.214 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.214 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.214 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.214 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.214 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.214 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-10T11:44:16.214 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.214 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.214 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.214 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.214 INFO:teuthology.orchestra.run.smithi100.stdout: "use_repo_digest": { 2024-05-10T11:44:16.214 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "use_repo_digest", 2024-05-10T11:44:16.215 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.215 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.215 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.215 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.215 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.215 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.215 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.215 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-10T11:44:16.215 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.215 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.215 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.215 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.215 INFO:teuthology.orchestra.run.smithi100.stdout: "warn_on_failed_host_check": { 2024-05-10T11:44:16.216 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "warn_on_failed_host_check", 2024-05-10T11:44:16.216 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.216 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.216 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.216 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.216 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.216 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.216 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.216 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "raise a health warning if the host check fails", 2024-05-10T11:44:16.216 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.216 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.216 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.216 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.216 INFO:teuthology.orchestra.run.smithi100.stdout: "warn_on_stray_daemons": { 2024-05-10T11:44:16.217 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "warn_on_stray_daemons", 2024-05-10T11:44:16.217 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.217 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.217 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.217 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.217 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.217 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.217 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.217 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-10T11:44:16.217 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.217 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.217 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.217 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.217 INFO:teuthology.orchestra.run.smithi100.stdout: "warn_on_stray_hosts": { 2024-05-10T11:44:16.218 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "warn_on_stray_hosts", 2024-05-10T11:44:16.218 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.218 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.218 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.218 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.218 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.218 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.218 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.218 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-10T11:44:16.218 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.218 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.218 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.218 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.218 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.219 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.219 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.219 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "crash", 2024-05-10T11:44:16.219 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.219 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.219 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.219 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.219 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.219 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.219 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.219 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.219 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.219 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.219 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.219 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.220 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.220 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.220 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.220 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.220 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.220 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.220 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.220 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.220 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.220 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.220 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.220 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.220 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.220 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.221 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.221 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.221 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.221 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.221 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.221 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.221 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.221 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.221 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.221 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.221 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.221 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.221 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.221 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.221 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.222 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.222 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.222 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.222 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.222 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.222 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.222 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.222 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.222 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.222 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.222 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.222 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.222 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.223 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.223 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.223 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.223 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.223 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.223 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.223 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.223 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.223 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.223 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.223 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.223 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.223 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.223 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.223 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.224 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.224 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.224 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.224 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.224 INFO:teuthology.orchestra.run.smithi100.stdout: "retain_interval": { 2024-05-10T11:44:16.224 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "retain_interval", 2024-05-10T11:44:16.224 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-05-10T11:44:16.224 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.224 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.224 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "31536000", 2024-05-10T11:44:16.224 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.224 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.224 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.224 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-10T11:44:16.224 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.225 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.225 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.225 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.225 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.225 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.225 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.225 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.225 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.225 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.225 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.225 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.225 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.225 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.225 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.225 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.225 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.226 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.226 INFO:teuthology.orchestra.run.smithi100.stdout: "warn_recent_interval": { 2024-05-10T11:44:16.226 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "warn_recent_interval", 2024-05-10T11:44:16.226 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-05-10T11:44:16.226 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.226 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.226 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "1209600", 2024-05-10T11:44:16.226 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.226 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.226 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.226 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-10T11:44:16.226 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.226 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.227 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.227 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.227 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.227 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.227 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.227 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "dashboard", 2024-05-10T11:44:16.227 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.227 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.227 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.227 INFO:teuthology.orchestra.run.smithi100.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-10T11:44:16.227 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-10T11:44:16.227 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.227 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.227 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.227 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10", 2024-05-10T11:44:16.228 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.228 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.228 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.228 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.228 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.228 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.228 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.228 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.228 INFO:teuthology.orchestra.run.smithi100.stdout: "ALERTMANAGER_API_HOST": { 2024-05-10T11:44:16.228 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-10T11:44:16.228 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.228 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.228 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.228 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.228 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.229 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.229 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.229 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.229 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.229 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.229 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.229 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.229 INFO:teuthology.orchestra.run.smithi100.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-10T11:44:16.229 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-10T11:44:16.229 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.229 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.229 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.229 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.229 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.229 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.230 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.230 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.230 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.230 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.230 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.230 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.230 INFO:teuthology.orchestra.run.smithi100.stdout: "AUDIT_API_ENABLED": { 2024-05-10T11:44:16.230 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "AUDIT_API_ENABLED", 2024-05-10T11:44:16.230 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.230 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.230 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.230 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.230 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.230 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.230 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.231 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.231 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.231 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.231 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.231 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.231 INFO:teuthology.orchestra.run.smithi100.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-10T11:44:16.231 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-10T11:44:16.231 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.231 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.231 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.231 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.231 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.231 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.231 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.231 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.232 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.232 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.232 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.232 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.232 INFO:teuthology.orchestra.run.smithi100.stdout: "ENABLE_BROWSABLE_API": { 2024-05-10T11:44:16.232 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-10T11:44:16.232 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.232 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.232 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.232 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.232 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.232 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.232 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.232 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.233 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.233 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.233 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.233 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.233 INFO:teuthology.orchestra.run.smithi100.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-10T11:44:16.233 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-10T11:44:16.233 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.233 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.233 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.233 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.233 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.233 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.233 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.233 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.233 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.234 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.234 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.234 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.234 INFO:teuthology.orchestra.run.smithi100.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-10T11:44:16.234 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-10T11:44:16.234 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.234 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.234 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.234 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.234 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.234 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.234 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.234 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.234 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.235 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.235 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.235 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.235 INFO:teuthology.orchestra.run.smithi100.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-10T11:44:16.235 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-10T11:44:16.235 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.235 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.235 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.235 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.235 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.235 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.235 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.235 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.235 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.235 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.235 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.236 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.236 INFO:teuthology.orchestra.run.smithi100.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-10T11:44:16.236 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-10T11:44:16.236 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.236 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.236 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.236 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.236 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.236 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.236 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.236 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.236 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.236 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.236 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.236 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.237 INFO:teuthology.orchestra.run.smithi100.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-10T11:44:16.237 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-10T11:44:16.237 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.237 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.237 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.237 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.237 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.237 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.237 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.237 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.237 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.237 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.237 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.237 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.237 INFO:teuthology.orchestra.run.smithi100.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-10T11:44:16.237 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-10T11:44:16.238 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.238 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.238 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.238 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.238 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.238 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.238 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.238 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.238 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.238 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.238 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.238 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.238 INFO:teuthology.orchestra.run.smithi100.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-10T11:44:16.238 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-10T11:44:16.239 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.239 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.239 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.239 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.239 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.239 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.239 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.239 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.239 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.239 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.239 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.239 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.239 INFO:teuthology.orchestra.run.smithi100.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-10T11:44:16.239 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-10T11:44:16.240 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.240 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.240 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.240 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.240 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.240 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.240 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.240 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.240 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.240 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.240 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.240 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.240 INFO:teuthology.orchestra.run.smithi100.stdout: "GRAFANA_API_PASSWORD": { 2024-05-10T11:44:16.240 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-10T11:44:16.240 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.241 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.241 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.241 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "admin", 2024-05-10T11:44:16.241 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.241 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.241 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.241 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.241 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.241 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.241 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.241 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.241 INFO:teuthology.orchestra.run.smithi100.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-10T11:44:16.241 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-10T11:44:16.241 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.241 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.242 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.242 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.242 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.242 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.242 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.242 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.242 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.242 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.242 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.242 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.242 INFO:teuthology.orchestra.run.smithi100.stdout: "GRAFANA_API_URL": { 2024-05-10T11:44:16.242 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "GRAFANA_API_URL", 2024-05-10T11:44:16.242 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.242 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.243 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.243 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.243 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.243 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.243 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.243 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.243 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.243 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.243 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.243 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.243 INFO:teuthology.orchestra.run.smithi100.stdout: "GRAFANA_API_USERNAME": { 2024-05-10T11:44:16.243 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-10T11:44:16.243 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.243 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.243 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.244 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "admin", 2024-05-10T11:44:16.244 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.244 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.244 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.244 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.244 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.244 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.244 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.244 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.244 INFO:teuthology.orchestra.run.smithi100.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-10T11:44:16.244 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-10T11:44:16.244 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.244 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.244 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.244 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.245 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.245 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.245 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.245 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.245 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.245 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.245 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.245 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.245 INFO:teuthology.orchestra.run.smithi100.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-10T11:44:16.245 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-10T11:44:16.245 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.245 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.245 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.245 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.245 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.246 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.246 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.246 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.246 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.246 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.246 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.246 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.246 INFO:teuthology.orchestra.run.smithi100.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-10T11:44:16.246 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-10T11:44:16.246 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.246 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.246 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.246 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.246 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.247 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.247 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.247 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.247 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.247 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.247 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.247 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.247 INFO:teuthology.orchestra.run.smithi100.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-10T11:44:16.247 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-10T11:44:16.247 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.247 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.247 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.247 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.247 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.247 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.248 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.248 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.248 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.248 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.248 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.248 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.248 INFO:teuthology.orchestra.run.smithi100.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-10T11:44:16.248 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-10T11:44:16.248 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.248 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.248 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.248 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "[]", 2024-05-10T11:44:16.248 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.248 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.248 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.249 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.249 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.249 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.249 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.249 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.249 INFO:teuthology.orchestra.run.smithi100.stdout: "MULTICLUSTER_CONFIG": { 2024-05-10T11:44:16.249 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-10T11:44:16.249 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.249 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.249 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.249 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "{}", 2024-05-10T11:44:16.249 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.249 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.249 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.249 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.250 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.250 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.250 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.250 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.250 INFO:teuthology.orchestra.run.smithi100.stdout: "PROMETHEUS_API_HOST": { 2024-05-10T11:44:16.250 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-10T11:44:16.250 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.250 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.250 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.250 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.250 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.250 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.250 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.251 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.251 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.251 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.251 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.251 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.251 INFO:teuthology.orchestra.run.smithi100.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-10T11:44:16.251 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-10T11:44:16.251 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.251 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.251 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.251 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.251 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.251 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.251 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.251 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.252 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.252 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.252 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.252 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.252 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-10T11:44:16.252 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-10T11:44:16.252 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.252 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.252 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.252 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.252 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.252 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.252 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.252 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.252 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.253 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.253 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.253 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.253 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-10T11:44:16.253 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-10T11:44:16.253 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.253 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.253 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.253 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.253 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.253 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.253 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.253 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.253 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.253 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.254 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.254 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.254 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-10T11:44:16.254 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-10T11:44:16.254 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.254 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.254 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.254 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.254 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.254 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.254 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.254 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.254 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.254 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.255 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.255 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.255 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-10T11:44:16.255 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-10T11:44:16.255 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.255 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.255 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.255 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.255 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.255 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.255 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.255 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.255 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.255 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.255 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.255 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.256 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-10T11:44:16.256 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-10T11:44:16.256 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.256 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.256 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.256 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.256 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.256 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.256 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.256 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.256 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.256 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.256 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.256 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.257 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-10T11:44:16.257 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-10T11:44:16.257 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.257 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.257 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.257 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.257 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.257 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.257 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.257 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.257 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.257 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.257 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.257 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.257 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-10T11:44:16.258 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-10T11:44:16.258 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.258 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.258 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.258 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.258 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.258 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.258 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.258 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.258 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.258 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.258 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.259 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.259 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_ENABLED": { 2024-05-10T11:44:16.259 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-10T11:44:16.259 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.259 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.259 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.259 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.259 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.259 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.259 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.259 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.259 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.259 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.259 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.260 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.260 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-10T11:44:16.260 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-10T11:44:16.260 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.260 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.260 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.260 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-10T11:44:16.260 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.260 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.260 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.260 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.260 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.260 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.260 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.260 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.261 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-10T11:44:16.261 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-10T11:44:16.261 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.261 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.261 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.261 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10", 2024-05-10T11:44:16.261 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.261 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.261 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.261 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.261 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.261 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.261 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.261 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.261 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-10T11:44:16.262 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-10T11:44:16.262 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.262 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.262 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.262 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "8", 2024-05-10T11:44:16.262 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.262 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.262 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.262 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.262 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.262 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.262 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.262 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.263 INFO:teuthology.orchestra.run.smithi100.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-10T11:44:16.263 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-10T11:44:16.263 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.263 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.263 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.263 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "45", 2024-05-10T11:44:16.263 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.263 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.263 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.263 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.263 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.263 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.263 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.263 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.263 INFO:teuthology.orchestra.run.smithi100.stdout: "RGW_API_ACCESS_KEY": { 2024-05-10T11:44:16.264 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-10T11:44:16.264 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.264 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.264 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.264 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.264 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.264 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.264 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.264 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.264 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.264 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.264 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.264 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.264 INFO:teuthology.orchestra.run.smithi100.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-10T11:44:16.265 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-10T11:44:16.265 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.265 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.265 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.265 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "admin", 2024-05-10T11:44:16.265 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.265 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.265 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.265 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.265 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.265 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.265 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.265 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.265 INFO:teuthology.orchestra.run.smithi100.stdout: "RGW_API_SECRET_KEY": { 2024-05-10T11:44:16.266 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-10T11:44:16.266 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.266 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.266 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.266 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.266 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.266 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.266 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.266 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.266 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.266 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.266 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.266 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.266 INFO:teuthology.orchestra.run.smithi100.stdout: "RGW_API_SSL_VERIFY": { 2024-05-10T11:44:16.267 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-10T11:44:16.267 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.267 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.267 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.267 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.267 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.267 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.267 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.267 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.267 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.267 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.267 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.267 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.267 INFO:teuthology.orchestra.run.smithi100.stdout: "UNSAFE_TLS_v1_2": { 2024-05-10T11:44:16.267 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-10T11:44:16.268 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.268 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.268 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.268 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.268 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.268 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.268 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.268 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.268 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.268 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.268 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.268 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.268 INFO:teuthology.orchestra.run.smithi100.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-10T11:44:16.268 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-10T11:44:16.268 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.269 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.269 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.269 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.269 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.269 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.269 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.269 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.269 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.269 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.269 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.269 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.269 INFO:teuthology.orchestra.run.smithi100.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-10T11:44:16.269 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-10T11:44:16.269 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.269 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.270 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.270 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10", 2024-05-10T11:44:16.270 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.270 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.270 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.270 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.270 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.270 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.270 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.270 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.270 INFO:teuthology.orchestra.run.smithi100.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-10T11:44:16.270 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-10T11:44:16.270 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.270 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.270 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.271 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "5", 2024-05-10T11:44:16.271 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.271 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.271 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.271 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.271 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.271 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.271 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.271 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.271 INFO:teuthology.orchestra.run.smithi100.stdout: "cross_origin_url": { 2024-05-10T11:44:16.271 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "cross_origin_url", 2024-05-10T11:44:16.271 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.271 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.271 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.271 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.272 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.272 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.272 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.272 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.272 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.272 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.272 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.272 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.272 INFO:teuthology.orchestra.run.smithi100.stdout: "crt_file": { 2024-05-10T11:44:16.272 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "crt_file", 2024-05-10T11:44:16.272 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.272 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.272 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.272 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.272 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.273 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.273 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.273 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.273 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.273 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.273 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.273 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.273 INFO:teuthology.orchestra.run.smithi100.stdout: "debug": { 2024-05-10T11:44:16.273 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "debug", 2024-05-10T11:44:16.273 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.273 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.273 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.273 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.273 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.273 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.274 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.274 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Enable/disable debug options", 2024-05-10T11:44:16.274 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.274 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.274 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.274 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.274 INFO:teuthology.orchestra.run.smithi100.stdout: "jwt_token_ttl": { 2024-05-10T11:44:16.274 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "jwt_token_ttl", 2024-05-10T11:44:16.274 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.274 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.274 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.274 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "28800", 2024-05-10T11:44:16.274 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.274 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.275 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.275 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.275 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.275 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.275 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.275 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.275 INFO:teuthology.orchestra.run.smithi100.stdout: "key_file": { 2024-05-10T11:44:16.275 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "key_file", 2024-05-10T11:44:16.275 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.275 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.275 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.275 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.275 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.275 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.275 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.276 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.276 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.276 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.276 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.276 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.276 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.276 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.276 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.276 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.276 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.276 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.276 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.276 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.276 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.276 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.277 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.277 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.277 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.277 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.277 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.277 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.277 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.277 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.277 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.277 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.277 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.277 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.278 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.278 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.278 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.278 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.278 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.278 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.278 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.278 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.278 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.278 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.278 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.278 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.278 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.278 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.278 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.279 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.279 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.279 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.279 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.279 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.279 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.279 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.279 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.279 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.279 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.279 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.279 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.279 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.279 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.280 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.280 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.280 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.280 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.280 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.280 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.280 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.280 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.280 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.280 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.280 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.280 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.280 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.281 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.281 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.281 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.281 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.281 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.281 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.281 INFO:teuthology.orchestra.run.smithi100.stdout: "motd": { 2024-05-10T11:44:16.281 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "motd", 2024-05-10T11:44:16.281 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.281 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.281 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.281 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.281 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.281 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.281 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.282 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "The message of the day", 2024-05-10T11:44:16.282 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.282 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.282 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.282 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.282 INFO:teuthology.orchestra.run.smithi100.stdout: "redirect_resolve_ip_addr": { 2024-05-10T11:44:16.282 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "redirect_resolve_ip_addr", 2024-05-10T11:44:16.282 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.282 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.282 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.282 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.282 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.283 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.283 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.283 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.283 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.283 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.283 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.283 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.283 INFO:teuthology.orchestra.run.smithi100.stdout: "server_addr": { 2024-05-10T11:44:16.283 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "server_addr", 2024-05-10T11:44:16.283 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.283 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.283 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.283 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "::", 2024-05-10T11:44:16.283 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.284 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.284 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.284 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.284 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.284 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.284 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.284 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.284 INFO:teuthology.orchestra.run.smithi100.stdout: "server_port": { 2024-05-10T11:44:16.284 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "server_port", 2024-05-10T11:44:16.284 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.284 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.284 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.284 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "8080", 2024-05-10T11:44:16.284 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.285 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.285 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.285 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.285 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.285 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.285 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.285 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.285 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.285 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.285 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.285 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.285 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.285 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.285 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.285 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.286 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.286 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.286 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.286 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.286 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.286 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.286 INFO:teuthology.orchestra.run.smithi100.stdout: "ssl": { 2024-05-10T11:44:16.286 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ssl", 2024-05-10T11:44:16.286 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.286 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.286 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.286 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.287 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.287 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.287 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.287 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.287 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.287 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.287 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.287 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.287 INFO:teuthology.orchestra.run.smithi100.stdout: "ssl_server_port": { 2024-05-10T11:44:16.287 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ssl_server_port", 2024-05-10T11:44:16.287 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.287 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.287 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.288 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "8443", 2024-05-10T11:44:16.288 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.288 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.288 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.288 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.288 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.288 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.288 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.288 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.288 INFO:teuthology.orchestra.run.smithi100.stdout: "standby_behaviour": { 2024-05-10T11:44:16.288 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "standby_behaviour", 2024-05-10T11:44:16.288 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.288 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.289 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.289 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "redirect", 2024-05-10T11:44:16.289 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.289 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.289 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.289 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.289 INFO:teuthology.orchestra.run.smithi100.stdout: "redirect" 2024-05-10T11:44:16.289 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.289 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.289 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.289 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.289 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.289 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.289 INFO:teuthology.orchestra.run.smithi100.stdout: "standby_error_status_code": { 2024-05-10T11:44:16.289 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "standby_error_status_code", 2024-05-10T11:44:16.290 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.290 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.290 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.290 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "500", 2024-05-10T11:44:16.290 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "400", 2024-05-10T11:44:16.290 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "599", 2024-05-10T11:44:16.290 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.290 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.290 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.290 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.290 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.290 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.290 INFO:teuthology.orchestra.run.smithi100.stdout: "url_prefix": { 2024-05-10T11:44:16.290 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "url_prefix", 2024-05-10T11:44:16.291 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.291 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.291 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.291 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.291 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.291 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.291 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.291 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.291 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.291 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.291 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.291 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.291 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.291 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.292 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.292 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "devicehealth", 2024-05-10T11:44:16.292 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.292 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.292 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.292 INFO:teuthology.orchestra.run.smithi100.stdout: "enable_monitoring": { 2024-05-10T11:44:16.292 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "enable_monitoring", 2024-05-10T11:44:16.292 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.292 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.292 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.292 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.292 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.292 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.292 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.293 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "monitor device health metrics", 2024-05-10T11:44:16.293 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.293 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.293 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.293 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.293 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.293 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.293 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.293 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.293 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.293 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.293 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.293 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.293 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.294 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.294 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.294 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.294 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.294 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.294 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.294 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.294 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.294 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.294 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.294 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.295 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.295 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.295 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.295 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.295 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.295 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.295 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.295 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.295 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.295 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.295 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.295 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.295 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.295 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.295 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.296 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.296 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.296 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.296 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.296 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.296 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.296 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.296 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.296 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.296 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.296 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.296 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.296 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.296 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.297 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.297 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.297 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.297 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.297 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.297 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.297 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.297 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.297 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.297 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.297 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.297 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.297 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.297 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.297 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.298 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.298 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.298 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.298 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.298 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.298 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.298 INFO:teuthology.orchestra.run.smithi100.stdout: "mark_out_threshold": { 2024-05-10T11:44:16.298 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "mark_out_threshold", 2024-05-10T11:44:16.298 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-05-10T11:44:16.298 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.298 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.298 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "2419200", 2024-05-10T11:44:16.298 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.298 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.299 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.299 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-10T11:44:16.299 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.299 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.299 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.299 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.299 INFO:teuthology.orchestra.run.smithi100.stdout: "pool_name": { 2024-05-10T11:44:16.299 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "pool_name", 2024-05-10T11:44:16.299 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.299 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.299 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.299 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "device_health_metrics", 2024-05-10T11:44:16.299 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.300 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.300 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.300 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-10T11:44:16.300 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.300 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.300 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.300 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.300 INFO:teuthology.orchestra.run.smithi100.stdout: "retention_period": { 2024-05-10T11:44:16.300 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "retention_period", 2024-05-10T11:44:16.300 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-05-10T11:44:16.300 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.300 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.300 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "15552000", 2024-05-10T11:44:16.300 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.301 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.301 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.301 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "how long to retain device health metrics", 2024-05-10T11:44:16.301 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.301 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.301 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.301 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.301 INFO:teuthology.orchestra.run.smithi100.stdout: "scrape_frequency": { 2024-05-10T11:44:16.301 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "scrape_frequency", 2024-05-10T11:44:16.301 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-05-10T11:44:16.301 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.301 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.301 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "86400", 2024-05-10T11:44:16.301 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.301 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.302 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.302 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-10T11:44:16.302 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.302 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.302 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.302 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.302 INFO:teuthology.orchestra.run.smithi100.stdout: "self_heal": { 2024-05-10T11:44:16.302 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "self_heal", 2024-05-10T11:44:16.302 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.302 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.302 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.302 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.302 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.302 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.302 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.303 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-10T11:44:16.303 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.303 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.303 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.303 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.303 INFO:teuthology.orchestra.run.smithi100.stdout: "sleep_interval": { 2024-05-10T11:44:16.303 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sleep_interval", 2024-05-10T11:44:16.303 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-05-10T11:44:16.303 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.303 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.303 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "600", 2024-05-10T11:44:16.303 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.303 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.303 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.304 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "how frequently to wake up and check device health", 2024-05-10T11:44:16.304 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.304 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.304 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.304 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.304 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.304 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.304 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.304 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.304 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.304 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.305 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.305 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.305 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.305 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.305 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.305 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.305 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.305 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.305 INFO:teuthology.orchestra.run.smithi100.stdout: "warn_threshold": { 2024-05-10T11:44:16.305 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "warn_threshold", 2024-05-10T11:44:16.305 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-05-10T11:44:16.305 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.305 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.305 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "7257600", 2024-05-10T11:44:16.305 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.306 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.306 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.306 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-10T11:44:16.306 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.306 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.306 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.306 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.306 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.306 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.306 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.306 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "diskprediction_local", 2024-05-10T11:44:16.306 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.306 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.306 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.307 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.307 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.307 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.307 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.307 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.307 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.307 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.307 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.307 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.307 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.307 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.307 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.307 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.307 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.307 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.308 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.308 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.308 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.308 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.308 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.308 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.308 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.308 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.308 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.308 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.308 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.308 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.308 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.308 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.308 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.308 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.309 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.309 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.309 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.309 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.309 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.309 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.309 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.309 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.309 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.309 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.309 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.309 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.309 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.309 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.309 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.310 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.310 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.310 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.310 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.310 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.310 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.310 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.310 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.310 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.310 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.310 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.310 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.310 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.310 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.311 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.311 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.311 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.311 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.311 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.311 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.311 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.311 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.311 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.311 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.311 INFO:teuthology.orchestra.run.smithi100.stdout: "predict_interval": { 2024-05-10T11:44:16.311 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "predict_interval", 2024-05-10T11:44:16.311 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.311 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.311 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.312 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "86400", 2024-05-10T11:44:16.312 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.312 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.312 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.312 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.312 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.312 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.312 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.312 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.312 INFO:teuthology.orchestra.run.smithi100.stdout: "predictor_model": { 2024-05-10T11:44:16.312 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "predictor_model", 2024-05-10T11:44:16.312 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.312 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.312 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.312 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "prophetstor", 2024-05-10T11:44:16.313 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.313 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.313 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.313 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.313 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.313 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.313 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.313 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.313 INFO:teuthology.orchestra.run.smithi100.stdout: "sleep_interval": { 2024-05-10T11:44:16.313 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sleep_interval", 2024-05-10T11:44:16.313 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.313 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.313 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.313 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "600", 2024-05-10T11:44:16.313 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.314 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.314 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.314 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.314 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.314 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.314 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.314 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.314 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.314 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.314 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.314 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.314 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.314 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.314 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.315 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.315 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.315 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.315 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.315 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.315 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.315 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.315 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.315 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.315 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.315 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "influx", 2024-05-10T11:44:16.315 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": false, 2024-05-10T11:44:16.315 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "influxdb python module not found", 2024-05-10T11:44:16.315 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.315 INFO:teuthology.orchestra.run.smithi100.stdout: "batch_size": { 2024-05-10T11:44:16.316 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "batch_size", 2024-05-10T11:44:16.316 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.316 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.316 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.316 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "5000", 2024-05-10T11:44:16.316 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.316 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.316 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.316 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-10T11:44:16.316 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.316 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.316 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.316 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.316 INFO:teuthology.orchestra.run.smithi100.stdout: "database": { 2024-05-10T11:44:16.316 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "database", 2024-05-10T11:44:16.317 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.317 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.317 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.317 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "ceph", 2024-05-10T11:44:16.317 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.317 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.317 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.317 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-10T11:44:16.317 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.317 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.317 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.317 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.317 INFO:teuthology.orchestra.run.smithi100.stdout: "hostname": { 2024-05-10T11:44:16.317 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "hostname", 2024-05-10T11:44:16.318 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.318 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.318 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.318 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.318 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.318 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.318 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.318 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "InfluxDB server hostname", 2024-05-10T11:44:16.318 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.318 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.318 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.318 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.318 INFO:teuthology.orchestra.run.smithi100.stdout: "interval": { 2024-05-10T11:44:16.318 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "interval", 2024-05-10T11:44:16.319 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-05-10T11:44:16.319 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.319 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.319 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "30", 2024-05-10T11:44:16.319 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "5", 2024-05-10T11:44:16.319 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.319 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.319 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-10T11:44:16.319 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.319 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.319 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.319 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.319 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.319 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.320 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.320 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.320 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.320 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.320 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.320 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.320 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.320 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.320 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.320 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.320 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.320 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.320 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.320 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.320 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.321 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.321 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.321 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.321 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.321 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.321 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.321 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.321 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.321 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.321 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.321 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.321 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.321 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.321 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.321 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.322 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.322 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.322 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.322 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.322 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.322 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.322 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.322 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.322 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.322 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.322 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.322 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.322 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.322 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.323 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.323 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.323 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.323 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.323 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.323 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.323 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.323 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.323 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.323 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.323 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.323 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.323 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.323 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.323 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.324 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.324 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.324 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.324 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.324 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.324 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.324 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.324 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.324 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.324 INFO:teuthology.orchestra.run.smithi100.stdout: "password": { 2024-05-10T11:44:16.324 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "password", 2024-05-10T11:44:16.324 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.324 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.324 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.324 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.325 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.325 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.325 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.325 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "password of InfluxDB server user", 2024-05-10T11:44:16.325 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.325 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.325 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.325 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.325 INFO:teuthology.orchestra.run.smithi100.stdout: "port": { 2024-05-10T11:44:16.325 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "port", 2024-05-10T11:44:16.325 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.325 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.325 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.325 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "8086", 2024-05-10T11:44:16.325 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.326 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.326 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.326 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "InfluxDB server port", 2024-05-10T11:44:16.326 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.326 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.326 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.326 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.326 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.326 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.326 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.326 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.327 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.327 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.327 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.327 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.327 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.327 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.327 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.327 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.327 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.327 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.327 INFO:teuthology.orchestra.run.smithi100.stdout: "ssl": { 2024-05-10T11:44:16.327 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ssl", 2024-05-10T11:44:16.327 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.327 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.328 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.328 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "false", 2024-05-10T11:44:16.328 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.328 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.328 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.328 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-10T11:44:16.328 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.328 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.328 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.328 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.328 INFO:teuthology.orchestra.run.smithi100.stdout: "threads": { 2024-05-10T11:44:16.328 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "threads", 2024-05-10T11:44:16.328 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.328 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.328 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.329 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "5", 2024-05-10T11:44:16.329 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "1", 2024-05-10T11:44:16.329 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "32", 2024-05-10T11:44:16.329 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.329 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-10T11:44:16.329 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.329 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.329 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.329 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.329 INFO:teuthology.orchestra.run.smithi100.stdout: "username": { 2024-05-10T11:44:16.329 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "username", 2024-05-10T11:44:16.329 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.329 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.329 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.329 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.330 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.330 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.330 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.330 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "username of InfluxDB server user", 2024-05-10T11:44:16.330 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.330 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.330 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.330 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.330 INFO:teuthology.orchestra.run.smithi100.stdout: "verify_ssl": { 2024-05-10T11:44:16.330 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "verify_ssl", 2024-05-10T11:44:16.330 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.330 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.330 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.330 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "true", 2024-05-10T11:44:16.331 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.331 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.331 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.331 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-10T11:44:16.331 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.331 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.331 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.331 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.331 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.331 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.331 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.331 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "insights", 2024-05-10T11:44:16.331 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.331 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.332 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.332 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.332 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.332 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.332 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.332 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.332 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.332 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.332 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.332 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.332 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.332 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.332 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.332 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.332 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.333 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.333 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.333 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.333 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.333 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.333 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.333 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.333 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.333 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.333 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.333 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.333 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.333 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.333 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.334 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.334 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.334 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.334 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.334 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.334 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.334 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.334 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.334 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.334 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.334 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.334 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.334 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.334 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.335 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.335 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.335 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.335 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.335 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.335 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.335 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.335 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.335 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.335 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.335 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.335 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.335 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.335 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.335 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.336 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.336 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.336 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.336 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.336 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.336 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.336 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.336 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.336 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.336 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.336 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.336 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.336 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.336 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.337 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.337 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.337 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.337 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.337 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.337 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.337 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.337 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.337 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.337 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.337 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.337 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.337 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.337 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.337 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.338 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.338 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "iostat", 2024-05-10T11:44:16.338 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.338 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.338 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.338 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.338 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.338 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.338 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.338 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.338 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.338 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.338 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.339 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.339 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.339 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.339 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.339 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.339 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.339 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.339 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.339 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.339 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.339 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.339 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.339 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.339 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.340 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.340 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.340 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.340 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.340 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.340 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.340 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.340 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.340 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.340 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.340 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.340 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.340 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.340 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.340 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.340 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.341 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.341 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.341 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.341 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.341 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.341 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.341 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.341 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.341 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.341 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.341 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.341 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.341 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.341 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.342 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.342 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.342 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.342 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.342 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.342 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.342 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.342 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.342 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.342 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.342 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.342 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.342 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.342 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.342 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.343 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.343 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.343 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.343 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.343 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.343 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.343 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.343 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.343 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.343 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.343 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.343 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.343 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.343 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.343 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.344 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.344 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.344 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.344 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.344 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.344 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "k8sevents", 2024-05-10T11:44:16.344 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.344 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.344 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.344 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_event_retention_days": { 2024-05-10T11:44:16.344 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ceph_event_retention_days", 2024-05-10T11:44:16.344 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.344 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.344 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.345 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "7", 2024-05-10T11:44:16.345 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.345 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.345 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.345 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-10T11:44:16.345 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.345 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.345 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.345 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.345 INFO:teuthology.orchestra.run.smithi100.stdout: "config_check_secs": { 2024-05-10T11:44:16.345 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "config_check_secs", 2024-05-10T11:44:16.345 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.345 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.345 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.345 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10", 2024-05-10T11:44:16.346 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "10", 2024-05-10T11:44:16.346 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.346 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.346 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-10T11:44:16.346 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.346 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.346 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.346 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.346 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.346 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.346 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.346 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.346 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.346 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.346 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.347 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.347 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.347 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.347 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.347 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.347 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.347 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.347 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.347 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.347 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.347 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.347 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.347 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.347 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.347 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.348 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.348 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.348 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.348 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.348 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.348 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.348 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.348 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.348 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.348 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.348 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.348 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.348 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.348 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.349 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.349 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.349 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.349 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.349 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.349 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.349 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.349 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.349 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.349 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.349 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.349 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.349 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.349 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.349 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.350 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.350 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.350 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.350 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.350 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.350 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.350 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.350 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.350 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.350 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.350 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.350 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.350 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.350 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.350 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.351 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.351 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.351 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.351 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.351 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.351 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.351 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.351 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.351 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.351 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.351 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.351 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.351 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.351 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.351 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.352 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.352 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.352 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.352 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.352 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.352 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.352 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "localpool", 2024-05-10T11:44:16.352 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.352 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.352 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.352 INFO:teuthology.orchestra.run.smithi100.stdout: "failure_domain": { 2024-05-10T11:44:16.352 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "failure_domain", 2024-05-10T11:44:16.352 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.352 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.352 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.353 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "host", 2024-05-10T11:44:16.353 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.353 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.353 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.353 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "failure domain for any created local pool", 2024-05-10T11:44:16.353 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-10T11:44:16.353 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.353 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.353 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.353 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.353 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.353 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.353 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.353 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.353 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.353 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.354 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.354 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.354 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.354 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.354 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.354 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.354 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.354 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.354 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.354 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.354 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.354 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.354 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.354 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.354 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.355 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.355 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.355 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.355 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.355 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.355 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.355 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.355 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.355 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.355 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.355 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.355 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.355 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.355 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.355 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.355 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.356 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.356 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.356 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.356 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.356 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.356 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.356 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.356 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.356 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.356 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.356 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.356 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.356 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.356 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.357 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.357 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.357 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.357 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.357 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.357 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.357 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.357 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.357 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.357 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.357 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.357 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.357 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.357 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.358 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.358 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.358 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.358 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.358 INFO:teuthology.orchestra.run.smithi100.stdout: "min_size": { 2024-05-10T11:44:16.358 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "min_size", 2024-05-10T11:44:16.358 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.358 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.358 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.358 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.358 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.358 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.358 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.358 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "default min_size for any created local pool", 2024-05-10T11:44:16.359 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-10T11:44:16.359 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.359 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.359 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.359 INFO:teuthology.orchestra.run.smithi100.stdout: "num_rep": { 2024-05-10T11:44:16.359 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "num_rep", 2024-05-10T11:44:16.359 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.359 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.359 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.359 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "3", 2024-05-10T11:44:16.359 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.359 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.359 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.359 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "default replica count for any created local pool", 2024-05-10T11:44:16.359 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.360 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.360 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.360 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.360 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num": { 2024-05-10T11:44:16.360 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "pg_num", 2024-05-10T11:44:16.360 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.360 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.360 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.360 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "128", 2024-05-10T11:44:16.360 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.360 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.360 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.360 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "default pg_num for any created local pool", 2024-05-10T11:44:16.360 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.360 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.361 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.361 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.361 INFO:teuthology.orchestra.run.smithi100.stdout: "prefix": { 2024-05-10T11:44:16.361 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "prefix", 2024-05-10T11:44:16.361 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.361 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.361 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.361 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.361 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.361 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.361 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.361 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "name prefix for any created local pool", 2024-05-10T11:44:16.361 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.361 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.362 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.362 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.362 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.362 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.362 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.362 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.362 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.362 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.362 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.362 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.362 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.362 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.362 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.362 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.362 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.363 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.363 INFO:teuthology.orchestra.run.smithi100.stdout: "subtree": { 2024-05-10T11:44:16.363 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "subtree", 2024-05-10T11:44:16.363 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.363 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.363 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.363 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "rack", 2024-05-10T11:44:16.363 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.363 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.363 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.363 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-10T11:44:16.363 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-10T11:44:16.363 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.363 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.363 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.364 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.364 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.364 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.364 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "mds_autoscaler", 2024-05-10T11:44:16.364 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.364 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.364 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.364 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.364 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.364 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.364 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.364 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.364 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.364 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.364 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.364 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.365 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.365 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.365 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.365 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.365 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.365 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.365 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.365 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.365 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.365 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.365 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.365 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.365 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.365 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.365 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.366 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.366 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.366 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.366 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.366 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.366 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.366 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.366 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.366 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.366 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.366 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.366 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.366 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.366 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.367 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.367 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.367 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.367 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.367 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.367 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.367 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.367 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.367 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.367 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.367 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.367 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.367 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.367 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.367 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.368 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.368 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.368 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.368 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.368 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.368 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.368 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.368 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.368 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.368 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.368 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.368 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.368 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.368 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.368 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.368 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.369 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.369 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.369 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.369 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.369 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.369 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.369 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.369 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.369 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.369 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.369 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.369 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.369 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.369 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.370 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.370 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.370 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.370 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.370 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "mirroring", 2024-05-10T11:44:16.370 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.370 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.370 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.370 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.370 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.370 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.370 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.370 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.370 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.370 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.371 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.371 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.371 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.371 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.371 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.371 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.371 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.371 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.371 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.371 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.371 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.371 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.371 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.371 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.371 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.372 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.372 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.372 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.372 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.372 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.372 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.372 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.372 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.372 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.372 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.372 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.372 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.372 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.372 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.372 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.373 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.373 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.373 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.373 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.373 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.373 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.373 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.373 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.373 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.373 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.373 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.373 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.373 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.374 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.374 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.374 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.374 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.374 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.374 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.374 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.374 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.374 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.374 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.374 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.374 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.374 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.374 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.375 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.375 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.375 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.375 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.375 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.375 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.375 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.375 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.375 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.375 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.375 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.375 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.375 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.375 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.375 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.375 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.376 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.376 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.376 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.376 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.376 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.376 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.376 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.376 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "nfs", 2024-05-10T11:44:16.376 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.376 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.376 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.376 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.376 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.376 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.376 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.376 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.377 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.377 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.377 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.377 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.377 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.377 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.377 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.377 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.377 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.377 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.377 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.377 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.377 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.377 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.378 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.378 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.378 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.378 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.378 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.378 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.378 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.378 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.378 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.378 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.378 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.378 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.378 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.378 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.378 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.379 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.379 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.379 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.379 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.379 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.379 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.379 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.379 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.379 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.379 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.379 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.379 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.379 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.379 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.379 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.380 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.380 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.380 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.380 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.380 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.380 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.380 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.380 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.380 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.380 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.380 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.380 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.380 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.380 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.380 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.381 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.381 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.381 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.381 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.381 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.381 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.381 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.381 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.381 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.381 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.381 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.381 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.381 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.381 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.381 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.382 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.382 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.382 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.382 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.382 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.382 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.382 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.382 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.382 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "orchestrator", 2024-05-10T11:44:16.382 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.382 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.382 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.382 INFO:teuthology.orchestra.run.smithi100.stdout: "fail_fs": { 2024-05-10T11:44:16.382 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "fail_fs", 2024-05-10T11:44:16.383 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.383 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.383 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.383 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.383 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.383 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.383 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.383 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-10T11:44:16.383 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.383 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.383 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.383 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.383 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.383 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.383 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.384 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.384 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.384 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.384 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.384 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.384 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.384 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.384 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.384 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.384 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.384 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.384 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.384 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.384 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.384 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.385 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.385 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.385 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.385 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.385 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.385 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.385 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.385 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.385 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.385 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.385 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.385 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.385 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.385 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.385 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.386 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.386 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.386 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.386 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.386 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.386 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.386 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.386 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.386 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.386 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.386 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.386 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.386 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.386 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.386 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.387 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.387 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.387 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.387 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.387 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.387 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.387 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.387 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.387 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.387 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.387 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.387 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.387 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.387 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.388 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.388 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.388 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.388 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.388 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.388 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.388 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.388 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.388 INFO:teuthology.orchestra.run.smithi100.stdout: "orchestrator": { 2024-05-10T11:44:16.388 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "orchestrator", 2024-05-10T11:44:16.388 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.388 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.388 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.388 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.388 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.388 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.389 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.389 INFO:teuthology.orchestra.run.smithi100.stdout: "cephadm", 2024-05-10T11:44:16.389 INFO:teuthology.orchestra.run.smithi100.stdout: "rook", 2024-05-10T11:44:16.389 INFO:teuthology.orchestra.run.smithi100.stdout: "test_orchestrator" 2024-05-10T11:44:16.389 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.389 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Orchestrator backend", 2024-05-10T11:44:16.389 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.389 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.389 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.389 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.389 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.389 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.389 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.389 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.389 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.390 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.390 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.390 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.390 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.390 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.390 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.390 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.390 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.390 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.390 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.390 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.390 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.390 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "osd_perf_query", 2024-05-10T11:44:16.390 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.390 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.391 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.391 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.391 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.391 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.391 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.391 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.391 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.391 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.391 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.391 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.391 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.391 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.391 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.391 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.392 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.392 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.392 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.392 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.392 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.392 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.392 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.392 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.392 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.392 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.392 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.392 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.392 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.392 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.392 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.393 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.393 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.393 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.393 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.393 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.393 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.393 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.393 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.393 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.393 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.393 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.393 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.393 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.393 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.393 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.394 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.394 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.394 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.394 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.394 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.394 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.394 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.394 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.394 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.394 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.394 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.394 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.394 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.394 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.394 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.395 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.395 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.395 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.395 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.395 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.395 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.395 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.395 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.395 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.395 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.395 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.395 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.395 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.395 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.396 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.396 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.396 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.396 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.396 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.396 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.396 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.396 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.396 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.396 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.396 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.396 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.396 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.396 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.396 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.396 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "osd_support", 2024-05-10T11:44:16.397 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.397 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.397 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.397 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.397 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.397 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.397 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.397 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.397 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.397 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.397 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.397 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.397 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.397 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.398 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.398 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.398 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.398 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.398 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.398 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.398 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.398 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.398 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.398 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.398 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.398 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.398 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.398 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.398 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.399 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.399 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.399 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.399 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.399 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.399 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.399 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.399 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.399 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.399 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.399 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.399 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.399 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.399 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.399 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.400 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.400 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.400 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.400 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.400 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.400 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.400 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.400 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.400 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.400 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.400 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.400 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.400 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.401 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.401 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.401 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.401 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.401 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.401 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.401 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.401 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.401 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.401 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.401 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.401 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.402 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.402 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.402 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.402 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.402 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.402 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.402 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.402 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.402 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.402 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.402 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.402 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.403 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.403 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.403 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.403 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.403 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.403 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.403 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.403 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.403 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.403 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "pg_autoscaler", 2024-05-10T11:44:16.403 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.403 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.403 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.403 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.403 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.404 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.404 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.404 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.404 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.404 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.404 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.404 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.404 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.404 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.404 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.404 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.404 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.404 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.404 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.405 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.405 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.405 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.405 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.405 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.405 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.405 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.405 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.405 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.405 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.405 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.406 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.406 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.406 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.406 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.406 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.406 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.406 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.406 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.406 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.406 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.406 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.406 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.406 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.406 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.407 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.407 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.407 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.407 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.407 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.407 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.407 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.407 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.407 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.407 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.407 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.407 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.407 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.407 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.408 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.408 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.408 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.408 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.408 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.408 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.408 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.408 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.408 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.408 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.408 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.408 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.408 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.408 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.409 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.409 INFO:teuthology.orchestra.run.smithi100.stdout: "sleep_interval": { 2024-05-10T11:44:16.409 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sleep_interval", 2024-05-10T11:44:16.409 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-05-10T11:44:16.409 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.409 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.409 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "60", 2024-05-10T11:44:16.409 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.409 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.409 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.409 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.409 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.410 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.410 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.410 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.410 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.410 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.410 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.410 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.410 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.410 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.410 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.410 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.410 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.410 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.410 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.411 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.411 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.411 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.411 INFO:teuthology.orchestra.run.smithi100.stdout: "threshold": { 2024-05-10T11:44:16.411 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "threshold", 2024-05-10T11:44:16.411 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "float", 2024-05-10T11:44:16.411 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.411 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.411 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "3.0", 2024-05-10T11:44:16.411 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "1.0", 2024-05-10T11:44:16.411 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.411 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.411 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "scaling threshold", 2024-05-10T11:44:16.411 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-10T11:44:16.411 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.412 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.412 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.412 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.412 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.412 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.412 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "progress", 2024-05-10T11:44:16.412 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.412 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.412 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.412 INFO:teuthology.orchestra.run.smithi100.stdout: "allow_pg_recovery_event": { 2024-05-10T11:44:16.412 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "allow_pg_recovery_event", 2024-05-10T11:44:16.412 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.412 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.412 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.413 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.413 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.413 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.413 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.413 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-10T11:44:16.413 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.413 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.413 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.413 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.413 INFO:teuthology.orchestra.run.smithi100.stdout: "enabled": { 2024-05-10T11:44:16.413 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "enabled", 2024-05-10T11:44:16.413 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.413 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.413 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.413 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.414 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.414 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.414 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.414 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.414 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.414 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.414 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.414 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.414 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.414 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.414 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.414 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.414 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.414 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.414 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.415 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.415 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.415 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.415 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.415 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.415 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.415 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.415 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.415 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.415 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.415 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.415 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.415 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.416 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.416 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.416 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.416 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.416 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.416 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.416 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.416 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.416 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.416 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.416 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.416 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.416 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.416 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.416 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.417 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.417 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.417 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.417 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.417 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.417 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.417 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.417 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.417 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.417 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.417 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.417 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.417 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.417 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.417 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.418 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.418 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.418 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.418 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.418 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.418 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.418 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.418 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.418 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.418 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.418 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.418 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.418 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.418 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.418 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.419 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.419 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.419 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.419 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.419 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.419 INFO:teuthology.orchestra.run.smithi100.stdout: "max_completed_events": { 2024-05-10T11:44:16.419 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "max_completed_events", 2024-05-10T11:44:16.419 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.419 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.419 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.419 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "50", 2024-05-10T11:44:16.419 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.419 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.419 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.419 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "number of past completed events to remember", 2024-05-10T11:44:16.420 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.420 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.420 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.420 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.420 INFO:teuthology.orchestra.run.smithi100.stdout: "sleep_interval": { 2024-05-10T11:44:16.420 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sleep_interval", 2024-05-10T11:44:16.420 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-05-10T11:44:16.420 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.420 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.420 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "5", 2024-05-10T11:44:16.420 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.420 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.420 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.420 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "how long the module is going to sleep", 2024-05-10T11:44:16.421 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.421 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.421 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.421 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.421 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.421 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.421 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.421 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.421 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.421 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.421 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.421 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.421 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.421 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.421 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.422 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.422 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.422 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.422 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.422 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.422 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.422 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "prometheus", 2024-05-10T11:44:16.422 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.422 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.422 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.422 INFO:teuthology.orchestra.run.smithi100.stdout: "cache": { 2024-05-10T11:44:16.423 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "cache", 2024-05-10T11:44:16.423 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.423 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.423 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.423 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.423 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.423 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.423 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.423 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.423 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.423 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.423 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.423 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.423 INFO:teuthology.orchestra.run.smithi100.stdout: "exclude_perf_counters": { 2024-05-10T11:44:16.424 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "exclude_perf_counters", 2024-05-10T11:44:16.424 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.424 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.424 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.424 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.424 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.424 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.424 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.424 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-10T11:44:16.424 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-10T11:44:16.424 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.424 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.424 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.424 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.424 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.425 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.425 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.425 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.425 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.425 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.425 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.425 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.425 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.425 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.425 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.425 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.425 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.425 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.425 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.425 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.426 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.426 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.426 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.426 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.426 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.426 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.426 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.426 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.426 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.426 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.426 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.426 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.426 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.426 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.427 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.427 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.427 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.427 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.427 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.427 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.427 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.427 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.427 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.427 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.427 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.427 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.427 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.427 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.428 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.428 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.428 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.428 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.428 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.428 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.428 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.428 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.428 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.428 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.428 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.428 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.428 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.428 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.428 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.429 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.429 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.429 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.429 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.429 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.429 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.429 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.429 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.429 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.429 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.429 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_stats_pools": { 2024-05-10T11:44:16.429 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rbd_stats_pools", 2024-05-10T11:44:16.429 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.429 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.430 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.430 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.430 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.430 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.430 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.430 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.430 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.430 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.430 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.430 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.430 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-10T11:44:16.430 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-10T11:44:16.430 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.430 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.431 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.431 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "300", 2024-05-10T11:44:16.431 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.431 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.431 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.431 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.431 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.431 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.431 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.431 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.431 INFO:teuthology.orchestra.run.smithi100.stdout: "scrape_interval": { 2024-05-10T11:44:16.431 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "scrape_interval", 2024-05-10T11:44:16.431 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "float", 2024-05-10T11:44:16.431 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.431 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.431 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "15.0", 2024-05-10T11:44:16.432 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.432 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.432 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.432 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.432 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.432 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.432 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.432 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.432 INFO:teuthology.orchestra.run.smithi100.stdout: "server_addr": { 2024-05-10T11:44:16.432 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "server_addr", 2024-05-10T11:44:16.432 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.432 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.432 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.432 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "::", 2024-05-10T11:44:16.432 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.433 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.433 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.433 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-10T11:44:16.433 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.433 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.433 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.433 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.433 INFO:teuthology.orchestra.run.smithi100.stdout: "server_port": { 2024-05-10T11:44:16.433 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "server_port", 2024-05-10T11:44:16.433 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.433 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.433 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.433 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "9283", 2024-05-10T11:44:16.433 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.434 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.434 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.434 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-10T11:44:16.434 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.434 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.434 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.434 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.434 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.434 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.434 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.434 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.434 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.434 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.434 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.434 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.435 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.435 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.435 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.435 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.435 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.435 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.435 INFO:teuthology.orchestra.run.smithi100.stdout: "stale_cache_strategy": { 2024-05-10T11:44:16.435 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "stale_cache_strategy", 2024-05-10T11:44:16.435 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.435 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.435 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.435 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "log", 2024-05-10T11:44:16.435 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.435 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.436 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.436 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.436 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.436 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.436 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.436 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.436 INFO:teuthology.orchestra.run.smithi100.stdout: "standby_behaviour": { 2024-05-10T11:44:16.436 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "standby_behaviour", 2024-05-10T11:44:16.436 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.436 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.436 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.436 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "default", 2024-05-10T11:44:16.436 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.436 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.437 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.437 INFO:teuthology.orchestra.run.smithi100.stdout: "default", 2024-05-10T11:44:16.437 INFO:teuthology.orchestra.run.smithi100.stdout: "error" 2024-05-10T11:44:16.437 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.437 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.437 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.437 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.437 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.437 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.437 INFO:teuthology.orchestra.run.smithi100.stdout: "standby_error_status_code": { 2024-05-10T11:44:16.437 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "standby_error_status_code", 2024-05-10T11:44:16.437 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.437 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.437 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.437 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "500", 2024-05-10T11:44:16.437 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "400", 2024-05-10T11:44:16.438 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "599", 2024-05-10T11:44:16.438 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.438 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.438 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.438 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.438 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.438 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.438 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.438 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.438 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.438 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rbd_support", 2024-05-10T11:44:16.438 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.438 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.438 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.439 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.439 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.439 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.439 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.439 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.439 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.439 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.439 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.439 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.439 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.439 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.439 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.439 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.439 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.440 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.440 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.440 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.440 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.440 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.440 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.440 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.440 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.440 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.440 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.440 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.440 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.440 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.440 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.441 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.441 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.441 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.441 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.441 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.441 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.441 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.441 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.441 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.441 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.441 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.441 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.441 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.442 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.442 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.442 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.442 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.442 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.442 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.442 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.442 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.442 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.442 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.442 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.442 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.442 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.442 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.442 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.443 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.443 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.443 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.443 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.443 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.443 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.443 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.443 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.443 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.443 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.443 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.443 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.443 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.443 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.444 INFO:teuthology.orchestra.run.smithi100.stdout: "max_concurrent_snap_create": { 2024-05-10T11:44:16.444 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "max_concurrent_snap_create", 2024-05-10T11:44:16.444 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.444 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.444 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.444 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10", 2024-05-10T11:44:16.444 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.444 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.444 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.444 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.444 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.444 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.444 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.444 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.444 INFO:teuthology.orchestra.run.smithi100.stdout: "mirror_snapshot_schedule": { 2024-05-10T11:44:16.444 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "mirror_snapshot_schedule", 2024-05-10T11:44:16.445 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.445 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.445 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.445 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.445 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.445 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.445 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.445 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.445 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.445 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.445 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.445 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.445 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.445 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.445 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.446 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.446 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.446 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.446 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.446 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.446 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.446 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.446 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.446 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.446 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.446 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.446 INFO:teuthology.orchestra.run.smithi100.stdout: "trash_purge_schedule": { 2024-05-10T11:44:16.446 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "trash_purge_schedule", 2024-05-10T11:44:16.446 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.446 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.447 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.447 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.447 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.447 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.447 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.447 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.447 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.447 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.447 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.447 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.447 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.447 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.447 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.447 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "restful", 2024-05-10T11:44:16.448 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.448 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.448 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.448 INFO:teuthology.orchestra.run.smithi100.stdout: "enable_auth": { 2024-05-10T11:44:16.448 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "enable_auth", 2024-05-10T11:44:16.448 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.448 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.448 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.448 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.448 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.448 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.448 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.448 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.448 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.448 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.449 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.449 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.449 INFO:teuthology.orchestra.run.smithi100.stdout: "key_file": { 2024-05-10T11:44:16.449 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "key_file", 2024-05-10T11:44:16.449 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.449 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.449 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.449 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.449 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.449 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.449 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.449 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.449 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.449 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.450 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.450 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.450 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.450 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.450 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.450 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.450 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.450 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.450 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.450 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.450 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.450 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.450 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.450 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.451 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.451 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.451 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.451 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.451 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.451 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.451 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.451 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.451 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.451 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.451 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.451 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.451 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.451 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.451 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.452 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.452 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.452 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.452 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.452 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.452 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.452 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.452 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.452 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.452 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.452 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.452 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.452 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.452 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.452 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.453 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.453 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.453 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.453 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.453 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.453 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.453 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.453 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.453 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.453 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.453 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.453 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.453 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.453 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.454 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.454 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.454 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.454 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.454 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.454 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.454 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.454 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.454 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.454 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.454 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.454 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.454 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.454 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.455 INFO:teuthology.orchestra.run.smithi100.stdout: "server_addr": { 2024-05-10T11:44:16.455 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "server_addr", 2024-05-10T11:44:16.455 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.455 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.455 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.455 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.455 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.455 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.455 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.455 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.455 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.455 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.455 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.455 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.455 INFO:teuthology.orchestra.run.smithi100.stdout: "server_port": { 2024-05-10T11:44:16.456 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "server_port", 2024-05-10T11:44:16.456 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.456 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.456 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.456 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.456 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.456 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.456 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.456 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.456 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.456 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.456 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.456 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.456 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.456 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.457 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.457 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.457 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.457 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.457 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.457 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.457 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.457 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.457 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.457 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.457 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.457 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.457 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.457 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.457 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.458 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rgw", 2024-05-10T11:44:16.458 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.458 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.458 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.458 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.458 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.458 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.458 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.458 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.458 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.458 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.458 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.458 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.458 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.458 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.459 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.459 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.459 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.459 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.459 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.459 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.459 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.459 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.459 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.459 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.459 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.459 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.459 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.459 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.459 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.460 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.460 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.460 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.460 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.460 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.460 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.460 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.460 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.460 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.460 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.460 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.460 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.460 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.460 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.460 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.460 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.461 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.461 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.461 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.461 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.461 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.461 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.461 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.461 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.461 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.461 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.461 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.461 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.461 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.461 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.461 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.462 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.462 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.462 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.462 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.462 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.462 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.462 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.462 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.462 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.462 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.462 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.462 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.462 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.462 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.463 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.463 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.463 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.463 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.463 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.463 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.463 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.463 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.463 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.463 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.463 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.463 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.463 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.463 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.463 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.464 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.464 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rook", 2024-05-10T11:44:16.464 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.464 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.464 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.464 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.464 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.464 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.464 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.464 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.464 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.464 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.464 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.464 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.464 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.464 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.465 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.465 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.465 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.465 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.465 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.465 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.465 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.465 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.465 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.465 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.465 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.465 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.465 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.465 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.465 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.466 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.466 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.466 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.466 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.466 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.466 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.466 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.466 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.466 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.466 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.466 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.466 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.466 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.466 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.467 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.467 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.467 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.467 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.467 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.467 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.467 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.467 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.467 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.467 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.467 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.467 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.467 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.467 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.468 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.468 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.468 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.468 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.468 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.468 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.468 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.468 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.468 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.468 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.468 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.468 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.468 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.468 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.468 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.468 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.469 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.469 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.469 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.469 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.469 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.469 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.469 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.469 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.469 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.469 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.469 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.469 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.469 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.469 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.469 INFO:teuthology.orchestra.run.smithi100.stdout: "storage_class": { 2024-05-10T11:44:16.470 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "storage_class", 2024-05-10T11:44:16.470 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.470 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.470 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.470 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "local", 2024-05-10T11:44:16.470 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.470 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.470 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.470 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-10T11:44:16.470 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.470 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.470 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.470 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.470 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.471 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.471 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.471 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "selftest", 2024-05-10T11:44:16.471 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.471 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.471 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.471 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.471 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.471 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.471 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.471 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.471 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.471 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.471 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.471 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.472 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.472 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.472 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.472 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.472 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.472 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.472 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.472 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.472 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.472 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.472 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.472 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.472 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.472 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.472 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.473 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.473 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.473 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.473 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.473 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.473 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.473 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.473 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.473 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.473 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.473 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.473 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.473 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.473 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.473 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.474 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.474 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.474 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.474 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.474 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.474 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.474 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.474 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.474 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.474 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.474 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.474 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.474 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.474 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.474 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.475 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.475 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.475 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.475 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.475 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.475 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.475 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.475 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.475 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.475 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.475 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.475 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.475 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.475 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.475 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.476 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.476 INFO:teuthology.orchestra.run.smithi100.stdout: "roption1": { 2024-05-10T11:44:16.476 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "roption1", 2024-05-10T11:44:16.476 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.476 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.476 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.476 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.476 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.476 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.476 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.476 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.476 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.476 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.476 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.477 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.477 INFO:teuthology.orchestra.run.smithi100.stdout: "roption2": { 2024-05-10T11:44:16.477 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "roption2", 2024-05-10T11:44:16.477 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.477 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.477 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.477 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "xyz", 2024-05-10T11:44:16.477 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.477 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.477 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.477 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.477 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.477 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.477 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.477 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.478 INFO:teuthology.orchestra.run.smithi100.stdout: "rwoption1": { 2024-05-10T11:44:16.478 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rwoption1", 2024-05-10T11:44:16.478 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.478 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.478 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.478 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.478 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.478 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.478 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.478 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.478 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.478 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.478 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.478 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.479 INFO:teuthology.orchestra.run.smithi100.stdout: "rwoption2": { 2024-05-10T11:44:16.479 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rwoption2", 2024-05-10T11:44:16.479 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.479 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.479 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.479 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.479 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.479 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.479 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.479 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.479 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.479 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.479 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.479 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.480 INFO:teuthology.orchestra.run.smithi100.stdout: "rwoption3": { 2024-05-10T11:44:16.480 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rwoption3", 2024-05-10T11:44:16.480 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "float", 2024-05-10T11:44:16.480 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.480 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.480 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.480 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.480 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.480 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.480 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.480 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.480 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.480 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.480 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.481 INFO:teuthology.orchestra.run.smithi100.stdout: "rwoption4": { 2024-05-10T11:44:16.481 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rwoption4", 2024-05-10T11:44:16.481 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.481 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.481 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.481 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.481 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.481 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.481 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.481 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.481 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.481 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.481 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.481 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.481 INFO:teuthology.orchestra.run.smithi100.stdout: "rwoption5": { 2024-05-10T11:44:16.482 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rwoption5", 2024-05-10T11:44:16.482 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.482 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.482 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.482 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.482 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.482 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.482 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.482 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.482 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.482 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.482 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.482 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.482 INFO:teuthology.orchestra.run.smithi100.stdout: "rwoption6": { 2024-05-10T11:44:16.483 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rwoption6", 2024-05-10T11:44:16.483 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.483 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.483 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.483 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.483 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.483 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.483 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.483 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.483 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.483 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.483 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.483 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.483 INFO:teuthology.orchestra.run.smithi100.stdout: "rwoption7": { 2024-05-10T11:44:16.484 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rwoption7", 2024-05-10T11:44:16.484 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.484 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.484 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.484 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.484 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "1", 2024-05-10T11:44:16.484 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "42", 2024-05-10T11:44:16.484 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.484 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.484 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.484 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.484 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.484 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.484 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.484 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.485 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.485 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.485 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.485 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.485 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.485 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.485 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.485 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.485 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.485 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.485 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.485 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.485 INFO:teuthology.orchestra.run.smithi100.stdout: "testkey": { 2024-05-10T11:44:16.485 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "testkey", 2024-05-10T11:44:16.486 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.486 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.486 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.486 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.486 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.486 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.486 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.486 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.486 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.486 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.486 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.486 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.486 INFO:teuthology.orchestra.run.smithi100.stdout: "testlkey": { 2024-05-10T11:44:16.486 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "testlkey", 2024-05-10T11:44:16.487 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.487 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.487 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.487 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.487 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.487 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.487 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.487 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.487 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.487 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.487 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.487 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.487 INFO:teuthology.orchestra.run.smithi100.stdout: "testnewline": { 2024-05-10T11:44:16.487 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "testnewline", 2024-05-10T11:44:16.488 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.488 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.488 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.488 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.488 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.488 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.488 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.488 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.488 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.488 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.488 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.488 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.488 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.488 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.488 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.489 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smb", 2024-05-10T11:44:16.489 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.489 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.489 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.489 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.489 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.489 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.489 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.489 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.489 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.489 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.489 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.490 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.490 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.490 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.490 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.490 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.490 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.490 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.490 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.490 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.490 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.490 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.490 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.490 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.490 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.491 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.491 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.491 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.491 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.491 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.491 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.491 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.491 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.491 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.491 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.491 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.491 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.491 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.491 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.491 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.492 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.492 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.492 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.492 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.492 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.492 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.492 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.492 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.492 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.492 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.492 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.492 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.492 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.492 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.492 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.493 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.493 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.493 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.493 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.493 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.493 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.493 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.493 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.493 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.493 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.493 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.493 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.493 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.493 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.493 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.494 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.494 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.494 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.494 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.494 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.494 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.494 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.494 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.494 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.494 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.494 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.494 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.494 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.494 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.495 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.495 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.495 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.495 INFO:teuthology.orchestra.run.smithi100.stdout: "update_orchestration": { 2024-05-10T11:44:16.495 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "update_orchestration", 2024-05-10T11:44:16.495 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.495 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.495 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.495 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.495 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.495 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.495 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.495 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-10T11:44:16.495 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.496 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.496 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.496 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.496 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.496 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.496 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.496 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "snap_schedule", 2024-05-10T11:44:16.496 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.496 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.496 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.496 INFO:teuthology.orchestra.run.smithi100.stdout: "allow_m_granularity": { 2024-05-10T11:44:16.496 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "allow_m_granularity", 2024-05-10T11:44:16.496 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.496 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.497 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.497 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.497 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.497 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.497 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.497 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "allow minute scheduled snapshots", 2024-05-10T11:44:16.497 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.497 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.497 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.497 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.497 INFO:teuthology.orchestra.run.smithi100.stdout: "dump_on_update": { 2024-05-10T11:44:16.497 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "dump_on_update", 2024-05-10T11:44:16.497 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.497 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.497 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.498 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.498 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.498 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.498 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.498 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "dump database to debug log on update", 2024-05-10T11:44:16.498 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.498 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.498 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.498 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.498 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.498 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.498 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.498 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.498 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.498 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.499 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.499 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.499 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.499 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.499 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.499 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.499 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.499 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.499 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.499 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.499 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.499 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.499 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.499 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.499 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.500 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.500 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.500 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.500 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.500 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.500 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.500 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.500 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.500 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.500 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.500 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.500 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.500 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.500 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.500 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.501 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.501 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.501 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.501 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.501 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.501 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.501 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.501 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.501 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.501 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.501 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.501 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.501 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.501 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.501 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.502 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.502 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.502 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.502 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.502 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.502 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.502 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.502 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.502 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.502 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.502 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.502 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.502 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.503 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.503 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.503 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.503 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.503 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.503 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.503 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.503 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.503 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.503 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.503 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.503 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.503 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.503 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.503 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.504 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.504 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.504 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.504 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.504 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.504 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.504 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.504 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.504 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "stats", 2024-05-10T11:44:16.504 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.504 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.504 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.504 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.504 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.504 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.505 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.505 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.505 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.505 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.505 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.505 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.505 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.505 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.505 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.505 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.505 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.505 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.505 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.505 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.505 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.506 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.506 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.506 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.506 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.506 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.506 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.506 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.506 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.506 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.506 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.506 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.506 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.506 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.506 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.506 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.507 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.507 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.507 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.507 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.507 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.507 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.507 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.507 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.507 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.507 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.507 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.507 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.507 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.507 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.507 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.508 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.508 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.508 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.508 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.508 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.508 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.508 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.508 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.508 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.508 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.508 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.508 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.508 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.508 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.509 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.509 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.509 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.509 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.509 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.509 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.509 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.509 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.509 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.509 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.509 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.509 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.509 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.509 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.509 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.509 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.510 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.510 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.510 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.510 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.510 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.510 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.510 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.510 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.510 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.510 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "status", 2024-05-10T11:44:16.510 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.510 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.510 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.510 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.511 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.511 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.511 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.511 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.511 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.511 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.511 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.511 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.511 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.511 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.511 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.511 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.511 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.511 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.511 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.511 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.512 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.512 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.512 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.512 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.512 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.512 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.512 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.512 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.512 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.512 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.512 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.512 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.512 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.512 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.513 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.513 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.513 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.513 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.513 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.513 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.513 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.513 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.513 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.513 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.513 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.513 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.513 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.513 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.513 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.514 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.514 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.514 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.514 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.514 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.514 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.514 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.514 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.514 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.514 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.514 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.514 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.514 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.514 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.515 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.515 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.515 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.515 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.515 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.515 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.515 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.515 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.515 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.515 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.515 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.515 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.515 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.515 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.515 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.516 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.516 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.516 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.516 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.516 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.516 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.516 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.516 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.516 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.516 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.516 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.516 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.516 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "telegraf", 2024-05-10T11:44:16.516 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.516 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.517 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.517 INFO:teuthology.orchestra.run.smithi100.stdout: "address": { 2024-05-10T11:44:16.517 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "address", 2024-05-10T11:44:16.517 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.517 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.517 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.517 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-10T11:44:16.517 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.517 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.517 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.517 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.517 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.517 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.517 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.517 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.518 INFO:teuthology.orchestra.run.smithi100.stdout: "interval": { 2024-05-10T11:44:16.518 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "interval", 2024-05-10T11:44:16.518 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-05-10T11:44:16.518 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.518 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.518 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "15", 2024-05-10T11:44:16.518 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.518 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.518 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.518 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.518 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.518 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.518 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.518 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.519 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.519 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.519 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.519 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.519 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.519 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.519 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.519 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.519 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.519 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.519 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.519 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.519 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.519 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.519 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.520 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.520 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.520 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.520 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.520 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.520 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.520 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.520 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.520 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.520 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.520 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.520 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.520 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.520 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.520 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.521 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.521 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.521 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.521 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.521 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.521 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.521 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.521 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.521 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.521 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.521 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.521 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.521 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.521 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.521 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.522 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.522 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.522 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.522 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.522 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.522 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.522 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.522 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.522 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.522 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.522 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.522 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.522 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.522 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.523 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.523 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.523 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.523 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.523 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.523 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.523 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.523 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.523 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.523 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.523 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.523 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.523 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.523 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.524 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.524 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.524 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.524 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.524 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.524 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.524 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.524 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.524 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.524 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.524 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.524 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.524 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.524 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.524 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "telemetry", 2024-05-10T11:44:16.525 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.525 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.525 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.525 INFO:teuthology.orchestra.run.smithi100.stdout: "channel_basic": { 2024-05-10T11:44:16.525 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "channel_basic", 2024-05-10T11:44:16.525 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.525 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.525 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.525 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.525 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.525 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.525 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.525 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-10T11:44:16.525 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.526 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.526 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.526 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.526 INFO:teuthology.orchestra.run.smithi100.stdout: "channel_crash": { 2024-05-10T11:44:16.526 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "channel_crash", 2024-05-10T11:44:16.526 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.526 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.526 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.526 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.526 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.526 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.526 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.526 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-10T11:44:16.526 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.526 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.527 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.527 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.527 INFO:teuthology.orchestra.run.smithi100.stdout: "channel_device": { 2024-05-10T11:44:16.527 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "channel_device", 2024-05-10T11:44:16.527 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.527 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.527 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.527 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.527 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.527 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.527 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.527 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-10T11:44:16.527 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.527 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.527 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.528 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.528 INFO:teuthology.orchestra.run.smithi100.stdout: "channel_ident": { 2024-05-10T11:44:16.528 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "channel_ident", 2024-05-10T11:44:16.528 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.528 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.528 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.528 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.528 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.528 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.528 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.528 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-10T11:44:16.528 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.528 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.528 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.529 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.529 INFO:teuthology.orchestra.run.smithi100.stdout: "channel_perf": { 2024-05-10T11:44:16.529 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "channel_perf", 2024-05-10T11:44:16.529 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.529 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.529 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.529 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.529 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.529 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.529 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.529 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-10T11:44:16.529 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.529 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.530 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.530 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.530 INFO:teuthology.orchestra.run.smithi100.stdout: "contact": { 2024-05-10T11:44:16.530 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "contact", 2024-05-10T11:44:16.530 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.530 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.530 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.530 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.530 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.530 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.530 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.530 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.530 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.530 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.531 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.531 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.531 INFO:teuthology.orchestra.run.smithi100.stdout: "description": { 2024-05-10T11:44:16.531 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "description", 2024-05-10T11:44:16.531 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.531 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.531 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.531 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.531 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.531 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.531 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.531 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.531 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.531 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.532 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.532 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.532 INFO:teuthology.orchestra.run.smithi100.stdout: "device_url": { 2024-05-10T11:44:16.532 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "device_url", 2024-05-10T11:44:16.532 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.532 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.532 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.532 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-10T11:44:16.532 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.532 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.532 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.532 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.532 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.532 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.532 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.533 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.533 INFO:teuthology.orchestra.run.smithi100.stdout: "enabled": { 2024-05-10T11:44:16.533 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "enabled", 2024-05-10T11:44:16.533 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.533 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.533 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.533 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.533 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.533 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.533 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.533 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.533 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.533 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.533 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.533 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.534 INFO:teuthology.orchestra.run.smithi100.stdout: "interval": { 2024-05-10T11:44:16.534 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "interval", 2024-05-10T11:44:16.534 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.534 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.534 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.534 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "24", 2024-05-10T11:44:16.534 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "8", 2024-05-10T11:44:16.534 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.534 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.534 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.534 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.534 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.534 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.534 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.535 INFO:teuthology.orchestra.run.smithi100.stdout: "last_opt_revision": { 2024-05-10T11:44:16.535 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "last_opt_revision", 2024-05-10T11:44:16.535 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.535 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.535 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.535 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "1", 2024-05-10T11:44:16.535 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.535 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.535 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.535 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.535 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.535 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.535 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.535 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.535 INFO:teuthology.orchestra.run.smithi100.stdout: "leaderboard": { 2024-05-10T11:44:16.535 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "leaderboard", 2024-05-10T11:44:16.536 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.536 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.536 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.536 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.536 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.536 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.536 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.536 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.536 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.536 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.536 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.536 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.536 INFO:teuthology.orchestra.run.smithi100.stdout: "leaderboard_description": { 2024-05-10T11:44:16.536 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "leaderboard_description", 2024-05-10T11:44:16.537 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.537 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.537 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.537 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.537 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.537 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.537 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.537 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.537 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.537 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.537 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.537 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.537 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.537 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.537 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.538 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.538 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.538 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.538 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.538 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.538 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.538 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.538 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.538 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.538 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.538 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.538 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.538 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.538 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.538 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.539 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.539 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.539 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.539 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.539 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.539 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.539 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.539 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.539 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.539 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.539 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.539 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.539 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.539 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.540 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.540 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.540 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.540 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.540 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.540 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.540 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.540 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.540 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.540 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.540 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.540 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.540 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.540 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.540 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.540 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.541 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.541 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.541 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.541 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.541 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.541 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.541 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.541 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.541 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.541 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.541 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.541 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.541 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.541 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.541 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.542 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.542 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.542 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.542 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.542 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.542 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.542 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.542 INFO:teuthology.orchestra.run.smithi100.stdout: "organization": { 2024-05-10T11:44:16.542 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "organization", 2024-05-10T11:44:16.542 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.542 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.542 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.542 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.542 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.543 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.543 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.543 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.543 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.543 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.543 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.543 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.543 INFO:teuthology.orchestra.run.smithi100.stdout: "proxy": { 2024-05-10T11:44:16.543 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "proxy", 2024-05-10T11:44:16.543 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.543 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.543 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.543 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.543 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.543 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.543 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.544 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.544 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.544 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.544 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.544 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.544 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.544 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.544 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.544 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.544 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.544 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.544 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.544 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.544 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.545 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.545 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.545 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.545 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.545 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.545 INFO:teuthology.orchestra.run.smithi100.stdout: "url": { 2024-05-10T11:44:16.545 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "url", 2024-05-10T11:44:16.545 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.545 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.545 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.545 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-10T11:44:16.545 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.545 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.545 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.545 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.546 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.546 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.546 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.546 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.546 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.546 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.546 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.546 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "test_orchestrator", 2024-05-10T11:44:16.546 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.546 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.546 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.546 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.546 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.546 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.547 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.547 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.547 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.547 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.547 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.547 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.547 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.547 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.547 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.547 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.547 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.547 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.547 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.547 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.547 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.548 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.548 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.548 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.548 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.548 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.548 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.548 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.548 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.548 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.548 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.548 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.548 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.548 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.548 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.548 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.548 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.549 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.549 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.549 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.549 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.549 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.549 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.549 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.549 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.549 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.549 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.549 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.549 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.549 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.549 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.550 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.550 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.550 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.550 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.550 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.550 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.550 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.550 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.550 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.550 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.550 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.550 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.550 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.550 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.550 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.550 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.551 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.551 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.551 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.551 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.551 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.551 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.551 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.551 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.551 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.551 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.551 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.551 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.551 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.551 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.551 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.552 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.552 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.552 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.552 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.552 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.552 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.552 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.552 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.552 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "volumes", 2024-05-10T11:44:16.552 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.552 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.552 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.552 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.552 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.553 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.553 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.553 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.553 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.553 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.553 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.553 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.553 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.553 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.553 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.553 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.553 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.553 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.553 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.554 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.554 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.554 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.554 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.554 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.554 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.554 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.554 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.554 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.554 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.554 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.554 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.554 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.554 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.555 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.555 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.555 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.555 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.555 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.555 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.555 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.555 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.555 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.555 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.555 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.555 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.555 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.555 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.556 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.556 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.556 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.556 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.556 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.556 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.556 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.556 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.556 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.556 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.556 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.556 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.556 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.556 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.556 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.557 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.557 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.557 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.557 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.557 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.557 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.557 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.557 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.557 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.557 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.557 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.557 INFO:teuthology.orchestra.run.smithi100.stdout: "max_concurrent_clones": { 2024-05-10T11:44:16.557 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "max_concurrent_clones", 2024-05-10T11:44:16.557 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.558 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.558 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.558 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "4", 2024-05-10T11:44:16.558 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.558 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.558 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.558 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-10T11:44:16.558 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.558 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.558 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.558 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.558 INFO:teuthology.orchestra.run.smithi100.stdout: "periodic_async_work": { 2024-05-10T11:44:16.558 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "periodic_async_work", 2024-05-10T11:44:16.558 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.558 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.559 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.559 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.559 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.559 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.559 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.559 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Periodically check for async work", 2024-05-10T11:44:16.559 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.559 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.559 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.559 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.559 INFO:teuthology.orchestra.run.smithi100.stdout: "snapshot_clone_delay": { 2024-05-10T11:44:16.559 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "snapshot_clone_delay", 2024-05-10T11:44:16.559 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.559 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.560 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.560 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.560 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.560 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.560 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.560 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-10T11:44:16.560 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.560 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.560 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.560 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.560 INFO:teuthology.orchestra.run.smithi100.stdout: "snapshot_clone_no_wait": { 2024-05-10T11:44:16.560 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "snapshot_clone_no_wait", 2024-05-10T11:44:16.560 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.560 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.560 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.561 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.561 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.561 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.561 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.561 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-10T11:44:16.561 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.561 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.561 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.561 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.561 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.561 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.561 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.561 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.561 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.562 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.562 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.562 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.562 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.562 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.562 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.562 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.562 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.562 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.562 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.562 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.562 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.562 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "zabbix", 2024-05-10T11:44:16.562 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.562 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.563 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.563 INFO:teuthology.orchestra.run.smithi100.stdout: "discovery_interval": { 2024-05-10T11:44:16.563 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "discovery_interval", 2024-05-10T11:44:16.563 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "uint", 2024-05-10T11:44:16.563 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.563 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.563 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "100", 2024-05-10T11:44:16.563 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.563 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.563 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.563 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.563 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.563 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.563 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.564 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.564 INFO:teuthology.orchestra.run.smithi100.stdout: "identifier": { 2024-05-10T11:44:16.564 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "identifier", 2024-05-10T11:44:16.564 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.564 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.564 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.564 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.564 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.564 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.564 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.564 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.564 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.564 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.564 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.565 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.565 INFO:teuthology.orchestra.run.smithi100.stdout: "interval": { 2024-05-10T11:44:16.565 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "interval", 2024-05-10T11:44:16.565 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-05-10T11:44:16.565 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.565 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.565 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "60", 2024-05-10T11:44:16.565 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.565 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.565 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.565 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.565 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.565 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.565 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.565 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.565 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.566 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.566 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.566 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.566 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.566 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.566 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.566 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.566 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.566 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.566 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.566 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.566 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.566 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.566 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.567 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.567 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.567 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.567 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.567 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.567 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.567 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.567 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.567 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.567 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.567 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.567 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.567 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.567 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.567 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.568 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.568 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.568 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.568 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.568 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.568 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.568 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.568 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.568 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.568 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.568 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.568 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.568 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.568 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.568 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.569 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.569 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.569 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.569 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.569 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.569 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.569 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.569 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.569 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.569 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.569 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.569 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.569 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.569 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.569 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.570 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.570 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.570 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.570 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.570 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.570 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.570 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.570 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.570 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.570 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.570 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.570 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.570 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.570 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.570 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.571 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.571 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.571 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.571 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.571 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.571 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.571 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.571 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.571 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.571 INFO:teuthology.orchestra.run.smithi100.stdout: "zabbix_host": { 2024-05-10T11:44:16.571 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "zabbix_host", 2024-05-10T11:44:16.571 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.571 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.571 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.571 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.572 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.572 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.572 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.572 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.572 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.572 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.572 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.572 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.572 INFO:teuthology.orchestra.run.smithi100.stdout: "zabbix_port": { 2024-05-10T11:44:16.572 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "zabbix_port", 2024-05-10T11:44:16.572 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.572 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.572 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.572 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10051", 2024-05-10T11:44:16.572 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.573 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.573 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.573 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.573 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.573 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.573 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.573 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.573 INFO:teuthology.orchestra.run.smithi100.stdout: "zabbix_sender": { 2024-05-10T11:44:16.573 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "zabbix_sender", 2024-05-10T11:44:16.573 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.573 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.573 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.573 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-10T11:44:16.573 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.573 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.574 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.574 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.574 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.574 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.574 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.574 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.574 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.574 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.574 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-05-10T11:44:16.574 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.574 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.574 INFO:teuthology.orchestra.run.smithi100.stdout: "modules": [ 2024-05-10T11:44:16.574 INFO:teuthology.orchestra.run.smithi100.stdout: "cephadm", 2024-05-10T11:44:16.574 INFO:teuthology.orchestra.run.smithi100.stdout: "dashboard", 2024-05-10T11:44:16.574 INFO:teuthology.orchestra.run.smithi100.stdout: "iostat", 2024-05-10T11:44:16.575 INFO:teuthology.orchestra.run.smithi100.stdout: "nfs", 2024-05-10T11:44:16.575 INFO:teuthology.orchestra.run.smithi100.stdout: "restful" 2024-05-10T11:44:16.575 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.575 INFO:teuthology.orchestra.run.smithi100.stdout: "available_modules": [ 2024-05-10T11:44:16.575 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.575 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "alerts", 2024-05-10T11:44:16.575 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.575 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.575 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.575 INFO:teuthology.orchestra.run.smithi100.stdout: "interval": { 2024-05-10T11:44:16.575 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "interval", 2024-05-10T11:44:16.575 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-05-10T11:44:16.575 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.575 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.575 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "60", 2024-05-10T11:44:16.576 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.576 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.576 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.576 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "How frequently to reexamine health status", 2024-05-10T11:44:16.576 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.576 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.576 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.576 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.576 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.576 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.576 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.576 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.576 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.576 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.577 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.577 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.577 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.577 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.577 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.577 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.577 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.577 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.577 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.577 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.577 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.577 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.577 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.577 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.577 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.578 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.578 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.578 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.578 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.578 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.578 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.578 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.578 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.578 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.578 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.578 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.578 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.578 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.578 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.578 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.579 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.579 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.579 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.579 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.579 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.579 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.579 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.579 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.579 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.579 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.579 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.579 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.579 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.579 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.579 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.580 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.580 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.580 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.580 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.580 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.580 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.580 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.580 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.580 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.580 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.580 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.580 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.580 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.580 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.580 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.581 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.581 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.581 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.581 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.581 INFO:teuthology.orchestra.run.smithi100.stdout: "smtp_destination": { 2024-05-10T11:44:16.581 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smtp_destination", 2024-05-10T11:44:16.581 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.581 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.581 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.581 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.581 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.581 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.581 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.581 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Email address to send alerts to", 2024-05-10T11:44:16.581 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.582 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.582 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.582 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.582 INFO:teuthology.orchestra.run.smithi100.stdout: "smtp_from_name": { 2024-05-10T11:44:16.582 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smtp_from_name", 2024-05-10T11:44:16.582 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.582 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.582 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.582 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "Ceph", 2024-05-10T11:44:16.582 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.582 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.582 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.582 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Email From: name", 2024-05-10T11:44:16.582 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.583 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.583 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.583 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.583 INFO:teuthology.orchestra.run.smithi100.stdout: "smtp_host": { 2024-05-10T11:44:16.583 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smtp_host", 2024-05-10T11:44:16.583 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.583 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.583 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.583 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.583 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.583 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.583 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.583 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "SMTP server", 2024-05-10T11:44:16.583 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.584 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.584 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.584 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.584 INFO:teuthology.orchestra.run.smithi100.stdout: "smtp_password": { 2024-05-10T11:44:16.584 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smtp_password", 2024-05-10T11:44:16.584 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.584 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.584 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.584 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.584 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.584 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.584 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.584 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Password to authenticate with", 2024-05-10T11:44:16.584 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.584 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.585 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.585 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.585 INFO:teuthology.orchestra.run.smithi100.stdout: "smtp_port": { 2024-05-10T11:44:16.585 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smtp_port", 2024-05-10T11:44:16.585 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.585 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.585 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.585 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "465", 2024-05-10T11:44:16.585 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.585 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.585 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.585 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "SMTP port", 2024-05-10T11:44:16.585 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.585 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.585 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.586 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.586 INFO:teuthology.orchestra.run.smithi100.stdout: "smtp_sender": { 2024-05-10T11:44:16.586 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smtp_sender", 2024-05-10T11:44:16.586 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.586 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.586 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.586 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.586 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.586 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.586 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.586 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "SMTP envelope sender", 2024-05-10T11:44:16.586 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.586 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.586 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.586 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.587 INFO:teuthology.orchestra.run.smithi100.stdout: "smtp_ssl": { 2024-05-10T11:44:16.587 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smtp_ssl", 2024-05-10T11:44:16.587 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.587 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.587 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.587 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.587 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.587 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.587 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.587 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-10T11:44:16.587 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.587 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.587 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.587 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.588 INFO:teuthology.orchestra.run.smithi100.stdout: "smtp_user": { 2024-05-10T11:44:16.588 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smtp_user", 2024-05-10T11:44:16.588 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.588 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.588 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.588 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.588 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.588 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.588 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.588 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "User to authenticate as", 2024-05-10T11:44:16.588 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.588 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.588 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.588 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.588 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.589 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.589 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.589 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.589 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.589 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.589 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.589 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.589 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.589 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.589 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.589 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.589 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.589 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.589 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.589 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.590 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.590 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "balancer", 2024-05-10T11:44:16.590 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.590 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.590 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.590 INFO:teuthology.orchestra.run.smithi100.stdout: "active": { 2024-05-10T11:44:16.590 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "active", 2024-05-10T11:44:16.590 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.590 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.590 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.590 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.590 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.590 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.591 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.591 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "automatically balance PGs across cluster", 2024-05-10T11:44:16.591 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.591 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.591 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.591 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.591 INFO:teuthology.orchestra.run.smithi100.stdout: "begin_time": { 2024-05-10T11:44:16.591 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "begin_time", 2024-05-10T11:44:16.591 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.591 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.591 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.591 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0000", 2024-05-10T11:44:16.591 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.591 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.591 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.592 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "beginning time of day to automatically balance", 2024-05-10T11:44:16.592 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-10T11:44:16.592 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.592 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.592 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.592 INFO:teuthology.orchestra.run.smithi100.stdout: "begin_weekday": { 2024-05-10T11:44:16.592 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "begin_weekday", 2024-05-10T11:44:16.592 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "uint", 2024-05-10T11:44:16.592 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.592 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.592 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.592 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "0", 2024-05-10T11:44:16.592 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "6", 2024-05-10T11:44:16.592 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.592 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-10T11:44:16.593 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-10T11:44:16.593 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.593 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.593 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.593 INFO:teuthology.orchestra.run.smithi100.stdout: "crush_compat_max_iterations": { 2024-05-10T11:44:16.593 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "crush_compat_max_iterations", 2024-05-10T11:44:16.593 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "uint", 2024-05-10T11:44:16.593 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.593 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.593 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "25", 2024-05-10T11:44:16.593 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "1", 2024-05-10T11:44:16.593 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "250", 2024-05-10T11:44:16.593 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.593 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-10T11:44:16.593 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.594 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.594 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.594 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.594 INFO:teuthology.orchestra.run.smithi100.stdout: "crush_compat_metrics": { 2024-05-10T11:44:16.594 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "crush_compat_metrics", 2024-05-10T11:44:16.594 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.594 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.594 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.594 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "pgs,objects,bytes", 2024-05-10T11:44:16.594 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.594 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.594 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.594 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-10T11:44:16.594 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-10T11:44:16.595 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.595 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.595 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.595 INFO:teuthology.orchestra.run.smithi100.stdout: "crush_compat_step": { 2024-05-10T11:44:16.595 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "crush_compat_step", 2024-05-10T11:44:16.595 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "float", 2024-05-10T11:44:16.595 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.595 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.595 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0.5", 2024-05-10T11:44:16.595 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "0.001", 2024-05-10T11:44:16.595 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "0.999", 2024-05-10T11:44:16.595 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.595 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "aggressiveness of optimization", 2024-05-10T11:44:16.595 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-10T11:44:16.595 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.596 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.596 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.596 INFO:teuthology.orchestra.run.smithi100.stdout: "end_time": { 2024-05-10T11:44:16.596 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "end_time", 2024-05-10T11:44:16.596 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.596 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.596 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.596 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "2359", 2024-05-10T11:44:16.596 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.596 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.596 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.596 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "ending time of day to automatically balance", 2024-05-10T11:44:16.596 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-10T11:44:16.596 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.596 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.597 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.597 INFO:teuthology.orchestra.run.smithi100.stdout: "end_weekday": { 2024-05-10T11:44:16.597 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "end_weekday", 2024-05-10T11:44:16.597 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "uint", 2024-05-10T11:44:16.597 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.597 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.597 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.597 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "0", 2024-05-10T11:44:16.597 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "6", 2024-05-10T11:44:16.597 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.597 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-10T11:44:16.597 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-10T11:44:16.597 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.597 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.597 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.598 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.598 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.598 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.598 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.598 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.598 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.598 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.598 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.598 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.598 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.598 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.598 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.598 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.598 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.599 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.599 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.599 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.599 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.599 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.599 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.599 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.599 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.599 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.599 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.599 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.599 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.599 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.599 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.600 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.600 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.600 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.600 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.600 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.600 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.600 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.600 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.600 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.600 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.600 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.600 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.600 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.600 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.600 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.601 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.601 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.601 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.601 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.601 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.601 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.601 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.601 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.601 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.601 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.601 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.601 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.601 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.601 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.602 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.602 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.602 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.602 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.602 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.602 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.602 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.602 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.602 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.602 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.602 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.602 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.602 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.602 INFO:teuthology.orchestra.run.smithi100.stdout: "min_score": { 2024-05-10T11:44:16.602 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "min_score", 2024-05-10T11:44:16.603 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "float", 2024-05-10T11:44:16.603 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.603 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.603 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.603 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.603 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.603 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.603 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-10T11:44:16.603 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.603 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.603 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.603 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.603 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": { 2024-05-10T11:44:16.604 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "mode", 2024-05-10T11:44:16.604 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.604 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.604 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.604 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "upmap", 2024-05-10T11:44:16.604 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.604 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.604 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.604 INFO:teuthology.orchestra.run.smithi100.stdout: "crush-compat", 2024-05-10T11:44:16.604 INFO:teuthology.orchestra.run.smithi100.stdout: "none", 2024-05-10T11:44:16.604 INFO:teuthology.orchestra.run.smithi100.stdout: "read", 2024-05-10T11:44:16.604 INFO:teuthology.orchestra.run.smithi100.stdout: "upmap", 2024-05-10T11:44:16.604 INFO:teuthology.orchestra.run.smithi100.stdout: "upmap-read" 2024-05-10T11:44:16.604 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.604 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Balancer mode", 2024-05-10T11:44:16.605 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.605 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.605 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.605 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.605 INFO:teuthology.orchestra.run.smithi100.stdout: "pool_ids": { 2024-05-10T11:44:16.605 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "pool_ids", 2024-05-10T11:44:16.605 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.605 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.605 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.605 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.605 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.605 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.605 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.605 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-10T11:44:16.605 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.606 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.606 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.606 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.606 INFO:teuthology.orchestra.run.smithi100.stdout: "sleep_interval": { 2024-05-10T11:44:16.606 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sleep_interval", 2024-05-10T11:44:16.606 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-05-10T11:44:16.606 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.606 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.606 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "60", 2024-05-10T11:44:16.606 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.606 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.606 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.606 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-10T11:44:16.606 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.606 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.607 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.607 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.607 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.607 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.607 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.607 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.607 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.607 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.607 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.607 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.607 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.607 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.607 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.607 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.608 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.608 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.608 INFO:teuthology.orchestra.run.smithi100.stdout: "upmap_max_deviation": { 2024-05-10T11:44:16.608 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "upmap_max_deviation", 2024-05-10T11:44:16.608 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.608 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.608 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.608 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "5", 2024-05-10T11:44:16.608 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "1", 2024-05-10T11:44:16.608 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.608 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.608 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-10T11:44:16.608 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-10T11:44:16.608 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.608 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.609 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.609 INFO:teuthology.orchestra.run.smithi100.stdout: "upmap_max_optimizations": { 2024-05-10T11:44:16.609 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "upmap_max_optimizations", 2024-05-10T11:44:16.609 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "uint", 2024-05-10T11:44:16.609 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.609 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.609 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10", 2024-05-10T11:44:16.609 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.609 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.609 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.609 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-10T11:44:16.609 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.609 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.609 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.609 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.609 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.610 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.610 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.610 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "cephadm", 2024-05-10T11:44:16.610 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.610 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.610 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.610 INFO:teuthology.orchestra.run.smithi100.stdout: "agent_down_multiplier": { 2024-05-10T11:44:16.610 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "agent_down_multiplier", 2024-05-10T11:44:16.610 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "float", 2024-05-10T11:44:16.610 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.610 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.610 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "3.0", 2024-05-10T11:44:16.610 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.610 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.611 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.611 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-10T11:44:16.611 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.611 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.611 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.611 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.611 INFO:teuthology.orchestra.run.smithi100.stdout: "agent_refresh_rate": { 2024-05-10T11:44:16.611 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "agent_refresh_rate", 2024-05-10T11:44:16.611 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-05-10T11:44:16.611 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.611 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.611 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "20", 2024-05-10T11:44:16.611 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.611 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.612 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.612 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-10T11:44:16.612 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.612 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.612 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.612 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.612 INFO:teuthology.orchestra.run.smithi100.stdout: "agent_starting_port": { 2024-05-10T11:44:16.612 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "agent_starting_port", 2024-05-10T11:44:16.612 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.612 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.612 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.612 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "4721", 2024-05-10T11:44:16.612 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.612 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.612 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.613 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-10T11:44:16.613 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.613 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.613 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.613 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.613 INFO:teuthology.orchestra.run.smithi100.stdout: "allow_ptrace": { 2024-05-10T11:44:16.613 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "allow_ptrace", 2024-05-10T11:44:16.613 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.613 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.613 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.613 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.613 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.613 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.613 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.614 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-10T11:44:16.614 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-10T11:44:16.614 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.614 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.614 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.614 INFO:teuthology.orchestra.run.smithi100.stdout: "autotune_interval": { 2024-05-10T11:44:16.614 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "autotune_interval", 2024-05-10T11:44:16.614 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-05-10T11:44:16.614 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.614 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.614 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "600", 2024-05-10T11:44:16.614 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.614 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.614 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.615 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-10T11:44:16.615 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.615 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.615 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.615 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.615 INFO:teuthology.orchestra.run.smithi100.stdout: "autotune_memory_target_ratio": { 2024-05-10T11:44:16.615 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "autotune_memory_target_ratio", 2024-05-10T11:44:16.615 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "float", 2024-05-10T11:44:16.615 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.615 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.615 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0.7", 2024-05-10T11:44:16.615 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.615 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.615 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.616 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-10T11:44:16.616 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.616 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.616 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.616 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.616 INFO:teuthology.orchestra.run.smithi100.stdout: "cephadm_log_destination": { 2024-05-10T11:44:16.616 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "cephadm_log_destination", 2024-05-10T11:44:16.616 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.616 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.616 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.616 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.616 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.616 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.616 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.616 INFO:teuthology.orchestra.run.smithi100.stdout: "file", 2024-05-10T11:44:16.617 INFO:teuthology.orchestra.run.smithi100.stdout: "file,syslog", 2024-05-10T11:44:16.617 INFO:teuthology.orchestra.run.smithi100.stdout: "syslog" 2024-05-10T11:44:16.617 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.617 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-10T11:44:16.617 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.617 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.617 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.617 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.617 INFO:teuthology.orchestra.run.smithi100.stdout: "cgroups_split": { 2024-05-10T11:44:16.617 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "cgroups_split", 2024-05-10T11:44:16.617 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.617 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.617 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.617 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.617 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.618 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.618 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.618 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-10T11:44:16.618 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.618 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.618 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.618 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.618 INFO:teuthology.orchestra.run.smithi100.stdout: "config_checks_enabled": { 2024-05-10T11:44:16.618 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "config_checks_enabled", 2024-05-10T11:44:16.618 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.618 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.618 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.618 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.618 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.619 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.619 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.619 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-10T11:44:16.619 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.619 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.619 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.619 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.619 INFO:teuthology.orchestra.run.smithi100.stdout: "config_dashboard": { 2024-05-10T11:44:16.619 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "config_dashboard", 2024-05-10T11:44:16.619 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.619 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.619 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.619 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.619 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.619 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.619 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.620 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-10T11:44:16.620 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.620 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.620 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.620 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.620 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_alertmanager": { 2024-05-10T11:44:16.620 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_alertmanager", 2024-05-10T11:44:16.620 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.620 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.620 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.620 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-10T11:44:16.620 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.620 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.620 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.621 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Prometheus container image", 2024-05-10T11:44:16.621 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.621 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.621 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.621 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.621 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_base": { 2024-05-10T11:44:16.621 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_base", 2024-05-10T11:44:16.621 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.621 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.621 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.621 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-10T11:44:16.621 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.621 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.621 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.621 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Container image name, without the tag", 2024-05-10T11:44:16.622 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.622 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.622 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.622 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.622 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_elasticsearch": { 2024-05-10T11:44:16.622 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_elasticsearch", 2024-05-10T11:44:16.622 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.622 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.622 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.622 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-10T11:44:16.622 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.622 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.622 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.622 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "elasticsearch container image", 2024-05-10T11:44:16.622 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.623 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.623 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.623 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.623 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_grafana": { 2024-05-10T11:44:16.623 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_grafana", 2024-05-10T11:44:16.623 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.623 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.623 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.623 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-10T11:44:16.623 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.623 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.623 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.623 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Prometheus container image", 2024-05-10T11:44:16.623 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.623 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.624 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.624 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.624 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_haproxy": { 2024-05-10T11:44:16.624 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_haproxy", 2024-05-10T11:44:16.624 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.624 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.624 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.624 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-10T11:44:16.624 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.624 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.624 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.624 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "HAproxy container image", 2024-05-10T11:44:16.624 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.624 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.624 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.625 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.625 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_jaeger_agent": { 2024-05-10T11:44:16.625 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_jaeger_agent", 2024-05-10T11:44:16.625 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.625 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.625 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.625 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-10T11:44:16.625 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.625 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.625 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.625 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Jaeger agent container image", 2024-05-10T11:44:16.625 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.625 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.625 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.626 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.626 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_jaeger_collector": { 2024-05-10T11:44:16.626 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_jaeger_collector", 2024-05-10T11:44:16.626 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.626 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.626 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.626 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-10T11:44:16.626 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.626 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.626 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.626 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Jaeger collector container image", 2024-05-10T11:44:16.626 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.627 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.627 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.627 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.627 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_jaeger_query": { 2024-05-10T11:44:16.627 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_jaeger_query", 2024-05-10T11:44:16.627 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.627 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.627 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.627 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-10T11:44:16.627 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.627 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.627 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.627 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Jaeger query container image", 2024-05-10T11:44:16.627 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.628 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.628 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.628 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.628 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_keepalived": { 2024-05-10T11:44:16.628 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_keepalived", 2024-05-10T11:44:16.628 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.628 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.628 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.628 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-10T11:44:16.628 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.628 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.628 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.628 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Keepalived container image", 2024-05-10T11:44:16.628 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.629 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.629 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.629 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.629 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_loki": { 2024-05-10T11:44:16.629 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_loki", 2024-05-10T11:44:16.629 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.629 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.629 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.629 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-10T11:44:16.629 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.629 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.629 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.629 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Loki container image", 2024-05-10T11:44:16.629 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.630 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.630 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.630 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.630 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_node_exporter": { 2024-05-10T11:44:16.630 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_node_exporter", 2024-05-10T11:44:16.630 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.630 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.630 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.630 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-10T11:44:16.630 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.630 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.630 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.630 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Prometheus container image", 2024-05-10T11:44:16.631 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.631 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.631 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.631 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.631 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_nvmeof": { 2024-05-10T11:44:16.631 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_nvmeof", 2024-05-10T11:44:16.631 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.631 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.631 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.631 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-10T11:44:16.631 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.631 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.631 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.631 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Nvme-of container image", 2024-05-10T11:44:16.632 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.632 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.632 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.632 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.632 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_prometheus": { 2024-05-10T11:44:16.632 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_prometheus", 2024-05-10T11:44:16.632 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.632 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.632 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.632 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-10T11:44:16.632 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.632 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.632 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.632 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Prometheus container image", 2024-05-10T11:44:16.633 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.633 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.633 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.633 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.633 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_promtail": { 2024-05-10T11:44:16.633 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_promtail", 2024-05-10T11:44:16.633 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.633 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.633 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.633 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-10T11:44:16.633 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.633 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.633 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.633 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Promtail container image", 2024-05-10T11:44:16.633 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.634 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.634 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.634 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.634 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_samba": { 2024-05-10T11:44:16.634 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_samba", 2024-05-10T11:44:16.634 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.634 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.634 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.634 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-10T11:44:16.634 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.634 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.634 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.634 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Samba/SMB container image", 2024-05-10T11:44:16.634 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.635 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.635 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.635 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.635 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image_snmp_gateway": { 2024-05-10T11:44:16.635 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_image_snmp_gateway", 2024-05-10T11:44:16.635 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.635 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.635 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.635 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-10T11:44:16.635 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.635 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.635 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.635 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "SNMP Gateway container image", 2024-05-10T11:44:16.635 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.635 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.636 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.636 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.636 INFO:teuthology.orchestra.run.smithi100.stdout: "container_init": { 2024-05-10T11:44:16.636 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "container_init", 2024-05-10T11:44:16.636 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.636 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.636 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.636 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.636 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.636 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.636 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.636 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Run podman/docker with `--init`", 2024-05-10T11:44:16.636 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.636 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.636 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.637 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.637 INFO:teuthology.orchestra.run.smithi100.stdout: "daemon_cache_timeout": { 2024-05-10T11:44:16.637 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "daemon_cache_timeout", 2024-05-10T11:44:16.637 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-05-10T11:44:16.637 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.637 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.637 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "600", 2024-05-10T11:44:16.637 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.637 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.637 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.637 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-10T11:44:16.637 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.637 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.638 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.638 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.638 INFO:teuthology.orchestra.run.smithi100.stdout: "default_cephadm_command_timeout": { 2024-05-10T11:44:16.638 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "default_cephadm_command_timeout", 2024-05-10T11:44:16.638 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.638 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.638 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.638 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "900", 2024-05-10T11:44:16.638 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.638 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.638 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.638 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-10T11:44:16.638 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.638 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.639 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.639 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.639 INFO:teuthology.orchestra.run.smithi100.stdout: "default_registry": { 2024-05-10T11:44:16.639 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "default_registry", 2024-05-10T11:44:16.639 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.639 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.639 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.639 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "docker.io", 2024-05-10T11:44:16.639 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.639 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.639 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.639 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-10T11:44:16.640 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.640 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.640 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.640 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.640 INFO:teuthology.orchestra.run.smithi100.stdout: "device_cache_timeout": { 2024-05-10T11:44:16.640 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "device_cache_timeout", 2024-05-10T11:44:16.640 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-05-10T11:44:16.640 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.640 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.640 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "1800", 2024-05-10T11:44:16.640 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.640 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.640 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.640 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "seconds to cache device inventory", 2024-05-10T11:44:16.641 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.641 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.641 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.641 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.641 INFO:teuthology.orchestra.run.smithi100.stdout: "device_enhanced_scan": { 2024-05-10T11:44:16.641 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "device_enhanced_scan", 2024-05-10T11:44:16.641 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.641 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.641 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.641 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.641 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.641 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.641 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.641 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-10T11:44:16.642 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.642 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.642 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.642 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.642 INFO:teuthology.orchestra.run.smithi100.stdout: "facts_cache_timeout": { 2024-05-10T11:44:16.642 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "facts_cache_timeout", 2024-05-10T11:44:16.642 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-05-10T11:44:16.642 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.642 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.642 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "60", 2024-05-10T11:44:16.642 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.642 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.642 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.642 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "seconds to cache host facts data", 2024-05-10T11:44:16.643 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.643 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.643 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.643 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.643 INFO:teuthology.orchestra.run.smithi100.stdout: "grafana_dashboards_path": { 2024-05-10T11:44:16.643 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "grafana_dashboards_path", 2024-05-10T11:44:16.643 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.643 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.643 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.643 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-10T11:44:16.643 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.643 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.643 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.643 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-10T11:44:16.644 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.644 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.644 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.644 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.644 INFO:teuthology.orchestra.run.smithi100.stdout: "host_check_interval": { 2024-05-10T11:44:16.644 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "host_check_interval", 2024-05-10T11:44:16.644 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-05-10T11:44:16.644 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.644 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.644 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "600", 2024-05-10T11:44:16.644 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.644 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.644 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.644 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "how frequently to perform a host check", 2024-05-10T11:44:16.645 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.645 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.645 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.645 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.645 INFO:teuthology.orchestra.run.smithi100.stdout: "hw_monitoring": { 2024-05-10T11:44:16.645 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "hw_monitoring", 2024-05-10T11:44:16.645 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.645 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.645 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.645 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.645 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.645 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.645 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.645 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-10T11:44:16.645 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.646 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.646 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.646 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.646 INFO:teuthology.orchestra.run.smithi100.stdout: "inventory_list_all": { 2024-05-10T11:44:16.646 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "inventory_list_all", 2024-05-10T11:44:16.646 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.646 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.646 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.646 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.646 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.646 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.646 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.646 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-10T11:44:16.646 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.647 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.647 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.647 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.647 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.647 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.647 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.647 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.647 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.647 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.647 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.647 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.647 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.647 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.647 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.648 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.648 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.648 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.648 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.648 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.648 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.648 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.648 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.648 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.648 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.648 INFO:teuthology.orchestra.run.smithi100.stdout: "log_refresh_metadata": { 2024-05-10T11:44:16.648 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_refresh_metadata", 2024-05-10T11:44:16.648 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.648 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.648 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.649 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.649 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.649 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.649 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.649 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-10T11:44:16.649 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.649 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.649 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.649 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.649 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.649 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.649 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.649 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.649 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.649 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.650 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.650 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.650 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.650 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-10T11:44:16.650 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.650 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.650 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.650 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.650 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.650 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.650 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.650 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.650 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.650 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.650 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.651 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.651 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.651 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.651 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.651 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.651 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.651 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.651 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.651 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.651 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.651 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.651 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.651 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.651 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.652 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.652 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.652 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.652 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.652 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.652 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.652 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.652 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.652 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.652 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.652 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.652 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.652 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.652 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.653 INFO:teuthology.orchestra.run.smithi100.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-10T11:44:16.653 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-10T11:44:16.653 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.653 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.653 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.653 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.653 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.653 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.653 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.653 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-10T11:44:16.653 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.653 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.653 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.653 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.654 INFO:teuthology.orchestra.run.smithi100.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-10T11:44:16.654 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-10T11:44:16.654 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.654 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.654 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.654 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "*", 2024-05-10T11:44:16.654 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.654 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.654 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.654 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-10T11:44:16.654 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.654 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.654 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.654 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.654 INFO:teuthology.orchestra.run.smithi100.stdout: "max_count_per_host": { 2024-05-10T11:44:16.655 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "max_count_per_host", 2024-05-10T11:44:16.655 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.655 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.655 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.655 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10", 2024-05-10T11:44:16.655 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.655 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.655 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.655 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "max number of daemons per service per host", 2024-05-10T11:44:16.655 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.655 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.655 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.655 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.655 INFO:teuthology.orchestra.run.smithi100.stdout: "max_osd_draining_count": { 2024-05-10T11:44:16.656 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "max_osd_draining_count", 2024-05-10T11:44:16.656 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.656 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.656 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.656 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10", 2024-05-10T11:44:16.656 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.656 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.656 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.656 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-10T11:44:16.656 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.656 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.657 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.657 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.657 INFO:teuthology.orchestra.run.smithi100.stdout: "migration_current": { 2024-05-10T11:44:16.657 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "migration_current", 2024-05-10T11:44:16.657 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.657 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.657 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.657 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.657 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.657 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.657 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.657 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "internal - do not modify", 2024-05-10T11:44:16.657 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.657 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.658 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.658 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.658 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": { 2024-05-10T11:44:16.658 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "mode", 2024-05-10T11:44:16.658 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.658 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.658 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.658 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "root", 2024-05-10T11:44:16.658 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.658 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.658 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.658 INFO:teuthology.orchestra.run.smithi100.stdout: "cephadm-package", 2024-05-10T11:44:16.658 INFO:teuthology.orchestra.run.smithi100.stdout: "root" 2024-05-10T11:44:16.658 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.658 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "mode for remote execution of cephadm", 2024-05-10T11:44:16.659 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.659 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.659 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.659 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.659 INFO:teuthology.orchestra.run.smithi100.stdout: "oob_default_addr": { 2024-05-10T11:44:16.659 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "oob_default_addr", 2024-05-10T11:44:16.659 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.659 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.659 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.659 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "169.254.1.1", 2024-05-10T11:44:16.659 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.659 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.659 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.659 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-10T11:44:16.659 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.659 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.660 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.660 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.660 INFO:teuthology.orchestra.run.smithi100.stdout: "prometheus_alerts_path": { 2024-05-10T11:44:16.660 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "prometheus_alerts_path", 2024-05-10T11:44:16.660 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.660 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.660 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.660 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-10T11:44:16.660 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.660 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.660 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.660 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-10T11:44:16.660 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.660 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.661 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.661 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.661 INFO:teuthology.orchestra.run.smithi100.stdout: "registry_insecure": { 2024-05-10T11:44:16.661 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "registry_insecure", 2024-05-10T11:44:16.661 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.661 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.661 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.661 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.661 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.661 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.661 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.661 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-10T11:44:16.661 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.661 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.661 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.662 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.662 INFO:teuthology.orchestra.run.smithi100.stdout: "registry_password": { 2024-05-10T11:44:16.662 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "registry_password", 2024-05-10T11:44:16.662 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.662 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.662 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.662 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.662 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.662 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.662 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.662 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-10T11:44:16.662 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.662 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.662 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.662 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.663 INFO:teuthology.orchestra.run.smithi100.stdout: "registry_url": { 2024-05-10T11:44:16.663 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "registry_url", 2024-05-10T11:44:16.663 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.663 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.663 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.663 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.663 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.663 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.663 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.663 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-10T11:44:16.663 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.663 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.663 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.663 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.663 INFO:teuthology.orchestra.run.smithi100.stdout: "registry_username": { 2024-05-10T11:44:16.664 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "registry_username", 2024-05-10T11:44:16.664 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.664 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.664 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.664 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.664 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.664 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.664 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.664 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-10T11:44:16.664 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.664 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.664 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.664 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.664 INFO:teuthology.orchestra.run.smithi100.stdout: "secure_monitoring_stack": { 2024-05-10T11:44:16.664 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "secure_monitoring_stack", 2024-05-10T11:44:16.665 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.665 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.665 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.665 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.665 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.665 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.665 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.665 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-10T11:44:16.665 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.665 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.665 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.665 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.665 INFO:teuthology.orchestra.run.smithi100.stdout: "service_discovery_port": { 2024-05-10T11:44:16.665 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "service_discovery_port", 2024-05-10T11:44:16.665 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.665 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.666 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.666 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "8765", 2024-05-10T11:44:16.666 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.666 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.666 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.666 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "cephadm service discovery port", 2024-05-10T11:44:16.666 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.666 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.666 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.666 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.666 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.666 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.666 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.667 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.667 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.667 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.667 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.667 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.667 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.667 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.667 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.667 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.667 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.667 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.667 INFO:teuthology.orchestra.run.smithi100.stdout: "ssh_config_file": { 2024-05-10T11:44:16.667 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ssh_config_file", 2024-05-10T11:44:16.667 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.667 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.668 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.668 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.668 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.668 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.668 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.668 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-10T11:44:16.668 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.668 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.668 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.668 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.668 INFO:teuthology.orchestra.run.smithi100.stdout: "use_agent": { 2024-05-10T11:44:16.668 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "use_agent", 2024-05-10T11:44:16.668 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.668 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.668 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.669 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.669 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.669 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.669 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.669 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-10T11:44:16.669 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.669 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.669 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.669 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.669 INFO:teuthology.orchestra.run.smithi100.stdout: "use_repo_digest": { 2024-05-10T11:44:16.669 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "use_repo_digest", 2024-05-10T11:44:16.669 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.669 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.669 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.669 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.670 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.670 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.670 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.670 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-10T11:44:16.670 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.670 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.670 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.670 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.670 INFO:teuthology.orchestra.run.smithi100.stdout: "warn_on_failed_host_check": { 2024-05-10T11:44:16.670 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "warn_on_failed_host_check", 2024-05-10T11:44:16.670 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.670 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.670 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.671 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.671 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.671 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.671 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.671 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "raise a health warning if the host check fails", 2024-05-10T11:44:16.671 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.671 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.671 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.671 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.671 INFO:teuthology.orchestra.run.smithi100.stdout: "warn_on_stray_daemons": { 2024-05-10T11:44:16.671 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "warn_on_stray_daemons", 2024-05-10T11:44:16.671 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.671 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.671 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.671 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.672 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.672 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.672 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.672 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-10T11:44:16.672 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.672 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.672 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.672 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.672 INFO:teuthology.orchestra.run.smithi100.stdout: "warn_on_stray_hosts": { 2024-05-10T11:44:16.672 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "warn_on_stray_hosts", 2024-05-10T11:44:16.672 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.672 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.672 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.672 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.672 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.673 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.673 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.673 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-10T11:44:16.673 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.673 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.673 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.673 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.673 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.673 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.673 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.673 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "crash", 2024-05-10T11:44:16.673 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.673 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.673 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.673 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.674 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.674 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.674 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.674 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.674 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.674 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.674 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.674 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.674 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.674 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.674 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.674 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.674 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.674 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.674 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.675 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.675 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.675 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.675 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.675 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.675 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.675 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.675 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.675 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.675 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.675 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.675 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.675 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.675 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.676 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.676 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.676 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.676 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.676 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.676 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.676 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.676 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.676 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.676 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.676 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.676 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.676 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.676 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.676 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.677 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.677 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.677 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.677 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.677 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.677 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.677 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.677 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.677 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.677 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.677 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.677 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.677 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.677 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.678 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.678 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.678 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.678 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.678 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.678 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.678 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.678 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.678 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.678 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.678 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.678 INFO:teuthology.orchestra.run.smithi100.stdout: "retain_interval": { 2024-05-10T11:44:16.678 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "retain_interval", 2024-05-10T11:44:16.678 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-05-10T11:44:16.678 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.679 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.679 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "31536000", 2024-05-10T11:44:16.679 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.679 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.679 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.679 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-10T11:44:16.679 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.679 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.679 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.679 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.679 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.679 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.679 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.680 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.680 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.680 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.680 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.680 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.680 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.680 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.680 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.680 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.680 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.680 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.680 INFO:teuthology.orchestra.run.smithi100.stdout: "warn_recent_interval": { 2024-05-10T11:44:16.680 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "warn_recent_interval", 2024-05-10T11:44:16.680 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-05-10T11:44:16.681 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.681 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.681 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "1209600", 2024-05-10T11:44:16.681 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.681 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.681 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.681 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-10T11:44:16.681 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.681 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.681 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.681 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.681 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.681 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.681 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.681 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "dashboard", 2024-05-10T11:44:16.681 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.682 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.682 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.682 INFO:teuthology.orchestra.run.smithi100.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-10T11:44:16.682 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-10T11:44:16.682 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.682 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.682 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.682 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10", 2024-05-10T11:44:16.682 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.682 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.682 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.682 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.682 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.682 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.683 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.683 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.683 INFO:teuthology.orchestra.run.smithi100.stdout: "ALERTMANAGER_API_HOST": { 2024-05-10T11:44:16.683 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-10T11:44:16.683 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.683 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.683 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.683 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.683 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.683 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.683 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.683 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.683 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.683 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.683 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.684 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.684 INFO:teuthology.orchestra.run.smithi100.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-10T11:44:16.684 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-10T11:44:16.684 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.684 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.684 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.684 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.684 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.684 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.684 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.684 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.684 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.684 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.684 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.684 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.685 INFO:teuthology.orchestra.run.smithi100.stdout: "AUDIT_API_ENABLED": { 2024-05-10T11:44:16.685 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "AUDIT_API_ENABLED", 2024-05-10T11:44:16.685 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.685 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.685 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.685 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.685 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.685 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.685 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.685 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.685 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.685 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.685 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.685 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.685 INFO:teuthology.orchestra.run.smithi100.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-10T11:44:16.686 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-10T11:44:16.686 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.686 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.686 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.686 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.686 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.686 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.686 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.686 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.686 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.686 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.686 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.686 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.686 INFO:teuthology.orchestra.run.smithi100.stdout: "ENABLE_BROWSABLE_API": { 2024-05-10T11:44:16.687 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-10T11:44:16.687 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.687 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.687 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.687 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.687 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.687 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.687 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.687 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.687 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.687 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.687 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.687 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.687 INFO:teuthology.orchestra.run.smithi100.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-10T11:44:16.687 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-10T11:44:16.688 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.688 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.688 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.688 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.688 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.688 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.688 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.688 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.688 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.688 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.688 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.688 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.688 INFO:teuthology.orchestra.run.smithi100.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-10T11:44:16.688 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-10T11:44:16.688 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.688 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.689 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.689 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.689 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.689 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.689 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.689 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.689 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.689 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.689 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.689 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.689 INFO:teuthology.orchestra.run.smithi100.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-10T11:44:16.689 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-10T11:44:16.689 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.689 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.689 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.690 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.690 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.690 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.690 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.690 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.690 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.690 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.690 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.690 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.690 INFO:teuthology.orchestra.run.smithi100.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-10T11:44:16.690 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-10T11:44:16.690 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.690 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.690 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.691 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.691 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.691 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.691 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.691 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.691 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.691 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.691 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.691 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.692 INFO:teuthology.orchestra.run.smithi100.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-10T11:44:16.692 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-10T11:44:16.692 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.692 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.692 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.692 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.692 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.692 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.692 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.693 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.693 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.693 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.693 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.693 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.693 INFO:teuthology.orchestra.run.smithi100.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-10T11:44:16.693 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-10T11:44:16.693 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.693 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.693 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.693 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.693 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.693 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.693 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.693 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.694 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.694 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.694 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.694 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.694 INFO:teuthology.orchestra.run.smithi100.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-10T11:44:16.694 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-10T11:44:16.694 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.694 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.694 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.694 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.694 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.694 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.694 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.694 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.694 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.695 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.695 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.695 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.695 INFO:teuthology.orchestra.run.smithi100.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-10T11:44:16.695 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-10T11:44:16.695 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.695 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.695 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.695 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.695 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.695 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.695 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.695 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.695 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.695 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.696 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.696 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.696 INFO:teuthology.orchestra.run.smithi100.stdout: "GRAFANA_API_PASSWORD": { 2024-05-10T11:44:16.696 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-10T11:44:16.696 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.696 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.696 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.696 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "admin", 2024-05-10T11:44:16.696 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.696 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.696 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.696 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.696 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.697 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.697 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.697 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.697 INFO:teuthology.orchestra.run.smithi100.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-10T11:44:16.697 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-10T11:44:16.697 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.697 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.697 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.697 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.697 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.697 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.697 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.697 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.697 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.697 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.698 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.698 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.698 INFO:teuthology.orchestra.run.smithi100.stdout: "GRAFANA_API_URL": { 2024-05-10T11:44:16.698 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "GRAFANA_API_URL", 2024-05-10T11:44:16.698 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.698 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.698 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.698 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.698 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.698 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.698 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.698 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.698 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.698 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.698 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.699 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.699 INFO:teuthology.orchestra.run.smithi100.stdout: "GRAFANA_API_USERNAME": { 2024-05-10T11:44:16.699 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-10T11:44:16.699 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.699 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.699 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.699 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "admin", 2024-05-10T11:44:16.699 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.699 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.699 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.699 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.699 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.699 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.699 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.699 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.700 INFO:teuthology.orchestra.run.smithi100.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-10T11:44:16.700 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-10T11:44:16.700 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.700 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.700 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.700 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.700 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.700 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.700 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.700 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.700 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.700 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.700 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.701 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.701 INFO:teuthology.orchestra.run.smithi100.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-10T11:44:16.701 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-10T11:44:16.701 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.701 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.701 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.701 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.701 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.701 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.701 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.701 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.701 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.701 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.701 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.701 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.702 INFO:teuthology.orchestra.run.smithi100.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-10T11:44:16.702 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-10T11:44:16.702 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.702 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.702 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.702 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.702 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.702 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.702 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.702 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.702 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.702 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.702 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.703 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.703 INFO:teuthology.orchestra.run.smithi100.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-10T11:44:16.703 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-10T11:44:16.703 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.703 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.703 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.703 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.703 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.703 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.703 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.703 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.703 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.703 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.703 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.703 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.704 INFO:teuthology.orchestra.run.smithi100.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-10T11:44:16.704 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-10T11:44:16.704 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.704 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.704 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.704 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "[]", 2024-05-10T11:44:16.704 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.704 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.704 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.704 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.704 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.704 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.704 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.704 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.705 INFO:teuthology.orchestra.run.smithi100.stdout: "MULTICLUSTER_CONFIG": { 2024-05-10T11:44:16.705 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-10T11:44:16.705 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.705 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.705 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.705 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "{}", 2024-05-10T11:44:16.705 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.705 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.705 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.705 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.705 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.705 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.705 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.705 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.705 INFO:teuthology.orchestra.run.smithi100.stdout: "PROMETHEUS_API_HOST": { 2024-05-10T11:44:16.706 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-10T11:44:16.706 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.706 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.706 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.706 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.706 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.706 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.706 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.706 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.706 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.706 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.706 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.706 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.706 INFO:teuthology.orchestra.run.smithi100.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-10T11:44:16.706 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-10T11:44:16.707 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.707 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.707 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.707 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.707 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.707 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.707 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.707 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.707 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.707 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.707 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.707 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.707 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-10T11:44:16.707 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-10T11:44:16.708 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.708 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.708 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.708 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.708 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.708 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.708 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.708 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.708 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.708 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.708 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.708 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.708 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-10T11:44:16.708 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-10T11:44:16.709 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.709 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.709 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.709 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.709 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.709 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.709 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.709 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.709 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.709 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.709 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.709 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.709 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-10T11:44:16.709 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-10T11:44:16.710 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.710 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.710 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.710 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.710 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.710 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.710 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.710 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.710 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.710 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.710 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.710 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.710 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-10T11:44:16.710 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-10T11:44:16.710 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.711 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.711 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.711 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.711 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.711 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.711 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.711 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.711 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.711 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.711 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.711 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.711 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-10T11:44:16.712 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-10T11:44:16.712 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.712 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.712 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.712 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.712 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.712 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.712 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.712 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.712 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.712 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.712 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.712 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.712 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-10T11:44:16.712 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-10T11:44:16.713 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.713 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.713 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.713 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.713 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.713 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.713 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.713 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.713 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.713 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.713 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.713 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.713 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-10T11:44:16.713 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-10T11:44:16.713 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.714 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.714 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.714 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.714 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.714 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.714 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.714 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.714 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.714 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.714 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.714 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.714 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_ENABLED": { 2024-05-10T11:44:16.714 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-10T11:44:16.714 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.714 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.715 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.715 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.715 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.715 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.715 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.715 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.715 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.715 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.715 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.715 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.715 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-10T11:44:16.715 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-10T11:44:16.715 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.715 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.715 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.716 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-10T11:44:16.716 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.716 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.716 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.716 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.716 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.716 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.716 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.716 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.716 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-10T11:44:16.716 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-10T11:44:16.716 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.716 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.716 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.717 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10", 2024-05-10T11:44:16.717 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.717 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.717 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.717 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.717 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.717 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.717 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.717 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.717 INFO:teuthology.orchestra.run.smithi100.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-10T11:44:16.717 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-10T11:44:16.717 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.717 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.717 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.717 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "8", 2024-05-10T11:44:16.718 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.718 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.718 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.718 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.718 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.718 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.718 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.718 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.718 INFO:teuthology.orchestra.run.smithi100.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-10T11:44:16.718 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-10T11:44:16.718 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.718 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.718 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.718 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "45", 2024-05-10T11:44:16.719 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.719 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.719 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.719 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.722 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.722 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.722 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.722 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.722 INFO:teuthology.orchestra.run.smithi100.stdout: "RGW_API_ACCESS_KEY": { 2024-05-10T11:44:16.722 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-10T11:44:16.722 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.722 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.723 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.723 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.723 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.723 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.723 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.723 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.723 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.723 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.723 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.723 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.723 INFO:teuthology.orchestra.run.smithi100.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-10T11:44:16.723 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-10T11:44:16.723 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.723 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.724 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.724 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "admin", 2024-05-10T11:44:16.724 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.724 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.724 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.724 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.724 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.724 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.724 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.724 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.724 INFO:teuthology.orchestra.run.smithi100.stdout: "RGW_API_SECRET_KEY": { 2024-05-10T11:44:16.724 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-10T11:44:16.724 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.724 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.724 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.725 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.725 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.725 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.725 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.725 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.725 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.725 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.725 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.725 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.725 INFO:teuthology.orchestra.run.smithi100.stdout: "RGW_API_SSL_VERIFY": { 2024-05-10T11:44:16.725 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-10T11:44:16.725 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.725 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.725 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.725 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.726 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.726 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.726 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.726 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.726 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.726 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.726 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.726 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.726 INFO:teuthology.orchestra.run.smithi100.stdout: "UNSAFE_TLS_v1_2": { 2024-05-10T11:44:16.726 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-10T11:44:16.726 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.726 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.726 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.726 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.726 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.727 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.727 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.727 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.727 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.727 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.727 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.727 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.727 INFO:teuthology.orchestra.run.smithi100.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-10T11:44:16.727 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-10T11:44:16.727 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.727 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.727 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.727 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.727 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.727 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.728 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.728 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.728 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.728 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.728 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.728 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.728 INFO:teuthology.orchestra.run.smithi100.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-10T11:44:16.728 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-10T11:44:16.728 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.728 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.728 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.728 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10", 2024-05-10T11:44:16.728 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.729 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.729 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.729 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.729 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.729 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.729 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.729 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.729 INFO:teuthology.orchestra.run.smithi100.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-10T11:44:16.729 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-10T11:44:16.729 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.729 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.729 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.729 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "5", 2024-05-10T11:44:16.729 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.729 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.730 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.730 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.730 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.730 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.730 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.730 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.730 INFO:teuthology.orchestra.run.smithi100.stdout: "cross_origin_url": { 2024-05-10T11:44:16.730 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "cross_origin_url", 2024-05-10T11:44:16.730 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.730 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.730 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.730 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.730 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.730 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.730 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.731 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.731 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.731 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.731 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.731 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.731 INFO:teuthology.orchestra.run.smithi100.stdout: "crt_file": { 2024-05-10T11:44:16.731 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "crt_file", 2024-05-10T11:44:16.731 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.731 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.731 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.731 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.731 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.731 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.731 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.732 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.732 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.732 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.732 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.732 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.732 INFO:teuthology.orchestra.run.smithi100.stdout: "debug": { 2024-05-10T11:44:16.732 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "debug", 2024-05-10T11:44:16.732 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.732 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.732 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.732 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.732 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.732 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.732 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.732 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Enable/disable debug options", 2024-05-10T11:44:16.733 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.733 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.733 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.733 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.733 INFO:teuthology.orchestra.run.smithi100.stdout: "jwt_token_ttl": { 2024-05-10T11:44:16.733 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "jwt_token_ttl", 2024-05-10T11:44:16.733 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.733 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.733 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.733 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "28800", 2024-05-10T11:44:16.733 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.733 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.733 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.733 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.733 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.733 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.734 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.734 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.734 INFO:teuthology.orchestra.run.smithi100.stdout: "key_file": { 2024-05-10T11:44:16.734 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "key_file", 2024-05-10T11:44:16.734 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.734 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.734 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.734 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.734 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.734 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.734 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.734 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.734 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.734 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.735 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.735 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.735 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.735 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.735 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.735 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.735 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.735 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.735 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.735 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.735 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.735 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.735 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.735 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.735 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.736 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.736 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.736 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.736 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.736 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.736 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.736 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.736 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.736 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.736 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.736 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.736 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.736 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.736 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.737 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.737 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.737 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.737 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.737 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.737 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.737 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.737 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.737 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.737 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.737 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.737 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.737 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.737 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.737 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.738 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.738 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.738 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.738 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.738 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.738 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.738 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.738 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.738 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.738 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.738 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.738 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.738 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.738 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.738 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.739 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.739 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.739 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.739 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.739 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.739 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.739 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.739 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.739 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.739 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.739 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.739 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.739 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.739 INFO:teuthology.orchestra.run.smithi100.stdout: "motd": { 2024-05-10T11:44:16.739 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "motd", 2024-05-10T11:44:16.740 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.740 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.740 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.740 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.740 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.740 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.740 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.740 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "The message of the day", 2024-05-10T11:44:16.740 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.740 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.740 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.740 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.740 INFO:teuthology.orchestra.run.smithi100.stdout: "redirect_resolve_ip_addr": { 2024-05-10T11:44:16.740 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "redirect_resolve_ip_addr", 2024-05-10T11:44:16.740 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.741 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.741 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.741 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.741 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.741 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.741 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.741 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.741 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.741 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.741 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.741 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.741 INFO:teuthology.orchestra.run.smithi100.stdout: "server_addr": { 2024-05-10T11:44:16.741 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "server_addr", 2024-05-10T11:44:16.741 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.741 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.742 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.742 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "::", 2024-05-10T11:44:16.742 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.742 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.742 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.742 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.742 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.742 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.742 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.742 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.742 INFO:teuthology.orchestra.run.smithi100.stdout: "server_port": { 2024-05-10T11:44:16.742 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "server_port", 2024-05-10T11:44:16.742 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.742 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.742 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.743 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "8080", 2024-05-10T11:44:16.743 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.743 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.743 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.743 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.743 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.743 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.743 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.743 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.743 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.743 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.743 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.743 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.743 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.743 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.744 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.744 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.744 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.744 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.744 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.744 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.744 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.744 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.744 INFO:teuthology.orchestra.run.smithi100.stdout: "ssl": { 2024-05-10T11:44:16.744 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ssl", 2024-05-10T11:44:16.744 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.744 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.744 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.744 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.745 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.745 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.745 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.745 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.745 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.745 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.745 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.745 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.745 INFO:teuthology.orchestra.run.smithi100.stdout: "ssl_server_port": { 2024-05-10T11:44:16.745 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ssl_server_port", 2024-05-10T11:44:16.745 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.745 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.745 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.745 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "8443", 2024-05-10T11:44:16.745 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.746 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.746 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.746 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.746 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.746 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.746 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.746 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.746 INFO:teuthology.orchestra.run.smithi100.stdout: "standby_behaviour": { 2024-05-10T11:44:16.746 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "standby_behaviour", 2024-05-10T11:44:16.746 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.746 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.746 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.746 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "redirect", 2024-05-10T11:44:16.746 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.746 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.747 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.747 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.747 INFO:teuthology.orchestra.run.smithi100.stdout: "redirect" 2024-05-10T11:44:16.747 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.747 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.747 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.747 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.747 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.747 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.747 INFO:teuthology.orchestra.run.smithi100.stdout: "standby_error_status_code": { 2024-05-10T11:44:16.747 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "standby_error_status_code", 2024-05-10T11:44:16.747 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.747 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.747 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.747 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "500", 2024-05-10T11:44:16.748 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "400", 2024-05-10T11:44:16.748 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "599", 2024-05-10T11:44:16.748 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.748 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.748 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.748 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.748 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.748 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.748 INFO:teuthology.orchestra.run.smithi100.stdout: "url_prefix": { 2024-05-10T11:44:16.748 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "url_prefix", 2024-05-10T11:44:16.748 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.748 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.748 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.748 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.748 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.748 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.749 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.749 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.749 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.749 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.749 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.749 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.749 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.749 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.749 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.749 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "devicehealth", 2024-05-10T11:44:16.749 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.749 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.749 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.749 INFO:teuthology.orchestra.run.smithi100.stdout: "enable_monitoring": { 2024-05-10T11:44:16.749 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "enable_monitoring", 2024-05-10T11:44:16.750 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.750 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.750 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.750 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.750 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.750 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.750 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.750 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "monitor device health metrics", 2024-05-10T11:44:16.750 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.750 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.750 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.750 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.750 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.750 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.750 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.751 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.751 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.751 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.751 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.751 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.751 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.751 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.751 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.751 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.751 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.751 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.751 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.751 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.751 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.751 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.752 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.752 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.752 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.752 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.752 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.752 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.752 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.752 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.752 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.752 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.752 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.752 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.752 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.752 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.752 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.753 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.753 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.753 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.753 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.753 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.753 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.753 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.753 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.753 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.753 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.753 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.753 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.753 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.753 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.753 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.754 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.754 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.754 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.754 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.754 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.754 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.754 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.754 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.754 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.754 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.754 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.754 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.754 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.754 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.755 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.755 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.755 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.755 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.755 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.755 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.755 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.755 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.755 INFO:teuthology.orchestra.run.smithi100.stdout: "mark_out_threshold": { 2024-05-10T11:44:16.755 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "mark_out_threshold", 2024-05-10T11:44:16.755 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-05-10T11:44:16.755 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.755 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.755 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "2419200", 2024-05-10T11:44:16.755 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.756 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.756 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.756 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-10T11:44:16.756 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.756 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.756 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.756 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.756 INFO:teuthology.orchestra.run.smithi100.stdout: "pool_name": { 2024-05-10T11:44:16.756 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "pool_name", 2024-05-10T11:44:16.756 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.756 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.756 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.756 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "device_health_metrics", 2024-05-10T11:44:16.756 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.756 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.757 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.757 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-10T11:44:16.757 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.757 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.757 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.757 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.757 INFO:teuthology.orchestra.run.smithi100.stdout: "retention_period": { 2024-05-10T11:44:16.757 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "retention_period", 2024-05-10T11:44:16.757 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-05-10T11:44:16.757 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.757 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.757 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "15552000", 2024-05-10T11:44:16.758 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.758 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.758 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.758 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "how long to retain device health metrics", 2024-05-10T11:44:16.758 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.758 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.758 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.758 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.758 INFO:teuthology.orchestra.run.smithi100.stdout: "scrape_frequency": { 2024-05-10T11:44:16.758 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "scrape_frequency", 2024-05-10T11:44:16.758 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-05-10T11:44:16.758 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.758 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.759 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "86400", 2024-05-10T11:44:16.759 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.759 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.759 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.759 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-10T11:44:16.759 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.759 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.759 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.759 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.759 INFO:teuthology.orchestra.run.smithi100.stdout: "self_heal": { 2024-05-10T11:44:16.759 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "self_heal", 2024-05-10T11:44:16.759 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.759 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.760 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.760 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.760 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.760 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.760 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.760 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-10T11:44:16.760 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.760 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.760 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.760 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.760 INFO:teuthology.orchestra.run.smithi100.stdout: "sleep_interval": { 2024-05-10T11:44:16.760 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sleep_interval", 2024-05-10T11:44:16.761 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-05-10T11:44:16.761 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.761 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.761 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "600", 2024-05-10T11:44:16.761 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.761 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.761 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.761 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "how frequently to wake up and check device health", 2024-05-10T11:44:16.761 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.761 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.761 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.761 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.761 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.762 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.762 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.762 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.762 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.762 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.762 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.762 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.762 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.762 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.762 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.762 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.762 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.762 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.762 INFO:teuthology.orchestra.run.smithi100.stdout: "warn_threshold": { 2024-05-10T11:44:16.763 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "warn_threshold", 2024-05-10T11:44:16.763 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-05-10T11:44:16.763 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.763 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.763 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "7257600", 2024-05-10T11:44:16.763 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.763 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.763 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.763 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-10T11:44:16.763 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.763 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.763 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.763 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.764 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.764 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.764 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.764 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "diskprediction_local", 2024-05-10T11:44:16.764 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.764 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.764 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.764 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.764 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.764 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.764 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.764 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.764 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.764 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.764 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.765 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.765 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.765 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.765 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.765 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.765 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.765 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.765 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.765 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.765 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.765 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.765 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.765 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.765 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.765 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.766 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.766 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.766 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.766 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.766 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.766 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.766 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.766 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.766 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.766 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.766 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.766 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.766 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.766 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.766 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.767 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.767 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.767 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.767 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.767 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.767 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.767 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.767 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.767 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.767 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.767 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.767 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.767 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.767 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.767 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.767 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.768 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.768 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.768 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.768 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.768 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.768 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.768 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.768 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.768 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.768 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.768 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.768 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.768 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.768 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.769 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.769 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.769 INFO:teuthology.orchestra.run.smithi100.stdout: "predict_interval": { 2024-05-10T11:44:16.769 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "predict_interval", 2024-05-10T11:44:16.769 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.769 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.769 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.769 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "86400", 2024-05-10T11:44:16.769 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.769 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.769 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.769 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.769 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.769 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.769 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.770 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.770 INFO:teuthology.orchestra.run.smithi100.stdout: "predictor_model": { 2024-05-10T11:44:16.770 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "predictor_model", 2024-05-10T11:44:16.770 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.770 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.770 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.770 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "prophetstor", 2024-05-10T11:44:16.770 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.770 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.770 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.770 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.770 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.770 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.770 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.770 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.771 INFO:teuthology.orchestra.run.smithi100.stdout: "sleep_interval": { 2024-05-10T11:44:16.771 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sleep_interval", 2024-05-10T11:44:16.771 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.771 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.771 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.771 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "600", 2024-05-10T11:44:16.771 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.771 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.771 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.771 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.771 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.771 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.771 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.771 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.771 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.772 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.772 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.772 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.772 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.772 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.772 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.772 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.772 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.772 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.772 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.772 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.772 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.772 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.772 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.772 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.773 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.773 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "influx", 2024-05-10T11:44:16.773 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": false, 2024-05-10T11:44:16.773 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "influxdb python module not found", 2024-05-10T11:44:16.773 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.773 INFO:teuthology.orchestra.run.smithi100.stdout: "batch_size": { 2024-05-10T11:44:16.773 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "batch_size", 2024-05-10T11:44:16.773 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.773 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.773 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.773 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "5000", 2024-05-10T11:44:16.773 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.773 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.773 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.773 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-10T11:44:16.774 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.774 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.774 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.774 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.774 INFO:teuthology.orchestra.run.smithi100.stdout: "database": { 2024-05-10T11:44:16.774 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "database", 2024-05-10T11:44:16.774 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.774 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.774 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.774 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "ceph", 2024-05-10T11:44:16.774 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.774 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.774 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.774 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-10T11:44:16.775 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.775 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.775 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.775 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.775 INFO:teuthology.orchestra.run.smithi100.stdout: "hostname": { 2024-05-10T11:44:16.775 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "hostname", 2024-05-10T11:44:16.775 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.775 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.775 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.775 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.775 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.775 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.775 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.775 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "InfluxDB server hostname", 2024-05-10T11:44:16.775 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.775 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.776 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.776 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.776 INFO:teuthology.orchestra.run.smithi100.stdout: "interval": { 2024-05-10T11:44:16.776 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "interval", 2024-05-10T11:44:16.776 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-05-10T11:44:16.776 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.776 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.776 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "30", 2024-05-10T11:44:16.776 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "5", 2024-05-10T11:44:16.776 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.776 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.776 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-10T11:44:16.776 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.776 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.776 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.777 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.777 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.777 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.777 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.777 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.777 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.777 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.777 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.777 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.777 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.777 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.777 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.777 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.777 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.777 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.777 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.778 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.778 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.778 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.778 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.778 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.778 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.778 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.778 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.778 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.778 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.778 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.778 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.778 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.778 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.778 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.779 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.779 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.779 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.779 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.779 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.779 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.779 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.779 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.779 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.779 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.779 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.779 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.779 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.779 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.779 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.780 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.780 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.780 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.780 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.780 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.780 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.780 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.780 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.780 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.780 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.780 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.780 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.780 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.780 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.780 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.781 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.781 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.781 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.781 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.781 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.781 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.781 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.781 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.781 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.781 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.781 INFO:teuthology.orchestra.run.smithi100.stdout: "password": { 2024-05-10T11:44:16.781 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "password", 2024-05-10T11:44:16.781 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.781 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.781 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.781 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.782 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.782 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.782 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.782 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "password of InfluxDB server user", 2024-05-10T11:44:16.782 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.782 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.782 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.782 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.782 INFO:teuthology.orchestra.run.smithi100.stdout: "port": { 2024-05-10T11:44:16.782 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "port", 2024-05-10T11:44:16.782 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.782 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.782 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.782 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "8086", 2024-05-10T11:44:16.783 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.783 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.783 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.783 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "InfluxDB server port", 2024-05-10T11:44:16.783 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.783 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.783 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.783 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.783 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.783 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.783 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.783 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.783 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.783 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.783 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.784 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.784 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.784 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.784 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.784 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.784 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.784 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.784 INFO:teuthology.orchestra.run.smithi100.stdout: "ssl": { 2024-05-10T11:44:16.784 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ssl", 2024-05-10T11:44:16.784 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.784 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.784 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.784 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "false", 2024-05-10T11:44:16.784 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.784 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.785 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.785 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-10T11:44:16.785 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.785 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.785 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.785 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.785 INFO:teuthology.orchestra.run.smithi100.stdout: "threads": { 2024-05-10T11:44:16.785 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "threads", 2024-05-10T11:44:16.785 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.785 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.785 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.785 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "5", 2024-05-10T11:44:16.785 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "1", 2024-05-10T11:44:16.785 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "32", 2024-05-10T11:44:16.785 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.786 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-10T11:44:16.786 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.786 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.786 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.786 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.786 INFO:teuthology.orchestra.run.smithi100.stdout: "username": { 2024-05-10T11:44:16.786 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "username", 2024-05-10T11:44:16.786 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.786 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.786 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.786 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.786 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.786 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.787 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.787 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "username of InfluxDB server user", 2024-05-10T11:44:16.787 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.787 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.787 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.787 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.787 INFO:teuthology.orchestra.run.smithi100.stdout: "verify_ssl": { 2024-05-10T11:44:16.787 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "verify_ssl", 2024-05-10T11:44:16.787 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.787 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.787 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.787 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "true", 2024-05-10T11:44:16.787 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.787 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.787 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.788 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-10T11:44:16.788 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.788 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.788 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.788 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.788 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.788 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.788 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.788 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "insights", 2024-05-10T11:44:16.788 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.788 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.788 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.788 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.788 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.789 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.789 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.789 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.789 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.789 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.789 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.789 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.789 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.789 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.789 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.789 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.789 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.789 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.789 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.789 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.790 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.790 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.790 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.790 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.790 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.790 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.790 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.790 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.790 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.790 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.790 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.790 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.790 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.790 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.790 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.791 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.791 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.791 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.791 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.791 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.791 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.791 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.791 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.791 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.791 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.791 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.791 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.791 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.791 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.792 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.792 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.792 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.792 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.792 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.792 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.792 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.792 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.792 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.792 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.792 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.792 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.792 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.792 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.792 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.793 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.793 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.793 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.793 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.793 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.793 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.793 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.793 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.793 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.793 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.793 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.793 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.793 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.793 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.794 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.794 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.794 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.794 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.794 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.794 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.794 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.794 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.794 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.794 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.794 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.794 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.794 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "iostat", 2024-05-10T11:44:16.794 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.794 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.795 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.795 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.795 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.795 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.795 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.795 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.795 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.795 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.795 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.795 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.795 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.795 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.795 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.795 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.796 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.796 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.796 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.796 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.796 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.796 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.796 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.796 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.796 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.796 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.796 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.796 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.796 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.796 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.796 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.797 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.797 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.797 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.797 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.797 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.797 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.797 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.797 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.797 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.797 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.797 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.797 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.797 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.797 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.797 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.798 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.798 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.798 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.798 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.798 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.798 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.798 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.798 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.798 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.798 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.798 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.798 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.798 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.798 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.799 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.799 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.799 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.799 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.799 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.799 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.799 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.799 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.799 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.799 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.799 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.799 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.799 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.800 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.800 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.800 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.800 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.800 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.800 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.800 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.800 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.800 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.800 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.800 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.800 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.801 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.801 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.801 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.801 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.801 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.801 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "k8sevents", 2024-05-10T11:44:16.801 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.801 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.801 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.801 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_event_retention_days": { 2024-05-10T11:44:16.801 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "ceph_event_retention_days", 2024-05-10T11:44:16.801 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.802 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.802 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.802 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "7", 2024-05-10T11:44:16.802 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.802 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.802 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.802 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-10T11:44:16.802 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.802 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.802 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.802 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.802 INFO:teuthology.orchestra.run.smithi100.stdout: "config_check_secs": { 2024-05-10T11:44:16.803 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "config_check_secs", 2024-05-10T11:44:16.803 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.803 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.803 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.803 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10", 2024-05-10T11:44:16.803 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "10", 2024-05-10T11:44:16.803 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.803 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.803 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-10T11:44:16.803 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.803 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.803 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.803 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.804 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.804 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.804 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.804 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.804 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.804 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.804 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.804 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.804 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.804 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.804 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.804 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.805 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.805 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.805 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.805 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.805 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.805 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.805 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.805 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.805 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.805 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.805 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.806 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.806 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.806 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.806 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.806 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.806 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.806 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.806 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.806 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.806 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.806 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.806 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.807 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.807 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.807 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.807 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.807 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.807 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.807 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.807 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.807 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.807 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.807 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.807 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.808 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.808 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.808 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.808 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.808 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.808 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.808 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.808 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.808 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.808 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.808 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.808 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.809 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.809 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.809 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.809 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.809 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.809 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.809 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.809 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.809 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.809 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.809 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.809 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.810 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.810 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.810 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.810 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.810 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.810 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.810 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.810 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.810 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.810 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.810 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.810 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.811 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.811 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.811 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.811 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.811 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "localpool", 2024-05-10T11:44:16.811 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.811 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.811 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.811 INFO:teuthology.orchestra.run.smithi100.stdout: "failure_domain": { 2024-05-10T11:44:16.811 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "failure_domain", 2024-05-10T11:44:16.811 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.811 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.811 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.812 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "host", 2024-05-10T11:44:16.812 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.812 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.812 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.812 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "failure domain for any created local pool", 2024-05-10T11:44:16.812 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-10T11:44:16.812 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.812 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.812 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.812 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.812 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.812 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.813 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.813 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.813 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.813 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.813 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.813 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.813 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.813 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.813 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.813 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.813 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.814 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.814 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.814 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.814 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.814 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.814 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.814 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.814 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.814 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.814 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.814 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.814 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.815 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.815 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.815 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.815 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.815 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.815 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.815 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.815 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.815 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.815 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.815 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.816 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.816 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.816 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.816 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.816 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.816 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.816 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.816 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.816 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.816 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.816 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.816 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.817 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.817 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.817 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.817 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.817 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.817 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.817 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.817 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.817 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.817 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.817 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.817 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.817 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.817 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.817 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.818 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.818 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.818 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.818 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.818 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.818 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.818 INFO:teuthology.orchestra.run.smithi100.stdout: "min_size": { 2024-05-10T11:44:16.818 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "min_size", 2024-05-10T11:44:16.818 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.818 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.818 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.818 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.818 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.818 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.819 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.819 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "default min_size for any created local pool", 2024-05-10T11:44:16.819 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-10T11:44:16.819 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.819 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.819 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.819 INFO:teuthology.orchestra.run.smithi100.stdout: "num_rep": { 2024-05-10T11:44:16.819 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "num_rep", 2024-05-10T11:44:16.819 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.819 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.819 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.819 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "3", 2024-05-10T11:44:16.819 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.819 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.819 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.820 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "default replica count for any created local pool", 2024-05-10T11:44:16.820 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.820 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.820 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.820 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.820 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num": { 2024-05-10T11:44:16.820 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "pg_num", 2024-05-10T11:44:16.820 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.820 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.820 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.820 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "128", 2024-05-10T11:44:16.820 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.820 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.820 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.820 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "default pg_num for any created local pool", 2024-05-10T11:44:16.821 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.821 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.821 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.821 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.821 INFO:teuthology.orchestra.run.smithi100.stdout: "prefix": { 2024-05-10T11:44:16.821 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "prefix", 2024-05-10T11:44:16.821 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.821 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.821 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.821 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.821 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.821 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.821 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.821 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "name prefix for any created local pool", 2024-05-10T11:44:16.821 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.821 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.822 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.822 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.822 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.822 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.822 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.822 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.822 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.822 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.822 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.822 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.822 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.822 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.822 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.822 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.823 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.823 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.823 INFO:teuthology.orchestra.run.smithi100.stdout: "subtree": { 2024-05-10T11:44:16.823 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "subtree", 2024-05-10T11:44:16.823 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.823 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.823 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.823 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "rack", 2024-05-10T11:44:16.823 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.823 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.823 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.823 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-10T11:44:16.823 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-10T11:44:16.824 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.824 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.824 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.824 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.824 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.824 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.824 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "mds_autoscaler", 2024-05-10T11:44:16.824 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.824 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.824 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.824 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.824 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.824 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.824 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.824 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.824 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.825 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.825 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.825 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.825 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.825 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.825 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.825 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.825 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.825 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.825 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.825 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.825 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.825 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.825 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.825 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.825 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.826 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.826 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.826 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.826 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.826 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.826 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.826 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.826 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.826 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.826 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.826 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.826 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.826 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.826 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.827 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.827 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.827 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.827 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.827 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.827 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.827 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.827 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.827 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.827 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.827 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.827 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.827 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.827 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.827 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.828 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.828 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.828 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.828 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.828 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.828 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.828 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.828 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.828 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.828 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.828 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.828 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.828 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.828 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.828 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.829 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.829 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.829 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.829 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.829 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.829 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.829 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.829 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.829 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.829 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.829 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.829 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.829 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.829 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.830 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.830 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.830 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.830 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.830 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.830 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.830 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.830 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "mirroring", 2024-05-10T11:44:16.830 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.830 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.830 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.830 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.830 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.830 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.830 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.831 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.831 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.831 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.831 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.831 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.831 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.831 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.831 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.831 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.831 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.831 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.831 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.831 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.831 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.832 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.832 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.832 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.832 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.832 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.832 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.832 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.832 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.832 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.832 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.832 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.832 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.832 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.832 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.832 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.833 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.833 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.833 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.833 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.833 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.833 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.833 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.833 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.833 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.833 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.833 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.833 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.833 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.833 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.833 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.834 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.834 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.834 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.834 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.834 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.834 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.834 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.834 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.834 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.834 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.834 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.834 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.834 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.834 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.835 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.835 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.835 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.835 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.835 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.835 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.835 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.835 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.835 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.835 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.835 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.835 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.835 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.835 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.835 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.836 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.836 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.836 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.836 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.836 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.836 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.836 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.836 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.836 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.836 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.836 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "nfs", 2024-05-10T11:44:16.836 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.836 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.836 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.836 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.837 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.837 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.837 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.837 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.837 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.837 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.837 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.837 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.837 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.837 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.837 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.837 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.837 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.837 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.837 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.838 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.838 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.838 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.838 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.838 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.838 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.838 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.838 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.838 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.838 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.838 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.838 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.838 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.838 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.838 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.839 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.839 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.839 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.839 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.839 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.839 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.839 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.839 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.839 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.839 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.839 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.839 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.839 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.839 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.839 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.840 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.840 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.840 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.840 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.840 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.840 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.840 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.840 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.840 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.840 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.840 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.840 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.840 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.840 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.840 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.841 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.841 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.841 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.841 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.841 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.841 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.841 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.841 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.841 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.841 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.841 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.841 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.841 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.841 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.841 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.842 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.842 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.842 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.842 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.842 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.842 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.842 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.842 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.842 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.842 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.842 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.842 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "orchestrator", 2024-05-10T11:44:16.842 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.842 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.842 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.843 INFO:teuthology.orchestra.run.smithi100.stdout: "fail_fs": { 2024-05-10T11:44:16.843 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "fail_fs", 2024-05-10T11:44:16.843 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.843 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.843 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.843 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.843 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.843 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.843 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.843 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-10T11:44:16.843 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.843 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.843 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.843 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.843 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.844 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.844 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.844 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.844 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.844 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.844 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.844 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.844 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.844 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.844 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.844 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.844 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.844 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.844 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.845 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.845 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.845 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.845 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.845 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.845 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.845 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.845 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.845 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.845 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.845 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.845 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.845 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.845 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.845 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.846 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.846 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.846 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.846 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.846 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.846 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.846 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.846 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.846 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.846 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.846 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.846 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.846 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.846 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.846 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.847 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.847 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.847 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.847 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.847 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.847 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.847 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.847 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.847 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.847 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.847 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.847 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.847 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.847 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.848 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.848 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.848 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.848 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.848 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.848 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.848 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.848 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.848 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.848 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.848 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.848 INFO:teuthology.orchestra.run.smithi100.stdout: "orchestrator": { 2024-05-10T11:44:16.848 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "orchestrator", 2024-05-10T11:44:16.848 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.848 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.848 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.849 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.849 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.849 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.849 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.849 INFO:teuthology.orchestra.run.smithi100.stdout: "cephadm", 2024-05-10T11:44:16.849 INFO:teuthology.orchestra.run.smithi100.stdout: "rook", 2024-05-10T11:44:16.849 INFO:teuthology.orchestra.run.smithi100.stdout: "test_orchestrator" 2024-05-10T11:44:16.849 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.849 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Orchestrator backend", 2024-05-10T11:44:16.849 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.849 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.849 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.849 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.850 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.850 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.850 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.850 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.850 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.850 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.850 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.850 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.850 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.850 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.850 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.850 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.850 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.850 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.850 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.851 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.851 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.851 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "osd_perf_query", 2024-05-10T11:44:16.851 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.851 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.851 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.851 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.851 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.851 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.851 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.851 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.851 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.851 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.851 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.851 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.852 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.852 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.852 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.852 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.852 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.852 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.852 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.852 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.852 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.852 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.852 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.852 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.852 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.852 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.852 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.853 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.853 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.853 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.853 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.853 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.853 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.853 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.853 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.853 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.853 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.853 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.853 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.853 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.853 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.853 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.854 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.854 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.854 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.854 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.854 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.854 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.854 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.854 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.854 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.854 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.854 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.854 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.854 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.854 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.855 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.855 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.855 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.855 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.855 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.855 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.855 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.855 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.855 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.855 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.855 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.855 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.855 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.855 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.855 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.856 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.856 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.856 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.856 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.856 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.856 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.856 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.856 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.856 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.856 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.856 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.856 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.856 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.856 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.856 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.857 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.857 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.857 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.857 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.857 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "osd_support", 2024-05-10T11:44:16.857 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.857 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.857 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.857 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.857 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.857 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.857 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.857 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.857 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.857 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.858 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.858 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.858 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.858 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.858 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.858 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.858 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.858 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.858 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.858 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.858 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.858 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.858 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.858 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.859 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.859 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.859 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.859 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.859 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.859 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.859 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.859 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.859 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.859 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.859 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.859 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.859 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.859 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.859 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.860 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.860 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.860 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.860 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.860 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.860 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.860 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.860 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.860 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.860 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.860 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.860 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.860 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.860 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.860 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.861 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.861 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.861 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.861 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.861 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.861 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.861 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.861 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.861 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.861 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.861 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.861 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.861 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.861 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.861 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.862 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.862 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.862 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.862 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.862 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.862 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.862 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.862 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.862 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.862 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.862 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.862 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.862 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.862 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.863 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.863 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.863 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.863 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.863 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.863 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.863 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.863 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "pg_autoscaler", 2024-05-10T11:44:16.863 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.863 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.863 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.863 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.863 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.863 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.863 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.864 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.864 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.864 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.864 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.864 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.864 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.864 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.864 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.864 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.864 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.864 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.864 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.864 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.864 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.864 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.864 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.865 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.865 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.865 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.865 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.865 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.865 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.865 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.865 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.865 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.865 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.865 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.865 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.865 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.865 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.865 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.866 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.866 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.866 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.866 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.866 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.866 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.866 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.866 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.866 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.866 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.866 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.866 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.866 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.866 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.867 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.867 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.867 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.867 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.867 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.867 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.867 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.867 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.867 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.867 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.867 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.867 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.867 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.867 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.867 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.868 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.868 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.868 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.868 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.868 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.868 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.868 INFO:teuthology.orchestra.run.smithi100.stdout: "sleep_interval": { 2024-05-10T11:44:16.868 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sleep_interval", 2024-05-10T11:44:16.868 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-05-10T11:44:16.868 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.868 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.868 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "60", 2024-05-10T11:44:16.868 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.868 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.868 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.869 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.869 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.869 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.869 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.869 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.869 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.869 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.869 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.869 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.869 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.869 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.869 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.869 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.869 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.870 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.870 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.870 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.870 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.870 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.870 INFO:teuthology.orchestra.run.smithi100.stdout: "threshold": { 2024-05-10T11:44:16.870 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "threshold", 2024-05-10T11:44:16.870 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "float", 2024-05-10T11:44:16.870 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.870 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.870 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "3.0", 2024-05-10T11:44:16.870 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "1.0", 2024-05-10T11:44:16.870 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.870 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.870 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "scaling threshold", 2024-05-10T11:44:16.871 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-10T11:44:16.871 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.871 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.871 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.871 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.871 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.871 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.871 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "progress", 2024-05-10T11:44:16.871 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.871 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.871 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.872 INFO:teuthology.orchestra.run.smithi100.stdout: "allow_pg_recovery_event": { 2024-05-10T11:44:16.872 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "allow_pg_recovery_event", 2024-05-10T11:44:16.872 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.872 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.872 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.872 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.872 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.872 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.872 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.872 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-10T11:44:16.872 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.872 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.873 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.873 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.873 INFO:teuthology.orchestra.run.smithi100.stdout: "enabled": { 2024-05-10T11:44:16.873 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "enabled", 2024-05-10T11:44:16.873 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.873 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.873 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.873 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.873 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.873 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.873 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.874 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.874 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.874 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.874 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.874 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.874 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.874 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.874 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.874 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.874 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.874 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.875 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.875 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.875 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.875 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.875 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.875 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.875 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.875 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.875 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.875 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.875 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.875 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.876 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.876 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.876 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.876 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.876 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.876 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.876 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.876 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.876 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.876 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.876 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.876 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.876 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.876 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.877 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.877 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.877 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.877 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.877 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.877 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.877 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.877 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.877 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.877 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.877 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.877 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.877 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.877 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.877 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.878 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.878 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.878 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.878 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.878 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.878 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.878 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.878 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.878 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.878 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.878 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.878 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.878 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.878 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.878 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.879 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.879 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.879 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.879 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.879 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.879 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.879 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.879 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.879 INFO:teuthology.orchestra.run.smithi100.stdout: "max_completed_events": { 2024-05-10T11:44:16.879 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "max_completed_events", 2024-05-10T11:44:16.879 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.879 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.879 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.880 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "50", 2024-05-10T11:44:16.880 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.880 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.880 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.880 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "number of past completed events to remember", 2024-05-10T11:44:16.880 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.880 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.880 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.880 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.880 INFO:teuthology.orchestra.run.smithi100.stdout: "sleep_interval": { 2024-05-10T11:44:16.880 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sleep_interval", 2024-05-10T11:44:16.880 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-05-10T11:44:16.880 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.880 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.880 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "5", 2024-05-10T11:44:16.881 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.881 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.881 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.881 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "how long the module is going to sleep", 2024-05-10T11:44:16.881 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.881 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.881 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.881 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.881 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.881 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.881 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.881 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.881 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.881 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.881 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.881 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.882 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.882 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.882 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.882 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.882 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.882 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.882 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.882 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.882 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.882 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "prometheus", 2024-05-10T11:44:16.882 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.882 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.882 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.882 INFO:teuthology.orchestra.run.smithi100.stdout: "cache": { 2024-05-10T11:44:16.883 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "cache", 2024-05-10T11:44:16.883 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.883 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.883 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.883 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.883 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.883 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.883 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.883 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.883 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.883 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.883 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.883 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.883 INFO:teuthology.orchestra.run.smithi100.stdout: "exclude_perf_counters": { 2024-05-10T11:44:16.883 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "exclude_perf_counters", 2024-05-10T11:44:16.884 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.884 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.884 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.884 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.884 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.884 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.884 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.884 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-10T11:44:16.884 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-10T11:44:16.884 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.884 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.884 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.884 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.884 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.884 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.885 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.885 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.885 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.885 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.885 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.885 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.885 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.885 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.885 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.885 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.885 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.885 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.885 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.885 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.885 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.886 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.886 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.886 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.886 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.886 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.886 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.886 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.886 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.886 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.886 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.886 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.886 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.886 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.886 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.887 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.887 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.887 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.887 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.887 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.887 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.887 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.887 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.887 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.887 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.887 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.887 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.887 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.887 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.887 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.887 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.888 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.888 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.888 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.888 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.888 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.888 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.888 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.888 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.888 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.888 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.888 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.888 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.888 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.888 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.889 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.889 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.889 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.889 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.889 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.889 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.889 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.889 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.889 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_stats_pools": { 2024-05-10T11:44:16.889 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rbd_stats_pools", 2024-05-10T11:44:16.889 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.889 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.889 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.889 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.889 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.889 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.890 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.890 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.890 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.890 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.890 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.890 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.890 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-10T11:44:16.890 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-10T11:44:16.890 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.890 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.890 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.890 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "300", 2024-05-10T11:44:16.890 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.891 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.891 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.891 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.891 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.891 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.891 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.891 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.891 INFO:teuthology.orchestra.run.smithi100.stdout: "scrape_interval": { 2024-05-10T11:44:16.891 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "scrape_interval", 2024-05-10T11:44:16.891 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "float", 2024-05-10T11:44:16.891 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.891 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.891 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "15.0", 2024-05-10T11:44:16.891 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.892 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.892 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.892 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.892 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.892 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.892 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.892 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.892 INFO:teuthology.orchestra.run.smithi100.stdout: "server_addr": { 2024-05-10T11:44:16.893 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "server_addr", 2024-05-10T11:44:16.893 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.893 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.893 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.893 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "::", 2024-05-10T11:44:16.893 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.893 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.893 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.893 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-10T11:44:16.893 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.893 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.894 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.894 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.894 INFO:teuthology.orchestra.run.smithi100.stdout: "server_port": { 2024-05-10T11:44:16.894 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "server_port", 2024-05-10T11:44:16.894 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.894 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.894 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.894 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "9283", 2024-05-10T11:44:16.894 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.894 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.894 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.894 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-10T11:44:16.894 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.894 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.895 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.895 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.895 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.895 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.895 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.895 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.895 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.895 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.895 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.895 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.895 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.895 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.895 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.895 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.895 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.896 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.896 INFO:teuthology.orchestra.run.smithi100.stdout: "stale_cache_strategy": { 2024-05-10T11:44:16.896 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "stale_cache_strategy", 2024-05-10T11:44:16.896 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.896 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.896 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.896 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "log", 2024-05-10T11:44:16.896 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.896 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.896 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.896 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.896 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.896 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.896 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.897 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.897 INFO:teuthology.orchestra.run.smithi100.stdout: "standby_behaviour": { 2024-05-10T11:44:16.897 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "standby_behaviour", 2024-05-10T11:44:16.897 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.897 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.897 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.897 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "default", 2024-05-10T11:44:16.897 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.897 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.897 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.897 INFO:teuthology.orchestra.run.smithi100.stdout: "default", 2024-05-10T11:44:16.897 INFO:teuthology.orchestra.run.smithi100.stdout: "error" 2024-05-10T11:44:16.897 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.897 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.897 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.897 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.898 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.898 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.898 INFO:teuthology.orchestra.run.smithi100.stdout: "standby_error_status_code": { 2024-05-10T11:44:16.898 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "standby_error_status_code", 2024-05-10T11:44:16.898 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.898 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.898 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.898 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "500", 2024-05-10T11:44:16.898 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "400", 2024-05-10T11:44:16.898 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "599", 2024-05-10T11:44:16.898 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.898 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.898 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.898 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.898 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.899 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.899 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.899 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.899 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.899 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rbd_support", 2024-05-10T11:44:16.899 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.899 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.899 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.899 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.899 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.899 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.899 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.899 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.899 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.899 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.900 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.900 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.900 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.900 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.900 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.900 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.900 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.900 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.900 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.900 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.900 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.900 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.900 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.900 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.901 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.901 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.901 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.901 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.901 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.901 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.901 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.901 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.901 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.901 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.901 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.901 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.901 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.901 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.901 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.902 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.902 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.902 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.902 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.902 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.902 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.902 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.902 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.902 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.902 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.902 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.902 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.902 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.902 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.902 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.903 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.903 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.903 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.903 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.903 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.903 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.903 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.903 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.903 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.903 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.903 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.903 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.903 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.903 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.903 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.904 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.904 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.904 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.904 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.904 INFO:teuthology.orchestra.run.smithi100.stdout: "max_concurrent_snap_create": { 2024-05-10T11:44:16.904 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "max_concurrent_snap_create", 2024-05-10T11:44:16.904 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.904 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.904 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.904 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10", 2024-05-10T11:44:16.904 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.904 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.904 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.904 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.904 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.905 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.905 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.905 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.905 INFO:teuthology.orchestra.run.smithi100.stdout: "mirror_snapshot_schedule": { 2024-05-10T11:44:16.905 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "mirror_snapshot_schedule", 2024-05-10T11:44:16.905 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.905 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.905 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.905 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.905 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.905 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.905 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.905 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.905 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.905 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.906 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.906 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.906 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.906 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.906 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.906 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.906 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.906 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.906 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.906 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.906 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.906 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.906 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.906 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.906 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.907 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.907 INFO:teuthology.orchestra.run.smithi100.stdout: "trash_purge_schedule": { 2024-05-10T11:44:16.907 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "trash_purge_schedule", 2024-05-10T11:44:16.907 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.907 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.907 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.907 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.907 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.907 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.907 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.907 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.907 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.907 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.907 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.907 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.908 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.908 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.908 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.908 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "restful", 2024-05-10T11:44:16.908 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.908 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.908 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.908 INFO:teuthology.orchestra.run.smithi100.stdout: "enable_auth": { 2024-05-10T11:44:16.908 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "enable_auth", 2024-05-10T11:44:16.908 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.908 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.908 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.908 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.908 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.908 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.909 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.909 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.909 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.909 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.909 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.909 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.909 INFO:teuthology.orchestra.run.smithi100.stdout: "key_file": { 2024-05-10T11:44:16.909 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "key_file", 2024-05-10T11:44:16.909 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.909 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.909 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.909 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.909 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.909 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.909 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.910 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.910 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.910 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.910 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.910 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.910 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.910 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.910 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.910 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.910 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.910 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.910 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.910 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.910 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.910 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.911 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.911 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.911 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.911 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.911 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.911 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.911 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.911 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.911 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.911 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.911 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.911 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.911 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.911 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.911 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.912 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.912 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.912 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.912 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.912 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.912 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.912 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.912 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.912 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.912 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.912 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.912 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.912 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.912 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.912 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.913 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.913 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.913 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.913 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.913 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.913 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.913 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.913 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.913 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.913 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.913 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.913 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.913 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.913 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.913 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.914 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.914 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.914 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.914 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.914 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.914 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.914 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.914 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.914 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.914 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.914 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.914 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.914 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.914 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.914 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.915 INFO:teuthology.orchestra.run.smithi100.stdout: "server_addr": { 2024-05-10T11:44:16.915 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "server_addr", 2024-05-10T11:44:16.915 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.915 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.915 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.915 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.915 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.915 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.915 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.915 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.915 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.915 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.915 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.915 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.915 INFO:teuthology.orchestra.run.smithi100.stdout: "server_port": { 2024-05-10T11:44:16.915 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "server_port", 2024-05-10T11:44:16.916 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.916 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.916 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.916 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.916 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.916 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.916 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.916 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.916 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.916 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.916 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.916 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.916 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.916 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.916 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.917 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.917 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.917 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.917 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.917 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.917 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.917 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.917 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.917 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.917 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.917 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.917 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.917 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.917 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.917 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rgw", 2024-05-10T11:44:16.918 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.918 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.918 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.918 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.918 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.918 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.918 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.918 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.918 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.918 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.918 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.918 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.918 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.918 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.918 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.919 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.919 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.919 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.919 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.919 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.919 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.919 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.919 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.919 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.919 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.919 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.919 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.919 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.919 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.919 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.920 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.920 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.920 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.920 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.920 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.920 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.920 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.920 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.920 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.920 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.920 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.920 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.920 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.920 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.920 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.921 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.921 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.921 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.921 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.921 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.921 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.921 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.921 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.921 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.921 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.921 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.921 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.921 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.921 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.921 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.921 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.922 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.922 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.922 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.922 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.922 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.922 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.922 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.922 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.922 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.922 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.922 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.922 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.922 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.922 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.923 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.923 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.923 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.923 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.923 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.923 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.923 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.923 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.923 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.923 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.923 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.923 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.923 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.923 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.923 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.924 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rook", 2024-05-10T11:44:16.924 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.924 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.924 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.924 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.924 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.924 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.924 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.924 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.924 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.924 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.924 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.924 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.924 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.924 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.924 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.925 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.925 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.925 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.925 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.925 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.925 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.925 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.925 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.925 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.925 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.925 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.925 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.925 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.925 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.925 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.926 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.926 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.926 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.926 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.926 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.926 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.926 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.926 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.926 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.926 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.926 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.926 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.926 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.926 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.927 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.927 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.927 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.927 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.927 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.927 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.927 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.927 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.927 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.927 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.927 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.927 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.927 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.927 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.927 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.930 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.930 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.930 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.930 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.930 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.930 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.930 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.930 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.930 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.930 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.931 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.931 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.931 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.931 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.931 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.931 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.931 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.931 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.931 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.931 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.931 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.931 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.931 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.931 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.931 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.932 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.932 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.932 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.932 INFO:teuthology.orchestra.run.smithi100.stdout: "storage_class": { 2024-05-10T11:44:16.932 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "storage_class", 2024-05-10T11:44:16.932 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.932 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.932 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.932 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "local", 2024-05-10T11:44:16.932 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.932 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.932 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.932 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-10T11:44:16.932 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.933 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.933 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.933 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.933 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.933 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.933 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.933 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "selftest", 2024-05-10T11:44:16.933 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.933 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.933 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.933 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.933 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.933 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.933 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.934 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.934 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.934 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.934 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.934 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.934 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.934 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.934 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.934 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.934 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.934 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.934 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.934 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.934 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.935 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.935 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.935 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.935 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.935 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.935 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.935 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.935 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.935 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.935 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.935 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.935 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.935 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.935 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.936 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.936 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.936 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.936 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.936 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.936 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.936 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.936 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.936 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.936 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.936 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.936 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.936 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.936 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.936 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.936 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.937 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.937 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.937 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.937 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.937 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.937 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.937 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.937 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.937 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.937 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.937 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.937 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.937 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.937 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.937 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.937 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.938 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.938 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.938 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.938 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.938 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.938 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.938 INFO:teuthology.orchestra.run.smithi100.stdout: "roption1": { 2024-05-10T11:44:16.938 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "roption1", 2024-05-10T11:44:16.938 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.938 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.938 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.938 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.938 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.938 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.939 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.939 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.939 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.939 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.939 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.939 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.939 INFO:teuthology.orchestra.run.smithi100.stdout: "roption2": { 2024-05-10T11:44:16.939 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "roption2", 2024-05-10T11:44:16.939 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.939 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.939 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.939 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "xyz", 2024-05-10T11:44:16.939 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.939 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.939 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.940 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.940 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.940 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.940 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.940 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.940 INFO:teuthology.orchestra.run.smithi100.stdout: "rwoption1": { 2024-05-10T11:44:16.940 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rwoption1", 2024-05-10T11:44:16.940 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.940 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.940 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.940 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.940 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.940 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.940 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.941 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.941 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.941 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.941 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.941 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.941 INFO:teuthology.orchestra.run.smithi100.stdout: "rwoption2": { 2024-05-10T11:44:16.941 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rwoption2", 2024-05-10T11:44:16.941 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.941 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.941 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.941 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.941 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.941 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.941 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.941 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.941 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.942 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.942 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.942 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.942 INFO:teuthology.orchestra.run.smithi100.stdout: "rwoption3": { 2024-05-10T11:44:16.942 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rwoption3", 2024-05-10T11:44:16.942 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "float", 2024-05-10T11:44:16.942 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.942 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.942 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.942 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.942 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.942 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.942 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.943 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.943 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.943 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.943 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.943 INFO:teuthology.orchestra.run.smithi100.stdout: "rwoption4": { 2024-05-10T11:44:16.943 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rwoption4", 2024-05-10T11:44:16.943 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.943 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.943 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.943 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.943 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.943 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.943 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.943 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.944 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.944 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.944 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.944 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.944 INFO:teuthology.orchestra.run.smithi100.stdout: "rwoption5": { 2024-05-10T11:44:16.944 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rwoption5", 2024-05-10T11:44:16.944 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.944 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.944 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.944 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.944 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.944 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.944 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.944 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.944 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.945 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.945 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.945 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.945 INFO:teuthology.orchestra.run.smithi100.stdout: "rwoption6": { 2024-05-10T11:44:16.945 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rwoption6", 2024-05-10T11:44:16.945 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.945 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.945 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.945 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.945 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.945 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.945 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.945 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.945 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.946 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.946 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.946 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.946 INFO:teuthology.orchestra.run.smithi100.stdout: "rwoption7": { 2024-05-10T11:44:16.946 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rwoption7", 2024-05-10T11:44:16.946 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.946 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.946 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.946 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.946 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "1", 2024-05-10T11:44:16.946 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "42", 2024-05-10T11:44:16.946 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.947 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.947 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.947 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.947 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.947 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.947 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.947 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.947 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.947 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.947 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.947 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.947 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.947 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.948 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.948 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.948 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.948 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.948 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.948 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.948 INFO:teuthology.orchestra.run.smithi100.stdout: "testkey": { 2024-05-10T11:44:16.948 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "testkey", 2024-05-10T11:44:16.948 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.948 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.948 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.948 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.949 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.949 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.949 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.949 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.949 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.949 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.949 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.949 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.949 INFO:teuthology.orchestra.run.smithi100.stdout: "testlkey": { 2024-05-10T11:44:16.949 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "testlkey", 2024-05-10T11:44:16.949 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.949 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.950 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.950 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.950 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.950 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.950 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.950 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.950 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.950 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.950 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.950 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.950 INFO:teuthology.orchestra.run.smithi100.stdout: "testnewline": { 2024-05-10T11:44:16.950 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "testnewline", 2024-05-10T11:44:16.950 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.950 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.951 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.951 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.951 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.951 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.951 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.951 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.951 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.951 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.951 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.951 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.951 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.951 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.951 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.951 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "smb", 2024-05-10T11:44:16.951 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.951 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.952 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.952 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.952 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.952 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.952 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.952 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.952 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.952 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.952 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.952 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.952 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.953 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.953 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.953 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.953 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.953 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.953 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.953 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.953 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.953 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.953 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.953 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.953 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.953 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.953 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.954 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.954 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.954 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.954 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.954 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.954 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.954 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.954 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.954 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.954 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.954 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.955 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.955 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.955 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.955 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.955 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.955 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.955 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.955 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.955 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.955 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.955 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.955 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.956 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.956 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.956 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.956 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.956 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.956 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.956 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.956 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.956 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.956 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.956 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.956 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.956 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.957 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.957 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.957 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.957 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.957 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.957 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.957 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.957 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.957 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.957 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.957 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.958 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.958 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.958 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.958 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.958 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.958 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.958 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.958 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.958 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.958 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.958 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.959 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.959 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.959 INFO:teuthology.orchestra.run.smithi100.stdout: "update_orchestration": { 2024-05-10T11:44:16.959 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "update_orchestration", 2024-05-10T11:44:16.959 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.959 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.959 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.959 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.959 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.959 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.959 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.960 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-10T11:44:16.960 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.960 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.960 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.960 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.960 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.960 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.960 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.960 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "snap_schedule", 2024-05-10T11:44:16.961 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.961 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.961 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.961 INFO:teuthology.orchestra.run.smithi100.stdout: "allow_m_granularity": { 2024-05-10T11:44:16.961 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "allow_m_granularity", 2024-05-10T11:44:16.961 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.961 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.961 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.961 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.961 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.961 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.962 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.962 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "allow minute scheduled snapshots", 2024-05-10T11:44:16.962 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.962 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.962 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.962 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.962 INFO:teuthology.orchestra.run.smithi100.stdout: "dump_on_update": { 2024-05-10T11:44:16.962 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "dump_on_update", 2024-05-10T11:44:16.962 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.962 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.962 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.963 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.963 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.963 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.963 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.963 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "dump database to debug log on update", 2024-05-10T11:44:16.963 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.963 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.963 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.963 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.963 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.963 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.964 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.964 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.964 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.964 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.964 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.964 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.964 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.964 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.964 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.964 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.964 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.964 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.964 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.964 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.964 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.965 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.965 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.965 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.965 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.965 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.965 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.965 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.965 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.965 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.965 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.965 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.965 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.965 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.965 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.965 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.966 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.966 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.966 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.966 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.966 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.966 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.966 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.966 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.966 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.966 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.966 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.966 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.966 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.966 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.966 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.967 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.967 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.967 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.967 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.967 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.967 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.967 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.967 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.967 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.967 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.967 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.967 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.967 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.967 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.967 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.967 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.968 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.968 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.968 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.968 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.968 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.968 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.968 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.968 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.968 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.968 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.968 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.968 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.968 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.968 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.968 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.969 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.969 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.969 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.969 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.969 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.969 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.969 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.969 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.969 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.969 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "stats", 2024-05-10T11:44:16.969 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.969 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.969 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.969 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.969 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.970 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.970 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.970 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.970 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.970 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.970 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.970 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.970 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.970 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.970 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.970 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.970 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.970 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.970 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.971 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.971 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.971 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.971 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.971 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.971 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.971 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.971 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.971 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.971 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.971 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.971 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.971 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.971 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.971 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.972 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.972 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.972 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.972 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.972 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.972 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.972 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.972 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.972 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.972 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.972 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.972 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.972 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.972 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.973 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.973 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.973 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.973 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.973 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.973 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.973 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.973 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.973 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.973 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.973 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.973 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.973 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.973 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.974 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.974 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.974 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.974 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.974 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.974 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.974 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.974 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.974 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.974 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.974 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.974 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.974 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.975 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.975 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.975 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.975 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.975 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.975 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.975 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.975 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.975 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.975 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.975 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.975 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.975 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.975 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.976 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.976 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "status", 2024-05-10T11:44:16.976 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.976 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.976 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.976 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.976 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.976 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.976 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.976 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.976 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.976 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.976 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.976 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.976 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.977 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.977 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.977 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.977 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.977 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.977 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.977 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.977 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.977 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.977 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.977 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.977 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.977 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.977 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.977 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.978 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.978 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.978 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.978 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.978 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.978 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.978 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.978 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.978 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.978 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.978 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.978 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.978 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.978 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.979 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.979 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.979 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.979 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.979 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.979 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.979 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.979 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.979 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.979 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.979 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.979 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.979 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.979 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.979 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.980 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.980 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.980 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.980 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.980 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.980 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.980 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.980 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.980 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.980 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.980 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.980 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.980 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.980 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.981 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.981 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.981 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.981 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.981 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.981 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.981 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.981 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.981 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.981 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.981 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.981 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.981 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.981 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.981 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.981 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.982 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.982 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.982 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.982 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "telegraf", 2024-05-10T11:44:16.982 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.982 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.982 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.982 INFO:teuthology.orchestra.run.smithi100.stdout: "address": { 2024-05-10T11:44:16.982 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "address", 2024-05-10T11:44:16.982 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.982 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.982 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.982 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-10T11:44:16.982 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.983 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.983 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.983 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.983 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.983 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.983 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.983 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.983 INFO:teuthology.orchestra.run.smithi100.stdout: "interval": { 2024-05-10T11:44:16.983 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "interval", 2024-05-10T11:44:16.983 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-05-10T11:44:16.983 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.983 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.983 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "15", 2024-05-10T11:44:16.983 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.983 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.983 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.984 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.984 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.984 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.984 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.984 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.984 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:16.984 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:16.984 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.984 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.984 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.984 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.984 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.984 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.984 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.984 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.985 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.985 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.985 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.985 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.985 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.985 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.985 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.985 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.985 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.985 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.985 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.985 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:16.985 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:16.985 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.985 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.986 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.986 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.986 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.986 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.986 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.986 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.986 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.986 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.986 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.986 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.986 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:16.986 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:16.986 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.986 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.987 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.987 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:16.987 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.987 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.987 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:16.987 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:16.987 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:16.987 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:16.987 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:16.987 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:16.987 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:16.987 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:16.987 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.987 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.987 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.988 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.988 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.988 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:16.988 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:16.988 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.988 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.988 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.988 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.988 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.988 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.988 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.988 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.988 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.988 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.988 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.989 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.989 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:16.989 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:16.989 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.989 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:16.989 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:16.989 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:16.989 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.989 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.989 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.989 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.989 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.989 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.989 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.989 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.990 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:16.990 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.990 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:16.990 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "telemetry", 2024-05-10T11:44:16.990 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:16.990 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:16.990 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:16.990 INFO:teuthology.orchestra.run.smithi100.stdout: "channel_basic": { 2024-05-10T11:44:16.990 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "channel_basic", 2024-05-10T11:44:16.990 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.990 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.990 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.990 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.990 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.991 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.991 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.991 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-10T11:44:16.991 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.991 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.991 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.991 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.991 INFO:teuthology.orchestra.run.smithi100.stdout: "channel_crash": { 2024-05-10T11:44:16.991 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "channel_crash", 2024-05-10T11:44:16.991 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.991 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.991 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.991 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.991 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.991 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.992 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.992 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-10T11:44:16.992 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.992 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.992 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.992 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.992 INFO:teuthology.orchestra.run.smithi100.stdout: "channel_device": { 2024-05-10T11:44:16.992 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "channel_device", 2024-05-10T11:44:16.992 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.992 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.992 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.992 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:16.992 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.992 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.992 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.993 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-10T11:44:16.993 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.993 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.993 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.993 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.993 INFO:teuthology.orchestra.run.smithi100.stdout: "channel_ident": { 2024-05-10T11:44:16.993 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "channel_ident", 2024-05-10T11:44:16.993 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.993 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.993 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.993 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.993 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.993 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.993 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.993 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-10T11:44:16.993 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.994 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.994 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.994 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.994 INFO:teuthology.orchestra.run.smithi100.stdout: "channel_perf": { 2024-05-10T11:44:16.994 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "channel_perf", 2024-05-10T11:44:16.994 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.994 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.994 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.994 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.994 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.994 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.994 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.994 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-10T11:44:16.994 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.995 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.995 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.995 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.995 INFO:teuthology.orchestra.run.smithi100.stdout: "contact": { 2024-05-10T11:44:16.995 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "contact", 2024-05-10T11:44:16.995 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.995 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.995 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.995 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.995 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.995 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.995 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.995 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.995 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.995 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.996 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.996 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.996 INFO:teuthology.orchestra.run.smithi100.stdout: "description": { 2024-05-10T11:44:16.996 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "description", 2024-05-10T11:44:16.996 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.996 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.996 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.996 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:16.996 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.996 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.996 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.996 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.996 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.996 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.996 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.997 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.997 INFO:teuthology.orchestra.run.smithi100.stdout: "device_url": { 2024-05-10T11:44:16.997 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "device_url", 2024-05-10T11:44:16.997 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:16.997 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.997 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.997 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-10T11:44:16.997 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.997 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.997 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.997 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.997 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.997 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.997 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.997 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.998 INFO:teuthology.orchestra.run.smithi100.stdout: "enabled": { 2024-05-10T11:44:16.998 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "enabled", 2024-05-10T11:44:16.998 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:16.998 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.998 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.998 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:16.998 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:16.998 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.998 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.998 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.998 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.998 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.998 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.998 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:16.999 INFO:teuthology.orchestra.run.smithi100.stdout: "interval": { 2024-05-10T11:44:16.999 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "interval", 2024-05-10T11:44:16.999 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:16.999 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:16.999 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:16.999 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "24", 2024-05-10T11:44:16.999 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "8", 2024-05-10T11:44:16.999 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:16.999 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:16.999 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:16.999 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:16.999 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:16.999 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:16.999 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:17.000 INFO:teuthology.orchestra.run.smithi100.stdout: "last_opt_revision": { 2024-05-10T11:44:17.000 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "last_opt_revision", 2024-05-10T11:44:17.000 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:17.000 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:17.000 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:17.000 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "1", 2024-05-10T11:44:17.000 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:17.000 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:17.000 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:17.000 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:17.000 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:17.000 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:17.000 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:17.000 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:17.000 INFO:teuthology.orchestra.run.smithi100.stdout: "leaderboard": { 2024-05-10T11:44:17.001 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "leaderboard", 2024-05-10T11:44:17.001 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:17.001 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:17.001 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:17.001 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:17.001 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:17.001 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:17.001 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:17.001 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:17.001 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:17.001 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:17.001 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:17.001 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:17.001 INFO:teuthology.orchestra.run.smithi100.stdout: "leaderboard_description": { 2024-05-10T11:44:17.001 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "leaderboard_description", 2024-05-10T11:44:17.002 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:17.002 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:17.002 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:17.002 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:17.002 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:17.002 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:17.002 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:17.002 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:17.002 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:17.002 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:17.002 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:17.002 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:17.002 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:17.002 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:17.002 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:17.003 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:17.003 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:17.003 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:17.003 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:17.003 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:17.003 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:17.003 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:17.003 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:17.003 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:17.003 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:17.003 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:17.003 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:17.003 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:17.003 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:17.003 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:17.004 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:17.004 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:17.004 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:17.004 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:17.004 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:17.004 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:17.004 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:17.004 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:17.004 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:17.004 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:17.004 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:17.004 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:17.004 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:17.004 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:17.005 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:17.005 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:17.005 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:17.005 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:17.005 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:17.005 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:17.005 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:17.005 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:17.005 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:17.005 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:17.005 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:17.005 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:17.005 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:17.005 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:17.006 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:17.006 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:17.006 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:17.006 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:17.006 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:17.006 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:17.006 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:17.006 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:17.006 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:17.006 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:17.006 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:17.006 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:17.006 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:17.006 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:17.006 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:17.007 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:17.007 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:17.007 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:17.007 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:17.007 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:17.007 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:17.007 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:17.007 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:17.007 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:17.007 INFO:teuthology.orchestra.run.smithi100.stdout: "organization": { 2024-05-10T11:44:17.007 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "organization", 2024-05-10T11:44:17.007 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:17.007 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:17.007 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:17.007 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:17.008 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:17.008 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:17.008 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:17.008 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:17.008 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:17.008 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:17.008 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:17.008 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:17.008 INFO:teuthology.orchestra.run.smithi100.stdout: "proxy": { 2024-05-10T11:44:17.008 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "proxy", 2024-05-10T11:44:17.008 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:17.008 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:17.008 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:17.008 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:17.008 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:17.009 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:17.009 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:17.009 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:17.009 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:17.009 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:17.009 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:17.009 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:17.009 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:17.009 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:17.009 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:17.009 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:17.009 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:17.009 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:17.009 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:17.009 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:17.010 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:17.010 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:17.010 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:17.010 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:17.010 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:17.010 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:17.010 INFO:teuthology.orchestra.run.smithi100.stdout: "url": { 2024-05-10T11:44:17.010 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "url", 2024-05-10T11:44:17.010 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:17.010 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:17.010 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:17.010 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-10T11:44:17.010 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:17.010 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:17.011 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:17.011 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:17.011 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:17.011 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:17.011 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:17.011 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:17.011 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:17.011 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:17.011 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:17.011 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "test_orchestrator", 2024-05-10T11:44:17.011 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:17.011 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:17.011 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:17.011 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:17.011 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:17.012 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:17.012 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:17.012 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:17.012 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:17.012 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:17.012 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:17.012 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:17.012 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:17.012 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:17.012 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:17.012 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:17.012 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:17.012 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:17.012 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:17.012 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:17.013 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:17.013 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:17.013 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:17.013 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:17.013 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:17.013 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:17.013 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:17.013 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:17.013 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:17.013 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:17.013 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:17.013 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:17.013 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:17.013 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:17.013 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:17.014 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:17.014 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:17.014 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:17.014 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:17.014 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:17.014 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:17.014 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:17.014 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:17.014 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:17.014 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:17.014 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:17.014 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:17.014 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:17.014 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:17.014 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:17.015 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:17.015 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:17.015 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:17.015 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:17.015 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:17.015 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:17.015 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:17.015 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:17.015 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:17.015 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:17.015 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:17.015 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:17.015 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:17.016 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:17.016 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:17.016 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:17.016 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:17.016 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:17.016 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:17.016 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:17.016 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:17.016 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:17.016 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:17.016 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:17.016 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:17.016 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:17.016 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:17.017 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:17.017 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:17.017 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:17.017 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:17.017 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:17.017 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:17.017 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:17.017 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:17.017 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:17.017 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:17.017 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:17.017 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:17.017 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:17.017 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "volumes", 2024-05-10T11:44:17.017 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:17.018 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:17.018 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:17.018 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:17.018 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:17.018 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:17.018 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:17.018 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:17.018 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:17.018 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:17.018 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:17.018 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:17.018 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:17.018 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:17.018 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:17.018 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:17.019 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:17.019 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:17.019 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:17.019 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:17.019 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:17.019 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:17.019 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:17.019 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:17.019 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:17.019 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:17.019 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:17.019 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:17.019 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:17.019 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:17.019 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:17.020 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:17.020 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:17.020 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:17.020 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:17.020 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:17.020 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:17.020 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:17.020 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:17.020 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:17.020 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:17.020 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:17.020 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:17.020 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:17.020 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:17.020 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:17.021 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:17.021 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:17.021 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:17.021 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:17.021 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:17.021 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:17.021 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:17.021 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:17.021 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:17.021 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:17.021 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:17.021 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:17.021 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:17.022 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:17.022 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:17.022 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:17.022 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:17.022 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:17.022 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:17.022 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:17.022 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:17.022 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:17.022 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:17.022 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:17.022 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:17.022 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:17.022 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:17.023 INFO:teuthology.orchestra.run.smithi100.stdout: "max_concurrent_clones": { 2024-05-10T11:44:17.023 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "max_concurrent_clones", 2024-05-10T11:44:17.023 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:17.023 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:17.023 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:17.023 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "4", 2024-05-10T11:44:17.023 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:17.023 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:17.023 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:17.023 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-10T11:44:17.023 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:17.023 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:17.023 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:17.023 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:17.023 INFO:teuthology.orchestra.run.smithi100.stdout: "periodic_async_work": { 2024-05-10T11:44:17.024 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "periodic_async_work", 2024-05-10T11:44:17.024 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:17.024 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:17.024 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:17.024 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:17.024 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:17.024 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:17.024 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:17.024 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Periodically check for async work", 2024-05-10T11:44:17.024 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:17.024 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:17.024 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:17.024 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:17.024 INFO:teuthology.orchestra.run.smithi100.stdout: "snapshot_clone_delay": { 2024-05-10T11:44:17.025 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "snapshot_clone_delay", 2024-05-10T11:44:17.025 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:17.025 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:17.025 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:17.025 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:17.025 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:17.025 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:17.025 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:17.025 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-10T11:44:17.025 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:17.025 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:17.025 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:17.025 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:17.025 INFO:teuthology.orchestra.run.smithi100.stdout: "snapshot_clone_no_wait": { 2024-05-10T11:44:17.025 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "snapshot_clone_no_wait", 2024-05-10T11:44:17.026 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:17.026 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:17.026 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:17.026 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "True", 2024-05-10T11:44:17.026 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:17.026 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:17.026 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:17.026 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-10T11:44:17.026 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:17.026 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:17.026 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:17.026 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:17.026 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:17.027 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:17.027 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:17.027 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:17.027 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:17.027 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:17.027 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:17.027 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:17.027 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:17.027 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:17.027 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:17.027 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:17.027 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:17.027 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:17.027 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:17.027 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:17.028 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:17.028 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "zabbix", 2024-05-10T11:44:17.028 INFO:teuthology.orchestra.run.smithi100.stdout: "can_run": true, 2024-05-10T11:44:17.028 INFO:teuthology.orchestra.run.smithi100.stdout: "error_string": "", 2024-05-10T11:44:17.028 INFO:teuthology.orchestra.run.smithi100.stdout: "module_options": { 2024-05-10T11:44:17.028 INFO:teuthology.orchestra.run.smithi100.stdout: "discovery_interval": { 2024-05-10T11:44:17.028 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "discovery_interval", 2024-05-10T11:44:17.028 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "uint", 2024-05-10T11:44:17.028 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:17.028 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:17.028 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "100", 2024-05-10T11:44:17.028 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:17.028 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:17.028 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:17.028 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:17.029 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:17.029 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:17.029 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:17.029 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:17.029 INFO:teuthology.orchestra.run.smithi100.stdout: "identifier": { 2024-05-10T11:44:17.029 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "identifier", 2024-05-10T11:44:17.029 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:17.029 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:17.029 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:17.029 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:17.029 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:17.029 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:17.029 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:17.029 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:17.030 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:17.030 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:17.030 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:17.030 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:17.030 INFO:teuthology.orchestra.run.smithi100.stdout: "interval": { 2024-05-10T11:44:17.030 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "interval", 2024-05-10T11:44:17.030 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "secs", 2024-05-10T11:44:17.030 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:17.030 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:17.030 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "60", 2024-05-10T11:44:17.030 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:17.030 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:17.030 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:17.030 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:17.030 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:17.031 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:17.031 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:17.031 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:17.031 INFO:teuthology.orchestra.run.smithi100.stdout: "log_level": { 2024-05-10T11:44:17.031 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_level", 2024-05-10T11:44:17.031 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:17.031 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:17.031 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:17.031 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:17.031 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:17.031 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:17.031 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:17.031 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:17.031 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:17.031 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:17.032 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:17.032 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:17.032 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:17.032 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:17.032 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:17.032 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:17.032 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:17.032 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:17.032 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:17.032 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster": { 2024-05-10T11:44:17.032 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster", 2024-05-10T11:44:17.032 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:17.032 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:17.032 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:17.032 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:17.033 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:17.033 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:17.033 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:17.033 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:17.033 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:17.033 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:17.033 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:17.033 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:17.033 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_cluster_level": { 2024-05-10T11:44:17.033 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_cluster_level", 2024-05-10T11:44:17.033 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:17.033 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:17.033 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:17.033 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "info", 2024-05-10T11:44:17.033 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:17.034 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:17.034 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [ 2024-05-10T11:44:17.034 INFO:teuthology.orchestra.run.smithi100.stdout: "", 2024-05-10T11:44:17.034 INFO:teuthology.orchestra.run.smithi100.stdout: "critical", 2024-05-10T11:44:17.034 INFO:teuthology.orchestra.run.smithi100.stdout: "debug", 2024-05-10T11:44:17.034 INFO:teuthology.orchestra.run.smithi100.stdout: "error", 2024-05-10T11:44:17.034 INFO:teuthology.orchestra.run.smithi100.stdout: "info", 2024-05-10T11:44:17.034 INFO:teuthology.orchestra.run.smithi100.stdout: "warning" 2024-05-10T11:44:17.034 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:17.034 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:17.034 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:17.034 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:17.034 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:17.034 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:17.035 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": { 2024-05-10T11:44:17.035 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "log_to_file", 2024-05-10T11:44:17.035 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "bool", 2024-05-10T11:44:17.035 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:17.035 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:17.035 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "False", 2024-05-10T11:44:17.035 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:17.035 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:17.035 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:17.035 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:17.035 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:17.035 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:17.035 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:17.035 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:17.035 INFO:teuthology.orchestra.run.smithi100.stdout: "sqlite3_killpoint": { 2024-05-10T11:44:17.036 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "sqlite3_killpoint", 2024-05-10T11:44:17.036 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:17.036 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "dev", 2024-05-10T11:44:17.036 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-05-10T11:44:17.036 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "0", 2024-05-10T11:44:17.036 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:17.036 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:17.036 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:17.036 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:17.036 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:17.036 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:17.036 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:17.036 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:17.036 INFO:teuthology.orchestra.run.smithi100.stdout: "zabbix_host": { 2024-05-10T11:44:17.036 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "zabbix_host", 2024-05-10T11:44:17.037 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:17.037 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:17.037 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:17.037 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "", 2024-05-10T11:44:17.037 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:17.037 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:17.037 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:17.037 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:17.037 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:17.037 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:17.037 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:17.037 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:17.037 INFO:teuthology.orchestra.run.smithi100.stdout: "zabbix_port": { 2024-05-10T11:44:17.037 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "zabbix_port", 2024-05-10T11:44:17.037 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "int", 2024-05-10T11:44:17.037 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:17.038 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:17.038 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "10051", 2024-05-10T11:44:17.038 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:17.038 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:17.038 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:17.038 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:17.038 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:17.038 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:17.038 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:17.038 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:17.038 INFO:teuthology.orchestra.run.smithi100.stdout: "zabbix_sender": { 2024-05-10T11:44:17.038 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "zabbix_sender", 2024-05-10T11:44:17.038 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "str", 2024-05-10T11:44:17.038 INFO:teuthology.orchestra.run.smithi100.stdout: "level": "advanced", 2024-05-10T11:44:17.038 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 0, 2024-05-10T11:44:17.039 INFO:teuthology.orchestra.run.smithi100.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-10T11:44:17.039 INFO:teuthology.orchestra.run.smithi100.stdout: "min": "", 2024-05-10T11:44:17.039 INFO:teuthology.orchestra.run.smithi100.stdout: "max": "", 2024-05-10T11:44:17.039 INFO:teuthology.orchestra.run.smithi100.stdout: "enum_allowed": [], 2024-05-10T11:44:17.039 INFO:teuthology.orchestra.run.smithi100.stdout: "desc": "", 2024-05-10T11:44:17.039 INFO:teuthology.orchestra.run.smithi100.stdout: "long_desc": "", 2024-05-10T11:44:17.039 INFO:teuthology.orchestra.run.smithi100.stdout: "tags": [], 2024-05-10T11:44:17.039 INFO:teuthology.orchestra.run.smithi100.stdout: "see_also": [] 2024-05-10T11:44:17.039 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:17.039 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:17.039 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:17.039 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:17.039 INFO:teuthology.orchestra.run.smithi100.stdout: "services": { 2024-05-10T11:44:17.039 INFO:teuthology.orchestra.run.smithi100.stdout: "dashboard": "https://172.21.15.100:8443/" 2024-05-10T11:44:17.039 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:17.040 INFO:teuthology.orchestra.run.smithi100.stdout: "always_on_modules": { 2024-05-10T11:44:17.040 INFO:teuthology.orchestra.run.smithi100.stdout: "octopus": [ 2024-05-10T11:44:17.040 INFO:teuthology.orchestra.run.smithi100.stdout: "balancer", 2024-05-10T11:44:17.040 INFO:teuthology.orchestra.run.smithi100.stdout: "crash", 2024-05-10T11:44:17.040 INFO:teuthology.orchestra.run.smithi100.stdout: "devicehealth", 2024-05-10T11:44:17.040 INFO:teuthology.orchestra.run.smithi100.stdout: "orchestrator", 2024-05-10T11:44:17.040 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_autoscaler", 2024-05-10T11:44:17.040 INFO:teuthology.orchestra.run.smithi100.stdout: "progress", 2024-05-10T11:44:17.040 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_support", 2024-05-10T11:44:17.040 INFO:teuthology.orchestra.run.smithi100.stdout: "status", 2024-05-10T11:44:17.040 INFO:teuthology.orchestra.run.smithi100.stdout: "telemetry", 2024-05-10T11:44:17.040 INFO:teuthology.orchestra.run.smithi100.stdout: "volumes" 2024-05-10T11:44:17.040 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:17.040 INFO:teuthology.orchestra.run.smithi100.stdout: "pacific": [ 2024-05-10T11:44:17.040 INFO:teuthology.orchestra.run.smithi100.stdout: "balancer", 2024-05-10T11:44:17.041 INFO:teuthology.orchestra.run.smithi100.stdout: "crash", 2024-05-10T11:44:17.041 INFO:teuthology.orchestra.run.smithi100.stdout: "devicehealth", 2024-05-10T11:44:17.041 INFO:teuthology.orchestra.run.smithi100.stdout: "orchestrator", 2024-05-10T11:44:17.041 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_autoscaler", 2024-05-10T11:44:17.041 INFO:teuthology.orchestra.run.smithi100.stdout: "progress", 2024-05-10T11:44:17.041 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_support", 2024-05-10T11:44:17.041 INFO:teuthology.orchestra.run.smithi100.stdout: "status", 2024-05-10T11:44:17.041 INFO:teuthology.orchestra.run.smithi100.stdout: "telemetry", 2024-05-10T11:44:17.041 INFO:teuthology.orchestra.run.smithi100.stdout: "volumes" 2024-05-10T11:44:17.041 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:17.041 INFO:teuthology.orchestra.run.smithi100.stdout: "quincy": [ 2024-05-10T11:44:17.041 INFO:teuthology.orchestra.run.smithi100.stdout: "balancer", 2024-05-10T11:44:17.041 INFO:teuthology.orchestra.run.smithi100.stdout: "crash", 2024-05-10T11:44:17.041 INFO:teuthology.orchestra.run.smithi100.stdout: "devicehealth", 2024-05-10T11:44:17.041 INFO:teuthology.orchestra.run.smithi100.stdout: "orchestrator", 2024-05-10T11:44:17.042 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_autoscaler", 2024-05-10T11:44:17.042 INFO:teuthology.orchestra.run.smithi100.stdout: "progress", 2024-05-10T11:44:17.042 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_support", 2024-05-10T11:44:17.042 INFO:teuthology.orchestra.run.smithi100.stdout: "status", 2024-05-10T11:44:17.042 INFO:teuthology.orchestra.run.smithi100.stdout: "telemetry", 2024-05-10T11:44:17.042 INFO:teuthology.orchestra.run.smithi100.stdout: "volumes" 2024-05-10T11:44:17.042 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:17.042 INFO:teuthology.orchestra.run.smithi100.stdout: "reef": [ 2024-05-10T11:44:17.042 INFO:teuthology.orchestra.run.smithi100.stdout: "balancer", 2024-05-10T11:44:17.042 INFO:teuthology.orchestra.run.smithi100.stdout: "crash", 2024-05-10T11:44:17.042 INFO:teuthology.orchestra.run.smithi100.stdout: "devicehealth", 2024-05-10T11:44:17.042 INFO:teuthology.orchestra.run.smithi100.stdout: "orchestrator", 2024-05-10T11:44:17.042 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_autoscaler", 2024-05-10T11:44:17.042 INFO:teuthology.orchestra.run.smithi100.stdout: "progress", 2024-05-10T11:44:17.042 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_support", 2024-05-10T11:44:17.043 INFO:teuthology.orchestra.run.smithi100.stdout: "status", 2024-05-10T11:44:17.043 INFO:teuthology.orchestra.run.smithi100.stdout: "telemetry", 2024-05-10T11:44:17.043 INFO:teuthology.orchestra.run.smithi100.stdout: "volumes" 2024-05-10T11:44:17.043 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-05-10T11:44:17.043 INFO:teuthology.orchestra.run.smithi100.stdout: "squid": [ 2024-05-10T11:44:17.043 INFO:teuthology.orchestra.run.smithi100.stdout: "balancer", 2024-05-10T11:44:17.043 INFO:teuthology.orchestra.run.smithi100.stdout: "crash", 2024-05-10T11:44:17.043 INFO:teuthology.orchestra.run.smithi100.stdout: "devicehealth", 2024-05-10T11:44:17.043 INFO:teuthology.orchestra.run.smithi100.stdout: "orchestrator", 2024-05-10T11:44:17.043 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_autoscaler", 2024-05-10T11:44:17.043 INFO:teuthology.orchestra.run.smithi100.stdout: "progress", 2024-05-10T11:44:17.043 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_support", 2024-05-10T11:44:17.043 INFO:teuthology.orchestra.run.smithi100.stdout: "status", 2024-05-10T11:44:17.043 INFO:teuthology.orchestra.run.smithi100.stdout: "telemetry", 2024-05-10T11:44:17.043 INFO:teuthology.orchestra.run.smithi100.stdout: "volumes" 2024-05-10T11:44:17.043 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-05-10T11:44:17.044 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:17.044 INFO:teuthology.orchestra.run.smithi100.stdout: "last_failure_osd_epoch": 3, 2024-05-10T11:44:17.044 INFO:teuthology.orchestra.run.smithi100.stdout: "active_clients": [ 2024-05-10T11:44:17.044 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:17.044 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "devicehealth", 2024-05-10T11:44:17.044 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-05-10T11:44:17.044 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:17.044 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-05-10T11:44:17.044 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:0", 2024-05-10T11:44:17.044 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2122358376 2024-05-10T11:44:17.044 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:17.044 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-05-10T11:44:17.044 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:17.044 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:17.044 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "libcephsqlite", 2024-05-10T11:44:17.045 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-05-10T11:44:17.045 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:17.045 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-05-10T11:44:17.045 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:0", 2024-05-10T11:44:17.045 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3925073324 2024-05-10T11:44:17.045 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:17.045 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-05-10T11:44:17.045 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:17.045 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:17.045 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "rbd_support", 2024-05-10T11:44:17.045 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-05-10T11:44:17.045 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:17.045 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-05-10T11:44:17.045 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:0", 2024-05-10T11:44:17.046 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3327438833 2024-05-10T11:44:17.046 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:17.046 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-05-10T11:44:17.046 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:17.046 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:17.046 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "volumes", 2024-05-10T11:44:17.046 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-05-10T11:44:17.046 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-05-10T11:44:17.046 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-05-10T11:44:17.046 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:0", 2024-05-10T11:44:17.046 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1927272520 2024-05-10T11:44:17.046 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:17.046 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-05-10T11:44:17.046 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:17.046 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-05-10T11:44:17.047 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:44:17.048 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:44:15 smithi178 ceph-mon[33359]: pgmap v1197: 129 pgs: 129 active+clean; 374 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 2.5 MiB/s wr, 333 op/s 2024-05-10T11:44:17.048 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:44:15 smithi178 ceph-mon[33359]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T11:44:17.048 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:44:15 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/2267235106' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T11:44:17.048 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:44:15 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/1009135002' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T11:44:17.048 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:44:15 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/290447203' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-10T11:44:17.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:44:15 smithi188 ceph-mon[34075]: pgmap v1197: 129 pgs: 129 active+clean; 374 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 2.5 MiB/s wr, 333 op/s 2024-05-10T11:44:17.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:44:15 smithi188 ceph-mon[34075]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T11:44:17.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:44:15 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/2267235106' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T11:44:17.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:44:15 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/1009135002' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T11:44:17.048 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:44:15 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/290447203' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-10T11:44:17.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.648+0000 7fd6f1bf7640 1 -- 172.21.15.100:0/290447203 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fd6d80b2870 msgr2=0x7fd6d80b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:17.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.648+0000 7fd6f1bf7640 1 --2- 172.21.15.100:0/290447203 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fd6d80b2870 0x7fd6d80b4d30 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc006010 tx=0x7fd6dc0095f0 comp rx=0 tx=0).stop 2024-05-10T11:44:17.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.648+0000 7fd6f1bf7640 1 -- 172.21.15.100:0/290447203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6ec1045b0 msgr2=0x7fd6ec1a0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:17.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.648+0000 7fd6f1bf7640 1 --2- 172.21.15.100:0/290447203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6ec1045b0 0x7fd6ec1a0170 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4009fd0 tx=0x7fd6e400b260 comp rx=0 tx=0).stop 2024-05-10T11:44:17.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.648+0000 7fd6f1bf7640 1 -- 172.21.15.100:0/290447203 shutdown_connections 2024-05-10T11:44:17.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.648+0000 7fd6f1bf7640 1 --2- 172.21.15.100:0/290447203 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fd6d80b2870 0x7fd6d80b4d30 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:17.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.648+0000 7fd6f1bf7640 1 --2- 172.21.15.100:0/290447203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6ec1045b0 0x7fd6ec1a0170 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:17.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.648+0000 7fd6f1bf7640 1 --2- 172.21.15.100:0/290447203 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd6ec103bf0 0x7fd6ec199100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:17.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.648+0000 7fd6f1bf7640 1 --2- 172.21.15.100:0/290447203 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6ec102a00 0x7fd6ec198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:17.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.648+0000 7fd6f1bf7640 1 -- 172.21.15.100:0/290447203 >> 172.21.15.100:0/290447203 conn(0x7fd6ec0fdf90 msgr2=0x7fd6ec107030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:44:17.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.648+0000 7fd6f1bf7640 1 -- 172.21.15.100:0/290447203 shutdown_connections 2024-05-10T11:44:17.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:15.648+0000 7fd6f1bf7640 1 -- 172.21.15.100:0/290447203 wait complete. 2024-05-10T11:44:17.057 DEBUG:tasks.check_counter:Getting stats from a 2024-05-10T11:44:17.057 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph daemon mds.a perf dump 2024-05-10T11:44:17.058 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:44:15 smithi100 ceph-mon[30296]: pgmap v1197: 129 pgs: 129 active+clean; 374 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 2.5 MiB/s wr, 333 op/s 2024-05-10T11:44:17.058 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:44:15 smithi100 ceph-mon[30296]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-10T11:44:17.058 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:44:15 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/2267235106' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T11:44:17.058 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:44:15 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/1009135002' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T11:44:17.058 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:44:15 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/290447203' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-10T11:44:17.315 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:44:18.076 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:44:17 smithi188 ceph-mon[34075]: pgmap v1198: 129 pgs: 129 active+clean; 350 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 2.0 MiB/s wr, 310 op/s 2024-05-10T11:44:18.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:44:17 smithi178 ceph-mon[33359]: pgmap v1198: 129 pgs: 129 active+clean; 350 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 2.0 MiB/s wr, 310 op/s 2024-05-10T11:44:18.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:44:17 smithi100 ceph-mon[30296]: pgmap v1198: 129 pgs: 129 active+clean; 350 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 2.0 MiB/s wr, 310 op/s 2024-05-10T11:44:20.153 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:44:19 smithi188 ceph-mon[34075]: pgmap v1199: 129 pgs: 129 active+clean; 349 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 1.3 MiB/s wr, 235 op/s 2024-05-10T11:44:20.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:44:19 smithi178 ceph-mon[33359]: pgmap v1199: 129 pgs: 129 active+clean; 349 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 1.3 MiB/s wr, 235 op/s 2024-05-10T11:44:20.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:44:19 smithi100 ceph-mon[30296]: pgmap v1199: 129 pgs: 129 active+clean; 349 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 1.3 MiB/s wr, 235 op/s 2024-05-10T11:44:22.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:44:22 smithi178 ceph-mon[33359]: pgmap v1200: 129 pgs: 129 active+clean; 348 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 900 KiB/s wr, 216 op/s 2024-05-10T11:44:22.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:44:22 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:44:22.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:44:22 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:44:22.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:44:22 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:44:22.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:44:22 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:44:22.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:44:22 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:44:22.513 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:44:22 smithi188 ceph-mon[34075]: pgmap v1200: 129 pgs: 129 active+clean; 348 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 900 KiB/s wr, 216 op/s 2024-05-10T11:44:22.513 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:44:22 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:44:22.513 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:44:22 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:44:22.513 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:44:22 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:44:22.513 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:44:22 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:44:22.513 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:44:22 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:44:22.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:44:22 smithi100 ceph-mon[30296]: pgmap v1200: 129 pgs: 129 active+clean; 348 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 900 KiB/s wr, 216 op/s 2024-05-10T11:44:22.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:44:22 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:44:22.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:44:22 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:44:22.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:44:22 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:44:22.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:44:22 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:44:22.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:44:22 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:44:23.691 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:44:23.692 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T11:44:23.692 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 484, 2024-05-10T11:44:23.692 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 481, 2024-05-10T11:44:23.692 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 589840, 2024-05-10T11:44:23.692 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 298208, 2024-05-10T11:44:23.692 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 4, 2024-05-10T11:44:23.692 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 0, 2024-05-10T11:44:23.692 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.057456750, 2024-05-10T11:44:23.692 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.025133284, 2024-05-10T11:44:23.692 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.018636787, 2024-05-10T11:44:23.693 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.009234018, 2024-05-10T11:44:23.693 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T11:44:23.693 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 480, 2024-05-10T11:44:23.693 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.014095151, 2024-05-10T11:44:23.693 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000029364 2024-05-10T11:44:23.693 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:23.693 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-05-10T11:44:23.693 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-05-10T11:44:23.693 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-05-10T11:44:23.693 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-05-10T11:44:23.693 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:23.693 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 589840, 2024-05-10T11:44:23.693 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 298208 2024-05-10T11:44:23.693 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:23.693 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T11:44:23.694 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 7, 2024-05-10T11:44:23.694 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 371, 2024-05-10T11:44:23.694 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 3392, 2024-05-10T11:44:23.694 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 301440, 2024-05-10T11:44:23.694 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 5, 2024-05-10T11:44:23.694 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 2, 2024-05-10T11:44:23.694 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.023107946, 2024-05-10T11:44:23.694 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.018223848, 2024-05-10T11:44:23.694 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.002096129, 2024-05-10T11:44:23.694 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-10T11:44:23.694 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T11:44:23.694 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 370, 2024-05-10T11:44:23.694 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.014583597, 2024-05-10T11:44:23.694 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000039415 2024-05-10T11:44:23.694 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:23.695 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-05-10T11:44:23.695 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-05-10T11:44:23.695 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-05-10T11:44:23.695 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-05-10T11:44:23.695 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:23.695 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 3392, 2024-05-10T11:44:23.695 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 301440 2024-05-10T11:44:23.695 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:23.695 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T11:44:23.695 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 0, 2024-05-10T11:44:23.695 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 2, 2024-05-10T11:44:23.695 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 0, 2024-05-10T11:44:23.695 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 0, 2024-05-10T11:44:23.695 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 2, 2024-05-10T11:44:23.695 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 1, 2024-05-10T11:44:23.696 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.000311700, 2024-05-10T11:44:23.696 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.000000000, 2024-05-10T11:44:23.696 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.000226575, 2024-05-10T11:44:23.696 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-10T11:44:23.696 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T11:44:23.696 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-05-10T11:44:23.696 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-05-10T11:44:23.696 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-05-10T11:44:23.696 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:23.696 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-05-10T11:44:23.696 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-05-10T11:44:23.696 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-05-10T11:44:23.696 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-05-10T11:44:23.696 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:23.696 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-10T11:44:23.697 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 0 2024-05-10T11:44:23.697 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:23.697 INFO:teuthology.orchestra.run.smithi100.stdout: "cct": { 2024-05-10T11:44:23.697 INFO:teuthology.orchestra.run.smithi100.stdout: "total_workers": 0, 2024-05-10T11:44:23.697 INFO:teuthology.orchestra.run.smithi100.stdout: "unhealthy_workers": 0 2024-05-10T11:44:23.697 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:23.697 INFO:teuthology.orchestra.run.smithi100.stdout: "mempool": { 2024-05-10T11:44:23.697 INFO:teuthology.orchestra.run.smithi100.stdout: "bloom_filter_bytes": 0, 2024-05-10T11:44:23.697 INFO:teuthology.orchestra.run.smithi100.stdout: "bloom_filter_items": 0, 2024-05-10T11:44:23.697 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T11:44:23.697 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_alloc_items": 0, 2024-05-10T11:44:23.697 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T11:44:23.697 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_data_items": 0, 2024-05-10T11:44:23.697 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T11:44:23.698 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T11:44:23.698 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T11:44:23.698 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T11:44:23.698 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T11:44:23.698 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_other_items": 0, 2024-05-10T11:44:23.698 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T11:44:23.698 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T11:44:23.698 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_extent_bytes": 0, 2024-05-10T11:44:23.698 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_extent_items": 0, 2024-05-10T11:44:23.698 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_blob_bytes": 0, 2024-05-10T11:44:23.698 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_blob_items": 0, 2024-05-10T11:44:23.698 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T11:44:23.698 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T11:44:23.698 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T11:44:23.698 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T11:44:23.699 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T11:44:23.699 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_items": 0, 2024-05-10T11:44:23.699 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_txc_bytes": 0, 2024-05-10T11:44:23.699 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_txc_items": 0, 2024-05-10T11:44:23.699 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T11:44:23.699 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T11:44:23.699 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_bytes": 0, 2024-05-10T11:44:23.699 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_items": 0, 2024-05-10T11:44:23.699 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_bytes": 0, 2024-05-10T11:44:23.699 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_items": 0, 2024-05-10T11:44:23.699 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T11:44:23.699 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_reader_items": 0, 2024-05-10T11:44:23.699 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T11:44:23.699 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_writer_items": 0, 2024-05-10T11:44:23.699 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_anon_bytes": 147960, 2024-05-10T11:44:23.700 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_anon_items": 28, 2024-05-10T11:44:23.700 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_meta_bytes": 0, 2024-05-10T11:44:23.700 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_meta_items": 0, 2024-05-10T11:44:23.700 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_bytes": 0, 2024-05-10T11:44:23.700 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_items": 0, 2024-05-10T11:44:23.700 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mapbl_bytes": 0, 2024-05-10T11:44:23.700 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mapbl_items": 0, 2024-05-10T11:44:23.700 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pglog_bytes": 0, 2024-05-10T11:44:23.700 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pglog_items": 0, 2024-05-10T11:44:23.700 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_bytes": 0, 2024-05-10T11:44:23.700 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_items": 0, 2024-05-10T11:44:23.700 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T11:44:23.700 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_mapping_items": 0, 2024-05-10T11:44:23.700 INFO:teuthology.orchestra.run.smithi100.stdout: "pgmap_bytes": 0, 2024-05-10T11:44:23.700 INFO:teuthology.orchestra.run.smithi100.stdout: "pgmap_items": 0, 2024-05-10T11:44:23.701 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_co_bytes": 928, 2024-05-10T11:44:23.701 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_co_items": 2, 2024-05-10T11:44:23.701 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_1_bytes": 0, 2024-05-10T11:44:23.701 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_1_items": 0, 2024-05-10T11:44:23.701 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_2_bytes": 0, 2024-05-10T11:44:23.701 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_2_items": 0 2024-05-10T11:44:23.701 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:23.701 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T11:44:23.701 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-05-10T11:44:23.701 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 104857600, 2024-05-10T11:44:23.701 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-05-10T11:44:23.701 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 487, 2024-05-10T11:44:23.701 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 521395, 2024-05-10T11:44:23.701 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-05-10T11:44:23.701 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 487, 2024-05-10T11:44:23.702 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 0, 2024-05-10T11:44:23.702 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 0, 2024-05-10T11:44:23.702 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 487, 2024-05-10T11:44:23.702 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 521395, 2024-05-10T11:44:23.702 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-05-10T11:44:23.702 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-05-10T11:44:23.702 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-05-10T11:44:23.702 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-05-10T11:44:23.702 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:23.702 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:23.702 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:44:24.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:44:24 smithi178 ceph-mon[33359]: pgmap v1201: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 191 KiB/s wr, 129 op/s 2024-05-10T11:44:24.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:44:24 smithi188 ceph-mon[34075]: pgmap v1201: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 191 KiB/s wr, 129 op/s 2024-05-10T11:44:24.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:44:24 smithi100 ceph-mon[30296]: pgmap v1201: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 191 KiB/s wr, 129 op/s 2024-05-10T11:44:24.579 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-05-10T11:44:24.579 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-05-10T11:44:24.579 DEBUG:tasks.check_counter:Getting stats from d 2024-05-10T11:44:24.580 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph daemon mds.d perf dump 2024-05-10T11:44:24.820 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:44:26.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:44:26 smithi100 ceph-mon[30296]: pgmap v1202: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 34 KiB/s wr, 75 op/s 2024-05-10T11:44:26.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:44:26 smithi178 ceph-mon[33359]: pgmap v1202: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 34 KiB/s wr, 75 op/s 2024-05-10T11:44:26.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:44:26 smithi188 ceph-mon[34075]: pgmap v1202: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 34 KiB/s wr, 75 op/s 2024-05-10T11:44:26.558 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:44:26.558 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T11:44:26.558 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 0, 2024-05-10T11:44:26.558 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 2, 2024-05-10T11:44:26.558 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 0, 2024-05-10T11:44:26.558 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 0, 2024-05-10T11:44:26.558 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 4, 2024-05-10T11:44:26.558 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 0, 2024-05-10T11:44:26.559 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.000968932, 2024-05-10T11:44:26.559 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.000000000, 2024-05-10T11:44:26.559 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.000101963, 2024-05-10T11:44:26.559 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-10T11:44:26.559 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T11:44:26.559 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-05-10T11:44:26.559 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-05-10T11:44:26.559 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-05-10T11:44:26.559 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:26.559 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-05-10T11:44:26.559 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-05-10T11:44:26.559 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-05-10T11:44:26.559 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-05-10T11:44:26.559 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:26.560 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-10T11:44:26.560 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 0 2024-05-10T11:44:26.560 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:26.560 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T11:44:26.560 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 495, 2024-05-10T11:44:26.560 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 490, 2024-05-10T11:44:26.560 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 594720, 2024-05-10T11:44:26.560 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 302656, 2024-05-10T11:44:26.560 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 2, 2024-05-10T11:44:26.560 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 1, 2024-05-10T11:44:26.560 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.056558975, 2024-05-10T11:44:26.560 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.023691903, 2024-05-10T11:44:26.560 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.020351098, 2024-05-10T11:44:26.560 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.009371554, 2024-05-10T11:44:26.560 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T11:44:26.560 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 490, 2024-05-10T11:44:26.561 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.014791541, 2024-05-10T11:44:26.561 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000030186 2024-05-10T11:44:26.561 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:26.561 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-05-10T11:44:26.561 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-05-10T11:44:26.561 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-05-10T11:44:26.561 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-05-10T11:44:26.561 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:26.561 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 594720, 2024-05-10T11:44:26.561 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 302656 2024-05-10T11:44:26.561 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:26.561 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T11:44:26.561 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 3, 2024-05-10T11:44:26.561 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 374, 2024-05-10T11:44:26.562 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 480, 2024-05-10T11:44:26.562 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 301872, 2024-05-10T11:44:26.562 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 5, 2024-05-10T11:44:26.562 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 2, 2024-05-10T11:44:26.562 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.022344369, 2024-05-10T11:44:26.562 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.017712155, 2024-05-10T11:44:26.562 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.001801915, 2024-05-10T11:44:26.562 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-10T11:44:26.562 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T11:44:26.562 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 372, 2024-05-10T11:44:26.562 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.015997581, 2024-05-10T11:44:26.562 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000043004 2024-05-10T11:44:26.562 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:26.562 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-05-10T11:44:26.562 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-05-10T11:44:26.563 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-05-10T11:44:26.563 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-05-10T11:44:26.563 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:26.563 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 480, 2024-05-10T11:44:26.563 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 301872 2024-05-10T11:44:26.563 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:26.563 INFO:teuthology.orchestra.run.smithi100.stdout: "cct": { 2024-05-10T11:44:26.563 INFO:teuthology.orchestra.run.smithi100.stdout: "total_workers": 0, 2024-05-10T11:44:26.563 INFO:teuthology.orchestra.run.smithi100.stdout: "unhealthy_workers": 0 2024-05-10T11:44:26.563 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:26.563 INFO:teuthology.orchestra.run.smithi100.stdout: "mempool": { 2024-05-10T11:44:26.563 INFO:teuthology.orchestra.run.smithi100.stdout: "bloom_filter_bytes": 0, 2024-05-10T11:44:26.563 INFO:teuthology.orchestra.run.smithi100.stdout: "bloom_filter_items": 0, 2024-05-10T11:44:26.563 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T11:44:26.563 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_alloc_items": 0, 2024-05-10T11:44:26.564 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T11:44:26.564 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_data_items": 0, 2024-05-10T11:44:26.564 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T11:44:26.564 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T11:44:26.564 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T11:44:26.564 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T11:44:26.564 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T11:44:26.564 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_other_items": 0, 2024-05-10T11:44:26.564 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T11:44:26.564 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T11:44:26.564 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_extent_bytes": 0, 2024-05-10T11:44:26.564 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_extent_items": 0, 2024-05-10T11:44:26.564 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_blob_bytes": 0, 2024-05-10T11:44:26.564 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_blob_items": 0, 2024-05-10T11:44:26.564 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T11:44:26.565 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T11:44:26.565 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T11:44:26.565 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T11:44:26.565 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T11:44:26.565 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_items": 0, 2024-05-10T11:44:26.565 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_txc_bytes": 0, 2024-05-10T11:44:26.565 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_txc_items": 0, 2024-05-10T11:44:26.565 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T11:44:26.565 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T11:44:26.565 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_bytes": 0, 2024-05-10T11:44:26.565 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_items": 0, 2024-05-10T11:44:26.565 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_bytes": 0, 2024-05-10T11:44:26.565 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_items": 0, 2024-05-10T11:44:26.565 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T11:44:26.566 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_reader_items": 0, 2024-05-10T11:44:26.566 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T11:44:26.566 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_writer_items": 0, 2024-05-10T11:44:26.566 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_anon_bytes": 148699, 2024-05-10T11:44:26.566 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_anon_items": 30, 2024-05-10T11:44:26.566 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_meta_bytes": 0, 2024-05-10T11:44:26.566 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_meta_items": 0, 2024-05-10T11:44:26.566 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_bytes": 0, 2024-05-10T11:44:26.566 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_items": 0, 2024-05-10T11:44:26.566 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mapbl_bytes": 0, 2024-05-10T11:44:26.566 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mapbl_items": 0, 2024-05-10T11:44:26.566 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pglog_bytes": 0, 2024-05-10T11:44:26.566 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pglog_items": 0, 2024-05-10T11:44:26.566 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_bytes": 0, 2024-05-10T11:44:26.566 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_items": 0, 2024-05-10T11:44:26.566 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T11:44:26.567 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_mapping_items": 0, 2024-05-10T11:44:26.567 INFO:teuthology.orchestra.run.smithi100.stdout: "pgmap_bytes": 0, 2024-05-10T11:44:26.567 INFO:teuthology.orchestra.run.smithi100.stdout: "pgmap_items": 0, 2024-05-10T11:44:26.567 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_co_bytes": 928, 2024-05-10T11:44:26.567 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_co_items": 2, 2024-05-10T11:44:26.567 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_1_bytes": 0, 2024-05-10T11:44:26.567 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_1_items": 0, 2024-05-10T11:44:26.567 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_2_bytes": 0, 2024-05-10T11:44:26.567 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_2_items": 0 2024-05-10T11:44:26.567 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:26.567 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T11:44:26.567 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-05-10T11:44:26.567 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 104857600, 2024-05-10T11:44:26.567 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-05-10T11:44:26.567 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 494, 2024-05-10T11:44:26.568 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 522369, 2024-05-10T11:44:26.568 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-05-10T11:44:26.568 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 494, 2024-05-10T11:44:26.568 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 0, 2024-05-10T11:44:26.568 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 0, 2024-05-10T11:44:26.568 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 494, 2024-05-10T11:44:26.568 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 522369, 2024-05-10T11:44:26.568 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-05-10T11:44:26.568 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-05-10T11:44:26.568 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-05-10T11:44:26.568 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-05-10T11:44:26.568 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:26.568 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:26.568 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:44:27.249 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-05-10T11:44:27.250 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-05-10T11:44:27.250 DEBUG:tasks.check_counter:Getting stats from g 2024-05-10T11:44:27.250 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph daemon mds.g perf dump 2024-05-10T11:44:27.494 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:44:28.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:44:28 smithi100 ceph-mon[30296]: pgmap v1203: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 33 KiB/s wr, 30 op/s 2024-05-10T11:44:28.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:44:28 smithi178 ceph-mon[33359]: pgmap v1203: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 33 KiB/s wr, 30 op/s 2024-05-10T11:44:28.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:44:28 smithi188 ceph-mon[34075]: pgmap v1203: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 33 KiB/s wr, 30 op/s 2024-05-10T11:44:29.463 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:44:29.463 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T11:44:29.463 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 0, 2024-05-10T11:44:29.464 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 2, 2024-05-10T11:44:29.464 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 0, 2024-05-10T11:44:29.464 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 0, 2024-05-10T11:44:29.464 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 4, 2024-05-10T11:44:29.464 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 0, 2024-05-10T11:44:29.464 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.001067745, 2024-05-10T11:44:29.464 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.000000000, 2024-05-10T11:44:29.464 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.000160350, 2024-05-10T11:44:29.464 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-10T11:44:29.464 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T11:44:29.464 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-05-10T11:44:29.464 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-05-10T11:44:29.464 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-05-10T11:44:29.464 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:29.464 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-05-10T11:44:29.465 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-05-10T11:44:29.465 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-05-10T11:44:29.465 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-05-10T11:44:29.465 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:29.465 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-10T11:44:29.465 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 0 2024-05-10T11:44:29.465 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:29.465 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T11:44:29.465 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 491, 2024-05-10T11:44:29.465 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 486, 2024-05-10T11:44:29.465 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 593632, 2024-05-10T11:44:29.465 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 300096, 2024-05-10T11:44:29.465 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 2, 2024-05-10T11:44:29.465 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 1, 2024-05-10T11:44:29.465 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.058647940, 2024-05-10T11:44:29.466 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.024818698, 2024-05-10T11:44:29.466 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.022152275, 2024-05-10T11:44:29.466 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.009279135, 2024-05-10T11:44:29.466 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T11:44:29.466 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 486, 2024-05-10T11:44:29.466 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.015937725, 2024-05-10T11:44:29.466 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000032793 2024-05-10T11:44:29.466 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:29.466 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-05-10T11:44:29.466 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-05-10T11:44:29.466 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-05-10T11:44:29.466 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-05-10T11:44:29.466 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:29.466 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 593632, 2024-05-10T11:44:29.466 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 300096 2024-05-10T11:44:29.467 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:29.467 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T11:44:29.467 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 3, 2024-05-10T11:44:29.467 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 371, 2024-05-10T11:44:29.467 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 480, 2024-05-10T11:44:29.467 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 301344, 2024-05-10T11:44:29.467 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 5, 2024-05-10T11:44:29.467 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 2, 2024-05-10T11:44:29.467 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.022668207, 2024-05-10T11:44:29.467 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.018183810, 2024-05-10T11:44:29.467 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.001629492, 2024-05-10T11:44:29.467 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-10T11:44:29.467 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T11:44:29.467 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 369, 2024-05-10T11:44:29.467 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.012989760, 2024-05-10T11:44:29.468 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000035202 2024-05-10T11:44:29.468 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:29.468 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-05-10T11:44:29.468 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-05-10T11:44:29.468 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-05-10T11:44:29.468 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-05-10T11:44:29.468 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:29.468 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 480, 2024-05-10T11:44:29.468 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 301344 2024-05-10T11:44:29.468 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:29.468 INFO:teuthology.orchestra.run.smithi100.stdout: "cct": { 2024-05-10T11:44:29.468 INFO:teuthology.orchestra.run.smithi100.stdout: "total_workers": 0, 2024-05-10T11:44:29.468 INFO:teuthology.orchestra.run.smithi100.stdout: "unhealthy_workers": 0 2024-05-10T11:44:29.468 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:29.468 INFO:teuthology.orchestra.run.smithi100.stdout: "mempool": { 2024-05-10T11:44:29.468 INFO:teuthology.orchestra.run.smithi100.stdout: "bloom_filter_bytes": 0, 2024-05-10T11:44:29.469 INFO:teuthology.orchestra.run.smithi100.stdout: "bloom_filter_items": 0, 2024-05-10T11:44:29.469 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T11:44:29.469 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_alloc_items": 0, 2024-05-10T11:44:29.469 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T11:44:29.469 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_data_items": 0, 2024-05-10T11:44:29.469 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T11:44:29.469 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T11:44:29.469 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T11:44:29.469 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T11:44:29.469 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T11:44:29.469 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_other_items": 0, 2024-05-10T11:44:29.469 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T11:44:29.469 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T11:44:29.469 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_extent_bytes": 0, 2024-05-10T11:44:29.469 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_extent_items": 0, 2024-05-10T11:44:29.470 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_blob_bytes": 0, 2024-05-10T11:44:29.470 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_blob_items": 0, 2024-05-10T11:44:29.470 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T11:44:29.470 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T11:44:29.470 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T11:44:29.470 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T11:44:29.470 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T11:44:29.470 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_items": 0, 2024-05-10T11:44:29.470 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_txc_bytes": 0, 2024-05-10T11:44:29.470 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_txc_items": 0, 2024-05-10T11:44:29.470 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T11:44:29.470 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T11:44:29.470 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_bytes": 0, 2024-05-10T11:44:29.470 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_items": 0, 2024-05-10T11:44:29.471 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_bytes": 0, 2024-05-10T11:44:29.471 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_items": 0, 2024-05-10T11:44:29.471 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T11:44:29.471 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_reader_items": 0, 2024-05-10T11:44:29.471 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T11:44:29.471 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_writer_items": 0, 2024-05-10T11:44:29.471 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_anon_bytes": 147876, 2024-05-10T11:44:29.471 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_anon_items": 27, 2024-05-10T11:44:29.471 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_meta_bytes": 0, 2024-05-10T11:44:29.472 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_meta_items": 0, 2024-05-10T11:44:29.472 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_bytes": 0, 2024-05-10T11:44:29.472 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_items": 0, 2024-05-10T11:44:29.472 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mapbl_bytes": 0, 2024-05-10T11:44:29.472 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mapbl_items": 0, 2024-05-10T11:44:29.472 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pglog_bytes": 0, 2024-05-10T11:44:29.472 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pglog_items": 0, 2024-05-10T11:44:29.472 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_bytes": 0, 2024-05-10T11:44:29.472 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_items": 0, 2024-05-10T11:44:29.472 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T11:44:29.472 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_mapping_items": 0, 2024-05-10T11:44:29.472 INFO:teuthology.orchestra.run.smithi100.stdout: "pgmap_bytes": 0, 2024-05-10T11:44:29.472 INFO:teuthology.orchestra.run.smithi100.stdout: "pgmap_items": 0, 2024-05-10T11:44:29.472 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_co_bytes": 928, 2024-05-10T11:44:29.472 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_co_items": 2, 2024-05-10T11:44:29.473 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_1_bytes": 0, 2024-05-10T11:44:29.473 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_1_items": 0, 2024-05-10T11:44:29.473 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_2_bytes": 0, 2024-05-10T11:44:29.473 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_2_items": 0 2024-05-10T11:44:29.473 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:29.473 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T11:44:29.473 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-05-10T11:44:29.473 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 104857600, 2024-05-10T11:44:29.473 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-05-10T11:44:29.473 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 490, 2024-05-10T11:44:29.473 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 521849, 2024-05-10T11:44:29.473 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-05-10T11:44:29.473 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 490, 2024-05-10T11:44:29.473 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 0, 2024-05-10T11:44:29.474 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 0, 2024-05-10T11:44:29.474 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 490, 2024-05-10T11:44:29.474 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 521849, 2024-05-10T11:44:29.474 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-05-10T11:44:29.474 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-05-10T11:44:29.474 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-05-10T11:44:29.474 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-05-10T11:44:29.474 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:29.474 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:29.474 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:44:30.240 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:44:30 smithi100 ceph-mon[30296]: pgmap v1204: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-05-10T11:44:30.271 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-05-10T11:44:30.271 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-05-10T11:44:30.272 DEBUG:tasks.check_counter:Getting stats from j 2024-05-10T11:44:30.272 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph daemon mds.j perf dump 2024-05-10T11:44:30.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:44:30 smithi178 ceph-mon[33359]: pgmap v1204: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-05-10T11:44:30.519 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:44:30.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:44:30 smithi188 ceph-mon[34075]: pgmap v1204: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-05-10T11:44:32.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:44:32 smithi100 ceph-mon[30296]: pgmap v1205: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.5 KiB/s wr, 10 op/s 2024-05-10T11:44:32.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:44:32 smithi188 ceph-mon[34075]: pgmap v1205: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.5 KiB/s wr, 10 op/s 2024-05-10T11:44:32.288 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-05-10T11:44:32.288 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T11:44:32.288 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 4, 2024-05-10T11:44:32.288 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 4, 2024-05-10T11:44:32.288 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 2912, 2024-05-10T11:44:32.288 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 448, 2024-05-10T11:44:32.288 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 4, 2024-05-10T11:44:32.288 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 0, 2024-05-10T11:44:32.288 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.001320799, 2024-05-10T11:44:32.288 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.000040039, 2024-05-10T11:44:32.289 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.000485756, 2024-05-10T11:44:32.289 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-10T11:44:32.289 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T11:44:32.289 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 3, 2024-05-10T11:44:32.289 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000697015, 2024-05-10T11:44:32.289 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000232338 2024-05-10T11:44:32.289 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:32.289 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-05-10T11:44:32.289 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-05-10T11:44:32.289 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-05-10T11:44:32.289 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-05-10T11:44:32.289 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:32.289 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 2912, 2024-05-10T11:44:32.289 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 448 2024-05-10T11:44:32.289 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:32.290 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T11:44:32.290 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 3, 2024-05-10T11:44:32.290 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 376, 2024-05-10T11:44:32.290 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 480, 2024-05-10T11:44:32.290 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 302224, 2024-05-10T11:44:32.290 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 5, 2024-05-10T11:44:32.290 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 2, 2024-05-10T11:44:32.290 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.022860712, 2024-05-10T11:44:32.290 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.017941089, 2024-05-10T11:44:32.290 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.002061356, 2024-05-10T11:44:32.290 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-10T11:44:32.290 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T11:44:32.290 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 374, 2024-05-10T11:44:32.291 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.014713444, 2024-05-10T11:44:32.291 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000039340 2024-05-10T11:44:32.291 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:32.291 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-05-10T11:44:32.291 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-05-10T11:44:32.291 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-05-10T11:44:32.291 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-05-10T11:44:32.291 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:32.291 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 480, 2024-05-10T11:44:32.291 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 302224 2024-05-10T11:44:32.291 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:32.291 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T11:44:32.291 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 493, 2024-05-10T11:44:32.291 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 490, 2024-05-10T11:44:32.291 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 592352, 2024-05-10T11:44:32.292 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 303488, 2024-05-10T11:44:32.292 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 2, 2024-05-10T11:44:32.292 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 1, 2024-05-10T11:44:32.292 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.058122821, 2024-05-10T11:44:32.292 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.024885524, 2024-05-10T11:44:32.292 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.020879774, 2024-05-10T11:44:32.292 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.009120325, 2024-05-10T11:44:32.292 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T11:44:32.292 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 489, 2024-05-10T11:44:32.292 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.013940998, 2024-05-10T11:44:32.292 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000028509 2024-05-10T11:44:32.292 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:32.292 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-05-10T11:44:32.292 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-05-10T11:44:32.292 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-05-10T11:44:32.292 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-05-10T11:44:32.293 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:32.293 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 592352, 2024-05-10T11:44:32.293 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 303488 2024-05-10T11:44:32.293 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:32.293 INFO:teuthology.orchestra.run.smithi100.stdout: "cct": { 2024-05-10T11:44:32.293 INFO:teuthology.orchestra.run.smithi100.stdout: "total_workers": 0, 2024-05-10T11:44:32.293 INFO:teuthology.orchestra.run.smithi100.stdout: "unhealthy_workers": 0 2024-05-10T11:44:32.293 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:32.293 INFO:teuthology.orchestra.run.smithi100.stdout: "mempool": { 2024-05-10T11:44:32.293 INFO:teuthology.orchestra.run.smithi100.stdout: "bloom_filter_bytes": 0, 2024-05-10T11:44:32.293 INFO:teuthology.orchestra.run.smithi100.stdout: "bloom_filter_items": 0, 2024-05-10T11:44:32.293 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T11:44:32.293 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_alloc_items": 0, 2024-05-10T11:44:32.293 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T11:44:32.293 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_data_items": 0, 2024-05-10T11:44:32.294 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T11:44:32.294 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T11:44:32.294 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T11:44:32.294 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T11:44:32.294 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T11:44:32.294 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_other_items": 0, 2024-05-10T11:44:32.294 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T11:44:32.294 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T11:44:32.294 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_extent_bytes": 0, 2024-05-10T11:44:32.294 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_extent_items": 0, 2024-05-10T11:44:32.294 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_blob_bytes": 0, 2024-05-10T11:44:32.294 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_blob_items": 0, 2024-05-10T11:44:32.294 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T11:44:32.294 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T11:44:32.295 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T11:44:32.295 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T11:44:32.295 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T11:44:32.295 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_items": 0, 2024-05-10T11:44:32.295 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_txc_bytes": 0, 2024-05-10T11:44:32.295 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_txc_items": 0, 2024-05-10T11:44:32.295 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T11:44:32.295 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T11:44:32.295 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_bytes": 0, 2024-05-10T11:44:32.295 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_items": 0, 2024-05-10T11:44:32.295 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_bytes": 0, 2024-05-10T11:44:32.295 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_items": 0, 2024-05-10T11:44:32.295 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T11:44:32.295 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_reader_items": 0, 2024-05-10T11:44:32.295 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T11:44:32.296 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_writer_items": 0, 2024-05-10T11:44:32.296 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_anon_bytes": 148000, 2024-05-10T11:44:32.296 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_anon_items": 29, 2024-05-10T11:44:32.296 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_meta_bytes": 0, 2024-05-10T11:44:32.296 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_meta_items": 0, 2024-05-10T11:44:32.296 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_bytes": 0, 2024-05-10T11:44:32.296 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_items": 0, 2024-05-10T11:44:32.296 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mapbl_bytes": 0, 2024-05-10T11:44:32.296 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mapbl_items": 0, 2024-05-10T11:44:32.296 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pglog_bytes": 0, 2024-05-10T11:44:32.296 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pglog_items": 0, 2024-05-10T11:44:32.296 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_bytes": 0, 2024-05-10T11:44:32.296 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_items": 0, 2024-05-10T11:44:32.296 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T11:44:32.296 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_mapping_items": 0, 2024-05-10T11:44:32.296 INFO:teuthology.orchestra.run.smithi100.stdout: "pgmap_bytes": 0, 2024-05-10T11:44:32.297 INFO:teuthology.orchestra.run.smithi100.stdout: "pgmap_items": 0, 2024-05-10T11:44:32.297 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_co_bytes": 928, 2024-05-10T11:44:32.297 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_co_items": 2, 2024-05-10T11:44:32.297 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_1_bytes": 0, 2024-05-10T11:44:32.297 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_1_items": 0, 2024-05-10T11:44:32.297 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_2_bytes": 0, 2024-05-10T11:44:32.297 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_2_items": 0 2024-05-10T11:44:32.297 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-05-10T11:44:32.297 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T11:44:32.297 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-05-10T11:44:32.297 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 104857600, 2024-05-10T11:44:32.297 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-05-10T11:44:32.297 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 496, 2024-05-10T11:44:32.297 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 522629, 2024-05-10T11:44:32.297 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-05-10T11:44:32.298 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 496, 2024-05-10T11:44:32.298 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 0, 2024-05-10T11:44:32.298 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 0, 2024-05-10T11:44:32.298 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 496, 2024-05-10T11:44:32.298 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 522629, 2024-05-10T11:44:32.298 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-05-10T11:44:32.298 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-05-10T11:44:32.298 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-05-10T11:44:32.298 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-05-10T11:44:32.298 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:32.298 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-05-10T11:44:32.298 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-05-10T11:44:32.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:44:32 smithi178 ceph-mon[33359]: pgmap v1205: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.5 KiB/s wr, 10 op/s 2024-05-10T11:44:33.004 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-05-10T11:44:33.004 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-05-10T11:44:33.005 DEBUG:tasks.check_counter:Getting stats from b 2024-05-10T11:44:33.005 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph daemon mds.b perf dump 2024-05-10T11:44:33.310 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T11:44:34.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:44:34 smithi178 ceph-mon[33359]: pgmap v1206: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-05-10T11:44:34.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:44:34 smithi188 ceph-mon[34075]: pgmap v1206: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-05-10T11:44:34.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:44:34 smithi100 ceph-mon[30296]: pgmap v1206: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-05-10T11:44:35.225 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-05-10T11:44:35.225 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T11:44:35.225 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 58705, 2024-05-10T11:44:35.225 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 58709, 2024-05-10T11:44:35.225 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 18215250, 2024-05-10T11:44:35.225 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 174000867, 2024-05-10T11:44:35.225 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 66, 2024-05-10T11:44:35.225 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 5, 2024-05-10T11:44:35.225 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 4.482360654, 2024-05-10T11:44:35.226 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 1.546880465, 2024-05-10T11:44:35.226 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.963012903, 2024-05-10T11:44:35.226 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 1.691643376, 2024-05-10T11:44:35.226 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T11:44:35.226 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 58705, 2024-05-10T11:44:35.226 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 2.354579202, 2024-05-10T11:44:35.226 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000040108 2024-05-10T11:44:35.226 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:35.226 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-05-10T11:44:35.226 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 56, 2024-05-10T11:44:35.226 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000005150, 2024-05-10T11:44:35.226 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000091 2024-05-10T11:44:35.226 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:35.226 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-10T11:44:35.227 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 0 2024-05-10T11:44:35.227 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:35.227 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T11:44:35.227 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 214023, 2024-05-10T11:44:35.227 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 189025, 2024-05-10T11:44:35.227 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 57466036, 2024-05-10T11:44:35.227 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 120996394, 2024-05-10T11:44:35.227 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 7, 2024-05-10T11:44:35.227 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 5, 2024-05-10T11:44:35.227 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 9.486978252, 2024-05-10T11:44:35.227 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 3.703530248, 2024-05-10T11:44:35.227 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 5.888128640, 2024-05-10T11:44:35.227 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.966263711, 2024-05-10T11:44:35.227 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T11:44:35.227 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 189027, 2024-05-10T11:44:35.228 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 3.930788711, 2024-05-10T11:44:35.228 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000020794 2024-05-10T11:44:35.228 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:35.228 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-05-10T11:44:35.228 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 334126, 2024-05-10T11:44:35.228 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.039254942, 2024-05-10T11:44:35.228 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000117 2024-05-10T11:44:35.228 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:35.228 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 713824, 2024-05-10T11:44:35.228 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 339168 2024-05-10T11:44:35.228 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:35.228 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T11:44:35.228 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 856141, 2024-05-10T11:44:35.228 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 992866, 2024-05-10T11:44:35.229 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 258184228, 2024-05-10T11:44:35.229 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 698616572, 2024-05-10T11:44:35.229 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 15, 2024-05-10T11:44:35.229 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 5, 2024-05-10T11:44:35.229 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 39.662449498, 2024-05-10T11:44:35.229 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 16.878285013, 2024-05-10T11:44:35.229 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 27.560153343, 2024-05-10T11:44:35.229 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.919783636, 2024-05-10T11:44:35.229 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T11:44:35.229 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 992864, 2024-05-10T11:44:35.229 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 17.299780004, 2024-05-10T11:44:35.229 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000017424 2024-05-10T11:44:35.229 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:35.230 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-05-10T11:44:35.230 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 1730209, 2024-05-10T11:44:35.230 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.213758718, 2024-05-10T11:44:35.230 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000123 2024-05-10T11:44:35.230 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:35.230 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 1904, 2024-05-10T11:44:35.230 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 807648 2024-05-10T11:44:35.230 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:35.230 INFO:teuthology.orchestra.run.smithi178.stdout: "cct": { 2024-05-10T11:44:35.230 INFO:teuthology.orchestra.run.smithi178.stdout: "total_workers": 1, 2024-05-10T11:44:35.230 INFO:teuthology.orchestra.run.smithi178.stdout: "unhealthy_workers": 0 2024-05-10T11:44:35.230 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:35.230 INFO:teuthology.orchestra.run.smithi178.stdout: "finisher-MDSRank": { 2024-05-10T11:44:35.231 INFO:teuthology.orchestra.run.smithi178.stdout: "queue_len": 0, 2024-05-10T11:44:35.231 INFO:teuthology.orchestra.run.smithi178.stdout: "complete_latency": { 2024-05-10T11:44:35.231 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 113283, 2024-05-10T11:44:35.231 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 76.442621775, 2024-05-10T11:44:35.231 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000674793 2024-05-10T11:44:35.231 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-05-10T11:44:35.231 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:35.231 INFO:teuthology.orchestra.run.smithi178.stdout: "finisher-PurgeQueue": { 2024-05-10T11:44:35.231 INFO:teuthology.orchestra.run.smithi178.stdout: "queue_len": 0, 2024-05-10T11:44:35.231 INFO:teuthology.orchestra.run.smithi178.stdout: "complete_latency": { 2024-05-10T11:44:35.231 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 48488, 2024-05-10T11:44:35.231 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 3.814876486, 2024-05-10T11:44:35.231 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000078676 2024-05-10T11:44:35.231 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-05-10T11:44:35.231 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:35.232 INFO:teuthology.orchestra.run.smithi178.stdout: "mds": { 2024-05-10T11:44:35.232 INFO:teuthology.orchestra.run.smithi178.stdout: "request": 726388, 2024-05-10T11:44:35.232 INFO:teuthology.orchestra.run.smithi178.stdout: "reply": 724207, 2024-05-10T11:44:35.232 INFO:teuthology.orchestra.run.smithi178.stdout: "reply_latency": { 2024-05-10T11:44:35.232 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 724207, 2024-05-10T11:44:35.232 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 471.803974077, 2024-05-10T11:44:35.232 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000651476 2024-05-10T11:44:35.232 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:35.232 INFO:teuthology.orchestra.run.smithi178.stdout: "slow_reply": 0, 2024-05-10T11:44:35.232 INFO:teuthology.orchestra.run.smithi178.stdout: "forward": 2181, 2024-05-10T11:44:35.232 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_fetch_complete": 1, 2024-05-10T11:44:35.232 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_fetch_keys": 0, 2024-05-10T11:44:35.232 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_commit": 16, 2024-05-10T11:44:35.232 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_split": 0, 2024-05-10T11:44:35.233 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_merge": 0, 2024-05-10T11:44:35.233 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes": 24, 2024-05-10T11:44:35.233 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_top": 6, 2024-05-10T11:44:35.233 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_bottom": 7, 2024-05-10T11:44:35.233 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_pin_tail": 11, 2024-05-10T11:44:35.233 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_pinned": 15, 2024-05-10T11:44:35.233 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_expired": 296, 2024-05-10T11:44:35.233 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_with_caps": 4, 2024-05-10T11:44:35.233 INFO:teuthology.orchestra.run.smithi178.stdout: "caps": 4, 2024-05-10T11:44:35.233 INFO:teuthology.orchestra.run.smithi178.stdout: "subtrees": 5, 2024-05-10T11:44:35.233 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse": 1428748, 2024-05-10T11:44:35.233 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_hit": 1341306, 2024-05-10T11:44:35.233 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_forward": 2173, 2024-05-10T11:44:35.233 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_discover": 33, 2024-05-10T11:44:35.233 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_dir_fetch": 0, 2024-05-10T11:44:35.234 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_remote_ino": 0, 2024-05-10T11:44:35.234 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_lock": 7, 2024-05-10T11:44:35.234 INFO:teuthology.orchestra.run.smithi178.stdout: "load_cent": 0, 2024-05-10T11:44:35.234 INFO:teuthology.orchestra.run.smithi178.stdout: "q": 0, 2024-05-10T11:44:35.234 INFO:teuthology.orchestra.run.smithi178.stdout: "exported": 11, 2024-05-10T11:44:35.234 INFO:teuthology.orchestra.run.smithi178.stdout: "exported_inodes": 108, 2024-05-10T11:44:35.234 INFO:teuthology.orchestra.run.smithi178.stdout: "imported": 12, 2024-05-10T11:44:35.234 INFO:teuthology.orchestra.run.smithi178.stdout: "imported_inodes": 396, 2024-05-10T11:44:35.234 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_dir_fetch": 0, 2024-05-10T11:44:35.234 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_backtrace_fetch": 3981, 2024-05-10T11:44:35.234 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_peer_discover": 0, 2024-05-10T11:44:35.234 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rfiles": 61, 2024-05-10T11:44:35.234 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rbytes": 12057077, 2024-05-10T11:44:35.234 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rsnaps": 0, 2024-05-10T11:44:35.235 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_backtrace_fetch": 3981, 2024-05-10T11:44:35.235 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_set_tag": 0, 2024-05-10T11:44:35.235 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_backtrace_repaired": 0, 2024-05-10T11:44:35.235 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_inotable_repaired": 0, 2024-05-10T11:44:35.235 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dir_inodes": 353, 2024-05-10T11:44:35.235 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dir_base_inodes": 0, 2024-05-10T11:44:35.235 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dirfrag_rstats": 353, 2024-05-10T11:44:35.235 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_file_inodes": 3628, 2024-05-10T11:44:35.235 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_inode_file_caps": 0, 2024-05-10T11:44:35.235 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_revoke": 50317, 2024-05-10T11:44:35.235 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_grant": 93586, 2024-05-10T11:44:35.235 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_trunc": 0, 2024-05-10T11:44:35.235 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-10T11:44:35.235 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-10T11:44:35.235 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_caps": 95894, 2024-05-10T11:44:35.236 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_caps_dirty": 34740, 2024-05-10T11:44:35.236 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_cap_release": 1385, 2024-05-10T11:44:35.236 INFO:teuthology.orchestra.run.smithi178.stdout: "process_request_cap_release": 92020 2024-05-10T11:44:35.236 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:35.236 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_cache": { 2024-05-10T11:44:35.236 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_update": 16, 2024-05-10T11:44:35.236 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_update_receipt": 20, 2024-05-10T11:44:35.236 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_try_discover": 13, 2024-05-10T11:44:35.236 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_send_discover": 39, 2024-05-10T11:44:35.236 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_handle_discover": 27, 2024-05-10T11:44:35.236 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays": 0, 2024-05-10T11:44:35.236 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays_delayed": 0, 2024-05-10T11:44:35.236 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays_enqueuing": 0, 2024-05-10T11:44:35.236 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_created": 25544, 2024-05-10T11:44:35.236 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_enqueued": 25544, 2024-05-10T11:44:35.236 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_reintegrated": 0, 2024-05-10T11:44:35.236 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_migrated": 0, 2024-05-10T11:44:35.237 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_processing": 0, 2024-05-10T11:44:35.237 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_enqueued": 0, 2024-05-10T11:44:35.237 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_prioritized": 0, 2024-05-10T11:44:35.237 INFO:teuthology.orchestra.run.smithi178.stdout: "recovery_started": 0, 2024-05-10T11:44:35.237 INFO:teuthology.orchestra.run.smithi178.stdout: "recovery_completed": 0, 2024-05-10T11:44:35.237 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_quiesce_path": 0, 2024-05-10T11:44:35.237 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_quiesce_inode": 0, 2024-05-10T11:44:35.237 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_enqueue_scrub": 0, 2024-05-10T11:44:35.237 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_exportdir": 11, 2024-05-10T11:44:35.237 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_flush": 0, 2024-05-10T11:44:35.237 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_fragmentdir": 0, 2024-05-10T11:44:35.237 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_fragstats": 0, 2024-05-10T11:44:35.237 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_inodestats": 0 2024-05-10T11:44:35.237 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:35.237 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_log": { 2024-05-10T11:44:35.237 INFO:teuthology.orchestra.run.smithi178.stdout: "evlrg": 0, 2024-05-10T11:44:35.238 INFO:teuthology.orchestra.run.smithi178.stdout: "evadd": 190453, 2024-05-10T11:44:35.238 INFO:teuthology.orchestra.run.smithi178.stdout: "evex": 99345, 2024-05-10T11:44:35.238 INFO:teuthology.orchestra.run.smithi178.stdout: "evtrm": 98321, 2024-05-10T11:44:35.238 INFO:teuthology.orchestra.run.smithi178.stdout: "ev": 92132, 2024-05-10T11:44:35.238 INFO:teuthology.orchestra.run.smithi178.stdout: "evexg": 0, 2024-05-10T11:44:35.238 INFO:teuthology.orchestra.run.smithi178.stdout: "evexd": 1024, 2024-05-10T11:44:35.238 INFO:teuthology.orchestra.run.smithi178.stdout: "segadd": 241, 2024-05-10T11:44:35.238 INFO:teuthology.orchestra.run.smithi178.stdout: "segex": 113, 2024-05-10T11:44:35.238 INFO:teuthology.orchestra.run.smithi178.stdout: "segtrm": 112, 2024-05-10T11:44:35.238 INFO:teuthology.orchestra.run.smithi178.stdout: "seg": 129, 2024-05-10T11:44:35.238 INFO:teuthology.orchestra.run.smithi178.stdout: "segmjr": 8, 2024-05-10T11:44:35.238 INFO:teuthology.orchestra.run.smithi178.stdout: "segexg": 0, 2024-05-10T11:44:35.238 INFO:teuthology.orchestra.run.smithi178.stdout: "segexd": 1, 2024-05-10T11:44:35.238 INFO:teuthology.orchestra.run.smithi178.stdout: "expos": 118327444, 2024-05-10T11:44:35.239 INFO:teuthology.orchestra.run.smithi178.stdout: "wrpos": 296980547, 2024-05-10T11:44:35.239 INFO:teuthology.orchestra.run.smithi178.stdout: "rdpos": 0, 2024-05-10T11:44:35.239 INFO:teuthology.orchestra.run.smithi178.stdout: "jlat": { 2024-05-10T11:44:35.239 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 56498, 2024-05-10T11:44:35.239 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 173.001224502, 2024-05-10T11:44:35.239 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.003062076 2024-05-10T11:44:35.239 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:35.239 INFO:teuthology.orchestra.run.smithi178.stdout: "replayed": 0 2024-05-10T11:44:35.239 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:35.239 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_mem": { 2024-05-10T11:44:35.239 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 29, 2024-05-10T11:44:35.239 INFO:teuthology.orchestra.run.smithi178.stdout: "ino+": 25721, 2024-05-10T11:44:35.239 INFO:teuthology.orchestra.run.smithi178.stdout: "ino-": 25692, 2024-05-10T11:44:35.239 INFO:teuthology.orchestra.run.smithi178.stdout: "dir": 27, 2024-05-10T11:44:35.239 INFO:teuthology.orchestra.run.smithi178.stdout: "dir+": 89, 2024-05-10T11:44:35.240 INFO:teuthology.orchestra.run.smithi178.stdout: "dir-": 62, 2024-05-10T11:44:35.240 INFO:teuthology.orchestra.run.smithi178.stdout: "dn": 24, 2024-05-10T11:44:35.240 INFO:teuthology.orchestra.run.smithi178.stdout: "dn+": 33543, 2024-05-10T11:44:35.240 INFO:teuthology.orchestra.run.smithi178.stdout: "dn-": 33519, 2024-05-10T11:44:35.240 INFO:teuthology.orchestra.run.smithi178.stdout: "cap": 4, 2024-05-10T11:44:35.240 INFO:teuthology.orchestra.run.smithi178.stdout: "cap+": 25651, 2024-05-10T11:44:35.240 INFO:teuthology.orchestra.run.smithi178.stdout: "cap-": 25647, 2024-05-10T11:44:35.240 INFO:teuthology.orchestra.run.smithi178.stdout: "rss": 84920, 2024-05-10T11:44:35.240 INFO:teuthology.orchestra.run.smithi178.stdout: "heap": 223608 2024-05-10T11:44:35.240 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:35.240 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_server": { 2024-05-10T11:44:35.240 INFO:teuthology.orchestra.run.smithi178.stdout: "dispatch_client_request": 821500, 2024-05-10T11:44:35.240 INFO:teuthology.orchestra.run.smithi178.stdout: "dispatch_server_request": 39, 2024-05-10T11:44:35.240 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_request": 726388, 2024-05-10T11:44:35.240 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_session": 1420, 2024-05-10T11:44:35.240 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_peer_request": 24862, 2024-05-10T11:44:35.240 INFO:teuthology.orchestra.run.smithi178.stdout: "req_create_latency": { 2024-05-10T11:44:35.241 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 25288, 2024-05-10T11:44:35.241 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 40.571719033, 2024-05-10T11:44:35.241 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.001604386 2024-05-10T11:44:35.241 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:35.241 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getattr_latency": { 2024-05-10T11:44:35.241 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 601759, 2024-05-10T11:44:35.241 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 246.672296968, 2024-05-10T11:44:35.241 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000409918 2024-05-10T11:44:35.241 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:35.241 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getfilelock_latency": { 2024-05-10T11:44:35.241 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:35.241 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:35.241 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:35.241 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:35.241 INFO:teuthology.orchestra.run.smithi178.stdout: "req_link_latency": { 2024-05-10T11:44:35.241 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:35.242 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:35.242 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:35.242 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:35.242 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookup_latency": { 2024-05-10T11:44:35.242 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 14912, 2024-05-10T11:44:35.242 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 7.456797954, 2024-05-10T11:44:35.242 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000500053 2024-05-10T11:44:35.242 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:35.242 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookuphash_latency": { 2024-05-10T11:44:35.242 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:35.242 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:35.242 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:35.242 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:35.243 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupino_latency": { 2024-05-10T11:44:35.243 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:35.243 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:35.243 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:35.243 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:35.243 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupname_latency": { 2024-05-10T11:44:35.243 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:35.243 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:35.243 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:35.243 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:35.243 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupparent_latency": { 2024-05-10T11:44:35.243 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:35.243 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:35.243 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:35.243 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:35.243 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupsnap_latency": { 2024-05-10T11:44:35.243 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:35.244 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:35.244 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:35.244 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:35.244 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lssnap_latency": { 2024-05-10T11:44:35.244 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:35.244 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:35.245 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:35.245 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:35.245 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mkdir_latency": { 2024-05-10T11:44:35.245 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:35.245 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:35.245 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:35.245 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:35.245 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mknod_latency": { 2024-05-10T11:44:35.245 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:35.245 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:35.245 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:35.245 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:35.245 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mksnap_latency": { 2024-05-10T11:44:35.245 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:35.245 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:35.245 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:35.246 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:35.246 INFO:teuthology.orchestra.run.smithi178.stdout: "req_open_latency": { 2024-05-10T11:44:35.246 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 17147, 2024-05-10T11:44:35.246 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 11.942473532, 2024-05-10T11:44:35.246 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000696475 2024-05-10T11:44:35.246 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:35.246 INFO:teuthology.orchestra.run.smithi178.stdout: "req_readdir_latency": { 2024-05-10T11:44:35.246 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 33889, 2024-05-10T11:44:35.246 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 95.700742958, 2024-05-10T11:44:35.246 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.002823947 2024-05-10T11:44:35.246 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:35.246 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rename_latency": { 2024-05-10T11:44:35.246 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 5212, 2024-05-10T11:44:35.246 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 10.542168227, 2024-05-10T11:44:35.246 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.002022672 2024-05-10T11:44:35.246 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:35.247 INFO:teuthology.orchestra.run.smithi178.stdout: "req_renamesnap_latency": { 2024-05-10T11:44:35.247 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:35.247 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:35.247 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:35.247 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:35.247 INFO:teuthology.orchestra.run.smithi178.stdout: "req_snapdiff_latency": { 2024-05-10T11:44:35.247 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:35.247 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:35.247 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:35.247 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:35.247 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmdir_latency": { 2024-05-10T11:44:35.247 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 30, 2024-05-10T11:44:35.247 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.129684759, 2024-05-10T11:44:35.247 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.004322825 2024-05-10T11:44:35.247 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:35.247 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmsnap_latency": { 2024-05-10T11:44:35.248 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:35.248 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:35.248 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:35.248 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:35.248 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmxattr_latency": { 2024-05-10T11:44:35.248 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:35.248 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:35.248 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:35.248 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:35.248 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setattr_latency": { 2024-05-10T11:44:35.248 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:35.248 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:35.248 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:35.248 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:35.248 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setdirlayout_latency": { 2024-05-10T11:44:35.248 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:35.248 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:35.249 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:35.249 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:35.249 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setfilelock_latency": { 2024-05-10T11:44:35.249 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 456, 2024-05-10T11:44:35.249 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.224180682, 2024-05-10T11:44:35.249 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000491624 2024-05-10T11:44:35.249 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:35.249 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setlayout_latency": { 2024-05-10T11:44:35.249 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:35.249 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:35.249 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:35.249 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:35.249 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setxattr_latency": { 2024-05-10T11:44:35.249 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:35.249 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:35.249 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:35.250 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:35.250 INFO:teuthology.orchestra.run.smithi178.stdout: "req_symlink_latency": { 2024-05-10T11:44:35.250 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:35.250 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:35.250 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:35.250 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:35.250 INFO:teuthology.orchestra.run.smithi178.stdout: "req_unlink_latency": { 2024-05-10T11:44:35.250 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 25514, 2024-05-10T11:44:35.250 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 58.563909964, 2024-05-10T11:44:35.250 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.002295363 2024-05-10T11:44:35.250 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:35.250 INFO:teuthology.orchestra.run.smithi178.stdout: "cap_revoke_eviction": 0, 2024-05-10T11:44:35.250 INFO:teuthology.orchestra.run.smithi178.stdout: "cap_acquisition_throttle": 0, 2024-05-10T11:44:35.250 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getvxattr_latency": { 2024-05-10T11:44:35.250 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:35.251 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:35.251 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:35.251 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-05-10T11:44:35.251 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:35.251 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_sessions": { 2024-05-10T11:44:35.251 INFO:teuthology.orchestra.run.smithi178.stdout: "session_count": 1, 2024-05-10T11:44:35.251 INFO:teuthology.orchestra.run.smithi178.stdout: "session_add": 2, 2024-05-10T11:44:35.251 INFO:teuthology.orchestra.run.smithi178.stdout: "session_remove": 1, 2024-05-10T11:44:35.251 INFO:teuthology.orchestra.run.smithi178.stdout: "sessions_open": 1, 2024-05-10T11:44:35.251 INFO:teuthology.orchestra.run.smithi178.stdout: "sessions_stale": 0, 2024-05-10T11:44:35.251 INFO:teuthology.orchestra.run.smithi178.stdout: "total_load": 78711, 2024-05-10T11:44:35.251 INFO:teuthology.orchestra.run.smithi178.stdout: "average_load": 78711, 2024-05-10T11:44:35.251 INFO:teuthology.orchestra.run.smithi178.stdout: "avg_session_uptime": 1567, 2024-05-10T11:44:35.251 INFO:teuthology.orchestra.run.smithi178.stdout: "mdthresh_evicted": 0 2024-05-10T11:44:35.251 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:35.251 INFO:teuthology.orchestra.run.smithi178.stdout: "mempool": { 2024-05-10T11:44:35.251 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_bytes": 120, 2024-05-10T11:44:35.252 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_items": 120, 2024-05-10T11:44:35.252 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T11:44:35.252 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_items": 0, 2024-05-10T11:44:35.252 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T11:44:35.252 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_items": 0, 2024-05-10T11:44:35.252 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T11:44:35.252 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T11:44:35.252 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T11:44:35.252 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T11:44:35.252 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T11:44:35.252 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_items": 0, 2024-05-10T11:44:35.252 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T11:44:35.252 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T11:44:35.252 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_bytes": 0, 2024-05-10T11:44:35.252 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_items": 0, 2024-05-10T11:44:35.253 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_bytes": 0, 2024-05-10T11:44:35.253 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_items": 0, 2024-05-10T11:44:35.253 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T11:44:35.253 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T11:44:35.253 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T11:44:35.253 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T11:44:35.253 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T11:44:35.253 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_items": 0, 2024-05-10T11:44:35.253 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_bytes": 0, 2024-05-10T11:44:35.253 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_items": 0, 2024-05-10T11:44:35.253 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T11:44:35.253 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T11:44:35.253 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_bytes": 0, 2024-05-10T11:44:35.253 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_items": 0, 2024-05-10T11:44:35.253 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_bytes": 0, 2024-05-10T11:44:35.254 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_items": 0, 2024-05-10T11:44:35.254 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T11:44:35.254 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_items": 0, 2024-05-10T11:44:35.254 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T11:44:35.254 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_items": 0, 2024-05-10T11:44:35.254 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_bytes": 299792, 2024-05-10T11:44:35.254 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_items": 113, 2024-05-10T11:44:35.254 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_bytes": 0, 2024-05-10T11:44:35.254 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_items": 0, 2024-05-10T11:44:35.254 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_bytes": 0, 2024-05-10T11:44:35.254 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_items": 0, 2024-05-10T11:44:35.254 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_bytes": 0, 2024-05-10T11:44:35.254 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_items": 0, 2024-05-10T11:44:35.254 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_bytes": 0, 2024-05-10T11:44:35.255 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_items": 0, 2024-05-10T11:44:35.255 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_bytes": 8696, 2024-05-10T11:44:35.255 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_items": 166, 2024-05-10T11:44:35.255 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T11:44:35.255 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_items": 0, 2024-05-10T11:44:35.255 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_bytes": 0, 2024-05-10T11:44:35.255 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_items": 0, 2024-05-10T11:44:35.255 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_bytes": 118997, 2024-05-10T11:44:35.255 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_items": 524, 2024-05-10T11:44:35.255 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_bytes": 0, 2024-05-10T11:44:35.255 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_items": 0, 2024-05-10T11:44:35.255 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_bytes": 0, 2024-05-10T11:44:35.255 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_items": 0 2024-05-10T11:44:35.255 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:35.255 INFO:teuthology.orchestra.run.smithi178.stdout: "objecter": { 2024-05-10T11:44:35.256 INFO:teuthology.orchestra.run.smithi178.stdout: "op_active": 0, 2024-05-10T11:44:35.256 INFO:teuthology.orchestra.run.smithi178.stdout: "op_laggy": 0, 2024-05-10T11:44:35.256 INFO:teuthology.orchestra.run.smithi178.stdout: "op_send": 119935, 2024-05-10T11:44:35.256 INFO:teuthology.orchestra.run.smithi178.stdout: "op_send_bytes": 297345771, 2024-05-10T11:44:35.256 INFO:teuthology.orchestra.run.smithi178.stdout: "op_resend": 29, 2024-05-10T11:44:35.256 INFO:teuthology.orchestra.run.smithi178.stdout: "op_reply": 119935, 2024-05-10T11:44:35.256 INFO:teuthology.orchestra.run.smithi178.stdout: "op_latency": { 2024-05-10T11:44:35.256 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 119935, 2024-05-10T11:44:35.256 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 493.537725219, 2024-05-10T11:44:35.256 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.004115043 2024-05-10T11:44:35.256 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:35.256 INFO:teuthology.orchestra.run.smithi178.stdout: "op_inflight": 0, 2024-05-10T11:44:35.256 INFO:teuthology.orchestra.run.smithi178.stdout: "oplen_avg": { 2024-05-10T11:44:35.256 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 119935, 2024-05-10T11:44:35.257 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 122798 2024-05-10T11:44:35.257 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:35.257 INFO:teuthology.orchestra.run.smithi178.stdout: "op": 119935, 2024-05-10T11:44:35.257 INFO:teuthology.orchestra.run.smithi178.stdout: "op_r": 6661, 2024-05-10T11:44:35.257 INFO:teuthology.orchestra.run.smithi178.stdout: "op_w": 113274, 2024-05-10T11:44:35.257 INFO:teuthology.orchestra.run.smithi178.stdout: "op_rmw": 0, 2024-05-10T11:44:35.257 INFO:teuthology.orchestra.run.smithi178.stdout: "op_pg": 0, 2024-05-10T11:44:35.257 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_stat": 5, 2024-05-10T11:44:35.257 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_create": 31, 2024-05-10T11:44:35.257 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_read": 2679, 2024-05-10T11:44:35.257 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_write": 59177, 2024-05-10T11:44:35.257 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_writefull": 1451, 2024-05-10T11:44:35.257 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_writesame": 0, 2024-05-10T11:44:35.257 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_append": 0, 2024-05-10T11:44:35.257 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_zero": 0, 2024-05-10T11:44:35.258 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_truncate": 0, 2024-05-10T11:44:35.258 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_delete": 51166, 2024-05-10T11:44:35.258 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_mapext": 0, 2024-05-10T11:44:35.258 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_sparse_read": 0, 2024-05-10T11:44:35.258 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_clonerange": 0, 2024-05-10T11:44:35.258 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_getxattr": 3982, 2024-05-10T11:44:35.258 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_setxattr": 57, 2024-05-10T11:44:35.258 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_cmpxattr": 0, 2024-05-10T11:44:35.258 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_rmxattr": 0, 2024-05-10T11:44:35.258 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_resetxattrs": 0, 2024-05-10T11:44:35.258 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_call": 0, 2024-05-10T11:44:35.258 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_watch": 0, 2024-05-10T11:44:35.258 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_notify": 0, 2024-05-10T11:44:35.259 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_src_cmpxattr": 0, 2024-05-10T11:44:35.259 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_pgls": 0, 2024-05-10T11:44:35.259 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_pgls_filter": 0, 2024-05-10T11:44:35.259 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_other": 0, 2024-05-10T11:44:35.259 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_active": 0, 2024-05-10T11:44:35.259 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_send": 0, 2024-05-10T11:44:35.259 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_resend": 0, 2024-05-10T11:44:35.259 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_ping": 0, 2024-05-10T11:44:35.259 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_active": 0, 2024-05-10T11:44:35.259 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_send": 0, 2024-05-10T11:44:35.259 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_resend": 0, 2024-05-10T11:44:35.259 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_active": 0, 2024-05-10T11:44:35.259 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_send": 0, 2024-05-10T11:44:35.259 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_resend": 0, 2024-05-10T11:44:35.259 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_active": 0, 2024-05-10T11:44:35.260 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_send": 0, 2024-05-10T11:44:35.260 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_resend": 0, 2024-05-10T11:44:35.260 INFO:teuthology.orchestra.run.smithi178.stdout: "command_active": 0, 2024-05-10T11:44:35.260 INFO:teuthology.orchestra.run.smithi178.stdout: "command_send": 0, 2024-05-10T11:44:35.260 INFO:teuthology.orchestra.run.smithi178.stdout: "command_resend": 0, 2024-05-10T11:44:35.260 INFO:teuthology.orchestra.run.smithi178.stdout: "map_epoch": 90, 2024-05-10T11:44:35.260 INFO:teuthology.orchestra.run.smithi178.stdout: "map_full": 0, 2024-05-10T11:44:35.260 INFO:teuthology.orchestra.run.smithi178.stdout: "map_inc": 1, 2024-05-10T11:44:35.260 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_sessions": 12, 2024-05-10T11:44:35.260 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_session_open": 12, 2024-05-10T11:44:35.260 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_session_close": 0, 2024-05-10T11:44:35.260 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_laggy": 0, 2024-05-10T11:44:35.260 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_wr": 2856, 2024-05-10T11:44:35.260 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_rd": 2, 2024-05-10T11:44:35.260 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_del": 1392 2024-05-10T11:44:35.261 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:35.261 INFO:teuthology.orchestra.run.smithi178.stdout: "oft": { 2024-05-10T11:44:35.261 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_objs": 1, 2024-05-10T11:44:35.261 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_kv_pairs": 0, 2024-05-10T11:44:35.261 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_updates": 0, 2024-05-10T11:44:35.261 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_removes": 0 2024-05-10T11:44:35.261 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:35.261 INFO:teuthology.orchestra.run.smithi178.stdout: "osd-slow-ops": { 2024-05-10T11:44:35.261 INFO:teuthology.orchestra.run.smithi178.stdout: "slow_ops_count": 0 2024-05-10T11:44:35.261 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:35.261 INFO:teuthology.orchestra.run.smithi178.stdout: "purge_queue": { 2024-05-10T11:44:35.261 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_ops": 0, 2024-05-10T11:44:35.261 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_ops_high_water": 16, 2024-05-10T11:44:35.261 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing": 0, 2024-05-10T11:44:35.261 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_high_water": 16, 2024-05-10T11:44:35.262 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executed_ops": 51058, 2024-05-10T11:44:35.262 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executed": 25544, 2024-05-10T11:44:35.262 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_item_in_journal": 0 2024-05-10T11:44:35.262 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:35.262 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T11:44:35.262 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-05-10T11:44:35.262 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 104857600, 2024-05-10T11:44:35.262 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-05-10T11:44:35.262 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 1128865, 2024-05-10T11:44:35.262 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 232234704, 2024-05-10T11:44:35.262 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-05-10T11:44:35.262 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 1128865, 2024-05-10T11:44:35.262 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-05-10T11:44:35.262 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-05-10T11:44:35.262 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 1128865, 2024-05-10T11:44:35.263 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 232234704, 2024-05-10T11:44:35.263 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-05-10T11:44:35.263 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:35.263 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:35.263 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:35.263 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-05-10T11:44:35.263 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:35.263 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-objecter_bytes": { 2024-05-10T11:44:35.263 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-05-10T11:44:35.263 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 104857600, 2024-05-10T11:44:35.263 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-05-10T11:44:35.263 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-05-10T11:44:35.263 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-05-10T11:44:35.263 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-05-10T11:44:35.264 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-05-10T11:44:35.264 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 119935, 2024-05-10T11:44:35.264 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 300129811, 2024-05-10T11:44:35.264 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 119935, 2024-05-10T11:44:35.264 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 300129811, 2024-05-10T11:44:35.264 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-05-10T11:44:35.264 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:35.264 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:35.264 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:35.264 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-05-10T11:44:35.264 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:35.264 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-objecter_ops": { 2024-05-10T11:44:35.264 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-05-10T11:44:35.264 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 1024, 2024-05-10T11:44:35.265 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-05-10T11:44:35.265 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-05-10T11:44:35.265 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-05-10T11:44:35.265 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-05-10T11:44:35.265 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-05-10T11:44:35.265 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 119935, 2024-05-10T11:44:35.265 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 119935, 2024-05-10T11:44:35.265 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 119935, 2024-05-10T11:44:35.265 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 119935, 2024-05-10T11:44:35.265 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-05-10T11:44:35.265 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:35.265 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:35.265 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:35.265 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-05-10T11:44:35.265 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:35.265 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-write_buf_throttle": { 2024-05-10T11:44:35.266 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-05-10T11:44:35.266 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 3758096384, 2024-05-10T11:44:35.266 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-05-10T11:44:35.266 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 25544, 2024-05-10T11:44:35.266 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 2784056, 2024-05-10T11:44:35.266 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-05-10T11:44:35.266 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 25544, 2024-05-10T11:44:35.266 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-05-10T11:44:35.266 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-05-10T11:44:35.266 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 2679, 2024-05-10T11:44:35.266 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 2784056, 2024-05-10T11:44:35.266 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-05-10T11:44:35.266 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:35.266 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:35.267 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:35.267 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-05-10T11:44:35.267 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:35.267 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-write_buf_throttle-0x55d5eb948500": { 2024-05-10T11:44:35.267 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-05-10T11:44:35.267 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 3758096384, 2024-05-10T11:44:35.267 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-05-10T11:44:35.267 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 190453, 2024-05-10T11:44:35.267 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 292786243, 2024-05-10T11:44:35.267 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-05-10T11:44:35.267 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 190453, 2024-05-10T11:44:35.267 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-05-10T11:44:35.267 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-05-10T11:44:35.267 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 56498, 2024-05-10T11:44:35.267 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 292786243, 2024-05-10T11:44:35.267 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-05-10T11:44:35.268 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:35.268 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:35.268 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:35.268 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-05-10T11:44:35.268 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-05-10T11:44:35.268 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-05-10T11:44:36.061 INFO:tasks.check_counter:Daemon mds.b mds.exported=11 2024-05-10T11:44:36.061 INFO:tasks.check_counter:Daemon mds.b mds.imported=12 2024-05-10T11:44:36.062 DEBUG:tasks.check_counter:Getting stats from e 2024-05-10T11:44:36.062 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph daemon mds.e perf dump 2024-05-10T11:44:36.312 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T11:44:36.343 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:44:36 smithi178 ceph-mon[33359]: pgmap v1207: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 853 B/s wr, 6 op/s 2024-05-10T11:44:36.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:44:36 smithi188 ceph-mon[34075]: pgmap v1207: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 853 B/s wr, 6 op/s 2024-05-10T11:44:36.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:44:36 smithi100 ceph-mon[30296]: pgmap v1207: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 853 B/s wr, 6 op/s 2024-05-10T11:44:38.140 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-05-10T11:44:38.140 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T11:44:38.140 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 1697, 2024-05-10T11:44:38.140 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 1697, 2024-05-10T11:44:38.141 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 92729936, 2024-05-10T11:44:38.141 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 538886, 2024-05-10T11:44:38.141 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 9, 2024-05-10T11:44:38.141 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 4, 2024-05-10T11:44:38.141 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.168221660, 2024-05-10T11:44:38.141 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.036928811, 2024-05-10T11:44:38.141 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.083443695, 2024-05-10T11:44:38.141 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.036911784, 2024-05-10T11:44:38.141 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T11:44:38.141 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 1696, 2024-05-10T11:44:38.141 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.045140947, 2024-05-10T11:44:38.141 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000026616 2024-05-10T11:44:38.141 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:38.141 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-05-10T11:44:38.141 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:38.142 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:38.142 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:38.142 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:38.142 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 2912, 2024-05-10T11:44:38.142 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 448 2024-05-10T11:44:38.142 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:38.142 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T11:44:38.142 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 1965, 2024-05-10T11:44:38.142 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 2337, 2024-05-10T11:44:38.142 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 92770198, 2024-05-10T11:44:38.142 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 1517793, 2024-05-10T11:44:38.142 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 8, 2024-05-10T11:44:38.142 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 5, 2024-05-10T11:44:38.142 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.202135223, 2024-05-10T11:44:38.143 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.048997352, 2024-05-10T11:44:38.143 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.095136880, 2024-05-10T11:44:38.143 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.043749021, 2024-05-10T11:44:38.143 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T11:44:38.143 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 2335, 2024-05-10T11:44:38.143 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.063207749, 2024-05-10T11:44:38.143 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000027069 2024-05-10T11:44:38.143 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:38.143 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-05-10T11:44:38.143 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:38.143 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:38.143 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:38.143 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:38.143 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 480, 2024-05-10T11:44:38.144 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 893872 2024-05-10T11:44:38.144 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:38.144 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T11:44:38.144 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 3148, 2024-05-10T11:44:38.144 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 3126, 2024-05-10T11:44:38.144 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 56023988, 2024-05-10T11:44:38.144 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 1140836, 2024-05-10T11:44:38.144 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 5, 2024-05-10T11:44:38.144 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 4, 2024-05-10T11:44:38.144 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.287707283, 2024-05-10T11:44:38.144 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.086412571, 2024-05-10T11:44:38.144 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.104579347, 2024-05-10T11:44:38.144 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.076574794, 2024-05-10T11:44:38.144 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T11:44:38.144 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 3125, 2024-05-10T11:44:38.145 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.098241900, 2024-05-10T11:44:38.145 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000031437 2024-05-10T11:44:38.145 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:38.145 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-05-10T11:44:38.145 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:38.145 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:38.145 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:38.145 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:38.145 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 689072, 2024-05-10T11:44:38.145 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 304496 2024-05-10T11:44:38.145 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:38.145 INFO:teuthology.orchestra.run.smithi178.stdout: "cct": { 2024-05-10T11:44:38.145 INFO:teuthology.orchestra.run.smithi178.stdout: "total_workers": 1, 2024-05-10T11:44:38.145 INFO:teuthology.orchestra.run.smithi178.stdout: "unhealthy_workers": 0 2024-05-10T11:44:38.146 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:38.146 INFO:teuthology.orchestra.run.smithi178.stdout: "finisher-MDSRank": { 2024-05-10T11:44:38.146 INFO:teuthology.orchestra.run.smithi178.stdout: "queue_len": 0, 2024-05-10T11:44:38.146 INFO:teuthology.orchestra.run.smithi178.stdout: "complete_latency": { 2024-05-10T11:44:38.146 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 10758, 2024-05-10T11:44:38.146 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.229527639, 2024-05-10T11:44:38.146 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000021335 2024-05-10T11:44:38.146 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-05-10T11:44:38.146 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:38.146 INFO:teuthology.orchestra.run.smithi178.stdout: "finisher-PurgeQueue": { 2024-05-10T11:44:38.146 INFO:teuthology.orchestra.run.smithi178.stdout: "queue_len": 0, 2024-05-10T11:44:38.146 INFO:teuthology.orchestra.run.smithi178.stdout: "complete_latency": { 2024-05-10T11:44:38.146 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:38.147 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:38.147 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:38.147 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-05-10T11:44:38.147 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:38.147 INFO:teuthology.orchestra.run.smithi178.stdout: "mds": { 2024-05-10T11:44:38.147 INFO:teuthology.orchestra.run.smithi178.stdout: "request": 0, 2024-05-10T11:44:38.147 INFO:teuthology.orchestra.run.smithi178.stdout: "reply": 0, 2024-05-10T11:44:38.147 INFO:teuthology.orchestra.run.smithi178.stdout: "reply_latency": { 2024-05-10T11:44:38.147 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:38.147 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:38.147 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:38.147 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:38.147 INFO:teuthology.orchestra.run.smithi178.stdout: "slow_reply": 0, 2024-05-10T11:44:38.147 INFO:teuthology.orchestra.run.smithi178.stdout: "forward": 0, 2024-05-10T11:44:38.147 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_fetch_complete": 0, 2024-05-10T11:44:38.148 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_fetch_keys": 0, 2024-05-10T11:44:38.148 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_commit": 0, 2024-05-10T11:44:38.148 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_split": 0, 2024-05-10T11:44:38.148 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_merge": 0, 2024-05-10T11:44:38.148 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes": 22000, 2024-05-10T11:44:38.148 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_top": 4657, 2024-05-10T11:44:38.148 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_bottom": 17343, 2024-05-10T11:44:38.148 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_pin_tail": 0, 2024-05-10T11:44:38.148 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_pinned": 15347, 2024-05-10T11:44:38.148 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_expired": 0, 2024-05-10T11:44:38.148 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_with_caps": 0, 2024-05-10T11:44:38.148 INFO:teuthology.orchestra.run.smithi178.stdout: "caps": 0, 2024-05-10T11:44:38.148 INFO:teuthology.orchestra.run.smithi178.stdout: "subtrees": 1, 2024-05-10T11:44:38.148 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse": 0, 2024-05-10T11:44:38.148 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_hit": 0, 2024-05-10T11:44:38.149 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_forward": 0, 2024-05-10T11:44:38.149 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_discover": 0, 2024-05-10T11:44:38.149 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_dir_fetch": 0, 2024-05-10T11:44:38.149 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_remote_ino": 0, 2024-05-10T11:44:38.149 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_lock": 0, 2024-05-10T11:44:38.149 INFO:teuthology.orchestra.run.smithi178.stdout: "load_cent": 0, 2024-05-10T11:44:38.149 INFO:teuthology.orchestra.run.smithi178.stdout: "q": 0, 2024-05-10T11:44:38.149 INFO:teuthology.orchestra.run.smithi178.stdout: "exported": 0, 2024-05-10T11:44:38.149 INFO:teuthology.orchestra.run.smithi178.stdout: "exported_inodes": 0, 2024-05-10T11:44:38.149 INFO:teuthology.orchestra.run.smithi178.stdout: "imported": 0, 2024-05-10T11:44:38.149 INFO:teuthology.orchestra.run.smithi178.stdout: "imported_inodes": 0, 2024-05-10T11:44:38.149 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_dir_fetch": 0, 2024-05-10T11:44:38.149 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_backtrace_fetch": 0, 2024-05-10T11:44:38.149 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_peer_discover": 0, 2024-05-10T11:44:38.149 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rfiles": 0, 2024-05-10T11:44:38.150 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rbytes": 0, 2024-05-10T11:44:38.150 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rsnaps": 0, 2024-05-10T11:44:38.150 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_backtrace_fetch": 0, 2024-05-10T11:44:38.150 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_set_tag": 0, 2024-05-10T11:44:38.150 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_backtrace_repaired": 0, 2024-05-10T11:44:38.150 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_inotable_repaired": 0, 2024-05-10T11:44:38.150 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dir_inodes": 0, 2024-05-10T11:44:38.150 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dir_base_inodes": 0, 2024-05-10T11:44:38.150 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dirfrag_rstats": 0, 2024-05-10T11:44:38.150 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_file_inodes": 0, 2024-05-10T11:44:38.150 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_inode_file_caps": 0, 2024-05-10T11:44:38.150 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_revoke": 0, 2024-05-10T11:44:38.150 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_grant": 0, 2024-05-10T11:44:38.150 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_trunc": 0, 2024-05-10T11:44:38.150 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-10T11:44:38.151 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-10T11:44:38.151 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_caps": 0, 2024-05-10T11:44:38.151 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_caps_dirty": 0, 2024-05-10T11:44:38.151 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_cap_release": 0, 2024-05-10T11:44:38.151 INFO:teuthology.orchestra.run.smithi178.stdout: "process_request_cap_release": 0 2024-05-10T11:44:38.151 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:38.151 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_cache": { 2024-05-10T11:44:38.151 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_update": 0, 2024-05-10T11:44:38.151 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_update_receipt": 0, 2024-05-10T11:44:38.151 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_try_discover": 0, 2024-05-10T11:44:38.151 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_send_discover": 0, 2024-05-10T11:44:38.151 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_handle_discover": 0, 2024-05-10T11:44:38.151 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays": 0, 2024-05-10T11:44:38.151 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays_delayed": 0, 2024-05-10T11:44:38.151 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays_enqueuing": 0, 2024-05-10T11:44:38.152 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_created": 0, 2024-05-10T11:44:38.152 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_enqueued": 0, 2024-05-10T11:44:38.152 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_reintegrated": 0, 2024-05-10T11:44:38.152 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_migrated": 0, 2024-05-10T11:44:38.152 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_processing": 0, 2024-05-10T11:44:38.152 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_enqueued": 0, 2024-05-10T11:44:38.152 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_prioritized": 0, 2024-05-10T11:44:38.152 INFO:teuthology.orchestra.run.smithi178.stdout: "recovery_started": 0, 2024-05-10T11:44:38.152 INFO:teuthology.orchestra.run.smithi178.stdout: "recovery_completed": 0, 2024-05-10T11:44:38.152 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_quiesce_path": 0, 2024-05-10T11:44:38.152 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_quiesce_inode": 0, 2024-05-10T11:44:38.152 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_enqueue_scrub": 0, 2024-05-10T11:44:38.152 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_exportdir": 0, 2024-05-10T11:44:38.152 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_flush": 0, 2024-05-10T11:44:38.152 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_fragmentdir": 0, 2024-05-10T11:44:38.153 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_fragstats": 0, 2024-05-10T11:44:38.153 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_inodestats": 0 2024-05-10T11:44:38.153 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:38.153 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_log": { 2024-05-10T11:44:38.153 INFO:teuthology.orchestra.run.smithi178.stdout: "evlrg": 0, 2024-05-10T11:44:38.153 INFO:teuthology.orchestra.run.smithi178.stdout: "evadd": 0, 2024-05-10T11:44:38.153 INFO:teuthology.orchestra.run.smithi178.stdout: "evex": 0, 2024-05-10T11:44:38.153 INFO:teuthology.orchestra.run.smithi178.stdout: "evtrm": 0, 2024-05-10T11:44:38.153 INFO:teuthology.orchestra.run.smithi178.stdout: "ev": 156483, 2024-05-10T11:44:38.153 INFO:teuthology.orchestra.run.smithi178.stdout: "evexg": 0, 2024-05-10T11:44:38.153 INFO:teuthology.orchestra.run.smithi178.stdout: "evexd": 0, 2024-05-10T11:44:38.153 INFO:teuthology.orchestra.run.smithi178.stdout: "segadd": 0, 2024-05-10T11:44:38.153 INFO:teuthology.orchestra.run.smithi178.stdout: "segex": 0, 2024-05-10T11:44:38.153 INFO:teuthology.orchestra.run.smithi178.stdout: "segtrm": 0, 2024-05-10T11:44:38.153 INFO:teuthology.orchestra.run.smithi178.stdout: "seg": 143, 2024-05-10T11:44:38.153 INFO:teuthology.orchestra.run.smithi178.stdout: "segmjr": 14, 2024-05-10T11:44:38.154 INFO:teuthology.orchestra.run.smithi178.stdout: "segexg": 0, 2024-05-10T11:44:38.154 INFO:teuthology.orchestra.run.smithi178.stdout: "segexd": 0, 2024-05-10T11:44:38.154 INFO:teuthology.orchestra.run.smithi178.stdout: "expos": 74596632, 2024-05-10T11:44:38.154 INFO:teuthology.orchestra.run.smithi178.stdout: "wrpos": 243289507, 2024-05-10T11:44:38.154 INFO:teuthology.orchestra.run.smithi178.stdout: "rdpos": 243288908, 2024-05-10T11:44:38.154 INFO:teuthology.orchestra.run.smithi178.stdout: "jlat": { 2024-05-10T11:44:38.154 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:38.154 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:38.154 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:38.154 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:38.154 INFO:teuthology.orchestra.run.smithi178.stdout: "replayed": 156483 2024-05-10T11:44:38.154 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:38.154 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_mem": { 2024-05-10T11:44:38.154 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 13, 2024-05-10T11:44:38.155 INFO:teuthology.orchestra.run.smithi178.stdout: "ino+": 22044, 2024-05-10T11:44:38.155 INFO:teuthology.orchestra.run.smithi178.stdout: "ino-": 22031, 2024-05-10T11:44:38.155 INFO:teuthology.orchestra.run.smithi178.stdout: "dir": 11, 2024-05-10T11:44:38.155 INFO:teuthology.orchestra.run.smithi178.stdout: "dir+": 82, 2024-05-10T11:44:38.155 INFO:teuthology.orchestra.run.smithi178.stdout: "dir-": 71, 2024-05-10T11:44:38.155 INFO:teuthology.orchestra.run.smithi178.stdout: "dn": 22000, 2024-05-10T11:44:38.155 INFO:teuthology.orchestra.run.smithi178.stdout: "dn+": 22736, 2024-05-10T11:44:38.155 INFO:teuthology.orchestra.run.smithi178.stdout: "dn-": 736, 2024-05-10T11:44:38.155 INFO:teuthology.orchestra.run.smithi178.stdout: "cap": 0, 2024-05-10T11:44:38.155 INFO:teuthology.orchestra.run.smithi178.stdout: "cap+": 0, 2024-05-10T11:44:38.155 INFO:teuthology.orchestra.run.smithi178.stdout: "cap-": 0, 2024-05-10T11:44:38.155 INFO:teuthology.orchestra.run.smithi178.stdout: "rss": 73656, 2024-05-10T11:44:38.155 INFO:teuthology.orchestra.run.smithi178.stdout: "heap": 215416 2024-05-10T11:44:38.155 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:38.155 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_server": { 2024-05-10T11:44:38.156 INFO:teuthology.orchestra.run.smithi178.stdout: "dispatch_client_request": 0, 2024-05-10T11:44:38.156 INFO:teuthology.orchestra.run.smithi178.stdout: "dispatch_server_request": 0, 2024-05-10T11:44:38.156 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_request": 0, 2024-05-10T11:44:38.156 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_session": 0, 2024-05-10T11:44:38.156 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_peer_request": 0, 2024-05-10T11:44:38.156 INFO:teuthology.orchestra.run.smithi178.stdout: "req_create_latency": { 2024-05-10T11:44:38.156 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:38.156 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:38.156 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:38.156 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:38.156 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getattr_latency": { 2024-05-10T11:44:38.156 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:38.156 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:38.156 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:38.156 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:38.157 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getfilelock_latency": { 2024-05-10T11:44:38.157 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:38.157 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:38.157 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:38.157 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:38.157 INFO:teuthology.orchestra.run.smithi178.stdout: "req_link_latency": { 2024-05-10T11:44:38.157 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:38.157 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:38.157 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:38.157 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:38.157 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookup_latency": { 2024-05-10T11:44:38.157 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:38.157 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:38.157 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:38.157 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:38.158 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookuphash_latency": { 2024-05-10T11:44:38.158 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:38.158 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:38.158 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:38.158 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:38.158 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupino_latency": { 2024-05-10T11:44:38.158 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:38.158 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:38.158 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:38.158 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:38.158 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupname_latency": { 2024-05-10T11:44:38.158 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:38.158 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:38.158 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:38.158 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:38.159 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupparent_latency": { 2024-05-10T11:44:38.159 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:38.159 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:38.159 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:38.159 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:38.159 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupsnap_latency": { 2024-05-10T11:44:38.159 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:38.159 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:38.159 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:38.159 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:38.159 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lssnap_latency": { 2024-05-10T11:44:38.159 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:38.159 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:38.159 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:38.159 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:38.160 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mkdir_latency": { 2024-05-10T11:44:38.160 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:38.160 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:38.160 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:38.161 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:38.161 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mknod_latency": { 2024-05-10T11:44:38.161 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:38.161 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:38.161 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:38.161 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:38.161 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mksnap_latency": { 2024-05-10T11:44:38.161 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:38.161 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:38.161 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:38.161 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:38.161 INFO:teuthology.orchestra.run.smithi178.stdout: "req_open_latency": { 2024-05-10T11:44:38.161 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:38.161 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:38.161 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:38.162 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:38.162 INFO:teuthology.orchestra.run.smithi178.stdout: "req_readdir_latency": { 2024-05-10T11:44:38.162 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:38.162 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:38.162 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:38.162 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:38.162 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rename_latency": { 2024-05-10T11:44:38.162 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:38.162 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:38.162 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:38.162 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:38.162 INFO:teuthology.orchestra.run.smithi178.stdout: "req_renamesnap_latency": { 2024-05-10T11:44:38.162 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:38.162 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:38.162 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:38.163 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:38.163 INFO:teuthology.orchestra.run.smithi178.stdout: "req_snapdiff_latency": { 2024-05-10T11:44:38.163 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:38.163 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:38.163 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:38.163 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:38.163 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmdir_latency": { 2024-05-10T11:44:38.163 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:38.163 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:38.163 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:38.163 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:38.163 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmsnap_latency": { 2024-05-10T11:44:38.163 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:38.163 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:38.164 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:38.164 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:38.164 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmxattr_latency": { 2024-05-10T11:44:38.164 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:38.164 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:38.164 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:38.164 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:38.164 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setattr_latency": { 2024-05-10T11:44:38.164 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:38.164 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:38.164 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:38.164 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:38.164 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setdirlayout_latency": { 2024-05-10T11:44:38.164 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:38.164 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:38.165 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:38.165 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:38.165 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setfilelock_latency": { 2024-05-10T11:44:38.165 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:38.165 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:38.165 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:38.165 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:38.165 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setlayout_latency": { 2024-05-10T11:44:38.165 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:38.165 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:38.165 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:38.165 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:38.165 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setxattr_latency": { 2024-05-10T11:44:38.165 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:38.165 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:38.166 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:38.166 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:38.166 INFO:teuthology.orchestra.run.smithi178.stdout: "req_symlink_latency": { 2024-05-10T11:44:38.166 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:38.166 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:38.166 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:38.166 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:38.166 INFO:teuthology.orchestra.run.smithi178.stdout: "req_unlink_latency": { 2024-05-10T11:44:38.166 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:38.166 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:38.166 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:38.166 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:38.166 INFO:teuthology.orchestra.run.smithi178.stdout: "cap_revoke_eviction": 0, 2024-05-10T11:44:38.167 INFO:teuthology.orchestra.run.smithi178.stdout: "cap_acquisition_throttle": 0, 2024-05-10T11:44:38.167 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getvxattr_latency": { 2024-05-10T11:44:38.167 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:38.167 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:38.167 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:38.167 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-05-10T11:44:38.167 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:38.167 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_sessions": { 2024-05-10T11:44:38.167 INFO:teuthology.orchestra.run.smithi178.stdout: "session_count": 1, 2024-05-10T11:44:38.167 INFO:teuthology.orchestra.run.smithi178.stdout: "session_add": 2, 2024-05-10T11:44:38.167 INFO:teuthology.orchestra.run.smithi178.stdout: "session_remove": 1, 2024-05-10T11:44:38.167 INFO:teuthology.orchestra.run.smithi178.stdout: "sessions_open": 1, 2024-05-10T11:44:38.167 INFO:teuthology.orchestra.run.smithi178.stdout: "sessions_stale": 0, 2024-05-10T11:44:38.167 INFO:teuthology.orchestra.run.smithi178.stdout: "total_load": 0, 2024-05-10T11:44:38.167 INFO:teuthology.orchestra.run.smithi178.stdout: "average_load": 0, 2024-05-10T11:44:38.168 INFO:teuthology.orchestra.run.smithi178.stdout: "avg_session_uptime": 2822, 2024-05-10T11:44:38.168 INFO:teuthology.orchestra.run.smithi178.stdout: "mdthresh_evicted": 0 2024-05-10T11:44:38.168 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:38.168 INFO:teuthology.orchestra.run.smithi178.stdout: "mempool": { 2024-05-10T11:44:38.168 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_bytes": 0, 2024-05-10T11:44:38.168 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_items": 0, 2024-05-10T11:44:38.168 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T11:44:38.168 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_items": 0, 2024-05-10T11:44:38.168 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T11:44:38.168 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_items": 0, 2024-05-10T11:44:38.168 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T11:44:38.168 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T11:44:38.168 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T11:44:38.168 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T11:44:38.169 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T11:44:38.169 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_items": 0, 2024-05-10T11:44:38.169 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T11:44:38.169 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T11:44:38.169 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_bytes": 0, 2024-05-10T11:44:38.169 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_items": 0, 2024-05-10T11:44:38.169 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_bytes": 0, 2024-05-10T11:44:38.169 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_items": 0, 2024-05-10T11:44:38.169 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T11:44:38.169 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T11:44:38.169 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T11:44:38.169 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T11:44:38.169 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T11:44:38.169 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_items": 0, 2024-05-10T11:44:38.169 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_bytes": 0, 2024-05-10T11:44:38.170 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_items": 0, 2024-05-10T11:44:38.170 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T11:44:38.170 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T11:44:38.170 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_bytes": 0, 2024-05-10T11:44:38.170 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_items": 0, 2024-05-10T11:44:38.170 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_bytes": 0, 2024-05-10T11:44:38.170 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_items": 0, 2024-05-10T11:44:38.170 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T11:44:38.170 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_items": 0, 2024-05-10T11:44:38.170 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T11:44:38.170 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_items": 0, 2024-05-10T11:44:38.170 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_bytes": 234471, 2024-05-10T11:44:38.170 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_items": 81, 2024-05-10T11:44:38.170 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_bytes": 0, 2024-05-10T11:44:38.170 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_items": 0, 2024-05-10T11:44:38.171 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_bytes": 0, 2024-05-10T11:44:38.171 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_items": 0, 2024-05-10T11:44:38.171 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_bytes": 0, 2024-05-10T11:44:38.171 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_items": 0, 2024-05-10T11:44:38.171 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_bytes": 0, 2024-05-10T11:44:38.171 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_items": 0, 2024-05-10T11:44:38.171 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_bytes": 8696, 2024-05-10T11:44:38.171 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_items": 166, 2024-05-10T11:44:38.171 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T11:44:38.171 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_items": 0, 2024-05-10T11:44:38.171 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_bytes": 0, 2024-05-10T11:44:38.171 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_items": 0, 2024-05-10T11:44:38.171 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_bytes": 15531592, 2024-05-10T11:44:38.171 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_items": 88128, 2024-05-10T11:44:38.171 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_bytes": 0, 2024-05-10T11:44:38.172 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_items": 0, 2024-05-10T11:44:38.172 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_bytes": 0, 2024-05-10T11:44:38.172 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_items": 0 2024-05-10T11:44:38.172 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:38.172 INFO:teuthology.orchestra.run.smithi178.stdout: "objecter": { 2024-05-10T11:44:38.172 INFO:teuthology.orchestra.run.smithi178.stdout: "op_active": 0, 2024-05-10T11:44:38.172 INFO:teuthology.orchestra.run.smithi178.stdout: "op_laggy": 0, 2024-05-10T11:44:38.172 INFO:teuthology.orchestra.run.smithi178.stdout: "op_send": 6285, 2024-05-10T11:44:38.172 INFO:teuthology.orchestra.run.smithi178.stdout: "op_send_bytes": 21, 2024-05-10T11:44:38.172 INFO:teuthology.orchestra.run.smithi178.stdout: "op_resend": 3, 2024-05-10T11:44:38.172 INFO:teuthology.orchestra.run.smithi178.stdout: "op_reply": 6285, 2024-05-10T11:44:38.172 INFO:teuthology.orchestra.run.smithi178.stdout: "op_latency": { 2024-05-10T11:44:38.172 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 6285, 2024-05-10T11:44:38.172 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 5.045915180, 2024-05-10T11:44:38.172 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000802850 2024-05-10T11:44:38.173 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:38.173 INFO:teuthology.orchestra.run.smithi178.stdout: "op_inflight": 0, 2024-05-10T11:44:38.173 INFO:teuthology.orchestra.run.smithi178.stdout: "oplen_avg": { 2024-05-10T11:44:38.173 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 6285, 2024-05-10T11:44:38.173 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 6286 2024-05-10T11:44:38.173 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:38.173 INFO:teuthology.orchestra.run.smithi178.stdout: "op": 6285, 2024-05-10T11:44:38.173 INFO:teuthology.orchestra.run.smithi178.stdout: "op_r": 6285, 2024-05-10T11:44:38.173 INFO:teuthology.orchestra.run.smithi178.stdout: "op_w": 0, 2024-05-10T11:44:38.173 INFO:teuthology.orchestra.run.smithi178.stdout: "op_rmw": 0, 2024-05-10T11:44:38.173 INFO:teuthology.orchestra.run.smithi178.stdout: "op_pg": 0, 2024-05-10T11:44:38.173 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_stat": 3374, 2024-05-10T11:44:38.173 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_create": 0, 2024-05-10T11:44:38.173 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_read": 2909, 2024-05-10T11:44:38.173 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_write": 0, 2024-05-10T11:44:38.174 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_writefull": 0, 2024-05-10T11:44:38.174 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_writesame": 0, 2024-05-10T11:44:38.174 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_append": 0, 2024-05-10T11:44:38.174 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_zero": 0, 2024-05-10T11:44:38.174 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_truncate": 0, 2024-05-10T11:44:38.174 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_delete": 0, 2024-05-10T11:44:38.174 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_mapext": 0, 2024-05-10T11:44:38.174 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_sparse_read": 0, 2024-05-10T11:44:38.174 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_clonerange": 0, 2024-05-10T11:44:38.174 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_getxattr": 1, 2024-05-10T11:44:38.174 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_setxattr": 0, 2024-05-10T11:44:38.174 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_cmpxattr": 0, 2024-05-10T11:44:38.174 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_rmxattr": 0, 2024-05-10T11:44:38.174 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_resetxattrs": 0, 2024-05-10T11:44:38.175 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_call": 0, 2024-05-10T11:44:38.175 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_watch": 0, 2024-05-10T11:44:38.175 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_notify": 0, 2024-05-10T11:44:38.175 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_src_cmpxattr": 0, 2024-05-10T11:44:38.175 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_pgls": 0, 2024-05-10T11:44:38.175 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_pgls_filter": 0, 2024-05-10T11:44:38.175 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_other": 0, 2024-05-10T11:44:38.175 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_active": 0, 2024-05-10T11:44:38.175 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_send": 0, 2024-05-10T11:44:38.175 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_resend": 0, 2024-05-10T11:44:38.175 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_ping": 0, 2024-05-10T11:44:38.175 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_active": 0, 2024-05-10T11:44:38.175 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_send": 0, 2024-05-10T11:44:38.175 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_resend": 0, 2024-05-10T11:44:38.175 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_active": 0, 2024-05-10T11:44:38.176 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_send": 0, 2024-05-10T11:44:38.176 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_resend": 0, 2024-05-10T11:44:38.176 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_active": 0, 2024-05-10T11:44:38.176 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_send": 0, 2024-05-10T11:44:38.176 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_resend": 0, 2024-05-10T11:44:38.176 INFO:teuthology.orchestra.run.smithi178.stdout: "command_active": 0, 2024-05-10T11:44:38.176 INFO:teuthology.orchestra.run.smithi178.stdout: "command_send": 0, 2024-05-10T11:44:38.176 INFO:teuthology.orchestra.run.smithi178.stdout: "command_resend": 0, 2024-05-10T11:44:38.176 INFO:teuthology.orchestra.run.smithi178.stdout: "map_epoch": 90, 2024-05-10T11:44:38.176 INFO:teuthology.orchestra.run.smithi178.stdout: "map_full": 0, 2024-05-10T11:44:38.176 INFO:teuthology.orchestra.run.smithi178.stdout: "map_inc": 1, 2024-05-10T11:44:38.176 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_sessions": 10, 2024-05-10T11:44:38.176 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_session_open": 11, 2024-05-10T11:44:38.176 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_session_close": 1, 2024-05-10T11:44:38.177 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_laggy": 0, 2024-05-10T11:44:38.177 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_wr": 0, 2024-05-10T11:44:38.177 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_rd": 2, 2024-05-10T11:44:38.177 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_del": 0 2024-05-10T11:44:38.177 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:38.177 INFO:teuthology.orchestra.run.smithi178.stdout: "oft": { 2024-05-10T11:44:38.177 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_objs": 0, 2024-05-10T11:44:38.177 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_kv_pairs": 0, 2024-05-10T11:44:38.177 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_updates": 0, 2024-05-10T11:44:38.177 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_removes": 0 2024-05-10T11:44:38.177 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:38.177 INFO:teuthology.orchestra.run.smithi178.stdout: "osd-slow-ops": { 2024-05-10T11:44:38.177 INFO:teuthology.orchestra.run.smithi178.stdout: "slow_ops_count": 0 2024-05-10T11:44:38.177 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:38.177 INFO:teuthology.orchestra.run.smithi178.stdout: "purge_queue": { 2024-05-10T11:44:38.178 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_ops": 0, 2024-05-10T11:44:38.178 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_ops_high_water": 0, 2024-05-10T11:44:38.178 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing": 0, 2024-05-10T11:44:38.178 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_high_water": 0, 2024-05-10T11:44:38.178 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executed_ops": 0, 2024-05-10T11:44:38.178 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executed": 0, 2024-05-10T11:44:38.178 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_item_in_journal": 0 2024-05-10T11:44:38.178 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:38.178 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T11:44:38.178 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-05-10T11:44:38.178 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 104857600, 2024-05-10T11:44:38.178 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-05-10T11:44:38.178 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 6806, 2024-05-10T11:44:38.178 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 240881960, 2024-05-10T11:44:38.178 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-05-10T11:44:38.179 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 6806, 2024-05-10T11:44:38.179 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-05-10T11:44:38.179 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-05-10T11:44:38.179 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 6806, 2024-05-10T11:44:38.179 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 240881960, 2024-05-10T11:44:38.179 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-05-10T11:44:38.179 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:38.179 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:38.179 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:38.179 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-05-10T11:44:38.179 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:38.179 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-objecter_bytes": { 2024-05-10T11:44:38.179 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-05-10T11:44:38.179 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 104857600, 2024-05-10T11:44:38.179 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-05-10T11:44:38.179 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-05-10T11:44:38.180 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-05-10T11:44:38.180 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-05-10T11:44:38.180 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-05-10T11:44:38.180 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 6285, 2024-05-10T11:44:38.180 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 239095660, 2024-05-10T11:44:38.180 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 6285, 2024-05-10T11:44:38.180 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 239095660, 2024-05-10T11:44:38.180 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-05-10T11:44:38.180 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:38.180 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:38.180 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:38.180 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-05-10T11:44:38.180 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:38.180 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-objecter_ops": { 2024-05-10T11:44:38.180 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-05-10T11:44:38.181 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 1024, 2024-05-10T11:44:38.181 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-05-10T11:44:38.181 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-05-10T11:44:38.181 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-05-10T11:44:38.181 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-05-10T11:44:38.181 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-05-10T11:44:38.181 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 6285, 2024-05-10T11:44:38.181 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 6285, 2024-05-10T11:44:38.181 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 6285, 2024-05-10T11:44:38.181 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 6285, 2024-05-10T11:44:38.181 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-05-10T11:44:38.181 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:38.181 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:38.181 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:38.181 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-05-10T11:44:38.182 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:38.182 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-write_buf_throttle": { 2024-05-10T11:44:38.182 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-05-10T11:44:38.182 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 3758096384, 2024-05-10T11:44:38.182 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-05-10T11:44:38.182 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-05-10T11:44:38.182 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-05-10T11:44:38.182 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-05-10T11:44:38.182 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-05-10T11:44:38.182 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-05-10T11:44:38.182 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-05-10T11:44:38.182 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 0, 2024-05-10T11:44:38.182 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 0, 2024-05-10T11:44:38.182 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-05-10T11:44:38.182 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:38.183 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:38.183 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:38.183 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-05-10T11:44:38.183 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:38.183 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-write_buf_throttle-0x55e3c74dc6e0": { 2024-05-10T11:44:38.183 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-05-10T11:44:38.183 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 3758096384, 2024-05-10T11:44:38.183 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-05-10T11:44:38.183 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-05-10T11:44:38.183 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-05-10T11:44:38.183 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-05-10T11:44:38.183 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-05-10T11:44:38.183 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-05-10T11:44:38.183 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-05-10T11:44:38.183 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 0, 2024-05-10T11:44:38.184 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 0, 2024-05-10T11:44:38.184 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-05-10T11:44:38.184 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:38.184 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:38.184 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:38.184 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-05-10T11:44:38.184 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-05-10T11:44:38.184 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-05-10T11:44:38.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:44:38 smithi178 ceph-mon[33359]: pgmap v1208: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 767 B/s wr, 7 op/s 2024-05-10T11:44:38.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:44:38 smithi188 ceph-mon[34075]: pgmap v1208: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 767 B/s wr, 7 op/s 2024-05-10T11:44:38.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:44:38 smithi100 ceph-mon[30296]: pgmap v1208: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 767 B/s wr, 7 op/s 2024-05-10T11:44:39.052 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-05-10T11:44:39.052 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-05-10T11:44:39.053 DEBUG:tasks.check_counter:Getting stats from h 2024-05-10T11:44:39.053 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph daemon mds.h perf dump 2024-05-10T11:44:39.294 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T11:44:40.432 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:44:40 smithi178 ceph-mon[33359]: pgmap v1209: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 682 B/s wr, 5 op/s 2024-05-10T11:44:41.286 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:44:40 smithi100 ceph-mon[30296]: pgmap v1209: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 682 B/s wr, 5 op/s 2024-05-10T11:44:41.286 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:44:40 smithi188 ceph-mon[34075]: pgmap v1209: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 682 B/s wr, 5 op/s 2024-05-10T11:44:41.286 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-05-10T11:44:41.286 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T11:44:41.286 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 3131, 2024-05-10T11:44:41.287 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 3109, 2024-05-10T11:44:41.287 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 67790875, 2024-05-10T11:44:41.287 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 1135108, 2024-05-10T11:44:41.287 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 7, 2024-05-10T11:44:41.287 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 3, 2024-05-10T11:44:41.287 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.284907453, 2024-05-10T11:44:41.287 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.093761811, 2024-05-10T11:44:41.287 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.101380549, 2024-05-10T11:44:41.287 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.068316276, 2024-05-10T11:44:41.287 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T11:44:41.287 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 3108, 2024-05-10T11:44:41.287 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.098406225, 2024-05-10T11:44:41.287 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000031662 2024-05-10T11:44:41.287 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:41.288 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-05-10T11:44:41.288 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:41.288 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:41.288 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:41.288 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:41.288 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 688800, 2024-05-10T11:44:41.288 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 303856 2024-05-10T11:44:41.288 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:41.288 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T11:44:41.288 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 1522, 2024-05-10T11:44:41.288 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 1891, 2024-05-10T11:44:41.288 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 101050978, 2024-05-10T11:44:41.288 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 1375976, 2024-05-10T11:44:41.288 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 9, 2024-05-10T11:44:41.289 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 6, 2024-05-10T11:44:41.289 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.156611253, 2024-05-10T11:44:41.289 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.038637672, 2024-05-10T11:44:41.289 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.078546111, 2024-05-10T11:44:41.289 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.029435498, 2024-05-10T11:44:41.289 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T11:44:41.289 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 1890, 2024-05-10T11:44:41.289 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.046495539, 2024-05-10T11:44:41.289 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000024600 2024-05-10T11:44:41.289 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:41.289 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-05-10T11:44:41.289 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:41.289 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:41.289 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:41.289 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:41.290 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 3392, 2024-05-10T11:44:41.290 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 894144 2024-05-10T11:44:41.290 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:41.290 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T11:44:41.290 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 2427, 2024-05-10T11:44:41.290 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 2429, 2024-05-10T11:44:41.290 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 126446089, 2024-05-10T11:44:41.290 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 771793, 2024-05-10T11:44:41.290 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 7, 2024-05-10T11:44:41.290 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 5, 2024-05-10T11:44:41.290 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.209981774, 2024-05-10T11:44:41.290 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.037424549, 2024-05-10T11:44:41.290 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.111726778, 2024-05-10T11:44:41.290 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.049688022, 2024-05-10T11:44:41.291 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T11:44:41.291 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 2427, 2024-05-10T11:44:41.291 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.049196097, 2024-05-10T11:44:41.291 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000020270 2024-05-10T11:44:41.291 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:41.291 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-05-10T11:44:41.291 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:41.291 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:41.291 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:41.291 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:41.291 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-10T11:44:41.291 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 0 2024-05-10T11:44:41.291 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:41.291 INFO:teuthology.orchestra.run.smithi178.stdout: "cct": { 2024-05-10T11:44:41.291 INFO:teuthology.orchestra.run.smithi178.stdout: "total_workers": 1, 2024-05-10T11:44:41.291 INFO:teuthology.orchestra.run.smithi178.stdout: "unhealthy_workers": 0 2024-05-10T11:44:41.292 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:41.292 INFO:teuthology.orchestra.run.smithi178.stdout: "finisher-MDSRank": { 2024-05-10T11:44:41.292 INFO:teuthology.orchestra.run.smithi178.stdout: "queue_len": 0, 2024-05-10T11:44:41.292 INFO:teuthology.orchestra.run.smithi178.stdout: "complete_latency": { 2024-05-10T11:44:41.292 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 11319, 2024-05-10T11:44:41.292 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.226135546, 2024-05-10T11:44:41.292 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000019978 2024-05-10T11:44:41.292 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-05-10T11:44:41.292 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:41.292 INFO:teuthology.orchestra.run.smithi178.stdout: "finisher-PurgeQueue": { 2024-05-10T11:44:41.292 INFO:teuthology.orchestra.run.smithi178.stdout: "queue_len": 0, 2024-05-10T11:44:41.292 INFO:teuthology.orchestra.run.smithi178.stdout: "complete_latency": { 2024-05-10T11:44:41.292 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:41.292 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:41.293 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:41.293 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-05-10T11:44:41.293 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:41.293 INFO:teuthology.orchestra.run.smithi178.stdout: "mds": { 2024-05-10T11:44:41.293 INFO:teuthology.orchestra.run.smithi178.stdout: "request": 0, 2024-05-10T11:44:41.293 INFO:teuthology.orchestra.run.smithi178.stdout: "reply": 0, 2024-05-10T11:44:41.293 INFO:teuthology.orchestra.run.smithi178.stdout: "reply_latency": { 2024-05-10T11:44:41.293 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:41.293 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:41.293 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:41.293 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:41.293 INFO:teuthology.orchestra.run.smithi178.stdout: "slow_reply": 0, 2024-05-10T11:44:41.293 INFO:teuthology.orchestra.run.smithi178.stdout: "forward": 0, 2024-05-10T11:44:41.293 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_fetch_complete": 0, 2024-05-10T11:44:41.293 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_fetch_keys": 0, 2024-05-10T11:44:41.293 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_commit": 0, 2024-05-10T11:44:41.294 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_split": 0, 2024-05-10T11:44:41.294 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_merge": 0, 2024-05-10T11:44:41.294 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes": 25559, 2024-05-10T11:44:41.294 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_top": 7813, 2024-05-10T11:44:41.294 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_bottom": 17746, 2024-05-10T11:44:41.294 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_pin_tail": 0, 2024-05-10T11:44:41.294 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_pinned": 14397, 2024-05-10T11:44:41.294 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_expired": 1, 2024-05-10T11:44:41.294 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_with_caps": 0, 2024-05-10T11:44:41.294 INFO:teuthology.orchestra.run.smithi178.stdout: "caps": 0, 2024-05-10T11:44:41.294 INFO:teuthology.orchestra.run.smithi178.stdout: "subtrees": 3, 2024-05-10T11:44:41.294 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse": 0, 2024-05-10T11:44:41.295 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_hit": 0, 2024-05-10T11:44:41.295 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_forward": 0, 2024-05-10T11:44:41.295 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_discover": 0, 2024-05-10T11:44:41.295 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_dir_fetch": 0, 2024-05-10T11:44:41.295 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_remote_ino": 0, 2024-05-10T11:44:41.296 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_lock": 0, 2024-05-10T11:44:41.296 INFO:teuthology.orchestra.run.smithi178.stdout: "load_cent": 0, 2024-05-10T11:44:41.296 INFO:teuthology.orchestra.run.smithi178.stdout: "q": 0, 2024-05-10T11:44:41.296 INFO:teuthology.orchestra.run.smithi178.stdout: "exported": 0, 2024-05-10T11:44:41.296 INFO:teuthology.orchestra.run.smithi178.stdout: "exported_inodes": 0, 2024-05-10T11:44:41.296 INFO:teuthology.orchestra.run.smithi178.stdout: "imported": 0, 2024-05-10T11:44:41.296 INFO:teuthology.orchestra.run.smithi178.stdout: "imported_inodes": 0, 2024-05-10T11:44:41.296 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_dir_fetch": 0, 2024-05-10T11:44:41.296 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_backtrace_fetch": 0, 2024-05-10T11:44:41.296 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_peer_discover": 0, 2024-05-10T11:44:41.296 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rfiles": 0, 2024-05-10T11:44:41.296 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rbytes": 0, 2024-05-10T11:44:41.296 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rsnaps": 0, 2024-05-10T11:44:41.296 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_backtrace_fetch": 0, 2024-05-10T11:44:41.297 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_set_tag": 0, 2024-05-10T11:44:41.297 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_backtrace_repaired": 0, 2024-05-10T11:44:41.297 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_inotable_repaired": 0, 2024-05-10T11:44:41.297 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dir_inodes": 0, 2024-05-10T11:44:41.297 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dir_base_inodes": 0, 2024-05-10T11:44:41.297 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dirfrag_rstats": 0, 2024-05-10T11:44:41.297 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_file_inodes": 0, 2024-05-10T11:44:41.297 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_inode_file_caps": 0, 2024-05-10T11:44:41.297 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_revoke": 0, 2024-05-10T11:44:41.297 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_grant": 0, 2024-05-10T11:44:41.297 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_trunc": 0, 2024-05-10T11:44:41.298 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-10T11:44:41.298 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-10T11:44:41.298 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_caps": 0, 2024-05-10T11:44:41.298 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_caps_dirty": 0, 2024-05-10T11:44:41.298 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_cap_release": 0, 2024-05-10T11:44:41.298 INFO:teuthology.orchestra.run.smithi178.stdout: "process_request_cap_release": 0 2024-05-10T11:44:41.298 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:41.298 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_cache": { 2024-05-10T11:44:41.298 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_update": 0, 2024-05-10T11:44:41.298 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_update_receipt": 0, 2024-05-10T11:44:41.298 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_try_discover": 0, 2024-05-10T11:44:41.298 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_send_discover": 0, 2024-05-10T11:44:41.298 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_handle_discover": 0, 2024-05-10T11:44:41.299 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays": 0, 2024-05-10T11:44:41.299 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays_delayed": 0, 2024-05-10T11:44:41.299 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays_enqueuing": 0, 2024-05-10T11:44:41.299 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_created": 0, 2024-05-10T11:44:41.299 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_enqueued": 0, 2024-05-10T11:44:41.299 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_reintegrated": 0, 2024-05-10T11:44:41.299 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_migrated": 0, 2024-05-10T11:44:41.299 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_processing": 0, 2024-05-10T11:44:41.299 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_enqueued": 0, 2024-05-10T11:44:41.299 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_prioritized": 0, 2024-05-10T11:44:41.299 INFO:teuthology.orchestra.run.smithi178.stdout: "recovery_started": 0, 2024-05-10T11:44:41.299 INFO:teuthology.orchestra.run.smithi178.stdout: "recovery_completed": 0, 2024-05-10T11:44:41.300 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_quiesce_path": 0, 2024-05-10T11:44:41.300 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_quiesce_inode": 0, 2024-05-10T11:44:41.300 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_enqueue_scrub": 0, 2024-05-10T11:44:41.300 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_exportdir": 0, 2024-05-10T11:44:41.300 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_flush": 0, 2024-05-10T11:44:41.300 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_fragmentdir": 0, 2024-05-10T11:44:41.300 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_fragstats": 0, 2024-05-10T11:44:41.300 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_inodestats": 0 2024-05-10T11:44:41.300 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:41.300 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_log": { 2024-05-10T11:44:41.300 INFO:teuthology.orchestra.run.smithi178.stdout: "evlrg": 0, 2024-05-10T11:44:41.300 INFO:teuthology.orchestra.run.smithi178.stdout: "evadd": 0, 2024-05-10T11:44:41.300 INFO:teuthology.orchestra.run.smithi178.stdout: "evex": 0, 2024-05-10T11:44:41.300 INFO:teuthology.orchestra.run.smithi178.stdout: "evtrm": 0, 2024-05-10T11:44:41.300 INFO:teuthology.orchestra.run.smithi178.stdout: "ev": 190453, 2024-05-10T11:44:41.301 INFO:teuthology.orchestra.run.smithi178.stdout: "evexg": 0, 2024-05-10T11:44:41.301 INFO:teuthology.orchestra.run.smithi178.stdout: "evexd": 0, 2024-05-10T11:44:41.301 INFO:teuthology.orchestra.run.smithi178.stdout: "segadd": 0, 2024-05-10T11:44:41.301 INFO:teuthology.orchestra.run.smithi178.stdout: "segex": 0, 2024-05-10T11:44:41.301 INFO:teuthology.orchestra.run.smithi178.stdout: "segtrm": 0, 2024-05-10T11:44:41.301 INFO:teuthology.orchestra.run.smithi178.stdout: "seg": 129, 2024-05-10T11:44:41.301 INFO:teuthology.orchestra.run.smithi178.stdout: "segmjr": 17, 2024-05-10T11:44:41.301 INFO:teuthology.orchestra.run.smithi178.stdout: "segexg": 0, 2024-05-10T11:44:41.301 INFO:teuthology.orchestra.run.smithi178.stdout: "segexd": 0, 2024-05-10T11:44:41.301 INFO:teuthology.orchestra.run.smithi178.stdout: "expos": 118327444, 2024-05-10T11:44:41.301 INFO:teuthology.orchestra.run.smithi178.stdout: "wrpos": 296980547, 2024-05-10T11:44:41.301 INFO:teuthology.orchestra.run.smithi178.stdout: "rdpos": 296979948, 2024-05-10T11:44:41.301 INFO:teuthology.orchestra.run.smithi178.stdout: "jlat": { 2024-05-10T11:44:41.301 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:41.302 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:41.302 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:41.302 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:41.302 INFO:teuthology.orchestra.run.smithi178.stdout: "replayed": 190453 2024-05-10T11:44:41.302 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:41.302 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_mem": { 2024-05-10T11:44:41.302 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 18, 2024-05-10T11:44:41.302 INFO:teuthology.orchestra.run.smithi178.stdout: "ino+": 25628, 2024-05-10T11:44:41.302 INFO:teuthology.orchestra.run.smithi178.stdout: "ino-": 25610, 2024-05-10T11:44:41.302 INFO:teuthology.orchestra.run.smithi178.stdout: "dir": 16, 2024-05-10T11:44:41.302 INFO:teuthology.orchestra.run.smithi178.stdout: "dir+": 70, 2024-05-10T11:44:41.302 INFO:teuthology.orchestra.run.smithi178.stdout: "dir-": 54, 2024-05-10T11:44:41.302 INFO:teuthology.orchestra.run.smithi178.stdout: "dn": 25559, 2024-05-10T11:44:41.303 INFO:teuthology.orchestra.run.smithi178.stdout: "dn+": 26178, 2024-05-10T11:44:41.303 INFO:teuthology.orchestra.run.smithi178.stdout: "dn-": 619, 2024-05-10T11:44:41.303 INFO:teuthology.orchestra.run.smithi178.stdout: "cap": 0, 2024-05-10T11:44:41.303 INFO:teuthology.orchestra.run.smithi178.stdout: "cap+": 0, 2024-05-10T11:44:41.303 INFO:teuthology.orchestra.run.smithi178.stdout: "cap-": 0, 2024-05-10T11:44:41.303 INFO:teuthology.orchestra.run.smithi178.stdout: "rss": 76424, 2024-05-10T11:44:41.303 INFO:teuthology.orchestra.run.smithi178.stdout: "heap": 215416 2024-05-10T11:44:41.303 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:41.303 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_server": { 2024-05-10T11:44:41.303 INFO:teuthology.orchestra.run.smithi178.stdout: "dispatch_client_request": 0, 2024-05-10T11:44:41.303 INFO:teuthology.orchestra.run.smithi178.stdout: "dispatch_server_request": 0, 2024-05-10T11:44:41.303 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_request": 0, 2024-05-10T11:44:41.303 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_session": 0, 2024-05-10T11:44:41.304 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_peer_request": 0, 2024-05-10T11:44:41.304 INFO:teuthology.orchestra.run.smithi178.stdout: "req_create_latency": { 2024-05-10T11:44:41.304 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:41.304 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:41.304 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:41.304 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:41.304 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getattr_latency": { 2024-05-10T11:44:41.304 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:41.304 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:41.304 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:41.304 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:41.304 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getfilelock_latency": { 2024-05-10T11:44:41.304 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:41.304 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:41.304 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:41.305 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:41.305 INFO:teuthology.orchestra.run.smithi178.stdout: "req_link_latency": { 2024-05-10T11:44:41.305 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:41.305 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:41.305 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:41.305 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:41.305 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookup_latency": { 2024-05-10T11:44:41.305 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:41.305 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:41.305 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:41.305 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:41.305 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookuphash_latency": { 2024-05-10T11:44:41.305 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:41.305 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:41.305 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:41.306 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:41.306 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupino_latency": { 2024-05-10T11:44:41.306 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:41.306 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:41.306 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:41.306 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:41.306 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupname_latency": { 2024-05-10T11:44:41.306 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:41.306 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:41.307 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:41.307 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:41.307 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupparent_latency": { 2024-05-10T11:44:41.307 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:41.307 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:41.307 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:41.307 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:41.307 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupsnap_latency": { 2024-05-10T11:44:41.307 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:41.307 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:41.307 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:41.307 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:41.307 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lssnap_latency": { 2024-05-10T11:44:41.307 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:41.307 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:41.307 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:41.308 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:41.308 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mkdir_latency": { 2024-05-10T11:44:41.308 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:41.308 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:41.308 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:41.308 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:41.308 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mknod_latency": { 2024-05-10T11:44:41.308 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:41.308 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:41.308 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:41.308 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:41.308 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mksnap_latency": { 2024-05-10T11:44:41.309 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:41.309 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:41.309 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:41.309 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:41.309 INFO:teuthology.orchestra.run.smithi178.stdout: "req_open_latency": { 2024-05-10T11:44:41.309 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:41.309 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:41.309 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:41.309 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:41.309 INFO:teuthology.orchestra.run.smithi178.stdout: "req_readdir_latency": { 2024-05-10T11:44:41.309 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:41.309 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:41.309 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:41.309 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:41.309 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rename_latency": { 2024-05-10T11:44:41.310 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:41.310 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:41.310 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:41.310 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:41.310 INFO:teuthology.orchestra.run.smithi178.stdout: "req_renamesnap_latency": { 2024-05-10T11:44:41.310 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:41.310 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:41.310 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:41.310 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:41.310 INFO:teuthology.orchestra.run.smithi178.stdout: "req_snapdiff_latency": { 2024-05-10T11:44:41.310 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:41.310 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:41.310 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:41.311 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:41.311 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmdir_latency": { 2024-05-10T11:44:41.311 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:41.311 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:41.311 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:41.311 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:41.311 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmsnap_latency": { 2024-05-10T11:44:41.311 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:41.311 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:41.311 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:41.311 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:41.311 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmxattr_latency": { 2024-05-10T11:44:41.311 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:41.311 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:41.312 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:41.312 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:41.312 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setattr_latency": { 2024-05-10T11:44:41.312 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:41.312 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:41.312 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:41.312 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:41.312 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setdirlayout_latency": { 2024-05-10T11:44:41.312 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:41.312 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:41.312 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:41.312 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:41.312 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setfilelock_latency": { 2024-05-10T11:44:41.312 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:41.313 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:41.313 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:41.313 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:41.313 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setlayout_latency": { 2024-05-10T11:44:41.313 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:41.313 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:41.313 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:41.313 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:41.313 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setxattr_latency": { 2024-05-10T11:44:41.313 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:41.313 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:41.313 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:41.313 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:41.313 INFO:teuthology.orchestra.run.smithi178.stdout: "req_symlink_latency": { 2024-05-10T11:44:41.313 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:41.314 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:41.314 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:41.314 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:41.314 INFO:teuthology.orchestra.run.smithi178.stdout: "req_unlink_latency": { 2024-05-10T11:44:41.314 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:41.314 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:41.314 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:41.314 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:41.314 INFO:teuthology.orchestra.run.smithi178.stdout: "cap_revoke_eviction": 0, 2024-05-10T11:44:41.314 INFO:teuthology.orchestra.run.smithi178.stdout: "cap_acquisition_throttle": 0, 2024-05-10T11:44:41.314 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getvxattr_latency": { 2024-05-10T11:44:41.314 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:41.314 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:41.315 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:41.315 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-05-10T11:44:41.315 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:41.315 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_sessions": { 2024-05-10T11:44:41.315 INFO:teuthology.orchestra.run.smithi178.stdout: "session_count": 1, 2024-05-10T11:44:41.315 INFO:teuthology.orchestra.run.smithi178.stdout: "session_add": 2, 2024-05-10T11:44:41.315 INFO:teuthology.orchestra.run.smithi178.stdout: "session_remove": 1, 2024-05-10T11:44:41.315 INFO:teuthology.orchestra.run.smithi178.stdout: "sessions_open": 1, 2024-05-10T11:44:41.315 INFO:teuthology.orchestra.run.smithi178.stdout: "sessions_stale": 0, 2024-05-10T11:44:41.315 INFO:teuthology.orchestra.run.smithi178.stdout: "total_load": 0, 2024-05-10T11:44:41.315 INFO:teuthology.orchestra.run.smithi178.stdout: "average_load": 0, 2024-05-10T11:44:41.315 INFO:teuthology.orchestra.run.smithi178.stdout: "avg_session_uptime": 2825, 2024-05-10T11:44:41.315 INFO:teuthology.orchestra.run.smithi178.stdout: "mdthresh_evicted": 0 2024-05-10T11:44:41.315 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:41.315 INFO:teuthology.orchestra.run.smithi178.stdout: "mempool": { 2024-05-10T11:44:41.316 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_bytes": 0, 2024-05-10T11:44:41.316 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_items": 0, 2024-05-10T11:44:41.316 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T11:44:41.316 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_items": 0, 2024-05-10T11:44:41.316 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T11:44:41.316 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_items": 0, 2024-05-10T11:44:41.316 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T11:44:41.316 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T11:44:41.316 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T11:44:41.316 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T11:44:41.316 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T11:44:41.316 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_items": 0, 2024-05-10T11:44:41.316 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T11:44:41.316 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T11:44:41.317 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_bytes": 0, 2024-05-10T11:44:41.317 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_items": 0, 2024-05-10T11:44:41.317 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_bytes": 0, 2024-05-10T11:44:41.317 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_items": 0, 2024-05-10T11:44:41.317 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T11:44:41.317 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T11:44:41.317 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T11:44:41.317 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T11:44:41.317 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T11:44:41.317 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_items": 0, 2024-05-10T11:44:41.317 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_bytes": 0, 2024-05-10T11:44:41.317 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_items": 0, 2024-05-10T11:44:41.317 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T11:44:41.317 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T11:44:41.317 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_bytes": 0, 2024-05-10T11:44:41.318 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_items": 0, 2024-05-10T11:44:41.318 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_bytes": 0, 2024-05-10T11:44:41.318 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_items": 0, 2024-05-10T11:44:41.318 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T11:44:41.318 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_items": 0, 2024-05-10T11:44:41.318 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T11:44:41.318 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_items": 0, 2024-05-10T11:44:41.318 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_bytes": 242014, 2024-05-10T11:44:41.318 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_items": 82, 2024-05-10T11:44:41.318 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_bytes": 0, 2024-05-10T11:44:41.318 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_items": 0, 2024-05-10T11:44:41.318 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_bytes": 0, 2024-05-10T11:44:41.318 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_items": 0, 2024-05-10T11:44:41.319 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_bytes": 0, 2024-05-10T11:44:41.319 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_items": 0, 2024-05-10T11:44:41.319 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_bytes": 0, 2024-05-10T11:44:41.319 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_items": 0, 2024-05-10T11:44:41.319 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_bytes": 8696, 2024-05-10T11:44:41.319 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_items": 166, 2024-05-10T11:44:41.319 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T11:44:41.319 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_items": 0, 2024-05-10T11:44:41.319 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_bytes": 0, 2024-05-10T11:44:41.319 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_items": 0, 2024-05-10T11:44:41.319 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_bytes": 18054533, 2024-05-10T11:44:41.319 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_items": 102433, 2024-05-10T11:44:41.319 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_bytes": 0, 2024-05-10T11:44:41.319 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_items": 0, 2024-05-10T11:44:41.320 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_bytes": 0, 2024-05-10T11:44:41.320 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_items": 0 2024-05-10T11:44:41.320 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:41.320 INFO:teuthology.orchestra.run.smithi178.stdout: "objecter": { 2024-05-10T11:44:41.320 INFO:teuthology.orchestra.run.smithi178.stdout: "op_active": 0, 2024-05-10T11:44:41.320 INFO:teuthology.orchestra.run.smithi178.stdout: "op_laggy": 0, 2024-05-10T11:44:41.320 INFO:teuthology.orchestra.run.smithi178.stdout: "op_send": 6556, 2024-05-10T11:44:41.320 INFO:teuthology.orchestra.run.smithi178.stdout: "op_send_bytes": 21, 2024-05-10T11:44:41.320 INFO:teuthology.orchestra.run.smithi178.stdout: "op_resend": 3, 2024-05-10T11:44:41.320 INFO:teuthology.orchestra.run.smithi178.stdout: "op_reply": 6556, 2024-05-10T11:44:41.320 INFO:teuthology.orchestra.run.smithi178.stdout: "op_latency": { 2024-05-10T11:44:41.320 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 6556, 2024-05-10T11:44:41.320 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 4.604922070, 2024-05-10T11:44:41.320 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000702398 2024-05-10T11:44:41.321 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:41.321 INFO:teuthology.orchestra.run.smithi178.stdout: "op_inflight": 0, 2024-05-10T11:44:41.321 INFO:teuthology.orchestra.run.smithi178.stdout: "oplen_avg": { 2024-05-10T11:44:41.321 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 6556, 2024-05-10T11:44:41.321 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 6557 2024-05-10T11:44:41.321 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:41.321 INFO:teuthology.orchestra.run.smithi178.stdout: "op": 6556, 2024-05-10T11:44:41.321 INFO:teuthology.orchestra.run.smithi178.stdout: "op_r": 6556, 2024-05-10T11:44:41.321 INFO:teuthology.orchestra.run.smithi178.stdout: "op_w": 0, 2024-05-10T11:44:41.321 INFO:teuthology.orchestra.run.smithi178.stdout: "op_rmw": 0, 2024-05-10T11:44:41.321 INFO:teuthology.orchestra.run.smithi178.stdout: "op_pg": 0, 2024-05-10T11:44:41.321 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_stat": 3371, 2024-05-10T11:44:41.321 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_create": 0, 2024-05-10T11:44:41.322 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_read": 3183, 2024-05-10T11:44:41.322 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_write": 0, 2024-05-10T11:44:41.322 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_writefull": 0, 2024-05-10T11:44:41.322 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_writesame": 0, 2024-05-10T11:44:41.322 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_append": 0, 2024-05-10T11:44:41.322 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_zero": 0, 2024-05-10T11:44:41.322 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_truncate": 0, 2024-05-10T11:44:41.322 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_delete": 0, 2024-05-10T11:44:41.323 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_mapext": 0, 2024-05-10T11:44:41.323 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_sparse_read": 0, 2024-05-10T11:44:41.323 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_clonerange": 0, 2024-05-10T11:44:41.323 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_getxattr": 1, 2024-05-10T11:44:41.323 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_setxattr": 0, 2024-05-10T11:44:41.323 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_cmpxattr": 0, 2024-05-10T11:44:41.323 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_rmxattr": 0, 2024-05-10T11:44:41.323 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_resetxattrs": 0, 2024-05-10T11:44:41.323 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_call": 0, 2024-05-10T11:44:41.323 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_watch": 0, 2024-05-10T11:44:41.323 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_notify": 0, 2024-05-10T11:44:41.323 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_src_cmpxattr": 0, 2024-05-10T11:44:41.323 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_pgls": 0, 2024-05-10T11:44:41.324 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_pgls_filter": 0, 2024-05-10T11:44:41.324 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_other": 0, 2024-05-10T11:44:41.324 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_active": 0, 2024-05-10T11:44:41.324 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_send": 0, 2024-05-10T11:44:41.324 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_resend": 0, 2024-05-10T11:44:41.324 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_ping": 0, 2024-05-10T11:44:41.324 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_active": 0, 2024-05-10T11:44:41.324 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_send": 0, 2024-05-10T11:44:41.324 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_resend": 0, 2024-05-10T11:44:41.324 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_active": 0, 2024-05-10T11:44:41.324 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_send": 0, 2024-05-10T11:44:41.324 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_resend": 0, 2024-05-10T11:44:41.324 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_active": 0, 2024-05-10T11:44:41.324 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_send": 0, 2024-05-10T11:44:41.325 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_resend": 0, 2024-05-10T11:44:41.325 INFO:teuthology.orchestra.run.smithi178.stdout: "command_active": 0, 2024-05-10T11:44:41.325 INFO:teuthology.orchestra.run.smithi178.stdout: "command_send": 0, 2024-05-10T11:44:41.325 INFO:teuthology.orchestra.run.smithi178.stdout: "command_resend": 0, 2024-05-10T11:44:41.325 INFO:teuthology.orchestra.run.smithi178.stdout: "map_epoch": 90, 2024-05-10T11:44:41.325 INFO:teuthology.orchestra.run.smithi178.stdout: "map_full": 0, 2024-05-10T11:44:41.325 INFO:teuthology.orchestra.run.smithi178.stdout: "map_inc": 1, 2024-05-10T11:44:41.325 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_sessions": 11, 2024-05-10T11:44:41.325 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_session_open": 12, 2024-05-10T11:44:41.325 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_session_close": 1, 2024-05-10T11:44:41.325 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_laggy": 0, 2024-05-10T11:44:41.325 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_wr": 0, 2024-05-10T11:44:41.325 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_rd": 2, 2024-05-10T11:44:41.325 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_del": 0 2024-05-10T11:44:41.325 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:41.326 INFO:teuthology.orchestra.run.smithi178.stdout: "oft": { 2024-05-10T11:44:41.326 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_objs": 0, 2024-05-10T11:44:41.326 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_kv_pairs": 0, 2024-05-10T11:44:41.326 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_updates": 0, 2024-05-10T11:44:41.326 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_removes": 0 2024-05-10T11:44:41.326 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:41.326 INFO:teuthology.orchestra.run.smithi178.stdout: "osd-slow-ops": { 2024-05-10T11:44:41.326 INFO:teuthology.orchestra.run.smithi178.stdout: "slow_ops_count": 0 2024-05-10T11:44:41.326 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:41.326 INFO:teuthology.orchestra.run.smithi178.stdout: "purge_queue": { 2024-05-10T11:44:41.326 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_ops": 0, 2024-05-10T11:44:41.326 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_ops_high_water": 0, 2024-05-10T11:44:41.327 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing": 0, 2024-05-10T11:44:41.327 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_high_water": 0, 2024-05-10T11:44:41.327 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executed_ops": 0, 2024-05-10T11:44:41.327 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executed": 0, 2024-05-10T11:44:41.327 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_item_in_journal": 0 2024-05-10T11:44:41.327 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:41.327 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T11:44:41.327 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-05-10T11:44:41.327 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 104857600, 2024-05-10T11:44:41.327 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-05-10T11:44:41.327 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 7076, 2024-05-10T11:44:41.327 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 294621532, 2024-05-10T11:44:41.327 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-05-10T11:44:41.327 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 7076, 2024-05-10T11:44:41.328 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-05-10T11:44:41.328 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-05-10T11:44:41.328 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 7076, 2024-05-10T11:44:41.328 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 294621532, 2024-05-10T11:44:41.328 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-05-10T11:44:41.328 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:41.328 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:41.328 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:41.328 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-05-10T11:44:41.328 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:41.328 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-objecter_bytes": { 2024-05-10T11:44:41.328 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-05-10T11:44:41.328 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 104857600, 2024-05-10T11:44:41.328 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-05-10T11:44:41.328 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-05-10T11:44:41.329 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-05-10T11:44:41.329 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-05-10T11:44:41.329 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-05-10T11:44:41.329 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 6556, 2024-05-10T11:44:41.329 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 292790468, 2024-05-10T11:44:41.329 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 6556, 2024-05-10T11:44:41.329 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 292790468, 2024-05-10T11:44:41.329 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-05-10T11:44:41.329 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:41.329 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:41.330 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:41.330 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-05-10T11:44:41.330 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:41.330 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-objecter_ops": { 2024-05-10T11:44:41.330 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-05-10T11:44:41.330 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 1024, 2024-05-10T11:44:41.330 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-05-10T11:44:41.330 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-05-10T11:44:41.331 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-05-10T11:44:41.331 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-05-10T11:44:41.331 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-05-10T11:44:41.331 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 6556, 2024-05-10T11:44:41.331 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 6556, 2024-05-10T11:44:41.331 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 6556, 2024-05-10T11:44:41.331 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 6556, 2024-05-10T11:44:41.331 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-05-10T11:44:41.331 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:41.331 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:41.331 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:41.331 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-05-10T11:44:41.332 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:41.332 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-write_buf_throttle": { 2024-05-10T11:44:41.332 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-05-10T11:44:41.332 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 3758096384, 2024-05-10T11:44:41.332 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-05-10T11:44:41.332 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-05-10T11:44:41.332 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-05-10T11:44:41.332 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-05-10T11:44:41.332 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-05-10T11:44:41.332 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-05-10T11:44:41.332 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-05-10T11:44:41.332 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 0, 2024-05-10T11:44:41.332 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 0, 2024-05-10T11:44:41.332 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-05-10T11:44:41.333 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:41.333 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:41.333 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:41.333 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-05-10T11:44:41.333 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:41.333 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-write_buf_throttle-0x55f6a70f26e0": { 2024-05-10T11:44:41.333 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-05-10T11:44:41.333 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 3758096384, 2024-05-10T11:44:41.333 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-05-10T11:44:41.333 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-05-10T11:44:41.333 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-05-10T11:44:41.333 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-05-10T11:44:41.333 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-05-10T11:44:41.333 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-05-10T11:44:41.334 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-05-10T11:44:41.334 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 0, 2024-05-10T11:44:41.334 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 0, 2024-05-10T11:44:41.334 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-05-10T11:44:41.334 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:41.334 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:41.334 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:41.334 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-05-10T11:44:41.334 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-05-10T11:44:41.334 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-05-10T11:44:41.837 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-05-10T11:44:41.837 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-05-10T11:44:41.837 DEBUG:tasks.check_counter:Getting stats from k 2024-05-10T11:44:41.838 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph daemon mds.k perf dump 2024-05-10T11:44:42.077 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T11:44:42.365 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:44:42 smithi100 ceph-mon[30296]: pgmap v1210: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-05-10T11:44:42.369 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:44:42 smithi178 ceph-mon[33359]: pgmap v1210: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-05-10T11:44:42.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:44:42 smithi188 ceph-mon[34075]: pgmap v1210: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-05-10T11:44:43.959 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-05-10T11:44:43.959 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T11:44:43.959 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 0, 2024-05-10T11:44:43.960 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 2, 2024-05-10T11:44:43.960 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 0, 2024-05-10T11:44:43.960 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 0, 2024-05-10T11:44:43.960 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 4, 2024-05-10T11:44:43.960 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 0, 2024-05-10T11:44:43.960 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.000965648, 2024-05-10T11:44:43.960 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.000000000, 2024-05-10T11:44:43.960 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.000173085, 2024-05-10T11:44:43.960 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-10T11:44:43.960 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T11:44:43.960 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:43.960 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:43.961 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:43.961 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:43.961 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-05-10T11:44:43.961 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:43.961 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:43.961 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:43.961 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:43.961 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-10T11:44:43.961 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 0 2024-05-10T11:44:43.961 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:43.961 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T11:44:43.961 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 501, 2024-05-10T11:44:43.962 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 496, 2024-05-10T11:44:43.962 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 596352, 2024-05-10T11:44:43.962 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 306496, 2024-05-10T11:44:43.962 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 2, 2024-05-10T11:44:43.962 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 1, 2024-05-10T11:44:43.962 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.058875767, 2024-05-10T11:44:43.962 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.023941490, 2024-05-10T11:44:43.962 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.022365201, 2024-05-10T11:44:43.962 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.009502095, 2024-05-10T11:44:43.962 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T11:44:43.962 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 496, 2024-05-10T11:44:43.963 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.018505382, 2024-05-10T11:44:43.963 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000037309 2024-05-10T11:44:43.963 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:43.963 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-05-10T11:44:43.963 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:43.963 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:43.963 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:43.963 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:43.963 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 596352, 2024-05-10T11:44:43.963 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 306496 2024-05-10T11:44:43.963 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:43.963 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T11:44:43.963 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 3, 2024-05-10T11:44:43.963 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 379, 2024-05-10T11:44:43.964 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 480, 2024-05-10T11:44:43.964 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 302752, 2024-05-10T11:44:43.964 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 5, 2024-05-10T11:44:43.964 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 2, 2024-05-10T11:44:43.964 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.018916372, 2024-05-10T11:44:43.964 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.014103868, 2024-05-10T11:44:43.964 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.001457012, 2024-05-10T11:44:43.964 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-10T11:44:43.964 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T11:44:43.964 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 377, 2024-05-10T11:44:43.964 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.017550038, 2024-05-10T11:44:43.964 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000046551 2024-05-10T11:44:43.964 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:43.964 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-05-10T11:44:43.965 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:43.965 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:43.965 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:43.965 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:43.965 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 480, 2024-05-10T11:44:43.965 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 302752 2024-05-10T11:44:43.965 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:43.965 INFO:teuthology.orchestra.run.smithi178.stdout: "cct": { 2024-05-10T11:44:43.965 INFO:teuthology.orchestra.run.smithi178.stdout: "total_workers": 0, 2024-05-10T11:44:43.965 INFO:teuthology.orchestra.run.smithi178.stdout: "unhealthy_workers": 0 2024-05-10T11:44:43.965 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:43.965 INFO:teuthology.orchestra.run.smithi178.stdout: "mempool": { 2024-05-10T11:44:43.965 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_bytes": 0, 2024-05-10T11:44:43.966 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_items": 0, 2024-05-10T11:44:43.966 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T11:44:43.966 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_items": 0, 2024-05-10T11:44:43.966 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T11:44:43.966 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_items": 0, 2024-05-10T11:44:43.966 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T11:44:43.966 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T11:44:43.966 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T11:44:43.966 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T11:44:43.966 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T11:44:43.966 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_items": 0, 2024-05-10T11:44:43.966 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T11:44:43.966 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T11:44:43.966 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_bytes": 0, 2024-05-10T11:44:43.967 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_items": 0, 2024-05-10T11:44:43.967 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_bytes": 0, 2024-05-10T11:44:43.967 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_items": 0, 2024-05-10T11:44:43.967 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T11:44:43.967 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T11:44:43.967 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T11:44:43.967 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T11:44:43.967 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T11:44:43.967 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_items": 0, 2024-05-10T11:44:43.967 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_bytes": 0, 2024-05-10T11:44:43.967 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_items": 0, 2024-05-10T11:44:43.967 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T11:44:43.967 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T11:44:43.967 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_bytes": 0, 2024-05-10T11:44:43.968 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_items": 0, 2024-05-10T11:44:43.968 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_bytes": 0, 2024-05-10T11:44:43.968 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_items": 0, 2024-05-10T11:44:43.968 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T11:44:43.968 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_items": 0, 2024-05-10T11:44:43.968 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T11:44:43.968 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_items": 0, 2024-05-10T11:44:43.968 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_bytes": 147960, 2024-05-10T11:44:43.968 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_items": 28, 2024-05-10T11:44:43.968 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_bytes": 0, 2024-05-10T11:44:43.968 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_items": 0, 2024-05-10T11:44:43.968 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_bytes": 0, 2024-05-10T11:44:43.968 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_items": 0, 2024-05-10T11:44:43.969 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_bytes": 0, 2024-05-10T11:44:43.969 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_items": 0, 2024-05-10T11:44:43.969 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_bytes": 0, 2024-05-10T11:44:43.969 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_items": 0, 2024-05-10T11:44:43.969 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_bytes": 0, 2024-05-10T11:44:43.969 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_items": 0, 2024-05-10T11:44:43.969 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T11:44:43.969 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_items": 0, 2024-05-10T11:44:43.969 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_bytes": 0, 2024-05-10T11:44:43.969 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_items": 0, 2024-05-10T11:44:43.969 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_bytes": 928, 2024-05-10T11:44:43.969 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_items": 2, 2024-05-10T11:44:43.969 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_bytes": 0, 2024-05-10T11:44:43.969 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_items": 0, 2024-05-10T11:44:43.969 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_bytes": 0, 2024-05-10T11:44:43.969 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_items": 0 2024-05-10T11:44:43.970 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-05-10T11:44:43.970 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T11:44:43.970 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-05-10T11:44:43.970 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 104857600, 2024-05-10T11:44:43.970 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-05-10T11:44:43.970 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 500, 2024-05-10T11:44:43.970 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 523149, 2024-05-10T11:44:43.970 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-05-10T11:44:43.970 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 500, 2024-05-10T11:44:43.970 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-05-10T11:44:43.970 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-05-10T11:44:43.970 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 500, 2024-05-10T11:44:43.970 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 523149, 2024-05-10T11:44:43.970 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-05-10T11:44:43.971 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-05-10T11:44:43.971 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-05-10T11:44:43.971 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-05-10T11:44:43.971 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-05-10T11:44:43.971 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-05-10T11:44:43.971 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-05-10T11:44:44.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:44:44 smithi178 ceph-mon[33359]: pgmap v1211: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-10T11:44:44.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:44:44 smithi188 ceph-mon[34075]: pgmap v1211: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-10T11:44:44.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:44:44 smithi100 ceph-mon[30296]: pgmap v1211: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-10T11:44:44.665 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-05-10T11:44:44.665 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-05-10T11:44:44.665 DEBUG:tasks.check_counter:Getting stats from c 2024-05-10T11:44:44.665 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph daemon mds.c perf dump 2024-05-10T11:44:44.968 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T11:44:46.414 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:44:46 smithi178 ceph-mon[33359]: pgmap v1212: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 5 op/s 2024-05-10T11:44:46.476 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:44:46 smithi188 ceph-mon[34075]: pgmap v1212: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 5 op/s 2024-05-10T11:44:46.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:44:46 smithi100 ceph-mon[30296]: pgmap v1212: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 5 op/s 2024-05-10T11:44:46.772 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-10T11:44:46.772 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T11:44:46.772 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 735623, 2024-05-10T11:44:46.772 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 847981, 2024-05-10T11:44:46.772 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 222118405, 2024-05-10T11:44:46.772 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 576603573, 2024-05-10T11:44:46.772 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 64, 2024-05-10T11:44:46.772 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 3, 2024-05-10T11:44:46.772 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 36.659770149, 2024-05-10T11:44:46.772 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 15.247278418, 2024-05-10T11:44:46.772 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 25.093076861, 2024-05-10T11:44:46.772 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 1.190624403, 2024-05-10T11:44:46.773 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T11:44:46.773 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 847980, 2024-05-10T11:44:46.773 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 23.337190218, 2024-05-10T11:44:46.773 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000027520 2024-05-10T11:44:46.773 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:46.773 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-05-10T11:44:46.773 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 1479292, 2024-05-10T11:44:46.773 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.223578008, 2024-05-10T11:44:46.773 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000151 2024-05-10T11:44:46.773 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:46.773 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 710096, 2024-05-10T11:44:46.774 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 335984 2024-05-10T11:44:46.774 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:46.774 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T11:44:46.774 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 67129, 2024-05-10T11:44:46.774 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 87187, 2024-05-10T11:44:46.774 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 15222466, 2024-05-10T11:44:46.774 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 102529049, 2024-05-10T11:44:46.774 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 9, 2024-05-10T11:44:46.774 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 6, 2024-05-10T11:44:46.774 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 5.647341978, 2024-05-10T11:44:46.774 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 2.405532895, 2024-05-10T11:44:46.774 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 1.929957399, 2024-05-10T11:44:46.774 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 1.285075379, 2024-05-10T11:44:46.774 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T11:44:46.775 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 87185, 2024-05-10T11:44:46.775 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 4.292811381, 2024-05-10T11:44:46.775 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000049237 2024-05-10T11:44:46.775 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:46.775 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-05-10T11:44:46.775 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 86575, 2024-05-10T11:44:46.775 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.013894018, 2024-05-10T11:44:46.775 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000160 2024-05-10T11:44:46.775 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:46.775 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-10T11:44:46.775 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 810688 2024-05-10T11:44:46.775 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:46.775 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T11:44:46.775 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 190952, 2024-05-10T11:44:46.776 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 190730, 2024-05-10T11:44:46.776 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 52017320, 2024-05-10T11:44:46.776 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 160348471, 2024-05-10T11:44:46.776 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 15, 2024-05-10T11:44:46.776 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 5, 2024-05-10T11:44:46.776 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 12.418441063, 2024-05-10T11:44:46.776 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 5.606321717, 2024-05-10T11:44:46.776 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 6.634180959, 2024-05-10T11:44:46.776 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 1.371260351, 2024-05-10T11:44:46.776 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T11:44:46.776 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 190729, 2024-05-10T11:44:46.776 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 6.635220891, 2024-05-10T11:44:46.776 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000034788 2024-05-10T11:44:46.776 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:46.776 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-05-10T11:44:46.777 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 300081, 2024-05-10T11:44:46.777 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.048498729, 2024-05-10T11:44:46.777 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000161 2024-05-10T11:44:46.777 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:46.777 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 2912, 2024-05-10T11:44:46.777 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 448 2024-05-10T11:44:46.777 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:46.777 INFO:teuthology.orchestra.run.smithi188.stdout: "cct": { 2024-05-10T11:44:46.777 INFO:teuthology.orchestra.run.smithi188.stdout: "total_workers": 1, 2024-05-10T11:44:46.777 INFO:teuthology.orchestra.run.smithi188.stdout: "unhealthy_workers": 0 2024-05-10T11:44:46.777 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:46.777 INFO:teuthology.orchestra.run.smithi188.stdout: "finisher-MDSRank": { 2024-05-10T11:44:46.777 INFO:teuthology.orchestra.run.smithi188.stdout: "queue_len": 0, 2024-05-10T11:44:46.777 INFO:teuthology.orchestra.run.smithi188.stdout: "complete_latency": { 2024-05-10T11:44:46.778 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 82977, 2024-05-10T11:44:46.778 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 87.492849225, 2024-05-10T11:44:46.778 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.001054422 2024-05-10T11:44:46.778 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-10T11:44:46.778 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:46.778 INFO:teuthology.orchestra.run.smithi188.stdout: "finisher-PurgeQueue": { 2024-05-10T11:44:46.778 INFO:teuthology.orchestra.run.smithi188.stdout: "queue_len": 0, 2024-05-10T11:44:46.778 INFO:teuthology.orchestra.run.smithi188.stdout: "complete_latency": { 2024-05-10T11:44:46.778 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 41390, 2024-05-10T11:44:46.778 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 5.186502671, 2024-05-10T11:44:46.778 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000125308 2024-05-10T11:44:46.778 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-10T11:44:46.778 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:46.779 INFO:teuthology.orchestra.run.smithi188.stdout: "mds": { 2024-05-10T11:44:46.779 INFO:teuthology.orchestra.run.smithi188.stdout: "request": 635330, 2024-05-10T11:44:46.779 INFO:teuthology.orchestra.run.smithi188.stdout: "reply": 633282, 2024-05-10T11:44:46.779 INFO:teuthology.orchestra.run.smithi188.stdout: "reply_latency": { 2024-05-10T11:44:46.779 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 633282, 2024-05-10T11:44:46.779 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 641.588688025, 2024-05-10T11:44:46.779 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.001013116 2024-05-10T11:44:46.779 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:46.779 INFO:teuthology.orchestra.run.smithi188.stdout: "slow_reply": 0, 2024-05-10T11:44:46.779 INFO:teuthology.orchestra.run.smithi188.stdout: "forward": 2048, 2024-05-10T11:44:46.779 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_fetch_complete": 0, 2024-05-10T11:44:46.779 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_fetch_keys": 0, 2024-05-10T11:44:46.779 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_commit": 14, 2024-05-10T11:44:46.779 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_split": 0, 2024-05-10T11:44:46.779 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_merge": 0, 2024-05-10T11:44:46.780 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes": 14, 2024-05-10T11:44:46.780 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_top": 1, 2024-05-10T11:44:46.780 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_bottom": 1, 2024-05-10T11:44:46.780 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_pin_tail": 12, 2024-05-10T11:44:46.780 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_pinned": 12, 2024-05-10T11:44:46.780 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_expired": 79, 2024-05-10T11:44:46.780 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_with_caps": 0, 2024-05-10T11:44:46.780 INFO:teuthology.orchestra.run.smithi188.stdout: "caps": 0, 2024-05-10T11:44:46.780 INFO:teuthology.orchestra.run.smithi188.stdout: "subtrees": 3, 2024-05-10T11:44:46.780 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse": 1259978, 2024-05-10T11:44:46.780 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_hit": 1173557, 2024-05-10T11:44:46.780 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_forward": 639, 2024-05-10T11:44:46.780 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_discover": 36, 2024-05-10T11:44:46.780 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_dir_fetch": 0, 2024-05-10T11:44:46.780 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_remote_ino": 0, 2024-05-10T11:44:46.781 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_lock": 77, 2024-05-10T11:44:46.781 INFO:teuthology.orchestra.run.smithi188.stdout: "load_cent": 0, 2024-05-10T11:44:46.781 INFO:teuthology.orchestra.run.smithi188.stdout: "q": 0, 2024-05-10T11:44:46.781 INFO:teuthology.orchestra.run.smithi188.stdout: "exported": 21, 2024-05-10T11:44:46.781 INFO:teuthology.orchestra.run.smithi188.stdout: "exported_inodes": 8, 2024-05-10T11:44:46.781 INFO:teuthology.orchestra.run.smithi188.stdout: "imported": 24, 2024-05-10T11:44:46.781 INFO:teuthology.orchestra.run.smithi188.stdout: "imported_inodes": 535, 2024-05-10T11:44:46.781 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_dir_fetch": 0, 2024-05-10T11:44:46.781 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_backtrace_fetch": 3830, 2024-05-10T11:44:46.781 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_peer_discover": 0, 2024-05-10T11:44:46.781 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rfiles": 61, 2024-05-10T11:44:46.781 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rbytes": 12057077, 2024-05-10T11:44:46.781 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rsnaps": 0, 2024-05-10T11:44:46.781 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_backtrace_fetch": 3830, 2024-05-10T11:44:46.781 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_set_tag": 0, 2024-05-10T11:44:46.782 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_backtrace_repaired": 0, 2024-05-10T11:44:46.782 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_inotable_repaired": 0, 2024-05-10T11:44:46.782 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dir_inodes": 330, 2024-05-10T11:44:46.782 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dir_base_inodes": 0, 2024-05-10T11:44:46.782 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dirfrag_rstats": 330, 2024-05-10T11:44:46.782 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_file_inodes": 3500, 2024-05-10T11:44:46.782 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_inode_file_caps": 1102, 2024-05-10T11:44:46.782 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_revoke": 42672, 2024-05-10T11:44:46.782 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_grant": 74282, 2024-05-10T11:44:46.782 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_trunc": 0, 2024-05-10T11:44:46.782 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-10T11:44:46.782 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-10T11:44:46.782 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_caps": 72188, 2024-05-10T11:44:46.783 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_caps_dirty": 21991, 2024-05-10T11:44:46.783 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_cap_release": 1116, 2024-05-10T11:44:46.783 INFO:teuthology.orchestra.run.smithi188.stdout: "process_request_cap_release": 78231 2024-05-10T11:44:46.783 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:46.783 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_cache": { 2024-05-10T11:44:46.783 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_update": 2, 2024-05-10T11:44:46.783 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_update_receipt": 63, 2024-05-10T11:44:46.783 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_try_discover": 40, 2024-05-10T11:44:46.783 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_send_discover": 80, 2024-05-10T11:44:46.783 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_handle_discover": 32, 2024-05-10T11:44:46.783 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays": 0, 2024-05-10T11:44:46.783 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays_delayed": 0, 2024-05-10T11:44:46.783 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays_enqueuing": 0, 2024-05-10T11:44:46.783 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_created": 21990, 2024-05-10T11:44:46.784 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_enqueued": 21990, 2024-05-10T11:44:46.784 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_reintegrated": 0, 2024-05-10T11:44:46.784 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_migrated": 0, 2024-05-10T11:44:46.784 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_processing": 0, 2024-05-10T11:44:46.784 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_enqueued": 0, 2024-05-10T11:44:46.784 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_prioritized": 0, 2024-05-10T11:44:46.784 INFO:teuthology.orchestra.run.smithi188.stdout: "recovery_started": 0, 2024-05-10T11:44:46.784 INFO:teuthology.orchestra.run.smithi188.stdout: "recovery_completed": 0, 2024-05-10T11:44:46.784 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_quiesce_path": 0, 2024-05-10T11:44:46.784 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_quiesce_inode": 0, 2024-05-10T11:44:46.784 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_enqueue_scrub": 0, 2024-05-10T11:44:46.784 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_exportdir": 22, 2024-05-10T11:44:46.784 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_flush": 0, 2024-05-10T11:44:46.784 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_fragmentdir": 0, 2024-05-10T11:44:46.784 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_fragstats": 0, 2024-05-10T11:44:46.784 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_inodestats": 0 2024-05-10T11:44:46.785 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:46.785 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_log": { 2024-05-10T11:44:46.785 INFO:teuthology.orchestra.run.smithi188.stdout: "evlrg": 0, 2024-05-10T11:44:46.785 INFO:teuthology.orchestra.run.smithi188.stdout: "evadd": 156483, 2024-05-10T11:44:46.785 INFO:teuthology.orchestra.run.smithi188.stdout: "evex": 49161, 2024-05-10T11:44:46.785 INFO:teuthology.orchestra.run.smithi188.stdout: "evtrm": 49161, 2024-05-10T11:44:46.785 INFO:teuthology.orchestra.run.smithi188.stdout: "ev": 107322, 2024-05-10T11:44:46.785 INFO:teuthology.orchestra.run.smithi188.stdout: "evexg": 0, 2024-05-10T11:44:46.785 INFO:teuthology.orchestra.run.smithi188.stdout: "evexd": 12290, 2024-05-10T11:44:46.785 INFO:teuthology.orchestra.run.smithi188.stdout: "segadd": 186, 2024-05-10T11:44:46.785 INFO:teuthology.orchestra.run.smithi188.stdout: "segex": 58, 2024-05-10T11:44:46.785 INFO:teuthology.orchestra.run.smithi188.stdout: "segtrm": 58, 2024-05-10T11:44:46.785 INFO:teuthology.orchestra.run.smithi188.stdout: "seg": 128, 2024-05-10T11:44:46.785 INFO:teuthology.orchestra.run.smithi188.stdout: "segmjr": 9, 2024-05-10T11:44:46.785 INFO:teuthology.orchestra.run.smithi188.stdout: "segexg": 0, 2024-05-10T11:44:46.786 INFO:teuthology.orchestra.run.smithi188.stdout: "segexd": 15, 2024-05-10T11:44:46.786 INFO:teuthology.orchestra.run.smithi188.stdout: "expos": 74596632, 2024-05-10T11:44:46.786 INFO:teuthology.orchestra.run.smithi188.stdout: "wrpos": 243289507, 2024-05-10T11:44:46.786 INFO:teuthology.orchestra.run.smithi188.stdout: "rdpos": 0, 2024-05-10T11:44:46.786 INFO:teuthology.orchestra.run.smithi188.stdout: "jlat": { 2024-05-10T11:44:46.786 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 41968, 2024-05-10T11:44:46.786 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 163.851787681, 2024-05-10T11:44:46.786 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.003904207 2024-05-10T11:44:46.786 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:46.786 INFO:teuthology.orchestra.run.smithi188.stdout: "replayed": 0 2024-05-10T11:44:46.786 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:46.786 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_mem": { 2024-05-10T11:44:46.786 INFO:teuthology.orchestra.run.smithi188.stdout: "ino": 19, 2024-05-10T11:44:46.786 INFO:teuthology.orchestra.run.smithi188.stdout: "ino+": 22049, 2024-05-10T11:44:46.787 INFO:teuthology.orchestra.run.smithi188.stdout: "ino-": 22030, 2024-05-10T11:44:46.787 INFO:teuthology.orchestra.run.smithi188.stdout: "dir": 17, 2024-05-10T11:44:46.787 INFO:teuthology.orchestra.run.smithi188.stdout: "dir+": 101, 2024-05-10T11:44:46.787 INFO:teuthology.orchestra.run.smithi188.stdout: "dir-": 84, 2024-05-10T11:44:46.787 INFO:teuthology.orchestra.run.smithi188.stdout: "dn": 14, 2024-05-10T11:44:46.787 INFO:teuthology.orchestra.run.smithi188.stdout: "dn+": 30590, 2024-05-10T11:44:46.787 INFO:teuthology.orchestra.run.smithi188.stdout: "dn-": 30576, 2024-05-10T11:44:46.787 INFO:teuthology.orchestra.run.smithi188.stdout: "cap": 0, 2024-05-10T11:44:46.787 INFO:teuthology.orchestra.run.smithi188.stdout: "cap+": 22011, 2024-05-10T11:44:46.787 INFO:teuthology.orchestra.run.smithi188.stdout: "cap-": 22011, 2024-05-10T11:44:46.787 INFO:teuthology.orchestra.run.smithi188.stdout: "rss": 81148, 2024-05-10T11:44:46.787 INFO:teuthology.orchestra.run.smithi188.stdout: "heap": 223608 2024-05-10T11:44:46.787 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:46.787 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_server": { 2024-05-10T11:44:46.787 INFO:teuthology.orchestra.run.smithi188.stdout: "dispatch_client_request": 732971, 2024-05-10T11:44:46.787 INFO:teuthology.orchestra.run.smithi188.stdout: "dispatch_server_request": 2387, 2024-05-10T11:44:46.788 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_request": 635330, 2024-05-10T11:44:46.788 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_session": 1580, 2024-05-10T11:44:46.788 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_peer_request": 27084, 2024-05-10T11:44:46.788 INFO:teuthology.orchestra.run.smithi188.stdout: "req_create_latency": { 2024-05-10T11:44:46.788 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 21684, 2024-05-10T11:44:46.788 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 70.834163716, 2024-05-10T11:44:46.788 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.003266655 2024-05-10T11:44:46.788 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:46.788 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getattr_latency": { 2024-05-10T11:44:46.788 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 525864, 2024-05-10T11:44:46.788 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 319.540409607, 2024-05-10T11:44:46.788 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000607648 2024-05-10T11:44:46.788 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:46.788 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getfilelock_latency": { 2024-05-10T11:44:46.788 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:46.789 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:46.789 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:46.789 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:46.789 INFO:teuthology.orchestra.run.smithi188.stdout: "req_link_latency": { 2024-05-10T11:44:46.789 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:46.789 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:46.789 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:46.789 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:46.789 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookup_latency": { 2024-05-10T11:44:46.789 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 17900, 2024-05-10T11:44:46.789 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 39.099968509, 2024-05-10T11:44:46.789 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.002184355 2024-05-10T11:44:46.789 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:46.789 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookuphash_latency": { 2024-05-10T11:44:46.789 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:46.790 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:46.790 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:46.790 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:46.790 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupino_latency": { 2024-05-10T11:44:46.790 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:46.790 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:46.790 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:46.790 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:46.790 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupname_latency": { 2024-05-10T11:44:46.790 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:46.790 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:46.790 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:46.790 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:46.790 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupparent_latency": { 2024-05-10T11:44:46.791 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:46.791 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:46.791 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:46.791 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:46.791 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupsnap_latency": { 2024-05-10T11:44:46.791 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:46.791 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:46.791 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:46.792 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:46.792 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lssnap_latency": { 2024-05-10T11:44:46.792 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:46.792 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:46.792 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:46.792 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:46.792 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mkdir_latency": { 2024-05-10T11:44:46.792 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:46.792 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:46.792 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:46.792 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:46.792 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mknod_latency": { 2024-05-10T11:44:46.792 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:46.792 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:46.792 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:46.793 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:46.793 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mksnap_latency": { 2024-05-10T11:44:46.793 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:46.793 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:46.793 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:46.793 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:46.793 INFO:teuthology.orchestra.run.smithi188.stdout: "req_open_latency": { 2024-05-10T11:44:46.793 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 12584, 2024-05-10T11:44:46.793 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 13.366939606, 2024-05-10T11:44:46.793 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.001062217 2024-05-10T11:44:46.793 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:46.793 INFO:teuthology.orchestra.run.smithi188.stdout: "req_readdir_latency": { 2024-05-10T11:44:46.793 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 28105, 2024-05-10T11:44:46.793 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 86.448647625, 2024-05-10T11:44:46.793 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.003075917 2024-05-10T11:44:46.794 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:46.794 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rename_latency": { 2024-05-10T11:44:46.794 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 4156, 2024-05-10T11:44:46.794 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 14.734384041, 2024-05-10T11:44:46.794 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.003545328 2024-05-10T11:44:46.794 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:46.794 INFO:teuthology.orchestra.run.smithi188.stdout: "req_renamesnap_latency": { 2024-05-10T11:44:46.794 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:46.794 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:46.794 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:46.794 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:46.794 INFO:teuthology.orchestra.run.smithi188.stdout: "req_snapdiff_latency": { 2024-05-10T11:44:46.794 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:46.794 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:46.795 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:46.795 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:46.795 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmdir_latency": { 2024-05-10T11:44:46.795 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 30, 2024-05-10T11:44:46.795 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.401630482, 2024-05-10T11:44:46.795 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.013387682 2024-05-10T11:44:46.795 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:46.795 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmsnap_latency": { 2024-05-10T11:44:46.795 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:46.795 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:46.795 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:46.795 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:46.795 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmxattr_latency": { 2024-05-10T11:44:46.795 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:46.796 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:46.796 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:46.796 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:46.796 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setattr_latency": { 2024-05-10T11:44:46.796 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 99, 2024-05-10T11:44:46.796 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.543910477, 2024-05-10T11:44:46.796 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.005494045 2024-05-10T11:44:46.796 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:46.796 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setdirlayout_latency": { 2024-05-10T11:44:46.796 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:46.796 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:46.796 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:46.796 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:46.796 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setfilelock_latency": { 2024-05-10T11:44:46.796 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 900, 2024-05-10T11:44:46.797 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.565128175, 2024-05-10T11:44:46.797 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000627920 2024-05-10T11:44:46.797 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:46.797 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setlayout_latency": { 2024-05-10T11:44:46.797 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:46.797 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:46.797 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:46.797 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:46.797 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setxattr_latency": { 2024-05-10T11:44:46.797 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:46.797 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:46.797 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:46.797 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:46.797 INFO:teuthology.orchestra.run.smithi188.stdout: "req_symlink_latency": { 2024-05-10T11:44:46.797 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:46.798 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:46.798 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:46.798 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:46.798 INFO:teuthology.orchestra.run.smithi188.stdout: "req_unlink_latency": { 2024-05-10T11:44:46.798 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 21960, 2024-05-10T11:44:46.798 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 96.053505787, 2024-05-10T11:44:46.798 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.004374021 2024-05-10T11:44:46.798 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:46.798 INFO:teuthology.orchestra.run.smithi188.stdout: "cap_revoke_eviction": 0, 2024-05-10T11:44:46.798 INFO:teuthology.orchestra.run.smithi188.stdout: "cap_acquisition_throttle": 0, 2024-05-10T11:44:46.798 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getvxattr_latency": { 2024-05-10T11:44:46.798 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:46.798 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:46.798 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:46.799 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-10T11:44:46.799 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:46.799 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_sessions": { 2024-05-10T11:44:46.799 INFO:teuthology.orchestra.run.smithi188.stdout: "session_count": 1, 2024-05-10T11:44:46.799 INFO:teuthology.orchestra.run.smithi188.stdout: "session_add": 2, 2024-05-10T11:44:46.799 INFO:teuthology.orchestra.run.smithi188.stdout: "session_remove": 1, 2024-05-10T11:44:46.799 INFO:teuthology.orchestra.run.smithi188.stdout: "sessions_open": 1, 2024-05-10T11:44:46.799 INFO:teuthology.orchestra.run.smithi188.stdout: "sessions_stale": 0, 2024-05-10T11:44:46.799 INFO:teuthology.orchestra.run.smithi188.stdout: "total_load": 72816, 2024-05-10T11:44:46.799 INFO:teuthology.orchestra.run.smithi188.stdout: "average_load": 72816, 2024-05-10T11:44:46.799 INFO:teuthology.orchestra.run.smithi188.stdout: "avg_session_uptime": 1581, 2024-05-10T11:44:46.799 INFO:teuthology.orchestra.run.smithi188.stdout: "mdthresh_evicted": 0 2024-05-10T11:44:46.799 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:46.799 INFO:teuthology.orchestra.run.smithi188.stdout: "mempool": { 2024-05-10T11:44:46.799 INFO:teuthology.orchestra.run.smithi188.stdout: "bloom_filter_bytes": 0, 2024-05-10T11:44:46.800 INFO:teuthology.orchestra.run.smithi188.stdout: "bloom_filter_items": 0, 2024-05-10T11:44:46.800 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T11:44:46.800 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_alloc_items": 0, 2024-05-10T11:44:46.800 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T11:44:46.800 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_data_items": 0, 2024-05-10T11:44:46.800 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T11:44:46.800 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T11:44:46.800 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T11:44:46.800 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T11:44:46.800 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T11:44:46.800 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_other_items": 0, 2024-05-10T11:44:46.800 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T11:44:46.800 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T11:44:46.800 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_extent_bytes": 0, 2024-05-10T11:44:46.800 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_extent_items": 0, 2024-05-10T11:44:46.801 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_blob_bytes": 0, 2024-05-10T11:44:46.801 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_blob_items": 0, 2024-05-10T11:44:46.801 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T11:44:46.801 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T11:44:46.801 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T11:44:46.801 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T11:44:46.801 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T11:44:46.801 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_fsck_items": 0, 2024-05-10T11:44:46.801 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_txc_bytes": 0, 2024-05-10T11:44:46.801 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_txc_items": 0, 2024-05-10T11:44:46.801 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T11:44:46.801 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T11:44:46.801 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_bytes": 0, 2024-05-10T11:44:46.801 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_items": 0, 2024-05-10T11:44:46.801 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_bytes": 0, 2024-05-10T11:44:46.801 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_items": 0, 2024-05-10T11:44:46.802 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T11:44:46.802 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_reader_items": 0, 2024-05-10T11:44:46.802 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T11:44:46.802 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_writer_items": 0, 2024-05-10T11:44:46.802 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_anon_bytes": 297580, 2024-05-10T11:44:46.802 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_anon_items": 105, 2024-05-10T11:44:46.802 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_meta_bytes": 0, 2024-05-10T11:44:46.802 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_meta_items": 0, 2024-05-10T11:44:46.802 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_bytes": 0, 2024-05-10T11:44:46.802 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_items": 0, 2024-05-10T11:44:46.802 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_mapbl_bytes": 0, 2024-05-10T11:44:46.802 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_mapbl_items": 0, 2024-05-10T11:44:46.802 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_pglog_bytes": 0, 2024-05-10T11:44:46.802 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_pglog_items": 0, 2024-05-10T11:44:46.802 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_bytes": 8696, 2024-05-10T11:44:46.803 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_items": 166, 2024-05-10T11:44:46.803 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T11:44:46.803 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_mapping_items": 0, 2024-05-10T11:44:46.803 INFO:teuthology.orchestra.run.smithi188.stdout: "pgmap_bytes": 0, 2024-05-10T11:44:46.803 INFO:teuthology.orchestra.run.smithi188.stdout: "pgmap_items": 0, 2024-05-10T11:44:46.803 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_co_bytes": 76120, 2024-05-10T11:44:46.803 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_co_items": 368, 2024-05-10T11:44:46.803 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_1_bytes": 0, 2024-05-10T11:44:46.803 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_1_items": 0, 2024-05-10T11:44:46.803 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_2_bytes": 0, 2024-05-10T11:44:46.803 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_2_items": 0 2024-05-10T11:44:46.803 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:46.803 INFO:teuthology.orchestra.run.smithi188.stdout: "objecter": { 2024-05-10T11:44:46.803 INFO:teuthology.orchestra.run.smithi188.stdout: "op_active": 0, 2024-05-10T11:44:46.803 INFO:teuthology.orchestra.run.smithi188.stdout: "op_laggy": 0, 2024-05-10T11:44:46.803 INFO:teuthology.orchestra.run.smithi188.stdout: "op_send": 96487, 2024-05-10T11:44:46.804 INFO:teuthology.orchestra.run.smithi188.stdout: "op_send_bytes": 242927957, 2024-05-10T11:44:46.804 INFO:teuthology.orchestra.run.smithi188.stdout: "op_resend": 29, 2024-05-10T11:44:46.804 INFO:teuthology.orchestra.run.smithi188.stdout: "op_reply": 96487, 2024-05-10T11:44:46.804 INFO:teuthology.orchestra.run.smithi188.stdout: "op_latency": { 2024-05-10T11:44:46.804 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 96487, 2024-05-10T11:44:46.804 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 494.909143372, 2024-05-10T11:44:46.804 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.005129283 2024-05-10T11:44:46.804 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:46.804 INFO:teuthology.orchestra.run.smithi188.stdout: "op_inflight": 0, 2024-05-10T11:44:46.804 INFO:teuthology.orchestra.run.smithi188.stdout: "oplen_avg": { 2024-05-10T11:44:46.804 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 96487, 2024-05-10T11:44:46.804 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 98766 2024-05-10T11:44:46.804 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:46.804 INFO:teuthology.orchestra.run.smithi188.stdout: "op": 96487, 2024-05-10T11:44:46.805 INFO:teuthology.orchestra.run.smithi188.stdout: "op_r": 5907, 2024-05-10T11:44:46.805 INFO:teuthology.orchestra.run.smithi188.stdout: "op_w": 90580, 2024-05-10T11:44:46.805 INFO:teuthology.orchestra.run.smithi188.stdout: "op_rmw": 0, 2024-05-10T11:44:46.805 INFO:teuthology.orchestra.run.smithi188.stdout: "op_pg": 0, 2024-05-10T11:44:46.805 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_stat": 3, 2024-05-10T11:44:46.805 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_create": 29, 2024-05-10T11:44:46.805 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_read": 2077, 2024-05-10T11:44:46.805 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_write": 44045, 2024-05-10T11:44:46.805 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_writefull": 1189, 2024-05-10T11:44:46.805 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_writesame": 0, 2024-05-10T11:44:46.805 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_append": 0, 2024-05-10T11:44:46.805 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_zero": 0, 2024-05-10T11:44:46.805 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_truncate": 0, 2024-05-10T11:44:46.805 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_delete": 44035, 2024-05-10T11:44:46.805 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_mapext": 0, 2024-05-10T11:44:46.806 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_sparse_read": 0, 2024-05-10T11:44:46.806 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_clonerange": 0, 2024-05-10T11:44:46.806 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_getxattr": 3830, 2024-05-10T11:44:46.806 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_setxattr": 54, 2024-05-10T11:44:46.806 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_cmpxattr": 0, 2024-05-10T11:44:46.806 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_rmxattr": 0, 2024-05-10T11:44:46.806 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_resetxattrs": 0, 2024-05-10T11:44:46.806 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_call": 0, 2024-05-10T11:44:46.806 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_watch": 0, 2024-05-10T11:44:46.806 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_notify": 0, 2024-05-10T11:44:46.806 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_src_cmpxattr": 0, 2024-05-10T11:44:46.806 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_pgls": 0, 2024-05-10T11:44:46.806 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_pgls_filter": 0, 2024-05-10T11:44:46.806 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_other": 0, 2024-05-10T11:44:46.807 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_active": 0, 2024-05-10T11:44:46.807 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_send": 0, 2024-05-10T11:44:46.807 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_resend": 0, 2024-05-10T11:44:46.807 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_ping": 0, 2024-05-10T11:44:46.807 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_active": 0, 2024-05-10T11:44:46.807 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_send": 0, 2024-05-10T11:44:46.807 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_resend": 0, 2024-05-10T11:44:46.807 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_active": 0, 2024-05-10T11:44:46.807 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_send": 0, 2024-05-10T11:44:46.807 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_resend": 0, 2024-05-10T11:44:46.807 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_active": 0, 2024-05-10T11:44:46.807 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_send": 0, 2024-05-10T11:44:46.807 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_resend": 0, 2024-05-10T11:44:46.807 INFO:teuthology.orchestra.run.smithi188.stdout: "command_active": 0, 2024-05-10T11:44:46.807 INFO:teuthology.orchestra.run.smithi188.stdout: "command_send": 0, 2024-05-10T11:44:46.807 INFO:teuthology.orchestra.run.smithi188.stdout: "command_resend": 0, 2024-05-10T11:44:46.808 INFO:teuthology.orchestra.run.smithi188.stdout: "map_epoch": 90, 2024-05-10T11:44:46.808 INFO:teuthology.orchestra.run.smithi188.stdout: "map_full": 0, 2024-05-10T11:44:46.808 INFO:teuthology.orchestra.run.smithi188.stdout: "map_inc": 1, 2024-05-10T11:44:46.808 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_sessions": 12, 2024-05-10T11:44:46.808 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_session_open": 12, 2024-05-10T11:44:46.808 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_session_close": 0, 2024-05-10T11:44:46.808 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_laggy": 0, 2024-05-10T11:44:46.808 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_wr": 2335, 2024-05-10T11:44:46.808 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_rd": 0, 2024-05-10T11:44:46.808 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_del": 1169 2024-05-10T11:44:46.808 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:46.808 INFO:teuthology.orchestra.run.smithi188.stdout: "oft": { 2024-05-10T11:44:46.808 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_objs": 1, 2024-05-10T11:44:46.808 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_kv_pairs": 0, 2024-05-10T11:44:46.809 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_updates": 0, 2024-05-10T11:44:46.809 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_removes": 0 2024-05-10T11:44:46.809 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:46.809 INFO:teuthology.orchestra.run.smithi188.stdout: "osd-slow-ops": { 2024-05-10T11:44:46.809 INFO:teuthology.orchestra.run.smithi188.stdout: "slow_ops_count": 0 2024-05-10T11:44:46.809 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:46.809 INFO:teuthology.orchestra.run.smithi188.stdout: "purge_queue": { 2024-05-10T11:44:46.809 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_ops": 0, 2024-05-10T11:44:46.809 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_ops_high_water": 16, 2024-05-10T11:44:46.809 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing": 0, 2024-05-10T11:44:46.809 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_high_water": 16, 2024-05-10T11:44:46.809 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executed_ops": 43950, 2024-05-10T11:44:46.809 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executed": 21990, 2024-05-10T11:44:46.809 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_item_in_journal": 0 2024-05-10T11:44:46.809 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:46.810 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T11:44:46.810 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-05-10T11:44:46.810 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 104857600, 2024-05-10T11:44:46.810 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-05-10T11:44:46.810 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 993700, 2024-05-10T11:44:46.810 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 199891994, 2024-05-10T11:44:46.810 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-05-10T11:44:46.810 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 993700, 2024-05-10T11:44:46.810 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-05-10T11:44:46.810 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-05-10T11:44:46.810 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 993700, 2024-05-10T11:44:46.810 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 199891994, 2024-05-10T11:44:46.810 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-05-10T11:44:46.810 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:46.810 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:46.811 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:46.811 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-10T11:44:46.811 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:46.811 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-objecter_bytes": { 2024-05-10T11:44:46.811 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-05-10T11:44:46.811 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 104857600, 2024-05-10T11:44:46.811 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-05-10T11:44:46.811 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-05-10T11:44:46.811 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-05-10T11:44:46.811 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-05-10T11:44:46.811 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-05-10T11:44:46.811 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 96487, 2024-05-10T11:44:46.811 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 245324627, 2024-05-10T11:44:46.811 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 96487, 2024-05-10T11:44:46.812 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 245324627, 2024-05-10T11:44:46.812 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-05-10T11:44:46.812 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:46.812 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:46.812 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:46.812 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-10T11:44:46.812 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:46.812 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-objecter_ops": { 2024-05-10T11:44:46.812 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-05-10T11:44:46.812 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 1024, 2024-05-10T11:44:46.812 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-05-10T11:44:46.812 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-05-10T11:44:46.812 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-05-10T11:44:46.812 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-05-10T11:44:46.812 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-05-10T11:44:46.813 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 96487, 2024-05-10T11:44:46.813 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 96487, 2024-05-10T11:44:46.813 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 96487, 2024-05-10T11:44:46.813 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 96487, 2024-05-10T11:44:46.813 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-05-10T11:44:46.813 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:46.813 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:46.813 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:46.813 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-10T11:44:46.813 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:46.813 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-write_buf_throttle": { 2024-05-10T11:44:46.813 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-05-10T11:44:46.813 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 3758096384, 2024-05-10T11:44:46.813 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-05-10T11:44:46.814 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 21990, 2024-05-10T11:44:46.814 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 2396670, 2024-05-10T11:44:46.814 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-05-10T11:44:46.814 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 21990, 2024-05-10T11:44:46.814 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-05-10T11:44:46.814 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-05-10T11:44:46.814 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 2077, 2024-05-10T11:44:46.814 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 2396670, 2024-05-10T11:44:46.814 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-05-10T11:44:46.814 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:46.814 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:46.815 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:46.815 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-10T11:44:46.815 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:46.815 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-write_buf_throttle-0x55a89bf16500": { 2024-05-10T11:44:46.815 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-05-10T11:44:46.815 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 3758096384, 2024-05-10T11:44:46.815 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-05-10T11:44:46.815 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 156483, 2024-05-10T11:44:46.815 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 239095203, 2024-05-10T11:44:46.815 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-05-10T11:44:46.815 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 156483, 2024-05-10T11:44:46.816 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-05-10T11:44:46.816 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-05-10T11:44:46.816 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 41968, 2024-05-10T11:44:46.816 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 239095203, 2024-05-10T11:44:46.816 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-05-10T11:44:46.816 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:46.816 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:46.816 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:46.816 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-10T11:44:46.816 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-10T11:44:46.816 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-10T11:44:47.558 INFO:tasks.check_counter:Daemon mds.c mds.exported=21 2024-05-10T11:44:47.559 INFO:tasks.check_counter:Daemon mds.c mds.imported=24 2024-05-10T11:44:47.559 DEBUG:tasks.check_counter:Getting stats from f 2024-05-10T11:44:47.559 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph daemon mds.f perf dump 2024-05-10T11:44:47.803 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T11:44:48.193 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:44:48 smithi188 ceph-mon[34075]: pgmap v1213: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-10T11:44:48.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:44:48 smithi178 ceph-mon[33359]: pgmap v1213: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-10T11:44:48.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:44:48 smithi100 ceph-mon[30296]: pgmap v1213: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-10T11:44:49.555 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-10T11:44:49.555 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T11:44:49.555 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 2020, 2024-05-10T11:44:49.555 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 2020, 2024-05-10T11:44:49.555 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 63416905, 2024-05-10T11:44:49.555 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 641546, 2024-05-10T11:44:49.556 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 8, 2024-05-10T11:44:49.556 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 4, 2024-05-10T11:44:49.556 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 0.172458431, 2024-05-10T11:44:49.556 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 0.038290972, 2024-05-10T11:44:49.556 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 0.075717298, 2024-05-10T11:44:49.556 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.047004343, 2024-05-10T11:44:49.556 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T11:44:49.556 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 2019, 2024-05-10T11:44:49.556 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.061635915, 2024-05-10T11:44:49.556 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000030527 2024-05-10T11:44:49.556 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:49.556 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-05-10T11:44:49.556 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:49.556 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:49.557 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:49.557 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:49.557 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 2912, 2024-05-10T11:44:49.557 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 448 2024-05-10T11:44:49.557 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:49.557 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T11:44:49.557 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 1188, 2024-05-10T11:44:49.557 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 1176, 2024-05-10T11:44:49.557 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 59240473, 2024-05-10T11:44:49.557 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 448420, 2024-05-10T11:44:49.557 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 6, 2024-05-10T11:44:49.557 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 5, 2024-05-10T11:44:49.557 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 0.118546555, 2024-05-10T11:44:49.557 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 0.028474665, 2024-05-10T11:44:49.557 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 0.059849189, 2024-05-10T11:44:49.558 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.025069269, 2024-05-10T11:44:49.558 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T11:44:49.558 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 1175, 2024-05-10T11:44:49.558 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.047104221, 2024-05-10T11:44:49.558 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000040088 2024-05-10T11:44:49.558 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:49.558 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-05-10T11:44:49.558 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:49.558 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:49.558 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:49.558 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:49.558 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 286464, 2024-05-10T11:44:49.558 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 156112 2024-05-10T11:44:49.558 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:49.559 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T11:44:49.559 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 802, 2024-05-10T11:44:49.559 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 993, 2024-05-10T11:44:49.559 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 35710710, 2024-05-10T11:44:49.559 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 681663, 2024-05-10T11:44:49.559 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 6, 2024-05-10T11:44:49.559 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 5, 2024-05-10T11:44:49.559 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 0.074583813, 2024-05-10T11:44:49.559 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 0.022837388, 2024-05-10T11:44:49.559 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 0.030433427, 2024-05-10T11:44:49.559 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.016169462, 2024-05-10T11:44:49.559 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T11:44:49.559 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 991, 2024-05-10T11:44:49.559 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.027376404, 2024-05-10T11:44:49.559 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000027625 2024-05-10T11:44:49.560 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:49.560 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-05-10T11:44:49.560 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:49.560 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:49.560 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:49.560 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:49.560 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-10T11:44:49.560 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 426944 2024-05-10T11:44:49.560 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:49.560 INFO:teuthology.orchestra.run.smithi188.stdout: "cct": { 2024-05-10T11:44:49.560 INFO:teuthology.orchestra.run.smithi188.stdout: "total_workers": 1, 2024-05-10T11:44:49.560 INFO:teuthology.orchestra.run.smithi188.stdout: "unhealthy_workers": 0 2024-05-10T11:44:49.560 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:49.560 INFO:teuthology.orchestra.run.smithi188.stdout: "finisher-MDSRank": { 2024-05-10T11:44:49.560 INFO:teuthology.orchestra.run.smithi188.stdout: "queue_len": 0, 2024-05-10T11:44:49.561 INFO:teuthology.orchestra.run.smithi188.stdout: "complete_latency": { 2024-05-10T11:44:49.561 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 6430, 2024-05-10T11:44:49.561 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.259019147, 2024-05-10T11:44:49.561 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000040282 2024-05-10T11:44:49.561 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-10T11:44:49.561 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:49.561 INFO:teuthology.orchestra.run.smithi188.stdout: "finisher-PurgeQueue": { 2024-05-10T11:44:49.561 INFO:teuthology.orchestra.run.smithi188.stdout: "queue_len": 0, 2024-05-10T11:44:49.561 INFO:teuthology.orchestra.run.smithi188.stdout: "complete_latency": { 2024-05-10T11:44:49.561 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:49.561 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:49.561 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:49.561 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-10T11:44:49.561 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:49.562 INFO:teuthology.orchestra.run.smithi188.stdout: "mds": { 2024-05-10T11:44:49.562 INFO:teuthology.orchestra.run.smithi188.stdout: "request": 0, 2024-05-10T11:44:49.562 INFO:teuthology.orchestra.run.smithi188.stdout: "reply": 0, 2024-05-10T11:44:49.562 INFO:teuthology.orchestra.run.smithi188.stdout: "reply_latency": { 2024-05-10T11:44:49.562 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:49.562 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:49.562 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:49.562 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:49.562 INFO:teuthology.orchestra.run.smithi188.stdout: "slow_reply": 0, 2024-05-10T11:44:49.562 INFO:teuthology.orchestra.run.smithi188.stdout: "forward": 0, 2024-05-10T11:44:49.562 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_fetch_complete": 0, 2024-05-10T11:44:49.562 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_fetch_keys": 0, 2024-05-10T11:44:49.562 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_commit": 0, 2024-05-10T11:44:49.562 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_split": 0, 2024-05-10T11:44:49.563 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_merge": 0, 2024-05-10T11:44:49.563 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes": 6794, 2024-05-10T11:44:49.563 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_top": 4539, 2024-05-10T11:44:49.563 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_bottom": 2255, 2024-05-10T11:44:49.563 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_pin_tail": 0, 2024-05-10T11:44:49.563 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_pinned": 309, 2024-05-10T11:44:49.563 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_expired": 0, 2024-05-10T11:44:49.563 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_with_caps": 0, 2024-05-10T11:44:49.563 INFO:teuthology.orchestra.run.smithi188.stdout: "caps": 0, 2024-05-10T11:44:49.563 INFO:teuthology.orchestra.run.smithi188.stdout: "subtrees": 3, 2024-05-10T11:44:49.563 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse": 0, 2024-05-10T11:44:49.563 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_hit": 0, 2024-05-10T11:44:49.563 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_forward": 0, 2024-05-10T11:44:49.563 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_discover": 0, 2024-05-10T11:44:49.564 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_dir_fetch": 0, 2024-05-10T11:44:49.564 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_remote_ino": 0, 2024-05-10T11:44:49.564 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_lock": 0, 2024-05-10T11:44:49.564 INFO:teuthology.orchestra.run.smithi188.stdout: "load_cent": 0, 2024-05-10T11:44:49.564 INFO:teuthology.orchestra.run.smithi188.stdout: "q": 0, 2024-05-10T11:44:49.564 INFO:teuthology.orchestra.run.smithi188.stdout: "exported": 0, 2024-05-10T11:44:49.564 INFO:teuthology.orchestra.run.smithi188.stdout: "exported_inodes": 0, 2024-05-10T11:44:49.564 INFO:teuthology.orchestra.run.smithi188.stdout: "imported": 0, 2024-05-10T11:44:49.564 INFO:teuthology.orchestra.run.smithi188.stdout: "imported_inodes": 0, 2024-05-10T11:44:49.564 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_dir_fetch": 0, 2024-05-10T11:44:49.564 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_backtrace_fetch": 0, 2024-05-10T11:44:49.564 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_peer_discover": 0, 2024-05-10T11:44:49.564 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rfiles": 61, 2024-05-10T11:44:49.564 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rbytes": 12057077, 2024-05-10T11:44:49.565 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rsnaps": 0, 2024-05-10T11:44:49.565 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_backtrace_fetch": 0, 2024-05-10T11:44:49.565 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_set_tag": 0, 2024-05-10T11:44:49.565 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_backtrace_repaired": 0, 2024-05-10T11:44:49.565 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_inotable_repaired": 0, 2024-05-10T11:44:49.565 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dir_inodes": 0, 2024-05-10T11:44:49.565 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dir_base_inodes": 0, 2024-05-10T11:44:49.565 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dirfrag_rstats": 0, 2024-05-10T11:44:49.565 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_file_inodes": 0, 2024-05-10T11:44:49.565 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_inode_file_caps": 0, 2024-05-10T11:44:49.565 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_revoke": 0, 2024-05-10T11:44:49.565 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_grant": 0, 2024-05-10T11:44:49.565 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_trunc": 0, 2024-05-10T11:44:49.565 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-10T11:44:49.565 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-10T11:44:49.565 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_caps": 0, 2024-05-10T11:44:49.566 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_caps_dirty": 0, 2024-05-10T11:44:49.566 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_cap_release": 0, 2024-05-10T11:44:49.566 INFO:teuthology.orchestra.run.smithi188.stdout: "process_request_cap_release": 0 2024-05-10T11:44:49.566 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:49.566 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_cache": { 2024-05-10T11:44:49.566 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_update": 0, 2024-05-10T11:44:49.566 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_update_receipt": 0, 2024-05-10T11:44:49.566 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_try_discover": 0, 2024-05-10T11:44:49.566 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_send_discover": 0, 2024-05-10T11:44:49.566 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_handle_discover": 0, 2024-05-10T11:44:49.566 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays": 0, 2024-05-10T11:44:49.566 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays_delayed": 0, 2024-05-10T11:44:49.566 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays_enqueuing": 0, 2024-05-10T11:44:49.566 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_created": 0, 2024-05-10T11:44:49.566 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_enqueued": 0, 2024-05-10T11:44:49.567 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_reintegrated": 0, 2024-05-10T11:44:49.567 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_migrated": 0, 2024-05-10T11:44:49.567 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_processing": 0, 2024-05-10T11:44:49.567 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_enqueued": 0, 2024-05-10T11:44:49.567 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_prioritized": 0, 2024-05-10T11:44:49.567 INFO:teuthology.orchestra.run.smithi188.stdout: "recovery_started": 0, 2024-05-10T11:44:49.567 INFO:teuthology.orchestra.run.smithi188.stdout: "recovery_completed": 0, 2024-05-10T11:44:49.567 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_quiesce_path": 0, 2024-05-10T11:44:49.567 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_quiesce_inode": 0, 2024-05-10T11:44:49.567 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_enqueue_scrub": 0, 2024-05-10T11:44:49.567 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_exportdir": 0, 2024-05-10T11:44:49.567 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_flush": 0, 2024-05-10T11:44:49.567 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_fragmentdir": 0, 2024-05-10T11:44:49.567 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_fragstats": 0, 2024-05-10T11:44:49.567 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_inodestats": 0 2024-05-10T11:44:49.568 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:49.568 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_log": { 2024-05-10T11:44:49.568 INFO:teuthology.orchestra.run.smithi188.stdout: "evlrg": 0, 2024-05-10T11:44:49.568 INFO:teuthology.orchestra.run.smithi188.stdout: "evadd": 0, 2024-05-10T11:44:49.568 INFO:teuthology.orchestra.run.smithi188.stdout: "evex": 0, 2024-05-10T11:44:49.568 INFO:teuthology.orchestra.run.smithi188.stdout: "evtrm": 0, 2024-05-10T11:44:49.568 INFO:teuthology.orchestra.run.smithi188.stdout: "ev": 74243, 2024-05-10T11:44:49.568 INFO:teuthology.orchestra.run.smithi188.stdout: "evexg": 0, 2024-05-10T11:44:49.568 INFO:teuthology.orchestra.run.smithi188.stdout: "evexd": 0, 2024-05-10T11:44:49.568 INFO:teuthology.orchestra.run.smithi188.stdout: "segadd": 0, 2024-05-10T11:44:49.568 INFO:teuthology.orchestra.run.smithi188.stdout: "segex": 0, 2024-05-10T11:44:49.568 INFO:teuthology.orchestra.run.smithi188.stdout: "segtrm": 0, 2024-05-10T11:44:49.569 INFO:teuthology.orchestra.run.smithi188.stdout: "seg": 3, 2024-05-10T11:44:49.569 INFO:teuthology.orchestra.run.smithi188.stdout: "segmjr": 68, 2024-05-10T11:44:49.569 INFO:teuthology.orchestra.run.smithi188.stdout: "segexg": 0, 2024-05-10T11:44:49.569 INFO:teuthology.orchestra.run.smithi188.stdout: "segexd": 0, 2024-05-10T11:44:49.569 INFO:teuthology.orchestra.run.smithi188.stdout: "expos": 209418875, 2024-05-10T11:44:49.569 INFO:teuthology.orchestra.run.smithi188.stdout: "wrpos": 211669065, 2024-05-10T11:44:49.569 INFO:teuthology.orchestra.run.smithi188.stdout: "rdpos": 211667078, 2024-05-10T11:44:49.569 INFO:teuthology.orchestra.run.smithi188.stdout: "jlat": { 2024-05-10T11:44:49.569 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:49.569 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:49.569 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:49.570 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:49.570 INFO:teuthology.orchestra.run.smithi188.stdout: "replayed": 74243 2024-05-10T11:44:49.570 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:49.570 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_mem": { 2024-05-10T11:44:49.570 INFO:teuthology.orchestra.run.smithi188.stdout: "ino": 24, 2024-05-10T11:44:49.570 INFO:teuthology.orchestra.run.smithi188.stdout: "ino+": 7306, 2024-05-10T11:44:49.570 INFO:teuthology.orchestra.run.smithi188.stdout: "ino-": 7282, 2024-05-10T11:44:49.570 INFO:teuthology.orchestra.run.smithi188.stdout: "dir": 23, 2024-05-10T11:44:49.570 INFO:teuthology.orchestra.run.smithi188.stdout: "dir+": 130, 2024-05-10T11:44:49.570 INFO:teuthology.orchestra.run.smithi188.stdout: "dir-": 107, 2024-05-10T11:44:49.570 INFO:teuthology.orchestra.run.smithi188.stdout: "dn": 6794, 2024-05-10T11:44:49.570 INFO:teuthology.orchestra.run.smithi188.stdout: "dn+": 7937, 2024-05-10T11:44:49.570 INFO:teuthology.orchestra.run.smithi188.stdout: "dn-": 1143, 2024-05-10T11:44:49.570 INFO:teuthology.orchestra.run.smithi188.stdout: "cap": 0, 2024-05-10T11:44:49.571 INFO:teuthology.orchestra.run.smithi188.stdout: "cap+": 0, 2024-05-10T11:44:49.571 INFO:teuthology.orchestra.run.smithi188.stdout: "cap-": 0, 2024-05-10T11:44:49.571 INFO:teuthology.orchestra.run.smithi188.stdout: "rss": 58592, 2024-05-10T11:44:49.571 INFO:teuthology.orchestra.run.smithi188.stdout: "heap": 215392 2024-05-10T11:44:49.571 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:49.571 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_server": { 2024-05-10T11:44:49.571 INFO:teuthology.orchestra.run.smithi188.stdout: "dispatch_client_request": 0, 2024-05-10T11:44:49.571 INFO:teuthology.orchestra.run.smithi188.stdout: "dispatch_server_request": 0, 2024-05-10T11:44:49.571 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_request": 0, 2024-05-10T11:44:49.572 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_session": 0, 2024-05-10T11:44:49.572 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_peer_request": 0, 2024-05-10T11:44:49.572 INFO:teuthology.orchestra.run.smithi188.stdout: "req_create_latency": { 2024-05-10T11:44:49.572 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:49.572 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:49.572 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:49.572 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:49.572 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getattr_latency": { 2024-05-10T11:44:49.572 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:49.572 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:49.572 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:49.572 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:49.572 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getfilelock_latency": { 2024-05-10T11:44:49.573 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:49.573 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:49.573 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:49.573 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:49.573 INFO:teuthology.orchestra.run.smithi188.stdout: "req_link_latency": { 2024-05-10T11:44:49.573 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:49.573 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:49.573 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:49.573 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:49.573 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookup_latency": { 2024-05-10T11:44:49.573 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:49.573 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:49.574 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:49.574 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:49.574 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookuphash_latency": { 2024-05-10T11:44:49.574 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:49.574 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:49.574 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:49.574 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:49.574 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupino_latency": { 2024-05-10T11:44:49.574 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:49.574 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:49.574 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:49.574 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:49.574 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupname_latency": { 2024-05-10T11:44:49.575 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:49.575 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:49.575 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:49.575 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:49.575 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupparent_latency": { 2024-05-10T11:44:49.575 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:49.575 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:49.575 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:49.575 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:49.575 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupsnap_latency": { 2024-05-10T11:44:49.575 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:49.576 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:49.576 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:49.576 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:49.576 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lssnap_latency": { 2024-05-10T11:44:49.576 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:49.576 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:49.576 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:49.576 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:49.577 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mkdir_latency": { 2024-05-10T11:44:49.577 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:49.577 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:49.577 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:49.577 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:49.577 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mknod_latency": { 2024-05-10T11:44:49.577 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:49.577 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:49.577 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:49.578 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:49.578 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mksnap_latency": { 2024-05-10T11:44:49.578 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:49.578 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:49.578 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:49.578 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:49.578 INFO:teuthology.orchestra.run.smithi188.stdout: "req_open_latency": { 2024-05-10T11:44:49.578 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:49.578 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:49.578 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:49.578 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:49.578 INFO:teuthology.orchestra.run.smithi188.stdout: "req_readdir_latency": { 2024-05-10T11:44:49.578 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:49.579 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:49.579 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:49.579 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:49.579 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rename_latency": { 2024-05-10T11:44:49.579 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:49.579 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:49.579 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:49.579 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:49.579 INFO:teuthology.orchestra.run.smithi188.stdout: "req_renamesnap_latency": { 2024-05-10T11:44:49.579 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:49.580 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:49.580 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:49.580 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:49.580 INFO:teuthology.orchestra.run.smithi188.stdout: "req_snapdiff_latency": { 2024-05-10T11:44:49.580 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:49.580 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:49.580 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:49.580 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:49.580 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmdir_latency": { 2024-05-10T11:44:49.580 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:49.581 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:49.581 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:49.581 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:49.581 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmsnap_latency": { 2024-05-10T11:44:49.581 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:49.581 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:49.581 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:49.581 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:49.581 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmxattr_latency": { 2024-05-10T11:44:49.581 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:49.581 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:49.582 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:49.582 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:49.582 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setattr_latency": { 2024-05-10T11:44:49.582 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:49.582 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:49.582 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:49.582 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:49.582 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setdirlayout_latency": { 2024-05-10T11:44:49.582 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:49.582 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:49.582 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:49.582 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:49.582 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setfilelock_latency": { 2024-05-10T11:44:49.582 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:49.582 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:49.583 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:49.583 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:49.583 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setlayout_latency": { 2024-05-10T11:44:49.583 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:49.583 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:49.583 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:49.583 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:49.583 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setxattr_latency": { 2024-05-10T11:44:49.583 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:49.583 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:49.583 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:49.583 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:49.583 INFO:teuthology.orchestra.run.smithi188.stdout: "req_symlink_latency": { 2024-05-10T11:44:49.583 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:49.583 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:49.584 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:49.584 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:49.584 INFO:teuthology.orchestra.run.smithi188.stdout: "req_unlink_latency": { 2024-05-10T11:44:49.584 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:49.584 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:49.584 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:49.584 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:49.584 INFO:teuthology.orchestra.run.smithi188.stdout: "cap_revoke_eviction": 0, 2024-05-10T11:44:49.584 INFO:teuthology.orchestra.run.smithi188.stdout: "cap_acquisition_throttle": 0, 2024-05-10T11:44:49.584 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getvxattr_latency": { 2024-05-10T11:44:49.584 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:49.584 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:49.584 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:49.584 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-10T11:44:49.585 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:49.585 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_sessions": { 2024-05-10T11:44:49.585 INFO:teuthology.orchestra.run.smithi188.stdout: "session_count": 1, 2024-05-10T11:44:49.585 INFO:teuthology.orchestra.run.smithi188.stdout: "session_add": 1, 2024-05-10T11:44:49.585 INFO:teuthology.orchestra.run.smithi188.stdout: "session_remove": 0, 2024-05-10T11:44:49.585 INFO:teuthology.orchestra.run.smithi188.stdout: "sessions_open": 0, 2024-05-10T11:44:49.585 INFO:teuthology.orchestra.run.smithi188.stdout: "sessions_stale": 0, 2024-05-10T11:44:49.585 INFO:teuthology.orchestra.run.smithi188.stdout: "total_load": 0, 2024-05-10T11:44:49.585 INFO:teuthology.orchestra.run.smithi188.stdout: "average_load": 0, 2024-05-10T11:44:49.585 INFO:teuthology.orchestra.run.smithi188.stdout: "avg_session_uptime": 2844, 2024-05-10T11:44:49.585 INFO:teuthology.orchestra.run.smithi188.stdout: "mdthresh_evicted": 0 2024-05-10T11:44:49.585 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:49.585 INFO:teuthology.orchestra.run.smithi188.stdout: "mempool": { 2024-05-10T11:44:49.585 INFO:teuthology.orchestra.run.smithi188.stdout: "bloom_filter_bytes": 0, 2024-05-10T11:44:49.585 INFO:teuthology.orchestra.run.smithi188.stdout: "bloom_filter_items": 0, 2024-05-10T11:44:49.586 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T11:44:49.586 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_alloc_items": 0, 2024-05-10T11:44:49.586 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T11:44:49.586 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_data_items": 0, 2024-05-10T11:44:49.586 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T11:44:49.586 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T11:44:49.586 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T11:44:49.586 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T11:44:49.586 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T11:44:49.586 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_other_items": 0, 2024-05-10T11:44:49.586 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T11:44:49.586 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T11:44:49.586 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_extent_bytes": 0, 2024-05-10T11:44:49.586 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_extent_items": 0, 2024-05-10T11:44:49.587 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_blob_bytes": 0, 2024-05-10T11:44:49.587 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_blob_items": 0, 2024-05-10T11:44:49.587 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T11:44:49.587 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T11:44:49.587 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T11:44:49.587 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T11:44:49.587 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T11:44:49.587 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_fsck_items": 0, 2024-05-10T11:44:49.587 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_txc_bytes": 0, 2024-05-10T11:44:49.587 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_txc_items": 0, 2024-05-10T11:44:49.587 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T11:44:49.587 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T11:44:49.587 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_bytes": 0, 2024-05-10T11:44:49.587 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_items": 0, 2024-05-10T11:44:49.587 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_bytes": 0, 2024-05-10T11:44:49.588 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_items": 0, 2024-05-10T11:44:49.588 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T11:44:49.588 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_reader_items": 0, 2024-05-10T11:44:49.588 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T11:44:49.588 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_writer_items": 0, 2024-05-10T11:44:49.588 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_anon_bytes": 240215, 2024-05-10T11:44:49.588 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_anon_items": 80, 2024-05-10T11:44:49.588 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_meta_bytes": 0, 2024-05-10T11:44:49.588 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_meta_items": 0, 2024-05-10T11:44:49.588 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_bytes": 0, 2024-05-10T11:44:49.588 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_items": 0, 2024-05-10T11:44:49.588 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_mapbl_bytes": 0, 2024-05-10T11:44:49.588 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_mapbl_items": 0, 2024-05-10T11:44:49.588 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_pglog_bytes": 0, 2024-05-10T11:44:49.588 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_pglog_items": 0, 2024-05-10T11:44:49.588 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_bytes": 8696, 2024-05-10T11:44:49.589 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_items": 166, 2024-05-10T11:44:49.589 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T11:44:49.589 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_mapping_items": 0, 2024-05-10T11:44:49.589 INFO:teuthology.orchestra.run.smithi188.stdout: "pgmap_bytes": 0, 2024-05-10T11:44:49.589 INFO:teuthology.orchestra.run.smithi188.stdout: "pgmap_items": 0, 2024-05-10T11:44:49.589 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_co_bytes": 4866943, 2024-05-10T11:44:49.589 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_co_items": 27929, 2024-05-10T11:44:49.589 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_1_bytes": 0, 2024-05-10T11:44:49.589 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_1_items": 0, 2024-05-10T11:44:49.589 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_2_bytes": 0, 2024-05-10T11:44:49.589 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_2_items": 0 2024-05-10T11:44:49.589 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:49.589 INFO:teuthology.orchestra.run.smithi188.stdout: "objecter": { 2024-05-10T11:44:49.589 INFO:teuthology.orchestra.run.smithi188.stdout: "op_active": 0, 2024-05-10T11:44:49.589 INFO:teuthology.orchestra.run.smithi188.stdout: "op_laggy": 0, 2024-05-10T11:44:49.590 INFO:teuthology.orchestra.run.smithi188.stdout: "op_send": 3736, 2024-05-10T11:44:49.590 INFO:teuthology.orchestra.run.smithi188.stdout: "op_send_bytes": 26, 2024-05-10T11:44:49.590 INFO:teuthology.orchestra.run.smithi188.stdout: "op_resend": 3, 2024-05-10T11:44:49.590 INFO:teuthology.orchestra.run.smithi188.stdout: "op_reply": 3736, 2024-05-10T11:44:49.590 INFO:teuthology.orchestra.run.smithi188.stdout: "op_latency": { 2024-05-10T11:44:49.590 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 3736, 2024-05-10T11:44:49.590 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 2.757950744, 2024-05-10T11:44:49.590 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000738209 2024-05-10T11:44:49.590 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:49.590 INFO:teuthology.orchestra.run.smithi188.stdout: "op_inflight": 0, 2024-05-10T11:44:49.590 INFO:teuthology.orchestra.run.smithi188.stdout: "oplen_avg": { 2024-05-10T11:44:49.590 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 3736, 2024-05-10T11:44:49.590 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 3737 2024-05-10T11:44:49.590 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:49.590 INFO:teuthology.orchestra.run.smithi188.stdout: "op": 3736, 2024-05-10T11:44:49.590 INFO:teuthology.orchestra.run.smithi188.stdout: "op_r": 3736, 2024-05-10T11:44:49.591 INFO:teuthology.orchestra.run.smithi188.stdout: "op_w": 0, 2024-05-10T11:44:49.591 INFO:teuthology.orchestra.run.smithi188.stdout: "op_rmw": 0, 2024-05-10T11:44:49.591 INFO:teuthology.orchestra.run.smithi188.stdout: "op_pg": 0, 2024-05-10T11:44:49.591 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_stat": 1862, 2024-05-10T11:44:49.591 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_create": 0, 2024-05-10T11:44:49.591 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_read": 1871, 2024-05-10T11:44:49.591 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_write": 0, 2024-05-10T11:44:49.591 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_writefull": 0, 2024-05-10T11:44:49.591 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_writesame": 0, 2024-05-10T11:44:49.591 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_append": 0, 2024-05-10T11:44:49.591 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_zero": 0, 2024-05-10T11:44:49.591 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_truncate": 0, 2024-05-10T11:44:49.591 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_delete": 0, 2024-05-10T11:44:49.591 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_mapext": 0, 2024-05-10T11:44:49.591 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_sparse_read": 0, 2024-05-10T11:44:49.591 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_clonerange": 0, 2024-05-10T11:44:49.592 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_getxattr": 2, 2024-05-10T11:44:49.592 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_setxattr": 0, 2024-05-10T11:44:49.592 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_cmpxattr": 0, 2024-05-10T11:44:49.592 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_rmxattr": 0, 2024-05-10T11:44:49.592 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_resetxattrs": 0, 2024-05-10T11:44:49.592 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_call": 0, 2024-05-10T11:44:49.592 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_watch": 0, 2024-05-10T11:44:49.592 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_notify": 0, 2024-05-10T11:44:49.592 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_src_cmpxattr": 0, 2024-05-10T11:44:49.592 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_pgls": 0, 2024-05-10T11:44:49.592 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_pgls_filter": 0, 2024-05-10T11:44:49.592 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_other": 0, 2024-05-10T11:44:49.592 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_active": 0, 2024-05-10T11:44:49.592 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_send": 0, 2024-05-10T11:44:49.592 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_resend": 0, 2024-05-10T11:44:49.592 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_ping": 0, 2024-05-10T11:44:49.593 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_active": 0, 2024-05-10T11:44:49.593 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_send": 0, 2024-05-10T11:44:49.593 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_resend": 0, 2024-05-10T11:44:49.593 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_active": 0, 2024-05-10T11:44:49.593 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_send": 0, 2024-05-10T11:44:49.593 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_resend": 0, 2024-05-10T11:44:49.593 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_active": 0, 2024-05-10T11:44:49.593 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_send": 0, 2024-05-10T11:44:49.593 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_resend": 0, 2024-05-10T11:44:49.593 INFO:teuthology.orchestra.run.smithi188.stdout: "command_active": 0, 2024-05-10T11:44:49.593 INFO:teuthology.orchestra.run.smithi188.stdout: "command_send": 0, 2024-05-10T11:44:49.593 INFO:teuthology.orchestra.run.smithi188.stdout: "command_resend": 0, 2024-05-10T11:44:49.594 INFO:teuthology.orchestra.run.smithi188.stdout: "map_epoch": 0, 2024-05-10T11:44:49.594 INFO:teuthology.orchestra.run.smithi188.stdout: "map_full": 0, 2024-05-10T11:44:49.594 INFO:teuthology.orchestra.run.smithi188.stdout: "map_inc": 0, 2024-05-10T11:44:49.594 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_sessions": 11, 2024-05-10T11:44:49.594 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_session_open": 11, 2024-05-10T11:44:49.594 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_session_close": 0, 2024-05-10T11:44:49.594 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_laggy": 0, 2024-05-10T11:44:49.594 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_wr": 0, 2024-05-10T11:44:49.594 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_rd": 2, 2024-05-10T11:44:49.594 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_del": 0 2024-05-10T11:44:49.594 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:49.594 INFO:teuthology.orchestra.run.smithi188.stdout: "oft": { 2024-05-10T11:44:49.594 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_objs": 0, 2024-05-10T11:44:49.595 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_kv_pairs": 0, 2024-05-10T11:44:49.595 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_updates": 0, 2024-05-10T11:44:49.595 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_removes": 0 2024-05-10T11:44:49.595 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:49.595 INFO:teuthology.orchestra.run.smithi188.stdout: "osd-slow-ops": { 2024-05-10T11:44:49.595 INFO:teuthology.orchestra.run.smithi188.stdout: "slow_ops_count": 0 2024-05-10T11:44:49.595 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:49.595 INFO:teuthology.orchestra.run.smithi188.stdout: "purge_queue": { 2024-05-10T11:44:49.595 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_ops": 0, 2024-05-10T11:44:49.595 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_ops_high_water": 0, 2024-05-10T11:44:49.595 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing": 0, 2024-05-10T11:44:49.595 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_high_water": 0, 2024-05-10T11:44:49.595 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executed_ops": 0, 2024-05-10T11:44:49.595 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executed": 0, 2024-05-10T11:44:49.595 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_item_in_journal": 0 2024-05-10T11:44:49.595 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:49.596 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T11:44:49.596 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-05-10T11:44:49.596 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 104857600, 2024-05-10T11:44:49.596 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-05-10T11:44:49.596 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 4006, 2024-05-10T11:44:49.596 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 157990807, 2024-05-10T11:44:49.596 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-05-10T11:44:49.596 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 4006, 2024-05-10T11:44:49.596 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-05-10T11:44:49.596 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-05-10T11:44:49.596 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 4006, 2024-05-10T11:44:49.596 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 157990807, 2024-05-10T11:44:49.596 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-05-10T11:44:49.596 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:49.596 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:49.596 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:49.597 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-10T11:44:49.597 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:49.597 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-objecter_bytes": { 2024-05-10T11:44:49.597 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-05-10T11:44:49.597 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 104857600, 2024-05-10T11:44:49.597 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-05-10T11:44:49.597 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-05-10T11:44:49.597 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-05-10T11:44:49.597 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-05-10T11:44:49.597 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-05-10T11:44:49.597 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 3736, 2024-05-10T11:44:49.597 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 157054929, 2024-05-10T11:44:49.597 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 3736, 2024-05-10T11:44:49.597 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 157054929, 2024-05-10T11:44:49.597 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-05-10T11:44:49.598 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:49.598 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:49.598 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:49.598 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-10T11:44:49.598 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:49.598 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-objecter_ops": { 2024-05-10T11:44:49.598 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-05-10T11:44:49.598 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 1024, 2024-05-10T11:44:49.598 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-05-10T11:44:49.598 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-05-10T11:44:49.598 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-05-10T11:44:49.598 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-05-10T11:44:49.598 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-05-10T11:44:49.598 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 3736, 2024-05-10T11:44:49.598 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 3736, 2024-05-10T11:44:49.599 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 3736, 2024-05-10T11:44:49.599 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 3736, 2024-05-10T11:44:49.599 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-05-10T11:44:49.599 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:49.599 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:49.599 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:49.599 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-10T11:44:49.599 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:49.599 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-write_buf_throttle": { 2024-05-10T11:44:49.599 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-05-10T11:44:49.599 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 3758096384, 2024-05-10T11:44:49.599 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-05-10T11:44:49.599 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-05-10T11:44:49.599 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-05-10T11:44:49.599 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-05-10T11:44:49.600 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-05-10T11:44:49.600 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-05-10T11:44:49.600 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-05-10T11:44:49.600 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 0, 2024-05-10T11:44:49.600 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 0, 2024-05-10T11:44:49.600 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-05-10T11:44:49.600 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:49.600 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:49.600 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:49.600 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-10T11:44:49.600 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:49.600 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-write_buf_throttle-0x55fa2e6846e0": { 2024-05-10T11:44:49.600 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-05-10T11:44:49.600 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 3758096384, 2024-05-10T11:44:49.600 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-05-10T11:44:49.600 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-05-10T11:44:49.601 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-05-10T11:44:49.601 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-05-10T11:44:49.601 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-05-10T11:44:49.601 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-05-10T11:44:49.601 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-05-10T11:44:49.601 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 0, 2024-05-10T11:44:49.601 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 0, 2024-05-10T11:44:49.601 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-05-10T11:44:49.601 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:49.601 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:49.601 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:49.601 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-10T11:44:49.601 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-10T11:44:49.601 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-10T11:44:50.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:44:50 smithi188 ceph-mon[34075]: pgmap v1214: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 597 B/s wr, 5 op/s 2024-05-10T11:44:50.403 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-05-10T11:44:50.403 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-05-10T11:44:50.404 DEBUG:tasks.check_counter:Getting stats from i 2024-05-10T11:44:50.404 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph daemon mds.i perf dump 2024-05-10T11:44:50.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:44:50 smithi178 ceph-mon[33359]: pgmap v1214: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 597 B/s wr, 5 op/s 2024-05-10T11:44:50.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:44:50 smithi100 ceph-mon[30296]: pgmap v1214: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 597 B/s wr, 5 op/s 2024-05-10T11:44:50.660 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T11:44:52.333 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:44:52 smithi188 ceph-mon[34075]: pgmap v1215: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 938 B/s wr, 6 op/s 2024-05-10T11:44:52.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:44:52 smithi100 ceph-mon[30296]: pgmap v1215: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 938 B/s wr, 6 op/s 2024-05-10T11:44:52.476 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-10T11:44:52.476 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T11:44:52.476 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 165279, 2024-05-10T11:44:52.476 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 170565, 2024-05-10T11:44:52.476 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 39885030, 2024-05-10T11:44:52.476 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 163853508, 2024-05-10T11:44:52.476 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 455, 2024-05-10T11:44:52.476 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 7, 2024-05-10T11:44:52.476 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 10.045362855, 2024-05-10T11:44:52.476 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 3.724045169, 2024-05-10T11:44:52.476 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 5.641122608, 2024-05-10T11:44:52.476 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 1.606653770, 2024-05-10T11:44:52.477 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T11:44:52.477 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 170567, 2024-05-10T11:44:52.477 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 11.023979212, 2024-05-10T11:44:52.477 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000064631 2024-05-10T11:44:52.477 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:52.477 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-05-10T11:44:52.477 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 251726, 2024-05-10T11:44:52.477 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.033106966, 2024-05-10T11:44:52.477 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000131 2024-05-10T11:44:52.477 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:52.477 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 2912, 2024-05-10T11:44:52.477 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 448 2024-05-10T11:44:52.477 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:52.477 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T11:44:52.477 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 183115, 2024-05-10T11:44:52.478 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 183312, 2024-05-10T11:44:52.478 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 45932096, 2024-05-10T11:44:52.478 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 121320555, 2024-05-10T11:44:52.478 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 7, 2024-05-10T11:44:52.478 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 5, 2024-05-10T11:44:52.478 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 12.190213059, 2024-05-10T11:44:52.478 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 5.267524918, 2024-05-10T11:44:52.478 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 6.670889686, 2024-05-10T11:44:52.478 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 1.510492912, 2024-05-10T11:44:52.478 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T11:44:52.478 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 183314, 2024-05-10T11:44:52.478 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 9.666551466, 2024-05-10T11:44:52.478 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000052732 2024-05-10T11:44:52.478 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:52.479 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-05-10T11:44:52.479 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 301229, 2024-05-10T11:44:52.479 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.043797956, 2024-05-10T11:44:52.479 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000145 2024-05-10T11:44:52.479 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:52.479 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 729168, 2024-05-10T11:44:52.479 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 349072 2024-05-10T11:44:52.479 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:52.479 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T11:44:52.479 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 1574881, 2024-05-10T11:44:52.479 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 1618225, 2024-05-10T11:44:52.479 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 463147972, 2024-05-10T11:44:52.479 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 985934089, 2024-05-10T11:44:52.479 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 19, 2024-05-10T11:44:52.479 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 8, 2024-05-10T11:44:52.480 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 65.345678327, 2024-05-10T11:44:52.480 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 27.007156481, 2024-05-10T11:44:52.480 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 46.418210315, 2024-05-10T11:44:52.480 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 1.437293560, 2024-05-10T11:44:52.480 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T11:44:52.480 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 1618217, 2024-05-10T11:44:52.480 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 40.198377324, 2024-05-10T11:44:52.480 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000024841 2024-05-10T11:44:52.480 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:52.480 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-05-10T11:44:52.480 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 3097756, 2024-05-10T11:44:52.480 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.398221753, 2024-05-10T11:44:52.480 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000128 2024-05-10T11:44:52.480 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:52.480 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 54304, 2024-05-10T11:44:52.481 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 870192 2024-05-10T11:44:52.481 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:52.481 INFO:teuthology.orchestra.run.smithi188.stdout: "cct": { 2024-05-10T11:44:52.481 INFO:teuthology.orchestra.run.smithi188.stdout: "total_workers": 1, 2024-05-10T11:44:52.481 INFO:teuthology.orchestra.run.smithi188.stdout: "unhealthy_workers": 0 2024-05-10T11:44:52.481 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:52.481 INFO:teuthology.orchestra.run.smithi188.stdout: "finisher-MDSRank": { 2024-05-10T11:44:52.481 INFO:teuthology.orchestra.run.smithi188.stdout: "queue_len": 0, 2024-05-10T11:44:52.481 INFO:teuthology.orchestra.run.smithi188.stdout: "complete_latency": { 2024-05-10T11:44:52.481 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 119741, 2024-05-10T11:44:52.481 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 82.364025428, 2024-05-10T11:44:52.481 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000687851 2024-05-10T11:44:52.481 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-10T11:44:52.481 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:52.482 INFO:teuthology.orchestra.run.smithi188.stdout: "finisher-PurgeQueue": { 2024-05-10T11:44:52.482 INFO:teuthology.orchestra.run.smithi188.stdout: "queue_len": 0, 2024-05-10T11:44:52.482 INFO:teuthology.orchestra.run.smithi188.stdout: "complete_latency": { 2024-05-10T11:44:52.482 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 14541, 2024-05-10T11:44:52.482 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 1.888534236, 2024-05-10T11:44:52.482 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000129876 2024-05-10T11:44:52.482 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-10T11:44:52.482 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:52.482 INFO:teuthology.orchestra.run.smithi188.stdout: "mds": { 2024-05-10T11:44:52.482 INFO:teuthology.orchestra.run.smithi188.stdout: "request": 1472107, 2024-05-10T11:44:52.482 INFO:teuthology.orchestra.run.smithi188.stdout: "reply": 1428880, 2024-05-10T11:44:52.482 INFO:teuthology.orchestra.run.smithi188.stdout: "reply_latency": { 2024-05-10T11:44:52.482 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 1428880, 2024-05-10T11:44:52.482 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 1195.983282460, 2024-05-10T11:44:52.482 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000837007 2024-05-10T11:44:52.483 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:52.483 INFO:teuthology.orchestra.run.smithi188.stdout: "slow_reply": 0, 2024-05-10T11:44:52.483 INFO:teuthology.orchestra.run.smithi188.stdout: "forward": 43227, 2024-05-10T11:44:52.483 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_fetch_complete": 7, 2024-05-10T11:44:52.483 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_fetch_keys": 2, 2024-05-10T11:44:52.483 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_commit": 2756, 2024-05-10T11:44:52.483 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_split": 0, 2024-05-10T11:44:52.483 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_merge": 0, 2024-05-10T11:44:52.483 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes": 45, 2024-05-10T11:44:52.483 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_top": 9, 2024-05-10T11:44:52.483 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_bottom": 25, 2024-05-10T11:44:52.483 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_pin_tail": 11, 2024-05-10T11:44:52.483 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_pinned": 31, 2024-05-10T11:44:52.483 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_expired": 1084, 2024-05-10T11:44:52.483 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_with_caps": 2, 2024-05-10T11:44:52.484 INFO:teuthology.orchestra.run.smithi188.stdout: "caps": 2, 2024-05-10T11:44:52.484 INFO:teuthology.orchestra.run.smithi188.stdout: "subtrees": 5, 2024-05-10T11:44:52.484 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse": 2869669, 2024-05-10T11:44:52.484 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_hit": 2852382, 2024-05-10T11:44:52.484 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_forward": 16739, 2024-05-10T11:44:52.484 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_discover": 26, 2024-05-10T11:44:52.484 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_dir_fetch": 0, 2024-05-10T11:44:52.484 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_remote_ino": 0, 2024-05-10T11:44:52.484 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_lock": 2, 2024-05-10T11:44:52.484 INFO:teuthology.orchestra.run.smithi188.stdout: "load_cent": 1, 2024-05-10T11:44:52.484 INFO:teuthology.orchestra.run.smithi188.stdout: "q": 0, 2024-05-10T11:44:52.484 INFO:teuthology.orchestra.run.smithi188.stdout: "exported": 34, 2024-05-10T11:44:52.484 INFO:teuthology.orchestra.run.smithi188.stdout: "exported_inodes": 931, 2024-05-10T11:44:52.484 INFO:teuthology.orchestra.run.smithi188.stdout: "imported": 30, 2024-05-10T11:44:52.484 INFO:teuthology.orchestra.run.smithi188.stdout: "imported_inodes": 116, 2024-05-10T11:44:52.485 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_dir_fetch": 0, 2024-05-10T11:44:52.485 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_backtrace_fetch": 2841, 2024-05-10T11:44:52.485 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_peer_discover": 0, 2024-05-10T11:44:52.485 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rfiles": 61, 2024-05-10T11:44:52.485 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rbytes": 12057077, 2024-05-10T11:44:52.485 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rsnaps": 0, 2024-05-10T11:44:52.485 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_backtrace_fetch": 2841, 2024-05-10T11:44:52.485 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_set_tag": 0, 2024-05-10T11:44:52.485 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_backtrace_repaired": 0, 2024-05-10T11:44:52.485 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_inotable_repaired": 0, 2024-05-10T11:44:52.485 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dir_inodes": 955, 2024-05-10T11:44:52.485 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dir_base_inodes": 28, 2024-05-10T11:44:52.485 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dirfrag_rstats": 927, 2024-05-10T11:44:52.485 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_file_inodes": 1886, 2024-05-10T11:44:52.485 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_inode_file_caps": 44, 2024-05-10T11:44:52.486 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_revoke": 63202, 2024-05-10T11:44:52.486 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_grant": 31985, 2024-05-10T11:44:52.486 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_trunc": 0, 2024-05-10T11:44:52.486 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-10T11:44:52.486 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-10T11:44:52.486 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_caps": 79317, 2024-05-10T11:44:52.486 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_caps_dirty": 12767, 2024-05-10T11:44:52.486 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_cap_release": 636, 2024-05-10T11:44:52.486 INFO:teuthology.orchestra.run.smithi188.stdout: "process_request_cap_release": 28186 2024-05-10T11:44:52.486 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:52.486 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_cache": { 2024-05-10T11:44:52.486 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_update": 14, 2024-05-10T11:44:52.486 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_update_receipt": 6, 2024-05-10T11:44:52.486 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_try_discover": 4, 2024-05-10T11:44:52.486 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_send_discover": 28, 2024-05-10T11:44:52.487 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_handle_discover": 88, 2024-05-10T11:44:52.487 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays": 6, 2024-05-10T11:44:52.487 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays_delayed": 0, 2024-05-10T11:44:52.487 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays_enqueuing": 0, 2024-05-10T11:44:52.487 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_created": 7182, 2024-05-10T11:44:52.487 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_enqueued": 7176, 2024-05-10T11:44:52.487 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_reintegrated": 0, 2024-05-10T11:44:52.487 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_migrated": 0, 2024-05-10T11:44:52.487 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_processing": 0, 2024-05-10T11:44:52.487 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_enqueued": 0, 2024-05-10T11:44:52.487 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_prioritized": 0, 2024-05-10T11:44:52.487 INFO:teuthology.orchestra.run.smithi188.stdout: "recovery_started": 0, 2024-05-10T11:44:52.487 INFO:teuthology.orchestra.run.smithi188.stdout: "recovery_completed": 0, 2024-05-10T11:44:52.487 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_quiesce_path": 0, 2024-05-10T11:44:52.487 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_quiesce_inode": 0, 2024-05-10T11:44:52.488 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_enqueue_scrub": 28, 2024-05-10T11:44:52.488 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_exportdir": 34, 2024-05-10T11:44:52.488 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_flush": 0, 2024-05-10T11:44:52.488 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_fragmentdir": 0, 2024-05-10T11:44:52.488 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_fragstats": 0, 2024-05-10T11:44:52.488 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_inodestats": 0 2024-05-10T11:44:52.488 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:52.488 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_log": { 2024-05-10T11:44:52.488 INFO:teuthology.orchestra.run.smithi188.stdout: "evlrg": 0, 2024-05-10T11:44:52.488 INFO:teuthology.orchestra.run.smithi188.stdout: "evadd": 98345, 2024-05-10T11:44:52.488 INFO:teuthology.orchestra.run.smithi188.stdout: "evex": 97182, 2024-05-10T11:44:52.489 INFO:teuthology.orchestra.run.smithi188.stdout: "evtrm": 97182, 2024-05-10T11:44:52.489 INFO:teuthology.orchestra.run.smithi188.stdout: "ev": 1163, 2024-05-10T11:44:52.489 INFO:teuthology.orchestra.run.smithi188.stdout: "evexg": 0, 2024-05-10T11:44:52.489 INFO:teuthology.orchestra.run.smithi188.stdout: "evexd": 1204, 2024-05-10T11:44:52.489 INFO:teuthology.orchestra.run.smithi188.stdout: "segadd": 199, 2024-05-10T11:44:52.489 INFO:teuthology.orchestra.run.smithi188.stdout: "segex": 196, 2024-05-10T11:44:52.489 INFO:teuthology.orchestra.run.smithi188.stdout: "segtrm": 196, 2024-05-10T11:44:52.489 INFO:teuthology.orchestra.run.smithi188.stdout: "seg": 3, 2024-05-10T11:44:52.489 INFO:teuthology.orchestra.run.smithi188.stdout: "segmjr": 1, 2024-05-10T11:44:52.489 INFO:teuthology.orchestra.run.smithi188.stdout: "segexg": 0, 2024-05-10T11:44:52.490 INFO:teuthology.orchestra.run.smithi188.stdout: "segexd": 2, 2024-05-10T11:44:52.490 INFO:teuthology.orchestra.run.smithi188.stdout: "expos": 209418875, 2024-05-10T11:44:52.490 INFO:teuthology.orchestra.run.smithi188.stdout: "wrpos": 211672629, 2024-05-10T11:44:52.490 INFO:teuthology.orchestra.run.smithi188.stdout: "rdpos": 0, 2024-05-10T11:44:52.490 INFO:teuthology.orchestra.run.smithi188.stdout: "jlat": { 2024-05-10T11:44:52.490 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 61218, 2024-05-10T11:44:52.490 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 277.579321622, 2024-05-10T11:44:52.490 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.004534276 2024-05-10T11:44:52.490 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:52.490 INFO:teuthology.orchestra.run.smithi188.stdout: "replayed": 0 2024-05-10T11:44:52.490 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:52.490 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_mem": { 2024-05-10T11:44:52.490 INFO:teuthology.orchestra.run.smithi188.stdout: "ino": 31, 2024-05-10T11:44:52.490 INFO:teuthology.orchestra.run.smithi188.stdout: "ino+": 7908, 2024-05-10T11:44:52.491 INFO:teuthology.orchestra.run.smithi188.stdout: "ino-": 7877, 2024-05-10T11:44:52.491 INFO:teuthology.orchestra.run.smithi188.stdout: "dir": 27, 2024-05-10T11:44:52.491 INFO:teuthology.orchestra.run.smithi188.stdout: "dir+": 159, 2024-05-10T11:44:52.491 INFO:teuthology.orchestra.run.smithi188.stdout: "dir-": 132, 2024-05-10T11:44:52.491 INFO:teuthology.orchestra.run.smithi188.stdout: "dn": 45, 2024-05-10T11:44:52.491 INFO:teuthology.orchestra.run.smithi188.stdout: "dn+": 12589, 2024-05-10T11:44:52.491 INFO:teuthology.orchestra.run.smithi188.stdout: "dn-": 12544, 2024-05-10T11:44:52.491 INFO:teuthology.orchestra.run.smithi188.stdout: "cap": 2, 2024-05-10T11:44:52.491 INFO:teuthology.orchestra.run.smithi188.stdout: "cap+": 7816, 2024-05-10T11:44:52.491 INFO:teuthology.orchestra.run.smithi188.stdout: "cap-": 7814, 2024-05-10T11:44:52.491 INFO:teuthology.orchestra.run.smithi188.stdout: "rss": 77648, 2024-05-10T11:44:52.491 INFO:teuthology.orchestra.run.smithi188.stdout: "heap": 231800 2024-05-10T11:44:52.491 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:52.491 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_server": { 2024-05-10T11:44:52.491 INFO:teuthology.orchestra.run.smithi188.stdout: "dispatch_client_request": 1528741, 2024-05-10T11:44:52.491 INFO:teuthology.orchestra.run.smithi188.stdout: "dispatch_server_request": 44738, 2024-05-10T11:44:52.492 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_request": 1472107, 2024-05-10T11:44:52.492 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_session": 1001, 2024-05-10T11:44:52.492 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_peer_request": 89554, 2024-05-10T11:44:52.493 INFO:teuthology.orchestra.run.smithi188.stdout: "req_create_latency": { 2024-05-10T11:44:52.493 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 7635, 2024-05-10T11:44:52.493 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 12.750846081, 2024-05-10T11:44:52.493 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.001670051 2024-05-10T11:44:52.493 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:52.493 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getattr_latency": { 2024-05-10T11:44:52.494 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 1350741, 2024-05-10T11:44:52.494 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 723.710994405, 2024-05-10T11:44:52.494 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000535788 2024-05-10T11:44:52.494 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:52.494 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getfilelock_latency": { 2024-05-10T11:44:52.494 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:52.494 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:52.494 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:52.494 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:52.494 INFO:teuthology.orchestra.run.smithi188.stdout: "req_link_latency": { 2024-05-10T11:44:52.494 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:52.494 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:52.494 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:52.494 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:52.495 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookup_latency": { 2024-05-10T11:44:52.495 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 44463, 2024-05-10T11:44:52.495 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 377.111396251, 2024-05-10T11:44:52.495 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.008481465 2024-05-10T11:44:52.495 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:52.495 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookuphash_latency": { 2024-05-10T11:44:52.495 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:52.495 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:52.495 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:52.495 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:52.495 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupino_latency": { 2024-05-10T11:44:52.495 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:52.495 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:52.495 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:52.495 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:52.496 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupname_latency": { 2024-05-10T11:44:52.496 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:52.496 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:52.496 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:52.496 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:52.496 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupparent_latency": { 2024-05-10T11:44:52.496 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:52.496 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:52.496 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:52.496 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:52.496 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupsnap_latency": { 2024-05-10T11:44:52.496 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:52.497 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:52.497 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:52.497 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:52.497 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lssnap_latency": { 2024-05-10T11:44:52.497 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:52.497 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:52.497 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:52.497 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:52.498 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mkdir_latency": { 2024-05-10T11:44:52.498 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 119, 2024-05-10T11:44:52.498 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.400200699, 2024-05-10T11:44:52.498 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.003363031 2024-05-10T11:44:52.498 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:52.498 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mknod_latency": { 2024-05-10T11:44:52.498 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:52.498 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:52.498 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:52.498 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:52.498 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mksnap_latency": { 2024-05-10T11:44:52.498 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:52.498 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:52.498 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:52.499 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:52.499 INFO:teuthology.orchestra.run.smithi188.stdout: "req_open_latency": { 2024-05-10T11:44:52.499 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 5585, 2024-05-10T11:44:52.499 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 6.866491182, 2024-05-10T11:44:52.499 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.001229452 2024-05-10T11:44:52.499 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:52.499 INFO:teuthology.orchestra.run.smithi188.stdout: "req_readdir_latency": { 2024-05-10T11:44:52.499 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 10743, 2024-05-10T11:44:52.499 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 51.694404274, 2024-05-10T11:44:52.499 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.004811915 2024-05-10T11:44:52.499 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:52.500 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rename_latency": { 2024-05-10T11:44:52.500 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 2007, 2024-05-10T11:44:52.500 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 4.306347140, 2024-05-10T11:44:52.500 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.002145663 2024-05-10T11:44:52.500 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:52.500 INFO:teuthology.orchestra.run.smithi188.stdout: "req_renamesnap_latency": { 2024-05-10T11:44:52.500 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:52.500 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:52.500 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:52.500 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:52.500 INFO:teuthology.orchestra.run.smithi188.stdout: "req_snapdiff_latency": { 2024-05-10T11:44:52.500 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:52.500 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:52.500 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:52.501 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:52.501 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmdir_latency": { 2024-05-10T11:44:52.501 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 64, 2024-05-10T11:44:52.501 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.535487600, 2024-05-10T11:44:52.501 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.008366993 2024-05-10T11:44:52.501 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:52.501 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmsnap_latency": { 2024-05-10T11:44:52.501 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:52.501 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:52.501 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:52.501 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:52.501 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmxattr_latency": { 2024-05-10T11:44:52.501 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:52.501 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:52.502 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:52.502 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:52.502 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setattr_latency": { 2024-05-10T11:44:52.502 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 6, 2024-05-10T11:44:52.502 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.036730488, 2024-05-10T11:44:52.502 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.006121748 2024-05-10T11:44:52.502 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:52.503 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setdirlayout_latency": { 2024-05-10T11:44:52.503 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:52.503 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:52.503 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:52.503 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:52.503 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setfilelock_latency": { 2024-05-10T11:44:52.503 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 380, 2024-05-10T11:44:52.503 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.282466929, 2024-05-10T11:44:52.503 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000743334 2024-05-10T11:44:52.503 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:52.503 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setlayout_latency": { 2024-05-10T11:44:52.503 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:52.503 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:52.503 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:52.504 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:52.504 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setxattr_latency": { 2024-05-10T11:44:52.504 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 8, 2024-05-10T11:44:52.504 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.022350107, 2024-05-10T11:44:52.504 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.002793763 2024-05-10T11:44:52.504 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:52.504 INFO:teuthology.orchestra.run.smithi188.stdout: "req_symlink_latency": { 2024-05-10T11:44:52.504 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:52.504 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:52.504 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:52.504 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:52.504 INFO:teuthology.orchestra.run.smithi188.stdout: "req_unlink_latency": { 2024-05-10T11:44:52.504 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 7129, 2024-05-10T11:44:52.504 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 18.265567304, 2024-05-10T11:44:52.504 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.002562149 2024-05-10T11:44:52.505 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:52.505 INFO:teuthology.orchestra.run.smithi188.stdout: "cap_revoke_eviction": 0, 2024-05-10T11:44:52.505 INFO:teuthology.orchestra.run.smithi188.stdout: "cap_acquisition_throttle": 0, 2024-05-10T11:44:52.505 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getvxattr_latency": { 2024-05-10T11:44:52.505 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:52.505 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:52.505 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:52.505 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-10T11:44:52.505 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:52.505 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_sessions": { 2024-05-10T11:44:52.505 INFO:teuthology.orchestra.run.smithi188.stdout: "session_count": 1, 2024-05-10T11:44:52.505 INFO:teuthology.orchestra.run.smithi188.stdout: "session_add": 3, 2024-05-10T11:44:52.505 INFO:teuthology.orchestra.run.smithi188.stdout: "session_remove": 2, 2024-05-10T11:44:52.505 INFO:teuthology.orchestra.run.smithi188.stdout: "sessions_open": 1, 2024-05-10T11:44:52.505 INFO:teuthology.orchestra.run.smithi188.stdout: "sessions_stale": 0, 2024-05-10T11:44:52.506 INFO:teuthology.orchestra.run.smithi188.stdout: "total_load": 91469, 2024-05-10T11:44:52.506 INFO:teuthology.orchestra.run.smithi188.stdout: "average_load": 91469, 2024-05-10T11:44:52.506 INFO:teuthology.orchestra.run.smithi188.stdout: "avg_session_uptime": 1586, 2024-05-10T11:44:52.506 INFO:teuthology.orchestra.run.smithi188.stdout: "mdthresh_evicted": 0 2024-05-10T11:44:52.506 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:52.506 INFO:teuthology.orchestra.run.smithi188.stdout: "mempool": { 2024-05-10T11:44:52.506 INFO:teuthology.orchestra.run.smithi188.stdout: "bloom_filter_bytes": 0, 2024-05-10T11:44:52.506 INFO:teuthology.orchestra.run.smithi188.stdout: "bloom_filter_items": 0, 2024-05-10T11:44:52.506 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T11:44:52.506 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_alloc_items": 0, 2024-05-10T11:44:52.506 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T11:44:52.506 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_data_items": 0, 2024-05-10T11:44:52.506 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T11:44:52.506 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T11:44:52.507 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T11:44:52.507 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T11:44:52.507 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T11:44:52.507 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_other_items": 0, 2024-05-10T11:44:52.507 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T11:44:52.507 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T11:44:52.507 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_extent_bytes": 0, 2024-05-10T11:44:52.507 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_extent_items": 0, 2024-05-10T11:44:52.507 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_blob_bytes": 0, 2024-05-10T11:44:52.507 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_blob_items": 0, 2024-05-10T11:44:52.507 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T11:44:52.507 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T11:44:52.507 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T11:44:52.507 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T11:44:52.508 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T11:44:52.508 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_fsck_items": 0, 2024-05-10T11:44:52.508 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_txc_bytes": 0, 2024-05-10T11:44:52.508 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_txc_items": 0, 2024-05-10T11:44:52.508 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T11:44:52.508 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T11:44:52.508 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_bytes": 0, 2024-05-10T11:44:52.508 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_items": 0, 2024-05-10T11:44:52.508 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_bytes": 0, 2024-05-10T11:44:52.508 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_items": 0, 2024-05-10T11:44:52.508 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T11:44:52.508 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_reader_items": 0, 2024-05-10T11:44:52.508 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T11:44:52.508 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_writer_items": 0, 2024-05-10T11:44:52.508 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_anon_bytes": 291912, 2024-05-10T11:44:52.509 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_anon_items": 117, 2024-05-10T11:44:52.509 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_meta_bytes": 0, 2024-05-10T11:44:52.509 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_meta_items": 0, 2024-05-10T11:44:52.509 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_bytes": 0, 2024-05-10T11:44:52.509 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_items": 0, 2024-05-10T11:44:52.509 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_mapbl_bytes": 0, 2024-05-10T11:44:52.509 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_mapbl_items": 0, 2024-05-10T11:44:52.509 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_pglog_bytes": 0, 2024-05-10T11:44:52.509 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_pglog_items": 0, 2024-05-10T11:44:52.509 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_bytes": 8696, 2024-05-10T11:44:52.509 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_items": 166, 2024-05-10T11:44:52.509 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T11:44:52.509 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_mapping_items": 0, 2024-05-10T11:44:52.509 INFO:teuthology.orchestra.run.smithi188.stdout: "pgmap_bytes": 0, 2024-05-10T11:44:52.509 INFO:teuthology.orchestra.run.smithi188.stdout: "pgmap_items": 0, 2024-05-10T11:44:52.510 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_co_bytes": 141847, 2024-05-10T11:44:52.510 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_co_items": 1457, 2024-05-10T11:44:52.510 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_1_bytes": 0, 2024-05-10T11:44:52.510 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_1_items": 0, 2024-05-10T11:44:52.510 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_2_bytes": 0, 2024-05-10T11:44:52.510 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_2_items": 0 2024-05-10T11:44:52.510 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:52.510 INFO:teuthology.orchestra.run.smithi188.stdout: "objecter": { 2024-05-10T11:44:52.510 INFO:teuthology.orchestra.run.smithi188.stdout: "op_active": 0, 2024-05-10T11:44:52.510 INFO:teuthology.orchestra.run.smithi188.stdout: "op_laggy": 0, 2024-05-10T11:44:52.510 INFO:teuthology.orchestra.run.smithi188.stdout: "op_send": 92335, 2024-05-10T11:44:52.510 INFO:teuthology.orchestra.run.smithi188.stdout: "op_send_bytes": 219083584, 2024-05-10T11:44:52.510 INFO:teuthology.orchestra.run.smithi188.stdout: "op_resend": 37, 2024-05-10T11:44:52.510 INFO:teuthology.orchestra.run.smithi188.stdout: "op_reply": 92335, 2024-05-10T11:44:52.511 INFO:teuthology.orchestra.run.smithi188.stdout: "op_latency": { 2024-05-10T11:44:52.511 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 92335, 2024-05-10T11:44:52.511 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 909.199732893, 2024-05-10T11:44:52.511 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.009846750 2024-05-10T11:44:52.511 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:52.511 INFO:teuthology.orchestra.run.smithi188.stdout: "op_inflight": 0, 2024-05-10T11:44:52.511 INFO:teuthology.orchestra.run.smithi188.stdout: "oplen_avg": { 2024-05-10T11:44:52.511 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 92335, 2024-05-10T11:44:52.511 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 109319 2024-05-10T11:44:52.511 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:52.511 INFO:teuthology.orchestra.run.smithi188.stdout: "op": 92335, 2024-05-10T11:44:52.511 INFO:teuthology.orchestra.run.smithi188.stdout: "op_r": 4094, 2024-05-10T11:44:52.511 INFO:teuthology.orchestra.run.smithi188.stdout: "op_w": 88241, 2024-05-10T11:44:52.511 INFO:teuthology.orchestra.run.smithi188.stdout: "op_rmw": 0, 2024-05-10T11:44:52.511 INFO:teuthology.orchestra.run.smithi188.stdout: "op_pg": 0, 2024-05-10T11:44:52.511 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_stat": 2625, 2024-05-10T11:44:52.512 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_create": 6204, 2024-05-10T11:44:52.512 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_read": 1216, 2024-05-10T11:44:52.512 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_write": 62406, 2024-05-10T11:44:52.512 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_writefull": 1030, 2024-05-10T11:44:52.512 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_writesame": 0, 2024-05-10T11:44:52.512 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_append": 0, 2024-05-10T11:44:52.512 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_zero": 0, 2024-05-10T11:44:52.512 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_truncate": 0, 2024-05-10T11:44:52.512 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_delete": 14914, 2024-05-10T11:44:52.512 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_mapext": 0, 2024-05-10T11:44:52.512 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_sparse_read": 0, 2024-05-10T11:44:52.512 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_clonerange": 0, 2024-05-10T11:44:52.512 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_getxattr": 2878, 2024-05-10T11:44:52.512 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_setxattr": 9418, 2024-05-10T11:44:52.513 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_cmpxattr": 0, 2024-05-10T11:44:52.513 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_rmxattr": 0, 2024-05-10T11:44:52.513 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_resetxattrs": 0, 2024-05-10T11:44:52.513 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_call": 0, 2024-05-10T11:44:52.513 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_watch": 0, 2024-05-10T11:44:52.513 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_notify": 0, 2024-05-10T11:44:52.513 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_src_cmpxattr": 0, 2024-05-10T11:44:52.513 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_pgls": 0, 2024-05-10T11:44:52.513 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_pgls_filter": 0, 2024-05-10T11:44:52.513 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_other": 0, 2024-05-10T11:44:52.513 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_active": 0, 2024-05-10T11:44:52.513 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_send": 0, 2024-05-10T11:44:52.513 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_resend": 0, 2024-05-10T11:44:52.513 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_ping": 0, 2024-05-10T11:44:52.513 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_active": 0, 2024-05-10T11:44:52.514 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_send": 0, 2024-05-10T11:44:52.514 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_resend": 0, 2024-05-10T11:44:52.514 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_active": 0, 2024-05-10T11:44:52.514 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_send": 0, 2024-05-10T11:44:52.514 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_resend": 0, 2024-05-10T11:44:52.514 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_active": 0, 2024-05-10T11:44:52.514 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_send": 0, 2024-05-10T11:44:52.514 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_resend": 0, 2024-05-10T11:44:52.514 INFO:teuthology.orchestra.run.smithi188.stdout: "command_active": 0, 2024-05-10T11:44:52.514 INFO:teuthology.orchestra.run.smithi188.stdout: "command_send": 0, 2024-05-10T11:44:52.514 INFO:teuthology.orchestra.run.smithi188.stdout: "command_resend": 0, 2024-05-10T11:44:52.514 INFO:teuthology.orchestra.run.smithi188.stdout: "map_epoch": 90, 2024-05-10T11:44:52.514 INFO:teuthology.orchestra.run.smithi188.stdout: "map_full": 0, 2024-05-10T11:44:52.514 INFO:teuthology.orchestra.run.smithi188.stdout: "map_inc": 10, 2024-05-10T11:44:52.515 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_sessions": 12, 2024-05-10T11:44:52.515 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_session_open": 12, 2024-05-10T11:44:52.515 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_session_close": 0, 2024-05-10T11:44:52.515 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_laggy": 0, 2024-05-10T11:44:52.515 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_wr": 7037, 2024-05-10T11:44:52.515 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_rd": 18, 2024-05-10T11:44:52.515 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_del": 1573 2024-05-10T11:44:52.515 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:52.515 INFO:teuthology.orchestra.run.smithi188.stdout: "oft": { 2024-05-10T11:44:52.515 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_objs": 1, 2024-05-10T11:44:52.515 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_kv_pairs": 0, 2024-05-10T11:44:52.515 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_updates": 0, 2024-05-10T11:44:52.515 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_removes": 0 2024-05-10T11:44:52.515 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:52.515 INFO:teuthology.orchestra.run.smithi188.stdout: "osd-slow-ops": { 2024-05-10T11:44:52.515 INFO:teuthology.orchestra.run.smithi188.stdout: "slow_ops_count": 0 2024-05-10T11:44:52.516 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:52.516 INFO:teuthology.orchestra.run.smithi188.stdout: "purge_queue": { 2024-05-10T11:44:52.516 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_ops": 0, 2024-05-10T11:44:52.516 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_ops_high_water": 17, 2024-05-10T11:44:52.516 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing": 0, 2024-05-10T11:44:52.516 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_high_water": 16, 2024-05-10T11:44:52.516 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executed_ops": 14305, 2024-05-10T11:44:52.516 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executed": 7176, 2024-05-10T11:44:52.516 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_item_in_journal": 0 2024-05-10T11:44:52.516 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:52.516 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T11:44:52.516 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-05-10T11:44:52.516 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 104857600, 2024-05-10T11:44:52.516 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-05-10T11:44:52.517 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 1923271, 2024-05-10T11:44:52.517 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 375830187, 2024-05-10T11:44:52.517 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-05-10T11:44:52.517 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 1923271, 2024-05-10T11:44:52.517 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-05-10T11:44:52.517 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-05-10T11:44:52.517 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 1923271, 2024-05-10T11:44:52.517 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 375830187, 2024-05-10T11:44:52.517 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-05-10T11:44:52.517 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:52.517 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:52.517 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:52.517 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-10T11:44:52.517 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:52.517 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-objecter_bytes": { 2024-05-10T11:44:52.517 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-05-10T11:44:52.518 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 104857600, 2024-05-10T11:44:52.518 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-05-10T11:44:52.518 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-05-10T11:44:52.518 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-05-10T11:44:52.518 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-05-10T11:44:52.518 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-05-10T11:44:52.518 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 92335, 2024-05-10T11:44:52.518 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 219865272, 2024-05-10T11:44:52.518 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 92335, 2024-05-10T11:44:52.518 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 219865272, 2024-05-10T11:44:52.518 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-05-10T11:44:52.518 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:52.518 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:52.518 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:52.519 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-10T11:44:52.519 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:52.519 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-objecter_ops": { 2024-05-10T11:44:52.519 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-05-10T11:44:52.519 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 1024, 2024-05-10T11:44:52.519 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-05-10T11:44:52.519 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-05-10T11:44:52.519 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-05-10T11:44:52.519 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-05-10T11:44:52.519 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-05-10T11:44:52.519 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 92335, 2024-05-10T11:44:52.519 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 92335, 2024-05-10T11:44:52.519 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 92335, 2024-05-10T11:44:52.519 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 92335, 2024-05-10T11:44:52.519 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-05-10T11:44:52.520 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:52.520 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:52.520 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:52.520 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-10T11:44:52.520 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:52.520 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-write_buf_throttle": { 2024-05-10T11:44:52.520 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-05-10T11:44:52.520 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 3758096384, 2024-05-10T11:44:52.520 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-05-10T11:44:52.520 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 7176, 2024-05-10T11:44:52.520 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 781808, 2024-05-10T11:44:52.520 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-05-10T11:44:52.520 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 7176, 2024-05-10T11:44:52.520 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-05-10T11:44:52.520 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-05-10T11:44:52.520 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 1188, 2024-05-10T11:44:52.521 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 781808, 2024-05-10T11:44:52.521 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-05-10T11:44:52.521 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:52.521 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:52.521 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:52.521 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-10T11:44:52.521 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:52.521 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-write_buf_throttle-0x560800a8e500": { 2024-05-10T11:44:52.521 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-05-10T11:44:52.521 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 3758096384, 2024-05-10T11:44:52.521 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-05-10T11:44:52.521 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 98345, 2024-05-10T11:44:52.521 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 207478325, 2024-05-10T11:44:52.521 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-05-10T11:44:52.522 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 98345, 2024-05-10T11:44:52.522 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-05-10T11:44:52.522 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-05-10T11:44:52.522 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 61218, 2024-05-10T11:44:52.522 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 207478325, 2024-05-10T11:44:52.522 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-05-10T11:44:52.522 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:52.522 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:52.522 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:52.522 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-10T11:44:52.522 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-10T11:44:52.522 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-10T11:44:52.523 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:44:52 smithi178 ceph-mon[33359]: pgmap v1215: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 938 B/s wr, 6 op/s 2024-05-10T11:44:53.303 INFO:tasks.check_counter:Daemon mds.i mds.exported=34 2024-05-10T11:44:53.303 INFO:tasks.check_counter:Daemon mds.i mds.imported=30 2024-05-10T11:44:53.303 DEBUG:tasks.check_counter:Getting stats from l 2024-05-10T11:44:53.303 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph daemon mds.l perf dump 2024-05-10T11:44:53.548 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T11:44:54.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:44:54 smithi188 ceph-mon[34075]: pgmap v1216: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-10T11:44:54.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:44:54 smithi178 ceph-mon[33359]: pgmap v1216: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-10T11:44:54.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:44:54 smithi100 ceph-mon[30296]: pgmap v1216: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-10T11:44:55.256 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-10T11:44:55.256 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T11:44:55.256 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 4, 2024-05-10T11:44:55.256 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 4, 2024-05-10T11:44:55.256 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 2912, 2024-05-10T11:44:55.256 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 448, 2024-05-10T11:44:55.256 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 4, 2024-05-10T11:44:55.256 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 0, 2024-05-10T11:44:55.256 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 0.001213606, 2024-05-10T11:44:55.256 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 0.000038792, 2024-05-10T11:44:55.256 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 0.000435829, 2024-05-10T11:44:55.256 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-10T11:44:55.256 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T11:44:55.257 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 3, 2024-05-10T11:44:55.257 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000637586, 2024-05-10T11:44:55.257 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000212528 2024-05-10T11:44:55.257 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:55.257 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-05-10T11:44:55.257 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:55.257 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:55.257 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:55.257 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:55.257 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 2912, 2024-05-10T11:44:55.257 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 448 2024-05-10T11:44:55.257 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:55.257 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T11:44:55.257 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 2, 2024-05-10T11:44:55.257 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 381, 2024-05-10T11:44:55.258 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 320, 2024-05-10T11:44:55.258 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 224304, 2024-05-10T11:44:55.258 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 4, 2024-05-10T11:44:55.258 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 2, 2024-05-10T11:44:55.258 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 0.017068922, 2024-05-10T11:44:55.258 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 0.013306724, 2024-05-10T11:44:55.258 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 0.000837271, 2024-05-10T11:44:55.258 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-10T11:44:55.258 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T11:44:55.258 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 379, 2024-05-10T11:44:55.258 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.017989433, 2024-05-10T11:44:55.258 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000047465 2024-05-10T11:44:55.258 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:55.259 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-05-10T11:44:55.259 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:55.259 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:55.259 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:55.259 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:55.259 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-10T11:44:55.259 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 224304 2024-05-10T11:44:55.259 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:55.259 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T11:44:55.259 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 501, 2024-05-10T11:44:55.259 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 498, 2024-05-10T11:44:55.259 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 594528, 2024-05-10T11:44:55.259 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 308608, 2024-05-10T11:44:55.259 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 2, 2024-05-10T11:44:55.260 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 1, 2024-05-10T11:44:55.260 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 0.060740589, 2024-05-10T11:44:55.260 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 0.024163389, 2024-05-10T11:44:55.260 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 0.023184467, 2024-05-10T11:44:55.260 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.010036082, 2024-05-10T11:44:55.260 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T11:44:55.260 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 497, 2024-05-10T11:44:55.260 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.016201777, 2024-05-10T11:44:55.260 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000032599 2024-05-10T11:44:55.260 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:55.260 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-05-10T11:44:55.260 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:55.260 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:55.260 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:55.261 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:55.261 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 594528, 2024-05-10T11:44:55.261 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 308608 2024-05-10T11:44:55.261 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:55.261 INFO:teuthology.orchestra.run.smithi188.stdout: "cct": { 2024-05-10T11:44:55.261 INFO:teuthology.orchestra.run.smithi188.stdout: "total_workers": 0, 2024-05-10T11:44:55.261 INFO:teuthology.orchestra.run.smithi188.stdout: "unhealthy_workers": 0 2024-05-10T11:44:55.261 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:55.261 INFO:teuthology.orchestra.run.smithi188.stdout: "mempool": { 2024-05-10T11:44:55.261 INFO:teuthology.orchestra.run.smithi188.stdout: "bloom_filter_bytes": 0, 2024-05-10T11:44:55.261 INFO:teuthology.orchestra.run.smithi188.stdout: "bloom_filter_items": 0, 2024-05-10T11:44:55.261 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T11:44:55.261 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_alloc_items": 0, 2024-05-10T11:44:55.261 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T11:44:55.262 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_data_items": 0, 2024-05-10T11:44:55.262 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T11:44:55.262 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T11:44:55.262 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T11:44:55.262 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T11:44:55.262 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T11:44:55.262 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_other_items": 0, 2024-05-10T11:44:55.262 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T11:44:55.262 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T11:44:55.262 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_extent_bytes": 0, 2024-05-10T11:44:55.262 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_extent_items": 0, 2024-05-10T11:44:55.262 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_blob_bytes": 0, 2024-05-10T11:44:55.262 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_blob_items": 0, 2024-05-10T11:44:55.262 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T11:44:55.263 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T11:44:55.263 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T11:44:55.263 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T11:44:55.263 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T11:44:55.263 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_fsck_items": 0, 2024-05-10T11:44:55.263 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_txc_bytes": 0, 2024-05-10T11:44:55.263 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_txc_items": 0, 2024-05-10T11:44:55.263 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T11:44:55.263 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T11:44:55.263 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_bytes": 0, 2024-05-10T11:44:55.263 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_items": 0, 2024-05-10T11:44:55.263 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_bytes": 0, 2024-05-10T11:44:55.263 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_items": 0, 2024-05-10T11:44:55.263 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T11:44:55.264 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_reader_items": 0, 2024-05-10T11:44:55.264 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T11:44:55.264 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_writer_items": 0, 2024-05-10T11:44:55.264 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_anon_bytes": 148650, 2024-05-10T11:44:55.264 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_anon_items": 28, 2024-05-10T11:44:55.264 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_meta_bytes": 0, 2024-05-10T11:44:55.264 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_meta_items": 0, 2024-05-10T11:44:55.264 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_bytes": 0, 2024-05-10T11:44:55.264 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_items": 0, 2024-05-10T11:44:55.264 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_mapbl_bytes": 0, 2024-05-10T11:44:55.264 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_mapbl_items": 0, 2024-05-10T11:44:55.264 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_pglog_bytes": 0, 2024-05-10T11:44:55.264 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_pglog_items": 0, 2024-05-10T11:44:55.264 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_bytes": 0, 2024-05-10T11:44:55.264 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_items": 0, 2024-05-10T11:44:55.265 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T11:44:55.265 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_mapping_items": 0, 2024-05-10T11:44:55.265 INFO:teuthology.orchestra.run.smithi188.stdout: "pgmap_bytes": 0, 2024-05-10T11:44:55.265 INFO:teuthology.orchestra.run.smithi188.stdout: "pgmap_items": 0, 2024-05-10T11:44:55.265 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_co_bytes": 928, 2024-05-10T11:44:55.265 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_co_items": 2, 2024-05-10T11:44:55.265 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_1_bytes": 0, 2024-05-10T11:44:55.265 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_1_items": 0, 2024-05-10T11:44:55.265 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_2_bytes": 0, 2024-05-10T11:44:55.265 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_2_items": 0 2024-05-10T11:44:55.265 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-10T11:44:55.265 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T11:44:55.265 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-05-10T11:44:55.265 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 104857600, 2024-05-10T11:44:55.266 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-05-10T11:44:55.266 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 503, 2024-05-10T11:44:55.266 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 523642, 2024-05-10T11:44:55.266 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-05-10T11:44:55.266 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 503, 2024-05-10T11:44:55.266 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-05-10T11:44:55.266 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-05-10T11:44:55.266 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 503, 2024-05-10T11:44:55.266 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 523642, 2024-05-10T11:44:55.266 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-05-10T11:44:55.266 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-10T11:44:55.266 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-10T11:44:55.266 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-10T11:44:55.266 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-10T11:44:55.266 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-10T11:44:55.266 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-10T11:44:55.996 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-05-10T11:44:55.996 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-05-10T11:44:55.997 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-05-10T11:44:56.007 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-05-10T11:44:56.008 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-05-10T11:44:56.008 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T11:44:56.036 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-05-10T11:44:56.036 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T11:44:56.096 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph osd blocklist ls 2024-05-10T11:44:56.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:44:56 smithi188 ceph-mon[34075]: pgmap v1217: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 341 B/s wr, 5 op/s 2024-05-10T11:44:56.389 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:44:56.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:44:56 smithi178 ceph-mon[33359]: pgmap v1217: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 341 B/s wr, 5 op/s 2024-05-10T11:44:56.512 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:44:56 smithi100 ceph-mon[30296]: pgmap v1217: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 341 B/s wr, 5 op/s 2024-05-10T11:44:58.206 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:44:58 smithi100 ceph-mon[30296]: pgmap v1218: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 7 op/s 2024-05-10T11:44:58.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:58.219+0000 7f8c7dc92700 1 -- 172.21.15.100:0/2168677217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c7806c150 msgr2=0x7f8c7806c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:58.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:58.219+0000 7f8c7dc92700 1 --2- 172.21.15.100:0/2168677217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c7806c150 0x7f8c7806c5d0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c009990 tx=0x7f8c6c009ca0 comp rx=0 tx=0).stop 2024-05-10T11:44:58.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:58.219+0000 7f8c7dc92700 1 -- 172.21.15.100:0/2168677217 shutdown_connections 2024-05-10T11:44:58.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:58.219+0000 7f8c7dc92700 1 --2- 172.21.15.100:0/2168677217 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c78105f30 0x7f8c781063b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:58.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:58.219+0000 7f8c7dc92700 1 --2- 172.21.15.100:0/2168677217 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c781084c0 0x7f8c7806bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:58.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:58.219+0000 7f8c7dc92700 1 --2- 172.21.15.100:0/2168677217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c7806c150 0x7f8c7806c5d0 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:58.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:58.219+0000 7f8c7dc92700 1 -- 172.21.15.100:0/2168677217 >> 172.21.15.100:0/2168677217 conn(0x7f8c780783f0 msgr2=0x7f8c78078800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:44:58.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:58.219+0000 7f8c7dc92700 1 -- 172.21.15.100:0/2168677217 shutdown_connections 2024-05-10T11:44:58.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:58.219+0000 7f8c7dc92700 1 -- 172.21.15.100:0/2168677217 wait complete. 2024-05-10T11:44:58.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:58.219+0000 7f8c7dc92700 1 Processor -- start 2024-05-10T11:44:58.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:58.219+0000 7f8c7dc92700 1 -- start start 2024-05-10T11:44:58.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:58.220+0000 7f8c7dc92700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c7806c150 0x7f8c7819ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:58.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:58.220+0000 7f8c7dc92700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c78105f30 0x7f8c781a0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:58.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:58.220+0000 7f8c7dc92700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c781084c0 0x7f8c781a4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:58.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:58.220+0000 7f8c7dc92700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8c78119b30 con 0x7f8c78105f30 2024-05-10T11:44:58.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:58.220+0000 7f8c7dc92700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8c781199b0 con 0x7f8c7806c150 2024-05-10T11:44:58.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:58.220+0000 7f8c7dc92700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8c78119cb0 con 0x7f8c781084c0 2024-05-10T11:44:58.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:58.220+0000 7f8c76ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c78105f30 0x7f8c781a0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:58.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:58.220+0000 7f8c76ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c78105f30 0x7f8c781a0500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60376/0 (socket says 172.21.15.100:60376) 2024-05-10T11:44:58.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:58.220+0000 7f8c76ffd700 1 -- 172.21.15.100:0/3768148887 learned_addr learned my addr 172.21.15.100:0/3768148887 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:44:58.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:58.220+0000 7f8c777fe700 1 --2- 172.21.15.100:0/3768148887 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c7806c150 0x7f8c7819ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:58.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:58.220+0000 7f8c77fff700 1 --2- 172.21.15.100:0/3768148887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c781084c0 0x7f8c781a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:58.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:58.220+0000 7f8c76ffd700 1 -- 172.21.15.100:0/3768148887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c781084c0 msgr2=0x7f8c781a4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:58.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:58.220+0000 7f8c76ffd700 1 --2- 172.21.15.100:0/3768148887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c781084c0 0x7f8c781a4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:58.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:58.220+0000 7f8c76ffd700 1 -- 172.21.15.100:0/3768148887 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c7806c150 msgr2=0x7f8c7819ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:58.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:58.220+0000 7f8c76ffd700 1 --2- 172.21.15.100:0/3768148887 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c7806c150 0x7f8c7819ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:58.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:58.220+0000 7f8c76ffd700 1 -- 172.21.15.100:0/3768148887 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c781a4fd0 con 0x7f8c78105f30 2024-05-10T11:44:58.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:58.220+0000 7f8c76ffd700 1 --2- 172.21.15.100:0/3768148887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c78105f30 0x7f8c781a0500 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c0059d0 tx=0x7f8c6c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:44:58.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:58.220+0000 7f8c74ff9700 1 -- 172.21.15.100:0/3768148887 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c027070 con 0x7f8c78105f30 2024-05-10T11:44:58.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:58.221+0000 7f8c74ff9700 1 -- 172.21.15.100:0/3768148887 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8c6c00fd40 con 0x7f8c78105f30 2024-05-10T11:44:58.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:58.221+0000 7f8c7dc92700 1 -- 172.21.15.100:0/3768148887 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c781a5260 con 0x7f8c78105f30 2024-05-10T11:44:58.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:58.221+0000 7f8c74ff9700 1 -- 172.21.15.100:0/3768148887 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c022db0 con 0x7f8c78105f30 2024-05-10T11:44:58.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:58.221+0000 7f8c7dc92700 1 -- 172.21.15.100:0/3768148887 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c781acb40 con 0x7f8c78105f30 2024-05-10T11:44:58.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:58.221+0000 7f8c7dc92700 1 -- 172.21.15.100:0/3768148887 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c7806a190 con 0x7f8c78105f30 2024-05-10T11:44:58.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:58.223+0000 7f8c74ff9700 1 -- 172.21.15.100:0/3768148887 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f8c6c02a070 con 0x7f8c78105f30 2024-05-10T11:44:58.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:58.223+0000 7f8c74ff9700 1 --2- 172.21.15.100:0/3768148887 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8c600b2880 0x7f8c600b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:44:58.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:58.224+0000 7f8c777fe700 1 --2- 172.21.15.100:0/3768148887 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8c600b2880 0x7f8c600b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:44:58.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:58.224+0000 7f8c74ff9700 1 -- 172.21.15.100:0/3768148887 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f8c6c0e3380 con 0x7f8c78105f30 2024-05-10T11:44:58.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:58.224+0000 7f8c777fe700 1 --2- 172.21.15.100:0/3768148887 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8c600b2880 0x7f8c600b4d40 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f8c68005950 tx=0x7f8c6800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:44:58.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:58.227+0000 7f8c74ff9700 1 -- 172.21.15.100:0/3768148887 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f8c6c013070 con 0x7f8c78105f30 2024-05-10T11:44:58.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:58.446+0000 7f8c7dc92700 1 -- 172.21.15.100:0/3768148887 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8c781acd10 con 0x7f8c78105f30 2024-05-10T11:44:58.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:58.447+0000 7f8c74ff9700 1 -- 172.21.15.100:0/3768148887 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v90) ==== 81+0+601 (secure 0 0 0) 0x7f8c6c0aae60 con 0x7f8c78105f30 2024-05-10T11:44:58.447 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.100:6800/1597433518 2024-05-11T11:07:00.092732+0000 2024-05-10T11:44:58.447 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.100:6800/1086450256 2024-05-11T11:06:33.114176+0000 2024-05-10T11:44:58.447 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.100:0/767964410 2024-05-11T11:07:00.092732+0000 2024-05-10T11:44:58.447 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.100:6801/1597433518 2024-05-11T11:07:00.092732+0000 2024-05-10T11:44:58.447 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.100:6801/1086450256 2024-05-11T11:06:33.114176+0000 2024-05-10T11:44:58.447 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.100:0/2770572070 2024-05-11T11:06:33.114176+0000 2024-05-10T11:44:58.447 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.100:0/1379340513 2024-05-11T11:06:33.114176+0000 2024-05-10T11:44:58.447 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.100:0/3025646953 2024-05-11T11:07:00.092732+0000 2024-05-10T11:44:58.447 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.100:0/4099258930 2024-05-11T11:07:00.092732+0000 2024-05-10T11:44:58.447 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.100:0/4267693567 2024-05-11T11:06:33.114176+0000 2024-05-10T11:44:58.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:58.449+0000 7f8c7dc92700 1 -- 172.21.15.100:0/3768148887 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8c600b2880 msgr2=0x7f8c600b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:58.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:58.449+0000 7f8c7dc92700 1 --2- 172.21.15.100:0/3768148887 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8c600b2880 0x7f8c600b4d40 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f8c68005950 tx=0x7f8c6800a300 comp rx=0 tx=0).stop 2024-05-10T11:44:58.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:58.449+0000 7f8c7dc92700 1 -- 172.21.15.100:0/3768148887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c78105f30 msgr2=0x7f8c781a0500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:44:58.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:58.449+0000 7f8c7dc92700 1 --2- 172.21.15.100:0/3768148887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c78105f30 0x7f8c781a0500 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c0059d0 tx=0x7f8c6c0047b0 comp rx=0 tx=0).stop 2024-05-10T11:44:58.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:58.449+0000 7f8c7dc92700 1 -- 172.21.15.100:0/3768148887 shutdown_connections 2024-05-10T11:44:58.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:58.449+0000 7f8c7dc92700 1 --2- 172.21.15.100:0/3768148887 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8c600b2880 0x7f8c600b4d40 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:58.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:58.449+0000 7f8c7dc92700 1 --2- 172.21.15.100:0/3768148887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c781084c0 0x7f8c781a4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:58.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:58.449+0000 7f8c7dc92700 1 --2- 172.21.15.100:0/3768148887 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c7806c150 0x7f8c7819ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:58.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:58.449+0000 7f8c7dc92700 1 --2- 172.21.15.100:0/3768148887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c78105f30 0x7f8c781a0500 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:44:58.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:58.449+0000 7f8c7dc92700 1 -- 172.21.15.100:0/3768148887 >> 172.21.15.100:0/3768148887 conn(0x7f8c780783f0 msgr2=0x7f8c78103d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:44:58.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:58.449+0000 7f8c7dc92700 1 -- 172.21.15.100:0/3768148887 shutdown_connections 2024-05-10T11:44:58.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:44:58.449+0000 7f8c7dc92700 1 -- 172.21.15.100:0/3768148887 wait complete. 2024-05-10T11:44:58.450 INFO:teuthology.orchestra.run.smithi100.stderr:listed 10 entries 2024-05-10T11:44:58.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:44:58 smithi178 ceph-mon[33359]: pgmap v1218: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 7 op/s 2024-05-10T11:44:58.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:44:58 smithi188 ceph-mon[34075]: pgmap v1218: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 7 op/s 2024-05-10T11:44:59.142 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-05-10T11:44:59.142 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T11:44:59.161 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph osd blocklist ls 2024-05-10T11:44:59.446 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:44:59.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:44:59 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/3768148887' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-10T11:44:59.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:44:59 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/3768148887' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-10T11:44:59.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:44:59 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/3768148887' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-10T11:45:00.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:00 smithi178 ceph-mon[33359]: pgmap v1219: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 5 op/s 2024-05-10T11:45:00.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:00 smithi188 ceph-mon[34075]: pgmap v1219: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 5 op/s 2024-05-10T11:45:00.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:00 smithi100 ceph-mon[30296]: pgmap v1219: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 5 op/s 2024-05-10T11:45:01.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:01.188+0000 7f42849c4700 1 -- 172.21.15.100:0/3366078304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4280075ff0 msgr2=0x7f4280114130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:45:01.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:01.188+0000 7f42849c4700 1 --2- 172.21.15.100:0/3366078304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4280075ff0 0x7f4280114130 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f4274009990 tx=0x7f4274009ca0 comp rx=0 tx=0).stop 2024-05-10T11:45:01.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:01.188+0000 7f42849c4700 1 -- 172.21.15.100:0/3366078304 shutdown_connections 2024-05-10T11:45:01.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:01.188+0000 7f42849c4700 1 --2- 172.21.15.100:0/3366078304 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4280114670 0x7f4280116a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:01.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:01.188+0000 7f42849c4700 1 --2- 172.21.15.100:0/3366078304 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42800756d0 0x7f4280075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:01.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:01.188+0000 7f42849c4700 1 --2- 172.21.15.100:0/3366078304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4280075ff0 0x7f4280114130 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:01.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:01.188+0000 7f42849c4700 1 -- 172.21.15.100:0/3366078304 >> 172.21.15.100:0/3366078304 conn(0x7f428007a600 msgr2=0x7f428007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:45:01.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:01.188+0000 7f42849c4700 1 -- 172.21.15.100:0/3366078304 shutdown_connections 2024-05-10T11:45:01.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:01.188+0000 7f42849c4700 1 -- 172.21.15.100:0/3366078304 wait complete. 2024-05-10T11:45:01.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:01.189+0000 7f42849c4700 1 Processor -- start 2024-05-10T11:45:01.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:01.189+0000 7f42849c4700 1 -- start start 2024-05-10T11:45:01.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:01.189+0000 7f42849c4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42800756d0 0x7f42801a00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:45:01.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:01.189+0000 7f42849c4700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4280075ff0 0x7f42801a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:45:01.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:01.189+0000 7f42849c4700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4280114670 0x7f42801a49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:45:01.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:01.189+0000 7f42849c4700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4280119b80 con 0x7f42800756d0 2024-05-10T11:45:01.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:01.189+0000 7f42849c4700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4280119a00 con 0x7f4280075ff0 2024-05-10T11:45:01.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:01.189+0000 7f42849c4700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4280119d00 con 0x7f4280114670 2024-05-10T11:45:01.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:01.189+0000 7f427e59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42800756d0 0x7f42801a00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:45:01.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:01.190+0000 7f427e59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42800756d0 0x7f42801a00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60394/0 (socket says 172.21.15.100:60394) 2024-05-10T11:45:01.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:01.190+0000 7f427e59c700 1 -- 172.21.15.100:0/4019182694 learned_addr learned my addr 172.21.15.100:0/4019182694 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:45:01.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:01.190+0000 7f427dd9b700 1 --2- 172.21.15.100:0/4019182694 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4280075ff0 0x7f42801a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:45:01.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:01.190+0000 7f427ed9d700 1 --2- 172.21.15.100:0/4019182694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4280114670 0x7f42801a49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:45:01.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:01.190+0000 7f427e59c700 1 -- 172.21.15.100:0/4019182694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4280114670 msgr2=0x7f42801a49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:45:01.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:01.190+0000 7f427e59c700 1 --2- 172.21.15.100:0/4019182694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4280114670 0x7f42801a49c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:01.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:01.190+0000 7f427e59c700 1 -- 172.21.15.100:0/4019182694 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4280075ff0 msgr2=0x7f42801a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:45:01.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:01.190+0000 7f427e59c700 1 --2- 172.21.15.100:0/4019182694 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4280075ff0 0x7f42801a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:01.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:01.190+0000 7f427e59c700 1 -- 172.21.15.100:0/4019182694 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42801a5100 con 0x7f42800756d0 2024-05-10T11:45:01.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:01.190+0000 7f427e59c700 1 --2- 172.21.15.100:0/4019182694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42800756d0 0x7f42801a00f0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f427000c920 tx=0x7f427000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:45:01.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:01.190+0000 7f426f7fe700 1 -- 172.21.15.100:0/4019182694 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4270007b00 con 0x7f42800756d0 2024-05-10T11:45:01.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:01.190+0000 7f42849c4700 1 -- 172.21.15.100:0/4019182694 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42801a53f0 con 0x7f42800756d0 2024-05-10T11:45:01.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:01.190+0000 7f426f7fe700 1 -- 172.21.15.100:0/4019182694 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4270004cc0 con 0x7f42800756d0 2024-05-10T11:45:01.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:01.190+0000 7f426f7fe700 1 -- 172.21.15.100:0/4019182694 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42700054a0 con 0x7f42800756d0 2024-05-10T11:45:01.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:01.190+0000 7f42849c4700 1 -- 172.21.15.100:0/4019182694 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42801acc70 con 0x7f42800756d0 2024-05-10T11:45:01.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:01.192+0000 7f42849c4700 1 -- 172.21.15.100:0/4019182694 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f428006a190 con 0x7f42800756d0 2024-05-10T11:45:01.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:01.193+0000 7f426f7fe700 1 -- 172.21.15.100:0/4019182694 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f4270004750 con 0x7f42800756d0 2024-05-10T11:45:01.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:01.193+0000 7f426f7fe700 1 --2- 172.21.15.100:0/4019182694 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f42680b2880 0x7f42680b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:45:01.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:01.193+0000 7f427dd9b700 1 --2- 172.21.15.100:0/4019182694 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f42680b2880 0x7f42680b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:45:01.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:01.193+0000 7f426f7fe700 1 -- 172.21.15.100:0/4019182694 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f42700e21d0 con 0x7f42800756d0 2024-05-10T11:45:01.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:01.194+0000 7f427dd9b700 1 --2- 172.21.15.100:0/4019182694 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f42680b2880 0x7f42680b4d40 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f4274005850 tx=0x7f427401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:45:01.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:01.198+0000 7f426f7fe700 1 -- 172.21.15.100:0/4019182694 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f4270014030 con 0x7f42800756d0 2024-05-10T11:45:01.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:01.424+0000 7f42849c4700 1 -- 172.21.15.100:0/4019182694 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f42801acf00 con 0x7f42800756d0 2024-05-10T11:45:01.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:01.424+0000 7f426f7fe700 1 -- 172.21.15.100:0/4019182694 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v90) ==== 81+0+601 (secure 0 0 0) 0x7f42700a9b90 con 0x7f42800756d0 2024-05-10T11:45:01.424 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.100:6800/1597433518 2024-05-11T11:07:00.092732+0000 2024-05-10T11:45:01.424 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.100:6800/1086450256 2024-05-11T11:06:33.114176+0000 2024-05-10T11:45:01.425 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.100:0/767964410 2024-05-11T11:07:00.092732+0000 2024-05-10T11:45:01.425 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.100:6801/1597433518 2024-05-11T11:07:00.092732+0000 2024-05-10T11:45:01.425 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.100:6801/1086450256 2024-05-11T11:06:33.114176+0000 2024-05-10T11:45:01.425 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.100:0/2770572070 2024-05-11T11:06:33.114176+0000 2024-05-10T11:45:01.425 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.100:0/1379340513 2024-05-11T11:06:33.114176+0000 2024-05-10T11:45:01.425 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.100:0/3025646953 2024-05-11T11:07:00.092732+0000 2024-05-10T11:45:01.425 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.100:0/4099258930 2024-05-11T11:07:00.092732+0000 2024-05-10T11:45:01.425 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.100:0/4267693567 2024-05-11T11:06:33.114176+0000 2024-05-10T11:45:01.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:01.427+0000 7f42849c4700 1 -- 172.21.15.100:0/4019182694 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f42680b2880 msgr2=0x7f42680b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:45:01.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:01.427+0000 7f42849c4700 1 --2- 172.21.15.100:0/4019182694 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f42680b2880 0x7f42680b4d40 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f4274005850 tx=0x7f427401a040 comp rx=0 tx=0).stop 2024-05-10T11:45:01.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:01.427+0000 7f42849c4700 1 -- 172.21.15.100:0/4019182694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42800756d0 msgr2=0x7f42801a00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:45:01.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:01.427+0000 7f42849c4700 1 --2- 172.21.15.100:0/4019182694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42800756d0 0x7f42801a00f0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f427000c920 tx=0x7f427000cce0 comp rx=0 tx=0).stop 2024-05-10T11:45:01.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:01.427+0000 7f42849c4700 1 -- 172.21.15.100:0/4019182694 shutdown_connections 2024-05-10T11:45:01.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:01.427+0000 7f42849c4700 1 --2- 172.21.15.100:0/4019182694 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f42680b2880 0x7f42680b4d40 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:01.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:01.427+0000 7f42849c4700 1 --2- 172.21.15.100:0/4019182694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4280114670 0x7f42801a49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:01.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:01.427+0000 7f42849c4700 1 --2- 172.21.15.100:0/4019182694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42800756d0 0x7f42801a00f0 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:01.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:01.427+0000 7f42849c4700 1 --2- 172.21.15.100:0/4019182694 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4280075ff0 0x7f42801a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:01.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:01.427+0000 7f42849c4700 1 -- 172.21.15.100:0/4019182694 >> 172.21.15.100:0/4019182694 conn(0x7f428007a600 msgr2=0x7f4280113580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:45:01.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:01.427+0000 7f42849c4700 1 -- 172.21.15.100:0/4019182694 shutdown_connections 2024-05-10T11:45:01.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:01.427+0000 7f42849c4700 1 -- 172.21.15.100:0/4019182694 wait complete. 2024-05-10T11:45:01.428 INFO:teuthology.orchestra.run.smithi100.stderr:listed 10 entries 2024-05-10T11:45:02.275 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi100.front.sepia.ceph.com... 2024-05-10T11:45:02.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T11:45:02.276 DEBUG:teuthology.orchestra.run.smithi100:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-10T11:45:02.324 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T11:45:02.324 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi100.stderr:ceph-fuse[86801]: fuse finished with error 0 and tester_r 0 2024-05-10T11:45:02.411 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:02 smithi100 ceph-mon[30296]: pgmap v1220: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 511 B/s wr, 6 op/s 2024-05-10T11:45:02.411 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:02 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/4019182694' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-10T11:45:02.490 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:02 smithi188 ceph-mon[34075]: pgmap v1220: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 511 B/s wr, 6 op/s 2024-05-10T11:45:02.490 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:02 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/4019182694' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-10T11:45:02.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:02 smithi178 ceph-mon[33359]: pgmap v1220: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 511 B/s wr, 6 op/s 2024-05-10T11:45:02.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:02 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/4019182694' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-10T11:45:04.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:04 smithi178 ceph-mon[33359]: pgmap v1221: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-10T11:45:04.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:04 smithi188 ceph-mon[34075]: pgmap v1221: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-10T11:45:04.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:04 smithi100 ceph-mon[30296]: pgmap v1221: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-10T11:45:06.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:06 smithi178 ceph-mon[33359]: pgmap v1222: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-10T11:45:06.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:06 smithi188 ceph-mon[34075]: pgmap v1222: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-10T11:45:06.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:06 smithi100 ceph-mon[30296]: pgmap v1222: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-10T11:45:08.426 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-05-10T11:45:08.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T11:45:08.426 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-10T11:45:08.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T11:45:08.456 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-05-10T11:45:08.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:08 smithi178 ceph-mon[33359]: pgmap v1223: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 853 B/s wr, 8 op/s 2024-05-10T11:45:08.515 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-10T11:45:08.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T11:45:08.515 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-10T11:45:08.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:08 smithi188 ceph-mon[34075]: pgmap v1223: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 853 B/s wr, 8 op/s 2024-05-10T11:45:08.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:08 smithi100 ceph-mon[30296]: pgmap v1223: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 853 B/s wr, 8 op/s 2024-05-10T11:45:08.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T11:45:08.569 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link delete ceph-brx 2024-05-10T11:45:08.681 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-05-10T11:45:08.691 INFO:tasks.cephadm:Teardown begin 2024-05-10T11:45:08.692 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-10T11:45:08.692 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T11:45:08.947 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:45:10.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:10 smithi100 ceph-mon[30296]: pgmap v1224: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 938 B/s wr, 6 op/s 2024-05-10T11:45:10.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:10 smithi178 ceph-mon[33359]: pgmap v1224: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 938 B/s wr, 6 op/s 2024-05-10T11:45:10.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:10 smithi188 ceph-mon[34075]: pgmap v1224: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 938 B/s wr, 6 op/s 2024-05-10T11:45:10.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:10.615+0000 7fe6d0e9d700 1 -- 172.21.15.100:0/3965466657 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6cc109500 msgr2=0x7fe6cc1098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:45:10.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:10.615+0000 7fe6d0e9d700 1 --2- 172.21.15.100:0/3965466657 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6cc109500 0x7fe6cc1098e0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc009990 tx=0x7fe6bc009ca0 comp rx=0 tx=0).stop 2024-05-10T11:45:10.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:10.615+0000 7fe6d0e9d700 1 -- 172.21.15.100:0/3965466657 shutdown_connections 2024-05-10T11:45:10.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:10.615+0000 7fe6d0e9d700 1 --2- 172.21.15.100:0/3965466657 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6cc06c3d0 0x7fe6cc110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:10.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:10.615+0000 7fe6d0e9d700 1 --2- 172.21.15.100:0/3965466657 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6cc109500 0x7fe6cc1098e0 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:10.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:10.615+0000 7fe6d0e9d700 1 --2- 172.21.15.100:0/3965466657 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6cc06ba10 0x7fe6cc06be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:10.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:10.615+0000 7fe6d0e9d700 1 -- 172.21.15.100:0/3965466657 >> 172.21.15.100:0/3965466657 conn(0x7fe6cc079310 msgr2=0x7fe6cc079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:45:10.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:10.615+0000 7fe6d0e9d700 1 -- 172.21.15.100:0/3965466657 shutdown_connections 2024-05-10T11:45:10.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:10.615+0000 7fe6d0e9d700 1 -- 172.21.15.100:0/3965466657 wait complete. 2024-05-10T11:45:10.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:10.615+0000 7fe6d0e9d700 1 Processor -- start 2024-05-10T11:45:10.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:10.616+0000 7fe6d0e9d700 1 -- start start 2024-05-10T11:45:10.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:10.616+0000 7fe6d0e9d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6cc06ba10 0x7fe6cc102460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:45:10.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:10.616+0000 7fe6d0e9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6cc06c3d0 0x7fe6cc1029a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:45:10.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:10.616+0000 7fe6d0e9d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6cc109500 0x7fe6cc102ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:45:10.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:10.616+0000 7fe6d0e9d700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe6cc1134c0 con 0x7fe6cc06c3d0 2024-05-10T11:45:10.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:10.616+0000 7fe6d0e9d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe6cc113340 con 0x7fe6cc06ba10 2024-05-10T11:45:10.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:10.616+0000 7fe6d0e9d700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe6cc113640 con 0x7fe6cc109500 2024-05-10T11:45:10.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:10.616+0000 7fe6c9d9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6cc06c3d0 0x7fe6cc1029a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:45:10.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:10.616+0000 7fe6c9d9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6cc06c3d0 0x7fe6cc1029a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36612/0 (socket says 172.21.15.100:36612) 2024-05-10T11:45:10.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:10.616+0000 7fe6c9d9b700 1 -- 172.21.15.100:0/3708508974 learned_addr learned my addr 172.21.15.100:0/3708508974 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:45:10.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:10.616+0000 7fe6cad9d700 1 --2- 172.21.15.100:0/3708508974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6cc109500 0x7fe6cc102ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:45:10.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:10.616+0000 7fe6ca59c700 1 --2- 172.21.15.100:0/3708508974 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6cc06ba10 0x7fe6cc102460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:45:10.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:10.616+0000 7fe6c9d9b700 1 -- 172.21.15.100:0/3708508974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6cc109500 msgr2=0x7fe6cc102ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:45:10.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:10.616+0000 7fe6c9d9b700 1 --2- 172.21.15.100:0/3708508974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6cc109500 0x7fe6cc102ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:10.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:10.616+0000 7fe6c9d9b700 1 -- 172.21.15.100:0/3708508974 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6cc06ba10 msgr2=0x7fe6cc102460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:45:10.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:10.616+0000 7fe6c9d9b700 1 --2- 172.21.15.100:0/3708508974 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6cc06ba10 0x7fe6cc102460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:10.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:10.616+0000 7fe6c9d9b700 1 -- 172.21.15.100:0/3708508974 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6cc1a7f60 con 0x7fe6cc06c3d0 2024-05-10T11:45:10.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:10.616+0000 7fe6ca59c700 1 --2- 172.21.15.100:0/3708508974 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6cc06ba10 0x7fe6cc102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:45:10.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:10.616+0000 7fe6c9d9b700 1 --2- 172.21.15.100:0/3708508974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6cc06c3d0 0x7fe6cc1029a0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fe6c000d950 tx=0x7fe6c000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:45:10.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:10.617+0000 7fe6bb7fe700 1 -- 172.21.15.100:0/3708508974 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c0016070 con 0x7fe6cc06c3d0 2024-05-10T11:45:10.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:10.617+0000 7fe6d0e9d700 1 -- 172.21.15.100:0/3708508974 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6cc1a8250 con 0x7fe6cc06c3d0 2024-05-10T11:45:10.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:10.617+0000 7fe6d0e9d700 1 -- 172.21.15.100:0/3708508974 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6cc1a87e0 con 0x7fe6cc06c3d0 2024-05-10T11:45:10.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:10.617+0000 7fe6bb7fe700 1 -- 172.21.15.100:0/3708508974 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe6c0009dd0 con 0x7fe6cc06c3d0 2024-05-10T11:45:10.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:10.617+0000 7fe6bb7fe700 1 -- 172.21.15.100:0/3708508974 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c000f3c0 con 0x7fe6cc06c3d0 2024-05-10T11:45:10.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:10.618+0000 7fe6d0e9d700 1 -- 172.21.15.100:0/3708508974 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6cc06a190 con 0x7fe6cc06c3d0 2024-05-10T11:45:10.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:10.619+0000 7fe6bb7fe700 1 -- 172.21.15.100:0/3708508974 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fe6c0019070 con 0x7fe6cc06c3d0 2024-05-10T11:45:10.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:10.619+0000 7fe6bb7fe700 1 --2- 172.21.15.100:0/3708508974 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe6b40b2900 0x7fe6b40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:45:10.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:10.620+0000 7fe6ca59c700 1 --2- 172.21.15.100:0/3708508974 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe6b40b2900 0x7fe6b40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:45:10.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:10.620+0000 7fe6bb7fe700 1 -- 172.21.15.100:0/3708508974 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fe6c00e2130 con 0x7fe6cc06c3d0 2024-05-10T11:45:10.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:10.620+0000 7fe6ca59c700 1 --2- 172.21.15.100:0/3708508974 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe6b40b2900 0x7fe6b40b4dc0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc000c00 tx=0x7fe6bc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:45:10.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:10.623+0000 7fe6bb7fe700 1 -- 172.21.15.100:0/3708508974 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fe6c00a9af0 con 0x7fe6cc06c3d0 2024-05-10T11:45:10.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:10.873+0000 7fe6d0e9d700 1 -- 172.21.15.100:0/3708508974 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe6cc103990 con 0x7fe6cc06c3d0 2024-05-10T11:45:10.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:10.874+0000 7fe6bb7fe700 1 -- 172.21.15.100:0/3708508974 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11903 (secure 0 0 0) 0x7fe6c00a93f0 con 0x7fe6cc06c3d0 2024-05-10T11:45:10.876 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:45:10.876 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2072225012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2072225012},{"type":"v1","addr":"172.21.15.100:6835","nonce":2072225012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/401939540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":401939540},{"type":"v1","addr":"172.21.15.100:6837","nonce":401939540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14571,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/3946311025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3946311025},{"type":"v1","addr":"172.21.15.100:6839","nonce":3946311025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3736371660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3736371660},{"type":"v1","addr":"172.21.15.100:6841","nonce":3736371660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/2741810791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2741810791},{"type":"v1","addr":"172.21.15.178:6833","nonce":2741810791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1704891041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1704891041},{"type":"v1","addr":"172.21.15.188:6833","nonce":1704891041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":44,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T11:14:27.242452+0000","modified":"2024-05-10T11:44:13.980352+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24439,"mds_1":24437,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/2124076312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2124076312},{"type":"v1","addr":"172.21.15.178:6835","nonce":2124076312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/3224358425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3224358425},{"type":"v1","addr":"172.21.15.178:6837","nonce":3224358425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/1943342620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1943342620},{"type":"v1","addr":"172.21.15.188:6837","nonce":1943342620}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.178:6839/3760985404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3760985404},{"type":"v1","addr":"172.21.15.178:6839","nonce":3760985404}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.188:6839/3194917070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":3194917070},{"type":"v1","addr":"172.21.15.188:6839","nonce":3194917070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27034":{"gid":27034,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/3139559450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3139559450},{"type":"v1","addr":"172.21.15.188:6835","nonce":3139559450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24439,"qdb_cluster":[24437,24439,24427]},"id":1}]} 2024-05-10T11:45:10.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:10.877+0000 7fe6d0e9d700 1 -- 172.21.15.100:0/3708508974 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe6b40b2900 msgr2=0x7fe6b40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:45:10.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:10.877+0000 7fe6d0e9d700 1 --2- 172.21.15.100:0/3708508974 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe6b40b2900 0x7fe6b40b4dc0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc000c00 tx=0x7fe6bc005fb0 comp rx=0 tx=0).stop 2024-05-10T11:45:10.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:10.877+0000 7fe6d0e9d700 1 -- 172.21.15.100:0/3708508974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6cc06c3d0 msgr2=0x7fe6cc1029a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:45:10.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:10.877+0000 7fe6d0e9d700 1 --2- 172.21.15.100:0/3708508974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6cc06c3d0 0x7fe6cc1029a0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fe6c000d950 tx=0x7fe6c000dc60 comp rx=0 tx=0).stop 2024-05-10T11:45:10.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:10.877+0000 7fe6d0e9d700 1 -- 172.21.15.100:0/3708508974 shutdown_connections 2024-05-10T11:45:10.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:10.877+0000 7fe6d0e9d700 1 --2- 172.21.15.100:0/3708508974 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fe6b40b2900 0x7fe6b40b4dc0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:10.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:10.877+0000 7fe6d0e9d700 1 --2- 172.21.15.100:0/3708508974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6cc109500 0x7fe6cc102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:10.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:10.877+0000 7fe6d0e9d700 1 --2- 172.21.15.100:0/3708508974 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6cc06ba10 0x7fe6cc102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:10.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:10.877+0000 7fe6d0e9d700 1 --2- 172.21.15.100:0/3708508974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6cc06c3d0 0x7fe6cc1029a0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:10.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:10.877+0000 7fe6d0e9d700 1 -- 172.21.15.100:0/3708508974 >> 172.21.15.100:0/3708508974 conn(0x7fe6cc079310 msgr2=0x7fe6cc1010d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:45:10.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:10.877+0000 7fe6d0e9d700 1 -- 172.21.15.100:0/3708508974 shutdown_connections 2024-05-10T11:45:10.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:10.877+0000 7fe6d0e9d700 1 -- 172.21.15.100:0/3708508974 wait complete. 2024-05-10T11:45:10.880 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 44 2024-05-10T11:45:11.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:11 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/3708508974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:45:11.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:11 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/3708508974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:45:11.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:11 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/3708508974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:45:11.655 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-10T11:45:11.901 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:45:12.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:12 smithi178 ceph-mon[33359]: pgmap v1225: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 938 B/s wr, 7 op/s 2024-05-10T11:45:12.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:12 smithi188 ceph-mon[34075]: pgmap v1225: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 938 B/s wr, 7 op/s 2024-05-10T11:45:12.538 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:12 smithi100 ceph-mon[30296]: pgmap v1225: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 938 B/s wr, 7 op/s 2024-05-10T11:45:13.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:13.633+0000 7f7da679e700 1 -- 172.21.15.100:0/2838827687 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7da0101f30 msgr2=0x7f7da0102310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:45:13.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:13.633+0000 7f7da679e700 1 --2- 172.21.15.100:0/2838827687 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7da0101f30 0x7f7da0102310 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f7d94009990 tx=0x7f7d94009ca0 comp rx=0 tx=0).stop 2024-05-10T11:45:13.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:13.633+0000 7f7da679e700 1 -- 172.21.15.100:0/2838827687 shutdown_connections 2024-05-10T11:45:13.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:13.633+0000 7f7da679e700 1 --2- 172.21.15.100:0/2838827687 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7da01087a0 0x7f7da0110350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:13.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:13.633+0000 7f7da679e700 1 --2- 172.21.15.100:0/2838827687 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7da0101f30 0x7f7da0102310 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:13.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:13.633+0000 7f7da679e700 1 --2- 172.21.15.100:0/2838827687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7da01028e0 0x7f7da01080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:13.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:13.633+0000 7f7da679e700 1 -- 172.21.15.100:0/2838827687 >> 172.21.15.100:0/2838827687 conn(0x7f7da0077650 msgr2=0x7f7da0077a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:45:13.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:13.634+0000 7f7da679e700 1 -- 172.21.15.100:0/2838827687 shutdown_connections 2024-05-10T11:45:13.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:13.634+0000 7f7da679e700 1 -- 172.21.15.100:0/2838827687 wait complete. 2024-05-10T11:45:13.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:13.634+0000 7f7da679e700 1 Processor -- start 2024-05-10T11:45:13.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:13.634+0000 7f7da679e700 1 -- start start 2024-05-10T11:45:13.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:13.635+0000 7f7da679e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7da0101f30 0x7f7da019f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:45:13.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:13.635+0000 7f7da679e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7da01028e0 0x7f7da019fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:45:13.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:13.635+0000 7f7da679e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7da01087a0 0x7f7da01997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:45:13.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:13.635+0000 7f7da679e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7da01134a0 con 0x7f7da01028e0 2024-05-10T11:45:13.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:13.635+0000 7f7da679e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7da0113320 con 0x7f7da01087a0 2024-05-10T11:45:13.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:13.635+0000 7f7da679e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7da0113620 con 0x7f7da0101f30 2024-05-10T11:45:13.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:13.635+0000 7f7da4f9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7da01028e0 0x7f7da019fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:45:13.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:13.635+0000 7f7da4f9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7da01028e0 0x7f7da019fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36620/0 (socket says 172.21.15.100:36620) 2024-05-10T11:45:13.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:13.635+0000 7f7da579c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7da0101f30 0x7f7da019f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:45:13.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:13.635+0000 7f7da4f9b700 1 -- 172.21.15.100:0/1246840389 learned_addr learned my addr 172.21.15.100:0/1246840389 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:45:13.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:13.635+0000 7f7da5f9d700 1 --2- 172.21.15.100:0/1246840389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7da01087a0 0x7f7da01997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:45:13.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:13.635+0000 7f7da4f9b700 1 -- 172.21.15.100:0/1246840389 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7da0101f30 msgr2=0x7f7da019f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:45:13.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:13.635+0000 7f7da4f9b700 1 --2- 172.21.15.100:0/1246840389 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7da0101f30 0x7f7da019f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:13.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:13.635+0000 7f7da4f9b700 1 -- 172.21.15.100:0/1246840389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7da01087a0 msgr2=0x7f7da01997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:45:13.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:13.635+0000 7f7da4f9b700 1 --2- 172.21.15.100:0/1246840389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7da01087a0 0x7f7da01997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:13.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:13.635+0000 7f7da4f9b700 1 -- 172.21.15.100:0/1246840389 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7da0199fa0 con 0x7f7da01028e0 2024-05-10T11:45:13.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:13.635+0000 7f7da4f9b700 1 --2- 172.21.15.100:0/1246840389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7da01028e0 0x7f7da019fb80 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c00d980 tx=0x7f7d9c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:45:13.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:13.635+0000 7f7d927fc700 1 -- 172.21.15.100:0/1246840389 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c017070 con 0x7f7da01028e0 2024-05-10T11:45:13.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:13.636+0000 7f7da679e700 1 -- 172.21.15.100:0/1246840389 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7da019a290 con 0x7f7da01028e0 2024-05-10T11:45:13.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:13.636+0000 7f7d927fc700 1 -- 172.21.15.100:0/1246840389 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7d9c00fd20 con 0x7f7da01028e0 2024-05-10T11:45:13.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:13.636+0000 7f7d927fc700 1 -- 172.21.15.100:0/1246840389 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c00e3b0 con 0x7f7da01028e0 2024-05-10T11:45:13.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:13.636+0000 7f7da679e700 1 -- 172.21.15.100:0/1246840389 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7da0102de0 con 0x7f7da01028e0 2024-05-10T11:45:13.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:13.636+0000 7f7da679e700 1 -- 172.21.15.100:0/1246840389 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7da010b9d0 con 0x7f7da01028e0 2024-05-10T11:45:13.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:13.638+0000 7f7d927fc700 1 -- 172.21.15.100:0/1246840389 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f7d9c00f890 con 0x7f7da01028e0 2024-05-10T11:45:13.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:13.638+0000 7f7d927fc700 1 --2- 172.21.15.100:0/1246840389 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f7d8c0b6c90 0x7f7d8c0b9150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:45:13.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:13.639+0000 7f7da579c700 1 --2- 172.21.15.100:0/1246840389 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f7d8c0b6c90 0x7f7d8c0b9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:45:13.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:13.639+0000 7f7d927fc700 1 -- 172.21.15.100:0/1246840389 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f7d9c0e2000 con 0x7f7da01028e0 2024-05-10T11:45:13.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:13.639+0000 7f7da579c700 1 --2- 172.21.15.100:0/1246840389 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f7d8c0b6c90 0x7f7d8c0b9150 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f7d940098e0 tx=0x7f7d94005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:45:13.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:13.642+0000 7f7d927fc700 1 -- 172.21.15.100:0/1246840389 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f7d9c0a99c0 con 0x7f7da01028e0 2024-05-10T11:45:13.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:13.886+0000 7f7da679e700 1 -- 172.21.15.100:0/1246840389 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7da019acf0 con 0x7f7da01028e0 2024-05-10T11:45:13.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:13.886+0000 7f7d927fc700 1 -- 172.21.15.100:0/1246840389 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11903 (secure 0 0 0) 0x7f7d9c0a92c0 con 0x7f7da01028e0 2024-05-10T11:45:13.887 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:45:13.887 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2072225012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2072225012},{"type":"v1","addr":"172.21.15.100:6835","nonce":2072225012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/401939540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":401939540},{"type":"v1","addr":"172.21.15.100:6837","nonce":401939540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14571,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/3946311025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":3946311025},{"type":"v1","addr":"172.21.15.100:6839","nonce":3946311025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/3736371660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":3736371660},{"type":"v1","addr":"172.21.15.100:6841","nonce":3736371660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/2741810791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2741810791},{"type":"v1","addr":"172.21.15.178:6833","nonce":2741810791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1704891041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1704891041},{"type":"v1","addr":"172.21.15.188:6833","nonce":1704891041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":44,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T11:14:27.242452+0000","modified":"2024-05-10T11:44:13.980352+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24439,"mds_1":24437,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24395":{"gid":24395,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/2124076312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2124076312},{"type":"v1","addr":"172.21.15.178:6835","nonce":2124076312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/3224358425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3224358425},{"type":"v1","addr":"172.21.15.178:6837","nonce":3224358425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.188:6837/1943342620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1943342620},{"type":"v1","addr":"172.21.15.188:6837","nonce":1943342620}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.178:6839/3760985404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3760985404},{"type":"v1","addr":"172.21.15.178:6839","nonce":3760985404}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.188:6839/3194917070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":3194917070},{"type":"v1","addr":"172.21.15.188:6839","nonce":3194917070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27034":{"gid":27034,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/3139559450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3139559450},{"type":"v1","addr":"172.21.15.188:6835","nonce":3139559450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24439,"qdb_cluster":[24437,24439,24427]},"id":1}]} 2024-05-10T11:45:13.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:13.889+0000 7f7da679e700 1 -- 172.21.15.100:0/1246840389 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f7d8c0b6c90 msgr2=0x7f7d8c0b9150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:45:13.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:13.889+0000 7f7da679e700 1 --2- 172.21.15.100:0/1246840389 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f7d8c0b6c90 0x7f7d8c0b9150 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f7d940098e0 tx=0x7f7d94005960 comp rx=0 tx=0).stop 2024-05-10T11:45:13.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:13.889+0000 7f7da679e700 1 -- 172.21.15.100:0/1246840389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7da01028e0 msgr2=0x7f7da019fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:45:13.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:13.889+0000 7f7da679e700 1 --2- 172.21.15.100:0/1246840389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7da01028e0 0x7f7da019fb80 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c00d980 tx=0x7f7d9c00dc90 comp rx=0 tx=0).stop 2024-05-10T11:45:13.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:13.889+0000 7f7da679e700 1 -- 172.21.15.100:0/1246840389 shutdown_connections 2024-05-10T11:45:13.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:13.889+0000 7f7da679e700 1 --2- 172.21.15.100:0/1246840389 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f7d8c0b6c90 0x7f7d8c0b9150 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:13.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:13.889+0000 7f7da679e700 1 --2- 172.21.15.100:0/1246840389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7da01087a0 0x7f7da01997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:13.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:13.889+0000 7f7da679e700 1 --2- 172.21.15.100:0/1246840389 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7da0101f30 0x7f7da019f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:13.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:13.889+0000 7f7da679e700 1 --2- 172.21.15.100:0/1246840389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7da01028e0 0x7f7da019fb80 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:13.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:13.889+0000 7f7da679e700 1 -- 172.21.15.100:0/1246840389 >> 172.21.15.100:0/1246840389 conn(0x7f7da0077650 msgr2=0x7f7da00ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:45:13.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:13.889+0000 7f7da679e700 1 -- 172.21.15.100:0/1246840389 shutdown_connections 2024-05-10T11:45:13.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:13.889+0000 7f7da679e700 1 -- 172.21.15.100:0/1246840389 wait complete. 2024-05-10T11:45:13.890 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 44 2024-05-10T11:45:14.460 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:14 smithi100 ceph-mon[30296]: pgmap v1226: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 8 op/s 2024-05-10T11:45:14.460 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:14 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/1246840389' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:45:14.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:14 smithi178 ceph-mon[33359]: pgmap v1226: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 8 op/s 2024-05-10T11:45:14.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:14 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/1246840389' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:45:14.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:14 smithi188 ceph-mon[34075]: pgmap v1226: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 8 op/s 2024-05-10T11:45:14.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:14 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/1246840389' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T11:45:14.773 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-10T11:45:15.015 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:45:16.477 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:16 smithi100 ceph-mon[30296]: pgmap v1227: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-10T11:45:16.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:16 smithi178 ceph-mon[33359]: pgmap v1227: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-10T11:45:16.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:16 smithi188 ceph-mon[34075]: pgmap v1227: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-10T11:45:16.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:16.701+0000 7f8fdbcfb700 1 -- 172.21.15.100:0/3424367851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8fd4101a90 msgr2=0x7f8fd4101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:45:16.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:16.701+0000 7f8fdbcfb700 1 --2- 172.21.15.100:0/3424367851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8fd4101a90 0x7f8fd4101f10 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4009990 tx=0x7f8fc4009ca0 comp rx=0 tx=0).stop 2024-05-10T11:45:16.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:16.701+0000 7f8fdbcfb700 1 -- 172.21.15.100:0/3424367851 shutdown_connections 2024-05-10T11:45:16.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:16.701+0000 7f8fdbcfb700 1 --2- 172.21.15.100:0/3424367851 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8fd4102450 0x7f8fd4110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:16.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:16.701+0000 7f8fdbcfb700 1 --2- 172.21.15.100:0/3424367851 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fd4109520 0x7f8fd4109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:16.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:16.701+0000 7f8fdbcfb700 1 --2- 172.21.15.100:0/3424367851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8fd4101a90 0x7f8fd4101f10 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:16.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:16.701+0000 7f8fdbcfb700 1 -- 172.21.15.100:0/3424367851 >> 172.21.15.100:0/3424367851 conn(0x7f8fd4078a10 msgr2=0x7f8fd40ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:45:16.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:16.701+0000 7f8fdbcfb700 1 -- 172.21.15.100:0/3424367851 shutdown_connections 2024-05-10T11:45:16.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:16.701+0000 7f8fdbcfb700 1 -- 172.21.15.100:0/3424367851 wait complete. 2024-05-10T11:45:16.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:16.702+0000 7f8fdbcfb700 1 Processor -- start 2024-05-10T11:45:16.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:16.702+0000 7f8fdbcfb700 1 -- start start 2024-05-10T11:45:16.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:16.702+0000 7f8fdbcfb700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fd4101a90 0x7f8fd419f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:45:16.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:16.702+0000 7f8fdbcfb700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8fd4102450 0x7f8fd419fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:45:16.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:16.702+0000 7f8fdbcfb700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8fd4109520 0x7f8fd41998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:45:16.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:16.702+0000 7f8fdbcfb700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd4113560 con 0x7f8fd4102450 2024-05-10T11:45:16.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:16.702+0000 7f8fdbcfb700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd41133e0 con 0x7f8fd4109520 2024-05-10T11:45:16.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:16.702+0000 7f8fdbcfb700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd41136e0 con 0x7f8fd4101a90 2024-05-10T11:45:16.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:16.702+0000 7f8fd9296700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8fd4102450 0x7f8fd419fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:45:16.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:16.702+0000 7f8fd9296700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8fd4102450 0x7f8fd419fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59614/0 (socket says 172.21.15.100:59614) 2024-05-10T11:45:16.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:16.702+0000 7f8fd9296700 1 -- 172.21.15.100:0/3714197242 learned_addr learned my addr 172.21.15.100:0/3714197242 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:45:16.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:16.702+0000 7f8fda298700 1 --2- 172.21.15.100:0/3714197242 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8fd4109520 0x7f8fd41998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:45:16.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:16.702+0000 7f8fd9296700 1 -- 172.21.15.100:0/3714197242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fd4101a90 msgr2=0x7f8fd419f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:45:16.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:16.702+0000 7f8fd9296700 1 --2- 172.21.15.100:0/3714197242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fd4101a90 0x7f8fd419f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:16.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:16.702+0000 7f8fd9296700 1 -- 172.21.15.100:0/3714197242 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8fd4109520 msgr2=0x7f8fd41998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:45:16.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:16.702+0000 7f8fd9296700 1 --2- 172.21.15.100:0/3714197242 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8fd4109520 0x7f8fd41998a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:16.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:16.703+0000 7f8fd9296700 1 -- 172.21.15.100:0/3714197242 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fd419a000 con 0x7f8fd4102450 2024-05-10T11:45:16.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:16.703+0000 7f8fda298700 1 --2- 172.21.15.100:0/3714197242 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8fd4109520 0x7f8fd41998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:45:16.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:16.703+0000 7f8fd9296700 1 --2- 172.21.15.100:0/3714197242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8fd4102450 0x7f8fd419fc20 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4000c00 tx=0x7f8fc40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:45:16.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:16.703+0000 7f8fcaffd700 1 -- 172.21.15.100:0/3714197242 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc4027070 con 0x7f8fd4102450 2024-05-10T11:45:16.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:16.703+0000 7f8fcaffd700 1 -- 172.21.15.100:0/3714197242 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8fc400fdb0 con 0x7f8fd4102450 2024-05-10T11:45:16.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:16.703+0000 7f8fcaffd700 1 -- 172.21.15.100:0/3714197242 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc40173f0 con 0x7f8fd4102450 2024-05-10T11:45:16.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:16.703+0000 7f8fdbcfb700 1 -- 172.21.15.100:0/3714197242 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fd419a290 con 0x7f8fd4102450 2024-05-10T11:45:16.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:16.703+0000 7f8fdbcfb700 1 -- 172.21.15.100:0/3714197242 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fd41a65b0 con 0x7f8fd4102450 2024-05-10T11:45:16.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:16.704+0000 7f8fdbcfb700 1 -- 172.21.15.100:0/3714197242 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fd4104db0 con 0x7f8fd4102450 2024-05-10T11:45:16.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:16.705+0000 7f8fcaffd700 1 -- 172.21.15.100:0/3714197242 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f8fc402a070 con 0x7f8fd4102450 2024-05-10T11:45:16.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:16.706+0000 7f8fcaffd700 1 --2- 172.21.15.100:0/3714197242 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8fc00b2880 0x7f8fc00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:45:16.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:16.706+0000 7f8fd9a97700 1 --2- 172.21.15.100:0/3714197242 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8fc00b2880 0x7f8fc00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:45:16.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:16.706+0000 7f8fcaffd700 1 -- 172.21.15.100:0/3714197242 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f8fc40e3650 con 0x7f8fd4102450 2024-05-10T11:45:16.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:16.706+0000 7f8fd9a97700 1 --2- 172.21.15.100:0/3714197242 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8fc00b2880 0x7f8fc00b4d40 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f8fd419a9e0 tx=0x7f8fd0009250 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:45:16.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:16.711+0000 7f8fcaffd700 1 -- 172.21.15.100:0/3714197242 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f8fc4013070 con 0x7f8fd4102450 2024-05-10T11:45:16.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:16.921+0000 7f8fdbcfb700 1 -- 172.21.15.100:0/3714197242 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8fd402e510 con 0x7f8fd4102450 2024-05-10T11:45:16.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:16.923+0000 7f8fcaffd700 1 -- 172.21.15.100:0/3714197242 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24737 (secure 0 0 0) 0x7f8fc40ab0a0 con 0x7f8fd4102450 2024-05-10T11:45:16.923 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-05-10T11:45:16.924 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":90,"fsid":"20a542cc-0ebd-11ef-bc98-c7b262605968","created":"2024-05-10T11:06:05.454446+0000","modified":"2024-05-10T11:29:07.126022+0000","last_up_change":"2024-05-10T11:13:18.185471+0000","last_in_change":"2024-05-10T11:12:57.268678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T11:09:26.176441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T11:14:21.004285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T11:14:24.179414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T11:14:33.308517+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5f031418-da23-4e85-8669-160960dadd09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":643028278},{"type":"v1","addr":"172.21.15.100:6803","nonce":643028278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":643028278},{"type":"v1","addr":"172.21.15.100:6805","nonce":643028278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":643028278},{"type":"v1","addr":"172.21.15.100:6809","nonce":643028278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":643028278},{"type":"v1","addr":"172.21.15.100:6807","nonce":643028278}]},"public_addr":"172.21.15.100:6803/643028278","cluster_addr":"172.21.15.100:6805/643028278","heartbeat_back_addr":"172.21.15.100:6809/643028278","heartbeat_front_addr":"172.21.15.100:6807/643028278","state":["exists","up"]},{"osd":1,"uuid":"a908cdc9-001e-4b65-9ab0-0dd08dba20f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":3103987017},{"type":"v1","addr":"172.21.15.178:6801","nonce":3103987017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":3103987017},{"type":"v1","addr":"172.21.15.178:6803","nonce":3103987017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":3103987017},{"type":"v1","addr":"172.21.15.178:6807","nonce":3103987017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":3103987017},{"type":"v1","addr":"172.21.15.178:6805","nonce":3103987017}]},"public_addr":"172.21.15.178:6801/3103987017","cluster_addr":"172.21.15.178:6803/3103987017","heartbeat_back_addr":"172.21.15.178:6807/3103987017","heartbeat_front_addr":"172.21.15.178:6805/3103987017","state":["exists","up"]},{"osd":2,"uuid":"4d8f89ae-9424-46e9-9c80-17c47cc5e8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3991466468},{"type":"v1","addr":"172.21.15.188:6801","nonce":3991466468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3991466468},{"type":"v1","addr":"172.21.15.188:6803","nonce":3991466468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3991466468},{"type":"v1","addr":"172.21.15.188:6807","nonce":3991466468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3991466468},{"type":"v1","addr":"172.21.15.188:6805","nonce":3991466468}]},"public_addr":"172.21.15.188:6801/3991466468","cluster_addr":"172.21.15.188:6803/3991466468","heartbeat_back_addr":"172.21.15.188:6807/3991466468","heartbeat_front_addr":"172.21.15.188:6805/3991466468","state":["exists","up"]},{"osd":3,"uuid":"4c8865aa-fef4-4710-b766-1385adb9462c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1713357192},{"type":"v1","addr":"172.21.15.100:6811","nonce":1713357192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1713357192},{"type":"v1","addr":"172.21.15.100:6813","nonce":1713357192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1713357192},{"type":"v1","addr":"172.21.15.100:6817","nonce":1713357192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1713357192},{"type":"v1","addr":"172.21.15.100:6815","nonce":1713357192}]},"public_addr":"172.21.15.100:6811/1713357192","cluster_addr":"172.21.15.100:6813/1713357192","heartbeat_back_addr":"172.21.15.100:6817/1713357192","heartbeat_front_addr":"172.21.15.100:6815/1713357192","state":["exists","up"]},{"osd":4,"uuid":"a9653800-da57-43eb-8781-3188e8a3949f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":2222814192},{"type":"v1","addr":"172.21.15.178:6809","nonce":2222814192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":2222814192},{"type":"v1","addr":"172.21.15.178:6811","nonce":2222814192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":2222814192},{"type":"v1","addr":"172.21.15.178:6815","nonce":2222814192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":2222814192},{"type":"v1","addr":"172.21.15.178:6813","nonce":2222814192}]},"public_addr":"172.21.15.178:6809/2222814192","cluster_addr":"172.21.15.178:6811/2222814192","heartbeat_back_addr":"172.21.15.178:6815/2222814192","heartbeat_front_addr":"172.21.15.178:6813/2222814192","state":["exists","up"]},{"osd":5,"uuid":"fb5f8af2-1eb0-4d8a-a7a3-ed6dd68e692e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3517375935},{"type":"v1","addr":"172.21.15.188:6809","nonce":3517375935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3517375935},{"type":"v1","addr":"172.21.15.188:6811","nonce":3517375935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3517375935},{"type":"v1","addr":"172.21.15.188:6815","nonce":3517375935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3517375935},{"type":"v1","addr":"172.21.15.188:6813","nonce":3517375935}]},"public_addr":"172.21.15.188:6809/3517375935","cluster_addr":"172.21.15.188:6811/3517375935","heartbeat_back_addr":"172.21.15.188:6815/3517375935","heartbeat_front_addr":"172.21.15.188:6813/3517375935","state":["exists","up"]},{"osd":6,"uuid":"84b57387-0a94-4e49-b099-493d329b87dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1410593862},{"type":"v1","addr":"172.21.15.100:6819","nonce":1410593862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1410593862},{"type":"v1","addr":"172.21.15.100:6821","nonce":1410593862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1410593862},{"type":"v1","addr":"172.21.15.100:6825","nonce":1410593862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1410593862},{"type":"v1","addr":"172.21.15.100:6823","nonce":1410593862}]},"public_addr":"172.21.15.100:6819/1410593862","cluster_addr":"172.21.15.100:6821/1410593862","heartbeat_back_addr":"172.21.15.100:6825/1410593862","heartbeat_front_addr":"172.21.15.100:6823/1410593862","state":["exists","up"]},{"osd":7,"uuid":"02ada28f-ac4f-47cf-983f-8a6da1140abe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2458690455},{"type":"v1","addr":"172.21.15.178:6817","nonce":2458690455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2458690455},{"type":"v1","addr":"172.21.15.178:6819","nonce":2458690455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2458690455},{"type":"v1","addr":"172.21.15.178:6823","nonce":2458690455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2458690455},{"type":"v1","addr":"172.21.15.178:6821","nonce":2458690455}]},"public_addr":"172.21.15.178:6817/2458690455","cluster_addr":"172.21.15.178:6819/2458690455","heartbeat_back_addr":"172.21.15.178:6823/2458690455","heartbeat_front_addr":"172.21.15.178:6821/2458690455","state":["exists","up"]},{"osd":8,"uuid":"e7e489bc-15fc-4a4c-b297-d1bb626513ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1523872826},{"type":"v1","addr":"172.21.15.188:6817","nonce":1523872826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1523872826},{"type":"v1","addr":"172.21.15.188:6819","nonce":1523872826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1523872826},{"type":"v1","addr":"172.21.15.188:6823","nonce":1523872826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1523872826},{"type":"v1","addr":"172.21.15.188:6821","nonce":1523872826}]},"public_addr":"172.21.15.188:6817/1523872826","cluster_addr":"172.21.15.188:6819/1523872826","heartbeat_back_addr":"172.21.15.188:6823/1523872826","heartbeat_front_addr":"172.21.15.188:6821/1523872826","state":["exists","up"]},{"osd":9,"uuid":"72328326-cd16-45d9-bfd9-40f6c5c20f5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3585761978},{"type":"v1","addr":"172.21.15.100:6827","nonce":3585761978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3585761978},{"type":"v1","addr":"172.21.15.100:6829","nonce":3585761978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3585761978},{"type":"v1","addr":"172.21.15.100:6833","nonce":3585761978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3585761978},{"type":"v1","addr":"172.21.15.100:6831","nonce":3585761978}]},"public_addr":"172.21.15.100:6827/3585761978","cluster_addr":"172.21.15.100:6829/3585761978","heartbeat_back_addr":"172.21.15.100:6833/3585761978","heartbeat_front_addr":"172.21.15.100:6831/3585761978","state":["exists","up"]},{"osd":10,"uuid":"a355f9a8-91a3-41d4-814d-5a9c9b953706","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2985841725},{"type":"v1","addr":"172.21.15.178:6825","nonce":2985841725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2985841725},{"type":"v1","addr":"172.21.15.178:6827","nonce":2985841725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2985841725},{"type":"v1","addr":"172.21.15.178:6831","nonce":2985841725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2985841725},{"type":"v1","addr":"172.21.15.178:6829","nonce":2985841725}]},"public_addr":"172.21.15.178:6825/2985841725","cluster_addr":"172.21.15.178:6827/2985841725","heartbeat_back_addr":"172.21.15.178:6831/2985841725","heartbeat_front_addr":"172.21.15.178:6829/2985841725","state":["exists","up"]},{"osd":11,"uuid":"76a5e13c-7e38-42f9-bfa7-7fedec7c7184","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2574955118},{"type":"v1","addr":"172.21.15.188:6825","nonce":2574955118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2574955118},{"type":"v1","addr":"172.21.15.188:6827","nonce":2574955118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2574955118},{"type":"v1","addr":"172.21.15.188:6831","nonce":2574955118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2574955118},{"type":"v1","addr":"172.21.15.188:6829","nonce":2574955118}]},"public_addr":"172.21.15.188:6825/2574955118","cluster_addr":"172.21.15.188:6827/2574955118","heartbeat_back_addr":"172.21.15.188:6831/2574955118","heartbeat_front_addr":"172.21.15.188:6829/2574955118","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:08:36.086637+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:08:58.948031+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:09:22.142517+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:09:47.655748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:10:12.853999+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:10:37.911383+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:11:04.448815+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:11:30.331740+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:11:55.819278+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:12:23.204612+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:12:49.484835+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T11:13:15.783531+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:6800/1597433518":"2024-05-11T11:07:00.092732+0000","172.21.15.100:6800/1086450256":"2024-05-11T11:06:33.114176+0000","172.21.15.100:0/767964410":"2024-05-11T11:07:00.092732+0000","172.21.15.100:6801/1597433518":"2024-05-11T11:07:00.092732+0000","172.21.15.100:6801/1086450256":"2024-05-11T11:06:33.114176+0000","172.21.15.100:0/2770572070":"2024-05-11T11:06:33.114176+0000","172.21.15.100:0/1379340513":"2024-05-11T11:06:33.114176+0000","172.21.15.100:0/3025646953":"2024-05-11T11:07:00.092732+0000","172.21.15.100:0/4099258930":"2024-05-11T11:07:00.092732+0000","172.21.15.100:0/4267693567":"2024-05-11T11:06:33.114176+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T11:45:16.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:16.925+0000 7f8fdbcfb700 1 -- 172.21.15.100:0/3714197242 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8fc00b2880 msgr2=0x7f8fc00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:45:16.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:16.925+0000 7f8fdbcfb700 1 --2- 172.21.15.100:0/3714197242 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8fc00b2880 0x7f8fc00b4d40 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f8fd419a9e0 tx=0x7f8fd0009250 comp rx=0 tx=0).stop 2024-05-10T11:45:16.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:16.925+0000 7f8fdbcfb700 1 -- 172.21.15.100:0/3714197242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8fd4102450 msgr2=0x7f8fd419fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:45:16.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:16.925+0000 7f8fdbcfb700 1 --2- 172.21.15.100:0/3714197242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8fd4102450 0x7f8fd419fc20 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4000c00 tx=0x7f8fc40047b0 comp rx=0 tx=0).stop 2024-05-10T11:45:16.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:16.925+0000 7f8fdbcfb700 1 -- 172.21.15.100:0/3714197242 shutdown_connections 2024-05-10T11:45:16.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:16.925+0000 7f8fdbcfb700 1 --2- 172.21.15.100:0/3714197242 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f8fc00b2880 0x7f8fc00b4d40 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:16.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:16.925+0000 7f8fdbcfb700 1 --2- 172.21.15.100:0/3714197242 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8fd4109520 0x7f8fd41998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:16.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:16.925+0000 7f8fdbcfb700 1 --2- 172.21.15.100:0/3714197242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fd4101a90 0x7f8fd419f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:16.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:16.925+0000 7f8fdbcfb700 1 --2- 172.21.15.100:0/3714197242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8fd4102450 0x7f8fd419fc20 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:16.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:16.925+0000 7f8fdbcfb700 1 -- 172.21.15.100:0/3714197242 >> 172.21.15.100:0/3714197242 conn(0x7f8fd4078a10 msgr2=0x7f8fd4100290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:45:16.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:16.925+0000 7f8fdbcfb700 1 -- 172.21.15.100:0/3714197242 shutdown_connections 2024-05-10T11:45:16.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:16.925+0000 7f8fdbcfb700 1 -- 172.21.15.100:0/3714197242 wait complete. 2024-05-10T11:45:17.236 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:17 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/3714197242' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T11:45:17.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:17 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/3714197242' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T11:45:17.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:17 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/3714197242' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T11:45:17.566 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph fs fail cephfs 2024-05-10T11:45:17.806 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:45:18.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:18 smithi100 ceph-mon[30296]: pgmap v1228: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 682 B/s wr, 8 op/s 2024-05-10T11:45:18.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:18 smithi178 ceph-mon[33359]: pgmap v1228: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 682 B/s wr, 8 op/s 2024-05-10T11:45:18.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:18 smithi188 ceph-mon[34075]: pgmap v1228: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 682 B/s wr, 8 op/s 2024-05-10T11:45:19.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:19.539+0000 7fcaf3d94700 1 -- 172.21.15.100:0/2881759847 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcaec1076c0 msgr2=0x7fcaec10f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:45:19.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:19.539+0000 7fcaf3d94700 1 --2- 172.21.15.100:0/2881759847 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcaec1076c0 0x7fcaec10f290 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fcae400b0d0 tx=0x7fcae400b3e0 comp rx=0 tx=0).stop 2024-05-10T11:45:19.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:19.539+0000 7fcaf3d94700 1 -- 172.21.15.100:0/2881759847 shutdown_connections 2024-05-10T11:45:19.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:19.539+0000 7fcaf3d94700 1 --2- 172.21.15.100:0/2881759847 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcaec1076c0 0x7fcaec10f290 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:19.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:19.539+0000 7fcaf3d94700 1 --2- 172.21.15.100:0/2881759847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcaec102630 0x7fcaec102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:19.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:19.539+0000 7fcaf3d94700 1 --2- 172.21.15.100:0/2881759847 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcaec102fe0 0x7fcaec106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:19.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:19.539+0000 7fcaf3d94700 1 -- 172.21.15.100:0/2881759847 >> 172.21.15.100:0/2881759847 conn(0x7fcaec079370 msgr2=0x7fcaec079780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:45:19.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:19.539+0000 7fcaf3d94700 1 -- 172.21.15.100:0/2881759847 shutdown_connections 2024-05-10T11:45:19.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:19.539+0000 7fcaf3d94700 1 -- 172.21.15.100:0/2881759847 wait complete. 2024-05-10T11:45:19.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:19.539+0000 7fcaf3d94700 1 Processor -- start 2024-05-10T11:45:19.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:19.539+0000 7fcaf3d94700 1 -- start start 2024-05-10T11:45:19.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:19.539+0000 7fcaf3d94700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcaec102630 0x7fcaec19f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:45:19.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:19.540+0000 7fcaf3d94700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcaec102fe0 0x7fcaec19fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:45:19.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:19.540+0000 7fcaf3d94700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcaec1076c0 0x7fcaec1997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:45:19.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:19.540+0000 7fcaf3d94700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fcaec112430 con 0x7fcaec102630 2024-05-10T11:45:19.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:19.540+0000 7fcaf3d94700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcaec1122b0 con 0x7fcaec102fe0 2024-05-10T11:45:19.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:19.540+0000 7fcaf3d94700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcaec1125b0 con 0x7fcaec1076c0 2024-05-10T11:45:19.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:19.540+0000 7fcaf1b30700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcaec102630 0x7fcaec19f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:45:19.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:19.540+0000 7fcaf1b30700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcaec102630 0x7fcaec19f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59618/0 (socket says 172.21.15.100:59618) 2024-05-10T11:45:19.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:19.540+0000 7fcaf1b30700 1 -- 172.21.15.100:0/316781171 learned_addr learned my addr 172.21.15.100:0/316781171 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:45:19.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:19.540+0000 7fcaf132f700 1 --2- 172.21.15.100:0/316781171 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcaec102fe0 0x7fcaec19fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:45:19.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:19.540+0000 7fcaf1b30700 1 -- 172.21.15.100:0/316781171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcaec1076c0 msgr2=0x7fcaec1997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:45:19.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:19.540+0000 7fcaf1b30700 1 --2- 172.21.15.100:0/316781171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcaec1076c0 0x7fcaec1997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:19.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:19.540+0000 7fcaf1b30700 1 -- 172.21.15.100:0/316781171 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcaec102fe0 msgr2=0x7fcaec19fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:45:19.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:19.540+0000 7fcaf1b30700 1 --2- 172.21.15.100:0/316781171 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcaec102fe0 0x7fcaec19fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:19.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:19.540+0000 7fcaf1b30700 1 -- 172.21.15.100:0/316781171 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaec199f90 con 0x7fcaec102630 2024-05-10T11:45:19.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:19.540+0000 7fcaf1b30700 1 --2- 172.21.15.100:0/316781171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcaec102630 0x7fcaec19f630 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fcae800d980 tx=0x7fcae800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:45:19.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:19.540+0000 7fcae2ffd700 1 -- 172.21.15.100:0/316781171 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae8017070 con 0x7fcaec102630 2024-05-10T11:45:19.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:19.540+0000 7fcae2ffd700 1 -- 172.21.15.100:0/316781171 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcae8004830 con 0x7fcaec102630 2024-05-10T11:45:19.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:19.540+0000 7fcaf3d94700 1 -- 172.21.15.100:0/316781171 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaec19a280 con 0x7fcaec102630 2024-05-10T11:45:19.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:19.540+0000 7fcae2ffd700 1 -- 172.21.15.100:0/316781171 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae80053a0 con 0x7fcaec102630 2024-05-10T11:45:19.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:19.540+0000 7fcaf3d94700 1 -- 172.21.15.100:0/316781171 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaec1a6490 con 0x7fcaec102630 2024-05-10T11:45:19.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:19.541+0000 7fcaf3d94700 1 -- 172.21.15.100:0/316781171 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcaec103ac0 con 0x7fcaec102630 2024-05-10T11:45:19.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:19.542+0000 7fcae2ffd700 1 -- 172.21.15.100:0/316781171 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fcae80049e0 con 0x7fcaec102630 2024-05-10T11:45:19.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:19.543+0000 7fcae2ffd700 1 --2- 172.21.15.100:0/316781171 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fcad80b2880 0x7fcad80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:45:19.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:19.543+0000 7fcaf132f700 1 --2- 172.21.15.100:0/316781171 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fcad80b2880 0x7fcad80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:45:19.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:19.543+0000 7fcae2ffd700 1 -- 172.21.15.100:0/316781171 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fcae80e2900 con 0x7fcaec102630 2024-05-10T11:45:19.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:19.544+0000 7fcaf132f700 1 --2- 172.21.15.100:0/316781171 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fcad80b2880 0x7fcad80b4d40 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fcadc005fd0 tx=0x7fcadc00af20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:45:19.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:19.546+0000 7fcae2ffd700 1 -- 172.21.15.100:0/316781171 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fcae8020020 con 0x7fcaec102630 2024-05-10T11:45:19.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:19.792+0000 7fcaf3d94700 1 -- 172.21.15.100:0/316781171 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fcaec02e6d0 con 0x7fcaec102630 2024-05-10T11:45:19.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:19.794+0000 7fcae2ffd700 1 -- 172.21.15.100:0/316781171 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v45) ==== 161+0+0 (secure 0 0 0) 0x7fcae80aa2c0 con 0x7fcaec102630 2024-05-10T11:45:19.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:19.797+0000 7fcaf3d94700 1 -- 172.21.15.100:0/316781171 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fcad80b2880 msgr2=0x7fcad80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:45:19.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:19.797+0000 7fcaf3d94700 1 --2- 172.21.15.100:0/316781171 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fcad80b2880 0x7fcad80b4d40 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fcadc005fd0 tx=0x7fcadc00af20 comp rx=0 tx=0).stop 2024-05-10T11:45:19.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:19.797+0000 7fcaf3d94700 1 -- 172.21.15.100:0/316781171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcaec102630 msgr2=0x7fcaec19f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:45:19.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:19.797+0000 7fcaf3d94700 1 --2- 172.21.15.100:0/316781171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcaec102630 0x7fcaec19f630 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fcae800d980 tx=0x7fcae800dd40 comp rx=0 tx=0).stop 2024-05-10T11:45:19.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:19.797+0000 7fcaf3d94700 1 -- 172.21.15.100:0/316781171 shutdown_connections 2024-05-10T11:45:19.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:19.797+0000 7fcaf3d94700 1 --2- 172.21.15.100:0/316781171 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fcad80b2880 0x7fcad80b4d40 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:19.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:19.797+0000 7fcaf3d94700 1 --2- 172.21.15.100:0/316781171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcaec1076c0 0x7fcaec1997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:19.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:19.797+0000 7fcaf3d94700 1 --2- 172.21.15.100:0/316781171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcaec102630 0x7fcaec19f630 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:19.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:19.797+0000 7fcaf3d94700 1 --2- 172.21.15.100:0/316781171 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcaec102fe0 0x7fcaec19fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:19.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:19.797+0000 7fcaf3d94700 1 -- 172.21.15.100:0/316781171 >> 172.21.15.100:0/316781171 conn(0x7fcaec079370 msgr2=0x7fcaec06bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:45:19.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:19.797+0000 7fcaf3d94700 1 -- 172.21.15.100:0/316781171 shutdown_connections 2024-05-10T11:45:19.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:19.797+0000 7fcaf3d94700 1 -- 172.21.15.100:0/316781171 wait complete. 2024-05-10T11:45:19.798 INFO:teuthology.orchestra.run.smithi100.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-10T11:45:20.115 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:19 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mon-a[30292]: 2024-05-10T11:45:19.793+0000 7f0f5d148700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-10T11:45:20.183 INFO:journalctl@ceph.mds.c.smithi188.stdout:May 10 11:45:19 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-c[64244]: ignoring --setuser ceph since I am not root 2024-05-10T11:45:20.184 INFO:journalctl@ceph.mds.c.smithi188.stdout:May 10 11:45:19 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-c[64244]: ignoring --setgroup ceph since I am not root 2024-05-10T11:45:20.184 INFO:journalctl@ceph.mds.c.smithi188.stdout:May 10 11:45:19 smithi188 ceph-mds[64248]: -- 172.21.15.188:0/2840965419 <== mon.1 v2:172.21.15.188:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d6d80fda40 con 0x55d6d80d0c00 2024-05-10T11:45:20.184 INFO:journalctl@ceph.mds.c.smithi188.stdout:May 10 11:45:19 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-c[64244]: starting mds.c at 2024-05-10T11:45:20.184 INFO:journalctl@ceph.mds.h.smithi178.stdout:May 10 11:45:19 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-h[63619]: ignoring --setuser ceph since I am not root 2024-05-10T11:45:20.184 INFO:journalctl@ceph.mds.h.smithi178.stdout:May 10 11:45:19 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-h[63619]: ignoring --setgroup ceph since I am not root 2024-05-10T11:45:20.184 INFO:journalctl@ceph.mds.h.smithi178.stdout:May 10 11:45:19 smithi178 ceph-mds[63623]: -- 172.21.15.178:0/3239857298 <== mon.2 v2:172.21.15.178:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5622490f1a40 con 0x5622490c5000 2024-05-10T11:45:20.184 INFO:journalctl@ceph.mds.h.smithi178.stdout:May 10 11:45:19 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-h[63619]: starting mds.h at 2024-05-10T11:45:20.185 INFO:journalctl@ceph.mds.f.smithi188.stdout:May 10 11:45:19 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-f[63940]: ignoring --setuser ceph since I am not root 2024-05-10T11:45:20.185 INFO:journalctl@ceph.mds.f.smithi188.stdout:May 10 11:45:19 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-f[63940]: ignoring --setgroup ceph since I am not root 2024-05-10T11:45:20.185 INFO:journalctl@ceph.mds.f.smithi188.stdout:May 10 11:45:19 smithi188 ceph-mds[63944]: -- 172.21.15.188:0/4137343851 <== mon.1 v2:172.21.15.188:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5587136bfa40 con 0x558713693000 2024-05-10T11:45:20.185 INFO:journalctl@ceph.mds.f.smithi188.stdout:May 10 11:45:19 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-f[63940]: starting mds.f at 2024-05-10T11:45:20.185 INFO:journalctl@ceph.mds.b.smithi178.stdout:May 10 11:45:19 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-b[63923]: ignoring --setuser ceph since I am not root 2024-05-10T11:45:20.185 INFO:journalctl@ceph.mds.b.smithi178.stdout:May 10 11:45:19 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-b[63923]: ignoring --setgroup ceph since I am not root 2024-05-10T11:45:20.185 INFO:journalctl@ceph.mds.b.smithi178.stdout:May 10 11:45:19 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-b[63923]: starting mds.b at 2024-05-10T11:45:20.186 INFO:journalctl@ceph.mds.i.smithi188.stdout:May 10 11:45:19 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-i[64554]: ignoring --setuser ceph since I am not root 2024-05-10T11:45:20.186 INFO:journalctl@ceph.mds.i.smithi188.stdout:May 10 11:45:19 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-i[64554]: ignoring --setgroup ceph since I am not root 2024-05-10T11:45:20.186 INFO:journalctl@ceph.mds.i.smithi188.stdout:May 10 11:45:19 smithi188 ceph-mds[64558]: -- 172.21.15.188:0/2552141647 <== mon.1 v2:172.21.15.188:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5652e1ecba40 con 0x5652e1e9e800 2024-05-10T11:45:20.186 INFO:journalctl@ceph.mds.i.smithi188.stdout:May 10 11:45:19 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-i[64554]: starting mds.i at 2024-05-10T11:45:20.186 INFO:journalctl@ceph.mds.e.smithi178.stdout:May 10 11:45:19 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-e[63314]: ignoring --setuser ceph since I am not root 2024-05-10T11:45:20.186 INFO:journalctl@ceph.mds.e.smithi178.stdout:May 10 11:45:19 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-e[63314]: ignoring --setgroup ceph since I am not root 2024-05-10T11:45:20.186 INFO:journalctl@ceph.mds.e.smithi178.stdout:May 10 11:45:19 smithi178 ceph-mds[63318]: -- 172.21.15.178:0/1280954431 <== mon.1 v2:172.21.15.188:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b12acf9a40 con 0x55b12accc800 2024-05-10T11:45:20.186 INFO:journalctl@ceph.mds.e.smithi178.stdout:May 10 11:45:19 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-e[63314]: starting mds.e at 2024-05-10T11:45:20.416 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:20 smithi100 ceph-mon[30296]: pgmap v1229: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-10T11:45:20.416 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:20 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/316781171' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-10T11:45:20.417 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:20 smithi100 ceph-mon[30296]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-10T11:45:20.417 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:20 smithi100 ceph-mon[30296]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-10T11:45:20.417 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:20 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/316781171' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-10T11:45:20.417 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:20 smithi100 ceph-mon[30296]: osdmap e91: 12 total, 12 up, 12 in 2024-05-10T11:45:20.417 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:20 smithi100 ceph-mon[30296]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-10T11:45:20.443 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-05-10T11:45:20.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:20 smithi178 ceph-mon[33359]: pgmap v1229: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-10T11:45:20.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:20 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/316781171' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-10T11:45:20.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:20 smithi178 ceph-mon[33359]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-10T11:45:20.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:20 smithi178 ceph-mon[33359]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-10T11:45:20.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:20 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/316781171' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-10T11:45:20.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:20 smithi178 ceph-mon[33359]: osdmap e91: 12 total, 12 up, 12 in 2024-05-10T11:45:20.503 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:20 smithi178 ceph-mon[33359]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-10T11:45:20.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:20 smithi188 ceph-mon[34075]: pgmap v1229: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-10T11:45:20.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:20 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/316781171' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-10T11:45:20.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:20 smithi188 ceph-mon[34075]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-10T11:45:20.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:20 smithi188 ceph-mon[34075]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-10T11:45:20.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:20 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/316781171' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-10T11:45:20.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:20 smithi188 ceph-mon[34075]: osdmap e91: 12 total, 12 up, 12 in 2024-05-10T11:45:20.529 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:20 smithi188 ceph-mon[34075]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-10T11:45:20.689 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:45:22.130 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:21 smithi100 ceph-mon[30296]: pgmap v1231: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-10T11:45:22.130 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:21 smithi100 ceph-mon[30296]: mds.? [v2:172.21.15.178:6834/4288218276,v1:172.21.15.178:6836/4288218276] up:boot 2024-05-10T11:45:22.130 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:21 smithi100 ceph-mon[30296]: mds.? [v2:172.21.15.178:6835/1023012259,v1:172.21.15.178:6837/1023012259] up:boot 2024-05-10T11:45:22.130 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:21 smithi100 ceph-mon[30296]: mds.? [v2:172.21.15.178:6838/3171369723,v1:172.21.15.178:6839/3171369723] up:boot 2024-05-10T11:45:22.130 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:21 smithi100 ceph-mon[30296]: mds.? [v2:172.21.15.188:6834/4249460433,v1:172.21.15.188:6835/4249460433] up:boot 2024-05-10T11:45:22.131 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:21 smithi100 ceph-mon[30296]: mds.? [v2:172.21.15.188:6836/2412040245,v1:172.21.15.188:6838/2412040245] up:boot 2024-05-10T11:45:22.131 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:21 smithi100 ceph-mon[30296]: mds.? [v2:172.21.15.188:6837/2603586310,v1:172.21.15.188:6839/2603586310] up:boot 2024-05-10T11:45:22.131 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:21 smithi100 ceph-mon[30296]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-05-10T11:45:22.131 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:21 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-10T11:45:22.131 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:21 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-10T11:45:22.131 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:21 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-10T11:45:22.131 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:21 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-10T11:45:22.131 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:21 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-10T11:45:22.131 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:21 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-10T11:45:22.131 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:21 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:45:22.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:21 smithi178 ceph-mon[33359]: pgmap v1231: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-10T11:45:22.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:21 smithi178 ceph-mon[33359]: mds.? [v2:172.21.15.178:6834/4288218276,v1:172.21.15.178:6836/4288218276] up:boot 2024-05-10T11:45:22.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:21 smithi178 ceph-mon[33359]: mds.? [v2:172.21.15.178:6835/1023012259,v1:172.21.15.178:6837/1023012259] up:boot 2024-05-10T11:45:22.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:21 smithi178 ceph-mon[33359]: mds.? [v2:172.21.15.178:6838/3171369723,v1:172.21.15.178:6839/3171369723] up:boot 2024-05-10T11:45:22.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:21 smithi178 ceph-mon[33359]: mds.? [v2:172.21.15.188:6834/4249460433,v1:172.21.15.188:6835/4249460433] up:boot 2024-05-10T11:45:22.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:21 smithi178 ceph-mon[33359]: mds.? [v2:172.21.15.188:6836/2412040245,v1:172.21.15.188:6838/2412040245] up:boot 2024-05-10T11:45:22.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:21 smithi178 ceph-mon[33359]: mds.? [v2:172.21.15.188:6837/2603586310,v1:172.21.15.188:6839/2603586310] up:boot 2024-05-10T11:45:22.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:21 smithi178 ceph-mon[33359]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-05-10T11:45:22.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:21 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-10T11:45:22.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:21 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-10T11:45:22.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:21 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-10T11:45:22.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:21 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-10T11:45:22.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:21 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-10T11:45:22.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:21 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-10T11:45:22.254 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:21 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:45:22.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:21 smithi188 ceph-mon[34075]: pgmap v1231: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-10T11:45:22.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:21 smithi188 ceph-mon[34075]: mds.? [v2:172.21.15.178:6834/4288218276,v1:172.21.15.178:6836/4288218276] up:boot 2024-05-10T11:45:22.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:21 smithi188 ceph-mon[34075]: mds.? [v2:172.21.15.178:6835/1023012259,v1:172.21.15.178:6837/1023012259] up:boot 2024-05-10T11:45:22.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:21 smithi188 ceph-mon[34075]: mds.? [v2:172.21.15.178:6838/3171369723,v1:172.21.15.178:6839/3171369723] up:boot 2024-05-10T11:45:22.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:21 smithi188 ceph-mon[34075]: mds.? [v2:172.21.15.188:6834/4249460433,v1:172.21.15.188:6835/4249460433] up:boot 2024-05-10T11:45:22.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:21 smithi188 ceph-mon[34075]: mds.? [v2:172.21.15.188:6836/2412040245,v1:172.21.15.188:6838/2412040245] up:boot 2024-05-10T11:45:22.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:21 smithi188 ceph-mon[34075]: mds.? [v2:172.21.15.188:6837/2603586310,v1:172.21.15.188:6839/2603586310] up:boot 2024-05-10T11:45:22.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:21 smithi188 ceph-mon[34075]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-05-10T11:45:22.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:21 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-10T11:45:22.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:21 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-10T11:45:22.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:21 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-10T11:45:22.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:21 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-10T11:45:22.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:21 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-10T11:45:22.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:21 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-10T11:45:22.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:21 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T11:45:22.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:22.358+0000 7fb837c0a700 1 -- 172.21.15.100:0/3917727540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb830102fe0 msgr2=0x7fb830106fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:45:22.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:22.358+0000 7fb837c0a700 1 --2- 172.21.15.100:0/3917727540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb830102fe0 0x7fb830106fe0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fb8200099c0 tx=0x7fb820009cd0 comp rx=0 tx=0).stop 2024-05-10T11:45:22.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:22.358+0000 7fb837c0a700 1 -- 172.21.15.100:0/3917727540 shutdown_connections 2024-05-10T11:45:22.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:22.358+0000 7fb837c0a700 1 --2- 172.21.15.100:0/3917727540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8301076c0 0x7fb83010f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:22.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:22.358+0000 7fb837c0a700 1 --2- 172.21.15.100:0/3917727540 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb830102630 0x7fb830102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:22.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:22.358+0000 7fb837c0a700 1 --2- 172.21.15.100:0/3917727540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb830102fe0 0x7fb830106fe0 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:22.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:22.358+0000 7fb837c0a700 1 -- 172.21.15.100:0/3917727540 >> 172.21.15.100:0/3917727540 conn(0x7fb830079370 msgr2=0x7fb830079780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:45:22.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:22.358+0000 7fb837c0a700 1 -- 172.21.15.100:0/3917727540 shutdown_connections 2024-05-10T11:45:22.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:22.358+0000 7fb837c0a700 1 -- 172.21.15.100:0/3917727540 wait complete. 2024-05-10T11:45:22.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:22.359+0000 7fb837c0a700 1 Processor -- start 2024-05-10T11:45:22.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:22.359+0000 7fb837c0a700 1 -- start start 2024-05-10T11:45:22.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:22.359+0000 7fb837c0a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb830102630 0x7fb83019f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:45:22.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:22.359+0000 7fb837c0a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb830102fe0 0x7fb83019fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:45:22.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:22.359+0000 7fb837c0a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8301076c0 0x7fb830199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:45:22.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:22.359+0000 7fb837c0a700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb830113560 con 0x7fb830102630 2024-05-10T11:45:22.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:22.359+0000 7fb837c0a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb8301133e0 con 0x7fb8301076c0 2024-05-10T11:45:22.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:22.359+0000 7fb837c0a700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb8301136e0 con 0x7fb830102fe0 2024-05-10T11:45:22.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:22.359+0000 7fb8359a6700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb830102630 0x7fb83019f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:45:22.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:22.359+0000 7fb8359a6700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb830102630 0x7fb83019f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59628/0 (socket says 172.21.15.100:59628) 2024-05-10T11:45:22.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:22.359+0000 7fb8359a6700 1 -- 172.21.15.100:0/3910722300 learned_addr learned my addr 172.21.15.100:0/3910722300 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:45:22.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:22.359+0000 7fb8351a5700 1 --2- 172.21.15.100:0/3910722300 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb830102fe0 0x7fb83019fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:45:22.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:22.360+0000 7fb8361a7700 1 --2- 172.21.15.100:0/3910722300 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8301076c0 0x7fb830199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:45:22.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:22.360+0000 7fb8359a6700 1 -- 172.21.15.100:0/3910722300 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb830102fe0 msgr2=0x7fb83019fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:45:22.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:22.360+0000 7fb8359a6700 1 --2- 172.21.15.100:0/3910722300 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb830102fe0 0x7fb83019fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:22.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:22.360+0000 7fb8359a6700 1 -- 172.21.15.100:0/3910722300 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8301076c0 msgr2=0x7fb830199a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:45:22.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:22.360+0000 7fb8359a6700 1 --2- 172.21.15.100:0/3910722300 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8301076c0 0x7fb830199a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:22.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:22.360+0000 7fb8359a6700 1 -- 172.21.15.100:0/3910722300 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb83019a170 con 0x7fb830102630 2024-05-10T11:45:22.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:22.360+0000 7fb8361a7700 1 --2- 172.21.15.100:0/3910722300 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8301076c0 0x7fb830199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:45:22.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:22.360+0000 7fb8359a6700 1 --2- 172.21.15.100:0/3910722300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb830102630 0x7fb83019f800 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fb82c00b530 tx=0x7fb82c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:45:22.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:22.360+0000 7fb826ffd700 1 -- 172.21.15.100:0/3910722300 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82c015020 con 0x7fb830102630 2024-05-10T11:45:22.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:22.360+0000 7fb837c0a700 1 -- 172.21.15.100:0/3910722300 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb83019a460 con 0x7fb830102630 2024-05-10T11:45:22.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:22.360+0000 7fb826ffd700 1 -- 172.21.15.100:0/3910722300 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb82c011cc0 con 0x7fb830102630 2024-05-10T11:45:22.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:22.360+0000 7fb826ffd700 1 -- 172.21.15.100:0/3910722300 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82c010dc0 con 0x7fb830102630 2024-05-10T11:45:22.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:22.360+0000 7fb837c0a700 1 -- 172.21.15.100:0/3910722300 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8301a66d0 con 0x7fb830102630 2024-05-10T11:45:22.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:22.361+0000 7fb837c0a700 1 -- 172.21.15.100:0/3910722300 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb83006a190 con 0x7fb830102630 2024-05-10T11:45:22.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:22.362+0000 7fb826ffd700 1 -- 172.21.15.100:0/3910722300 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fb82c018020 con 0x7fb830102630 2024-05-10T11:45:22.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:22.363+0000 7fb826ffd700 1 --2- 172.21.15.100:0/3910722300 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fb81c0b2880 0x7fb81c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:45:22.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:22.363+0000 7fb8351a5700 1 --2- 172.21.15.100:0/3910722300 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fb81c0b2880 0x7fb81c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:45:22.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:22.363+0000 7fb826ffd700 1 -- 172.21.15.100:0/3910722300 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fb82c0e1f50 con 0x7fb830102630 2024-05-10T11:45:22.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:22.364+0000 7fb8351a5700 1 --2- 172.21.15.100:0/3910722300 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fb81c0b2880 0x7fb81c0b4d40 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fb820009990 tx=0x7fb820005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:45:22.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:22.368+0000 7fb826ffd700 1 -- 172.21.15.100:0/3910722300 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fb82c01f020 con 0x7fb830102630 2024-05-10T11:45:22.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:22.639+0000 7fb837c0a700 1 -- 172.21.15.100:0/3910722300 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fb83019aef0 con 0x7fb830102630 2024-05-10T11:45:23.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:23.030+0000 7fb826ffd700 1 -- 172.21.15.100:0/3910722300 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v47) ==== 104+0+0 (secure 0 0 0) 0x7fb82c0a9810 con 0x7fb830102630 2024-05-10T11:45:23.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:23.033+0000 7fb837c0a700 1 -- 172.21.15.100:0/3910722300 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fb81c0b2880 msgr2=0x7fb81c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:45:23.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:23.033+0000 7fb837c0a700 1 --2- 172.21.15.100:0/3910722300 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fb81c0b2880 0x7fb81c0b4d40 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fb820009990 tx=0x7fb820005fb0 comp rx=0 tx=0).stop 2024-05-10T11:45:23.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:23.033+0000 7fb837c0a700 1 -- 172.21.15.100:0/3910722300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb830102630 msgr2=0x7fb83019f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:45:23.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:23.033+0000 7fb837c0a700 1 --2- 172.21.15.100:0/3910722300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb830102630 0x7fb83019f800 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fb82c00b530 tx=0x7fb82c00b840 comp rx=0 tx=0).stop 2024-05-10T11:45:23.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:23.033+0000 7fb837c0a700 1 -- 172.21.15.100:0/3910722300 shutdown_connections 2024-05-10T11:45:23.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:23.033+0000 7fb837c0a700 1 --2- 172.21.15.100:0/3910722300 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fb81c0b2880 0x7fb81c0b4d40 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:23.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:23.033+0000 7fb837c0a700 1 --2- 172.21.15.100:0/3910722300 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8301076c0 0x7fb830199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:23.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:23.033+0000 7fb837c0a700 1 --2- 172.21.15.100:0/3910722300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb830102630 0x7fb83019f800 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:23.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:23.033+0000 7fb837c0a700 1 --2- 172.21.15.100:0/3910722300 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb830102fe0 0x7fb83019fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:23.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:23.033+0000 7fb837c0a700 1 -- 172.21.15.100:0/3910722300 >> 172.21.15.100:0/3910722300 conn(0x7fb830079370 msgr2=0x7fb830100850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:45:23.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:23.033+0000 7fb837c0a700 1 -- 172.21.15.100:0/3910722300 shutdown_connections 2024-05-10T11:45:23.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:23.033+0000 7fb837c0a700 1 -- 172.21.15.100:0/3910722300 wait complete. 2024-05-10T11:45:23.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:22 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:45:23.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:22 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:45:23.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:22 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:45:23.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:22 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:45:23.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:22 smithi178 ceph-mon[33359]: Adjusting osd_memory_target on smithi188 to 176.1M 2024-05-10T11:45:23.254 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:22 smithi178 ceph-mon[33359]: Unable to set osd_memory_target on smithi188 to 184753049: error parsing value: Value '184753049' is below minimum 939524096 2024-05-10T11:45:23.254 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:22 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:45:23.254 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:22 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:45:23.254 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:22 smithi178 ceph-mon[33359]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:45:23.254 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:22 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/3910722300' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-10T11:45:23.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:22 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:45:23.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:22 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:45:23.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:22 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:45:23.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:22 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:45:23.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:22 smithi188 ceph-mon[34075]: Adjusting osd_memory_target on smithi188 to 176.1M 2024-05-10T11:45:23.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:22 smithi188 ceph-mon[34075]: Unable to set osd_memory_target on smithi188 to 184753049: error parsing value: Value '184753049' is below minimum 939524096 2024-05-10T11:45:23.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:22 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:45:23.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:22 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:45:23.280 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:22 smithi188 ceph-mon[34075]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:45:23.280 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:22 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/3910722300' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-10T11:45:23.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:22 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:45:23.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:22 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:45:23.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:22 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:45:23.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:22 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-10T11:45:23.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:22 smithi100 ceph-mon[30296]: Adjusting osd_memory_target on smithi188 to 176.1M 2024-05-10T11:45:23.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:22 smithi100 ceph-mon[30296]: Unable to set osd_memory_target on smithi188 to 184753049: error parsing value: Value '184753049' is below minimum 939524096 2024-05-10T11:45:23.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:22 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T11:45:23.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:22 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T11:45:23.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:22 smithi100 ceph-mon[30296]: from='mgr.14152 172.21.15.100:0/3064421823' entity='mgr.x' 2024-05-10T11:45:23.289 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:22 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/3910722300' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-10T11:45:23.686 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-10T11:45:23.930 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:45:24.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:23 smithi100 ceph-mon[30296]: pgmap v1232: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T11:45:24.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:23 smithi100 ceph-mon[30296]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-10T11:45:24.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:23 smithi100 ceph-mon[30296]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-10T11:45:24.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:23 smithi100 ceph-mon[30296]: Cluster is now healthy 2024-05-10T11:45:24.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:23 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/3910722300' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-10T11:45:24.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:23 smithi100 ceph-mon[30296]: fsmap 12 up:standby 2024-05-10T11:45:24.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:23 smithi178 ceph-mon[33359]: pgmap v1232: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T11:45:24.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:23 smithi178 ceph-mon[33359]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-10T11:45:24.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:23 smithi178 ceph-mon[33359]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-10T11:45:24.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:23 smithi178 ceph-mon[33359]: Cluster is now healthy 2024-05-10T11:45:24.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:23 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/3910722300' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-10T11:45:24.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:23 smithi178 ceph-mon[33359]: fsmap 12 up:standby 2024-05-10T11:45:24.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:23 smithi188 ceph-mon[34075]: pgmap v1232: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T11:45:24.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:23 smithi188 ceph-mon[34075]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-10T11:45:24.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:23 smithi188 ceph-mon[34075]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-10T11:45:24.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:23 smithi188 ceph-mon[34075]: Cluster is now healthy 2024-05-10T11:45:24.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:23 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/3910722300' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-10T11:45:24.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:23 smithi188 ceph-mon[34075]: fsmap 12 up:standby 2024-05-10T11:45:25.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:25.703+0000 7f1f840bd700 1 -- 172.21.15.100:0/205415493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f7c06be70 msgr2=0x7f1f7c06c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:45:25.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:25.703+0000 7f1f840bd700 1 --2- 172.21.15.100:0/205415493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f7c06be70 0x7f1f7c06c250 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f1f780099c0 tx=0x7f1f78009cd0 comp rx=0 tx=0).stop 2024-05-10T11:45:25.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:25.703+0000 7f1f840bd700 1 -- 172.21.15.100:0/205415493 shutdown_connections 2024-05-10T11:45:25.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:25.703+0000 7f1f840bd700 1 --2- 172.21.15.100:0/205415493 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f7c1087c0 0x7f1f7c1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:25.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:25.703+0000 7f1f840bd700 1 --2- 172.21.15.100:0/205415493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f7c06be70 0x7f1f7c06c250 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:25.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:25.703+0000 7f1f840bd700 1 --2- 172.21.15.100:0/205415493 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f7c06c820 0x7f1f7c1080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:25.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:25.703+0000 7f1f840bd700 1 -- 172.21.15.100:0/205415493 >> 172.21.15.100:0/205415493 conn(0x7f1f7c0783f0 msgr2=0x7f1f7c078800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:45:25.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:25.703+0000 7f1f840bd700 1 -- 172.21.15.100:0/205415493 shutdown_connections 2024-05-10T11:45:25.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:25.703+0000 7f1f840bd700 1 -- 172.21.15.100:0/205415493 wait complete. 2024-05-10T11:45:25.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:25.704+0000 7f1f840bd700 1 Processor -- start 2024-05-10T11:45:25.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:25.704+0000 7f1f840bd700 1 -- start start 2024-05-10T11:45:25.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:25.704+0000 7f1f840bd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f7c06be70 0x7f1f7c19f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:45:25.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:25.704+0000 7f1f840bd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f7c06c820 0x7f1f7c19fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:45:25.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:25.704+0000 7f1f840bd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f7c1087c0 0x7f1f7c1998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:45:25.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:25.705+0000 7f1f840bd700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1f7c113540 con 0x7f1f7c1087c0 2024-05-10T11:45:25.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:25.705+0000 7f1f840bd700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1f7c1133c0 con 0x7f1f7c06c820 2024-05-10T11:45:25.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:25.705+0000 7f1f840bd700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1f7c1136c0 con 0x7f1f7c06be70 2024-05-10T11:45:25.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:25.705+0000 7f1f8265a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f7c1087c0 0x7f1f7c1998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:45:25.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:25.705+0000 7f1f8265a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f7c1087c0 0x7f1f7c1998e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59648/0 (socket says 172.21.15.100:59648) 2024-05-10T11:45:25.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:25.705+0000 7f1f8265a700 1 -- 172.21.15.100:0/3088160441 learned_addr learned my addr 172.21.15.100:0/3088160441 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:45:25.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:25.705+0000 7f1f81e59700 1 --2- 172.21.15.100:0/3088160441 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f7c06be70 0x7f1f7c19f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:45:25.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:25.705+0000 7f1f81658700 1 --2- 172.21.15.100:0/3088160441 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f7c06c820 0x7f1f7c19fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:45:25.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:25.705+0000 7f1f8265a700 1 -- 172.21.15.100:0/3088160441 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f7c06be70 msgr2=0x7f1f7c19f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:45:25.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:25.705+0000 7f1f8265a700 1 --2- 172.21.15.100:0/3088160441 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f7c06be70 0x7f1f7c19f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:25.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:25.705+0000 7f1f8265a700 1 -- 172.21.15.100:0/3088160441 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f7c06c820 msgr2=0x7f1f7c19fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:45:25.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:25.705+0000 7f1f8265a700 1 --2- 172.21.15.100:0/3088160441 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f7c06c820 0x7f1f7c19fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:25.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:25.705+0000 7f1f8265a700 1 -- 172.21.15.100:0/3088160441 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f7c19a040 con 0x7f1f7c1087c0 2024-05-10T11:45:25.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:25.705+0000 7f1f81658700 1 --2- 172.21.15.100:0/3088160441 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f7c06c820 0x7f1f7c19fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T11:45:25.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:25.705+0000 7f1f8265a700 1 --2- 172.21.15.100:0/3088160441 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f7c1087c0 0x7f1f7c1998e0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f1f7400ada0 tx=0x7f1f7400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:45:25.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:25.705+0000 7f1f72ffd700 1 -- 172.21.15.100:0/3088160441 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f74017020 con 0x7f1f7c1087c0 2024-05-10T11:45:25.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:25.705+0000 7f1f72ffd700 1 -- 172.21.15.100:0/3088160441 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1f7400ce10 con 0x7f1f7c1087c0 2024-05-10T11:45:25.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:25.705+0000 7f1f840bd700 1 -- 172.21.15.100:0/3088160441 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f7c19a330 con 0x7f1f7c1087c0 2024-05-10T11:45:25.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:25.705+0000 7f1f72ffd700 1 -- 172.21.15.100:0/3088160441 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f74005440 con 0x7f1f7c1087c0 2024-05-10T11:45:25.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:25.705+0000 7f1f840bd700 1 -- 172.21.15.100:0/3088160441 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f7c1a65f0 con 0x7f1f7c1087c0 2024-05-10T11:45:25.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:25.706+0000 7f1f840bd700 1 -- 172.21.15.100:0/3088160441 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f7c06a190 con 0x7f1f7c1087c0 2024-05-10T11:45:25.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:25.708+0000 7f1f72ffd700 1 -- 172.21.15.100:0/3088160441 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f1f74004ad0 con 0x7f1f7c1087c0 2024-05-10T11:45:25.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:25.709+0000 7f1f72ffd700 1 --2- 172.21.15.100:0/3088160441 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f1f680b2900 0x7f1f680b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:45:25.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:25.709+0000 7f1f81e59700 1 --2- 172.21.15.100:0/3088160441 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f1f680b2900 0x7f1f680b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:45:25.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:25.709+0000 7f1f72ffd700 1 -- 172.21.15.100:0/3088160441 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f1f740e2370 con 0x7f1f7c1087c0 2024-05-10T11:45:25.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:25.710+0000 7f1f81e59700 1 --2- 172.21.15.100:0/3088160441 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f1f680b2900 0x7f1f680b4dc0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f1f78009990 tx=0x7f1f7801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:45:25.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:25.712+0000 7f1f72ffd700 1 -- 172.21.15.100:0/3088160441 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f1f74021020 con 0x7f1f7c1087c0 2024-05-10T11:45:25.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:25.921+0000 7f1f840bd700 1 -- 172.21.15.100:0/3088160441 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f1f7c02e510 con 0x7f1f7c1087c0 2024-05-10T11:45:26.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:25 smithi100 ceph-mon[30296]: pgmap v1233: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T11:45:26.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:25 smithi178 ceph-mon[33359]: pgmap v1233: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T11:45:26.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:25 smithi188 ceph-mon[34075]: pgmap v1233: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-10T11:45:26.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:26.809+0000 7f1f72ffd700 1 -- 172.21.15.100:0/3088160441 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v92) ==== 181+0+0 (secure 0 0 0) 0x7f1f740a9cc0 con 0x7f1f7c1087c0 2024-05-10T11:45:26.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:26.814+0000 7f1f840bd700 1 -- 172.21.15.100:0/3088160441 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f1f680b2900 msgr2=0x7f1f680b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:45:26.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:26.814+0000 7f1f840bd700 1 --2- 172.21.15.100:0/3088160441 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f1f680b2900 0x7f1f680b4dc0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f1f78009990 tx=0x7f1f7801a040 comp rx=0 tx=0).stop 2024-05-10T11:45:26.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:26.814+0000 7f1f840bd700 1 -- 172.21.15.100:0/3088160441 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f7c1087c0 msgr2=0x7f1f7c1998e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:45:26.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:26.814+0000 7f1f840bd700 1 --2- 172.21.15.100:0/3088160441 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f7c1087c0 0x7f1f7c1998e0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f1f7400ada0 tx=0x7f1f7400c330 comp rx=0 tx=0).stop 2024-05-10T11:45:26.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:26.814+0000 7f1f840bd700 1 -- 172.21.15.100:0/3088160441 shutdown_connections 2024-05-10T11:45:26.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:26.814+0000 7f1f840bd700 1 --2- 172.21.15.100:0/3088160441 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7f1f680b2900 0x7f1f680b4dc0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:26.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:26.814+0000 7f1f840bd700 1 --2- 172.21.15.100:0/3088160441 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f7c1087c0 0x7f1f7c1998e0 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:26.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:26.814+0000 7f1f840bd700 1 --2- 172.21.15.100:0/3088160441 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f7c06be70 0x7f1f7c19f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:26.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:26.814+0000 7f1f840bd700 1 --2- 172.21.15.100:0/3088160441 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f7c06c820 0x7f1f7c19fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:26.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:26.814+0000 7f1f840bd700 1 -- 172.21.15.100:0/3088160441 >> 172.21.15.100:0/3088160441 conn(0x7f1f7c0783f0 msgr2=0x7f1f7c101bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:45:26.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:26.815+0000 7f1f840bd700 1 -- 172.21.15.100:0/3088160441 shutdown_connections 2024-05-10T11:45:26.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:26.815+0000 7f1f840bd700 1 -- 172.21.15.100:0/3088160441 wait complete. 2024-05-10T11:45:26.817 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_metadata' removed 2024-05-10T11:45:27.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:26 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/3088160441' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-10T11:45:27.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:26 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/3088160441' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-10T11:45:27.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:26 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/3088160441' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-10T11:45:27.602 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-10T11:45:27.881 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:45:28.004 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:27 smithi100 ceph-mon[30296]: pgmap v1234: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 2 op/s 2024-05-10T11:45:28.004 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:27 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/3088160441' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-10T11:45:28.004 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:27 smithi100 ceph-mon[30296]: osdmap e92: 12 total, 12 up, 12 in 2024-05-10T11:45:28.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:27 smithi178 ceph-mon[33359]: pgmap v1234: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 2 op/s 2024-05-10T11:45:28.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:27 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/3088160441' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-10T11:45:28.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:27 smithi178 ceph-mon[33359]: osdmap e92: 12 total, 12 up, 12 in 2024-05-10T11:45:28.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:27 smithi188 ceph-mon[34075]: pgmap v1234: 129 pgs: 129 active+clean; 347 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 2 op/s 2024-05-10T11:45:28.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:27 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/3088160441' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-10T11:45:28.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:27 smithi188 ceph-mon[34075]: osdmap e92: 12 total, 12 up, 12 in 2024-05-10T11:45:29.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:29.714+0000 7fec246f1700 1 -- 172.21.15.100:0/3261436284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec1c06bb40 msgr2=0x7fec1c110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:45:29.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:29.714+0000 7fec246f1700 1 --2- 172.21.15.100:0/3261436284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec1c06bb40 0x7fec1c110330 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fec14009990 tx=0x7fec14009ca0 comp rx=0 tx=0).stop 2024-05-10T11:45:29.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:29.714+0000 7fec246f1700 1 -- 172.21.15.100:0/3261436284 shutdown_connections 2024-05-10T11:45:29.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:29.714+0000 7fec246f1700 1 --2- 172.21.15.100:0/3261436284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec1c06bb40 0x7fec1c110330 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:29.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:29.714+0000 7fec246f1700 1 --2- 172.21.15.100:0/3261436284 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec1c109510 0x7fec1c1098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:29.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:29.714+0000 7fec246f1700 1 --2- 172.21.15.100:0/3261436284 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec1c06b180 0x7fec1c06b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:29.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:29.714+0000 7fec246f1700 1 -- 172.21.15.100:0/3261436284 >> 172.21.15.100:0/3261436284 conn(0x7fec1c079350 msgr2=0x7fec1c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:45:29.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:29.714+0000 7fec246f1700 1 -- 172.21.15.100:0/3261436284 shutdown_connections 2024-05-10T11:45:29.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:29.714+0000 7fec246f1700 1 -- 172.21.15.100:0/3261436284 wait complete. 2024-05-10T11:45:29.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:29.715+0000 7fec246f1700 1 Processor -- start 2024-05-10T11:45:29.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:29.715+0000 7fec246f1700 1 -- start start 2024-05-10T11:45:29.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:29.715+0000 7fec246f1700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec1c06b180 0x7fec1c102470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:45:29.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:29.715+0000 7fec246f1700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec1c06bb40 0x7fec1c1029b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:45:29.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:29.715+0000 7fec246f1700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec1c109510 0x7fec1c102ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:45:29.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:29.715+0000 7fec246f1700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fec1c113550 con 0x7fec1c06bb40 2024-05-10T11:45:29.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:29.715+0000 7fec246f1700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fec1c1133d0 con 0x7fec1c109510 2024-05-10T11:45:29.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:29.715+0000 7fec246f1700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fec1c1136d0 con 0x7fec1c06b180 2024-05-10T11:45:29.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:29.715+0000 7fec21c8c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec1c06bb40 0x7fec1c1029b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:45:29.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:29.715+0000 7fec21c8c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec1c06bb40 0x7fec1c1029b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58062/0 (socket says 172.21.15.100:58062) 2024-05-10T11:45:29.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:29.715+0000 7fec21c8c700 1 -- 172.21.15.100:0/1362231933 learned_addr learned my addr 172.21.15.100:0/1362231933 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:45:29.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:29.716+0000 7fec2248d700 1 --2- 172.21.15.100:0/1362231933 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec1c06b180 0x7fec1c102470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:45:29.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:29.716+0000 7fec22c8e700 1 --2- 172.21.15.100:0/1362231933 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec1c109510 0x7fec1c102ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:45:29.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:29.716+0000 7fec21c8c700 1 -- 172.21.15.100:0/1362231933 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec1c06b180 msgr2=0x7fec1c102470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:45:29.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:29.716+0000 7fec21c8c700 1 --2- 172.21.15.100:0/1362231933 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec1c06b180 0x7fec1c102470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:29.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:29.716+0000 7fec21c8c700 1 -- 172.21.15.100:0/1362231933 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec1c109510 msgr2=0x7fec1c102ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:45:29.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:29.716+0000 7fec21c8c700 1 --2- 172.21.15.100:0/1362231933 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec1c109510 0x7fec1c102ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:29.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:29.716+0000 7fec21c8c700 1 -- 172.21.15.100:0/1362231933 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec1c1a7fb0 con 0x7fec1c06bb40 2024-05-10T11:45:29.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:29.716+0000 7fec21c8c700 1 --2- 172.21.15.100:0/1362231933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec1c06bb40 0x7fec1c1029b0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fec0c00d950 tx=0x7fec0c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:45:29.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:29.716+0000 7fec137fe700 1 -- 172.21.15.100:0/1362231933 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec0c016070 con 0x7fec1c06bb40 2024-05-10T11:45:29.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:29.716+0000 7fec246f1700 1 -- 172.21.15.100:0/1362231933 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec1c1a82a0 con 0x7fec1c06bb40 2024-05-10T11:45:29.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:29.716+0000 7fec137fe700 1 -- 172.21.15.100:0/1362231933 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fec0c009dd0 con 0x7fec1c06bb40 2024-05-10T11:45:29.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:29.716+0000 7fec137fe700 1 -- 172.21.15.100:0/1362231933 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec0c00f3c0 con 0x7fec1c06bb40 2024-05-10T11:45:29.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:29.716+0000 7fec246f1700 1 -- 172.21.15.100:0/1362231933 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec1c1a8830 con 0x7fec1c06bb40 2024-05-10T11:45:29.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:29.718+0000 7fec246f1700 1 -- 172.21.15.100:0/1362231933 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec1c10b9f0 con 0x7fec1c06bb40 2024-05-10T11:45:29.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:29.719+0000 7fec137fe700 1 -- 172.21.15.100:0/1362231933 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fec0c019070 con 0x7fec1c06bb40 2024-05-10T11:45:29.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:29.719+0000 7fec137fe700 1 --2- 172.21.15.100:0/1362231933 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fec080b2900 0x7fec080b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:45:29.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:29.719+0000 7fec2248d700 1 --2- 172.21.15.100:0/1362231933 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fec080b2900 0x7fec080b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:45:29.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:29.719+0000 7fec137fe700 1 -- 172.21.15.100:0/1362231933 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8854+0+0 (secure 0 0 0) 0x7fec0c0e1de0 con 0x7fec1c06bb40 2024-05-10T11:45:29.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:29.720+0000 7fec2248d700 1 --2- 172.21.15.100:0/1362231933 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fec080b2900 0x7fec080b4dc0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fec1800a400 tx=0x7fec1800d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:45:29.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:29.724+0000 7fec137fe700 1 -- 172.21.15.100:0/1362231933 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fec0c0a9890 con 0x7fec1c06bb40 2024-05-10T11:45:29.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:29.926+0000 7fec246f1700 1 -- 172.21.15.100:0/1362231933 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fec1c1039a0 con 0x7fec1c06bb40 2024-05-10T11:45:30.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:29 smithi100 ceph-mon[30296]: pgmap v1236: 97 pgs: 97 active+clean; 577 KiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:45:30.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:29 smithi178 ceph-mon[33359]: pgmap v1236: 97 pgs: 97 active+clean; 577 KiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:45:30.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:29 smithi188 ceph-mon[34075]: pgmap v1236: 97 pgs: 97 active+clean; 577 KiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:45:30.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:30.819+0000 7fec137fe700 1 -- 172.21.15.100:0/1362231933 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v93) ==== 169+0+0 (secure 0 0 0) 0x7fec0c0a9190 con 0x7fec1c06bb40 2024-05-10T11:45:30.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:30.822+0000 7fec246f1700 1 -- 172.21.15.100:0/1362231933 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fec080b2900 msgr2=0x7fec080b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:45:30.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:30.822+0000 7fec246f1700 1 --2- 172.21.15.100:0/1362231933 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fec080b2900 0x7fec080b4dc0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fec1800a400 tx=0x7fec1800d040 comp rx=0 tx=0).stop 2024-05-10T11:45:30.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:30.822+0000 7fec246f1700 1 -- 172.21.15.100:0/1362231933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec1c06bb40 msgr2=0x7fec1c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:45:30.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:30.822+0000 7fec246f1700 1 --2- 172.21.15.100:0/1362231933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec1c06bb40 0x7fec1c1029b0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fec0c00d950 tx=0x7fec0c00dc60 comp rx=0 tx=0).stop 2024-05-10T11:45:30.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:30.822+0000 7fec246f1700 1 -- 172.21.15.100:0/1362231933 shutdown_connections 2024-05-10T11:45:30.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:30.822+0000 7fec246f1700 1 --2- 172.21.15.100:0/1362231933 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fec080b2900 0x7fec080b4dc0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:30.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:30.822+0000 7fec246f1700 1 --2- 172.21.15.100:0/1362231933 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec1c109510 0x7fec1c102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:30.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:30.822+0000 7fec246f1700 1 --2- 172.21.15.100:0/1362231933 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec1c06b180 0x7fec1c102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:30.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:30.822+0000 7fec246f1700 1 --2- 172.21.15.100:0/1362231933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec1c06bb40 0x7fec1c1029b0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:30.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:30.822+0000 7fec246f1700 1 -- 172.21.15.100:0/1362231933 >> 172.21.15.100:0/1362231933 conn(0x7fec1c079350 msgr2=0x7fec1c1011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:45:30.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:30.822+0000 7fec246f1700 1 -- 172.21.15.100:0/1362231933 shutdown_connections 2024-05-10T11:45:30.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:30.822+0000 7fec246f1700 1 -- 172.21.15.100:0/1362231933 wait complete. 2024-05-10T11:45:30.824 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_data' removed 2024-05-10T11:45:31.147 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:30 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/1362231933' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-10T11:45:31.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:30 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/1362231933' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-10T11:45:31.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:30 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/1362231933' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-10T11:45:31.674 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b shell --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-05-10T11:45:31.922 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T11:45:32.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:31 smithi100 ceph-mon[30296]: pgmap v1237: 97 pgs: 97 active+clean; 577 KiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:45:32.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:31 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/1362231933' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-10T11:45:32.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:31 smithi100 ceph-mon[30296]: osdmap e93: 12 total, 12 up, 12 in 2024-05-10T11:45:32.180 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:31 smithi188 ceph-mon[34075]: pgmap v1237: 97 pgs: 97 active+clean; 577 KiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:45:32.180 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:31 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/1362231933' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-10T11:45:32.180 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:31 smithi188 ceph-mon[34075]: osdmap e93: 12 total, 12 up, 12 in 2024-05-10T11:45:32.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:31 smithi178 ceph-mon[33359]: pgmap v1237: 97 pgs: 97 active+clean; 577 KiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:45:32.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:31 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/1362231933' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-10T11:45:32.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:31 smithi178 ceph-mon[33359]: osdmap e93: 12 total, 12 up, 12 in 2024-05-10T11:45:33.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:33.695+0000 7feffdb10700 1 -- 172.21.15.100:0/75084989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feff8103550 msgr2=0x7feff81039b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:45:33.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:33.695+0000 7feffdb10700 1 --2- 172.21.15.100:0/75084989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feff8103550 0x7feff81039b0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fefe0009990 tx=0x7fefe0009ca0 comp rx=0 tx=0).stop 2024-05-10T11:45:33.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:33.695+0000 7feffdb10700 1 -- 172.21.15.100:0/75084989 shutdown_connections 2024-05-10T11:45:33.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:33.695+0000 7feffdb10700 1 --2- 172.21.15.100:0/75084989 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feff8103ef0 0x7feff81103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:33.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:33.695+0000 7feffdb10700 1 --2- 172.21.15.100:0/75084989 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feff8108460 0x7feff8108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:33.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:33.695+0000 7feffdb10700 1 --2- 172.21.15.100:0/75084989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feff8103550 0x7feff81039b0 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:33.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:33.695+0000 7feffdb10700 1 -- 172.21.15.100:0/75084989 >> 172.21.15.100:0/75084989 conn(0x7feff807a600 msgr2=0x7feff807aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:45:33.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:33.695+0000 7feffdb10700 1 -- 172.21.15.100:0/75084989 shutdown_connections 2024-05-10T11:45:33.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:33.695+0000 7feffdb10700 1 -- 172.21.15.100:0/75084989 wait complete. 2024-05-10T11:45:33.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:33.696+0000 7feffdb10700 1 Processor -- start 2024-05-10T11:45:33.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:33.696+0000 7feffdb10700 1 -- start start 2024-05-10T11:45:33.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:33.696+0000 7feffdb10700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feff8103550 0x7feff80757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:45:33.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:33.696+0000 7feffdb10700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feff8103ef0 0x7feff8075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:45:33.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:33.696+0000 7feffdb10700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feff8108460 0x7feff8076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:45:33.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:33.696+0000 7feffdb10700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7feff8113540 con 0x7feff8103ef0 2024-05-10T11:45:33.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:33.696+0000 7feffdb10700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7feff81133c0 con 0x7feff8108460 2024-05-10T11:45:33.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:33.696+0000 7feffdb10700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7feff81136c0 con 0x7feff8103550 2024-05-10T11:45:33.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:33.697+0000 7feff6ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feff8103ef0 0x7feff8075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:45:33.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:33.697+0000 7feff6ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feff8103ef0 0x7feff8075ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58090/0 (socket says 172.21.15.100:58090) 2024-05-10T11:45:33.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:33.697+0000 7feff6ffd700 1 -- 172.21.15.100:0/2230246734 learned_addr learned my addr 172.21.15.100:0/2230246734 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-05-10T11:45:33.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:33.697+0000 7feff77fe700 1 --2- 172.21.15.100:0/2230246734 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feff8103550 0x7feff80757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:45:33.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:33.697+0000 7feff7fff700 1 --2- 172.21.15.100:0/2230246734 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feff8108460 0x7feff8076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:45:33.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:33.697+0000 7feff6ffd700 1 -- 172.21.15.100:0/2230246734 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feff8103550 msgr2=0x7feff80757a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:45:33.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:33.697+0000 7feff6ffd700 1 --2- 172.21.15.100:0/2230246734 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feff8103550 0x7feff80757a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:33.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:33.697+0000 7feff6ffd700 1 -- 172.21.15.100:0/2230246734 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feff8108460 msgr2=0x7feff8076220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:45:33.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:33.697+0000 7feff6ffd700 1 --2- 172.21.15.100:0/2230246734 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feff8108460 0x7feff8076220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:33.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:33.697+0000 7feff6ffd700 1 -- 172.21.15.100:0/2230246734 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feff81a7fe0 con 0x7feff8103ef0 2024-05-10T11:45:33.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:33.697+0000 7feff77fe700 1 --2- 172.21.15.100:0/2230246734 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feff8103550 0x7feff80757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T11:45:33.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:33.697+0000 7feff6ffd700 1 --2- 172.21.15.100:0/2230246734 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feff8103ef0 0x7feff8075ce0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fefe0006010 tx=0x7fefe000bfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:45:33.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:33.697+0000 7feff4ff9700 1 -- 172.21.15.100:0/2230246734 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe0027070 con 0x7feff8103ef0 2024-05-10T11:45:33.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:33.697+0000 7feffdb10700 1 -- 172.21.15.100:0/2230246734 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feff81a8270 con 0x7feff8103ef0 2024-05-10T11:45:33.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:33.697+0000 7feff4ff9700 1 -- 172.21.15.100:0/2230246734 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fefe0022470 con 0x7feff8103ef0 2024-05-10T11:45:33.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:33.697+0000 7feff4ff9700 1 -- 172.21.15.100:0/2230246734 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe0017650 con 0x7feff8103ef0 2024-05-10T11:45:33.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:33.697+0000 7feffdb10700 1 -- 172.21.15.100:0/2230246734 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feff81a8720 con 0x7feff8103ef0 2024-05-10T11:45:33.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:33.698+0000 7feffdb10700 1 -- 172.21.15.100:0/2230246734 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feff806a190 con 0x7feff8103ef0 2024-05-10T11:45:33.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:33.700+0000 7feff4ff9700 1 -- 172.21.15.100:0/2230246734 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fefe0009d10 con 0x7feff8103ef0 2024-05-10T11:45:33.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:33.700+0000 7feff4ff9700 1 --2- 172.21.15.100:0/2230246734 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fefe40b2880 0x7fefe40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T11:45:33.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:33.700+0000 7feff77fe700 1 --2- 172.21.15.100:0/2230246734 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fefe40b2880 0x7fefe40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T11:45:33.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:33.700+0000 7feff4ff9700 1 -- 172.21.15.100:0/2230246734 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8324+0+0 (secure 0 0 0) 0x7fefe00e3210 con 0x7feff8103ef0 2024-05-10T11:45:33.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:33.701+0000 7feff77fe700 1 --2- 172.21.15.100:0/2230246734 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fefe40b2880 0x7fefe40b4d40 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7feff8104730 tx=0x7fefe8009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T11:45:33.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:33.704+0000 7feff4ff9700 1 -- 172.21.15.100:0/2230246734 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fefe0013070 con 0x7feff8103ef0 2024-05-10T11:45:33.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:33.906+0000 7feffdb10700 1 -- 172.21.15.100:0/2230246734 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7feff802e510 con 0x7feff8103ef0 2024-05-10T11:45:34.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:33 smithi100 ceph-mon[30296]: pgmap v1239: 33 pgs: 33 active+clean; 577 KiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:45:34.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:33 smithi178 ceph-mon[33359]: pgmap v1239: 33 pgs: 33 active+clean; 577 KiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:45:34.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:33 smithi188 ceph-mon[34075]: pgmap v1239: 33 pgs: 33 active+clean; 577 KiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:45:34.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:34.828+0000 7feff4ff9700 1 -- 172.21.15.100:0/2230246734 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v94) ==== 178+0+0 (secure 0 0 0) 0x7fefe00aaf60 con 0x7feff8103ef0 2024-05-10T11:45:34.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:34.832+0000 7feffdb10700 1 -- 172.21.15.100:0/2230246734 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fefe40b2880 msgr2=0x7fefe40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:45:34.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:34.832+0000 7feffdb10700 1 --2- 172.21.15.100:0/2230246734 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fefe40b2880 0x7fefe40b4d40 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7feff8104730 tx=0x7fefe8009040 comp rx=0 tx=0).stop 2024-05-10T11:45:34.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:34.832+0000 7feffdb10700 1 -- 172.21.15.100:0/2230246734 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feff8103ef0 msgr2=0x7feff8075ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T11:45:34.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:34.832+0000 7feffdb10700 1 --2- 172.21.15.100:0/2230246734 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feff8103ef0 0x7feff8075ce0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fefe0006010 tx=0x7fefe000bfc0 comp rx=0 tx=0).stop 2024-05-10T11:45:34.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:34.832+0000 7feffdb10700 1 -- 172.21.15.100:0/2230246734 shutdown_connections 2024-05-10T11:45:34.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:34.832+0000 7feffdb10700 1 --2- 172.21.15.100:0/2230246734 >> [v2:172.21.15.100:6800/2507287356,v1:172.21.15.100:6801/2507287356] conn(0x7fefe40b2880 0x7fefe40b4d40 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:34.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:34.832+0000 7feffdb10700 1 --2- 172.21.15.100:0/2230246734 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feff8108460 0x7feff8076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:34.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:34.832+0000 7feffdb10700 1 --2- 172.21.15.100:0/2230246734 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feff8103550 0x7feff80757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:34.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:34.832+0000 7feffdb10700 1 --2- 172.21.15.100:0/2230246734 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feff8103ef0 0x7feff8075ce0 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T11:45:34.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:34.832+0000 7feffdb10700 1 -- 172.21.15.100:0/2230246734 >> 172.21.15.100:0/2230246734 conn(0x7feff807a600 msgr2=0x7feff81016a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T11:45:34.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:34.833+0000 7feffdb10700 1 -- 172.21.15.100:0/2230246734 shutdown_connections 2024-05-10T11:45:34.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-05-10T11:45:34.834+0000 7feffdb10700 1 -- 172.21.15.100:0/2230246734 wait complete. 2024-05-10T11:45:34.840 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_data_ec' removed 2024-05-10T11:45:35.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:34 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/2230246734' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-10T11:45:35.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:34 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/2230246734' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-10T11:45:35.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:34 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/2230246734' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-10T11:45:35.773 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-10T11:45:35.822 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-10T11:45:35.851 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-10T11:45:35.879 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-05-10T11:45:35.879 DEBUG:teuthology.orchestra.run.smithi100:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-10T11:45:35.894 DEBUG:teuthology.orchestra.run.smithi178:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-10T11:45:35.910 DEBUG:teuthology.orchestra.run.smithi188:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-10T11:45:35.934 INFO:tasks.cephadm:Stopping all daemons... 2024-05-10T11:45:35.935 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-05-10T11:45:35.935 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mon.a 2024-05-10T11:45:36.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:35 smithi100 ceph-mon[30296]: pgmap v1240: 33 pgs: 33 active+clean; 577 KiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:45:36.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:35 smithi100 ceph-mon[30296]: from='client.? 172.21.15.100:0/2230246734' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-05-10T11:45:36.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:35 smithi100 ceph-mon[30296]: osdmap e94: 12 total, 12 up, 12 in 2024-05-10T11:45:36.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:35 smithi100 systemd[1]: Stopping Ceph mon.a for 20a542cc-0ebd-11ef-bc98-c7b262605968... 2024-05-10T11:45:36.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:35 smithi178 ceph-mon[33359]: pgmap v1240: 33 pgs: 33 active+clean; 577 KiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:45:36.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:35 smithi178 ceph-mon[33359]: from='client.? 172.21.15.100:0/2230246734' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-05-10T11:45:36.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:35 smithi178 ceph-mon[33359]: osdmap e94: 12 total, 12 up, 12 in 2024-05-10T11:45:36.278 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:35 smithi188 ceph-mon[34075]: pgmap v1240: 33 pgs: 33 active+clean; 577 KiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T11:45:36.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:35 smithi188 ceph-mon[34075]: from='client.? 172.21.15.100:0/2230246734' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-05-10T11:45:36.279 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:35 smithi188 ceph-mon[34075]: osdmap e94: 12 total, 12 up, 12 in 2024-05-10T11:45:36.409 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:36 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mon-a[30292]: 2024-05-10T11:45:36.256+0000 7f0f62953700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T11:45:36.409 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:36 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mon-a[30292]: 2024-05-10T11:45:36.256+0000 7f0f62953700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-05-10T11:45:36.715 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:36 smithi100 podman[140536]: 2024-05-10 11:45:36.410007484 +0000 UTC m=+0.275413413 container died 0469b48457b91a3da3da0578c68bc9d9af67a2aff2d0f427168c619c1078da34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mon-a, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-05-10T11:45:37.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:36 smithi100 podman[140536]: 2024-05-10 11:45:36.815539937 +0000 UTC m=+0.680945821 container cleanup 0469b48457b91a3da3da0578c68bc9d9af67a2aff2d0f427168c619c1078da34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mon-a, GIT_CLEAN=True, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-05-10T11:45:37.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:May 10 11:45:36 smithi100 bash[140536]: ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mon-a 2024-05-10T11:45:37.167 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mon.a.service' 2024-05-10T11:45:37.219 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T11:45:37.219 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-05-10T11:45:37.219 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-05-10T11:45:37.219 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mon.b 2024-05-10T11:45:37.502 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:37 smithi178 systemd[1]: Stopping Ceph mon.b for 20a542cc-0ebd-11ef-bc98-c7b262605968... 2024-05-10T11:45:38.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:37 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mon-b[33355]: 2024-05-10T11:45:37.652+0000 7f63ff0ad700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T11:45:38.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:37 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mon-b[33355]: 2024-05-10T11:45:37.652+0000 7f63ff0ad700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-05-10T11:45:38.003 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:37 smithi178 podman[76362]: 2024-05-10 11:45:37.745426069 +0000 UTC m=+0.300742297 container died 2b35ae6b2dd96afb36329937240d31f0539cfd9ad405d794e48e78e77203acc8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mon-b, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, ceph=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e) 2024-05-10T11:45:38.379 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:38 smithi178 podman[76362]: 2024-05-10 11:45:38.136679811 +0000 UTC m=+0.691996046 container cleanup 2b35ae6b2dd96afb36329937240d31f0539cfd9ad405d794e48e78e77203acc8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mon-b, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-10T11:45:38.379 INFO:journalctl@ceph.mon.b.smithi178.stdout:May 10 11:45:38 smithi178 bash[76362]: ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mon-b 2024-05-10T11:45:38.509 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mon.b.service' 2024-05-10T11:45:38.559 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T11:45:38.560 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-05-10T11:45:38.560 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-05-10T11:45:38.560 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mon.c 2024-05-10T11:45:38.911 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:38 smithi188 systemd[1]: Stopping Ceph mon.c for 20a542cc-0ebd-11ef-bc98-c7b262605968... 2024-05-10T11:45:39.186 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:38 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mon-c[34056]: 2024-05-10T11:45:38.910+0000 7f641628a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T11:45:39.186 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:38 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mon-c[34056]: 2024-05-10T11:45:38.910+0000 7f641628a700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-05-10T11:45:39.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:39 smithi188 podman[76001]: 2024-05-10 11:45:39.186864582 +0000 UTC m=+0.399779355 container died 7e0cb31e6fa7547ee407f9878ca96109d76a0d223e7dc02e5dcf6354bbd440bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-10T11:45:39.968 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:39 smithi188 podman[76001]: 2024-05-10 11:45:39.658738384 +0000 UTC m=+0.871653154 container cleanup 7e0cb31e6fa7547ee407f9878ca96109d76a0d223e7dc02e5dcf6354bbd440bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mon-c, ceph=True, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-05-10T11:45:39.968 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 10 11:45:39 smithi188 bash[76001]: ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mon-c 2024-05-10T11:45:40.087 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mon.c.service' 2024-05-10T11:45:40.136 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T11:45:40.136 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-05-10T11:45:40.136 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-05-10T11:45:40.137 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mgr.x 2024-05-10T11:45:40.462 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:45:40 smithi100 systemd[1]: Stopping Ceph mgr.x for 20a542cc-0ebd-11ef-bc98-c7b262605968... 2024-05-10T11:45:40.788 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:45:40 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x[30526]: 2024-05-10T11:45:40.461+0000 7f620b4ed700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T11:45:40.788 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:45:40 smithi100 podman[140658]: 2024-05-10 11:45:40.493031603 +0000 UTC m=+0.164817012 container died 81381d9bb2d19d72aa45e62cba5e45e37accf85602ab6b5a526393b06b68e01e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.build-date=20240507) 2024-05-10T11:45:41.190 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:45:41 smithi100 podman[140658]: 2024-05-10 11:45:41.189300411 +0000 UTC m=+0.861085819 container cleanup 81381d9bb2d19d72aa45e62cba5e45e37accf85602ab6b5a526393b06b68e01e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d39fa08867836ac070c47cb66c782fa559fd480b, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-10T11:45:41.190 INFO:journalctl@ceph.mgr.x.smithi100.stdout:May 10 11:45:41 smithi100 bash[140658]: ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-x 2024-05-10T11:45:41.616 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mgr.x.service' 2024-05-10T11:45:41.653 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T11:45:41.653 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-05-10T11:45:41.653 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-05-10T11:45:41.653 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mgr.y 2024-05-10T11:45:42.002 INFO:journalctl@ceph.mgr.y.smithi178.stdout:May 10 11:45:41 smithi178 systemd[1]: Stopping Ceph mgr.y for 20a542cc-0ebd-11ef-bc98-c7b262605968... 2024-05-10T11:45:42.408 INFO:journalctl@ceph.mgr.y.smithi178.stdout:May 10 11:45:42 smithi178 podman[76484]: 2024-05-10 11:45:42.103431687 +0000 UTC m=+0.250519345 container died b2f3553d842c4f6786ee9bf44783692a0e68037aab2764a18a764c631e91b2e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-y, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True) 2024-05-10T11:45:42.753 INFO:journalctl@ceph.mgr.y.smithi178.stdout:May 10 11:45:42 smithi178 podman[76484]: 2024-05-10 11:45:42.524652745 +0000 UTC m=+0.671740403 container cleanup b2f3553d842c4f6786ee9bf44783692a0e68037aab2764a18a764c631e91b2e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-y, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, CEPH_POINT_RELEASE=) 2024-05-10T11:45:42.753 INFO:journalctl@ceph.mgr.y.smithi178.stdout:May 10 11:45:42 smithi178 bash[76484]: ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-y 2024-05-10T11:45:43.077 INFO:journalctl@ceph.mgr.y.smithi178.stdout:May 10 11:45:42 smithi178 podman[76498]: 2024-05-10 11:45:42.810129351 +0000 UTC m=+0.701649204 container remove b2f3553d842c4f6786ee9bf44783692a0e68037aab2764a18a764c631e91b2e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-y, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08) 2024-05-10T11:45:43.077 INFO:journalctl@ceph.mgr.y.smithi178.stdout:May 10 11:45:42 smithi178 systemd[1]: ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-05-10T11:45:43.094 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mgr.y.service' 2024-05-10T11:45:43.129 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T11:45:43.129 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-05-10T11:45:43.129 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-05-10T11:45:43.129 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mgr.z 2024-05-10T11:45:43.445 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 10 11:45:43 smithi188 systemd[1]: Stopping Ceph mgr.z for 20a542cc-0ebd-11ef-bc98-c7b262605968... 2024-05-10T11:45:43.778 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 10 11:45:43 smithi188 podman[76124]: 2024-05-10 11:45:43.468332102 +0000 UTC m=+0.142725434 container died 2e5740865245418813b84e800c68aaa6ce3802b46ef7fa7996ea58d8736060c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-z, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240507, GIT_CLEAN=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-10T11:45:44.091 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 10 11:45:43 smithi188 podman[76124]: 2024-05-10 11:45:43.881204776 +0000 UTC m=+0.555598108 container cleanup 2e5740865245418813b84e800c68aaa6ce3802b46ef7fa7996ea58d8736060c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-z, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.schema-version=1.0, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-05-10T11:45:44.091 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 10 11:45:43 smithi188 bash[76124]: ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mgr-z 2024-05-10T11:45:44.294 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mgr.z.service' 2024-05-10T11:45:44.329 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T11:45:44.329 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-05-10T11:45:44.329 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-05-10T11:45:44.329 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@osd.0 2024-05-10T11:45:44.652 INFO:journalctl@ceph.osd.0.smithi100.stdout:May 10 11:45:44 smithi100 systemd[1]: Stopping Ceph osd.0 for 20a542cc-0ebd-11ef-bc98-c7b262605968... 2024-05-10T11:45:45.038 INFO:journalctl@ceph.osd.0.smithi100.stdout:May 10 11:45:44 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-0[42301]: 2024-05-10T11:45:44.651+0000 7f0be51ad700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T11:45:45.038 INFO:journalctl@ceph.osd.0.smithi100.stdout:May 10 11:45:44 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-0[42301]: 2024-05-10T11:45:44.651+0000 7f0be51ad700 -1 osd.0 94 *** Got signal Terminated *** 2024-05-10T11:45:45.038 INFO:journalctl@ceph.osd.0.smithi100.stdout:May 10 11:45:44 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-0[42301]: 2024-05-10T11:45:44.651+0000 7f0be51ad700 -1 osd.0 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T11:45:50.445 INFO:journalctl@ceph.osd.0.smithi100.stdout:May 10 11:45:50 smithi100 podman[140778]: 2024-05-10 11:45:50.116446771 +0000 UTC m=+5.592565521 container died 368e76af2a3067e6e5425d705a9e754016057e6c1cc259f0f8dd12bb003f619a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-05-10T11:45:50.707 INFO:journalctl@ceph.osd.0.smithi100.stdout:May 10 11:45:50 smithi100 podman[140778]: 2024-05-10 11:45:50.686470999 +0000 UTC m=+6.162589759 container cleanup 368e76af2a3067e6e5425d705a9e754016057e6c1cc259f0f8dd12bb003f619a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, ceph=True, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-05-10T11:45:50.707 INFO:journalctl@ceph.osd.0.smithi100.stdout:May 10 11:45:50 smithi100 bash[140778]: ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-0 2024-05-10T11:45:51.288 INFO:journalctl@ceph.osd.0.smithi100.stdout:May 10 11:45:51 smithi100 podman[140796]: 2024-05-10 11:45:51.02184899 +0000 UTC m=+0.898359683 container remove 368e76af2a3067e6e5425d705a9e754016057e6c1cc259f0f8dd12bb003f619a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-0, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux ) 2024-05-10T11:45:51.633 INFO:journalctl@ceph.osd.0.smithi100.stdout:May 10 11:45:51 smithi100 podman[140888]: 2024-05-10 11:45:51.261460508 +0000 UTC m=+0.017587359 image pull ee5e4f21ef58047cb638cccd5cfbe8f6f43412b56c94b1d7fbd9cf3cb90a53a5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f 2024-05-10T11:45:51.634 INFO:journalctl@ceph.osd.0.smithi100.stdout:May 10 11:45:51 smithi100 podman[140888]: 2024-05-10 11:45:51.364353606 +0000 UTC m=+0.120480442 container create 1fa3515dd2409d409549e1e6c4084a49084a2728869d7ad8972e8afa35c8e5f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, io.buildah.version=1.33.5) 2024-05-10T11:45:51.981 INFO:journalctl@ceph.osd.0.smithi100.stdout:May 10 11:45:51 smithi100 podman[140888]: 2024-05-10 11:45:51.633688815 +0000 UTC m=+0.389815652 container init 1fa3515dd2409d409549e1e6c4084a49084a2728869d7ad8972e8afa35c8e5f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_CLEAN=True) 2024-05-10T11:45:51.981 INFO:journalctl@ceph.osd.0.smithi100.stdout:May 10 11:45:51 smithi100 podman[140888]: 2024-05-10 11:45:51.637926019 +0000 UTC m=+0.394052857 container start 1fa3515dd2409d409549e1e6c4084a49084a2728869d7ad8972e8afa35c8e5f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-0-deactivate, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240507, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-05-10T11:45:51.982 INFO:journalctl@ceph.osd.0.smithi100.stdout:May 10 11:45:51 smithi100 podman[140888]: 2024-05-10 11:45:51.742220065 +0000 UTC m=+0.498346911 container attach 1fa3515dd2409d409549e1e6c4084a49084a2728869d7ad8972e8afa35c8e5f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-0-deactivate, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_CLEAN=True, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-05-10T11:45:52.288 INFO:journalctl@ceph.osd.0.smithi100.stdout:May 10 11:45:51 smithi100 podman[140888]: 2024-05-10 11:45:51.982301849 +0000 UTC m=+0.738428683 container died 1fa3515dd2409d409549e1e6c4084a49084a2728869d7ad8972e8afa35c8e5f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507) 2024-05-10T11:45:52.781 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@osd.0.service' 2024-05-10T11:45:52.821 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T11:45:52.821 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-05-10T11:45:52.821 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-05-10T11:45:52.821 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@osd.1 2024-05-10T11:45:53.229 INFO:journalctl@ceph.osd.1.smithi178.stdout:May 10 11:45:52 smithi178 systemd[1]: Stopping Ceph osd.1 for 20a542cc-0ebd-11ef-bc98-c7b262605968... 2024-05-10T11:45:53.502 INFO:journalctl@ceph.osd.1.smithi178.stdout:May 10 11:45:53 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-1[37942]: 2024-05-10T11:45:53.228+0000 7f900f246700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T11:45:53.502 INFO:journalctl@ceph.osd.1.smithi178.stdout:May 10 11:45:53 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-1[37942]: 2024-05-10T11:45:53.228+0000 7f900f246700 -1 osd.1 94 *** Got signal Terminated *** 2024-05-10T11:45:53.502 INFO:journalctl@ceph.osd.1.smithi178.stdout:May 10 11:45:53 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-1[37942]: 2024-05-10T11:45:53.228+0000 7f900f246700 -1 osd.1 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T11:45:58.911 INFO:journalctl@ceph.osd.1.smithi178.stdout:May 10 11:45:58 smithi178 podman[76607]: 2024-05-10 11:45:58.619085088 +0000 UTC m=+5.589311681 container died 00a623888b3236e83a85cd7992b91491421708f702e866d683f1ced6632bb0f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-1, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08) 2024-05-10T11:45:58.911 INFO:journalctl@ceph.osd.1.smithi178.stdout:May 10 11:45:58 smithi178 podman[76607]: 2024-05-10 11:45:58.911217394 +0000 UTC m=+5.881443986 container cleanup 00a623888b3236e83a85cd7992b91491421708f702e866d683f1ced6632bb0f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-1, org.label-schema.license=GPLv2, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e) 2024-05-10T11:45:59.187 INFO:journalctl@ceph.osd.1.smithi178.stdout:May 10 11:45:58 smithi178 bash[76607]: ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-1 2024-05-10T11:45:59.495 INFO:journalctl@ceph.osd.1.smithi178.stdout:May 10 11:45:59 smithi178 podman[76624]: 2024-05-10 11:45:59.187689543 +0000 UTC m=+0.562792896 container remove 00a623888b3236e83a85cd7992b91491421708f702e866d683f1ced6632bb0f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, ceph=True, maintainer=Guillaume Abrioux ) 2024-05-10T11:45:59.752 INFO:journalctl@ceph.osd.1.smithi178.stdout:May 10 11:45:59 smithi178 podman[76718]: 2024-05-10 11:45:59.399122392 +0000 UTC m=+0.017806839 image pull ee5e4f21ef58047cb638cccd5cfbe8f6f43412b56c94b1d7fbd9cf3cb90a53a5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f 2024-05-10T11:45:59.753 INFO:journalctl@ceph.osd.1.smithi178.stdout:May 10 11:45:59 smithi178 podman[76718]: 2024-05-10 11:45:59.513572394 +0000 UTC m=+0.132256833 container create 27701ae3ed199837caeca497ac26c163a04bdbd121be3aa460c1e6975fab6401 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-1-deactivate, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-05-10T11:46:00.110 INFO:journalctl@ceph.osd.1.smithi178.stdout:May 10 11:45:59 smithi178 podman[76718]: 2024-05-10 11:45:59.758768016 +0000 UTC m=+0.377452456 container init 27701ae3ed199837caeca497ac26c163a04bdbd121be3aa460c1e6975fab6401 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-05-10T11:46:00.111 INFO:journalctl@ceph.osd.1.smithi178.stdout:May 10 11:45:59 smithi178 podman[76718]: 2024-05-10 11:45:59.762504922 +0000 UTC m=+0.381189365 container start 27701ae3ed199837caeca497ac26c163a04bdbd121be3aa460c1e6975fab6401 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240507, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, CEPH_POINT_RELEASE=) 2024-05-10T11:46:00.111 INFO:journalctl@ceph.osd.1.smithi178.stdout:May 10 11:45:59 smithi178 podman[76718]: 2024-05-10 11:45:59.858332924 +0000 UTC m=+0.477017363 container attach 27701ae3ed199837caeca497ac26c163a04bdbd121be3aa460c1e6975fab6401 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-10T11:46:00.378 INFO:journalctl@ceph.osd.1.smithi178.stdout:May 10 11:46:00 smithi178 podman[76718]: 2024-05-10 11:46:00.111193106 +0000 UTC m=+0.729877553 container died 27701ae3ed199837caeca497ac26c163a04bdbd121be3aa460c1e6975fab6401 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-05-10T11:46:00.737 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@osd.1.service' 2024-05-10T11:46:00.777 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T11:46:00.778 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-05-10T11:46:00.778 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-05-10T11:46:00.778 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@osd.2 2024-05-10T11:46:01.075 INFO:journalctl@ceph.osd.2.smithi188.stdout:May 10 11:46:00 smithi188 systemd[1]: Stopping Ceph osd.2 for 20a542cc-0ebd-11ef-bc98-c7b262605968... 2024-05-10T11:46:01.528 INFO:journalctl@ceph.osd.2.smithi188.stdout:May 10 11:46:01 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-2[39236]: 2024-05-10T11:46:01.075+0000 7f1cfb589700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T11:46:01.529 INFO:journalctl@ceph.osd.2.smithi188.stdout:May 10 11:46:01 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-2[39236]: 2024-05-10T11:46:01.075+0000 7f1cfb589700 -1 osd.2 94 *** Got signal Terminated *** 2024-05-10T11:46:01.529 INFO:journalctl@ceph.osd.2.smithi188.stdout:May 10 11:46:01 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-2[39236]: 2024-05-10T11:46:01.075+0000 7f1cfb589700 -1 osd.2 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T11:46:06.885 INFO:journalctl@ceph.osd.2.smithi188.stdout:May 10 11:46:06 smithi188 podman[76247]: 2024-05-10 11:46:06.569129073 +0000 UTC m=+5.599581371 container died 66f6e1fdd848836b1a372225aef8d409308236f08c7adf11a0ebc58046753862 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-2, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-10T11:46:07.252 INFO:journalctl@ceph.osd.2.smithi188.stdout:May 10 11:46:06 smithi188 podman[76247]: 2024-05-10 11:46:06.984200658 +0000 UTC m=+6.014652953 container cleanup 66f6e1fdd848836b1a372225aef8d409308236f08c7adf11a0ebc58046753862 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-10T11:46:07.252 INFO:journalctl@ceph.osd.2.smithi188.stdout:May 10 11:46:06 smithi188 bash[76247]: ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-2 2024-05-10T11:46:07.528 INFO:journalctl@ceph.osd.2.smithi188.stdout:May 10 11:46:07 smithi188 podman[76279]: 2024-05-10 11:46:07.252432698 +0000 UTC m=+0.665796578 container remove 66f6e1fdd848836b1a372225aef8d409308236f08c7adf11a0ebc58046753862 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-2, org.label-schema.build-date=20240507, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-05-10T11:46:07.939 INFO:journalctl@ceph.osd.2.smithi188.stdout:May 10 11:46:07 smithi188 podman[76372]: 2024-05-10 11:46:07.514558064 +0000 UTC m=+0.018178594 image pull ee5e4f21ef58047cb638cccd5cfbe8f6f43412b56c94b1d7fbd9cf3cb90a53a5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f 2024-05-10T11:46:07.939 INFO:journalctl@ceph.osd.2.smithi188.stdout:May 10 11:46:07 smithi188 podman[76372]: 2024-05-10 11:46:07.620137811 +0000 UTC m=+0.123758328 container create 1acee92df744a13248fe225ca91e42cc1f4fb796a7a9e1d56b859bd40e2b6f54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-05-10T11:46:07.939 INFO:journalctl@ceph.osd.2.smithi188.stdout:May 10 11:46:07 smithi188 podman[76372]: 2024-05-10 11:46:07.840056043 +0000 UTC m=+0.343676569 container init 1acee92df744a13248fe225ca91e42cc1f4fb796a7a9e1d56b859bd40e2b6f54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-05-10T11:46:07.939 INFO:journalctl@ceph.osd.2.smithi188.stdout:May 10 11:46:07 smithi188 podman[76372]: 2024-05-10 11:46:07.843684085 +0000 UTC m=+0.347304608 container start 1acee92df744a13248fe225ca91e42cc1f4fb796a7a9e1d56b859bd40e2b6f54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0) 2024-05-10T11:46:07.939 INFO:journalctl@ceph.osd.2.smithi188.stdout:May 10 11:46:07 smithi188 podman[76372]: 2024-05-10 11:46:07.939548385 +0000 UTC m=+0.443168902 container attach 1acee92df744a13248fe225ca91e42cc1f4fb796a7a9e1d56b859bd40e2b6f54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-10T11:46:08.195 INFO:journalctl@ceph.osd.2.smithi188.stdout:May 10 11:46:08 smithi188 podman[76372]: 2024-05-10 11:46:08.162621558 +0000 UTC m=+0.666242078 container died 1acee92df744a13248fe225ca91e42cc1f4fb796a7a9e1d56b859bd40e2b6f54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux ) 2024-05-10T11:46:08.834 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@osd.2.service' 2024-05-10T11:46:08.876 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T11:46:08.876 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-05-10T11:46:08.876 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-05-10T11:46:08.877 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@osd.3 2024-05-10T11:46:09.199 INFO:journalctl@ceph.osd.3.smithi100.stdout:May 10 11:46:08 smithi100 systemd[1]: Stopping Ceph osd.3 for 20a542cc-0ebd-11ef-bc98-c7b262605968... 2024-05-10T11:46:09.538 INFO:journalctl@ceph.osd.3.smithi100.stdout:May 10 11:46:09 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-3[49281]: 2024-05-10T11:46:09.198+0000 7f950e575700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T11:46:09.538 INFO:journalctl@ceph.osd.3.smithi100.stdout:May 10 11:46:09 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-3[49281]: 2024-05-10T11:46:09.198+0000 7f950e575700 -1 osd.3 94 *** Got signal Terminated *** 2024-05-10T11:46:09.538 INFO:journalctl@ceph.osd.3.smithi100.stdout:May 10 11:46:09 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-3[49281]: 2024-05-10T11:46:09.198+0000 7f950e575700 -1 osd.3 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T11:46:11.002 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:10 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:10.635+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:44.388044+0000 front 2024-05-10T11:45:44.388108+0000 (oldest deadline 2024-05-10T11:46:10.288056+0000) 2024-05-10T11:46:11.038 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:10 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:10.653+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.171852+0000 front 2024-05-10T11:45:45.171825+0000 (oldest deadline 2024-05-10T11:46:10.372558+0000) 2024-05-10T11:46:11.538 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:11 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6[56225]: 2024-05-10T11:46:11.076+0000 7ff3d7bd0700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.438944+0000 front 2024-05-10T11:45:45.438900+0000 (oldest deadline 2024-05-10T11:46:10.739184+0000) 2024-05-10T11:46:11.752 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:11 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:11.608+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:44.388044+0000 front 2024-05-10T11:45:44.388108+0000 (oldest deadline 2024-05-10T11:46:10.288056+0000) 2024-05-10T11:46:12.038 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:11 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:11.692+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.171852+0000 front 2024-05-10T11:45:45.171825+0000 (oldest deadline 2024-05-10T11:46:10.372558+0000) 2024-05-10T11:46:12.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:11 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:11.843+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.703618+0000 front 2024-05-10T11:45:45.703296+0000 (oldest deadline 2024-05-10T11:46:11.003467+0000) 2024-05-10T11:46:12.325 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:12 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6[56225]: 2024-05-10T11:46:12.102+0000 7ff3d7bd0700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.438944+0000 front 2024-05-10T11:45:45.438900+0000 (oldest deadline 2024-05-10T11:46:10.739184+0000) 2024-05-10T11:46:13.002 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:12 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:12.651+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:44.388044+0000 front 2024-05-10T11:45:44.388108+0000 (oldest deadline 2024-05-10T11:46:10.288056+0000) 2024-05-10T11:46:13.038 INFO:journalctl@ceph.osd.3.smithi100.stdout:May 10 11:46:12 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-3[49281]: 2024-05-10T11:46:12.857+0000 7f9506b61700 -1 osd.3 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:46.171886+0000 front 2024-05-10T11:45:46.171953+0000 (oldest deadline 2024-05-10T11:46:12.072058+0000) 2024-05-10T11:46:13.038 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:12 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:12.706+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.171852+0000 front 2024-05-10T11:45:45.171825+0000 (oldest deadline 2024-05-10T11:46:10.372558+0000) 2024-05-10T11:46:13.278 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:12 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:12.859+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.703618+0000 front 2024-05-10T11:45:45.703296+0000 (oldest deadline 2024-05-10T11:46:11.003467+0000) 2024-05-10T11:46:13.538 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:13 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6[56225]: 2024-05-10T11:46:13.124+0000 7ff3d7bd0700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.438944+0000 front 2024-05-10T11:45:45.438900+0000 (oldest deadline 2024-05-10T11:46:10.739184+0000) 2024-05-10T11:46:14.002 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:13 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:13.684+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:44.388044+0000 front 2024-05-10T11:45:44.388108+0000 (oldest deadline 2024-05-10T11:46:10.288056+0000) 2024-05-10T11:46:14.038 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:13 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:13.686+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.171852+0000 front 2024-05-10T11:45:45.171825+0000 (oldest deadline 2024-05-10T11:46:10.372558+0000) 2024-05-10T11:46:14.038 INFO:journalctl@ceph.osd.3.smithi100.stdout:May 10 11:46:13 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-3[49281]: 2024-05-10T11:46:13.873+0000 7f9506b61700 -1 osd.3 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:46.171886+0000 front 2024-05-10T11:45:46.171953+0000 (oldest deadline 2024-05-10T11:46:12.072058+0000) 2024-05-10T11:46:14.278 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:13 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:13.826+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.703618+0000 front 2024-05-10T11:45:45.703296+0000 (oldest deadline 2024-05-10T11:46:11.003467+0000) 2024-05-10T11:46:14.538 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:14 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6[56225]: 2024-05-10T11:46:14.147+0000 7ff3d7bd0700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.438944+0000 front 2024-05-10T11:45:45.438900+0000 (oldest deadline 2024-05-10T11:46:10.739184+0000) 2024-05-10T11:46:14.778 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 10 11:46:14 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-5[45716]: 2024-05-10T11:46:14.429+0000 7fca39e83700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:48.830864+0000 front 2024-05-10T11:45:48.830948+0000 (oldest deadline 2024-05-10T11:46:13.530938+0000) 2024-05-10T11:46:15.002 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:14 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:14.684+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:44.388044+0000 front 2024-05-10T11:45:44.388108+0000 (oldest deadline 2024-05-10T11:46:10.288056+0000) 2024-05-10T11:46:15.008 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:14 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:14.699+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.171852+0000 front 2024-05-10T11:45:45.171825+0000 (oldest deadline 2024-05-10T11:46:10.372558+0000) 2024-05-10T11:46:15.008 INFO:journalctl@ceph.osd.3.smithi100.stdout:May 10 11:46:14 smithi100 podman[140994]: 2024-05-10 11:46:14.683461285 +0000 UTC m=+5.604809418 container died 2e5cf00d9067e333a44bbe22841ee112913b2268611a8b35f45049fe1e7ff752 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, GIT_CLEAN=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True) 2024-05-10T11:46:15.278 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:14 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:14.855+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.703618+0000 front 2024-05-10T11:45:45.703296+0000 (oldest deadline 2024-05-10T11:46:11.003467+0000) 2024-05-10T11:46:15.288 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:15 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6[56225]: 2024-05-10T11:46:15.161+0000 7ff3d7bd0700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.438944+0000 front 2024-05-10T11:45:45.438900+0000 (oldest deadline 2024-05-10T11:46:10.739184+0000) 2024-05-10T11:46:15.288 INFO:journalctl@ceph.osd.3.smithi100.stdout:May 10 11:46:15 smithi100 podman[140994]: 2024-05-10 11:46:15.2249544 +0000 UTC m=+6.146302576 container cleanup 2e5cf00d9067e333a44bbe22841ee112913b2268611a8b35f45049fe1e7ff752 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-3, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_CLEAN=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.schema-version=1.0, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-05-10T11:46:15.288 INFO:journalctl@ceph.osd.3.smithi100.stdout:May 10 11:46:15 smithi100 bash[140994]: ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-3 2024-05-10T11:46:15.778 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 10 11:46:15 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-5[45716]: 2024-05-10T11:46:15.429+0000 7fca39e83700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:48.830864+0000 front 2024-05-10T11:45:48.830948+0000 (oldest deadline 2024-05-10T11:46:13.530938+0000) 2024-05-10T11:46:15.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:15 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:15.693+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.171852+0000 front 2024-05-10T11:45:45.171825+0000 (oldest deadline 2024-05-10T11:46:10.372558+0000) 2024-05-10T11:46:15.788 INFO:journalctl@ceph.osd.3.smithi100.stdout:May 10 11:46:15 smithi100 podman[141023]: 2024-05-10 11:46:15.460039474 +0000 UTC m=+0.769887527 container remove 2e5cf00d9067e333a44bbe22841ee112913b2268611a8b35f45049fe1e7ff752 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-3, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e) 2024-05-10T11:46:16.002 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:15 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:15.724+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:44.388044+0000 front 2024-05-10T11:45:44.388108+0000 (oldest deadline 2024-05-10T11:46:10.288056+0000) 2024-05-10T11:46:16.179 INFO:journalctl@ceph.osd.3.smithi100.stdout:May 10 11:46:15 smithi100 podman[141113]: 2024-05-10 11:46:15.798126388 +0000 UTC m=+0.018032383 image pull ee5e4f21ef58047cb638cccd5cfbe8f6f43412b56c94b1d7fbd9cf3cb90a53a5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f 2024-05-10T11:46:16.180 INFO:journalctl@ceph.osd.3.smithi100.stdout:May 10 11:46:15 smithi100 podman[141113]: 2024-05-10 11:46:15.927774639 +0000 UTC m=+0.147680664 container create aac7eee4003e9f787a963f7f277cfc3d8a9e61c720c50d03ae3d807afcb88ad1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-3-deactivate, ceph=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_BRANCH=HEAD, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-10T11:46:16.278 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:15 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:15.900+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.703618+0000 front 2024-05-10T11:45:45.703296+0000 (oldest deadline 2024-05-10T11:46:11.003467+0000) 2024-05-10T11:46:16.479 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:16 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6[56225]: 2024-05-10T11:46:16.179+0000 7ff3d7bd0700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.438944+0000 front 2024-05-10T11:45:45.438900+0000 (oldest deadline 2024-05-10T11:46:10.739184+0000) 2024-05-10T11:46:16.479 INFO:journalctl@ceph.osd.3.smithi100.stdout:May 10 11:46:16 smithi100 podman[141113]: 2024-05-10 11:46:16.180451639 +0000 UTC m=+0.400357618 container init aac7eee4003e9f787a963f7f277cfc3d8a9e61c720c50d03ae3d807afcb88ad1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_CLEAN=True) 2024-05-10T11:46:16.479 INFO:journalctl@ceph.osd.3.smithi100.stdout:May 10 11:46:16 smithi100 podman[141113]: 2024-05-10 11:46:16.184399744 +0000 UTC m=+0.404305728 container start aac7eee4003e9f787a963f7f277cfc3d8a9e61c720c50d03ae3d807afcb88ad1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e) 2024-05-10T11:46:16.479 INFO:journalctl@ceph.osd.3.smithi100.stdout:May 10 11:46:16 smithi100 podman[141113]: 2024-05-10 11:46:16.288729431 +0000 UTC m=+0.508635429 container attach aac7eee4003e9f787a963f7f277cfc3d8a9e61c720c50d03ae3d807afcb88ad1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240507, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-10T11:46:16.479 INFO:journalctl@ceph.osd.3.smithi100.stdout:May 10 11:46:16 smithi100 podman[141113]: 2024-05-10 11:46:16.479833455 +0000 UTC m=+0.699739435 container died aac7eee4003e9f787a963f7f277cfc3d8a9e61c720c50d03ae3d807afcb88ad1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-05-10T11:46:16.778 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 10 11:46:16 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-5[45716]: 2024-05-10T11:46:16.449+0000 7fca39e83700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:48.830864+0000 front 2024-05-10T11:45:48.830948+0000 (oldest deadline 2024-05-10T11:46:13.530938+0000) 2024-05-10T11:46:16.787 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:16 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:16.677+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.171852+0000 front 2024-05-10T11:45:45.171825+0000 (oldest deadline 2024-05-10T11:46:10.372558+0000) 2024-05-10T11:46:17.134 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@osd.3.service' 2024-05-10T11:46:17.174 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T11:46:17.175 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-05-10T11:46:17.175 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-05-10T11:46:17.175 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@osd.4 2024-05-10T11:46:17.202 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:16 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:16.762+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:44.388044+0000 front 2024-05-10T11:45:44.388108+0000 (oldest deadline 2024-05-10T11:46:10.288056+0000) 2024-05-10T11:46:17.278 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:16 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:16.922+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.703618+0000 front 2024-05-10T11:45:45.703296+0000 (oldest deadline 2024-05-10T11:46:11.003467+0000) 2024-05-10T11:46:17.288 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:17 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6[56225]: 2024-05-10T11:46:17.175+0000 7ff3d7bd0700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.438944+0000 front 2024-05-10T11:45:45.438900+0000 (oldest deadline 2024-05-10T11:46:10.739184+0000) 2024-05-10T11:46:17.502 INFO:journalctl@ceph.osd.4.smithi178.stdout:May 10 11:46:17 smithi178 systemd[1]: Stopping Ceph osd.4 for 20a542cc-0ebd-11ef-bc98-c7b262605968... 2024-05-10T11:46:17.779 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 10 11:46:17 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-5[45716]: 2024-05-10T11:46:17.425+0000 7fca39e83700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:48.830864+0000 front 2024-05-10T11:45:48.830948+0000 (oldest deadline 2024-05-10T11:46:13.530938+0000) 2024-05-10T11:46:17.811 INFO:journalctl@ceph.osd.4.smithi178.stdout:May 10 11:46:17 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-4[44733]: 2024-05-10T11:46:17.505+0000 7f56376b0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T11:46:17.811 INFO:journalctl@ceph.osd.4.smithi178.stdout:May 10 11:46:17 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-4[44733]: 2024-05-10T11:46:17.505+0000 7f56376b0700 -1 osd.4 94 *** Got signal Terminated *** 2024-05-10T11:46:17.811 INFO:journalctl@ceph.osd.4.smithi178.stdout:May 10 11:46:17 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-4[44733]: 2024-05-10T11:46:17.505+0000 7f56376b0700 -1 osd.4 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T11:46:18.038 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:17 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:17.653+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.171852+0000 front 2024-05-10T11:45:45.171825+0000 (oldest deadline 2024-05-10T11:46:10.372558+0000) 2024-05-10T11:46:18.229 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:17 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:17.950+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.703618+0000 front 2024-05-10T11:45:45.703296+0000 (oldest deadline 2024-05-10T11:46:11.003467+0000) 2024-05-10T11:46:18.253 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:17 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:17.810+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:44.388044+0000 front 2024-05-10T11:45:44.388108+0000 (oldest deadline 2024-05-10T11:46:10.288056+0000) 2024-05-10T11:46:18.529 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 10 11:46:18 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-5[45716]: 2024-05-10T11:46:18.454+0000 7fca39e83700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:48.830864+0000 front 2024-05-10T11:45:48.830948+0000 (oldest deadline 2024-05-10T11:46:13.530938+0000) 2024-05-10T11:46:18.538 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:18 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6[56225]: 2024-05-10T11:46:18.133+0000 7ff3d7bd0700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.438944+0000 front 2024-05-10T11:45:45.438900+0000 (oldest deadline 2024-05-10T11:46:10.739184+0000) 2024-05-10T11:46:19.038 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:18 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:18.664+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.171852+0000 front 2024-05-10T11:45:45.171825+0000 (oldest deadline 2024-05-10T11:46:10.372558+0000) 2024-05-10T11:46:19.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:18 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:18.848+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:44.388044+0000 front 2024-05-10T11:45:44.388108+0000 (oldest deadline 2024-05-10T11:46:10.288056+0000) 2024-05-10T11:46:19.278 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:18 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:18.978+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.703618+0000 front 2024-05-10T11:45:45.703296+0000 (oldest deadline 2024-05-10T11:46:11.003467+0000) 2024-05-10T11:46:19.538 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:19 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6[56225]: 2024-05-10T11:46:19.161+0000 7ff3d7bd0700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.438944+0000 front 2024-05-10T11:45:45.438900+0000 (oldest deadline 2024-05-10T11:46:10.739184+0000) 2024-05-10T11:46:19.538 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:19 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6[56225]: 2024-05-10T11:46:19.161+0000 7ff3d7bd0700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.141219+0000 front 2024-05-10T11:45:57.141270+0000 (oldest deadline 2024-05-10T11:46:18.841368+0000) 2024-05-10T11:46:19.778 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 10 11:46:19 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-5[45716]: 2024-05-10T11:46:19.491+0000 7fca39e83700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:48.830864+0000 front 2024-05-10T11:45:48.830948+0000 (oldest deadline 2024-05-10T11:46:13.530938+0000) 2024-05-10T11:46:20.038 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:19 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:19.634+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.171852+0000 front 2024-05-10T11:45:45.171825+0000 (oldest deadline 2024-05-10T11:46:10.372558+0000) 2024-05-10T11:46:20.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:19 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:19.846+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:44.388044+0000 front 2024-05-10T11:45:44.388108+0000 (oldest deadline 2024-05-10T11:46:10.288056+0000) 2024-05-10T11:46:20.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:19 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:19.846+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.190342+0000 front 2024-05-10T11:45:57.190423+0000 (oldest deadline 2024-05-10T11:46:19.490457+0000) 2024-05-10T11:46:20.278 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:20 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:20.010+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.703618+0000 front 2024-05-10T11:45:45.703296+0000 (oldest deadline 2024-05-10T11:46:11.003467+0000) 2024-05-10T11:46:20.538 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:20 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6[56225]: 2024-05-10T11:46:20.153+0000 7ff3d7bd0700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.438944+0000 front 2024-05-10T11:45:45.438900+0000 (oldest deadline 2024-05-10T11:46:10.739184+0000) 2024-05-10T11:46:20.538 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:20 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6[56225]: 2024-05-10T11:46:20.153+0000 7ff3d7bd0700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.141219+0000 front 2024-05-10T11:45:57.141270+0000 (oldest deadline 2024-05-10T11:46:18.841368+0000) 2024-05-10T11:46:20.778 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 10 11:46:20 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-5[45716]: 2024-05-10T11:46:20.524+0000 7fca39e83700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:48.830864+0000 front 2024-05-10T11:45:48.830948+0000 (oldest deadline 2024-05-10T11:46:13.530938+0000) 2024-05-10T11:46:20.779 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 10 11:46:20 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-5[45716]: 2024-05-10T11:46:20.524+0000 7fca39e83700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:58.033547+0000 front 2024-05-10T11:45:58.033589+0000 (oldest deadline 2024-05-10T11:46:20.333670+0000) 2024-05-10T11:46:21.038 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:20 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:20.634+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.171852+0000 front 2024-05-10T11:45:45.171825+0000 (oldest deadline 2024-05-10T11:46:10.372558+0000) 2024-05-10T11:46:21.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:20 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:20.864+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:44.388044+0000 front 2024-05-10T11:45:44.388108+0000 (oldest deadline 2024-05-10T11:46:10.288056+0000) 2024-05-10T11:46:21.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:20 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:20.864+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.190342+0000 front 2024-05-10T11:45:57.190423+0000 (oldest deadline 2024-05-10T11:46:19.490457+0000) 2024-05-10T11:46:21.278 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:20 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:20.968+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.703618+0000 front 2024-05-10T11:45:45.703296+0000 (oldest deadline 2024-05-10T11:46:11.003467+0000) 2024-05-10T11:46:21.538 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:21 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6[56225]: 2024-05-10T11:46:21.130+0000 7ff3d7bd0700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.438944+0000 front 2024-05-10T11:45:45.438900+0000 (oldest deadline 2024-05-10T11:46:10.739184+0000) 2024-05-10T11:46:21.538 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:21 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6[56225]: 2024-05-10T11:46:21.130+0000 7ff3d7bd0700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.141219+0000 front 2024-05-10T11:45:57.141270+0000 (oldest deadline 2024-05-10T11:46:18.841368+0000) 2024-05-10T11:46:21.752 INFO:journalctl@ceph.osd.4.smithi178.stdout:May 10 11:46:21 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-4[44733]: 2024-05-10T11:46:21.337+0000 7f562fc9c700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:56.512021+0000 front 2024-05-10T11:45:56.512049+0000 (oldest deadline 2024-05-10T11:46:20.612163+0000) 2024-05-10T11:46:21.753 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:21 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7[51714]: 2024-05-10T11:46:21.350+0000 7f2b921f1700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:55.454191+0000 front 2024-05-10T11:45:55.454223+0000 (oldest deadline 2024-05-10T11:46:20.754336+0000) 2024-05-10T11:46:21.944 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 10 11:46:21 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-5[45716]: 2024-05-10T11:46:21.540+0000 7fca39e83700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:48.830864+0000 front 2024-05-10T11:45:48.830948+0000 (oldest deadline 2024-05-10T11:46:13.530938+0000) 2024-05-10T11:46:21.944 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 10 11:46:21 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-5[45716]: 2024-05-10T11:46:21.540+0000 7fca39e83700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:58.033547+0000 front 2024-05-10T11:45:58.033589+0000 (oldest deadline 2024-05-10T11:46:20.333670+0000) 2024-05-10T11:46:22.038 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:21 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:21.592+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.171852+0000 front 2024-05-10T11:45:45.171825+0000 (oldest deadline 2024-05-10T11:46:10.372558+0000) 2024-05-10T11:46:22.038 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:21 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:21.592+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.274899+0000 front 2024-05-10T11:45:57.274943+0000 (oldest deadline 2024-05-10T11:46:21.375013+0000) 2024-05-10T11:46:22.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:21 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:21.909+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:44.388044+0000 front 2024-05-10T11:45:44.388108+0000 (oldest deadline 2024-05-10T11:46:10.288056+0000) 2024-05-10T11:46:22.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:21 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:21.909+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.190342+0000 front 2024-05-10T11:45:57.190423+0000 (oldest deadline 2024-05-10T11:46:19.490457+0000) 2024-05-10T11:46:22.278 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:21 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:21.943+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.703618+0000 front 2024-05-10T11:45:45.703296+0000 (oldest deadline 2024-05-10T11:46:11.003467+0000) 2024-05-10T11:46:22.538 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:22 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6[56225]: 2024-05-10T11:46:22.101+0000 7ff3d7bd0700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.438944+0000 front 2024-05-10T11:45:45.438900+0000 (oldest deadline 2024-05-10T11:46:10.739184+0000) 2024-05-10T11:46:22.538 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:22 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6[56225]: 2024-05-10T11:46:22.101+0000 7ff3d7bd0700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.141219+0000 front 2024-05-10T11:45:57.141270+0000 (oldest deadline 2024-05-10T11:46:18.841368+0000) 2024-05-10T11:46:22.752 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:22 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7[51714]: 2024-05-10T11:46:22.354+0000 7f2b921f1700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:55.454191+0000 front 2024-05-10T11:45:55.454223+0000 (oldest deadline 2024-05-10T11:46:20.754336+0000) 2024-05-10T11:46:22.753 INFO:journalctl@ceph.osd.4.smithi178.stdout:May 10 11:46:22 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-4[44733]: 2024-05-10T11:46:22.355+0000 7f562fc9c700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:56.512021+0000 front 2024-05-10T11:45:56.512049+0000 (oldest deadline 2024-05-10T11:46:20.612163+0000) 2024-05-10T11:46:22.778 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 10 11:46:22 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-5[45716]: 2024-05-10T11:46:22.506+0000 7fca39e83700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:48.830864+0000 front 2024-05-10T11:45:48.830948+0000 (oldest deadline 2024-05-10T11:46:13.530938+0000) 2024-05-10T11:46:22.778 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 10 11:46:22 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-5[45716]: 2024-05-10T11:46:22.507+0000 7fca39e83700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:58.033547+0000 front 2024-05-10T11:45:58.033589+0000 (oldest deadline 2024-05-10T11:46:20.333670+0000) 2024-05-10T11:46:23.038 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:22 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:22.629+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.171852+0000 front 2024-05-10T11:45:45.171825+0000 (oldest deadline 2024-05-10T11:46:10.372558+0000) 2024-05-10T11:46:23.038 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:22 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:22.629+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.274899+0000 front 2024-05-10T11:45:57.274943+0000 (oldest deadline 2024-05-10T11:46:21.375013+0000) 2024-05-10T11:46:23.253 INFO:journalctl@ceph.osd.4.smithi178.stdout:May 10 11:46:22 smithi178 podman[76834]: 2024-05-10 11:46:22.990840576 +0000 UTC m=+5.610881292 container died 2273740b90eb23e08720777181884dac17ecee29c37c8f39cec34f1829786705 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-4, ceph=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-05-10T11:46:23.253 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:22 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:22.935+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:44.388044+0000 front 2024-05-10T11:45:44.388108+0000 (oldest deadline 2024-05-10T11:46:10.288056+0000) 2024-05-10T11:46:23.253 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:22 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:22.935+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.190342+0000 front 2024-05-10T11:45:57.190423+0000 (oldest deadline 2024-05-10T11:46:19.490457+0000) 2024-05-10T11:46:23.278 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:22 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:22.962+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.703618+0000 front 2024-05-10T11:45:45.703296+0000 (oldest deadline 2024-05-10T11:46:11.003467+0000) 2024-05-10T11:46:23.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:22 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:22.962+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:58.005499+0000 front 2024-05-10T11:45:58.005443+0000 (oldest deadline 2024-05-10T11:46:22.705582+0000) 2024-05-10T11:46:23.538 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:23 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6[56225]: 2024-05-10T11:46:23.075+0000 7ff3d7bd0700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.438944+0000 front 2024-05-10T11:45:45.438900+0000 (oldest deadline 2024-05-10T11:46:10.739184+0000) 2024-05-10T11:46:23.538 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:23 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6[56225]: 2024-05-10T11:46:23.075+0000 7ff3d7bd0700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.141219+0000 front 2024-05-10T11:45:57.141270+0000 (oldest deadline 2024-05-10T11:46:18.841368+0000) 2024-05-10T11:46:23.648 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:23 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7[51714]: 2024-05-10T11:46:23.372+0000 7f2b921f1700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:55.454191+0000 front 2024-05-10T11:45:55.454223+0000 (oldest deadline 2024-05-10T11:46:20.754336+0000) 2024-05-10T11:46:23.648 INFO:journalctl@ceph.osd.4.smithi178.stdout:May 10 11:46:23 smithi178 podman[76834]: 2024-05-10 11:46:23.363796286 +0000 UTC m=+5.983837001 container cleanup 2273740b90eb23e08720777181884dac17ecee29c37c8f39cec34f1829786705 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-4, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-05-10T11:46:23.648 INFO:journalctl@ceph.osd.4.smithi178.stdout:May 10 11:46:23 smithi178 bash[76834]: ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-4 2024-05-10T11:46:23.779 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 10 11:46:23 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-5[45716]: 2024-05-10T11:46:23.481+0000 7fca39e83700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:48.830864+0000 front 2024-05-10T11:45:48.830948+0000 (oldest deadline 2024-05-10T11:46:13.530938+0000) 2024-05-10T11:46:23.779 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 10 11:46:23 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-5[45716]: 2024-05-10T11:46:23.481+0000 7fca39e83700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:58.033547+0000 front 2024-05-10T11:45:58.033589+0000 (oldest deadline 2024-05-10T11:46:20.333670+0000) 2024-05-10T11:46:23.779 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:23 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:23.532+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:53.194818+0000 front 2024-05-10T11:45:53.194735+0000 (oldest deadline 2024-05-10T11:46:23.195612+0000) 2024-05-10T11:46:23.910 INFO:journalctl@ceph.osd.4.smithi178.stdout:May 10 11:46:23 smithi178 podman[76851]: 2024-05-10 11:46:23.648688351 +0000 UTC m=+0.655520313 container remove 2273740b90eb23e08720777181884dac17ecee29c37c8f39cec34f1829786705 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-4, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.vendor=CentOS) 2024-05-10T11:46:24.038 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:23 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:23.602+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.171852+0000 front 2024-05-10T11:45:45.171825+0000 (oldest deadline 2024-05-10T11:46:10.372558+0000) 2024-05-10T11:46:24.038 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:23 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:23.602+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.274899+0000 front 2024-05-10T11:45:57.274943+0000 (oldest deadline 2024-05-10T11:46:21.375013+0000) 2024-05-10T11:46:24.165 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:23 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:23.909+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:44.388044+0000 front 2024-05-10T11:45:44.388108+0000 (oldest deadline 2024-05-10T11:46:10.288056+0000) 2024-05-10T11:46:24.166 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:23 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:23.909+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.190342+0000 front 2024-05-10T11:45:57.190423+0000 (oldest deadline 2024-05-10T11:46:19.490457+0000) 2024-05-10T11:46:24.166 INFO:journalctl@ceph.osd.4.smithi178.stdout:May 10 11:46:23 smithi178 podman[76942]: 2024-05-10 11:46:23.886979453 +0000 UTC m=+0.018094811 image pull ee5e4f21ef58047cb638cccd5cfbe8f6f43412b56c94b1d7fbd9cf3cb90a53a5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f 2024-05-10T11:46:24.166 INFO:journalctl@ceph.osd.4.smithi178.stdout:May 10 11:46:24 smithi178 podman[76942]: 2024-05-10 11:46:24.016222822 +0000 UTC m=+0.147338181 container create 4f4f3eaafe7c6dd59a189bd4caba6cfe60a9869fe11cb878f54b641ce6d73478 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-05-10T11:46:24.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:23 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:23.970+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.703618+0000 front 2024-05-10T11:45:45.703296+0000 (oldest deadline 2024-05-10T11:46:11.003467+0000) 2024-05-10T11:46:24.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:23 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:23.970+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:58.005499+0000 front 2024-05-10T11:45:58.005443+0000 (oldest deadline 2024-05-10T11:46:22.705582+0000) 2024-05-10T11:46:24.538 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:24 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6[56225]: 2024-05-10T11:46:24.078+0000 7ff3d7bd0700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.438944+0000 front 2024-05-10T11:45:45.438900+0000 (oldest deadline 2024-05-10T11:46:10.739184+0000) 2024-05-10T11:46:24.538 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:24 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6[56225]: 2024-05-10T11:46:24.078+0000 7ff3d7bd0700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.141219+0000 front 2024-05-10T11:45:57.141270+0000 (oldest deadline 2024-05-10T11:46:18.841368+0000) 2024-05-10T11:46:24.574 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:24 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7[51714]: 2024-05-10T11:46:24.410+0000 7f2b921f1700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:55.454191+0000 front 2024-05-10T11:45:55.454223+0000 (oldest deadline 2024-05-10T11:46:20.754336+0000) 2024-05-10T11:46:24.574 INFO:journalctl@ceph.osd.4.smithi178.stdout:May 10 11:46:24 smithi178 podman[76942]: 2024-05-10 11:46:24.277289805 +0000 UTC m=+0.408405164 container init 4f4f3eaafe7c6dd59a189bd4caba6cfe60a9869fe11cb878f54b641ce6d73478 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-05-10T11:46:24.575 INFO:journalctl@ceph.osd.4.smithi178.stdout:May 10 11:46:24 smithi178 podman[76942]: 2024-05-10 11:46:24.280678296 +0000 UTC m=+0.411793662 container start 4f4f3eaafe7c6dd59a189bd4caba6cfe60a9869fe11cb878f54b641ce6d73478 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507) 2024-05-10T11:46:24.575 INFO:journalctl@ceph.osd.4.smithi178.stdout:May 10 11:46:24 smithi178 podman[76942]: 2024-05-10 11:46:24.377176952 +0000 UTC m=+0.508292310 container attach 4f4f3eaafe7c6dd59a189bd4caba6cfe60a9869fe11cb878f54b641ce6d73478 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True) 2024-05-10T11:46:24.575 INFO:journalctl@ceph.osd.4.smithi178.stdout:May 10 11:46:24 smithi178 podman[76942]: 2024-05-10 11:46:24.575310157 +0000 UTC m=+0.706425516 container died 4f4f3eaafe7c6dd59a189bd4caba6cfe60a9869fe11cb878f54b641ce6d73478 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-05-10T11:46:24.778 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:24 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:24.513+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:53.194818+0000 front 2024-05-10T11:45:53.194735+0000 (oldest deadline 2024-05-10T11:46:23.195612+0000) 2024-05-10T11:46:24.779 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 10 11:46:24 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-5[45716]: 2024-05-10T11:46:24.518+0000 7fca39e83700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:48.830864+0000 front 2024-05-10T11:45:48.830948+0000 (oldest deadline 2024-05-10T11:46:13.530938+0000) 2024-05-10T11:46:24.779 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 10 11:46:24 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-5[45716]: 2024-05-10T11:46:24.518+0000 7fca39e83700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:58.033547+0000 front 2024-05-10T11:45:58.033589+0000 (oldest deadline 2024-05-10T11:46:20.333670+0000) 2024-05-10T11:46:25.038 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:24 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:24.563+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.171852+0000 front 2024-05-10T11:45:45.171825+0000 (oldest deadline 2024-05-10T11:46:10.372558+0000) 2024-05-10T11:46:25.038 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:24 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:24.563+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.274899+0000 front 2024-05-10T11:45:57.274943+0000 (oldest deadline 2024-05-10T11:46:21.375013+0000) 2024-05-10T11:46:25.089 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@osd.4.service' 2024-05-10T11:46:25.152 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:24 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:24.872+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:44.388044+0000 front 2024-05-10T11:45:44.388108+0000 (oldest deadline 2024-05-10T11:46:10.288056+0000) 2024-05-10T11:46:25.153 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:24 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:24.872+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.190342+0000 front 2024-05-10T11:45:57.190423+0000 (oldest deadline 2024-05-10T11:46:19.490457+0000) 2024-05-10T11:46:25.153 INFO:journalctl@ceph.osd.4.smithi178.stdout:May 10 11:46:25 smithi178 podman[76967]: 2024-05-10 11:46:25.064137084 +0000 UTC m=+0.483450968 container remove 4f4f3eaafe7c6dd59a189bd4caba6cfe60a9869fe11cb878f54b641ce6d73478 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240507, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.license=GPLv2) 2024-05-10T11:46:25.153 INFO:journalctl@ceph.osd.4.smithi178.stdout:May 10 11:46:25 smithi178 systemd[1]: ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@osd.4.service: Deactivated successfully. 2024-05-10T11:46:25.153 INFO:journalctl@ceph.osd.4.smithi178.stdout:May 10 11:46:25 smithi178 systemd[1]: Stopped Ceph osd.4 for 20a542cc-0ebd-11ef-bc98-c7b262605968. 2024-05-10T11:46:25.153 INFO:journalctl@ceph.osd.4.smithi178.stdout:May 10 11:46:25 smithi178 systemd[1]: ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@osd.4.service: Consumed 4min 16.065s CPU time. 2024-05-10T11:46:25.167 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T11:46:25.167 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-05-10T11:46:25.168 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-05-10T11:46:25.168 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@osd.5 2024-05-10T11:46:25.279 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 10 11:46:25 smithi188 systemd[1]: Stopping Ceph osd.5 for 20a542cc-0ebd-11ef-bc98-c7b262605968... 2024-05-10T11:46:25.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:25 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:25.016+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.703618+0000 front 2024-05-10T11:45:45.703296+0000 (oldest deadline 2024-05-10T11:46:11.003467+0000) 2024-05-10T11:46:25.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:25 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:25.016+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:58.005499+0000 front 2024-05-10T11:45:58.005443+0000 (oldest deadline 2024-05-10T11:46:22.705582+0000) 2024-05-10T11:46:25.403 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:25 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7[51714]: 2024-05-10T11:46:25.402+0000 7f2b921f1700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:55.454191+0000 front 2024-05-10T11:45:55.454223+0000 (oldest deadline 2024-05-10T11:46:20.754336+0000) 2024-05-10T11:46:25.538 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:25 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6[56225]: 2024-05-10T11:46:25.062+0000 7ff3d7bd0700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.438944+0000 front 2024-05-10T11:45:45.438900+0000 (oldest deadline 2024-05-10T11:46:10.739184+0000) 2024-05-10T11:46:25.538 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:25 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6[56225]: 2024-05-10T11:46:25.062+0000 7ff3d7bd0700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.141219+0000 front 2024-05-10T11:45:57.141270+0000 (oldest deadline 2024-05-10T11:46:18.841368+0000) 2024-05-10T11:46:25.778 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:25 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:25.524+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:53.194818+0000 front 2024-05-10T11:45:53.194735+0000 (oldest deadline 2024-05-10T11:46:23.195612+0000) 2024-05-10T11:46:25.779 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 10 11:46:25 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-5[45716]: 2024-05-10T11:46:25.487+0000 7fca3d88f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T11:46:25.779 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 10 11:46:25 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-5[45716]: 2024-05-10T11:46:25.487+0000 7fca3d88f700 -1 osd.5 94 *** Got signal Terminated *** 2024-05-10T11:46:25.779 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 10 11:46:25 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-5[45716]: 2024-05-10T11:46:25.487+0000 7fca3d88f700 -1 osd.5 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T11:46:25.779 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 10 11:46:25 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-5[45716]: 2024-05-10T11:46:25.552+0000 7fca39e83700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:48.830864+0000 front 2024-05-10T11:45:48.830948+0000 (oldest deadline 2024-05-10T11:46:13.530938+0000) 2024-05-10T11:46:25.779 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 10 11:46:25 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-5[45716]: 2024-05-10T11:46:25.552+0000 7fca39e83700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:58.033547+0000 front 2024-05-10T11:45:58.033589+0000 (oldest deadline 2024-05-10T11:46:20.333670+0000) 2024-05-10T11:46:26.018 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:25 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:25.586+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.171852+0000 front 2024-05-10T11:45:45.171825+0000 (oldest deadline 2024-05-10T11:46:10.372558+0000) 2024-05-10T11:46:26.019 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:25 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:25.586+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.274899+0000 front 2024-05-10T11:45:57.274943+0000 (oldest deadline 2024-05-10T11:46:21.375013+0000) 2024-05-10T11:46:26.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:25 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:25.919+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:44.388044+0000 front 2024-05-10T11:45:44.388108+0000 (oldest deadline 2024-05-10T11:46:10.288056+0000) 2024-05-10T11:46:26.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:25 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:25.919+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.190342+0000 front 2024-05-10T11:45:57.190423+0000 (oldest deadline 2024-05-10T11:46:19.490457+0000) 2024-05-10T11:46:26.288 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:26 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6[56225]: 2024-05-10T11:46:26.019+0000 7ff3d7bd0700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.438944+0000 front 2024-05-10T11:45:45.438900+0000 (oldest deadline 2024-05-10T11:46:10.739184+0000) 2024-05-10T11:46:26.288 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:26 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6[56225]: 2024-05-10T11:46:26.019+0000 7ff3d7bd0700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.141219+0000 front 2024-05-10T11:45:57.141270+0000 (oldest deadline 2024-05-10T11:46:18.841368+0000) 2024-05-10T11:46:26.481 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:26 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:26.033+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.703618+0000 front 2024-05-10T11:45:45.703296+0000 (oldest deadline 2024-05-10T11:46:11.003467+0000) 2024-05-10T11:46:26.481 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:26 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:26.033+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:58.005499+0000 front 2024-05-10T11:45:58.005443+0000 (oldest deadline 2024-05-10T11:46:22.705582+0000) 2024-05-10T11:46:26.752 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:26 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7[51714]: 2024-05-10T11:46:26.423+0000 7f2b921f1700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:55.454191+0000 front 2024-05-10T11:45:55.454223+0000 (oldest deadline 2024-05-10T11:46:20.754336+0000) 2024-05-10T11:46:26.778 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:26 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:26.481+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:53.194818+0000 front 2024-05-10T11:45:53.194735+0000 (oldest deadline 2024-05-10T11:46:23.195612+0000) 2024-05-10T11:46:26.779 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 10 11:46:26 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-5[45716]: 2024-05-10T11:46:26.597+0000 7fca39e83700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:48.830864+0000 front 2024-05-10T11:45:48.830948+0000 (oldest deadline 2024-05-10T11:46:13.530938+0000) 2024-05-10T11:46:26.779 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 10 11:46:26 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-5[45716]: 2024-05-10T11:46:26.597+0000 7fca39e83700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:58.033547+0000 front 2024-05-10T11:45:58.033589+0000 (oldest deadline 2024-05-10T11:46:20.333670+0000) 2024-05-10T11:46:26.990 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:26 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:26.594+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.171852+0000 front 2024-05-10T11:45:45.171825+0000 (oldest deadline 2024-05-10T11:46:10.372558+0000) 2024-05-10T11:46:26.990 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:26 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:26.594+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.274899+0000 front 2024-05-10T11:45:57.274943+0000 (oldest deadline 2024-05-10T11:46:21.375013+0000) 2024-05-10T11:46:27.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:26 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:26.898+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:44.388044+0000 front 2024-05-10T11:45:44.388108+0000 (oldest deadline 2024-05-10T11:46:10.288056+0000) 2024-05-10T11:46:27.253 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:26 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:26.898+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.190342+0000 front 2024-05-10T11:45:57.190423+0000 (oldest deadline 2024-05-10T11:46:19.490457+0000) 2024-05-10T11:46:27.253 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:26 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:26.898+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.691618+0000 front 2024-05-10T11:46:01.691702+0000 (oldest deadline 2024-05-10T11:46:26.892345+0000) 2024-05-10T11:46:27.278 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:27 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:27.008+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.703618+0000 front 2024-05-10T11:45:45.703296+0000 (oldest deadline 2024-05-10T11:46:11.003467+0000) 2024-05-10T11:46:27.278 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:27 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:27.008+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:58.005499+0000 front 2024-05-10T11:45:58.005443+0000 (oldest deadline 2024-05-10T11:46:22.705582+0000) 2024-05-10T11:46:27.288 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:26 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6[56225]: 2024-05-10T11:46:26.990+0000 7ff3d7bd0700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.438944+0000 front 2024-05-10T11:45:45.438900+0000 (oldest deadline 2024-05-10T11:46:10.739184+0000) 2024-05-10T11:46:27.288 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:26 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6[56225]: 2024-05-10T11:46:26.990+0000 7ff3d7bd0700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.141219+0000 front 2024-05-10T11:45:57.141270+0000 (oldest deadline 2024-05-10T11:46:18.841368+0000) 2024-05-10T11:46:27.752 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:27 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7[51714]: 2024-05-10T11:46:27.407+0000 7f2b921f1700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:55.454191+0000 front 2024-05-10T11:45:55.454223+0000 (oldest deadline 2024-05-10T11:46:20.754336+0000) 2024-05-10T11:46:27.752 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:27 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7[51714]: 2024-05-10T11:46:27.407+0000 7f2b921f1700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:00.754955+0000 front 2024-05-10T11:46:00.754989+0000 (oldest deadline 2024-05-10T11:46:26.654989+0000) 2024-05-10T11:46:27.778 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:27 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:27.513+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:53.194818+0000 front 2024-05-10T11:45:53.194735+0000 (oldest deadline 2024-05-10T11:46:23.195612+0000) 2024-05-10T11:46:27.779 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 10 11:46:27 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-5[45716]: 2024-05-10T11:46:27.634+0000 7fca39e83700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:48.830864+0000 front 2024-05-10T11:45:48.830948+0000 (oldest deadline 2024-05-10T11:46:13.530938+0000) 2024-05-10T11:46:27.779 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 10 11:46:27 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-5[45716]: 2024-05-10T11:46:27.634+0000 7fca39e83700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:58.033547+0000 front 2024-05-10T11:45:58.033589+0000 (oldest deadline 2024-05-10T11:46:20.333670+0000) 2024-05-10T11:46:27.980 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:27 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:27.574+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.171852+0000 front 2024-05-10T11:45:45.171825+0000 (oldest deadline 2024-05-10T11:46:10.372558+0000) 2024-05-10T11:46:27.980 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:27 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:27.574+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.274899+0000 front 2024-05-10T11:45:57.274943+0000 (oldest deadline 2024-05-10T11:46:21.375013+0000) 2024-05-10T11:46:27.980 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:27 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:27.574+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.375471+0000 front 2024-05-10T11:46:01.375489+0000 (oldest deadline 2024-05-10T11:46:26.675500+0000) 2024-05-10T11:46:28.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:27 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:27.885+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:44.388044+0000 front 2024-05-10T11:45:44.388108+0000 (oldest deadline 2024-05-10T11:46:10.288056+0000) 2024-05-10T11:46:28.253 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:27 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:27.885+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.190342+0000 front 2024-05-10T11:45:57.190423+0000 (oldest deadline 2024-05-10T11:46:19.490457+0000) 2024-05-10T11:46:28.253 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:27 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:27.885+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.691618+0000 front 2024-05-10T11:46:01.691702+0000 (oldest deadline 2024-05-10T11:46:26.892345+0000) 2024-05-10T11:46:28.278 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:27 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:27.991+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.703618+0000 front 2024-05-10T11:45:45.703296+0000 (oldest deadline 2024-05-10T11:46:11.003467+0000) 2024-05-10T11:46:28.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:27 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:27.991+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:58.005499+0000 front 2024-05-10T11:45:58.005443+0000 (oldest deadline 2024-05-10T11:46:22.705582+0000) 2024-05-10T11:46:28.288 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:27 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6[56225]: 2024-05-10T11:46:27.979+0000 7ff3d7bd0700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.438944+0000 front 2024-05-10T11:45:45.438900+0000 (oldest deadline 2024-05-10T11:46:10.739184+0000) 2024-05-10T11:46:28.288 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:27 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6[56225]: 2024-05-10T11:46:27.979+0000 7ff3d7bd0700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.141219+0000 front 2024-05-10T11:45:57.141270+0000 (oldest deadline 2024-05-10T11:46:18.841368+0000) 2024-05-10T11:46:28.752 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:28 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7[51714]: 2024-05-10T11:46:28.415+0000 7f2b921f1700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:55.454191+0000 front 2024-05-10T11:45:55.454223+0000 (oldest deadline 2024-05-10T11:46:20.754336+0000) 2024-05-10T11:46:28.752 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:28 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7[51714]: 2024-05-10T11:46:28.415+0000 7f2b921f1700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:00.754955+0000 front 2024-05-10T11:46:00.754989+0000 (oldest deadline 2024-05-10T11:46:26.654989+0000) 2024-05-10T11:46:28.778 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:28 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:28.506+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:53.194818+0000 front 2024-05-10T11:45:53.194735+0000 (oldest deadline 2024-05-10T11:46:23.195612+0000) 2024-05-10T11:46:28.779 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 10 11:46:28 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-5[45716]: 2024-05-10T11:46:28.677+0000 7fca39e83700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:48.830864+0000 front 2024-05-10T11:45:48.830948+0000 (oldest deadline 2024-05-10T11:46:13.530938+0000) 2024-05-10T11:46:28.779 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 10 11:46:28 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-5[45716]: 2024-05-10T11:46:28.677+0000 7fca39e83700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:58.033547+0000 front 2024-05-10T11:45:58.033589+0000 (oldest deadline 2024-05-10T11:46:20.333670+0000) 2024-05-10T11:46:28.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:28 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:28.530+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.171852+0000 front 2024-05-10T11:45:45.171825+0000 (oldest deadline 2024-05-10T11:46:10.372558+0000) 2024-05-10T11:46:28.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:28 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:28.530+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.274899+0000 front 2024-05-10T11:45:57.274943+0000 (oldest deadline 2024-05-10T11:46:21.375013+0000) 2024-05-10T11:46:28.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:28 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:28.530+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.375471+0000 front 2024-05-10T11:46:01.375489+0000 (oldest deadline 2024-05-10T11:46:26.675500+0000) 2024-05-10T11:46:29.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:28 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:28.859+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:44.388044+0000 front 2024-05-10T11:45:44.388108+0000 (oldest deadline 2024-05-10T11:46:10.288056+0000) 2024-05-10T11:46:29.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:28 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:28.859+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.190342+0000 front 2024-05-10T11:45:57.190423+0000 (oldest deadline 2024-05-10T11:46:19.490457+0000) 2024-05-10T11:46:29.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:28 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:28.859+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.691618+0000 front 2024-05-10T11:46:01.691702+0000 (oldest deadline 2024-05-10T11:46:26.892345+0000) 2024-05-10T11:46:29.278 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:28 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:28.997+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.703618+0000 front 2024-05-10T11:45:45.703296+0000 (oldest deadline 2024-05-10T11:46:11.003467+0000) 2024-05-10T11:46:29.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:28 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:28.998+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:58.005499+0000 front 2024-05-10T11:45:58.005443+0000 (oldest deadline 2024-05-10T11:46:22.705582+0000) 2024-05-10T11:46:29.288 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:29 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6[56225]: 2024-05-10T11:46:29.023+0000 7ff3d7bd0700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.438944+0000 front 2024-05-10T11:45:45.438900+0000 (oldest deadline 2024-05-10T11:46:10.739184+0000) 2024-05-10T11:46:29.288 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:29 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6[56225]: 2024-05-10T11:46:29.023+0000 7ff3d7bd0700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.141219+0000 front 2024-05-10T11:45:57.141270+0000 (oldest deadline 2024-05-10T11:46:18.841368+0000) 2024-05-10T11:46:29.752 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:29 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7[51714]: 2024-05-10T11:46:29.428+0000 7f2b921f1700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:55.454191+0000 front 2024-05-10T11:45:55.454223+0000 (oldest deadline 2024-05-10T11:46:20.754336+0000) 2024-05-10T11:46:29.752 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:29 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7[51714]: 2024-05-10T11:46:29.428+0000 7f2b921f1700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:00.754955+0000 front 2024-05-10T11:46:00.754989+0000 (oldest deadline 2024-05-10T11:46:26.654989+0000) 2024-05-10T11:46:29.778 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 10 11:46:29 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-5[45716]: 2024-05-10T11:46:29.649+0000 7fca39e83700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:48.830864+0000 front 2024-05-10T11:45:48.830948+0000 (oldest deadline 2024-05-10T11:46:13.530938+0000) 2024-05-10T11:46:29.778 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 10 11:46:29 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-5[45716]: 2024-05-10T11:46:29.649+0000 7fca39e83700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:58.033547+0000 front 2024-05-10T11:45:58.033589+0000 (oldest deadline 2024-05-10T11:46:20.333670+0000) 2024-05-10T11:46:29.779 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:29 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:29.478+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:53.194818+0000 front 2024-05-10T11:45:53.194735+0000 (oldest deadline 2024-05-10T11:46:23.195612+0000) 2024-05-10T11:46:29.779 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:29 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:29.478+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:03.196106+0000 front 2024-05-10T11:46:03.196084+0000 (oldest deadline 2024-05-10T11:46:29.096288+0000) 2024-05-10T11:46:30.038 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:29 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:29.542+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.171852+0000 front 2024-05-10T11:45:45.171825+0000 (oldest deadline 2024-05-10T11:46:10.372558+0000) 2024-05-10T11:46:30.038 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:29 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:29.542+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.274899+0000 front 2024-05-10T11:45:57.274943+0000 (oldest deadline 2024-05-10T11:46:21.375013+0000) 2024-05-10T11:46:30.038 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:29 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:29.542+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.375471+0000 front 2024-05-10T11:46:01.375489+0000 (oldest deadline 2024-05-10T11:46:26.675500+0000) 2024-05-10T11:46:30.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:29 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:29.893+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:44.388044+0000 front 2024-05-10T11:45:44.388108+0000 (oldest deadline 2024-05-10T11:46:10.288056+0000) 2024-05-10T11:46:30.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:29 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:29.893+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.190342+0000 front 2024-05-10T11:45:57.190423+0000 (oldest deadline 2024-05-10T11:46:19.490457+0000) 2024-05-10T11:46:30.253 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:29 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:29.893+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.691618+0000 front 2024-05-10T11:46:01.691702+0000 (oldest deadline 2024-05-10T11:46:26.892345+0000) 2024-05-10T11:46:30.278 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:29 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:29.976+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.703618+0000 front 2024-05-10T11:45:45.703296+0000 (oldest deadline 2024-05-10T11:46:11.003467+0000) 2024-05-10T11:46:30.278 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:29 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:29.976+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:58.005499+0000 front 2024-05-10T11:45:58.005443+0000 (oldest deadline 2024-05-10T11:46:22.705582+0000) 2024-05-10T11:46:30.499 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:30 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6[56225]: 2024-05-10T11:46:30.058+0000 7ff3d7bd0700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.438944+0000 front 2024-05-10T11:45:45.438900+0000 (oldest deadline 2024-05-10T11:46:10.739184+0000) 2024-05-10T11:46:30.499 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:30 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6[56225]: 2024-05-10T11:46:30.058+0000 7ff3d7bd0700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.141219+0000 front 2024-05-10T11:45:57.141270+0000 (oldest deadline 2024-05-10T11:46:18.841368+0000) 2024-05-10T11:46:30.752 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:30 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7[51714]: 2024-05-10T11:46:30.455+0000 7f2b921f1700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:55.454191+0000 front 2024-05-10T11:45:55.454223+0000 (oldest deadline 2024-05-10T11:46:20.754336+0000) 2024-05-10T11:46:30.753 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:30 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7[51714]: 2024-05-10T11:46:30.455+0000 7f2b921f1700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:00.754955+0000 front 2024-05-10T11:46:00.754989+0000 (oldest deadline 2024-05-10T11:46:26.654989+0000) 2024-05-10T11:46:30.778 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:30 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:30.435+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:53.194818+0000 front 2024-05-10T11:45:53.194735+0000 (oldest deadline 2024-05-10T11:46:23.195612+0000) 2024-05-10T11:46:30.778 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:30 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:30.435+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:03.196106+0000 front 2024-05-10T11:46:03.196084+0000 (oldest deadline 2024-05-10T11:46:29.096288+0000) 2024-05-10T11:46:30.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:30 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:30.498+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.171852+0000 front 2024-05-10T11:45:45.171825+0000 (oldest deadline 2024-05-10T11:46:10.372558+0000) 2024-05-10T11:46:30.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:30 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:30.498+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.274899+0000 front 2024-05-10T11:45:57.274943+0000 (oldest deadline 2024-05-10T11:46:21.375013+0000) 2024-05-10T11:46:30.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:30 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:30.498+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.375471+0000 front 2024-05-10T11:46:01.375489+0000 (oldest deadline 2024-05-10T11:46:26.675500+0000) 2024-05-10T11:46:31.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:30 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:30.881+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:44.388044+0000 front 2024-05-10T11:45:44.388108+0000 (oldest deadline 2024-05-10T11:46:10.288056+0000) 2024-05-10T11:46:31.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:30 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:30.881+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.190342+0000 front 2024-05-10T11:45:57.190423+0000 (oldest deadline 2024-05-10T11:46:19.490457+0000) 2024-05-10T11:46:31.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:30 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:30.881+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.691618+0000 front 2024-05-10T11:46:01.691702+0000 (oldest deadline 2024-05-10T11:46:26.892345+0000) 2024-05-10T11:46:31.278 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 10 11:46:31 smithi188 podman[76478]: 2024-05-10 11:46:31.004280285 +0000 UTC m=+5.642565236 container died 083c0cb86b40a183827198736bc772669fec58cddc1a0f814d0b547a4a317eaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-5, org.label-schema.license=GPLv2, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240507, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-10T11:46:31.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:30 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:30.996+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.703618+0000 front 2024-05-10T11:45:45.703296+0000 (oldest deadline 2024-05-10T11:46:11.003467+0000) 2024-05-10T11:46:31.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:30 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:30.996+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:58.005499+0000 front 2024-05-10T11:45:58.005443+0000 (oldest deadline 2024-05-10T11:46:22.705582+0000) 2024-05-10T11:46:31.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:30 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:30.996+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:05.606931+0000 front 2024-05-10T11:46:05.606957+0000 (oldest deadline 2024-05-10T11:46:30.907138+0000) 2024-05-10T11:46:31.470 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:31 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6[56225]: 2024-05-10T11:46:31.040+0000 7ff3d7bd0700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.438944+0000 front 2024-05-10T11:45:45.438900+0000 (oldest deadline 2024-05-10T11:46:10.739184+0000) 2024-05-10T11:46:31.470 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:31 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6[56225]: 2024-05-10T11:46:31.040+0000 7ff3d7bd0700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.141219+0000 front 2024-05-10T11:45:57.141270+0000 (oldest deadline 2024-05-10T11:46:18.841368+0000) 2024-05-10T11:46:31.680 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 10 11:46:31 smithi188 podman[76478]: 2024-05-10 11:46:31.41244425 +0000 UTC m=+6.050729201 container cleanup 083c0cb86b40a183827198736bc772669fec58cddc1a0f814d0b547a4a317eaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-5, CEPH_POINT_RELEASE=, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-05-10T11:46:31.680 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 10 11:46:31 smithi188 bash[76478]: ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-5 2024-05-10T11:46:31.680 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:31 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:31.473+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:53.194818+0000 front 2024-05-10T11:45:53.194735+0000 (oldest deadline 2024-05-10T11:46:23.195612+0000) 2024-05-10T11:46:31.680 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:31 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:31.473+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:03.196106+0000 front 2024-05-10T11:46:03.196084+0000 (oldest deadline 2024-05-10T11:46:29.096288+0000) 2024-05-10T11:46:31.752 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:31 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7[51714]: 2024-05-10T11:46:31.411+0000 7f2b921f1700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:55.454191+0000 front 2024-05-10T11:45:55.454223+0000 (oldest deadline 2024-05-10T11:46:20.754336+0000) 2024-05-10T11:46:31.752 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:31 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7[51714]: 2024-05-10T11:46:31.411+0000 7f2b921f1700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:00.754955+0000 front 2024-05-10T11:46:00.754989+0000 (oldest deadline 2024-05-10T11:46:26.654989+0000) 2024-05-10T11:46:31.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:31 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:31.469+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.171852+0000 front 2024-05-10T11:45:45.171825+0000 (oldest deadline 2024-05-10T11:46:10.372558+0000) 2024-05-10T11:46:31.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:31 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:31.469+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.274899+0000 front 2024-05-10T11:45:57.274943+0000 (oldest deadline 2024-05-10T11:46:21.375013+0000) 2024-05-10T11:46:31.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:31 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:31.469+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.375471+0000 front 2024-05-10T11:46:01.375489+0000 (oldest deadline 2024-05-10T11:46:26.675500+0000) 2024-05-10T11:46:32.000 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 10 11:46:31 smithi188 podman[76497]: 2024-05-10 11:46:31.680661108 +0000 UTC m=+0.673165559 container remove 083c0cb86b40a183827198736bc772669fec58cddc1a0f814d0b547a4a317eaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, CEPH_POINT_RELEASE=, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08) 2024-05-10T11:46:32.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:31 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:31.867+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:44.388044+0000 front 2024-05-10T11:45:44.388108+0000 (oldest deadline 2024-05-10T11:46:10.288056+0000) 2024-05-10T11:46:32.253 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:31 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:31.867+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.190342+0000 front 2024-05-10T11:45:57.190423+0000 (oldest deadline 2024-05-10T11:46:19.490457+0000) 2024-05-10T11:46:32.253 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:31 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:31.867+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.691618+0000 front 2024-05-10T11:46:01.691702+0000 (oldest deadline 2024-05-10T11:46:26.892345+0000) 2024-05-10T11:46:32.279 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 10 11:46:32 smithi188 podman[76588]: 2024-05-10 11:46:31.934228859 +0000 UTC m=+0.018341344 image pull ee5e4f21ef58047cb638cccd5cfbe8f6f43412b56c94b1d7fbd9cf3cb90a53a5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f 2024-05-10T11:46:32.279 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 10 11:46:32 smithi188 podman[76588]: 2024-05-10 11:46:32.065268394 +0000 UTC m=+0.149380861 container create 066a2dd4e4b340be96a4fd6d264306786b464fcecf66196114571cd27cc19956 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-5-deactivate, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.build-date=20240507) 2024-05-10T11:46:32.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:32 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:32.000+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.703618+0000 front 2024-05-10T11:45:45.703296+0000 (oldest deadline 2024-05-10T11:46:11.003467+0000) 2024-05-10T11:46:32.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:32 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:32.000+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:58.005499+0000 front 2024-05-10T11:45:58.005443+0000 (oldest deadline 2024-05-10T11:46:22.705582+0000) 2024-05-10T11:46:32.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:32 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:32.000+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:05.606931+0000 front 2024-05-10T11:46:05.606957+0000 (oldest deadline 2024-05-10T11:46:30.907138+0000) 2024-05-10T11:46:32.288 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:32 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6[56225]: 2024-05-10T11:46:32.016+0000 7ff3d7bd0700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.438944+0000 front 2024-05-10T11:45:45.438900+0000 (oldest deadline 2024-05-10T11:46:10.739184+0000) 2024-05-10T11:46:32.288 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:32 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6[56225]: 2024-05-10T11:46:32.016+0000 7ff3d7bd0700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.141219+0000 front 2024-05-10T11:45:57.141270+0000 (oldest deadline 2024-05-10T11:46:18.841368+0000) 2024-05-10T11:46:32.288 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:32 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6[56225]: 2024-05-10T11:46:32.016+0000 7ff3d7bd0700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:05.843273+0000 front 2024-05-10T11:46:05.843296+0000 (oldest deadline 2024-05-10T11:46:31.143260+0000) 2024-05-10T11:46:32.577 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:32 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:32.472+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:53.194818+0000 front 2024-05-10T11:45:53.194735+0000 (oldest deadline 2024-05-10T11:46:23.195612+0000) 2024-05-10T11:46:32.577 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:32 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:32.472+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:03.196106+0000 front 2024-05-10T11:46:03.196084+0000 (oldest deadline 2024-05-10T11:46:29.096288+0000) 2024-05-10T11:46:32.577 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 10 11:46:32 smithi188 podman[76588]: 2024-05-10 11:46:32.284724522 +0000 UTC m=+0.368837000 container init 066a2dd4e4b340be96a4fd6d264306786b464fcecf66196114571cd27cc19956 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-05-10T11:46:32.577 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 10 11:46:32 smithi188 podman[76588]: 2024-05-10 11:46:32.287982319 +0000 UTC m=+0.372094786 container start 066a2dd4e4b340be96a4fd6d264306786b464fcecf66196114571cd27cc19956 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-10T11:46:32.577 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 10 11:46:32 smithi188 podman[76588]: 2024-05-10 11:46:32.376300107 +0000 UTC m=+0.460412613 container attach 066a2dd4e4b340be96a4fd6d264306786b464fcecf66196114571cd27cc19956 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=) 2024-05-10T11:46:32.753 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:32 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7[51714]: 2024-05-10T11:46:32.455+0000 7f2b921f1700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:55.454191+0000 front 2024-05-10T11:45:55.454223+0000 (oldest deadline 2024-05-10T11:46:20.754336+0000) 2024-05-10T11:46:32.753 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:32 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7[51714]: 2024-05-10T11:46:32.455+0000 7f2b921f1700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:00.754955+0000 front 2024-05-10T11:46:00.754989+0000 (oldest deadline 2024-05-10T11:46:26.654989+0000) 2024-05-10T11:46:32.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:32 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:32.450+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.171852+0000 front 2024-05-10T11:45:45.171825+0000 (oldest deadline 2024-05-10T11:46:10.372558+0000) 2024-05-10T11:46:32.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:32 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:32.450+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.274899+0000 front 2024-05-10T11:45:57.274943+0000 (oldest deadline 2024-05-10T11:46:21.375013+0000) 2024-05-10T11:46:32.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:32 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:32.450+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.375471+0000 front 2024-05-10T11:46:01.375489+0000 (oldest deadline 2024-05-10T11:46:26.675500+0000) 2024-05-10T11:46:32.870 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 10 11:46:32 smithi188 podman[76588]: 2024-05-10 11:46:32.577459495 +0000 UTC m=+0.661571977 container died 066a2dd4e4b340be96a4fd6d264306786b464fcecf66196114571cd27cc19956 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507) 2024-05-10T11:46:33.180 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:33 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:33.040+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.703618+0000 front 2024-05-10T11:45:45.703296+0000 (oldest deadline 2024-05-10T11:46:11.003467+0000) 2024-05-10T11:46:33.180 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:33 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:33.040+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:58.005499+0000 front 2024-05-10T11:45:58.005443+0000 (oldest deadline 2024-05-10T11:46:22.705582+0000) 2024-05-10T11:46:33.180 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:33 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:33.040+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:05.606931+0000 front 2024-05-10T11:46:05.606957+0000 (oldest deadline 2024-05-10T11:46:30.907138+0000) 2024-05-10T11:46:33.201 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@osd.5.service' 2024-05-10T11:46:33.240 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T11:46:33.241 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-05-10T11:46:33.241 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-05-10T11:46:33.241 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@osd.6 2024-05-10T11:46:33.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:32 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:32.904+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:44.388044+0000 front 2024-05-10T11:45:44.388108+0000 (oldest deadline 2024-05-10T11:46:10.288056+0000) 2024-05-10T11:46:33.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:32 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:32.904+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.190342+0000 front 2024-05-10T11:45:57.190423+0000 (oldest deadline 2024-05-10T11:46:19.490457+0000) 2024-05-10T11:46:33.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:32 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:32.904+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.691618+0000 front 2024-05-10T11:46:01.691702+0000 (oldest deadline 2024-05-10T11:46:26.892345+0000) 2024-05-10T11:46:33.294 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:33 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6[56225]: 2024-05-10T11:46:33.039+0000 7ff3d7bd0700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.438944+0000 front 2024-05-10T11:45:45.438900+0000 (oldest deadline 2024-05-10T11:46:10.739184+0000) 2024-05-10T11:46:33.294 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:33 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6[56225]: 2024-05-10T11:46:33.039+0000 7ff3d7bd0700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.141219+0000 front 2024-05-10T11:45:57.141270+0000 (oldest deadline 2024-05-10T11:46:18.841368+0000) 2024-05-10T11:46:33.294 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:33 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6[56225]: 2024-05-10T11:46:33.039+0000 7ff3d7bd0700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:05.843273+0000 front 2024-05-10T11:46:05.843296+0000 (oldest deadline 2024-05-10T11:46:31.143260+0000) 2024-05-10T11:46:33.562 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:33 smithi100 systemd[1]: Stopping Ceph osd.6 for 20a542cc-0ebd-11ef-bc98-c7b262605968... 2024-05-10T11:46:33.562 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:33 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:33.428+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.171852+0000 front 2024-05-10T11:45:45.171825+0000 (oldest deadline 2024-05-10T11:46:10.372558+0000) 2024-05-10T11:46:33.563 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:33 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:33.428+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.274899+0000 front 2024-05-10T11:45:57.274943+0000 (oldest deadline 2024-05-10T11:46:21.375013+0000) 2024-05-10T11:46:33.563 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:33 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:33.428+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.375471+0000 front 2024-05-10T11:46:01.375489+0000 (oldest deadline 2024-05-10T11:46:26.675500+0000) 2024-05-10T11:46:33.752 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:33 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7[51714]: 2024-05-10T11:46:33.469+0000 7f2b921f1700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:55.454191+0000 front 2024-05-10T11:45:55.454223+0000 (oldest deadline 2024-05-10T11:46:20.754336+0000) 2024-05-10T11:46:33.752 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:33 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7[51714]: 2024-05-10T11:46:33.469+0000 7f2b921f1700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:00.754955+0000 front 2024-05-10T11:46:00.754989+0000 (oldest deadline 2024-05-10T11:46:26.654989+0000) 2024-05-10T11:46:33.778 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:33 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:33.517+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:53.194818+0000 front 2024-05-10T11:45:53.194735+0000 (oldest deadline 2024-05-10T11:46:23.195612+0000) 2024-05-10T11:46:33.778 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:33 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:33.517+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:03.196106+0000 front 2024-05-10T11:46:03.196084+0000 (oldest deadline 2024-05-10T11:46:29.096288+0000) 2024-05-10T11:46:34.038 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:33 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6[56225]: 2024-05-10T11:46:33.562+0000 7ff3db5dc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T11:46:34.038 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:33 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6[56225]: 2024-05-10T11:46:33.562+0000 7ff3db5dc700 -1 osd.6 94 *** Got signal Terminated *** 2024-05-10T11:46:34.038 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:33 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6[56225]: 2024-05-10T11:46:33.562+0000 7ff3db5dc700 -1 osd.6 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T11:46:34.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:33 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:33.903+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:44.388044+0000 front 2024-05-10T11:45:44.388108+0000 (oldest deadline 2024-05-10T11:46:10.288056+0000) 2024-05-10T11:46:34.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:33 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:33.903+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.190342+0000 front 2024-05-10T11:45:57.190423+0000 (oldest deadline 2024-05-10T11:46:19.490457+0000) 2024-05-10T11:46:34.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:33 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:33.903+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.691618+0000 front 2024-05-10T11:46:01.691702+0000 (oldest deadline 2024-05-10T11:46:26.892345+0000) 2024-05-10T11:46:34.385 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:34 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:34.049+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.703618+0000 front 2024-05-10T11:45:45.703296+0000 (oldest deadline 2024-05-10T11:46:11.003467+0000) 2024-05-10T11:46:34.385 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:34 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:34.049+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:58.005499+0000 front 2024-05-10T11:45:58.005443+0000 (oldest deadline 2024-05-10T11:46:22.705582+0000) 2024-05-10T11:46:34.385 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:34 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:34.049+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:05.606931+0000 front 2024-05-10T11:46:05.606957+0000 (oldest deadline 2024-05-10T11:46:30.907138+0000) 2024-05-10T11:46:34.405 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:34 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6[56225]: 2024-05-10T11:46:34.060+0000 7ff3d7bd0700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.438944+0000 front 2024-05-10T11:45:45.438900+0000 (oldest deadline 2024-05-10T11:46:10.739184+0000) 2024-05-10T11:46:34.405 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:34 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6[56225]: 2024-05-10T11:46:34.060+0000 7ff3d7bd0700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.141219+0000 front 2024-05-10T11:45:57.141270+0000 (oldest deadline 2024-05-10T11:46:18.841368+0000) 2024-05-10T11:46:34.405 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:34 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6[56225]: 2024-05-10T11:46:34.060+0000 7ff3d7bd0700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:05.843273+0000 front 2024-05-10T11:46:05.843296+0000 (oldest deadline 2024-05-10T11:46:31.143260+0000) 2024-05-10T11:46:34.752 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:34 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7[51714]: 2024-05-10T11:46:34.423+0000 7f2b921f1700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:55.454191+0000 front 2024-05-10T11:45:55.454223+0000 (oldest deadline 2024-05-10T11:46:20.754336+0000) 2024-05-10T11:46:34.753 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:34 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7[51714]: 2024-05-10T11:46:34.423+0000 7f2b921f1700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:00.754955+0000 front 2024-05-10T11:46:00.754989+0000 (oldest deadline 2024-05-10T11:46:26.654989+0000) 2024-05-10T11:46:34.778 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:34 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:34.536+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:53.194818+0000 front 2024-05-10T11:45:53.194735+0000 (oldest deadline 2024-05-10T11:46:23.195612+0000) 2024-05-10T11:46:34.778 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:34 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:34.536+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:03.196106+0000 front 2024-05-10T11:46:03.196084+0000 (oldest deadline 2024-05-10T11:46:29.096288+0000) 2024-05-10T11:46:34.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:34 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:34.405+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.171852+0000 front 2024-05-10T11:45:45.171825+0000 (oldest deadline 2024-05-10T11:46:10.372558+0000) 2024-05-10T11:46:34.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:34 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:34.405+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.274899+0000 front 2024-05-10T11:45:57.274943+0000 (oldest deadline 2024-05-10T11:46:21.375013+0000) 2024-05-10T11:46:34.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:34 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:34.405+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.375471+0000 front 2024-05-10T11:46:01.375489+0000 (oldest deadline 2024-05-10T11:46:26.675500+0000) 2024-05-10T11:46:35.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:34 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:34.890+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:44.388044+0000 front 2024-05-10T11:45:44.388108+0000 (oldest deadline 2024-05-10T11:46:10.288056+0000) 2024-05-10T11:46:35.253 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:34 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:34.890+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.190342+0000 front 2024-05-10T11:45:57.190423+0000 (oldest deadline 2024-05-10T11:46:19.490457+0000) 2024-05-10T11:46:35.253 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:34 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:34.890+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.691618+0000 front 2024-05-10T11:46:01.691702+0000 (oldest deadline 2024-05-10T11:46:26.892345+0000) 2024-05-10T11:46:35.429 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:35 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6[56225]: 2024-05-10T11:46:35.099+0000 7ff3d7bd0700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.438944+0000 front 2024-05-10T11:45:45.438900+0000 (oldest deadline 2024-05-10T11:46:10.739184+0000) 2024-05-10T11:46:35.429 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:35 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6[56225]: 2024-05-10T11:46:35.099+0000 7ff3d7bd0700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.141219+0000 front 2024-05-10T11:45:57.141270+0000 (oldest deadline 2024-05-10T11:46:18.841368+0000) 2024-05-10T11:46:35.429 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:35 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6[56225]: 2024-05-10T11:46:35.099+0000 7ff3d7bd0700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:05.843273+0000 front 2024-05-10T11:46:05.843296+0000 (oldest deadline 2024-05-10T11:46:31.143260+0000) 2024-05-10T11:46:35.529 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:35 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:35.033+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.703618+0000 front 2024-05-10T11:45:45.703296+0000 (oldest deadline 2024-05-10T11:46:11.003467+0000) 2024-05-10T11:46:35.529 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:35 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:35.033+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:58.005499+0000 front 2024-05-10T11:45:58.005443+0000 (oldest deadline 2024-05-10T11:46:22.705582+0000) 2024-05-10T11:46:35.529 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:35 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:35.033+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:05.606931+0000 front 2024-05-10T11:46:05.606957+0000 (oldest deadline 2024-05-10T11:46:30.907138+0000) 2024-05-10T11:46:35.752 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:35 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7[51714]: 2024-05-10T11:46:35.396+0000 7f2b921f1700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:55.454191+0000 front 2024-05-10T11:45:55.454223+0000 (oldest deadline 2024-05-10T11:46:20.754336+0000) 2024-05-10T11:46:35.752 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:35 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7[51714]: 2024-05-10T11:46:35.396+0000 7f2b921f1700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:00.754955+0000 front 2024-05-10T11:46:00.754989+0000 (oldest deadline 2024-05-10T11:46:26.654989+0000) 2024-05-10T11:46:35.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:35 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:35.428+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.171852+0000 front 2024-05-10T11:45:45.171825+0000 (oldest deadline 2024-05-10T11:46:10.372558+0000) 2024-05-10T11:46:35.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:35 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:35.428+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.274899+0000 front 2024-05-10T11:45:57.274943+0000 (oldest deadline 2024-05-10T11:46:21.375013+0000) 2024-05-10T11:46:35.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:35 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:35.428+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.375471+0000 front 2024-05-10T11:46:01.375489+0000 (oldest deadline 2024-05-10T11:46:26.675500+0000) 2024-05-10T11:46:36.028 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:35 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:35.550+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:53.194818+0000 front 2024-05-10T11:45:53.194735+0000 (oldest deadline 2024-05-10T11:46:23.195612+0000) 2024-05-10T11:46:36.029 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:35 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:35.550+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:03.196106+0000 front 2024-05-10T11:46:03.196084+0000 (oldest deadline 2024-05-10T11:46:29.096288+0000) 2024-05-10T11:46:36.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:35 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:35.931+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:44.388044+0000 front 2024-05-10T11:45:44.388108+0000 (oldest deadline 2024-05-10T11:46:10.288056+0000) 2024-05-10T11:46:36.253 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:35 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:35.931+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.190342+0000 front 2024-05-10T11:45:57.190423+0000 (oldest deadline 2024-05-10T11:46:19.490457+0000) 2024-05-10T11:46:36.253 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:35 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:35.931+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.691618+0000 front 2024-05-10T11:46:01.691702+0000 (oldest deadline 2024-05-10T11:46:26.892345+0000) 2024-05-10T11:46:36.449 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:36 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6[56225]: 2024-05-10T11:46:36.089+0000 7ff3d7bd0700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.438944+0000 front 2024-05-10T11:45:45.438900+0000 (oldest deadline 2024-05-10T11:46:10.739184+0000) 2024-05-10T11:46:36.449 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:36 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6[56225]: 2024-05-10T11:46:36.089+0000 7ff3d7bd0700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.141219+0000 front 2024-05-10T11:45:57.141270+0000 (oldest deadline 2024-05-10T11:46:18.841368+0000) 2024-05-10T11:46:36.449 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:36 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6[56225]: 2024-05-10T11:46:36.089+0000 7ff3d7bd0700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:05.843273+0000 front 2024-05-10T11:46:05.843296+0000 (oldest deadline 2024-05-10T11:46:31.143260+0000) 2024-05-10T11:46:36.450 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:36 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6[56225]: 2024-05-10T11:46:36.089+0000 7ff3d7bd0700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:13.444225+0000 front 2024-05-10T11:46:13.444201+0000 (oldest deadline 2024-05-10T11:46:35.744326+0000) 2024-05-10T11:46:36.528 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:36 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:36.054+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.703618+0000 front 2024-05-10T11:45:45.703296+0000 (oldest deadline 2024-05-10T11:46:11.003467+0000) 2024-05-10T11:46:36.528 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:36 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:36.054+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:58.005499+0000 front 2024-05-10T11:45:58.005443+0000 (oldest deadline 2024-05-10T11:46:22.705582+0000) 2024-05-10T11:46:36.529 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:36 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:36.054+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:05.606931+0000 front 2024-05-10T11:46:05.606957+0000 (oldest deadline 2024-05-10T11:46:30.907138+0000) 2024-05-10T11:46:36.752 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:36 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7[51714]: 2024-05-10T11:46:36.381+0000 7f2b921f1700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:55.454191+0000 front 2024-05-10T11:45:55.454223+0000 (oldest deadline 2024-05-10T11:46:20.754336+0000) 2024-05-10T11:46:36.752 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:36 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7[51714]: 2024-05-10T11:46:36.381+0000 7f2b921f1700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:00.754955+0000 front 2024-05-10T11:46:00.754989+0000 (oldest deadline 2024-05-10T11:46:26.654989+0000) 2024-05-10T11:46:36.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:36 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:36.449+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.171852+0000 front 2024-05-10T11:45:45.171825+0000 (oldest deadline 2024-05-10T11:46:10.372558+0000) 2024-05-10T11:46:36.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:36 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:36.449+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.274899+0000 front 2024-05-10T11:45:57.274943+0000 (oldest deadline 2024-05-10T11:46:21.375013+0000) 2024-05-10T11:46:36.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:36 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:36.449+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.375471+0000 front 2024-05-10T11:46:01.375489+0000 (oldest deadline 2024-05-10T11:46:26.675500+0000) 2024-05-10T11:46:36.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:36 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:36.449+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:12.877843+0000 front 2024-05-10T11:46:12.877870+0000 (oldest deadline 2024-05-10T11:46:36.378001+0000) 2024-05-10T11:46:37.012 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:36 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:36.583+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:53.194818+0000 front 2024-05-10T11:45:53.194735+0000 (oldest deadline 2024-05-10T11:46:23.195612+0000) 2024-05-10T11:46:37.012 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:36 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:36.583+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:03.196106+0000 front 2024-05-10T11:46:03.196084+0000 (oldest deadline 2024-05-10T11:46:29.096288+0000) 2024-05-10T11:46:37.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:36 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:36.928+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:44.388044+0000 front 2024-05-10T11:45:44.388108+0000 (oldest deadline 2024-05-10T11:46:10.288056+0000) 2024-05-10T11:46:37.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:36 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:36.928+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.190342+0000 front 2024-05-10T11:45:57.190423+0000 (oldest deadline 2024-05-10T11:46:19.490457+0000) 2024-05-10T11:46:37.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:36 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:36.928+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.691618+0000 front 2024-05-10T11:46:01.691702+0000 (oldest deadline 2024-05-10T11:46:26.892345+0000) 2024-05-10T11:46:37.278 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:37 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:37.012+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.703618+0000 front 2024-05-10T11:45:45.703296+0000 (oldest deadline 2024-05-10T11:46:11.003467+0000) 2024-05-10T11:46:37.278 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:37 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:37.012+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:58.005499+0000 front 2024-05-10T11:45:58.005443+0000 (oldest deadline 2024-05-10T11:46:22.705582+0000) 2024-05-10T11:46:37.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:37 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:37.012+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:05.606931+0000 front 2024-05-10T11:46:05.606957+0000 (oldest deadline 2024-05-10T11:46:30.907138+0000) 2024-05-10T11:46:37.466 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:37 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6[56225]: 2024-05-10T11:46:37.048+0000 7ff3d7bd0700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.438944+0000 front 2024-05-10T11:45:45.438900+0000 (oldest deadline 2024-05-10T11:46:10.739184+0000) 2024-05-10T11:46:37.467 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:37 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6[56225]: 2024-05-10T11:46:37.048+0000 7ff3d7bd0700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.141219+0000 front 2024-05-10T11:45:57.141270+0000 (oldest deadline 2024-05-10T11:46:18.841368+0000) 2024-05-10T11:46:37.467 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:37 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6[56225]: 2024-05-10T11:46:37.048+0000 7ff3d7bd0700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:05.843273+0000 front 2024-05-10T11:46:05.843296+0000 (oldest deadline 2024-05-10T11:46:31.143260+0000) 2024-05-10T11:46:37.467 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:37 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6[56225]: 2024-05-10T11:46:37.048+0000 7ff3d7bd0700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:13.444225+0000 front 2024-05-10T11:46:13.444201+0000 (oldest deadline 2024-05-10T11:46:35.744326+0000) 2024-05-10T11:46:37.752 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:37 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7[51714]: 2024-05-10T11:46:37.413+0000 7f2b921f1700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:55.454191+0000 front 2024-05-10T11:45:55.454223+0000 (oldest deadline 2024-05-10T11:46:20.754336+0000) 2024-05-10T11:46:37.752 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:37 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7[51714]: 2024-05-10T11:46:37.413+0000 7f2b921f1700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:00.754955+0000 front 2024-05-10T11:46:00.754989+0000 (oldest deadline 2024-05-10T11:46:26.654989+0000) 2024-05-10T11:46:37.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:37 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:37.466+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.171852+0000 front 2024-05-10T11:45:45.171825+0000 (oldest deadline 2024-05-10T11:46:10.372558+0000) 2024-05-10T11:46:37.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:37 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:37.466+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.274899+0000 front 2024-05-10T11:45:57.274943+0000 (oldest deadline 2024-05-10T11:46:21.375013+0000) 2024-05-10T11:46:37.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:37 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:37.466+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.375471+0000 front 2024-05-10T11:46:01.375489+0000 (oldest deadline 2024-05-10T11:46:26.675500+0000) 2024-05-10T11:46:37.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:37 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:37.466+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:12.877843+0000 front 2024-05-10T11:46:12.877870+0000 (oldest deadline 2024-05-10T11:46:36.378001+0000) 2024-05-10T11:46:38.028 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:37 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:37.607+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:53.194818+0000 front 2024-05-10T11:45:53.194735+0000 (oldest deadline 2024-05-10T11:46:23.195612+0000) 2024-05-10T11:46:38.029 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:37 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:37.607+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:03.196106+0000 front 2024-05-10T11:46:03.196084+0000 (oldest deadline 2024-05-10T11:46:29.096288+0000) 2024-05-10T11:46:38.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:37 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:37.954+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:44.388044+0000 front 2024-05-10T11:45:44.388108+0000 (oldest deadline 2024-05-10T11:46:10.288056+0000) 2024-05-10T11:46:38.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:37 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:37.954+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.190342+0000 front 2024-05-10T11:45:57.190423+0000 (oldest deadline 2024-05-10T11:46:19.490457+0000) 2024-05-10T11:46:38.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:37 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:37.954+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.691618+0000 front 2024-05-10T11:46:01.691702+0000 (oldest deadline 2024-05-10T11:46:26.892345+0000) 2024-05-10T11:46:38.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:37 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:37.954+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:13.794432+0000 front 2024-05-10T11:46:13.794449+0000 (oldest deadline 2024-05-10T11:46:37.294473+0000) 2024-05-10T11:46:38.501 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:38 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6[56225]: 2024-05-10T11:46:38.069+0000 7ff3d7bd0700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.438944+0000 front 2024-05-10T11:45:45.438900+0000 (oldest deadline 2024-05-10T11:46:10.739184+0000) 2024-05-10T11:46:38.501 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:38 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6[56225]: 2024-05-10T11:46:38.069+0000 7ff3d7bd0700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.141219+0000 front 2024-05-10T11:45:57.141270+0000 (oldest deadline 2024-05-10T11:46:18.841368+0000) 2024-05-10T11:46:38.501 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:38 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6[56225]: 2024-05-10T11:46:38.069+0000 7ff3d7bd0700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:05.843273+0000 front 2024-05-10T11:46:05.843296+0000 (oldest deadline 2024-05-10T11:46:31.143260+0000) 2024-05-10T11:46:38.501 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:38 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6[56225]: 2024-05-10T11:46:38.069+0000 7ff3d7bd0700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:13.444225+0000 front 2024-05-10T11:46:13.444201+0000 (oldest deadline 2024-05-10T11:46:35.744326+0000) 2024-05-10T11:46:38.528 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:38 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:38.062+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.703618+0000 front 2024-05-10T11:45:45.703296+0000 (oldest deadline 2024-05-10T11:46:11.003467+0000) 2024-05-10T11:46:38.529 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:38 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:38.062+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:58.005499+0000 front 2024-05-10T11:45:58.005443+0000 (oldest deadline 2024-05-10T11:46:22.705582+0000) 2024-05-10T11:46:38.529 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:38 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:38.062+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:05.606931+0000 front 2024-05-10T11:46:05.606957+0000 (oldest deadline 2024-05-10T11:46:30.907138+0000) 2024-05-10T11:46:38.752 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:38 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7[51714]: 2024-05-10T11:46:38.445+0000 7f2b921f1700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:55.454191+0000 front 2024-05-10T11:45:55.454223+0000 (oldest deadline 2024-05-10T11:46:20.754336+0000) 2024-05-10T11:46:38.752 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:38 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7[51714]: 2024-05-10T11:46:38.445+0000 7f2b921f1700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:00.754955+0000 front 2024-05-10T11:46:00.754989+0000 (oldest deadline 2024-05-10T11:46:26.654989+0000) 2024-05-10T11:46:38.753 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:38 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7[51714]: 2024-05-10T11:46:38.445+0000 7f2b921f1700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:13.057027+0000 front 2024-05-10T11:46:13.056878+0000 (oldest deadline 2024-05-10T11:46:38.357046+0000) 2024-05-10T11:46:38.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:38 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:38.500+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.171852+0000 front 2024-05-10T11:45:45.171825+0000 (oldest deadline 2024-05-10T11:46:10.372558+0000) 2024-05-10T11:46:38.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:38 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:38.500+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.274899+0000 front 2024-05-10T11:45:57.274943+0000 (oldest deadline 2024-05-10T11:46:21.375013+0000) 2024-05-10T11:46:38.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:38 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:38.500+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.375471+0000 front 2024-05-10T11:46:01.375489+0000 (oldest deadline 2024-05-10T11:46:26.675500+0000) 2024-05-10T11:46:38.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:38 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:38.500+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:12.877843+0000 front 2024-05-10T11:46:12.877870+0000 (oldest deadline 2024-05-10T11:46:36.378001+0000) 2024-05-10T11:46:39.028 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:38 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:38.572+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:53.194818+0000 front 2024-05-10T11:45:53.194735+0000 (oldest deadline 2024-05-10T11:46:23.195612+0000) 2024-05-10T11:46:39.029 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:38 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:38.572+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:03.196106+0000 front 2024-05-10T11:46:03.196084+0000 (oldest deadline 2024-05-10T11:46:29.096288+0000) 2024-05-10T11:46:39.029 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:38 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:38.572+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:13.797420+0000 front 2024-05-10T11:46:13.797446+0000 (oldest deadline 2024-05-10T11:46:38.497608+0000) 2024-05-10T11:46:39.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:38 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:38.958+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:44.388044+0000 front 2024-05-10T11:45:44.388108+0000 (oldest deadline 2024-05-10T11:46:10.288056+0000) 2024-05-10T11:46:39.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:38 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:38.958+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.190342+0000 front 2024-05-10T11:45:57.190423+0000 (oldest deadline 2024-05-10T11:46:19.490457+0000) 2024-05-10T11:46:39.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:38 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:38.958+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.691618+0000 front 2024-05-10T11:46:01.691702+0000 (oldest deadline 2024-05-10T11:46:26.892345+0000) 2024-05-10T11:46:39.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:38 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:38.958+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:13.794432+0000 front 2024-05-10T11:46:13.794449+0000 (oldest deadline 2024-05-10T11:46:37.294473+0000) 2024-05-10T11:46:39.437 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:39 smithi100 podman[141219]: 2024-05-10 11:46:39.141337512 +0000 UTC m=+5.698439250 container died 3d6c7a1eadacd9df40ba0678bd2e52f6c6a5cfdc258f59fa1c2e84490893a395 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6, org.label-schema.vendor=CentOS, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507) 2024-05-10T11:46:39.528 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:39 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:39.087+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.703618+0000 front 2024-05-10T11:45:45.703296+0000 (oldest deadline 2024-05-10T11:46:11.003467+0000) 2024-05-10T11:46:39.529 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:39 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:39.087+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:58.005499+0000 front 2024-05-10T11:45:58.005443+0000 (oldest deadline 2024-05-10T11:46:22.705582+0000) 2024-05-10T11:46:39.529 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:39 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:39.087+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:05.606931+0000 front 2024-05-10T11:46:05.606957+0000 (oldest deadline 2024-05-10T11:46:30.907138+0000) 2024-05-10T11:46:39.752 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:39 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7[51714]: 2024-05-10T11:46:39.409+0000 7f2b921f1700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:55.454191+0000 front 2024-05-10T11:45:55.454223+0000 (oldest deadline 2024-05-10T11:46:20.754336+0000) 2024-05-10T11:46:39.752 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:39 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7[51714]: 2024-05-10T11:46:39.409+0000 7f2b921f1700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:00.754955+0000 front 2024-05-10T11:46:00.754989+0000 (oldest deadline 2024-05-10T11:46:26.654989+0000) 2024-05-10T11:46:39.753 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:39 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7[51714]: 2024-05-10T11:46:39.409+0000 7f2b921f1700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:13.057027+0000 front 2024-05-10T11:46:13.056878+0000 (oldest deadline 2024-05-10T11:46:38.357046+0000) 2024-05-10T11:46:39.788 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:39 smithi100 podman[141219]: 2024-05-10 11:46:39.620398406 +0000 UTC m=+6.177500150 container cleanup 3d6c7a1eadacd9df40ba0678bd2e52f6c6a5cfdc258f59fa1c2e84490893a395 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6, org.label-schema.vendor=CentOS, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_CLEAN=True, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-05-10T11:46:39.788 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:39 smithi100 bash[141219]: ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6 2024-05-10T11:46:39.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:39 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:39.496+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.171852+0000 front 2024-05-10T11:45:45.171825+0000 (oldest deadline 2024-05-10T11:46:10.372558+0000) 2024-05-10T11:46:39.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:39 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:39.496+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.274899+0000 front 2024-05-10T11:45:57.274943+0000 (oldest deadline 2024-05-10T11:46:21.375013+0000) 2024-05-10T11:46:39.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:39 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:39.496+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.375471+0000 front 2024-05-10T11:46:01.375489+0000 (oldest deadline 2024-05-10T11:46:26.675500+0000) 2024-05-10T11:46:39.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:39 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:39.496+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:12.877843+0000 front 2024-05-10T11:46:12.877870+0000 (oldest deadline 2024-05-10T11:46:36.378001+0000) 2024-05-10T11:46:40.029 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:39 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:39.579+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:53.194818+0000 front 2024-05-10T11:45:53.194735+0000 (oldest deadline 2024-05-10T11:46:23.195612+0000) 2024-05-10T11:46:40.029 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:39 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:39.579+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:03.196106+0000 front 2024-05-10T11:46:03.196084+0000 (oldest deadline 2024-05-10T11:46:29.096288+0000) 2024-05-10T11:46:40.029 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:39 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:39.579+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:13.797420+0000 front 2024-05-10T11:46:13.797446+0000 (oldest deadline 2024-05-10T11:46:38.497608+0000) 2024-05-10T11:46:40.230 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:39 smithi100 podman[141236]: 2024-05-10 11:46:39.880690429 +0000 UTC m=+0.735281468 container remove 3d6c7a1eadacd9df40ba0678bd2e52f6c6a5cfdc258f59fa1c2e84490893a395 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-10T11:46:40.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:39 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:39.923+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:44.388044+0000 front 2024-05-10T11:45:44.388108+0000 (oldest deadline 2024-05-10T11:46:10.288056+0000) 2024-05-10T11:46:40.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:39 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:39.923+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.190342+0000 front 2024-05-10T11:45:57.190423+0000 (oldest deadline 2024-05-10T11:46:19.490457+0000) 2024-05-10T11:46:40.253 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:39 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:39.923+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.691618+0000 front 2024-05-10T11:46:01.691702+0000 (oldest deadline 2024-05-10T11:46:26.892345+0000) 2024-05-10T11:46:40.253 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:39 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:39.923+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:13.794432+0000 front 2024-05-10T11:46:13.794449+0000 (oldest deadline 2024-05-10T11:46:37.294473+0000) 2024-05-10T11:46:40.528 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:40 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:40.068+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.703618+0000 front 2024-05-10T11:45:45.703296+0000 (oldest deadline 2024-05-10T11:46:11.003467+0000) 2024-05-10T11:46:40.529 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:40 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:40.068+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:58.005499+0000 front 2024-05-10T11:45:58.005443+0000 (oldest deadline 2024-05-10T11:46:22.705582+0000) 2024-05-10T11:46:40.529 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:40 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:40.068+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:05.606931+0000 front 2024-05-10T11:46:05.606957+0000 (oldest deadline 2024-05-10T11:46:30.907138+0000) 2024-05-10T11:46:40.537 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:40 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7[51714]: 2024-05-10T11:46:40.396+0000 7f2b921f1700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:55.454191+0000 front 2024-05-10T11:45:55.454223+0000 (oldest deadline 2024-05-10T11:46:20.754336+0000) 2024-05-10T11:46:40.537 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:40 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7[51714]: 2024-05-10T11:46:40.396+0000 7f2b921f1700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:00.754955+0000 front 2024-05-10T11:46:00.754989+0000 (oldest deadline 2024-05-10T11:46:26.654989+0000) 2024-05-10T11:46:40.537 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:40 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7[51714]: 2024-05-10T11:46:40.396+0000 7f2b921f1700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:13.057027+0000 front 2024-05-10T11:46:13.056878+0000 (oldest deadline 2024-05-10T11:46:38.357046+0000) 2024-05-10T11:46:40.538 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:40 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:40.479+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.171852+0000 front 2024-05-10T11:45:45.171825+0000 (oldest deadline 2024-05-10T11:46:10.372558+0000) 2024-05-10T11:46:40.538 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:40 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:40.479+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.274899+0000 front 2024-05-10T11:45:57.274943+0000 (oldest deadline 2024-05-10T11:46:21.375013+0000) 2024-05-10T11:46:40.538 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:40 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:40.479+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.375471+0000 front 2024-05-10T11:46:01.375489+0000 (oldest deadline 2024-05-10T11:46:26.675500+0000) 2024-05-10T11:46:40.538 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:40 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:40.479+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:12.877843+0000 front 2024-05-10T11:46:12.877870+0000 (oldest deadline 2024-05-10T11:46:36.378001+0000) 2024-05-10T11:46:40.538 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:40 smithi100 podman[141325]: 2024-05-10 11:46:40.231421265 +0000 UTC m=+0.106366574 container create 32ad2ee24767c2f7e8b0760e127f4105c0373048df012bb716aa16066ec4cb5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, ceph=True, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-05-10T11:46:40.538 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:40 smithi100 podman[141325]: 2024-05-10 11:46:40.143093653 +0000 UTC m=+0.018038964 image pull ee5e4f21ef58047cb638cccd5cfbe8f6f43412b56c94b1d7fbd9cf3cb90a53a5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f 2024-05-10T11:46:40.538 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:40 smithi100 podman[141325]: 2024-05-10 11:46:40.442124195 +0000 UTC m=+0.317069506 container init 32ad2ee24767c2f7e8b0760e127f4105c0373048df012bb716aa16066ec4cb5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6-deactivate, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240507) 2024-05-10T11:46:40.539 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:40 smithi100 podman[141325]: 2024-05-10 11:46:40.445578056 +0000 UTC m=+0.320523375 container start 32ad2ee24767c2f7e8b0760e127f4105c0373048df012bb716aa16066ec4cb5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-10T11:46:41.028 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:40 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:40.570+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:53.194818+0000 front 2024-05-10T11:45:53.194735+0000 (oldest deadline 2024-05-10T11:46:23.195612+0000) 2024-05-10T11:46:41.029 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:40 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:40.570+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:03.196106+0000 front 2024-05-10T11:46:03.196084+0000 (oldest deadline 2024-05-10T11:46:29.096288+0000) 2024-05-10T11:46:41.029 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:40 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:40.570+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:13.797420+0000 front 2024-05-10T11:46:13.797446+0000 (oldest deadline 2024-05-10T11:46:38.497608+0000) 2024-05-10T11:46:41.038 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:40 smithi100 podman[141325]: 2024-05-10 11:46:40.542147054 +0000 UTC m=+0.417092394 container attach 32ad2ee24767c2f7e8b0760e127f4105c0373048df012bb716aa16066ec4cb5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, io.buildah.version=1.33.5, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True) 2024-05-10T11:46:41.038 INFO:journalctl@ceph.osd.6.smithi100.stdout:May 10 11:46:40 smithi100 podman[141325]: 2024-05-10 11:46:40.738685513 +0000 UTC m=+0.613630830 container died 32ad2ee24767c2f7e8b0760e127f4105c0373048df012bb716aa16066ec4cb5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-6-deactivate, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-05-10T11:46:41.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:40 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:40.943+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:44.388044+0000 front 2024-05-10T11:45:44.388108+0000 (oldest deadline 2024-05-10T11:46:10.288056+0000) 2024-05-10T11:46:41.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:40 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:40.943+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.190342+0000 front 2024-05-10T11:45:57.190423+0000 (oldest deadline 2024-05-10T11:46:19.490457+0000) 2024-05-10T11:46:41.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:40 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:40.943+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.691618+0000 front 2024-05-10T11:46:01.691702+0000 (oldest deadline 2024-05-10T11:46:26.892345+0000) 2024-05-10T11:46:41.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:40 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:40.943+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:13.794432+0000 front 2024-05-10T11:46:13.794449+0000 (oldest deadline 2024-05-10T11:46:37.294473+0000) 2024-05-10T11:46:41.419 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@osd.6.service' 2024-05-10T11:46:41.458 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T11:46:41.458 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-05-10T11:46:41.458 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-05-10T11:46:41.458 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@osd.7 2024-05-10T11:46:41.528 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:41 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:41.064+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.703618+0000 front 2024-05-10T11:45:45.703296+0000 (oldest deadline 2024-05-10T11:46:11.003467+0000) 2024-05-10T11:46:41.528 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:41 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:41.064+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:58.005499+0000 front 2024-05-10T11:45:58.005443+0000 (oldest deadline 2024-05-10T11:46:22.705582+0000) 2024-05-10T11:46:41.528 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:41 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:41.064+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:05.606931+0000 front 2024-05-10T11:46:05.606957+0000 (oldest deadline 2024-05-10T11:46:30.907138+0000) 2024-05-10T11:46:41.752 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:41 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7[51714]: 2024-05-10T11:46:41.404+0000 7f2b921f1700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:55.454191+0000 front 2024-05-10T11:45:55.454223+0000 (oldest deadline 2024-05-10T11:46:20.754336+0000) 2024-05-10T11:46:41.753 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:41 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7[51714]: 2024-05-10T11:46:41.404+0000 7f2b921f1700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:00.754955+0000 front 2024-05-10T11:46:00.754989+0000 (oldest deadline 2024-05-10T11:46:26.654989+0000) 2024-05-10T11:46:41.753 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:41 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7[51714]: 2024-05-10T11:46:41.404+0000 7f2b921f1700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:13.057027+0000 front 2024-05-10T11:46:13.056878+0000 (oldest deadline 2024-05-10T11:46:38.357046+0000) 2024-05-10T11:46:41.753 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:41 smithi178 systemd[1]: Stopping Ceph osd.7 for 20a542cc-0ebd-11ef-bc98-c7b262605968... 2024-05-10T11:46:41.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:41 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:41.507+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.171852+0000 front 2024-05-10T11:45:45.171825+0000 (oldest deadline 2024-05-10T11:46:10.372558+0000) 2024-05-10T11:46:41.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:41 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:41.507+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.274899+0000 front 2024-05-10T11:45:57.274943+0000 (oldest deadline 2024-05-10T11:46:21.375013+0000) 2024-05-10T11:46:41.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:41 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:41.507+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.375471+0000 front 2024-05-10T11:46:01.375489+0000 (oldest deadline 2024-05-10T11:46:26.675500+0000) 2024-05-10T11:46:41.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:41 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:41.507+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:12.877843+0000 front 2024-05-10T11:46:12.877870+0000 (oldest deadline 2024-05-10T11:46:36.378001+0000) 2024-05-10T11:46:42.028 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:41 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:41.555+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:53.194818+0000 front 2024-05-10T11:45:53.194735+0000 (oldest deadline 2024-05-10T11:46:23.195612+0000) 2024-05-10T11:46:42.029 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:41 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:41.555+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:03.196106+0000 front 2024-05-10T11:46:03.196084+0000 (oldest deadline 2024-05-10T11:46:29.096288+0000) 2024-05-10T11:46:42.029 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:41 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:41.555+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:13.797420+0000 front 2024-05-10T11:46:13.797446+0000 (oldest deadline 2024-05-10T11:46:38.497608+0000) 2024-05-10T11:46:42.253 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:41 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7[51714]: 2024-05-10T11:46:41.973+0000 7f2b95bfd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T11:46:42.253 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:41 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7[51714]: 2024-05-10T11:46:41.973+0000 7f2b95bfd700 -1 osd.7 94 *** Got signal Terminated *** 2024-05-10T11:46:42.253 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:41 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7[51714]: 2024-05-10T11:46:41.973+0000 7f2b95bfd700 -1 osd.7 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T11:46:42.253 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:41 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:41.958+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:44.388044+0000 front 2024-05-10T11:45:44.388108+0000 (oldest deadline 2024-05-10T11:46:10.288056+0000) 2024-05-10T11:46:42.253 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:41 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:41.958+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.190342+0000 front 2024-05-10T11:45:57.190423+0000 (oldest deadline 2024-05-10T11:46:19.490457+0000) 2024-05-10T11:46:42.253 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:41 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:41.958+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.691618+0000 front 2024-05-10T11:46:01.691702+0000 (oldest deadline 2024-05-10T11:46:26.892345+0000) 2024-05-10T11:46:42.253 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:41 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:41.958+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:13.794432+0000 front 2024-05-10T11:46:13.794449+0000 (oldest deadline 2024-05-10T11:46:37.294473+0000) 2024-05-10T11:46:42.529 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:42 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:42.059+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.703618+0000 front 2024-05-10T11:45:45.703296+0000 (oldest deadline 2024-05-10T11:46:11.003467+0000) 2024-05-10T11:46:42.529 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:42 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:42.059+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:58.005499+0000 front 2024-05-10T11:45:58.005443+0000 (oldest deadline 2024-05-10T11:46:22.705582+0000) 2024-05-10T11:46:42.529 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:42 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:42.059+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:05.606931+0000 front 2024-05-10T11:46:05.606957+0000 (oldest deadline 2024-05-10T11:46:30.907138+0000) 2024-05-10T11:46:42.752 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:42 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7[51714]: 2024-05-10T11:46:42.428+0000 7f2b921f1700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:55.454191+0000 front 2024-05-10T11:45:55.454223+0000 (oldest deadline 2024-05-10T11:46:20.754336+0000) 2024-05-10T11:46:42.752 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:42 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7[51714]: 2024-05-10T11:46:42.428+0000 7f2b921f1700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:00.754955+0000 front 2024-05-10T11:46:00.754989+0000 (oldest deadline 2024-05-10T11:46:26.654989+0000) 2024-05-10T11:46:42.752 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:42 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7[51714]: 2024-05-10T11:46:42.428+0000 7f2b921f1700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:13.057027+0000 front 2024-05-10T11:46:13.056878+0000 (oldest deadline 2024-05-10T11:46:38.357046+0000) 2024-05-10T11:46:42.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:42 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:42.495+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.171852+0000 front 2024-05-10T11:45:45.171825+0000 (oldest deadline 2024-05-10T11:46:10.372558+0000) 2024-05-10T11:46:42.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:42 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:42.495+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.274899+0000 front 2024-05-10T11:45:57.274943+0000 (oldest deadline 2024-05-10T11:46:21.375013+0000) 2024-05-10T11:46:42.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:42 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:42.495+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.375471+0000 front 2024-05-10T11:46:01.375489+0000 (oldest deadline 2024-05-10T11:46:26.675500+0000) 2024-05-10T11:46:42.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:42 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:42.495+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:12.877843+0000 front 2024-05-10T11:46:12.877870+0000 (oldest deadline 2024-05-10T11:46:36.378001+0000) 2024-05-10T11:46:43.019 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:42 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:42.534+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:53.194818+0000 front 2024-05-10T11:45:53.194735+0000 (oldest deadline 2024-05-10T11:46:23.195612+0000) 2024-05-10T11:46:43.019 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:42 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:42.534+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:03.196106+0000 front 2024-05-10T11:46:03.196084+0000 (oldest deadline 2024-05-10T11:46:29.096288+0000) 2024-05-10T11:46:43.019 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:42 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:42.534+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:13.797420+0000 front 2024-05-10T11:46:13.797446+0000 (oldest deadline 2024-05-10T11:46:38.497608+0000) 2024-05-10T11:46:43.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:42 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:42.979+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:44.388044+0000 front 2024-05-10T11:45:44.388108+0000 (oldest deadline 2024-05-10T11:46:10.288056+0000) 2024-05-10T11:46:43.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:42 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:42.979+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.190342+0000 front 2024-05-10T11:45:57.190423+0000 (oldest deadline 2024-05-10T11:46:19.490457+0000) 2024-05-10T11:46:43.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:42 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:42.979+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.691618+0000 front 2024-05-10T11:46:01.691702+0000 (oldest deadline 2024-05-10T11:46:26.892345+0000) 2024-05-10T11:46:43.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:42 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:42.979+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:13.794432+0000 front 2024-05-10T11:46:13.794449+0000 (oldest deadline 2024-05-10T11:46:37.294473+0000) 2024-05-10T11:46:43.278 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:43 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:43.019+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.703618+0000 front 2024-05-10T11:45:45.703296+0000 (oldest deadline 2024-05-10T11:46:11.003467+0000) 2024-05-10T11:46:43.278 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:43 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:43.019+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:58.005499+0000 front 2024-05-10T11:45:58.005443+0000 (oldest deadline 2024-05-10T11:46:22.705582+0000) 2024-05-10T11:46:43.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:43 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:43.019+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:05.606931+0000 front 2024-05-10T11:46:05.606957+0000 (oldest deadline 2024-05-10T11:46:30.907138+0000) 2024-05-10T11:46:43.752 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:43 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7[51714]: 2024-05-10T11:46:43.379+0000 7f2b921f1700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:55.454191+0000 front 2024-05-10T11:45:55.454223+0000 (oldest deadline 2024-05-10T11:46:20.754336+0000) 2024-05-10T11:46:43.752 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:43 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7[51714]: 2024-05-10T11:46:43.379+0000 7f2b921f1700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:00.754955+0000 front 2024-05-10T11:46:00.754989+0000 (oldest deadline 2024-05-10T11:46:26.654989+0000) 2024-05-10T11:46:43.753 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:43 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7[51714]: 2024-05-10T11:46:43.379+0000 7f2b921f1700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:13.057027+0000 front 2024-05-10T11:46:13.056878+0000 (oldest deadline 2024-05-10T11:46:38.357046+0000) 2024-05-10T11:46:43.778 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:43 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:43.513+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:53.194818+0000 front 2024-05-10T11:45:53.194735+0000 (oldest deadline 2024-05-10T11:46:23.195612+0000) 2024-05-10T11:46:43.778 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:43 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:43.513+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:03.196106+0000 front 2024-05-10T11:46:03.196084+0000 (oldest deadline 2024-05-10T11:46:29.096288+0000) 2024-05-10T11:46:43.779 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:43 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:43.513+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:13.797420+0000 front 2024-05-10T11:46:13.797446+0000 (oldest deadline 2024-05-10T11:46:38.497608+0000) 2024-05-10T11:46:43.779 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:43 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:43.513+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:18.498272+0000 front 2024-05-10T11:46:18.498261+0000 (oldest deadline 2024-05-10T11:46:43.198180+0000) 2024-05-10T11:46:43.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:43 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:43.480+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.171852+0000 front 2024-05-10T11:45:45.171825+0000 (oldest deadline 2024-05-10T11:46:10.372558+0000) 2024-05-10T11:46:43.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:43 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:43.480+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.274899+0000 front 2024-05-10T11:45:57.274943+0000 (oldest deadline 2024-05-10T11:46:21.375013+0000) 2024-05-10T11:46:43.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:43 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:43.480+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.375471+0000 front 2024-05-10T11:46:01.375489+0000 (oldest deadline 2024-05-10T11:46:26.675500+0000) 2024-05-10T11:46:43.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:43 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:43.480+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:12.877843+0000 front 2024-05-10T11:46:12.877870+0000 (oldest deadline 2024-05-10T11:46:36.378001+0000) 2024-05-10T11:46:43.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:43 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:43.480+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:18.679081+0000 front 2024-05-10T11:46:18.679227+0000 (oldest deadline 2024-05-10T11:46:43.379060+0000) 2024-05-10T11:46:44.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:43 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:43.995+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:44.388044+0000 front 2024-05-10T11:45:44.388108+0000 (oldest deadline 2024-05-10T11:46:10.288056+0000) 2024-05-10T11:46:44.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:43 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:43.995+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.190342+0000 front 2024-05-10T11:45:57.190423+0000 (oldest deadline 2024-05-10T11:46:19.490457+0000) 2024-05-10T11:46:44.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:43 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:43.995+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.691618+0000 front 2024-05-10T11:46:01.691702+0000 (oldest deadline 2024-05-10T11:46:26.892345+0000) 2024-05-10T11:46:44.253 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:43 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:43.995+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:13.794432+0000 front 2024-05-10T11:46:13.794449+0000 (oldest deadline 2024-05-10T11:46:37.294473+0000) 2024-05-10T11:46:44.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:44 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:44.040+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.703618+0000 front 2024-05-10T11:45:45.703296+0000 (oldest deadline 2024-05-10T11:46:11.003467+0000) 2024-05-10T11:46:44.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:44 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:44.040+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:58.005499+0000 front 2024-05-10T11:45:58.005443+0000 (oldest deadline 2024-05-10T11:46:22.705582+0000) 2024-05-10T11:46:44.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:44 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:44.040+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:05.606931+0000 front 2024-05-10T11:46:05.606957+0000 (oldest deadline 2024-05-10T11:46:30.907138+0000) 2024-05-10T11:46:44.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:44 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:44.040+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:20.110322+0000 front 2024-05-10T11:46:20.110411+0000 (oldest deadline 2024-05-10T11:46:43.610132+0000) 2024-05-10T11:46:44.752 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:44 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7[51714]: 2024-05-10T11:46:44.366+0000 7f2b921f1700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:55.454191+0000 front 2024-05-10T11:45:55.454223+0000 (oldest deadline 2024-05-10T11:46:20.754336+0000) 2024-05-10T11:46:44.753 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:44 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7[51714]: 2024-05-10T11:46:44.366+0000 7f2b921f1700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:00.754955+0000 front 2024-05-10T11:46:00.754989+0000 (oldest deadline 2024-05-10T11:46:26.654989+0000) 2024-05-10T11:46:44.753 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:44 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7[51714]: 2024-05-10T11:46:44.366+0000 7f2b921f1700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:13.057027+0000 front 2024-05-10T11:46:13.056878+0000 (oldest deadline 2024-05-10T11:46:38.357046+0000) 2024-05-10T11:46:44.753 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:44 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7[51714]: 2024-05-10T11:46:44.366+0000 7f2b921f1700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:21.858533+0000 front 2024-05-10T11:46:21.858272+0000 (oldest deadline 2024-05-10T11:46:43.458920+0000) 2024-05-10T11:46:44.778 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:44 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:44.468+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:53.194818+0000 front 2024-05-10T11:45:53.194735+0000 (oldest deadline 2024-05-10T11:46:23.195612+0000) 2024-05-10T11:46:44.779 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:44 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:44.468+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:03.196106+0000 front 2024-05-10T11:46:03.196084+0000 (oldest deadline 2024-05-10T11:46:29.096288+0000) 2024-05-10T11:46:44.779 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:44 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:44.468+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:13.797420+0000 front 2024-05-10T11:46:13.797446+0000 (oldest deadline 2024-05-10T11:46:38.497608+0000) 2024-05-10T11:46:44.779 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:44 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:44.468+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:18.498272+0000 front 2024-05-10T11:46:18.498261+0000 (oldest deadline 2024-05-10T11:46:43.198180+0000) 2024-05-10T11:46:44.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:44 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:44.467+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.171852+0000 front 2024-05-10T11:45:45.171825+0000 (oldest deadline 2024-05-10T11:46:10.372558+0000) 2024-05-10T11:46:44.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:44 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:44.467+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.274899+0000 front 2024-05-10T11:45:57.274943+0000 (oldest deadline 2024-05-10T11:46:21.375013+0000) 2024-05-10T11:46:44.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:44 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:44.467+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.375471+0000 front 2024-05-10T11:46:01.375489+0000 (oldest deadline 2024-05-10T11:46:26.675500+0000) 2024-05-10T11:46:44.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:44 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:44.467+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:12.877843+0000 front 2024-05-10T11:46:12.877870+0000 (oldest deadline 2024-05-10T11:46:36.378001+0000) 2024-05-10T11:46:44.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:44 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:44.467+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:18.679081+0000 front 2024-05-10T11:46:18.679227+0000 (oldest deadline 2024-05-10T11:46:43.379060+0000) 2024-05-10T11:46:45.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:44 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:44.946+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:44.388044+0000 front 2024-05-10T11:45:44.388108+0000 (oldest deadline 2024-05-10T11:46:10.288056+0000) 2024-05-10T11:46:45.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:44 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:44.946+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.190342+0000 front 2024-05-10T11:45:57.190423+0000 (oldest deadline 2024-05-10T11:46:19.490457+0000) 2024-05-10T11:46:45.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:44 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:44.946+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.691618+0000 front 2024-05-10T11:46:01.691702+0000 (oldest deadline 2024-05-10T11:46:26.892345+0000) 2024-05-10T11:46:45.253 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:44 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:44.946+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:13.794432+0000 front 2024-05-10T11:46:13.794449+0000 (oldest deadline 2024-05-10T11:46:37.294473+0000) 2024-05-10T11:46:45.278 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:45 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:45.026+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.703618+0000 front 2024-05-10T11:45:45.703296+0000 (oldest deadline 2024-05-10T11:46:11.003467+0000) 2024-05-10T11:46:45.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:45 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:45.026+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:58.005499+0000 front 2024-05-10T11:45:58.005443+0000 (oldest deadline 2024-05-10T11:46:22.705582+0000) 2024-05-10T11:46:45.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:45 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:45.026+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:05.606931+0000 front 2024-05-10T11:46:05.606957+0000 (oldest deadline 2024-05-10T11:46:30.907138+0000) 2024-05-10T11:46:45.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:45 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:45.026+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:20.110322+0000 front 2024-05-10T11:46:20.110411+0000 (oldest deadline 2024-05-10T11:46:43.610132+0000) 2024-05-10T11:46:45.752 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:45 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7[51714]: 2024-05-10T11:46:45.319+0000 7f2b921f1700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:55.454191+0000 front 2024-05-10T11:45:55.454223+0000 (oldest deadline 2024-05-10T11:46:20.754336+0000) 2024-05-10T11:46:45.753 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:45 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7[51714]: 2024-05-10T11:46:45.319+0000 7f2b921f1700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:00.754955+0000 front 2024-05-10T11:46:00.754989+0000 (oldest deadline 2024-05-10T11:46:26.654989+0000) 2024-05-10T11:46:45.753 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:45 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7[51714]: 2024-05-10T11:46:45.319+0000 7f2b921f1700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:13.057027+0000 front 2024-05-10T11:46:13.056878+0000 (oldest deadline 2024-05-10T11:46:38.357046+0000) 2024-05-10T11:46:45.753 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:45 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7[51714]: 2024-05-10T11:46:45.319+0000 7f2b921f1700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:21.858533+0000 front 2024-05-10T11:46:21.858272+0000 (oldest deadline 2024-05-10T11:46:43.458920+0000) 2024-05-10T11:46:45.778 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:45 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:45.451+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:53.194818+0000 front 2024-05-10T11:45:53.194735+0000 (oldest deadline 2024-05-10T11:46:23.195612+0000) 2024-05-10T11:46:45.779 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:45 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:45.451+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:03.196106+0000 front 2024-05-10T11:46:03.196084+0000 (oldest deadline 2024-05-10T11:46:29.096288+0000) 2024-05-10T11:46:45.779 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:45 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:45.451+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:13.797420+0000 front 2024-05-10T11:46:13.797446+0000 (oldest deadline 2024-05-10T11:46:38.497608+0000) 2024-05-10T11:46:45.779 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:45 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:45.451+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:18.498272+0000 front 2024-05-10T11:46:18.498261+0000 (oldest deadline 2024-05-10T11:46:43.198180+0000) 2024-05-10T11:46:45.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:45 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:45.499+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.171852+0000 front 2024-05-10T11:45:45.171825+0000 (oldest deadline 2024-05-10T11:46:10.372558+0000) 2024-05-10T11:46:45.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:45 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:45.499+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.274899+0000 front 2024-05-10T11:45:57.274943+0000 (oldest deadline 2024-05-10T11:46:21.375013+0000) 2024-05-10T11:46:45.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:45 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:45.499+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.375471+0000 front 2024-05-10T11:46:01.375489+0000 (oldest deadline 2024-05-10T11:46:26.675500+0000) 2024-05-10T11:46:45.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:45 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:45.499+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:12.877843+0000 front 2024-05-10T11:46:12.877870+0000 (oldest deadline 2024-05-10T11:46:36.378001+0000) 2024-05-10T11:46:45.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:45 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:45.499+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:18.679081+0000 front 2024-05-10T11:46:18.679227+0000 (oldest deadline 2024-05-10T11:46:43.379060+0000) 2024-05-10T11:46:46.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:45 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:45.933+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:44.388044+0000 front 2024-05-10T11:45:44.388108+0000 (oldest deadline 2024-05-10T11:46:10.288056+0000) 2024-05-10T11:46:46.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:45 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:45.933+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.190342+0000 front 2024-05-10T11:45:57.190423+0000 (oldest deadline 2024-05-10T11:46:19.490457+0000) 2024-05-10T11:46:46.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:45 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:45.933+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.691618+0000 front 2024-05-10T11:46:01.691702+0000 (oldest deadline 2024-05-10T11:46:26.892345+0000) 2024-05-10T11:46:46.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:45 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:45.933+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:13.794432+0000 front 2024-05-10T11:46:13.794449+0000 (oldest deadline 2024-05-10T11:46:37.294473+0000) 2024-05-10T11:46:46.278 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:45 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:45.989+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.703618+0000 front 2024-05-10T11:45:45.703296+0000 (oldest deadline 2024-05-10T11:46:11.003467+0000) 2024-05-10T11:46:46.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:45 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:45.989+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:58.005499+0000 front 2024-05-10T11:45:58.005443+0000 (oldest deadline 2024-05-10T11:46:22.705582+0000) 2024-05-10T11:46:46.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:45 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:45.989+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:05.606931+0000 front 2024-05-10T11:46:05.606957+0000 (oldest deadline 2024-05-10T11:46:30.907138+0000) 2024-05-10T11:46:46.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:45 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:45.989+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:20.110322+0000 front 2024-05-10T11:46:20.110411+0000 (oldest deadline 2024-05-10T11:46:43.610132+0000) 2024-05-10T11:46:46.752 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:46 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7[51714]: 2024-05-10T11:46:46.319+0000 7f2b921f1700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:55.454191+0000 front 2024-05-10T11:45:55.454223+0000 (oldest deadline 2024-05-10T11:46:20.754336+0000) 2024-05-10T11:46:46.753 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:46 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7[51714]: 2024-05-10T11:46:46.319+0000 7f2b921f1700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:00.754955+0000 front 2024-05-10T11:46:00.754989+0000 (oldest deadline 2024-05-10T11:46:26.654989+0000) 2024-05-10T11:46:46.753 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:46 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7[51714]: 2024-05-10T11:46:46.319+0000 7f2b921f1700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:13.057027+0000 front 2024-05-10T11:46:13.056878+0000 (oldest deadline 2024-05-10T11:46:38.357046+0000) 2024-05-10T11:46:46.753 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:46 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7[51714]: 2024-05-10T11:46:46.319+0000 7f2b921f1700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:21.858533+0000 front 2024-05-10T11:46:21.858272+0000 (oldest deadline 2024-05-10T11:46:43.458920+0000) 2024-05-10T11:46:46.778 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:46 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:46.501+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:53.194818+0000 front 2024-05-10T11:45:53.194735+0000 (oldest deadline 2024-05-10T11:46:23.195612+0000) 2024-05-10T11:46:46.779 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:46 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:46.501+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:03.196106+0000 front 2024-05-10T11:46:03.196084+0000 (oldest deadline 2024-05-10T11:46:29.096288+0000) 2024-05-10T11:46:46.779 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:46 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:46.501+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:13.797420+0000 front 2024-05-10T11:46:13.797446+0000 (oldest deadline 2024-05-10T11:46:38.497608+0000) 2024-05-10T11:46:46.779 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:46 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:46.501+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:18.498272+0000 front 2024-05-10T11:46:18.498261+0000 (oldest deadline 2024-05-10T11:46:43.198180+0000) 2024-05-10T11:46:46.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:46 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:46.458+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.171852+0000 front 2024-05-10T11:45:45.171825+0000 (oldest deadline 2024-05-10T11:46:10.372558+0000) 2024-05-10T11:46:46.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:46 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:46.458+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.274899+0000 front 2024-05-10T11:45:57.274943+0000 (oldest deadline 2024-05-10T11:46:21.375013+0000) 2024-05-10T11:46:46.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:46 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:46.458+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.375471+0000 front 2024-05-10T11:46:01.375489+0000 (oldest deadline 2024-05-10T11:46:26.675500+0000) 2024-05-10T11:46:46.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:46 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:46.458+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:12.877843+0000 front 2024-05-10T11:46:12.877870+0000 (oldest deadline 2024-05-10T11:46:36.378001+0000) 2024-05-10T11:46:46.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:46 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:46.458+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:18.679081+0000 front 2024-05-10T11:46:18.679227+0000 (oldest deadline 2024-05-10T11:46:43.379060+0000) 2024-05-10T11:46:47.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:46 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:46.888+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:44.388044+0000 front 2024-05-10T11:45:44.388108+0000 (oldest deadline 2024-05-10T11:46:10.288056+0000) 2024-05-10T11:46:47.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:46 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:46.888+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.190342+0000 front 2024-05-10T11:45:57.190423+0000 (oldest deadline 2024-05-10T11:46:19.490457+0000) 2024-05-10T11:46:47.253 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:46 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:46.888+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.691618+0000 front 2024-05-10T11:46:01.691702+0000 (oldest deadline 2024-05-10T11:46:26.892345+0000) 2024-05-10T11:46:47.253 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:46 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:46.888+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:13.794432+0000 front 2024-05-10T11:46:13.794449+0000 (oldest deadline 2024-05-10T11:46:37.294473+0000) 2024-05-10T11:46:47.253 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:46 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:46.888+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:21.995604+0000 front 2024-05-10T11:46:21.995643+0000 (oldest deadline 2024-05-10T11:46:46.695720+0000) 2024-05-10T11:46:47.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:46 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:46.967+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.703618+0000 front 2024-05-10T11:45:45.703296+0000 (oldest deadline 2024-05-10T11:46:11.003467+0000) 2024-05-10T11:46:47.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:46 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:46.967+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:58.005499+0000 front 2024-05-10T11:45:58.005443+0000 (oldest deadline 2024-05-10T11:46:22.705582+0000) 2024-05-10T11:46:47.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:46 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:46.967+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:05.606931+0000 front 2024-05-10T11:46:05.606957+0000 (oldest deadline 2024-05-10T11:46:30.907138+0000) 2024-05-10T11:46:47.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:46 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:46.967+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:20.110322+0000 front 2024-05-10T11:46:20.110411+0000 (oldest deadline 2024-05-10T11:46:43.610132+0000) 2024-05-10T11:46:47.749 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:47 smithi178 podman[77058]: 2024-05-10 11:46:47.450849429 +0000 UTC m=+5.801139806 container died 8450ce57819a517ac4f9e397fe2ce0142e630c0a3757873b046af7fc9f87c880 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e) 2024-05-10T11:46:47.779 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:47 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:47.503+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:53.194818+0000 front 2024-05-10T11:45:53.194735+0000 (oldest deadline 2024-05-10T11:46:23.195612+0000) 2024-05-10T11:46:47.779 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:47 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:47.503+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:03.196106+0000 front 2024-05-10T11:46:03.196084+0000 (oldest deadline 2024-05-10T11:46:29.096288+0000) 2024-05-10T11:46:47.779 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:47 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:47.503+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:13.797420+0000 front 2024-05-10T11:46:13.797446+0000 (oldest deadline 2024-05-10T11:46:38.497608+0000) 2024-05-10T11:46:47.779 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:47 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:47.503+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:18.498272+0000 front 2024-05-10T11:46:18.498261+0000 (oldest deadline 2024-05-10T11:46:43.198180+0000) 2024-05-10T11:46:47.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:47 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:47.507+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.171852+0000 front 2024-05-10T11:45:45.171825+0000 (oldest deadline 2024-05-10T11:46:10.372558+0000) 2024-05-10T11:46:47.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:47 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:47.507+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.274899+0000 front 2024-05-10T11:45:57.274943+0000 (oldest deadline 2024-05-10T11:46:21.375013+0000) 2024-05-10T11:46:47.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:47 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:47.507+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.375471+0000 front 2024-05-10T11:46:01.375489+0000 (oldest deadline 2024-05-10T11:46:26.675500+0000) 2024-05-10T11:46:47.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:47 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:47.507+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:12.877843+0000 front 2024-05-10T11:46:12.877870+0000 (oldest deadline 2024-05-10T11:46:36.378001+0000) 2024-05-10T11:46:47.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:47 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:47.507+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:18.679081+0000 front 2024-05-10T11:46:18.679227+0000 (oldest deadline 2024-05-10T11:46:43.379060+0000) 2024-05-10T11:46:48.002 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:47 smithi178 podman[77058]: 2024-05-10 11:46:47.84824103 +0000 UTC m=+6.198531404 container cleanup 8450ce57819a517ac4f9e397fe2ce0142e630c0a3757873b046af7fc9f87c880 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.build-date=20240507, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08) 2024-05-10T11:46:48.003 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:47 smithi178 bash[77058]: ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7 2024-05-10T11:46:48.003 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:47 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:47.918+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:44.388044+0000 front 2024-05-10T11:45:44.388108+0000 (oldest deadline 2024-05-10T11:46:10.288056+0000) 2024-05-10T11:46:48.003 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:47 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:47.918+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.190342+0000 front 2024-05-10T11:45:57.190423+0000 (oldest deadline 2024-05-10T11:46:19.490457+0000) 2024-05-10T11:46:48.003 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:47 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:47.918+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.691618+0000 front 2024-05-10T11:46:01.691702+0000 (oldest deadline 2024-05-10T11:46:26.892345+0000) 2024-05-10T11:46:48.003 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:47 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:47.918+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:13.794432+0000 front 2024-05-10T11:46:13.794449+0000 (oldest deadline 2024-05-10T11:46:37.294473+0000) 2024-05-10T11:46:48.003 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:47 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:47.918+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:21.995604+0000 front 2024-05-10T11:46:21.995643+0000 (oldest deadline 2024-05-10T11:46:46.695720+0000) 2024-05-10T11:46:48.278 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:47 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:47.962+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.703618+0000 front 2024-05-10T11:45:45.703296+0000 (oldest deadline 2024-05-10T11:46:11.003467+0000) 2024-05-10T11:46:48.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:47 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:47.962+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:58.005499+0000 front 2024-05-10T11:45:58.005443+0000 (oldest deadline 2024-05-10T11:46:22.705582+0000) 2024-05-10T11:46:48.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:47 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:47.962+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:05.606931+0000 front 2024-05-10T11:46:05.606957+0000 (oldest deadline 2024-05-10T11:46:30.907138+0000) 2024-05-10T11:46:48.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:47 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:47.962+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:20.110322+0000 front 2024-05-10T11:46:20.110411+0000 (oldest deadline 2024-05-10T11:46:43.610132+0000) 2024-05-10T11:46:48.463 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:48 smithi178 podman[77075]: 2024-05-10 11:46:48.108149456 +0000 UTC m=+0.647444554 container remove 8450ce57819a517ac4f9e397fe2ce0142e630c0a3757873b046af7fc9f87c880 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, ceph=True, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-05-10T11:46:48.744 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:48 smithi178 podman[77164]: 2024-05-10 11:46:48.36719022 +0000 UTC m=+0.017896982 image pull ee5e4f21ef58047cb638cccd5cfbe8f6f43412b56c94b1d7fbd9cf3cb90a53a5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f 2024-05-10T11:46:48.744 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:48 smithi178 podman[77164]: 2024-05-10 11:46:48.493313595 +0000 UTC m=+0.144020382 container create 10d7f0258ccf2b75a04dfaa04728917bf5ecdf0b94267e5528fbc61e657aabe4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e) 2024-05-10T11:46:48.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:48 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:48.522+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.171852+0000 front 2024-05-10T11:45:45.171825+0000 (oldest deadline 2024-05-10T11:46:10.372558+0000) 2024-05-10T11:46:48.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:48 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:48.522+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.274899+0000 front 2024-05-10T11:45:57.274943+0000 (oldest deadline 2024-05-10T11:46:21.375013+0000) 2024-05-10T11:46:48.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:48 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:48.522+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.375471+0000 front 2024-05-10T11:46:01.375489+0000 (oldest deadline 2024-05-10T11:46:26.675500+0000) 2024-05-10T11:46:48.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:48 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:48.522+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:12.877843+0000 front 2024-05-10T11:46:12.877870+0000 (oldest deadline 2024-05-10T11:46:36.378001+0000) 2024-05-10T11:46:48.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:48 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:48.522+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:18.679081+0000 front 2024-05-10T11:46:18.679227+0000 (oldest deadline 2024-05-10T11:46:43.379060+0000) 2024-05-10T11:46:48.986 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:48 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:48.538+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:53.194818+0000 front 2024-05-10T11:45:53.194735+0000 (oldest deadline 2024-05-10T11:46:23.195612+0000) 2024-05-10T11:46:48.987 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:48 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:48.538+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:03.196106+0000 front 2024-05-10T11:46:03.196084+0000 (oldest deadline 2024-05-10T11:46:29.096288+0000) 2024-05-10T11:46:48.987 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:48 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:48.538+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:13.797420+0000 front 2024-05-10T11:46:13.797446+0000 (oldest deadline 2024-05-10T11:46:38.497608+0000) 2024-05-10T11:46:48.987 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:48 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:48.538+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:18.498272+0000 front 2024-05-10T11:46:18.498261+0000 (oldest deadline 2024-05-10T11:46:43.198180+0000) 2024-05-10T11:46:49.003 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:48 smithi178 podman[77164]: 2024-05-10 11:46:48.744605326 +0000 UTC m=+0.395312070 container init 10d7f0258ccf2b75a04dfaa04728917bf5ecdf0b94267e5528fbc61e657aabe4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7-deactivate, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-10T11:46:49.003 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:48 smithi178 podman[77164]: 2024-05-10 11:46:48.748735616 +0000 UTC m=+0.399442361 container start 10d7f0258ccf2b75a04dfaa04728917bf5ecdf0b94267e5528fbc61e657aabe4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, CEPH_POINT_RELEASE=) 2024-05-10T11:46:49.003 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:48 smithi178 podman[77164]: 2024-05-10 11:46:48.84464727 +0000 UTC m=+0.495354015 container attach 10d7f0258ccf2b75a04dfaa04728917bf5ecdf0b94267e5528fbc61e657aabe4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=) 2024-05-10T11:46:49.003 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:48 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:48.906+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:44.388044+0000 front 2024-05-10T11:45:44.388108+0000 (oldest deadline 2024-05-10T11:46:10.288056+0000) 2024-05-10T11:46:49.003 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:48 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:48.906+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.190342+0000 front 2024-05-10T11:45:57.190423+0000 (oldest deadline 2024-05-10T11:46:19.490457+0000) 2024-05-10T11:46:49.003 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:48 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:48.906+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.691618+0000 front 2024-05-10T11:46:01.691702+0000 (oldest deadline 2024-05-10T11:46:26.892345+0000) 2024-05-10T11:46:49.003 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:48 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:48.906+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:13.794432+0000 front 2024-05-10T11:46:13.794449+0000 (oldest deadline 2024-05-10T11:46:37.294473+0000) 2024-05-10T11:46:49.004 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:48 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:48.906+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:21.995604+0000 front 2024-05-10T11:46:21.995643+0000 (oldest deadline 2024-05-10T11:46:46.695720+0000) 2024-05-10T11:46:49.278 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:48 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:48.986+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.703618+0000 front 2024-05-10T11:45:45.703296+0000 (oldest deadline 2024-05-10T11:46:11.003467+0000) 2024-05-10T11:46:49.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:48 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:48.986+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:58.005499+0000 front 2024-05-10T11:45:58.005443+0000 (oldest deadline 2024-05-10T11:46:22.705582+0000) 2024-05-10T11:46:49.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:48 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:48.986+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:05.606931+0000 front 2024-05-10T11:46:05.606957+0000 (oldest deadline 2024-05-10T11:46:30.907138+0000) 2024-05-10T11:46:49.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:48 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:48.986+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:20.110322+0000 front 2024-05-10T11:46:20.110411+0000 (oldest deadline 2024-05-10T11:46:43.610132+0000) 2024-05-10T11:46:49.338 INFO:journalctl@ceph.osd.7.smithi178.stdout:May 10 11:46:49 smithi178 podman[77164]: 2024-05-10 11:46:49.058477969 +0000 UTC m=+0.709184715 container died 10d7f0258ccf2b75a04dfaa04728917bf5ecdf0b94267e5528fbc61e657aabe4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True) 2024-05-10T11:46:49.667 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@osd.7.service' 2024-05-10T11:46:49.706 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T11:46:49.707 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-05-10T11:46:49.707 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-05-10T11:46:49.707 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@osd.8 2024-05-10T11:46:49.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:49 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:49.508+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.171852+0000 front 2024-05-10T11:45:45.171825+0000 (oldest deadline 2024-05-10T11:46:10.372558+0000) 2024-05-10T11:46:49.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:49 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:49.508+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.274899+0000 front 2024-05-10T11:45:57.274943+0000 (oldest deadline 2024-05-10T11:46:21.375013+0000) 2024-05-10T11:46:49.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:49 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:49.508+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.375471+0000 front 2024-05-10T11:46:01.375489+0000 (oldest deadline 2024-05-10T11:46:26.675500+0000) 2024-05-10T11:46:49.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:49 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:49.508+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:12.877843+0000 front 2024-05-10T11:46:12.877870+0000 (oldest deadline 2024-05-10T11:46:36.378001+0000) 2024-05-10T11:46:49.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:49 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:49.508+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:18.679081+0000 front 2024-05-10T11:46:18.679227+0000 (oldest deadline 2024-05-10T11:46:43.379060+0000) 2024-05-10T11:46:50.015 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:49 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:49.567+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:53.194818+0000 front 2024-05-10T11:45:53.194735+0000 (oldest deadline 2024-05-10T11:46:23.195612+0000) 2024-05-10T11:46:50.016 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:49 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:49.567+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:03.196106+0000 front 2024-05-10T11:46:03.196084+0000 (oldest deadline 2024-05-10T11:46:29.096288+0000) 2024-05-10T11:46:50.016 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:49 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:49.567+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:13.797420+0000 front 2024-05-10T11:46:13.797446+0000 (oldest deadline 2024-05-10T11:46:38.497608+0000) 2024-05-10T11:46:50.016 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:49 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:49.567+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:18.498272+0000 front 2024-05-10T11:46:18.498261+0000 (oldest deadline 2024-05-10T11:46:43.198180+0000) 2024-05-10T11:46:50.016 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:49 smithi188 systemd[1]: Stopping Ceph osd.8 for 20a542cc-0ebd-11ef-bc98-c7b262605968... 2024-05-10T11:46:50.253 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:49 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:49.939+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:44.388044+0000 front 2024-05-10T11:45:44.388108+0000 (oldest deadline 2024-05-10T11:46:10.288056+0000) 2024-05-10T11:46:50.253 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:49 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:49.939+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.190342+0000 front 2024-05-10T11:45:57.190423+0000 (oldest deadline 2024-05-10T11:46:19.490457+0000) 2024-05-10T11:46:50.253 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:49 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:49.939+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.691618+0000 front 2024-05-10T11:46:01.691702+0000 (oldest deadline 2024-05-10T11:46:26.892345+0000) 2024-05-10T11:46:50.253 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:49 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:49.939+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:13.794432+0000 front 2024-05-10T11:46:13.794449+0000 (oldest deadline 2024-05-10T11:46:37.294473+0000) 2024-05-10T11:46:50.253 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:49 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:49.939+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:21.995604+0000 front 2024-05-10T11:46:21.995643+0000 (oldest deadline 2024-05-10T11:46:46.695720+0000) 2024-05-10T11:46:50.279 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:50 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:50.040+0000 7f01afe22700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T11:46:50.279 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:50 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:50.040+0000 7f01afe22700 -1 osd.8 94 *** Got signal Terminated *** 2024-05-10T11:46:50.279 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:50 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:50.040+0000 7f01afe22700 -1 osd.8 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T11:46:50.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:50 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:50.015+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.703618+0000 front 2024-05-10T11:45:45.703296+0000 (oldest deadline 2024-05-10T11:46:11.003467+0000) 2024-05-10T11:46:50.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:50 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:50.015+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:58.005499+0000 front 2024-05-10T11:45:58.005443+0000 (oldest deadline 2024-05-10T11:46:22.705582+0000) 2024-05-10T11:46:50.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:50 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:50.015+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:05.606931+0000 front 2024-05-10T11:46:05.606957+0000 (oldest deadline 2024-05-10T11:46:30.907138+0000) 2024-05-10T11:46:50.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:50 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:50.015+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:20.110322+0000 front 2024-05-10T11:46:20.110411+0000 (oldest deadline 2024-05-10T11:46:43.610132+0000) 2024-05-10T11:46:50.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:50 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:50.519+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.171852+0000 front 2024-05-10T11:45:45.171825+0000 (oldest deadline 2024-05-10T11:46:10.372558+0000) 2024-05-10T11:46:50.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:50 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:50.519+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.274899+0000 front 2024-05-10T11:45:57.274943+0000 (oldest deadline 2024-05-10T11:46:21.375013+0000) 2024-05-10T11:46:50.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:50 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:50.519+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.375471+0000 front 2024-05-10T11:46:01.375489+0000 (oldest deadline 2024-05-10T11:46:26.675500+0000) 2024-05-10T11:46:50.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:50 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:50.519+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:12.877843+0000 front 2024-05-10T11:46:12.877870+0000 (oldest deadline 2024-05-10T11:46:36.378001+0000) 2024-05-10T11:46:50.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:50 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:50.519+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:18.679081+0000 front 2024-05-10T11:46:18.679227+0000 (oldest deadline 2024-05-10T11:46:43.379060+0000) 2024-05-10T11:46:50.991 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:50 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:50.544+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:53.194818+0000 front 2024-05-10T11:45:53.194735+0000 (oldest deadline 2024-05-10T11:46:23.195612+0000) 2024-05-10T11:46:50.991 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:50 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:50.544+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:03.196106+0000 front 2024-05-10T11:46:03.196084+0000 (oldest deadline 2024-05-10T11:46:29.096288+0000) 2024-05-10T11:46:50.991 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:50 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:50.544+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:13.797420+0000 front 2024-05-10T11:46:13.797446+0000 (oldest deadline 2024-05-10T11:46:38.497608+0000) 2024-05-10T11:46:50.992 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:50 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:50.544+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:18.498272+0000 front 2024-05-10T11:46:18.498261+0000 (oldest deadline 2024-05-10T11:46:43.198180+0000) 2024-05-10T11:46:51.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:50 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:50.905+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:44.388044+0000 front 2024-05-10T11:45:44.388108+0000 (oldest deadline 2024-05-10T11:46:10.288056+0000) 2024-05-10T11:46:51.253 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:50 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:50.905+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.190342+0000 front 2024-05-10T11:45:57.190423+0000 (oldest deadline 2024-05-10T11:46:19.490457+0000) 2024-05-10T11:46:51.253 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:50 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:50.905+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.691618+0000 front 2024-05-10T11:46:01.691702+0000 (oldest deadline 2024-05-10T11:46:26.892345+0000) 2024-05-10T11:46:51.253 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:50 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:50.905+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:13.794432+0000 front 2024-05-10T11:46:13.794449+0000 (oldest deadline 2024-05-10T11:46:37.294473+0000) 2024-05-10T11:46:51.253 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:50 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:50.905+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:21.995604+0000 front 2024-05-10T11:46:21.995643+0000 (oldest deadline 2024-05-10T11:46:46.695720+0000) 2024-05-10T11:46:51.278 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:50 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:50.991+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.703618+0000 front 2024-05-10T11:45:45.703296+0000 (oldest deadline 2024-05-10T11:46:11.003467+0000) 2024-05-10T11:46:51.278 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:50 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:50.991+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:58.005499+0000 front 2024-05-10T11:45:58.005443+0000 (oldest deadline 2024-05-10T11:46:22.705582+0000) 2024-05-10T11:46:51.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:50 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:50.991+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:05.606931+0000 front 2024-05-10T11:46:05.606957+0000 (oldest deadline 2024-05-10T11:46:30.907138+0000) 2024-05-10T11:46:51.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:50 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:50.991+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:20.110322+0000 front 2024-05-10T11:46:20.110411+0000 (oldest deadline 2024-05-10T11:46:43.610132+0000) 2024-05-10T11:46:51.984 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:51 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:51.551+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:53.194818+0000 front 2024-05-10T11:45:53.194735+0000 (oldest deadline 2024-05-10T11:46:23.195612+0000) 2024-05-10T11:46:51.985 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:51 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:51.551+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:03.196106+0000 front 2024-05-10T11:46:03.196084+0000 (oldest deadline 2024-05-10T11:46:29.096288+0000) 2024-05-10T11:46:51.985 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:51 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:51.551+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:13.797420+0000 front 2024-05-10T11:46:13.797446+0000 (oldest deadline 2024-05-10T11:46:38.497608+0000) 2024-05-10T11:46:51.985 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:51 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:51.551+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:18.498272+0000 front 2024-05-10T11:46:18.498261+0000 (oldest deadline 2024-05-10T11:46:43.198180+0000) 2024-05-10T11:46:52.038 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:51 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:51.567+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.171852+0000 front 2024-05-10T11:45:45.171825+0000 (oldest deadline 2024-05-10T11:46:10.372558+0000) 2024-05-10T11:46:52.038 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:51 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:51.567+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.274899+0000 front 2024-05-10T11:45:57.274943+0000 (oldest deadline 2024-05-10T11:46:21.375013+0000) 2024-05-10T11:46:52.038 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:51 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:51.567+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.375471+0000 front 2024-05-10T11:46:01.375489+0000 (oldest deadline 2024-05-10T11:46:26.675500+0000) 2024-05-10T11:46:52.038 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:51 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:51.567+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:12.877843+0000 front 2024-05-10T11:46:12.877870+0000 (oldest deadline 2024-05-10T11:46:36.378001+0000) 2024-05-10T11:46:52.038 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:51 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:51.567+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:18.679081+0000 front 2024-05-10T11:46:18.679227+0000 (oldest deadline 2024-05-10T11:46:43.379060+0000) 2024-05-10T11:46:52.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:51 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:51.913+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:44.388044+0000 front 2024-05-10T11:45:44.388108+0000 (oldest deadline 2024-05-10T11:46:10.288056+0000) 2024-05-10T11:46:52.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:51 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:51.913+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.190342+0000 front 2024-05-10T11:45:57.190423+0000 (oldest deadline 2024-05-10T11:46:19.490457+0000) 2024-05-10T11:46:52.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:51 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:51.913+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.691618+0000 front 2024-05-10T11:46:01.691702+0000 (oldest deadline 2024-05-10T11:46:26.892345+0000) 2024-05-10T11:46:52.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:51 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:51.913+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:13.794432+0000 front 2024-05-10T11:46:13.794449+0000 (oldest deadline 2024-05-10T11:46:37.294473+0000) 2024-05-10T11:46:52.253 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:51 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:51.913+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:21.995604+0000 front 2024-05-10T11:46:21.995643+0000 (oldest deadline 2024-05-10T11:46:46.695720+0000) 2024-05-10T11:46:52.278 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:51 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:51.984+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.703618+0000 front 2024-05-10T11:45:45.703296+0000 (oldest deadline 2024-05-10T11:46:11.003467+0000) 2024-05-10T11:46:52.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:51 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:51.984+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:58.005499+0000 front 2024-05-10T11:45:58.005443+0000 (oldest deadline 2024-05-10T11:46:22.705582+0000) 2024-05-10T11:46:52.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:51 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:51.984+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:05.606931+0000 front 2024-05-10T11:46:05.606957+0000 (oldest deadline 2024-05-10T11:46:30.907138+0000) 2024-05-10T11:46:52.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:51 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:51.984+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:20.110322+0000 front 2024-05-10T11:46:20.110411+0000 (oldest deadline 2024-05-10T11:46:43.610132+0000) 2024-05-10T11:46:53.019 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:52 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:52.584+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:53.194818+0000 front 2024-05-10T11:45:53.194735+0000 (oldest deadline 2024-05-10T11:46:23.195612+0000) 2024-05-10T11:46:53.019 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:52 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:52.584+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:03.196106+0000 front 2024-05-10T11:46:03.196084+0000 (oldest deadline 2024-05-10T11:46:29.096288+0000) 2024-05-10T11:46:53.019 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:52 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:52.584+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:13.797420+0000 front 2024-05-10T11:46:13.797446+0000 (oldest deadline 2024-05-10T11:46:38.497608+0000) 2024-05-10T11:46:53.020 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:52 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:52.584+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:18.498272+0000 front 2024-05-10T11:46:18.498261+0000 (oldest deadline 2024-05-10T11:46:43.198180+0000) 2024-05-10T11:46:53.038 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:52 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:52.545+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.171852+0000 front 2024-05-10T11:45:45.171825+0000 (oldest deadline 2024-05-10T11:46:10.372558+0000) 2024-05-10T11:46:53.038 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:52 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:52.545+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.274899+0000 front 2024-05-10T11:45:57.274943+0000 (oldest deadline 2024-05-10T11:46:21.375013+0000) 2024-05-10T11:46:53.038 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:52 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:52.545+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.375471+0000 front 2024-05-10T11:46:01.375489+0000 (oldest deadline 2024-05-10T11:46:26.675500+0000) 2024-05-10T11:46:53.038 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:52 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:52.545+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:12.877843+0000 front 2024-05-10T11:46:12.877870+0000 (oldest deadline 2024-05-10T11:46:36.378001+0000) 2024-05-10T11:46:53.038 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:52 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:52.545+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:18.679081+0000 front 2024-05-10T11:46:18.679227+0000 (oldest deadline 2024-05-10T11:46:43.379060+0000) 2024-05-10T11:46:53.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:52 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:52.950+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:44.388044+0000 front 2024-05-10T11:45:44.388108+0000 (oldest deadline 2024-05-10T11:46:10.288056+0000) 2024-05-10T11:46:53.253 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:52 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:52.950+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.190342+0000 front 2024-05-10T11:45:57.190423+0000 (oldest deadline 2024-05-10T11:46:19.490457+0000) 2024-05-10T11:46:53.253 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:52 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:52.950+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.691618+0000 front 2024-05-10T11:46:01.691702+0000 (oldest deadline 2024-05-10T11:46:26.892345+0000) 2024-05-10T11:46:53.253 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:52 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:52.950+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:13.794432+0000 front 2024-05-10T11:46:13.794449+0000 (oldest deadline 2024-05-10T11:46:37.294473+0000) 2024-05-10T11:46:53.253 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:52 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:52.950+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:21.995604+0000 front 2024-05-10T11:46:21.995643+0000 (oldest deadline 2024-05-10T11:46:46.695720+0000) 2024-05-10T11:46:53.253 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:52 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:52.950+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-10T11:46:26.696333+0000 front 2024-05-10T11:46:26.696307+0000 (oldest deadline 2024-05-10T11:46:51.996382+0000) 2024-05-10T11:46:53.278 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:53 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:53.019+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.703618+0000 front 2024-05-10T11:45:45.703296+0000 (oldest deadline 2024-05-10T11:46:11.003467+0000) 2024-05-10T11:46:53.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:53 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:53.019+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:58.005499+0000 front 2024-05-10T11:45:58.005443+0000 (oldest deadline 2024-05-10T11:46:22.705582+0000) 2024-05-10T11:46:53.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:53 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:53.019+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:05.606931+0000 front 2024-05-10T11:46:05.606957+0000 (oldest deadline 2024-05-10T11:46:30.907138+0000) 2024-05-10T11:46:53.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:53 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:53.019+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:20.110322+0000 front 2024-05-10T11:46:20.110411+0000 (oldest deadline 2024-05-10T11:46:43.610132+0000) 2024-05-10T11:46:53.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:53 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:53.019+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-10T11:46:28.311217+0000 front 2024-05-10T11:46:28.311184+0000 (oldest deadline 2024-05-10T11:46:52.411332+0000) 2024-05-10T11:46:53.794 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:53 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:53.521+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.171852+0000 front 2024-05-10T11:45:45.171825+0000 (oldest deadline 2024-05-10T11:46:10.372558+0000) 2024-05-10T11:46:53.794 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:53 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:53.521+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.274899+0000 front 2024-05-10T11:45:57.274943+0000 (oldest deadline 2024-05-10T11:46:21.375013+0000) 2024-05-10T11:46:53.794 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:53 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:53.521+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.375471+0000 front 2024-05-10T11:46:01.375489+0000 (oldest deadline 2024-05-10T11:46:26.675500+0000) 2024-05-10T11:46:53.794 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:53 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:53.521+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:12.877843+0000 front 2024-05-10T11:46:12.877870+0000 (oldest deadline 2024-05-10T11:46:36.378001+0000) 2024-05-10T11:46:53.794 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:53 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:53.521+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:18.679081+0000 front 2024-05-10T11:46:18.679227+0000 (oldest deadline 2024-05-10T11:46:43.379060+0000) 2024-05-10T11:46:53.794 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:53 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:53.521+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-10T11:46:27.980824+0000 front 2024-05-10T11:46:27.980941+0000 (oldest deadline 2024-05-10T11:46:53.280786+0000) 2024-05-10T11:46:53.979 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:53 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:53.543+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:53.194818+0000 front 2024-05-10T11:45:53.194735+0000 (oldest deadline 2024-05-10T11:46:23.195612+0000) 2024-05-10T11:46:53.979 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:53 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:53.543+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:03.196106+0000 front 2024-05-10T11:46:03.196084+0000 (oldest deadline 2024-05-10T11:46:29.096288+0000) 2024-05-10T11:46:53.979 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:53 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:53.543+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:13.797420+0000 front 2024-05-10T11:46:13.797446+0000 (oldest deadline 2024-05-10T11:46:38.497608+0000) 2024-05-10T11:46:53.979 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:53 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:53.543+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:18.498272+0000 front 2024-05-10T11:46:18.498261+0000 (oldest deadline 2024-05-10T11:46:43.198180+0000) 2024-05-10T11:46:54.253 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:53 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:53.958+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:44.388044+0000 front 2024-05-10T11:45:44.388108+0000 (oldest deadline 2024-05-10T11:46:10.288056+0000) 2024-05-10T11:46:54.253 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:53 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:53.958+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.190342+0000 front 2024-05-10T11:45:57.190423+0000 (oldest deadline 2024-05-10T11:46:19.490457+0000) 2024-05-10T11:46:54.253 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:53 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:53.958+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.691618+0000 front 2024-05-10T11:46:01.691702+0000 (oldest deadline 2024-05-10T11:46:26.892345+0000) 2024-05-10T11:46:54.253 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:53 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:53.958+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:13.794432+0000 front 2024-05-10T11:46:13.794449+0000 (oldest deadline 2024-05-10T11:46:37.294473+0000) 2024-05-10T11:46:54.253 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:53 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:53.958+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:21.995604+0000 front 2024-05-10T11:46:21.995643+0000 (oldest deadline 2024-05-10T11:46:46.695720+0000) 2024-05-10T11:46:54.253 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:53 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:53.958+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-10T11:46:26.696333+0000 front 2024-05-10T11:46:26.696307+0000 (oldest deadline 2024-05-10T11:46:51.996382+0000) 2024-05-10T11:46:54.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:53 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:53.977+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.703618+0000 front 2024-05-10T11:45:45.703296+0000 (oldest deadline 2024-05-10T11:46:11.003467+0000) 2024-05-10T11:46:54.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:53 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:53.977+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:58.005499+0000 front 2024-05-10T11:45:58.005443+0000 (oldest deadline 2024-05-10T11:46:22.705582+0000) 2024-05-10T11:46:54.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:53 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:53.977+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:05.606931+0000 front 2024-05-10T11:46:05.606957+0000 (oldest deadline 2024-05-10T11:46:30.907138+0000) 2024-05-10T11:46:54.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:53 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:53.977+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:20.110322+0000 front 2024-05-10T11:46:20.110411+0000 (oldest deadline 2024-05-10T11:46:43.610132+0000) 2024-05-10T11:46:54.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:53 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:53.977+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-10T11:46:28.311217+0000 front 2024-05-10T11:46:28.311184+0000 (oldest deadline 2024-05-10T11:46:52.411332+0000) 2024-05-10T11:46:54.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:54 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:54.497+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.171852+0000 front 2024-05-10T11:45:45.171825+0000 (oldest deadline 2024-05-10T11:46:10.372558+0000) 2024-05-10T11:46:54.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:54 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:54.497+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.274899+0000 front 2024-05-10T11:45:57.274943+0000 (oldest deadline 2024-05-10T11:46:21.375013+0000) 2024-05-10T11:46:54.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:54 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:54.497+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.375471+0000 front 2024-05-10T11:46:01.375489+0000 (oldest deadline 2024-05-10T11:46:26.675500+0000) 2024-05-10T11:46:54.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:54 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:54.497+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:12.877843+0000 front 2024-05-10T11:46:12.877870+0000 (oldest deadline 2024-05-10T11:46:36.378001+0000) 2024-05-10T11:46:54.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:54 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:54.497+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:18.679081+0000 front 2024-05-10T11:46:18.679227+0000 (oldest deadline 2024-05-10T11:46:43.379060+0000) 2024-05-10T11:46:54.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:54 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:54.497+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-10T11:46:27.980824+0000 front 2024-05-10T11:46:27.980941+0000 (oldest deadline 2024-05-10T11:46:53.280786+0000) 2024-05-10T11:46:54.954 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:54 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:54.577+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:53.194818+0000 front 2024-05-10T11:45:53.194735+0000 (oldest deadline 2024-05-10T11:46:23.195612+0000) 2024-05-10T11:46:54.954 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:54 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:54.577+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:03.196106+0000 front 2024-05-10T11:46:03.196084+0000 (oldest deadline 2024-05-10T11:46:29.096288+0000) 2024-05-10T11:46:54.954 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:54 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:54.577+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:13.797420+0000 front 2024-05-10T11:46:13.797446+0000 (oldest deadline 2024-05-10T11:46:38.497608+0000) 2024-05-10T11:46:54.954 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:54 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8[52244]: 2024-05-10T11:46:54.577+0000 7f01a840e700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:18.498272+0000 front 2024-05-10T11:46:18.498261+0000 (oldest deadline 2024-05-10T11:46:43.198180+0000) 2024-05-10T11:46:55.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:54 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:54.913+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:44.388044+0000 front 2024-05-10T11:45:44.388108+0000 (oldest deadline 2024-05-10T11:46:10.288056+0000) 2024-05-10T11:46:55.253 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:54 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:54.913+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.190342+0000 front 2024-05-10T11:45:57.190423+0000 (oldest deadline 2024-05-10T11:46:19.490457+0000) 2024-05-10T11:46:55.253 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:54 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:54.913+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.691618+0000 front 2024-05-10T11:46:01.691702+0000 (oldest deadline 2024-05-10T11:46:26.892345+0000) 2024-05-10T11:46:55.253 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:54 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:54.913+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:13.794432+0000 front 2024-05-10T11:46:13.794449+0000 (oldest deadline 2024-05-10T11:46:37.294473+0000) 2024-05-10T11:46:55.253 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:54 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:54.913+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:21.995604+0000 front 2024-05-10T11:46:21.995643+0000 (oldest deadline 2024-05-10T11:46:46.695720+0000) 2024-05-10T11:46:55.253 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:54 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:54.913+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-10T11:46:26.696333+0000 front 2024-05-10T11:46:26.696307+0000 (oldest deadline 2024-05-10T11:46:51.996382+0000) 2024-05-10T11:46:55.278 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:54 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:54.954+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.703618+0000 front 2024-05-10T11:45:45.703296+0000 (oldest deadline 2024-05-10T11:46:11.003467+0000) 2024-05-10T11:46:55.278 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:54 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:54.954+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:58.005499+0000 front 2024-05-10T11:45:58.005443+0000 (oldest deadline 2024-05-10T11:46:22.705582+0000) 2024-05-10T11:46:55.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:54 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:54.954+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:05.606931+0000 front 2024-05-10T11:46:05.606957+0000 (oldest deadline 2024-05-10T11:46:30.907138+0000) 2024-05-10T11:46:55.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:54 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:54.954+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:20.110322+0000 front 2024-05-10T11:46:20.110411+0000 (oldest deadline 2024-05-10T11:46:43.610132+0000) 2024-05-10T11:46:55.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:54 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:54.954+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-10T11:46:28.311217+0000 front 2024-05-10T11:46:28.311184+0000 (oldest deadline 2024-05-10T11:46:52.411332+0000) 2024-05-10T11:46:55.889 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:55 smithi188 podman[76706]: 2024-05-10 11:46:55.595197911 +0000 UTC m=+5.690410663 container died dca79fe29f17d4cb485d6057c5ffc42f4c56f408d38c0a4eecc5e64c045c9a02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-05-10T11:46:56.038 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:55 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:55.538+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.171852+0000 front 2024-05-10T11:45:45.171825+0000 (oldest deadline 2024-05-10T11:46:10.372558+0000) 2024-05-10T11:46:56.038 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:55 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:55.538+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.274899+0000 front 2024-05-10T11:45:57.274943+0000 (oldest deadline 2024-05-10T11:46:21.375013+0000) 2024-05-10T11:46:56.038 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:55 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:55.538+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.375471+0000 front 2024-05-10T11:46:01.375489+0000 (oldest deadline 2024-05-10T11:46:26.675500+0000) 2024-05-10T11:46:56.038 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:55 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:55.538+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:12.877843+0000 front 2024-05-10T11:46:12.877870+0000 (oldest deadline 2024-05-10T11:46:36.378001+0000) 2024-05-10T11:46:56.038 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:55 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:55.538+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:18.679081+0000 front 2024-05-10T11:46:18.679227+0000 (oldest deadline 2024-05-10T11:46:43.379060+0000) 2024-05-10T11:46:56.038 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:55 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:55.538+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-10T11:46:27.980824+0000 front 2024-05-10T11:46:27.980941+0000 (oldest deadline 2024-05-10T11:46:53.280786+0000) 2024-05-10T11:46:56.250 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:55 smithi188 podman[76706]: 2024-05-10 11:46:55.997972305 +0000 UTC m=+6.093185060 container cleanup dca79fe29f17d4cb485d6057c5ffc42f4c56f408d38c0a4eecc5e64c045c9a02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-05-10T11:46:56.250 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:55 smithi188 bash[76706]: ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8 2024-05-10T11:46:56.250 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:56 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:56.000+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.703618+0000 front 2024-05-10T11:45:45.703296+0000 (oldest deadline 2024-05-10T11:46:11.003467+0000) 2024-05-10T11:46:56.251 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:56 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:56.000+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:58.005499+0000 front 2024-05-10T11:45:58.005443+0000 (oldest deadline 2024-05-10T11:46:22.705582+0000) 2024-05-10T11:46:56.251 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:56 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:56.000+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:05.606931+0000 front 2024-05-10T11:46:05.606957+0000 (oldest deadline 2024-05-10T11:46:30.907138+0000) 2024-05-10T11:46:56.251 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:56 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:56.000+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:20.110322+0000 front 2024-05-10T11:46:20.110411+0000 (oldest deadline 2024-05-10T11:46:43.610132+0000) 2024-05-10T11:46:56.251 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:56 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:56.000+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-10T11:46:28.311217+0000 front 2024-05-10T11:46:28.311184+0000 (oldest deadline 2024-05-10T11:46:52.411332+0000) 2024-05-10T11:46:56.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:55 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:55.951+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:44.388044+0000 front 2024-05-10T11:45:44.388108+0000 (oldest deadline 2024-05-10T11:46:10.288056+0000) 2024-05-10T11:46:56.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:55 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:55.951+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.190342+0000 front 2024-05-10T11:45:57.190423+0000 (oldest deadline 2024-05-10T11:46:19.490457+0000) 2024-05-10T11:46:56.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:55 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:55.951+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.691618+0000 front 2024-05-10T11:46:01.691702+0000 (oldest deadline 2024-05-10T11:46:26.892345+0000) 2024-05-10T11:46:56.253 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:55 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:55.951+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:13.794432+0000 front 2024-05-10T11:46:13.794449+0000 (oldest deadline 2024-05-10T11:46:37.294473+0000) 2024-05-10T11:46:56.253 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:55 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:55.951+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:21.995604+0000 front 2024-05-10T11:46:21.995643+0000 (oldest deadline 2024-05-10T11:46:46.695720+0000) 2024-05-10T11:46:56.253 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:55 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:55.951+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-10T11:46:26.696333+0000 front 2024-05-10T11:46:26.696307+0000 (oldest deadline 2024-05-10T11:46:51.996382+0000) 2024-05-10T11:46:56.529 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:56 smithi188 podman[76726]: 2024-05-10 11:46:56.249264493 +0000 UTC m=+0.651667607 container remove dca79fe29f17d4cb485d6057c5ffc42f4c56f408d38c0a4eecc5e64c045c9a02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, maintainer=Guillaume Abrioux ) 2024-05-10T11:46:56.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:56 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:56.525+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.171852+0000 front 2024-05-10T11:45:45.171825+0000 (oldest deadline 2024-05-10T11:46:10.372558+0000) 2024-05-10T11:46:56.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:56 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:56.525+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.274899+0000 front 2024-05-10T11:45:57.274943+0000 (oldest deadline 2024-05-10T11:46:21.375013+0000) 2024-05-10T11:46:56.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:56 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:56.525+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.375471+0000 front 2024-05-10T11:46:01.375489+0000 (oldest deadline 2024-05-10T11:46:26.675500+0000) 2024-05-10T11:46:56.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:56 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:56.525+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:12.877843+0000 front 2024-05-10T11:46:12.877870+0000 (oldest deadline 2024-05-10T11:46:36.378001+0000) 2024-05-10T11:46:56.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:56 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:56.525+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:18.679081+0000 front 2024-05-10T11:46:18.679227+0000 (oldest deadline 2024-05-10T11:46:43.379060+0000) 2024-05-10T11:46:56.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:56 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:56.525+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-10T11:46:27.980824+0000 front 2024-05-10T11:46:27.980941+0000 (oldest deadline 2024-05-10T11:46:53.280786+0000) 2024-05-10T11:46:56.827 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:56 smithi188 podman[76816]: 2024-05-10 11:46:56.468481389 +0000 UTC m=+0.018128049 image pull ee5e4f21ef58047cb638cccd5cfbe8f6f43412b56c94b1d7fbd9cf3cb90a53a5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f 2024-05-10T11:46:56.827 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:56 smithi188 podman[76816]: 2024-05-10 11:46:56.583648816 +0000 UTC m=+0.133295509 container create c6d9a2cd006dea32f8b43968921e4563390db61706818a75eada85f893f5993d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240507, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-05-10T11:46:56.827 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:56 smithi188 podman[76816]: 2024-05-10 11:46:56.827720638 +0000 UTC m=+0.377367288 container init c6d9a2cd006dea32f8b43968921e4563390db61706818a75eada85f893f5993d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8-deactivate, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-05-10T11:46:57.120 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:56 smithi188 podman[76816]: 2024-05-10 11:46:56.831027569 +0000 UTC m=+0.380674224 container start c6d9a2cd006dea32f8b43968921e4563390db61706818a75eada85f893f5993d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8-deactivate, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-05-10T11:46:57.120 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:56 smithi188 podman[76816]: 2024-05-10 11:46:56.919443093 +0000 UTC m=+0.469089747 container attach c6d9a2cd006dea32f8b43968921e4563390db61706818a75eada85f893f5993d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2) 2024-05-10T11:46:57.121 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:56 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:56.971+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.703618+0000 front 2024-05-10T11:45:45.703296+0000 (oldest deadline 2024-05-10T11:46:11.003467+0000) 2024-05-10T11:46:57.121 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:56 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:56.971+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:58.005499+0000 front 2024-05-10T11:45:58.005443+0000 (oldest deadline 2024-05-10T11:46:22.705582+0000) 2024-05-10T11:46:57.121 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:56 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:56.971+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:05.606931+0000 front 2024-05-10T11:46:05.606957+0000 (oldest deadline 2024-05-10T11:46:30.907138+0000) 2024-05-10T11:46:57.121 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:56 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:56.971+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:20.110322+0000 front 2024-05-10T11:46:20.110411+0000 (oldest deadline 2024-05-10T11:46:43.610132+0000) 2024-05-10T11:46:57.121 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:56 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:56.971+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-10T11:46:28.311217+0000 front 2024-05-10T11:46:28.311184+0000 (oldest deadline 2024-05-10T11:46:52.411332+0000) 2024-05-10T11:46:57.253 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:56 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:56.974+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:44.388044+0000 front 2024-05-10T11:45:44.388108+0000 (oldest deadline 2024-05-10T11:46:10.288056+0000) 2024-05-10T11:46:57.253 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:56 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:56.974+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.190342+0000 front 2024-05-10T11:45:57.190423+0000 (oldest deadline 2024-05-10T11:46:19.490457+0000) 2024-05-10T11:46:57.253 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:56 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:56.974+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.691618+0000 front 2024-05-10T11:46:01.691702+0000 (oldest deadline 2024-05-10T11:46:26.892345+0000) 2024-05-10T11:46:57.253 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:56 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:56.974+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:13.794432+0000 front 2024-05-10T11:46:13.794449+0000 (oldest deadline 2024-05-10T11:46:37.294473+0000) 2024-05-10T11:46:57.253 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:56 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:56.974+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:21.995604+0000 front 2024-05-10T11:46:21.995643+0000 (oldest deadline 2024-05-10T11:46:46.695720+0000) 2024-05-10T11:46:57.253 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:56 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:56.974+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-10T11:46:26.696333+0000 front 2024-05-10T11:46:26.696307+0000 (oldest deadline 2024-05-10T11:46:51.996382+0000) 2024-05-10T11:46:57.410 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 10 11:46:57 smithi188 podman[76816]: 2024-05-10 11:46:57.120756468 +0000 UTC m=+0.670403122 container died c6d9a2cd006dea32f8b43968921e4563390db61706818a75eada85f893f5993d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-8-deactivate, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240507, GIT_CLEAN=True, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-05-10T11:46:57.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:57 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:57.486+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.171852+0000 front 2024-05-10T11:45:45.171825+0000 (oldest deadline 2024-05-10T11:46:10.372558+0000) 2024-05-10T11:46:57.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:57 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:57.486+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.274899+0000 front 2024-05-10T11:45:57.274943+0000 (oldest deadline 2024-05-10T11:46:21.375013+0000) 2024-05-10T11:46:57.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:57 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:57.486+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.375471+0000 front 2024-05-10T11:46:01.375489+0000 (oldest deadline 2024-05-10T11:46:26.675500+0000) 2024-05-10T11:46:57.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:57 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:57.486+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:12.877843+0000 front 2024-05-10T11:46:12.877870+0000 (oldest deadline 2024-05-10T11:46:36.378001+0000) 2024-05-10T11:46:57.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:57 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:57.486+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:18.679081+0000 front 2024-05-10T11:46:18.679227+0000 (oldest deadline 2024-05-10T11:46:43.379060+0000) 2024-05-10T11:46:57.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:57 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:57.486+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-10T11:46:27.980824+0000 front 2024-05-10T11:46:27.980941+0000 (oldest deadline 2024-05-10T11:46:53.280786+0000) 2024-05-10T11:46:57.824 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@osd.8.service' 2024-05-10T11:46:57.863 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T11:46:57.863 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-05-10T11:46:57.863 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-05-10T11:46:57.863 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@osd.9 2024-05-10T11:46:58.181 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:57 smithi100 systemd[1]: Stopping Ceph osd.9 for 20a542cc-0ebd-11ef-bc98-c7b262605968... 2024-05-10T11:46:58.278 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:57 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:57.966+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.703618+0000 front 2024-05-10T11:45:45.703296+0000 (oldest deadline 2024-05-10T11:46:11.003467+0000) 2024-05-10T11:46:58.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:57 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:57.966+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:58.005499+0000 front 2024-05-10T11:45:58.005443+0000 (oldest deadline 2024-05-10T11:46:22.705582+0000) 2024-05-10T11:46:58.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:57 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:57.966+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:05.606931+0000 front 2024-05-10T11:46:05.606957+0000 (oldest deadline 2024-05-10T11:46:30.907138+0000) 2024-05-10T11:46:58.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:57 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:57.966+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:20.110322+0000 front 2024-05-10T11:46:20.110411+0000 (oldest deadline 2024-05-10T11:46:43.610132+0000) 2024-05-10T11:46:58.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:57 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:57.966+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-10T11:46:28.311217+0000 front 2024-05-10T11:46:28.311184+0000 (oldest deadline 2024-05-10T11:46:52.411332+0000) 2024-05-10T11:46:58.444 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:58 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:58.181+0000 7fb62ca9d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T11:46:58.444 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:58 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:58.181+0000 7fb62ca9d700 -1 osd.9 94 *** Got signal Terminated *** 2024-05-10T11:46:58.444 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:58 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:58.181+0000 7fb62ca9d700 -1 osd.9 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T11:46:58.444 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:58 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:58.444+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.171852+0000 front 2024-05-10T11:45:45.171825+0000 (oldest deadline 2024-05-10T11:46:10.372558+0000) 2024-05-10T11:46:58.444 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:58 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:58.444+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.274899+0000 front 2024-05-10T11:45:57.274943+0000 (oldest deadline 2024-05-10T11:46:21.375013+0000) 2024-05-10T11:46:58.444 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:58 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:58.444+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.375471+0000 front 2024-05-10T11:46:01.375489+0000 (oldest deadline 2024-05-10T11:46:26.675500+0000) 2024-05-10T11:46:58.444 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:58 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:58.444+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:12.877843+0000 front 2024-05-10T11:46:12.877870+0000 (oldest deadline 2024-05-10T11:46:36.378001+0000) 2024-05-10T11:46:58.444 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:58 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:58.444+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:18.679081+0000 front 2024-05-10T11:46:18.679227+0000 (oldest deadline 2024-05-10T11:46:43.379060+0000) 2024-05-10T11:46:58.502 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:58 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:58.023+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:44.388044+0000 front 2024-05-10T11:45:44.388108+0000 (oldest deadline 2024-05-10T11:46:10.288056+0000) 2024-05-10T11:46:58.503 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:58 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:58.023+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.190342+0000 front 2024-05-10T11:45:57.190423+0000 (oldest deadline 2024-05-10T11:46:19.490457+0000) 2024-05-10T11:46:58.503 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:58 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:58.023+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.691618+0000 front 2024-05-10T11:46:01.691702+0000 (oldest deadline 2024-05-10T11:46:26.892345+0000) 2024-05-10T11:46:58.503 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:58 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:58.023+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:13.794432+0000 front 2024-05-10T11:46:13.794449+0000 (oldest deadline 2024-05-10T11:46:37.294473+0000) 2024-05-10T11:46:58.503 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:58 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:58.023+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:21.995604+0000 front 2024-05-10T11:46:21.995643+0000 (oldest deadline 2024-05-10T11:46:46.695720+0000) 2024-05-10T11:46:58.503 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:58 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:58.023+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-10T11:46:26.696333+0000 front 2024-05-10T11:46:26.696307+0000 (oldest deadline 2024-05-10T11:46:51.996382+0000) 2024-05-10T11:46:59.278 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:58 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:58.993+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.703618+0000 front 2024-05-10T11:45:45.703296+0000 (oldest deadline 2024-05-10T11:46:11.003467+0000) 2024-05-10T11:46:59.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:58 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:58.993+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:58.005499+0000 front 2024-05-10T11:45:58.005443+0000 (oldest deadline 2024-05-10T11:46:22.705582+0000) 2024-05-10T11:46:59.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:58 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:58.993+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:05.606931+0000 front 2024-05-10T11:46:05.606957+0000 (oldest deadline 2024-05-10T11:46:30.907138+0000) 2024-05-10T11:46:59.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:58 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:58.993+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:20.110322+0000 front 2024-05-10T11:46:20.110411+0000 (oldest deadline 2024-05-10T11:46:43.610132+0000) 2024-05-10T11:46:59.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:58 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:58.993+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-10T11:46:28.311217+0000 front 2024-05-10T11:46:28.311184+0000 (oldest deadline 2024-05-10T11:46:52.411332+0000) 2024-05-10T11:46:59.502 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:59 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:59.071+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:44.388044+0000 front 2024-05-10T11:45:44.388108+0000 (oldest deadline 2024-05-10T11:46:10.288056+0000) 2024-05-10T11:46:59.502 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:59 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:59.071+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.190342+0000 front 2024-05-10T11:45:57.190423+0000 (oldest deadline 2024-05-10T11:46:19.490457+0000) 2024-05-10T11:46:59.503 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:59 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:59.071+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.691618+0000 front 2024-05-10T11:46:01.691702+0000 (oldest deadline 2024-05-10T11:46:26.892345+0000) 2024-05-10T11:46:59.503 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:59 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:59.071+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:13.794432+0000 front 2024-05-10T11:46:13.794449+0000 (oldest deadline 2024-05-10T11:46:37.294473+0000) 2024-05-10T11:46:59.503 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:59 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:59.071+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:21.995604+0000 front 2024-05-10T11:46:21.995643+0000 (oldest deadline 2024-05-10T11:46:46.695720+0000) 2024-05-10T11:46:59.503 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:46:59 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:46:59.071+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-10T11:46:26.696333+0000 front 2024-05-10T11:46:26.696307+0000 (oldest deadline 2024-05-10T11:46:51.996382+0000) 2024-05-10T11:46:59.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:58 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:58.444+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-10T11:46:27.980824+0000 front 2024-05-10T11:46:27.980941+0000 (oldest deadline 2024-05-10T11:46:53.280786+0000) 2024-05-10T11:46:59.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:59 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:59.403+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.171852+0000 front 2024-05-10T11:45:45.171825+0000 (oldest deadline 2024-05-10T11:46:10.372558+0000) 2024-05-10T11:46:59.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:59 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:59.403+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.274899+0000 front 2024-05-10T11:45:57.274943+0000 (oldest deadline 2024-05-10T11:46:21.375013+0000) 2024-05-10T11:46:59.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:59 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:59.403+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.375471+0000 front 2024-05-10T11:46:01.375489+0000 (oldest deadline 2024-05-10T11:46:26.675500+0000) 2024-05-10T11:46:59.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:59 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:59.403+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:12.877843+0000 front 2024-05-10T11:46:12.877870+0000 (oldest deadline 2024-05-10T11:46:36.378001+0000) 2024-05-10T11:46:59.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:59 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:59.403+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:18.679081+0000 front 2024-05-10T11:46:18.679227+0000 (oldest deadline 2024-05-10T11:46:43.379060+0000) 2024-05-10T11:46:59.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:46:59 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:46:59.403+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-10T11:46:27.980824+0000 front 2024-05-10T11:46:27.980941+0000 (oldest deadline 2024-05-10T11:46:53.280786+0000) 2024-05-10T11:47:00.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:59 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:59.953+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.703618+0000 front 2024-05-10T11:45:45.703296+0000 (oldest deadline 2024-05-10T11:46:11.003467+0000) 2024-05-10T11:47:00.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:59 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:59.953+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:58.005499+0000 front 2024-05-10T11:45:58.005443+0000 (oldest deadline 2024-05-10T11:46:22.705582+0000) 2024-05-10T11:47:00.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:59 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:59.953+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:05.606931+0000 front 2024-05-10T11:46:05.606957+0000 (oldest deadline 2024-05-10T11:46:30.907138+0000) 2024-05-10T11:47:00.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:59 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:59.953+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:20.110322+0000 front 2024-05-10T11:46:20.110411+0000 (oldest deadline 2024-05-10T11:46:43.610132+0000) 2024-05-10T11:47:00.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:46:59 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:46:59.953+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-10T11:46:28.311217+0000 front 2024-05-10T11:46:28.311184+0000 (oldest deadline 2024-05-10T11:46:52.411332+0000) 2024-05-10T11:47:00.502 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:00 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:00.107+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:44.388044+0000 front 2024-05-10T11:45:44.388108+0000 (oldest deadline 2024-05-10T11:46:10.288056+0000) 2024-05-10T11:47:00.503 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:00 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:00.107+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.190342+0000 front 2024-05-10T11:45:57.190423+0000 (oldest deadline 2024-05-10T11:46:19.490457+0000) 2024-05-10T11:47:00.503 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:00 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:00.107+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.691618+0000 front 2024-05-10T11:46:01.691702+0000 (oldest deadline 2024-05-10T11:46:26.892345+0000) 2024-05-10T11:47:00.503 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:00 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:00.107+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:13.794432+0000 front 2024-05-10T11:46:13.794449+0000 (oldest deadline 2024-05-10T11:46:37.294473+0000) 2024-05-10T11:47:00.503 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:00 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:00.107+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:21.995604+0000 front 2024-05-10T11:46:21.995643+0000 (oldest deadline 2024-05-10T11:46:46.695720+0000) 2024-05-10T11:47:00.503 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:00 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:00.107+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-10T11:46:26.696333+0000 front 2024-05-10T11:46:26.696307+0000 (oldest deadline 2024-05-10T11:46:51.996382+0000) 2024-05-10T11:47:00.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:47:00 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:47:00.444+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.171852+0000 front 2024-05-10T11:45:45.171825+0000 (oldest deadline 2024-05-10T11:46:10.372558+0000) 2024-05-10T11:47:00.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:47:00 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:47:00.444+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.274899+0000 front 2024-05-10T11:45:57.274943+0000 (oldest deadline 2024-05-10T11:46:21.375013+0000) 2024-05-10T11:47:00.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:47:00 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:47:00.444+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.375471+0000 front 2024-05-10T11:46:01.375489+0000 (oldest deadline 2024-05-10T11:46:26.675500+0000) 2024-05-10T11:47:00.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:47:00 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:47:00.444+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:12.877843+0000 front 2024-05-10T11:46:12.877870+0000 (oldest deadline 2024-05-10T11:46:36.378001+0000) 2024-05-10T11:47:00.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:47:00 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:47:00.444+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:18.679081+0000 front 2024-05-10T11:46:18.679227+0000 (oldest deadline 2024-05-10T11:46:43.379060+0000) 2024-05-10T11:47:00.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:47:00 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:47:00.444+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-10T11:46:27.980824+0000 front 2024-05-10T11:46:27.980941+0000 (oldest deadline 2024-05-10T11:46:53.280786+0000) 2024-05-10T11:47:01.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:00 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:00.910+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.703618+0000 front 2024-05-10T11:45:45.703296+0000 (oldest deadline 2024-05-10T11:46:11.003467+0000) 2024-05-10T11:47:01.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:00 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:00.910+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:58.005499+0000 front 2024-05-10T11:45:58.005443+0000 (oldest deadline 2024-05-10T11:46:22.705582+0000) 2024-05-10T11:47:01.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:00 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:00.910+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:05.606931+0000 front 2024-05-10T11:46:05.606957+0000 (oldest deadline 2024-05-10T11:46:30.907138+0000) 2024-05-10T11:47:01.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:00 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:00.910+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:20.110322+0000 front 2024-05-10T11:46:20.110411+0000 (oldest deadline 2024-05-10T11:46:43.610132+0000) 2024-05-10T11:47:01.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:00 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:00.910+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-10T11:46:28.311217+0000 front 2024-05-10T11:46:28.311184+0000 (oldest deadline 2024-05-10T11:46:52.411332+0000) 2024-05-10T11:47:01.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:00 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:00.910+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6822 osd.6 since back 2024-05-10T11:46:34.012719+0000 front 2024-05-10T11:46:34.012678+0000 (oldest deadline 2024-05-10T11:47:00.413058+0000) 2024-05-10T11:47:01.502 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:01 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:01.114+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:44.388044+0000 front 2024-05-10T11:45:44.388108+0000 (oldest deadline 2024-05-10T11:46:10.288056+0000) 2024-05-10T11:47:01.503 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:01 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:01.114+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.190342+0000 front 2024-05-10T11:45:57.190423+0000 (oldest deadline 2024-05-10T11:46:19.490457+0000) 2024-05-10T11:47:01.503 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:01 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:01.114+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.691618+0000 front 2024-05-10T11:46:01.691702+0000 (oldest deadline 2024-05-10T11:46:26.892345+0000) 2024-05-10T11:47:01.503 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:01 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:01.114+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:13.794432+0000 front 2024-05-10T11:46:13.794449+0000 (oldest deadline 2024-05-10T11:46:37.294473+0000) 2024-05-10T11:47:01.503 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:01 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:01.114+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:21.995604+0000 front 2024-05-10T11:46:21.995643+0000 (oldest deadline 2024-05-10T11:46:46.695720+0000) 2024-05-10T11:47:01.503 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:01 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:01.114+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-10T11:46:26.696333+0000 front 2024-05-10T11:46:26.696307+0000 (oldest deadline 2024-05-10T11:46:51.996382+0000) 2024-05-10T11:47:01.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:47:01 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:47:01.455+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.171852+0000 front 2024-05-10T11:45:45.171825+0000 (oldest deadline 2024-05-10T11:46:10.372558+0000) 2024-05-10T11:47:01.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:47:01 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:47:01.455+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.274899+0000 front 2024-05-10T11:45:57.274943+0000 (oldest deadline 2024-05-10T11:46:21.375013+0000) 2024-05-10T11:47:01.788 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:47:01 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:47:01.455+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.375471+0000 front 2024-05-10T11:46:01.375489+0000 (oldest deadline 2024-05-10T11:46:26.675500+0000) 2024-05-10T11:47:01.789 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:47:01 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:47:01.455+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:12.877843+0000 front 2024-05-10T11:46:12.877870+0000 (oldest deadline 2024-05-10T11:46:36.378001+0000) 2024-05-10T11:47:01.789 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:47:01 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:47:01.455+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:18.679081+0000 front 2024-05-10T11:46:18.679227+0000 (oldest deadline 2024-05-10T11:46:43.379060+0000) 2024-05-10T11:47:01.789 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:47:01 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:47:01.455+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-10T11:46:27.980824+0000 front 2024-05-10T11:46:27.980941+0000 (oldest deadline 2024-05-10T11:46:53.280786+0000) 2024-05-10T11:47:02.180 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:01 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:01.901+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.703618+0000 front 2024-05-10T11:45:45.703296+0000 (oldest deadline 2024-05-10T11:46:11.003467+0000) 2024-05-10T11:47:02.180 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:01 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:01.901+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:58.005499+0000 front 2024-05-10T11:45:58.005443+0000 (oldest deadline 2024-05-10T11:46:22.705582+0000) 2024-05-10T11:47:02.180 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:01 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:01.901+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:05.606931+0000 front 2024-05-10T11:46:05.606957+0000 (oldest deadline 2024-05-10T11:46:30.907138+0000) 2024-05-10T11:47:02.180 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:01 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:01.901+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:20.110322+0000 front 2024-05-10T11:46:20.110411+0000 (oldest deadline 2024-05-10T11:46:43.610132+0000) 2024-05-10T11:47:02.180 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:01 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:01.901+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-10T11:46:28.311217+0000 front 2024-05-10T11:46:28.311184+0000 (oldest deadline 2024-05-10T11:46:52.411332+0000) 2024-05-10T11:47:02.181 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:01 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:01.901+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6822 osd.6 since back 2024-05-10T11:46:34.012719+0000 front 2024-05-10T11:46:34.012678+0000 (oldest deadline 2024-05-10T11:47:00.413058+0000) 2024-05-10T11:47:02.502 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:02 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:02.092+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:44.388044+0000 front 2024-05-10T11:45:44.388108+0000 (oldest deadline 2024-05-10T11:46:10.288056+0000) 2024-05-10T11:47:02.502 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:02 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:02.092+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.190342+0000 front 2024-05-10T11:45:57.190423+0000 (oldest deadline 2024-05-10T11:46:19.490457+0000) 2024-05-10T11:47:02.503 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:02 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:02.092+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.691618+0000 front 2024-05-10T11:46:01.691702+0000 (oldest deadline 2024-05-10T11:46:26.892345+0000) 2024-05-10T11:47:02.503 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:02 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:02.092+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:13.794432+0000 front 2024-05-10T11:46:13.794449+0000 (oldest deadline 2024-05-10T11:46:37.294473+0000) 2024-05-10T11:47:02.503 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:02 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:02.092+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:21.995604+0000 front 2024-05-10T11:46:21.995643+0000 (oldest deadline 2024-05-10T11:46:46.695720+0000) 2024-05-10T11:47:02.503 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:02 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:02.092+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-10T11:46:26.696333+0000 front 2024-05-10T11:46:26.696307+0000 (oldest deadline 2024-05-10T11:46:51.996382+0000) 2024-05-10T11:47:02.503 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:02 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:02.092+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6822 osd.6 since back 2024-05-10T11:46:36.097215+0000 front 2024-05-10T11:46:36.097243+0000 (oldest deadline 2024-05-10T11:47:01.397239+0000) 2024-05-10T11:47:02.790 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:47:02 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:47:02.436+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.171852+0000 front 2024-05-10T11:45:45.171825+0000 (oldest deadline 2024-05-10T11:46:10.372558+0000) 2024-05-10T11:47:02.790 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:47:02 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:47:02.436+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.274899+0000 front 2024-05-10T11:45:57.274943+0000 (oldest deadline 2024-05-10T11:46:21.375013+0000) 2024-05-10T11:47:02.790 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:47:02 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:47:02.436+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.375471+0000 front 2024-05-10T11:46:01.375489+0000 (oldest deadline 2024-05-10T11:46:26.675500+0000) 2024-05-10T11:47:02.790 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:47:02 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:47:02.436+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:12.877843+0000 front 2024-05-10T11:46:12.877870+0000 (oldest deadline 2024-05-10T11:46:36.378001+0000) 2024-05-10T11:47:02.790 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:47:02 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:47:02.436+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:18.679081+0000 front 2024-05-10T11:46:18.679227+0000 (oldest deadline 2024-05-10T11:46:43.379060+0000) 2024-05-10T11:47:02.790 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:47:02 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9[62826]: 2024-05-10T11:47:02.436+0000 7fb625089700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-10T11:46:27.980824+0000 front 2024-05-10T11:46:27.980941+0000 (oldest deadline 2024-05-10T11:46:53.280786+0000) 2024-05-10T11:47:03.192 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:02 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:02.903+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.703618+0000 front 2024-05-10T11:45:45.703296+0000 (oldest deadline 2024-05-10T11:46:11.003467+0000) 2024-05-10T11:47:03.192 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:02 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:02.903+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:58.005499+0000 front 2024-05-10T11:45:58.005443+0000 (oldest deadline 2024-05-10T11:46:22.705582+0000) 2024-05-10T11:47:03.192 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:02 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:02.903+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:05.606931+0000 front 2024-05-10T11:46:05.606957+0000 (oldest deadline 2024-05-10T11:46:30.907138+0000) 2024-05-10T11:47:03.192 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:02 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:02.903+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:20.110322+0000 front 2024-05-10T11:46:20.110411+0000 (oldest deadline 2024-05-10T11:46:43.610132+0000) 2024-05-10T11:47:03.192 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:02 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:02.903+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-10T11:46:28.311217+0000 front 2024-05-10T11:46:28.311184+0000 (oldest deadline 2024-05-10T11:46:52.411332+0000) 2024-05-10T11:47:03.192 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:02 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:02.903+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6822 osd.6 since back 2024-05-10T11:46:34.012719+0000 front 2024-05-10T11:46:34.012678+0000 (oldest deadline 2024-05-10T11:47:00.413058+0000) 2024-05-10T11:47:03.502 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:03 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:03.112+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:44.388044+0000 front 2024-05-10T11:45:44.388108+0000 (oldest deadline 2024-05-10T11:46:10.288056+0000) 2024-05-10T11:47:03.503 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:03 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:03.112+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.190342+0000 front 2024-05-10T11:45:57.190423+0000 (oldest deadline 2024-05-10T11:46:19.490457+0000) 2024-05-10T11:47:03.503 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:03 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:03.112+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.691618+0000 front 2024-05-10T11:46:01.691702+0000 (oldest deadline 2024-05-10T11:46:26.892345+0000) 2024-05-10T11:47:03.503 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:03 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:03.112+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:13.794432+0000 front 2024-05-10T11:46:13.794449+0000 (oldest deadline 2024-05-10T11:46:37.294473+0000) 2024-05-10T11:47:03.503 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:03 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:03.112+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:21.995604+0000 front 2024-05-10T11:46:21.995643+0000 (oldest deadline 2024-05-10T11:46:46.695720+0000) 2024-05-10T11:47:03.503 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:03 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:03.112+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-10T11:46:26.696333+0000 front 2024-05-10T11:46:26.696307+0000 (oldest deadline 2024-05-10T11:46:51.996382+0000) 2024-05-10T11:47:03.503 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:03 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:03.112+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6822 osd.6 since back 2024-05-10T11:46:36.097215+0000 front 2024-05-10T11:46:36.097243+0000 (oldest deadline 2024-05-10T11:47:01.397239+0000) 2024-05-10T11:47:03.890 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:47:03 smithi100 podman[141441]: 2024-05-10 11:47:03.604895264 +0000 UTC m=+5.544480511 container died f887ad41d083dd418a660fa0e39c965ca2f2f5d0b597bfbfd4e6023fbc6d754d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-10T11:47:04.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:03 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:03.908+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.703618+0000 front 2024-05-10T11:45:45.703296+0000 (oldest deadline 2024-05-10T11:46:11.003467+0000) 2024-05-10T11:47:04.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:03 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:03.908+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:58.005499+0000 front 2024-05-10T11:45:58.005443+0000 (oldest deadline 2024-05-10T11:46:22.705582+0000) 2024-05-10T11:47:04.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:03 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:03.908+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:05.606931+0000 front 2024-05-10T11:46:05.606957+0000 (oldest deadline 2024-05-10T11:46:30.907138+0000) 2024-05-10T11:47:04.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:03 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:03.908+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:20.110322+0000 front 2024-05-10T11:46:20.110411+0000 (oldest deadline 2024-05-10T11:46:43.610132+0000) 2024-05-10T11:47:04.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:03 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:03.908+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-10T11:46:28.311217+0000 front 2024-05-10T11:46:28.311184+0000 (oldest deadline 2024-05-10T11:46:52.411332+0000) 2024-05-10T11:47:04.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:03 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:03.908+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6822 osd.6 since back 2024-05-10T11:46:34.012719+0000 front 2024-05-10T11:46:34.012678+0000 (oldest deadline 2024-05-10T11:47:00.413058+0000) 2024-05-10T11:47:04.288 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:47:04 smithi100 podman[141441]: 2024-05-10 11:47:04.097967285 +0000 UTC m=+6.037552525 container cleanup f887ad41d083dd418a660fa0e39c965ca2f2f5d0b597bfbfd4e6023fbc6d754d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, CEPH_POINT_RELEASE=) 2024-05-10T11:47:04.288 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:47:04 smithi100 bash[141441]: ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9 2024-05-10T11:47:04.502 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:04 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:04.146+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:44.388044+0000 front 2024-05-10T11:45:44.388108+0000 (oldest deadline 2024-05-10T11:46:10.288056+0000) 2024-05-10T11:47:04.502 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:04 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:04.146+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.190342+0000 front 2024-05-10T11:45:57.190423+0000 (oldest deadline 2024-05-10T11:46:19.490457+0000) 2024-05-10T11:47:04.503 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:04 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:04.146+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.691618+0000 front 2024-05-10T11:46:01.691702+0000 (oldest deadline 2024-05-10T11:46:26.892345+0000) 2024-05-10T11:47:04.503 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:04 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:04.146+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:13.794432+0000 front 2024-05-10T11:46:13.794449+0000 (oldest deadline 2024-05-10T11:46:37.294473+0000) 2024-05-10T11:47:04.503 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:04 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:04.146+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:21.995604+0000 front 2024-05-10T11:46:21.995643+0000 (oldest deadline 2024-05-10T11:46:46.695720+0000) 2024-05-10T11:47:04.503 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:04 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:04.146+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-10T11:46:26.696333+0000 front 2024-05-10T11:46:26.696307+0000 (oldest deadline 2024-05-10T11:46:51.996382+0000) 2024-05-10T11:47:04.503 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:04 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:04.146+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6822 osd.6 since back 2024-05-10T11:46:36.097215+0000 front 2024-05-10T11:46:36.097243+0000 (oldest deadline 2024-05-10T11:47:01.397239+0000) 2024-05-10T11:47:04.750 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:47:04 smithi100 podman[141458]: 2024-05-10 11:47:04.449844685 +0000 UTC m=+0.842194931 container remove f887ad41d083dd418a660fa0e39c965ca2f2f5d0b597bfbfd4e6023fbc6d754d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, maintainer=Guillaume Abrioux , RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-10T11:47:05.038 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:47:04 smithi100 podman[141547]: 2024-05-10 11:47:04.654050676 +0000 UTC m=+0.018483538 image pull ee5e4f21ef58047cb638cccd5cfbe8f6f43412b56c94b1d7fbd9cf3cb90a53a5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f 2024-05-10T11:47:05.038 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:47:04 smithi100 podman[141547]: 2024-05-10 11:47:04.767306305 +0000 UTC m=+0.131739138 container create 73c6f03433e929f7157e04cc77022f691c495d4622ff4672b360af60486264b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507) 2024-05-10T11:47:05.038 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:47:04 smithi100 podman[141547]: 2024-05-10 11:47:04.945035215 +0000 UTC m=+0.309468050 container init 73c6f03433e929f7157e04cc77022f691c495d4622ff4672b360af60486264b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0) 2024-05-10T11:47:05.039 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:47:04 smithi100 podman[141547]: 2024-05-10 11:47:04.948251093 +0000 UTC m=+0.312683928 container start 73c6f03433e929f7157e04cc77022f691c495d4622ff4672b360af60486264b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-10T11:47:05.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:04 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:04.927+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.703618+0000 front 2024-05-10T11:45:45.703296+0000 (oldest deadline 2024-05-10T11:46:11.003467+0000) 2024-05-10T11:47:05.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:04 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:04.927+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:58.005499+0000 front 2024-05-10T11:45:58.005443+0000 (oldest deadline 2024-05-10T11:46:22.705582+0000) 2024-05-10T11:47:05.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:04 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:04.927+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:05.606931+0000 front 2024-05-10T11:46:05.606957+0000 (oldest deadline 2024-05-10T11:46:30.907138+0000) 2024-05-10T11:47:05.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:04 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:04.927+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:20.110322+0000 front 2024-05-10T11:46:20.110411+0000 (oldest deadline 2024-05-10T11:46:43.610132+0000) 2024-05-10T11:47:05.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:04 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:04.927+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-10T11:46:28.311217+0000 front 2024-05-10T11:46:28.311184+0000 (oldest deadline 2024-05-10T11:46:52.411332+0000) 2024-05-10T11:47:05.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:04 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:04.927+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6822 osd.6 since back 2024-05-10T11:46:34.012719+0000 front 2024-05-10T11:46:34.012678+0000 (oldest deadline 2024-05-10T11:47:00.413058+0000) 2024-05-10T11:47:05.502 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:05 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:05.119+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:44.388044+0000 front 2024-05-10T11:45:44.388108+0000 (oldest deadline 2024-05-10T11:46:10.288056+0000) 2024-05-10T11:47:05.503 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:05 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:05.119+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.190342+0000 front 2024-05-10T11:45:57.190423+0000 (oldest deadline 2024-05-10T11:46:19.490457+0000) 2024-05-10T11:47:05.503 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:05 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:05.119+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.691618+0000 front 2024-05-10T11:46:01.691702+0000 (oldest deadline 2024-05-10T11:46:26.892345+0000) 2024-05-10T11:47:05.503 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:05 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:05.119+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:13.794432+0000 front 2024-05-10T11:46:13.794449+0000 (oldest deadline 2024-05-10T11:46:37.294473+0000) 2024-05-10T11:47:05.503 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:05 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:05.119+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:21.995604+0000 front 2024-05-10T11:46:21.995643+0000 (oldest deadline 2024-05-10T11:46:46.695720+0000) 2024-05-10T11:47:05.503 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:05 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:05.119+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-10T11:46:26.696333+0000 front 2024-05-10T11:46:26.696307+0000 (oldest deadline 2024-05-10T11:46:51.996382+0000) 2024-05-10T11:47:05.503 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:05 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:05.119+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6822 osd.6 since back 2024-05-10T11:46:36.097215+0000 front 2024-05-10T11:46:36.097243+0000 (oldest deadline 2024-05-10T11:47:01.397239+0000) 2024-05-10T11:47:05.538 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:47:05 smithi100 podman[141547]: 2024-05-10 11:47:05.044970562 +0000 UTC m=+0.409403407 container attach 73c6f03433e929f7157e04cc77022f691c495d4622ff4672b360af60486264b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-05-10T11:47:05.538 INFO:journalctl@ceph.osd.9.smithi100.stdout:May 10 11:47:05 smithi100 podman[141547]: 2024-05-10 11:47:05.243038431 +0000 UTC m=+0.607471268 container died 73c6f03433e929f7157e04cc77022f691c495d4622ff4672b360af60486264b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-05-10T11:47:05.836 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@osd.9.service' 2024-05-10T11:47:05.874 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T11:47:05.875 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-05-10T11:47:05.875 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-05-10T11:47:05.875 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@osd.10 2024-05-10T11:47:06.199 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:05 smithi178 systemd[1]: Stopping Ceph osd.10 for 20a542cc-0ebd-11ef-bc98-c7b262605968... 2024-05-10T11:47:06.199 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:06 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:06.140+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:44.388044+0000 front 2024-05-10T11:45:44.388108+0000 (oldest deadline 2024-05-10T11:46:10.288056+0000) 2024-05-10T11:47:06.199 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:06 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:06.140+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.190342+0000 front 2024-05-10T11:45:57.190423+0000 (oldest deadline 2024-05-10T11:46:19.490457+0000) 2024-05-10T11:47:06.199 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:06 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:06.140+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.691618+0000 front 2024-05-10T11:46:01.691702+0000 (oldest deadline 2024-05-10T11:46:26.892345+0000) 2024-05-10T11:47:06.199 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:06 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:06.140+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:13.794432+0000 front 2024-05-10T11:46:13.794449+0000 (oldest deadline 2024-05-10T11:46:37.294473+0000) 2024-05-10T11:47:06.200 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:06 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:06.140+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:21.995604+0000 front 2024-05-10T11:46:21.995643+0000 (oldest deadline 2024-05-10T11:46:46.695720+0000) 2024-05-10T11:47:06.200 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:06 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:06.140+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-10T11:46:26.696333+0000 front 2024-05-10T11:46:26.696307+0000 (oldest deadline 2024-05-10T11:46:51.996382+0000) 2024-05-10T11:47:06.200 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:06 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:06.140+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6822 osd.6 since back 2024-05-10T11:46:36.097215+0000 front 2024-05-10T11:46:36.097243+0000 (oldest deadline 2024-05-10T11:47:01.397239+0000) 2024-05-10T11:47:06.200 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:06 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:06.198+0000 7fbce6790700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T11:47:06.200 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:06 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:06.198+0000 7fbce6790700 -1 osd.10 94 *** Got signal Terminated *** 2024-05-10T11:47:06.200 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:06 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:06.198+0000 7fbce6790700 -1 osd.10 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T11:47:06.278 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:05 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:05.920+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.703618+0000 front 2024-05-10T11:45:45.703296+0000 (oldest deadline 2024-05-10T11:46:11.003467+0000) 2024-05-10T11:47:06.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:05 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:05.920+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:58.005499+0000 front 2024-05-10T11:45:58.005443+0000 (oldest deadline 2024-05-10T11:46:22.705582+0000) 2024-05-10T11:47:06.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:05 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:05.920+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:05.606931+0000 front 2024-05-10T11:46:05.606957+0000 (oldest deadline 2024-05-10T11:46:30.907138+0000) 2024-05-10T11:47:06.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:05 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:05.920+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:20.110322+0000 front 2024-05-10T11:46:20.110411+0000 (oldest deadline 2024-05-10T11:46:43.610132+0000) 2024-05-10T11:47:06.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:05 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:05.920+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-10T11:46:28.311217+0000 front 2024-05-10T11:46:28.311184+0000 (oldest deadline 2024-05-10T11:46:52.411332+0000) 2024-05-10T11:47:06.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:05 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:05.920+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6822 osd.6 since back 2024-05-10T11:46:34.012719+0000 front 2024-05-10T11:46:34.012678+0000 (oldest deadline 2024-05-10T11:47:00.413058+0000) 2024-05-10T11:47:07.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:06 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:06.940+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.703618+0000 front 2024-05-10T11:45:45.703296+0000 (oldest deadline 2024-05-10T11:46:11.003467+0000) 2024-05-10T11:47:07.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:06 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:06.940+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:58.005499+0000 front 2024-05-10T11:45:58.005443+0000 (oldest deadline 2024-05-10T11:46:22.705582+0000) 2024-05-10T11:47:07.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:06 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:06.940+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:05.606931+0000 front 2024-05-10T11:46:05.606957+0000 (oldest deadline 2024-05-10T11:46:30.907138+0000) 2024-05-10T11:47:07.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:06 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:06.940+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:20.110322+0000 front 2024-05-10T11:46:20.110411+0000 (oldest deadline 2024-05-10T11:46:43.610132+0000) 2024-05-10T11:47:07.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:06 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:06.940+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-10T11:46:28.311217+0000 front 2024-05-10T11:46:28.311184+0000 (oldest deadline 2024-05-10T11:46:52.411332+0000) 2024-05-10T11:47:07.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:06 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:06.940+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6822 osd.6 since back 2024-05-10T11:46:34.012719+0000 front 2024-05-10T11:46:34.012678+0000 (oldest deadline 2024-05-10T11:47:00.413058+0000) 2024-05-10T11:47:07.502 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:07 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:07.101+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:44.388044+0000 front 2024-05-10T11:45:44.388108+0000 (oldest deadline 2024-05-10T11:46:10.288056+0000) 2024-05-10T11:47:07.502 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:07 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:07.101+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.190342+0000 front 2024-05-10T11:45:57.190423+0000 (oldest deadline 2024-05-10T11:46:19.490457+0000) 2024-05-10T11:47:07.502 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:07 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:07.101+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.691618+0000 front 2024-05-10T11:46:01.691702+0000 (oldest deadline 2024-05-10T11:46:26.892345+0000) 2024-05-10T11:47:07.503 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:07 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:07.101+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:13.794432+0000 front 2024-05-10T11:46:13.794449+0000 (oldest deadline 2024-05-10T11:46:37.294473+0000) 2024-05-10T11:47:07.503 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:07 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:07.101+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:21.995604+0000 front 2024-05-10T11:46:21.995643+0000 (oldest deadline 2024-05-10T11:46:46.695720+0000) 2024-05-10T11:47:07.503 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:07 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:07.101+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-10T11:46:26.696333+0000 front 2024-05-10T11:46:26.696307+0000 (oldest deadline 2024-05-10T11:46:51.996382+0000) 2024-05-10T11:47:07.503 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:07 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:07.101+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6822 osd.6 since back 2024-05-10T11:46:36.097215+0000 front 2024-05-10T11:46:36.097243+0000 (oldest deadline 2024-05-10T11:47:01.397239+0000) 2024-05-10T11:47:08.278 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:07 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:07.963+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.703618+0000 front 2024-05-10T11:45:45.703296+0000 (oldest deadline 2024-05-10T11:46:11.003467+0000) 2024-05-10T11:47:08.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:07 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:07.963+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:58.005499+0000 front 2024-05-10T11:45:58.005443+0000 (oldest deadline 2024-05-10T11:46:22.705582+0000) 2024-05-10T11:47:08.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:07 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:07.963+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:05.606931+0000 front 2024-05-10T11:46:05.606957+0000 (oldest deadline 2024-05-10T11:46:30.907138+0000) 2024-05-10T11:47:08.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:07 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:07.963+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:20.110322+0000 front 2024-05-10T11:46:20.110411+0000 (oldest deadline 2024-05-10T11:46:43.610132+0000) 2024-05-10T11:47:08.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:07 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:07.963+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-10T11:46:28.311217+0000 front 2024-05-10T11:46:28.311184+0000 (oldest deadline 2024-05-10T11:46:52.411332+0000) 2024-05-10T11:47:08.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:07 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:07.963+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6822 osd.6 since back 2024-05-10T11:46:34.012719+0000 front 2024-05-10T11:46:34.012678+0000 (oldest deadline 2024-05-10T11:47:00.413058+0000) 2024-05-10T11:47:08.502 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:08 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:08.088+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:44.388044+0000 front 2024-05-10T11:45:44.388108+0000 (oldest deadline 2024-05-10T11:46:10.288056+0000) 2024-05-10T11:47:08.502 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:08 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:08.088+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.190342+0000 front 2024-05-10T11:45:57.190423+0000 (oldest deadline 2024-05-10T11:46:19.490457+0000) 2024-05-10T11:47:08.503 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:08 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:08.088+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.691618+0000 front 2024-05-10T11:46:01.691702+0000 (oldest deadline 2024-05-10T11:46:26.892345+0000) 2024-05-10T11:47:08.503 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:08 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:08.088+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:13.794432+0000 front 2024-05-10T11:46:13.794449+0000 (oldest deadline 2024-05-10T11:46:37.294473+0000) 2024-05-10T11:47:08.503 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:08 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:08.088+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:21.995604+0000 front 2024-05-10T11:46:21.995643+0000 (oldest deadline 2024-05-10T11:46:46.695720+0000) 2024-05-10T11:47:08.503 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:08 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:08.088+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-10T11:46:26.696333+0000 front 2024-05-10T11:46:26.696307+0000 (oldest deadline 2024-05-10T11:46:51.996382+0000) 2024-05-10T11:47:08.503 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:08 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:08.088+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6822 osd.6 since back 2024-05-10T11:46:36.097215+0000 front 2024-05-10T11:46:36.097243+0000 (oldest deadline 2024-05-10T11:47:01.397239+0000) 2024-05-10T11:47:09.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:08 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:08.947+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.703618+0000 front 2024-05-10T11:45:45.703296+0000 (oldest deadline 2024-05-10T11:46:11.003467+0000) 2024-05-10T11:47:09.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:08 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:08.947+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:58.005499+0000 front 2024-05-10T11:45:58.005443+0000 (oldest deadline 2024-05-10T11:46:22.705582+0000) 2024-05-10T11:47:09.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:08 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:08.947+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:05.606931+0000 front 2024-05-10T11:46:05.606957+0000 (oldest deadline 2024-05-10T11:46:30.907138+0000) 2024-05-10T11:47:09.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:08 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:08.947+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:20.110322+0000 front 2024-05-10T11:46:20.110411+0000 (oldest deadline 2024-05-10T11:46:43.610132+0000) 2024-05-10T11:47:09.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:08 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:08.947+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-10T11:46:28.311217+0000 front 2024-05-10T11:46:28.311184+0000 (oldest deadline 2024-05-10T11:46:52.411332+0000) 2024-05-10T11:47:09.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:08 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:08.947+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6822 osd.6 since back 2024-05-10T11:46:34.012719+0000 front 2024-05-10T11:46:34.012678+0000 (oldest deadline 2024-05-10T11:47:00.413058+0000) 2024-05-10T11:47:09.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:08 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:08.947+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6820 osd.7 since back 2024-05-10T11:46:46.814517+0000 front 2024-05-10T11:46:46.814639+0000 (oldest deadline 2024-05-10T11:47:08.414926+0000) 2024-05-10T11:47:09.502 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:09 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:09.113+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:44.388044+0000 front 2024-05-10T11:45:44.388108+0000 (oldest deadline 2024-05-10T11:46:10.288056+0000) 2024-05-10T11:47:09.502 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:09 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:09.113+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.190342+0000 front 2024-05-10T11:45:57.190423+0000 (oldest deadline 2024-05-10T11:46:19.490457+0000) 2024-05-10T11:47:09.503 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:09 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:09.113+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.691618+0000 front 2024-05-10T11:46:01.691702+0000 (oldest deadline 2024-05-10T11:46:26.892345+0000) 2024-05-10T11:47:09.503 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:09 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:09.113+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:13.794432+0000 front 2024-05-10T11:46:13.794449+0000 (oldest deadline 2024-05-10T11:46:37.294473+0000) 2024-05-10T11:47:09.503 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:09 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:09.113+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:21.995604+0000 front 2024-05-10T11:46:21.995643+0000 (oldest deadline 2024-05-10T11:46:46.695720+0000) 2024-05-10T11:47:09.503 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:09 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:09.113+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-10T11:46:26.696333+0000 front 2024-05-10T11:46:26.696307+0000 (oldest deadline 2024-05-10T11:46:51.996382+0000) 2024-05-10T11:47:09.503 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:09 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:09.113+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6822 osd.6 since back 2024-05-10T11:46:36.097215+0000 front 2024-05-10T11:46:36.097243+0000 (oldest deadline 2024-05-10T11:47:01.397239+0000) 2024-05-10T11:47:10.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:09 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:09.898+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.703618+0000 front 2024-05-10T11:45:45.703296+0000 (oldest deadline 2024-05-10T11:46:11.003467+0000) 2024-05-10T11:47:10.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:09 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:09.898+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:58.005499+0000 front 2024-05-10T11:45:58.005443+0000 (oldest deadline 2024-05-10T11:46:22.705582+0000) 2024-05-10T11:47:10.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:09 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:09.898+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:05.606931+0000 front 2024-05-10T11:46:05.606957+0000 (oldest deadline 2024-05-10T11:46:30.907138+0000) 2024-05-10T11:47:10.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:09 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:09.898+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:20.110322+0000 front 2024-05-10T11:46:20.110411+0000 (oldest deadline 2024-05-10T11:46:43.610132+0000) 2024-05-10T11:47:10.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:09 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:09.898+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-10T11:46:28.311217+0000 front 2024-05-10T11:46:28.311184+0000 (oldest deadline 2024-05-10T11:46:52.411332+0000) 2024-05-10T11:47:10.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:09 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:09.898+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6822 osd.6 since back 2024-05-10T11:46:34.012719+0000 front 2024-05-10T11:46:34.012678+0000 (oldest deadline 2024-05-10T11:47:00.413058+0000) 2024-05-10T11:47:10.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:09 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:09.898+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6820 osd.7 since back 2024-05-10T11:46:46.814517+0000 front 2024-05-10T11:46:46.814639+0000 (oldest deadline 2024-05-10T11:47:08.414926+0000) 2024-05-10T11:47:10.404 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:10 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:10.141+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:44.388044+0000 front 2024-05-10T11:45:44.388108+0000 (oldest deadline 2024-05-10T11:46:10.288056+0000) 2024-05-10T11:47:10.404 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:10 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:10.141+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.190342+0000 front 2024-05-10T11:45:57.190423+0000 (oldest deadline 2024-05-10T11:46:19.490457+0000) 2024-05-10T11:47:10.404 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:10 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:10.141+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.691618+0000 front 2024-05-10T11:46:01.691702+0000 (oldest deadline 2024-05-10T11:46:26.892345+0000) 2024-05-10T11:47:10.404 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:10 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:10.141+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:13.794432+0000 front 2024-05-10T11:46:13.794449+0000 (oldest deadline 2024-05-10T11:46:37.294473+0000) 2024-05-10T11:47:10.404 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:10 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:10.141+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:21.995604+0000 front 2024-05-10T11:46:21.995643+0000 (oldest deadline 2024-05-10T11:46:46.695720+0000) 2024-05-10T11:47:10.404 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:10 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:10.141+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-10T11:46:26.696333+0000 front 2024-05-10T11:46:26.696307+0000 (oldest deadline 2024-05-10T11:46:51.996382+0000) 2024-05-10T11:47:10.404 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:10 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:10.141+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6822 osd.6 since back 2024-05-10T11:46:36.097215+0000 front 2024-05-10T11:46:36.097243+0000 (oldest deadline 2024-05-10T11:47:01.397239+0000) 2024-05-10T11:47:11.278 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:10 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:10.875+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.703618+0000 front 2024-05-10T11:45:45.703296+0000 (oldest deadline 2024-05-10T11:46:11.003467+0000) 2024-05-10T11:47:11.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:10 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:10.875+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:58.005499+0000 front 2024-05-10T11:45:58.005443+0000 (oldest deadline 2024-05-10T11:46:22.705582+0000) 2024-05-10T11:47:11.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:10 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:10.875+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:05.606931+0000 front 2024-05-10T11:46:05.606957+0000 (oldest deadline 2024-05-10T11:46:30.907138+0000) 2024-05-10T11:47:11.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:10 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:10.875+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:20.110322+0000 front 2024-05-10T11:46:20.110411+0000 (oldest deadline 2024-05-10T11:46:43.610132+0000) 2024-05-10T11:47:11.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:10 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:10.875+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-10T11:46:28.311217+0000 front 2024-05-10T11:46:28.311184+0000 (oldest deadline 2024-05-10T11:46:52.411332+0000) 2024-05-10T11:47:11.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:10 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:10.875+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6822 osd.6 since back 2024-05-10T11:46:34.012719+0000 front 2024-05-10T11:46:34.012678+0000 (oldest deadline 2024-05-10T11:47:00.413058+0000) 2024-05-10T11:47:11.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:10 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:10.875+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6820 osd.7 since back 2024-05-10T11:46:46.814517+0000 front 2024-05-10T11:46:46.814639+0000 (oldest deadline 2024-05-10T11:47:08.414926+0000) 2024-05-10T11:47:11.502 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:11 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:11.182+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:44.388044+0000 front 2024-05-10T11:45:44.388108+0000 (oldest deadline 2024-05-10T11:46:10.288056+0000) 2024-05-10T11:47:11.503 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:11 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:11.182+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:57.190342+0000 front 2024-05-10T11:45:57.190423+0000 (oldest deadline 2024-05-10T11:46:19.490457+0000) 2024-05-10T11:47:11.503 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:11 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:11.182+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:01.691618+0000 front 2024-05-10T11:46:01.691702+0000 (oldest deadline 2024-05-10T11:46:26.892345+0000) 2024-05-10T11:47:11.503 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:11 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:11.182+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6814 osd.3 since back 2024-05-10T11:46:13.794432+0000 front 2024-05-10T11:46:13.794449+0000 (oldest deadline 2024-05-10T11:46:37.294473+0000) 2024-05-10T11:47:11.503 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:11 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:11.182+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:21.995604+0000 front 2024-05-10T11:46:21.995643+0000 (oldest deadline 2024-05-10T11:46:46.695720+0000) 2024-05-10T11:47:11.503 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:11 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:11.182+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-10T11:46:26.696333+0000 front 2024-05-10T11:46:26.696307+0000 (oldest deadline 2024-05-10T11:46:51.996382+0000) 2024-05-10T11:47:11.503 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:11 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10[58743]: 2024-05-10T11:47:11.182+0000 7fbcded7c700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.100:6822 osd.6 since back 2024-05-10T11:46:36.097215+0000 front 2024-05-10T11:46:36.097243+0000 (oldest deadline 2024-05-10T11:47:01.397239+0000) 2024-05-10T11:47:11.931 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:11 smithi178 podman[77270]: 2024-05-10 11:47:11.627806525 +0000 UTC m=+5.553152164 container died e1ed9b9a15fbca00d54df715377bbb1f01c9a44d1cc914b05bab0b7e65bef023 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.vendor=CentOS, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, io.buildah.version=1.33.5) 2024-05-10T11:47:11.931 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:11 smithi178 podman[77270]: 2024-05-10 11:47:11.93174288 +0000 UTC m=+5.857088518 container cleanup e1ed9b9a15fbca00d54df715377bbb1f01c9a44d1cc914b05bab0b7e65bef023 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10, org.label-schema.license=GPLv2, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-05-10T11:47:12.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:11 smithi178 bash[77270]: ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10 2024-05-10T11:47:12.252 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:12 smithi178 podman[77299]: 2024-05-10 11:47:12.141322395 +0000 UTC m=+0.504015714 container remove e1ed9b9a15fbca00d54df715377bbb1f01c9a44d1cc914b05bab0b7e65bef023 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e) 2024-05-10T11:47:12.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:11 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:11.829+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.703618+0000 front 2024-05-10T11:45:45.703296+0000 (oldest deadline 2024-05-10T11:46:11.003467+0000) 2024-05-10T11:47:12.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:11 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:11.829+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:58.005499+0000 front 2024-05-10T11:45:58.005443+0000 (oldest deadline 2024-05-10T11:46:22.705582+0000) 2024-05-10T11:47:12.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:11 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:11.829+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:05.606931+0000 front 2024-05-10T11:46:05.606957+0000 (oldest deadline 2024-05-10T11:46:30.907138+0000) 2024-05-10T11:47:12.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:11 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:11.829+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:20.110322+0000 front 2024-05-10T11:46:20.110411+0000 (oldest deadline 2024-05-10T11:46:43.610132+0000) 2024-05-10T11:47:12.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:11 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:11.829+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-10T11:46:28.311217+0000 front 2024-05-10T11:46:28.311184+0000 (oldest deadline 2024-05-10T11:46:52.411332+0000) 2024-05-10T11:47:12.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:11 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:11.829+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6822 osd.6 since back 2024-05-10T11:46:34.012719+0000 front 2024-05-10T11:46:34.012678+0000 (oldest deadline 2024-05-10T11:47:00.413058+0000) 2024-05-10T11:47:12.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:11 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:11.829+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6820 osd.7 since back 2024-05-10T11:46:46.814517+0000 front 2024-05-10T11:46:46.814639+0000 (oldest deadline 2024-05-10T11:47:08.414926+0000) 2024-05-10T11:47:12.702 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:12 smithi178 podman[77390]: 2024-05-10 11:47:12.44201647 +0000 UTC m=+0.103835532 container create 43a52f727082a030732cb7412f87db2a04af8c6fa0551c0c2a616a5a5fe5a2e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, io.buildah.version=1.33.5, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.license=GPLv2) 2024-05-10T11:47:12.702 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:12 smithi178 podman[77390]: 2024-05-10 11:47:12.356311885 +0000 UTC m=+0.018130949 image pull ee5e4f21ef58047cb638cccd5cfbe8f6f43412b56c94b1d7fbd9cf3cb90a53a5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f 2024-05-10T11:47:12.702 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:12 smithi178 podman[77390]: 2024-05-10 11:47:12.619211172 +0000 UTC m=+0.281030239 container init 43a52f727082a030732cb7412f87db2a04af8c6fa0551c0c2a616a5a5fe5a2e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-10T11:47:12.702 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:12 smithi178 podman[77390]: 2024-05-10 11:47:12.622397446 +0000 UTC m=+0.284216514 container start 43a52f727082a030732cb7412f87db2a04af8c6fa0551c0c2a616a5a5fe5a2e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True) 2024-05-10T11:47:12.702 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:12 smithi178 podman[77390]: 2024-05-10 11:47:12.702463764 +0000 UTC m=+0.364282827 container attach 43a52f727082a030732cb7412f87db2a04af8c6fa0551c0c2a616a5a5fe5a2e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e) 2024-05-10T11:47:13.002 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:12 smithi178 podman[77390]: 2024-05-10 11:47:12.909530713 +0000 UTC m=+0.571349766 container died 43a52f727082a030732cb7412f87db2a04af8c6fa0551c0c2a616a5a5fe5a2e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_CLEAN=True, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-05-10T11:47:13.235 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:12 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:12.813+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.703618+0000 front 2024-05-10T11:45:45.703296+0000 (oldest deadline 2024-05-10T11:46:11.003467+0000) 2024-05-10T11:47:13.235 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:12 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:12.813+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:58.005499+0000 front 2024-05-10T11:45:58.005443+0000 (oldest deadline 2024-05-10T11:46:22.705582+0000) 2024-05-10T11:47:13.235 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:12 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:12.813+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:05.606931+0000 front 2024-05-10T11:46:05.606957+0000 (oldest deadline 2024-05-10T11:46:30.907138+0000) 2024-05-10T11:47:13.235 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:12 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:12.813+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:20.110322+0000 front 2024-05-10T11:46:20.110411+0000 (oldest deadline 2024-05-10T11:46:43.610132+0000) 2024-05-10T11:47:13.236 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:12 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:12.813+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-10T11:46:28.311217+0000 front 2024-05-10T11:46:28.311184+0000 (oldest deadline 2024-05-10T11:46:52.411332+0000) 2024-05-10T11:47:13.236 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:12 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:12.813+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6822 osd.6 since back 2024-05-10T11:46:34.012719+0000 front 2024-05-10T11:46:34.012678+0000 (oldest deadline 2024-05-10T11:47:00.413058+0000) 2024-05-10T11:47:13.236 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:12 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:12.813+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6820 osd.7 since back 2024-05-10T11:46:46.814517+0000 front 2024-05-10T11:46:46.814639+0000 (oldest deadline 2024-05-10T11:47:08.414926+0000) 2024-05-10T11:47:13.463 INFO:journalctl@ceph.osd.10.smithi178.stdout:May 10 11:47:13 smithi178 podman[77416]: 2024-05-10 11:47:13.447149214 +0000 UTC m=+0.532021036 container remove 43a52f727082a030732cb7412f87db2a04af8c6fa0551c0c2a616a5a5fe5a2e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-10-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-10T11:47:13.469 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@osd.10.service' 2024-05-10T11:47:13.509 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T11:47:13.509 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-05-10T11:47:13.509 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-05-10T11:47:13.509 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@osd.11 2024-05-10T11:47:13.824 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:13 smithi188 systemd[1]: Stopping Ceph osd.11 for 20a542cc-0ebd-11ef-bc98-c7b262605968... 2024-05-10T11:47:14.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:13 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:13.824+0000 7f872f35e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T11:47:14.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:13 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:13.824+0000 7f872f35e700 -1 osd.11 94 *** Got signal Terminated *** 2024-05-10T11:47:14.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:13 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:13.824+0000 7f872f35e700 -1 osd.11 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T11:47:14.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:13 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:13.825+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.703618+0000 front 2024-05-10T11:45:45.703296+0000 (oldest deadline 2024-05-10T11:46:11.003467+0000) 2024-05-10T11:47:14.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:13 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:13.825+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:58.005499+0000 front 2024-05-10T11:45:58.005443+0000 (oldest deadline 2024-05-10T11:46:22.705582+0000) 2024-05-10T11:47:14.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:13 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:13.825+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:05.606931+0000 front 2024-05-10T11:46:05.606957+0000 (oldest deadline 2024-05-10T11:46:30.907138+0000) 2024-05-10T11:47:14.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:13 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:13.825+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:20.110322+0000 front 2024-05-10T11:46:20.110411+0000 (oldest deadline 2024-05-10T11:46:43.610132+0000) 2024-05-10T11:47:14.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:13 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:13.825+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-10T11:46:28.311217+0000 front 2024-05-10T11:46:28.311184+0000 (oldest deadline 2024-05-10T11:46:52.411332+0000) 2024-05-10T11:47:14.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:13 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:13.825+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6822 osd.6 since back 2024-05-10T11:46:34.012719+0000 front 2024-05-10T11:46:34.012678+0000 (oldest deadline 2024-05-10T11:47:00.413058+0000) 2024-05-10T11:47:14.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:13 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:13.825+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6820 osd.7 since back 2024-05-10T11:46:46.814517+0000 front 2024-05-10T11:46:46.814639+0000 (oldest deadline 2024-05-10T11:47:08.414926+0000) 2024-05-10T11:47:15.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:14 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:14.862+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.703618+0000 front 2024-05-10T11:45:45.703296+0000 (oldest deadline 2024-05-10T11:46:11.003467+0000) 2024-05-10T11:47:15.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:14 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:14.862+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:58.005499+0000 front 2024-05-10T11:45:58.005443+0000 (oldest deadline 2024-05-10T11:46:22.705582+0000) 2024-05-10T11:47:15.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:14 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:14.862+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:05.606931+0000 front 2024-05-10T11:46:05.606957+0000 (oldest deadline 2024-05-10T11:46:30.907138+0000) 2024-05-10T11:47:15.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:14 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:14.862+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:20.110322+0000 front 2024-05-10T11:46:20.110411+0000 (oldest deadline 2024-05-10T11:46:43.610132+0000) 2024-05-10T11:47:15.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:14 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:14.862+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-10T11:46:28.311217+0000 front 2024-05-10T11:46:28.311184+0000 (oldest deadline 2024-05-10T11:46:52.411332+0000) 2024-05-10T11:47:15.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:14 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:14.862+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6822 osd.6 since back 2024-05-10T11:46:34.012719+0000 front 2024-05-10T11:46:34.012678+0000 (oldest deadline 2024-05-10T11:47:00.413058+0000) 2024-05-10T11:47:15.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:14 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:14.862+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6820 osd.7 since back 2024-05-10T11:46:46.814517+0000 front 2024-05-10T11:46:46.814639+0000 (oldest deadline 2024-05-10T11:47:08.414926+0000) 2024-05-10T11:47:16.283 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:15 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:15.893+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.703618+0000 front 2024-05-10T11:45:45.703296+0000 (oldest deadline 2024-05-10T11:46:11.003467+0000) 2024-05-10T11:47:16.283 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:15 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:15.893+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:58.005499+0000 front 2024-05-10T11:45:58.005443+0000 (oldest deadline 2024-05-10T11:46:22.705582+0000) 2024-05-10T11:47:16.283 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:15 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:15.893+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:05.606931+0000 front 2024-05-10T11:46:05.606957+0000 (oldest deadline 2024-05-10T11:46:30.907138+0000) 2024-05-10T11:47:16.283 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:15 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:15.893+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:20.110322+0000 front 2024-05-10T11:46:20.110411+0000 (oldest deadline 2024-05-10T11:46:43.610132+0000) 2024-05-10T11:47:16.283 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:15 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:15.893+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-10T11:46:28.311217+0000 front 2024-05-10T11:46:28.311184+0000 (oldest deadline 2024-05-10T11:46:52.411332+0000) 2024-05-10T11:47:16.283 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:15 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:15.893+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6822 osd.6 since back 2024-05-10T11:46:34.012719+0000 front 2024-05-10T11:46:34.012678+0000 (oldest deadline 2024-05-10T11:47:00.413058+0000) 2024-05-10T11:47:16.283 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:15 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:15.893+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6820 osd.7 since back 2024-05-10T11:46:46.814517+0000 front 2024-05-10T11:46:46.814639+0000 (oldest deadline 2024-05-10T11:47:08.414926+0000) 2024-05-10T11:47:17.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:16 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:16.898+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.703618+0000 front 2024-05-10T11:45:45.703296+0000 (oldest deadline 2024-05-10T11:46:11.003467+0000) 2024-05-10T11:47:17.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:16 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:16.898+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:58.005499+0000 front 2024-05-10T11:45:58.005443+0000 (oldest deadline 2024-05-10T11:46:22.705582+0000) 2024-05-10T11:47:17.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:16 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:16.898+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:05.606931+0000 front 2024-05-10T11:46:05.606957+0000 (oldest deadline 2024-05-10T11:46:30.907138+0000) 2024-05-10T11:47:17.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:16 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:16.898+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:20.110322+0000 front 2024-05-10T11:46:20.110411+0000 (oldest deadline 2024-05-10T11:46:43.610132+0000) 2024-05-10T11:47:17.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:16 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:16.898+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-10T11:46:28.311217+0000 front 2024-05-10T11:46:28.311184+0000 (oldest deadline 2024-05-10T11:46:52.411332+0000) 2024-05-10T11:47:17.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:16 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:16.898+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6822 osd.6 since back 2024-05-10T11:46:34.012719+0000 front 2024-05-10T11:46:34.012678+0000 (oldest deadline 2024-05-10T11:47:00.413058+0000) 2024-05-10T11:47:17.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:16 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:16.898+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6820 osd.7 since back 2024-05-10T11:46:46.814517+0000 front 2024-05-10T11:46:46.814639+0000 (oldest deadline 2024-05-10T11:47:08.414926+0000) 2024-05-10T11:47:17.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:16 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:16.898+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6820 osd.8 since back 2024-05-10T11:46:54.816262+0000 front 2024-05-10T11:46:54.816306+0000 (oldest deadline 2024-05-10T11:47:16.516317+0000) 2024-05-10T11:47:18.278 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:17 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:17.856+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-05-10T11:45:45.703618+0000 front 2024-05-10T11:45:45.703296+0000 (oldest deadline 2024-05-10T11:46:11.003467+0000) 2024-05-10T11:47:18.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:17 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:17.857+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2024-05-10T11:45:58.005499+0000 front 2024-05-10T11:45:58.005443+0000 (oldest deadline 2024-05-10T11:46:22.705582+0000) 2024-05-10T11:47:18.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:17 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:17.857+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-10T11:46:05.606931+0000 front 2024-05-10T11:46:05.606957+0000 (oldest deadline 2024-05-10T11:46:30.907138+0000) 2024-05-10T11:47:18.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:17 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:17.857+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6812 osd.4 since back 2024-05-10T11:46:20.110322+0000 front 2024-05-10T11:46:20.110411+0000 (oldest deadline 2024-05-10T11:46:43.610132+0000) 2024-05-10T11:47:18.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:17 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:17.857+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-10T11:46:28.311217+0000 front 2024-05-10T11:46:28.311184+0000 (oldest deadline 2024-05-10T11:46:52.411332+0000) 2024-05-10T11:47:18.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:17 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:17.857+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.100:6822 osd.6 since back 2024-05-10T11:46:34.012719+0000 front 2024-05-10T11:46:34.012678+0000 (oldest deadline 2024-05-10T11:47:00.413058+0000) 2024-05-10T11:47:18.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:17 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:17.857+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.178:6820 osd.7 since back 2024-05-10T11:46:46.814517+0000 front 2024-05-10T11:46:46.814639+0000 (oldest deadline 2024-05-10T11:47:08.414926+0000) 2024-05-10T11:47:18.279 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:17 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11[58809]: 2024-05-10T11:47:17.857+0000 7f872794a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.188:6820 osd.8 since back 2024-05-10T11:46:54.816262+0000 front 2024-05-10T11:46:54.816306+0000 (oldest deadline 2024-05-10T11:47:16.516317+0000) 2024-05-10T11:47:19.529 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:19 smithi188 podman[76935]: 2024-05-10 11:47:19.251428713 +0000 UTC m=+5.557122366 container died 3bba8eb71c40b550ae6fb72741c7561ab1940f6426c70f3ebc3483defd1b2b16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.build-date=20240507, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_CLEAN=True) 2024-05-10T11:47:20.028 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:19 smithi188 podman[76935]: 2024-05-10 11:47:19.573825978 +0000 UTC m=+5.879519618 container cleanup 3bba8eb71c40b550ae6fb72741c7561ab1940f6426c70f3ebc3483defd1b2b16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507) 2024-05-10T11:47:20.029 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:19 smithi188 bash[76935]: ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11 2024-05-10T11:47:20.029 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:19 smithi188 podman[76953]: 2024-05-10 11:47:19.791604955 +0000 UTC m=+0.536668001 container remove 3bba8eb71c40b550ae6fb72741c7561ab1940f6426c70f3ebc3483defd1b2b16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08) 2024-05-10T11:47:20.519 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:20 smithi188 podman[77046]: 2024-05-10 11:47:20.104486119 +0000 UTC m=+0.018351369 image pull ee5e4f21ef58047cb638cccd5cfbe8f6f43412b56c94b1d7fbd9cf3cb90a53a5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f 2024-05-10T11:47:20.520 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:20 smithi188 podman[77046]: 2024-05-10 11:47:20.209204936 +0000 UTC m=+0.123070177 container create c9c9fa10018a266b0665d08327ca52d7182d08fe3c9667864f7fa45c73f6dc6a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-05-10T11:47:20.520 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:20 smithi188 podman[77046]: 2024-05-10 11:47:20.436694806 +0000 UTC m=+0.350560046 container init c9c9fa10018a266b0665d08327ca52d7182d08fe3c9667864f7fa45c73f6dc6a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.schema-version=1.0) 2024-05-10T11:47:20.520 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:20 smithi188 podman[77046]: 2024-05-10 11:47:20.440025432 +0000 UTC m=+0.353890680 container start c9c9fa10018a266b0665d08327ca52d7182d08fe3c9667864f7fa45c73f6dc6a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507) 2024-05-10T11:47:20.778 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:20 smithi188 podman[77046]: 2024-05-10 11:47:20.519944713 +0000 UTC m=+0.433809973 container attach c9c9fa10018a266b0665d08327ca52d7182d08fe3c9667864f7fa45c73f6dc6a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-10T11:47:20.782 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 10 11:47:20 smithi188 podman[77046]: 2024-05-10 11:47:20.742766172 +0000 UTC m=+0.656631405 container died c9c9fa10018a266b0665d08327ca52d7182d08fe3c9667864f7fa45c73f6dc6a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-osd-11-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-05-10T11:47:21.317 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@osd.11.service' 2024-05-10T11:47:21.354 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T11:47:21.354 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-05-10T11:47:21.354 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-05-10T11:47:21.354 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mds.a 2024-05-10T11:47:21.649 INFO:journalctl@ceph.mds.a.smithi100.stdout:May 10 11:47:21 smithi100 systemd[1]: Stopping Ceph mds.a for 20a542cc-0ebd-11ef-bc98-c7b262605968... 2024-05-10T11:47:22.038 INFO:journalctl@ceph.mds.a.smithi100.stdout:May 10 11:47:21 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-a[69827]: 2024-05-10T11:47:21.648+0000 7f9a4b9f2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T11:47:22.038 INFO:journalctl@ceph.mds.a.smithi100.stdout:May 10 11:47:21 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-a[69827]: 2024-05-10T11:47:21.648+0000 7f9a4b9f2700 -1 mds.a *** got signal Terminated *** 2024-05-10T11:47:23.966 INFO:journalctl@ceph.mds.a.smithi100.stdout:May 10 11:47:23 smithi100 podman[141651]: 2024-05-10 11:47:23.686670936 +0000 UTC m=+2.139963668 container died e7edb955eb9705efb9011e0c3a3c21258d85efef0780dcaa146c6253fc046a7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD) 2024-05-10T11:47:24.288 INFO:journalctl@ceph.mds.a.smithi100.stdout:May 10 11:47:24 smithi100 podman[141651]: 2024-05-10 11:47:24.141365588 +0000 UTC m=+2.594658351 container cleanup e7edb955eb9705efb9011e0c3a3c21258d85efef0780dcaa146c6253fc046a7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-a, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-05-10T11:47:24.288 INFO:journalctl@ceph.mds.a.smithi100.stdout:May 10 11:47:24 smithi100 bash[141651]: ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-a 2024-05-10T11:47:24.571 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mds.a.service' 2024-05-10T11:47:24.608 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T11:47:24.608 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-05-10T11:47:24.609 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-05-10T11:47:24.609 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mds.d 2024-05-10T11:47:24.937 INFO:journalctl@ceph.mds.d.smithi100.stdout:May 10 11:47:24 smithi100 systemd[1]: Stopping Ceph mds.d for 20a542cc-0ebd-11ef-bc98-c7b262605968... 2024-05-10T11:47:25.288 INFO:journalctl@ceph.mds.d.smithi100.stdout:May 10 11:47:24 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-d[68554]: 2024-05-10T11:47:24.937+0000 7f6550519700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T11:47:25.288 INFO:journalctl@ceph.mds.d.smithi100.stdout:May 10 11:47:24 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-d[68554]: 2024-05-10T11:47:24.937+0000 7f6550519700 -1 mds.d *** got signal Terminated *** 2024-05-10T11:47:29.229 INFO:journalctl@ceph.mds.d.smithi100.stdout:May 10 11:47:28 smithi100 podman[141783]: 2024-05-10 11:47:28.961958585 +0000 UTC m=+4.128671219 container died 6bdce29d310a345ccbc028835efff3a2c069e22d88c609a294a9b35a4ed5ccd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-05-10T11:47:29.538 INFO:journalctl@ceph.mds.d.smithi100.stdout:May 10 11:47:29 smithi100 podman[141783]: 2024-05-10 11:47:29.328370109 +0000 UTC m=+4.495082740 container cleanup 6bdce29d310a345ccbc028835efff3a2c069e22d88c609a294a9b35a4ed5ccd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-d, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08) 2024-05-10T11:47:29.538 INFO:journalctl@ceph.mds.d.smithi100.stdout:May 10 11:47:29 smithi100 bash[141783]: ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-d 2024-05-10T11:47:29.708 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mds.d.service' 2024-05-10T11:47:29.742 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T11:47:29.742 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-05-10T11:47:29.742 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-05-10T11:47:29.742 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mds.g 2024-05-10T11:47:30.288 INFO:journalctl@ceph.mds.g.smithi100.stdout:May 10 11:47:29 smithi100 systemd[1]: Stopping Ceph mds.g for 20a542cc-0ebd-11ef-bc98-c7b262605968... 2024-05-10T11:47:30.289 INFO:journalctl@ceph.mds.g.smithi100.stdout:May 10 11:47:30 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-g[69359]: 2024-05-10T11:47:30.073+0000 7f9f97d67700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T11:47:30.289 INFO:journalctl@ceph.mds.g.smithi100.stdout:May 10 11:47:30 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-g[69359]: 2024-05-10T11:47:30.073+0000 7f9f97d67700 -1 mds.g *** got signal Terminated *** 2024-05-10T11:47:34.391 INFO:journalctl@ceph.mds.g.smithi100.stdout:May 10 11:47:34 smithi100 podman[141905]: 2024-05-10 11:47:34.080312997 +0000 UTC m=+4.109975145 container died b9c096f696f7e84ad69bf1dbb8b1600e6bc4880e08d747d4bb4a10a56e14b59d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-g, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-05-10T11:47:34.691 INFO:journalctl@ceph.mds.g.smithi100.stdout:May 10 11:47:34 smithi100 podman[141905]: 2024-05-10 11:47:34.523520729 +0000 UTC m=+4.553182880 container cleanup b9c096f696f7e84ad69bf1dbb8b1600e6bc4880e08d747d4bb4a10a56e14b59d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-g, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, ceph=True) 2024-05-10T11:47:34.691 INFO:journalctl@ceph.mds.g.smithi100.stdout:May 10 11:47:34 smithi100 bash[141905]: ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-g 2024-05-10T11:47:34.917 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mds.g.service' 2024-05-10T11:47:34.949 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T11:47:34.950 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-05-10T11:47:34.950 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-05-10T11:47:34.950 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mds.j 2024-05-10T11:47:35.277 INFO:journalctl@ceph.mds.j.smithi100.stdout:May 10 11:47:35 smithi100 systemd[1]: Stopping Ceph mds.j for 20a542cc-0ebd-11ef-bc98-c7b262605968... 2024-05-10T11:47:35.538 INFO:journalctl@ceph.mds.j.smithi100.stdout:May 10 11:47:35 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-j[68109]: 2024-05-10T11:47:35.277+0000 7fd998c9d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T11:47:35.538 INFO:journalctl@ceph.mds.j.smithi100.stdout:May 10 11:47:35 smithi100 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-j[68109]: 2024-05-10T11:47:35.277+0000 7fd998c9d700 -1 mds.j *** got signal Terminated *** 2024-05-10T11:47:37.418 INFO:journalctl@ceph.mds.j.smithi100.stdout:May 10 11:47:37 smithi100 podman[142025]: 2024-05-10 11:47:37.148120761 +0000 UTC m=+1.974516013 container died cfaf0ab1a1056904a0b34e7fe9dbad07a7e1b65b5cdf8f41d016b2b57c0003ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-j, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-05-10T11:47:37.682 INFO:journalctl@ceph.mds.j.smithi100.stdout:May 10 11:47:37 smithi100 podman[142025]: 2024-05-10 11:47:37.502124838 +0000 UTC m=+2.328520098 container cleanup cfaf0ab1a1056904a0b34e7fe9dbad07a7e1b65b5cdf8f41d016b2b57c0003ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-j, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.build-date=20240507, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-05-10T11:47:37.682 INFO:journalctl@ceph.mds.j.smithi100.stdout:May 10 11:47:37 smithi100 bash[142025]: ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-j 2024-05-10T11:47:37.880 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mds.j.service' 2024-05-10T11:47:37.913 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T11:47:37.913 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-05-10T11:47:37.913 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-05-10T11:47:37.913 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mds.b 2024-05-10T11:47:38.221 INFO:journalctl@ceph.mds.b.smithi178.stdout:May 10 11:47:37 smithi178 systemd[1]: Stopping Ceph mds.b for 20a542cc-0ebd-11ef-bc98-c7b262605968... 2024-05-10T11:47:38.502 INFO:journalctl@ceph.mds.b.smithi178.stdout:May 10 11:47:38 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-b[63923]: 2024-05-10T11:47:38.221+0000 7f6b008d0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T11:47:38.502 INFO:journalctl@ceph.mds.b.smithi178.stdout:May 10 11:47:38 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-b[63923]: 2024-05-10T11:47:38.221+0000 7f6b008d0700 -1 mds.b *** got signal Terminated *** 2024-05-10T11:47:40.252 INFO:journalctl@ceph.mds.b.smithi178.stdout:May 10 11:47:39 smithi178 podman[77494]: 2024-05-10 11:47:39.912011877 +0000 UTC m=+1.804720113 container died e8a32cd93f44fff212080401c21b906de675c13d364b63ebe7c01fae96f9f285 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-b, org.label-schema.build-date=20240507, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_BRANCH=HEAD, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS) 2024-05-10T11:47:40.572 INFO:journalctl@ceph.mds.b.smithi178.stdout:May 10 11:47:40 smithi178 podman[77494]: 2024-05-10 11:47:40.354221597 +0000 UTC m=+2.246929825 container cleanup e8a32cd93f44fff212080401c21b906de675c13d364b63ebe7c01fae96f9f285 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-05-10T11:47:40.572 INFO:journalctl@ceph.mds.b.smithi178.stdout:May 10 11:47:40 smithi178 bash[77494]: ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-b 2024-05-10T11:47:40.822 INFO:journalctl@ceph.mds.b.smithi178.stdout:May 10 11:47:40 smithi178 podman[77508]: 2024-05-10 11:47:40.572019802 +0000 UTC m=+0.661760818 container remove e8a32cd93f44fff212080401c21b906de675c13d364b63ebe7c01fae96f9f285 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-b, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08) 2024-05-10T11:47:40.822 INFO:journalctl@ceph.mds.b.smithi178.stdout:May 10 11:47:40 smithi178 systemd[1]: ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mds.b.service: Deactivated successfully. 2024-05-10T11:47:40.822 INFO:journalctl@ceph.mds.b.smithi178.stdout:May 10 11:47:40 smithi178 systemd[1]: Stopped Ceph mds.b for 20a542cc-0ebd-11ef-bc98-c7b262605968. 2024-05-10T11:47:40.822 INFO:journalctl@ceph.mds.b.smithi178.stdout:May 10 11:47:40 smithi178 systemd[1]: ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mds.b.service: Consumed 13min 37.827s CPU time. 2024-05-10T11:47:40.823 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mds.b.service' 2024-05-10T11:47:40.857 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T11:47:40.857 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-05-10T11:47:40.857 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-05-10T11:47:40.858 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mds.e 2024-05-10T11:47:41.174 INFO:journalctl@ceph.mds.e.smithi178.stdout:May 10 11:47:40 smithi178 systemd[1]: Stopping Ceph mds.e for 20a542cc-0ebd-11ef-bc98-c7b262605968... 2024-05-10T11:47:41.502 INFO:journalctl@ceph.mds.e.smithi178.stdout:May 10 11:47:41 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-e[63314]: 2024-05-10T11:47:41.173+0000 7f0a8c200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T11:47:41.502 INFO:journalctl@ceph.mds.e.smithi178.stdout:May 10 11:47:41 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-e[63314]: 2024-05-10T11:47:41.173+0000 7f0a8c200700 -1 mds.e *** got signal Terminated *** 2024-05-10T11:47:44.198 INFO:journalctl@ceph.mds.e.smithi178.stdout:May 10 11:47:43 smithi178 podman[77614]: 2024-05-10 11:47:43.902423162 +0000 UTC m=+2.832501882 container died b445c9f9eac521c55765b83e47ce5c952ff374ccaf60c4a76f699cd2cc2cf982 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-e, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-05-10T11:47:44.198 INFO:journalctl@ceph.mds.e.smithi178.stdout:May 10 11:47:44 smithi178 podman[77614]: 2024-05-10 11:47:44.199274583 +0000 UTC m=+3.129353313 container cleanup b445c9f9eac521c55765b83e47ce5c952ff374ccaf60c4a76f699cd2cc2cf982 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-e, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0) 2024-05-10T11:47:44.502 INFO:journalctl@ceph.mds.e.smithi178.stdout:May 10 11:47:44 smithi178 bash[77614]: ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-e 2024-05-10T11:47:44.503 INFO:journalctl@ceph.mds.e.smithi178.stdout:May 10 11:47:44 smithi178 podman[77639]: 2024-05-10 11:47:44.425409402 +0000 UTC m=+0.516594920 container remove b445c9f9eac521c55765b83e47ce5c952ff374ccaf60c4a76f699cd2cc2cf982 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-10T11:47:44.546 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mds.e.service' 2024-05-10T11:47:44.580 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T11:47:44.580 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-05-10T11:47:44.581 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-05-10T11:47:44.581 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mds.h 2024-05-10T11:47:44.903 INFO:journalctl@ceph.mds.h.smithi178.stdout:May 10 11:47:44 smithi178 systemd[1]: Stopping Ceph mds.h for 20a542cc-0ebd-11ef-bc98-c7b262605968... 2024-05-10T11:47:45.252 INFO:journalctl@ceph.mds.h.smithi178.stdout:May 10 11:47:44 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-h[63619]: 2024-05-10T11:47:44.903+0000 7f4e21936700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T11:47:45.252 INFO:journalctl@ceph.mds.h.smithi178.stdout:May 10 11:47:44 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-h[63619]: 2024-05-10T11:47:44.903+0000 7f4e21936700 -1 mds.h *** got signal Terminated *** 2024-05-10T11:47:48.211 INFO:journalctl@ceph.mds.h.smithi178.stdout:May 10 11:47:47 smithi178 podman[77745]: 2024-05-10 11:47:47.905566046 +0000 UTC m=+3.103706635 container died 6939a764e16d473fc77acbb004d30986f0a4e6fbb3bc4f271b653143fe097d8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-h, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507) 2024-05-10T11:47:48.502 INFO:journalctl@ceph.mds.h.smithi178.stdout:May 10 11:47:48 smithi178 podman[77745]: 2024-05-10 11:47:48.29410717 +0000 UTC m=+3.492247739 container cleanup 6939a764e16d473fc77acbb004d30986f0a4e6fbb3bc4f271b653143fe097d8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-h, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-10T11:47:48.502 INFO:journalctl@ceph.mds.h.smithi178.stdout:May 10 11:47:48 smithi178 bash[77745]: ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-h 2024-05-10T11:47:48.746 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mds.h.service' 2024-05-10T11:47:48.779 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T11:47:48.780 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-05-10T11:47:48.780 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-05-10T11:47:48.780 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mds.k 2024-05-10T11:47:49.002 INFO:journalctl@ceph.mds.k.smithi178.stdout:May 10 11:47:48 smithi178 systemd[1]: Stopping Ceph mds.k for 20a542cc-0ebd-11ef-bc98-c7b262605968... 2024-05-10T11:47:49.502 INFO:journalctl@ceph.mds.k.smithi178.stdout:May 10 11:47:49 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-k[62996]: 2024-05-10T11:47:49.096+0000 7f6b8d8a1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T11:47:49.502 INFO:journalctl@ceph.mds.k.smithi178.stdout:May 10 11:47:49 smithi178 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-k[62996]: 2024-05-10T11:47:49.096+0000 7f6b8d8a1700 -1 mds.k *** got signal Terminated *** 2024-05-10T11:47:52.630 INFO:journalctl@ceph.mds.k.smithi178.stdout:May 10 11:47:52 smithi178 podman[77865]: 2024-05-10 11:47:52.345631208 +0000 UTC m=+3.349541984 container died 348fd0498391e5b3e79ac16d3fcbc08d472a81825d9d5d0f2d7778d22281ed38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-k, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-10T11:47:52.889 INFO:journalctl@ceph.mds.k.smithi178.stdout:May 10 11:47:52 smithi178 podman[77865]: 2024-05-10 11:47:52.697182031 +0000 UTC m=+3.701092807 container cleanup 348fd0498391e5b3e79ac16d3fcbc08d472a81825d9d5d0f2d7778d22281ed38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-k, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240507, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-10T11:47:52.889 INFO:journalctl@ceph.mds.k.smithi178.stdout:May 10 11:47:52 smithi178 bash[77865]: ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-k 2024-05-10T11:47:53.128 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mds.k.service' 2024-05-10T11:47:53.149 INFO:journalctl@ceph.mds.k.smithi178.stdout:May 10 11:47:52 smithi178 podman[77879]: 2024-05-10 11:47:52.889794868 +0000 UTC m=+0.534489200 container remove 348fd0498391e5b3e79ac16d3fcbc08d472a81825d9d5d0f2d7778d22281ed38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-k, org.label-schema.schema-version=1.0, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, ceph=True, CEPH_POINT_RELEASE=) 2024-05-10T11:47:53.149 INFO:journalctl@ceph.mds.k.smithi178.stdout:May 10 11:47:53 smithi178 systemd[1]: ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mds.k.service: Deactivated successfully. 2024-05-10T11:47:53.149 INFO:journalctl@ceph.mds.k.smithi178.stdout:May 10 11:47:53 smithi178 systemd[1]: Stopped Ceph mds.k for 20a542cc-0ebd-11ef-bc98-c7b262605968. 2024-05-10T11:47:53.149 INFO:journalctl@ceph.mds.k.smithi178.stdout:May 10 11:47:53 smithi178 systemd[1]: ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mds.k.service: Consumed 1.217s CPU time. 2024-05-10T11:47:53.160 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T11:47:53.160 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-05-10T11:47:53.160 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-05-10T11:47:53.160 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mds.c 2024-05-10T11:47:53.529 INFO:journalctl@ceph.mds.c.smithi188.stdout:May 10 11:47:53 smithi188 systemd[1]: Stopping Ceph mds.c for 20a542cc-0ebd-11ef-bc98-c7b262605968... 2024-05-10T11:47:54.029 INFO:journalctl@ceph.mds.c.smithi188.stdout:May 10 11:47:53 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-c[64244]: 2024-05-10T11:47:53.580+0000 7f50ed66f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T11:47:54.029 INFO:journalctl@ceph.mds.c.smithi188.stdout:May 10 11:47:53 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-c[64244]: 2024-05-10T11:47:53.580+0000 7f50ed66f700 -1 mds.c *** got signal Terminated *** 2024-05-10T11:47:56.279 INFO:journalctl@ceph.mds.c.smithi188.stdout:May 10 11:47:55 smithi188 podman[77152]: 2024-05-10 11:47:55.914875624 +0000 UTC m=+2.560249998 container died ae08c30f24b6214ef74e834ae26ba3cf711e91f9984ebab7babb9f463c5e4b9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-c, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_BRANCH=HEAD) 2024-05-10T11:47:56.646 INFO:journalctl@ceph.mds.c.smithi188.stdout:May 10 11:47:56 smithi188 podman[77152]: 2024-05-10 11:47:56.378579248 +0000 UTC m=+3.023953622 container cleanup ae08c30f24b6214ef74e834ae26ba3cf711e91f9984ebab7babb9f463c5e4b9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-c, org.label-schema.build-date=20240507, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-10T11:47:56.646 INFO:journalctl@ceph.mds.c.smithi188.stdout:May 10 11:47:56 smithi188 bash[77152]: ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-c 2024-05-10T11:47:56.761 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mds.c.service' 2024-05-10T11:47:56.797 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T11:47:56.797 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-05-10T11:47:56.797 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-05-10T11:47:56.798 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mds.f 2024-05-10T11:47:57.028 INFO:journalctl@ceph.mds.f.smithi188.stdout:May 10 11:47:56 smithi188 systemd[1]: Stopping Ceph mds.f for 20a542cc-0ebd-11ef-bc98-c7b262605968... 2024-05-10T11:47:57.528 INFO:journalctl@ceph.mds.f.smithi188.stdout:May 10 11:47:57 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-f[63940]: 2024-05-10T11:47:57.115+0000 7f555f6fa700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T11:47:57.529 INFO:journalctl@ceph.mds.f.smithi188.stdout:May 10 11:47:57 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-f[63940]: 2024-05-10T11:47:57.115+0000 7f555f6fa700 -1 mds.f *** got signal Terminated *** 2024-05-10T11:48:00.174 INFO:journalctl@ceph.mds.f.smithi188.stdout:May 10 11:47:59 smithi188 podman[77282]: 2024-05-10 11:47:59.922743849 +0000 UTC m=+2.897936725 container died 172d7613ab043329f870e210e9e87ffb3cd7f44b242e16848ec6acc269ce6a32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-f, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507) 2024-05-10T11:48:00.528 INFO:journalctl@ceph.mds.f.smithi188.stdout:May 10 11:48:00 smithi188 podman[77282]: 2024-05-10 11:48:00.273694382 +0000 UTC m=+3.248887266 container cleanup 172d7613ab043329f870e210e9e87ffb3cd7f44b242e16848ec6acc269ce6a32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-f, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-05-10T11:48:00.529 INFO:journalctl@ceph.mds.f.smithi188.stdout:May 10 11:48:00 smithi188 bash[77282]: ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-f 2024-05-10T11:48:00.661 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mds.f.service' 2024-05-10T11:48:00.694 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T11:48:00.694 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-05-10T11:48:00.694 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-05-10T11:48:00.694 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mds.i 2024-05-10T11:48:01.019 INFO:journalctl@ceph.mds.i.smithi188.stdout:May 10 11:48:00 smithi188 systemd[1]: Stopping Ceph mds.i for 20a542cc-0ebd-11ef-bc98-c7b262605968... 2024-05-10T11:48:01.278 INFO:journalctl@ceph.mds.i.smithi188.stdout:May 10 11:48:01 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-i[64554]: 2024-05-10T11:48:01.018+0000 7fabe9f5a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T11:48:01.278 INFO:journalctl@ceph.mds.i.smithi188.stdout:May 10 11:48:01 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-i[64554]: 2024-05-10T11:48:01.018+0000 7fabe9f5a700 -1 mds.i *** got signal Terminated *** 2024-05-10T11:48:04.278 INFO:journalctl@ceph.mds.i.smithi188.stdout:May 10 11:48:03 smithi188 podman[77404]: 2024-05-10 11:48:03.91525877 +0000 UTC m=+3.002569404 container died 9d9fc5b815d34f96217acd961717fc89203a306fcaf730ddceff700ed14d3cdd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08) 2024-05-10T11:48:04.583 INFO:journalctl@ceph.mds.i.smithi188.stdout:May 10 11:48:04 smithi188 podman[77404]: 2024-05-10 11:48:04.360373374 +0000 UTC m=+3.447684016 container cleanup 9d9fc5b815d34f96217acd961717fc89203a306fcaf730ddceff700ed14d3cdd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-i, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, io.buildah.version=1.33.5) 2024-05-10T11:48:04.583 INFO:journalctl@ceph.mds.i.smithi188.stdout:May 10 11:48:04 smithi188 bash[77404]: ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-i 2024-05-10T11:48:04.823 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mds.i.service' 2024-05-10T11:48:04.844 INFO:journalctl@ceph.mds.i.smithi188.stdout:May 10 11:48:04 smithi188 podman[77417]: 2024-05-10 11:48:04.703428699 +0000 UTC m=+0.789596629 container remove 9d9fc5b815d34f96217acd961717fc89203a306fcaf730ddceff700ed14d3cdd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-i, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=) 2024-05-10T11:48:04.844 INFO:journalctl@ceph.mds.i.smithi188.stdout:May 10 11:48:04 smithi188 systemd[1]: ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mds.i.service: Deactivated successfully. 2024-05-10T11:48:04.844 INFO:journalctl@ceph.mds.i.smithi188.stdout:May 10 11:48:04 smithi188 systemd[1]: Stopped Ceph mds.i for 20a542cc-0ebd-11ef-bc98-c7b262605968. 2024-05-10T11:48:04.845 INFO:journalctl@ceph.mds.i.smithi188.stdout:May 10 11:48:04 smithi188 systemd[1]: ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mds.i.service: Consumed 22min 55.800s CPU time. 2024-05-10T11:48:04.855 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T11:48:04.855 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-05-10T11:48:04.856 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-05-10T11:48:04.856 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mds.l 2024-05-10T11:48:05.172 INFO:journalctl@ceph.mds.l.smithi188.stdout:May 10 11:48:04 smithi188 systemd[1]: Stopping Ceph mds.l for 20a542cc-0ebd-11ef-bc98-c7b262605968... 2024-05-10T11:48:05.528 INFO:journalctl@ceph.mds.l.smithi188.stdout:May 10 11:48:05 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-l[63617]: 2024-05-10T11:48:05.172+0000 7ff19554c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T11:48:05.528 INFO:journalctl@ceph.mds.l.smithi188.stdout:May 10 11:48:05 smithi188 ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-l[63617]: 2024-05-10T11:48:05.172+0000 7ff19554c700 -1 mds.l *** got signal Terminated *** 2024-05-10T11:48:09.529 INFO:journalctl@ceph.mds.l.smithi188.stdout:May 10 11:48:09 smithi188 podman[77527]: 2024-05-10 11:48:09.179818669 +0000 UTC m=+4.101426921 container died f411399636efe8355c2c672777fdab789ea85b8a377c4a410547d3d83cc28943 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-l, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-05-10T11:48:09.942 INFO:journalctl@ceph.mds.l.smithi188.stdout:May 10 11:48:09 smithi188 podman[77527]: 2024-05-10 11:48:09.659091466 +0000 UTC m=+4.580699746 container cleanup f411399636efe8355c2c672777fdab789ea85b8a377c4a410547d3d83cc28943 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c77785140a1aa226ee46fbc1bec1768a7bea4786cdf8c717e77f2b0ad3d7725f, name=ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-l, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-63896-from-wip-rishabh-testing-20240501.193033-d39fa08, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-05-10T11:48:09.943 INFO:journalctl@ceph.mds.l.smithi188.stdout:May 10 11:48:09 smithi188 bash[77527]: ceph-20a542cc-0ebd-11ef-bc98-c7b262605968-mds-l 2024-05-10T11:48:10.086 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-20a542cc-0ebd-11ef-bc98-c7b262605968@mds.l.service' 2024-05-10T11:48:10.118 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T11:48:10.118 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-05-10T11:48:10.118 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 --force --keep-logs 2024-05-10T11:48:10.312 INFO:teuthology.orchestra.run.smithi100.stdout:Deleting cluster with fsid: 20a542cc-0ebd-11ef-bc98-c7b262605968 2024-05-10T11:48:12.813 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 --force --keep-logs 2024-05-10T11:48:13.006 INFO:teuthology.orchestra.run.smithi178.stdout:Deleting cluster with fsid: 20a542cc-0ebd-11ef-bc98-c7b262605968 2024-05-10T11:48:15.562 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 --force --keep-logs 2024-05-10T11:48:15.753 INFO:teuthology.orchestra.run.smithi188.stdout:Deleting cluster with fsid: 20a542cc-0ebd-11ef-bc98-c7b262605968 2024-05-10T11:48:18.130 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-10T11:48:18.164 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-10T11:48:18.198 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-10T11:48:18.224 INFO:tasks.cephadm:Archiving crash dumps... 2024-05-10T11:48:18.227 DEBUG:teuthology.misc:Transferring archived files from smithi100:/var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/crash to /home/teuthworker/archive/dparmar-2024-05-10_07:38:09-fs-wip-63896-from-wip-rishabh-testing-20240501.193033-distro-default-smithi/7700928/remote/smithi100/crash 2024-05-10T11:48:18.228 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar c -f - -C /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/crash -- . 2024-05-10T11:48:18.263 INFO:teuthology.orchestra.run.smithi100.stderr:tar: /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/crash: Cannot open: No such file or directory 2024-05-10T11:48:18.264 INFO:teuthology.orchestra.run.smithi100.stderr:tar: Error is not recoverable: exiting now 2024-05-10T11:48:18.266 DEBUG:teuthology.misc:Transferring archived files from smithi178:/var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/crash to /home/teuthworker/archive/dparmar-2024-05-10_07:38:09-fs-wip-63896-from-wip-rishabh-testing-20240501.193033-distro-default-smithi/7700928/remote/smithi178/crash 2024-05-10T11:48:18.266 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar c -f - -C /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/crash -- . 2024-05-10T11:48:18.329 INFO:teuthology.orchestra.run.smithi178.stderr:tar: /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/crash: Cannot open: No such file or directory 2024-05-10T11:48:18.329 INFO:teuthology.orchestra.run.smithi178.stderr:tar: Error is not recoverable: exiting now 2024-05-10T11:48:18.331 DEBUG:teuthology.misc:Transferring archived files from smithi188:/var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/crash to /home/teuthworker/archive/dparmar-2024-05-10_07:38:09-fs-wip-63896-from-wip-rishabh-testing-20240501.193033-distro-default-smithi/7700928/remote/smithi188/crash 2024-05-10T11:48:18.332 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar c -f - -C /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/crash -- . 2024-05-10T11:48:18.373 INFO:teuthology.orchestra.run.smithi188.stderr:tar: /var/lib/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/crash: Cannot open: No such file or directory 2024-05-10T11:48:18.373 INFO:teuthology.orchestra.run.smithi188.stderr:tar: Error is not recoverable: exiting now 2024-05-10T11:48:18.374 INFO:tasks.cephadm:Checking cluster log for badness... 2024-05-10T11:48:18.374 DEBUG:teuthology.orchestra.run.smithi100:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-05-10T11:48:18.482 INFO:teuthology.orchestra.run.smithi100.stdout:2024-05-10T11:32:55.494468+0000 mds.i (mds.0) 89 : cluster [WRN] Scrub error on inode 0x100000004a2 (/volumes/qa/sv_0/96765652-39ef-4cf3-b20a-a70f7e9d774d/client.0/tmp/clients/client7/~dmtmp/PWRPNT) see mds.i log and `damage ls` output for details 2024-05-10T11:48:18.483 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-05-10T11:48:18.483 DEBUG:teuthology.orchestra.run.smithi100:> sudo egrep '\[SEC\]' /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-05-10T11:48:18.519 DEBUG:teuthology.orchestra.run.smithi100:> sudo egrep '\[ERR\]' /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-05-10T11:48:18.594 DEBUG:teuthology.orchestra.run.smithi100:> sudo egrep '\[WRN\]' /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-05-10T11:48:18.667 INFO:teuthology.orchestra.run.smithi100.stdout:2024-05-10T11:32:55.494468+0000 mds.i (mds.0) 89 : cluster [WRN] Scrub error on inode 0x100000004a2 (/volumes/qa/sv_0/96765652-39ef-4cf3-b20a-a70f7e9d774d/client.0/tmp/clients/client7/~dmtmp/PWRPNT) see mds.i log and `damage ls` output for details 2024-05-10T11:48:18.667 INFO:tasks.cephadm:Compressing logs... 2024-05-10T11:48:18.668 DEBUG:teuthology.orchestra.run.smithi100:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-10T11:48:18.710 DEBUG:teuthology.orchestra.run.smithi178:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-10T11:48:18.713 DEBUG:teuthology.orchestra.run.smithi188:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-10T11:48:18.745 INFO:teuthology.orchestra.run.smithi188.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-10T11:48:18.746 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph.log 2024-05-10T11:48:18.747 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-osd.2.log 2024-05-10T11:48:18.747 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-mds.f.log 2024-05-10T11:48:18.748 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph.log: /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph.audit.log 2024-05-10T11:48:18.748 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-osd.5.log 2024-05-10T11:48:18.748 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-mds.c.log 2024-05-10T11:48:18.748 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-mon.c.log 2024-05-10T11:48:18.751 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-mgr.z.log 2024-05-10T11:48:18.751 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-mds.i.log 2024-05-10T11:48:18.751 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph.cephadm.log 2024-05-10T11:48:18.751 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-osd.11.log 2024-05-10T11:48:18.751 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-osd.8.log 2024-05-10T11:48:18.751 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-volume.log 2024-05-10T11:48:18.751 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-mds.l.log 2024-05-10T11:48:18.751 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-10T11:48:18.755 INFO:teuthology.orchestra.run.smithi100.stderr:find: ‘/var/log/rbd-target-api’: No such file or directorygzip 2024-05-10T11:48:18.755 INFO:teuthology.orchestra.run.smithi100.stderr: -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-05-10T11:48:18.755 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph.log 2024-05-10T11:48:18.755 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.0.log: -- /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph.audit.log 2024-05-10T11:48:18.755 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-mds.g.log 2024-05-10T11:48:18.763 INFO:teuthology.orchestra.run.smithi178.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-10T11:48:18.763 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph.log 2024-05-10T11:48:18.763 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph.audit.log 2024-05-10T11:48:18.763 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-osd.7.log 2024-05-10T11:48:18.763 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph.log: /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-mds.b.log 2024-05-10T11:48:18.763 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-osd.4.log 2024-05-10T11:48:18.764 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-osd.1.log 2024-05-10T11:48:18.764 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-mgr.y.log 2024-05-10T11:48:18.764 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-mds.h.log 2024-05-10T11:48:18.764 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-osd.10.log 2024-05-10T11:48:18.764 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph.cephadm.log 2024-05-10T11:48:18.764 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-mds.e.log 2024-05-10T11:48:18.764 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-volume.log 2024-05-10T11:48:18.764 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-mon.b.log 2024-05-10T11:48:18.764 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-mds.k.log 2024-05-10T11:48:18.764 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-10T11:48:18.764 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph.log: /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-mds.j.log 2024-05-10T11:48:18.764 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-osd.3.log 2024-05-10T11:48:18.764 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-osd.0.log 2024-05-10T11:48:18.764 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-mgr.x.log 2024-05-10T11:48:18.765 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-mon.a.log 2024-05-10T11:48:18.765 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-osd.6.log 2024-05-10T11:48:18.765 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph.cephadm.log 2024-05-10T11:48:18.765 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-mds.a.log 2024-05-10T11:48:18.765 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-volume.log 2024-05-10T11:48:18.765 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-mds.d.log 2024-05-10T11:48:18.765 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-osd.9.log 2024-05-10T11:48:18.765 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-10T11:48:18.765 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-05-10T11:48:18.765 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/cephadm.log: 87.5% -- replaced with /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph.log.gz/var/log/ceph/ceph-client.admin.log: 2024-05-10T11:48:18.817 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-mds.l.log: /var/log/ceph/cephadm.log: 84.1% -- replaced with /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph.cephadm.log.gz 2024-05-10T11:48:18.864 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-mds.k.log: /var/log/ceph/cephadm.log: 84.1% -- replaced with /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph.cephadm.log.gz 2024-05-10T11:48:18.895 INFO:teuthology.orchestra.run.smithi100.stderr: 94.8% -- replaced with /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-mds.a.log.gz 2024-05-10T11:48:18.898 INFO:teuthology.orchestra.run.smithi100.stderr: 94.7% -- replaced with /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-mds.g.log.gz 2024-05-10T11:48:18.910 INFO:teuthology.orchestra.run.smithi100.stderr: 92.8% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-05-10T11:48:18.932 INFO:teuthology.orchestra.run.smithi178.stderr: 94.5% -- replaced with /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-mds.k.log.gz 2024-05-10T11:48:18.952 INFO:teuthology.orchestra.run.smithi178.stderr: 93.6% -- replaced with /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-mgr.y.log.gz 2024-05-10T11:48:18.958 INFO:teuthology.orchestra.run.smithi100.stderr: 94.7% -- replaced with /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-mds.j.log.gz 2024-05-10T11:48:18.965 INFO:teuthology.orchestra.run.smithi188.stderr: 91.1% -- replaced with /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-volume.log.gz 2024-05-10T11:48:18.974 INFO:teuthology.orchestra.run.smithi178.stderr: 87.5% -- replaced with /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph.log.gz 2024-05-10T11:48:19.003 INFO:teuthology.orchestra.run.smithi178.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-10T11:48:19.041 INFO:teuthology.orchestra.run.smithi188.stderr: 93.7% -- replaced with /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-mgr.z.log.gz 2024-05-10T11:48:19.054 INFO:teuthology.orchestra.run.smithi100.stderr: 90.2% -- replaced with /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-mgr.x.log.gz 2024-05-10T11:48:19.131 INFO:teuthology.orchestra.run.smithi100.stderr: 91.1% -- replaced with /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-volume.log.gz 2024-05-10T11:48:19.139 INFO:teuthology.orchestra.run.smithi100.stderr: 84.0% -- replaced with /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph.cephadm.log.gz 2024-05-10T11:48:19.142 INFO:teuthology.orchestra.run.smithi100.stderr: 90.4% -- replaced with /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph.audit.log.gz 2024-05-10T11:48:19.191 INFO:teuthology.orchestra.run.smithi178.stderr: 91.1% -- replaced with /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-volume.log.gz 2024-05-10T11:48:19.200 INFO:teuthology.orchestra.run.smithi100.stderr: 94.6% -- replaced with /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-mds.d.log.gz 2024-05-10T11:48:19.237 INFO:teuthology.orchestra.run.smithi100.stderr: 94.7% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-10T11:48:19.365 INFO:teuthology.orchestra.run.smithi178.stderr: 90.5% -- replaced with /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph.audit.log.gz 2024-05-10T11:48:19.406 INFO:teuthology.orchestra.run.smithi188.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-10T11:48:19.416 INFO:teuthology.orchestra.run.smithi188.stderr: 87.5% -- replaced with /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph.log.gz 2024-05-10T11:48:19.423 INFO:teuthology.orchestra.run.smithi188.stderr: 90.6% -- replaced with /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph.audit.log.gz 2024-05-10T11:48:19.534 INFO:teuthology.orchestra.run.smithi188.stderr: 94.7% -- replaced with /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-mds.l.log.gz 2024-05-10T11:48:25.638 INFO:teuthology.orchestra.run.smithi188.stderr: 91.6% -- replaced with /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-mon.c.log.gz 2024-05-10T11:48:31.209 INFO:teuthology.orchestra.run.smithi178.stderr: 91.7% -- replaced with /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-mon.b.log.gz 2024-05-10T11:49:17.005 INFO:teuthology.orchestra.run.smithi100.stderr: 91.1% -- replaced with /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-mon.a.log.gz 2024-05-10T11:51:12.706 INFO:teuthology.orchestra.run.smithi178.stderr: 95.3% -- replaced with /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-mds.h.log.gz 2024-05-10T11:51:21.395 INFO:teuthology.orchestra.run.smithi188.stderr: 95.2% -- replaced with /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-mds.f.log.gz 2024-05-10T11:51:57.930 INFO:teuthology.orchestra.run.smithi178.stderr: 94.8% -- replaced with /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-mds.e.log.gz 2024-05-10T11:53:21.577 INFO:teuthology.orchestra.run.smithi178.stderr: 91.5% -- replaced with /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-osd.1.log.gz 2024-05-10T11:53:46.618 INFO:teuthology.orchestra.run.smithi100.stderr: 91.6% -- replaced with /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-osd.0.log.gz 2024-05-10T11:54:01.023 INFO:teuthology.orchestra.run.smithi100.stderr: 91.2% -- replaced with /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-osd.6.log.gz 2024-05-10T11:54:19.040 INFO:teuthology.orchestra.run.smithi178.stderr: 91.3% -- replaced with /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-osd.10.log.gz 2024-05-10T11:54:42.963 INFO:teuthology.orchestra.run.smithi178.stderr: 91.3% -- replaced with /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-osd.7.log.gz 2024-05-10T11:55:43.114 INFO:teuthology.orchestra.run.smithi100.stderr: 91.3% -- replaced with /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-osd.3.log.gz 2024-05-10T11:56:01.433 INFO:teuthology.orchestra.run.smithi178.stderr: 91.2% -- replaced with /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-osd.4.log.gz 2024-05-10T11:56:43.275 INFO:teuthology.orchestra.run.smithi100.stderr: 91.1% -- replaced with /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-osd.9.log.gz 2024-05-10T11:57:13.498 INFO:teuthology.orchestra.run.smithi188.stderr: 91.4% -- replaced with /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-osd.11.log.gz 2024-05-10T11:58:23.232 INFO:teuthology.orchestra.run.smithi188.stderr: 91.2% -- replaced with /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-osd.8.log.gz 2024-05-10T11:58:45.067 INFO:teuthology.orchestra.run.smithi188.stderr: 91.3% -- replaced with /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-osd.2.log.gz 2024-05-10T11:59:14.905 INFO:teuthology.orchestra.run.smithi178.stderr: 94.3% -- replaced with /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-mds.b.log.gz 2024-05-10T11:59:14.907 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-05-10T11:59:14.907 INFO:teuthology.orchestra.run.smithi178.stderr:real 10m56.179s 2024-05-10T11:59:14.908 INFO:teuthology.orchestra.run.smithi178.stderr:user 6m4.216s 2024-05-10T11:59:14.908 INFO:teuthology.orchestra.run.smithi178.stderr:sys 0m15.641s 2024-05-10T11:59:18.746 INFO:teuthology.orchestra.run.smithi188.stderr: 91.0% -- replaced with /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-osd.5.log.gz 2024-05-10T12:03:18.463 INFO:teuthology.orchestra.run.smithi100.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-05-10T12:03:18.465 INFO:teuthology.orchestra.run.smithi100.stderr: 2024-05-10T12:03:18.465 INFO:teuthology.orchestra.run.smithi100.stderr:real 14m59.741s 2024-05-10T12:03:18.465 INFO:teuthology.orchestra.run.smithi100.stderr:user 8m49.700s 2024-05-10T12:03:18.465 INFO:teuthology.orchestra.run.smithi100.stderr:sys 0m23.088s 2024-05-10T12:21:27.310 INFO:teuthology.orchestra.run.smithi188.stderr: 93.8% -- replaced with /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-mds.c.log.gz 2024-05-10T12:24:40.494 INFO:teuthology.orchestra.run.smithi188.stderr: 95.3% -- replaced with /var/log/ceph/20a542cc-0ebd-11ef-bc98-c7b262605968/ceph-mds.i.log.gz 2024-05-10T12:24:40.497 INFO:teuthology.orchestra.run.smithi188.stderr: 2024-05-10T12:24:40.497 INFO:teuthology.orchestra.run.smithi188.stderr:real 36m21.768s 2024-05-10T12:24:40.497 INFO:teuthology.orchestra.run.smithi188.stderr:user 11m33.632s 2024-05-10T12:24:40.497 INFO:teuthology.orchestra.run.smithi188.stderr:sys 0m34.902s 2024-05-10T12:24:40.497 INFO:tasks.cephadm:Archiving logs... 2024-05-10T12:24:40.499 DEBUG:teuthology.misc:Transferring archived files from smithi100:/var/log/ceph to /home/teuthworker/archive/dparmar-2024-05-10_07:38:09-fs-wip-63896-from-wip-rishabh-testing-20240501.193033-distro-default-smithi/7700928/remote/smithi100/log 2024-05-10T12:24:40.500 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-10T12:26:14.021 DEBUG:teuthology.misc:Transferring archived files from smithi178:/var/log/ceph to /home/teuthworker/archive/dparmar-2024-05-10_07:38:09-fs-wip-63896-from-wip-rishabh-testing-20240501.193033-distro-default-smithi/7700928/remote/smithi178/log 2024-05-10T12:26:14.023 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-10T12:27:10.410 DEBUG:teuthology.misc:Transferring archived files from smithi188:/var/log/ceph to /home/teuthworker/archive/dparmar-2024-05-10_07:38:09-fs-wip-63896-from-wip-rishabh-testing-20240501.193033-distro-default-smithi/7700928/remote/smithi188/log 2024-05-10T12:27:10.412 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-10T12:28:46.009 INFO:tasks.cephadm:Removing cluster... 2024-05-10T12:28:46.010 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 --force 2024-05-10T12:28:46.330 INFO:teuthology.orchestra.run.smithi100.stdout:Deleting cluster with fsid: 20a542cc-0ebd-11ef-bc98-c7b262605968 2024-05-10T12:28:46.700 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 --force 2024-05-10T12:28:47.015 INFO:teuthology.orchestra.run.smithi178.stdout:Deleting cluster with fsid: 20a542cc-0ebd-11ef-bc98-c7b262605968 2024-05-10T12:28:47.676 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 20a542cc-0ebd-11ef-bc98-c7b262605968 --force 2024-05-10T12:28:48.415 INFO:teuthology.orchestra.run.smithi188.stdout:Deleting cluster with fsid: 20a542cc-0ebd-11ef-bc98-c7b262605968 2024-05-10T12:28:49.180 INFO:tasks.cephadm:Removing cephadm ... 2024-05-10T12:28:49.180 DEBUG:teuthology.orchestra.run.smithi100:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-10T12:28:49.200 DEBUG:teuthology.orchestra.run.smithi178:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-10T12:28:49.219 DEBUG:teuthology.orchestra.run.smithi188:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-10T12:28:49.235 INFO:tasks.cephadm:Teardown complete 2024-05-10T12:28:49.236 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-10T12:28:49.247 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-10T12:28:49.247 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-10T12:28:49.249 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-10T12:28:49.262 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-10T12:28:49.396 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-10T12:28:49.406 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-10T12:28:49.407 DEBUG:teuthology.orchestra.run.smithi100:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-10T12:28:49.409 DEBUG:teuthology.orchestra.run.smithi178:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-10T12:28:49.413 DEBUG:teuthology.orchestra.run.smithi188:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-10T12:28:49.424 INFO:teuthology.orchestra.run.smithi100.stderr:bash: line 1: ntpq: command not found 2024-05-10T12:28:49.427 INFO:teuthology.orchestra.run.smithi178.stderr:bash: line 1: ntpq: command not found 2024-05-10T12:28:49.427 INFO:teuthology.orchestra.run.smithi188.stderr:bash: line 1: ntpq: command not found 2024-05-10T12:28:49.459 INFO:teuthology.orchestra.run.smithi178.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-10T12:28:49.459 INFO:teuthology.orchestra.run.smithi178.stdout:=============================================================================== 2024-05-10T12:28:49.459 INFO:teuthology.orchestra.run.smithi178.stdout:^* hv01.front.sepia.ceph.com 3 6 377 56 +1624us[+1623us] +/- 59ms 2024-05-10T12:28:49.459 INFO:teuthology.orchestra.run.smithi178.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 55 -1313us[-1313us] +/- 65ms 2024-05-10T12:28:49.459 INFO:teuthology.orchestra.run.smithi178.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 54 -1055us[-1055us] +/- 80ms 2024-05-10T12:28:49.459 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T12:28:49.463 INFO:teuthology.orchestra.run.smithi100.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-10T12:28:49.463 INFO:teuthology.orchestra.run.smithi100.stdout:=============================================================================== 2024-05-10T12:28:49.463 INFO:teuthology.orchestra.run.smithi100.stdout:^* hv01.front.sepia.ceph.com 3 6 377 48 +1739us[+1730us] +/- 59ms 2024-05-10T12:28:49.463 INFO:teuthology.orchestra.run.smithi100.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 52 -1220us[-1228us] +/- 65ms 2024-05-10T12:28:49.463 INFO:teuthology.orchestra.run.smithi100.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 53 -949us[ -958us] +/- 80ms 2024-05-10T12:28:49.463 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T12:28:49.512 INFO:teuthology.orchestra.run.smithi188.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-10T12:28:49.513 INFO:teuthology.orchestra.run.smithi188.stdout:=============================================================================== 2024-05-10T12:28:49.513 INFO:teuthology.orchestra.run.smithi188.stdout:^* hv01.front.sepia.ceph.com 3 6 377 53 +1751us[+1577us] +/- 59ms 2024-05-10T12:28:49.513 INFO:teuthology.orchestra.run.smithi188.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 54 -1197us[-1371us] +/- 65ms 2024-05-10T12:28:49.513 INFO:teuthology.orchestra.run.smithi188.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 52 -924us[ -924us] +/- 80ms 2024-05-10T12:28:49.513 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T12:28:49.545 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-10T12:28:49.556 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-10T12:28:49.557 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-10T12:28:49.582 DEBUG:teuthology.orchestra.run.smithi100:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-10T12:28:49.750 DEBUG:teuthology.orchestra.run.smithi178:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-10T12:28:49.859 DEBUG:teuthology.orchestra.run.smithi188:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-10T12:28:50.015 DEBUG:teuthology.orchestra.run.smithi100:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-10T12:28:50.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T12:28:50.049 DEBUG:teuthology.orchestra.run.smithi178:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-10T12:28:50.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T12:28:50.082 DEBUG:teuthology.orchestra.run.smithi188:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-10T12:28:50.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T12:28:50.110 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-10T12:28:50.121 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-10T12:28:50.173 INFO:teuthology.task.internal:Duration was 5418.089764 seconds 2024-05-10T12:28:50.174 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-10T12:28:50.223 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-10T12:28:50.223 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-10T12:28:50.226 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-10T12:28:50.229 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-10T12:28:50.295 INFO:teuthology.orchestra.run.smithi100.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-10T12:28:50.299 INFO:teuthology.orchestra.run.smithi178.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-10T12:28:50.304 INFO:teuthology.orchestra.run.smithi188.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-10T12:28:50.619 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-10T12:28:50.620 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi100.front.sepia.ceph.com 2024-05-10T12:28:50.620 DEBUG:teuthology.orchestra.run.smithi100:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-10T12:28:50.642 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi178.front.sepia.ceph.com 2024-05-10T12:28:50.642 DEBUG:teuthology.orchestra.run.smithi178:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-10T12:28:50.672 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi188.front.sepia.ceph.com 2024-05-10T12:28:50.672 DEBUG:teuthology.orchestra.run.smithi188:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-10T12:28:50.701 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-10T12:28:50.701 DEBUG:teuthology.orchestra.run.smithi100:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-10T12:28:50.704 DEBUG:teuthology.orchestra.run.smithi178:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-10T12:28:50.715 DEBUG:teuthology.orchestra.run.smithi188:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-10T12:28:50.775 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-10T12:28:50.776 DEBUG:teuthology.orchestra.run.smithi100:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-10T12:28:50.778 DEBUG:teuthology.orchestra.run.smithi178:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-10T12:28:50.785 DEBUG:teuthology.orchestra.run.smithi188:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-10T12:28:52.293 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-10T12:28:52.305 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-10T12:28:52.305 DEBUG:teuthology.orchestra.run.smithi100:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-10T12:28:52.351 DEBUG:teuthology.orchestra.run.smithi178:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-10T12:28:52.384 DEBUG:teuthology.orchestra.run.smithi188:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-10T12:28:52.416 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-10T12:28:52.427 DEBUG:teuthology.orchestra.run.smithi100:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-10T12:28:52.430 DEBUG:teuthology.orchestra.run.smithi178:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-10T12:28:52.433 DEBUG:teuthology.orchestra.run.smithi188:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-10T12:28:52.467 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern = core 2024-05-10T12:28:52.481 INFO:teuthology.orchestra.run.smithi178.stdout:kernel.core_pattern = core 2024-05-10T12:28:52.526 INFO:teuthology.orchestra.run.smithi188.stdout:kernel.core_pattern = core 2024-05-10T12:28:52.546 DEBUG:teuthology.orchestra.run.smithi100:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-10T12:28:52.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T12:28:52.561 DEBUG:teuthology.orchestra.run.smithi178:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-10T12:28:52.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T12:28:52.577 DEBUG:teuthology.orchestra.run.smithi188:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-10T12:28:52.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T12:28:52.594 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-10T12:28:52.606 INFO:teuthology.task.internal:Transferring archived files... 2024-05-10T12:28:52.607 DEBUG:teuthology.misc:Transferring archived files from smithi100:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/dparmar-2024-05-10_07:38:09-fs-wip-63896-from-wip-rishabh-testing-20240501.193033-distro-default-smithi/7700928/remote/smithi100 2024-05-10T12:28:52.608 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-10T12:28:52.682 DEBUG:teuthology.misc:Transferring archived files from smithi178:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/dparmar-2024-05-10_07:38:09-fs-wip-63896-from-wip-rishabh-testing-20240501.193033-distro-default-smithi/7700928/remote/smithi178 2024-05-10T12:28:52.682 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-10T12:28:52.723 DEBUG:teuthology.misc:Transferring archived files from smithi188:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/dparmar-2024-05-10_07:38:09-fs-wip-63896-from-wip-rishabh-testing-20240501.193033-distro-default-smithi/7700928/remote/smithi188 2024-05-10T12:28:52.723 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-10T12:28:52.764 INFO:teuthology.task.internal:Removing archive directory... 2024-05-10T12:28:52.764 DEBUG:teuthology.orchestra.run.smithi100:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-10T12:28:52.767 DEBUG:teuthology.orchestra.run.smithi178:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-10T12:28:52.769 DEBUG:teuthology.orchestra.run.smithi188:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-10T12:28:52.814 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-10T12:28:52.827 INFO:teuthology.task.internal:Not uploading archives. 2024-05-10T12:28:52.827 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-10T12:28:52.852 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-10T12:28:52.852 DEBUG:teuthology.orchestra.run.smithi100:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-10T12:28:52.855 DEBUG:teuthology.orchestra.run.smithi178:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-10T12:28:52.858 DEBUG:teuthology.orchestra.run.smithi188:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-10T12:28:52.868 INFO:teuthology.orchestra.run.smithi100.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 10 12:28 /home/ubuntu/cephtest 2024-05-10T12:28:52.872 INFO:teuthology.orchestra.run.smithi178.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 10 12:28 /home/ubuntu/cephtest 2024-05-10T12:28:52.875 INFO:teuthology.orchestra.run.smithi188.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 10 12:28 /home/ubuntu/cephtest 2024-05-10T12:28:52.877 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-10T12:28:52.901 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-10T12:28:53.004 INFO:teuthology.nuke:Checking targets against current locks 2024-05-10T12:28:53.031 DEBUG:teuthology.nuke:shortname: smithi100 2024-05-10T12:28:53.032 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-10T12:28:53.056 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi100.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dparmar-2024-05-10_07:38:09-fs-wip-63896-from-wip-rishabh-testing-20240501.193033-distro-default-smithi/7700928', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-10 10:49:19.642624', 'locked_by': 'scheduled_dparmar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJt3ipYRQ432lUpm4/AGPX3yBj/0GAz5K85floybCoPuHpQc06vfLSR2OIARet5cxu9aU4b2at+bhV65hOP1tPI='} 2024-05-10T12:28:53.059 DEBUG:teuthology.nuke:shortname: smithi178 2024-05-10T12:28:53.059 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-10T12:28:53.083 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi178.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dparmar-2024-05-10_07:38:09-fs-wip-63896-from-wip-rishabh-testing-20240501.193033-distro-default-smithi/7700928', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-10 10:49:19.639563', 'locked_by': 'scheduled_dparmar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJuZmTHNWAwjSGV2n2XRxRLiENiiqEvpNwqO/nd1+ixZENKQRPBK7HHWqEDjaqfB3U6h5G4+woAQdEh1pWGj2E4='} 2024-05-10T12:28:53.094 DEBUG:teuthology.nuke:shortname: smithi188 2024-05-10T12:28:53.094 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-10T12:28:53.114 INFO:teuthology.orchestra.console.smithi100:Power off 2024-05-10T12:28:53.115 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-10T12:28:53.125 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi188.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dparmar-2024-05-10_07:38:09-fs-wip-63896-from-wip-rishabh-testing-20240501.193033-distro-default-smithi/7700928', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-10 10:49:19.641318', 'locked_by': 'scheduled_dparmar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDLA5TUZHoJYRNjyOKi56vQBMgDVnKiHQBobOj4YJoXW8pAlMrQohjTezCcLcajoTSSCXKmYYIyKUsCTdJkyWjs='} 2024-05-10T12:28:53.149 INFO:teuthology.orchestra.console.smithi178:Power off 2024-05-10T12:28:53.150 DEBUG:teuthology.orchestra.console.smithi178:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-10T12:28:53.189 INFO:teuthology.orchestra.console.smithi188:Power off 2024-05-10T12:28:53.189 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-10T12:28:53.231 DEBUG:teuthology.orchestra.console.smithi188:power off output: Chassis Power Control: Down/Off 2024-05-10T12:28:53.232 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-10T12:28:53.240 DEBUG:teuthology.orchestra.console.smithi178:power off output: Chassis Power Control: Down/Off 2024-05-10T12:28:53.241 DEBUG:teuthology.orchestra.console.smithi178:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-10T12:28:53.249 DEBUG:teuthology.orchestra.console.smithi100:power off output: Chassis Power Control: Down/Off 2024-05-10T12:28:53.249 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-10T12:28:53.258 DEBUG:teuthology.orchestra.console.smithi188:check power output: Chassis Power is on 2024-05-10T12:28:53.264 DEBUG:teuthology.orchestra.console.smithi178:check power output: Chassis Power is on 2024-05-10T12:28:53.271 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is on 2024-05-10T12:28:57.260 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-10T12:28:57.268 DEBUG:teuthology.orchestra.console.smithi178:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-10T12:28:57.276 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-10T12:28:57.371 DEBUG:teuthology.orchestra.console.smithi188:check power output: Chassis Power is on 2024-05-10T12:28:57.378 DEBUG:teuthology.orchestra.console.smithi178:check power output: Chassis Power is on 2024-05-10T12:28:57.384 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is on 2024-05-10T12:29:01.373 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-10T12:29:01.381 DEBUG:teuthology.orchestra.console.smithi178:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-10T12:29:01.387 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-10T12:29:01.483 DEBUG:teuthology.orchestra.console.smithi188:check power output: Chassis Power is off 2024-05-10T12:29:01.489 DEBUG:teuthology.orchestra.console.smithi178:check power output: Chassis Power is off 2024-05-10T12:29:01.500 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is on 2024-05-10T12:29:01.584 INFO:teuthology.orchestra.console.smithi188:Power off completed 2024-05-10T12:29:01.590 INFO:teuthology.orchestra.console.smithi178:Power off completed 2024-05-10T12:29:05.504 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-10T12:29:05.614 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is off 2024-05-10T12:29:05.715 INFO:teuthology.orchestra.console.smithi100:Power off completed 2024-05-10T12:29:05.820 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/suites/dbench}} duration: 5418.089764356613 failure_reason: '"2024-05-10T11:32:55.494468+0000 mds.i (mds.0) 89 : cluster [WRN] Scrub error on inode 0x100000004a2 (/volumes/qa/sv_0/96765652-39ef-4cf3-b20a-a70f7e9d774d/client.0/tmp/clients/client7/~dmtmp/PWRPNT) see mds.i log and `damage ls` output for details" in cluster log' flavor: default owner: scheduled_dparmar@teuthology success: false 2024-05-10T12:29:05.820 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-10T12:29:05.913 INFO:teuthology.run:FAIL